Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on kv-engine-jepsen-host-2-ubuntu1804 (jepsen non-vm-host) in workspace /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly The recommended git tool is: NONE No credentials specified > git rev-parse --resolve-git-dir /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/.git # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url ssh://buildbot@review.couchbase.org:29418/jepsen.couchbase # timeout=10 Fetching upstream changes from ssh://buildbot@review.couchbase.org:29418/jepsen.couchbase > git --version # timeout=10 > git --version # 'git version 2.17.1' > git fetch --tags --progress -- ssh://buildbot@review.couchbase.org:29418/jepsen.couchbase +refs/heads/*:refs/remotes/origin/* # timeout=10 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 Checking out Revision 7a53fac81fdd4c21e7b4a3d63887911ad58baa52 (refs/remotes/origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 7a53fac81fdd4c21e7b4a3d63887911ad58baa52 # timeout=10 Commit message: "MB-49923: Upgrade Couchbase dcp-client" > git rev-list --no-walk 7a53fac81fdd4c21e7b4a3d63887911ad58baa52 # timeout=10 The recommended git tool is: NONE No credentials specified > git rev-parse 7a53fac81fdd4c21e7b4a3d63887911ad58baa52^{commit} # timeout=10 The recommended git tool is: NONE No credentials specified [GitCheckoutListener] Recording commits of 'git ssh://buildbot@review.couchbase.org:29418/jepsen.couchbase' [GitCheckoutListener] Found previous build 'kv-engine-neo-jepsen-nightly #183' that contains recorded Git commits [GitCheckoutListener] -> Starting recording of new commits since '7a53fac81fdd4c21e7b4a3d63887911ad58baa52' [GitCheckoutListener] -> Git commit decorator could not be created for SCM 'hudson.plugins.git.GitSCM@23932eaa' [GitCheckoutListener] -> No new commits found [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content CC=/usr/lib/ccache/gcc-8 CXX=/usr/lib/ccache/g++-8 [EnvInject] - Variables injected successfully. [kv-engine-neo-jepsen-nightly] $ /bin/bash /tmp/jenkins938948730534178730.sh Setting NETWORK_INTERFACE to eth0 ------ Testing Jepsen changes against latest build ------ grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-174/20211231T053655.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-154/20211211T083302.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-166/20211222T124058.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-163/20211220T104806.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T042208.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T073946.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T083827.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T110553.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T094055.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T082141.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T070517.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T102112.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T062923.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T100023.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T074320.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T074421.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T104526.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T080939.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T105555.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T065546.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T103339.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T074357.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-159/20211216T082817.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-155/20211212T083938.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-155/20211212T100323.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-155/20211212T120153.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-155/20211212T103538.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-155/20211212T121258.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-155/20211212T054023.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-161/20211218T112100.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-161/20211218T054027.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-162/20211219T053837.000-0800/results.edn: No such file or directory grep: ./store/Couchbase-jenkins-kv-engine-neo-jepsen-nightly-169/20211226T113343.000-0800/results.edn: No such file or directory Couchbase Server: 7.1.0-2028-1 Using Node IPs 172.23.105.187 172.23.107.228 172.23.106.61 172.23.107.227 ################################################################ Running test suit: suites/kv-engine-cv/cheshire-cat/cv-nightly.conf # Syncwrite Sanity tests workload=register,replicas=2,no-autofailover,rate=0,durability=25:25:25:25 # Syncwrite failover # We're only Sequentially consistent in hard failover situations, so linizrable checker is likely to fail. So just test graceful. workload=failover,failover-type=graceful,recovery-type=delta,replicas=1,no-autofailover,disrupt-count=1,rate=0,durability=25:25:25:25 workload=failover,scopes=3,collections=1,failover-type=graceful,recovery-type=delta,replicas=2,no-autofailover,disrupt-count=1,rate=0,durability=0:0:100:0,cas workload=failover,failover-type=graceful,recovery-type=full,replicas=1,no-autofailover,disrupt-count=1,rate=0,durability=0:100:0:0,use-json-docs,use-subdoc # Syncwrite rebalance workload=rebalance,scenario=sequential-rebalance-out-in,replicas=1,no-autofailover,disrupt-count=1,rate=0,durability=25:25:25:25,cas workload=rebalance,scenario=sequential-rebalance-out-in,replicas=2,no-autofailover,disrupt-count=1,rate=0,durability=0:100:0:0 workload=rebalance,scenario=bulk-rebalance-out-in,no-autofailover,replicas=2,disrupt-count=2,kv-timeout=1.5,durability=0:0:100:0,cas workload=rebalance,scenario=swap-rebalance,no-autofailover,replicas=1,disrupt-count=1,kv-timeout=30,durability=0:0:0:100,use-json-docs,doc-padding-size=4096 # Syncwrite partition workload=partition,no-autofailover,replicas=1,disrupt-count=1,disrupt-time=10,kv-timeout=15,durability=0:0:100:0,cas # Syncwrite partition-failover (These tests are commented out till we have a sequential consistency checker as they require hard failover to work) #workload=partition-failover,failover-type=hard,recovery-type=delta,replicas=2,no-autofailover,disrupt-count=1,durability=0:100:0:0 #workload=partition-failover,failover-type=hard,recovery-type=full,replicas=1,no-autofailover,disrupt-count=1,durability=0:0:100:0 # Syncwrite kill workload=kill,scenario=kill-ns-server,replicas=2,no-autofailover,durability=0:0:0:100,eviction-policy=value workload=kill,scopes=1,collections=3,scenario=kill-babysitter,replicas=2,no-autofailover,durability=0:0:0:100,eviction-policy=value,cas workload=kill,scenario=kill-memcached,replicas=2,no-autofailover,durability=0:0:0:100,eviction-policy=value,use-json-docs,doc-padding-size=4096 workload=kill,scenario=hard-reboot,replicas=2,no-autofailover,durability=0:0:0:100,eviction-policy=full,cas workload=kill,scenario=suspend-process,process-to-suspend=memcached,process-suspend-time=35,replicas=1,no-autofailover,durability=0:0:100:0,eviction-policy=full,cas # Syncwrite set-kill workload=set-kill,scenario=kill-memcached,replicas=1,no-autofailover,durability=0:0:100:0,eviction-policy=full,use-json-docs,doc-padding-size=64 workload=set-kill,scenario=kill-memcached,replicas=1,no-autofailover,durability=0:0:0:100,eviction-policy=value workload=set-kill,scenario=kill-ns-server,replicas=2,no-autofailover,durability=0:100:0:0,eviction-policy=value workload=set-kill,scenario=kill-babysitter,replicas=2,no-autofailover,durability=0:100:0:0,eviction-policy=value workload=set-kill,scenario=hard-reboot,replicas=1,no-autofailover,durability=0:0:100:0,eviction-policy=value workload=set-kill,scenario=suspend-process,process-to-suspend=memcached,process-suspend-time=10,replicas=2,autofailover,durability=0:0:100:0,eviction-policy=value workload=set-kill,scenario=kill-memcached-on-slow-disk,replicas=2,no-autofailover,durability=0:0:0:100,eviction-policy=value,manipulate-disks workload=set-kill,scenario=kill-memcached-on-slow-disk,replicas=2,no-autofailover,durability=0:0:0:100,eviction-policy=full,manipulate-disks # Ephemeral Syncwrite tests workload=register,replicas=2,no-autofailover,rate=0,durability=50:50:0:0,bucket-type=ephemeral,cas workload=failover,failover-type=hard,recovery-type=delta,replicas=2,no-autofailover,disrupt-count=1,rate=0,durability=0:100:0:0,bucket-type=ephemeral,cas workload=rebalance,scenario=sequential-rebalance-out-in,replicas=2,no-autofailover,disrupt-count=1,rate=0,durability=0:100:0:0,bucket-type=ephemeral,cas workload=set-kill,scenario=kill-memcached,replicas=2,no-autofailover,durability=0:100:0:0,bucket-type=ephemeral workload=set-kill,scenario=kill-memcached,replicas=1,no-autofailover,durability=0:100:0:0,bucket-type=ephemeral,use-json-docs,use-subdoc workload=set-kill,scenario=kill-ns-server,replicas=2,no-autofailover,durability=0:100:0:0,bucket-type=ephemeral,use-json-docs,doc-padding-size=64 workload=set-kill,scenario=kill-babysitter,replicas=2,no-autofailover,durability=0:100:0:0,bucket-type=ephemeral # Disk Failure Tests workload=disk-failure,manipulate-disks,replicas=1,durability=0:100:0:0,recovery-type=full,eviction-policy=value,use-json-docs,use-subdoc workload=disk-failure,manipulate-disks,replicas=1,durability=0:0:100:0,recovery-type=delta,eviction-policy=full,cas workload=disk-failure,manipulate-disks,replicas=1,durability=0:0:0:100,recovery-type=delta,eviction-policy=full,cas workload=disk-failure,manipulate-disks,disrupt-count=2,replicas=2,durability=0:0:0:100,recovery-type=delta,eviction-policy=full # Disk Failure and kill Tests workload=kill-n-disk-failure,manipulate-disks,replicas=1,durability=0:0:0:100,recovery-type=delta,eviction-policy=full################################################################ Creating key-pair ./resources/my.key and ./resources/my.key.pub. Key pair already exists. Removing node entries from ~/.ssh/known_hosts. # Host 172.23.105.187 found: line 10 /home/couchbase/.ssh/known_hosts updated. Original contents retained as /home/couchbase/.ssh/known_hosts.old # Host 172.23.107.228 found: line 10 /home/couchbase/.ssh/known_hosts updated. Original contents retained as /home/couchbase/.ssh/known_hosts.old # Host 172.23.106.61 found: line 10 /home/couchbase/.ssh/known_hosts updated. Original contents retained as /home/couchbase/.ssh/known_hosts.old # Host 172.23.107.227 found: line 10 /home/couchbase/.ssh/known_hosts updated. Original contents retained as /home/couchbase/.ssh/known_hosts.old Publishing public key ./resources/my.key.pub to all nodes. ################################################################ Running test: #1 Test params: workload=register,replicas=2,no-autofailover,rate=0,durability=25:25:25:25 Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --package couchbase-server-enterprise_neo-ubuntu18.04_amd64.deb --workload=register --replicas=2 --no-autofailover --rate=0 --durability=25:25:25:25 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-1.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 1 unknown: 0 fail: 0 crash: 0 memcached errors (not reported as total): 0 1/1 = 100.00% ################################################################ ################################################################ Running test: #2 Test params: workload=failover,failover-type=graceful,recovery-type=delta,replicas=1,no-autofailover,disrupt-count=1,rate=0,durability=25:25:25:25 Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=failover --failover-type=graceful --recovery-type=delta --replicas=1 --no-autofailover --disrupt-count=1 --rate=0 --durability=25:25:25:25 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-2.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 2 unknown: 0 fail: 0 crash: 0 memcached errors (not reported as total): 0 2/2 = 100.00% ################################################################ ################################################################ Running test: #3 Test params: workload=failover,scopes=3,collections=1,failover-type=graceful,recovery-type=delta,replicas=2,no-autofailover,disrupt-count=1,rate=0,durability=0:0:100:0,cas Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=failover --scopes=3 --collections=1 --failover-type=graceful --recovery-type=delta --replicas=2 --no-autofailover --disrupt-count=1 --rate=0 --durability=0:0:100:0 --cas --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-3.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 3 unknown: 0 fail: 0 crash: 0 memcached errors (not reported as total): 0 3/3 = 100.00% ################################################################ ################################################################ Running test: #4 Test params: workload=failover,failover-type=graceful,recovery-type=full,replicas=1,no-autofailover,disrupt-count=1,rate=0,durability=0:100:0:0,use-json-docs,use-subdoc Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=failover --failover-type=graceful --recovery-type=full --replicas=1 --no-autofailover --disrupt-count=1 --rate=0 --durability=0:100:0:0 --use-json-docs --use-subdoc --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-4.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 4 unknown: 0 fail: 0 crash: 0 memcached errors (not reported as total): 0 4/4 = 100.00% ################################################################ ################################################################ Running test: #5 Test params: workload=rebalance,scenario=sequential-rebalance-out-in,replicas=1,no-autofailover,disrupt-count=1,rate=0,durability=25:25:25:25,cas Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=rebalance --scenario=sequential-rebalance-out-in --replicas=1 --no-autofailover --disrupt-count=1 --rate=0 --durability=25:25:25:25 --cas --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-5.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 5 unknown: 0 fail: 0 crash: 0 memcached errors (not reported as total): 0 5/5 = 100.00% ################################################################ ################################################################ Running test: #6 Test params: workload=rebalance,scenario=sequential-rebalance-out-in,replicas=2,no-autofailover,disrupt-count=1,rate=0,durability=0:100:0:0 Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=rebalance --scenario=sequential-rebalance-out-in --replicas=2 --no-autofailover --disrupt-count=1 --rate=0 --durability=0:100:0:0 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-6.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 6 unknown: 0 fail: 0 crash: 0 memcached errors (not reported as total): 0 6/6 = 100.00% ################################################################ ################################################################ Running test: #7 Test params: workload=rebalance,scenario=bulk-rebalance-out-in,no-autofailover,replicas=2,disrupt-count=2,kv-timeout=1.5,durability=0:0:100:0,cas Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=rebalance --scenario=bulk-rebalance-out-in --no-autofailover --replicas=2 --disrupt-count=2 --kv-timeout=1.5 --durability=0:0:100:0 --cas --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-7.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 7 unknown: 0 fail: 0 crash: 0 memcached errors (not reported as total): 0 7/7 = 100.00% ################################################################ ################################################################ Running test: #8 Test params: workload=rebalance,scenario=swap-rebalance,no-autofailover,replicas=1,disrupt-count=1,kv-timeout=30,durability=0:0:0:100,use-json-docs,doc-padding-size=4096 Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-8.log file to use error_msg_from_memcached.log 05:34:41.183 [main] INFO jepsen.cli - Test options: {:scopes nil, :hashdump true, :cycles 1, :ssh-private-key "./resources/my.key", :enable-memcached-debug-log-level true, :bucket-type :couchbase, :persist-to 0, :collect-core-files false, :manipulate-disks false, :concurrency 4, :pool-size 6, :autoreprovision-maxnodes 1, :leave-db-running? false, :use-checker nil, :skip-teardown false, :logging-json? false, :dcp-set-read false, :replicate-to 0, :cluster-run false, :cas false, :ssh {:dummy? false, :username "root", :password "root", :strict-host-key-checking false, :private-key-path "./resources/my.key"}, :net-interface "eth0", :collections nil, :doc-padding-size 4096, :use-subdoc false, :argv ("test" "--nodes-file" "./nodes" "--username" "root" "--ssh-private-key=./resources/my.key" "--workload=rebalance" "--scenario=swap-rebalance" "--no-autofailover" "--replicas=1" "--disrupt-count=1" "--kv-timeout=30" "--durability=0:0:0:100" "--use-json-docs" "--doc-padding-size=4096" "--enable-tcp-capture" "--hashdump" "--enable-memcached-debug-log-level" "--net-interface=eth0"), :durability (0 0 0 100), :disable-out-of-order-execution false, :nodes ["172.23.105.187" "172.23.107.228" "172.23.106.61" "172.23.107.227"], :test-count 1, :install-path "/opt/couchbase", :doc-count 30, :process-suspend-time 10, :perf-graphs false, :replicas 1, :autofailover false, :storage-backend :couchstore, :cbcollect true, :connect-timeout 10, :autoreprovision true, :disrupt-count 1, :collect-data-files false, :doc-threads 3, :kv-timeout #object[java.time.Duration 0x478cebdb "PT30S"], :disable-auto-compaction false, :time-limit 60, :use-json-docs true, :eviction-policy "fullEviction", :scenario :swap-rebalance, :workload "rebalance", :recovery-type :delta, :enable-tcp-capture true} INFO [2022-01-10 05:34:41,422] jepsen test runner - jepsen.core Test version 7a53fac81fdd4c21e7b4a3d63887911ad58baa52 (plus uncommitted changes) INFO [2022-01-10 05:34:41,423] jepsen test runner - jepsen.core Command line: lein run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 INFO [2022-01-10 05:34:41,515] jepsen test runner - jepsen.core Running test: {:scopes nil :remote #jepsen.control.retry.Remote{:remote #jepsen.control.scp.Remote{:cmd-remote #jepsen.control.clj_ssh.Remote{:concurrency-limit 8, :conn-spec nil, :session nil, :semaphore nil}, :conn-spec nil}, :conn nil} :hashdump true :cycles 1 :ssh-private-key "./resources/my.key" :enable-memcached-debug-log-level true :bucket-type :couchbase :persist-to 0 :collect-core-files false :manipulate-disks false :concurrency 90 :pool-size 6 :autoreprovision-maxnodes 1 :db #object[couchbase.core$couchbase_remote$reify__3667 "0x7c751692" "couchbase.core$couchbase_remote$reify__3667@7c751692"] :leave-db-running? false :use-checker nil :skip-teardown false :name "Couchbase" :logging-json? false :dcp-set-read false :start-time #object[org.joda.time.DateTime "0x820b78a" "2022-01-10T05:34:41.000-08:00"] :net #object[jepsen.net$reify__9402 "0x15a41088" "jepsen.net$reify__9402@15a41088"] :client #couchbase.clients.NewRegisterClient{:cluster nil, :bucket nil, :collection nil, :env nil} :replicate-to 0 :barrier #object[java.util.concurrent.CyclicBarrier "0x53335328" "java.util.concurrent.CyclicBarrier@53335328"] :cluster-run false :cas false :pure-generators true :ssh {:dummy? false, :username "root", :password "root", :strict-host-key-checking false, :private-key-path "./resources/my.key"} :net-interface "eth0" :collections nil :doc-padding-size 4096 :checker #object[jepsen.checker$compose$reify__8903 "0xf1e1d66" "jepsen.checker$compose$reify__8903@f1e1d66"] :use-subdoc false :argv ("test" "--nodes-file" "./nodes" "--username" "root" "--ssh-private-key=./resources/my.key" "--workload=rebalance" "--scenario=swap-rebalance" "--no-autofailover" "--replicas=1" "--disrupt-count=1" "--kv-timeout=30" "--durability=0:0:0:100" "--use-json-docs" "--doc-padding-size=4096" "--enable-tcp-capture" "--hashdump" "--enable-memcached-debug-log-level" "--net-interface=eth0") :db-intialized #object[clojure.lang.Atom "0x43edd2ea" {:status :ready, :val false}] :nemesis #unprintable "couchbase.nemesis$couchbase$reify__3984@4cd9ee2c" :durability (0 0 0 100) :disable-out-of-order-execution false :workload-type :register :nodes ["172.23.105.187" "172.23.107.228" "172.23.106.61" "172.23.107.227"] :test-count 1 :install-path "/opt/couchbase" :doc-count 30 :process-suspend-time 10 :perf-graphs false :replicas 1 :autofailover false :storage-backend :couchstore :cbcollect true :connect-timeout 10 :autoreprovision true :disrupt-count 1 :generator (couchbase.workload.Stopable {:gen (jepsen.generator.Any {:gens [(jepsen.generator.OnThreads {:f #{:nemesis}, :gen [[{:type :sleep, :value 5} {:type :info, :f :rebalance-out, :targeter #object[couchbase.nemesis$start_stop_targeter$start_stop_targeter_invoke__3737 "0x47f416d0" "couchbase.nemesis$start_stop_targeter$start_stop_targeter_invoke__3737@47f416d0"], :target-count 1, :target-action :start}] ([{:type :sleep, :value 5} {:type :info, :f :swap-rebalance, :targeter #object[couchbase.nemesis$start_stop_targeter$start_stop_targeter_invoke__3737 "0x47f416d0" "couchbase.nemesis$start_stop_targeter$start_stop_targeter_invoke__3737@47f416d0"], :target-count 1, :target-action :swap}]) {:type :sleep, :value 10} {:type :stop-test}]}) (jepsen.generator.OnThreads {:f #object[clojure.core$complement$fn__5686 "0x4acdf6e7" "clojure.core$complement$fn__5686@4acdf6e7"], :gen (jepsen.generator.OnThreads {:f #object[clojure.core$complement$fn__5686 "0x2de82bf8" "clojure.core$complement$fn__5686@2de82bf8"], :gen (jepsen.independent.ConcurrentGenerator {:n 3, :fgen #object[couchbase.workload$register_client_gen$fn__3426 "0x54ec8ab3" "couchbase.workload$register_client_gen$fn__3426@54ec8ab3"], :group->threads nil, :thread->group nil, :keys (0 1 2 3 4 5 6 7 ...), :gens nil})})})]})}) :collect-data-files false :doc-threads 3 :kv-timeout #object[java.time.Duration "0x478cebdb" "PT30S"] :disable-auto-compaction false :os #object[jepsen.os$reify__2507 "0x1d686622" "jepsen.os$reify__2507@1d686622"] :time-limit 60 :use-json-docs true :eviction-policy "fullEviction" :scenario :swap-rebalance :workload "rebalance" :recovery-type :delta :enable-tcp-capture true} INFO [2022-01-10 05:34:42,337] jepsen test runner - jepsen.db Tearing down DB INFO [2022-01-10 05:34:42,350] jepsen node 172.23.106.61 - couchbase.util Tearing down couchbase node INFO [2022-01-10 05:34:42,350] jepsen node 172.23.105.187 - couchbase.util Tearing down couchbase node INFO [2022-01-10 05:34:42,350] jepsen node 172.23.107.227 - couchbase.util Tearing down couchbase node INFO [2022-01-10 05:34:42,351] jepsen node 172.23.107.228 - couchbase.util Tearing down couchbase node INFO [2022-01-10 05:34:44,201] jepsen node 172.23.107.228 - couchbase.util Teardown Complete INFO [2022-01-10 05:34:44,408] jepsen node 172.23.105.187 - couchbase.util Teardown Complete INFO [2022-01-10 05:34:44,615] jepsen node 172.23.106.61 - couchbase.util Teardown Complete INFO [2022-01-10 05:34:44,826] jepsen node 172.23.107.227 - couchbase.util Teardown Complete INFO [2022-01-10 05:34:44,830] jepsen test runner - jepsen.db Setting up DB INFO [2022-01-10 05:34:44,834] jepsen node 172.23.107.227 - couchbase.util Setting up Couchbase Server INFO [2022-01-10 05:34:44,834] jepsen node 172.23.105.187 - couchbase.util Setting up Couchbase Server INFO [2022-01-10 05:34:44,834] jepsen node 172.23.107.228 - couchbase.util Setting up Couchbase Server INFO [2022-01-10 05:34:44,834] jepsen node 172.23.106.61 - couchbase.util Setting up Couchbase Server INFO [2022-01-10 05:34:44,834] jepsen node 172.23.107.227 - couchbase.util Setting up data paths INFO [2022-01-10 05:34:44,834] jepsen node 172.23.105.187 - couchbase.util Setting up data paths INFO [2022-01-10 05:34:44,834] jepsen node 172.23.106.61 - couchbase.util Setting up data paths INFO [2022-01-10 05:34:44,834] jepsen node 172.23.107.228 - couchbase.util Setting up data paths INFO [2022-01-10 05:34:44,936] jepsen node 172.23.106.61 - couchbase.util Changing permissions for /opt/couchbase INFO [2022-01-10 05:34:44,936] jepsen node 172.23.105.187 - couchbase.util Changing permissions for /opt/couchbase INFO [2022-01-10 05:34:44,936] jepsen node 172.23.107.227 - couchbase.util Changing permissions for /opt/couchbase INFO [2022-01-10 05:34:44,936] jepsen node 172.23.107.228 - couchbase.util Changing permissions for /opt/couchbase INFO [2022-01-10 05:34:45,139] jepsen node 172.23.107.228 - couchbase.util Starting daemon INFO [2022-01-10 05:34:45,139] jepsen node 172.23.107.227 - couchbase.util Starting daemon INFO [2022-01-10 05:34:45,239] jepsen node 172.23.106.61 - couchbase.util Starting daemon INFO [2022-01-10 05:34:45,239] jepsen node 172.23.105.187 - couchbase.util Starting daemon INFO [2022-01-10 05:35:02,003] jepsen node 172.23.106.61 - couchbase.util Daemon started INFO [2022-01-10 05:35:02,106] jepsen node 172.23.106.61 - couchbase.util packet dump file name Couchbase-172.23.106.61.pcap INFO [2022-01-10 05:35:02,109] jepsen node 172.23.107.227 - couchbase.util Daemon started INFO [2022-01-10 05:35:02,211] jepsen node 172.23.107.227 - couchbase.util packet dump file name Couchbase-172.23.107.227.pcap INFO [2022-01-10 05:35:02,451] jepsen node 172.23.107.228 - couchbase.util Daemon started INFO [2022-01-10 05:35:02,454] jepsen node 172.23.105.187 - couchbase.util Daemon started INFO [2022-01-10 05:35:02,553] jepsen node 172.23.107.228 - couchbase.util packet dump file name Couchbase-172.23.107.228.pcap INFO [2022-01-10 05:35:02,556] jepsen node 172.23.105.187 - couchbase.util packet dump file name Couchbase-172.23.105.187.pcap INFO [2022-01-10 05:35:02,759] jepsen test runner - jepsen.db Setting up primary 172.23.105.187 INFO [2022-01-10 05:35:02,761] jepsen node 172.23.105.187 - couchbase.util Creating couchbase cluster from 172.23.105.187 INFO [2022-01-10 05:35:03,012] jepsen node 172.23.105.187 - couchbase.util Adding node 172.23.107.228 to cluster INFO [2022-01-10 05:35:18,241] jepsen node 172.23.105.187 - couchbase.util Adding node 172.23.106.61 to cluster INFO [2022-01-10 05:35:24,535] jepsen node 172.23.105.187 - couchbase.util Adding node 172.23.107.227 to cluster INFO [2022-01-10 05:35:31,114] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {} INFO [2022-01-10 05:35:31,115] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 0 INFO [2022-01-10 05:35:31,115] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:35:32,136] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:35:32,136] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 1 INFO [2022-01-10 05:35:32,137] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:35:33,145] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:35:33,145] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 2 INFO [2022-01-10 05:35:33,146] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 1 INFO [2022-01-10 05:35:34,153] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:35:34,153] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 3 INFO [2022-01-10 05:35:34,153] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 2 INFO [2022-01-10 05:35:35,160] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:35:35,160] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 4 INFO [2022-01-10 05:35:35,160] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 3 INFO [2022-01-10 05:35:36,167] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:35:36,168] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 5 INFO [2022-01-10 05:35:36,168] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 4 INFO [2022-01-10 05:35:37,175] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:35:37,176] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 6 INFO [2022-01-10 05:35:37,176] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 5 INFO [2022-01-10 05:35:38,183] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status none} INFO [2022-01-10 05:35:38,184] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 7 INFO [2022-01-10 05:35:38,184] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 5 INFO [2022-01-10 05:35:38,185] jepsen node 172.23.105.187 - couchbase.util Rebalance complete INFO [2022-01-10 05:35:38,271] jepsen node 172.23.105.187 - couchbase.util Waiting for bucket warmup to complete... INFO [2022-01-10 05:35:38,290] jepsen node 172.23.105.187 - couchbase.util Setting memcached log level to debug INFO [2022-01-10 05:35:38,315] jepsen node 172.23.105.187 - couchbase.util Setup complete INFO [2022-01-10 05:35:38,318] jepsen test runner - jepsen.core Relative time begins now INFO [2022-01-10 05:35:38,328] jepsen node 172.23.105.187 - couchbase.cbclients Opening new client INFO [2022-01-10 05:35:40,637] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":30000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0xefe9906200000001","seedNodes":[{"address":"172.23.105.187"}]} INFO [2022-01-10 05:35:40,651] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000001","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:35:40,670] jepsen node 172.23.105.187 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 05:35:40,671] jepsen node 172.23.105.187 - couchbase.cbclients Opening new client INFO [2022-01-10 05:35:40,707] jepsen node 172.23.105.187 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 05:35:40,707] jepsen node 172.23.105.187 - couchbase.cbclients Opening new client INFO [2022-01-10 05:35:40,727] jepsen node 172.23.105.187 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 05:35:40,728] jepsen node 172.23.105.187 - couchbase.cbclients Opening new client INFO [2022-01-10 05:35:40,753] jepsen node 172.23.105.187 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 05:35:40,753] jepsen node 172.23.105.187 - couchbase.cbclients Opening new client INFO [2022-01-10 05:35:40,779] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":30000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0xefe9906200000002","seedNodes":[{"address":"172.23.107.227"}]} INFO [2022-01-10 05:35:40,779] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000002","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 05:35:40,795] jepsen node 172.23.105.187 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 05:35:40,796] jepsen node 172.23.105.187 - couchbase.cbclients Opening new client INFO [2022-01-10 05:35:40,814] jepsen node 172.23.105.187 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 05:35:40,821] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":30000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0xefe9906200000003","seedNodes":[{"address":"172.23.106.61"}]} INFO [2022-01-10 05:35:40,824] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000003","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 05:35:40,836] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":30000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0xefe9906200000004","seedNodes":[{"address":"172.23.107.228"}]} INFO [2022-01-10 05:35:40,837] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000004","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 05:35:40,882] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":30000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0xefe9906200000005","seedNodes":[{"address":"172.23.107.228"}]} INFO [2022-01-10 05:35:40,883] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000005","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 05:35:40,905] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":30000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0xefe9906200000006","seedNodes":[{"address":"172.23.106.61"}]} INFO [2022-01-10 05:35:40,906] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000006","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 05:35:41,381] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000002","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:35:41,383] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][659ms] Opened bucket "default" {"coreId":"0xefe9906200000002"} INFO [2022-01-10 05:35:41,384] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000002","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 05:35:41,388] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000005","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:35:41,388] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000005","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 05:35:41,388] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000005","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 05:35:41,414] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000006","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:35:41,414] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000006","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 05:35:41,415] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000006","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 05:35:41,434] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000003","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:35:41,435] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][638ms] Opened bucket "default" {"coreId":"0xefe9906200000003"} INFO [2022-01-10 05:35:41,435] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000003","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 05:35:41,435] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000003","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 05:35:41,438] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000004","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:35:41,439] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][618ms] Opened bucket "default" {"coreId":"0xefe9906200000004"} INFO [2022-01-10 05:35:41,439] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000004","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 05:35:41,439] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000004","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 05:35:41,455] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000001","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 05:35:41,455] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000001","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 05:35:41,455] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000001","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 05:35:41,456] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][849ms] Opened bucket "default" {"coreId":"0xefe9906200000001"} INFO [2022-01-10 05:35:41,485] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000002","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 05:35:41,515] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][684ms] Opened bucket "default" {"coreId":"0xefe9906200000006"} INFO [2022-01-10 05:35:41,589] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][704ms] Opened bucket "default" {"coreId":"0xefe9906200000005"} INFO [2022-01-10 05:35:46,000] jepsen worker 56 - jepsen.util 56 :invoke :write [18 14] INFO [2022-01-10 05:35:46,000] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 05:35:46,002] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 05:35:46,018] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 05:35:46,030] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 05:35:46,054] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 05:35:46,064] jepsen worker 35 - jepsen.util 35 :invoke :write [11 28] INFO [2022-01-10 05:35:46,073] jepsen worker 10 - jepsen.util 10 :invoke :write [3 1] INFO [2022-01-10 05:35:46,081] jepsen worker 19 - jepsen.util 19 :invoke :write [6 25] INFO [2022-01-10 05:35:46,090] jepsen worker 30 - jepsen.util 30 :ok :read [10 :nil] INFO [2022-01-10 05:35:46,093] jepsen worker 72 - jepsen.util 72 :ok :read [24 :nil] INFO [2022-01-10 05:35:46,098] jepsen worker 24 - jepsen.util 24 :ok :read [8 :nil] INFO [2022-01-10 05:35:46,099] jepsen worker 1 - jepsen.util 1 :ok :read [0 :nil] INFO [2022-01-10 05:35:46,101] jepsen worker 6 - jepsen.util 6 :ok :read [2 :nil] INFO [2022-01-10 05:35:46,101] jepsen worker 65 - jepsen.util 65 :invoke :write [21 26] INFO [2022-01-10 05:35:46,110] jepsen worker 59 - jepsen.util 59 :invoke :write [19 30] INFO [2022-01-10 05:35:46,127] jepsen worker 43 - jepsen.util 43 :invoke :write [14 2] INFO [2022-01-10 05:35:46,152] jepsen worker 72 - jepsen.util 72 :invoke :write [24 2] INFO [2022-01-10 05:35:46,170] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 05:35:46,177] jepsen worker 18 - jepsen.util 18 :ok :read [6 :nil] INFO [2022-01-10 05:35:46,204] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 05:35:46,212] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 05:35:46,219] jepsen worker 82 - jepsen.util 82 :ok :read [27 :nil] INFO [2022-01-10 05:35:46,223] jepsen worker 47 - jepsen.util 47 :ok :read [15 :nil] INFO [2022-01-10 05:35:46,226] jepsen worker 30 - jepsen.util 30 :invoke :write [10 7] INFO [2022-01-10 05:35:46,232] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 05:35:46,251] jepsen worker 74 - jepsen.util 74 :ok :read [24 :nil] INFO [2022-01-10 05:35:46,278] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 05:35:46,321] jepsen worker 77 - jepsen.util 77 :ok :read [25 :nil] INFO [2022-01-10 05:35:46,324] jepsen worker 7 - jepsen.util 7 :invoke :write [2 21] INFO [2022-01-10 05:35:46,356] jepsen worker 9 - jepsen.util 9 :invoke :write [3 10] INFO [2022-01-10 05:35:46,363] jepsen worker 16 - jepsen.util 16 :invoke :write [5 18] INFO [2022-01-10 05:35:46,403] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 05:35:46,431] jepsen worker 38 - jepsen.util 38 :ok :read [12 :nil] INFO [2022-01-10 05:35:46,432] jepsen worker 15 - jepsen.util 15 :invoke :write [5 24] INFO [2022-01-10 05:35:46,477] jepsen worker 57 - jepsen.util 57 :invoke :write [19 15] INFO [2022-01-10 05:35:46,522] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 05:35:46,542] jepsen worker 66 - jepsen.util 66 :ok :read [22 :nil] INFO [2022-01-10 05:35:46,577] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 05:35:46,595] jepsen worker 53 - jepsen.util 53 :ok :read [17 :nil] INFO [2022-01-10 05:35:46,610] jepsen worker 45 - jepsen.util 45 :invoke :write [15 0] INFO [2022-01-10 05:35:46,616] jepsen worker 18 - jepsen.util 18 :invoke :write [6 22] INFO [2022-01-10 05:35:46,646] jepsen worker 64 - jepsen.util 64 :invoke :write [21 7] INFO [2022-01-10 05:35:46,652] jepsen worker 4 - jepsen.util 4 :invoke :write [1 13] INFO [2022-01-10 05:35:46,682] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 05:35:46,707] jepsen worker 61 - jepsen.util 61 :ok :read [20 :nil] INFO [2022-01-10 05:35:46,862] jepsen worker 62 - jepsen.util 62 :invoke :write [20 46] INFO [2022-01-10 05:35:46,875] jepsen worker 83 - jepsen.util 83 :invoke :write [27 30] INFO [2022-01-10 05:35:46,946] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 05:35:46,946] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 05:35:46,966] jepsen worker 0 - jepsen.util 0 :ok :read [0 :nil] INFO [2022-01-10 05:35:46,966] jepsen worker 80 - jepsen.util 80 :ok :read [26 :nil] INFO [2022-01-10 05:35:46,987] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 05:35:47,001] jepsen worker 82 - jepsen.util 82 :ok :read [27 :nil] INFO [2022-01-10 05:35:47,056] jepsen worker 31 - jepsen.util 31 :invoke :write [10 0] INFO [2022-01-10 05:35:47,093] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 05:35:47,104] jepsen worker 66 - jepsen.util 66 :ok :read [22 :nil] INFO [2022-01-10 05:35:47,160] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 05:35:47,172] jepsen worker 51 - jepsen.util 51 :ok :read [17 :nil] INFO [2022-01-10 05:35:47,247] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 05:35:47,253] jepsen worker 6 - jepsen.util 6 :ok :read [2 :nil] INFO [2022-01-10 05:35:47,323] jepsen worker 22 - jepsen.util 22 :invoke :write [7 36] INFO [2022-01-10 05:35:47,335] jepsen worker 32 - jepsen.util 32 :invoke :write [10 39] INFO [2022-01-10 05:35:47,361] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 05:35:47,386] jepsen worker 82 - jepsen.util 82 :ok :read [27 :nil] INFO [2022-01-10 05:35:47,447] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 05:35:47,456] jepsen worker 34 - jepsen.util 34 :ok :read [11 :nil] INFO [2022-01-10 05:35:47,616] jepsen worker 58 - jepsen.util 58 :invoke :write [19 12] INFO [2022-01-10 05:35:47,639] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 05:35:47,646] jepsen worker 47 - jepsen.util 47 :ok :read [15 :nil] INFO [2022-01-10 05:35:47,730] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 05:35:47,586] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 05:35:47,931] jepsen worker 17 - jepsen.util 17 :ok :read [5 :nil] INFO [2022-01-10 05:35:47,945] jepsen worker 36 - jepsen.util 36 :ok :read [12 :nil] INFO [2022-01-10 05:35:47,996] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 05:35:48,001] jepsen worker 80 - jepsen.util 80 :invoke :write [26 10] INFO [2022-01-10 05:35:48,027] jepsen worker 38 - jepsen.util 38 :ok :read [12 :nil] INFO [2022-01-10 05:35:48,056] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 05:35:48,062] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 05:35:48,081] jepsen worker 76 - jepsen.util 76 :ok :read [25 :nil] INFO [2022-01-10 05:35:48,142] jepsen worker 88 - jepsen.util 88 :invoke :write [29 6] INFO [2022-01-10 05:35:48,146] jepsen worker 5 - jepsen.util 5 :invoke :write [1 3] INFO [2022-01-10 05:35:48,174] jepsen worker 86 - jepsen.util 86 :ok :read [28 :nil] INFO [2022-01-10 05:35:48,177] jepsen worker 73 - jepsen.util 73 :invoke :write [24 13] INFO [2022-01-10 05:35:48,181] jepsen worker 79 - jepsen.util 79 :invoke :read [26 nil] INFO [2022-01-10 05:35:48,189] jepsen worker 79 - jepsen.util 79 :ok :read [26 :nil] INFO [2022-01-10 05:35:48,315] jepsen worker 51 - jepsen.util 51 :invoke :write [17 28] INFO [2022-01-10 05:35:48,319] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 05:35:48,327] jepsen worker 42 - jepsen.util 42 :invoke :write [14 14] INFO [2022-01-10 05:35:48,327] jepsen worker 44 - jepsen.util 44 :invoke :write [14 41] INFO [2022-01-10 05:35:48,351] jepsen worker 87 - jepsen.util 87 :ok :read [29 :nil] INFO [2022-01-10 05:35:48,383] jepsen worker 28 - jepsen.util 28 :invoke :read [9 nil] INFO [2022-01-10 05:35:48,440] jepsen worker 63 - jepsen.util 63 :invoke :write [21 37] INFO [2022-01-10 05:35:48,513] jepsen worker 28 - jepsen.util 28 :ok :read [9 :nil] INFO [2022-01-10 05:35:48,599] jepsen worker 23 - jepsen.util 23 :invoke :write [7 47] INFO [2022-01-10 05:35:48,603] jepsen worker 84 - jepsen.util 84 :invoke :write [28 41] INFO [2022-01-10 05:35:48,607] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 05:35:48,626] jepsen worker 55 - jepsen.util 55 :ok :read [18 :nil] INFO [2022-01-10 05:35:48,635] jepsen worker 71 - jepsen.util 71 :invoke :write [23 42] INFO [2022-01-10 05:35:48,636] jepsen worker 89 - jepsen.util 89 :invoke :write [29 14] INFO [2022-01-10 05:35:48,638] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 05:35:48,653] jepsen worker 26 - jepsen.util 26 :ok :read [8 :nil] INFO [2022-01-10 05:35:48,743] jepsen worker 47 - jepsen.util 47 :invoke :write [15 19] INFO [2022-01-10 05:35:48,747] jepsen worker 20 - jepsen.util 20 :invoke :read [6 nil] INFO [2022-01-10 05:35:48,765] jepsen worker 20 - jepsen.util 20 :ok :read [6 :nil] INFO [2022-01-10 05:35:48,783] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 05:35:48,787] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 05:35:48,797] jepsen worker 6 - jepsen.util 6 :ok :read [2 :nil] INFO [2022-01-10 05:35:48,873] jepsen worker 33 - jepsen.util 33 :ok :read [11 :nil] INFO [2022-01-10 05:35:48,880] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 05:35:48,910] jepsen worker 82 - jepsen.util 82 :ok :read [27 :nil] INFO [2022-01-10 05:35:48,911] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 05:35:48,915] jepsen worker 81 - jepsen.util 81 :invoke :write [27 1] INFO [2022-01-10 05:35:49,019] jepsen worker 2 - jepsen.util 2 :invoke :write [0 10] INFO [2022-01-10 05:35:49,116] jepsen worker 27 - jepsen.util 27 :ok :read [9 :nil] INFO [2022-01-10 05:35:49,172] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 05:35:49,179] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 05:35:49,213] jepsen worker 8 - jepsen.util 8 :ok :read [2 :nil] INFO [2022-01-10 05:35:49,251] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 05:35:49,256] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 05:35:49,273] jepsen worker 85 - jepsen.util 85 :ok :read [28 :nil] INFO [2022-01-10 05:35:49,348] jepsen worker 27 - jepsen.util 27 :invoke :write [9 26] INFO [2022-01-10 05:35:49,351] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 05:35:49,365] jepsen worker 29 - jepsen.util 29 :ok :read [9 :nil] INFO [2022-01-10 05:35:49,377] jepsen worker 74 - jepsen.util 74 :ok :read [24 :nil] INFO [2022-01-10 05:35:49,437] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 05:35:49,441] jepsen worker 50 - jepsen.util 50 :ok :read [16 :nil] INFO [2022-01-10 05:35:49,441] jepsen worker 14 - jepsen.util 14 :invoke :write [4 6] INFO [2022-01-10 05:35:49,454] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 05:35:49,457] jepsen worker 55 - jepsen.util 55 :ok :read [18 :nil] INFO [2022-01-10 05:35:49,497] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 05:35:49,501] jepsen worker 17 - jepsen.util 17 :ok :read [5 :nil] INFO [2022-01-10 05:35:49,533] jepsen worker 82 - jepsen.util 82 :invoke :write [27 2] INFO [2022-01-10 05:35:49,566] jepsen worker 8 - jepsen.util 8 :invoke :write [2 12] INFO [2022-01-10 05:35:49,603] jepsen worker 66 - jepsen.util 66 :invoke :write [22 33] INFO [2022-01-10 05:35:49,618] jepsen worker 20 - jepsen.util 20 :invoke :read [6 nil] INFO [2022-01-10 05:35:49,630] jepsen worker 20 - jepsen.util 20 :ok :read [6 :nil] INFO [2022-01-10 05:35:49,734] jepsen worker 87 - jepsen.util 87 :invoke :write [29 17] INFO [2022-01-10 05:35:49,739] jepsen worker 52 - jepsen.util 52 :invoke :write [17 27] INFO [2022-01-10 05:35:49,941] jepsen worker 74 - jepsen.util 74 :invoke :write [24 35] INFO [2022-01-10 05:35:49,947] jepsen worker 55 - jepsen.util 55 :invoke :write [18 42] INFO [2022-01-10 05:35:50,031] jepsen worker 54 - jepsen.util 54 :invoke :write [18 14] INFO [2022-01-10 05:35:50,076] jepsen worker 11 - jepsen.util 11 :invoke :write [3 27] INFO [2022-01-10 05:35:50,112] jepsen worker 21 - jepsen.util 21 :invoke :write [7 41] INFO [2022-01-10 05:35:50,145] jepsen worker 25 - jepsen.util 25 :invoke :write [8 9] INFO [2022-01-10 05:35:50,173] jepsen worker 86 - jepsen.util 86 :invoke :write [28 21] INFO [2022-01-10 05:35:50,198] jepsen worker 36 - jepsen.util 36 :invoke :write [12 30] INFO [2022-01-10 05:35:50,226] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 05:35:50,229] jepsen worker 17 - jepsen.util 17 :ok :read [5 :nil] INFO [2022-01-10 05:35:50,254] jepsen worker 29 - jepsen.util 29 :invoke :write [9 22] INFO [2022-01-10 05:35:50,323] jepsen worker 20 - jepsen.util 20 :invoke :write [6 34] INFO [2022-01-10 05:35:50,356] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 05:35:50,359] jepsen worker 76 - jepsen.util 76 :ok :read [25 :nil] INFO [2022-01-10 05:35:50,392] jepsen worker 17 - jepsen.util 17 :invoke :write [5 4] INFO [2022-01-10 05:35:50,420] jepsen worker 37 - jepsen.util 37 :invoke :write [12 47] INFO [2022-01-10 05:35:50,452] jepsen worker 39 - jepsen.util 39 :invoke :write [13 32] INFO [2022-01-10 05:35:50,454] jepsen worker 85 - jepsen.util 85 :invoke :write [28 31] INFO [2022-01-10 05:35:50,488] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 05:35:50,490] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 05:35:50,491] jepsen worker 38 - jepsen.util 38 :ok :read [12 :nil] INFO [2022-01-10 05:35:50,504] jepsen worker 33 - jepsen.util 33 :ok :read [11 :nil] INFO [2022-01-10 05:35:50,631] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 05:35:50,632] jepsen worker 48 - jepsen.util 48 :invoke :write [16 48] INFO [2022-01-10 05:35:50,636] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 05:35:50,639] jepsen worker 1 - jepsen.util 1 :ok :read [0 :nil] INFO [2022-01-10 05:35:50,639] jepsen worker 53 - jepsen.util 53 :ok :read [17 :nil] INFO [2022-01-10 05:35:50,643] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 05:35:50,643] jepsen worker 33 - jepsen.util 33 :invoke :write [11 13] INFO [2022-01-10 05:35:50,647] jepsen worker 77 - jepsen.util 77 :ok :read [25 :nil] INFO [2022-01-10 05:35:50,749] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 05:35:50,750] jepsen worker 67 - jepsen.util 67 :invoke :write [22 29] INFO [2022-01-10 05:35:50,752] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 05:35:50,753] jepsen worker 49 - jepsen.util 49 :invoke :write [16 2] INFO [2022-01-10 05:35:50,815] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 05:35:50,824] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 05:35:50,826] jepsen worker 68 - jepsen.util 68 :invoke :write [22 23] INFO [2022-01-10 05:35:50,826] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 05:35:50,839] jepsen worker 12 - jepsen.util 12 :ok :read [4 :nil] INFO [2022-01-10 05:35:50,849] jepsen worker 70 - jepsen.util 70 :ok :read [23 :nil] INFO [2022-01-10 05:35:50,852] jepsen worker 34 - jepsen.util 34 :ok :read [11 :nil] INFO [2022-01-10 05:35:50,855] jepsen worker 60 - jepsen.util 60 :ok :read [20 :nil] INFO [2022-01-10 05:35:50,868] jepsen worker 1 - jepsen.util 1 :invoke :write [0 31] INFO [2022-01-10 05:35:50,869] jepsen worker 28 - jepsen.util 28 :invoke :write [9 19] INFO [2022-01-10 05:35:50,874] jepsen worker 0 - jepsen.util 0 :invoke :write [0 20] INFO [2022-01-10 05:35:50,904] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 05:35:50,905] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 05:35:50,906] jepsen worker 24 - jepsen.util 24 :invoke :write [8 49] INFO [2022-01-10 05:35:50,907] jepsen worker 46 - jepsen.util 46 :ok :read [15 :nil] INFO [2022-01-10 05:35:50,948] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 05:35:50,949] jepsen worker 79 - jepsen.util 79 :invoke :write [26 45] INFO [2022-01-10 05:35:50,950] jepsen worker 78 - jepsen.util 78 :ok :read [26 :nil] INFO [2022-01-10 05:35:50,951] jepsen worker 76 - jepsen.util 76 :invoke :write [25 32] INFO [2022-01-10 05:35:50,952] jepsen worker 13 - jepsen.util 13 :invoke :write [4 18] INFO [2022-01-10 05:35:50,958] jepsen worker 12 - jepsen.util 12 :invoke :write [4 6] INFO [2022-01-10 05:35:50,962] jepsen worker 78 - jepsen.util 78 :invoke :write [26 26] INFO [2022-01-10 05:35:51,011] jepsen worker 70 - jepsen.util 70 :invoke :write [23 10] INFO [2022-01-10 05:35:51,052] jepsen worker 75 - jepsen.util 75 :invoke :write [25 26] INFO [2022-01-10 05:35:51,053] jepsen worker 40 - jepsen.util 40 :invoke :write [13 7] INFO [2022-01-10 05:35:51,202] jepsen worker 53 - jepsen.util 53 :invoke :write [17 25] INFO [2022-01-10 05:35:51,202] jepsen worker 46 - jepsen.util 46 :invoke :write [15 6] INFO [2022-01-10 05:35:51,206] jepsen worker nemesis - jepsen.util :nemesis :info :rebalance-out nil INFO [2022-01-10 05:35:51,250] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 05:35:51,251] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 05:35:51,252] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 05:35:51,253] jepsen worker 6 - jepsen.util 6 :invoke :write [2 31] INFO [2022-01-10 05:35:51,253] jepsen worker 26 - jepsen.util 26 :ok :read [8 :nil] INFO [2022-01-10 05:35:51,253] jepsen worker 41 - jepsen.util 41 :ok :read [13 :nil] INFO [2022-01-10 05:35:51,254] jepsen worker 34 - jepsen.util 34 :ok :read [11 :nil] INFO [2022-01-10 05:35:51,260] jepsen worker 26 - jepsen.util 26 :invoke :write [8 44] INFO [2022-01-10 05:35:51,261] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 05:35:51,262] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 05:35:51,262] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 05:35:51,264] jepsen worker 69 - jepsen.util 69 :ok :read [23 :nil] INFO [2022-01-10 05:35:51,264] jepsen worker 41 - jepsen.util 41 :ok :read [13 :nil] INFO [2022-01-10 05:35:51,264] jepsen worker 61 - jepsen.util 61 :ok :read [20 :nil] INFO [2022-01-10 05:35:51,266] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 05:35:51,267] jepsen worker 34 - jepsen.util 34 :invoke :write [11 30] INFO [2022-01-10 05:35:51,271] jepsen worker 77 - jepsen.util 77 :ok :read [25 :nil] INFO [2022-01-10 05:35:51,328] jepsen worker 61 - jepsen.util 61 :invoke :write [20 18] INFO [2022-01-10 05:35:51,329] jepsen worker 69 - jepsen.util 69 :invoke :write [23 31] INFO [2022-01-10 05:35:51,329] jepsen worker 41 - jepsen.util 41 :invoke :write [13 39] INFO [2022-01-10 05:35:51,329] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 05:35:51,334] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 05:35:51,336] jepsen worker 77 - jepsen.util 77 :ok :read [25 :nil] INFO [2022-01-10 05:35:51,371] jepsen worker 60 - jepsen.util 60 :ok :read [20 :nil] INFO [2022-01-10 05:35:51,371] jepsen worker 77 - jepsen.util 77 :invoke :write [25 5] INFO [2022-01-10 05:35:51,372] jepsen worker 60 - jepsen.util 60 :invoke :write [20 2] INFO [2022-01-10 05:35:52,638] jepsen worker nemesis - couchbase.util Rebalancing nodes (172.23.105.187) out of cluster INFO [2022-01-10 05:35:52,919] jepsen worker nemesis - couchbase.util Rebalance status: {} INFO [2022-01-10 05:35:52,920] jepsen worker nemesis - couchbase.util Rebalance check count: 0 INFO [2022-01-10 05:35:52,920] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:35:54,158] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.01162790697674421}, :ns_1@172.23.105.187 {:progress 0.01171875}, :ns_1@172.23.107.227 {:progress 0.007843137254901933}, :ns_1@172.23.107.228 {:progress 0.02352941176470591}} INFO [2022-01-10 05:35:54,163] jepsen worker nemesis - couchbase.util Rebalance check count: 1 INFO [2022-01-10 05:35:54,164] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:35:55,181] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1046511627906976}, :ns_1@172.23.105.187 {:progress 0.08984375}, :ns_1@172.23.107.227 {:progress 0.1098039215686275}, :ns_1@172.23.107.228 {:progress 0.1098039215686275}} INFO [2022-01-10 05:35:55,181] jepsen worker nemesis - couchbase.util Rebalance check count: 2 INFO [2022-01-10 05:35:55,181] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:35:56,190] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1782945736434108}, :ns_1@172.23.105.187 {:progress 0.1640625}, :ns_1@172.23.107.227 {:progress 0.1764705882352942}, :ns_1@172.23.107.228 {:progress 0.192156862745098}} INFO [2022-01-10 05:35:56,191] jepsen worker nemesis - couchbase.util Rebalance check count: 3 INFO [2022-01-10 05:35:56,191] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:35:57,201] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3178294573643411}, :ns_1@172.23.105.187 {:progress 0.25}, :ns_1@172.23.107.227 {:progress 0.2588235294117647}, :ns_1@172.23.107.228 {:progress 0.2901960784313725}} INFO [2022-01-10 05:35:57,202] jepsen worker nemesis - couchbase.util Rebalance check count: 4 INFO [2022-01-10 05:35:57,202] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:35:58,211] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4108527131782945}, :ns_1@172.23.105.187 {:progress 0.29296875}, :ns_1@172.23.107.227 {:progress 0.3372549019607843}, :ns_1@172.23.107.228 {:progress 0.3647058823529412}} INFO [2022-01-10 05:35:58,211] jepsen worker nemesis - couchbase.util Rebalance check count: 5 INFO [2022-01-10 05:35:58,211] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:35:59,265] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5}, :ns_1@172.23.105.187 {:progress 0.3671875}, :ns_1@172.23.107.227 {:progress 0.4117647058823529}, :ns_1@172.23.107.228 {:progress 0.4470588235294117}} INFO [2022-01-10 05:35:59,265] jepsen worker nemesis - couchbase.util Rebalance check count: 6 INFO [2022-01-10 05:35:59,265] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:36:00,274] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5697674418604651}, :ns_1@172.23.105.187 {:progress 0.4375}, :ns_1@172.23.107.227 {:progress 0.4823529411764705}, :ns_1@172.23.107.228 {:progress 0.5254901960784314}} INFO [2022-01-10 05:36:00,274] jepsen worker nemesis - couchbase.util Rebalance check count: 7 INFO [2022-01-10 05:36:00,274] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:36:01,284] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.7325581395348837}, :ns_1@172.23.105.187 {:progress 0.5}, :ns_1@172.23.107.227 {:progress 0.580392156862745}, :ns_1@172.23.107.228 {:progress 0.6235294117647059}} INFO [2022-01-10 05:36:01,284] jepsen worker nemesis - couchbase.util Rebalance check count: 8 INFO [2022-01-10 05:36:01,284] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:36:02,295] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.8488372093023255}, :ns_1@172.23.105.187 {:progress 0.5}, :ns_1@172.23.107.227 {:progress 0.6509803921568628}, :ns_1@172.23.107.228 {:progress 0.6862745098039216}} INFO [2022-01-10 05:36:02,296] jepsen worker nemesis - couchbase.util Rebalance check count: 9 INFO [2022-01-10 05:36:02,296] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:36:03,317] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.8643410852713178}, :ns_1@172.23.105.187 {:progress 0.546875}, :ns_1@172.23.107.227 {:progress 0.7058823529411764}, :ns_1@172.23.107.228 {:progress 0.7647058823529411}} INFO [2022-01-10 05:36:03,323] jepsen worker nemesis - couchbase.util Rebalance check count: 10 INFO [2022-01-10 05:36:03,323] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:36:04,335] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.9147286821705426}, :ns_1@172.23.105.187 {:progress 0.69140625}, :ns_1@172.23.107.227 {:progress 0.7960784313725491}, :ns_1@172.23.107.228 {:progress 0.8627450980392157}} INFO [2022-01-10 05:36:04,335] jepsen worker nemesis - couchbase.util Rebalance check count: 11 INFO [2022-01-10 05:36:04,335] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:36:05,356] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.9302325581395349}, :ns_1@172.23.105.187 {:progress 0.7890625}, :ns_1@172.23.107.227 {:progress 0.9215686274509804}, :ns_1@172.23.107.228 {:progress 0.9372549019607843}} INFO [2022-01-10 05:36:05,356] jepsen worker nemesis - couchbase.util Rebalance check count: 12 INFO [2022-01-10 05:36:05,356] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:36:06,371] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.9922480620155039}, :ns_1@172.23.105.187 {:progress 0.91015625}, :ns_1@172.23.107.227 {:progress 0.9529411764705882}, :ns_1@172.23.107.228 {:progress 0.9647058823529412}} INFO [2022-01-10 05:36:06,372] jepsen worker nemesis - couchbase.util Rebalance check count: 13 INFO [2022-01-10 05:36:06,372] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:36:06,816] jepsen worker 35 - jepsen.util 35 :ok :write [11 28] INFO [2022-01-10 05:36:06,997] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:07,375] jepsen worker 35 - jepsen.util 35 :ok :read [11 28] INFO [2022-01-10 05:36:07,412] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:07,427] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:07,427] jepsen worker nemesis - couchbase.util Rebalance check count: 14 INFO [2022-01-10 05:36:07,427] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:36:07,520] jepsen worker 35 - jepsen.util 35 :ok :read [11 28] INFO [2022-01-10 05:36:07,617] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:07,767] jepsen worker 35 - jepsen.util 35 :ok :read [11 28] INFO [2022-01-10 05:36:07,772] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:07,941] jepsen worker 35 - jepsen.util 35 :ok :read [11 28] INFO [2022-01-10 05:36:07,950] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:08,112] jepsen worker 35 - jepsen.util 35 :ok :read [11 28] INFO [2022-01-10 05:36:08,151] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:08,342] jepsen worker 35 - jepsen.util 35 :ok :read [11 28] INFO [2022-01-10 05:36:08,404] jepsen worker 35 - jepsen.util 35 :invoke :write [11 12] INFO [2022-01-10 05:36:08,434] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:08,434] jepsen worker nemesis - couchbase.util Rebalance check count: 15 INFO [2022-01-10 05:36:08,434] jepsen worker nemesis - couchbase.util Rebalance stuck count: 1 INFO [2022-01-10 05:36:08,630] jepsen worker 35 - jepsen.util 35 :ok :write [11 12] INFO [2022-01-10 05:36:08,631] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:08,653] jepsen worker 35 - jepsen.util 35 :ok :read [11 12] INFO [2022-01-10 05:36:08,679] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:08,756] jepsen worker 35 - jepsen.util 35 :ok :read [11 12] INFO [2022-01-10 05:36:08,814] jepsen worker 35 - jepsen.util 35 :invoke :write [11 3] INFO [2022-01-10 05:36:09,035] jepsen worker 35 - jepsen.util 35 :ok :write [11 3] INFO [2022-01-10 05:36:09,036] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:09,103] jepsen worker 35 - jepsen.util 35 :ok :read [11 3] INFO [2022-01-10 05:36:09,124] jepsen worker 35 - jepsen.util 35 :invoke :write [11 9] INFO [2022-01-10 05:36:09,327] jepsen worker 35 - jepsen.util 35 :ok :write [11 9] INFO [2022-01-10 05:36:09,440] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:09,440] jepsen worker nemesis - couchbase.util Rebalance check count: 16 INFO [2022-01-10 05:36:09,440] jepsen worker nemesis - couchbase.util Rebalance stuck count: 2 INFO [2022-01-10 05:36:09,492] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:09,533] jepsen worker 35 - jepsen.util 35 :ok :read [11 9] INFO [2022-01-10 05:36:09,539] jepsen worker 35 - jepsen.util 35 :invoke :write [11 37] INFO [2022-01-10 05:36:09,700] jepsen worker 35 - jepsen.util 35 :ok :write [11 37] INFO [2022-01-10 05:36:09,703] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:09,726] jepsen worker 35 - jepsen.util 35 :ok :read [11 37] INFO [2022-01-10 05:36:09,809] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:09,893] jepsen worker 35 - jepsen.util 35 :ok :read [11 37] INFO [2022-01-10 05:36:09,943] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:09,996] jepsen worker 35 - jepsen.util 35 :ok :read [11 37] INFO [2022-01-10 05:36:10,063] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:10,129] jepsen worker 35 - jepsen.util 35 :ok :read [11 37] INFO [2022-01-10 05:36:10,219] jepsen worker 35 - jepsen.util 35 :invoke :write [11 45] INFO [2022-01-10 05:36:10,396] jepsen worker 35 - jepsen.util 35 :ok :write [11 45] INFO [2022-01-10 05:36:10,451] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:10,451] jepsen worker nemesis - couchbase.util Rebalance check count: 17 INFO [2022-01-10 05:36:10,451] jepsen worker nemesis - couchbase.util Rebalance stuck count: 3 INFO [2022-01-10 05:36:10,463] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:10,494] jepsen worker 35 - jepsen.util 35 :ok :read [11 45] INFO [2022-01-10 05:36:10,503] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:10,569] jepsen worker 35 - jepsen.util 35 :ok :read [11 45] INFO [2022-01-10 05:36:10,570] jepsen worker 59 - jepsen.util 59 :ok :write [19 30] INFO [2022-01-10 05:36:10,571] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 05:36:10,650] jepsen worker 59 - jepsen.util 59 :ok :read [19 30] INFO [2022-01-10 05:36:10,658] jepsen worker 59 - jepsen.util 59 :invoke :write [19 11] INFO [2022-01-10 05:36:10,658] jepsen worker 35 - jepsen.util 35 :invoke :write [11 35] INFO [2022-01-10 05:36:10,779] jepsen worker 16 - jepsen.util 16 :ok :write [5 18] INFO [2022-01-10 05:36:10,805] jepsen worker 16 - jepsen.util 16 :invoke :write [5 9] INFO [2022-01-10 05:36:10,850] jepsen worker 35 - jepsen.util 35 :ok :write [11 35] INFO [2022-01-10 05:36:10,875] jepsen worker 35 - jepsen.util 35 :invoke :write [11 9] WARN [2022-01-10 05:36:10,915] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":346881,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":346881,"total_server_duration_us":282750,"encode_duration_us":1201019,"operation_id":"0x43d","timeout_ms":10000,"last_server_duration_us":282750,"total_duration_us":1003942}],"total_count":1}} INFO [2022-01-10 05:36:10,991] jepsen worker 35 - jepsen.util 35 :ok :write [11 9] INFO [2022-01-10 05:36:10,991] jepsen worker 35 - jepsen.util 35 :invoke :write [11 40] INFO [2022-01-10 05:36:11,032] jepsen worker 59 - jepsen.util 59 :ok :write [19 11] INFO [2022-01-10 05:36:11,032] jepsen worker 59 - jepsen.util 59 :invoke :write [19 49] INFO [2022-01-10 05:36:11,168] jepsen worker 16 - jepsen.util 16 :ok :write [5 9] INFO [2022-01-10 05:36:11,168] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 05:36:11,190] jepsen worker 35 - jepsen.util 35 :ok :write [11 40] INFO [2022-01-10 05:36:11,192] jepsen worker 35 - jepsen.util 35 :invoke :write [11 12] INFO [2022-01-10 05:36:11,202] jepsen worker 16 - jepsen.util 16 :ok :read [5 9] INFO [2022-01-10 05:36:11,219] jepsen worker 16 - jepsen.util 16 :invoke :write [5 35] INFO [2022-01-10 05:36:11,224] jepsen worker 59 - jepsen.util 59 :ok :write [19 49] INFO [2022-01-10 05:36:11,231] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 05:36:11,266] jepsen worker 59 - jepsen.util 59 :ok :read [19 49] INFO [2022-01-10 05:36:11,279] jepsen worker 59 - jepsen.util 59 :invoke :write [19 0] INFO [2022-01-10 05:36:11,310] jepsen worker 35 - jepsen.util 35 :ok :write [11 12] INFO [2022-01-10 05:36:11,311] jepsen worker 35 - jepsen.util 35 :invoke :write [11 41] INFO [2022-01-10 05:36:11,393] jepsen worker 16 - jepsen.util 16 :ok :write [5 35] INFO [2022-01-10 05:36:11,399] jepsen worker 16 - jepsen.util 16 :invoke :write [5 46] INFO [2022-01-10 05:36:11,457] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:11,457] jepsen worker nemesis - couchbase.util Rebalance check count: 18 INFO [2022-01-10 05:36:11,457] jepsen worker nemesis - couchbase.util Rebalance stuck count: 4 INFO [2022-01-10 05:36:11,488] jepsen worker 4 - jepsen.util 4 :ok :write [1 13] INFO [2022-01-10 05:36:11,526] jepsen worker 35 - jepsen.util 35 :ok :write [11 41] INFO [2022-01-10 05:36:11,547] jepsen worker 59 - jepsen.util 59 :ok :write [19 0] INFO [2022-01-10 05:36:11,590] jepsen worker 35 - jepsen.util 35 :invoke :write [11 45] INFO [2022-01-10 05:36:11,590] jepsen worker 16 - jepsen.util 16 :ok :write [5 46] INFO [2022-01-10 05:36:11,590] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 05:36:11,591] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 05:36:11,591] jepsen worker 16 - jepsen.util 16 :invoke :write [5 14] INFO [2022-01-10 05:36:11,644] jepsen worker 4 - jepsen.util 4 :ok :read [1 13] INFO [2022-01-10 05:36:11,648] jepsen worker 4 - jepsen.util 4 :invoke :write [1 15] INFO [2022-01-10 05:36:11,810] jepsen worker 16 - jepsen.util 16 :ok :write [5 14] INFO [2022-01-10 05:36:11,811] jepsen worker 16 - jepsen.util 16 :invoke :write [5 37] INFO [2022-01-10 05:36:11,835] jepsen worker 59 - jepsen.util 59 :ok :read [19 0] INFO [2022-01-10 05:36:11,838] jepsen worker 59 - jepsen.util 59 :invoke :write [19 28] INFO [2022-01-10 05:36:11,901] jepsen worker 35 - jepsen.util 35 :ok :write [11 45] INFO [2022-01-10 05:36:11,906] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:12,013] jepsen worker 4 - jepsen.util 4 :ok :write [1 15] INFO [2022-01-10 05:36:12,017] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 05:36:12,031] jepsen worker 65 - jepsen.util 65 :ok :write [21 26] INFO [2022-01-10 05:36:12,036] jepsen worker 65 - jepsen.util 65 :invoke :write [21 49] INFO [2022-01-10 05:36:12,057] jepsen worker 35 - jepsen.util 35 :ok :read [11 45] INFO [2022-01-10 05:36:12,058] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:12,087] jepsen worker 16 - jepsen.util 16 :ok :write [5 37] INFO [2022-01-10 05:36:12,090] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 05:36:12,119] jepsen worker 4 - jepsen.util 4 :ok :read [1 15] INFO [2022-01-10 05:36:12,148] jepsen worker 72 - jepsen.util 72 :ok :write [24 2] INFO [2022-01-10 05:36:12,167] jepsen worker 72 - jepsen.util 72 :invoke :write [24 45] INFO [2022-01-10 05:36:12,167] jepsen worker 4 - jepsen.util 4 :invoke :write [1 22] INFO [2022-01-10 05:36:12,182] jepsen worker 59 - jepsen.util 59 :ok :write [19 28] INFO [2022-01-10 05:36:12,184] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 05:36:12,185] jepsen worker 16 - jepsen.util 16 :ok :read [5 37] INFO [2022-01-10 05:36:12,185] jepsen worker 16 - jepsen.util 16 :invoke :write [5 21] INFO [2022-01-10 05:36:12,198] jepsen worker 65 - jepsen.util 65 :ok :write [21 49] INFO [2022-01-10 05:36:12,200] jepsen worker 65 - jepsen.util 65 :invoke :write [21 46] INFO [2022-01-10 05:36:12,210] jepsen worker 59 - jepsen.util 59 :ok :read [19 28] INFO [2022-01-10 05:36:12,215] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 05:36:12,273] jepsen worker 59 - jepsen.util 59 :ok :read [19 28] INFO [2022-01-10 05:36:12,291] jepsen worker 35 - jepsen.util 35 :ok :read [11 45] INFO [2022-01-10 05:36:12,303] jepsen worker 35 - jepsen.util 35 :invoke :write [11 1] INFO [2022-01-10 05:36:12,347] jepsen worker 59 - jepsen.util 59 :invoke :write [19 19] INFO [2022-01-10 05:36:12,394] jepsen worker 65 - jepsen.util 65 :ok :write [21 46] INFO [2022-01-10 05:36:12,399] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 05:36:12,439] jepsen worker 65 - jepsen.util 65 :ok :read [21 46] INFO [2022-01-10 05:36:12,464] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:12,465] jepsen worker nemesis - couchbase.util Rebalance check count: 19 INFO [2022-01-10 05:36:12,465] jepsen worker nemesis - couchbase.util Rebalance stuck count: 5 INFO [2022-01-10 05:36:12,484] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 05:36:12,682] jepsen worker 65 - jepsen.util 65 :ok :read [21 46] INFO [2022-01-10 05:36:12,721] jepsen worker 72 - jepsen.util 72 :ok :write [24 45] INFO [2022-01-10 05:36:12,723] jepsen worker 16 - jepsen.util 16 :ok :write [5 21] INFO [2022-01-10 05:36:12,723] jepsen worker 4 - jepsen.util 4 :ok :write [1 22] INFO [2022-01-10 05:36:12,785] jepsen worker 4 - jepsen.util 4 :invoke :write [1 38] INFO [2022-01-10 05:36:12,786] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 05:36:12,786] jepsen worker 72 - jepsen.util 72 :invoke :write [24 33] INFO [2022-01-10 05:36:12,791] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 05:36:12,930] jepsen worker 65 - jepsen.util 65 :ok :read [21 46] INFO [2022-01-10 05:36:12,955] jepsen worker 16 - jepsen.util 16 :ok :read [5 21] INFO [2022-01-10 05:36:12,955] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 05:36:13,091] jepsen worker 65 - jepsen.util 65 :invoke :write [21 25] INFO [2022-01-10 05:36:13,095] jepsen worker 35 - jepsen.util 35 :ok :write [11 1] INFO [2022-01-10 05:36:13,097] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 05:36:13,190] jepsen worker 16 - jepsen.util 16 :ok :read [5 21] INFO [2022-01-10 05:36:13,242] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 05:36:13,294] jepsen worker 35 - jepsen.util 35 :ok :read [11 1] INFO [2022-01-10 05:36:13,294] jepsen worker 35 - jepsen.util 35 :invoke :write [11 47] INFO [2022-01-10 05:36:13,417] jepsen worker 59 - jepsen.util 59 :ok :write [19 19] INFO [2022-01-10 05:36:13,418] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 05:36:13,472] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:13,472] jepsen worker nemesis - couchbase.util Rebalance check count: 20 INFO [2022-01-10 05:36:13,472] jepsen worker nemesis - couchbase.util Rebalance stuck count: 6 INFO [2022-01-10 05:36:13,473] jepsen worker 59 - jepsen.util 59 :ok :read [19 19] INFO [2022-01-10 05:36:13,440] jepsen worker 16 - jepsen.util 16 :ok :read [5 21] INFO [2022-01-10 05:36:13,487] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 05:36:13,503] jepsen worker 59 - jepsen.util 59 :invoke :write [19 4] INFO [2022-01-10 05:36:13,688] jepsen worker 16 - jepsen.util 16 :ok :read [5 21] INFO [2022-01-10 05:36:13,740] jepsen worker 30 - jepsen.util 30 :ok :write [10 7] INFO [2022-01-10 05:36:13,745] jepsen worker 30 - jepsen.util 30 :invoke :write [10 36] INFO [2022-01-10 05:36:13,771] jepsen worker 16 - jepsen.util 16 :invoke :write [5 48] INFO [2022-01-10 05:36:14,505] jepsen worker 43 - jepsen.util 43 :ok :write [14 2] INFO [2022-01-10 05:36:14,505] jepsen worker 43 - jepsen.util 43 :invoke :write [14 40] INFO [2022-01-10 05:36:14,507] jepsen worker 4 - jepsen.util 4 :ok :write [1 38] INFO [2022-01-10 05:36:14,507] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 05:36:14,515] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:14,516] jepsen worker nemesis - couchbase.util Rebalance check count: 21 INFO [2022-01-10 05:36:14,516] jepsen worker nemesis - couchbase.util Rebalance stuck count: 7 INFO [2022-01-10 05:36:14,535] jepsen worker 4 - jepsen.util 4 :ok :read [1 38] INFO [2022-01-10 05:36:14,542] jepsen worker 4 - jepsen.util 4 :invoke :write [1 41] INFO [2022-01-10 05:36:14,939] jepsen worker 35 - jepsen.util 35 :ok :write [11 47] INFO [2022-01-10 05:36:14,940] jepsen worker 35 - jepsen.util 35 :invoke :write [11 21] INFO [2022-01-10 05:36:15,197] jepsen worker 9 - jepsen.util 9 :ok :write [3 10] INFO [2022-01-10 05:36:15,197] jepsen worker 9 - jepsen.util 9 :invoke :write [3 22] INFO [2022-01-10 05:36:15,521] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:15,521] jepsen worker nemesis - couchbase.util Rebalance check count: 22 INFO [2022-01-10 05:36:15,522] jepsen worker nemesis - couchbase.util Rebalance stuck count: 8 INFO [2022-01-10 05:36:15,590] jepsen worker 15 - jepsen.util 15 :ok :write [5 24] INFO [2022-01-10 05:36:15,591] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 05:36:15,621] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 05:36:15,626] jepsen worker 15 - jepsen.util 15 :invoke :write [5 28] INFO [2022-01-10 05:36:15,821] jepsen worker 72 - jepsen.util 72 :ok :write [24 33] INFO [2022-01-10 05:36:15,822] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 05:36:15,855] jepsen worker 72 - jepsen.util 72 :ok :read [24 33] INFO [2022-01-10 05:36:15,892] jepsen worker 72 - jepsen.util 72 :invoke :write [24 2] INFO [2022-01-10 05:36:16,019] jepsen worker 56 - jepsen.util 56 :ok :write [18 14] INFO [2022-01-10 05:36:16,023] jepsen worker 51 - jepsen.util 51 :ok :write [17 28] INFO [2022-01-10 05:36:16,140] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 05:36:16,140] jepsen worker 51 - jepsen.util 51 :invoke :write [17 37] INFO [2022-01-10 05:36:16,519] jepsen worker 43 - jepsen.util 43 :ok :write [14 40] INFO [2022-01-10 05:36:16,523] jepsen worker 43 - jepsen.util 43 :invoke :write [14 32] INFO [2022-01-10 05:36:16,566] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:16,574] jepsen worker nemesis - couchbase.util Rebalance check count: 23 INFO [2022-01-10 05:36:16,574] jepsen worker nemesis - couchbase.util Rebalance stuck count: 9 INFO [2022-01-10 05:36:16,707] jepsen worker 64 - jepsen.util 64 :ok :write [21 7] INFO [2022-01-10 05:36:16,711] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 05:36:17,103] jepsen worker 7 - jepsen.util 7 :ok :write [2 21] INFO [2022-01-10 05:36:17,104] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 05:36:17,191] jepsen worker 19 - jepsen.util 19 :ok :write [6 25] INFO [2022-01-10 05:36:17,191] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 05:36:17,251] jepsen worker 64 - jepsen.util 64 :ok :read [21 7] INFO [2022-01-10 05:36:17,283] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 05:36:17,286] jepsen worker 19 - jepsen.util 19 :ok :read [6 25] INFO [2022-01-10 05:36:17,295] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 05:36:17,389] jepsen worker 19 - jepsen.util 19 :ok :read [6 25] INFO [2022-01-10 05:36:17,473] jepsen worker 64 - jepsen.util 64 :ok :read [21 7] INFO [2022-01-10 05:36:17,489] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 05:36:17,490] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 05:36:17,549] jepsen worker 51 - jepsen.util 51 :ok :write [17 37] INFO [2022-01-10 05:36:17,685] jepsen worker 51 - jepsen.util 51 :invoke :write [17 6] WARN [2022-01-10 05:36:17,687] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][UnexpectedEndpointDisconnectedEvent] The remote side disconnected the endpoint unexpectedly {"bucket":"default","channelId":"EFE9906200000001/000000003D424AA2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43918","remote":"172.23.105.187:11210","type":"KV"} WARN [2022-01-10 05:36:17,687] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][UnexpectedEndpointDisconnectedEvent] The remote side disconnected the endpoint unexpectedly {"bucket":"default","channelId":"EFE9906200000003/00000000E460FFFE","circuitBreaker":"DISABLED","coreId":"0xefe9906200000003","local":"172.23.104.79:43988","remote":"172.23.105.187:11210","type":"KV"} WARN [2022-01-10 05:36:17,688] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][UnexpectedEndpointDisconnectedEvent] The remote side disconnected the endpoint unexpectedly {"bucket":"default","channelId":"EFE9906200000005/000000000FD69664","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43960","remote":"172.23.105.187:11210","type":"KV"} WARN [2022-01-10 05:36:17,691] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][UnexpectedEndpointDisconnectedEvent] The remote side disconnected the endpoint unexpectedly {"bucket":"default","channelId":"EFE9906200000002/000000003C4278A1","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43962","remote":"172.23.105.187:11210","type":"KV"} WARN [2022-01-10 05:36:17,691] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][UnexpectedEndpointDisconnectedEvent] The remote side disconnected the endpoint unexpectedly {"bucket":"default","channelId":"EFE9906200000004/00000000E88FBBF3","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43952","remote":"172.23.105.187:11210","type":"KV"} WARN [2022-01-10 05:36:17,691] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][UnexpectedEndpointDisconnectedEvent] The remote side disconnected the endpoint unexpectedly {"bucket":"default","channelId":"EFE9906200000006/000000005C9D4E4F","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43944","remote":"172.23.105.187:11210","type":"KV"} INFO [2022-01-10 05:36:17,700] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:17,700] jepsen worker nemesis - couchbase.util Rebalance check count: 24 INFO [2022-01-10 05:36:17,700] jepsen worker nemesis - couchbase.util Rebalance stuck count: 10 INFO [2022-01-10 05:36:18,023] jepsen worker 57 - jepsen.util 57 :ok :write [19 15] INFO [2022-01-10 05:36:18,041] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 05:36:18,263] jepsen worker 19 - jepsen.util 19 :ok :read [6 25] INFO [2022-01-10 05:36:18,270] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 05:36:18,321] jepsen worker 30 - jepsen.util 30 :ok :write [10 36] INFO [2022-01-10 05:36:18,379] jepsen worker 64 - jepsen.util 64 :ok :read [21 7] INFO [2022-01-10 05:36:18,388] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 05:36:18,388] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 05:36:18,414] jepsen worker 19 - jepsen.util 19 :ok :read [6 25] INFO [2022-01-10 05:36:18,494] jepsen worker 19 - jepsen.util 19 :invoke :write [6 47] INFO [2022-01-10 05:36:18,549] jepsen worker 83 - jepsen.util 83 :ok :write [27 30] INFO [2022-01-10 05:36:18,583] jepsen worker 64 - jepsen.util 64 :ok :read [21 7] INFO [2022-01-10 05:36:18,600] jepsen worker 80 - jepsen.util 80 :ok :write [26 10] INFO [2022-01-10 05:36:18,621] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 05:36:18,647] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 05:36:18,651] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 05:36:18,724] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:18,724] jepsen worker nemesis - couchbase.util Rebalance check count: 25 INFO [2022-01-10 05:36:18,724] jepsen worker nemesis - couchbase.util Rebalance stuck count: 11 INFO [2022-01-10 05:36:18,791] jepsen worker 30 - jepsen.util 30 :ok :read [10 36] INFO [2022-01-10 05:36:18,795] jepsen worker 83 - jepsen.util 83 :ok :read [27 30] INFO [2022-01-10 05:36:18,858] jepsen worker 64 - jepsen.util 64 :ok :read [21 7] INFO [2022-01-10 05:36:18,899] jepsen worker 83 - jepsen.util 83 :invoke :write [27 19] INFO [2022-01-10 05:36:18,903] jepsen worker 30 - jepsen.util 30 :invoke :write [10 6] INFO [2022-01-10 05:36:18,911] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 05:36:18,944] jepsen worker 80 - jepsen.util 80 :ok :read [26 10] INFO [2022-01-10 05:36:19,015] jepsen worker 56 - jepsen.util 56 :ok :read [18 14] INFO [2022-01-10 05:36:19,015] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 05:36:19,036] jepsen worker 80 - jepsen.util 80 :invoke :write [26 41] INFO [2022-01-10 05:36:19,082] jepsen worker 4 - jepsen.util 4 :ok :write [1 41] INFO [2022-01-10 05:36:19,167] jepsen worker 4 - jepsen.util 4 :invoke :write [1 10] INFO [2022-01-10 05:36:19,276] jepsen worker 64 - jepsen.util 64 :ok :read [21 7] INFO [2022-01-10 05:36:19,322] jepsen worker 35 - jepsen.util 35 :ok :write [11 21] INFO [2022-01-10 05:36:19,400] jepsen worker 35 - jepsen.util 35 :invoke :write [11 0] INFO [2022-01-10 05:36:19,404] jepsen worker 64 - jepsen.util 64 :invoke :write [21 14] WARN [2022-01-10 05:36:19,412] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":4126950,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":4126950,"total_server_duration_us":4106744,"encode_duration_us":212040,"operation_id":"0x586","timeout_ms":10000,"last_server_duration_us":4106744,"total_duration_us":4131025},{"operation_name":"upsert","last_dispatch_duration_us":3754353,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3754353,"total_server_duration_us":3729687,"encode_duration_us":41006,"operation_id":"0x584","timeout_ms":10000,"last_server_duration_us":3729687,"total_duration_us":3760993},{"operation_name":"upsert","last_dispatch_duration_us":3425676,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":3425676,"total_server_duration_us":2556073,"encode_duration_us":227675,"operation_id":"0x5ed","timeout_ms":10000,"last_server_duration_us":2556073,"total_duration_us":3429387},{"operation_name":"get","last_dispatch_duration_us":2719466,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":2719466,"total_server_duration_us":32,"operation_id":"0x61c","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":2728170},{"operation_name":"upsert","last_dispatch_duration_us":1958143,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":1958143,"total_server_duration_us":1943277,"encode_duration_us":166636,"operation_id":"0x5b7","timeout_ms":10000,"last_server_duration_us":1943277,"total_duration_us":1961382},{"operation_name":"get","last_dispatch_duration_us":753058,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":753058,"total_server_duration_us":15,"operation_id":"0x650","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":755940},{"operation_name":"get","last_dispatch_duration_us":538293,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":538293,"total_server_duration_us":69,"operation_id":"0x64f","timeout_ms":30000,"last_server_duration_us":69,"total_duration_us":550485}],"total_count":7}} INFO [2022-01-10 05:36:19,437] jepsen worker 56 - jepsen.util 56 :ok :read [18 14] INFO [2022-01-10 05:36:19,451] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 05:36:19,588] jepsen worker 56 - jepsen.util 56 :ok :read [18 14] INFO [2022-01-10 05:36:19,595] jepsen worker 56 - jepsen.util 56 :invoke :write [18 12] INFO [2022-01-10 05:36:19,747] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 05:36:19,748] jepsen worker nemesis - couchbase.util Rebalance check count: 26 INFO [2022-01-10 05:36:19,748] jepsen worker nemesis - couchbase.util Rebalance stuck count: 12 INFO [2022-01-10 05:36:20,126] jepsen worker 9 - jepsen.util 9 :ok :write [3 22] INFO [2022-01-10 05:36:20,127] jepsen worker 9 - jepsen.util 9 :invoke :write [3 40] INFO [2022-01-10 05:36:20,754] jepsen worker nemesis - couchbase.util Rebalance status: {:status none} INFO [2022-01-10 05:36:20,755] jepsen worker nemesis - couchbase.util Rebalance check count: 27 INFO [2022-01-10 05:36:20,755] jepsen worker nemesis - couchbase.util Rebalance stuck count: 12 INFO [2022-01-10 05:36:20,755] jepsen worker nemesis - couchbase.util Rebalance complete INFO [2022-01-10 05:36:20,766] jepsen worker nemesis - jepsen.util :nemesis :info :rebalance-out ("172.23.105.187") WARN [2022-01-10 05:36:21,010] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":4505369,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":4505369,"total_server_duration_us":4488094,"encode_duration_us":65115,"operation_id":"0x5b4","timeout_ms":10000,"last_server_duration_us":4488094,"total_duration_us":4509398},{"operation_name":"upsert","last_dispatch_duration_us":1070335,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":1070335,"total_server_duration_us":1052164,"encode_duration_us":278490,"operation_id":"0x560","timeout_ms":10000,"last_server_duration_us":1052164,"total_duration_us":1075591}],"total_count":2}} WARN [2022-01-10 05:36:21,014] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":4913446,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":4913446,"total_server_duration_us":4899281,"encode_duration_us":11221,"operation_id":"0x5ee","timeout_ms":10000,"last_server_duration_us":4899281,"total_duration_us":4916994},{"operation_name":"upsert","last_dispatch_duration_us":4302027,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4302027,"total_server_duration_us":4290592,"encode_duration_us":75220,"operation_id":"0x5e6","timeout_ms":10000,"last_server_duration_us":4290592,"total_duration_us":4305884},{"operation_name":"upsert","last_dispatch_duration_us":4103560,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":4103560,"total_server_duration_us":4090063,"encode_duration_us":197182,"operation_id":"0x5c1","timeout_ms":10000,"last_server_duration_us":4090063,"total_duration_us":4108449},{"operation_name":"upsert","last_dispatch_duration_us":2484340,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":2484340,"total_server_duration_us":2425405,"encode_duration_us":286253,"operation_id":"0x561","timeout_ms":10000,"last_server_duration_us":2425405,"total_duration_us":2512867},{"operation_name":"upsert","last_dispatch_duration_us":1408430,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1408430,"total_server_duration_us":1388983,"encode_duration_us":230895,"operation_id":"0x593","timeout_ms":10000,"last_server_duration_us":1388983,"total_duration_us":1413509},{"operation_name":"upsert","last_dispatch_duration_us":622440,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":622440,"total_server_duration_us":598552,"encode_duration_us":164494,"operation_id":"0x55e","timeout_ms":10000,"last_server_duration_us":598552,"total_duration_us":626322}],"total_count":6}} WARN [2022-01-10 05:36:21,020] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":4493417,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":4493417,"total_server_duration_us":4478641,"encode_duration_us":41361,"operation_id":"0x5c4","timeout_ms":10000,"last_server_duration_us":4478641,"total_duration_us":4497693},{"operation_name":"upsert","last_dispatch_duration_us":1753992,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":1753992,"total_server_duration_us":1739393,"encode_duration_us":243361,"operation_id":"0x5b5","timeout_ms":10000,"last_server_duration_us":1739393,"total_duration_us":1758011},{"operation_name":"upsert","last_dispatch_duration_us":1655075,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":1655075,"total_server_duration_us":1520247,"encode_duration_us":41115,"operation_id":"0x582","timeout_ms":10000,"last_server_duration_us":1520247,"total_duration_us":1659095}],"total_count":3}} INFO [2022-01-10 05:36:21,036] jepsen worker 29 - jepsen.util 29 :ok :write [9 22] INFO [2022-01-10 05:36:21,037] jepsen worker 29 - jepsen.util 29 :invoke :write [9 47] WARN [2022-01-10 05:36:21,121] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2805859,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2805859,"total_server_duration_us":2600901,"encode_duration_us":36775,"operation_id":"0x5c0","timeout_ms":10000,"last_server_duration_us":2600901,"total_duration_us":2810789},{"operation_name":"upsert","last_dispatch_duration_us":1932356,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":1932356,"total_server_duration_us":1920699,"encode_duration_us":68575,"operation_id":"0x5c3","timeout_ms":10000,"last_server_duration_us":1920699,"total_duration_us":1936899},{"operation_name":"upsert","last_dispatch_duration_us":1305953,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":1305953,"total_server_duration_us":1219003,"encode_duration_us":433924,"operation_id":"0x58e","timeout_ms":10000,"last_server_duration_us":1219003,"total_duration_us":1314113}],"total_count":3}} WARN [2022-01-10 05:36:21,132] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":4400419,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":4400419,"total_server_duration_us":4248158,"encode_duration_us":277752,"operation_id":"0x594","timeout_ms":10000,"last_server_duration_us":4248158,"total_duration_us":4405314},{"operation_name":"upsert","last_dispatch_duration_us":2924910,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2924910,"total_server_duration_us":2912103,"encode_duration_us":102209,"operation_id":"0x583","timeout_ms":10000,"last_server_duration_us":2912103,"total_duration_us":2929723},{"operation_name":"upsert","last_dispatch_duration_us":1778373,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":1778373,"total_server_duration_us":1531205,"encode_duration_us":95288,"operation_id":"0x5bf","timeout_ms":10000,"last_server_duration_us":1531205,"total_duration_us":1783367},{"operation_name":"upsert","last_dispatch_duration_us":1345829,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":1345829,"total_server_duration_us":1323609,"encode_duration_us":57252,"operation_id":"0x61d","timeout_ms":10000,"last_server_duration_us":1323609,"total_duration_us":1349934},{"operation_name":"upsert","last_dispatch_duration_us":971358,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":971358,"total_server_duration_us":959156,"encode_duration_us":94242,"operation_id":"0x55d","timeout_ms":10000,"last_server_duration_us":959156,"total_duration_us":975693}],"total_count":5}} INFO [2022-01-10 05:36:23,235] jepsen worker 65 - jepsen.util 65 :info :write [21 25] :RequestTimeoutException INFO [2022-01-10 05:36:23,237] jepsen worker 65 - jepsen.util 155 :invoke :read [21 nil] INFO [2022-01-10 05:36:23,619] jepsen worker 42 - jepsen.util 42 :ok :write [14 14] INFO [2022-01-10 05:36:23,619] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 05:36:23,662] jepsen worker 59 - jepsen.util 59 :info :write [19 4] :RequestTimeoutException INFO [2022-01-10 05:36:23,662] jepsen worker 59 - jepsen.util 149 :invoke :write [19 6] INFO [2022-01-10 05:36:23,868] jepsen worker 18 - jepsen.util 18 :info :write [6 22] :RequestTimeoutException INFO [2022-01-10 05:36:23,869] jepsen worker 18 - jepsen.util 108 :invoke :write [6 6] INFO [2022-01-10 05:36:24,173] jepsen worker 16 - jepsen.util 16 :info :write [5 48] :RequestTimeoutException INFO [2022-01-10 05:36:24,175] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 05:36:24,733] jepsen worker 10 - jepsen.util 10 :info :write [3 1] :RequestTimeoutException INFO [2022-01-10 05:36:24,733] jepsen worker 10 - jepsen.util 100 :invoke :write [3 18] INFO [2022-01-10 05:36:24,954] jepsen worker 62 - jepsen.util 62 :info :write [20 46] :RequestTimeoutException INFO [2022-01-10 05:36:24,955] jepsen worker 62 - jepsen.util 152 :invoke :write [20 29] INFO [2022-01-10 05:36:25,476] jepsen worker 5 - jepsen.util 5 :info :write [1 3] :RequestTimeoutException INFO [2022-01-10 05:36:25,477] jepsen worker 5 - jepsen.util 95 :invoke :write [1 17] INFO [2022-01-10 05:36:25,591] jepsen worker 72 - jepsen.util 72 :info :write [24 2] :SyncWriteAmbiguous INFO [2022-01-10 05:36:25,591] jepsen worker 72 - jepsen.util 162 :invoke :write [24 5] INFO [2022-01-10 05:36:25,598] jepsen worker 45 - jepsen.util 45 :info :write [15 0] :SyncWriteAmbiguous INFO [2022-01-10 05:36:25,598] jepsen worker 45 - jepsen.util 135 :invoke :read [15 nil] INFO [2022-01-10 05:36:25,749] jepsen worker 15 - jepsen.util 15 :info :write [5 28] :RequestTimeoutException INFO [2022-01-10 05:36:25,750] jepsen worker 15 - jepsen.util 105 :invoke :write [5 42] INFO [2022-01-10 05:36:25,767] jepsen worker nemesis - jepsen.util :nemesis :info :swap-rebalance nil INFO [2022-01-10 05:36:25,801] jepsen worker 31 - jepsen.util 31 :info :write [10 0] :RequestTimeoutException INFO [2022-01-10 05:36:25,802] jepsen worker 31 - jepsen.util 121 :invoke :read [10 nil] INFO [2022-01-10 05:36:25,856] jepsen worker 82 - jepsen.util 82 :info :write [27 2] :RequestTimeoutException INFO [2022-01-10 05:36:25,857] jepsen worker 82 - jepsen.util 172 :invoke :read [27 nil] INFO [2022-01-10 05:36:25,869] jepsen worker 32 - jepsen.util 32 :info :write [10 39] :RequestTimeoutException INFO [2022-01-10 05:36:25,870] jepsen worker 32 - jepsen.util 122 :invoke :read [10 nil] INFO [2022-01-10 05:36:25,911] jepsen worker 32 - jepsen.util 122 :ok :read [10 36] INFO [2022-01-10 05:36:25,912] jepsen worker 32 - jepsen.util 122 :invoke :read [10 nil] INFO [2022-01-10 05:36:25,953] jepsen worker 32 - jepsen.util 122 :ok :read [10 36] INFO [2022-01-10 05:36:25,953] jepsen worker 32 - jepsen.util 122 :invoke :write [10 24] INFO [2022-01-10 05:36:26,583] jepsen worker 71 - jepsen.util 71 :info :write [23 42] :RequestTimeoutException INFO [2022-01-10 05:36:26,583] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 05:36:26,624] jepsen worker 43 - jepsen.util 43 :info :write [14 32] :RequestTimeoutException INFO [2022-01-10 05:36:26,625] jepsen worker 43 - jepsen.util 133 :invoke :write [14 41] INFO [2022-01-10 05:36:26,716] jepsen worker 14 - jepsen.util 14 :info :write [4 6] :RequestTimeoutException INFO [2022-01-10 05:36:26,717] jepsen worker 14 - jepsen.util 104 :invoke :write [4 21] INFO [2022-01-10 05:36:26,908] jepsen worker 63 - jepsen.util 63 :info :write [21 37] :RequestTimeoutException INFO [2022-01-10 05:36:26,910] jepsen worker 63 - jepsen.util 153 :invoke :write [21 6] INFO [2022-01-10 05:36:26,923] jepsen worker 22 - jepsen.util 22 :info :write [7 36] :RequestTimeoutException INFO [2022-01-10 05:36:26,924] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 05:36:26,988] jepsen worker 55 - jepsen.util 55 :info :write [18 42] :RequestTimeoutException INFO [2022-01-10 05:36:26,989] jepsen worker 55 - jepsen.util 145 :invoke :read [18 nil] INFO [2022-01-10 05:36:27,001] jepsen worker 23 - jepsen.util 23 :info :write [7 47] :RequestTimeoutException INFO [2022-01-10 05:36:27,003] jepsen worker 23 - jepsen.util 113 :invoke :write [7 12] INFO [2022-01-10 05:36:27,013] jepsen worker 55 - jepsen.util 145 :ok :read [18 14] INFO [2022-01-10 05:36:27,014] jepsen worker 55 - jepsen.util 145 :invoke :write [18 40] INFO [2022-01-10 05:36:27,198] jepsen worker 51 - jepsen.util 51 :info :write [17 6] :SyncWriteAmbiguous INFO [2022-01-10 05:36:27,199] jepsen worker 51 - jepsen.util 141 :invoke :write [17 11] INFO [2022-01-10 05:36:27,232] jepsen worker 66 - jepsen.util 66 :info :write [22 33] :SyncWriteAmbiguous INFO [2022-01-10 05:36:27,234] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 05:36:27,437] jepsen worker 31 - jepsen.util 121 :ok :read [10 36] INFO [2022-01-10 05:36:27,438] jepsen worker 31 - jepsen.util 121 :invoke :read [10 nil] INFO [2022-01-10 05:36:27,465] jepsen worker 31 - jepsen.util 121 :ok :read [10 36] INFO [2022-01-10 05:36:27,466] jepsen worker 31 - jepsen.util 121 :invoke :read [10 nil] INFO [2022-01-10 05:36:27,489] jepsen worker 31 - jepsen.util 121 :ok :read [10 36] INFO [2022-01-10 05:36:27,490] jepsen worker 31 - jepsen.util 121 :invoke :write [10 45] INFO [2022-01-10 05:36:27,525] jepsen worker 88 - jepsen.util 88 :info :write [29 6] :SyncWriteAmbiguous INFO [2022-01-10 05:36:27,525] jepsen worker 88 - jepsen.util 178 :invoke :write [29 44] INFO [2022-01-10 05:36:27,742] jepsen worker 81 - jepsen.util 81 :info :write [27 1] :RequestTimeoutException INFO [2022-01-10 05:36:27,743] jepsen worker 81 - jepsen.util 171 :invoke :write [27 49] INFO [2022-01-10 05:36:27,769] jepsen worker 71 - jepsen.util 161 :ok :read [23 :nil] INFO [2022-01-10 05:36:27,769] jepsen worker 71 - jepsen.util 161 :invoke :write [23 27] INFO [2022-01-10 05:36:27,780] jepsen worker 21 - jepsen.util 21 :info :write [7 41] :RequestTimeoutException INFO [2022-01-10 05:36:27,781] jepsen worker 21 - jepsen.util 111 :invoke :write [7 43] INFO [2022-01-10 05:36:27,826] jepsen worker 58 - jepsen.util 58 :info :write [19 12] :RequestTimeoutException INFO [2022-01-10 05:36:27,828] jepsen worker 58 - jepsen.util 148 :invoke :read [19 nil] INFO [2022-01-10 05:36:28,053] jepsen worker 47 - jepsen.util 47 :info :write [15 19] :RequestTimeoutException INFO [2022-01-10 05:36:28,054] jepsen worker 47 - jepsen.util 137 :invoke :read [15 nil] INFO [2022-01-10 05:36:28,103] jepsen worker 50 - jepsen.util 50 :info :write [16 30] :RequestTimeoutException INFO [2022-01-10 05:36:28,104] jepsen worker 50 - jepsen.util 140 :invoke :write [16 3] INFO [2022-01-10 05:36:28,158] jepsen worker 85 - jepsen.util 85 :info :write [28 31] :SyncWriteAmbiguous INFO [2022-01-10 05:36:28,158] jepsen worker 85 - jepsen.util 175 :invoke :write [28 19] INFO [2022-01-10 05:36:28,381] jepsen worker 8 - jepsen.util 8 :info :write [2 12] :RequestTimeoutException INFO [2022-01-10 05:36:28,382] jepsen worker 8 - jepsen.util 98 :invoke :write [2 14] INFO [2022-01-10 05:36:28,399] jepsen worker 67 - jepsen.util 67 :info :write [22 29] :RequestTimeoutException INFO [2022-01-10 05:36:28,400] jepsen worker 67 - jepsen.util 157 :invoke :read [22 nil] INFO [2022-01-10 05:36:28,463] jepsen worker 80 - jepsen.util 80 :info :write [26 41] :SyncWriteAmbiguous INFO [2022-01-10 05:36:28,464] jepsen worker 80 - jepsen.util 170 :invoke :write [26 26] INFO [2022-01-10 05:36:28,532] jepsen worker 57 - jepsen.util 57 :ok :read [19 15] INFO [2022-01-10 05:36:28,533] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 05:36:28,686] jepsen worker 89 - jepsen.util 89 :info :write [29 14] :RequestTimeoutException INFO [2022-01-10 05:36:28,687] jepsen worker 89 - jepsen.util 179 :invoke :write [29 16] INFO [2022-01-10 05:36:28,733] jepsen worker 56 - jepsen.util 56 :info :write [18 12] :SyncWriteAmbiguous INFO [2022-01-10 05:36:28,739] jepsen worker 56 - jepsen.util 146 :invoke :write [18 11] INFO [2022-01-10 05:36:28,776] jepsen worker 42 - jepsen.util 42 :ok :read [14 14] INFO [2022-01-10 05:36:28,777] jepsen worker 42 - jepsen.util 42 :invoke :write [14 27] INFO [2022-01-10 05:36:28,800] jepsen worker 65 - jepsen.util 155 :ok :read [21 7] INFO [2022-01-10 05:36:28,801] jepsen worker 65 - jepsen.util 155 :invoke :write [21 20] INFO [2022-01-10 05:36:28,827] jepsen worker 73 - jepsen.util 73 :info :write [24 13] :RequestTimeoutException INFO [2022-01-10 05:36:28,828] jepsen worker 73 - jepsen.util 163 :invoke :read [24 nil] INFO [2022-01-10 05:36:28,843] jepsen worker 44 - jepsen.util 44 :info :write [14 41] :RequestTimeoutException INFO [2022-01-10 05:36:28,843] jepsen worker 44 - jepsen.util 134 :invoke :write [14 31] INFO [2022-01-10 05:36:28,936] jepsen worker 19 - jepsen.util 19 :info :write [6 47] :RequestTimeoutException INFO [2022-01-10 05:36:28,937] jepsen worker 19 - jepsen.util 109 :invoke :read [6 nil] INFO [2022-01-10 05:36:28,942] jepsen worker 30 - jepsen.util 30 :info :write [10 6] :RequestTimeoutException INFO [2022-01-10 05:36:28,943] jepsen worker 30 - jepsen.util 120 :invoke :write [10 1] INFO [2022-01-10 05:36:29,029] jepsen worker 83 - jepsen.util 83 :info :write [27 19] :RequestTimeoutException INFO [2022-01-10 05:36:29,029] jepsen worker 83 - jepsen.util 173 :invoke :write [27 37] INFO [2022-01-10 05:36:29,078] jepsen worker 38 - jepsen.util 38 :info :write [12 11] :SyncWriteAmbiguous INFO [2022-01-10 05:36:29,079] jepsen worker 38 - jepsen.util 128 :invoke :read [12 nil] INFO [2022-01-10 05:36:29,082] jepsen worker 19 - jepsen.util 109 :ok :read [6 25] INFO [2022-01-10 05:36:29,082] jepsen worker 19 - jepsen.util 109 :invoke :write [6 17] INFO [2022-01-10 05:36:29,108] jepsen worker 38 - jepsen.util 128 :ok :read [12 :nil] INFO [2022-01-10 05:36:29,109] jepsen worker 38 - jepsen.util 128 :invoke :read [12 nil] INFO [2022-01-10 05:36:29,170] jepsen worker 27 - jepsen.util 27 :info :write [9 26] :RequestTimeoutException INFO [2022-01-10 05:36:29,171] jepsen worker 27 - jepsen.util 117 :invoke :read [9 nil] INFO [2022-01-10 05:36:29,208] jepsen worker 4 - jepsen.util 4 :info :write [1 10] :RequestTimeoutException INFO [2022-01-10 05:36:29,210] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 05:36:29,223] jepsen worker 38 - jepsen.util 128 :ok :read [12 :nil] INFO [2022-01-10 05:36:29,228] jepsen worker 38 - jepsen.util 128 :invoke :write [12 49] INFO [2022-01-10 05:36:29,243] jepsen worker 52 - jepsen.util 52 :info :write [17 27] :RequestTimeoutException INFO [2022-01-10 05:36:29,244] jepsen worker 11 - jepsen.util 11 :info :write [3 27] :SyncWriteAmbiguous INFO [2022-01-10 05:36:29,246] jepsen worker 11 - jepsen.util 101 :invoke :read [3 nil] INFO [2022-01-10 05:36:29,248] jepsen worker 52 - jepsen.util 142 :invoke :write [17 30] INFO [2022-01-10 05:36:29,358] jepsen worker 3 - jepsen.util 3 :info :write [1 40] :RequestTimeoutException INFO [2022-01-10 05:36:29,359] jepsen worker 3 - jepsen.util 93 :invoke :write [1 4] INFO [2022-01-10 05:36:29,388] jepsen worker 4 - jepsen.util 94 :ok :read [1 41] INFO [2022-01-10 05:36:29,389] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] WARN [2022-01-10 05:36:29,430] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10955755,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10955755,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9033121,"encode_duration_us":147883,"operation_id":"0x627","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10001417},{"operation_name":"upsert","last_dispatch_duration_us":9197710,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10176215,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9574505,"encode_duration_us":31422,"operation_id":"0x612","timeout_ms":10000,"last_server_duration_us":9122444,"total_duration_us":10006477},{"operation_name":"upsert","last_dispatch_duration_us":9094801,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11566875,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9057226,"encode_duration_us":15454,"operation_id":"0x5ef","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10004492}],"total_count":3}} WARN [2022-01-10 05:36:29,431] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":68518,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2113935,"total_server_duration_us":192957,"encode_duration_us":312128,"operation_id":"0x6df","timeout_ms":10000,"last_server_duration_us":36627,"total_duration_us":10007822},{"operation_name":"upsert","last_dispatch_duration_us":406168,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":1324022,"total_server_duration_us":474153,"encode_duration_us":77107,"operation_id":"0x689","timeout_ms":10000,"last_server_duration_us":107430,"total_duration_us":10007805},{"operation_name":"upsert","last_dispatch_duration_us":9197710,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10176215,"total_server_duration_us":9574505,"encode_duration_us":31422,"operation_id":"0x612","timeout_ms":10000,"last_server_duration_us":9122444,"total_duration_us":10006477},{"operation_name":"upsert","last_dispatch_duration_us":9094801,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":11566875,"total_server_duration_us":9057226,"encode_duration_us":15454,"operation_id":"0x5ef","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10004492},{"operation_name":"upsert","last_dispatch_duration_us":7693369,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9053294,"total_server_duration_us":504900,"encode_duration_us":37106,"operation_id":"0x6c2","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10003853},{"operation_name":"upsert","last_dispatch_duration_us":10955755,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10955755,"total_server_duration_us":9033121,"encode_duration_us":147883,"operation_id":"0x627","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10001417},{"operation_name":"upsert","last_dispatch_duration_us":9287970,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9287970,"total_server_duration_us":9052231,"encode_duration_us":81644,"operation_id":"0x6e9","timeout_ms":10000,"last_server_duration_us":9052231,"total_duration_us":9342781},{"operation_name":"upsert","last_dispatch_duration_us":9078262,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9078262,"total_server_duration_us":9050106,"encode_duration_us":138916,"operation_id":"0x734","timeout_ms":10000,"last_server_duration_us":9050106,"total_duration_us":9083448},{"operation_name":"upsert","last_dispatch_duration_us":9062232,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9062232,"total_server_duration_us":9034183,"encode_duration_us":63894,"operation_id":"0x705","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":9066894}],"total_count":9}} INFO [2022-01-10 05:36:29,447] jepsen worker 64 - jepsen.util 64 :info :write [21 14] :RequestTimeoutException INFO [2022-01-10 05:36:29,448] jepsen worker 64 - jepsen.util 154 :invoke :write [21 49] INFO [2022-01-10 05:36:29,459] jepsen worker 35 - jepsen.util 35 :info :write [11 0] :RequestTimeoutException INFO [2022-01-10 05:36:29,462] jepsen worker 35 - jepsen.util 125 :invoke :write [11 15] INFO [2022-01-10 05:36:29,479] jepsen worker 4 - jepsen.util 94 :ok :read [1 41] INFO [2022-01-10 05:36:29,480] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 05:36:29,484] jepsen worker 39 - jepsen.util 39 :ok :write [13 32] INFO [2022-01-10 05:36:29,485] jepsen worker 39 - jepsen.util 39 :invoke :write [13 43] INFO [2022-01-10 05:36:29,544] jepsen worker 4 - jepsen.util 94 :ok :read [1 41] INFO [2022-01-10 05:36:29,544] jepsen worker 4 - jepsen.util 94 :invoke :write [1 27] INFO [2022-01-10 05:36:29,564] jepsen worker 69 - jepsen.util 69 :info :write [23 31] :RequestTimeoutException INFO [2022-01-10 05:36:29,567] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 05:36:29,581] jepsen worker 77 - jepsen.util 77 :info :write [25 5] :RequestTimeoutException INFO [2022-01-10 05:36:29,583] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 05:36:29,675] jepsen worker 77 - jepsen.util 167 :ok :read [25 :nil] INFO [2022-01-10 05:36:29,676] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 05:36:29,723] jepsen worker 69 - jepsen.util 159 :ok :read [23 :nil] INFO [2022-01-10 05:36:29,723] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 05:36:29,771] jepsen worker 77 - jepsen.util 167 :ok :read [25 :nil] INFO [2022-01-10 05:36:29,772] jepsen worker 77 - jepsen.util 167 :invoke :write [25 8] INFO [2022-01-10 05:36:29,780] jepsen worker 74 - jepsen.util 74 :info :write [24 35] :RequestTimeoutException INFO [2022-01-10 05:36:29,786] jepsen worker 74 - jepsen.util 164 :invoke :write [24 5] INFO [2022-01-10 05:36:29,798] jepsen worker 69 - jepsen.util 159 :ok :read [23 :nil] INFO [2022-01-10 05:36:29,801] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 05:36:29,860] jepsen worker 20 - jepsen.util 20 :info :write [6 34] :RequestTimeoutException INFO [2022-01-10 05:36:29,862] jepsen worker 20 - jepsen.util 110 :invoke :read [6 nil] INFO [2022-01-10 05:36:29,865] jepsen worker 69 - jepsen.util 159 :ok :read [23 :nil] INFO [2022-01-10 05:36:29,866] jepsen worker 69 - jepsen.util 159 :invoke :write [23 34] INFO [2022-01-10 05:36:29,877] jepsen worker 60 - jepsen.util 60 :info :write [20 2] :RequestTimeoutException INFO [2022-01-10 05:36:29,877] jepsen worker 60 - jepsen.util 150 :invoke :write [20 10] INFO [2022-01-10 05:36:29,890] jepsen worker 54 - jepsen.util 54 :info :write [18 14] :RequestTimeoutException INFO [2022-01-10 05:36:29,891] jepsen worker 54 - jepsen.util 144 :invoke :read [18 nil] INFO [2022-01-10 05:36:29,892] jepsen worker 46 - jepsen.util 46 :info :write [15 6] :RequestTimeoutException INFO [2022-01-10 05:36:29,893] jepsen worker 46 - jepsen.util 136 :invoke :read [15 nil] INFO [2022-01-10 05:36:29,912] jepsen worker 33 - jepsen.util 33 :info :write [11 13] :RequestTimeoutException INFO [2022-01-10 05:36:29,913] jepsen worker 33 - jepsen.util 123 :invoke :write [11 5] INFO [2022-01-10 05:36:29,946] jepsen worker 68 - jepsen.util 68 :info :write [22 23] :RequestTimeoutException INFO [2022-01-10 05:36:29,946] jepsen worker 17 - jepsen.util 17 :info :write [5 4] :RequestTimeoutException INFO [2022-01-10 05:36:29,947] jepsen worker 68 - jepsen.util 158 :invoke :write [22 34] INFO [2022-01-10 05:36:29,947] jepsen worker 17 - jepsen.util 107 :invoke :read [5 nil] INFO [2022-01-10 05:36:29,972] jepsen worker 0 - jepsen.util 0 :info :write [0 20] :RequestTimeoutException INFO [2022-01-10 05:36:29,972] jepsen worker 25 - jepsen.util 25 :info :write [8 9] :RequestTimeoutException INFO [2022-01-10 05:36:29,973] jepsen worker 0 - jepsen.util 90 :invoke :write [0 34] INFO [2022-01-10 05:36:29,977] jepsen worker 25 - jepsen.util 115 :invoke :read [8 nil] INFO [2022-01-10 05:36:30,002] jepsen worker 24 - jepsen.util 24 :info :write [8 49] :RequestTimeoutException INFO [2022-01-10 05:36:30,003] jepsen worker 24 - jepsen.util 114 :invoke :read [8 nil] INFO [2022-01-10 05:36:30,037] jepsen worker 86 - jepsen.util 86 :info :write [28 21] :RequestTimeoutException INFO [2022-01-10 05:36:30,038] jepsen worker 86 - jepsen.util 176 :invoke :write [28 30] INFO [2022-01-10 05:36:30,042] jepsen worker 76 - jepsen.util 76 :info :write [25 32] :RequestTimeoutException INFO [2022-01-10 05:36:30,056] jepsen worker 76 - jepsen.util 166 :invoke :write [25 43] INFO [2022-01-10 05:36:30,081] jepsen worker 13 - jepsen.util 13 :info :write [4 18] :RequestTimeoutException INFO [2022-01-10 05:36:30,084] jepsen worker 13 - jepsen.util 103 :invoke :read [4 nil] INFO [2022-01-10 05:36:30,099] jepsen worker 84 - jepsen.util 84 :info :write [28 41] :RequestTimeoutException INFO [2022-01-10 05:36:30,100] jepsen worker 84 - jepsen.util 174 :invoke :write [28 25] INFO [2022-01-10 05:36:30,107] jepsen worker 6 - jepsen.util 6 :info :write [2 31] :RequestTimeoutException INFO [2022-01-10 05:36:30,108] jepsen worker 6 - jepsen.util 96 :invoke :read [2 nil] INFO [2022-01-10 05:36:30,112] jepsen worker 12 - jepsen.util 12 :info :write [4 6] :RequestTimeoutException INFO [2022-01-10 05:36:30,113] jepsen worker 12 - jepsen.util 102 :invoke :read [4 nil] INFO [2022-01-10 05:36:30,122] jepsen worker 87 - jepsen.util 87 :info :write [29 17] :RequestTimeoutException INFO [2022-01-10 05:36:30,123] jepsen worker 87 - jepsen.util 177 :invoke :read [29 nil] INFO [2022-01-10 05:36:30,150] jepsen worker 36 - jepsen.util 36 :info :write [12 30] :RequestTimeoutException INFO [2022-01-10 05:36:30,150] jepsen worker 36 - jepsen.util 126 :invoke :read [12 nil] INFO [2022-01-10 05:36:30,169] jepsen worker 13 - jepsen.util 103 :ok :read [4 :nil] INFO [2022-01-10 05:36:30,170] jepsen worker 13 - jepsen.util 103 :invoke :write [4 40] INFO [2022-01-10 05:36:30,180] jepsen worker 79 - jepsen.util 79 :info :write [26 45] :RequestTimeoutException INFO [2022-01-10 05:36:30,181] jepsen worker 79 - jepsen.util 169 :invoke :read [26 nil] INFO [2022-01-10 05:36:30,183] jepsen worker 12 - jepsen.util 102 :ok :read [4 :nil] INFO [2022-01-10 05:36:30,184] jepsen worker 12 - jepsen.util 102 :invoke :write [4 5] INFO [2022-01-10 05:36:30,187] jepsen worker 9 - jepsen.util 9 :info :write [3 40] :RequestTimeoutException INFO [2022-01-10 05:36:30,192] jepsen worker 9 - jepsen.util 99 :invoke :read [3 nil] INFO [2022-01-10 05:36:30,212] jepsen worker 34 - jepsen.util 34 :info :write [11 30] :RequestTimeoutException INFO [2022-01-10 05:36:30,213] jepsen worker 34 - jepsen.util 124 :invoke :write [11 11] INFO [2022-01-10 05:36:30,221] jepsen worker 9 - jepsen.util 99 :ok :read [3 22] INFO [2022-01-10 05:36:30,221] jepsen worker 9 - jepsen.util 99 :invoke :read [3 nil] INFO [2022-01-10 05:36:30,255] jepsen worker 9 - jepsen.util 99 :ok :read [3 22] INFO [2022-01-10 05:36:30,256] jepsen worker 9 - jepsen.util 99 :invoke :write [3 3] INFO [2022-01-10 05:36:30,280] jepsen worker 70 - jepsen.util 70 :info :write [23 10] :RequestTimeoutException INFO [2022-01-10 05:36:30,280] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 05:36:30,284] jepsen worker 79 - jepsen.util 169 :ok :read [26 10] INFO [2022-01-10 05:36:30,285] jepsen worker 79 - jepsen.util 169 :invoke :write [26 15] INFO [2022-01-10 05:36:30,285] jepsen worker 75 - jepsen.util 75 :info :write [25 26] :RequestTimeoutException INFO [2022-01-10 05:36:30,286] jepsen worker 75 - jepsen.util 165 :invoke :write [25 41] INFO [2022-01-10 05:36:30,301] jepsen worker 48 - jepsen.util 48 :info :write [16 48] :RequestTimeoutException INFO [2022-01-10 05:36:30,302] jepsen worker 48 - jepsen.util 138 :invoke :read [16 nil] INFO [2022-01-10 05:36:30,305] jepsen worker 28 - jepsen.util 28 :info :write [9 19] :RequestTimeoutException INFO [2022-01-10 05:36:30,306] jepsen worker 28 - jepsen.util 118 :invoke :read [9 nil] INFO [2022-01-10 05:36:30,311] jepsen worker 49 - jepsen.util 49 :info :write [16 2] :RequestTimeoutException INFO [2022-01-10 05:36:30,312] jepsen worker 49 - jepsen.util 139 :invoke :write [16 23] INFO [2022-01-10 05:36:30,346] jepsen worker 26 - jepsen.util 26 :info :write [8 44] :RequestTimeoutException INFO [2022-01-10 05:36:30,347] jepsen worker 26 - jepsen.util 116 :invoke :write [8 2] INFO [2022-01-10 05:36:30,361] jepsen worker 70 - jepsen.util 160 :ok :read [23 :nil] INFO [2022-01-10 05:36:30,362] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 05:36:30,381] jepsen worker 61 - jepsen.util 61 :info :write [20 18] :RequestTimeoutException INFO [2022-01-10 05:36:30,382] jepsen worker 61 - jepsen.util 151 :invoke :write [20 39] INFO [2022-01-10 05:36:30,401] jepsen worker 41 - jepsen.util 41 :info :write [13 39] :RequestTimeoutException INFO [2022-01-10 05:36:30,401] jepsen worker 41 - jepsen.util 131 :invoke :read [13 nil] INFO [2022-01-10 05:36:30,416] jepsen worker 2 - jepsen.util 2 :info :write [0 10] :RequestTimeoutException INFO [2022-01-10 05:36:30,417] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 05:36:30,442] jepsen worker 70 - jepsen.util 160 :ok :read [23 :nil] INFO [2022-01-10 05:36:30,443] jepsen worker 70 - jepsen.util 160 :invoke :write [23 27] INFO [2022-01-10 05:36:30,455] jepsen worker 37 - jepsen.util 37 :info :write [12 47] :RequestTimeoutException INFO [2022-01-10 05:36:30,457] jepsen worker 37 - jepsen.util 127 :invoke :write [12 27] INFO [2022-01-10 05:36:30,476] jepsen worker 40 - jepsen.util 40 :info :write [13 7] :RequestTimeoutException INFO [2022-01-10 05:36:30,476] jepsen worker 78 - jepsen.util 78 :info :write [26 26] :RequestTimeoutException INFO [2022-01-10 05:36:30,477] jepsen worker 1 - jepsen.util 1 :info :write [0 31] :RequestTimeoutException INFO [2022-01-10 05:36:30,477] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 05:36:30,478] jepsen worker 40 - jepsen.util 130 :invoke :write [13 8] INFO [2022-01-10 05:36:30,479] jepsen worker 1 - jepsen.util 91 :invoke :write [0 48] INFO [2022-01-10 05:36:30,506] jepsen worker 53 - jepsen.util 53 :info :write [17 25] :RequestTimeoutException INFO [2022-01-10 05:36:30,508] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 05:36:30,566] jepsen worker 7 - jepsen.util 7 :ok :read [2 21] INFO [2022-01-10 05:36:30,567] jepsen worker 7 - jepsen.util 7 :invoke :write [2 29] INFO [2022-01-10 05:36:30,698] jepsen worker 78 - jepsen.util 168 :ok :read [26 10] INFO [2022-01-10 05:36:30,699] jepsen worker 78 - jepsen.util 168 :invoke :write [26 11] WARN [2022-01-10 05:36:30,952] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":13334546,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13402188,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9236114,"encode_duration_us":149062,"operation_id":"0x595","timeout_ms":10000,"last_server_duration_us":9190751,"total_duration_us":10007611}],"total_count":1}} WARN [2022-01-10 05:36:31,026] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10846724,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10846724,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9046921,"encode_duration_us":244125,"operation_id":"0x61e","timeout_ms":10000,"last_server_duration_us":9046921,"total_duration_us":10002536},{"operation_name":"upsert","last_dispatch_duration_us":9056083,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9668150,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9297308,"encode_duration_us":392476,"operation_id":"0x5be","timeout_ms":10000,"last_server_duration_us":9042674,"total_duration_us":10007067}],"total_count":2}} WARN [2022-01-10 05:36:31,026] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":62936,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":8772376,"total_server_duration_us":62139,"encode_duration_us":29833,"operation_id":"0x6e0","timeout_ms":10000,"last_server_duration_us":21649,"total_duration_us":10008068},{"operation_name":"upsert","last_dispatch_duration_us":9056083,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9668150,"total_server_duration_us":9297308,"encode_duration_us":392476,"operation_id":"0x5be","timeout_ms":10000,"last_server_duration_us":9042674,"total_duration_us":10007067},{"operation_name":"upsert","encode_duration_us":18798,"operation_id":"0x747","timeout_ms":10000,"total_duration_us":10006752},{"operation_name":"upsert","last_dispatch_duration_us":8675393,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":12477058,"total_server_duration_us":162134,"encode_duration_us":127295,"operation_id":"0x5d1","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10005704},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":156532,"operation_id":"0x730","timeout_ms":10000,"total_duration_us":10005383},{"operation_name":"upsert","last_dispatch_duration_us":8921613,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9000928,"total_server_duration_us":126265,"encode_duration_us":160012,"operation_id":"0x675","timeout_ms":10000,"last_server_duration_us":85303,"total_duration_us":10005317},{"operation_name":"upsert","last_dispatch_duration_us":10325732,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10325732,"total_server_duration_us":23813,"encode_duration_us":149408,"operation_id":"0x699","timeout_ms":10000,"last_server_duration_us":23813,"total_duration_us":10004723},{"operation_name":"upsert","encode_duration_us":151923,"operation_id":"0x731","timeout_ms":10000,"total_duration_us":10003981},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":75063,"operation_id":"0x751","timeout_ms":10000,"total_duration_us":10003716},{"operation_name":"upsert","last_dispatch_duration_us":4293469,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4293469,"total_server_duration_us":41983,"encode_duration_us":129588,"operation_id":"0x6d9","timeout_ms":10000,"last_server_duration_us":41983,"total_duration_us":10002894}],"total_count":16}} WARN [2022-01-10 05:36:31,035] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12332491,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12332491,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9025695,"encode_duration_us":160531,"operation_id":"0x60a","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10002664},{"operation_name":"upsert","last_dispatch_duration_us":9153024,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9924166,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9482009,"encode_duration_us":114457,"operation_id":"0x58f","timeout_ms":10000,"last_server_duration_us":9129905,"total_duration_us":10017094},{"operation_name":"upsert","last_dispatch_duration_us":9835819,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10020790,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9608950,"encode_duration_us":21156,"operation_id":"0x752","timeout_ms":10000,"last_server_duration_us":9587707,"total_duration_us":10010205}],"total_count":3}} WARN [2022-01-10 05:36:31,035] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9153024,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9924166,"total_server_duration_us":9482009,"encode_duration_us":114457,"operation_id":"0x58f","timeout_ms":10000,"last_server_duration_us":9129905,"total_duration_us":10017094},{"operation_name":"upsert","last_dispatch_duration_us":51047,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1385777,"total_server_duration_us":328630,"encode_duration_us":155372,"operation_id":"0x70f","timeout_ms":10000,"last_server_duration_us":39109,"total_duration_us":10011002},{"operation_name":"upsert","last_dispatch_duration_us":9835819,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10020790,"total_server_duration_us":9608950,"encode_duration_us":21156,"operation_id":"0x752","timeout_ms":10000,"last_server_duration_us":9587707,"total_duration_us":10010205},{"operation_name":"upsert","last_dispatch_duration_us":177727,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8765367,"total_server_duration_us":70742,"encode_duration_us":37378,"operation_id":"0x6f9","timeout_ms":10000,"last_server_duration_us":21001,"total_duration_us":10008791},{"operation_name":"upsert","last_dispatch_duration_us":69720,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1366237,"total_server_duration_us":444687,"encode_duration_us":94093,"operation_id":"0x73e","timeout_ms":10000,"last_server_duration_us":59374,"total_duration_us":10008754},{"operation_name":"upsert","last_dispatch_duration_us":154869,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":2332066,"total_server_duration_us":658424,"encode_duration_us":112439,"operation_id":"0x5fd","timeout_ms":10000,"last_server_duration_us":29788,"total_duration_us":10007680},{"operation_name":"upsert","last_dispatch_duration_us":196190,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":3059721,"total_server_duration_us":607558,"encode_duration_us":17318,"operation_id":"0x75e","timeout_ms":10000,"last_server_duration_us":30918,"total_duration_us":10007143},{"operation_name":"upsert","last_dispatch_duration_us":104113,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1223181,"total_server_duration_us":330556,"encode_duration_us":135542,"operation_id":"0x729","timeout_ms":10000,"last_server_duration_us":75478,"total_duration_us":10006264},{"operation_name":"upsert","last_dispatch_duration_us":75721,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1555943,"total_server_duration_us":520774,"encode_duration_us":153863,"operation_id":"0x640","timeout_ms":10000,"last_server_duration_us":56524,"total_duration_us":10005248},{"operation_name":"upsert","last_dispatch_duration_us":860089,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":3381351,"total_server_duration_us":458112,"encode_duration_us":154126,"operation_id":"0x72e","timeout_ms":10000,"last_server_duration_us":52183,"total_duration_us":10004850}],"total_count":15}} WARN [2022-01-10 05:36:31,042] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":995912,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":3407579,"total_server_duration_us":867645,"encode_duration_us":118160,"operation_id":"0x6a8","timeout_ms":10000,"last_server_duration_us":657963,"total_duration_us":10011466},{"operation_name":"upsert","last_dispatch_duration_us":39136,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":3928730,"total_server_duration_us":270803,"encode_duration_us":182807,"operation_id":"0x642","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10010390},{"operation_name":"upsert","last_dispatch_duration_us":1017979,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":3460187,"total_server_duration_us":341161,"encode_duration_us":104466,"operation_id":"0x738","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10009907},{"operation_name":"upsert","last_dispatch_duration_us":40758,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":527233,"total_server_duration_us":280705,"encode_duration_us":26848,"operation_id":"0x75d","timeout_ms":10000,"last_server_duration_us":23898,"total_duration_us":10008757},{"operation_name":"upsert","last_dispatch_duration_us":10923794,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10923794,"total_server_duration_us":26064,"encode_duration_us":284843,"operation_id":"0x67c","timeout_ms":10000,"last_server_duration_us":26064,"total_duration_us":10008458},{"operation_name":"upsert","last_dispatch_duration_us":961720,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":3839508,"total_server_duration_us":192208,"encode_duration_us":34946,"operation_id":"0x75f","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10007863},{"operation_name":"upsert","last_dispatch_duration_us":30021,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":1210813,"total_server_duration_us":411644,"encode_duration_us":24814,"operation_id":"0x762","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10007363},{"operation_name":"upsert","last_dispatch_duration_us":68181,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":5716848,"total_server_duration_us":460514,"encode_duration_us":134721,"operation_id":"0x613","timeout_ms":10000,"last_server_duration_us":30445,"total_duration_us":10006751},{"operation_name":"upsert","last_dispatch_duration_us":49247,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":1538839,"total_server_duration_us":561013,"encode_duration_us":52684,"operation_id":"0x746","timeout_ms":10000,"last_server_duration_us":29137,"total_duration_us":10006068},{"operation_name":"upsert","last_dispatch_duration_us":1322840,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":6291077,"total_server_duration_us":284269,"encode_duration_us":300106,"operation_id":"0x6ed","timeout_ms":10000,"last_server_duration_us":27764,"total_duration_us":10005971}],"total_count":17}} INFO [2022-01-10 05:36:31,057] jepsen worker 29 - jepsen.util 29 :info :write [9 47] :RequestTimeoutException INFO [2022-01-10 05:36:31,058] jepsen worker 29 - jepsen.util 119 :invoke :write [9 6] INFO [2022-01-10 05:36:31,114] jepsen worker 22 - jepsen.util 112 :ok :read [7 :nil] INFO [2022-01-10 05:36:31,114] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] WARN [2022-01-10 05:36:31,136] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13437876,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13437876,"total_server_duration_us":62,"operation_id":"0x645","timeout_ms":30000,"last_server_duration_us":62,"total_duration_us":13447281},{"operation_name":"upsert","last_dispatch_duration_us":12903743,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12903743,"total_server_duration_us":95482,"encode_duration_us":87881,"operation_id":"0x61f","timeout_ms":10000,"last_server_duration_us":95482,"total_duration_us":10010147},{"operation_name":"upsert","last_dispatch_duration_us":8417601,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8978911,"total_server_duration_us":352388,"encode_duration_us":83896,"operation_id":"0x73d","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10009963},{"operation_name":"upsert","last_dispatch_duration_us":11279158,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11279158,"total_server_duration_us":9110726,"encode_duration_us":43898,"operation_id":"0x5e4","timeout_ms":10000,"last_server_duration_us":9110726,"total_duration_us":10009671},{"operation_name":"upsert","last_dispatch_duration_us":69965,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9604937,"total_server_duration_us":385936,"encode_duration_us":239685,"operation_id":"0x701","timeout_ms":10000,"last_server_duration_us":29508,"total_duration_us":10005634},{"operation_name":"upsert","encode_duration_us":65866,"operation_id":"0x735","timeout_ms":10000,"total_duration_us":10005486},{"operation_name":"upsert","last_dispatch_duration_us":75870,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9684978,"total_server_duration_us":398725,"encode_duration_us":51556,"operation_id":"0x73f","timeout_ms":10000,"last_server_duration_us":21977,"total_duration_us":10005071},{"operation_name":"upsert","encode_duration_us":27738,"operation_id":"0x739","timeout_ms":10000,"total_duration_us":10004938},{"operation_name":"upsert","last_dispatch_duration_us":67592,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9388054,"total_server_duration_us":267418,"encode_duration_us":100456,"operation_id":"0x740","timeout_ms":10000,"last_server_duration_us":22223,"total_duration_us":10003831},{"operation_name":"upsert","encode_duration_us":79572,"operation_id":"0x72c","timeout_ms":10000,"total_duration_us":10003488}],"total_count":15}} WARN [2022-01-10 05:36:31,153] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10379466,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10379466,"total_server_duration_us":32,"operation_id":"0x693","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":10475448},{"operation_name":"upsert","last_dispatch_duration_us":79585,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":394147,"total_server_duration_us":61924,"encode_duration_us":24115,"operation_id":"0x755","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10012312},{"operation_name":"upsert","last_dispatch_duration_us":55002,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":6617664,"total_server_duration_us":301479,"encode_duration_us":110472,"operation_id":"0x745","timeout_ms":10000,"last_server_duration_us":42417,"total_duration_us":10009774},{"operation_name":"upsert","last_dispatch_duration_us":129517,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8592299,"total_server_duration_us":255274,"encode_duration_us":228491,"operation_id":"0x6a7","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10009227},{"operation_name":"upsert","last_dispatch_duration_us":31621,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7494812,"total_server_duration_us":336290,"encode_duration_us":169918,"operation_id":"0x704","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10008735},{"operation_name":"upsert","last_dispatch_duration_us":13334546,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13402188,"total_server_duration_us":9236114,"encode_duration_us":149062,"operation_id":"0x595","timeout_ms":10000,"last_server_duration_us":9190751,"total_duration_us":10007611},{"operation_name":"upsert","last_dispatch_duration_us":98965,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9571984,"total_server_duration_us":161240,"encode_duration_us":137394,"operation_id":"0x643","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10007374},{"operation_name":"upsert","last_dispatch_duration_us":937344,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8234226,"total_server_duration_us":1152459,"encode_duration_us":138713,"operation_id":"0x728","timeout_ms":10000,"last_server_duration_us":922258,"total_duration_us":10007312},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":48921,"operation_id":"0x74f","timeout_ms":10000,"total_duration_us":10005407},{"operation_name":"upsert","last_dispatch_duration_us":55483,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":6137438,"total_server_duration_us":236261,"encode_duration_us":57643,"operation_id":"0x74d","timeout_ms":10000,"last_server_duration_us":22971,"total_duration_us":10004924}],"total_count":16}} INFO [2022-01-10 05:36:31,201] jepsen worker 82 - jepsen.util 172 :ok :read [27 30] INFO [2022-01-10 05:36:31,201] jepsen worker 82 - jepsen.util 172 :invoke :read [27 nil] WARN [2022-01-10 05:36:31,236] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11279158,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11279158,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9110726,"encode_duration_us":43898,"operation_id":"0x5e4","timeout_ms":10000,"last_server_duration_us":9110726,"total_duration_us":10009671}],"total_count":1}} WARN [2022-01-10 05:36:31,243] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12458469,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13164266,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9433878,"encode_duration_us":216917,"operation_id":"0x5b6","timeout_ms":10000,"last_server_duration_us":9218562,"total_duration_us":10004501}],"total_count":1}} INFO [2022-01-10 05:36:31,722] jepsen worker nemesis - couchbase.util Adding node 172.23.105.187 to cluster INFO [2022-01-10 05:36:32,607] jepsen worker 58 - jepsen.util 148 :ok :read [19 15] INFO [2022-01-10 05:36:32,607] jepsen worker 58 - jepsen.util 148 :invoke :write [19 25] INFO [2022-01-10 05:36:32,786] jepsen worker 17 - jepsen.util 107 :ok :read [5 24] INFO [2022-01-10 05:36:32,787] jepsen worker 17 - jepsen.util 107 :invoke :read [5 nil] INFO [2022-01-10 05:36:32,805] jepsen worker 28 - jepsen.util 118 :ok :read [9 22] INFO [2022-01-10 05:36:32,806] jepsen worker 28 - jepsen.util 118 :invoke :write [9 24] INFO [2022-01-10 05:36:32,856] jepsen worker 17 - jepsen.util 107 :ok :read [5 24] INFO [2022-01-10 05:36:32,856] jepsen worker 17 - jepsen.util 107 :invoke :write [5 17] INFO [2022-01-10 05:36:33,681] jepsen worker 59 - jepsen.util 149 :info :write [19 6] :RequestTimeoutException INFO [2022-01-10 05:36:33,682] jepsen worker 59 - jepsen.util 239 :invoke :write [19 25] INFO [2022-01-10 05:36:33,895] jepsen worker 18 - jepsen.util 108 :info :write [6 6] :RequestTimeoutException INFO [2022-01-10 05:36:33,897] jepsen worker 18 - jepsen.util 198 :invoke :read [6 nil] INFO [2022-01-10 05:36:34,760] jepsen worker 10 - jepsen.util 100 :info :write [3 18] :RequestTimeoutException INFO [2022-01-10 05:36:34,761] jepsen worker 10 - jepsen.util 190 :invoke :write [3 28] INFO [2022-01-10 05:36:34,983] jepsen worker 62 - jepsen.util 152 :info :write [20 29] :RequestTimeoutException INFO [2022-01-10 05:36:34,983] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 05:36:35,497] jepsen worker 5 - jepsen.util 95 :info :write [1 17] :RequestTimeoutException INFO [2022-01-10 05:36:35,497] jepsen worker 5 - jepsen.util 185 :invoke :write [1 7] INFO [2022-01-10 05:36:35,621] jepsen worker 72 - jepsen.util 162 :info :write [24 5] :RequestTimeoutException INFO [2022-01-10 05:36:35,623] jepsen worker 72 - jepsen.util 252 :invoke :read [24 nil] INFO [2022-01-10 05:36:35,776] jepsen worker 15 - jepsen.util 105 :info :write [5 42] :RequestTimeoutException INFO [2022-01-10 05:36:35,777] jepsen worker 15 - jepsen.util 195 :invoke :write [5 21] INFO [2022-01-10 05:36:35,977] jepsen worker 32 - jepsen.util 122 :info :write [10 24] :RequestTimeoutException INFO [2022-01-10 05:36:35,978] jepsen worker 32 - jepsen.util 212 :invoke :write [10 45] INFO [2022-01-10 05:36:36,069] jepsen worker 16 - jepsen.util 106 :ok :read [5 24] INFO [2022-01-10 05:36:36,069] jepsen worker 16 - jepsen.util 106 :invoke :write [5 13] INFO [2022-01-10 05:36:36,146] jepsen worker 57 - jepsen.util 57 :ok :read [19 15] INFO [2022-01-10 05:36:36,147] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 05:36:36,185] jepsen worker 27 - jepsen.util 117 :ok :read [9 22] INFO [2022-01-10 05:36:36,186] jepsen worker 27 - jepsen.util 117 :invoke :write [9 49] INFO [2022-01-10 05:36:36,310] jepsen worker 67 - jepsen.util 157 :ok :read [22 :nil] INFO [2022-01-10 05:36:36,311] jepsen worker 67 - jepsen.util 157 :invoke :read [22 nil] INFO [2022-01-10 05:36:36,381] jepsen worker 57 - jepsen.util 57 :ok :read [19 15] INFO [2022-01-10 05:36:36,382] jepsen worker 57 - jepsen.util 57 :invoke :write [19 10] INFO [2022-01-10 05:36:36,430] jepsen worker 41 - jepsen.util 131 :ok :read [13 32] INFO [2022-01-10 05:36:36,431] jepsen worker 41 - jepsen.util 131 :invoke :read [13 nil] INFO [2022-01-10 05:36:36,482] jepsen worker 67 - jepsen.util 157 :ok :read [22 :nil] INFO [2022-01-10 05:36:36,483] jepsen worker 67 - jepsen.util 157 :invoke :read [22 nil] INFO [2022-01-10 05:36:36,500] jepsen worker 67 - jepsen.util 157 :ok :read [22 :nil] INFO [2022-01-10 05:36:36,500] jepsen worker 67 - jepsen.util 157 :invoke :read [22 nil] INFO [2022-01-10 05:36:36,512] jepsen worker 41 - jepsen.util 131 :ok :read [13 32] INFO [2022-01-10 05:36:36,513] jepsen worker 41 - jepsen.util 131 :invoke :read [13 nil] INFO [2022-01-10 05:36:36,526] jepsen worker 67 - jepsen.util 157 :ok :read [22 :nil] INFO [2022-01-10 05:36:36,527] jepsen worker 67 - jepsen.util 157 :invoke :read [22 nil] INFO [2022-01-10 05:36:36,538] jepsen worker 67 - jepsen.util 157 :ok :read [22 :nil] INFO [2022-01-10 05:36:36,540] jepsen worker 67 - jepsen.util 157 :invoke :write [22 6] INFO [2022-01-10 05:36:36,553] jepsen worker 41 - jepsen.util 131 :ok :read [13 32] INFO [2022-01-10 05:36:36,556] jepsen worker 41 - jepsen.util 131 :invoke :read [13 nil] INFO [2022-01-10 05:36:36,644] jepsen worker 88 - jepsen.util 178 :info :write [29 44] :SyncWriteAmbiguous INFO [2022-01-10 05:36:36,645] jepsen worker 43 - jepsen.util 133 :info :write [14 41] :RequestTimeoutException INFO [2022-01-10 05:36:36,646] jepsen worker 88 - jepsen.util 268 :invoke :write [29 33] INFO [2022-01-10 05:36:36,646] jepsen worker 43 - jepsen.util 223 :invoke :read [14 nil] INFO [2022-01-10 05:36:36,654] jepsen worker 20 - jepsen.util 110 :ok :read [6 25] INFO [2022-01-10 05:36:36,655] jepsen worker 20 - jepsen.util 110 :invoke :read [6 nil] INFO [2022-01-10 05:36:36,702] jepsen worker 41 - jepsen.util 131 :ok :read [13 32] INFO [2022-01-10 05:36:36,703] jepsen worker 41 - jepsen.util 131 :invoke :read [13 nil] INFO [2022-01-10 05:36:36,746] jepsen worker 14 - jepsen.util 104 :info :write [4 21] :RequestTimeoutException INFO [2022-01-10 05:36:36,747] jepsen worker 14 - jepsen.util 194 :invoke :write [4 29] INFO [2022-01-10 05:36:36,777] jepsen worker 41 - jepsen.util 131 :ok :read [13 32] INFO [2022-01-10 05:36:36,778] jepsen worker 41 - jepsen.util 131 :invoke :read [13 nil] INFO [2022-01-10 05:36:36,864] jepsen worker 41 - jepsen.util 131 :ok :read [13 32] INFO [2022-01-10 05:36:36,865] jepsen worker 41 - jepsen.util 131 :invoke :read [13 nil] INFO [2022-01-10 05:36:36,918] jepsen worker 41 - jepsen.util 131 :ok :read [13 32] INFO [2022-01-10 05:36:36,919] jepsen worker 41 - jepsen.util 131 :invoke :read [13 nil] INFO [2022-01-10 05:36:36,930] jepsen worker 63 - jepsen.util 153 :info :write [21 6] :RequestTimeoutException INFO [2022-01-10 05:36:36,931] jepsen worker 63 - jepsen.util 243 :invoke :read [21 nil] INFO [2022-01-10 05:36:36,965] jepsen worker 41 - jepsen.util 131 :ok :read [13 32] INFO [2022-01-10 05:36:36,966] jepsen worker 41 - jepsen.util 131 :invoke :write [13 23] INFO [2022-01-10 05:36:37,007] jepsen worker 46 - jepsen.util 136 :ok :read [15 :nil] INFO [2022-01-10 05:36:37,007] jepsen worker 46 - jepsen.util 136 :invoke :read [15 nil] INFO [2022-01-10 05:36:37,032] jepsen worker 23 - jepsen.util 113 :info :write [7 12] :RequestTimeoutException INFO [2022-01-10 05:36:37,033] jepsen worker 23 - jepsen.util 203 :invoke :read [7 nil] INFO [2022-01-10 05:36:37,036] jepsen worker 55 - jepsen.util 145 :info :write [18 40] :RequestTimeoutException INFO [2022-01-10 05:36:37,037] jepsen worker 55 - jepsen.util 235 :invoke :write [18 25] INFO [2022-01-10 05:36:37,074] jepsen worker 71 - jepsen.util 161 :info :write [23 27] :SyncWriteAmbiguous INFO [2022-01-10 05:36:37,076] jepsen worker 71 - jepsen.util 251 :invoke :read [23 nil] INFO [2022-01-10 05:36:37,198] jepsen worker 31 - jepsen.util 121 :info :write [10 45] :SyncWriteAmbiguous INFO [2022-01-10 05:36:37,199] jepsen worker 31 - jepsen.util 211 :invoke :read [10 nil] INFO [2022-01-10 05:36:37,222] jepsen worker 31 - jepsen.util 211 :ok :read [10 36] INFO [2022-01-10 05:36:37,222] jepsen worker 31 - jepsen.util 211 :invoke :write [10 23] INFO [2022-01-10 05:36:37,225] jepsen worker 51 - jepsen.util 141 :info :write [17 11] :RequestTimeoutException INFO [2022-01-10 05:36:37,226] jepsen worker 51 - jepsen.util 231 :invoke :read [17 nil] INFO [2022-01-10 05:36:37,358] jepsen worker 72 - jepsen.util 252 :ok :read [24 33] INFO [2022-01-10 05:36:37,358] jepsen worker 72 - jepsen.util 252 :invoke :write [24 49] INFO [2022-01-10 05:36:37,383] jepsen worker 24 - jepsen.util 114 :ok :read [8 :nil] INFO [2022-01-10 05:36:37,383] jepsen worker 24 - jepsen.util 114 :invoke :write [8 16] INFO [2022-01-10 05:36:37,399] jepsen worker 63 - jepsen.util 243 :ok :read [21 7] INFO [2022-01-10 05:36:37,399] jepsen worker 63 - jepsen.util 243 :invoke :read [21 nil] INFO [2022-01-10 05:36:37,621] jepsen worker 47 - jepsen.util 137 :ok :read [15 :nil] INFO [2022-01-10 05:36:37,621] jepsen worker 45 - jepsen.util 135 :ok :read [15 :nil] INFO [2022-01-10 05:36:37,621] jepsen worker 47 - jepsen.util 137 :invoke :write [15 23] INFO [2022-01-10 05:36:37,622] jepsen worker 45 - jepsen.util 135 :invoke :write [15 25] INFO [2022-01-10 05:36:37,769] jepsen worker 22 - jepsen.util 112 :ok :read [7 :nil] INFO [2022-01-10 05:36:37,769] jepsen worker 22 - jepsen.util 112 :invoke :write [7 16] INFO [2022-01-10 05:36:37,771] jepsen worker 62 - jepsen.util 242 :ok :read [20 :nil] INFO [2022-01-10 05:36:37,772] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 05:36:37,773] jepsen worker 46 - jepsen.util 136 :ok :read [15 :nil] INFO [2022-01-10 05:36:37,773] jepsen worker 46 - jepsen.util 136 :invoke :read [15 nil] INFO [2022-01-10 05:36:37,776] jepsen worker 81 - jepsen.util 171 :info :write [27 49] :RequestTimeoutException INFO [2022-01-10 05:36:37,777] jepsen worker 81 - jepsen.util 261 :invoke :write [27 0] INFO [2022-01-10 05:36:37,783] jepsen worker 43 - jepsen.util 223 :ok :read [14 14] INFO [2022-01-10 05:36:37,784] jepsen worker 43 - jepsen.util 223 :invoke :write [14 42] INFO [2022-01-10 05:36:37,807] jepsen worker 21 - jepsen.util 111 :info :write [7 43] :RequestTimeoutException INFO [2022-01-10 05:36:37,808] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 05:36:38,084] jepsen worker 18 - jepsen.util 198 :ok :read [6 25] INFO [2022-01-10 05:36:38,085] jepsen worker 18 - jepsen.util 198 :invoke :write [6 35] INFO [2022-01-10 05:36:38,094] jepsen worker 63 - jepsen.util 243 :ok :read [21 7] INFO [2022-01-10 05:36:38,094] jepsen worker 63 - jepsen.util 243 :invoke :write [21 11] INFO [2022-01-10 05:36:38,127] jepsen worker 50 - jepsen.util 140 :info :write [16 3] :RequestTimeoutException INFO [2022-01-10 05:36:38,128] jepsen worker 50 - jepsen.util 230 :invoke :write [16 32] INFO [2022-01-10 05:36:38,190] jepsen worker 85 - jepsen.util 175 :info :write [28 19] :RequestTimeoutException INFO [2022-01-10 05:36:38,191] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 05:36:38,264] jepsen worker 62 - jepsen.util 242 :ok :read [20 :nil] INFO [2022-01-10 05:36:38,264] jepsen worker 46 - jepsen.util 136 :ok :read [15 :nil] INFO [2022-01-10 05:36:38,265] jepsen worker 46 - jepsen.util 136 :invoke :write [15 11] INFO [2022-01-10 05:36:38,265] jepsen worker 62 - jepsen.util 242 :invoke :write [20 33] INFO [2022-01-10 05:36:38,412] jepsen worker 8 - jepsen.util 98 :info :write [2 14] :RequestTimeoutException INFO [2022-01-10 05:36:38,413] jepsen worker 8 - jepsen.util 188 :invoke :read [2 nil] INFO [2022-01-10 05:36:38,491] jepsen worker 80 - jepsen.util 170 :info :write [26 26] :RequestTimeoutException INFO [2022-01-10 05:36:38,492] jepsen worker 80 - jepsen.util 260 :invoke :write [26 44] INFO [2022-01-10 05:36:38,716] jepsen worker 89 - jepsen.util 179 :info :write [29 16] :RequestTimeoutException INFO [2022-01-10 05:36:38,717] jepsen worker 89 - jepsen.util 269 :invoke :write [29 24] INFO [2022-01-10 05:36:38,767] jepsen worker 56 - jepsen.util 146 :info :write [18 11] :RequestTimeoutException INFO [2022-01-10 05:36:38,768] jepsen worker 56 - jepsen.util 236 :invoke :write [18 8] INFO [2022-01-10 05:36:38,807] jepsen worker 42 - jepsen.util 42 :info :write [14 27] :RequestTimeoutException INFO [2022-01-10 05:36:38,808] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 05:36:38,827] jepsen worker 65 - jepsen.util 155 :info :write [21 20] :RequestTimeoutException INFO [2022-01-10 05:36:38,827] jepsen worker 65 - jepsen.util 245 :invoke :write [21 29] INFO [2022-01-10 05:36:38,872] jepsen worker 44 - jepsen.util 134 :info :write [14 31] :RequestTimeoutException INFO [2022-01-10 05:36:38,873] jepsen worker 44 - jepsen.util 224 :invoke :write [14 28] INFO [2022-01-10 05:36:38,971] jepsen worker 23 - jepsen.util 203 :ok :read [7 :nil] INFO [2022-01-10 05:36:38,971] jepsen worker 30 - jepsen.util 120 :info :write [10 1] :RequestTimeoutException INFO [2022-01-10 05:36:38,973] jepsen worker 23 - jepsen.util 203 :invoke :read [7 nil] INFO [2022-01-10 05:36:38,975] jepsen worker 30 - jepsen.util 210 :invoke :read [10 nil] INFO [2022-01-10 05:36:39,055] jepsen worker 83 - jepsen.util 173 :info :write [27 37] :RequestTimeoutException INFO [2022-01-10 05:36:39,056] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 05:36:39,071] jepsen worker 11 - jepsen.util 101 :ok :read [3 22] INFO [2022-01-10 05:36:39,071] jepsen worker 11 - jepsen.util 101 :invoke :write [3 11] INFO [2022-01-10 05:36:39,107] jepsen worker 19 - jepsen.util 109 :info :write [6 17] :RequestTimeoutException INFO [2022-01-10 05:36:39,108] jepsen worker 19 - jepsen.util 199 :invoke :write [6 11] INFO [2022-01-10 05:36:39,127] jepsen worker 87 - jepsen.util 177 :ok :read [29 :nil] INFO [2022-01-10 05:36:39,128] jepsen worker 87 - jepsen.util 177 :invoke :write [29 23] INFO [2022-01-10 05:36:39,150] jepsen worker 53 - jepsen.util 143 :ok :read [17 37] INFO [2022-01-10 05:36:39,151] jepsen worker 53 - jepsen.util 143 :invoke :write [17 20] INFO [2022-01-10 05:36:39,256] jepsen worker 38 - jepsen.util 128 :info :write [12 49] :RequestTimeoutException INFO [2022-01-10 05:36:39,257] jepsen worker 38 - jepsen.util 218 :invoke :read [12 nil] INFO [2022-01-10 05:36:39,272] jepsen worker 52 - jepsen.util 142 :info :write [17 30] :RequestTimeoutException INFO [2022-01-10 05:36:39,272] jepsen worker 52 - jepsen.util 232 :invoke :write [17 45] INFO [2022-01-10 05:36:39,386] jepsen worker 3 - jepsen.util 93 :info :write [1 4] :RequestTimeoutException INFO [2022-01-10 05:36:39,386] jepsen worker 3 - jepsen.util 183 :invoke :read [1 nil] WARN [2022-01-10 05:36:39,447] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9917047,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9917047,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9454235,"encode_duration_us":39373,"operation_id":"0x74e","timeout_ms":10000,"last_server_duration_us":9454235,"total_duration_us":10005673}],"total_count":1}} WARN [2022-01-10 05:36:39,448] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":1745397,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1745397,"total_server_duration_us":11,"operation_id":"0x826","timeout_ms":30000,"last_server_duration_us":11,"total_duration_us":12021496},{"operation_name":"upsert","last_dispatch_duration_us":4381329,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":5910472,"total_server_duration_us":145188,"encode_duration_us":17278,"operation_id":"0x895","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10010290},{"operation_name":"upsert","last_dispatch_duration_us":571656,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":6839653,"total_server_duration_us":270522,"encode_duration_us":16842,"operation_id":"0x8b5","timeout_ms":10000,"last_server_duration_us":55668,"total_duration_us":10007825},{"operation_name":"upsert","last_dispatch_duration_us":4990847,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":4990847,"total_server_duration_us":23222,"encode_duration_us":18640,"operation_id":"0x800","timeout_ms":10000,"last_server_duration_us":23222,"total_duration_us":10005993},{"operation_name":"upsert","last_dispatch_duration_us":311258,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":3261199,"total_server_duration_us":160735,"encode_duration_us":18980,"operation_id":"0x866","timeout_ms":10000,"last_server_duration_us":32451,"total_duration_us":10005929},{"operation_name":"upsert","last_dispatch_duration_us":9917047,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9917047,"total_server_duration_us":9454235,"encode_duration_us":39373,"operation_id":"0x74e","timeout_ms":10000,"last_server_duration_us":9454235,"total_duration_us":10005673},{"operation_name":"upsert","last_dispatch_duration_us":10293290,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11529230,"total_server_duration_us":290010,"encode_duration_us":32492,"operation_id":"0x750","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10004475},{"operation_name":"upsert","last_dispatch_duration_us":182857,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9388367,"total_server_duration_us":430708,"encode_duration_us":43975,"operation_id":"0x75c","timeout_ms":10000,"last_server_duration_us":20521,"total_duration_us":10004246},{"operation_name":"upsert","last_dispatch_duration_us":8837399,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10140182,"total_server_duration_us":436431,"encode_duration_us":23774,"operation_id":"0x72f","timeout_ms":10000,"last_server_duration_us":53135,"total_duration_us":10002915},{"operation_name":"upsert","last_dispatch_duration_us":10816244,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11924121,"total_server_duration_us":635655,"encode_duration_us":38123,"operation_id":"0x6fc","timeout_ms":10000,"last_server_duration_us":539626,"total_duration_us":10002710}],"total_count":17}} INFO [2022-01-10 05:36:39,486] jepsen worker 64 - jepsen.util 154 :info :write [21 49] :RequestTimeoutException INFO [2022-01-10 05:36:39,488] jepsen worker 64 - jepsen.util 244 :invoke :read [21 nil] INFO [2022-01-10 05:36:39,490] jepsen worker 35 - jepsen.util 125 :info :write [11 15] :RequestTimeoutException INFO [2022-01-10 05:36:39,491] jepsen worker 35 - jepsen.util 215 :invoke :write [11 23] INFO [2022-01-10 05:36:39,506] jepsen worker 39 - jepsen.util 39 :info :write [13 43] :RequestTimeoutException INFO [2022-01-10 05:36:39,506] jepsen worker 39 - jepsen.util 129 :invoke :write [13 47] INFO [2022-01-10 05:36:39,572] jepsen worker 4 - jepsen.util 94 :info :write [1 27] :RequestTimeoutException INFO [2022-01-10 05:36:39,573] jepsen worker 4 - jepsen.util 184 :invoke :read [1 nil] INFO [2022-01-10 05:36:39,608] jepsen worker 3 - jepsen.util 183 :ok :read [1 41] INFO [2022-01-10 05:36:39,608] jepsen worker 3 - jepsen.util 183 :invoke :write [1 13] INFO [2022-01-10 05:36:39,796] jepsen worker 77 - jepsen.util 167 :info :write [25 8] :RequestTimeoutException INFO [2022-01-10 05:36:39,797] jepsen worker 77 - jepsen.util 257 :invoke :write [25 38] INFO [2022-01-10 05:36:39,811] jepsen worker 74 - jepsen.util 164 :info :write [24 5] :RequestTimeoutException INFO [2022-01-10 05:36:39,812] jepsen worker 74 - jepsen.util 254 :invoke :write [24 49] INFO [2022-01-10 05:36:39,892] jepsen worker 69 - jepsen.util 159 :info :write [23 34] :RequestTimeoutException INFO [2022-01-10 05:36:39,893] jepsen worker 69 - jepsen.util 249 :invoke :write [23 37] INFO [2022-01-10 05:36:39,906] jepsen worker 60 - jepsen.util 150 :info :write [20 10] :RequestTimeoutException INFO [2022-01-10 05:36:39,907] jepsen worker 60 - jepsen.util 240 :invoke :write [20 30] INFO [2022-01-10 05:36:39,937] jepsen worker 33 - jepsen.util 123 :info :write [11 5] :RequestTimeoutException INFO [2022-01-10 05:36:39,938] jepsen worker 33 - jepsen.util 213 :invoke :read [11 nil] INFO [2022-01-10 05:36:39,971] jepsen worker 68 - jepsen.util 158 :info :write [22 34] :RequestTimeoutException INFO [2022-01-10 05:36:39,972] jepsen worker 68 - jepsen.util 248 :invoke :read [22 nil] INFO [2022-01-10 05:36:39,996] jepsen worker 0 - jepsen.util 90 :info :write [0 34] :RequestTimeoutException INFO [2022-01-10 05:36:39,997] jepsen worker 0 - jepsen.util 180 :invoke :write [0 14] INFO [2022-01-10 05:36:40,066] jepsen worker 86 - jepsen.util 176 :info :write [28 30] :RequestTimeoutException INFO [2022-01-10 05:36:40,067] jepsen worker 86 - jepsen.util 266 :invoke :write [28 41] INFO [2022-01-10 05:36:40,081] jepsen worker 76 - jepsen.util 166 :info :write [25 43] :RequestTimeoutException INFO [2022-01-10 05:36:40,082] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 05:36:40,126] jepsen worker 84 - jepsen.util 174 :info :write [28 25] :RequestTimeoutException INFO [2022-01-10 05:36:40,127] jepsen worker 84 - jepsen.util 264 :invoke :read [28 nil] INFO [2022-01-10 05:36:40,195] jepsen worker 13 - jepsen.util 103 :info :write [4 40] :RequestTimeoutException INFO [2022-01-10 05:36:40,196] jepsen worker 13 - jepsen.util 193 :invoke :write [4 34] INFO [2022-01-10 05:36:40,212] jepsen worker 12 - jepsen.util 102 :info :write [4 5] :RequestTimeoutException INFO [2022-01-10 05:36:40,213] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 05:36:40,240] jepsen worker 34 - jepsen.util 124 :info :write [11 11] :RequestTimeoutException INFO [2022-01-10 05:36:40,241] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 05:36:40,276] jepsen worker 9 - jepsen.util 99 :info :write [3 3] :RequestTimeoutException INFO [2022-01-10 05:36:40,277] jepsen worker 9 - jepsen.util 189 :invoke :write [3 2] INFO [2022-01-10 05:36:40,305] jepsen worker 79 - jepsen.util 169 :info :write [26 15] :RequestTimeoutException INFO [2022-01-10 05:36:40,306] jepsen worker 79 - jepsen.util 259 :invoke :write [26 37] INFO [2022-01-10 05:36:40,306] jepsen worker 75 - jepsen.util 165 :info :write [25 41] :RequestTimeoutException INFO [2022-01-10 05:36:40,307] jepsen worker 75 - jepsen.util 255 :invoke :write [25 13] INFO [2022-01-10 05:36:40,337] jepsen worker 49 - jepsen.util 139 :info :write [16 23] :RequestTimeoutException INFO [2022-01-10 05:36:40,337] jepsen worker 49 - jepsen.util 229 :invoke :write [16 24] INFO [2022-01-10 05:36:40,370] jepsen worker 26 - jepsen.util 116 :info :write [8 2] :RequestTimeoutException INFO [2022-01-10 05:36:40,371] jepsen worker 26 - jepsen.util 206 :invoke :write [8 46] INFO [2022-01-10 05:36:40,402] jepsen worker 61 - jepsen.util 151 :info :write [20 39] :RequestTimeoutException INFO [2022-01-10 05:36:40,403] jepsen worker 61 - jepsen.util 241 :invoke :write [20 33] INFO [2022-01-10 05:36:40,472] jepsen worker 70 - jepsen.util 160 :info :write [23 27] :RequestTimeoutException INFO [2022-01-10 05:36:40,472] jepsen worker 70 - jepsen.util 250 :invoke :read [23 nil] INFO [2022-01-10 05:36:40,485] jepsen worker 37 - jepsen.util 127 :info :write [12 27] :RequestTimeoutException INFO [2022-01-10 05:36:40,485] jepsen worker 37 - jepsen.util 217 :invoke :write [12 7] INFO [2022-01-10 05:36:40,500] jepsen worker 40 - jepsen.util 130 :info :write [13 8] :RequestTimeoutException INFO [2022-01-10 05:36:40,500] jepsen worker 40 - jepsen.util 220 :invoke :read [13 nil] INFO [2022-01-10 05:36:40,511] jepsen worker 1 - jepsen.util 91 :info :write [0 48] :RequestTimeoutException INFO [2022-01-10 05:36:40,512] jepsen worker 1 - jepsen.util 181 :invoke :write [0 21] INFO [2022-01-10 05:36:40,592] jepsen worker 7 - jepsen.util 7 :info :write [2 29] :RequestTimeoutException INFO [2022-01-10 05:36:40,593] jepsen worker 7 - jepsen.util 97 :invoke :read [2 nil] INFO [2022-01-10 05:36:40,726] jepsen worker 78 - jepsen.util 168 :info :write [26 11] :RequestTimeoutException INFO [2022-01-10 05:36:40,727] jepsen worker 78 - jepsen.util 258 :invoke :write [26 6] WARN [2022-01-10 05:36:40,966] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9183497,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18755482,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9278357,"encode_duration_us":137394,"operation_id":"0x643","timeout_ms":10000,"last_server_duration_us":9117117,"total_duration_us":10007374},{"operation_name":"upsert","last_dispatch_duration_us":9128356,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17720655,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9361740,"encode_duration_us":228491,"operation_id":"0x6a7","timeout_ms":10000,"last_server_duration_us":9106466,"total_duration_us":10009227},{"operation_name":"upsert","last_dispatch_duration_us":12224777,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12618924,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9891444,"encode_duration_us":24115,"operation_id":"0x755","timeout_ms":10000,"last_server_duration_us":9829520,"total_duration_us":10012312}],"total_count":3}} WARN [2022-01-10 05:36:41,040] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16699982,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16699982,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9108596,"encode_duration_us":54743,"operation_id":"0x73c","timeout_ms":10000,"last_server_duration_us":9108596,"total_duration_us":10002274}],"total_count":1}} WARN [2022-01-10 05:36:41,041] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":5043843,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5043843,"total_server_duration_us":134160,"encode_duration_us":16757,"operation_id":"0x861","timeout_ms":10000,"last_server_duration_us":134160,"total_duration_us":10011337},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":17794,"operation_id":"0x8b4","timeout_ms":10000,"total_duration_us":10009203},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":16638,"operation_id":"0x890","timeout_ms":10000,"total_duration_us":10008864},{"operation_name":"upsert","last_dispatch_duration_us":5844101,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5844101,"total_server_duration_us":66168,"encode_duration_us":20787,"operation_id":"0x827","timeout_ms":10000,"last_server_duration_us":66168,"total_duration_us":10008297},{"operation_name":"upsert","last_dispatch_duration_us":213890,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1554668,"total_server_duration_us":121339,"encode_duration_us":16679,"operation_id":"0x8f6","timeout_ms":10000,"last_server_duration_us":35011,"total_duration_us":10007314},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":16355,"operation_id":"0x8f1","timeout_ms":10000,"total_duration_us":10006674},{"operation_name":"upsert","last_dispatch_duration_us":656129,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":656129,"total_server_duration_us":45614,"encode_duration_us":18016,"operation_id":"0x8e3","timeout_ms":10000,"last_server_duration_us":45614,"total_duration_us":10005841},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":17302,"operation_id":"0x8c3","timeout_ms":10000,"total_duration_us":10005349},{"operation_name":"upsert","last_dispatch_duration_us":4848922,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":4848922,"total_server_duration_us":104546,"encode_duration_us":16423,"operation_id":"0x7ea","timeout_ms":10000,"last_server_duration_us":104546,"total_duration_us":10001226},{"operation_name":"upsert","last_dispatch_duration_us":9271682,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9271682,"total_server_duration_us":9193958,"encode_duration_us":26566,"operation_id":"0x879","timeout_ms":10000,"last_server_duration_us":9193958,"total_duration_us":9276077}],"total_count":13}} WARN [2022-01-10 05:36:41,054] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9350420,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18791279,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9412856,"encode_duration_us":34341,"operation_id":"0x6f3","timeout_ms":10000,"last_server_duration_us":9293616,"total_duration_us":10003859},{"operation_name":"upsert","last_dispatch_duration_us":10415903,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10738608,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":10314789,"encode_duration_us":22091,"operation_id":"0x8c2","timeout_ms":10000,"last_server_duration_us":10293868,"total_duration_us":10005328}],"total_count":2}} WARN [2022-01-10 05:36:41,055] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11876875,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11876875,"total_server_duration_us":27,"operation_id":"0x80a","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":11878149},{"operation_name":"upsert","last_dispatch_duration_us":6886492,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6886492,"total_server_duration_us":119129,"encode_duration_us":17595,"operation_id":"0x8ee","timeout_ms":10000,"last_server_duration_us":119129,"total_duration_us":10010032},{"operation_name":"upsert","last_dispatch_duration_us":182435,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7206559,"total_server_duration_us":208833,"encode_duration_us":26657,"operation_id":"0x8c6","timeout_ms":10000,"last_server_duration_us":20841,"total_duration_us":10009676},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":16762,"operation_id":"0x834","timeout_ms":10000,"total_duration_us":10008125},{"operation_name":"upsert","last_dispatch_duration_us":1816513,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":4456119,"total_server_duration_us":289195,"encode_duration_us":19771,"operation_id":"0x851","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10007811},{"operation_name":"upsert","last_dispatch_duration_us":810829,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":810829,"total_server_duration_us":41445,"encode_duration_us":19957,"operation_id":"0x922","timeout_ms":10000,"last_server_duration_us":41445,"total_duration_us":10006311},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":16841,"operation_id":"0x90c","timeout_ms":10000,"total_duration_us":10006264},{"operation_name":"upsert","last_dispatch_duration_us":10415903,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10738608,"total_server_duration_us":10314789,"encode_duration_us":22091,"operation_id":"0x8c2","timeout_ms":10000,"last_server_duration_us":10293868,"total_duration_us":10005328},{"operation_name":"upsert","encode_duration_us":16354,"operation_id":"0x883","timeout_ms":10000,"total_duration_us":10005044},{"operation_name":"upsert","encode_duration_us":17866,"operation_id":"0x8f3","timeout_ms":10000,"total_duration_us":10004267}],"total_count":15}} WARN [2022-01-10 05:36:41,057] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":43799,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":338180,"total_server_duration_us":71310,"encode_duration_us":16525,"operation_id":"0x8f5","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10010489},{"operation_name":"upsert","last_dispatch_duration_us":158012,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4830691,"total_server_duration_us":843061,"encode_duration_us":20108,"operation_id":"0x8a9","timeout_ms":10000,"last_server_duration_us":43618,"total_duration_us":10008403},{"operation_name":"upsert","last_dispatch_duration_us":708133,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":3899845,"total_server_duration_us":644777,"encode_duration_us":16911,"operation_id":"0x8b7","timeout_ms":10000,"last_server_duration_us":35412,"total_duration_us":10006691},{"operation_name":"upsert","last_dispatch_duration_us":119044,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":2214122,"total_server_duration_us":818663,"encode_duration_us":16260,"operation_id":"0x8ef","timeout_ms":10000,"last_server_duration_us":44500,"total_duration_us":10006086},{"operation_name":"upsert","last_dispatch_duration_us":116389,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4863592,"total_server_duration_us":1385991,"encode_duration_us":18947,"operation_id":"0x87a","timeout_ms":10000,"last_server_duration_us":35011,"total_duration_us":10005330},{"operation_name":"upsert","last_dispatch_duration_us":136977,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":5154816,"total_server_duration_us":911448,"encode_duration_us":21642,"operation_id":"0x8b0","timeout_ms":10000,"last_server_duration_us":124398,"total_duration_us":10003012},{"operation_name":"upsert","last_dispatch_duration_us":64149,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":2732851,"total_server_duration_us":567261,"encode_duration_us":16870,"operation_id":"0x90a","timeout_ms":10000,"last_server_duration_us":33328,"total_duration_us":10002935},{"operation_name":"upsert","last_dispatch_duration_us":9707222,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10462132,"total_server_duration_us":380204,"encode_duration_us":19702,"operation_id":"0x845","timeout_ms":10000,"last_server_duration_us":44279,"total_duration_us":10002834},{"operation_name":"upsert","last_dispatch_duration_us":10627180,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":14070033,"total_server_duration_us":449144,"encode_duration_us":16597,"operation_id":"0x793","timeout_ms":10000,"last_server_duration_us":22887,"total_duration_us":10002428},{"operation_name":"upsert","last_dispatch_duration_us":9700220,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10379913,"total_server_duration_us":736314,"encode_duration_us":16322,"operation_id":"0x825","timeout_ms":10000,"last_server_duration_us":415247,"total_duration_us":10002071}],"total_count":17}} INFO [2022-01-10 05:36:41,081] jepsen worker 29 - jepsen.util 119 :info :write [9 6] :RequestTimeoutException INFO [2022-01-10 05:36:41,082] jepsen worker 29 - jepsen.util 209 :invoke :read [9 nil] WARN [2022-01-10 05:36:41,150] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":35530,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":5240458,"total_server_duration_us":935304,"encode_duration_us":17872,"operation_id":"0x912","timeout_ms":10000,"last_server_duration_us":25188,"total_duration_us":10010466},{"operation_name":"upsert","last_dispatch_duration_us":7154006,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7154006,"total_server_duration_us":106305,"encode_duration_us":16362,"operation_id":"0x821","timeout_ms":10000,"last_server_duration_us":106305,"total_duration_us":10010008},{"operation_name":"upsert","last_dispatch_duration_us":6540508,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6540508,"total_server_duration_us":300241,"encode_duration_us":17474,"operation_id":"0x8b2","timeout_ms":10000,"last_server_duration_us":300241,"total_duration_us":10009644},{"operation_name":"upsert","last_dispatch_duration_us":12675002,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":12675002,"total_server_duration_us":9106466,"encode_duration_us":17591,"operation_id":"0x875","timeout_ms":10000,"last_server_duration_us":9106466,"total_duration_us":10009639},{"operation_name":"upsert","last_dispatch_duration_us":33219,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":5260489,"total_server_duration_us":342456,"encode_duration_us":18935,"operation_id":"0x903","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10008625},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":22610,"operation_id":"0x916","timeout_ms":10000,"total_duration_us":10008284},{"operation_name":"upsert","last_dispatch_duration_us":984712,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":984712,"total_server_duration_us":41230,"encode_duration_us":21026,"operation_id":"0x88f","timeout_ms":10000,"last_server_duration_us":41230,"total_duration_us":10007101},{"operation_name":"upsert","last_dispatch_duration_us":783118,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":5801038,"total_server_duration_us":1095858,"encode_duration_us":18079,"operation_id":"0x8eb","timeout_ms":10000,"last_server_duration_us":379281,"total_duration_us":10006672},{"operation_name":"upsert","last_dispatch_duration_us":474298,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":6216496,"total_server_duration_us":1169836,"encode_duration_us":20658,"operation_id":"0x8ca","timeout_ms":10000,"last_server_duration_us":392886,"total_duration_us":10006139},{"operation_name":"upsert","last_dispatch_duration_us":4536469,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":4536469,"total_server_duration_us":44058,"encode_duration_us":18286,"operation_id":"0x918","timeout_ms":10000,"last_server_duration_us":44058,"total_duration_us":10005716}],"total_count":19}} WARN [2022-01-10 05:36:41,166] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","encode_duration_us":17058,"operation_id":"0x905","timeout_ms":10000,"total_duration_us":10008819},{"operation_name":"upsert","last_dispatch_duration_us":267047,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8706443,"total_server_duration_us":167112,"encode_duration_us":22750,"operation_id":"0x88b","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10006047},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","encode_duration_us":21460,"operation_id":"0x8c8","timeout_ms":10000,"total_duration_us":10005176},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","encode_duration_us":18790,"operation_id":"0x90e","timeout_ms":10000,"total_duration_us":10003568},{"operation_name":"upsert","last_dispatch_duration_us":241284,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":6010244,"total_server_duration_us":41448,"encode_duration_us":17479,"operation_id":"0x915","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10003291},{"operation_name":"upsert","last_dispatch_duration_us":59003,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":189387,"total_server_duration_us":60129,"encode_duration_us":22371,"operation_id":"0x817","timeout_ms":10000,"last_server_duration_us":22223,"total_duration_us":10002573},{"operation_name":"upsert","last_dispatch_duration_us":1417173,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2036944,"total_server_duration_us":232420,"encode_duration_us":17357,"operation_id":"0x85f","timeout_ms":10000,"last_server_duration_us":36220,"total_duration_us":10001306},{"operation_name":"upsert","last_dispatch_duration_us":9080789,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9584020,"total_server_duration_us":9184918,"encode_duration_us":18358,"operation_id":"0x86a","timeout_ms":10000,"last_server_duration_us":9062854,"total_duration_us":9687989},{"operation_name":"upsert","last_dispatch_duration_us":9093541,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9093541,"total_server_duration_us":9050106,"encode_duration_us":17219,"operation_id":"0x86b","timeout_ms":10000,"last_server_duration_us":9050106,"total_duration_us":9097891},{"operation_name":"get","last_dispatch_duration_us":7596030,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7596030,"total_server_duration_us":84,"operation_id":"0x891","timeout_ms":30000,"last_server_duration_us":84,"total_duration_us":7597201}],"total_count":11}} WARN [2022-01-10 05:36:41,251] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12675002,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12675002,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9106466,"encode_duration_us":17591,"operation_id":"0x875","timeout_ms":10000,"last_server_duration_us":9106466,"total_duration_us":10009639}],"total_count":1}} WARN [2022-01-10 05:36:41,258] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9090518,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9665974,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9332185,"encode_duration_us":19552,"operation_id":"0x862","timeout_ms":10000,"last_server_duration_us":9072420,"total_duration_us":10001914},{"operation_name":"upsert","last_dispatch_duration_us":14572664,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18412173,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9218964,"encode_duration_us":34946,"operation_id":"0x75f","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10007863},{"operation_name":"upsert","last_dispatch_duration_us":9191728,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10402541,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9558611,"encode_duration_us":24814,"operation_id":"0x762","timeout_ms":10000,"last_server_duration_us":9146967,"total_duration_us":10007363},{"operation_name":"upsert","last_dispatch_duration_us":9129670,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9656903,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9395691,"encode_duration_us":26848,"operation_id":"0x75d","timeout_ms":10000,"last_server_duration_us":9114986,"total_duration_us":10008757}],"total_count":4}} INFO [2022-01-10 05:36:41,263] jepsen worker 73 - jepsen.util 163 :ok :read [24 33] INFO [2022-01-10 05:36:41,264] jepsen worker 73 - jepsen.util 163 :invoke :read [24 nil] INFO [2022-01-10 05:36:41,646] jepsen worker 70 - jepsen.util 250 :ok :read [23 :nil] INFO [2022-01-10 05:36:41,647] jepsen worker 70 - jepsen.util 250 :invoke :write [23 27] INFO [2022-01-10 05:36:41,944] jepsen worker 28 - jepsen.util 118 :info :write [9 24] :SyncWriteAmbiguous INFO [2022-01-10 05:36:41,945] jepsen worker 28 - jepsen.util 208 :invoke :write [9 14] INFO [2022-01-10 05:36:42,631] jepsen worker 58 - jepsen.util 148 :info :write [19 25] :RequestTimeoutException INFO [2022-01-10 05:36:42,632] jepsen worker 58 - jepsen.util 238 :invoke :read [19 nil] INFO [2022-01-10 05:36:42,885] jepsen worker 17 - jepsen.util 107 :info :write [5 17] :RequestTimeoutException INFO [2022-01-10 05:36:42,886] jepsen worker 17 - jepsen.util 197 :invoke :read [5 nil] INFO [2022-01-10 05:36:42,952] jepsen worker 36 - jepsen.util 126 :ok :read [12 :nil] INFO [2022-01-10 05:36:42,953] jepsen worker 36 - jepsen.util 126 :invoke :read [12 nil] INFO [2022-01-10 05:36:43,427] jepsen worker 25 - jepsen.util 115 :ok :read [8 :nil] INFO [2022-01-10 05:36:43,427] jepsen worker 21 - jepsen.util 201 :ok :read [7 :nil] INFO [2022-01-10 05:36:43,427] jepsen worker 25 - jepsen.util 115 :invoke :write [8 36] INFO [2022-01-10 05:36:43,428] jepsen worker 21 - jepsen.util 201 :invoke :write [7 46] INFO [2022-01-10 05:36:43,632] jepsen worker 6 - jepsen.util 96 :ok :read [2 21] INFO [2022-01-10 05:36:43,632] jepsen worker 6 - jepsen.util 96 :invoke :write [2 29] INFO [2022-01-10 05:36:43,705] jepsen worker 59 - jepsen.util 239 :info :write [19 25] :RequestTimeoutException INFO [2022-01-10 05:36:43,706] jepsen worker 59 - jepsen.util 329 :invoke :read [19 nil] INFO [2022-01-10 05:36:44,373] jepsen worker 36 - jepsen.util 126 :ok :read [12 :nil] INFO [2022-01-10 05:36:44,391] jepsen worker 36 - jepsen.util 126 :invoke :read [12 nil] INFO [2022-01-10 05:36:44,445] jepsen worker 71 - jepsen.util 251 :ok :read [23 :nil] INFO [2022-01-10 05:36:44,445] jepsen worker 71 - jepsen.util 251 :invoke :write [23 14] INFO [2022-01-10 05:36:44,782] jepsen worker 2 - jepsen.util 92 :ok :read [0 :nil] INFO [2022-01-10 05:36:44,782] jepsen worker 84 - jepsen.util 264 :ok :read [28 :nil] INFO [2022-01-10 05:36:44,782] jepsen worker 48 - jepsen.util 138 :ok :read [16 :nil] INFO [2022-01-10 05:36:44,783] jepsen worker 48 - jepsen.util 138 :invoke :read [16 nil] INFO [2022-01-10 05:36:44,784] jepsen worker 2 - jepsen.util 92 :invoke :write [0 18] INFO [2022-01-10 05:36:44,784] jepsen worker 84 - jepsen.util 264 :invoke :write [28 7] INFO [2022-01-10 05:36:44,790] jepsen worker 10 - jepsen.util 190 :info :write [3 28] :RequestTimeoutException INFO [2022-01-10 05:36:44,791] jepsen worker 10 - jepsen.util 280 :invoke :read [3 nil] INFO [2022-01-10 05:36:44,795] jepsen worker 40 - jepsen.util 220 :ok :read [13 32] INFO [2022-01-10 05:36:44,796] jepsen worker 40 - jepsen.util 220 :invoke :read [13 nil] INFO [2022-01-10 05:36:44,808] jepsen worker 83 - jepsen.util 263 :ok :read [27 30] INFO [2022-01-10 05:36:44,809] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 05:36:44,851] jepsen worker 58 - jepsen.util 238 :ok :read [19 15] INFO [2022-01-10 05:36:44,852] jepsen worker 58 - jepsen.util 238 :invoke :read [19 nil] INFO [2022-01-10 05:36:45,516] jepsen worker 5 - jepsen.util 185 :info :write [1 7] :RequestTimeoutException INFO [2022-01-10 05:36:45,517] jepsen worker 5 - jepsen.util 275 :invoke :read [1 nil] INFO [2022-01-10 05:36:45,545] jepsen worker 73 - jepsen.util 163 :ok :read [24 33] INFO [2022-01-10 05:36:45,546] jepsen worker 73 - jepsen.util 163 :invoke :read [24 nil] INFO [2022-01-10 05:36:45,551] jepsen worker 34 - jepsen.util 214 :ok :read [11 21] INFO [2022-01-10 05:36:45,552] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 05:36:45,721] jepsen worker 73 - jepsen.util 163 :ok :read [24 33] INFO [2022-01-10 05:36:45,722] jepsen worker 73 - jepsen.util 163 :invoke :write [24 8] INFO [2022-01-10 05:36:45,737] jepsen worker 34 - jepsen.util 214 :ok :read [11 21] INFO [2022-01-10 05:36:45,737] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 05:36:45,745] jepsen worker 23 - jepsen.util 203 :ok :read [7 :nil] INFO [2022-01-10 05:36:45,745] jepsen worker 23 - jepsen.util 203 :invoke :write [7 39] INFO [2022-01-10 05:36:45,773] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][1198us] Node disconnected {"coreId":"0xefe9906200000003","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:36:45,805] jepsen worker 15 - jepsen.util 195 :info :write [5 21] :RequestTimeoutException INFO [2022-01-10 05:36:45,805] jepsen worker 15 - jepsen.util 285 :invoke :read [5 nil] INFO [2022-01-10 05:36:45,807] jepsen worker 34 - jepsen.util 214 :ok :read [11 21] INFO [2022-01-10 05:36:45,808] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 05:36:45,890] jepsen worker 67 - jepsen.util 157 :info :write [22 6] :SyncWriteAmbiguous INFO [2022-01-10 05:36:45,890] jepsen worker 67 - jepsen.util 247 :invoke :read [22 nil] INFO [2022-01-10 05:36:45,938] jepsen worker 34 - jepsen.util 214 :ok :read [11 21] INFO [2022-01-10 05:36:45,939] jepsen worker 34 - jepsen.util 214 :invoke :write [11 16] INFO [2022-01-10 05:36:46,006] jepsen worker 32 - jepsen.util 212 :info :write [10 45] :RequestTimeoutException INFO [2022-01-10 05:36:46,007] jepsen worker 32 - jepsen.util 302 :invoke :read [10 nil] INFO [2022-01-10 05:36:46,069] jepsen worker 32 - jepsen.util 302 :ok :read [10 36] INFO [2022-01-10 05:36:46,069] jepsen worker 32 - jepsen.util 302 :invoke :read [10 nil] INFO [2022-01-10 05:36:46,096] jepsen worker 16 - jepsen.util 106 :info :write [5 13] :RequestTimeoutException INFO [2022-01-10 05:36:46,096] jepsen worker 16 - jepsen.util 196 :invoke :read [5 nil] INFO [2022-01-10 05:36:46,141] jepsen worker 32 - jepsen.util 302 :ok :read [10 36] INFO [2022-01-10 05:36:46,142] jepsen worker 32 - jepsen.util 302 :invoke :read [10 nil] INFO [2022-01-10 05:36:46,212] jepsen worker 32 - jepsen.util 302 :ok :read [10 36] INFO [2022-01-10 05:36:46,213] jepsen worker 32 - jepsen.util 302 :invoke :write [10 32] INFO [2022-01-10 05:36:46,219] jepsen worker 27 - jepsen.util 117 :info :write [9 49] :RequestTimeoutException INFO [2022-01-10 05:36:46,221] jepsen worker 27 - jepsen.util 207 :invoke :read [9 nil] INFO [2022-01-10 05:36:46,310] jepsen worker 31 - jepsen.util 211 :info :write [10 23] :SyncWriteAmbiguous INFO [2022-01-10 05:36:46,311] jepsen worker 31 - jepsen.util 301 :invoke :write [10 38] INFO [2022-01-10 05:36:46,411] jepsen worker 57 - jepsen.util 57 :info :write [19 10] :RequestTimeoutException INFO [2022-01-10 05:36:46,412] jepsen worker 57 - jepsen.util 147 :invoke :write [19 31] INFO [2022-01-10 05:36:46,671] jepsen worker 88 - jepsen.util 268 :info :write [29 33] :RequestTimeoutException INFO [2022-01-10 05:36:46,671] jepsen worker 88 - jepsen.util 358 :invoke :read [29 nil] INFO [2022-01-10 05:36:46,776] jepsen worker 14 - jepsen.util 194 :info :write [4 29] :RequestTimeoutException INFO [2022-01-10 05:36:46,777] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 05:36:46,823] jepsen worker 42 - jepsen.util 132 :ok :read [14 14] INFO [2022-01-10 05:36:46,824] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 05:36:46,881] jepsen worker 14 - jepsen.util 284 :ok :read [4 :nil] INFO [2022-01-10 05:36:46,881] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 05:36:46,930] jepsen worker 14 - jepsen.util 284 :ok :read [4 :nil] INFO [2022-01-10 05:36:46,930] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 05:36:46,987] jepsen worker 41 - jepsen.util 131 :info :write [13 23] :RequestTimeoutException INFO [2022-01-10 05:36:46,987] jepsen worker 41 - jepsen.util 221 :invoke :write [13 7] INFO [2022-01-10 05:36:47,050] jepsen worker 14 - jepsen.util 284 :ok :read [4 :nil] INFO [2022-01-10 05:36:47,050] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 05:36:47,066] jepsen worker 55 - jepsen.util 235 :info :write [18 25] :RequestTimeoutException INFO [2022-01-10 05:36:47,067] jepsen worker 55 - jepsen.util 325 :invoke :read [18 nil] INFO [2022-01-10 05:36:47,090] jepsen worker 14 - jepsen.util 284 :ok :read [4 :nil] INFO [2022-01-10 05:36:47,090] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 05:36:47,129] jepsen worker 14 - jepsen.util 284 :ok :read [4 :nil] INFO [2022-01-10 05:36:47,130] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] WARN [2022-01-10 05:36:47,349] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][UnexpectedEndpointDisconnectedEvent] The remote side disconnected the endpoint unexpectedly {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","type":"KV"} WARN [2022-01-10 05:36:47,359] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][UnexpectedEndpointDisconnectedEvent] The remote side disconnected the endpoint unexpectedly {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","type":"KV"} WARN [2022-01-10 05:36:47,361] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][5831us] Connect attempt 1 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,363] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][6299us] Connect attempt 1 failed because of : finishConnect(..) failed: Connection reset by peer: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection reset by peer: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,364] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][3063us] Connect attempt 2 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,364] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][UnexpectedEndpointDisconnectedEvent] The remote side disconnected the endpoint unexpectedly {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","type":"KV"} WARN [2022-01-10 05:36:47,364] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][UnexpectedEndpointDisconnectedEvent] The remote side disconnected the endpoint unexpectedly {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","type":"KV"} WARN [2022-01-10 05:36:47,365] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][6326us] Connect attempt 1 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,365] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][5096us] Connect attempt 1 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,366] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][2868us] Connect attempt 2 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,366] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][4965us] Connect attempt 2 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,367] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][UnexpectedEndpointDisconnectedEvent] The remote side disconnected the endpoint unexpectedly {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","type":"KV"} WARN [2022-01-10 05:36:47,368] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][6344us] Connect attempt 1 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,369] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][3750us] Connect attempt 2 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. INFO [2022-01-10 05:36:47,382] jepsen worker 72 - jepsen.util 252 :info :write [24 49] :RequestTimeoutException INFO [2022-01-10 05:36:47,382] jepsen worker 72 - jepsen.util 342 :invoke :write [24 36] INFO [2022-01-10 05:36:47,412] jepsen worker 24 - jepsen.util 114 :info :write [8 16] :RequestTimeoutException INFO [2022-01-10 05:36:47,413] jepsen worker 24 - jepsen.util 204 :invoke :read [8 nil] WARN [2022-01-10 05:36:47,463] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][14ms] Connect attempt 2 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,464] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][1801us] Connect attempt 3 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,465] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][2502us] Connect attempt 3 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,468] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][2221us] Connect attempt 3 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,469] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][1811us] Connect attempt 3 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. WARN [2022-01-10 05:36:47,470] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][3405us] Connect attempt 3 failed because of : finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: /172.23.105.187:11210 - Check server ports and cluster encryption setting. INFO [2022-01-10 05:36:47,645] jepsen worker 45 - jepsen.util 135 :info :write [15 25] :RequestTimeoutException INFO [2022-01-10 05:36:47,645] jepsen worker 47 - jepsen.util 137 :info :write [15 23] :RequestTimeoutException INFO [2022-01-10 05:36:47,645] jepsen worker 47 - jepsen.util 227 :invoke :write [15 21] INFO [2022-01-10 05:36:47,646] jepsen worker 45 - jepsen.util 225 :invoke :read [15 nil] ERROR [2022-01-10 05:36:47,765] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][165ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"cdf94e43-71ef-450b-633b-8340bae54772"}} ERROR [2022-01-10 05:36:47,766] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][164ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"704d0497-121b-400c-8637-1bf09e58851b"}} ERROR [2022-01-10 05:36:47,769] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][144ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"2c46b9c5-6d74-4ae8-b4bf-3e7893527bae"}} WARN [2022-01-10 05:36:47,770] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][157ms] Connect attempt 4 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"2c46b9c5-6d74-4ae8-b4bf-3e7893527bae"}} {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"2c46b9c5-6d74-4ae8-b4bf-3e7893527bae"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:47,769] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][168ms] Connect attempt 4 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"cdf94e43-71ef-450b-633b-8340bae54772"}} {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"cdf94e43-71ef-450b-633b-8340bae54772"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:47,769] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][171ms] Connect attempt 4 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"704d0497-121b-400c-8637-1bf09e58851b"}} {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"704d0497-121b-400c-8637-1bf09e58851b"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) ERROR [2022-01-10 05:36:47,770] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][153ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"b5a45422-7439-4626-4fc4-385676e2fead"}} ERROR [2022-01-10 05:36:47,771] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][159ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"ffa2bdf7-24c3-40a3-9e1f-62911238b452"}} WARN [2022-01-10 05:36:47,772] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][156ms] Connect attempt 4 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"b5a45422-7439-4626-4fc4-385676e2fead"}} {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"b5a45422-7439-4626-4fc4-385676e2fead"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:47,772] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][162ms] Connect attempt 4 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"ffa2bdf7-24c3-40a3-9e1f-62911238b452"}} {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"ffa2bdf7-24c3-40a3-9e1f-62911238b452"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) INFO [2022-01-10 05:36:47,793] jepsen worker 22 - jepsen.util 112 :info :write [7 16] :RequestTimeoutException INFO [2022-01-10 05:36:47,793] jepsen worker 22 - jepsen.util 202 :invoke :read [7 nil] INFO [2022-01-10 05:36:47,812] jepsen worker 81 - jepsen.util 261 :info :write [27 0] :RequestTimeoutException INFO [2022-01-10 05:36:47,813] jepsen worker 43 - jepsen.util 223 :info :write [14 42] :RequestTimeoutException INFO [2022-01-10 05:36:47,813] jepsen worker 81 - jepsen.util 351 :invoke :write [27 1] INFO [2022-01-10 05:36:47,813] jepsen worker 43 - jepsen.util 313 :invoke :read [14 nil] ERROR [2022-01-10 05:36:48,071] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][50ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"a3d2f92f-86a9-4cec-ce5b-4377b4a7daa9"}} ERROR [2022-01-10 05:36:48,072] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][52ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"67aba15c-b0d9-4413-d10a-749548311fa4"}} ERROR [2022-01-10 05:36:48,072] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][57ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"92a77317-caf5-4a01-0b48-41144d50d247"}} ERROR [2022-01-10 05:36:48,073] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][69ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"36ab91b2-5f6c-4a86-b1dc-f41f0a4591f5"}} ERROR [2022-01-10 05:36:48,073] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][57ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"4c5bbd04-653d-43da-6792-9d58048f5a8f"}} WARN [2022-01-10 05:36:48,075] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][55ms] Connect attempt 5 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"67aba15c-b0d9-4413-d10a-749548311fa4"}} {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"67aba15c-b0d9-4413-d10a-749548311fa4"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:48,075] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][60ms] Connect attempt 5 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"92a77317-caf5-4a01-0b48-41144d50d247"}} {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"92a77317-caf5-4a01-0b48-41144d50d247"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:48,075] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][75ms] Connect attempt 5 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"36ab91b2-5f6c-4a86-b1dc-f41f0a4591f5"}} {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"36ab91b2-5f6c-4a86-b1dc-f41f0a4591f5"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:48,075] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][55ms] Connect attempt 5 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"a3d2f92f-86a9-4cec-ce5b-4377b4a7daa9"}} {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"a3d2f92f-86a9-4cec-ce5b-4377b4a7daa9"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:48,076] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][59ms] Connect attempt 5 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"4c5bbd04-653d-43da-6792-9d58048f5a8f"}} {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"4c5bbd04-653d-43da-6792-9d58048f5a8f"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) INFO [2022-01-10 05:36:48,106] jepsen worker 18 - jepsen.util 198 :info :write [6 35] :RequestTimeoutException INFO [2022-01-10 05:36:48,107] jepsen worker 18 - jepsen.util 288 :invoke :read [6 nil] INFO [2022-01-10 05:36:48,121] jepsen worker 63 - jepsen.util 243 :info :write [21 11] :RequestTimeoutException INFO [2022-01-10 05:36:48,122] jepsen worker 63 - jepsen.util 333 :invoke :read [21 nil] INFO [2022-01-10 05:36:48,156] jepsen worker 50 - jepsen.util 230 :info :write [16 32] :RequestTimeoutException INFO [2022-01-10 05:36:48,156] jepsen worker 50 - jepsen.util 320 :invoke :read [16 nil] INFO [2022-01-10 05:36:48,292] jepsen worker 46 - jepsen.util 136 :info :write [15 11] :RequestTimeoutException INFO [2022-01-10 05:36:48,292] jepsen worker 62 - jepsen.util 242 :info :write [20 33] :RequestTimeoutException INFO [2022-01-10 05:36:48,293] jepsen worker 46 - jepsen.util 226 :invoke :read [15 nil] INFO [2022-01-10 05:36:48,293] jepsen worker 62 - jepsen.util 332 :invoke :read [20 nil] INFO [2022-01-10 05:36:48,516] jepsen worker 80 - jepsen.util 260 :info :write [26 44] :RequestTimeoutException INFO [2022-01-10 05:36:48,517] jepsen worker 80 - jepsen.util 350 :invoke :read [26 nil] ERROR [2022-01-10 05:36:48,676] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][48ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"c4aa0f35-7938-4d71-48a7-498aee575175"}} ERROR [2022-01-10 05:36:48,676] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][47ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"80bb5e6d-737c-4c85-4582-d60addf4a936"}} ERROR [2022-01-10 05:36:48,677] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][52ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"be77cccf-52d6-4807-82b1-cd45f7f8cf6d"}} WARN [2022-01-10 05:36:48,677] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][53ms] Connect attempt 6 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"80bb5e6d-737c-4c85-4582-d60addf4a936"}} {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"80bb5e6d-737c-4c85-4582-d60addf4a936"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) ERROR [2022-01-10 05:36:48,678] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][61ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"4715c9c7-4b84-49c3-3722-a8386d5e886e"}} ERROR [2022-01-10 05:36:48,678] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][59ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"47575867-0988-4323-202a-f390ca770994"}} WARN [2022-01-10 05:36:48,678] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][50ms] Connect attempt 6 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"c4aa0f35-7938-4d71-48a7-498aee575175"}} {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"c4aa0f35-7938-4d71-48a7-498aee575175"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:48,679] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][55ms] Connect attempt 6 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"be77cccf-52d6-4807-82b1-cd45f7f8cf6d"}} {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"be77cccf-52d6-4807-82b1-cd45f7f8cf6d"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:48,679] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][64ms] Connect attempt 6 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"4715c9c7-4b84-49c3-3722-a8386d5e886e"}} {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"4715c9c7-4b84-49c3-3722-a8386d5e886e"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:48,680] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][62ms] Connect attempt 6 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"47575867-0988-4323-202a-f390ca770994"}} {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"47575867-0988-4323-202a-f390ca770994"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) INFO [2022-01-10 05:36:48,687] jepsen worker 62 - jepsen.util 332 :ok :read [20 :nil] INFO [2022-01-10 05:36:48,688] jepsen worker 62 - jepsen.util 332 :invoke :write [20 18] INFO [2022-01-10 05:36:48,742] jepsen worker 89 - jepsen.util 269 :info :write [29 24] :RequestTimeoutException INFO [2022-01-10 05:36:48,743] jepsen worker 89 - jepsen.util 359 :invoke :write [29 29] INFO [2022-01-10 05:36:48,744] jepsen worker 45 - jepsen.util 225 :ok :read [15 :nil] INFO [2022-01-10 05:36:48,744] jepsen worker 45 - jepsen.util 225 :invoke :write [15 17] INFO [2022-01-10 05:36:48,795] jepsen worker 56 - jepsen.util 236 :info :write [18 8] :RequestTimeoutException INFO [2022-01-10 05:36:48,796] jepsen worker 56 - jepsen.util 326 :invoke :write [18 33] INFO [2022-01-10 05:36:48,850] jepsen worker 65 - jepsen.util 245 :info :write [21 29] :RequestTimeoutException INFO [2022-01-10 05:36:48,850] jepsen worker 65 - jepsen.util 335 :invoke :read [21 nil] INFO [2022-01-10 05:36:48,902] jepsen worker 44 - jepsen.util 224 :info :write [14 28] :RequestTimeoutException INFO [2022-01-10 05:36:48,903] jepsen worker 44 - jepsen.util 314 :invoke :write [14 30] INFO [2022-01-10 05:36:49,097] jepsen worker 11 - jepsen.util 101 :info :write [3 11] :RequestTimeoutException INFO [2022-01-10 05:36:49,097] jepsen worker 11 - jepsen.util 191 :invoke :read [3 nil] INFO [2022-01-10 05:36:49,135] jepsen worker 19 - jepsen.util 199 :info :write [6 11] :RequestTimeoutException INFO [2022-01-10 05:36:49,136] jepsen worker 19 - jepsen.util 289 :invoke :write [6 5] INFO [2022-01-10 05:36:49,157] jepsen worker 87 - jepsen.util 177 :info :write [29 23] :RequestTimeoutException INFO [2022-01-10 05:36:49,158] jepsen worker 87 - jepsen.util 267 :invoke :write [29 7] INFO [2022-01-10 05:36:49,177] jepsen worker 53 - jepsen.util 143 :info :write [17 20] :RequestTimeoutException INFO [2022-01-10 05:36:49,178] jepsen worker 53 - jepsen.util 233 :invoke :write [17 44] INFO [2022-01-10 05:36:49,300] jepsen worker 52 - jepsen.util 232 :info :write [17 45] :RequestTimeoutException INFO [2022-01-10 05:36:49,301] jepsen worker 52 - jepsen.util 322 :invoke :read [17 nil] INFO [2022-01-10 05:36:49,319] jepsen worker 7 - jepsen.util 97 :ok :read [2 21] INFO [2022-01-10 05:36:49,320] jepsen worker 7 - jepsen.util 97 :invoke :read [2 nil] INFO [2022-01-10 05:36:49,346] jepsen worker 22 - jepsen.util 202 :ok :read [7 :nil] INFO [2022-01-10 05:36:49,346] jepsen worker 22 - jepsen.util 202 :invoke :write [7 42] INFO [2022-01-10 05:36:49,347] jepsen worker 46 - jepsen.util 226 :ok :read [15 :nil] INFO [2022-01-10 05:36:49,347] jepsen worker 46 - jepsen.util 226 :invoke :read [15 nil] INFO [2022-01-10 05:36:49,356] jepsen worker 42 - jepsen.util 132 :ok :read [14 14] INFO [2022-01-10 05:36:49,357] jepsen worker 42 - jepsen.util 132 :invoke :write [14 31] INFO [2022-01-10 05:36:49,359] jepsen worker 63 - jepsen.util 333 :ok :read [21 7] INFO [2022-01-10 05:36:49,359] jepsen worker 63 - jepsen.util 333 :invoke :read [21 nil] INFO [2022-01-10 05:36:49,382] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][89us] Node disconnected {"coreId":"0xefe9906200000002","managerPort":"8091","remote":"172.23.105.187"} WARN [2022-01-10 05:36:49,479] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11562110,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11562110,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9158705,"encode_duration_us":17061,"operation_id":"0x97e","timeout_ms":10000,"last_server_duration_us":9158705,"total_duration_us":10004851}],"total_count":1}} WARN [2022-01-10 05:36:49,480] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2710618,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":3126854,"total_server_duration_us":209894,"encode_duration_us":17457,"operation_id":"0x974","timeout_ms":10000,"last_server_duration_us":60885,"total_duration_us":10010318},{"operation_name":"upsert","last_dispatch_duration_us":3571242,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3571242,"total_server_duration_us":32355,"encode_duration_us":16508,"operation_id":"0xa55","timeout_ms":10000,"last_server_duration_us":32355,"total_duration_us":10009670},{"operation_name":"upsert","last_dispatch_duration_us":11979368,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12505138,"total_server_duration_us":95199,"encode_duration_us":16807,"operation_id":"0x9bd","timeout_ms":10000,"last_server_duration_us":25450,"total_duration_us":10009586},{"operation_name":"upsert","last_dispatch_duration_us":2415202,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":2415202,"total_server_duration_us":211909,"encode_duration_us":16958,"operation_id":"0xa44","timeout_ms":10000,"last_server_duration_us":211909,"total_duration_us":10009352},{"operation_name":"upsert","last_dispatch_duration_us":12068654,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":14434567,"total_server_duration_us":200152,"encode_duration_us":16493,"operation_id":"0x901","timeout_ms":10000,"last_server_duration_us":41123,"total_duration_us":10007714},{"operation_name":"upsert","last_dispatch_duration_us":908550,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":1880615,"total_server_duration_us":125379,"encode_duration_us":18018,"operation_id":"0x8c4","timeout_ms":10000,"last_server_duration_us":21895,"total_duration_us":10006935},{"operation_name":"upsert","last_dispatch_duration_us":7962104,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8296946,"total_server_duration_us":83710,"encode_duration_us":20071,"operation_id":"0x913","timeout_ms":10000,"last_server_duration_us":64214,"total_duration_us":10006673},{"operation_name":"upsert","last_dispatch_duration_us":4235195,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8089853,"total_server_duration_us":113689,"encode_duration_us":16793,"operation_id":"0xa0c","timeout_ms":10000,"last_server_duration_us":54939,"total_duration_us":10004981},{"operation_name":"upsert","last_dispatch_duration_us":11562110,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11562110,"total_server_duration_us":9158705,"encode_duration_us":17061,"operation_id":"0x97e","timeout_ms":10000,"last_server_duration_us":9158705,"total_duration_us":10004851},{"operation_name":"upsert","last_dispatch_duration_us":3204405,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8984006,"total_server_duration_us":98348,"encode_duration_us":17802,"operation_id":"0xa0d","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10004850}],"total_count":15}} INFO [2022-01-10 05:36:49,512] jepsen worker 35 - jepsen.util 215 :info :write [11 23] :RequestTimeoutException INFO [2022-01-10 05:36:49,512] jepsen worker 35 - jepsen.util 305 :invoke :write [11 5] INFO [2022-01-10 05:36:49,525] jepsen worker 39 - jepsen.util 129 :info :write [13 47] :RequestTimeoutException INFO [2022-01-10 05:36:49,527] jepsen worker 39 - jepsen.util 219 :invoke :read [13 nil] INFO [2022-01-10 05:36:49,542] jepsen worker 63 - jepsen.util 333 :ok :read [21 7] INFO [2022-01-10 05:36:49,543] jepsen worker 63 - jepsen.util 333 :invoke :read [21 nil] INFO [2022-01-10 05:36:49,617] jepsen worker 46 - jepsen.util 226 :ok :read [15 :nil] INFO [2022-01-10 05:36:49,617] jepsen worker 46 - jepsen.util 226 :invoke :read [15 nil] INFO [2022-01-10 05:36:49,632] jepsen worker 3 - jepsen.util 183 :info :write [1 13] :RequestTimeoutException INFO [2022-01-10 05:36:49,633] jepsen worker 3 - jepsen.util 273 :invoke :write [1 6] ERROR [2022-01-10 05:36:49,682] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][54ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000002/00000000FC7A8DD2","circuitBreaker":"DISABLED","coreId":"0xefe9906200000002","local":"172.23.104.79:43948","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"11077029-83a3-4c83-5523-750ea11403bc"}} INFO [2022-01-10 05:36:49,750] jepsen worker 63 - jepsen.util 333 :ok :read [21 7] INFO [2022-01-10 05:36:49,751] jepsen worker 63 - jepsen.util 333 :invoke :read [21 nil] INFO [2022-01-10 05:36:49,752] jepsen worker 7 - jepsen.util 97 :ok :read [2 21] INFO [2022-01-10 05:36:49,753] jepsen worker 7 - jepsen.util 97 :invoke :write [2 8] ERROR [2022-01-10 05:36:49,779] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][49ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"8fbd02c6-3f6b-4fc8-df4d-a59cb3cf0464"}} ERROR [2022-01-10 05:36:49,781] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][67ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"998b7586-3daf-4b15-c2e3-35502a51d5ad"}} WARN [2022-01-10 05:36:49,781] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][51ms] Connect attempt 7 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"8fbd02c6-3f6b-4fc8-df4d-a59cb3cf0464"}} {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"8fbd02c6-3f6b-4fc8-df4d-a59cb3cf0464"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) ERROR [2022-01-10 05:36:49,781] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][77ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"086eae2b-6a0c-4540-505b-2f79e0bfa8d7"}} WARN [2022-01-10 05:36:49,782] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][69ms] Connect attempt 7 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"998b7586-3daf-4b15-c2e3-35502a51d5ad"}} {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"998b7586-3daf-4b15-c2e3-35502a51d5ad"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:49,782] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][80ms] Connect attempt 7 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"086eae2b-6a0c-4540-505b-2f79e0bfa8d7"}} {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"086eae2b-6a0c-4540-505b-2f79e0bfa8d7"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) ERROR [2022-01-10 05:36:49,784] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][53ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"f28892ce-4841-4064-603d-43977ce9ba95"}} WARN [2022-01-10 05:36:49,785] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][57ms] Connect attempt 7 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"f28892ce-4841-4064-603d-43977ce9ba95"}} {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"f28892ce-4841-4064-603d-43977ce9ba95"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) INFO [2022-01-10 05:36:49,804] jepsen worker 38 - jepsen.util 218 :ok :read [12 :nil] INFO [2022-01-10 05:36:49,804] jepsen worker 38 - jepsen.util 218 :invoke :read [12 nil] INFO [2022-01-10 05:36:49,815] jepsen worker 12 - jepsen.util 192 :ok :read [4 :nil] INFO [2022-01-10 05:36:49,816] jepsen worker 38 - jepsen.util 218 :ok :read [12 :nil] INFO [2022-01-10 05:36:49,816] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 05:36:49,817] jepsen worker 38 - jepsen.util 218 :invoke :read [12 nil] INFO [2022-01-10 05:36:49,820] jepsen worker 77 - jepsen.util 257 :info :write [25 38] :RequestTimeoutException INFO [2022-01-10 05:36:49,821] jepsen worker 12 - jepsen.util 192 :ok :read [4 :nil] INFO [2022-01-10 05:36:49,821] jepsen worker 38 - jepsen.util 218 :ok :read [12 :nil] INFO [2022-01-10 05:36:49,822] jepsen worker 77 - jepsen.util 347 :invoke :write [25 15] INFO [2022-01-10 05:36:49,823] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 05:36:49,823] jepsen worker 38 - jepsen.util 218 :invoke :read [12 nil] INFO [2022-01-10 05:36:49,825] jepsen worker 12 - jepsen.util 192 :ok :read [4 :nil] INFO [2022-01-10 05:36:49,825] jepsen worker 38 - jepsen.util 218 :ok :read [12 :nil] INFO [2022-01-10 05:36:49,826] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 05:36:49,826] jepsen worker 38 - jepsen.util 218 :invoke :write [12 34] INFO [2022-01-10 05:36:49,828] jepsen worker 12 - jepsen.util 192 :ok :read [4 :nil] INFO [2022-01-10 05:36:49,828] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 05:36:49,832] jepsen worker 12 - jepsen.util 192 :ok :read [4 :nil] INFO [2022-01-10 05:36:49,832] jepsen worker 74 - jepsen.util 254 :info :write [24 49] :RequestTimeoutException INFO [2022-01-10 05:36:49,833] jepsen worker 12 - jepsen.util 192 :invoke :write [4 13] INFO [2022-01-10 05:36:49,834] jepsen worker 74 - jepsen.util 344 :invoke :read [24 nil] INFO [2022-01-10 05:36:49,837] jepsen worker 4 - jepsen.util 184 :ok :read [1 41] INFO [2022-01-10 05:36:49,838] jepsen worker 4 - jepsen.util 184 :invoke :read [1 nil] INFO [2022-01-10 05:36:49,864] jepsen worker 4 - jepsen.util 184 :ok :read [1 41] INFO [2022-01-10 05:36:49,865] jepsen worker 4 - jepsen.util 184 :invoke :read [1 nil] INFO [2022-01-10 05:36:49,892] jepsen worker 43 - jepsen.util 313 :ok :read [14 14] INFO [2022-01-10 05:36:49,893] jepsen worker 43 - jepsen.util 313 :invoke :read [14 nil] INFO [2022-01-10 05:36:49,916] jepsen worker 69 - jepsen.util 249 :info :write [23 37] :RequestTimeoutException INFO [2022-01-10 05:36:49,917] jepsen worker 69 - jepsen.util 339 :invoke :write [23 36] INFO [2022-01-10 05:36:49,931] jepsen worker 60 - jepsen.util 240 :info :write [20 30] :RequestTimeoutException INFO [2022-01-10 05:36:49,932] jepsen worker 60 - jepsen.util 330 :invoke :read [20 nil] INFO [2022-01-10 05:36:49,969] jepsen worker 63 - jepsen.util 333 :ok :read [21 7] INFO [2022-01-10 05:36:49,971] jepsen worker 63 - jepsen.util 333 :invoke :write [21 22] INFO [2022-01-10 05:36:49,977] jepsen worker 4 - jepsen.util 184 :ok :read [1 41] INFO [2022-01-10 05:36:49,977] jepsen worker 4 - jepsen.util 184 :invoke :write [1 0] INFO [2022-01-10 05:36:50,020] jepsen worker 0 - jepsen.util 180 :info :write [0 14] :RequestTimeoutException INFO [2022-01-10 05:36:50,021] jepsen worker 0 - jepsen.util 270 :invoke :read [0 nil] INFO [2022-01-10 05:36:50,092] jepsen worker 86 - jepsen.util 266 :info :write [28 41] :RequestTimeoutException INFO [2022-01-10 05:36:50,093] jepsen worker 86 - jepsen.util 356 :invoke :read [28 nil] INFO [2022-01-10 05:36:50,099] jepsen worker 8 - jepsen.util 188 :ok :read [2 21] INFO [2022-01-10 05:36:50,100] jepsen worker 8 - jepsen.util 188 :invoke :write [2 7] INFO [2022-01-10 05:36:50,225] jepsen worker 13 - jepsen.util 193 :info :write [4 34] :RequestTimeoutException INFO [2022-01-10 05:36:50,226] jepsen worker 13 - jepsen.util 283 :invoke :read [4 nil] INFO [2022-01-10 05:36:50,302] jepsen worker 9 - jepsen.util 189 :info :write [3 2] :RequestTimeoutException INFO [2022-01-10 05:36:50,303] jepsen worker 9 - jepsen.util 279 :invoke :write [3 12] INFO [2022-01-10 05:36:50,332] jepsen worker 75 - jepsen.util 255 :info :write [25 13] :RequestTimeoutException INFO [2022-01-10 05:36:50,332] jepsen worker 75 - jepsen.util 345 :invoke :read [25 nil] INFO [2022-01-10 05:36:50,337] jepsen worker 79 - jepsen.util 259 :info :write [26 37] :RequestTimeoutException INFO [2022-01-10 05:36:50,338] jepsen worker 79 - jepsen.util 349 :invoke :read [26 nil] INFO [2022-01-10 05:36:50,365] jepsen worker 49 - jepsen.util 229 :info :write [16 24] :RequestTimeoutException INFO [2022-01-10 05:36:50,366] jepsen worker 49 - jepsen.util 319 :invoke :read [16 nil] INFO [2022-01-10 05:36:50,396] jepsen worker 26 - jepsen.util 206 :info :write [8 46] :RequestTimeoutException INFO [2022-01-10 05:36:50,397] jepsen worker 26 - jepsen.util 296 :invoke :read [8 nil] INFO [2022-01-10 05:36:50,430] jepsen worker 61 - jepsen.util 241 :info :write [20 33] :RequestTimeoutException INFO [2022-01-10 05:36:50,431] jepsen worker 61 - jepsen.util 331 :invoke :read [20 nil] INFO [2022-01-10 05:36:50,477] jepsen worker 74 - jepsen.util 344 :ok :read [24 33] INFO [2022-01-10 05:36:50,477] jepsen worker 74 - jepsen.util 344 :invoke :read [24 nil] INFO [2022-01-10 05:36:50,507] jepsen worker 37 - jepsen.util 217 :info :write [12 7] :RequestTimeoutException INFO [2022-01-10 05:36:50,507] jepsen worker 37 - jepsen.util 307 :invoke :write [12 43] INFO [2022-01-10 05:36:50,535] jepsen worker 1 - jepsen.util 181 :info :write [0 21] :RequestTimeoutException INFO [2022-01-10 05:36:50,535] jepsen worker 1 - jepsen.util 271 :invoke :read [0 nil] INFO [2022-01-10 05:36:50,622] jepsen worker 74 - jepsen.util 344 :ok :read [24 33] INFO [2022-01-10 05:36:50,623] jepsen worker 74 - jepsen.util 344 :invoke :write [24 40] INFO [2022-01-10 05:36:50,750] jepsen worker 78 - jepsen.util 258 :info :write [26 6] :RequestTimeoutException INFO [2022-01-10 05:36:50,751] jepsen worker 78 - jepsen.util 348 :invoke :read [26 nil] INFO [2022-01-10 05:36:50,945] jepsen worker 60 - jepsen.util 330 :ok :read [20 :nil] INFO [2022-01-10 05:36:50,946] jepsen worker 60 - jepsen.util 330 :invoke :write [20 42] WARN [2022-01-10 05:36:50,982] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9352769,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15363014,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9074569,"encode_duration_us":17479,"operation_id":"0x915","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10003291}],"total_count":1}} WARN [2022-01-10 05:36:50,983] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":13996568,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13996568,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9151235,"encode_duration_us":75063,"operation_id":"0x751","timeout_ms":10000,"last_server_duration_us":9151235,"total_duration_us":10003716}],"total_count":1}} WARN [2022-01-10 05:36:50,983] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":2680323,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":2680323,"total_server_duration_us":19,"operation_id":"0x909","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":14478701},{"operation_name":"get","last_dispatch_duration_us":2561027,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":2561027,"total_server_duration_us":23,"operation_id":"0x911","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":14363800},{"operation_name":"get","last_dispatch_duration_us":4558982,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":4558982,"total_server_duration_us":32,"operation_id":"0x8ff","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":12799275},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":25742,"operation_id":"0xa24","timeout_ms":10000,"total_duration_us":10008159},{"operation_name":"upsert","last_dispatch_duration_us":2507494,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":2507494,"total_server_duration_us":58004,"encode_duration_us":16466,"operation_id":"0xa40","timeout_ms":10000,"last_server_duration_us":58004,"total_duration_us":10007144},{"operation_name":"upsert","last_dispatch_duration_us":186155,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":186155,"total_server_duration_us":49835,"encode_duration_us":16452,"operation_id":"0xa7b","timeout_ms":10000,"last_server_duration_us":49835,"total_duration_us":10006492},{"operation_name":"upsert","last_dispatch_duration_us":11163561,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11163561,"total_server_duration_us":21568,"encode_duration_us":16867,"operation_id":"0x959","timeout_ms":10000,"last_server_duration_us":21568,"total_duration_us":10006061},{"operation_name":"upsert","last_dispatch_duration_us":4741244,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5559650,"total_server_duration_us":113104,"encode_duration_us":16456,"operation_id":"0xa09","timeout_ms":10000,"last_server_duration_us":28952,"total_duration_us":10006023},{"operation_name":"upsert","last_dispatch_duration_us":3181113,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3181113,"total_server_duration_us":44279,"encode_duration_us":20445,"operation_id":"0xa2a","timeout_ms":10000,"last_server_duration_us":44279,"total_duration_us":10005119},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":17376,"operation_id":"0x92f","timeout_ms":10000,"total_duration_us":10004302}],"total_count":19}} WARN [2022-01-10 05:36:50,985] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":20379195,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":23760547,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9664903,"encode_duration_us":154126,"operation_id":"0x72e","timeout_ms":10000,"last_server_duration_us":9206791,"total_duration_us":10004850},{"operation_name":"upsert","last_dispatch_duration_us":18121160,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18931989,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9514080,"encode_duration_us":19957,"operation_id":"0x922","timeout_ms":10000,"last_server_duration_us":9472635,"total_duration_us":10006311}],"total_count":2}} WARN [2022-01-10 05:36:50,987] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","encode_duration_us":19850,"operation_id":"0xa89","timeout_ms":10000,"total_duration_us":10009664},{"operation_name":"upsert","last_dispatch_duration_us":12672405,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12672405,"total_server_duration_us":21568,"encode_duration_us":18540,"operation_id":"0x9ed","timeout_ms":10000,"last_server_duration_us":21568,"total_duration_us":10009416},{"operation_name":"upsert","encode_duration_us":16368,"operation_id":"0xa5a","timeout_ms":10000,"total_duration_us":10008679},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":20036,"operation_id":"0xa59","timeout_ms":10000,"total_duration_us":10007941},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":16230,"operation_id":"0xa36","timeout_ms":10000,"total_duration_us":10006988},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":18440,"operation_id":"0xa54","timeout_ms":10000,"total_duration_us":10005907},{"operation_name":"upsert","encode_duration_us":16760,"operation_id":"0xa7a","timeout_ms":10000,"total_duration_us":10005378},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":16998,"operation_id":"0xa90","timeout_ms":10000,"total_duration_us":10003030},{"operation_name":"upsert","last_dispatch_duration_us":36491,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":416717,"total_server_duration_us":213667,"encode_duration_us":17367,"operation_id":"0x9fb","timeout_ms":10000,"last_server_duration_us":24069,"total_duration_us":10002943},{"operation_name":"upsert","last_dispatch_duration_us":13534931,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":13534931,"total_server_duration_us":23729,"encode_duration_us":16359,"operation_id":"0x9ac","timeout_ms":10000,"last_server_duration_us":23729,"total_duration_us":10001187}],"total_count":11}} WARN [2022-01-10 05:36:51,084] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":379398,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":379398,"total_server_duration_us":56,"operation_id":"0xa35","timeout_ms":30000,"last_server_duration_us":56,"total_duration_us":11669951},{"operation_name":"upsert","last_dispatch_duration_us":10235193,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10235193,"total_server_duration_us":20282,"encode_duration_us":17204,"operation_id":"0xa50","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10010583},{"operation_name":"upsert","last_dispatch_duration_us":4278318,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":4278318,"total_server_duration_us":34711,"encode_duration_us":18856,"operation_id":"0xa23","timeout_ms":10000,"last_server_duration_us":34711,"total_duration_us":10009016},{"operation_name":"upsert","encode_duration_us":17357,"operation_id":"0xa33","timeout_ms":10000,"total_duration_us":10008975},{"operation_name":"upsert","encode_duration_us":18752,"operation_id":"0xa34","timeout_ms":10000,"total_duration_us":10007529},{"operation_name":"upsert","last_dispatch_duration_us":1347522,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":1347522,"total_server_duration_us":821015,"encode_duration_us":16225,"operation_id":"0xa88","timeout_ms":10000,"last_server_duration_us":821015,"total_duration_us":10007395},{"operation_name":"upsert","last_dispatch_duration_us":852997,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":852997,"total_server_duration_us":31489,"encode_duration_us":21380,"operation_id":"0xa0b","timeout_ms":10000,"last_server_duration_us":31489,"total_duration_us":10006326},{"operation_name":"upsert","last_dispatch_duration_us":10184510,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10184510,"total_server_duration_us":9395889,"encode_duration_us":16525,"operation_id":"0xa62","timeout_ms":10000,"last_server_duration_us":9395889,"total_duration_us":10006222},{"operation_name":"upsert","last_dispatch_duration_us":6046216,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8507390,"total_server_duration_us":46405,"encode_duration_us":17553,"operation_id":"0xa1c","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10005162},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":19562,"operation_id":"0xa80","timeout_ms":10000,"total_duration_us":10004449}],"total_count":14}} WARN [2022-01-10 05:36:51,085] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":7926939,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":7926939,"total_server_duration_us":23,"operation_id":"0xa5b","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":10545695},{"operation_name":"get","last_dispatch_duration_us":7626689,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":7626689,"total_server_duration_us":19,"operation_id":"0xa61","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":10241412},{"operation_name":"upsert","last_dispatch_duration_us":10632544,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10632544,"total_server_duration_us":21813,"encode_duration_us":17317,"operation_id":"0x9fe","timeout_ms":10000,"last_server_duration_us":21813,"total_duration_us":10010071},{"operation_name":"upsert","last_dispatch_duration_us":13771355,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":13771355,"total_server_duration_us":18876,"encode_duration_us":17166,"operation_id":"0x9c4","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10009685},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":18077,"operation_id":"0xa31","timeout_ms":10000,"total_duration_us":10008633},{"operation_name":"upsert","last_dispatch_duration_us":347225,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9215972,"total_server_duration_us":49102,"encode_duration_us":16557,"operation_id":"0xa8c","timeout_ms":10000,"last_server_duration_us":25712,"total_duration_us":10007396},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":22075,"operation_id":"0x9c5","timeout_ms":10000,"total_duration_us":10003353},{"operation_name":"upsert","last_dispatch_duration_us":8891469,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11035724,"total_server_duration_us":38994,"encode_duration_us":18589,"operation_id":"0xa29","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10001106},{"operation_name":"get","last_dispatch_duration_us":7499150,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":7499150,"total_server_duration_us":15,"operation_id":"0xa85","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":9601845},{"operation_name":"get","last_dispatch_duration_us":7115570,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7115570,"total_server_duration_us":32,"operation_id":"0xa98","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":8711133}],"total_count":14}} INFO [2022-01-10 05:36:51,162] jepsen worker 82 - jepsen.util 172 :ok :read [27 30] INFO [2022-01-10 05:36:51,163] jepsen worker 82 - jepsen.util 172 :invoke :read [27 nil] WARN [2022-01-10 05:36:51,183] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13505864,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":13505864,"total_server_duration_us":181498,"operation_id":"0x8f8","timeout_ms":30000,"last_server_duration_us":181498,"total_duration_us":13508676},{"operation_name":"get","last_dispatch_duration_us":13448043,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":13448043,"total_server_duration_us":23,"operation_id":"0x8f2","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":13448665},{"operation_name":"get","last_dispatch_duration_us":12417377,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12417377,"total_server_duration_us":10451,"operation_id":"0x8b1","timeout_ms":30000,"last_server_duration_us":10451,"total_duration_us":12420062},{"operation_name":"upsert","last_dispatch_duration_us":457921,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":457921,"total_server_duration_us":70690,"encode_duration_us":17875,"operation_id":"0x99e","timeout_ms":10000,"last_server_duration_us":70690,"total_duration_us":10010012},{"operation_name":"upsert","last_dispatch_duration_us":457425,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":5298546,"total_server_duration_us":224508,"encode_duration_us":17303,"operation_id":"0xa5c","timeout_ms":10000,"last_server_duration_us":158208,"total_duration_us":10009747},{"operation_name":"upsert","last_dispatch_duration_us":342944,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":564019,"total_server_duration_us":41363,"encode_duration_us":23923,"operation_id":"0x9cd","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10008670},{"operation_name":"upsert","last_dispatch_duration_us":373382,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2274459,"total_server_duration_us":642750,"encode_duration_us":17469,"operation_id":"0xa8d","timeout_ms":10000,"last_server_duration_us":97638,"total_duration_us":10008421},{"operation_name":"upsert","last_dispatch_duration_us":309658,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":309658,"total_server_duration_us":19730,"encode_duration_us":21823,"operation_id":"0x9cf","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10006070},{"operation_name":"upsert","last_dispatch_duration_us":270816,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5817430,"total_server_duration_us":224404,"encode_duration_us":16622,"operation_id":"0xa78","timeout_ms":10000,"last_server_duration_us":57756,"total_duration_us":10005296},{"operation_name":"upsert","last_dispatch_duration_us":1846909,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":1846909,"total_server_duration_us":243158,"encode_duration_us":17022,"operation_id":"0xaa8","timeout_ms":10000,"last_server_duration_us":243158,"total_duration_us":10005268}],"total_count":16}} WARN [2022-01-10 05:36:51,285] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10184510,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10184510,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9395889,"encode_duration_us":16525,"operation_id":"0xa62","timeout_ms":10000,"last_server_duration_us":9395889,"total_duration_us":10006222},{"operation_name":"upsert","last_dispatch_duration_us":9066714,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14307172,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9989659,"encode_duration_us":17872,"operation_id":"0x912","timeout_ms":10000,"last_server_duration_us":9054355,"total_duration_us":10010466},{"operation_name":"upsert","last_dispatch_duration_us":9171710,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14432199,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9500094,"encode_duration_us":18935,"operation_id":"0x903","timeout_ms":10000,"last_server_duration_us":9157638,"total_duration_us":10008625}],"total_count":3}} WARN [2022-01-10 05:36:51,287] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":19082633,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":20621473,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":10004431,"encode_duration_us":52684,"operation_id":"0x746","timeout_ms":10000,"last_server_duration_us":9443418,"total_duration_us":10006068},{"operation_name":"upsert","last_dispatch_duration_us":9675975,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13575820,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9971714,"encode_duration_us":16911,"operation_id":"0x8b7","timeout_ms":10000,"last_server_duration_us":9326937,"total_duration_us":10006691}],"total_count":2}} INFO [2022-01-10 05:36:51,610] jepsen worker 54 - jepsen.util 144 :ok :read [18 14] INFO [2022-01-10 05:36:51,611] jepsen worker 54 - jepsen.util 144 :invoke :read [18 nil] INFO [2022-01-10 05:36:51,672] jepsen worker 70 - jepsen.util 250 :info :write [23 27] :RequestTimeoutException INFO [2022-01-10 05:36:51,673] jepsen worker 70 - jepsen.util 340 :invoke :write [23 12] INFO [2022-01-10 05:36:51,700] jepsen worker 76 - jepsen.util 256 :ok :read [25 :nil] INFO [2022-01-10 05:36:51,700] jepsen worker 24 - jepsen.util 204 :ok :read [8 :nil] INFO [2022-01-10 05:36:51,700] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 05:36:51,701] jepsen worker 24 - jepsen.util 204 :invoke :write [8 48] INFO [2022-01-10 05:36:51,784] jepsen worker 10 - jepsen.util 280 :ok :read [3 22] INFO [2022-01-10 05:36:51,785] jepsen worker 10 - jepsen.util 280 :invoke :read [3 nil] INFO [2022-01-10 05:36:51,799] jepsen worker 64 - jepsen.util 244 :ok :read [21 7] INFO [2022-01-10 05:36:51,810] jepsen worker 18 - jepsen.util 288 :ok :read [6 25] INFO [2022-01-10 05:36:51,810] jepsen worker 64 - jepsen.util 244 :invoke :read [21 nil] INFO [2022-01-10 05:36:51,810] jepsen worker 18 - jepsen.util 288 :invoke :write [6 17] INFO [2022-01-10 05:36:51,836] jepsen worker 61 - jepsen.util 331 :ok :read [20 :nil] INFO [2022-01-10 05:36:51,836] jepsen worker 76 - jepsen.util 256 :ok :read [25 :nil] INFO [2022-01-10 05:36:51,837] jepsen worker 61 - jepsen.util 331 :invoke :read [20 nil] INFO [2022-01-10 05:36:51,837] jepsen worker 76 - jepsen.util 256 :invoke :write [25 19] ERROR [2022-01-10 05:36:51,885] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][57ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"ae551c20-a78f-4bd1-bc23-8dc4f59cbe64"}} WARN [2022-01-10 05:36:51,889] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][61ms] Connect attempt 8 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"ae551c20-a78f-4bd1-bc23-8dc4f59cbe64"}} {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"ae551c20-a78f-4bd1-bc23-8dc4f59cbe64"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) ERROR [2022-01-10 05:36:51,889] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][60ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"a2a401dc-8617-4138-435e-d213d5c28316"}} ERROR [2022-01-10 05:36:51,890] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][68ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"3ed123fb-365f-47b1-28f1-bd1103e271d6"}} ERROR [2022-01-10 05:36:51,890] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][49ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"2a312a3c-aafe-4876-c7ab-8cceec282abf"}} WARN [2022-01-10 05:36:51,890] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][64ms] Connect attempt 8 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"a2a401dc-8617-4138-435e-d213d5c28316"}} {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"a2a401dc-8617-4138-435e-d213d5c28316"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:51,890] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][53ms] Connect attempt 8 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"2a312a3c-aafe-4876-c7ab-8cceec282abf"}} {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"2a312a3c-aafe-4876-c7ab-8cceec282abf"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) WARN [2022-01-10 05:36:51,890] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][70ms] Connect attempt 8 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"3ed123fb-365f-47b1-28f1-bd1103e271d6"}} {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"3ed123fb-365f-47b1-28f1-bd1103e271d6"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) INFO [2022-01-10 05:36:51,935] jepsen worker 64 - jepsen.util 244 :ok :read [21 7] INFO [2022-01-10 05:36:51,935] jepsen worker 64 - jepsen.util 244 :invoke :read [21 nil] INFO [2022-01-10 05:36:51,967] jepsen worker 28 - jepsen.util 208 :info :write [9 14] :RequestTimeoutException INFO [2022-01-10 05:36:51,967] jepsen worker 28 - jepsen.util 298 :invoke :read [9 nil] INFO [2022-01-10 05:36:52,106] jepsen worker 66 - jepsen.util 156 :ok :read [22 :nil] INFO [2022-01-10 05:36:52,107] jepsen worker 66 - jepsen.util 156 :invoke :write [22 34] INFO [2022-01-10 05:36:52,130] jepsen worker 61 - jepsen.util 331 :ok :read [20 :nil] INFO [2022-01-10 05:36:52,130] jepsen worker 61 - jepsen.util 331 :invoke :write [20 6] INFO [2022-01-10 05:36:52,310] jepsen worker 16 - jepsen.util 196 :ok :read [5 24] INFO [2022-01-10 05:36:52,310] jepsen worker 59 - jepsen.util 329 :ok :read [19 15] INFO [2022-01-10 05:36:52,311] jepsen worker 59 - jepsen.util 329 :invoke :write [19 11] INFO [2022-01-10 05:36:52,311] jepsen worker 16 - jepsen.util 196 :invoke :write [5 10] INFO [2022-01-10 05:36:52,357] jepsen worker 68 - jepsen.util 248 :ok :read [22 :nil] INFO [2022-01-10 05:36:52,358] jepsen worker 68 - jepsen.util 248 :invoke :write [22 33] INFO [2022-01-10 05:36:52,373] jepsen worker 30 - jepsen.util 210 :ok :read [10 36] INFO [2022-01-10 05:36:52,374] jepsen worker 30 - jepsen.util 210 :invoke :read [10 nil] INFO [2022-01-10 05:36:52,487] jepsen worker 36 - jepsen.util 126 :ok :read [12 :nil] INFO [2022-01-10 05:36:52,488] jepsen worker 36 - jepsen.util 126 :invoke :write [12 29] INFO [2022-01-10 05:36:52,523] jepsen worker 64 - jepsen.util 244 :ok :read [21 7] INFO [2022-01-10 05:36:52,523] jepsen worker 64 - jepsen.util 244 :invoke :write [21 26] INFO [2022-01-10 05:36:52,532] jepsen worker 46 - jepsen.util 226 :ok :read [15 :nil] INFO [2022-01-10 05:36:52,533] jepsen worker 46 - jepsen.util 226 :invoke :write [15 42] INFO [2022-01-10 05:36:52,677] jepsen worker 55 - jepsen.util 325 :ok :read [18 14] INFO [2022-01-10 05:36:52,680] jepsen worker 55 - jepsen.util 325 :invoke :read [18 nil] INFO [2022-01-10 05:36:52,686] jepsen worker 79 - jepsen.util 349 :ok :read [26 10] INFO [2022-01-10 05:36:52,687] jepsen worker 79 - jepsen.util 349 :invoke :write [26 29] INFO [2022-01-10 05:36:52,694] jepsen worker 11 - jepsen.util 191 :ok :read [3 22] INFO [2022-01-10 05:36:52,695] jepsen worker 11 - jepsen.util 191 :invoke :write [3 24] INFO [2022-01-10 05:36:52,697] jepsen worker 54 - jepsen.util 144 :ok :read [18 14] INFO [2022-01-10 05:36:52,698] jepsen worker 54 - jepsen.util 144 :invoke :write [18 31] INFO [2022-01-10 05:36:53,442] jepsen worker 10 - jepsen.util 280 :ok :read [3 22] INFO [2022-01-10 05:36:53,442] jepsen worker 10 - jepsen.util 280 :invoke :read [3 nil] INFO [2022-01-10 05:36:53,450] jepsen worker 25 - jepsen.util 115 :info :write [8 36] :RequestTimeoutException INFO [2022-01-10 05:36:53,451] jepsen worker 25 - jepsen.util 205 :invoke :read [8 nil] INFO [2022-01-10 05:36:53,455] jepsen worker 21 - jepsen.util 201 :info :write [7 46] :RequestTimeoutException INFO [2022-01-10 05:36:53,457] jepsen worker 21 - jepsen.util 291 :invoke :write [7 14] INFO [2022-01-10 05:36:53,633] jepsen worker 55 - jepsen.util 325 :ok :read [18 14] INFO [2022-01-10 05:36:53,633] jepsen worker 55 - jepsen.util 325 :invoke :read [18 nil] INFO [2022-01-10 05:36:53,660] jepsen worker 6 - jepsen.util 96 :info :write [2 29] :RequestTimeoutException INFO [2022-01-10 05:36:53,660] jepsen worker 6 - jepsen.util 186 :invoke :write [2 16] INFO [2022-01-10 05:36:53,711] jepsen worker 25 - jepsen.util 205 :ok :read [8 :nil] INFO [2022-01-10 05:36:53,712] jepsen worker 25 - jepsen.util 205 :invoke :write [8 22] INFO [2022-01-10 05:36:53,826] jepsen worker 48 - jepsen.util 138 :ok :read [16 :nil] INFO [2022-01-10 05:36:53,827] jepsen worker 48 - jepsen.util 138 :invoke :read [16 nil] INFO [2022-01-10 05:36:53,848] jepsen worker 40 - jepsen.util 220 :ok :read [13 32] INFO [2022-01-10 05:36:53,849] jepsen worker 40 - jepsen.util 220 :invoke :read [13 nil] INFO [2022-01-10 05:36:53,925] jepsen worker 83 - jepsen.util 263 :ok :read [27 30] INFO [2022-01-10 05:36:53,926] jepsen worker 83 - jepsen.util 263 :invoke :write [27 31] INFO [2022-01-10 05:36:53,943] jepsen worker 67 - jepsen.util 247 :ok :read [22 :nil] INFO [2022-01-10 05:36:53,944] jepsen worker 67 - jepsen.util 247 :invoke :write [22 23] INFO [2022-01-10 05:36:53,957] jepsen worker 48 - jepsen.util 138 :ok :read [16 :nil] INFO [2022-01-10 05:36:53,958] jepsen worker 48 - jepsen.util 138 :invoke :write [16 16] INFO [2022-01-10 05:36:53,960] jepsen worker 58 - jepsen.util 238 :ok :read [19 15] INFO [2022-01-10 05:36:53,961] jepsen worker 58 - jepsen.util 238 :invoke :read [19 nil] INFO [2022-01-10 05:36:53,992] jepsen worker 40 - jepsen.util 220 :ok :read [13 32] INFO [2022-01-10 05:36:53,993] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][74us] Node disconnected {"coreId":"0xefe9906200000006","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:36:53,993] jepsen worker 40 - jepsen.util 220 :invoke :write [13 7] INFO [2022-01-10 05:36:54,014] jepsen worker 30 - jepsen.util 210 :ok :read [10 36] INFO [2022-01-10 05:36:54,015] jepsen worker 30 - jepsen.util 210 :invoke :read [10 nil] INFO [2022-01-10 05:36:54,040] jepsen worker 58 - jepsen.util 238 :ok :read [19 15] INFO [2022-01-10 05:36:54,041] jepsen worker 58 - jepsen.util 238 :invoke :read [19 nil] INFO [2022-01-10 05:36:54,201] jepsen worker 58 - jepsen.util 238 :ok :read [19 15] INFO [2022-01-10 05:36:54,202] jepsen worker 58 - jepsen.util 238 :invoke :write [19 37] INFO [2022-01-10 05:36:54,401] jepsen worker 13 - jepsen.util 283 :ok :read [4 :nil] INFO [2022-01-10 05:36:54,402] jepsen worker 13 - jepsen.util 283 :invoke :write [4 9] INFO [2022-01-10 05:36:54,440] jepsen worker 10 - jepsen.util 280 :ok :read [3 22] INFO [2022-01-10 05:36:54,440] jepsen worker 10 - jepsen.util 280 :invoke :read [3 nil] INFO [2022-01-10 05:36:54,470] jepsen worker 71 - jepsen.util 251 :info :write [23 14] :RequestTimeoutException INFO [2022-01-10 05:36:54,471] jepsen worker 71 - jepsen.util 341 :invoke :read [23 nil] INFO [2022-01-10 05:36:54,489] jepsen worker 55 - jepsen.util 325 :ok :read [18 14] INFO [2022-01-10 05:36:54,490] jepsen worker 55 - jepsen.util 325 :invoke :read [18 nil] INFO [2022-01-10 05:36:54,592] jepsen worker 10 - jepsen.util 280 :ok :read [3 22] INFO [2022-01-10 05:36:54,592] jepsen worker 10 - jepsen.util 280 :invoke :read [3 nil] INFO [2022-01-10 05:36:54,632] jepsen worker 55 - jepsen.util 325 :ok :read [18 14] INFO [2022-01-10 05:36:54,633] jepsen worker 55 - jepsen.util 325 :invoke :write [18 1] INFO [2022-01-10 05:36:54,678] jepsen worker 10 - jepsen.util 280 :ok :read [3 22] INFO [2022-01-10 05:36:54,679] jepsen worker 10 - jepsen.util 280 :invoke :read [3 nil] INFO [2022-01-10 05:36:54,804] jepsen worker 10 - jepsen.util 280 :ok :read [3 22] INFO [2022-01-10 05:36:54,805] jepsen worker 10 - jepsen.util 280 :invoke :write [3 14] INFO [2022-01-10 05:36:54,811] jepsen worker 84 - jepsen.util 264 :info :write [28 7] :RequestTimeoutException INFO [2022-01-10 05:36:54,811] jepsen worker 2 - jepsen.util 92 :info :write [0 18] :RequestTimeoutException INFO [2022-01-10 05:36:54,812] jepsen worker 2 - jepsen.util 182 :invoke :read [0 nil] INFO [2022-01-10 05:36:54,812] jepsen worker 84 - jepsen.util 354 :invoke :read [28 nil] INFO [2022-01-10 05:36:54,842] jepsen worker 78 - jepsen.util 348 :ok :read [26 10] INFO [2022-01-10 05:36:54,843] jepsen worker 78 - jepsen.util 348 :invoke :read [26 nil] INFO [2022-01-10 05:36:55,475] jepsen worker 32 - jepsen.util 302 :info :write [10 32] :SyncWriteAmbiguous INFO [2022-01-10 05:36:55,476] jepsen worker 32 - jepsen.util 392 :invoke :read [10 nil] INFO [2022-01-10 05:36:55,491] jepsen worker 51 - jepsen.util 231 :ok :read [17 37] INFO [2022-01-10 05:36:55,492] jepsen worker 51 - jepsen.util 231 :invoke :read [17 nil] INFO [2022-01-10 05:36:55,495] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][89us] Node disconnected {"coreId":"0xefe9906200000005","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:36:55,750] jepsen worker 73 - jepsen.util 163 :info :write [24 8] :RequestTimeoutException INFO [2022-01-10 05:36:55,751] jepsen worker 73 - jepsen.util 253 :invoke :read [24 nil] INFO [2022-01-10 05:36:55,774] jepsen worker 23 - jepsen.util 203 :info :write [7 39] :RequestTimeoutException INFO [2022-01-10 05:36:55,775] jepsen worker 23 - jepsen.util 293 :invoke :read [7 nil] INFO [2022-01-10 05:36:55,960] jepsen worker 34 - jepsen.util 214 :info :write [11 16] :RequestTimeoutException INFO [2022-01-10 05:36:55,960] jepsen worker 34 - jepsen.util 304 :invoke :write [11 48] ERROR [2022-01-10 05:36:55,995] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][69ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"037ecd63-e81f-444a-e553-89d6dca1ba05"}} WARN [2022-01-10 05:36:55,996] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][70ms] Connect attempt 9 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"037ecd63-e81f-444a-e553-89d6dca1ba05"}} {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"037ecd63-e81f-444a-e553-89d6dca1ba05"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) ERROR [2022-01-10 05:36:55,996] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][59ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000005/00000000E817EC43","circuitBreaker":"DISABLED","coreId":"0xefe9906200000005","local":"172.23.104.79:43980","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"9fa04027-ad6b-40f7-1b6e-76b7cea8dec7"}} ERROR [2022-01-10 05:36:55,996] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][49ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000006/000000002D218D66","circuitBreaker":"DISABLED","coreId":"0xefe9906200000006","local":"172.23.104.79:43982","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"ad507d7b-300b-45e2-6734-cc8963b5a975"}} ERROR [2022-01-10 05:36:55,999] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][50ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"049714d1-1db6-4dc9-007d-d4d0e5dfda2b"}} WARN [2022-01-10 05:36:56,000] cb-events - com.couchbase.endpoint [com.couchbase.endpoint][EndpointConnectionFailedEvent][56ms] Connect attempt 9 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"049714d1-1db6-4dc9-007d-d4d0e5dfda2b"}} {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","type":"KV"} com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"049714d1-1db6-4dc9-007d-d4d0e5dfda2b"}} at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:486) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:294) at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:251) at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:85) at java.lang.Thread.run(Thread.java:748) INFO [2022-01-10 05:36:56,332] jepsen worker 31 - jepsen.util 301 :info :write [10 38] :RequestTimeoutException INFO [2022-01-10 05:36:56,333] jepsen worker 31 - jepsen.util 391 :invoke :read [10 nil] INFO [2022-01-10 05:36:56,407] jepsen worker 14 - jepsen.util 284 :ok :read [4 :nil] INFO [2022-01-10 05:36:56,408] jepsen worker 14 - jepsen.util 284 :invoke :write [4 32] INFO [2022-01-10 05:36:56,437] jepsen worker 57 - jepsen.util 147 :info :write [19 31] :RequestTimeoutException INFO [2022-01-10 05:36:56,438] jepsen worker 57 - jepsen.util 237 :invoke :write [19 17] INFO [2022-01-10 05:36:56,827] jepsen worker 30 - jepsen.util 210 :ok :read [10 36] INFO [2022-01-10 05:36:56,828] jepsen worker 30 - jepsen.util 210 :invoke :write [10 33] INFO [2022-01-10 05:36:56,876] jepsen worker 50 - jepsen.util 320 :ok :read [16 :nil] INFO [2022-01-10 05:36:56,877] jepsen worker 50 - jepsen.util 320 :invoke :write [16 44] INFO [2022-01-10 05:36:56,885] jepsen worker 1 - jepsen.util 271 :ok :read [0 :nil] INFO [2022-01-10 05:36:56,885] jepsen worker 1 - jepsen.util 271 :invoke :read [0 nil] INFO [2022-01-10 05:36:56,893] jepsen worker 1 - jepsen.util 271 :ok :read [0 :nil] INFO [2022-01-10 05:36:56,893] jepsen worker 1 - jepsen.util 271 :invoke :read [0 nil] INFO [2022-01-10 05:36:56,895] jepsen worker 1 - jepsen.util 271 :ok :read [0 :nil] INFO [2022-01-10 05:36:56,896] jepsen worker 1 - jepsen.util 271 :invoke :read [0 nil] INFO [2022-01-10 05:36:56,901] jepsen worker 17 - jepsen.util 197 :ok :read [5 24] INFO [2022-01-10 05:36:56,902] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][110us] Node disconnected {"coreId":"0xefe9906200000001","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:36:56,902] jepsen worker 17 - jepsen.util 197 :invoke :read [5 nil] INFO [2022-01-10 05:36:56,952] jepsen worker 1 - jepsen.util 271 :ok :read [0 :nil] INFO [2022-01-10 05:36:56,952] jepsen worker 1 - jepsen.util 271 :invoke :write [0 49] INFO [2022-01-10 05:36:57,006] jepsen worker 41 - jepsen.util 221 :info :write [13 7] :RequestTimeoutException INFO [2022-01-10 05:36:57,009] jepsen worker 41 - jepsen.util 311 :invoke :read [13 nil] INFO [2022-01-10 05:36:57,042] jepsen worker 52 - jepsen.util 322 :ok :read [17 37] INFO [2022-01-10 05:36:57,043] jepsen worker 52 - jepsen.util 322 :invoke :write [17 43] INFO [2022-01-10 05:36:57,300] jepsen worker 31 - jepsen.util 391 :ok :read [10 36] INFO [2022-01-10 05:36:57,300] jepsen worker 31 - jepsen.util 391 :invoke :read [10 nil] INFO [2022-01-10 05:36:57,402] jepsen worker 72 - jepsen.util 342 :info :write [24 36] :RequestTimeoutException INFO [2022-01-10 05:36:57,403] jepsen worker 72 - jepsen.util 432 :invoke :read [24 nil] INFO [2022-01-10 05:36:57,574] jepsen worker 31 - jepsen.util 391 :ok :read [10 36] INFO [2022-01-10 05:36:57,575] jepsen worker 31 - jepsen.util 391 :invoke :write [10 41] INFO [2022-01-10 05:36:57,672] jepsen worker 47 - jepsen.util 227 :info :write [15 21] :RequestTimeoutException INFO [2022-01-10 05:36:57,674] jepsen worker 47 - jepsen.util 317 :invoke :write [15 30] INFO [2022-01-10 05:36:57,840] jepsen worker 81 - jepsen.util 351 :info :write [27 1] :RequestTimeoutException INFO [2022-01-10 05:36:57,841] jepsen worker 81 - jepsen.util 441 :invoke :read [27 nil] INFO [2022-01-10 05:36:58,158] jepsen worker 17 - jepsen.util 197 :ok :read [5 24] INFO [2022-01-10 05:36:58,158] jepsen worker 17 - jepsen.util 197 :invoke :write [5 42] INFO [2022-01-10 05:36:58,320] jepsen worker 85 - jepsen.util 265 :ok :read [28 :nil] INFO [2022-01-10 05:36:58,321] jepsen worker 85 - jepsen.util 265 :invoke :write [28 18] INFO [2022-01-10 05:36:58,433] jepsen worker 20 - jepsen.util 110 :ok :read [6 25] INFO [2022-01-10 05:36:58,433] jepsen worker 20 - jepsen.util 110 :invoke :read [6 nil] INFO [2022-01-10 05:36:58,710] jepsen worker 62 - jepsen.util 332 :info :write [20 18] :RequestTimeoutException INFO [2022-01-10 05:36:58,711] jepsen worker 62 - jepsen.util 422 :invoke :write [20 32] INFO [2022-01-10 05:36:58,765] jepsen worker 45 - jepsen.util 225 :info :write [15 17] :RequestTimeoutException INFO [2022-01-10 05:36:58,765] jepsen worker 45 - jepsen.util 315 :invoke :write [15 38] INFO [2022-01-10 05:36:58,767] jepsen worker 89 - jepsen.util 359 :info :write [29 29] :RequestTimeoutException INFO [2022-01-10 05:36:58,767] jepsen worker 89 - jepsen.util 449 :invoke :write [29 46] INFO [2022-01-10 05:36:58,821] jepsen worker 56 - jepsen.util 326 :info :write [18 33] :RequestTimeoutException INFO [2022-01-10 05:36:58,822] jepsen worker 56 - jepsen.util 416 :invoke :read [18 nil] INFO [2022-01-10 05:36:58,926] jepsen worker 44 - jepsen.util 314 :info :write [14 30] :RequestTimeoutException INFO [2022-01-10 05:36:58,926] jepsen worker 44 - jepsen.util 404 :invoke :read [14 nil] INFO [2022-01-10 05:36:58,953] jepsen worker 5 - jepsen.util 275 :ok :read [1 41] INFO [2022-01-10 05:36:58,954] jepsen worker 5 - jepsen.util 275 :invoke :read [1 nil] INFO [2022-01-10 05:36:59,163] jepsen worker 19 - jepsen.util 289 :info :write [6 5] :RequestTimeoutException INFO [2022-01-10 05:36:59,163] jepsen worker 19 - jepsen.util 379 :invoke :read [6 nil] INFO [2022-01-10 05:36:59,186] jepsen worker 87 - jepsen.util 267 :info :write [29 7] :RequestTimeoutException INFO [2022-01-10 05:36:59,187] jepsen worker 87 - jepsen.util 357 :invoke :read [29 nil] INFO [2022-01-10 05:36:59,211] jepsen worker 53 - jepsen.util 233 :info :write [17 44] :RequestTimeoutException INFO [2022-01-10 05:36:59,212] jepsen worker 53 - jepsen.util 323 :invoke :write [17 12] INFO [2022-01-10 05:36:59,247] jepsen worker 43 - jepsen.util 313 :ok :read [14 14] INFO [2022-01-10 05:36:59,248] jepsen worker 43 - jepsen.util 313 :invoke :write [14 31] INFO [2022-01-10 05:36:59,318] jepsen worker 80 - jepsen.util 350 :ok :read [26 10] INFO [2022-01-10 05:36:59,319] jepsen worker 80 - jepsen.util 350 :invoke :read [26 nil] INFO [2022-01-10 05:36:59,367] jepsen worker 22 - jepsen.util 202 :info :write [7 42] :RequestTimeoutException INFO [2022-01-10 05:36:59,367] jepsen worker 22 - jepsen.util 292 :invoke :write [7 25] INFO [2022-01-10 05:36:59,376] jepsen worker 42 - jepsen.util 132 :info :write [14 31] :RequestTimeoutException INFO [2022-01-10 05:36:59,376] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] WARN [2022-01-10 05:36:59,496] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9468305,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13955601,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9489275,"encode_duration_us":23011,"operation_id":"0xa86","timeout_ms":10000,"last_server_duration_us":9276438,"total_duration_us":10004926},{"operation_name":"upsert","last_dispatch_duration_us":19121706,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":22248561,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9379276,"encode_duration_us":17457,"operation_id":"0x974","timeout_ms":10000,"last_server_duration_us":9169382,"total_duration_us":10010318},{"operation_name":"upsert","last_dispatch_duration_us":9094166,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11509369,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9262015,"encode_duration_us":16958,"operation_id":"0xa44","timeout_ms":10000,"last_server_duration_us":9050106,"total_duration_us":10009352}],"total_count":3}} WARN [2022-01-10 05:36:59,510] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":7411820,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7411820,"total_server_duration_us":865915,"operation_id":"0x9de","timeout_ms":30000,"last_server_duration_us":865915,"total_duration_us":21761972},{"operation_name":"get","last_dispatch_duration_us":8310394,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8310394,"total_server_duration_us":19,"operation_id":"0xae1","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":13996269},{"operation_name":"upsert","last_dispatch_duration_us":16487439,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":16487439,"total_server_duration_us":19030,"encode_duration_us":16208,"operation_id":"0xa8b","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10010257},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","encode_duration_us":16476,"operation_id":"0xae6","timeout_ms":10000,"total_duration_us":10009216},{"operation_name":"upsert","encode_duration_us":23333,"operation_id":"0xb37","timeout_ms":10000,"total_duration_us":10004958},{"operation_name":"upsert","last_dispatch_duration_us":9468305,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":13955601,"total_server_duration_us":9489275,"encode_duration_us":23011,"operation_id":"0xa86","timeout_ms":10000,"last_server_duration_us":9276438,"total_duration_us":10004926},{"operation_name":"upsert","last_dispatch_duration_us":10666638,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10666638,"total_server_duration_us":20124,"encode_duration_us":16695,"operation_id":"0xa91","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10004804},{"operation_name":"upsert","last_dispatch_duration_us":7801790,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7801790,"total_server_duration_us":403293,"encode_duration_us":17047,"operation_id":"0xbf9","timeout_ms":10000,"last_server_duration_us":403293,"total_duration_us":10002420},{"operation_name":"upsert","last_dispatch_duration_us":17289430,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":17289430,"total_server_duration_us":20680,"encode_duration_us":16343,"operation_id":"0xa60","timeout_ms":10000,"last_server_duration_us":20680,"total_duration_us":10001498},{"operation_name":"get","last_dispatch_duration_us":9276239,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9276239,"total_server_duration_us":38,"operation_id":"0xb74","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":9276648}],"total_count":15}} INFO [2022-01-10 05:36:59,536] jepsen worker 35 - jepsen.util 305 :info :write [11 5] :RequestTimeoutException INFO [2022-01-10 05:36:59,537] jepsen worker 35 - jepsen.util 395 :invoke :write [11 9] INFO [2022-01-10 05:36:59,598] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][79us] Node disconnected {"coreId":"0xefe9906200000004","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:36:59,662] jepsen worker 3 - jepsen.util 273 :info :write [1 6] :RequestTimeoutException INFO [2022-01-10 05:36:59,663] jepsen worker 3 - jepsen.util 363 :invoke :write [1 39] INFO [2022-01-10 05:36:59,777] jepsen worker 7 - jepsen.util 97 :info :write [2 8] :RequestTimeoutException INFO [2022-01-10 05:36:59,778] jepsen worker 7 - jepsen.util 187 :invoke :read [2 nil] INFO [2022-01-10 05:36:59,785] jepsen worker 5 - jepsen.util 275 :ok :read [1 41] INFO [2022-01-10 05:36:59,785] jepsen worker 5 - jepsen.util 275 :invoke :write [1 20] INFO [2022-01-10 05:36:59,819] jepsen worker 26 - jepsen.util 296 :ok :read [8 :nil] INFO [2022-01-10 05:36:59,819] jepsen worker 26 - jepsen.util 296 :invoke :write [8 30] INFO [2022-01-10 05:36:59,846] jepsen worker 77 - jepsen.util 347 :info :write [25 15] :RequestTimeoutException INFO [2022-01-10 05:36:59,848] jepsen worker 77 - jepsen.util 437 :invoke :write [25 33] INFO [2022-01-10 05:36:59,856] jepsen worker 38 - jepsen.util 218 :info :write [12 34] :RequestTimeoutException INFO [2022-01-10 05:36:59,856] jepsen worker 12 - jepsen.util 192 :info :write [4 13] :RequestTimeoutException INFO [2022-01-10 05:36:59,857] jepsen worker 38 - jepsen.util 308 :invoke :write [12 23] INFO [2022-01-10 05:36:59,857] jepsen worker 12 - jepsen.util 282 :invoke :read [4 nil] INFO [2022-01-10 05:36:59,945] jepsen worker 69 - jepsen.util 339 :info :write [23 36] :RequestTimeoutException INFO [2022-01-10 05:36:59,945] jepsen worker 69 - jepsen.util 429 :invoke :read [23 nil] INFO [2022-01-10 05:36:59,992] jepsen worker 63 - jepsen.util 333 :info :write [21 22] :RequestTimeoutException INFO [2022-01-10 05:36:59,992] jepsen worker 80 - jepsen.util 350 :ok :read [26 10] INFO [2022-01-10 05:36:59,992] jepsen worker 63 - jepsen.util 423 :invoke :read [21 nil] INFO [2022-01-10 05:36:59,993] jepsen worker 80 - jepsen.util 350 :invoke :read [26 nil] INFO [2022-01-10 05:36:59,996] jepsen worker 4 - jepsen.util 184 :info :write [1 0] :RequestTimeoutException INFO [2022-01-10 05:36:59,999] jepsen worker 4 - jepsen.util 274 :invoke :read [1 nil] INFO [2022-01-10 05:37:00,073] jepsen worker 73 - jepsen.util 253 :ok :read [24 33] INFO [2022-01-10 05:37:00,074] jepsen worker 73 - jepsen.util 253 :invoke :write [24 48] INFO [2022-01-10 05:37:00,122] jepsen worker 8 - jepsen.util 188 :info :write [2 7] :RequestTimeoutException INFO [2022-01-10 05:37:00,122] jepsen worker 8 - jepsen.util 278 :invoke :read [2 nil] INFO [2022-01-10 05:37:00,193] jepsen worker 86 - jepsen.util 356 :ok :read [28 :nil] INFO [2022-01-10 05:37:00,193] jepsen worker 86 - jepsen.util 356 :invoke :read [28 nil] ERROR [2022-01-10 05:37:00,199] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][46ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000004/000000004085468E","circuitBreaker":"DISABLED","coreId":"0xefe9906200000004","local":"172.23.104.79:43940","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"d13e9c33-1230-44d0-d34b-171120e97798"}} INFO [2022-01-10 05:37:00,207] jepsen worker 29 - jepsen.util 209 :ok :read [9 22] INFO [2022-01-10 05:37:00,208] jepsen worker 29 - jepsen.util 209 :invoke :write [9 14] ERROR [2022-01-10 05:37:00,212] cb-events - com.couchbase.io [com.couchbase.io][SaslAuthenticationFailedEvent][47ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"channelId":"EFE9906200000001/00000000601F9E85","circuitBreaker":"DISABLED","coreId":"0xefe9906200000001","local":"172.23.104.79:43916","remote":"172.23.105.187:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"25f24a7a-94eb-4735-ce03-43fb6258f557"}} INFO [2022-01-10 05:37:00,219] jepsen worker 82 - jepsen.util 172 :ok :read [27 30] INFO [2022-01-10 05:37:00,220] jepsen worker 82 - jepsen.util 172 :invoke :read [27 nil] INFO [2022-01-10 05:37:00,238] jepsen worker 84 - jepsen.util 354 :ok :read [28 :nil] INFO [2022-01-10 05:37:00,239] jepsen worker 84 - jepsen.util 354 :invoke :write [28 40] INFO [2022-01-10 05:37:00,251] jepsen worker 28 - jepsen.util 298 :ok :read [9 22] INFO [2022-01-10 05:37:00,252] jepsen worker 28 - jepsen.util 298 :invoke :write [9 42] INFO [2022-01-10 05:37:00,308] jepsen worker 80 - jepsen.util 350 :ok :read [26 10] INFO [2022-01-10 05:37:00,308] jepsen worker 80 - jepsen.util 350 :invoke :write [26 12] INFO [2022-01-10 05:37:00,332] jepsen worker 9 - jepsen.util 279 :info :write [3 12] :RequestTimeoutException INFO [2022-01-10 05:37:00,332] jepsen worker 9 - jepsen.util 369 :invoke :write [3 10] INFO [2022-01-10 05:37:00,531] jepsen worker 37 - jepsen.util 307 :info :write [12 43] :RequestTimeoutException INFO [2022-01-10 05:37:00,532] jepsen worker 37 - jepsen.util 397 :invoke :write [12 23] INFO [2022-01-10 05:37:00,573] jepsen worker 42 - jepsen.util 222 :ok :read [14 14] INFO [2022-01-10 05:37:00,573] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 05:37:00,587] jepsen worker 72 - jepsen.util 432 :ok :read [24 33] INFO [2022-01-10 05:37:00,587] jepsen worker 72 - jepsen.util 432 :invoke :write [24 6] INFO [2022-01-10 05:37:00,651] jepsen worker 74 - jepsen.util 344 :info :write [24 40] :RequestTimeoutException INFO [2022-01-10 05:37:00,652] jepsen worker 74 - jepsen.util 434 :invoke :read [24 nil] INFO [2022-01-10 05:37:00,667] jepsen worker 0 - jepsen.util 270 :ok :read [0 :nil] INFO [2022-01-10 05:37:00,668] jepsen worker 0 - jepsen.util 270 :invoke :read [0 nil] INFO [2022-01-10 05:37:00,671] jepsen worker 49 - jepsen.util 319 :ok :read [16 :nil] INFO [2022-01-10 05:37:00,672] jepsen worker 49 - jepsen.util 319 :invoke :write [16 35] INFO [2022-01-10 05:37:00,676] jepsen worker 87 - jepsen.util 357 :ok :read [29 33] INFO [2022-01-10 05:37:00,676] jepsen worker 87 - jepsen.util 357 :invoke :write [29 7] INFO [2022-01-10 05:37:00,677] jepsen worker 2 - jepsen.util 182 :ok :read [0 :nil] INFO [2022-01-10 05:37:00,677] jepsen worker 2 - jepsen.util 182 :invoke :write [0 22] INFO [2022-01-10 05:37:00,683] jepsen worker 27 - jepsen.util 207 :ok :read [9 22] INFO [2022-01-10 05:37:00,686] jepsen worker 27 - jepsen.util 207 :invoke :write [9 22] INFO [2022-01-10 05:37:00,690] jepsen worker 39 - jepsen.util 219 :ok :read [13 23] INFO [2022-01-10 05:37:00,691] jepsen worker 39 - jepsen.util 219 :invoke :write [13 40] INFO [2022-01-10 05:37:00,941] jepsen worker 65 - jepsen.util 335 :ok :read [21 7] INFO [2022-01-10 05:37:00,942] jepsen worker 65 - jepsen.util 335 :invoke :write [21 39] INFO [2022-01-10 05:37:00,967] jepsen worker 60 - jepsen.util 330 :info :write [20 42] :RequestTimeoutException INFO [2022-01-10 05:37:00,967] jepsen worker 60 - jepsen.util 420 :invoke :write [20 17] WARN [2022-01-10 05:37:01,099] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":21211852,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":21211852,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9028877,"encode_duration_us":17376,"operation_id":"0x92f","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10004302},{"operation_name":"upsert","last_dispatch_duration_us":17039781,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":20220895,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9530995,"encode_duration_us":20445,"operation_id":"0xa2a","timeout_ms":10000,"last_server_duration_us":9486716,"total_duration_us":10005119},{"operation_name":"upsert","last_dispatch_duration_us":11361091,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11361091,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9028877,"encode_duration_us":25742,"operation_id":"0xa24","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10008159},{"operation_name":"upsert","last_dispatch_duration_us":13970792,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13970792,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9485632,"encode_duration_us":17302,"operation_id":"0x8c3","timeout_ms":10000,"last_server_duration_us":9485632,"total_duration_us":10005349}],"total_count":4}} WARN [2022-01-10 05:37:01,100] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16139580,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":16139580,"total_server_duration_us":145,"operation_id":"0x867","timeout_ms":30000,"last_server_duration_us":145,"total_duration_us":24871030},{"operation_name":"get","last_dispatch_duration_us":60086,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":60086,"total_server_duration_us":49,"operation_id":"0xa7d","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":12383831},{"operation_name":"upsert","last_dispatch_duration_us":146236,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3431711,"total_server_duration_us":351178,"encode_duration_us":16416,"operation_id":"0xc3e","timeout_ms":10000,"last_server_duration_us":95942,"total_duration_us":10006452},{"operation_name":"upsert","last_dispatch_duration_us":977199,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3257011,"total_server_duration_us":204203,"encode_duration_us":16321,"operation_id":"0xc02","timeout_ms":10000,"last_server_duration_us":58128,"total_duration_us":10005527},{"operation_name":"upsert","last_dispatch_duration_us":430250,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":558608,"total_server_duration_us":384845,"encode_duration_us":23420,"operation_id":"0xc4f","timeout_ms":10000,"last_server_duration_us":334720,"total_duration_us":10005229},{"operation_name":"upsert","last_dispatch_duration_us":7233686,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8174517,"total_server_duration_us":523525,"encode_duration_us":17917,"operation_id":"0xc50","timeout_ms":10000,"last_server_duration_us":97947,"total_duration_us":10004563},{"operation_name":"upsert","last_dispatch_duration_us":408400,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2198029,"total_server_duration_us":276999,"encode_duration_us":19185,"operation_id":"0xc14","timeout_ms":10000,"last_server_duration_us":70690,"total_duration_us":10003543},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":17118,"operation_id":"0xc7e","timeout_ms":10000,"total_duration_us":10003052},{"operation_name":"upsert","last_dispatch_duration_us":240081,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3269709,"total_server_duration_us":279102,"encode_duration_us":16460,"operation_id":"0xc11","timeout_ms":10000,"last_server_duration_us":52896,"total_duration_us":10002827},{"operation_name":"upsert","last_dispatch_duration_us":1324657,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":1413211,"total_server_duration_us":623076,"encode_duration_us":16483,"operation_id":"0xc56","timeout_ms":10000,"last_server_duration_us":596212,"total_duration_us":10001803}],"total_count":18}} WARN [2022-01-10 05:37:01,100] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","encode_duration_us":16809,"operation_id":"0xc5e","timeout_ms":10000,"total_duration_us":10011052},{"operation_name":"upsert","last_dispatch_duration_us":43903,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9692202,"total_server_duration_us":162661,"encode_duration_us":17965,"operation_id":"0xb16","timeout_ms":10000,"last_server_duration_us":20045,"total_duration_us":10008011},{"operation_name":"upsert","last_dispatch_duration_us":6311503,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7374333,"total_server_duration_us":392190,"encode_duration_us":17204,"operation_id":"0xb8f","timeout_ms":10000,"last_server_duration_us":204225,"total_duration_us":10007566},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":24357,"operation_id":"0xc0c","timeout_ms":10000,"total_duration_us":10007534},{"operation_name":"upsert","last_dispatch_duration_us":117286,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9672527,"total_server_duration_us":114496,"encode_duration_us":19147,"operation_id":"0xb17","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10007464},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":16666,"operation_id":"0xc00","timeout_ms":10000,"total_duration_us":10007196},{"operation_name":"upsert","encode_duration_us":16425,"operation_id":"0xc65","timeout_ms":10000,"total_duration_us":10006065},{"operation_name":"upsert","last_dispatch_duration_us":1563801,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3175325,"total_server_duration_us":629373,"encode_duration_us":21766,"operation_id":"0xc67","timeout_ms":10000,"last_server_duration_us":131866,"total_duration_us":10005418},{"operation_name":"upsert","last_dispatch_duration_us":6950668,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8936411,"total_server_duration_us":520567,"encode_duration_us":17337,"operation_id":"0xc55","timeout_ms":10000,"last_server_duration_us":202112,"total_duration_us":10002127},{"operation_name":"get","last_dispatch_duration_us":9077600,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9077600,"total_server_duration_us":43,"operation_id":"0xb1a","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":9101371}],"total_count":18}} WARN [2022-01-10 05:37:01,100] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":2369040,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":2369040,"total_server_duration_us":298496,"operation_id":"0xa07","timeout_ms":30000,"last_server_duration_us":298496,"total_duration_us":18250424},{"operation_name":"get","last_dispatch_duration_us":13271547,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":13271547,"total_server_duration_us":307,"operation_id":"0xa52","timeout_ms":30000,"last_server_duration_us":307,"total_duration_us":13382958},{"operation_name":"get","last_dispatch_duration_us":3556409,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":3556409,"total_server_duration_us":22223,"operation_id":"0xa5e","timeout_ms":30000,"last_server_duration_us":22223,"total_duration_us":12295519},{"operation_name":"get","last_dispatch_duration_us":3393609,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":3393609,"total_server_duration_us":15,"operation_id":"0xa7f","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":11617159},{"operation_name":"upsert","last_dispatch_duration_us":2234487,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":6678361,"total_server_duration_us":478789,"encode_duration_us":17071,"operation_id":"0xc0b","timeout_ms":10000,"last_server_duration_us":223881,"total_duration_us":10009667},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":16660,"operation_id":"0xb52","timeout_ms":10000,"total_duration_us":10007087},{"operation_name":"upsert","last_dispatch_duration_us":289143,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":2279778,"total_server_duration_us":98883,"encode_duration_us":16805,"operation_id":"0xc4e","timeout_ms":10000,"last_server_duration_us":47762,"total_duration_us":10006187},{"operation_name":"upsert","last_dispatch_duration_us":2000066,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":2000066,"total_server_duration_us":137364,"encode_duration_us":16325,"operation_id":"0xbf8","timeout_ms":10000,"last_server_duration_us":137364,"total_duration_us":10006088},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":16505,"operation_id":"0xaca","timeout_ms":10000,"total_duration_us":10004164},{"operation_name":"get","last_dispatch_duration_us":3723089,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":3723089,"total_server_duration_us":38,"operation_id":"0xb79","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":4285917}],"total_count":16}} WARN [2022-01-10 05:37:01,100] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8727784,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":8727784,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":6868255,"encode_duration_us":19273,"operation_id":"0x9df","timeout_ms":10000,"last_server_duration_us":6868255,"total_duration_us":10004335},{"operation_name":"upsert","last_dispatch_duration_us":21586345,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":21896003,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9132586,"encode_duration_us":21823,"operation_id":"0x9cf","timeout_ms":10000,"last_server_duration_us":9112856,"total_duration_us":10006070},{"operation_name":"upsert","last_dispatch_duration_us":10518693,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12807918,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":10432855,"encode_duration_us":16764,"operation_id":"0xb33","timeout_ms":10000,"last_server_duration_us":10051669,"total_duration_us":10010631}],"total_count":3}} WARN [2022-01-10 05:37:01,100] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":19942086,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":19942086,"total_server_duration_us":56,"operation_id":"0x934","timeout_ms":30000,"last_server_duration_us":56,"total_duration_us":19945049},{"operation_name":"get","last_dispatch_duration_us":19108288,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":19108288,"total_server_duration_us":43,"operation_id":"0xab2","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":19109446},{"operation_name":"get","last_dispatch_duration_us":13418534,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":13418534,"total_server_duration_us":9114,"operation_id":"0xb23","timeout_ms":30000,"last_server_duration_us":9114,"total_duration_us":13421171},{"operation_name":"get","last_dispatch_duration_us":10784376,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10784376,"total_server_duration_us":16050,"operation_id":"0xbd1","timeout_ms":30000,"last_server_duration_us":16050,"total_duration_us":10786573},{"operation_name":"get","last_dispatch_duration_us":10098274,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10098274,"total_server_duration_us":27,"operation_id":"0xc58","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":10098739},{"operation_name":"upsert","last_dispatch_duration_us":9194802,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9755256,"total_server_duration_us":348806,"encode_duration_us":16277,"operation_id":"0xc0a","timeout_ms":10000,"last_server_duration_us":314095,"total_duration_us":10009341},{"operation_name":"upsert","last_dispatch_duration_us":425547,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10264152,"total_server_duration_us":415728,"encode_duration_us":24546,"operation_id":"0xc27","timeout_ms":10000,"last_server_duration_us":184530,"total_duration_us":10003781},{"operation_name":"upsert","last_dispatch_duration_us":9696428,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9696428,"total_server_duration_us":358317,"encode_duration_us":17397,"operation_id":"0xc59","timeout_ms":10000,"last_server_duration_us":358317,"total_duration_us":10003711},{"operation_name":"upsert","last_dispatch_duration_us":17237067,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":17237067,"total_server_duration_us":9050106,"encode_duration_us":21400,"operation_id":"0xab7","timeout_ms":10000,"last_server_duration_us":9050106,"total_duration_us":10003123},{"operation_name":"upsert","last_dispatch_duration_us":9828788,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10384869,"total_server_duration_us":381657,"encode_duration_us":16167,"operation_id":"0xb78","timeout_ms":10000,"last_server_duration_us":337599,"total_duration_us":10002583}],"total_count":18}} WARN [2022-01-10 05:37:01,200] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":19709378,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20126096,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":8713555,"encode_duration_us":17367,"operation_id":"0x9fb","timeout_ms":10000,"last_server_duration_us":8499888,"total_duration_us":10002943}],"total_count":1}} WARN [2022-01-10 05:37:01,200] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":6845156,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":6845156,"total_server_duration_us":32,"operation_id":"0x8ec","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":21703925},{"operation_name":"get","last_dispatch_duration_us":20127447,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":20127447,"total_server_duration_us":23,"operation_id":"0xa32","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":20128019},{"operation_name":"get","last_dispatch_duration_us":8255089,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8255089,"total_server_duration_us":43,"operation_id":"0xb4f","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":14444662},{"operation_name":"get","last_dispatch_duration_us":8021089,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8021089,"total_server_duration_us":362,"operation_id":"0xc13","timeout_ms":30000,"last_server_duration_us":362,"total_duration_us":11148632},{"operation_name":"get","last_dispatch_duration_us":8026865,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8026865,"total_server_duration_us":27,"operation_id":"0xc57","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":10645150},{"operation_name":"get","last_dispatch_duration_us":8196537,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8196537,"total_server_duration_us":32,"operation_id":"0xc61","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":10299934},{"operation_name":"upsert","last_dispatch_duration_us":10518693,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12807918,"total_server_duration_us":10432855,"encode_duration_us":16764,"operation_id":"0xb33","timeout_ms":10000,"last_server_duration_us":10051669,"total_duration_us":10010631},{"operation_name":"upsert","last_dispatch_duration_us":12249212,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":12249212,"total_server_duration_us":19730,"encode_duration_us":16236,"operation_id":"0xbac","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10010209},{"operation_name":"upsert","last_dispatch_duration_us":9757217,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":14355250,"total_server_duration_us":460377,"encode_duration_us":17271,"operation_id":"0xae8","timeout_ms":10000,"last_server_duration_us":110674,"total_duration_us":10009445},{"operation_name":"upsert","last_dispatch_duration_us":149854,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10052091,"total_server_duration_us":669880,"encode_duration_us":17081,"operation_id":"0xb03","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10006240}],"total_count":24}} WARN [2022-01-10 05:37:01,201] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17237067,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17237067,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9050106,"encode_duration_us":21400,"operation_id":"0xab7","timeout_ms":10000,"last_server_duration_us":9050106,"total_duration_us":10003123},{"operation_name":"upsert","last_dispatch_duration_us":15780389,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":20316858,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9153719,"encode_duration_us":18286,"operation_id":"0x918","timeout_ms":10000,"last_server_duration_us":9109661,"total_duration_us":10005716},{"operation_name":"upsert","last_dispatch_duration_us":16519268,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17866791,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9167427,"encode_duration_us":16225,"operation_id":"0xa88","timeout_ms":10000,"last_server_duration_us":8346412,"total_duration_us":10007395},{"operation_name":"upsert","last_dispatch_duration_us":21795379,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":30774291,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9381265,"encode_duration_us":83896,"operation_id":"0x73d","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10009963},{"operation_name":"upsert","last_dispatch_duration_us":16751999,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17604997,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9475989,"encode_duration_us":21380,"operation_id":"0xa0b","timeout_ms":10000,"last_server_duration_us":9444500,"total_duration_us":10006326},{"operation_name":"upsert","last_dispatch_duration_us":29637064,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":29637064,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9032060,"encode_duration_us":22610,"operation_id":"0x916","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10008284}],"total_count":6}} INFO [2022-01-10 05:37:01,293] jepsen worker 42 - jepsen.util 222 :ok :read [14 14] INFO [2022-01-10 05:37:01,293] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 05:37:01,297] jepsen worker 8 - jepsen.util 278 :ok :read [2 21] INFO [2022-01-10 05:37:01,297] jepsen worker 8 - jepsen.util 278 :invoke :read [2 nil] INFO [2022-01-10 05:37:01,651] jepsen worker 66 - jepsen.util 156 :info :write [22 34] :SyncWriteAmbiguous INFO [2022-01-10 05:37:01,652] jepsen worker 66 - jepsen.util 246 :invoke :read [22 nil] INFO [2022-01-10 05:37:01,700] jepsen worker 70 - jepsen.util 340 :info :write [23 12] :RequestTimeoutException INFO [2022-01-10 05:37:01,700] jepsen worker 70 - jepsen.util 430 :invoke :read [23 nil] INFO [2022-01-10 05:37:01,726] jepsen worker 24 - jepsen.util 204 :info :write [8 48] :RequestTimeoutException INFO [2022-01-10 05:37:01,727] jepsen worker 24 - jepsen.util 294 :invoke :read [8 nil] INFO [2022-01-10 05:37:01,836] jepsen worker 18 - jepsen.util 288 :info :write [6 17] :RequestTimeoutException INFO [2022-01-10 05:37:01,837] jepsen worker 18 - jepsen.util 378 :invoke :read [6 nil] INFO [2022-01-10 05:37:01,866] jepsen worker 76 - jepsen.util 256 :info :write [25 19] :RequestTimeoutException INFO [2022-01-10 05:37:01,867] jepsen worker 76 - jepsen.util 346 :invoke :read [25 nil] INFO [2022-01-10 05:37:01,971] jepsen worker 42 - jepsen.util 222 :ok :read [14 14] INFO [2022-01-10 05:37:01,972] jepsen worker 42 - jepsen.util 222 :invoke :write [14 20] INFO [2022-01-10 05:37:02,003] jepsen worker 70 - jepsen.util 430 :ok :read [23 :nil] INFO [2022-01-10 05:37:02,003] jepsen worker 70 - jepsen.util 430 :invoke :write [23 30] INFO [2022-01-10 05:37:02,156] jepsen worker 61 - jepsen.util 331 :info :write [20 6] :RequestTimeoutException INFO [2022-01-10 05:37:02,157] jepsen worker 61 - jepsen.util 421 :invoke :read [20 nil] INFO [2022-01-10 05:37:02,336] jepsen worker 59 - jepsen.util 329 :info :write [19 11] :RequestTimeoutException INFO [2022-01-10 05:37:02,336] jepsen worker 16 - jepsen.util 196 :info :write [5 10] :RequestTimeoutException INFO [2022-01-10 05:37:02,337] jepsen worker 16 - jepsen.util 286 :invoke :write [5 44] INFO [2022-01-10 05:37:02,337] jepsen worker 59 - jepsen.util 419 :invoke :write [19 48] INFO [2022-01-10 05:37:02,386] jepsen worker 68 - jepsen.util 248 :info :write [22 33] :RequestTimeoutException INFO [2022-01-10 05:37:02,387] jepsen worker 68 - jepsen.util 338 :invoke :read [22 nil] INFO [2022-01-10 05:37:02,474] jepsen worker 8 - jepsen.util 278 :ok :read [2 21] INFO [2022-01-10 05:37:02,474] jepsen worker 8 - jepsen.util 278 :invoke :write [2 39] INFO [2022-01-10 05:37:02,522] jepsen worker 36 - jepsen.util 126 :info :write [12 29] :RequestTimeoutException INFO [2022-01-10 05:37:02,523] jepsen worker 36 - jepsen.util 216 :invoke :read [12 nil] INFO [2022-01-10 05:37:02,547] jepsen worker 64 - jepsen.util 244 :info :write [21 26] :RequestTimeoutException INFO [2022-01-10 05:37:02,547] jepsen worker 64 - jepsen.util 334 :invoke :read [21 nil] INFO [2022-01-10 05:37:02,566] jepsen worker 46 - jepsen.util 226 :info :write [15 42] :RequestTimeoutException INFO [2022-01-10 05:37:02,566] jepsen worker 46 - jepsen.util 316 :invoke :read [15 nil] INFO [2022-01-10 05:37:02,706] jepsen worker 79 - jepsen.util 349 :info :write [26 29] :RequestTimeoutException INFO [2022-01-10 05:37:02,707] jepsen worker 79 - jepsen.util 439 :invoke :read [26 nil] INFO [2022-01-10 05:37:02,720] jepsen worker 11 - jepsen.util 191 :info :write [3 24] :RequestTimeoutException INFO [2022-01-10 05:37:02,720] jepsen worker 54 - jepsen.util 144 :info :write [18 31] :RequestTimeoutException INFO [2022-01-10 05:37:02,721] jepsen worker 11 - jepsen.util 281 :invoke :read [3 nil] INFO [2022-01-10 05:37:02,722] jepsen worker 54 - jepsen.util 234 :invoke :read [18 nil] INFO [2022-01-10 05:37:02,751] jepsen worker 11 - jepsen.util 281 :ok :read [3 2] INFO [2022-01-10 05:37:02,752] jepsen worker 11 - jepsen.util 281 :invoke :read [3 nil] INFO [2022-01-10 05:37:02,791] jepsen worker 76 - jepsen.util 346 :ok :read [25 :nil] INFO [2022-01-10 05:37:02,791] jepsen worker 76 - jepsen.util 346 :invoke :read [25 nil] INFO [2022-01-10 05:37:02,792] jepsen worker 23 - jepsen.util 293 :ok :read [7 :nil] INFO [2022-01-10 05:37:02,792] jepsen worker 23 - jepsen.util 293 :invoke :read [7 nil] INFO [2022-01-10 05:37:02,832] jepsen worker 64 - jepsen.util 334 :ok :read [21 7] INFO [2022-01-10 05:37:02,834] jepsen worker 64 - jepsen.util 334 :invoke :read [21 nil] INFO [2022-01-10 05:37:02,861] jepsen worker 33 - jepsen.util 213 :ok :read [11 21] INFO [2022-01-10 05:37:02,861] jepsen worker 33 - jepsen.util 213 :invoke :write [11 12] INFO [2022-01-10 05:37:03,030] jepsen worker 44 - jepsen.util 404 :ok :read [14 14] INFO [2022-01-10 05:37:03,031] jepsen worker 44 - jepsen.util 404 :invoke :read [14 nil] INFO [2022-01-10 05:37:03,049] jepsen worker 83 - jepsen.util 263 :info :write [27 31] :SyncWriteAmbiguous INFO [2022-01-10 05:37:03,050] jepsen worker 83 - jepsen.util 353 :invoke :write [27 35] INFO [2022-01-10 05:37:03,377] jepsen worker 69 - jepsen.util 429 :ok :read [23 :nil] INFO [2022-01-10 05:37:03,378] jepsen worker 69 - jepsen.util 429 :invoke :read [23 nil] INFO [2022-01-10 05:37:03,491] jepsen worker 21 - jepsen.util 291 :info :write [7 14] :RequestTimeoutException INFO [2022-01-10 05:37:03,492] jepsen worker 21 - jepsen.util 381 :invoke :write [7 6] INFO [2022-01-10 05:37:03,501] jepsen worker 88 - jepsen.util 358 :ok :read [29 33] INFO [2022-01-10 05:37:03,502] jepsen worker 88 - jepsen.util 358 :invoke :write [29 5] INFO [2022-01-10 05:37:03,685] jepsen worker 6 - jepsen.util 186 :info :write [2 16] :RequestTimeoutException INFO [2022-01-10 05:37:03,686] jepsen worker 6 - jepsen.util 276 :invoke :write [2 2] INFO [2022-01-10 05:37:03,697] jepsen worker 71 - jepsen.util 341 :ok :read [23 :nil] INFO [2022-01-10 05:37:03,698] jepsen worker 71 - jepsen.util 341 :invoke :write [23 9] INFO [2022-01-10 05:37:03,733] jepsen worker 7 - jepsen.util 187 :ok :read [2 21] INFO [2022-01-10 05:37:03,734] jepsen worker 7 - jepsen.util 187 :invoke :read [2 nil] INFO [2022-01-10 05:37:03,738] jepsen worker 25 - jepsen.util 205 :info :write [8 22] :RequestTimeoutException INFO [2022-01-10 05:37:03,739] jepsen worker 25 - jepsen.util 295 :invoke :read [8 nil] INFO [2022-01-10 05:37:03,760] jepsen worker 78 - jepsen.util 348 :ok :read [26 10] INFO [2022-01-10 05:37:03,761] jepsen worker 78 - jepsen.util 348 :invoke :write [26 43] INFO [2022-01-10 05:37:03,971] jepsen worker 67 - jepsen.util 247 :info :write [22 23] :RequestTimeoutException INFO [2022-01-10 05:37:03,975] jepsen worker 67 - jepsen.util 337 :invoke :read [22 nil] INFO [2022-01-10 05:37:03,981] jepsen worker 48 - jepsen.util 138 :info :write [16 16] :RequestTimeoutException INFO [2022-01-10 05:37:03,981] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 05:37:04,005] jepsen worker 69 - jepsen.util 429 :ok :read [23 :nil] INFO [2022-01-10 05:37:04,006] jepsen worker 69 - jepsen.util 429 :invoke :read [23 nil] INFO [2022-01-10 05:37:04,031] jepsen worker 40 - jepsen.util 220 :info :write [13 7] :RequestTimeoutException INFO [2022-01-10 05:37:04,032] jepsen worker 40 - jepsen.util 310 :invoke :read [13 nil] INFO [2022-01-10 05:37:04,077] jepsen worker 61 - jepsen.util 421 :ok :read [20 :nil] INFO [2022-01-10 05:37:04,078] jepsen worker 61 - jepsen.util 421 :invoke :write [20 39] INFO [2022-01-10 05:37:04,222] jepsen worker 58 - jepsen.util 238 :info :write [19 37] :RequestTimeoutException INFO [2022-01-10 05:37:04,222] jepsen worker 58 - jepsen.util 328 :invoke :write [19 18] INFO [2022-01-10 05:37:04,426] jepsen worker 13 - jepsen.util 283 :info :write [4 9] :RequestTimeoutException INFO [2022-01-10 05:37:04,427] jepsen worker 13 - jepsen.util 373 :invoke :read [4 nil] INFO [2022-01-10 05:37:04,523] jepsen worker 46 - jepsen.util 316 :ok :read [15 :nil] INFO [2022-01-10 05:37:04,540] jepsen worker 46 - jepsen.util 316 :invoke :read [15 nil] INFO [2022-01-10 05:37:04,642] jepsen worker 18 - jepsen.util 378 :ok :read [6 25] INFO [2022-01-10 05:37:04,642] jepsen worker 18 - jepsen.util 378 :invoke :read [6 nil] INFO [2022-01-10 05:37:04,644] jepsen worker 63 - jepsen.util 423 :ok :read [21 7] INFO [2022-01-10 05:37:04,644] jepsen worker 63 - jepsen.util 423 :invoke :read [21 nil] INFO [2022-01-10 05:37:04,660] jepsen worker 55 - jepsen.util 325 :info :write [18 1] :RequestTimeoutException INFO [2022-01-10 05:37:04,661] jepsen worker 55 - jepsen.util 415 :invoke :read [18 nil] INFO [2022-01-10 05:37:04,686] jepsen worker 25 - jepsen.util 295 :ok :read [8 :nil] INFO [2022-01-10 05:37:04,687] jepsen worker 25 - jepsen.util 295 :invoke :read [8 nil] INFO [2022-01-10 05:37:04,700] jepsen worker 7 - jepsen.util 187 :ok :read [2 21] INFO [2022-01-10 05:37:04,700] jepsen worker 7 - jepsen.util 187 :invoke :write [2 10] INFO [2022-01-10 05:37:04,830] jepsen worker 10 - jepsen.util 280 :info :write [3 14] :RequestTimeoutException INFO [2022-01-10 05:37:04,831] jepsen worker 10 - jepsen.util 370 :invoke :read [3 nil] INFO [2022-01-10 05:37:05,322] jepsen worker 64 - jepsen.util 334 :ok :read [21 7] INFO [2022-01-10 05:37:05,322] jepsen worker 64 - jepsen.util 334 :invoke :write [21 13] INFO [2022-01-10 05:37:05,383] jepsen worker 69 - jepsen.util 429 :ok :read [23 :nil] INFO [2022-01-10 05:37:05,383] jepsen worker 69 - jepsen.util 429 :invoke :write [23 6] INFO [2022-01-10 05:37:05,439] jepsen worker 25 - jepsen.util 295 :ok :read [8 :nil] INFO [2022-01-10 05:37:05,439] jepsen worker 25 - jepsen.util 295 :invoke :read [8 nil] INFO [2022-01-10 05:37:05,523] jepsen worker 44 - jepsen.util 404 :ok :read [14 14] INFO [2022-01-10 05:37:05,524] jepsen worker 44 - jepsen.util 404 :invoke :read [14 nil] INFO [2022-01-10 05:37:05,551] jepsen worker 13 - jepsen.util 373 :ok :read [4 :nil] INFO [2022-01-10 05:37:05,551] jepsen worker 13 - jepsen.util 373 :invoke :write [4 45] INFO [2022-01-10 05:37:05,719] jepsen worker 24 - jepsen.util 294 :ok :read [8 :nil] INFO [2022-01-10 05:37:05,719] jepsen worker 24 - jepsen.util 294 :invoke :write [8 28] INFO [2022-01-10 05:37:05,986] jepsen worker 34 - jepsen.util 304 :info :write [11 48] :RequestTimeoutException INFO [2022-01-10 05:37:05,987] jepsen worker 34 - jepsen.util 394 :invoke :write [11 16] INFO [2022-01-10 05:37:06,002] jepsen worker 76 - jepsen.util 346 :ok :read [25 :nil] INFO [2022-01-10 05:37:06,002] jepsen worker 75 - jepsen.util 345 :ok :read [25 :nil] INFO [2022-01-10 05:37:06,003] jepsen worker 76 - jepsen.util 346 :invoke :read [25 nil] INFO [2022-01-10 05:37:06,004] jepsen worker 75 - jepsen.util 345 :invoke :read [25 nil] INFO [2022-01-10 05:37:06,006] jepsen worker 23 - jepsen.util 293 :ok :read [7 :nil] INFO [2022-01-10 05:37:06,007] jepsen worker 23 - jepsen.util 293 :invoke :write [7 3] INFO [2022-01-10 05:37:06,018] jepsen worker 20 - jepsen.util 110 :ok :read [6 25] INFO [2022-01-10 05:37:06,019] jepsen worker 20 - jepsen.util 110 :invoke :write [6 13] INFO [2022-01-10 05:37:06,164] jepsen worker 18 - jepsen.util 378 :ok :read [6 25] INFO [2022-01-10 05:37:06,165] jepsen worker 18 - jepsen.util 378 :invoke :read [6 nil] INFO [2022-01-10 05:37:06,435] jepsen worker 14 - jepsen.util 284 :info :write [4 32] :RequestTimeoutException INFO [2022-01-10 05:37:06,436] jepsen worker 14 - jepsen.util 374 :invoke :write [4 8] INFO [2022-01-10 05:37:06,462] jepsen worker 57 - jepsen.util 237 :info :write [19 17] :RequestTimeoutException INFO [2022-01-10 05:37:06,463] jepsen worker 57 - jepsen.util 327 :invoke :write [19 46] INFO [2022-01-10 05:37:06,547] jepsen worker 25 - jepsen.util 295 :ok :read [8 :nil] INFO [2022-01-10 05:37:06,547] jepsen worker 25 - jepsen.util 295 :invoke :read [8 nil] INFO [2022-01-10 05:37:06,622] jepsen worker 40 - jepsen.util 310 :ok :read [13 23] INFO [2022-01-10 05:37:06,622] jepsen worker 40 - jepsen.util 310 :invoke :write [13 22] INFO [2022-01-10 05:37:06,700] jepsen worker 74 - jepsen.util 434 :ok :read [24 33] INFO [2022-01-10 05:37:06,701] jepsen worker 74 - jepsen.util 434 :invoke :write [24 29] INFO [2022-01-10 05:37:06,857] jepsen worker 30 - jepsen.util 210 :info :write [10 33] :RequestTimeoutException INFO [2022-01-10 05:37:06,857] jepsen worker 30 - jepsen.util 300 :invoke :read [10 nil] INFO [2022-01-10 05:37:06,905] jepsen worker 50 - jepsen.util 320 :info :write [16 44] :RequestTimeoutException INFO [2022-01-10 05:37:06,906] jepsen worker 50 - jepsen.util 410 :invoke :read [16 nil] INFO [2022-01-10 05:37:06,975] jepsen worker 1 - jepsen.util 271 :info :write [0 49] :RequestTimeoutException INFO [2022-01-10 05:37:06,976] jepsen worker 1 - jepsen.util 361 :invoke :write [0 1] INFO [2022-01-10 05:37:07,065] jepsen worker 52 - jepsen.util 322 :info :write [17 43] :RequestTimeoutException INFO [2022-01-10 05:37:07,066] jepsen worker 52 - jepsen.util 412 :invoke :read [17 nil] INFO [2022-01-10 05:37:07,451] jepsen worker 76 - jepsen.util 346 :ok :read [25 :nil] INFO [2022-01-10 05:37:07,451] jepsen worker 75 - jepsen.util 345 :ok :read [25 :nil] INFO [2022-01-10 05:37:07,452] jepsen worker 75 - jepsen.util 345 :invoke :read [25 nil] INFO [2022-01-10 05:37:07,452] jepsen worker 76 - jepsen.util 346 :invoke :read [25 nil] INFO [2022-01-10 05:37:07,524] jepsen worker 30 - jepsen.util 300 :ok :read [10 36] INFO [2022-01-10 05:37:07,524] jepsen worker 30 - jepsen.util 300 :invoke :read [10 nil] INFO [2022-01-10 05:37:07,600] jepsen worker 31 - jepsen.util 391 :info :write [10 41] :RequestTimeoutException INFO [2022-01-10 05:37:07,601] jepsen worker 31 - jepsen.util 481 :invoke :write [10 42] INFO [2022-01-10 05:37:07,621] jepsen worker 32 - jepsen.util 392 :ok :read [10 36] INFO [2022-01-10 05:37:07,622] jepsen worker 32 - jepsen.util 392 :invoke :write [10 41] INFO [2022-01-10 05:37:07,695] jepsen worker 47 - jepsen.util 317 :info :write [15 30] :RequestTimeoutException INFO [2022-01-10 05:37:07,696] jepsen worker 47 - jepsen.util 407 :invoke :write [15 2] INFO [2022-01-10 05:37:07,807] jepsen worker 76 - jepsen.util 346 :ok :read [25 :nil] INFO [2022-01-10 05:37:07,807] jepsen worker 75 - jepsen.util 345 :ok :read [25 :nil] INFO [2022-01-10 05:37:07,808] jepsen worker 75 - jepsen.util 345 :invoke :read [25 nil] INFO [2022-01-10 05:37:07,808] jepsen worker 76 - jepsen.util 346 :invoke :read [25 nil] INFO [2022-01-10 05:37:07,898] jepsen worker 17 - jepsen.util 197 :ok :write [5 42] INFO [2022-01-10 05:37:07,899] jepsen worker 17 - jepsen.util 197 :invoke :read [5 nil] INFO [2022-01-10 05:37:07,945] jepsen worker 12 - jepsen.util 282 :ok :read [4 :nil] INFO [2022-01-10 05:37:07,945] jepsen worker 36 - jepsen.util 216 :ok :read [12 :nil] INFO [2022-01-10 05:37:07,945] jepsen worker 12 - jepsen.util 282 :invoke :write [4 36] INFO [2022-01-10 05:37:07,946] jepsen worker 36 - jepsen.util 216 :invoke :read [12 nil] INFO [2022-01-10 05:37:08,065] jepsen worker 30 - jepsen.util 300 :ok :read [10 36] INFO [2022-01-10 05:37:08,066] jepsen worker 30 - jepsen.util 300 :invoke :read [10 nil] INFO [2022-01-10 05:37:08,067] jepsen worker 17 - jepsen.util 197 :ok :read [5 42] INFO [2022-01-10 05:37:08,067] jepsen worker 17 - jepsen.util 197 :invoke :read [5 nil] INFO [2022-01-10 05:37:08,163] jepsen worker 17 - jepsen.util 197 :ok :read [5 42] INFO [2022-01-10 05:37:08,164] jepsen worker 17 - jepsen.util 197 :invoke :write [5 39] INFO [2022-01-10 05:37:08,240] jepsen worker 76 - jepsen.util 346 :ok :read [25 :nil] INFO [2022-01-10 05:37:08,240] jepsen worker 75 - jepsen.util 345 :ok :read [25 :nil] INFO [2022-01-10 05:37:08,241] jepsen worker 76 - jepsen.util 346 :invoke :read [25 nil] INFO [2022-01-10 05:37:08,241] jepsen worker 75 - jepsen.util 345 :invoke :read [25 nil] INFO [2022-01-10 05:37:08,340] jepsen worker 85 - jepsen.util 265 :info :write [28 18] :RequestTimeoutException INFO [2022-01-10 05:37:08,341] jepsen worker 85 - jepsen.util 355 :invoke :write [28 4] INFO [2022-01-10 05:37:08,375] jepsen worker 51 - jepsen.util 231 :ok :read [17 37] INFO [2022-01-10 05:37:08,376] jepsen worker 51 - jepsen.util 231 :invoke :write [17 11] INFO [2022-01-10 05:37:08,380] jepsen worker 56 - jepsen.util 416 :ok :read [18 14] INFO [2022-01-10 05:37:08,380] jepsen worker 56 - jepsen.util 416 :invoke :read [18 nil] INFO [2022-01-10 05:37:08,430] jepsen worker 25 - jepsen.util 295 :ok :read [8 :nil] INFO [2022-01-10 05:37:08,431] jepsen worker 25 - jepsen.util 295 :invoke :read [8 nil] INFO [2022-01-10 05:37:08,434] jepsen worker 75 - jepsen.util 345 :ok :read [25 :nil] INFO [2022-01-10 05:37:08,434] jepsen worker 76 - jepsen.util 346 :ok :read [25 :nil] INFO [2022-01-10 05:37:08,435] jepsen worker 76 - jepsen.util 346 :invoke :read [25 nil] INFO [2022-01-10 05:37:08,443] jepsen worker 75 - jepsen.util 345 :invoke :read [25 nil] INFO [2022-01-10 05:37:08,499] jepsen worker 30 - jepsen.util 300 :ok :read [10 36] INFO [2022-01-10 05:37:08,500] jepsen worker 30 - jepsen.util 300 :invoke :write [10 37] INFO [2022-01-10 05:37:08,531] jepsen worker 76 - jepsen.util 346 :ok :read [25 :nil] INFO [2022-01-10 05:37:08,532] jepsen worker 76 - jepsen.util 346 :invoke :write [25 48] INFO [2022-01-10 05:37:08,532] jepsen worker 75 - jepsen.util 345 :ok :read [25 :nil] INFO [2022-01-10 05:37:08,533] jepsen worker 75 - jepsen.util 345 :invoke :read [25 nil] INFO [2022-01-10 05:37:08,612] jepsen worker nemesis - couchbase.util Rebalancing nodes (172.23.107.227) out of cluster INFO [2022-01-10 05:37:08,670] jepsen worker 75 - jepsen.util 345 :ok :read [25 :nil] INFO [2022-01-10 05:37:08,671] jepsen worker 75 - jepsen.util 345 :invoke :write [25 37] INFO [2022-01-10 05:37:08,731] jepsen worker 62 - jepsen.util 422 :info :write [20 32] :RequestTimeoutException INFO [2022-01-10 05:37:08,731] jepsen worker 62 - jepsen.util 512 :invoke :read [20 nil] INFO [2022-01-10 05:37:08,786] jepsen worker 89 - jepsen.util 449 :info :write [29 46] :RequestTimeoutException INFO [2022-01-10 05:37:08,787] jepsen worker 89 - jepsen.util 539 :invoke :write [29 38] INFO [2022-01-10 05:37:08,792] jepsen worker 45 - jepsen.util 315 :info :write [15 38] :RequestTimeoutException INFO [2022-01-10 05:37:08,793] jepsen worker 45 - jepsen.util 405 :invoke :write [15 40] INFO [2022-01-10 05:37:08,902] jepsen worker 44 - jepsen.util 404 :ok :read [14 14] INFO [2022-01-10 05:37:08,903] jepsen worker 44 - jepsen.util 404 :invoke :write [14 9] INFO [2022-01-10 05:37:09,194] jepsen worker 4 - jepsen.util 274 :ok :read [1 41] INFO [2022-01-10 05:37:09,195] jepsen worker 4 - jepsen.util 274 :invoke :write [1 2] INFO [2022-01-10 05:37:09,240] jepsen worker 86 - jepsen.util 356 :ok :read [28 :nil] INFO [2022-01-10 05:37:09,241] jepsen worker 86 - jepsen.util 356 :invoke :read [28 nil] INFO [2022-01-10 05:37:09,242] jepsen worker 53 - jepsen.util 323 :info :write [17 12] :RequestTimeoutException INFO [2022-01-10 05:37:09,243] jepsen worker 53 - jepsen.util 413 :invoke :read [17 nil] INFO [2022-01-10 05:37:09,268] jepsen worker 82 - jepsen.util 172 :ok :read [27 30] INFO [2022-01-10 05:37:09,268] jepsen worker 82 - jepsen.util 172 :invoke :write [27 32] INFO [2022-01-10 05:37:09,272] jepsen worker 43 - jepsen.util 313 :info :write [14 31] :RequestTimeoutException INFO [2022-01-10 05:37:09,273] jepsen worker 43 - jepsen.util 403 :invoke :read [14 nil] INFO [2022-01-10 05:37:09,291] jepsen worker 36 - jepsen.util 216 :ok :read [12 :nil] INFO [2022-01-10 05:37:09,292] jepsen worker 36 - jepsen.util 216 :invoke :read [12 nil] INFO [2022-01-10 05:37:09,356] jepsen worker 15 - jepsen.util 285 :ok :read [5 42] INFO [2022-01-10 05:37:09,357] jepsen worker 15 - jepsen.util 285 :invoke :read [5 nil] INFO [2022-01-10 05:37:09,385] jepsen worker 86 - jepsen.util 356 :ok :read [28 :nil] INFO [2022-01-10 05:37:09,386] jepsen worker 86 - jepsen.util 356 :invoke :read [28 nil] INFO [2022-01-10 05:37:09,397] jepsen worker 22 - jepsen.util 292 :info :write [7 25] :RequestTimeoutException INFO [2022-01-10 05:37:09,398] jepsen worker 22 - jepsen.util 382 :invoke :read [7 nil] INFO [2022-01-10 05:37:09,446] jepsen worker 86 - jepsen.util 356 :ok :read [28 :nil] INFO [2022-01-10 05:37:09,447] jepsen worker 86 - jepsen.util 356 :invoke :write [28 17] WARN [2022-01-10 05:37:09,526] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":389932,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":389932,"total_server_duration_us":17290,"operation_id":"0xa7c","timeout_ms":30000,"last_server_duration_us":17290,"total_duration_us":22903989},{"operation_name":"get","last_dispatch_duration_us":2857217,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2857217,"total_server_duration_us":8,"operation_id":"0xc5f","timeout_ms":30000,"last_server_duration_us":8,"total_duration_us":15668940},{"operation_name":"get","last_dispatch_duration_us":3847203,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3847203,"total_server_duration_us":202322,"operation_id":"0xc01","timeout_ms":30000,"last_server_duration_us":202322,"total_duration_us":12073155},{"operation_name":"upsert","last_dispatch_duration_us":7567510,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9909734,"total_server_duration_us":263890,"encode_duration_us":17534,"operation_id":"0xd7c","timeout_ms":10000,"last_server_duration_us":24755,"total_duration_us":10009709},{"operation_name":"upsert","last_dispatch_duration_us":10133133,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10133133,"total_server_duration_us":104387,"encode_duration_us":17738,"operation_id":"0xc54","timeout_ms":10000,"last_server_duration_us":104387,"total_duration_us":10009023},{"operation_name":"upsert","encode_duration_us":16476,"operation_id":"0xce3","timeout_ms":10000,"total_duration_us":10007192},{"operation_name":"upsert","last_dispatch_duration_us":141455,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9848173,"total_server_duration_us":465808,"encode_duration_us":19508,"operation_id":"0xd62","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10007058},{"operation_name":"upsert","last_dispatch_duration_us":7441117,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9851763,"total_server_duration_us":261371,"encode_duration_us":17341,"operation_id":"0xd7f","timeout_ms":10000,"last_server_duration_us":21731,"total_duration_us":10004237},{"operation_name":"upsert","last_dispatch_duration_us":640802,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10101158,"total_server_duration_us":418896,"encode_duration_us":18237,"operation_id":"0xd81","timeout_ms":10000,"last_server_duration_us":82835,"total_duration_us":10003543},{"operation_name":"upsert","last_dispatch_duration_us":3387755,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5868652,"total_server_duration_us":201245,"encode_duration_us":16471,"operation_id":"0xd88","timeout_ms":10000,"last_server_duration_us":83124,"total_duration_us":10002449}],"total_count":26}} INFO [2022-01-10 05:37:09,565] jepsen worker 35 - jepsen.util 395 :info :write [11 9] :RequestTimeoutException INFO [2022-01-10 05:37:09,566] jepsen worker 35 - jepsen.util 485 :invoke :read [11 nil] INFO [2022-01-10 05:37:09,661] jepsen worker 0 - jepsen.util 270 :ok :read [0 :nil] INFO [2022-01-10 05:37:09,661] jepsen worker 0 - jepsen.util 270 :invoke :read [0 nil] INFO [2022-01-10 05:37:09,670] jepsen worker 62 - jepsen.util 512 :ok :read [20 :nil] INFO [2022-01-10 05:37:09,671] jepsen worker 62 - jepsen.util 512 :invoke :write [20 23] INFO [2022-01-10 05:37:09,686] jepsen worker 3 - jepsen.util 363 :info :write [1 39] :RequestTimeoutException INFO [2022-01-10 05:37:09,687] jepsen worker 3 - jepsen.util 453 :invoke :read [1 nil] INFO [2022-01-10 05:37:09,830] jepsen worker 0 - jepsen.util 270 :ok :read [0 :nil] INFO [2022-01-10 05:37:09,834] jepsen worker 0 - jepsen.util 270 :invoke :write [0 35] INFO [2022-01-10 05:37:09,835] jepsen worker 5 - jepsen.util 275 :info :write [1 20] :RequestTimeoutException INFO [2022-01-10 05:37:09,835] jepsen worker 5 - jepsen.util 365 :invoke :read [1 nil] INFO [2022-01-10 05:37:09,842] jepsen worker 26 - jepsen.util 296 :info :write [8 30] :RequestTimeoutException INFO [2022-01-10 05:37:09,843] jepsen worker 26 - jepsen.util 386 :invoke :write [8 47] INFO [2022-01-10 05:37:09,872] jepsen worker 77 - jepsen.util 437 :info :write [25 33] :RequestTimeoutException INFO [2022-01-10 05:37:09,872] jepsen worker 77 - jepsen.util 527 :invoke :read [25 nil] INFO [2022-01-10 05:37:09,877] jepsen worker 38 - jepsen.util 308 :info :write [12 23] :RequestTimeoutException INFO [2022-01-10 05:37:09,877] jepsen worker 38 - jepsen.util 398 :invoke :write [12 41] INFO [2022-01-10 05:37:09,914] jepsen worker 66 - jepsen.util 246 :ok :read [22 :nil] INFO [2022-01-10 05:37:09,915] jepsen worker 66 - jepsen.util 246 :invoke :write [22 41] INFO [2022-01-10 05:37:10,016] jepsen worker 80 - jepsen.util 350 :info :write [26 12] :SyncWriteAmbiguous INFO [2022-01-10 05:37:10,017] jepsen worker 80 - jepsen.util 440 :invoke :write [26 40] INFO [2022-01-10 05:37:10,045] jepsen worker 54 - jepsen.util 234 :ok :read [18 14] INFO [2022-01-10 05:37:10,047] jepsen worker 54 - jepsen.util 234 :invoke :write [18 49] INFO [2022-01-10 05:37:10,091] jepsen worker 25 - jepsen.util 295 :ok :read [8 :nil] INFO [2022-01-10 05:37:10,091] jepsen worker 25 - jepsen.util 295 :invoke :write [8 11] INFO [2022-01-10 05:37:10,101] jepsen worker 73 - jepsen.util 253 :info :write [24 48] :RequestTimeoutException INFO [2022-01-10 05:37:10,102] jepsen worker 73 - jepsen.util 343 :invoke :read [24 nil] INFO [2022-01-10 05:37:10,142] jepsen worker 50 - jepsen.util 410 :ok :read [16 :nil] INFO [2022-01-10 05:37:10,142] jepsen worker 50 - jepsen.util 410 :invoke :write [16 25] INFO [2022-01-10 05:37:10,232] jepsen worker 29 - jepsen.util 209 :info :write [9 14] :RequestTimeoutException INFO [2022-01-10 05:37:10,233] jepsen worker 29 - jepsen.util 299 :invoke :write [9 33] INFO [2022-01-10 05:37:10,262] jepsen worker 84 - jepsen.util 354 :info :write [28 40] :RequestTimeoutException INFO [2022-01-10 05:37:10,262] jepsen worker 84 - jepsen.util 444 :invoke :write [28 37] INFO [2022-01-10 05:37:10,272] jepsen worker 28 - jepsen.util 298 :info :write [9 42] :RequestTimeoutException INFO [2022-01-10 05:37:10,273] jepsen worker 28 - jepsen.util 388 :invoke :write [9 41] INFO [2022-01-10 05:37:10,357] jepsen worker 9 - jepsen.util 369 :info :write [3 10] :RequestTimeoutException INFO [2022-01-10 05:37:10,357] jepsen worker 9 - jepsen.util 459 :invoke :write [3 10] INFO [2022-01-10 05:37:10,376] jepsen worker 5 - jepsen.util 365 :ok :read [1 41] INFO [2022-01-10 05:37:10,380] jepsen worker 5 - jepsen.util 365 :invoke :read [1 nil] INFO [2022-01-10 05:37:10,536] jepsen worker 52 - jepsen.util 412 :ok :read [17 37] INFO [2022-01-10 05:37:10,536] jepsen worker 52 - jepsen.util 412 :invoke :read [17 nil] INFO [2022-01-10 05:37:10,572] jepsen worker 37 - jepsen.util 397 :info :write [12 23] :RequestTimeoutException INFO [2022-01-10 05:37:10,573] jepsen worker 37 - jepsen.util 487 :invoke :write [12 38] INFO [2022-01-10 05:37:10,612] jepsen worker 72 - jepsen.util 432 :info :write [24 6] :RequestTimeoutException INFO [2022-01-10 05:37:10,612] jepsen worker 72 - jepsen.util 522 :invoke :write [24 3] INFO [2022-01-10 05:37:10,700] jepsen worker 87 - jepsen.util 357 :info :write [29 7] :RequestTimeoutException INFO [2022-01-10 05:37:10,700] jepsen worker 49 - jepsen.util 319 :info :write [16 35] :RequestTimeoutException INFO [2022-01-10 05:37:10,700] jepsen worker 2 - jepsen.util 182 :info :write [0 22] :RequestTimeoutException INFO [2022-01-10 05:37:10,701] jepsen worker 2 - jepsen.util 272 :invoke :read [0 nil] INFO [2022-01-10 05:37:10,701] jepsen worker 49 - jepsen.util 409 :invoke :read [16 nil] INFO [2022-01-10 05:37:10,702] jepsen worker 87 - jepsen.util 447 :invoke :read [29 nil] INFO [2022-01-10 05:37:10,710] jepsen worker 27 - jepsen.util 207 :info :write [9 22] :RequestTimeoutException INFO [2022-01-10 05:37:10,711] jepsen worker 27 - jepsen.util 297 :invoke :read [9 nil] INFO [2022-01-10 05:37:10,719] jepsen worker 39 - jepsen.util 219 :info :write [13 40] :RequestTimeoutException INFO [2022-01-10 05:37:10,720] jepsen worker 39 - jepsen.util 309 :invoke :read [13 nil] INFO [2022-01-10 05:37:10,846] jepsen worker 39 - jepsen.util 309 :ok :read [13 23] INFO [2022-01-10 05:37:10,848] jepsen worker 39 - jepsen.util 309 :invoke :write [13 14] INFO [2022-01-10 05:37:10,866] jepsen worker 5 - jepsen.util 365 :ok :read [1 41] INFO [2022-01-10 05:37:10,867] jepsen worker 5 - jepsen.util 365 :invoke :write [1 18] INFO [2022-01-10 05:37:10,875] jepsen worker 87 - jepsen.util 447 :ok :read [29 33] INFO [2022-01-10 05:37:10,876] jepsen worker 87 - jepsen.util 447 :invoke :read [29 nil] INFO [2022-01-10 05:37:10,976] jepsen worker 65 - jepsen.util 335 :info :write [21 39] :RequestTimeoutException INFO [2022-01-10 05:37:10,977] jepsen worker 65 - jepsen.util 425 :invoke :read [21 nil] INFO [2022-01-10 05:37:10,987] jepsen worker 60 - jepsen.util 420 :info :write [20 17] :RequestTimeoutException INFO [2022-01-10 05:37:10,987] jepsen worker 60 - jepsen.util 510 :invoke :read [20 nil] INFO [2022-01-10 05:37:11,030] jepsen worker 87 - jepsen.util 447 :ok :read [29 33] INFO [2022-01-10 05:37:11,031] jepsen worker 87 - jepsen.util 447 :invoke :write [29 9] INFO [2022-01-10 05:37:11,037] jepsen worker 52 - jepsen.util 412 :ok :read [17 37] INFO [2022-01-10 05:37:11,037] jepsen worker 52 - jepsen.util 412 :invoke :write [17 41] WARN [2022-01-10 05:37:11,132] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":23530776,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":23530776,"total_server_duration_us":38,"operation_id":"0xb3e","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":23532008},{"operation_name":"get","last_dispatch_duration_us":12856798,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12856798,"total_server_duration_us":303993,"operation_id":"0xd31","timeout_ms":30000,"last_server_duration_us":303993,"total_duration_us":12859456},{"operation_name":"get","last_dispatch_duration_us":12123388,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12123388,"total_server_duration_us":400187,"operation_id":"0xd30","timeout_ms":30000,"last_server_duration_us":400187,"total_duration_us":12129185},{"operation_name":"upsert","last_dispatch_duration_us":11366145,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11702999,"total_server_duration_us":133238,"encode_duration_us":16999,"operation_id":"0xced","timeout_ms":10000,"last_server_duration_us":76591,"total_duration_us":10008942},{"operation_name":"upsert","last_dispatch_duration_us":12411058,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12411058,"total_server_duration_us":229415,"encode_duration_us":18812,"operation_id":"0xd79","timeout_ms":10000,"last_server_duration_us":229415,"total_duration_us":10008467},{"operation_name":"upsert","last_dispatch_duration_us":362395,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1634636,"total_server_duration_us":286199,"encode_duration_us":17316,"operation_id":"0xcbb","timeout_ms":10000,"last_server_duration_us":95482,"total_duration_us":10007793},{"operation_name":"upsert","last_dispatch_duration_us":10134790,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10362512,"total_server_duration_us":9816898,"encode_duration_us":17574,"operation_id":"0xcb2","timeout_ms":10000,"last_server_duration_us":9693527,"total_duration_us":10007313},{"operation_name":"upsert","last_dispatch_duration_us":9801848,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11759964,"total_server_duration_us":541332,"encode_duration_us":21049,"operation_id":"0xcb4","timeout_ms":10000,"last_server_duration_us":120818,"total_duration_us":10007037},{"operation_name":"upsert","last_dispatch_duration_us":2977630,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":2977630,"total_server_duration_us":235681,"encode_duration_us":16421,"operation_id":"0xdfd","timeout_ms":10000,"last_server_duration_us":235681,"total_duration_us":10007025},{"operation_name":"upsert","last_dispatch_duration_us":10473227,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12626041,"total_server_duration_us":579143,"encode_duration_us":18328,"operation_id":"0xc8f","timeout_ms":10000,"last_server_duration_us":146065,"total_duration_us":10005948}],"total_count":19}} WARN [2022-01-10 05:37:11,132] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":25295861,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25295861,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9024634,"encode_duration_us":23522,"operation_id":"0x8fb","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10001322},{"operation_name":"upsert","last_dispatch_duration_us":10173942,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10173942,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9244265,"encode_duration_us":17118,"operation_id":"0xc7e","timeout_ms":10000,"last_server_duration_us":9244265,"total_duration_us":10003052},{"operation_name":"upsert","last_dispatch_duration_us":25288429,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25288429,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9102208,"encode_duration_us":16841,"operation_id":"0x90c","timeout_ms":10000,"last_server_duration_us":9102208,"total_duration_us":10006264},{"operation_name":"upsert","last_dispatch_duration_us":9569091,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12826103,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9546205,"encode_duration_us":16321,"operation_id":"0xc02","timeout_ms":10000,"last_server_duration_us":9342002,"total_duration_us":10005527},{"operation_name":"upsert","last_dispatch_duration_us":12634816,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13193425,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9857480,"encode_duration_us":23420,"operation_id":"0xc4f","timeout_ms":10000,"last_server_duration_us":9472635,"total_duration_us":10005229}],"total_count":5}} WARN [2022-01-10 05:37:11,132] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16812497,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":16812497,"total_server_duration_us":32,"operation_id":"0xb53","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":16813959},{"operation_name":"upsert","last_dispatch_duration_us":596749,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4481675,"total_server_duration_us":513506,"encode_duration_us":16692,"operation_id":"0xded","timeout_ms":10000,"last_server_duration_us":21895,"total_duration_us":10010093},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":22800,"operation_id":"0xcc0","timeout_ms":10000,"total_duration_us":10009854},{"operation_name":"upsert","last_dispatch_duration_us":871469,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2125256,"total_server_duration_us":310862,"encode_duration_us":23354,"operation_id":"0xce1","timeout_ms":10000,"last_server_duration_us":195413,"total_duration_us":10009555},{"operation_name":"upsert","last_dispatch_duration_us":2837025,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":2837025,"total_server_duration_us":205709,"encode_duration_us":29450,"operation_id":"0xdfe","timeout_ms":10000,"last_server_duration_us":205709,"total_duration_us":10009133},{"operation_name":"upsert","last_dispatch_duration_us":32825,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":529239,"total_server_duration_us":158841,"encode_duration_us":28215,"operation_id":"0xd04","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10009029},{"operation_name":"upsert","last_dispatch_duration_us":8527981,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9336115,"total_server_duration_us":228585,"encode_duration_us":18666,"operation_id":"0xd01","timeout_ms":10000,"last_server_duration_us":22720,"total_duration_us":10007343},{"operation_name":"upsert","last_dispatch_duration_us":1259589,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4906427,"total_server_duration_us":406364,"encode_duration_us":17268,"operation_id":"0xe00","timeout_ms":10000,"last_server_duration_us":28218,"total_duration_us":10006030},{"operation_name":"upsert","last_dispatch_duration_us":36706,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":443771,"total_server_duration_us":144921,"encode_duration_us":16932,"operation_id":"0xd02","timeout_ms":10000,"last_server_duration_us":23390,"total_duration_us":10005354},{"operation_name":"upsert","last_dispatch_duration_us":6611688,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7101241,"total_server_duration_us":168285,"encode_duration_us":17363,"operation_id":"0xd09","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10001581}],"total_count":20}} WARN [2022-01-10 05:37:11,132] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17507958,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":20015452,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9523060,"encode_duration_us":16466,"operation_id":"0xa40","timeout_ms":10000,"last_server_duration_us":9465056,"total_duration_us":10007144}],"total_count":1}} WARN [2022-01-10 05:37:11,133] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":29596693,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":29596693,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9033121,"encode_duration_us":17988,"operation_id":"0xa7e","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10004173},{"operation_name":"upsert","last_dispatch_duration_us":10389833,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14371244,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":10024397,"encode_duration_us":18952,"operation_id":"0xccb","timeout_ms":10000,"last_server_duration_us":9340925,"total_duration_us":10004221},{"operation_name":"upsert","last_dispatch_duration_us":10301071,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11059202,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9952487,"encode_duration_us":16473,"operation_id":"0xd87","timeout_ms":10000,"last_server_duration_us":9617112,"total_duration_us":10008354},{"operation_name":"upsert","last_dispatch_duration_us":10044993,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12556408,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9927783,"encode_duration_us":16931,"operation_id":"0xccc","timeout_ms":10000,"last_server_duration_us":9467221,"total_duration_us":10004255}],"total_count":4}} WARN [2022-01-10 05:37:11,232] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":92183,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":306286,"total_server_duration_us":63587,"encode_duration_us":16933,"operation_id":"0xcbf","timeout_ms":10000,"last_server_duration_us":20680,"total_duration_us":10010502},{"operation_name":"upsert","last_dispatch_duration_us":9357323,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10705648,"total_server_duration_us":450301,"encode_duration_us":24441,"operation_id":"0xcc2","timeout_ms":10000,"last_server_duration_us":86182,"total_duration_us":10007732},{"operation_name":"upsert","last_dispatch_duration_us":2352117,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2352117,"total_server_duration_us":90182,"encode_duration_us":16928,"operation_id":"0xd5a","timeout_ms":10000,"last_server_duration_us":90182,"total_duration_us":10007617},{"operation_name":"upsert","last_dispatch_duration_us":10579058,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10951823,"total_server_duration_us":127993,"encode_duration_us":16633,"operation_id":"0xcbc","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10007210},{"operation_name":"upsert","last_dispatch_duration_us":11970959,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12399839,"total_server_duration_us":136400,"encode_duration_us":16603,"operation_id":"0xcbd","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10007183},{"operation_name":"upsert","last_dispatch_duration_us":114191,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8329128,"total_server_duration_us":643912,"encode_duration_us":16631,"operation_id":"0xdc8","timeout_ms":10000,"last_server_duration_us":21001,"total_duration_us":10005364},{"operation_name":"upsert","last_dispatch_duration_us":2005493,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2005493,"total_server_duration_us":89883,"encode_duration_us":16332,"operation_id":"0xe21","timeout_ms":10000,"last_server_duration_us":89883,"total_duration_us":10001984},{"operation_name":"upsert","last_dispatch_duration_us":10988062,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10988062,"total_server_duration_us":312064,"encode_duration_us":16333,"operation_id":"0xcca","timeout_ms":10000,"last_server_duration_us":312064,"total_duration_us":10001797},{"operation_name":"upsert","last_dispatch_duration_us":9521532,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9521532,"total_server_duration_us":9316183,"encode_duration_us":16579,"operation_id":"0xcb7","timeout_ms":10000,"last_server_duration_us":9316183,"total_duration_us":9525524},{"operation_name":"get","last_dispatch_duration_us":9224000,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9224000,"total_server_duration_us":27,"operation_id":"0xd0c","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":9224503}],"total_count":17}} WARN [2022-01-10 05:37:11,233] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":481112,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":1994150,"total_server_duration_us":860811,"encode_duration_us":16443,"operation_id":"0xdd9","timeout_ms":10000,"last_server_duration_us":387864,"total_duration_us":10031638},{"operation_name":"upsert","last_dispatch_duration_us":10139307,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12162672,"total_server_duration_us":9526148,"encode_duration_us":16459,"operation_id":"0xdb2","timeout_ms":10000,"last_server_duration_us":9300061,"total_duration_us":10008793},{"operation_name":"upsert","last_dispatch_duration_us":7100749,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11797230,"total_server_duration_us":464890,"encode_duration_us":16713,"operation_id":"0xdc3","timeout_ms":10000,"last_server_duration_us":128891,"total_duration_us":10006976},{"operation_name":"upsert","last_dispatch_duration_us":1602189,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2786446,"total_server_duration_us":289633,"encode_duration_us":16384,"operation_id":"0xddf","timeout_ms":10000,"last_server_duration_us":116115,"total_duration_us":10006852},{"operation_name":"upsert","last_dispatch_duration_us":165515,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9561284,"total_server_duration_us":104506,"encode_duration_us":16599,"operation_id":"0xdf9","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10006380},{"operation_name":"upsert","last_dispatch_duration_us":12778153,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":12778153,"total_server_duration_us":9028877,"encode_duration_us":17797,"operation_id":"0xd63","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10005871},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":16815,"operation_id":"0xdfa","timeout_ms":10000,"total_duration_us":10005307},{"operation_name":"upsert","last_dispatch_duration_us":1285172,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2585682,"total_server_duration_us":279500,"encode_duration_us":16635,"operation_id":"0xddd","timeout_ms":10000,"last_server_duration_us":130113,"total_duration_us":10004995},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":17527,"operation_id":"0xdfb","timeout_ms":10000,"total_duration_us":10001519},{"operation_name":"upsert","last_dispatch_duration_us":546164,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":2687319,"total_server_duration_us":1157432,"encode_duration_us":26397,"operation_id":"0xdc2","timeout_ms":10000,"last_server_duration_us":153864,"total_duration_us":10001461}],"total_count":15}} WARN [2022-01-10 05:37:11,333] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12002198,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12002198,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9192889,"encode_duration_us":20923,"operation_id":"0xd0a","timeout_ms":10000,"last_server_duration_us":9192889,"total_duration_us":10002262},{"operation_name":"upsert","last_dispatch_duration_us":27344978,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":27344978,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9034183,"encode_duration_us":16505,"operation_id":"0xaca","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10004164},{"operation_name":"upsert","last_dispatch_duration_us":10064944,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12344722,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9756354,"encode_duration_us":16805,"operation_id":"0xc4e","timeout_ms":10000,"last_server_duration_us":9657471,"total_duration_us":10006187},{"operation_name":"upsert","last_dispatch_duration_us":10134790,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10362512,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9816898,"encode_duration_us":17574,"operation_id":"0xcb2","timeout_ms":10000,"last_server_duration_us":9693527,"total_duration_us":10007313}],"total_count":4}} WARN [2022-01-10 05:37:11,333] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12778153,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12778153,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9028877,"encode_duration_us":17797,"operation_id":"0xd63","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10005871},{"operation_name":"upsert","last_dispatch_duration_us":10139307,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12162672,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9526148,"encode_duration_us":16459,"operation_id":"0xdb2","timeout_ms":10000,"last_server_duration_us":9300061,"total_duration_us":10008793}],"total_count":2}} WARN [2022-01-10 05:37:11,334] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":110785,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":225012,"total_server_duration_us":39720,"encode_duration_us":17591,"operation_id":"0xe0d","timeout_ms":10000,"last_server_duration_us":20921,"total_duration_us":10010304},{"operation_name":"upsert","last_dispatch_duration_us":10301071,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":11059202,"total_server_duration_us":9952487,"encode_duration_us":16473,"operation_id":"0xd87","timeout_ms":10000,"last_server_duration_us":9617112,"total_duration_us":10008354},{"operation_name":"upsert","last_dispatch_duration_us":1773516,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":5444532,"total_server_duration_us":1120721,"encode_duration_us":17216,"operation_id":"0xc84","timeout_ms":10000,"last_server_duration_us":277180,"total_duration_us":10008170},{"operation_name":"upsert","last_dispatch_duration_us":1552825,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":4350378,"total_server_duration_us":936943,"encode_duration_us":18606,"operation_id":"0xd0f","timeout_ms":10000,"last_server_duration_us":350820,"total_duration_us":10007817},{"operation_name":"upsert","last_dispatch_duration_us":102878,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9188754,"total_server_duration_us":60016,"encode_duration_us":17087,"operation_id":"0xe0c","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10006269},{"operation_name":"upsert","last_dispatch_duration_us":9470729,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9470729,"total_server_duration_us":108237,"encode_duration_us":16489,"operation_id":"0xe03","timeout_ms":10000,"last_server_duration_us":108237,"total_duration_us":10006205},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":17224,"operation_id":"0xe04","timeout_ms":10000,"total_duration_us":10004599},{"operation_name":"upsert","last_dispatch_duration_us":10044993,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":12556408,"total_server_duration_us":9927783,"encode_duration_us":16931,"operation_id":"0xccc","timeout_ms":10000,"last_server_duration_us":9467221,"total_duration_us":10004255},{"operation_name":"upsert","last_dispatch_duration_us":10389833,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":14371244,"total_server_duration_us":10024397,"encode_duration_us":18952,"operation_id":"0xccb","timeout_ms":10000,"last_server_duration_us":9340925,"total_duration_us":10004221},{"operation_name":"upsert","last_dispatch_duration_us":8822085,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":13358162,"total_server_duration_us":1518591,"encode_duration_us":21429,"operation_id":"0xd28","timeout_ms":10000,"last_server_duration_us":22887,"total_duration_us":10003130}],"total_count":25}} INFO [2022-01-10 05:37:11,480] jepsen worker 65 - jepsen.util 425 :ok :read [21 7] INFO [2022-01-10 05:37:11,480] jepsen worker 65 - jepsen.util 425 :invoke :read [21 nil] INFO [2022-01-10 05:37:11,486] jepsen worker 73 - jepsen.util 343 :ok :read [24 33] INFO [2022-01-10 05:37:11,487] jepsen worker 73 - jepsen.util 343 :invoke :write [24 6] INFO [2022-01-10 05:37:11,602] jepsen worker 35 - jepsen.util 485 :ok :read [11 21] INFO [2022-01-10 05:37:11,602] jepsen worker 35 - jepsen.util 485 :invoke :read [11 nil] INFO [2022-01-10 05:37:11,607] jepsen worker 11 - jepsen.util 281 :ok :read [3 2] INFO [2022-01-10 05:37:11,607] jepsen worker 11 - jepsen.util 281 :invoke :read [3 nil] INFO [2022-01-10 05:37:11,749] jepsen worker 65 - jepsen.util 425 :ok :read [21 7] INFO [2022-01-10 05:37:11,750] jepsen worker 65 - jepsen.util 425 :invoke :read [21 nil] INFO [2022-01-10 05:37:11,831] jepsen worker 10 - jepsen.util 370 :ok :read [3 2] INFO [2022-01-10 05:37:11,832] jepsen worker 10 - jepsen.util 370 :invoke :write [3 37] INFO [2022-01-10 05:37:11,963] jepsen worker 65 - jepsen.util 425 :ok :read [21 7] INFO [2022-01-10 05:37:11,964] jepsen worker 65 - jepsen.util 425 :invoke :write [21 23] INFO [2022-01-10 05:37:11,973] jepsen worker 19 - jepsen.util 379 :ok :read [6 25] INFO [2022-01-10 05:37:11,973] jepsen worker 19 - jepsen.util 379 :invoke :write [6 16] INFO [2022-01-10 05:37:11,991] jepsen worker 42 - jepsen.util 222 :info :write [14 20] :RequestTimeoutException INFO [2022-01-10 05:37:11,992] jepsen worker 42 - jepsen.util 312 :invoke :read [14 nil] INFO [2022-01-10 05:37:12,032] jepsen worker 70 - jepsen.util 430 :info :write [23 30] :RequestTimeoutException INFO [2022-01-10 05:37:12,033] jepsen worker 70 - jepsen.util 520 :invoke :read [23 nil] INFO [2022-01-10 05:37:12,154] jepsen worker 70 - jepsen.util 520 :ok :read [23 :nil] INFO [2022-01-10 05:37:12,155] jepsen worker 70 - jepsen.util 520 :invoke :write [23 27] INFO [2022-01-10 05:37:12,178] jepsen worker 3 - jepsen.util 453 :ok :read [1 41] INFO [2022-01-10 05:37:12,179] jepsen worker 3 - jepsen.util 453 :invoke :write [1 0] INFO [2022-01-10 05:37:12,329] jepsen worker 46 - jepsen.util 316 :ok :read [15 :nil] INFO [2022-01-10 05:37:12,330] jepsen worker 46 - jepsen.util 316 :invoke :read [15 nil] INFO [2022-01-10 05:37:12,360] jepsen worker 16 - jepsen.util 286 :info :write [5 44] :RequestTimeoutException INFO [2022-01-10 05:37:12,362] jepsen worker 16 - jepsen.util 376 :invoke :write [5 33] INFO [2022-01-10 05:37:12,367] jepsen worker 59 - jepsen.util 419 :info :write [19 48] :RequestTimeoutException INFO [2022-01-10 05:37:12,368] jepsen worker 59 - jepsen.util 509 :invoke :write [19 41] INFO [2022-01-10 05:37:12,376] jepsen worker 11 - jepsen.util 281 :ok :read [3 2] INFO [2022-01-10 05:37:12,376] jepsen worker 11 - jepsen.util 281 :invoke :write [3 17] INFO [2022-01-10 05:37:12,502] jepsen worker 8 - jepsen.util 278 :info :write [2 39] :RequestTimeoutException INFO [2022-01-10 05:37:12,503] jepsen worker 8 - jepsen.util 368 :invoke :read [2 nil] INFO [2022-01-10 05:37:12,645] jepsen worker 63 - jepsen.util 423 :ok :read [21 7] INFO [2022-01-10 05:37:12,646] jepsen worker 63 - jepsen.util 423 :invoke :write [21 33] INFO [2022-01-10 05:37:12,648] jepsen worker 18 - jepsen.util 378 :ok :read [6 25] INFO [2022-01-10 05:37:12,649] jepsen worker 18 - jepsen.util 378 :invoke :write [6 31] INFO [2022-01-10 05:37:12,775] jepsen worker nemesis - couchbase.util Rebalance status: {} INFO [2022-01-10 05:37:12,775] jepsen worker nemesis - couchbase.util Rebalance check count: 0 INFO [2022-01-10 05:37:12,775] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:37:12,885] jepsen worker 33 - jepsen.util 213 :info :write [11 12] :RequestTimeoutException INFO [2022-01-10 05:37:12,886] jepsen worker 33 - jepsen.util 303 :invoke :read [11 nil] INFO [2022-01-10 05:37:13,081] jepsen worker 83 - jepsen.util 353 :info :write [27 35] :RequestTimeoutException INFO [2022-01-10 05:37:13,082] jepsen worker 83 - jepsen.util 443 :invoke :write [27 28] INFO [2022-01-10 05:37:13,095] jepsen worker 22 - jepsen.util 382 :ok :read [7 :nil] INFO [2022-01-10 05:37:13,096] jepsen worker 22 - jepsen.util 382 :invoke :write [7 38] INFO [2022-01-10 05:37:13,318] jepsen worker 43 - jepsen.util 403 :ok :read [14 14] INFO [2022-01-10 05:37:13,318] jepsen worker 43 - jepsen.util 403 :invoke :write [14 4] INFO [2022-01-10 05:37:13,466] jepsen worker 8 - jepsen.util 368 :ok :read [2 21] INFO [2022-01-10 05:37:13,466] jepsen worker 42 - jepsen.util 312 :ok :read [14 14] INFO [2022-01-10 05:37:13,466] jepsen worker 8 - jepsen.util 368 :invoke :write [2 12] INFO [2022-01-10 05:37:13,467] jepsen worker 42 - jepsen.util 312 :invoke :write [14 15] INFO [2022-01-10 05:37:13,515] jepsen worker 21 - jepsen.util 381 :info :write [7 6] :RequestTimeoutException INFO [2022-01-10 05:37:13,516] jepsen worker 21 - jepsen.util 471 :invoke :write [7 24] INFO [2022-01-10 05:37:13,521] jepsen worker 88 - jepsen.util 358 :info :write [29 5] :RequestTimeoutException INFO [2022-01-10 05:37:13,521] jepsen worker 88 - jepsen.util 448 :invoke :write [29 31] INFO [2022-01-10 05:37:13,615] jepsen worker 46 - jepsen.util 316 :ok :read [15 :nil] INFO [2022-01-10 05:37:13,615] jepsen worker 46 - jepsen.util 316 :invoke :write [15 49] INFO [2022-01-10 05:37:13,717] jepsen worker 6 - jepsen.util 276 :info :write [2 2] :RequestTimeoutException INFO [2022-01-10 05:37:13,717] jepsen worker 6 - jepsen.util 366 :invoke :read [2 nil] INFO [2022-01-10 05:37:13,726] jepsen worker 71 - jepsen.util 341 :info :write [23 9] :RequestTimeoutException INFO [2022-01-10 05:37:13,727] jepsen worker 71 - jepsen.util 431 :invoke :write [23 24] INFO [2022-01-10 05:37:13,782] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:13,782] jepsen worker nemesis - couchbase.util Rebalance check count: 1 INFO [2022-01-10 05:37:13,782] jepsen worker nemesis - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 05:37:13,786] jepsen worker 78 - jepsen.util 348 :info :write [26 43] :RequestTimeoutException INFO [2022-01-10 05:37:13,787] jepsen worker 78 - jepsen.util 438 :invoke :write [26 21] INFO [2022-01-10 05:37:13,869] jepsen worker 81 - jepsen.util 441 :ok :read [27 30] INFO [2022-01-10 05:37:13,869] jepsen worker 81 - jepsen.util 441 :invoke :read [27 nil] INFO [2022-01-10 05:37:13,907] jepsen worker 27 - jepsen.util 297 :ok :read [9 22] INFO [2022-01-10 05:37:13,908] jepsen worker 27 - jepsen.util 297 :invoke :read [9 nil] INFO [2022-01-10 05:37:13,951] jepsen worker 81 - jepsen.util 441 :ok :read [27 30] INFO [2022-01-10 05:37:13,952] jepsen worker 81 - jepsen.util 441 :invoke :read [27 nil] INFO [2022-01-10 05:37:13,989] jepsen worker 27 - jepsen.util 297 :ok :read [9 22] INFO [2022-01-10 05:37:13,990] jepsen worker 27 - jepsen.util 297 :invoke :read [9 nil] INFO [2022-01-10 05:37:14,029] jepsen worker 81 - jepsen.util 441 :ok :read [27 30] INFO [2022-01-10 05:37:14,030] jepsen worker 81 - jepsen.util 441 :invoke :write [27 27] INFO [2022-01-10 05:37:14,058] jepsen worker 27 - jepsen.util 297 :ok :read [9 22] INFO [2022-01-10 05:37:14,058] jepsen worker 27 - jepsen.util 297 :invoke :write [9 22] INFO [2022-01-10 05:37:14,106] jepsen worker 61 - jepsen.util 421 :info :write [20 39] :RequestTimeoutException INFO [2022-01-10 05:37:14,108] jepsen worker 61 - jepsen.util 511 :invoke :write [20 32] INFO [2022-01-10 05:37:14,245] jepsen worker 58 - jepsen.util 328 :info :write [19 18] :RequestTimeoutException INFO [2022-01-10 05:37:14,246] jepsen worker 58 - jepsen.util 418 :invoke :read [19 nil] INFO [2022-01-10 05:37:14,730] jepsen worker 7 - jepsen.util 187 :info :write [2 10] :RequestTimeoutException INFO [2022-01-10 05:37:14,731] jepsen worker 7 - jepsen.util 277 :invoke :read [2 nil] INFO [2022-01-10 05:37:14,751] jepsen worker 35 - jepsen.util 485 :ok :read [11 21] INFO [2022-01-10 05:37:14,751] jepsen worker 35 - jepsen.util 485 :invoke :write [11 2] INFO [2022-01-10 05:37:14,754] jepsen worker 60 - jepsen.util 510 :ok :read [20 :nil] INFO [2022-01-10 05:37:14,754] jepsen worker 60 - jepsen.util 510 :invoke :read [20 nil] INFO [2022-01-10 05:37:14,789] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:14,789] jepsen worker nemesis - couchbase.util Rebalance check count: 2 INFO [2022-01-10 05:37:14,789] jepsen worker nemesis - couchbase.util Rebalance stuck count: 1 INFO [2022-01-10 05:37:15,209] jepsen worker 36 - jepsen.util 216 :ok :read [12 :nil] INFO [2022-01-10 05:37:15,209] jepsen worker 36 - jepsen.util 216 :invoke :read [12 nil] INFO [2022-01-10 05:37:15,222] jepsen worker 53 - jepsen.util 413 :ok :read [17 37] INFO [2022-01-10 05:37:15,223] jepsen worker 53 - jepsen.util 413 :invoke :read [17 nil] INFO [2022-01-10 05:37:15,345] jepsen worker 64 - jepsen.util 334 :info :write [21 13] :RequestTimeoutException INFO [2022-01-10 05:37:15,346] jepsen worker 64 - jepsen.util 424 :invoke :read [21 nil] INFO [2022-01-10 05:37:15,405] jepsen worker 69 - jepsen.util 429 :info :write [23 6] :RequestTimeoutException INFO [2022-01-10 05:37:15,406] jepsen worker 69 - jepsen.util 519 :invoke :read [23 nil] INFO [2022-01-10 05:37:15,576] jepsen worker 13 - jepsen.util 373 :info :write [4 45] :RequestTimeoutException INFO [2022-01-10 05:37:15,577] jepsen worker 13 - jepsen.util 463 :invoke :read [4 nil] INFO [2022-01-10 05:37:15,740] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000003","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:37:15,740] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000002","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:37:15,747] jepsen worker 24 - jepsen.util 294 :info :write [8 28] :RequestTimeoutException INFO [2022-01-10 05:37:15,748] jepsen worker 24 - jepsen.util 384 :invoke :read [8 nil] INFO [2022-01-10 05:37:15,787] jepsen worker 68 - jepsen.util 338 :ok :read [22 :nil] INFO [2022-01-10 05:37:15,787] jepsen worker 48 - jepsen.util 228 :ok :read [16 :nil] INFO [2022-01-10 05:37:15,788] jepsen worker 48 - jepsen.util 228 :invoke :write [16 18] INFO [2022-01-10 05:37:15,789] jepsen worker 68 - jepsen.util 338 :invoke :read [22 nil] INFO [2022-01-10 05:37:15,797] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:15,797] jepsen worker nemesis - couchbase.util Rebalance check count: 3 INFO [2022-01-10 05:37:15,797] jepsen worker nemesis - couchbase.util Rebalance stuck count: 2 INFO [2022-01-10 05:37:15,803] jepsen worker 41 - jepsen.util 311 :ok :read [13 23] INFO [2022-01-10 05:37:15,803] jepsen worker 41 - jepsen.util 311 :invoke :write [13 18] INFO [2022-01-10 05:37:15,812] jepsen worker 15 - jepsen.util 285 :ok :read [5 42] INFO [2022-01-10 05:37:15,813] jepsen worker 15 - jepsen.util 285 :invoke :write [5 49] INFO [2022-01-10 05:37:15,823] jepsen worker 2 - jepsen.util 272 :ok :read [0 :nil] INFO [2022-01-10 05:37:15,824] jepsen worker 2 - jepsen.util 272 :invoke :read [0 nil] INFO [2022-01-10 05:37:15,824] jepsen worker 68 - jepsen.util 338 :ok :read [22 :nil] INFO [2022-01-10 05:37:15,825] jepsen worker 68 - jepsen.util 338 :invoke :read [22 nil] INFO [2022-01-10 05:37:15,840] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000005","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:37:15,840] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000006","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:37:15,840] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000004","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:37:15,857] jepsen worker 68 - jepsen.util 338 :ok :read [22 :nil] INFO [2022-01-10 05:37:15,858] jepsen worker 2 - jepsen.util 272 :ok :read [0 :nil] INFO [2022-01-10 05:37:15,858] jepsen worker 68 - jepsen.util 338 :invoke :write [22 1] INFO [2022-01-10 05:37:15,859] jepsen worker 2 - jepsen.util 272 :invoke :read [0 nil] INFO [2022-01-10 05:37:15,921] jepsen worker 2 - jepsen.util 272 :ok :read [0 :nil] INFO [2022-01-10 05:37:15,922] jepsen worker 2 - jepsen.util 272 :invoke :read [0 nil] INFO [2022-01-10 05:37:15,924] jepsen worker 56 - jepsen.util 416 :ok :read [18 14] INFO [2022-01-10 05:37:15,924] jepsen worker 56 - jepsen.util 416 :invoke :write [18 8] INFO [2022-01-10 05:37:16,004] jepsen worker 33 - jepsen.util 303 :ok :read [11 21] INFO [2022-01-10 05:37:16,005] jepsen worker 33 - jepsen.util 303 :invoke :write [11 11] INFO [2022-01-10 05:37:16,017] jepsen worker 34 - jepsen.util 394 :info :write [11 16] :RequestTimeoutException INFO [2022-01-10 05:37:16,018] jepsen worker 34 - jepsen.util 484 :invoke :write [11 35] INFO [2022-01-10 05:37:16,030] jepsen worker 60 - jepsen.util 510 :ok :read [20 :nil] INFO [2022-01-10 05:37:16,031] jepsen worker 60 - jepsen.util 510 :invoke :write [20 35] INFO [2022-01-10 05:37:16,035] jepsen worker 23 - jepsen.util 293 :info :write [7 3] :RequestTimeoutException INFO [2022-01-10 05:37:16,036] jepsen worker 23 - jepsen.util 383 :invoke :read [7 nil] INFO [2022-01-10 05:37:16,045] jepsen worker 20 - jepsen.util 110 :info :write [6 13] :RequestTimeoutException INFO [2022-01-10 05:37:16,045] jepsen worker 20 - jepsen.util 200 :invoke :write [6 12] INFO [2022-01-10 05:37:16,073] jepsen worker 2 - jepsen.util 272 :ok :read [0 :nil] INFO [2022-01-10 05:37:16,074] jepsen worker 2 - jepsen.util 272 :invoke :read [0 nil] INFO [2022-01-10 05:37:16,142] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xefe9906200000001","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:37:16,188] jepsen worker 2 - jepsen.util 272 :ok :read [0 :nil] INFO [2022-01-10 05:37:16,189] jepsen worker 2 - jepsen.util 272 :invoke :read [0 nil] INFO [2022-01-10 05:37:16,192] jepsen worker 7 - jepsen.util 277 :ok :read [2 21] INFO [2022-01-10 05:37:16,193] jepsen worker 7 - jepsen.util 277 :invoke :read [2 nil] INFO [2022-01-10 05:37:16,281] jepsen worker 2 - jepsen.util 272 :ok :read [0 :nil] INFO [2022-01-10 05:37:16,282] jepsen worker 2 - jepsen.util 272 :invoke :write [0 45] INFO [2022-01-10 05:37:16,461] jepsen worker 14 - jepsen.util 374 :info :write [4 8] :RequestTimeoutException INFO [2022-01-10 05:37:16,462] jepsen worker 14 - jepsen.util 464 :invoke :read [4 nil] INFO [2022-01-10 05:37:16,484] jepsen worker 57 - jepsen.util 327 :info :write [19 46] :RequestTimeoutException INFO [2022-01-10 05:37:16,486] jepsen worker 57 - jepsen.util 417 :invoke :write [19 37] INFO [2022-01-10 05:37:16,642] jepsen worker 40 - jepsen.util 310 :info :write [13 22] :RequestTimeoutException INFO [2022-01-10 05:37:16,642] jepsen worker 40 - jepsen.util 400 :invoke :read [13 nil] INFO [2022-01-10 05:37:16,725] jepsen worker 74 - jepsen.util 434 :info :write [24 29] :RequestTimeoutException INFO [2022-01-10 05:37:16,726] jepsen worker 74 - jepsen.util 524 :invoke :write [24 27] INFO [2022-01-10 05:37:16,784] jepsen worker 23 - jepsen.util 383 :ok :read [7 :nil] INFO [2022-01-10 05:37:16,785] jepsen worker 23 - jepsen.util 383 :invoke :read [7 nil] INFO [2022-01-10 05:37:16,805] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:16,806] jepsen worker nemesis - couchbase.util Rebalance check count: 4 INFO [2022-01-10 05:37:16,806] jepsen worker nemesis - couchbase.util Rebalance stuck count: 3 INFO [2022-01-10 05:37:16,980] jepsen worker 30 - jepsen.util 300 :ok :write [10 37] INFO [2022-01-10 05:37:16,981] jepsen worker 30 - jepsen.util 300 :invoke :read [10 nil] INFO [2022-01-10 05:37:16,985] jepsen worker 24 - jepsen.util 384 :ok :read [8 :nil] INFO [2022-01-10 05:37:16,985] jepsen worker 24 - jepsen.util 384 :invoke :read [8 nil] INFO [2022-01-10 05:37:17,002] jepsen worker 1 - jepsen.util 361 :info :write [0 1] :RequestTimeoutException INFO [2022-01-10 05:37:17,003] jepsen worker 1 - jepsen.util 451 :invoke :read [0 nil] INFO [2022-01-10 05:37:17,240] jepsen worker 17 - jepsen.util 197 :info :write [5 39] :SyncWriteAmbiguous INFO [2022-01-10 05:37:17,241] jepsen worker 17 - jepsen.util 287 :invoke :write [5 21] INFO [2022-01-10 05:37:17,326] jepsen worker 7 - jepsen.util 277 :ok :read [2 21] INFO [2022-01-10 05:37:17,327] jepsen worker 7 - jepsen.util 277 :invoke :write [2 48] INFO [2022-01-10 05:37:17,356] jepsen worker 58 - jepsen.util 418 :ok :read [19 31] INFO [2022-01-10 05:37:17,356] jepsen worker 58 - jepsen.util 418 :invoke :read [19 nil] INFO [2022-01-10 05:37:17,394] jepsen worker 58 - jepsen.util 418 :ok :read [19 31] INFO [2022-01-10 05:37:17,395] jepsen worker 58 - jepsen.util 418 :invoke :write [19 22] INFO [2022-01-10 05:37:17,622] jepsen worker 31 - jepsen.util 481 :info :write [10 42] :RequestTimeoutException INFO [2022-01-10 05:37:17,623] jepsen worker 31 - jepsen.util 571 :invoke :write [10 17] INFO [2022-01-10 05:37:17,631] jepsen worker 24 - jepsen.util 384 :ok :read [8 :nil] INFO [2022-01-10 05:37:17,633] jepsen worker 24 - jepsen.util 384 :invoke :read [8 nil] INFO [2022-01-10 05:37:17,647] jepsen worker 32 - jepsen.util 392 :info :write [10 41] :RequestTimeoutException INFO [2022-01-10 05:37:17,648] jepsen worker 32 - jepsen.util 482 :invoke :read [10 nil] INFO [2022-01-10 05:37:17,725] jepsen worker 47 - jepsen.util 407 :info :write [15 2] :RequestTimeoutException INFO [2022-01-10 05:37:17,725] jepsen worker 47 - jepsen.util 497 :invoke :write [15 2] INFO [2022-01-10 05:37:17,815] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:17,815] jepsen worker nemesis - couchbase.util Rebalance check count: 5 INFO [2022-01-10 05:37:17,815] jepsen worker nemesis - couchbase.util Rebalance stuck count: 4 INFO [2022-01-10 05:37:17,934] jepsen worker 66 - jepsen.util 246 :ok :write [22 41] INFO [2022-01-10 05:37:17,934] jepsen worker 66 - jepsen.util 246 :invoke :write [22 14] INFO [2022-01-10 05:37:17,971] jepsen worker 12 - jepsen.util 282 :info :write [4 36] :RequestTimeoutException INFO [2022-01-10 05:37:17,972] jepsen worker 12 - jepsen.util 372 :invoke :write [4 0] INFO [2022-01-10 05:37:18,186] jepsen worker 24 - jepsen.util 384 :ok :read [8 :nil] INFO [2022-01-10 05:37:18,187] jepsen worker 24 - jepsen.util 384 :invoke :read [8 nil] INFO [2022-01-10 05:37:18,238] jepsen worker 77 - jepsen.util 527 :ok :read [25 :nil] INFO [2022-01-10 05:37:18,240] jepsen worker 76 - jepsen.util 346 :info :write [25 48] :SyncWriteAmbiguous INFO [2022-01-10 05:37:18,241] jepsen worker 77 - jepsen.util 527 :invoke :write [25 15] INFO [2022-01-10 05:37:18,241] jepsen worker 76 - jepsen.util 436 :invoke :write [25 26] INFO [2022-01-10 05:37:18,366] jepsen worker 85 - jepsen.util 355 :info :write [28 4] :RequestTimeoutException INFO [2022-01-10 05:37:18,367] jepsen worker 85 - jepsen.util 445 :invoke :read [28 nil] INFO [2022-01-10 05:37:18,396] jepsen worker 51 - jepsen.util 231 :info :write [17 11] :RequestTimeoutException INFO [2022-01-10 05:37:18,397] jepsen worker 51 - jepsen.util 321 :invoke :write [17 22] INFO [2022-01-10 05:37:18,695] jepsen worker 75 - jepsen.util 345 :info :write [25 37] :RequestTimeoutException INFO [2022-01-10 05:37:18,696] jepsen worker 75 - jepsen.util 435 :invoke :write [25 20] INFO [2022-01-10 05:37:18,812] jepsen worker 89 - jepsen.util 539 :info :write [29 38] :RequestTimeoutException INFO [2022-01-10 05:37:18,813] jepsen worker 89 - jepsen.util 629 :invoke :write [29 19] INFO [2022-01-10 05:37:18,816] jepsen worker 45 - jepsen.util 405 :info :write [15 40] :RequestTimeoutException INFO [2022-01-10 05:37:18,817] jepsen worker 45 - jepsen.util 495 :invoke :read [15 nil] INFO [2022-01-10 05:37:18,821] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:18,822] jepsen worker nemesis - couchbase.util Rebalance check count: 6 INFO [2022-01-10 05:37:18,822] jepsen worker nemesis - couchbase.util Rebalance stuck count: 5 INFO [2022-01-10 05:37:18,857] jepsen worker 24 - jepsen.util 384 :ok :read [8 :nil] INFO [2022-01-10 05:37:18,857] jepsen worker 24 - jepsen.util 384 :invoke :write [8 21] INFO [2022-01-10 05:37:18,862] jepsen worker 67 - jepsen.util 337 :ok :read [22 41] INFO [2022-01-10 05:37:18,862] jepsen worker 67 - jepsen.util 337 :invoke :read [22 nil] INFO [2022-01-10 05:37:18,910] jepsen worker 67 - jepsen.util 337 :ok :read [22 41] INFO [2022-01-10 05:37:18,911] jepsen worker 67 - jepsen.util 337 :invoke :write [22 43] INFO [2022-01-10 05:37:18,920] jepsen worker 79 - jepsen.util 439 :ok :read [26 10] INFO [2022-01-10 05:37:18,921] jepsen worker 79 - jepsen.util 439 :invoke :write [26 5] INFO [2022-01-10 05:37:18,935] jepsen worker 44 - jepsen.util 404 :info :write [14 9] :RequestTimeoutException INFO [2022-01-10 05:37:18,936] jepsen worker 44 - jepsen.util 494 :invoke :read [14 nil] INFO [2022-01-10 05:37:18,963] jepsen worker 45 - jepsen.util 495 :ok :read [15 :nil] INFO [2022-01-10 05:37:18,963] jepsen worker 85 - jepsen.util 445 :ok :read [28 :nil] INFO [2022-01-10 05:37:18,965] jepsen worker 85 - jepsen.util 445 :invoke :read [28 nil] INFO [2022-01-10 05:37:18,966] jepsen worker 45 - jepsen.util 495 :invoke :write [15 39] INFO [2022-01-10 05:37:18,966] jepsen worker 85 - jepsen.util 445 :ok :read [28 :nil] INFO [2022-01-10 05:37:18,967] jepsen worker 85 - jepsen.util 445 :invoke :write [28 16] INFO [2022-01-10 05:37:19,168] jepsen worker 55 - jepsen.util 415 :ok :read [18 14] INFO [2022-01-10 05:37:19,168] jepsen worker 55 - jepsen.util 415 :invoke :write [18 22] INFO [2022-01-10 05:37:19,220] jepsen worker 4 - jepsen.util 274 :info :write [1 2] :RequestTimeoutException INFO [2022-01-10 05:37:19,221] jepsen worker 4 - jepsen.util 364 :invoke :write [1 23] INFO [2022-01-10 05:37:19,292] jepsen worker 82 - jepsen.util 172 :info :write [27 32] :RequestTimeoutException INFO [2022-01-10 05:37:19,293] jepsen worker 82 - jepsen.util 262 :invoke :write [27 18] WARN [2022-01-10 05:37:19,448] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9438300,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11610643,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9640670,"encode_duration_us":20895,"operation_id":"0xeda","timeout_ms":10000,"last_server_duration_us":9229268,"total_duration_us":10006041}],"total_count":1}} INFO [2022-01-10 05:37:19,472] jepsen worker 86 - jepsen.util 356 :info :write [28 17] :RequestTimeoutException INFO [2022-01-10 05:37:19,473] jepsen worker 86 - jepsen.util 446 :invoke :read [28 nil] WARN [2022-01-10 05:37:19,549] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":213919,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1269653,"total_server_duration_us":423739,"encode_duration_us":20485,"operation_id":"0xf4e","timeout_ms":10000,"last_server_duration_us":42091,"total_duration_us":10010771},{"operation_name":"upsert","last_dispatch_duration_us":2192159,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3234180,"total_server_duration_us":396779,"encode_duration_us":17702,"operation_id":"0xf0c","timeout_ms":10000,"last_server_duration_us":39319,"total_duration_us":10010032},{"operation_name":"upsert","last_dispatch_duration_us":3354207,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7095683,"total_server_duration_us":375910,"encode_duration_us":17932,"operation_id":"0xdc7","timeout_ms":10000,"last_server_duration_us":53854,"total_duration_us":10008928},{"operation_name":"upsert","last_dispatch_duration_us":207462,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1000677,"total_server_duration_us":371892,"encode_duration_us":24742,"operation_id":"0xe20","timeout_ms":10000,"last_server_duration_us":126118,"total_duration_us":10007570},{"operation_name":"upsert","last_dispatch_duration_us":2750100,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2959282,"total_server_duration_us":193443,"encode_duration_us":17152,"operation_id":"0xedb","timeout_ms":10000,"last_server_duration_us":71096,"total_duration_us":10007376},{"operation_name":"upsert","last_dispatch_duration_us":9438300,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11610643,"total_server_duration_us":9640670,"encode_duration_us":20895,"operation_id":"0xeda","timeout_ms":10000,"last_server_duration_us":9229268,"total_duration_us":10006041},{"operation_name":"upsert","last_dispatch_duration_us":587767,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1636583,"total_server_duration_us":711593,"encode_duration_us":17616,"operation_id":"0xf35","timeout_ms":10000,"last_server_duration_us":454669,"total_duration_us":10005529},{"operation_name":"upsert","last_dispatch_duration_us":8083321,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12406948,"total_server_duration_us":284743,"encode_duration_us":16928,"operation_id":"0xea1","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10004887},{"operation_name":"upsert","last_dispatch_duration_us":1004438,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1361994,"total_server_duration_us":384773,"encode_duration_us":16686,"operation_id":"0xebf","timeout_ms":10000,"last_server_duration_us":361551,"total_duration_us":10004861},{"operation_name":"upsert","last_dispatch_duration_us":927813,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":5609306,"total_server_duration_us":1869973,"encode_duration_us":16294,"operation_id":"0xec0","timeout_ms":10000,"last_server_duration_us":358048,"total_duration_us":10004823}],"total_count":19}} INFO [2022-01-10 05:37:19,700] jepsen worker 62 - jepsen.util 512 :info :write [20 23] :RequestTimeoutException INFO [2022-01-10 05:37:19,702] jepsen worker 62 - jepsen.util 602 :invoke :write [20 5] INFO [2022-01-10 05:37:19,828] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:19,828] jepsen worker nemesis - couchbase.util Rebalance check count: 7 INFO [2022-01-10 05:37:19,835] jepsen worker nemesis - couchbase.util Rebalance stuck count: 6 INFO [2022-01-10 05:37:19,860] jepsen worker 0 - jepsen.util 270 :info :write [0 35] :RequestTimeoutException INFO [2022-01-10 05:37:19,861] jepsen worker 0 - jepsen.util 360 :invoke :write [0 31] INFO [2022-01-10 05:37:19,872] jepsen worker 26 - jepsen.util 386 :info :write [8 47] :RequestTimeoutException INFO [2022-01-10 05:37:19,873] jepsen worker 26 - jepsen.util 476 :invoke :write [8 23] INFO [2022-01-10 05:37:19,896] jepsen worker 38 - jepsen.util 398 :info :write [12 41] :RequestTimeoutException INFO [2022-01-10 05:37:19,898] jepsen worker 38 - jepsen.util 488 :invoke :write [12 9] INFO [2022-01-10 05:37:19,911] jepsen worker 6 - jepsen.util 366 :ok :read [2 21] INFO [2022-01-10 05:37:19,913] jepsen worker 6 - jepsen.util 366 :invoke :write [2 31] INFO [2022-01-10 05:37:20,051] jepsen worker 80 - jepsen.util 440 :info :write [26 40] :RequestTimeoutException INFO [2022-01-10 05:37:20,052] jepsen worker 80 - jepsen.util 530 :invoke :read [26 nil] INFO [2022-01-10 05:37:20,066] jepsen worker 54 - jepsen.util 234 :info :write [18 49] :RequestTimeoutException INFO [2022-01-10 05:37:20,067] jepsen worker 54 - jepsen.util 324 :invoke :read [18 nil] INFO [2022-01-10 05:37:20,086] jepsen worker 30 - jepsen.util 300 :ok :read [10 37] INFO [2022-01-10 05:37:20,086] jepsen worker 30 - jepsen.util 300 :invoke :write [10 45] INFO [2022-01-10 05:37:20,110] jepsen worker 25 - jepsen.util 295 :info :write [8 11] :RequestTimeoutException INFO [2022-01-10 05:37:20,112] jepsen worker 25 - jepsen.util 385 :invoke :write [8 15] INFO [2022-01-10 05:37:20,123] jepsen worker 64 - jepsen.util 424 :ok :read [21 7] INFO [2022-01-10 05:37:20,124] jepsen worker 64 - jepsen.util 424 :invoke :read [21 nil] INFO [2022-01-10 05:37:20,170] jepsen worker 50 - jepsen.util 410 :info :write [16 25] :RequestTimeoutException INFO [2022-01-10 05:37:20,171] jepsen worker 50 - jepsen.util 500 :invoke :write [16 41] INFO [2022-01-10 05:37:20,256] jepsen worker 29 - jepsen.util 299 :info :write [9 33] :RequestTimeoutException INFO [2022-01-10 05:37:20,257] jepsen worker 29 - jepsen.util 389 :invoke :write [9 37] INFO [2022-01-10 05:37:20,282] jepsen worker 84 - jepsen.util 444 :info :write [28 37] :RequestTimeoutException INFO [2022-01-10 05:37:20,282] jepsen worker 84 - jepsen.util 534 :invoke :write [28 39] INFO [2022-01-10 05:37:20,305] jepsen worker 28 - jepsen.util 388 :info :write [9 41] :RequestTimeoutException INFO [2022-01-10 05:37:20,306] jepsen worker 28 - jepsen.util 478 :invoke :write [9 9] INFO [2022-01-10 05:37:20,386] jepsen worker 9 - jepsen.util 459 :info :write [3 10] :RequestTimeoutException INFO [2022-01-10 05:37:20,387] jepsen worker 9 - jepsen.util 549 :invoke :read [3 nil] INFO [2022-01-10 05:37:20,601] jepsen worker 37 - jepsen.util 487 :info :write [12 38] :RequestTimeoutException INFO [2022-01-10 05:37:20,601] jepsen worker 37 - jepsen.util 577 :invoke :read [12 nil] INFO [2022-01-10 05:37:20,633] jepsen worker 72 - jepsen.util 522 :info :write [24 3] :RequestTimeoutException INFO [2022-01-10 05:37:20,633] jepsen worker 72 - jepsen.util 612 :invoke :write [24 26] INFO [2022-01-10 05:37:20,656] jepsen worker 64 - jepsen.util 424 :ok :read [21 7] INFO [2022-01-10 05:37:20,657] jepsen worker 64 - jepsen.util 424 :invoke :read [21 nil] INFO [2022-01-10 05:37:20,841] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:20,842] jepsen worker nemesis - couchbase.util Rebalance check count: 8 INFO [2022-01-10 05:37:20,842] jepsen worker nemesis - couchbase.util Rebalance stuck count: 7 INFO [2022-01-10 05:37:20,870] jepsen worker 39 - jepsen.util 309 :info :write [13 14] :RequestTimeoutException INFO [2022-01-10 05:37:20,871] jepsen worker 39 - jepsen.util 399 :invoke :write [13 20] INFO [2022-01-10 05:37:20,886] jepsen worker 5 - jepsen.util 365 :info :write [1 18] :RequestTimeoutException INFO [2022-01-10 05:37:20,887] jepsen worker 5 - jepsen.util 455 :invoke :write [1 26] WARN [2022-01-10 05:37:21,047] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":670177,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2726547,"total_server_duration_us":227558,"encode_duration_us":17394,"operation_id":"0xf7d","timeout_ms":10000,"last_server_duration_us":62412,"total_duration_us":10009977},{"operation_name":"upsert","last_dispatch_duration_us":11953275,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11953275,"total_server_duration_us":9024634,"encode_duration_us":19902,"operation_id":"0xe7b","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10009718},{"operation_name":"upsert","last_dispatch_duration_us":1746528,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":1746528,"total_server_duration_us":100753,"encode_duration_us":16573,"operation_id":"0xee3","timeout_ms":10000,"last_server_duration_us":100753,"total_duration_us":10007371},{"operation_name":"upsert","last_dispatch_duration_us":2867005,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":4204628,"total_server_duration_us":628475,"encode_duration_us":19781,"operation_id":"0xf23","timeout_ms":10000,"last_server_duration_us":363714,"total_duration_us":10004834},{"operation_name":"upsert","last_dispatch_duration_us":3759375,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":3759375,"total_server_duration_us":206560,"encode_duration_us":16615,"operation_id":"0xf07","timeout_ms":10000,"last_server_duration_us":206560,"total_duration_us":10002373},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":16671,"operation_id":"0xee6","timeout_ms":10000,"total_duration_us":10001871},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":16334,"operation_id":"0xf8a","timeout_ms":10000,"total_duration_us":10001787},{"operation_name":"upsert","last_dispatch_duration_us":7486668,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":11341559,"total_server_duration_us":86284,"encode_duration_us":16732,"operation_id":"0xe81","timeout_ms":10000,"last_server_duration_us":67485,"total_duration_us":10001373},{"operation_name":"upsert","last_dispatch_duration_us":10324115,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10324115,"total_server_duration_us":9325861,"encode_duration_us":16591,"operation_id":"0xe47","timeout_ms":10000,"last_server_duration_us":9325861,"total_duration_us":10001318},{"operation_name":"get","last_dispatch_duration_us":6436431,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":6436431,"total_server_duration_us":791020,"operation_id":"0xf2d","timeout_ms":30000,"last_server_duration_us":791020,"total_duration_us":7529007}],"total_count":18}} WARN [2022-01-10 05:37:21,047] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10324115,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10324115,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9325861,"encode_duration_us":16591,"operation_id":"0xe47","timeout_ms":10000,"last_server_duration_us":9325861,"total_duration_us":10001318},{"operation_name":"upsert","last_dispatch_duration_us":18501157,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18944928,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9169555,"encode_duration_us":16932,"operation_id":"0xd02","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10005354},{"operation_name":"upsert","last_dispatch_duration_us":11953275,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11953275,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9024634,"encode_duration_us":19902,"operation_id":"0xe7b","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10009718},{"operation_name":"upsert","last_dispatch_duration_us":23212765,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":23212765,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9113921,"encode_duration_us":22800,"operation_id":"0xcc0","timeout_ms":10000,"last_server_duration_us":9113921,"total_duration_us":10009854}],"total_count":4}} INFO [2022-01-10 05:37:21,055] jepsen worker 87 - jepsen.util 447 :info :write [29 9] :RequestTimeoutException INFO [2022-01-10 05:37:21,056] jepsen worker 87 - jepsen.util 537 :invoke :write [29 25] INFO [2022-01-10 05:37:21,067] jepsen worker 52 - jepsen.util 412 :info :write [17 41] :RequestTimeoutException INFO [2022-01-10 05:37:21,067] jepsen worker 52 - jepsen.util 502 :invoke :write [17 36] WARN [2022-01-10 05:37:21,147] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18775177,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":18775177,"total_server_duration_us":27,"operation_id":"0xd80","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":18776267},{"operation_name":"get","last_dispatch_duration_us":13397774,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":13397774,"total_server_duration_us":15,"operation_id":"0xe52","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":13398206},{"operation_name":"get","last_dispatch_duration_us":11803889,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11803889,"total_server_duration_us":11,"operation_id":"0xe9a","timeout_ms":30000,"last_server_duration_us":11,"total_duration_us":11804239},{"operation_name":"upsert","last_dispatch_duration_us":1391862,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10382566,"total_server_duration_us":914294,"encode_duration_us":16785,"operation_id":"0xeca","timeout_ms":10000,"last_server_duration_us":112147,"total_duration_us":10009688},{"operation_name":"upsert","last_dispatch_duration_us":680912,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10233264,"total_server_duration_us":609631,"encode_duration_us":20256,"operation_id":"0xed9","timeout_ms":10000,"last_server_duration_us":103434,"total_duration_us":10008666},{"operation_name":"upsert","last_dispatch_duration_us":7762977,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10764575,"total_server_duration_us":673389,"encode_duration_us":16190,"operation_id":"0xf08","timeout_ms":10000,"last_server_duration_us":234557,"total_duration_us":10007045},{"operation_name":"upsert","last_dispatch_duration_us":599872,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9802773,"total_server_duration_us":600214,"encode_duration_us":18378,"operation_id":"0xf4d","timeout_ms":10000,"last_server_duration_us":37138,"total_duration_us":10004407},{"operation_name":"upsert","last_dispatch_duration_us":3816975,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12203667,"total_server_duration_us":785098,"encode_duration_us":16811,"operation_id":"0xf2e","timeout_ms":10000,"last_server_duration_us":445482,"total_duration_us":10002460},{"operation_name":"upsert","last_dispatch_duration_us":5871036,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":6790227,"total_server_duration_us":280314,"encode_duration_us":16592,"operation_id":"0xf82","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10001792},{"operation_name":"upsert","last_dispatch_duration_us":3872818,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9821618,"total_server_duration_us":89045,"encode_duration_us":17510,"operation_id":"0xfb0","timeout_ms":10000,"last_server_duration_us":67882,"total_duration_us":10001193}],"total_count":16}} WARN [2022-01-10 05:37:21,147] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14976,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":14976,"total_server_duration_us":293,"operation_id":"0xe99","timeout_ms":30000,"last_server_duration_us":293,"total_duration_us":14871726},{"operation_name":"upsert","last_dispatch_duration_us":980778,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10750381,"total_server_duration_us":592081,"encode_duration_us":16665,"operation_id":"0xe53","timeout_ms":10000,"last_server_duration_us":99035,"total_duration_us":10010305},{"operation_name":"upsert","last_dispatch_duration_us":8601869,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":16041590,"total_server_duration_us":106054,"encode_duration_us":16532,"operation_id":"0xe51","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10008173},{"operation_name":"upsert","last_dispatch_duration_us":530726,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10252282,"total_server_duration_us":602655,"encode_duration_us":21397,"operation_id":"0xe48","timeout_ms":10000,"last_server_duration_us":183315,"total_duration_us":10006855},{"operation_name":"upsert","last_dispatch_duration_us":180466,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9399466,"total_server_duration_us":227103,"encode_duration_us":16847,"operation_id":"0xf59","timeout_ms":10000,"last_server_duration_us":31872,"total_duration_us":10006146},{"operation_name":"upsert","last_dispatch_duration_us":214937,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":214937,"total_server_duration_us":25712,"encode_duration_us":19085,"operation_id":"0xeff","timeout_ms":10000,"last_server_duration_us":25712,"total_duration_us":10006080},{"operation_name":"upsert","last_dispatch_duration_us":1418854,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":6613382,"total_server_duration_us":1330524,"encode_duration_us":18639,"operation_id":"0xf49","timeout_ms":10000,"last_server_duration_us":296012,"total_duration_us":10005708},{"operation_name":"upsert","last_dispatch_duration_us":244439,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":347578,"total_server_duration_us":42827,"encode_duration_us":19108,"operation_id":"0xf5e","timeout_ms":10000,"last_server_duration_us":22306,"total_duration_us":10005292},{"operation_name":"upsert","last_dispatch_duration_us":1139809,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":5503287,"total_server_duration_us":408742,"encode_duration_us":16500,"operation_id":"0xf90","timeout_ms":10000,"last_server_duration_us":50418,"total_duration_us":10002625},{"operation_name":"get","last_dispatch_duration_us":8835423,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8835423,"total_server_duration_us":32,"operation_id":"0xe69","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":8836849}],"total_count":15}} WARN [2022-01-10 05:37:21,147] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":20552950,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20649506,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9045113,"encode_duration_us":16262,"operation_id":"0xb6d","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10001234},{"operation_name":"upsert","last_dispatch_duration_us":11096935,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11096935,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9370006,"encode_duration_us":24185,"operation_id":"0xe89","timeout_ms":10000,"last_server_duration_us":9370006,"total_duration_us":10003088},{"operation_name":"upsert","last_dispatch_duration_us":18057631,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":20255660,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9427167,"encode_duration_us":19185,"operation_id":"0xc14","timeout_ms":10000,"last_server_duration_us":9150168,"total_duration_us":10003543},{"operation_name":"upsert","last_dispatch_duration_us":18956323,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19262609,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9199888,"encode_duration_us":16933,"operation_id":"0xcbf","timeout_ms":10000,"last_server_duration_us":9136301,"total_duration_us":10010502},{"operation_name":"upsert","last_dispatch_duration_us":10678094,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12405482,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":10475429,"encode_duration_us":16635,"operation_id":"0xec3","timeout_ms":10000,"last_server_duration_us":10179718,"total_duration_us":10007219}],"total_count":5}} WARN [2022-01-10 05:37:21,148] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18011358,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18011358,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9026756,"encode_duration_us":17224,"operation_id":"0xe04","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10004599},{"operation_name":"upsert","last_dispatch_duration_us":10593694,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14861775,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9305811,"encode_duration_us":19336,"operation_id":"0xeae","timeout_ms":10000,"last_server_duration_us":9044797,"total_duration_us":10009327}],"total_count":2}} WARN [2022-01-10 05:37:21,248] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9554327,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12241647,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":10626819,"encode_duration_us":26397,"operation_id":"0xdc2","timeout_ms":10000,"last_server_duration_us":9469387,"total_duration_us":10001461},{"operation_name":"upsert","last_dispatch_duration_us":18075695,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18075695,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9026756,"encode_duration_us":17527,"operation_id":"0xdfb","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10001519},{"operation_name":"upsert","last_dispatch_duration_us":18058759,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18058759,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9027816,"encode_duration_us":16815,"operation_id":"0xdfa","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10005307},{"operation_name":"upsert","last_dispatch_duration_us":9618155,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11612306,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":10311800,"encode_duration_us":16443,"operation_id":"0xdd9","timeout_ms":10000,"last_server_duration_us":9450989,"total_duration_us":10031638}],"total_count":4}} WARN [2022-01-10 05:37:21,248] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10847252,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10847252,"total_server_duration_us":32,"operation_id":"0xd9e","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":16013919},{"operation_name":"get","last_dispatch_duration_us":9160666,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9160666,"total_server_duration_us":7962,"operation_id":"0xdc0","timeout_ms":30000,"last_server_duration_us":7962,"total_duration_us":12795164},{"operation_name":"upsert","last_dispatch_duration_us":9400801,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9400801,"total_server_duration_us":269986,"encode_duration_us":17993,"operation_id":"0xf8c","timeout_ms":10000,"last_server_duration_us":269986,"total_duration_us":10009462},{"operation_name":"upsert","last_dispatch_duration_us":1950967,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11566186,"total_server_duration_us":184866,"encode_duration_us":21591,"operation_id":"0xe85","timeout_ms":10000,"last_server_duration_us":60506,"total_duration_us":10007579},{"operation_name":"upsert","last_dispatch_duration_us":9530789,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":13992308,"total_server_duration_us":747161,"encode_duration_us":16886,"operation_id":"0xe92","timeout_ms":10000,"last_server_duration_us":127502,"total_duration_us":10007463},{"operation_name":"upsert","last_dispatch_duration_us":10678094,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":12405482,"total_server_duration_us":10475429,"encode_duration_us":16635,"operation_id":"0xec3","timeout_ms":10000,"last_server_duration_us":10179718,"total_duration_us":10007219},{"operation_name":"upsert","last_dispatch_duration_us":81674,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":6571121,"total_server_duration_us":256076,"encode_duration_us":17342,"operation_id":"0xf2c","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10006611},{"operation_name":"upsert","last_dispatch_duration_us":972524,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9196727,"total_server_duration_us":131738,"encode_duration_us":21832,"operation_id":"0xe9d","timeout_ms":10000,"last_server_duration_us":68813,"total_duration_us":10005228},{"operation_name":"upsert","last_dispatch_duration_us":11096935,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11096935,"total_server_duration_us":9370006,"encode_duration_us":24185,"operation_id":"0xe89","timeout_ms":10000,"last_server_duration_us":9370006,"total_duration_us":10003088},{"operation_name":"upsert","last_dispatch_duration_us":1857313,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11097099,"total_server_duration_us":262952,"encode_duration_us":16253,"operation_id":"0xf7f","timeout_ms":10000,"last_server_duration_us":137006,"total_duration_us":10001450}],"total_count":20}} WARN [2022-01-10 05:37:21,348] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":21838227,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":23838294,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9476137,"encode_duration_us":16325,"operation_id":"0xbf8","timeout_ms":10000,"last_server_duration_us":9338773,"total_duration_us":10006088},{"operation_name":"upsert","last_dispatch_duration_us":19070049,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":20704686,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9951313,"encode_duration_us":17316,"operation_id":"0xcbb","timeout_ms":10000,"last_server_duration_us":9665114,"total_duration_us":10007793},{"operation_name":"upsert","last_dispatch_duration_us":29340489,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":29340489,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":6424068,"encode_duration_us":16660,"operation_id":"0xb52","timeout_ms":10000,"last_server_duration_us":6424068,"total_duration_us":10007087}],"total_count":3}} WARN [2022-01-10 05:37:21,348] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9492233,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9492233,"total_server_duration_us":13488,"operation_id":"0xe5d","timeout_ms":30000,"last_server_duration_us":13488,"total_duration_us":16186655},{"operation_name":"get","last_dispatch_duration_us":9312178,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9312178,"total_server_duration_us":121327,"operation_id":"0xea6","timeout_ms":30000,"last_server_duration_us":121327,"total_duration_us":14487737},{"operation_name":"upsert","last_dispatch_duration_us":7453182,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9878354,"total_server_duration_us":183230,"encode_duration_us":17106,"operation_id":"0xf62","timeout_ms":10000,"last_server_duration_us":62412,"total_duration_us":10011150},{"operation_name":"upsert","last_dispatch_duration_us":9458513,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9458513,"total_server_duration_us":124741,"encode_duration_us":21825,"operation_id":"0xf81","timeout_ms":10000,"last_server_duration_us":124741,"total_duration_us":10009865},{"operation_name":"upsert","last_dispatch_duration_us":236342,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7231716,"total_server_duration_us":220797,"encode_duration_us":16881,"operation_id":"0xf88","timeout_ms":10000,"last_server_duration_us":39319,"total_duration_us":10009768},{"operation_name":"upsert","last_dispatch_duration_us":10593694,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":14861775,"total_server_duration_us":9305811,"encode_duration_us":19336,"operation_id":"0xeae","timeout_ms":10000,"last_server_duration_us":9044797,"total_duration_us":10009327},{"operation_name":"upsert","last_dispatch_duration_us":4268996,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":4268996,"total_server_duration_us":138081,"encode_duration_us":18497,"operation_id":"0xf8f","timeout_ms":10000,"last_server_duration_us":138081,"total_duration_us":10007872},{"operation_name":"upsert","last_dispatch_duration_us":2493334,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":11490123,"total_server_duration_us":272631,"encode_duration_us":17535,"operation_id":"0xf55","timeout_ms":10000,"last_server_duration_us":149943,"total_duration_us":10006616},{"operation_name":"upsert","last_dispatch_duration_us":133651,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":133651,"total_server_duration_us":19340,"encode_duration_us":16570,"operation_id":"0xe50","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10005828},{"operation_name":"upsert","last_dispatch_duration_us":417509,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7286012,"total_server_duration_us":434484,"encode_duration_us":16733,"operation_id":"0xfae","timeout_ms":10000,"last_server_duration_us":208906,"total_duration_us":10004560}],"total_count":19}} INFO [2022-01-10 05:37:21,412] jepsen worker 64 - jepsen.util 424 :ok :read [21 7] INFO [2022-01-10 05:37:21,413] jepsen worker 64 - jepsen.util 424 :invoke :write [21 22] INFO [2022-01-10 05:37:21,512] jepsen worker 73 - jepsen.util 343 :info :write [24 6] :RequestTimeoutException INFO [2022-01-10 05:37:21,513] jepsen worker 73 - jepsen.util 433 :invoke :read [24 nil] INFO [2022-01-10 05:37:21,849] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:21,849] jepsen worker nemesis - couchbase.util Rebalance check count: 9 INFO [2022-01-10 05:37:21,849] jepsen worker nemesis - couchbase.util Rebalance stuck count: 8 INFO [2022-01-10 05:37:21,862] jepsen worker 10 - jepsen.util 370 :info :write [3 37] :RequestTimeoutException INFO [2022-01-10 05:37:21,863] jepsen worker 10 - jepsen.util 460 :invoke :write [3 21] INFO [2022-01-10 05:37:21,989] jepsen worker 73 - jepsen.util 433 :ok :read [24 33] INFO [2022-01-10 05:37:21,990] jepsen worker 73 - jepsen.util 433 :invoke :write [24 45] INFO [2022-01-10 05:37:21,992] jepsen worker 65 - jepsen.util 425 :info :write [21 23] :RequestTimeoutException INFO [2022-01-10 05:37:21,993] jepsen worker 65 - jepsen.util 515 :invoke :write [21 31] INFO [2022-01-10 05:37:21,997] jepsen worker 19 - jepsen.util 379 :info :write [6 16] :RequestTimeoutException INFO [2022-01-10 05:37:21,997] jepsen worker 19 - jepsen.util 469 :invoke :write [6 31] INFO [2022-01-10 05:37:22,185] jepsen worker 70 - jepsen.util 520 :info :write [23 27] :RequestTimeoutException INFO [2022-01-10 05:37:22,185] jepsen worker 70 - jepsen.util 610 :invoke :write [23 32] INFO [2022-01-10 05:37:22,202] jepsen worker 3 - jepsen.util 453 :info :write [1 0] :RequestTimeoutException INFO [2022-01-10 05:37:22,203] jepsen worker 3 - jepsen.util 543 :invoke :write [1 31] INFO [2022-01-10 05:37:22,386] jepsen worker 16 - jepsen.util 376 :info :write [5 33] :RequestTimeoutException INFO [2022-01-10 05:37:22,386] jepsen worker 16 - jepsen.util 466 :invoke :write [5 11] INFO [2022-01-10 05:37:22,400] jepsen worker 59 - jepsen.util 509 :info :write [19 41] :RequestTimeoutException INFO [2022-01-10 05:37:22,400] jepsen worker 59 - jepsen.util 599 :invoke :write [19 19] INFO [2022-01-10 05:37:22,402] jepsen worker 11 - jepsen.util 281 :info :write [3 17] :RequestTimeoutException INFO [2022-01-10 05:37:22,403] jepsen worker 11 - jepsen.util 371 :invoke :write [3 31] INFO [2022-01-10 05:37:22,666] jepsen worker 63 - jepsen.util 423 :info :write [21 33] :RequestTimeoutException INFO [2022-01-10 05:37:22,667] jepsen worker 63 - jepsen.util 513 :invoke :write [21 36] INFO [2022-01-10 05:37:22,671] jepsen worker 18 - jepsen.util 378 :info :write [6 31] :RequestTimeoutException INFO [2022-01-10 05:37:22,675] jepsen worker 18 - jepsen.util 468 :invoke :write [6 37] INFO [2022-01-10 05:37:22,849] jepsen worker 69 - jepsen.util 519 :ok :read [23 :nil] INFO [2022-01-10 05:37:22,851] jepsen worker 69 - jepsen.util 519 :invoke :write [23 15] INFO [2022-01-10 05:37:22,854] jepsen worker 36 - jepsen.util 216 :ok :read [12 :nil] INFO [2022-01-10 05:37:22,854] jepsen worker 36 - jepsen.util 216 :invoke :read [12 nil] INFO [2022-01-10 05:37:22,856] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:22,856] jepsen worker nemesis - couchbase.util Rebalance check count: 10 INFO [2022-01-10 05:37:22,856] jepsen worker nemesis - couchbase.util Rebalance stuck count: 9 INFO [2022-01-10 05:37:23,102] jepsen worker 83 - jepsen.util 443 :info :write [27 28] :RequestTimeoutException INFO [2022-01-10 05:37:23,103] jepsen worker 83 - jepsen.util 533 :invoke :write [27 45] INFO [2022-01-10 05:37:23,126] jepsen worker 22 - jepsen.util 382 :info :write [7 38] :RequestTimeoutException INFO [2022-01-10 05:37:23,127] jepsen worker 22 - jepsen.util 472 :invoke :read [7 nil] INFO [2022-01-10 05:37:23,345] jepsen worker 43 - jepsen.util 403 :info :write [14 4] :RequestTimeoutException INFO [2022-01-10 05:37:23,345] jepsen worker 43 - jepsen.util 493 :invoke :write [14 11] INFO [2022-01-10 05:37:23,492] jepsen worker 42 - jepsen.util 312 :info :write [14 15] :RequestTimeoutException INFO [2022-01-10 05:37:23,492] jepsen worker 8 - jepsen.util 368 :info :write [2 12] :RequestTimeoutException INFO [2022-01-10 05:37:23,493] jepsen worker 42 - jepsen.util 402 :invoke :read [14 nil] INFO [2022-01-10 05:37:23,493] jepsen worker 8 - jepsen.util 458 :invoke :write [2 35] INFO [2022-01-10 05:37:23,535] jepsen worker 21 - jepsen.util 471 :info :write [7 24] :RequestTimeoutException INFO [2022-01-10 05:37:23,536] jepsen worker 21 - jepsen.util 561 :invoke :write [7 25] INFO [2022-01-10 05:37:23,546] jepsen worker 88 - jepsen.util 448 :info :write [29 31] :RequestTimeoutException INFO [2022-01-10 05:37:23,547] jepsen worker 88 - jepsen.util 538 :invoke :write [29 28] INFO [2022-01-10 05:37:23,646] jepsen worker 46 - jepsen.util 316 :info :write [15 49] :RequestTimeoutException INFO [2022-01-10 05:37:23,647] jepsen worker 46 - jepsen.util 406 :invoke :write [15 7] INFO [2022-01-10 05:37:23,736] jepsen worker 22 - jepsen.util 472 :ok :read [7 :nil] INFO [2022-01-10 05:37:23,736] jepsen worker 22 - jepsen.util 472 :invoke :read [7 nil] INFO [2022-01-10 05:37:23,752] jepsen worker 71 - jepsen.util 431 :info :write [23 24] :RequestTimeoutException INFO [2022-01-10 05:37:23,753] jepsen worker 71 - jepsen.util 521 :invoke :read [23 nil] INFO [2022-01-10 05:37:23,807] jepsen worker 78 - jepsen.util 438 :info :write [26 21] :RequestTimeoutException INFO [2022-01-10 05:37:23,808] jepsen worker 78 - jepsen.util 528 :invoke :read [26 nil] INFO [2022-01-10 05:37:23,864] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:23,864] jepsen worker nemesis - couchbase.util Rebalance check count: 11 INFO [2022-01-10 05:37:23,864] jepsen worker nemesis - couchbase.util Rebalance stuck count: 10 INFO [2022-01-10 05:37:23,906] jepsen worker 37 - jepsen.util 577 :ok :read [12 :nil] INFO [2022-01-10 05:37:23,907] jepsen worker 37 - jepsen.util 577 :invoke :write [12 8] INFO [2022-01-10 05:37:23,919] jepsen worker 53 - jepsen.util 413 :ok :read [17 37] INFO [2022-01-10 05:37:23,920] jepsen worker 53 - jepsen.util 413 :invoke :read [17 nil] INFO [2022-01-10 05:37:23,945] jepsen worker 22 - jepsen.util 472 :ok :read [7 :nil] INFO [2022-01-10 05:37:23,945] jepsen worker 23 - jepsen.util 383 :ok :read [7 :nil] INFO [2022-01-10 05:37:23,946] jepsen worker 22 - jepsen.util 472 :invoke :read [7 nil] INFO [2022-01-10 05:37:23,947] jepsen worker 23 - jepsen.util 383 :invoke :write [7 20] INFO [2022-01-10 05:37:24,056] jepsen worker 81 - jepsen.util 441 :info :write [27 27] :RequestTimeoutException INFO [2022-01-10 05:37:24,057] jepsen worker 81 - jepsen.util 531 :invoke :read [27 nil] INFO [2022-01-10 05:37:24,086] jepsen worker 27 - jepsen.util 297 :info :write [9 22] :RequestTimeoutException INFO [2022-01-10 05:37:24,087] jepsen worker 27 - jepsen.util 387 :invoke :write [9 23] INFO [2022-01-10 05:37:24,131] jepsen worker 61 - jepsen.util 511 :info :write [20 32] :RequestTimeoutException INFO [2022-01-10 05:37:24,132] jepsen worker 61 - jepsen.util 601 :invoke :write [20 35] INFO [2022-01-10 05:37:24,181] jepsen worker 14 - jepsen.util 464 :ok :read [4 :nil] INFO [2022-01-10 05:37:24,181] jepsen worker 14 - jepsen.util 464 :invoke :read [4 nil] INFO [2022-01-10 05:37:24,243] jepsen worker 13 - jepsen.util 463 :ok :read [4 :nil] INFO [2022-01-10 05:37:24,243] jepsen worker 13 - jepsen.util 463 :invoke :read [4 nil] INFO [2022-01-10 05:37:24,304] jepsen worker 22 - jepsen.util 472 :ok :read [7 :nil] INFO [2022-01-10 05:37:24,304] jepsen worker 22 - jepsen.util 472 :invoke :read [7 nil] INFO [2022-01-10 05:37:24,478] jepsen worker 22 - jepsen.util 472 :ok :read [7 :nil] INFO [2022-01-10 05:37:24,479] jepsen worker 22 - jepsen.util 472 :invoke :write [7 15] INFO [2022-01-10 05:37:24,660] jepsen worker 1 - jepsen.util 451 :ok :read [0 :nil] INFO [2022-01-10 05:37:24,661] jepsen worker 1 - jepsen.util 451 :invoke :write [0 43] INFO [2022-01-10 05:37:24,680] jepsen worker 49 - jepsen.util 409 :ok :read [16 :nil] INFO [2022-01-10 05:37:24,680] jepsen worker 49 - jepsen.util 409 :invoke :read [16 nil] INFO [2022-01-10 05:37:24,758] jepsen worker 40 - jepsen.util 400 :ok :read [13 23] INFO [2022-01-10 05:37:24,759] jepsen worker 40 - jepsen.util 400 :invoke :read [13 nil] INFO [2022-01-10 05:37:24,770] jepsen worker 35 - jepsen.util 485 :info :write [11 2] :RequestTimeoutException INFO [2022-01-10 05:37:24,771] jepsen worker 35 - jepsen.util 575 :invoke :write [11 34] INFO [2022-01-10 05:37:24,870] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:24,871] jepsen worker nemesis - couchbase.util Rebalance check count: 12 INFO [2022-01-10 05:37:24,873] jepsen worker 48 - jepsen.util 228 :info :write [16 18] :SyncWriteAmbiguous INFO [2022-01-10 05:37:24,882] jepsen worker nemesis - couchbase.util Rebalance stuck count: 11 INFO [2022-01-10 05:37:24,882] jepsen worker 48 - jepsen.util 318 :invoke :read [16 nil] INFO [2022-01-10 05:37:24,897] jepsen worker 42 - jepsen.util 402 :ok :read [14 14] INFO [2022-01-10 05:37:24,898] jepsen worker 42 - jepsen.util 402 :invoke :write [14 12] INFO [2022-01-10 05:37:24,914] jepsen worker 44 - jepsen.util 494 :ok :read [14 14] INFO [2022-01-10 05:37:24,915] jepsen worker 44 - jepsen.util 494 :invoke :read [14 nil] INFO [2022-01-10 05:37:24,930] jepsen worker 48 - jepsen.util 318 :ok :read [16 :nil] INFO [2022-01-10 05:37:24,931] jepsen worker 48 - jepsen.util 318 :invoke :read [16 nil] INFO [2022-01-10 05:37:24,932] jepsen worker 48 - jepsen.util 318 :ok :read [16 :nil] INFO [2022-01-10 05:37:24,936] jepsen worker 48 - jepsen.util 318 :invoke :read [16 nil] INFO [2022-01-10 05:37:24,937] jepsen worker 48 - jepsen.util 318 :ok :read [16 :nil] INFO [2022-01-10 05:37:24,938] jepsen worker 48 - jepsen.util 318 :invoke :write [16 1] INFO [2022-01-10 05:37:25,085] jepsen worker 32 - jepsen.util 482 :ok :read [10 37] INFO [2022-01-10 05:37:25,086] jepsen worker 32 - jepsen.util 482 :invoke :read [10 nil] INFO [2022-01-10 05:37:25,353] jepsen worker 44 - jepsen.util 494 :ok :read [14 14] INFO [2022-01-10 05:37:25,354] jepsen worker 44 - jepsen.util 494 :invoke :write [14 12] INFO [2022-01-10 05:37:25,562] jepsen worker 54 - jepsen.util 324 :ok :read [18 14] INFO [2022-01-10 05:37:25,563] jepsen worker 54 - jepsen.util 324 :invoke :read [18 nil] INFO [2022-01-10 05:37:25,743] jepsen worker 57 - jepsen.util 417 :info :write [19 37] :SyncWriteAmbiguous INFO [2022-01-10 05:37:25,751] jepsen worker 57 - jepsen.util 507 :invoke :write [19 37] INFO [2022-01-10 05:37:25,782] jepsen worker 9 - jepsen.util 549 :ok :read [3 2] INFO [2022-01-10 05:37:25,783] jepsen worker 9 - jepsen.util 549 :invoke :write [3 26] INFO [2022-01-10 05:37:25,837] jepsen worker 41 - jepsen.util 311 :info :write [13 18] :RequestTimeoutException INFO [2022-01-10 05:37:25,837] jepsen worker 41 - jepsen.util 401 :invoke :read [13 nil] INFO [2022-01-10 05:37:25,846] jepsen worker 15 - jepsen.util 285 :info :write [5 49] :RequestTimeoutException INFO [2022-01-10 05:37:25,847] jepsen worker 15 - jepsen.util 375 :invoke :read [5 nil] INFO [2022-01-10 05:37:25,876] jepsen worker 68 - jepsen.util 338 :info :write [22 1] :RequestTimeoutException INFO [2022-01-10 05:37:25,877] jepsen worker 68 - jepsen.util 428 :invoke :read [22 nil] INFO [2022-01-10 05:37:25,898] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:25,898] jepsen worker nemesis - couchbase.util Rebalance check count: 13 INFO [2022-01-10 05:37:25,898] jepsen worker nemesis - couchbase.util Rebalance stuck count: 12 INFO [2022-01-10 05:37:25,951] jepsen worker 56 - jepsen.util 416 :info :write [18 8] :RequestTimeoutException INFO [2022-01-10 05:37:25,952] jepsen worker 56 - jepsen.util 506 :invoke :write [18 16] INFO [2022-01-10 05:37:26,027] jepsen worker 33 - jepsen.util 303 :info :write [11 11] :RequestTimeoutException INFO [2022-01-10 05:37:26,028] jepsen worker 33 - jepsen.util 393 :invoke :read [11 nil] INFO [2022-01-10 05:37:26,045] jepsen worker 34 - jepsen.util 484 :info :write [11 35] :RequestTimeoutException INFO [2022-01-10 05:37:26,046] jepsen worker 34 - jepsen.util 574 :invoke :write [11 13] INFO [2022-01-10 05:37:26,056] jepsen worker 60 - jepsen.util 510 :info :write [20 35] :RequestTimeoutException INFO [2022-01-10 05:37:26,057] jepsen worker 60 - jepsen.util 600 :invoke :read [20 nil] INFO [2022-01-10 05:37:26,070] jepsen worker 20 - jepsen.util 200 :info :write [6 12] :RequestTimeoutException INFO [2022-01-10 05:37:26,071] jepsen worker 20 - jepsen.util 290 :invoke :read [6 nil] INFO [2022-01-10 05:37:26,306] jepsen worker 2 - jepsen.util 272 :info :write [0 45] :RequestTimeoutException INFO [2022-01-10 05:37:26,307] jepsen worker 2 - jepsen.util 362 :invoke :write [0 35] INFO [2022-01-10 05:37:26,636] jepsen worker 36 - jepsen.util 216 :ok :read [12 :nil] INFO [2022-01-10 05:37:26,637] jepsen worker 36 - jepsen.util 216 :invoke :read [12 nil] INFO [2022-01-10 05:37:26,650] jepsen worker 53 - jepsen.util 413 :ok :read [17 41] INFO [2022-01-10 05:37:26,651] jepsen worker 53 - jepsen.util 413 :invoke :read [17 nil] INFO [2022-01-10 05:37:26,755] jepsen worker 74 - jepsen.util 524 :info :write [24 27] :RequestTimeoutException INFO [2022-01-10 05:37:26,756] jepsen worker 74 - jepsen.util 614 :invoke :read [24 nil] INFO [2022-01-10 05:37:26,791] jepsen worker 74 - jepsen.util 614 :ok :read [24 33] INFO [2022-01-10 05:37:26,792] jepsen worker 74 - jepsen.util 614 :invoke :write [24 18] INFO [2022-01-10 05:37:26,904] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:26,904] jepsen worker nemesis - couchbase.util Rebalance check count: 14 INFO [2022-01-10 05:37:26,904] jepsen worker nemesis - couchbase.util Rebalance stuck count: 13 INFO [2022-01-10 05:37:27,270] jepsen worker 17 - jepsen.util 287 :info :write [5 21] :RequestTimeoutException INFO [2022-01-10 05:37:27,271] jepsen worker 17 - jepsen.util 377 :invoke :read [5 nil] INFO [2022-01-10 05:37:27,353] jepsen worker 32 - jepsen.util 482 :ok :read [10 37] INFO [2022-01-10 05:37:27,353] jepsen worker 32 - jepsen.util 482 :invoke :write [10 36] INFO [2022-01-10 05:37:27,366] jepsen worker 7 - jepsen.util 277 :info :write [2 48] :RequestTimeoutException INFO [2022-01-10 05:37:27,366] jepsen worker 54 - jepsen.util 324 :ok :read [18 14] INFO [2022-01-10 05:37:27,366] jepsen worker 54 - jepsen.util 324 :invoke :write [18 9] INFO [2022-01-10 05:37:27,367] jepsen worker 7 - jepsen.util 367 :invoke :read [2 nil] INFO [2022-01-10 05:37:27,415] jepsen worker 58 - jepsen.util 418 :info :write [19 22] :RequestTimeoutException INFO [2022-01-10 05:37:27,416] jepsen worker 58 - jepsen.util 508 :invoke :write [19 42] INFO [2022-01-10 05:37:27,642] jepsen worker 31 - jepsen.util 571 :info :write [10 17] :RequestTimeoutException INFO [2022-01-10 05:37:27,642] jepsen worker 31 - jepsen.util 661 :invoke :read [10 nil] INFO [2022-01-10 05:37:27,751] jepsen worker 47 - jepsen.util 497 :info :write [15 2] :RequestTimeoutException INFO [2022-01-10 05:37:27,752] jepsen worker 47 - jepsen.util 587 :invoke :read [15 nil] INFO [2022-01-10 05:37:27,767] jepsen worker 7 - jepsen.util 367 :ok :read [2 21] INFO [2022-01-10 05:37:27,768] jepsen worker 7 - jepsen.util 367 :invoke :write [2 1] INFO [2022-01-10 05:37:27,805] jepsen worker 13 - jepsen.util 463 :ok :read [4 :nil] INFO [2022-01-10 05:37:27,806] jepsen worker 13 - jepsen.util 463 :invoke :read [4 nil] INFO [2022-01-10 05:37:27,910] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:27,910] jepsen worker nemesis - couchbase.util Rebalance check count: 15 INFO [2022-01-10 05:37:27,910] jepsen worker nemesis - couchbase.util Rebalance stuck count: 14 INFO [2022-01-10 05:37:27,960] jepsen worker 66 - jepsen.util 246 :info :write [22 14] :RequestTimeoutException INFO [2022-01-10 05:37:27,960] jepsen worker 66 - jepsen.util 336 :invoke :write [22 10] INFO [2022-01-10 05:37:27,995] jepsen worker 12 - jepsen.util 372 :info :write [4 0] :RequestTimeoutException INFO [2022-01-10 05:37:27,996] jepsen worker 12 - jepsen.util 462 :invoke :write [4 48] INFO [2022-01-10 05:37:27,999] jepsen worker 14 - jepsen.util 464 :ok :read [4 :nil] INFO [2022-01-10 05:37:27,999] jepsen worker 14 - jepsen.util 464 :invoke :write [4 3] INFO [2022-01-10 05:37:28,234] jepsen worker 86 - jepsen.util 446 :ok :read [28 :nil] INFO [2022-01-10 05:37:28,234] jepsen worker 86 - jepsen.util 446 :invoke :read [28 nil] INFO [2022-01-10 05:37:28,249] jepsen worker 68 - jepsen.util 428 :ok :read [22 41] INFO [2022-01-10 05:37:28,249] jepsen worker 68 - jepsen.util 428 :invoke :write [22 8] INFO [2022-01-10 05:37:28,265] jepsen worker 77 - jepsen.util 527 :info :write [25 15] :RequestTimeoutException INFO [2022-01-10 05:37:28,266] jepsen worker 76 - jepsen.util 436 :info :write [25 26] :RequestTimeoutException INFO [2022-01-10 05:37:28,266] jepsen worker 77 - jepsen.util 617 :invoke :read [25 nil] INFO [2022-01-10 05:37:28,267] jepsen worker 76 - jepsen.util 526 :invoke :read [25 nil] INFO [2022-01-10 05:37:28,302] jepsen worker 77 - jepsen.util 617 :ok :read [25 :nil] INFO [2022-01-10 05:37:28,303] jepsen worker 77 - jepsen.util 617 :invoke :write [25 10] INFO [2022-01-10 05:37:28,422] jepsen worker 51 - jepsen.util 321 :info :write [17 22] :RequestTimeoutException INFO [2022-01-10 05:37:28,423] jepsen worker 51 - jepsen.util 411 :invoke :read [17 nil] INFO [2022-01-10 05:37:28,723] jepsen worker 47 - jepsen.util 587 :ok :read [15 2] INFO [2022-01-10 05:37:28,724] jepsen worker 47 - jepsen.util 587 :invoke :write [15 6] INFO [2022-01-10 05:37:28,726] jepsen worker 75 - jepsen.util 435 :info :write [25 20] :RequestTimeoutException INFO [2022-01-10 05:37:28,727] jepsen worker 75 - jepsen.util 525 :invoke :write [25 32] INFO [2022-01-10 05:37:28,841] jepsen worker 89 - jepsen.util 629 :info :write [29 19] :RequestTimeoutException INFO [2022-01-10 05:37:28,842] jepsen worker 89 - jepsen.util 719 :invoke :write [29 12] INFO [2022-01-10 05:37:28,891] jepsen worker 24 - jepsen.util 384 :info :write [8 21] :RequestTimeoutException INFO [2022-01-10 05:37:28,892] jepsen worker 24 - jepsen.util 474 :invoke :read [8 nil] INFO [2022-01-10 05:37:28,920] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:28,921] jepsen worker nemesis - couchbase.util Rebalance check count: 16 INFO [2022-01-10 05:37:28,921] jepsen worker nemesis - couchbase.util Rebalance stuck count: 15 INFO [2022-01-10 05:37:28,932] jepsen worker 67 - jepsen.util 337 :info :write [22 43] :RequestTimeoutException INFO [2022-01-10 05:37:28,935] jepsen worker 67 - jepsen.util 427 :invoke :write [22 7] INFO [2022-01-10 05:37:28,940] jepsen worker 33 - jepsen.util 393 :ok :read [11 21] INFO [2022-01-10 05:37:28,940] jepsen worker 33 - jepsen.util 393 :invoke :read [11 nil] INFO [2022-01-10 05:37:28,950] jepsen worker 79 - jepsen.util 439 :info :write [26 5] :RequestTimeoutException INFO [2022-01-10 05:37:28,951] jepsen worker 79 - jepsen.util 529 :invoke :write [26 47] INFO [2022-01-10 05:37:28,985] jepsen worker 24 - jepsen.util 474 :ok :read [8 :nil] INFO [2022-01-10 05:37:28,986] jepsen worker 24 - jepsen.util 474 :invoke :write [8 18] INFO [2022-01-10 05:37:28,990] jepsen worker 85 - jepsen.util 445 :info :write [28 16] :RequestTimeoutException INFO [2022-01-10 05:37:28,991] jepsen worker 85 - jepsen.util 535 :invoke :write [28 41] INFO [2022-01-10 05:37:28,995] jepsen worker 45 - jepsen.util 495 :info :write [15 39] :RequestTimeoutException INFO [2022-01-10 05:37:28,997] jepsen worker 45 - jepsen.util 585 :invoke :write [15 6] INFO [2022-01-10 05:37:29,141] jepsen worker 33 - jepsen.util 393 :ok :read [11 21] INFO [2022-01-10 05:37:29,141] jepsen worker 33 - jepsen.util 393 :invoke :write [11 28] INFO [2022-01-10 05:37:29,190] jepsen worker 55 - jepsen.util 415 :info :write [18 22] :RequestTimeoutException INFO [2022-01-10 05:37:29,191] jepsen worker 55 - jepsen.util 505 :invoke :write [18 21] INFO [2022-01-10 05:37:29,240] jepsen worker 4 - jepsen.util 364 :info :write [1 23] :RequestTimeoutException INFO [2022-01-10 05:37:29,241] jepsen worker 4 - jepsen.util 454 :invoke :read [1 nil] INFO [2022-01-10 05:37:29,322] jepsen worker 82 - jepsen.util 262 :info :write [27 18] :RequestTimeoutException INFO [2022-01-10 05:37:29,323] jepsen worker 82 - jepsen.util 352 :invoke :write [27 0] WARN [2022-01-10 05:37:29,464] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9439655,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10234271,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9327953,"encode_duration_us":16526,"operation_id":"0x1001","timeout_ms":10000,"last_server_duration_us":9079863,"total_duration_us":10001749},{"operation_name":"upsert","last_dispatch_duration_us":4854129,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":8088309,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":4533165,"encode_duration_us":17702,"operation_id":"0xf0c","timeout_ms":10000,"last_server_duration_us":4136386,"total_duration_us":10010032}],"total_count":2}} WARN [2022-01-10 05:37:29,564] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":469816,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1968454,"total_server_duration_us":735746,"encode_duration_us":17871,"operation_id":"0x10f9","timeout_ms":10000,"last_server_duration_us":224983,"total_duration_us":10011083},{"operation_name":"upsert","last_dispatch_duration_us":384149,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":7934604,"total_server_duration_us":342525,"encode_duration_us":20173,"operation_id":"0xf8b","timeout_ms":10000,"last_server_duration_us":45167,"total_duration_us":10010886},{"operation_name":"upsert","last_dispatch_duration_us":202407,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1269929,"total_server_duration_us":398496,"encode_duration_us":19024,"operation_id":"0x10a3","timeout_ms":10000,"last_server_duration_us":116448,"total_duration_us":10009520},{"operation_name":"upsert","last_dispatch_duration_us":165769,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5072415,"total_server_duration_us":717368,"encode_duration_us":16652,"operation_id":"0xffe","timeout_ms":10000,"last_server_duration_us":64863,"total_duration_us":10008909},{"operation_name":"upsert","last_dispatch_duration_us":6146464,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8427555,"total_server_duration_us":724911,"encode_duration_us":17862,"operation_id":"0x1081","timeout_ms":10000,"last_server_duration_us":39529,"total_duration_us":10008339},{"operation_name":"upsert","last_dispatch_duration_us":6142737,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9210906,"total_server_duration_us":1261522,"encode_duration_us":17076,"operation_id":"0x10d7","timeout_ms":10000,"last_server_duration_us":37653,"total_duration_us":10006541},{"operation_name":"upsert","last_dispatch_duration_us":3504836,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10190169,"total_server_duration_us":2627152,"encode_duration_us":19052,"operation_id":"0xfb3","timeout_ms":10000,"last_server_duration_us":257006,"total_duration_us":10004372},{"operation_name":"upsert","last_dispatch_duration_us":43834,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":1233020,"total_server_duration_us":615503,"encode_duration_us":16276,"operation_id":"0x10b7","timeout_ms":10000,"last_server_duration_us":33622,"total_duration_us":10003332},{"operation_name":"upsert","last_dispatch_duration_us":5011014,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":13284456,"total_server_duration_us":4070341,"encode_duration_us":25754,"operation_id":"0xfdd","timeout_ms":10000,"last_server_duration_us":140962,"total_duration_us":10003214},{"operation_name":"upsert","last_dispatch_duration_us":1351695,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":2853706,"total_server_duration_us":1342766,"encode_duration_us":19875,"operation_id":"0x10d2","timeout_ms":10000,"last_server_duration_us":787633,"total_duration_us":10001875}],"total_count":12}} INFO [2022-01-10 05:37:29,721] jepsen worker 62 - jepsen.util 602 :info :write [20 5] :RequestTimeoutException INFO [2022-01-10 05:37:29,721] jepsen worker 62 - jepsen.util 692 :invoke :read [20 nil] INFO [2022-01-10 05:37:29,885] jepsen worker 0 - jepsen.util 360 :info :write [0 31] :RequestTimeoutException INFO [2022-01-10 05:37:29,886] jepsen worker 0 - jepsen.util 450 :invoke :write [0 8] INFO [2022-01-10 05:37:29,896] jepsen worker 26 - jepsen.util 476 :info :write [8 23] :RequestTimeoutException INFO [2022-01-10 05:37:29,897] jepsen worker 26 - jepsen.util 566 :invoke :write [8 38] INFO [2022-01-10 05:37:29,928] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:29,928] jepsen worker nemesis - couchbase.util Rebalance check count: 17 INFO [2022-01-10 05:37:29,928] jepsen worker nemesis - couchbase.util Rebalance stuck count: 16 INFO [2022-01-10 05:37:29,930] jepsen worker 38 - jepsen.util 488 :info :write [12 9] :RequestTimeoutException INFO [2022-01-10 05:37:29,931] jepsen worker 38 - jepsen.util 578 :invoke :read [12 nil] INFO [2022-01-10 05:37:29,940] jepsen worker 6 - jepsen.util 366 :info :write [2 31] :RequestTimeoutException INFO [2022-01-10 05:37:29,941] jepsen worker 6 - jepsen.util 456 :invoke :read [2 nil] INFO [2022-01-10 05:37:30,035] jepsen worker 80 - jepsen.util 530 :ok :read [26 10] INFO [2022-01-10 05:37:30,036] jepsen worker 80 - jepsen.util 530 :invoke :write [26 34] INFO [2022-01-10 05:37:30,101] jepsen worker 6 - jepsen.util 456 :ok :read [2 21] INFO [2022-01-10 05:37:30,101] jepsen worker 6 - jepsen.util 456 :invoke :read [2 nil] INFO [2022-01-10 05:37:30,106] jepsen worker 30 - jepsen.util 300 :info :write [10 45] :RequestTimeoutException INFO [2022-01-10 05:37:30,109] jepsen worker 30 - jepsen.util 390 :invoke :read [10 nil] INFO [2022-01-10 05:37:30,141] jepsen worker 25 - jepsen.util 385 :info :write [8 15] :RequestTimeoutException INFO [2022-01-10 05:37:30,142] jepsen worker 25 - jepsen.util 475 :invoke :read [8 nil] INFO [2022-01-10 05:37:30,195] jepsen worker 50 - jepsen.util 500 :info :write [16 41] :RequestTimeoutException INFO [2022-01-10 05:37:30,196] jepsen worker 50 - jepsen.util 590 :invoke :write [16 38] INFO [2022-01-10 05:37:30,212] jepsen worker 13 - jepsen.util 463 :ok :read [4 :nil] INFO [2022-01-10 05:37:30,212] jepsen worker 13 - jepsen.util 463 :invoke :write [4 25] INFO [2022-01-10 05:37:30,244] jepsen worker 36 - jepsen.util 216 :ok :read [12 :nil] INFO [2022-01-10 05:37:30,246] jepsen worker 36 - jepsen.util 216 :invoke :read [12 nil] INFO [2022-01-10 05:37:30,257] jepsen worker 53 - jepsen.util 413 :ok :read [17 41] INFO [2022-01-10 05:37:30,258] jepsen worker 53 - jepsen.util 413 :invoke :read [17 nil] INFO [2022-01-10 05:37:30,261] jepsen worker 6 - jepsen.util 456 :ok :read [2 21] INFO [2022-01-10 05:37:30,262] jepsen worker 6 - jepsen.util 456 :invoke :read [2 nil] INFO [2022-01-10 05:37:30,276] jepsen worker 29 - jepsen.util 389 :info :write [9 37] :RequestTimeoutException INFO [2022-01-10 05:37:30,277] jepsen worker 29 - jepsen.util 479 :invoke :read [9 nil] INFO [2022-01-10 05:37:30,301] jepsen worker 6 - jepsen.util 456 :ok :read [2 21] INFO [2022-01-10 05:37:30,303] jepsen worker 6 - jepsen.util 456 :invoke :write [2 40] INFO [2022-01-10 05:37:30,310] jepsen worker 84 - jepsen.util 534 :info :write [28 39] :RequestTimeoutException INFO [2022-01-10 05:37:30,312] jepsen worker 84 - jepsen.util 624 :invoke :read [28 nil] INFO [2022-01-10 05:37:30,332] jepsen worker 28 - jepsen.util 478 :info :write [9 9] :RequestTimeoutException INFO [2022-01-10 05:37:30,333] jepsen worker 28 - jepsen.util 568 :invoke :write [9 32] INFO [2022-01-10 05:37:30,443] jepsen worker 31 - jepsen.util 661 :ok :read [10 37] INFO [2022-01-10 05:37:30,443] jepsen worker 31 - jepsen.util 661 :invoke :read [10 nil] INFO [2022-01-10 05:37:30,610] jepsen worker 41 - jepsen.util 401 :ok :read [13 23] INFO [2022-01-10 05:37:30,611] jepsen worker 41 - jepsen.util 401 :invoke :read [13 nil] INFO [2022-01-10 05:37:30,655] jepsen worker 72 - jepsen.util 612 :info :write [24 26] :RequestTimeoutException INFO [2022-01-10 05:37:30,655] jepsen worker 72 - jepsen.util 702 :invoke :write [24 33] INFO [2022-01-10 05:37:30,750] jepsen worker 15 - jepsen.util 375 :ok :read [5 42] INFO [2022-01-10 05:37:30,750] jepsen worker 15 - jepsen.util 375 :invoke :read [5 nil] INFO [2022-01-10 05:37:30,879] jepsen worker 81 - jepsen.util 531 :ok :read [27 30] INFO [2022-01-10 05:37:30,880] jepsen worker 81 - jepsen.util 531 :invoke :write [27 41] INFO [2022-01-10 05:37:30,895] jepsen worker 39 - jepsen.util 399 :info :write [13 20] :RequestTimeoutException INFO [2022-01-10 05:37:30,896] jepsen worker 39 - jepsen.util 489 :invoke :read [13 nil] INFO [2022-01-10 05:37:30,909] jepsen worker 41 - jepsen.util 401 :ok :read [13 23] INFO [2022-01-10 05:37:30,910] jepsen worker 5 - jepsen.util 455 :info :write [1 26] :RequestTimeoutException INFO [2022-01-10 05:37:30,910] jepsen worker 41 - jepsen.util 401 :invoke :write [13 43] INFO [2022-01-10 05:37:30,910] jepsen worker 5 - jepsen.util 545 :invoke :read [1 nil] INFO [2022-01-10 05:37:30,938] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:30,938] jepsen worker nemesis - couchbase.util Rebalance check count: 18 INFO [2022-01-10 05:37:30,938] jepsen worker nemesis - couchbase.util Rebalance stuck count: 17 INFO [2022-01-10 05:37:31,008] jepsen worker 39 - jepsen.util 489 :ok :read [13 23] INFO [2022-01-10 05:37:31,009] jepsen worker 39 - jepsen.util 489 :invoke :read [13 nil] WARN [2022-01-10 05:37:31,061] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13976594,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":13976594,"total_server_duration_us":38,"operation_id":"0xf98","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":13977039},{"operation_name":"upsert","last_dispatch_duration_us":2849927,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":8965914,"total_server_duration_us":1906835,"encode_duration_us":16243,"operation_id":"0x10f2","timeout_ms":10000,"last_server_duration_us":963248,"total_duration_us":10010713},{"operation_name":"upsert","last_dispatch_duration_us":358929,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3782489,"total_server_duration_us":1067393,"encode_duration_us":22964,"operation_id":"0x10f4","timeout_ms":10000,"last_server_duration_us":57756,"total_duration_us":10010497},{"operation_name":"upsert","last_dispatch_duration_us":148055,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3007195,"total_server_duration_us":419433,"encode_duration_us":18777,"operation_id":"0x1126","timeout_ms":10000,"last_server_duration_us":48105,"total_duration_us":10009234},{"operation_name":"upsert","last_dispatch_duration_us":3859165,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":10173313,"total_server_duration_us":1238131,"encode_duration_us":17501,"operation_id":"0x107f","timeout_ms":10000,"last_server_duration_us":300989,"total_duration_us":10008265},{"operation_name":"upsert","last_dispatch_duration_us":447308,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5893011,"total_server_duration_us":1527922,"encode_duration_us":16502,"operation_id":"0x1000","timeout_ms":10000,"last_server_duration_us":363714,"total_duration_us":10007467},{"operation_name":"upsert","last_dispatch_duration_us":190029,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3365519,"total_server_duration_us":361391,"encode_duration_us":16485,"operation_id":"0x1022","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10005542},{"operation_name":"upsert","last_dispatch_duration_us":609429,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5009563,"total_server_duration_us":400608,"encode_duration_us":16439,"operation_id":"0xfe5","timeout_ms":10000,"last_server_duration_us":48794,"total_duration_us":10005104},{"operation_name":"upsert","last_dispatch_duration_us":98602,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4465564,"total_server_duration_us":788795,"encode_duration_us":20607,"operation_id":"0x10c7","timeout_ms":10000,"last_server_duration_us":25712,"total_duration_us":10004064},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":17201,"operation_id":"0xff9","timeout_ms":10000,"total_duration_us":10001452}],"total_count":20}} WARN [2022-01-10 05:37:31,062] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17993589,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17993589,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9026756,"encode_duration_us":16671,"operation_id":"0xee6","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10001871}],"total_count":1}} INFO [2022-01-10 05:37:31,082] jepsen worker 87 - jepsen.util 537 :info :write [29 25] :RequestTimeoutException INFO [2022-01-10 05:37:31,083] jepsen worker 87 - jepsen.util 627 :invoke :read [29 nil] INFO [2022-01-10 05:37:31,096] jepsen worker 52 - jepsen.util 502 :info :write [17 36] :RequestTimeoutException INFO [2022-01-10 05:37:31,097] jepsen worker 52 - jepsen.util 592 :invoke :read [17 nil] WARN [2022-01-10 05:37:31,161] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10840461,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11935524,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9139085,"encode_duration_us":19271,"operation_id":"0xfd5","timeout_ms":10000,"last_server_duration_us":9076673,"total_duration_us":10002708},{"operation_name":"upsert","last_dispatch_duration_us":9040557,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9934934,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9205297,"encode_duration_us":20284,"operation_id":"0x1020","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10005498}],"total_count":2}} WARN [2022-01-10 05:37:31,161] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14411427,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":14411427,"total_server_duration_us":8585898,"encode_duration_us":17403,"operation_id":"0xfb4","timeout_ms":10000,"last_server_duration_us":8585898,"total_duration_us":10011197},{"operation_name":"upsert","last_dispatch_duration_us":211395,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":825043,"total_server_duration_us":118642,"encode_duration_us":22505,"operation_id":"0x1078","timeout_ms":10000,"last_server_duration_us":20045,"total_duration_us":10009961},{"operation_name":"upsert","last_dispatch_duration_us":235741,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10007936,"total_server_duration_us":367115,"encode_duration_us":20672,"operation_id":"0x10e4","timeout_ms":10000,"last_server_duration_us":22306,"total_duration_us":10008363},{"operation_name":"upsert","last_dispatch_duration_us":9578304,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12324943,"total_server_duration_us":9769782,"encode_duration_us":16419,"operation_id":"0x1083","timeout_ms":10000,"last_server_duration_us":9259273,"total_duration_us":10008363},{"operation_name":"upsert","last_dispatch_duration_us":2191381,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11639850,"total_server_duration_us":490852,"encode_duration_us":16777,"operation_id":"0x1090","timeout_ms":10000,"last_server_duration_us":388978,"total_duration_us":10006052},{"operation_name":"upsert","last_dispatch_duration_us":182102,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":842231,"total_server_duration_us":123043,"encode_duration_us":26017,"operation_id":"0x107a","timeout_ms":10000,"last_server_duration_us":22389,"total_duration_us":10005421},{"operation_name":"upsert","last_dispatch_duration_us":8385981,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12218717,"total_server_duration_us":1606217,"encode_duration_us":18288,"operation_id":"0x1080","timeout_ms":10000,"last_server_duration_us":41875,"total_duration_us":10002155},{"operation_name":"upsert","last_dispatch_duration_us":7311991,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":7311991,"total_server_duration_us":189630,"encode_duration_us":17115,"operation_id":"0x1015","timeout_ms":10000,"last_server_duration_us":189630,"total_duration_us":10001413},{"operation_name":"upsert","last_dispatch_duration_us":9698746,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10666539,"total_server_duration_us":504501,"encode_duration_us":16394,"operation_id":"0x107d","timeout_ms":10000,"last_server_duration_us":382594,"total_duration_us":10000970},{"operation_name":"upsert","last_dispatch_duration_us":9219880,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9219880,"total_server_duration_us":9038428,"encode_duration_us":17855,"operation_id":"0x1092","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":9236168}],"total_count":22}} WARN [2022-01-10 05:37:31,162] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10169979,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":16708459,"total_server_duration_us":9970669,"encode_duration_us":16754,"operation_id":"0xfd3","timeout_ms":10000,"last_server_duration_us":9459645,"total_duration_us":10010900},{"operation_name":"upsert","last_dispatch_duration_us":7571157,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8849853,"total_server_duration_us":877203,"encode_duration_us":17367,"operation_id":"0x1103","timeout_ms":10000,"last_server_duration_us":274773,"total_duration_us":10010457},{"operation_name":"upsert","last_dispatch_duration_us":10588528,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10588528,"total_server_duration_us":10079447,"encode_duration_us":20027,"operation_id":"0xfd2","timeout_ms":10000,"last_server_duration_us":10079447,"total_duration_us":10009086},{"operation_name":"upsert","last_dispatch_duration_us":8246435,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":17014386,"total_server_duration_us":1162450,"encode_duration_us":17006,"operation_id":"0xfff","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10008049},{"operation_name":"upsert","last_dispatch_duration_us":660896,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8592886,"total_server_duration_us":1538646,"encode_duration_us":16956,"operation_id":"0xfb9","timeout_ms":10000,"last_server_duration_us":115948,"total_duration_us":10007535},{"operation_name":"upsert","last_dispatch_duration_us":5053517,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":5053517,"total_server_duration_us":277904,"encode_duration_us":16570,"operation_id":"0x10da","timeout_ms":10000,"last_server_duration_us":277904,"total_duration_us":10007213},{"operation_name":"upsert","last_dispatch_duration_us":7681046,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9366062,"total_server_duration_us":479773,"encode_duration_us":17443,"operation_id":"0x1011","timeout_ms":10000,"last_server_duration_us":265708,"total_duration_us":10006674},{"operation_name":"upsert","last_dispatch_duration_us":7499514,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8118780,"total_server_duration_us":520660,"encode_duration_us":18735,"operation_id":"0x112d","timeout_ms":10000,"last_server_duration_us":242474,"total_duration_us":10006470},{"operation_name":"upsert","last_dispatch_duration_us":7634636,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10421841,"total_server_duration_us":1199553,"encode_duration_us":19749,"operation_id":"0xfe2","timeout_ms":10000,"last_server_duration_us":331851,"total_duration_us":10004960},{"operation_name":"upsert","last_dispatch_duration_us":7660570,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10277226,"total_server_duration_us":837379,"encode_duration_us":18220,"operation_id":"0xfde","timeout_ms":10000,"last_server_duration_us":26064,"total_duration_us":10003860}],"total_count":18}} WARN [2022-01-10 05:37:31,162] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14125523,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16695255,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9465279,"encode_duration_us":20946,"operation_id":"0xf7c","timeout_ms":10000,"last_server_duration_us":9199304,"total_duration_us":10003749},{"operation_name":"upsert","last_dispatch_duration_us":9661212,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11894864,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9607197,"encode_duration_us":16716,"operation_id":"0x10b4","timeout_ms":10000,"last_server_duration_us":9387258,"total_duration_us":10010567}],"total_count":2}} INFO [2022-01-10 05:37:31,172] jepsen worker 15 - jepsen.util 375 :ok :read [5 42] INFO [2022-01-10 05:37:31,173] jepsen worker 15 - jepsen.util 375 :invoke :write [5 42] WARN [2022-01-10 05:37:31,262] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7064626,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10387271,"total_server_duration_us":144928,"encode_duration_us":19506,"operation_id":"0x1003","timeout_ms":10000,"last_server_duration_us":48679,"total_duration_us":10010826},{"operation_name":"upsert","last_dispatch_duration_us":7778670,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8876606,"total_server_duration_us":262576,"encode_duration_us":16621,"operation_id":"0x1021","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10010677},{"operation_name":"upsert","last_dispatch_duration_us":7289213,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10844034,"total_server_duration_us":173307,"encode_duration_us":19392,"operation_id":"0xffa","timeout_ms":10000,"last_server_duration_us":55425,"total_duration_us":10009524},{"operation_name":"upsert","last_dispatch_duration_us":154114,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8993857,"total_server_duration_us":292697,"encode_duration_us":30957,"operation_id":"0x10b6","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10007445},{"operation_name":"upsert","last_dispatch_duration_us":855923,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":3008061,"total_server_duration_us":310292,"encode_duration_us":16882,"operation_id":"0x1144","timeout_ms":10000,"last_server_duration_us":62284,"total_duration_us":10007102},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":16622,"operation_id":"0x10d8","timeout_ms":10000,"total_duration_us":10006985},{"operation_name":"upsert","last_dispatch_duration_us":6496696,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":6496696,"total_server_duration_us":145698,"encode_duration_us":17205,"operation_id":"0x1002","timeout_ms":10000,"last_server_duration_us":145698,"total_duration_us":10006860},{"operation_name":"upsert","last_dispatch_duration_us":7555889,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10176615,"total_server_duration_us":280661,"encode_duration_us":16691,"operation_id":"0xfe0","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10006182},{"operation_name":"upsert","last_dispatch_duration_us":9040557,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9934934,"total_server_duration_us":9205297,"encode_duration_us":20284,"operation_id":"0x1020","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10005498},{"operation_name":"upsert","last_dispatch_duration_us":487372,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":487372,"total_server_duration_us":43727,"encode_duration_us":18485,"operation_id":"0x111f","timeout_ms":10000,"last_server_duration_us":43727,"total_duration_us":10003614}],"total_count":18}} WARN [2022-01-10 05:37:31,262] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9578304,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12324943,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9769782,"encode_duration_us":16419,"operation_id":"0x1083","timeout_ms":10000,"last_server_duration_us":9259273,"total_duration_us":10008363},{"operation_name":"upsert","last_dispatch_duration_us":14411427,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14411427,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":8585898,"encode_duration_us":17403,"operation_id":"0xfb4","timeout_ms":10000,"last_server_duration_us":8585898,"total_duration_us":10011197}],"total_count":2}} INFO [2022-01-10 05:37:31,289] jepsen worker 20 - jepsen.util 290 :ok :read [6 25] INFO [2022-01-10 05:37:31,289] jepsen worker 20 - jepsen.util 290 :invoke :read [6 nil] INFO [2022-01-10 05:37:31,329] jepsen worker 39 - jepsen.util 489 :ok :read [13 23] INFO [2022-01-10 05:37:31,330] jepsen worker 39 - jepsen.util 489 :invoke :write [13 3] WARN [2022-01-10 05:37:31,362] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9681527,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":15184815,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9522663,"encode_duration_us":16500,"operation_id":"0xf90","timeout_ms":10000,"last_server_duration_us":9113921,"total_duration_us":10002625},{"operation_name":"upsert","last_dispatch_duration_us":9457356,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10487117,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10045676,"encode_duration_us":17285,"operation_id":"0x10f5","timeout_ms":10000,"last_server_duration_us":9428284,"total_duration_us":10003272},{"operation_name":"upsert","last_dispatch_duration_us":17727345,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":27288630,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9222688,"encode_duration_us":16599,"operation_id":"0xdf9","timeout_ms":10000,"last_server_duration_us":9118182,"total_duration_us":10006380},{"operation_name":"upsert","last_dispatch_duration_us":10588528,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10588528,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":10079447,"encode_duration_us":20027,"operation_id":"0xfd2","timeout_ms":10000,"last_server_duration_us":10079447,"total_duration_us":10009086},{"operation_name":"upsert","last_dispatch_duration_us":17734980,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18082559,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9230371,"encode_duration_us":19108,"operation_id":"0xf5e","timeout_ms":10000,"last_server_duration_us":9187544,"total_duration_us":10005292},{"operation_name":"upsert","last_dispatch_duration_us":10169979,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16708459,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9970669,"encode_duration_us":16754,"operation_id":"0xfd3","timeout_ms":10000,"last_server_duration_us":9459645,"total_duration_us":10010900}],"total_count":6}} WARN [2022-01-10 05:37:31,362] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9661212,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11894864,"total_server_duration_us":9607197,"encode_duration_us":16716,"operation_id":"0x10b4","timeout_ms":10000,"last_server_duration_us":9387258,"total_duration_us":10010567},{"operation_name":"upsert","last_dispatch_duration_us":4332725,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8239710,"total_server_duration_us":342029,"encode_duration_us":18314,"operation_id":"0x10f7","timeout_ms":10000,"last_server_duration_us":203590,"total_duration_us":10010092},{"operation_name":"upsert","last_dispatch_duration_us":9660972,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":15058316,"total_server_duration_us":182949,"encode_duration_us":20859,"operation_id":"0xfe1","timeout_ms":10000,"last_server_duration_us":85449,"total_duration_us":10009761},{"operation_name":"upsert","last_dispatch_duration_us":32740,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4405604,"total_server_duration_us":995219,"encode_duration_us":16925,"operation_id":"0x1121","timeout_ms":10000,"last_server_duration_us":22471,"total_duration_us":10009393},{"operation_name":"upsert","last_dispatch_duration_us":10281352,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10281352,"total_server_duration_us":62796,"encode_duration_us":18307,"operation_id":"0x112c","timeout_ms":10000,"last_server_duration_us":62796,"total_duration_us":10008531},{"operation_name":"upsert","last_dispatch_duration_us":4319008,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":4319008,"total_server_duration_us":381765,"encode_duration_us":23755,"operation_id":"0x1120","timeout_ms":10000,"last_server_duration_us":381765,"total_duration_us":10007734},{"operation_name":"upsert","last_dispatch_duration_us":160359,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8565974,"total_server_duration_us":882460,"encode_duration_us":16545,"operation_id":"0x1084","timeout_ms":10000,"last_server_duration_us":144051,"total_duration_us":10007322},{"operation_name":"upsert","last_dispatch_duration_us":224748,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":366394,"total_server_duration_us":57314,"encode_duration_us":17344,"operation_id":"0x10d1","timeout_ms":10000,"last_server_duration_us":24669,"total_duration_us":10007320},{"operation_name":"upsert","last_dispatch_duration_us":6234880,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":6234880,"total_server_duration_us":360471,"encode_duration_us":16760,"operation_id":"0x1145","timeout_ms":10000,"last_server_duration_us":360471,"total_duration_us":10004754},{"operation_name":"upsert","last_dispatch_duration_us":9555645,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10362103,"total_server_duration_us":197871,"encode_duration_us":17839,"operation_id":"0x10fa","timeout_ms":10000,"last_server_duration_us":34413,"total_duration_us":10004180}],"total_count":16}} INFO [2022-01-10 05:37:31,440] jepsen worker 64 - jepsen.util 424 :info :write [21 22] :RequestTimeoutException INFO [2022-01-10 05:37:31,442] jepsen worker 64 - jepsen.util 514 :invoke :write [21 32] INFO [2022-01-10 05:37:31,525] jepsen worker 20 - jepsen.util 290 :ok :read [6 25] INFO [2022-01-10 05:37:31,525] jepsen worker 20 - jepsen.util 290 :invoke :write [6 7] INFO [2022-01-10 05:37:31,885] jepsen worker 10 - jepsen.util 460 :info :write [3 21] :RequestTimeoutException INFO [2022-01-10 05:37:31,886] jepsen worker 10 - jepsen.util 550 :invoke :read [3 nil] INFO [2022-01-10 05:37:31,951] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:31,951] jepsen worker nemesis - couchbase.util Rebalance check count: 19 INFO [2022-01-10 05:37:31,951] jepsen worker nemesis - couchbase.util Rebalance stuck count: 18 INFO [2022-01-10 05:37:32,011] jepsen worker 73 - jepsen.util 433 :info :write [24 45] :RequestTimeoutException INFO [2022-01-10 05:37:32,014] jepsen worker 73 - jepsen.util 523 :invoke :read [24 nil] INFO [2022-01-10 05:37:32,019] jepsen worker 19 - jepsen.util 469 :info :write [6 31] :RequestTimeoutException INFO [2022-01-10 05:37:32,021] jepsen worker 19 - jepsen.util 559 :invoke :write [6 20] INFO [2022-01-10 05:37:32,027] jepsen worker 65 - jepsen.util 515 :info :write [21 31] :RequestTimeoutException INFO [2022-01-10 05:37:32,028] jepsen worker 65 - jepsen.util 605 :invoke :write [21 16] INFO [2022-01-10 05:37:32,115] jepsen worker 36 - jepsen.util 216 :ok :read [12 :nil] INFO [2022-01-10 05:37:32,117] jepsen worker 36 - jepsen.util 216 :invoke :write [12 16] INFO [2022-01-10 05:37:32,130] jepsen worker 53 - jepsen.util 413 :ok :read [17 41] INFO [2022-01-10 05:37:32,132] jepsen worker 53 - jepsen.util 413 :invoke :write [17 48] INFO [2022-01-10 05:37:32,212] jepsen worker 70 - jepsen.util 610 :info :write [23 32] :RequestTimeoutException INFO [2022-01-10 05:37:32,213] jepsen worker 70 - jepsen.util 700 :invoke :read [23 nil] INFO [2022-01-10 05:37:32,227] jepsen worker 3 - jepsen.util 543 :info :write [1 31] :RequestTimeoutException INFO [2022-01-10 05:37:32,228] jepsen worker 3 - jepsen.util 633 :invoke :read [1 nil] INFO [2022-01-10 05:37:32,412] jepsen worker 16 - jepsen.util 466 :info :write [5 11] :RequestTimeoutException INFO [2022-01-10 05:37:32,412] jepsen worker 16 - jepsen.util 556 :invoke :read [5 nil] INFO [2022-01-10 05:37:32,425] jepsen worker 59 - jepsen.util 599 :info :write [19 19] :RequestTimeoutException INFO [2022-01-10 05:37:32,426] jepsen worker 59 - jepsen.util 689 :invoke :read [19 nil] INFO [2022-01-10 05:37:32,437] jepsen worker 11 - jepsen.util 371 :info :write [3 31] :RequestTimeoutException INFO [2022-01-10 05:37:32,437] jepsen worker 11 - jepsen.util 461 :invoke :write [3 42] INFO [2022-01-10 05:37:32,690] jepsen worker 63 - jepsen.util 513 :info :write [21 36] :RequestTimeoutException INFO [2022-01-10 05:37:32,691] jepsen worker 63 - jepsen.util 603 :invoke :read [21 nil] INFO [2022-01-10 05:37:32,702] jepsen worker 18 - jepsen.util 468 :info :write [6 37] :RequestTimeoutException INFO [2022-01-10 05:37:32,703] jepsen worker 18 - jepsen.util 558 :invoke :read [6 nil] INFO [2022-01-10 05:37:32,755] jepsen worker 51 - jepsen.util 411 :ok :read [17 41] INFO [2022-01-10 05:37:32,755] jepsen worker 51 - jepsen.util 411 :invoke :read [17 nil] INFO [2022-01-10 05:37:32,872] jepsen worker 69 - jepsen.util 519 :info :write [23 15] :RequestTimeoutException INFO [2022-01-10 05:37:32,873] jepsen worker 69 - jepsen.util 609 :invoke :read [23 nil] INFO [2022-01-10 05:37:32,957] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:32,958] jepsen worker nemesis - couchbase.util Rebalance check count: 20 INFO [2022-01-10 05:37:32,958] jepsen worker nemesis - couchbase.util Rebalance stuck count: 19 INFO [2022-01-10 05:37:33,092] jepsen worker 59 - jepsen.util 689 :ok :read [19 31] INFO [2022-01-10 05:37:33,092] jepsen worker 59 - jepsen.util 689 :invoke :read [19 nil] INFO [2022-01-10 05:37:33,127] jepsen worker 83 - jepsen.util 533 :info :write [27 45] :RequestTimeoutException INFO [2022-01-10 05:37:33,127] jepsen worker 83 - jepsen.util 623 :invoke :write [27 12] INFO [2022-01-10 05:37:33,365] jepsen worker 43 - jepsen.util 493 :info :write [14 11] :RequestTimeoutException INFO [2022-01-10 05:37:33,365] jepsen worker 43 - jepsen.util 583 :invoke :write [14 2] INFO [2022-01-10 05:37:33,520] jepsen worker 8 - jepsen.util 458 :info :write [2 35] :RequestTimeoutException INFO [2022-01-10 05:37:33,521] jepsen worker 8 - jepsen.util 548 :invoke :read [2 nil] INFO [2022-01-10 05:37:33,544] jepsen worker 8 - jepsen.util 548 :ok :read [2 21] INFO [2022-01-10 05:37:33,545] jepsen worker 8 - jepsen.util 548 :invoke :write [2 40] INFO [2022-01-10 05:37:33,562] jepsen worker 21 - jepsen.util 561 :info :write [7 25] :RequestTimeoutException INFO [2022-01-10 05:37:33,564] jepsen worker 21 - jepsen.util 651 :invoke :write [7 22] INFO [2022-01-10 05:37:33,577] jepsen worker 88 - jepsen.util 538 :info :write [29 28] :RequestTimeoutException INFO [2022-01-10 05:37:33,578] jepsen worker 88 - jepsen.util 628 :invoke :read [29 nil] INFO [2022-01-10 05:37:33,643] jepsen worker 59 - jepsen.util 689 :ok :read [19 31] INFO [2022-01-10 05:37:33,644] jepsen worker 59 - jepsen.util 689 :invoke :read [19 nil] INFO [2022-01-10 05:37:33,672] jepsen worker 46 - jepsen.util 406 :info :write [15 7] :RequestTimeoutException INFO [2022-01-10 05:37:33,672] jepsen worker 46 - jepsen.util 496 :invoke :write [15 40] INFO [2022-01-10 05:37:33,725] jepsen worker 31 - jepsen.util 661 :ok :read [10 37] INFO [2022-01-10 05:37:33,725] jepsen worker 31 - jepsen.util 661 :invoke :write [10 39] INFO [2022-01-10 05:37:33,733] jepsen worker 5 - jepsen.util 545 :ok :read [1 41] INFO [2022-01-10 05:37:33,733] jepsen worker 5 - jepsen.util 545 :invoke :read [1 nil] INFO [2022-01-10 05:37:33,781] jepsen worker 71 - jepsen.util 521 :ok :read [23 :nil] INFO [2022-01-10 05:37:33,782] jepsen worker 71 - jepsen.util 521 :invoke :write [23 35] INFO [2022-01-10 05:37:33,888] jepsen worker 60 - jepsen.util 600 :ok :read [20 :nil] INFO [2022-01-10 05:37:33,889] jepsen worker 60 - jepsen.util 600 :invoke :write [20 43] INFO [2022-01-10 05:37:33,932] jepsen worker 37 - jepsen.util 577 :info :write [12 8] :RequestTimeoutException INFO [2022-01-10 05:37:33,933] jepsen worker 37 - jepsen.util 667 :invoke :read [12 nil] INFO [2022-01-10 05:37:33,964] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:33,964] jepsen worker nemesis - couchbase.util Rebalance check count: 21 INFO [2022-01-10 05:37:33,964] jepsen worker nemesis - couchbase.util Rebalance stuck count: 20 INFO [2022-01-10 05:37:33,976] jepsen worker 23 - jepsen.util 383 :info :write [7 20] :RequestTimeoutException INFO [2022-01-10 05:37:33,977] jepsen worker 23 - jepsen.util 473 :invoke :read [7 nil] INFO [2022-01-10 05:37:34,026] jepsen worker 49 - jepsen.util 409 :ok :read [16 :nil] INFO [2022-01-10 05:37:34,028] jepsen worker 49 - jepsen.util 409 :invoke :read [16 nil] INFO [2022-01-10 05:37:34,054] jepsen worker 40 - jepsen.util 400 :ok :read [13 23] INFO [2022-01-10 05:37:34,054] jepsen worker 40 - jepsen.util 400 :invoke :read [13 nil] INFO [2022-01-10 05:37:34,112] jepsen worker 27 - jepsen.util 387 :info :write [9 23] :RequestTimeoutException INFO [2022-01-10 05:37:34,112] jepsen worker 27 - jepsen.util 477 :invoke :write [9 40] INFO [2022-01-10 05:37:34,156] jepsen worker 61 - jepsen.util 601 :info :write [20 35] :RequestTimeoutException INFO [2022-01-10 05:37:34,157] jepsen worker 61 - jepsen.util 691 :invoke :read [20 nil] INFO [2022-01-10 05:37:34,159] jepsen worker 52 - jepsen.util 592 :ok :read [17 41] INFO [2022-01-10 05:37:34,160] jepsen worker 52 - jepsen.util 592 :invoke :write [17 14] INFO [2022-01-10 05:37:34,181] jepsen worker 84 - jepsen.util 624 :ok :read [28 :nil] INFO [2022-01-10 05:37:34,182] jepsen worker 84 - jepsen.util 624 :invoke :write [28 1] INFO [2022-01-10 05:37:34,195] jepsen worker 49 - jepsen.util 409 :ok :read [16 :nil] INFO [2022-01-10 05:37:34,196] jepsen worker 49 - jepsen.util 409 :invoke :read [16 nil] INFO [2022-01-10 05:37:34,203] jepsen worker 62 - jepsen.util 692 :ok :read [20 :nil] INFO [2022-01-10 05:37:34,204] jepsen worker 62 - jepsen.util 692 :invoke :read [20 nil] INFO [2022-01-10 05:37:34,207] jepsen worker 25 - jepsen.util 475 :ok :read [8 :nil] INFO [2022-01-10 05:37:34,208] jepsen worker 25 - jepsen.util 475 :invoke :read [8 nil] INFO [2022-01-10 05:37:34,234] jepsen worker 76 - jepsen.util 526 :ok :read [25 :nil] INFO [2022-01-10 05:37:34,234] jepsen worker 76 - jepsen.util 526 :invoke :write [25 30] INFO [2022-01-10 05:37:34,271] jepsen worker 62 - jepsen.util 692 :ok :read [20 :nil] INFO [2022-01-10 05:37:34,271] jepsen worker 23 - jepsen.util 473 :ok :read [7 :nil] INFO [2022-01-10 05:37:34,273] jepsen worker 62 - jepsen.util 692 :invoke :write [20 34] INFO [2022-01-10 05:37:34,273] jepsen worker 23 - jepsen.util 473 :invoke :write [7 40] INFO [2022-01-10 05:37:34,283] jepsen worker 25 - jepsen.util 475 :ok :read [8 :nil] INFO [2022-01-10 05:37:34,284] jepsen worker 25 - jepsen.util 475 :invoke :write [8 21] INFO [2022-01-10 05:37:34,289] jepsen worker 59 - jepsen.util 689 :ok :read [19 31] INFO [2022-01-10 05:37:34,291] jepsen worker 59 - jepsen.util 689 :invoke :read [19 nil] INFO [2022-01-10 05:37:34,297] jepsen worker 73 - jepsen.util 523 :ok :read [24 33] INFO [2022-01-10 05:37:34,297] jepsen worker 73 - jepsen.util 523 :invoke :read [24 nil] INFO [2022-01-10 05:37:34,381] jepsen worker 40 - jepsen.util 400 :ok :read [13 23] INFO [2022-01-10 05:37:34,382] jepsen worker 40 - jepsen.util 400 :invoke :write [13 46] INFO [2022-01-10 05:37:34,436] jepsen worker 73 - jepsen.util 523 :ok :read [24 33] INFO [2022-01-10 05:37:34,438] jepsen worker 73 - jepsen.util 523 :invoke :write [24 33] INFO [2022-01-10 05:37:34,464] jepsen worker 4 - jepsen.util 454 :ok :read [1 41] INFO [2022-01-10 05:37:34,465] jepsen worker 4 - jepsen.util 454 :invoke :write [1 0] INFO [2022-01-10 05:37:34,489] jepsen worker 51 - jepsen.util 411 :ok :read [17 41] INFO [2022-01-10 05:37:34,490] jepsen worker 51 - jepsen.util 411 :invoke :write [17 26] INFO [2022-01-10 05:37:34,501] jepsen worker 22 - jepsen.util 472 :info :write [7 15] :RequestTimeoutException INFO [2022-01-10 05:37:34,502] jepsen worker 22 - jepsen.util 562 :invoke :read [7 nil] INFO [2022-01-10 05:37:34,573] jepsen worker 49 - jepsen.util 409 :ok :read [16 :nil] INFO [2022-01-10 05:37:34,574] jepsen worker 49 - jepsen.util 409 :invoke :read [16 nil] INFO [2022-01-10 05:37:34,626] jepsen worker 59 - jepsen.util 689 :ok :read [19 31] INFO [2022-01-10 05:37:34,627] jepsen worker 59 - jepsen.util 689 :invoke :write [19 3] INFO [2022-01-10 05:37:34,686] jepsen worker 1 - jepsen.util 451 :info :write [0 43] :RequestTimeoutException INFO [2022-01-10 05:37:34,687] jepsen worker 1 - jepsen.util 541 :invoke :write [0 2] INFO [2022-01-10 05:37:34,775] jepsen worker 38 - jepsen.util 578 :ok :read [12 :nil] INFO [2022-01-10 05:37:34,776] jepsen worker 38 - jepsen.util 578 :invoke :read [12 nil] INFO [2022-01-10 05:37:34,792] jepsen worker 35 - jepsen.util 575 :info :write [11 34] :RequestTimeoutException INFO [2022-01-10 05:37:34,793] jepsen worker 35 - jepsen.util 665 :invoke :write [11 44] INFO [2022-01-10 05:37:34,927] jepsen worker 42 - jepsen.util 402 :info :write [14 12] :RequestTimeoutException INFO [2022-01-10 05:37:34,927] jepsen worker 42 - jepsen.util 492 :invoke :read [14 nil] INFO [2022-01-10 05:37:34,966] jepsen worker 48 - jepsen.util 318 :info :write [16 1] :RequestTimeoutException INFO [2022-01-10 05:37:34,967] jepsen worker 48 - jepsen.util 408 :invoke :write [16 1] INFO [2022-01-10 05:37:35,007] jepsen worker 88 - jepsen.util 628 :ok :read [29 33] INFO [2022-01-10 05:37:35,008] jepsen worker 88 - jepsen.util 628 :invoke :read [29 nil] INFO [2022-01-10 05:37:35,008] jepsen worker 87 - jepsen.util 627 :ok :read [29 33] INFO [2022-01-10 05:37:35,009] jepsen worker 87 - jepsen.util 627 :invoke :write [29 44] INFO [2022-01-10 05:37:35,013] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:35,013] jepsen worker nemesis - couchbase.util Rebalance check count: 22 INFO [2022-01-10 05:37:35,013] jepsen worker nemesis - couchbase.util Rebalance stuck count: 21 INFO [2022-01-10 05:37:35,082] jepsen worker 86 - jepsen.util 446 :ok :read [28 :nil] INFO [2022-01-10 05:37:35,082] jepsen worker 86 - jepsen.util 446 :invoke :read [28 nil] INFO [2022-01-10 05:37:35,117] jepsen worker 49 - jepsen.util 409 :ok :read [16 :nil] INFO [2022-01-10 05:37:35,118] jepsen worker 49 - jepsen.util 409 :invoke :read [16 nil] INFO [2022-01-10 05:37:35,127] jepsen worker 17 - jepsen.util 377 :ok :read [5 42] INFO [2022-01-10 05:37:35,128] jepsen worker 17 - jepsen.util 377 :invoke :write [5 46] INFO [2022-01-10 05:37:35,153] jepsen worker 42 - jepsen.util 492 :ok :read [14 14] INFO [2022-01-10 05:37:35,153] jepsen worker 42 - jepsen.util 492 :invoke :write [14 32] INFO [2022-01-10 05:37:35,204] jepsen worker 5 - jepsen.util 545 :ok :read [1 41] INFO [2022-01-10 05:37:35,205] jepsen worker 5 - jepsen.util 545 :invoke :write [1 12] INFO [2022-01-10 05:37:35,376] jepsen worker 44 - jepsen.util 494 :info :write [14 12] :RequestTimeoutException INFO [2022-01-10 05:37:35,377] jepsen worker 44 - jepsen.util 584 :invoke :read [14 nil] INFO [2022-01-10 05:37:35,383] jepsen worker 69 - jepsen.util 609 :ok :read [23 :nil] INFO [2022-01-10 05:37:35,383] jepsen worker 69 - jepsen.util 609 :invoke :read [23 nil] INFO [2022-01-10 05:37:35,399] jepsen worker 3 - jepsen.util 633 :ok :read [1 41] INFO [2022-01-10 05:37:35,399] jepsen worker 3 - jepsen.util 633 :invoke :write [1 49] INFO [2022-01-10 05:37:35,775] jepsen worker 70 - jepsen.util 700 :ok :read [23 :nil] INFO [2022-01-10 05:37:35,775] jepsen worker 38 - jepsen.util 578 :ok :read [12 :nil] INFO [2022-01-10 05:37:35,775] jepsen worker 57 - jepsen.util 507 :info :write [19 37] :RequestTimeoutException INFO [2022-01-10 05:37:35,777] jepsen worker 38 - jepsen.util 578 :invoke :write [12 5] INFO [2022-01-10 05:37:35,777] jepsen worker 70 - jepsen.util 700 :invoke :read [23 nil] INFO [2022-01-10 05:37:35,778] jepsen worker 57 - jepsen.util 597 :invoke :write [19 48] INFO [2022-01-10 05:37:35,789] jepsen worker 88 - jepsen.util 628 :ok :read [29 33] INFO [2022-01-10 05:37:35,789] jepsen worker 88 - jepsen.util 628 :invoke :read [29 nil] INFO [2022-01-10 05:37:35,806] jepsen worker 9 - jepsen.util 549 :info :write [3 26] :RequestTimeoutException INFO [2022-01-10 05:37:35,807] jepsen worker 9 - jepsen.util 639 :invoke :write [3 13] INFO [2022-01-10 05:37:35,843] jepsen worker 29 - jepsen.util 479 :ok :read [9 22] INFO [2022-01-10 05:37:35,844] jepsen worker 29 - jepsen.util 479 :invoke :read [9 nil] INFO [2022-01-10 05:37:35,867] jepsen worker 34 - jepsen.util 574 :info :write [11 13] :SyncWriteAmbiguous INFO [2022-01-10 05:37:35,867] jepsen worker 34 - jepsen.util 664 :invoke :write [11 49] INFO [2022-01-10 05:37:35,971] jepsen worker 56 - jepsen.util 506 :info :write [18 16] :RequestTimeoutException INFO [2022-01-10 05:37:35,972] jepsen worker 56 - jepsen.util 596 :invoke :write [18 6] INFO [2022-01-10 05:37:36,020] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:36,020] jepsen worker nemesis - couchbase.util Rebalance check count: 23 INFO [2022-01-10 05:37:36,020] jepsen worker nemesis - couchbase.util Rebalance stuck count: 22 INFO [2022-01-10 05:37:36,327] jepsen worker 2 - jepsen.util 362 :info :write [0 35] :RequestTimeoutException INFO [2022-01-10 05:37:36,328] jepsen worker 2 - jepsen.util 452 :invoke :write [0 22] INFO [2022-01-10 05:37:36,341] jepsen worker 78 - jepsen.util 528 :ok :read [26 10] INFO [2022-01-10 05:37:36,342] jepsen worker 78 - jepsen.util 528 :invoke :write [26 30] INFO [2022-01-10 05:37:36,350] jepsen worker 49 - jepsen.util 409 :ok :read [16 :nil] INFO [2022-01-10 05:37:36,351] jepsen worker 49 - jepsen.util 409 :invoke :read [16 nil] INFO [2022-01-10 05:37:36,821] jepsen worker 74 - jepsen.util 614 :info :write [24 18] :RequestTimeoutException INFO [2022-01-10 05:37:36,821] jepsen worker 74 - jepsen.util 704 :invoke :read [24 nil] INFO [2022-01-10 05:37:36,877] jepsen worker 7 - jepsen.util 367 :info :write [2 1] :SyncWriteAmbiguous INFO [2022-01-10 05:37:36,878] jepsen worker 7 - jepsen.util 457 :invoke :write [2 20] INFO [2022-01-10 05:37:36,953] jepsen worker 49 - jepsen.util 409 :ok :read [16 :nil] INFO [2022-01-10 05:37:36,954] jepsen worker 49 - jepsen.util 409 :invoke :write [16 39] INFO [2022-01-10 05:37:37,028] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:37,028] jepsen worker nemesis - couchbase.util Rebalance check count: 24 INFO [2022-01-10 05:37:37,028] jepsen worker nemesis - couchbase.util Rebalance stuck count: 23 INFO [2022-01-10 05:37:37,216] jepsen worker 70 - jepsen.util 700 :ok :read [23 :nil] INFO [2022-01-10 05:37:37,217] jepsen worker 70 - jepsen.util 700 :invoke :write [23 14] INFO [2022-01-10 05:37:37,236] jepsen worker 88 - jepsen.util 628 :ok :read [29 33] INFO [2022-01-10 05:37:37,237] jepsen worker 88 - jepsen.util 628 :invoke :read [29 nil] INFO [2022-01-10 05:37:37,376] jepsen worker 32 - jepsen.util 482 :info :write [10 36] :RequestTimeoutException INFO [2022-01-10 05:37:37,377] jepsen worker 32 - jepsen.util 572 :invoke :write [10 29] INFO [2022-01-10 05:37:37,387] jepsen worker 54 - jepsen.util 324 :info :write [18 9] :RequestTimeoutException INFO [2022-01-10 05:37:37,387] jepsen worker 54 - jepsen.util 414 :invoke :write [18 1] INFO [2022-01-10 05:37:37,440] jepsen worker 58 - jepsen.util 508 :info :write [19 42] :RequestTimeoutException INFO [2022-01-10 05:37:37,441] jepsen worker 58 - jepsen.util 598 :invoke :write [19 16] INFO [2022-01-10 05:37:37,889] jepsen worker 88 - jepsen.util 628 :ok :read [29 33] INFO [2022-01-10 05:37:37,890] jepsen worker 88 - jepsen.util 628 :invoke :write [29 14] INFO [2022-01-10 05:37:37,982] jepsen worker 66 - jepsen.util 336 :info :write [22 10] :RequestTimeoutException INFO [2022-01-10 05:37:37,982] jepsen worker 66 - jepsen.util 426 :invoke :read [22 nil] INFO [2022-01-10 05:37:38,025] jepsen worker 12 - jepsen.util 462 :info :write [4 48] :RequestTimeoutException INFO [2022-01-10 05:37:38,025] jepsen worker 12 - jepsen.util 552 :invoke :read [4 nil] INFO [2022-01-10 05:37:38,032] jepsen worker 14 - jepsen.util 464 :info :write [4 3] :RequestTimeoutException INFO [2022-01-10 05:37:38,033] jepsen worker 14 - jepsen.util 554 :invoke :write [4 25] INFO [2022-01-10 05:37:38,034] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:38,034] jepsen worker nemesis - couchbase.util Rebalance check count: 25 INFO [2022-01-10 05:37:38,034] jepsen worker nemesis - couchbase.util Rebalance stuck count: 24 INFO [2022-01-10 05:37:38,276] jepsen worker 68 - jepsen.util 428 :info :write [22 8] :RequestTimeoutException INFO [2022-01-10 05:37:38,277] jepsen worker 68 - jepsen.util 518 :invoke :write [22 45] INFO [2022-01-10 05:37:38,327] jepsen worker 77 - jepsen.util 617 :info :write [25 10] :RequestTimeoutException INFO [2022-01-10 05:37:38,327] jepsen worker 77 - jepsen.util 707 :invoke :read [25 nil] INFO [2022-01-10 05:37:38,364] jepsen worker 77 - jepsen.util 707 :ok :read [25 :nil] INFO [2022-01-10 05:37:38,365] jepsen worker 77 - jepsen.util 707 :invoke :read [25 nil] INFO [2022-01-10 05:37:38,367] jepsen worker 77 - jepsen.util 707 :ok :read [25 :nil] INFO [2022-01-10 05:37:38,368] jepsen worker 77 - jepsen.util 707 :invoke :write [25 43] INFO [2022-01-10 05:37:38,747] jepsen worker 47 - jepsen.util 587 :info :write [15 6] :RequestTimeoutException INFO [2022-01-10 05:37:38,747] jepsen worker 47 - jepsen.util 677 :invoke :write [15 7] INFO [2022-01-10 05:37:38,756] jepsen worker 75 - jepsen.util 525 :info :write [25 32] :RequestTimeoutException INFO [2022-01-10 05:37:38,757] jepsen worker 75 - jepsen.util 615 :invoke :read [25 nil] INFO [2022-01-10 05:37:38,871] jepsen worker 89 - jepsen.util 719 :info :write [29 12] :RequestTimeoutException INFO [2022-01-10 05:37:38,872] jepsen worker 89 - jepsen.util 809 :invoke :read [29 nil] INFO [2022-01-10 05:37:38,912] jepsen worker 61 - jepsen.util 691 :ok :read [20 :nil] INFO [2022-01-10 05:37:38,913] jepsen worker 61 - jepsen.util 691 :invoke :write [20 11] INFO [2022-01-10 05:37:38,966] jepsen worker 67 - jepsen.util 427 :info :write [22 7] :RequestTimeoutException INFO [2022-01-10 05:37:38,967] jepsen worker 67 - jepsen.util 517 :invoke :write [22 23] INFO [2022-01-10 05:37:38,974] jepsen worker 63 - jepsen.util 603 :ok :read [21 7] INFO [2022-01-10 05:37:38,975] jepsen worker 63 - jepsen.util 603 :invoke :read [21 nil] INFO [2022-01-10 05:37:38,979] jepsen worker 74 - jepsen.util 704 :ok :read [24 33] INFO [2022-01-10 05:37:38,979] jepsen worker 74 - jepsen.util 704 :invoke :read [24 nil] INFO [2022-01-10 05:37:38,980] jepsen worker 79 - jepsen.util 529 :info :write [26 47] :RequestTimeoutException INFO [2022-01-10 05:37:38,980] jepsen worker 79 - jepsen.util 619 :invoke :read [26 nil] INFO [2022-01-10 05:37:39,012] jepsen worker 85 - jepsen.util 535 :info :write [28 41] :RequestTimeoutException INFO [2022-01-10 05:37:39,013] jepsen worker 85 - jepsen.util 625 :invoke :read [28 nil] INFO [2022-01-10 05:37:39,015] jepsen worker 24 - jepsen.util 474 :info :write [8 18] :RequestTimeoutException INFO [2022-01-10 05:37:39,015] jepsen worker 24 - jepsen.util 564 :invoke :write [8 18] INFO [2022-01-10 05:37:39,027] jepsen worker 45 - jepsen.util 585 :info :write [15 6] :RequestTimeoutException INFO [2022-01-10 05:37:39,027] jepsen worker 45 - jepsen.util 675 :invoke :read [15 nil] INFO [2022-01-10 05:37:39,080] jepsen worker 63 - jepsen.util 603 :ok :read [21 7] INFO [2022-01-10 05:37:39,080] jepsen worker 63 - jepsen.util 603 :invoke :read [21 nil] INFO [2022-01-10 05:37:39,110] jepsen worker 30 - jepsen.util 390 :ok :read [10 37] INFO [2022-01-10 05:37:39,111] jepsen worker 30 - jepsen.util 390 :invoke :write [10 5] INFO [2022-01-10 05:37:39,112] jepsen worker 74 - jepsen.util 704 :ok :read [24 33] INFO [2022-01-10 05:37:39,113] jepsen worker 74 - jepsen.util 704 :invoke :write [24 47] INFO [2022-01-10 05:37:39,165] jepsen worker 33 - jepsen.util 393 :info :write [11 28] :RequestTimeoutException INFO [2022-01-10 05:37:39,165] jepsen worker 33 - jepsen.util 483 :invoke :read [11 nil] INFO [2022-01-10 05:37:39,167] jepsen worker 44 - jepsen.util 584 :ok :read [14 14] INFO [2022-01-10 05:37:39,168] jepsen worker 44 - jepsen.util 584 :invoke :read [14 nil] INFO [2022-01-10 05:37:39,198] jepsen worker 63 - jepsen.util 603 :ok :read [21 7] INFO [2022-01-10 05:37:39,198] jepsen worker 63 - jepsen.util 603 :invoke :read [21 nil] INFO [2022-01-10 05:37:39,212] jepsen worker 55 - jepsen.util 505 :info :write [18 21] :RequestTimeoutException INFO [2022-01-10 05:37:39,212] jepsen worker 55 - jepsen.util 595 :invoke :read [18 nil] INFO [2022-01-10 05:37:39,274] jepsen worker 33 - jepsen.util 483 :ok :read [11 21] INFO [2022-01-10 05:37:39,275] jepsen worker 33 - jepsen.util 483 :invoke :read [11 nil] INFO [2022-01-10 05:37:39,309] jepsen worker 44 - jepsen.util 584 :ok :read [14 14] INFO [2022-01-10 05:37:39,310] jepsen worker 44 - jepsen.util 584 :invoke :read [14 nil] INFO [2022-01-10 05:37:39,318] jepsen worker 63 - jepsen.util 603 :ok :read [21 7] INFO [2022-01-10 05:37:39,319] jepsen worker 63 - jepsen.util 603 :invoke :write [21 26] INFO [2022-01-10 05:37:39,346] jepsen worker 82 - jepsen.util 352 :info :write [27 0] :RequestTimeoutException INFO [2022-01-10 05:37:39,347] jepsen worker 82 - jepsen.util 442 :invoke :read [27 nil] INFO [2022-01-10 05:37:39,362] jepsen worker 33 - jepsen.util 483 :ok :read [11 21] INFO [2022-01-10 05:37:39,363] jepsen worker 33 - jepsen.util 483 :invoke :write [11 23] INFO [2022-01-10 05:37:39,389] jepsen worker 44 - jepsen.util 584 :ok :read [14 14] INFO [2022-01-10 05:37:39,390] jepsen worker 44 - jepsen.util 584 :invoke :write [14 26] INFO [2022-01-10 05:37:39,433] jepsen worker 18 - jepsen.util 558 :ok :read [6 25] INFO [2022-01-10 05:37:39,433] jepsen worker 18 - jepsen.util 558 :invoke :write [6 25] WARN [2022-01-10 05:37:39,490] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12895603,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15749310,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":10860925,"encode_duration_us":19875,"operation_id":"0x10d2","timeout_ms":10000,"last_server_duration_us":9518159,"total_duration_us":10001875},{"operation_name":"upsert","last_dispatch_duration_us":9149376,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10160505,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9486791,"encode_duration_us":19499,"operation_id":"0x112a","timeout_ms":10000,"last_server_duration_us":9133103,"total_duration_us":10003119},{"operation_name":"upsert","last_dispatch_duration_us":9286916,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11255371,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9795413,"encode_duration_us":17871,"operation_id":"0x10f9","timeout_ms":10000,"last_server_duration_us":9059667,"total_duration_us":10011083}],"total_count":3}} WARN [2022-01-10 05:37:39,590] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10026580,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10026580,"total_server_duration_us":6927,"operation_id":"0x11a2","timeout_ms":30000,"last_server_duration_us":6927,"total_duration_us":10027039},{"operation_name":"upsert","last_dispatch_duration_us":6403151,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":6403151,"total_server_duration_us":411245,"encode_duration_us":17876,"operation_id":"0x1241","timeout_ms":10000,"last_server_duration_us":411245,"total_duration_us":10009858},{"operation_name":"upsert","last_dispatch_duration_us":9093043,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9256785,"total_server_duration_us":302793,"encode_duration_us":17522,"operation_id":"0x11d6","timeout_ms":10000,"last_server_duration_us":218622,"total_duration_us":10009037},{"operation_name":"upsert","last_dispatch_duration_us":9098289,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9652946,"total_server_duration_us":425766,"encode_duration_us":16752,"operation_id":"0x116c","timeout_ms":10000,"last_server_duration_us":126982,"total_duration_us":10007247},{"operation_name":"upsert","last_dispatch_duration_us":96214,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2025181,"total_server_duration_us":860295,"encode_duration_us":16999,"operation_id":"0x126e","timeout_ms":10000,"last_server_duration_us":72045,"total_duration_us":10005558},{"operation_name":"upsert","last_dispatch_duration_us":8491767,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":12093181,"total_server_duration_us":369473,"encode_duration_us":16317,"operation_id":"0x115f","timeout_ms":10000,"last_server_duration_us":186563,"total_duration_us":10004934},{"operation_name":"upsert","last_dispatch_duration_us":32375,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1928517,"total_server_duration_us":448467,"encode_duration_us":23488,"operation_id":"0x1269","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10004325},{"operation_name":"upsert","last_dispatch_duration_us":9097671,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10297764,"total_server_duration_us":810074,"encode_duration_us":19474,"operation_id":"0x111e","timeout_ms":10000,"last_server_duration_us":115449,"total_duration_us":10003958},{"operation_name":"upsert","last_dispatch_duration_us":347558,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9187872,"total_server_duration_us":624183,"encode_duration_us":17031,"operation_id":"0x1130","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10003178},{"operation_name":"upsert","last_dispatch_duration_us":9149376,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10160505,"total_server_duration_us":9486791,"encode_duration_us":19499,"operation_id":"0x112a","timeout_ms":10000,"last_server_duration_us":9133103,"total_duration_us":10003119}],"total_count":15}} INFO [2022-01-10 05:37:39,759] jepsen worker 22 - jepsen.util 562 :ok :read [7 :nil] INFO [2022-01-10 05:37:39,760] jepsen worker 22 - jepsen.util 562 :invoke :write [7 13] INFO [2022-01-10 05:37:39,863] jepsen worker 37 - jepsen.util 667 :ok :read [12 :nil] INFO [2022-01-10 05:37:39,864] jepsen worker 37 - jepsen.util 667 :invoke :write [12 23] INFO [2022-01-10 05:37:39,884] jepsen worker 10 - jepsen.util 550 :ok :read [3 2] INFO [2022-01-10 05:37:39,885] jepsen worker 10 - jepsen.util 550 :invoke :read [3 nil] INFO [2022-01-10 05:37:39,917] jepsen worker 26 - jepsen.util 566 :info :write [8 38] :RequestTimeoutException INFO [2022-01-10 05:37:39,919] jepsen worker 26 - jepsen.util 656 :invoke :write [8 3] INFO [2022-01-10 05:37:39,922] jepsen worker 0 - jepsen.util 450 :info :write [0 8] :RequestTimeoutException INFO [2022-01-10 05:37:39,923] jepsen worker 0 - jepsen.util 540 :invoke :read [0 nil] INFO [2022-01-10 05:37:40,062] jepsen worker 80 - jepsen.util 530 :info :write [26 34] :RequestTimeoutException INFO [2022-01-10 05:37:40,070] jepsen worker 80 - jepsen.util 620 :invoke :write [26 1] INFO [2022-01-10 05:37:40,222] jepsen worker 50 - jepsen.util 590 :info :write [16 38] :RequestTimeoutException INFO [2022-01-10 05:37:40,223] jepsen worker 50 - jepsen.util 680 :invoke :read [16 nil] INFO [2022-01-10 05:37:40,246] jepsen worker 13 - jepsen.util 463 :info :write [4 25] :RequestTimeoutException INFO [2022-01-10 05:37:40,247] jepsen worker 13 - jepsen.util 553 :invoke :read [4 nil] INFO [2022-01-10 05:37:40,335] jepsen worker 6 - jepsen.util 456 :info :write [2 40] :RequestTimeoutException INFO [2022-01-10 05:37:40,335] jepsen worker 6 - jepsen.util 546 :invoke :read [2 nil] INFO [2022-01-10 05:37:40,364] jepsen worker 28 - jepsen.util 568 :info :write [9 32] :RequestTimeoutException INFO [2022-01-10 05:37:40,365] jepsen worker 28 - jepsen.util 658 :invoke :read [9 nil] INFO [2022-01-10 05:37:40,553] jepsen worker 64 - jepsen.util 514 :info :write [21 32] :SyncWriteAmbiguous INFO [2022-01-10 05:37:40,554] jepsen worker 64 - jepsen.util 604 :invoke :write [21 4] INFO [2022-01-10 05:37:40,555] jepsen worker 12 - jepsen.util 552 :ok :read [4 :nil] INFO [2022-01-10 05:37:40,556] jepsen worker 12 - jepsen.util 552 :invoke :write [4 29] INFO [2022-01-10 05:37:40,676] jepsen worker 72 - jepsen.util 702 :info :write [24 33] :RequestTimeoutException INFO [2022-01-10 05:37:40,677] jepsen worker 72 - jepsen.util 792 :invoke :read [24 nil] INFO [2022-01-10 05:37:40,831] jepsen worker 6 - jepsen.util 546 :ok :read [2 21] INFO [2022-01-10 05:37:40,832] jepsen worker 6 - jepsen.util 546 :invoke :write [2 19] INFO [2022-01-10 05:37:40,910] jepsen worker 81 - jepsen.util 531 :info :write [27 41] :RequestTimeoutException INFO [2022-01-10 05:37:40,911] jepsen worker 81 - jepsen.util 621 :invoke :write [27 9] INFO [2022-01-10 05:37:40,930] jepsen worker 41 - jepsen.util 401 :info :write [13 43] :RequestTimeoutException INFO [2022-01-10 05:37:40,930] jepsen worker 41 - jepsen.util 491 :invoke :read [13 nil] INFO [2022-01-10 05:37:41,030] jepsen worker 89 - jepsen.util 809 :ok :read [29 33] INFO [2022-01-10 05:37:41,031] jepsen worker 89 - jepsen.util 809 :invoke :read [29 nil] WARN [2022-01-10 05:37:41,079] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":20712672,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20712672,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9122444,"encode_duration_us":17201,"operation_id":"0xff9","timeout_ms":10000,"last_server_duration_us":9122444,"total_duration_us":10001452},{"operation_name":"upsert","last_dispatch_duration_us":7592362,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10201783,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":7781149,"encode_duration_us":20021,"operation_id":"0x121e","timeout_ms":10000,"last_server_duration_us":7405761,"total_duration_us":10007955}],"total_count":2}} WARN [2022-01-10 05:37:41,079] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":785753,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":4915281,"total_server_duration_us":812270,"encode_duration_us":17078,"operation_id":"0x1263","timeout_ms":10000,"last_server_duration_us":174705,"total_duration_us":10010410},{"operation_name":"upsert","last_dispatch_duration_us":7592362,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10201783,"total_server_duration_us":7781149,"encode_duration_us":20021,"operation_id":"0x121e","timeout_ms":10000,"last_server_duration_us":7405761,"total_duration_us":10007955},{"operation_name":"upsert","last_dispatch_duration_us":3051811,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9940012,"total_server_duration_us":891323,"encode_duration_us":17778,"operation_id":"0x11ab","timeout_ms":10000,"last_server_duration_us":235006,"total_duration_us":10006368},{"operation_name":"upsert","last_dispatch_duration_us":7122769,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8900598,"total_server_duration_us":337266,"encode_duration_us":16752,"operation_id":"0x1191","timeout_ms":10000,"last_server_duration_us":24583,"total_duration_us":10006018},{"operation_name":"upsert","last_dispatch_duration_us":11615800,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11615800,"total_server_duration_us":67353,"encode_duration_us":17906,"operation_id":"0x116b","timeout_ms":10000,"last_server_duration_us":67353,"total_duration_us":10005975},{"operation_name":"upsert","last_dispatch_duration_us":7232625,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9535849,"total_server_duration_us":761924,"encode_duration_us":16469,"operation_id":"0x11b7","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10004864},{"operation_name":"upsert","last_dispatch_duration_us":373764,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":6572597,"total_server_duration_us":101070,"encode_duration_us":16803,"operation_id":"0x1240","timeout_ms":10000,"last_server_duration_us":50652,"total_duration_us":10003447},{"operation_name":"upsert","last_dispatch_duration_us":954561,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":4880388,"total_server_duration_us":1303466,"encode_duration_us":16201,"operation_id":"0x126f","timeout_ms":10000,"last_server_duration_us":360471,"total_duration_us":10003018},{"operation_name":"upsert","last_dispatch_duration_us":7074165,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9814909,"total_server_duration_us":383102,"encode_duration_us":16756,"operation_id":"0x11ca","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10002953},{"operation_name":"upsert","last_dispatch_duration_us":1181688,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":7212544,"total_server_duration_us":1036009,"encode_duration_us":16594,"operation_id":"0x11fa","timeout_ms":10000,"last_server_duration_us":22554,"total_duration_us":10001894}],"total_count":30}} WARN [2022-01-10 05:37:41,175] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2704232,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":11391635,"total_server_duration_us":1284567,"encode_duration_us":17380,"operation_id":"0x1147","timeout_ms":10000,"last_server_duration_us":395126,"total_duration_us":10010562},{"operation_name":"upsert","last_dispatch_duration_us":6686036,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9158532,"total_server_duration_us":367713,"encode_duration_us":19896,"operation_id":"0x126a","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10008714},{"operation_name":"upsert","last_dispatch_duration_us":6760602,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9342914,"total_server_duration_us":524960,"encode_duration_us":19720,"operation_id":"0x125a","timeout_ms":10000,"last_server_duration_us":34612,"total_duration_us":10008611},{"operation_name":"upsert","last_dispatch_duration_us":2158607,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10553778,"total_server_duration_us":1586280,"encode_duration_us":25560,"operation_id":"0x1291","timeout_ms":10000,"last_server_duration_us":248658,"total_duration_us":10007401},{"operation_name":"upsert","last_dispatch_duration_us":11083283,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":15643323,"total_server_duration_us":9821788,"encode_duration_us":16482,"operation_id":"0x116a","timeout_ms":10000,"last_server_duration_us":9481299,"total_duration_us":10006514},{"operation_name":"upsert","last_dispatch_duration_us":2600689,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10034542,"total_server_duration_us":1235086,"encode_duration_us":22344,"operation_id":"0x1190","timeout_ms":10000,"last_server_duration_us":190657,"total_duration_us":10006486},{"operation_name":"upsert","last_dispatch_duration_us":9262305,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":14364574,"total_server_duration_us":9176338,"encode_duration_us":16756,"operation_id":"0x11b3","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10006443},{"operation_name":"upsert","last_dispatch_duration_us":3884714,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12230515,"total_server_duration_us":1311191,"encode_duration_us":16770,"operation_id":"0x1227","timeout_ms":10000,"last_server_duration_us":548944,"total_duration_us":10005315},{"operation_name":"upsert","last_dispatch_duration_us":6945829,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":6945829,"total_server_duration_us":48563,"encode_duration_us":21266,"operation_id":"0x1249","timeout_ms":10000,"last_server_duration_us":48563,"total_duration_us":10005005},{"operation_name":"upsert","last_dispatch_duration_us":3572180,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10649027,"total_server_duration_us":1719954,"encode_duration_us":18311,"operation_id":"0x11f0","timeout_ms":10000,"last_server_duration_us":368055,"total_duration_us":10004429}],"total_count":23}} WARN [2022-01-10 05:37:41,175] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":76605,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9589034,"total_server_duration_us":281577,"encode_duration_us":16952,"operation_id":"0x116f","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10009201},{"operation_name":"upsert","last_dispatch_duration_us":888295,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":7351152,"total_server_duration_us":794534,"encode_duration_us":16250,"operation_id":"0x1285","timeout_ms":10000,"last_server_duration_us":202956,"total_duration_us":10009004},{"operation_name":"upsert","last_dispatch_duration_us":787367,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":4264750,"total_server_duration_us":626124,"encode_duration_us":16637,"operation_id":"0x1290","timeout_ms":10000,"last_server_duration_us":34811,"total_duration_us":10008826},{"operation_name":"upsert","last_dispatch_duration_us":3656835,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8414810,"total_server_duration_us":1323179,"encode_duration_us":16799,"operation_id":"0x1146","timeout_ms":10000,"last_server_duration_us":376256,"total_duration_us":10008604},{"operation_name":"upsert","last_dispatch_duration_us":2988796,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10627279,"total_server_duration_us":892294,"encode_duration_us":18343,"operation_id":"0x1169","timeout_ms":10000,"last_server_duration_us":206773,"total_duration_us":10006703},{"operation_name":"upsert","last_dispatch_duration_us":926491,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10553800,"total_server_duration_us":720280,"encode_duration_us":16680,"operation_id":"0x11af","timeout_ms":10000,"last_server_duration_us":115283,"total_duration_us":10006423},{"operation_name":"upsert","last_dispatch_duration_us":3930943,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9029413,"total_server_duration_us":883701,"encode_duration_us":19483,"operation_id":"0x1289","timeout_ms":10000,"last_server_duration_us":24239,"total_duration_us":10005569},{"operation_name":"upsert","last_dispatch_duration_us":2938825,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9124947,"total_server_duration_us":2336228,"encode_duration_us":27198,"operation_id":"0x1242","timeout_ms":10000,"last_server_duration_us":1265093,"total_duration_us":10005209},{"operation_name":"upsert","last_dispatch_duration_us":30645,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8888956,"total_server_duration_us":280151,"encode_duration_us":20094,"operation_id":"0x118f","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10004973},{"operation_name":"upsert","last_dispatch_duration_us":1119537,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":5949211,"total_server_duration_us":1052824,"encode_duration_us":17017,"operation_id":"0x1268","timeout_ms":10000,"last_server_duration_us":228081,"total_duration_us":10003308}],"total_count":17}} WARN [2022-01-10 05:37:41,176] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9382575,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11612704,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9414281,"encode_duration_us":18502,"operation_id":"0x1161","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10002313}],"total_count":1}} WARN [2022-01-10 05:37:41,176] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14312236,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14312236,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9903335,"encode_duration_us":18061,"operation_id":"0x1124","timeout_ms":10000,"last_server_duration_us":9903335,"total_duration_us":10001097},{"operation_name":"upsert","last_dispatch_duration_us":9497117,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9984489,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9068361,"encode_duration_us":18485,"operation_id":"0x111f","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10003614},{"operation_name":"upsert","last_dispatch_duration_us":11393460,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14596411,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":11018235,"encode_duration_us":17277,"operation_id":"0x11ec","timeout_ms":10000,"last_server_duration_us":10228893,"total_duration_us":10005516},{"operation_name":"upsert","last_dispatch_duration_us":13252886,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":19749582,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9543745,"encode_duration_us":17205,"operation_id":"0x1002","timeout_ms":10000,"last_server_duration_us":9398047,"total_duration_us":10006860},{"operation_name":"upsert","last_dispatch_duration_us":9240500,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11660296,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9175934,"encode_duration_us":19038,"operation_id":"0x11d3","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10009083},{"operation_name":"upsert","last_dispatch_duration_us":9335884,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11839734,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9276082,"encode_duration_us":18970,"operation_id":"0x11ad","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10009166},{"operation_name":"upsert","last_dispatch_duration_us":9516133,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9516133,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9025695,"encode_duration_us":16622,"operation_id":"0x10d8","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10006985}],"total_count":7}} WARN [2022-01-10 05:37:41,177] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12511090,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":12511090,"total_server_duration_us":342334,"operation_id":"0x11a6","timeout_ms":30000,"last_server_duration_us":342334,"total_duration_us":12513287},{"operation_name":"upsert","last_dispatch_duration_us":12301655,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":15803742,"total_server_duration_us":631962,"encode_duration_us":22568,"operation_id":"0x116d","timeout_ms":10000,"last_server_duration_us":365068,"total_duration_us":10010047},{"operation_name":"upsert","last_dispatch_duration_us":9335884,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11839734,"total_server_duration_us":9276082,"encode_duration_us":18970,"operation_id":"0x11ad","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10009166},{"operation_name":"upsert","last_dispatch_duration_us":9240500,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11660296,"total_server_duration_us":9175934,"encode_duration_us":19038,"operation_id":"0x11d3","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10009083},{"operation_name":"upsert","last_dispatch_duration_us":8227920,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9781362,"total_server_duration_us":258204,"encode_duration_us":23450,"operation_id":"0x1162","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10008817},{"operation_name":"upsert","last_dispatch_duration_us":1299520,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":3589464,"total_server_duration_us":829560,"encode_duration_us":17764,"operation_id":"0x11b8","timeout_ms":10000,"last_server_duration_us":56524,"total_duration_us":10006544},{"operation_name":"upsert","last_dispatch_duration_us":1244394,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":1244394,"total_server_duration_us":155369,"encode_duration_us":16347,"operation_id":"0x1271","timeout_ms":10000,"last_server_duration_us":155369,"total_duration_us":10006270},{"operation_name":"upsert","last_dispatch_duration_us":44280,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9669264,"total_server_duration_us":331576,"encode_duration_us":16599,"operation_id":"0x124a","timeout_ms":10000,"last_server_duration_us":28860,"total_duration_us":10006063},{"operation_name":"upsert","last_dispatch_duration_us":11393460,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":14596411,"total_server_duration_us":11018235,"encode_duration_us":17277,"operation_id":"0x11ec","timeout_ms":10000,"last_server_duration_us":10228893,"total_duration_us":10005516},{"operation_name":"upsert","last_dispatch_duration_us":1063761,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":1063761,"total_server_duration_us":270941,"encode_duration_us":24456,"operation_id":"0x1266","timeout_ms":10000,"last_server_duration_us":270941,"total_duration_us":10005281}],"total_count":17}} INFO [2022-01-10 05:37:41,192] jepsen worker 15 - jepsen.util 375 :info :write [5 42] :RequestTimeoutException INFO [2022-01-10 05:37:41,193] jepsen worker 15 - jepsen.util 465 :invoke :read [5 nil] WARN [2022-01-10 05:37:41,276] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9262305,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14364574,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9176338,"encode_duration_us":16756,"operation_id":"0x11b3","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10006443},{"operation_name":"upsert","last_dispatch_duration_us":18240699,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19065742,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9380060,"encode_duration_us":22505,"operation_id":"0x1078","timeout_ms":10000,"last_server_duration_us":9261418,"total_duration_us":10009961},{"operation_name":"upsert","last_dispatch_duration_us":11083283,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15643323,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9821788,"encode_duration_us":16482,"operation_id":"0x116a","timeout_ms":10000,"last_server_duration_us":9481299,"total_duration_us":10006514}],"total_count":3}} INFO [2022-01-10 05:37:41,360] jepsen worker 39 - jepsen.util 489 :info :write [13 3] :RequestTimeoutException INFO [2022-01-10 05:37:41,360] jepsen worker 39 - jepsen.util 579 :invoke :write [13 27] WARN [2022-01-10 05:37:41,376] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7974486,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11250419,"total_server_duration_us":453954,"encode_duration_us":16530,"operation_id":"0x114b","timeout_ms":10000,"last_server_duration_us":20841,"total_duration_us":10009532},{"operation_name":"upsert","last_dispatch_duration_us":8088962,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8577229,"total_server_duration_us":349111,"encode_duration_us":16367,"operation_id":"0x118e","timeout_ms":10000,"last_server_duration_us":282021,"total_duration_us":10009522},{"operation_name":"upsert","last_dispatch_duration_us":683970,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":4783550,"total_server_duration_us":898948,"encode_duration_us":22802,"operation_id":"0x12b3","timeout_ms":10000,"last_server_duration_us":25800,"total_duration_us":10006387},{"operation_name":"upsert","last_dispatch_duration_us":7929466,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10044096,"total_server_duration_us":311224,"encode_duration_us":17482,"operation_id":"0x116e","timeout_ms":10000,"last_server_duration_us":41983,"total_duration_us":10004421},{"operation_name":"upsert","last_dispatch_duration_us":495158,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":3985820,"total_server_duration_us":747349,"encode_duration_us":18839,"operation_id":"0x1229","timeout_ms":10000,"last_server_duration_us":37653,"total_duration_us":10004256},{"operation_name":"upsert","last_dispatch_duration_us":9382575,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11612704,"total_server_duration_us":9414281,"encode_duration_us":18502,"operation_id":"0x1161","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10002313},{"operation_name":"upsert","last_dispatch_duration_us":359302,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":3670460,"total_server_duration_us":1049189,"encode_duration_us":17120,"operation_id":"0x12b5","timeout_ms":10000,"last_server_duration_us":116949,"total_duration_us":10001825},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":25686,"operation_id":"0x1267","timeout_ms":10000,"total_duration_us":10001758},{"operation_name":"get","last_dispatch_duration_us":7829511,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7829511,"total_server_duration_us":23,"operation_id":"0x11fc","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":7830049},{"operation_name":"get","last_dispatch_duration_us":294096,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":294096,"total_server_duration_us":32,"operation_id":"0x1248","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":5966350}],"total_count":16}} INFO [2022-01-10 05:37:41,552] jepsen worker 20 - jepsen.util 290 :info :write [6 7] :RequestTimeoutException INFO [2022-01-10 05:37:41,553] jepsen worker 20 - jepsen.util 380 :invoke :write [6 45] INFO [2022-01-10 05:37:41,743] jepsen worker 69 - jepsen.util 609 :ok :read [23 :nil] INFO [2022-01-10 05:37:41,744] jepsen worker 69 - jepsen.util 609 :invoke :write [23 49] INFO [2022-01-10 05:37:42,042] jepsen worker 19 - jepsen.util 559 :info :write [6 20] :RequestTimeoutException INFO [2022-01-10 05:37:42,043] jepsen worker 19 - jepsen.util 649 :invoke :write [6 46] INFO [2022-01-10 05:37:42,046] jepsen worker 65 - jepsen.util 605 :info :write [21 16] :RequestTimeoutException INFO [2022-01-10 05:37:42,046] jepsen worker 65 - jepsen.util 695 :invoke :write [21 38] INFO [2022-01-10 05:37:42,152] jepsen worker 36 - jepsen.util 216 :info :write [12 16] :RequestTimeoutException INFO [2022-01-10 05:37:42,152] jepsen worker 36 - jepsen.util 306 :invoke :read [12 nil] INFO [2022-01-10 05:37:42,161] jepsen worker 53 - jepsen.util 413 :info :write [17 48] :RequestTimeoutException INFO [2022-01-10 05:37:42,161] jepsen worker 53 - jepsen.util 503 :invoke :read [17 nil] INFO [2022-01-10 05:37:42,403] jepsen worker 89 - jepsen.util 809 :ok :read [29 33] INFO [2022-01-10 05:37:42,403] jepsen worker 89 - jepsen.util 809 :invoke :write [29 23] INFO [2022-01-10 05:37:42,462] jepsen worker 11 - jepsen.util 461 :info :write [3 42] :RequestTimeoutException INFO [2022-01-10 05:37:42,463] jepsen worker 11 - jepsen.util 551 :invoke :read [3 nil] INFO [2022-01-10 05:37:43,082] jepsen worker 28 - jepsen.util 658 :ok :read [9 22] INFO [2022-01-10 05:37:43,083] jepsen worker 28 - jepsen.util 658 :invoke :read [9 nil] INFO [2022-01-10 05:37:43,151] jepsen worker 28 - jepsen.util 658 :ok :read [9 22] INFO [2022-01-10 05:37:43,152] jepsen worker 28 - jepsen.util 658 :invoke :read [9 nil] INFO [2022-01-10 05:37:43,156] jepsen worker 83 - jepsen.util 623 :info :write [27 12] :RequestTimeoutException INFO [2022-01-10 05:37:43,157] jepsen worker 83 - jepsen.util 713 :invoke :write [27 48] INFO [2022-01-10 05:37:43,390] jepsen worker 43 - jepsen.util 583 :info :write [14 2] :RequestTimeoutException INFO [2022-01-10 05:37:43,390] jepsen worker 43 - jepsen.util 673 :invoke :read [14 nil] INFO [2022-01-10 05:37:43,517] jepsen worker 75 - jepsen.util 615 :ok :read [25 :nil] INFO [2022-01-10 05:37:43,518] jepsen worker 75 - jepsen.util 615 :invoke :read [25 nil] INFO [2022-01-10 05:37:43,547] jepsen worker 45 - jepsen.util 675 :ok :read [15 2] INFO [2022-01-10 05:37:43,547] jepsen worker 45 - jepsen.util 675 :invoke :write [15 11] INFO [2022-01-10 05:37:43,554] jepsen worker 72 - jepsen.util 792 :ok :read [24 18] INFO [2022-01-10 05:37:43,554] jepsen worker 72 - jepsen.util 792 :invoke :read [24 nil] INFO [2022-01-10 05:37:43,572] jepsen worker 76 - jepsen.util 526 :info :write [25 30] :SyncWriteAmbiguous INFO [2022-01-10 05:37:43,573] jepsen worker 8 - jepsen.util 548 :info :write [2 40] :RequestTimeoutException INFO [2022-01-10 05:37:43,573] jepsen worker 76 - jepsen.util 616 :invoke :write [25 35] INFO [2022-01-10 05:37:43,574] jepsen worker 8 - jepsen.util 638 :invoke :read [2 nil] INFO [2022-01-10 05:37:43,586] jepsen worker 75 - jepsen.util 615 :ok :read [25 :nil] INFO [2022-01-10 05:37:43,587] jepsen worker 75 - jepsen.util 615 :invoke :read [25 nil] INFO [2022-01-10 05:37:43,593] jepsen worker 21 - jepsen.util 651 :info :write [7 22] :RequestTimeoutException INFO [2022-01-10 05:37:43,593] jepsen worker 21 - jepsen.util 741 :invoke :read [7 nil] INFO [2022-01-10 05:37:43,598] jepsen worker 21 - jepsen.util 741 :ok :read [7 :nil] INFO [2022-01-10 05:37:43,598] jepsen worker 21 - jepsen.util 741 :invoke :read [7 nil] INFO [2022-01-10 05:37:43,599] jepsen worker 43 - jepsen.util 673 :ok :read [14 14] INFO [2022-01-10 05:37:43,599] jepsen worker 43 - jepsen.util 673 :invoke :read [14 nil] INFO [2022-01-10 05:37:43,600] jepsen worker 21 - jepsen.util 741 :ok :read [7 :nil] INFO [2022-01-10 05:37:43,600] jepsen worker 21 - jepsen.util 741 :invoke :read [7 nil] INFO [2022-01-10 05:37:43,601] jepsen worker 21 - jepsen.util 741 :ok :read [7 :nil] INFO [2022-01-10 05:37:43,602] jepsen worker 21 - jepsen.util 741 :invoke :write [7 42] INFO [2022-01-10 05:37:43,605] jepsen worker 72 - jepsen.util 792 :ok :read [24 18] INFO [2022-01-10 05:37:43,606] jepsen worker 72 - jepsen.util 792 :invoke :write [24 13] INFO [2022-01-10 05:37:43,634] jepsen worker 25 - jepsen.util 475 :info :write [8 21] :SyncWriteAmbiguous INFO [2022-01-10 05:37:43,635] jepsen worker 25 - jepsen.util 565 :invoke :read [8 nil] INFO [2022-01-10 05:37:43,656] jepsen worker 75 - jepsen.util 615 :ok :read [25 :nil] INFO [2022-01-10 05:37:43,657] jepsen worker 75 - jepsen.util 615 :invoke :write [25 19] INFO [2022-01-10 05:37:43,678] jepsen worker 43 - jepsen.util 673 :ok :read [14 14] INFO [2022-01-10 05:37:43,679] jepsen worker 43 - jepsen.util 673 :invoke :write [14 10] INFO [2022-01-10 05:37:43,695] jepsen worker 46 - jepsen.util 496 :info :write [15 40] :RequestTimeoutException INFO [2022-01-10 05:37:43,696] jepsen worker 46 - jepsen.util 586 :invoke :read [15 nil] INFO [2022-01-10 05:37:43,752] jepsen worker 31 - jepsen.util 661 :info :write [10 39] :RequestTimeoutException INFO [2022-01-10 05:37:43,752] jepsen worker 31 - jepsen.util 751 :invoke :write [10 11] INFO [2022-01-10 05:37:43,805] jepsen worker 71 - jepsen.util 521 :info :write [23 35] :RequestTimeoutException INFO [2022-01-10 05:37:43,806] jepsen worker 71 - jepsen.util 611 :invoke :read [23 nil] INFO [2022-01-10 05:37:43,910] jepsen worker 60 - jepsen.util 600 :info :write [20 43] :RequestTimeoutException INFO [2022-01-10 05:37:43,911] jepsen worker 60 - jepsen.util 690 :invoke :write [20 33] INFO [2022-01-10 05:37:43,925] jepsen worker 85 - jepsen.util 625 :ok :read [28 :nil] INFO [2022-01-10 05:37:43,927] jepsen worker 85 - jepsen.util 625 :invoke :read [28 nil] INFO [2022-01-10 05:37:43,939] jepsen worker 28 - jepsen.util 658 :ok :read [9 22] INFO [2022-01-10 05:37:43,940] jepsen worker 28 - jepsen.util 658 :invoke :write [9 46] INFO [2022-01-10 05:37:43,968] jepsen worker 16 - jepsen.util 556 :ok :read [5 42] INFO [2022-01-10 05:37:43,969] jepsen worker 16 - jepsen.util 556 :invoke :write [5 34] INFO [2022-01-10 05:37:43,987] jepsen worker 85 - jepsen.util 625 :ok :read [28 :nil] INFO [2022-01-10 05:37:43,988] jepsen worker 85 - jepsen.util 625 :invoke :read [28 nil] INFO [2022-01-10 05:37:44,001] jepsen worker 10 - jepsen.util 550 :ok :read [3 2] INFO [2022-01-10 05:37:44,002] jepsen worker 10 - jepsen.util 550 :invoke :read [3 nil] INFO [2022-01-10 05:37:44,018] jepsen worker 85 - jepsen.util 625 :ok :read [28 :nil] INFO [2022-01-10 05:37:44,019] jepsen worker 85 - jepsen.util 625 :invoke :write [28 16] INFO [2022-01-10 05:37:44,088] jepsen worker 71 - jepsen.util 611 :ok :read [23 :nil] INFO [2022-01-10 05:37:44,089] jepsen worker 71 - jepsen.util 611 :invoke :read [23 nil] INFO [2022-01-10 05:37:44,142] jepsen worker 27 - jepsen.util 477 :info :write [9 40] :RequestTimeoutException INFO [2022-01-10 05:37:44,143] jepsen worker 27 - jepsen.util 567 :invoke :write [9 26] INFO [2022-01-10 05:37:44,182] jepsen worker 52 - jepsen.util 592 :info :write [17 14] :RequestTimeoutException INFO [2022-01-10 05:37:44,182] jepsen worker 52 - jepsen.util 682 :invoke :read [17 nil] INFO [2022-01-10 05:37:44,222] jepsen worker 84 - jepsen.util 624 :info :write [28 1] :RequestTimeoutException INFO [2022-01-10 05:37:44,223] jepsen worker 84 - jepsen.util 714 :invoke :write [28 13] INFO [2022-01-10 05:37:44,272] jepsen worker 15 - jepsen.util 465 :ok :read [5 42] INFO [2022-01-10 05:37:44,273] jepsen worker 15 - jepsen.util 465 :invoke :read [5 nil] INFO [2022-01-10 05:37:44,276] jepsen worker 82 - jepsen.util 442 :ok :read [27 30] INFO [2022-01-10 05:37:44,277] jepsen worker 82 - jepsen.util 442 :invoke :write [27 3] INFO [2022-01-10 05:37:44,300] jepsen worker 23 - jepsen.util 473 :info :write [7 40] :RequestTimeoutException INFO [2022-01-10 05:37:44,301] jepsen worker 23 - jepsen.util 563 :invoke :write [7 5] INFO [2022-01-10 05:37:44,309] jepsen worker 62 - jepsen.util 692 :info :write [20 34] :RequestTimeoutException INFO [2022-01-10 05:37:44,310] jepsen worker 62 - jepsen.util 782 :invoke :read [20 nil] INFO [2022-01-10 05:37:44,411] jepsen worker 40 - jepsen.util 400 :info :write [13 46] :RequestTimeoutException INFO [2022-01-10 05:37:44,411] jepsen worker 40 - jepsen.util 490 :invoke :read [13 nil] INFO [2022-01-10 05:37:44,414] jepsen worker 15 - jepsen.util 465 :ok :read [5 42] INFO [2022-01-10 05:37:44,414] jepsen worker 15 - jepsen.util 465 :invoke :read [5 nil] INFO [2022-01-10 05:37:44,460] jepsen worker 73 - jepsen.util 523 :info :write [24 33] :RequestTimeoutException INFO [2022-01-10 05:37:44,461] jepsen worker 73 - jepsen.util 613 :invoke :write [24 43] INFO [2022-01-10 05:37:44,496] jepsen worker 4 - jepsen.util 454 :info :write [1 0] :RequestTimeoutException INFO [2022-01-10 05:37:44,497] jepsen worker 4 - jepsen.util 544 :invoke :read [1 nil] INFO [2022-01-10 05:37:44,522] jepsen worker 51 - jepsen.util 411 :info :write [17 26] :RequestTimeoutException INFO [2022-01-10 05:37:44,523] jepsen worker 51 - jepsen.util 501 :invoke :read [17 nil] INFO [2022-01-10 05:37:44,590] jepsen worker 15 - jepsen.util 465 :ok :read [5 42] INFO [2022-01-10 05:37:44,590] jepsen worker 15 - jepsen.util 465 :invoke :read [5 nil] INFO [2022-01-10 05:37:44,618] jepsen worker 71 - jepsen.util 611 :ok :read [23 :nil] INFO [2022-01-10 05:37:44,619] jepsen worker 71 - jepsen.util 611 :invoke :write [23 40] INFO [2022-01-10 05:37:44,650] jepsen worker 59 - jepsen.util 689 :info :write [19 3] :RequestTimeoutException INFO [2022-01-10 05:37:44,651] jepsen worker 59 - jepsen.util 779 :invoke :write [19 36] INFO [2022-01-10 05:37:44,715] jepsen worker 1 - jepsen.util 541 :info :write [0 2] :RequestTimeoutException INFO [2022-01-10 05:37:44,716] jepsen worker 1 - jepsen.util 631 :invoke :write [0 3] INFO [2022-01-10 05:37:44,730] jepsen worker 15 - jepsen.util 465 :ok :read [5 42] INFO [2022-01-10 05:37:44,731] jepsen worker 15 - jepsen.util 465 :invoke :read [5 nil] INFO [2022-01-10 05:37:44,806] jepsen worker 86 - jepsen.util 446 :ok :read [28 :nil] INFO [2022-01-10 05:37:44,807] jepsen worker 86 - jepsen.util 446 :invoke :write [28 46] INFO [2022-01-10 05:37:44,815] jepsen worker 35 - jepsen.util 665 :info :write [11 44] :RequestTimeoutException INFO [2022-01-10 05:37:44,816] jepsen worker 35 - jepsen.util 755 :invoke :write [11 44] INFO [2022-01-10 05:37:44,857] jepsen worker 50 - jepsen.util 680 :ok :read [16 :nil] INFO [2022-01-10 05:37:44,858] jepsen worker 50 - jepsen.util 680 :invoke :write [16 20] INFO [2022-01-10 05:37:44,871] jepsen worker 29 - jepsen.util 479 :ok :read [9 22] INFO [2022-01-10 05:37:44,871] jepsen worker 29 - jepsen.util 479 :invoke :write [9 0] INFO [2022-01-10 05:37:44,885] jepsen worker 15 - jepsen.util 465 :ok :read [5 42] INFO [2022-01-10 05:37:44,885] jepsen worker 15 - jepsen.util 465 :invoke :write [5 0] INFO [2022-01-10 05:37:44,992] jepsen worker 48 - jepsen.util 408 :info :write [16 1] :RequestTimeoutException INFO [2022-01-10 05:37:44,996] jepsen worker 48 - jepsen.util 498 :invoke :write [16 18] INFO [2022-01-10 05:37:45,046] jepsen worker 87 - jepsen.util 627 :info :write [29 44] :RequestTimeoutException INFO [2022-01-10 05:37:45,047] jepsen worker 87 - jepsen.util 717 :invoke :write [29 1] INFO [2022-01-10 05:37:45,155] jepsen worker 17 - jepsen.util 377 :info :write [5 46] :RequestTimeoutException INFO [2022-01-10 05:37:45,176] jepsen worker 17 - jepsen.util 467 :invoke :read [5 nil] INFO [2022-01-10 05:37:45,180] jepsen worker 42 - jepsen.util 492 :info :write [14 32] :RequestTimeoutException INFO [2022-01-10 05:37:45,182] jepsen worker 42 - jepsen.util 582 :invoke :read [14 nil] INFO [2022-01-10 05:37:45,233] jepsen worker 5 - jepsen.util 545 :info :write [1 12] :RequestTimeoutException INFO [2022-01-10 05:37:45,233] jepsen worker 5 - jepsen.util 635 :invoke :read [1 nil] INFO [2022-01-10 05:37:45,425] jepsen worker 3 - jepsen.util 633 :info :write [1 49] :RequestTimeoutException INFO [2022-01-10 05:37:45,426] jepsen worker 3 - jepsen.util 723 :invoke :read [1 nil] INFO [2022-01-10 05:37:45,485] jepsen worker 13 - jepsen.util 553 :ok :read [4 :nil] INFO [2022-01-10 05:37:45,486] jepsen worker 13 - jepsen.util 553 :invoke :read [4 nil] INFO [2022-01-10 05:37:45,576] jepsen worker 41 - jepsen.util 491 :ok :read [13 23] INFO [2022-01-10 05:37:45,577] jepsen worker 41 - jepsen.util 491 :invoke :read [13 nil] INFO [2022-01-10 05:37:45,590] jepsen worker 42 - jepsen.util 582 :ok :read [14 14] INFO [2022-01-10 05:37:45,591] jepsen worker 42 - jepsen.util 582 :invoke :write [14 41] INFO [2022-01-10 05:37:45,654] jepsen worker 40 - jepsen.util 490 :ok :read [13 23] INFO [2022-01-10 05:37:45,654] jepsen worker 40 - jepsen.util 490 :invoke :write [13 29] INFO [2022-01-10 05:37:45,754] jepsen worker 41 - jepsen.util 491 :ok :read [13 23] INFO [2022-01-10 05:37:45,755] jepsen worker 41 - jepsen.util 491 :invoke :read [13 nil] INFO [2022-01-10 05:37:45,801] jepsen worker 57 - jepsen.util 597 :info :write [19 48] :RequestTimeoutException INFO [2022-01-10 05:37:45,801] jepsen worker 38 - jepsen.util 578 :info :write [12 5] :RequestTimeoutException INFO [2022-01-10 05:37:45,801] jepsen worker 57 - jepsen.util 687 :invoke :write [19 9] INFO [2022-01-10 05:37:45,802] jepsen worker 38 - jepsen.util 668 :invoke :read [12 nil] INFO [2022-01-10 05:37:45,835] jepsen worker 9 - jepsen.util 639 :info :write [3 13] :RequestTimeoutException INFO [2022-01-10 05:37:45,836] jepsen worker 9 - jepsen.util 729 :invoke :read [3 nil] INFO [2022-01-10 05:37:45,896] jepsen worker 34 - jepsen.util 664 :info :write [11 49] :RequestTimeoutException INFO [2022-01-10 05:37:45,896] jepsen worker 34 - jepsen.util 754 :invoke :read [11 nil] INFO [2022-01-10 05:37:45,978] jepsen worker 41 - jepsen.util 491 :ok :read [13 23] INFO [2022-01-10 05:37:45,979] jepsen worker 41 - jepsen.util 491 :invoke :write [13 3] INFO [2022-01-10 05:37:46,000] jepsen worker 56 - jepsen.util 596 :info :write [18 6] :RequestTimeoutException INFO [2022-01-10 05:37:46,000] jepsen worker 56 - jepsen.util 686 :invoke :write [18 45] INFO [2022-01-10 05:37:46,011] jepsen worker 11 - jepsen.util 551 :ok :read [3 2] INFO [2022-01-10 05:37:46,012] jepsen worker 11 - jepsen.util 551 :invoke :read [3 nil] INFO [2022-01-10 05:37:46,076] jepsen worker 79 - jepsen.util 619 :ok :read [26 10] INFO [2022-01-10 05:37:46,077] jepsen worker 79 - jepsen.util 619 :invoke :write [26 5] INFO [2022-01-10 05:37:46,084] jepsen worker 51 - jepsen.util 501 :ok :read [17 41] INFO [2022-01-10 05:37:46,087] jepsen worker 51 - jepsen.util 501 :invoke :write [17 28] INFO [2022-01-10 05:37:46,099] jepsen worker 55 - jepsen.util 595 :ok :read [18 14] INFO [2022-01-10 05:37:46,100] jepsen worker 55 - jepsen.util 595 :invoke :write [18 44] INFO [2022-01-10 05:37:46,306] jepsen worker 9 - jepsen.util 729 :ok :read [3 2] INFO [2022-01-10 05:37:46,307] jepsen worker 9 - jepsen.util 729 :invoke :write [3 14] INFO [2022-01-10 05:37:46,352] jepsen worker 2 - jepsen.util 452 :info :write [0 22] :RequestTimeoutException INFO [2022-01-10 05:37:46,353] jepsen worker 2 - jepsen.util 542 :invoke :read [0 nil] INFO [2022-01-10 05:37:46,376] jepsen worker 78 - jepsen.util 528 :info :write [26 30] :RequestTimeoutException INFO [2022-01-10 05:37:46,377] jepsen worker 78 - jepsen.util 618 :invoke :read [26 nil] INFO [2022-01-10 05:37:46,432] jepsen worker 0 - jepsen.util 540 :ok :read [0 :nil] INFO [2022-01-10 05:37:46,433] jepsen worker 0 - jepsen.util 540 :invoke :read [0 nil] INFO [2022-01-10 05:37:46,448] jepsen worker 17 - jepsen.util 467 :ok :read [5 42] INFO [2022-01-10 05:37:46,449] jepsen worker 17 - jepsen.util 467 :invoke :write [5 8] INFO [2022-01-10 05:37:46,585] jepsen worker 34 - jepsen.util 754 :ok :read [11 21] INFO [2022-01-10 05:37:46,586] jepsen worker 34 - jepsen.util 754 :invoke :write [11 18] INFO [2022-01-10 05:37:46,700] jepsen worker 53 - jepsen.util 503 :ok :read [17 41] INFO [2022-01-10 05:37:46,701] jepsen worker 53 - jepsen.util 503 :invoke :read [17 nil] INFO [2022-01-10 05:37:46,902] jepsen worker 7 - jepsen.util 457 :info :write [2 20] :RequestTimeoutException INFO [2022-01-10 05:37:46,902] jepsen worker 7 - jepsen.util 547 :invoke :write [2 39] INFO [2022-01-10 05:37:46,981] jepsen worker 49 - jepsen.util 409 :info :write [16 39] :RequestTimeoutException INFO [2022-01-10 05:37:46,982] jepsen worker 49 - jepsen.util 499 :invoke :read [16 nil] INFO [2022-01-10 05:37:47,023] jepsen worker 2 - jepsen.util 542 :ok :read [0 :nil] INFO [2022-01-10 05:37:47,024] jepsen worker 2 - jepsen.util 542 :invoke :read [0 nil] INFO [2022-01-10 05:37:47,204] jepsen worker 10 - jepsen.util 550 :ok :read [3 2] INFO [2022-01-10 05:37:47,205] jepsen worker 10 - jepsen.util 550 :invoke :write [3 26] INFO [2022-01-10 05:37:47,241] jepsen worker 70 - jepsen.util 700 :info :write [23 14] :RequestTimeoutException INFO [2022-01-10 05:37:47,242] jepsen worker 70 - jepsen.util 790 :invoke :write [23 16] INFO [2022-01-10 05:37:47,385] jepsen worker 2 - jepsen.util 542 :ok :read [0 :nil] INFO [2022-01-10 05:37:47,386] jepsen worker 2 - jepsen.util 542 :invoke :write [0 10] INFO [2022-01-10 05:37:47,405] jepsen worker 32 - jepsen.util 572 :info :write [10 29] :RequestTimeoutException INFO [2022-01-10 05:37:47,405] jepsen worker 32 - jepsen.util 662 :invoke :read [10 nil] INFO [2022-01-10 05:37:47,406] jepsen worker 54 - jepsen.util 414 :info :write [18 1] :RequestTimeoutException INFO [2022-01-10 05:37:47,407] jepsen worker 54 - jepsen.util 504 :invoke :write [18 42] INFO [2022-01-10 05:37:47,470] jepsen worker 58 - jepsen.util 598 :info :write [19 16] :RequestTimeoutException INFO [2022-01-10 05:37:47,471] jepsen worker 58 - jepsen.util 688 :invoke :read [19 nil] INFO [2022-01-10 05:37:47,496] jepsen worker 66 - jepsen.util 426 :ok :read [22 41] INFO [2022-01-10 05:37:47,497] jepsen worker 66 - jepsen.util 426 :invoke :read [22 nil] INFO [2022-01-10 05:37:47,912] jepsen worker 88 - jepsen.util 628 :info :write [29 14] :RequestTimeoutException INFO [2022-01-10 05:37:47,913] jepsen worker 88 - jepsen.util 718 :invoke :write [29 26] INFO [2022-01-10 05:37:48,061] jepsen worker 14 - jepsen.util 554 :info :write [4 25] :RequestTimeoutException INFO [2022-01-10 05:37:48,062] jepsen worker 14 - jepsen.util 644 :invoke :read [4 nil] INFO [2022-01-10 05:37:48,136] jepsen worker 61 - jepsen.util 691 :info :write [20 11] :SyncWriteAmbiguous INFO [2022-01-10 05:37:48,137] jepsen worker 61 - jepsen.util 781 :invoke :read [20 nil] INFO [2022-01-10 05:37:48,144] jepsen worker 52 - jepsen.util 682 :ok :read [17 41] INFO [2022-01-10 05:37:48,145] jepsen worker 52 - jepsen.util 682 :invoke :read [17 nil] INFO [2022-01-10 05:37:48,176] jepsen worker 4 - jepsen.util 544 :ok :read [1 41] INFO [2022-01-10 05:37:48,177] jepsen worker 4 - jepsen.util 544 :invoke :write [1 0] INFO [2022-01-10 05:37:48,241] jepsen worker 0 - jepsen.util 540 :ok :read [0 :nil] INFO [2022-01-10 05:37:48,242] jepsen worker 0 - jepsen.util 540 :invoke :write [0 16] INFO [2022-01-10 05:37:48,305] jepsen worker 68 - jepsen.util 518 :info :write [22 45] :RequestTimeoutException INFO [2022-01-10 05:37:48,306] jepsen worker 68 - jepsen.util 608 :invoke :write [22 24] INFO [2022-01-10 05:37:48,322] jepsen worker 49 - jepsen.util 499 :ok :read [16 38] INFO [2022-01-10 05:37:48,322] jepsen worker 49 - jepsen.util 499 :invoke :read [16 nil] INFO [2022-01-10 05:37:48,396] jepsen worker 77 - jepsen.util 707 :info :write [25 43] :RequestTimeoutException INFO [2022-01-10 05:37:48,396] jepsen worker 77 - jepsen.util 797 :invoke :write [25 6] INFO [2022-01-10 05:37:48,403] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 05:37:48,403] jepsen worker nemesis - couchbase.util Rebalance check count: 26 INFO [2022-01-10 05:37:48,403] jepsen worker nemesis - couchbase.util Rebalance stuck count: 25 INFO [2022-01-10 05:37:48,477] jepsen worker 66 - jepsen.util 426 :ok :read [22 41] INFO [2022-01-10 05:37:48,477] jepsen worker 66 - jepsen.util 426 :invoke :read [22 nil] INFO [2022-01-10 05:37:48,492] jepsen worker 49 - jepsen.util 499 :ok :read [16 38] INFO [2022-01-10 05:37:48,492] jepsen worker 49 - jepsen.util 499 :invoke :write [16 49] INFO [2022-01-10 05:37:48,510] jepsen worker 25 - jepsen.util 565 :ok :read [8 :nil] INFO [2022-01-10 05:37:48,511] jepsen worker 25 - jepsen.util 565 :invoke :write [8 49] INFO [2022-01-10 05:37:48,770] jepsen worker 47 - jepsen.util 677 :info :write [15 7] :RequestTimeoutException INFO [2022-01-10 05:37:48,770] jepsen worker 47 - jepsen.util 767 :invoke :write [15 5] INFO [2022-01-10 05:37:48,862] jepsen worker 74 - jepsen.util 704 :info :write [24 47] :SyncWriteAmbiguous INFO [2022-01-10 05:37:48,862] jepsen worker 74 - jepsen.util 794 :invoke :write [24 9] INFO [2022-01-10 05:37:49,001] jepsen worker 67 - jepsen.util 517 :info :write [22 23] :RequestTimeoutException INFO [2022-01-10 05:37:49,002] jepsen worker 67 - jepsen.util 607 :invoke :write [22 3] INFO [2022-01-10 05:37:49,040] jepsen worker 24 - jepsen.util 564 :info :write [8 18] :RequestTimeoutException INFO [2022-01-10 05:37:49,040] jepsen worker 24 - jepsen.util 654 :invoke :read [8 nil] INFO [2022-01-10 05:37:49,063] jepsen worker 62 - jepsen.util 782 :ok :read [20 :nil] INFO [2022-01-10 05:37:49,064] jepsen worker 62 - jepsen.util 782 :invoke :read [20 nil] INFO [2022-01-10 05:37:49,071] jepsen worker 62 - jepsen.util 782 :ok :read [20 :nil] INFO [2022-01-10 05:37:49,072] jepsen worker 62 - jepsen.util 782 :invoke :write [20 14] INFO [2022-01-10 05:37:49,136] jepsen worker 30 - jepsen.util 390 :info :write [10 5] :RequestTimeoutException INFO [2022-01-10 05:37:49,137] jepsen worker 30 - jepsen.util 480 :invoke :read [10 nil] INFO [2022-01-10 05:37:49,221] jepsen worker 78 - jepsen.util 618 :ok :read [26 10] INFO [2022-01-10 05:37:49,222] jepsen worker 78 - jepsen.util 618 :invoke :read [26 nil] INFO [2022-01-10 05:37:49,279] jepsen worker 36 - jepsen.util 306 :ok :read [12 :nil] INFO [2022-01-10 05:37:49,280] jepsen worker 36 - jepsen.util 306 :invoke :write [12 44] INFO [2022-01-10 05:37:49,293] jepsen worker 3 - jepsen.util 723 :ok :read [1 41] INFO [2022-01-10 05:37:49,294] jepsen worker 3 - jepsen.util 723 :invoke :read [1 nil] INFO [2022-01-10 05:37:49,346] jepsen worker 63 - jepsen.util 603 :info :write [21 26] :RequestTimeoutException INFO [2022-01-10 05:37:49,347] jepsen worker 63 - jepsen.util 693 :invoke :read [21 nil] INFO [2022-01-10 05:37:49,386] jepsen worker 33 - jepsen.util 483 :info :write [11 23] :RequestTimeoutException INFO [2022-01-10 05:37:49,387] jepsen worker 33 - jepsen.util 573 :invoke :write [11 2] INFO [2022-01-10 05:37:49,410] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0.00588235294117645}} INFO [2022-01-10 05:37:49,410] jepsen worker nemesis - couchbase.util Rebalance check count: 27 INFO [2022-01-10 05:37:49,410] jepsen worker nemesis - couchbase.util Rebalance stuck count: 25 INFO [2022-01-10 05:37:49,417] jepsen worker 44 - jepsen.util 584 :info :write [14 26] :RequestTimeoutException INFO [2022-01-10 05:37:49,418] jepsen worker 44 - jepsen.util 674 :invoke :write [14 15] INFO [2022-01-10 05:37:49,462] jepsen worker 18 - jepsen.util 558 :info :write [6 25] :RequestTimeoutException INFO [2022-01-10 05:37:49,462] jepsen worker 18 - jepsen.util 648 :invoke :read [6 nil] WARN [2022-01-10 05:37:49,504] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":13733291,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13733291,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9291468,"encode_duration_us":23287,"operation_id":"0x1298","timeout_ms":10000,"last_server_duration_us":9291468,"total_duration_us":10007139}],"total_count":1}} WARN [2022-01-10 05:37:49,605] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7253682,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8757157,"total_server_duration_us":433141,"encode_duration_us":19942,"operation_id":"0x1297","timeout_ms":10000,"last_server_duration_us":39635,"total_duration_us":10010620},{"operation_name":"upsert","last_dispatch_duration_us":7973793,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":7973793,"total_server_duration_us":436960,"encode_duration_us":16673,"operation_id":"0x13c2","timeout_ms":10000,"last_server_duration_us":436960,"total_duration_us":10010206},{"operation_name":"upsert","last_dispatch_duration_us":3338293,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":11491951,"total_server_duration_us":126523,"encode_duration_us":18104,"operation_id":"0x139b","timeout_ms":10000,"last_server_duration_us":104546,"total_duration_us":10009098},{"operation_name":"upsert","last_dispatch_duration_us":170686,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9889661,"total_server_duration_us":172136,"encode_duration_us":17136,"operation_id":"0x1379","timeout_ms":10000,"last_server_duration_us":30823,"total_duration_us":10008989},{"operation_name":"upsert","last_dispatch_duration_us":4384776,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":12253895,"total_server_duration_us":147422,"encode_duration_us":16904,"operation_id":"0x1382","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10008148},{"operation_name":"upsert","last_dispatch_duration_us":5002679,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":12085072,"total_server_duration_us":213637,"encode_duration_us":21777,"operation_id":"0x1353","timeout_ms":10000,"last_server_duration_us":194375,"total_duration_us":10008117},{"operation_name":"upsert","last_dispatch_duration_us":13733291,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13733291,"total_server_duration_us":9291468,"encode_duration_us":23287,"operation_id":"0x1298","timeout_ms":10000,"last_server_duration_us":9291468,"total_duration_us":10007139},{"operation_name":"upsert","last_dispatch_duration_us":7115443,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":11144668,"total_server_duration_us":1502402,"encode_duration_us":17043,"operation_id":"0x132c","timeout_ms":10000,"last_server_duration_us":30069,"total_duration_us":10005691},{"operation_name":"upsert","last_dispatch_duration_us":859985,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8478143,"total_server_duration_us":540525,"encode_duration_us":21218,"operation_id":"0x13e7","timeout_ms":10000,"last_server_duration_us":162411,"total_duration_us":10005432},{"operation_name":"upsert","last_dispatch_duration_us":120421,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10005062,"total_server_duration_us":103015,"encode_duration_us":24401,"operation_id":"0x135b","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10002109}],"total_count":18}} INFO [2022-01-10 05:37:49,644] jepsen worker 66 - jepsen.util 426 :ok :read [22 41] INFO [2022-01-10 05:37:49,644] jepsen worker 66 - jepsen.util 426 :invoke :read [22 nil] INFO [2022-01-10 05:37:49,686] jepsen worker 66 - jepsen.util 426 :ok :read [22 41] INFO [2022-01-10 05:37:49,687] jepsen worker 66 - jepsen.util 426 :invoke :write [22 2] INFO [2022-01-10 05:37:49,791] jepsen worker 22 - jepsen.util 562 :info :write [7 13] :RequestTimeoutException INFO [2022-01-10 05:37:49,792] jepsen worker 22 - jepsen.util 652 :invoke :read [7 nil] INFO [2022-01-10 05:37:49,897] jepsen worker 37 - jepsen.util 667 :info :write [12 23] :RequestTimeoutException INFO [2022-01-10 05:37:49,897] jepsen worker 37 - jepsen.util 757 :invoke :read [12 nil] INFO [2022-01-10 05:37:49,946] jepsen worker 26 - jepsen.util 656 :info :write [8 3] :RequestTimeoutException INFO [2022-01-10 05:37:49,947] jepsen worker 26 - jepsen.util 746 :invoke :read [8 nil] INFO [2022-01-10 05:37:50,092] jepsen worker 80 - jepsen.util 620 :info :write [26 1] :RequestTimeoutException INFO [2022-01-10 05:37:50,092] jepsen worker 80 - jepsen.util 710 :invoke :read [26 nil] INFO [2022-01-10 05:37:50,296] jepsen worker 53 - jepsen.util 503 :ok :read [17 41] INFO [2022-01-10 05:37:50,297] jepsen worker 53 - jepsen.util 503 :invoke :write [17 22] INFO [2022-01-10 05:37:50,310] jepsen worker 46 - jepsen.util 586 :ok :read [15 2] INFO [2022-01-10 05:37:50,311] jepsen worker 46 - jepsen.util 586 :invoke :write [15 9] INFO [2022-01-10 05:37:50,417] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0.00588235294117645}} INFO [2022-01-10 05:37:50,417] jepsen worker nemesis - couchbase.util Rebalance check count: 28 INFO [2022-01-10 05:37:50,417] jepsen worker nemesis - couchbase.util Rebalance stuck count: 26 INFO [2022-01-10 05:37:50,493] jepsen worker 14 - jepsen.util 644 :ok :read [4 :nil] INFO [2022-01-10 05:37:50,494] jepsen worker 14 - jepsen.util 644 :invoke :read [4 nil] INFO [2022-01-10 05:37:50,499] jepsen worker 8 - jepsen.util 638 :ok :read [2 21] INFO [2022-01-10 05:37:50,500] jepsen worker 8 - jepsen.util 638 :invoke :write [2 11] INFO [2022-01-10 05:37:50,535] jepsen worker 61 - jepsen.util 781 :ok :read [20 :nil] INFO [2022-01-10 05:37:50,536] jepsen worker 61 - jepsen.util 781 :invoke :write [20 13] INFO [2022-01-10 05:37:50,565] jepsen worker 13 - jepsen.util 553 :ok :read [4 :nil] INFO [2022-01-10 05:37:50,566] jepsen worker 13 - jepsen.util 553 :invoke :write [4 33] INFO [2022-01-10 05:37:50,577] jepsen worker 12 - jepsen.util 552 :info :write [4 29] :RequestTimeoutException INFO [2022-01-10 05:37:50,578] jepsen worker 12 - jepsen.util 642 :invoke :write [4 45] INFO [2022-01-10 05:37:50,580] jepsen worker 64 - jepsen.util 604 :info :write [21 4] :RequestTimeoutException INFO [2022-01-10 05:37:50,581] jepsen worker 64 - jepsen.util 694 :invoke :read [21 nil] INFO [2022-01-10 05:37:50,611] jepsen worker 63 - jepsen.util 693 :ok :read [21 7] INFO [2022-01-10 05:37:50,612] jepsen worker 63 - jepsen.util 693 :invoke :write [21 15] INFO [2022-01-10 05:37:50,855] jepsen worker 6 - jepsen.util 546 :info :write [2 19] :RequestTimeoutException INFO [2022-01-10 05:37:50,856] jepsen worker 6 - jepsen.util 636 :invoke :read [2 nil] INFO [2022-01-10 05:37:50,937] jepsen worker 81 - jepsen.util 621 :info :write [27 9] :RequestTimeoutException INFO [2022-01-10 05:37:50,938] jepsen worker 81 - jepsen.util 711 :invoke :write [27 22] WARN [2022-01-10 05:37:51,090] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9635536,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9635536,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9532268,"encode_duration_us":17649,"operation_id":"0x139d","timeout_ms":10000,"last_server_duration_us":9532268,"total_duration_us":10009943}],"total_count":1}} WARN [2022-01-10 05:37:51,090] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12463225,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12463225,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9750458,"encode_duration_us":25686,"operation_id":"0x1267","timeout_ms":10000,"last_server_duration_us":9750458,"total_duration_us":10001758},{"operation_name":"upsert","last_dispatch_duration_us":10194437,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13864898,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":10229252,"encode_duration_us":17120,"operation_id":"0x12b5","timeout_ms":10000,"last_server_duration_us":9180063,"total_duration_us":10001825},{"operation_name":"upsert","last_dispatch_duration_us":9311506,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9976766,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9411520,"encode_duration_us":17086,"operation_id":"0x1314","timeout_ms":10000,"last_server_duration_us":9300061,"total_duration_us":10006026},{"operation_name":"upsert","last_dispatch_duration_us":9241602,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10643803,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9512773,"encode_duration_us":20418,"operation_id":"0x134d","timeout_ms":10000,"last_server_duration_us":9229268,"total_duration_us":10005596}],"total_count":4}} WARN [2022-01-10 05:37:51,098] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10415259,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16987857,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9300374,"encode_duration_us":16803,"operation_id":"0x1240","timeout_ms":10000,"last_server_duration_us":9199304,"total_duration_us":10003447},{"operation_name":"upsert","last_dispatch_duration_us":10715359,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11326738,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9905850,"encode_duration_us":19435,"operation_id":"0x13c0","timeout_ms":10000,"last_server_duration_us":9692433,"total_duration_us":10004107},{"operation_name":"upsert","last_dispatch_duration_us":9439860,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9439860,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9228197,"encode_duration_us":16366,"operation_id":"0x13bb","timeout_ms":10000,"last_server_duration_us":9228197,"total_duration_us":10006051}],"total_count":3}} WARN [2022-01-10 05:37:51,098] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10535513,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10867379,"total_server_duration_us":278225,"encode_duration_us":16465,"operation_id":"0x1424","timeout_ms":10000,"last_server_duration_us":23222,"total_duration_us":10010878},{"operation_name":"upsert","last_dispatch_duration_us":8163708,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10811970,"total_server_duration_us":215081,"encode_duration_us":18504,"operation_id":"0x1351","timeout_ms":10000,"last_server_duration_us":71773,"total_duration_us":10009628},{"operation_name":"upsert","last_dispatch_duration_us":971624,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":3210078,"total_server_duration_us":502192,"encode_duration_us":17994,"operation_id":"0x12ea","timeout_ms":10000,"last_server_duration_us":186156,"total_duration_us":10009596},{"operation_name":"upsert","last_dispatch_duration_us":651490,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10044343,"total_server_duration_us":503733,"encode_duration_us":17022,"operation_id":"0x13bc","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10009446},{"operation_name":"upsert","last_dispatch_duration_us":4038820,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":4783537,"total_server_duration_us":364472,"encode_duration_us":17439,"operation_id":"0x13e3","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10009233},{"operation_name":"upsert","last_dispatch_duration_us":8107048,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10815944,"total_server_duration_us":316557,"encode_duration_us":29574,"operation_id":"0x1348","timeout_ms":10000,"last_server_duration_us":49024,"total_duration_us":10008970},{"operation_name":"upsert","last_dispatch_duration_us":1471249,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":2471700,"total_server_duration_us":204680,"encode_duration_us":24609,"operation_id":"0x1410","timeout_ms":10000,"last_server_duration_us":22554,"total_duration_us":10007731},{"operation_name":"upsert","last_dispatch_duration_us":10601655,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10654474,"total_server_duration_us":44413,"encode_duration_us":24123,"operation_id":"0x1437","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10006083},{"operation_name":"upsert","last_dispatch_duration_us":9439860,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9439860,"total_server_duration_us":9228197,"encode_duration_us":16366,"operation_id":"0x13bb","timeout_ms":10000,"last_server_duration_us":9228197,"total_duration_us":10006051},{"operation_name":"upsert","last_dispatch_duration_us":1650152,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":5244302,"total_server_duration_us":2509694,"encode_duration_us":28077,"operation_id":"0x12e9","timeout_ms":10000,"last_server_duration_us":813757,"total_duration_us":10006046}],"total_count":25}} WARN [2022-01-10 05:37:51,191] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":761384,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":761384,"total_server_duration_us":135936,"operation_id":"0x12f3","timeout_ms":30000,"last_server_duration_us":135936,"total_duration_us":11536423},{"operation_name":"upsert","last_dispatch_duration_us":9635536,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9635536,"total_server_duration_us":9532268,"encode_duration_us":17649,"operation_id":"0x139d","timeout_ms":10000,"last_server_duration_us":9532268,"total_duration_us":10009943},{"operation_name":"upsert","last_dispatch_duration_us":4967751,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9591438,"total_server_duration_us":534202,"encode_duration_us":25682,"operation_id":"0x13a4","timeout_ms":10000,"last_server_duration_us":256539,"total_duration_us":10007736},{"operation_name":"upsert","last_dispatch_duration_us":3482330,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":7749720,"total_server_duration_us":172341,"encode_duration_us":17475,"operation_id":"0x1415","timeout_ms":10000,"last_server_duration_us":101381,"total_duration_us":10006907},{"operation_name":"upsert","last_dispatch_duration_us":2371289,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11808028,"total_server_duration_us":291449,"encode_duration_us":20559,"operation_id":"0x13ea","timeout_ms":10000,"last_server_duration_us":22471,"total_duration_us":10006745},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":19144,"operation_id":"0x1447","timeout_ms":10000,"total_duration_us":10006521},{"operation_name":"upsert","last_dispatch_duration_us":4741530,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":4741530,"total_server_duration_us":41767,"encode_duration_us":26179,"operation_id":"0x1445","timeout_ms":10000,"last_server_duration_us":41767,"total_duration_us":10005305},{"operation_name":"upsert","last_dispatch_duration_us":9053541,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9096870,"total_server_duration_us":54347,"encode_duration_us":17506,"operation_id":"0x136c","timeout_ms":10000,"last_server_duration_us":33426,"total_duration_us":10004018},{"operation_name":"upsert","last_dispatch_duration_us":71237,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":71237,"total_server_duration_us":20045,"encode_duration_us":26110,"operation_id":"0x1313","timeout_ms":10000,"last_server_duration_us":20045,"total_duration_us":10001821},{"operation_name":"upsert","last_dispatch_duration_us":3395898,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8703334,"total_server_duration_us":263040,"encode_duration_us":16337,"operation_id":"0x13c3","timeout_ms":10000,"last_server_duration_us":69080,"total_duration_us":10001039}],"total_count":19}} WARN [2022-01-10 05:37:51,194] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":3949511,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":3949511,"total_server_duration_us":19887,"encode_duration_us":16276,"operation_id":"0x146f","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10008807},{"operation_name":"upsert","last_dispatch_duration_us":7092701,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8451095,"total_server_duration_us":382404,"encode_duration_us":18287,"operation_id":"0x1420","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10008448},{"operation_name":"upsert","last_dispatch_duration_us":624915,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9528902,"total_server_duration_us":2307291,"encode_duration_us":28032,"operation_id":"0x1370","timeout_ms":10000,"last_server_duration_us":146801,"total_duration_us":10007852},{"operation_name":"upsert","last_dispatch_duration_us":33210,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1873176,"total_server_duration_us":306392,"encode_duration_us":17140,"operation_id":"0x1422","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10005181},{"operation_name":"upsert","last_dispatch_duration_us":7024337,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9009412,"total_server_duration_us":322356,"encode_duration_us":21268,"operation_id":"0x1423","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10004814},{"operation_name":"upsert","last_dispatch_duration_us":7072268,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8153437,"total_server_duration_us":110650,"encode_duration_us":18114,"operation_id":"0x1318","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10002911},{"operation_name":"upsert","last_dispatch_duration_us":570289,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":4549187,"total_server_duration_us":1793503,"encode_duration_us":17426,"operation_id":"0x1450","timeout_ms":10000,"last_server_duration_us":402727,"total_duration_us":10002506},{"operation_name":"upsert","last_dispatch_duration_us":30806,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8531655,"total_server_duration_us":519626,"encode_duration_us":16599,"operation_id":"0x12e5","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10000716},{"operation_name":"upsert","last_dispatch_duration_us":9430008,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9677305,"total_server_duration_us":9136067,"encode_duration_us":20578,"operation_id":"0x1419","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":9726396},{"operation_name":"get","last_dispatch_duration_us":9722681,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9722681,"total_server_duration_us":7231,"operation_id":"0x1371","timeout_ms":30000,"last_server_duration_us":7231,"total_duration_us":9723112}],"total_count":17}} WARN [2022-01-10 05:37:51,195] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7950196,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":16136261,"total_server_duration_us":607934,"encode_duration_us":20416,"operation_id":"0x1355","timeout_ms":10000,"last_server_duration_us":92286,"total_duration_us":10010567},{"operation_name":"upsert","last_dispatch_duration_us":6551291,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7275529,"total_server_duration_us":543936,"encode_duration_us":16227,"operation_id":"0x1316","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10010412},{"operation_name":"upsert","last_dispatch_duration_us":9672714,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":12607476,"total_server_duration_us":143691,"encode_duration_us":19339,"operation_id":"0x133b","timeout_ms":10000,"last_server_duration_us":46513,"total_duration_us":10009180},{"operation_name":"upsert","last_dispatch_duration_us":14483491,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":14483491,"total_server_duration_us":9151235,"encode_duration_us":17997,"operation_id":"0x137b","timeout_ms":10000,"last_server_duration_us":9151235,"total_duration_us":10008511},{"operation_name":"upsert","last_dispatch_duration_us":8181812,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":16791780,"total_server_duration_us":1328626,"encode_duration_us":17405,"operation_id":"0x1328","timeout_ms":10000,"last_server_duration_us":132923,"total_duration_us":10007990},{"operation_name":"upsert","last_dispatch_duration_us":8151510,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":15845825,"total_server_duration_us":500865,"encode_duration_us":16665,"operation_id":"0x12f5","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10007524},{"operation_name":"upsert","last_dispatch_duration_us":10026063,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10886494,"total_server_duration_us":129442,"encode_duration_us":16862,"operation_id":"0x13a2","timeout_ms":10000,"last_server_duration_us":42525,"total_duration_us":10006916},{"operation_name":"upsert","last_dispatch_duration_us":9765354,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11584830,"total_server_duration_us":133190,"encode_duration_us":18263,"operation_id":"0x136e","timeout_ms":10000,"last_server_duration_us":58128,"total_duration_us":10006090},{"operation_name":"upsert","last_dispatch_duration_us":9052031,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10239893,"total_server_duration_us":9279884,"encode_duration_us":20843,"operation_id":"0x12c6","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10005310},{"operation_name":"upsert","last_dispatch_duration_us":6639824,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7749384,"total_server_duration_us":359980,"encode_duration_us":16552,"operation_id":"0x12e4","timeout_ms":10000,"last_server_duration_us":22554,"total_duration_us":10004212}],"total_count":20}} WARN [2022-01-10 05:37:51,294] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":15578497,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18455977,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9204122,"encode_duration_us":16603,"operation_id":"0x1205","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10002179}],"total_count":1}} WARN [2022-01-10 05:37:51,295] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9997235,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15034228,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9777775,"encode_duration_us":16812,"operation_id":"0x12c2","timeout_ms":10000,"last_server_duration_us":9199304,"total_duration_us":10001528},{"operation_name":"upsert","last_dispatch_duration_us":14483491,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14483491,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9151235,"encode_duration_us":17997,"operation_id":"0x137b","timeout_ms":10000,"last_server_duration_us":9151235,"total_duration_us":10008511},{"operation_name":"upsert","last_dispatch_duration_us":9052031,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10239893,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9279884,"encode_duration_us":20843,"operation_id":"0x12c6","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10005310},{"operation_name":"upsert","last_dispatch_duration_us":11002846,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15267597,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9725138,"encode_duration_us":16637,"operation_id":"0x1290","timeout_ms":10000,"last_server_duration_us":9099014,"total_duration_us":10008826}],"total_count":4}} INFO [2022-01-10 05:37:51,387] jepsen worker 39 - jepsen.util 579 :info :write [13 27] :RequestTimeoutException INFO [2022-01-10 05:37:51,387] jepsen worker 39 - jepsen.util 669 :invoke :write [13 9] WARN [2022-01-10 05:37:51,391] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2459237,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10762609,"total_server_duration_us":121437,"encode_duration_us":16991,"operation_id":"0x13c4","timeout_ms":10000,"last_server_duration_us":20841,"total_duration_us":10011125},{"operation_name":"upsert","last_dispatch_duration_us":907431,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10148912,"total_server_duration_us":358748,"encode_duration_us":16929,"operation_id":"0x139e","timeout_ms":10000,"last_server_duration_us":57509,"total_duration_us":10009377},{"operation_name":"upsert","last_dispatch_duration_us":1816381,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11268593,"total_server_duration_us":409161,"encode_duration_us":16308,"operation_id":"0x1413","timeout_ms":10000,"last_server_duration_us":28310,"total_duration_us":10007281},{"operation_name":"upsert","last_dispatch_duration_us":9357957,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13611707,"total_server_duration_us":1208023,"encode_duration_us":21772,"operation_id":"0x12c4","timeout_ms":10000,"last_server_duration_us":50068,"total_duration_us":10007001},{"operation_name":"upsert","last_dispatch_duration_us":8845130,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9857515,"total_server_duration_us":335924,"encode_duration_us":19243,"operation_id":"0x137a","timeout_ms":10000,"last_server_duration_us":28768,"total_duration_us":10006673},{"operation_name":"upsert","last_dispatch_duration_us":9311506,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9976766,"total_server_duration_us":9411520,"encode_duration_us":17086,"operation_id":"0x1314","timeout_ms":10000,"last_server_duration_us":9300061,"total_duration_us":10006026},{"operation_name":"upsert","last_dispatch_duration_us":9241602,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10643803,"total_server_duration_us":9512773,"encode_duration_us":20418,"operation_id":"0x134d","timeout_ms":10000,"last_server_duration_us":9229268,"total_duration_us":10005596},{"operation_name":"upsert","last_dispatch_duration_us":9501886,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10645975,"total_server_duration_us":102081,"encode_duration_us":16256,"operation_id":"0x1357","timeout_ms":10000,"last_server_duration_us":82115,"total_duration_us":10005574},{"operation_name":"upsert","last_dispatch_duration_us":1976634,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":1976634,"total_server_duration_us":24755,"encode_duration_us":20127,"operation_id":"0x1451","timeout_ms":10000,"last_server_duration_us":24755,"total_duration_us":10004747},{"operation_name":"upsert","last_dispatch_duration_us":898094,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10205369,"total_server_duration_us":79231,"encode_duration_us":17341,"operation_id":"0x13fe","timeout_ms":10000,"last_server_duration_us":39846,"total_duration_us":10003929}],"total_count":22}} INFO [2022-01-10 05:37:51,427] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0.00588235294117645}} INFO [2022-01-10 05:37:51,427] jepsen worker nemesis - couchbase.util Rebalance check count: 29 INFO [2022-01-10 05:37:51,428] jepsen worker nemesis - couchbase.util Rebalance stuck count: 27 INFO [2022-01-10 05:37:51,551] jepsen worker 26 - jepsen.util 746 :ok :read [8 :nil] INFO [2022-01-10 05:37:51,552] jepsen worker 26 - jepsen.util 746 :invoke :read [8 nil] INFO [2022-01-10 05:37:51,580] jepsen worker 20 - jepsen.util 380 :info :write [6 45] :RequestTimeoutException INFO [2022-01-10 05:37:51,581] jepsen worker 20 - jepsen.util 470 :invoke :write [6 17] INFO [2022-01-10 05:37:51,765] jepsen worker 69 - jepsen.util 609 :info :write [23 49] :RequestTimeoutException INFO [2022-01-10 05:37:51,766] jepsen worker 69 - jepsen.util 699 :invoke :write [23 42] INFO [2022-01-10 05:37:51,785] jepsen worker 26 - jepsen.util 746 :ok :read [8 :nil] INFO [2022-01-10 05:37:51,786] jepsen worker 26 - jepsen.util 746 :invoke :read [8 nil] INFO [2022-01-10 05:37:52,062] jepsen worker 19 - jepsen.util 649 :info :write [6 46] :RequestTimeoutException INFO [2022-01-10 05:37:52,063] jepsen worker 19 - jepsen.util 739 :invoke :write [6 1] INFO [2022-01-10 05:37:52,066] jepsen worker 65 - jepsen.util 695 :info :write [21 38] :RequestTimeoutException INFO [2022-01-10 05:37:52,067] jepsen worker 65 - jepsen.util 785 :invoke :read [21 nil] INFO [2022-01-10 05:37:52,329] jepsen worker 14 - jepsen.util 644 :ok :read [4 :nil] INFO [2022-01-10 05:37:52,330] jepsen worker 14 - jepsen.util 644 :invoke :write [4 7] INFO [2022-01-10 05:37:52,427] jepsen worker 89 - jepsen.util 809 :info :write [29 23] :RequestTimeoutException INFO [2022-01-10 05:37:52,428] jepsen worker 89 - jepsen.util 899 :invoke :read [29 nil] INFO [2022-01-10 05:37:52,434] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0.00588235294117645}} INFO [2022-01-10 05:37:52,434] jepsen worker nemesis - couchbase.util Rebalance check count: 30 INFO [2022-01-10 05:37:52,434] jepsen worker nemesis - couchbase.util Rebalance stuck count: 28 INFO [2022-01-10 05:37:52,474] jepsen worker 3 - jepsen.util 723 :ok :read [1 41] INFO [2022-01-10 05:37:52,475] jepsen worker 3 - jepsen.util 723 :invoke :read [1 nil] INFO [2022-01-10 05:37:52,843] jepsen worker 3 - jepsen.util 723 :ok :read [1 41] INFO [2022-01-10 05:37:52,843] jepsen worker 3 - jepsen.util 723 :invoke :read [1 nil] INFO [2022-01-10 05:37:53,125] jepsen worker 26 - jepsen.util 746 :ok :read [8 :nil] INFO [2022-01-10 05:37:53,125] jepsen worker 26 - jepsen.util 746 :invoke :write [8 10] INFO [2022-01-10 05:37:53,132] jepsen worker 85 - jepsen.util 625 :info :write [28 16] :SyncWriteAmbiguous INFO [2022-01-10 05:37:53,134] jepsen worker 85 - jepsen.util 715 :invoke :write [28 18] INFO [2022-01-10 05:37:53,181] jepsen worker 83 - jepsen.util 713 :info :write [27 48] :RequestTimeoutException INFO [2022-01-10 05:37:53,181] jepsen worker 83 - jepsen.util 803 :invoke :read [27 nil] INFO [2022-01-10 05:37:53,197] jepsen worker 28 - jepsen.util 658 :info :write [9 46] :SyncWriteAmbiguous INFO [2022-01-10 05:37:53,197] jepsen worker 28 - jepsen.util 748 :invoke :read [9 nil] INFO [2022-01-10 05:37:53,279] jepsen worker 28 - jepsen.util 748 :ok :read [9 22] INFO [2022-01-10 05:37:53,280] jepsen worker 28 - jepsen.util 748 :invoke :write [9 14] INFO [2022-01-10 05:37:53,447] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0.00588235294117645}} INFO [2022-01-10 05:37:53,447] jepsen worker nemesis - couchbase.util Rebalance check count: 31 INFO [2022-01-10 05:37:53,447] jepsen worker nemesis - couchbase.util Rebalance stuck count: 29 INFO [2022-01-10 05:37:53,466] jepsen worker 82 - jepsen.util 442 :info :write [27 3] :SyncWriteAmbiguous INFO [2022-01-10 05:37:53,468] jepsen worker 82 - jepsen.util 532 :invoke :write [27 39] INFO [2022-01-10 05:37:53,567] jepsen worker 58 - jepsen.util 688 :ok :read [19 31] INFO [2022-01-10 05:37:53,568] jepsen worker 58 - jepsen.util 688 :invoke :read [19 nil] INFO [2022-01-10 05:37:53,572] jepsen worker 45 - jepsen.util 675 :info :write [15 11] :RequestTimeoutException INFO [2022-01-10 05:37:53,573] jepsen worker 45 - jepsen.util 765 :invoke :write [15 13] INFO [2022-01-10 05:37:53,588] jepsen worker 3 - jepsen.util 723 :ok :read [1 41] INFO [2022-01-10 05:37:53,589] jepsen worker 3 - jepsen.util 723 :invoke :write [1 45] INFO [2022-01-10 05:37:53,597] jepsen worker 76 - jepsen.util 616 :info :write [25 35] :RequestTimeoutException INFO [2022-01-10 05:37:53,597] jepsen worker 76 - jepsen.util 706 :invoke :read [25 nil] INFO [2022-01-10 05:37:53,605] jepsen worker 24 - jepsen.util 654 :ok :read [8 :nil] INFO [2022-01-10 05:37:53,605] jepsen worker 24 - jepsen.util 654 :invoke :write [8 27] INFO [2022-01-10 05:37:53,625] jepsen worker 21 - jepsen.util 741 :info :write [7 42] :RequestTimeoutException INFO [2022-01-10 05:37:53,626] jepsen worker 21 - jepsen.util 831 :invoke :write [7 25] INFO [2022-01-10 05:37:53,632] jepsen worker 72 - jepsen.util 792 :info :write [24 13] :RequestTimeoutException INFO [2022-01-10 05:37:53,633] jepsen worker 72 - jepsen.util 882 :invoke :read [24 nil] INFO [2022-01-10 05:37:53,682] jepsen worker 75 - jepsen.util 615 :info :write [25 19] :RequestTimeoutException INFO [2022-01-10 05:37:53,682] jepsen worker 75 - jepsen.util 705 :invoke :read [25 nil] INFO [2022-01-10 05:37:53,703] jepsen worker 43 - jepsen.util 673 :info :write [14 10] :RequestTimeoutException INFO [2022-01-10 05:37:53,704] jepsen worker 43 - jepsen.util 763 :invoke :read [14 nil] INFO [2022-01-10 05:37:53,762] jepsen worker 83 - jepsen.util 803 :ok :read [27 30] INFO [2022-01-10 05:37:53,762] jepsen worker 83 - jepsen.util 803 :invoke :read [27 nil] INFO [2022-01-10 05:37:53,773] jepsen worker 31 - jepsen.util 751 :info :write [10 11] :RequestTimeoutException INFO [2022-01-10 05:37:53,775] jepsen worker 31 - jepsen.util 841 :invoke :write [10 27] INFO [2022-01-10 05:37:53,799] jepsen worker 58 - jepsen.util 688 :ok :read [19 31] INFO [2022-01-10 05:37:53,800] jepsen worker 58 - jepsen.util 688 :invoke :read [19 nil] INFO [2022-01-10 05:37:53,831] jepsen worker 83 - jepsen.util 803 :ok :read [27 30] INFO [2022-01-10 05:37:53,831] jepsen worker 83 - jepsen.util 803 :invoke :write [27 29] INFO [2022-01-10 05:37:53,890] jepsen worker 58 - jepsen.util 688 :ok :read [19 31] INFO [2022-01-10 05:37:53,891] jepsen worker 58 - jepsen.util 688 :invoke :read [19 nil] INFO [2022-01-10 05:37:53,927] jepsen worker 58 - jepsen.util 688 :ok :read [19 31] INFO [2022-01-10 05:37:53,927] jepsen worker 58 - jepsen.util 688 :invoke :read [19 nil] INFO [2022-01-10 05:37:53,931] jepsen worker 60 - jepsen.util 690 :info :write [20 33] :RequestTimeoutException INFO [2022-01-10 05:37:53,932] jepsen worker 60 - jepsen.util 780 :invoke :write [20 10] INFO [2022-01-10 05:37:53,972] jepsen worker 1 - jepsen.util 631 :info :write [0 3] :SyncWriteAmbiguous INFO [2022-01-10 05:37:53,972] jepsen worker 1 - jepsen.util 721 :invoke :read [0 nil] INFO [2022-01-10 05:37:53,996] jepsen worker 16 - jepsen.util 556 :info :write [5 34] :RequestTimeoutException INFO [2022-01-10 05:37:53,997] jepsen worker 16 - jepsen.util 646 :invoke :read [5 nil] INFO [2022-01-10 05:37:54,000] jepsen worker 58 - jepsen.util 688 :ok :read [19 31] INFO [2022-01-10 05:37:54,001] jepsen worker 58 - jepsen.util 688 :invoke :write [19 3] INFO [2022-01-10 05:37:54,165] jepsen worker 27 - jepsen.util 567 :info :write [9 26] :RequestTimeoutException INFO [2022-01-10 05:37:54,167] jepsen worker 27 - jepsen.util 657 :invoke :write [9 3] INFO [2022-01-10 05:37:54,255] jepsen worker 84 - jepsen.util 714 :info :write [28 13] :RequestTimeoutException INFO [2022-01-10 05:37:54,256] jepsen worker 84 - jepsen.util 804 :invoke :read [28 nil] INFO [2022-01-10 05:37:54,261] jepsen worker 16 - jepsen.util 646 :ok :read [5 42] INFO [2022-01-10 05:37:54,262] jepsen worker 16 - jepsen.util 646 :invoke :write [5 44] INFO [2022-01-10 05:37:54,322] jepsen worker 23 - jepsen.util 563 :info :write [7 5] :RequestTimeoutException INFO [2022-01-10 05:37:54,323] jepsen worker 23 - jepsen.util 653 :invoke :write [7 45] INFO [2022-01-10 05:37:54,454] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.005847953216374324}, :ns_1@172.23.107.227 {:progress 0.005865102639296182}, :ns_1@172.23.105.187 {:progress 0.005865102639296182}, :ns_1@172.23.107.228 {:progress 0.00588235294117645}} INFO [2022-01-10 05:37:54,454] jepsen worker nemesis - couchbase.util Rebalance check count: 32 INFO [2022-01-10 05:37:54,454] jepsen worker nemesis - couchbase.util Rebalance stuck count: 29 INFO [2022-01-10 05:37:54,485] jepsen worker 73 - jepsen.util 613 :info :write [24 43] :RequestTimeoutException INFO [2022-01-10 05:37:54,485] jepsen worker 73 - jepsen.util 703 :invoke :read [24 nil] INFO [2022-01-10 05:37:54,646] jepsen worker 71 - jepsen.util 611 :info :write [23 40] :RequestTimeoutException INFO [2022-01-10 05:37:54,647] jepsen worker 71 - jepsen.util 701 :invoke :read [23 nil] INFO [2022-01-10 05:37:54,672] jepsen worker 59 - jepsen.util 779 :info :write [19 36] :RequestTimeoutException INFO [2022-01-10 05:37:54,673] jepsen worker 59 - jepsen.util 869 :invoke :read [19 nil] INFO [2022-01-10 05:37:54,796] jepsen worker 72 - jepsen.util 882 :ok :read [24 18] INFO [2022-01-10 05:37:54,796] jepsen worker 72 - jepsen.util 882 :invoke :read [24 nil] INFO [2022-01-10 05:37:54,836] jepsen worker 86 - jepsen.util 446 :info :write [28 46] :RequestTimeoutException INFO [2022-01-10 05:37:54,837] jepsen worker 86 - jepsen.util 536 :invoke :read [28 nil] INFO [2022-01-10 05:37:54,841] jepsen worker 35 - jepsen.util 755 :info :write [11 44] :RequestTimeoutException INFO [2022-01-10 05:37:54,842] jepsen worker 35 - jepsen.util 845 :invoke :write [11 33] INFO [2022-01-10 05:37:54,876] jepsen worker 50 - jepsen.util 680 :info :write [16 20] :RequestTimeoutException INFO [2022-01-10 05:37:54,877] jepsen worker 50 - jepsen.util 770 :invoke :write [16 0] INFO [2022-01-10 05:37:54,897] jepsen worker 29 - jepsen.util 479 :info :write [9 0] :RequestTimeoutException INFO [2022-01-10 05:37:54,898] jepsen worker 29 - jepsen.util 569 :invoke :write [9 17] INFO [2022-01-10 05:37:54,905] jepsen worker 15 - jepsen.util 465 :info :write [5 0] :RequestTimeoutException INFO [2022-01-10 05:37:54,905] jepsen worker 15 - jepsen.util 555 :invoke :read [5 nil] INFO [2022-01-10 05:37:55,015] jepsen worker 48 - jepsen.util 498 :info :write [16 18] :RequestTimeoutException INFO [2022-01-10 05:37:55,016] jepsen worker 48 - jepsen.util 588 :invoke :write [16 12] INFO [2022-01-10 05:37:55,076] jepsen worker 87 - jepsen.util 717 :info :write [29 1] :RequestTimeoutException INFO [2022-01-10 05:37:55,078] jepsen worker 87 - jepsen.util 807 :invoke :write [29 37] INFO [2022-01-10 05:37:55,462] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.01169590643274854}, :ns_1@172.23.107.227 {:progress 0.005865102639296182}, :ns_1@172.23.105.187 {:progress 0.005865102639296182}, :ns_1@172.23.107.228 {:progress 0.0117647058823529}} INFO [2022-01-10 05:37:55,462] jepsen worker nemesis - couchbase.util Rebalance check count: 33 INFO [2022-01-10 05:37:55,478] jepsen worker nemesis - couchbase.util Rebalance stuck count: 29 INFO [2022-01-10 05:37:55,596] jepsen worker 59 - jepsen.util 869 :ok :read [19 31] INFO [2022-01-10 05:37:55,596] jepsen worker 59 - jepsen.util 869 :invoke :read [19 nil] INFO [2022-01-10 05:37:55,612] jepsen worker 42 - jepsen.util 582 :info :write [14 41] :RequestTimeoutException INFO [2022-01-10 05:37:55,612] jepsen worker 42 - jepsen.util 672 :invoke :write [14 6] INFO [2022-01-10 05:37:55,681] jepsen worker 40 - jepsen.util 490 :info :write [13 29] :RequestTimeoutException INFO [2022-01-10 05:37:55,682] jepsen worker 40 - jepsen.util 580 :invoke :read [13 nil] INFO [2022-01-10 05:37:55,825] jepsen worker 57 - jepsen.util 687 :info :write [19 9] :RequestTimeoutException INFO [2022-01-10 05:37:55,826] jepsen worker 57 - jepsen.util 777 :invoke :write [19 39] INFO [2022-01-10 05:37:56,012] jepsen worker 41 - jepsen.util 491 :info :write [13 3] :RequestTimeoutException INFO [2022-01-10 05:37:56,013] jepsen worker 41 - jepsen.util 581 :invoke :write [13 7] INFO [2022-01-10 05:37:56,026] jepsen worker 56 - jepsen.util 686 :info :write [18 45] :RequestTimeoutException INFO [2022-01-10 05:37:56,027] jepsen worker 56 - jepsen.util 776 :invoke :read [18 nil] INFO [2022-01-10 05:37:56,105] jepsen worker 79 - jepsen.util 619 :info :write [26 5] :RequestTimeoutException INFO [2022-01-10 05:37:56,106] jepsen worker 79 - jepsen.util 709 :invoke :read [26 nil] INFO [2022-01-10 05:37:56,109] jepsen worker 51 - jepsen.util 501 :info :write [17 28] :RequestTimeoutException INFO [2022-01-10 05:37:56,110] jepsen worker 51 - jepsen.util 591 :invoke :write [17 22] INFO [2022-01-10 05:37:56,113] jepsen worker 5 - jepsen.util 635 :ok :read [1 41] INFO [2022-01-10 05:37:56,113] jepsen worker 5 - jepsen.util 635 :invoke :read [1 nil] INFO [2022-01-10 05:37:56,122] jepsen worker 55 - jepsen.util 595 :info :write [18 44] :RequestTimeoutException INFO [2022-01-10 05:37:56,123] jepsen worker 55 - jepsen.util 685 :invoke :write [18 19] INFO [2022-01-10 05:37:56,330] jepsen worker 9 - jepsen.util 729 :info :write [3 14] :RequestTimeoutException INFO [2022-01-10 05:37:56,331] jepsen worker 9 - jepsen.util 819 :invoke :write [3 12] INFO [2022-01-10 05:37:56,470] jepsen worker 17 - jepsen.util 467 :info :write [5 8] :RequestTimeoutException INFO [2022-01-10 05:37:56,471] jepsen worker 17 - jepsen.util 557 :invoke :write [5 27] INFO [2022-01-10 05:37:56,485] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.01169590643274854}, :ns_1@172.23.107.227 {:progress 0.005865102639296182}, :ns_1@172.23.105.187 {:progress 0.005865102639296182}, :ns_1@172.23.107.228 {:progress 0.0117647058823529}} INFO [2022-01-10 05:37:56,485] jepsen worker nemesis - couchbase.util Rebalance check count: 34 INFO [2022-01-10 05:37:56,485] jepsen worker nemesis - couchbase.util Rebalance stuck count: 30 INFO [2022-01-10 05:37:56,612] jepsen worker 34 - jepsen.util 754 :info :write [11 18] :RequestTimeoutException INFO [2022-01-10 05:37:56,613] jepsen worker 34 - jepsen.util 844 :invoke :write [11 9] INFO [2022-01-10 05:37:56,619] jepsen worker 71 - jepsen.util 701 :ok :read [23 :nil] INFO [2022-01-10 05:37:56,619] jepsen worker 71 - jepsen.util 701 :invoke :write [23 22] INFO [2022-01-10 05:37:56,727] jepsen worker 59 - jepsen.util 869 :ok :read [19 31] INFO [2022-01-10 05:37:56,727] jepsen worker 59 - jepsen.util 869 :invoke :read [19 nil] INFO [2022-01-10 05:37:56,926] jepsen worker 7 - jepsen.util 547 :info :write [2 39] :RequestTimeoutException INFO [2022-01-10 05:37:56,927] jepsen worker 7 - jepsen.util 637 :invoke :read [2 nil] INFO [2022-01-10 05:37:57,164] jepsen worker 72 - jepsen.util 882 :ok :read [24 18] INFO [2022-01-10 05:37:57,165] jepsen worker 72 - jepsen.util 882 :invoke :write [24 25] INFO [2022-01-10 05:37:57,166] jepsen worker 18 - jepsen.util 648 :ok :read [6 25] INFO [2022-01-10 05:37:57,166] jepsen worker 18 - jepsen.util 648 :invoke :read [6 nil] INFO [2022-01-10 05:37:57,226] jepsen worker 10 - jepsen.util 550 :info :write [3 26] :RequestTimeoutException INFO [2022-01-10 05:37:57,227] jepsen worker 10 - jepsen.util 640 :invoke :read [3 nil] INFO [2022-01-10 05:37:57,228] jepsen worker 64 - jepsen.util 694 :ok :read [21 7] INFO [2022-01-10 05:37:57,229] jepsen worker 64 - jepsen.util 694 :invoke :write [21 16] INFO [2022-01-10 05:37:57,272] jepsen worker 70 - jepsen.util 790 :info :write [23 16] :RequestTimeoutException INFO [2022-01-10 05:37:57,274] jepsen worker 70 - jepsen.util 880 :invoke :write [23 42] INFO [2022-01-10 05:37:57,283] jepsen worker 43 - jepsen.util 763 :ok :read [14 14] INFO [2022-01-10 05:37:57,283] jepsen worker 43 - jepsen.util 763 :invoke :write [14 11] INFO [2022-01-10 05:37:57,368] jepsen worker 18 - jepsen.util 648 :ok :read [6 25] INFO [2022-01-10 05:37:57,369] jepsen worker 18 - jepsen.util 648 :invoke :write [6 10] INFO [2022-01-10 05:37:57,411] jepsen worker 2 - jepsen.util 542 :info :write [0 10] :RequestTimeoutException INFO [2022-01-10 05:37:57,412] jepsen worker 2 - jepsen.util 632 :invoke :write [0 46] INFO [2022-01-10 05:37:57,431] jepsen worker 54 - jepsen.util 504 :info :write [18 42] :RequestTimeoutException INFO [2022-01-10 05:37:57,432] jepsen worker 54 - jepsen.util 594 :invoke :write [18 19] INFO [2022-01-10 05:37:57,497] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.01169590643274854}, :ns_1@172.23.107.227 {:progress 0.005865102639296182}, :ns_1@172.23.105.187 {:progress 0.005865102639296182}, :ns_1@172.23.107.228 {:progress 0.0117647058823529}} INFO [2022-01-10 05:37:57,497] jepsen worker nemesis - couchbase.util Rebalance check count: 35 INFO [2022-01-10 05:37:57,498] jepsen worker nemesis - couchbase.util Rebalance stuck count: 31 INFO [2022-01-10 05:37:57,518] jepsen worker 59 - jepsen.util 869 :ok :read [19 31] INFO [2022-01-10 05:37:57,518] jepsen worker 59 - jepsen.util 869 :invoke :read [19 nil] INFO [2022-01-10 05:37:57,672] jepsen worker 32 - jepsen.util 662 :ok :read [10 37] INFO [2022-01-10 05:37:57,673] jepsen worker 32 - jepsen.util 662 :invoke :write [10 10] INFO [2022-01-10 05:37:57,693] jepsen worker 38 - jepsen.util 668 :ok :read [12 :nil] INFO [2022-01-10 05:37:57,693] jepsen worker 38 - jepsen.util 668 :invoke :read [12 nil] INFO [2022-01-10 05:37:57,729] jepsen worker 10 - jepsen.util 640 :ok :read [3 2] INFO [2022-01-10 05:37:57,730] jepsen worker 10 - jepsen.util 640 :invoke :write [3 12] INFO [2022-01-10 05:37:57,749] jepsen worker 15 - jepsen.util 555 :ok :read [5 42] INFO [2022-01-10 05:37:57,750] jepsen worker 15 - jepsen.util 555 :invoke :read [5 nil] INFO [2022-01-10 05:37:57,830] jepsen worker 49 - jepsen.util 499 :info :write [16 49] :SyncWriteAmbiguous INFO [2022-01-10 05:37:57,831] jepsen worker 49 - jepsen.util 589 :invoke :read [16 nil] INFO [2022-01-10 05:37:57,833] jepsen worker 37 - jepsen.util 757 :ok :read [12 :nil] INFO [2022-01-10 05:37:57,834] jepsen worker 37 - jepsen.util 757 :invoke :read [12 nil] INFO [2022-01-10 05:37:57,838] jepsen worker 1 - jepsen.util 721 :ok :read [0 :nil] INFO [2022-01-10 05:37:57,838] jepsen worker 1 - jepsen.util 721 :invoke :read [0 nil] INFO [2022-01-10 05:37:57,848] jepsen worker 30 - jepsen.util 480 :ok :read [10 37] INFO [2022-01-10 05:37:57,849] jepsen worker 30 - jepsen.util 480 :invoke :read [10 nil] INFO [2022-01-10 05:37:57,874] jepsen worker 59 - jepsen.util 869 :ok :read [19 31] INFO [2022-01-10 05:37:57,875] jepsen worker 59 - jepsen.util 869 :invoke :read [19 nil] INFO [2022-01-10 05:37:57,946] jepsen worker 88 - jepsen.util 718 :info :write [29 26] :RequestTimeoutException INFO [2022-01-10 05:37:57,947] jepsen worker 88 - jepsen.util 808 :invoke :write [29 42] INFO [2022-01-10 05:37:58,004] jepsen worker 89 - jepsen.util 899 :ok :read [29 33] INFO [2022-01-10 05:37:58,005] jepsen worker 89 - jepsen.util 899 :invoke :read [29 nil] INFO [2022-01-10 05:37:58,022] jepsen worker 11 - jepsen.util 551 :ok :read [3 2] INFO [2022-01-10 05:37:58,023] jepsen worker 11 - jepsen.util 551 :invoke :write [3 19] INFO [2022-01-10 05:37:58,057] jepsen worker 5 - jepsen.util 635 :ok :read [1 41] INFO [2022-01-10 05:37:58,058] jepsen worker 5 - jepsen.util 635 :invoke :read [1 nil] INFO [2022-01-10 05:37:58,064] jepsen worker 15 - jepsen.util 555 :ok :read [5 42] INFO [2022-01-10 05:37:58,065] jepsen worker 15 - jepsen.util 555 :invoke :read [5 nil] INFO [2022-01-10 05:37:58,196] jepsen worker 4 - jepsen.util 544 :info :write [1 0] :RequestTimeoutException INFO [2022-01-10 05:37:58,197] jepsen worker 4 - jepsen.util 634 :invoke :write [1 42] INFO [2022-01-10 05:37:58,271] jepsen worker 0 - jepsen.util 540 :info :write [0 16] :RequestTimeoutException INFO [2022-01-10 05:37:58,272] jepsen worker 0 - jepsen.util 630 :invoke :write [0 29] INFO [2022-01-10 05:37:58,326] jepsen worker 68 - jepsen.util 608 :info :write [22 24] :RequestTimeoutException INFO [2022-01-10 05:37:58,328] jepsen worker 68 - jepsen.util 698 :invoke :write [22 36] INFO [2022-01-10 05:37:58,359] jepsen worker 52 - jepsen.util 682 :ok :read [17 41] INFO [2022-01-10 05:37:58,359] jepsen worker 52 - jepsen.util 682 :invoke :read [17 nil] INFO [2022-01-10 05:37:58,421] jepsen worker 77 - jepsen.util 797 :info :write [25 6] :RequestTimeoutException INFO [2022-01-10 05:37:58,422] jepsen worker 77 - jepsen.util 887 :invoke :read [25 nil] INFO [2022-01-10 05:37:58,504] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.03508771929824561}, :ns_1@172.23.107.227 {:progress 0.01173020527859236}, :ns_1@172.23.105.187 {:progress 0.01173020527859236}, :ns_1@172.23.107.228 {:progress 0.03529411764705881}} INFO [2022-01-10 05:37:58,504] jepsen worker nemesis - couchbase.util Rebalance check count: 36 INFO [2022-01-10 05:37:58,504] jepsen worker nemesis - couchbase.util Rebalance stuck count: 31 INFO [2022-01-10 05:37:58,536] jepsen worker 25 - jepsen.util 565 :info :write [8 49] :RequestTimeoutException INFO [2022-01-10 05:37:58,538] jepsen worker 25 - jepsen.util 655 :invoke :write [8 28] INFO [2022-01-10 05:37:58,644] jepsen worker 38 - jepsen.util 668 :ok :read [12 :nil] INFO [2022-01-10 05:37:58,644] jepsen worker 38 - jepsen.util 668 :invoke :read [12 nil] INFO [2022-01-10 05:37:58,795] jepsen worker 47 - jepsen.util 767 :info :write [15 5] :RequestTimeoutException INFO [2022-01-10 05:37:58,796] jepsen worker 47 - jepsen.util 857 :invoke :read [15 nil] INFO [2022-01-10 05:37:58,835] jepsen worker 62 - jepsen.util 782 :info :write [20 14] :SyncWriteAmbiguous INFO [2022-01-10 05:37:58,836] jepsen worker 62 - jepsen.util 872 :invoke :read [20 nil] INFO [2022-01-10 05:37:58,865] jepsen worker 84 - jepsen.util 804 :ok :read [28 :nil] INFO [2022-01-10 05:37:58,865] jepsen worker 84 - jepsen.util 804 :invoke :read [28 nil] INFO [2022-01-10 05:37:58,886] jepsen worker 74 - jepsen.util 794 :info :write [24 9] :RequestTimeoutException INFO [2022-01-10 05:37:58,887] jepsen worker 74 - jepsen.util 884 :invoke :write [24 32] INFO [2022-01-10 05:37:58,963] jepsen worker 52 - jepsen.util 682 :ok :read [17 41] INFO [2022-01-10 05:37:58,963] jepsen worker 52 - jepsen.util 682 :invoke :read [17 nil] INFO [2022-01-10 05:37:59,030] jepsen worker 67 - jepsen.util 607 :info :write [22 3] :RequestTimeoutException INFO [2022-01-10 05:37:59,031] jepsen worker 67 - jepsen.util 697 :invoke :write [22 40] INFO [2022-01-10 05:37:59,145] jepsen worker 84 - jepsen.util 804 :ok :read [28 :nil] INFO [2022-01-10 05:37:59,145] jepsen worker 84 - jepsen.util 804 :invoke :read [28 nil] INFO [2022-01-10 05:37:59,147] jepsen worker 84 - jepsen.util 804 :ok :read [28 :nil] INFO [2022-01-10 05:37:59,147] jepsen worker 84 - jepsen.util 804 :invoke :read [28 nil] INFO [2022-01-10 05:37:59,149] jepsen worker 84 - jepsen.util 804 :ok :read [28 :nil] INFO [2022-01-10 05:37:59,150] jepsen worker 84 - jepsen.util 804 :invoke :write [28 5] INFO [2022-01-10 05:37:59,301] jepsen worker 36 - jepsen.util 306 :info :write [12 44] :RequestTimeoutException INFO [2022-01-10 05:37:59,302] jepsen worker 36 - jepsen.util 396 :invoke :read [12 nil] INFO [2022-01-10 05:37:59,415] jepsen worker 33 - jepsen.util 573 :info :write [11 2] :RequestTimeoutException INFO [2022-01-10 05:37:59,416] jepsen worker 33 - jepsen.util 663 :invoke :write [11 25] INFO [2022-01-10 05:37:59,436] jepsen worker 44 - jepsen.util 674 :info :write [14 15] :RequestTimeoutException INFO [2022-01-10 05:37:59,437] jepsen worker 44 - jepsen.util 764 :invoke :write [14 23] WARN [2022-01-10 05:37:59,520] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9429037,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9810746,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9492823,"encode_duration_us":20031,"operation_id":"0x131a","timeout_ms":10000,"last_server_duration_us":9412081,"total_duration_us":10001961},{"operation_name":"upsert","last_dispatch_duration_us":12477219,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":20451013,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9535974,"encode_duration_us":16673,"operation_id":"0x13c2","timeout_ms":10000,"last_server_duration_us":9099014,"total_duration_us":10010206}],"total_count":2}} INFO [2022-01-10 05:37:59,528] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.04093567251461994}, :ns_1@172.23.107.227 {:progress 0.01173020527859236}, :ns_1@172.23.105.187 {:progress 0.01173020527859236}, :ns_1@172.23.107.228 {:progress 0.04117647058823526}} INFO [2022-01-10 05:37:59,528] jepsen worker nemesis - couchbase.util Rebalance check count: 37 INFO [2022-01-10 05:37:59,528] jepsen worker nemesis - couchbase.util Rebalance stuck count: 31 WARN [2022-01-10 05:37:59,620] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1267487,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3156400,"total_server_duration_us":114902,"encode_duration_us":18799,"operation_id":"0x1604","timeout_ms":10000,"last_server_duration_us":25976,"total_duration_us":10008343},{"operation_name":"upsert","last_dispatch_duration_us":367860,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":638256,"total_server_duration_us":153631,"encode_duration_us":16470,"operation_id":"0x15e7","timeout_ms":10000,"last_server_duration_us":47647,"total_duration_us":10007222},{"operation_name":"upsert","last_dispatch_duration_us":2501756,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":2501756,"total_server_duration_us":187583,"encode_duration_us":19952,"operation_id":"0x156a","timeout_ms":10000,"last_server_duration_us":187583,"total_duration_us":10007210},{"operation_name":"upsert","last_dispatch_duration_us":229241,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9129789,"total_server_duration_us":430761,"encode_duration_us":16355,"operation_id":"0x150c","timeout_ms":10000,"last_server_duration_us":50652,"total_duration_us":10006522},{"operation_name":"upsert","last_dispatch_duration_us":736691,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10398338,"total_server_duration_us":631498,"encode_duration_us":16442,"operation_id":"0x14d2","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10006057},{"operation_name":"upsert","last_dispatch_duration_us":6930721,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8915033,"total_server_duration_us":1697404,"encode_duration_us":17785,"operation_id":"0x146e","timeout_ms":10000,"last_server_duration_us":24154,"total_duration_us":10004563},{"operation_name":"upsert","last_dispatch_duration_us":1046311,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9791713,"total_server_duration_us":888656,"encode_duration_us":19120,"operation_id":"0x1564","timeout_ms":10000,"last_server_duration_us":396248,"total_duration_us":10003709},{"operation_name":"upsert","last_dispatch_duration_us":5269625,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":5269625,"total_server_duration_us":35815,"encode_duration_us":16787,"operation_id":"0x148a","timeout_ms":10000,"last_server_duration_us":35815,"total_duration_us":10003225},{"operation_name":"upsert","last_dispatch_duration_us":5500018,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8439232,"total_server_duration_us":520174,"encode_duration_us":19044,"operation_id":"0x1503","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10002457},{"operation_name":"upsert","last_dispatch_duration_us":1095021,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9756323,"total_server_duration_us":838954,"encode_duration_us":16229,"operation_id":"0x1534","timeout_ms":10000,"last_server_duration_us":86622,"total_duration_us":10001313}],"total_count":20}} INFO [2022-01-10 05:37:59,636] jepsen worker 38 - jepsen.util 668 :ok :read [12 :nil] INFO [2022-01-10 05:37:59,637] jepsen worker 38 - jepsen.util 668 :invoke :read [12 nil] INFO [2022-01-10 05:37:59,666] jepsen worker 79 - jepsen.util 709 :ok :read [26 10] INFO [2022-01-10 05:37:59,666] jepsen worker 79 - jepsen.util 709 :invoke :read [26 nil] INFO [2022-01-10 05:37:59,712] jepsen worker 66 - jepsen.util 426 :info :write [22 2] :RequestTimeoutException INFO [2022-01-10 05:37:59,713] jepsen worker 66 - jepsen.util 516 :invoke :write [22 31] INFO [2022-01-10 05:37:59,948] jepsen worker 78 - jepsen.util 618 :ok :read [26 10] INFO [2022-01-10 05:37:59,948] jepsen worker 78 - jepsen.util 618 :invoke :read [26 nil] INFO [2022-01-10 05:37:59,952] jepsen worker 76 - jepsen.util 706 :ok :read [25 :nil] INFO [2022-01-10 05:37:59,953] jepsen worker 76 - jepsen.util 706 :invoke :read [25 nil] INFO [2022-01-10 05:37:59,970] jepsen worker 80 - jepsen.util 710 :ok :read [26 10] INFO [2022-01-10 05:37:59,970] jepsen worker 80 - jepsen.util 710 :invoke :read [26 nil] INFO [2022-01-10 05:37:59,998] jepsen worker 49 - jepsen.util 589 :ok :read [16 38] INFO [2022-01-10 05:37:59,999] jepsen worker 49 - jepsen.util 589 :invoke :write [16 8] INFO [2022-01-10 05:38:00,231] jepsen worker 52 - jepsen.util 682 :ok :read [17 41] INFO [2022-01-10 05:38:00,232] jepsen worker 52 - jepsen.util 682 :invoke :read [17 nil] INFO [2022-01-10 05:38:00,297] jepsen worker 56 - jepsen.util 776 :ok :read [18 14] INFO [2022-01-10 05:38:00,298] jepsen worker 56 - jepsen.util 776 :invoke :write [18 48] INFO [2022-01-10 05:38:00,325] jepsen worker 53 - jepsen.util 503 :info :write [17 22] :RequestTimeoutException INFO [2022-01-10 05:38:00,326] jepsen worker 53 - jepsen.util 593 :invoke :write [17 43] INFO [2022-01-10 05:38:00,340] jepsen worker 46 - jepsen.util 586 :info :write [15 9] :RequestTimeoutException INFO [2022-01-10 05:38:00,340] jepsen worker 46 - jepsen.util 676 :invoke :write [15 14] INFO [2022-01-10 05:38:00,349] jepsen worker 78 - jepsen.util 618 :ok :read [26 10] INFO [2022-01-10 05:38:00,349] jepsen worker 78 - jepsen.util 618 :invoke :write [26 1] INFO [2022-01-10 05:38:00,358] jepsen worker 80 - jepsen.util 710 :ok :read [26 10] INFO [2022-01-10 05:38:00,359] jepsen worker 80 - jepsen.util 710 :invoke :read [26 nil] INFO [2022-01-10 05:38:00,509] jepsen worker 36 - jepsen.util 396 :ok :read [12 :nil] INFO [2022-01-10 05:38:00,510] jepsen worker 36 - jepsen.util 396 :invoke :read [12 nil] INFO [2022-01-10 05:38:00,521] jepsen worker 8 - jepsen.util 638 :info :write [2 11] :RequestTimeoutException INFO [2022-01-10 05:38:00,534] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05263157894736847}, :ns_1@172.23.107.227 {:progress 0.01173020527859236}, :ns_1@172.23.105.187 {:progress 0.01173020527859236}, :ns_1@172.23.107.228 {:progress 0.05294117647058827}} INFO [2022-01-10 05:38:00,606] jepsen worker 8 - jepsen.util 728 :invoke :read [2 nil] INFO [2022-01-10 05:38:00,562] jepsen worker 61 - jepsen.util 781 :info :write [20 13] :RequestTimeoutException INFO [2022-01-10 05:38:00,591] jepsen worker 13 - jepsen.util 553 :info :write [4 33] :RequestTimeoutException INFO [2022-01-10 05:38:00,606] jepsen worker 61 - jepsen.util 871 :invoke :write [20 37] INFO [2022-01-10 05:38:00,606] jepsen worker 13 - jepsen.util 643 :invoke :read [4 nil] INFO [2022-01-10 05:38:00,606] jepsen worker nemesis - couchbase.util Rebalance check count: 38 INFO [2022-01-10 05:38:00,606] jepsen worker 12 - jepsen.util 642 :info :write [4 45] :RequestTimeoutException INFO [2022-01-10 05:38:00,606] jepsen worker nemesis - couchbase.util Rebalance stuck count: 31 INFO [2022-01-10 05:38:00,607] jepsen worker 12 - jepsen.util 732 :invoke :read [4 nil] INFO [2022-01-10 05:38:00,632] jepsen worker 63 - jepsen.util 693 :info :write [21 15] :RequestTimeoutException INFO [2022-01-10 05:38:00,632] jepsen worker 63 - jepsen.util 783 :invoke :write [21 22] INFO [2022-01-10 05:38:00,635] jepsen worker 38 - jepsen.util 668 :ok :read [12 :nil] INFO [2022-01-10 05:38:00,635] jepsen worker 38 - jepsen.util 668 :invoke :read [12 nil] INFO [2022-01-10 05:38:00,648] jepsen worker 79 - jepsen.util 709 :ok :read [26 10] INFO [2022-01-10 05:38:00,649] jepsen worker 79 - jepsen.util 709 :invoke :write [26 30] INFO [2022-01-10 05:38:00,715] jepsen worker 36 - jepsen.util 396 :ok :read [12 :nil] INFO [2022-01-10 05:38:00,716] jepsen worker 36 - jepsen.util 396 :invoke :write [12 21] INFO [2022-01-10 05:38:00,796] jepsen worker 80 - jepsen.util 710 :ok :read [26 10] INFO [2022-01-10 05:38:00,797] jepsen worker 80 - jepsen.util 710 :invoke :read [26 nil] INFO [2022-01-10 05:38:00,967] jepsen worker 81 - jepsen.util 711 :info :write [27 22] :RequestTimeoutException INFO [2022-01-10 05:38:00,967] jepsen worker 81 - jepsen.util 801 :invoke :read [27 nil] INFO [2022-01-10 05:38:01,075] jepsen worker 13 - jepsen.util 643 :ok :read [4 :nil] INFO [2022-01-10 05:38:01,075] jepsen worker 13 - jepsen.util 643 :invoke :read [4 nil] WARN [2022-01-10 05:38:01,105] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12854344,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17595875,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9313912,"encode_duration_us":26179,"operation_id":"0x1445","timeout_ms":10000,"last_server_duration_us":9272145,"total_duration_us":10005305},{"operation_name":"upsert","last_dispatch_duration_us":17547382,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17547382,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9373240,"encode_duration_us":19144,"operation_id":"0x1447","timeout_ms":10000,"last_server_duration_us":9373240,"total_duration_us":10006521}],"total_count":2}} WARN [2022-01-10 05:38:01,113] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9432431,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10616439,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9321821,"encode_duration_us":24406,"operation_id":"0x1567","timeout_ms":10000,"last_server_duration_us":9214281,"total_duration_us":10007046}],"total_count":1}} WARN [2022-01-10 05:38:01,113] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7039434,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11649586,"total_server_duration_us":530400,"encode_duration_us":16719,"operation_id":"0x1543","timeout_ms":10000,"last_server_duration_us":88840,"total_duration_us":10009548},{"operation_name":"upsert","last_dispatch_duration_us":2598204,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3633793,"total_server_duration_us":52176,"encode_duration_us":16883,"operation_id":"0x1530","timeout_ms":10000,"last_server_duration_us":21731,"total_duration_us":10007709},{"operation_name":"upsert","last_dispatch_duration_us":9432431,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10616439,"total_server_duration_us":9321821,"encode_duration_us":24406,"operation_id":"0x1567","timeout_ms":10000,"last_server_duration_us":9214281,"total_duration_us":10007046},{"operation_name":"upsert","last_dispatch_duration_us":468827,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":1575999,"total_server_duration_us":193659,"encode_duration_us":16683,"operation_id":"0x15a5","timeout_ms":10000,"last_server_duration_us":103275,"total_duration_us":10006778},{"operation_name":"upsert","last_dispatch_duration_us":1258655,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1258655,"total_server_duration_us":21082,"encode_duration_us":16968,"operation_id":"0x15d4","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10006694},{"operation_name":"upsert","last_dispatch_duration_us":7056585,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9982021,"total_server_duration_us":591346,"encode_duration_us":18411,"operation_id":"0x15a3","timeout_ms":10000,"last_server_duration_us":55790,"total_duration_us":10005939},{"operation_name":"upsert","last_dispatch_duration_us":123317,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":4017379,"total_server_duration_us":339472,"encode_duration_us":20194,"operation_id":"0x1634","timeout_ms":10000,"last_server_duration_us":103117,"total_duration_us":10004760},{"operation_name":"upsert","last_dispatch_duration_us":1439077,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2594930,"total_server_duration_us":63065,"encode_duration_us":17343,"operation_id":"0x1631","timeout_ms":10000,"last_server_duration_us":35211,"total_duration_us":10003406},{"operation_name":"upsert","last_dispatch_duration_us":1383614,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":6240680,"total_server_duration_us":591516,"encode_duration_us":17612,"operation_id":"0x156c","timeout_ms":10000,"last_server_duration_us":47420,"total_duration_us":10003341},{"operation_name":"upsert","last_dispatch_duration_us":5784829,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5784829,"total_server_duration_us":20362,"encode_duration_us":16784,"operation_id":"0x1636","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10002151}],"total_count":20}} WARN [2022-01-10 05:38:01,205] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9255566,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11225303,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9169583,"encode_duration_us":18263,"operation_id":"0x1577","timeout_ms":10000,"last_server_duration_us":9116052,"total_duration_us":10001909},{"operation_name":"upsert","last_dispatch_duration_us":9357715,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9357715,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9024634,"encode_duration_us":16394,"operation_id":"0x147a","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10009033}],"total_count":2}} WARN [2022-01-10 05:38:01,205] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11988506,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11988506,"total_server_duration_us":103592,"operation_id":"0x1568","timeout_ms":30000,"last_server_duration_us":103592,"total_duration_us":11990858},{"operation_name":"get","last_dispatch_duration_us":11889801,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11889801,"total_server_duration_us":142412,"operation_id":"0x155d","timeout_ms":30000,"last_server_duration_us":142412,"total_duration_us":11890231},{"operation_name":"get","last_dispatch_duration_us":10185219,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10185219,"total_server_duration_us":8557,"operation_id":"0x15cf","timeout_ms":30000,"last_server_duration_us":8557,"total_duration_us":10197163},{"operation_name":"upsert","last_dispatch_duration_us":1351275,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":2296510,"total_server_duration_us":219145,"encode_duration_us":16551,"operation_id":"0x1635","timeout_ms":10000,"last_server_duration_us":162988,"total_duration_us":10010594},{"operation_name":"upsert","encode_duration_us":21502,"operation_id":"0x15c6","timeout_ms":10000,"total_duration_us":10010520},{"operation_name":"upsert","last_dispatch_duration_us":10335371,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":13552539,"total_server_duration_us":76118,"encode_duration_us":18788,"operation_id":"0x1535","timeout_ms":10000,"last_server_duration_us":45390,"total_duration_us":10008928},{"operation_name":"upsert","last_dispatch_duration_us":7506953,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8210754,"total_server_duration_us":139257,"encode_duration_us":16693,"operation_id":"0x1479","timeout_ms":10000,"last_server_duration_us":21813,"total_duration_us":10008308},{"operation_name":"upsert","last_dispatch_duration_us":1498481,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4665755,"total_server_duration_us":90830,"encode_duration_us":17483,"operation_id":"0x15d7","timeout_ms":10000,"last_server_duration_us":51828,"total_duration_us":10006857},{"operation_name":"upsert","last_dispatch_duration_us":7166183,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8127490,"total_server_duration_us":376986,"encode_duration_us":19273,"operation_id":"0x14f9","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10006589},{"operation_name":"upsert","last_dispatch_duration_us":1794916,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4180345,"total_server_duration_us":160882,"encode_duration_us":16574,"operation_id":"0x1598","timeout_ms":10000,"last_server_duration_us":140600,"total_duration_us":10006379}],"total_count":23}} WARN [2022-01-10 05:38:01,208] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1687151,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":3240769,"total_server_duration_us":209551,"encode_duration_us":18259,"operation_id":"0x15a2","timeout_ms":10000,"last_server_duration_us":53614,"total_duration_us":10011013},{"operation_name":"upsert","last_dispatch_duration_us":2455804,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2455804,"total_server_duration_us":317409,"encode_duration_us":17347,"operation_id":"0x1632","timeout_ms":10000,"last_server_duration_us":317409,"total_duration_us":10007733},{"operation_name":"upsert","last_dispatch_duration_us":2018292,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":5686540,"total_server_duration_us":685477,"encode_duration_us":16983,"operation_id":"0x1578","timeout_ms":10000,"last_server_duration_us":43070,"total_duration_us":10007590},{"operation_name":"upsert","last_dispatch_duration_us":3302987,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7221038,"total_server_duration_us":1146407,"encode_duration_us":18146,"operation_id":"0x14d3","timeout_ms":10000,"last_server_duration_us":853816,"total_duration_us":10007345},{"operation_name":"upsert","last_dispatch_duration_us":914617,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":1708080,"total_server_duration_us":150034,"encode_duration_us":19178,"operation_id":"0x160b","timeout_ms":10000,"last_server_duration_us":27764,"total_duration_us":10005334},{"operation_name":"upsert","last_dispatch_duration_us":3354785,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6426576,"total_server_duration_us":189080,"encode_duration_us":18764,"operation_id":"0x14ca","timeout_ms":10000,"last_server_duration_us":21649,"total_duration_us":10005017},{"operation_name":"upsert","last_dispatch_duration_us":2213031,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6423838,"total_server_duration_us":1325979,"encode_duration_us":17555,"operation_id":"0x14d7","timeout_ms":10000,"last_server_duration_us":24325,"total_duration_us":10004970},{"operation_name":"upsert","last_dispatch_duration_us":151693,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":2123324,"total_server_duration_us":232748,"encode_duration_us":16951,"operation_id":"0x1511","timeout_ms":10000,"last_server_duration_us":26507,"total_duration_us":10003723},{"operation_name":"upsert","last_dispatch_duration_us":9948592,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12895087,"total_server_duration_us":9628173,"encode_duration_us":20820,"operation_id":"0x14d5","timeout_ms":10000,"last_server_duration_us":9141633,"total_duration_us":10003216},{"operation_name":"upsert","last_dispatch_duration_us":2407491,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6083469,"total_server_duration_us":347069,"encode_duration_us":17484,"operation_id":"0x1509","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10002580}],"total_count":14}} WARN [2022-01-10 05:38:01,208] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10861280,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10861280,"total_server_duration_us":43,"operation_id":"0x153e","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":10863313},{"operation_name":"get","last_dispatch_duration_us":10243652,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10243652,"total_server_duration_us":6777,"operation_id":"0x15a4","timeout_ms":30000,"last_server_duration_us":6777,"total_duration_us":10246840},{"operation_name":"upsert","last_dispatch_duration_us":863485,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":1739981,"total_server_duration_us":689397,"encode_duration_us":16442,"operation_id":"0x1653","timeout_ms":10000,"last_server_duration_us":546689,"total_duration_us":10010991},{"operation_name":"upsert","last_dispatch_duration_us":11549781,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":11549781,"total_server_duration_us":121157,"encode_duration_us":17362,"operation_id":"0x1569","timeout_ms":10000,"last_server_duration_us":121157,"total_duration_us":10007607},{"operation_name":"upsert","last_dispatch_duration_us":439001,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10281469,"total_server_duration_us":306604,"encode_duration_us":21048,"operation_id":"0x152f","timeout_ms":10000,"last_server_duration_us":61139,"total_duration_us":10007326},{"operation_name":"upsert","last_dispatch_duration_us":9891393,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10037043,"total_server_duration_us":53462,"encode_duration_us":17808,"operation_id":"0x14cd","timeout_ms":10000,"last_server_duration_us":24325,"total_duration_us":10004526},{"operation_name":"upsert","last_dispatch_duration_us":9722697,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10206257,"total_server_duration_us":9522521,"encode_duration_us":18706,"operation_id":"0x14a1","timeout_ms":10000,"last_server_duration_us":9345231,"total_duration_us":10003978},{"operation_name":"upsert","last_dispatch_duration_us":492931,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10381798,"total_server_duration_us":155049,"encode_duration_us":20996,"operation_id":"0x1532","timeout_ms":10000,"last_server_duration_us":50068,"total_duration_us":10003775},{"operation_name":"upsert","last_dispatch_duration_us":10973858,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":11698397,"total_server_duration_us":245303,"encode_duration_us":22831,"operation_id":"0x1510","timeout_ms":10000,"last_server_duration_us":93498,"total_duration_us":10003310},{"operation_name":"upsert","last_dispatch_duration_us":9153699,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":16641038,"total_server_duration_us":9351924,"encode_duration_us":16951,"operation_id":"0x1498","timeout_ms":10000,"last_server_duration_us":9134169,"total_duration_us":10002045}],"total_count":19}} WARN [2022-01-10 05:38:01,308] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10147732,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11300645,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9215449,"encode_duration_us":17428,"operation_id":"0x1542","timeout_ms":10000,"last_server_duration_us":9146967,"total_duration_us":10001952},{"operation_name":"upsert","last_dispatch_duration_us":9948592,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12895087,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9628173,"encode_duration_us":20820,"operation_id":"0x14d5","timeout_ms":10000,"last_server_duration_us":9141633,"total_duration_us":10003216}],"total_count":2}} WARN [2022-01-10 05:38:01,308] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9153699,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16641038,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9351924,"encode_duration_us":16951,"operation_id":"0x1498","timeout_ms":10000,"last_server_duration_us":9134169,"total_duration_us":10002045},{"operation_name":"upsert","last_dispatch_duration_us":16359252,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18232429,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9560304,"encode_duration_us":17140,"operation_id":"0x1422","timeout_ms":10000,"last_server_duration_us":9253912,"total_duration_us":10005181},{"operation_name":"upsert","last_dispatch_duration_us":9722697,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10206257,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9522521,"encode_duration_us":18706,"operation_id":"0x14a1","timeout_ms":10000,"last_server_duration_us":9345231,"total_duration_us":10003978}],"total_count":3}} INFO [2022-01-10 05:38:01,392] jepsen worker 13 - jepsen.util 643 :ok :read [4 :nil] INFO [2022-01-10 05:38:01,394] jepsen worker 13 - jepsen.util 643 :invoke :write [4 1] WARN [2022-01-10 05:38:01,405] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9379544,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11910902,"total_server_duration_us":71228,"encode_duration_us":16781,"operation_id":"0x1505","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10014952},{"operation_name":"upsert","last_dispatch_duration_us":411708,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10084902,"total_server_duration_us":454721,"encode_duration_us":16603,"operation_id":"0x15f5","timeout_ms":10000,"last_server_duration_us":66300,"total_duration_us":10010478},{"operation_name":"upsert","last_dispatch_duration_us":844971,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10743252,"total_server_duration_us":370098,"encode_duration_us":18010,"operation_id":"0x15d1","timeout_ms":10000,"last_server_duration_us":113133,"total_duration_us":10010094},{"operation_name":"upsert","last_dispatch_duration_us":9357715,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9357715,"total_server_duration_us":9024634,"encode_duration_us":16394,"operation_id":"0x147a","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10009033},{"operation_name":"upsert","last_dispatch_duration_us":118637,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8324530,"total_server_duration_us":199290,"encode_duration_us":20603,"operation_id":"0x162f","timeout_ms":10000,"last_server_duration_us":24583,"total_duration_us":10006802},{"operation_name":"upsert","last_dispatch_duration_us":2035138,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9841358,"total_server_duration_us":256030,"encode_duration_us":16270,"operation_id":"0x1573","timeout_ms":10000,"last_server_duration_us":36525,"total_duration_us":10006700},{"operation_name":"upsert","last_dispatch_duration_us":9207033,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11753964,"total_server_duration_us":138799,"encode_duration_us":16702,"operation_id":"0x14c6","timeout_ms":10000,"last_server_duration_us":79829,"total_duration_us":10005861},{"operation_name":"upsert","last_dispatch_duration_us":2062789,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10245211,"total_server_duration_us":611628,"encode_duration_us":17337,"operation_id":"0x156b","timeout_ms":10000,"last_server_duration_us":50652,"total_duration_us":10004557},{"operation_name":"upsert","last_dispatch_duration_us":9255566,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11225303,"total_server_duration_us":9169583,"encode_duration_us":18263,"operation_id":"0x1577","timeout_ms":10000,"last_server_duration_us":9116052,"total_duration_us":10001909},{"operation_name":"upsert","last_dispatch_duration_us":9306670,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9306670,"total_server_duration_us":9165110,"encode_duration_us":21814,"operation_id":"0x15d6","timeout_ms":10000,"last_server_duration_us":9165110,"total_duration_us":9312422}],"total_count":15}} INFO [2022-01-10 05:38:01,406] jepsen worker 80 - jepsen.util 710 :ok :read [26 10] INFO [2022-01-10 05:38:01,406] jepsen worker 80 - jepsen.util 710 :invoke :write [26 2] INFO [2022-01-10 05:38:01,414] jepsen worker 52 - jepsen.util 682 :ok :read [17 41] INFO [2022-01-10 05:38:01,415] jepsen worker 52 - jepsen.util 682 :invoke :read [17 nil] INFO [2022-01-10 05:38:01,421] jepsen worker 39 - jepsen.util 669 :info :write [13 9] :RequestTimeoutException INFO [2022-01-10 05:38:01,422] jepsen worker 39 - jepsen.util 759 :invoke :write [13 24] INFO [2022-01-10 05:38:01,605] jepsen worker 20 - jepsen.util 470 :info :write [6 17] :RequestTimeoutException INFO [2022-01-10 05:38:01,606] jepsen worker 20 - jepsen.util 560 :invoke :write [6 13] INFO [2022-01-10 05:38:01,613] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05263157894736847}, :ns_1@172.23.107.227 {:progress 0.01173020527859236}, :ns_1@172.23.105.187 {:progress 0.01173020527859236}, :ns_1@172.23.107.228 {:progress 0.05294117647058827}} INFO [2022-01-10 05:38:01,613] jepsen worker nemesis - couchbase.util Rebalance check count: 39 INFO [2022-01-10 05:38:01,613] jepsen worker nemesis - couchbase.util Rebalance stuck count: 32 INFO [2022-01-10 05:38:01,618] jepsen worker 38 - jepsen.util 668 :ok :read [12 :nil] INFO [2022-01-10 05:38:01,619] jepsen worker 38 - jepsen.util 668 :invoke :read [12 nil] INFO [2022-01-10 05:38:01,785] jepsen worker 69 - jepsen.util 699 :info :write [23 42] :RequestTimeoutException INFO [2022-01-10 05:38:01,786] jepsen worker 69 - jepsen.util 789 :invoke :write [23 30] INFO [2022-01-10 05:38:01,974] jepsen worker 52 - jepsen.util 682 :ok :read [17 41] INFO [2022-01-10 05:38:01,975] jepsen worker 52 - jepsen.util 682 :invoke :write [17 29] INFO [2022-01-10 05:38:02,090] jepsen worker 19 - jepsen.util 739 :info :write [6 1] :RequestTimeoutException INFO [2022-01-10 05:38:02,091] jepsen worker 19 - jepsen.util 829 :invoke :write [6 27] INFO [2022-01-10 05:38:02,226] jepsen worker 38 - jepsen.util 668 :ok :read [12 :nil] INFO [2022-01-10 05:38:02,227] jepsen worker 38 - jepsen.util 668 :invoke :write [12 17] INFO [2022-01-10 05:38:02,350] jepsen worker 14 - jepsen.util 644 :info :write [4 7] :RequestTimeoutException INFO [2022-01-10 05:38:02,351] jepsen worker 14 - jepsen.util 734 :invoke :write [4 24] INFO [2022-01-10 05:38:02,491] jepsen worker 59 - jepsen.util 869 :ok :read [19 31] INFO [2022-01-10 05:38:02,491] jepsen worker 59 - jepsen.util 869 :invoke :write [19 13] INFO [2022-01-10 05:38:02,620] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05263157894736847}, :ns_1@172.23.107.227 {:progress 0.01173020527859236}, :ns_1@172.23.105.187 {:progress 0.01173020527859236}, :ns_1@172.23.107.228 {:progress 0.05294117647058827}} INFO [2022-01-10 05:38:02,620] jepsen worker nemesis - couchbase.util Rebalance check count: 40 INFO [2022-01-10 05:38:02,620] jepsen worker nemesis - couchbase.util Rebalance stuck count: 33 INFO [2022-01-10 05:38:02,907] jepsen worker 22 - jepsen.util 652 :ok :read [7 :nil] INFO [2022-01-10 05:38:02,907] jepsen worker 22 - jepsen.util 652 :invoke :read [7 nil] INFO [2022-01-10 05:38:03,083] jepsen worker 73 - jepsen.util 703 :ok :read [24 18] INFO [2022-01-10 05:38:03,084] jepsen worker 73 - jepsen.util 703 :invoke :read [24 nil] INFO [2022-01-10 05:38:03,151] jepsen worker 26 - jepsen.util 746 :info :write [8 10] :RequestTimeoutException INFO [2022-01-10 05:38:03,152] jepsen worker 26 - jepsen.util 836 :invoke :write [8 34] INFO [2022-01-10 05:38:03,161] jepsen worker 85 - jepsen.util 715 :info :write [28 18] :RequestTimeoutException INFO [2022-01-10 05:38:03,161] jepsen worker 85 - jepsen.util 805 :invoke :write [28 0] INFO [2022-01-10 05:38:03,306] jepsen worker 28 - jepsen.util 748 :info :write [9 14] :RequestTimeoutException INFO [2022-01-10 05:38:03,307] jepsen worker 28 - jepsen.util 838 :invoke :read [9 nil] INFO [2022-01-10 05:38:03,490] jepsen worker 82 - jepsen.util 532 :info :write [27 39] :RequestTimeoutException INFO [2022-01-10 05:38:03,491] jepsen worker 82 - jepsen.util 622 :invoke :write [27 29] INFO [2022-01-10 05:38:03,592] jepsen worker 45 - jepsen.util 765 :info :write [15 13] :RequestTimeoutException INFO [2022-01-10 05:38:03,593] jepsen worker 45 - jepsen.util 855 :invoke :write [15 7] INFO [2022-01-10 05:38:03,611] jepsen worker 3 - jepsen.util 723 :info :write [1 45] :RequestTimeoutException INFO [2022-01-10 05:38:03,612] jepsen worker 3 - jepsen.util 813 :invoke :read [1 nil] INFO [2022-01-10 05:38:03,626] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05263157894736847}, :ns_1@172.23.107.227 {:progress 0.01173020527859236}, :ns_1@172.23.105.187 {:progress 0.01173020527859236}, :ns_1@172.23.107.228 {:progress 0.05294117647058827}} INFO [2022-01-10 05:38:03,626] jepsen worker nemesis - couchbase.util Rebalance check count: 41 INFO [2022-01-10 05:38:03,626] jepsen worker nemesis - couchbase.util Rebalance stuck count: 34 INFO [2022-01-10 05:38:03,632] jepsen worker 24 - jepsen.util 654 :info :write [8 27] :RequestTimeoutException INFO [2022-01-10 05:38:03,632] jepsen worker 24 - jepsen.util 744 :invoke :write [8 24] INFO [2022-01-10 05:38:03,651] jepsen worker 21 - jepsen.util 831 :info :write [7 25] :RequestTimeoutException INFO [2022-01-10 05:38:03,656] jepsen worker 21 - jepsen.util 921 :invoke :write [7 48] INFO [2022-01-10 05:38:03,802] jepsen worker 31 - jepsen.util 841 :info :write [10 27] :RequestTimeoutException INFO [2022-01-10 05:38:03,803] jepsen worker 31 - jepsen.util 931 :invoke :read [10 nil] INFO [2022-01-10 05:38:03,855] jepsen worker 83 - jepsen.util 803 :info :write [27 29] :RequestTimeoutException INFO [2022-01-10 05:38:03,855] jepsen worker 83 - jepsen.util 893 :invoke :write [27 40] INFO [2022-01-10 05:38:03,956] jepsen worker 60 - jepsen.util 780 :info :write [20 10] :RequestTimeoutException INFO [2022-01-10 05:38:03,957] jepsen worker 60 - jepsen.util 870 :invoke :write [20 49] INFO [2022-01-10 05:38:04,025] jepsen worker 58 - jepsen.util 688 :info :write [19 3] :RequestTimeoutException INFO [2022-01-10 05:38:04,025] jepsen worker 58 - jepsen.util 778 :invoke :read [19 nil] INFO [2022-01-10 05:38:04,186] jepsen worker 27 - jepsen.util 657 :info :write [9 3] :RequestTimeoutException INFO [2022-01-10 05:38:04,187] jepsen worker 27 - jepsen.util 747 :invoke :read [9 nil] INFO [2022-01-10 05:38:04,285] jepsen worker 16 - jepsen.util 646 :info :write [5 44] :RequestTimeoutException INFO [2022-01-10 05:38:04,286] jepsen worker 16 - jepsen.util 736 :invoke :write [5 35] INFO [2022-01-10 05:38:04,342] jepsen worker 23 - jepsen.util 653 :info :write [7 45] :RequestTimeoutException INFO [2022-01-10 05:38:04,343] jepsen worker 23 - jepsen.util 743 :invoke :write [7 36] INFO [2022-01-10 05:38:04,632] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05263157894736847}, :ns_1@172.23.107.227 {:progress 0.01173020527859236}, :ns_1@172.23.105.187 {:progress 0.01173020527859236}, :ns_1@172.23.107.228 {:progress 0.05294117647058827}} INFO [2022-01-10 05:38:04,633] jepsen worker nemesis - couchbase.util Rebalance check count: 42 INFO [2022-01-10 05:38:04,633] jepsen worker nemesis - couchbase.util Rebalance stuck count: 35 INFO [2022-01-10 05:38:04,870] jepsen worker 35 - jepsen.util 845 :info :write [11 33] :RequestTimeoutException INFO [2022-01-10 05:38:04,871] jepsen worker 35 - jepsen.util 935 :invoke :write [11 22] INFO [2022-01-10 05:38:04,902] jepsen worker 50 - jepsen.util 770 :info :write [16 0] :RequestTimeoutException INFO [2022-01-10 05:38:04,903] jepsen worker 50 - jepsen.util 860 :invoke :write [16 31] INFO [2022-01-10 05:38:04,928] jepsen worker 31 - jepsen.util 931 :ok :read [10 37] INFO [2022-01-10 05:38:04,929] jepsen worker 29 - jepsen.util 569 :info :write [9 17] :RequestTimeoutException INFO [2022-01-10 05:38:04,929] jepsen worker 31 - jepsen.util 931 :invoke :read [10 nil] INFO [2022-01-10 05:38:04,929] jepsen worker 29 - jepsen.util 659 :invoke :read [9 nil] INFO [2022-01-10 05:38:05,044] jepsen worker 48 - jepsen.util 588 :info :write [16 12] :RequestTimeoutException INFO [2022-01-10 05:38:05,045] jepsen worker 48 - jepsen.util 678 :invoke :read [16 nil] INFO [2022-01-10 05:38:05,106] jepsen worker 87 - jepsen.util 807 :info :write [29 37] :RequestTimeoutException INFO [2022-01-10 05:38:05,107] jepsen worker 87 - jepsen.util 897 :invoke :write [29 20] INFO [2022-01-10 05:38:05,567] jepsen worker 31 - jepsen.util 931 :ok :read [10 37] INFO [2022-01-10 05:38:05,567] jepsen worker 31 - jepsen.util 931 :invoke :write [10 13] INFO [2022-01-10 05:38:05,641] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05263157894736847}, :ns_1@172.23.107.227 {:progress 0.01173020527859236}, :ns_1@172.23.105.187 {:progress 0.01173020527859236}, :ns_1@172.23.107.228 {:progress 0.05294117647058827}} INFO [2022-01-10 05:38:05,641] jepsen worker nemesis - couchbase.util Rebalance check count: 43 INFO [2022-01-10 05:38:05,641] jepsen worker nemesis - couchbase.util Rebalance stuck count: 36 INFO [2022-01-10 05:38:05,650] jepsen worker 42 - jepsen.util 672 :info :write [14 6] :RequestTimeoutException INFO [2022-01-10 05:38:05,651] jepsen worker 42 - jepsen.util 762 :invoke :write [14 47] INFO [2022-01-10 05:38:05,847] jepsen worker 57 - jepsen.util 777 :info :write [19 39] :RequestTimeoutException INFO [2022-01-10 05:38:05,847] jepsen worker 57 - jepsen.util 867 :invoke :write [19 26] INFO [2022-01-10 05:38:06,041] jepsen worker 41 - jepsen.util 581 :info :write [13 7] :RequestTimeoutException INFO [2022-01-10 05:38:06,043] jepsen worker 41 - jepsen.util 671 :invoke :read [13 nil] INFO [2022-01-10 05:38:06,130] jepsen worker 51 - jepsen.util 591 :info :write [17 22] :RequestTimeoutException INFO [2022-01-10 05:38:06,130] jepsen worker 51 - jepsen.util 681 :invoke :read [17 nil] INFO [2022-01-10 05:38:06,142] jepsen worker 55 - jepsen.util 685 :info :write [18 19] :RequestTimeoutException INFO [2022-01-10 05:38:06,142] jepsen worker 55 - jepsen.util 775 :invoke :read [18 nil] INFO [2022-01-10 05:38:06,356] jepsen worker 9 - jepsen.util 819 :info :write [3 12] :RequestTimeoutException INFO [2022-01-10 05:38:06,357] jepsen worker 9 - jepsen.util 909 :invoke :write [3 9] INFO [2022-01-10 05:38:06,501] jepsen worker 17 - jepsen.util 557 :info :write [5 27] :RequestTimeoutException INFO [2022-01-10 05:38:06,502] jepsen worker 17 - jepsen.util 647 :invoke :write [5 11] INFO [2022-01-10 05:38:06,635] jepsen worker 34 - jepsen.util 844 :info :write [11 9] :RequestTimeoutException INFO [2022-01-10 05:38:06,636] jepsen worker 34 - jepsen.util 934 :invoke :read [11 nil] INFO [2022-01-10 05:38:06,641] jepsen worker 71 - jepsen.util 701 :info :write [23 22] :RequestTimeoutException INFO [2022-01-10 05:38:06,642] jepsen worker 71 - jepsen.util 791 :invoke :write [23 36] INFO [2022-01-10 05:38:06,647] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05263157894736847}, :ns_1@172.23.107.227 {:progress 0.01173020527859236}, :ns_1@172.23.105.187 {:progress 0.01173020527859236}, :ns_1@172.23.107.228 {:progress 0.05294117647058827}} INFO [2022-01-10 05:38:06,648] jepsen worker nemesis - couchbase.util Rebalance check count: 44 INFO [2022-01-10 05:38:06,648] jepsen worker nemesis - couchbase.util Rebalance stuck count: 37 INFO [2022-01-10 05:38:06,748] jepsen worker 6 - jepsen.util 636 :ok :read [2 21] INFO [2022-01-10 05:38:06,748] jepsen worker 6 - jepsen.util 636 :invoke :write [2 31] INFO [2022-01-10 05:38:06,771] jepsen worker 77 - jepsen.util 887 :ok :read [25 :nil] INFO [2022-01-10 05:38:06,771] jepsen worker 77 - jepsen.util 887 :invoke :write [25 21] INFO [2022-01-10 05:38:07,082] jepsen worker 75 - jepsen.util 705 :ok :read [25 :nil] INFO [2022-01-10 05:38:07,084] jepsen worker 75 - jepsen.util 705 :invoke :read [25 nil] INFO [2022-01-10 05:38:07,184] jepsen worker 72 - jepsen.util 882 :info :write [24 25] :RequestTimeoutException INFO [2022-01-10 05:38:07,185] jepsen worker 72 - jepsen.util 972 :invoke :write [24 15] INFO [2022-01-10 05:38:07,195] jepsen worker 37 - jepsen.util 757 :ok :read [12 :nil] INFO [2022-01-10 05:38:07,197] jepsen worker 7 - jepsen.util 637 :ok :read [2 21] INFO [2022-01-10 05:38:07,200] jepsen worker 7 - jepsen.util 637 :invoke :write [2 14] INFO [2022-01-10 05:38:07,201] jepsen worker 37 - jepsen.util 757 :invoke :write [12 9] INFO [2022-01-10 05:38:07,209] jepsen worker 30 - jepsen.util 480 :ok :read [10 37] INFO [2022-01-10 05:38:07,210] jepsen worker 30 - jepsen.util 480 :invoke :read [10 nil] INFO [2022-01-10 05:38:07,223] jepsen worker 89 - jepsen.util 899 :ok :read [29 33] INFO [2022-01-10 05:38:07,223] jepsen worker 89 - jepsen.util 899 :invoke :read [29 nil] INFO [2022-01-10 05:38:07,241] jepsen worker 5 - jepsen.util 635 :ok :read [1 41] INFO [2022-01-10 05:38:07,242] jepsen worker 5 - jepsen.util 635 :invoke :read [1 nil] INFO [2022-01-10 05:38:07,250] jepsen worker 64 - jepsen.util 694 :info :write [21 16] :RequestTimeoutException INFO [2022-01-10 05:38:07,250] jepsen worker 64 - jepsen.util 784 :invoke :read [21 nil] INFO [2022-01-10 05:38:07,295] jepsen worker 40 - jepsen.util 580 :ok :read [13 23] INFO [2022-01-10 05:38:07,295] jepsen worker 40 - jepsen.util 580 :invoke :read [13 nil] INFO [2022-01-10 05:38:07,302] jepsen worker 70 - jepsen.util 880 :info :write [23 42] :RequestTimeoutException INFO [2022-01-10 05:38:07,302] jepsen worker 70 - jepsen.util 970 :invoke :write [23 41] INFO [2022-01-10 05:38:07,310] jepsen worker 43 - jepsen.util 763 :info :write [14 11] :RequestTimeoutException INFO [2022-01-10 05:38:07,310] jepsen worker 43 - jepsen.util 853 :invoke :read [14 nil] INFO [2022-01-10 05:38:07,390] jepsen worker 18 - jepsen.util 648 :info :write [6 10] :RequestTimeoutException INFO [2022-01-10 05:38:07,391] jepsen worker 18 - jepsen.util 738 :invoke :read [6 nil] INFO [2022-01-10 05:38:07,436] jepsen worker 2 - jepsen.util 632 :info :write [0 46] :RequestTimeoutException INFO [2022-01-10 05:38:07,437] jepsen worker 2 - jepsen.util 722 :invoke :write [0 39] INFO [2022-01-10 05:38:07,448] jepsen worker 55 - jepsen.util 775 :ok :read [18 14] INFO [2022-01-10 05:38:07,448] jepsen worker 55 - jepsen.util 775 :invoke :read [18 nil] INFO [2022-01-10 05:38:07,461] jepsen worker 54 - jepsen.util 594 :info :write [18 19] :RequestTimeoutException INFO [2022-01-10 05:38:07,462] jepsen worker 54 - jepsen.util 684 :invoke :write [18 37] INFO [2022-01-10 05:38:07,659] jepsen worker 30 - jepsen.util 480 :ok :read [10 37] INFO [2022-01-10 05:38:07,663] jepsen worker 30 - jepsen.util 480 :invoke :read [10 nil] INFO [2022-01-10 05:38:07,668] jepsen worker 51 - jepsen.util 681 :ok :read [17 41] INFO [2022-01-10 05:38:07,668] jepsen worker 51 - jepsen.util 681 :invoke :write [17 36] INFO [2022-01-10 05:38:07,673] jepsen worker 89 - jepsen.util 899 :ok :read [29 33] INFO [2022-01-10 05:38:07,674] jepsen worker 89 - jepsen.util 899 :invoke :read [29 nil] INFO [2022-01-10 05:38:07,674] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05847953216374269}, :ns_1@172.23.107.227 {:progress 0.02346041055718473}, :ns_1@172.23.105.187 {:progress 0.02346041055718473}, :ns_1@172.23.107.228 {:progress 0.05882352941176472}} INFO [2022-01-10 05:38:07,674] jepsen worker nemesis - couchbase.util Rebalance check count: 45 INFO [2022-01-10 05:38:07,674] jepsen worker nemesis - couchbase.util Rebalance stuck count: 37 INFO [2022-01-10 05:38:07,675] jepsen worker 3 - jepsen.util 813 :ok :read [1 41] INFO [2022-01-10 05:38:07,676] jepsen worker 3 - jepsen.util 813 :invoke :write [1 19] INFO [2022-01-10 05:38:07,696] jepsen worker 32 - jepsen.util 662 :info :write [10 10] :RequestTimeoutException INFO [2022-01-10 05:38:07,697] jepsen worker 32 - jepsen.util 752 :invoke :read [10 nil] INFO [2022-01-10 05:38:07,759] jepsen worker 10 - jepsen.util 640 :info :write [3 12] :RequestTimeoutException INFO [2022-01-10 05:38:07,760] jepsen worker 10 - jepsen.util 730 :invoke :read [3 nil] INFO [2022-01-10 05:38:07,809] jepsen worker 12 - jepsen.util 732 :ok :read [4 :nil] INFO [2022-01-10 05:38:07,811] jepsen worker 12 - jepsen.util 732 :invoke :read [4 nil] INFO [2022-01-10 05:38:07,830] jepsen worker 58 - jepsen.util 778 :ok :read [19 31] INFO [2022-01-10 05:38:07,831] jepsen worker 58 - jepsen.util 778 :invoke :write [19 41] INFO [2022-01-10 05:38:07,851] jepsen worker 5 - jepsen.util 635 :ok :read [1 41] INFO [2022-01-10 05:38:07,855] jepsen worker 5 - jepsen.util 635 :invoke :write [1 17] INFO [2022-01-10 05:38:07,861] jepsen worker 55 - jepsen.util 775 :ok :read [18 14] INFO [2022-01-10 05:38:07,863] jepsen worker 55 - jepsen.util 775 :invoke :write [18 35] INFO [2022-01-10 05:38:07,987] jepsen worker 88 - jepsen.util 808 :info :write [29 42] :RequestTimeoutException INFO [2022-01-10 05:38:07,988] jepsen worker 88 - jepsen.util 898 :invoke :read [29 nil] INFO [2022-01-10 05:38:08,004] jepsen worker 30 - jepsen.util 480 :ok :read [10 37] INFO [2022-01-10 05:38:08,005] jepsen worker 30 - jepsen.util 480 :invoke :read [10 nil] INFO [2022-01-10 05:38:08,024] jepsen worker 89 - jepsen.util 899 :ok :read [29 33] INFO [2022-01-10 05:38:08,024] jepsen worker 89 - jepsen.util 899 :invoke :read [29 nil] INFO [2022-01-10 05:38:08,033] jepsen worker 32 - jepsen.util 752 :ok :read [10 37] INFO [2022-01-10 05:38:08,033] jepsen worker 32 - jepsen.util 752 :invoke :write [10 42] INFO [2022-01-10 05:38:08,066] jepsen worker 11 - jepsen.util 551 :info :write [3 19] :RequestTimeoutException INFO [2022-01-10 05:38:08,067] jepsen worker 11 - jepsen.util 641 :invoke :read [3 nil] INFO [2022-01-10 05:38:08,162] jepsen worker 1 - jepsen.util 721 :ok :read [0 :nil] INFO [2022-01-10 05:38:08,163] jepsen worker 1 - jepsen.util 721 :invoke :write [0 40] INFO [2022-01-10 05:38:08,189] jepsen worker 29 - jepsen.util 659 :ok :read [9 22] INFO [2022-01-10 05:38:08,189] jepsen worker 29 - jepsen.util 659 :invoke :read [9 nil] INFO [2022-01-10 05:38:08,197] jepsen worker 40 - jepsen.util 580 :ok :read [13 23] INFO [2022-01-10 05:38:08,198] jepsen worker 40 - jepsen.util 580 :invoke :read [13 nil] INFO [2022-01-10 05:38:08,231] jepsen worker 4 - jepsen.util 634 :info :write [1 42] :RequestTimeoutException INFO [2022-01-10 05:38:08,231] jepsen worker 4 - jepsen.util 724 :invoke :write [1 9] INFO [2022-01-10 05:38:08,302] jepsen worker 0 - jepsen.util 630 :info :write [0 29] :RequestTimeoutException INFO [2022-01-10 05:38:08,303] jepsen worker 0 - jepsen.util 720 :invoke :read [0 nil] INFO [2022-01-10 05:38:08,356] jepsen worker 68 - jepsen.util 698 :info :write [22 36] :RequestTimeoutException INFO [2022-01-10 05:38:08,357] jepsen worker 68 - jepsen.util 788 :invoke :read [22 nil] INFO [2022-01-10 05:38:08,468] jepsen worker 65 - jepsen.util 785 :ok :read [21 7] INFO [2022-01-10 05:38:08,469] jepsen worker 65 - jepsen.util 785 :invoke :write [21 3] INFO [2022-01-10 05:38:08,565] jepsen worker 25 - jepsen.util 655 :info :write [8 28] :RequestTimeoutException INFO [2022-01-10 05:38:08,566] jepsen worker 12 - jepsen.util 732 :ok :read [4 :nil] INFO [2022-01-10 05:38:08,566] jepsen worker 25 - jepsen.util 745 :invoke :read [8 nil] INFO [2022-01-10 05:38:08,567] jepsen worker 12 - jepsen.util 732 :invoke :read [4 nil] INFO [2022-01-10 05:38:08,680] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05847953216374269}, :ns_1@172.23.107.227 {:progress 0.02346041055718473}, :ns_1@172.23.105.187 {:progress 0.02346041055718473}, :ns_1@172.23.107.228 {:progress 0.05882352941176472}} INFO [2022-01-10 05:38:08,681] jepsen worker nemesis - couchbase.util Rebalance check count: 46 INFO [2022-01-10 05:38:08,681] jepsen worker nemesis - couchbase.util Rebalance stuck count: 38 INFO [2022-01-10 05:38:08,734] jepsen worker 15 - jepsen.util 555 :ok :read [5 42] INFO [2022-01-10 05:38:08,735] jepsen worker 15 - jepsen.util 555 :invoke :read [5 nil] INFO [2022-01-10 05:38:08,792] jepsen worker 30 - jepsen.util 480 :ok :read [10 37] INFO [2022-01-10 05:38:08,793] jepsen worker 30 - jepsen.util 480 :invoke :write [10 42] INFO [2022-01-10 05:38:08,797] jepsen worker 89 - jepsen.util 899 :ok :read [29 33] INFO [2022-01-10 05:38:08,797] jepsen worker 89 - jepsen.util 899 :invoke :read [29 nil] INFO [2022-01-10 05:38:08,911] jepsen worker 74 - jepsen.util 884 :info :write [24 32] :RequestTimeoutException INFO [2022-01-10 05:38:08,912] jepsen worker 74 - jepsen.util 974 :invoke :write [24 45] INFO [2022-01-10 05:38:09,056] jepsen worker 67 - jepsen.util 697 :info :write [22 40] :RequestTimeoutException INFO [2022-01-10 05:38:09,057] jepsen worker 67 - jepsen.util 787 :invoke :read [22 nil] INFO [2022-01-10 05:38:09,100] jepsen worker 12 - jepsen.util 732 :ok :read [4 :nil] INFO [2022-01-10 05:38:09,101] jepsen worker 12 - jepsen.util 732 :invoke :write [4 12] INFO [2022-01-10 05:38:09,167] jepsen worker 0 - jepsen.util 720 :ok :read [0 :nil] INFO [2022-01-10 05:38:09,167] jepsen worker 0 - jepsen.util 720 :invoke :read [0 nil] INFO [2022-01-10 05:38:09,170] jepsen worker 84 - jepsen.util 804 :info :write [28 5] :RequestTimeoutException INFO [2022-01-10 05:38:09,170] jepsen worker 84 - jepsen.util 894 :invoke :read [28 nil] INFO [2022-01-10 05:38:09,172] jepsen worker 29 - jepsen.util 659 :ok :read [9 22] INFO [2022-01-10 05:38:09,173] jepsen worker 29 - jepsen.util 659 :invoke :write [9 6] INFO [2022-01-10 05:38:09,181] jepsen worker 40 - jepsen.util 580 :ok :read [13 23] INFO [2022-01-10 05:38:09,182] jepsen worker 40 - jepsen.util 580 :invoke :write [13 34] INFO [2022-01-10 05:38:09,445] jepsen worker 33 - jepsen.util 663 :info :write [11 25] :RequestTimeoutException INFO [2022-01-10 05:38:09,446] jepsen worker 33 - jepsen.util 753 :invoke :write [11 27] INFO [2022-01-10 05:38:09,466] jepsen worker 44 - jepsen.util 764 :info :write [14 23] :RequestTimeoutException INFO [2022-01-10 05:38:09,466] jepsen worker 44 - jepsen.util 854 :invoke :write [14 4] WARN [2022-01-10 05:38:09,571] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11004816,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14645128,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":11164082,"encode_duration_us":16749,"operation_id":"0x16d7","timeout_ms":10000,"last_server_duration_us":9991780,"total_duration_us":10005554},{"operation_name":"upsert","last_dispatch_duration_us":11896897,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":21026687,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10540245,"encode_duration_us":16355,"operation_id":"0x150c","timeout_ms":10000,"last_server_duration_us":10109484,"total_duration_us":10006522},{"operation_name":"upsert","last_dispatch_duration_us":9313473,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12469874,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9186259,"encode_duration_us":18799,"operation_id":"0x1604","timeout_ms":10000,"last_server_duration_us":9071357,"total_duration_us":10008343},{"operation_name":"upsert","last_dispatch_duration_us":11949176,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12587432,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10058069,"encode_duration_us":16470,"operation_id":"0x15e7","timeout_ms":10000,"last_server_duration_us":9904438,"total_duration_us":10007222}],"total_count":4}} WARN [2022-01-10 05:38:09,671] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10708891,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10708891,"total_server_duration_us":43,"operation_id":"0x1600","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":10710988},{"operation_name":"upsert","encode_duration_us":18246,"operation_id":"0x17d7","timeout_ms":10000,"total_duration_us":10010175},{"operation_name":"upsert","last_dispatch_duration_us":4745554,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":7616811,"total_server_duration_us":565517,"encode_duration_us":18169,"operation_id":"0x170f","timeout_ms":10000,"last_server_duration_us":215363,"total_duration_us":10009782},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","encode_duration_us":16725,"operation_id":"0x17aa","timeout_ms":10000,"total_duration_us":10009599},{"operation_name":"upsert","last_dispatch_duration_us":220785,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10098856,"total_server_duration_us":148266,"encode_duration_us":17377,"operation_id":"0x162e","timeout_ms":10000,"last_server_duration_us":94412,"total_duration_us":10009557},{"operation_name":"upsert","last_dispatch_duration_us":233132,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2579062,"total_server_duration_us":142844,"encode_duration_us":16556,"operation_id":"0x165c","timeout_ms":10000,"last_server_duration_us":97638,"total_duration_us":10006792},{"operation_name":"upsert","last_dispatch_duration_us":4619417,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8386600,"total_server_duration_us":509414,"encode_duration_us":16913,"operation_id":"0x16dd","timeout_ms":10000,"last_server_duration_us":104387,"total_duration_us":10006264},{"operation_name":"upsert","last_dispatch_duration_us":1737272,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":5138693,"total_server_duration_us":375180,"encode_duration_us":16474,"operation_id":"0x16e6","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10006117},{"operation_name":"upsert","last_dispatch_duration_us":11004816,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14645128,"total_server_duration_us":11164082,"encode_duration_us":16749,"operation_id":"0x16d7","timeout_ms":10000,"last_server_duration_us":9991780,"total_duration_us":10005554},{"operation_name":"upsert","last_dispatch_duration_us":4332556,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5042118,"total_server_duration_us":221909,"encode_duration_us":17274,"operation_id":"0x176d","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10001517}],"total_count":18}} INFO [2022-01-10 05:38:09,687] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05847953216374269}, :ns_1@172.23.107.227 {:progress 0.02346041055718473}, :ns_1@172.23.105.187 {:progress 0.02346041055718473}, :ns_1@172.23.107.228 {:progress 0.05882352941176472}} INFO [2022-01-10 05:38:09,687] jepsen worker nemesis - couchbase.util Rebalance check count: 47 INFO [2022-01-10 05:38:09,687] jepsen worker nemesis - couchbase.util Rebalance stuck count: 39 INFO [2022-01-10 05:38:09,740] jepsen worker 66 - jepsen.util 516 :info :write [22 31] :RequestTimeoutException INFO [2022-01-10 05:38:09,741] jepsen worker 66 - jepsen.util 606 :invoke :read [22 nil] INFO [2022-01-10 05:38:09,876] jepsen worker 15 - jepsen.util 555 :ok :read [5 42] INFO [2022-01-10 05:38:09,877] jepsen worker 15 - jepsen.util 555 :invoke :read [5 nil] INFO [2022-01-10 05:38:10,024] jepsen worker 49 - jepsen.util 589 :info :write [16 8] :RequestTimeoutException INFO [2022-01-10 05:38:10,025] jepsen worker 49 - jepsen.util 679 :invoke :write [16 42] INFO [2022-01-10 05:38:10,325] jepsen worker 56 - jepsen.util 776 :info :write [18 48] :RequestTimeoutException INFO [2022-01-10 05:38:10,325] jepsen worker 56 - jepsen.util 866 :invoke :read [18 nil] INFO [2022-01-10 05:38:10,351] jepsen worker 53 - jepsen.util 593 :info :write [17 43] :RequestTimeoutException INFO [2022-01-10 05:38:10,352] jepsen worker 53 - jepsen.util 683 :invoke :read [17 nil] INFO [2022-01-10 05:38:10,366] jepsen worker 46 - jepsen.util 676 :info :write [15 14] :RequestTimeoutException INFO [2022-01-10 05:38:10,368] jepsen worker 46 - jepsen.util 766 :invoke :write [15 23] INFO [2022-01-10 05:38:10,375] jepsen worker 78 - jepsen.util 618 :info :write [26 1] :RequestTimeoutException INFO [2022-01-10 05:38:10,375] jepsen worker 78 - jepsen.util 708 :invoke :read [26 nil] INFO [2022-01-10 05:38:10,636] jepsen worker 61 - jepsen.util 871 :info :write [20 37] :RequestTimeoutException INFO [2022-01-10 05:38:10,636] jepsen worker 61 - jepsen.util 961 :invoke :read [20 nil] INFO [2022-01-10 05:38:10,651] jepsen worker 63 - jepsen.util 783 :info :write [21 22] :RequestTimeoutException INFO [2022-01-10 05:38:10,676] jepsen worker 79 - jepsen.util 709 :info :write [26 30] :RequestTimeoutException INFO [2022-01-10 05:38:10,755] jepsen worker 63 - jepsen.util 873 :invoke :read [21 nil] INFO [2022-01-10 05:38:10,755] jepsen worker 79 - jepsen.util 799 :invoke :read [26 nil] INFO [2022-01-10 05:38:10,692] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05847953216374269}, :ns_1@172.23.107.227 {:progress 0.02346041055718473}, :ns_1@172.23.105.187 {:progress 0.02346041055718473}, :ns_1@172.23.107.228 {:progress 0.05882352941176472}} INFO [2022-01-10 05:38:10,741] jepsen worker 36 - jepsen.util 396 :info :write [12 21] :RequestTimeoutException INFO [2022-01-10 05:38:10,755] jepsen worker nemesis - couchbase.util Rebalance check count: 48 INFO [2022-01-10 05:38:10,755] jepsen worker nemesis - couchbase.util Rebalance stuck count: 40 INFO [2022-01-10 05:38:10,755] jepsen worker 36 - jepsen.util 486 :invoke :read [12 nil] INFO [2022-01-10 05:38:10,832] jepsen worker 11 - jepsen.util 641 :ok :read [3 2] INFO [2022-01-10 05:38:10,832] jepsen worker 11 - jepsen.util 641 :invoke :write [3 43] WARN [2022-01-10 05:38:11,119] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10626360,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10626360,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":10614544,"encode_duration_us":16625,"operation_id":"0x16ab","timeout_ms":10000,"last_server_duration_us":10614544,"total_duration_us":10008726}],"total_count":1}} WARN [2022-01-10 05:38:11,126] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10650217,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10650217,"total_server_duration_us":32,"operation_id":"0x179e","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":10652304},{"operation_name":"get","last_dispatch_duration_us":10290944,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10290944,"total_server_duration_us":1115,"operation_id":"0x1797","timeout_ms":30000,"last_server_duration_us":1115,"total_duration_us":10323147},{"operation_name":"upsert","last_dispatch_duration_us":430032,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":3102489,"total_server_duration_us":1024007,"encode_duration_us":18284,"operation_id":"0x1774","timeout_ms":10000,"last_server_duration_us":95635,"total_duration_us":10010069},{"operation_name":"upsert","last_dispatch_duration_us":10208770,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10208770,"total_server_duration_us":9328013,"encode_duration_us":19642,"operation_id":"0x1746","timeout_ms":10000,"last_server_duration_us":9328013,"total_duration_us":10009788},{"operation_name":"upsert","last_dispatch_duration_us":10667484,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":17264743,"total_server_duration_us":349865,"encode_duration_us":23504,"operation_id":"0x165a","timeout_ms":10000,"last_server_duration_us":183922,"total_duration_us":10009261},{"operation_name":"upsert","last_dispatch_duration_us":10346173,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10346173,"total_server_duration_us":355632,"encode_duration_us":17158,"operation_id":"0x16a8","timeout_ms":10000,"last_server_duration_us":355632,"total_duration_us":10009140},{"operation_name":"upsert","last_dispatch_duration_us":212009,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":2088180,"total_server_duration_us":769425,"encode_duration_us":17370,"operation_id":"0x1818","timeout_ms":10000,"last_server_duration_us":101224,"total_duration_us":10006726},{"operation_name":"upsert","last_dispatch_duration_us":12035711,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":12035711,"total_server_duration_us":19108,"encode_duration_us":18949,"operation_id":"0x16b3","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10005321},{"operation_name":"upsert","last_dispatch_duration_us":264080,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":5116626,"total_server_duration_us":1213781,"encode_duration_us":16375,"operation_id":"0x1748","timeout_ms":10000,"last_server_duration_us":99035,"total_duration_us":10004924},{"operation_name":"upsert","last_dispatch_duration_us":309880,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":309880,"total_server_duration_us":76451,"encode_duration_us":23695,"operation_id":"0x173a","timeout_ms":10000,"last_server_duration_us":76451,"total_duration_us":10003644}],"total_count":14}} WARN [2022-01-10 05:38:11,126] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17691131,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":20286062,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":10581194,"encode_duration_us":17343,"operation_id":"0x1631","timeout_ms":10000,"last_server_duration_us":10518129,"total_duration_us":10003406},{"operation_name":"upsert","last_dispatch_duration_us":10208770,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10208770,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9328013,"encode_duration_us":19642,"operation_id":"0x1746","timeout_ms":10000,"last_server_duration_us":9328013,"total_duration_us":10009788}],"total_count":2}} WARN [2022-01-10 05:38:11,219] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9631246,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10867058,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9483741,"encode_duration_us":16565,"operation_id":"0x1740","timeout_ms":10000,"last_server_duration_us":9285025,"total_duration_us":10002020},{"operation_name":"upsert","last_dispatch_duration_us":9270469,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10962751,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9060203,"encode_duration_us":28620,"operation_id":"0x1718","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10007952},{"operation_name":"upsert","last_dispatch_duration_us":10213250,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10548730,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9657044,"encode_duration_us":16377,"operation_id":"0x1772","timeout_ms":10000,"last_server_duration_us":9632375,"total_duration_us":10004516}],"total_count":3}} WARN [2022-01-10 05:38:11,219] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11295613,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11295613,"total_server_duration_us":23,"operation_id":"0x16b4","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":13398438},{"operation_name":"get","last_dispatch_duration_us":10251161,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10251161,"total_server_duration_us":23,"operation_id":"0x1765","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":10255079},{"operation_name":"upsert","last_dispatch_duration_us":2045333,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8299801,"total_server_duration_us":473405,"encode_duration_us":32115,"operation_id":"0x179c","timeout_ms":10000,"last_server_duration_us":77010,"total_duration_us":10010287},{"operation_name":"upsert","last_dispatch_duration_us":1458321,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":4591418,"total_server_duration_us":233396,"encode_duration_us":17149,"operation_id":"0x1710","timeout_ms":10000,"last_server_duration_us":18493,"total_duration_us":10010284},{"operation_name":"upsert","last_dispatch_duration_us":2029863,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9950870,"total_server_duration_us":850822,"encode_duration_us":16247,"operation_id":"0x180e","timeout_ms":10000,"last_server_duration_us":69615,"total_duration_us":10009948},{"operation_name":"upsert","last_dispatch_duration_us":11345043,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11345043,"total_server_duration_us":2351879,"encode_duration_us":19557,"operation_id":"0x17d8","timeout_ms":10000,"last_server_duration_us":2351879,"total_duration_us":10008767},{"operation_name":"upsert","last_dispatch_duration_us":10626360,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10626360,"total_server_duration_us":10614544,"encode_duration_us":16625,"operation_id":"0x16ab","timeout_ms":10000,"last_server_duration_us":10614544,"total_duration_us":10008726},{"operation_name":"upsert","last_dispatch_duration_us":1620126,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8870816,"total_server_duration_us":884989,"encode_duration_us":30019,"operation_id":"0x179a","timeout_ms":10000,"last_server_duration_us":329511,"total_duration_us":10008192},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":24065,"operation_id":"0x180b","timeout_ms":10000,"total_duration_us":10004478},{"operation_name":"upsert","last_dispatch_duration_us":697779,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":2679362,"total_server_duration_us":930506,"encode_duration_us":16285,"operation_id":"0x166c","timeout_ms":10000,"last_server_duration_us":96096,"total_duration_us":10002572}],"total_count":16}} WARN [2022-01-10 05:38:11,220] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":354939,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":2712046,"total_server_duration_us":772919,"encode_duration_us":16895,"operation_id":"0x170e","timeout_ms":10000,"last_server_duration_us":74786,"total_duration_us":10011153},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":18865,"operation_id":"0x17a7","timeout_ms":10000,"total_duration_us":10008645},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":16987,"operation_id":"0x1804","timeout_ms":10000,"total_duration_us":10008065},{"operation_name":"upsert","last_dispatch_duration_us":10679899,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10679899,"total_server_duration_us":9042674,"encode_duration_us":17404,"operation_id":"0x1745","timeout_ms":10000,"last_server_duration_us":9042674,"total_duration_us":10006785},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":16973,"operation_id":"0x16da","timeout_ms":10000,"total_duration_us":10006447},{"operation_name":"upsert","last_dispatch_duration_us":394093,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10116609,"total_server_duration_us":339880,"encode_duration_us":16404,"operation_id":"0x1776","timeout_ms":10000,"last_server_duration_us":139518,"total_duration_us":10006161},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":19583,"operation_id":"0x17cc","timeout_ms":10000,"total_duration_us":10004831},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":19080,"operation_id":"0x1773","timeout_ms":10000,"total_duration_us":10003867},{"operation_name":"upsert","last_dispatch_duration_us":390955,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":2215749,"total_server_duration_us":303230,"encode_duration_us":16911,"operation_id":"0x1738","timeout_ms":10000,"last_server_duration_us":52301,"total_duration_us":10002724},{"operation_name":"get","last_dispatch_duration_us":9359598,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9359598,"total_server_duration_us":4078,"operation_id":"0x1796","timeout_ms":30000,"last_server_duration_us":4078,"total_duration_us":9359993}],"total_count":18}} WARN [2022-01-10 05:38:11,221] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15821023,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":15821023,"total_server_duration_us":43,"operation_id":"0x167a","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":16387239},{"operation_name":"get","last_dispatch_duration_us":14789781,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":14789781,"total_server_duration_us":2549254,"operation_id":"0x1652","timeout_ms":30000,"last_server_duration_us":2549254,"total_duration_us":15876001},{"operation_name":"get","last_dispatch_duration_us":11004358,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11004358,"total_server_duration_us":27,"operation_id":"0x161e","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":13113733},{"operation_name":"get","last_dispatch_duration_us":11593503,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11593503,"total_server_duration_us":420415,"operation_id":"0x1719","timeout_ms":30000,"last_server_duration_us":420415,"total_duration_us":11596115},{"operation_name":"upsert","last_dispatch_duration_us":717265,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10678728,"total_server_duration_us":441894,"encode_duration_us":16463,"operation_id":"0x1770","timeout_ms":10000,"last_server_duration_us":325112,"total_duration_us":10010668},{"operation_name":"upsert","last_dispatch_duration_us":10159632,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":13313456,"total_server_duration_us":128538,"encode_duration_us":16514,"operation_id":"0x16cf","timeout_ms":10000,"last_server_duration_us":22554,"total_duration_us":10009926},{"operation_name":"upsert","last_dispatch_duration_us":896344,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10203987,"total_server_duration_us":485958,"encode_duration_us":20430,"operation_id":"0x17a6","timeout_ms":10000,"last_server_duration_us":327179,"total_duration_us":10009348},{"operation_name":"upsert","encode_duration_us":16771,"operation_id":"0x16b1","timeout_ms":10000,"total_duration_us":10009062},{"operation_name":"upsert","last_dispatch_duration_us":2304082,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10663553,"total_server_duration_us":444390,"encode_duration_us":16636,"operation_id":"0x1803","timeout_ms":10000,"last_server_duration_us":139698,"total_duration_us":10008529},{"operation_name":"upsert","last_dispatch_duration_us":10966790,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10966790,"total_server_duration_us":134692,"encode_duration_us":16882,"operation_id":"0x170c","timeout_ms":10000,"last_server_duration_us":134692,"total_duration_us":10007385}],"total_count":23}} INFO [2022-01-10 05:38:11,238] jepsen worker 89 - jepsen.util 899 :ok :read [29 33] INFO [2022-01-10 05:38:11,239] jepsen worker 89 - jepsen.util 899 :invoke :write [29 39] INFO [2022-01-10 05:38:11,265] jepsen worker 62 - jepsen.util 872 :ok :read [20 :nil] INFO [2022-01-10 05:38:11,265] jepsen worker 62 - jepsen.util 872 :invoke :write [20 37] INFO [2022-01-10 05:38:11,292] jepsen worker 34 - jepsen.util 934 :ok :read [11 21] INFO [2022-01-10 05:38:11,293] jepsen worker 34 - jepsen.util 934 :invoke :read [11 nil] WARN [2022-01-10 05:38:11,320] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10679899,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10679899,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9042674,"encode_duration_us":17404,"operation_id":"0x1745","timeout_ms":10000,"last_server_duration_us":9042674,"total_duration_us":10006785}],"total_count":1}} WARN [2022-01-10 05:38:11,321] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12300747,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18384217,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":10007815,"encode_duration_us":17484,"operation_id":"0x1509","timeout_ms":10000,"last_server_duration_us":9660746,"total_duration_us":10002580},{"operation_name":"upsert","last_dispatch_duration_us":11430332,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13553657,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9854220,"encode_duration_us":16951,"operation_id":"0x1511","timeout_ms":10000,"last_server_duration_us":9621472,"total_duration_us":10003723},{"operation_name":"upsert","last_dispatch_duration_us":12471087,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14179168,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10564229,"encode_duration_us":19178,"operation_id":"0x160b","timeout_ms":10000,"last_server_duration_us":10414195,"total_duration_us":10005334},{"operation_name":"upsert","last_dispatch_duration_us":11850091,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15090860,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9497797,"encode_duration_us":18259,"operation_id":"0x15a2","timeout_ms":10000,"last_server_duration_us":9288246,"total_duration_us":10011013}],"total_count":4}} INFO [2022-01-10 05:38:11,414] jepsen worker 13 - jepsen.util 643 :info :write [4 1] :RequestTimeoutException INFO [2022-01-10 05:38:11,416] jepsen worker 13 - jepsen.util 733 :invoke :write [4 26] WARN [2022-01-10 05:38:11,419] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12231302,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":15955749,"total_server_duration_us":372678,"encode_duration_us":16454,"operation_id":"0x170b","timeout_ms":10000,"last_server_duration_us":164724,"total_duration_us":10010015},{"operation_name":"upsert","last_dispatch_duration_us":99690,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":6088117,"total_server_duration_us":87563,"encode_duration_us":16190,"operation_id":"0x167b","timeout_ms":10000,"last_server_duration_us":26596,"total_duration_us":10009872},{"operation_name":"upsert","last_dispatch_duration_us":32603,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":385118,"total_server_duration_us":226577,"encode_duration_us":16868,"operation_id":"0x17e4","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10009237},{"operation_name":"upsert","last_dispatch_duration_us":12450505,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":13636246,"total_server_duration_us":554172,"encode_duration_us":17543,"operation_id":"0x1771","timeout_ms":10000,"last_server_duration_us":229637,"total_duration_us":10008555},{"operation_name":"upsert","last_dispatch_duration_us":1000021,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10444469,"total_server_duration_us":248713,"encode_duration_us":20773,"operation_id":"0x1788","timeout_ms":10000,"last_server_duration_us":135580,"total_duration_us":10008266},{"operation_name":"upsert","last_dispatch_duration_us":9270469,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10962751,"total_server_duration_us":9060203,"encode_duration_us":28620,"operation_id":"0x1718","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10007952},{"operation_name":"upsert","last_dispatch_duration_us":750369,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10329003,"total_server_duration_us":689062,"encode_duration_us":26491,"operation_id":"0x17a5","timeout_ms":10000,"last_server_duration_us":316643,"total_duration_us":10005655},{"operation_name":"upsert","last_dispatch_duration_us":10213250,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10548730,"total_server_duration_us":9657044,"encode_duration_us":16377,"operation_id":"0x1772","timeout_ms":10000,"last_server_duration_us":9632375,"total_duration_us":10004516},{"operation_name":"upsert","last_dispatch_duration_us":115188,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":1101084,"total_server_duration_us":172777,"encode_duration_us":17063,"operation_id":"0x176f","timeout_ms":10000,"last_server_duration_us":28493,"total_duration_us":10003417},{"operation_name":"upsert","last_dispatch_duration_us":9260143,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":14906389,"total_server_duration_us":715153,"encode_duration_us":16300,"operation_id":"0x1682","timeout_ms":10000,"last_server_duration_us":157639,"total_duration_us":10003205}],"total_count":16}} INFO [2022-01-10 05:38:11,435] jepsen worker 80 - jepsen.util 710 :info :write [26 2] :RequestTimeoutException INFO [2022-01-10 05:38:11,436] jepsen worker 80 - jepsen.util 800 :invoke :read [26 nil] INFO [2022-01-10 05:38:11,440] jepsen worker 39 - jepsen.util 759 :info :write [13 24] :RequestTimeoutException INFO [2022-01-10 05:38:11,441] jepsen worker 39 - jepsen.util 849 :invoke :write [13 44] INFO [2022-01-10 05:38:11,559] jepsen worker 76 - jepsen.util 706 :ok :read [25 :nil] INFO [2022-01-10 05:38:11,560] jepsen worker 76 - jepsen.util 706 :invoke :write [25 34] INFO [2022-01-10 05:38:11,587] jepsen worker 78 - jepsen.util 708 :ok :read [26 10] INFO [2022-01-10 05:38:11,587] jepsen worker 78 - jepsen.util 708 :invoke :read [26 nil] INFO [2022-01-10 05:38:11,635] jepsen worker 20 - jepsen.util 560 :info :write [6 13] :RequestTimeoutException INFO [2022-01-10 05:38:11,635] jepsen worker 20 - jepsen.util 650 :invoke :read [6 nil] INFO [2022-01-10 05:38:11,763] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05847953216374269}, :ns_1@172.23.107.227 {:progress 0.02346041055718473}, :ns_1@172.23.105.187 {:progress 0.02346041055718473}, :ns_1@172.23.107.228 {:progress 0.05882352941176472}} INFO [2022-01-10 05:38:11,763] jepsen worker nemesis - couchbase.util Rebalance check count: 49 INFO [2022-01-10 05:38:11,763] jepsen worker nemesis - couchbase.util Rebalance stuck count: 41 INFO [2022-01-10 05:38:11,805] jepsen worker 69 - jepsen.util 789 :info :write [23 30] :RequestTimeoutException INFO [2022-01-10 05:38:11,806] jepsen worker 69 - jepsen.util 879 :invoke :read [23 nil] INFO [2022-01-10 05:38:11,894] jepsen worker 56 - jepsen.util 866 :ok :read [18 14] INFO [2022-01-10 05:38:11,895] jepsen worker 56 - jepsen.util 866 :invoke :write [18 23] INFO [2022-01-10 05:38:11,903] jepsen worker 79 - jepsen.util 799 :ok :read [26 10] INFO [2022-01-10 05:38:11,904] jepsen worker 79 - jepsen.util 799 :invoke :read [26 nil] INFO [2022-01-10 05:38:11,997] jepsen worker 52 - jepsen.util 682 :info :write [17 29] :RequestTimeoutException INFO [2022-01-10 05:38:11,997] jepsen worker 52 - jepsen.util 772 :invoke :read [17 nil] INFO [2022-01-10 05:38:12,110] jepsen worker 19 - jepsen.util 829 :info :write [6 27] :RequestTimeoutException INFO [2022-01-10 05:38:12,111] jepsen worker 19 - jepsen.util 919 :invoke :read [6 nil] INFO [2022-01-10 05:38:12,256] jepsen worker 38 - jepsen.util 668 :info :write [12 17] :RequestTimeoutException INFO [2022-01-10 05:38:12,257] jepsen worker 38 - jepsen.util 758 :invoke :write [12 49] INFO [2022-01-10 05:38:12,364] jepsen worker 80 - jepsen.util 800 :ok :read [26 10] INFO [2022-01-10 05:38:12,365] jepsen worker 80 - jepsen.util 800 :invoke :read [26 nil] INFO [2022-01-10 05:38:12,372] jepsen worker 14 - jepsen.util 734 :info :write [4 24] :RequestTimeoutException INFO [2022-01-10 05:38:12,372] jepsen worker 14 - jepsen.util 824 :invoke :read [4 nil] INFO [2022-01-10 05:38:12,514] jepsen worker 59 - jepsen.util 869 :info :write [19 13] :RequestTimeoutException INFO [2022-01-10 05:38:12,515] jepsen worker 59 - jepsen.util 959 :invoke :read [19 nil] INFO [2022-01-10 05:38:12,769] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05847953216374269}, :ns_1@172.23.107.227 {:progress 0.02346041055718473}, :ns_1@172.23.105.187 {:progress 0.02346041055718473}, :ns_1@172.23.107.228 {:progress 0.05882352941176472}} INFO [2022-01-10 05:38:12,769] jepsen worker nemesis - couchbase.util Rebalance check count: 50 INFO [2022-01-10 05:38:12,769] jepsen worker nemesis - couchbase.util Rebalance stuck count: 42 INFO [2022-01-10 05:38:12,909] jepsen worker 78 - jepsen.util 708 :ok :read [26 10] INFO [2022-01-10 05:38:12,909] jepsen worker 78 - jepsen.util 708 :invoke :read [26 nil] INFO [2022-01-10 05:38:13,176] jepsen worker 26 - jepsen.util 836 :info :write [8 34] :RequestTimeoutException INFO [2022-01-10 05:38:13,177] jepsen worker 26 - jepsen.util 926 :invoke :read [8 nil] INFO [2022-01-10 05:38:13,181] jepsen worker 85 - jepsen.util 805 :info :write [28 0] :RequestTimeoutException INFO [2022-01-10 05:38:13,181] jepsen worker 85 - jepsen.util 895 :invoke :write [28 40] INFO [2022-01-10 05:38:13,516] jepsen worker 82 - jepsen.util 622 :info :write [27 29] :RequestTimeoutException INFO [2022-01-10 05:38:13,516] jepsen worker 82 - jepsen.util 712 :invoke :write [27 16] INFO [2022-01-10 05:38:13,535] jepsen worker 14 - jepsen.util 824 :ok :read [4 :nil] INFO [2022-01-10 05:38:13,536] jepsen worker 14 - jepsen.util 824 :invoke :write [4 19] INFO [2022-01-10 05:38:13,600] jepsen worker 10 - jepsen.util 730 :ok :read [3 2] INFO [2022-01-10 05:38:13,600] jepsen worker 10 - jepsen.util 730 :invoke :write [3 34] INFO [2022-01-10 05:38:13,620] jepsen worker 45 - jepsen.util 855 :info :write [15 7] :RequestTimeoutException INFO [2022-01-10 05:38:13,621] jepsen worker 45 - jepsen.util 945 :invoke :read [15 nil] INFO [2022-01-10 05:38:13,656] jepsen worker 24 - jepsen.util 744 :info :write [8 24] :RequestTimeoutException INFO [2022-01-10 05:38:13,656] jepsen worker 24 - jepsen.util 834 :invoke :read [8 nil] INFO [2022-01-10 05:38:13,682] jepsen worker 21 - jepsen.util 921 :info :write [7 48] :RequestTimeoutException INFO [2022-01-10 05:38:13,683] jepsen worker 21 - jepsen.util 1011 :invoke :write [7 38] INFO [2022-01-10 05:38:13,696] jepsen worker 80 - jepsen.util 800 :ok :read [26 10] INFO [2022-01-10 05:38:13,696] jepsen worker 80 - jepsen.util 800 :invoke :write [26 29] INFO [2022-01-10 05:38:13,775] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05847953216374269}, :ns_1@172.23.107.227 {:progress 0.02346041055718473}, :ns_1@172.23.105.187 {:progress 0.02346041055718473}, :ns_1@172.23.107.228 {:progress 0.05882352941176472}} INFO [2022-01-10 05:38:13,776] jepsen worker nemesis - couchbase.util Rebalance check count: 51 INFO [2022-01-10 05:38:13,776] jepsen worker nemesis - couchbase.util Rebalance stuck count: 43 INFO [2022-01-10 05:38:13,876] jepsen worker 83 - jepsen.util 893 :info :write [27 40] :RequestTimeoutException INFO [2022-01-10 05:38:13,877] jepsen worker 83 - jepsen.util 983 :invoke :read [27 nil] INFO [2022-01-10 05:38:13,980] jepsen worker 60 - jepsen.util 870 :info :write [20 49] :RequestTimeoutException INFO [2022-01-10 05:38:13,980] jepsen worker 60 - jepsen.util 960 :invoke :read [20 nil] INFO [2022-01-10 05:38:14,311] jepsen worker 16 - jepsen.util 736 :info :write [5 35] :RequestTimeoutException INFO [2022-01-10 05:38:14,312] jepsen worker 16 - jepsen.util 826 :invoke :write [5 44] INFO [2022-01-10 05:38:14,365] jepsen worker 23 - jepsen.util 743 :info :write [7 36] :RequestTimeoutException INFO [2022-01-10 05:38:14,366] jepsen worker 23 - jepsen.util 833 :invoke :read [7 nil] INFO [2022-01-10 05:38:14,782] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05847953216374269}, :ns_1@172.23.107.227 {:progress 0.02346041055718473}, :ns_1@172.23.105.187 {:progress 0.02346041055718473}, :ns_1@172.23.107.228 {:progress 0.05882352941176472}} INFO [2022-01-10 05:38:14,782] jepsen worker nemesis - couchbase.util Rebalance check count: 52 INFO [2022-01-10 05:38:14,782] jepsen worker nemesis - couchbase.util Rebalance stuck count: 44 INFO [2022-01-10 05:38:14,867] jepsen worker 73 - jepsen.util 703 :ok :read [24 18] INFO [2022-01-10 05:38:14,868] jepsen worker 73 - jepsen.util 703 :invoke :write [24 28] INFO [2022-01-10 05:38:14,896] jepsen worker 35 - jepsen.util 935 :info :write [11 22] :RequestTimeoutException INFO [2022-01-10 05:38:14,897] jepsen worker 35 - jepsen.util 1025 :invoke :read [11 nil] INFO [2022-01-10 05:38:14,913] jepsen worker 64 - jepsen.util 784 :ok :read [21 7] INFO [2022-01-10 05:38:14,914] jepsen worker 64 - jepsen.util 784 :invoke :read [21 nil] INFO [2022-01-10 05:38:14,930] jepsen worker 50 - jepsen.util 860 :info :write [16 31] :RequestTimeoutException INFO [2022-01-10 05:38:14,931] jepsen worker 50 - jepsen.util 950 :invoke :write [16 48] INFO [2022-01-10 05:38:15,132] jepsen worker 87 - jepsen.util 897 :info :write [29 20] :RequestTimeoutException INFO [2022-01-10 05:38:15,132] jepsen worker 87 - jepsen.util 987 :invoke :write [29 45] INFO [2022-01-10 05:38:15,615] jepsen worker 23 - jepsen.util 833 :ok :read [7 :nil] INFO [2022-01-10 05:38:15,615] jepsen worker 31 - jepsen.util 931 :info :write [10 13] :RequestTimeoutException INFO [2022-01-10 05:38:15,616] jepsen worker 23 - jepsen.util 833 :invoke :write [7 21] INFO [2022-01-10 05:38:15,616] jepsen worker 31 - jepsen.util 1021 :invoke :read [10 nil] INFO [2022-01-10 05:38:15,625] jepsen worker 45 - jepsen.util 945 :ok :read [15 2] INFO [2022-01-10 05:38:15,626] jepsen worker 45 - jepsen.util 945 :invoke :read [15 nil] INFO [2022-01-10 05:38:15,674] jepsen worker 42 - jepsen.util 762 :info :write [14 47] :RequestTimeoutException INFO [2022-01-10 05:38:15,675] jepsen worker 42 - jepsen.util 852 :invoke :read [14 nil] INFO [2022-01-10 05:38:15,791] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.05847953216374269}, :ns_1@172.23.107.227 {:progress 0.02346041055718473}, :ns_1@172.23.105.187 {:progress 0.02346041055718473}, :ns_1@172.23.107.228 {:progress 0.05882352941176472}} INFO [2022-01-10 05:38:15,791] jepsen worker nemesis - couchbase.util Rebalance check count: 53 INFO [2022-01-10 05:38:15,791] jepsen worker nemesis - couchbase.util Rebalance stuck count: 45 INFO [2022-01-10 05:38:15,876] jepsen worker 57 - jepsen.util 867 :info :write [19 26] :RequestTimeoutException INFO [2022-01-10 05:38:15,877] jepsen worker 57 - jepsen.util 957 :invoke :read [19 nil] INFO [2022-01-10 05:38:16,023] jepsen worker 64 - jepsen.util 784 :ok :read [21 7] INFO [2022-01-10 05:38:16,023] jepsen worker 64 - jepsen.util 784 :invoke :read [21 nil] INFO [2022-01-10 05:38:16,049] jepsen worker 88 - jepsen.util 898 :ok :read [29 33] INFO [2022-01-10 05:38:16,049] jepsen worker 88 - jepsen.util 898 :invoke :read [29 nil] INFO [2022-01-10 05:38:16,381] jepsen worker 9 - jepsen.util 909 :info :write [3 9] :RequestTimeoutException INFO [2022-01-10 05:38:16,382] jepsen worker 9 - jepsen.util 999 :invoke :write [3 44] INFO [2022-01-10 05:38:16,524] jepsen worker 17 - jepsen.util 647 :info :write [5 11] :RequestTimeoutException INFO [2022-01-10 05:38:16,525] jepsen worker 17 - jepsen.util 737 :invoke :read [5 nil] INFO [2022-01-10 05:38:16,603] jepsen worker 60 - jepsen.util 960 :ok :read [20 :nil] INFO [2022-01-10 05:38:16,603] jepsen worker 60 - jepsen.util 960 :invoke :write [20 4] INFO [2022-01-10 05:38:16,669] jepsen worker 71 - jepsen.util 791 :info :write [23 36] :RequestTimeoutException INFO [2022-01-10 05:38:16,670] jepsen worker 71 - jepsen.util 881 :invoke :read [23 nil] INFO [2022-01-10 05:38:16,693] jepsen worker 86 - jepsen.util 536 :ok :read [28 :nil] INFO [2022-01-10 05:38:16,694] jepsen worker 86 - jepsen.util 536 :invoke :write [28 44] INFO [2022-01-10 05:38:16,772] jepsen worker 6 - jepsen.util 636 :info :write [2 31] :RequestTimeoutException INFO [2022-01-10 05:38:16,773] jepsen worker 6 - jepsen.util 726 :invoke :read [2 nil] INFO [2022-01-10 05:38:16,791] jepsen worker 77 - jepsen.util 887 :info :write [25 21] :RequestTimeoutException INFO [2022-01-10 05:38:16,792] jepsen worker 77 - jepsen.util 977 :invoke :write [25 21] INFO [2022-01-10 05:38:16,797] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.06432748538011701}, :ns_1@172.23.107.227 {:progress 0.02932551319648091}, :ns_1@172.23.105.187 {:progress 0.02932551319648091}, :ns_1@172.23.107.228 {:progress 0.06470588235294117}} INFO [2022-01-10 05:38:16,797] jepsen worker nemesis - couchbase.util Rebalance check count: 54 INFO [2022-01-10 05:38:16,797] jepsen worker nemesis - couchbase.util Rebalance stuck count: 45 INFO [2022-01-10 05:38:17,041] jepsen worker 45 - jepsen.util 945 :ok :read [15 2] INFO [2022-01-10 05:38:17,042] jepsen worker 45 - jepsen.util 945 :invoke :write [15 24] INFO [2022-01-10 05:38:17,130] jepsen worker 28 - jepsen.util 838 :ok :read [9 22] INFO [2022-01-10 05:38:17,130] jepsen worker 28 - jepsen.util 838 :invoke :read [9 nil] INFO [2022-01-10 05:38:17,212] jepsen worker 72 - jepsen.util 972 :info :write [24 15] :RequestTimeoutException INFO [2022-01-10 05:38:17,213] jepsen worker 72 - jepsen.util 1062 :invoke :read [24 nil] INFO [2022-01-10 05:38:17,226] jepsen worker 7 - jepsen.util 637 :info :write [2 14] :RequestTimeoutException INFO [2022-01-10 05:38:17,227] jepsen worker 7 - jepsen.util 727 :invoke :write [2 0] INFO [2022-01-10 05:38:17,232] jepsen worker 88 - jepsen.util 898 :ok :read [29 33] INFO [2022-01-10 05:38:17,232] jepsen worker 88 - jepsen.util 898 :invoke :read [29 nil] INFO [2022-01-10 05:38:17,236] jepsen worker 37 - jepsen.util 757 :info :write [12 9] :RequestTimeoutException INFO [2022-01-10 05:38:17,237] jepsen worker 37 - jepsen.util 847 :invoke :write [12 7] INFO [2022-01-10 05:38:17,321] jepsen worker 70 - jepsen.util 970 :info :write [23 41] :RequestTimeoutException INFO [2022-01-10 05:38:17,322] jepsen worker 70 - jepsen.util 1060 :invoke :write [23 12] INFO [2022-01-10 05:38:17,390] jepsen worker 88 - jepsen.util 898 :ok :read [29 33] INFO [2022-01-10 05:38:17,391] jepsen worker 88 - jepsen.util 898 :invoke :read [29 nil] INFO [2022-01-10 05:38:17,460] jepsen worker 2 - jepsen.util 722 :info :write [0 39] :RequestTimeoutException INFO [2022-01-10 05:38:17,461] jepsen worker 2 - jepsen.util 812 :invoke :write [0 46] INFO [2022-01-10 05:38:17,492] jepsen worker 54 - jepsen.util 684 :info :write [18 37] :RequestTimeoutException INFO [2022-01-10 05:38:17,493] jepsen worker 54 - jepsen.util 774 :invoke :write [18 39] INFO [2022-01-10 05:38:17,593] jepsen worker 63 - jepsen.util 873 :ok :read [21 7] INFO [2022-01-10 05:38:17,593] jepsen worker 63 - jepsen.util 873 :invoke :read [21 nil] INFO [2022-01-10 05:38:17,653] jepsen worker 88 - jepsen.util 898 :ok :read [29 33] INFO [2022-01-10 05:38:17,654] jepsen worker 88 - jepsen.util 898 :invoke :write [29 39] INFO [2022-01-10 05:38:17,692] jepsen worker 51 - jepsen.util 681 :info :write [17 36] :RequestTimeoutException INFO [2022-01-10 05:38:17,693] jepsen worker 51 - jepsen.util 771 :invoke :write [17 28] INFO [2022-01-10 05:38:17,701] jepsen worker 3 - jepsen.util 813 :info :write [1 19] :RequestTimeoutException INFO [2022-01-10 05:38:17,702] jepsen worker 3 - jepsen.util 903 :invoke :write [1 47] INFO [2022-01-10 05:38:17,803] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.06432748538011701}, :ns_1@172.23.107.227 {:progress 0.02932551319648091}, :ns_1@172.23.105.187 {:progress 0.02932551319648091}, :ns_1@172.23.107.228 {:progress 0.06470588235294117}} INFO [2022-01-10 05:38:17,803] jepsen worker nemesis - couchbase.util Rebalance check count: 55 INFO [2022-01-10 05:38:17,803] jepsen worker nemesis - couchbase.util Rebalance stuck count: 46 INFO [2022-01-10 05:38:17,861] jepsen worker 58 - jepsen.util 778 :info :write [19 41] :RequestTimeoutException INFO [2022-01-10 05:38:17,862] jepsen worker 58 - jepsen.util 868 :invoke :read [19 nil] INFO [2022-01-10 05:38:17,876] jepsen worker 5 - jepsen.util 635 :info :write [1 17] :RequestTimeoutException INFO [2022-01-10 05:38:17,876] jepsen worker 5 - jepsen.util 725 :invoke :write [1 19] INFO [2022-01-10 05:38:17,886] jepsen worker 55 - jepsen.util 775 :info :write [18 35] :RequestTimeoutException INFO [2022-01-10 05:38:17,887] jepsen worker 55 - jepsen.util 865 :invoke :read [18 nil] INFO [2022-01-10 05:38:18,056] jepsen worker 32 - jepsen.util 752 :info :write [10 42] :RequestTimeoutException INFO [2022-01-10 05:38:18,057] jepsen worker 32 - jepsen.util 842 :invoke :read [10 nil] INFO [2022-01-10 05:38:18,201] jepsen worker 1 - jepsen.util 721 :info :write [0 40] :RequestTimeoutException INFO [2022-01-10 05:38:18,202] jepsen worker 1 - jepsen.util 811 :invoke :read [0 nil] INFO [2022-01-10 05:38:18,260] jepsen worker 4 - jepsen.util 724 :info :write [1 9] :RequestTimeoutException INFO [2022-01-10 05:38:18,260] jepsen worker 4 - jepsen.util 814 :invoke :write [1 17] INFO [2022-01-10 05:38:18,491] jepsen worker 65 - jepsen.util 785 :info :write [21 3] :RequestTimeoutException INFO [2022-01-10 05:38:18,492] jepsen worker 65 - jepsen.util 875 :invoke :write [21 29] INFO [2022-01-10 05:38:18,809] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.06432748538011701}, :ns_1@172.23.107.227 {:progress 0.02932551319648091}, :ns_1@172.23.105.187 {:progress 0.02932551319648091}, :ns_1@172.23.107.228 {:progress 0.06470588235294117}} INFO [2022-01-10 05:38:18,809] jepsen worker nemesis - couchbase.util Rebalance check count: 56 INFO [2022-01-10 05:38:18,809] jepsen worker nemesis - couchbase.util Rebalance stuck count: 47 INFO [2022-01-10 05:38:18,817] jepsen worker 30 - jepsen.util 480 :info :write [10 42] :RequestTimeoutException INFO [2022-01-10 05:38:18,817] jepsen worker 30 - jepsen.util 570 :invoke :read [10 nil] INFO [2022-01-10 05:38:18,819] jepsen worker 6 - jepsen.util 726 :ok :read [2 21] INFO [2022-01-10 05:38:18,820] jepsen worker 6 - jepsen.util 726 :invoke :read [2 nil] INFO [2022-01-10 05:38:18,935] jepsen worker 74 - jepsen.util 974 :info :write [24 45] :RequestTimeoutException INFO [2022-01-10 05:38:18,936] jepsen worker 74 - jepsen.util 1064 :invoke :write [24 12] INFO [2022-01-10 05:38:19,120] jepsen worker 12 - jepsen.util 732 :info :write [4 12] :RequestTimeoutException INFO [2022-01-10 05:38:19,121] jepsen worker 12 - jepsen.util 822 :invoke :write [4 5] INFO [2022-01-10 05:38:19,192] jepsen worker 29 - jepsen.util 659 :info :write [9 6] :RequestTimeoutException INFO [2022-01-10 05:38:19,193] jepsen worker 29 - jepsen.util 749 :invoke :write [9 14] INFO [2022-01-10 05:38:19,201] jepsen worker 40 - jepsen.util 580 :info :write [13 34] :RequestTimeoutException INFO [2022-01-10 05:38:19,202] jepsen worker 40 - jepsen.util 670 :invoke :read [13 nil] INFO [2022-01-10 05:38:19,472] jepsen worker 33 - jepsen.util 753 :info :write [11 27] :RequestTimeoutException INFO [2022-01-10 05:38:19,473] jepsen worker 33 - jepsen.util 843 :invoke :write [11 35] INFO [2022-01-10 05:38:19,497] jepsen worker 44 - jepsen.util 854 :info :write [14 4] :RequestTimeoutException INFO [2022-01-10 05:38:19,497] jepsen worker 44 - jepsen.util 944 :invoke :read [14 nil] INFO [2022-01-10 05:38:19,560] jepsen worker 0 - jepsen.util 720 :ok :read [0 :nil] INFO [2022-01-10 05:38:19,561] jepsen worker 0 - jepsen.util 720 :invoke :write [0 48] WARN [2022-01-10 05:38:19,585] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9852901,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14631403,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9649727,"encode_duration_us":17758,"operation_id":"0x1806","timeout_ms":10000,"last_server_duration_us":9188613,"total_duration_us":10006696}],"total_count":1}} WARN [2022-01-10 05:38:19,685] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":8129614,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8129614,"total_server_duration_us":38,"operation_id":"0x1892","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":13808513},{"operation_name":"upsert","last_dispatch_duration_us":8614643,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":11573156,"total_server_duration_us":1094178,"encode_duration_us":16275,"operation_id":"0x18ed","timeout_ms":10000,"last_server_duration_us":525935,"total_duration_us":10011010},{"operation_name":"upsert","last_dispatch_duration_us":8069177,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":13725667,"total_server_duration_us":1339038,"encode_duration_us":16986,"operation_id":"0x1836","timeout_ms":10000,"last_server_duration_us":170369,"total_duration_us":10010831},{"operation_name":"upsert","last_dispatch_duration_us":7994253,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":15085560,"total_server_duration_us":1380384,"encode_duration_us":16915,"operation_id":"0x1802","timeout_ms":10000,"last_server_duration_us":424454,"total_duration_us":10009147},{"operation_name":"upsert","last_dispatch_duration_us":13638797,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13638797,"total_server_duration_us":21405,"encode_duration_us":16714,"operation_id":"0x17fa","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10007814},{"operation_name":"upsert","last_dispatch_duration_us":9852901,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":14631403,"total_server_duration_us":9649727,"encode_duration_us":17758,"operation_id":"0x1806","timeout_ms":10000,"last_server_duration_us":9188613,"total_duration_us":10006696},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","encode_duration_us":16340,"operation_id":"0x18ee","timeout_ms":10000,"total_duration_us":10006476},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","encode_duration_us":16949,"operation_id":"0x19ad","timeout_ms":10000,"total_duration_us":10005361},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","encode_duration_us":16244,"operation_id":"0x18c2","timeout_ms":10000,"total_duration_us":10004990},{"operation_name":"upsert","encode_duration_us":23077,"operation_id":"0x1839","timeout_ms":10000,"total_duration_us":10004758}],"total_count":16}} INFO [2022-01-10 05:38:19,819] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.06432748538011701}, :ns_1@172.23.107.227 {:progress 0.02932551319648091}, :ns_1@172.23.105.187 {:progress 0.02932551319648091}, :ns_1@172.23.107.228 {:progress 0.06470588235294117}} INFO [2022-01-10 05:38:19,819] jepsen worker nemesis - couchbase.util Rebalance check count: 57 INFO [2022-01-10 05:38:19,819] jepsen worker nemesis - couchbase.util Rebalance stuck count: 48 INFO [2022-01-10 05:38:20,044] jepsen worker 49 - jepsen.util 679 :info :write [16 42] :RequestTimeoutException INFO [2022-01-10 05:38:20,045] jepsen worker 49 - jepsen.util 769 :invoke :read [16 nil] INFO [2022-01-10 05:38:20,392] jepsen worker 46 - jepsen.util 766 :info :write [15 23] :RequestTimeoutException INFO [2022-01-10 05:38:20,392] jepsen worker 46 - jepsen.util 856 :invoke :write [15 27] INFO [2022-01-10 05:38:20,435] jepsen worker 36 - jepsen.util 486 :ok :read [12 :nil] INFO [2022-01-10 05:38:20,436] jepsen worker 36 - jepsen.util 486 :invoke :write [12 35] INFO [2022-01-10 05:38:20,448] jepsen worker 53 - jepsen.util 683 :ok :read [17 41] INFO [2022-01-10 05:38:20,449] jepsen worker 53 - jepsen.util 683 :invoke :write [17 36] INFO [2022-01-10 05:38:20,840] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.06432748538011701}, :ns_1@172.23.107.227 {:progress 0.02932551319648091}, :ns_1@172.23.105.187 {:progress 0.02932551319648091}, :ns_1@172.23.107.228 {:progress 0.06470588235294117}} INFO [2022-01-10 05:38:20,840] jepsen worker nemesis - couchbase.util Rebalance check count: 58 INFO [2022-01-10 05:38:20,844] jepsen worker nemesis - couchbase.util Rebalance stuck count: 49 INFO [2022-01-10 05:38:20,856] jepsen worker 11 - jepsen.util 641 :info :write [3 43] :RequestTimeoutException INFO [2022-01-10 05:38:20,856] jepsen worker 11 - jepsen.util 731 :invoke :read [3 nil] WARN [2022-01-10 05:38:21,139] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":16273,"operation_id":"0x18c1","timeout_ms":10000,"total_duration_us":10008650},{"operation_name":"upsert","last_dispatch_duration_us":4453004,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4453004,"total_server_duration_us":344182,"encode_duration_us":16673,"operation_id":"0x1897","timeout_ms":10000,"last_server_duration_us":344182,"total_duration_us":10008262},{"operation_name":"upsert","last_dispatch_duration_us":629583,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":2340294,"total_server_duration_us":809414,"encode_duration_us":16030,"operation_id":"0x18e6","timeout_ms":10000,"last_server_duration_us":100127,"total_duration_us":10007762},{"operation_name":"upsert","last_dispatch_duration_us":1649420,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":1649420,"total_server_duration_us":358855,"encode_duration_us":31305,"operation_id":"0x1984","timeout_ms":10000,"last_server_duration_us":358855,"total_duration_us":10005908},{"operation_name":"upsert","last_dispatch_duration_us":908001,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":2734364,"total_server_duration_us":439391,"encode_duration_us":18159,"operation_id":"0x1917","timeout_ms":10000,"last_server_duration_us":141143,"total_duration_us":10005190},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":17363,"operation_id":"0x1934","timeout_ms":10000,"total_duration_us":10002037},{"operation_name":"upsert","last_dispatch_duration_us":1811202,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":1811202,"total_server_duration_us":302990,"encode_duration_us":16587,"operation_id":"0x1837","timeout_ms":10000,"last_server_duration_us":302990,"total_duration_us":10001513},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":17166,"operation_id":"0x1891","timeout_ms":10000,"total_duration_us":10001175},{"operation_name":"upsert","last_dispatch_duration_us":921681,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":3556925,"total_server_duration_us":1146479,"encode_duration_us":16789,"operation_id":"0x194b","timeout_ms":10000,"last_server_duration_us":373787,"total_duration_us":10000613},{"operation_name":"get","last_dispatch_duration_us":1163076,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":1163076,"total_server_duration_us":23,"operation_id":"0x1967","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":5824848}],"total_count":11}} WARN [2022-01-10 05:38:21,139] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":21102314,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":21412195,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10629700,"encode_duration_us":23695,"operation_id":"0x173a","timeout_ms":10000,"last_server_duration_us":10553249,"total_duration_us":10003644},{"operation_name":"upsert","last_dispatch_duration_us":9486362,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12588851,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":10432848,"encode_duration_us":18284,"operation_id":"0x1774","timeout_ms":10000,"last_server_duration_us":9408841,"total_duration_us":10010069}],"total_count":2}} WARN [2022-01-10 05:38:21,234] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":21855265,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":21855265,"total_server_duration_us":9686,"operation_id":"0x170a","timeout_ms":30000,"last_server_duration_us":9686,"total_duration_us":21855676},{"operation_name":"upsert","last_dispatch_duration_us":902616,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8678631,"total_server_duration_us":717788,"encode_duration_us":18297,"operation_id":"0x1865","timeout_ms":10000,"last_server_duration_us":281778,"total_duration_us":10009919},{"operation_name":"upsert","last_dispatch_duration_us":14926627,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":14926627,"total_server_duration_us":413244,"encode_duration_us":18691,"operation_id":"0x190d","timeout_ms":10000,"last_server_duration_us":413244,"total_duration_us":10009115},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":16498,"operation_id":"0x18e1","timeout_ms":10000,"total_duration_us":10008289},{"operation_name":"upsert","last_dispatch_duration_us":14402863,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":14402863,"total_server_duration_us":202112,"encode_duration_us":17000,"operation_id":"0x1893","timeout_ms":10000,"last_server_duration_us":202112,"total_duration_us":10007147},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":18888,"operation_id":"0x1944","timeout_ms":10000,"total_duration_us":10006296},{"operation_name":"upsert","last_dispatch_duration_us":2109239,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8899200,"total_server_duration_us":478878,"encode_duration_us":16427,"operation_id":"0x1859","timeout_ms":10000,"last_server_duration_us":233211,"total_duration_us":10004296},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","encode_duration_us":19167,"operation_id":"0x1a09","timeout_ms":10000,"total_duration_us":10003708},{"operation_name":"upsert","last_dispatch_duration_us":2053699,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":5796877,"total_server_duration_us":589677,"encode_duration_us":16543,"operation_id":"0x1852","timeout_ms":10000,"last_server_duration_us":162219,"total_duration_us":10001900},{"operation_name":"upsert","last_dispatch_duration_us":15874179,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":15874179,"total_server_duration_us":197914,"encode_duration_us":18477,"operation_id":"0x18b7","timeout_ms":10000,"last_server_duration_us":197914,"total_duration_us":10001090}],"total_count":13}} WARN [2022-01-10 05:38:21,234] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17328637,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17328637,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":17301368,"encode_duration_us":16750,"operation_id":"0x17d5","timeout_ms":10000,"last_server_duration_us":17301368,"total_duration_us":10002600},{"operation_name":"upsert","last_dispatch_duration_us":11566016,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13720814,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":11735318,"encode_duration_us":17283,"operation_id":"0x191a","timeout_ms":10000,"last_server_duration_us":11277729,"total_duration_us":10009395}],"total_count":2}} WARN [2022-01-10 05:38:21,234] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11766574,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11766574,"total_server_duration_us":682889,"operation_id":"0x1889","timeout_ms":30000,"last_server_duration_us":682889,"total_duration_us":11769025},{"operation_name":"get","last_dispatch_duration_us":11605699,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11605699,"total_server_duration_us":32,"operation_id":"0x17f8","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":11606047},{"operation_name":"upsert","last_dispatch_duration_us":1219663,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6900619,"total_server_duration_us":650722,"encode_duration_us":18301,"operation_id":"0x19bb","timeout_ms":10000,"last_server_duration_us":386197,"total_duration_us":10011152},{"operation_name":"upsert","last_dispatch_duration_us":1723960,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":6385059,"total_server_duration_us":1029477,"encode_duration_us":26869,"operation_id":"0x1946","timeout_ms":10000,"last_server_duration_us":91382,"total_duration_us":10008041},{"operation_name":"upsert","last_dispatch_duration_us":10224902,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10224902,"total_server_duration_us":671606,"encode_duration_us":16489,"operation_id":"0x1890","timeout_ms":10000,"last_server_duration_us":671606,"total_duration_us":10007070},{"operation_name":"upsert","last_dispatch_duration_us":1251308,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":3838544,"total_server_duration_us":566354,"encode_duration_us":17258,"operation_id":"0x19ac","timeout_ms":10000,"last_server_duration_us":400469,"total_duration_us":10006183},{"operation_name":"upsert","last_dispatch_duration_us":1298581,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":4788430,"total_server_duration_us":394949,"encode_duration_us":16505,"operation_id":"0x197b","timeout_ms":10000,"last_server_duration_us":98258,"total_duration_us":10005750},{"operation_name":"upsert","last_dispatch_duration_us":11226588,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11226588,"total_server_duration_us":18953,"encode_duration_us":17048,"operation_id":"0x1896","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10005497},{"operation_name":"upsert","last_dispatch_duration_us":1812209,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":5650136,"total_server_duration_us":837009,"encode_duration_us":18049,"operation_id":"0x1977","timeout_ms":10000,"last_server_duration_us":438718,"total_duration_us":10004452},{"operation_name":"upsert","last_dispatch_duration_us":1464134,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":5216783,"total_server_duration_us":2162878,"encode_duration_us":17649,"operation_id":"0x1976","timeout_ms":10000,"last_server_duration_us":221245,"total_duration_us":10002347}],"total_count":20}} WARN [2022-01-10 05:38:21,235] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10391019,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10391019,"total_server_duration_us":2513,"operation_id":"0x19b6","timeout_ms":30000,"last_server_duration_us":2513,"total_duration_us":10391935},{"operation_name":"upsert","last_dispatch_duration_us":1045586,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":7187750,"total_server_duration_us":1234076,"encode_duration_us":18898,"operation_id":"0x194f","timeout_ms":10000,"last_server_duration_us":230305,"total_duration_us":10010100},{"operation_name":"upsert","last_dispatch_duration_us":886827,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":7638803,"total_server_duration_us":1348035,"encode_duration_us":16242,"operation_id":"0x1953","timeout_ms":10000,"last_server_duration_us":270702,"total_duration_us":10008916},{"operation_name":"upsert","encode_duration_us":16309,"operation_id":"0x19ba","timeout_ms":10000,"total_duration_us":10008794},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":19000,"operation_id":"0x1942","timeout_ms":10000,"total_duration_us":10006811},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16146,"operation_id":"0x19e6","timeout_ms":10000,"total_duration_us":10006707},{"operation_name":"upsert","last_dispatch_duration_us":11697036,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11697036,"total_server_duration_us":10830267,"encode_duration_us":23150,"operation_id":"0x1975","timeout_ms":10000,"last_server_duration_us":10830267,"total_duration_us":10006417},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16766,"operation_id":"0x192d","timeout_ms":10000,"total_duration_us":10006243},{"operation_name":"upsert","last_dispatch_duration_us":1611115,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":6528964,"total_server_duration_us":2331292,"encode_duration_us":17731,"operation_id":"0x1952","timeout_ms":10000,"last_server_duration_us":190452,"total_duration_us":10005451},{"operation_name":"upsert","encode_duration_us":16923,"operation_id":"0x1989","timeout_ms":10000,"total_duration_us":10005269}],"total_count":13}} INFO [2022-01-10 05:38:21,266] jepsen worker 89 - jepsen.util 899 :info :write [29 39] :RequestTimeoutException INFO [2022-01-10 05:38:21,267] jepsen worker 89 - jepsen.util 989 :invoke :read [29 nil] INFO [2022-01-10 05:38:21,290] jepsen worker 62 - jepsen.util 872 :info :write [20 37] :RequestTimeoutException INFO [2022-01-10 05:38:21,290] jepsen worker 62 - jepsen.util 962 :invoke :read [20 nil] WARN [2022-01-10 05:38:21,334] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12427155,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12427155,"total_server_duration_us":32,"operation_id":"0x17c8","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":12427589},{"operation_name":"get","last_dispatch_duration_us":10080304,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10080304,"total_server_duration_us":7962,"operation_id":"0x19e4","timeout_ms":30000,"last_server_duration_us":7962,"total_duration_us":10082165},{"operation_name":"upsert","last_dispatch_duration_us":9776522,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":12720353,"total_server_duration_us":735350,"encode_duration_us":17548,"operation_id":"0x1986","timeout_ms":10000,"last_server_duration_us":182506,"total_duration_us":10009840},{"operation_name":"upsert","last_dispatch_duration_us":4655075,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4655075,"total_server_duration_us":388978,"encode_duration_us":16184,"operation_id":"0x1894","timeout_ms":10000,"last_server_duration_us":388978,"total_duration_us":10009780},{"operation_name":"upsert","last_dispatch_duration_us":11566016,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":13720814,"total_server_duration_us":11735318,"encode_duration_us":17283,"operation_id":"0x191a","timeout_ms":10000,"last_server_duration_us":11277729,"total_duration_us":10009395},{"operation_name":"upsert","last_dispatch_duration_us":15449240,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":15449240,"total_server_duration_us":215363,"encode_duration_us":20211,"operation_id":"0x18e2","timeout_ms":10000,"last_server_duration_us":215363,"total_duration_us":10006265},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":16801,"operation_id":"0x194e","timeout_ms":10000,"total_duration_us":10005393},{"operation_name":"upsert","last_dispatch_duration_us":4002918,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4002918,"total_server_duration_us":282021,"encode_duration_us":16930,"operation_id":"0x18b8","timeout_ms":10000,"last_server_duration_us":282021,"total_duration_us":10005108},{"operation_name":"upsert","last_dispatch_duration_us":9721491,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":12163673,"total_server_duration_us":423054,"encode_duration_us":16379,"operation_id":"0x19af","timeout_ms":10000,"last_server_duration_us":93802,"total_duration_us":10002239},{"operation_name":"upsert","last_dispatch_duration_us":4738896,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4738896,"total_server_duration_us":587223,"encode_duration_us":16862,"operation_id":"0x185a","timeout_ms":10000,"last_server_duration_us":587223,"total_duration_us":10001574}],"total_count":13}} WARN [2022-01-10 05:38:21,334] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":19062539,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19062539,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":18738764,"encode_duration_us":19080,"operation_id":"0x1773","timeout_ms":10000,"last_server_duration_us":18738764,"total_duration_us":10003867},{"operation_name":"upsert","last_dispatch_duration_us":17604800,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17604800,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":11233451,"encode_duration_us":16973,"operation_id":"0x16da","timeout_ms":10000,"last_server_duration_us":11233451,"total_duration_us":10006447}],"total_count":2}} WARN [2022-01-10 05:38:21,335] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11697036,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11697036,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10830267,"encode_duration_us":23150,"operation_id":"0x1975","timeout_ms":10000,"last_server_duration_us":10830267,"total_duration_us":10006417}],"total_count":1}} INFO [2022-01-10 05:38:21,419] jepsen worker 52 - jepsen.util 772 :ok :read [17 41] INFO [2022-01-10 05:38:21,420] jepsen worker 52 - jepsen.util 772 :invoke :read [17 nil] INFO [2022-01-10 05:38:21,442] jepsen worker 13 - jepsen.util 733 :info :write [4 26] :RequestTimeoutException INFO [2022-01-10 05:38:21,443] jepsen worker 13 - jepsen.util 823 :invoke :read [4 nil] INFO [2022-01-10 05:38:21,461] jepsen worker 39 - jepsen.util 849 :info :write [13 44] :RequestTimeoutException INFO [2022-01-10 05:38:21,461] jepsen worker 39 - jepsen.util 939 :invoke :write [13 44] INFO [2022-01-10 05:38:21,525] jepsen worker 31 - jepsen.util 1021 :ok :read [10 37] INFO [2022-01-10 05:38:21,525] jepsen worker 31 - jepsen.util 1021 :invoke :write [10 31] INFO [2022-01-10 05:38:21,533] jepsen worker 55 - jepsen.util 865 :ok :read [18 14] INFO [2022-01-10 05:38:21,533] jepsen worker 55 - jepsen.util 865 :invoke :read [18 nil] INFO [2022-01-10 05:38:21,587] jepsen worker 76 - jepsen.util 706 :info :write [25 34] :RequestTimeoutException INFO [2022-01-10 05:38:21,588] jepsen worker 76 - jepsen.util 796 :invoke :read [25 nil] INFO [2022-01-10 05:38:21,591] jepsen worker 69 - jepsen.util 879 :ok :read [23 :nil] INFO [2022-01-10 05:38:21,592] jepsen worker 69 - jepsen.util 879 :invoke :read [23 nil] INFO [2022-01-10 05:38:21,605] jepsen worker 79 - jepsen.util 799 :ok :read [26 10] INFO [2022-01-10 05:38:21,605] jepsen worker 79 - jepsen.util 799 :invoke :write [26 17] INFO [2022-01-10 05:38:21,851] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.06432748538011701}, :ns_1@172.23.107.227 {:progress 0.02932551319648091}, :ns_1@172.23.105.187 {:progress 0.02932551319648091}, :ns_1@172.23.107.228 {:progress 0.06470588235294117}} INFO [2022-01-10 05:38:21,851] jepsen worker nemesis - couchbase.util Rebalance check count: 59 INFO [2022-01-10 05:38:21,851] jepsen worker nemesis - couchbase.util Rebalance stuck count: 50 INFO [2022-01-10 05:38:21,915] jepsen worker 56 - jepsen.util 866 :info :write [18 23] :RequestTimeoutException INFO [2022-01-10 05:38:21,916] jepsen worker 56 - jepsen.util 956 :invoke :read [18 nil] INFO [2022-01-10 05:38:22,139] jepsen worker 52 - jepsen.util 772 :ok :read [17 41] INFO [2022-01-10 05:38:22,140] jepsen worker 52 - jepsen.util 772 :invoke :write [17 0] INFO [2022-01-10 05:38:22,147] jepsen worker 55 - jepsen.util 865 :ok :read [18 14] INFO [2022-01-10 05:38:22,151] jepsen worker 55 - jepsen.util 865 :invoke :read [18 nil] INFO [2022-01-10 05:38:22,286] jepsen worker 38 - jepsen.util 758 :info :write [12 49] :RequestTimeoutException INFO [2022-01-10 05:38:22,287] jepsen worker 38 - jepsen.util 848 :invoke :write [12 43] INFO [2022-01-10 05:38:22,342] jepsen worker 89 - jepsen.util 989 :ok :read [29 33] INFO [2022-01-10 05:38:22,343] jepsen worker 89 - jepsen.util 989 :invoke :read [29 nil] INFO [2022-01-10 05:38:22,561] jepsen worker 41 - jepsen.util 671 :ok :read [13 23] INFO [2022-01-10 05:38:22,562] jepsen worker 41 - jepsen.util 671 :invoke :read [13 nil] INFO [2022-01-10 05:38:22,563] jepsen worker 59 - jepsen.util 959 :ok :read [19 31] INFO [2022-01-10 05:38:22,564] jepsen worker 59 - jepsen.util 959 :invoke :write [19 26] INFO [2022-01-10 05:38:22,857] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.06432748538011701}, :ns_1@172.23.107.227 {:progress 0.02932551319648091}, :ns_1@172.23.105.187 {:progress 0.02932551319648091}, :ns_1@172.23.107.228 {:progress 0.06470588235294117}} INFO [2022-01-10 05:38:22,857] jepsen worker nemesis - couchbase.util Rebalance check count: 60 INFO [2022-01-10 05:38:22,857] jepsen worker nemesis - couchbase.util Rebalance stuck count: 51 INFO [2022-01-10 05:38:23,027] jepsen worker 25 - jepsen.util 745 :ok :read [8 :nil] INFO [2022-01-10 05:38:23,028] jepsen worker 25 - jepsen.util 745 :invoke :write [8 31] INFO [2022-01-10 05:38:23,041] jepsen worker 8 - jepsen.util 728 :ok :read [2 21] INFO [2022-01-10 05:38:23,041] jepsen worker 8 - jepsen.util 728 :invoke :read [2 nil] INFO [2022-01-10 05:38:23,200] jepsen worker 85 - jepsen.util 895 :info :write [28 40] :RequestTimeoutException INFO [2022-01-10 05:38:23,201] jepsen worker 85 - jepsen.util 985 :invoke :read [28 nil] INFO [2022-01-10 05:38:23,241] jepsen worker 64 - jepsen.util 784 :ok :read [21 7] INFO [2022-01-10 05:38:23,242] jepsen worker 64 - jepsen.util 784 :invoke :write [21 3] INFO [2022-01-10 05:38:23,380] jepsen worker 71 - jepsen.util 881 :ok :read [23 :nil] INFO [2022-01-10 05:38:23,381] jepsen worker 71 - jepsen.util 881 :invoke :read [23 nil] INFO [2022-01-10 05:38:23,542] jepsen worker 82 - jepsen.util 712 :info :write [27 16] :RequestTimeoutException INFO [2022-01-10 05:38:23,543] jepsen worker 82 - jepsen.util 802 :invoke :read [27 nil] INFO [2022-01-10 05:38:23,561] jepsen worker 14 - jepsen.util 824 :info :write [4 19] :RequestTimeoutException INFO [2022-01-10 05:38:23,561] jepsen worker 14 - jepsen.util 914 :invoke :read [4 nil] INFO [2022-01-10 05:38:23,621] jepsen worker 10 - jepsen.util 730 :info :write [3 34] :RequestTimeoutException INFO [2022-01-10 05:38:23,622] jepsen worker 10 - jepsen.util 820 :invoke :write [3 13] INFO [2022-01-10 05:38:23,705] jepsen worker 21 - jepsen.util 1011 :info :write [7 38] :RequestTimeoutException INFO [2022-01-10 05:38:23,706] jepsen worker 21 - jepsen.util 1101 :invoke :read [7 nil] INFO [2022-01-10 05:38:23,716] jepsen worker 80 - jepsen.util 800 :info :write [26 29] :RequestTimeoutException INFO [2022-01-10 05:38:23,717] jepsen worker 80 - jepsen.util 890 :invoke :read [26 nil] INFO [2022-01-10 05:38:23,864] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.06432748538011701}, :ns_1@172.23.107.227 {:progress 0.02932551319648091}, :ns_1@172.23.105.187 {:progress 0.02932551319648091}, :ns_1@172.23.107.228 {:progress 0.06470588235294117}} INFO [2022-01-10 05:38:23,864] jepsen worker nemesis - couchbase.util Rebalance check count: 61 INFO [2022-01-10 05:38:23,864] jepsen worker nemesis - couchbase.util Rebalance stuck count: 52 INFO [2022-01-10 05:38:24,081] jepsen worker 69 - jepsen.util 879 :ok :read [23 :nil] INFO [2022-01-10 05:38:24,081] jepsen worker 13 - jepsen.util 823 :ok :read [4 :nil] INFO [2022-01-10 05:38:24,082] jepsen worker 13 - jepsen.util 823 :invoke :write [4 4] INFO [2022-01-10 05:38:24,082] jepsen worker 69 - jepsen.util 879 :invoke :write [23 14] INFO [2022-01-10 05:38:24,100] jepsen worker 30 - jepsen.util 570 :ok :read [10 37] INFO [2022-01-10 05:38:24,100] jepsen worker 30 - jepsen.util 570 :invoke :read [10 nil] INFO [2022-01-10 05:38:24,207] jepsen worker 55 - jepsen.util 865 :ok :read [18 14] INFO [2022-01-10 05:38:24,208] jepsen worker 55 - jepsen.util 865 :invoke :read [18 nil] INFO [2022-01-10 05:38:24,332] jepsen worker 16 - jepsen.util 826 :info :write [5 44] :RequestTimeoutException INFO [2022-01-10 05:38:24,333] jepsen worker 16 - jepsen.util 916 :invoke :read [5 nil] INFO [2022-01-10 05:38:24,376] jepsen worker 71 - jepsen.util 881 :ok :read [23 :nil] INFO [2022-01-10 05:38:24,377] jepsen worker 71 - jepsen.util 881 :invoke :write [23 32] INFO [2022-01-10 05:38:24,830] jepsen worker 67 - jepsen.util 787 :ok :read [22 41] INFO [2022-01-10 05:38:24,831] jepsen worker 67 - jepsen.util 787 :invoke :write [22 6] INFO [2022-01-10 05:38:24,845] jepsen worker 83 - jepsen.util 983 :ok :read [27 30] INFO [2022-01-10 05:38:24,846] jepsen worker 83 - jepsen.util 983 :invoke :read [27 nil] INFO [2022-01-10 05:38:24,871] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.06432748538011701}, :ns_1@172.23.107.227 {:progress 0.02932551319648091}, :ns_1@172.23.105.187 {:progress 0.02932551319648091}, :ns_1@172.23.107.228 {:progress 0.06470588235294117}} INFO [2022-01-10 05:38:24,871] jepsen worker nemesis - couchbase.util Rebalance check count: 62 INFO [2022-01-10 05:38:24,871] jepsen worker nemesis - couchbase.util Rebalance stuck count: 53 INFO [2022-01-10 05:38:24,887] jepsen worker 73 - jepsen.util 703 :info :write [24 28] :RequestTimeoutException INFO [2022-01-10 05:38:24,888] jepsen worker 73 - jepsen.util 793 :invoke :read [24 nil] INFO [2022-01-10 05:38:24,898] jepsen worker 18 - jepsen.util 738 :ok :read [6 25] INFO [2022-01-10 05:38:24,899] jepsen worker 18 - jepsen.util 738 :invoke :read [6 nil] INFO [2022-01-10 05:38:24,954] jepsen worker 50 - jepsen.util 950 :info :write [16 48] :RequestTimeoutException INFO [2022-01-10 05:38:24,955] jepsen worker 50 - jepsen.util 1040 :invoke :read [16 nil] INFO [2022-01-10 05:38:24,964] jepsen worker 32 - jepsen.util 842 :ok :read [10 37] INFO [2022-01-10 05:38:24,965] jepsen worker 32 - jepsen.util 842 :invoke :read [10 nil] INFO [2022-01-10 05:38:25,089] jepsen worker 72 - jepsen.util 1062 :ok :read [24 18] INFO [2022-01-10 05:38:25,090] jepsen worker 72 - jepsen.util 1062 :invoke :write [24 10] INFO [2022-01-10 05:38:25,156] jepsen worker 87 - jepsen.util 987 :info :write [29 45] :RequestTimeoutException INFO [2022-01-10 05:38:25,156] jepsen worker 87 - jepsen.util 1077 :invoke :write [29 14] INFO [2022-01-10 05:38:25,268] jepsen worker 22 - jepsen.util 652 :ok :read [7 :nil] INFO [2022-01-10 05:38:25,269] jepsen worker 22 - jepsen.util 652 :invoke :write [7 7] INFO [2022-01-10 05:38:25,277] jepsen worker 30 - jepsen.util 570 :ok :read [10 37] INFO [2022-01-10 05:38:25,278] jepsen worker 30 - jepsen.util 570 :invoke :read [10 nil] INFO [2022-01-10 05:38:25,384] jepsen worker 80 - jepsen.util 890 :ok :read [26 10] INFO [2022-01-10 05:38:25,385] jepsen worker 80 - jepsen.util 890 :invoke :read [26 nil] INFO [2022-01-10 05:38:25,636] jepsen worker 23 - jepsen.util 833 :info :write [7 21] :RequestTimeoutException INFO [2022-01-10 05:38:25,637] jepsen worker 23 - jepsen.util 923 :invoke :write [7 24] INFO [2022-01-10 05:38:25,877] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.06432748538011701}, :ns_1@172.23.107.227 {:progress 0.02932551319648091}, :ns_1@172.23.105.187 {:progress 0.02932551319648091}, :ns_1@172.23.107.228 {:progress 0.06470588235294117}} INFO [2022-01-10 05:38:25,877] jepsen worker nemesis - couchbase.util Rebalance check count: 63 INFO [2022-01-10 05:38:25,993] jepsen worker 32 - jepsen.util 842 :ok :read [10 37] INFO [2022-01-10 05:38:26,192] jepsen worker nemesis - couchbase.util Rebalance stuck count: 54 INFO [2022-01-10 05:38:26,192] jepsen worker 32 - jepsen.util 842 :invoke :read [10 nil] INFO [2022-01-10 05:38:26,292] jepsen worker 75 - jepsen.util 705 :ok :read [25 :nil] INFO [2022-01-10 05:38:26,292] jepsen worker 75 - jepsen.util 705 :invoke :read [25 nil] INFO [2022-01-10 05:38:26,411] jepsen worker 9 - jepsen.util 999 :info :write [3 44] :RequestTimeoutException INFO [2022-01-10 05:38:26,411] jepsen worker 9 - jepsen.util 1089 :invoke :read [3 nil] INFO [2022-01-10 05:38:26,492] jepsen worker 66 - jepsen.util 606 :ok :read [22 41] INFO [2022-01-10 05:38:26,492] jepsen worker 66 - jepsen.util 606 :invoke :read [22 nil] INFO [2022-01-10 05:38:26,630] jepsen worker 60 - jepsen.util 960 :info :write [20 4] :RequestTimeoutException INFO [2022-01-10 05:38:26,630] jepsen worker 60 - jepsen.util 1050 :invoke :write [20 31] INFO [2022-01-10 05:38:26,715] jepsen worker 86 - jepsen.util 536 :info :write [28 44] :RequestTimeoutException INFO [2022-01-10 05:38:26,716] jepsen worker 86 - jepsen.util 626 :invoke :read [28 nil] INFO [2022-01-10 05:38:26,812] jepsen worker 77 - jepsen.util 977 :info :write [25 21] :RequestTimeoutException INFO [2022-01-10 05:38:26,813] jepsen worker 77 - jepsen.util 1067 :invoke :read [25 nil] INFO [2022-01-10 05:38:27,066] jepsen worker 45 - jepsen.util 945 :info :write [15 24] :RequestTimeoutException INFO [2022-01-10 05:38:27,067] jepsen worker 45 - jepsen.util 1035 :invoke :read [15 nil] INFO [2022-01-10 05:38:27,138] jepsen worker 21 - jepsen.util 1101 :ok :read [7 :nil] INFO [2022-01-10 05:38:27,140] jepsen worker 21 - jepsen.util 1101 :invoke :write [7 20] INFO [2022-01-10 05:38:27,153] jepsen worker 47 - jepsen.util 857 :ok :read [15 2] INFO [2022-01-10 05:38:27,154] jepsen worker 47 - jepsen.util 857 :invoke :read [15 nil] INFO [2022-01-10 05:38:27,198] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.07017543859649122}, :ns_1@172.23.107.227 {:progress 0.04985337243401755}, :ns_1@172.23.105.187 {:progress 0.04985337243401755}, :ns_1@172.23.107.228 {:progress 0.07058823529411762}} INFO [2022-01-10 05:38:27,198] jepsen worker nemesis - couchbase.util Rebalance check count: 64 INFO [2022-01-10 05:38:27,198] jepsen worker nemesis - couchbase.util Rebalance stuck count: 54 INFO [2022-01-10 05:38:27,252] jepsen worker 7 - jepsen.util 727 :info :write [2 0] :RequestTimeoutException INFO [2022-01-10 05:38:27,252] jepsen worker 7 - jepsen.util 817 :invoke :read [2 nil] INFO [2022-01-10 05:38:27,265] jepsen worker 37 - jepsen.util 847 :info :write [12 7] :RequestTimeoutException INFO [2022-01-10 05:38:27,266] jepsen worker 37 - jepsen.util 937 :invoke :read [12 nil] INFO [2022-01-10 05:38:27,340] jepsen worker 26 - jepsen.util 926 :ok :read [8 :nil] INFO [2022-01-10 05:38:27,340] jepsen worker 26 - jepsen.util 926 :invoke :write [8 24] INFO [2022-01-10 05:38:27,346] jepsen worker 70 - jepsen.util 1060 :info :write [23 12] :RequestTimeoutException INFO [2022-01-10 05:38:27,346] jepsen worker 70 - jepsen.util 1150 :invoke :write [23 13] INFO [2022-01-10 05:38:27,354] jepsen worker 34 - jepsen.util 934 :ok :read [11 21] INFO [2022-01-10 05:38:27,354] jepsen worker 34 - jepsen.util 934 :invoke :read [11 nil] INFO [2022-01-10 05:38:27,490] jepsen worker 2 - jepsen.util 812 :info :write [0 46] :RequestTimeoutException INFO [2022-01-10 05:38:27,490] jepsen worker 2 - jepsen.util 902 :invoke :write [0 39] INFO [2022-01-10 05:38:27,492] jepsen worker 40 - jepsen.util 670 :ok :read [13 23] INFO [2022-01-10 05:38:27,493] jepsen worker 40 - jepsen.util 670 :invoke :write [13 15] INFO [2022-01-10 05:38:27,498] jepsen worker 15 - jepsen.util 555 :ok :read [5 42] INFO [2022-01-10 05:38:27,499] jepsen worker 15 - jepsen.util 555 :invoke :write [5 11] INFO [2022-01-10 05:38:27,515] jepsen worker 66 - jepsen.util 606 :ok :read [22 41] INFO [2022-01-10 05:38:27,516] jepsen worker 66 - jepsen.util 606 :invoke :read [22 nil] INFO [2022-01-10 05:38:27,522] jepsen worker 54 - jepsen.util 774 :info :write [18 39] :RequestTimeoutException INFO [2022-01-10 05:38:27,523] jepsen worker 54 - jepsen.util 864 :invoke :read [18 nil] INFO [2022-01-10 05:38:27,675] jepsen worker 88 - jepsen.util 898 :info :write [29 39] :RequestTimeoutException INFO [2022-01-10 05:38:27,675] jepsen worker 88 - jepsen.util 988 :invoke :read [29 nil] INFO [2022-01-10 05:38:27,677] jepsen worker 14 - jepsen.util 914 :ok :read [4 :nil] INFO [2022-01-10 05:38:27,677] jepsen worker 14 - jepsen.util 914 :invoke :write [4 9] INFO [2022-01-10 05:38:27,716] jepsen worker 51 - jepsen.util 771 :info :write [17 28] :RequestTimeoutException INFO [2022-01-10 05:38:27,718] jepsen worker 51 - jepsen.util 861 :invoke :write [17 43] INFO [2022-01-10 05:38:27,720] jepsen worker 55 - jepsen.util 865 :ok :read [18 14] INFO [2022-01-10 05:38:27,721] jepsen worker 3 - jepsen.util 903 :info :write [1 47] :RequestTimeoutException INFO [2022-01-10 05:38:27,721] jepsen worker 55 - jepsen.util 865 :invoke :read [18 nil] INFO [2022-01-10 05:38:27,722] jepsen worker 3 - jepsen.util 993 :invoke :write [1 18] INFO [2022-01-10 05:38:27,744] jepsen worker 17 - jepsen.util 737 :ok :read [5 42] INFO [2022-01-10 05:38:27,745] jepsen worker 17 - jepsen.util 737 :invoke :read [5 nil] INFO [2022-01-10 05:38:27,746] jepsen worker 66 - jepsen.util 606 :ok :read [22 41] INFO [2022-01-10 05:38:27,746] jepsen worker 66 - jepsen.util 606 :invoke :write [22 46] INFO [2022-01-10 05:38:27,773] jepsen worker 20 - jepsen.util 650 :ok :read [6 25] INFO [2022-01-10 05:38:27,773] jepsen worker 20 - jepsen.util 650 :invoke :write [6 43] INFO [2022-01-10 05:38:27,896] jepsen worker 5 - jepsen.util 725 :info :write [1 19] :RequestTimeoutException INFO [2022-01-10 05:38:27,897] jepsen worker 5 - jepsen.util 815 :invoke :write [1 24] INFO [2022-01-10 05:38:28,080] jepsen worker 85 - jepsen.util 985 :ok :read [28 :nil] INFO [2022-01-10 05:38:28,081] jepsen worker 85 - jepsen.util 985 :invoke :write [28 22] INFO [2022-01-10 05:38:28,085] jepsen worker 28 - jepsen.util 838 :ok :read [9 22] INFO [2022-01-10 05:38:28,086] jepsen worker 28 - jepsen.util 838 :invoke :write [9 22] INFO [2022-01-10 05:38:28,108] jepsen worker 58 - jepsen.util 868 :ok :read [19 31] INFO [2022-01-10 05:38:28,109] jepsen worker 58 - jepsen.util 868 :invoke :write [19 29] INFO [2022-01-10 05:38:28,110] jepsen worker 16 - jepsen.util 916 :ok :read [5 42] INFO [2022-01-10 05:38:28,111] jepsen worker 16 - jepsen.util 916 :invoke :read [5 nil] INFO [2022-01-10 05:38:28,121] jepsen worker 17 - jepsen.util 737 :ok :read [5 42] INFO [2022-01-10 05:38:28,121] jepsen worker 17 - jepsen.util 737 :invoke :read [5 nil] INFO [2022-01-10 05:38:28,179] jepsen worker 44 - jepsen.util 944 :ok :read [14 14] INFO [2022-01-10 05:38:28,180] jepsen worker 44 - jepsen.util 944 :invoke :read [14 nil] INFO [2022-01-10 05:38:28,183] jepsen worker 75 - jepsen.util 705 :ok :read [25 :nil] INFO [2022-01-10 05:38:28,184] jepsen worker 75 - jepsen.util 705 :invoke :write [25 36] INFO [2022-01-10 05:38:28,185] jepsen worker 73 - jepsen.util 793 :ok :read [24 18] INFO [2022-01-10 05:38:28,186] jepsen worker 73 - jepsen.util 793 :invoke :read [24 nil] INFO [2022-01-10 05:38:28,197] jepsen worker 18 - jepsen.util 738 :ok :read [6 25] INFO [2022-01-10 05:38:28,198] jepsen worker 18 - jepsen.util 738 :invoke :read [6 nil] INFO [2022-01-10 05:38:28,204] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.07017543859649122}, :ns_1@172.23.107.227 {:progress 0.04985337243401755}, :ns_1@172.23.105.187 {:progress 0.04985337243401755}, :ns_1@172.23.107.228 {:progress 0.07058823529411762}} INFO [2022-01-10 05:38:28,205] jepsen worker nemesis - couchbase.util Rebalance check count: 65 INFO [2022-01-10 05:38:28,205] jepsen worker nemesis - couchbase.util Rebalance stuck count: 55 INFO [2022-01-10 05:38:28,211] jepsen worker 32 - jepsen.util 842 :ok :read [10 37] INFO [2022-01-10 05:38:28,212] jepsen worker 32 - jepsen.util 842 :invoke :write [10 12] INFO [2022-01-10 05:38:28,225] jepsen worker 78 - jepsen.util 708 :ok :read [26 10] INFO [2022-01-10 05:38:28,226] jepsen worker 78 - jepsen.util 708 :invoke :read [26 nil] INFO [2022-01-10 05:38:28,281] jepsen worker 4 - jepsen.util 814 :info :write [1 17] :RequestTimeoutException INFO [2022-01-10 05:38:28,281] jepsen worker 4 - jepsen.util 904 :invoke :read [1 nil] INFO [2022-01-10 05:38:28,378] jepsen worker 43 - jepsen.util 853 :ok :read [14 14] INFO [2022-01-10 05:38:28,379] jepsen worker 43 - jepsen.util 853 :invoke :write [14 0] INFO [2022-01-10 05:38:28,516] jepsen worker 65 - jepsen.util 875 :info :write [21 29] :RequestTimeoutException INFO [2022-01-10 05:38:28,517] jepsen worker 65 - jepsen.util 965 :invoke :write [21 7] INFO [2022-01-10 05:38:28,584] jepsen worker 16 - jepsen.util 916 :ok :read [5 42] INFO [2022-01-10 05:38:28,585] jepsen worker 16 - jepsen.util 916 :invoke :write [5 4] INFO [2022-01-10 05:38:28,643] jepsen worker 17 - jepsen.util 737 :ok :read [5 42] INFO [2022-01-10 05:38:28,644] jepsen worker 17 - jepsen.util 737 :invoke :read [5 nil] INFO [2022-01-10 05:38:28,763] jepsen worker 54 - jepsen.util 864 :ok :read [18 14] INFO [2022-01-10 05:38:28,763] jepsen worker 54 - jepsen.util 864 :invoke :read [18 nil] INFO [2022-01-10 05:38:28,887] jepsen worker 88 - jepsen.util 988 :ok :read [29 33] INFO [2022-01-10 05:38:28,888] jepsen worker 88 - jepsen.util 988 :invoke :read [29 nil] INFO [2022-01-10 05:38:28,960] jepsen worker 74 - jepsen.util 1064 :info :write [24 12] :RequestTimeoutException INFO [2022-01-10 05:38:28,961] jepsen worker 74 - jepsen.util 1154 :invoke :write [24 4] INFO [2022-01-10 05:38:29,069] jepsen worker 17 - jepsen.util 737 :ok :read [5 42] INFO [2022-01-10 05:38:29,070] jepsen worker 17 - jepsen.util 737 :invoke :read [5 nil] INFO [2022-01-10 05:38:29,142] jepsen worker 12 - jepsen.util 822 :info :write [4 5] :RequestTimeoutException INFO [2022-01-10 05:38:29,144] jepsen worker 12 - jepsen.util 912 :invoke :read [4 nil] INFO [2022-01-10 05:38:29,212] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.07017543859649122}, :ns_1@172.23.107.227 {:progress 0.05278592375366564}, :ns_1@172.23.105.187 {:progress 0.05278592375366564}, :ns_1@172.23.107.228 {:progress 0.07058823529411762}} INFO [2022-01-10 05:38:29,212] jepsen worker nemesis - couchbase.util Rebalance check count: 66 INFO [2022-01-10 05:38:29,212] jepsen worker nemesis - couchbase.util Rebalance stuck count: 55 INFO [2022-01-10 05:38:29,216] jepsen worker 29 - jepsen.util 749 :info :write [9 14] :RequestTimeoutException INFO [2022-01-10 05:38:29,217] jepsen worker 29 - jepsen.util 839 :invoke :write [9 43] INFO [2022-01-10 05:38:29,259] jepsen worker 17 - jepsen.util 737 :ok :read [5 42] INFO [2022-01-10 05:38:29,260] jepsen worker 17 - jepsen.util 737 :invoke :read [5 nil] INFO [2022-01-10 05:38:29,395] jepsen worker 17 - jepsen.util 737 :ok :read [5 42] INFO [2022-01-10 05:38:29,396] jepsen worker 17 - jepsen.util 737 :invoke :write [5 17] INFO [2022-01-10 05:38:29,495] jepsen worker 33 - jepsen.util 843 :info :write [11 35] :RequestTimeoutException INFO [2022-01-10 05:38:29,495] jepsen worker 33 - jepsen.util 933 :invoke :read [11 nil] INFO [2022-01-10 05:38:29,582] jepsen worker 0 - jepsen.util 720 :info :write [0 48] :RequestTimeoutException INFO [2022-01-10 05:38:29,582] jepsen worker 0 - jepsen.util 810 :invoke :read [0 nil] WARN [2022-01-10 05:38:29,605] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18330503,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18330503,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9742786,"encode_duration_us":16485,"operation_id":"0x1918","timeout_ms":10000,"last_server_duration_us":9742786,"total_duration_us":10004633},{"operation_name":"upsert","last_dispatch_duration_us":23938763,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":23938763,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9458563,"encode_duration_us":16725,"operation_id":"0x17aa","timeout_ms":10000,"last_server_duration_us":9458563,"total_duration_us":10009599}],"total_count":2}} INFO [2022-01-10 05:38:29,697] jepsen worker 6 - jepsen.util 726 :ok :read [2 21] INFO [2022-01-10 05:38:29,697] jepsen worker 6 - jepsen.util 726 :invoke :write [2 6] WARN [2022-01-10 05:38:29,706] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14867114,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":14867114,"total_server_duration_us":615379,"operation_id":"0x194a","timeout_ms":30000,"last_server_duration_us":615379,"total_duration_us":21052578},{"operation_name":"get","last_dispatch_duration_us":14522058,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":14522058,"total_server_duration_us":632404,"operation_id":"0x1a18","timeout_ms":30000,"last_server_duration_us":632404,"total_duration_us":16122562},{"operation_name":"get","last_dispatch_duration_us":11198992,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11198992,"total_server_duration_us":43,"operation_id":"0x1af1","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":11201633},{"operation_name":"get","last_dispatch_duration_us":10931853,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10931853,"total_server_duration_us":45055,"operation_id":"0x1b19","timeout_ms":30000,"last_server_duration_us":45055,"total_duration_us":10934136},{"operation_name":"get","last_dispatch_duration_us":10214471,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10214471,"total_server_duration_us":59374,"operation_id":"0x1b47","timeout_ms":30000,"last_server_duration_us":59374,"total_duration_us":10216831},{"operation_name":"get","last_dispatch_duration_us":8428205,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8428205,"total_server_duration_us":38,"operation_id":"0x1a45","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":10029782},{"operation_name":"upsert","last_dispatch_duration_us":2537657,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9470956,"total_server_duration_us":2432764,"encode_duration_us":18489,"operation_id":"0x1b20","timeout_ms":10000,"last_server_duration_us":166661,"total_duration_us":10008263},{"operation_name":"upsert","last_dispatch_duration_us":12738362,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12738362,"total_server_duration_us":151432,"encode_duration_us":16402,"operation_id":"0x1abd","timeout_ms":10000,"last_server_duration_us":151432,"total_duration_us":10006371},{"operation_name":"upsert","encode_duration_us":16382,"operation_id":"0x1b53","timeout_ms":10000,"total_duration_us":10006362},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","encode_duration_us":16498,"operation_id":"0x1a78","timeout_ms":10000,"total_duration_us":10004342}],"total_count":16}} INFO [2022-01-10 05:38:29,771] jepsen worker 55 - jepsen.util 865 :ok :read [18 14] INFO [2022-01-10 05:38:29,772] jepsen worker 55 - jepsen.util 865 :invoke :write [18 8] INFO [2022-01-10 05:38:30,219] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.07017543859649122}, :ns_1@172.23.107.227 {:progress 0.05278592375366564}, :ns_1@172.23.105.187 {:progress 0.05278592375366564}, :ns_1@172.23.107.228 {:progress 0.07058823529411762}} INFO [2022-01-10 05:38:30,219] jepsen worker nemesis - couchbase.util Rebalance check count: 67 INFO [2022-01-10 05:38:30,219] jepsen worker nemesis - couchbase.util Rebalance stuck count: 56 INFO [2022-01-10 05:38:30,321] jepsen worker 47 - jepsen.util 857 :ok :read [15 2] INFO [2022-01-10 05:38:30,322] jepsen worker 47 - jepsen.util 857 :invoke :read [15 nil] INFO [2022-01-10 05:38:30,369] jepsen worker 7 - jepsen.util 817 :ok :read [2 21] INFO [2022-01-10 05:38:30,369] jepsen worker 7 - jepsen.util 817 :invoke :write [2 14] INFO [2022-01-10 05:38:30,412] jepsen worker 46 - jepsen.util 856 :info :write [15 27] :RequestTimeoutException INFO [2022-01-10 05:38:30,412] jepsen worker 46 - jepsen.util 946 :invoke :read [15 nil] INFO [2022-01-10 05:38:30,420] jepsen worker 56 - jepsen.util 956 :ok :read [18 14] INFO [2022-01-10 05:38:30,421] jepsen worker 56 - jepsen.util 956 :invoke :write [18 18] INFO [2022-01-10 05:38:30,460] jepsen worker 36 - jepsen.util 486 :info :write [12 35] :RequestTimeoutException INFO [2022-01-10 05:38:30,461] jepsen worker 36 - jepsen.util 576 :invoke :read [12 nil] INFO [2022-01-10 05:38:30,470] jepsen worker 53 - jepsen.util 683 :info :write [17 36] :RequestTimeoutException INFO [2022-01-10 05:38:30,471] jepsen worker 53 - jepsen.util 773 :invoke :read [17 nil] INFO [2022-01-10 05:38:30,525] jepsen worker 61 - jepsen.util 961 :ok :read [20 :nil] INFO [2022-01-10 05:38:30,530] jepsen worker 61 - jepsen.util 961 :invoke :write [20 31] INFO [2022-01-10 05:38:30,546] jepsen worker 78 - jepsen.util 708 :ok :read [26 10] INFO [2022-01-10 05:38:30,546] jepsen worker 78 - jepsen.util 708 :invoke :write [26 41] INFO [2022-01-10 05:38:30,637] jepsen worker 88 - jepsen.util 988 :ok :read [29 33] INFO [2022-01-10 05:38:30,638] jepsen worker 88 - jepsen.util 988 :invoke :write [29 7] INFO [2022-01-10 05:38:30,721] jepsen worker 12 - jepsen.util 912 :ok :read [4 :nil] INFO [2022-01-10 05:38:30,722] jepsen worker 12 - jepsen.util 912 :invoke :read [4 nil] INFO [2022-01-10 05:38:30,745] jepsen worker 11 - jepsen.util 731 :ok :read [3 2] INFO [2022-01-10 05:38:30,745] jepsen worker 11 - jepsen.util 731 :invoke :write [3 2] INFO [2022-01-10 05:38:30,978] jepsen worker 81 - jepsen.util 801 :fail :read [27 nil] :UnambiguousTimeoutException INFO [2022-01-10 05:38:30,997] jepsen worker 81 - jepsen.util 801 :invoke :write [27 1] INFO [2022-01-10 05:38:31,006] jepsen worker 36 - jepsen.util 576 :ok :read [12 :nil] INFO [2022-01-10 05:38:31,006] jepsen worker 36 - jepsen.util 576 :invoke :write [12 13] INFO [2022-01-10 05:38:30,984] jepsen worker 34 - jepsen.util 934 :ok :read [11 21] INFO [2022-01-10 05:38:31,011] jepsen worker 34 - jepsen.util 934 :invoke :read [11 nil] INFO [2022-01-10 05:38:31,068] jepsen worker 33 - jepsen.util 933 :ok :read [11 21] INFO [2022-01-10 05:38:31,069] jepsen worker 33 - jepsen.util 933 :invoke :read [11 nil] WARN [2022-01-10 05:38:31,147] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10433623,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10433623,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9392652,"encode_duration_us":19167,"operation_id":"0x1a09","timeout_ms":10000,"last_server_duration_us":9392652,"total_duration_us":10003708}],"total_count":1}} WARN [2022-01-10 05:38:31,153] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14703462,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":14703462,"total_server_duration_us":76,"operation_id":"0x1808","timeout_ms":30000,"last_server_duration_us":76,"total_duration_us":22420052},{"operation_name":"get","last_dispatch_duration_us":19769233,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":19769233,"total_server_duration_us":32,"operation_id":"0x19f9","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":19769654},{"operation_name":"get","last_dispatch_duration_us":2242224,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":2242224,"total_server_duration_us":12569,"operation_id":"0x19db","timeout_ms":30000,"last_server_duration_us":12569,"total_duration_us":17606874},{"operation_name":"get","last_dispatch_duration_us":16735042,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":16735042,"total_server_duration_us":1115,"operation_id":"0x19c8","timeout_ms":30000,"last_server_duration_us":1115,"total_duration_us":16736900},{"operation_name":"get","last_dispatch_duration_us":14459857,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":14459857,"total_server_duration_us":852261,"operation_id":"0x198a","timeout_ms":30000,"last_server_duration_us":852261,"total_duration_us":14460307},{"operation_name":"upsert","last_dispatch_duration_us":1104183,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":5850709,"total_server_duration_us":732267,"encode_duration_us":18040,"operation_id":"0x1af0","timeout_ms":10000,"last_server_duration_us":172334,"total_duration_us":10009287},{"operation_name":"upsert","last_dispatch_duration_us":1670683,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9663840,"total_server_duration_us":1977323,"encode_duration_us":16481,"operation_id":"0x1a6e","timeout_ms":10000,"last_server_duration_us":662847,"total_duration_us":10007610},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":16388,"operation_id":"0x1b1f","timeout_ms":10000,"total_duration_us":10006848},{"operation_name":"upsert","last_dispatch_duration_us":2051675,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":2051675,"total_server_duration_us":278145,"encode_duration_us":16310,"operation_id":"0x1b52","timeout_ms":10000,"last_server_duration_us":278145,"total_duration_us":10003213},{"operation_name":"upsert","last_dispatch_duration_us":1175082,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":6188345,"total_server_duration_us":2692996,"encode_duration_us":17678,"operation_id":"0x1a6f","timeout_ms":10000,"last_server_duration_us":696035,"total_duration_us":10002157}],"total_count":16}} WARN [2022-01-10 05:38:31,154] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":23715846,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":23715846,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":10219944,"encode_duration_us":17363,"operation_id":"0x1934","timeout_ms":10000,"last_server_duration_us":10219944,"total_duration_us":10002037}],"total_count":1}} INFO [2022-01-10 05:38:31,226] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.07602339181286555}, :ns_1@172.23.107.227 {:progress 0.05865102639296182}, :ns_1@172.23.105.187 {:progress 0.05865102639296182}, :ns_1@172.23.107.228 {:progress 0.07647058823529407}} INFO [2022-01-10 05:38:31,226] jepsen worker nemesis - couchbase.util Rebalance check count: 68 INFO [2022-01-10 05:38:31,226] jepsen worker nemesis - couchbase.util Rebalance stuck count: 56 INFO [2022-01-10 05:38:31,243] jepsen worker 53 - jepsen.util 773 :ok :read [17 41] INFO [2022-01-10 05:38:31,244] jepsen worker 53 - jepsen.util 773 :invoke :read [17 nil] WARN [2022-01-10 05:38:31,247] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","operation_id":"0x182c","timeout_ms":30000,"total_duration_us":30008393},{"operation_name":"get","last_dispatch_duration_us":15297282,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":15297282,"total_server_duration_us":43,"operation_id":"0x1a64","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":15301168},{"operation_name":"upsert","last_dispatch_duration_us":13701462,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":13701462,"total_server_duration_us":13011050,"encode_duration_us":16409,"operation_id":"0x1a10","timeout_ms":10000,"last_server_duration_us":13011050,"total_duration_us":10010594},{"operation_name":"upsert","last_dispatch_duration_us":1782474,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":1782474,"total_server_duration_us":479658,"encode_duration_us":18643,"operation_id":"0x1a42","timeout_ms":10000,"last_server_duration_us":479658,"total_duration_us":10009878},{"operation_name":"upsert","last_dispatch_duration_us":7219105,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12642132,"total_server_duration_us":1375388,"encode_duration_us":17216,"operation_id":"0x1a16","timeout_ms":10000,"last_server_duration_us":392886,"total_duration_us":10008540},{"operation_name":"upsert","last_dispatch_duration_us":3079987,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11078661,"total_server_duration_us":1004964,"encode_duration_us":17770,"operation_id":"0x1b15","timeout_ms":10000,"last_server_duration_us":602237,"total_duration_us":10006075},{"operation_name":"upsert","encode_duration_us":16683,"operation_id":"0x1b2d","timeout_ms":10000,"total_duration_us":10005981},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":16768,"operation_id":"0x1b7d","timeout_ms":10000,"total_duration_us":10005695},{"operation_name":"upsert","encode_duration_us":16329,"operation_id":"0x1a79","timeout_ms":10000,"total_duration_us":10002473},{"operation_name":"upsert","last_dispatch_duration_us":3485991,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":13311610,"total_server_duration_us":1109474,"encode_duration_us":17024,"operation_id":"0x1ac7","timeout_ms":10000,"last_server_duration_us":387864,"total_duration_us":10002457}],"total_count":19}} WARN [2022-01-10 05:38:31,248] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18580424,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":18580424,"total_server_duration_us":27,"operation_id":"0x17c1","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":28341074},{"operation_name":"get","last_dispatch_duration_us":18125196,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":18125196,"total_server_duration_us":405275,"operation_id":"0x193b","timeout_ms":30000,"last_server_duration_us":405275,"total_duration_us":19207363},{"operation_name":"get","last_dispatch_duration_us":17477031,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":17477031,"total_server_duration_us":414388,"operation_id":"0x194c","timeout_ms":30000,"last_server_duration_us":414388,"total_duration_us":17492879},{"operation_name":"get","last_dispatch_duration_us":16043514,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":16043514,"total_server_duration_us":844499,"operation_id":"0x1a12","timeout_ms":30000,"last_server_duration_us":844499,"total_duration_us":16045535},{"operation_name":"get","last_dispatch_duration_us":11607372,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11607372,"total_server_duration_us":329252,"operation_id":"0x19ae","timeout_ms":30000,"last_server_duration_us":329252,"total_duration_us":15757742},{"operation_name":"get","last_dispatch_duration_us":14161218,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":14161218,"total_server_duration_us":190863,"operation_id":"0x1a6b","timeout_ms":30000,"last_server_duration_us":190863,"total_duration_us":14161619},{"operation_name":"get","last_dispatch_duration_us":10949767,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10949767,"total_server_duration_us":407545,"operation_id":"0x1a92","timeout_ms":30000,"last_server_duration_us":407545,"total_duration_us":10951987},{"operation_name":"upsert","last_dispatch_duration_us":3835970,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11184386,"total_server_duration_us":937042,"encode_duration_us":16610,"operation_id":"0x1b21","timeout_ms":10000,"last_server_duration_us":595210,"total_duration_us":10006481},{"operation_name":"upsert","last_dispatch_duration_us":1430392,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9109131,"total_server_duration_us":280923,"encode_duration_us":16992,"operation_id":"0x1ac4","timeout_ms":10000,"last_server_duration_us":186359,"total_duration_us":10005547},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":16404,"operation_id":"0x1afa","timeout_ms":10000,"total_duration_us":10004773}],"total_count":26}} WARN [2022-01-10 05:38:31,248] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17704842,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":17704842,"total_server_duration_us":49,"operation_id":"0x1888","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":22359472},{"operation_name":"upsert","last_dispatch_duration_us":2093038,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10512335,"total_server_duration_us":892437,"encode_duration_us":18078,"operation_id":"0x1b24","timeout_ms":10000,"last_server_duration_us":237259,"total_duration_us":10010312},{"operation_name":"upsert","last_dispatch_duration_us":9495080,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":13014804,"total_server_duration_us":604822,"encode_duration_us":16380,"operation_id":"0x1a13","timeout_ms":10000,"last_server_duration_us":97793,"total_duration_us":10008647},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":16433,"operation_id":"0x1a6d","timeout_ms":10000,"total_duration_us":10008414},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":16547,"operation_id":"0x1b80","timeout_ms":10000,"total_duration_us":10003434},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16181,"operation_id":"0x1b0e","timeout_ms":10000,"total_duration_us":10003291},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":16204,"operation_id":"0x1a9a","timeout_ms":10000,"total_duration_us":10002865},{"operation_name":"upsert","last_dispatch_duration_us":1644625,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":11345819,"total_server_duration_us":2605098,"encode_duration_us":18412,"operation_id":"0x1b7b","timeout_ms":10000,"last_server_duration_us":172531,"total_duration_us":10001973},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":17008,"operation_id":"0x1ba6","timeout_ms":10000,"total_duration_us":10001556},{"operation_name":"get","last_dispatch_duration_us":6889450,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":6889450,"total_server_duration_us":32,"operation_id":"0x1b4a","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":6891504}],"total_count":15}} WARN [2022-01-10 05:38:31,249] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12104754,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18171141,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":11401815,"encode_duration_us":17686,"operation_id":"0x1a11","timeout_ms":10000,"last_server_duration_us":10476277,"total_duration_us":10006259}],"total_count":1}} INFO [2022-01-10 05:38:31,305] jepsen worker 63 - jepsen.util 873 :ok :read [21 7] INFO [2022-01-10 05:38:31,306] jepsen worker 63 - jepsen.util 873 :invoke :write [21 17] WARN [2022-01-10 05:38:31,348] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":13701462,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13701462,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":13011050,"encode_duration_us":16409,"operation_id":"0x1a10","timeout_ms":10000,"last_server_duration_us":13011050,"total_duration_us":10010594},{"operation_name":"upsert","last_dispatch_duration_us":28590401,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":30330382,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":10249911,"encode_duration_us":16442,"operation_id":"0x1653","timeout_ms":10000,"last_server_duration_us":9560514,"total_duration_us":10010991}],"total_count":2}} WARN [2022-01-10 05:38:31,349] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9872932,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16401896,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":11508150,"encode_duration_us":17731,"operation_id":"0x1952","timeout_ms":10000,"last_server_duration_us":9176858,"total_duration_us":10005451}],"total_count":1}} WARN [2022-01-10 05:38:31,349] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16499916,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":16499916,"total_server_duration_us":32,"operation_id":"0x190e","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":16502289},{"operation_name":"get","last_dispatch_duration_us":10852811,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10852811,"total_server_duration_us":27,"operation_id":"0x1b73","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":10855476},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":17279,"operation_id":"0x1b23","timeout_ms":10000,"total_duration_us":10010388},{"operation_name":"upsert","last_dispatch_duration_us":1441719,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":3409441,"total_server_duration_us":1028309,"encode_duration_us":17566,"operation_id":"0x1af2","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10008129},{"operation_name":"upsert","last_dispatch_duration_us":11820031,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11820031,"total_server_duration_us":48219,"encode_duration_us":16491,"operation_id":"0x1b74","timeout_ms":10000,"last_server_duration_us":48219,"total_duration_us":10006459},{"operation_name":"upsert","last_dispatch_duration_us":12104754,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":18171141,"total_server_duration_us":11401815,"encode_duration_us":17686,"operation_id":"0x1a11","timeout_ms":10000,"last_server_duration_us":10476277,"total_duration_us":10006259},{"operation_name":"upsert","last_dispatch_duration_us":2068836,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9367260,"total_server_duration_us":958707,"encode_duration_us":16380,"operation_id":"0x1ba7","timeout_ms":10000,"last_server_duration_us":135758,"total_duration_us":10006049},{"operation_name":"upsert","last_dispatch_duration_us":991178,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10554859,"total_server_duration_us":1135991,"encode_duration_us":16386,"operation_id":"0x1ba8","timeout_ms":10000,"last_server_duration_us":274773,"total_duration_us":10003697},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":16336,"operation_id":"0x1a6c","timeout_ms":10000,"total_duration_us":10001871},{"operation_name":"get","last_dispatch_duration_us":9405420,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9405420,"total_server_duration_us":32,"operation_id":"0x1a3a","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":9407275}],"total_count":19}} INFO [2022-01-10 05:38:31,490] jepsen worker 39 - jepsen.util 939 :info :write [13 44] :RequestTimeoutException INFO [2022-01-10 05:38:31,491] jepsen worker 39 - jepsen.util 1029 :invoke :read [13 nil] INFO [2022-01-10 05:38:31,560] jepsen worker 31 - jepsen.util 1021 :info :write [10 31] :RequestTimeoutException INFO [2022-01-10 05:38:31,561] jepsen worker 31 - jepsen.util 1111 :invoke :write [10 48] INFO [2022-01-10 05:38:31,626] jepsen worker 79 - jepsen.util 799 :info :write [26 17] :RequestTimeoutException INFO [2022-01-10 05:38:31,627] jepsen worker 79 - jepsen.util 889 :invoke :read [26 nil] INFO [2022-01-10 05:38:31,865] jepsen worker 12 - jepsen.util 912 :ok :read [4 :nil] INFO [2022-01-10 05:38:31,865] jepsen worker 12 - jepsen.util 912 :invoke :write [4 17] INFO [2022-01-10 05:38:31,905] jepsen worker 62 - jepsen.util 962 :ok :read [20 :nil] INFO [2022-01-10 05:38:31,905] jepsen worker 62 - jepsen.util 962 :invoke :write [20 39] INFO [2022-01-10 05:38:31,950] jepsen worker 33 - jepsen.util 933 :ok :read [11 21] INFO [2022-01-10 05:38:31,951] jepsen worker 33 - jepsen.util 933 :invoke :read [11 nil] INFO [2022-01-10 05:38:32,007] jepsen worker 77 - jepsen.util 1067 :ok :read [25 :nil] INFO [2022-01-10 05:38:32,008] jepsen worker 77 - jepsen.util 1067 :invoke :read [25 nil] INFO [2022-01-10 05:38:32,114] jepsen worker 44 - jepsen.util 944 :ok :read [14 14] INFO [2022-01-10 05:38:32,114] jepsen worker 44 - jepsen.util 944 :invoke :read [14 nil] INFO [2022-01-10 05:38:32,135] jepsen worker 18 - jepsen.util 738 :ok :read [6 25] INFO [2022-01-10 05:38:32,135] jepsen worker 18 - jepsen.util 738 :invoke :read [6 nil] INFO [2022-01-10 05:38:32,161] jepsen worker 52 - jepsen.util 772 :info :write [17 0] :RequestTimeoutException INFO [2022-01-10 05:38:32,162] jepsen worker 52 - jepsen.util 862 :invoke :write [17 41] INFO [2022-01-10 05:38:32,187] jepsen worker 53 - jepsen.util 773 :ok :read [17 41] INFO [2022-01-10 05:38:32,188] jepsen worker 53 - jepsen.util 773 :invoke :write [17 21] INFO [2022-01-10 05:38:32,232] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.07602339181286555}, :ns_1@172.23.107.227 {:progress 0.05865102639296182}, :ns_1@172.23.105.187 {:progress 0.05865102639296182}, :ns_1@172.23.107.228 {:progress 0.07647058823529407}} INFO [2022-01-10 05:38:32,232] jepsen worker nemesis - couchbase.util Rebalance check count: 69 INFO [2022-01-10 05:38:32,232] jepsen worker nemesis - couchbase.util Rebalance stuck count: 57 INFO [2022-01-10 05:38:32,312] jepsen worker 38 - jepsen.util 848 :info :write [12 43] :RequestTimeoutException INFO [2022-01-10 05:38:32,312] jepsen worker 38 - jepsen.util 938 :invoke :write [12 29] INFO [2022-01-10 05:38:32,393] jepsen worker 77 - jepsen.util 1067 :ok :read [25 :nil] INFO [2022-01-10 05:38:32,394] jepsen worker 77 - jepsen.util 1067 :invoke :write [25 45] INFO [2022-01-10 05:38:32,400] jepsen worker 30 - jepsen.util 570 :ok :read [10 37] INFO [2022-01-10 05:38:32,400] jepsen worker 54 - jepsen.util 864 :ok :read [18 14] INFO [2022-01-10 05:38:32,408] jepsen worker 30 - jepsen.util 570 :invoke :read [10 nil] INFO [2022-01-10 05:38:32,408] jepsen worker 46 - jepsen.util 946 :ok :read [15 2] INFO [2022-01-10 05:38:32,409] jepsen worker 54 - jepsen.util 864 :invoke :write [18 3] INFO [2022-01-10 05:38:32,410] jepsen worker 46 - jepsen.util 946 :invoke :write [15 41] INFO [2022-01-10 05:38:32,494] jepsen worker 4 - jepsen.util 904 :ok :read [1 41] INFO [2022-01-10 05:38:32,495] jepsen worker 4 - jepsen.util 904 :invoke :write [1 1] INFO [2022-01-10 05:38:32,586] jepsen worker 59 - jepsen.util 959 :info :write [19 26] :RequestTimeoutException INFO [2022-01-10 05:38:32,587] jepsen worker 59 - jepsen.util 1049 :invoke :read [19 nil] INFO [2022-01-10 05:38:32,904] jepsen worker 24 - jepsen.util 834 :ok :read [8 :nil] INFO [2022-01-10 05:38:32,906] jepsen worker 24 - jepsen.util 834 :invoke :read [8 nil] INFO [2022-01-10 05:38:32,940] jepsen worker 73 - jepsen.util 793 :ok :read [24 18] INFO [2022-01-10 05:38:32,942] jepsen worker 73 - jepsen.util 793 :invoke :write [24 17] INFO [2022-01-10 05:38:32,951] jepsen worker 35 - jepsen.util 1025 :ok :read [11 21] INFO [2022-01-10 05:38:32,952] jepsen worker 35 - jepsen.util 1025 :invoke :write [11 37] INFO [2022-01-10 05:38:33,052] jepsen worker 25 - jepsen.util 745 :info :write [8 31] :RequestTimeoutException INFO [2022-01-10 05:38:33,054] jepsen worker 25 - jepsen.util 835 :invoke :write [8 36] INFO [2022-01-10 05:38:33,062] jepsen worker 42 - jepsen.util 852 :ok :read [14 14] INFO [2022-01-10 05:38:33,063] jepsen worker 42 - jepsen.util 852 :invoke :write [14 46] INFO [2022-01-10 05:38:33,081] jepsen worker 19 - jepsen.util 919 :ok :read [6 25] INFO [2022-01-10 05:38:33,081] jepsen worker 19 - jepsen.util 919 :invoke :write [6 2] INFO [2022-01-10 05:38:33,082] jepsen worker 47 - jepsen.util 857 :ok :read [15 2] INFO [2022-01-10 05:38:33,083] jepsen worker 47 - jepsen.util 857 :invoke :read [15 nil] INFO [2022-01-10 05:38:33,238] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.08771929824561409}, :ns_1@172.23.107.227 {:progress 0.05865102639296182}, :ns_1@172.23.105.187 {:progress 0.05865102639296182}, :ns_1@172.23.107.228 {:progress 0.08235294117647063}} INFO [2022-01-10 05:38:33,238] jepsen worker nemesis - couchbase.util Rebalance check count: 70 INFO [2022-01-10 05:38:33,238] jepsen worker nemesis - couchbase.util Rebalance stuck count: 57 INFO [2022-01-10 05:38:33,266] jepsen worker 64 - jepsen.util 784 :info :write [21 3] :RequestTimeoutException INFO [2022-01-10 05:38:33,267] jepsen worker 64 - jepsen.util 874 :invoke :read [21 nil] INFO [2022-01-10 05:38:33,287] jepsen worker 52 - jepsen.util 862 :fail :write [17 41] :DurabilityImpossible INFO [2022-01-10 05:38:33,287] jepsen worker 52 - jepsen.util 862 :invoke :write [17 21] INFO [2022-01-10 05:38:33,403] jepsen worker 51 - jepsen.util 861 :fail :write [17 43] :DurabilityImpossible INFO [2022-01-10 05:38:33,404] jepsen worker 51 - jepsen.util 861 :invoke :write [17 29] INFO [2022-01-10 05:38:33,437] jepsen worker 53 - jepsen.util 773 :fail :write [17 21] :DurabilityImpossible INFO [2022-01-10 05:38:33,437] jepsen worker 53 - jepsen.util 773 :invoke :read [17 nil] INFO [2022-01-10 05:38:33,553] jepsen worker 34 - jepsen.util 934 :ok :read [11 21] INFO [2022-01-10 05:38:33,553] jepsen worker 34 - jepsen.util 934 :invoke :read [11 nil] INFO [2022-01-10 05:38:33,642] jepsen worker 10 - jepsen.util 820 :info :write [3 13] :RequestTimeoutException INFO [2022-01-10 05:38:33,642] jepsen worker 10 - jepsen.util 910 :invoke :write [3 30] INFO [2022-01-10 05:38:33,778] jepsen worker 44 - jepsen.util 944 :ok :read [14 14] INFO [2022-01-10 05:38:33,779] jepsen worker 44 - jepsen.util 944 :invoke :write [14 8] INFO [2022-01-10 05:38:33,793] jepsen worker 18 - jepsen.util 738 :ok :read [6 25] INFO [2022-01-10 05:38:33,794] jepsen worker 18 - jepsen.util 738 :invoke :read [6 nil] INFO [2022-01-10 05:38:33,840] jepsen worker 52 - jepsen.util 862 :fail :write [17 21] :DurabilityImpossible INFO [2022-01-10 05:38:33,841] jepsen worker 52 - jepsen.util 862 :invoke :read [17 nil] INFO [2022-01-10 05:38:34,044] jepsen worker 45 - jepsen.util 1035 :ok :read [15 2] INFO [2022-01-10 05:38:34,045] jepsen worker 45 - jepsen.util 1035 :invoke :write [15 19] INFO [2022-01-10 05:38:34,046] jepsen worker 8 - jepsen.util 728 :ok :read [2 21] INFO [2022-01-10 05:38:34,047] jepsen worker 8 - jepsen.util 728 :invoke :read [2 nil] INFO [2022-01-10 05:38:34,083] jepsen worker 84 - jepsen.util 894 :ok :read [28 :nil] INFO [2022-01-10 05:38:34,083] jepsen worker 84 - jepsen.util 894 :invoke :read [28 nil] INFO [2022-01-10 05:38:34,106] jepsen worker 69 - jepsen.util 879 :info :write [23 14] :RequestTimeoutException INFO [2022-01-10 05:38:34,107] jepsen worker 69 - jepsen.util 969 :invoke :write [23 18] INFO [2022-01-10 05:38:34,116] jepsen worker 13 - jepsen.util 823 :info :write [4 4] :RequestTimeoutException INFO [2022-01-10 05:38:34,117] jepsen worker 13 - jepsen.util 913 :invoke :write [4 7] INFO [2022-01-10 05:38:34,184] jepsen worker 52 - jepsen.util 862 :ok :read [17 41] INFO [2022-01-10 05:38:34,185] jepsen worker 52 - jepsen.util 862 :invoke :write [17 14] INFO [2022-01-10 05:38:34,198] jepsen worker 27 - jepsen.util 747 :fail :read [9 nil] :UnambiguousTimeoutException INFO [2022-01-10 05:38:34,198] jepsen worker 27 - jepsen.util 747 :invoke :write [9 20] INFO [2022-01-10 05:38:34,241] jepsen worker 40 - jepsen.util 670 :ok :write [13 15] INFO [2022-01-10 05:38:34,242] jepsen worker 40 - jepsen.util 670 :invoke :write [13 28] INFO [2022-01-10 05:38:34,245] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.08771929824561409}, :ns_1@172.23.107.227 {:progress 0.05865102639296182}, :ns_1@172.23.105.187 {:progress 0.05865102639296182}, :ns_1@172.23.107.228 {:progress 0.08235294117647063}} INFO [2022-01-10 05:38:34,245] jepsen worker nemesis - couchbase.util Rebalance check count: 71 INFO [2022-01-10 05:38:34,245] jepsen worker nemesis - couchbase.util Rebalance stuck count: 58 INFO [2022-01-10 05:38:34,396] jepsen worker 71 - jepsen.util 881 :info :write [23 32] :RequestTimeoutException INFO [2022-01-10 05:38:34,396] jepsen worker 71 - jepsen.util 971 :invoke :write [23 46] INFO [2022-01-10 05:38:34,412] jepsen worker 76 - jepsen.util 796 :ok :read [25 :nil] INFO [2022-01-10 05:38:34,413] jepsen worker 76 - jepsen.util 796 :invoke :read [25 nil] INFO [2022-01-10 05:38:34,620] jepsen worker 47 - jepsen.util 857 :ok :read [15 2] INFO [2022-01-10 05:38:34,621] jepsen worker 47 - jepsen.util 857 :invoke :write [15 26] INFO [2022-01-10 05:38:34,647] jepsen worker 89 - jepsen.util 989 :ok :read [29 33] INFO [2022-01-10 05:38:34,648] jepsen worker 89 - jepsen.util 989 :invoke :read [29 nil] INFO [2022-01-10 05:38:34,857] jepsen worker 67 - jepsen.util 787 :info :write [22 6] :RequestTimeoutException INFO [2022-01-10 05:38:34,858] jepsen worker 67 - jepsen.util 877 :invoke :read [22 nil] INFO [2022-01-10 05:38:34,861] jepsen worker 24 - jepsen.util 834 :ok :read [8 :nil] INFO [2022-01-10 05:38:34,862] jepsen worker 24 - jepsen.util 834 :invoke :write [8 25] INFO [2022-01-10 05:38:34,874] jepsen worker 8 - jepsen.util 728 :ok :read [2 21] INFO [2022-01-10 05:38:34,874] jepsen worker 8 - jepsen.util 728 :invoke :read [2 nil] INFO [2022-01-10 05:38:34,877] jepsen worker 82 - jepsen.util 802 :ok :read [27 30] INFO [2022-01-10 05:38:34,877] jepsen worker 82 - jepsen.util 802 :invoke :read [27 nil] INFO [2022-01-10 05:38:34,888] jepsen worker 86 - jepsen.util 626 :ok :read [28 :nil] INFO [2022-01-10 05:38:34,888] jepsen worker 86 - jepsen.util 626 :invoke :write [28 13] INFO [2022-01-10 05:38:34,900] jepsen worker 83 - jepsen.util 983 :ok :read [27 30] INFO [2022-01-10 05:38:34,902] jepsen worker 34 - jepsen.util 934 :ok :read [11 21] INFO [2022-01-10 05:38:34,902] jepsen worker 83 - jepsen.util 983 :invoke :write [27 35] INFO [2022-01-10 05:38:34,910] jepsen worker 34 - jepsen.util 934 :invoke :read [11 nil] INFO [2022-01-10 05:38:34,914] jepsen worker 0 - jepsen.util 810 :ok :read [0 :nil] INFO [2022-01-10 05:38:34,915] jepsen worker 0 - jepsen.util 810 :invoke :write [0 49] INFO [2022-01-10 05:38:34,943] jepsen worker 82 - jepsen.util 802 :ok :read [27 30] INFO [2022-01-10 05:38:34,944] jepsen worker 82 - jepsen.util 802 :invoke :read [27 nil] INFO [2022-01-10 05:38:35,050] jepsen worker 8 - jepsen.util 728 :ok :read [2 21] INFO [2022-01-10 05:38:35,051] jepsen worker 8 - jepsen.util 728 :invoke :read [2 nil] INFO [2022-01-10 05:38:35,058] jepsen worker 48 - jepsen.util 678 :fail :read [16 nil] :UnambiguousTimeoutException INFO [2022-01-10 05:38:35,058] jepsen worker 48 - jepsen.util 678 :invoke :write [16 28] INFO [2022-01-10 05:38:35,117] jepsen worker 72 - jepsen.util 1062 :info :write [24 10] :RequestTimeoutException INFO [2022-01-10 05:38:35,117] jepsen worker 72 - jepsen.util 1152 :invoke :write [24 25] INFO [2022-01-10 05:38:35,151] jepsen worker 82 - jepsen.util 802 :ok :read [27 30] INFO [2022-01-10 05:38:35,152] jepsen worker 82 - jepsen.util 802 :invoke :write [27 35] INFO [2022-01-10 05:38:35,182] jepsen worker 87 - jepsen.util 1077 :info :write [29 14] :RequestTimeoutException INFO [2022-01-10 05:38:35,182] jepsen worker 87 - jepsen.util 1167 :invoke :write [29 6] INFO [2022-01-10 05:38:35,242] jepsen worker 8 - jepsen.util 728 :ok :read [2 21] INFO [2022-01-10 05:38:35,243] jepsen worker 8 - jepsen.util 728 :invoke :read [2 nil] INFO [2022-01-10 05:38:35,252] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.0935672514619883}, :ns_1@172.23.107.227 {:progress 0.05865102639296182}, :ns_1@172.23.105.187 {:progress 0.05865102639296182}, :ns_1@172.23.107.228 {:progress 0.08235294117647063}} INFO [2022-01-10 05:38:35,252] jepsen worker nemesis - couchbase.util Rebalance check count: 72 INFO [2022-01-10 05:38:35,252] jepsen worker nemesis - couchbase.util Rebalance stuck count: 58 INFO [2022-01-10 05:38:35,285] jepsen worker 8 - jepsen.util 728 :ok :read [2 21] INFO [2022-01-10 05:38:35,287] jepsen worker 8 - jepsen.util 728 :invoke :read [2 nil] INFO [2022-01-10 05:38:35,293] jepsen worker 22 - jepsen.util 652 :info :write [7 7] :RequestTimeoutException INFO [2022-01-10 05:38:35,293] jepsen worker 22 - jepsen.util 742 :invoke :write [7 44] INFO [2022-01-10 05:38:35,311] jepsen worker 57 - jepsen.util 957 :ok :read [19 31] INFO [2022-01-10 05:38:35,312] jepsen worker 57 - jepsen.util 957 :invoke :write [19 8] INFO [2022-01-10 05:38:35,325] jepsen worker 68 - jepsen.util 788 :ok :read [22 41] INFO [2022-01-10 05:38:35,326] jepsen worker 68 - jepsen.util 788 :invoke :write [22 7] INFO [2022-01-10 05:38:35,339] jepsen worker 8 - jepsen.util 728 :ok :read [2 21] INFO [2022-01-10 05:38:35,340] jepsen worker 8 - jepsen.util 728 :invoke :read [2 nil] INFO [2022-01-10 05:38:35,379] jepsen worker 1 - jepsen.util 811 :ok :read [0 :nil] INFO [2022-01-10 05:38:35,379] jepsen worker 1 - jepsen.util 811 :invoke :write [0 39] INFO [2022-01-10 05:38:35,398] jepsen worker 8 - jepsen.util 728 :ok :read [2 21] INFO [2022-01-10 05:38:35,400] jepsen worker 8 - jepsen.util 728 :invoke :read [2 nil] INFO [2022-01-10 05:38:35,533] jepsen worker 8 - jepsen.util 728 :ok :read [2 21] INFO [2022-01-10 05:38:35,534] jepsen worker 8 - jepsen.util 728 :invoke :write [2 47] INFO [2022-01-10 05:38:35,656] jepsen worker 23 - jepsen.util 923 :info :write [7 24] :RequestTimeoutException INFO [2022-01-10 05:38:35,657] jepsen worker 23 - jepsen.util 1013 :invoke :read [7 nil] INFO [2022-01-10 05:38:35,708] jepsen worker 89 - jepsen.util 989 :ok :read [29 33] INFO [2022-01-10 05:38:35,709] jepsen worker 89 - jepsen.util 989 :invoke :write [29 32] INFO [2022-01-10 05:38:35,856] jepsen worker 37 - jepsen.util 937 :ok :read [12 :nil] INFO [2022-01-10 05:38:35,856] jepsen worker 37 - jepsen.util 937 :invoke :write [12 2] INFO [2022-01-10 05:38:35,859] jepsen worker 80 - jepsen.util 890 :ok :read [26 10] INFO [2022-01-10 05:38:35,860] jepsen worker 80 - jepsen.util 890 :invoke :read [26 nil] INFO [2022-01-10 05:38:35,874] jepsen worker 9 - jepsen.util 1089 :ok :read [3 2] INFO [2022-01-10 05:38:35,875] jepsen worker 9 - jepsen.util 1089 :invoke :read [3 nil] INFO [2022-01-10 05:38:35,889] jepsen worker 52 - jepsen.util 862 :fail :write [17 14] :DurabilityImpossible INFO [2022-01-10 05:38:35,890] jepsen worker 52 - jepsen.util 862 :invoke :write [17 47] INFO [2022-01-10 05:38:35,926] jepsen worker 80 - jepsen.util 890 :ok :read [26 10] INFO [2022-01-10 05:38:35,927] jepsen worker 80 - jepsen.util 890 :invoke :read [26 nil] INFO [2022-01-10 05:38:36,258] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.09941520467836262}, :ns_1@172.23.107.227 {:progress 0.05865102639296182}, :ns_1@172.23.105.187 {:progress 0.05865102639296182}, :ns_1@172.23.107.228 {:progress 0.08235294117647063}} INFO [2022-01-10 05:38:36,258] jepsen worker nemesis - couchbase.util Rebalance check count: 73 INFO [2022-01-10 05:38:36,308] jepsen worker nemesis - couchbase.util Rebalance stuck count: 58 INFO [2022-01-10 05:38:36,355] jepsen worker 9 - jepsen.util 1089 :ok :read [3 2] INFO [2022-01-10 05:38:36,355] jepsen worker 9 - jepsen.util 1089 :invoke :read [3 nil] INFO [2022-01-10 05:38:36,655] jepsen worker 60 - jepsen.util 1050 :info :write [20 31] :RequestTimeoutException INFO [2022-01-10 05:38:36,655] jepsen worker 60 - jepsen.util 1140 :invoke :write [20 5] INFO [2022-01-10 05:38:36,796] jepsen worker 80 - jepsen.util 890 :ok :read [26 10] INFO [2022-01-10 05:38:36,796] jepsen worker 80 - jepsen.util 890 :invoke :read [26 nil] INFO [2022-01-10 05:38:36,844] jepsen worker 7 - jepsen.util 817 :ok :write [2 14] INFO [2022-01-10 05:38:36,844] jepsen worker 7 - jepsen.util 817 :invoke :write [2 43] INFO [2022-01-10 05:38:36,994] jepsen worker 9 - jepsen.util 1089 :ok :read [3 2] INFO [2022-01-10 05:38:36,994] jepsen worker 9 - jepsen.util 1089 :invoke :read [3 nil] INFO [2022-01-10 05:38:37,166] jepsen worker 21 - jepsen.util 1101 :info :write [7 20] :RequestTimeoutException INFO [2022-01-10 05:38:37,167] jepsen worker 21 - jepsen.util 1191 :invoke :write [7 49] INFO [2022-01-10 05:38:37,316] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.09941520467836262}, :ns_1@172.23.107.227 {:progress 0.05865102639296182}, :ns_1@172.23.105.187 {:progress 0.05865102639296182}, :ns_1@172.23.107.228 {:progress 0.08235294117647063}} INFO [2022-01-10 05:38:37,316] jepsen worker nemesis - couchbase.util Rebalance check count: 74 INFO [2022-01-10 05:38:37,316] jepsen worker nemesis - couchbase.util Rebalance stuck count: 59 INFO [2022-01-10 05:38:37,338] jepsen worker 80 - jepsen.util 890 :ok :read [26 10] INFO [2022-01-10 05:38:37,339] jepsen worker 80 - jepsen.util 890 :invoke :write [26 35] INFO [2022-01-10 05:38:37,365] jepsen worker 70 - jepsen.util 1150 :info :write [23 13] :RequestTimeoutException INFO [2022-01-10 05:38:37,366] jepsen worker 26 - jepsen.util 926 :info :write [8 24] :RequestTimeoutException INFO [2022-01-10 05:38:37,366] jepsen worker 26 - jepsen.util 1016 :invoke :write [8 15] INFO [2022-01-10 05:38:37,367] jepsen worker 70 - jepsen.util 1240 :invoke :write [23 22] INFO [2022-01-10 05:38:37,454] jepsen worker 9 - jepsen.util 1089 :ok :read [3 2] INFO [2022-01-10 05:38:37,454] jepsen worker 9 - jepsen.util 1089 :invoke :read [3 nil] INFO [2022-01-10 05:38:37,516] jepsen worker 2 - jepsen.util 902 :info :write [0 39] :RequestTimeoutException INFO [2022-01-10 05:38:37,517] jepsen worker 2 - jepsen.util 992 :invoke :write [0 30] INFO [2022-01-10 05:38:37,521] jepsen worker 15 - jepsen.util 555 :info :write [5 11] :RequestTimeoutException INFO [2022-01-10 05:38:37,522] jepsen worker 15 - jepsen.util 645 :invoke :read [5 nil] INFO [2022-01-10 05:38:37,609] jepsen worker 58 - jepsen.util 868 :info :write [19 29] :SyncWriteAmbiguous INFO [2022-01-10 05:38:37,609] jepsen worker 58 - jepsen.util 958 :invoke :read [19 nil] INFO [2022-01-10 05:38:37,662] jepsen worker 9 - jepsen.util 1089 :ok :read [3 2] INFO [2022-01-10 05:38:37,663] jepsen worker 9 - jepsen.util 1089 :invoke :read [3 nil] INFO [2022-01-10 05:38:37,697] jepsen worker 9 - jepsen.util 1089 :ok :read [3 2] INFO [2022-01-10 05:38:37,698] jepsen worker 9 - jepsen.util 1089 :invoke :read [3 nil] INFO [2022-01-10 05:38:37,710] jepsen worker 14 - jepsen.util 914 :info :write [4 9] :RequestTimeoutException INFO [2022-01-10 05:38:37,711] jepsen worker 14 - jepsen.util 1004 :invoke :write [4 42] INFO [2022-01-10 05:38:37,721] jepsen worker 39 - jepsen.util 1029 :ok :read [13 15] INFO [2022-01-10 05:38:37,722] jepsen worker 39 - jepsen.util 1029 :invoke :write [13 38] INFO [2022-01-10 05:38:37,745] jepsen worker 9 - jepsen.util 1089 :ok :read [3 2] INFO [2022-01-10 05:38:37,746] jepsen worker 9 - jepsen.util 1089 :invoke :write [3 28] INFO [2022-01-10 05:38:37,751] jepsen worker 3 - jepsen.util 993 :info :write [1 18] :RequestTimeoutException INFO [2022-01-10 05:38:37,752] jepsen worker 3 - jepsen.util 1083 :invoke :write [1 23] INFO [2022-01-10 05:38:37,771] jepsen worker 66 - jepsen.util 606 :info :write [22 46] :RequestTimeoutException INFO [2022-01-10 05:38:37,772] jepsen worker 66 - jepsen.util 696 :invoke :write [22 44] INFO [2022-01-10 05:38:37,795] jepsen worker 20 - jepsen.util 650 :info :write [6 43] :RequestTimeoutException INFO [2022-01-10 05:38:37,797] jepsen worker 20 - jepsen.util 740 :invoke :write [6 24] INFO [2022-01-10 05:38:37,924] jepsen worker 5 - jepsen.util 815 :info :write [1 24] :RequestTimeoutException INFO [2022-01-10 05:38:37,925] jepsen worker 5 - jepsen.util 905 :invoke :read [1 nil] INFO [2022-01-10 05:38:38,105] jepsen worker 85 - jepsen.util 985 :info :write [28 22] :RequestTimeoutException INFO [2022-01-10 05:38:38,105] jepsen worker 85 - jepsen.util 1075 :invoke :read [28 nil] INFO [2022-01-10 05:38:38,115] jepsen worker 28 - jepsen.util 838 :info :write [9 22] :RequestTimeoutException INFO [2022-01-10 05:38:38,116] jepsen worker 28 - jepsen.util 928 :invoke :write [9 19] INFO [2022-01-10 05:38:38,151] jepsen worker 67 - jepsen.util 877 :ok :read [22 41] INFO [2022-01-10 05:38:38,152] jepsen worker 67 - jepsen.util 877 :invoke :read [22 nil] INFO [2022-01-10 05:38:38,206] jepsen worker 75 - jepsen.util 705 :info :write [25 36] :RequestTimeoutException INFO [2022-01-10 05:38:38,207] jepsen worker 75 - jepsen.util 795 :invoke :read [25 nil] INFO [2022-01-10 05:38:38,232] jepsen worker 32 - jepsen.util 842 :info :write [10 12] :RequestTimeoutException INFO [2022-01-10 05:38:38,233] jepsen worker 32 - jepsen.util 932 :invoke :write [10 11] INFO [2022-01-10 05:38:38,323] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1052631578947368}, :ns_1@172.23.107.227 {:progress 0.05865102639296182}, :ns_1@172.23.105.187 {:progress 0.05865102639296182}, :ns_1@172.23.107.228 {:progress 0.08235294117647063}} INFO [2022-01-10 05:38:38,323] jepsen worker nemesis - couchbase.util Rebalance check count: 75 INFO [2022-01-10 05:38:38,323] jepsen worker nemesis - couchbase.util Rebalance stuck count: 59 INFO [2022-01-10 05:38:38,340] jepsen worker 67 - jepsen.util 877 :ok :read [22 41] INFO [2022-01-10 05:38:38,341] jepsen worker 67 - jepsen.util 877 :invoke :read [22 nil] INFO [2022-01-10 05:38:38,405] jepsen worker 43 - jepsen.util 853 :info :write [14 0] :RequestTimeoutException INFO [2022-01-10 05:38:38,406] jepsen worker 43 - jepsen.util 943 :invoke :write [14 26] INFO [2022-01-10 05:38:38,457] jepsen worker 67 - jepsen.util 877 :ok :read [22 41] INFO [2022-01-10 05:38:38,457] jepsen worker 67 - jepsen.util 877 :invoke :write [22 17] INFO [2022-01-10 05:38:38,540] jepsen worker 65 - jepsen.util 965 :info :write [21 7] :RequestTimeoutException INFO [2022-01-10 05:38:38,541] jepsen worker 65 - jepsen.util 1055 :invoke :read [21 nil] INFO [2022-01-10 05:38:38,615] jepsen worker 16 - jepsen.util 916 :info :write [5 4] :RequestTimeoutException INFO [2022-01-10 05:38:38,616] jepsen worker 16 - jepsen.util 1006 :invoke :write [5 4] INFO [2022-01-10 05:38:38,982] jepsen worker 74 - jepsen.util 1154 :info :write [24 4] :RequestTimeoutException INFO [2022-01-10 05:38:38,983] jepsen worker 74 - jepsen.util 1244 :invoke :read [24 nil] INFO [2022-01-10 05:38:39,241] jepsen worker 29 - jepsen.util 839 :info :write [9 43] :RequestTimeoutException INFO [2022-01-10 05:38:39,242] jepsen worker 29 - jepsen.util 929 :invoke :read [9 nil] INFO [2022-01-10 05:38:39,329] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1111111111111112}, :ns_1@172.23.107.227 {:progress 0.05865102639296182}, :ns_1@172.23.105.187 {:progress 0.05865102639296182}, :ns_1@172.23.107.228 {:progress 0.09411764705882353}} INFO [2022-01-10 05:38:39,329] jepsen worker nemesis - couchbase.util Rebalance check count: 76 INFO [2022-01-10 05:38:39,329] jepsen worker nemesis - couchbase.util Rebalance stuck count: 59 INFO [2022-01-10 05:38:39,415] jepsen worker 17 - jepsen.util 737 :info :write [5 17] :RequestTimeoutException INFO [2022-01-10 05:38:39,416] jepsen worker 17 - jepsen.util 827 :invoke :write [5 13] WARN [2022-01-10 05:38:39,619] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":19683612,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19683612,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10031689,"encode_duration_us":16340,"operation_id":"0x18ee","timeout_ms":10000,"last_server_duration_us":10031689,"total_duration_us":10006476}],"total_count":1}} WARN [2022-01-10 05:38:39,720] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":364122,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":364122,"total_server_duration_us":10996,"operation_id":"0x1b25","timeout_ms":30000,"last_server_duration_us":10996,"total_duration_us":13696683},{"operation_name":"get","last_dispatch_duration_us":859562,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":859562,"total_server_duration_us":27,"operation_id":"0x1bcf","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":10613454},{"operation_name":"upsert","last_dispatch_duration_us":2804855,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":2804855,"total_server_duration_us":288612,"encode_duration_us":16479,"operation_id":"0x1d0d","timeout_ms":10000,"last_server_duration_us":288612,"total_duration_us":10010103},{"operation_name":"upsert","last_dispatch_duration_us":116245,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9092181,"total_server_duration_us":273916,"encode_duration_us":19460,"operation_id":"0x1d22","timeout_ms":10000,"last_server_duration_us":45838,"total_duration_us":10009954},{"operation_name":"upsert","last_dispatch_duration_us":7695369,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9424610,"total_server_duration_us":238896,"encode_duration_us":19112,"operation_id":"0x1d0f","timeout_ms":10000,"last_server_duration_us":58128,"total_duration_us":10009532},{"operation_name":"upsert","last_dispatch_duration_us":884474,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":884474,"total_server_duration_us":607950,"encode_duration_us":16568,"operation_id":"0x1d20","timeout_ms":10000,"last_server_duration_us":607950,"total_duration_us":10009007},{"operation_name":"upsert","last_dispatch_duration_us":1916084,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3674535,"total_server_duration_us":395155,"encode_duration_us":17301,"operation_id":"0x1cb3","timeout_ms":10000,"last_server_duration_us":155181,"total_duration_us":10006417},{"operation_name":"upsert","last_dispatch_duration_us":7633080,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14748448,"total_server_duration_us":1043582,"encode_duration_us":17099,"operation_id":"0x1c02","timeout_ms":10000,"last_server_duration_us":20045,"total_duration_us":10004810},{"operation_name":"upsert","last_dispatch_duration_us":301741,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8475779,"total_server_duration_us":155634,"encode_duration_us":16135,"operation_id":"0x1d4f","timeout_ms":10000,"last_server_duration_us":30162,"total_duration_us":10002638},{"operation_name":"upsert","last_dispatch_duration_us":140916,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9419224,"total_server_duration_us":377174,"encode_duration_us":20899,"operation_id":"0x1d0e","timeout_ms":10000,"last_server_duration_us":45950,"total_duration_us":10002143}],"total_count":15}} INFO [2022-01-10 05:38:39,720] jepsen worker 6 - jepsen.util 726 :info :write [2 6] :RequestTimeoutException INFO [2022-01-10 05:38:39,721] jepsen worker 6 - jepsen.util 816 :invoke :write [2 9] INFO [2022-01-10 05:38:39,800] jepsen worker 55 - jepsen.util 865 :info :write [18 8] :RequestTimeoutException INFO [2022-01-10 05:38:39,801] jepsen worker 55 - jepsen.util 955 :invoke :write [18 11] INFO [2022-01-10 05:38:40,338] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1111111111111112}, :ns_1@172.23.107.227 {:progress 0.05865102639296182}, :ns_1@172.23.105.187 {:progress 0.05865102639296182}, :ns_1@172.23.107.228 {:progress 0.09411764705882353}} INFO [2022-01-10 05:38:40,338] jepsen worker nemesis - couchbase.util Rebalance check count: 77 INFO [2022-01-10 05:38:40,338] jepsen worker nemesis - couchbase.util Rebalance stuck count: 60 INFO [2022-01-10 05:38:40,447] jepsen worker 56 - jepsen.util 956 :info :write [18 18] :RequestTimeoutException INFO [2022-01-10 05:38:40,447] jepsen worker 56 - jepsen.util 1046 :invoke :read [18 nil] INFO [2022-01-10 05:38:40,561] jepsen worker 61 - jepsen.util 961 :info :write [20 31] :RequestTimeoutException INFO [2022-01-10 05:38:40,561] jepsen worker 61 - jepsen.util 1051 :invoke :write [20 16] INFO [2022-01-10 05:38:40,566] jepsen worker 78 - jepsen.util 708 :info :write [26 41] :RequestTimeoutException INFO [2022-01-10 05:38:40,567] jepsen worker 78 - jepsen.util 798 :invoke :write [26 33] INFO [2022-01-10 05:38:40,661] jepsen worker 88 - jepsen.util 988 :info :write [29 7] :RequestTimeoutException INFO [2022-01-10 05:38:40,662] jepsen worker 88 - jepsen.util 1078 :invoke :write [29 43] INFO [2022-01-10 05:38:40,776] jepsen worker 11 - jepsen.util 731 :info :write [3 2] :RequestTimeoutException INFO [2022-01-10 05:38:40,777] jepsen worker 11 - jepsen.util 821 :invoke :write [3 11] INFO [2022-01-10 05:38:40,957] jepsen worker 79 - jepsen.util 889 :ok :read [26 10] INFO [2022-01-10 05:38:40,957] jepsen worker 79 - jepsen.util 889 :invoke :write [26 43] INFO [2022-01-10 05:38:41,024] jepsen worker 30 - jepsen.util 570 :ok :read [10 37] INFO [2022-01-10 05:38:41,025] jepsen worker 30 - jepsen.util 570 :invoke :write [10 48] INFO [2022-01-10 05:38:41,026] jepsen worker 81 - jepsen.util 801 :info :write [27 1] :RequestTimeoutException INFO [2022-01-10 05:38:41,027] jepsen worker 81 - jepsen.util 891 :invoke :write [27 23] INFO [2022-01-10 05:38:41,042] jepsen worker 36 - jepsen.util 576 :info :write [12 13] :RequestTimeoutException INFO [2022-01-10 05:38:41,043] jepsen worker 36 - jepsen.util 666 :invoke :read [12 nil] WARN [2022-01-10 05:38:41,164] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14941245,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18699326,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9333806,"encode_duration_us":16405,"operation_id":"0x1bd7","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10003742},{"operation_name":"upsert","last_dispatch_duration_us":18127909,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18127909,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9762519,"encode_duration_us":16404,"operation_id":"0x1afa","timeout_ms":10000,"last_server_duration_us":9762519,"total_duration_us":10004773}],"total_count":2}} WARN [2022-01-10 05:38:41,182] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":19246004,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":19246004,"total_server_duration_us":18799,"operation_id":"0x1a71","timeout_ms":30000,"last_server_duration_us":18799,"total_duration_us":19246453},{"operation_name":"get","last_dispatch_duration_us":18034359,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":18034359,"total_server_duration_us":49,"operation_id":"0x1abb","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":18036496},{"operation_name":"get","last_dispatch_duration_us":10180826,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":10180826,"total_server_duration_us":112147,"operation_id":"0x1c00","timeout_ms":30000,"last_server_duration_us":112147,"total_duration_us":12290768},{"operation_name":"get","last_dispatch_duration_us":1738274,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1738274,"total_server_duration_us":10332,"operation_id":"0x1c21","timeout_ms":30000,"last_server_duration_us":10332,"total_duration_us":10990820},{"operation_name":"get","last_dispatch_duration_us":1210920,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":1210920,"total_server_duration_us":151245,"operation_id":"0x1c89","timeout_ms":30000,"last_server_duration_us":151245,"total_duration_us":10459715},{"operation_name":"upsert","last_dispatch_duration_us":235891,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":2176441,"total_server_duration_us":176270,"encode_duration_us":18181,"operation_id":"0x1cf6","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10010327},{"operation_name":"upsert","last_dispatch_duration_us":492716,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3371584,"total_server_duration_us":171892,"encode_duration_us":21941,"operation_id":"0x1d7e","timeout_ms":10000,"last_server_duration_us":70825,"total_duration_us":10008254},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":16409,"operation_id":"0x1c22","timeout_ms":10000,"total_duration_us":10007630},{"operation_name":"upsert","last_dispatch_duration_us":10607523,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":10607523,"total_server_duration_us":299493,"encode_duration_us":16214,"operation_id":"0x1bff","timeout_ms":10000,"last_server_duration_us":299493,"total_duration_us":10007468},{"operation_name":"upsert","last_dispatch_duration_us":96096,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9416400,"total_server_duration_us":215171,"encode_duration_us":16525,"operation_id":"0x1d4d","timeout_ms":10000,"last_server_duration_us":24583,"total_duration_us":10007012}],"total_count":23}} WARN [2022-01-10 05:38:41,182] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9221964,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10056924,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9656569,"encode_duration_us":18369,"operation_id":"0x1ce5","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10002817},{"operation_name":"upsert","last_dispatch_duration_us":11321540,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17172250,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":10031254,"encode_duration_us":18040,"operation_id":"0x1af0","timeout_ms":10000,"last_server_duration_us":9298987,"total_duration_us":10009287}],"total_count":2}} WARN [2022-01-10 05:38:41,262] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18532218,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":18532218,"total_server_duration_us":27132,"operation_id":"0x18e5","timeout_ms":30000,"last_server_duration_us":27132,"total_duration_us":30010505},{"operation_name":"get","last_dispatch_duration_us":18293074,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":18293074,"total_server_duration_us":27,"operation_id":"0x18c0","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":30009165},{"operation_name":"get","last_dispatch_duration_us":18210290,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":18210290,"total_server_duration_us":49,"operation_id":"0x1988","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":26951723},{"operation_name":"get","last_dispatch_duration_us":18325584,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":18325584,"total_server_duration_us":27,"operation_id":"0x1ae7","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":19416172},{"operation_name":"get","last_dispatch_duration_us":17175362,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":17175362,"total_server_duration_us":2134,"operation_id":"0x1b51","timeout_ms":30000,"last_server_duration_us":2134,"total_duration_us":17175757},{"operation_name":"upsert","last_dispatch_duration_us":1204171,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":2442996,"total_server_duration_us":466862,"encode_duration_us":19396,"operation_id":"0x1d94","timeout_ms":10000,"last_server_duration_us":399905,"total_duration_us":10010803},{"operation_name":"upsert","last_dispatch_duration_us":5023139,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11788484,"total_server_duration_us":865431,"encode_duration_us":16695,"operation_id":"0x1c7f","timeout_ms":10000,"last_server_duration_us":20841,"total_duration_us":10009425},{"operation_name":"upsert","last_dispatch_duration_us":1024015,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":1690076,"total_server_duration_us":286685,"encode_duration_us":16483,"operation_id":"0x1d7b","timeout_ms":10000,"last_server_duration_us":112969,"total_duration_us":10008727},{"operation_name":"upsert","last_dispatch_duration_us":6022716,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8116623,"total_server_duration_us":39980,"encode_duration_us":16861,"operation_id":"0x1cdc","timeout_ms":10000,"last_server_duration_us":18493,"total_duration_us":10008290},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":23644,"operation_id":"0x1da2","timeout_ms":10000,"total_duration_us":10004571}],"total_count":19}} WARN [2022-01-10 05:38:41,262] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9152750,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9152750,"total_server_duration_us":251659,"operation_id":"0x1a3b","timeout_ms":30000,"last_server_duration_us":251659,"total_duration_us":20955318},{"operation_name":"get","last_dispatch_duration_us":9143499,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9143499,"total_server_duration_us":334459,"operation_id":"0x1ac8","timeout_ms":30000,"last_server_duration_us":334459,"total_duration_us":17372004},{"operation_name":"get","last_dispatch_duration_us":6129915,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6129915,"total_server_duration_us":43,"operation_id":"0x1bd5","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":12822603},{"operation_name":"upsert","last_dispatch_duration_us":8839629,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10049748,"total_server_duration_us":196779,"encode_duration_us":26855,"operation_id":"0x1da3","timeout_ms":10000,"last_server_duration_us":25625,"total_duration_us":10008352},{"operation_name":"upsert","last_dispatch_duration_us":9170863,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":15831656,"total_server_duration_us":1484085,"encode_duration_us":16203,"operation_id":"0x1c86","timeout_ms":10000,"last_server_duration_us":20680,"total_duration_us":10008029},{"operation_name":"upsert","last_dispatch_duration_us":8602531,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10504564,"total_server_duration_us":173257,"encode_duration_us":16539,"operation_id":"0x1d80","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10006338},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16302,"operation_id":"0x1d44","timeout_ms":10000,"total_duration_us":10004021},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":19009,"operation_id":"0x1c88","timeout_ms":10000,"total_duration_us":10003564},{"operation_name":"upsert","last_dispatch_duration_us":1623817,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":6917591,"total_server_duration_us":1358050,"encode_duration_us":16149,"operation_id":"0x1c2e","timeout_ms":10000,"last_server_duration_us":582585,"total_duration_us":10002883},{"operation_name":"upsert","last_dispatch_duration_us":10009283,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":12770973,"total_server_duration_us":9297706,"encode_duration_us":16809,"operation_id":"0x1d1e","timeout_ms":10000,"last_server_duration_us":9079863,"total_duration_us":10002678}],"total_count":11}} WARN [2022-01-10 05:38:41,264] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11307784,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11307784,"total_server_duration_us":27,"operation_id":"0x1c2c","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":11310464},{"operation_name":"get","last_dispatch_duration_us":10037948,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10037948,"total_server_duration_us":15,"operation_id":"0x1c79","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":10040783},{"operation_name":"upsert","last_dispatch_duration_us":9894636,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":16781790,"total_server_duration_us":384628,"encode_duration_us":24614,"operation_id":"0x1c51","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10008577},{"operation_name":"upsert","last_dispatch_duration_us":113195,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8103148,"total_server_duration_us":176765,"encode_duration_us":16619,"operation_id":"0x1ce2","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10007953},{"operation_name":"upsert","last_dispatch_duration_us":1486810,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7326622,"total_server_duration_us":810119,"encode_duration_us":16596,"operation_id":"0x1ce0","timeout_ms":10000,"last_server_duration_us":97947,"total_duration_us":10007736},{"operation_name":"upsert","last_dispatch_duration_us":10034100,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10034100,"total_server_duration_us":20045,"encode_duration_us":17866,"operation_id":"0x1c7a","timeout_ms":10000,"last_server_duration_us":20045,"total_duration_us":10007332},{"operation_name":"upsert","last_dispatch_duration_us":8637029,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":16851799,"total_server_duration_us":528208,"encode_duration_us":16312,"operation_id":"0x1c4f","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10006965},{"operation_name":"upsert","last_dispatch_duration_us":14941245,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":18699326,"total_server_duration_us":9333806,"encode_duration_us":16405,"operation_id":"0x1bd7","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10003742},{"operation_name":"upsert","last_dispatch_duration_us":931776,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5641296,"total_server_duration_us":180243,"encode_duration_us":17605,"operation_id":"0x1d1c","timeout_ms":10000,"last_server_duration_us":92589,"total_duration_us":10003571},{"operation_name":"upsert","last_dispatch_duration_us":8667008,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":16255220,"total_server_duration_us":1722295,"encode_duration_us":16385,"operation_id":"0x1c5a","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10001786}],"total_count":25}} WARN [2022-01-10 05:38:41,265] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":26619593,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":26619593,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9029938,"encode_duration_us":16801,"operation_id":"0x194e","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10005393}],"total_count":1}} INFO [2022-01-10 05:38:41,297] jepsen worker 5 - jepsen.util 905 :ok :read [1 41] INFO [2022-01-10 05:38:41,298] jepsen worker 5 - jepsen.util 905 :invoke :read [1 nil] INFO [2022-01-10 05:38:41,325] jepsen worker 63 - jepsen.util 873 :info :write [21 17] :RequestTimeoutException INFO [2022-01-10 05:38:41,326] jepsen worker 63 - jepsen.util 963 :invoke :read [21 nil] INFO [2022-01-10 05:38:41,346] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1169590643274854}, :ns_1@172.23.107.227 {:progress 0.05865102639296182}, :ns_1@172.23.105.187 {:progress 0.05865102639296182}, :ns_1@172.23.107.228 {:progress 0.09411764705882353}} INFO [2022-01-10 05:38:41,346] jepsen worker nemesis - couchbase.util Rebalance check count: 78 INFO [2022-01-10 05:38:41,346] jepsen worker nemesis - couchbase.util Rebalance stuck count: 60 WARN [2022-01-10 05:38:41,363] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":36195775,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":36195775,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9389415,"encode_duration_us":19583,"operation_id":"0x17cc","timeout_ms":10000,"last_server_duration_us":9389415,"total_duration_us":10004831},{"operation_name":"get","last_dispatch_duration_us":18437967,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18437967,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":38,"operation_id":"0x182c","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":30008393},{"operation_name":"get","last_dispatch_duration_us":18293074,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18293074,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":27,"operation_id":"0x18c0","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":30009165},{"operation_name":"get","last_dispatch_duration_us":18532218,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18532218,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":27132,"operation_id":"0x18e5","timeout_ms":30000,"last_server_duration_us":27132,"total_duration_us":30010505}],"total_count":4}} WARN [2022-01-10 05:38:41,364] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":24884316,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":24884316,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9219632,"encode_duration_us":16426,"operation_id":"0x19b8","timeout_ms":10000,"last_server_duration_us":9219632,"total_duration_us":10002304},{"operation_name":"upsert","last_dispatch_duration_us":27084351,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":27084351,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":5812823,"encode_duration_us":16146,"operation_id":"0x19e6","timeout_ms":10000,"last_server_duration_us":5812823,"total_duration_us":10006707},{"operation_name":"upsert","last_dispatch_duration_us":10009283,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12770973,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9297706,"encode_duration_us":16809,"operation_id":"0x1d1e","timeout_ms":10000,"last_server_duration_us":9079863,"total_duration_us":10002678}],"total_count":3}} WARN [2022-01-10 05:38:41,466] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":24910417,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":24910417,"total_server_duration_us":3763,"operation_id":"0x19b7","timeout_ms":30000,"last_server_duration_us":3763,"total_duration_us":24911708},{"operation_name":"upsert","last_dispatch_duration_us":902397,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8406086,"total_server_duration_us":2268079,"encode_duration_us":23603,"operation_id":"0x1bd4","timeout_ms":10000,"last_server_duration_us":93650,"total_duration_us":10010539},{"operation_name":"upsert","last_dispatch_duration_us":530583,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":4650030,"total_server_duration_us":644733,"encode_duration_us":16814,"operation_id":"0x1d6b","timeout_ms":10000,"last_server_duration_us":105664,"total_duration_us":10010308},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":20424,"operation_id":"0x1bd2","timeout_ms":10000,"total_duration_us":10007217},{"operation_name":"upsert","last_dispatch_duration_us":3661394,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":3661394,"total_server_duration_us":164917,"encode_duration_us":16801,"operation_id":"0x1d21","timeout_ms":10000,"last_server_duration_us":164917,"total_duration_us":10005812},{"operation_name":"upsert","last_dispatch_duration_us":921250,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":7221593,"total_server_duration_us":844706,"encode_duration_us":26453,"operation_id":"0x1cee","timeout_ms":10000,"last_server_duration_us":108237,"total_duration_us":10005247},{"operation_name":"upsert","last_dispatch_duration_us":507695,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":3148817,"total_server_duration_us":371300,"encode_duration_us":17689,"operation_id":"0x1d58","timeout_ms":10000,"last_server_duration_us":219495,"total_duration_us":10004215},{"operation_name":"upsert","last_dispatch_duration_us":1196204,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":7684625,"total_server_duration_us":723415,"encode_duration_us":16351,"operation_id":"0x1bfe","timeout_ms":10000,"last_server_duration_us":22637,"total_duration_us":10002997},{"operation_name":"get","last_dispatch_duration_us":5192825,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5192825,"total_server_duration_us":32,"operation_id":"0x1cd3","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":5193294},{"operation_name":"get","last_dispatch_duration_us":1511193,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":1511193,"total_server_duration_us":19809,"operation_id":"0x1d36","timeout_ms":30000,"last_server_duration_us":19809,"total_duration_us":3619312}],"total_count":14}} INFO [2022-01-10 05:38:41,533] jepsen worker 64 - jepsen.util 874 :ok :read [21 7] INFO [2022-01-10 05:38:41,534] jepsen worker 64 - jepsen.util 874 :invoke :read [21 nil] INFO [2022-01-10 05:38:41,585] jepsen worker 31 - jepsen.util 1111 :info :write [10 48] :RequestTimeoutException INFO [2022-01-10 05:38:41,586] jepsen worker 31 - jepsen.util 1201 :invoke :write [10 35] INFO [2022-01-10 05:38:41,604] jepsen worker 5 - jepsen.util 905 :ok :read [1 41] INFO [2022-01-10 05:38:41,605] jepsen worker 5 - jepsen.util 905 :invoke :write [1 35] INFO [2022-01-10 05:38:41,886] jepsen worker 12 - jepsen.util 912 :info :write [4 17] :RequestTimeoutException INFO [2022-01-10 05:38:41,887] jepsen worker 12 - jepsen.util 1002 :invoke :write [4 20] INFO [2022-01-10 05:38:41,925] jepsen worker 62 - jepsen.util 962 :info :write [20 39] :RequestTimeoutException INFO [2022-01-10 05:38:41,925] jepsen worker 62 - jepsen.util 1052 :invoke :write [20 23] INFO [2022-01-10 05:38:42,342] jepsen worker 38 - jepsen.util 938 :info :write [12 29] :RequestTimeoutException INFO [2022-01-10 05:38:42,343] jepsen worker 38 - jepsen.util 1028 :invoke :read [12 nil] INFO [2022-01-10 05:38:42,354] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1169590643274854}, :ns_1@172.23.107.227 {:progress 0.05865102639296182}, :ns_1@172.23.105.187 {:progress 0.05865102639296182}, :ns_1@172.23.107.228 {:progress 0.09411764705882353}} INFO [2022-01-10 05:38:42,354] jepsen worker nemesis - couchbase.util Rebalance check count: 79 INFO [2022-01-10 05:38:42,354] jepsen worker nemesis - couchbase.util Rebalance stuck count: 61 INFO [2022-01-10 05:38:42,420] jepsen worker 77 - jepsen.util 1067 :info :write [25 45] :RequestTimeoutException INFO [2022-01-10 05:38:42,421] jepsen worker 77 - jepsen.util 1157 :invoke :read [25 nil] INFO [2022-01-10 05:38:42,430] jepsen worker 54 - jepsen.util 864 :info :write [18 3] :RequestTimeoutException INFO [2022-01-10 05:38:42,430] jepsen worker 54 - jepsen.util 954 :invoke :write [18 34] INFO [2022-01-10 05:38:42,440] jepsen worker 46 - jepsen.util 946 :info :write [15 41] :RequestTimeoutException INFO [2022-01-10 05:38:42,441] jepsen worker 46 - jepsen.util 1036 :invoke :read [15 nil] INFO [2022-01-10 05:38:42,516] jepsen worker 4 - jepsen.util 904 :info :write [1 1] :RequestTimeoutException INFO [2022-01-10 05:38:42,517] jepsen worker 4 - jepsen.util 994 :invoke :read [1 nil] INFO [2022-01-10 05:38:42,671] jepsen worker 75 - jepsen.util 795 :ok :read [25 :nil] INFO [2022-01-10 05:38:42,672] jepsen worker 75 - jepsen.util 795 :invoke :read [25 nil] INFO [2022-01-10 05:38:42,683] jepsen worker 64 - jepsen.util 874 :ok :read [21 7] INFO [2022-01-10 05:38:42,684] jepsen worker 64 - jepsen.util 874 :invoke :write [21 32] INFO [2022-01-10 05:38:42,739] jepsen worker 77 - jepsen.util 1157 :ok :read [25 :nil] INFO [2022-01-10 05:38:42,740] jepsen worker 77 - jepsen.util 1157 :invoke :read [25 nil] INFO [2022-01-10 05:38:42,754] jepsen worker 49 - jepsen.util 769 :ok :read [16 38] INFO [2022-01-10 05:38:42,755] jepsen worker 49 - jepsen.util 769 :invoke :read [16 nil] INFO [2022-01-10 05:38:42,804] jepsen worker 41 - jepsen.util 671 :ok :read [13 15] INFO [2022-01-10 05:38:42,804] jepsen worker 41 - jepsen.util 671 :invoke :read [13 nil] INFO [2022-01-10 05:38:42,808] jepsen worker 50 - jepsen.util 1040 :ok :read [16 38] INFO [2022-01-10 05:38:42,808] jepsen worker 50 - jepsen.util 1040 :invoke :read [16 nil] INFO [2022-01-10 05:38:42,815] jepsen worker 84 - jepsen.util 894 :ok :read [28 :nil] INFO [2022-01-10 05:38:42,816] jepsen worker 84 - jepsen.util 894 :invoke :read [28 nil] INFO [2022-01-10 05:38:42,831] jepsen worker 77 - jepsen.util 1157 :ok :read [25 :nil] INFO [2022-01-10 05:38:42,832] jepsen worker 77 - jepsen.util 1157 :invoke :read [25 nil] INFO [2022-01-10 05:38:42,836] jepsen worker 77 - jepsen.util 1157 :ok :read [25 :nil] INFO [2022-01-10 05:38:42,836] jepsen worker 77 - jepsen.util 1157 :invoke :write [25 20] INFO [2022-01-10 05:38:42,843] jepsen worker 38 - jepsen.util 1028 :ok :read [12 :nil] INFO [2022-01-10 05:38:42,843] jepsen worker 38 - jepsen.util 1028 :invoke :write [12 4] INFO [2022-01-10 05:38:42,889] jepsen worker 84 - jepsen.util 894 :ok :read [28 :nil] INFO [2022-01-10 05:38:42,890] jepsen worker 84 - jepsen.util 894 :invoke :write [28 21] INFO [2022-01-10 05:38:42,891] jepsen worker 49 - jepsen.util 769 :ok :read [16 38] INFO [2022-01-10 05:38:42,892] jepsen worker 49 - jepsen.util 769 :invoke :read [16 nil] INFO [2022-01-10 05:38:42,896] jepsen worker 41 - jepsen.util 671 :ok :read [13 15] INFO [2022-01-10 05:38:42,897] jepsen worker 41 - jepsen.util 671 :invoke :write [13 44] INFO [2022-01-10 05:38:42,903] jepsen worker 50 - jepsen.util 1040 :ok :read [16 38] INFO [2022-01-10 05:38:42,904] jepsen worker 50 - jepsen.util 1040 :invoke :read [16 nil] INFO [2022-01-10 05:38:42,916] jepsen worker 53 - jepsen.util 773 :ok :read [17 41] INFO [2022-01-10 05:38:42,917] jepsen worker 53 - jepsen.util 773 :invoke :read [17 nil] INFO [2022-01-10 05:38:42,953] jepsen worker 53 - jepsen.util 773 :ok :read [17 41] INFO [2022-01-10 05:38:42,954] jepsen worker 53 - jepsen.util 773 :invoke :read [17 nil] INFO [2022-01-10 05:38:42,966] jepsen worker 73 - jepsen.util 793 :info :write [24 17] :RequestTimeoutException INFO [2022-01-10 05:38:42,967] jepsen worker 73 - jepsen.util 883 :invoke :write [24 8] INFO [2022-01-10 05:38:42,981] jepsen worker 35 - jepsen.util 1025 :info :write [11 37] :RequestTimeoutException INFO [2022-01-10 05:38:42,982] jepsen worker 49 - jepsen.util 769 :ok :read [16 38] INFO [2022-01-10 05:38:42,982] jepsen worker 35 - jepsen.util 1115 :invoke :read [11 nil] INFO [2022-01-10 05:38:42,992] jepsen worker 49 - jepsen.util 769 :invoke :write [16 44] INFO [2022-01-10 05:38:43,004] jepsen worker 50 - jepsen.util 1040 :ok :read [16 38] INFO [2022-01-10 05:38:43,005] jepsen worker 50 - jepsen.util 1040 :invoke :write [16 7] INFO [2022-01-10 05:38:43,023] jepsen worker 53 - jepsen.util 773 :ok :read [17 41] INFO [2022-01-10 05:38:43,023] jepsen worker 53 - jepsen.util 773 :invoke :read [17 nil] INFO [2022-01-10 05:38:43,068] jepsen worker 36 - jepsen.util 666 :ok :read [12 :nil] INFO [2022-01-10 05:38:43,069] jepsen worker 36 - jepsen.util 666 :invoke :write [12 24] INFO [2022-01-10 05:38:43,081] jepsen worker 25 - jepsen.util 835 :info :write [8 36] :RequestTimeoutException INFO [2022-01-10 05:38:43,081] jepsen worker 42 - jepsen.util 852 :info :write [14 46] :RequestTimeoutException INFO [2022-01-10 05:38:43,082] jepsen worker 25 - jepsen.util 925 :invoke :write [8 44] INFO [2022-01-10 05:38:43,082] jepsen worker 42 - jepsen.util 942 :invoke :read [14 nil] INFO [2022-01-10 05:38:43,083] jepsen worker 56 - jepsen.util 1046 :ok :read [18 14] INFO [2022-01-10 05:38:43,083] jepsen worker 56 - jepsen.util 1046 :invoke :write [18 48] INFO [2022-01-10 05:38:43,092] jepsen worker 53 - jepsen.util 773 :ok :read [17 41] INFO [2022-01-10 05:38:43,093] jepsen worker 53 - jepsen.util 773 :invoke :write [17 23] INFO [2022-01-10 05:38:43,102] jepsen worker 19 - jepsen.util 919 :info :write [6 2] :RequestTimeoutException INFO [2022-01-10 05:38:43,103] jepsen worker 19 - jepsen.util 1009 :invoke :read [6 nil] INFO [2022-01-10 05:38:43,168] jepsen worker 33 - jepsen.util 933 :ok :read [11 21] INFO [2022-01-10 05:38:43,168] jepsen worker 33 - jepsen.util 933 :invoke :write [11 19] INFO [2022-01-10 05:38:43,172] jepsen worker 86 - jepsen.util 626 :ok :write [28 13] INFO [2022-01-10 05:38:43,172] jepsen worker 86 - jepsen.util 626 :invoke :write [28 40] INFO [2022-01-10 05:38:43,361] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1228070175438597}, :ns_1@172.23.107.227 {:progress 0.06451612903225812}, :ns_1@172.23.105.187 {:progress 0.06451612903225812}, :ns_1@172.23.107.228 {:progress 0.09999999999999998}} INFO [2022-01-10 05:38:43,361] jepsen worker nemesis - couchbase.util Rebalance check count: 80 INFO [2022-01-10 05:38:43,361] jepsen worker nemesis - couchbase.util Rebalance stuck count: 61 INFO [2022-01-10 05:38:43,426] jepsen worker 51 - jepsen.util 861 :info :write [17 29] :RequestTimeoutException INFO [2022-01-10 05:38:43,427] jepsen worker 51 - jepsen.util 951 :invoke :write [17 42] INFO [2022-01-10 05:38:43,643] jepsen worker 15 - jepsen.util 645 :ok :read [5 42] INFO [2022-01-10 05:38:43,643] jepsen worker 15 - jepsen.util 645 :invoke :write [5 19] INFO [2022-01-10 05:38:43,666] jepsen worker 10 - jepsen.util 910 :info :write [3 30] :RequestTimeoutException INFO [2022-01-10 05:38:43,667] jepsen worker 10 - jepsen.util 1000 :invoke :write [3 5] INFO [2022-01-10 05:38:43,712] jepsen worker 18 - jepsen.util 738 :ok :read [6 25] INFO [2022-01-10 05:38:43,713] jepsen worker 18 - jepsen.util 738 :invoke :write [6 23] INFO [2022-01-10 05:38:43,805] jepsen worker 44 - jepsen.util 944 :info :write [14 8] :RequestTimeoutException INFO [2022-01-10 05:38:43,807] jepsen worker 44 - jepsen.util 1034 :invoke :read [14 nil] INFO [2022-01-10 05:38:44,072] jepsen worker 45 - jepsen.util 1035 :info :write [15 19] :RequestTimeoutException INFO [2022-01-10 05:38:44,072] jepsen worker 45 - jepsen.util 1125 :invoke :write [15 37] INFO [2022-01-10 05:38:44,130] jepsen worker 69 - jepsen.util 969 :info :write [23 18] :RequestTimeoutException INFO [2022-01-10 05:38:44,131] jepsen worker 69 - jepsen.util 1059 :invoke :read [23 nil] INFO [2022-01-10 05:38:44,142] jepsen worker 13 - jepsen.util 913 :info :write [4 7] :RequestTimeoutException INFO [2022-01-10 05:38:44,142] jepsen worker 13 - jepsen.util 1003 :invoke :write [4 39] INFO [2022-01-10 05:38:44,227] jepsen worker 27 - jepsen.util 747 :info :write [9 20] :RequestTimeoutException INFO [2022-01-10 05:38:44,227] jepsen worker 27 - jepsen.util 837 :invoke :write [9 2] INFO [2022-01-10 05:38:44,271] jepsen worker 40 - jepsen.util 670 :info :write [13 28] :RequestTimeoutException INFO [2022-01-10 05:38:44,272] jepsen worker 40 - jepsen.util 760 :invoke :read [13 nil] INFO [2022-01-10 05:38:44,367] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1228070175438597}, :ns_1@172.23.107.227 {:progress 0.06451612903225812}, :ns_1@172.23.105.187 {:progress 0.06451612903225812}, :ns_1@172.23.107.228 {:progress 0.09999999999999998}} INFO [2022-01-10 05:38:44,367] jepsen worker nemesis - couchbase.util Rebalance check count: 81 INFO [2022-01-10 05:38:44,368] jepsen worker nemesis - couchbase.util Rebalance stuck count: 62 INFO [2022-01-10 05:38:44,417] jepsen worker 71 - jepsen.util 971 :info :write [23 46] :RequestTimeoutException INFO [2022-01-10 05:38:44,417] jepsen worker 71 - jepsen.util 1061 :invoke :write [23 24] INFO [2022-01-10 05:38:44,646] jepsen worker 47 - jepsen.util 857 :info :write [15 26] :RequestTimeoutException INFO [2022-01-10 05:38:44,647] jepsen worker 47 - jepsen.util 947 :invoke :write [15 5] INFO [2022-01-10 05:38:44,881] jepsen worker 24 - jepsen.util 834 :info :write [8 25] :RequestTimeoutException INFO [2022-01-10 05:38:44,882] jepsen worker 24 - jepsen.util 924 :invoke :read [8 nil] INFO [2022-01-10 05:38:44,936] jepsen worker 83 - jepsen.util 983 :info :write [27 35] :RequestTimeoutException INFO [2022-01-10 05:38:44,936] jepsen worker 0 - jepsen.util 810 :info :write [0 49] :RequestTimeoutException INFO [2022-01-10 05:38:44,936] jepsen worker 83 - jepsen.util 1073 :invoke :write [27 32] INFO [2022-01-10 05:38:44,937] jepsen worker 0 - jepsen.util 900 :invoke :read [0 nil] INFO [2022-01-10 05:38:45,086] jepsen worker 48 - jepsen.util 678 :info :write [16 28] :RequestTimeoutException INFO [2022-01-10 05:38:45,088] jepsen worker 48 - jepsen.util 768 :invoke :read [16 nil] INFO [2022-01-10 05:38:45,145] jepsen worker 72 - jepsen.util 1152 :info :write [24 25] :RequestTimeoutException INFO [2022-01-10 05:38:45,146] jepsen worker 72 - jepsen.util 1242 :invoke :read [24 nil] INFO [2022-01-10 05:38:45,176] jepsen worker 82 - jepsen.util 802 :info :write [27 35] :RequestTimeoutException INFO [2022-01-10 05:38:45,176] jepsen worker 82 - jepsen.util 892 :invoke :read [27 nil] INFO [2022-01-10 05:38:45,206] jepsen worker 87 - jepsen.util 1167 :info :write [29 6] :RequestTimeoutException INFO [2022-01-10 05:38:45,208] jepsen worker 87 - jepsen.util 1257 :invoke :write [29 19] INFO [2022-01-10 05:38:45,239] jepsen worker 59 - jepsen.util 1049 :ok :read [19 31] INFO [2022-01-10 05:38:45,240] jepsen worker 59 - jepsen.util 1049 :invoke :write [19 41] INFO [2022-01-10 05:38:45,320] jepsen worker 22 - jepsen.util 742 :info :write [7 44] :RequestTimeoutException INFO [2022-01-10 05:38:45,321] jepsen worker 22 - jepsen.util 832 :invoke :write [7 25] INFO [2022-01-10 05:38:45,337] jepsen worker 57 - jepsen.util 957 :info :write [19 8] :RequestTimeoutException INFO [2022-01-10 05:38:45,337] jepsen worker 57 - jepsen.util 1047 :invoke :read [19 nil] INFO [2022-01-10 05:38:45,346] jepsen worker 68 - jepsen.util 788 :info :write [22 7] :RequestTimeoutException INFO [2022-01-10 05:38:45,347] jepsen worker 68 - jepsen.util 878 :invoke :write [22 15] INFO [2022-01-10 05:38:45,374] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1286549707602339}, :ns_1@172.23.107.227 {:progress 0.0703812316715543}, :ns_1@172.23.105.187 {:progress 0.0703812316715543}, :ns_1@172.23.107.228 {:progress 0.1058823529411764}} INFO [2022-01-10 05:38:45,374] jepsen worker nemesis - couchbase.util Rebalance check count: 82 INFO [2022-01-10 05:38:45,374] jepsen worker nemesis - couchbase.util Rebalance stuck count: 62 INFO [2022-01-10 05:38:45,406] jepsen worker 1 - jepsen.util 811 :info :write [0 39] :RequestTimeoutException INFO [2022-01-10 05:38:45,407] jepsen worker 1 - jepsen.util 901 :invoke :read [0 nil] INFO [2022-01-10 05:38:45,435] jepsen worker 1 - jepsen.util 901 :ok :read [0 46] INFO [2022-01-10 05:38:45,436] jepsen worker 1 - jepsen.util 901 :invoke :write [0 35] INFO [2022-01-10 05:38:45,463] jepsen worker 40 - jepsen.util 760 :ok :read [13 15] INFO [2022-01-10 05:38:45,464] jepsen worker 40 - jepsen.util 760 :invoke :write [13 25] INFO [2022-01-10 05:38:45,561] jepsen worker 8 - jepsen.util 728 :info :write [2 47] :RequestTimeoutException INFO [2022-01-10 05:38:45,561] jepsen worker 8 - jepsen.util 818 :invoke :read [2 nil] INFO [2022-01-10 05:38:45,639] jepsen worker 89 - jepsen.util 989 :info :write [29 32] :SyncWriteAmbiguous INFO [2022-01-10 05:38:45,639] jepsen worker 89 - jepsen.util 1079 :invoke :read [29 nil] INFO [2022-01-10 05:38:45,881] jepsen worker 37 - jepsen.util 937 :info :write [12 2] :RequestTimeoutException INFO [2022-01-10 05:38:45,882] jepsen worker 37 - jepsen.util 1027 :invoke :write [12 31] INFO [2022-01-10 05:38:45,920] jepsen worker 52 - jepsen.util 862 :info :write [17 47] :RequestTimeoutException INFO [2022-01-10 05:38:45,921] jepsen worker 52 - jepsen.util 952 :invoke :write [17 49] INFO [2022-01-10 05:38:45,957] jepsen worker 82 - jepsen.util 892 :ok :read [27 30] INFO [2022-01-10 05:38:45,957] jepsen worker 0 - jepsen.util 900 :ok :read [0 46] INFO [2022-01-10 05:38:45,957] jepsen worker 82 - jepsen.util 892 :invoke :write [27 16] INFO [2022-01-10 05:38:45,958] jepsen worker 0 - jepsen.util 900 :invoke :read [0 nil] INFO [2022-01-10 05:38:46,022] jepsen worker 85 - jepsen.util 1075 :ok :read [28 13] INFO [2022-01-10 05:38:46,023] jepsen worker 85 - jepsen.util 1075 :invoke :read [28 nil] INFO [2022-01-10 05:38:46,043] jepsen worker 8 - jepsen.util 818 :ok :read [2 14] INFO [2022-01-10 05:38:46,044] jepsen worker 8 - jepsen.util 818 :invoke :write [2 0] INFO [2022-01-10 05:38:46,094] jepsen worker 58 - jepsen.util 958 :ok :read [19 31] INFO [2022-01-10 05:38:46,095] jepsen worker 58 - jepsen.util 958 :invoke :read [19 nil] INFO [2022-01-10 05:38:46,181] jepsen worker 75 - jepsen.util 795 :ok :read [25 :nil] INFO [2022-01-10 05:38:46,181] jepsen worker 75 - jepsen.util 795 :invoke :read [25 nil] INFO [2022-01-10 05:38:46,195] jepsen worker 46 - jepsen.util 1036 :ok :read [15 23] INFO [2022-01-10 05:38:46,195] jepsen worker 46 - jepsen.util 1036 :invoke :write [15 15] INFO [2022-01-10 05:38:46,387] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1286549707602339}, :ns_1@172.23.107.227 {:progress 0.0703812316715543}, :ns_1@172.23.105.187 {:progress 0.0703812316715543}, :ns_1@172.23.107.228 {:progress 0.1058823529411764}} INFO [2022-01-10 05:38:46,388] jepsen worker nemesis - couchbase.util Rebalance check count: 83 INFO [2022-01-10 05:38:46,437] jepsen worker nemesis - couchbase.util Rebalance stuck count: 63 INFO [2022-01-10 05:38:46,439] jepsen worker 75 - jepsen.util 795 :ok :read [25 :nil] INFO [2022-01-10 05:38:46,440] jepsen worker 75 - jepsen.util 795 :invoke :read [25 nil] INFO [2022-01-10 05:38:46,610] jepsen worker 57 - jepsen.util 1047 :ok :read [19 31] INFO [2022-01-10 05:38:46,611] jepsen worker 57 - jepsen.util 1047 :invoke :read [19 nil] INFO [2022-01-10 05:38:46,619] jepsen worker 19 - jepsen.util 1009 :ok :read [6 25] INFO [2022-01-10 05:38:46,620] jepsen worker 19 - jepsen.util 1009 :invoke :read [6 nil] INFO [2022-01-10 05:38:46,677] jepsen worker 60 - jepsen.util 1140 :info :write [20 5] :RequestTimeoutException INFO [2022-01-10 05:38:46,678] jepsen worker 60 - jepsen.util 1230 :invoke :read [20 nil] INFO [2022-01-10 05:38:46,711] jepsen worker 0 - jepsen.util 900 :ok :read [0 46] INFO [2022-01-10 05:38:46,712] jepsen worker 0 - jepsen.util 900 :invoke :write [0 8] INFO [2022-01-10 05:38:46,713] jepsen worker 85 - jepsen.util 1075 :ok :read [28 13] INFO [2022-01-10 05:38:46,715] jepsen worker 85 - jepsen.util 1075 :invoke :read [28 nil] INFO [2022-01-10 05:38:46,734] jepsen worker 58 - jepsen.util 958 :ok :read [19 31] INFO [2022-01-10 05:38:46,736] jepsen worker 58 - jepsen.util 958 :invoke :read [19 nil] INFO [2022-01-10 05:38:46,861] jepsen worker 57 - jepsen.util 1047 :ok :read [19 31] INFO [2022-01-10 05:38:46,862] jepsen worker 57 - jepsen.util 1047 :invoke :write [19 31] INFO [2022-01-10 05:38:46,867] jepsen worker 7 - jepsen.util 817 :info :write [2 43] :RequestTimeoutException INFO [2022-01-10 05:38:46,868] jepsen worker 7 - jepsen.util 907 :invoke :read [2 nil] INFO [2022-01-10 05:38:46,939] jepsen worker 85 - jepsen.util 1075 :ok :read [28 13] INFO [2022-01-10 05:38:46,939] jepsen worker 85 - jepsen.util 1075 :invoke :write [28 47] INFO [2022-01-10 05:38:46,969] jepsen worker 58 - jepsen.util 958 :ok :read [19 31] INFO [2022-01-10 05:38:46,969] jepsen worker 58 - jepsen.util 958 :invoke :read [19 nil] INFO [2022-01-10 05:38:47,069] jepsen worker 75 - jepsen.util 795 :ok :read [25 :nil] INFO [2022-01-10 05:38:47,070] jepsen worker 75 - jepsen.util 795 :invoke :write [25 32] INFO [2022-01-10 05:38:47,135] jepsen worker 58 - jepsen.util 958 :ok :read [19 31] INFO [2022-01-10 05:38:47,135] jepsen worker 58 - jepsen.util 958 :invoke :write [19 16] INFO [2022-01-10 05:38:47,182] jepsen worker 9 - jepsen.util 1089 :info :write [3 28] :SyncWriteAmbiguous INFO [2022-01-10 05:38:47,183] jepsen worker 9 - jepsen.util 1179 :invoke :read [3 nil] INFO [2022-01-10 05:38:47,186] jepsen worker 39 - jepsen.util 1029 :info :write [13 38] :SyncWriteAmbiguous INFO [2022-01-10 05:38:47,187] jepsen worker 39 - jepsen.util 1119 :invoke :read [13 nil] INFO [2022-01-10 05:38:47,191] jepsen worker 21 - jepsen.util 1191 :info :write [7 49] :RequestTimeoutException INFO [2022-01-10 05:38:47,191] jepsen worker 21 - jepsen.util 1281 :invoke :read [7 nil] INFO [2022-01-10 05:38:47,249] jepsen worker 19 - jepsen.util 1009 :ok :read [6 25] INFO [2022-01-10 05:38:47,250] jepsen worker 19 - jepsen.util 1009 :invoke :read [6 nil] INFO [2022-01-10 05:38:47,310] jepsen worker 29 - jepsen.util 929 :ok :read [9 22] INFO [2022-01-10 05:38:47,311] jepsen worker 29 - jepsen.util 929 :invoke :write [9 22] INFO [2022-01-10 05:38:47,371] jepsen worker 80 - jepsen.util 890 :info :write [26 35] :RequestTimeoutException INFO [2022-01-10 05:38:47,372] jepsen worker 80 - jepsen.util 980 :invoke :read [26 nil] INFO [2022-01-10 05:38:47,386] jepsen worker 70 - jepsen.util 1240 :info :write [23 22] :RequestTimeoutException INFO [2022-01-10 05:38:47,386] jepsen worker 70 - jepsen.util 1330 :invoke :write [23 26] INFO [2022-01-10 05:38:47,391] jepsen worker 39 - jepsen.util 1119 :ok :read [13 15] INFO [2022-01-10 05:38:47,391] jepsen worker 39 - jepsen.util 1119 :invoke :read [13 nil] INFO [2022-01-10 05:38:47,395] jepsen worker 26 - jepsen.util 1016 :info :write [8 15] :RequestTimeoutException INFO [2022-01-10 05:38:47,396] jepsen worker 26 - jepsen.util 1106 :invoke :read [8 nil] INFO [2022-01-10 05:38:47,443] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1286549707602339}, :ns_1@172.23.107.227 {:progress 0.0703812316715543}, :ns_1@172.23.105.187 {:progress 0.0703812316715543}, :ns_1@172.23.107.228 {:progress 0.1058823529411764}} INFO [2022-01-10 05:38:47,443] jepsen worker nemesis - couchbase.util Rebalance check count: 84 INFO [2022-01-10 05:38:47,443] jepsen worker nemesis - couchbase.util Rebalance stuck count: 64 INFO [2022-01-10 05:38:47,470] jepsen worker 39 - jepsen.util 1119 :ok :read [13 15] INFO [2022-01-10 05:38:47,471] jepsen worker 39 - jepsen.util 1119 :invoke :write [13 38] INFO [2022-01-10 05:38:47,540] jepsen worker 2 - jepsen.util 992 :info :write [0 30] :RequestTimeoutException INFO [2022-01-10 05:38:47,541] jepsen worker 2 - jepsen.util 1082 :invoke :read [0 nil] INFO [2022-01-10 05:38:47,741] jepsen worker 14 - jepsen.util 1004 :info :write [4 42] :RequestTimeoutException INFO [2022-01-10 05:38:47,742] jepsen worker 14 - jepsen.util 1094 :invoke :write [4 5] INFO [2022-01-10 05:38:47,758] jepsen worker 23 - jepsen.util 1013 :ok :read [7 :nil] INFO [2022-01-10 05:38:47,758] jepsen worker 76 - jepsen.util 796 :ok :read [25 :nil] INFO [2022-01-10 05:38:47,761] jepsen worker 23 - jepsen.util 1013 :invoke :read [7 nil] INFO [2022-01-10 05:38:47,762] jepsen worker 76 - jepsen.util 796 :invoke :write [25 48] INFO [2022-01-10 05:38:47,775] jepsen worker 21 - jepsen.util 1281 :ok :read [7 :nil] INFO [2022-01-10 05:38:47,775] jepsen worker 3 - jepsen.util 1083 :info :write [1 23] :RequestTimeoutException INFO [2022-01-10 05:38:47,776] jepsen worker 3 - jepsen.util 1173 :invoke :write [1 13] INFO [2022-01-10 05:38:47,776] jepsen worker 21 - jepsen.util 1281 :invoke :read [7 nil] INFO [2022-01-10 05:38:47,792] jepsen worker 19 - jepsen.util 1009 :ok :read [6 25] INFO [2022-01-10 05:38:47,793] jepsen worker 19 - jepsen.util 1009 :invoke :write [6 48] INFO [2022-01-10 05:38:47,796] jepsen worker 66 - jepsen.util 696 :info :write [22 44] :RequestTimeoutException INFO [2022-01-10 05:38:47,797] jepsen worker 66 - jepsen.util 786 :invoke :read [22 nil] INFO [2022-01-10 05:38:47,814] jepsen worker 65 - jepsen.util 1055 :ok :read [21 7] INFO [2022-01-10 05:38:47,814] jepsen worker 65 - jepsen.util 1055 :invoke :write [21 1] INFO [2022-01-10 05:38:47,820] jepsen worker 20 - jepsen.util 740 :info :write [6 24] :RequestTimeoutException INFO [2022-01-10 05:38:47,821] jepsen worker 20 - jepsen.util 830 :invoke :read [6 nil] INFO [2022-01-10 05:38:47,906] jepsen worker 35 - jepsen.util 1115 :ok :read [11 21] INFO [2022-01-10 05:38:47,906] jepsen worker 35 - jepsen.util 1115 :invoke :read [11 nil] INFO [2022-01-10 05:38:48,014] jepsen worker 44 - jepsen.util 1034 :ok :read [14 14] INFO [2022-01-10 05:38:48,015] jepsen worker 44 - jepsen.util 1034 :invoke :write [14 22] INFO [2022-01-10 05:38:48,022] jepsen worker 74 - jepsen.util 1244 :ok :read [24 18] INFO [2022-01-10 05:38:48,023] jepsen worker 74 - jepsen.util 1244 :invoke :write [24 19] INFO [2022-01-10 05:38:48,141] jepsen worker 28 - jepsen.util 928 :info :write [9 19] :RequestTimeoutException INFO [2022-01-10 05:38:48,142] jepsen worker 28 - jepsen.util 1018 :invoke :read [9 nil] INFO [2022-01-10 05:38:48,256] jepsen worker 32 - jepsen.util 932 :info :write [10 11] :RequestTimeoutException INFO [2022-01-10 05:38:48,257] jepsen worker 32 - jepsen.util 1022 :invoke :read [10 nil] INFO [2022-01-10 05:38:48,306] jepsen worker 21 - jepsen.util 1281 :ok :read [7 :nil] INFO [2022-01-10 05:38:48,306] jepsen worker 21 - jepsen.util 1281 :invoke :write [7 22] INFO [2022-01-10 05:38:48,354] jepsen worker 23 - jepsen.util 1013 :ok :read [7 :nil] INFO [2022-01-10 05:38:48,354] jepsen worker 26 - jepsen.util 1106 :ok :read [8 :nil] INFO [2022-01-10 05:38:48,355] jepsen worker 23 - jepsen.util 1013 :invoke :read [7 nil] INFO [2022-01-10 05:38:48,356] jepsen worker 26 - jepsen.util 1106 :invoke :read [8 nil] INFO [2022-01-10 05:38:48,359] jepsen worker 72 - jepsen.util 1242 :ok :read [24 18] INFO [2022-01-10 05:38:48,360] jepsen worker 72 - jepsen.util 1242 :invoke :read [24 nil] INFO [2022-01-10 05:38:48,360] jepsen worker 42 - jepsen.util 942 :ok :read [14 14] INFO [2022-01-10 05:38:48,360] jepsen worker 34 - jepsen.util 934 :ok :read [11 21] INFO [2022-01-10 05:38:48,361] jepsen worker 42 - jepsen.util 942 :invoke :write [14 37] INFO [2022-01-10 05:38:48,362] jepsen worker 34 - jepsen.util 934 :invoke :read [11 nil] INFO [2022-01-10 05:38:48,367] jepsen worker 7 - jepsen.util 907 :ok :read [2 14] INFO [2022-01-10 05:38:48,368] jepsen worker 7 - jepsen.util 907 :invoke :read [2 nil] INFO [2022-01-10 05:38:48,430] jepsen worker 43 - jepsen.util 943 :info :write [14 26] :RequestTimeoutException INFO [2022-01-10 05:38:48,431] jepsen worker 43 - jepsen.util 1033 :invoke :write [14 23] INFO [2022-01-10 05:38:48,450] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1345029239766082}, :ns_1@172.23.107.227 {:progress 0.07624633431085048}, :ns_1@172.23.105.187 {:progress 0.07624633431085048}, :ns_1@172.23.107.228 {:progress 0.1176470588235294}} INFO [2022-01-10 05:38:48,450] jepsen worker nemesis - couchbase.util Rebalance check count: 85 INFO [2022-01-10 05:38:48,450] jepsen worker nemesis - couchbase.util Rebalance stuck count: 64 INFO [2022-01-10 05:38:48,481] jepsen worker 67 - jepsen.util 877 :info :write [22 17] :RequestTimeoutException INFO [2022-01-10 05:38:48,482] jepsen worker 67 - jepsen.util 967 :invoke :write [22 39] INFO [2022-01-10 05:38:48,636] jepsen worker 16 - jepsen.util 1006 :info :write [5 4] :RequestTimeoutException INFO [2022-01-10 05:38:48,637] jepsen worker 16 - jepsen.util 1096 :invoke :read [5 nil] INFO [2022-01-10 05:38:48,715] jepsen worker 2 - jepsen.util 1082 :ok :read [0 46] INFO [2022-01-10 05:38:48,716] jepsen worker 2 - jepsen.util 1082 :invoke :read [0 nil] INFO [2022-01-10 05:38:48,764] jepsen worker 66 - jepsen.util 786 :ok :read [22 41] INFO [2022-01-10 05:38:48,764] jepsen worker 66 - jepsen.util 786 :invoke :write [22 11] INFO [2022-01-10 05:38:49,121] jepsen worker 24 - jepsen.util 924 :ok :read [8 :nil] INFO [2022-01-10 05:38:49,121] jepsen worker 24 - jepsen.util 924 :invoke :read [8 nil] INFO [2022-01-10 05:38:49,200] jepsen worker 16 - jepsen.util 1096 :ok :read [5 42] INFO [2022-01-10 05:38:49,200] jepsen worker 16 - jepsen.util 1096 :invoke :write [5 32] INFO [2022-01-10 05:38:49,267] jepsen worker 63 - jepsen.util 963 :ok :read [21 7] INFO [2022-01-10 05:38:49,268] jepsen worker 63 - jepsen.util 963 :invoke :read [21 nil] INFO [2022-01-10 05:38:49,436] jepsen worker 17 - jepsen.util 827 :info :write [5 13] :RequestTimeoutException INFO [2022-01-10 05:38:49,437] jepsen worker 17 - jepsen.util 917 :invoke :read [5 nil] INFO [2022-01-10 05:38:49,456] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1345029239766082}, :ns_1@172.23.107.227 {:progress 0.07624633431085048}, :ns_1@172.23.105.187 {:progress 0.07624633431085048}, :ns_1@172.23.107.228 {:progress 0.1176470588235294}} INFO [2022-01-10 05:38:49,456] jepsen worker nemesis - couchbase.util Rebalance check count: 86 INFO [2022-01-10 05:38:49,456] jepsen worker nemesis - couchbase.util Rebalance stuck count: 65 WARN [2022-01-10 05:38:49,633] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14062616,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":15434192,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9361636,"encode_duration_us":19645,"operation_id":"0x1d06","timeout_ms":10000,"last_server_duration_us":9152302,"total_duration_us":10001445},{"operation_name":"upsert","last_dispatch_duration_us":10269491,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13074346,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9385497,"encode_duration_us":16479,"operation_id":"0x1d0d","timeout_ms":10000,"last_server_duration_us":9096885,"total_duration_us":10010103},{"operation_name":"upsert","last_dispatch_duration_us":9166983,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17642762,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9189817,"encode_duration_us":16135,"operation_id":"0x1d4f","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10002638}],"total_count":3}} WARN [2022-01-10 05:38:49,733] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":1446652,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1446652,"total_server_duration_us":319711,"operation_id":"0x1dd0","timeout_ms":30000,"last_server_duration_us":319711,"total_duration_us":11202312},{"operation_name":"upsert","last_dispatch_duration_us":659739,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8228208,"total_server_duration_us":363149,"encode_duration_us":16763,"operation_id":"0x1ed9","timeout_ms":10000,"last_server_duration_us":133276,"total_duration_us":10010854},{"operation_name":"upsert","last_dispatch_duration_us":3979619,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11010970,"total_server_duration_us":190945,"encode_duration_us":18149,"operation_id":"0x1e76","timeout_ms":10000,"last_server_duration_us":81541,"total_duration_us":10008878},{"operation_name":"upsert","last_dispatch_duration_us":377623,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10297331,"total_server_duration_us":196190,"encode_duration_us":16549,"operation_id":"0x1dae","timeout_ms":10000,"last_server_duration_us":59249,"total_duration_us":10006178},{"operation_name":"upsert","last_dispatch_duration_us":188195,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":5318329,"total_server_duration_us":344459,"encode_duration_us":18243,"operation_id":"0x1ef5","timeout_ms":10000,"last_server_duration_us":45614,"total_duration_us":10003878},{"operation_name":"upsert","last_dispatch_duration_us":767251,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":767251,"total_server_duration_us":144782,"encode_duration_us":16144,"operation_id":"0x1dac","timeout_ms":10000,"last_server_duration_us":144782,"total_duration_us":10002318},{"operation_name":"upsert","last_dispatch_duration_us":1995423,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8245038,"total_server_duration_us":151735,"encode_duration_us":16432,"operation_id":"0x1dcf","timeout_ms":10000,"last_server_duration_us":115616,"total_duration_us":10002200},{"operation_name":"get","last_dispatch_duration_us":3590433,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3590433,"total_server_duration_us":9686,"operation_id":"0x1e0e","timeout_ms":30000,"last_server_duration_us":9686,"total_duration_us":8249426},{"operation_name":"get","last_dispatch_duration_us":8051243,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8051243,"total_server_duration_us":32,"operation_id":"0x1f3a","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":8053767},{"operation_name":"get","last_dispatch_duration_us":825271,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":825271,"total_server_duration_us":15,"operation_id":"0x1f0e","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":4462419}],"total_count":19}} INFO [2022-01-10 05:38:49,740] jepsen worker 6 - jepsen.util 816 :info :write [2 9] :RequestTimeoutException INFO [2022-01-10 05:38:49,740] jepsen worker 6 - jepsen.util 906 :invoke :write [2 17] INFO [2022-01-10 05:38:49,770] jepsen worker 34 - jepsen.util 934 :ok :read [11 21] INFO [2022-01-10 05:38:49,771] jepsen worker 34 - jepsen.util 934 :invoke :write [11 11] INFO [2022-01-10 05:38:49,822] jepsen worker 55 - jepsen.util 955 :info :write [18 11] :RequestTimeoutException INFO [2022-01-10 05:38:49,822] jepsen worker 55 - jepsen.util 1045 :invoke :read [18 nil] INFO [2022-01-10 05:38:50,349] jepsen worker 4 - jepsen.util 994 :ok :read [1 41] INFO [2022-01-10 05:38:50,349] jepsen worker 4 - jepsen.util 994 :invoke :write [1 24] INFO [2022-01-10 05:38:50,462] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1345029239766082}, :ns_1@172.23.107.227 {:progress 0.07624633431085048}, :ns_1@172.23.105.187 {:progress 0.07624633431085048}, :ns_1@172.23.107.228 {:progress 0.1176470588235294}} INFO [2022-01-10 05:38:50,462] jepsen worker nemesis - couchbase.util Rebalance check count: 87 INFO [2022-01-10 05:38:50,462] jepsen worker nemesis - couchbase.util Rebalance stuck count: 66 INFO [2022-01-10 05:38:50,550] jepsen worker 35 - jepsen.util 1115 :ok :read [11 21] INFO [2022-01-10 05:38:50,551] jepsen worker 35 - jepsen.util 1115 :invoke :read [11 nil] INFO [2022-01-10 05:38:50,581] jepsen worker 61 - jepsen.util 1051 :info :write [20 16] :RequestTimeoutException INFO [2022-01-10 05:38:50,582] jepsen worker 61 - jepsen.util 1141 :invoke :write [20 38] INFO [2022-01-10 05:38:50,595] jepsen worker 78 - jepsen.util 798 :info :write [26 33] :RequestTimeoutException INFO [2022-01-10 05:38:50,595] jepsen worker 78 - jepsen.util 888 :invoke :write [26 42] INFO [2022-01-10 05:38:50,603] jepsen worker 80 - jepsen.util 980 :ok :read [26 10] INFO [2022-01-10 05:38:50,603] jepsen worker 80 - jepsen.util 980 :invoke :write [26 16] INFO [2022-01-10 05:38:50,686] jepsen worker 88 - jepsen.util 1078 :info :write [29 43] :RequestTimeoutException INFO [2022-01-10 05:38:50,686] jepsen worker 88 - jepsen.util 1168 :invoke :read [29 nil] INFO [2022-01-10 05:38:50,800] jepsen worker 11 - jepsen.util 821 :info :write [3 11] :RequestTimeoutException INFO [2022-01-10 05:38:50,801] jepsen worker 11 - jepsen.util 911 :invoke :write [3 49] INFO [2022-01-10 05:38:50,824] jepsen worker 88 - jepsen.util 1168 :ok :read [29 33] INFO [2022-01-10 05:38:50,824] jepsen worker 88 - jepsen.util 1168 :invoke :read [29 nil] INFO [2022-01-10 05:38:50,976] jepsen worker 79 - jepsen.util 889 :info :write [26 43] :RequestTimeoutException INFO [2022-01-10 05:38:50,977] jepsen worker 79 - jepsen.util 979 :invoke :write [26 16] INFO [2022-01-10 05:38:51,045] jepsen worker 30 - jepsen.util 570 :info :write [10 48] :RequestTimeoutException INFO [2022-01-10 05:38:51,046] jepsen worker 30 - jepsen.util 660 :invoke :read [10 nil] INFO [2022-01-10 05:38:51,052] jepsen worker 81 - jepsen.util 891 :info :write [27 23] :RequestTimeoutException INFO [2022-01-10 05:38:51,052] jepsen worker 81 - jepsen.util 981 :invoke :write [27 6] WARN [2022-01-10 05:38:51,177] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17533935,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17533935,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9029938,"encode_duration_us":17012,"operation_id":"0x1de1","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10003261},{"operation_name":"upsert","last_dispatch_duration_us":10665996,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10665996,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9585530,"encode_duration_us":16692,"operation_id":"0x1e02","timeout_ms":10000,"last_server_duration_us":9585530,"total_duration_us":10006460}],"total_count":2}} WARN [2022-01-10 05:38:51,197] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9437121,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10510970,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9342369,"encode_duration_us":20594,"operation_id":"0x1ea4","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10003288},{"operation_name":"upsert","last_dispatch_duration_us":11333246,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11333246,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9215351,"encode_duration_us":16409,"operation_id":"0x1c22","timeout_ms":10000,"last_server_duration_us":9215351,"total_duration_us":10007630},{"operation_name":"upsert","last_dispatch_duration_us":9359114,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12730698,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9497753,"encode_duration_us":21941,"operation_id":"0x1d7e","timeout_ms":10000,"last_server_duration_us":9325861,"total_duration_us":10008254},{"operation_name":"upsert","last_dispatch_duration_us":9370892,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10673029,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9200127,"encode_duration_us":19604,"operation_id":"0x1e03","timeout_ms":10000,"last_server_duration_us":9078800,"total_duration_us":10008400}],"total_count":4}} WARN [2022-01-10 05:38:51,197] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9341961,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9796292,"total_server_duration_us":168305,"encode_duration_us":21045,"operation_id":"0x1ed7","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10009893},{"operation_name":"upsert","last_dispatch_duration_us":387874,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":387874,"total_server_duration_us":25976,"encode_duration_us":19224,"operation_id":"0x1eec","timeout_ms":10000,"last_server_duration_us":25976,"total_duration_us":10009625},{"operation_name":"upsert","last_dispatch_duration_us":63510,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":4268684,"total_server_duration_us":192685,"encode_duration_us":19910,"operation_id":"0x1e42","timeout_ms":10000,"last_server_duration_us":22554,"total_duration_us":10009293},{"operation_name":"upsert","last_dispatch_duration_us":10840993,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11147807,"total_server_duration_us":175765,"encode_duration_us":17051,"operation_id":"0x1e81","timeout_ms":10000,"last_server_duration_us":105344,"total_duration_us":10008776},{"operation_name":"upsert","last_dispatch_duration_us":9370892,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10673029,"total_server_duration_us":9200127,"encode_duration_us":19604,"operation_id":"0x1e03","timeout_ms":10000,"last_server_duration_us":9078800,"total_duration_us":10008400},{"operation_name":"upsert","encode_duration_us":16195,"operation_id":"0x1ed6","timeout_ms":10000,"total_duration_us":10007002},{"operation_name":"upsert","last_dispatch_duration_us":11912002,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":12779950,"total_server_duration_us":152652,"encode_duration_us":19033,"operation_id":"0x1e04","timeout_ms":10000,"last_server_duration_us":59625,"total_duration_us":10006786},{"operation_name":"upsert","last_dispatch_duration_us":157100,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9426365,"total_server_duration_us":260438,"encode_duration_us":16516,"operation_id":"0x1f12","timeout_ms":10000,"last_server_duration_us":73273,"total_duration_us":10006519},{"operation_name":"upsert","last_dispatch_duration_us":9437121,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":10510970,"total_server_duration_us":9342369,"encode_duration_us":20594,"operation_id":"0x1ea4","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10003288},{"operation_name":"upsert","last_dispatch_duration_us":2354806,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2354806,"total_server_duration_us":762613,"encode_duration_us":16565,"operation_id":"0x1f63","timeout_ms":10000,"last_server_duration_us":762613,"total_duration_us":10002321}],"total_count":19}} WARN [2022-01-10 05:38:51,277] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12630552,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":12630552,"total_server_duration_us":23,"operation_id":"0x1de2","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":12632919},{"operation_name":"upsert","last_dispatch_duration_us":10342426,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10342426,"total_server_duration_us":18493,"encode_duration_us":17110,"operation_id":"0x1e6f","timeout_ms":10000,"last_server_duration_us":18493,"total_duration_us":10009953},{"operation_name":"upsert","encode_duration_us":16698,"operation_id":"0x1e7f","timeout_ms":10000,"total_duration_us":10009776},{"operation_name":"upsert","last_dispatch_duration_us":11085288,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11085288,"total_server_duration_us":19418,"encode_duration_us":19148,"operation_id":"0x1e41","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10008225},{"operation_name":"upsert","last_dispatch_duration_us":10524362,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10524362,"total_server_duration_us":22306,"encode_duration_us":18062,"operation_id":"0x1e7c","timeout_ms":10000,"last_server_duration_us":22306,"total_duration_us":10005886},{"operation_name":"upsert","last_dispatch_duration_us":610993,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8440116,"total_server_duration_us":1234519,"encode_duration_us":16375,"operation_id":"0x1eae","timeout_ms":10000,"last_server_duration_us":107269,"total_duration_us":10003930},{"operation_name":"upsert","last_dispatch_duration_us":725304,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8061356,"total_server_duration_us":1174904,"encode_duration_us":17321,"operation_id":"0x1ece","timeout_ms":10000,"last_server_duration_us":187992,"total_duration_us":10003926},{"operation_name":"upsert","last_dispatch_duration_us":32954,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9734682,"total_server_duration_us":238761,"encode_duration_us":17291,"operation_id":"0x1e10","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10003849},{"operation_name":"upsert","last_dispatch_duration_us":249865,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":985247,"total_server_duration_us":41099,"encode_duration_us":16148,"operation_id":"0x1f14","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10003538},{"operation_name":"upsert","last_dispatch_duration_us":10576185,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10576185,"total_server_duration_us":18876,"encode_duration_us":16866,"operation_id":"0x1e7e","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10002763}],"total_count":21}} WARN [2022-01-10 05:38:51,277] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13344293,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13344293,"total_server_duration_us":146249,"operation_id":"0x1e43","timeout_ms":30000,"last_server_duration_us":146249,"total_duration_us":13344621},{"operation_name":"get","last_dispatch_duration_us":12099692,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12099692,"total_server_duration_us":15,"operation_id":"0x1e82","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":12099932},{"operation_name":"upsert","last_dispatch_duration_us":613623,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10268264,"total_server_duration_us":482955,"encode_duration_us":18696,"operation_id":"0x1ddc","timeout_ms":10000,"last_server_duration_us":61139,"total_duration_us":10010223},{"operation_name":"upsert","last_dispatch_duration_us":13692386,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13692386,"total_server_duration_us":81828,"encode_duration_us":16293,"operation_id":"0x1e36","timeout_ms":10000,"last_server_duration_us":81828,"total_duration_us":10010120},{"operation_name":"upsert","last_dispatch_duration_us":93438,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9538525,"total_server_duration_us":395316,"encode_duration_us":17565,"operation_id":"0x1e3f","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10006567},{"operation_name":"upsert","last_dispatch_duration_us":523786,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":3829825,"total_server_duration_us":1120947,"encode_duration_us":18426,"operation_id":"0x1f88","timeout_ms":10000,"last_server_duration_us":413816,"total_duration_us":10005432},{"operation_name":"upsert","last_dispatch_duration_us":567045,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":912826,"total_server_duration_us":220923,"encode_duration_us":16210,"operation_id":"0x1f5b","timeout_ms":10000,"last_server_duration_us":54456,"total_duration_us":10004001},{"operation_name":"upsert","last_dispatch_duration_us":13830165,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13830165,"total_server_duration_us":76870,"encode_duration_us":16431,"operation_id":"0x1e07","timeout_ms":10000,"last_server_duration_us":76870,"total_duration_us":10003586},{"operation_name":"upsert","last_dispatch_duration_us":12565469,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12565469,"total_server_duration_us":26685,"encode_duration_us":16671,"operation_id":"0x1e05","timeout_ms":10000,"last_server_duration_us":26685,"total_duration_us":10001459},{"operation_name":"get","last_dispatch_duration_us":9255047,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9255047,"total_server_duration_us":13958,"operation_id":"0x1f13","timeout_ms":30000,"last_server_duration_us":13958,"total_duration_us":9257361}],"total_count":19}} WARN [2022-01-10 05:38:51,278] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":3165963,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3165963,"total_server_duration_us":38,"operation_id":"0x1e6a","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":13433759},{"operation_name":"upsert","last_dispatch_duration_us":10299250,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10299250,"total_server_duration_us":378731,"encode_duration_us":16698,"operation_id":"0x1e2d","timeout_ms":10000,"last_server_duration_us":378731,"total_duration_us":10009315},{"operation_name":"upsert","last_dispatch_duration_us":16417101,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":16417101,"total_server_duration_us":18953,"encode_duration_us":16233,"operation_id":"0x1e13","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10006864},{"operation_name":"upsert","last_dispatch_duration_us":10665996,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10665996,"total_server_duration_us":9585530,"encode_duration_us":16692,"operation_id":"0x1e02","timeout_ms":10000,"last_server_duration_us":9585530,"total_duration_us":10006460},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","encode_duration_us":16313,"operation_id":"0x1f65","timeout_ms":10000,"total_duration_us":10006331},{"operation_name":"upsert","last_dispatch_duration_us":1263639,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10746283,"total_server_duration_us":235580,"encode_duration_us":17401,"operation_id":"0x1efc","timeout_ms":10000,"last_server_duration_us":26507,"total_duration_us":10005607},{"operation_name":"upsert","encode_duration_us":17932,"operation_id":"0x1f0f","timeout_ms":10000,"total_duration_us":10004551},{"operation_name":"upsert","last_dispatch_duration_us":2238975,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10599673,"total_server_duration_us":430934,"encode_duration_us":18797,"operation_id":"0x1e77","timeout_ms":10000,"last_server_duration_us":46400,"total_duration_us":10004221},{"operation_name":"upsert","last_dispatch_duration_us":2199859,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10846610,"total_server_duration_us":170244,"encode_duration_us":28550,"operation_id":"0x1e6b","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10003801},{"operation_name":"upsert","encode_duration_us":19455,"operation_id":"0x1e78","timeout_ms":10000,"total_duration_us":10003569}],"total_count":27}} WARN [2022-01-10 05:38:51,278] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9559951,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16781545,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9970347,"encode_duration_us":26453,"operation_id":"0x1cee","timeout_ms":10000,"last_server_duration_us":9125641,"total_duration_us":10005247},{"operation_name":"upsert","last_dispatch_duration_us":9684419,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13345814,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9657051,"encode_duration_us":16801,"operation_id":"0x1d21","timeout_ms":10000,"last_server_duration_us":9492134,"total_duration_us":10005812},{"operation_name":"upsert","last_dispatch_duration_us":10366378,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10366378,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9271072,"encode_duration_us":16573,"operation_id":"0x1e38","timeout_ms":10000,"last_server_duration_us":9271072,"total_duration_us":10005889},{"operation_name":"upsert","last_dispatch_duration_us":25247426,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25247426,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":8607717,"encode_duration_us":17279,"operation_id":"0x1b23","timeout_ms":10000,"last_server_duration_us":8607717,"total_duration_us":10010388},{"operation_name":"upsert","last_dispatch_duration_us":9073998,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10093404,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":9074757,"encode_duration_us":23311,"operation_id":"0x1ea5","timeout_ms":10000,"last_server_duration_us":9055417,"total_duration_us":10006428}],"total_count":5}} WARN [2022-01-10 05:38:51,378] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":25681255,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25681255,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9110726,"encode_duration_us":16768,"operation_id":"0x1b7d","timeout_ms":10000,"last_server_duration_us":9110726,"total_duration_us":10005695},{"operation_name":"upsert","last_dispatch_duration_us":10340133,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12030209,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9530950,"encode_duration_us":16483,"operation_id":"0x1d7b","timeout_ms":10000,"last_server_duration_us":9244265,"total_duration_us":10008727}],"total_count":2}} WARN [2022-01-10 05:38:51,378] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16234304,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16234304,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9362463,"encode_duration_us":19009,"operation_id":"0x1c88","timeout_ms":10000,"last_server_duration_us":9362463,"total_duration_us":10003564},{"operation_name":"upsert","last_dispatch_duration_us":29602611,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":29602611,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9040551,"encode_duration_us":16433,"operation_id":"0x1a6d","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10008414}],"total_count":2}} INFO [2022-01-10 05:38:51,468] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1345029239766082}, :ns_1@172.23.107.227 {:progress 0.07624633431085048}, :ns_1@172.23.105.187 {:progress 0.07624633431085048}, :ns_1@172.23.107.228 {:progress 0.1176470588235294}} INFO [2022-01-10 05:38:51,469] jepsen worker nemesis - couchbase.util Rebalance check count: 88 INFO [2022-01-10 05:38:51,476] jepsen worker nemesis - couchbase.util Rebalance stuck count: 67 WARN [2022-01-10 05:38:51,478] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":22690504,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":22690504,"total_server_duration_us":27,"operation_id":"0x1b9f","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":22692710},{"operation_name":"get","last_dispatch_duration_us":20221120,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":20221120,"total_server_duration_us":38,"operation_id":"0x1c01","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":20223942},{"operation_name":"get","last_dispatch_duration_us":17834930,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":17834930,"total_server_duration_us":23,"operation_id":"0x1c7d","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":17837267},{"operation_name":"upsert","last_dispatch_duration_us":10136251,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10136251,"total_server_duration_us":80540,"encode_duration_us":16252,"operation_id":"0x1e7b","timeout_ms":10000,"last_server_duration_us":80540,"total_duration_us":10010028},{"operation_name":"upsert","last_dispatch_duration_us":11283535,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12298110,"total_server_duration_us":273541,"encode_duration_us":21115,"operation_id":"0x1de0","timeout_ms":10000,"last_server_duration_us":120988,"total_duration_us":10008533},{"operation_name":"upsert","last_dispatch_duration_us":9073998,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10093404,"total_server_duration_us":9074757,"encode_duration_us":23311,"operation_id":"0x1ea5","timeout_ms":10000,"last_server_duration_us":9055417,"total_duration_us":10006428},{"operation_name":"upsert","last_dispatch_duration_us":10366378,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10366378,"total_server_duration_us":9271072,"encode_duration_us":16573,"operation_id":"0x1e38","timeout_ms":10000,"last_server_duration_us":9271072,"total_duration_us":10005889},{"operation_name":"upsert","last_dispatch_duration_us":1718794,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7794896,"total_server_duration_us":122142,"encode_duration_us":16808,"operation_id":"0x1edc","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10005169},{"operation_name":"upsert","last_dispatch_duration_us":11152296,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11776288,"total_server_duration_us":343499,"encode_duration_us":20303,"operation_id":"0x1dde","timeout_ms":10000,"last_server_duration_us":203801,"total_duration_us":10004920},{"operation_name":"upsert","last_dispatch_duration_us":8347689,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8347689,"total_server_duration_us":144599,"encode_duration_us":17817,"operation_id":"0x1f03","timeout_ms":10000,"last_server_duration_us":144599,"total_duration_us":10004772}],"total_count":16}} INFO [2022-01-10 05:38:51,604] jepsen worker 31 - jepsen.util 1201 :info :write [10 35] :RequestTimeoutException INFO [2022-01-10 05:38:51,605] jepsen worker 31 - jepsen.util 1291 :invoke :write [10 4] INFO [2022-01-10 05:38:51,632] jepsen worker 5 - jepsen.util 905 :info :write [1 35] :RequestTimeoutException INFO [2022-01-10 05:38:51,633] jepsen worker 5 - jepsen.util 995 :invoke :write [1 14] INFO [2022-01-10 05:38:51,906] jepsen worker 12 - jepsen.util 1002 :info :write [4 20] :RequestTimeoutException INFO [2022-01-10 05:38:51,907] jepsen worker 12 - jepsen.util 1092 :invoke :read [4 nil] INFO [2022-01-10 05:38:51,950] jepsen worker 62 - jepsen.util 1052 :info :write [20 23] :RequestTimeoutException INFO [2022-01-10 05:38:51,951] jepsen worker 62 - jepsen.util 1142 :invoke :write [20 13] INFO [2022-01-10 05:38:52,190] jepsen worker 60 - jepsen.util 1230 :ok :read [20 :nil] INFO [2022-01-10 05:38:52,191] jepsen worker 60 - jepsen.util 1230 :invoke :write [20 30] INFO [2022-01-10 05:38:52,279] jepsen worker 84 - jepsen.util 894 :info :write [28 21] :SyncWriteAmbiguous INFO [2022-01-10 05:38:52,280] jepsen worker 84 - jepsen.util 984 :invoke :write [28 21] INFO [2022-01-10 05:38:52,448] jepsen worker 48 - jepsen.util 768 :ok :read [16 38] INFO [2022-01-10 05:38:52,449] jepsen worker 48 - jepsen.util 768 :invoke :read [16 nil] INFO [2022-01-10 05:38:52,451] jepsen worker 54 - jepsen.util 954 :info :write [18 34] :RequestTimeoutException INFO [2022-01-10 05:38:52,452] jepsen worker 54 - jepsen.util 1044 :invoke :write [18 38] INFO [2022-01-10 05:38:52,484] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1403508771929824}, :ns_1@172.23.107.227 {:progress 0.08504398826979476}, :ns_1@172.23.105.187 {:progress 0.08504398826979476}, :ns_1@172.23.107.228 {:progress 0.1352941176470588}} INFO [2022-01-10 05:38:52,484] jepsen worker nemesis - couchbase.util Rebalance check count: 89 INFO [2022-01-10 05:38:52,484] jepsen worker nemesis - couchbase.util Rebalance stuck count: 67 INFO [2022-01-10 05:38:52,705] jepsen worker 64 - jepsen.util 874 :info :write [21 32] :RequestTimeoutException INFO [2022-01-10 05:38:52,705] jepsen worker 64 - jepsen.util 964 :invoke :write [21 18] INFO [2022-01-10 05:38:52,805] jepsen worker 48 - jepsen.util 768 :ok :read [16 38] INFO [2022-01-10 05:38:52,806] jepsen worker 48 - jepsen.util 768 :invoke :read [16 nil] INFO [2022-01-10 05:38:52,856] jepsen worker 77 - jepsen.util 1157 :info :write [25 20] :RequestTimeoutException INFO [2022-01-10 05:38:52,857] jepsen worker 77 - jepsen.util 1247 :invoke :read [25 nil] INFO [2022-01-10 05:38:52,871] jepsen worker 38 - jepsen.util 1028 :info :write [12 4] :RequestTimeoutException INFO [2022-01-10 05:38:52,872] jepsen worker 38 - jepsen.util 1118 :invoke :write [12 35] INFO [2022-01-10 05:38:52,920] jepsen worker 41 - jepsen.util 671 :info :write [13 44] :RequestTimeoutException INFO [2022-01-10 05:38:52,921] jepsen worker 41 - jepsen.util 761 :invoke :write [13 47] INFO [2022-01-10 05:38:52,941] jepsen worker 23 - jepsen.util 1013 :ok :read [7 :nil] INFO [2022-01-10 05:38:52,942] jepsen worker 23 - jepsen.util 1013 :invoke :write [7 4] INFO [2022-01-10 05:38:52,990] jepsen worker 73 - jepsen.util 883 :info :write [24 8] :RequestTimeoutException INFO [2022-01-10 05:38:52,991] jepsen worker 73 - jepsen.util 973 :invoke :write [24 21] INFO [2022-01-10 05:38:53,020] jepsen worker 49 - jepsen.util 769 :info :write [16 44] :RequestTimeoutException INFO [2022-01-10 05:38:53,021] jepsen worker 49 - jepsen.util 859 :invoke :read [16 nil] INFO [2022-01-10 05:38:53,040] jepsen worker 50 - jepsen.util 1040 :info :write [16 7] :RequestTimeoutException INFO [2022-01-10 05:38:53,041] jepsen worker 50 - jepsen.util 1130 :invoke :read [16 nil] INFO [2022-01-10 05:38:53,096] jepsen worker 36 - jepsen.util 666 :info :write [12 24] :RequestTimeoutException INFO [2022-01-10 05:38:53,097] jepsen worker 36 - jepsen.util 756 :invoke :read [12 nil] INFO [2022-01-10 05:38:53,106] jepsen worker 25 - jepsen.util 925 :info :write [8 44] :RequestTimeoutException INFO [2022-01-10 05:38:53,107] jepsen worker 25 - jepsen.util 1015 :invoke :read [8 nil] INFO [2022-01-10 05:38:53,116] jepsen worker 56 - jepsen.util 1046 :info :write [18 48] :RequestTimeoutException INFO [2022-01-10 05:38:53,117] jepsen worker 56 - jepsen.util 1136 :invoke :read [18 nil] INFO [2022-01-10 05:38:53,127] jepsen worker 53 - jepsen.util 773 :info :write [17 23] :RequestTimeoutException INFO [2022-01-10 05:38:53,127] jepsen worker 53 - jepsen.util 863 :invoke :write [17 14] INFO [2022-01-10 05:38:53,161] jepsen worker 48 - jepsen.util 768 :ok :read [16 38] INFO [2022-01-10 05:38:53,161] jepsen worker 48 - jepsen.util 768 :invoke :write [16 19] INFO [2022-01-10 05:38:53,176] jepsen worker 56 - jepsen.util 1136 :ok :read [18 14] INFO [2022-01-10 05:38:53,177] jepsen worker 56 - jepsen.util 1136 :invoke :write [18 42] INFO [2022-01-10 05:38:53,195] jepsen worker 33 - jepsen.util 933 :info :write [11 19] :RequestTimeoutException INFO [2022-01-10 05:38:53,196] jepsen worker 33 - jepsen.util 1023 :invoke :write [11 34] INFO [2022-01-10 05:38:53,196] jepsen worker 86 - jepsen.util 626 :info :write [28 40] :RequestTimeoutException INFO [2022-01-10 05:38:53,196] jepsen worker 86 - jepsen.util 716 :invoke :write [28 42] INFO [2022-01-10 05:38:53,256] jepsen worker 17 - jepsen.util 917 :ok :read [5 42] INFO [2022-01-10 05:38:53,261] jepsen worker 17 - jepsen.util 917 :invoke :write [5 34] INFO [2022-01-10 05:38:53,446] jepsen worker 51 - jepsen.util 951 :info :write [17 42] :RequestTimeoutException INFO [2022-01-10 05:38:53,447] jepsen worker 51 - jepsen.util 1041 :invoke :read [17 nil] INFO [2022-01-10 05:38:53,491] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1403508771929824}, :ns_1@172.23.107.227 {:progress 0.08504398826979476}, :ns_1@172.23.105.187 {:progress 0.08504398826979476}, :ns_1@172.23.107.228 {:progress 0.1352941176470588}} INFO [2022-01-10 05:38:53,491] jepsen worker nemesis - couchbase.util Rebalance check count: 90 INFO [2022-01-10 05:38:53,491] jepsen worker nemesis - couchbase.util Rebalance stuck count: 68 INFO [2022-01-10 05:38:53,672] jepsen worker 15 - jepsen.util 645 :info :write [5 19] :RequestTimeoutException INFO [2022-01-10 05:38:53,673] jepsen worker 15 - jepsen.util 735 :invoke :write [5 44] INFO [2022-01-10 05:38:53,690] jepsen worker 10 - jepsen.util 1000 :info :write [3 5] :RequestTimeoutException INFO [2022-01-10 05:38:53,690] jepsen worker 10 - jepsen.util 1090 :invoke :write [3 3] INFO [2022-01-10 05:38:53,746] jepsen worker 18 - jepsen.util 738 :info :write [6 23] :RequestTimeoutException INFO [2022-01-10 05:38:53,747] jepsen worker 18 - jepsen.util 828 :invoke :write [6 48] INFO [2022-01-10 05:38:54,096] jepsen worker 45 - jepsen.util 1125 :info :write [15 37] :RequestTimeoutException INFO [2022-01-10 05:38:54,097] jepsen worker 45 - jepsen.util 1215 :invoke :write [15 33] INFO [2022-01-10 05:38:54,165] jepsen worker 13 - jepsen.util 1003 :info :write [4 39] :RequestTimeoutException INFO [2022-01-10 05:38:54,165] jepsen worker 13 - jepsen.util 1093 :invoke :read [4 nil] INFO [2022-01-10 05:38:54,255] jepsen worker 27 - jepsen.util 837 :info :write [9 2] :RequestTimeoutException INFO [2022-01-10 05:38:54,256] jepsen worker 27 - jepsen.util 927 :invoke :write [9 46] INFO [2022-01-10 05:38:54,289] jepsen worker 28 - jepsen.util 1018 :ok :read [9 22] INFO [2022-01-10 05:38:54,290] jepsen worker 28 - jepsen.util 1018 :invoke :read [9 nil] INFO [2022-01-10 05:38:54,442] jepsen worker 71 - jepsen.util 1061 :info :write [23 24] :RequestTimeoutException INFO [2022-01-10 05:38:54,447] jepsen worker 71 - jepsen.util 1151 :invoke :write [23 11] INFO [2022-01-10 05:38:54,498] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1403508771929824}, :ns_1@172.23.107.227 {:progress 0.08504398826979476}, :ns_1@172.23.105.187 {:progress 0.08504398826979476}, :ns_1@172.23.107.228 {:progress 0.1352941176470588}} INFO [2022-01-10 05:38:54,498] jepsen worker nemesis - couchbase.util Rebalance check count: 91 INFO [2022-01-10 05:38:54,498] jepsen worker nemesis - couchbase.util Rebalance stuck count: 69 INFO [2022-01-10 05:38:54,503] jepsen worker 28 - jepsen.util 1018 :ok :read [9 22] INFO [2022-01-10 05:38:54,504] jepsen worker 28 - jepsen.util 1018 :invoke :write [9 21] INFO [2022-01-10 05:38:54,659] jepsen worker 26 - jepsen.util 1106 :ok :read [8 :nil] INFO [2022-01-10 05:38:54,659] jepsen worker 26 - jepsen.util 1106 :invoke :read [8 nil] INFO [2022-01-10 05:38:54,666] jepsen worker 47 - jepsen.util 947 :info :write [15 5] :RequestTimeoutException INFO [2022-01-10 05:38:54,667] jepsen worker 47 - jepsen.util 1037 :invoke :write [15 1] INFO [2022-01-10 05:38:54,678] jepsen worker 7 - jepsen.util 907 :ok :read [2 14] INFO [2022-01-10 05:38:54,679] jepsen worker 7 - jepsen.util 907 :invoke :write [2 24] INFO [2022-01-10 05:38:54,964] jepsen worker 20 - jepsen.util 830 :ok :read [6 25] INFO [2022-01-10 05:38:54,964] jepsen worker 20 - jepsen.util 830 :invoke :write [6 28] INFO [2022-01-10 05:38:54,964] jepsen worker 83 - jepsen.util 1073 :info :write [27 32] :RequestTimeoutException INFO [2022-01-10 05:38:54,965] jepsen worker 83 - jepsen.util 1163 :invoke :read [27 nil] INFO [2022-01-10 05:38:55,035] jepsen worker 52 - jepsen.util 952 :info :write [17 49] :SyncWriteAmbiguous INFO [2022-01-10 05:38:55,036] jepsen worker 52 - jepsen.util 1042 :invoke :read [17 nil] INFO [2022-01-10 05:38:55,067] jepsen worker 52 - jepsen.util 1042 :ok :read [17 41] INFO [2022-01-10 05:38:55,068] jepsen worker 52 - jepsen.util 1042 :invoke :write [17 3] INFO [2022-01-10 05:38:55,231] jepsen worker 87 - jepsen.util 1257 :info :write [29 19] :RequestTimeoutException INFO [2022-01-10 05:38:55,232] jepsen worker 87 - jepsen.util 1347 :invoke :read [29 nil] INFO [2022-01-10 05:38:55,267] jepsen worker 59 - jepsen.util 1049 :info :write [19 41] :RequestTimeoutException INFO [2022-01-10 05:38:55,267] jepsen worker 59 - jepsen.util 1139 :invoke :read [19 nil] INFO [2022-01-10 05:38:55,345] jepsen worker 22 - jepsen.util 832 :info :write [7 25] :RequestTimeoutException INFO [2022-01-10 05:38:55,346] jepsen worker 22 - jepsen.util 922 :invoke :read [7 nil] INFO [2022-01-10 05:38:55,370] jepsen worker 68 - jepsen.util 878 :info :write [22 15] :RequestTimeoutException INFO [2022-01-10 05:38:55,371] jepsen worker 68 - jepsen.util 968 :invoke :write [22 15] INFO [2022-01-10 05:38:55,462] jepsen worker 1 - jepsen.util 901 :info :write [0 35] :RequestTimeoutException INFO [2022-01-10 05:38:55,462] jepsen worker 1 - jepsen.util 991 :invoke :write [0 7] INFO [2022-01-10 05:38:55,490] jepsen worker 40 - jepsen.util 760 :info :write [13 25] :RequestTimeoutException INFO [2022-01-10 05:38:55,490] jepsen worker 40 - jepsen.util 850 :invoke :read [13 nil] INFO [2022-01-10 05:38:55,504] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1403508771929824}, :ns_1@172.23.107.227 {:progress 0.08504398826979476}, :ns_1@172.23.105.187 {:progress 0.08504398826979476}, :ns_1@172.23.107.228 {:progress 0.1352941176470588}} INFO [2022-01-10 05:38:55,504] jepsen worker nemesis - couchbase.util Rebalance check count: 92 INFO [2022-01-10 05:38:55,504] jepsen worker nemesis - couchbase.util Rebalance stuck count: 70 INFO [2022-01-10 05:38:55,567] jepsen worker 9 - jepsen.util 1179 :ok :read [3 2] INFO [2022-01-10 05:38:55,568] jepsen worker 9 - jepsen.util 1179 :invoke :read [3 nil] INFO [2022-01-10 05:38:55,576] jepsen worker 32 - jepsen.util 1022 :ok :read [10 37] INFO [2022-01-10 05:38:55,576] jepsen worker 32 - jepsen.util 1022 :invoke :read [10 nil] INFO [2022-01-10 05:38:55,589] jepsen worker 55 - jepsen.util 1045 :ok :read [18 14] INFO [2022-01-10 05:38:55,589] jepsen worker 55 - jepsen.util 1045 :invoke :write [18 28] INFO [2022-01-10 05:38:55,597] jepsen worker 30 - jepsen.util 660 :ok :read [10 37] INFO [2022-01-10 05:38:55,597] jepsen worker 30 - jepsen.util 660 :invoke :write [10 22] INFO [2022-01-10 05:38:55,613] jepsen worker 70 - jepsen.util 1330 :ok :write [23 26] INFO [2022-01-10 05:38:55,618] jepsen worker 70 - jepsen.util 1330 :invoke :write [23 2] INFO [2022-01-10 05:38:55,618] jepsen worker 9 - jepsen.util 1179 :ok :read [3 2] INFO [2022-01-10 05:38:55,619] jepsen worker 9 - jepsen.util 1179 :invoke :read [3 nil] INFO [2022-01-10 05:38:55,628] jepsen worker 32 - jepsen.util 1022 :ok :read [10 37] INFO [2022-01-10 05:38:55,629] jepsen worker 32 - jepsen.util 1022 :invoke :read [10 nil] INFO [2022-01-10 05:38:55,724] jepsen worker 9 - jepsen.util 1179 :ok :read [3 2] INFO [2022-01-10 05:38:55,725] jepsen worker 9 - jepsen.util 1179 :invoke :write [3 28] INFO [2022-01-10 05:38:55,755] jepsen worker 32 - jepsen.util 1022 :ok :read [10 37] INFO [2022-01-10 05:38:55,756] jepsen worker 32 - jepsen.util 1022 :invoke :read [10 nil] INFO [2022-01-10 05:38:55,905] jepsen worker 37 - jepsen.util 1027 :info :write [12 31] :RequestTimeoutException INFO [2022-01-10 05:38:55,906] jepsen worker 37 - jepsen.util 1117 :invoke :write [12 22] INFO [2022-01-10 05:38:55,918] jepsen worker 32 - jepsen.util 1022 :ok :read [10 37] INFO [2022-01-10 05:38:55,919] jepsen worker 32 - jepsen.util 1022 :invoke :read [10 nil] INFO [2022-01-10 05:38:55,976] jepsen worker 82 - jepsen.util 892 :info :write [27 16] :RequestTimeoutException INFO [2022-01-10 05:38:55,976] jepsen worker 82 - jepsen.util 982 :invoke :read [27 nil] INFO [2022-01-10 05:38:56,066] jepsen worker 8 - jepsen.util 818 :info :write [2 0] :RequestTimeoutException INFO [2022-01-10 05:38:56,067] jepsen worker 8 - jepsen.util 908 :invoke :read [2 nil] INFO [2022-01-10 05:38:56,071] jepsen worker 32 - jepsen.util 1022 :ok :read [10 37] INFO [2022-01-10 05:38:56,072] jepsen worker 32 - jepsen.util 1022 :invoke :read [10 nil] INFO [2022-01-10 05:38:56,199] jepsen worker 32 - jepsen.util 1022 :ok :read [10 37] INFO [2022-01-10 05:38:56,200] jepsen worker 32 - jepsen.util 1022 :invoke :write [10 5] INFO [2022-01-10 05:38:56,215] jepsen worker 46 - jepsen.util 1036 :info :write [15 15] :RequestTimeoutException INFO [2022-01-10 05:38:56,216] jepsen worker 46 - jepsen.util 1126 :invoke :write [15 13] INFO [2022-01-10 05:38:56,251] jepsen worker 72 - jepsen.util 1242 :ok :read [24 18] INFO [2022-01-10 05:38:56,251] jepsen worker 72 - jepsen.util 1242 :invoke :read [24 nil] INFO [2022-01-10 05:38:56,367] jepsen worker 89 - jepsen.util 1079 :ok :read [29 33] INFO [2022-01-10 05:38:56,367] jepsen worker 89 - jepsen.util 1079 :invoke :read [29 nil] INFO [2022-01-10 05:38:56,512] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1461988304093568}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1411764705882353}} INFO [2022-01-10 05:38:56,512] jepsen worker nemesis - couchbase.util Rebalance check count: 93 INFO [2022-01-10 05:38:56,512] jepsen worker nemesis - couchbase.util Rebalance stuck count: 70 INFO [2022-01-10 05:38:56,655] jepsen worker 25 - jepsen.util 1015 :ok :read [8 :nil] INFO [2022-01-10 05:38:56,656] jepsen worker 25 - jepsen.util 1015 :invoke :write [8 36] INFO [2022-01-10 05:38:56,723] jepsen worker 69 - jepsen.util 1059 :ok :read [23 26] INFO [2022-01-10 05:38:56,724] jepsen worker 69 - jepsen.util 1059 :invoke :write [23 42] INFO [2022-01-10 05:38:56,736] jepsen worker 0 - jepsen.util 900 :info :write [0 8] :RequestTimeoutException INFO [2022-01-10 05:38:56,736] jepsen worker 0 - jepsen.util 990 :invoke :read [0 nil] INFO [2022-01-10 05:38:56,881] jepsen worker 89 - jepsen.util 1079 :ok :read [29 33] INFO [2022-01-10 05:38:56,881] jepsen worker 89 - jepsen.util 1079 :invoke :write [29 13] INFO [2022-01-10 05:38:56,885] jepsen worker 57 - jepsen.util 1047 :info :write [19 31] :RequestTimeoutException INFO [2022-01-10 05:38:56,886] jepsen worker 57 - jepsen.util 1137 :invoke :read [19 nil] INFO [2022-01-10 05:38:56,967] jepsen worker 85 - jepsen.util 1075 :info :write [28 47] :RequestTimeoutException INFO [2022-01-10 05:38:56,968] jepsen worker 85 - jepsen.util 1165 :invoke :read [28 nil] INFO [2022-01-10 05:38:57,095] jepsen worker 75 - jepsen.util 795 :info :write [25 32] :RequestTimeoutException INFO [2022-01-10 05:38:57,096] jepsen worker 75 - jepsen.util 885 :invoke :read [25 nil] INFO [2022-01-10 05:38:57,166] jepsen worker 58 - jepsen.util 958 :info :write [19 16] :RequestTimeoutException INFO [2022-01-10 05:38:57,166] jepsen worker 58 - jepsen.util 1048 :invoke :read [19 nil] INFO [2022-01-10 05:38:57,335] jepsen worker 29 - jepsen.util 929 :info :write [9 22] :RequestTimeoutException INFO [2022-01-10 05:38:57,335] jepsen worker 29 - jepsen.util 1019 :invoke :write [9 19] INFO [2022-01-10 05:38:57,368] jepsen worker 22 - jepsen.util 922 :ok :read [7 :nil] INFO [2022-01-10 05:38:57,369] jepsen worker 22 - jepsen.util 922 :invoke :read [7 nil] INFO [2022-01-10 05:38:57,431] jepsen worker 39 - jepsen.util 1119 :info :write [13 38] :SyncWriteAmbiguous INFO [2022-01-10 05:38:57,432] jepsen worker 39 - jepsen.util 1209 :invoke :read [13 nil] INFO [2022-01-10 05:38:57,518] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1461988304093568}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1411764705882353}} INFO [2022-01-10 05:38:57,518] jepsen worker nemesis - couchbase.util Rebalance check count: 94 INFO [2022-01-10 05:38:57,519] jepsen worker nemesis - couchbase.util Rebalance stuck count: 71 INFO [2022-01-10 05:38:57,728] jepsen worker 8 - jepsen.util 908 :ok :read [2 14] INFO [2022-01-10 05:38:57,729] jepsen worker 8 - jepsen.util 908 :invoke :write [2 10] INFO [2022-01-10 05:38:57,771] jepsen worker 14 - jepsen.util 1094 :info :write [4 5] :RequestTimeoutException INFO [2022-01-10 05:38:57,772] jepsen worker 14 - jepsen.util 1184 :invoke :read [4 nil] INFO [2022-01-10 05:38:57,792] jepsen worker 76 - jepsen.util 796 :info :write [25 48] :RequestTimeoutException INFO [2022-01-10 05:38:57,792] jepsen worker 76 - jepsen.util 886 :invoke :read [25 nil] INFO [2022-01-10 05:38:57,795] jepsen worker 3 - jepsen.util 1173 :info :write [1 13] :RequestTimeoutException INFO [2022-01-10 05:38:57,795] jepsen worker 3 - jepsen.util 1263 :invoke :write [1 48] INFO [2022-01-10 05:38:57,814] jepsen worker 19 - jepsen.util 1009 :info :write [6 48] :RequestTimeoutException INFO [2022-01-10 05:38:57,815] jepsen worker 19 - jepsen.util 1099 :invoke :write [6 4] INFO [2022-01-10 05:38:57,819] jepsen worker 77 - jepsen.util 1247 :ok :read [25 :nil] INFO [2022-01-10 05:38:57,820] jepsen worker 77 - jepsen.util 1247 :invoke :read [25 nil] INFO [2022-01-10 05:38:57,842] jepsen worker 65 - jepsen.util 1055 :info :write [21 1] :RequestTimeoutException INFO [2022-01-10 05:38:57,842] jepsen worker 65 - jepsen.util 1145 :invoke :read [21 nil] INFO [2022-01-10 05:38:57,860] jepsen worker 14 - jepsen.util 1184 :ok :read [4 :nil] INFO [2022-01-10 05:38:57,861] jepsen worker 14 - jepsen.util 1184 :invoke :read [4 nil] INFO [2022-01-10 05:38:57,950] jepsen worker 14 - jepsen.util 1184 :ok :read [4 :nil] INFO [2022-01-10 05:38:57,951] jepsen worker 14 - jepsen.util 1184 :invoke :write [4 33] INFO [2022-01-10 05:38:58,023] jepsen worker 58 - jepsen.util 1048 :ok :read [19 31] INFO [2022-01-10 05:38:58,024] jepsen worker 58 - jepsen.util 1048 :invoke :write [19 27] INFO [2022-01-10 05:38:58,042] jepsen worker 44 - jepsen.util 1034 :info :write [14 22] :RequestTimeoutException INFO [2022-01-10 05:38:58,042] jepsen worker 44 - jepsen.util 1124 :invoke :read [14 nil] INFO [2022-01-10 05:38:58,052] jepsen worker 74 - jepsen.util 1244 :info :write [24 19] :RequestTimeoutException INFO [2022-01-10 05:38:58,052] jepsen worker 74 - jepsen.util 1334 :invoke :read [24 nil] INFO [2022-01-10 05:38:58,121] jepsen worker 49 - jepsen.util 859 :ok :read [16 38] INFO [2022-01-10 05:38:58,122] jepsen worker 49 - jepsen.util 859 :invoke :write [16 14] INFO [2022-01-10 05:38:58,141] jepsen worker 40 - jepsen.util 850 :ok :read [13 15] INFO [2022-01-10 05:38:58,142] jepsen worker 40 - jepsen.util 850 :invoke :read [13 nil] INFO [2022-01-10 05:38:58,149] jepsen worker 57 - jepsen.util 1137 :ok :read [19 31] INFO [2022-01-10 05:38:58,149] jepsen worker 57 - jepsen.util 1137 :invoke :write [19 10] INFO [2022-01-10 05:38:58,335] jepsen worker 21 - jepsen.util 1281 :info :write [7 22] :RequestTimeoutException INFO [2022-01-10 05:38:58,335] jepsen worker 21 - jepsen.util 1371 :invoke :read [7 nil] INFO [2022-01-10 05:38:58,345] jepsen worker 75 - jepsen.util 885 :ok :read [25 :nil] INFO [2022-01-10 05:38:58,346] jepsen worker 75 - jepsen.util 885 :invoke :read [25 nil] INFO [2022-01-10 05:38:58,347] jepsen worker 77 - jepsen.util 1247 :ok :read [25 :nil] INFO [2022-01-10 05:38:58,348] jepsen worker 77 - jepsen.util 1247 :invoke :write [25 36] INFO [2022-01-10 05:38:58,382] jepsen worker 42 - jepsen.util 942 :info :write [14 37] :RequestTimeoutException INFO [2022-01-10 05:38:58,382] jepsen worker 42 - jepsen.util 1032 :invoke :read [14 nil] INFO [2022-01-10 05:38:58,451] jepsen worker 43 - jepsen.util 1033 :info :write [14 23] :RequestTimeoutException INFO [2022-01-10 05:38:58,452] jepsen worker 43 - jepsen.util 1123 :invoke :write [14 33] INFO [2022-01-10 05:38:58,505] jepsen worker 67 - jepsen.util 967 :info :write [22 39] :RequestTimeoutException INFO [2022-01-10 05:38:58,506] jepsen worker 67 - jepsen.util 1057 :invoke :read [22 nil] INFO [2022-01-10 05:38:58,525] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1461988304093568}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1411764705882353}} INFO [2022-01-10 05:38:58,525] jepsen worker nemesis - couchbase.util Rebalance check count: 95 INFO [2022-01-10 05:38:58,525] jepsen worker nemesis - couchbase.util Rebalance stuck count: 72 INFO [2022-01-10 05:38:58,773] jepsen worker 2 - jepsen.util 1082 :ok :read [0 46] INFO [2022-01-10 05:38:58,773] jepsen worker 2 - jepsen.util 1082 :invoke :read [0 nil] INFO [2022-01-10 05:38:58,786] jepsen worker 66 - jepsen.util 786 :info :write [22 11] :RequestTimeoutException INFO [2022-01-10 05:38:58,787] jepsen worker 66 - jepsen.util 876 :invoke :read [22 nil] INFO [2022-01-10 05:38:58,865] jepsen worker 40 - jepsen.util 850 :ok :read [13 15] INFO [2022-01-10 05:38:58,866] jepsen worker 40 - jepsen.util 850 :invoke :read [13 nil] INFO [2022-01-10 05:38:58,880] jepsen worker 50 - jepsen.util 1130 :ok :read [16 38] INFO [2022-01-10 05:38:58,881] jepsen worker 50 - jepsen.util 1130 :invoke :read [16 nil] INFO [2022-01-10 05:38:58,949] jepsen worker 85 - jepsen.util 1165 :ok :read [28 13] INFO [2022-01-10 05:38:58,949] jepsen worker 85 - jepsen.util 1165 :invoke :read [28 nil] INFO [2022-01-10 05:38:58,967] jepsen worker 50 - jepsen.util 1130 :ok :read [16 38] INFO [2022-01-10 05:38:58,967] jepsen worker 50 - jepsen.util 1130 :invoke :read [16 nil] INFO [2022-01-10 05:38:59,012] jepsen worker 39 - jepsen.util 1209 :ok :read [13 15] INFO [2022-01-10 05:38:59,012] jepsen worker 39 - jepsen.util 1209 :invoke :write [13 2] INFO [2022-01-10 05:38:59,042] jepsen worker 85 - jepsen.util 1165 :ok :read [28 13] INFO [2022-01-10 05:38:59,043] jepsen worker 85 - jepsen.util 1165 :invoke :read [28 nil] INFO [2022-01-10 05:38:59,054] jepsen worker 50 - jepsen.util 1130 :ok :read [16 38] INFO [2022-01-10 05:38:59,054] jepsen worker 50 - jepsen.util 1130 :invoke :read [16 nil] INFO [2022-01-10 05:38:59,128] jepsen worker 85 - jepsen.util 1165 :ok :read [28 13] INFO [2022-01-10 05:38:59,130] jepsen worker 85 - jepsen.util 1165 :invoke :read [28 nil] INFO [2022-01-10 05:38:59,130] jepsen worker 50 - jepsen.util 1130 :ok :read [16 38] INFO [2022-01-10 05:38:59,131] jepsen worker 50 - jepsen.util 1130 :invoke :read [16 nil] INFO [2022-01-10 05:38:59,140] jepsen worker 83 - jepsen.util 1163 :ok :read [27 30] INFO [2022-01-10 05:38:59,141] jepsen worker 83 - jepsen.util 1163 :invoke :write [27 20] INFO [2022-01-10 05:38:59,167] jepsen worker 22 - jepsen.util 922 :ok :read [7 :nil] INFO [2022-01-10 05:38:59,167] jepsen worker 22 - jepsen.util 922 :invoke :read [7 nil] INFO [2022-01-10 05:38:59,217] jepsen worker 85 - jepsen.util 1165 :ok :read [28 13] INFO [2022-01-10 05:38:59,218] jepsen worker 85 - jepsen.util 1165 :invoke :read [28 nil] INFO [2022-01-10 05:38:59,226] jepsen worker 16 - jepsen.util 1096 :info :write [5 32] :RequestTimeoutException INFO [2022-01-10 05:38:59,227] jepsen worker 16 - jepsen.util 1186 :invoke :write [5 5] INFO [2022-01-10 05:38:59,233] jepsen worker 50 - jepsen.util 1130 :ok :read [16 38] INFO [2022-01-10 05:38:59,234] jepsen worker 50 - jepsen.util 1130 :invoke :read [16 nil] INFO [2022-01-10 05:38:59,307] jepsen worker 67 - jepsen.util 1057 :ok :read [22 41] INFO [2022-01-10 05:38:59,307] jepsen worker 67 - jepsen.util 1057 :invoke :read [22 nil] INFO [2022-01-10 05:38:59,309] jepsen worker 40 - jepsen.util 850 :ok :read [13 15] INFO [2022-01-10 05:38:59,310] jepsen worker 40 - jepsen.util 850 :invoke :read [13 nil] INFO [2022-01-10 05:38:59,316] jepsen worker 88 - jepsen.util 1168 :ok :read [29 33] INFO [2022-01-10 05:38:59,317] jepsen worker 88 - jepsen.util 1168 :invoke :write [29 35] INFO [2022-01-10 05:38:59,327] jepsen worker 85 - jepsen.util 1165 :ok :read [28 13] INFO [2022-01-10 05:38:59,327] jepsen worker 85 - jepsen.util 1165 :invoke :write [28 45] INFO [2022-01-10 05:38:59,334] jepsen worker 50 - jepsen.util 1130 :ok :read [16 38] INFO [2022-01-10 05:38:59,335] jepsen worker 50 - jepsen.util 1130 :invoke :read [16 nil] INFO [2022-01-10 05:38:59,386] jepsen worker 36 - jepsen.util 756 :ok :read [12 :nil] INFO [2022-01-10 05:38:59,387] jepsen worker 36 - jepsen.util 756 :invoke :write [12 9] INFO [2022-01-10 05:38:59,411] jepsen worker 0 - jepsen.util 990 :ok :read [0 46] INFO [2022-01-10 05:38:59,412] jepsen worker 0 - jepsen.util 990 :invoke :write [0 21] INFO [2022-01-10 05:38:59,439] jepsen worker 50 - jepsen.util 1130 :ok :read [16 38] INFO [2022-01-10 05:38:59,440] jepsen worker 50 - jepsen.util 1130 :invoke :write [16 0] INFO [2022-01-10 05:38:59,480] jepsen worker 2 - jepsen.util 1082 :ok :read [0 46] INFO [2022-01-10 05:38:59,480] jepsen worker 2 - jepsen.util 1082 :invoke :read [0 nil] INFO [2022-01-10 05:38:59,523] jepsen worker 66 - jepsen.util 876 :ok :read [22 41] INFO [2022-01-10 05:38:59,524] jepsen worker 66 - jepsen.util 876 :invoke :read [22 nil] INFO [2022-01-10 05:38:59,532] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1461988304093568}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1411764705882353}} INFO [2022-01-10 05:38:59,532] jepsen worker nemesis - couchbase.util Rebalance check count: 96 INFO [2022-01-10 05:38:59,532] jepsen worker nemesis - couchbase.util Rebalance stuck count: 73 WARN [2022-01-10 05:38:59,647] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9359950,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10186995,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9451546,"encode_duration_us":16240,"operation_id":"0x2107","timeout_ms":10000,"last_server_duration_us":9312957,"total_duration_us":10006390},{"operation_name":"upsert","last_dispatch_duration_us":9327290,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11192798,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9704746,"encode_duration_us":16648,"operation_id":"0x1f64","timeout_ms":10000,"last_server_duration_us":9091564,"total_duration_us":10010516},{"operation_name":"upsert","last_dispatch_duration_us":9437280,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10517033,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9676998,"encode_duration_us":16148,"operation_id":"0x20c6","timeout_ms":10000,"last_server_duration_us":9367851,"total_duration_us":10006584}],"total_count":3}} WARN [2022-01-10 05:38:59,747] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9327290,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":11192798,"total_server_duration_us":9704746,"encode_duration_us":16648,"operation_id":"0x1f64","timeout_ms":10000,"last_server_duration_us":9091564,"total_duration_us":10010516},{"operation_name":"upsert","last_dispatch_duration_us":8107287,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12758808,"total_server_duration_us":1571124,"encode_duration_us":17333,"operation_id":"0x2050","timeout_ms":10000,"last_server_duration_us":42091,"total_duration_us":10010029},{"operation_name":"upsert","last_dispatch_duration_us":1752007,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5686597,"total_server_duration_us":434697,"encode_duration_us":16222,"operation_id":"0x20ba","timeout_ms":10000,"last_server_duration_us":86769,"total_duration_us":10008728},{"operation_name":"upsert","last_dispatch_duration_us":1345875,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2303134,"total_server_duration_us":274401,"encode_duration_us":17818,"operation_id":"0x1ffe","timeout_ms":10000,"last_server_duration_us":75755,"total_duration_us":10008280},{"operation_name":"upsert","last_dispatch_duration_us":9437280,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10517033,"total_server_duration_us":9676998,"encode_duration_us":16148,"operation_id":"0x20c6","timeout_ms":10000,"last_server_duration_us":9367851,"total_duration_us":10006584},{"operation_name":"upsert","last_dispatch_duration_us":90255,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":6304010,"total_server_duration_us":58491,"encode_duration_us":16298,"operation_id":"0x2082","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10006444},{"operation_name":"upsert","last_dispatch_duration_us":9359950,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10186995,"total_server_duration_us":9451546,"encode_duration_us":16240,"operation_id":"0x2107","timeout_ms":10000,"last_server_duration_us":9312957,"total_duration_us":10006390},{"operation_name":"upsert","last_dispatch_duration_us":2760757,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6843361,"total_server_duration_us":1422206,"encode_duration_us":21325,"operation_id":"0x20ff","timeout_ms":10000,"last_server_duration_us":30350,"total_duration_us":10006217},{"operation_name":"upsert","last_dispatch_duration_us":3244662,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6413505,"total_server_duration_us":802145,"encode_duration_us":18275,"operation_id":"0x205d","timeout_ms":10000,"last_server_duration_us":298496,"total_duration_us":10004733},{"operation_name":"upsert","last_dispatch_duration_us":775939,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6305828,"total_server_duration_us":831244,"encode_duration_us":16995,"operation_id":"0x1fc6","timeout_ms":10000,"last_server_duration_us":235681,"total_duration_us":10002946}],"total_count":17}} INFO [2022-01-10 05:38:59,772] jepsen worker 6 - jepsen.util 906 :info :write [2 17] :RequestTimeoutException INFO [2022-01-10 05:38:59,772] jepsen worker 6 - jepsen.util 996 :invoke :write [2 49] INFO [2022-01-10 05:38:59,796] jepsen worker 34 - jepsen.util 934 :info :write [11 11] :RequestTimeoutException INFO [2022-01-10 05:38:59,797] jepsen worker 34 - jepsen.util 1024 :invoke :write [11 20] INFO [2022-01-10 05:39:00,376] jepsen worker 4 - jepsen.util 994 :info :write [1 24] :RequestTimeoutException INFO [2022-01-10 05:39:00,377] jepsen worker 4 - jepsen.util 1084 :invoke :read [1 nil] INFO [2022-01-10 05:39:00,559] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.152046783625731}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1470588235294118}} INFO [2022-01-10 05:39:00,559] jepsen worker nemesis - couchbase.util Rebalance check count: 97 INFO [2022-01-10 05:39:00,559] jepsen worker nemesis - couchbase.util Rebalance stuck count: 73 INFO [2022-01-10 05:39:00,581] jepsen worker 22 - jepsen.util 922 :ok :read [7 :nil] INFO [2022-01-10 05:39:00,581] jepsen worker 22 - jepsen.util 922 :invoke :write [7 48] INFO [2022-01-10 05:39:00,611] jepsen worker 61 - jepsen.util 1141 :info :write [20 38] :RequestTimeoutException INFO [2022-01-10 05:39:00,612] jepsen worker 61 - jepsen.util 1231 :invoke :read [20 nil] INFO [2022-01-10 05:39:00,615] jepsen worker 78 - jepsen.util 888 :info :write [26 42] :RequestTimeoutException INFO [2022-01-10 05:39:00,615] jepsen worker 78 - jepsen.util 978 :invoke :read [26 nil] INFO [2022-01-10 05:39:00,627] jepsen worker 80 - jepsen.util 980 :info :write [26 16] :RequestTimeoutException INFO [2022-01-10 05:39:00,630] jepsen worker 80 - jepsen.util 1070 :invoke :read [26 nil] INFO [2022-01-10 05:39:00,646] jepsen worker 67 - jepsen.util 1057 :ok :read [22 41] INFO [2022-01-10 05:39:00,647] jepsen worker 40 - jepsen.util 850 :ok :read [13 15] INFO [2022-01-10 05:39:00,647] jepsen worker 67 - jepsen.util 1057 :invoke :write [22 30] INFO [2022-01-10 05:39:00,648] jepsen worker 40 - jepsen.util 850 :invoke :write [13 40] INFO [2022-01-10 05:39:00,722] jepsen worker 63 - jepsen.util 963 :ok :read [21 7] INFO [2022-01-10 05:39:00,723] jepsen worker 63 - jepsen.util 963 :invoke :read [21 nil] INFO [2022-01-10 05:39:00,830] jepsen worker 11 - jepsen.util 911 :info :write [3 49] :RequestTimeoutException INFO [2022-01-10 05:39:00,831] jepsen worker 11 - jepsen.util 1001 :invoke :read [3 nil] INFO [2022-01-10 05:39:01,002] jepsen worker 79 - jepsen.util 979 :info :write [26 16] :RequestTimeoutException INFO [2022-01-10 05:39:01,003] jepsen worker 79 - jepsen.util 1069 :invoke :write [26 39] INFO [2022-01-10 05:39:01,071] jepsen worker 81 - jepsen.util 981 :info :write [27 6] :RequestTimeoutException INFO [2022-01-10 05:39:01,071] jepsen worker 81 - jepsen.util 1071 :invoke :read [27 nil] INFO [2022-01-10 05:39:01,165] jepsen worker 59 - jepsen.util 1139 :ok :read [19 31] INFO [2022-01-10 05:39:01,166] jepsen worker 59 - jepsen.util 1139 :invoke :write [19 40] WARN [2022-01-10 05:39:01,192] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18160904,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18160904,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9028877,"encode_duration_us":16313,"operation_id":"0x1f65","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10006331},{"operation_name":"upsert","last_dispatch_duration_us":9404175,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11814915,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9259242,"encode_duration_us":16385,"operation_id":"0x20b7","timeout_ms":10000,"last_server_duration_us":9130971,"total_duration_us":10006361}],"total_count":2}} WARN [2022-01-10 05:39:01,211] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18112298,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18500173,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9052732,"encode_duration_us":19224,"operation_id":"0x1eec","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10009625}],"total_count":1}} WARN [2022-01-10 05:39:01,212] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9444893,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9444893,"total_server_duration_us":57880,"operation_id":"0x2027","timeout_ms":30000,"last_server_duration_us":57880,"total_duration_us":12575580},{"operation_name":"get","last_dispatch_duration_us":3725266,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3725266,"total_server_duration_us":10272,"operation_id":"0x2131","timeout_ms":30000,"last_server_duration_us":10272,"total_duration_us":11440203},{"operation_name":"get","last_dispatch_duration_us":9119113,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9119113,"total_server_duration_us":13091,"operation_id":"0x2063","timeout_ms":30000,"last_server_duration_us":13091,"total_duration_us":10711926},{"operation_name":"upsert","last_dispatch_duration_us":2070677,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2070677,"total_server_duration_us":234557,"encode_duration_us":17179,"operation_id":"0x20f6","timeout_ms":10000,"last_server_duration_us":234557,"total_duration_us":10011071},{"operation_name":"upsert","last_dispatch_duration_us":173249,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":592922,"total_server_duration_us":92322,"encode_duration_us":18234,"operation_id":"0x20df","timeout_ms":10000,"last_server_duration_us":67221,"total_duration_us":10010203},{"operation_name":"upsert","last_dispatch_duration_us":7454952,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7454952,"total_server_duration_us":232763,"encode_duration_us":19228,"operation_id":"0x215c","timeout_ms":10000,"last_server_duration_us":232763,"total_duration_us":10008925},{"operation_name":"upsert","last_dispatch_duration_us":13875376,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":13875376,"total_server_duration_us":39109,"encode_duration_us":16962,"operation_id":"0x1fbd","timeout_ms":10000,"last_server_duration_us":39109,"total_duration_us":10003080},{"operation_name":"upsert","last_dispatch_duration_us":9449098,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9449098,"total_server_duration_us":31203,"encode_duration_us":18771,"operation_id":"0x205a","timeout_ms":10000,"last_server_duration_us":31203,"total_duration_us":10003037},{"operation_name":"upsert","last_dispatch_duration_us":9144298,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9144298,"total_server_duration_us":234332,"encode_duration_us":16797,"operation_id":"0x2106","timeout_ms":10000,"last_server_duration_us":234332,"total_duration_us":10002452},{"operation_name":"upsert","last_dispatch_duration_us":3806804,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3806804,"total_server_duration_us":55912,"encode_duration_us":17276,"operation_id":"0x2105","timeout_ms":10000,"last_server_duration_us":55912,"total_duration_us":10002420}],"total_count":17}} WARN [2022-01-10 05:39:01,290] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9093093,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14918182,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9257192,"encode_duration_us":16878,"operation_id":"0x1f7e","timeout_ms":10000,"last_server_duration_us":9050106,"total_duration_us":10004751}],"total_count":1}} WARN [2022-01-10 05:39:01,290] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10041643,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10041643,"total_server_duration_us":27,"operation_id":"0x2111","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":10042970},{"operation_name":"upsert","last_dispatch_duration_us":10044899,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11738443,"total_server_duration_us":674256,"encode_duration_us":16434,"operation_id":"0x20b8","timeout_ms":10000,"last_server_duration_us":569092,"total_duration_us":10010225},{"operation_name":"upsert","last_dispatch_duration_us":9515256,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":12747109,"total_server_duration_us":209843,"encode_duration_us":16977,"operation_id":"0x205b","timeout_ms":10000,"last_server_duration_us":28493,"total_duration_us":10008907},{"operation_name":"upsert","last_dispatch_duration_us":9036289,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10262847,"total_server_duration_us":9317243,"encode_duration_us":17709,"operation_id":"0x1ff3","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10008729},{"operation_name":"upsert","last_dispatch_duration_us":9907286,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10755595,"total_server_duration_us":62628,"encode_duration_us":21120,"operation_id":"0x20c1","timeout_ms":10000,"last_server_duration_us":23729,"total_duration_us":10008643},{"operation_name":"upsert","last_dispatch_duration_us":33597,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":1074544,"total_server_duration_us":582411,"encode_duration_us":24318,"operation_id":"0x1ff7","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10008505},{"operation_name":"upsert","last_dispatch_duration_us":8468009,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9769930,"total_server_duration_us":602299,"encode_duration_us":25927,"operation_id":"0x1ff6","timeout_ms":10000,"last_server_duration_us":21163,"total_duration_us":10006234},{"operation_name":"upsert","last_dispatch_duration_us":1038719,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9313161,"total_server_duration_us":501030,"encode_duration_us":17219,"operation_id":"0x2023","timeout_ms":10000,"last_server_duration_us":235456,"total_duration_us":10006021},{"operation_name":"upsert","last_dispatch_duration_us":1422829,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10219311,"total_server_duration_us":1393008,"encode_duration_us":17328,"operation_id":"0x1ff4","timeout_ms":10000,"last_server_duration_us":418977,"total_duration_us":10005828},{"operation_name":"upsert","last_dispatch_duration_us":1498800,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9132816,"total_server_duration_us":631878,"encode_duration_us":17480,"operation_id":"0x2087","timeout_ms":10000,"last_server_duration_us":33524,"total_duration_us":10004085}],"total_count":22}} WARN [2022-01-10 05:39:01,292] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":594470,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10220633,"total_server_duration_us":378726,"encode_duration_us":17077,"operation_id":"0x215a","timeout_ms":10000,"last_server_duration_us":26596,"total_duration_us":10008807},{"operation_name":"upsert","last_dispatch_duration_us":9241523,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9720846,"total_server_duration_us":115320,"encode_duration_us":16838,"operation_id":"0x2130","timeout_ms":10000,"last_server_duration_us":72589,"total_duration_us":10008332},{"operation_name":"upsert","last_dispatch_duration_us":1781647,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8544206,"total_server_duration_us":430548,"encode_duration_us":16446,"operation_id":"0x2146","timeout_ms":10000,"last_server_duration_us":239067,"total_duration_us":10008187},{"operation_name":"upsert","last_dispatch_duration_us":9404175,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11814915,"total_server_duration_us":9259242,"encode_duration_us":16385,"operation_id":"0x20b7","timeout_ms":10000,"last_server_duration_us":9130971,"total_duration_us":10006361},{"operation_name":"upsert","last_dispatch_duration_us":11223192,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":13676647,"total_server_duration_us":964487,"encode_duration_us":21015,"operation_id":"0x202c","timeout_ms":10000,"last_server_duration_us":495911,"total_duration_us":10006034},{"operation_name":"upsert","last_dispatch_duration_us":10109492,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":13057079,"total_server_duration_us":612664,"encode_duration_us":17631,"operation_id":"0x20a2","timeout_ms":10000,"last_server_duration_us":60002,"total_duration_us":10005500},{"operation_name":"upsert","last_dispatch_duration_us":6329350,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":14042442,"total_server_duration_us":1628849,"encode_duration_us":27146,"operation_id":"0x200f","timeout_ms":10000,"last_server_duration_us":197078,"total_duration_us":10004950},{"operation_name":"upsert","last_dispatch_duration_us":9952360,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10894717,"total_server_duration_us":186058,"encode_duration_us":16862,"operation_id":"0x211c","timeout_ms":10000,"last_server_duration_us":105664,"total_duration_us":10004460},{"operation_name":"upsert","last_dispatch_duration_us":9602697,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":15797249,"total_server_duration_us":306638,"encode_duration_us":18695,"operation_id":"0x1fff","timeout_ms":10000,"last_server_duration_us":75616,"total_duration_us":10004093},{"operation_name":"upsert","last_dispatch_duration_us":569095,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10055808,"total_server_duration_us":553470,"encode_duration_us":19276,"operation_id":"0x215e","timeout_ms":10000,"last_server_duration_us":374609,"total_duration_us":10003233}],"total_count":19}} WARN [2022-01-10 05:39:01,292] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8264907,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10229518,"total_server_duration_us":377016,"encode_duration_us":17447,"operation_id":"0x2001","timeout_ms":10000,"last_server_duration_us":103117,"total_duration_us":10010733},{"operation_name":"upsert","last_dispatch_duration_us":4904981,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":4904981,"total_server_duration_us":26418,"encode_duration_us":16255,"operation_id":"0x20f5","timeout_ms":10000,"last_server_duration_us":26418,"total_duration_us":10009892},{"operation_name":"upsert","last_dispatch_duration_us":11383793,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11383793,"total_server_duration_us":10027251,"encode_duration_us":18716,"operation_id":"0x20e2","timeout_ms":10000,"last_server_duration_us":10027251,"total_duration_us":10009788},{"operation_name":"upsert","last_dispatch_duration_us":2492568,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2492568,"total_server_duration_us":201269,"encode_duration_us":16817,"operation_id":"0x20f3","timeout_ms":10000,"last_server_duration_us":201269,"total_duration_us":10009772},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":20595,"operation_id":"0x2145","timeout_ms":10000,"total_duration_us":10009502},{"operation_name":"upsert","last_dispatch_duration_us":8970642,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":11079968,"total_server_duration_us":697173,"encode_duration_us":18660,"operation_id":"0x1f93","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10008044},{"operation_name":"upsert","last_dispatch_duration_us":45445,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9107905,"total_server_duration_us":282317,"encode_duration_us":16368,"operation_id":"0x202e","timeout_ms":10000,"last_server_duration_us":34413,"total_duration_us":10007674},{"operation_name":"upsert","last_dispatch_duration_us":276424,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":3164009,"total_server_duration_us":464174,"encode_duration_us":16374,"operation_id":"0x2180","timeout_ms":10000,"last_server_duration_us":55425,"total_duration_us":10007533},{"operation_name":"upsert","last_dispatch_duration_us":1321891,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10963594,"total_server_duration_us":1077134,"encode_duration_us":17574,"operation_id":"0x2060","timeout_ms":10000,"last_server_duration_us":79829,"total_duration_us":10006908},{"operation_name":"upsert","last_dispatch_duration_us":195917,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10038809,"total_server_duration_us":439334,"encode_duration_us":24270,"operation_id":"0x1fe6","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10003272}],"total_count":17}} INFO [2022-01-10 05:39:01,316] jepsen worker 2 - jepsen.util 1082 :ok :read [0 46] INFO [2022-01-10 05:39:01,317] jepsen worker 2 - jepsen.util 1082 :invoke :read [0 nil] INFO [2022-01-10 05:39:01,343] jepsen worker 82 - jepsen.util 982 :ok :read [27 30] INFO [2022-01-10 05:39:01,344] jepsen worker 82 - jepsen.util 982 :invoke :write [27 6] INFO [2022-01-10 05:39:01,349] jepsen worker 66 - jepsen.util 876 :ok :read [22 41] INFO [2022-01-10 05:39:01,350] jepsen worker 66 - jepsen.util 876 :invoke :write [22 0] WARN [2022-01-10 05:39:01,391] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10076150,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11061398,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9107141,"encode_duration_us":16148,"operation_id":"0x1f14","timeout_ms":10000,"last_server_duration_us":9066042,"total_duration_us":10003538},{"operation_name":"upsert","last_dispatch_duration_us":9036289,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10262847,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9317243,"encode_duration_us":17709,"operation_id":"0x1ff3","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10008729}],"total_count":2}} WARN [2022-01-10 05:39:01,393] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9360279,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10273105,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9249800,"encode_duration_us":16210,"operation_id":"0x1f5b","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10004001},{"operation_name":"upsert","last_dispatch_duration_us":9126083,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12955908,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10168930,"encode_duration_us":18426,"operation_id":"0x1f88","timeout_ms":10000,"last_server_duration_us":9047983,"total_duration_us":10005432},{"operation_name":"upsert","last_dispatch_duration_us":11383793,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11383793,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":10027251,"encode_duration_us":18716,"operation_id":"0x20e2","timeout_ms":10000,"last_server_duration_us":10027251,"total_duration_us":10009788}],"total_count":3}} WARN [2022-01-10 05:39:01,491] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":371630,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":3122428,"total_server_duration_us":1119775,"encode_duration_us":17220,"operation_id":"0x1fef","timeout_ms":10000,"last_server_duration_us":41445,"total_duration_us":10009608},{"operation_name":"upsert","last_dispatch_duration_us":334202,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10033123,"total_server_duration_us":233612,"encode_duration_us":26060,"operation_id":"0x1ff1","timeout_ms":10000,"last_server_duration_us":62796,"total_duration_us":10007773},{"operation_name":"upsert","last_dispatch_duration_us":1301652,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10943289,"total_server_duration_us":309575,"encode_duration_us":17279,"operation_id":"0x2061","timeout_ms":10000,"last_server_duration_us":68015,"total_duration_us":10007274},{"operation_name":"upsert","last_dispatch_duration_us":7532506,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12402534,"total_server_duration_us":318261,"encode_duration_us":16354,"operation_id":"0x1fb3","timeout_ms":10000,"last_server_duration_us":267843,"total_duration_us":10007253},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":20599,"operation_id":"0x217e","timeout_ms":10000,"total_duration_us":10007024},{"operation_name":"upsert","last_dispatch_duration_us":7172492,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12163214,"total_server_duration_us":264165,"encode_duration_us":16823,"operation_id":"0x1fee","timeout_ms":10000,"last_server_duration_us":210191,"total_duration_us":10005712},{"operation_name":"upsert","last_dispatch_duration_us":1472293,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11060004,"total_server_duration_us":295168,"encode_duration_us":16949,"operation_id":"0x205e","timeout_ms":10000,"last_server_duration_us":64214,"total_duration_us":10004784},{"operation_name":"upsert","last_dispatch_duration_us":390839,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9913387,"total_server_duration_us":167240,"encode_duration_us":19368,"operation_id":"0x1fea","timeout_ms":10000,"last_server_duration_us":54818,"total_duration_us":10002336},{"operation_name":"upsert","last_dispatch_duration_us":75617,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":3050941,"total_server_duration_us":249561,"encode_duration_us":19071,"operation_id":"0x2002","timeout_ms":10000,"last_server_duration_us":21163,"total_duration_us":10002258},{"operation_name":"upsert","last_dispatch_duration_us":9253557,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9350299,"total_server_duration_us":9167874,"encode_duration_us":20511,"operation_id":"0x1fe9","timeout_ms":10000,"last_server_duration_us":9141633,"total_duration_us":9367002}],"total_count":21}} INFO [2022-01-10 05:39:01,565] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.152046783625731}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1470588235294118}} INFO [2022-01-10 05:39:01,565] jepsen worker nemesis - couchbase.util Rebalance check count: 98 INFO [2022-01-10 05:39:01,569] jepsen worker nemesis - couchbase.util Rebalance stuck count: 74 INFO [2022-01-10 05:39:01,624] jepsen worker 31 - jepsen.util 1291 :info :write [10 4] :RequestTimeoutException INFO [2022-01-10 05:39:01,625] jepsen worker 31 - jepsen.util 1381 :invoke :read [10 nil] INFO [2022-01-10 05:39:01,629] jepsen worker 13 - jepsen.util 1093 :ok :read [4 :nil] INFO [2022-01-10 05:39:01,633] jepsen worker 13 - jepsen.util 1093 :invoke :read [4 nil] INFO [2022-01-10 05:39:01,656] jepsen worker 87 - jepsen.util 1347 :ok :read [29 33] INFO [2022-01-10 05:39:01,656] jepsen worker 5 - jepsen.util 995 :info :write [1 14] :RequestTimeoutException INFO [2022-01-10 05:39:01,657] jepsen worker 5 - jepsen.util 1085 :invoke :read [1 nil] INFO [2022-01-10 05:39:01,657] jepsen worker 87 - jepsen.util 1347 :invoke :write [29 20] INFO [2022-01-10 05:39:01,706] jepsen worker 13 - jepsen.util 1093 :ok :read [4 :nil] INFO [2022-01-10 05:39:01,707] jepsen worker 13 - jepsen.util 1093 :invoke :read [4 nil] INFO [2022-01-10 05:39:01,722] jepsen worker 11 - jepsen.util 1001 :ok :read [3 2] INFO [2022-01-10 05:39:01,723] jepsen worker 11 - jepsen.util 1001 :invoke :read [3 nil] INFO [2022-01-10 05:39:01,823] jepsen worker 13 - jepsen.util 1093 :ok :read [4 :nil] INFO [2022-01-10 05:39:01,824] jepsen worker 13 - jepsen.util 1093 :invoke :write [4 47] INFO [2022-01-10 05:39:01,856] jepsen worker 11 - jepsen.util 1001 :ok :read [3 2] INFO [2022-01-10 05:39:01,856] jepsen worker 11 - jepsen.util 1001 :invoke :write [3 30] INFO [2022-01-10 05:39:01,935] jepsen worker 81 - jepsen.util 1071 :ok :read [27 30] INFO [2022-01-10 05:39:01,935] jepsen worker 81 - jepsen.util 1071 :invoke :read [27 nil] INFO [2022-01-10 05:39:01,982] jepsen worker 62 - jepsen.util 1142 :info :write [20 13] :RequestTimeoutException INFO [2022-01-10 05:39:01,982] jepsen worker 62 - jepsen.util 1232 :invoke :read [20 nil] INFO [2022-01-10 05:39:02,210] jepsen worker 60 - jepsen.util 1230 :info :write [20 30] :RequestTimeoutException INFO [2022-01-10 05:39:02,211] jepsen worker 60 - jepsen.util 1320 :invoke :write [20 20] INFO [2022-01-10 05:39:02,218] jepsen worker 44 - jepsen.util 1124 :ok :read [14 14] INFO [2022-01-10 05:39:02,218] jepsen worker 44 - jepsen.util 1124 :invoke :read [14 nil] INFO [2022-01-10 05:39:02,221] jepsen worker 75 - jepsen.util 885 :ok :read [25 :nil] INFO [2022-01-10 05:39:02,222] jepsen worker 75 - jepsen.util 885 :invoke :read [25 nil] INFO [2022-01-10 05:39:02,262] jepsen worker 2 - jepsen.util 1082 :ok :read [0 46] INFO [2022-01-10 05:39:02,262] jepsen worker 2 - jepsen.util 1082 :invoke :write [0 21] INFO [2022-01-10 05:39:02,307] jepsen worker 84 - jepsen.util 984 :info :write [28 21] :RequestTimeoutException INFO [2022-01-10 05:39:02,308] jepsen worker 84 - jepsen.util 1074 :invoke :write [28 25] INFO [2022-01-10 05:39:02,481] jepsen worker 54 - jepsen.util 1044 :info :write [18 38] :RequestTimeoutException INFO [2022-01-10 05:39:02,481] jepsen worker 54 - jepsen.util 1134 :invoke :write [18 27] INFO [2022-01-10 05:39:02,576] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.152046783625731}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1470588235294118}} INFO [2022-01-10 05:39:02,577] jepsen worker nemesis - couchbase.util Rebalance check count: 99 INFO [2022-01-10 05:39:02,577] jepsen worker nemesis - couchbase.util Rebalance stuck count: 75 INFO [2022-01-10 05:39:02,583] jepsen worker 81 - jepsen.util 1071 :ok :read [27 30] INFO [2022-01-10 05:39:02,584] jepsen worker 81 - jepsen.util 1071 :invoke :write [27 20] INFO [2022-01-10 05:39:02,735] jepsen worker 64 - jepsen.util 964 :info :write [21 18] :RequestTimeoutException INFO [2022-01-10 05:39:02,736] jepsen worker 64 - jepsen.util 1054 :invoke :write [21 20] INFO [2022-01-10 05:39:02,899] jepsen worker 38 - jepsen.util 1118 :info :write [12 35] :RequestTimeoutException INFO [2022-01-10 05:39:02,901] jepsen worker 38 - jepsen.util 1208 :invoke :write [12 28] INFO [2022-01-10 05:39:02,942] jepsen worker 41 - jepsen.util 761 :info :write [13 47] :RequestTimeoutException INFO [2022-01-10 05:39:02,942] jepsen worker 41 - jepsen.util 851 :invoke :read [13 nil] INFO [2022-01-10 05:39:02,962] jepsen worker 23 - jepsen.util 1013 :info :write [7 4] :RequestTimeoutException INFO [2022-01-10 05:39:02,963] jepsen worker 23 - jepsen.util 1103 :invoke :write [7 21] INFO [2022-01-10 05:39:03,016] jepsen worker 73 - jepsen.util 973 :info :write [24 21] :RequestTimeoutException INFO [2022-01-10 05:39:03,017] jepsen worker 73 - jepsen.util 1063 :invoke :read [24 nil] INFO [2022-01-10 05:39:03,053] jepsen worker 41 - jepsen.util 851 :ok :read [13 15] INFO [2022-01-10 05:39:03,053] jepsen worker 41 - jepsen.util 851 :invoke :read [13 nil] INFO [2022-01-10 05:39:03,157] jepsen worker 53 - jepsen.util 863 :info :write [17 14] :RequestTimeoutException INFO [2022-01-10 05:39:03,159] jepsen worker 53 - jepsen.util 953 :invoke :write [17 5] INFO [2022-01-10 05:39:03,183] jepsen worker 48 - jepsen.util 768 :info :write [16 19] :RequestTimeoutException INFO [2022-01-10 05:39:03,184] jepsen worker 48 - jepsen.util 858 :invoke :read [16 nil] INFO [2022-01-10 05:39:03,202] jepsen worker 56 - jepsen.util 1136 :info :write [18 42] :RequestTimeoutException INFO [2022-01-10 05:39:03,203] jepsen worker 56 - jepsen.util 1226 :invoke :write [18 44] INFO [2022-01-10 05:39:03,221] jepsen worker 33 - jepsen.util 1023 :info :write [11 34] :RequestTimeoutException INFO [2022-01-10 05:39:03,222] jepsen worker 33 - jepsen.util 1113 :invoke :write [11 29] INFO [2022-01-10 05:39:03,224] jepsen worker 86 - jepsen.util 716 :info :write [28 42] :RequestTimeoutException INFO [2022-01-10 05:39:03,225] jepsen worker 86 - jepsen.util 806 :invoke :read [28 nil] INFO [2022-01-10 05:39:03,279] jepsen worker 17 - jepsen.util 917 :info :write [5 34] :RequestTimeoutException INFO [2022-01-10 05:39:03,280] jepsen worker 17 - jepsen.util 1007 :invoke :write [5 32] INFO [2022-01-10 05:39:03,586] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.152046783625731}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1470588235294118}} INFO [2022-01-10 05:39:03,586] jepsen worker nemesis - couchbase.util Rebalance check count: 100 INFO [2022-01-10 05:39:03,587] jepsen worker nemesis - couchbase.util Rebalance stuck count: 76 INFO [2022-01-10 05:39:03,700] jepsen worker 15 - jepsen.util 735 :info :write [5 44] :RequestTimeoutException INFO [2022-01-10 05:39:03,700] jepsen worker 15 - jepsen.util 825 :invoke :write [5 41] INFO [2022-01-10 05:39:03,711] jepsen worker 10 - jepsen.util 1090 :info :write [3 3] :RequestTimeoutException INFO [2022-01-10 05:39:03,712] jepsen worker 10 - jepsen.util 1180 :invoke :write [3 26] INFO [2022-01-10 05:39:03,776] jepsen worker 18 - jepsen.util 828 :info :write [6 48] :RequestTimeoutException INFO [2022-01-10 05:39:03,777] jepsen worker 18 - jepsen.util 918 :invoke :read [6 nil] INFO [2022-01-10 05:39:03,949] jepsen worker 41 - jepsen.util 851 :ok :read [13 15] INFO [2022-01-10 05:39:03,949] jepsen worker 41 - jepsen.util 851 :invoke :write [13 19] INFO [2022-01-10 05:39:04,106] jepsen worker 51 - jepsen.util 1041 :ok :read [17 41] INFO [2022-01-10 05:39:04,106] jepsen worker 51 - jepsen.util 1041 :invoke :write [17 34] INFO [2022-01-10 05:39:04,121] jepsen worker 45 - jepsen.util 1215 :info :write [15 33] :RequestTimeoutException INFO [2022-01-10 05:39:04,122] jepsen worker 45 - jepsen.util 1305 :invoke :read [15 nil] INFO [2022-01-10 05:39:04,286] jepsen worker 27 - jepsen.util 927 :info :write [9 46] :RequestTimeoutException INFO [2022-01-10 05:39:04,287] jepsen worker 27 - jepsen.util 1017 :invoke :write [9 45] INFO [2022-01-10 05:39:04,470] jepsen worker 71 - jepsen.util 1151 :info :write [23 11] :RequestTimeoutException INFO [2022-01-10 05:39:04,471] jepsen worker 71 - jepsen.util 1241 :invoke :read [23 nil] INFO [2022-01-10 05:39:04,522] jepsen worker 28 - jepsen.util 1018 :info :write [9 21] :RequestTimeoutException INFO [2022-01-10 05:39:04,522] jepsen worker 28 - jepsen.util 1108 :invoke :write [9 46] INFO [2022-01-10 05:39:04,592] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1578947368421053}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1470588235294118}} INFO [2022-01-10 05:39:04,592] jepsen worker nemesis - couchbase.util Rebalance check count: 101 INFO [2022-01-10 05:39:04,592] jepsen worker nemesis - couchbase.util Rebalance stuck count: 76 INFO [2022-01-10 05:39:04,692] jepsen worker 47 - jepsen.util 1037 :info :write [15 1] :RequestTimeoutException INFO [2022-01-10 05:39:04,693] jepsen worker 47 - jepsen.util 1127 :invoke :read [15 nil] INFO [2022-01-10 05:39:04,712] jepsen worker 7 - jepsen.util 907 :info :write [2 24] :RequestTimeoutException INFO [2022-01-10 05:39:04,712] jepsen worker 7 - jepsen.util 997 :invoke :write [2 22] INFO [2022-01-10 05:39:04,965] jepsen worker 74 - jepsen.util 1334 :ok :read [24 18] INFO [2022-01-10 05:39:04,965] jepsen worker 74 - jepsen.util 1334 :invoke :write [24 32] INFO [2022-01-10 05:39:04,990] jepsen worker 20 - jepsen.util 830 :info :write [6 28] :RequestTimeoutException INFO [2022-01-10 05:39:04,991] jepsen worker 20 - jepsen.util 920 :invoke :read [6 nil] INFO [2022-01-10 05:39:05,077] jepsen worker 48 - jepsen.util 858 :ok :read [16 38] INFO [2022-01-10 05:39:05,077] jepsen worker 48 - jepsen.util 858 :invoke :read [16 nil] INFO [2022-01-10 05:39:05,101] jepsen worker 52 - jepsen.util 1042 :info :write [17 3] :RequestTimeoutException INFO [2022-01-10 05:39:05,102] jepsen worker 52 - jepsen.util 1132 :invoke :read [17 nil] INFO [2022-01-10 05:39:05,126] jepsen worker 52 - jepsen.util 1132 :ok :read [17 41] INFO [2022-01-10 05:39:05,126] jepsen worker 52 - jepsen.util 1132 :invoke :read [17 nil] INFO [2022-01-10 05:39:05,149] jepsen worker 52 - jepsen.util 1132 :ok :read [17 41] INFO [2022-01-10 05:39:05,149] jepsen worker 52 - jepsen.util 1132 :invoke :read [17 nil] INFO [2022-01-10 05:39:05,171] jepsen worker 52 - jepsen.util 1132 :ok :read [17 41] INFO [2022-01-10 05:39:05,172] jepsen worker 52 - jepsen.util 1132 :invoke :read [17 nil] INFO [2022-01-10 05:39:05,194] jepsen worker 52 - jepsen.util 1132 :ok :read [17 41] INFO [2022-01-10 05:39:05,195] jepsen worker 52 - jepsen.util 1132 :invoke :write [17 14] INFO [2022-01-10 05:39:05,392] jepsen worker 68 - jepsen.util 968 :info :write [22 15] :RequestTimeoutException INFO [2022-01-10 05:39:05,392] jepsen worker 68 - jepsen.util 1058 :invoke :write [22 25] INFO [2022-01-10 05:39:05,497] jepsen worker 1 - jepsen.util 991 :info :write [0 7] :RequestTimeoutException INFO [2022-01-10 05:39:05,497] jepsen worker 1 - jepsen.util 1081 :invoke :write [0 25] INFO [2022-01-10 05:39:05,535] jepsen worker 42 - jepsen.util 1032 :ok :read [14 14] INFO [2022-01-10 05:39:05,536] jepsen worker 42 - jepsen.util 1032 :invoke :write [14 21] INFO [2022-01-10 05:39:05,599] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1578947368421053}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1470588235294118}} INFO [2022-01-10 05:39:05,599] jepsen worker nemesis - couchbase.util Rebalance check count: 102 INFO [2022-01-10 05:39:05,599] jepsen worker nemesis - couchbase.util Rebalance stuck count: 77 INFO [2022-01-10 05:39:05,627] jepsen worker 30 - jepsen.util 660 :info :write [10 22] :RequestTimeoutException INFO [2022-01-10 05:39:05,627] jepsen worker 55 - jepsen.util 1045 :info :write [18 28] :RequestTimeoutException INFO [2022-01-10 05:39:05,628] jepsen worker 55 - jepsen.util 1135 :invoke :write [18 37] INFO [2022-01-10 05:39:05,629] jepsen worker 30 - jepsen.util 750 :invoke :write [10 9] INFO [2022-01-10 05:39:05,650] jepsen worker 70 - jepsen.util 1330 :info :write [23 2] :RequestTimeoutException INFO [2022-01-10 05:39:05,650] jepsen worker 70 - jepsen.util 1420 :invoke :write [23 7] INFO [2022-01-10 05:39:05,756] jepsen worker 9 - jepsen.util 1179 :info :write [3 28] :RequestTimeoutException INFO [2022-01-10 05:39:05,757] jepsen worker 9 - jepsen.util 1269 :invoke :read [3 nil] INFO [2022-01-10 05:39:05,806] jepsen worker 48 - jepsen.util 858 :ok :read [16 38] INFO [2022-01-10 05:39:05,807] jepsen worker 48 - jepsen.util 858 :invoke :read [16 nil] INFO [2022-01-10 05:39:05,935] jepsen worker 37 - jepsen.util 1117 :info :write [12 22] :RequestTimeoutException INFO [2022-01-10 05:39:05,936] jepsen worker 37 - jepsen.util 1207 :invoke :read [12 nil] INFO [2022-01-10 05:39:05,951] jepsen worker 48 - jepsen.util 858 :ok :read [16 38] INFO [2022-01-10 05:39:05,952] jepsen worker 48 - jepsen.util 858 :invoke :read [16 nil] INFO [2022-01-10 05:39:06,226] jepsen worker 32 - jepsen.util 1022 :info :write [10 5] :RequestTimeoutException INFO [2022-01-10 05:39:06,227] jepsen worker 32 - jepsen.util 1112 :invoke :read [10 nil] INFO [2022-01-10 05:39:06,242] jepsen worker 46 - jepsen.util 1126 :info :write [15 13] :RequestTimeoutException INFO [2022-01-10 05:39:06,242] jepsen worker 46 - jepsen.util 1216 :invoke :read [15 nil] INFO [2022-01-10 05:39:06,413] jepsen worker 62 - jepsen.util 1232 :ok :read [20 :nil] INFO [2022-01-10 05:39:06,413] jepsen worker 62 - jepsen.util 1232 :invoke :read [20 nil] INFO [2022-01-10 05:39:06,534] jepsen worker 75 - jepsen.util 885 :ok :read [25 :nil] INFO [2022-01-10 05:39:06,534] jepsen worker 75 - jepsen.util 885 :invoke :read [25 nil] INFO [2022-01-10 05:39:06,606] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1578947368421053}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1470588235294118}} INFO [2022-01-10 05:39:06,606] jepsen worker nemesis - couchbase.util Rebalance check count: 103 INFO [2022-01-10 05:39:06,630] jepsen worker nemesis - couchbase.util Rebalance stuck count: 78 INFO [2022-01-10 05:39:06,680] jepsen worker 25 - jepsen.util 1015 :info :write [8 36] :RequestTimeoutException INFO [2022-01-10 05:39:06,680] jepsen worker 25 - jepsen.util 1105 :invoke :read [8 nil] INFO [2022-01-10 05:39:06,694] jepsen worker 48 - jepsen.util 858 :ok :read [16 38] INFO [2022-01-10 05:39:06,695] jepsen worker 48 - jepsen.util 858 :invoke :write [16 15] INFO [2022-01-10 05:39:06,751] jepsen worker 69 - jepsen.util 1059 :info :write [23 42] :RequestTimeoutException INFO [2022-01-10 05:39:06,752] jepsen worker 69 - jepsen.util 1149 :invoke :write [23 0] INFO [2022-01-10 05:39:06,901] jepsen worker 89 - jepsen.util 1079 :info :write [29 13] :RequestTimeoutException INFO [2022-01-10 05:39:06,901] jepsen worker 89 - jepsen.util 1169 :invoke :write [29 12] INFO [2022-01-10 05:39:07,020] jepsen worker 35 - jepsen.util 1115 :ok :read [11 21] INFO [2022-01-10 05:39:07,021] jepsen worker 35 - jepsen.util 1115 :invoke :write [11 19] INFO [2022-01-10 05:39:07,059] jepsen worker 31 - jepsen.util 1381 :ok :read [10 37] INFO [2022-01-10 05:39:07,060] jepsen worker 31 - jepsen.util 1381 :invoke :write [10 29] INFO [2022-01-10 05:39:07,362] jepsen worker 29 - jepsen.util 1019 :info :write [9 19] :RequestTimeoutException INFO [2022-01-10 05:39:07,363] jepsen worker 29 - jepsen.util 1109 :invoke :write [9 41] INFO [2022-01-10 05:39:07,384] jepsen worker 80 - jepsen.util 1070 :ok :read [26 10] INFO [2022-01-10 05:39:07,385] jepsen worker 80 - jepsen.util 1070 :invoke :read [26 nil] INFO [2022-01-10 05:39:07,409] jepsen worker 80 - jepsen.util 1070 :ok :read [26 10] INFO [2022-01-10 05:39:07,410] jepsen worker 80 - jepsen.util 1070 :invoke :write [26 0] INFO [2022-01-10 05:39:07,637] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1578947368421053}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1529411764705882}} INFO [2022-01-10 05:39:07,637] jepsen worker nemesis - couchbase.util Rebalance check count: 104 INFO [2022-01-10 05:39:07,637] jepsen worker nemesis - couchbase.util Rebalance stuck count: 78 INFO [2022-01-10 05:39:07,750] jepsen worker 8 - jepsen.util 908 :info :write [2 10] :RequestTimeoutException INFO [2022-01-10 05:39:07,751] jepsen worker 8 - jepsen.util 998 :invoke :write [2 23] INFO [2022-01-10 05:39:07,816] jepsen worker 3 - jepsen.util 1263 :info :write [1 48] :RequestTimeoutException INFO [2022-01-10 05:39:07,816] jepsen worker 3 - jepsen.util 1353 :invoke :write [1 15] INFO [2022-01-10 05:39:07,840] jepsen worker 19 - jepsen.util 1099 :info :write [6 4] :RequestTimeoutException INFO [2022-01-10 05:39:07,841] jepsen worker 19 - jepsen.util 1189 :invoke :write [6 25] INFO [2022-01-10 05:39:07,981] jepsen worker 14 - jepsen.util 1184 :info :write [4 33] :RequestTimeoutException INFO [2022-01-10 05:39:07,982] jepsen worker 14 - jepsen.util 1274 :invoke :write [4 1] INFO [2022-01-10 05:39:08,011] jepsen worker 24 - jepsen.util 924 :ok :read [8 :nil] INFO [2022-01-10 05:39:08,011] jepsen worker 24 - jepsen.util 924 :invoke :write [8 49] INFO [2022-01-10 05:39:08,024] jepsen worker 63 - jepsen.util 963 :ok :read [21 7] INFO [2022-01-10 05:39:08,025] jepsen worker 63 - jepsen.util 963 :invoke :read [21 nil] INFO [2022-01-10 05:39:08,028] jepsen worker 73 - jepsen.util 1063 :ok :read [24 18] INFO [2022-01-10 05:39:08,029] jepsen worker 73 - jepsen.util 1063 :invoke :write [24 43] INFO [2022-01-10 05:39:08,050] jepsen worker 58 - jepsen.util 1048 :info :write [19 27] :RequestTimeoutException INFO [2022-01-10 05:39:08,051] jepsen worker 58 - jepsen.util 1138 :invoke :write [19 1] INFO [2022-01-10 05:39:08,161] jepsen worker 49 - jepsen.util 859 :info :write [16 14] :RequestTimeoutException INFO [2022-01-10 05:39:08,162] jepsen worker 49 - jepsen.util 949 :invoke :read [16 nil] INFO [2022-01-10 05:39:08,171] jepsen worker 57 - jepsen.util 1137 :info :write [19 10] :RequestTimeoutException INFO [2022-01-10 05:39:08,172] jepsen worker 57 - jepsen.util 1227 :invoke :read [19 nil] INFO [2022-01-10 05:39:08,366] jepsen worker 77 - jepsen.util 1247 :info :write [25 36] :RequestTimeoutException INFO [2022-01-10 05:39:08,367] jepsen worker 77 - jepsen.util 1337 :invoke :write [25 37] INFO [2022-01-10 05:39:08,479] jepsen worker 43 - jepsen.util 1123 :info :write [14 33] :RequestTimeoutException INFO [2022-01-10 05:39:08,480] jepsen worker 43 - jepsen.util 1213 :invoke :write [14 37] INFO [2022-01-10 05:39:08,643] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1578947368421053}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1529411764705882}} INFO [2022-01-10 05:39:08,644] jepsen worker nemesis - couchbase.util Rebalance check count: 105 INFO [2022-01-10 05:39:08,644] jepsen worker nemesis - couchbase.util Rebalance stuck count: 79 INFO [2022-01-10 05:39:08,894] jepsen worker 57 - jepsen.util 1227 :ok :read [19 31] INFO [2022-01-10 05:39:08,895] jepsen worker 57 - jepsen.util 1227 :invoke :read [19 nil] INFO [2022-01-10 05:39:08,962] jepsen worker 49 - jepsen.util 949 :ok :read [16 38] INFO [2022-01-10 05:39:08,963] jepsen worker 49 - jepsen.util 949 :invoke :read [16 nil] INFO [2022-01-10 05:39:09,037] jepsen worker 39 - jepsen.util 1209 :info :write [13 2] :RequestTimeoutException INFO [2022-01-10 05:39:09,037] jepsen worker 39 - jepsen.util 1299 :invoke :write [13 11] INFO [2022-01-10 05:39:09,167] jepsen worker 83 - jepsen.util 1163 :info :write [27 20] :RequestTimeoutException INFO [2022-01-10 05:39:09,167] jepsen worker 83 - jepsen.util 1253 :invoke :read [27 nil] INFO [2022-01-10 05:39:09,251] jepsen worker 16 - jepsen.util 1186 :info :write [5 5] :RequestTimeoutException INFO [2022-01-10 05:39:09,252] jepsen worker 16 - jepsen.util 1276 :invoke :write [5 2] INFO [2022-01-10 05:39:09,346] jepsen worker 88 - jepsen.util 1168 :info :write [29 35] :RequestTimeoutException INFO [2022-01-10 05:39:09,347] jepsen worker 88 - jepsen.util 1258 :invoke :read [29 nil] INFO [2022-01-10 05:39:09,354] jepsen worker 85 - jepsen.util 1165 :info :write [28 45] :RequestTimeoutException INFO [2022-01-10 05:39:09,356] jepsen worker 85 - jepsen.util 1255 :invoke :write [28 14] INFO [2022-01-10 05:39:09,416] jepsen worker 36 - jepsen.util 756 :info :write [12 9] :RequestTimeoutException INFO [2022-01-10 05:39:09,418] jepsen worker 36 - jepsen.util 846 :invoke :read [12 nil] INFO [2022-01-10 05:39:09,436] jepsen worker 0 - jepsen.util 990 :info :write [0 21] :RequestTimeoutException INFO [2022-01-10 05:39:09,437] jepsen worker 0 - jepsen.util 1080 :invoke :write [0 32] INFO [2022-01-10 05:39:09,475] jepsen worker 50 - jepsen.util 1130 :info :write [16 0] :RequestTimeoutException INFO [2022-01-10 05:39:09,476] jepsen worker 50 - jepsen.util 1220 :invoke :write [16 43] INFO [2022-01-10 05:39:09,568] jepsen worker 86 - jepsen.util 806 :ok :read [28 13] INFO [2022-01-10 05:39:09,568] jepsen worker 86 - jepsen.util 806 :invoke :read [28 nil] INFO [2022-01-10 05:39:09,653] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1578947368421053}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1529411764705882}} INFO [2022-01-10 05:39:09,653] jepsen worker nemesis - couchbase.util Rebalance check count: 106 INFO [2022-01-10 05:39:09,653] jepsen worker nemesis - couchbase.util Rebalance stuck count: 80 WARN [2022-01-10 05:39:09,668] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9140933,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13367153,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9064371,"encode_duration_us":17252,"operation_id":"0x20e9","timeout_ms":10000,"last_server_duration_us":9044797,"total_duration_us":10000995},{"operation_name":"upsert","last_dispatch_duration_us":9234592,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15637539,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9575523,"encode_duration_us":22579,"operation_id":"0x21e9","timeout_ms":10000,"last_server_duration_us":9077736,"total_duration_us":10004434}],"total_count":2}} INFO [2022-01-10 05:39:09,719] jepsen worker 86 - jepsen.util 806 :ok :read [28 13] INFO [2022-01-10 05:39:09,719] jepsen worker 86 - jepsen.util 806 :invoke :read [28 nil] INFO [2022-01-10 05:39:09,729] jepsen worker 83 - jepsen.util 1253 :ok :read [27 30] INFO [2022-01-10 05:39:09,729] jepsen worker 83 - jepsen.util 1253 :invoke :read [27 nil] WARN [2022-01-10 05:39:09,769] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":31573,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":6653728,"total_server_duration_us":204193,"encode_duration_us":17687,"operation_id":"0x2272","timeout_ms":10000,"last_server_duration_us":20045,"total_duration_us":10010074},{"operation_name":"upsert","last_dispatch_duration_us":8906908,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10121929,"total_server_duration_us":1082300,"encode_duration_us":16728,"operation_id":"0x2320","timeout_ms":10000,"last_server_duration_us":479354,"total_duration_us":10009464},{"operation_name":"upsert","last_dispatch_duration_us":5868236,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5868236,"total_server_duration_us":118121,"encode_duration_us":23536,"operation_id":"0x21c1","timeout_ms":10000,"last_server_duration_us":118121,"total_duration_us":10005131},{"operation_name":"upsert","last_dispatch_duration_us":9234592,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":15637539,"total_server_duration_us":9575523,"encode_duration_us":22579,"operation_id":"0x21e9","timeout_ms":10000,"last_server_duration_us":9077736,"total_duration_us":10004434},{"operation_name":"upsert","last_dispatch_duration_us":9188874,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9188874,"total_server_duration_us":47306,"encode_duration_us":16493,"operation_id":"0x215d","timeout_ms":10000,"last_server_duration_us":47306,"total_duration_us":10002347},{"operation_name":"upsert","last_dispatch_duration_us":8399883,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10034377,"total_server_duration_us":640381,"encode_duration_us":32661,"operation_id":"0x2323","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10002291},{"operation_name":"upsert","last_dispatch_duration_us":9300093,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9432825,"total_server_duration_us":44022,"encode_duration_us":16597,"operation_id":"0x2189","timeout_ms":10000,"last_server_duration_us":23898,"total_duration_us":10001634},{"operation_name":"get","last_dispatch_duration_us":7462978,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":7462978,"total_server_duration_us":32,"operation_id":"0x2214","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":7463378},{"operation_name":"get","last_dispatch_duration_us":6407503,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":6407503,"total_server_duration_us":32,"operation_id":"0x2244","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":6409552},{"operation_name":"get","last_dispatch_duration_us":6327189,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":6327189,"total_server_duration_us":23,"operation_id":"0x23e1","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":6328284}],"total_count":11}} INFO [2022-01-10 05:39:09,771] jepsen worker 75 - jepsen.util 885 :ok :read [25 :nil] INFO [2022-01-10 05:39:09,772] jepsen worker 75 - jepsen.util 885 :invoke :read [25 nil] INFO [2022-01-10 05:39:09,795] jepsen worker 6 - jepsen.util 996 :info :write [2 49] :RequestTimeoutException INFO [2022-01-10 05:39:09,796] jepsen worker 6 - jepsen.util 1086 :invoke :write [2 8] INFO [2022-01-10 05:39:09,799] jepsen worker 49 - jepsen.util 949 :ok :read [16 38] INFO [2022-01-10 05:39:09,800] jepsen worker 49 - jepsen.util 949 :invoke :write [16 12] INFO [2022-01-10 05:39:09,826] jepsen worker 34 - jepsen.util 1024 :info :write [11 20] :RequestTimeoutException INFO [2022-01-10 05:39:09,827] jepsen worker 34 - jepsen.util 1114 :invoke :write [11 37] INFO [2022-01-10 05:39:09,883] jepsen worker 86 - jepsen.util 806 :ok :read [28 13] INFO [2022-01-10 05:39:09,883] jepsen worker 86 - jepsen.util 806 :invoke :write [28 9] INFO [2022-01-10 05:39:09,892] jepsen worker 83 - jepsen.util 1253 :ok :read [27 30] INFO [2022-01-10 05:39:09,893] jepsen worker 83 - jepsen.util 1253 :invoke :write [27 49] INFO [2022-01-10 05:39:09,929] jepsen worker 45 - jepsen.util 1305 :ok :read [15 23] INFO [2022-01-10 05:39:09,929] jepsen worker 45 - jepsen.util 1305 :invoke :write [15 33] INFO [2022-01-10 05:39:09,959] jepsen worker 20 - jepsen.util 920 :ok :read [6 25] INFO [2022-01-10 05:39:09,960] jepsen worker 20 - jepsen.util 920 :invoke :read [6 nil] INFO [2022-01-10 05:39:10,183] jepsen worker 62 - jepsen.util 1232 :ok :read [20 :nil] INFO [2022-01-10 05:39:10,184] jepsen worker 62 - jepsen.util 1232 :invoke :write [20 49] INFO [2022-01-10 05:39:10,319] jepsen worker 21 - jepsen.util 1371 :ok :read [7 :nil] INFO [2022-01-10 05:39:10,319] jepsen worker 76 - jepsen.util 886 :ok :read [25 :nil] INFO [2022-01-10 05:39:10,321] jepsen worker 21 - jepsen.util 1371 :invoke :read [7 nil] INFO [2022-01-10 05:39:10,321] jepsen worker 76 - jepsen.util 886 :invoke :write [25 18] INFO [2022-01-10 05:39:10,610] jepsen worker 22 - jepsen.util 922 :info :write [7 48] :RequestTimeoutException INFO [2022-01-10 05:39:10,611] jepsen worker 22 - jepsen.util 1012 :invoke :write [7 32] INFO [2022-01-10 05:39:10,632] jepsen worker 63 - jepsen.util 963 :ok :read [21 7] INFO [2022-01-10 05:39:10,633] jepsen worker 63 - jepsen.util 963 :invoke :read [21 nil] INFO [2022-01-10 05:39:10,659] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1578947368421053}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1529411764705882}} INFO [2022-01-10 05:39:10,659] jepsen worker nemesis - couchbase.util Rebalance check count: 107 INFO [2022-01-10 05:39:10,659] jepsen worker nemesis - couchbase.util Rebalance stuck count: 81 INFO [2022-01-10 05:39:10,671] jepsen worker 40 - jepsen.util 850 :info :write [13 40] :RequestTimeoutException INFO [2022-01-10 05:39:10,672] jepsen worker 40 - jepsen.util 940 :invoke :write [13 19] INFO [2022-01-10 05:39:10,672] jepsen worker 67 - jepsen.util 1057 :info :write [22 30] :RequestTimeoutException INFO [2022-01-10 05:39:10,673] jepsen worker 67 - jepsen.util 1147 :invoke :read [22 nil] INFO [2022-01-10 05:39:10,997] jepsen worker 13 - jepsen.util 1093 :info :write [4 47] :SyncWriteAmbiguous INFO [2022-01-10 05:39:10,998] jepsen worker 13 - jepsen.util 1183 :invoke :read [4 nil] INFO [2022-01-10 05:39:11,026] jepsen worker 79 - jepsen.util 1069 :info :write [26 39] :RequestTimeoutException INFO [2022-01-10 05:39:11,027] jepsen worker 79 - jepsen.util 1159 :invoke :read [26 nil] INFO [2022-01-10 05:39:11,130] jepsen worker 9 - jepsen.util 1269 :ok :read [3 2] INFO [2022-01-10 05:39:11,130] jepsen worker 9 - jepsen.util 1269 :invoke :read [3 nil] INFO [2022-01-10 05:39:11,143] jepsen worker 13 - jepsen.util 1183 :ok :read [4 :nil] INFO [2022-01-10 05:39:11,143] jepsen worker 13 - jepsen.util 1183 :invoke :write [4 3] INFO [2022-01-10 05:39:11,189] jepsen worker 9 - jepsen.util 1269 :ok :read [3 2] INFO [2022-01-10 05:39:11,190] jepsen worker 9 - jepsen.util 1269 :invoke :write [3 5] INFO [2022-01-10 05:39:11,195] jepsen worker 59 - jepsen.util 1139 :info :write [19 40] :RequestTimeoutException INFO [2022-01-10 05:39:11,196] jepsen worker 59 - jepsen.util 1229 :invoke :write [19 43] WARN [2022-01-10 05:39:11,204] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16928552,"last_local_id":"EFE9906200000002/00000000598DAEAE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":17458763,"last_local_socket":"172.23.104.79:44342","total_server_duration_us":9359849,"encode_duration_us":16267,"operation_id":"0x2000","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10002173},{"operation_name":"upsert","last_dispatch_duration_us":9146108,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9847293,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9164750,"encode_duration_us":16388,"operation_id":"0x218a","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10006178},{"operation_name":"upsert","last_dispatch_duration_us":9346758,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11075737,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9244492,"encode_duration_us":18357,"operation_id":"0x2321","timeout_ms":10000,"last_server_duration_us":9061792,"total_duration_us":10010454},{"operation_name":"upsert","last_dispatch_duration_us":9210317,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10929171,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9245688,"encode_duration_us":18095,"operation_id":"0x231e","timeout_ms":10000,"last_server_duration_us":9045859,"total_duration_us":10010581}],"total_count":4}} WARN [2022-01-10 05:39:11,272] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9140169,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10172590,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9241311,"encode_duration_us":18027,"operation_id":"0x228e","timeout_ms":10000,"last_server_duration_us":9083054,"total_duration_us":10008478}],"total_count":1}} WARN [2022-01-10 05:39:11,272] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":5558977,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5558977,"total_server_duration_us":120649,"operation_id":"0x2129","timeout_ms":30000,"last_server_duration_us":120649,"total_duration_us":18888508},{"operation_name":"upsert","last_dispatch_duration_us":33072,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":953931,"total_server_duration_us":570663,"encode_duration_us":21956,"operation_id":"0x223d","timeout_ms":10000,"last_server_duration_us":22306,"total_duration_us":10011025},{"operation_name":"upsert","last_dispatch_duration_us":333125,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9589133,"total_server_duration_us":555687,"encode_duration_us":20266,"operation_id":"0x22d5","timeout_ms":10000,"last_server_duration_us":88098,"total_duration_us":10008887},{"operation_name":"upsert","last_dispatch_duration_us":9140169,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":10172590,"total_server_duration_us":9241311,"encode_duration_us":18027,"operation_id":"0x228e","timeout_ms":10000,"last_server_duration_us":9083054,"total_duration_us":10008478},{"operation_name":"upsert","last_dispatch_duration_us":8077051,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":8878371,"total_server_duration_us":208711,"encode_duration_us":19852,"operation_id":"0x21b4","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10008440},{"operation_name":"upsert","last_dispatch_duration_us":618583,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7063627,"total_server_duration_us":1187754,"encode_duration_us":30006,"operation_id":"0x22e0","timeout_ms":10000,"last_server_duration_us":27402,"total_duration_us":10008409},{"operation_name":"upsert","last_dispatch_duration_us":7647578,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11720474,"total_server_duration_us":465123,"encode_duration_us":16764,"operation_id":"0x220d","timeout_ms":10000,"last_server_duration_us":326145,"total_duration_us":10006497},{"operation_name":"upsert","last_dispatch_duration_us":443888,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":3856977,"total_server_duration_us":280569,"encode_duration_us":17025,"operation_id":"0x2352","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10005658},{"operation_name":"upsert","last_dispatch_duration_us":6656531,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":6656531,"total_server_duration_us":109209,"encode_duration_us":21229,"operation_id":"0x21e8","timeout_ms":10000,"last_server_duration_us":109209,"total_duration_us":10003254},{"operation_name":"upsert","last_dispatch_duration_us":434013,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":5872838,"total_server_duration_us":598478,"encode_duration_us":18416,"operation_id":"0x22e1","timeout_ms":10000,"last_server_duration_us":31203,"total_duration_us":10002815}],"total_count":21}} WARN [2022-01-10 05:39:11,303] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17745172,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":20796113,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9321981,"encode_duration_us":19071,"operation_id":"0x2002","timeout_ms":10000,"last_server_duration_us":9072420,"total_duration_us":10002258}],"total_count":1}} WARN [2022-01-10 05:39:11,304] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9551268,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10502675,"total_server_duration_us":9473985,"encode_duration_us":18327,"operation_id":"0x224c","timeout_ms":10000,"last_server_duration_us":9288246,"total_duration_us":10010374},{"operation_name":"upsert","last_dispatch_duration_us":9331698,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10517103,"total_server_duration_us":336939,"encode_duration_us":20483,"operation_id":"0x2263","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10010144},{"operation_name":"upsert","last_dispatch_duration_us":3757993,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7416237,"total_server_duration_us":631182,"encode_duration_us":18780,"operation_id":"0x2200","timeout_ms":10000,"last_server_duration_us":256073,"total_duration_us":10009802},{"operation_name":"upsert","last_dispatch_duration_us":5740088,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10897872,"total_server_duration_us":216567,"encode_duration_us":23705,"operation_id":"0x2248","timeout_ms":10000,"last_server_duration_us":54456,"total_duration_us":10009797},{"operation_name":"upsert","last_dispatch_duration_us":1012841,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10375365,"total_server_duration_us":631651,"encode_duration_us":16806,"operation_id":"0x21b2","timeout_ms":10000,"last_server_duration_us":126118,"total_duration_us":10008815},{"operation_name":"upsert","last_dispatch_duration_us":29768,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":953764,"total_server_duration_us":593078,"encode_duration_us":19747,"operation_id":"0x21e5","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10008780},{"operation_name":"upsert","last_dispatch_duration_us":625283,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":1159544,"total_server_duration_us":185662,"encode_duration_us":16862,"operation_id":"0x227c","timeout_ms":10000,"last_server_duration_us":66168,"total_duration_us":10008736},{"operation_name":"upsert","last_dispatch_duration_us":2033779,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6120888,"total_server_duration_us":928086,"encode_duration_us":16550,"operation_id":"0x21d9","timeout_ms":10000,"last_server_duration_us":191275,"total_duration_us":10008080},{"operation_name":"upsert","last_dispatch_duration_us":239263,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":1128653,"total_server_duration_us":328986,"encode_duration_us":28977,"operation_id":"0x224d","timeout_ms":10000,"last_server_duration_us":92589,"total_duration_us":10008016},{"operation_name":"upsert","last_dispatch_duration_us":1745608,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":6509760,"total_server_duration_us":765794,"encode_duration_us":17555,"operation_id":"0x2317","timeout_ms":10000,"last_server_duration_us":180894,"total_duration_us":10007401}],"total_count":16}} WARN [2022-01-10 05:39:11,305] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10640936,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":10640936,"total_server_duration_us":27,"operation_id":"0x21eb","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":10642759},{"operation_name":"upsert","last_dispatch_duration_us":9210317,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10929171,"total_server_duration_us":9245688,"encode_duration_us":18095,"operation_id":"0x231e","timeout_ms":10000,"last_server_duration_us":9045859,"total_duration_us":10010581},{"operation_name":"upsert","last_dispatch_duration_us":3780647,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10486518,"total_server_duration_us":247043,"encode_duration_us":17840,"operation_id":"0x2345","timeout_ms":10000,"last_server_duration_us":135224,"total_duration_us":10010482},{"operation_name":"upsert","last_dispatch_duration_us":9346758,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11075737,"total_server_duration_us":9244492,"encode_duration_us":18357,"operation_id":"0x2321","timeout_ms":10000,"last_server_duration_us":9061792,"total_duration_us":10010454},{"operation_name":"upsert","last_dispatch_duration_us":1393095,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":4158448,"total_server_duration_us":151139,"encode_duration_us":19377,"operation_id":"0x2215","timeout_ms":10000,"last_server_duration_us":59751,"total_duration_us":10009862},{"operation_name":"upsert","last_dispatch_duration_us":9146108,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9847293,"total_server_duration_us":9164750,"encode_duration_us":16388,"operation_id":"0x218a","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10006178},{"operation_name":"upsert","last_dispatch_duration_us":7892591,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9156444,"total_server_duration_us":277678,"encode_duration_us":19611,"operation_id":"0x2373","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10002739},{"operation_name":"upsert","last_dispatch_duration_us":86243,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":945021,"total_server_duration_us":161411,"encode_duration_us":17986,"operation_id":"0x22d0","timeout_ms":10000,"last_server_duration_us":21813,"total_duration_us":10001683},{"operation_name":"upsert","last_dispatch_duration_us":4168416,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11014726,"total_server_duration_us":496724,"encode_duration_us":16714,"operation_id":"0x22e4","timeout_ms":10000,"last_server_duration_us":266419,"total_duration_us":10001288},{"operation_name":"get","last_dispatch_duration_us":7130169,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7130169,"total_server_duration_us":281050,"operation_id":"0x22e5","timeout_ms":30000,"last_server_duration_us":281050,"total_duration_us":7132598}],"total_count":17}} WARN [2022-01-10 05:39:11,308] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16452532,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":16452532,"total_server_duration_us":18876,"operation_id":"0x215b","timeout_ms":30000,"last_server_duration_us":18876,"total_duration_us":16455060},{"operation_name":"upsert","last_dispatch_duration_us":671365,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8940960,"total_server_duration_us":1784968,"encode_duration_us":16983,"operation_id":"0x22ac","timeout_ms":10000,"last_server_duration_us":23475,"total_duration_us":10008685},{"operation_name":"upsert","encode_duration_us":23398,"operation_id":"0x2221","timeout_ms":10000,"total_duration_us":10008522},{"operation_name":"upsert","last_dispatch_duration_us":167415,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":6472193,"total_server_duration_us":490080,"encode_duration_us":16156,"operation_id":"0x2351","timeout_ms":10000,"last_server_duration_us":26241,"total_duration_us":10008136},{"operation_name":"upsert","last_dispatch_duration_us":7687261,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9642803,"total_server_duration_us":605017,"encode_duration_us":17143,"operation_id":"0x21e7","timeout_ms":10000,"last_server_duration_us":22554,"total_duration_us":10007848},{"operation_name":"upsert","encode_duration_us":18030,"operation_id":"0x227d","timeout_ms":10000,"total_duration_us":10007312},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":22663,"operation_id":"0x21aa","timeout_ms":10000,"total_duration_us":10007110},{"operation_name":"upsert","last_dispatch_duration_us":7753944,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9621705,"total_server_duration_us":1110721,"encode_duration_us":16403,"operation_id":"0x21f3","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10003953},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":20844,"operation_id":"0x221a","timeout_ms":10000,"total_duration_us":10003568},{"operation_name":"upsert","last_dispatch_duration_us":1286009,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9792394,"total_server_duration_us":910278,"encode_duration_us":16232,"operation_id":"0x2247","timeout_ms":10000,"last_server_duration_us":61520,"total_duration_us":10003564}],"total_count":17}} INFO [2022-01-10 05:39:11,366] jepsen worker 82 - jepsen.util 982 :info :write [27 6] :RequestTimeoutException INFO [2022-01-10 05:39:11,367] jepsen worker 82 - jepsen.util 1072 :invoke :read [27 nil] INFO [2022-01-10 05:39:11,376] jepsen worker 66 - jepsen.util 876 :info :write [22 0] :RequestTimeoutException INFO [2022-01-10 05:39:11,376] jepsen worker 66 - jepsen.util 966 :invoke :write [22 10] WARN [2022-01-10 05:39:11,404] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10831203,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10831203,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9077736,"encode_duration_us":20714,"operation_id":"0x2322","timeout_ms":10000,"last_server_duration_us":9077736,"total_duration_us":10002921},{"operation_name":"upsert","last_dispatch_duration_us":9551268,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10502675,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9473985,"encode_duration_us":18327,"operation_id":"0x224c","timeout_ms":10000,"last_server_duration_us":9288246,"total_duration_us":10010374}],"total_count":2}} WARN [2022-01-10 05:39:11,408] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9112138,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12276148,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9554674,"encode_duration_us":16374,"operation_id":"0x2180","timeout_ms":10000,"last_server_duration_us":9090500,"total_duration_us":10007533}],"total_count":1}} WARN [2022-01-10 05:39:11,504] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":411364,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4264232,"total_server_duration_us":452015,"encode_duration_us":17172,"operation_id":"0x234d","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10010407},{"operation_name":"upsert","last_dispatch_duration_us":1400035,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4322772,"total_server_duration_us":599109,"encode_duration_us":16872,"operation_id":"0x22e6","timeout_ms":10000,"last_server_duration_us":124913,"total_duration_us":10010130},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":16547,"operation_id":"0x21d6","timeout_ms":10000,"total_duration_us":10009981},{"operation_name":"upsert","last_dispatch_duration_us":1666768,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10119691,"total_server_duration_us":810004,"encode_duration_us":16241,"operation_id":"0x21ec","timeout_ms":10000,"last_server_duration_us":24154,"total_duration_us":10009333},{"operation_name":"upsert","last_dispatch_duration_us":1064321,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8816637,"total_server_duration_us":272181,"encode_duration_us":16538,"operation_id":"0x22d6","timeout_ms":10000,"last_server_duration_us":25712,"total_duration_us":10009085},{"operation_name":"upsert","last_dispatch_duration_us":1695622,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5958452,"total_server_duration_us":554781,"encode_duration_us":16433,"operation_id":"0x223b","timeout_ms":10000,"last_server_duration_us":386475,"total_duration_us":10008866},{"operation_name":"upsert","encode_duration_us":22180,"operation_id":"0x2250","timeout_ms":10000,"total_duration_us":10008596},{"operation_name":"upsert","last_dispatch_duration_us":1337680,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":6879234,"total_server_duration_us":703847,"encode_duration_us":16679,"operation_id":"0x22d2","timeout_ms":10000,"last_server_duration_us":122688,"total_duration_us":10007260},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":16321,"operation_id":"0x2217","timeout_ms":10000,"total_duration_us":10006166},{"operation_name":"upsert","last_dispatch_duration_us":2111080,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5976900,"total_server_duration_us":816772,"encode_duration_us":16801,"operation_id":"0x2280","timeout_ms":10000,"last_server_duration_us":72589,"total_duration_us":10005857}],"total_count":17}} INFO [2022-01-10 05:39:11,669] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1578947368421053}, :ns_1@172.23.107.227 {:progress 0.08797653958944285}, :ns_1@172.23.105.187 {:progress 0.08797653958944285}, :ns_1@172.23.107.228 {:progress 0.1529411764705882}} INFO [2022-01-10 05:39:11,669] jepsen worker nemesis - couchbase.util Rebalance check count: 108 INFO [2022-01-10 05:39:11,670] jepsen worker nemesis - couchbase.util Rebalance stuck count: 82 INFO [2022-01-10 05:39:11,684] jepsen worker 87 - jepsen.util 1347 :info :write [29 20] :RequestTimeoutException INFO [2022-01-10 05:39:11,685] jepsen worker 87 - jepsen.util 1437 :invoke :read [29 nil] INFO [2022-01-10 05:39:11,732] jepsen worker 75 - jepsen.util 885 :ok :read [25 :nil] INFO [2022-01-10 05:39:11,732] jepsen worker 75 - jepsen.util 885 :invoke :read [25 nil] INFO [2022-01-10 05:39:11,875] jepsen worker 11 - jepsen.util 1001 :info :write [3 30] :RequestTimeoutException INFO [2022-01-10 05:39:11,876] jepsen worker 11 - jepsen.util 1091 :invoke :read [3 nil] INFO [2022-01-10 05:39:12,041] jepsen worker 21 - jepsen.util 1371 :ok :read [7 :nil] INFO [2022-01-10 05:39:12,042] jepsen worker 21 - jepsen.util 1371 :invoke :write [7 47] INFO [2022-01-10 05:39:12,234] jepsen worker 60 - jepsen.util 1320 :info :write [20 20] :RequestTimeoutException INFO [2022-01-10 05:39:12,236] jepsen worker 60 - jepsen.util 1410 :invoke :write [20 19] INFO [2022-01-10 05:39:12,245] jepsen worker 11 - jepsen.util 1091 :ok :read [3 2] INFO [2022-01-10 05:39:12,246] jepsen worker 11 - jepsen.util 1091 :invoke :read [3 nil] INFO [2022-01-10 05:39:12,286] jepsen worker 2 - jepsen.util 1082 :info :write [0 21] :RequestTimeoutException INFO [2022-01-10 05:39:12,286] jepsen worker 2 - jepsen.util 1172 :invoke :read [0 nil] INFO [2022-01-10 05:39:12,336] jepsen worker 84 - jepsen.util 1074 :info :write [28 25] :RequestTimeoutException INFO [2022-01-10 05:39:12,337] jepsen worker 84 - jepsen.util 1164 :invoke :write [28 1] INFO [2022-01-10 05:39:12,417] jepsen worker 11 - jepsen.util 1091 :ok :read [3 2] INFO [2022-01-10 05:39:12,418] jepsen worker 11 - jepsen.util 1091 :invoke :write [3 25] INFO [2022-01-10 05:39:12,500] jepsen worker 54 - jepsen.util 1134 :info :write [18 27] :RequestTimeoutException INFO [2022-01-10 05:39:12,501] jepsen worker 54 - jepsen.util 1224 :invoke :write [18 9] INFO [2022-01-10 05:39:12,610] jepsen worker 81 - jepsen.util 1071 :info :write [27 20] :RequestTimeoutException INFO [2022-01-10 05:39:12,610] jepsen worker 81 - jepsen.util 1161 :invoke :read [27 nil] INFO [2022-01-10 05:39:12,677] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1578947368421053}, :ns_1@172.23.107.227 {:progress 0.09970674486803521}, :ns_1@172.23.105.187 {:progress 0.09970674486803521}, :ns_1@172.23.107.228 {:progress 0.1588235294117647}} INFO [2022-01-10 05:39:12,677] jepsen worker nemesis - couchbase.util Rebalance check count: 109 INFO [2022-01-10 05:39:12,677] jepsen worker nemesis - couchbase.util Rebalance stuck count: 82 INFO [2022-01-10 05:39:12,762] jepsen worker 64 - jepsen.util 1054 :info :write [21 20] :RequestTimeoutException INFO [2022-01-10 05:39:12,763] jepsen worker 64 - jepsen.util 1144 :invoke :write [21 6] INFO [2022-01-10 05:39:12,926] jepsen worker 38 - jepsen.util 1208 :info :write [12 28] :RequestTimeoutException INFO [2022-01-10 05:39:12,927] jepsen worker 38 - jepsen.util 1298 :invoke :write [12 7] INFO [2022-01-10 05:39:12,985] jepsen worker 23 - jepsen.util 1103 :info :write [7 21] :RequestTimeoutException INFO [2022-01-10 05:39:12,985] jepsen worker 23 - jepsen.util 1193 :invoke :read [7 nil] INFO [2022-01-10 05:39:13,172] jepsen worker 51 - jepsen.util 1041 :info :write [17 34] :SyncWriteAmbiguous INFO [2022-01-10 05:39:13,173] jepsen worker 51 - jepsen.util 1131 :invoke :write [17 45] INFO [2022-01-10 05:39:13,180] jepsen worker 53 - jepsen.util 953 :info :write [17 5] :RequestTimeoutException INFO [2022-01-10 05:39:13,181] jepsen worker 53 - jepsen.util 1043 :invoke :read [17 nil] INFO [2022-01-10 05:39:13,216] jepsen worker 53 - jepsen.util 1043 :ok :read [17 41] INFO [2022-01-10 05:39:13,216] jepsen worker 53 - jepsen.util 1043 :invoke :write [17 47] INFO [2022-01-10 05:39:13,226] jepsen worker 56 - jepsen.util 1226 :info :write [18 44] :RequestTimeoutException INFO [2022-01-10 05:39:13,227] jepsen worker 56 - jepsen.util 1316 :invoke :read [18 nil] INFO [2022-01-10 05:39:13,241] jepsen worker 33 - jepsen.util 1113 :info :write [11 29] :RequestTimeoutException INFO [2022-01-10 05:39:13,242] jepsen worker 33 - jepsen.util 1203 :invoke :read [11 nil] INFO [2022-01-10 05:39:13,245] jepsen worker 44 - jepsen.util 1124 :ok :read [14 14] INFO [2022-01-10 05:39:13,246] jepsen worker 44 - jepsen.util 1124 :invoke :write [14 39] INFO [2022-01-10 05:39:13,278] jepsen worker 18 - jepsen.util 918 :ok :read [6 25] INFO [2022-01-10 05:39:13,278] jepsen worker 18 - jepsen.util 918 :invoke :read [6 nil] INFO [2022-01-10 05:39:13,306] jepsen worker 17 - jepsen.util 1007 :info :write [5 32] :RequestTimeoutException INFO [2022-01-10 05:39:13,307] jepsen worker 17 - jepsen.util 1097 :invoke :read [5 nil] INFO [2022-01-10 05:39:13,684] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1578947368421053}, :ns_1@172.23.107.227 {:progress 0.1026392961876833}, :ns_1@172.23.105.187 {:progress 0.1026392961876833}, :ns_1@172.23.107.228 {:progress 0.1588235294117647}} INFO [2022-01-10 05:39:13,684] jepsen worker nemesis - couchbase.util Rebalance check count: 110 INFO [2022-01-10 05:39:13,684] jepsen worker nemesis - couchbase.util Rebalance stuck count: 82 INFO [2022-01-10 05:39:13,720] jepsen worker 15 - jepsen.util 825 :info :write [5 41] :RequestTimeoutException INFO [2022-01-10 05:39:13,721] jepsen worker 15 - jepsen.util 915 :invoke :read [5 nil] INFO [2022-01-10 05:39:13,732] jepsen worker 10 - jepsen.util 1180 :info :write [3 26] :RequestTimeoutException INFO [2022-01-10 05:39:13,733] jepsen worker 10 - jepsen.util 1270 :invoke :write [3 32] INFO [2022-01-10 05:39:13,823] jepsen worker 25 - jepsen.util 1105 :ok :read [8 :nil] INFO [2022-01-10 05:39:13,824] jepsen worker 25 - jepsen.util 1105 :invoke :write [8 25] INFO [2022-01-10 05:39:13,972] jepsen worker 41 - jepsen.util 851 :info :write [13 19] :RequestTimeoutException INFO [2022-01-10 05:39:13,973] jepsen worker 41 - jepsen.util 941 :invoke :read [13 nil] INFO [2022-01-10 05:39:14,006] jepsen worker 20 - jepsen.util 920 :ok :read [6 25] INFO [2022-01-10 05:39:14,007] jepsen worker 20 - jepsen.util 920 :invoke :write [6 2] INFO [2022-01-10 05:39:14,315] jepsen worker 27 - jepsen.util 1017 :info :write [9 45] :RequestTimeoutException INFO [2022-01-10 05:39:14,316] jepsen worker 27 - jepsen.util 1107 :invoke :write [9 49] INFO [2022-01-10 05:39:14,347] jepsen worker 33 - jepsen.util 1203 :ok :read [11 21] INFO [2022-01-10 05:39:14,348] jepsen worker 33 - jepsen.util 1203 :invoke :read [11 nil] INFO [2022-01-10 05:39:14,483] jepsen worker 23 - jepsen.util 1193 :ok :read [7 :nil] INFO [2022-01-10 05:39:14,483] jepsen worker 23 - jepsen.util 1193 :invoke :read [7 nil] INFO [2022-01-10 05:39:14,491] jepsen worker 23 - jepsen.util 1193 :ok :read [7 :nil] INFO [2022-01-10 05:39:14,491] jepsen worker 23 - jepsen.util 1193 :invoke :read [7 nil] INFO [2022-01-10 05:39:14,493] jepsen worker 23 - jepsen.util 1193 :ok :read [7 :nil] INFO [2022-01-10 05:39:14,493] jepsen worker 23 - jepsen.util 1193 :invoke :read [7 nil] INFO [2022-01-10 05:39:14,550] jepsen worker 28 - jepsen.util 1108 :info :write [9 46] :RequestTimeoutException INFO [2022-01-10 05:39:14,551] jepsen worker 28 - jepsen.util 1198 :invoke :read [9 nil] INFO [2022-01-10 05:39:14,622] jepsen worker 23 - jepsen.util 1193 :ok :read [7 :nil] INFO [2022-01-10 05:39:14,623] jepsen worker 23 - jepsen.util 1193 :invoke :read [7 nil] INFO [2022-01-10 05:39:14,670] jepsen worker 12 - jepsen.util 1092 :ok :read [4 :nil] INFO [2022-01-10 05:39:14,671] jepsen worker 12 - jepsen.util 1092 :invoke :read [4 nil] INFO [2022-01-10 05:39:14,691] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1578947368421053}, :ns_1@172.23.107.227 {:progress 0.1026392961876833}, :ns_1@172.23.105.187 {:progress 0.1026392961876833}, :ns_1@172.23.107.228 {:progress 0.1588235294117647}} INFO [2022-01-10 05:39:14,691] jepsen worker nemesis - couchbase.util Rebalance check count: 111 INFO [2022-01-10 05:39:14,691] jepsen worker nemesis - couchbase.util Rebalance stuck count: 83 INFO [2022-01-10 05:39:14,736] jepsen worker 7 - jepsen.util 997 :info :write [2 22] :RequestTimeoutException INFO [2022-01-10 05:39:14,737] jepsen worker 7 - jepsen.util 1087 :invoke :write [2 36] INFO [2022-01-10 05:39:14,790] jepsen worker 4 - jepsen.util 1084 :ok :read [1 41] INFO [2022-01-10 05:39:14,796] jepsen worker 4 - jepsen.util 1084 :invoke :write [1 25] INFO [2022-01-10 05:39:14,814] jepsen worker 32 - jepsen.util 1112 :ok :read [10 37] INFO [2022-01-10 05:39:14,815] jepsen worker 32 - jepsen.util 1112 :invoke :write [10 45] INFO [2022-01-10 05:39:14,827] jepsen worker 12 - jepsen.util 1092 :ok :read [4 :nil] INFO [2022-01-10 05:39:14,828] jepsen worker 12 - jepsen.util 1092 :invoke :read [4 nil] INFO [2022-01-10 05:39:14,833] jepsen worker 17 - jepsen.util 1097 :ok :read [5 42] INFO [2022-01-10 05:39:14,833] jepsen worker 17 - jepsen.util 1097 :invoke :write [5 5] INFO [2022-01-10 05:39:14,842] jepsen worker 88 - jepsen.util 1258 :ok :read [29 33] INFO [2022-01-10 05:39:14,842] jepsen worker 88 - jepsen.util 1258 :invoke :read [29 nil] INFO [2022-01-10 05:39:14,872] jepsen worker 12 - jepsen.util 1092 :ok :read [4 :nil] INFO [2022-01-10 05:39:14,873] jepsen worker 12 - jepsen.util 1092 :invoke :read [4 nil] INFO [2022-01-10 05:39:14,919] jepsen worker 12 - jepsen.util 1092 :ok :read [4 :nil] INFO [2022-01-10 05:39:14,919] jepsen worker 12 - jepsen.util 1092 :invoke :write [4 27] INFO [2022-01-10 05:39:14,933] jepsen worker 88 - jepsen.util 1258 :ok :read [29 33] INFO [2022-01-10 05:39:14,934] jepsen worker 88 - jepsen.util 1258 :invoke :write [29 13] INFO [2022-01-10 05:39:14,991] jepsen worker 74 - jepsen.util 1334 :info :write [24 32] :RequestTimeoutException INFO [2022-01-10 05:39:14,992] jepsen worker 74 - jepsen.util 1424 :invoke :write [24 38] INFO [2022-01-10 05:39:15,226] jepsen worker 52 - jepsen.util 1132 :info :write [17 14] :RequestTimeoutException INFO [2022-01-10 05:39:15,226] jepsen worker 52 - jepsen.util 1222 :invoke :write [17 4] INFO [2022-01-10 05:39:15,416] jepsen worker 68 - jepsen.util 1058 :info :write [22 25] :RequestTimeoutException INFO [2022-01-10 05:39:15,417] jepsen worker 68 - jepsen.util 1148 :invoke :read [22 nil] INFO [2022-01-10 05:39:15,466] jepsen worker 78 - jepsen.util 978 :ok :read [26 10] INFO [2022-01-10 05:39:15,467] jepsen worker 78 - jepsen.util 978 :invoke :write [26 38] INFO [2022-01-10 05:39:15,478] jepsen worker 5 - jepsen.util 1085 :ok :read [1 41] INFO [2022-01-10 05:39:15,478] jepsen worker 5 - jepsen.util 1085 :invoke :read [1 nil] INFO [2022-01-10 05:39:15,520] jepsen worker 1 - jepsen.util 1081 :info :write [0 25] :RequestTimeoutException INFO [2022-01-10 05:39:15,521] jepsen worker 1 - jepsen.util 1171 :invoke :write [0 28] INFO [2022-01-10 05:39:15,566] jepsen worker 42 - jepsen.util 1032 :info :write [14 21] :RequestTimeoutException INFO [2022-01-10 05:39:15,566] jepsen worker 42 - jepsen.util 1122 :invoke :write [14 25] INFO [2022-01-10 05:39:15,580] jepsen worker 23 - jepsen.util 1193 :ok :read [7 :nil] INFO [2022-01-10 05:39:15,580] jepsen worker 23 - jepsen.util 1193 :invoke :read [7 nil] INFO [2022-01-10 05:39:15,601] jepsen worker 36 - jepsen.util 846 :ok :read [12 :nil] INFO [2022-01-10 05:39:15,601] jepsen worker 36 - jepsen.util 846 :invoke :read [12 nil] INFO [2022-01-10 05:39:15,652] jepsen worker 55 - jepsen.util 1135 :info :write [18 37] :RequestTimeoutException INFO [2022-01-10 05:39:15,653] jepsen worker 55 - jepsen.util 1225 :invoke :write [18 48] INFO [2022-01-10 05:39:15,656] jepsen worker 30 - jepsen.util 750 :info :write [10 9] :RequestTimeoutException INFO [2022-01-10 05:39:15,657] jepsen worker 30 - jepsen.util 840 :invoke :read [10 nil] INFO [2022-01-10 05:39:15,671] jepsen worker 70 - jepsen.util 1420 :info :write [23 7] :RequestTimeoutException INFO [2022-01-10 05:39:15,672] jepsen worker 70 - jepsen.util 1510 :invoke :write [23 3] INFO [2022-01-10 05:39:15,693] jepsen worker 5 - jepsen.util 1085 :ok :read [1 41] INFO [2022-01-10 05:39:15,694] jepsen worker 5 - jepsen.util 1085 :invoke :read [1 nil] INFO [2022-01-10 05:39:15,698] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1578947368421053}, :ns_1@172.23.107.227 {:progress 0.1026392961876833}, :ns_1@172.23.105.187 {:progress 0.1026392961876833}, :ns_1@172.23.107.228 {:progress 0.1588235294117647}} INFO [2022-01-10 05:39:15,698] jepsen worker nemesis - couchbase.util Rebalance check count: 112 INFO [2022-01-10 05:39:15,698] jepsen worker nemesis - couchbase.util Rebalance stuck count: 84 INFO [2022-01-10 05:39:15,720] jepsen worker 36 - jepsen.util 846 :ok :read [12 :nil] INFO [2022-01-10 05:39:15,721] jepsen worker 36 - jepsen.util 846 :invoke :read [12 nil] INFO [2022-01-10 05:39:15,825] jepsen worker 5 - jepsen.util 1085 :ok :read [1 41] INFO [2022-01-10 05:39:15,826] jepsen worker 5 - jepsen.util 1085 :invoke :write [1 10] INFO [2022-01-10 05:39:15,893] jepsen worker 36 - jepsen.util 846 :ok :read [12 :nil] INFO [2022-01-10 05:39:15,893] jepsen worker 36 - jepsen.util 846 :invoke :read [12 nil] INFO [2022-01-10 05:39:15,980] jepsen worker 30 - jepsen.util 840 :ok :read [10 37] INFO [2022-01-10 05:39:15,981] jepsen worker 30 - jepsen.util 840 :invoke :write [10 43] INFO [2022-01-10 05:39:16,006] jepsen worker 36 - jepsen.util 846 :ok :read [12 :nil] INFO [2022-01-10 05:39:16,006] jepsen worker 36 - jepsen.util 846 :invoke :write [12 24] INFO [2022-01-10 05:39:16,334] jepsen worker 78 - jepsen.util 978 :fail :write [26 38] :DurabilityImpossible INFO [2022-01-10 05:39:16,335] jepsen worker 78 - jepsen.util 978 :invoke :write [26 17] INFO [2022-01-10 05:39:16,541] jepsen worker 78 - jepsen.util 978 :fail :write [26 17] :DurabilityImpossible INFO [2022-01-10 05:39:16,542] jepsen worker 78 - jepsen.util 978 :invoke :read [26 nil] INFO [2022-01-10 05:39:16,640] jepsen worker 78 - jepsen.util 978 :ok :read [26 10] INFO [2022-01-10 05:39:16,641] jepsen worker 78 - jepsen.util 978 :invoke :write [26 8] INFO [2022-01-10 05:39:16,704] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1637426900584795}, :ns_1@172.23.107.227 {:progress 0.1055718475073314}, :ns_1@172.23.105.187 {:progress 0.1055718475073314}, :ns_1@172.23.107.228 {:progress 0.1764705882352942}} INFO [2022-01-10 05:39:16,704] jepsen worker nemesis - couchbase.util Rebalance check count: 113 INFO [2022-01-10 05:39:16,717] jepsen worker nemesis - couchbase.util Rebalance stuck count: 84 INFO [2022-01-10 05:39:16,721] jepsen worker 48 - jepsen.util 858 :info :write [16 15] :RequestTimeoutException INFO [2022-01-10 05:39:16,722] jepsen worker 48 - jepsen.util 948 :invoke :write [16 6] INFO [2022-01-10 05:39:16,771] jepsen worker 69 - jepsen.util 1149 :info :write [23 0] :RequestTimeoutException INFO [2022-01-10 05:39:16,772] jepsen worker 69 - jepsen.util 1239 :invoke :read [23 nil] INFO [2022-01-10 05:39:16,934] jepsen worker 89 - jepsen.util 1169 :info :write [29 12] :RequestTimeoutException INFO [2022-01-10 05:39:16,935] jepsen worker 89 - jepsen.util 1259 :invoke :write [29 10] INFO [2022-01-10 05:39:17,004] jepsen worker 78 - jepsen.util 978 :fail :write [26 8] :DurabilityImpossible INFO [2022-01-10 05:39:17,005] jepsen worker 78 - jepsen.util 978 :invoke :read [26 nil] INFO [2022-01-10 05:39:17,042] jepsen worker 35 - jepsen.util 1115 :info :write [11 19] :RequestTimeoutException INFO [2022-01-10 05:39:17,043] jepsen worker 35 - jepsen.util 1205 :invoke :read [11 nil] INFO [2022-01-10 05:39:17,082] jepsen worker 31 - jepsen.util 1381 :info :write [10 29] :RequestTimeoutException INFO [2022-01-10 05:39:17,083] jepsen worker 31 - jepsen.util 1471 :invoke :read [10 nil] INFO [2022-01-10 05:39:17,197] jepsen worker 31 - jepsen.util 1471 :ok :read [10 37] INFO [2022-01-10 05:39:17,198] jepsen worker 31 - jepsen.util 1471 :invoke :read [10 nil] INFO [2022-01-10 05:39:17,198] jepsen worker 78 - jepsen.util 978 :ok :read [26 10] INFO [2022-01-10 05:39:17,199] jepsen worker 78 - jepsen.util 978 :invoke :read [26 nil] INFO [2022-01-10 05:39:17,284] jepsen worker 81 - jepsen.util 1161 :ok :read [27 30] INFO [2022-01-10 05:39:17,285] jepsen worker 81 - jepsen.util 1161 :invoke :read [27 nil] INFO [2022-01-10 05:39:17,289] jepsen worker 67 - jepsen.util 1147 :ok :read [22 41] INFO [2022-01-10 05:39:17,290] jepsen worker 67 - jepsen.util 1147 :invoke :write [22 8] INFO [2022-01-10 05:39:17,298] jepsen worker 75 - jepsen.util 885 :ok :read [25 :nil] INFO [2022-01-10 05:39:17,298] jepsen worker 75 - jepsen.util 885 :invoke :write [25 33] INFO [2022-01-10 05:39:17,396] jepsen worker 29 - jepsen.util 1109 :info :write [9 41] :RequestTimeoutException INFO [2022-01-10 05:39:17,399] jepsen worker 29 - jepsen.util 1199 :invoke :write [9 27] INFO [2022-01-10 05:39:17,432] jepsen worker 80 - jepsen.util 1070 :info :write [26 0] :RequestTimeoutException INFO [2022-01-10 05:39:17,432] jepsen worker 80 - jepsen.util 1160 :invoke :read [26 nil] INFO [2022-01-10 05:39:17,452] jepsen worker 78 - jepsen.util 978 :ok :read [26 10] INFO [2022-01-10 05:39:17,452] jepsen worker 78 - jepsen.util 978 :invoke :read [26 nil] INFO [2022-01-10 05:39:17,479] jepsen worker 13 - jepsen.util 1183 :info :write [4 3] :SyncWriteAmbiguous INFO [2022-01-10 05:39:17,481] jepsen worker 13 - jepsen.util 1273 :invoke :read [4 nil] INFO [2022-01-10 05:39:17,512] jepsen worker 31 - jepsen.util 1471 :ok :read [10 37] INFO [2022-01-10 05:39:17,513] jepsen worker 31 - jepsen.util 1471 :invoke :read [10 nil] INFO [2022-01-10 05:39:17,515] jepsen worker 81 - jepsen.util 1161 :ok :read [27 30] INFO [2022-01-10 05:39:17,516] jepsen worker 81 - jepsen.util 1161 :invoke :write [27 18] INFO [2022-01-10 05:39:17,519] jepsen worker 82 - jepsen.util 1072 :ok :read [27 30] INFO [2022-01-10 05:39:17,520] jepsen worker 82 - jepsen.util 1072 :invoke :write [27 35] INFO [2022-01-10 05:39:17,548] jepsen worker 80 - jepsen.util 1160 :ok :read [26 10] INFO [2022-01-10 05:39:17,548] jepsen worker 80 - jepsen.util 1160 :invoke :read [26 nil] INFO [2022-01-10 05:39:17,557] jepsen worker 78 - jepsen.util 978 :ok :read [26 10] INFO [2022-01-10 05:39:17,557] jepsen worker 78 - jepsen.util 978 :invoke :read [26 nil] INFO [2022-01-10 05:39:17,643] jepsen worker 31 - jepsen.util 1471 :ok :read [10 37] INFO [2022-01-10 05:39:17,643] jepsen worker 31 - jepsen.util 1471 :invoke :read [10 nil] INFO [2022-01-10 05:39:17,670] jepsen worker 80 - jepsen.util 1160 :ok :read [26 10] INFO [2022-01-10 05:39:17,671] jepsen worker 80 - jepsen.util 1160 :invoke :read [26 nil] INFO [2022-01-10 05:39:17,682] jepsen worker 31 - jepsen.util 1471 :ok :read [10 37] INFO [2022-01-10 05:39:17,683] jepsen worker 31 - jepsen.util 1471 :invoke :read [10 nil] INFO [2022-01-10 05:39:17,683] jepsen worker 78 - jepsen.util 978 :ok :read [26 10] INFO [2022-01-10 05:39:17,684] jepsen worker 78 - jepsen.util 978 :invoke :read [26 nil] INFO [2022-01-10 05:39:17,726] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1637426900584795}, :ns_1@172.23.107.227 {:progress 0.1055718475073314}, :ns_1@172.23.105.187 {:progress 0.1055718475073314}, :ns_1@172.23.107.228 {:progress 0.1764705882352942}} INFO [2022-01-10 05:39:17,726] jepsen worker nemesis - couchbase.util Rebalance check count: 114 INFO [2022-01-10 05:39:17,726] jepsen worker nemesis - couchbase.util Rebalance stuck count: 85 INFO [2022-01-10 05:39:17,780] jepsen worker 8 - jepsen.util 998 :info :write [2 23] :RequestTimeoutException INFO [2022-01-10 05:39:17,780] jepsen worker 8 - jepsen.util 1088 :invoke :write [2 34] INFO [2022-01-10 05:39:17,798] jepsen worker 31 - jepsen.util 1471 :ok :read [10 37] INFO [2022-01-10 05:39:17,799] jepsen worker 31 - jepsen.util 1471 :invoke :write [10 10] INFO [2022-01-10 05:39:17,821] jepsen worker 80 - jepsen.util 1160 :ok :read [26 10] INFO [2022-01-10 05:39:17,823] jepsen worker 80 - jepsen.util 1160 :invoke :read [26 nil] INFO [2022-01-10 05:39:17,834] jepsen worker 78 - jepsen.util 978 :ok :read [26 10] INFO [2022-01-10 05:39:17,835] jepsen worker 78 - jepsen.util 978 :invoke :write [26 16] INFO [2022-01-10 05:39:17,842] jepsen worker 3 - jepsen.util 1353 :info :write [1 15] :RequestTimeoutException INFO [2022-01-10 05:39:17,843] jepsen worker 3 - jepsen.util 1443 :invoke :write [1 24] INFO [2022-01-10 05:39:17,866] jepsen worker 19 - jepsen.util 1189 :info :write [6 25] :RequestTimeoutException INFO [2022-01-10 05:39:17,867] jepsen worker 19 - jepsen.util 1279 :invoke :read [6 nil] INFO [2022-01-10 05:39:18,001] jepsen worker 14 - jepsen.util 1274 :info :write [4 1] :RequestTimeoutException INFO [2022-01-10 05:39:18,002] jepsen worker 14 - jepsen.util 1364 :invoke :write [4 34] INFO [2022-01-10 05:39:18,041] jepsen worker 24 - jepsen.util 924 :info :write [8 49] :RequestTimeoutException INFO [2022-01-10 05:39:18,042] jepsen worker 24 - jepsen.util 1014 :invoke :write [8 34] INFO [2022-01-10 05:39:18,056] jepsen worker 73 - jepsen.util 1063 :info :write [24 43] :RequestTimeoutException INFO [2022-01-10 05:39:18,057] jepsen worker 73 - jepsen.util 1153 :invoke :write [24 7] INFO [2022-01-10 05:39:18,075] jepsen worker 58 - jepsen.util 1138 :info :write [19 1] :RequestTimeoutException INFO [2022-01-10 05:39:18,075] jepsen worker 58 - jepsen.util 1228 :invoke :read [19 nil] INFO [2022-01-10 05:39:18,230] jepsen worker 63 - jepsen.util 963 :ok :read [21 7] INFO [2022-01-10 05:39:18,231] jepsen worker 63 - jepsen.util 963 :invoke :write [21 47] INFO [2022-01-10 05:39:18,233] jepsen worker 80 - jepsen.util 1160 :ok :read [26 10] INFO [2022-01-10 05:39:18,234] jepsen worker 80 - jepsen.util 1160 :invoke :write [26 37] INFO [2022-01-10 05:39:18,287] jepsen worker 78 - jepsen.util 978 :fail :write [26 16] :DurabilityImpossible INFO [2022-01-10 05:39:18,287] jepsen worker 78 - jepsen.util 978 :invoke :read [26 nil] INFO [2022-01-10 05:39:18,383] jepsen worker 80 - jepsen.util 1160 :fail :write [26 37] :DurabilityImpossible INFO [2022-01-10 05:39:18,385] jepsen worker 80 - jepsen.util 1160 :invoke :write [26 2] INFO [2022-01-10 05:39:18,391] jepsen worker 77 - jepsen.util 1337 :info :write [25 37] :RequestTimeoutException INFO [2022-01-10 05:39:18,392] jepsen worker 77 - jepsen.util 1427 :invoke :write [25 47] INFO [2022-01-10 05:39:18,399] jepsen worker 57 - jepsen.util 1227 :ok :read [19 31] INFO [2022-01-10 05:39:18,400] jepsen worker 57 - jepsen.util 1227 :invoke :write [19 29] INFO [2022-01-10 05:39:18,403] jepsen worker 78 - jepsen.util 978 :ok :read [26 10] INFO [2022-01-10 05:39:18,403] jepsen worker 78 - jepsen.util 978 :invoke :write [26 1] INFO [2022-01-10 05:39:18,505] jepsen worker 2 - jepsen.util 1172 :ok :read [0 46] INFO [2022-01-10 05:39:18,505] jepsen worker 2 - jepsen.util 1172 :invoke :read [0 nil] INFO [2022-01-10 05:39:18,506] jepsen worker 43 - jepsen.util 1213 :info :write [14 37] :RequestTimeoutException INFO [2022-01-10 05:39:18,507] jepsen worker 43 - jepsen.util 1303 :invoke :read [14 nil] INFO [2022-01-10 05:39:18,512] jepsen worker 80 - jepsen.util 1160 :fail :write [26 2] :DurabilityImpossible INFO [2022-01-10 05:39:18,513] jepsen worker 80 - jepsen.util 1160 :invoke :read [26 nil] INFO [2022-01-10 05:39:18,515] jepsen worker 26 - jepsen.util 1106 :ok :read [8 :nil] INFO [2022-01-10 05:39:18,515] jepsen worker 26 - jepsen.util 1106 :invoke :read [8 nil] INFO [2022-01-10 05:39:18,520] jepsen worker 15 - jepsen.util 915 :ok :read [5 42] INFO [2022-01-10 05:39:18,520] jepsen worker 15 - jepsen.util 915 :invoke :write [5 49] INFO [2022-01-10 05:39:18,548] jepsen worker 78 - jepsen.util 978 :fail :write [26 1] :DurabilityImpossible INFO [2022-01-10 05:39:18,549] jepsen worker 78 - jepsen.util 978 :invoke :write [26 46] INFO [2022-01-10 05:39:18,573] jepsen worker 80 - jepsen.util 1160 :ok :read [26 10] INFO [2022-01-10 05:39:18,574] jepsen worker 80 - jepsen.util 1160 :invoke :read [26 nil] INFO [2022-01-10 05:39:18,636] jepsen worker 2 - jepsen.util 1172 :ok :read [0 46] INFO [2022-01-10 05:39:18,636] jepsen worker 2 - jepsen.util 1172 :invoke :read [0 nil] INFO [2022-01-10 05:39:18,654] jepsen worker 78 - jepsen.util 978 :fail :write [26 46] :DurabilityImpossible INFO [2022-01-10 05:39:18,655] jepsen worker 78 - jepsen.util 978 :invoke :read [26 nil] INFO [2022-01-10 05:39:18,675] jepsen worker 80 - jepsen.util 1160 :ok :read [26 10] INFO [2022-01-10 05:39:18,675] jepsen worker 80 - jepsen.util 1160 :invoke :write [26 8] INFO [2022-01-10 05:39:18,695] jepsen worker 78 - jepsen.util 978 :ok :read [26 10] INFO [2022-01-10 05:39:18,695] jepsen worker 78 - jepsen.util 978 :invoke :write [26 7] INFO [2022-01-10 05:39:18,733] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1637426900584795}, :ns_1@172.23.107.227 {:progress 0.1055718475073314}, :ns_1@172.23.105.187 {:progress 0.1055718475073314}, :ns_1@172.23.107.228 {:progress 0.1764705882352942}} INFO [2022-01-10 05:39:18,734] jepsen worker nemesis - couchbase.util Rebalance check count: 115 INFO [2022-01-10 05:39:18,734] jepsen worker nemesis - couchbase.util Rebalance stuck count: 86 INFO [2022-01-10 05:39:18,772] jepsen worker 80 - jepsen.util 1160 :fail :write [26 8] :DurabilityImpossible INFO [2022-01-10 05:39:18,773] jepsen worker 80 - jepsen.util 1160 :invoke :read [26 nil] INFO [2022-01-10 05:39:18,816] jepsen worker 78 - jepsen.util 978 :fail :write [26 7] :DurabilityImpossible INFO [2022-01-10 05:39:18,818] jepsen worker 12 - jepsen.util 1092 :fail :write [4 27] :DurabilityImpossible INFO [2022-01-10 05:39:18,819] jepsen worker 12 - jepsen.util 1092 :invoke :read [4 nil] INFO [2022-01-10 05:39:18,819] jepsen worker 2 - jepsen.util 1172 :ok :read [0 46] INFO [2022-01-10 05:39:18,820] jepsen worker 78 - jepsen.util 978 :invoke :write [26 49] INFO [2022-01-10 05:39:18,820] jepsen worker 2 - jepsen.util 1172 :invoke :write [0 25] INFO [2022-01-10 05:39:18,840] jepsen worker 80 - jepsen.util 1160 :ok :read [26 10] INFO [2022-01-10 05:39:18,841] jepsen worker 80 - jepsen.util 1160 :invoke :write [26 3] INFO [2022-01-10 05:39:18,940] jepsen worker 61 - jepsen.util 1231 :ok :read [20 :nil] INFO [2022-01-10 05:39:18,941] jepsen worker 61 - jepsen.util 1231 :invoke :read [20 nil] INFO [2022-01-10 05:39:18,957] jepsen worker 65 - jepsen.util 1145 :ok :read [21 7] INFO [2022-01-10 05:39:18,957] jepsen worker 65 - jepsen.util 1145 :invoke :read [21 nil] INFO [2022-01-10 05:39:18,969] jepsen worker 46 - jepsen.util 1216 :ok :read [15 23] INFO [2022-01-10 05:39:18,970] jepsen worker 46 - jepsen.util 1216 :invoke :read [15 nil] INFO [2022-01-10 05:39:18,987] jepsen worker 47 - jepsen.util 1127 :ok :read [15 23] INFO [2022-01-10 05:39:18,988] jepsen worker 47 - jepsen.util 1127 :invoke :write [15 36] INFO [2022-01-10 05:39:19,061] jepsen worker 39 - jepsen.util 1299 :info :write [13 11] :RequestTimeoutException INFO [2022-01-10 05:39:19,062] jepsen worker 39 - jepsen.util 1389 :invoke :read [13 nil] INFO [2022-01-10 05:39:19,276] jepsen worker 16 - jepsen.util 1276 :info :write [5 2] :RequestTimeoutException INFO [2022-01-10 05:39:19,276] jepsen worker 16 - jepsen.util 1366 :invoke :write [5 42] INFO [2022-01-10 05:39:19,382] jepsen worker 85 - jepsen.util 1255 :info :write [28 14] :RequestTimeoutException INFO [2022-01-10 05:39:19,382] jepsen worker 85 - jepsen.util 1345 :invoke :read [28 nil] INFO [2022-01-10 05:39:19,462] jepsen worker 0 - jepsen.util 1080 :info :write [0 32] :RequestTimeoutException INFO [2022-01-10 05:39:19,462] jepsen worker 0 - jepsen.util 1170 :invoke :write [0 0] INFO [2022-01-10 05:39:19,504] jepsen worker 50 - jepsen.util 1220 :info :write [16 43] :RequestTimeoutException INFO [2022-01-10 05:39:19,505] jepsen worker 50 - jepsen.util 1310 :invoke :write [16 8] INFO [2022-01-10 05:39:19,741] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1695906432748538}, :ns_1@172.23.107.227 {:progress 0.1114369501466276}, :ns_1@172.23.105.187 {:progress 0.1114369501466276}, :ns_1@172.23.107.228 {:progress 0.1823529411764706}} INFO [2022-01-10 05:39:19,741] jepsen worker nemesis - couchbase.util Rebalance check count: 116 INFO [2022-01-10 05:39:19,741] jepsen worker nemesis - couchbase.util Rebalance stuck count: 86 WARN [2022-01-10 05:39:19,782] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":2773928,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2773928,"total_server_duration_us":43,"operation_id":"0x22c9","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":12525587},{"operation_name":"get","last_dispatch_duration_us":2723609,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2723609,"total_server_duration_us":19,"operation_id":"0x22e2","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":11982711},{"operation_name":"upsert","last_dispatch_duration_us":133344,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8892626,"total_server_duration_us":255782,"encode_duration_us":16175,"operation_id":"0x2382","timeout_ms":10000,"last_server_duration_us":32065,"total_duration_us":10009954},{"operation_name":"upsert","last_dispatch_duration_us":1231255,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9961190,"total_server_duration_us":501783,"encode_duration_us":17680,"operation_id":"0x240e","timeout_ms":10000,"last_server_duration_us":68946,"total_duration_us":10009368},{"operation_name":"upsert","last_dispatch_duration_us":1256854,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4739559,"total_server_duration_us":1588403,"encode_duration_us":17767,"operation_id":"0x24fb","timeout_ms":10000,"last_server_duration_us":1141753,"total_duration_us":10008681},{"operation_name":"upsert","last_dispatch_duration_us":41887,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":959350,"total_server_duration_us":583668,"encode_duration_us":22171,"operation_id":"0x243e","timeout_ms":10000,"last_server_duration_us":29601,"total_duration_us":10007558},{"operation_name":"upsert","last_dispatch_duration_us":1386634,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":6375579,"total_server_duration_us":559575,"encode_duration_us":16597,"operation_id":"0x24c3","timeout_ms":10000,"last_server_duration_us":158969,"total_duration_us":10006743},{"operation_name":"upsert","last_dispatch_duration_us":1861876,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5802048,"total_server_duration_us":837331,"encode_duration_us":17288,"operation_id":"0x23af","timeout_ms":10000,"last_server_duration_us":698892,"total_duration_us":10005103},{"operation_name":"upsert","last_dispatch_duration_us":2967672,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10222182,"total_server_duration_us":345158,"encode_duration_us":16665,"operation_id":"0x233e","timeout_ms":10000,"last_server_duration_us":153677,"total_duration_us":10004971},{"operation_name":"upsert","last_dispatch_duration_us":118646,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":5466047,"total_server_duration_us":372844,"encode_duration_us":29082,"operation_id":"0x2492","timeout_ms":10000,"last_server_duration_us":56157,"total_duration_us":10002860}],"total_count":18}} INFO [2022-01-10 05:39:19,801] jepsen worker 37 - jepsen.util 1207 :ok :read [12 :nil] INFO [2022-01-10 05:39:19,803] jepsen worker 37 - jepsen.util 1207 :invoke :write [12 39] INFO [2022-01-10 05:39:19,820] jepsen worker 71 - jepsen.util 1241 :ok :read [23 26] INFO [2022-01-10 05:39:19,820] jepsen worker 71 - jepsen.util 1241 :invoke :write [23 14] INFO [2022-01-10 05:39:19,826] jepsen worker 6 - jepsen.util 1086 :info :write [2 8] :RequestTimeoutException INFO [2022-01-10 05:39:19,826] jepsen worker 49 - jepsen.util 949 :info :write [16 12] :RequestTimeoutException INFO [2022-01-10 05:39:19,827] jepsen worker 6 - jepsen.util 1176 :invoke :write [2 18] INFO [2022-01-10 05:39:19,827] jepsen worker 49 - jepsen.util 1039 :invoke :write [16 1] INFO [2022-01-10 05:39:19,836] jepsen worker 79 - jepsen.util 1159 :ok :read [26 10] INFO [2022-01-10 05:39:19,836] jepsen worker 79 - jepsen.util 1159 :invoke :read [26 nil] INFO [2022-01-10 05:39:19,849] jepsen worker 34 - jepsen.util 1114 :info :write [11 37] :RequestTimeoutException INFO [2022-01-10 05:39:19,851] jepsen worker 34 - jepsen.util 1204 :invoke :write [11 29] INFO [2022-01-10 05:39:19,900] jepsen worker 72 - jepsen.util 1242 :ok :read [24 18] INFO [2022-01-10 05:39:19,901] jepsen worker 72 - jepsen.util 1242 :invoke :write [24 45] INFO [2022-01-10 05:39:19,904] jepsen worker 86 - jepsen.util 806 :info :write [28 9] :RequestTimeoutException INFO [2022-01-10 05:39:19,905] jepsen worker 86 - jepsen.util 896 :invoke :read [28 nil] INFO [2022-01-10 05:39:19,915] jepsen worker 83 - jepsen.util 1253 :info :write [27 49] :RequestTimeoutException INFO [2022-01-10 05:39:19,916] jepsen worker 83 - jepsen.util 1343 :invoke :write [27 48] INFO [2022-01-10 05:39:19,951] jepsen worker 45 - jepsen.util 1305 :info :write [15 33] :RequestTimeoutException INFO [2022-01-10 05:39:19,952] jepsen worker 45 - jepsen.util 1395 :invoke :read [15 nil] INFO [2022-01-10 05:39:20,211] jepsen worker 62 - jepsen.util 1232 :info :write [20 49] :RequestTimeoutException INFO [2022-01-10 05:39:20,211] jepsen worker 62 - jepsen.util 1322 :invoke :read [20 nil] INFO [2022-01-10 05:39:20,345] jepsen worker 76 - jepsen.util 886 :info :write [25 18] :RequestTimeoutException INFO [2022-01-10 05:39:20,345] jepsen worker 76 - jepsen.util 976 :invoke :read [25 nil] INFO [2022-01-10 05:39:20,584] jepsen worker 26 - jepsen.util 1106 :ok :read [8 :nil] INFO [2022-01-10 05:39:20,585] jepsen worker 61 - jepsen.util 1231 :ok :read [20 :nil] INFO [2022-01-10 05:39:20,585] jepsen worker 26 - jepsen.util 1106 :invoke :read [8 nil] INFO [2022-01-10 05:39:20,585] jepsen worker 61 - jepsen.util 1231 :invoke :read [20 nil] INFO [2022-01-10 05:39:20,624] jepsen worker 65 - jepsen.util 1145 :ok :read [21 7] INFO [2022-01-10 05:39:20,625] jepsen worker 65 - jepsen.util 1145 :invoke :write [21 45] INFO [2022-01-10 05:39:20,632] jepsen worker 22 - jepsen.util 1012 :info :write [7 32] :RequestTimeoutException INFO [2022-01-10 05:39:20,633] jepsen worker 22 - jepsen.util 1102 :invoke :read [7 nil] INFO [2022-01-10 05:39:20,697] jepsen worker 40 - jepsen.util 940 :info :write [13 19] :RequestTimeoutException INFO [2022-01-10 05:39:20,697] jepsen worker 40 - jepsen.util 1030 :invoke :read [13 nil] INFO [2022-01-10 05:39:20,756] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1695906432748538}, :ns_1@172.23.107.227 {:progress 0.1114369501466276}, :ns_1@172.23.105.187 {:progress 0.1114369501466276}, :ns_1@172.23.107.228 {:progress 0.1823529411764706}} INFO [2022-01-10 05:39:20,756] jepsen worker nemesis - couchbase.util Rebalance check count: 117 INFO [2022-01-10 05:39:20,756] jepsen worker nemesis - couchbase.util Rebalance stuck count: 87 INFO [2022-01-10 05:39:20,859] jepsen worker 39 - jepsen.util 1389 :ok :read [13 15] INFO [2022-01-10 05:39:20,860] jepsen worker 85 - jepsen.util 1345 :ok :read [28 13] INFO [2022-01-10 05:39:20,860] jepsen worker 39 - jepsen.util 1389 :invoke :read [13 nil] INFO [2022-01-10 05:39:20,860] jepsen worker 85 - jepsen.util 1345 :invoke :read [28 nil] INFO [2022-01-10 05:39:20,964] jepsen worker 46 - jepsen.util 1216 :ok :read [15 23] INFO [2022-01-10 05:39:20,965] jepsen worker 46 - jepsen.util 1216 :invoke :write [15 8] INFO [2022-01-10 05:39:21,215] jepsen worker 9 - jepsen.util 1269 :info :write [3 5] :RequestTimeoutException INFO [2022-01-10 05:39:21,216] jepsen worker 9 - jepsen.util 1359 :invoke :write [3 15] INFO [2022-01-10 05:39:21,220] jepsen worker 59 - jepsen.util 1229 :info :write [19 43] :RequestTimeoutException INFO [2022-01-10 05:39:21,220] jepsen worker 59 - jepsen.util 1319 :invoke :read [19 nil] WARN [2022-01-10 05:39:21,222] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16877734,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17822756,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9199839,"encode_duration_us":17986,"operation_id":"0x22d0","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":10001683}],"total_count":1}} WARN [2022-01-10 05:39:21,292] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9547602,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10124560,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9157984,"encode_duration_us":17231,"operation_id":"0x2485","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10001364},{"operation_name":"upsert","last_dispatch_duration_us":9168134,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16231762,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10225120,"encode_duration_us":30006,"operation_id":"0x22e0","timeout_ms":10000,"last_server_duration_us":9037366,"total_duration_us":10008409}],"total_count":2}} WARN [2022-01-10 05:39:21,292] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":5998654,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7688477,"total_server_duration_us":780149,"encode_duration_us":19626,"operation_id":"0x24c1","timeout_ms":10000,"last_server_duration_us":105025,"total_duration_us":10009447},{"operation_name":"upsert","last_dispatch_duration_us":6688876,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10776165,"total_server_duration_us":322339,"encode_duration_us":16896,"operation_id":"0x2431","timeout_ms":10000,"last_server_duration_us":118961,"total_duration_us":10008102},{"operation_name":"upsert","last_dispatch_duration_us":423415,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":423415,"total_server_duration_us":42634,"encode_duration_us":16228,"operation_id":"0x24fa","timeout_ms":10000,"last_server_duration_us":42634,"total_duration_us":10007634},{"operation_name":"upsert","last_dispatch_duration_us":5870381,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5870381,"total_server_duration_us":23729,"encode_duration_us":18884,"operation_id":"0x252b","timeout_ms":10000,"last_server_duration_us":23729,"total_duration_us":10007171},{"operation_name":"upsert","last_dispatch_duration_us":967491,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10414326,"total_server_duration_us":467387,"encode_duration_us":16170,"operation_id":"0x24ee","timeout_ms":10000,"last_server_duration_us":32451,"total_duration_us":10006927},{"operation_name":"upsert","last_dispatch_duration_us":7511186,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10528557,"total_server_duration_us":1180008,"encode_duration_us":16542,"operation_id":"0x2405","timeout_ms":10000,"last_server_duration_us":36322,"total_duration_us":10005075},{"operation_name":"upsert","last_dispatch_duration_us":295126,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":736711,"total_server_duration_us":222730,"encode_duration_us":16376,"operation_id":"0x2444","timeout_ms":10000,"last_server_duration_us":28952,"total_duration_us":10003885},{"operation_name":"upsert","last_dispatch_duration_us":832730,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3210328,"total_server_duration_us":225681,"encode_duration_us":18586,"operation_id":"0x2523","timeout_ms":10000,"last_server_duration_us":100596,"total_duration_us":10002998},{"operation_name":"upsert","last_dispatch_duration_us":9058672,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":10473022,"total_server_duration_us":633108,"encode_duration_us":19383,"operation_id":"0x249d","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10001916},{"operation_name":"upsert","last_dispatch_duration_us":9036592,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":10850657,"total_server_duration_us":1034388,"encode_duration_us":16600,"operation_id":"0x24bf","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10001641}],"total_count":17}} INFO [2022-01-10 05:39:21,299] jepsen worker 86 - jepsen.util 896 :ok :read [28 13] INFO [2022-01-10 05:39:21,300] jepsen worker 86 - jepsen.util 896 :invoke :write [28 28] WARN [2022-01-10 05:39:21,316] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":23843153,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":23843153,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9033121,"encode_duration_us":20599,"operation_id":"0x217e","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10007024},{"operation_name":"upsert","last_dispatch_duration_us":10085097,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14544776,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9144223,"encode_duration_us":18017,"operation_id":"0x23b3","timeout_ms":10000,"last_server_duration_us":9059667,"total_duration_us":10007557},{"operation_name":"upsert","last_dispatch_duration_us":10260039,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19076677,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9750230,"encode_duration_us":16538,"operation_id":"0x22d6","timeout_ms":10000,"last_server_duration_us":9478049,"total_duration_us":10009085},{"operation_name":"upsert","last_dispatch_duration_us":11463636,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17422089,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":10797102,"encode_duration_us":16433,"operation_id":"0x223b","timeout_ms":10000,"last_server_duration_us":10242321,"total_duration_us":10008866}],"total_count":4}} WARN [2022-01-10 05:39:21,319] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14834212,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":14834212,"total_server_duration_us":38,"operation_id":"0x234f","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":14836166},{"operation_name":"get","last_dispatch_duration_us":13802888,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":13802888,"total_server_duration_us":32,"operation_id":"0x2381","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":13805002},{"operation_name":"get","last_dispatch_duration_us":11007012,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11007012,"total_server_duration_us":18341,"operation_id":"0x23ad","timeout_ms":30000,"last_server_duration_us":18341,"total_duration_us":11010594},{"operation_name":"upsert","last_dispatch_duration_us":8860420,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8860420,"total_server_duration_us":38899,"encode_duration_us":16233,"operation_id":"0x2442","timeout_ms":10000,"last_server_duration_us":38899,"total_duration_us":10010183},{"operation_name":"upsert","last_dispatch_duration_us":12570512,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12570512,"total_server_duration_us":19418,"encode_duration_us":16423,"operation_id":"0x23d3","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10008199},{"operation_name":"upsert","last_dispatch_duration_us":698776,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":2466525,"total_server_duration_us":59401,"encode_duration_us":16263,"operation_id":"0x2531","timeout_ms":10000,"last_server_duration_us":28768,"total_duration_us":10007663},{"operation_name":"upsert","last_dispatch_duration_us":6066746,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6066746,"total_server_duration_us":52896,"encode_duration_us":17008,"operation_id":"0x24be","timeout_ms":10000,"last_server_duration_us":52896,"total_duration_us":10007562},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":17586,"operation_id":"0x24cd","timeout_ms":10000,"total_duration_us":10007544},{"operation_name":"upsert","last_dispatch_duration_us":9085282,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9085282,"total_server_duration_us":544759,"encode_duration_us":16236,"operation_id":"0x241e","timeout_ms":10000,"last_server_duration_us":544759,"total_duration_us":10007239},{"operation_name":"upsert","last_dispatch_duration_us":5393983,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9899568,"total_server_duration_us":185251,"encode_duration_us":19346,"operation_id":"0x237e","timeout_ms":10000,"last_server_duration_us":30539,"total_duration_us":10006231}],"total_count":16}} WARN [2022-01-10 05:39:21,322] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16142568,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":16142568,"total_server_duration_us":88246,"operation_id":"0x2219","timeout_ms":30000,"last_server_duration_us":88246,"total_duration_us":23854553},{"operation_name":"get","last_dispatch_duration_us":2139020,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2139020,"total_server_duration_us":261930,"operation_id":"0x227e","timeout_ms":30000,"last_server_duration_us":261930,"total_duration_us":23630589},{"operation_name":"get","last_dispatch_duration_us":12865510,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12865510,"total_server_duration_us":49,"operation_id":"0x22d3","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":21096264},{"operation_name":"get","last_dispatch_duration_us":12647905,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12647905,"total_server_duration_us":134692,"operation_id":"0x234e","timeout_ms":30000,"last_server_duration_us":134692,"total_duration_us":18327284},{"operation_name":"get","last_dispatch_duration_us":12680016,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12680016,"total_server_duration_us":143322,"operation_id":"0x2411","timeout_ms":30000,"last_server_duration_us":143322,"total_duration_us":14280150},{"operation_name":"get","last_dispatch_duration_us":12709933,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12709933,"total_server_duration_us":148646,"operation_id":"0x246e","timeout_ms":30000,"last_server_duration_us":148646,"total_duration_us":12712563},{"operation_name":"upsert","last_dispatch_duration_us":8553016,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10483415,"total_server_duration_us":421756,"encode_duration_us":18567,"operation_id":"0x2478","timeout_ms":10000,"last_server_duration_us":22223,"total_duration_us":10007561},{"operation_name":"upsert","last_dispatch_duration_us":1120403,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10521976,"total_server_duration_us":122037,"encode_duration_us":17738,"operation_id":"0x24f8","timeout_ms":10000,"last_server_duration_us":39319,"total_duration_us":10007204},{"operation_name":"upsert","last_dispatch_duration_us":90778,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":2560184,"total_server_duration_us":1372078,"encode_duration_us":17772,"operation_id":"0x24bd","timeout_ms":10000,"last_server_duration_us":58750,"total_duration_us":10006901},{"operation_name":"upsert","last_dispatch_duration_us":12828903,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12828903,"total_server_duration_us":88394,"encode_duration_us":19136,"operation_id":"0x23c6","timeout_ms":10000,"last_server_duration_us":88394,"total_duration_us":10006185}],"total_count":22}} WARN [2022-01-10 05:39:21,323] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15328219,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":15328219,"total_server_duration_us":43,"operation_id":"0x240f","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":15329246},{"operation_name":"get","last_dispatch_duration_us":13864154,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":13864154,"total_server_duration_us":1046,"operation_id":"0x2465","timeout_ms":30000,"last_server_duration_us":1046,"total_duration_us":13864552},{"operation_name":"upsert","last_dispatch_duration_us":1948443,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":6557337,"total_server_duration_us":715249,"encode_duration_us":21188,"operation_id":"0x249b","timeout_ms":10000,"last_server_duration_us":35613,"total_duration_us":10010508},{"operation_name":"upsert","last_dispatch_duration_us":1606280,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11244436,"total_server_duration_us":231652,"encode_duration_us":16595,"operation_id":"0x24c2","timeout_ms":10000,"last_server_duration_us":90182,"total_duration_us":10009922},{"operation_name":"upsert","last_dispatch_duration_us":2011673,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9469392,"total_server_duration_us":274133,"encode_duration_us":20201,"operation_id":"0x251e","timeout_ms":10000,"last_server_duration_us":167828,"total_duration_us":10009365},{"operation_name":"upsert","last_dispatch_duration_us":286176,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7175287,"total_server_duration_us":782442,"encode_duration_us":19460,"operation_id":"0x237b","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10009240},{"operation_name":"upsert","last_dispatch_duration_us":889657,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9396661,"total_server_duration_us":895382,"encode_duration_us":16467,"operation_id":"0x23e3","timeout_ms":10000,"last_server_duration_us":29788,"total_duration_us":10008395},{"operation_name":"upsert","last_dispatch_duration_us":1049542,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":3679238,"total_server_duration_us":140590,"encode_duration_us":16853,"operation_id":"0x24f6","timeout_ms":10000,"last_server_duration_us":32451,"total_duration_us":10005974},{"operation_name":"upsert","last_dispatch_duration_us":1154091,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":6245632,"total_server_duration_us":300455,"encode_duration_us":16453,"operation_id":"0x243f","timeout_ms":10000,"last_server_duration_us":54939,"total_duration_us":10005728},{"operation_name":"upsert","last_dispatch_duration_us":1817152,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10060428,"total_server_duration_us":1314344,"encode_duration_us":16600,"operation_id":"0x237d","timeout_ms":10000,"last_server_duration_us":992930,"total_duration_us":10004669}],"total_count":18}} INFO [2022-01-10 05:39:21,396] jepsen worker 66 - jepsen.util 966 :info :write [22 10] :RequestTimeoutException INFO [2022-01-10 05:39:21,397] jepsen worker 66 - jepsen.util 1056 :invoke :write [22 31] WARN [2022-01-10 05:39:21,419] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18417715,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":19546369,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9357863,"encode_duration_us":28977,"operation_id":"0x224d","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10008016},{"operation_name":"upsert","last_dispatch_duration_us":13396035,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19516923,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":10403969,"encode_duration_us":16550,"operation_id":"0x21d9","timeout_ms":10000,"last_server_duration_us":9475883,"total_duration_us":10008080},{"operation_name":"upsert","last_dispatch_duration_us":18006648,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":19166193,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9211357,"encode_duration_us":16862,"operation_id":"0x227c","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10008736}],"total_count":3}} WARN [2022-01-10 05:39:21,423] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":25696814,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":25696814,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9112856,"encode_duration_us":22663,"operation_id":"0x21aa","timeout_ms":10000,"last_server_duration_us":9112856,"total_duration_us":10007110},{"operation_name":"upsert","last_dispatch_duration_us":9163663,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18104623,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10824457,"encode_duration_us":16983,"operation_id":"0x22ac","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10008685},{"operation_name":"upsert","last_dispatch_duration_us":9504146,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15976340,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9952972,"encode_duration_us":16156,"operation_id":"0x2351","timeout_ms":10000,"last_server_duration_us":9462892,"total_duration_us":10008136}],"total_count":3}} WARN [2022-01-10 05:39:21,517] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":22762382,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":22762382,"total_server_duration_us":38,"operation_id":"0x21a9","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":22762745},{"operation_name":"get","last_dispatch_duration_us":8709970,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8709970,"total_server_duration_us":32,"operation_id":"0x234c","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":14384359},{"operation_name":"upsert","last_dispatch_duration_us":986216,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2128694,"total_server_duration_us":811439,"encode_duration_us":18986,"operation_id":"0x24b0","timeout_ms":10000,"last_server_duration_us":683243,"total_duration_us":10008936},{"operation_name":"upsert","last_dispatch_duration_us":10085097,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":14544776,"total_server_duration_us":9144223,"encode_duration_us":18017,"operation_id":"0x23b3","timeout_ms":10000,"last_server_duration_us":9059667,"total_duration_us":10007557},{"operation_name":"upsert","last_dispatch_duration_us":205731,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9820945,"total_server_duration_us":894452,"encode_duration_us":17107,"operation_id":"0x255b","timeout_ms":10000,"last_server_duration_us":52301,"total_duration_us":10005754},{"operation_name":"upsert","last_dispatch_duration_us":9436621,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13724808,"total_server_duration_us":1401360,"encode_duration_us":21210,"operation_id":"0x2410","timeout_ms":10000,"last_server_duration_us":862786,"total_duration_us":10005727},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","encode_duration_us":16397,"operation_id":"0x2520","timeout_ms":10000,"total_duration_us":10004608},{"operation_name":"upsert","last_dispatch_duration_us":8642218,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11897023,"total_server_duration_us":187525,"encode_duration_us":18907,"operation_id":"0x2440","timeout_ms":10000,"last_server_duration_us":91834,"total_duration_us":10002578},{"operation_name":"upsert","last_dispatch_duration_us":8616422,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13587272,"total_server_duration_us":254415,"encode_duration_us":16553,"operation_id":"0x23f0","timeout_ms":10000,"last_server_duration_us":53135,"total_duration_us":10001943},{"operation_name":"upsert","last_dispatch_duration_us":8650541,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8650541,"total_server_duration_us":19574,"encode_duration_us":16343,"operation_id":"0x23b2","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10001730}],"total_count":19}} INFO [2022-01-10 05:39:21,520] jepsen worker 85 - jepsen.util 1345 :ok :read [28 13] INFO [2022-01-10 05:39:21,520] jepsen worker 85 - jepsen.util 1345 :invoke :read [28 nil] INFO [2022-01-10 05:39:21,531] jepsen worker 40 - jepsen.util 1030 :ok :read [13 15] INFO [2022-01-10 05:39:21,532] jepsen worker 40 - jepsen.util 1030 :invoke :read [13 nil] INFO [2022-01-10 05:39:21,762] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1695906432748538}, :ns_1@172.23.107.227 {:progress 0.1114369501466276}, :ns_1@172.23.105.187 {:progress 0.1114369501466276}, :ns_1@172.23.107.228 {:progress 0.1823529411764706}} INFO [2022-01-10 05:39:21,762] jepsen worker nemesis - couchbase.util Rebalance check count: 118 INFO [2022-01-10 05:39:21,763] jepsen worker nemesis - couchbase.util Rebalance stuck count: 88 INFO [2022-01-10 05:39:22,045] jepsen worker 85 - jepsen.util 1345 :ok :read [28 13] INFO [2022-01-10 05:39:22,046] jepsen worker 85 - jepsen.util 1345 :invoke :read [28 nil] INFO [2022-01-10 05:39:22,057] jepsen worker 40 - jepsen.util 1030 :ok :read [13 15] INFO [2022-01-10 05:39:22,057] jepsen worker 40 - jepsen.util 1030 :invoke :read [13 nil] INFO [2022-01-10 05:39:22,065] jepsen worker 21 - jepsen.util 1371 :info :write [7 47] :RequestTimeoutException INFO [2022-01-10 05:39:22,065] jepsen worker 21 - jepsen.util 1461 :invoke :read [7 nil] INFO [2022-01-10 05:39:22,256] jepsen worker 60 - jepsen.util 1410 :info :write [20 19] :RequestTimeoutException INFO [2022-01-10 05:39:22,256] jepsen worker 60 - jepsen.util 1500 :invoke :write [20 27] INFO [2022-01-10 05:39:22,260] jepsen worker 51 - jepsen.util 1131 :info :write [17 45] :SyncWriteAmbiguous INFO [2022-01-10 05:39:22,261] jepsen worker 51 - jepsen.util 1221 :invoke :read [17 nil] INFO [2022-01-10 05:39:22,298] jepsen worker 51 - jepsen.util 1221 :ok :read [17 41] INFO [2022-01-10 05:39:22,299] jepsen worker 51 - jepsen.util 1221 :invoke :write [17 13] INFO [2022-01-10 05:39:22,362] jepsen worker 84 - jepsen.util 1164 :info :write [28 1] :RequestTimeoutException INFO [2022-01-10 05:39:22,363] jepsen worker 84 - jepsen.util 1254 :invoke :write [28 17] INFO [2022-01-10 05:39:22,365] jepsen worker 85 - jepsen.util 1345 :ok :read [28 13] INFO [2022-01-10 05:39:22,366] jepsen worker 85 - jepsen.util 1345 :invoke :read [28 nil] INFO [2022-01-10 05:39:22,372] jepsen worker 40 - jepsen.util 1030 :ok :read [13 15] INFO [2022-01-10 05:39:22,372] jepsen worker 40 - jepsen.util 1030 :invoke :write [13 31] INFO [2022-01-10 05:39:22,445] jepsen worker 11 - jepsen.util 1091 :info :write [3 25] :RequestTimeoutException INFO [2022-01-10 05:39:22,446] jepsen worker 11 - jepsen.util 1181 :invoke :write [3 1] INFO [2022-01-10 05:39:22,511] jepsen worker 85 - jepsen.util 1345 :ok :read [28 13] INFO [2022-01-10 05:39:22,512] jepsen worker 85 - jepsen.util 1345 :invoke :write [28 3] INFO [2022-01-10 05:39:22,527] jepsen worker 54 - jepsen.util 1224 :info :write [18 9] :RequestTimeoutException INFO [2022-01-10 05:39:22,527] jepsen worker 54 - jepsen.util 1314 :invoke :read [18 nil] INFO [2022-01-10 05:39:22,771] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1695906432748538}, :ns_1@172.23.107.227 {:progress 0.1114369501466276}, :ns_1@172.23.105.187 {:progress 0.1114369501466276}, :ns_1@172.23.107.228 {:progress 0.1823529411764706}} INFO [2022-01-10 05:39:22,771] jepsen worker nemesis - couchbase.util Rebalance check count: 119 INFO [2022-01-10 05:39:22,771] jepsen worker nemesis - couchbase.util Rebalance stuck count: 89 INFO [2022-01-10 05:39:22,784] jepsen worker 64 - jepsen.util 1144 :info :write [21 6] :RequestTimeoutException INFO [2022-01-10 05:39:22,785] jepsen worker 64 - jepsen.util 1234 :invoke :write [21 18] INFO [2022-01-10 05:39:22,946] jepsen worker 38 - jepsen.util 1298 :info :write [12 7] :RequestTimeoutException INFO [2022-01-10 05:39:22,948] jepsen worker 38 - jepsen.util 1388 :invoke :read [12 nil] INFO [2022-01-10 05:39:22,958] jepsen worker 45 - jepsen.util 1395 :ok :read [15 23] INFO [2022-01-10 05:39:22,959] jepsen worker 45 - jepsen.util 1395 :invoke :write [15 3] INFO [2022-01-10 05:39:22,965] jepsen worker 35 - jepsen.util 1205 :ok :read [11 21] INFO [2022-01-10 05:39:22,966] jepsen worker 35 - jepsen.util 1205 :invoke :write [11 18] INFO [2022-01-10 05:39:23,247] jepsen worker 53 - jepsen.util 1043 :info :write [17 47] :RequestTimeoutException INFO [2022-01-10 05:39:23,248] jepsen worker 53 - jepsen.util 1133 :invoke :write [17 34] INFO [2022-01-10 05:39:23,276] jepsen worker 44 - jepsen.util 1124 :info :write [14 39] :RequestTimeoutException INFO [2022-01-10 05:39:23,277] jepsen worker 44 - jepsen.util 1214 :invoke :read [14 nil] INFO [2022-01-10 05:39:23,668] jepsen worker 41 - jepsen.util 941 :ok :read [13 15] INFO [2022-01-10 05:39:23,669] jepsen worker 41 - jepsen.util 941 :invoke :write [13 15] INFO [2022-01-10 05:39:23,762] jepsen worker 10 - jepsen.util 1270 :info :write [3 32] :RequestTimeoutException INFO [2022-01-10 05:39:23,763] jepsen worker 10 - jepsen.util 1360 :invoke :read [3 nil] INFO [2022-01-10 05:39:23,801] jepsen worker 68 - jepsen.util 1148 :ok :read [22 41] INFO [2022-01-10 05:39:23,802] jepsen worker 68 - jepsen.util 1148 :invoke :write [22 6] INFO [2022-01-10 05:39:23,805] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1754385964912281}, :ns_1@172.23.107.227 {:progress 0.1143695014662757}, :ns_1@172.23.105.187 {:progress 0.1143695014662757}, :ns_1@172.23.107.228 {:progress 0.1882352941176471}} INFO [2022-01-10 05:39:23,805] jepsen worker nemesis - couchbase.util Rebalance check count: 120 INFO [2022-01-10 05:39:23,805] jepsen worker nemesis - couchbase.util Rebalance stuck count: 89 INFO [2022-01-10 05:39:23,847] jepsen worker 25 - jepsen.util 1105 :info :write [8 25] :RequestTimeoutException INFO [2022-01-10 05:39:23,847] jepsen worker 25 - jepsen.util 1195 :invoke :read [8 nil] INFO [2022-01-10 05:39:23,865] jepsen worker 33 - jepsen.util 1203 :ok :read [11 21] INFO [2022-01-10 05:39:23,866] jepsen worker 33 - jepsen.util 1203 :invoke :write [11 29] INFO [2022-01-10 05:39:23,898] jepsen worker 23 - jepsen.util 1193 :ok :read [7 :nil] INFO [2022-01-10 05:39:23,898] jepsen worker 23 - jepsen.util 1193 :invoke :write [7 42] INFO [2022-01-10 05:39:24,031] jepsen worker 20 - jepsen.util 920 :info :write [6 2] :RequestTimeoutException INFO [2022-01-10 05:39:24,032] jepsen worker 20 - jepsen.util 1010 :invoke :read [6 nil] INFO [2022-01-10 05:39:24,052] jepsen worker 18 - jepsen.util 918 :ok :read [6 25] INFO [2022-01-10 05:39:24,052] jepsen worker 18 - jepsen.util 918 :invoke :read [6 nil] INFO [2022-01-10 05:39:24,229] jepsen worker 43 - jepsen.util 1303 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:24,230] jepsen worker 43 - jepsen.util 1303 :invoke :read [14 nil] INFO [2022-01-10 05:39:24,240] jepsen worker 19 - jepsen.util 1279 :ok :read [6 25] INFO [2022-01-10 05:39:24,241] jepsen worker 19 - jepsen.util 1279 :invoke :read [6 nil] INFO [2022-01-10 05:39:24,341] jepsen worker 27 - jepsen.util 1107 :info :write [9 49] :RequestTimeoutException INFO [2022-01-10 05:39:24,342] jepsen worker 27 - jepsen.util 1197 :invoke :read [9 nil] INFO [2022-01-10 05:39:24,451] jepsen worker 62 - jepsen.util 1322 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:24,453] jepsen worker 62 - jepsen.util 1322 :invoke :write [20 46] INFO [2022-01-10 05:39:24,456] jepsen worker 25 - jepsen.util 1195 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:24,456] jepsen worker 25 - jepsen.util 1195 :invoke :read [8 nil] INFO [2022-01-10 05:39:24,755] jepsen worker 7 - jepsen.util 1087 :info :write [2 36] :RequestTimeoutException INFO [2022-01-10 05:39:24,756] jepsen worker 7 - jepsen.util 1177 :invoke :write [2 14] INFO [2022-01-10 05:39:24,776] jepsen worker 43 - jepsen.util 1303 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:24,777] jepsen worker 43 - jepsen.util 1303 :invoke :write [14 48] INFO [2022-01-10 05:39:24,790] jepsen worker 25 - jepsen.util 1195 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:24,791] jepsen worker 25 - jepsen.util 1195 :invoke :write [8 37] INFO [2022-01-10 05:39:24,811] jepsen worker 19 - jepsen.util 1279 :ok :read [6 25] INFO [2022-01-10 05:39:24,812] jepsen worker 19 - jepsen.util 1279 :invoke :read [6 nil] INFO [2022-01-10 05:39:24,813] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1754385964912281}, :ns_1@172.23.107.227 {:progress 0.1143695014662757}, :ns_1@172.23.105.187 {:progress 0.1143695014662757}, :ns_1@172.23.107.228 {:progress 0.1882352941176471}} INFO [2022-01-10 05:39:24,813] jepsen worker nemesis - couchbase.util Rebalance check count: 121 INFO [2022-01-10 05:39:24,813] jepsen worker nemesis - couchbase.util Rebalance stuck count: 90 INFO [2022-01-10 05:39:24,830] jepsen worker 4 - jepsen.util 1084 :info :write [1 25] :RequestTimeoutException INFO [2022-01-10 05:39:24,831] jepsen worker 4 - jepsen.util 1174 :invoke :write [1 22] INFO [2022-01-10 05:39:24,840] jepsen worker 32 - jepsen.util 1112 :info :write [10 45] :RequestTimeoutException INFO [2022-01-10 05:39:24,841] jepsen worker 32 - jepsen.util 1202 :invoke :write [10 27] INFO [2022-01-10 05:39:24,856] jepsen worker 17 - jepsen.util 1097 :info :write [5 5] :RequestTimeoutException INFO [2022-01-10 05:39:24,856] jepsen worker 17 - jepsen.util 1187 :invoke :read [5 nil] INFO [2022-01-10 05:39:24,960] jepsen worker 88 - jepsen.util 1258 :info :write [29 13] :RequestTimeoutException INFO [2022-01-10 05:39:24,961] jepsen worker 88 - jepsen.util 1348 :invoke :read [29 nil] INFO [2022-01-10 05:39:25,020] jepsen worker 74 - jepsen.util 1424 :info :write [24 38] :RequestTimeoutException INFO [2022-01-10 05:39:25,021] jepsen worker 74 - jepsen.util 1514 :invoke :write [24 5] INFO [2022-01-10 05:39:25,176] jepsen worker 54 - jepsen.util 1314 :ok :read [18 14] INFO [2022-01-10 05:39:25,176] jepsen worker 54 - jepsen.util 1314 :invoke :write [18 46] INFO [2022-01-10 05:39:25,206] jepsen worker 18 - jepsen.util 918 :ok :read [6 25] INFO [2022-01-10 05:39:25,207] jepsen worker 18 - jepsen.util 918 :invoke :write [6 7] INFO [2022-01-10 05:39:25,252] jepsen worker 52 - jepsen.util 1222 :info :write [17 4] :RequestTimeoutException INFO [2022-01-10 05:39:25,253] jepsen worker 52 - jepsen.util 1312 :invoke :read [17 nil] INFO [2022-01-10 05:39:25,287] jepsen worker 52 - jepsen.util 1312 :ok :read [17 41] INFO [2022-01-10 05:39:25,288] jepsen worker 52 - jepsen.util 1312 :invoke :read [17 nil] INFO [2022-01-10 05:39:25,317] jepsen worker 52 - jepsen.util 1312 :ok :read [17 41] INFO [2022-01-10 05:39:25,317] jepsen worker 52 - jepsen.util 1312 :invoke :read [17 nil] INFO [2022-01-10 05:39:25,337] jepsen worker 22 - jepsen.util 1102 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:25,337] jepsen worker 22 - jepsen.util 1102 :invoke :read [7 nil] INFO [2022-01-10 05:39:25,353] jepsen worker 52 - jepsen.util 1312 :ok :read [17 41] INFO [2022-01-10 05:39:25,353] jepsen worker 52 - jepsen.util 1312 :invoke :write [17 17] INFO [2022-01-10 05:39:25,541] jepsen worker 1 - jepsen.util 1171 :info :write [0 28] :RequestTimeoutException INFO [2022-01-10 05:39:25,542] jepsen worker 1 - jepsen.util 1261 :invoke :write [0 13] INFO [2022-01-10 05:39:25,568] jepsen worker 22 - jepsen.util 1102 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:25,568] jepsen worker 22 - jepsen.util 1102 :invoke :read [7 nil] INFO [2022-01-10 05:39:25,584] jepsen worker 42 - jepsen.util 1122 :info :write [14 25] :RequestTimeoutException INFO [2022-01-10 05:39:25,585] jepsen worker 42 - jepsen.util 1212 :invoke :read [14 nil] INFO [2022-01-10 05:39:25,613] jepsen worker 44 - jepsen.util 1214 :ok :read [14 14] INFO [2022-01-10 05:39:25,613] jepsen worker 44 - jepsen.util 1214 :invoke :read [14 nil] INFO [2022-01-10 05:39:25,676] jepsen worker 55 - jepsen.util 1225 :info :write [18 48] :RequestTimeoutException INFO [2022-01-10 05:39:25,676] jepsen worker 55 - jepsen.util 1315 :invoke :read [18 nil] INFO [2022-01-10 05:39:25,692] jepsen worker 70 - jepsen.util 1510 :info :write [23 3] :RequestTimeoutException INFO [2022-01-10 05:39:25,693] jepsen worker 70 - jepsen.util 1600 :invoke :write [23 40] INFO [2022-01-10 05:39:25,700] jepsen worker 55 - jepsen.util 1315 :ok :read [18 14] INFO [2022-01-10 05:39:25,701] jepsen worker 55 - jepsen.util 1315 :invoke :read [18 nil] INFO [2022-01-10 05:39:25,725] jepsen worker 55 - jepsen.util 1315 :ok :read [18 14] INFO [2022-01-10 05:39:25,726] jepsen worker 55 - jepsen.util 1315 :invoke :write [18 31] INFO [2022-01-10 05:39:25,728] jepsen worker 22 - jepsen.util 1102 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:25,728] jepsen worker 22 - jepsen.util 1102 :invoke :write [7 32] INFO [2022-01-10 05:39:25,780] jepsen worker 19 - jepsen.util 1279 :ok :read [6 25] INFO [2022-01-10 05:39:25,781] jepsen worker 19 - jepsen.util 1279 :invoke :write [6 16] INFO [2022-01-10 05:39:25,818] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1754385964912281}, :ns_1@172.23.107.227 {:progress 0.1143695014662757}, :ns_1@172.23.105.187 {:progress 0.1143695014662757}, :ns_1@172.23.107.228 {:progress 0.1882352941176471}} INFO [2022-01-10 05:39:25,818] jepsen worker nemesis - couchbase.util Rebalance check count: 122 INFO [2022-01-10 05:39:25,818] jepsen worker nemesis - couchbase.util Rebalance stuck count: 91 INFO [2022-01-10 05:39:25,846] jepsen worker 5 - jepsen.util 1085 :info :write [1 10] :RequestTimeoutException INFO [2022-01-10 05:39:25,847] jepsen worker 5 - jepsen.util 1175 :invoke :write [1 14] INFO [2022-01-10 05:39:25,896] jepsen worker 44 - jepsen.util 1214 :ok :read [14 14] INFO [2022-01-10 05:39:25,896] jepsen worker 44 - jepsen.util 1214 :invoke :read [14 nil] INFO [2022-01-10 05:39:25,981] jepsen worker 44 - jepsen.util 1214 :ok :read [14 14] INFO [2022-01-10 05:39:25,982] jepsen worker 44 - jepsen.util 1214 :invoke :read [14 nil] INFO [2022-01-10 05:39:26,010] jepsen worker 30 - jepsen.util 840 :info :write [10 43] :RequestTimeoutException INFO [2022-01-10 05:39:26,010] jepsen worker 30 - jepsen.util 930 :invoke :read [10 nil] INFO [2022-01-10 05:39:26,026] jepsen worker 36 - jepsen.util 846 :info :write [12 24] :RequestTimeoutException INFO [2022-01-10 05:39:26,026] jepsen worker 36 - jepsen.util 936 :invoke :read [12 nil] INFO [2022-01-10 05:39:26,314] jepsen worker 44 - jepsen.util 1214 :ok :read [14 14] INFO [2022-01-10 05:39:26,315] jepsen worker 44 - jepsen.util 1214 :invoke :read [14 nil] INFO [2022-01-10 05:39:26,746] jepsen worker 48 - jepsen.util 948 :info :write [16 6] :RequestTimeoutException INFO [2022-01-10 05:39:26,746] jepsen worker 48 - jepsen.util 1038 :invoke :read [16 nil] INFO [2022-01-10 05:39:26,824] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1754385964912281}, :ns_1@172.23.107.227 {:progress 0.1143695014662757}, :ns_1@172.23.105.187 {:progress 0.1143695014662757}, :ns_1@172.23.107.228 {:progress 0.1882352941176471}} INFO [2022-01-10 05:39:26,824] jepsen worker nemesis - couchbase.util Rebalance check count: 123 INFO [2022-01-10 05:39:26,830] jepsen worker nemesis - couchbase.util Rebalance stuck count: 92 INFO [2022-01-10 05:39:26,955] jepsen worker 89 - jepsen.util 1259 :info :write [29 10] :RequestTimeoutException INFO [2022-01-10 05:39:26,955] jepsen worker 89 - jepsen.util 1349 :invoke :write [29 32] INFO [2022-01-10 05:39:26,970] jepsen worker 44 - jepsen.util 1214 :ok :read [14 14] INFO [2022-01-10 05:39:26,971] jepsen worker 44 - jepsen.util 1214 :invoke :write [14 24] INFO [2022-01-10 05:39:27,321] jepsen worker 67 - jepsen.util 1147 :info :write [22 8] :RequestTimeoutException INFO [2022-01-10 05:39:27,322] jepsen worker 67 - jepsen.util 1237 :invoke :read [22 nil] INFO [2022-01-10 05:39:27,325] jepsen worker 75 - jepsen.util 885 :info :write [25 33] :RequestTimeoutException INFO [2022-01-10 05:39:27,326] jepsen worker 75 - jepsen.util 975 :invoke :write [25 3] INFO [2022-01-10 05:39:27,422] jepsen worker 29 - jepsen.util 1199 :info :write [9 27] :RequestTimeoutException INFO [2022-01-10 05:39:27,423] jepsen worker 29 - jepsen.util 1289 :invoke :write [9 16] INFO [2022-01-10 05:39:27,541] jepsen worker 81 - jepsen.util 1161 :info :write [27 18] :RequestTimeoutException INFO [2022-01-10 05:39:27,542] jepsen worker 82 - jepsen.util 1072 :info :write [27 35] :RequestTimeoutException INFO [2022-01-10 05:39:27,543] jepsen worker 82 - jepsen.util 1162 :invoke :write [27 30] INFO [2022-01-10 05:39:27,544] jepsen worker 81 - jepsen.util 1251 :invoke :write [27 13] INFO [2022-01-10 05:39:27,698] jepsen worker 38 - jepsen.util 1388 :ok :read [12 :nil] INFO [2022-01-10 05:39:27,699] jepsen worker 38 - jepsen.util 1388 :invoke :read [12 nil] INFO [2022-01-10 05:39:27,727] jepsen worker 38 - jepsen.util 1388 :ok :read [12 :nil] INFO [2022-01-10 05:39:27,728] jepsen worker 38 - jepsen.util 1388 :invoke :write [12 34] INFO [2022-01-10 05:39:27,805] jepsen worker 8 - jepsen.util 1088 :info :write [2 34] :RequestTimeoutException INFO [2022-01-10 05:39:27,806] jepsen worker 8 - jepsen.util 1178 :invoke :write [2 13] INFO [2022-01-10 05:39:27,830] jepsen worker 31 - jepsen.util 1471 :info :write [10 10] :RequestTimeoutException INFO [2022-01-10 05:39:27,831] jepsen worker 31 - jepsen.util 1561 :invoke :write [10 13] INFO [2022-01-10 05:39:27,836] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1754385964912281}, :ns_1@172.23.107.227 {:progress 0.1143695014662757}, :ns_1@172.23.105.187 {:progress 0.1143695014662757}, :ns_1@172.23.107.228 {:progress 0.1882352941176471}} INFO [2022-01-10 05:39:27,836] jepsen worker nemesis - couchbase.util Rebalance check count: 124 INFO [2022-01-10 05:39:27,836] jepsen worker nemesis - couchbase.util Rebalance stuck count: 93 INFO [2022-01-10 05:39:27,866] jepsen worker 3 - jepsen.util 1443 :info :write [1 24] :RequestTimeoutException INFO [2022-01-10 05:39:27,866] jepsen worker 3 - jepsen.util 1533 :invoke :write [1 24] INFO [2022-01-10 05:39:28,022] jepsen worker 14 - jepsen.util 1364 :info :write [4 34] :RequestTimeoutException INFO [2022-01-10 05:39:28,022] jepsen worker 14 - jepsen.util 1454 :invoke :write [4 11] INFO [2022-01-10 05:39:28,067] jepsen worker 24 - jepsen.util 1014 :info :write [8 34] :RequestTimeoutException INFO [2022-01-10 05:39:28,067] jepsen worker 24 - jepsen.util 1104 :invoke :write [8 17] INFO [2022-01-10 05:39:28,081] jepsen worker 73 - jepsen.util 1153 :info :write [24 7] :RequestTimeoutException INFO [2022-01-10 05:39:28,082] jepsen worker 73 - jepsen.util 1243 :invoke :read [24 nil] INFO [2022-01-10 05:39:28,088] jepsen worker 59 - jepsen.util 1319 :ok :read [19 31] INFO [2022-01-10 05:39:28,088] jepsen worker 59 - jepsen.util 1319 :invoke :read [19 nil] INFO [2022-01-10 05:39:28,131] jepsen worker 87 - jepsen.util 1437 :ok :read [29 33] INFO [2022-01-10 05:39:28,132] jepsen worker 87 - jepsen.util 1437 :invoke :read [29 nil] INFO [2022-01-10 05:39:28,143] jepsen worker 56 - jepsen.util 1316 :ok :read [18 14] INFO [2022-01-10 05:39:28,144] jepsen worker 56 - jepsen.util 1316 :invoke :read [18 nil] INFO [2022-01-10 05:39:28,151] jepsen worker 30 - jepsen.util 930 :ok :read [10 37] INFO [2022-01-10 05:39:28,152] jepsen worker 30 - jepsen.util 930 :invoke :write [10 16] INFO [2022-01-10 05:39:28,156] jepsen worker 69 - jepsen.util 1239 :ok :read [23 26] INFO [2022-01-10 05:39:28,157] jepsen worker 69 - jepsen.util 1239 :invoke :write [23 45] INFO [2022-01-10 05:39:28,160] jepsen worker 87 - jepsen.util 1437 :ok :read [29 33] INFO [2022-01-10 05:39:28,161] jepsen worker 87 - jepsen.util 1437 :invoke :write [29 5] INFO [2022-01-10 05:39:28,181] jepsen worker 56 - jepsen.util 1316 :ok :read [18 14] INFO [2022-01-10 05:39:28,182] jepsen worker 56 - jepsen.util 1316 :invoke :write [18 25] INFO [2022-01-10 05:39:28,252] jepsen worker 63 - jepsen.util 963 :info :write [21 47] :RequestTimeoutException INFO [2022-01-10 05:39:28,252] jepsen worker 63 - jepsen.util 1053 :invoke :write [21 19] INFO [2022-01-10 05:39:28,420] jepsen worker 57 - jepsen.util 1227 :info :write [19 29] :RequestTimeoutException INFO [2022-01-10 05:39:28,421] jepsen worker 57 - jepsen.util 1317 :invoke :read [19 nil] INFO [2022-01-10 05:39:28,426] jepsen worker 77 - jepsen.util 1427 :info :write [25 47] :RequestTimeoutException INFO [2022-01-10 05:39:28,426] jepsen worker 77 - jepsen.util 1517 :invoke :read [25 nil] INFO [2022-01-10 05:39:28,550] jepsen worker 15 - jepsen.util 915 :info :write [5 49] :RequestTimeoutException INFO [2022-01-10 05:39:28,550] jepsen worker 15 - jepsen.util 1005 :invoke :read [5 nil] INFO [2022-01-10 05:39:28,843] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1754385964912281}, :ns_1@172.23.107.227 {:progress 0.1143695014662757}, :ns_1@172.23.105.187 {:progress 0.1143695014662757}, :ns_1@172.23.107.228 {:progress 0.1882352941176471}} INFO [2022-01-10 05:39:28,843] jepsen worker nemesis - couchbase.util Rebalance check count: 125 INFO [2022-01-10 05:39:28,843] jepsen worker nemesis - couchbase.util Rebalance stuck count: 94 INFO [2022-01-10 05:39:28,847] jepsen worker 78 - jepsen.util 978 :info :write [26 49] :RequestTimeoutException INFO [2022-01-10 05:39:28,848] jepsen worker 78 - jepsen.util 1068 :invoke :write [26 48] INFO [2022-01-10 05:39:28,850] jepsen worker 2 - jepsen.util 1172 :info :write [0 25] :RequestTimeoutException INFO [2022-01-10 05:39:28,851] jepsen worker 2 - jepsen.util 1262 :invoke :read [0 nil] INFO [2022-01-10 05:39:28,866] jepsen worker 80 - jepsen.util 1160 :info :write [26 3] :RequestTimeoutException INFO [2022-01-10 05:39:28,867] jepsen worker 80 - jepsen.util 1250 :invoke :write [26 42] INFO [2022-01-10 05:39:28,949] jepsen worker 77 - jepsen.util 1517 :ok :read [25 :nil] INFO [2022-01-10 05:39:28,949] jepsen worker 77 - jepsen.util 1517 :invoke :read [25 nil] INFO [2022-01-10 05:39:29,012] jepsen worker 47 - jepsen.util 1127 :info :write [15 36] :RequestTimeoutException INFO [2022-01-10 05:39:29,012] jepsen worker 47 - jepsen.util 1217 :invoke :write [15 37] INFO [2022-01-10 05:39:29,296] jepsen worker 16 - jepsen.util 1366 :info :write [5 42] :RequestTimeoutException INFO [2022-01-10 05:39:29,297] jepsen worker 16 - jepsen.util 1456 :invoke :write [5 43] INFO [2022-01-10 05:39:29,358] jepsen worker 79 - jepsen.util 1159 :ok :read [26 10] INFO [2022-01-10 05:39:29,359] jepsen worker 79 - jepsen.util 1159 :invoke :read [26 nil] INFO [2022-01-10 05:39:29,383] jepsen worker 79 - jepsen.util 1159 :ok :read [26 10] INFO [2022-01-10 05:39:29,384] jepsen worker 79 - jepsen.util 1159 :invoke :write [26 18] INFO [2022-01-10 05:39:29,485] jepsen worker 0 - jepsen.util 1170 :info :write [0 0] :RequestTimeoutException INFO [2022-01-10 05:39:29,485] jepsen worker 0 - jepsen.util 1260 :invoke :read [0 nil] INFO [2022-01-10 05:39:29,526] jepsen worker 50 - jepsen.util 1310 :info :write [16 8] :RequestTimeoutException INFO [2022-01-10 05:39:29,527] jepsen worker 50 - jepsen.util 1400 :invoke :write [16 21] INFO [2022-01-10 05:39:29,619] jepsen worker 28 - jepsen.util 1198 :ok :read [9 22] INFO [2022-01-10 05:39:29,620] jepsen worker 28 - jepsen.util 1198 :invoke :read [9 nil] INFO [2022-01-10 05:39:29,630] jepsen worker 58 - jepsen.util 1228 :ok :read [19 31] INFO [2022-01-10 05:39:29,630] jepsen worker 58 - jepsen.util 1228 :invoke :write [19 23] WARN [2022-01-10 05:39:29,697] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9123198,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14589246,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9407027,"encode_duration_us":29082,"operation_id":"0x2492","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10002860},{"operation_name":"upsert","last_dispatch_duration_us":10298735,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13596794,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":10626644,"encode_duration_us":16817,"operation_id":"0x2521","timeout_ms":10000,"last_server_duration_us":9144833,"total_duration_us":10003469},{"operation_name":"upsert","last_dispatch_duration_us":10468560,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11718621,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9806933,"encode_duration_us":17391,"operation_id":"0x257f","timeout_ms":10000,"last_server_duration_us":9765810,"total_duration_us":10004880}],"total_count":3}} WARN [2022-01-10 05:39:29,797] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15051161,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":15051161,"total_server_duration_us":5830429,"operation_id":"0x25ea","timeout_ms":30000,"last_server_duration_us":5830429,"total_duration_us":15053126},{"operation_name":"get","last_dispatch_duration_us":11537951,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11537951,"total_server_duration_us":23,"operation_id":"0x26b6","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":11540230},{"operation_name":"upsert","last_dispatch_duration_us":9004829,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9707066,"total_server_duration_us":194857,"encode_duration_us":16393,"operation_id":"0x258a","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10010383},{"operation_name":"upsert","last_dispatch_duration_us":9002819,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10649659,"total_server_duration_us":573356,"encode_duration_us":17514,"operation_id":"0x255a","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10007033},{"operation_name":"upsert","last_dispatch_duration_us":10468560,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11718621,"total_server_duration_us":9806933,"encode_duration_us":17391,"operation_id":"0x257f","timeout_ms":10000,"last_server_duration_us":9765810,"total_duration_us":10004880},{"operation_name":"upsert","last_dispatch_duration_us":9221260,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13723036,"total_server_duration_us":351660,"encode_duration_us":16664,"operation_id":"0x2522","timeout_ms":10000,"last_server_duration_us":74786,"total_duration_us":10004877},{"operation_name":"upsert","last_dispatch_duration_us":9293169,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":13538395,"total_server_duration_us":720578,"encode_duration_us":18487,"operation_id":"0x252d","timeout_ms":10000,"last_server_duration_us":74648,"total_duration_us":10004591},{"operation_name":"upsert","last_dispatch_duration_us":484441,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2072570,"total_server_duration_us":811448,"encode_duration_us":19302,"operation_id":"0x26a7","timeout_ms":10000,"last_server_duration_us":160113,"total_duration_us":10004151},{"operation_name":"upsert","last_dispatch_duration_us":10298735,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13596794,"total_server_duration_us":10626644,"encode_duration_us":16817,"operation_id":"0x2521","timeout_ms":10000,"last_server_duration_us":9144833,"total_duration_us":10003469},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":17911,"operation_id":"0x26ff","timeout_ms":10000,"total_duration_us":10003444}],"total_count":16}} INFO [2022-01-10 05:39:29,826] jepsen worker 37 - jepsen.util 1207 :info :write [12 39] :RequestTimeoutException INFO [2022-01-10 05:39:29,827] jepsen worker 37 - jepsen.util 1297 :invoke :write [12 22] INFO [2022-01-10 05:39:29,833] jepsen worker 27 - jepsen.util 1197 :ok :read [9 22] INFO [2022-01-10 05:39:29,833] jepsen worker 27 - jepsen.util 1197 :invoke :read [9 nil] INFO [2022-01-10 05:39:29,839] jepsen worker 48 - jepsen.util 1038 :ok :read [16 38] INFO [2022-01-10 05:39:29,840] jepsen worker 48 - jepsen.util 1038 :invoke :read [16 nil] INFO [2022-01-10 05:39:29,849] jepsen worker 6 - jepsen.util 1176 :info :write [2 18] :RequestTimeoutException INFO [2022-01-10 05:39:29,850] jepsen worker 6 - jepsen.util 1266 :invoke :read [2 nil] INFO [2022-01-10 05:39:29,852] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1754385964912281}, :ns_1@172.23.107.227 {:progress 0.1143695014662757}, :ns_1@172.23.105.187 {:progress 0.1143695014662757}, :ns_1@172.23.107.228 {:progress 0.1882352941176471}} INFO [2022-01-10 05:39:29,852] jepsen worker nemesis - couchbase.util Rebalance check count: 126 INFO [2022-01-10 05:39:29,852] jepsen worker nemesis - couchbase.util Rebalance stuck count: 95 INFO [2022-01-10 05:39:29,856] jepsen worker 71 - jepsen.util 1241 :info :write [23 14] :RequestTimeoutException INFO [2022-01-10 05:39:29,858] jepsen worker 71 - jepsen.util 1331 :invoke :write [23 9] INFO [2022-01-10 05:39:29,862] jepsen worker 49 - jepsen.util 1039 :info :write [16 1] :RequestTimeoutException INFO [2022-01-10 05:39:29,862] jepsen worker 49 - jepsen.util 1129 :invoke :read [16 nil] INFO [2022-01-10 05:39:29,877] jepsen worker 34 - jepsen.util 1204 :info :write [11 29] :RequestTimeoutException INFO [2022-01-10 05:39:29,877] jepsen worker 34 - jepsen.util 1294 :invoke :read [11 nil] INFO [2022-01-10 05:39:29,932] jepsen worker 72 - jepsen.util 1242 :info :write [24 45] :RequestTimeoutException INFO [2022-01-10 05:39:29,933] jepsen worker 72 - jepsen.util 1332 :invoke :write [24 4] INFO [2022-01-10 05:39:29,938] jepsen worker 10 - jepsen.util 1360 :ok :read [3 2] INFO [2022-01-10 05:39:29,939] jepsen worker 10 - jepsen.util 1360 :invoke :write [3 3] INFO [2022-01-10 05:39:29,945] jepsen worker 83 - jepsen.util 1343 :info :write [27 48] :RequestTimeoutException INFO [2022-01-10 05:39:29,946] jepsen worker 83 - jepsen.util 1433 :invoke :write [27 23] INFO [2022-01-10 05:39:30,652] jepsen worker 65 - jepsen.util 1145 :info :write [21 45] :RequestTimeoutException INFO [2022-01-10 05:39:30,653] jepsen worker 65 - jepsen.util 1235 :invoke :read [21 nil] INFO [2022-01-10 05:39:30,838] jepsen worker 77 - jepsen.util 1517 :ok :read [25 :nil] INFO [2022-01-10 05:39:30,839] jepsen worker 77 - jepsen.util 1517 :invoke :read [25 nil] INFO [2022-01-10 05:39:30,861] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1754385964912281}, :ns_1@172.23.107.227 {:progress 0.1143695014662757}, :ns_1@172.23.105.187 {:progress 0.1143695014662757}, :ns_1@172.23.107.228 {:progress 0.1882352941176471}} INFO [2022-01-10 05:39:30,861] jepsen worker nemesis - couchbase.util Rebalance check count: 127 INFO [2022-01-10 05:39:30,861] jepsen worker nemesis - couchbase.util Rebalance stuck count: 96 INFO [2022-01-10 05:39:30,991] jepsen worker 46 - jepsen.util 1216 :info :write [15 8] :RequestTimeoutException INFO [2022-01-10 05:39:30,992] jepsen worker 46 - jepsen.util 1306 :invoke :write [15 26] INFO [2022-01-10 05:39:31,142] jepsen worker 59 - jepsen.util 1319 :ok :read [19 31] INFO [2022-01-10 05:39:31,142] jepsen worker 59 - jepsen.util 1319 :invoke :write [19 10] INFO [2022-01-10 05:39:31,159] jepsen worker 2 - jepsen.util 1262 :ok :read [0 46] INFO [2022-01-10 05:39:31,160] jepsen worker 2 - jepsen.util 1262 :invoke :read [0 nil] INFO [2022-01-10 05:39:31,183] jepsen worker 48 - jepsen.util 1038 :ok :read [16 38] INFO [2022-01-10 05:39:31,183] jepsen worker 48 - jepsen.util 1038 :invoke :read [16 nil] INFO [2022-01-10 05:39:31,215] jepsen worker 26 - jepsen.util 1106 :ok :read [8 :nil] INFO [2022-01-10 05:39:31,215] jepsen worker 76 - jepsen.util 976 :ok :read [25 :nil] INFO [2022-01-10 05:39:31,216] jepsen worker 61 - jepsen.util 1231 :ok :read [20 :nil] INFO [2022-01-10 05:39:31,216] jepsen worker 26 - jepsen.util 1106 :invoke :read [8 nil] INFO [2022-01-10 05:39:31,216] jepsen worker 76 - jepsen.util 976 :invoke :read [25 nil] INFO [2022-01-10 05:39:31,217] jepsen worker 61 - jepsen.util 1231 :invoke :write [20 38] INFO [2022-01-10 05:39:31,235] jepsen worker 9 - jepsen.util 1359 :info :write [3 15] :RequestTimeoutException INFO [2022-01-10 05:39:31,236] jepsen worker 9 - jepsen.util 1449 :invoke :write [3 26] WARN [2022-01-10 05:39:31,237] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":13384788,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13384788,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9217491,"encode_duration_us":18728,"operation_id":"0x2624","timeout_ms":10000,"last_server_duration_us":9217491,"total_duration_us":10003508},{"operation_name":"upsert","last_dispatch_duration_us":10692347,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14371585,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9215136,"encode_duration_us":16853,"operation_id":"0x24f6","timeout_ms":10000,"last_server_duration_us":9074546,"total_duration_us":10005974},{"operation_name":"upsert","last_dispatch_duration_us":12230248,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12230248,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":10021705,"encode_duration_us":24824,"operation_id":"0x26c2","timeout_ms":10000,"last_server_duration_us":10021705,"total_duration_us":10008084}],"total_count":3}} INFO [2022-01-10 05:39:31,269] jepsen worker 77 - jepsen.util 1517 :ok :read [25 :nil] INFO [2022-01-10 05:39:31,270] jepsen worker 77 - jepsen.util 1517 :invoke :write [25 7] WARN [2022-01-10 05:39:31,306] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17552776,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18289488,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9248425,"encode_duration_us":16376,"operation_id":"0x2444","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10003885},{"operation_name":"upsert","last_dispatch_duration_us":11635635,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12658758,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":10259275,"encode_duration_us":16828,"operation_id":"0x25de","timeout_ms":10000,"last_server_duration_us":10121732,"total_duration_us":10006774}],"total_count":2}} WARN [2022-01-10 05:39:31,306] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16421160,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":16421160,"total_server_duration_us":43,"operation_id":"0x255e","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":16424263},{"operation_name":"get","last_dispatch_duration_us":14889283,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":14889283,"total_server_duration_us":32,"operation_id":"0x25b6","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":14891820},{"operation_name":"get","last_dispatch_duration_us":11353130,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":11353130,"total_server_duration_us":23,"operation_id":"0x2666","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":11354243},{"operation_name":"upsert","last_dispatch_duration_us":1167441,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":6483494,"total_server_duration_us":1741394,"encode_duration_us":19689,"operation_id":"0x2680","timeout_ms":10000,"last_server_duration_us":22720,"total_duration_us":10011045},{"operation_name":"upsert","last_dispatch_duration_us":1441050,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":6737537,"total_server_duration_us":2266489,"encode_duration_us":17054,"operation_id":"0x2687","timeout_ms":10000,"last_server_duration_us":443125,"total_duration_us":10007473},{"operation_name":"upsert","last_dispatch_duration_us":473987,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":3152849,"total_server_duration_us":380818,"encode_duration_us":17273,"operation_id":"0x25e5","timeout_ms":10000,"last_server_duration_us":119804,"total_duration_us":10007125},{"operation_name":"upsert","last_dispatch_duration_us":11635635,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":12658758,"total_server_duration_us":10259275,"encode_duration_us":16828,"operation_id":"0x25de","timeout_ms":10000,"last_server_duration_us":10121732,"total_duration_us":10006774},{"operation_name":"upsert","last_dispatch_duration_us":1819108,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4577944,"total_server_duration_us":1268421,"encode_duration_us":16176,"operation_id":"0x26b5","timeout_ms":10000,"last_server_duration_us":688916,"total_duration_us":10006732},{"operation_name":"upsert","last_dispatch_duration_us":404583,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":3420503,"total_server_duration_us":232929,"encode_duration_us":16222,"operation_id":"0x261f","timeout_ms":10000,"last_server_duration_us":25625,"total_duration_us":10003680},{"operation_name":"upsert","last_dispatch_duration_us":8281556,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":12915052,"total_server_duration_us":397547,"encode_duration_us":16647,"operation_id":"0x26bd","timeout_ms":10000,"last_server_duration_us":65384,"total_duration_us":10002896}],"total_count":16}} INFO [2022-01-10 05:39:31,321] jepsen worker 86 - jepsen.util 896 :info :write [28 28] :RequestTimeoutException INFO [2022-01-10 05:39:31,322] jepsen worker 86 - jepsen.util 986 :invoke :write [28 8] WARN [2022-01-10 05:39:31,331] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9272377,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9929063,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9193476,"encode_duration_us":22103,"operation_id":"0x260e","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10002632},{"operation_name":"upsert","last_dispatch_duration_us":10807111,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10807111,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9673853,"encode_duration_us":16397,"operation_id":"0x2520","timeout_ms":10000,"last_server_duration_us":9673853,"total_duration_us":10004608},{"operation_name":"upsert","last_dispatch_duration_us":9568092,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19389037,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":10224616,"encode_duration_us":17107,"operation_id":"0x255b","timeout_ms":10000,"last_server_duration_us":9330164,"total_duration_us":10005754}],"total_count":3}} WARN [2022-01-10 05:39:31,333] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10751384,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10751384,"total_server_duration_us":43179,"operation_id":"0x25ba","timeout_ms":30000,"last_server_duration_us":43179,"total_duration_us":10753465},{"operation_name":"upsert","last_dispatch_duration_us":8795177,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":8795177,"total_server_duration_us":18493,"encode_duration_us":17582,"operation_id":"0x26ef","timeout_ms":10000,"last_server_duration_us":18493,"total_duration_us":10008905},{"operation_name":"upsert","last_dispatch_duration_us":8795563,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":13442560,"total_server_duration_us":503896,"encode_duration_us":16247,"operation_id":"0x258b","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10008468},{"operation_name":"upsert","last_dispatch_duration_us":34755,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":8906534,"total_server_duration_us":63629,"encode_duration_us":16695,"operation_id":"0x26f1","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10007857},{"operation_name":"upsert","last_dispatch_duration_us":35809,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":960430,"total_server_duration_us":573595,"encode_duration_us":22211,"operation_id":"0x25b7","timeout_ms":10000,"last_server_duration_us":22471,"total_duration_us":10007586},{"operation_name":"upsert","last_dispatch_duration_us":10642023,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10642023,"total_server_duration_us":72317,"encode_duration_us":22189,"operation_id":"0x25b9","timeout_ms":10000,"last_server_duration_us":72317,"total_duration_us":10007552},{"operation_name":"upsert","last_dispatch_duration_us":1113681,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4020391,"total_server_duration_us":990224,"encode_duration_us":19490,"operation_id":"0x2724","timeout_ms":10000,"last_server_duration_us":611998,"total_duration_us":10005153},{"operation_name":"upsert","last_dispatch_duration_us":802640,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1376312,"total_server_duration_us":265682,"encode_duration_us":19183,"operation_id":"0x26b4","timeout_ms":10000,"last_server_duration_us":130813,"total_duration_us":10004618},{"operation_name":"upsert","last_dispatch_duration_us":10696138,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10696138,"total_server_duration_us":242702,"encode_duration_us":18031,"operation_id":"0x25dc","timeout_ms":10000,"last_server_duration_us":242702,"total_duration_us":10003702},{"operation_name":"upsert","last_dispatch_duration_us":5092864,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10355762,"total_server_duration_us":1338055,"encode_duration_us":16547,"operation_id":"0x26fd","timeout_ms":10000,"last_server_duration_us":1196006,"total_duration_us":10002778}],"total_count":22}} WARN [2022-01-10 05:39:31,337] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":526008,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10256281,"total_server_duration_us":285255,"encode_duration_us":24326,"operation_id":"0x2722","timeout_ms":10000,"last_server_duration_us":40270,"total_duration_us":10010880},{"operation_name":"upsert","last_dispatch_duration_us":9037713,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10755937,"total_server_duration_us":150911,"encode_duration_us":17200,"operation_id":"0x2681","timeout_ms":10000,"last_server_duration_us":71366,"total_duration_us":10009591},{"operation_name":"upsert","last_dispatch_duration_us":12230248,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12230248,"total_server_duration_us":10021705,"encode_duration_us":24824,"operation_id":"0x26c2","timeout_ms":10000,"last_server_duration_us":10021705,"total_duration_us":10008084},{"operation_name":"upsert","last_dispatch_duration_us":11281048,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11281048,"total_server_duration_us":59625,"encode_duration_us":16376,"operation_id":"0x26b1","timeout_ms":10000,"last_server_duration_us":59625,"total_duration_us":10005812},{"operation_name":"upsert","last_dispatch_duration_us":4452855,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":13588757,"total_server_duration_us":556321,"encode_duration_us":16739,"operation_id":"0x2610","timeout_ms":10000,"last_server_duration_us":362361,"total_duration_us":10005028},{"operation_name":"upsert","last_dispatch_duration_us":13384788,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":13384788,"total_server_duration_us":9217491,"encode_duration_us":18728,"operation_id":"0x2624","timeout_ms":10000,"last_server_duration_us":9217491,"total_duration_us":10003508},{"operation_name":"upsert","last_dispatch_duration_us":643387,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10081664,"total_server_duration_us":222681,"encode_duration_us":19002,"operation_id":"0x271f","timeout_ms":10000,"last_server_duration_us":113958,"total_duration_us":10002886},{"operation_name":"upsert","last_dispatch_duration_us":2252404,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10528011,"total_server_duration_us":1091257,"encode_duration_us":16682,"operation_id":"0x2587","timeout_ms":10000,"last_server_duration_us":1034367,"total_duration_us":10002017},{"operation_name":"upsert","last_dispatch_duration_us":2750429,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11205518,"total_server_duration_us":314908,"encode_duration_us":18220,"operation_id":"0x2700","timeout_ms":10000,"last_server_duration_us":295021,"total_duration_us":10001586},{"operation_name":"upsert","last_dispatch_duration_us":1602838,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8232516,"total_server_duration_us":248878,"encode_duration_us":16385,"operation_id":"0x25ff","timeout_ms":10000,"last_server_duration_us":96403,"total_duration_us":10001199}],"total_count":20}} WARN [2022-01-10 05:39:31,338] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10867617,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10867617,"total_server_duration_us":70421,"operation_id":"0x2730","timeout_ms":30000,"last_server_duration_us":70421,"total_duration_us":10867944},{"operation_name":"get","last_dispatch_duration_us":10628805,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10628805,"total_server_duration_us":7856,"operation_id":"0x2731","timeout_ms":30000,"last_server_duration_us":7856,"total_duration_us":10628998},{"operation_name":"get","last_dispatch_duration_us":10628305,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10628305,"total_server_duration_us":7750,"operation_id":"0x2732","timeout_ms":30000,"last_server_duration_us":7750,"total_duration_us":10628506},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16279,"operation_id":"0x2734","timeout_ms":10000,"total_duration_us":10010374},{"operation_name":"upsert","last_dispatch_duration_us":11280219,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11280219,"total_server_duration_us":81971,"encode_duration_us":20991,"operation_id":"0x2726","timeout_ms":10000,"last_server_duration_us":81971,"total_duration_us":10009337},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":17250,"operation_id":"0x2756","timeout_ms":10000,"total_duration_us":10008705},{"operation_name":"upsert","last_dispatch_duration_us":16055284,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":16055284,"total_server_duration_us":9027816,"encode_duration_us":16432,"operation_id":"0x261b","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10008423},{"operation_name":"upsert","last_dispatch_duration_us":1433042,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8650912,"total_server_duration_us":189667,"encode_duration_us":16198,"operation_id":"0x2589","timeout_ms":10000,"last_server_duration_us":85449,"total_duration_us":10007488},{"operation_name":"upsert","last_dispatch_duration_us":10867165,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10867165,"total_server_duration_us":9271072,"encode_duration_us":27008,"operation_id":"0x2723","timeout_ms":10000,"last_server_duration_us":9271072,"total_duration_us":10006448},{"operation_name":"upsert","last_dispatch_duration_us":1459638,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":3292841,"total_server_duration_us":549938,"encode_duration_us":16512,"operation_id":"0x2688","timeout_ms":10000,"last_server_duration_us":62029,"total_duration_us":10004841}],"total_count":16}} INFO [2022-01-10 05:39:31,420] jepsen worker 66 - jepsen.util 1056 :info :write [22 31] :RequestTimeoutException INFO [2022-01-10 05:39:31,421] jepsen worker 66 - jepsen.util 1146 :invoke :read [22 nil] WARN [2022-01-10 05:39:31,433] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9038872,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10940775,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9307500,"encode_duration_us":17727,"operation_id":"0x2649","timeout_ms":10000,"last_server_duration_us":9019331,"total_duration_us":10000932},{"operation_name":"upsert","last_dispatch_duration_us":10464306,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12930831,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9154158,"encode_duration_us":16263,"operation_id":"0x2531","timeout_ms":10000,"last_server_duration_us":9094757,"total_duration_us":10007663},{"operation_name":"upsert","last_dispatch_duration_us":15291770,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":15291770,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9820722,"encode_duration_us":17586,"operation_id":"0x24cd","timeout_ms":10000,"last_server_duration_us":9820722,"total_duration_us":10007544}],"total_count":3}} WARN [2022-01-10 05:39:31,438] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9044796,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9509929,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9269178,"encode_duration_us":26876,"operation_id":"0x25cf","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10001306},{"operation_name":"upsert","last_dispatch_duration_us":10867165,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10867165,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9271072,"encode_duration_us":27008,"operation_id":"0x2723","timeout_ms":10000,"last_server_duration_us":9271072,"total_duration_us":10006448},{"operation_name":"upsert","last_dispatch_duration_us":11818359,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11818359,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9995103,"encode_duration_us":18362,"operation_id":"0x26f5","timeout_ms":10000,"last_server_duration_us":9995103,"total_duration_us":10004773},{"operation_name":"upsert","last_dispatch_duration_us":16055284,"last_local_id":"EFE9906200000004/000000008B42EFB5","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":16055284,"last_local_socket":"172.23.104.79:44292","total_server_duration_us":9027816,"encode_duration_us":16432,"operation_id":"0x261b","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10008423}],"total_count":4}} WARN [2022-01-10 05:39:31,532] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":49841,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":7501921,"total_server_duration_us":203094,"encode_duration_us":19046,"operation_id":"0x26ae","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10010809},{"operation_name":"upsert","last_dispatch_duration_us":47970,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":1431804,"total_server_duration_us":484202,"encode_duration_us":17329,"operation_id":"0x2648","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10010664},{"operation_name":"upsert","last_dispatch_duration_us":7125850,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8817997,"total_server_duration_us":402109,"encode_duration_us":17103,"operation_id":"0x2613","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10008280},{"operation_name":"upsert","last_dispatch_duration_us":662292,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":3339669,"total_server_duration_us":1303732,"encode_duration_us":20451,"operation_id":"0x26c8","timeout_ms":10000,"last_server_duration_us":346300,"total_duration_us":10007955},{"operation_name":"upsert","last_dispatch_duration_us":132261,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":1639742,"total_server_duration_us":433503,"encode_duration_us":26142,"operation_id":"0x260c","timeout_ms":10000,"last_server_duration_us":68414,"total_duration_us":10007004},{"operation_name":"upsert","last_dispatch_duration_us":469852,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10143186,"total_server_duration_us":342911,"encode_duration_us":19720,"operation_id":"0x2614","timeout_ms":10000,"last_server_duration_us":52777,"total_duration_us":10006912},{"operation_name":"upsert","last_dispatch_duration_us":561720,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":3237805,"total_server_duration_us":296845,"encode_duration_us":23793,"operation_id":"0x26f0","timeout_ms":10000,"last_server_duration_us":53734,"total_duration_us":10005054},{"operation_name":"upsert","last_dispatch_duration_us":1536628,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9650735,"total_server_duration_us":1259555,"encode_duration_us":17526,"operation_id":"0x2721","timeout_ms":10000,"last_server_duration_us":609298,"total_duration_us":10003960},{"operation_name":"upsert","last_dispatch_duration_us":9272377,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9929063,"total_server_duration_us":9193476,"encode_duration_us":22103,"operation_id":"0x260e","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10002632},{"operation_name":"upsert","last_dispatch_duration_us":634038,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":3507339,"total_server_duration_us":377927,"encode_duration_us":17356,"operation_id":"0x26c1","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10001952}],"total_count":19}} INFO [2022-01-10 05:39:31,568] jepsen worker 15 - jepsen.util 1005 :ok :read [5 42] INFO [2022-01-10 05:39:31,568] jepsen worker 15 - jepsen.util 1005 :invoke :read [5 nil] INFO [2022-01-10 05:39:31,572] jepsen worker 39 - jepsen.util 1389 :ok :read [13 15] INFO [2022-01-10 05:39:31,573] jepsen worker 39 - jepsen.util 1389 :invoke :write [13 9] INFO [2022-01-10 05:39:31,736] jepsen worker 6 - jepsen.util 1266 :ok :read [2 14] INFO [2022-01-10 05:39:31,736] jepsen worker 6 - jepsen.util 1266 :invoke :write [2 42] INFO [2022-01-10 05:39:31,797] jepsen worker 65 - jepsen.util 1235 :ok :read [21 7] INFO [2022-01-10 05:39:31,797] jepsen worker 65 - jepsen.util 1235 :invoke :write [21 3] INFO [2022-01-10 05:39:31,868] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1812865497076024}, :ns_1@172.23.107.227 {:progress 0.126099706744868}, :ns_1@172.23.105.187 {:progress 0.126099706744868}, :ns_1@172.23.107.228 {:progress 0.1941176470588235}} INFO [2022-01-10 05:39:31,951] jepsen worker nemesis - couchbase.util Rebalance check count: 128 INFO [2022-01-10 05:39:31,951] jepsen worker nemesis - couchbase.util Rebalance stuck count: 96 INFO [2022-01-10 05:39:32,164] jepsen worker 73 - jepsen.util 1243 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:32,165] jepsen worker 42 - jepsen.util 1212 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:32,167] jepsen worker 42 - jepsen.util 1212 :invoke :read [14 nil] INFO [2022-01-10 05:39:32,168] jepsen worker 73 - jepsen.util 1243 :invoke :write [24 31] INFO [2022-01-10 05:39:32,176] jepsen worker 21 - jepsen.util 1461 :ok :read [7 :nil] INFO [2022-01-10 05:39:32,177] jepsen worker 21 - jepsen.util 1461 :invoke :write [7 35] INFO [2022-01-10 05:39:32,277] jepsen worker 60 - jepsen.util 1500 :info :write [20 27] :RequestTimeoutException INFO [2022-01-10 05:39:32,277] jepsen worker 60 - jepsen.util 1590 :invoke :read [20 nil] INFO [2022-01-10 05:39:32,321] jepsen worker 51 - jepsen.util 1221 :info :write [17 13] :RequestTimeoutException INFO [2022-01-10 05:39:32,322] jepsen worker 51 - jepsen.util 1311 :invoke :write [17 8] INFO [2022-01-10 05:39:32,386] jepsen worker 84 - jepsen.util 1254 :info :write [28 17] :RequestTimeoutException INFO [2022-01-10 05:39:32,387] jepsen worker 84 - jepsen.util 1344 :invoke :read [28 nil] INFO [2022-01-10 05:39:32,401] jepsen worker 40 - jepsen.util 1030 :info :write [13 31] :RequestTimeoutException INFO [2022-01-10 05:39:32,402] jepsen worker 40 - jepsen.util 1120 :invoke :read [13 nil] INFO [2022-01-10 05:39:32,419] jepsen worker 34 - jepsen.util 1294 :ok :read [11 21] INFO [2022-01-10 05:39:32,419] jepsen worker 34 - jepsen.util 1294 :invoke :read [11 nil] INFO [2022-01-10 05:39:32,466] jepsen worker 11 - jepsen.util 1181 :info :write [3 1] :RequestTimeoutException INFO [2022-01-10 05:39:32,467] jepsen worker 11 - jepsen.util 1271 :invoke :read [3 nil] INFO [2022-01-10 05:39:32,532] jepsen worker 85 - jepsen.util 1345 :info :write [28 3] :RequestTimeoutException INFO [2022-01-10 05:39:32,533] jepsen worker 85 - jepsen.util 1435 :invoke :write [28 22] INFO [2022-01-10 05:39:32,608] jepsen worker 20 - jepsen.util 1010 :fail :read [6 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:32,608] jepsen worker 20 - jepsen.util 1010 :invoke :write [6 48] INFO [2022-01-10 05:39:32,795] jepsen worker 34 - jepsen.util 1294 :ok :read [11 21] INFO [2022-01-10 05:39:32,795] jepsen worker 34 - jepsen.util 1294 :invoke :write [11 19] INFO [2022-01-10 05:39:32,812] jepsen worker 64 - jepsen.util 1234 :info :write [21 18] :RequestTimeoutException INFO [2022-01-10 05:39:32,812] jepsen worker 64 - jepsen.util 1324 :invoke :read [21 nil] INFO [2022-01-10 05:39:32,957] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1812865497076024}, :ns_1@172.23.107.227 {:progress 0.126099706744868}, :ns_1@172.23.105.187 {:progress 0.126099706744868}, :ns_1@172.23.107.228 {:progress 0.1941176470588235}} INFO [2022-01-10 05:39:32,957] jepsen worker nemesis - couchbase.util Rebalance check count: 129 INFO [2022-01-10 05:39:32,957] jepsen worker nemesis - couchbase.util Rebalance stuck count: 97 INFO [2022-01-10 05:39:32,986] jepsen worker 45 - jepsen.util 1395 :info :write [15 3] :RequestTimeoutException INFO [2022-01-10 05:39:32,986] jepsen worker 45 - jepsen.util 1485 :invoke :read [15 nil] INFO [2022-01-10 05:39:32,996] jepsen worker 35 - jepsen.util 1205 :info :write [11 18] :RequestTimeoutException INFO [2022-01-10 05:39:32,997] jepsen worker 35 - jepsen.util 1295 :invoke :write [11 39] INFO [2022-01-10 05:39:33,209] jepsen worker 42 - jepsen.util 1212 :ok :read [14 14] INFO [2022-01-10 05:39:33,210] jepsen worker 42 - jepsen.util 1212 :invoke :read [14 nil] INFO [2022-01-10 05:39:33,281] jepsen worker 53 - jepsen.util 1133 :info :write [17 34] :RequestTimeoutException INFO [2022-01-10 05:39:33,282] jepsen worker 53 - jepsen.util 1223 :invoke :write [17 35] INFO [2022-01-10 05:39:33,416] jepsen worker 28 - jepsen.util 1198 :ok :read [9 22] INFO [2022-01-10 05:39:33,417] jepsen worker 28 - jepsen.util 1198 :invoke :write [9 31] INFO [2022-01-10 05:39:33,600] jepsen worker 88 - jepsen.util 1348 :ok :read [29 33] INFO [2022-01-10 05:39:33,600] jepsen worker 88 - jepsen.util 1348 :invoke :read [29 nil] INFO [2022-01-10 05:39:33,622] jepsen worker 45 - jepsen.util 1485 :ok :read [15 23] INFO [2022-01-10 05:39:33,623] jepsen worker 45 - jepsen.util 1485 :invoke :read [15 nil] INFO [2022-01-10 05:39:33,696] jepsen worker 41 - jepsen.util 941 :info :write [13 15] :RequestTimeoutException INFO [2022-01-10 05:39:33,697] jepsen worker 41 - jepsen.util 1031 :invoke :read [13 nil] INFO [2022-01-10 05:39:33,698] jepsen worker 88 - jepsen.util 1348 :ok :read [29 33] INFO [2022-01-10 05:39:33,698] jepsen worker 88 - jepsen.util 1348 :invoke :read [29 nil] INFO [2022-01-10 05:39:33,760] jepsen worker 26 - jepsen.util 1106 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:33,760] jepsen worker 60 - jepsen.util 1590 :ok :read [20 :nil] INFO [2022-01-10 05:39:33,761] jepsen worker 76 - jepsen.util 976 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:33,762] jepsen worker 76 - jepsen.util 976 :invoke :read [25 nil] INFO [2022-01-10 05:39:33,763] jepsen worker 60 - jepsen.util 1590 :invoke :read [20 nil] INFO [2022-01-10 05:39:33,765] jepsen worker 26 - jepsen.util 1106 :invoke :read [8 nil] INFO [2022-01-10 05:39:33,799] jepsen worker 88 - jepsen.util 1348 :ok :read [29 33] INFO [2022-01-10 05:39:33,800] jepsen worker 88 - jepsen.util 1348 :invoke :read [29 nil] INFO [2022-01-10 05:39:33,825] jepsen worker 27 - jepsen.util 1197 :ok :read [9 22] INFO [2022-01-10 05:39:33,826] jepsen worker 27 - jepsen.util 1197 :invoke :write [9 42] INFO [2022-01-10 05:39:33,836] jepsen worker 68 - jepsen.util 1148 :info :write [22 6] :RequestTimeoutException INFO [2022-01-10 05:39:33,837] jepsen worker 68 - jepsen.util 1238 :invoke :write [22 3] INFO [2022-01-10 05:39:33,895] jepsen worker 33 - jepsen.util 1203 :info :write [11 29] :RequestTimeoutException INFO [2022-01-10 05:39:33,896] jepsen worker 33 - jepsen.util 1293 :invoke :read [11 nil] INFO [2022-01-10 05:39:33,903] jepsen worker 88 - jepsen.util 1348 :ok :read [29 33] INFO [2022-01-10 05:39:33,904] jepsen worker 88 - jepsen.util 1348 :invoke :write [29 37] INFO [2022-01-10 05:39:33,919] jepsen worker 23 - jepsen.util 1193 :info :write [7 42] :RequestTimeoutException INFO [2022-01-10 05:39:33,920] jepsen worker 23 - jepsen.util 1283 :invoke :read [7 nil] INFO [2022-01-10 05:39:33,964] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1812865497076024}, :ns_1@172.23.107.227 {:progress 0.126099706744868}, :ns_1@172.23.105.187 {:progress 0.126099706744868}, :ns_1@172.23.107.228 {:progress 0.1941176470588235}} INFO [2022-01-10 05:39:33,964] jepsen worker nemesis - couchbase.util Rebalance check count: 130 INFO [2022-01-10 05:39:33,964] jepsen worker nemesis - couchbase.util Rebalance stuck count: 98 INFO [2022-01-10 05:39:34,046] jepsen worker 49 - jepsen.util 1129 :ok :read [16 38] INFO [2022-01-10 05:39:34,047] jepsen worker 49 - jepsen.util 1129 :invoke :read [16 nil] INFO [2022-01-10 05:39:34,337] jepsen worker 42 - jepsen.util 1212 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:34,338] jepsen worker 42 - jepsen.util 1212 :invoke :read [14 nil] INFO [2022-01-10 05:39:34,441] jepsen worker 64 - jepsen.util 1324 :ok :read [21 7] INFO [2022-01-10 05:39:34,442] jepsen worker 64 - jepsen.util 1324 :invoke :write [21 47] INFO [2022-01-10 05:39:34,447] jepsen worker 0 - jepsen.util 1260 :ok :read [0 46] INFO [2022-01-10 05:39:34,448] jepsen worker 0 - jepsen.util 1260 :invoke :read [0 nil] INFO [2022-01-10 05:39:34,480] jepsen worker 62 - jepsen.util 1322 :info :write [20 46] :RequestTimeoutException INFO [2022-01-10 05:39:34,480] jepsen worker 62 - jepsen.util 1412 :invoke :read [20 nil] INFO [2022-01-10 05:39:34,489] jepsen worker 45 - jepsen.util 1485 :ok :read [15 23] INFO [2022-01-10 05:39:34,489] jepsen worker 45 - jepsen.util 1485 :invoke :read [15 nil] INFO [2022-01-10 05:39:34,731] jepsen worker 23 - jepsen.util 1283 :ok :read [7 :nil] INFO [2022-01-10 05:39:34,732] jepsen worker 23 - jepsen.util 1283 :invoke :write [7 46] INFO [2022-01-10 05:39:34,782] jepsen worker 7 - jepsen.util 1177 :info :write [2 14] :RequestTimeoutException INFO [2022-01-10 05:39:34,782] jepsen worker 7 - jepsen.util 1267 :invoke :read [2 nil] INFO [2022-01-10 05:39:34,809] jepsen worker 43 - jepsen.util 1303 :info :write [14 48] :RequestTimeoutException INFO [2022-01-10 05:39:34,810] jepsen worker 43 - jepsen.util 1393 :invoke :write [14 46] INFO [2022-01-10 05:39:34,820] jepsen worker 25 - jepsen.util 1195 :info :write [8 37] :RequestTimeoutException INFO [2022-01-10 05:39:34,820] jepsen worker 25 - jepsen.util 1285 :invoke :read [8 nil] INFO [2022-01-10 05:39:34,855] jepsen worker 4 - jepsen.util 1174 :info :write [1 22] :RequestTimeoutException INFO [2022-01-10 05:39:34,856] jepsen worker 4 - jepsen.util 1264 :invoke :write [1 9] INFO [2022-01-10 05:39:34,866] jepsen worker 32 - jepsen.util 1202 :info :write [10 27] :RequestTimeoutException INFO [2022-01-10 05:39:34,867] jepsen worker 32 - jepsen.util 1292 :invoke :read [10 nil] INFO [2022-01-10 05:39:34,891] jepsen worker 60 - jepsen.util 1590 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:34,893] jepsen worker 33 - jepsen.util 1293 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:34,894] jepsen worker 42 - jepsen.util 1212 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:34,894] jepsen worker 33 - jepsen.util 1293 :invoke :read [11 nil] INFO [2022-01-10 05:39:34,895] jepsen worker 60 - jepsen.util 1590 :invoke :write [20 33] INFO [2022-01-10 05:39:34,895] jepsen worker 42 - jepsen.util 1212 :invoke :write [14 15] INFO [2022-01-10 05:39:34,970] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1812865497076024}, :ns_1@172.23.107.227 {:progress 0.126099706744868}, :ns_1@172.23.105.187 {:progress 0.126099706744868}, :ns_1@172.23.107.228 {:progress 0.1941176470588235}} INFO [2022-01-10 05:39:34,970] jepsen worker nemesis - couchbase.util Rebalance check count: 131 INFO [2022-01-10 05:39:34,970] jepsen worker nemesis - couchbase.util Rebalance stuck count: 99 INFO [2022-01-10 05:39:35,057] jepsen worker 74 - jepsen.util 1514 :info :write [24 5] :RequestTimeoutException INFO [2022-01-10 05:39:35,057] jepsen worker 74 - jepsen.util 1604 :invoke :write [24 1] INFO [2022-01-10 05:39:35,200] jepsen worker 54 - jepsen.util 1314 :info :write [18 46] :RequestTimeoutException INFO [2022-01-10 05:39:35,200] jepsen worker 54 - jepsen.util 1404 :invoke :read [18 nil] INFO [2022-01-10 05:39:35,226] jepsen worker 18 - jepsen.util 918 :info :write [6 7] :RequestTimeoutException INFO [2022-01-10 05:39:35,227] jepsen worker 18 - jepsen.util 1008 :invoke :read [6 nil] INFO [2022-01-10 05:39:35,304] jepsen worker 7 - jepsen.util 1267 :ok :read [2 14] INFO [2022-01-10 05:39:35,305] jepsen worker 7 - jepsen.util 1267 :invoke :write [2 40] INFO [2022-01-10 05:39:35,381] jepsen worker 52 - jepsen.util 1312 :info :write [17 17] :RequestTimeoutException INFO [2022-01-10 05:39:35,386] jepsen worker 52 - jepsen.util 1402 :invoke :write [17 39] INFO [2022-01-10 05:39:35,389] jepsen worker 45 - jepsen.util 1485 :ok :read [15 23] INFO [2022-01-10 05:39:35,390] jepsen worker 45 - jepsen.util 1485 :invoke :write [15 23] INFO [2022-01-10 05:39:35,493] jepsen worker 33 - jepsen.util 1293 :ok :read [11 21] INFO [2022-01-10 05:39:35,493] jepsen worker 33 - jepsen.util 1293 :invoke :write [11 33] INFO [2022-01-10 05:39:35,575] jepsen worker 1 - jepsen.util 1261 :info :write [0 13] :RequestTimeoutException INFO [2022-01-10 05:39:35,576] jepsen worker 1 - jepsen.util 1351 :invoke :read [0 nil] INFO [2022-01-10 05:39:35,722] jepsen worker 70 - jepsen.util 1600 :info :write [23 40] :RequestTimeoutException INFO [2022-01-10 05:39:35,722] jepsen worker 70 - jepsen.util 1690 :invoke :read [23 nil] INFO [2022-01-10 05:39:35,756] jepsen worker 22 - jepsen.util 1102 :info :write [7 32] :RequestTimeoutException INFO [2022-01-10 05:39:35,757] jepsen worker 22 - jepsen.util 1192 :invoke :write [7 30] INFO [2022-01-10 05:39:35,760] jepsen worker 55 - jepsen.util 1315 :info :write [18 31] :RequestTimeoutException INFO [2022-01-10 05:39:35,760] jepsen worker 55 - jepsen.util 1405 :invoke :write [18 22] INFO [2022-01-10 05:39:35,800] jepsen worker 19 - jepsen.util 1279 :info :write [6 16] :RequestTimeoutException INFO [2022-01-10 05:39:35,800] jepsen worker 19 - jepsen.util 1369 :invoke :write [6 15] INFO [2022-01-10 05:39:35,866] jepsen worker 5 - jepsen.util 1175 :info :write [1 14] :RequestTimeoutException INFO [2022-01-10 05:39:35,867] jepsen worker 5 - jepsen.util 1265 :invoke :write [1 47] INFO [2022-01-10 05:39:35,943] jepsen worker 57 - jepsen.util 1317 :ok :read [19 31] INFO [2022-01-10 05:39:35,943] jepsen worker 57 - jepsen.util 1317 :invoke :read [19 nil] INFO [2022-01-10 05:39:35,976] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1812865497076024}, :ns_1@172.23.107.227 {:progress 0.126099706744868}, :ns_1@172.23.105.187 {:progress 0.126099706744868}, :ns_1@172.23.107.228 {:progress 0.1941176470588235}} INFO [2022-01-10 05:39:35,976] jepsen worker nemesis - couchbase.util Rebalance check count: 132 INFO [2022-01-10 05:39:35,976] jepsen worker nemesis - couchbase.util Rebalance stuck count: 100 INFO [2022-01-10 05:39:36,351] jepsen worker 67 - jepsen.util 1237 :ok :read [22 41] INFO [2022-01-10 05:39:36,351] jepsen worker 2 - jepsen.util 1262 :ok :read [0 46] INFO [2022-01-10 05:39:36,352] jepsen worker 67 - jepsen.util 1237 :invoke :write [22 29] INFO [2022-01-10 05:39:36,352] jepsen worker 2 - jepsen.util 1262 :invoke :write [0 11] INFO [2022-01-10 05:39:36,979] jepsen worker 89 - jepsen.util 1349 :info :write [29 32] :RequestTimeoutException INFO [2022-01-10 05:39:36,980] jepsen worker 89 - jepsen.util 1439 :invoke :write [29 8] INFO [2022-01-10 05:39:36,982] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1871345029239766}, :ns_1@172.23.107.227 {:progress 0.1319648093841642}, :ns_1@172.23.105.187 {:progress 0.1319648093841642}, :ns_1@172.23.107.228 {:progress 0.2}} INFO [2022-01-10 05:39:36,995] jepsen worker 44 - jepsen.util 1214 :info :write [14 24] :RequestTimeoutException INFO [2022-01-10 05:39:37,217] jepsen worker nemesis - couchbase.util Rebalance check count: 133 INFO [2022-01-10 05:39:37,217] jepsen worker nemesis - couchbase.util Rebalance stuck count: 100 INFO [2022-01-10 05:39:37,217] jepsen worker 44 - jepsen.util 1304 :invoke :write [14 48] INFO [2022-01-10 05:39:37,294] jepsen worker 18 - jepsen.util 1008 :ok :read [6 25] INFO [2022-01-10 05:39:37,295] jepsen worker 18 - jepsen.util 1008 :invoke :read [6 nil] INFO [2022-01-10 05:39:37,347] jepsen worker 75 - jepsen.util 975 :info :write [25 3] :RequestTimeoutException INFO [2022-01-10 05:39:37,348] jepsen worker 75 - jepsen.util 1065 :invoke :read [25 nil] INFO [2022-01-10 05:39:37,451] jepsen worker 29 - jepsen.util 1289 :info :write [9 16] :RequestTimeoutException INFO [2022-01-10 05:39:37,451] jepsen worker 29 - jepsen.util 1379 :invoke :read [9 nil] INFO [2022-01-10 05:39:37,565] jepsen worker 82 - jepsen.util 1162 :info :write [27 30] :RequestTimeoutException INFO [2022-01-10 05:39:37,566] jepsen worker 81 - jepsen.util 1251 :info :write [27 13] :RequestTimeoutException INFO [2022-01-10 05:39:37,566] jepsen worker 82 - jepsen.util 1252 :invoke :read [27 nil] INFO [2022-01-10 05:39:37,567] jepsen worker 81 - jepsen.util 1341 :invoke :read [27 nil] INFO [2022-01-10 05:39:37,581] jepsen worker 76 - jepsen.util 976 :ok :read [25 :nil] INFO [2022-01-10 05:39:37,581] jepsen worker 62 - jepsen.util 1412 :ok :read [20 :nil] INFO [2022-01-10 05:39:37,582] jepsen worker 62 - jepsen.util 1412 :invoke :write [20 14] INFO [2022-01-10 05:39:37,582] jepsen worker 76 - jepsen.util 976 :invoke :read [25 nil] INFO [2022-01-10 05:39:37,583] jepsen worker 26 - jepsen.util 1106 :ok :read [8 :nil] INFO [2022-01-10 05:39:37,583] jepsen worker 26 - jepsen.util 1106 :invoke :read [8 nil] INFO [2022-01-10 05:39:37,602] jepsen worker 84 - jepsen.util 1344 :ok :read [28 13] INFO [2022-01-10 05:39:37,602] jepsen worker 84 - jepsen.util 1344 :invoke :read [28 nil] INFO [2022-01-10 05:39:37,756] jepsen worker 38 - jepsen.util 1388 :info :write [12 34] :RequestTimeoutException INFO [2022-01-10 05:39:37,757] jepsen worker 38 - jepsen.util 1478 :invoke :read [12 nil] INFO [2022-01-10 05:39:37,803] jepsen worker 57 - jepsen.util 1317 :ok :read [19 31] INFO [2022-01-10 05:39:37,804] jepsen worker 57 - jepsen.util 1317 :invoke :write [19 21] INFO [2022-01-10 05:39:37,829] jepsen worker 8 - jepsen.util 1178 :info :write [2 13] :RequestTimeoutException INFO [2022-01-10 05:39:37,830] jepsen worker 8 - jepsen.util 1268 :invoke :read [2 nil] INFO [2022-01-10 05:39:37,852] jepsen worker 31 - jepsen.util 1561 :info :write [10 13] :RequestTimeoutException INFO [2022-01-10 05:39:37,852] jepsen worker 31 - jepsen.util 1651 :invoke :read [10 nil] INFO [2022-01-10 05:39:37,886] jepsen worker 3 - jepsen.util 1533 :info :write [1 24] :RequestTimeoutException INFO [2022-01-10 05:39:37,886] jepsen worker 3 - jepsen.util 1623 :invoke :read [1 nil] INFO [2022-01-10 05:39:38,041] jepsen worker 14 - jepsen.util 1454 :info :write [4 11] :RequestTimeoutException INFO [2022-01-10 05:39:38,042] jepsen worker 14 - jepsen.util 1544 :invoke :read [4 nil] INFO [2022-01-10 05:39:38,095] jepsen worker 24 - jepsen.util 1104 :info :write [8 17] :RequestTimeoutException INFO [2022-01-10 05:39:38,096] jepsen worker 24 - jepsen.util 1194 :invoke :write [8 10] INFO [2022-01-10 05:39:38,181] jepsen worker 87 - jepsen.util 1437 :info :write [29 5] :RequestTimeoutException INFO [2022-01-10 05:39:38,181] jepsen worker 30 - jepsen.util 930 :info :write [10 16] :RequestTimeoutException INFO [2022-01-10 05:39:38,181] jepsen worker 69 - jepsen.util 1239 :info :write [23 45] :RequestTimeoutException INFO [2022-01-10 05:39:38,182] jepsen worker 87 - jepsen.util 1527 :invoke :write [29 36] INFO [2022-01-10 05:39:38,183] jepsen worker 30 - jepsen.util 1020 :invoke :write [10 26] INFO [2022-01-10 05:39:38,184] jepsen worker 69 - jepsen.util 1329 :invoke :write [23 6] INFO [2022-01-10 05:39:38,199] jepsen worker 8 - jepsen.util 1268 :ok :read [2 14] INFO [2022-01-10 05:39:38,200] jepsen worker 8 - jepsen.util 1268 :invoke :write [2 48] INFO [2022-01-10 05:39:38,201] jepsen worker 56 - jepsen.util 1316 :info :write [18 25] :RequestTimeoutException INFO [2022-01-10 05:39:38,202] jepsen worker 56 - jepsen.util 1406 :invoke :read [18 nil] INFO [2022-01-10 05:39:38,226] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1871345029239766}, :ns_1@172.23.107.227 {:progress 0.1319648093841642}, :ns_1@172.23.105.187 {:progress 0.1319648093841642}, :ns_1@172.23.107.228 {:progress 0.2}} INFO [2022-01-10 05:39:38,226] jepsen worker nemesis - couchbase.util Rebalance check count: 134 INFO [2022-01-10 05:39:38,226] jepsen worker nemesis - couchbase.util Rebalance stuck count: 101 INFO [2022-01-10 05:39:38,280] jepsen worker 63 - jepsen.util 1053 :info :write [21 19] :RequestTimeoutException INFO [2022-01-10 05:39:38,280] jepsen worker 63 - jepsen.util 1143 :invoke :write [21 32] INFO [2022-01-10 05:39:38,507] jepsen worker 18 - jepsen.util 1008 :ok :read [6 25] INFO [2022-01-10 05:39:38,507] jepsen worker 18 - jepsen.util 1008 :invoke :read [6 nil] INFO [2022-01-10 05:39:38,821] jepsen worker 41 - jepsen.util 1031 :ok :read [13 15] INFO [2022-01-10 05:39:38,821] jepsen worker 41 - jepsen.util 1031 :invoke :write [13 16] INFO [2022-01-10 05:39:38,874] jepsen worker 78 - jepsen.util 1068 :info :write [26 48] :RequestTimeoutException INFO [2022-01-10 05:39:38,875] jepsen worker 78 - jepsen.util 1158 :invoke :write [26 33] INFO [2022-01-10 05:39:38,889] jepsen worker 80 - jepsen.util 1250 :info :write [26 42] :RequestTimeoutException INFO [2022-01-10 05:39:38,890] jepsen worker 80 - jepsen.util 1340 :invoke :read [26 nil] INFO [2022-01-10 05:39:39,042] jepsen worker 47 - jepsen.util 1217 :info :write [15 37] :RequestTimeoutException INFO [2022-01-10 05:39:39,042] jepsen worker 47 - jepsen.util 1307 :invoke :read [15 nil] INFO [2022-01-10 05:39:39,232] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1871345029239766}, :ns_1@172.23.107.227 {:progress 0.1319648093841642}, :ns_1@172.23.105.187 {:progress 0.1319648093841642}, :ns_1@172.23.107.228 {:progress 0.2}} INFO [2022-01-10 05:39:39,232] jepsen worker nemesis - couchbase.util Rebalance check count: 135 INFO [2022-01-10 05:39:39,232] jepsen worker nemesis - couchbase.util Rebalance stuck count: 102 INFO [2022-01-10 05:39:39,326] jepsen worker 16 - jepsen.util 1456 :info :write [5 43] :RequestTimeoutException INFO [2022-01-10 05:39:39,327] jepsen worker 16 - jepsen.util 1546 :invoke :write [5 9] INFO [2022-01-10 05:39:39,414] jepsen worker 75 - jepsen.util 1065 :ok :read [25 :nil] INFO [2022-01-10 05:39:39,416] jepsen worker 75 - jepsen.util 1065 :invoke :read [25 nil] INFO [2022-01-10 05:39:39,416] jepsen worker 79 - jepsen.util 1159 :info :write [26 18] :RequestTimeoutException INFO [2022-01-10 05:39:39,417] jepsen worker 79 - jepsen.util 1249 :invoke :write [26 38] INFO [2022-01-10 05:39:39,555] jepsen worker 50 - jepsen.util 1400 :info :write [16 21] :RequestTimeoutException INFO [2022-01-10 05:39:39,555] jepsen worker 50 - jepsen.util 1490 :invoke :read [16 nil] INFO [2022-01-10 05:39:39,655] jepsen worker 58 - jepsen.util 1228 :info :write [19 23] :RequestTimeoutException INFO [2022-01-10 05:39:39,656] jepsen worker 58 - jepsen.util 1318 :invoke :read [19 nil] INFO [2022-01-10 05:39:39,665] jepsen worker 18 - jepsen.util 1008 :ok :read [6 25] INFO [2022-01-10 05:39:39,665] jepsen worker 18 - jepsen.util 1008 :invoke :write [6 39] WARN [2022-01-10 05:39:39,719] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14308835,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14308835,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9037366,"encode_duration_us":17933,"operation_id":"0x2673","timeout_ms":10000,"last_server_duration_us":9037366,"total_duration_us":10000902},{"operation_name":"upsert","last_dispatch_duration_us":10032810,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13213136,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10896623,"encode_duration_us":17915,"operation_id":"0x27df","timeout_ms":10000,"last_server_duration_us":9633465,"total_duration_us":10010621}],"total_count":2}} WARN [2022-01-10 05:39:39,819] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10032810,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":13213136,"total_server_duration_us":10896623,"encode_duration_us":17915,"operation_id":"0x27df","timeout_ms":10000,"last_server_duration_us":9633465,"total_duration_us":10010621},{"operation_name":"upsert","last_dispatch_duration_us":6162839,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":6162839,"total_server_duration_us":382318,"encode_duration_us":22845,"operation_id":"0x2822","timeout_ms":10000,"last_server_duration_us":382318,"total_duration_us":10009398},{"operation_name":"upsert","last_dispatch_duration_us":4351980,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4351980,"total_server_duration_us":392326,"encode_duration_us":17665,"operation_id":"0x28e6","timeout_ms":10000,"last_server_duration_us":392326,"total_duration_us":10009304},{"operation_name":"upsert","last_dispatch_duration_us":10760510,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10760510,"total_server_duration_us":873759,"encode_duration_us":20042,"operation_id":"0x2727","timeout_ms":10000,"last_server_duration_us":873759,"total_duration_us":10008178},{"operation_name":"upsert","last_dispatch_duration_us":439017,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":1379080,"total_server_duration_us":608191,"encode_duration_us":19252,"operation_id":"0x28d8","timeout_ms":10000,"last_server_duration_us":21649,"total_duration_us":10006552},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":17239,"operation_id":"0x2810","timeout_ms":10000,"total_duration_us":10006098},{"operation_name":"upsert","last_dispatch_duration_us":7347452,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":7347452,"total_server_duration_us":562233,"encode_duration_us":16616,"operation_id":"0x278e","timeout_ms":10000,"last_server_duration_us":562233,"total_duration_us":10005359},{"operation_name":"upsert","last_dispatch_duration_us":8862009,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10337153,"total_server_duration_us":711434,"encode_duration_us":17991,"operation_id":"0x2876","timeout_ms":10000,"last_server_duration_us":52183,"total_duration_us":10004998},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":16293,"operation_id":"0x2880","timeout_ms":10000,"total_duration_us":10004991},{"operation_name":"upsert","last_dispatch_duration_us":149131,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":149131,"total_server_duration_us":127502,"encode_duration_us":22089,"operation_id":"0x28a8","timeout_ms":10000,"last_server_duration_us":127502,"total_duration_us":10004499}],"total_count":12}} INFO [2022-01-10 05:39:39,856] jepsen worker 37 - jepsen.util 1297 :info :write [12 22] :RequestTimeoutException INFO [2022-01-10 05:39:39,856] jepsen worker 37 - jepsen.util 1387 :invoke :read [12 nil] INFO [2022-01-10 05:39:39,881] jepsen worker 71 - jepsen.util 1331 :info :write [23 9] :RequestTimeoutException INFO [2022-01-10 05:39:39,882] jepsen worker 71 - jepsen.util 1421 :invoke :write [23 6] INFO [2022-01-10 05:39:39,897] jepsen worker 26 - jepsen.util 1106 :ok :read [8 :nil] INFO [2022-01-10 05:39:39,897] jepsen worker 76 - jepsen.util 976 :ok :read [25 :nil] INFO [2022-01-10 05:39:39,898] jepsen worker 76 - jepsen.util 976 :invoke :read [25 nil] INFO [2022-01-10 05:39:39,898] jepsen worker 26 - jepsen.util 1106 :invoke :read [8 nil] INFO [2022-01-10 05:39:39,955] jepsen worker 72 - jepsen.util 1332 :info :write [24 4] :RequestTimeoutException INFO [2022-01-10 05:39:39,956] jepsen worker 72 - jepsen.util 1422 :invoke :write [24 28] INFO [2022-01-10 05:39:39,966] jepsen worker 10 - jepsen.util 1360 :info :write [3 3] :RequestTimeoutException INFO [2022-01-10 05:39:39,967] jepsen worker 10 - jepsen.util 1450 :invoke :read [3 nil] INFO [2022-01-10 05:39:39,967] jepsen worker 83 - jepsen.util 1433 :info :write [27 23] :RequestTimeoutException INFO [2022-01-10 05:39:39,967] jepsen worker 83 - jepsen.util 1523 :invoke :write [27 2] INFO [2022-01-10 05:39:40,062] jepsen worker 0 - jepsen.util 1260 :ok :read [0 46] INFO [2022-01-10 05:39:40,063] jepsen worker 0 - jepsen.util 1260 :invoke :read [0 nil] INFO [2022-01-10 05:39:40,242] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1929824561403509}, :ns_1@172.23.107.227 {:progress 0.1378299120234604}, :ns_1@172.23.105.187 {:progress 0.1378299120234604}, :ns_1@172.23.107.228 {:progress 0.2058823529411765}} INFO [2022-01-10 05:39:40,242] jepsen worker nemesis - couchbase.util Rebalance check count: 136 INFO [2022-01-10 05:39:40,242] jepsen worker nemesis - couchbase.util Rebalance stuck count: 102 INFO [2022-01-10 05:39:40,405] jepsen worker 80 - jepsen.util 1340 :ok :read [26 10] INFO [2022-01-10 05:39:40,406] jepsen worker 80 - jepsen.util 1340 :invoke :read [26 nil] INFO [2022-01-10 05:39:40,440] jepsen worker 80 - jepsen.util 1340 :ok :read [26 10] INFO [2022-01-10 05:39:40,441] jepsen worker 80 - jepsen.util 1340 :invoke :write [26 8] INFO [2022-01-10 05:39:40,496] jepsen worker 75 - jepsen.util 1065 :ok :read [25 :nil] INFO [2022-01-10 05:39:40,496] jepsen worker 75 - jepsen.util 1065 :invoke :write [25 40] INFO [2022-01-10 05:39:40,500] jepsen worker 48 - jepsen.util 1038 :ok :read [16 38] INFO [2022-01-10 05:39:40,501] jepsen worker 48 - jepsen.util 1038 :invoke :write [16 41] INFO [2022-01-10 05:39:40,911] jepsen worker 15 - jepsen.util 1005 :ok :read [5 42] INFO [2022-01-10 05:39:40,912] jepsen worker 15 - jepsen.util 1005 :invoke :read [5 nil] INFO [2022-01-10 05:39:41,011] jepsen worker 46 - jepsen.util 1306 :info :write [15 26] :RequestTimeoutException INFO [2022-01-10 05:39:41,012] jepsen worker 46 - jepsen.util 1396 :invoke :write [15 19] INFO [2022-01-10 05:39:41,090] jepsen worker 11 - jepsen.util 1271 :ok :read [3 2] INFO [2022-01-10 05:39:41,091] jepsen worker 54 - jepsen.util 1404 :ok :read [18 14] INFO [2022-01-10 05:39:41,091] jepsen worker 11 - jepsen.util 1271 :invoke :write [3 14] INFO [2022-01-10 05:39:41,092] jepsen worker 54 - jepsen.util 1404 :invoke :write [18 5] INFO [2022-01-10 05:39:41,131] jepsen worker 84 - jepsen.util 1344 :ok :read [28 13] INFO [2022-01-10 05:39:41,131] jepsen worker 84 - jepsen.util 1344 :invoke :write [28 31] INFO [2022-01-10 05:39:41,165] jepsen worker 3 - jepsen.util 1623 :ok :read [1 41] INFO [2022-01-10 05:39:41,165] jepsen worker 3 - jepsen.util 1623 :invoke :write [1 2] INFO [2022-01-10 05:39:41,166] jepsen worker 59 - jepsen.util 1319 :info :write [19 10] :RequestTimeoutException INFO [2022-01-10 05:39:41,167] jepsen worker 59 - jepsen.util 1409 :invoke :read [19 nil] INFO [2022-01-10 05:39:41,175] jepsen worker 56 - jepsen.util 1406 :ok :read [18 14] INFO [2022-01-10 05:39:41,176] jepsen worker 56 - jepsen.util 1406 :invoke :write [18 26] INFO [2022-01-10 05:39:41,188] jepsen worker 10 - jepsen.util 1450 :ok :read [3 2] INFO [2022-01-10 05:39:41,189] jepsen worker 10 - jepsen.util 1450 :invoke :read [3 nil] INFO [2022-01-10 05:39:41,242] jepsen worker 61 - jepsen.util 1231 :info :write [20 38] :RequestTimeoutException INFO [2022-01-10 05:39:41,242] jepsen worker 61 - jepsen.util 1321 :invoke :read [20 nil] INFO [2022-01-10 05:39:41,248] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1929824561403509}, :ns_1@172.23.107.227 {:progress 0.1378299120234604}, :ns_1@172.23.105.187 {:progress 0.1378299120234604}, :ns_1@172.23.107.228 {:progress 0.2058823529411765}} INFO [2022-01-10 05:39:41,248] jepsen worker nemesis - couchbase.util Rebalance check count: 137 INFO [2022-01-10 05:39:41,248] jepsen worker nemesis - couchbase.util Rebalance stuck count: 103 INFO [2022-01-10 05:39:41,256] jepsen worker 9 - jepsen.util 1449 :info :write [3 26] :RequestTimeoutException INFO [2022-01-10 05:39:41,256] jepsen worker 9 - jepsen.util 1539 :invoke :read [3 nil] INFO [2022-01-10 05:39:41,297] jepsen worker 77 - jepsen.util 1517 :info :write [25 7] :RequestTimeoutException INFO [2022-01-10 05:39:41,297] jepsen worker 77 - jepsen.util 1607 :invoke :write [25 0] INFO [2022-01-10 05:39:41,316] jepsen worker 82 - jepsen.util 1252 :ok :read [27 30] INFO [2022-01-10 05:39:41,317] jepsen worker 82 - jepsen.util 1252 :invoke :read [27 nil] WARN [2022-01-10 05:39:41,320] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9942040,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12666428,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9829401,"encode_duration_us":17311,"operation_id":"0x28b3","timeout_ms":10000,"last_server_duration_us":9063917,"total_duration_us":10001456},{"operation_name":"upsert","last_dispatch_duration_us":9033335,"last_local_id":"EFE9906200000006/0000000021E2C07D","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":9813542,"last_local_socket":"172.23.104.79:44302","total_server_duration_us":9433657,"encode_duration_us":21302,"operation_id":"0x2821","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10005922},{"operation_name":"upsert","last_dispatch_duration_us":9768292,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12673036,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9720011,"encode_duration_us":18834,"operation_id":"0x28b2","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10004691}],"total_count":3}} WARN [2022-01-10 05:39:41,320] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":6556296,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9246683,"total_server_duration_us":1208758,"encode_duration_us":16966,"operation_id":"0x287f","timeout_ms":10000,"last_server_duration_us":200008,"total_duration_us":10010475},{"operation_name":"upsert","last_dispatch_duration_us":9139623,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":12732816,"total_server_duration_us":913036,"encode_duration_us":17345,"operation_id":"0x28b1","timeout_ms":10000,"last_server_duration_us":38274,"total_duration_us":10010230},{"operation_name":"upsert","last_dispatch_duration_us":30884,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":903855,"total_server_duration_us":509873,"encode_duration_us":22394,"operation_id":"0x27bb","timeout_ms":10000,"last_server_duration_us":20841,"total_duration_us":10010123},{"operation_name":"upsert","last_dispatch_duration_us":1218349,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11143074,"total_server_duration_us":1253520,"encode_duration_us":16832,"operation_id":"0x2800","timeout_ms":10000,"last_server_duration_us":479354,"total_duration_us":10007682},{"operation_name":"upsert","last_dispatch_duration_us":5130791,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":13327120,"total_server_duration_us":2471678,"encode_duration_us":20787,"operation_id":"0x278f","timeout_ms":10000,"last_server_duration_us":398778,"total_duration_us":10007450},{"operation_name":"upsert","last_dispatch_duration_us":9033335,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9813542,"total_server_duration_us":9433657,"encode_duration_us":21302,"operation_id":"0x2821","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10005922},{"operation_name":"upsert","last_dispatch_duration_us":9768292,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":12673036,"total_server_duration_us":9720011,"encode_duration_us":18834,"operation_id":"0x28b2","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10004691},{"operation_name":"upsert","last_dispatch_duration_us":8484829,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9449847,"total_server_duration_us":631963,"encode_duration_us":17123,"operation_id":"0x278c","timeout_ms":10000,"last_server_duration_us":55303,"total_duration_us":10004267},{"operation_name":"upsert","last_dispatch_duration_us":3185950,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7969029,"total_server_duration_us":1249288,"encode_duration_us":17435,"operation_id":"0x275f","timeout_ms":10000,"last_server_duration_us":369962,"total_duration_us":10002938},{"operation_name":"upsert","last_dispatch_duration_us":4628146,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11536528,"total_server_duration_us":2763406,"encode_duration_us":16655,"operation_id":"0x2792","timeout_ms":10000,"last_server_duration_us":589548,"total_duration_us":10002243}],"total_count":23}} INFO [2022-01-10 05:39:41,341] jepsen worker 86 - jepsen.util 986 :info :write [28 8] :RequestTimeoutException INFO [2022-01-10 05:39:41,342] jepsen worker 86 - jepsen.util 1076 :invoke :read [28 nil] WARN [2022-01-10 05:39:41,346] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16142965,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17574769,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9512018,"encode_duration_us":17329,"operation_id":"0x2648","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10010664}],"total_count":1}} WARN [2022-01-10 05:39:41,347] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2839102,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":2839102,"total_server_duration_us":363714,"encode_duration_us":18008,"operation_id":"0x28e2","timeout_ms":10000,"last_server_duration_us":363714,"total_duration_us":10009948},{"operation_name":"upsert","last_dispatch_duration_us":9086228,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":11064789,"total_server_duration_us":9841523,"encode_duration_us":19230,"operation_id":"0x2895","timeout_ms":10000,"last_server_duration_us":9021452,"total_duration_us":10008642},{"operation_name":"upsert","last_dispatch_duration_us":5486849,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":12196884,"total_server_duration_us":1235461,"encode_duration_us":18383,"operation_id":"0x27bd","timeout_ms":10000,"last_server_duration_us":656918,"total_duration_us":10008132},{"operation_name":"upsert","last_dispatch_duration_us":3025201,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":7103078,"total_server_duration_us":930455,"encode_duration_us":25577,"operation_id":"0x27dd","timeout_ms":10000,"last_server_duration_us":177090,"total_duration_us":10007822},{"operation_name":"upsert","last_dispatch_duration_us":2011824,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9530847,"total_server_duration_us":1501086,"encode_duration_us":28152,"operation_id":"0x2814","timeout_ms":10000,"last_server_duration_us":213417,"total_duration_us":10006896},{"operation_name":"upsert","last_dispatch_duration_us":1009086,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8467587,"total_server_duration_us":1230434,"encode_duration_us":16399,"operation_id":"0x287e","timeout_ms":10000,"last_server_duration_us":458844,"total_duration_us":10003382},{"operation_name":"upsert","last_dispatch_duration_us":2081630,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9137249,"total_server_duration_us":1924591,"encode_duration_us":16542,"operation_id":"0x281c","timeout_ms":10000,"last_server_duration_us":60128,"total_duration_us":10002299},{"operation_name":"upsert","last_dispatch_duration_us":7858934,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12002940,"total_server_duration_us":1348224,"encode_duration_us":16289,"operation_id":"0x284a","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10001801},{"operation_name":"upsert","last_dispatch_duration_us":7819639,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10529966,"total_server_duration_us":1100089,"encode_duration_us":17147,"operation_id":"0x28a4","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10001661},{"operation_name":"upsert","last_dispatch_duration_us":782144,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7649587,"total_server_duration_us":955759,"encode_duration_us":25885,"operation_id":"0x283b","timeout_ms":10000,"last_server_duration_us":64343,"total_duration_us":10001601}],"total_count":18}} WARN [2022-01-10 05:39:41,352] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10693764,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10693764,"total_server_duration_us":38,"operation_id":"0x2754","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":10697346},{"operation_name":"get","last_dispatch_duration_us":10104423,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10104423,"total_server_duration_us":9513,"operation_id":"0x2783","timeout_ms":30000,"last_server_duration_us":9513,"total_duration_us":10104819},{"operation_name":"upsert","last_dispatch_duration_us":3166033,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7293899,"total_server_duration_us":219275,"encode_duration_us":19935,"operation_id":"0x28db","timeout_ms":10000,"last_server_duration_us":52420,"total_duration_us":10008457},{"operation_name":"upsert","last_dispatch_duration_us":2829771,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12629239,"total_server_duration_us":170618,"encode_duration_us":17708,"operation_id":"0x27ab","timeout_ms":10000,"last_server_duration_us":90931,"total_duration_us":10008365},{"operation_name":"upsert","last_dispatch_duration_us":42021,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8600088,"total_server_duration_us":209423,"encode_duration_us":21048,"operation_id":"0x282e","timeout_ms":10000,"last_server_duration_us":23559,"total_duration_us":10006645},{"operation_name":"upsert","last_dispatch_duration_us":38895,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":4500923,"total_server_duration_us":242052,"encode_duration_us":16924,"operation_id":"0x290c","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10005626},{"operation_name":"upsert","last_dispatch_duration_us":90985,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":5963935,"total_server_duration_us":59426,"encode_duration_us":17110,"operation_id":"0x28a3","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10003110},{"operation_name":"upsert","last_dispatch_duration_us":2256759,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8798576,"total_server_duration_us":204681,"encode_duration_us":16679,"operation_id":"0x2937","timeout_ms":10000,"last_server_duration_us":45390,"total_duration_us":10001709},{"operation_name":"get","last_dispatch_duration_us":9007092,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9007092,"total_server_duration_us":1085125,"operation_id":"0x287d","timeout_ms":30000,"last_server_duration_us":1085125,"total_duration_us":9010287},{"operation_name":"get","last_dispatch_duration_us":8617801,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8617801,"total_server_duration_us":32,"operation_id":"0x2813","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":8619849}],"total_count":24}} WARN [2022-01-10 05:39:41,353] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":32224,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":3190296,"total_server_duration_us":971398,"encode_duration_us":16386,"operation_id":"0x290f","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10010047},{"operation_name":"upsert","last_dispatch_duration_us":1827924,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":2702024,"total_server_duration_us":532857,"encode_duration_us":21397,"operation_id":"0x28e4","timeout_ms":10000,"last_server_duration_us":21977,"total_duration_us":10009832},{"operation_name":"upsert","last_dispatch_duration_us":1795560,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11328003,"total_server_duration_us":484786,"encode_duration_us":17928,"operation_id":"0x27b3","timeout_ms":10000,"last_server_duration_us":276939,"total_duration_us":10007955},{"operation_name":"upsert","last_dispatch_duration_us":7231964,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7231964,"total_server_duration_us":596212,"encode_duration_us":17997,"operation_id":"0x293f","timeout_ms":10000,"last_server_duration_us":596212,"total_duration_us":10005218},{"operation_name":"upsert","last_dispatch_duration_us":33319,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":2293153,"total_server_duration_us":886299,"encode_duration_us":19012,"operation_id":"0x2812","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10004966},{"operation_name":"upsert","last_dispatch_duration_us":4812610,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7451964,"total_server_duration_us":1144886,"encode_duration_us":17398,"operation_id":"0x2881","timeout_ms":10000,"last_server_duration_us":776014,"total_duration_us":10003454},{"operation_name":"upsert","last_dispatch_duration_us":1769713,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":1769713,"total_server_duration_us":366968,"encode_duration_us":16658,"operation_id":"0x2910","timeout_ms":10000,"last_server_duration_us":366968,"total_duration_us":10003215},{"operation_name":"upsert","last_dispatch_duration_us":34710,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":3248561,"total_server_duration_us":654851,"encode_duration_us":26111,"operation_id":"0x290a","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10001696},{"operation_name":"upsert","last_dispatch_duration_us":781610,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10186246,"total_server_duration_us":478390,"encode_duration_us":17616,"operation_id":"0x2790","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10001024},{"operation_name":"upsert","last_dispatch_duration_us":820959,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9983968,"total_server_duration_us":123332,"encode_duration_us":28336,"operation_id":"0x27b4","timeout_ms":10000,"last_server_duration_us":56770,"total_duration_us":10000909}],"total_count":23}} WARN [2022-01-10 05:39:41,447] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9086228,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11064789,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9841523,"encode_duration_us":19230,"operation_id":"0x2895","timeout_ms":10000,"last_server_duration_us":9021452,"total_duration_us":10008642},{"operation_name":"upsert","last_dispatch_duration_us":9071091,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":17866268,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":9065414,"encode_duration_us":17582,"operation_id":"0x26ef","timeout_ms":10000,"last_server_duration_us":9046921,"total_duration_us":10008905}],"total_count":2}} WARN [2022-01-10 05:39:41,452] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17366042,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":19926227,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":10399894,"encode_duration_us":17772,"operation_id":"0x24bd","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10006901}],"total_count":1}} WARN [2022-01-10 05:39:41,546] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":773108,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7517425,"total_server_duration_us":595449,"encode_duration_us":17209,"operation_id":"0x280e","timeout_ms":10000,"last_server_duration_us":61774,"total_duration_us":10010322},{"operation_name":"upsert","last_dispatch_duration_us":1431790,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10065388,"total_server_duration_us":2582936,"encode_duration_us":21169,"operation_id":"0x2807","timeout_ms":10000,"last_server_duration_us":585233,"total_duration_us":10010197},{"operation_name":"upsert","last_dispatch_duration_us":373626,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8211120,"total_server_duration_us":963080,"encode_duration_us":17636,"operation_id":"0x28b5","timeout_ms":10000,"last_server_duration_us":92438,"total_duration_us":10008658},{"operation_name":"upsert","last_dispatch_duration_us":637750,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":6600972,"total_server_duration_us":918309,"encode_duration_us":18366,"operation_id":"0x27ed","timeout_ms":10000,"last_server_duration_us":246819,"total_duration_us":10007874},{"operation_name":"upsert","last_dispatch_duration_us":1308090,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7630507,"total_server_duration_us":874481,"encode_duration_us":16334,"operation_id":"0x28a2","timeout_ms":10000,"last_server_duration_us":44389,"total_duration_us":10006466},{"operation_name":"upsert","last_dispatch_duration_us":45172,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":45172,"total_server_duration_us":33721,"encode_duration_us":16998,"operation_id":"0x2875","timeout_ms":10000,"last_server_duration_us":33721,"total_duration_us":10005973},{"operation_name":"upsert","last_dispatch_duration_us":52078,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8519324,"total_server_duration_us":242172,"encode_duration_us":17151,"operation_id":"0x281b","timeout_ms":10000,"last_server_duration_us":40376,"total_duration_us":10005433},{"operation_name":"upsert","last_dispatch_duration_us":13195666,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13799147,"total_server_duration_us":241641,"encode_duration_us":17195,"operation_id":"0x2760","timeout_ms":10000,"last_server_duration_us":202112,"total_duration_us":10004798},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":28894,"operation_id":"0x283c","timeout_ms":10000,"total_duration_us":10004510},{"operation_name":"upsert","last_dispatch_duration_us":927357,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8333253,"total_server_duration_us":1133987,"encode_duration_us":16457,"operation_id":"0x2843","timeout_ms":10000,"last_server_duration_us":89584,"total_duration_us":10001813}],"total_count":18}} INFO [2022-01-10 05:39:41,592] jepsen worker 39 - jepsen.util 1389 :info :write [13 9] :RequestTimeoutException INFO [2022-01-10 05:39:41,592] jepsen worker 39 - jepsen.util 1479 :invoke :write [13 49] INFO [2022-01-10 05:39:41,759] jepsen worker 6 - jepsen.util 1266 :info :write [2 42] :RequestTimeoutException INFO [2022-01-10 05:39:41,760] jepsen worker 6 - jepsen.util 1356 :invoke :read [2 nil] INFO [2022-01-10 05:39:41,819] jepsen worker 65 - jepsen.util 1235 :info :write [21 3] :RequestTimeoutException INFO [2022-01-10 05:39:41,820] jepsen worker 65 - jepsen.util 1325 :invoke :read [21 nil] INFO [2022-01-10 05:39:42,009] jepsen worker 76 - jepsen.util 976 :ok :read [25 :nil] INFO [2022-01-10 05:39:42,009] jepsen worker 26 - jepsen.util 1106 :ok :read [8 :nil] INFO [2022-01-10 05:39:42,010] jepsen worker 76 - jepsen.util 976 :invoke :read [25 nil] INFO [2022-01-10 05:39:42,010] jepsen worker 26 - jepsen.util 1106 :invoke :read [8 nil] INFO [2022-01-10 05:39:42,196] jepsen worker 73 - jepsen.util 1243 :info :write [24 31] :RequestTimeoutException INFO [2022-01-10 05:39:42,197] jepsen worker 73 - jepsen.util 1333 :invoke :read [24 nil] INFO [2022-01-10 05:39:42,202] jepsen worker 21 - jepsen.util 1461 :info :write [7 35] :RequestTimeoutException INFO [2022-01-10 05:39:42,202] jepsen worker 21 - jepsen.util 1551 :invoke :read [7 nil] INFO [2022-01-10 05:39:42,228] jepsen worker 36 - jepsen.util 936 :ok :read [12 :nil] INFO [2022-01-10 05:39:42,228] jepsen worker 36 - jepsen.util 936 :invoke :read [12 nil] INFO [2022-01-10 05:39:42,254] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1929824561403509}, :ns_1@172.23.107.227 {:progress 0.1378299120234604}, :ns_1@172.23.105.187 {:progress 0.1378299120234604}, :ns_1@172.23.107.228 {:progress 0.2058823529411765}} INFO [2022-01-10 05:39:42,254] jepsen worker nemesis - couchbase.util Rebalance check count: 138 INFO [2022-01-10 05:39:42,254] jepsen worker nemesis - couchbase.util Rebalance stuck count: 104 INFO [2022-01-10 05:39:42,350] jepsen worker 51 - jepsen.util 1311 :info :write [17 8] :RequestTimeoutException INFO [2022-01-10 05:39:42,351] jepsen worker 51 - jepsen.util 1401 :invoke :read [17 nil] INFO [2022-01-10 05:39:42,555] jepsen worker 85 - jepsen.util 1435 :info :write [28 22] :RequestTimeoutException INFO [2022-01-10 05:39:42,556] jepsen worker 85 - jepsen.util 1525 :invoke :read [28 nil] INFO [2022-01-10 05:39:42,631] jepsen worker 20 - jepsen.util 1010 :info :write [6 48] :RequestTimeoutException INFO [2022-01-10 05:39:42,632] jepsen worker 20 - jepsen.util 1100 :invoke :write [6 11] INFO [2022-01-10 05:39:42,637] jepsen worker 36 - jepsen.util 936 :ok :read [12 :nil] INFO [2022-01-10 05:39:42,637] jepsen worker 36 - jepsen.util 936 :invoke :write [12 33] INFO [2022-01-10 05:39:42,821] jepsen worker 34 - jepsen.util 1294 :info :write [11 19] :RequestTimeoutException INFO [2022-01-10 05:39:42,821] jepsen worker 34 - jepsen.util 1384 :invoke :read [11 nil] INFO [2022-01-10 05:39:42,997] jepsen worker 76 - jepsen.util 976 :ok :read [25 :nil] INFO [2022-01-10 05:39:42,997] jepsen worker 26 - jepsen.util 1106 :ok :read [8 :nil] INFO [2022-01-10 05:39:42,997] jepsen worker 76 - jepsen.util 976 :invoke :read [25 nil] INFO [2022-01-10 05:39:42,998] jepsen worker 26 - jepsen.util 1106 :invoke :write [8 37] INFO [2022-01-10 05:39:43,022] jepsen worker 35 - jepsen.util 1295 :info :write [11 39] :RequestTimeoutException INFO [2022-01-10 05:39:43,023] jepsen worker 35 - jepsen.util 1385 :invoke :read [11 nil] INFO [2022-01-10 05:39:43,262] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1929824561403509}, :ns_1@172.23.107.227 {:progress 0.1378299120234604}, :ns_1@172.23.105.187 {:progress 0.1378299120234604}, :ns_1@172.23.107.228 {:progress 0.2058823529411765}} INFO [2022-01-10 05:39:43,262] jepsen worker nemesis - couchbase.util Rebalance check count: 139 INFO [2022-01-10 05:39:43,262] jepsen worker nemesis - couchbase.util Rebalance stuck count: 105 INFO [2022-01-10 05:39:43,306] jepsen worker 53 - jepsen.util 1223 :info :write [17 35] :RequestTimeoutException INFO [2022-01-10 05:39:43,306] jepsen worker 53 - jepsen.util 1313 :invoke :write [17 49] INFO [2022-01-10 05:39:43,393] jepsen worker 73 - jepsen.util 1333 :ok :read [24 18] INFO [2022-01-10 05:39:43,393] jepsen worker 73 - jepsen.util 1333 :invoke :read [24 nil] INFO [2022-01-10 05:39:43,445] jepsen worker 28 - jepsen.util 1198 :info :write [9 31] :RequestTimeoutException INFO [2022-01-10 05:39:43,445] jepsen worker 28 - jepsen.util 1288 :invoke :read [9 nil] INFO [2022-01-10 05:39:43,778] jepsen worker 50 - jepsen.util 1490 :ok :read [16 38] INFO [2022-01-10 05:39:43,779] jepsen worker 50 - jepsen.util 1490 :invoke :write [16 31] INFO [2022-01-10 05:39:43,781] jepsen worker 66 - jepsen.util 1146 :ok :read [22 41] INFO [2022-01-10 05:39:43,781] jepsen worker 66 - jepsen.util 1146 :invoke :write [22 43] INFO [2022-01-10 05:39:43,846] jepsen worker 27 - jepsen.util 1197 :info :write [9 42] :RequestTimeoutException INFO [2022-01-10 05:39:43,847] jepsen worker 27 - jepsen.util 1287 :invoke :write [9 42] INFO [2022-01-10 05:39:43,865] jepsen worker 68 - jepsen.util 1238 :info :write [22 3] :RequestTimeoutException INFO [2022-01-10 05:39:43,866] jepsen worker 68 - jepsen.util 1328 :invoke :read [22 nil] INFO [2022-01-10 05:39:43,922] jepsen worker 88 - jepsen.util 1348 :info :write [29 37] :RequestTimeoutException INFO [2022-01-10 05:39:43,923] jepsen worker 88 - jepsen.util 1438 :invoke :read [29 nil] INFO [2022-01-10 05:39:44,013] jepsen worker 76 - jepsen.util 976 :ok :read [25 :nil] INFO [2022-01-10 05:39:44,013] jepsen worker 76 - jepsen.util 976 :invoke :read [25 nil] INFO [2022-01-10 05:39:44,254] jepsen worker 35 - jepsen.util 1385 :ok :read [11 21] INFO [2022-01-10 05:39:44,255] jepsen worker 35 - jepsen.util 1385 :invoke :read [11 nil] INFO [2022-01-10 05:39:44,267] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1988304093567251}, :ns_1@172.23.107.227 {:progress 0.1436950146627566}, :ns_1@172.23.105.187 {:progress 0.1436950146627566}, :ns_1@172.23.107.228 {:progress 0.211764705882353}} INFO [2022-01-10 05:39:44,268] jepsen worker nemesis - couchbase.util Rebalance check count: 140 INFO [2022-01-10 05:39:44,268] jepsen worker nemesis - couchbase.util Rebalance stuck count: 105 INFO [2022-01-10 05:39:44,466] jepsen worker 64 - jepsen.util 1324 :info :write [21 47] :RequestTimeoutException INFO [2022-01-10 05:39:44,466] jepsen worker 64 - jepsen.util 1414 :invoke :write [21 22] INFO [2022-01-10 05:39:44,760] jepsen worker 23 - jepsen.util 1283 :info :write [7 46] :RequestTimeoutException INFO [2022-01-10 05:39:44,760] jepsen worker 23 - jepsen.util 1373 :invoke :write [7 47] INFO [2022-01-10 05:39:44,831] jepsen worker 43 - jepsen.util 1393 :info :write [14 46] :RequestTimeoutException INFO [2022-01-10 05:39:44,837] jepsen worker 43 - jepsen.util 1483 :invoke :write [14 2] INFO [2022-01-10 05:39:44,857] jepsen worker 10 - jepsen.util 1450 :ok :read [3 2] INFO [2022-01-10 05:39:44,858] jepsen worker 32 - jepsen.util 1292 :ok :read [10 37] INFO [2022-01-10 05:39:44,858] jepsen worker 10 - jepsen.util 1450 :invoke :write [3 21] INFO [2022-01-10 05:39:44,858] jepsen worker 32 - jepsen.util 1292 :invoke :read [10 nil] INFO [2022-01-10 05:39:44,864] jepsen worker 70 - jepsen.util 1690 :ok :read [23 26] INFO [2022-01-10 05:39:44,865] jepsen worker 70 - jepsen.util 1690 :invoke :read [23 nil] INFO [2022-01-10 05:39:44,875] jepsen worker 4 - jepsen.util 1264 :info :write [1 9] :RequestTimeoutException INFO [2022-01-10 05:39:44,876] jepsen worker 4 - jepsen.util 1354 :invoke :read [1 nil] INFO [2022-01-10 05:39:44,916] jepsen worker 42 - jepsen.util 1212 :info :write [14 15] :RequestTimeoutException INFO [2022-01-10 05:39:44,916] jepsen worker 60 - jepsen.util 1590 :info :write [20 33] :RequestTimeoutException INFO [2022-01-10 05:39:44,917] jepsen worker 42 - jepsen.util 1302 :invoke :read [14 nil] INFO [2022-01-10 05:39:44,917] jepsen worker 60 - jepsen.util 1680 :invoke :read [20 nil] INFO [2022-01-10 05:39:45,082] jepsen worker 74 - jepsen.util 1604 :info :write [24 1] :RequestTimeoutException INFO [2022-01-10 05:39:45,083] jepsen worker 74 - jepsen.util 1694 :invoke :write [24 20] INFO [2022-01-10 05:39:45,274] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1988304093567251}, :ns_1@172.23.107.227 {:progress 0.1436950146627566}, :ns_1@172.23.105.187 {:progress 0.1436950146627566}, :ns_1@172.23.107.228 {:progress 0.211764705882353}} INFO [2022-01-10 05:39:45,274] jepsen worker nemesis - couchbase.util Rebalance check count: 141 INFO [2022-01-10 05:39:45,274] jepsen worker nemesis - couchbase.util Rebalance stuck count: 106 INFO [2022-01-10 05:39:45,327] jepsen worker 7 - jepsen.util 1267 :info :write [2 40] :RequestTimeoutException INFO [2022-01-10 05:39:45,327] jepsen worker 7 - jepsen.util 1357 :invoke :read [2 nil] INFO [2022-01-10 05:39:45,415] jepsen worker 52 - jepsen.util 1402 :info :write [17 39] :RequestTimeoutException INFO [2022-01-10 05:39:45,416] jepsen worker 52 - jepsen.util 1492 :invoke :read [17 nil] INFO [2022-01-10 05:39:45,419] jepsen worker 45 - jepsen.util 1485 :info :write [15 23] :RequestTimeoutException INFO [2022-01-10 05:39:45,420] jepsen worker 45 - jepsen.util 1575 :invoke :read [15 nil] INFO [2022-01-10 05:39:45,516] jepsen worker 33 - jepsen.util 1293 :info :write [11 33] :RequestTimeoutException INFO [2022-01-10 05:39:45,517] jepsen worker 33 - jepsen.util 1383 :invoke :read [11 nil] INFO [2022-01-10 05:39:45,657] jepsen worker 73 - jepsen.util 1333 :ok :read [24 18] INFO [2022-01-10 05:39:45,657] jepsen worker 73 - jepsen.util 1333 :invoke :read [24 nil] INFO [2022-01-10 05:39:45,729] jepsen worker 88 - jepsen.util 1438 :ok :read [29 33] INFO [2022-01-10 05:39:45,729] jepsen worker 88 - jepsen.util 1438 :invoke :read [29 nil] INFO [2022-01-10 05:39:45,740] jepsen worker 32 - jepsen.util 1292 :ok :read [10 37] INFO [2022-01-10 05:39:45,741] jepsen worker 32 - jepsen.util 1292 :invoke :write [10 43] INFO [2022-01-10 05:39:45,751] jepsen worker 70 - jepsen.util 1690 :ok :read [23 26] INFO [2022-01-10 05:39:45,752] jepsen worker 70 - jepsen.util 1690 :invoke :write [23 7] INFO [2022-01-10 05:39:45,782] jepsen worker 22 - jepsen.util 1192 :info :write [7 30] :RequestTimeoutException INFO [2022-01-10 05:39:45,783] jepsen worker 22 - jepsen.util 1282 :invoke :read [7 nil] INFO [2022-01-10 05:39:45,786] jepsen worker 55 - jepsen.util 1405 :info :write [18 22] :RequestTimeoutException INFO [2022-01-10 05:39:45,787] jepsen worker 55 - jepsen.util 1495 :invoke :write [18 21] INFO [2022-01-10 05:39:45,813] jepsen worker 76 - jepsen.util 976 :ok :read [25 :nil] INFO [2022-01-10 05:39:45,814] jepsen worker 76 - jepsen.util 976 :invoke :write [25 10] INFO [2022-01-10 05:39:45,821] jepsen worker 19 - jepsen.util 1369 :info :write [6 15] :RequestTimeoutException INFO [2022-01-10 05:39:45,822] jepsen worker 19 - jepsen.util 1459 :invoke :write [6 11] INFO [2022-01-10 05:39:45,860] jepsen worker 1 - jepsen.util 1351 :ok :read [0 46] INFO [2022-01-10 05:39:45,861] jepsen worker 1 - jepsen.util 1351 :invoke :write [0 10] INFO [2022-01-10 05:39:45,862] jepsen worker 0 - jepsen.util 1260 :ok :read [0 46] INFO [2022-01-10 05:39:45,863] jepsen worker 0 - jepsen.util 1260 :invoke :read [0 nil] INFO [2022-01-10 05:39:45,884] jepsen worker 5 - jepsen.util 1265 :info :write [1 47] :RequestTimeoutException INFO [2022-01-10 05:39:45,885] jepsen worker 5 - jepsen.util 1355 :invoke :read [1 nil] INFO [2022-01-10 05:39:45,924] jepsen worker 4 - jepsen.util 1354 :ok :read [1 41] INFO [2022-01-10 05:39:45,924] jepsen worker 4 - jepsen.util 1354 :invoke :write [1 28] INFO [2022-01-10 05:39:46,281] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1988304093567251}, :ns_1@172.23.107.227 {:progress 0.1436950146627566}, :ns_1@172.23.105.187 {:progress 0.1436950146627566}, :ns_1@172.23.107.228 {:progress 0.211764705882353}} INFO [2022-01-10 05:39:46,281] jepsen worker nemesis - couchbase.util Rebalance check count: 142 INFO [2022-01-10 05:39:46,281] jepsen worker nemesis - couchbase.util Rebalance stuck count: 107 INFO [2022-01-10 05:39:46,333] jepsen worker 31 - jepsen.util 1651 :ok :read [10 37] INFO [2022-01-10 05:39:46,334] jepsen worker 31 - jepsen.util 1651 :invoke :read [10 nil] INFO [2022-01-10 05:39:46,372] jepsen worker 67 - jepsen.util 1237 :info :write [22 29] :RequestTimeoutException INFO [2022-01-10 05:39:46,373] jepsen worker 67 - jepsen.util 1327 :invoke :write [22 28] INFO [2022-01-10 05:39:46,378] jepsen worker 2 - jepsen.util 1262 :info :write [0 11] :RequestTimeoutException INFO [2022-01-10 05:39:46,378] jepsen worker 2 - jepsen.util 1352 :invoke :write [0 42] INFO [2022-01-10 05:39:46,754] jepsen worker 9 - jepsen.util 1539 :ok :read [3 2] INFO [2022-01-10 05:39:46,755] jepsen worker 9 - jepsen.util 1539 :invoke :write [3 27] INFO [2022-01-10 05:39:47,184] jepsen worker 58 - jepsen.util 1318 :fail :read [19 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:47,184] jepsen worker 58 - jepsen.util 1318 :invoke :read [19 nil] INFO [2022-01-10 05:39:47,227] jepsen worker 82 - jepsen.util 1252 :ok :read [27 30] INFO [2022-01-10 05:39:47,227] jepsen worker 82 - jepsen.util 1252 :invoke :read [27 nil] INFO [2022-01-10 05:39:47,240] jepsen worker 44 - jepsen.util 1304 :info :write [14 48] :RequestTimeoutException INFO [2022-01-10 05:39:47,240] jepsen worker 44 - jepsen.util 1394 :invoke :write [14 35] INFO [2022-01-10 05:39:47,242] jepsen worker 86 - jepsen.util 1076 :ok :read [28 13] INFO [2022-01-10 05:39:47,242] jepsen worker 86 - jepsen.util 1076 :invoke :write [28 4] INFO [2022-01-10 05:39:47,246] jepsen worker 89 - jepsen.util 1439 :info :write [29 8] :RequestTimeoutException INFO [2022-01-10 05:39:47,247] jepsen worker 89 - jepsen.util 1529 :invoke :write [29 35] INFO [2022-01-10 05:39:47,288] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1988304093567251}, :ns_1@172.23.107.227 {:progress 0.1436950146627566}, :ns_1@172.23.105.187 {:progress 0.1436950146627566}, :ns_1@172.23.107.228 {:progress 0.211764705882353}} INFO [2022-01-10 05:39:47,288] jepsen worker nemesis - couchbase.util Rebalance check count: 143 INFO [2022-01-10 05:39:47,288] jepsen worker nemesis - couchbase.util Rebalance stuck count: 108 INFO [2022-01-10 05:39:47,444] jepsen worker 85 - jepsen.util 1525 :ok :read [28 13] INFO [2022-01-10 05:39:47,444] jepsen worker 85 - jepsen.util 1525 :invoke :write [28 45] INFO [2022-01-10 05:39:47,492] jepsen worker 13 - jepsen.util 1273 :fail :read [4 nil] :UnambiguousTimeoutException INFO [2022-01-10 05:39:47,493] jepsen worker 13 - jepsen.util 1273 :invoke :read [4 nil] INFO [2022-01-10 05:39:47,591] jepsen worker 25 - jepsen.util 1285 :ok :read [8 :nil] INFO [2022-01-10 05:39:47,592] jepsen worker 25 - jepsen.util 1285 :invoke :write [8 5] INFO [2022-01-10 05:39:47,612] jepsen worker 62 - jepsen.util 1412 :info :write [20 14] :RequestTimeoutException INFO [2022-01-10 05:39:47,613] jepsen worker 62 - jepsen.util 1502 :invoke :write [20 43] INFO [2022-01-10 05:39:47,631] jepsen worker 73 - jepsen.util 1333 :ok :read [24 18] INFO [2022-01-10 05:39:47,632] jepsen worker 73 - jepsen.util 1333 :invoke :write [24 20] INFO [2022-01-10 05:39:47,672] jepsen worker 15 - jepsen.util 1005 :ok :read [5 42] INFO [2022-01-10 05:39:47,672] jepsen worker 15 - jepsen.util 1005 :invoke :read [5 nil] INFO [2022-01-10 05:39:47,748] jepsen worker 21 - jepsen.util 1551 :ok :read [7 :nil] INFO [2022-01-10 05:39:47,749] jepsen worker 21 - jepsen.util 1551 :invoke :read [7 nil] INFO [2022-01-10 05:39:47,832] jepsen worker 57 - jepsen.util 1317 :info :write [19 21] :RequestTimeoutException INFO [2022-01-10 05:39:47,832] jepsen worker 57 - jepsen.util 1407 :invoke :read [19 nil] INFO [2022-01-10 05:39:47,925] jepsen worker 45 - jepsen.util 1575 :ok :read [15 23] INFO [2022-01-10 05:39:47,926] jepsen worker 45 - jepsen.util 1575 :invoke :read [15 nil] INFO [2022-01-10 05:39:47,937] jepsen worker 47 - jepsen.util 1307 :ok :read [15 23] INFO [2022-01-10 05:39:47,938] jepsen worker 47 - jepsen.util 1307 :invoke :read [15 nil] INFO [2022-01-10 05:39:47,941] jepsen worker 7 - jepsen.util 1357 :ok :read [2 14] INFO [2022-01-10 05:39:47,941] jepsen worker 7 - jepsen.util 1357 :invoke :read [2 nil] INFO [2022-01-10 05:39:48,125] jepsen worker 24 - jepsen.util 1194 :info :write [8 10] :RequestTimeoutException INFO [2022-01-10 05:39:48,128] jepsen worker 24 - jepsen.util 1284 :invoke :read [8 nil] INFO [2022-01-10 05:39:48,206] jepsen worker 87 - jepsen.util 1527 :info :write [29 36] :RequestTimeoutException INFO [2022-01-10 05:39:48,207] jepsen worker 87 - jepsen.util 1617 :invoke :read [29 nil] INFO [2022-01-10 05:39:48,209] jepsen worker 30 - jepsen.util 1020 :info :write [10 26] :RequestTimeoutException INFO [2022-01-10 05:39:48,210] jepsen worker 30 - jepsen.util 1110 :invoke :read [10 nil] INFO [2022-01-10 05:39:48,211] jepsen worker 69 - jepsen.util 1329 :info :write [23 6] :RequestTimeoutException INFO [2022-01-10 05:39:48,212] jepsen worker 69 - jepsen.util 1419 :invoke :write [23 27] INFO [2022-01-10 05:39:48,230] jepsen worker 8 - jepsen.util 1268 :info :write [2 48] :RequestTimeoutException INFO [2022-01-10 05:39:48,231] jepsen worker 8 - jepsen.util 1358 :invoke :write [2 8] INFO [2022-01-10 05:39:48,297] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1988304093567251}, :ns_1@172.23.107.227 {:progress 0.1436950146627566}, :ns_1@172.23.105.187 {:progress 0.1436950146627566}, :ns_1@172.23.107.228 {:progress 0.211764705882353}} INFO [2022-01-10 05:39:48,297] jepsen worker nemesis - couchbase.util Rebalance check count: 144 INFO [2022-01-10 05:39:48,297] jepsen worker nemesis - couchbase.util Rebalance stuck count: 109 INFO [2022-01-10 05:39:48,302] jepsen worker 63 - jepsen.util 1143 :info :write [21 32] :RequestTimeoutException INFO [2022-01-10 05:39:48,302] jepsen worker 63 - jepsen.util 1233 :invoke :read [21 nil] INFO [2022-01-10 05:39:48,311] jepsen worker 5 - jepsen.util 1355 :ok :read [1 41] INFO [2022-01-10 05:39:48,312] jepsen worker 5 - jepsen.util 1355 :invoke :write [1 14] INFO [2022-01-10 05:39:48,318] jepsen worker 88 - jepsen.util 1438 :ok :read [29 33] INFO [2022-01-10 05:39:48,318] jepsen worker 88 - jepsen.util 1438 :invoke :read [29 nil] INFO [2022-01-10 05:39:48,325] jepsen worker 31 - jepsen.util 1651 :ok :read [10 37] INFO [2022-01-10 05:39:48,326] jepsen worker 31 - jepsen.util 1651 :invoke :write [10 20] INFO [2022-01-10 05:39:48,358] jepsen worker 61 - jepsen.util 1321 :ok :read [20 :nil] INFO [2022-01-10 05:39:48,359] jepsen worker 61 - jepsen.util 1321 :invoke :read [20 nil] INFO [2022-01-10 05:39:48,391] jepsen worker 45 - jepsen.util 1575 :ok :read [15 23] INFO [2022-01-10 05:39:48,391] jepsen worker 45 - jepsen.util 1575 :invoke :read [15 nil] INFO [2022-01-10 05:39:48,494] jepsen worker 81 - jepsen.util 1341 :ok :read [27 30] INFO [2022-01-10 05:39:48,495] jepsen worker 81 - jepsen.util 1341 :invoke :read [27 nil] INFO [2022-01-10 05:39:48,509] jepsen worker 24 - jepsen.util 1284 :ok :read [8 :nil] INFO [2022-01-10 05:39:48,510] jepsen worker 24 - jepsen.util 1284 :invoke :read [8 nil] INFO [2022-01-10 05:39:48,535] jepsen worker 65 - jepsen.util 1325 :ok :read [21 7] INFO [2022-01-10 05:39:48,535] jepsen worker 65 - jepsen.util 1325 :invoke :read [21 nil] INFO [2022-01-10 05:39:48,539] jepsen worker 34 - jepsen.util 1384 :ok :read [11 21] INFO [2022-01-10 05:39:48,539] jepsen worker 34 - jepsen.util 1384 :invoke :write [11 18] INFO [2022-01-10 05:39:48,559] jepsen worker 22 - jepsen.util 1282 :ok :read [7 :nil] INFO [2022-01-10 05:39:48,559] jepsen worker 60 - jepsen.util 1680 :ok :read [20 :nil] INFO [2022-01-10 05:39:48,560] jepsen worker 22 - jepsen.util 1282 :invoke :read [7 nil] INFO [2022-01-10 05:39:48,560] jepsen worker 60 - jepsen.util 1680 :invoke :write [20 1] INFO [2022-01-10 05:39:48,562] jepsen worker 21 - jepsen.util 1551 :ok :read [7 :nil] INFO [2022-01-10 05:39:48,563] jepsen worker 21 - jepsen.util 1551 :invoke :write [7 26] INFO [2022-01-10 05:39:48,831] jepsen worker 12 - jepsen.util 1092 :fail :read [4 nil] :UnambiguousTimeoutException INFO [2022-01-10 05:39:48,832] jepsen worker 12 - jepsen.util 1092 :invoke :write [4 46] INFO [2022-01-10 05:39:48,841] jepsen worker 41 - jepsen.util 1031 :info :write [13 16] :RequestTimeoutException INFO [2022-01-10 05:39:48,841] jepsen worker 41 - jepsen.util 1121 :invoke :read [13 nil] INFO [2022-01-10 05:39:48,905] jepsen worker 78 - jepsen.util 1158 :info :write [26 33] :RequestTimeoutException INFO [2022-01-10 05:39:48,905] jepsen worker 78 - jepsen.util 1248 :invoke :read [26 nil] INFO [2022-01-10 05:39:48,946] jepsen worker 78 - jepsen.util 1248 :ok :read [26 10] INFO [2022-01-10 05:39:48,947] jepsen worker 78 - jepsen.util 1248 :invoke :write [26 0] INFO [2022-01-10 05:39:49,120] jepsen worker 59 - jepsen.util 1409 :ok :read [19 31] INFO [2022-01-10 05:39:49,121] jepsen worker 59 - jepsen.util 1409 :invoke :write [19 13] INFO [2022-01-10 05:39:49,121] jepsen worker 17 - jepsen.util 1187 :ok :read [5 42] INFO [2022-01-10 05:39:49,122] jepsen worker 17 - jepsen.util 1187 :invoke :write [5 10] INFO [2022-01-10 05:39:49,304] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1988304093567251}, :ns_1@172.23.107.227 {:progress 0.1495601173020528}, :ns_1@172.23.105.187 {:progress 0.1495601173020528}, :ns_1@172.23.107.228 {:progress 0.2176470588235294}} INFO [2022-01-10 05:39:49,304] jepsen worker nemesis - couchbase.util Rebalance check count: 145 INFO [2022-01-10 05:39:49,304] jepsen worker nemesis - couchbase.util Rebalance stuck count: 109 INFO [2022-01-10 05:39:49,321] jepsen worker 0 - jepsen.util 1260 :ok :read [0 46] INFO [2022-01-10 05:39:49,322] jepsen worker 0 - jepsen.util 1260 :invoke :write [0 30] INFO [2022-01-10 05:39:49,352] jepsen worker 16 - jepsen.util 1546 :info :write [5 9] :RequestTimeoutException INFO [2022-01-10 05:39:49,352] jepsen worker 16 - jepsen.util 1636 :invoke :read [5 nil] INFO [2022-01-10 05:39:49,436] jepsen worker 79 - jepsen.util 1249 :info :write [26 38] :RequestTimeoutException INFO [2022-01-10 05:39:49,437] jepsen worker 79 - jepsen.util 1339 :invoke :write [26 13] INFO [2022-01-10 05:39:49,494] jepsen worker 51 - jepsen.util 1401 :ok :read [17 41] INFO [2022-01-10 05:39:49,495] jepsen worker 51 - jepsen.util 1401 :invoke :write [17 26] INFO [2022-01-10 05:39:49,498] jepsen worker 52 - jepsen.util 1492 :ok :read [17 41] INFO [2022-01-10 05:39:49,499] jepsen worker 52 - jepsen.util 1492 :invoke :write [17 31] INFO [2022-01-10 05:39:49,530] jepsen worker 58 - jepsen.util 1318 :ok :read [19 31] INFO [2022-01-10 05:39:49,531] jepsen worker 58 - jepsen.util 1318 :invoke :read [19 nil] INFO [2022-01-10 05:39:49,622] jepsen worker 47 - jepsen.util 1307 :ok :read [15 23] INFO [2022-01-10 05:39:49,623] jepsen worker 7 - jepsen.util 1357 :ok :read [2 14] INFO [2022-01-10 05:39:49,623] jepsen worker 47 - jepsen.util 1307 :invoke :read [15 nil] INFO [2022-01-10 05:39:49,623] jepsen worker 7 - jepsen.util 1357 :invoke :write [2 3] INFO [2022-01-10 05:39:49,667] jepsen worker 45 - jepsen.util 1575 :ok :read [15 23] INFO [2022-01-10 05:39:49,667] jepsen worker 45 - jepsen.util 1575 :invoke :read [15 nil] INFO [2022-01-10 05:39:49,692] jepsen worker 18 - jepsen.util 1008 :info :write [6 39] :RequestTimeoutException INFO [2022-01-10 05:39:49,694] jepsen worker 18 - jepsen.util 1098 :invoke :write [6 25] WARN [2022-01-10 05:39:49,736] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":19638331,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":19638331,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9265708,"encode_duration_us":16409,"operation_id":"0x275e","timeout_ms":10000,"last_server_duration_us":9265708,"total_duration_us":10001882},{"operation_name":"upsert","last_dispatch_duration_us":15769922,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15769922,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":11541415,"encode_duration_us":16293,"operation_id":"0x2880","timeout_ms":10000,"last_server_duration_us":11541415,"total_duration_us":10004991},{"operation_name":"upsert","last_dispatch_duration_us":18236339,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18385470,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9354628,"encode_duration_us":22089,"operation_id":"0x28a8","timeout_ms":10000,"last_server_duration_us":9227126,"total_duration_us":10004499}],"total_count":3}} INFO [2022-01-10 05:39:49,752] jepsen worker 61 - jepsen.util 1321 :ok :read [20 :nil] INFO [2022-01-10 05:39:49,753] jepsen worker 61 - jepsen.util 1321 :invoke :read [20 nil] INFO [2022-01-10 05:39:49,768] jepsen worker 65 - jepsen.util 1325 :ok :read [21 7] INFO [2022-01-10 05:39:49,769] jepsen worker 65 - jepsen.util 1325 :invoke :read [21 nil] INFO [2022-01-10 05:39:49,792] jepsen worker 24 - jepsen.util 1284 :ok :read [8 :nil] INFO [2022-01-10 05:39:49,792] jepsen worker 24 - jepsen.util 1284 :invoke :read [8 nil] WARN [2022-01-10 05:39:49,836] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16200246,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":16200246,"total_server_duration_us":1139993,"operation_id":"0x284e","timeout_ms":30000,"last_server_duration_us":1139993,"total_duration_us":16200554},{"operation_name":"get","last_dispatch_duration_us":10155855,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10155855,"total_server_duration_us":772654,"operation_id":"0x29fd","timeout_ms":30000,"last_server_duration_us":772654,"total_duration_us":12769918},{"operation_name":"get","last_dispatch_duration_us":12342051,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12342051,"total_server_duration_us":416106,"operation_id":"0x2947","timeout_ms":30000,"last_server_duration_us":416106,"total_duration_us":12343946},{"operation_name":"get","last_dispatch_duration_us":2172221,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":2172221,"total_server_duration_us":362,"operation_id":"0x2a68","timeout_ms":30000,"last_server_duration_us":362,"total_duration_us":10910897},{"operation_name":"upsert","last_dispatch_duration_us":29548,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":1028115,"total_server_duration_us":583409,"encode_duration_us":18213,"operation_id":"0x2abf","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10010676},{"operation_name":"upsert","last_dispatch_duration_us":12603483,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12603483,"total_server_duration_us":393725,"encode_duration_us":16543,"operation_id":"0x29a5","timeout_ms":10000,"last_server_duration_us":393725,"total_duration_us":10010648},{"operation_name":"upsert","last_dispatch_duration_us":3073459,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":3073459,"total_server_duration_us":215146,"encode_duration_us":17535,"operation_id":"0x29cd","timeout_ms":10000,"last_server_duration_us":215146,"total_duration_us":10009636},{"operation_name":"upsert","last_dispatch_duration_us":36020,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":1411667,"total_server_duration_us":539702,"encode_duration_us":18566,"operation_id":"0x2a0e","timeout_ms":10000,"last_server_duration_us":24411,"total_duration_us":10009483},{"operation_name":"upsert","last_dispatch_duration_us":12635202,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":16779891,"total_server_duration_us":783539,"encode_duration_us":16459,"operation_id":"0x28e8","timeout_ms":10000,"last_server_duration_us":393725,"total_duration_us":10007571},{"operation_name":"upsert","last_dispatch_duration_us":17621168,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":17621168,"total_server_duration_us":291563,"encode_duration_us":16893,"operation_id":"0x290e","timeout_ms":10000,"last_server_duration_us":291563,"total_duration_us":10004000}],"total_count":19}} INFO [2022-01-10 05:39:49,847] jepsen worker 68 - jepsen.util 1328 :ok :read [22 41] INFO [2022-01-10 05:39:49,847] jepsen worker 68 - jepsen.util 1328 :invoke :read [22 nil] INFO [2022-01-10 05:39:49,910] jepsen worker 71 - jepsen.util 1421 :info :write [23 6] :RequestTimeoutException INFO [2022-01-10 05:39:49,911] jepsen worker 71 - jepsen.util 1511 :invoke :read [23 nil] INFO [2022-01-10 05:39:49,983] jepsen worker 72 - jepsen.util 1422 :info :write [24 28] :RequestTimeoutException INFO [2022-01-10 05:39:49,983] jepsen worker 72 - jepsen.util 1512 :invoke :write [24 26] INFO [2022-01-10 05:39:49,991] jepsen worker 83 - jepsen.util 1523 :info :write [27 2] :RequestTimeoutException INFO [2022-01-10 05:39:49,992] jepsen worker 83 - jepsen.util 1613 :invoke :write [27 4] INFO [2022-01-10 05:39:50,015] jepsen worker 15 - jepsen.util 1005 :ok :read [5 42] INFO [2022-01-10 05:39:50,015] jepsen worker 15 - jepsen.util 1005 :invoke :write [5 29] INFO [2022-01-10 05:39:50,219] jepsen worker 22 - jepsen.util 1282 :ok :read [7 :nil] INFO [2022-01-10 05:39:50,220] jepsen worker 22 - jepsen.util 1282 :invoke :read [7 nil] INFO [2022-01-10 05:39:50,252] jepsen worker 45 - jepsen.util 1575 :ok :read [15 23] INFO [2022-01-10 05:39:50,253] jepsen worker 45 - jepsen.util 1575 :invoke :read [15 nil] INFO [2022-01-10 05:39:50,261] jepsen worker 30 - jepsen.util 1110 :ok :read [10 37] INFO [2022-01-10 05:39:50,262] jepsen worker 30 - jepsen.util 1110 :invoke :read [10 nil] INFO [2022-01-10 05:39:50,311] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1988304093567251}, :ns_1@172.23.107.227 {:progress 0.1495601173020528}, :ns_1@172.23.105.187 {:progress 0.1495601173020528}, :ns_1@172.23.107.228 {:progress 0.2176470588235294}} INFO [2022-01-10 05:39:50,311] jepsen worker nemesis - couchbase.util Rebalance check count: 146 INFO [2022-01-10 05:39:50,311] jepsen worker nemesis - couchbase.util Rebalance stuck count: 110 INFO [2022-01-10 05:39:50,405] jepsen worker 82 - jepsen.util 1252 :ok :read [27 30] INFO [2022-01-10 05:39:50,406] jepsen worker 82 - jepsen.util 1252 :invoke :read [27 nil] INFO [2022-01-10 05:39:50,441] jepsen worker 24 - jepsen.util 1284 :ok :read [8 :nil] INFO [2022-01-10 05:39:50,442] jepsen worker 24 - jepsen.util 1284 :invoke :write [8 41] INFO [2022-01-10 05:39:50,466] jepsen worker 80 - jepsen.util 1340 :info :write [26 8] :RequestTimeoutException INFO [2022-01-10 05:39:50,466] jepsen worker 80 - jepsen.util 1430 :invoke :read [26 nil] INFO [2022-01-10 05:39:50,467] jepsen worker 63 - jepsen.util 1233 :ok :read [21 7] INFO [2022-01-10 05:39:50,467] jepsen worker 63 - jepsen.util 1233 :invoke :read [21 nil] INFO [2022-01-10 05:39:50,506] jepsen worker 80 - jepsen.util 1430 :ok :read [26 10] INFO [2022-01-10 05:39:50,506] jepsen worker 80 - jepsen.util 1430 :invoke :write [26 18] INFO [2022-01-10 05:39:50,522] jepsen worker 75 - jepsen.util 1065 :info :write [25 40] :RequestTimeoutException INFO [2022-01-10 05:39:50,522] jepsen worker 75 - jepsen.util 1155 :invoke :read [25 nil] INFO [2022-01-10 05:39:50,526] jepsen worker 48 - jepsen.util 1038 :info :write [16 41] :RequestTimeoutException INFO [2022-01-10 05:39:50,526] jepsen worker 48 - jepsen.util 1128 :invoke :read [16 nil] INFO [2022-01-10 05:39:50,641] jepsen worker 81 - jepsen.util 1341 :ok :read [27 30] INFO [2022-01-10 05:39:50,642] jepsen worker 81 - jepsen.util 1341 :invoke :write [27 15] INFO [2022-01-10 05:39:50,651] jepsen worker 57 - jepsen.util 1407 :ok :read [19 31] INFO [2022-01-10 05:39:50,652] jepsen worker 57 - jepsen.util 1407 :invoke :write [19 6] INFO [2022-01-10 05:39:51,036] jepsen worker 46 - jepsen.util 1396 :info :write [15 19] :RequestTimeoutException INFO [2022-01-10 05:39:51,037] jepsen worker 46 - jepsen.util 1486 :invoke :read [15 nil] INFO [2022-01-10 05:39:51,111] jepsen worker 11 - jepsen.util 1271 :info :write [3 14] :RequestTimeoutException INFO [2022-01-10 05:39:51,111] jepsen worker 11 - jepsen.util 1361 :invoke :read [3 nil] INFO [2022-01-10 05:39:51,116] jepsen worker 54 - jepsen.util 1404 :info :write [18 5] :RequestTimeoutException INFO [2022-01-10 05:39:51,116] jepsen worker 54 - jepsen.util 1494 :invoke :write [18 11] INFO [2022-01-10 05:39:51,162] jepsen worker 84 - jepsen.util 1344 :info :write [28 31] :RequestTimeoutException INFO [2022-01-10 05:39:51,162] jepsen worker 84 - jepsen.util 1434 :invoke :read [28 nil] INFO [2022-01-10 05:39:51,186] jepsen worker 3 - jepsen.util 1623 :info :write [1 2] :RequestTimeoutException INFO [2022-01-10 05:39:51,187] jepsen worker 3 - jepsen.util 1713 :invoke :read [1 nil] INFO [2022-01-10 05:39:51,194] jepsen worker 45 - jepsen.util 1575 :ok :read [15 23] INFO [2022-01-10 05:39:51,194] jepsen worker 45 - jepsen.util 1575 :invoke :read [15 nil] INFO [2022-01-10 05:39:51,206] jepsen worker 56 - jepsen.util 1406 :info :write [18 26] :RequestTimeoutException INFO [2022-01-10 05:39:51,207] jepsen worker 56 - jepsen.util 1496 :invoke :write [18 45] WARN [2022-01-10 05:39:51,266] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10632897,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10632897,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":10615681,"encode_duration_us":16499,"operation_id":"0x2a00","timeout_ms":10000,"last_server_duration_us":10615681,"total_duration_us":10002375},{"operation_name":"upsert","last_dispatch_duration_us":12667752,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12667752,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":10146247,"encode_duration_us":17128,"operation_id":"0x2a03","timeout_ms":10000,"last_server_duration_us":10146247,"total_duration_us":10003903}],"total_count":2}} INFO [2022-01-10 05:39:51,273] jepsen worker 16 - jepsen.util 1636 :ok :read [5 42] INFO [2022-01-10 05:39:51,273] jepsen worker 16 - jepsen.util 1636 :invoke :read [5 nil] INFO [2022-01-10 05:39:51,316] jepsen worker 77 - jepsen.util 1607 :info :write [25 0] :RequestTimeoutException INFO [2022-01-10 05:39:51,317] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1988304093567251}, :ns_1@172.23.107.227 {:progress 0.1495601173020528}, :ns_1@172.23.105.187 {:progress 0.1495601173020528}, :ns_1@172.23.107.228 {:progress 0.2176470588235294}} INFO [2022-01-10 05:39:51,317] jepsen worker 77 - jepsen.util 1697 :invoke :read [25 nil] INFO [2022-01-10 05:39:51,317] jepsen worker nemesis - couchbase.util Rebalance check count: 147 INFO [2022-01-10 05:39:51,317] jepsen worker nemesis - couchbase.util Rebalance stuck count: 111 WARN [2022-01-10 05:39:51,337] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17532086,"last_local_id":"EFE9906200000006/0000000021E2C07D","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":18435941,"last_local_socket":"172.23.104.79:44302","total_server_duration_us":9535568,"encode_duration_us":22394,"operation_id":"0x27bb","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10010123}],"total_count":1}} WARN [2022-01-10 05:39:51,337] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":8528027,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":28305059,"total_server_duration_us":766,"operation_id":"0x2685","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":30009246},{"operation_name":"upsert","last_dispatch_duration_us":2267947,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10172623,"total_server_duration_us":1024038,"encode_duration_us":17896,"operation_id":"0x2acc","timeout_ms":10000,"last_server_duration_us":287632,"total_duration_us":10007689},{"operation_name":"upsert","last_dispatch_duration_us":418463,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7651851,"total_server_duration_us":785275,"encode_duration_us":17850,"operation_id":"0x299a","timeout_ms":10000,"last_server_duration_us":99346,"total_duration_us":10006945},{"operation_name":"upsert","last_dispatch_duration_us":5437751,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":5437751,"total_server_duration_us":195621,"encode_duration_us":16254,"operation_id":"0x2af1","timeout_ms":10000,"last_server_duration_us":195621,"total_duration_us":10006687},{"operation_name":"upsert","last_dispatch_duration_us":2124248,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8907863,"total_server_duration_us":2033908,"encode_duration_us":16400,"operation_id":"0x2a9d","timeout_ms":10000,"last_server_duration_us":468147,"total_duration_us":10004010},{"operation_name":"upsert","last_dispatch_duration_us":1743448,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8469829,"total_server_duration_us":673148,"encode_duration_us":16335,"operation_id":"0x29fe","timeout_ms":10000,"last_server_duration_us":105184,"total_duration_us":10003677},{"operation_name":"upsert","last_dispatch_duration_us":1683317,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5129662,"total_server_duration_us":636590,"encode_duration_us":17120,"operation_id":"0x2a33","timeout_ms":10000,"last_server_duration_us":114123,"total_duration_us":10002868},{"operation_name":"upsert","last_dispatch_duration_us":5114517,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9401226,"total_server_duration_us":785152,"encode_duration_us":16425,"operation_id":"0x2abd","timeout_ms":10000,"last_server_duration_us":585564,"total_duration_us":10002369},{"operation_name":"upsert","last_dispatch_duration_us":6133767,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":12206238,"total_server_duration_us":619484,"encode_duration_us":17107,"operation_id":"0x2946","timeout_ms":10000,"last_server_duration_us":416106,"total_duration_us":10001801},{"operation_name":"upsert","last_dispatch_duration_us":1475041,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":8852869,"total_server_duration_us":3815543,"encode_duration_us":18154,"operation_id":"0x2b1e","timeout_ms":10000,"last_server_duration_us":710367,"total_duration_us":10000614}],"total_count":24}} WARN [2022-01-10 05:39:51,360] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":25905959,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":29413299,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":10025574,"encode_duration_us":17356,"operation_id":"0x26c1","timeout_ms":10000,"last_server_duration_us":9647647,"total_duration_us":10001952},{"operation_name":"upsert","last_dispatch_duration_us":9653438,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13165977,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9711582,"encode_duration_us":16849,"operation_id":"0x2969","timeout_ms":10000,"last_server_duration_us":9414240,"total_duration_us":10004447},{"operation_name":"upsert","last_dispatch_duration_us":17296350,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17296350,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9107531,"encode_duration_us":28894,"operation_id":"0x283c","timeout_ms":10000,"last_server_duration_us":9107531,"total_duration_us":10004510},{"operation_name":"upsert","last_dispatch_duration_us":11084980,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11084980,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":10209881,"encode_duration_us":21885,"operation_id":"0x2a0f","timeout_ms":10000,"last_server_duration_us":10209881,"total_duration_us":10007051}],"total_count":4}} WARN [2022-01-10 05:39:51,365] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":3440701,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8951408,"total_server_duration_us":2161355,"encode_duration_us":19002,"operation_id":"0x2b29","timeout_ms":10000,"last_server_duration_us":694965,"total_duration_us":10010708},{"operation_name":"upsert","last_dispatch_duration_us":2756552,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12507114,"total_server_duration_us":957004,"encode_duration_us":16332,"operation_id":"0x2a26","timeout_ms":10000,"last_server_duration_us":90781,"total_duration_us":10008857},{"operation_name":"upsert","last_dispatch_duration_us":1966366,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9266073,"total_server_duration_us":536282,"encode_duration_us":16478,"operation_id":"0x2b1d","timeout_ms":10000,"last_server_duration_us":223660,"total_duration_us":10006271},{"operation_name":"upsert","last_dispatch_duration_us":12494411,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":12494411,"total_server_duration_us":20521,"encode_duration_us":17437,"operation_id":"0x29a4","timeout_ms":10000,"last_server_duration_us":20521,"total_duration_us":10005872},{"operation_name":"upsert","last_dispatch_duration_us":8896533,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11394458,"total_server_duration_us":570404,"encode_duration_us":16531,"operation_id":"0x2a0d","timeout_ms":10000,"last_server_duration_us":231868,"total_duration_us":10004813},{"operation_name":"upsert","last_dispatch_duration_us":9594736,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":14240678,"total_server_duration_us":3010526,"encode_duration_us":21586,"operation_id":"0x2945","timeout_ms":10000,"last_server_duration_us":874938,"total_duration_us":10004420},{"operation_name":"upsert","last_dispatch_duration_us":135387,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9788538,"total_server_duration_us":717614,"encode_duration_us":16602,"operation_id":"0x2944","timeout_ms":10000,"last_server_duration_us":23222,"total_duration_us":10002436},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":17157,"operation_id":"0x29cc","timeout_ms":10000,"total_duration_us":10002370},{"operation_name":"upsert","last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","encode_duration_us":17170,"operation_id":"0x2ac9","timeout_ms":10000,"total_duration_us":10001647},{"operation_name":"upsert","last_dispatch_duration_us":1777195,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9619854,"total_server_duration_us":1426412,"encode_duration_us":19259,"operation_id":"0x2b27","timeout_ms":10000,"last_server_duration_us":203590,"total_duration_us":10000562}],"total_count":15}} WARN [2022-01-10 05:39:51,365] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":6832608,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11935397,"total_server_duration_us":1682359,"encode_duration_us":16296,"operation_id":"0x2a8c","timeout_ms":10000,"last_server_duration_us":460639,"total_duration_us":10010574},{"operation_name":"upsert","last_dispatch_duration_us":4125297,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7339581,"total_server_duration_us":195958,"encode_duration_us":16209,"operation_id":"0x2af0","timeout_ms":10000,"last_server_duration_us":84865,"total_duration_us":10009646},{"operation_name":"upsert","last_dispatch_duration_us":1976848,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":5442345,"total_server_duration_us":220877,"encode_duration_us":16542,"operation_id":"0x2afb","timeout_ms":10000,"last_server_duration_us":58999,"total_duration_us":10007949},{"operation_name":"upsert","last_dispatch_duration_us":4760992,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11662425,"total_server_duration_us":760577,"encode_duration_us":16612,"operation_id":"0x2ac7","timeout_ms":10000,"last_server_duration_us":371599,"total_duration_us":10007523},{"operation_name":"upsert","last_dispatch_duration_us":2146415,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2146415,"total_server_duration_us":52896,"encode_duration_us":16388,"operation_id":"0x299c","timeout_ms":10000,"last_server_duration_us":52896,"total_duration_us":10007516},{"operation_name":"upsert","last_dispatch_duration_us":2363253,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9959817,"total_server_duration_us":277212,"encode_duration_us":16301,"operation_id":"0x2a04","timeout_ms":10000,"last_server_duration_us":57632,"total_duration_us":10007425},{"operation_name":"upsert","last_dispatch_duration_us":7494875,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":7494875,"total_server_duration_us":399905,"encode_duration_us":22871,"operation_id":"0x2b25","timeout_ms":10000,"last_server_duration_us":399905,"total_duration_us":10006604},{"operation_name":"upsert","last_dispatch_duration_us":2140277,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":5261533,"total_server_duration_us":148478,"encode_duration_us":17712,"operation_id":"0x2a25","timeout_ms":10000,"last_server_duration_us":62412,"total_duration_us":10006442},{"operation_name":"upsert","last_dispatch_duration_us":1127623,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7377503,"total_server_duration_us":266290,"encode_duration_us":18930,"operation_id":"0x2943","timeout_ms":10000,"last_server_duration_us":85596,"total_duration_us":10005365},{"operation_name":"upsert","last_dispatch_duration_us":2198419,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10278934,"total_server_duration_us":581312,"encode_duration_us":18788,"operation_id":"0x2972","timeout_ms":10000,"last_server_duration_us":105984,"total_duration_us":10005222}],"total_count":29}} WARN [2022-01-10 05:39:51,367] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10266613,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10266613,"total_server_duration_us":203167,"operation_id":"0x2a11","timeout_ms":30000,"last_server_duration_us":203167,"total_duration_us":10269284},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","encode_duration_us":18709,"operation_id":"0x2a63","timeout_ms":10000,"total_duration_us":10009626},{"operation_name":"upsert","last_dispatch_duration_us":7326589,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7326589,"total_server_duration_us":420415,"encode_duration_us":16682,"operation_id":"0x2afc","timeout_ms":10000,"last_server_duration_us":420415,"total_duration_us":10007382},{"operation_name":"upsert","last_dispatch_duration_us":9915517,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":15480425,"total_server_duration_us":705005,"encode_duration_us":20665,"operation_id":"0x2971","timeout_ms":10000,"last_server_duration_us":83848,"total_duration_us":10006826},{"operation_name":"upsert","last_dispatch_duration_us":9869574,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9869574,"total_server_duration_us":192719,"encode_duration_us":18643,"operation_id":"0x2a3d","timeout_ms":10000,"last_server_duration_us":192719,"total_duration_us":10006301},{"operation_name":"upsert","last_dispatch_duration_us":32143,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":1030046,"total_server_duration_us":625923,"encode_duration_us":18974,"operation_id":"0x2afa","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10004904},{"operation_name":"upsert","last_dispatch_duration_us":8766896,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":13045970,"total_server_duration_us":248783,"encode_duration_us":17566,"operation_id":"0x2a10","timeout_ms":10000,"last_server_duration_us":128369,"total_duration_us":10004593},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","encode_duration_us":18938,"operation_id":"0x2a99","timeout_ms":10000,"total_duration_us":10004371},{"operation_name":"upsert","last_dispatch_duration_us":12667752,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12667752,"total_server_duration_us":10146247,"encode_duration_us":17128,"operation_id":"0x2a03","timeout_ms":10000,"last_server_duration_us":10146247,"total_duration_us":10003903},{"operation_name":"upsert","last_dispatch_duration_us":10052970,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":13632126,"total_server_duration_us":470791,"encode_duration_us":16471,"operation_id":"0x2a02","timeout_ms":10000,"last_server_duration_us":205922,"total_duration_us":10003885}],"total_count":14}} INFO [2022-01-10 05:39:51,443] jepsen worker 41 - jepsen.util 1121 :ok :read [13 15] INFO [2022-01-10 05:39:51,444] jepsen worker 41 - jepsen.util 1121 :invoke :read [13 nil] WARN [2022-01-10 05:39:51,465] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10496757,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18964344,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":11454852,"encode_duration_us":16399,"operation_id":"0x287e","timeout_ms":10000,"last_server_duration_us":10224418,"total_duration_us":10003382},{"operation_name":"upsert","last_dispatch_duration_us":17950590,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":26857124,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":9099934,"encode_duration_us":16695,"operation_id":"0x26f1","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10007857},{"operation_name":"upsert","last_dispatch_duration_us":18836507,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25939585,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":11093432,"encode_duration_us":25577,"operation_id":"0x27dd","timeout_ms":10000,"last_server_duration_us":10162977,"total_duration_us":10007822}],"total_count":3}} WARN [2022-01-10 05:39:51,465] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9206498,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15170433,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9188265,"encode_duration_us":17110,"operation_id":"0x28a3","timeout_ms":10000,"last_server_duration_us":9128839,"total_duration_us":10003110}],"total_count":1}} WARN [2022-01-10 05:39:51,560] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":608,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":2281736,"total_server_duration_us":3027,"operation_id":"0x26ee","timeout_ms":30000,"last_server_duration_us":84,"total_duration_us":30010681},{"operation_name":"get","last_dispatch_duration_us":24245817,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":24245817,"total_server_duration_us":43,"operation_id":"0x2811","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":24248277},{"operation_name":"upsert","last_dispatch_duration_us":32601,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":2955019,"total_server_duration_us":502808,"encode_duration_us":16614,"operation_id":"0x2974","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10011015},{"operation_name":"upsert","last_dispatch_duration_us":3202651,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":11530375,"total_server_duration_us":1792411,"encode_duration_us":17118,"operation_id":"0x2aee","timeout_ms":10000,"last_server_duration_us":398497,"total_duration_us":10010305},{"operation_name":"upsert","last_dispatch_duration_us":8777253,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9859892,"total_server_duration_us":645652,"encode_duration_us":19171,"operation_id":"0x2a9c","timeout_ms":10000,"last_server_duration_us":180492,"total_duration_us":10009766},{"operation_name":"upsert","last_dispatch_duration_us":8776433,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12894343,"total_server_duration_us":448552,"encode_duration_us":17655,"operation_id":"0x29ef","timeout_ms":10000,"last_server_duration_us":59124,"total_duration_us":10009638},{"operation_name":"upsert","last_dispatch_duration_us":2378842,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10674713,"total_server_duration_us":1594585,"encode_duration_us":17367,"operation_id":"0x2a98","timeout_ms":10000,"last_server_duration_us":183112,"total_duration_us":10008572},{"operation_name":"upsert","last_dispatch_duration_us":11084980,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11084980,"total_server_duration_us":10209881,"encode_duration_us":21885,"operation_id":"0x2a0f","timeout_ms":10000,"last_server_duration_us":10209881,"total_duration_us":10007051},{"operation_name":"upsert","last_dispatch_duration_us":8425328,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9997836,"total_server_duration_us":410269,"encode_duration_us":16565,"operation_id":"0x2a62","timeout_ms":10000,"last_server_duration_us":193546,"total_duration_us":10005127},{"operation_name":"upsert","last_dispatch_duration_us":9653438,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":13165977,"total_server_duration_us":9711582,"encode_duration_us":16849,"operation_id":"0x2969","timeout_ms":10000,"last_server_duration_us":9414240,"total_duration_us":10004447}],"total_count":29}} INFO [2022-01-10 05:39:51,615] jepsen worker 39 - jepsen.util 1479 :info :write [13 49] :RequestTimeoutException INFO [2022-01-10 05:39:51,615] jepsen worker 39 - jepsen.util 1569 :invoke :write [13 28] INFO [2022-01-10 05:39:51,872] jepsen worker 40 - jepsen.util 1120 :ok :read [13 15] INFO [2022-01-10 05:39:51,872] jepsen worker 16 - jepsen.util 1636 :ok :read [5 42] INFO [2022-01-10 05:39:51,873] jepsen worker 16 - jepsen.util 1636 :invoke :write [5 38] INFO [2022-01-10 05:39:51,873] jepsen worker 58 - jepsen.util 1318 :ok :read [19 31] INFO [2022-01-10 05:39:51,874] jepsen worker 40 - jepsen.util 1120 :invoke :read [13 nil] INFO [2022-01-10 05:39:51,875] jepsen worker 58 - jepsen.util 1318 :invoke :write [19 12] INFO [2022-01-10 05:39:51,937] jepsen worker 6 - jepsen.util 1356 :ok :read [2 14] INFO [2022-01-10 05:39:51,937] jepsen worker 6 - jepsen.util 1356 :invoke :write [2 47] INFO [2022-01-10 05:39:52,020] jepsen worker 45 - jepsen.util 1575 :ok :read [15 23] INFO [2022-01-10 05:39:52,021] jepsen worker 45 - jepsen.util 1575 :invoke :write [15 6] INFO [2022-01-10 05:39:52,060] jepsen worker 84 - jepsen.util 1434 :ok :read [28 13] INFO [2022-01-10 05:39:52,061] jepsen worker 84 - jepsen.util 1434 :invoke :read [28 nil] INFO [2022-01-10 05:39:52,099] jepsen worker 63 - jepsen.util 1233 :ok :read [21 7] INFO [2022-01-10 05:39:52,100] jepsen worker 63 - jepsen.util 1233 :invoke :read [21 nil] INFO [2022-01-10 05:39:52,297] jepsen worker 46 - jepsen.util 1486 :ok :read [15 23] INFO [2022-01-10 05:39:52,298] jepsen worker 46 - jepsen.util 1486 :invoke :read [15 nil] INFO [2022-01-10 05:39:52,323] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.1988304093567251}, :ns_1@172.23.107.227 {:progress 0.1495601173020528}, :ns_1@172.23.105.187 {:progress 0.1495601173020528}, :ns_1@172.23.107.228 {:progress 0.2176470588235294}} INFO [2022-01-10 05:39:52,323] jepsen worker nemesis - couchbase.util Rebalance check count: 148 INFO [2022-01-10 05:39:52,323] jepsen worker nemesis - couchbase.util Rebalance stuck count: 112 INFO [2022-01-10 05:39:52,377] jepsen worker 77 - jepsen.util 1697 :ok :read [25 :nil] INFO [2022-01-10 05:39:52,378] jepsen worker 77 - jepsen.util 1697 :invoke :write [25 39] INFO [2022-01-10 05:39:52,405] jepsen worker 42 - jepsen.util 1302 :ok :read [14 14] INFO [2022-01-10 05:39:52,405] jepsen worker 42 - jepsen.util 1302 :invoke :write [14 48] INFO [2022-01-10 05:39:52,520] jepsen worker 71 - jepsen.util 1511 :ok :read [23 26] INFO [2022-01-10 05:39:52,520] jepsen worker 71 - jepsen.util 1511 :invoke :read [23 nil] INFO [2022-01-10 05:39:52,656] jepsen worker 20 - jepsen.util 1100 :info :write [6 11] :RequestTimeoutException INFO [2022-01-10 05:39:52,657] jepsen worker 20 - jepsen.util 1190 :invoke :write [6 42] INFO [2022-01-10 05:39:52,666] jepsen worker 36 - jepsen.util 936 :info :write [12 33] :RequestTimeoutException INFO [2022-01-10 05:39:52,667] jepsen worker 36 - jepsen.util 1026 :invoke :write [12 1] INFO [2022-01-10 05:39:52,744] jepsen worker 63 - jepsen.util 1233 :ok :read [21 7] INFO [2022-01-10 05:39:52,745] jepsen worker 63 - jepsen.util 1233 :invoke :write [21 26] INFO [2022-01-10 05:39:52,793] jepsen worker 75 - jepsen.util 1155 :ok :read [25 :nil] INFO [2022-01-10 05:39:52,801] jepsen worker 75 - jepsen.util 1155 :invoke :read [25 nil] INFO [2022-01-10 05:39:52,805] jepsen worker 33 - jepsen.util 1383 :ok :read [11 21] INFO [2022-01-10 05:39:52,807] jepsen worker 33 - jepsen.util 1383 :invoke :write [11 48] INFO [2022-01-10 05:39:52,810] jepsen worker 35 - jepsen.util 1385 :ok :read [11 21] INFO [2022-01-10 05:39:52,811] jepsen worker 35 - jepsen.util 1385 :invoke :read [11 nil] INFO [2022-01-10 05:39:52,816] jepsen worker 46 - jepsen.util 1486 :ok :read [15 23] INFO [2022-01-10 05:39:52,816] jepsen worker 46 - jepsen.util 1486 :invoke :write [15 34] INFO [2022-01-10 05:39:53,021] jepsen worker 26 - jepsen.util 1106 :info :write [8 37] :RequestTimeoutException INFO [2022-01-10 05:39:53,022] jepsen worker 26 - jepsen.util 1196 :invoke :read [8 nil] INFO [2022-01-10 05:39:53,326] jepsen worker 53 - jepsen.util 1313 :info :write [17 49] :RequestTimeoutException INFO [2022-01-10 05:39:53,327] jepsen worker 53 - jepsen.util 1403 :invoke :write [17 20] INFO [2022-01-10 05:39:53,330] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2105263157894737}, :ns_1@172.23.107.227 {:progress 0.1554252199413489}, :ns_1@172.23.105.187 {:progress 0.1554252199413489}, :ns_1@172.23.107.228 {:progress 0.2235294117647059}} INFO [2022-01-10 05:39:53,330] jepsen worker nemesis - couchbase.util Rebalance check count: 149 INFO [2022-01-10 05:39:53,331] jepsen worker nemesis - couchbase.util Rebalance stuck count: 112 INFO [2022-01-10 05:39:53,351] jepsen worker 88 - jepsen.util 1438 :ok :read [29 33] INFO [2022-01-10 05:39:53,351] jepsen worker 88 - jepsen.util 1438 :invoke :read [29 nil] INFO [2022-01-10 05:39:53,577] jepsen worker 49 - jepsen.util 1129 :ok :read [16 38] INFO [2022-01-10 05:39:53,578] jepsen worker 49 - jepsen.util 1129 :invoke :read [16 nil] INFO [2022-01-10 05:39:53,587] jepsen worker 84 - jepsen.util 1434 :ok :read [28 13] INFO [2022-01-10 05:39:53,588] jepsen worker 84 - jepsen.util 1434 :invoke :write [28 29] INFO [2022-01-10 05:39:53,592] jepsen worker 29 - jepsen.util 1379 :ok :read [9 22] INFO [2022-01-10 05:39:53,593] jepsen worker 29 - jepsen.util 1379 :invoke :read [9 nil] INFO [2022-01-10 05:39:53,784] jepsen worker 48 - jepsen.util 1128 :ok :read [16 38] INFO [2022-01-10 05:39:53,784] jepsen worker 48 - jepsen.util 1128 :invoke :read [16 nil] INFO [2022-01-10 05:39:53,801] jepsen worker 50 - jepsen.util 1490 :info :write [16 31] :RequestTimeoutException INFO [2022-01-10 05:39:53,802] jepsen worker 50 - jepsen.util 1580 :invoke :read [16 nil] INFO [2022-01-10 05:39:53,804] jepsen worker 66 - jepsen.util 1146 :info :write [22 43] :RequestTimeoutException INFO [2022-01-10 05:39:53,805] jepsen worker 66 - jepsen.util 1236 :invoke :write [22 7] INFO [2022-01-10 05:39:53,807] jepsen worker 41 - jepsen.util 1121 :ok :read [13 15] INFO [2022-01-10 05:39:53,807] jepsen worker 41 - jepsen.util 1121 :invoke :write [13 16] INFO [2022-01-10 05:39:53,874] jepsen worker 27 - jepsen.util 1287 :info :write [9 42] :RequestTimeoutException INFO [2022-01-10 05:39:53,876] jepsen worker 27 - jepsen.util 1377 :invoke :read [9 nil] INFO [2022-01-10 05:39:53,986] jepsen worker 40 - jepsen.util 1120 :ok :read [13 15] INFO [2022-01-10 05:39:53,987] jepsen worker 40 - jepsen.util 1120 :invoke :read [13 nil] INFO [2022-01-10 05:39:53,994] jepsen worker 49 - jepsen.util 1129 :ok :read [16 38] INFO [2022-01-10 05:39:53,995] jepsen worker 49 - jepsen.util 1129 :invoke :read [16 nil] INFO [2022-01-10 05:39:54,338] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2105263157894737}, :ns_1@172.23.107.227 {:progress 0.1554252199413489}, :ns_1@172.23.105.187 {:progress 0.1554252199413489}, :ns_1@172.23.107.228 {:progress 0.2235294117647059}} INFO [2022-01-10 05:39:54,338] jepsen worker nemesis - couchbase.util Rebalance check count: 150 INFO [2022-01-10 05:39:54,338] jepsen worker nemesis - couchbase.util Rebalance stuck count: 113 INFO [2022-01-10 05:39:54,492] jepsen worker 64 - jepsen.util 1414 :info :write [21 22] :RequestTimeoutException INFO [2022-01-10 05:39:54,492] jepsen worker 64 - jepsen.util 1504 :invoke :read [21 nil] INFO [2022-01-10 05:39:54,595] jepsen worker 30 - jepsen.util 1110 :ok :read [10 37] INFO [2022-01-10 05:39:54,596] jepsen worker 30 - jepsen.util 1110 :invoke :read [10 nil] INFO [2022-01-10 05:39:54,602] jepsen worker 71 - jepsen.util 1511 :ok :read [23 26] INFO [2022-01-10 05:39:54,607] jepsen worker 71 - jepsen.util 1511 :invoke :read [23 nil] INFO [2022-01-10 05:39:54,617] jepsen worker 29 - jepsen.util 1379 :ok :read [9 22] INFO [2022-01-10 05:39:54,618] jepsen worker 29 - jepsen.util 1379 :invoke :read [9 nil] INFO [2022-01-10 05:39:54,634] jepsen worker 68 - jepsen.util 1328 :ok :read [22 41] INFO [2022-01-10 05:39:54,634] jepsen worker 68 - jepsen.util 1328 :invoke :write [22 0] INFO [2022-01-10 05:39:54,786] jepsen worker 23 - jepsen.util 1373 :info :write [7 47] :RequestTimeoutException INFO [2022-01-10 05:39:54,787] jepsen worker 23 - jepsen.util 1463 :invoke :write [7 30] INFO [2022-01-10 05:39:54,799] jepsen worker 48 - jepsen.util 1128 :ok :read [16 38] INFO [2022-01-10 05:39:54,800] jepsen worker 48 - jepsen.util 1128 :invoke :read [16 nil] INFO [2022-01-10 05:39:54,804] jepsen worker 40 - jepsen.util 1120 :ok :read [13 15] INFO [2022-01-10 05:39:54,805] jepsen worker 40 - jepsen.util 1120 :invoke :read [13 nil] INFO [2022-01-10 05:39:54,861] jepsen worker 43 - jepsen.util 1483 :info :write [14 2] :RequestTimeoutException INFO [2022-01-10 05:39:54,861] jepsen worker 43 - jepsen.util 1573 :invoke :read [14 nil] INFO [2022-01-10 05:39:54,887] jepsen worker 10 - jepsen.util 1450 :info :write [3 21] :RequestTimeoutException INFO [2022-01-10 05:39:54,887] jepsen worker 10 - jepsen.util 1540 :invoke :read [3 nil] INFO [2022-01-10 05:39:55,102] jepsen worker 74 - jepsen.util 1694 :info :write [24 20] :RequestTimeoutException INFO [2022-01-10 05:39:55,103] jepsen worker 74 - jepsen.util 1784 :invoke :write [24 23] INFO [2022-01-10 05:39:55,345] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2105263157894737}, :ns_1@172.23.107.227 {:progress 0.1554252199413489}, :ns_1@172.23.105.187 {:progress 0.1554252199413489}, :ns_1@172.23.107.228 {:progress 0.2235294117647059}} INFO [2022-01-10 05:39:55,345] jepsen worker nemesis - couchbase.util Rebalance check count: 151 INFO [2022-01-10 05:39:55,345] jepsen worker nemesis - couchbase.util Rebalance stuck count: 114 INFO [2022-01-10 05:39:55,402] jepsen worker 71 - jepsen.util 1511 :ok :read [23 26] INFO [2022-01-10 05:39:55,403] jepsen worker 71 - jepsen.util 1511 :invoke :write [23 48] INFO [2022-01-10 05:39:55,769] jepsen worker 32 - jepsen.util 1292 :info :write [10 43] :RequestTimeoutException INFO [2022-01-10 05:39:55,769] jepsen worker 70 - jepsen.util 1690 :info :write [23 7] :RequestTimeoutException INFO [2022-01-10 05:39:55,770] jepsen worker 32 - jepsen.util 1382 :invoke :read [10 nil] INFO [2022-01-10 05:39:55,771] jepsen worker 70 - jepsen.util 1780 :invoke :read [23 nil] INFO [2022-01-10 05:39:55,812] jepsen worker 55 - jepsen.util 1495 :info :write [18 21] :RequestTimeoutException INFO [2022-01-10 05:39:55,812] jepsen worker 55 - jepsen.util 1585 :invoke :write [18 41] INFO [2022-01-10 05:39:55,842] jepsen worker 76 - jepsen.util 976 :info :write [25 10] :RequestTimeoutException INFO [2022-01-10 05:39:55,843] jepsen worker 76 - jepsen.util 1066 :invoke :write [25 43] INFO [2022-01-10 05:39:55,846] jepsen worker 19 - jepsen.util 1459 :info :write [6 11] :RequestTimeoutException INFO [2022-01-10 05:39:55,847] jepsen worker 19 - jepsen.util 1549 :invoke :read [6 nil] INFO [2022-01-10 05:39:55,885] jepsen worker 1 - jepsen.util 1351 :info :write [0 10] :RequestTimeoutException INFO [2022-01-10 05:39:55,886] jepsen worker 1 - jepsen.util 1441 :invoke :write [0 22] INFO [2022-01-10 05:39:55,944] jepsen worker 4 - jepsen.util 1354 :info :write [1 28] :RequestTimeoutException INFO [2022-01-10 05:39:55,945] jepsen worker 4 - jepsen.util 1444 :invoke :read [1 nil] INFO [2022-01-10 05:39:56,111] jepsen worker 49 - jepsen.util 1129 :ok :read [16 38] INFO [2022-01-10 05:39:56,111] jepsen worker 49 - jepsen.util 1129 :invoke :read [16 nil] INFO [2022-01-10 05:39:56,139] jepsen worker 29 - jepsen.util 1379 :ok :read [9 22] INFO [2022-01-10 05:39:56,140] jepsen worker 29 - jepsen.util 1379 :invoke :write [9 18] INFO [2022-01-10 05:39:56,216] jepsen worker 88 - jepsen.util 1438 :ok :read [29 33] INFO [2022-01-10 05:39:56,217] jepsen worker 88 - jepsen.util 1438 :invoke :read [29 nil] INFO [2022-01-10 05:39:56,218] jepsen worker 10 - jepsen.util 1540 :ok :read [3 2] INFO [2022-01-10 05:39:56,219] jepsen worker 10 - jepsen.util 1540 :invoke :write [3 26] INFO [2022-01-10 05:39:56,265] jepsen worker 48 - jepsen.util 1128 :ok :read [16 38] INFO [2022-01-10 05:39:56,265] jepsen worker 48 - jepsen.util 1128 :invoke :write [16 47] INFO [2022-01-10 05:39:56,279] jepsen worker 40 - jepsen.util 1120 :ok :read [13 15] INFO [2022-01-10 05:39:56,280] jepsen worker 40 - jepsen.util 1120 :invoke :read [13 nil] INFO [2022-01-10 05:39:56,294] jepsen worker 49 - jepsen.util 1129 :ok :read [16 38] INFO [2022-01-10 05:39:56,294] jepsen worker 49 - jepsen.util 1129 :invoke :write [16 38] INFO [2022-01-10 05:39:56,334] jepsen worker 40 - jepsen.util 1120 :ok :read [13 15] INFO [2022-01-10 05:39:56,335] jepsen worker 40 - jepsen.util 1120 :invoke :read [13 nil] INFO [2022-01-10 05:39:56,362] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2105263157894737}, :ns_1@172.23.107.227 {:progress 0.1554252199413489}, :ns_1@172.23.105.187 {:progress 0.1554252199413489}, :ns_1@172.23.107.228 {:progress 0.2235294117647059}} INFO [2022-01-10 05:39:56,362] jepsen worker nemesis - couchbase.util Rebalance check count: 152 INFO [2022-01-10 05:39:56,362] jepsen worker nemesis - couchbase.util Rebalance stuck count: 115 INFO [2022-01-10 05:39:56,395] jepsen worker 67 - jepsen.util 1327 :info :write [22 28] :RequestTimeoutException INFO [2022-01-10 05:39:56,396] jepsen worker 67 - jepsen.util 1417 :invoke :read [22 nil] INFO [2022-01-10 05:39:56,406] jepsen worker 2 - jepsen.util 1352 :info :write [0 42] :RequestTimeoutException INFO [2022-01-10 05:39:56,406] jepsen worker 2 - jepsen.util 1442 :invoke :read [0 nil] INFO [2022-01-10 05:39:56,781] jepsen worker 9 - jepsen.util 1539 :info :write [3 27] :RequestTimeoutException INFO [2022-01-10 05:39:56,782] jepsen worker 9 - jepsen.util 1629 :invoke :write [3 10] INFO [2022-01-10 05:39:57,003] jepsen worker 2 - jepsen.util 1442 :ok :read [0 46] INFO [2022-01-10 05:39:57,004] jepsen worker 2 - jepsen.util 1442 :invoke :read [0 nil] INFO [2022-01-10 05:39:57,027] jepsen worker 2 - jepsen.util 1442 :ok :read [0 46] INFO [2022-01-10 05:39:57,027] jepsen worker 2 - jepsen.util 1442 :invoke :write [0 22] INFO [2022-01-10 05:39:57,058] jepsen worker 40 - jepsen.util 1120 :ok :read [13 15] INFO [2022-01-10 05:39:57,059] jepsen worker 40 - jepsen.util 1120 :invoke :write [13 28] INFO [2022-01-10 05:39:57,260] jepsen worker 44 - jepsen.util 1394 :info :write [14 35] :RequestTimeoutException INFO [2022-01-10 05:39:57,261] jepsen worker 44 - jepsen.util 1484 :invoke :read [14 nil] INFO [2022-01-10 05:39:57,271] jepsen worker 86 - jepsen.util 1076 :info :write [28 4] :RequestTimeoutException INFO [2022-01-10 05:39:57,272] jepsen worker 86 - jepsen.util 1166 :invoke :read [28 nil] INFO [2022-01-10 05:39:57,272] jepsen worker 89 - jepsen.util 1529 :info :write [29 35] :RequestTimeoutException INFO [2022-01-10 05:39:57,272] jepsen worker 89 - jepsen.util 1619 :invoke :write [29 25] INFO [2022-01-10 05:39:57,368] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.216374269005848}, :ns_1@172.23.107.227 {:progress 0.158357771260997}, :ns_1@172.23.105.187 {:progress 0.158357771260997}, :ns_1@172.23.107.228 {:progress 0.2294117647058823}} INFO [2022-01-10 05:39:57,368] jepsen worker nemesis - couchbase.util Rebalance check count: 153 INFO [2022-01-10 05:39:57,447] jepsen worker nemesis - couchbase.util Rebalance stuck count: 115 INFO [2022-01-10 05:39:57,465] jepsen worker 85 - jepsen.util 1525 :info :write [28 45] :RequestTimeoutException INFO [2022-01-10 05:39:57,466] jepsen worker 85 - jepsen.util 1615 :invoke :write [28 47] INFO [2022-01-10 05:39:57,615] jepsen worker 25 - jepsen.util 1285 :info :write [8 5] :RequestTimeoutException INFO [2022-01-10 05:39:57,616] jepsen worker 25 - jepsen.util 1375 :invoke :read [8 nil] INFO [2022-01-10 05:39:57,636] jepsen worker 62 - jepsen.util 1502 :info :write [20 43] :RequestTimeoutException INFO [2022-01-10 05:39:57,636] jepsen worker 62 - jepsen.util 1592 :invoke :read [20 nil] INFO [2022-01-10 05:39:57,661] jepsen worker 73 - jepsen.util 1333 :info :write [24 20] :RequestTimeoutException INFO [2022-01-10 05:39:57,663] jepsen worker 73 - jepsen.util 1423 :invoke :write [24 15] INFO [2022-01-10 05:39:57,688] jepsen worker 30 - jepsen.util 1110 :ok :read [10 37] INFO [2022-01-10 05:39:57,689] jepsen worker 30 - jepsen.util 1110 :invoke :read [10 nil] INFO [2022-01-10 05:39:57,797] jepsen worker 44 - jepsen.util 1484 :ok :read [14 14] INFO [2022-01-10 05:39:57,798] jepsen worker 44 - jepsen.util 1484 :invoke :read [14 nil] INFO [2022-01-10 05:39:58,231] jepsen worker 69 - jepsen.util 1419 :info :write [23 27] :RequestTimeoutException INFO [2022-01-10 05:39:58,232] jepsen worker 69 - jepsen.util 1509 :invoke :read [23 nil] INFO [2022-01-10 05:39:58,256] jepsen worker 8 - jepsen.util 1358 :info :write [2 8] :RequestTimeoutException INFO [2022-01-10 05:39:58,257] jepsen worker 8 - jepsen.util 1448 :invoke :write [2 31] INFO [2022-01-10 05:39:58,331] jepsen worker 5 - jepsen.util 1355 :info :write [1 14] :RequestTimeoutException INFO [2022-01-10 05:39:58,332] jepsen worker 5 - jepsen.util 1445 :invoke :read [1 nil] INFO [2022-01-10 05:39:58,352] jepsen worker 31 - jepsen.util 1651 :info :write [10 20] :RequestTimeoutException INFO [2022-01-10 05:39:58,353] jepsen worker 31 - jepsen.util 1741 :invoke :write [10 48] INFO [2022-01-10 05:39:58,364] jepsen worker 19 - jepsen.util 1549 :ok :read [6 25] INFO [2022-01-10 05:39:58,365] jepsen worker 19 - jepsen.util 1549 :invoke :write [6 34] INFO [2022-01-10 05:39:58,421] jepsen worker 38 - jepsen.util 1478 :ok :read [12 :nil] INFO [2022-01-10 05:39:58,421] jepsen worker 38 - jepsen.util 1478 :invoke :write [12 45] INFO [2022-01-10 05:39:58,454] jepsen worker 11 - jepsen.util 1361 :ok :read [3 2] INFO [2022-01-10 05:39:58,455] jepsen worker 11 - jepsen.util 1361 :invoke :read [3 nil] INFO [2022-01-10 05:39:58,457] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.216374269005848}, :ns_1@172.23.107.227 {:progress 0.158357771260997}, :ns_1@172.23.105.187 {:progress 0.158357771260997}, :ns_1@172.23.107.228 {:progress 0.2294117647058823}} INFO [2022-01-10 05:39:58,457] jepsen worker nemesis - couchbase.util Rebalance check count: 154 INFO [2022-01-10 05:39:58,457] jepsen worker nemesis - couchbase.util Rebalance stuck count: 116 INFO [2022-01-10 05:39:58,466] jepsen worker 69 - jepsen.util 1509 :ok :read [23 26] INFO [2022-01-10 05:39:58,467] jepsen worker 69 - jepsen.util 1509 :invoke :write [23 22] INFO [2022-01-10 05:39:58,509] jepsen worker 11 - jepsen.util 1361 :ok :read [3 2] INFO [2022-01-10 05:39:58,509] jepsen worker 11 - jepsen.util 1361 :invoke :write [3 18] INFO [2022-01-10 05:39:58,564] jepsen worker 51 - jepsen.util 1401 :info :write [17 26] :SyncWriteAmbiguous INFO [2022-01-10 05:39:58,564] jepsen worker 51 - jepsen.util 1491 :invoke :write [17 32] INFO [2022-01-10 05:39:58,570] jepsen worker 34 - jepsen.util 1384 :info :write [11 18] :RequestTimeoutException INFO [2022-01-10 05:39:58,571] jepsen worker 34 - jepsen.util 1474 :invoke :read [11 nil] INFO [2022-01-10 05:39:58,580] jepsen worker 60 - jepsen.util 1680 :info :write [20 1] :RequestTimeoutException INFO [2022-01-10 05:39:58,580] jepsen worker 60 - jepsen.util 1770 :invoke :write [20 47] INFO [2022-01-10 05:39:58,587] jepsen worker 21 - jepsen.util 1551 :info :write [7 26] :RequestTimeoutException INFO [2022-01-10 05:39:58,587] jepsen worker 21 - jepsen.util 1641 :invoke :read [7 nil] INFO [2022-01-10 05:39:58,699] jepsen worker 44 - jepsen.util 1484 :ok :read [14 14] INFO [2022-01-10 05:39:58,700] jepsen worker 44 - jepsen.util 1484 :invoke :write [14 16] INFO [2022-01-10 05:39:58,849] jepsen worker 37 - jepsen.util 1387 :ok :read [12 :nil] INFO [2022-01-10 05:39:58,850] jepsen worker 12 - jepsen.util 1092 :info :write [4 46] :RequestTimeoutException INFO [2022-01-10 05:39:58,850] jepsen worker 37 - jepsen.util 1387 :invoke :write [12 39] INFO [2022-01-10 05:39:58,850] jepsen worker 12 - jepsen.util 1182 :invoke :write [4 39] INFO [2022-01-10 05:39:58,891] jepsen worker 87 - jepsen.util 1617 :ok :read [29 33] INFO [2022-01-10 05:39:58,892] jepsen worker 87 - jepsen.util 1617 :invoke :write [29 45] INFO [2022-01-10 05:39:58,904] jepsen worker 61 - jepsen.util 1321 :ok :read [20 :nil] INFO [2022-01-10 05:39:58,904] jepsen worker 61 - jepsen.util 1321 :invoke :write [20 8] INFO [2022-01-10 05:39:58,966] jepsen worker 78 - jepsen.util 1248 :info :write [26 0] :RequestTimeoutException INFO [2022-01-10 05:39:58,967] jepsen worker 78 - jepsen.util 1338 :invoke :write [26 46] INFO [2022-01-10 05:39:59,060] jepsen worker 34 - jepsen.util 1474 :ok :read [11 21] INFO [2022-01-10 05:39:59,060] jepsen worker 34 - jepsen.util 1474 :invoke :write [11 47] INFO [2022-01-10 05:39:59,074] jepsen worker 22 - jepsen.util 1282 :ok :read [7 :nil] INFO [2022-01-10 05:39:59,074] jepsen worker 22 - jepsen.util 1282 :invoke :read [7 nil] INFO [2022-01-10 05:39:59,093] jepsen worker 47 - jepsen.util 1307 :ok :read [15 23] INFO [2022-01-10 05:39:59,093] jepsen worker 47 - jepsen.util 1307 :invoke :read [15 nil] INFO [2022-01-10 05:39:59,136] jepsen worker 65 - jepsen.util 1325 :ok :read [21 7] INFO [2022-01-10 05:39:59,136] jepsen worker 65 - jepsen.util 1325 :invoke :write [21 0] INFO [2022-01-10 05:39:59,146] jepsen worker 59 - jepsen.util 1409 :info :write [19 13] :RequestTimeoutException INFO [2022-01-10 05:39:59,146] jepsen worker 59 - jepsen.util 1499 :invoke :read [19 nil] INFO [2022-01-10 05:39:59,150] jepsen worker 17 - jepsen.util 1187 :info :write [5 10] :RequestTimeoutException INFO [2022-01-10 05:39:59,151] jepsen worker 17 - jepsen.util 1277 :invoke :read [5 nil] INFO [2022-01-10 05:39:59,209] jepsen worker 82 - jepsen.util 1252 :ok :read [27 30] INFO [2022-01-10 05:39:59,210] jepsen worker 82 - jepsen.util 1252 :invoke :write [27 27] INFO [2022-01-10 05:39:59,261] jepsen worker 64 - jepsen.util 1504 :ok :read [21 7] INFO [2022-01-10 05:39:59,262] jepsen worker 64 - jepsen.util 1504 :invoke :write [21 44] INFO [2022-01-10 05:39:59,273] jepsen worker 17 - jepsen.util 1277 :ok :read [5 42] INFO [2022-01-10 05:39:59,273] jepsen worker 17 - jepsen.util 1277 :invoke :write [5 27] INFO [2022-01-10 05:39:59,352] jepsen worker 0 - jepsen.util 1260 :info :write [0 30] :RequestTimeoutException INFO [2022-01-10 05:39:59,353] jepsen worker 0 - jepsen.util 1350 :invoke :write [0 28] INFO [2022-01-10 05:39:59,419] jepsen worker 62 - jepsen.util 1592 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:59,420] jepsen worker 62 - jepsen.util 1592 :invoke :write [20 47] INFO [2022-01-10 05:39:59,433] jepsen worker 43 - jepsen.util 1573 :ok :read [14 14] INFO [2022-01-10 05:39:59,434] jepsen worker 43 - jepsen.util 1573 :invoke :write [14 15] INFO [2022-01-10 05:39:59,462] jepsen worker 79 - jepsen.util 1339 :info :write [26 13] :RequestTimeoutException INFO [2022-01-10 05:39:59,463] jepsen worker 79 - jepsen.util 1429 :invoke :write [26 8] INFO [2022-01-10 05:39:59,466] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.216374269005848}, :ns_1@172.23.107.227 {:progress 0.158357771260997}, :ns_1@172.23.105.187 {:progress 0.158357771260997}, :ns_1@172.23.107.228 {:progress 0.2294117647058823}} INFO [2022-01-10 05:39:59,466] jepsen worker nemesis - couchbase.util Rebalance check count: 155 INFO [2022-01-10 05:39:59,466] jepsen worker nemesis - couchbase.util Rebalance stuck count: 117 INFO [2022-01-10 05:39:59,490] jepsen worker 22 - jepsen.util 1282 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:39:59,491] jepsen worker 22 - jepsen.util 1282 :invoke :write [7 36] INFO [2022-01-10 05:39:59,538] jepsen worker 47 - jepsen.util 1307 :ok :read [15 23] INFO [2022-01-10 05:39:59,538] jepsen worker 47 - jepsen.util 1307 :invoke :read [15 nil] INFO [2022-01-10 05:39:59,541] jepsen worker 52 - jepsen.util 1492 :info :write [17 31] :RequestTimeoutException INFO [2022-01-10 05:39:59,542] jepsen worker 52 - jepsen.util 1582 :invoke :write [17 26] INFO [2022-01-10 05:39:59,590] jepsen worker 28 - jepsen.util 1288 :ok :read [9 22] INFO [2022-01-10 05:39:59,590] jepsen worker 28 - jepsen.util 1288 :invoke :read [9 nil] INFO [2022-01-10 05:39:59,645] jepsen worker 50 - jepsen.util 1580 :ok :read [16 38] INFO [2022-01-10 05:39:59,646] jepsen worker 50 - jepsen.util 1580 :invoke :write [16 8] INFO [2022-01-10 05:39:59,657] jepsen worker 7 - jepsen.util 1357 :info :write [2 3] :RequestTimeoutException INFO [2022-01-10 05:39:59,657] jepsen worker 7 - jepsen.util 1447 :invoke :write [2 17] INFO [2022-01-10 05:39:59,717] jepsen worker 18 - jepsen.util 1098 :info :write [6 25] :RequestTimeoutException INFO [2022-01-10 05:39:59,717] jepsen worker 18 - jepsen.util 1188 :invoke :write [6 6] WARN [2022-01-10 05:39:59,749] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11975781,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14940781,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9591778,"encode_duration_us":16332,"operation_id":"0x2b5e","timeout_ms":10000,"last_server_duration_us":9319408,"total_duration_us":10012336}],"total_count":1}} INFO [2022-01-10 05:39:59,779] jepsen worker 28 - jepsen.util 1288 :ok :read [9 22] INFO [2022-01-10 05:39:59,780] jepsen worker 28 - jepsen.util 1288 :invoke :read [9 nil] WARN [2022-01-10 05:39:59,850] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11975781,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":14940781,"total_server_duration_us":9591778,"encode_duration_us":16332,"operation_id":"0x2b5e","timeout_ms":10000,"last_server_duration_us":9319408,"total_duration_us":10012336},{"operation_name":"upsert","last_dispatch_duration_us":2255264,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4705641,"total_server_duration_us":555583,"encode_duration_us":17051,"operation_id":"0x2baa","timeout_ms":10000,"last_server_duration_us":346035,"total_duration_us":10009548},{"operation_name":"upsert","last_dispatch_duration_us":953625,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10086745,"total_server_duration_us":1035335,"encode_duration_us":16614,"operation_id":"0x2b1c","timeout_ms":10000,"last_server_duration_us":400469,"total_duration_us":10005499},{"operation_name":"upsert","last_dispatch_duration_us":1344260,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7019597,"total_server_duration_us":1935232,"encode_duration_us":17066,"operation_id":"0x2c46","timeout_ms":10000,"last_server_duration_us":526252,"total_duration_us":10005380},{"operation_name":"upsert","last_dispatch_duration_us":2242190,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4337737,"total_server_duration_us":793509,"encode_duration_us":16624,"operation_id":"0x2b2f","timeout_ms":10000,"last_server_duration_us":376256,"total_duration_us":10004928},{"operation_name":"upsert","last_dispatch_duration_us":1896705,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8961878,"total_server_duration_us":1812738,"encode_duration_us":16681,"operation_id":"0x2c18","timeout_ms":10000,"last_server_duration_us":674772,"total_duration_us":10004713},{"operation_name":"upsert","last_dispatch_duration_us":9759085,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":12458739,"total_server_duration_us":683822,"encode_duration_us":16236,"operation_id":"0x2c10","timeout_ms":10000,"last_server_duration_us":401315,"total_duration_us":10003313},{"operation_name":"upsert","last_dispatch_duration_us":2699540,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10718679,"total_server_duration_us":1440780,"encode_duration_us":19448,"operation_id":"0x2ba3","timeout_ms":10000,"last_server_duration_us":379557,"total_duration_us":10002975},{"operation_name":"upsert","last_dispatch_duration_us":2674415,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4974462,"total_server_duration_us":828962,"encode_duration_us":17090,"operation_id":"0x2c44","timeout_ms":10000,"last_server_duration_us":589215,"total_duration_us":10002926},{"operation_name":"get","last_dispatch_duration_us":7323727,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":7323727,"total_server_duration_us":4698,"operation_id":"0x2d10","timeout_ms":30000,"last_server_duration_us":4698,"total_duration_us":7325917}],"total_count":19}} INFO [2022-01-10 05:39:59,897] jepsen worker 28 - jepsen.util 1288 :ok :read [9 22] INFO [2022-01-10 05:39:59,898] jepsen worker 28 - jepsen.util 1288 :invoke :write [9 19] INFO [2022-01-10 05:40:00,005] jepsen worker 72 - jepsen.util 1512 :info :write [24 26] :RequestTimeoutException INFO [2022-01-10 05:40:00,005] jepsen worker 72 - jepsen.util 1602 :invoke :read [24 nil] INFO [2022-01-10 05:40:00,077] jepsen worker 15 - jepsen.util 1005 :info :write [5 29] :RequestTimeoutException INFO [2022-01-10 05:40:00,077] jepsen worker 83 - jepsen.util 1613 :info :write [27 4] :RequestTimeoutException INFO [2022-01-10 05:40:00,078] jepsen worker 83 - jepsen.util 1703 :invoke :write [27 31] INFO [2022-01-10 05:40:00,079] jepsen worker 15 - jepsen.util 1095 :invoke :write [5 28] INFO [2022-01-10 05:40:00,411] jepsen worker 25 - jepsen.util 1375 :ok :read [8 :nil] INFO [2022-01-10 05:40:00,411] jepsen worker 26 - jepsen.util 1196 :ok :read [8 :nil] INFO [2022-01-10 05:40:00,412] jepsen worker 25 - jepsen.util 1375 :invoke :read [8 nil] INFO [2022-01-10 05:40:00,412] jepsen worker 26 - jepsen.util 1196 :invoke :read [8 nil] INFO [2022-01-10 05:40:00,465] jepsen worker 24 - jepsen.util 1284 :info :write [8 41] :RequestTimeoutException INFO [2022-01-10 05:40:00,465] jepsen worker 24 - jepsen.util 1374 :invoke :write [8 24] INFO [2022-01-10 05:40:00,480] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2222222222222222}, :ns_1@172.23.107.227 {:progress 0.1612903225806451}, :ns_1@172.23.105.187 {:progress 0.1612903225806451}, :ns_1@172.23.107.228 {:progress 0.2352941176470589}} INFO [2022-01-10 05:40:00,481] jepsen worker nemesis - couchbase.util Rebalance check count: 156 INFO [2022-01-10 05:40:00,481] jepsen worker nemesis - couchbase.util Rebalance stuck count: 117 INFO [2022-01-10 05:40:00,529] jepsen worker 80 - jepsen.util 1430 :info :write [26 18] :RequestTimeoutException INFO [2022-01-10 05:40:00,530] jepsen worker 80 - jepsen.util 1520 :invoke :read [26 nil] INFO [2022-01-10 05:40:00,637] jepsen worker 75 - jepsen.util 1155 :ok :read [25 :nil] INFO [2022-01-10 05:40:00,638] jepsen worker 75 - jepsen.util 1155 :invoke :read [25 nil] INFO [2022-01-10 05:40:00,663] jepsen worker 47 - jepsen.util 1307 :ok :read [15 23] INFO [2022-01-10 05:40:00,663] jepsen worker 47 - jepsen.util 1307 :invoke :write [15 18] INFO [2022-01-10 05:40:00,665] jepsen worker 81 - jepsen.util 1341 :info :write [27 15] :RequestTimeoutException INFO [2022-01-10 05:40:00,665] jepsen worker 81 - jepsen.util 1431 :invoke :read [27 nil] INFO [2022-01-10 05:40:00,669] jepsen worker 35 - jepsen.util 1385 :ok :read [11 21] INFO [2022-01-10 05:40:00,669] jepsen worker 35 - jepsen.util 1385 :invoke :write [11 14] INFO [2022-01-10 05:40:00,681] jepsen worker 57 - jepsen.util 1407 :info :write [19 6] :RequestTimeoutException INFO [2022-01-10 05:40:00,681] jepsen worker 57 - jepsen.util 1497 :invoke :read [19 nil] INFO [2022-01-10 05:40:00,784] jepsen worker 57 - jepsen.util 1497 :ok :read [19 31] INFO [2022-01-10 05:40:00,785] jepsen worker 57 - jepsen.util 1497 :invoke :write [19 14] INFO [2022-01-10 05:40:00,947] jepsen worker 75 - jepsen.util 1155 :ok :read [25 :nil] INFO [2022-01-10 05:40:00,947] jepsen worker 25 - jepsen.util 1375 :ok :read [8 :nil] INFO [2022-01-10 05:40:00,947] jepsen worker 26 - jepsen.util 1196 :ok :read [8 :nil] INFO [2022-01-10 05:40:00,947] jepsen worker 25 - jepsen.util 1375 :invoke :read [8 nil] INFO [2022-01-10 05:40:00,948] jepsen worker 75 - jepsen.util 1155 :invoke :read [25 nil] INFO [2022-01-10 05:40:00,948] jepsen worker 26 - jepsen.util 1196 :invoke :write [8 10] INFO [2022-01-10 05:40:01,137] jepsen worker 54 - jepsen.util 1494 :info :write [18 11] :RequestTimeoutException INFO [2022-01-10 05:40:01,140] jepsen worker 54 - jepsen.util 1584 :invoke :write [18 8] INFO [2022-01-10 05:40:01,226] jepsen worker 75 - jepsen.util 1155 :ok :read [25 :nil] INFO [2022-01-10 05:40:01,226] jepsen worker 25 - jepsen.util 1375 :ok :read [8 :nil] INFO [2022-01-10 05:40:01,227] jepsen worker 75 - jepsen.util 1155 :invoke :read [25 nil] INFO [2022-01-10 05:40:01,228] jepsen worker 25 - jepsen.util 1375 :invoke :read [8 nil] INFO [2022-01-10 05:40:01,236] jepsen worker 56 - jepsen.util 1496 :info :write [18 45] :RequestTimeoutException INFO [2022-01-10 05:40:01,237] jepsen worker 56 - jepsen.util 1586 :invoke :write [18 47] WARN [2022-01-10 05:40:01,278] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":13066827,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18504579,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9464548,"encode_duration_us":16254,"operation_id":"0x2af1","timeout_ms":10000,"last_server_duration_us":9268927,"total_duration_us":10006687}],"total_count":1}} WARN [2022-01-10 05:40:01,278] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":249792,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":249792,"total_server_duration_us":9002,"operation_id":"0x2b89","timeout_ms":30000,"last_server_duration_us":9002,"total_duration_us":16128201},{"operation_name":"upsert","last_dispatch_duration_us":73993,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":8174431,"total_server_duration_us":42617,"encode_duration_us":16870,"operation_id":"0x2cde","timeout_ms":10000,"last_server_duration_us":22887,"total_duration_us":10044656},{"operation_name":"upsert","last_dispatch_duration_us":8751126,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":8751126,"total_server_duration_us":29975,"encode_duration_us":18590,"operation_id":"0x2caf","timeout_ms":10000,"last_server_duration_us":29975,"total_duration_us":10010658},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":18341,"operation_id":"0x2cf0","timeout_ms":10000,"total_duration_us":10009833},{"operation_name":"upsert","last_dispatch_duration_us":2776247,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":2776247,"total_server_duration_us":211479,"encode_duration_us":19125,"operation_id":"0x2c42","timeout_ms":10000,"last_server_duration_us":211479,"total_duration_us":10009557},{"operation_name":"upsert","last_dispatch_duration_us":34963,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9219187,"total_server_duration_us":305081,"encode_duration_us":32652,"operation_id":"0x2cb4","timeout_ms":10000,"last_server_duration_us":21731,"total_duration_us":10009342},{"operation_name":"upsert","last_dispatch_duration_us":3344731,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9352037,"total_server_duration_us":1501241,"encode_duration_us":17333,"operation_id":"0x2bd5","timeout_ms":10000,"last_server_duration_us":184530,"total_duration_us":10005740},{"operation_name":"upsert","last_dispatch_duration_us":9331094,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":14255487,"total_server_duration_us":683382,"encode_duration_us":17554,"operation_id":"0x2ba2","timeout_ms":10000,"last_server_duration_us":68680,"total_duration_us":10004040},{"operation_name":"upsert","last_dispatch_duration_us":1160529,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9997629,"total_server_duration_us":1796897,"encode_duration_us":16125,"operation_id":"0x2c7a","timeout_ms":10000,"last_server_duration_us":354828,"total_duration_us":10001948},{"operation_name":"upsert","last_dispatch_duration_us":9044354,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9044354,"total_server_duration_us":9026756,"encode_duration_us":16849,"operation_id":"0x2cb2","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":9049515}],"total_count":23}} WARN [2022-01-10 05:40:01,281] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":21812781,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":25003078,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":11851002,"encode_duration_us":16386,"operation_id":"0x290f","timeout_ms":10000,"last_server_duration_us":10879604,"total_duration_us":10010047}],"total_count":1}} WARN [2022-01-10 05:40:01,379] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":19512696,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":19512696,"total_server_duration_us":476314,"operation_id":"0x29d1","timeout_ms":30000,"last_server_duration_us":476314,"total_duration_us":19515078},{"operation_name":"get","last_dispatch_duration_us":19448939,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":19448939,"total_server_duration_us":406693,"operation_id":"0x2976","timeout_ms":30000,"last_server_duration_us":406693,"total_duration_us":19451292},{"operation_name":"get","last_dispatch_duration_us":16119196,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":16119196,"total_server_duration_us":425321,"operation_id":"0x2a66","timeout_ms":30000,"last_server_duration_us":425321,"total_duration_us":16121678},{"operation_name":"upsert","last_dispatch_duration_us":216497,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":661800,"total_server_duration_us":276353,"encode_duration_us":22028,"operation_id":"0x2cb6","timeout_ms":10000,"last_server_duration_us":136114,"total_duration_us":10010565},{"operation_name":"upsert","last_dispatch_duration_us":565179,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":6689493,"total_server_duration_us":3223816,"encode_duration_us":18248,"operation_id":"0x2d1b","timeout_ms":10000,"last_server_duration_us":301739,"total_duration_us":10010101},{"operation_name":"upsert","last_dispatch_duration_us":1206560,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10844251,"total_server_duration_us":2456173,"encode_duration_us":18103,"operation_id":"0x2bd8","timeout_ms":10000,"last_server_duration_us":808047,"total_duration_us":10009955},{"operation_name":"upsert","last_dispatch_duration_us":8056465,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12847394,"total_server_duration_us":1184144,"encode_duration_us":16116,"operation_id":"0x2bc8","timeout_ms":10000,"last_server_duration_us":107269,"total_duration_us":10008600},{"operation_name":"upsert","last_dispatch_duration_us":649956,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":3100775,"total_server_duration_us":743641,"encode_duration_us":18122,"operation_id":"0x2c77","timeout_ms":10000,"last_server_duration_us":230305,"total_duration_us":10006735},{"operation_name":"upsert","last_dispatch_duration_us":8099642,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10474666,"total_server_duration_us":117202,"encode_duration_us":17130,"operation_id":"0x2c84","timeout_ms":10000,"last_server_duration_us":26596,"total_duration_us":10006308},{"operation_name":"upsert","last_dispatch_duration_us":10732309,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12928377,"total_server_duration_us":9252151,"encode_duration_us":18329,"operation_id":"0x2c0c","timeout_ms":10000,"last_server_duration_us":9146967,"total_duration_us":10005133}],"total_count":26}} WARN [2022-01-10 05:40:01,379] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7689305,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":14044440,"total_server_duration_us":557994,"encode_duration_us":17587,"operation_id":"0x2c0b","timeout_ms":10000,"last_server_duration_us":261224,"total_duration_us":10009806},{"operation_name":"upsert","last_dispatch_duration_us":10900374,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10900374,"total_server_duration_us":24755,"encode_duration_us":16707,"operation_id":"0x2c2c","timeout_ms":10000,"last_server_duration_us":24755,"total_duration_us":10008874},{"operation_name":"upsert","last_dispatch_duration_us":7841818,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12254869,"total_server_duration_us":222602,"encode_duration_us":20598,"operation_id":"0x2c48","timeout_ms":10000,"last_server_duration_us":132570,"total_duration_us":10008254},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":16438,"operation_id":"0x2c43","timeout_ms":10000,"total_duration_us":10008185},{"operation_name":"upsert","last_dispatch_duration_us":7619056,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11765862,"total_server_duration_us":337247,"encode_duration_us":16334,"operation_id":"0x2bb5","timeout_ms":10000,"last_server_duration_us":159159,"total_duration_us":10008155},{"operation_name":"upsert","last_dispatch_duration_us":11820766,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":11820766,"total_server_duration_us":9375396,"encode_duration_us":16237,"operation_id":"0x2c04","timeout_ms":10000,"last_server_duration_us":9375396,"total_duration_us":10007563},{"operation_name":"upsert","last_dispatch_duration_us":31263,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":664785,"total_server_duration_us":409500,"encode_duration_us":16527,"operation_id":"0x2cb1","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10007538},{"operation_name":"upsert","last_dispatch_duration_us":8325768,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":16600757,"total_server_duration_us":678654,"encode_duration_us":17163,"operation_id":"0x2b7f","timeout_ms":10000,"last_server_duration_us":93498,"total_duration_us":10005943},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":19441,"operation_id":"0x2c7b","timeout_ms":10000,"total_duration_us":10005719},{"operation_name":"upsert","last_dispatch_duration_us":8468655,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10581512,"total_server_duration_us":220544,"encode_duration_us":16504,"operation_id":"0x2bdc","timeout_ms":10000,"last_server_duration_us":148091,"total_duration_us":10002008}],"total_count":20}} WARN [2022-01-10 05:40:01,382] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":20662214,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":20662214,"total_server_duration_us":408967,"operation_id":"0x2a7f","timeout_ms":30000,"last_server_duration_us":408967,"total_duration_us":20662581},{"operation_name":"get","last_dispatch_duration_us":18991836,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":18991836,"total_server_duration_us":363443,"operation_id":"0x2aeb","timeout_ms":30000,"last_server_duration_us":363443,"total_duration_us":18992220},{"operation_name":"get","last_dispatch_duration_us":10665404,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10665404,"total_server_duration_us":32,"operation_id":"0x2c74","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":10668780},{"operation_name":"get","last_dispatch_duration_us":3465523,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3465523,"total_server_duration_us":19262,"operation_id":"0x2b42","timeout_ms":30000,"last_server_duration_us":19262,"total_duration_us":10162569},{"operation_name":"upsert","last_dispatch_duration_us":4556179,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":4556179,"total_server_duration_us":387030,"encode_duration_us":23729,"operation_id":"0x2cdd","timeout_ms":10000,"last_server_duration_us":387030,"total_duration_us":10060663},{"operation_name":"upsert","last_dispatch_duration_us":3461925,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9763097,"total_server_duration_us":427191,"encode_duration_us":16420,"operation_id":"0x2c19","timeout_ms":10000,"last_server_duration_us":32065,"total_duration_us":10010361},{"operation_name":"upsert","last_dispatch_duration_us":4819734,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":4819734,"total_server_duration_us":190246,"encode_duration_us":16828,"operation_id":"0x2cad","timeout_ms":10000,"last_server_duration_us":190246,"total_duration_us":10007360},{"operation_name":"upsert","last_dispatch_duration_us":4513461,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":4513461,"total_server_duration_us":394285,"encode_duration_us":16312,"operation_id":"0x2c0e","timeout_ms":10000,"last_server_duration_us":394285,"total_duration_us":10007204},{"operation_name":"upsert","last_dispatch_duration_us":1077018,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5146815,"total_server_duration_us":670487,"encode_duration_us":17098,"operation_id":"0x2b5d","timeout_ms":10000,"last_server_duration_us":125085,"total_duration_us":10005872},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","encode_duration_us":16254,"operation_id":"0x2d14","timeout_ms":10000,"total_duration_us":10002912}],"total_count":14}} WARN [2022-01-10 05:40:01,478] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10656000,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":20515893,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9796887,"encode_duration_us":19171,"operation_id":"0x2a9c","timeout_ms":10000,"last_server_duration_us":9151235,"total_duration_us":10009766}],"total_count":1}} WARN [2022-01-10 05:40:01,479] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":15392612,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":15392612,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":9023573,"encode_duration_us":17170,"operation_id":"0x2ac9","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10001647},{"operation_name":"upsert","last_dispatch_duration_us":10732309,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12928377,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9252151,"encode_duration_us":18329,"operation_id":"0x2c0c","timeout_ms":10000,"last_server_duration_us":9146967,"total_duration_us":10005133}],"total_count":2}} WARN [2022-01-10 05:40:01,479] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11100782,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":19408108,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9505676,"encode_duration_us":21392,"operation_id":"0x2a9b","timeout_ms":10000,"last_server_duration_us":9312957,"total_duration_us":10005142},{"operation_name":"upsert","last_dispatch_duration_us":11820766,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11820766,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9375396,"encode_duration_us":16237,"operation_id":"0x2c04","timeout_ms":10000,"last_server_duration_us":9375396,"total_duration_us":10007563},{"operation_name":"upsert","last_dispatch_duration_us":12947375,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18389721,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9598429,"encode_duration_us":16542,"operation_id":"0x2afb","timeout_ms":10000,"last_server_duration_us":9377552,"total_duration_us":10007949}],"total_count":3}} INFO [2022-01-10 05:40:01,487] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2222222222222222}, :ns_1@172.23.107.227 {:progress 0.1612903225806451}, :ns_1@172.23.105.187 {:progress 0.1612903225806451}, :ns_1@172.23.107.228 {:progress 0.2352941176470589}} INFO [2022-01-10 05:40:01,487] jepsen worker nemesis - couchbase.util Rebalance check count: 157 INFO [2022-01-10 05:40:01,487] jepsen worker nemesis - couchbase.util Rebalance stuck count: 118 INFO [2022-01-10 05:40:01,642] jepsen worker 75 - jepsen.util 1155 :ok :read [25 :nil] INFO [2022-01-10 05:40:01,642] jepsen worker 75 - jepsen.util 1155 :invoke :write [25 26] INFO [2022-01-10 05:40:01,645] jepsen worker 39 - jepsen.util 1569 :info :write [13 28] :RequestTimeoutException INFO [2022-01-10 05:40:01,645] jepsen worker 39 - jepsen.util 1659 :invoke :write [13 16] INFO [2022-01-10 05:40:01,645] jepsen worker 25 - jepsen.util 1375 :ok :read [8 :nil] INFO [2022-01-10 05:40:01,646] jepsen worker 25 - jepsen.util 1375 :invoke :write [8 15] WARN [2022-01-10 05:40:01,678] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8789861,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10423056,"total_server_duration_us":486542,"encode_duration_us":18793,"operation_id":"0x2c81","timeout_ms":10000,"last_server_duration_us":72726,"total_duration_us":10011235},{"operation_name":"upsert","last_dispatch_duration_us":2602112,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9441378,"total_server_duration_us":674458,"encode_duration_us":18477,"operation_id":"0x2bfb","timeout_ms":10000,"last_server_duration_us":390930,"total_duration_us":10009518},{"operation_name":"upsert","last_dispatch_duration_us":1062222,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":3733339,"total_server_duration_us":608956,"encode_duration_us":18312,"operation_id":"0x2cae","timeout_ms":10000,"last_server_duration_us":200428,"total_duration_us":10009104},{"operation_name":"upsert","last_dispatch_duration_us":1513580,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":2383568,"total_server_duration_us":777580,"encode_duration_us":19237,"operation_id":"0x2ced","timeout_ms":10000,"last_server_duration_us":241564,"total_duration_us":10003365},{"operation_name":"upsert","last_dispatch_duration_us":9319720,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11625168,"total_server_duration_us":1072463,"encode_duration_us":16398,"operation_id":"0x2c41","timeout_ms":10000,"last_server_duration_us":89883,"total_duration_us":10002495},{"operation_name":"upsert","last_dispatch_duration_us":8843627,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10636781,"total_server_duration_us":182607,"encode_duration_us":17403,"operation_id":"0x2c83","timeout_ms":10000,"last_server_duration_us":96865,"total_duration_us":10001514},{"operation_name":"upsert","last_dispatch_duration_us":32048,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":1009314,"total_server_duration_us":547197,"encode_duration_us":16101,"operation_id":"0x2ca4","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10001051},{"operation_name":"upsert","last_dispatch_duration_us":2870531,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":11446645,"total_server_duration_us":1410573,"encode_duration_us":16171,"operation_id":"0x2bfc","timeout_ms":10000,"last_server_duration_us":399623,"total_duration_us":10000828},{"operation_name":"get","last_dispatch_duration_us":9447633,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9447633,"total_server_duration_us":100440,"operation_id":"0x2cb5","timeout_ms":30000,"last_server_duration_us":100440,"total_duration_us":9449738},{"operation_name":"get","last_dispatch_duration_us":9349235,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9349235,"total_server_duration_us":127848,"operation_id":"0x2ccc","timeout_ms":30000,"last_server_duration_us":127848,"total_duration_us":9351900}],"total_count":21}} INFO [2022-01-10 05:40:01,895] jepsen worker 58 - jepsen.util 1318 :info :write [19 12] :RequestTimeoutException INFO [2022-01-10 05:40:01,896] jepsen worker 58 - jepsen.util 1408 :invoke :write [19 16] INFO [2022-01-10 05:40:01,899] jepsen worker 16 - jepsen.util 1636 :info :write [5 38] :RequestTimeoutException INFO [2022-01-10 05:40:01,900] jepsen worker 16 - jepsen.util 1726 :invoke :write [5 46] INFO [2022-01-10 05:40:01,956] jepsen worker 6 - jepsen.util 1356 :info :write [2 47] :RequestTimeoutException INFO [2022-01-10 05:40:01,956] jepsen worker 6 - jepsen.util 1446 :invoke :read [2 nil] INFO [2022-01-10 05:40:02,045] jepsen worker 45 - jepsen.util 1575 :info :write [15 6] :RequestTimeoutException INFO [2022-01-10 05:40:02,046] jepsen worker 45 - jepsen.util 1665 :invoke :read [15 nil] INFO [2022-01-10 05:40:02,401] jepsen worker 77 - jepsen.util 1697 :info :write [25 39] :RequestTimeoutException INFO [2022-01-10 05:40:02,402] jepsen worker 77 - jepsen.util 1787 :invoke :write [25 37] INFO [2022-01-10 05:40:02,426] jepsen worker 42 - jepsen.util 1302 :info :write [14 48] :RequestTimeoutException INFO [2022-01-10 05:40:02,432] jepsen worker 42 - jepsen.util 1392 :invoke :read [14 nil] INFO [2022-01-10 05:40:02,495] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2280701754385965}, :ns_1@172.23.107.227 {:progress 0.1671554252199413}, :ns_1@172.23.105.187 {:progress 0.1671554252199413}, :ns_1@172.23.107.228 {:progress 0.2352941176470589}} INFO [2022-01-10 05:40:02,496] jepsen worker nemesis - couchbase.util Rebalance check count: 158 INFO [2022-01-10 05:40:02,496] jepsen worker nemesis - couchbase.util Rebalance stuck count: 118 INFO [2022-01-10 05:40:02,676] jepsen worker 20 - jepsen.util 1190 :info :write [6 42] :RequestTimeoutException INFO [2022-01-10 05:40:02,677] jepsen worker 20 - jepsen.util 1280 :invoke :read [6 nil] INFO [2022-01-10 05:40:02,690] jepsen worker 36 - jepsen.util 1026 :info :write [12 1] :RequestTimeoutException INFO [2022-01-10 05:40:02,690] jepsen worker 36 - jepsen.util 1116 :invoke :read [12 nil] INFO [2022-01-10 05:40:02,771] jepsen worker 63 - jepsen.util 1233 :info :write [21 26] :RequestTimeoutException INFO [2022-01-10 05:40:02,772] jepsen worker 63 - jepsen.util 1323 :invoke :write [21 18] INFO [2022-01-10 05:40:02,826] jepsen worker 33 - jepsen.util 1383 :info :write [11 48] :RequestTimeoutException INFO [2022-01-10 05:40:02,827] jepsen worker 33 - jepsen.util 1473 :invoke :write [11 24] INFO [2022-01-10 05:40:02,841] jepsen worker 46 - jepsen.util 1486 :info :write [15 34] :RequestTimeoutException INFO [2022-01-10 05:40:02,843] jepsen worker 46 - jepsen.util 1576 :invoke :write [15 49] INFO [2022-01-10 05:40:03,356] jepsen worker 53 - jepsen.util 1403 :info :write [17 20] :RequestTimeoutException INFO [2022-01-10 05:40:03,357] jepsen worker 53 - jepsen.util 1493 :invoke :read [17 nil] INFO [2022-01-10 05:40:03,389] jepsen worker 20 - jepsen.util 1280 :ok :read [6 25] INFO [2022-01-10 05:40:03,389] jepsen worker 20 - jepsen.util 1280 :invoke :write [6 16] INFO [2022-01-10 05:40:03,503] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2280701754385965}, :ns_1@172.23.107.227 {:progress 0.1671554252199413}, :ns_1@172.23.105.187 {:progress 0.1671554252199413}, :ns_1@172.23.107.228 {:progress 0.2352941176470589}} INFO [2022-01-10 05:40:03,503] jepsen worker nemesis - couchbase.util Rebalance check count: 159 INFO [2022-01-10 05:40:03,503] jepsen worker nemesis - couchbase.util Rebalance stuck count: 119 INFO [2022-01-10 05:40:03,610] jepsen worker 84 - jepsen.util 1434 :info :write [28 29] :RequestTimeoutException INFO [2022-01-10 05:40:03,610] jepsen worker 84 - jepsen.util 1524 :invoke :read [28 nil] INFO [2022-01-10 05:40:03,824] jepsen worker 66 - jepsen.util 1236 :info :write [22 7] :RequestTimeoutException INFO [2022-01-10 05:40:03,825] jepsen worker 66 - jepsen.util 1326 :invoke :read [22 nil] INFO [2022-01-10 05:40:03,835] jepsen worker 41 - jepsen.util 1121 :info :write [13 16] :RequestTimeoutException INFO [2022-01-10 05:40:03,836] jepsen worker 41 - jepsen.util 1211 :invoke :write [13 3] INFO [2022-01-10 05:40:03,860] jepsen worker 84 - jepsen.util 1524 :ok :read [28 13] INFO [2022-01-10 05:40:03,861] jepsen worker 84 - jepsen.util 1524 :invoke :write [28 39] INFO [2022-01-10 05:40:04,184] jepsen worker 27 - jepsen.util 1377 :ok :read [9 22] INFO [2022-01-10 05:40:04,185] jepsen worker 27 - jepsen.util 1377 :invoke :read [9 nil] INFO [2022-01-10 05:40:04,422] jepsen worker 53 - jepsen.util 1493 :ok :read [17 41] INFO [2022-01-10 05:40:04,423] jepsen worker 53 - jepsen.util 1493 :invoke :read [17 nil] INFO [2022-01-10 05:40:04,456] jepsen worker 86 - jepsen.util 1166 :ok :read [28 13] INFO [2022-01-10 05:40:04,456] jepsen worker 86 - jepsen.util 1166 :invoke :write [28 5] INFO [2022-01-10 05:40:04,509] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2280701754385965}, :ns_1@172.23.107.227 {:progress 0.1700879765395894}, :ns_1@172.23.105.187 {:progress 0.1700879765395894}, :ns_1@172.23.107.228 {:progress 0.2352941176470589}} INFO [2022-01-10 05:40:04,509] jepsen worker nemesis - couchbase.util Rebalance check count: 160 INFO [2022-01-10 05:40:04,509] jepsen worker nemesis - couchbase.util Rebalance stuck count: 119 INFO [2022-01-10 05:40:04,609] jepsen worker 80 - jepsen.util 1520 :ok :read [26 10] INFO [2022-01-10 05:40:04,609] jepsen worker 80 - jepsen.util 1520 :invoke :read [26 nil] INFO [2022-01-10 05:40:04,656] jepsen worker 68 - jepsen.util 1328 :info :write [22 0] :RequestTimeoutException INFO [2022-01-10 05:40:04,657] jepsen worker 68 - jepsen.util 1418 :invoke :write [22 17] INFO [2022-01-10 05:40:04,750] jepsen worker 42 - jepsen.util 1392 :ok :read [14 14] INFO [2022-01-10 05:40:04,751] jepsen worker 42 - jepsen.util 1392 :invoke :write [14 41] INFO [2022-01-10 05:40:04,812] jepsen worker 23 - jepsen.util 1463 :info :write [7 30] :RequestTimeoutException INFO [2022-01-10 05:40:04,813] jepsen worker 23 - jepsen.util 1553 :invoke :read [7 nil] INFO [2022-01-10 05:40:05,124] jepsen worker 74 - jepsen.util 1784 :info :write [24 23] :RequestTimeoutException INFO [2022-01-10 05:40:05,125] jepsen worker 74 - jepsen.util 1874 :invoke :write [24 41] INFO [2022-01-10 05:40:05,249] jepsen worker 80 - jepsen.util 1520 :ok :read [26 10] INFO [2022-01-10 05:40:05,250] jepsen worker 80 - jepsen.util 1520 :invoke :write [26 47] INFO [2022-01-10 05:40:05,431] jepsen worker 71 - jepsen.util 1511 :info :write [23 48] :RequestTimeoutException INFO [2022-01-10 05:40:05,431] jepsen worker 71 - jepsen.util 1601 :invoke :read [23 nil] INFO [2022-01-10 05:40:05,517] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2280701754385965}, :ns_1@172.23.107.227 {:progress 0.1700879765395894}, :ns_1@172.23.105.187 {:progress 0.1700879765395894}, :ns_1@172.23.107.228 {:progress 0.2352941176470589}} INFO [2022-01-10 05:40:05,517] jepsen worker nemesis - couchbase.util Rebalance check count: 161 INFO [2022-01-10 05:40:05,517] jepsen worker nemesis - couchbase.util Rebalance stuck count: 120 INFO [2022-01-10 05:40:05,535] jepsen worker 27 - jepsen.util 1377 :ok :read [9 22] INFO [2022-01-10 05:40:05,536] jepsen worker 27 - jepsen.util 1377 :invoke :write [9 45] INFO [2022-01-10 05:40:05,832] jepsen worker 55 - jepsen.util 1585 :info :write [18 41] :RequestTimeoutException INFO [2022-01-10 05:40:05,832] jepsen worker 55 - jepsen.util 1675 :invoke :read [18 nil] INFO [2022-01-10 05:40:05,846] jepsen worker 53 - jepsen.util 1493 :ok :read [17 41] INFO [2022-01-10 05:40:05,847] jepsen worker 53 - jepsen.util 1493 :invoke :write [17 22] INFO [2022-01-10 05:40:05,866] jepsen worker 76 - jepsen.util 1066 :info :write [25 43] :RequestTimeoutException INFO [2022-01-10 05:40:05,867] jepsen worker 76 - jepsen.util 1156 :invoke :read [25 nil] INFO [2022-01-10 05:40:05,905] jepsen worker 1 - jepsen.util 1441 :info :write [0 22] :RequestTimeoutException INFO [2022-01-10 05:40:05,906] jepsen worker 1 - jepsen.util 1531 :invoke :read [0 nil] INFO [2022-01-10 05:40:06,167] jepsen worker 29 - jepsen.util 1379 :info :write [9 18] :RequestTimeoutException INFO [2022-01-10 05:40:06,167] jepsen worker 29 - jepsen.util 1469 :invoke :write [9 19] INFO [2022-01-10 05:40:06,241] jepsen worker 10 - jepsen.util 1540 :info :write [3 26] :RequestTimeoutException INFO [2022-01-10 05:40:06,242] jepsen worker 10 - jepsen.util 1630 :invoke :read [3 nil] INFO [2022-01-10 05:40:06,286] jepsen worker 48 - jepsen.util 1128 :info :write [16 47] :RequestTimeoutException INFO [2022-01-10 05:40:06,288] jepsen worker 48 - jepsen.util 1218 :invoke :read [16 nil] INFO [2022-01-10 05:40:06,316] jepsen worker 49 - jepsen.util 1129 :info :write [16 38] :RequestTimeoutException INFO [2022-01-10 05:40:06,317] jepsen worker 49 - jepsen.util 1219 :invoke :write [16 23] INFO [2022-01-10 05:40:06,318] jepsen worker 1 - jepsen.util 1531 :ok :read [0 46] INFO [2022-01-10 05:40:06,319] jepsen worker 1 - jepsen.util 1531 :invoke :write [0 21] INFO [2022-01-10 05:40:06,523] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2280701754385965}, :ns_1@172.23.107.227 {:progress 0.1700879765395894}, :ns_1@172.23.105.187 {:progress 0.1700879765395894}, :ns_1@172.23.107.228 {:progress 0.2352941176470589}} INFO [2022-01-10 05:40:06,523] jepsen worker nemesis - couchbase.util Rebalance check count: 162 INFO [2022-01-10 05:40:06,524] jepsen worker nemesis - couchbase.util Rebalance stuck count: 121 INFO [2022-01-10 05:40:06,656] jepsen worker 9 - jepsen.util 1629 :info :write [3 10] :SyncWriteAmbiguous INFO [2022-01-10 05:40:06,656] jepsen worker 9 - jepsen.util 1719 :invoke :read [3 nil] INFO [2022-01-10 05:40:06,684] jepsen worker 36 - jepsen.util 1116 :ok :read [12 :nil] INFO [2022-01-10 05:40:06,687] jepsen worker 36 - jepsen.util 1116 :invoke :read [12 nil] INFO [2022-01-10 05:40:06,722] jepsen worker 70 - jepsen.util 1780 :ok :read [23 26] INFO [2022-01-10 05:40:06,723] jepsen worker 70 - jepsen.util 1780 :invoke :read [23 nil] INFO [2022-01-10 05:40:06,726] jepsen worker 55 - jepsen.util 1675 :ok :read [18 14] INFO [2022-01-10 05:40:06,727] jepsen worker 55 - jepsen.util 1675 :invoke :read [18 nil] INFO [2022-01-10 05:40:06,732] jepsen worker 88 - jepsen.util 1438 :ok :read [29 33] INFO [2022-01-10 05:40:06,733] jepsen worker 88 - jepsen.util 1438 :invoke :read [29 nil] INFO [2022-01-10 05:40:06,736] jepsen worker 30 - jepsen.util 1110 :ok :read [10 37] INFO [2022-01-10 05:40:06,737] jepsen worker 30 - jepsen.util 1110 :invoke :read [10 nil] INFO [2022-01-10 05:40:06,741] jepsen worker 36 - jepsen.util 1116 :ok :read [12 1] INFO [2022-01-10 05:40:06,742] jepsen worker 36 - jepsen.util 1116 :invoke :write [12 27] INFO [2022-01-10 05:40:06,751] jepsen worker 5 - jepsen.util 1445 :ok :read [1 41] INFO [2022-01-10 05:40:06,759] jepsen worker 55 - jepsen.util 1675 :ok :read [18 14] INFO [2022-01-10 05:40:06,760] jepsen worker 48 - jepsen.util 1218 :ok :read [16 38] INFO [2022-01-10 05:40:06,760] jepsen worker 55 - jepsen.util 1675 :invoke :read [18 nil] INFO [2022-01-10 05:40:06,760] jepsen worker 5 - jepsen.util 1445 :invoke :read [1 nil] INFO [2022-01-10 05:40:06,760] jepsen worker 48 - jepsen.util 1218 :invoke :write [16 32] INFO [2022-01-10 05:40:06,771] jepsen worker 70 - jepsen.util 1780 :ok :read [23 26] INFO [2022-01-10 05:40:06,771] jepsen worker 70 - jepsen.util 1780 :invoke :write [23 4] INFO [2022-01-10 05:40:06,773] jepsen worker 10 - jepsen.util 1630 :ok :read [3 2] INFO [2022-01-10 05:40:06,774] jepsen worker 10 - jepsen.util 1630 :invoke :read [3 nil] INFO [2022-01-10 05:40:06,785] jepsen worker 88 - jepsen.util 1438 :ok :read [29 33] INFO [2022-01-10 05:40:06,786] jepsen worker 88 - jepsen.util 1438 :invoke :read [29 nil] INFO [2022-01-10 05:40:06,804] jepsen worker 30 - jepsen.util 1110 :ok :read [10 37] INFO [2022-01-10 05:40:06,805] jepsen worker 30 - jepsen.util 1110 :invoke :write [10 47] INFO [2022-01-10 05:40:06,805] jepsen worker 55 - jepsen.util 1675 :ok :read [18 14] INFO [2022-01-10 05:40:06,805] jepsen worker 55 - jepsen.util 1675 :invoke :write [18 5] INFO [2022-01-10 05:40:06,814] jepsen worker 5 - jepsen.util 1445 :ok :read [1 41] INFO [2022-01-10 05:40:06,815] jepsen worker 5 - jepsen.util 1445 :invoke :write [1 44] INFO [2022-01-10 05:40:06,840] jepsen worker 10 - jepsen.util 1630 :ok :read [3 2] INFO [2022-01-10 05:40:06,840] jepsen worker 10 - jepsen.util 1630 :invoke :read [3 nil] INFO [2022-01-10 05:40:06,857] jepsen worker 88 - jepsen.util 1438 :ok :read [29 33] INFO [2022-01-10 05:40:06,860] jepsen worker 88 - jepsen.util 1438 :invoke :read [29 nil] INFO [2022-01-10 05:40:06,990] jepsen worker 10 - jepsen.util 1630 :ok :read [3 2] INFO [2022-01-10 05:40:06,990] jepsen worker 10 - jepsen.util 1630 :invoke :read [3 nil] INFO [2022-01-10 05:40:07,001] jepsen worker 88 - jepsen.util 1438 :ok :read [29 33] INFO [2022-01-10 05:40:07,002] jepsen worker 88 - jepsen.util 1438 :invoke :read [29 nil] INFO [2022-01-10 05:40:07,052] jepsen worker 2 - jepsen.util 1442 :info :write [0 22] :RequestTimeoutException INFO [2022-01-10 05:40:07,052] jepsen worker 2 - jepsen.util 1532 :invoke :read [0 nil] INFO [2022-01-10 05:40:07,074] jepsen worker 10 - jepsen.util 1630 :ok :read [3 2] INFO [2022-01-10 05:40:07,075] jepsen worker 10 - jepsen.util 1630 :invoke :read [3 nil] INFO [2022-01-10 05:40:07,087] jepsen worker 40 - jepsen.util 1120 :info :write [13 28] :RequestTimeoutException INFO [2022-01-10 05:40:07,087] jepsen worker 40 - jepsen.util 1210 :invoke :read [13 nil] INFO [2022-01-10 05:40:07,102] jepsen worker 88 - jepsen.util 1438 :ok :read [29 33] INFO [2022-01-10 05:40:07,102] jepsen worker 88 - jepsen.util 1438 :invoke :read [29 nil] INFO [2022-01-10 05:40:07,143] jepsen worker 10 - jepsen.util 1630 :ok :read [3 2] INFO [2022-01-10 05:40:07,144] jepsen worker 10 - jepsen.util 1630 :invoke :read [3 nil] INFO [2022-01-10 05:40:07,185] jepsen worker 88 - jepsen.util 1438 :ok :read [29 33] INFO [2022-01-10 05:40:07,185] jepsen worker 88 - jepsen.util 1438 :invoke :write [29 19] INFO [2022-01-10 05:40:07,222] jepsen worker 10 - jepsen.util 1630 :ok :read [3 2] INFO [2022-01-10 05:40:07,222] jepsen worker 10 - jepsen.util 1630 :invoke :write [3 39] INFO [2022-01-10 05:40:07,295] jepsen worker 89 - jepsen.util 1619 :info :write [29 25] :RequestTimeoutException INFO [2022-01-10 05:40:07,296] jepsen worker 89 - jepsen.util 1709 :invoke :write [29 24] INFO [2022-01-10 05:40:07,362] jepsen worker 72 - jepsen.util 1602 :ok :read [24 18] INFO [2022-01-10 05:40:07,364] jepsen worker 72 - jepsen.util 1602 :invoke :read [24 nil] INFO [2022-01-10 05:40:07,489] jepsen worker 85 - jepsen.util 1615 :info :write [28 47] :RequestTimeoutException INFO [2022-01-10 05:40:07,507] jepsen worker 6 - jepsen.util 1446 :ok :read [2 14] INFO [2022-01-10 05:40:07,554] jepsen worker 85 - jepsen.util 1705 :invoke :read [28 nil] INFO [2022-01-10 05:40:07,554] jepsen worker 6 - jepsen.util 1446 :invoke :read [2 nil] INFO [2022-01-10 05:40:07,531] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2339181286549707}, :ns_1@172.23.107.227 {:progress 0.1788856304985337}, :ns_1@172.23.105.187 {:progress 0.1788856304985337}, :ns_1@172.23.107.228 {:progress 0.2411764705882353}} INFO [2022-01-10 05:40:07,555] jepsen worker nemesis - couchbase.util Rebalance check count: 163 INFO [2022-01-10 05:40:07,555] jepsen worker nemesis - couchbase.util Rebalance stuck count: 121 INFO [2022-01-10 05:40:07,610] jepsen worker 9 - jepsen.util 1719 :ok :read [3 2] INFO [2022-01-10 05:40:07,611] jepsen worker 9 - jepsen.util 1719 :invoke :write [3 31] INFO [2022-01-10 05:40:07,644] jepsen worker 72 - jepsen.util 1602 :ok :read [24 18] INFO [2022-01-10 05:40:07,649] jepsen worker 2 - jepsen.util 1532 :ok :read [0 46] INFO [2022-01-10 05:40:07,650] jepsen worker 72 - jepsen.util 1602 :invoke :read [24 nil] INFO [2022-01-10 05:40:07,650] jepsen worker 2 - jepsen.util 1532 :invoke :write [0 16] INFO [2022-01-10 05:40:07,688] jepsen worker 23 - jepsen.util 1553 :ok :read [7 :nil] INFO [2022-01-10 05:40:07,688] jepsen worker 21 - jepsen.util 1641 :ok :read [7 :nil] INFO [2022-01-10 05:40:07,689] jepsen worker 23 - jepsen.util 1553 :invoke :read [7 nil] INFO [2022-01-10 05:40:07,689] jepsen worker 21 - jepsen.util 1641 :invoke :read [7 nil] INFO [2022-01-10 05:40:07,691] jepsen worker 73 - jepsen.util 1423 :info :write [24 15] :RequestTimeoutException INFO [2022-01-10 05:40:07,692] jepsen worker 73 - jepsen.util 1513 :invoke :read [24 nil] INFO [2022-01-10 05:40:07,727] jepsen worker 85 - jepsen.util 1705 :ok :read [28 13] INFO [2022-01-10 05:40:07,728] jepsen worker 85 - jepsen.util 1705 :invoke :read [28 nil] INFO [2022-01-10 05:40:07,731] jepsen worker 6 - jepsen.util 1446 :ok :read [2 14] INFO [2022-01-10 05:40:07,732] jepsen worker 6 - jepsen.util 1446 :invoke :write [2 47] INFO [2022-01-10 05:40:07,843] jepsen worker 72 - jepsen.util 1602 :ok :read [24 18] INFO [2022-01-10 05:40:07,843] jepsen worker 72 - jepsen.util 1602 :invoke :read [24 nil] INFO [2022-01-10 05:40:07,859] jepsen worker 40 - jepsen.util 1210 :ok :read [13 15] INFO [2022-01-10 05:40:07,860] jepsen worker 40 - jepsen.util 1210 :invoke :write [13 45] INFO [2022-01-10 05:40:07,863] jepsen worker 21 - jepsen.util 1641 :ok :read [7 :nil] INFO [2022-01-10 05:40:07,864] jepsen worker 21 - jepsen.util 1641 :invoke :write [7 42] INFO [2022-01-10 05:40:07,870] jepsen worker 85 - jepsen.util 1705 :ok :read [28 13] INFO [2022-01-10 05:40:07,871] jepsen worker 85 - jepsen.util 1705 :invoke :write [28 24] INFO [2022-01-10 05:40:07,902] jepsen worker 23 - jepsen.util 1553 :ok :read [7 :nil] INFO [2022-01-10 05:40:07,903] jepsen worker 23 - jepsen.util 1553 :invoke :read [7 nil] INFO [2022-01-10 05:40:07,970] jepsen worker 32 - jepsen.util 1382 :ok :read [10 37] INFO [2022-01-10 05:40:07,979] jepsen worker 4 - jepsen.util 1444 :ok :read [1 41] INFO [2022-01-10 05:40:07,981] jepsen worker 32 - jepsen.util 1382 :invoke :write [10 20] INFO [2022-01-10 05:40:07,983] jepsen worker 4 - jepsen.util 1444 :invoke :read [1 nil] INFO [2022-01-10 05:40:07,984] jepsen worker 59 - jepsen.util 1499 :ok :read [19 31] INFO [2022-01-10 05:40:07,984] jepsen worker 59 - jepsen.util 1499 :invoke :read [19 nil] INFO [2022-01-10 05:40:08,007] jepsen worker 72 - jepsen.util 1602 :ok :read [24 18] INFO [2022-01-10 05:40:08,007] jepsen worker 72 - jepsen.util 1602 :invoke :write [24 47] INFO [2022-01-10 05:40:08,040] jepsen worker 23 - jepsen.util 1553 :ok :read [7 :nil] INFO [2022-01-10 05:40:08,041] jepsen worker 23 - jepsen.util 1553 :invoke :write [7 22] INFO [2022-01-10 05:40:08,053] jepsen worker 14 - jepsen.util 1544 :fail :read [4 nil] :UnambiguousTimeoutException INFO [2022-01-10 05:40:08,054] jepsen worker 14 - jepsen.util 1544 :invoke :write [4 23] INFO [2022-01-10 05:40:08,094] jepsen worker 81 - jepsen.util 1431 :ok :read [27 30] INFO [2022-01-10 05:40:08,094] jepsen worker 81 - jepsen.util 1431 :invoke :read [27 nil] INFO [2022-01-10 05:40:08,111] jepsen worker 76 - jepsen.util 1156 :ok :read [25 :nil] INFO [2022-01-10 05:40:08,111] jepsen worker 76 - jepsen.util 1156 :invoke :write [25 15] INFO [2022-01-10 05:40:08,186] jepsen worker 4 - jepsen.util 1444 :ok :read [1 41] INFO [2022-01-10 05:40:08,187] jepsen worker 4 - jepsen.util 1444 :invoke :write [1 40] INFO [2022-01-10 05:40:08,276] jepsen worker 8 - jepsen.util 1448 :info :write [2 31] :RequestTimeoutException INFO [2022-01-10 05:40:08,277] jepsen worker 8 - jepsen.util 1538 :invoke :write [2 26] INFO [2022-01-10 05:40:08,382] jepsen worker 31 - jepsen.util 1741 :info :write [10 48] :RequestTimeoutException INFO [2022-01-10 05:40:08,382] jepsen worker 31 - jepsen.util 1831 :invoke :read [10 nil] INFO [2022-01-10 05:40:08,391] jepsen worker 19 - jepsen.util 1549 :info :write [6 34] :RequestTimeoutException INFO [2022-01-10 05:40:08,392] jepsen worker 19 - jepsen.util 1639 :invoke :read [6 nil] INFO [2022-01-10 05:40:08,456] jepsen worker 38 - jepsen.util 1478 :info :write [12 45] :RequestTimeoutException INFO [2022-01-10 05:40:08,457] jepsen worker 38 - jepsen.util 1568 :invoke :read [12 nil] INFO [2022-01-10 05:40:08,494] jepsen worker 69 - jepsen.util 1509 :info :write [23 22] :RequestTimeoutException INFO [2022-01-10 05:40:08,495] jepsen worker 69 - jepsen.util 1599 :invoke :write [23 44] INFO [2022-01-10 05:40:08,535] jepsen worker 11 - jepsen.util 1361 :info :write [3 18] :RequestTimeoutException INFO [2022-01-10 05:40:08,536] jepsen worker 11 - jepsen.util 1451 :invoke :read [3 nil] INFO [2022-01-10 05:40:08,567] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2339181286549707}, :ns_1@172.23.107.227 {:progress 0.1788856304985337}, :ns_1@172.23.105.187 {:progress 0.1788856304985337}, :ns_1@172.23.107.228 {:progress 0.2411764705882353}} INFO [2022-01-10 05:40:08,567] jepsen worker nemesis - couchbase.util Rebalance check count: 164 INFO [2022-01-10 05:40:08,567] jepsen worker nemesis - couchbase.util Rebalance stuck count: 122 INFO [2022-01-10 05:40:08,586] jepsen worker 51 - jepsen.util 1491 :info :write [17 32] :RequestTimeoutException INFO [2022-01-10 05:40:08,587] jepsen worker 51 - jepsen.util 1581 :invoke :write [17 39] INFO [2022-01-10 05:40:08,604] jepsen worker 60 - jepsen.util 1770 :info :write [20 47] :RequestTimeoutException INFO [2022-01-10 05:40:08,605] jepsen worker 60 - jepsen.util 1860 :invoke :write [20 26] INFO [2022-01-10 05:40:08,721] jepsen worker 44 - jepsen.util 1484 :info :write [14 16] :RequestTimeoutException INFO [2022-01-10 05:40:08,721] jepsen worker 44 - jepsen.util 1574 :invoke :write [14 9] INFO [2022-01-10 05:40:08,731] jepsen worker 19 - jepsen.util 1639 :ok :read [6 25] INFO [2022-01-10 05:40:08,731] jepsen worker 19 - jepsen.util 1639 :invoke :read [6 nil] INFO [2022-01-10 05:40:08,867] jepsen worker 19 - jepsen.util 1639 :ok :read [6 25] INFO [2022-01-10 05:40:08,868] jepsen worker 19 - jepsen.util 1639 :invoke :write [6 17] INFO [2022-01-10 05:40:08,880] jepsen worker 12 - jepsen.util 1182 :info :write [4 39] :RequestTimeoutException INFO [2022-01-10 05:40:08,880] jepsen worker 12 - jepsen.util 1272 :invoke :read [4 nil] INFO [2022-01-10 05:40:08,885] jepsen worker 37 - jepsen.util 1387 :info :write [12 39] :RequestTimeoutException INFO [2022-01-10 05:40:08,886] jepsen worker 37 - jepsen.util 1477 :invoke :write [12 43] INFO [2022-01-10 05:40:08,916] jepsen worker 87 - jepsen.util 1617 :info :write [29 45] :RequestTimeoutException INFO [2022-01-10 05:40:08,917] jepsen worker 87 - jepsen.util 1707 :invoke :read [29 nil] INFO [2022-01-10 05:40:08,932] jepsen worker 61 - jepsen.util 1321 :info :write [20 8] :RequestTimeoutException INFO [2022-01-10 05:40:08,933] jepsen worker 61 - jepsen.util 1411 :invoke :read [20 nil] INFO [2022-01-10 05:40:08,991] jepsen worker 78 - jepsen.util 1338 :info :write [26 46] :RequestTimeoutException INFO [2022-01-10 05:40:08,992] jepsen worker 78 - jepsen.util 1428 :invoke :write [26 43] INFO [2022-01-10 05:40:08,999] jepsen worker 82 - jepsen.util 1252 :info :write [27 27] :SyncWriteAmbiguous INFO [2022-01-10 05:40:09,000] jepsen worker 82 - jepsen.util 1342 :invoke :read [27 nil] INFO [2022-01-10 05:40:09,027] jepsen worker 66 - jepsen.util 1326 :ok :read [22 41] INFO [2022-01-10 05:40:09,027] jepsen worker 66 - jepsen.util 1326 :invoke :read [22 nil] INFO [2022-01-10 05:40:09,091] jepsen worker 34 - jepsen.util 1474 :info :write [11 47] :RequestTimeoutException INFO [2022-01-10 05:40:09,092] jepsen worker 34 - jepsen.util 1564 :invoke :write [11 13] INFO [2022-01-10 05:40:09,170] jepsen worker 65 - jepsen.util 1325 :info :write [21 0] :RequestTimeoutException INFO [2022-01-10 05:40:09,170] jepsen worker 65 - jepsen.util 1415 :invoke :read [21 nil] INFO [2022-01-10 05:40:09,189] jepsen worker 81 - jepsen.util 1431 :ok :read [27 30] INFO [2022-01-10 05:40:09,189] jepsen worker 81 - jepsen.util 1431 :invoke :write [27 49] INFO [2022-01-10 05:40:09,280] jepsen worker 64 - jepsen.util 1504 :info :write [21 44] :RequestTimeoutException INFO [2022-01-10 05:40:09,281] jepsen worker 64 - jepsen.util 1594 :invoke :read [21 nil] INFO [2022-01-10 05:40:09,300] jepsen worker 59 - jepsen.util 1499 :ok :read [19 31] INFO [2022-01-10 05:40:09,301] jepsen worker 59 - jepsen.util 1499 :invoke :write [19 43] INFO [2022-01-10 05:40:09,301] jepsen worker 17 - jepsen.util 1277 :info :write [5 27] :RequestTimeoutException INFO [2022-01-10 05:40:09,302] jepsen worker 17 - jepsen.util 1367 :invoke :read [5 nil] INFO [2022-01-10 05:40:09,310] jepsen worker 61 - jepsen.util 1411 :ok :read [20 :nil] INFO [2022-01-10 05:40:09,310] jepsen worker 61 - jepsen.util 1411 :invoke :read [20 nil] INFO [2022-01-10 05:40:09,385] jepsen worker 0 - jepsen.util 1350 :info :write [0 28] :RequestTimeoutException INFO [2022-01-10 05:40:09,386] jepsen worker 0 - jepsen.util 1440 :invoke :read [0 nil] INFO [2022-01-10 05:40:09,398] jepsen worker 66 - jepsen.util 1326 :ok :read [22 41] INFO [2022-01-10 05:40:09,399] jepsen worker 66 - jepsen.util 1326 :invoke :read [22 nil] INFO [2022-01-10 05:40:09,428] jepsen worker 61 - jepsen.util 1411 :ok :read [20 :nil] INFO [2022-01-10 05:40:09,429] jepsen worker 61 - jepsen.util 1411 :invoke :read [20 nil] INFO [2022-01-10 05:40:09,447] jepsen worker 62 - jepsen.util 1592 :info :write [20 47] :RequestTimeoutException INFO [2022-01-10 05:40:09,447] jepsen worker 62 - jepsen.util 1682 :invoke :write [20 41] INFO [2022-01-10 05:40:09,457] jepsen worker 43 - jepsen.util 1573 :info :write [14 15] :RequestTimeoutException INFO [2022-01-10 05:40:09,457] jepsen worker 43 - jepsen.util 1663 :invoke :write [14 36] INFO [2022-01-10 05:40:09,486] jepsen worker 79 - jepsen.util 1429 :info :write [26 8] :RequestTimeoutException INFO [2022-01-10 05:40:09,487] jepsen worker 79 - jepsen.util 1519 :invoke :read [26 nil] INFO [2022-01-10 05:40:09,496] jepsen worker 61 - jepsen.util 1411 :ok :read [20 :nil] INFO [2022-01-10 05:40:09,497] jepsen worker 61 - jepsen.util 1411 :invoke :write [20 0] INFO [2022-01-10 05:40:09,519] jepsen worker 22 - jepsen.util 1282 :info :write [7 36] :RequestTimeoutException INFO [2022-01-10 05:40:09,520] jepsen worker 22 - jepsen.util 1372 :invoke :read [7 nil] INFO [2022-01-10 05:40:09,569] jepsen worker 52 - jepsen.util 1582 :info :write [17 26] :RequestTimeoutException INFO [2022-01-10 05:40:09,570] jepsen worker 52 - jepsen.util 1672 :invoke :write [17 8] INFO [2022-01-10 05:40:09,577] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2397660818713451}, :ns_1@172.23.107.227 {:progress 0.1788856304985337}, :ns_1@172.23.105.187 {:progress 0.1788856304985337}, :ns_1@172.23.107.228 {:progress 0.2470588235294118}} INFO [2022-01-10 05:40:09,577] jepsen worker nemesis - couchbase.util Rebalance check count: 165 INFO [2022-01-10 05:40:09,577] jepsen worker nemesis - couchbase.util Rebalance stuck count: 122 INFO [2022-01-10 05:40:09,671] jepsen worker 50 - jepsen.util 1580 :info :write [16 8] :RequestTimeoutException INFO [2022-01-10 05:40:09,672] jepsen worker 50 - jepsen.util 1670 :invoke :read [16 nil] INFO [2022-01-10 05:40:09,682] jepsen worker 7 - jepsen.util 1447 :info :write [2 17] :RequestTimeoutException INFO [2022-01-10 05:40:09,682] jepsen worker 7 - jepsen.util 1537 :invoke :read [2 nil] WARN [2022-01-10 05:40:09,689] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10245082,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17120490,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10845438,"encode_duration_us":18722,"operation_id":"0x2cdc","timeout_ms":10000,"last_server_duration_us":9164043,"total_duration_us":10001445},{"operation_name":"upsert","last_dispatch_duration_us":11308345,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16282808,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":10020782,"encode_duration_us":17090,"operation_id":"0x2c44","timeout_ms":10000,"last_server_duration_us":9191820,"total_duration_us":10002926},{"operation_name":"upsert","last_dispatch_duration_us":10967334,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14819484,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10675903,"encode_duration_us":19194,"operation_id":"0x2d42","timeout_ms":10000,"last_server_duration_us":9898921,"total_duration_us":10004116},{"operation_name":"upsert","last_dispatch_duration_us":11698719,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16404360,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9781639,"encode_duration_us":17051,"operation_id":"0x2baa","timeout_ms":10000,"last_server_duration_us":9226056,"total_duration_us":10009548},{"operation_name":"upsert","last_dispatch_duration_us":10140783,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16556319,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10424980,"encode_duration_us":18809,"operation_id":"0x2ce9","timeout_ms":10000,"last_server_duration_us":9116052,"total_duration_us":10003552}],"total_count":5}} INFO [2022-01-10 05:40:09,746] jepsen worker 18 - jepsen.util 1188 :info :write [6 6] :RequestTimeoutException INFO [2022-01-10 05:40:09,747] jepsen worker 18 - jepsen.util 1278 :invoke :write [6 36] WARN [2022-01-10 05:40:09,890] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":90654,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":1094997,"total_server_duration_us":639525,"encode_duration_us":20368,"operation_id":"0x2ea5","timeout_ms":10000,"last_server_duration_us":56279,"total_duration_us":10010569},{"operation_name":"upsert","last_dispatch_duration_us":8583721,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11122547,"total_server_duration_us":405642,"encode_duration_us":17965,"operation_id":"0x2d1f","timeout_ms":10000,"last_server_duration_us":37859,"total_duration_us":10010446},{"operation_name":"upsert","last_dispatch_duration_us":8029093,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10412229,"total_server_duration_us":136430,"encode_duration_us":16729,"operation_id":"0x2da1","timeout_ms":10000,"last_server_duration_us":113793,"total_duration_us":10010297},{"operation_name":"upsert","last_dispatch_duration_us":44414,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":1436534,"total_server_duration_us":939434,"encode_duration_us":16542,"operation_id":"0x2e6c","timeout_ms":10000,"last_server_duration_us":24928,"total_duration_us":10010139},{"operation_name":"upsert","last_dispatch_duration_us":29748,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":1336898,"total_server_duration_us":472196,"encode_duration_us":17361,"operation_id":"0x2e6d","timeout_ms":10000,"last_server_duration_us":18570,"total_duration_us":10007658},{"operation_name":"upsert","last_dispatch_duration_us":384956,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2799381,"total_server_duration_us":927640,"encode_duration_us":16193,"operation_id":"0x2dd3","timeout_ms":10000,"last_server_duration_us":148276,"total_duration_us":10004547},{"operation_name":"upsert","last_dispatch_duration_us":10967334,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":14819484,"total_server_duration_us":10675903,"encode_duration_us":19194,"operation_id":"0x2d42","timeout_ms":10000,"last_server_duration_us":9898921,"total_duration_us":10004116},{"operation_name":"upsert","last_dispatch_duration_us":8751685,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8751685,"total_server_duration_us":19652,"encode_duration_us":19289,"operation_id":"0x2e71","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10004030},{"operation_name":"upsert","last_dispatch_duration_us":10140783,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":16556319,"total_server_duration_us":10424980,"encode_duration_us":18809,"operation_id":"0x2ce9","timeout_ms":10000,"last_server_duration_us":9116052,"total_duration_us":10003552},{"operation_name":"upsert","last_dispatch_duration_us":2592755,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":2592755,"total_server_duration_us":1084695,"encode_duration_us":16459,"operation_id":"0x2d40","timeout_ms":10000,"last_server_duration_us":1084695,"total_duration_us":10003134}],"total_count":16}} INFO [2022-01-10 05:40:09,906] jepsen worker 7 - jepsen.util 1537 :ok :read [2 14] INFO [2022-01-10 05:40:09,906] jepsen worker 7 - jepsen.util 1537 :invoke :write [2 21] INFO [2022-01-10 05:40:09,921] jepsen worker 28 - jepsen.util 1288 :info :write [9 19] :RequestTimeoutException INFO [2022-01-10 05:40:09,921] jepsen worker 28 - jepsen.util 1378 :invoke :write [9 31] INFO [2022-01-10 05:40:10,104] jepsen worker 83 - jepsen.util 1703 :info :write [27 31] :RequestTimeoutException INFO [2022-01-10 05:40:10,105] jepsen worker 83 - jepsen.util 1793 :invoke :read [27 nil] INFO [2022-01-10 05:40:10,105] jepsen worker 15 - jepsen.util 1095 :info :write [5 28] :RequestTimeoutException INFO [2022-01-10 05:40:10,106] jepsen worker 15 - jepsen.util 1185 :invoke :read [5 nil] INFO [2022-01-10 05:40:10,172] jepsen worker 82 - jepsen.util 1342 :ok :read [27 30] INFO [2022-01-10 05:40:10,173] jepsen worker 82 - jepsen.util 1342 :invoke :read [27 nil] INFO [2022-01-10 05:40:10,489] jepsen worker 24 - jepsen.util 1374 :info :write [8 24] :RequestTimeoutException INFO [2022-01-10 05:40:10,490] jepsen worker 24 - jepsen.util 1464 :invoke :read [8 nil] INFO [2022-01-10 05:40:10,491] jepsen worker 66 - jepsen.util 1326 :ok :read [22 41] INFO [2022-01-10 05:40:10,492] jepsen worker 66 - jepsen.util 1326 :invoke :write [22 32] INFO [2022-01-10 05:40:10,583] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2397660818713451}, :ns_1@172.23.107.227 {:progress 0.1788856304985337}, :ns_1@172.23.105.187 {:progress 0.1788856304985337}, :ns_1@172.23.107.228 {:progress 0.2470588235294118}} INFO [2022-01-10 05:40:10,583] jepsen worker nemesis - couchbase.util Rebalance check count: 166 INFO [2022-01-10 05:40:10,583] jepsen worker nemesis - couchbase.util Rebalance stuck count: 123 INFO [2022-01-10 05:40:10,592] jepsen worker 79 - jepsen.util 1519 :ok :read [26 10] INFO [2022-01-10 05:40:10,592] jepsen worker 79 - jepsen.util 1519 :invoke :write [26 19] INFO [2022-01-10 05:40:10,690] jepsen worker 47 - jepsen.util 1307 :info :write [15 18] :RequestTimeoutException INFO [2022-01-10 05:40:10,691] jepsen worker 47 - jepsen.util 1397 :invoke :read [15 nil] INFO [2022-01-10 05:40:10,692] jepsen worker 35 - jepsen.util 1385 :info :write [11 14] :RequestTimeoutException INFO [2022-01-10 05:40:10,692] jepsen worker 35 - jepsen.util 1475 :invoke :read [11 nil] INFO [2022-01-10 05:40:10,812] jepsen worker 57 - jepsen.util 1497 :info :write [19 14] :RequestTimeoutException INFO [2022-01-10 05:40:10,813] jepsen worker 57 - jepsen.util 1587 :invoke :write [19 29] INFO [2022-01-10 05:40:10,933] jepsen worker 47 - jepsen.util 1397 :ok :read [15 23] INFO [2022-01-10 05:40:10,933] jepsen worker 47 - jepsen.util 1397 :invoke :read [15 nil] INFO [2022-01-10 05:40:10,964] jepsen worker 47 - jepsen.util 1397 :ok :read [15 23] INFO [2022-01-10 05:40:10,964] jepsen worker 47 - jepsen.util 1397 :invoke :write [15 44] INFO [2022-01-10 05:40:10,973] jepsen worker 26 - jepsen.util 1196 :info :write [8 10] :RequestTimeoutException INFO [2022-01-10 05:40:10,976] jepsen worker 26 - jepsen.util 1286 :invoke :read [8 nil] INFO [2022-01-10 05:40:11,149] jepsen worker 17 - jepsen.util 1367 :ok :read [5 42] INFO [2022-01-10 05:40:11,150] jepsen worker 17 - jepsen.util 1367 :invoke :read [5 nil] INFO [2022-01-10 05:40:11,164] jepsen worker 15 - jepsen.util 1185 :ok :read [5 42] INFO [2022-01-10 05:40:11,164] jepsen worker 15 - jepsen.util 1185 :invoke :write [5 8] INFO [2022-01-10 05:40:11,165] jepsen worker 54 - jepsen.util 1584 :info :write [18 8] :RequestTimeoutException INFO [2022-01-10 05:40:11,166] jepsen worker 54 - jepsen.util 1674 :invoke :read [18 nil] INFO [2022-01-10 05:40:11,195] jepsen worker 54 - jepsen.util 1674 :ok :read [18 14] INFO [2022-01-10 05:40:11,196] jepsen worker 54 - jepsen.util 1674 :invoke :read [18 nil] INFO [2022-01-10 05:40:11,201] jepsen worker 82 - jepsen.util 1342 :ok :read [27 30] INFO [2022-01-10 05:40:11,202] jepsen worker 82 - jepsen.util 1342 :invoke :read [27 nil] INFO [2022-01-10 05:40:11,222] jepsen worker 54 - jepsen.util 1674 :ok :read [18 14] INFO [2022-01-10 05:40:11,223] jepsen worker 54 - jepsen.util 1674 :invoke :read [18 nil] INFO [2022-01-10 05:40:11,245] jepsen worker 54 - jepsen.util 1674 :ok :read [18 14] INFO [2022-01-10 05:40:11,246] jepsen worker 54 - jepsen.util 1674 :invoke :read [18 nil] INFO [2022-01-10 05:40:11,266] jepsen worker 56 - jepsen.util 1586 :info :write [18 47] :RequestTimeoutException INFO [2022-01-10 05:40:11,266] jepsen worker 56 - jepsen.util 1676 :invoke :write [18 46] INFO [2022-01-10 05:40:11,270] jepsen worker 54 - jepsen.util 1674 :ok :read [18 14] INFO [2022-01-10 05:40:11,271] jepsen worker 54 - jepsen.util 1674 :invoke :read [18 nil] WARN [2022-01-10 05:40:11,295] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11869147,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14421161,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9366224,"encode_duration_us":17003,"operation_id":"0x2d72","timeout_ms":10000,"last_server_duration_us":9215351,"total_duration_us":10001137},{"operation_name":"upsert","last_dispatch_duration_us":12672111,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17185572,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9763214,"encode_duration_us":16312,"operation_id":"0x2c0e","timeout_ms":10000,"last_server_duration_us":9368929,"total_duration_us":10007204},{"operation_name":"upsert","last_dispatch_duration_us":16799413,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16799413,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9027816,"encode_duration_us":16254,"operation_id":"0x2d14","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10002912}],"total_count":3}} INFO [2022-01-10 05:40:11,296] jepsen worker 54 - jepsen.util 1674 :ok :read [18 14] INFO [2022-01-10 05:40:11,297] jepsen worker 54 - jepsen.util 1674 :invoke :write [18 38] WARN [2022-01-10 05:40:11,392] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9637046,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11153628,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9364410,"encode_duration_us":16530,"operation_id":"0x2d63","timeout_ms":10000,"last_server_duration_us":9249624,"total_duration_us":10009077},{"operation_name":"upsert","last_dispatch_duration_us":9471432,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12053184,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9302450,"encode_duration_us":16941,"operation_id":"0x2dda","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10010480},{"operation_name":"upsert","last_dispatch_duration_us":9580788,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9580788,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9423962,"encode_duration_us":18341,"operation_id":"0x2cf0","timeout_ms":10000,"last_server_duration_us":9423962,"total_duration_us":10009833}],"total_count":3}} WARN [2022-01-10 05:40:11,392] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":310469,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":19302933,"total_server_duration_us":2363720,"operation_id":"0x2a90","timeout_ms":30000,"last_server_duration_us":309785,"total_duration_us":30008991},{"operation_name":"upsert","last_dispatch_duration_us":9471432,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":12053184,"total_server_duration_us":9302450,"encode_duration_us":16941,"operation_id":"0x2dda","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10010480},{"operation_name":"upsert","last_dispatch_duration_us":9395186,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9427057,"total_server_duration_us":66922,"encode_duration_us":16596,"operation_id":"0x2efa","timeout_ms":10000,"last_server_duration_us":47192,"total_duration_us":10009641},{"operation_name":"upsert","last_dispatch_duration_us":8439108,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9370815,"total_server_duration_us":418074,"encode_duration_us":18271,"operation_id":"0x2ea4","timeout_ms":10000,"last_server_duration_us":126291,"total_duration_us":10009098},{"operation_name":"upsert","last_dispatch_duration_us":9637046,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11153628,"total_server_duration_us":9364410,"encode_duration_us":16530,"operation_id":"0x2d63","timeout_ms":10000,"last_server_duration_us":9249624,"total_duration_us":10009077},{"operation_name":"upsert","last_dispatch_duration_us":173331,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9983132,"total_server_duration_us":425517,"encode_duration_us":18509,"operation_id":"0x2e3e","timeout_ms":10000,"last_server_duration_us":59249,"total_duration_us":10009032},{"operation_name":"upsert","last_dispatch_duration_us":209872,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":849841,"total_server_duration_us":327272,"encode_duration_us":16254,"operation_id":"0x2e69","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10008744},{"operation_name":"upsert","last_dispatch_duration_us":8272344,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":13168837,"total_server_duration_us":1167288,"encode_duration_us":16416,"operation_id":"0x2d73","timeout_ms":10000,"last_server_duration_us":22389,"total_duration_us":10008131},{"operation_name":"upsert","last_dispatch_duration_us":57876,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":719080,"total_server_duration_us":197658,"encode_duration_us":17246,"operation_id":"0x2ead","timeout_ms":10000,"last_server_duration_us":21895,"total_duration_us":10006903},{"operation_name":"upsert","last_dispatch_duration_us":182501,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":360645,"total_server_duration_us":77033,"encode_duration_us":24292,"operation_id":"0x2e97","timeout_ms":10000,"last_server_duration_us":36017,"total_duration_us":10006200}],"total_count":21}} WARN [2022-01-10 05:40:11,392] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11716236,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13993517,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9348944,"encode_duration_us":16884,"operation_id":"0x2d50","timeout_ms":10000,"last_server_duration_us":8982248,"total_duration_us":10004412},{"operation_name":"upsert","last_dispatch_duration_us":10506656,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10506656,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9250696,"encode_duration_us":16222,"operation_id":"0x2d3f","timeout_ms":10000,"last_server_duration_us":9250696,"total_duration_us":10009324}],"total_count":2}} WARN [2022-01-10 05:40:11,392] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10059574,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10551494,"total_server_duration_us":9776670,"encode_duration_us":16811,"operation_id":"0x2e31","timeout_ms":10000,"last_server_duration_us":9672760,"total_duration_us":10010325},{"operation_name":"upsert","last_dispatch_duration_us":306325,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":4101065,"total_server_duration_us":1496950,"encode_duration_us":18347,"operation_id":"0x2d7b","timeout_ms":10000,"last_server_duration_us":292549,"total_duration_us":10009772},{"operation_name":"upsert","last_dispatch_duration_us":9904649,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11621513,"total_server_duration_us":956786,"encode_duration_us":16658,"operation_id":"0x2e07","timeout_ms":10000,"last_server_duration_us":43618,"total_duration_us":10009667},{"operation_name":"upsert","last_dispatch_duration_us":885351,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1616754,"total_server_duration_us":555555,"encode_duration_us":18187,"operation_id":"0x2ebb","timeout_ms":10000,"last_server_duration_us":319199,"total_duration_us":10009580},{"operation_name":"upsert","last_dispatch_duration_us":903610,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":3158066,"total_server_duration_us":514416,"encode_duration_us":17022,"operation_id":"0x2ea6","timeout_ms":10000,"last_server_duration_us":57632,"total_duration_us":10008869},{"operation_name":"upsert","last_dispatch_duration_us":1101201,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1951461,"total_server_duration_us":362113,"encode_duration_us":16304,"operation_id":"0x2dfd","timeout_ms":10000,"last_server_duration_us":79120,"total_duration_us":10005800},{"operation_name":"upsert","last_dispatch_duration_us":1303908,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":2864747,"total_server_duration_us":962894,"encode_duration_us":16881,"operation_id":"0x2ea7","timeout_ms":10000,"last_server_duration_us":603915,"total_duration_us":10005287},{"operation_name":"upsert","last_dispatch_duration_us":9606825,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10734191,"total_server_duration_us":9151489,"encode_duration_us":17067,"operation_id":"0x2e0c","timeout_ms":10000,"last_server_duration_us":9107531,"total_duration_us":10004405},{"operation_name":"upsert","last_dispatch_duration_us":9551722,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11575096,"total_server_duration_us":562106,"encode_duration_us":17621,"operation_id":"0x2e0b","timeout_ms":10000,"last_server_duration_us":40482,"total_duration_us":10002845},{"operation_name":"upsert","last_dispatch_duration_us":322245,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":789490,"total_server_duration_us":578794,"encode_duration_us":18569,"operation_id":"0x2e6f","timeout_ms":10000,"last_server_duration_us":311303,"total_duration_us":10002511}],"total_count":19}} WARN [2022-01-10 05:40:11,395] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12172321,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":12172321,"total_server_duration_us":32,"operation_id":"0x2ded","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":12174245},{"operation_name":"get","last_dispatch_duration_us":12012238,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":12012238,"total_server_duration_us":15,"operation_id":"0x2dff","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":12014889},{"operation_name":"get","last_dispatch_duration_us":10292043,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10292043,"total_server_duration_us":32,"operation_id":"0x2da2","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":10293986},{"operation_name":"upsert","last_dispatch_duration_us":1026224,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10059402,"total_server_duration_us":1027696,"encode_duration_us":16489,"operation_id":"0x2ed1","timeout_ms":10000,"last_server_duration_us":34911,"total_duration_us":10009399},{"operation_name":"upsert","last_dispatch_duration_us":127217,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9821758,"total_server_duration_us":110386,"encode_duration_us":25616,"operation_id":"0x2e92","timeout_ms":10000,"last_server_duration_us":22720,"total_duration_us":10009155},{"operation_name":"upsert","last_dispatch_duration_us":157876,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10091545,"total_server_duration_us":79182,"encode_duration_us":25239,"operation_id":"0x2e6a","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10008977},{"operation_name":"upsert","last_dispatch_duration_us":128885,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9822422,"total_server_duration_us":101468,"encode_duration_us":16588,"operation_id":"0x2e93","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10007508},{"operation_name":"upsert","last_dispatch_duration_us":2215037,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":10080869,"total_server_duration_us":1262606,"encode_duration_us":17203,"operation_id":"0x2ea8","timeout_ms":10000,"last_server_duration_us":212555,"total_duration_us":10005162},{"operation_name":"upsert","last_dispatch_duration_us":1096447,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10962209,"total_server_duration_us":143907,"encode_duration_us":16311,"operation_id":"0x2daf","timeout_ms":10000,"last_server_duration_us":65775,"total_duration_us":10004740},{"operation_name":"upsert","last_dispatch_duration_us":9785301,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":15101732,"total_server_duration_us":729598,"encode_duration_us":16214,"operation_id":"0x2d4d","timeout_ms":10000,"last_server_duration_us":38690,"total_duration_us":10003535}],"total_count":22}} WARN [2022-01-10 05:40:11,492] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":21276,"operation_id":"0x2e68","timeout_ms":10000,"total_duration_us":10006814},{"operation_name":"upsert","last_dispatch_duration_us":486648,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":3885842,"total_server_duration_us":377375,"encode_duration_us":17320,"operation_id":"0x2f03","timeout_ms":10000,"last_server_duration_us":29508,"total_duration_us":10006605},{"operation_name":"upsert","last_dispatch_duration_us":506424,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":1667090,"total_server_duration_us":319768,"encode_duration_us":18156,"operation_id":"0x2dca","timeout_ms":10000,"last_server_duration_us":118289,"total_duration_us":10006418},{"operation_name":"upsert","last_dispatch_duration_us":438560,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":4545762,"total_server_duration_us":648427,"encode_duration_us":18313,"operation_id":"0x2eae","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10005341},{"operation_name":"upsert","last_dispatch_duration_us":295814,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":3798525,"total_server_duration_us":2138368,"encode_duration_us":18292,"operation_id":"0x2e30","timeout_ms":10000,"last_server_duration_us":82403,"total_duration_us":10005321},{"operation_name":"upsert","last_dispatch_duration_us":334711,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":4167183,"total_server_duration_us":1127212,"encode_duration_us":17563,"operation_id":"0x2ee1","timeout_ms":10000,"last_server_duration_us":57632,"total_duration_us":10004366},{"operation_name":"upsert","last_dispatch_duration_us":997110,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":10793850,"total_server_duration_us":3194273,"encode_duration_us":19837,"operation_id":"0x2e95","timeout_ms":10000,"last_server_duration_us":602572,"total_duration_us":10003849},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":16426,"operation_id":"0x2dfb","timeout_ms":10000,"total_duration_us":10002415},{"operation_name":"get","last_dispatch_duration_us":4060043,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":4060043,"total_server_duration_us":152179,"operation_id":"0x2edc","timeout_ms":30000,"last_server_duration_us":152179,"total_duration_us":4062495},{"operation_name":"get","last_dispatch_duration_us":1087469,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":1087469,"total_server_duration_us":158398,"operation_id":"0x30a9","timeout_ms":30000,"last_server_duration_us":158398,"total_duration_us":1089433}],"total_count":12}} WARN [2022-01-10 05:40:11,492] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9555140,"last_local_id":"EFE9906200000004/000000008B42EFB5","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10219926,"last_local_socket":"172.23.104.79:44292","total_server_duration_us":9953714,"encode_duration_us":16527,"operation_id":"0x2cb1","timeout_ms":10000,"last_server_duration_us":9544214,"total_duration_us":10007538},{"operation_name":"upsert","last_dispatch_duration_us":19483449,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":19483449,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9053293,"encode_duration_us":16438,"operation_id":"0x2c43","timeout_ms":10000,"last_server_duration_us":9053293,"total_duration_us":10008185}],"total_count":2}} WARN [2022-01-10 05:40:11,493] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9606825,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10734191,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9151489,"encode_duration_us":17067,"operation_id":"0x2e0c","timeout_ms":10000,"last_server_duration_us":9107531,"total_duration_us":10004405},{"operation_name":"upsert","last_dispatch_duration_us":10059574,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10551494,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9776670,"encode_duration_us":16811,"operation_id":"0x2e31","timeout_ms":10000,"last_server_duration_us":9672760,"total_duration_us":10010325}],"total_count":2}} INFO [2022-01-10 05:40:11,589] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2397660818713451}, :ns_1@172.23.107.227 {:progress 0.1788856304985337}, :ns_1@172.23.105.187 {:progress 0.1788856304985337}, :ns_1@172.23.107.228 {:progress 0.2470588235294118}} INFO [2022-01-10 05:40:11,589] jepsen worker nemesis - couchbase.util Rebalance check count: 167 INFO [2022-01-10 05:40:11,589] jepsen worker nemesis - couchbase.util Rebalance stuck count: 124 WARN [2022-01-10 05:40:11,592] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10932160,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10932160,"total_server_duration_us":43,"operation_id":"0x2dee","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":10933595},{"operation_name":"get","last_dispatch_duration_us":10493055,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10493055,"total_server_duration_us":32,"operation_id":"0x2e08","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":10495776},{"operation_name":"upsert","last_dispatch_duration_us":265845,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4840149,"total_server_duration_us":572525,"encode_duration_us":17472,"operation_id":"0x2ea9","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10010152},{"operation_name":"upsert","last_dispatch_duration_us":10506656,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10506656,"total_server_duration_us":9250696,"encode_duration_us":16222,"operation_id":"0x2d3f","timeout_ms":10000,"last_server_duration_us":9250696,"total_duration_us":10009324},{"operation_name":"upsert","last_dispatch_duration_us":2398010,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9126796,"total_server_duration_us":2430924,"encode_duration_us":19424,"operation_id":"0x2e91","timeout_ms":10000,"last_server_duration_us":213201,"total_duration_us":10008981},{"operation_name":"upsert","last_dispatch_duration_us":445005,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5355363,"total_server_duration_us":990164,"encode_duration_us":24398,"operation_id":"0x2ea1","timeout_ms":10000,"last_server_duration_us":65514,"total_duration_us":10008324},{"operation_name":"upsert","last_dispatch_duration_us":378233,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4204408,"total_server_duration_us":1086164,"encode_duration_us":20358,"operation_id":"0x2ee0","timeout_ms":10000,"last_server_duration_us":220807,"total_duration_us":10005884},{"operation_name":"upsert","last_dispatch_duration_us":204575,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5344727,"total_server_duration_us":837424,"encode_duration_us":18888,"operation_id":"0x2e94","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10005847},{"operation_name":"upsert","last_dispatch_duration_us":301011,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":3964550,"total_server_duration_us":434990,"encode_duration_us":17294,"operation_id":"0x2edb","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10005788},{"operation_name":"upsert","last_dispatch_duration_us":1236562,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":11125850,"total_server_duration_us":2520677,"encode_duration_us":21035,"operation_id":"0x2eab","timeout_ms":10000,"last_server_duration_us":555085,"total_duration_us":10005402}],"total_count":20}} INFO [2022-01-10 05:40:11,627] jepsen worker 22 - jepsen.util 1372 :ok :read [7 :nil] INFO [2022-01-10 05:40:11,628] jepsen worker 22 - jepsen.util 1372 :invoke :read [7 nil] INFO [2022-01-10 05:40:11,648] jepsen worker 73 - jepsen.util 1513 :ok :read [24 18] INFO [2022-01-10 05:40:11,649] jepsen worker 22 - jepsen.util 1372 :ok :read [7 :nil] INFO [2022-01-10 05:40:11,649] jepsen worker 26 - jepsen.util 1286 :ok :read [8 :nil] INFO [2022-01-10 05:40:11,649] jepsen worker 73 - jepsen.util 1513 :invoke :write [24 11] INFO [2022-01-10 05:40:11,650] jepsen worker 26 - jepsen.util 1286 :invoke :write [8 32] INFO [2022-01-10 05:40:11,651] jepsen worker 22 - jepsen.util 1372 :invoke :read [7 nil] INFO [2022-01-10 05:40:11,653] jepsen worker 67 - jepsen.util 1417 :ok :read [22 41] INFO [2022-01-10 05:40:11,653] jepsen worker 82 - jepsen.util 1342 :ok :read [27 30] INFO [2022-01-10 05:40:11,655] jepsen worker 82 - jepsen.util 1342 :invoke :write [27 12] INFO [2022-01-10 05:40:11,655] jepsen worker 67 - jepsen.util 1417 :invoke :write [22 49] INFO [2022-01-10 05:40:11,657] jepsen worker 64 - jepsen.util 1594 :ok :read [21 7] INFO [2022-01-10 05:40:11,658] jepsen worker 22 - jepsen.util 1372 :ok :read [7 :nil] INFO [2022-01-10 05:40:11,659] jepsen worker 17 - jepsen.util 1367 :ok :read [5 42] INFO [2022-01-10 05:40:11,659] jepsen worker 64 - jepsen.util 1594 :invoke :write [21 36] INFO [2022-01-10 05:40:11,661] jepsen worker 22 - jepsen.util 1372 :invoke :write [7 3] INFO [2022-01-10 05:40:11,661] jepsen worker 17 - jepsen.util 1367 :invoke :write [5 18] INFO [2022-01-10 05:40:11,661] jepsen worker 75 - jepsen.util 1155 :info :write [25 26] :RequestTimeoutException INFO [2022-01-10 05:40:11,669] jepsen worker 39 - jepsen.util 1659 :info :write [13 16] :RequestTimeoutException INFO [2022-01-10 05:40:11,669] jepsen worker 75 - jepsen.util 1245 :invoke :read [25 nil] INFO [2022-01-10 05:40:11,670] jepsen worker 39 - jepsen.util 1749 :invoke :read [13 nil] INFO [2022-01-10 05:40:11,672] jepsen worker 25 - jepsen.util 1375 :info :write [8 15] :RequestTimeoutException INFO [2022-01-10 05:40:11,672] jepsen worker 25 - jepsen.util 1465 :invoke :write [8 2] INFO [2022-01-10 05:40:11,749] jepsen worker 75 - jepsen.util 1245 :ok :read [25 :nil] INFO [2022-01-10 05:40:11,749] jepsen worker 75 - jepsen.util 1245 :invoke :read [25 nil] INFO [2022-01-10 05:40:11,922] jepsen worker 58 - jepsen.util 1408 :info :write [19 16] :RequestTimeoutException INFO [2022-01-10 05:40:11,923] jepsen worker 58 - jepsen.util 1498 :invoke :write [19 37] INFO [2022-01-10 05:40:11,926] jepsen worker 16 - jepsen.util 1726 :info :write [5 46] :RequestTimeoutException INFO [2022-01-10 05:40:11,927] jepsen worker 16 - jepsen.util 1816 :invoke :write [5 2] INFO [2022-01-10 05:40:12,355] jepsen worker 39 - jepsen.util 1749 :ok :read [13 15] INFO [2022-01-10 05:40:12,355] jepsen worker 39 - jepsen.util 1749 :invoke :write [13 25] INFO [2022-01-10 05:40:12,456] jepsen worker 77 - jepsen.util 1787 :info :write [25 37] :RequestTimeoutException INFO [2022-01-10 05:40:12,457] jepsen worker 77 - jepsen.util 1877 :invoke :write [25 12] INFO [2022-01-10 05:40:12,596] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2397660818713451}, :ns_1@172.23.107.227 {:progress 0.1788856304985337}, :ns_1@172.23.105.187 {:progress 0.1788856304985337}, :ns_1@172.23.107.228 {:progress 0.2470588235294118}} INFO [2022-01-10 05:40:12,692] jepsen worker nemesis - couchbase.util Rebalance check count: 168 INFO [2022-01-10 05:40:12,693] jepsen worker nemesis - couchbase.util Rebalance stuck count: 125 INFO [2022-01-10 05:40:12,801] jepsen worker 63 - jepsen.util 1323 :info :write [21 18] :RequestTimeoutException INFO [2022-01-10 05:40:12,801] jepsen worker 63 - jepsen.util 1413 :invoke :read [21 nil] INFO [2022-01-10 05:40:12,854] jepsen worker 33 - jepsen.util 1473 :info :write [11 24] :RequestTimeoutException INFO [2022-01-10 05:40:12,855] jepsen worker 33 - jepsen.util 1563 :invoke :read [11 nil] INFO [2022-01-10 05:40:12,866] jepsen worker 46 - jepsen.util 1576 :info :write [15 49] :RequestTimeoutException INFO [2022-01-10 05:40:12,866] jepsen worker 46 - jepsen.util 1666 :invoke :write [15 19] INFO [2022-01-10 05:40:13,422] jepsen worker 20 - jepsen.util 1280 :info :write [6 16] :RequestTimeoutException INFO [2022-01-10 05:40:13,422] jepsen worker 20 - jepsen.util 1370 :invoke :write [6 31] INFO [2022-01-10 05:40:13,699] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2397660818713451}, :ns_1@172.23.107.227 {:progress 0.1788856304985337}, :ns_1@172.23.105.187 {:progress 0.1788856304985337}, :ns_1@172.23.107.228 {:progress 0.2529411764705882}} INFO [2022-01-10 05:40:13,699] jepsen worker nemesis - couchbase.util Rebalance check count: 169 INFO [2022-01-10 05:40:13,699] jepsen worker nemesis - couchbase.util Rebalance stuck count: 125 INFO [2022-01-10 05:40:13,861] jepsen worker 41 - jepsen.util 1211 :info :write [13 3] :RequestTimeoutException INFO [2022-01-10 05:40:13,862] jepsen worker 41 - jepsen.util 1301 :invoke :write [13 19] INFO [2022-01-10 05:40:13,882] jepsen worker 84 - jepsen.util 1524 :info :write [28 39] :RequestTimeoutException INFO [2022-01-10 05:40:13,883] jepsen worker 84 - jepsen.util 1614 :invoke :write [28 5] INFO [2022-01-10 05:40:14,096] jepsen worker 35 - jepsen.util 1475 :ok :read [11 21] INFO [2022-01-10 05:40:14,097] jepsen worker 35 - jepsen.util 1475 :invoke :write [11 37] INFO [2022-01-10 05:40:14,137] jepsen worker 33 - jepsen.util 1563 :ok :read [11 21] INFO [2022-01-10 05:40:14,138] jepsen worker 33 - jepsen.util 1563 :invoke :read [11 nil] INFO [2022-01-10 05:40:14,248] jepsen worker 33 - jepsen.util 1563 :ok :read [11 21] INFO [2022-01-10 05:40:14,248] jepsen worker 33 - jepsen.util 1563 :invoke :read [11 nil] INFO [2022-01-10 05:40:14,317] jepsen worker 33 - jepsen.util 1563 :ok :read [11 21] INFO [2022-01-10 05:40:14,317] jepsen worker 33 - jepsen.util 1563 :invoke :read [11 nil] INFO [2022-01-10 05:40:14,425] jepsen worker 33 - jepsen.util 1563 :ok :read [11 21] INFO [2022-01-10 05:40:14,426] jepsen worker 33 - jepsen.util 1563 :invoke :write [11 43] INFO [2022-01-10 05:40:14,484] jepsen worker 86 - jepsen.util 1166 :info :write [28 5] :RequestTimeoutException INFO [2022-01-10 05:40:14,485] jepsen worker 86 - jepsen.util 1256 :invoke :read [28 nil] INFO [2022-01-10 05:40:14,685] jepsen worker 68 - jepsen.util 1418 :info :write [22 17] :RequestTimeoutException INFO [2022-01-10 05:40:14,686] jepsen worker 68 - jepsen.util 1508 :invoke :write [22 16] INFO [2022-01-10 05:40:14,705] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2397660818713451}, :ns_1@172.23.107.227 {:progress 0.1788856304985337}, :ns_1@172.23.105.187 {:progress 0.1788856304985337}, :ns_1@172.23.107.228 {:progress 0.2529411764705882}} INFO [2022-01-10 05:40:14,705] jepsen worker nemesis - couchbase.util Rebalance check count: 170 INFO [2022-01-10 05:40:14,706] jepsen worker nemesis - couchbase.util Rebalance stuck count: 126 INFO [2022-01-10 05:40:14,770] jepsen worker 42 - jepsen.util 1392 :info :write [14 41] :RequestTimeoutException INFO [2022-01-10 05:40:14,771] jepsen worker 42 - jepsen.util 1482 :invoke :read [14 nil] INFO [2022-01-10 05:40:15,150] jepsen worker 74 - jepsen.util 1874 :info :write [24 41] :RequestTimeoutException INFO [2022-01-10 05:40:15,150] jepsen worker 74 - jepsen.util 1964 :invoke :write [24 22] INFO [2022-01-10 05:40:15,272] jepsen worker 80 - jepsen.util 1520 :info :write [26 47] :RequestTimeoutException INFO [2022-01-10 05:40:15,272] jepsen worker 80 - jepsen.util 1610 :invoke :read [26 nil] INFO [2022-01-10 05:40:15,556] jepsen worker 27 - jepsen.util 1377 :info :write [9 45] :RequestTimeoutException INFO [2022-01-10 05:40:15,556] jepsen worker 27 - jepsen.util 1467 :invoke :write [9 32] INFO [2022-01-10 05:40:15,712] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2397660818713451}, :ns_1@172.23.107.227 {:progress 0.1818181818181818}, :ns_1@172.23.105.187 {:progress 0.1818181818181818}, :ns_1@172.23.107.228 {:progress 0.2529411764705882}} INFO [2022-01-10 05:40:15,712] jepsen worker nemesis - couchbase.util Rebalance check count: 171 INFO [2022-01-10 05:40:15,712] jepsen worker nemesis - couchbase.util Rebalance stuck count: 126 INFO [2022-01-10 05:40:15,766] jepsen worker 31 - jepsen.util 1831 :ok :read [10 37] INFO [2022-01-10 05:40:15,767] jepsen worker 31 - jepsen.util 1831 :invoke :write [10 33] INFO [2022-01-10 05:40:15,868] jepsen worker 36 - jepsen.util 1116 :info :write [12 27] :SyncWriteAmbiguous INFO [2022-01-10 05:40:15,868] jepsen worker 36 - jepsen.util 1206 :invoke :write [12 12] INFO [2022-01-10 05:40:15,869] jepsen worker 3 - jepsen.util 1713 :ok :read [1 41] INFO [2022-01-10 05:40:15,870] jepsen worker 3 - jepsen.util 1713 :invoke :read [1 nil] INFO [2022-01-10 05:40:15,873] jepsen worker 53 - jepsen.util 1493 :info :write [17 22] :RequestTimeoutException INFO [2022-01-10 05:40:15,874] jepsen worker 53 - jepsen.util 1583 :invoke :read [17 nil] INFO [2022-01-10 05:40:15,941] jepsen worker 11 - jepsen.util 1451 :ok :read [3 2] INFO [2022-01-10 05:40:15,942] jepsen worker 11 - jepsen.util 1451 :invoke :write [3 11] INFO [2022-01-10 05:40:15,962] jepsen worker 87 - jepsen.util 1707 :ok :read [29 33] INFO [2022-01-10 05:40:15,962] jepsen worker 87 - jepsen.util 1707 :invoke :write [29 28] INFO [2022-01-10 05:40:15,990] jepsen worker 5 - jepsen.util 1445 :info :write [1 44] :SyncWriteAmbiguous INFO [2022-01-10 05:40:15,993] jepsen worker 5 - jepsen.util 1535 :invoke :read [1 nil] INFO [2022-01-10 05:40:16,012] jepsen worker 71 - jepsen.util 1601 :ok :read [23 26] INFO [2022-01-10 05:40:16,013] jepsen worker 71 - jepsen.util 1601 :invoke :write [23 30] INFO [2022-01-10 05:40:16,186] jepsen worker 29 - jepsen.util 1469 :info :write [9 19] :RequestTimeoutException INFO [2022-01-10 05:40:16,187] jepsen worker 29 - jepsen.util 1559 :invoke :write [9 32] INFO [2022-01-10 05:40:16,336] jepsen worker 49 - jepsen.util 1219 :info :write [16 23] :RequestTimeoutException INFO [2022-01-10 05:40:16,339] jepsen worker 49 - jepsen.util 1309 :invoke :write [16 37] INFO [2022-01-10 05:40:16,345] jepsen worker 1 - jepsen.util 1531 :info :write [0 21] :RequestTimeoutException INFO [2022-01-10 05:40:16,347] jepsen worker 1 - jepsen.util 1621 :invoke :write [0 40] INFO [2022-01-10 05:40:16,367] jepsen worker 12 - jepsen.util 1272 :ok :read [4 3] INFO [2022-01-10 05:40:16,368] jepsen worker 12 - jepsen.util 1272 :invoke :write [4 17] INFO [2022-01-10 05:40:16,369] jepsen worker 80 - jepsen.util 1610 :ok :read [26 10] INFO [2022-01-10 05:40:16,370] jepsen worker 80 - jepsen.util 1610 :invoke :write [26 27] INFO [2022-01-10 05:40:16,374] jepsen worker 53 - jepsen.util 1583 :ok :read [17 41] INFO [2022-01-10 05:40:16,375] jepsen worker 53 - jepsen.util 1583 :invoke :write [17 46] INFO [2022-01-10 05:40:16,388] jepsen worker 13 - jepsen.util 1273 :ok :read [4 3] INFO [2022-01-10 05:40:16,389] jepsen worker 13 - jepsen.util 1273 :invoke :read [4 nil] INFO [2022-01-10 05:40:16,441] jepsen worker 89 - jepsen.util 1709 :info :write [29 24] :SyncWriteAmbiguous INFO [2022-01-10 05:40:16,442] jepsen worker 89 - jepsen.util 1799 :invoke :write [29 47] INFO [2022-01-10 05:40:16,453] jepsen worker 45 - jepsen.util 1665 :ok :read [15 23] INFO [2022-01-10 05:40:16,454] jepsen worker 45 - jepsen.util 1665 :invoke :read [15 nil] INFO [2022-01-10 05:40:16,620] jepsen worker 45 - jepsen.util 1665 :ok :read [15 23] INFO [2022-01-10 05:40:16,620] jepsen worker 45 - jepsen.util 1665 :invoke :write [15 18] INFO [2022-01-10 05:40:16,718] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2397660818713451}, :ns_1@172.23.107.227 {:progress 0.1818181818181818}, :ns_1@172.23.105.187 {:progress 0.1818181818181818}, :ns_1@172.23.107.228 {:progress 0.2529411764705882}} INFO [2022-01-10 05:40:16,718] jepsen worker nemesis - couchbase.util Rebalance check count: 172 INFO [2022-01-10 05:40:16,719] jepsen worker nemesis - couchbase.util Rebalance stuck count: 127 INFO [2022-01-10 05:40:16,745] jepsen worker 50 - jepsen.util 1670 :ok :read [16 38] INFO [2022-01-10 05:40:16,746] jepsen worker 50 - jepsen.util 1670 :invoke :read [16 nil] INFO [2022-01-10 05:40:16,749] jepsen worker 24 - jepsen.util 1464 :ok :read [8 :nil] INFO [2022-01-10 05:40:16,752] jepsen worker 24 - jepsen.util 1464 :invoke :read [8 nil] INFO [2022-01-10 05:40:16,764] jepsen worker 70 - jepsen.util 1780 :info :write [23 4] :SyncWriteAmbiguous INFO [2022-01-10 05:40:16,764] jepsen worker 70 - jepsen.util 1870 :invoke :read [23 nil] INFO [2022-01-10 05:40:16,781] jepsen worker 24 - jepsen.util 1464 :ok :read [8 :nil] INFO [2022-01-10 05:40:16,781] jepsen worker 48 - jepsen.util 1218 :info :write [16 32] :RequestTimeoutException INFO [2022-01-10 05:40:16,782] jepsen worker 48 - jepsen.util 1308 :invoke :read [16 nil] INFO [2022-01-10 05:40:16,783] jepsen worker 24 - jepsen.util 1464 :invoke :read [8 nil] INFO [2022-01-10 05:40:16,815] jepsen worker 24 - jepsen.util 1464 :ok :read [8 :nil] INFO [2022-01-10 05:40:16,816] jepsen worker 24 - jepsen.util 1464 :invoke :read [8 nil] INFO [2022-01-10 05:40:16,830] jepsen worker 30 - jepsen.util 1110 :info :write [10 47] :RequestTimeoutException INFO [2022-01-10 05:40:16,831] jepsen worker 30 - jepsen.util 1200 :invoke :write [10 0] INFO [2022-01-10 05:40:16,836] jepsen worker 55 - jepsen.util 1675 :info :write [18 5] :RequestTimeoutException INFO [2022-01-10 05:40:16,837] jepsen worker 55 - jepsen.util 1765 :invoke :write [18 28] INFO [2022-01-10 05:40:16,852] jepsen worker 24 - jepsen.util 1464 :ok :read [8 :nil] INFO [2022-01-10 05:40:16,853] jepsen worker 24 - jepsen.util 1464 :invoke :write [8 31] INFO [2022-01-10 05:40:16,894] jepsen worker 70 - jepsen.util 1870 :ok :read [23 26] INFO [2022-01-10 05:40:16,895] jepsen worker 70 - jepsen.util 1870 :invoke :read [23 nil] INFO [2022-01-10 05:40:16,965] jepsen worker 70 - jepsen.util 1870 :ok :read [23 26] INFO [2022-01-10 05:40:16,966] jepsen worker 70 - jepsen.util 1870 :invoke :write [23 23] INFO [2022-01-10 05:40:17,220] jepsen worker 88 - jepsen.util 1438 :info :write [29 19] :RequestTimeoutException INFO [2022-01-10 05:40:17,222] jepsen worker 88 - jepsen.util 1528 :invoke :write [29 16] INFO [2022-01-10 05:40:17,230] jepsen worker 50 - jepsen.util 1670 :ok :read [16 38] INFO [2022-01-10 05:40:17,231] jepsen worker 50 - jepsen.util 1670 :invoke :write [16 22] INFO [2022-01-10 05:40:17,250] jepsen worker 10 - jepsen.util 1630 :info :write [3 39] :RequestTimeoutException INFO [2022-01-10 05:40:17,253] jepsen worker 10 - jepsen.util 1720 :invoke :write [3 47] INFO [2022-01-10 05:40:17,281] jepsen worker 38 - jepsen.util 1568 :ok :read [12 1] INFO [2022-01-10 05:40:17,281] jepsen worker 38 - jepsen.util 1568 :invoke :read [12 nil] INFO [2022-01-10 05:40:17,317] jepsen worker 38 - jepsen.util 1568 :ok :read [12 1] INFO [2022-01-10 05:40:17,317] jepsen worker 38 - jepsen.util 1568 :invoke :write [12 8] INFO [2022-01-10 05:40:17,367] jepsen worker 13 - jepsen.util 1273 :ok :read [4 3] INFO [2022-01-10 05:40:17,368] jepsen worker 13 - jepsen.util 1273 :invoke :read [4 nil] INFO [2022-01-10 05:40:17,466] jepsen worker 13 - jepsen.util 1273 :ok :read [4 3] INFO [2022-01-10 05:40:17,466] jepsen worker 13 - jepsen.util 1273 :invoke :read [4 nil] INFO [2022-01-10 05:40:17,492] jepsen worker 13 - jepsen.util 1273 :ok :read [4 3] INFO [2022-01-10 05:40:17,493] jepsen worker 13 - jepsen.util 1273 :invoke :write [4 45] INFO [2022-01-10 05:40:17,641] jepsen worker 9 - jepsen.util 1719 :info :write [3 31] :RequestTimeoutException INFO [2022-01-10 05:40:17,644] jepsen worker 9 - jepsen.util 1809 :invoke :read [3 nil] INFO [2022-01-10 05:40:17,675] jepsen worker 2 - jepsen.util 1532 :info :write [0 16] :RequestTimeoutException INFO [2022-01-10 05:40:17,676] jepsen worker 2 - jepsen.util 1622 :invoke :read [0 nil] INFO [2022-01-10 05:40:17,697] jepsen worker 9 - jepsen.util 1809 :ok :read [3 2] INFO [2022-01-10 05:40:17,698] jepsen worker 9 - jepsen.util 1809 :invoke :write [3 31] INFO [2022-01-10 05:40:17,724] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2397660818713451}, :ns_1@172.23.107.227 {:progress 0.1818181818181818}, :ns_1@172.23.105.187 {:progress 0.1818181818181818}, :ns_1@172.23.107.228 {:progress 0.2529411764705882}} INFO [2022-01-10 05:40:17,724] jepsen worker nemesis - couchbase.util Rebalance check count: 173 INFO [2022-01-10 05:40:17,724] jepsen worker nemesis - couchbase.util Rebalance stuck count: 128 INFO [2022-01-10 05:40:17,761] jepsen worker 6 - jepsen.util 1446 :info :write [2 47] :RequestTimeoutException INFO [2022-01-10 05:40:17,761] jepsen worker 6 - jepsen.util 1536 :invoke :read [2 nil] INFO [2022-01-10 05:40:17,831] jepsen worker 0 - jepsen.util 1440 :ok :read [0 46] INFO [2022-01-10 05:40:17,831] jepsen worker 0 - jepsen.util 1440 :invoke :read [0 nil] INFO [2022-01-10 05:40:17,886] jepsen worker 40 - jepsen.util 1210 :info :write [13 45] :RequestTimeoutException INFO [2022-01-10 05:40:17,887] jepsen worker 40 - jepsen.util 1300 :invoke :read [13 nil] INFO [2022-01-10 05:40:17,896] jepsen worker 21 - jepsen.util 1641 :info :write [7 42] :RequestTimeoutException INFO [2022-01-10 05:40:17,897] jepsen worker 21 - jepsen.util 1731 :invoke :read [7 nil] INFO [2022-01-10 05:40:17,902] jepsen worker 85 - jepsen.util 1705 :info :write [28 24] :RequestTimeoutException INFO [2022-01-10 05:40:17,902] jepsen worker 85 - jepsen.util 1795 :invoke :write [28 46] INFO [2022-01-10 05:40:18,006] jepsen worker 32 - jepsen.util 1382 :info :write [10 20] :RequestTimeoutException INFO [2022-01-10 05:40:18,006] jepsen worker 32 - jepsen.util 1472 :invoke :write [10 28] INFO [2022-01-10 05:40:18,031] jepsen worker 72 - jepsen.util 1602 :info :write [24 47] :RequestTimeoutException INFO [2022-01-10 05:40:18,031] jepsen worker 72 - jepsen.util 1692 :invoke :read [24 nil] INFO [2022-01-10 05:40:18,066] jepsen worker 23 - jepsen.util 1553 :info :write [7 22] :RequestTimeoutException INFO [2022-01-10 05:40:18,066] jepsen worker 23 - jepsen.util 1643 :invoke :read [7 nil] INFO [2022-01-10 05:40:18,080] jepsen worker 14 - jepsen.util 1544 :info :write [4 23] :RequestTimeoutException INFO [2022-01-10 05:40:18,081] jepsen worker 14 - jepsen.util 1634 :invoke :write [4 49] INFO [2022-01-10 05:40:18,131] jepsen worker 76 - jepsen.util 1156 :info :write [25 15] :RequestTimeoutException INFO [2022-01-10 05:40:18,132] jepsen worker 76 - jepsen.util 1246 :invoke :read [25 nil] INFO [2022-01-10 05:40:18,199] jepsen worker 44 - jepsen.util 1574 :info :write [14 9] :SyncWriteAmbiguous INFO [2022-01-10 05:40:18,200] jepsen worker 44 - jepsen.util 1664 :invoke :write [14 33] INFO [2022-01-10 05:40:18,215] jepsen worker 4 - jepsen.util 1444 :info :write [1 40] :RequestTimeoutException INFO [2022-01-10 05:40:18,216] jepsen worker 4 - jepsen.util 1534 :invoke :read [1 nil] INFO [2022-01-10 05:40:18,311] jepsen worker 8 - jepsen.util 1538 :info :write [2 26] :RequestTimeoutException INFO [2022-01-10 05:40:18,312] jepsen worker 8 - jepsen.util 1628 :invoke :read [2 nil] INFO [2022-01-10 05:40:18,438] jepsen worker 0 - jepsen.util 1440 :ok :read [0 46] INFO [2022-01-10 05:40:18,438] jepsen worker 0 - jepsen.util 1440 :invoke :read [0 nil] INFO [2022-01-10 05:40:18,522] jepsen worker 69 - jepsen.util 1599 :info :write [23 44] :RequestTimeoutException INFO [2022-01-10 05:40:18,522] jepsen worker 69 - jepsen.util 1689 :invoke :read [23 nil] INFO [2022-01-10 05:40:18,611] jepsen worker 51 - jepsen.util 1581 :info :write [17 39] :RequestTimeoutException INFO [2022-01-10 05:40:18,612] jepsen worker 51 - jepsen.util 1671 :invoke :read [17 nil] INFO [2022-01-10 05:40:18,624] jepsen worker 60 - jepsen.util 1860 :info :write [20 26] :RequestTimeoutException INFO [2022-01-10 05:40:18,625] jepsen worker 60 - jepsen.util 1950 :invoke :write [20 5] INFO [2022-01-10 05:40:18,730] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2456140350877193}, :ns_1@172.23.107.227 {:progress 0.1876832844574781}, :ns_1@172.23.105.187 {:progress 0.1876832844574781}, :ns_1@172.23.107.228 {:progress 0.2647058823529411}} INFO [2022-01-10 05:40:18,730] jepsen worker nemesis - couchbase.util Rebalance check count: 174 INFO [2022-01-10 05:40:18,730] jepsen worker nemesis - couchbase.util Rebalance stuck count: 128 INFO [2022-01-10 05:40:18,757] jepsen worker 40 - jepsen.util 1300 :ok :read [13 15] INFO [2022-01-10 05:40:18,758] jepsen worker 40 - jepsen.util 1300 :invoke :read [13 nil] INFO [2022-01-10 05:40:18,886] jepsen worker 19 - jepsen.util 1639 :info :write [6 17] :RequestTimeoutException INFO [2022-01-10 05:40:18,887] jepsen worker 19 - jepsen.util 1729 :invoke :read [6 nil] INFO [2022-01-10 05:40:18,912] jepsen worker 37 - jepsen.util 1477 :info :write [12 43] :RequestTimeoutException INFO [2022-01-10 05:40:18,912] jepsen worker 37 - jepsen.util 1567 :invoke :write [12 13] INFO [2022-01-10 05:40:19,015] jepsen worker 78 - jepsen.util 1428 :info :write [26 43] :RequestTimeoutException INFO [2022-01-10 05:40:19,016] jepsen worker 78 - jepsen.util 1518 :invoke :read [26 nil] INFO [2022-01-10 05:40:19,072] jepsen worker 78 - jepsen.util 1518 :ok :read [26 10] INFO [2022-01-10 05:40:19,072] jepsen worker 78 - jepsen.util 1518 :invoke :read [26 nil] INFO [2022-01-10 05:40:19,085] jepsen worker 0 - jepsen.util 1440 :ok :read [0 46] INFO [2022-01-10 05:40:19,085] jepsen worker 0 - jepsen.util 1440 :invoke :read [0 nil] INFO [2022-01-10 05:40:19,096] jepsen worker 78 - jepsen.util 1518 :ok :read [26 10] INFO [2022-01-10 05:40:19,097] jepsen worker 78 - jepsen.util 1518 :invoke :write [26 47] INFO [2022-01-10 05:40:19,120] jepsen worker 34 - jepsen.util 1564 :info :write [11 13] :RequestTimeoutException INFO [2022-01-10 05:40:19,121] jepsen worker 34 - jepsen.util 1654 :invoke :write [11 17] INFO [2022-01-10 05:40:19,160] jepsen worker 40 - jepsen.util 1300 :ok :read [13 15] INFO [2022-01-10 05:40:19,160] jepsen worker 40 - jepsen.util 1300 :invoke :read [13 nil] INFO [2022-01-10 05:40:19,216] jepsen worker 81 - jepsen.util 1431 :info :write [27 49] :RequestTimeoutException INFO [2022-01-10 05:40:19,217] jepsen worker 81 - jepsen.util 1521 :invoke :write [27 2] INFO [2022-01-10 05:40:19,326] jepsen worker 59 - jepsen.util 1499 :info :write [19 43] :RequestTimeoutException INFO [2022-01-10 05:40:19,326] jepsen worker 59 - jepsen.util 1589 :invoke :write [19 3] INFO [2022-01-10 05:40:19,476] jepsen worker 62 - jepsen.util 1682 :info :write [20 41] :RequestTimeoutException INFO [2022-01-10 05:40:19,477] jepsen worker 62 - jepsen.util 1772 :invoke :read [20 nil] INFO [2022-01-10 05:40:19,480] jepsen worker 43 - jepsen.util 1663 :info :write [14 36] :RequestTimeoutException INFO [2022-01-10 05:40:19,480] jepsen worker 43 - jepsen.util 1753 :invoke :read [14 nil] INFO [2022-01-10 05:40:19,531] jepsen worker 61 - jepsen.util 1411 :info :write [20 0] :RequestTimeoutException INFO [2022-01-10 05:40:19,532] jepsen worker 61 - jepsen.util 1501 :invoke :read [20 nil] INFO [2022-01-10 05:40:19,591] jepsen worker 52 - jepsen.util 1672 :info :write [17 8] :RequestTimeoutException INFO [2022-01-10 05:40:19,592] jepsen worker 52 - jepsen.util 1762 :invoke :read [17 nil] INFO [2022-01-10 05:40:19,614] jepsen worker 86 - jepsen.util 1256 :ok :read [28 13] INFO [2022-01-10 05:40:19,614] jepsen worker 86 - jepsen.util 1256 :invoke :write [28 41] INFO [2022-01-10 05:40:19,615] jepsen worker 52 - jepsen.util 1762 :ok :read [17 41] INFO [2022-01-10 05:40:19,616] jepsen worker 52 - jepsen.util 1762 :invoke :write [17 49] WARN [2022-01-10 05:40:19,702] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17380825,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":20180206,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9967129,"encode_duration_us":16193,"operation_id":"0x2dd3","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10004547},{"operation_name":"upsert","last_dispatch_duration_us":9554239,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10649237,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9737475,"encode_duration_us":20368,"operation_id":"0x2ea5","timeout_ms":10000,"last_server_duration_us":9097950,"total_duration_us":10010569},{"operation_name":"upsert","last_dispatch_duration_us":9048369,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10385268,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9508501,"encode_duration_us":17361,"operation_id":"0x2e6d","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10007658}],"total_count":3}} INFO [2022-01-10 05:40:19,747] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2456140350877193}, :ns_1@172.23.107.227 {:progress 0.1876832844574781}, :ns_1@172.23.105.187 {:progress 0.1876832844574781}, :ns_1@172.23.107.228 {:progress 0.2647058823529411}} INFO [2022-01-10 05:40:19,747] jepsen worker nemesis - couchbase.util Rebalance check count: 175 INFO [2022-01-10 05:40:19,747] jepsen worker nemesis - couchbase.util Rebalance stuck count: 129 INFO [2022-01-10 05:40:19,780] jepsen worker 18 - jepsen.util 1278 :info :write [6 36] :RequestTimeoutException INFO [2022-01-10 05:40:19,780] jepsen worker 18 - jepsen.util 1368 :invoke :read [6 nil] WARN [2022-01-10 05:40:19,803] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14381931,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":14381931,"total_server_duration_us":6335,"operation_id":"0x2f32","timeout_ms":30000,"last_server_duration_us":6335,"total_duration_us":14385180},{"operation_name":"upsert","last_dispatch_duration_us":8248229,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11057224,"total_server_duration_us":431348,"encode_duration_us":16765,"operation_id":"0x2f92","timeout_ms":10000,"last_server_duration_us":93043,"total_duration_us":10010396},{"operation_name":"upsert","last_dispatch_duration_us":13615415,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":13615415,"total_server_duration_us":19418,"encode_duration_us":17727,"operation_id":"0x2f5c","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10009123},{"operation_name":"upsert","last_dispatch_duration_us":621526,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10550268,"total_server_duration_us":759919,"encode_duration_us":16786,"operation_id":"0x2fe9","timeout_ms":10000,"last_server_duration_us":138978,"total_duration_us":10008809},{"operation_name":"upsert","last_dispatch_duration_us":8707498,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11779539,"total_server_duration_us":891749,"encode_duration_us":19140,"operation_id":"0x2f90","timeout_ms":10000,"last_server_duration_us":39529,"total_duration_us":10008190},{"operation_name":"upsert","last_dispatch_duration_us":8144339,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8854869,"total_server_duration_us":435818,"encode_duration_us":18283,"operation_id":"0x2f0a","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10006170},{"operation_name":"upsert","last_dispatch_duration_us":8596022,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10793636,"total_server_duration_us":667112,"encode_duration_us":20424,"operation_id":"0x2ed2","timeout_ms":10000,"last_server_duration_us":116281,"total_duration_us":10004431},{"operation_name":"upsert","last_dispatch_duration_us":31452,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":6491716,"total_server_duration_us":4270848,"encode_duration_us":17951,"operation_id":"0x3095","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10004033},{"operation_name":"upsert","last_dispatch_duration_us":918602,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10419112,"total_server_duration_us":333273,"encode_duration_us":20667,"operation_id":"0x302f","timeout_ms":10000,"last_server_duration_us":185545,"total_duration_us":10003469},{"operation_name":"upsert","last_dispatch_duration_us":203625,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8378490,"total_server_duration_us":139960,"encode_duration_us":17029,"operation_id":"0x306e","timeout_ms":10000,"last_server_duration_us":29415,"total_duration_us":10001493}],"total_count":16}} INFO [2022-01-10 05:40:19,922] jepsen worker 0 - jepsen.util 1440 :ok :read [0 46] INFO [2022-01-10 05:40:19,923] jepsen worker 0 - jepsen.util 1440 :invoke :write [0 45] INFO [2022-01-10 05:40:19,926] jepsen worker 7 - jepsen.util 1537 :info :write [2 21] :RequestTimeoutException INFO [2022-01-10 05:40:19,927] jepsen worker 7 - jepsen.util 1627 :invoke :write [2 18] INFO [2022-01-10 05:40:19,942] jepsen worker 28 - jepsen.util 1378 :info :write [9 31] :RequestTimeoutException INFO [2022-01-10 05:40:19,943] jepsen worker 28 - jepsen.util 1468 :invoke :write [9 18] INFO [2022-01-10 05:40:20,195] jepsen worker 75 - jepsen.util 1245 :ok :read [25 :nil] INFO [2022-01-10 05:40:20,196] jepsen worker 75 - jepsen.util 1245 :invoke :write [25 16] INFO [2022-01-10 05:40:20,204] jepsen worker 62 - jepsen.util 1772 :ok :read [20 :nil] INFO [2022-01-10 05:40:20,204] jepsen worker 62 - jepsen.util 1772 :invoke :read [20 nil] INFO [2022-01-10 05:40:20,291] jepsen worker 62 - jepsen.util 1772 :ok :read [20 :nil] INFO [2022-01-10 05:40:20,291] jepsen worker 62 - jepsen.util 1772 :invoke :write [20 35] INFO [2022-01-10 05:40:20,292] jepsen worker 19 - jepsen.util 1729 :ok :read [6 25] INFO [2022-01-10 05:40:20,295] jepsen worker 83 - jepsen.util 1793 :ok :read [27 30] INFO [2022-01-10 05:40:20,295] jepsen worker 19 - jepsen.util 1729 :invoke :read [6 nil] INFO [2022-01-10 05:40:20,296] jepsen worker 83 - jepsen.util 1793 :invoke :write [27 19] INFO [2022-01-10 05:40:20,304] jepsen worker 63 - jepsen.util 1413 :ok :read [21 7] INFO [2022-01-10 05:40:20,305] jepsen worker 63 - jepsen.util 1413 :invoke :read [21 nil] INFO [2022-01-10 05:40:20,374] jepsen worker 19 - jepsen.util 1729 :ok :read [6 25] INFO [2022-01-10 05:40:20,374] jepsen worker 19 - jepsen.util 1729 :invoke :read [6 nil] INFO [2022-01-10 05:40:20,391] jepsen worker 63 - jepsen.util 1413 :ok :read [21 7] INFO [2022-01-10 05:40:20,392] jepsen worker 63 - jepsen.util 1413 :invoke :read [21 nil] INFO [2022-01-10 05:40:20,505] jepsen worker 19 - jepsen.util 1729 :ok :read [6 25] INFO [2022-01-10 05:40:20,506] jepsen worker 19 - jepsen.util 1729 :invoke :read [6 nil] INFO [2022-01-10 05:40:20,509] jepsen worker 40 - jepsen.util 1300 :ok :read [13 15] INFO [2022-01-10 05:40:20,510] jepsen worker 40 - jepsen.util 1300 :invoke :write [13 11] INFO [2022-01-10 05:40:20,515] jepsen worker 63 - jepsen.util 1413 :ok :read [21 7] INFO [2022-01-10 05:40:20,516] jepsen worker 63 - jepsen.util 1413 :invoke :write [21 32] INFO [2022-01-10 05:40:20,516] jepsen worker 66 - jepsen.util 1326 :info :write [22 32] :RequestTimeoutException INFO [2022-01-10 05:40:20,517] jepsen worker 66 - jepsen.util 1416 :invoke :read [22 nil] INFO [2022-01-10 05:40:20,600] jepsen worker 19 - jepsen.util 1729 :ok :read [6 25] INFO [2022-01-10 05:40:20,601] jepsen worker 19 - jepsen.util 1729 :invoke :write [6 31] INFO [2022-01-10 05:40:20,611] jepsen worker 79 - jepsen.util 1519 :info :write [26 19] :RequestTimeoutException INFO [2022-01-10 05:40:20,612] jepsen worker 79 - jepsen.util 1609 :invoke :read [26 nil] INFO [2022-01-10 05:40:20,635] jepsen worker 79 - jepsen.util 1609 :ok :read [26 10] INFO [2022-01-10 05:40:20,636] jepsen worker 79 - jepsen.util 1609 :invoke :write [26 27] INFO [2022-01-10 05:40:20,752] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2631578947368421}, :ns_1@172.23.107.227 {:progress 0.1876832844574781}, :ns_1@172.23.105.187 {:progress 0.1876832844574781}, :ns_1@172.23.107.228 {:progress 0.2647058823529411}} INFO [2022-01-10 05:40:20,752] jepsen worker nemesis - couchbase.util Rebalance check count: 176 INFO [2022-01-10 05:40:20,753] jepsen worker nemesis - couchbase.util Rebalance stuck count: 129 INFO [2022-01-10 05:40:20,772] jepsen worker 26 - jepsen.util 1286 :info :write [8 32] :SyncWriteAmbiguous INFO [2022-01-10 05:40:20,772] jepsen worker 26 - jepsen.util 1376 :invoke :write [8 27] INFO [2022-01-10 05:40:20,832] jepsen worker 57 - jepsen.util 1587 :info :write [19 29] :RequestTimeoutException INFO [2022-01-10 05:40:20,832] jepsen worker 57 - jepsen.util 1677 :invoke :read [19 nil] INFO [2022-01-10 05:40:20,985] jepsen worker 47 - jepsen.util 1397 :info :write [15 44] :RequestTimeoutException INFO [2022-01-10 05:40:20,986] jepsen worker 47 - jepsen.util 1487 :invoke :read [15 nil] INFO [2022-01-10 05:40:21,191] jepsen worker 15 - jepsen.util 1185 :info :write [5 8] :RequestTimeoutException INFO [2022-01-10 05:40:21,192] jepsen worker 15 - jepsen.util 1275 :invoke :write [5 46] INFO [2022-01-10 05:40:21,285] jepsen worker 56 - jepsen.util 1676 :info :write [18 46] :RequestTimeoutException INFO [2022-01-10 05:40:21,286] jepsen worker 56 - jepsen.util 1766 :invoke :read [18 nil] WARN [2022-01-10 05:40:21,307] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9132848,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9132848,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9026756,"encode_duration_us":19534,"operation_id":"0x2f55","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10008411}],"total_count":1}} WARN [2022-01-10 05:40:21,307] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":1026179,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":18158790,"total_server_duration_us":744240,"operation_id":"0x2c45","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":28864685},{"operation_name":"upsert","last_dispatch_duration_us":9132848,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9132848,"total_server_duration_us":9026756,"encode_duration_us":19534,"operation_id":"0x2f55","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10008411},{"operation_name":"upsert","last_dispatch_duration_us":29996,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":1053372,"total_server_duration_us":463630,"encode_duration_us":21997,"operation_id":"0x302d","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10007157},{"operation_name":"upsert","last_dispatch_duration_us":45492,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":8664614,"total_server_duration_us":2983342,"encode_duration_us":16663,"operation_id":"0x306d","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10007153},{"operation_name":"upsert","last_dispatch_duration_us":32096,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7804542,"total_server_duration_us":653296,"encode_duration_us":27068,"operation_id":"0x30ab","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10006509},{"operation_name":"upsert","last_dispatch_duration_us":58010,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9018233,"total_server_duration_us":2908084,"encode_duration_us":19811,"operation_id":"0x305d","timeout_ms":10000,"last_server_duration_us":21813,"total_duration_us":10006431},{"operation_name":"upsert","last_dispatch_duration_us":351034,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8347637,"total_server_duration_us":250896,"encode_duration_us":16945,"operation_id":"0x305b","timeout_ms":10000,"last_server_duration_us":24755,"total_duration_us":10005927},{"operation_name":"upsert","last_dispatch_duration_us":836427,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":5908470,"total_server_duration_us":1448674,"encode_duration_us":20768,"operation_id":"0x3108","timeout_ms":10000,"last_server_duration_us":56401,"total_duration_us":10005604},{"operation_name":"upsert","last_dispatch_duration_us":299118,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1497727,"total_server_duration_us":202446,"encode_duration_us":27334,"operation_id":"0x3067","timeout_ms":10000,"last_server_duration_us":20841,"total_duration_us":10005412},{"operation_name":"upsert","last_dispatch_duration_us":445893,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":3216859,"total_server_duration_us":707216,"encode_duration_us":19209,"operation_id":"0x2f86","timeout_ms":10000,"last_server_duration_us":107753,"total_duration_us":10004933}],"total_count":17}} WARN [2022-01-10 05:40:21,308] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9166884,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9625644,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9068375,"encode_duration_us":18376,"operation_id":"0x2f5d","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10003273},{"operation_name":"upsert","last_dispatch_duration_us":15884031,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15884031,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9033121,"encode_duration_us":24390,"operation_id":"0x2f0d","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10003508}],"total_count":2}} INFO [2022-01-10 05:40:21,321] jepsen worker 54 - jepsen.util 1674 :info :write [18 38] :RequestTimeoutException INFO [2022-01-10 05:40:21,321] jepsen worker 54 - jepsen.util 1764 :invoke :read [18 nil] WARN [2022-01-10 05:40:21,404] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9178655,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14523382,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9867362,"encode_duration_us":18888,"operation_id":"0x2e94","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10005847},{"operation_name":"upsert","last_dispatch_duration_us":18684467,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":22417806,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9799707,"encode_duration_us":18312,"operation_id":"0x2cae","timeout_ms":10000,"last_server_duration_us":9190751,"total_duration_us":10009104}],"total_count":2}} WARN [2022-01-10 05:40:21,407] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":24662348,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":24662348,"total_server_duration_us":38,"operation_id":"0x2d19","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":24664386},{"operation_name":"get","last_dispatch_duration_us":10563620,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10563620,"total_server_duration_us":27,"operation_id":"0x2fbb","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":10564733},{"operation_name":"upsert","last_dispatch_duration_us":10764329,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":13617021,"total_server_duration_us":10627731,"encode_duration_us":16427,"operation_id":"0x2f2f","timeout_ms":10000,"last_server_duration_us":9704464,"total_duration_us":10009285},{"operation_name":"upsert","last_dispatch_duration_us":981750,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":2435698,"total_server_duration_us":915017,"encode_duration_us":21068,"operation_id":"0x3056","timeout_ms":10000,"last_server_duration_us":65906,"total_duration_us":10009168},{"operation_name":"upsert","last_dispatch_duration_us":9032015,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11987062,"total_server_duration_us":245120,"encode_duration_us":16357,"operation_id":"0x2f0f","timeout_ms":10000,"last_server_duration_us":23054,"total_duration_us":10009014},{"operation_name":"upsert","last_dispatch_duration_us":414212,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":1381772,"total_server_duration_us":353237,"encode_duration_us":22979,"operation_id":"0x2f65","timeout_ms":10000,"last_server_duration_us":63182,"total_duration_us":10008744},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":16641,"operation_id":"0x3091","timeout_ms":10000,"total_duration_us":10008004},{"operation_name":"upsert","last_dispatch_duration_us":7540995,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":7540995,"total_server_duration_us":42743,"encode_duration_us":18180,"operation_id":"0x306b","timeout_ms":10000,"last_server_duration_us":42743,"total_duration_us":10007596},{"operation_name":"upsert","last_dispatch_duration_us":11019316,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":14251910,"total_server_duration_us":1117223,"encode_duration_us":16719,"operation_id":"0x3006","timeout_ms":10000,"last_server_duration_us":94870,"total_duration_us":10004090},{"operation_name":"upsert","last_dispatch_duration_us":1337572,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":9636285,"total_server_duration_us":2982645,"encode_duration_us":17465,"operation_id":"0x2fba","timeout_ms":10000,"last_server_duration_us":192512,"total_duration_us":10003509}],"total_count":15}} WARN [2022-01-10 05:40:21,409] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7935678,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10258414,"total_server_duration_us":270716,"encode_duration_us":22121,"operation_id":"0x3055","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10008658},{"operation_name":"upsert","last_dispatch_duration_us":154951,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":473882,"total_server_duration_us":59799,"encode_duration_us":18876,"operation_id":"0x3066","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10008657},{"operation_name":"upsert","last_dispatch_duration_us":87893,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1335040,"total_server_duration_us":123725,"encode_duration_us":16847,"operation_id":"0x2f3a","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10006197},{"operation_name":"upsert","last_dispatch_duration_us":7930957,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9221510,"total_server_duration_us":156394,"encode_duration_us":22402,"operation_id":"0x30a7","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10005675},{"operation_name":"upsert","last_dispatch_duration_us":76754,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2532272,"total_server_duration_us":275473,"encode_duration_us":19049,"operation_id":"0x305c","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10005294},{"operation_name":"upsert","last_dispatch_duration_us":4262738,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11680651,"total_server_duration_us":630801,"encode_duration_us":21306,"operation_id":"0x3054","timeout_ms":10000,"last_server_duration_us":53974,"total_duration_us":10004365},{"operation_name":"upsert","last_dispatch_duration_us":8490634,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9247620,"total_server_duration_us":134032,"encode_duration_us":19374,"operation_id":"0x305a","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10004349},{"operation_name":"upsert","last_dispatch_duration_us":9166884,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9625644,"total_server_duration_us":9068375,"encode_duration_us":18376,"operation_id":"0x2f5d","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10003273},{"operation_name":"upsert","last_dispatch_duration_us":7941153,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9316944,"total_server_duration_us":240193,"encode_duration_us":16428,"operation_id":"0x30ce","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10002436},{"operation_name":"upsert","last_dispatch_duration_us":4899872,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12233625,"total_server_duration_us":1336630,"encode_duration_us":16330,"operation_id":"0x2fe5","timeout_ms":10000,"last_server_duration_us":153489,"total_duration_us":10001802}],"total_count":19}} WARN [2022-01-10 05:40:21,507] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10171179,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10171179,"total_server_duration_us":10451,"operation_id":"0x30d0","timeout_ms":30000,"last_server_duration_us":10451,"total_duration_us":10173863},{"operation_name":"upsert","last_dispatch_duration_us":30429,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9690638,"total_server_duration_us":206538,"encode_duration_us":20702,"operation_id":"0x3016","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10009567},{"operation_name":"upsert","last_dispatch_duration_us":10345199,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11327920,"total_server_duration_us":183341,"encode_duration_us":17119,"operation_id":"0x309f","timeout_ms":10000,"last_server_duration_us":150501,"total_duration_us":10009263},{"operation_name":"upsert","last_dispatch_duration_us":10925504,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":19438727,"total_server_duration_us":927728,"encode_duration_us":16381,"operation_id":"0x2f30","timeout_ms":10000,"last_server_duration_us":59249,"total_duration_us":10009225},{"operation_name":"upsert","last_dispatch_duration_us":10951155,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11947446,"total_server_duration_us":613417,"encode_duration_us":17245,"operation_id":"0x30a3","timeout_ms":10000,"last_server_duration_us":545402,"total_duration_us":10007021},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":16534,"operation_id":"0x30da","timeout_ms":10000,"total_duration_us":10006908},{"operation_name":"upsert","last_dispatch_duration_us":10698295,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":13868870,"total_server_duration_us":9284935,"encode_duration_us":16162,"operation_id":"0x2fe8","timeout_ms":10000,"last_server_duration_us":9169382,"total_duration_us":10001968},{"operation_name":"upsert","last_dispatch_duration_us":9092402,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9092402,"total_server_duration_us":9053293,"encode_duration_us":23663,"operation_id":"0x3004","timeout_ms":10000,"last_server_duration_us":9053293,"total_duration_us":9098516},{"operation_name":"upsert","last_dispatch_duration_us":9087047,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9087047,"total_server_duration_us":9033121,"encode_duration_us":24149,"operation_id":"0x3116","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":9095320},{"operation_name":"get","last_dispatch_duration_us":7389095,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":7389095,"total_server_duration_us":32,"operation_id":"0x306c","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":7391043}],"total_count":15}} WARN [2022-01-10 05:40:21,508] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10698295,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13868870,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9284935,"encode_duration_us":16162,"operation_id":"0x2fe8","timeout_ms":10000,"last_server_duration_us":9169382,"total_duration_us":10001968},{"operation_name":"upsert","last_dispatch_duration_us":10072298,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11689053,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9636481,"encode_duration_us":18187,"operation_id":"0x2ebb","timeout_ms":10000,"last_server_duration_us":9080926,"total_duration_us":10009580},{"operation_name":"upsert","last_dispatch_duration_us":10570133,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":11359624,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":10321580,"encode_duration_us":18569,"operation_id":"0x2e6f","timeout_ms":10000,"last_server_duration_us":9742786,"total_duration_us":10002511}],"total_count":3}} WARN [2022-01-10 05:40:21,605] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15233255,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":15233255,"total_server_duration_us":11427,"operation_id":"0x2e0e","timeout_ms":30000,"last_server_duration_us":11427,"total_duration_us":15235230},{"operation_name":"upsert","last_dispatch_duration_us":78149,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":78149,"total_server_duration_us":21243,"encode_duration_us":17198,"operation_id":"0x3097","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10009685},{"operation_name":"upsert","last_dispatch_duration_us":171044,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":354787,"total_server_duration_us":39619,"encode_duration_us":22038,"operation_id":"0x30c1","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10009319},{"operation_name":"upsert","last_dispatch_duration_us":173947,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9825206,"total_server_duration_us":156718,"encode_duration_us":24932,"operation_id":"0x3027","timeout_ms":10000,"last_server_duration_us":26774,"total_duration_us":10008785},{"operation_name":"upsert","last_dispatch_duration_us":8937641,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9808043,"total_server_duration_us":178584,"encode_duration_us":16443,"operation_id":"0x3015","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10008514},{"operation_name":"upsert","last_dispatch_duration_us":165939,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":361654,"total_server_duration_us":45775,"encode_duration_us":16432,"operation_id":"0x2fb9","timeout_ms":10000,"last_server_duration_us":22804,"total_duration_us":10007618},{"operation_name":"upsert","last_dispatch_duration_us":616501,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9851656,"total_server_duration_us":3380688,"encode_duration_us":16134,"operation_id":"0x2fdc","timeout_ms":10000,"last_server_duration_us":189630,"total_duration_us":10005961},{"operation_name":"upsert","last_dispatch_duration_us":84509,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":84509,"total_server_duration_us":20203,"encode_duration_us":16469,"operation_id":"0x30a8","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10005497},{"operation_name":"upsert","last_dispatch_duration_us":10290594,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10290594,"total_server_duration_us":89286,"encode_duration_us":17752,"operation_id":"0x2f10","timeout_ms":10000,"last_server_duration_us":89286,"total_duration_us":10005486},{"operation_name":"upsert","last_dispatch_duration_us":104972,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9597903,"total_server_duration_us":132500,"encode_duration_us":22078,"operation_id":"0x3028","timeout_ms":10000,"last_server_duration_us":28127,"total_duration_us":10004539}],"total_count":19}} WARN [2022-01-10 05:40:21,608] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9391589,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11903187,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9379229,"encode_duration_us":17040,"operation_id":"0x2f0e","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10001335},{"operation_name":"upsert","last_dispatch_duration_us":27493482,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":27493482,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9034183,"encode_duration_us":19441,"operation_id":"0x2c7b","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10005719},{"operation_name":"upsert","last_dispatch_duration_us":9333728,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13879491,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9677304,"encode_duration_us":18313,"operation_id":"0x2eae","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10005341},{"operation_name":"upsert","last_dispatch_duration_us":9802796,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11469886,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9420911,"encode_duration_us":18156,"operation_id":"0x2dca","timeout_ms":10000,"last_server_duration_us":9101143,"total_duration_us":10006418},{"operation_name":"upsert","last_dispatch_duration_us":10764329,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13617021,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10627731,"encode_duration_us":16427,"operation_id":"0x2f2f","timeout_ms":10000,"last_server_duration_us":9704464,"total_duration_us":10009285}],"total_count":5}} INFO [2022-01-10 05:40:21,676] jepsen worker 82 - jepsen.util 1342 :info :write [27 12] :RequestTimeoutException INFO [2022-01-10 05:40:21,676] jepsen worker 73 - jepsen.util 1513 :info :write [24 11] :RequestTimeoutException INFO [2022-01-10 05:40:21,677] jepsen worker 73 - jepsen.util 1603 :invoke :write [24 6] INFO [2022-01-10 05:40:21,677] jepsen worker 82 - jepsen.util 1432 :invoke :read [27 nil] INFO [2022-01-10 05:40:21,680] jepsen worker 67 - jepsen.util 1417 :info :write [22 49] :RequestTimeoutException INFO [2022-01-10 05:40:21,680] jepsen worker 67 - jepsen.util 1507 :invoke :read [22 nil] INFO [2022-01-10 05:40:21,687] jepsen worker 64 - jepsen.util 1594 :info :write [21 36] :RequestTimeoutException INFO [2022-01-10 05:40:21,687] jepsen worker 22 - jepsen.util 1372 :info :write [7 3] :RequestTimeoutException INFO [2022-01-10 05:40:21,688] jepsen worker 64 - jepsen.util 1684 :invoke :write [21 43] INFO [2022-01-10 05:40:21,688] jepsen worker 22 - jepsen.util 1462 :invoke :read [7 nil] INFO [2022-01-10 05:40:21,696] jepsen worker 17 - jepsen.util 1367 :info :write [5 18] :RequestTimeoutException INFO [2022-01-10 05:40:21,696] jepsen worker 25 - jepsen.util 1465 :info :write [8 2] :RequestTimeoutException INFO [2022-01-10 05:40:21,697] jepsen worker 25 - jepsen.util 1555 :invoke :read [8 nil] INFO [2022-01-10 05:40:21,697] jepsen worker 17 - jepsen.util 1457 :invoke :write [5 0] INFO [2022-01-10 05:40:21,760] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2631578947368421}, :ns_1@172.23.107.227 {:progress 0.1876832844574781}, :ns_1@172.23.105.187 {:progress 0.1876832844574781}, :ns_1@172.23.107.228 {:progress 0.2647058823529411}} INFO [2022-01-10 05:40:21,760] jepsen worker nemesis - couchbase.util Rebalance check count: 177 INFO [2022-01-10 05:40:21,760] jepsen worker nemesis - couchbase.util Rebalance stuck count: 130 INFO [2022-01-10 05:40:21,815] jepsen worker 23 - jepsen.util 1643 :ok :read [7 :nil] INFO [2022-01-10 05:40:21,816] jepsen worker 23 - jepsen.util 1643 :invoke :read [7 nil] INFO [2022-01-10 05:40:21,939] jepsen worker 23 - jepsen.util 1643 :ok :read [7 :nil] INFO [2022-01-10 05:40:21,940] jepsen worker 23 - jepsen.util 1643 :invoke :read [7 nil] INFO [2022-01-10 05:40:21,946] jepsen worker 58 - jepsen.util 1498 :info :write [19 37] :RequestTimeoutException INFO [2022-01-10 05:40:21,947] jepsen worker 58 - jepsen.util 1588 :invoke :write [19 38] INFO [2022-01-10 05:40:21,951] jepsen worker 16 - jepsen.util 1816 :info :write [5 2] :RequestTimeoutException INFO [2022-01-10 05:40:21,952] jepsen worker 16 - jepsen.util 1906 :invoke :write [5 32] INFO [2022-01-10 05:40:22,005] jepsen worker 23 - jepsen.util 1643 :ok :read [7 :nil] INFO [2022-01-10 05:40:22,006] jepsen worker 23 - jepsen.util 1643 :invoke :write [7 26] INFO [2022-01-10 05:40:22,018] jepsen worker 6 - jepsen.util 1536 :ok :read [2 14] INFO [2022-01-10 05:40:22,019] jepsen worker 6 - jepsen.util 1536 :invoke :read [2 nil] INFO [2022-01-10 05:40:22,093] jepsen worker 6 - jepsen.util 1536 :ok :read [2 14] INFO [2022-01-10 05:40:22,093] jepsen worker 6 - jepsen.util 1536 :invoke :write [2 19] INFO [2022-01-10 05:40:22,379] jepsen worker 39 - jepsen.util 1749 :info :write [13 25] :RequestTimeoutException INFO [2022-01-10 05:40:22,380] jepsen worker 39 - jepsen.util 1839 :invoke :read [13 nil] INFO [2022-01-10 05:40:22,475] jepsen worker 77 - jepsen.util 1877 :info :write [25 12] :RequestTimeoutException INFO [2022-01-10 05:40:22,476] jepsen worker 77 - jepsen.util 1967 :invoke :write [25 20] INFO [2022-01-10 05:40:22,732] jepsen worker 67 - jepsen.util 1507 :ok :read [22 41] INFO [2022-01-10 05:40:22,732] jepsen worker 67 - jepsen.util 1507 :invoke :read [22 nil] INFO [2022-01-10 05:40:22,768] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2631578947368421}, :ns_1@172.23.107.227 {:progress 0.1876832844574781}, :ns_1@172.23.105.187 {:progress 0.1876832844574781}, :ns_1@172.23.107.228 {:progress 0.2647058823529411}} INFO [2022-01-10 05:40:22,780] jepsen worker nemesis - couchbase.util Rebalance check count: 178 INFO [2022-01-10 05:40:22,780] jepsen worker nemesis - couchbase.util Rebalance stuck count: 131 INFO [2022-01-10 05:40:22,891] jepsen worker 46 - jepsen.util 1666 :info :write [15 19] :RequestTimeoutException INFO [2022-01-10 05:40:22,893] jepsen worker 46 - jepsen.util 1756 :invoke :write [15 28] INFO [2022-01-10 05:40:22,898] jepsen worker 65 - jepsen.util 1415 :ok :read [21 7] INFO [2022-01-10 05:40:22,899] jepsen worker 65 - jepsen.util 1415 :invoke :write [21 47] INFO [2022-01-10 05:40:23,008] jepsen worker 42 - jepsen.util 1482 :ok :read [14 14] INFO [2022-01-10 05:40:23,009] jepsen worker 42 - jepsen.util 1482 :invoke :read [14 nil] INFO [2022-01-10 05:40:23,018] jepsen worker 72 - jepsen.util 1692 :ok :read [24 18] INFO [2022-01-10 05:40:23,018] jepsen worker 72 - jepsen.util 1692 :invoke :write [24 1] INFO [2022-01-10 05:40:23,044] jepsen worker 61 - jepsen.util 1501 :ok :read [20 :nil] INFO [2022-01-10 05:40:23,044] jepsen worker 25 - jepsen.util 1555 :ok :read [8 :nil] INFO [2022-01-10 05:40:23,045] jepsen worker 61 - jepsen.util 1501 :invoke :write [20 26] INFO [2022-01-10 05:40:23,045] jepsen worker 25 - jepsen.util 1555 :invoke :write [8 3] INFO [2022-01-10 05:40:23,057] jepsen worker 8 - jepsen.util 1628 :ok :read [2 14] INFO [2022-01-10 05:40:23,058] jepsen worker 8 - jepsen.util 1628 :invoke :read [2 nil] INFO [2022-01-10 05:40:23,103] jepsen worker 42 - jepsen.util 1482 :ok :read [14 14] INFO [2022-01-10 05:40:23,104] jepsen worker 42 - jepsen.util 1482 :invoke :write [14 15] INFO [2022-01-10 05:40:23,447] jepsen worker 20 - jepsen.util 1370 :info :write [6 31] :RequestTimeoutException INFO [2022-01-10 05:40:23,448] jepsen worker 20 - jepsen.util 1460 :invoke :write [6 15] INFO [2022-01-10 05:40:23,554] jepsen worker 39 - jepsen.util 1839 :ok :read [13 15] INFO [2022-01-10 05:40:23,555] jepsen worker 39 - jepsen.util 1839 :invoke :write [13 29] INFO [2022-01-10 05:40:23,565] jepsen worker 67 - jepsen.util 1507 :ok :read [22 41] INFO [2022-01-10 05:40:23,566] jepsen worker 67 - jepsen.util 1507 :invoke :write [22 47] INFO [2022-01-10 05:40:23,660] jepsen worker 8 - jepsen.util 1628 :ok :read [2 14] INFO [2022-01-10 05:40:23,660] jepsen worker 8 - jepsen.util 1628 :invoke :read [2 nil] INFO [2022-01-10 05:40:23,721] jepsen worker 35 - jepsen.util 1475 :info :write [11 37] :SyncWriteAmbiguous INFO [2022-01-10 05:40:23,722] jepsen worker 35 - jepsen.util 1565 :invoke :write [11 13] INFO [2022-01-10 05:40:23,787] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2748538011695907}, :ns_1@172.23.107.227 {:progress 0.1876832844574781}, :ns_1@172.23.105.187 {:progress 0.1876832844574781}, :ns_1@172.23.107.228 {:progress 0.2705882352941177}} INFO [2022-01-10 05:40:23,787] jepsen worker nemesis - couchbase.util Rebalance check count: 179 INFO [2022-01-10 05:40:23,787] jepsen worker nemesis - couchbase.util Rebalance stuck count: 131 INFO [2022-01-10 05:40:23,861] jepsen worker 8 - jepsen.util 1628 :ok :read [2 14] INFO [2022-01-10 05:40:23,862] jepsen worker 8 - jepsen.util 1628 :invoke :write [2 45] INFO [2022-01-10 05:40:23,891] jepsen worker 41 - jepsen.util 1301 :info :write [13 19] :RequestTimeoutException INFO [2022-01-10 05:40:23,892] jepsen worker 41 - jepsen.util 1391 :invoke :read [13 nil] INFO [2022-01-10 05:40:23,906] jepsen worker 84 - jepsen.util 1614 :info :write [28 5] :RequestTimeoutException INFO [2022-01-10 05:40:23,907] jepsen worker 84 - jepsen.util 1704 :invoke :write [28 16] INFO [2022-01-10 05:40:23,989] jepsen worker 22 - jepsen.util 1462 :ok :read [7 :nil] INFO [2022-01-10 05:40:23,989] jepsen worker 22 - jepsen.util 1462 :invoke :write [7 27] INFO [2022-01-10 05:40:23,991] jepsen worker 43 - jepsen.util 1753 :ok :read [14 14] INFO [2022-01-10 05:40:23,991] jepsen worker 43 - jepsen.util 1753 :invoke :write [14 19] INFO [2022-01-10 05:40:24,373] jepsen worker 41 - jepsen.util 1391 :ok :read [13 15] INFO [2022-01-10 05:40:24,373] jepsen worker 41 - jepsen.util 1391 :invoke :write [13 17] INFO [2022-01-10 05:40:24,450] jepsen worker 33 - jepsen.util 1563 :info :write [11 43] :RequestTimeoutException INFO [2022-01-10 05:40:24,450] jepsen worker 33 - jepsen.util 1653 :invoke :read [11 nil] INFO [2022-01-10 05:40:24,710] jepsen worker 68 - jepsen.util 1508 :info :write [22 16] :RequestTimeoutException INFO [2022-01-10 05:40:24,711] jepsen worker 68 - jepsen.util 1598 :invoke :read [22 nil] INFO [2022-01-10 05:40:24,793] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2748538011695907}, :ns_1@172.23.107.227 {:progress 0.1876832844574781}, :ns_1@172.23.105.187 {:progress 0.1876832844574781}, :ns_1@172.23.107.228 {:progress 0.2705882352941177}} INFO [2022-01-10 05:40:24,793] jepsen worker nemesis - couchbase.util Rebalance check count: 180 INFO [2022-01-10 05:40:24,793] jepsen worker nemesis - couchbase.util Rebalance stuck count: 132 INFO [2022-01-10 05:40:24,903] jepsen worker 31 - jepsen.util 1831 :info :write [10 33] :SyncWriteAmbiguous INFO [2022-01-10 05:40:24,904] jepsen worker 31 - jepsen.util 1921 :invoke :read [10 nil] INFO [2022-01-10 05:40:25,052] jepsen worker 33 - jepsen.util 1653 :ok :read [11 21] INFO [2022-01-10 05:40:25,053] jepsen worker 33 - jepsen.util 1653 :invoke :write [11 31] INFO [2022-01-10 05:40:25,126] jepsen worker 3 - jepsen.util 1713 :ok :read [1 41] INFO [2022-01-10 05:40:25,126] jepsen worker 3 - jepsen.util 1713 :invoke :read [1 nil] INFO [2022-01-10 05:40:25,129] jepsen worker 5 - jepsen.util 1535 :ok :read [1 41] INFO [2022-01-10 05:40:25,130] jepsen worker 5 - jepsen.util 1535 :invoke :write [1 15] INFO [2022-01-10 05:40:25,176] jepsen worker 74 - jepsen.util 1964 :info :write [24 22] :RequestTimeoutException INFO [2022-01-10 05:40:25,177] jepsen worker 74 - jepsen.util 2054 :invoke :write [24 26] INFO [2022-01-10 05:40:25,204] jepsen worker 54 - jepsen.util 1764 :ok :read [18 14] INFO [2022-01-10 05:40:25,205] jepsen worker 54 - jepsen.util 1764 :invoke :read [18 nil] INFO [2022-01-10 05:40:25,213] jepsen worker 3 - jepsen.util 1713 :ok :read [1 41] INFO [2022-01-10 05:40:25,214] jepsen worker 3 - jepsen.util 1713 :invoke :read [1 nil] INFO [2022-01-10 05:40:25,282] jepsen worker 54 - jepsen.util 1764 :ok :read [18 14] INFO [2022-01-10 05:40:25,282] jepsen worker 54 - jepsen.util 1764 :invoke :read [18 nil] INFO [2022-01-10 05:40:25,290] jepsen worker 3 - jepsen.util 1713 :ok :read [1 41] INFO [2022-01-10 05:40:25,291] jepsen worker 3 - jepsen.util 1713 :invoke :read [1 nil] INFO [2022-01-10 05:40:25,363] jepsen worker 54 - jepsen.util 1764 :ok :read [18 14] INFO [2022-01-10 05:40:25,364] jepsen worker 54 - jepsen.util 1764 :invoke :write [18 8] INFO [2022-01-10 05:40:25,391] jepsen worker 3 - jepsen.util 1713 :ok :read [1 41] INFO [2022-01-10 05:40:25,392] jepsen worker 3 - jepsen.util 1713 :invoke :read [1 nil] INFO [2022-01-10 05:40:25,507] jepsen worker 3 - jepsen.util 1713 :ok :read [1 41] INFO [2022-01-10 05:40:25,508] jepsen worker 3 - jepsen.util 1713 :invoke :write [1 7] INFO [2022-01-10 05:40:25,532] jepsen worker 87 - jepsen.util 1707 :info :write [29 28] :SyncWriteAmbiguous INFO [2022-01-10 05:40:25,533] jepsen worker 87 - jepsen.util 1797 :invoke :write [29 0] INFO [2022-01-10 05:40:25,574] jepsen worker 27 - jepsen.util 1467 :info :write [9 32] :RequestTimeoutException INFO [2022-01-10 05:40:25,575] jepsen worker 27 - jepsen.util 1557 :invoke :read [9 nil] INFO [2022-01-10 05:40:25,799] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2748538011695907}, :ns_1@172.23.107.227 {:progress 0.1876832844574781}, :ns_1@172.23.105.187 {:progress 0.1876832844574781}, :ns_1@172.23.107.228 {:progress 0.2705882352941177}} INFO [2022-01-10 05:40:25,800] jepsen worker nemesis - couchbase.util Rebalance check count: 181 INFO [2022-01-10 05:40:25,800] jepsen worker nemesis - couchbase.util Rebalance stuck count: 133 INFO [2022-01-10 05:40:25,895] jepsen worker 36 - jepsen.util 1206 :info :write [12 12] :RequestTimeoutException INFO [2022-01-10 05:40:25,896] jepsen worker 36 - jepsen.util 1296 :invoke :write [12 34] INFO [2022-01-10 05:40:25,966] jepsen worker 11 - jepsen.util 1451 :info :write [3 11] :RequestTimeoutException INFO [2022-01-10 05:40:25,967] jepsen worker 11 - jepsen.util 1541 :invoke :read [3 nil] INFO [2022-01-10 05:40:26,032] jepsen worker 71 - jepsen.util 1601 :info :write [23 30] :RequestTimeoutException INFO [2022-01-10 05:40:26,033] jepsen worker 71 - jepsen.util 1691 :invoke :read [23 nil] INFO [2022-01-10 05:40:26,086] jepsen worker 11 - jepsen.util 1541 :ok :read [3 2] INFO [2022-01-10 05:40:26,086] jepsen worker 11 - jepsen.util 1541 :invoke :write [3 21] INFO [2022-01-10 05:40:26,099] jepsen worker 70 - jepsen.util 1870 :info :write [23 23] :SyncWriteAmbiguous INFO [2022-01-10 05:40:26,100] jepsen worker 70 - jepsen.util 1960 :invoke :read [23 nil] INFO [2022-01-10 05:40:26,206] jepsen worker 29 - jepsen.util 1559 :info :write [9 32] :RequestTimeoutException INFO [2022-01-10 05:40:26,207] jepsen worker 29 - jepsen.util 1649 :invoke :write [9 22] INFO [2022-01-10 05:40:26,260] jepsen worker 4 - jepsen.util 1534 :ok :read [1 41] INFO [2022-01-10 05:40:26,260] jepsen worker 4 - jepsen.util 1534 :invoke :write [1 22] INFO [2022-01-10 05:40:26,317] jepsen worker 70 - jepsen.util 1960 :ok :read [23 26] INFO [2022-01-10 05:40:26,317] jepsen worker 70 - jepsen.util 1960 :invoke :read [23 nil] INFO [2022-01-10 05:40:26,332] jepsen worker 80 - jepsen.util 1610 :info :write [26 27] :SyncWriteAmbiguous INFO [2022-01-10 05:40:26,332] jepsen worker 80 - jepsen.util 1700 :invoke :write [26 32] INFO [2022-01-10 05:40:26,345] jepsen worker 12 - jepsen.util 1272 :info :write [4 17] :SyncWriteAmbiguous INFO [2022-01-10 05:40:26,347] jepsen worker 12 - jepsen.util 1362 :invoke :write [4 33] INFO [2022-01-10 05:40:26,347] jepsen worker 70 - jepsen.util 1960 :ok :read [23 26] INFO [2022-01-10 05:40:26,348] jepsen worker 70 - jepsen.util 1960 :invoke :read [23 nil] INFO [2022-01-10 05:40:26,377] jepsen worker 70 - jepsen.util 1960 :ok :read [23 26] INFO [2022-01-10 05:40:26,378] jepsen worker 70 - jepsen.util 1960 :invoke :write [23 17] INFO [2022-01-10 05:40:26,381] jepsen worker 49 - jepsen.util 1309 :info :write [16 37] :RequestTimeoutException INFO [2022-01-10 05:40:26,381] jepsen worker 49 - jepsen.util 1399 :invoke :read [16 nil] INFO [2022-01-10 05:40:26,384] jepsen worker 1 - jepsen.util 1621 :info :write [0 40] :RequestTimeoutException INFO [2022-01-10 05:40:26,385] jepsen worker 1 - jepsen.util 1711 :invoke :read [0 nil] INFO [2022-01-10 05:40:26,400] jepsen worker 53 - jepsen.util 1583 :info :write [17 46] :RequestTimeoutException INFO [2022-01-10 05:40:26,400] jepsen worker 53 - jepsen.util 1673 :invoke :write [17 4] INFO [2022-01-10 05:40:26,466] jepsen worker 89 - jepsen.util 1799 :info :write [29 47] :RequestTimeoutException INFO [2022-01-10 05:40:26,466] jepsen worker 89 - jepsen.util 1889 :invoke :read [29 nil] INFO [2022-01-10 05:40:26,491] jepsen worker 76 - jepsen.util 1246 :ok :read [25 :nil] INFO [2022-01-10 05:40:26,492] jepsen worker 76 - jepsen.util 1246 :invoke :write [25 46] INFO [2022-01-10 05:40:26,496] jepsen worker 51 - jepsen.util 1671 :ok :read [17 41] INFO [2022-01-10 05:40:26,497] jepsen worker 51 - jepsen.util 1671 :invoke :write [17 44] INFO [2022-01-10 05:40:26,634] jepsen worker 89 - jepsen.util 1889 :ok :read [29 33] INFO [2022-01-10 05:40:26,635] jepsen worker 89 - jepsen.util 1889 :invoke :write [29 23] INFO [2022-01-10 05:40:26,645] jepsen worker 45 - jepsen.util 1665 :info :write [15 18] :RequestTimeoutException INFO [2022-01-10 05:40:26,646] jepsen worker 45 - jepsen.util 1755 :invoke :read [15 nil] INFO [2022-01-10 05:40:26,714] jepsen worker 21 - jepsen.util 1731 :ok :read [7 :nil] INFO [2022-01-10 05:40:26,714] jepsen worker 21 - jepsen.util 1731 :invoke :read [7 nil] INFO [2022-01-10 05:40:26,806] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2748538011695907}, :ns_1@172.23.107.227 {:progress 0.1876832844574781}, :ns_1@172.23.105.187 {:progress 0.1876832844574781}, :ns_1@172.23.107.228 {:progress 0.2705882352941177}} INFO [2022-01-10 05:40:26,806] jepsen worker nemesis - couchbase.util Rebalance check count: 182 INFO [2022-01-10 05:40:26,806] jepsen worker nemesis - couchbase.util Rebalance stuck count: 134 INFO [2022-01-10 05:40:26,856] jepsen worker 30 - jepsen.util 1200 :info :write [10 0] :RequestTimeoutException INFO [2022-01-10 05:40:26,856] jepsen worker 30 - jepsen.util 1290 :invoke :read [10 nil] INFO [2022-01-10 05:40:26,861] jepsen worker 55 - jepsen.util 1765 :info :write [18 28] :RequestTimeoutException INFO [2022-01-10 05:40:26,862] jepsen worker 21 - jepsen.util 1731 :ok :read [7 :nil] INFO [2022-01-10 05:40:26,862] jepsen worker 55 - jepsen.util 1855 :invoke :write [18 31] INFO [2022-01-10 05:40:26,863] jepsen worker 21 - jepsen.util 1731 :invoke :read [7 nil] INFO [2022-01-10 05:40:26,875] jepsen worker 24 - jepsen.util 1464 :info :write [8 31] :RequestTimeoutException INFO [2022-01-10 05:40:26,875] jepsen worker 24 - jepsen.util 1554 :invoke :write [8 39] INFO [2022-01-10 05:40:26,908] jepsen worker 21 - jepsen.util 1731 :ok :read [7 :nil] INFO [2022-01-10 05:40:26,909] jepsen worker 21 - jepsen.util 1731 :invoke :read [7 nil] INFO [2022-01-10 05:40:26,940] jepsen worker 21 - jepsen.util 1731 :ok :read [7 :nil] INFO [2022-01-10 05:40:26,941] jepsen worker 21 - jepsen.util 1731 :invoke :write [7 26] INFO [2022-01-10 05:40:27,217] jepsen worker 47 - jepsen.util 1487 :ok :read [15 23] INFO [2022-01-10 05:40:27,218] jepsen worker 47 - jepsen.util 1487 :invoke :write [15 35] INFO [2022-01-10 05:40:27,255] jepsen worker 50 - jepsen.util 1670 :info :write [16 22] :RequestTimeoutException INFO [2022-01-10 05:40:27,255] jepsen worker 88 - jepsen.util 1528 :info :write [29 16] :RequestTimeoutException INFO [2022-01-10 05:40:27,256] jepsen worker 50 - jepsen.util 1760 :invoke :read [16 nil] INFO [2022-01-10 05:40:27,256] jepsen worker 88 - jepsen.util 1618 :invoke :read [29 nil] INFO [2022-01-10 05:40:27,276] jepsen worker 10 - jepsen.util 1720 :info :write [3 47] :RequestTimeoutException INFO [2022-01-10 05:40:27,277] jepsen worker 10 - jepsen.util 1810 :invoke :read [3 nil] INFO [2022-01-10 05:40:27,303] jepsen worker 10 - jepsen.util 1810 :ok :read [3 2] INFO [2022-01-10 05:40:27,304] jepsen worker 10 - jepsen.util 1810 :invoke :read [3 nil] INFO [2022-01-10 05:40:27,328] jepsen worker 10 - jepsen.util 1810 :ok :read [3 2] INFO [2022-01-10 05:40:27,329] jepsen worker 10 - jepsen.util 1810 :invoke :read [3 nil] INFO [2022-01-10 05:40:27,349] jepsen worker 38 - jepsen.util 1568 :info :write [12 8] :RequestTimeoutException INFO [2022-01-10 05:40:27,350] jepsen worker 38 - jepsen.util 1658 :invoke :write [12 4] INFO [2022-01-10 05:40:27,354] jepsen worker 10 - jepsen.util 1810 :ok :read [3 2] INFO [2022-01-10 05:40:27,354] jepsen worker 10 - jepsen.util 1810 :invoke :write [3 27] INFO [2022-01-10 05:40:27,511] jepsen worker 13 - jepsen.util 1273 :info :write [4 45] :RequestTimeoutException INFO [2022-01-10 05:40:27,512] jepsen worker 13 - jepsen.util 1363 :invoke :write [4 32] INFO [2022-01-10 05:40:27,720] jepsen worker 9 - jepsen.util 1809 :info :write [3 31] :RequestTimeoutException INFO [2022-01-10 05:40:27,720] jepsen worker 9 - jepsen.util 1899 :invoke :read [3 nil] INFO [2022-01-10 05:40:27,812] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2865497076023392}, :ns_1@172.23.107.227 {:progress 0.1876832844574781}, :ns_1@172.23.105.187 {:progress 0.1876832844574781}, :ns_1@172.23.107.228 {:progress 0.2764705882352941}} INFO [2022-01-10 05:40:27,812] jepsen worker nemesis - couchbase.util Rebalance check count: 183 INFO [2022-01-10 05:40:27,923] jepsen worker nemesis - couchbase.util Rebalance stuck count: 134 INFO [2022-01-10 05:40:27,926] jepsen worker 85 - jepsen.util 1795 :info :write [28 46] :RequestTimeoutException INFO [2022-01-10 05:40:27,927] jepsen worker 85 - jepsen.util 1885 :invoke :write [28 15] INFO [2022-01-10 05:40:28,030] jepsen worker 32 - jepsen.util 1472 :info :write [10 28] :RequestTimeoutException INFO [2022-01-10 05:40:28,031] jepsen worker 32 - jepsen.util 1562 :invoke :write [10 37] INFO [2022-01-10 05:40:28,101] jepsen worker 14 - jepsen.util 1634 :info :write [4 49] :RequestTimeoutException INFO [2022-01-10 05:40:28,102] jepsen worker 14 - jepsen.util 1724 :invoke :write [4 26] INFO [2022-01-10 05:40:28,227] jepsen worker 44 - jepsen.util 1664 :info :write [14 33] :RequestTimeoutException INFO [2022-01-10 05:40:28,228] jepsen worker 44 - jepsen.util 1754 :invoke :read [14 nil] INFO [2022-01-10 05:40:28,300] jepsen worker 1 - jepsen.util 1711 :ok :read [0 46] INFO [2022-01-10 05:40:28,300] jepsen worker 1 - jepsen.util 1711 :invoke :read [0 nil] INFO [2022-01-10 05:40:28,432] jepsen worker 27 - jepsen.util 1557 :ok :read [9 22] INFO [2022-01-10 05:40:28,433] jepsen worker 27 - jepsen.util 1557 :invoke :read [9 nil] INFO [2022-01-10 05:40:28,581] jepsen worker 26 - jepsen.util 1376 :ok :write [8 27] INFO [2022-01-10 05:40:28,581] jepsen worker 26 - jepsen.util 1376 :invoke :read [8 nil] INFO [2022-01-10 05:40:28,645] jepsen worker 60 - jepsen.util 1950 :info :write [20 5] :RequestTimeoutException INFO [2022-01-10 05:40:28,646] jepsen worker 60 - jepsen.util 2040 :invoke :read [20 nil] INFO [2022-01-10 05:40:28,796] jepsen worker 26 - jepsen.util 1376 :ok :read [8 27] INFO [2022-01-10 05:40:28,796] jepsen worker 26 - jepsen.util 1376 :invoke :write [8 41] INFO [2022-01-10 05:40:28,934] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2923976608187134}, :ns_1@172.23.107.227 {:progress 0.1935483870967742}, :ns_1@172.23.105.187 {:progress 0.1935483870967742}, :ns_1@172.23.107.228 {:progress 0.2823529411764706}} INFO [2022-01-10 05:40:28,934] jepsen worker nemesis - couchbase.util Rebalance check count: 184 INFO [2022-01-10 05:40:28,934] jepsen worker nemesis - couchbase.util Rebalance stuck count: 134 INFO [2022-01-10 05:40:28,940] jepsen worker 37 - jepsen.util 1567 :info :write [12 13] :RequestTimeoutException INFO [2022-01-10 05:40:28,941] jepsen worker 37 - jepsen.util 1657 :invoke :write [12 6] INFO [2022-01-10 05:40:28,958] jepsen worker 18 - jepsen.util 1368 :ok :read [6 25] INFO [2022-01-10 05:40:28,959] jepsen worker 18 - jepsen.util 1368 :invoke :read [6 nil] INFO [2022-01-10 05:40:29,122] jepsen worker 78 - jepsen.util 1518 :info :write [26 47] :RequestTimeoutException INFO [2022-01-10 05:40:29,123] jepsen worker 78 - jepsen.util 1608 :invoke :read [26 nil] INFO [2022-01-10 05:40:29,146] jepsen worker 34 - jepsen.util 1654 :info :write [11 17] :RequestTimeoutException INFO [2022-01-10 05:40:29,147] jepsen worker 34 - jepsen.util 1744 :invoke :read [11 nil] INFO [2022-01-10 05:40:29,148] jepsen worker 78 - jepsen.util 1608 :ok :read [26 10] INFO [2022-01-10 05:40:29,149] jepsen worker 78 - jepsen.util 1608 :invoke :read [26 nil] INFO [2022-01-10 05:40:29,170] jepsen worker 78 - jepsen.util 1608 :ok :read [26 10] INFO [2022-01-10 05:40:29,171] jepsen worker 78 - jepsen.util 1608 :invoke :read [26 nil] INFO [2022-01-10 05:40:29,197] jepsen worker 78 - jepsen.util 1608 :ok :read [26 10] INFO [2022-01-10 05:40:29,197] jepsen worker 78 - jepsen.util 1608 :invoke :read [26 nil] INFO [2022-01-10 05:40:29,205] jepsen worker 1 - jepsen.util 1711 :ok :read [0 46] INFO [2022-01-10 05:40:29,206] jepsen worker 1 - jepsen.util 1711 :invoke :write [0 28] INFO [2022-01-10 05:40:29,211] jepsen worker 66 - jepsen.util 1416 :ok :read [22 41] INFO [2022-01-10 05:40:29,211] jepsen worker 66 - jepsen.util 1416 :invoke :read [22 nil] INFO [2022-01-10 05:40:29,237] jepsen worker 78 - jepsen.util 1608 :ok :read [26 10] INFO [2022-01-10 05:40:29,238] jepsen worker 78 - jepsen.util 1608 :invoke :write [26 26] INFO [2022-01-10 05:40:29,241] jepsen worker 81 - jepsen.util 1521 :info :write [27 2] :RequestTimeoutException INFO [2022-01-10 05:40:29,242] jepsen worker 81 - jepsen.util 1611 :invoke :read [27 nil] INFO [2022-01-10 05:40:29,280] jepsen worker 66 - jepsen.util 1416 :ok :read [22 41] INFO [2022-01-10 05:40:29,281] jepsen worker 66 - jepsen.util 1416 :invoke :write [22 25] INFO [2022-01-10 05:40:29,354] jepsen worker 59 - jepsen.util 1589 :info :write [19 3] :RequestTimeoutException INFO [2022-01-10 05:40:29,355] jepsen worker 59 - jepsen.util 1679 :invoke :write [19 40] INFO [2022-01-10 05:40:29,495] jepsen worker 57 - jepsen.util 1677 :ok :read [19 31] INFO [2022-01-10 05:40:29,496] jepsen worker 57 - jepsen.util 1677 :invoke :read [19 nil] INFO [2022-01-10 05:40:29,636] jepsen worker 86 - jepsen.util 1256 :info :write [28 41] :RequestTimeoutException INFO [2022-01-10 05:40:29,636] jepsen worker 52 - jepsen.util 1762 :info :write [17 49] :RequestTimeoutException INFO [2022-01-10 05:40:29,636] jepsen worker 86 - jepsen.util 1346 :invoke :write [28 29] INFO [2022-01-10 05:40:29,639] jepsen worker 52 - jepsen.util 1852 :invoke :write [17 38] WARN [2022-01-10 05:40:29,715] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9229393,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17607883,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9209191,"encode_duration_us":17029,"operation_id":"0x306e","timeout_ms":10000,"last_server_duration_us":9069231,"total_duration_us":10001493}],"total_count":1}} WARN [2022-01-10 05:40:29,816] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":806543,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":7554586,"total_server_duration_us":1049964,"encode_duration_us":17570,"operation_id":"0x3295","timeout_ms":10000,"last_server_duration_us":271179,"total_duration_us":10009510},{"operation_name":"upsert","last_dispatch_duration_us":65271,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8297040,"total_server_duration_us":309816,"encode_duration_us":22554,"operation_id":"0x3227","timeout_ms":10000,"last_server_duration_us":22887,"total_duration_us":10009371},{"operation_name":"upsert","last_dispatch_duration_us":598781,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8080406,"total_server_duration_us":1645388,"encode_duration_us":28693,"operation_id":"0x31f0","timeout_ms":10000,"last_server_duration_us":78555,"total_duration_us":10007113},{"operation_name":"upsert","last_dispatch_duration_us":642985,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":5012387,"total_server_duration_us":594446,"encode_duration_us":16792,"operation_id":"0x318c","timeout_ms":10000,"last_server_duration_us":87211,"total_duration_us":10005893},{"operation_name":"upsert","last_dispatch_duration_us":125288,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9947073,"total_server_duration_us":153937,"encode_duration_us":17771,"operation_id":"0x321e","timeout_ms":10000,"last_server_duration_us":26953,"total_duration_us":10002999},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","encode_duration_us":16825,"operation_id":"0x3266","timeout_ms":10000,"total_duration_us":10002941},{"operation_name":"upsert","last_dispatch_duration_us":247928,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":7798964,"total_server_duration_us":1560775,"encode_duration_us":20170,"operation_id":"0x3228","timeout_ms":10000,"last_server_duration_us":28036,"total_duration_us":10002922},{"operation_name":"upsert","last_dispatch_duration_us":84395,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9936197,"total_server_duration_us":259066,"encode_duration_us":20673,"operation_id":"0x31f7","timeout_ms":10000,"last_server_duration_us":24497,"total_duration_us":10002907},{"operation_name":"upsert","last_dispatch_duration_us":8753743,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":13558849,"total_server_duration_us":205548,"encode_duration_us":16209,"operation_id":"0x310e","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10001585},{"operation_name":"upsert","last_dispatch_duration_us":8747386,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8807745,"total_server_duration_us":39620,"encode_duration_us":16250,"operation_id":"0x3144","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10001321}],"total_count":18}} INFO [2022-01-10 05:40:29,945] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2923976608187134}, :ns_1@172.23.107.227 {:progress 0.1935483870967742}, :ns_1@172.23.105.187 {:progress 0.1935483870967742}, :ns_1@172.23.107.228 {:progress 0.2823529411764706}} INFO [2022-01-10 05:40:29,945] jepsen worker nemesis - couchbase.util Rebalance check count: 185 INFO [2022-01-10 05:40:29,945] jepsen worker nemesis - couchbase.util Rebalance stuck count: 135 INFO [2022-01-10 05:40:29,946] jepsen worker 0 - jepsen.util 1440 :info :write [0 45] :RequestTimeoutException INFO [2022-01-10 05:40:29,947] jepsen worker 75 - jepsen.util 1245 :info :write [25 16] :SyncWriteAmbiguous INFO [2022-01-10 05:40:29,949] jepsen worker 63 - jepsen.util 1413 :info :write [21 32] :SyncWriteAmbiguous INFO [2022-01-10 05:40:29,949] jepsen worker 0 - jepsen.util 1530 :invoke :read [0 nil] INFO [2022-01-10 05:40:29,950] jepsen worker 75 - jepsen.util 1335 :invoke :read [25 nil] INFO [2022-01-10 05:40:29,950] jepsen worker 63 - jepsen.util 1503 :invoke :read [21 nil] INFO [2022-01-10 05:40:29,957] jepsen worker 57 - jepsen.util 1677 :ok :read [19 31] INFO [2022-01-10 05:40:29,958] jepsen worker 57 - jepsen.util 1677 :invoke :read [19 nil] INFO [2022-01-10 05:40:29,965] jepsen worker 7 - jepsen.util 1627 :info :write [2 18] :RequestTimeoutException INFO [2022-01-10 05:40:29,966] jepsen worker 7 - jepsen.util 1717 :invoke :write [2 39] INFO [2022-01-10 05:40:29,966] jepsen worker 28 - jepsen.util 1468 :info :write [9 18] :RequestTimeoutException INFO [2022-01-10 05:40:29,967] jepsen worker 28 - jepsen.util 1558 :invoke :write [9 15] INFO [2022-01-10 05:40:30,161] jepsen worker 27 - jepsen.util 1557 :ok :read [9 22] INFO [2022-01-10 05:40:30,163] jepsen worker 27 - jepsen.util 1557 :invoke :write [9 20] INFO [2022-01-10 05:40:30,314] jepsen worker 82 - jepsen.util 1432 :ok :read [27 30] INFO [2022-01-10 05:40:30,315] jepsen worker 82 - jepsen.util 1432 :invoke :write [27 9] INFO [2022-01-10 05:40:30,316] jepsen worker 62 - jepsen.util 1772 :info :write [20 35] :RequestTimeoutException INFO [2022-01-10 05:40:30,316] jepsen worker 62 - jepsen.util 1862 :invoke :write [20 34] INFO [2022-01-10 05:40:30,316] jepsen worker 83 - jepsen.util 1793 :info :write [27 19] :RequestTimeoutException INFO [2022-01-10 05:40:30,317] jepsen worker 83 - jepsen.util 1883 :invoke :read [27 nil] INFO [2022-01-10 05:40:30,322] jepsen worker 57 - jepsen.util 1677 :ok :read [19 31] INFO [2022-01-10 05:40:30,323] jepsen worker 57 - jepsen.util 1677 :invoke :read [19 nil] INFO [2022-01-10 05:40:30,486] jepsen worker 81 - jepsen.util 1611 :ok :read [27 30] INFO [2022-01-10 05:40:30,486] jepsen worker 81 - jepsen.util 1611 :invoke :read [27 nil] INFO [2022-01-10 05:40:30,532] jepsen worker 40 - jepsen.util 1300 :info :write [13 11] :RequestTimeoutException INFO [2022-01-10 05:40:30,533] jepsen worker 40 - jepsen.util 1390 :invoke :read [13 nil] INFO [2022-01-10 05:40:30,544] jepsen worker 50 - jepsen.util 1760 :ok :read [16 38] INFO [2022-01-10 05:40:30,544] jepsen worker 50 - jepsen.util 1760 :invoke :write [16 36] INFO [2022-01-10 05:40:30,552] jepsen worker 83 - jepsen.util 1883 :ok :read [27 30] INFO [2022-01-10 05:40:30,553] jepsen worker 83 - jepsen.util 1883 :invoke :write [27 47] INFO [2022-01-10 05:40:30,626] jepsen worker 19 - jepsen.util 1729 :info :write [6 31] :RequestTimeoutException INFO [2022-01-10 05:40:30,627] jepsen worker 19 - jepsen.util 1819 :invoke :read [6 nil] INFO [2022-01-10 05:40:30,662] jepsen worker 79 - jepsen.util 1609 :info :write [26 27] :RequestTimeoutException INFO [2022-01-10 05:40:30,662] jepsen worker 79 - jepsen.util 1699 :invoke :read [26 nil] INFO [2022-01-10 05:40:30,692] jepsen worker 79 - jepsen.util 1699 :ok :read [26 10] INFO [2022-01-10 05:40:30,692] jepsen worker 79 - jepsen.util 1699 :invoke :read [26 nil] INFO [2022-01-10 05:40:30,721] jepsen worker 79 - jepsen.util 1699 :ok :read [26 10] INFO [2022-01-10 05:40:30,722] jepsen worker 79 - jepsen.util 1699 :invoke :read [26 nil] INFO [2022-01-10 05:40:30,748] jepsen worker 79 - jepsen.util 1699 :ok :read [26 10] INFO [2022-01-10 05:40:30,749] jepsen worker 79 - jepsen.util 1699 :invoke :read [26 nil] INFO [2022-01-10 05:40:30,761] jepsen worker 18 - jepsen.util 1368 :ok :read [6 25] INFO [2022-01-10 05:40:30,762] jepsen worker 18 - jepsen.util 1368 :invoke :read [6 nil] INFO [2022-01-10 05:40:30,812] jepsen worker 79 - jepsen.util 1699 :ok :read [26 10] INFO [2022-01-10 05:40:30,813] jepsen worker 79 - jepsen.util 1699 :invoke :read [26 nil] INFO [2022-01-10 05:40:30,831] jepsen worker 57 - jepsen.util 1677 :ok :read [19 31] INFO [2022-01-10 05:40:30,832] jepsen worker 57 - jepsen.util 1677 :invoke :read [19 nil] INFO [2022-01-10 05:40:30,850] jepsen worker 79 - jepsen.util 1699 :ok :read [26 10] INFO [2022-01-10 05:40:30,851] jepsen worker 79 - jepsen.util 1699 :invoke :write [26 48] INFO [2022-01-10 05:40:30,868] jepsen worker 0 - jepsen.util 1530 :ok :read [0 46] INFO [2022-01-10 05:40:30,870] jepsen worker 0 - jepsen.util 1530 :invoke :read [0 nil] INFO [2022-01-10 05:40:30,878] jepsen worker 40 - jepsen.util 1390 :ok :read [13 15] INFO [2022-01-10 05:40:30,879] jepsen worker 40 - jepsen.util 1390 :invoke :write [13 40] INFO [2022-01-10 05:40:30,957] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.2923976608187134}, :ns_1@172.23.107.227 {:progress 0.1935483870967742}, :ns_1@172.23.105.187 {:progress 0.1935483870967742}, :ns_1@172.23.107.228 {:progress 0.288235294117647}} INFO [2022-01-10 05:40:30,958] jepsen worker nemesis - couchbase.util Rebalance check count: 186 INFO [2022-01-10 05:40:30,958] jepsen worker nemesis - couchbase.util Rebalance stuck count: 135 INFO [2022-01-10 05:40:31,216] jepsen worker 15 - jepsen.util 1275 :info :write [5 46] :RequestTimeoutException INFO [2022-01-10 05:40:31,217] jepsen worker 15 - jepsen.util 1365 :invoke :read [5 nil] WARN [2022-01-10 05:40:31,323] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17471501,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17945383,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9095043,"encode_duration_us":18876,"operation_id":"0x3066","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10008657}],"total_count":1}} WARN [2022-01-10 05:40:31,325] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9150777,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10196228,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9413644,"encode_duration_us":18489,"operation_id":"0x310f","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10004552},{"operation_name":"upsert","last_dispatch_duration_us":9118711,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10172083,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9507365,"encode_duration_us":21997,"operation_id":"0x302d","timeout_ms":10000,"last_server_duration_us":9043735,"total_duration_us":10007157},{"operation_name":"upsert","last_dispatch_duration_us":9436273,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10519854,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9219492,"encode_duration_us":23517,"operation_id":"0x3046","timeout_ms":10000,"last_server_duration_us":9086245,"total_duration_us":10004752},{"operation_name":"upsert","last_dispatch_duration_us":10328773,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14948906,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10790621,"encode_duration_us":17881,"operation_id":"0x318b","timeout_ms":10000,"last_server_duration_us":9376474,"total_duration_us":10010765}],"total_count":4}} WARN [2022-01-10 05:40:31,325] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10328773,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":14948906,"total_server_duration_us":10790621,"encode_duration_us":17881,"operation_id":"0x318b","timeout_ms":10000,"last_server_duration_us":9376474,"total_duration_us":10010765},{"operation_name":"upsert","last_dispatch_duration_us":7661840,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":13358562,"total_server_duration_us":1314521,"encode_duration_us":34316,"operation_id":"0x31ef","timeout_ms":10000,"last_server_duration_us":101853,"total_duration_us":10006454},{"operation_name":"upsert","last_dispatch_duration_us":1484480,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":1484480,"total_server_duration_us":492519,"encode_duration_us":16730,"operation_id":"0x3294","timeout_ms":10000,"last_server_duration_us":492519,"total_duration_us":10006424},{"operation_name":"upsert","last_dispatch_duration_us":8626667,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":14174372,"total_server_duration_us":1689603,"encode_duration_us":18274,"operation_id":"0x313c","timeout_ms":10000,"last_server_duration_us":123371,"total_duration_us":10004681},{"operation_name":"upsert","last_dispatch_duration_us":100888,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9170680,"total_server_duration_us":300568,"encode_duration_us":19275,"operation_id":"0x321d","timeout_ms":10000,"last_server_duration_us":67485,"total_duration_us":10004561},{"operation_name":"upsert","last_dispatch_duration_us":9150777,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":10196228,"total_server_duration_us":9413644,"encode_duration_us":18489,"operation_id":"0x310f","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10004552},{"operation_name":"upsert","last_dispatch_duration_us":253177,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":2217964,"total_server_duration_us":373085,"encode_duration_us":17463,"operation_id":"0x32ce","timeout_ms":10000,"last_server_duration_us":158588,"total_duration_us":10004389},{"operation_name":"upsert","last_dispatch_duration_us":93515,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":264408,"total_server_duration_us":61836,"encode_duration_us":16322,"operation_id":"0x3258","timeout_ms":10000,"last_server_duration_us":31108,"total_duration_us":10003235},{"operation_name":"upsert","last_dispatch_duration_us":8486378,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":14734583,"total_server_duration_us":1463561,"encode_duration_us":27195,"operation_id":"0x311a","timeout_ms":10000,"last_server_duration_us":108076,"total_duration_us":10002443},{"operation_name":"upsert","last_dispatch_duration_us":332065,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9428234,"total_server_duration_us":325760,"encode_duration_us":16615,"operation_id":"0x322e","timeout_ms":10000,"last_server_duration_us":73684,"total_duration_us":10000991}],"total_count":18}} INFO [2022-01-10 05:40:31,359] jepsen worker 57 - jepsen.util 1677 :ok :read [19 31] INFO [2022-01-10 05:40:31,359] jepsen worker 57 - jepsen.util 1677 :invoke :write [19 4] WARN [2022-01-10 05:40:31,423] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10382148,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10382148,"total_server_duration_us":48794,"encode_duration_us":16993,"operation_id":"0x31e3","timeout_ms":10000,"last_server_duration_us":48794,"total_duration_us":10009317},{"operation_name":"upsert","last_dispatch_duration_us":90078,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":222921,"total_server_duration_us":40435,"encode_duration_us":17243,"operation_id":"0x3261","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10008066},{"operation_name":"upsert","last_dispatch_duration_us":90590,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":90590,"total_server_duration_us":19262,"encode_duration_us":17742,"operation_id":"0x32d1","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10006981},{"operation_name":"upsert","last_dispatch_duration_us":113203,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":388050,"total_server_duration_us":85311,"encode_duration_us":16940,"operation_id":"0x32c8","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10006560},{"operation_name":"upsert","last_dispatch_duration_us":3041675,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":6947373,"total_server_duration_us":258305,"encode_duration_us":17013,"operation_id":"0x32fa","timeout_ms":10000,"last_server_duration_us":53254,"total_duration_us":10006294},{"operation_name":"upsert","last_dispatch_duration_us":1656158,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8848062,"total_server_duration_us":650760,"encode_duration_us":17110,"operation_id":"0x32bd","timeout_ms":10000,"last_server_duration_us":53254,"total_duration_us":10005617},{"operation_name":"upsert","last_dispatch_duration_us":1218199,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10287107,"total_server_duration_us":849948,"encode_duration_us":16143,"operation_id":"0x329a","timeout_ms":10000,"last_server_duration_us":22223,"total_duration_us":10004593},{"operation_name":"upsert","last_dispatch_duration_us":54873,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9788504,"total_server_duration_us":228139,"encode_duration_us":19171,"operation_id":"0x3229","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10003221},{"operation_name":"upsert","last_dispatch_duration_us":2658858,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10687032,"total_server_duration_us":1450059,"encode_duration_us":20808,"operation_id":"0x3254","timeout_ms":10000,"last_server_duration_us":54335,"total_duration_us":10002206},{"operation_name":"upsert","last_dispatch_duration_us":8919805,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8919805,"total_server_duration_us":41230,"encode_duration_us":21334,"operation_id":"0x31f6","timeout_ms":10000,"last_server_duration_us":41230,"total_duration_us":10001774}],"total_count":17}} WARN [2022-01-10 05:40:31,425] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13710383,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13710383,"total_server_duration_us":11677,"operation_id":"0x309e","timeout_ms":30000,"last_server_duration_us":11677,"total_duration_us":13712306},{"operation_name":"upsert","last_dispatch_duration_us":44074,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":1035892,"total_server_duration_us":631614,"encode_duration_us":16449,"operation_id":"0x32d2","timeout_ms":10000,"last_server_duration_us":24325,"total_duration_us":10008795},{"operation_name":"upsert","last_dispatch_duration_us":33403,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":1043649,"total_server_duration_us":644527,"encode_duration_us":16417,"operation_id":"0x328b","timeout_ms":10000,"last_server_duration_us":21731,"total_duration_us":10007743},{"operation_name":"upsert","last_dispatch_duration_us":11221147,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11221147,"total_server_duration_us":28127,"encode_duration_us":17663,"operation_id":"0x311b","timeout_ms":10000,"last_server_duration_us":28127,"total_duration_us":10005232},{"operation_name":"upsert","last_dispatch_duration_us":54984,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9777907,"total_server_duration_us":203125,"encode_duration_us":16261,"operation_id":"0x31e7","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10002114},{"operation_name":"upsert","last_dispatch_duration_us":8973132,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8973132,"total_server_duration_us":21487,"encode_duration_us":23112,"operation_id":"0x3165","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10001411},{"operation_name":"get","last_dispatch_duration_us":9233712,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9233712,"total_server_duration_us":38,"operation_id":"0x31e1","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":9235546},{"operation_name":"get","last_dispatch_duration_us":9118260,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9118260,"total_server_duration_us":43,"operation_id":"0x31e6","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":9120247},{"operation_name":"get","last_dispatch_duration_us":8217296,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8217296,"total_server_duration_us":11241,"operation_id":"0x31ac","timeout_ms":30000,"last_server_duration_us":11241,"total_duration_us":8219272},{"operation_name":"get","last_dispatch_duration_us":4968375,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":4968375,"total_server_duration_us":8015,"operation_id":"0x3256","timeout_ms":30000,"last_server_duration_us":8015,"total_duration_us":4970721}],"total_count":15}} WARN [2022-01-10 05:40:31,427] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":612846,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9958551,"total_server_duration_us":545816,"encode_duration_us":16778,"operation_id":"0x3112","timeout_ms":10000,"last_server_duration_us":469051,"total_duration_us":10009293},{"operation_name":"upsert","last_dispatch_duration_us":624957,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10064927,"total_server_duration_us":105465,"encode_duration_us":17914,"operation_id":"0x3117","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10008811},{"operation_name":"upsert","last_dispatch_duration_us":5769098,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12925964,"total_server_duration_us":194573,"encode_duration_us":16098,"operation_id":"0x31bf","timeout_ms":10000,"last_server_duration_us":21001,"total_duration_us":10008469},{"operation_name":"upsert","last_dispatch_duration_us":5770082,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6493126,"total_server_duration_us":88334,"encode_duration_us":16288,"operation_id":"0x3292","timeout_ms":10000,"last_server_duration_us":49255,"total_duration_us":10007445},{"operation_name":"upsert","last_dispatch_duration_us":169196,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8604224,"total_server_duration_us":116523,"encode_duration_us":16278,"operation_id":"0x316c","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10007238},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":16905,"operation_id":"0x321c","timeout_ms":10000,"total_duration_us":10007212},{"operation_name":"upsert","last_dispatch_duration_us":134440,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9410725,"total_server_duration_us":80115,"encode_duration_us":19979,"operation_id":"0x3119","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10005645},{"operation_name":"upsert","last_dispatch_duration_us":2210545,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":2210545,"total_server_duration_us":93043,"encode_duration_us":16983,"operation_id":"0x313b","timeout_ms":10000,"last_server_duration_us":93043,"total_duration_us":10005466},{"operation_name":"upsert","last_dispatch_duration_us":30929,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":618739,"total_server_duration_us":204985,"encode_duration_us":20240,"operation_id":"0x31e4","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10003676},{"operation_name":"upsert","last_dispatch_duration_us":1021506,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10583272,"total_server_duration_us":1018466,"encode_duration_us":16706,"operation_id":"0x32c9","timeout_ms":10000,"last_server_duration_us":163180,"total_duration_us":10003402}],"total_count":18}} WARN [2022-01-10 05:40:31,517] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14295864,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16477404,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":10397076,"encode_duration_us":22653,"operation_id":"0x3118","timeout_ms":10000,"last_server_duration_us":10073889,"total_duration_us":10001425},{"operation_name":"upsert","last_dispatch_duration_us":9129027,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10002410,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":9259242,"encode_duration_us":18709,"operation_id":"0x31f4","timeout_ms":10000,"last_server_duration_us":9058604,"total_duration_us":10004614}],"total_count":2}} WARN [2022-01-10 05:40:31,525] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16187767,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16187767,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9023573,"encode_duration_us":16641,"operation_id":"0x3091","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10008004},{"operation_name":"upsert","last_dispatch_duration_us":18007944,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19389716,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9416091,"encode_duration_us":22979,"operation_id":"0x2f65","timeout_ms":10000,"last_server_duration_us":9062854,"total_duration_us":10008744}],"total_count":2}} WARN [2022-01-10 05:40:31,618] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":17594,"operation_id":"0x3287","timeout_ms":10000,"total_duration_us":10009436},{"operation_name":"upsert","last_dispatch_duration_us":8186431,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8186431,"total_server_duration_us":28768,"encode_duration_us":16462,"operation_id":"0x3255","timeout_ms":10000,"last_server_duration_us":28768,"total_duration_us":10006295},{"operation_name":"upsert","last_dispatch_duration_us":47601,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":306174,"total_server_duration_us":145126,"encode_duration_us":25417,"operation_id":"0x322b","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10005848},{"operation_name":"upsert","last_dispatch_duration_us":10692196,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":16049986,"total_server_duration_us":1223571,"encode_duration_us":17993,"operation_id":"0x3143","timeout_ms":10000,"last_server_duration_us":34512,"total_duration_us":10005502},{"operation_name":"upsert","last_dispatch_duration_us":57729,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9931190,"total_server_duration_us":331652,"encode_duration_us":17454,"operation_id":"0x3197","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10004885},{"operation_name":"upsert","last_dispatch_duration_us":9129027,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10002410,"total_server_duration_us":9259242,"encode_duration_us":18709,"operation_id":"0x31f4","timeout_ms":10000,"last_server_duration_us":9058604,"total_duration_us":10004614},{"operation_name":"upsert","last_dispatch_duration_us":10362612,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":14453753,"total_server_duration_us":106780,"encode_duration_us":19119,"operation_id":"0x319f","timeout_ms":10000,"last_server_duration_us":43727,"total_duration_us":10003893},{"operation_name":"upsert","last_dispatch_duration_us":8412920,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8412920,"total_server_duration_us":23983,"encode_duration_us":18619,"operation_id":"0x323d","timeout_ms":10000,"last_server_duration_us":23983,"total_duration_us":10003222},{"operation_name":"upsert","last_dispatch_duration_us":14295864,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":16477404,"total_server_duration_us":10397076,"encode_duration_us":22653,"operation_id":"0x3118","timeout_ms":10000,"last_server_duration_us":10073889,"total_duration_us":10001425},{"operation_name":"upsert","last_dispatch_duration_us":9319274,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9937869,"total_server_duration_us":9634542,"encode_duration_us":23955,"operation_id":"0x31f3","timeout_ms":10000,"last_server_duration_us":9047983,"total_duration_us":9949750}],"total_count":17}} INFO [2022-01-10 05:40:31,633] jepsen worker 0 - jepsen.util 1530 :ok :read [0 46] INFO [2022-01-10 05:40:31,634] jepsen worker 0 - jepsen.util 1530 :invoke :write [0 40] INFO [2022-01-10 05:40:31,706] jepsen worker 73 - jepsen.util 1603 :info :write [24 6] :RequestTimeoutException INFO [2022-01-10 05:40:31,706] jepsen worker 73 - jepsen.util 1693 :invoke :write [24 9] INFO [2022-01-10 05:40:31,716] jepsen worker 17 - jepsen.util 1457 :info :write [5 0] :RequestTimeoutException INFO [2022-01-10 05:40:31,716] jepsen worker 64 - jepsen.util 1684 :info :write [21 43] :RequestTimeoutException INFO [2022-01-10 05:40:31,718] jepsen worker 64 - jepsen.util 1774 :invoke :write [21 7] INFO [2022-01-10 05:40:31,718] jepsen worker 17 - jepsen.util 1547 :invoke :read [5 nil] INFO [2022-01-10 05:40:31,840] jepsen worker 2 - jepsen.util 1622 :ok :read [0 46] INFO [2022-01-10 05:40:31,841] jepsen worker 2 - jepsen.util 1622 :invoke :read [0 nil] INFO [2022-01-10 05:40:31,874] jepsen worker 48 - jepsen.util 1308 :ok :read [16 38] INFO [2022-01-10 05:40:31,874] jepsen worker 48 - jepsen.util 1308 :invoke :read [16 nil] INFO [2022-01-10 05:40:31,895] jepsen worker 68 - jepsen.util 1598 :ok :read [22 41] INFO [2022-01-10 05:40:31,896] jepsen worker 68 - jepsen.util 1598 :invoke :write [22 17] INFO [2022-01-10 05:40:31,964] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.304093567251462}, :ns_1@172.23.107.227 {:progress 0.2023460410557185}, :ns_1@172.23.105.187 {:progress 0.2023460410557185}, :ns_1@172.23.107.228 {:progress 0.2941176470588235}} INFO [2022-01-10 05:40:31,964] jepsen worker nemesis - couchbase.util Rebalance check count: 187 INFO [2022-01-10 05:40:31,964] jepsen worker nemesis - couchbase.util Rebalance stuck count: 135 INFO [2022-01-10 05:40:31,971] jepsen worker 58 - jepsen.util 1588 :info :write [19 38] :RequestTimeoutException INFO [2022-01-10 05:40:31,971] jepsen worker 58 - jepsen.util 1678 :invoke :write [19 2] INFO [2022-01-10 05:40:31,975] jepsen worker 16 - jepsen.util 1906 :info :write [5 32] :RequestTimeoutException INFO [2022-01-10 05:40:31,976] jepsen worker 16 - jepsen.util 1996 :invoke :read [5 nil] INFO [2022-01-10 05:40:31,995] jepsen worker 49 - jepsen.util 1399 :ok :read [16 38] INFO [2022-01-10 05:40:31,996] jepsen worker 49 - jepsen.util 1399 :invoke :read [16 nil] INFO [2022-01-10 05:40:32,027] jepsen worker 23 - jepsen.util 1643 :info :write [7 26] :RequestTimeoutException INFO [2022-01-10 05:40:32,027] jepsen worker 23 - jepsen.util 1733 :invoke :write [7 16] INFO [2022-01-10 05:40:32,062] jepsen worker 17 - jepsen.util 1547 :ok :read [5 42] INFO [2022-01-10 05:40:32,063] jepsen worker 17 - jepsen.util 1547 :invoke :write [5 26] INFO [2022-01-10 05:40:32,085] jepsen worker 46 - jepsen.util 1756 :info :write [15 28] :SyncWriteAmbiguous INFO [2022-01-10 05:40:32,086] jepsen worker 46 - jepsen.util 1846 :invoke :write [15 48] INFO [2022-01-10 05:40:32,116] jepsen worker 6 - jepsen.util 1536 :info :write [2 19] :RequestTimeoutException INFO [2022-01-10 05:40:32,117] jepsen worker 6 - jepsen.util 1626 :invoke :write [2 22] INFO [2022-01-10 05:40:32,133] jepsen worker 72 - jepsen.util 1692 :info :write [24 1] :SyncWriteAmbiguous INFO [2022-01-10 05:40:32,134] jepsen worker 72 - jepsen.util 1782 :invoke :read [24 nil] INFO [2022-01-10 05:40:32,145] jepsen worker 18 - jepsen.util 1368 :ok :read [6 25] INFO [2022-01-10 05:40:32,146] jepsen worker 18 - jepsen.util 1368 :invoke :read [6 nil] INFO [2022-01-10 05:40:32,220] jepsen worker 81 - jepsen.util 1611 :ok :read [27 30] INFO [2022-01-10 05:40:32,220] jepsen worker 81 - jepsen.util 1611 :invoke :write [27 32] INFO [2022-01-10 05:40:32,222] jepsen worker 63 - jepsen.util 1503 :ok :read [21 7] INFO [2022-01-10 05:40:32,223] jepsen worker 63 - jepsen.util 1503 :invoke :read [21 nil] INFO [2022-01-10 05:40:32,252] jepsen worker 63 - jepsen.util 1503 :ok :read [21 7] INFO [2022-01-10 05:40:32,252] jepsen worker 63 - jepsen.util 1503 :invoke :read [21 nil] INFO [2022-01-10 05:40:32,302] jepsen worker 19 - jepsen.util 1819 :ok :read [6 25] INFO [2022-01-10 05:40:32,303] jepsen worker 19 - jepsen.util 1819 :invoke :write [6 4] INFO [2022-01-10 05:40:32,312] jepsen worker 63 - jepsen.util 1503 :ok :read [21 7] INFO [2022-01-10 05:40:32,313] jepsen worker 63 - jepsen.util 1503 :invoke :write [21 27] INFO [2022-01-10 05:40:32,350] jepsen worker 16 - jepsen.util 1996 :ok :read [5 42] INFO [2022-01-10 05:40:32,351] jepsen worker 16 - jepsen.util 1996 :invoke :read [5 nil] INFO [2022-01-10 05:40:32,432] jepsen worker 15 - jepsen.util 1365 :ok :read [5 42] INFO [2022-01-10 05:40:32,432] jepsen worker 15 - jepsen.util 1365 :invoke :read [5 nil] INFO [2022-01-10 05:40:32,458] jepsen worker 45 - jepsen.util 1755 :ok :read [15 23] INFO [2022-01-10 05:40:32,459] jepsen worker 45 - jepsen.util 1755 :invoke :read [15 nil] INFO [2022-01-10 05:40:32,500] jepsen worker 77 - jepsen.util 1967 :info :write [25 20] :RequestTimeoutException INFO [2022-01-10 05:40:32,502] jepsen worker 77 - jepsen.util 2057 :invoke :read [25 nil] INFO [2022-01-10 05:40:32,510] jepsen worker 2 - jepsen.util 1622 :ok :read [0 46] INFO [2022-01-10 05:40:32,510] jepsen worker 2 - jepsen.util 1622 :invoke :write [0 9] INFO [2022-01-10 05:40:32,720] jepsen worker 45 - jepsen.util 1755 :ok :read [15 23] INFO [2022-01-10 05:40:32,721] jepsen worker 45 - jepsen.util 1755 :invoke :read [15 nil] INFO [2022-01-10 05:40:32,809] jepsen worker 16 - jepsen.util 1996 :ok :read [5 42] INFO [2022-01-10 05:40:32,810] jepsen worker 16 - jepsen.util 1996 :invoke :read [5 nil] INFO [2022-01-10 05:40:32,878] jepsen worker 48 - jepsen.util 1308 :ok :read [16 38] INFO [2022-01-10 05:40:32,879] jepsen worker 48 - jepsen.util 1308 :invoke :write [16 29] INFO [2022-01-10 05:40:32,922] jepsen worker 65 - jepsen.util 1415 :info :write [21 47] :RequestTimeoutException INFO [2022-01-10 05:40:32,923] jepsen worker 65 - jepsen.util 1505 :invoke :write [21 5] INFO [2022-01-10 05:40:32,968] jepsen worker 49 - jepsen.util 1399 :ok :read [16 38] INFO [2022-01-10 05:40:32,969] jepsen worker 49 - jepsen.util 1399 :invoke :read [16 nil] INFO [2022-01-10 05:40:32,973] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.304093567251462}, :ns_1@172.23.107.227 {:progress 0.2023460410557185}, :ns_1@172.23.105.187 {:progress 0.2023460410557185}, :ns_1@172.23.107.228 {:progress 0.2941176470588235}} INFO [2022-01-10 05:40:32,973] jepsen worker nemesis - couchbase.util Rebalance check count: 188 INFO [2022-01-10 05:40:32,973] jepsen worker nemesis - couchbase.util Rebalance stuck count: 136 INFO [2022-01-10 05:40:33,072] jepsen worker 61 - jepsen.util 1501 :info :write [20 26] :RequestTimeoutException INFO [2022-01-10 05:40:33,072] jepsen worker 61 - jepsen.util 1591 :invoke :read [20 nil] INFO [2022-01-10 05:40:33,082] jepsen worker 25 - jepsen.util 1555 :info :write [8 3] :RequestTimeoutException INFO [2022-01-10 05:40:33,083] jepsen worker 25 - jepsen.util 1645 :invoke :write [8 37] INFO [2022-01-10 05:40:33,113] jepsen worker 77 - jepsen.util 2057 :ok :read [25 :nil] INFO [2022-01-10 05:40:33,113] jepsen worker 77 - jepsen.util 2057 :invoke :write [25 34] INFO [2022-01-10 05:40:33,123] jepsen worker 42 - jepsen.util 1482 :info :write [14 15] :RequestTimeoutException INFO [2022-01-10 05:40:33,124] jepsen worker 42 - jepsen.util 1572 :invoke :write [14 25] INFO [2022-01-10 05:40:33,185] jepsen worker 87 - jepsen.util 1797 :ok :write [29 0] INFO [2022-01-10 05:40:33,186] jepsen worker 87 - jepsen.util 1797 :invoke :write [29 7] INFO [2022-01-10 05:40:33,200] jepsen worker 16 - jepsen.util 1996 :ok :read [5 42] INFO [2022-01-10 05:40:33,200] jepsen worker 16 - jepsen.util 1996 :invoke :write [5 31] INFO [2022-01-10 05:40:33,248] jepsen worker 88 - jepsen.util 1618 :ok :read [29 0] INFO [2022-01-10 05:40:33,248] jepsen worker 88 - jepsen.util 1618 :invoke :write [29 29] INFO [2022-01-10 05:40:33,405] jepsen worker 15 - jepsen.util 1365 :ok :read [5 42] INFO [2022-01-10 05:40:33,406] jepsen worker 15 - jepsen.util 1365 :invoke :write [5 16] INFO [2022-01-10 05:40:33,476] jepsen worker 20 - jepsen.util 1460 :info :write [6 15] :RequestTimeoutException INFO [2022-01-10 05:40:33,476] jepsen worker 20 - jepsen.util 1550 :invoke :write [6 39] INFO [2022-01-10 05:40:33,576] jepsen worker 39 - jepsen.util 1839 :info :write [13 29] :RequestTimeoutException INFO [2022-01-10 05:40:33,577] jepsen worker 39 - jepsen.util 1929 :invoke :read [13 nil] INFO [2022-01-10 05:40:33,595] jepsen worker 67 - jepsen.util 1507 :info :write [22 47] :RequestTimeoutException INFO [2022-01-10 05:40:33,596] jepsen worker 67 - jepsen.util 1597 :invoke :read [22 nil] INFO [2022-01-10 05:40:33,643] jepsen worker 45 - jepsen.util 1755 :ok :read [15 23] INFO [2022-01-10 05:40:33,644] jepsen worker 45 - jepsen.util 1755 :invoke :read [15 nil] INFO [2022-01-10 05:40:33,737] jepsen worker 22 - jepsen.util 1462 :info :write [7 27] :SyncWriteAmbiguous INFO [2022-01-10 05:40:33,738] jepsen worker 22 - jepsen.util 1552 :invoke :read [7 nil] INFO [2022-01-10 05:40:33,751] jepsen worker 35 - jepsen.util 1565 :info :write [11 13] :RequestTimeoutException INFO [2022-01-10 05:40:33,752] jepsen worker 35 - jepsen.util 1655 :invoke :read [11 nil] INFO [2022-01-10 05:40:33,892] jepsen worker 8 - jepsen.util 1628 :info :write [2 45] :RequestTimeoutException INFO [2022-01-10 05:40:33,893] jepsen worker 8 - jepsen.util 1718 :invoke :write [2 38] INFO [2022-01-10 05:40:33,936] jepsen worker 84 - jepsen.util 1704 :info :write [28 16] :RequestTimeoutException INFO [2022-01-10 05:40:33,937] jepsen worker 84 - jepsen.util 1794 :invoke :write [28 20] INFO [2022-01-10 05:40:33,992] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.304093567251462}, :ns_1@172.23.107.227 {:progress 0.2023460410557185}, :ns_1@172.23.105.187 {:progress 0.2023460410557185}, :ns_1@172.23.107.228 {:progress 0.2941176470588235}} INFO [2022-01-10 05:40:33,992] jepsen worker nemesis - couchbase.util Rebalance check count: 189 INFO [2022-01-10 05:40:33,992] jepsen worker nemesis - couchbase.util Rebalance stuck count: 137 INFO [2022-01-10 05:40:34,009] jepsen worker 43 - jepsen.util 1753 :info :write [14 19] :RequestTimeoutException INFO [2022-01-10 05:40:34,010] jepsen worker 43 - jepsen.util 1843 :invoke :read [14 nil] INFO [2022-01-10 05:40:34,050] jepsen worker 49 - jepsen.util 1399 :ok :read [16 38] INFO [2022-01-10 05:40:34,050] jepsen worker 49 - jepsen.util 1399 :invoke :write [16 21] INFO [2022-01-10 05:40:34,394] jepsen worker 41 - jepsen.util 1391 :info :write [13 17] :RequestTimeoutException INFO [2022-01-10 05:40:34,395] jepsen worker 41 - jepsen.util 1481 :invoke :write [13 2] INFO [2022-01-10 05:40:34,671] jepsen worker 69 - jepsen.util 1689 :ok :read [23 26] INFO [2022-01-10 05:40:34,671] jepsen worker 69 - jepsen.util 1689 :invoke :read [23 nil] INFO [2022-01-10 05:40:34,676] jepsen worker 31 - jepsen.util 1921 :ok :read [10 37] INFO [2022-01-10 05:40:34,677] jepsen worker 31 - jepsen.util 1921 :invoke :read [10 nil] INFO [2022-01-10 05:40:34,725] jepsen worker 71 - jepsen.util 1691 :ok :read [23 26] INFO [2022-01-10 05:40:34,727] jepsen worker 71 - jepsen.util 1691 :invoke :read [23 nil] INFO [2022-01-10 05:40:34,727] jepsen worker 9 - jepsen.util 1899 :ok :read [3 2] INFO [2022-01-10 05:40:34,728] jepsen worker 9 - jepsen.util 1899 :invoke :write [3 21] INFO [2022-01-10 05:40:34,734] jepsen worker 69 - jepsen.util 1689 :ok :read [23 26] INFO [2022-01-10 05:40:34,735] jepsen worker 69 - jepsen.util 1689 :invoke :read [23 nil] INFO [2022-01-10 05:40:34,738] jepsen worker 31 - jepsen.util 1921 :ok :read [10 37] INFO [2022-01-10 05:40:34,738] jepsen worker 31 - jepsen.util 1921 :invoke :write [10 42] INFO [2022-01-10 05:40:34,740] jepsen worker 67 - jepsen.util 1597 :ok :read [22 41] INFO [2022-01-10 05:40:34,749] jepsen worker 67 - jepsen.util 1597 :invoke :read [22 nil] INFO [2022-01-10 05:40:34,794] jepsen worker 71 - jepsen.util 1691 :ok :read [23 26] INFO [2022-01-10 05:40:34,794] jepsen worker 71 - jepsen.util 1691 :invoke :read [23 nil] INFO [2022-01-10 05:40:34,805] jepsen worker 69 - jepsen.util 1689 :ok :read [23 26] INFO [2022-01-10 05:40:34,806] jepsen worker 69 - jepsen.util 1689 :invoke :write [23 31] INFO [2022-01-10 05:40:34,998] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3099415204678363}, :ns_1@172.23.107.227 {:progress 0.2082111436950147}, :ns_1@172.23.105.187 {:progress 0.2082111436950147}, :ns_1@172.23.107.228 {:progress 0.3}} INFO [2022-01-10 05:40:34,998] jepsen worker nemesis - couchbase.util Rebalance check count: 190 INFO [2022-01-10 05:40:34,998] jepsen worker nemesis - couchbase.util Rebalance stuck count: 137 INFO [2022-01-10 05:40:35,020] jepsen worker 60 - jepsen.util 2040 :ok :read [20 :nil] INFO [2022-01-10 05:40:35,020] jepsen worker 60 - jepsen.util 2040 :invoke :write [20 10] INFO [2022-01-10 05:40:35,079] jepsen worker 33 - jepsen.util 1653 :info :write [11 31] :RequestTimeoutException INFO [2022-01-10 05:40:35,080] jepsen worker 33 - jepsen.util 1743 :invoke :write [11 17] INFO [2022-01-10 05:40:35,152] jepsen worker 5 - jepsen.util 1535 :info :write [1 15] :RequestTimeoutException INFO [2022-01-10 05:40:35,153] jepsen worker 5 - jepsen.util 1625 :invoke :write [1 27] INFO [2022-01-10 05:40:35,201] jepsen worker 74 - jepsen.util 2054 :info :write [24 26] :RequestTimeoutException INFO [2022-01-10 05:40:35,201] jepsen worker 74 - jepsen.util 2144 :invoke :write [24 1] INFO [2022-01-10 05:40:35,243] jepsen worker 54 - jepsen.util 1764 :info :write [18 8] :SyncWriteAmbiguous INFO [2022-01-10 05:40:35,244] jepsen worker 54 - jepsen.util 1854 :invoke :read [18 nil] INFO [2022-01-10 05:40:35,295] jepsen worker 3 - jepsen.util 1713 :info :write [1 7] :SyncWriteAmbiguous INFO [2022-01-10 05:40:35,297] jepsen worker 3 - jepsen.util 1803 :invoke :read [1 nil] INFO [2022-01-10 05:40:35,320] jepsen worker 3 - jepsen.util 1803 :ok :read [1 41] INFO [2022-01-10 05:40:35,321] jepsen worker 3 - jepsen.util 1803 :invoke :read [1 nil] INFO [2022-01-10 05:40:35,348] jepsen worker 3 - jepsen.util 1803 :ok :read [1 41] INFO [2022-01-10 05:40:35,349] jepsen worker 3 - jepsen.util 1803 :invoke :write [1 16] INFO [2022-01-10 05:40:35,639] jepsen worker 61 - jepsen.util 1591 :ok :read [20 :nil] INFO [2022-01-10 05:40:35,640] jepsen worker 61 - jepsen.util 1591 :invoke :write [20 25] INFO [2022-01-10 05:40:35,676] jepsen worker 30 - jepsen.util 1290 :ok :read [10 37] INFO [2022-01-10 05:40:35,680] jepsen worker 30 - jepsen.util 1290 :invoke :write [10 43] INFO [2022-01-10 05:40:35,682] jepsen worker 56 - jepsen.util 1766 :ok :read [18 14] INFO [2022-01-10 05:40:35,683] jepsen worker 56 - jepsen.util 1766 :invoke :read [18 nil] INFO [2022-01-10 05:40:35,710] jepsen worker 56 - jepsen.util 1766 :ok :read [18 14] INFO [2022-01-10 05:40:35,711] jepsen worker 56 - jepsen.util 1766 :invoke :write [18 9] INFO [2022-01-10 05:40:35,748] jepsen worker 39 - jepsen.util 1929 :ok :read [13 15] INFO [2022-01-10 05:40:35,748] jepsen worker 39 - jepsen.util 1929 :invoke :write [13 10] INFO [2022-01-10 05:40:35,864] jepsen worker 67 - jepsen.util 1597 :ok :read [22 41] INFO [2022-01-10 05:40:35,864] jepsen worker 67 - jepsen.util 1597 :invoke :write [22 37] INFO [2022-01-10 05:40:35,920] jepsen worker 36 - jepsen.util 1296 :info :write [12 34] :RequestTimeoutException INFO [2022-01-10 05:40:35,920] jepsen worker 36 - jepsen.util 1386 :invoke :write [12 45] INFO [2022-01-10 05:40:36,005] jepsen worker 70 - jepsen.util 1960 :info :write [23 17] :SyncWriteAmbiguous INFO [2022-01-10 05:40:36,006] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3099415204678363}, :ns_1@172.23.107.227 {:progress 0.2082111436950147}, :ns_1@172.23.105.187 {:progress 0.2082111436950147}, :ns_1@172.23.107.228 {:progress 0.3}} INFO [2022-01-10 05:40:36,006] jepsen worker 70 - jepsen.util 2050 :invoke :write [23 20] INFO [2022-01-10 05:40:36,006] jepsen worker nemesis - couchbase.util Rebalance check count: 191 INFO [2022-01-10 05:40:36,006] jepsen worker nemesis - couchbase.util Rebalance stuck count: 138 INFO [2022-01-10 05:40:36,112] jepsen worker 11 - jepsen.util 1541 :info :write [3 21] :RequestTimeoutException INFO [2022-01-10 05:40:36,113] jepsen worker 11 - jepsen.util 1631 :invoke :read [3 nil] INFO [2022-01-10 05:40:36,155] jepsen worker 45 - jepsen.util 1755 :ok :read [15 23] INFO [2022-01-10 05:40:36,156] jepsen worker 45 - jepsen.util 1755 :invoke :write [15 30] INFO [2022-01-10 05:40:36,235] jepsen worker 29 - jepsen.util 1649 :info :write [9 22] :RequestTimeoutException INFO [2022-01-10 05:40:36,236] jepsen worker 29 - jepsen.util 1739 :invoke :read [9 nil] INFO [2022-01-10 05:40:36,280] jepsen worker 4 - jepsen.util 1534 :info :write [1 22] :RequestTimeoutException INFO [2022-01-10 05:40:36,281] jepsen worker 4 - jepsen.util 1624 :invoke :write [1 24] INFO [2022-01-10 05:40:36,366] jepsen worker 80 - jepsen.util 1700 :info :write [26 32] :RequestTimeoutException INFO [2022-01-10 05:40:36,367] jepsen worker 80 - jepsen.util 1790 :invoke :read [26 nil] INFO [2022-01-10 05:40:36,369] jepsen worker 12 - jepsen.util 1362 :info :write [4 33] :RequestTimeoutException INFO [2022-01-10 05:40:36,370] jepsen worker 12 - jepsen.util 1452 :invoke :read [4 nil] INFO [2022-01-10 05:40:36,431] jepsen worker 12 - jepsen.util 1452 :ok :read [4 3] INFO [2022-01-10 05:40:36,431] jepsen worker 53 - jepsen.util 1673 :info :write [17 4] :RequestTimeoutException INFO [2022-01-10 05:40:36,432] jepsen worker 53 - jepsen.util 1763 :invoke :read [17 nil] INFO [2022-01-10 05:40:36,432] jepsen worker 12 - jepsen.util 1452 :invoke :read [4 nil] INFO [2022-01-10 05:40:36,456] jepsen worker 12 - jepsen.util 1452 :ok :read [4 3] INFO [2022-01-10 05:40:36,457] jepsen worker 12 - jepsen.util 1452 :invoke :read [4 nil] INFO [2022-01-10 05:40:36,460] jepsen worker 53 - jepsen.util 1763 :ok :read [17 41] INFO [2022-01-10 05:40:36,461] jepsen worker 53 - jepsen.util 1763 :invoke :write [17 42] INFO [2022-01-10 05:40:36,479] jepsen worker 12 - jepsen.util 1452 :ok :read [4 3] INFO [2022-01-10 05:40:36,480] jepsen worker 12 - jepsen.util 1452 :invoke :write [4 15] INFO [2022-01-10 05:40:36,514] jepsen worker 76 - jepsen.util 1246 :info :write [25 46] :RequestTimeoutException INFO [2022-01-10 05:40:36,515] jepsen worker 76 - jepsen.util 1336 :invoke :write [25 9] INFO [2022-01-10 05:40:36,521] jepsen worker 51 - jepsen.util 1671 :info :write [17 44] :RequestTimeoutException INFO [2022-01-10 05:40:36,522] jepsen worker 51 - jepsen.util 1761 :invoke :read [17 nil] INFO [2022-01-10 05:40:36,554] jepsen worker 51 - jepsen.util 1761 :ok :read [17 41] INFO [2022-01-10 05:40:36,555] jepsen worker 51 - jepsen.util 1761 :invoke :read [17 nil] INFO [2022-01-10 05:40:36,587] jepsen worker 51 - jepsen.util 1761 :ok :read [17 41] INFO [2022-01-10 05:40:36,588] jepsen worker 51 - jepsen.util 1761 :invoke :read [17 nil] INFO [2022-01-10 05:40:36,619] jepsen worker 51 - jepsen.util 1761 :ok :read [17 41] INFO [2022-01-10 05:40:36,620] jepsen worker 51 - jepsen.util 1761 :invoke :write [17 19] INFO [2022-01-10 05:40:36,665] jepsen worker 89 - jepsen.util 1889 :info :write [29 23] :RequestTimeoutException INFO [2022-01-10 05:40:36,666] jepsen worker 89 - jepsen.util 1979 :invoke :read [29 nil] INFO [2022-01-10 05:40:36,892] jepsen worker 55 - jepsen.util 1855 :info :write [18 31] :RequestTimeoutException INFO [2022-01-10 05:40:36,893] jepsen worker 55 - jepsen.util 1945 :invoke :write [18 4] INFO [2022-01-10 05:40:36,896] jepsen worker 24 - jepsen.util 1554 :info :write [8 39] :RequestTimeoutException INFO [2022-01-10 05:40:36,897] jepsen worker 24 - jepsen.util 1644 :invoke :read [8 nil] INFO [2022-01-10 05:40:36,965] jepsen worker 21 - jepsen.util 1731 :info :write [7 26] :RequestTimeoutException INFO [2022-01-10 05:40:36,965] jepsen worker 21 - jepsen.util 1821 :invoke :write [7 20] INFO [2022-01-10 05:40:37,012] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3099415204678363}, :ns_1@172.23.107.227 {:progress 0.2082111436950147}, :ns_1@172.23.105.187 {:progress 0.2082111436950147}, :ns_1@172.23.107.228 {:progress 0.3}} INFO [2022-01-10 05:40:37,012] jepsen worker nemesis - couchbase.util Rebalance check count: 192 INFO [2022-01-10 05:40:37,012] jepsen worker nemesis - couchbase.util Rebalance stuck count: 139 INFO [2022-01-10 05:40:37,060] jepsen worker 22 - jepsen.util 1552 :ok :read [7 :nil] INFO [2022-01-10 05:40:37,061] jepsen worker 22 - jepsen.util 1552 :invoke :write [7 24] INFO [2022-01-10 05:40:37,245] jepsen worker 47 - jepsen.util 1487 :info :write [15 35] :RequestTimeoutException INFO [2022-01-10 05:40:37,245] jepsen worker 47 - jepsen.util 1577 :invoke :write [15 38] INFO [2022-01-10 05:40:37,370] jepsen worker 38 - jepsen.util 1658 :info :write [12 4] :RequestTimeoutException INFO [2022-01-10 05:40:37,371] jepsen worker 38 - jepsen.util 1748 :invoke :read [12 nil] INFO [2022-01-10 05:40:37,376] jepsen worker 10 - jepsen.util 1810 :info :write [3 27] :RequestTimeoutException INFO [2022-01-10 05:40:37,377] jepsen worker 10 - jepsen.util 1900 :invoke :read [3 nil] INFO [2022-01-10 05:40:37,395] jepsen worker 38 - jepsen.util 1748 :ok :read [12 1] INFO [2022-01-10 05:40:37,396] jepsen worker 38 - jepsen.util 1748 :invoke :write [12 23] INFO [2022-01-10 05:40:37,552] jepsen worker 13 - jepsen.util 1363 :info :write [4 32] :RequestTimeoutException INFO [2022-01-10 05:40:37,552] jepsen worker 13 - jepsen.util 1453 :invoke :write [4 39] INFO [2022-01-10 05:40:37,951] jepsen worker 85 - jepsen.util 1885 :info :write [28 15] :RequestTimeoutException INFO [2022-01-10 05:40:37,952] jepsen worker 85 - jepsen.util 1975 :invoke :write [28 48] INFO [2022-01-10 05:40:38,019] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3099415204678363}, :ns_1@172.23.107.227 {:progress 0.2082111436950147}, :ns_1@172.23.105.187 {:progress 0.2082111436950147}, :ns_1@172.23.107.228 {:progress 0.3}} INFO [2022-01-10 05:40:38,019] jepsen worker nemesis - couchbase.util Rebalance check count: 193 INFO [2022-01-10 05:40:38,019] jepsen worker nemesis - couchbase.util Rebalance stuck count: 140 INFO [2022-01-10 05:40:38,055] jepsen worker 32 - jepsen.util 1562 :info :write [10 37] :RequestTimeoutException INFO [2022-01-10 05:40:38,055] jepsen worker 32 - jepsen.util 1652 :invoke :write [10 34] INFO [2022-01-10 05:40:38,060] jepsen worker 44 - jepsen.util 1754 :ok :read [14 14] INFO [2022-01-10 05:40:38,061] jepsen worker 44 - jepsen.util 1754 :invoke :write [14 35] INFO [2022-01-10 05:40:38,125] jepsen worker 14 - jepsen.util 1724 :info :write [4 26] :RequestTimeoutException INFO [2022-01-10 05:40:38,127] jepsen worker 14 - jepsen.util 1814 :invoke :read [4 nil] INFO [2022-01-10 05:40:38,589] jepsen worker 24 - jepsen.util 1644 :ok :read [8 27] INFO [2022-01-10 05:40:38,589] jepsen worker 24 - jepsen.util 1644 :invoke :write [8 49] INFO [2022-01-10 05:40:38,816] jepsen worker 26 - jepsen.util 1376 :info :write [8 41] :RequestTimeoutException INFO [2022-01-10 05:40:38,817] jepsen worker 26 - jepsen.util 1466 :invoke :read [8 nil] INFO [2022-01-10 05:40:38,967] jepsen worker 37 - jepsen.util 1657 :info :write [12 6] :RequestTimeoutException INFO [2022-01-10 05:40:38,968] jepsen worker 37 - jepsen.util 1747 :invoke :write [12 22] INFO [2022-01-10 05:40:39,025] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3157894736842105}, :ns_1@172.23.107.227 {:progress 0.2140762463343109}, :ns_1@172.23.105.187 {:progress 0.2140762463343109}, :ns_1@172.23.107.228 {:progress 0.3058823529411765}} INFO [2022-01-10 05:40:39,025] jepsen worker nemesis - couchbase.util Rebalance check count: 194 INFO [2022-01-10 05:40:39,025] jepsen worker nemesis - couchbase.util Rebalance stuck count: 140 INFO [2022-01-10 05:40:39,226] jepsen worker 1 - jepsen.util 1711 :info :write [0 28] :RequestTimeoutException INFO [2022-01-10 05:40:39,227] jepsen worker 1 - jepsen.util 1801 :invoke :read [0 nil] INFO [2022-01-10 05:40:39,261] jepsen worker 78 - jepsen.util 1608 :info :write [26 26] :RequestTimeoutException INFO [2022-01-10 05:40:39,261] jepsen worker 78 - jepsen.util 1698 :invoke :write [26 39] INFO [2022-01-10 05:40:39,309] jepsen worker 66 - jepsen.util 1416 :info :write [22 25] :RequestTimeoutException INFO [2022-01-10 05:40:39,310] jepsen worker 66 - jepsen.util 1506 :invoke :write [22 46] INFO [2022-01-10 05:40:39,379] jepsen worker 59 - jepsen.util 1679 :info :write [19 40] :RequestTimeoutException INFO [2022-01-10 05:40:39,380] jepsen worker 59 - jepsen.util 1769 :invoke :read [19 nil] INFO [2022-01-10 05:40:39,517] jepsen worker 59 - jepsen.util 1769 :ok :read [19 31] INFO [2022-01-10 05:40:39,517] jepsen worker 59 - jepsen.util 1769 :invoke :read [19 nil] INFO [2022-01-10 05:40:39,628] jepsen worker 59 - jepsen.util 1769 :ok :read [19 31] INFO [2022-01-10 05:40:39,628] jepsen worker 59 - jepsen.util 1769 :invoke :read [19 nil] INFO [2022-01-10 05:40:39,662] jepsen worker 86 - jepsen.util 1346 :info :write [28 29] :RequestTimeoutException INFO [2022-01-10 05:40:39,662] jepsen worker 86 - jepsen.util 1436 :invoke :write [28 26] INFO [2022-01-10 05:40:39,666] jepsen worker 52 - jepsen.util 1852 :info :write [17 38] :RequestTimeoutException INFO [2022-01-10 05:40:39,667] jepsen worker 52 - jepsen.util 1942 :invoke :write [17 16] INFO [2022-01-10 05:40:39,711] jepsen worker 59 - jepsen.util 1769 :ok :read [19 31] INFO [2022-01-10 05:40:39,712] jepsen worker 59 - jepsen.util 1769 :invoke :write [19 1] WARN [2022-01-10 05:40:39,729] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9457602,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9457602,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9358154,"encode_duration_us":16825,"operation_id":"0x3266","timeout_ms":10000,"last_server_duration_us":9358154,"total_duration_us":10002941},{"operation_name":"upsert","last_dispatch_duration_us":11865443,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12001146,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9535256,"encode_duration_us":29997,"operation_id":"0x32bc","timeout_ms":10000,"last_server_duration_us":9507311,"total_duration_us":10007462}],"total_count":2}} WARN [2022-01-10 05:40:39,930] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":427623,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":7874224,"total_server_duration_us":2189938,"encode_duration_us":16375,"operation_id":"0x33ea","timeout_ms":10000,"last_server_duration_us":271897,"total_duration_us":10010776},{"operation_name":"upsert","last_dispatch_duration_us":606804,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":7683745,"total_server_duration_us":1222796,"encode_duration_us":17545,"operation_id":"0x3360","timeout_ms":10000,"last_server_duration_us":89883,"total_duration_us":10010412},{"operation_name":"upsert","last_dispatch_duration_us":52330,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":434738,"total_server_duration_us":162956,"encode_duration_us":20421,"operation_id":"0x33f8","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10009120},{"operation_name":"upsert","last_dispatch_duration_us":2459146,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2517779,"total_server_duration_us":43874,"encode_duration_us":18585,"operation_id":"0x3424","timeout_ms":10000,"last_server_duration_us":22306,"total_duration_us":10007480},{"operation_name":"upsert","last_dispatch_duration_us":11865443,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":12001146,"total_server_duration_us":9535256,"encode_duration_us":29997,"operation_id":"0x32bc","timeout_ms":10000,"last_server_duration_us":9507311,"total_duration_us":10007462},{"operation_name":"upsert","last_dispatch_duration_us":5206262,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":5206262,"total_server_duration_us":18799,"encode_duration_us":16585,"operation_id":"0x344d","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10006589},{"operation_name":"upsert","last_dispatch_duration_us":276796,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":5701248,"total_server_duration_us":1006804,"encode_duration_us":16337,"operation_id":"0x3325","timeout_ms":10000,"last_server_duration_us":113628,"total_duration_us":10006252},{"operation_name":"upsert","last_dispatch_duration_us":531376,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":7357446,"total_server_duration_us":1753147,"encode_duration_us":16392,"operation_id":"0x338c","timeout_ms":10000,"last_server_duration_us":96403,"total_duration_us":10004183},{"operation_name":"upsert","last_dispatch_duration_us":2083486,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2654423,"total_server_duration_us":434843,"encode_duration_us":18315,"operation_id":"0x33f5","timeout_ms":10000,"last_server_duration_us":255607,"total_duration_us":10003283},{"operation_name":"upsert","last_dispatch_duration_us":876441,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1134620,"total_server_duration_us":945866,"encode_duration_us":20650,"operation_id":"0x341d","timeout_ms":10000,"last_server_duration_us":843725,"total_duration_us":10002358}],"total_count":13}} INFO [2022-01-10 05:40:39,938] jepsen worker 34 - jepsen.util 1744 :ok :read [11 21] INFO [2022-01-10 05:40:39,939] jepsen worker 34 - jepsen.util 1744 :invoke :write [11 27] INFO [2022-01-10 05:40:39,972] jepsen worker 1 - jepsen.util 1801 :ok :read [0 46] INFO [2022-01-10 05:40:39,973] jepsen worker 1 - jepsen.util 1801 :invoke :write [0 6] INFO [2022-01-10 05:40:39,989] jepsen worker 7 - jepsen.util 1717 :info :write [2 39] :RequestTimeoutException INFO [2022-01-10 05:40:39,990] jepsen worker 7 - jepsen.util 1807 :invoke :read [2 nil] INFO [2022-01-10 05:40:39,992] jepsen worker 28 - jepsen.util 1558 :info :write [9 15] :RequestTimeoutException INFO [2022-01-10 05:40:39,992] jepsen worker 28 - jepsen.util 1648 :invoke :read [9 nil] INFO [2022-01-10 05:40:40,031] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3157894736842105}, :ns_1@172.23.107.227 {:progress 0.2140762463343109}, :ns_1@172.23.105.187 {:progress 0.2140762463343109}, :ns_1@172.23.107.228 {:progress 0.3058823529411765}} INFO [2022-01-10 05:40:40,031] jepsen worker nemesis - couchbase.util Rebalance check count: 195 INFO [2022-01-10 05:40:40,031] jepsen worker nemesis - couchbase.util Rebalance stuck count: 141 INFO [2022-01-10 05:40:40,186] jepsen worker 27 - jepsen.util 1557 :info :write [9 20] :RequestTimeoutException INFO [2022-01-10 05:40:40,186] jepsen worker 27 - jepsen.util 1647 :invoke :read [9 nil] INFO [2022-01-10 05:40:40,207] jepsen worker 83 - jepsen.util 1883 :info :write [27 47] :SyncWriteAmbiguous INFO [2022-01-10 05:40:40,208] jepsen worker 83 - jepsen.util 1973 :invoke :write [27 46] INFO [2022-01-10 05:40:40,336] jepsen worker 62 - jepsen.util 1862 :info :write [20 34] :RequestTimeoutException INFO [2022-01-10 05:40:40,337] jepsen worker 62 - jepsen.util 1952 :invoke :read [20 nil] INFO [2022-01-10 05:40:40,341] jepsen worker 82 - jepsen.util 1432 :info :write [27 9] :RequestTimeoutException INFO [2022-01-10 05:40:40,341] jepsen worker 82 - jepsen.util 1522 :invoke :write [27 36] INFO [2022-01-10 05:40:40,571] jepsen worker 50 - jepsen.util 1760 :info :write [16 36] :RequestTimeoutException INFO [2022-01-10 05:40:40,572] jepsen worker 50 - jepsen.util 1850 :invoke :write [16 36] INFO [2022-01-10 05:40:40,879] jepsen worker 79 - jepsen.util 1699 :info :write [26 48] :RequestTimeoutException INFO [2022-01-10 05:40:40,880] jepsen worker 79 - jepsen.util 1789 :invoke :read [26 nil] INFO [2022-01-10 05:40:40,904] jepsen worker 40 - jepsen.util 1390 :info :write [13 40] :RequestTimeoutException INFO [2022-01-10 05:40:40,905] jepsen worker 40 - jepsen.util 1480 :invoke :read [13 nil] INFO [2022-01-10 05:40:40,927] jepsen worker 79 - jepsen.util 1789 :ok :read [26 10] INFO [2022-01-10 05:40:40,927] jepsen worker 79 - jepsen.util 1789 :invoke :write [26 47] INFO [2022-01-10 05:40:41,045] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3157894736842105}, :ns_1@172.23.107.227 {:progress 0.2140762463343109}, :ns_1@172.23.105.187 {:progress 0.2140762463343109}, :ns_1@172.23.107.228 {:progress 0.3058823529411765}} INFO [2022-01-10 05:40:41,045] jepsen worker nemesis - couchbase.util Rebalance check count: 196 INFO [2022-01-10 05:40:41,045] jepsen worker nemesis - couchbase.util Rebalance stuck count: 142 INFO [2022-01-10 05:40:41,122] jepsen worker 27 - jepsen.util 1647 :ok :read [9 22] INFO [2022-01-10 05:40:41,122] jepsen worker 27 - jepsen.util 1647 :invoke :read [9 nil] INFO [2022-01-10 05:40:41,244] jepsen worker 28 - jepsen.util 1648 :ok :read [9 22] INFO [2022-01-10 05:40:41,244] jepsen worker 28 - jepsen.util 1648 :invoke :read [9 nil] WARN [2022-01-10 05:40:41,339] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12021674,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17326298,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9210207,"encode_duration_us":17809,"operation_id":"0x336e","timeout_ms":10000,"last_server_duration_us":9138434,"total_duration_us":10010286}],"total_count":1}} WARN [2022-01-10 05:40:41,340] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":46557,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":1754504,"total_server_duration_us":632954,"encode_duration_us":19157,"operation_id":"0x33f3","timeout_ms":10000,"last_server_duration_us":22720,"total_duration_us":10011064},{"operation_name":"upsert","last_dispatch_duration_us":12021674,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":17326298,"total_server_duration_us":9210207,"encode_duration_us":17809,"operation_id":"0x336e","timeout_ms":10000,"last_server_duration_us":9138434,"total_duration_us":10010286},{"operation_name":"upsert","last_dispatch_duration_us":904606,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":2378103,"total_server_duration_us":407765,"encode_duration_us":17131,"operation_id":"0x34c0","timeout_ms":10000,"last_server_duration_us":123884,"total_duration_us":10009421},{"operation_name":"upsert","last_dispatch_duration_us":655626,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":3182199,"total_server_duration_us":572908,"encode_duration_us":16232,"operation_id":"0x34bc","timeout_ms":10000,"last_server_duration_us":25712,"total_duration_us":10009011},{"operation_name":"upsert","last_dispatch_duration_us":9253480,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":16108373,"total_server_duration_us":384285,"encode_duration_us":17582,"operation_id":"0x33b7","timeout_ms":10000,"last_server_duration_us":65123,"total_duration_us":10005754},{"operation_name":"upsert","last_dispatch_duration_us":1061516,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":2171938,"total_server_duration_us":383278,"encode_duration_us":18284,"operation_id":"0x344c","timeout_ms":10000,"last_server_duration_us":119635,"total_duration_us":10005694},{"operation_name":"upsert","last_dispatch_duration_us":38785,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":1709305,"total_server_duration_us":652783,"encode_duration_us":17120,"operation_id":"0x33f6","timeout_ms":10000,"last_server_duration_us":23475,"total_duration_us":10005589},{"operation_name":"upsert","last_dispatch_duration_us":32591,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":1065227,"total_server_duration_us":642141,"encode_duration_us":17884,"operation_id":"0x3489","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10004379},{"operation_name":"upsert","last_dispatch_duration_us":384923,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":990251,"total_server_duration_us":138719,"encode_duration_us":19519,"operation_id":"0x3324","timeout_ms":10000,"last_server_duration_us":67353,"total_duration_us":10003635},{"operation_name":"upsert","last_dispatch_duration_us":9630632,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9630632,"total_server_duration_us":9152302,"encode_duration_us":16851,"operation_id":"0x34c1","timeout_ms":10000,"last_server_duration_us":9152302,"total_duration_us":9635659}],"total_count":14}} INFO [2022-01-10 05:40:41,381] jepsen worker 57 - jepsen.util 1677 :info :write [19 4] :RequestTimeoutException INFO [2022-01-10 05:40:41,382] jepsen worker 57 - jepsen.util 1767 :invoke :write [19 42] WARN [2022-01-10 05:40:41,440] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10925396,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10925396,"total_server_duration_us":8339,"operation_id":"0x320d","timeout_ms":30000,"last_server_duration_us":8339,"total_duration_us":15075660},{"operation_name":"get","last_dispatch_duration_us":11022790,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11022790,"total_server_duration_us":38,"operation_id":"0x3230","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":14149802},{"operation_name":"upsert","last_dispatch_duration_us":1667235,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10904800,"total_server_duration_us":323895,"encode_duration_us":16664,"operation_id":"0x3355","timeout_ms":10000,"last_server_duration_us":245901,"total_duration_us":10009805},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":20734,"operation_id":"0x341a","timeout_ms":10000,"total_duration_us":10007979},{"operation_name":"upsert","last_dispatch_duration_us":431364,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8957278,"total_server_duration_us":1352114,"encode_duration_us":17014,"operation_id":"0x34b2","timeout_ms":10000,"last_server_duration_us":43508,"total_duration_us":10007294},{"operation_name":"upsert","last_dispatch_duration_us":3241483,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12924564,"total_server_duration_us":604758,"encode_duration_us":22637,"operation_id":"0x3381","timeout_ms":10000,"last_server_duration_us":218840,"total_duration_us":10007282},{"operation_name":"upsert","last_dispatch_duration_us":8565871,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9117491,"total_server_duration_us":289827,"encode_duration_us":19307,"operation_id":"0x33e3","timeout_ms":10000,"last_server_duration_us":21568,"total_duration_us":10006216},{"operation_name":"upsert","last_dispatch_duration_us":35923,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":1123952,"total_server_duration_us":647459,"encode_duration_us":32827,"operation_id":"0x342c","timeout_ms":10000,"last_server_duration_us":22471,"total_duration_us":10005959},{"operation_name":"upsert","last_dispatch_duration_us":1960718,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11263992,"total_server_duration_us":406536,"encode_duration_us":30516,"operation_id":"0x3356","timeout_ms":10000,"last_server_duration_us":335504,"total_duration_us":10005627},{"operation_name":"upsert","last_dispatch_duration_us":34966,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":939287,"total_server_duration_us":243673,"encode_duration_us":16624,"operation_id":"0x33b6","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10004716}],"total_count":29}} WARN [2022-01-10 05:40:41,442] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2013690,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11033039,"total_server_duration_us":328847,"encode_duration_us":16731,"operation_id":"0x3383","timeout_ms":10000,"last_server_duration_us":82259,"total_duration_us":10011106},{"operation_name":"upsert","last_dispatch_duration_us":10451829,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10451829,"total_server_duration_us":23054,"encode_duration_us":17969,"operation_id":"0x3308","timeout_ms":10000,"last_server_duration_us":23054,"total_duration_us":10009482},{"operation_name":"upsert","last_dispatch_duration_us":51651,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":791502,"total_server_duration_us":460429,"encode_duration_us":16514,"operation_id":"0x344e","timeout_ms":10000,"last_server_duration_us":29415,"total_duration_us":10005952},{"operation_name":"upsert","last_dispatch_duration_us":36258,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":1338870,"total_server_duration_us":664080,"encode_duration_us":26676,"operation_id":"0x33ee","timeout_ms":10000,"last_server_duration_us":25275,"total_duration_us":10005720},{"operation_name":"upsert","last_dispatch_duration_us":895903,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10039671,"total_server_duration_us":784334,"encode_duration_us":16684,"operation_id":"0x34b9","timeout_ms":10000,"last_server_duration_us":60128,"total_duration_us":10005455},{"operation_name":"upsert","last_dispatch_duration_us":822416,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8495095,"total_server_duration_us":502113,"encode_duration_us":16908,"operation_id":"0x335f","timeout_ms":10000,"last_server_duration_us":60759,"total_duration_us":10003852},{"operation_name":"upsert","last_dispatch_duration_us":31500,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":1597172,"total_server_duration_us":544387,"encode_duration_us":18016,"operation_id":"0x342b","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10002795},{"operation_name":"get","last_dispatch_duration_us":107116,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":107116,"total_server_duration_us":13355,"operation_id":"0x33f7","timeout_ms":30000,"last_server_duration_us":13355,"total_duration_us":5791605},{"operation_name":"get","last_dispatch_duration_us":2495826,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2495826,"total_server_duration_us":143322,"operation_id":"0x3565","timeout_ms":30000,"last_server_duration_us":143322,"total_duration_us":2497665},{"operation_name":"get","last_dispatch_duration_us":148063,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":148063,"total_server_duration_us":31012,"operation_id":"0x34af","timeout_ms":30000,"last_server_duration_us":31012,"total_duration_us":2255190}],"total_count":16}} WARN [2022-01-10 05:40:41,532] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10111280,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10584141,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9389359,"encode_duration_us":16334,"operation_id":"0x3385","timeout_ms":10000,"last_server_duration_us":9275365,"total_duration_us":10001024},{"operation_name":"upsert","last_dispatch_duration_us":17384906,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17691081,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9169760,"encode_duration_us":25417,"operation_id":"0x322b","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10005848}],"total_count":2}} WARN [2022-01-10 05:40:41,540] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":20066601,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25275627,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10854210,"encode_duration_us":18244,"operation_id":"0x2f87","timeout_ms":10000,"last_server_duration_us":9197165,"total_duration_us":10002474}],"total_count":1}} WARN [2022-01-10 05:40:41,549] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16127372,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":16127372,"total_server_duration_us":32,"operation_id":"0x3264","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":16128584},{"operation_name":"upsert","last_dispatch_duration_us":286781,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":977943,"total_server_duration_us":181066,"encode_duration_us":18428,"operation_id":"0x3300","timeout_ms":10000,"last_server_duration_us":22141,"total_duration_us":10009802},{"operation_name":"upsert","last_dispatch_duration_us":10207979,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10865837,"total_server_duration_us":247801,"encode_duration_us":18429,"operation_id":"0x335e","timeout_ms":10000,"last_server_duration_us":182305,"total_duration_us":10008805},{"operation_name":"upsert","last_dispatch_duration_us":32958,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":549361,"total_server_duration_us":357929,"encode_duration_us":17329,"operation_id":"0x348e","timeout_ms":10000,"last_server_duration_us":21001,"total_duration_us":10008626},{"operation_name":"upsert","last_dispatch_duration_us":9798637,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":17120040,"total_server_duration_us":517785,"encode_duration_us":16811,"operation_id":"0x3328","timeout_ms":10000,"last_server_duration_us":257941,"total_duration_us":10005583},{"operation_name":"upsert","last_dispatch_duration_us":99866,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":5892210,"total_server_duration_us":74878,"encode_duration_us":19774,"operation_id":"0x347a","timeout_ms":10000,"last_server_duration_us":23475,"total_duration_us":10004915},{"operation_name":"upsert","last_dispatch_duration_us":308361,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":7665533,"total_server_duration_us":1107195,"encode_duration_us":17197,"operation_id":"0x3487","timeout_ms":10000,"last_server_duration_us":75616,"total_duration_us":10002532},{"operation_name":"upsert","last_dispatch_duration_us":10393396,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10393396,"total_server_duration_us":184327,"encode_duration_us":16599,"operation_id":"0x34bd","timeout_ms":10000,"last_server_duration_us":184327,"total_duration_us":10002258},{"operation_name":"upsert","last_dispatch_duration_us":220959,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1117393,"total_server_duration_us":134621,"encode_duration_us":17164,"operation_id":"0x3478","timeout_ms":10000,"last_server_duration_us":32840,"total_duration_us":10002001},{"operation_name":"upsert","last_dispatch_duration_us":9979962,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9979962,"total_server_duration_us":9559427,"encode_duration_us":17980,"operation_id":"0x341b","timeout_ms":10000,"last_server_duration_us":9559427,"total_duration_us":10001907}],"total_count":15}} WARN [2022-01-10 05:40:41,549] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9979962,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9979962,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9559427,"encode_duration_us":17980,"operation_id":"0x341b","timeout_ms":10000,"last_server_duration_us":9559427,"total_duration_us":10001907},{"operation_name":"upsert","last_dispatch_duration_us":17725398,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17725398,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9038428,"encode_duration_us":16905,"operation_id":"0x321c","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":10007212},{"operation_name":"upsert","last_dispatch_duration_us":17721567,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18340306,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9247659,"encode_duration_us":20240,"operation_id":"0x31e4","timeout_ms":10000,"last_server_duration_us":9042674,"total_duration_us":10003676}],"total_count":3}} INFO [2022-01-10 05:40:41,661] jepsen worker 0 - jepsen.util 1530 :info :write [0 40] :RequestTimeoutException INFO [2022-01-10 05:40:41,662] jepsen worker 0 - jepsen.util 1620 :invoke :write [0 11] WARN [2022-01-10 05:40:41,733] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":34666,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":34666,"total_server_duration_us":38,"operation_id":"0x32fb","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":14381637},{"operation_name":"upsert","last_dispatch_duration_us":36500,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":1125910,"total_server_duration_us":604742,"encode_duration_us":18144,"operation_id":"0x34e8","timeout_ms":10000,"last_server_duration_us":23222,"total_duration_us":10009556},{"operation_name":"upsert","last_dispatch_duration_us":1666702,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8667114,"total_server_duration_us":976740,"encode_duration_us":20105,"operation_id":"0x348a","timeout_ms":10000,"last_server_duration_us":227859,"total_duration_us":10007866},{"operation_name":"upsert","last_dispatch_duration_us":10763974,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":13215284,"total_server_duration_us":759370,"encode_duration_us":16483,"operation_id":"0x3330","timeout_ms":10000,"last_server_duration_us":181901,"total_duration_us":10007275},{"operation_name":"upsert","last_dispatch_duration_us":12506794,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12506794,"total_server_duration_us":310797,"encode_duration_us":18619,"operation_id":"0x33ae","timeout_ms":10000,"last_server_duration_us":310797,"total_duration_us":10007231},{"operation_name":"upsert","last_dispatch_duration_us":658929,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10219012,"total_server_duration_us":841181,"encode_duration_us":16411,"operation_id":"0x34b1","timeout_ms":10000,"last_server_duration_us":189425,"total_duration_us":10006279},{"operation_name":"upsert","last_dispatch_duration_us":34143,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9106219,"total_server_duration_us":41445,"encode_duration_us":16588,"operation_id":"0x33df","timeout_ms":10000,"last_server_duration_us":21163,"total_duration_us":10006059},{"operation_name":"upsert","last_dispatch_duration_us":1273576,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9695744,"total_server_duration_us":1217311,"encode_duration_us":19625,"operation_id":"0x348b","timeout_ms":10000,"last_server_duration_us":61392,"total_duration_us":10003681},{"operation_name":"upsert","last_dispatch_duration_us":37222,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":37222,"total_server_duration_us":22141,"encode_duration_us":16684,"operation_id":"0x342a","timeout_ms":10000,"last_server_duration_us":22141,"total_duration_us":10002549},{"operation_name":"upsert","last_dispatch_duration_us":33078,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":33078,"total_server_duration_us":19185,"encode_duration_us":16710,"operation_id":"0x33eb","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10002547}],"total_count":19}} INFO [2022-01-10 05:40:41,736] jepsen worker 73 - jepsen.util 1693 :info :write [24 9] :RequestTimeoutException INFO [2022-01-10 05:40:41,736] jepsen worker 73 - jepsen.util 1783 :invoke :read [24 nil] INFO [2022-01-10 05:40:41,741] jepsen worker 64 - jepsen.util 1774 :info :write [21 7] :RequestTimeoutException INFO [2022-01-10 05:40:41,741] jepsen worker 64 - jepsen.util 1864 :invoke :read [21 nil] INFO [2022-01-10 05:40:41,922] jepsen worker 68 - jepsen.util 1598 :info :write [22 17] :RequestTimeoutException INFO [2022-01-10 05:40:41,923] jepsen worker 68 - jepsen.util 1688 :invoke :read [22 nil] INFO [2022-01-10 05:40:41,996] jepsen worker 58 - jepsen.util 1678 :info :write [19 2] :RequestTimeoutException INFO [2022-01-10 05:40:41,997] jepsen worker 58 - jepsen.util 1768 :invoke :write [19 43] INFO [2022-01-10 05:40:42,051] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3157894736842105}, :ns_1@172.23.107.227 {:progress 0.2140762463343109}, :ns_1@172.23.105.187 {:progress 0.2140762463343109}, :ns_1@172.23.107.228 {:progress 0.3058823529411765}} INFO [2022-01-10 05:40:42,051] jepsen worker nemesis - couchbase.util Rebalance check count: 197 INFO [2022-01-10 05:40:42,051] jepsen worker nemesis - couchbase.util Rebalance stuck count: 143 INFO [2022-01-10 05:40:42,052] jepsen worker 23 - jepsen.util 1733 :info :write [7 16] :RequestTimeoutException INFO [2022-01-10 05:40:42,053] jepsen worker 23 - jepsen.util 1823 :invoke :write [7 38] INFO [2022-01-10 05:40:42,085] jepsen worker 17 - jepsen.util 1547 :info :write [5 26] :RequestTimeoutException INFO [2022-01-10 05:40:42,085] jepsen worker 17 - jepsen.util 1637 :invoke :read [5 nil] INFO [2022-01-10 05:40:42,116] jepsen worker 46 - jepsen.util 1846 :info :write [15 48] :RequestTimeoutException INFO [2022-01-10 05:40:42,116] jepsen worker 46 - jepsen.util 1936 :invoke :write [15 45] INFO [2022-01-10 05:40:42,141] jepsen worker 6 - jepsen.util 1626 :info :write [2 22] :RequestTimeoutException INFO [2022-01-10 05:40:42,142] jepsen worker 6 - jepsen.util 1716 :invoke :write [2 13] INFO [2022-01-10 05:40:42,170] jepsen worker 28 - jepsen.util 1648 :ok :read [9 22] INFO [2022-01-10 05:40:42,170] jepsen worker 28 - jepsen.util 1648 :invoke :write [9 48] INFO [2022-01-10 05:40:42,245] jepsen worker 81 - jepsen.util 1611 :info :write [27 32] :RequestTimeoutException INFO [2022-01-10 05:40:42,246] jepsen worker 81 - jepsen.util 1701 :invoke :write [27 21] INFO [2022-01-10 05:40:42,326] jepsen worker 19 - jepsen.util 1819 :info :write [6 4] :RequestTimeoutException INFO [2022-01-10 05:40:42,327] jepsen worker 19 - jepsen.util 1909 :invoke :write [6 13] INFO [2022-01-10 05:40:42,336] jepsen worker 63 - jepsen.util 1503 :info :write [21 27] :RequestTimeoutException INFO [2022-01-10 05:40:42,337] jepsen worker 63 - jepsen.util 1593 :invoke :write [21 36] INFO [2022-01-10 05:40:42,454] jepsen worker 14 - jepsen.util 1814 :ok :read [4 3] INFO [2022-01-10 05:40:42,455] jepsen worker 14 - jepsen.util 1814 :invoke :read [4 nil] INFO [2022-01-10 05:40:42,491] jepsen worker 14 - jepsen.util 1814 :ok :read [4 3] INFO [2022-01-10 05:40:42,492] jepsen worker 14 - jepsen.util 1814 :invoke :read [4 nil] INFO [2022-01-10 05:40:42,516] jepsen worker 14 - jepsen.util 1814 :ok :read [4 3] INFO [2022-01-10 05:40:42,517] jepsen worker 14 - jepsen.util 1814 :invoke :read [4 nil] INFO [2022-01-10 05:40:42,526] jepsen worker 80 - jepsen.util 1790 :ok :read [26 10] INFO [2022-01-10 05:40:42,527] jepsen worker 80 - jepsen.util 1790 :invoke :write [26 21] INFO [2022-01-10 05:40:42,532] jepsen worker 2 - jepsen.util 1622 :info :write [0 9] :RequestTimeoutException INFO [2022-01-10 05:40:42,532] jepsen worker 2 - jepsen.util 1712 :invoke :write [0 48] INFO [2022-01-10 05:40:42,538] jepsen worker 14 - jepsen.util 1814 :ok :read [4 3] INFO [2022-01-10 05:40:42,539] jepsen worker 14 - jepsen.util 1814 :invoke :read [4 nil] INFO [2022-01-10 05:40:42,566] jepsen worker 14 - jepsen.util 1814 :ok :read [4 3] INFO [2022-01-10 05:40:42,568] jepsen worker 14 - jepsen.util 1814 :invoke :read [4 nil] INFO [2022-01-10 05:40:42,577] jepsen worker 72 - jepsen.util 1782 :ok :read [24 18] INFO [2022-01-10 05:40:42,578] jepsen worker 72 - jepsen.util 1782 :invoke :read [24 nil] INFO [2022-01-10 05:40:42,594] jepsen worker 14 - jepsen.util 1814 :ok :read [4 3] INFO [2022-01-10 05:40:42,595] jepsen worker 14 - jepsen.util 1814 :invoke :write [4 0] INFO [2022-01-10 05:40:42,644] jepsen worker 62 - jepsen.util 1952 :ok :read [20 :nil] INFO [2022-01-10 05:40:42,645] jepsen worker 62 - jepsen.util 1952 :invoke :write [20 1] INFO [2022-01-10 05:40:42,654] jepsen worker 26 - jepsen.util 1466 :ok :read [8 27] INFO [2022-01-10 05:40:42,654] jepsen worker 26 - jepsen.util 1466 :invoke :write [8 11] INFO [2022-01-10 05:40:42,661] jepsen worker 18 - jepsen.util 1368 :ok :read [6 25] INFO [2022-01-10 05:40:42,661] jepsen worker 18 - jepsen.util 1368 :invoke :read [6 nil] INFO [2022-01-10 05:40:42,813] jepsen worker 18 - jepsen.util 1368 :ok :read [6 25] INFO [2022-01-10 05:40:42,814] jepsen worker 18 - jepsen.util 1368 :invoke :read [6 nil] INFO [2022-01-10 05:40:42,902] jepsen worker 48 - jepsen.util 1308 :info :write [16 29] :RequestTimeoutException INFO [2022-01-10 05:40:42,903] jepsen worker 48 - jepsen.util 1398 :invoke :write [16 27] INFO [2022-01-10 05:40:42,949] jepsen worker 65 - jepsen.util 1505 :info :write [21 5] :RequestTimeoutException INFO [2022-01-10 05:40:42,950] jepsen worker 65 - jepsen.util 1595 :invoke :write [21 8] INFO [2022-01-10 05:40:43,058] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3216374269005848}, :ns_1@172.23.107.227 {:progress 0.217008797653959}, :ns_1@172.23.105.187 {:progress 0.217008797653959}, :ns_1@172.23.107.228 {:progress 0.3117647058823529}} INFO [2022-01-10 05:40:43,058] jepsen worker nemesis - couchbase.util Rebalance check count: 198 INFO [2022-01-10 05:40:43,064] jepsen worker nemesis - couchbase.util Rebalance stuck count: 143 INFO [2022-01-10 05:40:43,106] jepsen worker 25 - jepsen.util 1645 :info :write [8 37] :RequestTimeoutException INFO [2022-01-10 05:40:43,107] jepsen worker 25 - jepsen.util 1735 :invoke :write [8 30] INFO [2022-01-10 05:40:43,135] jepsen worker 77 - jepsen.util 2057 :info :write [25 34] :RequestTimeoutException INFO [2022-01-10 05:40:43,136] jepsen worker 77 - jepsen.util 2147 :invoke :read [25 nil] INFO [2022-01-10 05:40:43,151] jepsen worker 42 - jepsen.util 1572 :info :write [14 25] :RequestTimeoutException INFO [2022-01-10 05:40:43,152] jepsen worker 42 - jepsen.util 1662 :invoke :write [14 18] INFO [2022-01-10 05:40:43,205] jepsen worker 87 - jepsen.util 1797 :info :write [29 7] :RequestTimeoutException INFO [2022-01-10 05:40:43,206] jepsen worker 87 - jepsen.util 1887 :invoke :read [29 nil] INFO [2022-01-10 05:40:43,220] jepsen worker 16 - jepsen.util 1996 :info :write [5 31] :RequestTimeoutException INFO [2022-01-10 05:40:43,221] jepsen worker 16 - jepsen.util 2086 :invoke :write [5 28] INFO [2022-01-10 05:40:43,235] jepsen worker 87 - jepsen.util 1887 :ok :read [29 0] INFO [2022-01-10 05:40:43,235] jepsen worker 87 - jepsen.util 1887 :invoke :read [29 nil] INFO [2022-01-10 05:40:43,276] jepsen worker 88 - jepsen.util 1618 :info :write [29 29] :RequestTimeoutException INFO [2022-01-10 05:40:43,276] jepsen worker 88 - jepsen.util 1708 :invoke :write [29 21] INFO [2022-01-10 05:40:43,280] jepsen worker 87 - jepsen.util 1887 :ok :read [29 0] INFO [2022-01-10 05:40:43,281] jepsen worker 87 - jepsen.util 1887 :invoke :write [29 42] INFO [2022-01-10 05:40:43,282] jepsen worker 68 - jepsen.util 1688 :ok :read [22 41] INFO [2022-01-10 05:40:43,282] jepsen worker 68 - jepsen.util 1688 :invoke :read [22 nil] INFO [2022-01-10 05:40:43,317] jepsen worker 29 - jepsen.util 1739 :ok :read [9 22] INFO [2022-01-10 05:40:43,318] jepsen worker 29 - jepsen.util 1739 :invoke :read [9 nil] INFO [2022-01-10 05:40:43,432] jepsen worker 15 - jepsen.util 1365 :info :write [5 16] :RequestTimeoutException INFO [2022-01-10 05:40:43,433] jepsen worker 15 - jepsen.util 1455 :invoke :read [5 nil] INFO [2022-01-10 05:40:43,486] jepsen worker 17 - jepsen.util 1637 :ok :read [5 42] INFO [2022-01-10 05:40:43,486] jepsen worker 17 - jepsen.util 1637 :invoke :write [5 41] INFO [2022-01-10 05:40:43,505] jepsen worker 20 - jepsen.util 1550 :info :write [6 39] :RequestTimeoutException INFO [2022-01-10 05:40:43,505] jepsen worker 20 - jepsen.util 1640 :invoke :read [6 nil] INFO [2022-01-10 05:40:43,828] jepsen worker 9 - jepsen.util 1899 :info :write [3 21] :SyncWriteAmbiguous INFO [2022-01-10 05:40:43,829] jepsen worker 9 - jepsen.util 1989 :invoke :read [3 nil] INFO [2022-01-10 05:40:43,850] jepsen worker 31 - jepsen.util 1921 :info :write [10 42] :SyncWriteAmbiguous INFO [2022-01-10 05:40:43,851] jepsen worker 31 - jepsen.util 2011 :invoke :write [10 35] INFO [2022-01-10 05:40:43,873] jepsen worker 71 - jepsen.util 1691 :ok :read [23 26] INFO [2022-01-10 05:40:43,873] jepsen worker 71 - jepsen.util 1691 :invoke :read [23 nil] INFO [2022-01-10 05:40:43,921] jepsen worker 8 - jepsen.util 1718 :info :write [2 38] :RequestTimeoutException INFO [2022-01-10 05:40:43,921] jepsen worker 8 - jepsen.util 1808 :invoke :write [2 3] INFO [2022-01-10 05:40:43,965] jepsen worker 84 - jepsen.util 1794 :info :write [28 20] :RequestTimeoutException INFO [2022-01-10 05:40:43,965] jepsen worker 84 - jepsen.util 1884 :invoke :read [28 nil] INFO [2022-01-10 05:40:44,024] jepsen worker 11 - jepsen.util 1631 :ok :read [3 2] INFO [2022-01-10 05:40:44,025] jepsen worker 11 - jepsen.util 1631 :invoke :write [3 33] INFO [2022-01-10 05:40:44,041] jepsen worker 29 - jepsen.util 1739 :ok :read [9 22] INFO [2022-01-10 05:40:44,042] jepsen worker 29 - jepsen.util 1739 :invoke :read [9 nil] INFO [2022-01-10 05:40:44,072] jepsen worker 49 - jepsen.util 1399 :info :write [16 21] :RequestTimeoutException INFO [2022-01-10 05:40:44,072] jepsen worker 49 - jepsen.util 1489 :invoke :read [16 nil] INFO [2022-01-10 05:40:44,072] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3216374269005848}, :ns_1@172.23.107.227 {:progress 0.217008797653959}, :ns_1@172.23.105.187 {:progress 0.217008797653959}, :ns_1@172.23.107.228 {:progress 0.3117647058823529}} INFO [2022-01-10 05:40:44,072] jepsen worker nemesis - couchbase.util Rebalance check count: 199 INFO [2022-01-10 05:40:44,073] jepsen worker nemesis - couchbase.util Rebalance stuck count: 144 INFO [2022-01-10 05:40:44,419] jepsen worker 41 - jepsen.util 1481 :info :write [13 2] :RequestTimeoutException INFO [2022-01-10 05:40:44,420] jepsen worker 41 - jepsen.util 1571 :invoke :read [13 nil] INFO [2022-01-10 05:40:44,587] jepsen worker 75 - jepsen.util 1335 :ok :read [25 :nil] INFO [2022-01-10 05:40:44,587] jepsen worker 75 - jepsen.util 1335 :invoke :read [25 nil] INFO [2022-01-10 05:40:44,689] jepsen worker 43 - jepsen.util 1843 :ok :read [14 14] INFO [2022-01-10 05:40:44,690] jepsen worker 43 - jepsen.util 1843 :invoke :read [14 nil] INFO [2022-01-10 05:40:44,761] jepsen worker 29 - jepsen.util 1739 :ok :read [9 22] INFO [2022-01-10 05:40:44,762] jepsen worker 29 - jepsen.util 1739 :invoke :read [9 nil] INFO [2022-01-10 05:40:44,836] jepsen worker 69 - jepsen.util 1689 :info :write [23 31] :RequestTimeoutException INFO [2022-01-10 05:40:44,837] jepsen worker 69 - jepsen.util 1779 :invoke :write [23 49] INFO [2022-01-10 05:40:44,954] jepsen worker 29 - jepsen.util 1739 :ok :read [9 22] INFO [2022-01-10 05:40:44,955] jepsen worker 29 - jepsen.util 1739 :invoke :read [9 nil] INFO [2022-01-10 05:40:45,042] jepsen worker 60 - jepsen.util 2040 :info :write [20 10] :RequestTimeoutException INFO [2022-01-10 05:40:45,042] jepsen worker 60 - jepsen.util 2130 :invoke :write [20 2] INFO [2022-01-10 05:40:45,081] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3216374269005848}, :ns_1@172.23.107.227 {:progress 0.2258064516129032}, :ns_1@172.23.105.187 {:progress 0.2258064516129032}, :ns_1@172.23.107.228 {:progress 0.3176470588235294}} INFO [2022-01-10 05:40:45,081] jepsen worker nemesis - couchbase.util Rebalance check count: 200 INFO [2022-01-10 05:40:45,081] jepsen worker nemesis - couchbase.util Rebalance stuck count: 144 INFO [2022-01-10 05:40:45,102] jepsen worker 33 - jepsen.util 1743 :info :write [11 17] :RequestTimeoutException INFO [2022-01-10 05:40:45,103] jepsen worker 33 - jepsen.util 1833 :invoke :read [11 nil] INFO [2022-01-10 05:40:45,156] jepsen worker 29 - jepsen.util 1739 :ok :read [9 22] INFO [2022-01-10 05:40:45,157] jepsen worker 29 - jepsen.util 1739 :invoke :read [9 nil] INFO [2022-01-10 05:40:45,176] jepsen worker 5 - jepsen.util 1625 :info :write [1 27] :RequestTimeoutException INFO [2022-01-10 05:40:45,177] jepsen worker 5 - jepsen.util 1715 :invoke :write [1 20] INFO [2022-01-10 05:40:45,221] jepsen worker 74 - jepsen.util 2144 :info :write [24 1] :RequestTimeoutException INFO [2022-01-10 05:40:45,221] jepsen worker 74 - jepsen.util 2234 :invoke :write [24 19] INFO [2022-01-10 05:40:45,354] jepsen worker 29 - jepsen.util 1739 :ok :read [9 22] INFO [2022-01-10 05:40:45,355] jepsen worker 29 - jepsen.util 1739 :invoke :write [9 0] INFO [2022-01-10 05:40:45,376] jepsen worker 3 - jepsen.util 1803 :info :write [1 16] :RequestTimeoutException INFO [2022-01-10 05:40:45,376] jepsen worker 3 - jepsen.util 1893 :invoke :read [1 nil] INFO [2022-01-10 05:40:45,465] jepsen worker 84 - jepsen.util 1884 :ok :read [28 13] INFO [2022-01-10 05:40:45,466] jepsen worker 84 - jepsen.util 1884 :invoke :read [28 nil] INFO [2022-01-10 05:40:45,543] jepsen worker 68 - jepsen.util 1688 :ok :read [22 41] INFO [2022-01-10 05:40:45,544] jepsen worker 68 - jepsen.util 1688 :invoke :write [22 11] INFO [2022-01-10 05:40:45,672] jepsen worker 61 - jepsen.util 1591 :info :write [20 25] :RequestTimeoutException INFO [2022-01-10 05:40:45,673] jepsen worker 61 - jepsen.util 1681 :invoke :write [20 34] INFO [2022-01-10 05:40:45,709] jepsen worker 30 - jepsen.util 1290 :info :write [10 43] :RequestTimeoutException INFO [2022-01-10 05:40:45,710] jepsen worker 30 - jepsen.util 1380 :invoke :write [10 29] INFO [2022-01-10 05:40:45,740] jepsen worker 56 - jepsen.util 1766 :info :write [18 9] :RequestTimeoutException INFO [2022-01-10 05:40:45,741] jepsen worker 56 - jepsen.util 1856 :invoke :write [18 32] INFO [2022-01-10 05:40:45,776] jepsen worker 39 - jepsen.util 1929 :info :write [13 10] :RequestTimeoutException INFO [2022-01-10 05:40:45,777] jepsen worker 39 - jepsen.util 2019 :invoke :write [13 21] INFO [2022-01-10 05:40:45,849] jepsen worker 73 - jepsen.util 1783 :ok :read [24 18] INFO [2022-01-10 05:40:45,851] jepsen worker 73 - jepsen.util 1783 :invoke :read [24 nil] INFO [2022-01-10 05:40:45,890] jepsen worker 67 - jepsen.util 1597 :info :write [22 37] :RequestTimeoutException INFO [2022-01-10 05:40:45,891] jepsen worker 67 - jepsen.util 1687 :invoke :read [22 nil] INFO [2022-01-10 05:40:45,950] jepsen worker 36 - jepsen.util 1386 :info :write [12 45] :RequestTimeoutException INFO [2022-01-10 05:40:45,950] jepsen worker 36 - jepsen.util 1476 :invoke :read [12 nil] INFO [2022-01-10 05:40:46,011] jepsen worker 72 - jepsen.util 1782 :ok :read [24 18] INFO [2022-01-10 05:40:46,012] jepsen worker 72 - jepsen.util 1782 :invoke :write [24 8] INFO [2022-01-10 05:40:46,031] jepsen worker 70 - jepsen.util 2050 :info :write [23 20] :RequestTimeoutException INFO [2022-01-10 05:40:46,032] jepsen worker 70 - jepsen.util 2140 :invoke :read [23 nil] INFO [2022-01-10 05:40:46,087] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.327485380116959}, :ns_1@172.23.107.227 {:progress 0.2287390029325513}, :ns_1@172.23.105.187 {:progress 0.2287390029325513}, :ns_1@172.23.107.228 {:progress 0.3235294117647058}} INFO [2022-01-10 05:40:46,087] jepsen worker nemesis - couchbase.util Rebalance check count: 201 INFO [2022-01-10 05:40:46,087] jepsen worker nemesis - couchbase.util Rebalance stuck count: 144 INFO [2022-01-10 05:40:46,169] jepsen worker 7 - jepsen.util 1807 :ok :read [2 14] INFO [2022-01-10 05:40:46,169] jepsen worker 7 - jepsen.util 1807 :invoke :write [2 43] INFO [2022-01-10 05:40:46,176] jepsen worker 45 - jepsen.util 1755 :info :write [15 30] :RequestTimeoutException INFO [2022-01-10 05:40:46,177] jepsen worker 45 - jepsen.util 1845 :invoke :write [15 20] INFO [2022-01-10 05:40:46,219] jepsen worker 15 - jepsen.util 1455 :ok :read [5 42] INFO [2022-01-10 05:40:46,220] jepsen worker 15 - jepsen.util 1455 :invoke :write [5 1] INFO [2022-01-10 05:40:46,276] jepsen worker 64 - jepsen.util 1864 :ok :read [21 7] INFO [2022-01-10 05:40:46,276] jepsen worker 64 - jepsen.util 1864 :invoke :write [21 11] INFO [2022-01-10 05:40:46,305] jepsen worker 4 - jepsen.util 1624 :info :write [1 24] :RequestTimeoutException INFO [2022-01-10 05:40:46,306] jepsen worker 4 - jepsen.util 1714 :invoke :write [1 40] INFO [2022-01-10 05:40:46,368] jepsen worker 49 - jepsen.util 1489 :ok :read [16 38] INFO [2022-01-10 05:40:46,369] jepsen worker 49 - jepsen.util 1489 :invoke :write [16 48] INFO [2022-01-10 05:40:46,457] jepsen worker 73 - jepsen.util 1783 :ok :read [24 18] INFO [2022-01-10 05:40:46,458] jepsen worker 73 - jepsen.util 1783 :invoke :write [24 9] INFO [2022-01-10 05:40:46,482] jepsen worker 53 - jepsen.util 1763 :info :write [17 42] :RequestTimeoutException INFO [2022-01-10 05:40:46,483] jepsen worker 53 - jepsen.util 1853 :invoke :read [17 nil] INFO [2022-01-10 05:40:46,499] jepsen worker 12 - jepsen.util 1452 :info :write [4 15] :RequestTimeoutException INFO [2022-01-10 05:40:46,500] jepsen worker 12 - jepsen.util 1542 :invoke :read [4 nil] INFO [2022-01-10 05:40:46,509] jepsen worker 53 - jepsen.util 1853 :ok :read [17 41] INFO [2022-01-10 05:40:46,510] jepsen worker 53 - jepsen.util 1853 :invoke :read [17 nil] INFO [2022-01-10 05:40:46,532] jepsen worker 53 - jepsen.util 1853 :ok :read [17 41] INFO [2022-01-10 05:40:46,532] jepsen worker 53 - jepsen.util 1853 :invoke :write [17 26] INFO [2022-01-10 05:40:46,536] jepsen worker 76 - jepsen.util 1336 :info :write [25 9] :RequestTimeoutException INFO [2022-01-10 05:40:46,537] jepsen worker 76 - jepsen.util 1426 :invoke :read [25 nil] INFO [2022-01-10 05:40:46,651] jepsen worker 51 - jepsen.util 1761 :info :write [17 19] :RequestTimeoutException INFO [2022-01-10 05:40:46,652] jepsen worker 51 - jepsen.util 1851 :invoke :read [17 nil] INFO [2022-01-10 05:40:46,916] jepsen worker 55 - jepsen.util 1945 :info :write [18 4] :RequestTimeoutException INFO [2022-01-10 05:40:46,916] jepsen worker 55 - jepsen.util 2035 :invoke :read [18 nil] INFO [2022-01-10 05:40:46,940] jepsen worker 84 - jepsen.util 1884 :ok :read [28 13] INFO [2022-01-10 05:40:46,940] jepsen worker 84 - jepsen.util 1884 :invoke :write [28 15] INFO [2022-01-10 05:40:46,992] jepsen worker 21 - jepsen.util 1821 :info :write [7 20] :RequestTimeoutException INFO [2022-01-10 05:40:46,992] jepsen worker 21 - jepsen.util 1911 :invoke :write [7 25] INFO [2022-01-10 05:40:47,071] jepsen worker 43 - jepsen.util 1843 :ok :read [14 14] INFO [2022-01-10 05:40:47,071] jepsen worker 43 - jepsen.util 1843 :invoke :read [14 nil] INFO [2022-01-10 05:40:47,082] jepsen worker 22 - jepsen.util 1552 :info :write [7 24] :RequestTimeoutException INFO [2022-01-10 05:40:47,083] jepsen worker 22 - jepsen.util 1642 :invoke :read [7 nil] INFO [2022-01-10 05:40:47,093] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.327485380116959}, :ns_1@172.23.107.227 {:progress 0.2287390029325513}, :ns_1@172.23.105.187 {:progress 0.2287390029325513}, :ns_1@172.23.107.228 {:progress 0.3235294117647058}} INFO [2022-01-10 05:40:47,093] jepsen worker nemesis - couchbase.util Rebalance check count: 202 INFO [2022-01-10 05:40:47,093] jepsen worker nemesis - couchbase.util Rebalance stuck count: 145 INFO [2022-01-10 05:40:47,167] jepsen worker 67 - jepsen.util 1687 :ok :read [22 41] INFO [2022-01-10 05:40:47,167] jepsen worker 67 - jepsen.util 1687 :invoke :write [22 47] INFO [2022-01-10 05:40:47,247] jepsen worker 75 - jepsen.util 1335 :ok :read [25 :nil] INFO [2022-01-10 05:40:47,247] jepsen worker 75 - jepsen.util 1335 :invoke :read [25 nil] INFO [2022-01-10 05:40:47,260] jepsen worker 33 - jepsen.util 1833 :ok :read [11 21] INFO [2022-01-10 05:40:47,262] jepsen worker 33 - jepsen.util 1833 :invoke :read [11 nil] INFO [2022-01-10 05:40:47,276] jepsen worker 47 - jepsen.util 1577 :info :write [15 38] :RequestTimeoutException INFO [2022-01-10 05:40:47,277] jepsen worker 47 - jepsen.util 1667 :invoke :read [15 nil] INFO [2022-01-10 05:40:47,405] jepsen worker 22 - jepsen.util 1642 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:40:47,406] jepsen worker 22 - jepsen.util 1642 :invoke :read [7 nil] INFO [2022-01-10 05:40:47,419] jepsen worker 43 - jepsen.util 1843 :ok :read [14 14] INFO [2022-01-10 05:40:47,419] jepsen worker 43 - jepsen.util 1843 :invoke :read [14 nil] INFO [2022-01-10 05:40:47,421] jepsen worker 38 - jepsen.util 1748 :info :write [12 23] :RequestTimeoutException INFO [2022-01-10 05:40:47,422] jepsen worker 38 - jepsen.util 1838 :invoke :write [12 42] INFO [2022-01-10 05:40:47,577] jepsen worker 13 - jepsen.util 1453 :info :write [4 39] :RequestTimeoutException INFO [2022-01-10 05:40:47,577] jepsen worker 13 - jepsen.util 1543 :invoke :write [4 32] INFO [2022-01-10 05:40:47,773] jepsen worker 22 - jepsen.util 1642 :ok :read [7 :nil] INFO [2022-01-10 05:40:47,774] jepsen worker 22 - jepsen.util 1642 :invoke :read [7 nil] INFO [2022-01-10 05:40:47,786] jepsen worker 43 - jepsen.util 1843 :ok :read [14 14] INFO [2022-01-10 05:40:47,787] jepsen worker 43 - jepsen.util 1843 :invoke :read [14 nil] INFO [2022-01-10 05:40:47,980] jepsen worker 85 - jepsen.util 1975 :info :write [28 48] :RequestTimeoutException INFO [2022-01-10 05:40:47,980] jepsen worker 85 - jepsen.util 2065 :invoke :read [28 nil] INFO [2022-01-10 05:40:48,080] jepsen worker 44 - jepsen.util 1754 :info :write [14 35] :RequestTimeoutException INFO [2022-01-10 05:40:48,080] jepsen worker 44 - jepsen.util 1844 :invoke :write [14 1] INFO [2022-01-10 05:40:48,081] jepsen worker 32 - jepsen.util 1652 :info :write [10 34] :RequestTimeoutException INFO [2022-01-10 05:40:48,100] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3333333333333334}, :ns_1@172.23.107.227 {:progress 0.2316715542521994}, :ns_1@172.23.105.187 {:progress 0.2316715542521994}, :ns_1@172.23.107.228 {:progress 0.3235294117647058}} INFO [2022-01-10 05:40:48,488] jepsen worker 32 - jepsen.util 1742 :invoke :write [10 23] INFO [2022-01-10 05:40:48,488] jepsen worker nemesis - couchbase.util Rebalance check count: 203 INFO [2022-01-10 05:40:48,488] jepsen worker nemesis - couchbase.util Rebalance stuck count: 145 INFO [2022-01-10 05:40:48,462] jepsen worker 22 - jepsen.util 1642 :ok :read [7 :nil] INFO [2022-01-10 05:40:48,477] jepsen worker 43 - jepsen.util 1843 :ok :read [14 14] INFO [2022-01-10 05:40:48,488] jepsen worker 22 - jepsen.util 1642 :invoke :read [7 nil] INFO [2022-01-10 05:40:48,488] jepsen worker 43 - jepsen.util 1843 :invoke :read [14 nil] INFO [2022-01-10 05:40:48,610] jepsen worker 24 - jepsen.util 1644 :info :write [8 49] :RequestTimeoutException INFO [2022-01-10 05:40:48,610] jepsen worker 24 - jepsen.util 1734 :invoke :write [8 6] INFO [2022-01-10 05:40:48,995] jepsen worker 37 - jepsen.util 1747 :info :write [12 22] :RequestTimeoutException INFO [2022-01-10 05:40:48,996] jepsen worker 37 - jepsen.util 1837 :invoke :read [12 nil] INFO [2022-01-10 05:40:49,147] jepsen worker 40 - jepsen.util 1480 :ok :read [13 15] INFO [2022-01-10 05:40:49,147] jepsen worker 40 - jepsen.util 1480 :invoke :read [13 nil] INFO [2022-01-10 05:40:49,157] jepsen worker 27 - jepsen.util 1647 :ok :read [9 22] INFO [2022-01-10 05:40:49,158] jepsen worker 27 - jepsen.util 1647 :invoke :write [9 30] INFO [2022-01-10 05:40:49,280] jepsen worker 78 - jepsen.util 1698 :info :write [26 39] :RequestTimeoutException INFO [2022-01-10 05:40:49,281] jepsen worker 78 - jepsen.util 1788 :invoke :write [26 37] INFO [2022-01-10 05:40:49,293] jepsen worker 41 - jepsen.util 1571 :ok :read [13 15] INFO [2022-01-10 05:40:49,293] jepsen worker 41 - jepsen.util 1571 :invoke :read [13 nil] INFO [2022-01-10 05:40:49,317] jepsen worker 40 - jepsen.util 1480 :ok :read [13 15] INFO [2022-01-10 05:40:49,317] jepsen worker 40 - jepsen.util 1480 :invoke :read [13 nil] INFO [2022-01-10 05:40:49,332] jepsen worker 66 - jepsen.util 1506 :info :write [22 46] :RequestTimeoutException INFO [2022-01-10 05:40:49,332] jepsen worker 66 - jepsen.util 1596 :invoke :read [22 nil] INFO [2022-01-10 05:40:49,494] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3333333333333334}, :ns_1@172.23.107.227 {:progress 0.2316715542521994}, :ns_1@172.23.105.187 {:progress 0.2316715542521994}, :ns_1@172.23.107.228 {:progress 0.3235294117647058}} INFO [2022-01-10 05:40:49,494] jepsen worker nemesis - couchbase.util Rebalance check count: 204 INFO [2022-01-10 05:40:49,494] jepsen worker nemesis - couchbase.util Rebalance stuck count: 146 INFO [2022-01-10 05:40:49,691] jepsen worker 86 - jepsen.util 1436 :info :write [28 26] :RequestTimeoutException INFO [2022-01-10 05:40:49,692] jepsen worker 86 - jepsen.util 1526 :invoke :write [28 15] INFO [2022-01-10 05:40:49,694] jepsen worker 52 - jepsen.util 1942 :info :write [17 16] :RequestTimeoutException INFO [2022-01-10 05:40:49,695] jepsen worker 52 - jepsen.util 2032 :invoke :write [17 33] INFO [2022-01-10 05:40:49,736] jepsen worker 59 - jepsen.util 1769 :info :write [19 1] :RequestTimeoutException INFO [2022-01-10 05:40:49,737] jepsen worker 59 - jepsen.util 1859 :invoke :write [19 32] WARN [2022-01-10 05:40:49,779] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9292831,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15663380,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":11053951,"encode_duration_us":20151,"operation_id":"0x34ea","timeout_ms":10000,"last_server_duration_us":9276438,"total_duration_us":10004530},{"operation_name":"upsert","last_dispatch_duration_us":15061001,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15495740,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9187590,"encode_duration_us":20421,"operation_id":"0x33f8","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10009120}],"total_count":2}} WARN [2022-01-10 05:40:49,879] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":2841924,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2841924,"total_server_duration_us":502415,"operation_id":"0x34ae","timeout_ms":30000,"last_server_duration_us":502415,"total_duration_us":14635911},{"operation_name":"get","last_dispatch_duration_us":10770755,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10770755,"total_server_duration_us":567783,"operation_id":"0x3482","timeout_ms":30000,"last_server_duration_us":567783,"total_duration_us":10772989},{"operation_name":"get","last_dispatch_duration_us":2695738,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2695738,"total_server_duration_us":423587,"operation_id":"0x3524","timeout_ms":30000,"last_server_duration_us":423587,"total_duration_us":10418878},{"operation_name":"upsert","last_dispatch_duration_us":2863349,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2863349,"total_server_duration_us":38066,"encode_duration_us":16504,"operation_id":"0x3562","timeout_ms":10000,"last_server_duration_us":38066,"total_duration_us":10011004},{"operation_name":"upsert","last_dispatch_duration_us":34526,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":1070254,"total_server_duration_us":624047,"encode_duration_us":16956,"operation_id":"0x3685","timeout_ms":10000,"last_server_duration_us":20680,"total_duration_us":10010140},{"operation_name":"upsert","last_dispatch_duration_us":8997226,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13690553,"total_server_duration_us":1817818,"encode_duration_us":17475,"operation_id":"0x3587","timeout_ms":10000,"last_server_duration_us":31012,"total_duration_us":10009832},{"operation_name":"upsert","last_dispatch_duration_us":9086480,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9086480,"total_server_duration_us":18876,"encode_duration_us":17242,"operation_id":"0x3560","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10008679},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":17538,"operation_id":"0x3678","timeout_ms":10000,"total_duration_us":10008475},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":18695,"operation_id":"0x35f5","timeout_ms":10000,"total_duration_us":10004712},{"operation_name":"upsert","last_dispatch_duration_us":9292831,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":15663380,"total_server_duration_us":11053951,"encode_duration_us":20151,"operation_id":"0x34ea","timeout_ms":10000,"last_server_duration_us":9276438,"total_duration_us":10004530}],"total_count":17}} INFO [2022-01-10 05:40:49,965] jepsen worker 34 - jepsen.util 1744 :info :write [11 27] :RequestTimeoutException INFO [2022-01-10 05:40:49,965] jepsen worker 34 - jepsen.util 1834 :invoke :write [11 40] INFO [2022-01-10 05:40:49,996] jepsen worker 1 - jepsen.util 1801 :info :write [0 6] :RequestTimeoutException INFO [2022-01-10 05:40:49,997] jepsen worker 1 - jepsen.util 1891 :invoke :read [0 nil] INFO [2022-01-10 05:40:50,236] jepsen worker 83 - jepsen.util 1973 :info :write [27 46] :RequestTimeoutException INFO [2022-01-10 05:40:50,237] jepsen worker 83 - jepsen.util 2063 :invoke :read [27 nil] INFO [2022-01-10 05:40:50,364] jepsen worker 82 - jepsen.util 1522 :info :write [27 36] :RequestTimeoutException INFO [2022-01-10 05:40:50,365] jepsen worker 82 - jepsen.util 1612 :invoke :write [27 24] INFO [2022-01-10 05:40:50,407] jepsen worker 41 - jepsen.util 1571 :ok :read [13 15] INFO [2022-01-10 05:40:50,408] jepsen worker 41 - jepsen.util 1571 :invoke :read [13 nil] INFO [2022-01-10 05:40:50,442] jepsen worker 40 - jepsen.util 1480 :ok :read [13 15] INFO [2022-01-10 05:40:50,443] jepsen worker 40 - jepsen.util 1480 :invoke :read [13 nil] INFO [2022-01-10 05:40:50,501] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3333333333333334}, :ns_1@172.23.107.227 {:progress 0.2316715542521994}, :ns_1@172.23.105.187 {:progress 0.2316715542521994}, :ns_1@172.23.107.228 {:progress 0.3235294117647058}} INFO [2022-01-10 05:40:50,501] jepsen worker nemesis - couchbase.util Rebalance check count: 205 INFO [2022-01-10 05:40:50,501] jepsen worker nemesis - couchbase.util Rebalance stuck count: 147 INFO [2022-01-10 05:40:50,596] jepsen worker 50 - jepsen.util 1850 :info :write [16 36] :RequestTimeoutException INFO [2022-01-10 05:40:50,597] jepsen worker 50 - jepsen.util 1940 :invoke :write [16 31] INFO [2022-01-10 05:40:50,950] jepsen worker 79 - jepsen.util 1789 :info :write [26 47] :RequestTimeoutException INFO [2022-01-10 05:40:50,951] jepsen worker 79 - jepsen.util 1879 :invoke :write [26 1] INFO [2022-01-10 05:40:51,318] jepsen worker 54 - jepsen.util 1854 :ok :read [18 14] INFO [2022-01-10 05:40:51,318] jepsen worker 54 - jepsen.util 1854 :invoke :read [18 nil] INFO [2022-01-10 05:40:51,362] jepsen worker 89 - jepsen.util 1979 :ok :read [29 0] INFO [2022-01-10 05:40:51,363] jepsen worker 89 - jepsen.util 1979 :invoke :read [29 nil] INFO [2022-01-10 05:40:51,366] jepsen worker 75 - jepsen.util 1335 :ok :read [25 :nil] INFO [2022-01-10 05:40:51,368] jepsen worker 75 - jepsen.util 1335 :invoke :read [25 nil] INFO [2022-01-10 05:40:51,377] jepsen worker 10 - jepsen.util 1900 :ok :read [3 2] INFO [2022-01-10 05:40:51,377] jepsen worker 10 - jepsen.util 1900 :invoke :write [3 45] WARN [2022-01-10 05:40:51,381] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10316615,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10908999,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9286746,"encode_duration_us":16593,"operation_id":"0x34f2","timeout_ms":10000,"last_server_duration_us":9201443,"total_duration_us":10004247}],"total_count":1}} WARN [2022-01-10 05:40:51,382] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":741747,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":741747,"total_server_duration_us":36220,"operation_id":"0x3526","timeout_ms":30000,"last_server_duration_us":36220,"total_duration_us":10499084},{"operation_name":"upsert","last_dispatch_duration_us":31998,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":1072261,"total_server_duration_us":668656,"encode_duration_us":20739,"operation_id":"0x3601","timeout_ms":10000,"last_server_duration_us":21813,"total_duration_us":10010034},{"operation_name":"upsert","last_dispatch_duration_us":662035,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1144863,"total_server_duration_us":278459,"encode_duration_us":16626,"operation_id":"0x3586","timeout_ms":10000,"last_server_duration_us":87802,"total_duration_us":10010000},{"operation_name":"upsert","last_dispatch_duration_us":8737605,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10525172,"total_server_duration_us":920053,"encode_duration_us":16868,"operation_id":"0x34f3","timeout_ms":10000,"last_server_duration_us":506771,"total_duration_us":10009682},{"operation_name":"upsert","last_dispatch_duration_us":34327,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":851940,"total_server_duration_us":491704,"encode_duration_us":17097,"operation_id":"0x362e","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10007593},{"operation_name":"upsert","last_dispatch_duration_us":557730,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10057763,"total_server_duration_us":151831,"encode_duration_us":16324,"operation_id":"0x3502","timeout_ms":10000,"last_server_duration_us":23983,"total_duration_us":10006127},{"operation_name":"upsert","last_dispatch_duration_us":1218963,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1218963,"total_server_duration_us":36017,"encode_duration_us":20529,"operation_id":"0x355d","timeout_ms":10000,"last_server_duration_us":36017,"total_duration_us":10005524},{"operation_name":"upsert","last_dispatch_duration_us":19216999,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":19216999,"total_server_duration_us":10019487,"encode_duration_us":17765,"operation_id":"0x3525","timeout_ms":10000,"last_server_duration_us":10019487,"total_duration_us":10005063},{"operation_name":"upsert","last_dispatch_duration_us":10316615,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10908999,"total_server_duration_us":9286746,"encode_duration_us":16593,"operation_id":"0x34f2","timeout_ms":10000,"last_server_duration_us":9201443,"total_duration_us":10004247},{"operation_name":"upsert","last_dispatch_duration_us":278018,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":1993638,"total_server_duration_us":569605,"encode_duration_us":24872,"operation_id":"0x3686","timeout_ms":10000,"last_server_duration_us":87506,"total_duration_us":10003408}],"total_count":17}} INFO [2022-01-10 05:40:51,403] jepsen worker 47 - jepsen.util 1667 :ok :read [15 23] INFO [2022-01-10 05:40:51,404] jepsen worker 47 - jepsen.util 1667 :invoke :write [15 14] INFO [2022-01-10 05:40:51,406] jepsen worker 57 - jepsen.util 1767 :info :write [19 42] :RequestTimeoutException INFO [2022-01-10 05:40:51,407] jepsen worker 57 - jepsen.util 1857 :invoke :write [19 7] WARN [2022-01-10 05:40:51,481] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10413095,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11668520,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9334296,"encode_duration_us":16872,"operation_id":"0x352a","timeout_ms":10000,"last_server_duration_us":9137368,"total_duration_us":10005521},{"operation_name":"upsert","last_dispatch_duration_us":9045252,"last_local_id":"EFE9906200000002/00000000598DAEAE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10384122,"last_local_socket":"172.23.104.79:44342","total_server_duration_us":9691896,"encode_duration_us":26676,"operation_id":"0x33ee","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10005720}],"total_count":2}} WARN [2022-01-10 05:40:51,481] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10662378,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10662378,"total_server_duration_us":77851,"operation_id":"0x3588","timeout_ms":30000,"last_server_duration_us":77851,"total_duration_us":10664550},{"operation_name":"upsert","last_dispatch_duration_us":77815,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":976214,"total_server_duration_us":544869,"encode_duration_us":17065,"operation_id":"0x35c5","timeout_ms":10000,"last_server_duration_us":35111,"total_duration_us":10009576},{"operation_name":"upsert","last_dispatch_duration_us":1392095,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10445246,"total_server_duration_us":524445,"encode_duration_us":19687,"operation_id":"0x35dc","timeout_ms":10000,"last_server_duration_us":126463,"total_duration_us":10007624},{"operation_name":"upsert","last_dispatch_duration_us":1369597,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9851803,"total_server_duration_us":546632,"encode_duration_us":16437,"operation_id":"0x36b4","timeout_ms":10000,"last_server_duration_us":57139,"total_duration_us":10006829},{"operation_name":"upsert","last_dispatch_duration_us":10413095,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11668520,"total_server_duration_us":9334296,"encode_duration_us":16872,"operation_id":"0x352a","timeout_ms":10000,"last_server_duration_us":9137368,"total_duration_us":10005521},{"operation_name":"upsert","last_dispatch_duration_us":38103,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":6084047,"total_server_duration_us":395015,"encode_duration_us":17459,"operation_id":"0x362f","timeout_ms":10000,"last_server_duration_us":22141,"total_duration_us":10005463},{"operation_name":"upsert","last_dispatch_duration_us":968538,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7943743,"total_server_duration_us":392453,"encode_duration_us":17080,"operation_id":"0x36a7","timeout_ms":10000,"last_server_duration_us":51828,"total_duration_us":10005370},{"operation_name":"upsert","last_dispatch_duration_us":33590,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":883169,"total_server_duration_us":509063,"encode_duration_us":17784,"operation_id":"0x3622","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10004109},{"operation_name":"upsert","last_dispatch_duration_us":10076161,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11736515,"total_server_duration_us":239604,"encode_duration_us":18389,"operation_id":"0x352b","timeout_ms":10000,"last_server_duration_us":47990,"total_duration_us":10003870},{"operation_name":"upsert","last_dispatch_duration_us":421285,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10306790,"total_server_duration_us":233774,"encode_duration_us":16530,"operation_id":"0x35f3","timeout_ms":10000,"last_server_duration_us":74234,"total_duration_us":10002890}],"total_count":21}} WARN [2022-01-10 05:40:51,481] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11801557,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11801557,"total_server_duration_us":128717,"encode_duration_us":16928,"operation_id":"0x3623","timeout_ms":10000,"last_server_duration_us":128717,"total_duration_us":10008658},{"operation_name":"upsert","last_dispatch_duration_us":528071,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":7259528,"total_server_duration_us":801240,"encode_duration_us":17210,"operation_id":"0x3517","timeout_ms":10000,"last_server_duration_us":76451,"total_duration_us":10008507},{"operation_name":"upsert","last_dispatch_duration_us":4278869,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7392188,"total_server_duration_us":485936,"encode_duration_us":17004,"operation_id":"0x351b","timeout_ms":10000,"last_server_duration_us":79687,"total_duration_us":10006967},{"operation_name":"upsert","last_dispatch_duration_us":7958841,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":14159481,"total_server_duration_us":598282,"encode_duration_us":16237,"operation_id":"0x3551","timeout_ms":10000,"last_server_duration_us":41337,"total_duration_us":10006317},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":19072,"operation_id":"0x364f","timeout_ms":10000,"total_duration_us":10005904},{"operation_name":"upsert","last_dispatch_duration_us":7945380,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11151156,"total_server_duration_us":550335,"encode_duration_us":19605,"operation_id":"0x3561","timeout_ms":10000,"last_server_duration_us":22720,"total_duration_us":10005860},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":19672,"operation_id":"0x35eb","timeout_ms":10000,"total_duration_us":10005139},{"operation_name":"upsert","last_dispatch_duration_us":3453807,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":3453807,"total_server_duration_us":205497,"encode_duration_us":16386,"operation_id":"0x35b9","timeout_ms":10000,"last_server_duration_us":205497,"total_duration_us":10004806},{"operation_name":"upsert","last_dispatch_duration_us":12776889,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12776889,"total_server_duration_us":9251768,"encode_duration_us":26822,"operation_id":"0x35c6","timeout_ms":10000,"last_server_duration_us":9251768,"total_duration_us":10004754},{"operation_name":"upsert","last_dispatch_duration_us":7942696,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":14458847,"total_server_duration_us":761092,"encode_duration_us":17216,"operation_id":"0x3530","timeout_ms":10000,"last_server_duration_us":53374,"total_duration_us":10003922}],"total_count":20}} INFO [2022-01-10 05:40:51,507] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3333333333333334}, :ns_1@172.23.107.227 {:progress 0.2316715542521994}, :ns_1@172.23.105.187 {:progress 0.2316715542521994}, :ns_1@172.23.107.228 {:progress 0.3235294117647058}} INFO [2022-01-10 05:40:51,507] jepsen worker nemesis - couchbase.util Rebalance check count: 206 INFO [2022-01-10 05:40:51,507] jepsen worker nemesis - couchbase.util Rebalance stuck count: 148 INFO [2022-01-10 05:40:51,559] jepsen worker 33 - jepsen.util 1833 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:40:51,560] jepsen worker 33 - jepsen.util 1833 :invoke :read [11 nil] WARN [2022-01-10 05:40:51,581] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1551834,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":6538873,"total_server_duration_us":740358,"encode_duration_us":16973,"operation_id":"0x36b1","timeout_ms":10000,"last_server_duration_us":39109,"total_duration_us":10010726},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":20803,"operation_id":"0x35b7","timeout_ms":10000,"total_duration_us":10008904},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":20899,"operation_id":"0x351d","timeout_ms":10000,"total_duration_us":10008320},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":20347,"operation_id":"0x3501","timeout_ms":10000,"total_duration_us":10008033},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":23061,"operation_id":"0x362b","timeout_ms":10000,"total_duration_us":10006556},{"operation_name":"upsert","last_dispatch_duration_us":8818361,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8818361,"total_server_duration_us":39846,"encode_duration_us":17228,"operation_id":"0x35c0","timeout_ms":10000,"last_server_duration_us":39846,"total_duration_us":10005607},{"operation_name":"upsert","last_dispatch_duration_us":609088,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8001286,"total_server_duration_us":806046,"encode_duration_us":19434,"operation_id":"0x3693","timeout_ms":10000,"last_server_duration_us":86182,"total_duration_us":10004484},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":17849,"operation_id":"0x3555","timeout_ms":10000,"total_duration_us":10004341},{"operation_name":"upsert","last_dispatch_duration_us":1390764,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1390764,"total_server_duration_us":308522,"encode_duration_us":17016,"operation_id":"0x35fe","timeout_ms":10000,"last_server_duration_us":308522,"total_duration_us":10003100},{"operation_name":"upsert","last_dispatch_duration_us":9085256,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9085256,"total_server_duration_us":9022513,"encode_duration_us":19288,"operation_id":"0x35a9","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":9090582}],"total_count":14}} WARN [2022-01-10 05:40:51,581] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10150774,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12078547,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":10409058,"encode_duration_us":17209,"operation_id":"0x3650","timeout_ms":10000,"last_server_duration_us":9648738,"total_duration_us":10000932},{"operation_name":"upsert","last_dispatch_duration_us":25276091,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":25276091,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9024634,"encode_duration_us":17594,"operation_id":"0x3287","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10009436}],"total_count":2}} WARN [2022-01-10 05:40:51,582] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9659448,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11757570,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9210592,"encode_duration_us":16436,"operation_id":"0x3589","timeout_ms":10000,"last_server_duration_us":9089436,"total_duration_us":10003905},{"operation_name":"upsert","last_dispatch_duration_us":17192309,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18131596,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9270429,"encode_duration_us":16624,"operation_id":"0x33b6","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10004716},{"operation_name":"upsert","last_dispatch_duration_us":12776889,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12776889,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9251768,"encode_duration_us":26822,"operation_id":"0x35c6","timeout_ms":10000,"last_server_duration_us":9251768,"total_duration_us":10004754},{"operation_name":"upsert","last_dispatch_duration_us":17481967,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17481967,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9027816,"encode_duration_us":20734,"operation_id":"0x341a","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10007979}],"total_count":4}} WARN [2022-01-10 05:40:51,681] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18573838,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19691231,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9687527,"encode_duration_us":17164,"operation_id":"0x3478","timeout_ms":10000,"last_server_duration_us":9552906,"total_duration_us":10002001},{"operation_name":"upsert","last_dispatch_duration_us":9054713,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":9604074,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":9399541,"encode_duration_us":17329,"operation_id":"0x348e","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":10008626}],"total_count":2}} WARN [2022-01-10 05:40:51,682] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2698764,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7931646,"total_server_duration_us":317797,"encode_duration_us":16657,"operation_id":"0x364e","timeout_ms":10000,"last_server_duration_us":101067,"total_duration_us":10010218},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":18432,"operation_id":"0x35ea","timeout_ms":10000,"total_duration_us":10010108},{"operation_name":"upsert","last_dispatch_duration_us":9478003,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":17530734,"total_server_duration_us":975264,"encode_duration_us":16361,"operation_id":"0x3552","timeout_ms":10000,"last_server_duration_us":314095,"total_duration_us":10009475},{"operation_name":"upsert","last_dispatch_duration_us":2108300,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8923093,"total_server_duration_us":499039,"encode_duration_us":16337,"operation_id":"0x35e9","timeout_ms":10000,"last_server_duration_us":77851,"total_duration_us":10008338},{"operation_name":"upsert","last_dispatch_duration_us":2267265,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10322507,"total_server_duration_us":689414,"encode_duration_us":16685,"operation_id":"0x3590","timeout_ms":10000,"last_server_duration_us":42417,"total_duration_us":10006738},{"operation_name":"upsert","last_dispatch_duration_us":8507327,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8507327,"total_server_duration_us":20601,"encode_duration_us":22339,"operation_id":"0x35d5","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10006350},{"operation_name":"upsert","last_dispatch_duration_us":33484,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":755148,"total_server_duration_us":358907,"encode_duration_us":16774,"operation_id":"0x36d5","timeout_ms":10000,"last_server_duration_us":22223,"total_duration_us":10005531},{"operation_name":"upsert","last_dispatch_duration_us":31757,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8993712,"total_server_duration_us":224348,"encode_duration_us":23083,"operation_id":"0x35ce","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10005519},{"operation_name":"upsert","last_dispatch_duration_us":30686,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":1171757,"total_server_duration_us":543013,"encode_duration_us":16358,"operation_id":"0x35fc","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10002205},{"operation_name":"upsert","last_dispatch_duration_us":1340537,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10160800,"total_server_duration_us":520776,"encode_duration_us":17263,"operation_id":"0x355f","timeout_ms":10000,"last_server_duration_us":44279,"total_duration_us":10001699}],"total_count":20}} INFO [2022-01-10 05:40:51,691] jepsen worker 0 - jepsen.util 1620 :info :write [0 11] :RequestTimeoutException INFO [2022-01-10 05:40:51,692] jepsen worker 0 - jepsen.util 1710 :invoke :read [0 nil] INFO [2022-01-10 05:40:52,022] jepsen worker 58 - jepsen.util 1768 :info :write [19 43] :RequestTimeoutException INFO [2022-01-10 05:40:52,022] jepsen worker 58 - jepsen.util 1858 :invoke :write [19 13] INFO [2022-01-10 05:40:52,076] jepsen worker 23 - jepsen.util 1823 :info :write [7 38] :RequestTimeoutException INFO [2022-01-10 05:40:52,077] jepsen worker 23 - jepsen.util 1913 :invoke :read [7 nil] INFO [2022-01-10 05:40:52,134] jepsen worker 46 - jepsen.util 1936 :info :write [15 45] :RequestTimeoutException INFO [2022-01-10 05:40:52,135] jepsen worker 46 - jepsen.util 2026 :invoke :write [15 36] INFO [2022-01-10 05:40:52,165] jepsen worker 6 - jepsen.util 1716 :info :write [2 13] :RequestTimeoutException INFO [2022-01-10 05:40:52,166] jepsen worker 6 - jepsen.util 1806 :invoke :write [2 34] INFO [2022-01-10 05:40:52,189] jepsen worker 28 - jepsen.util 1648 :info :write [9 48] :RequestTimeoutException INFO [2022-01-10 05:40:52,190] jepsen worker 28 - jepsen.util 1738 :invoke :write [9 11] INFO [2022-01-10 05:40:52,280] jepsen worker 81 - jepsen.util 1701 :info :write [27 21] :RequestTimeoutException INFO [2022-01-10 05:40:52,281] jepsen worker 81 - jepsen.util 1791 :invoke :read [27 nil] INFO [2022-01-10 05:40:52,352] jepsen worker 19 - jepsen.util 1909 :info :write [6 13] :RequestTimeoutException INFO [2022-01-10 05:40:52,353] jepsen worker 19 - jepsen.util 1999 :invoke :read [6 nil] INFO [2022-01-10 05:40:52,356] jepsen worker 63 - jepsen.util 1593 :info :write [21 36] :RequestTimeoutException INFO [2022-01-10 05:40:52,357] jepsen worker 63 - jepsen.util 1683 :invoke :write [21 5] INFO [2022-01-10 05:40:52,512] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3333333333333334}, :ns_1@172.23.107.227 {:progress 0.2316715542521994}, :ns_1@172.23.105.187 {:progress 0.2316715542521994}, :ns_1@172.23.107.228 {:progress 0.3235294117647058}} INFO [2022-01-10 05:40:52,512] jepsen worker nemesis - couchbase.util Rebalance check count: 207 INFO [2022-01-10 05:40:52,512] jepsen worker nemesis - couchbase.util Rebalance stuck count: 149 INFO [2022-01-10 05:40:52,561] jepsen worker 2 - jepsen.util 1712 :info :write [0 48] :RequestTimeoutException INFO [2022-01-10 05:40:52,562] jepsen worker 2 - jepsen.util 1802 :invoke :write [0 22] INFO [2022-01-10 05:40:52,566] jepsen worker 80 - jepsen.util 1790 :info :write [26 21] :RequestTimeoutException INFO [2022-01-10 05:40:52,566] jepsen worker 80 - jepsen.util 1880 :invoke :read [26 nil] INFO [2022-01-10 05:40:52,617] jepsen worker 14 - jepsen.util 1814 :info :write [4 0] :RequestTimeoutException INFO [2022-01-10 05:40:52,617] jepsen worker 14 - jepsen.util 1904 :invoke :read [4 nil] INFO [2022-01-10 05:40:52,671] jepsen worker 62 - jepsen.util 1952 :info :write [20 1] :RequestTimeoutException INFO [2022-01-10 05:40:52,671] jepsen worker 62 - jepsen.util 2042 :invoke :read [20 nil] INFO [2022-01-10 05:40:52,681] jepsen worker 26 - jepsen.util 1466 :info :write [8 11] :RequestTimeoutException INFO [2022-01-10 05:40:52,682] jepsen worker 26 - jepsen.util 1556 :invoke :write [8 39] INFO [2022-01-10 05:40:52,925] jepsen worker 48 - jepsen.util 1398 :info :write [16 27] :RequestTimeoutException INFO [2022-01-10 05:40:52,925] jepsen worker 48 - jepsen.util 1488 :invoke :read [16 nil] INFO [2022-01-10 05:40:52,928] jepsen worker 9 - jepsen.util 1989 :ok :read [3 2] INFO [2022-01-10 05:40:52,929] jepsen worker 9 - jepsen.util 1989 :invoke :write [3 34] INFO [2022-01-10 05:40:52,931] jepsen worker 71 - jepsen.util 1691 :ok :read [23 26] INFO [2022-01-10 05:40:52,932] jepsen worker 71 - jepsen.util 1691 :invoke :read [23 nil] INFO [2022-01-10 05:40:52,955] jepsen worker 31 - jepsen.util 2011 :info :write [10 35] :SyncWriteAmbiguous INFO [2022-01-10 05:40:52,956] jepsen worker 31 - jepsen.util 2101 :invoke :read [10 nil] INFO [2022-01-10 05:40:52,976] jepsen worker 65 - jepsen.util 1595 :info :write [21 8] :RequestTimeoutException INFO [2022-01-10 05:40:52,977] jepsen worker 65 - jepsen.util 1685 :invoke :write [21 16] INFO [2022-01-10 05:40:53,022] jepsen worker 23 - jepsen.util 1913 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:40:53,023] jepsen worker 23 - jepsen.util 1913 :invoke :write [7 31] INFO [2022-01-10 05:40:53,048] jepsen worker 33 - jepsen.util 1833 :ok :read [11 21] INFO [2022-01-10 05:40:53,049] jepsen worker 33 - jepsen.util 1833 :invoke :write [11 15] INFO [2022-01-10 05:40:53,130] jepsen worker 25 - jepsen.util 1735 :info :write [8 30] :RequestTimeoutException INFO [2022-01-10 05:40:53,131] jepsen worker 25 - jepsen.util 1825 :invoke :read [8 nil] INFO [2022-01-10 05:40:53,177] jepsen worker 42 - jepsen.util 1662 :info :write [14 18] :RequestTimeoutException INFO [2022-01-10 05:40:53,244] jepsen worker 16 - jepsen.util 2086 :info :write [5 28] :RequestTimeoutException INFO [2022-01-10 05:40:53,292] jepsen worker 14 - jepsen.util 1904 :ok :read [4 3] INFO [2022-01-10 05:40:53,331] jepsen worker 16 - jepsen.util 2176 :invoke :read [5 nil] INFO [2022-01-10 05:40:53,306] jepsen worker 88 - jepsen.util 1708 :info :write [29 21] :RequestTimeoutException INFO [2022-01-10 05:40:53,321] jepsen worker 87 - jepsen.util 1887 :info :write [29 42] :RequestTimeoutException INFO [2022-01-10 05:40:53,332] jepsen worker 42 - jepsen.util 1752 :invoke :write [14 21] INFO [2022-01-10 05:40:53,331] jepsen worker 14 - jepsen.util 1904 :invoke :read [4 nil] INFO [2022-01-10 05:40:53,332] jepsen worker 88 - jepsen.util 1798 :invoke :write [29 29] INFO [2022-01-10 05:40:53,332] jepsen worker 87 - jepsen.util 1977 :invoke :read [29 nil] INFO [2022-01-10 05:40:53,358] jepsen worker 14 - jepsen.util 1904 :ok :read [4 3] INFO [2022-01-10 05:40:53,359] jepsen worker 14 - jepsen.util 1904 :invoke :write [4 27] INFO [2022-01-10 05:40:53,395] jepsen worker 12 - jepsen.util 1542 :ok :read [4 3] INFO [2022-01-10 05:40:53,396] jepsen worker 12 - jepsen.util 1542 :invoke :write [4 38] INFO [2022-01-10 05:40:53,446] jepsen worker 80 - jepsen.util 1880 :ok :read [26 10] INFO [2022-01-10 05:40:53,446] jepsen worker 80 - jepsen.util 1880 :invoke :read [26 nil] INFO [2022-01-10 05:40:53,454] jepsen worker 51 - jepsen.util 1851 :ok :read [17 41] INFO [2022-01-10 05:40:53,454] jepsen worker 51 - jepsen.util 1851 :invoke :read [17 nil] INFO [2022-01-10 05:40:53,511] jepsen worker 17 - jepsen.util 1637 :info :write [5 41] :RequestTimeoutException INFO [2022-01-10 05:40:53,513] jepsen worker 17 - jepsen.util 1727 :invoke :write [5 27] INFO [2022-01-10 05:40:53,520] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3333333333333334}, :ns_1@172.23.107.227 {:progress 0.2316715542521994}, :ns_1@172.23.105.187 {:progress 0.2316715542521994}, :ns_1@172.23.107.228 {:progress 0.3235294117647058}} INFO [2022-01-10 05:40:53,520] jepsen worker nemesis - couchbase.util Rebalance check count: 208 INFO [2022-01-10 05:40:53,520] jepsen worker nemesis - couchbase.util Rebalance stuck count: 150 INFO [2022-01-10 05:40:53,535] jepsen worker 51 - jepsen.util 1851 :ok :read [17 41] INFO [2022-01-10 05:40:53,536] jepsen worker 51 - jepsen.util 1851 :invoke :write [17 42] INFO [2022-01-10 05:40:53,551] jepsen worker 3 - jepsen.util 1893 :ok :read [1 41] INFO [2022-01-10 05:40:53,552] jepsen worker 3 - jepsen.util 1893 :invoke :read [1 nil] INFO [2022-01-10 05:40:53,559] jepsen worker 36 - jepsen.util 1476 :ok :read [12 1] INFO [2022-01-10 05:40:53,559] jepsen worker 36 - jepsen.util 1476 :invoke :write [12 16] INFO [2022-01-10 05:40:53,573] jepsen worker 48 - jepsen.util 1488 :ok :read [16 38] INFO [2022-01-10 05:40:53,573] jepsen worker 48 - jepsen.util 1488 :invoke :read [16 nil] INFO [2022-01-10 05:40:53,596] jepsen worker 37 - jepsen.util 1837 :ok :read [12 1] INFO [2022-01-10 05:40:53,596] jepsen worker 37 - jepsen.util 1837 :invoke :read [12 nil] INFO [2022-01-10 05:40:53,610] jepsen worker 3 - jepsen.util 1893 :ok :read [1 41] INFO [2022-01-10 05:40:53,610] jepsen worker 3 - jepsen.util 1893 :invoke :write [1 41] INFO [2022-01-10 05:40:53,696] jepsen worker 37 - jepsen.util 1837 :ok :read [12 1] INFO [2022-01-10 05:40:53,697] jepsen worker 37 - jepsen.util 1837 :invoke :write [12 23] INFO [2022-01-10 05:40:53,761] jepsen worker 80 - jepsen.util 1880 :ok :read [26 10] INFO [2022-01-10 05:40:53,762] jepsen worker 80 - jepsen.util 1880 :invoke :write [26 28] INFO [2022-01-10 05:40:53,874] jepsen worker 48 - jepsen.util 1488 :ok :read [16 38] INFO [2022-01-10 05:40:53,874] jepsen worker 48 - jepsen.util 1488 :invoke :write [16 47] INFO [2022-01-10 05:40:53,944] jepsen worker 8 - jepsen.util 1808 :info :write [2 3] :RequestTimeoutException INFO [2022-01-10 05:40:53,945] jepsen worker 8 - jepsen.util 1898 :invoke :write [2 35] INFO [2022-01-10 05:40:54,046] jepsen worker 11 - jepsen.util 1631 :info :write [3 33] :RequestTimeoutException INFO [2022-01-10 05:40:54,047] jepsen worker 11 - jepsen.util 1721 :invoke :read [3 nil] INFO [2022-01-10 05:40:54,526] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3333333333333334}, :ns_1@172.23.107.227 {:progress 0.2316715542521994}, :ns_1@172.23.105.187 {:progress 0.2316715542521994}, :ns_1@172.23.107.228 {:progress 0.3235294117647058}} INFO [2022-01-10 05:40:54,526] jepsen worker nemesis - couchbase.util Rebalance check count: 209 INFO [2022-01-10 05:40:54,526] jepsen worker nemesis - couchbase.util Rebalance stuck count: 151 INFO [2022-01-10 05:40:54,857] jepsen worker 69 - jepsen.util 1779 :info :write [23 49] :RequestTimeoutException INFO [2022-01-10 05:40:54,857] jepsen worker 69 - jepsen.util 1869 :invoke :read [23 nil] INFO [2022-01-10 05:40:55,061] jepsen worker 60 - jepsen.util 2130 :info :write [20 2] :RequestTimeoutException INFO [2022-01-10 05:40:55,062] jepsen worker 60 - jepsen.util 2220 :invoke :read [20 nil] INFO [2022-01-10 05:40:55,206] jepsen worker 5 - jepsen.util 1715 :info :write [1 20] :RequestTimeoutException INFO [2022-01-10 05:40:55,206] jepsen worker 5 - jepsen.util 1805 :invoke :read [1 nil] INFO [2022-01-10 05:40:55,240] jepsen worker 74 - jepsen.util 2234 :info :write [24 19] :RequestTimeoutException INFO [2022-01-10 05:40:55,240] jepsen worker 74 - jepsen.util 2324 :invoke :write [24 23] INFO [2022-01-10 05:40:55,381] jepsen worker 29 - jepsen.util 1739 :info :write [9 0] :RequestTimeoutException INFO [2022-01-10 05:40:55,381] jepsen worker 29 - jepsen.util 1829 :invoke :write [9 3] INFO [2022-01-10 05:40:55,421] jepsen worker 31 - jepsen.util 2101 :ok :read [10 37] INFO [2022-01-10 05:40:55,422] jepsen worker 31 - jepsen.util 2101 :invoke :write [10 48] INFO [2022-01-10 05:40:55,469] jepsen worker 5 - jepsen.util 1805 :ok :read [1 41] INFO [2022-01-10 05:40:55,469] jepsen worker 5 - jepsen.util 1805 :invoke :write [1 47] INFO [2022-01-10 05:40:55,533] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3333333333333334}, :ns_1@172.23.107.227 {:progress 0.2316715542521994}, :ns_1@172.23.105.187 {:progress 0.2316715542521994}, :ns_1@172.23.107.228 {:progress 0.3235294117647058}} INFO [2022-01-10 05:40:55,533] jepsen worker nemesis - couchbase.util Rebalance check count: 210 INFO [2022-01-10 05:40:55,533] jepsen worker nemesis - couchbase.util Rebalance stuck count: 152 INFO [2022-01-10 05:40:55,570] jepsen worker 68 - jepsen.util 1688 :info :write [22 11] :RequestTimeoutException INFO [2022-01-10 05:40:55,570] jepsen worker 68 - jepsen.util 1778 :invoke :read [22 nil] INFO [2022-01-10 05:40:55,696] jepsen worker 61 - jepsen.util 1681 :info :write [20 34] :RequestTimeoutException INFO [2022-01-10 05:40:55,697] jepsen worker 61 - jepsen.util 1771 :invoke :write [20 46] INFO [2022-01-10 05:40:55,731] jepsen worker 30 - jepsen.util 1380 :info :write [10 29] :RequestTimeoutException INFO [2022-01-10 05:40:55,732] jepsen worker 30 - jepsen.util 1470 :invoke :write [10 3] INFO [2022-01-10 05:40:55,765] jepsen worker 56 - jepsen.util 1856 :info :write [18 32] :RequestTimeoutException INFO [2022-01-10 05:40:55,766] jepsen worker 56 - jepsen.util 1946 :invoke :write [18 15] INFO [2022-01-10 05:40:55,806] jepsen worker 39 - jepsen.util 2019 :info :write [13 21] :RequestTimeoutException INFO [2022-01-10 05:40:55,807] jepsen worker 39 - jepsen.util 2109 :invoke :write [13 12] INFO [2022-01-10 05:40:56,035] jepsen worker 72 - jepsen.util 1782 :info :write [24 8] :RequestTimeoutException INFO [2022-01-10 05:40:56,036] jepsen worker 72 - jepsen.util 1872 :invoke :write [24 30] INFO [2022-01-10 05:40:56,195] jepsen worker 7 - jepsen.util 1807 :info :write [2 43] :RequestTimeoutException INFO [2022-01-10 05:40:56,196] jepsen worker 7 - jepsen.util 1897 :invoke :write [2 28] INFO [2022-01-10 05:40:56,201] jepsen worker 45 - jepsen.util 1845 :info :write [15 20] :RequestTimeoutException INFO [2022-01-10 05:40:56,202] jepsen worker 45 - jepsen.util 1935 :invoke :write [15 9] INFO [2022-01-10 05:40:56,250] jepsen worker 15 - jepsen.util 1455 :info :write [5 1] :RequestTimeoutException INFO [2022-01-10 05:40:56,251] jepsen worker 15 - jepsen.util 1545 :invoke :read [5 nil] INFO [2022-01-10 05:40:56,296] jepsen worker 64 - jepsen.util 1864 :info :write [21 11] :RequestTimeoutException INFO [2022-01-10 05:40:56,296] jepsen worker 64 - jepsen.util 1954 :invoke :read [21 nil] INFO [2022-01-10 05:40:56,324] jepsen worker 4 - jepsen.util 1714 :info :write [1 40] :RequestTimeoutException INFO [2022-01-10 05:40:56,325] jepsen worker 4 - jepsen.util 1804 :invoke :write [1 34] INFO [2022-01-10 05:40:56,390] jepsen worker 49 - jepsen.util 1489 :info :write [16 48] :RequestTimeoutException INFO [2022-01-10 05:40:56,391] jepsen worker 49 - jepsen.util 1579 :invoke :read [16 nil] INFO [2022-01-10 05:40:56,485] jepsen worker 73 - jepsen.util 1783 :info :write [24 9] :RequestTimeoutException INFO [2022-01-10 05:40:56,486] jepsen worker 73 - jepsen.util 1873 :invoke :read [24 nil] INFO [2022-01-10 05:40:56,539] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3333333333333334}, :ns_1@172.23.107.227 {:progress 0.2316715542521994}, :ns_1@172.23.105.187 {:progress 0.2316715542521994}, :ns_1@172.23.107.228 {:progress 0.3235294117647058}} INFO [2022-01-10 05:40:56,539] jepsen worker nemesis - couchbase.util Rebalance check count: 211 INFO [2022-01-10 05:40:56,539] jepsen worker nemesis - couchbase.util Rebalance stuck count: 153 INFO [2022-01-10 05:40:56,556] jepsen worker 53 - jepsen.util 1853 :info :write [17 26] :RequestTimeoutException INFO [2022-01-10 05:40:56,557] jepsen worker 53 - jepsen.util 1943 :invoke :read [17 nil] INFO [2022-01-10 05:40:56,663] jepsen worker 53 - jepsen.util 1943 :ok :read [17 41] INFO [2022-01-10 05:40:56,663] jepsen worker 53 - jepsen.util 1943 :invoke :read [17 nil] INFO [2022-01-10 05:40:56,871] jepsen worker 53 - jepsen.util 1943 :ok :read [17 41] INFO [2022-01-10 05:40:56,871] jepsen worker 53 - jepsen.util 1943 :invoke :read [17 nil] INFO [2022-01-10 05:40:56,965] jepsen worker 84 - jepsen.util 1884 :info :write [28 15] :RequestTimeoutException INFO [2022-01-10 05:40:56,966] jepsen worker 84 - jepsen.util 1974 :invoke :write [28 49] INFO [2022-01-10 05:40:57,015] jepsen worker 21 - jepsen.util 1911 :info :write [7 25] :RequestTimeoutException INFO [2022-01-10 05:40:57,015] jepsen worker 21 - jepsen.util 2001 :invoke :read [7 nil] INFO [2022-01-10 05:40:57,071] jepsen worker 53 - jepsen.util 1943 :ok :read [17 41] INFO [2022-01-10 05:40:57,072] jepsen worker 53 - jepsen.util 1943 :invoke :write [17 44] INFO [2022-01-10 05:40:57,190] jepsen worker 67 - jepsen.util 1687 :info :write [22 47] :RequestTimeoutException INFO [2022-01-10 05:40:57,191] jepsen worker 67 - jepsen.util 1777 :invoke :read [22 nil] INFO [2022-01-10 05:40:57,442] jepsen worker 38 - jepsen.util 1838 :info :write [12 42] :RequestTimeoutException INFO [2022-01-10 05:40:57,442] jepsen worker 38 - jepsen.util 1928 :invoke :read [12 nil] INFO [2022-01-10 05:40:57,493] jepsen worker 85 - jepsen.util 2065 :ok :read [28 13] INFO [2022-01-10 05:40:57,494] jepsen worker 85 - jepsen.util 2065 :invoke :write [28 1] INFO [2022-01-10 05:40:57,517] jepsen worker 66 - jepsen.util 1596 :ok :read [22 41] INFO [2022-01-10 05:40:57,518] jepsen worker 66 - jepsen.util 1596 :invoke :write [22 24] INFO [2022-01-10 05:40:57,524] jepsen worker 15 - jepsen.util 1545 :ok :read [5 42] INFO [2022-01-10 05:40:57,524] jepsen worker 15 - jepsen.util 1545 :invoke :write [5 25] INFO [2022-01-10 05:40:57,548] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3391812865497076}, :ns_1@172.23.107.227 {:progress 0.2346041055718475}, :ns_1@172.23.105.187 {:progress 0.2346041055718475}, :ns_1@172.23.107.228 {:progress 0.3294117647058824}} INFO [2022-01-10 05:40:57,548] jepsen worker nemesis - couchbase.util Rebalance check count: 212 INFO [2022-01-10 05:40:57,548] jepsen worker nemesis - couchbase.util Rebalance stuck count: 153 INFO [2022-01-10 05:40:57,559] jepsen worker 83 - jepsen.util 2063 :ok :read [27 30] INFO [2022-01-10 05:40:57,559] jepsen worker 83 - jepsen.util 2063 :invoke :read [27 nil] INFO [2022-01-10 05:40:57,607] jepsen worker 13 - jepsen.util 1543 :info :write [4 32] :RequestTimeoutException INFO [2022-01-10 05:40:57,607] jepsen worker 13 - jepsen.util 1633 :invoke :write [4 5] INFO [2022-01-10 05:40:57,629] jepsen worker 1 - jepsen.util 1891 :ok :read [0 46] INFO [2022-01-10 05:40:57,629] jepsen worker 1 - jepsen.util 1891 :invoke :read [0 nil] INFO [2022-01-10 05:40:57,635] jepsen worker 38 - jepsen.util 1928 :ok :read [12 1] INFO [2022-01-10 05:40:57,636] jepsen worker 38 - jepsen.util 1928 :invoke :read [12 nil] INFO [2022-01-10 05:40:57,647] jepsen worker 0 - jepsen.util 1710 :ok :read [0 46] INFO [2022-01-10 05:40:57,648] jepsen worker 0 - jepsen.util 1710 :invoke :write [0 25] INFO [2022-01-10 05:40:57,696] jepsen worker 38 - jepsen.util 1928 :ok :read [12 1] INFO [2022-01-10 05:40:57,697] jepsen worker 38 - jepsen.util 1928 :invoke :write [12 42] INFO [2022-01-10 05:40:57,756] jepsen worker 81 - jepsen.util 1791 :ok :read [27 30] INFO [2022-01-10 05:40:57,757] jepsen worker 81 - jepsen.util 1791 :invoke :write [27 3] INFO [2022-01-10 05:40:57,793] jepsen worker 68 - jepsen.util 1778 :ok :read [22 41] INFO [2022-01-10 05:40:57,794] jepsen worker 68 - jepsen.util 1778 :invoke :read [22 nil] INFO [2022-01-10 05:40:57,794] jepsen worker 16 - jepsen.util 2176 :ok :read [5 42] INFO [2022-01-10 05:40:57,795] jepsen worker 16 - jepsen.util 2176 :invoke :write [5 38] INFO [2022-01-10 05:40:57,927] jepsen worker 1 - jepsen.util 1891 :ok :read [0 46] INFO [2022-01-10 05:40:57,928] jepsen worker 1 - jepsen.util 1891 :invoke :write [0 47] INFO [2022-01-10 05:40:58,109] jepsen worker 68 - jepsen.util 1778 :ok :read [22 41] INFO [2022-01-10 05:40:58,109] jepsen worker 68 - jepsen.util 1778 :invoke :read [22 nil] INFO [2022-01-10 05:40:58,300] jepsen worker 41 - jepsen.util 1571 :ok :read [13 15] INFO [2022-01-10 05:40:58,301] jepsen worker 83 - jepsen.util 2063 :ok :read [27 30] INFO [2022-01-10 05:40:58,306] jepsen worker 40 - jepsen.util 1480 :ok :read [13 15] INFO [2022-01-10 05:40:58,319] jepsen worker 83 - jepsen.util 2063 :invoke :read [27 nil] INFO [2022-01-10 05:40:58,319] jepsen worker 41 - jepsen.util 1571 :invoke :read [13 nil] INFO [2022-01-10 05:40:58,319] jepsen worker 40 - jepsen.util 1480 :invoke :read [13 nil] INFO [2022-01-10 05:40:58,514] jepsen worker 44 - jepsen.util 1844 :info :write [14 1] :RequestTimeoutException INFO [2022-01-10 05:40:58,515] jepsen worker 44 - jepsen.util 1934 :invoke :read [14 nil] INFO [2022-01-10 05:40:58,515] jepsen worker 32 - jepsen.util 1742 :info :write [10 23] :RequestTimeoutException INFO [2022-01-10 05:40:58,516] jepsen worker 32 - jepsen.util 1832 :invoke :write [10 9] INFO [2022-01-10 05:40:58,555] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3391812865497076}, :ns_1@172.23.107.227 {:progress 0.2346041055718475}, :ns_1@172.23.105.187 {:progress 0.2346041055718475}, :ns_1@172.23.107.228 {:progress 0.3294117647058824}} INFO [2022-01-10 05:40:58,555] jepsen worker nemesis - couchbase.util Rebalance check count: 213 INFO [2022-01-10 05:40:58,555] jepsen worker nemesis - couchbase.util Rebalance stuck count: 154 INFO [2022-01-10 05:40:58,630] jepsen worker 24 - jepsen.util 1734 :info :write [8 6] :RequestTimeoutException INFO [2022-01-10 05:40:58,631] jepsen worker 24 - jepsen.util 1824 :invoke :read [8 nil] INFO [2022-01-10 05:40:58,978] jepsen worker 35 - jepsen.util 1655 :ok :read [11 21] INFO [2022-01-10 05:40:58,978] jepsen worker 35 - jepsen.util 1655 :invoke :read [11 nil] INFO [2022-01-10 05:40:59,182] jepsen worker 27 - jepsen.util 1647 :info :write [9 30] :RequestTimeoutException INFO [2022-01-10 05:40:59,183] jepsen worker 27 - jepsen.util 1737 :invoke :read [9 nil] INFO [2022-01-10 05:40:59,306] jepsen worker 78 - jepsen.util 1788 :info :write [26 37] :RequestTimeoutException INFO [2022-01-10 05:40:59,307] jepsen worker 78 - jepsen.util 1878 :invoke :write [26 18] INFO [2022-01-10 05:40:59,498] jepsen worker 83 - jepsen.util 2063 :ok :read [27 30] INFO [2022-01-10 05:40:59,499] jepsen worker 83 - jepsen.util 2063 :invoke :read [27 nil] INFO [2022-01-10 05:40:59,561] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3391812865497076}, :ns_1@172.23.107.227 {:progress 0.2346041055718475}, :ns_1@172.23.105.187 {:progress 0.2346041055718475}, :ns_1@172.23.107.228 {:progress 0.3294117647058824}} INFO [2022-01-10 05:40:59,561] jepsen worker nemesis - couchbase.util Rebalance check count: 214 INFO [2022-01-10 05:40:59,561] jepsen worker nemesis - couchbase.util Rebalance stuck count: 155 INFO [2022-01-10 05:40:59,593] jepsen worker 18 - jepsen.util 1368 :ok :read [6 25] INFO [2022-01-10 05:40:59,594] jepsen worker 18 - jepsen.util 1368 :invoke :write [6 20] INFO [2022-01-10 05:40:59,715] jepsen worker 86 - jepsen.util 1526 :info :write [28 15] :RequestTimeoutException INFO [2022-01-10 05:40:59,716] jepsen worker 86 - jepsen.util 1616 :invoke :read [28 nil] INFO [2022-01-10 05:40:59,716] jepsen worker 52 - jepsen.util 2032 :info :write [17 33] :RequestTimeoutException INFO [2022-01-10 05:40:59,717] jepsen worker 52 - jepsen.util 2122 :invoke :write [17 16] INFO [2022-01-10 05:40:59,760] jepsen worker 59 - jepsen.util 1859 :info :write [19 32] :RequestTimeoutException INFO [2022-01-10 05:40:59,760] jepsen worker 59 - jepsen.util 1949 :invoke :read [19 nil] INFO [2022-01-10 05:40:59,774] jepsen worker 75 - jepsen.util 1335 :ok :read [25 :nil] INFO [2022-01-10 05:40:59,774] jepsen worker 60 - jepsen.util 2220 :ok :read [20 :nil] INFO [2022-01-10 05:40:59,775] jepsen worker 75 - jepsen.util 1335 :invoke :read [25 nil] INFO [2022-01-10 05:40:59,776] jepsen worker 60 - jepsen.util 2220 :invoke :write [20 24] WARN [2022-01-10 05:40:59,792] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18080809,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18080809,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9025695,"encode_duration_us":16396,"operation_id":"0x355e","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10001720},{"operation_name":"upsert","last_dispatch_duration_us":11564312,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11564312,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10373645,"encode_duration_us":17996,"operation_id":"0x37e2","timeout_ms":10000,"last_server_duration_us":10373645,"total_duration_us":10008534}],"total_count":2}} WARN [2022-01-10 05:40:59,892] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16054972,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":16054972,"total_server_duration_us":43,"operation_id":"0x35c2","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":16056982},{"operation_name":"get","last_dispatch_duration_us":14678814,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":14678814,"total_server_duration_us":62,"operation_id":"0x3602","timeout_ms":30000,"last_server_duration_us":62,"total_duration_us":14679998},{"operation_name":"get","last_dispatch_duration_us":13979542,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":13979542,"total_server_duration_us":32,"operation_id":"0x362d","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":13981967},{"operation_name":"upsert","last_dispatch_duration_us":6760226,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7513384,"total_server_duration_us":546090,"encode_duration_us":16864,"operation_id":"0x3844","timeout_ms":10000,"last_server_duration_us":394285,"total_duration_us":10008589},{"operation_name":"upsert","last_dispatch_duration_us":11564312,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11564312,"total_server_duration_us":10373645,"encode_duration_us":17996,"operation_id":"0x37e2","timeout_ms":10000,"last_server_duration_us":10373645,"total_duration_us":10008534},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":16318,"operation_id":"0x37c3","timeout_ms":10000,"total_duration_us":10007316},{"operation_name":"upsert","last_dispatch_duration_us":1423166,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2374168,"total_server_duration_us":236438,"encode_duration_us":16312,"operation_id":"0x3809","timeout_ms":10000,"last_server_duration_us":217098,"total_duration_us":10005712},{"operation_name":"upsert","last_dispatch_duration_us":6883500,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9592615,"total_server_duration_us":1343368,"encode_duration_us":16492,"operation_id":"0x37d4","timeout_ms":10000,"last_server_duration_us":596880,"total_duration_us":10005710},{"operation_name":"upsert","last_dispatch_duration_us":828384,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":6899767,"total_server_duration_us":2341057,"encode_duration_us":16383,"operation_id":"0x3877","timeout_ms":10000,"last_server_duration_us":161260,"total_duration_us":10005706},{"operation_name":"upsert","last_dispatch_duration_us":2039523,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11811575,"total_server_duration_us":1338372,"encode_duration_us":19374,"operation_id":"0x36a6","timeout_ms":10000,"last_server_duration_us":392886,"total_duration_us":10005558}],"total_count":24}} INFO [2022-01-10 05:40:59,918] jepsen worker 41 - jepsen.util 1571 :ok :read [13 15] INFO [2022-01-10 05:40:59,919] jepsen worker 41 - jepsen.util 1571 :invoke :read [13 nil] INFO [2022-01-10 05:40:59,925] jepsen worker 40 - jepsen.util 1480 :ok :read [13 15] INFO [2022-01-10 05:40:59,925] jepsen worker 40 - jepsen.util 1480 :invoke :read [13 nil] INFO [2022-01-10 05:40:59,992] jepsen worker 34 - jepsen.util 1834 :info :write [11 40] :RequestTimeoutException INFO [2022-01-10 05:40:59,992] jepsen worker 34 - jepsen.util 1924 :invoke :read [11 nil] INFO [2022-01-10 05:41:00,060] jepsen worker 49 - jepsen.util 1579 :ok :read [16 38] INFO [2022-01-10 05:41:00,061] jepsen worker 49 - jepsen.util 1579 :invoke :write [16 20] INFO [2022-01-10 05:41:00,062] jepsen worker 83 - jepsen.util 2063 :ok :read [27 30] INFO [2022-01-10 05:41:00,063] jepsen worker 83 - jepsen.util 2063 :invoke :write [27 38] INFO [2022-01-10 05:41:00,088] jepsen worker 68 - jepsen.util 1778 :ok :read [22 41] INFO [2022-01-10 05:41:00,088] jepsen worker 68 - jepsen.util 1778 :invoke :write [22 48] INFO [2022-01-10 05:41:00,385] jepsen worker 82 - jepsen.util 1612 :info :write [27 24] :RequestTimeoutException INFO [2022-01-10 05:41:00,386] jepsen worker 82 - jepsen.util 1702 :invoke :read [27 nil] INFO [2022-01-10 05:41:00,516] jepsen worker 54 - jepsen.util 1854 :ok :read [18 14] INFO [2022-01-10 05:41:00,517] jepsen worker 54 - jepsen.util 1854 :invoke :read [18 nil] INFO [2022-01-10 05:41:00,525] jepsen worker 89 - jepsen.util 1979 :ok :read [29 0] INFO [2022-01-10 05:41:00,526] jepsen worker 89 - jepsen.util 1979 :invoke :read [29 nil] INFO [2022-01-10 05:41:00,567] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3391812865497076}, :ns_1@172.23.107.227 {:progress 0.2346041055718475}, :ns_1@172.23.105.187 {:progress 0.2346041055718475}, :ns_1@172.23.107.228 {:progress 0.3294117647058824}} INFO [2022-01-10 05:41:00,567] jepsen worker nemesis - couchbase.util Rebalance check count: 215 INFO [2022-01-10 05:41:00,567] jepsen worker nemesis - couchbase.util Rebalance stuck count: 156 INFO [2022-01-10 05:41:00,570] jepsen worker 87 - jepsen.util 1977 :ok :read [29 0] INFO [2022-01-10 05:41:00,570] jepsen worker 87 - jepsen.util 1977 :invoke :write [29 30] INFO [2022-01-10 05:41:00,616] jepsen worker 50 - jepsen.util 1940 :info :write [16 31] :RequestTimeoutException INFO [2022-01-10 05:41:00,617] jepsen worker 50 - jepsen.util 2030 :invoke :write [16 15] INFO [2022-01-10 05:41:00,643] jepsen worker 41 - jepsen.util 1571 :ok :read [13 15] INFO [2022-01-10 05:41:00,643] jepsen worker 41 - jepsen.util 1571 :invoke :write [13 15] INFO [2022-01-10 05:41:00,647] jepsen worker 40 - jepsen.util 1480 :ok :read [13 15] INFO [2022-01-10 05:41:00,647] jepsen worker 40 - jepsen.util 1480 :invoke :read [13 nil] INFO [2022-01-10 05:41:00,653] jepsen worker 54 - jepsen.util 1854 :ok :read [18 14] INFO [2022-01-10 05:41:00,653] jepsen worker 54 - jepsen.util 1854 :invoke :read [18 nil] INFO [2022-01-10 05:41:00,691] jepsen worker 89 - jepsen.util 1979 :ok :read [29 0] INFO [2022-01-10 05:41:00,691] jepsen worker 89 - jepsen.util 1979 :invoke :write [29 40] INFO [2022-01-10 05:41:00,808] jepsen worker 54 - jepsen.util 1854 :ok :read [18 14] INFO [2022-01-10 05:41:00,809] jepsen worker 54 - jepsen.util 1854 :invoke :read [18 nil] INFO [2022-01-10 05:41:00,970] jepsen worker 79 - jepsen.util 1879 :info :write [26 1] :RequestTimeoutException INFO [2022-01-10 05:41:00,970] jepsen worker 79 - jepsen.util 1969 :invoke :write [26 6] INFO [2022-01-10 05:41:00,990] jepsen worker 54 - jepsen.util 1854 :ok :read [18 14] INFO [2022-01-10 05:41:00,991] jepsen worker 54 - jepsen.util 1854 :invoke :read [18 nil] INFO [2022-01-10 05:41:01,405] jepsen worker 10 - jepsen.util 1900 :info :write [3 45] :RequestTimeoutException INFO [2022-01-10 05:41:01,405] jepsen worker 10 - jepsen.util 1990 :invoke :write [3 29] INFO [2022-01-10 05:41:01,411] jepsen worker 75 - jepsen.util 1335 :ok :read [25 :nil] INFO [2022-01-10 05:41:01,411] jepsen worker 75 - jepsen.util 1335 :invoke :read [25 nil] INFO [2022-01-10 05:41:01,430] jepsen worker 47 - jepsen.util 1667 :info :write [15 14] :RequestTimeoutException INFO [2022-01-10 05:41:01,431] jepsen worker 47 - jepsen.util 1757 :invoke :write [15 9] INFO [2022-01-10 05:41:01,432] jepsen worker 57 - jepsen.util 1857 :info :write [19 7] :RequestTimeoutException INFO [2022-01-10 05:41:01,432] jepsen worker 57 - jepsen.util 1947 :invoke :write [19 38] WARN [2022-01-10 05:41:01,494] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11801019,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13245393,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":11802814,"encode_duration_us":16574,"operation_id":"0x37df","timeout_ms":10000,"last_server_duration_us":11395774,"total_duration_us":10002376},{"operation_name":"upsert","last_dispatch_duration_us":9039813,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10016027,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9571625,"encode_duration_us":17065,"operation_id":"0x35c5","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10009576},{"operation_name":"upsert","last_dispatch_duration_us":10246594,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18190338,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9933409,"encode_duration_us":17080,"operation_id":"0x36a7","timeout_ms":10000,"last_server_duration_us":9540956,"total_duration_us":10005370}],"total_count":3}} WARN [2022-01-10 05:41:01,494] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8769873,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":13959988,"total_server_duration_us":482309,"encode_duration_us":17248,"operation_id":"0x37ca","timeout_ms":10000,"last_server_duration_us":122858,"total_duration_us":10010726},{"operation_name":"upsert","last_dispatch_duration_us":39044,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9470285,"total_server_duration_us":428748,"encode_duration_us":16712,"operation_id":"0x3843","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10009990},{"operation_name":"upsert","last_dispatch_duration_us":10116113,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10116113,"total_server_duration_us":1597652,"encode_duration_us":18113,"operation_id":"0x37a9","timeout_ms":10000,"last_server_duration_us":1597652,"total_duration_us":10009891},{"operation_name":"upsert","last_dispatch_duration_us":849745,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3075563,"total_server_duration_us":394368,"encode_duration_us":16625,"operation_id":"0x373c","timeout_ms":10000,"last_server_duration_us":185952,"total_duration_us":10008495},{"operation_name":"upsert","last_dispatch_duration_us":13457660,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":15173632,"total_server_duration_us":2299280,"encode_duration_us":17042,"operation_id":"0x37dc","timeout_ms":10000,"last_server_duration_us":1825595,"total_duration_us":10008307},{"operation_name":"upsert","last_dispatch_duration_us":8645103,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12381279,"total_server_duration_us":263826,"encode_duration_us":16834,"operation_id":"0x3808","timeout_ms":10000,"last_server_duration_us":33524,"total_duration_us":10007415},{"operation_name":"upsert","last_dispatch_duration_us":34326,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":1451537,"total_server_duration_us":558816,"encode_duration_us":31308,"operation_id":"0x3715","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10006974},{"operation_name":"upsert","last_dispatch_duration_us":575161,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5949512,"total_server_duration_us":576086,"encode_duration_us":16354,"operation_id":"0x370b","timeout_ms":10000,"last_server_duration_us":185342,"total_duration_us":10006725},{"operation_name":"upsert","last_dispatch_duration_us":1126938,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":3191818,"total_server_duration_us":914147,"encode_duration_us":16439,"operation_id":"0x37e7","timeout_ms":10000,"last_server_duration_us":202322,"total_duration_us":10006212},{"operation_name":"upsert","last_dispatch_duration_us":1529948,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":5369197,"total_server_duration_us":1603175,"encode_duration_us":16194,"operation_id":"0x3878","timeout_ms":10000,"last_server_duration_us":601231,"total_duration_us":10003833}],"total_count":14}} WARN [2022-01-10 05:41:01,495] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16755497,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":16755497,"total_server_duration_us":2897039,"operation_id":"0x373a","timeout_ms":30000,"last_server_duration_us":2897039,"total_duration_us":16757644},{"operation_name":"upsert","last_dispatch_duration_us":9434758,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11366097,"total_server_duration_us":405111,"encode_duration_us":18813,"operation_id":"0x36df","timeout_ms":10000,"last_server_duration_us":110837,"total_duration_us":10010099},{"operation_name":"upsert","last_dispatch_duration_us":183216,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":183216,"total_server_duration_us":21487,"encode_duration_us":16231,"operation_id":"0x371b","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10009603},{"operation_name":"upsert","last_dispatch_duration_us":9891645,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":13712093,"total_server_duration_us":2110140,"encode_duration_us":18776,"operation_id":"0x3714","timeout_ms":10000,"last_server_duration_us":713248,"total_duration_us":10008951},{"operation_name":"upsert","last_dispatch_duration_us":158163,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":158163,"total_server_duration_us":19185,"encode_duration_us":16963,"operation_id":"0x371a","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10007850},{"operation_name":"upsert","last_dispatch_duration_us":10137049,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10433761,"total_server_duration_us":9954791,"encode_duration_us":17861,"operation_id":"0x37ab","timeout_ms":10000,"last_server_duration_us":9870254,"total_duration_us":10007347},{"operation_name":"upsert","last_dispatch_duration_us":10017718,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":12534817,"total_server_duration_us":10319605,"encode_duration_us":16947,"operation_id":"0x374e","timeout_ms":10000,"last_server_duration_us":9826220,"total_duration_us":10007228},{"operation_name":"upsert","encode_duration_us":17190,"operation_id":"0x37dd","timeout_ms":10000,"total_duration_us":10006251},{"operation_name":"upsert","last_dispatch_duration_us":6955468,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":7526917,"total_server_duration_us":346904,"encode_duration_us":35380,"operation_id":"0x374b","timeout_ms":10000,"last_server_duration_us":22306,"total_duration_us":10004023},{"operation_name":"upsert","last_dispatch_duration_us":31118,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":62682,"total_server_duration_us":41926,"encode_duration_us":17336,"operation_id":"0x37bc","timeout_ms":10000,"last_server_duration_us":20521,"total_duration_us":10002768}],"total_count":18}} WARN [2022-01-10 05:41:01,495] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":19216999,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19216999,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":10019487,"encode_duration_us":17765,"operation_id":"0x3525","timeout_ms":10000,"last_server_duration_us":10019487,"total_duration_us":10005063},{"operation_name":"upsert","last_dispatch_duration_us":10137049,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10433761,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9954791,"encode_duration_us":17861,"operation_id":"0x37ab","timeout_ms":10000,"last_server_duration_us":9870254,"total_duration_us":10007347},{"operation_name":"upsert","last_dispatch_duration_us":10017718,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12534817,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10319605,"encode_duration_us":16947,"operation_id":"0x374e","timeout_ms":10000,"last_server_duration_us":9826220,"total_duration_us":10007228}],"total_count":3}} WARN [2022-01-10 05:41:01,496] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18005145,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":18005145,"total_server_duration_us":38,"operation_id":"0x3579","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":25209673},{"operation_name":"upsert","encode_duration_us":16259,"operation_id":"0x3745","timeout_ms":10000,"total_duration_us":10007935},{"operation_name":"upsert","last_dispatch_duration_us":1776874,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":10294667,"total_server_duration_us":1798477,"encode_duration_us":16995,"operation_id":"0x386e","timeout_ms":10000,"last_server_duration_us":397653,"total_duration_us":10007262},{"operation_name":"upsert","encode_duration_us":16344,"operation_id":"0x3702","timeout_ms":10000,"total_duration_us":10006691},{"operation_name":"upsert","encode_duration_us":16680,"operation_id":"0x37af","timeout_ms":10000,"total_duration_us":10006172},{"operation_name":"upsert","last_dispatch_duration_us":9578755,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":15923553,"total_server_duration_us":1791196,"encode_duration_us":19091,"operation_id":"0x36de","timeout_ms":10000,"last_server_duration_us":838312,"total_duration_us":10004322},{"operation_name":"upsert","last_dispatch_duration_us":880753,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9701614,"total_server_duration_us":1622405,"encode_duration_us":24251,"operation_id":"0x3819","timeout_ms":10000,"last_server_duration_us":191893,"total_duration_us":10004095},{"operation_name":"upsert","last_dispatch_duration_us":9050485,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9664424,"total_server_duration_us":9375115,"encode_duration_us":17161,"operation_id":"0x3718","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":10004065},{"operation_name":"upsert","encode_duration_us":16421,"operation_id":"0x370f","timeout_ms":10000,"total_duration_us":10002124},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":17379,"operation_id":"0x379e","timeout_ms":10000,"total_duration_us":10001624}],"total_count":20}} WARN [2022-01-10 05:41:01,497] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9045481,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":9816948,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":9402881,"encode_duration_us":16088,"operation_id":"0x3680","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10001583},{"operation_name":"upsert","last_dispatch_duration_us":10687080,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12049912,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9585936,"encode_duration_us":18546,"operation_id":"0x37e1","timeout_ms":10000,"last_server_duration_us":9492134,"total_duration_us":10002170},{"operation_name":"upsert","last_dispatch_duration_us":9136956,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17644283,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9056906,"encode_duration_us":22339,"operation_id":"0x35d5","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10006350},{"operation_name":"upsert","last_dispatch_duration_us":11175256,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19106903,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":10185847,"encode_duration_us":16657,"operation_id":"0x364e","timeout_ms":10000,"last_server_duration_us":9868050,"total_duration_us":10010218},{"operation_name":"upsert","last_dispatch_duration_us":17378733,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17378733,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9034183,"encode_duration_us":18432,"operation_id":"0x35ea","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10010108}],"total_count":5}} INFO [2022-01-10 05:41:01,573] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3391812865497076}, :ns_1@172.23.107.227 {:progress 0.2346041055718475}, :ns_1@172.23.105.187 {:progress 0.2346041055718475}, :ns_1@172.23.107.228 {:progress 0.3294117647058824}} INFO [2022-01-10 05:41:01,573] jepsen worker nemesis - couchbase.util Rebalance check count: 216 INFO [2022-01-10 05:41:01,573] jepsen worker nemesis - couchbase.util Rebalance stuck count: 157 WARN [2022-01-10 05:41:01,595] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","encode_duration_us":16595,"operation_id":"0x3892","timeout_ms":10000,"total_duration_us":10008921},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16936,"operation_id":"0x370e","timeout_ms":10000,"total_duration_us":10007979},{"operation_name":"upsert","last_dispatch_duration_us":8077741,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9152441,"total_server_duration_us":1094168,"encode_duration_us":18281,"operation_id":"0x3701","timeout_ms":10000,"last_server_duration_us":106626,"total_duration_us":10006127},{"operation_name":"upsert","last_dispatch_duration_us":852687,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10078261,"total_server_duration_us":2038255,"encode_duration_us":16291,"operation_id":"0x3818","timeout_ms":10000,"last_server_duration_us":20441,"total_duration_us":10002722},{"operation_name":"upsert","last_dispatch_duration_us":1400094,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10825826,"total_server_duration_us":250644,"encode_duration_us":22020,"operation_id":"0x386c","timeout_ms":10000,"last_server_duration_us":51828,"total_duration_us":10001837},{"operation_name":"get","last_dispatch_duration_us":8153887,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8153887,"total_server_duration_us":49,"operation_id":"0x37ac","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":8156291},{"operation_name":"get","last_dispatch_duration_us":7872394,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":7872394,"total_server_duration_us":6432,"operation_id":"0x389c","timeout_ms":30000,"last_server_duration_us":6432,"total_duration_us":7874946},{"operation_name":"get","last_dispatch_duration_us":7845993,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":7845993,"total_server_duration_us":5954,"operation_id":"0x389d","timeout_ms":30000,"last_server_duration_us":5954,"total_duration_us":7848254},{"operation_name":"get","last_dispatch_duration_us":7589150,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":7589150,"total_server_duration_us":27,"operation_id":"0x37d3","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":7590260},{"operation_name":"get","last_dispatch_duration_us":4579536,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":4579536,"total_server_duration_us":23,"operation_id":"0x3864","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":4580672}],"total_count":16}} WARN [2022-01-10 05:41:01,596] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9050485,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":9664424,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":9375115,"encode_duration_us":17161,"operation_id":"0x3718","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":10004065},{"operation_name":"upsert","last_dispatch_duration_us":18071354,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18071354,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9028877,"encode_duration_us":20803,"operation_id":"0x35b7","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10008904},{"operation_name":"upsert","last_dispatch_duration_us":9664129,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16203002,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9764992,"encode_duration_us":16973,"operation_id":"0x36b1","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10010726}],"total_count":3}} WARN [2022-01-10 05:41:01,695] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9043618,"last_local_id":"EFE9906200000004/000000008B42EFB5","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":9995935,"last_local_socket":"172.23.104.79:44292","total_server_duration_us":9622558,"encode_duration_us":16698,"operation_id":"0x35fb","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10003720},{"operation_name":"upsert","last_dispatch_duration_us":18081409,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":27198901,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9324010,"encode_duration_us":19307,"operation_id":"0x33e3","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10006216},{"operation_name":"upsert","last_dispatch_duration_us":17422735,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":24682264,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10097004,"encode_duration_us":17210,"operation_id":"0x3517","timeout_ms":10000,"last_server_duration_us":9295764,"total_duration_us":10008507}],"total_count":3}} WARN [2022-01-10 05:41:01,698] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9739668,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":14069124,"total_server_duration_us":309330,"encode_duration_us":19601,"operation_id":"0x37de","timeout_ms":10000,"last_server_duration_us":67221,"total_duration_us":10010231},{"operation_name":"upsert","last_dispatch_duration_us":1652518,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":6772103,"total_server_duration_us":245112,"encode_duration_us":23917,"operation_id":"0x370d","timeout_ms":10000,"last_server_duration_us":115117,"total_duration_us":10009250},{"operation_name":"upsert","last_dispatch_duration_us":9948379,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":15204025,"total_server_duration_us":384126,"encode_duration_us":16537,"operation_id":"0x37ae","timeout_ms":10000,"last_server_duration_us":130638,"total_duration_us":10008514},{"operation_name":"upsert","last_dispatch_duration_us":2453713,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4038270,"total_server_duration_us":808345,"encode_duration_us":17067,"operation_id":"0x38c8","timeout_ms":10000,"last_server_duration_us":396529,"total_duration_us":10008447},{"operation_name":"upsert","last_dispatch_duration_us":2075029,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11111318,"total_server_duration_us":1774961,"encode_duration_us":16349,"operation_id":"0x373d","timeout_ms":10000,"last_server_duration_us":1040285,"total_duration_us":10006059},{"operation_name":"upsert","last_dispatch_duration_us":1790163,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":2931883,"total_server_duration_us":1124874,"encode_duration_us":16628,"operation_id":"0x3812","timeout_ms":10000,"last_server_duration_us":1051738,"total_duration_us":10005526},{"operation_name":"upsert","last_dispatch_duration_us":9715757,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9715757,"total_server_duration_us":76033,"encode_duration_us":16600,"operation_id":"0x3885","timeout_ms":10000,"last_server_duration_us":76033,"total_duration_us":10005462},{"operation_name":"upsert","last_dispatch_duration_us":10687080,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":12049912,"total_server_duration_us":9585936,"encode_duration_us":18546,"operation_id":"0x37e1","timeout_ms":10000,"last_server_duration_us":9492134,"total_duration_us":10002170},{"operation_name":"upsert","last_dispatch_duration_us":10190522,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":16679627,"total_server_duration_us":893991,"encode_duration_us":16616,"operation_id":"0x370c","timeout_ms":10000,"last_server_duration_us":145698,"total_duration_us":10001877},{"operation_name":"upsert","last_dispatch_duration_us":2690739,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":8626568,"total_server_duration_us":1916802,"encode_duration_us":16305,"operation_id":"0x38bd","timeout_ms":10000,"last_server_duration_us":803491,"total_duration_us":10001844}],"total_count":14}} INFO [2022-01-10 05:41:02,041] jepsen worker 58 - jepsen.util 1858 :info :write [19 13] :RequestTimeoutException INFO [2022-01-10 05:41:02,042] jepsen worker 58 - jepsen.util 1948 :invoke :write [19 47] INFO [2022-01-10 05:41:02,156] jepsen worker 46 - jepsen.util 2026 :info :write [15 36] :RequestTimeoutException INFO [2022-01-10 05:41:02,157] jepsen worker 46 - jepsen.util 2116 :invoke :write [15 8] INFO [2022-01-10 05:41:02,190] jepsen worker 6 - jepsen.util 1806 :info :write [2 34] :RequestTimeoutException INFO [2022-01-10 05:41:02,190] jepsen worker 6 - jepsen.util 1896 :invoke :read [2 nil] INFO [2022-01-10 05:41:02,212] jepsen worker 28 - jepsen.util 1738 :info :write [9 11] :RequestTimeoutException INFO [2022-01-10 05:41:02,212] jepsen worker 28 - jepsen.util 1828 :invoke :read [9 nil] INFO [2022-01-10 05:41:02,353] jepsen worker 67 - jepsen.util 1777 :ok :read [22 41] INFO [2022-01-10 05:41:02,353] jepsen worker 67 - jepsen.util 1777 :invoke :write [22 10] INFO [2022-01-10 05:41:02,358] jepsen worker 86 - jepsen.util 1616 :ok :read [28 13] INFO [2022-01-10 05:41:02,359] jepsen worker 86 - jepsen.util 1616 :invoke :write [28 38] INFO [2022-01-10 05:41:02,375] jepsen worker 63 - jepsen.util 1683 :info :write [21 5] :RequestTimeoutException INFO [2022-01-10 05:41:02,376] jepsen worker 63 - jepsen.util 1773 :invoke :read [21 nil] INFO [2022-01-10 05:41:02,387] jepsen worker 40 - jepsen.util 1480 :ok :read [13 15] INFO [2022-01-10 05:41:02,388] jepsen worker 40 - jepsen.util 1480 :invoke :read [13 nil] INFO [2022-01-10 05:41:02,578] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3450292397660819}, :ns_1@172.23.107.227 {:progress 0.2375366568914956}, :ns_1@172.23.105.187 {:progress 0.2375366568914956}, :ns_1@172.23.107.228 {:progress 0.3352941176470589}} INFO [2022-01-10 05:41:02,578] jepsen worker nemesis - couchbase.util Rebalance check count: 217 INFO [2022-01-10 05:41:02,578] jepsen worker nemesis - couchbase.util Rebalance stuck count: 157 INFO [2022-01-10 05:41:02,586] jepsen worker 2 - jepsen.util 1802 :info :write [0 22] :RequestTimeoutException INFO [2022-01-10 05:41:02,587] jepsen worker 2 - jepsen.util 1892 :invoke :write [0 25] INFO [2022-01-10 05:41:02,616] jepsen worker 55 - jepsen.util 2035 :ok :read [18 14] INFO [2022-01-10 05:41:02,617] jepsen worker 55 - jepsen.util 2035 :invoke :read [18 nil] INFO [2022-01-10 05:41:02,627] jepsen worker 69 - jepsen.util 1869 :ok :read [23 26] INFO [2022-01-10 05:41:02,628] jepsen worker 69 - jepsen.util 1869 :invoke :read [23 nil] INFO [2022-01-10 05:41:02,711] jepsen worker 26 - jepsen.util 1556 :info :write [8 39] :RequestTimeoutException INFO [2022-01-10 05:41:02,711] jepsen worker 26 - jepsen.util 1646 :invoke :read [8 nil] INFO [2022-01-10 05:41:02,815] jepsen worker 70 - jepsen.util 2140 :ok :read [23 26] INFO [2022-01-10 05:41:02,815] jepsen worker 70 - jepsen.util 2140 :invoke :read [23 nil] INFO [2022-01-10 05:41:02,819] jepsen worker 71 - jepsen.util 1691 :ok :read [23 26] INFO [2022-01-10 05:41:02,820] jepsen worker 71 - jepsen.util 1691 :invoke :read [23 nil] INFO [2022-01-10 05:41:02,956] jepsen worker 9 - jepsen.util 1989 :info :write [3 34] :RequestTimeoutException INFO [2022-01-10 05:41:02,957] jepsen worker 9 - jepsen.util 2079 :invoke :write [3 13] INFO [2022-01-10 05:41:03,000] jepsen worker 65 - jepsen.util 1685 :info :write [21 16] :RequestTimeoutException INFO [2022-01-10 05:41:03,000] jepsen worker 65 - jepsen.util 1775 :invoke :read [21 nil] INFO [2022-01-10 05:41:03,044] jepsen worker 23 - jepsen.util 1913 :info :write [7 31] :RequestTimeoutException INFO [2022-01-10 05:41:03,045] jepsen worker 23 - jepsen.util 2003 :invoke :write [7 8] INFO [2022-01-10 05:41:03,075] jepsen worker 33 - jepsen.util 1833 :info :write [11 15] :RequestTimeoutException INFO [2022-01-10 05:41:03,076] jepsen worker 33 - jepsen.util 1923 :invoke :read [11 nil] INFO [2022-01-10 05:41:03,220] jepsen worker 55 - jepsen.util 2035 :ok :read [18 14] INFO [2022-01-10 05:41:03,220] jepsen worker 55 - jepsen.util 2035 :invoke :read [18 nil] INFO [2022-01-10 05:41:03,225] jepsen worker 69 - jepsen.util 1869 :ok :read [23 26] INFO [2022-01-10 05:41:03,225] jepsen worker 69 - jepsen.util 1869 :invoke :read [23 nil] INFO [2022-01-10 05:41:03,351] jepsen worker 42 - jepsen.util 1752 :info :write [14 21] :RequestTimeoutException INFO [2022-01-10 05:41:03,356] jepsen worker 88 - jepsen.util 1798 :info :write [29 29] :RequestTimeoutException INFO [2022-01-10 05:41:03,370] jepsen worker 28 - jepsen.util 1828 :ok :read [9 22] INFO [2022-01-10 05:41:04,135] jepsen worker 42 - jepsen.util 1842 :invoke :write [14 41] INFO [2022-01-10 05:41:04,135] jepsen worker 88 - jepsen.util 1888 :invoke :read [29 nil] INFO [2022-01-10 05:41:04,135] jepsen worker 28 - jepsen.util 1828 :invoke :read [9 nil] INFO [2022-01-10 05:41:03,382] jepsen worker 14 - jepsen.util 1904 :info :write [4 27] :RequestTimeoutException INFO [2022-01-10 05:41:03,406] jepsen worker 82 - jepsen.util 1702 :ok :read [27 30] INFO [2022-01-10 05:41:04,135] jepsen worker 14 - jepsen.util 1994 :invoke :read [4 nil] INFO [2022-01-10 05:41:03,420] jepsen worker 12 - jepsen.util 1542 :info :write [4 38] :RequestTimeoutException INFO [2022-01-10 05:41:03,539] jepsen worker 17 - jepsen.util 1727 :info :write [5 27] :RequestTimeoutException INFO [2022-01-10 05:41:03,556] jepsen worker 51 - jepsen.util 1851 :info :write [17 42] :RequestTimeoutException INFO [2022-01-10 05:41:04,135] jepsen worker 17 - jepsen.util 1817 :invoke :write [5 29] INFO [2022-01-10 05:41:04,135] jepsen worker 51 - jepsen.util 1941 :invoke :read [17 nil] INFO [2022-01-10 05:41:03,582] jepsen worker 36 - jepsen.util 1476 :info :write [12 16] :RequestTimeoutException INFO [2022-01-10 05:41:04,135] jepsen worker 82 - jepsen.util 1702 :invoke :read [27 nil] INFO [2022-01-10 05:41:03,585] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3450292397660819}, :ns_1@172.23.107.227 {:progress 0.2375366568914956}, :ns_1@172.23.105.187 {:progress 0.2375366568914956}, :ns_1@172.23.107.228 {:progress 0.3352941176470589}} INFO [2022-01-10 05:41:04,136] jepsen worker 36 - jepsen.util 1566 :invoke :write [12 44] INFO [2022-01-10 05:41:04,136] jepsen worker nemesis - couchbase.util Rebalance check count: 218 INFO [2022-01-10 05:41:04,136] jepsen worker nemesis - couchbase.util Rebalance stuck count: 158 INFO [2022-01-10 05:41:03,611] jepsen worker 27 - jepsen.util 1737 :ok :read [9 22] INFO [2022-01-10 05:41:04,136] jepsen worker 27 - jepsen.util 1737 :invoke :read [9 nil] INFO [2022-01-10 05:41:03,631] jepsen worker 3 - jepsen.util 1893 :info :write [1 41] :RequestTimeoutException INFO [2022-01-10 05:41:03,722] jepsen worker 37 - jepsen.util 1837 :info :write [12 23] :RequestTimeoutException INFO [2022-01-10 05:41:03,789] jepsen worker 80 - jepsen.util 1880 :info :write [26 28] :RequestTimeoutException INFO [2022-01-10 05:41:04,145] jepsen worker 37 - jepsen.util 1927 :invoke :write [12 22] INFO [2022-01-10 05:41:04,145] jepsen worker 3 - jepsen.util 1983 :invoke :read [1 nil] INFO [2022-01-10 05:41:04,145] jepsen worker 80 - jepsen.util 1970 :invoke :read [26 nil] INFO [2022-01-10 05:41:03,883] jepsen worker 75 - jepsen.util 1335 :ok :read [25 :nil] INFO [2022-01-10 05:41:04,145] jepsen worker 75 - jepsen.util 1335 :invoke :read [25 nil] INFO [2022-01-10 05:41:03,895] jepsen worker 48 - jepsen.util 1488 :info :write [16 47] :RequestTimeoutException INFO [2022-01-10 05:41:04,145] jepsen worker 48 - jepsen.util 1578 :invoke :read [16 nil] INFO [2022-01-10 05:41:03,972] jepsen worker 8 - jepsen.util 1898 :info :write [2 35] :RequestTimeoutException INFO [2022-01-10 05:41:04,135] jepsen worker 12 - jepsen.util 1632 :invoke :write [4 15] INFO [2022-01-10 05:41:04,146] jepsen worker 8 - jepsen.util 1988 :invoke :write [2 13] INFO [2022-01-10 05:41:04,222] jepsen worker 28 - jepsen.util 1828 :ok :read [9 22] INFO [2022-01-10 05:41:04,223] jepsen worker 28 - jepsen.util 1828 :invoke :read [9 nil] INFO [2022-01-10 05:41:04,389] jepsen worker 55 - jepsen.util 2035 :ok :read [18 14] INFO [2022-01-10 05:41:04,390] jepsen worker 55 - jepsen.util 2035 :invoke :write [18 22] INFO [2022-01-10 05:41:04,407] jepsen worker 69 - jepsen.util 1869 :ok :read [23 26] INFO [2022-01-10 05:41:04,407] jepsen worker 69 - jepsen.util 1869 :invoke :read [23 nil] INFO [2022-01-10 05:41:04,579] jepsen worker 28 - jepsen.util 1828 :ok :read [9 22] INFO [2022-01-10 05:41:04,580] jepsen worker 28 - jepsen.util 1828 :invoke :read [9 nil] INFO [2022-01-10 05:41:04,806] jepsen worker 82 - jepsen.util 1702 :ok :read [27 30] INFO [2022-01-10 05:41:04,806] jepsen worker 82 - jepsen.util 1702 :invoke :write [27 21] INFO [2022-01-10 05:41:04,849] jepsen worker 40 - jepsen.util 1480 :ok :read [13 15] INFO [2022-01-10 05:41:04,849] jepsen worker 40 - jepsen.util 1480 :invoke :write [13 7] INFO [2022-01-10 05:41:04,894] jepsen worker 27 - jepsen.util 1737 :ok :read [9 22] INFO [2022-01-10 05:41:04,895] jepsen worker 27 - jepsen.util 1737 :invoke :write [9 6] INFO [2022-01-10 05:41:04,936] jepsen worker 54 - jepsen.util 1854 :ok :read [18 14] INFO [2022-01-10 05:41:04,936] jepsen worker 54 - jepsen.util 1854 :invoke :read [18 nil] INFO [2022-01-10 05:41:05,081] jepsen worker 28 - jepsen.util 1828 :ok :read [9 22] INFO [2022-01-10 05:41:05,081] jepsen worker 28 - jepsen.util 1828 :invoke :read [9 nil] INFO [2022-01-10 05:41:05,143] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3450292397660819}, :ns_1@172.23.107.227 {:progress 0.2375366568914956}, :ns_1@172.23.105.187 {:progress 0.2375366568914956}, :ns_1@172.23.107.228 {:progress 0.3352941176470589}} INFO [2022-01-10 05:41:05,143] jepsen worker nemesis - couchbase.util Rebalance check count: 219 INFO [2022-01-10 05:41:05,143] jepsen worker nemesis - couchbase.util Rebalance stuck count: 159 INFO [2022-01-10 05:41:05,152] jepsen worker 11 - jepsen.util 1721 :ok :read [3 2] INFO [2022-01-10 05:41:05,152] jepsen worker 11 - jepsen.util 1721 :invoke :read [3 nil] INFO [2022-01-10 05:41:05,163] jepsen worker 71 - jepsen.util 1691 :ok :read [23 26] INFO [2022-01-10 05:41:05,164] jepsen worker 71 - jepsen.util 1691 :invoke :read [23 nil] INFO [2022-01-10 05:41:05,164] jepsen worker 70 - jepsen.util 2140 :ok :read [23 26] INFO [2022-01-10 05:41:05,165] jepsen worker 70 - jepsen.util 2140 :invoke :write [23 14] INFO [2022-01-10 05:41:05,259] jepsen worker 74 - jepsen.util 2324 :info :write [24 23] :RequestTimeoutException INFO [2022-01-10 05:41:05,260] jepsen worker 74 - jepsen.util 2414 :invoke :write [24 17] INFO [2022-01-10 05:41:05,341] jepsen worker 63 - jepsen.util 1773 :ok :read [21 7] INFO [2022-01-10 05:41:05,342] jepsen worker 63 - jepsen.util 1773 :invoke :read [21 nil] INFO [2022-01-10 05:41:05,358] jepsen worker 11 - jepsen.util 1721 :ok :read [3 2] INFO [2022-01-10 05:41:05,358] jepsen worker 11 - jepsen.util 1721 :invoke :read [3 nil] INFO [2022-01-10 05:41:05,366] jepsen worker 80 - jepsen.util 1970 :ok :read [26 10] INFO [2022-01-10 05:41:05,369] jepsen worker 80 - jepsen.util 1970 :invoke :write [26 41] INFO [2022-01-10 05:41:05,371] jepsen worker 71 - jepsen.util 1691 :ok :read [23 26] INFO [2022-01-10 05:41:05,372] jepsen worker 71 - jepsen.util 1691 :invoke :read [23 nil] INFO [2022-01-10 05:41:05,384] jepsen worker 48 - jepsen.util 1578 :ok :read [16 38] INFO [2022-01-10 05:41:05,385] jepsen worker 48 - jepsen.util 1578 :invoke :write [16 19] INFO [2022-01-10 05:41:05,402] jepsen worker 29 - jepsen.util 1829 :info :write [9 3] :RequestTimeoutException INFO [2022-01-10 05:41:05,402] jepsen worker 29 - jepsen.util 1919 :invoke :write [9 49] INFO [2022-01-10 05:41:05,445] jepsen worker 31 - jepsen.util 2101 :info :write [10 48] :RequestTimeoutException INFO [2022-01-10 05:41:05,446] jepsen worker 31 - jepsen.util 2191 :invoke :read [10 nil] INFO [2022-01-10 05:41:05,496] jepsen worker 5 - jepsen.util 1805 :info :write [1 47] :RequestTimeoutException INFO [2022-01-10 05:41:05,497] jepsen worker 5 - jepsen.util 1895 :invoke :write [1 20] INFO [2022-01-10 05:41:05,721] jepsen worker 61 - jepsen.util 1771 :info :write [20 46] :RequestTimeoutException INFO [2022-01-10 05:41:05,722] jepsen worker 61 - jepsen.util 1861 :invoke :write [20 6] INFO [2022-01-10 05:41:05,755] jepsen worker 30 - jepsen.util 1470 :info :write [10 3] :RequestTimeoutException INFO [2022-01-10 05:41:05,755] jepsen worker 30 - jepsen.util 1560 :invoke :write [10 27] INFO [2022-01-10 05:41:05,771] jepsen worker 69 - jepsen.util 1869 :ok :read [23 26] INFO [2022-01-10 05:41:05,772] jepsen worker 69 - jepsen.util 1869 :invoke :write [23 47] INFO [2022-01-10 05:41:05,781] jepsen worker 11 - jepsen.util 1721 :ok :read [3 2] INFO [2022-01-10 05:41:05,781] jepsen worker 11 - jepsen.util 1721 :invoke :read [3 nil] INFO [2022-01-10 05:41:05,786] jepsen worker 56 - jepsen.util 1946 :info :write [18 15] :RequestTimeoutException INFO [2022-01-10 05:41:05,786] jepsen worker 56 - jepsen.util 2036 :invoke :read [18 nil] INFO [2022-01-10 05:41:05,793] jepsen worker 71 - jepsen.util 1691 :ok :read [23 26] INFO [2022-01-10 05:41:05,794] jepsen worker 71 - jepsen.util 1691 :invoke :read [23 nil] INFO [2022-01-10 05:41:05,840] jepsen worker 39 - jepsen.util 2109 :info :write [13 12] :RequestTimeoutException INFO [2022-01-10 05:41:05,841] jepsen worker 39 - jepsen.util 2199 :invoke :write [13 1] INFO [2022-01-10 05:41:06,047] jepsen worker 22 - jepsen.util 1642 :ok :read [7 :nil] INFO [2022-01-10 05:41:06,048] jepsen worker 22 - jepsen.util 1642 :invoke :read [7 nil] INFO [2022-01-10 05:41:06,062] jepsen worker 72 - jepsen.util 1872 :info :write [24 30] :RequestTimeoutException INFO [2022-01-10 05:41:06,062] jepsen worker 72 - jepsen.util 1962 :invoke :read [24 nil] INFO [2022-01-10 05:41:06,065] jepsen worker 14 - jepsen.util 1994 :ok :read [4 3] INFO [2022-01-10 05:41:06,065] jepsen worker 14 - jepsen.util 1994 :invoke :write [4 29] INFO [2022-01-10 05:41:06,149] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3450292397660819}, :ns_1@172.23.107.227 {:progress 0.2375366568914956}, :ns_1@172.23.105.187 {:progress 0.2375366568914956}, :ns_1@172.23.107.228 {:progress 0.3352941176470589}} INFO [2022-01-10 05:41:06,149] jepsen worker nemesis - couchbase.util Rebalance check count: 220 INFO [2022-01-10 05:41:06,149] jepsen worker nemesis - couchbase.util Rebalance stuck count: 160 INFO [2022-01-10 05:41:06,216] jepsen worker 7 - jepsen.util 1897 :info :write [2 28] :RequestTimeoutException INFO [2022-01-10 05:41:06,217] jepsen worker 7 - jepsen.util 1987 :invoke :read [2 nil] INFO [2022-01-10 05:41:06,231] jepsen worker 45 - jepsen.util 1935 :info :write [15 9] :RequestTimeoutException INFO [2022-01-10 05:41:06,232] jepsen worker 45 - jepsen.util 2025 :invoke :write [15 16] INFO [2022-01-10 05:41:06,233] jepsen worker 11 - jepsen.util 1721 :ok :read [3 2] INFO [2022-01-10 05:41:06,234] jepsen worker 11 - jepsen.util 1721 :invoke :write [3 30] INFO [2022-01-10 05:41:06,349] jepsen worker 4 - jepsen.util 1804 :info :write [1 34] :RequestTimeoutException INFO [2022-01-10 05:41:06,350] jepsen worker 4 - jepsen.util 1894 :invoke :write [1 9] INFO [2022-01-10 05:41:06,454] jepsen worker 73 - jepsen.util 1873 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:06,455] jepsen worker 73 - jepsen.util 1873 :invoke :read [24 nil] INFO [2022-01-10 05:41:06,529] jepsen worker 88 - jepsen.util 1888 :ok :read [29 0] INFO [2022-01-10 05:41:06,530] jepsen worker 88 - jepsen.util 1888 :invoke :read [29 nil] INFO [2022-01-10 05:41:06,555] jepsen worker 88 - jepsen.util 1888 :ok :read [29 0] INFO [2022-01-10 05:41:06,555] jepsen worker 88 - jepsen.util 1888 :invoke :write [29 16] INFO [2022-01-10 05:41:06,731] jepsen worker 51 - jepsen.util 1941 :ok :read [17 41] INFO [2022-01-10 05:41:06,732] jepsen worker 51 - jepsen.util 1941 :invoke :read [17 nil] INFO [2022-01-10 05:41:06,898] jepsen worker 43 - jepsen.util 1843 :ok :read [14 14] INFO [2022-01-10 05:41:06,898] jepsen worker 43 - jepsen.util 1843 :invoke :read [14 nil] INFO [2022-01-10 05:41:06,985] jepsen worker 84 - jepsen.util 1974 :info :write [28 49] :RequestTimeoutException INFO [2022-01-10 05:41:06,986] jepsen worker 84 - jepsen.util 2064 :invoke :write [28 23] INFO [2022-01-10 05:41:06,996] jepsen worker 56 - jepsen.util 2036 :ok :read [18 14] INFO [2022-01-10 05:41:06,997] jepsen worker 56 - jepsen.util 2036 :invoke :read [18 nil] INFO [2022-01-10 05:41:07,031] jepsen worker 56 - jepsen.util 2036 :ok :read [18 14] INFO [2022-01-10 05:41:07,032] jepsen worker 56 - jepsen.util 2036 :invoke :write [18 48] INFO [2022-01-10 05:41:07,091] jepsen worker 53 - jepsen.util 1943 :info :write [17 44] :RequestTimeoutException INFO [2022-01-10 05:41:07,092] jepsen worker 53 - jepsen.util 2033 :invoke :write [17 30] INFO [2022-01-10 05:41:07,155] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2375366568914956}, :ns_1@172.23.105.187 {:progress 0.2375366568914956}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:07,156] jepsen worker nemesis - couchbase.util Rebalance check count: 221 INFO [2022-01-10 05:41:07,156] jepsen worker nemesis - couchbase.util Rebalance stuck count: 160 INFO [2022-01-10 05:41:07,495] jepsen worker 15 - jepsen.util 1545 :info :write [5 25] :SyncWriteAmbiguous INFO [2022-01-10 05:41:07,499] jepsen worker 15 - jepsen.util 1635 :invoke :read [5 nil] INFO [2022-01-10 05:41:07,520] jepsen worker 85 - jepsen.util 2065 :info :write [28 1] :RequestTimeoutException INFO [2022-01-10 05:41:07,521] jepsen worker 85 - jepsen.util 2155 :invoke :write [28 27] INFO [2022-01-10 05:41:07,551] jepsen worker 66 - jepsen.util 1596 :info :write [22 24] :RequestTimeoutException INFO [2022-01-10 05:41:07,552] jepsen worker 66 - jepsen.util 1686 :invoke :read [22 nil] INFO [2022-01-10 05:41:07,636] jepsen worker 13 - jepsen.util 1633 :info :write [4 5] :RequestTimeoutException INFO [2022-01-10 05:41:07,637] jepsen worker 13 - jepsen.util 1723 :invoke :write [4 21] INFO [2022-01-10 05:41:07,676] jepsen worker 0 - jepsen.util 1710 :info :write [0 25] :RequestTimeoutException INFO [2022-01-10 05:41:07,677] jepsen worker 0 - jepsen.util 1800 :invoke :write [0 15] INFO [2022-01-10 05:41:07,721] jepsen worker 38 - jepsen.util 1928 :info :write [12 42] :RequestTimeoutException INFO [2022-01-10 05:41:07,722] jepsen worker 38 - jepsen.util 2018 :invoke :write [12 28] INFO [2022-01-10 05:41:07,786] jepsen worker 81 - jepsen.util 1791 :info :write [27 3] :RequestTimeoutException INFO [2022-01-10 05:41:07,787] jepsen worker 81 - jepsen.util 1881 :invoke :write [27 11] INFO [2022-01-10 05:41:07,817] jepsen worker 16 - jepsen.util 2176 :info :write [5 38] :RequestTimeoutException INFO [2022-01-10 05:41:07,817] jepsen worker 16 - jepsen.util 2266 :invoke :write [5 22] INFO [2022-01-10 05:41:07,839] jepsen worker 51 - jepsen.util 1941 :ok :read [17 41] INFO [2022-01-10 05:41:07,840] jepsen worker 51 - jepsen.util 1941 :invoke :read [17 nil] INFO [2022-01-10 05:41:07,883] jepsen worker 75 - jepsen.util 1335 :ok :read [25 :nil] INFO [2022-01-10 05:41:07,888] jepsen worker 75 - jepsen.util 1335 :invoke :read [25 nil] INFO [2022-01-10 05:41:07,899] jepsen worker 33 - jepsen.util 1923 :ok :read [11 21] INFO [2022-01-10 05:41:07,900] jepsen worker 33 - jepsen.util 1923 :invoke :write [11 38] INFO [2022-01-10 05:41:07,946] jepsen worker 1 - jepsen.util 1891 :info :write [0 47] :RequestTimeoutException INFO [2022-01-10 05:41:07,947] jepsen worker 1 - jepsen.util 1981 :invoke :read [0 nil] INFO [2022-01-10 05:41:08,161] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:08,161] jepsen worker nemesis - couchbase.util Rebalance check count: 222 INFO [2022-01-10 05:41:08,161] jepsen worker nemesis - couchbase.util Rebalance stuck count: 160 INFO [2022-01-10 05:41:08,351] jepsen worker 59 - jepsen.util 1949 :ok :read [19 31] INFO [2022-01-10 05:41:08,351] jepsen worker 59 - jepsen.util 1949 :invoke :read [19 nil] INFO [2022-01-10 05:41:08,489] jepsen worker 51 - jepsen.util 1941 :ok :read [17 41] INFO [2022-01-10 05:41:08,520] jepsen worker 51 - jepsen.util 1941 :invoke :read [17 nil] INFO [2022-01-10 05:41:08,534] jepsen worker 32 - jepsen.util 1832 :info :write [10 9] :RequestTimeoutException INFO [2022-01-10 05:41:08,535] jepsen worker 32 - jepsen.util 1922 :invoke :write [10 21] INFO [2022-01-10 05:41:08,549] jepsen worker 59 - jepsen.util 1949 :ok :read [19 31] INFO [2022-01-10 05:41:08,549] jepsen worker 59 - jepsen.util 1949 :invoke :read [19 nil] INFO [2022-01-10 05:41:08,586] jepsen worker 59 - jepsen.util 1949 :ok :read [19 31] INFO [2022-01-10 05:41:08,586] jepsen worker 59 - jepsen.util 1949 :invoke :read [19 nil] INFO [2022-01-10 05:41:08,634] jepsen worker 59 - jepsen.util 1949 :ok :read [19 31] INFO [2022-01-10 05:41:08,634] jepsen worker 59 - jepsen.util 1949 :invoke :write [19 9] INFO [2022-01-10 05:41:09,120] jepsen worker 51 - jepsen.util 1941 :ok :read [17 41] INFO [2022-01-10 05:41:09,121] jepsen worker 51 - jepsen.util 1941 :invoke :read [17 nil] INFO [2022-01-10 05:41:09,167] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:09,167] jepsen worker nemesis - couchbase.util Rebalance check count: 223 INFO [2022-01-10 05:41:09,167] jepsen worker nemesis - couchbase.util Rebalance stuck count: 161 INFO [2022-01-10 05:41:09,332] jepsen worker 78 - jepsen.util 1878 :info :write [26 18] :RequestTimeoutException INFO [2022-01-10 05:41:09,332] jepsen worker 78 - jepsen.util 1968 :invoke :read [26 nil] INFO [2022-01-10 05:41:09,458] jepsen worker 21 - jepsen.util 2001 :ok :read [7 :nil] INFO [2022-01-10 05:41:09,458] jepsen worker 21 - jepsen.util 2001 :invoke :read [7 nil] INFO [2022-01-10 05:41:09,471] jepsen worker 34 - jepsen.util 1924 :ok :read [11 21] INFO [2022-01-10 05:41:09,471] jepsen worker 34 - jepsen.util 1924 :invoke :read [11 nil] INFO [2022-01-10 05:41:09,621] jepsen worker 18 - jepsen.util 1368 :info :write [6 20] :RequestTimeoutException INFO [2022-01-10 05:41:09,622] jepsen worker 18 - jepsen.util 1458 :invoke :write [6 32] INFO [2022-01-10 05:41:09,741] jepsen worker 52 - jepsen.util 2122 :info :write [17 16] :RequestTimeoutException INFO [2022-01-10 05:41:09,742] jepsen worker 52 - jepsen.util 2212 :invoke :read [17 nil] INFO [2022-01-10 05:41:09,754] jepsen worker 51 - jepsen.util 1941 :ok :read [17 41] INFO [2022-01-10 05:41:09,757] jepsen worker 51 - jepsen.util 1941 :invoke :read [17 nil] INFO [2022-01-10 05:41:09,780] jepsen worker 78 - jepsen.util 1968 :ok :read [26 10] INFO [2022-01-10 05:41:09,780] jepsen worker 78 - jepsen.util 1968 :invoke :write [26 11] INFO [2022-01-10 05:41:09,794] jepsen worker 60 - jepsen.util 2220 :info :write [20 24] :RequestTimeoutException INFO [2022-01-10 05:41:09,795] jepsen worker 60 - jepsen.util 2310 :invoke :read [20 nil] WARN [2022-01-10 05:41:09,807] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18058305,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":27144785,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9044571,"encode_duration_us":17242,"operation_id":"0x3560","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10008679}],"total_count":1}} WARN [2022-01-10 05:41:09,907] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":4112252,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":13674622,"total_server_duration_us":281278,"encode_duration_us":16842,"operation_id":"0x38c7","timeout_ms":10000,"last_server_duration_us":193546,"total_duration_us":10009976},{"operation_name":"upsert","last_dispatch_duration_us":2367459,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":4112131,"total_server_duration_us":752238,"encode_duration_us":16916,"operation_id":"0x3921","timeout_ms":10000,"last_server_duration_us":360471,"total_duration_us":10009014},{"operation_name":"upsert","last_dispatch_duration_us":3341402,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10143883,"total_server_duration_us":1056210,"encode_duration_us":16442,"operation_id":"0x39a1","timeout_ms":10000,"last_server_duration_us":384532,"total_duration_us":10005926},{"operation_name":"upsert","last_dispatch_duration_us":9378133,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13757475,"total_server_duration_us":1853345,"encode_duration_us":16297,"operation_id":"0x3959","timeout_ms":10000,"last_server_duration_us":129589,"total_duration_us":10003717},{"operation_name":"upsert","last_dispatch_duration_us":9185642,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14346534,"total_server_duration_us":1529815,"encode_duration_us":16926,"operation_id":"0x389b","timeout_ms":10000,"last_server_duration_us":85157,"total_duration_us":10002433},{"operation_name":"upsert","last_dispatch_duration_us":9278336,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11012046,"total_server_duration_us":462086,"encode_duration_us":16334,"operation_id":"0x39df","timeout_ms":10000,"last_server_duration_us":123713,"total_duration_us":10001807},{"operation_name":"upsert","last_dispatch_duration_us":2679884,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":12086216,"total_server_duration_us":1334721,"encode_duration_us":16384,"operation_id":"0x38f7","timeout_ms":10000,"last_server_duration_us":171744,"total_duration_us":10001397},{"operation_name":"upsert","last_dispatch_duration_us":1538125,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2997934,"total_server_duration_us":780465,"encode_duration_us":19295,"operation_id":"0x3920","timeout_ms":10000,"last_server_duration_us":389256,"total_duration_us":10001289},{"operation_name":"upsert","last_dispatch_duration_us":3928301,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":6310781,"total_server_duration_us":390918,"encode_duration_us":16343,"operation_id":"0x3a20","timeout_ms":10000,"last_server_duration_us":331330,"total_duration_us":10001151},{"operation_name":"upsert","last_dispatch_duration_us":1177772,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":4417960,"total_server_duration_us":1585294,"encode_duration_us":17023,"operation_id":"0x3a67","timeout_ms":10000,"last_server_duration_us":392047,"total_duration_us":10001051}],"total_count":21}} INFO [2022-01-10 05:41:10,082] jepsen worker 49 - jepsen.util 1579 :info :write [16 20] :RequestTimeoutException INFO [2022-01-10 05:41:10,083] jepsen worker 49 - jepsen.util 1669 :invoke :write [16 1] INFO [2022-01-10 05:41:10,091] jepsen worker 83 - jepsen.util 2063 :info :write [27 38] :RequestTimeoutException INFO [2022-01-10 05:41:10,092] jepsen worker 83 - jepsen.util 2153 :invoke :write [27 20] INFO [2022-01-10 05:41:10,092] jepsen worker 24 - jepsen.util 1824 :ok :read [8 27] INFO [2022-01-10 05:41:10,095] jepsen worker 24 - jepsen.util 1824 :invoke :read [8 nil] INFO [2022-01-10 05:41:10,114] jepsen worker 64 - jepsen.util 1954 :ok :read [21 7] INFO [2022-01-10 05:41:10,115] jepsen worker 64 - jepsen.util 1954 :invoke :write [21 15] INFO [2022-01-10 05:41:10,116] jepsen worker 68 - jepsen.util 1778 :info :write [22 48] :RequestTimeoutException INFO [2022-01-10 05:41:10,117] jepsen worker 68 - jepsen.util 1868 :invoke :write [22 30] INFO [2022-01-10 05:41:10,173] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:10,173] jepsen worker nemesis - couchbase.util Rebalance check count: 224 INFO [2022-01-10 05:41:10,173] jepsen worker nemesis - couchbase.util Rebalance stuck count: 162 INFO [2022-01-10 05:41:10,289] jepsen worker 52 - jepsen.util 2212 :ok :read [17 41] INFO [2022-01-10 05:41:10,289] jepsen worker 52 - jepsen.util 2212 :invoke :read [17 nil] INFO [2022-01-10 05:41:10,595] jepsen worker 87 - jepsen.util 1977 :info :write [29 30] :RequestTimeoutException INFO [2022-01-10 05:41:10,595] jepsen worker 87 - jepsen.util 2067 :invoke :read [29 nil] INFO [2022-01-10 05:41:10,626] jepsen worker 87 - jepsen.util 2067 :ok :read [29 0] INFO [2022-01-10 05:41:10,627] jepsen worker 87 - jepsen.util 2067 :invoke :write [29 33] INFO [2022-01-10 05:41:10,652] jepsen worker 50 - jepsen.util 2030 :info :write [16 15] :RequestTimeoutException INFO [2022-01-10 05:41:10,653] jepsen worker 50 - jepsen.util 2120 :invoke :write [16 39] INFO [2022-01-10 05:41:10,672] jepsen worker 41 - jepsen.util 1571 :info :write [13 15] :RequestTimeoutException INFO [2022-01-10 05:41:10,672] jepsen worker 41 - jepsen.util 1661 :invoke :read [13 nil] INFO [2022-01-10 05:41:10,675] jepsen worker 51 - jepsen.util 1941 :ok :read [17 41] INFO [2022-01-10 05:41:10,676] jepsen worker 51 - jepsen.util 1941 :invoke :write [17 33] INFO [2022-01-10 05:41:10,715] jepsen worker 89 - jepsen.util 1979 :info :write [29 40] :RequestTimeoutException INFO [2022-01-10 05:41:10,716] jepsen worker 89 - jepsen.util 2069 :invoke :write [29 4] INFO [2022-01-10 05:41:10,965] jepsen worker 52 - jepsen.util 2212 :ok :read [17 41] INFO [2022-01-10 05:41:10,965] jepsen worker 52 - jepsen.util 2212 :invoke :read [17 nil] INFO [2022-01-10 05:41:10,996] jepsen worker 79 - jepsen.util 1969 :info :write [26 6] :RequestTimeoutException INFO [2022-01-10 05:41:10,997] jepsen worker 79 - jepsen.util 2059 :invoke :write [26 25] INFO [2022-01-10 05:41:11,066] jepsen worker 41 - jepsen.util 1661 :ok :read [13 15] INFO [2022-01-10 05:41:11,066] jepsen worker 41 - jepsen.util 1661 :invoke :read [13 nil] INFO [2022-01-10 05:41:11,092] jepsen worker 41 - jepsen.util 1661 :ok :read [13 15] INFO [2022-01-10 05:41:11,092] jepsen worker 41 - jepsen.util 1661 :invoke :write [13 31] INFO [2022-01-10 05:41:11,113] jepsen worker 63 - jepsen.util 1773 :ok :read [21 7] INFO [2022-01-10 05:41:11,114] jepsen worker 63 - jepsen.util 1773 :invoke :read [21 nil] INFO [2022-01-10 05:41:11,178] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:11,178] jepsen worker nemesis - couchbase.util Rebalance check count: 225 INFO [2022-01-10 05:41:11,178] jepsen worker nemesis - couchbase.util Rebalance stuck count: 163 INFO [2022-01-10 05:41:11,220] jepsen worker 76 - jepsen.util 1426 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:11,222] jepsen worker 20 - jepsen.util 1640 :fail :read [6 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:11,222] jepsen worker 25 - jepsen.util 1825 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:11,223] jepsen worker 77 - jepsen.util 2147 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:11,222] jepsen worker 76 - jepsen.util 1426 :invoke :read [25 nil] INFO [2022-01-10 05:41:11,223] jepsen worker 20 - jepsen.util 1640 :invoke :read [6 nil] INFO [2022-01-10 05:41:11,225] jepsen worker 77 - jepsen.util 2147 :invoke :write [25 1] INFO [2022-01-10 05:41:11,225] jepsen worker 25 - jepsen.util 1825 :invoke :write [8 3] INFO [2022-01-10 05:41:11,323] jepsen worker 52 - jepsen.util 2212 :ok :read [17 41] INFO [2022-01-10 05:41:11,324] jepsen worker 52 - jepsen.util 2212 :invoke :read [17 nil] WARN [2022-01-10 05:41:11,411] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16771707,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16834390,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9661218,"encode_duration_us":17336,"operation_id":"0x37bc","timeout_ms":10000,"last_server_duration_us":9619292,"total_duration_us":10002768}],"total_count":1}} WARN [2022-01-10 05:41:11,411] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15682745,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":15682745,"total_server_duration_us":56,"operation_id":"0x3807","timeout_ms":30000,"last_server_duration_us":56,"total_duration_us":15684999},{"operation_name":"upsert","last_dispatch_duration_us":420010,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":4127692,"total_server_duration_us":417353,"encode_duration_us":17589,"operation_id":"0x3a72","timeout_ms":10000,"last_server_duration_us":62668,"total_duration_us":10010239},{"operation_name":"upsert","last_dispatch_duration_us":7586843,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7586843,"total_server_duration_us":586228,"encode_duration_us":17965,"operation_id":"0x3902","timeout_ms":10000,"last_server_duration_us":586228,"total_duration_us":10009979},{"operation_name":"upsert","last_dispatch_duration_us":1604078,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":4560391,"total_server_duration_us":1928651,"encode_duration_us":23110,"operation_id":"0x39eb","timeout_ms":10000,"last_server_duration_us":1074376,"total_duration_us":10009886},{"operation_name":"upsert","last_dispatch_duration_us":859223,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":3566000,"total_server_duration_us":688673,"encode_duration_us":16607,"operation_id":"0x39ea","timeout_ms":10000,"last_server_duration_us":96865,"total_duration_us":10009208},{"operation_name":"upsert","encode_duration_us":19351,"operation_id":"0x39b8","timeout_ms":10000,"total_duration_us":10008795},{"operation_name":"upsert","last_dispatch_duration_us":594679,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10088399,"total_server_duration_us":3546169,"encode_duration_us":16493,"operation_id":"0x3a58","timeout_ms":10000,"last_server_duration_us":152366,"total_duration_us":10007308},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":23725,"operation_id":"0x3a4a","timeout_ms":10000,"total_duration_us":10003518},{"operation_name":"upsert","last_dispatch_duration_us":1794936,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":11021855,"total_server_duration_us":6432474,"encode_duration_us":16556,"operation_id":"0x39e1","timeout_ms":10000,"last_server_duration_us":440185,"total_duration_us":10002505},{"operation_name":"upsert","encode_duration_us":16377,"operation_id":"0x392c","timeout_ms":10000,"total_duration_us":10002349}],"total_count":19}} INFO [2022-01-10 05:41:11,430] jepsen worker 10 - jepsen.util 1990 :info :write [3 29] :RequestTimeoutException INFO [2022-01-10 05:41:11,431] jepsen worker 10 - jepsen.util 2080 :invoke :read [3 nil] INFO [2022-01-10 05:41:11,455] jepsen worker 47 - jepsen.util 1757 :info :write [15 9] :RequestTimeoutException INFO [2022-01-10 05:41:11,456] jepsen worker 47 - jepsen.util 1847 :invoke :write [15 3] INFO [2022-01-10 05:41:11,456] jepsen worker 57 - jepsen.util 1947 :info :write [19 38] :RequestTimeoutException INFO [2022-01-10 05:41:11,457] jepsen worker 57 - jepsen.util 2037 :invoke :write [19 3] WARN [2022-01-10 05:41:11,508] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10353057,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10759547,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9115573,"encode_duration_us":16339,"operation_id":"0x39a2","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10002746}],"total_count":1}} WARN [2022-01-10 05:41:11,508] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18393683,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":18393683,"total_server_duration_us":739767,"operation_id":"0x3842","timeout_ms":30000,"last_server_duration_us":739767,"total_duration_us":18395764},{"operation_name":"get","last_dispatch_duration_us":17557250,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":17557250,"total_server_duration_us":617073,"operation_id":"0x3841","timeout_ms":30000,"last_server_duration_us":617073,"total_duration_us":17557555},{"operation_name":"get","last_dispatch_duration_us":6590132,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6590132,"total_server_duration_us":31681,"operation_id":"0x39ba","timeout_ms":30000,"last_server_duration_us":31681,"total_duration_us":13802508},{"operation_name":"get","last_dispatch_duration_us":6251921,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6251921,"total_server_duration_us":9977,"operation_id":"0x39e0","timeout_ms":30000,"last_server_duration_us":9977,"total_duration_us":12441492},{"operation_name":"get","last_dispatch_duration_us":6787290,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6787290,"total_server_duration_us":14230,"operation_id":"0x3a21","timeout_ms":30000,"last_server_duration_us":14230,"total_duration_us":11443273},{"operation_name":"upsert","last_dispatch_duration_us":7795660,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10489367,"total_server_duration_us":963994,"encode_duration_us":16934,"operation_id":"0x398c","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10009293},{"operation_name":"upsert","last_dispatch_duration_us":7468199,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10164657,"total_server_duration_us":402499,"encode_duration_us":16878,"operation_id":"0x398b","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10005941},{"operation_name":"upsert","last_dispatch_duration_us":1394268,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9931332,"total_server_duration_us":283854,"encode_duration_us":17429,"operation_id":"0x38f9","timeout_ms":10000,"last_server_duration_us":90781,"total_duration_us":10005810},{"operation_name":"upsert","last_dispatch_duration_us":17075347,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":17075347,"total_server_duration_us":472677,"encode_duration_us":17011,"operation_id":"0x38f2","timeout_ms":10000,"last_server_duration_us":472677,"total_duration_us":10002976},{"operation_name":"upsert","last_dispatch_duration_us":10353057,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10759547,"total_server_duration_us":9115573,"encode_duration_us":16339,"operation_id":"0x39a2","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10002746}],"total_count":13}} WARN [2022-01-10 05:41:11,508] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16764688,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":16764688,"total_server_duration_us":27,"operation_id":"0x37d5","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":16765692},{"operation_name":"get","last_dispatch_duration_us":11083149,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":11083149,"total_server_duration_us":32,"operation_id":"0x395b","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":11085184},{"operation_name":"upsert","last_dispatch_duration_us":311108,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":7547836,"total_server_duration_us":495270,"encode_duration_us":16577,"operation_id":"0x3a73","timeout_ms":10000,"last_server_duration_us":42417,"total_duration_us":10010876},{"operation_name":"upsert","last_dispatch_duration_us":11398726,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":11398726,"total_server_duration_us":1538695,"encode_duration_us":16457,"operation_id":"0x391d","timeout_ms":10000,"last_server_duration_us":1538695,"total_duration_us":10010185},{"operation_name":"upsert","last_dispatch_duration_us":693566,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":10211902,"total_server_duration_us":2671683,"encode_duration_us":16846,"operation_id":"0x3aa2","timeout_ms":10000,"last_server_duration_us":39319,"total_duration_us":10008429},{"operation_name":"upsert","last_dispatch_duration_us":814938,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10506401,"total_server_duration_us":1230847,"encode_duration_us":21779,"operation_id":"0x39f1","timeout_ms":10000,"last_server_duration_us":357779,"total_duration_us":10006423},{"operation_name":"upsert","last_dispatch_duration_us":456844,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":7903540,"total_server_duration_us":585800,"encode_duration_us":22923,"operation_id":"0x3a05","timeout_ms":10000,"last_server_duration_us":74510,"total_duration_us":10005184},{"operation_name":"upsert","last_dispatch_duration_us":1040972,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":7700848,"total_server_duration_us":754849,"encode_duration_us":18146,"operation_id":"0x3a13","timeout_ms":10000,"last_server_duration_us":74786,"total_duration_us":10003056},{"operation_name":"upsert","last_dispatch_duration_us":3421883,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":11316716,"total_server_duration_us":4167267,"encode_duration_us":16691,"operation_id":"0x3934","timeout_ms":10000,"last_server_duration_us":1193762,"total_duration_us":10002329},{"operation_name":"upsert","last_dispatch_duration_us":10061881,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10061881,"total_server_duration_us":574013,"encode_duration_us":16770,"operation_id":"0x39b7","timeout_ms":10000,"last_server_duration_us":574013,"total_duration_us":10002125}],"total_count":24}} WARN [2022-01-10 05:41:11,511] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9828560,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10139638,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":10072629,"encode_duration_us":17308,"operation_id":"0x39bb","timeout_ms":10000,"last_server_duration_us":9816324,"total_duration_us":10006031}],"total_count":1}} INFO [2022-01-10 05:41:11,606] jepsen worker 52 - jepsen.util 2212 :ok :read [17 41] INFO [2022-01-10 05:41:11,606] jepsen worker 52 - jepsen.util 2212 :invoke :read [17 nil] WARN [2022-01-10 05:41:11,609] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17929335,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17929335,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9601860,"encode_duration_us":17379,"operation_id":"0x379e","timeout_ms":10000,"last_server_duration_us":9601860,"total_duration_us":10001624}],"total_count":1}} WARN [2022-01-10 05:41:11,609] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":6596816,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6596816,"total_server_duration_us":15,"operation_id":"0x3744","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":28084572},{"operation_name":"get","last_dispatch_duration_us":6676583,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6676583,"total_server_duration_us":19,"operation_id":"0x374f","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":27712207},{"operation_name":"get","last_dispatch_duration_us":6766496,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6766496,"total_server_duration_us":49,"operation_id":"0x37e6","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":24680728},{"operation_name":"get","last_dispatch_duration_us":6596708,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6596708,"total_server_duration_us":15,"operation_id":"0x3928","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":17886778},{"operation_name":"upsert","last_dispatch_duration_us":11573559,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11573559,"total_server_duration_us":258409,"encode_duration_us":16259,"operation_id":"0x395a","timeout_ms":10000,"last_server_duration_us":258409,"total_duration_us":10009738},{"operation_name":"upsert","last_dispatch_duration_us":342802,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":5355736,"total_server_duration_us":533031,"encode_duration_us":20097,"operation_id":"0x3a81","timeout_ms":10000,"last_server_duration_us":65645,"total_duration_us":10007666},{"operation_name":"upsert","last_dispatch_duration_us":2553355,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":12108979,"total_server_duration_us":328806,"encode_duration_us":16507,"operation_id":"0x38c9","timeout_ms":10000,"last_server_duration_us":126291,"total_duration_us":10007558},{"operation_name":"upsert","last_dispatch_duration_us":61920,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":61920,"total_server_duration_us":19966,"encode_duration_us":16570,"operation_id":"0x39f8","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10007188},{"operation_name":"upsert","last_dispatch_duration_us":927442,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":10432182,"total_server_duration_us":5322577,"encode_duration_us":18494,"operation_id":"0x3a48","timeout_ms":10000,"last_server_duration_us":370780,"total_duration_us":10005656},{"operation_name":"upsert","last_dispatch_duration_us":8703735,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":12708261,"total_server_duration_us":303536,"encode_duration_us":18638,"operation_id":"0x393f","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10005614}],"total_count":23}} WARN [2022-01-10 05:41:11,609] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10044559,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12809130,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":10061836,"encode_duration_us":20839,"operation_id":"0x3936","timeout_ms":10000,"last_server_duration_us":9702276,"total_duration_us":10000652}],"total_count":1}} WARN [2022-01-10 05:41:11,712] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":907879,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":5770263,"total_server_duration_us":210776,"encode_duration_us":23023,"operation_id":"0x39af","timeout_ms":10000,"last_server_duration_us":58501,"total_duration_us":10009415},{"operation_name":"upsert","last_dispatch_duration_us":1497419,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10176667,"total_server_duration_us":3851473,"encode_duration_us":18336,"operation_id":"0x394c","timeout_ms":10000,"last_server_duration_us":1423139,"total_duration_us":10008399},{"operation_name":"upsert","last_dispatch_duration_us":3094585,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":3094585,"total_server_duration_us":85596,"encode_duration_us":19070,"operation_id":"0x3932","timeout_ms":10000,"last_server_duration_us":85596,"total_duration_us":10006711},{"operation_name":"upsert","last_dispatch_duration_us":9828560,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10139638,"total_server_duration_us":10072629,"encode_duration_us":17308,"operation_id":"0x39bb","timeout_ms":10000,"last_server_duration_us":9816324,"total_duration_us":10006031},{"operation_name":"upsert","last_dispatch_duration_us":1941522,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10778270,"total_server_duration_us":3386128,"encode_duration_us":16801,"operation_id":"0x392f","timeout_ms":10000,"last_server_duration_us":1008750,"total_duration_us":10005870},{"operation_name":"upsert","last_dispatch_duration_us":4881346,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":6972648,"total_server_duration_us":808862,"encode_duration_us":16744,"operation_id":"0x391f","timeout_ms":10000,"last_server_duration_us":407829,"total_duration_us":10005378},{"operation_name":"upsert","last_dispatch_duration_us":2020658,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4119258,"total_server_duration_us":416248,"encode_duration_us":19392,"operation_id":"0x38f3","timeout_ms":10000,"last_server_duration_us":397372,"total_duration_us":10003502},{"operation_name":"upsert","last_dispatch_duration_us":11189627,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11189627,"total_server_duration_us":405842,"encode_duration_us":16336,"operation_id":"0x3989","timeout_ms":10000,"last_server_duration_us":405842,"total_duration_us":10002148},{"operation_name":"get","last_dispatch_duration_us":3545674,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":3545674,"total_server_duration_us":27,"operation_id":"0x39e8","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":5146905},{"operation_name":"get","last_dispatch_duration_us":2815388,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":2815388,"total_server_duration_us":43,"operation_id":"0x3a47","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":4413499}],"total_count":15}} INFO [2022-01-10 05:41:11,834] jepsen worker 52 - jepsen.util 2212 :ok :read [17 41] INFO [2022-01-10 05:41:11,835] jepsen worker 52 - jepsen.util 2212 :invoke :write [17 17] INFO [2022-01-10 05:41:12,060] jepsen worker 75 - jepsen.util 1335 :ok :read [25 :nil] INFO [2022-01-10 05:41:12,060] jepsen worker 75 - jepsen.util 1335 :invoke :read [25 nil] INFO [2022-01-10 05:41:12,070] jepsen worker 58 - jepsen.util 1948 :info :write [19 47] :RequestTimeoutException INFO [2022-01-10 05:41:12,070] jepsen worker 58 - jepsen.util 2038 :invoke :read [19 nil] INFO [2022-01-10 05:41:12,183] jepsen worker 46 - jepsen.util 2116 :info :write [15 8] :RequestTimeoutException INFO [2022-01-10 05:41:12,183] jepsen worker 46 - jepsen.util 2206 :invoke :write [15 18] INFO [2022-01-10 05:41:12,188] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:12,188] jepsen worker nemesis - couchbase.util Rebalance check count: 226 INFO [2022-01-10 05:41:12,188] jepsen worker nemesis - couchbase.util Rebalance stuck count: 164 INFO [2022-01-10 05:41:12,379] jepsen worker 67 - jepsen.util 1777 :info :write [22 10] :RequestTimeoutException INFO [2022-01-10 05:41:12,380] jepsen worker 67 - jepsen.util 1867 :invoke :read [22 nil] INFO [2022-01-10 05:41:12,386] jepsen worker 86 - jepsen.util 1616 :info :write [28 38] :RequestTimeoutException INFO [2022-01-10 05:41:12,387] jepsen worker 86 - jepsen.util 1706 :invoke :read [28 nil] INFO [2022-01-10 05:41:12,616] jepsen worker 2 - jepsen.util 1892 :info :write [0 25] :RequestTimeoutException INFO [2022-01-10 05:41:12,616] jepsen worker 2 - jepsen.util 1982 :invoke :read [0 nil] INFO [2022-01-10 05:41:12,982] jepsen worker 9 - jepsen.util 2079 :info :write [3 13] :RequestTimeoutException INFO [2022-01-10 05:41:12,983] jepsen worker 9 - jepsen.util 2169 :invoke :read [3 nil] INFO [2022-01-10 05:41:13,057] jepsen worker 86 - jepsen.util 1706 :ok :read [28 13] INFO [2022-01-10 05:41:13,057] jepsen worker 86 - jepsen.util 1706 :invoke :read [28 nil] INFO [2022-01-10 05:41:13,066] jepsen worker 28 - jepsen.util 1828 :ok :read [9 22] INFO [2022-01-10 05:41:13,067] jepsen worker 28 - jepsen.util 1828 :invoke :write [9 37] INFO [2022-01-10 05:41:13,071] jepsen worker 23 - jepsen.util 2003 :info :write [7 8] :RequestTimeoutException INFO [2022-01-10 05:41:13,071] jepsen worker 23 - jepsen.util 2093 :invoke :read [7 nil] INFO [2022-01-10 05:41:13,135] jepsen worker 66 - jepsen.util 1686 :ok :read [22 41] INFO [2022-01-10 05:41:13,136] jepsen worker 66 - jepsen.util 1686 :invoke :write [22 38] INFO [2022-01-10 05:41:13,159] jepsen worker 1 - jepsen.util 1981 :ok :read [0 46] INFO [2022-01-10 05:41:13,160] jepsen worker 1 - jepsen.util 1981 :invoke :read [0 nil] INFO [2022-01-10 05:41:13,201] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:13,202] jepsen worker nemesis - couchbase.util Rebalance check count: 227 INFO [2022-01-10 05:41:13,203] jepsen worker nemesis - couchbase.util Rebalance stuck count: 165 INFO [2022-01-10 05:41:13,370] jepsen worker 86 - jepsen.util 1706 :ok :read [28 13] INFO [2022-01-10 05:41:13,371] jepsen worker 86 - jepsen.util 1706 :invoke :read [28 nil] INFO [2022-01-10 05:41:13,453] jepsen worker 86 - jepsen.util 1706 :ok :read [28 13] INFO [2022-01-10 05:41:13,454] jepsen worker 86 - jepsen.util 1706 :invoke :read [28 nil] INFO [2022-01-10 05:41:13,540] jepsen worker 86 - jepsen.util 1706 :ok :read [28 13] INFO [2022-01-10 05:41:13,540] jepsen worker 86 - jepsen.util 1706 :invoke :read [28 nil] INFO [2022-01-10 05:41:13,618] jepsen worker 1 - jepsen.util 1981 :ok :read [0 46] INFO [2022-01-10 05:41:13,618] jepsen worker 1 - jepsen.util 1981 :invoke :read [0 nil] INFO [2022-01-10 05:41:13,643] jepsen worker 86 - jepsen.util 1706 :ok :read [28 13] INFO [2022-01-10 05:41:13,643] jepsen worker 86 - jepsen.util 1706 :invoke :read [28 nil] INFO [2022-01-10 05:41:13,703] jepsen worker 86 - jepsen.util 1706 :ok :read [28 13] INFO [2022-01-10 05:41:13,704] jepsen worker 86 - jepsen.util 1706 :invoke :read [28 nil] INFO [2022-01-10 05:41:13,786] jepsen worker 63 - jepsen.util 1773 :ok :read [21 7] INFO [2022-01-10 05:41:13,787] jepsen worker 63 - jepsen.util 1773 :invoke :write [21 13] INFO [2022-01-10 05:41:13,799] jepsen worker 1 - jepsen.util 1981 :ok :read [0 46] INFO [2022-01-10 05:41:13,800] jepsen worker 1 - jepsen.util 1981 :invoke :write [0 40] INFO [2022-01-10 05:41:14,161] jepsen worker 17 - jepsen.util 1817 :info :write [5 29] :RequestTimeoutException INFO [2022-01-10 05:41:14,163] jepsen worker 17 - jepsen.util 1907 :invoke :write [5 38] INFO [2022-01-10 05:41:14,165] jepsen worker 36 - jepsen.util 1566 :info :write [12 44] :RequestTimeoutException INFO [2022-01-10 05:41:14,166] jepsen worker 42 - jepsen.util 1842 :info :write [14 41] :RequestTimeoutException INFO [2022-01-10 05:41:14,166] jepsen worker 36 - jepsen.util 1656 :invoke :read [12 nil] INFO [2022-01-10 05:41:14,166] jepsen worker 37 - jepsen.util 1927 :info :write [12 22] :RequestTimeoutException INFO [2022-01-10 05:41:14,167] jepsen worker 42 - jepsen.util 1932 :invoke :write [14 2] INFO [2022-01-10 05:41:14,168] jepsen worker 37 - jepsen.util 2017 :invoke :write [12 12] INFO [2022-01-10 05:41:14,181] jepsen worker 12 - jepsen.util 1632 :info :write [4 15] :RequestTimeoutException INFO [2022-01-10 05:41:14,182] jepsen worker 12 - jepsen.util 1722 :invoke :read [4 nil] INFO [2022-01-10 05:41:14,186] jepsen worker 8 - jepsen.util 1988 :info :write [2 13] :RequestTimeoutException INFO [2022-01-10 05:41:14,187] jepsen worker 8 - jepsen.util 2078 :invoke :read [2 nil] INFO [2022-01-10 05:41:14,208] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:14,208] jepsen worker nemesis - couchbase.util Rebalance check count: 228 INFO [2022-01-10 05:41:14,208] jepsen worker nemesis - couchbase.util Rebalance stuck count: 166 INFO [2022-01-10 05:41:14,259] jepsen worker 12 - jepsen.util 1722 :ok :read [4 3] INFO [2022-01-10 05:41:14,259] jepsen worker 12 - jepsen.util 1722 :invoke :read [4 nil] INFO [2022-01-10 05:41:14,285] jepsen worker 12 - jepsen.util 1722 :ok :read [4 3] INFO [2022-01-10 05:41:14,286] jepsen worker 12 - jepsen.util 1722 :invoke :write [4 14] INFO [2022-01-10 05:41:14,411] jepsen worker 55 - jepsen.util 2035 :info :write [18 22] :RequestTimeoutException INFO [2022-01-10 05:41:14,412] jepsen worker 55 - jepsen.util 2125 :invoke :write [18 30] INFO [2022-01-10 05:41:14,558] jepsen worker 62 - jepsen.util 2042 :ok :read [20 :nil] INFO [2022-01-10 05:41:14,559] jepsen worker 62 - jepsen.util 2042 :invoke :write [20 26] INFO [2022-01-10 05:41:14,559] jepsen worker 75 - jepsen.util 1335 :ok :read [25 :nil] INFO [2022-01-10 05:41:14,560] jepsen worker 75 - jepsen.util 1335 :invoke :read [25 nil] INFO [2022-01-10 05:41:14,620] jepsen worker 6 - jepsen.util 1896 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,623] jepsen worker 6 - jepsen.util 1896 :invoke :read [2 nil] INFO [2022-01-10 05:41:14,623] jepsen worker 7 - jepsen.util 1987 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,624] jepsen worker 73 - jepsen.util 1873 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,624] jepsen worker 65 - jepsen.util 1775 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,626] jepsen worker 23 - jepsen.util 2093 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,626] jepsen worker 60 - jepsen.util 2310 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,626] jepsen worker 7 - jepsen.util 1987 :invoke :read [2 nil] INFO [2022-01-10 05:41:14,626] jepsen worker 73 - jepsen.util 1873 :invoke :write [24 31] INFO [2022-01-10 05:41:14,627] jepsen worker 23 - jepsen.util 2093 :invoke :read [7 nil] INFO [2022-01-10 05:41:14,627] jepsen worker 65 - jepsen.util 1775 :invoke :read [21 nil] INFO [2022-01-10 05:41:14,628] jepsen worker 60 - jepsen.util 2310 :invoke :read [20 nil] INFO [2022-01-10 05:41:14,629] jepsen worker 8 - jepsen.util 2078 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,630] jepsen worker 8 - jepsen.util 2078 :invoke :read [2 nil] INFO [2022-01-10 05:41:14,639] jepsen worker 35 - jepsen.util 1655 :ok :read [11 21] INFO [2022-01-10 05:41:14,640] jepsen worker 35 - jepsen.util 1655 :invoke :write [11 30] INFO [2022-01-10 05:41:14,691] jepsen worker 26 - jepsen.util 1646 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,692] jepsen worker 26 - jepsen.util 1646 :invoke :read [8 nil] INFO [2022-01-10 05:41:14,815] jepsen worker 7 - jepsen.util 1987 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,815] jepsen worker 6 - jepsen.util 1896 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,816] jepsen worker 6 - jepsen.util 1896 :invoke :read [2 nil] INFO [2022-01-10 05:41:14,816] jepsen worker 7 - jepsen.util 1987 :invoke :write [2 1] INFO [2022-01-10 05:41:14,821] jepsen worker 65 - jepsen.util 1775 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,822] jepsen worker 60 - jepsen.util 2310 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,825] jepsen worker 65 - jepsen.util 1775 :invoke :write [21 0] INFO [2022-01-10 05:41:14,826] jepsen worker 60 - jepsen.util 2310 :invoke :write [20 10] INFO [2022-01-10 05:41:14,817] jepsen worker 23 - jepsen.util 2093 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,826] jepsen worker 23 - jepsen.util 2093 :invoke :read [7 nil] INFO [2022-01-10 05:41:14,830] jepsen worker 82 - jepsen.util 1702 :info :write [27 21] :RequestTimeoutException INFO [2022-01-10 05:41:14,831] jepsen worker 82 - jepsen.util 1792 :invoke :write [27 9] INFO [2022-01-10 05:41:14,823] jepsen worker 8 - jepsen.util 2078 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,831] jepsen worker 8 - jepsen.util 2078 :invoke :write [2 30] INFO [2022-01-10 05:41:14,872] jepsen worker 40 - jepsen.util 1480 :info :write [13 7] :RequestTimeoutException INFO [2022-01-10 05:41:14,872] jepsen worker 40 - jepsen.util 1570 :invoke :read [13 nil] INFO [2022-01-10 05:41:14,920] jepsen worker 27 - jepsen.util 1737 :info :write [9 6] :RequestTimeoutException INFO [2022-01-10 05:41:14,921] jepsen worker 27 - jepsen.util 1827 :invoke :read [9 nil] INFO [2022-01-10 05:41:14,945] jepsen worker 22 - jepsen.util 1642 :ok :read [7 :nil] INFO [2022-01-10 05:41:14,947] jepsen worker 24 - jepsen.util 1824 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,953] jepsen worker 21 - jepsen.util 2001 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,953] jepsen worker 24 - jepsen.util 1824 :invoke :write [8 37] INFO [2022-01-10 05:41:14,953] jepsen worker 22 - jepsen.util 1642 :invoke :read [7 nil] INFO [2022-01-10 05:41:14,956] jepsen worker 34 - jepsen.util 1924 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:14,958] jepsen worker 34 - jepsen.util 1924 :invoke :read [11 nil] INFO [2022-01-10 05:41:14,957] jepsen worker 21 - jepsen.util 2001 :invoke :write [7 45] INFO [2022-01-10 05:41:14,963] jepsen worker 43 - jepsen.util 1843 :ok :read [14 14] INFO [2022-01-10 05:41:14,963] jepsen worker 43 - jepsen.util 1843 :invoke :write [14 2] INFO [2022-01-10 05:41:15,186] jepsen worker 70 - jepsen.util 2140 :info :write [23 14] :RequestTimeoutException INFO [2022-01-10 05:41:15,187] jepsen worker 70 - jepsen.util 2230 :invoke :read [23 nil] INFO [2022-01-10 05:41:15,219] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:15,219] jepsen worker nemesis - couchbase.util Rebalance check count: 229 INFO [2022-01-10 05:41:15,219] jepsen worker nemesis - couchbase.util Rebalance stuck count: 167 INFO [2022-01-10 05:41:15,250] jepsen worker 71 - jepsen.util 1691 :ok :read [23 26] INFO [2022-01-10 05:41:15,250] jepsen worker 71 - jepsen.util 1691 :invoke :read [23 nil] INFO [2022-01-10 05:41:15,286] jepsen worker 74 - jepsen.util 2414 :info :write [24 17] :RequestTimeoutException INFO [2022-01-10 05:41:15,287] jepsen worker 74 - jepsen.util 2504 :invoke :read [24 nil] INFO [2022-01-10 05:41:15,342] jepsen worker 54 - jepsen.util 1854 :ok :read [18 14] INFO [2022-01-10 05:41:15,343] jepsen worker 54 - jepsen.util 1854 :invoke :write [18 25] INFO [2022-01-10 05:41:15,390] jepsen worker 80 - jepsen.util 1970 :info :write [26 41] :RequestTimeoutException INFO [2022-01-10 05:41:15,391] jepsen worker 80 - jepsen.util 2060 :invoke :read [26 nil] INFO [2022-01-10 05:41:15,406] jepsen worker 48 - jepsen.util 1578 :info :write [16 19] :RequestTimeoutException INFO [2022-01-10 05:41:15,407] jepsen worker 48 - jepsen.util 1668 :invoke :read [16 nil] INFO [2022-01-10 05:41:15,430] jepsen worker 29 - jepsen.util 1919 :info :write [9 49] :RequestTimeoutException INFO [2022-01-10 05:41:15,431] jepsen worker 29 - jepsen.util 2009 :invoke :read [9 nil] INFO [2022-01-10 05:41:15,516] jepsen worker 5 - jepsen.util 1895 :info :write [1 20] :RequestTimeoutException INFO [2022-01-10 05:41:15,517] jepsen worker 5 - jepsen.util 1985 :invoke :write [1 4] INFO [2022-01-10 05:41:15,541] jepsen worker 80 - jepsen.util 2060 :ok :read [26 10] INFO [2022-01-10 05:41:15,542] jepsen worker 80 - jepsen.util 2060 :invoke :read [26 nil] INFO [2022-01-10 05:41:15,565] jepsen worker 80 - jepsen.util 2060 :ok :read [26 10] INFO [2022-01-10 05:41:15,566] jepsen worker 80 - jepsen.util 2060 :invoke :write [26 5] INFO [2022-01-10 05:41:15,633] jepsen worker 3 - jepsen.util 1983 :ok :read [1 41] INFO [2022-01-10 05:41:15,635] jepsen worker 31 - jepsen.util 2191 :ok :read [10 37] INFO [2022-01-10 05:41:15,636] jepsen worker 3 - jepsen.util 1983 :invoke :read [1 nil] INFO [2022-01-10 05:41:15,642] jepsen worker 31 - jepsen.util 2191 :invoke :write [10 15] INFO [2022-01-10 05:41:15,652] jepsen worker 9 - jepsen.util 2169 :ok :read [3 2] INFO [2022-01-10 05:41:15,652] jepsen worker 9 - jepsen.util 2169 :invoke :write [3 39] INFO [2022-01-10 05:41:15,665] jepsen worker 70 - jepsen.util 2230 :ok :read [23 26] INFO [2022-01-10 05:41:15,666] jepsen worker 70 - jepsen.util 2230 :invoke :read [23 nil] INFO [2022-01-10 05:41:15,674] jepsen worker 3 - jepsen.util 1983 :ok :read [1 41] INFO [2022-01-10 05:41:15,675] jepsen worker 3 - jepsen.util 1983 :invoke :write [1 47] INFO [2022-01-10 05:41:15,707] jepsen worker 70 - jepsen.util 2230 :ok :read [23 26] INFO [2022-01-10 05:41:15,708] jepsen worker 70 - jepsen.util 2230 :invoke :read [23 nil] INFO [2022-01-10 05:41:15,751] jepsen worker 61 - jepsen.util 1861 :info :write [20 6] :RequestTimeoutException INFO [2022-01-10 05:41:15,752] jepsen worker 61 - jepsen.util 1951 :invoke :read [20 nil] INFO [2022-01-10 05:41:15,775] jepsen worker 30 - jepsen.util 1560 :info :write [10 27] :RequestTimeoutException INFO [2022-01-10 05:41:15,776] jepsen worker 30 - jepsen.util 1650 :invoke :read [10 nil] INFO [2022-01-10 05:41:15,791] jepsen worker 69 - jepsen.util 1869 :info :write [23 47] :RequestTimeoutException INFO [2022-01-10 05:41:15,791] jepsen worker 69 - jepsen.util 1959 :invoke :write [23 13] INFO [2022-01-10 05:41:15,805] jepsen worker 15 - jepsen.util 1635 :ok :read [5 42] INFO [2022-01-10 05:41:15,806] jepsen worker 15 - jepsen.util 1635 :invoke :write [5 13] INFO [2022-01-10 05:41:15,854] jepsen worker 70 - jepsen.util 2230 :ok :read [23 26] INFO [2022-01-10 05:41:15,854] jepsen worker 70 - jepsen.util 2230 :invoke :write [23 35] INFO [2022-01-10 05:41:15,860] jepsen worker 39 - jepsen.util 2199 :info :write [13 1] :RequestTimeoutException INFO [2022-01-10 05:41:15,861] jepsen worker 39 - jepsen.util 2289 :invoke :read [13 nil] INFO [2022-01-10 05:41:15,967] jepsen worker 36 - jepsen.util 1656 :ok :read [12 42] INFO [2022-01-10 05:41:15,968] jepsen worker 36 - jepsen.util 1656 :invoke :read [12 nil] INFO [2022-01-10 05:41:15,985] jepsen worker 2 - jepsen.util 1982 :ok :read [0 46] INFO [2022-01-10 05:41:15,986] jepsen worker 2 - jepsen.util 1982 :invoke :read [0 nil] INFO [2022-01-10 05:41:16,001] jepsen worker 36 - jepsen.util 1656 :ok :read [12 42] INFO [2022-01-10 05:41:16,001] jepsen worker 36 - jepsen.util 1656 :invoke :write [12 3] INFO [2022-01-10 05:41:16,022] jepsen worker 2 - jepsen.util 1982 :ok :read [0 46] INFO [2022-01-10 05:41:16,023] jepsen worker 2 - jepsen.util 1982 :invoke :write [0 28] INFO [2022-01-10 05:41:16,043] jepsen worker 27 - jepsen.util 1827 :ok :read [9 22] INFO [2022-01-10 05:41:16,044] jepsen worker 27 - jepsen.util 1827 :invoke :read [9 nil] INFO [2022-01-10 05:41:16,092] jepsen worker 14 - jepsen.util 1994 :info :write [4 29] :RequestTimeoutException INFO [2022-01-10 05:41:16,093] jepsen worker 14 - jepsen.util 2084 :invoke :read [4 nil] INFO [2022-01-10 05:41:16,099] jepsen worker 27 - jepsen.util 1827 :ok :read [9 22] INFO [2022-01-10 05:41:16,100] jepsen worker 27 - jepsen.util 1827 :invoke :read [9 nil] INFO [2022-01-10 05:41:16,112] jepsen worker 56 - jepsen.util 2036 :info :write [18 48] :SyncWriteAmbiguous INFO [2022-01-10 05:41:16,113] jepsen worker 56 - jepsen.util 2126 :invoke :write [18 0] INFO [2022-01-10 05:41:16,121] jepsen worker 14 - jepsen.util 2084 :ok :read [4 3] INFO [2022-01-10 05:41:16,122] jepsen worker 14 - jepsen.util 2084 :invoke :read [4 nil] INFO [2022-01-10 05:41:16,148] jepsen worker 27 - jepsen.util 1827 :ok :read [9 22] INFO [2022-01-10 05:41:16,148] jepsen worker 27 - jepsen.util 1827 :invoke :write [9 12] INFO [2022-01-10 05:41:16,153] jepsen worker 14 - jepsen.util 2084 :ok :read [4 3] INFO [2022-01-10 05:41:16,153] jepsen worker 14 - jepsen.util 2084 :invoke :write [4 28] INFO [2022-01-10 05:41:16,224] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:16,224] jepsen worker nemesis - couchbase.util Rebalance check count: 230 INFO [2022-01-10 05:41:16,225] jepsen worker nemesis - couchbase.util Rebalance stuck count: 168 INFO [2022-01-10 05:41:16,252] jepsen worker 45 - jepsen.util 2025 :info :write [15 16] :RequestTimeoutException INFO [2022-01-10 05:41:16,253] jepsen worker 45 - jepsen.util 2115 :invoke :read [15 nil] INFO [2022-01-10 05:41:16,256] jepsen worker 11 - jepsen.util 1721 :info :write [3 30] :RequestTimeoutException INFO [2022-01-10 05:41:16,257] jepsen worker 11 - jepsen.util 1811 :invoke :read [3 nil] INFO [2022-01-10 05:41:16,267] jepsen worker 10 - jepsen.util 2080 :ok :read [3 2] INFO [2022-01-10 05:41:16,268] jepsen worker 10 - jepsen.util 2080 :invoke :write [3 18] INFO [2022-01-10 05:41:16,375] jepsen worker 4 - jepsen.util 1894 :info :write [1 9] :RequestTimeoutException INFO [2022-01-10 05:41:16,376] jepsen worker 4 - jepsen.util 1984 :invoke :write [1 15] INFO [2022-01-10 05:41:16,580] jepsen worker 88 - jepsen.util 1888 :info :write [29 16] :RequestTimeoutException INFO [2022-01-10 05:41:16,580] jepsen worker 88 - jepsen.util 1978 :invoke :read [29 nil] INFO [2022-01-10 05:41:17,006] jepsen worker 84 - jepsen.util 2064 :info :write [28 23] :RequestTimeoutException INFO [2022-01-10 05:41:17,007] jepsen worker 84 - jepsen.util 2154 :invoke :read [28 nil] INFO [2022-01-10 05:41:17,029] jepsen worker 26 - jepsen.util 1646 :ok :read [8 27] INFO [2022-01-10 05:41:17,030] jepsen worker 26 - jepsen.util 1646 :invoke :write [8 3] INFO [2022-01-10 05:41:17,120] jepsen worker 53 - jepsen.util 2033 :info :write [17 30] :RequestTimeoutException INFO [2022-01-10 05:41:17,120] jepsen worker 53 - jepsen.util 2123 :invoke :write [17 20] INFO [2022-01-10 05:41:17,230] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:17,231] jepsen worker nemesis - couchbase.util Rebalance check count: 231 INFO [2022-01-10 05:41:17,231] jepsen worker nemesis - couchbase.util Rebalance stuck count: 169 INFO [2022-01-10 05:41:17,546] jepsen worker 85 - jepsen.util 2155 :info :write [28 27] :RequestTimeoutException INFO [2022-01-10 05:41:17,546] jepsen worker 85 - jepsen.util 2245 :invoke :write [28 15] INFO [2022-01-10 05:41:17,665] jepsen worker 13 - jepsen.util 1723 :info :write [4 21] :RequestTimeoutException INFO [2022-01-10 05:41:17,665] jepsen worker 13 - jepsen.util 1813 :invoke :write [4 22] INFO [2022-01-10 05:41:17,706] jepsen worker 0 - jepsen.util 1800 :info :write [0 15] :RequestTimeoutException INFO [2022-01-10 05:41:17,707] jepsen worker 0 - jepsen.util 1890 :invoke :write [0 1] INFO [2022-01-10 05:41:17,750] jepsen worker 38 - jepsen.util 2018 :info :write [12 28] :RequestTimeoutException INFO [2022-01-10 05:41:17,750] jepsen worker 38 - jepsen.util 2108 :invoke :write [12 43] INFO [2022-01-10 05:41:17,822] jepsen worker 81 - jepsen.util 1881 :info :write [27 11] :RequestTimeoutException INFO [2022-01-10 05:41:17,823] jepsen worker 81 - jepsen.util 1971 :invoke :write [27 37] INFO [2022-01-10 05:41:17,847] jepsen worker 16 - jepsen.util 2266 :info :write [5 22] :RequestTimeoutException INFO [2022-01-10 05:41:17,847] jepsen worker 16 - jepsen.util 2356 :invoke :read [5 nil] INFO [2022-01-10 05:41:17,871] jepsen worker 16 - jepsen.util 2356 :ok :read [5 42] INFO [2022-01-10 05:41:17,872] jepsen worker 16 - jepsen.util 2356 :invoke :write [5 46] INFO [2022-01-10 05:41:17,925] jepsen worker 33 - jepsen.util 1923 :info :write [11 38] :RequestTimeoutException INFO [2022-01-10 05:41:17,925] jepsen worker 33 - jepsen.util 2013 :invoke :read [11 nil] INFO [2022-01-10 05:41:18,237] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:18,237] jepsen worker nemesis - couchbase.util Rebalance check count: 232 INFO [2022-01-10 05:41:18,237] jepsen worker nemesis - couchbase.util Rebalance stuck count: 170 INFO [2022-01-10 05:41:18,555] jepsen worker 32 - jepsen.util 1922 :info :write [10 21] :RequestTimeoutException INFO [2022-01-10 05:41:18,556] jepsen worker 32 - jepsen.util 2012 :invoke :write [10 47] INFO [2022-01-10 05:41:18,655] jepsen worker 59 - jepsen.util 1949 :info :write [19 9] :RequestTimeoutException INFO [2022-01-10 05:41:18,656] jepsen worker 59 - jepsen.util 2039 :invoke :write [19 22] INFO [2022-01-10 05:41:19,243] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:19,243] jepsen worker nemesis - couchbase.util Rebalance check count: 233 INFO [2022-01-10 05:41:19,243] jepsen worker nemesis - couchbase.util Rebalance stuck count: 171 INFO [2022-01-10 05:41:19,650] jepsen worker 18 - jepsen.util 1458 :info :write [6 32] :RequestTimeoutException INFO [2022-01-10 05:41:19,650] jepsen worker 18 - jepsen.util 1548 :invoke :read [6 nil] INFO [2022-01-10 05:41:19,805] jepsen worker 78 - jepsen.util 1968 :info :write [26 11] :RequestTimeoutException INFO [2022-01-10 05:41:19,806] jepsen worker 78 - jepsen.util 2058 :invoke :write [26 20] WARN [2022-01-10 05:41:19,821] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10875403,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14609656,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":10231192,"encode_duration_us":16279,"operation_id":"0x3a7d","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10007639}],"total_count":1}} WARN [2022-01-10 05:41:20,021] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10082391,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10082391,"total_server_duration_us":69,"operation_id":"0x3b63","timeout_ms":30000,"last_server_duration_us":69,"total_duration_us":10391991},{"operation_name":"upsert","last_dispatch_duration_us":5903032,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5903032,"total_server_duration_us":446073,"encode_duration_us":28107,"operation_id":"0x3b3c","timeout_ms":10000,"last_server_duration_us":446073,"total_duration_us":10010625},{"operation_name":"upsert","last_dispatch_duration_us":57740,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":6347657,"total_server_duration_us":2430129,"encode_duration_us":17535,"operation_id":"0x3bd9","timeout_ms":10000,"last_server_duration_us":37550,"total_duration_us":10009633},{"operation_name":"upsert","last_dispatch_duration_us":10875403,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":14609656,"total_server_duration_us":10231192,"encode_duration_us":16279,"operation_id":"0x3a7d","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10007639},{"operation_name":"upsert","last_dispatch_duration_us":4156604,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":4798128,"total_server_duration_us":773219,"encode_duration_us":16235,"operation_id":"0x3a88","timeout_ms":10000,"last_server_duration_us":393445,"total_duration_us":10006693},{"operation_name":"upsert","last_dispatch_duration_us":9077588,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9077588,"total_server_duration_us":19966,"encode_duration_us":17625,"operation_id":"0x3ba8","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10006249},{"operation_name":"upsert","last_dispatch_duration_us":2999568,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2999568,"total_server_duration_us":434038,"encode_duration_us":18082,"operation_id":"0x3bfe","timeout_ms":10000,"last_server_duration_us":434038,"total_duration_us":10006041},{"operation_name":"upsert","last_dispatch_duration_us":9211110,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":11245523,"total_server_duration_us":239902,"encode_duration_us":24323,"operation_id":"0x3b3a","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10004457},{"operation_name":"upsert","last_dispatch_duration_us":9418092,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10869817,"total_server_duration_us":10497856,"encode_duration_us":16498,"operation_id":"0x3c0b","timeout_ms":10000,"last_server_duration_us":9336620,"total_duration_us":10003624},{"operation_name":"upsert","last_dispatch_duration_us":6007390,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10572443,"total_server_duration_us":1784876,"encode_duration_us":19910,"operation_id":"0x3aac","timeout_ms":10000,"last_server_duration_us":676180,"total_duration_us":10003441}],"total_count":17}} INFO [2022-01-10 05:41:20,060] jepsen worker 58 - jepsen.util 2038 :ok :read [19 31] INFO [2022-01-10 05:41:20,060] jepsen worker 58 - jepsen.util 2038 :invoke :write [19 11] INFO [2022-01-10 05:41:20,111] jepsen worker 49 - jepsen.util 1669 :info :write [16 1] :RequestTimeoutException INFO [2022-01-10 05:41:20,112] jepsen worker 49 - jepsen.util 1759 :invoke :read [16 nil] INFO [2022-01-10 05:41:20,114] jepsen worker 83 - jepsen.util 2153 :info :write [27 20] :RequestTimeoutException INFO [2022-01-10 05:41:20,115] jepsen worker 83 - jepsen.util 2243 :invoke :write [27 16] INFO [2022-01-10 05:41:20,145] jepsen worker 64 - jepsen.util 1954 :info :write [21 15] :RequestTimeoutException INFO [2022-01-10 05:41:20,145] jepsen worker 68 - jepsen.util 1868 :info :write [22 30] :RequestTimeoutException INFO [2022-01-10 05:41:20,147] jepsen worker 64 - jepsen.util 2044 :invoke :read [21 nil] INFO [2022-01-10 05:41:20,147] jepsen worker 68 - jepsen.util 1958 :invoke :write [22 29] INFO [2022-01-10 05:41:20,177] jepsen worker 49 - jepsen.util 1759 :ok :read [16 38] INFO [2022-01-10 05:41:20,178] jepsen worker 49 - jepsen.util 1759 :invoke :write [16 26] INFO [2022-01-10 05:41:20,249] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:20,249] jepsen worker nemesis - couchbase.util Rebalance check count: 234 INFO [2022-01-10 05:41:20,249] jepsen worker nemesis - couchbase.util Rebalance stuck count: 172 INFO [2022-01-10 05:41:20,340] jepsen worker 19 - jepsen.util 1999 :ok :read [6 25] INFO [2022-01-10 05:41:20,340] jepsen worker 19 - jepsen.util 1999 :invoke :write [6 16] INFO [2022-01-10 05:41:20,596] jepsen worker 72 - jepsen.util 1962 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:20,598] jepsen worker 76 - jepsen.util 1426 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:20,599] jepsen worker 72 - jepsen.util 1962 :invoke :write [24 35] INFO [2022-01-10 05:41:20,599] jepsen worker 76 - jepsen.util 1426 :invoke :read [25 nil] INFO [2022-01-10 05:41:20,623] jepsen worker 44 - jepsen.util 1934 :ok :read [14 14] INFO [2022-01-10 05:41:20,624] jepsen worker 44 - jepsen.util 1934 :invoke :write [14 45] INFO [2022-01-10 05:41:20,650] jepsen worker 87 - jepsen.util 2067 :info :write [29 33] :RequestTimeoutException INFO [2022-01-10 05:41:20,651] jepsen worker 87 - jepsen.util 2157 :invoke :read [29 nil] INFO [2022-01-10 05:41:20,672] jepsen worker 50 - jepsen.util 2120 :info :write [16 39] :RequestTimeoutException INFO [2022-01-10 05:41:20,673] jepsen worker 50 - jepsen.util 2210 :invoke :read [16 nil] INFO [2022-01-10 05:41:20,706] jepsen worker 51 - jepsen.util 1941 :info :write [17 33] :RequestTimeoutException INFO [2022-01-10 05:41:20,707] jepsen worker 51 - jepsen.util 2031 :invoke :write [17 4] INFO [2022-01-10 05:41:20,741] jepsen worker 89 - jepsen.util 2069 :info :write [29 4] :RequestTimeoutException INFO [2022-01-10 05:41:20,742] jepsen worker 89 - jepsen.util 2159 :invoke :write [29 20] INFO [2022-01-10 05:41:20,904] jepsen worker 20 - jepsen.util 1640 :ok :read [6 25] INFO [2022-01-10 05:41:20,905] jepsen worker 20 - jepsen.util 1640 :invoke :read [6 nil] INFO [2022-01-10 05:41:21,025] jepsen worker 79 - jepsen.util 2059 :info :write [26 25] :RequestTimeoutException INFO [2022-01-10 05:41:21,027] jepsen worker 79 - jepsen.util 2149 :invoke :write [26 8] INFO [2022-01-10 05:41:21,117] jepsen worker 41 - jepsen.util 1661 :info :write [13 31] :RequestTimeoutException INFO [2022-01-10 05:41:21,118] jepsen worker 41 - jepsen.util 1751 :invoke :read [13 nil] INFO [2022-01-10 05:41:21,252] jepsen worker 77 - jepsen.util 2147 :info :write [25 1] :RequestTimeoutException INFO [2022-01-10 05:41:21,253] jepsen worker 77 - jepsen.util 2237 :invoke :write [25 29] INFO [2022-01-10 05:41:21,255] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:21,255] jepsen worker nemesis - couchbase.util Rebalance check count: 235 INFO [2022-01-10 05:41:21,255] jepsen worker nemesis - couchbase.util Rebalance stuck count: 173 INFO [2022-01-10 05:41:21,261] jepsen worker 25 - jepsen.util 1825 :info :write [8 3] :RequestTimeoutException INFO [2022-01-10 05:41:21,262] jepsen worker 25 - jepsen.util 1915 :invoke :read [8 nil] INFO [2022-01-10 05:41:21,320] jepsen worker 26 - jepsen.util 1646 :fail :write [8 3] :ServerOutOfMemoryException INFO [2022-01-10 05:41:21,321] jepsen worker 26 - jepsen.util 1646 :invoke :write [8 6] WARN [2022-01-10 05:41:21,424] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9693919,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19625251,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9314853,"encode_duration_us":17429,"operation_id":"0x38f9","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10005810}],"total_count":1}} WARN [2022-01-10 05:41:21,424] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9812623,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13688434,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":11995960,"encode_duration_us":17404,"operation_id":"0x3aab","timeout_ms":10000,"last_server_duration_us":9370006,"total_duration_us":10006828},{"operation_name":"upsert","last_dispatch_duration_us":10488219,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15048611,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":11469607,"encode_duration_us":23110,"operation_id":"0x39eb","timeout_ms":10000,"last_server_duration_us":9540956,"total_duration_us":10009886},{"operation_name":"upsert","last_dispatch_duration_us":11114352,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14680352,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10043595,"encode_duration_us":16607,"operation_id":"0x39ea","timeout_ms":10000,"last_server_duration_us":9354922,"total_duration_us":10009208}],"total_count":3}} WARN [2022-01-10 05:41:21,424] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10557899,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10557899,"total_server_duration_us":24583,"operation_id":"0x38f6","timeout_ms":30000,"last_server_duration_us":24583,"total_duration_us":27970581},{"operation_name":"get","last_dispatch_duration_us":10285700,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10285700,"total_server_duration_us":147537,"operation_id":"0x3a1f","timeout_ms":30000,"last_server_duration_us":147537,"total_duration_us":22084478},{"operation_name":"upsert","last_dispatch_duration_us":10760261,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10760261,"total_server_duration_us":281535,"encode_duration_us":18334,"operation_id":"0x3b83","timeout_ms":10000,"last_server_duration_us":281535,"total_duration_us":10009742},{"operation_name":"upsert","last_dispatch_duration_us":83220,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":3680488,"total_server_duration_us":171480,"encode_duration_us":17423,"operation_id":"0x3c58","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10009111},{"operation_name":"upsert","last_dispatch_duration_us":10382480,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10382480,"total_server_duration_us":403576,"encode_duration_us":17202,"operation_id":"0x3b00","timeout_ms":10000,"last_server_duration_us":403576,"total_duration_us":10007568},{"operation_name":"upsert","last_dispatch_duration_us":9812623,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":13688434,"total_server_duration_us":11995960,"encode_duration_us":17404,"operation_id":"0x3aab","timeout_ms":10000,"last_server_duration_us":9370006,"total_duration_us":10006828},{"operation_name":"upsert","last_dispatch_duration_us":207574,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":5042283,"total_server_duration_us":179441,"encode_duration_us":18026,"operation_id":"0x3c74","timeout_ms":10000,"last_server_duration_us":94412,"total_duration_us":10006512},{"operation_name":"upsert","last_dispatch_duration_us":7983857,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":8853845,"total_server_duration_us":198944,"encode_duration_us":20275,"operation_id":"0x3b38","timeout_ms":10000,"last_server_duration_us":87064,"total_duration_us":10004932},{"operation_name":"upsert","last_dispatch_duration_us":8913203,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":10747966,"total_server_duration_us":462199,"encode_duration_us":16534,"operation_id":"0x3b3f","timeout_ms":10000,"last_server_duration_us":18493,"total_duration_us":10004312},{"operation_name":"upsert","last_dispatch_duration_us":74443,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":8272377,"total_server_duration_us":4187653,"encode_duration_us":30322,"operation_id":"0x3b3d","timeout_ms":10000,"last_server_duration_us":55061,"total_duration_us":10004033}],"total_count":15}} INFO [2022-01-10 05:41:21,480] jepsen worker 47 - jepsen.util 1847 :info :write [15 3] :RequestTimeoutException INFO [2022-01-10 05:41:21,480] jepsen worker 47 - jepsen.util 1937 :invoke :write [15 0] INFO [2022-01-10 05:41:21,481] jepsen worker 57 - jepsen.util 2037 :info :write [19 3] :RequestTimeoutException INFO [2022-01-10 05:41:21,482] jepsen worker 57 - jepsen.util 2127 :invoke :write [19 39] WARN [2022-01-10 05:41:21,523] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10893092,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10893092,"total_server_duration_us":43,"operation_id":"0x3b9d","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":14531555},{"operation_name":"get","last_dispatch_duration_us":11461075,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":11461075,"total_server_duration_us":43,"operation_id":"0x3b35","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":11463531},{"operation_name":"get","last_dispatch_duration_us":10168106,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10168106,"total_server_duration_us":27,"operation_id":"0x3b75","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":10170742},{"operation_name":"upsert","last_dispatch_duration_us":10220137,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10220137,"total_server_duration_us":262166,"encode_duration_us":16243,"operation_id":"0x3c94","timeout_ms":10000,"last_server_duration_us":262166,"total_duration_us":10010530},{"operation_name":"upsert","last_dispatch_duration_us":48712,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":6893701,"total_server_duration_us":4094395,"encode_duration_us":16816,"operation_id":"0x3b9e","timeout_ms":10000,"last_server_duration_us":33035,"total_duration_us":10009483},{"operation_name":"upsert","last_dispatch_duration_us":13786592,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13786592,"total_server_duration_us":162603,"encode_duration_us":16170,"operation_id":"0x3ad3","timeout_ms":10000,"last_server_duration_us":162603,"total_duration_us":10008834},{"operation_name":"upsert","last_dispatch_duration_us":12624257,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":12624257,"total_server_duration_us":50418,"encode_duration_us":16258,"operation_id":"0x3afe","timeout_ms":10000,"last_server_duration_us":50418,"total_duration_us":10008332},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":29228,"operation_id":"0x3c95","timeout_ms":10000,"total_duration_us":10006509},{"operation_name":"upsert","last_dispatch_duration_us":8030239,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8030239,"total_server_duration_us":21487,"encode_duration_us":28815,"operation_id":"0x3bfa","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10005594},{"operation_name":"upsert","last_dispatch_duration_us":867331,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":2222214,"total_server_duration_us":697012,"encode_duration_us":17665,"operation_id":"0x3b61","timeout_ms":10000,"last_server_duration_us":466340,"total_duration_us":10004257}],"total_count":15}} WARN [2022-01-10 05:41:21,524] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":345189,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":5617447,"total_server_duration_us":129272,"encode_duration_us":17457,"operation_id":"0x3bda","timeout_ms":10000,"last_server_duration_us":64473,"total_duration_us":10010733},{"operation_name":"upsert","last_dispatch_duration_us":9151710,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9151710,"total_server_duration_us":560280,"encode_duration_us":19367,"operation_id":"0x3c60","timeout_ms":10000,"last_server_duration_us":560280,"total_duration_us":10010317},{"operation_name":"upsert","last_dispatch_duration_us":137598,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":3616024,"total_server_duration_us":380853,"encode_duration_us":18766,"operation_id":"0x3c61","timeout_ms":10000,"last_server_duration_us":125257,"total_duration_us":10009065},{"operation_name":"upsert","last_dispatch_duration_us":7037597,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7037597,"total_server_duration_us":134514,"encode_duration_us":16434,"operation_id":"0x3b6e","timeout_ms":10000,"last_server_duration_us":134514,"total_duration_us":10008163},{"operation_name":"upsert","last_dispatch_duration_us":72089,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":4355349,"total_server_duration_us":2275617,"encode_duration_us":16790,"operation_id":"0x3c50","timeout_ms":10000,"last_server_duration_us":32161,"total_duration_us":10007640},{"operation_name":"upsert","last_dispatch_duration_us":719082,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10528992,"total_server_duration_us":205306,"encode_duration_us":20108,"operation_id":"0x3ada","timeout_ms":10000,"last_server_duration_us":46626,"total_duration_us":10007454},{"operation_name":"upsert","last_dispatch_duration_us":10071459,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":16691986,"total_server_duration_us":540122,"encode_duration_us":23538,"operation_id":"0x3b39","timeout_ms":10000,"last_server_duration_us":260284,"total_duration_us":10006959},{"operation_name":"upsert","last_dispatch_duration_us":9395738,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10806491,"total_server_duration_us":107641,"encode_duration_us":19794,"operation_id":"0x3aad","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10002947},{"operation_name":"upsert","last_dispatch_duration_us":569607,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":5908485,"total_server_duration_us":498839,"encode_duration_us":16676,"operation_id":"0x3bcd","timeout_ms":10000,"last_server_duration_us":479963,"total_duration_us":10002766},{"operation_name":"upsert","last_dispatch_duration_us":8336567,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11449651,"total_server_duration_us":454697,"encode_duration_us":18336,"operation_id":"0x3b3b","timeout_ms":10000,"last_server_duration_us":25625,"total_duration_us":10002467}],"total_count":18}} WARN [2022-01-10 05:41:21,525] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10249991,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10785882,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9176205,"encode_duration_us":16665,"operation_id":"0x3b60","timeout_ms":10000,"last_server_duration_us":9089436,"total_duration_us":10005933},{"operation_name":"upsert","last_dispatch_duration_us":9617035,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":13076004,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":11141396,"encode_duration_us":16716,"operation_id":"0x3bcf","timeout_ms":10000,"last_server_duration_us":9262491,"total_duration_us":10010014}],"total_count":2}} WARN [2022-01-10 05:41:21,622] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16721947,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":16721947,"total_server_duration_us":779379,"operation_id":"0x38fb","timeout_ms":30000,"last_server_duration_us":779379,"total_duration_us":21886500},{"operation_name":"get","last_dispatch_duration_us":15637496,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":15637496,"total_server_duration_us":31203,"operation_id":"0x3a40","timeout_ms":30000,"last_server_duration_us":31203,"total_duration_us":15639283},{"operation_name":"get","last_dispatch_duration_us":12427199,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12427199,"total_server_duration_us":27,"operation_id":"0x3ad4","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":12427471},{"operation_name":"get","last_dispatch_duration_us":11617652,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11617652,"total_server_duration_us":27,"operation_id":"0x3aff","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":11617844},{"operation_name":"upsert","last_dispatch_duration_us":368023,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":4420464,"total_server_duration_us":1732930,"encode_duration_us":19207,"operation_id":"0x3c67","timeout_ms":10000,"last_server_duration_us":45838,"total_duration_us":10010704},{"operation_name":"upsert","last_dispatch_duration_us":7970515,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":14574231,"total_server_duration_us":997240,"encode_duration_us":17502,"operation_id":"0x3ad8","timeout_ms":10000,"last_server_duration_us":32548,"total_duration_us":10009490},{"operation_name":"upsert","last_dispatch_duration_us":54668,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8570985,"total_server_duration_us":228025,"encode_duration_us":17444,"operation_id":"0x3c8a","timeout_ms":10000,"last_server_duration_us":30350,"total_duration_us":10006363},{"operation_name":"upsert","last_dispatch_duration_us":7838245,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10982177,"total_server_duration_us":924551,"encode_duration_us":18073,"operation_id":"0x3bd7","timeout_ms":10000,"last_server_duration_us":21649,"total_duration_us":10005864},{"operation_name":"upsert","last_dispatch_duration_us":9283154,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11930673,"total_server_duration_us":9672916,"encode_duration_us":16622,"operation_id":"0x3b73","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10003997},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":18545,"operation_id":"0x3ba7","timeout_ms":10000,"total_duration_us":10003281}],"total_count":19}} WARN [2022-01-10 05:41:21,623] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17712913,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":21709773,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9477855,"encode_duration_us":17318,"operation_id":"0x3938","timeout_ms":10000,"last_server_duration_us":9042674,"total_duration_us":10003137},{"operation_name":"upsert","last_dispatch_duration_us":16970843,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17032763,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":8320260,"encode_duration_us":16570,"operation_id":"0x39f8","timeout_ms":10000,"last_server_duration_us":8300294,"total_duration_us":10007188},{"operation_name":"upsert","last_dispatch_duration_us":9397461,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14753197,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9600136,"encode_duration_us":20097,"operation_id":"0x3a81","timeout_ms":10000,"last_server_duration_us":9067105,"total_duration_us":10007666},{"operation_name":"upsert","last_dispatch_duration_us":10220137,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10220137,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":262166,"encode_duration_us":16243,"operation_id":"0x3c94","timeout_ms":10000,"last_server_duration_us":262166,"total_duration_us":10010530}],"total_count":4}} WARN [2022-01-10 05:41:21,723] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9461354,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10019919,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9431961,"encode_duration_us":18370,"operation_id":"0x3b6d","timeout_ms":10000,"last_server_duration_us":9047983,"total_duration_us":10002442},{"operation_name":"upsert","last_dispatch_duration_us":32219273,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":33610038,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9786571,"encode_duration_us":17016,"operation_id":"0x35fe","timeout_ms":10000,"last_server_duration_us":9478049,"total_duration_us":10003100},{"operation_name":"upsert","last_dispatch_duration_us":9283154,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11930673,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9672916,"encode_duration_us":16622,"operation_id":"0x3b73","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10003997}],"total_count":3}} WARN [2022-01-10 05:41:21,725] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9870424,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9870424,"total_server_duration_us":38,"operation_id":"0x3ae3","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":11978077},{"operation_name":"upsert","last_dispatch_duration_us":9934599,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9934599,"total_server_duration_us":22971,"encode_duration_us":16253,"operation_id":"0x3b74","timeout_ms":10000,"last_server_duration_us":22971,"total_duration_us":10010831},{"operation_name":"upsert","last_dispatch_duration_us":90884,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":852352,"total_server_duration_us":466871,"encode_duration_us":16339,"operation_id":"0x3be7","timeout_ms":10000,"last_server_duration_us":22971,"total_duration_us":10010331},{"operation_name":"upsert","last_dispatch_duration_us":9617035,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":13076004,"total_server_duration_us":11141396,"encode_duration_us":16716,"operation_id":"0x3bcf","timeout_ms":10000,"last_server_duration_us":9262491,"total_duration_us":10010014},{"operation_name":"upsert","last_dispatch_duration_us":9384806,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13392570,"total_server_duration_us":315416,"encode_duration_us":16935,"operation_id":"0x3ad6","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10008646},{"operation_name":"upsert","last_dispatch_duration_us":9520590,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13658824,"total_server_duration_us":444860,"encode_duration_us":19423,"operation_id":"0x3acc","timeout_ms":10000,"last_server_duration_us":25537,"total_duration_us":10008092},{"operation_name":"upsert","last_dispatch_duration_us":859392,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10147489,"total_server_duration_us":946791,"encode_duration_us":19580,"operation_id":"0x3c34","timeout_ms":10000,"last_server_duration_us":268557,"total_duration_us":10007053},{"operation_name":"upsert","last_dispatch_duration_us":10249991,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10785882,"total_server_duration_us":9176205,"encode_duration_us":16665,"operation_id":"0x3b60","timeout_ms":10000,"last_server_duration_us":9089436,"total_duration_us":10005933},{"operation_name":"upsert","last_dispatch_duration_us":93302,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":1260701,"total_server_duration_us":823298,"encode_duration_us":17987,"operation_id":"0x3bab","timeout_ms":10000,"last_server_duration_us":23475,"total_duration_us":10005656},{"operation_name":"upsert","last_dispatch_duration_us":75388,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":680231,"total_server_duration_us":393530,"encode_duration_us":17114,"operation_id":"0x3c64","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10005581}],"total_count":18}} INFO [2022-01-10 05:41:21,862] jepsen worker 52 - jepsen.util 2212 :info :write [17 17] :RequestTimeoutException INFO [2022-01-10 05:41:21,863] jepsen worker 52 - jepsen.util 2302 :invoke :write [17 20] INFO [2022-01-10 05:41:22,211] jepsen worker 46 - jepsen.util 2206 :info :write [15 18] :RequestTimeoutException INFO [2022-01-10 05:41:22,212] jepsen worker 46 - jepsen.util 2296 :invoke :write [15 6] INFO [2022-01-10 05:41:22,261] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:22,261] jepsen worker nemesis - couchbase.util Rebalance check count: 236 INFO [2022-01-10 05:41:22,261] jepsen worker nemesis - couchbase.util Rebalance stuck count: 174 INFO [2022-01-10 05:41:22,462] jepsen worker 86 - jepsen.util 1706 :ok :read [28 13] INFO [2022-01-10 05:41:22,462] jepsen worker 86 - jepsen.util 1706 :invoke :read [28 nil] INFO [2022-01-10 05:41:22,479] jepsen worker 28 - jepsen.util 1828 :info :write [9 37] :SyncWriteAmbiguous INFO [2022-01-10 05:41:22,480] jepsen worker 28 - jepsen.util 1918 :invoke :read [9 nil] INFO [2022-01-10 05:41:22,822] jepsen worker 41 - jepsen.util 1751 :ok :read [13 15] INFO [2022-01-10 05:41:22,823] jepsen worker 41 - jepsen.util 1751 :invoke :read [13 nil] INFO [2022-01-10 05:41:22,889] jepsen worker 45 - jepsen.util 2115 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:22,889] jepsen worker 61 - jepsen.util 1951 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:22,889] jepsen worker 33 - jepsen.util 2013 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:22,891] jepsen worker 76 - jepsen.util 1426 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:22,891] jepsen worker 61 - jepsen.util 1951 :invoke :write [20 44] INFO [2022-01-10 05:41:22,892] jepsen worker 33 - jepsen.util 2013 :invoke :write [11 39] INFO [2022-01-10 05:41:22,892] jepsen worker 45 - jepsen.util 2115 :invoke :write [15 40] INFO [2022-01-10 05:41:22,893] jepsen worker 76 - jepsen.util 1426 :invoke :read [25 nil] INFO [2022-01-10 05:41:23,161] jepsen worker 66 - jepsen.util 1686 :info :write [22 38] :RequestTimeoutException INFO [2022-01-10 05:41:23,161] jepsen worker 66 - jepsen.util 1776 :invoke :read [22 nil] INFO [2022-01-10 05:41:23,267] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:23,267] jepsen worker nemesis - couchbase.util Rebalance check count: 237 INFO [2022-01-10 05:41:23,267] jepsen worker nemesis - couchbase.util Rebalance stuck count: 175 INFO [2022-01-10 05:41:23,280] jepsen worker 22 - jepsen.util 1642 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:23,280] jepsen worker 34 - jepsen.util 1924 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:23,280] jepsen worker 22 - jepsen.util 1642 :invoke :read [7 nil] INFO [2022-01-10 05:41:23,281] jepsen worker 34 - jepsen.util 1924 :invoke :write [11 34] INFO [2022-01-10 05:41:23,814] jepsen worker 63 - jepsen.util 1773 :info :write [21 13] :RequestTimeoutException INFO [2022-01-10 05:41:23,815] jepsen worker 63 - jepsen.util 1863 :invoke :write [21 13] INFO [2022-01-10 05:41:23,817] jepsen worker 20 - jepsen.util 1640 :fail :read [6 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:23,840] jepsen worker 20 - jepsen.util 1640 :invoke :write [6 44] INFO [2022-01-10 05:41:23,819] jepsen worker 76 - jepsen.util 1426 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:23,841] jepsen worker 76 - jepsen.util 1426 :invoke :write [25 18] INFO [2022-01-10 05:41:23,821] jepsen worker 1 - jepsen.util 1981 :info :write [0 40] :RequestTimeoutException INFO [2022-01-10 05:41:23,842] jepsen worker 1 - jepsen.util 2071 :invoke :read [0 nil] INFO [2022-01-10 05:41:23,961] jepsen worker 37 - jepsen.util 2017 :info :write [12 12] :SyncWriteAmbiguous INFO [2022-01-10 05:41:23,962] jepsen worker 37 - jepsen.util 2107 :invoke :read [12 nil] INFO [2022-01-10 05:41:24,186] jepsen worker 42 - jepsen.util 1932 :info :write [14 2] :RequestTimeoutException INFO [2022-01-10 05:41:24,187] jepsen worker 42 - jepsen.util 2022 :invoke :write [14 32] INFO [2022-01-10 05:41:24,191] jepsen worker 17 - jepsen.util 1907 :info :write [5 38] :RequestTimeoutException INFO [2022-01-10 05:41:24,192] jepsen worker 17 - jepsen.util 1997 :invoke :write [5 44] INFO [2022-01-10 05:41:24,273] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:24,273] jepsen worker nemesis - couchbase.util Rebalance check count: 238 INFO [2022-01-10 05:41:24,273] jepsen worker nemesis - couchbase.util Rebalance stuck count: 176 INFO [2022-01-10 05:41:24,312] jepsen worker 12 - jepsen.util 1722 :info :write [4 14] :RequestTimeoutException INFO [2022-01-10 05:41:24,312] jepsen worker 12 - jepsen.util 1812 :invoke :read [4 nil] INFO [2022-01-10 05:41:24,327] jepsen worker 71 - jepsen.util 1691 :ok :read [23 26] INFO [2022-01-10 05:41:24,328] jepsen worker 71 - jepsen.util 1691 :invoke :read [23 nil] INFO [2022-01-10 05:41:24,368] jepsen worker 12 - jepsen.util 1812 :ok :read [4 3] INFO [2022-01-10 05:41:24,368] jepsen worker 12 - jepsen.util 1812 :invoke :read [4 nil] INFO [2022-01-10 05:41:24,371] jepsen worker 30 - jepsen.util 1650 :ok :read [10 37] INFO [2022-01-10 05:41:24,372] jepsen worker 30 - jepsen.util 1650 :invoke :write [10 47] INFO [2022-01-10 05:41:24,385] jepsen worker 11 - jepsen.util 1811 :ok :read [3 2] INFO [2022-01-10 05:41:24,385] jepsen worker 11 - jepsen.util 1811 :invoke :write [3 14] INFO [2022-01-10 05:41:24,418] jepsen worker 12 - jepsen.util 1812 :ok :read [4 3] INFO [2022-01-10 05:41:24,419] jepsen worker 12 - jepsen.util 1812 :invoke :read [4 nil] INFO [2022-01-10 05:41:24,431] jepsen worker 55 - jepsen.util 2125 :info :write [18 30] :RequestTimeoutException INFO [2022-01-10 05:41:24,432] jepsen worker 55 - jepsen.util 2215 :invoke :write [18 15] INFO [2022-01-10 05:41:24,459] jepsen worker 12 - jepsen.util 1812 :ok :read [4 3] INFO [2022-01-10 05:41:24,460] jepsen worker 12 - jepsen.util 1812 :invoke :write [4 28] INFO [2022-01-10 05:41:24,587] jepsen worker 62 - jepsen.util 2042 :info :write [20 26] :RequestTimeoutException INFO [2022-01-10 05:41:24,587] jepsen worker 62 - jepsen.util 2132 :invoke :write [20 22] INFO [2022-01-10 05:41:24,647] jepsen worker 73 - jepsen.util 1873 :info :write [24 31] :RequestTimeoutException INFO [2022-01-10 05:41:24,647] jepsen worker 73 - jepsen.util 1963 :invoke :read [24 nil] INFO [2022-01-10 05:41:24,666] jepsen worker 35 - jepsen.util 1655 :info :write [11 30] :RequestTimeoutException INFO [2022-01-10 05:41:24,667] jepsen worker 35 - jepsen.util 1745 :invoke :write [11 26] INFO [2022-01-10 05:41:24,847] jepsen worker 65 - jepsen.util 1775 :info :write [21 0] :RequestTimeoutException INFO [2022-01-10 05:41:24,850] jepsen worker 65 - jepsen.util 1865 :invoke :write [21 8] INFO [2022-01-10 05:41:24,855] jepsen worker 82 - jepsen.util 1792 :info :write [27 9] :RequestTimeoutException INFO [2022-01-10 05:41:24,855] jepsen worker 82 - jepsen.util 1882 :invoke :read [27 nil] INFO [2022-01-10 05:41:24,856] jepsen worker 7 - jepsen.util 1987 :info :write [2 1] :RequestTimeoutException INFO [2022-01-10 05:41:24,857] jepsen worker 7 - jepsen.util 2077 :invoke :read [2 nil] INFO [2022-01-10 05:41:24,866] jepsen worker 8 - jepsen.util 2078 :info :write [2 30] :RequestTimeoutException INFO [2022-01-10 05:41:24,866] jepsen worker 60 - jepsen.util 2310 :info :write [20 10] :RequestTimeoutException INFO [2022-01-10 05:41:24,872] jepsen worker 8 - jepsen.util 2168 :invoke :write [2 36] INFO [2022-01-10 05:41:24,872] jepsen worker 60 - jepsen.util 2400 :invoke :read [20 nil] INFO [2022-01-10 05:41:24,986] jepsen worker 21 - jepsen.util 2001 :info :write [7 45] :RequestTimeoutException INFO [2022-01-10 05:41:24,986] jepsen worker 43 - jepsen.util 1843 :info :write [14 2] :RequestTimeoutException INFO [2022-01-10 05:41:24,987] jepsen worker 24 - jepsen.util 1824 :info :write [8 37] :RequestTimeoutException INFO [2022-01-10 05:41:24,987] jepsen worker 24 - jepsen.util 1914 :invoke :read [8 nil] INFO [2022-01-10 05:41:24,988] jepsen worker 43 - jepsen.util 1933 :invoke :write [14 13] INFO [2022-01-10 05:41:24,989] jepsen worker 21 - jepsen.util 2091 :invoke :read [7 nil] INFO [2022-01-10 05:41:25,030] jepsen worker 1 - jepsen.util 2071 :ok :read [0 46] INFO [2022-01-10 05:41:25,030] jepsen worker 1 - jepsen.util 2071 :invoke :read [0 nil] INFO [2022-01-10 05:41:25,128] jepsen worker 1 - jepsen.util 2071 :ok :read [0 46] INFO [2022-01-10 05:41:25,129] jepsen worker 1 - jepsen.util 2071 :invoke :read [0 nil] INFO [2022-01-10 05:41:25,212] jepsen worker 1 - jepsen.util 2071 :ok :read [0 46] INFO [2022-01-10 05:41:25,213] jepsen worker 1 - jepsen.util 2071 :invoke :write [0 46] INFO [2022-01-10 05:41:25,216] jepsen worker 54 - jepsen.util 1854 :info :write [18 25] :SyncWriteAmbiguous INFO [2022-01-10 05:41:25,216] jepsen worker 54 - jepsen.util 1944 :invoke :read [18 nil] INFO [2022-01-10 05:41:25,252] jepsen worker 54 - jepsen.util 1944 :ok :read [18 14] INFO [2022-01-10 05:41:25,252] jepsen worker 54 - jepsen.util 1944 :invoke :read [18 nil] INFO [2022-01-10 05:41:25,284] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:25,285] jepsen worker nemesis - couchbase.util Rebalance check count: 239 INFO [2022-01-10 05:41:25,285] jepsen worker nemesis - couchbase.util Rebalance stuck count: 177 INFO [2022-01-10 05:41:25,286] jepsen worker 54 - jepsen.util 1944 :ok :read [18 14] INFO [2022-01-10 05:41:25,287] jepsen worker 54 - jepsen.util 1944 :invoke :write [18 46] INFO [2022-01-10 05:41:25,311] jepsen worker 88 - jepsen.util 1978 :ok :read [29 0] INFO [2022-01-10 05:41:25,311] jepsen worker 88 - jepsen.util 1978 :invoke :read [29 nil] INFO [2022-01-10 05:41:25,357] jepsen worker 87 - jepsen.util 2157 :ok :read [29 0] INFO [2022-01-10 05:41:25,360] jepsen worker 87 - jepsen.util 2157 :invoke :read [29 nil] INFO [2022-01-10 05:41:25,394] jepsen worker 88 - jepsen.util 1978 :ok :read [29 0] INFO [2022-01-10 05:41:25,395] jepsen worker 88 - jepsen.util 1978 :invoke :read [29 nil] INFO [2022-01-10 05:41:25,447] jepsen worker 87 - jepsen.util 2157 :ok :read [29 0] INFO [2022-01-10 05:41:25,447] jepsen worker 87 - jepsen.util 2157 :invoke :read [29 nil] INFO [2022-01-10 05:41:25,491] jepsen worker 88 - jepsen.util 1978 :ok :read [29 0] INFO [2022-01-10 05:41:25,492] jepsen worker 88 - jepsen.util 1978 :invoke :write [29 34] INFO [2022-01-10 05:41:25,539] jepsen worker 5 - jepsen.util 1985 :info :write [1 4] :RequestTimeoutException INFO [2022-01-10 05:41:25,540] jepsen worker 5 - jepsen.util 2075 :invoke :write [1 28] INFO [2022-01-10 05:41:25,550] jepsen worker 87 - jepsen.util 2157 :ok :read [29 0] INFO [2022-01-10 05:41:25,550] jepsen worker 87 - jepsen.util 2157 :invoke :read [29 nil] INFO [2022-01-10 05:41:25,591] jepsen worker 80 - jepsen.util 2060 :info :write [26 5] :RequestTimeoutException INFO [2022-01-10 05:41:25,592] jepsen worker 80 - jepsen.util 2150 :invoke :read [26 nil] INFO [2022-01-10 05:41:25,671] jepsen worker 87 - jepsen.util 2157 :ok :read [29 0] INFO [2022-01-10 05:41:25,672] jepsen worker 31 - jepsen.util 2191 :info :write [10 15] :RequestTimeoutException INFO [2022-01-10 05:41:25,672] jepsen worker 87 - jepsen.util 2157 :invoke :read [29 nil] INFO [2022-01-10 05:41:25,672] jepsen worker 31 - jepsen.util 2281 :invoke :read [10 nil] INFO [2022-01-10 05:41:25,681] jepsen worker 80 - jepsen.util 2150 :ok :read [26 10] INFO [2022-01-10 05:41:25,682] jepsen worker 80 - jepsen.util 2150 :invoke :read [26 nil] INFO [2022-01-10 05:41:25,682] jepsen worker 9 - jepsen.util 2169 :info :write [3 39] :RequestTimeoutException INFO [2022-01-10 05:41:25,687] jepsen worker 9 - jepsen.util 2259 :invoke :read [3 nil] INFO [2022-01-10 05:41:25,702] jepsen worker 3 - jepsen.util 1983 :info :write [1 47] :RequestTimeoutException INFO [2022-01-10 05:41:25,703] jepsen worker 3 - jepsen.util 2073 :invoke :write [1 30] INFO [2022-01-10 05:41:25,791] jepsen worker 80 - jepsen.util 2150 :ok :read [26 10] INFO [2022-01-10 05:41:25,792] jepsen worker 80 - jepsen.util 2150 :invoke :write [26 49] INFO [2022-01-10 05:41:25,795] jepsen worker 87 - jepsen.util 2157 :ok :read [29 0] INFO [2022-01-10 05:41:25,796] jepsen worker 87 - jepsen.util 2157 :invoke :write [29 34] INFO [2022-01-10 05:41:25,811] jepsen worker 69 - jepsen.util 1959 :info :write [23 13] :RequestTimeoutException INFO [2022-01-10 05:41:25,812] jepsen worker 69 - jepsen.util 2049 :invoke :read [23 nil] INFO [2022-01-10 05:41:25,832] jepsen worker 15 - jepsen.util 1635 :info :write [5 13] :RequestTimeoutException INFO [2022-01-10 05:41:25,832] jepsen worker 15 - jepsen.util 1725 :invoke :read [5 nil] INFO [2022-01-10 05:41:25,834] jepsen worker 9 - jepsen.util 2259 :ok :read [3 2] INFO [2022-01-10 05:41:25,835] jepsen worker 9 - jepsen.util 2259 :invoke :write [3 12] INFO [2022-01-10 05:41:25,882] jepsen worker 70 - jepsen.util 2230 :info :write [23 35] :RequestTimeoutException INFO [2022-01-10 05:41:25,883] jepsen worker 70 - jepsen.util 2320 :invoke :write [23 46] INFO [2022-01-10 05:41:25,988] jepsen worker 22 - jepsen.util 1642 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:25,988] jepsen worker 25 - jepsen.util 1915 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:25,990] jepsen worker 22 - jepsen.util 1642 :invoke :write [7 21] INFO [2022-01-10 05:41:25,990] jepsen worker 25 - jepsen.util 1915 :invoke :write [8 3] INFO [2022-01-10 05:41:26,035] jepsen worker 36 - jepsen.util 1656 :info :write [12 3] :RequestTimeoutException INFO [2022-01-10 05:41:26,035] jepsen worker 36 - jepsen.util 1746 :invoke :read [12 nil] INFO [2022-01-10 05:41:26,050] jepsen worker 2 - jepsen.util 1982 :info :write [0 28] :RequestTimeoutException INFO [2022-01-10 05:41:26,051] jepsen worker 2 - jepsen.util 2072 :invoke :read [0 nil] INFO [2022-01-10 05:41:26,139] jepsen worker 56 - jepsen.util 2126 :info :write [18 0] :RequestTimeoutException INFO [2022-01-10 05:41:26,140] jepsen worker 56 - jepsen.util 2216 :invoke :read [18 nil] INFO [2022-01-10 05:41:26,145] jepsen worker 24 - jepsen.util 1914 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:26,146] jepsen worker 24 - jepsen.util 1914 :invoke :read [8 nil] INFO [2022-01-10 05:41:26,163] jepsen worker 23 - jepsen.util 2093 :ok :read [7 :nil] INFO [2022-01-10 05:41:26,164] jepsen worker 23 - jepsen.util 2093 :invoke :write [7 24] INFO [2022-01-10 05:41:26,170] jepsen worker 27 - jepsen.util 1827 :info :write [9 12] :RequestTimeoutException INFO [2022-01-10 05:41:26,171] jepsen worker 27 - jepsen.util 1917 :invoke :read [9 nil] INFO [2022-01-10 05:41:26,176] jepsen worker 14 - jepsen.util 2084 :info :write [4 28] :RequestTimeoutException INFO [2022-01-10 05:41:26,177] jepsen worker 6 - jepsen.util 1896 :ok :read [2 14] INFO [2022-01-10 05:41:26,177] jepsen worker 14 - jepsen.util 2174 :invoke :write [4 28] INFO [2022-01-10 05:41:26,177] jepsen worker 6 - jepsen.util 1896 :invoke :write [2 22] INFO [2022-01-10 05:41:26,291] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:26,291] jepsen worker nemesis - couchbase.util Rebalance check count: 240 INFO [2022-01-10 05:41:26,291] jepsen worker nemesis - couchbase.util Rebalance stuck count: 178 INFO [2022-01-10 05:41:26,296] jepsen worker 10 - jepsen.util 2080 :info :write [3 18] :RequestTimeoutException INFO [2022-01-10 05:41:26,297] jepsen worker 10 - jepsen.util 2170 :invoke :write [3 18] INFO [2022-01-10 05:41:26,363] jepsen worker 7 - jepsen.util 2077 :ok :read [2 14] INFO [2022-01-10 05:41:26,364] jepsen worker 7 - jepsen.util 2077 :invoke :read [2 nil] INFO [2022-01-10 05:41:26,401] jepsen worker 4 - jepsen.util 1984 :info :write [1 15] :RequestTimeoutException INFO [2022-01-10 05:41:26,402] jepsen worker 4 - jepsen.util 2074 :invoke :write [1 16] INFO [2022-01-10 05:41:27,084] jepsen worker 21 - jepsen.util 2091 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:27,085] jepsen worker 21 - jepsen.util 2091 :invoke :read [7 nil] INFO [2022-01-10 05:41:27,140] jepsen worker 53 - jepsen.util 2123 :info :write [17 20] :RequestTimeoutException INFO [2022-01-10 05:41:27,140] jepsen worker 53 - jepsen.util 2213 :invoke :read [17 nil] INFO [2022-01-10 05:41:27,213] jepsen worker 7 - jepsen.util 2077 :ok :read [2 14] INFO [2022-01-10 05:41:27,214] jepsen worker 7 - jepsen.util 2077 :invoke :write [2 19] INFO [2022-01-10 05:41:27,297] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:27,297] jepsen worker nemesis - couchbase.util Rebalance check count: 241 INFO [2022-01-10 05:41:27,297] jepsen worker nemesis - couchbase.util Rebalance stuck count: 179 INFO [2022-01-10 05:41:27,460] jepsen worker 53 - jepsen.util 2213 :ok :read [17 41] INFO [2022-01-10 05:41:27,461] jepsen worker 53 - jepsen.util 2213 :invoke :read [17 nil] INFO [2022-01-10 05:41:27,544] jepsen worker 53 - jepsen.util 2213 :ok :read [17 41] INFO [2022-01-10 05:41:27,544] jepsen worker 53 - jepsen.util 2213 :invoke :read [17 nil] INFO [2022-01-10 05:41:27,572] jepsen worker 85 - jepsen.util 2245 :info :write [28 15] :RequestTimeoutException INFO [2022-01-10 05:41:27,573] jepsen worker 85 - jepsen.util 2335 :invoke :read [28 nil] INFO [2022-01-10 05:41:27,660] jepsen worker 53 - jepsen.util 2213 :ok :read [17 41] INFO [2022-01-10 05:41:27,661] jepsen worker 53 - jepsen.util 2213 :invoke :write [17 7] INFO [2022-01-10 05:41:27,686] jepsen worker 13 - jepsen.util 1813 :info :write [4 22] :RequestTimeoutException INFO [2022-01-10 05:41:27,687] jepsen worker 13 - jepsen.util 1903 :invoke :read [4 nil] INFO [2022-01-10 05:41:27,711] jepsen worker 13 - jepsen.util 1903 :ok :read [4 3] INFO [2022-01-10 05:41:27,712] jepsen worker 13 - jepsen.util 1903 :invoke :write [4 12] INFO [2022-01-10 05:41:27,731] jepsen worker 0 - jepsen.util 1890 :info :write [0 1] :RequestTimeoutException INFO [2022-01-10 05:41:27,732] jepsen worker 0 - jepsen.util 1980 :invoke :read [0 nil] INFO [2022-01-10 05:41:27,775] jepsen worker 38 - jepsen.util 2108 :info :write [12 43] :RequestTimeoutException INFO [2022-01-10 05:41:27,776] jepsen worker 38 - jepsen.util 2198 :invoke :read [12 nil] INFO [2022-01-10 05:41:27,856] jepsen worker 81 - jepsen.util 1971 :info :write [27 37] :RequestTimeoutException INFO [2022-01-10 05:41:27,857] jepsen worker 81 - jepsen.util 2061 :invoke :write [27 44] INFO [2022-01-10 05:41:27,900] jepsen worker 16 - jepsen.util 2356 :info :write [5 46] :RequestTimeoutException INFO [2022-01-10 05:41:27,900] jepsen worker 16 - jepsen.util 2446 :invoke :read [5 nil] INFO [2022-01-10 05:41:28,303] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:28,303] jepsen worker nemesis - couchbase.util Rebalance check count: 242 INFO [2022-01-10 05:41:28,303] jepsen worker nemesis - couchbase.util Rebalance stuck count: 180 INFO [2022-01-10 05:41:28,626] jepsen worker 32 - jepsen.util 2012 :info :write [10 47] :RequestTimeoutException INFO [2022-01-10 05:41:28,626] jepsen worker 32 - jepsen.util 2102 :invoke :read [10 nil] INFO [2022-01-10 05:41:28,681] jepsen worker 59 - jepsen.util 2039 :info :write [19 22] :RequestTimeoutException INFO [2022-01-10 05:41:28,682] jepsen worker 59 - jepsen.util 2129 :invoke :write [19 34] INFO [2022-01-10 05:41:28,946] jepsen worker 67 - jepsen.util 1867 :ok :read [22 41] INFO [2022-01-10 05:41:28,955] jepsen worker 67 - jepsen.util 1867 :invoke :read [22 nil] INFO [2022-01-10 05:41:29,007] jepsen worker 21 - jepsen.util 2091 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:29,008] jepsen worker 21 - jepsen.util 2091 :invoke :write [7 24] INFO [2022-01-10 05:41:29,192] jepsen worker 58 - jepsen.util 2038 :info :write [19 11] :SyncWriteAmbiguous INFO [2022-01-10 05:41:29,192] jepsen worker 58 - jepsen.util 2128 :invoke :write [19 30] INFO [2022-01-10 05:41:29,309] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:29,309] jepsen worker nemesis - couchbase.util Rebalance check count: 243 INFO [2022-01-10 05:41:29,309] jepsen worker nemesis - couchbase.util Rebalance stuck count: 181 WARN [2022-01-10 05:41:29,834] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":19142715,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":25453497,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9415552,"encode_duration_us":16343,"operation_id":"0x3a20","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10001151},{"operation_name":"upsert","last_dispatch_duration_us":9100063,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18177652,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9056271,"encode_duration_us":17625,"operation_id":"0x3ba8","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10006249},{"operation_name":"upsert","last_dispatch_duration_us":17766442,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18660161,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9351667,"encode_duration_us":19570,"operation_id":"0x3c59","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10001924},{"operation_name":"upsert","last_dispatch_duration_us":12786285,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18689317,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10297602,"encode_duration_us":28107,"operation_id":"0x3b3c","timeout_ms":10000,"last_server_duration_us":9851529,"total_duration_us":10010625},{"operation_name":"upsert","last_dispatch_duration_us":18813343,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":23611471,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9801035,"encode_duration_us":16235,"operation_id":"0x3a88","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10006693},{"operation_name":"upsert","last_dispatch_duration_us":9418092,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10869817,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":10497856,"encode_duration_us":16498,"operation_id":"0x3c0b","timeout_ms":10000,"last_server_duration_us":9336620,"total_duration_us":10003624}],"total_count":6}} INFO [2022-01-10 05:41:29,836] jepsen worker 78 - jepsen.util 2058 :info :write [26 20] :RequestTimeoutException INFO [2022-01-10 05:41:29,836] jepsen worker 78 - jepsen.util 2148 :invoke :write [26 18] WARN [2022-01-10 05:41:30,034] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16549072,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":16549072,"total_server_duration_us":6384,"operation_id":"0x3cc3","timeout_ms":30000,"last_server_duration_us":6384,"total_duration_us":16551493},{"operation_name":"upsert","last_dispatch_duration_us":131236,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9793115,"total_server_duration_us":264946,"encode_duration_us":22393,"operation_id":"0x3d98","timeout_ms":10000,"last_server_duration_us":23475,"total_duration_us":10010325},{"operation_name":"upsert","last_dispatch_duration_us":341901,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":4491983,"total_server_duration_us":1879787,"encode_duration_us":18502,"operation_id":"0x3c88","timeout_ms":10000,"last_server_duration_us":331330,"total_duration_us":10008639},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","encode_duration_us":18453,"operation_id":"0x3d0e","timeout_ms":10000,"total_duration_us":10008609},{"operation_name":"upsert","last_dispatch_duration_us":42007,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8576041,"total_server_duration_us":269634,"encode_duration_us":21353,"operation_id":"0x3de8","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10002644},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":20187,"operation_id":"0x3d52","timeout_ms":10000,"total_duration_us":10002622},{"operation_name":"upsert","last_dispatch_duration_us":17766442,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":18660161,"total_server_duration_us":9351667,"encode_duration_us":19570,"operation_id":"0x3c59","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10001924},{"operation_name":"upsert","last_dispatch_duration_us":9146017,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9757498,"total_server_duration_us":9216121,"encode_duration_us":17860,"operation_id":"0x3d65","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":9852455},{"operation_name":"get","last_dispatch_duration_us":8714155,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8714155,"total_server_duration_us":43,"operation_id":"0x3dab","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":8715414},{"operation_name":"get","last_dispatch_duration_us":4681537,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":4681537,"total_server_duration_us":38,"operation_id":"0x3e55","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":4683386}],"total_count":10}} INFO [2022-01-10 05:41:30,080] jepsen worker 73 - jepsen.util 1963 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:30,080] jepsen worker 64 - jepsen.util 2044 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:30,080] jepsen worker 60 - jepsen.util 2400 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:30,080] jepsen worker 74 - jepsen.util 2504 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:30,080] jepsen worker 73 - jepsen.util 1963 :invoke :write [24 26] INFO [2022-01-10 05:41:30,081] jepsen worker 60 - jepsen.util 2400 :invoke :write [20 29] INFO [2022-01-10 05:41:30,081] jepsen worker 64 - jepsen.util 2044 :invoke :write [21 10] INFO [2022-01-10 05:41:30,082] jepsen worker 74 - jepsen.util 2504 :invoke :read [24 nil] INFO [2022-01-10 05:41:30,142] jepsen worker 83 - jepsen.util 2243 :info :write [27 16] :RequestTimeoutException INFO [2022-01-10 05:41:30,143] jepsen worker 83 - jepsen.util 2333 :invoke :write [27 23] INFO [2022-01-10 05:41:30,171] jepsen worker 68 - jepsen.util 1958 :info :write [22 29] :RequestTimeoutException INFO [2022-01-10 05:41:30,171] jepsen worker 68 - jepsen.util 2048 :invoke :write [22 6] INFO [2022-01-10 05:41:30,200] jepsen worker 49 - jepsen.util 1759 :info :write [16 26] :RequestTimeoutException INFO [2022-01-10 05:41:30,200] jepsen worker 49 - jepsen.util 1849 :invoke :read [16 nil] INFO [2022-01-10 05:41:30,314] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:30,314] jepsen worker nemesis - couchbase.util Rebalance check count: 244 INFO [2022-01-10 05:41:30,315] jepsen worker nemesis - couchbase.util Rebalance stuck count: 182 INFO [2022-01-10 05:41:30,361] jepsen worker 19 - jepsen.util 1999 :info :write [6 16] :RequestTimeoutException INFO [2022-01-10 05:41:30,362] jepsen worker 19 - jepsen.util 2089 :invoke :write [6 15] INFO [2022-01-10 05:41:30,622] jepsen worker 72 - jepsen.util 1962 :info :write [24 35] :RequestTimeoutException INFO [2022-01-10 05:41:30,623] jepsen worker 72 - jepsen.util 2052 :invoke :write [24 10] INFO [2022-01-10 05:41:30,651] jepsen worker 44 - jepsen.util 1934 :info :write [14 45] :RequestTimeoutException INFO [2022-01-10 05:41:30,652] jepsen worker 44 - jepsen.util 2024 :invoke :read [14 nil] INFO [2022-01-10 05:41:30,730] jepsen worker 51 - jepsen.util 2031 :info :write [17 4] :RequestTimeoutException INFO [2022-01-10 05:41:30,730] jepsen worker 51 - jepsen.util 2121 :invoke :read [17 nil] INFO [2022-01-10 05:41:30,762] jepsen worker 89 - jepsen.util 2159 :info :write [29 20] :RequestTimeoutException INFO [2022-01-10 05:41:30,763] jepsen worker 89 - jepsen.util 2249 :invoke :write [29 12] INFO [2022-01-10 05:41:31,047] jepsen worker 79 - jepsen.util 2149 :info :write [26 8] :RequestTimeoutException INFO [2022-01-10 05:41:31,047] jepsen worker 79 - jepsen.util 2239 :invoke :read [26 nil] INFO [2022-01-10 05:41:31,275] jepsen worker 77 - jepsen.util 2237 :info :write [25 29] :RequestTimeoutException INFO [2022-01-10 05:41:31,276] jepsen worker 77 - jepsen.util 2327 :invoke :read [25 nil] INFO [2022-01-10 05:41:31,329] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:31,329] jepsen worker nemesis - couchbase.util Rebalance check count: 245 INFO [2022-01-10 05:41:31,329] jepsen worker nemesis - couchbase.util Rebalance stuck count: 183 INFO [2022-01-10 05:41:31,340] jepsen worker 26 - jepsen.util 1646 :info :write [8 6] :RequestTimeoutException INFO [2022-01-10 05:41:31,341] jepsen worker 26 - jepsen.util 1736 :invoke :read [8 nil] INFO [2022-01-10 05:41:31,386] jepsen worker 24 - jepsen.util 1914 :ok :read [8 27] INFO [2022-01-10 05:41:31,386] jepsen worker 24 - jepsen.util 1914 :invoke :read [8 nil] WARN [2022-01-10 05:41:31,437] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9318024,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15226510,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9561693,"encode_duration_us":16676,"operation_id":"0x3bcd","timeout_ms":10000,"last_server_duration_us":9062854,"total_duration_us":10002766},{"operation_name":"upsert","last_dispatch_duration_us":9075244,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14692691,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9159210,"encode_duration_us":17457,"operation_id":"0x3bda","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10010733}],"total_count":2}} WARN [2022-01-10 05:41:31,437] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9191649,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18596680,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9105060,"encode_duration_us":17666,"operation_id":"0x3b93","timeout_ms":10000,"last_server_duration_us":9066042,"total_duration_us":10000681},{"operation_name":"upsert","last_dispatch_duration_us":9253824,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18107669,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9267112,"encode_duration_us":20275,"operation_id":"0x3b38","timeout_ms":10000,"last_server_duration_us":9068168,"total_duration_us":10004932}],"total_count":2}} WARN [2022-01-10 05:41:31,437] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7418773,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7418773,"total_server_duration_us":611660,"encode_duration_us":16940,"operation_id":"0x3cc2","timeout_ms":10000,"last_server_duration_us":611660,"total_duration_us":10010225},{"operation_name":"upsert","last_dispatch_duration_us":2616950,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9524601,"total_server_duration_us":640576,"encode_duration_us":17920,"operation_id":"0x3e54","timeout_ms":10000,"last_server_duration_us":153489,"total_duration_us":10008573},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":19212,"operation_id":"0x3d1d","timeout_ms":10000,"total_duration_us":10008324},{"operation_name":"upsert","last_dispatch_duration_us":140951,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":2658762,"total_server_duration_us":2162483,"encode_duration_us":16207,"operation_id":"0x3d6b","timeout_ms":10000,"last_server_duration_us":108076,"total_duration_us":10007976},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":16970,"operation_id":"0x3e00","timeout_ms":10000,"total_duration_us":10007687},{"operation_name":"upsert","last_dispatch_duration_us":58888,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":58888,"total_server_duration_us":30445,"encode_duration_us":17896,"operation_id":"0x3e4f","timeout_ms":10000,"last_server_duration_us":30445,"total_duration_us":10004739},{"operation_name":"upsert","last_dispatch_duration_us":2778133,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8544938,"total_server_duration_us":790455,"encode_duration_us":16476,"operation_id":"0x3e51","timeout_ms":10000,"last_server_duration_us":200008,"total_duration_us":10003873},{"operation_name":"upsert","last_dispatch_duration_us":116322,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9067138,"total_server_duration_us":102056,"encode_duration_us":18099,"operation_id":"0x3d15","timeout_ms":10000,"last_server_duration_us":29415,"total_duration_us":10002984},{"operation_name":"upsert","last_dispatch_duration_us":93240,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9657292,"total_server_duration_us":229672,"encode_duration_us":21279,"operation_id":"0x3cee","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10002740},{"operation_name":"upsert","last_dispatch_duration_us":113457,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":113457,"total_server_duration_us":23138,"encode_duration_us":16539,"operation_id":"0x3d8c","timeout_ms":10000,"last_server_duration_us":23138,"total_duration_us":10002673}],"total_count":16}} INFO [2022-01-10 05:41:31,500] jepsen worker 47 - jepsen.util 1937 :info :write [15 0] :RequestTimeoutException INFO [2022-01-10 05:41:31,501] jepsen worker 47 - jepsen.util 2027 :invoke :write [15 41] INFO [2022-01-10 05:41:31,501] jepsen worker 57 - jepsen.util 2127 :info :write [19 39] :RequestTimeoutException INFO [2022-01-10 05:41:31,502] jepsen worker 57 - jepsen.util 2217 :invoke :write [19 20] WARN [2022-01-10 05:41:31,537] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":4932993,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4932993,"total_server_duration_us":235906,"encode_duration_us":16744,"operation_id":"0x3d59","timeout_ms":10000,"last_server_duration_us":235906,"total_duration_us":10011090},{"operation_name":"upsert","last_dispatch_duration_us":43504,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":865727,"total_server_duration_us":506765,"encode_duration_us":17437,"operation_id":"0x3da9","timeout_ms":10000,"last_server_duration_us":22141,"total_duration_us":10010179},{"operation_name":"upsert","last_dispatch_duration_us":579839,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":5576864,"total_server_duration_us":1783799,"encode_duration_us":19266,"operation_id":"0x3e44","timeout_ms":10000,"last_server_duration_us":106144,"total_duration_us":10009302},{"operation_name":"upsert","last_dispatch_duration_us":6474838,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6474838,"total_server_duration_us":21649,"encode_duration_us":16541,"operation_id":"0x3d5a","timeout_ms":10000,"last_server_duration_us":21649,"total_duration_us":10005880},{"operation_name":"upsert","last_dispatch_duration_us":6849230,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6849230,"total_server_duration_us":22637,"encode_duration_us":27440,"operation_id":"0x3d5b","timeout_ms":10000,"last_server_duration_us":22637,"total_duration_us":10005771},{"operation_name":"upsert","last_dispatch_duration_us":172979,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":3406633,"total_server_duration_us":1095291,"encode_duration_us":16719,"operation_id":"0x3da6","timeout_ms":10000,"last_server_duration_us":46287,"total_duration_us":10005422},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":29525,"operation_id":"0x3df6","timeout_ms":10000,"total_duration_us":10003370},{"operation_name":"upsert","last_dispatch_duration_us":3128705,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12217624,"total_server_duration_us":299749,"encode_duration_us":16642,"operation_id":"0x3d1f","timeout_ms":10000,"last_server_duration_us":60002,"total_duration_us":10001244},{"operation_name":"get","last_dispatch_duration_us":8740996,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8740996,"total_server_duration_us":100,"operation_id":"0x3cfb","timeout_ms":30000,"last_server_duration_us":100,"total_duration_us":8743155},{"operation_name":"get","last_dispatch_duration_us":4172121,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4172121,"total_server_duration_us":3647,"operation_id":"0x3d58","timeout_ms":30000,"last_server_duration_us":3647,"total_duration_us":8319638}],"total_count":15}} WARN [2022-01-10 05:41:31,539] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9057641,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9570306,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9287938,"encode_duration_us":19047,"operation_id":"0x3e50","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10002030}],"total_count":1}} WARN [2022-01-10 05:41:31,635] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":5033641,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":5033641,"total_server_duration_us":145,"operation_id":"0x3d64","timeout_ms":30000,"last_server_duration_us":145,"total_duration_us":14792712},{"operation_name":"get","last_dispatch_duration_us":11335965,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11335965,"total_server_duration_us":6927,"operation_id":"0x3d4e","timeout_ms":30000,"last_server_duration_us":6927,"total_duration_us":11345134},{"operation_name":"get","last_dispatch_duration_us":11330705,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11330705,"total_server_duration_us":7028,"operation_id":"0x3d4f","timeout_ms":30000,"last_server_duration_us":7028,"total_duration_us":11336408},{"operation_name":"upsert","last_dispatch_duration_us":1059947,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1173211,"total_server_duration_us":443479,"encode_duration_us":16475,"operation_id":"0x3d26","timeout_ms":10000,"last_server_duration_us":379006,"total_duration_us":10010356},{"operation_name":"upsert","last_dispatch_duration_us":13447792,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":13447792,"total_server_duration_us":190041,"encode_duration_us":29787,"operation_id":"0x3d53","timeout_ms":10000,"last_server_duration_us":190041,"total_duration_us":10009525},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":17753,"operation_id":"0x3dff","timeout_ms":10000,"total_duration_us":10008651},{"operation_name":"upsert","last_dispatch_duration_us":699815,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":699815,"total_server_duration_us":274773,"encode_duration_us":17485,"operation_id":"0x3d2e","timeout_ms":10000,"last_server_duration_us":274773,"total_duration_us":10007875},{"operation_name":"upsert","last_dispatch_duration_us":12733154,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12733154,"total_server_duration_us":192512,"encode_duration_us":30893,"operation_id":"0x3d51","timeout_ms":10000,"last_server_duration_us":192512,"total_duration_us":10007315},{"operation_name":"upsert","last_dispatch_duration_us":14907214,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":14907214,"total_server_duration_us":225645,"encode_duration_us":27469,"operation_id":"0x3d54","timeout_ms":10000,"last_server_duration_us":225645,"total_duration_us":10006447},{"operation_name":"upsert","last_dispatch_duration_us":11586397,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11586397,"total_server_duration_us":81971,"encode_duration_us":17653,"operation_id":"0x3d50","timeout_ms":10000,"last_server_duration_us":81971,"total_duration_us":10002987}],"total_count":20}} WARN [2022-01-10 05:41:31,636] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":84075,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":3319301,"total_server_duration_us":1605727,"encode_duration_us":16925,"operation_id":"0x3cb9","timeout_ms":10000,"last_server_duration_us":27945,"total_duration_us":10010152},{"operation_name":"upsert","last_dispatch_duration_us":45982,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":1634989,"total_server_duration_us":628626,"encode_duration_us":19575,"operation_id":"0x3d6f","timeout_ms":10000,"last_server_duration_us":32548,"total_duration_us":10009451},{"operation_name":"upsert","last_dispatch_duration_us":628094,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":766186,"total_server_duration_us":42953,"encode_duration_us":17060,"operation_id":"0x3e4a","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10009403},{"operation_name":"upsert","last_dispatch_duration_us":392327,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":3191455,"total_server_duration_us":2637957,"encode_duration_us":16341,"operation_id":"0x3d23","timeout_ms":10000,"last_server_duration_us":378731,"total_duration_us":10008496},{"operation_name":"upsert","last_dispatch_duration_us":618854,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8388191,"total_server_duration_us":163158,"encode_duration_us":16286,"operation_id":"0x3dd3","timeout_ms":10000,"last_server_duration_us":35412,"total_duration_us":10008470},{"operation_name":"upsert","last_dispatch_duration_us":67963,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":1724134,"total_server_duration_us":567836,"encode_duration_us":18235,"operation_id":"0x3d70","timeout_ms":10000,"last_server_duration_us":35011,"total_duration_us":10008196},{"operation_name":"upsert","last_dispatch_duration_us":8731115,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8731115,"total_server_duration_us":20282,"encode_duration_us":16279,"operation_id":"0x3c98","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10007518},{"operation_name":"upsert","last_dispatch_duration_us":52917,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":1578263,"total_server_duration_us":555826,"encode_duration_us":21568,"operation_id":"0x3d95","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10005410},{"operation_name":"upsert","last_dispatch_duration_us":4182944,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10341425,"total_server_duration_us":128323,"encode_duration_us":17387,"operation_id":"0x3e53","timeout_ms":10000,"last_server_duration_us":24497,"total_duration_us":10005047},{"operation_name":"upsert","last_dispatch_duration_us":32186,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":910220,"total_server_duration_us":527702,"encode_duration_us":16407,"operation_id":"0x3e6a","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10003052}],"total_count":19}} WARN [2022-01-10 05:41:31,636] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17571049,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19793264,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9766243,"encode_duration_us":17665,"operation_id":"0x3b61","timeout_ms":10000,"last_server_duration_us":9069231,"total_duration_us":10004257}],"total_count":1}} WARN [2022-01-10 05:41:31,736] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10012146,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10012146,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9832820,"encode_duration_us":18156,"operation_id":"0x3d2d","timeout_ms":10000,"last_server_duration_us":9832820,"total_duration_us":10001394},{"operation_name":"upsert","last_dispatch_duration_us":18045462,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18045462,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9027816,"encode_duration_us":18545,"operation_id":"0x3ba7","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10003281}],"total_count":2}} WARN [2022-01-10 05:41:31,740] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8419111,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9242054,"total_server_duration_us":473950,"encode_duration_us":17041,"operation_id":"0x3d99","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10009026},{"operation_name":"upsert","last_dispatch_duration_us":8689475,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9438897,"total_server_duration_us":433391,"encode_duration_us":21211,"operation_id":"0x3df7","timeout_ms":10000,"last_server_duration_us":21813,"total_duration_us":10005684},{"operation_name":"upsert","last_dispatch_duration_us":65098,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8683632,"total_server_duration_us":245691,"encode_duration_us":20479,"operation_id":"0x3da4","timeout_ms":10000,"last_server_duration_us":22637,"total_duration_us":10005458},{"operation_name":"upsert","last_dispatch_duration_us":8747247,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9588597,"total_server_duration_us":444414,"encode_duration_us":17007,"operation_id":"0x3da5","timeout_ms":10000,"last_server_duration_us":24154,"total_duration_us":10004081},{"operation_name":"upsert","last_dispatch_duration_us":230769,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":5805550,"total_server_duration_us":1466556,"encode_duration_us":18192,"operation_id":"0x3e63","timeout_ms":10000,"last_server_duration_us":83413,"total_duration_us":10004021},{"operation_name":"upsert","last_dispatch_duration_us":91042,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8843365,"total_server_duration_us":161471,"encode_duration_us":18262,"operation_id":"0x3d69","timeout_ms":10000,"last_server_duration_us":21001,"total_duration_us":10003414},{"operation_name":"upsert","last_dispatch_duration_us":9057641,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9570306,"total_server_duration_us":9287938,"encode_duration_us":19047,"operation_id":"0x3e50","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10002030},{"operation_name":"upsert","last_dispatch_duration_us":710295,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7937098,"total_server_duration_us":2626848,"encode_duration_us":17412,"operation_id":"0x3e83","timeout_ms":10000,"last_server_duration_us":326403,"total_duration_us":10001368},{"operation_name":"upsert","last_dispatch_duration_us":565125,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7773198,"total_server_duration_us":2377600,"encode_duration_us":17269,"operation_id":"0x3e82","timeout_ms":10000,"last_server_duration_us":149572,"total_duration_us":10001163},{"operation_name":"upsert","last_dispatch_duration_us":304407,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":7347320,"total_server_duration_us":1322809,"encode_duration_us":17239,"operation_id":"0x3dd2","timeout_ms":10000,"last_server_duration_us":39846,"total_duration_us":10001120}],"total_count":13}} INFO [2022-01-10 05:41:31,886] jepsen worker 52 - jepsen.util 2302 :info :write [17 20] :RequestTimeoutException INFO [2022-01-10 05:41:31,887] jepsen worker 52 - jepsen.util 2392 :invoke :read [17 nil] INFO [2022-01-10 05:41:31,968] jepsen worker 40 - jepsen.util 1570 :ok :read [13 15] INFO [2022-01-10 05:41:31,968] jepsen worker 40 - jepsen.util 1570 :invoke :read [13 nil] INFO [2022-01-10 05:41:32,002] jepsen worker 29 - jepsen.util 2009 :ok :read [9 22] INFO [2022-01-10 05:41:32,002] jepsen worker 27 - jepsen.util 1917 :ok :read [9 22] INFO [2022-01-10 05:41:32,003] jepsen worker 29 - jepsen.util 2009 :invoke :write [9 36] INFO [2022-01-10 05:41:32,003] jepsen worker 27 - jepsen.util 1917 :invoke :write [9 14] INFO [2022-01-10 05:41:32,011] jepsen worker 84 - jepsen.util 2154 :ok :read [28 13] INFO [2022-01-10 05:41:32,012] jepsen worker 84 - jepsen.util 2154 :invoke :write [28 37] INFO [2022-01-10 05:41:32,059] jepsen worker 41 - jepsen.util 1751 :ok :read [13 15] INFO [2022-01-10 05:41:32,060] jepsen worker 41 - jepsen.util 1751 :invoke :read [13 nil] INFO [2022-01-10 05:41:32,116] jepsen worker 41 - jepsen.util 1751 :ok :read [13 15] INFO [2022-01-10 05:41:32,117] jepsen worker 41 - jepsen.util 1751 :invoke :read [13 nil] INFO [2022-01-10 05:41:32,149] jepsen worker 50 - jepsen.util 2210 :ok :read [16 38] INFO [2022-01-10 05:41:32,150] jepsen worker 50 - jepsen.util 2210 :invoke :write [16 1] INFO [2022-01-10 05:41:32,150] jepsen worker 41 - jepsen.util 1751 :ok :read [13 15] INFO [2022-01-10 05:41:32,151] jepsen worker 41 - jepsen.util 1751 :invoke :read [13 nil] INFO [2022-01-10 05:41:32,162] jepsen worker 86 - jepsen.util 1706 :ok :read [28 13] INFO [2022-01-10 05:41:32,163] jepsen worker 86 - jepsen.util 1706 :invoke :read [28 nil] INFO [2022-01-10 05:41:32,178] jepsen worker 66 - jepsen.util 1776 :ok :read [22 41] INFO [2022-01-10 05:41:32,179] jepsen worker 66 - jepsen.util 1776 :invoke :read [22 nil] INFO [2022-01-10 05:41:32,180] jepsen worker 41 - jepsen.util 1751 :ok :read [13 15] INFO [2022-01-10 05:41:32,181] jepsen worker 41 - jepsen.util 1751 :invoke :write [13 20] INFO [2022-01-10 05:41:32,219] jepsen worker 82 - jepsen.util 1882 :ok :read [27 30] INFO [2022-01-10 05:41:32,219] jepsen worker 82 - jepsen.util 1882 :invoke :write [27 19] INFO [2022-01-10 05:41:32,231] jepsen worker 46 - jepsen.util 2296 :info :write [15 6] :RequestTimeoutException INFO [2022-01-10 05:41:32,232] jepsen worker 46 - jepsen.util 2386 :invoke :write [15 7] INFO [2022-01-10 05:41:32,255] jepsen worker 85 - jepsen.util 2335 :ok :read [28 13] INFO [2022-01-10 05:41:32,256] jepsen worker 85 - jepsen.util 2335 :invoke :write [28 39] INFO [2022-01-10 05:41:32,307] jepsen worker 49 - jepsen.util 1849 :ok :read [16 38] INFO [2022-01-10 05:41:32,308] jepsen worker 49 - jepsen.util 1849 :invoke :write [16 44] INFO [2022-01-10 05:41:32,336] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:32,336] jepsen worker nemesis - couchbase.util Rebalance check count: 246 INFO [2022-01-10 05:41:32,336] jepsen worker nemesis - couchbase.util Rebalance stuck count: 184 INFO [2022-01-10 05:41:32,357] jepsen worker 40 - jepsen.util 1570 :ok :read [13 15] INFO [2022-01-10 05:41:32,358] jepsen worker 40 - jepsen.util 1570 :invoke :read [13 nil] INFO [2022-01-10 05:41:32,365] jepsen worker 77 - jepsen.util 2327 :ok :read [25 :nil] INFO [2022-01-10 05:41:32,365] jepsen worker 77 - jepsen.util 2327 :invoke :read [25 nil] INFO [2022-01-10 05:41:32,408] jepsen worker 86 - jepsen.util 1706 :ok :read [28 13] INFO [2022-01-10 05:41:32,408] jepsen worker 86 - jepsen.util 1706 :invoke :read [28 nil] INFO [2022-01-10 05:41:32,463] jepsen worker 66 - jepsen.util 1776 :ok :read [22 41] INFO [2022-01-10 05:41:32,463] jepsen worker 66 - jepsen.util 1776 :invoke :write [22 24] INFO [2022-01-10 05:41:32,557] jepsen worker 51 - jepsen.util 2121 :ok :read [17 41] INFO [2022-01-10 05:41:32,558] jepsen worker 51 - jepsen.util 2121 :invoke :read [17 nil] INFO [2022-01-10 05:41:32,561] jepsen worker 24 - jepsen.util 1914 :ok :read [8 27] INFO [2022-01-10 05:41:32,562] jepsen worker 24 - jepsen.util 1914 :invoke :write [8 49] INFO [2022-01-10 05:41:32,563] jepsen worker 79 - jepsen.util 2239 :ok :read [26 10] INFO [2022-01-10 05:41:32,563] jepsen worker 79 - jepsen.util 2239 :invoke :read [26 nil] INFO [2022-01-10 05:41:32,611] jepsen worker 79 - jepsen.util 2239 :ok :read [26 10] INFO [2022-01-10 05:41:32,612] jepsen worker 79 - jepsen.util 2239 :invoke :read [26 nil] INFO [2022-01-10 05:41:32,619] jepsen worker 52 - jepsen.util 2392 :ok :read [17 41] INFO [2022-01-10 05:41:32,620] jepsen worker 52 - jepsen.util 2392 :invoke :write [17 20] INFO [2022-01-10 05:41:32,620] jepsen worker 51 - jepsen.util 2121 :ok :read [17 41] INFO [2022-01-10 05:41:32,621] jepsen worker 51 - jepsen.util 2121 :invoke :read [17 nil] INFO [2022-01-10 05:41:32,672] jepsen worker 51 - jepsen.util 2121 :ok :read [17 41] INFO [2022-01-10 05:41:32,673] jepsen worker 51 - jepsen.util 2121 :invoke :write [17 12] INFO [2022-01-10 05:41:32,679] jepsen worker 79 - jepsen.util 2239 :ok :read [26 10] INFO [2022-01-10 05:41:32,679] jepsen worker 79 - jepsen.util 2239 :invoke :write [26 1] INFO [2022-01-10 05:41:32,859] jepsen worker 77 - jepsen.util 2327 :ok :read [25 :nil] INFO [2022-01-10 05:41:32,860] jepsen worker 77 - jepsen.util 2327 :invoke :read [25 nil] INFO [2022-01-10 05:41:32,912] jepsen worker 61 - jepsen.util 1951 :info :write [20 44] :RequestTimeoutException INFO [2022-01-10 05:41:32,912] jepsen worker 45 - jepsen.util 2115 :info :write [15 40] :RequestTimeoutException INFO [2022-01-10 05:41:32,913] jepsen worker 45 - jepsen.util 2205 :invoke :write [15 39] INFO [2022-01-10 05:41:32,913] jepsen worker 61 - jepsen.util 2041 :invoke :write [20 33] INFO [2022-01-10 05:41:32,931] jepsen worker 33 - jepsen.util 2013 :info :write [11 39] :RequestTimeoutException INFO [2022-01-10 05:41:32,932] jepsen worker 33 - jepsen.util 2103 :invoke :write [11 49] INFO [2022-01-10 05:41:33,211] jepsen worker 77 - jepsen.util 2327 :ok :read [25 :nil] INFO [2022-01-10 05:41:33,211] jepsen worker 77 - jepsen.util 2327 :invoke :write [25 41] INFO [2022-01-10 05:41:33,316] jepsen worker 34 - jepsen.util 1924 :info :write [11 34] :RequestTimeoutException INFO [2022-01-10 05:41:33,316] jepsen worker 34 - jepsen.util 2014 :invoke :write [11 40] INFO [2022-01-10 05:41:33,342] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:33,342] jepsen worker nemesis - couchbase.util Rebalance check count: 247 INFO [2022-01-10 05:41:33,342] jepsen worker nemesis - couchbase.util Rebalance stuck count: 185 INFO [2022-01-10 05:41:33,488] jepsen worker 37 - jepsen.util 2107 :ok :read [12 42] INFO [2022-01-10 05:41:33,489] jepsen worker 37 - jepsen.util 2107 :invoke :read [12 nil] INFO [2022-01-10 05:41:33,492] jepsen worker 71 - jepsen.util 1691 :ok :read [23 26] INFO [2022-01-10 05:41:33,494] jepsen worker 71 - jepsen.util 1691 :invoke :read [23 nil] INFO [2022-01-10 05:41:33,545] jepsen worker 56 - jepsen.util 2216 :ok :read [18 14] INFO [2022-01-10 05:41:33,546] jepsen worker 56 - jepsen.util 2216 :invoke :read [18 nil] INFO [2022-01-10 05:41:33,557] jepsen worker 32 - jepsen.util 2102 :ok :read [10 37] INFO [2022-01-10 05:41:33,557] jepsen worker 32 - jepsen.util 2102 :invoke :write [10 15] INFO [2022-01-10 05:41:33,601] jepsen worker 56 - jepsen.util 2216 :ok :read [18 14] INFO [2022-01-10 05:41:33,602] jepsen worker 56 - jepsen.util 2216 :invoke :write [18 30] INFO [2022-01-10 05:41:33,678] jepsen worker 31 - jepsen.util 2281 :ok :read [10 37] INFO [2022-01-10 05:41:33,679] jepsen worker 31 - jepsen.util 2281 :invoke :write [10 33] INFO [2022-01-10 05:41:33,683] jepsen worker 36 - jepsen.util 1746 :ok :read [12 42] INFO [2022-01-10 05:41:33,684] jepsen worker 36 - jepsen.util 1746 :invoke :read [12 nil] INFO [2022-01-10 05:41:33,862] jepsen worker 20 - jepsen.util 1640 :info :write [6 44] :RequestTimeoutException INFO [2022-01-10 05:41:33,863] jepsen worker 20 - jepsen.util 1730 :invoke :write [6 9] INFO [2022-01-10 05:41:33,880] jepsen worker 63 - jepsen.util 1863 :info :write [21 13] :RequestTimeoutException INFO [2022-01-10 05:41:33,881] jepsen worker 63 - jepsen.util 1953 :invoke :read [21 nil] INFO [2022-01-10 05:41:33,881] jepsen worker 76 - jepsen.util 1426 :info :write [25 18] :RequestTimeoutException INFO [2022-01-10 05:41:33,883] jepsen worker 76 - jepsen.util 1516 :invoke :read [25 nil] INFO [2022-01-10 05:41:34,211] jepsen worker 42 - jepsen.util 2022 :info :write [14 32] :RequestTimeoutException INFO [2022-01-10 05:41:34,211] jepsen worker 42 - jepsen.util 2112 :invoke :read [14 nil] INFO [2022-01-10 05:41:34,214] jepsen worker 17 - jepsen.util 1997 :info :write [5 44] :RequestTimeoutException INFO [2022-01-10 05:41:34,276] jepsen worker 55 - jepsen.util 2215 :info :write [18 15] :SyncWriteAmbiguous INFO [2022-01-10 05:41:34,311] jepsen worker 55 - jepsen.util 2305 :invoke :read [18 nil] INFO [2022-01-10 05:41:34,311] jepsen worker 17 - jepsen.util 2087 :invoke :read [5 nil] INFO [2022-01-10 05:41:34,349] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:34,349] jepsen worker nemesis - couchbase.util Rebalance check count: 248 INFO [2022-01-10 05:41:34,349] jepsen worker nemesis - couchbase.util Rebalance stuck count: 186 INFO [2022-01-10 05:41:34,356] jepsen worker 69 - jepsen.util 2049 :ok :read [23 26] INFO [2022-01-10 05:41:34,357] jepsen worker 69 - jepsen.util 2049 :invoke :write [23 42] INFO [2022-01-10 05:41:34,406] jepsen worker 30 - jepsen.util 1650 :info :write [10 47] :RequestTimeoutException INFO [2022-01-10 05:41:34,407] jepsen worker 30 - jepsen.util 1740 :invoke :read [10 nil] INFO [2022-01-10 05:41:34,416] jepsen worker 11 - jepsen.util 1811 :info :write [3 14] :RequestTimeoutException INFO [2022-01-10 05:41:34,417] jepsen worker 11 - jepsen.util 1901 :invoke :read [3 nil] INFO [2022-01-10 05:41:34,442] jepsen worker 11 - jepsen.util 1901 :ok :read [3 2] INFO [2022-01-10 05:41:34,442] jepsen worker 11 - jepsen.util 1901 :invoke :read [3 nil] INFO [2022-01-10 05:41:34,474] jepsen worker 11 - jepsen.util 1901 :ok :read [3 2] INFO [2022-01-10 05:41:34,474] jepsen worker 11 - jepsen.util 1901 :invoke :write [3 39] INFO [2022-01-10 05:41:34,476] jepsen worker 30 - jepsen.util 1740 :ok :read [10 37] INFO [2022-01-10 05:41:34,477] jepsen worker 30 - jepsen.util 1740 :invoke :read [10 nil] INFO [2022-01-10 05:41:34,486] jepsen worker 12 - jepsen.util 1812 :info :write [4 28] :RequestTimeoutException INFO [2022-01-10 05:41:34,487] jepsen worker 12 - jepsen.util 1902 :invoke :read [4 nil] INFO [2022-01-10 05:41:34,507] jepsen worker 30 - jepsen.util 1740 :ok :read [10 37] INFO [2022-01-10 05:41:34,508] jepsen worker 30 - jepsen.util 1740 :invoke :write [10 26] INFO [2022-01-10 05:41:34,551] jepsen worker 12 - jepsen.util 1902 :ok :read [4 3] INFO [2022-01-10 05:41:34,551] jepsen worker 12 - jepsen.util 1902 :invoke :read [4 nil] INFO [2022-01-10 05:41:34,588] jepsen worker 12 - jepsen.util 1902 :ok :read [4 3] INFO [2022-01-10 05:41:34,589] jepsen worker 12 - jepsen.util 1902 :invoke :read [4 nil] INFO [2022-01-10 05:41:34,611] jepsen worker 40 - jepsen.util 1570 :ok :read [13 15] INFO [2022-01-10 05:41:34,611] jepsen worker 62 - jepsen.util 2132 :info :write [20 22] :RequestTimeoutException INFO [2022-01-10 05:41:34,612] jepsen worker 40 - jepsen.util 1570 :invoke :write [13 40] INFO [2022-01-10 05:41:34,613] jepsen worker 62 - jepsen.util 2222 :invoke :write [20 27] INFO [2022-01-10 05:41:34,613] jepsen worker 12 - jepsen.util 1902 :ok :read [4 3] INFO [2022-01-10 05:41:34,615] jepsen worker 12 - jepsen.util 1902 :invoke :read [4 nil] INFO [2022-01-10 05:41:34,649] jepsen worker 86 - jepsen.util 1706 :ok :read [28 13] INFO [2022-01-10 05:41:34,650] jepsen worker 86 - jepsen.util 1706 :invoke :read [28 nil] INFO [2022-01-10 05:41:34,671] jepsen worker 12 - jepsen.util 1902 :ok :read [4 3] INFO [2022-01-10 05:41:34,671] jepsen worker 12 - jepsen.util 1902 :invoke :read [4 nil] INFO [2022-01-10 05:41:34,691] jepsen worker 35 - jepsen.util 1745 :info :write [11 26] :RequestTimeoutException INFO [2022-01-10 05:41:34,692] jepsen worker 35 - jepsen.util 1835 :invoke :read [11 nil] INFO [2022-01-10 05:41:34,708] jepsen worker 12 - jepsen.util 1902 :ok :read [4 3] INFO [2022-01-10 05:41:34,709] jepsen worker 12 - jepsen.util 1902 :invoke :write [4 48] INFO [2022-01-10 05:41:34,884] jepsen worker 65 - jepsen.util 1865 :info :write [21 8] :RequestTimeoutException INFO [2022-01-10 05:41:34,885] jepsen worker 65 - jepsen.util 1955 :invoke :write [21 39] INFO [2022-01-10 05:41:34,891] jepsen worker 8 - jepsen.util 2168 :info :write [2 36] :RequestTimeoutException INFO [2022-01-10 05:41:34,892] jepsen worker 8 - jepsen.util 2258 :invoke :write [2 34] INFO [2022-01-10 05:41:35,014] jepsen worker 43 - jepsen.util 1933 :info :write [14 13] :RequestTimeoutException INFO [2022-01-10 05:41:35,015] jepsen worker 43 - jepsen.util 2023 :invoke :read [14 nil] INFO [2022-01-10 05:41:35,242] jepsen worker 1 - jepsen.util 2071 :info :write [0 46] :RequestTimeoutException INFO [2022-01-10 05:41:35,243] jepsen worker 1 - jepsen.util 2161 :invoke :write [0 48] INFO [2022-01-10 05:41:35,309] jepsen worker 54 - jepsen.util 1944 :info :write [18 46] :RequestTimeoutException INFO [2022-01-10 05:41:35,310] jepsen worker 54 - jepsen.util 2034 :invoke :write [18 1] INFO [2022-01-10 05:41:35,355] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:35,356] jepsen worker nemesis - couchbase.util Rebalance check count: 249 INFO [2022-01-10 05:41:35,356] jepsen worker nemesis - couchbase.util Rebalance stuck count: 187 INFO [2022-01-10 05:41:35,494] jepsen worker 15 - jepsen.util 1725 :ok :read [5 42] INFO [2022-01-10 05:41:35,495] jepsen worker 15 - jepsen.util 1725 :invoke :write [5 19] INFO [2022-01-10 05:41:35,525] jepsen worker 88 - jepsen.util 1978 :info :write [29 34] :RequestTimeoutException INFO [2022-01-10 05:41:35,525] jepsen worker 88 - jepsen.util 2068 :invoke :read [29 nil] INFO [2022-01-10 05:41:35,572] jepsen worker 5 - jepsen.util 2075 :info :write [1 28] :RequestTimeoutException INFO [2022-01-10 05:41:35,573] jepsen worker 5 - jepsen.util 2165 :invoke :write [1 46] INFO [2022-01-10 05:41:35,726] jepsen worker 3 - jepsen.util 2073 :info :write [1 30] :RequestTimeoutException INFO [2022-01-10 05:41:35,727] jepsen worker 3 - jepsen.util 2163 :invoke :write [1 38] INFO [2022-01-10 05:41:35,815] jepsen worker 87 - jepsen.util 2157 :info :write [29 34] :RequestTimeoutException INFO [2022-01-10 05:41:35,816] jepsen worker 87 - jepsen.util 2247 :invoke :read [29 nil] INFO [2022-01-10 05:41:35,816] jepsen worker 80 - jepsen.util 2150 :info :write [26 49] :RequestTimeoutException INFO [2022-01-10 05:41:35,817] jepsen worker 80 - jepsen.util 2240 :invoke :read [26 nil] INFO [2022-01-10 05:41:35,838] jepsen worker 80 - jepsen.util 2240 :ok :read [26 10] INFO [2022-01-10 05:41:35,839] jepsen worker 80 - jepsen.util 2240 :invoke :read [26 nil] INFO [2022-01-10 05:41:35,855] jepsen worker 9 - jepsen.util 2259 :info :write [3 12] :RequestTimeoutException INFO [2022-01-10 05:41:35,855] jepsen worker 9 - jepsen.util 2349 :invoke :write [3 19] INFO [2022-01-10 05:41:35,869] jepsen worker 80 - jepsen.util 2240 :ok :read [26 10] INFO [2022-01-10 05:41:35,870] jepsen worker 80 - jepsen.util 2240 :invoke :read [26 nil] INFO [2022-01-10 05:41:35,903] jepsen worker 80 - jepsen.util 2240 :ok :read [26 10] INFO [2022-01-10 05:41:35,904] jepsen worker 80 - jepsen.util 2240 :invoke :read [26 nil] INFO [2022-01-10 05:41:35,907] jepsen worker 70 - jepsen.util 2320 :info :write [23 46] :RequestTimeoutException INFO [2022-01-10 05:41:35,907] jepsen worker 70 - jepsen.util 2410 :invoke :read [23 nil] INFO [2022-01-10 05:41:35,927] jepsen worker 80 - jepsen.util 2240 :ok :read [26 10] INFO [2022-01-10 05:41:35,927] jepsen worker 80 - jepsen.util 2240 :invoke :write [26 24] INFO [2022-01-10 05:41:36,016] jepsen worker 22 - jepsen.util 1642 :info :write [7 21] :RequestTimeoutException INFO [2022-01-10 05:41:36,016] jepsen worker 25 - jepsen.util 1915 :info :write [8 3] :RequestTimeoutException INFO [2022-01-10 05:41:36,016] jepsen worker 25 - jepsen.util 2005 :invoke :read [8 nil] INFO [2022-01-10 05:41:36,017] jepsen worker 22 - jepsen.util 1732 :invoke :write [7 22] INFO [2022-01-10 05:41:36,186] jepsen worker 23 - jepsen.util 2093 :info :write [7 24] :RequestTimeoutException INFO [2022-01-10 05:41:36,187] jepsen worker 23 - jepsen.util 2183 :invoke :read [7 nil] INFO [2022-01-10 05:41:36,196] jepsen worker 6 - jepsen.util 1896 :info :write [2 22] :RequestTimeoutException INFO [2022-01-10 05:41:36,197] jepsen worker 6 - jepsen.util 1986 :invoke :read [2 nil] INFO [2022-01-10 05:41:36,202] jepsen worker 14 - jepsen.util 2174 :info :write [4 28] :RequestTimeoutException INFO [2022-01-10 05:41:36,202] jepsen worker 14 - jepsen.util 2264 :invoke :write [4 40] INFO [2022-01-10 05:41:36,317] jepsen worker 10 - jepsen.util 2170 :info :write [3 18] :RequestTimeoutException INFO [2022-01-10 05:41:36,317] jepsen worker 10 - jepsen.util 2260 :invoke :read [3 nil] INFO [2022-01-10 05:41:36,362] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:36,362] jepsen worker nemesis - couchbase.util Rebalance check count: 250 INFO [2022-01-10 05:41:36,362] jepsen worker nemesis - couchbase.util Rebalance stuck count: 188 INFO [2022-01-10 05:41:36,421] jepsen worker 4 - jepsen.util 2074 :info :write [1 16] :RequestTimeoutException INFO [2022-01-10 05:41:36,422] jepsen worker 4 - jepsen.util 2164 :invoke :write [1 18] INFO [2022-01-10 05:41:36,591] jepsen worker 75 - jepsen.util 1335 :ok :read [25 :nil] INFO [2022-01-10 05:41:36,591] jepsen worker 75 - jepsen.util 1335 :invoke :read [25 nil] INFO [2022-01-10 05:41:36,724] jepsen worker 18 - jepsen.util 1548 :ok :read [6 25] INFO [2022-01-10 05:41:36,724] jepsen worker 18 - jepsen.util 1548 :invoke :write [6 43] INFO [2022-01-10 05:41:36,901] jepsen worker 26 - jepsen.util 1736 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:36,902] jepsen worker 26 - jepsen.util 1736 :invoke :write [8 25] INFO [2022-01-10 05:41:36,983] jepsen worker 0 - jepsen.util 1980 :ok :read [0 46] INFO [2022-01-10 05:41:36,983] jepsen worker 0 - jepsen.util 1980 :invoke :read [0 nil] INFO [2022-01-10 05:41:37,153] jepsen worker 74 - jepsen.util 2504 :ok :read [24 18] INFO [2022-01-10 05:41:37,154] jepsen worker 74 - jepsen.util 2504 :invoke :write [24 24] INFO [2022-01-10 05:41:37,240] jepsen worker 7 - jepsen.util 2077 :info :write [2 19] :RequestTimeoutException INFO [2022-01-10 05:41:37,241] jepsen worker 7 - jepsen.util 2167 :invoke :read [2 nil] INFO [2022-01-10 05:41:37,368] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:37,368] jepsen worker nemesis - couchbase.util Rebalance check count: 251 INFO [2022-01-10 05:41:37,368] jepsen worker nemesis - couchbase.util Rebalance stuck count: 189 INFO [2022-01-10 05:41:37,684] jepsen worker 53 - jepsen.util 2213 :info :write [17 7] :RequestTimeoutException INFO [2022-01-10 05:41:37,685] jepsen worker 53 - jepsen.util 2303 :invoke :write [17 22] INFO [2022-01-10 05:41:37,740] jepsen worker 13 - jepsen.util 1903 :info :write [4 12] :RequestTimeoutException INFO [2022-01-10 05:41:37,740] jepsen worker 13 - jepsen.util 1993 :invoke :write [4 17] INFO [2022-01-10 05:41:37,806] jepsen worker 0 - jepsen.util 1980 :ok :read [0 46] INFO [2022-01-10 05:41:37,806] jepsen worker 0 - jepsen.util 1980 :invoke :write [0 30] INFO [2022-01-10 05:41:37,881] jepsen worker 81 - jepsen.util 2061 :info :write [27 44] :RequestTimeoutException INFO [2022-01-10 05:41:37,882] jepsen worker 81 - jepsen.util 2151 :invoke :write [27 10] INFO [2022-01-10 05:41:38,292] jepsen worker 44 - jepsen.util 2024 :ok :read [14 14] INFO [2022-01-10 05:41:38,293] jepsen worker 44 - jepsen.util 2024 :invoke :read [14 nil] INFO [2022-01-10 05:41:38,374] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:38,374] jepsen worker nemesis - couchbase.util Rebalance check count: 252 INFO [2022-01-10 05:41:38,374] jepsen worker nemesis - couchbase.util Rebalance stuck count: 190 INFO [2022-01-10 05:41:38,595] jepsen worker 7 - jepsen.util 2167 :ok :read [2 14] INFO [2022-01-10 05:41:38,596] jepsen worker 7 - jepsen.util 2167 :invoke :write [2 48] INFO [2022-01-10 05:41:38,710] jepsen worker 59 - jepsen.util 2129 :info :write [19 34] :RequestTimeoutException INFO [2022-01-10 05:41:38,711] jepsen worker 59 - jepsen.util 2219 :invoke :read [19 nil] INFO [2022-01-10 05:41:38,835] jepsen worker 75 - jepsen.util 1335 :ok :read [25 :nil] INFO [2022-01-10 05:41:38,835] jepsen worker 75 - jepsen.util 1335 :invoke :write [25 34] INFO [2022-01-10 05:41:39,026] jepsen worker 21 - jepsen.util 2091 :info :write [7 24] :RequestTimeoutException INFO [2022-01-10 05:41:39,027] jepsen worker 21 - jepsen.util 2181 :invoke :read [7 nil] INFO [2022-01-10 05:41:39,212] jepsen worker 58 - jepsen.util 2128 :info :write [19 30] :RequestTimeoutException INFO [2022-01-10 05:41:39,213] jepsen worker 58 - jepsen.util 2218 :invoke :read [19 nil] INFO [2022-01-10 05:41:39,380] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:39,380] jepsen worker nemesis - couchbase.util Rebalance check count: 253 INFO [2022-01-10 05:41:39,380] jepsen worker nemesis - couchbase.util Rebalance stuck count: 191 INFO [2022-01-10 05:41:39,782] jepsen worker 39 - jepsen.util 2289 :ok :read [13 15] INFO [2022-01-10 05:41:39,783] jepsen worker 39 - jepsen.util 2289 :invoke :write [13 10] INFO [2022-01-10 05:41:39,857] jepsen worker 78 - jepsen.util 2148 :info :write [26 18] :RequestTimeoutException INFO [2022-01-10 05:41:39,857] jepsen worker 78 - jepsen.util 2238 :invoke :read [26 nil] WARN [2022-01-10 05:41:39,868] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":24891879,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":24891879,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":10807352,"encode_duration_us":20187,"operation_id":"0x3d52","timeout_ms":10000,"last_server_duration_us":10807352,"total_duration_us":10002622},{"operation_name":"upsert","last_dispatch_duration_us":23360418,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":26359986,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":12835379,"encode_duration_us":18082,"operation_id":"0x3bfe","timeout_ms":10000,"last_server_duration_us":12401341,"total_duration_us":10006041}],"total_count":2}} INFO [2022-01-10 05:41:39,879] jepsen worker 78 - jepsen.util 2238 :ok :read [26 10] INFO [2022-01-10 05:41:39,880] jepsen worker 78 - jepsen.util 2238 :invoke :read [26 nil] INFO [2022-01-10 05:41:39,902] jepsen worker 78 - jepsen.util 2238 :ok :read [26 10] INFO [2022-01-10 05:41:39,902] jepsen worker 78 - jepsen.util 2238 :invoke :read [26 nil] INFO [2022-01-10 05:41:39,931] jepsen worker 78 - jepsen.util 2238 :ok :read [26 10] INFO [2022-01-10 05:41:39,933] jepsen worker 78 - jepsen.util 2238 :invoke :write [26 3] INFO [2022-01-10 05:41:39,934] jepsen worker 48 - jepsen.util 1668 :ok :read [16 38] INFO [2022-01-10 05:41:39,950] jepsen worker 48 - jepsen.util 1668 :invoke :write [16 9] INFO [2022-01-10 05:41:39,952] jepsen worker 17 - jepsen.util 2087 :ok :read [5 42] INFO [2022-01-10 05:41:39,953] jepsen worker 17 - jepsen.util 2087 :invoke :read [5 nil] INFO [2022-01-10 05:41:39,960] jepsen worker 28 - jepsen.util 1918 :ok :read [9 22] INFO [2022-01-10 05:41:39,960] jepsen worker 28 - jepsen.util 1918 :invoke :write [9 35] WARN [2022-01-10 05:41:39,969] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":23903944,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":23903944,"total_server_duration_us":23,"operation_id":"0x3d94","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":23906280},{"operation_name":"get","last_dispatch_duration_us":22030122,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":22030122,"total_server_duration_us":4118,"operation_id":"0x3d25","timeout_ms":30000,"last_server_duration_us":4118,"total_duration_us":22030450},{"operation_name":"get","last_dispatch_duration_us":17055062,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":17055062,"total_server_duration_us":9058,"operation_id":"0x3e25","timeout_ms":30000,"last_server_duration_us":9058,"total_duration_us":17056987},{"operation_name":"upsert","last_dispatch_duration_us":47753,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":1470041,"total_server_duration_us":473216,"encode_duration_us":22450,"operation_id":"0x3f45","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10010097},{"operation_name":"upsert","encode_duration_us":16768,"operation_id":"0x3f34","timeout_ms":10000,"total_duration_us":10009011},{"operation_name":"upsert","last_dispatch_duration_us":1571432,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":5386837,"total_server_duration_us":1928037,"encode_duration_us":17001,"operation_id":"0x3faf","timeout_ms":10000,"last_server_duration_us":140962,"total_duration_us":10006025},{"operation_name":"upsert","last_dispatch_duration_us":12561862,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":12561862,"total_server_duration_us":522457,"encode_duration_us":27467,"operation_id":"0x3f30","timeout_ms":10000,"last_server_duration_us":522457,"total_duration_us":10005562},{"operation_name":"upsert","last_dispatch_duration_us":15631851,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":15631851,"total_server_duration_us":191687,"encode_duration_us":16985,"operation_id":"0x3e81","timeout_ms":10000,"last_server_duration_us":191687,"total_duration_us":10004796},{"operation_name":"upsert","last_dispatch_duration_us":125013,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":1595177,"total_server_duration_us":492406,"encode_duration_us":16575,"operation_id":"0x3f70","timeout_ms":10000,"last_server_duration_us":22720,"total_duration_us":10002889},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":19854,"operation_id":"0x3f03","timeout_ms":10000,"total_duration_us":10002097}],"total_count":14}} INFO [2022-01-10 05:41:40,106] jepsen worker 64 - jepsen.util 2044 :info :write [21 10] :RequestTimeoutException INFO [2022-01-10 05:41:40,106] jepsen worker 60 - jepsen.util 2400 :info :write [20 29] :RequestTimeoutException INFO [2022-01-10 05:41:40,107] jepsen worker 73 - jepsen.util 1963 :info :write [24 26] :RequestTimeoutException INFO [2022-01-10 05:41:40,107] jepsen worker 60 - jepsen.util 2490 :invoke :read [20 nil] INFO [2022-01-10 05:41:40,107] jepsen worker 64 - jepsen.util 2134 :invoke :read [21 nil] INFO [2022-01-10 05:41:40,108] jepsen worker 73 - jepsen.util 2053 :invoke :read [24 nil] INFO [2022-01-10 05:41:40,161] jepsen worker 83 - jepsen.util 2333 :info :write [27 23] :RequestTimeoutException INFO [2022-01-10 05:41:40,161] jepsen worker 83 - jepsen.util 2423 :invoke :write [27 48] INFO [2022-01-10 05:41:40,195] jepsen worker 68 - jepsen.util 2048 :info :write [22 6] :RequestTimeoutException INFO [2022-01-10 05:41:40,196] jepsen worker 68 - jepsen.util 2138 :invoke :write [22 35] INFO [2022-01-10 05:41:40,306] jepsen worker 2 - jepsen.util 2072 :ok :read [0 46] INFO [2022-01-10 05:41:40,307] jepsen worker 2 - jepsen.util 2072 :invoke :read [0 nil] INFO [2022-01-10 05:41:40,386] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:40,386] jepsen worker nemesis - couchbase.util Rebalance check count: 254 INFO [2022-01-10 05:41:40,386] jepsen worker nemesis - couchbase.util Rebalance stuck count: 192 INFO [2022-01-10 05:41:40,390] jepsen worker 19 - jepsen.util 2089 :info :write [6 15] :RequestTimeoutException INFO [2022-01-10 05:41:40,391] jepsen worker 19 - jepsen.util 2179 :invoke :write [6 19] INFO [2022-01-10 05:41:40,434] jepsen worker 59 - jepsen.util 2219 :ok :read [19 31] INFO [2022-01-10 05:41:40,435] jepsen worker 59 - jepsen.util 2219 :invoke :read [19 nil] INFO [2022-01-10 05:41:40,446] jepsen worker 86 - jepsen.util 1706 :ok :read [28 13] INFO [2022-01-10 05:41:40,446] jepsen worker 86 - jepsen.util 1706 :invoke :read [28 nil] INFO [2022-01-10 05:41:40,581] jepsen worker 17 - jepsen.util 2087 :ok :read [5 42] INFO [2022-01-10 05:41:40,582] jepsen worker 17 - jepsen.util 2087 :invoke :write [5 23] INFO [2022-01-10 05:41:40,642] jepsen worker 72 - jepsen.util 2052 :info :write [24 10] :RequestTimeoutException INFO [2022-01-10 05:41:40,643] jepsen worker 72 - jepsen.util 2142 :invoke :read [24 nil] INFO [2022-01-10 05:41:40,784] jepsen worker 89 - jepsen.util 2249 :info :write [29 12] :RequestTimeoutException INFO [2022-01-10 05:41:40,787] jepsen worker 89 - jepsen.util 2339 :invoke :write [29 38] INFO [2022-01-10 05:41:40,798] jepsen worker 59 - jepsen.util 2219 :ok :read [19 31] INFO [2022-01-10 05:41:40,798] jepsen worker 59 - jepsen.util 2219 :invoke :read [19 nil] INFO [2022-01-10 05:41:41,245] jepsen worker 44 - jepsen.util 2024 :ok :read [14 14] INFO [2022-01-10 05:41:41,246] jepsen worker 44 - jepsen.util 2024 :invoke :read [14 nil] INFO [2022-01-10 05:41:41,396] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:41,396] jepsen worker nemesis - couchbase.util Rebalance check count: 255 INFO [2022-01-10 05:41:41,396] jepsen worker nemesis - couchbase.util Rebalance stuck count: 193 WARN [2022-01-10 05:41:41,469] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17681348,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17794805,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9054137,"encode_duration_us":16539,"operation_id":"0x3d8c","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10002673},{"operation_name":"upsert","last_dispatch_duration_us":17455043,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17455043,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9026756,"encode_duration_us":19212,"operation_id":"0x3d1d","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10008324},{"operation_name":"upsert","last_dispatch_duration_us":17618108,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":22660391,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9217869,"encode_duration_us":18026,"operation_id":"0x3c74","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":10006512}],"total_count":3}} INFO [2022-01-10 05:41:41,521] jepsen worker 57 - jepsen.util 2217 :info :write [19 20] :RequestTimeoutException INFO [2022-01-10 05:41:41,522] jepsen worker 57 - jepsen.util 2307 :invoke :read [19 nil] INFO [2022-01-10 05:41:41,526] jepsen worker 47 - jepsen.util 2027 :info :write [15 41] :RequestTimeoutException INFO [2022-01-10 05:41:41,526] jepsen worker 47 - jepsen.util 2117 :invoke :read [15 nil] WARN [2022-01-10 05:41:41,569] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17625100,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":21241125,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9422465,"encode_duration_us":18766,"operation_id":"0x3c61","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":10009065},{"operation_name":"upsert","last_dispatch_duration_us":9096360,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9962087,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9579185,"encode_duration_us":17437,"operation_id":"0x3da9","timeout_ms":10000,"last_server_duration_us":9072420,"total_duration_us":10010179}],"total_count":2}} WARN [2022-01-10 05:41:41,570] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":30518,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":189840,"total_server_duration_us":95941,"encode_duration_us":16119,"operation_id":"0x3fd2","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10007374},{"operation_name":"upsert","last_dispatch_duration_us":1513919,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5644437,"total_server_duration_us":1076945,"encode_duration_us":17021,"operation_id":"0x3f0e","timeout_ms":10000,"last_server_duration_us":195621,"total_duration_us":10006532},{"operation_name":"upsert","last_dispatch_duration_us":5194932,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8337478,"total_server_duration_us":300550,"encode_duration_us":16497,"operation_id":"0x3f02","timeout_ms":10000,"last_server_duration_us":101381,"total_duration_us":10006490},{"operation_name":"upsert","last_dispatch_duration_us":1960276,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7832578,"total_server_duration_us":517338,"encode_duration_us":16617,"operation_id":"0x3eaa","timeout_ms":10000,"last_server_duration_us":256306,"total_duration_us":10001781},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":16968,"operation_id":"0x3f84","timeout_ms":10000,"total_duration_us":10001380},{"operation_name":"upsert","last_dispatch_duration_us":37095,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":302985,"total_server_duration_us":241579,"encode_duration_us":16316,"operation_id":"0x402c","timeout_ms":10000,"last_server_duration_us":22887,"total_duration_us":10000692},{"operation_name":"get","last_dispatch_duration_us":9219963,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9219963,"total_server_duration_us":326662,"operation_id":"0x3ecb","timeout_ms":30000,"last_server_duration_us":326662,"total_duration_us":9222387},{"operation_name":"get","last_dispatch_duration_us":7987830,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":7987830,"total_server_duration_us":62,"operation_id":"0x3f4a","timeout_ms":30000,"last_server_duration_us":62,"total_duration_us":7989722},{"operation_name":"get","last_dispatch_duration_us":7631979,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":7631979,"total_server_duration_us":27,"operation_id":"0x3f74","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":7633249},{"operation_name":"get","last_dispatch_duration_us":1587695,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1587695,"total_server_duration_us":11241,"operation_id":"0x3f7d","timeout_ms":30000,"last_server_duration_us":11241,"total_duration_us":5223516}],"total_count":12}} WARN [2022-01-10 05:41:41,571] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9993911,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10083245,"total_server_duration_us":9939938,"encode_duration_us":18189,"operation_id":"0x3f3e","timeout_ms":10000,"last_server_duration_us":9919893,"total_duration_us":10010396},{"operation_name":"upsert","last_dispatch_duration_us":3820312,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6919437,"total_server_duration_us":230431,"encode_duration_us":28812,"operation_id":"0x3ecf","timeout_ms":10000,"last_server_duration_us":65514,"total_duration_us":10010304},{"operation_name":"upsert","last_dispatch_duration_us":5814985,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":5814985,"total_server_duration_us":169585,"encode_duration_us":29593,"operation_id":"0x3efa","timeout_ms":10000,"last_server_duration_us":169585,"total_duration_us":10009992},{"operation_name":"upsert","last_dispatch_duration_us":30468,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":3621325,"total_server_duration_us":3140766,"encode_duration_us":16965,"operation_id":"0x3f80","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10007234},{"operation_name":"upsert","last_dispatch_duration_us":3969038,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":3969038,"total_server_duration_us":183719,"encode_duration_us":16383,"operation_id":"0x3f0c","timeout_ms":10000,"last_server_duration_us":183719,"total_duration_us":10007097},{"operation_name":"upsert","last_dispatch_duration_us":32646,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":1033002,"total_server_duration_us":590697,"encode_duration_us":25613,"operation_id":"0x3f47","timeout_ms":10000,"last_server_duration_us":20045,"total_duration_us":10005210},{"operation_name":"upsert","last_dispatch_duration_us":3939683,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":3939683,"total_server_duration_us":175300,"encode_duration_us":17210,"operation_id":"0x3ef8","timeout_ms":10000,"last_server_duration_us":175300,"total_duration_us":10004551},{"operation_name":"upsert","last_dispatch_duration_us":2007871,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10160382,"total_server_duration_us":1958680,"encode_duration_us":16340,"operation_id":"0x4002","timeout_ms":10000,"last_server_duration_us":1568308,"total_duration_us":10002745},{"operation_name":"upsert","last_dispatch_duration_us":3276811,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9323291,"total_server_duration_us":690911,"encode_duration_us":17997,"operation_id":"0x3ecd","timeout_ms":10000,"last_server_duration_us":152179,"total_duration_us":10002248},{"operation_name":"upsert","last_dispatch_duration_us":6275855,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12782668,"total_server_duration_us":3371275,"encode_duration_us":17142,"operation_id":"0x3ece","timeout_ms":10000,"last_server_duration_us":3032474,"total_duration_us":10002151}],"total_count":17}} WARN [2022-01-10 05:41:41,571] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9145841,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17989207,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9186105,"encode_duration_us":18262,"operation_id":"0x3d69","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10003414},{"operation_name":"upsert","last_dispatch_duration_us":17199808,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18052160,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9501054,"encode_duration_us":16339,"operation_id":"0x3be7","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10010331}],"total_count":2}} WARN [2022-01-10 05:41:41,669] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":25052,"operation_id":"0x3f07","timeout_ms":10000,"total_duration_us":10008650},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":21962,"operation_id":"0x3f08","timeout_ms":10000,"total_duration_us":10007678},{"operation_name":"upsert","last_dispatch_duration_us":1478937,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8710336,"total_server_duration_us":431150,"encode_duration_us":17028,"operation_id":"0x4023","timeout_ms":10000,"last_server_duration_us":243158,"total_duration_us":10007214},{"operation_name":"upsert","last_dispatch_duration_us":32170,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":4680469,"total_server_duration_us":1447245,"encode_duration_us":16610,"operation_id":"0x3f6c","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10006880},{"operation_name":"upsert","last_dispatch_duration_us":2722246,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10170197,"total_server_duration_us":335354,"encode_duration_us":17445,"operation_id":"0x4024","timeout_ms":10000,"last_server_duration_us":128369,"total_duration_us":10006505},{"operation_name":"upsert","last_dispatch_duration_us":1467436,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9024303,"total_server_duration_us":400055,"encode_duration_us":19066,"operation_id":"0x4025","timeout_ms":10000,"last_server_duration_us":300241,"total_duration_us":10006032},{"operation_name":"upsert","last_dispatch_duration_us":341583,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":5127810,"total_server_duration_us":2030666,"encode_duration_us":16418,"operation_id":"0x3ea3","timeout_ms":10000,"last_server_duration_us":69080,"total_duration_us":10005821},{"operation_name":"upsert","last_dispatch_duration_us":5405263,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11029247,"total_server_duration_us":282707,"encode_duration_us":16740,"operation_id":"0x3f7f","timeout_ms":10000,"last_server_duration_us":99190,"total_duration_us":10004447},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":18629,"operation_id":"0x3f71","timeout_ms":10000,"total_duration_us":10003981},{"operation_name":"upsert","last_dispatch_duration_us":47829,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":4750985,"total_server_duration_us":1408724,"encode_duration_us":16469,"operation_id":"0x4021","timeout_ms":10000,"last_server_duration_us":35916,"total_duration_us":10003069}],"total_count":20}} WARN [2022-01-10 05:41:41,670] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17078367,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":17078367,"total_server_duration_us":433455,"operation_id":"0x3d55","timeout_ms":30000,"last_server_duration_us":433455,"total_duration_us":17080278},{"operation_name":"get","last_dispatch_duration_us":16552638,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":16552638,"total_server_duration_us":453479,"operation_id":"0x3d68","timeout_ms":30000,"last_server_duration_us":453479,"total_duration_us":16554999},{"operation_name":"get","last_dispatch_duration_us":14988000,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":14988000,"total_server_duration_us":21243,"operation_id":"0x3dca","timeout_ms":30000,"last_server_duration_us":21243,"total_duration_us":14990260},{"operation_name":"get","last_dispatch_duration_us":11458049,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11458049,"total_server_duration_us":36525,"operation_id":"0x3e56","timeout_ms":30000,"last_server_duration_us":36525,"total_duration_us":11460559},{"operation_name":"upsert","last_dispatch_duration_us":1643188,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6992882,"total_server_duration_us":669869,"encode_duration_us":24858,"operation_id":"0x3ed8","timeout_ms":10000,"last_server_duration_us":89137,"total_duration_us":10009070},{"operation_name":"upsert","last_dispatch_duration_us":874327,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":4824134,"total_server_duration_us":1991494,"encode_duration_us":17029,"operation_id":"0x3f0b","timeout_ms":10000,"last_server_duration_us":31393,"total_duration_us":10008381},{"operation_name":"upsert","last_dispatch_duration_us":2433904,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6705967,"total_server_duration_us":689300,"encode_duration_us":18901,"operation_id":"0x3f72","timeout_ms":10000,"last_server_duration_us":177689,"total_duration_us":10006737},{"operation_name":"upsert","last_dispatch_duration_us":5384311,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10457595,"total_server_duration_us":327044,"encode_duration_us":19118,"operation_id":"0x3f73","timeout_ms":10000,"last_server_duration_us":203801,"total_duration_us":10006353},{"operation_name":"upsert","last_dispatch_duration_us":82345,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8528666,"total_server_duration_us":221369,"encode_duration_us":16914,"operation_id":"0x3f59","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10005818},{"operation_name":"upsert","last_dispatch_duration_us":5569073,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8082853,"total_server_duration_us":295729,"encode_duration_us":16577,"operation_id":"0x3ffb","timeout_ms":10000,"last_server_duration_us":204648,"total_duration_us":10000968}],"total_count":21}} WARN [2022-01-10 05:41:41,671] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9993911,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10083245,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9939938,"encode_duration_us":18189,"operation_id":"0x3f3e","timeout_ms":10000,"last_server_duration_us":9919893,"total_duration_us":10010396}],"total_count":1}} INFO [2022-01-10 05:41:41,717] jepsen worker 35 - jepsen.util 1835 :ok :read [11 21] INFO [2022-01-10 05:41:41,717] jepsen worker 35 - jepsen.util 1835 :invoke :write [11 31] WARN [2022-01-10 05:41:41,770] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":19089367,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":27660352,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":10937077,"encode_duration_us":17444,"operation_id":"0x3c8a","timeout_ms":10000,"last_server_duration_us":10709052,"total_duration_us":10006363},{"operation_name":"upsert","last_dispatch_duration_us":14839777,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14839777,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9086245,"encode_duration_us":17753,"operation_id":"0x3dff","timeout_ms":10000,"last_server_duration_us":9086245,"total_duration_us":10008651}],"total_count":2}} WARN [2022-01-10 05:41:41,771] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":5798126,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":13662649,"total_server_duration_us":1636722,"encode_duration_us":28637,"operation_id":"0x3ef9","timeout_ms":10000,"last_server_duration_us":141143,"total_duration_us":10010942},{"operation_name":"upsert","last_dispatch_duration_us":3397983,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5856405,"total_server_duration_us":3644679,"encode_duration_us":16336,"operation_id":"0x402f","timeout_ms":10000,"last_server_duration_us":2960795,"total_duration_us":10010732},{"operation_name":"upsert","last_dispatch_duration_us":574970,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5759420,"total_server_duration_us":2276614,"encode_duration_us":16258,"operation_id":"0x3fab","timeout_ms":10000,"last_server_duration_us":308017,"total_duration_us":10009609},{"operation_name":"upsert","last_dispatch_duration_us":1589133,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":5289842,"total_server_duration_us":3465964,"encode_duration_us":18128,"operation_id":"0x3fbd","timeout_ms":10000,"last_server_duration_us":1515277,"total_duration_us":10009323},{"operation_name":"upsert","last_dispatch_duration_us":4791945,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11945335,"total_server_duration_us":780768,"encode_duration_us":19822,"operation_id":"0x3fe1","timeout_ms":10000,"last_server_duration_us":198123,"total_duration_us":10007794},{"operation_name":"upsert","last_dispatch_duration_us":34919,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9040891,"total_server_duration_us":291927,"encode_duration_us":16990,"operation_id":"0x3f40","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10004781},{"operation_name":"upsert","last_dispatch_duration_us":9050797,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9583132,"total_server_duration_us":9205989,"encode_duration_us":17216,"operation_id":"0x3f0a","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":9824060},{"operation_name":"get","last_dispatch_duration_us":8527424,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8527424,"total_server_duration_us":43,"operation_id":"0x3f6e","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":8528385},{"operation_name":"get","last_dispatch_duration_us":5814535,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":5814535,"total_server_duration_us":425610,"operation_id":"0x3f7e","timeout_ms":30000,"last_server_duration_us":425610,"total_duration_us":5816719},{"operation_name":"get","last_dispatch_duration_us":1087839,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":1087839,"total_server_duration_us":6827,"operation_id":"0x4059","timeout_ms":30000,"last_server_duration_us":6827,"total_duration_us":1088129}],"total_count":10}} INFO [2022-01-10 05:41:42,020] jepsen worker 84 - jepsen.util 2154 :info :write [28 37] :SyncWriteAmbiguous INFO [2022-01-10 05:41:42,021] jepsen worker 84 - jepsen.util 2244 :invoke :read [28 nil] INFO [2022-01-10 05:41:42,025] jepsen worker 29 - jepsen.util 2009 :info :write [9 36] :RequestTimeoutException INFO [2022-01-10 05:41:42,026] jepsen worker 29 - jepsen.util 2099 :invoke :write [9 44] INFO [2022-01-10 05:41:42,029] jepsen worker 27 - jepsen.util 1917 :info :write [9 14] :RequestTimeoutException INFO [2022-01-10 05:41:42,030] jepsen worker 27 - jepsen.util 2007 :invoke :write [9 24] INFO [2022-01-10 05:41:42,055] jepsen worker 59 - jepsen.util 2219 :ok :read [19 31] INFO [2022-01-10 05:41:42,056] jepsen worker 59 - jepsen.util 2219 :invoke :write [19 29] INFO [2022-01-10 05:41:42,175] jepsen worker 50 - jepsen.util 2210 :info :write [16 1] :RequestTimeoutException INFO [2022-01-10 05:41:42,176] jepsen worker 50 - jepsen.util 2300 :invoke :read [16 nil] INFO [2022-01-10 05:41:42,211] jepsen worker 41 - jepsen.util 1751 :info :write [13 20] :RequestTimeoutException INFO [2022-01-10 05:41:42,212] jepsen worker 41 - jepsen.util 1841 :invoke :write [13 31] INFO [2022-01-10 05:41:42,246] jepsen worker 82 - jepsen.util 1882 :info :write [27 19] :RequestTimeoutException INFO [2022-01-10 05:41:42,247] jepsen worker 82 - jepsen.util 1972 :invoke :write [27 42] INFO [2022-01-10 05:41:42,255] jepsen worker 46 - jepsen.util 2386 :info :write [15 7] :RequestTimeoutException INFO [2022-01-10 05:41:42,256] jepsen worker 46 - jepsen.util 2476 :invoke :write [15 7] INFO [2022-01-10 05:41:42,276] jepsen worker 85 - jepsen.util 2335 :info :write [28 39] :RequestTimeoutException INFO [2022-01-10 05:41:42,276] jepsen worker 85 - jepsen.util 2425 :invoke :write [28 27] INFO [2022-01-10 05:41:42,326] jepsen worker 49 - jepsen.util 1849 :info :write [16 44] :RequestTimeoutException INFO [2022-01-10 05:41:42,326] jepsen worker 49 - jepsen.util 1939 :invoke :read [16 nil] INFO [2022-01-10 05:41:42,402] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:42,402] jepsen worker nemesis - couchbase.util Rebalance check count: 256 INFO [2022-01-10 05:41:42,402] jepsen worker nemesis - couchbase.util Rebalance stuck count: 194 INFO [2022-01-10 05:41:42,453] jepsen worker 57 - jepsen.util 2307 :ok :read [19 31] INFO [2022-01-10 05:41:42,454] jepsen worker 57 - jepsen.util 2307 :invoke :write [19 21] INFO [2022-01-10 05:41:42,486] jepsen worker 66 - jepsen.util 1776 :info :write [22 24] :RequestTimeoutException INFO [2022-01-10 05:41:42,486] jepsen worker 66 - jepsen.util 1866 :invoke :write [22 20] INFO [2022-01-10 05:41:42,581] jepsen worker 24 - jepsen.util 1914 :info :write [8 49] :RequestTimeoutException INFO [2022-01-10 05:41:42,581] jepsen worker 24 - jepsen.util 2004 :invoke :read [8 nil] INFO [2022-01-10 05:41:42,645] jepsen worker 52 - jepsen.util 2392 :info :write [17 20] :RequestTimeoutException INFO [2022-01-10 05:41:42,645] jepsen worker 52 - jepsen.util 2482 :invoke :read [17 nil] INFO [2022-01-10 05:41:42,671] jepsen worker 52 - jepsen.util 2482 :ok :read [17 41] INFO [2022-01-10 05:41:42,672] jepsen worker 52 - jepsen.util 2482 :invoke :read [17 nil] INFO [2022-01-10 05:41:42,701] jepsen worker 51 - jepsen.util 2121 :info :write [17 12] :RequestTimeoutException INFO [2022-01-10 05:41:42,702] jepsen worker 51 - jepsen.util 2211 :invoke :read [17 nil] INFO [2022-01-10 05:41:42,706] jepsen worker 79 - jepsen.util 2239 :info :write [26 1] :RequestTimeoutException INFO [2022-01-10 05:41:42,707] jepsen worker 79 - jepsen.util 2329 :invoke :read [26 nil] INFO [2022-01-10 05:41:42,717] jepsen worker 52 - jepsen.util 2482 :ok :read [17 41] INFO [2022-01-10 05:41:42,718] jepsen worker 52 - jepsen.util 2482 :invoke :read [17 nil] INFO [2022-01-10 05:41:42,726] jepsen worker 51 - jepsen.util 2211 :ok :read [17 41] INFO [2022-01-10 05:41:42,727] jepsen worker 51 - jepsen.util 2211 :invoke :write [17 4] INFO [2022-01-10 05:41:42,735] jepsen worker 36 - jepsen.util 1746 :ok :read [12 42] INFO [2022-01-10 05:41:42,736] jepsen worker 36 - jepsen.util 1746 :invoke :write [12 10] INFO [2022-01-10 05:41:42,741] jepsen worker 52 - jepsen.util 2482 :ok :read [17 41] INFO [2022-01-10 05:41:42,741] jepsen worker 52 - jepsen.util 2482 :invoke :read [17 nil] INFO [2022-01-10 05:41:42,766] jepsen worker 52 - jepsen.util 2482 :ok :read [17 41] INFO [2022-01-10 05:41:42,766] jepsen worker 52 - jepsen.util 2482 :invoke :write [17 5] INFO [2022-01-10 05:41:42,940] jepsen worker 45 - jepsen.util 2205 :info :write [15 39] :RequestTimeoutException INFO [2022-01-10 05:41:42,941] jepsen worker 45 - jepsen.util 2295 :invoke :write [15 31] INFO [2022-01-10 05:41:42,942] jepsen worker 61 - jepsen.util 2041 :info :write [20 33] :RequestTimeoutException INFO [2022-01-10 05:41:42,943] jepsen worker 61 - jepsen.util 2131 :invoke :write [20 31] INFO [2022-01-10 05:41:42,956] jepsen worker 33 - jepsen.util 2103 :info :write [11 49] :RequestTimeoutException INFO [2022-01-10 05:41:42,957] jepsen worker 33 - jepsen.util 2193 :invoke :write [11 21] INFO [2022-01-10 05:41:43,230] jepsen worker 77 - jepsen.util 2327 :info :write [25 41] :RequestTimeoutException INFO [2022-01-10 05:41:43,230] jepsen worker 77 - jepsen.util 2417 :invoke :read [25 nil] INFO [2022-01-10 05:41:43,304] jepsen worker 56 - jepsen.util 2216 :info :write [18 30] :SyncWriteAmbiguous INFO [2022-01-10 05:41:43,306] jepsen worker 56 - jepsen.util 2306 :invoke :read [18 nil] INFO [2022-01-10 05:41:43,341] jepsen worker 34 - jepsen.util 2014 :info :write [11 40] :RequestTimeoutException INFO [2022-01-10 05:41:43,341] jepsen worker 34 - jepsen.util 2104 :invoke :read [11 nil] INFO [2022-01-10 05:41:43,409] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:43,410] jepsen worker nemesis - couchbase.util Rebalance check count: 257 INFO [2022-01-10 05:41:43,410] jepsen worker nemesis - couchbase.util Rebalance stuck count: 195 INFO [2022-01-10 05:41:43,415] jepsen worker 87 - jepsen.util 2247 :ok :read [29 0] INFO [2022-01-10 05:41:43,415] jepsen worker 87 - jepsen.util 2247 :invoke :read [29 nil] INFO [2022-01-10 05:41:43,457] jepsen worker 87 - jepsen.util 2247 :ok :read [29 0] INFO [2022-01-10 05:41:43,458] jepsen worker 87 - jepsen.util 2247 :invoke :write [29 31] INFO [2022-01-10 05:41:43,475] jepsen worker 69 - jepsen.util 2049 :info :write [23 42] :SyncWriteAmbiguous INFO [2022-01-10 05:41:43,476] jepsen worker 69 - jepsen.util 2139 :invoke :write [23 6] INFO [2022-01-10 05:41:43,586] jepsen worker 32 - jepsen.util 2102 :info :write [10 15] :RequestTimeoutException INFO [2022-01-10 05:41:43,587] jepsen worker 32 - jepsen.util 2192 :invoke :read [10 nil] INFO [2022-01-10 05:41:43,610] jepsen worker 32 - jepsen.util 2192 :ok :read [10 37] INFO [2022-01-10 05:41:43,611] jepsen worker 32 - jepsen.util 2192 :invoke :read [10 nil] INFO [2022-01-10 05:41:43,647] jepsen worker 32 - jepsen.util 2192 :ok :read [10 37] INFO [2022-01-10 05:41:43,647] jepsen worker 32 - jepsen.util 2192 :invoke :write [10 7] INFO [2022-01-10 05:41:43,654] jepsen worker 73 - jepsen.util 2053 :ok :read [24 18] INFO [2022-01-10 05:41:43,654] jepsen worker 73 - jepsen.util 2053 :invoke :read [24 nil] INFO [2022-01-10 05:41:43,662] jepsen worker 44 - jepsen.util 2024 :ok :read [14 14] INFO [2022-01-10 05:41:43,663] jepsen worker 44 - jepsen.util 2024 :invoke :write [14 22] INFO [2022-01-10 05:41:43,700] jepsen worker 31 - jepsen.util 2281 :info :write [10 33] :RequestTimeoutException INFO [2022-01-10 05:41:43,701] jepsen worker 31 - jepsen.util 2371 :invoke :read [10 nil] INFO [2022-01-10 05:41:43,885] jepsen worker 20 - jepsen.util 1730 :info :write [6 9] :RequestTimeoutException INFO [2022-01-10 05:41:43,886] jepsen worker 20 - jepsen.util 1820 :invoke :read [6 nil] INFO [2022-01-10 05:41:44,416] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:44,416] jepsen worker nemesis - couchbase.util Rebalance check count: 258 INFO [2022-01-10 05:41:44,506] jepsen worker 11 - jepsen.util 1901 :info :write [3 39] :RequestTimeoutException INFO [2022-01-10 05:41:44,557] jepsen worker nemesis - couchbase.util Rebalance stuck count: 196 INFO [2022-01-10 05:41:44,535] jepsen worker 30 - jepsen.util 1740 :info :write [10 26] :RequestTimeoutException INFO [2022-01-10 05:41:44,557] jepsen worker 11 - jepsen.util 1991 :invoke :read [3 nil] INFO [2022-01-10 05:41:44,557] jepsen worker 30 - jepsen.util 1830 :invoke :read [10 nil] INFO [2022-01-10 05:41:44,599] jepsen worker 30 - jepsen.util 1830 :ok :read [10 37] INFO [2022-01-10 05:41:44,600] jepsen worker 30 - jepsen.util 1830 :invoke :write [10 12] INFO [2022-01-10 05:41:44,636] jepsen worker 40 - jepsen.util 1570 :info :write [13 40] :RequestTimeoutException INFO [2022-01-10 05:41:44,636] jepsen worker 40 - jepsen.util 1660 :invoke :read [13 nil] INFO [2022-01-10 05:41:44,642] jepsen worker 62 - jepsen.util 2222 :info :write [20 27] :RequestTimeoutException INFO [2022-01-10 05:41:44,642] jepsen worker 62 - jepsen.util 2312 :invoke :write [20 12] INFO [2022-01-10 05:41:44,730] jepsen worker 12 - jepsen.util 1902 :info :write [4 48] :RequestTimeoutException INFO [2022-01-10 05:41:44,731] jepsen worker 12 - jepsen.util 1992 :invoke :write [4 10] INFO [2022-01-10 05:41:44,911] jepsen worker 65 - jepsen.util 1955 :info :write [21 39] :RequestTimeoutException INFO [2022-01-10 05:41:44,912] jepsen worker 65 - jepsen.util 2045 :invoke :read [21 nil] INFO [2022-01-10 05:41:44,915] jepsen worker 8 - jepsen.util 2258 :info :write [2 34] :RequestTimeoutException INFO [2022-01-10 05:41:44,916] jepsen worker 8 - jepsen.util 2348 :invoke :write [2 14] INFO [2022-01-10 05:41:45,269] jepsen worker 1 - jepsen.util 2161 :info :write [0 48] :RequestTimeoutException INFO [2022-01-10 05:41:45,270] jepsen worker 1 - jepsen.util 2251 :invoke :write [0 26] INFO [2022-01-10 05:41:45,331] jepsen worker 54 - jepsen.util 2034 :info :write [18 1] :RequestTimeoutException INFO [2022-01-10 05:41:45,332] jepsen worker 54 - jepsen.util 2124 :invoke :read [18 nil] INFO [2022-01-10 05:41:45,521] jepsen worker 15 - jepsen.util 1725 :info :write [5 19] :RequestTimeoutException INFO [2022-01-10 05:41:45,522] jepsen worker 15 - jepsen.util 1815 :invoke :write [5 1] INFO [2022-01-10 05:41:45,563] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:45,563] jepsen worker nemesis - couchbase.util Rebalance check count: 259 INFO [2022-01-10 05:41:45,563] jepsen worker nemesis - couchbase.util Rebalance stuck count: 197 INFO [2022-01-10 05:41:45,601] jepsen worker 5 - jepsen.util 2165 :info :write [1 46] :RequestTimeoutException INFO [2022-01-10 05:41:45,602] jepsen worker 5 - jepsen.util 2255 :invoke :write [1 4] INFO [2022-01-10 05:41:45,620] jepsen worker 75 - jepsen.util 1335 :fail :write [25 34] :ServerOutOfMemoryException INFO [2022-01-10 05:41:45,620] jepsen worker 75 - jepsen.util 1335 :invoke :write [25 17] INFO [2022-01-10 05:41:45,754] jepsen worker 3 - jepsen.util 2163 :info :write [1 38] :RequestTimeoutException INFO [2022-01-10 05:41:45,755] jepsen worker 3 - jepsen.util 2253 :invoke :write [1 20] INFO [2022-01-10 05:41:45,819] jepsen worker 63 - jepsen.util 1953 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:45,819] jepsen worker 43 - jepsen.util 2023 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:45,820] jepsen worker 23 - jepsen.util 2183 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:45,820] jepsen worker 63 - jepsen.util 1953 :invoke :read [21 nil] INFO [2022-01-10 05:41:45,821] jepsen worker 23 - jepsen.util 2183 :invoke :read [7 nil] INFO [2022-01-10 05:41:45,822] jepsen worker 43 - jepsen.util 2023 :invoke :read [14 nil] INFO [2022-01-10 05:41:45,881] jepsen worker 9 - jepsen.util 2349 :info :write [3 19] :RequestTimeoutException INFO [2022-01-10 05:41:45,882] jepsen worker 9 - jepsen.util 2439 :invoke :read [3 nil] INFO [2022-01-10 05:41:45,950] jepsen worker 80 - jepsen.util 2240 :info :write [26 24] :RequestTimeoutException INFO [2022-01-10 05:41:45,952] jepsen worker 80 - jepsen.util 2330 :invoke :write [26 11] INFO [2022-01-10 05:41:46,045] jepsen worker 22 - jepsen.util 1732 :info :write [7 22] :RequestTimeoutException INFO [2022-01-10 05:41:46,046] jepsen worker 22 - jepsen.util 1822 :invoke :write [7 29] INFO [2022-01-10 05:41:46,232] jepsen worker 14 - jepsen.util 2264 :info :write [4 40] :RequestTimeoutException INFO [2022-01-10 05:41:46,232] jepsen worker 14 - jepsen.util 2354 :invoke :read [4 nil] INFO [2022-01-10 05:41:46,256] jepsen worker 14 - jepsen.util 2354 :ok :read [4 3] INFO [2022-01-10 05:41:46,257] jepsen worker 14 - jepsen.util 2354 :invoke :write [4 10] INFO [2022-01-10 05:41:46,441] jepsen worker 4 - jepsen.util 2164 :info :write [1 18] :RequestTimeoutException INFO [2022-01-10 05:41:46,442] jepsen worker 4 - jepsen.util 2254 :invoke :write [1 20] INFO [2022-01-10 05:41:46,569] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:46,570] jepsen worker nemesis - couchbase.util Rebalance check count: 260 INFO [2022-01-10 05:41:46,570] jepsen worker nemesis - couchbase.util Rebalance stuck count: 198 INFO [2022-01-10 05:41:46,744] jepsen worker 18 - jepsen.util 1548 :info :write [6 43] :RequestTimeoutException INFO [2022-01-10 05:41:46,745] jepsen worker 18 - jepsen.util 1638 :invoke :write [6 26] INFO [2022-01-10 05:41:46,932] jepsen worker 26 - jepsen.util 1736 :info :write [8 25] :RequestTimeoutException INFO [2022-01-10 05:41:46,932] jepsen worker 26 - jepsen.util 1826 :invoke :read [8 nil] INFO [2022-01-10 05:41:47,029] jepsen worker 77 - jepsen.util 2417 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:47,029] jepsen worker 65 - jepsen.util 2045 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:47,030] jepsen worker 47 - jepsen.util 2117 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:47,030] jepsen worker 20 - jepsen.util 1820 :fail :read [6 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:47,030] jepsen worker 25 - jepsen.util 2005 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:47,030] jepsen worker 21 - jepsen.util 2181 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:47,031] jepsen worker 65 - jepsen.util 2045 :invoke :write [21 4] INFO [2022-01-10 05:41:47,032] jepsen worker 21 - jepsen.util 2181 :invoke :read [7 nil] INFO [2022-01-10 05:41:47,032] jepsen worker 47 - jepsen.util 2117 :invoke :write [15 23] INFO [2022-01-10 05:41:47,033] jepsen worker 25 - jepsen.util 2005 :invoke :read [8 nil] INFO [2022-01-10 05:41:47,034] jepsen worker 20 - jepsen.util 1820 :invoke :read [6 nil] INFO [2022-01-10 05:41:47,034] jepsen worker 77 - jepsen.util 2417 :invoke :write [25 47] INFO [2022-01-10 05:41:47,176] jepsen worker 74 - jepsen.util 2504 :info :write [24 24] :RequestTimeoutException INFO [2022-01-10 05:41:47,176] jepsen worker 74 - jepsen.util 2594 :invoke :read [24 nil] INFO [2022-01-10 05:41:47,575] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:47,575] jepsen worker nemesis - couchbase.util Rebalance check count: 261 INFO [2022-01-10 05:41:47,575] jepsen worker nemesis - couchbase.util Rebalance stuck count: 199 INFO [2022-01-10 05:41:47,715] jepsen worker 53 - jepsen.util 2303 :info :write [17 22] :RequestTimeoutException INFO [2022-01-10 05:41:47,717] jepsen worker 53 - jepsen.util 2393 :invoke :write [17 33] INFO [2022-01-10 05:41:47,759] jepsen worker 13 - jepsen.util 1993 :info :write [4 17] :RequestTimeoutException INFO [2022-01-10 05:41:47,760] jepsen worker 13 - jepsen.util 2083 :invoke :write [4 33] INFO [2022-01-10 05:41:47,832] jepsen worker 0 - jepsen.util 1980 :info :write [0 30] :RequestTimeoutException INFO [2022-01-10 05:41:47,832] jepsen worker 0 - jepsen.util 2070 :invoke :read [0 nil] INFO [2022-01-10 05:41:47,902] jepsen worker 81 - jepsen.util 2151 :info :write [27 10] :RequestTimeoutException INFO [2022-01-10 05:41:47,903] jepsen worker 81 - jepsen.util 2241 :invoke :write [27 45] INFO [2022-01-10 05:41:48,211] jepsen worker 20 - jepsen.util 1820 :fail :read [6 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:48,211] jepsen worker 20 - jepsen.util 1820 :invoke :write [6 6] INFO [2022-01-10 05:41:48,245] jepsen worker 63 - jepsen.util 1953 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:48,245] jepsen worker 74 - jepsen.util 2594 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:48,245] jepsen worker 43 - jepsen.util 2023 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:48,245] jepsen worker 23 - jepsen.util 2183 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:48,245] jepsen worker 43 - jepsen.util 2023 :invoke :read [14 nil] INFO [2022-01-10 05:41:48,246] jepsen worker 74 - jepsen.util 2594 :invoke :write [24 37] INFO [2022-01-10 05:41:48,246] jepsen worker 63 - jepsen.util 1953 :invoke :write [21 18] INFO [2022-01-10 05:41:48,254] jepsen worker 23 - jepsen.util 2183 :invoke :write [7 6] INFO [2022-01-10 05:41:48,264] jepsen worker 21 - jepsen.util 2181 :ok :read [7 :nil] INFO [2022-01-10 05:41:48,265] jepsen worker 34 - jepsen.util 2104 :ok :read [11 21] INFO [2022-01-10 05:41:48,271] jepsen worker 34 - jepsen.util 2104 :invoke :read [11 nil] INFO [2022-01-10 05:41:48,271] jepsen worker 21 - jepsen.util 2181 :invoke :read [7 nil] INFO [2022-01-10 05:41:48,280] jepsen worker 25 - jepsen.util 2005 :ok :read [8 27] INFO [2022-01-10 05:41:48,280] jepsen worker 25 - jepsen.util 2005 :invoke :read [8 nil] INFO [2022-01-10 05:41:48,355] jepsen worker 49 - jepsen.util 1939 :ok :read [16 38] INFO [2022-01-10 05:41:48,356] jepsen worker 49 - jepsen.util 1939 :invoke :read [16 nil] INFO [2022-01-10 05:41:48,365] jepsen worker 40 - jepsen.util 1660 :ok :read [13 15] INFO [2022-01-10 05:41:48,366] jepsen worker 40 - jepsen.util 1660 :invoke :read [13 nil] INFO [2022-01-10 05:41:48,582] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:48,582] jepsen worker nemesis - couchbase.util Rebalance check count: 262 INFO [2022-01-10 05:41:48,582] jepsen worker nemesis - couchbase.util Rebalance stuck count: 200 INFO [2022-01-10 05:41:48,587] jepsen worker 79 - jepsen.util 2329 :ok :read [26 10] INFO [2022-01-10 05:41:48,587] jepsen worker 79 - jepsen.util 2329 :invoke :write [26 49] INFO [2022-01-10 05:41:48,615] jepsen worker 7 - jepsen.util 2167 :info :write [2 48] :RequestTimeoutException INFO [2022-01-10 05:41:48,615] jepsen worker 7 - jepsen.util 2257 :invoke :write [2 7] INFO [2022-01-10 05:41:49,588] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:49,588] jepsen worker nemesis - couchbase.util Rebalance check count: 263 INFO [2022-01-10 05:41:49,804] jepsen worker 39 - jepsen.util 2289 :info :write [13 10] :RequestTimeoutException INFO [2022-01-10 05:41:49,883] jepsen worker nemesis - couchbase.util Rebalance stuck count: 201 INFO [2022-01-10 05:41:49,883] jepsen worker 39 - jepsen.util 2379 :invoke :write [13 10] INFO [2022-01-10 05:41:49,956] jepsen worker 78 - jepsen.util 2238 :info :write [26 3] :RequestTimeoutException INFO [2022-01-10 05:41:49,957] jepsen worker 78 - jepsen.util 2328 :invoke :read [26 nil] INFO [2022-01-10 05:41:49,975] jepsen worker 48 - jepsen.util 1668 :info :write [16 9] :RequestTimeoutException INFO [2022-01-10 05:41:49,975] jepsen worker 48 - jepsen.util 1758 :invoke :write [16 0] WARN [2022-01-10 05:41:49,983] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9079048,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10549090,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9503154,"encode_duration_us":22450,"operation_id":"0x3f45","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10010097}],"total_count":1}} INFO [2022-01-10 05:41:49,995] jepsen worker 28 - jepsen.util 1918 :info :write [9 35] :RequestTimeoutException INFO [2022-01-10 05:41:49,996] jepsen worker 28 - jepsen.util 2008 :invoke :write [9 8] WARN [2022-01-10 05:41:50,084] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":24509624,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":24509624,"total_server_duration_us":9002,"operation_id":"0x3d67","timeout_ms":30000,"last_server_duration_us":9002,"total_duration_us":24512081},{"operation_name":"get","last_dispatch_duration_us":17455197,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":17455197,"total_server_duration_us":13025,"operation_id":"0x3eac","timeout_ms":30000,"last_server_duration_us":13025,"total_duration_us":17457881},{"operation_name":"get","last_dispatch_duration_us":14239349,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14239349,"total_server_duration_us":140600,"operation_id":"0x3f75","timeout_ms":30000,"last_server_duration_us":140600,"total_duration_us":14241366},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":16586,"operation_id":"0x4148","timeout_ms":10000,"total_duration_us":10009878},{"operation_name":"upsert","last_dispatch_duration_us":2080769,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3277084,"total_server_duration_us":735494,"encode_duration_us":18301,"operation_id":"0x415e","timeout_ms":10000,"last_server_duration_us":487296,"total_duration_us":10009061},{"operation_name":"upsert","last_dispatch_duration_us":31244,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":3838770,"total_server_duration_us":2928430,"encode_duration_us":16256,"operation_id":"0x409b","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10008082},{"operation_name":"upsert","last_dispatch_duration_us":94922,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8270214,"total_server_duration_us":189709,"encode_duration_us":20607,"operation_id":"0x4100","timeout_ms":10000,"last_server_duration_us":21163,"total_duration_us":10006070},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":17118,"operation_id":"0x402d","timeout_ms":10000,"total_duration_us":10005760},{"operation_name":"upsert","last_dispatch_duration_us":7717107,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8443813,"total_server_duration_us":343190,"encode_duration_us":16332,"operation_id":"0x4045","timeout_ms":10000,"last_server_duration_us":20441,"total_duration_us":10004329},{"operation_name":"upsert","last_dispatch_duration_us":953461,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2122253,"total_server_duration_us":501227,"encode_duration_us":17630,"operation_id":"0x4129","timeout_ms":10000,"last_server_duration_us":289103,"total_duration_us":10004281}],"total_count":15}} INFO [2022-01-10 05:41:50,181] jepsen worker 83 - jepsen.util 2423 :info :write [27 48] :RequestTimeoutException INFO [2022-01-10 05:41:50,182] jepsen worker 83 - jepsen.util 2513 :invoke :read [27 nil] INFO [2022-01-10 05:41:50,215] jepsen worker 68 - jepsen.util 2138 :info :write [22 35] :RequestTimeoutException INFO [2022-01-10 05:41:50,216] jepsen worker 68 - jepsen.util 2228 :invoke :write [22 49] INFO [2022-01-10 05:41:50,415] jepsen worker 19 - jepsen.util 2179 :info :write [6 19] :RequestTimeoutException INFO [2022-01-10 05:41:50,416] jepsen worker 19 - jepsen.util 2269 :invoke :write [6 38] INFO [2022-01-10 05:41:50,607] jepsen worker 17 - jepsen.util 2087 :info :write [5 23] :RequestTimeoutException INFO [2022-01-10 05:41:50,607] jepsen worker 17 - jepsen.util 2177 :invoke :read [5 nil] INFO [2022-01-10 05:41:50,801] jepsen worker 49 - jepsen.util 1939 :ok :read [16 38] INFO [2022-01-10 05:41:50,801] jepsen worker 49 - jepsen.util 1939 :invoke :read [16 nil] INFO [2022-01-10 05:41:50,812] jepsen worker 89 - jepsen.util 2339 :info :write [29 38] :RequestTimeoutException INFO [2022-01-10 05:41:50,813] jepsen worker 89 - jepsen.util 2429 :invoke :read [29 nil] INFO [2022-01-10 05:41:50,821] jepsen worker 40 - jepsen.util 1660 :ok :read [13 15] INFO [2022-01-10 05:41:50,822] jepsen worker 40 - jepsen.util 1660 :invoke :read [13 nil] INFO [2022-01-10 05:41:50,827] jepsen worker 16 - jepsen.util 2446 :ok :read [5 42] INFO [2022-01-10 05:41:50,827] jepsen worker 16 - jepsen.util 2446 :invoke :read [5 nil] INFO [2022-01-10 05:41:50,888] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:50,889] jepsen worker nemesis - couchbase.util Rebalance check count: 264 INFO [2022-01-10 05:41:50,889] jepsen worker nemesis - couchbase.util Rebalance stuck count: 202 INFO [2022-01-10 05:41:51,167] jepsen worker 78 - jepsen.util 2328 :ok :read [26 10] INFO [2022-01-10 05:41:51,167] jepsen worker 78 - jepsen.util 2328 :invoke :write [26 3] INFO [2022-01-10 05:41:51,423] jepsen worker 67 - jepsen.util 1867 :ok :read [22 41] INFO [2022-01-10 05:41:51,424] jepsen worker 67 - jepsen.util 1867 :invoke :write [22 30] WARN [2022-01-10 05:41:51,486] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":1768579,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1768579,"total_server_duration_us":23,"operation_id":"0x415d","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":11012714},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":17185,"operation_id":"0x4135","timeout_ms":10000,"total_duration_us":10010908},{"operation_name":"upsert","last_dispatch_duration_us":30737,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":3767993,"total_server_duration_us":3413658,"encode_duration_us":16963,"operation_id":"0x409c","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10010692},{"operation_name":"upsert","last_dispatch_duration_us":8484486,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8484486,"total_server_duration_us":170957,"encode_duration_us":17236,"operation_id":"0x4128","timeout_ms":10000,"last_server_duration_us":170957,"total_duration_us":10007721},{"operation_name":"upsert","last_dispatch_duration_us":11857903,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11857903,"total_server_duration_us":199588,"encode_duration_us":16806,"operation_id":"0x40c7","timeout_ms":10000,"last_server_duration_us":199588,"total_duration_us":10006697},{"operation_name":"upsert","last_dispatch_duration_us":9283918,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9909809,"total_server_duration_us":765826,"encode_duration_us":25932,"operation_id":"0x408d","timeout_ms":10000,"last_server_duration_us":391767,"total_duration_us":10003421},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":18143,"operation_id":"0x40ce","timeout_ms":10000,"total_duration_us":10002844},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":16335,"operation_id":"0x4169","timeout_ms":10000,"total_duration_us":10002356},{"operation_name":"upsert","last_dispatch_duration_us":509767,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":6604502,"total_server_duration_us":2113425,"encode_duration_us":16711,"operation_id":"0x4215","timeout_ms":10000,"last_server_duration_us":373514,"total_duration_us":10002273},{"operation_name":"upsert","last_dispatch_duration_us":9421648,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9711823,"total_server_duration_us":9523232,"encode_duration_us":17137,"operation_id":"0x405d","timeout_ms":10000,"last_server_duration_us":9387258,"total_duration_us":10001080}],"total_count":20}} WARN [2022-01-10 05:41:51,486] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9421648,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9711823,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9523232,"encode_duration_us":17137,"operation_id":"0x405d","timeout_ms":10000,"last_server_duration_us":9387258,"total_duration_us":10001080},{"operation_name":"upsert","last_dispatch_duration_us":16284397,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16284397,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9035244,"encode_duration_us":16968,"operation_id":"0x3f84","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10001380},{"operation_name":"upsert","last_dispatch_duration_us":11857903,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11857903,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":199588,"encode_duration_us":16806,"operation_id":"0x40c7","timeout_ms":10000,"last_server_duration_us":199588,"total_duration_us":10006697},{"operation_name":"upsert","last_dispatch_duration_us":12761883,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18406320,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":1472071,"encode_duration_us":17021,"operation_id":"0x3f0e","timeout_ms":10000,"last_server_duration_us":395126,"total_duration_us":10006532},{"operation_name":"upsert","last_dispatch_duration_us":8484486,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":8484486,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":170957,"encode_duration_us":17236,"operation_id":"0x4128","timeout_ms":10000,"last_server_duration_us":170957,"total_duration_us":10007721}],"total_count":5}} INFO [2022-01-10 05:41:51,571] jepsen worker 76 - jepsen.util 1516 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:51,572] jepsen worker 76 - jepsen.util 1516 :invoke :write [25 39] WARN [2022-01-10 05:41:51,585] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9707517,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17790370,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":488861,"encode_duration_us":16577,"operation_id":"0x3ffb","timeout_ms":10000,"last_server_duration_us":193132,"total_duration_us":10000968},{"operation_name":"upsert","last_dispatch_duration_us":16100934,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16100934,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":10833706,"encode_duration_us":20745,"operation_id":"0x4105","timeout_ms":10000,"last_server_duration_us":10833706,"total_duration_us":10002013},{"operation_name":"upsert","last_dispatch_duration_us":9068745,"last_local_id":"EFE9906200000002/00000000598DAEAE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10602931,"last_local_socket":"172.23.104.79:44342","total_server_duration_us":9382855,"encode_duration_us":19052,"operation_id":"0x419a","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10010333},{"operation_name":"upsert","last_dispatch_duration_us":13068238,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19774205,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":895222,"encode_duration_us":18901,"operation_id":"0x3f72","timeout_ms":10000,"last_server_duration_us":205922,"total_duration_us":10006737}],"total_count":4}} WARN [2022-01-10 05:41:51,585] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14437028,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":14437028,"total_server_duration_us":350020,"encode_duration_us":16933,"operation_id":"0x40bd","timeout_ms":10000,"last_server_duration_us":350020,"total_duration_us":10010982},{"operation_name":"upsert","last_dispatch_duration_us":11879568,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11879568,"total_server_duration_us":366153,"encode_duration_us":17136,"operation_id":"0x4104","timeout_ms":10000,"last_server_duration_us":366153,"total_duration_us":10010490},{"operation_name":"upsert","last_dispatch_duration_us":33832,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":2819224,"total_server_duration_us":2506229,"encode_duration_us":19034,"operation_id":"0x4167","timeout_ms":10000,"last_server_duration_us":20841,"total_duration_us":10009461},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":21551,"operation_id":"0x418a","timeout_ms":10000,"total_duration_us":10008180},{"operation_name":"upsert","last_dispatch_duration_us":1130731,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":3554957,"total_server_duration_us":838749,"encode_duration_us":17298,"operation_id":"0x41b4","timeout_ms":10000,"last_server_duration_us":440185,"total_duration_us":10007684},{"operation_name":"upsert","last_dispatch_duration_us":1020632,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":5403845,"total_server_duration_us":944164,"encode_duration_us":18797,"operation_id":"0x4133","timeout_ms":10000,"last_server_duration_us":356168,"total_duration_us":10006956},{"operation_name":"upsert","last_dispatch_duration_us":31940,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":833268,"total_server_duration_us":458034,"encode_duration_us":19174,"operation_id":"0x4159","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10005962},{"operation_name":"upsert","last_dispatch_duration_us":31533,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":391289,"total_server_duration_us":242611,"encode_duration_us":19436,"operation_id":"0x4236","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10004610},{"operation_name":"upsert","last_dispatch_duration_us":35836,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":889022,"total_server_duration_us":465976,"encode_duration_us":16341,"operation_id":"0x4132","timeout_ms":10000,"last_server_duration_us":22306,"total_duration_us":10004138},{"operation_name":"upsert","last_dispatch_duration_us":812506,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":2654664,"total_server_duration_us":472959,"encode_duration_us":17604,"operation_id":"0x41b5","timeout_ms":10000,"last_server_duration_us":282507,"total_duration_us":10001506}],"total_count":10}} WARN [2022-01-10 05:41:51,586] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":3487837,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":3487837,"total_server_duration_us":2018971,"encode_duration_us":17134,"operation_id":"0x405c","timeout_ms":10000,"last_server_duration_us":2018971,"total_duration_us":10007100},{"operation_name":"upsert","last_dispatch_duration_us":2323343,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":3736067,"total_server_duration_us":222342,"encode_duration_us":17586,"operation_id":"0x421b","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10005973},{"operation_name":"upsert","last_dispatch_duration_us":3901043,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8556973,"total_server_duration_us":627633,"encode_duration_us":21894,"operation_id":"0x40bf","timeout_ms":10000,"last_server_duration_us":526252,"total_duration_us":10001273},{"operation_name":"upsert","last_dispatch_duration_us":3354293,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4549870,"total_server_duration_us":583828,"encode_duration_us":20174,"operation_id":"0x4192","timeout_ms":10000,"last_server_duration_us":317409,"total_duration_us":10000793},{"operation_name":"get","last_dispatch_duration_us":3373495,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":3373495,"total_server_duration_us":163180,"operation_id":"0x4108","timeout_ms":30000,"last_server_duration_us":163180,"total_duration_us":7008281},{"operation_name":"get","last_dispatch_duration_us":1931030,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1931030,"total_server_duration_us":509579,"operation_id":"0x420e","timeout_ms":30000,"last_server_duration_us":509579,"total_duration_us":3531393},{"operation_name":"get","last_dispatch_duration_us":2400259,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":2400259,"total_server_duration_us":509892,"operation_id":"0x4243","timeout_ms":30000,"last_server_duration_us":509892,"total_duration_us":2402315},{"operation_name":"get","last_dispatch_duration_us":1239615,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":1239615,"total_server_duration_us":84,"operation_id":"0x4235","timeout_ms":30000,"last_server_duration_us":84,"total_duration_us":1241641},{"operation_name":"get","last_dispatch_duration_us":914984,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":914984,"total_server_duration_us":1617,"operation_id":"0x4244","timeout_ms":30000,"last_server_duration_us":1617,"total_duration_us":916856}],"total_count":9}} INFO [2022-01-10 05:41:51,653] jepsen worker 38 - jepsen.util 2198 :ok :read [12 42] INFO [2022-01-10 05:41:51,653] jepsen worker 38 - jepsen.util 2198 :invoke :read [12 nil] INFO [2022-01-10 05:41:51,659] jepsen worker 37 - jepsen.util 2107 :ok :read [12 42] INFO [2022-01-10 05:41:51,660] jepsen worker 37 - jepsen.util 2107 :invoke :read [12 nil] WARN [2022-01-10 05:41:51,685] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9196900,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10563095,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":9266691,"encode_duration_us":17377,"operation_id":"0x41a7","timeout_ms":10000,"last_server_duration_us":9046921,"total_duration_us":10000761},{"operation_name":"upsert","last_dispatch_duration_us":14568860,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14568860,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":396810,"encode_duration_us":16772,"operation_id":"0x40c6","timeout_ms":10000,"last_server_duration_us":396810,"total_duration_us":10000999}],"total_count":2}} WARN [2022-01-10 05:41:51,686] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11936905,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11936905,"total_server_duration_us":307,"operation_id":"0x40ed","timeout_ms":30000,"last_server_duration_us":307,"total_duration_us":11937296},{"operation_name":"get","last_dispatch_duration_us":10803089,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10803089,"total_server_duration_us":23,"operation_id":"0x412a","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":10803296},{"operation_name":"upsert","last_dispatch_duration_us":8390700,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9138074,"total_server_duration_us":225817,"encode_duration_us":17442,"operation_id":"0x40cb","timeout_ms":10000,"last_server_duration_us":18570,"total_duration_us":10010631},{"operation_name":"upsert","last_dispatch_duration_us":9068745,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":10602931,"total_server_duration_us":9382855,"encode_duration_us":19052,"operation_id":"0x419a","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10010333},{"operation_name":"upsert","last_dispatch_duration_us":7485851,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":11661685,"total_server_duration_us":1991921,"encode_duration_us":16654,"operation_id":"0x409d","timeout_ms":10000,"last_server_duration_us":21813,"total_duration_us":10009605},{"operation_name":"upsert","last_dispatch_duration_us":7385872,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7385872,"total_server_duration_us":387308,"encode_duration_us":16078,"operation_id":"0x4218","timeout_ms":10000,"last_server_duration_us":387308,"total_duration_us":10008081},{"operation_name":"upsert","last_dispatch_duration_us":32369,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9131927,"total_server_duration_us":216742,"encode_duration_us":23238,"operation_id":"0x40ff","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10007554},{"operation_name":"upsert","last_dispatch_duration_us":1309141,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9636710,"total_server_duration_us":323435,"encode_duration_us":16871,"operation_id":"0x4208","timeout_ms":10000,"last_server_duration_us":53015,"total_duration_us":10005798},{"operation_name":"upsert","last_dispatch_duration_us":2529969,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":2857526,"total_server_duration_us":143258,"encode_duration_us":20549,"operation_id":"0x408e","timeout_ms":10000,"last_server_duration_us":90481,"total_duration_us":10005607},{"operation_name":"upsert","last_dispatch_duration_us":12728531,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12728531,"total_server_duration_us":196453,"encode_duration_us":17490,"operation_id":"0x408f","timeout_ms":10000,"last_server_duration_us":196453,"total_duration_us":10005188}],"total_count":26}} WARN [2022-01-10 05:41:51,686] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14318441,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17601335,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":533391,"encode_duration_us":17318,"operation_id":"0x3f31","timeout_ms":10000,"last_server_duration_us":381212,"total_duration_us":10001595},{"operation_name":"upsert","last_dispatch_duration_us":13900087,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17869125,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":556138,"encode_duration_us":16383,"operation_id":"0x3f0c","timeout_ms":10000,"last_server_duration_us":372419,"total_duration_us":10007097}],"total_count":2}} INFO [2022-01-10 05:41:51,686] jepsen worker 71 - jepsen.util 1691 :ok :read [23 26] WARN [2022-01-10 05:41:51,686] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9059908,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":13810894,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":10448213,"encode_duration_us":16469,"operation_id":"0x4021","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10003069},{"operation_name":"upsert","last_dispatch_duration_us":18106781,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18106781,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9029938,"encode_duration_us":25052,"operation_id":"0x3f07","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10008650},{"operation_name":"upsert","last_dispatch_duration_us":18128514,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18128514,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9025695,"encode_duration_us":21962,"operation_id":"0x3f08","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10007678}],"total_count":3}} INFO [2022-01-10 05:41:51,687] jepsen worker 71 - jepsen.util 1691 :invoke :write [23 40] INFO [2022-01-10 05:41:51,693] jepsen worker 88 - jepsen.util 2068 :ok :read [29 0] INFO [2022-01-10 05:41:51,693] jepsen worker 88 - jepsen.util 2068 :invoke :write [29 29] INFO [2022-01-10 05:41:51,698] jepsen worker 70 - jepsen.util 2410 :ok :read [23 26] INFO [2022-01-10 05:41:51,699] jepsen worker 70 - jepsen.util 2410 :invoke :read [23 nil] INFO [2022-01-10 05:41:51,736] jepsen worker 35 - jepsen.util 1835 :info :write [11 31] :RequestTimeoutException INFO [2022-01-10 05:41:51,737] jepsen worker 35 - jepsen.util 1925 :invoke :write [11 26] INFO [2022-01-10 05:41:51,737] jepsen worker 10 - jepsen.util 2260 :ok :read [3 2] INFO [2022-01-10 05:41:51,737] jepsen worker 10 - jepsen.util 2260 :invoke :read [3 nil] INFO [2022-01-10 05:41:51,740] jepsen worker 31 - jepsen.util 2371 :ok :read [10 37] INFO [2022-01-10 05:41:51,741] jepsen worker 31 - jepsen.util 2371 :invoke :write [10 21] INFO [2022-01-10 05:41:51,772] jepsen worker 38 - jepsen.util 2198 :ok :read [12 42] INFO [2022-01-10 05:41:51,773] jepsen worker 38 - jepsen.util 2198 :invoke :read [12 nil] INFO [2022-01-10 05:41:51,783] jepsen worker 37 - jepsen.util 2107 :ok :read [12 42] INFO [2022-01-10 05:41:51,784] jepsen worker 37 - jepsen.util 2107 :invoke :write [12 46] INFO [2022-01-10 05:41:51,793] jepsen worker 55 - jepsen.util 2305 :ok :read [18 14] INFO [2022-01-10 05:41:51,793] jepsen worker 55 - jepsen.util 2305 :invoke :write [18 43] INFO [2022-01-10 05:41:51,819] jepsen worker 70 - jepsen.util 2410 :ok :read [23 26] INFO [2022-01-10 05:41:51,820] jepsen worker 70 - jepsen.util 2410 :invoke :write [23 28] INFO [2022-01-10 05:41:51,836] jepsen worker 83 - jepsen.util 2513 :ok :read [27 30] INFO [2022-01-10 05:41:51,836] jepsen worker 83 - jepsen.util 2513 :invoke :read [27 nil] INFO [2022-01-10 05:41:51,880] jepsen worker 10 - jepsen.util 2260 :ok :read [3 2] INFO [2022-01-10 05:41:51,881] jepsen worker 10 - jepsen.util 2260 :invoke :write [3 17] WARN [2022-01-10 05:41:51,885] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17687131,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":17687131,"total_server_duration_us":603,"operation_id":"0x40ef","timeout_ms":30000,"last_server_duration_us":603,"total_duration_us":17687480},{"operation_name":"upsert","last_dispatch_duration_us":2888565,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10680755,"total_server_duration_us":1330557,"encode_duration_us":16496,"operation_id":"0x407e","timeout_ms":10000,"last_server_duration_us":393445,"total_duration_us":10009179},{"operation_name":"upsert","last_dispatch_duration_us":4499441,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11244308,"total_server_duration_us":1638086,"encode_duration_us":16512,"operation_id":"0x4131","timeout_ms":10000,"last_server_duration_us":31108,"total_duration_us":10009015},{"operation_name":"upsert","last_dispatch_duration_us":8384485,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":11102575,"total_server_duration_us":353295,"encode_duration_us":16748,"operation_id":"0x4115","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10003206},{"operation_name":"upsert","last_dispatch_duration_us":8358468,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":11339266,"total_server_duration_us":513420,"encode_duration_us":19219,"operation_id":"0x415c","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10003161},{"operation_name":"upsert","last_dispatch_duration_us":11037191,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11037191,"total_server_duration_us":186359,"encode_duration_us":23405,"operation_id":"0x40be","timeout_ms":10000,"last_server_duration_us":186359,"total_duration_us":10002606},{"operation_name":"upsert","last_dispatch_duration_us":14568860,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":14568860,"total_server_duration_us":396810,"encode_duration_us":16772,"operation_id":"0x40c6","timeout_ms":10000,"last_server_duration_us":396810,"total_duration_us":10000999},{"operation_name":"upsert","last_dispatch_duration_us":9196900,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10563095,"total_server_duration_us":9266691,"encode_duration_us":17377,"operation_id":"0x41a7","timeout_ms":10000,"last_server_duration_us":9046921,"total_duration_us":10000761},{"operation_name":"upsert","last_dispatch_duration_us":9081452,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9081452,"total_server_duration_us":9035244,"encode_duration_us":16647,"operation_id":"0x40f9","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":9092546},{"operation_name":"get","last_dispatch_duration_us":6010528,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":6010528,"total_server_duration_us":409821,"operation_id":"0x4274","timeout_ms":30000,"last_server_duration_us":409821,"total_duration_us":6012715}],"total_count":14}} INFO [2022-01-10 05:41:51,895] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:51,895] jepsen worker nemesis - couchbase.util Rebalance check count: 265 INFO [2022-01-10 05:41:51,895] jepsen worker nemesis - couchbase.util Rebalance stuck count: 203 INFO [2022-01-10 05:41:51,917] jepsen worker 38 - jepsen.util 2198 :ok :read [12 42] INFO [2022-01-10 05:41:51,918] jepsen worker 38 - jepsen.util 2198 :invoke :write [12 42] INFO [2022-01-10 05:41:52,056] jepsen worker 29 - jepsen.util 2099 :info :write [9 44] :RequestTimeoutException INFO [2022-01-10 05:41:52,057] jepsen worker 29 - jepsen.util 2189 :invoke :write [9 29] INFO [2022-01-10 05:41:52,070] jepsen worker 17 - jepsen.util 2177 :ok :read [5 42] INFO [2022-01-10 05:41:52,071] jepsen worker 27 - jepsen.util 2007 :info :write [9 24] :RequestTimeoutException INFO [2022-01-10 05:41:52,072] jepsen worker 27 - jepsen.util 2097 :invoke :read [9 nil] INFO [2022-01-10 05:41:52,073] jepsen worker 17 - jepsen.util 2177 :invoke :write [5 13] INFO [2022-01-10 05:41:52,074] jepsen worker 11 - jepsen.util 1991 :ok :read [3 2] INFO [2022-01-10 05:41:52,076] jepsen worker 11 - jepsen.util 1991 :invoke :read [3 nil] INFO [2022-01-10 05:41:52,077] jepsen worker 59 - jepsen.util 2219 :info :write [19 29] :RequestTimeoutException INFO [2022-01-10 05:41:52,077] jepsen worker 59 - jepsen.util 2309 :invoke :read [19 nil] INFO [2022-01-10 05:41:52,084] jepsen worker 56 - jepsen.util 2306 :ok :read [18 14] INFO [2022-01-10 05:41:52,085] jepsen worker 49 - jepsen.util 1939 :ok :read [16 38] INFO [2022-01-10 05:41:52,085] jepsen worker 56 - jepsen.util 2306 :invoke :write [18 38] INFO [2022-01-10 05:41:52,086] jepsen worker 49 - jepsen.util 1939 :invoke :write [16 35] INFO [2022-01-10 05:41:52,106] jepsen worker 40 - jepsen.util 1660 :ok :read [13 15] INFO [2022-01-10 05:41:52,107] jepsen worker 40 - jepsen.util 1660 :invoke :read [13 nil] INFO [2022-01-10 05:41:52,117] jepsen worker 9 - jepsen.util 2439 :ok :read [3 2] INFO [2022-01-10 05:41:52,117] jepsen worker 9 - jepsen.util 2439 :invoke :write [3 44] INFO [2022-01-10 05:41:52,127] jepsen worker 11 - jepsen.util 1991 :ok :read [3 2] INFO [2022-01-10 05:41:52,131] jepsen worker 11 - jepsen.util 1991 :invoke :read [3 nil] INFO [2022-01-10 05:41:52,177] jepsen worker 11 - jepsen.util 1991 :ok :read [3 2] INFO [2022-01-10 05:41:52,177] jepsen worker 11 - jepsen.util 1991 :invoke :read [3 nil] INFO [2022-01-10 05:41:52,203] jepsen worker 64 - jepsen.util 2134 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:52,203] jepsen worker 72 - jepsen.util 2142 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:52,204] jepsen worker 64 - jepsen.util 2134 :invoke :write [21 47] INFO [2022-01-10 05:41:52,205] jepsen worker 72 - jepsen.util 2142 :invoke :read [24 nil] INFO [2022-01-10 05:41:52,217] jepsen worker 6 - jepsen.util 1986 :ok :read [2 14] INFO [2022-01-10 05:41:52,220] jepsen worker 6 - jepsen.util 1986 :invoke :write [2 6] INFO [2022-01-10 05:41:52,236] jepsen worker 41 - jepsen.util 1841 :info :write [13 31] :RequestTimeoutException INFO [2022-01-10 05:41:52,238] jepsen worker 41 - jepsen.util 1931 :invoke :write [13 11] INFO [2022-01-10 05:41:52,247] jepsen worker 58 - jepsen.util 2218 :ok :read [19 31] INFO [2022-01-10 05:41:52,247] jepsen worker 58 - jepsen.util 2218 :invoke :read [19 nil] INFO [2022-01-10 05:41:52,271] jepsen worker 82 - jepsen.util 1972 :info :write [27 42] :RequestTimeoutException INFO [2022-01-10 05:41:52,272] jepsen worker 82 - jepsen.util 2062 :invoke :read [27 nil] INFO [2022-01-10 05:41:52,274] jepsen worker 46 - jepsen.util 2476 :info :write [15 7] :RequestTimeoutException INFO [2022-01-10 05:41:52,274] jepsen worker 46 - jepsen.util 2566 :invoke :read [15 nil] INFO [2022-01-10 05:41:52,280] jepsen worker 11 - jepsen.util 1991 :ok :read [3 2] INFO [2022-01-10 05:41:52,281] jepsen worker 11 - jepsen.util 1991 :invoke :read [3 nil] INFO [2022-01-10 05:41:52,295] jepsen worker 85 - jepsen.util 2425 :info :write [28 27] :RequestTimeoutException INFO [2022-01-10 05:41:52,296] jepsen worker 85 - jepsen.util 2515 :invoke :write [28 39] INFO [2022-01-10 05:41:52,353] jepsen worker 11 - jepsen.util 1991 :ok :read [3 2] INFO [2022-01-10 05:41:52,354] jepsen worker 11 - jepsen.util 1991 :invoke :write [3 43] INFO [2022-01-10 05:41:52,476] jepsen worker 57 - jepsen.util 2307 :info :write [19 21] :RequestTimeoutException INFO [2022-01-10 05:41:52,477] jepsen worker 57 - jepsen.util 2397 :invoke :write [19 30] INFO [2022-01-10 05:41:52,511] jepsen worker 66 - jepsen.util 1866 :info :write [22 20] :RequestTimeoutException INFO [2022-01-10 05:41:52,512] jepsen worker 66 - jepsen.util 1956 :invoke :read [22 nil] INFO [2022-01-10 05:41:52,574] jepsen worker 69 - jepsen.util 2139 :info :write [23 6] :SyncWriteAmbiguous INFO [2022-01-10 05:41:52,575] jepsen worker 69 - jepsen.util 2229 :invoke :read [23 nil] INFO [2022-01-10 05:41:52,598] jepsen worker 54 - jepsen.util 2124 :ok :read [18 14] INFO [2022-01-10 05:41:52,599] jepsen worker 54 - jepsen.util 2124 :invoke :write [18 37] INFO [2022-01-10 05:41:52,606] jepsen worker 69 - jepsen.util 2229 :ok :read [23 26] INFO [2022-01-10 05:41:52,606] jepsen worker 69 - jepsen.util 2229 :invoke :write [23 34] INFO [2022-01-10 05:41:52,665] jepsen worker 89 - jepsen.util 2429 :ok :read [29 0] INFO [2022-01-10 05:41:52,666] jepsen worker 89 - jepsen.util 2429 :invoke :read [29 nil] INFO [2022-01-10 05:41:52,744] jepsen worker 89 - jepsen.util 2429 :ok :read [29 0] INFO [2022-01-10 05:41:52,744] jepsen worker 89 - jepsen.util 2429 :invoke :read [29 nil] INFO [2022-01-10 05:41:52,755] jepsen worker 51 - jepsen.util 2211 :info :write [17 4] :RequestTimeoutException INFO [2022-01-10 05:41:52,755] jepsen worker 51 - jepsen.util 2301 :invoke :write [17 49] INFO [2022-01-10 05:41:52,760] jepsen worker 36 - jepsen.util 1746 :info :write [12 10] :RequestTimeoutException INFO [2022-01-10 05:41:52,761] jepsen worker 36 - jepsen.util 1836 :invoke :read [12 nil] INFO [2022-01-10 05:41:52,791] jepsen worker 52 - jepsen.util 2482 :info :write [17 5] :RequestTimeoutException INFO [2022-01-10 05:41:52,792] jepsen worker 52 - jepsen.util 2572 :invoke :read [17 nil] INFO [2022-01-10 05:41:52,843] jepsen worker 52 - jepsen.util 2572 :ok :read [17 41] INFO [2022-01-10 05:41:52,843] jepsen worker 52 - jepsen.util 2572 :invoke :read [17 nil] INFO [2022-01-10 05:41:52,845] jepsen worker 89 - jepsen.util 2429 :ok :read [29 0] INFO [2022-01-10 05:41:52,846] jepsen worker 89 - jepsen.util 2429 :invoke :write [29 47] INFO [2022-01-10 05:41:52,903] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:52,903] jepsen worker nemesis - couchbase.util Rebalance check count: 266 INFO [2022-01-10 05:41:52,903] jepsen worker nemesis - couchbase.util Rebalance stuck count: 204 INFO [2022-01-10 05:41:52,970] jepsen worker 45 - jepsen.util 2295 :info :write [15 31] :RequestTimeoutException INFO [2022-01-10 05:41:52,970] jepsen worker 45 - jepsen.util 2385 :invoke :write [15 45] INFO [2022-01-10 05:41:52,971] jepsen worker 61 - jepsen.util 2131 :info :write [20 31] :RequestTimeoutException INFO [2022-01-10 05:41:52,972] jepsen worker 61 - jepsen.util 2221 :invoke :read [20 nil] INFO [2022-01-10 05:41:52,986] jepsen worker 33 - jepsen.util 2193 :info :write [11 21] :RequestTimeoutException INFO [2022-01-10 05:41:52,987] jepsen worker 33 - jepsen.util 2283 :invoke :write [11 17] INFO [2022-01-10 05:41:53,000] jepsen worker 52 - jepsen.util 2572 :ok :read [17 41] INFO [2022-01-10 05:41:53,000] jepsen worker 52 - jepsen.util 2572 :invoke :write [17 42] INFO [2022-01-10 05:41:53,144] jepsen worker 36 - jepsen.util 1836 :ok :read [12 42] INFO [2022-01-10 05:41:53,145] jepsen worker 36 - jepsen.util 1836 :invoke :write [12 9] INFO [2022-01-10 05:41:53,251] jepsen worker 72 - jepsen.util 2142 :ok :read [24 18] INFO [2022-01-10 05:41:53,252] jepsen worker 72 - jepsen.util 2142 :invoke :write [24 21] INFO [2022-01-10 05:41:53,486] jepsen worker 87 - jepsen.util 2247 :info :write [29 31] :RequestTimeoutException INFO [2022-01-10 05:41:53,486] jepsen worker 87 - jepsen.util 2337 :invoke :read [29 nil] INFO [2022-01-10 05:41:53,672] jepsen worker 32 - jepsen.util 2192 :info :write [10 7] :RequestTimeoutException INFO [2022-01-10 05:41:53,672] jepsen worker 32 - jepsen.util 2282 :invoke :write [10 40] INFO [2022-01-10 05:41:53,685] jepsen worker 83 - jepsen.util 2513 :ok :read [27 30] INFO [2022-01-10 05:41:53,686] jepsen worker 44 - jepsen.util 2024 :info :write [14 22] :RequestTimeoutException INFO [2022-01-10 05:41:53,687] jepsen worker 83 - jepsen.util 2513 :invoke :read [27 nil] INFO [2022-01-10 05:41:53,688] jepsen worker 44 - jepsen.util 2114 :invoke :write [14 14] INFO [2022-01-10 05:41:53,750] jepsen worker 87 - jepsen.util 2337 :ok :read [29 0] INFO [2022-01-10 05:41:53,750] jepsen worker 87 - jepsen.util 2337 :invoke :write [29 33] INFO [2022-01-10 05:41:53,845] jepsen worker 26 - jepsen.util 1826 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:41:53,846] jepsen worker 26 - jepsen.util 1826 :invoke :read [8 nil] INFO [2022-01-10 05:41:53,909] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:53,909] jepsen worker nemesis - couchbase.util Rebalance check count: 267 INFO [2022-01-10 05:41:53,909] jepsen worker nemesis - couchbase.util Rebalance stuck count: 205 INFO [2022-01-10 05:41:53,920] jepsen worker 0 - jepsen.util 2070 :ok :read [0 46] INFO [2022-01-10 05:41:53,920] jepsen worker 0 - jepsen.util 2070 :invoke :write [0 31] INFO [2022-01-10 05:41:54,626] jepsen worker 30 - jepsen.util 1830 :info :write [10 12] :RequestTimeoutException INFO [2022-01-10 05:41:54,627] jepsen worker 30 - jepsen.util 1920 :invoke :write [10 30] INFO [2022-01-10 05:41:54,662] jepsen worker 62 - jepsen.util 2312 :info :write [20 12] :RequestTimeoutException INFO [2022-01-10 05:41:54,711] jepsen worker 62 - jepsen.util 2402 :invoke :read [20 nil] INFO [2022-01-10 05:41:54,757] jepsen worker 12 - jepsen.util 1992 :info :write [4 10] :RequestTimeoutException INFO [2022-01-10 05:41:54,757] jepsen worker 12 - jepsen.util 2082 :invoke :read [4 nil] INFO [2022-01-10 05:41:54,915] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:54,915] jepsen worker nemesis - couchbase.util Rebalance check count: 268 INFO [2022-01-10 05:41:54,915] jepsen worker nemesis - couchbase.util Rebalance stuck count: 206 INFO [2022-01-10 05:41:54,935] jepsen worker 8 - jepsen.util 2348 :info :write [2 14] :RequestTimeoutException INFO [2022-01-10 05:41:54,935] jepsen worker 8 - jepsen.util 2438 :invoke :write [2 27] INFO [2022-01-10 05:41:55,148] jepsen worker 2 - jepsen.util 2072 :ok :read [0 46] INFO [2022-01-10 05:41:55,148] jepsen worker 2 - jepsen.util 2072 :invoke :write [0 26] INFO [2022-01-10 05:41:55,160] jepsen worker 84 - jepsen.util 2244 :ok :read [28 13] INFO [2022-01-10 05:41:55,160] jepsen worker 84 - jepsen.util 2244 :invoke :read [28 nil] INFO [2022-01-10 05:41:55,296] jepsen worker 1 - jepsen.util 2251 :info :write [0 26] :RequestTimeoutException INFO [2022-01-10 05:41:55,297] jepsen worker 1 - jepsen.util 2341 :invoke :read [0 nil] INFO [2022-01-10 05:41:55,541] jepsen worker 15 - jepsen.util 1815 :info :write [5 1] :RequestTimeoutException INFO [2022-01-10 05:41:55,542] jepsen worker 15 - jepsen.util 1905 :invoke :read [5 nil] INFO [2022-01-10 05:41:55,626] jepsen worker 5 - jepsen.util 2255 :info :write [1 4] :RequestTimeoutException INFO [2022-01-10 05:41:55,627] jepsen worker 5 - jepsen.util 2345 :invoke :read [1 nil] INFO [2022-01-10 05:41:55,644] jepsen worker 75 - jepsen.util 1335 :info :write [25 17] :RequestTimeoutException INFO [2022-01-10 05:41:55,646] jepsen worker 75 - jepsen.util 1425 :invoke :read [25 nil] INFO [2022-01-10 05:41:55,670] jepsen worker 12 - jepsen.util 2082 :ok :read [4 3] INFO [2022-01-10 05:41:55,671] jepsen worker 12 - jepsen.util 2082 :invoke :write [4 8] INFO [2022-01-10 05:41:55,781] jepsen worker 3 - jepsen.util 2253 :info :write [1 20] :RequestTimeoutException INFO [2022-01-10 05:41:55,782] jepsen worker 3 - jepsen.util 2343 :invoke :read [1 nil] INFO [2022-01-10 05:41:55,925] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:55,925] jepsen worker nemesis - couchbase.util Rebalance check count: 269 INFO [2022-01-10 05:41:55,925] jepsen worker nemesis - couchbase.util Rebalance stuck count: 207 INFO [2022-01-10 05:41:55,976] jepsen worker 80 - jepsen.util 2330 :info :write [26 11] :RequestTimeoutException INFO [2022-01-10 05:41:55,977] jepsen worker 80 - jepsen.util 2420 :invoke :write [26 33] INFO [2022-01-10 05:41:55,982] jepsen worker 40 - jepsen.util 1660 :ok :read [13 15] INFO [2022-01-10 05:41:55,983] jepsen worker 40 - jepsen.util 1660 :invoke :read [13 nil] INFO [2022-01-10 05:41:56,070] jepsen worker 22 - jepsen.util 1822 :info :write [7 29] :RequestTimeoutException INFO [2022-01-10 05:41:56,070] jepsen worker 22 - jepsen.util 1912 :invoke :write [7 5] INFO [2022-01-10 05:41:56,282] jepsen worker 14 - jepsen.util 2354 :info :write [4 10] :RequestTimeoutException INFO [2022-01-10 05:41:56,283] jepsen worker 14 - jepsen.util 2444 :invoke :write [4 35] INFO [2022-01-10 05:41:56,386] jepsen worker 27 - jepsen.util 2097 :ok :read [9 22] INFO [2022-01-10 05:41:56,386] jepsen worker 27 - jepsen.util 2097 :invoke :read [9 nil] INFO [2022-01-10 05:41:56,466] jepsen worker 4 - jepsen.util 2254 :info :write [1 20] :RequestTimeoutException INFO [2022-01-10 05:41:56,466] jepsen worker 4 - jepsen.util 2344 :invoke :write [1 49] INFO [2022-01-10 05:41:56,771] jepsen worker 18 - jepsen.util 1638 :info :write [6 26] :RequestTimeoutException INFO [2022-01-10 05:41:56,772] jepsen worker 18 - jepsen.util 1728 :invoke :write [6 32] INFO [2022-01-10 05:41:56,931] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:56,931] jepsen worker nemesis - couchbase.util Rebalance check count: 270 INFO [2022-01-10 05:41:56,931] jepsen worker nemesis - couchbase.util Rebalance stuck count: 208 INFO [2022-01-10 05:41:57,051] jepsen worker 65 - jepsen.util 2045 :info :write [21 4] :RequestTimeoutException INFO [2022-01-10 05:41:57,051] jepsen worker 65 - jepsen.util 2135 :invoke :read [21 nil] INFO [2022-01-10 05:41:57,061] jepsen worker 47 - jepsen.util 2117 :info :write [15 23] :RequestTimeoutException INFO [2022-01-10 05:41:57,061] jepsen worker 77 - jepsen.util 2417 :info :write [25 47] :RequestTimeoutException INFO [2022-01-10 05:41:57,062] jepsen worker 47 - jepsen.util 2207 :invoke :read [15 nil] INFO [2022-01-10 05:41:57,062] jepsen worker 77 - jepsen.util 2507 :invoke :write [25 26] INFO [2022-01-10 05:41:57,740] jepsen worker 53 - jepsen.util 2393 :info :write [17 33] :RequestTimeoutException INFO [2022-01-10 05:41:57,740] jepsen worker 53 - jepsen.util 2483 :invoke :read [17 nil] INFO [2022-01-10 05:41:57,781] jepsen worker 13 - jepsen.util 2083 :info :write [4 33] :RequestTimeoutException INFO [2022-01-10 05:41:57,782] jepsen worker 13 - jepsen.util 2173 :invoke :write [4 48] INFO [2022-01-10 05:41:57,912] jepsen worker 53 - jepsen.util 2483 :ok :read [17 41] INFO [2022-01-10 05:41:57,913] jepsen worker 53 - jepsen.util 2483 :invoke :write [17 47] INFO [2022-01-10 05:41:57,932] jepsen worker 81 - jepsen.util 2241 :info :write [27 45] :RequestTimeoutException INFO [2022-01-10 05:41:57,932] jepsen worker 81 - jepsen.util 2331 :invoke :write [27 22] INFO [2022-01-10 05:41:57,936] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:57,936] jepsen worker nemesis - couchbase.util Rebalance check count: 271 INFO [2022-01-10 05:41:57,936] jepsen worker nemesis - couchbase.util Rebalance stuck count: 209 INFO [2022-01-10 05:41:58,241] jepsen worker 20 - jepsen.util 1820 :info :write [6 6] :RequestTimeoutException INFO [2022-01-10 05:41:58,241] jepsen worker 20 - jepsen.util 1910 :invoke :write [6 7] INFO [2022-01-10 05:41:58,275] jepsen worker 74 - jepsen.util 2594 :info :write [24 37] :RequestTimeoutException INFO [2022-01-10 05:41:58,275] jepsen worker 63 - jepsen.util 1953 :info :write [21 18] :RequestTimeoutException INFO [2022-01-10 05:41:58,276] jepsen worker 63 - jepsen.util 2043 :invoke :write [21 15] INFO [2022-01-10 05:41:58,279] jepsen worker 74 - jepsen.util 2684 :invoke :write [24 31] INFO [2022-01-10 05:41:58,286] jepsen worker 23 - jepsen.util 2183 :info :write [7 6] :RequestTimeoutException INFO [2022-01-10 05:41:58,286] jepsen worker 23 - jepsen.util 2273 :invoke :write [7 31] INFO [2022-01-10 05:41:58,606] jepsen worker 79 - jepsen.util 2329 :info :write [26 49] :RequestTimeoutException INFO [2022-01-10 05:41:58,606] jepsen worker 79 - jepsen.util 2419 :invoke :read [26 nil] INFO [2022-01-10 05:41:58,645] jepsen worker 7 - jepsen.util 2257 :info :write [2 7] :RequestTimeoutException INFO [2022-01-10 05:41:58,645] jepsen worker 7 - jepsen.util 2347 :invoke :read [2 nil] INFO [2022-01-10 05:41:58,870] jepsen worker 16 - jepsen.util 2446 :ok :read [5 42] INFO [2022-01-10 05:41:58,870] jepsen worker 16 - jepsen.util 2446 :invoke :read [5 nil] INFO [2022-01-10 05:41:58,873] jepsen worker 58 - jepsen.util 2218 :ok :read [19 31] INFO [2022-01-10 05:41:58,873] jepsen worker 58 - jepsen.util 2218 :invoke :read [19 nil] INFO [2022-01-10 05:41:58,942] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:58,942] jepsen worker nemesis - couchbase.util Rebalance check count: 272 INFO [2022-01-10 05:41:58,942] jepsen worker nemesis - couchbase.util Rebalance stuck count: 210 INFO [2022-01-10 05:41:59,046] jepsen worker 86 - jepsen.util 1706 :ok :read [28 13] INFO [2022-01-10 05:41:59,046] jepsen worker 86 - jepsen.util 1706 :invoke :write [28 6] INFO [2022-01-10 05:41:59,130] jepsen worker 79 - jepsen.util 2419 :ok :read [26 10] INFO [2022-01-10 05:41:59,131] jepsen worker 79 - jepsen.util 2419 :invoke :write [26 22] INFO [2022-01-10 05:41:59,666] jepsen worker 50 - jepsen.util 2300 :ok :read [16 38] INFO [2022-01-10 05:41:59,666] jepsen worker 50 - jepsen.util 2300 :invoke :write [16 21] INFO [2022-01-10 05:41:59,906] jepsen worker 39 - jepsen.util 2379 :info :write [13 10] :RequestTimeoutException INFO [2022-01-10 05:41:59,947] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:41:59,965] jepsen worker 39 - jepsen.util 2469 :invoke :write [13 13] INFO [2022-01-10 05:41:59,965] jepsen worker nemesis - couchbase.util Rebalance check count: 273 INFO [2022-01-10 05:41:59,965] jepsen worker nemesis - couchbase.util Rebalance stuck count: 211 WARN [2022-01-10 05:41:59,997] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":25952939,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25952939,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":10208763,"encode_duration_us":19854,"operation_id":"0x3f03","timeout_ms":10000,"last_server_duration_us":10208763,"total_duration_us":10002097},{"operation_name":"upsert","last_dispatch_duration_us":12342470,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14464724,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10601807,"encode_duration_us":17630,"operation_id":"0x4129","timeout_ms":10000,"last_server_duration_us":10100580,"total_duration_us":10004281},{"operation_name":"upsert","last_dispatch_duration_us":16254575,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16254575,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9024634,"encode_duration_us":16586,"operation_id":"0x4148","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10009878}],"total_count":3}} WARN [2022-01-10 05:41:59,997] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":22895187,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":22895187,"total_server_duration_us":416679,"operation_id":"0x3fd3","timeout_ms":30000,"last_server_duration_us":416679,"total_duration_us":22903685},{"operation_name":"get","last_dispatch_duration_us":22447437,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":22447437,"total_server_duration_us":210620,"operation_id":"0x3ffa","timeout_ms":30000,"last_server_duration_us":210620,"total_duration_us":22451979},{"operation_name":"get","last_dispatch_duration_us":3534238,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":3534238,"total_server_duration_us":32,"operation_id":"0x4217","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":14823931},{"operation_name":"get","last_dispatch_duration_us":3369637,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":3369637,"total_server_duration_us":12633,"operation_id":"0x4265","timeout_ms":30000,"last_server_duration_us":12633,"total_duration_us":13123998},{"operation_name":"get","last_dispatch_duration_us":12989178,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12989178,"total_server_duration_us":84,"operation_id":"0x41e5","timeout_ms":30000,"last_server_duration_us":84,"total_duration_us":12991730},{"operation_name":"upsert","last_dispatch_duration_us":607859,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":2408687,"total_server_duration_us":711754,"encode_duration_us":17079,"operation_id":"0x4290","timeout_ms":10000,"last_server_duration_us":57139,"total_duration_us":10010305},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","encode_duration_us":16400,"operation_id":"0x4309","timeout_ms":10000,"total_duration_us":10006697},{"operation_name":"upsert","last_dispatch_duration_us":13268795,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13268795,"total_server_duration_us":390930,"encode_duration_us":17301,"operation_id":"0x420a","timeout_ms":10000,"last_server_duration_us":390930,"total_duration_us":10006577},{"operation_name":"upsert","last_dispatch_duration_us":13670995,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13670995,"total_server_duration_us":411531,"encode_duration_us":28482,"operation_id":"0x420b","timeout_ms":10000,"last_server_duration_us":411531,"total_duration_us":10005629},{"operation_name":"upsert","last_dispatch_duration_us":4154364,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":4154364,"total_server_duration_us":206985,"encode_duration_us":25542,"operation_id":"0x4398","timeout_ms":10000,"last_server_duration_us":206985,"total_duration_us":10002932}],"total_count":21}} INFO [2022-01-10 05:42:00,002] jepsen worker 48 - jepsen.util 1758 :info :write [16 0] :RequestTimeoutException INFO [2022-01-10 05:42:00,002] jepsen worker 48 - jepsen.util 1848 :invoke :write [16 13] INFO [2022-01-10 05:42:00,016] jepsen worker 28 - jepsen.util 2008 :info :write [9 8] :RequestTimeoutException INFO [2022-01-10 05:42:00,017] jepsen worker 28 - jepsen.util 2098 :invoke :write [9 15] INFO [2022-01-10 05:42:00,245] jepsen worker 68 - jepsen.util 2228 :info :write [22 49] :RequestTimeoutException INFO [2022-01-10 05:42:00,246] jepsen worker 68 - jepsen.util 2318 :invoke :read [22 nil] INFO [2022-01-10 05:42:00,280] jepsen worker 43 - jepsen.util 2023 :ok :read [14 14] INFO [2022-01-10 05:42:00,281] jepsen worker 43 - jepsen.util 2023 :invoke :read [14 nil] INFO [2022-01-10 05:42:00,436] jepsen worker 19 - jepsen.util 2269 :info :write [6 38] :RequestTimeoutException INFO [2022-01-10 05:42:00,436] jepsen worker 19 - jepsen.util 2359 :invoke :read [6 nil] INFO [2022-01-10 05:42:00,476] jepsen worker 1 - jepsen.util 2341 :ok :read [0 46] INFO [2022-01-10 05:42:00,477] jepsen worker 1 - jepsen.util 2341 :invoke :read [0 nil] INFO [2022-01-10 05:42:00,907] jepsen worker 31 - jepsen.util 2371 :info :write [10 21] :SyncWriteAmbiguous INFO [2022-01-10 05:42:00,907] jepsen worker 31 - jepsen.util 2461 :invoke :write [10 28] INFO [2022-01-10 05:42:00,971] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:00,971] jepsen worker nemesis - couchbase.util Rebalance check count: 274 INFO [2022-01-10 05:42:00,971] jepsen worker nemesis - couchbase.util Rebalance stuck count: 212 INFO [2022-01-10 05:42:00,999] jepsen worker 88 - jepsen.util 2068 :info :write [29 29] :SyncWriteAmbiguous INFO [2022-01-10 05:42:01,000] jepsen worker 88 - jepsen.util 2158 :invoke :read [29 nil] INFO [2022-01-10 05:42:01,163] jepsen worker 10 - jepsen.util 2260 :info :write [3 17] :SyncWriteAmbiguous INFO [2022-01-10 05:42:01,164] jepsen worker 10 - jepsen.util 2350 :invoke :read [3 nil] INFO [2022-01-10 05:42:01,190] jepsen worker 78 - jepsen.util 2328 :info :write [26 3] :RequestTimeoutException INFO [2022-01-10 05:42:01,191] jepsen worker 78 - jepsen.util 2418 :invoke :write [26 13] INFO [2022-01-10 05:42:01,228] jepsen worker 10 - jepsen.util 2350 :ok :read [3 2] INFO [2022-01-10 05:42:01,228] jepsen worker 10 - jepsen.util 2350 :invoke :read [3 nil] INFO [2022-01-10 05:42:01,251] jepsen worker 10 - jepsen.util 2350 :ok :read [3 2] INFO [2022-01-10 05:42:01,252] jepsen worker 10 - jepsen.util 2350 :invoke :write [3 5] INFO [2022-01-10 05:42:01,309] jepsen worker 82 - jepsen.util 2062 :ok :read [27 30] INFO [2022-01-10 05:42:01,309] jepsen worker 82 - jepsen.util 2062 :invoke :write [27 23] INFO [2022-01-10 05:42:01,444] jepsen worker 67 - jepsen.util 1867 :info :write [22 30] :RequestTimeoutException INFO [2022-01-10 05:42:01,445] jepsen worker 67 - jepsen.util 1957 :invoke :write [22 5] WARN [2022-01-10 05:42:01,499] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17463082,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":17463082,"total_server_duration_us":376,"operation_id":"0x40f8","timeout_ms":30000,"last_server_duration_us":376,"total_duration_us":17465069},{"operation_name":"upsert","last_dispatch_duration_us":15148336,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":15148336,"total_server_duration_us":18570,"encode_duration_us":16328,"operation_id":"0x431c","timeout_ms":10000,"last_server_duration_us":18570,"total_duration_us":10009299},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":17309,"operation_id":"0x435c","timeout_ms":10000,"total_duration_us":10009137},{"operation_name":"upsert","last_dispatch_duration_us":1819607,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":5666415,"total_server_duration_us":3220650,"encode_duration_us":18711,"operation_id":"0x4386","timeout_ms":10000,"last_server_duration_us":1202751,"total_duration_us":10008706},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":20115,"operation_id":"0x435d","timeout_ms":10000,"total_duration_us":10008335},{"operation_name":"upsert","last_dispatch_duration_us":1444282,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1444282,"total_server_duration_us":194790,"encode_duration_us":16897,"operation_id":"0x434a","timeout_ms":10000,"last_server_duration_us":194790,"total_duration_us":10008145},{"operation_name":"upsert","last_dispatch_duration_us":722732,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":6259689,"total_server_duration_us":2032881,"encode_duration_us":16501,"operation_id":"0x4271","timeout_ms":10000,"last_server_duration_us":389256,"total_duration_us":10007250},{"operation_name":"upsert","last_dispatch_duration_us":561555,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":2260266,"total_server_duration_us":1931429,"encode_duration_us":18076,"operation_id":"0x4292","timeout_ms":10000,"last_server_duration_us":548622,"total_duration_us":10006085},{"operation_name":"upsert","encode_duration_us":23229,"operation_id":"0x438d","timeout_ms":10000,"total_duration_us":10005437},{"operation_name":"upsert","last_dispatch_duration_us":18131423,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":18131423,"total_server_duration_us":9030999,"encode_duration_us":20757,"operation_id":"0x4291","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10003622}],"total_count":13}} WARN [2022-01-10 05:42:01,499] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18050549,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18050549,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9025695,"encode_duration_us":18143,"operation_id":"0x40ce","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10002844},{"operation_name":"upsert","last_dispatch_duration_us":18131423,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18131423,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9030999,"encode_duration_us":20757,"operation_id":"0x4291","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10003622}],"total_count":2}} INFO [2022-01-10 05:42:01,506] jepsen worker 56 - jepsen.util 2306 :info :write [18 38] :SyncWriteAmbiguous INFO [2022-01-10 05:42:01,506] jepsen worker 56 - jepsen.util 2396 :invoke :write [18 26] INFO [2022-01-10 05:42:01,599] jepsen worker 76 - jepsen.util 1516 :info :write [25 39] :RequestTimeoutException WARN [2022-01-10 05:42:01,600] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":20334313,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":23640832,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9632421,"encode_duration_us":17539,"operation_id":"0x4090","timeout_ms":10000,"last_server_duration_us":9502974,"total_duration_us":10001675}],"total_count":1}} WARN [2022-01-10 05:42:01,600] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18015068,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":18015068,"total_server_duration_us":43,"operation_id":"0x421a","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":18585493},{"operation_name":"get","last_dispatch_duration_us":12011232,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12011232,"total_server_duration_us":838312,"operation_id":"0x438e","timeout_ms":30000,"last_server_duration_us":838312,"total_duration_us":12013671},{"operation_name":"upsert","encode_duration_us":16454,"operation_id":"0x4305","timeout_ms":10000,"total_duration_us":10008758},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":22574,"operation_id":"0x4393","timeout_ms":10000,"total_duration_us":10008757},{"operation_name":"upsert","last_dispatch_duration_us":615543,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":8710473,"total_server_duration_us":1280081,"encode_duration_us":16068,"operation_id":"0x432d","timeout_ms":10000,"last_server_duration_us":137722,"total_duration_us":10007630},{"operation_name":"upsert","last_dispatch_duration_us":1515744,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":2486529,"total_server_duration_us":2039190,"encode_duration_us":18668,"operation_id":"0x42db","timeout_ms":10000,"last_server_duration_us":1504864,"total_duration_us":10007335},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":22932,"operation_id":"0x438f","timeout_ms":10000,"total_duration_us":10005821},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":23233,"operation_id":"0x4390","timeout_ms":10000,"total_duration_us":10005170},{"operation_name":"upsert","last_dispatch_duration_us":19219956,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":19219956,"total_server_duration_us":790266,"encode_duration_us":25156,"operation_id":"0x4266","timeout_ms":10000,"last_server_duration_us":790266,"total_duration_us":10004118},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":19177,"operation_id":"0x43db","timeout_ms":10000,"total_duration_us":10002982}],"total_count":14}} INFO [2022-01-10 05:42:01,600] jepsen worker 76 - jepsen.util 1606 :invoke :read [25 nil] WARN [2022-01-10 05:42:01,601] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17474051,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":17474051,"total_server_duration_us":3492608,"operation_id":"0x426f","timeout_ms":30000,"last_server_duration_us":3492608,"total_duration_us":17475989},{"operation_name":"upsert","last_dispatch_duration_us":14682177,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":14682177,"total_server_duration_us":829452,"encode_duration_us":16578,"operation_id":"0x4276","timeout_ms":10000,"last_server_duration_us":829452,"total_duration_us":10007667},{"operation_name":"upsert","last_dispatch_duration_us":948558,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":5011093,"total_server_duration_us":887715,"encode_duration_us":16629,"operation_id":"0x4336","timeout_ms":10000,"last_server_duration_us":303491,"total_duration_us":10007591},{"operation_name":"upsert","last_dispatch_duration_us":10664000,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10664000,"total_server_duration_us":393165,"encode_duration_us":17854,"operation_id":"0x43dd","timeout_ms":10000,"last_server_duration_us":393165,"total_duration_us":10007478},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":19692,"operation_id":"0x42a0","timeout_ms":10000,"total_duration_us":10007346},{"operation_name":"upsert","last_dispatch_duration_us":263098,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9713897,"total_server_duration_us":489807,"encode_duration_us":17268,"operation_id":"0x42b1","timeout_ms":10000,"last_server_duration_us":107269,"total_duration_us":10006601},{"operation_name":"upsert","last_dispatch_duration_us":970055,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":5681568,"total_server_duration_us":4297134,"encode_duration_us":16698,"operation_id":"0x4378","timeout_ms":10000,"last_server_duration_us":592543,"total_duration_us":10003671},{"operation_name":"upsert","last_dispatch_duration_us":14711173,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":14711173,"total_server_duration_us":596546,"encode_duration_us":16638,"operation_id":"0x4307","timeout_ms":10000,"last_server_duration_us":596546,"total_duration_us":10002021},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":17248,"operation_id":"0x42dd","timeout_ms":10000,"total_duration_us":10001601},{"operation_name":"get","last_dispatch_duration_us":9019356,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9019356,"total_server_duration_us":216013,"operation_id":"0x4457","timeout_ms":30000,"last_server_duration_us":216013,"total_duration_us":9022113}],"total_count":11}} WARN [2022-01-10 05:42:01,607] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":23861583,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":23861583,"total_server_duration_us":32,"operation_id":"0x3fc5","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":23862637},{"operation_name":"get","last_dispatch_duration_us":18176464,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":18176464,"total_server_duration_us":15,"operation_id":"0x40c9","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":18177720},{"operation_name":"get","last_dispatch_duration_us":18154299,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":18154299,"total_server_duration_us":27,"operation_id":"0x40c8","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":18155375},{"operation_name":"get","last_dispatch_duration_us":16150701,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":16150701,"total_server_duration_us":43,"operation_id":"0x4134","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":16151795},{"operation_name":"get","last_dispatch_duration_us":15774235,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":15774235,"total_server_duration_us":15,"operation_id":"0x415b","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":15775626},{"operation_name":"get","last_dispatch_duration_us":15397910,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":15397910,"total_server_duration_us":23,"operation_id":"0x4168","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":15400312},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":18770,"operation_id":"0x42a1","timeout_ms":10000,"total_duration_us":10008706},{"operation_name":"upsert","last_dispatch_duration_us":6157499,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":12255021,"total_server_duration_us":978768,"encode_duration_us":16158,"operation_id":"0x4270","timeout_ms":10000,"last_server_duration_us":580271,"total_duration_us":10007902},{"operation_name":"upsert","last_dispatch_duration_us":332441,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":7266752,"total_server_duration_us":3289923,"encode_duration_us":17304,"operation_id":"0x42f0","timeout_ms":10000,"last_server_duration_us":308522,"total_duration_us":10007864},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":16447,"operation_id":"0x42b2","timeout_ms":10000,"total_duration_us":10006343}],"total_count":22}} WARN [2022-01-10 05:42:01,701] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11641897,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14296561,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":11264283,"encode_duration_us":17604,"operation_id":"0x41b5","timeout_ms":10000,"last_server_duration_us":10791324,"total_duration_us":10001506},{"operation_name":"upsert","last_dispatch_duration_us":9043558,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9434848,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9273610,"encode_duration_us":19436,"operation_id":"0x4236","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10004610},{"operation_name":"upsert","last_dispatch_duration_us":9049704,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9938726,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9503342,"encode_duration_us":16341,"operation_id":"0x4132","timeout_ms":10000,"last_server_duration_us":9037366,"total_duration_us":10004138}],"total_count":3}} INFO [2022-01-10 05:42:01,706] jepsen worker 71 - jepsen.util 1691 :info :write [23 40] :RequestTimeoutException INFO [2022-01-10 05:42:01,706] jepsen worker 71 - jepsen.util 1781 :invoke :write [23 20] WARN [2022-01-10 05:42:01,708] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":25314782,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":25314782,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9024634,"encode_duration_us":16498,"operation_id":"0x3f82","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10002647},{"operation_name":"upsert","last_dispatch_duration_us":12935748,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":21646084,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":10172840,"encode_duration_us":17028,"operation_id":"0x4023","timeout_ms":10000,"last_server_duration_us":9741690,"total_duration_us":10007214},{"operation_name":"upsert","last_dispatch_duration_us":9222672,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15412169,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9066219,"encode_duration_us":18675,"operation_id":"0x4308","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10004260}],"total_count":3}} INFO [2022-01-10 05:42:01,756] jepsen worker 35 - jepsen.util 1925 :info :write [11 26] :RequestTimeoutException INFO [2022-01-10 05:42:01,757] jepsen worker 35 - jepsen.util 2015 :invoke :read [11 nil] INFO [2022-01-10 05:42:01,806] jepsen worker 37 - jepsen.util 2107 :info :write [12 46] :RequestTimeoutException INFO [2022-01-10 05:42:01,807] jepsen worker 37 - jepsen.util 2197 :invoke :read [12 nil] INFO [2022-01-10 05:42:01,820] jepsen worker 55 - jepsen.util 2305 :info :write [18 43] :RequestTimeoutException INFO [2022-01-10 05:42:01,821] jepsen worker 55 - jepsen.util 2395 :invoke :read [18 nil] INFO [2022-01-10 05:42:01,846] jepsen worker 70 - jepsen.util 2410 :info :write [23 28] :RequestTimeoutException INFO [2022-01-10 05:42:01,847] jepsen worker 70 - jepsen.util 2500 :invoke :read [23 nil] INFO [2022-01-10 05:42:01,860] jepsen worker 55 - jepsen.util 2395 :ok :read [18 14] INFO [2022-01-10 05:42:01,860] jepsen worker 55 - jepsen.util 2395 :invoke :read [18 nil] WARN [2022-01-10 05:42:01,898] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16003283,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":16003283,"total_server_duration_us":23475,"operation_id":"0x4166","timeout_ms":30000,"last_server_duration_us":23475,"total_duration_us":16005280},{"operation_name":"get","last_dispatch_duration_us":12094698,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12094698,"total_server_duration_us":43,"operation_id":"0x420c","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":12095005},{"operation_name":"get","last_dispatch_duration_us":11559799,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11559799,"total_server_duration_us":19,"operation_id":"0x421c","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":11559993},{"operation_name":"upsert","last_dispatch_duration_us":4861383,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9772993,"total_server_duration_us":1169734,"encode_duration_us":17445,"operation_id":"0x4413","timeout_ms":10000,"last_server_duration_us":386197,"total_duration_us":10009891},{"operation_name":"upsert","last_dispatch_duration_us":2446554,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12089600,"total_server_duration_us":798124,"encode_duration_us":21395,"operation_id":"0x429f","timeout_ms":10000,"last_server_duration_us":397091,"total_duration_us":10006945},{"operation_name":"upsert","last_dispatch_duration_us":119619,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":7905882,"total_server_duration_us":1654831,"encode_duration_us":16276,"operation_id":"0x4411","timeout_ms":10000,"last_server_duration_us":96250,"total_duration_us":10005943},{"operation_name":"upsert","last_dispatch_duration_us":645116,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":7231501,"total_server_duration_us":2427126,"encode_duration_us":16424,"operation_id":"0x4371","timeout_ms":10000,"last_server_duration_us":372966,"total_duration_us":10005551},{"operation_name":"upsert","last_dispatch_duration_us":4947238,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":14346514,"total_server_duration_us":3260551,"encode_duration_us":18642,"operation_id":"0x432e","timeout_ms":10000,"last_server_duration_us":2671923,"total_duration_us":10004304},{"operation_name":"upsert","last_dispatch_duration_us":2249199,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11561451,"total_server_duration_us":2157610,"encode_duration_us":25999,"operation_id":"0x4267","timeout_ms":10000,"last_server_duration_us":805768,"total_duration_us":10003302},{"operation_name":"upsert","last_dispatch_duration_us":9066664,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9066664,"total_server_duration_us":9050106,"encode_duration_us":25301,"operation_id":"0x42ad","timeout_ms":10000,"last_server_duration_us":9050106,"total_duration_us":9070694}],"total_count":17}} INFO [2022-01-10 05:42:01,931] jepsen worker 40 - jepsen.util 1660 :ok :read [13 15] INFO [2022-01-10 05:42:01,931] jepsen worker 40 - jepsen.util 1660 :invoke :read [13 nil] INFO [2022-01-10 05:42:01,937] jepsen worker 38 - jepsen.util 2198 :info :write [12 42] :RequestTimeoutException INFO [2022-01-10 05:42:01,937] jepsen worker 38 - jepsen.util 2288 :invoke :write [12 37] INFO [2022-01-10 05:42:01,977] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:01,977] jepsen worker nemesis - couchbase.util Rebalance check count: 275 INFO [2022-01-10 05:42:01,977] jepsen worker nemesis - couchbase.util Rebalance stuck count: 213 INFO [2022-01-10 05:42:02,081] jepsen worker 29 - jepsen.util 2189 :info :write [9 29] :RequestTimeoutException INFO [2022-01-10 05:42:02,081] jepsen worker 29 - jepsen.util 2279 :invoke :read [9 nil] INFO [2022-01-10 05:42:02,100] jepsen worker 17 - jepsen.util 2177 :info :write [5 13] :RequestTimeoutException INFO [2022-01-10 05:42:02,100] jepsen worker 17 - jepsen.util 2267 :invoke :read [5 nil] INFO [2022-01-10 05:42:02,120] jepsen worker 49 - jepsen.util 1939 :info :write [16 35] :RequestTimeoutException INFO [2022-01-10 05:42:02,121] jepsen worker 49 - jepsen.util 2029 :invoke :write [16 26] INFO [2022-01-10 05:42:02,145] jepsen worker 9 - jepsen.util 2439 :info :write [3 44] :RequestTimeoutException INFO [2022-01-10 05:42:02,146] jepsen worker 9 - jepsen.util 2529 :invoke :read [3 nil] INFO [2022-01-10 05:42:02,229] jepsen worker 64 - jepsen.util 2134 :info :write [21 47] :RequestTimeoutException INFO [2022-01-10 05:42:02,230] jepsen worker 64 - jepsen.util 2224 :invoke :read [21 nil] INFO [2022-01-10 05:42:02,250] jepsen worker 6 - jepsen.util 1986 :info :write [2 6] :RequestTimeoutException INFO [2022-01-10 05:42:02,250] jepsen worker 6 - jepsen.util 2076 :invoke :write [2 24] INFO [2022-01-10 05:42:02,259] jepsen worker 41 - jepsen.util 1931 :info :write [13 11] :RequestTimeoutException INFO [2022-01-10 05:42:02,260] jepsen worker 41 - jepsen.util 2021 :invoke :write [13 27] INFO [2022-01-10 05:42:02,321] jepsen worker 85 - jepsen.util 2515 :info :write [28 39] :RequestTimeoutException INFO [2022-01-10 05:42:02,322] jepsen worker 85 - jepsen.util 2605 :invoke :read [28 nil] INFO [2022-01-10 05:42:02,375] jepsen worker 11 - jepsen.util 1991 :info :write [3 43] :RequestTimeoutException INFO [2022-01-10 05:42:02,375] jepsen worker 11 - jepsen.util 2081 :invoke :read [3 nil] INFO [2022-01-10 05:42:02,504] jepsen worker 57 - jepsen.util 2397 :info :write [19 30] :RequestTimeoutException INFO [2022-01-10 05:42:02,505] jepsen worker 57 - jepsen.util 2487 :invoke :write [19 37] INFO [2022-01-10 05:42:02,518] jepsen worker 65 - jepsen.util 2135 :ok :read [21 7] INFO [2022-01-10 05:42:02,518] jepsen worker 65 - jepsen.util 2135 :invoke :read [21 nil] INFO [2022-01-10 05:42:02,576] jepsen worker 55 - jepsen.util 2395 :ok :read [18 14] INFO [2022-01-10 05:42:02,577] jepsen worker 55 - jepsen.util 2395 :invoke :read [18 nil] INFO [2022-01-10 05:42:02,597] jepsen worker 9 - jepsen.util 2529 :ok :read [3 2] INFO [2022-01-10 05:42:02,598] jepsen worker 9 - jepsen.util 2529 :invoke :write [3 31] INFO [2022-01-10 05:42:02,609] jepsen worker 55 - jepsen.util 2395 :ok :read [18 14] INFO [2022-01-10 05:42:02,609] jepsen worker 55 - jepsen.util 2395 :invoke :read [18 nil] INFO [2022-01-10 05:42:02,620] jepsen worker 54 - jepsen.util 2124 :info :write [18 37] :RequestTimeoutException INFO [2022-01-10 05:42:02,621] jepsen worker 54 - jepsen.util 2214 :invoke :read [18 nil] INFO [2022-01-10 05:42:02,629] jepsen worker 69 - jepsen.util 2229 :info :write [23 34] :RequestTimeoutException INFO [2022-01-10 05:42:02,630] jepsen worker 69 - jepsen.util 2319 :invoke :read [23 nil] INFO [2022-01-10 05:42:02,632] jepsen worker 55 - jepsen.util 2395 :ok :read [18 14] INFO [2022-01-10 05:42:02,632] jepsen worker 55 - jepsen.util 2395 :invoke :read [18 nil] INFO [2022-01-10 05:42:02,700] jepsen worker 55 - jepsen.util 2395 :ok :read [18 14] INFO [2022-01-10 05:42:02,700] jepsen worker 55 - jepsen.util 2395 :invoke :read [18 nil] INFO [2022-01-10 05:42:02,703] jepsen worker 69 - jepsen.util 2319 :ok :read [23 26] INFO [2022-01-10 05:42:02,703] jepsen worker 69 - jepsen.util 2319 :invoke :read [23 nil] INFO [2022-01-10 05:42:02,722] jepsen worker 24 - jepsen.util 2004 :ok :read [8 27] INFO [2022-01-10 05:42:02,723] jepsen worker 24 - jepsen.util 2004 :invoke :read [8 nil] INFO [2022-01-10 05:42:02,727] jepsen worker 84 - jepsen.util 2244 :ok :read [28 13] INFO [2022-01-10 05:42:02,728] jepsen worker 84 - jepsen.util 2244 :invoke :read [28 nil] INFO [2022-01-10 05:42:02,782] jepsen worker 51 - jepsen.util 2301 :info :write [17 49] :RequestTimeoutException INFO [2022-01-10 05:42:02,782] jepsen worker 51 - jepsen.util 2391 :invoke :write [17 48] INFO [2022-01-10 05:42:02,872] jepsen worker 89 - jepsen.util 2429 :info :write [29 47] :RequestTimeoutException INFO [2022-01-10 05:42:02,872] jepsen worker 89 - jepsen.util 2519 :invoke :write [29 48] INFO [2022-01-10 05:42:02,983] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:02,983] jepsen worker nemesis - couchbase.util Rebalance check count: 276 INFO [2022-01-10 05:42:02,983] jepsen worker nemesis - couchbase.util Rebalance stuck count: 214 INFO [2022-01-10 05:42:02,994] jepsen worker 55 - jepsen.util 2395 :ok :read [18 14] INFO [2022-01-10 05:42:02,995] jepsen worker 45 - jepsen.util 2385 :info :write [15 45] :RequestTimeoutException INFO [2022-01-10 05:42:02,995] jepsen worker 55 - jepsen.util 2395 :invoke :write [18 32] INFO [2022-01-10 05:42:02,996] jepsen worker 45 - jepsen.util 2475 :invoke :write [15 9] INFO [2022-01-10 05:42:03,002] jepsen worker 37 - jepsen.util 2197 :ok :read [12 42] INFO [2022-01-10 05:42:03,002] jepsen worker 37 - jepsen.util 2197 :invoke :read [12 nil] INFO [2022-01-10 05:42:03,010] jepsen worker 33 - jepsen.util 2283 :info :write [11 17] :RequestTimeoutException INFO [2022-01-10 05:42:03,010] jepsen worker 33 - jepsen.util 2373 :invoke :write [11 23] INFO [2022-01-10 05:42:03,016] jepsen worker 11 - jepsen.util 2081 :ok :read [3 2] INFO [2022-01-10 05:42:03,017] jepsen worker 11 - jepsen.util 2081 :invoke :write [3 10] INFO [2022-01-10 05:42:03,021] jepsen worker 52 - jepsen.util 2572 :info :write [17 42] :RequestTimeoutException INFO [2022-01-10 05:42:03,022] jepsen worker 52 - jepsen.util 2662 :invoke :read [17 nil] INFO [2022-01-10 05:42:03,056] jepsen worker 37 - jepsen.util 2197 :ok :read [12 42] INFO [2022-01-10 05:42:03,057] jepsen worker 37 - jepsen.util 2197 :invoke :write [12 8] INFO [2022-01-10 05:42:03,166] jepsen worker 36 - jepsen.util 1836 :info :write [12 9] :RequestTimeoutException INFO [2022-01-10 05:42:03,166] jepsen worker 36 - jepsen.util 1926 :invoke :read [12 nil] INFO [2022-01-10 05:42:03,270] jepsen worker 72 - jepsen.util 2142 :info :write [24 21] :RequestTimeoutException INFO [2022-01-10 05:42:03,271] jepsen worker 72 - jepsen.util 2232 :invoke :write [24 8] INFO [2022-01-10 05:42:03,371] jepsen worker 52 - jepsen.util 2662 :ok :read [17 41] INFO [2022-01-10 05:42:03,371] jepsen worker 52 - jepsen.util 2662 :invoke :write [17 14] INFO [2022-01-10 05:42:03,696] jepsen worker 32 - jepsen.util 2282 :info :write [10 40] :RequestTimeoutException INFO [2022-01-10 05:42:03,696] jepsen worker 32 - jepsen.util 2372 :invoke :read [10 nil] INFO [2022-01-10 05:42:03,711] jepsen worker 44 - jepsen.util 2114 :info :write [14 14] :RequestTimeoutException INFO [2022-01-10 05:42:03,712] jepsen worker 44 - jepsen.util 2204 :invoke :read [14 nil] INFO [2022-01-10 05:42:03,772] jepsen worker 87 - jepsen.util 2337 :info :write [29 33] :RequestTimeoutException INFO [2022-01-10 05:42:03,772] jepsen worker 87 - jepsen.util 2427 :invoke :read [29 nil] INFO [2022-01-10 05:42:03,946] jepsen worker 0 - jepsen.util 2070 :info :write [0 31] :RequestTimeoutException INFO [2022-01-10 05:42:03,947] jepsen worker 0 - jepsen.util 2160 :invoke :read [0 nil] INFO [2022-01-10 05:42:03,990] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:03,990] jepsen worker nemesis - couchbase.util Rebalance check count: 277 INFO [2022-01-10 05:42:03,991] jepsen worker nemesis - couchbase.util Rebalance stuck count: 215 INFO [2022-01-10 05:42:04,185] jepsen worker 85 - jepsen.util 2605 :ok :read [28 13] INFO [2022-01-10 05:42:04,185] jepsen worker 85 - jepsen.util 2605 :invoke :read [28 nil] INFO [2022-01-10 05:42:04,322] jepsen worker 42 - jepsen.util 2112 :fail :read [14 nil] :UnambiguousTimeoutException INFO [2022-01-10 05:42:04,323] jepsen worker 42 - jepsen.util 2112 :invoke :write [14 29] INFO [2022-01-10 05:42:04,420] jepsen worker 87 - jepsen.util 2427 :ok :read [29 0] INFO [2022-01-10 05:42:04,421] jepsen worker 87 - jepsen.util 2427 :invoke :write [29 47] INFO [2022-01-10 05:42:04,498] jepsen worker 3 - jepsen.util 2343 :ok :read [1 41] INFO [2022-01-10 05:42:04,498] jepsen worker 3 - jepsen.util 2343 :invoke :read [1 nil] INFO [2022-01-10 05:42:04,504] jepsen worker 88 - jepsen.util 2158 :ok :read [29 0] INFO [2022-01-10 05:42:04,504] jepsen worker 88 - jepsen.util 2158 :invoke :write [29 44] INFO [2022-01-10 05:42:04,736] jepsen worker 30 - jepsen.util 1920 :info :write [10 30] :RequestTimeoutException INFO [2022-01-10 05:42:04,737] jepsen worker 30 - jepsen.util 2010 :invoke :write [10 34] INFO [2022-01-10 05:42:04,808] jepsen worker 17 - jepsen.util 2267 :ok :read [5 42] INFO [2022-01-10 05:42:04,808] jepsen worker 17 - jepsen.util 2267 :invoke :read [5 nil] INFO [2022-01-10 05:42:04,962] jepsen worker 8 - jepsen.util 2438 :info :write [2 27] :RequestTimeoutException INFO [2022-01-10 05:42:04,963] jepsen worker 8 - jepsen.util 2528 :invoke :write [2 31] INFO [2022-01-10 05:42:04,997] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:04,997] jepsen worker nemesis - couchbase.util Rebalance check count: 278 INFO [2022-01-10 05:42:04,998] jepsen worker nemesis - couchbase.util Rebalance stuck count: 216 INFO [2022-01-10 05:42:04,999] jepsen worker 1 - jepsen.util 2341 :ok :read [0 46] INFO [2022-01-10 05:42:04,999] jepsen worker 1 - jepsen.util 2341 :invoke :read [0 nil] INFO [2022-01-10 05:42:05,008] jepsen worker 60 - jepsen.util 2490 :ok :read [20 :nil] INFO [2022-01-10 05:42:05,009] jepsen worker 60 - jepsen.util 2490 :invoke :write [20 31] INFO [2022-01-10 05:42:05,174] jepsen worker 2 - jepsen.util 2072 :info :write [0 26] :RequestTimeoutException INFO [2022-01-10 05:42:05,175] jepsen worker 2 - jepsen.util 2162 :invoke :write [0 21] INFO [2022-01-10 05:42:05,690] jepsen worker 12 - jepsen.util 2082 :info :write [4 8] :RequestTimeoutException INFO [2022-01-10 05:42:05,691] jepsen worker 12 - jepsen.util 2172 :invoke :read [4 nil] INFO [2022-01-10 05:42:05,840] jepsen worker 83 - jepsen.util 2513 :ok :read [27 30] INFO [2022-01-10 05:42:05,840] jepsen worker 83 - jepsen.util 2513 :invoke :write [27 33] INFO [2022-01-10 05:42:05,852] jepsen worker 15 - jepsen.util 1905 :ok :read [5 42] INFO [2022-01-10 05:42:05,853] jepsen worker 15 - jepsen.util 1905 :invoke :write [5 32] INFO [2022-01-10 05:42:06,001] jepsen worker 80 - jepsen.util 2420 :info :write [26 33] :RequestTimeoutException INFO [2022-01-10 05:42:06,002] jepsen worker 80 - jepsen.util 2510 :invoke :read [26 nil] INFO [2022-01-10 05:42:06,003] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:06,003] jepsen worker nemesis - couchbase.util Rebalance check count: 279 INFO [2022-01-10 05:42:06,003] jepsen worker nemesis - couchbase.util Rebalance stuck count: 217 INFO [2022-01-10 05:42:06,097] jepsen worker 22 - jepsen.util 1912 :info :write [7 5] :RequestTimeoutException INFO [2022-01-10 05:42:06,097] jepsen worker 22 - jepsen.util 2002 :invoke :read [7 nil] INFO [2022-01-10 05:42:06,170] jepsen worker 80 - jepsen.util 2510 :ok :read [26 10] INFO [2022-01-10 05:42:06,171] jepsen worker 80 - jepsen.util 2510 :invoke :read [26 nil] INFO [2022-01-10 05:42:06,196] jepsen worker 80 - jepsen.util 2510 :ok :read [26 10] INFO [2022-01-10 05:42:06,196] jepsen worker 80 - jepsen.util 2510 :invoke :read [26 nil] INFO [2022-01-10 05:42:06,248] jepsen worker 85 - jepsen.util 2605 :ok :read [28 13] INFO [2022-01-10 05:42:06,249] jepsen worker 85 - jepsen.util 2605 :invoke :read [28 nil] INFO [2022-01-10 05:42:06,294] jepsen worker 12 - jepsen.util 2172 :ok :read [4 3] INFO [2022-01-10 05:42:06,295] jepsen worker 12 - jepsen.util 2172 :invoke :write [4 18] INFO [2022-01-10 05:42:06,311] jepsen worker 14 - jepsen.util 2444 :info :write [4 35] :RequestTimeoutException INFO [2022-01-10 05:42:06,312] jepsen worker 14 - jepsen.util 2534 :invoke :read [4 nil] INFO [2022-01-10 05:42:06,377] jepsen worker 80 - jepsen.util 2510 :ok :read [26 10] INFO [2022-01-10 05:42:06,378] jepsen worker 80 - jepsen.util 2510 :invoke :read [26 nil] INFO [2022-01-10 05:42:06,484] jepsen worker 4 - jepsen.util 2344 :info :write [1 49] :RequestTimeoutException INFO [2022-01-10 05:42:06,485] jepsen worker 4 - jepsen.util 2434 :invoke :write [1 7] INFO [2022-01-10 05:42:06,491] jepsen worker 14 - jepsen.util 2534 :ok :read [4 3] INFO [2022-01-10 05:42:06,492] jepsen worker 14 - jepsen.util 2534 :invoke :read [4 nil] INFO [2022-01-10 05:42:06,496] jepsen worker 80 - jepsen.util 2510 :ok :read [26 10] INFO [2022-01-10 05:42:06,496] jepsen worker 80 - jepsen.util 2510 :invoke :write [26 3] INFO [2022-01-10 05:42:06,586] jepsen worker 14 - jepsen.util 2534 :ok :read [4 3] INFO [2022-01-10 05:42:06,587] jepsen worker 14 - jepsen.util 2534 :invoke :read [4 nil] INFO [2022-01-10 05:42:06,796] jepsen worker 18 - jepsen.util 1728 :info :write [6 32] :RequestTimeoutException INFO [2022-01-10 05:42:06,797] jepsen worker 18 - jepsen.util 1818 :invoke :write [6 21] INFO [2022-01-10 05:42:06,811] jepsen worker 14 - jepsen.util 2534 :ok :read [4 3] INFO [2022-01-10 05:42:06,812] jepsen worker 14 - jepsen.util 2534 :invoke :read [4 nil] INFO [2022-01-10 05:42:06,919] jepsen worker 14 - jepsen.util 2534 :ok :read [4 3] INFO [2022-01-10 05:42:06,920] jepsen worker 14 - jepsen.util 2534 :invoke :write [4 0] INFO [2022-01-10 05:42:07,009] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:07,009] jepsen worker nemesis - couchbase.util Rebalance check count: 280 INFO [2022-01-10 05:42:07,009] jepsen worker nemesis - couchbase.util Rebalance stuck count: 218 INFO [2022-01-10 05:42:07,035] jepsen worker 29 - jepsen.util 2279 :fail :read [9 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:07,035] jepsen worker 40 - jepsen.util 1660 :fail :read [13 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:07,036] jepsen worker 40 - jepsen.util 1660 :invoke :write [13 5] INFO [2022-01-10 05:42:07,037] jepsen worker 29 - jepsen.util 2279 :invoke :read [9 nil] INFO [2022-01-10 05:42:07,075] jepsen worker 59 - jepsen.util 2309 :ok :read [19 31] INFO [2022-01-10 05:42:07,076] jepsen worker 59 - jepsen.util 2309 :invoke :read [19 nil] INFO [2022-01-10 05:42:07,086] jepsen worker 34 - jepsen.util 2104 :ok :read [11 21] INFO [2022-01-10 05:42:07,086] jepsen worker 34 - jepsen.util 2104 :invoke :read [11 nil] INFO [2022-01-10 05:42:07,086] jepsen worker 77 - jepsen.util 2507 :info :write [25 26] :RequestTimeoutException INFO [2022-01-10 05:42:07,087] jepsen worker 77 - jepsen.util 2597 :invoke :read [25 nil] INFO [2022-01-10 05:42:07,469] jepsen worker 68 - jepsen.util 2318 :ok :read [22 41] INFO [2022-01-10 05:42:07,470] jepsen worker 68 - jepsen.util 2318 :invoke :write [22 1] INFO [2022-01-10 05:42:07,470] jepsen worker 17 - jepsen.util 2267 :ok :read [5 42] INFO [2022-01-10 05:42:07,471] jepsen worker 17 - jepsen.util 2267 :invoke :write [5 18] INFO [2022-01-10 05:42:07,671] jepsen worker 61 - jepsen.util 2221 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:07,672] jepsen worker 61 - jepsen.util 2221 :invoke :read [20 nil] INFO [2022-01-10 05:42:07,804] jepsen worker 13 - jepsen.util 2173 :info :write [4 48] :RequestTimeoutException INFO [2022-01-10 05:42:07,805] jepsen worker 13 - jepsen.util 2263 :invoke :write [4 23] INFO [2022-01-10 05:42:07,941] jepsen worker 53 - jepsen.util 2483 :info :write [17 47] :RequestTimeoutException INFO [2022-01-10 05:42:07,941] jepsen worker 53 - jepsen.util 2573 :invoke :write [17 22] INFO [2022-01-10 05:42:07,956] jepsen worker 81 - jepsen.util 2331 :info :write [27 22] :RequestTimeoutException INFO [2022-01-10 05:42:07,957] jepsen worker 81 - jepsen.util 2421 :invoke :write [27 36] INFO [2022-01-10 05:42:08,014] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:08,014] jepsen worker nemesis - couchbase.util Rebalance check count: 281 INFO [2022-01-10 05:42:08,014] jepsen worker nemesis - couchbase.util Rebalance stuck count: 219 INFO [2022-01-10 05:42:08,079] jepsen worker 47 - jepsen.util 2207 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:08,080] jepsen worker 47 - jepsen.util 2207 :invoke :read [15 nil] INFO [2022-01-10 05:42:08,259] jepsen worker 20 - jepsen.util 1910 :info :write [6 7] :RequestTimeoutException INFO [2022-01-10 05:42:08,260] jepsen worker 20 - jepsen.util 2000 :invoke :read [6 nil] INFO [2022-01-10 05:42:08,302] jepsen worker 63 - jepsen.util 2043 :info :write [21 15] :RequestTimeoutException INFO [2022-01-10 05:42:08,302] jepsen worker 63 - jepsen.util 2133 :invoke :read [21 nil] INFO [2022-01-10 05:42:08,307] jepsen worker 74 - jepsen.util 2684 :info :write [24 31] :RequestTimeoutException INFO [2022-01-10 05:42:08,307] jepsen worker 74 - jepsen.util 2774 :invoke :read [24 nil] INFO [2022-01-10 05:42:08,311] jepsen worker 23 - jepsen.util 2273 :info :write [7 31] :RequestTimeoutException INFO [2022-01-10 05:42:08,312] jepsen worker 23 - jepsen.util 2363 :invoke :read [7 nil] INFO [2022-01-10 05:42:08,484] jepsen worker 66 - jepsen.util 1956 :ok :read [22 41] INFO [2022-01-10 05:42:08,485] jepsen worker 66 - jepsen.util 1956 :invoke :read [22 nil] INFO [2022-01-10 05:42:08,497] jepsen worker 1 - jepsen.util 2341 :ok :read [0 46] INFO [2022-01-10 05:42:08,498] jepsen worker 1 - jepsen.util 2341 :invoke :read [0 nil] INFO [2022-01-10 05:42:09,020] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:09,020] jepsen worker nemesis - couchbase.util Rebalance check count: 282 INFO [2022-01-10 05:42:09,020] jepsen worker nemesis - couchbase.util Rebalance stuck count: 220 INFO [2022-01-10 05:42:09,066] jepsen worker 86 - jepsen.util 1706 :info :write [28 6] :RequestTimeoutException INFO [2022-01-10 05:42:09,066] jepsen worker 86 - jepsen.util 1796 :invoke :read [28 nil] INFO [2022-01-10 05:42:09,140] jepsen worker 35 - jepsen.util 2015 :ok :read [11 21] INFO [2022-01-10 05:42:09,141] jepsen worker 35 - jepsen.util 2015 :invoke :read [11 nil] INFO [2022-01-10 05:42:09,155] jepsen worker 79 - jepsen.util 2419 :info :write [26 22] :RequestTimeoutException INFO [2022-01-10 05:42:09,155] jepsen worker 79 - jepsen.util 2509 :invoke :write [26 17] INFO [2022-01-10 05:42:09,511] jepsen worker 21 - jepsen.util 2181 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:09,511] jepsen worker 25 - jepsen.util 2005 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:09,511] jepsen worker 21 - jepsen.util 2181 :invoke :read [7 nil] INFO [2022-01-10 05:42:09,512] jepsen worker 25 - jepsen.util 2005 :invoke :read [8 nil] INFO [2022-01-10 05:42:09,915] jepsen worker 23 - jepsen.util 2363 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:09,915] jepsen worker 76 - jepsen.util 1606 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:09,915] jepsen worker 25 - jepsen.util 2005 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:09,915] jepsen worker 7 - jepsen.util 2347 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:09,916] jepsen worker 21 - jepsen.util 2181 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:09,916] jepsen worker 76 - jepsen.util 1606 :invoke :read [25 nil] INFO [2022-01-10 05:42:09,917] jepsen worker 21 - jepsen.util 2181 :invoke :write [7 45] INFO [2022-01-10 05:42:09,917] jepsen worker 7 - jepsen.util 2347 :invoke :read [2 nil] INFO [2022-01-10 05:42:09,918] jepsen worker 25 - jepsen.util 2005 :invoke :read [8 nil] INFO [2022-01-10 05:42:09,918] jepsen worker 23 - jepsen.util 2363 :invoke :read [7 nil] INFO [2022-01-10 05:42:09,919] jepsen worker 25 - jepsen.util 2005 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:09,919] jepsen worker 23 - jepsen.util 2363 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:09,920] jepsen worker 25 - jepsen.util 2005 :invoke :write [8 23] INFO [2022-01-10 05:42:09,920] jepsen worker 23 - jepsen.util 2363 :invoke :read [7 nil] INFO [2022-01-10 05:42:09,921] jepsen worker 23 - jepsen.util 2363 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:09,921] jepsen worker 23 - jepsen.util 2363 :invoke :write [7 38] INFO [2022-01-10 05:42:09,970] jepsen worker 31 - jepsen.util 2461 :info :write [10 28] :SyncWriteAmbiguous INFO [2022-01-10 05:42:09,971] jepsen worker 31 - jepsen.util 2551 :invoke :write [10 3] INFO [2022-01-10 05:42:09,990] jepsen worker 39 - jepsen.util 2469 :info :write [13 13] :RequestTimeoutException INFO [2022-01-10 05:42:09,990] jepsen worker 39 - jepsen.util 2559 :invoke :read [13 nil] INFO [2022-01-10 05:42:09,992] jepsen worker 50 - jepsen.util 2300 :info :write [16 21] :RequestTimeoutException INFO [2022-01-10 05:42:09,992] jepsen worker 50 - jepsen.util 2390 :invoke :read [16 nil] WARN [2022-01-10 05:42:10,010] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10864688,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14527595,"total_server_duration_us":790104,"encode_duration_us":16447,"operation_id":"0x445c","timeout_ms":10000,"last_server_duration_us":383978,"total_duration_us":10010340},{"operation_name":"upsert","last_dispatch_duration_us":3903584,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":3903584,"total_server_duration_us":592209,"encode_duration_us":18661,"operation_id":"0x43e6","timeout_ms":10000,"last_server_duration_us":592209,"total_duration_us":10009745},{"operation_name":"upsert","last_dispatch_duration_us":8367429,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8367429,"total_server_duration_us":199169,"encode_duration_us":16838,"operation_id":"0x44e6","timeout_ms":10000,"last_server_duration_us":199169,"total_duration_us":10008525},{"operation_name":"upsert","last_dispatch_duration_us":879231,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":8771892,"total_server_duration_us":3262018,"encode_duration_us":16702,"operation_id":"0x4594","timeout_ms":10000,"last_server_duration_us":402162,"total_duration_us":10006514},{"operation_name":"upsert","last_dispatch_duration_us":13290108,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":13290108,"total_server_duration_us":1410109,"encode_duration_us":17528,"operation_id":"0x4487","timeout_ms":10000,"last_server_duration_us":1410109,"total_duration_us":10006308},{"operation_name":"upsert","last_dispatch_duration_us":936589,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":8882829,"total_server_duration_us":3924447,"encode_duration_us":16678,"operation_id":"0x4558","timeout_ms":10000,"last_server_duration_us":124227,"total_duration_us":10004448},{"operation_name":"upsert","last_dispatch_duration_us":9336882,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10194497,"total_server_duration_us":1268071,"encode_duration_us":23397,"operation_id":"0x4450","timeout_ms":10000,"last_server_duration_us":1019216,"total_duration_us":10003696},{"operation_name":"upsert","last_dispatch_duration_us":104631,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":762418,"total_server_duration_us":181923,"encode_duration_us":16958,"operation_id":"0x445b","timeout_ms":10000,"last_server_duration_us":41660,"total_duration_us":10003224},{"operation_name":"upsert","last_dispatch_duration_us":4521257,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":7771983,"total_server_duration_us":974836,"encode_duration_us":18494,"operation_id":"0x4412","timeout_ms":10000,"last_server_duration_us":364526,"total_duration_us":10001786},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":16390,"operation_id":"0x4516","timeout_ms":10000,"total_duration_us":10000988}],"total_count":12}} INFO [2022-01-10 05:42:10,021] jepsen worker 48 - jepsen.util 1848 :info :write [16 13] :RequestTimeoutException INFO [2022-01-10 05:42:10,022] jepsen worker 48 - jepsen.util 1938 :invoke :write [16 17] INFO [2022-01-10 05:42:10,026] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:10,026] jepsen worker nemesis - couchbase.util Rebalance check count: 283 INFO [2022-01-10 05:42:10,026] jepsen worker nemesis - couchbase.util Rebalance stuck count: 221 INFO [2022-01-10 05:42:10,036] jepsen worker 28 - jepsen.util 2098 :info :write [9 15] :RequestTimeoutException INFO [2022-01-10 05:42:10,037] jepsen worker 28 - jepsen.util 2188 :invoke :read [9 nil] INFO [2022-01-10 05:42:10,584] jepsen worker 29 - jepsen.util 2279 :ok :read [9 22] INFO [2022-01-10 05:42:10,585] jepsen worker 29 - jepsen.util 2279 :invoke :write [9 33] INFO [2022-01-10 05:42:10,752] jepsen worker 75 - jepsen.util 1425 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:10,753] jepsen worker 75 - jepsen.util 1425 :invoke :read [25 nil] INFO [2022-01-10 05:42:10,769] jepsen worker 34 - jepsen.util 2104 :ok :read [11 21] INFO [2022-01-10 05:42:10,773] jepsen worker 34 - jepsen.util 2104 :invoke :write [11 25] INFO [2022-01-10 05:42:10,793] jepsen worker 85 - jepsen.util 2605 :ok :read [28 13] INFO [2022-01-10 05:42:10,795] jepsen worker 85 - jepsen.util 2605 :invoke :write [28 45] INFO [2022-01-10 05:42:10,953] jepsen worker 61 - jepsen.util 2221 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:10,953] jepsen worker 61 - jepsen.util 2221 :invoke :write [20 20] INFO [2022-01-10 05:42:10,955] jepsen worker 7 - jepsen.util 2347 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:10,955] jepsen worker 76 - jepsen.util 1606 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:10,955] jepsen worker 75 - jepsen.util 1425 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:10,955] jepsen worker 76 - jepsen.util 1606 :invoke :write [25 39] INFO [2022-01-10 05:42:10,956] jepsen worker 7 - jepsen.util 2347 :invoke :read [2 nil] INFO [2022-01-10 05:42:10,957] jepsen worker 75 - jepsen.util 1425 :invoke :write [25 13] INFO [2022-01-10 05:42:10,966] jepsen worker 43 - jepsen.util 2023 :ok :read [14 14] INFO [2022-01-10 05:42:10,967] jepsen worker 43 - jepsen.util 2023 :invoke :write [14 23] INFO [2022-01-10 05:42:11,031] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:11,031] jepsen worker nemesis - couchbase.util Rebalance check count: 284 INFO [2022-01-10 05:42:11,031] jepsen worker nemesis - couchbase.util Rebalance stuck count: 222 INFO [2022-01-10 05:42:11,216] jepsen worker 78 - jepsen.util 2418 :info :write [26 13] :RequestTimeoutException INFO [2022-01-10 05:42:11,217] jepsen worker 78 - jepsen.util 2508 :invoke :write [26 37] INFO [2022-01-10 05:42:11,272] jepsen worker 10 - jepsen.util 2350 :info :write [3 5] :RequestTimeoutException INFO [2022-01-10 05:42:11,272] jepsen worker 10 - jepsen.util 2440 :invoke :read [3 nil] INFO [2022-01-10 05:42:11,332] jepsen worker 82 - jepsen.util 2062 :info :write [27 23] :RequestTimeoutException INFO [2022-01-10 05:42:11,333] jepsen worker 82 - jepsen.util 2152 :invoke :write [27 48] INFO [2022-01-10 05:42:11,399] jepsen worker 27 - jepsen.util 2097 :ok :read [9 22] INFO [2022-01-10 05:42:11,400] jepsen worker 27 - jepsen.util 2097 :invoke :write [9 38] INFO [2022-01-10 05:42:11,421] jepsen worker 16 - jepsen.util 2446 :ok :read [5 42] INFO [2022-01-10 05:42:11,422] jepsen worker 16 - jepsen.util 2446 :invoke :read [5 nil] INFO [2022-01-10 05:42:11,441] jepsen worker 59 - jepsen.util 2309 :ok :read [19 31] INFO [2022-01-10 05:42:11,442] jepsen worker 59 - jepsen.util 2309 :invoke :write [19 15] INFO [2022-01-10 05:42:11,471] jepsen worker 67 - jepsen.util 1957 :info :write [22 5] :RequestTimeoutException INFO [2022-01-10 05:42:11,472] jepsen worker 67 - jepsen.util 2047 :invoke :write [22 20] WARN [2022-01-10 05:42:11,512] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":3829630,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3829630,"total_server_duration_us":56,"operation_id":"0x4392","timeout_ms":30000,"last_server_duration_us":56,"total_duration_us":21238868},{"operation_name":"get","last_dispatch_duration_us":3819779,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3819779,"total_server_duration_us":23,"operation_id":"0x4394","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":21230013},{"operation_name":"get","last_dispatch_duration_us":12137095,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":12137095,"total_server_duration_us":10392,"operation_id":"0x4493","timeout_ms":30000,"last_server_duration_us":10392,"total_duration_us":12138975},{"operation_name":"get","last_dispatch_duration_us":10293345,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10293345,"total_server_duration_us":32,"operation_id":"0x44e8","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":10295213},{"operation_name":"upsert","last_dispatch_duration_us":350401,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9823514,"total_server_duration_us":586242,"encode_duration_us":17422,"operation_id":"0x443c","timeout_ms":10000,"last_server_duration_us":333415,"total_duration_us":10009058},{"operation_name":"upsert","last_dispatch_duration_us":844224,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":8105218,"total_server_duration_us":3097798,"encode_duration_us":18803,"operation_id":"0x4514","timeout_ms":10000,"last_server_duration_us":84283,"total_duration_us":10008809},{"operation_name":"upsert","last_dispatch_duration_us":254248,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":8348477,"total_server_duration_us":5080795,"encode_duration_us":19341,"operation_id":"0x455f","timeout_ms":10000,"last_server_duration_us":228303,"total_duration_us":10007665},{"operation_name":"upsert","last_dispatch_duration_us":10074680,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":14112564,"total_server_duration_us":799345,"encode_duration_us":17085,"operation_id":"0x445a","timeout_ms":10000,"last_server_duration_us":203801,"total_duration_us":10006930},{"operation_name":"upsert","last_dispatch_duration_us":4498869,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4498869,"total_server_duration_us":393165,"encode_duration_us":17636,"operation_id":"0x456a","timeout_ms":10000,"last_server_duration_us":393165,"total_duration_us":10006329},{"operation_name":"upsert","encode_duration_us":17067,"operation_id":"0x449b","timeout_ms":10000,"total_duration_us":10005572}],"total_count":16}} WARN [2022-01-10 05:42:11,512] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12688293,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12688293,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":11083693,"encode_duration_us":18880,"operation_id":"0x4442","timeout_ms":10000,"last_server_duration_us":11083693,"total_duration_us":10004395}],"total_count":1}} INFO [2022-01-10 05:42:11,535] jepsen worker 56 - jepsen.util 2396 :info :write [18 26] :RequestTimeoutException INFO [2022-01-10 05:42:11,535] jepsen worker 56 - jepsen.util 2486 :invoke :read [18 nil] WARN [2022-01-10 05:42:11,611] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10867908,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11617131,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":11025466,"encode_duration_us":17652,"operation_id":"0x4460","timeout_ms":10000,"last_server_duration_us":10759297,"total_duration_us":10004842}],"total_count":1}} WARN [2022-01-10 05:42:11,612] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18791363,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":18791363,"total_server_duration_us":1224441,"operation_id":"0x4391","timeout_ms":30000,"last_server_duration_us":1224441,"total_duration_us":18800181},{"operation_name":"get","last_dispatch_duration_us":6200379,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":6200379,"total_server_duration_us":614364,"operation_id":"0x445d","timeout_ms":30000,"last_server_duration_us":614364,"total_duration_us":15955653},{"operation_name":"get","last_dispatch_duration_us":3305897,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3305897,"total_server_duration_us":23,"operation_id":"0x44ea","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":15106067},{"operation_name":"get","last_dispatch_duration_us":4711474,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":4711474,"total_server_duration_us":614025,"operation_id":"0x4444","timeout_ms":30000,"last_server_duration_us":614025,"total_duration_us":14980809},{"operation_name":"get","last_dispatch_duration_us":13615433,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":13615433,"total_server_duration_us":76,"operation_id":"0x4486","timeout_ms":30000,"last_server_duration_us":76,"total_duration_us":14698503},{"operation_name":"get","last_dispatch_duration_us":5079658,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5079658,"total_server_duration_us":27,"operation_id":"0x456c","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":11269410},{"operation_name":"get","last_dispatch_duration_us":3464411,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3464411,"total_server_duration_us":15,"operation_id":"0x45be","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":10670875},{"operation_name":"upsert","last_dispatch_duration_us":857793,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":2175101,"total_server_duration_us":465665,"encode_duration_us":16525,"operation_id":"0x44ca","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10008004},{"operation_name":"upsert","last_dispatch_duration_us":5914855,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5914855,"total_server_duration_us":402445,"encode_duration_us":16276,"operation_id":"0x452f","timeout_ms":10000,"last_server_duration_us":402445,"total_duration_us":10007063},{"operation_name":"upsert","last_dispatch_duration_us":4267839,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10608057,"total_server_duration_us":600521,"encode_duration_us":17372,"operation_id":"0x4560","timeout_ms":10000,"last_server_duration_us":222341,"total_duration_us":10005594}],"total_count":18}} WARN [2022-01-10 05:42:11,614] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":19007880,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":19007880,"total_server_duration_us":7437,"operation_id":"0x40f6","timeout_ms":30000,"last_server_duration_us":7437,"total_duration_us":30010148},{"operation_name":"get","last_dispatch_duration_us":18716238,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":18716238,"total_server_duration_us":1811,"operation_id":"0x420d","timeout_ms":30000,"last_server_duration_us":1811,"total_duration_us":24899500},{"operation_name":"get","last_dispatch_duration_us":11015733,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11015733,"total_server_duration_us":43,"operation_id":"0x4537","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":11016078},{"operation_name":"upsert","last_dispatch_duration_us":1631061,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":8675078,"total_server_duration_us":3961977,"encode_duration_us":16483,"operation_id":"0x4476","timeout_ms":10000,"last_server_duration_us":1409627,"total_duration_us":10009038},{"operation_name":"upsert","last_dispatch_duration_us":3276095,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8435660,"total_server_duration_us":1339317,"encode_duration_us":17653,"operation_id":"0x45b4","timeout_ms":10000,"last_server_duration_us":388143,"total_duration_us":10008757},{"operation_name":"upsert","last_dispatch_duration_us":44035,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":2744764,"total_server_duration_us":2025921,"encode_duration_us":16844,"operation_id":"0x4485","timeout_ms":10000,"last_server_duration_us":27222,"total_duration_us":10008517},{"operation_name":"upsert","last_dispatch_duration_us":13121355,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13121355,"total_server_duration_us":385642,"encode_duration_us":17559,"operation_id":"0x44de","timeout_ms":10000,"last_server_duration_us":385642,"total_duration_us":10008259},{"operation_name":"upsert","last_dispatch_duration_us":10801155,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10801155,"total_server_duration_us":387030,"encode_duration_us":16486,"operation_id":"0x4568","timeout_ms":10000,"last_server_duration_us":387030,"total_duration_us":10008012},{"operation_name":"upsert","last_dispatch_duration_us":758189,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":7845199,"total_server_duration_us":4086558,"encode_duration_us":17702,"operation_id":"0x450c","timeout_ms":10000,"last_server_duration_us":746737,"total_duration_us":10005523},{"operation_name":"upsert","last_dispatch_duration_us":4326738,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10013988,"total_server_duration_us":1790978,"encode_duration_us":16764,"operation_id":"0x45eb","timeout_ms":10000,"last_server_duration_us":202956,"total_duration_us":10005157}],"total_count":19}} WARN [2022-01-10 05:42:11,622] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":20122794,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":20122794,"total_server_duration_us":1029306,"operation_id":"0x4277","timeout_ms":30000,"last_server_duration_us":1029306,"total_duration_us":20124786},{"operation_name":"upsert","encode_duration_us":16413,"operation_id":"0x4569","timeout_ms":10000,"total_duration_us":10010101},{"operation_name":"upsert","last_dispatch_duration_us":4295596,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9995014,"total_server_duration_us":1195451,"encode_duration_us":16083,"operation_id":"0x44b6","timeout_ms":10000,"last_server_duration_us":602908,"total_duration_us":10008984},{"operation_name":"upsert","encode_duration_us":16830,"operation_id":"0x450d","timeout_ms":10000,"total_duration_us":10008705},{"operation_name":"upsert","last_dispatch_duration_us":2438371,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9678832,"total_server_duration_us":2127378,"encode_duration_us":16359,"operation_id":"0x45e8","timeout_ms":10000,"last_server_duration_us":152179,"total_duration_us":10008335},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":16274,"operation_id":"0x4538","timeout_ms":10000,"total_duration_us":10006624},{"operation_name":"upsert","last_dispatch_duration_us":1037175,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":2244109,"total_server_duration_us":214247,"encode_duration_us":17054,"operation_id":"0x4494","timeout_ms":10000,"last_server_duration_us":189836,"total_duration_us":10005583},{"operation_name":"upsert","last_dispatch_duration_us":1177842,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10611729,"total_server_duration_us":1194190,"encode_duration_us":17544,"operation_id":"0x4435","timeout_ms":10000,"last_server_duration_us":989613,"total_duration_us":10004161},{"operation_name":"upsert","last_dispatch_duration_us":1230969,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10689668,"total_server_duration_us":249620,"encode_duration_us":22419,"operation_id":"0x443e","timeout_ms":10000,"last_server_duration_us":39004,"total_duration_us":10002615},{"operation_name":"upsert","last_dispatch_duration_us":3280218,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8007432,"total_server_duration_us":1198246,"encode_duration_us":16399,"operation_id":"0x45b6","timeout_ms":10000,"last_server_duration_us":991686,"total_duration_us":10002222}],"total_count":19}} INFO [2022-01-10 05:42:11,628] jepsen worker 5 - jepsen.util 2345 :ok :read [1 41] INFO [2022-01-10 05:42:11,629] jepsen worker 5 - jepsen.util 2345 :invoke :read [1 nil] INFO [2022-01-10 05:42:11,679] jepsen worker 38 - jepsen.util 2288 :info :write [12 37] :SyncWriteAmbiguous INFO [2022-01-10 05:42:11,681] jepsen worker 38 - jepsen.util 2378 :invoke :read [12 nil] INFO [2022-01-10 05:42:11,713] jepsen worker 70 - jepsen.util 2500 :ok :read [23 26] WARN [2022-01-10 05:42:11,714] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9056174,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11235023,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9587677,"encode_duration_us":16401,"operation_id":"0x44a8","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10004572},{"operation_name":"get","last_dispatch_duration_us":19007880,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19007880,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":7437,"operation_id":"0x40f6","timeout_ms":30000,"last_server_duration_us":7437,"total_duration_us":30010148}],"total_count":2}} INFO [2022-01-10 05:42:11,716] jepsen worker 70 - jepsen.util 2500 :invoke :read [23 nil] INFO [2022-01-10 05:42:11,719] jepsen worker 69 - jepsen.util 2319 :ok :read [23 26] INFO [2022-01-10 05:42:11,719] jepsen worker 69 - jepsen.util 2319 :invoke :write [23 18] INFO [2022-01-10 05:42:11,729] jepsen worker 71 - jepsen.util 1781 :info :write [23 20] :RequestTimeoutException INFO [2022-01-10 05:42:11,730] jepsen worker 71 - jepsen.util 1871 :invoke :read [23 nil] INFO [2022-01-10 05:42:11,731] jepsen worker 54 - jepsen.util 2214 :ok :read [18 14] INFO [2022-01-10 05:42:11,731] jepsen worker 54 - jepsen.util 2214 :invoke :write [18 46] INFO [2022-01-10 05:42:11,772] jepsen worker 5 - jepsen.util 2345 :ok :read [1 41] INFO [2022-01-10 05:42:11,772] jepsen worker 5 - jepsen.util 2345 :invoke :write [1 36] INFO [2022-01-10 05:42:11,793] jepsen worker 36 - jepsen.util 1926 :ok :read [12 42] INFO [2022-01-10 05:42:11,793] jepsen worker 36 - jepsen.util 1926 :invoke :write [12 35] INFO [2022-01-10 05:42:11,800] jepsen worker 38 - jepsen.util 2378 :ok :read [12 42] INFO [2022-01-10 05:42:11,801] jepsen worker 38 - jepsen.util 2378 :invoke :write [12 39] WARN [2022-01-10 05:42:11,812] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":5339159,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10233011,"total_server_duration_us":1407892,"encode_duration_us":16583,"operation_id":"0x4453","timeout_ms":10000,"last_server_duration_us":403576,"total_duration_us":10008366},{"operation_name":"upsert","last_dispatch_duration_us":4135852,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11184869,"total_server_duration_us":988379,"encode_duration_us":16392,"operation_id":"0x45b3","timeout_ms":10000,"last_server_duration_us":375706,"total_duration_us":10007655},{"operation_name":"upsert","last_dispatch_duration_us":5744990,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11035020,"total_server_duration_us":1992747,"encode_duration_us":21923,"operation_id":"0x4454","timeout_ms":10000,"last_server_duration_us":804629,"total_duration_us":10007464},{"operation_name":"upsert","last_dispatch_duration_us":6345577,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11212269,"total_server_duration_us":607459,"encode_duration_us":26505,"operation_id":"0x4448","timeout_ms":10000,"last_server_duration_us":201901,"total_duration_us":10006640},{"operation_name":"upsert","last_dispatch_duration_us":4725542,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7837740,"total_server_duration_us":1012386,"encode_duration_us":17417,"operation_id":"0x4488","timeout_ms":10000,"last_server_duration_us":386475,"total_duration_us":10004989},{"operation_name":"upsert","last_dispatch_duration_us":10867908,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":11617131,"total_server_duration_us":11025466,"encode_duration_us":17652,"operation_id":"0x4460","timeout_ms":10000,"last_server_duration_us":10759297,"total_duration_us":10004842},{"operation_name":"upsert","last_dispatch_duration_us":6343462,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":12290573,"total_server_duration_us":608586,"encode_duration_us":17646,"operation_id":"0x4456","timeout_ms":10000,"last_server_duration_us":202744,"total_duration_us":10002844},{"operation_name":"upsert","last_dispatch_duration_us":824937,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":8218340,"total_server_duration_us":4602650,"encode_duration_us":16326,"operation_id":"0x44eb","timeout_ms":10000,"last_server_duration_us":81398,"total_duration_us":10002014},{"operation_name":"upsert","last_dispatch_duration_us":5562444,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13296627,"total_server_duration_us":429713,"encode_duration_us":24433,"operation_id":"0x4445","timeout_ms":10000,"last_server_duration_us":32840,"total_duration_us":10001617},{"operation_name":"upsert","last_dispatch_duration_us":8946292,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":11494069,"total_server_duration_us":655784,"encode_duration_us":19057,"operation_id":"0x445f","timeout_ms":10000,"last_server_duration_us":23559,"total_duration_us":10001591}],"total_count":19}} INFO [2022-01-10 05:42:11,814] jepsen worker 70 - jepsen.util 2500 :ok :read [23 26] INFO [2022-01-10 05:42:11,814] jepsen worker 70 - jepsen.util 2500 :invoke :read [23 nil] INFO [2022-01-10 05:42:11,882] jepsen worker 71 - jepsen.util 1871 :ok :read [23 26] INFO [2022-01-10 05:42:11,883] jepsen worker 71 - jepsen.util 1871 :invoke :read [23 nil] INFO [2022-01-10 05:42:11,966] jepsen worker 70 - jepsen.util 2500 :ok :read [23 26] INFO [2022-01-10 05:42:11,967] jepsen worker 70 - jepsen.util 2500 :invoke :write [23 17] INFO [2022-01-10 05:42:12,037] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:12,037] jepsen worker nemesis - couchbase.util Rebalance check count: 285 INFO [2022-01-10 05:42:12,037] jepsen worker nemesis - couchbase.util Rebalance stuck count: 223 INFO [2022-01-10 05:42:12,147] jepsen worker 49 - jepsen.util 2029 :info :write [16 26] :RequestTimeoutException INFO [2022-01-10 05:42:12,147] jepsen worker 49 - jepsen.util 2119 :invoke :write [16 47] INFO [2022-01-10 05:42:12,234] jepsen worker 58 - jepsen.util 2218 :ok :read [19 31] INFO [2022-01-10 05:42:12,234] jepsen worker 58 - jepsen.util 2218 :invoke :read [19 nil] INFO [2022-01-10 05:42:12,276] jepsen worker 6 - jepsen.util 2076 :info :write [2 24] :RequestTimeoutException INFO [2022-01-10 05:42:12,276] jepsen worker 6 - jepsen.util 2166 :invoke :write [2 42] INFO [2022-01-10 05:42:12,280] jepsen worker 41 - jepsen.util 2021 :info :write [13 27] :RequestTimeoutException INFO [2022-01-10 05:42:12,281] jepsen worker 41 - jepsen.util 2111 :invoke :write [13 40] INFO [2022-01-10 05:42:12,531] jepsen worker 57 - jepsen.util 2487 :info :write [19 37] :RequestTimeoutException INFO [2022-01-10 05:42:12,532] jepsen worker 57 - jepsen.util 2577 :invoke :write [19 42] INFO [2022-01-10 05:42:12,621] jepsen worker 9 - jepsen.util 2529 :info :write [3 31] :RequestTimeoutException INFO [2022-01-10 05:42:12,621] jepsen worker 9 - jepsen.util 2619 :invoke :write [3 18] INFO [2022-01-10 05:42:12,643] jepsen worker 84 - jepsen.util 2244 :ok :read [28 13] INFO [2022-01-10 05:42:12,644] jepsen worker 84 - jepsen.util 2244 :invoke :read [28 nil] INFO [2022-01-10 05:42:12,737] jepsen worker 73 - jepsen.util 2053 :ok :read [24 18] INFO [2022-01-10 05:42:12,738] jepsen worker 73 - jepsen.util 2053 :invoke :read [24 nil] INFO [2022-01-10 05:42:12,810] jepsen worker 51 - jepsen.util 2391 :info :write [17 48] :RequestTimeoutException INFO [2022-01-10 05:42:12,811] jepsen worker 51 - jepsen.util 2481 :invoke :write [17 25] INFO [2022-01-10 05:42:12,901] jepsen worker 89 - jepsen.util 2519 :info :write [29 48] :RequestTimeoutException INFO [2022-01-10 05:42:12,902] jepsen worker 89 - jepsen.util 2609 :invoke :read [29 nil] INFO [2022-01-10 05:42:12,919] jepsen worker 7 - jepsen.util 2347 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:12,920] jepsen worker 7 - jepsen.util 2347 :invoke :write [2 43] INFO [2022-01-10 05:42:13,016] jepsen worker 45 - jepsen.util 2475 :info :write [15 9] :RequestTimeoutException INFO [2022-01-10 05:42:13,017] jepsen worker 45 - jepsen.util 2565 :invoke :write [15 32] INFO [2022-01-10 05:42:13,031] jepsen worker 33 - jepsen.util 2373 :info :write [11 23] :RequestTimeoutException INFO [2022-01-10 05:42:13,031] jepsen worker 33 - jepsen.util 2463 :invoke :read [11 nil] INFO [2022-01-10 05:42:13,036] jepsen worker 11 - jepsen.util 2081 :info :write [3 10] :RequestTimeoutException INFO [2022-01-10 05:42:13,036] jepsen worker 11 - jepsen.util 2171 :invoke :write [3 37] INFO [2022-01-10 05:42:13,043] jepsen worker 55 - jepsen.util 2395 :info :write [18 32] :RequestTimeoutException INFO [2022-01-10 05:42:13,043] jepsen worker 55 - jepsen.util 2485 :invoke :write [18 31] INFO [2022-01-10 05:42:13,045] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:13,045] jepsen worker nemesis - couchbase.util Rebalance check count: 286 INFO [2022-01-10 05:42:13,045] jepsen worker nemesis - couchbase.util Rebalance stuck count: 224 INFO [2022-01-10 05:42:13,077] jepsen worker 37 - jepsen.util 2197 :info :write [12 8] :RequestTimeoutException INFO [2022-01-10 05:42:13,077] jepsen worker 37 - jepsen.util 2287 :invoke :read [12 nil] INFO [2022-01-10 05:42:13,255] jepsen worker 66 - jepsen.util 1956 :ok :read [22 41] INFO [2022-01-10 05:42:13,255] jepsen worker 66 - jepsen.util 1956 :invoke :read [22 nil] INFO [2022-01-10 05:42:13,290] jepsen worker 72 - jepsen.util 2232 :info :write [24 8] :RequestTimeoutException INFO [2022-01-10 05:42:13,291] jepsen worker 72 - jepsen.util 2322 :invoke :write [24 25] INFO [2022-01-10 05:42:13,395] jepsen worker 52 - jepsen.util 2662 :info :write [17 14] :RequestTimeoutException INFO [2022-01-10 05:42:13,395] jepsen worker 52 - jepsen.util 2752 :invoke :read [17 nil] INFO [2022-01-10 05:42:13,610] jepsen worker 52 - jepsen.util 2752 :ok :read [17 41] INFO [2022-01-10 05:42:13,611] jepsen worker 52 - jepsen.util 2752 :invoke :read [17 nil] INFO [2022-01-10 05:42:13,693] jepsen worker 0 - jepsen.util 2160 :ok :read [0 46] INFO [2022-01-10 05:42:13,694] jepsen worker 0 - jepsen.util 2160 :invoke :write [0 30] INFO [2022-01-10 05:42:13,713] jepsen worker 86 - jepsen.util 1796 :ok :read [28 13] INFO [2022-01-10 05:42:13,713] jepsen worker 86 - jepsen.util 1796 :invoke :read [28 nil] INFO [2022-01-10 05:42:13,737] jepsen worker 39 - jepsen.util 2559 :ok :read [13 15] INFO [2022-01-10 05:42:13,738] jepsen worker 39 - jepsen.util 2559 :invoke :write [13 44] INFO [2022-01-10 05:42:13,812] jepsen worker 52 - jepsen.util 2752 :ok :read [17 41] INFO [2022-01-10 05:42:13,812] jepsen worker 52 - jepsen.util 2752 :invoke :write [17 10] INFO [2022-01-10 05:42:13,879] jepsen worker 28 - jepsen.util 2188 :ok :read [9 22] INFO [2022-01-10 05:42:13,879] jepsen worker 28 - jepsen.util 2188 :invoke :read [9 nil] INFO [2022-01-10 05:42:14,051] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:14,051] jepsen worker nemesis - couchbase.util Rebalance check count: 287 INFO [2022-01-10 05:42:14,051] jepsen worker nemesis - couchbase.util Rebalance stuck count: 225 INFO [2022-01-10 05:42:14,116] jepsen worker 71 - jepsen.util 1871 :ok :read [23 26] INFO [2022-01-10 05:42:14,119] jepsen worker 71 - jepsen.util 1871 :invoke :read [23 nil] INFO [2022-01-10 05:42:14,194] jepsen worker 37 - jepsen.util 2287 :ok :read [12 42] INFO [2022-01-10 05:42:14,195] jepsen worker 37 - jepsen.util 2287 :invoke :read [12 nil] INFO [2022-01-10 05:42:14,225] jepsen worker 3 - jepsen.util 2343 :ok :read [1 41] INFO [2022-01-10 05:42:14,225] jepsen worker 3 - jepsen.util 2343 :invoke :read [1 nil] INFO [2022-01-10 05:42:14,340] jepsen worker 16 - jepsen.util 2446 :ok :read [5 42] INFO [2022-01-10 05:42:14,340] jepsen worker 16 - jepsen.util 2446 :invoke :read [5 nil] INFO [2022-01-10 05:42:14,342] jepsen worker 42 - jepsen.util 2112 :info :write [14 29] :RequestTimeoutException INFO [2022-01-10 05:42:14,342] jepsen worker 42 - jepsen.util 2202 :invoke :read [14 nil] INFO [2022-01-10 05:42:14,441] jepsen worker 87 - jepsen.util 2427 :info :write [29 47] :RequestTimeoutException INFO [2022-01-10 05:42:14,442] jepsen worker 87 - jepsen.util 2517 :invoke :read [29 nil] INFO [2022-01-10 05:42:14,471] jepsen worker 71 - jepsen.util 1871 :ok :read [23 26] INFO [2022-01-10 05:42:14,472] jepsen worker 71 - jepsen.util 1871 :invoke :read [23 nil] INFO [2022-01-10 05:42:14,530] jepsen worker 88 - jepsen.util 2158 :info :write [29 44] :RequestTimeoutException INFO [2022-01-10 05:42:14,531] jepsen worker 88 - jepsen.util 2248 :invoke :read [29 nil] INFO [2022-01-10 05:42:14,648] jepsen worker 37 - jepsen.util 2287 :ok :read [12 42] INFO [2022-01-10 05:42:14,648] jepsen worker 37 - jepsen.util 2287 :invoke :write [12 29] INFO [2022-01-10 05:42:14,757] jepsen worker 30 - jepsen.util 2010 :info :write [10 34] :RequestTimeoutException INFO [2022-01-10 05:42:14,758] jepsen worker 30 - jepsen.util 2100 :invoke :read [10 nil] INFO [2022-01-10 05:42:14,777] jepsen worker 46 - jepsen.util 2566 :ok :read [15 23] INFO [2022-01-10 05:42:14,778] jepsen worker 46 - jepsen.util 2566 :invoke :read [15 nil] INFO [2022-01-10 05:42:14,989] jepsen worker 8 - jepsen.util 2528 :info :write [2 31] :RequestTimeoutException INFO [2022-01-10 05:42:14,990] jepsen worker 8 - jepsen.util 2618 :invoke :read [2 nil] INFO [2022-01-10 05:42:15,039] jepsen worker 71 - jepsen.util 1871 :ok :read [23 26] INFO [2022-01-10 05:42:15,040] jepsen worker 71 - jepsen.util 1871 :invoke :read [23 nil] INFO [2022-01-10 05:42:15,041] jepsen worker 60 - jepsen.util 2490 :info :write [20 31] :RequestTimeoutException INFO [2022-01-10 05:42:15,042] jepsen worker 60 - jepsen.util 2580 :invoke :write [20 34] INFO [2022-01-10 05:42:15,063] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:15,063] jepsen worker nemesis - couchbase.util Rebalance check count: 288 INFO [2022-01-10 05:42:15,063] jepsen worker nemesis - couchbase.util Rebalance stuck count: 226 INFO [2022-01-10 05:42:15,162] jepsen worker 44 - jepsen.util 2204 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:15,163] jepsen worker 44 - jepsen.util 2204 :invoke :read [14 nil] INFO [2022-01-10 05:42:15,186] jepsen worker 10 - jepsen.util 2440 :ok :read [3 2] INFO [2022-01-10 05:42:15,187] jepsen worker 10 - jepsen.util 2440 :invoke :read [3 nil] INFO [2022-01-10 05:42:15,193] jepsen worker 89 - jepsen.util 2609 :ok :read [29 0] INFO [2022-01-10 05:42:15,193] jepsen worker 89 - jepsen.util 2609 :invoke :write [29 10] INFO [2022-01-10 05:42:15,201] jepsen worker 2 - jepsen.util 2162 :info :write [0 21] :RequestTimeoutException INFO [2022-01-10 05:42:15,202] jepsen worker 2 - jepsen.util 2252 :invoke :read [0 nil] INFO [2022-01-10 05:42:15,690] jepsen worker 3 - jepsen.util 2343 :ok :read [1 41] INFO [2022-01-10 05:42:15,690] jepsen worker 3 - jepsen.util 2343 :invoke :read [1 nil] INFO [2022-01-10 05:42:15,692] jepsen worker 66 - jepsen.util 1956 :ok :read [22 41] INFO [2022-01-10 05:42:15,693] jepsen worker 66 - jepsen.util 1956 :invoke :write [22 10] INFO [2022-01-10 05:42:15,861] jepsen worker 83 - jepsen.util 2513 :info :write [27 33] :RequestTimeoutException INFO [2022-01-10 05:42:15,861] jepsen worker 83 - jepsen.util 2603 :invoke :read [27 nil] INFO [2022-01-10 05:42:15,881] jepsen worker 15 - jepsen.util 1905 :info :write [5 32] :RequestTimeoutException INFO [2022-01-10 05:42:15,881] jepsen worker 15 - jepsen.util 1995 :invoke :read [5 nil] INFO [2022-01-10 05:42:16,069] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:16,069] jepsen worker nemesis - couchbase.util Rebalance check count: 289 INFO [2022-01-10 05:42:16,069] jepsen worker nemesis - couchbase.util Rebalance stuck count: 227 INFO [2022-01-10 05:42:16,316] jepsen worker 12 - jepsen.util 2172 :info :write [4 18] :RequestTimeoutException INFO [2022-01-10 05:42:16,317] jepsen worker 12 - jepsen.util 2262 :invoke :write [4 31] INFO [2022-01-10 05:42:16,509] jepsen worker 4 - jepsen.util 2434 :info :write [1 7] :RequestTimeoutException INFO [2022-01-10 05:42:16,510] jepsen worker 4 - jepsen.util 2524 :invoke :write [1 25] INFO [2022-01-10 05:42:16,521] jepsen worker 80 - jepsen.util 2510 :info :write [26 3] :RequestTimeoutException INFO [2022-01-10 05:42:16,521] jepsen worker 80 - jepsen.util 2600 :invoke :read [26 nil] INFO [2022-01-10 05:42:16,746] jepsen worker 10 - jepsen.util 2440 :ok :read [3 2] INFO [2022-01-10 05:42:16,746] jepsen worker 10 - jepsen.util 2440 :invoke :read [3 nil] INFO [2022-01-10 05:42:16,782] jepsen worker 3 - jepsen.util 2343 :ok :read [1 41] INFO [2022-01-10 05:42:16,783] jepsen worker 3 - jepsen.util 2343 :invoke :read [1 nil] INFO [2022-01-10 05:42:16,821] jepsen worker 18 - jepsen.util 1818 :info :write [6 21] :RequestTimeoutException INFO [2022-01-10 05:42:16,822] jepsen worker 18 - jepsen.util 1908 :invoke :write [6 20] INFO [2022-01-10 05:42:16,884] jepsen worker 10 - jepsen.util 2440 :ok :read [3 2] INFO [2022-01-10 05:42:16,884] jepsen worker 10 - jepsen.util 2440 :invoke :read [3 nil] INFO [2022-01-10 05:42:16,946] jepsen worker 14 - jepsen.util 2534 :info :write [4 0] :RequestTimeoutException INFO [2022-01-10 05:42:16,946] jepsen worker 14 - jepsen.util 2624 :invoke :write [4 38] INFO [2022-01-10 05:42:17,026] jepsen worker 47 - jepsen.util 2207 :ok :read [15 23] INFO [2022-01-10 05:42:17,027] jepsen worker 35 - jepsen.util 2015 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:17,030] jepsen worker 73 - jepsen.util 2053 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:17,030] jepsen worker 47 - jepsen.util 2207 :invoke :write [15 42] INFO [2022-01-10 05:42:17,030] jepsen worker 35 - jepsen.util 2015 :invoke :read [11 nil] INFO [2022-01-10 05:42:17,031] jepsen worker 24 - jepsen.util 2004 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:17,031] jepsen worker 74 - jepsen.util 2774 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:17,031] jepsen worker 73 - jepsen.util 2053 :invoke :read [24 nil] INFO [2022-01-10 05:42:17,031] jepsen worker 33 - jepsen.util 2463 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:17,031] jepsen worker 24 - jepsen.util 2004 :invoke :read [8 nil] INFO [2022-01-10 05:42:17,032] jepsen worker 77 - jepsen.util 2597 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:17,032] jepsen worker 42 - jepsen.util 2202 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:17,032] jepsen worker 74 - jepsen.util 2774 :invoke :read [24 nil] INFO [2022-01-10 05:42:17,032] jepsen worker 33 - jepsen.util 2463 :invoke :write [11 25] INFO [2022-01-10 05:42:17,038] jepsen worker 77 - jepsen.util 2597 :invoke :write [25 5] INFO [2022-01-10 05:42:17,041] jepsen worker 63 - jepsen.util 2133 :ok :read [21 7] INFO [2022-01-10 05:42:17,042] jepsen worker 63 - jepsen.util 2133 :invoke :write [21 36] INFO [2022-01-10 05:42:17,044] jepsen worker 42 - jepsen.util 2202 :invoke :read [14 nil] INFO [2022-01-10 05:42:17,063] jepsen worker 3 - jepsen.util 2343 :ok :read [1 41] INFO [2022-01-10 05:42:17,064] jepsen worker 3 - jepsen.util 2343 :invoke :write [1 10] INFO [2022-01-10 05:42:17,070] jepsen worker 40 - jepsen.util 1660 :info :write [13 5] :RequestTimeoutException INFO [2022-01-10 05:42:17,071] jepsen worker 40 - jepsen.util 1750 :invoke :write [13 0] INFO [2022-01-10 05:42:17,075] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:17,075] jepsen worker nemesis - couchbase.util Rebalance check count: 290 INFO [2022-01-10 05:42:17,075] jepsen worker nemesis - couchbase.util Rebalance stuck count: 228 INFO [2022-01-10 05:42:17,125] jepsen worker 10 - jepsen.util 2440 :ok :read [3 2] INFO [2022-01-10 05:42:17,126] jepsen worker 10 - jepsen.util 2440 :invoke :write [3 2] INFO [2022-01-10 05:42:17,189] jepsen worker 26 - jepsen.util 1826 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:17,190] jepsen worker 62 - jepsen.util 2402 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:17,190] jepsen worker 26 - jepsen.util 1826 :invoke :write [8 49] INFO [2022-01-10 05:42:17,191] jepsen worker 62 - jepsen.util 2402 :invoke :write [20 8] INFO [2022-01-10 05:42:17,356] jepsen worker 80 - jepsen.util 2600 :ok :read [26 10] INFO [2022-01-10 05:42:17,357] jepsen worker 80 - jepsen.util 2600 :invoke :write [26 17] INFO [2022-01-10 05:42:17,414] jepsen worker 44 - jepsen.util 2204 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:17,414] jepsen worker 44 - jepsen.util 2204 :invoke :write [14 13] INFO [2022-01-10 05:42:17,425] jepsen worker 46 - jepsen.util 2566 :ok :read [15 23] INFO [2022-01-10 05:42:17,426] jepsen worker 46 - jepsen.util 2566 :invoke :write [15 0] INFO [2022-01-10 05:42:17,491] jepsen worker 68 - jepsen.util 2318 :info :write [22 1] :RequestTimeoutException INFO [2022-01-10 05:42:17,491] jepsen worker 17 - jepsen.util 2267 :info :write [5 18] :RequestTimeoutException INFO [2022-01-10 05:42:17,492] jepsen worker 68 - jepsen.util 2408 :invoke :read [22 nil] INFO [2022-01-10 05:42:17,492] jepsen worker 17 - jepsen.util 2357 :invoke :write [5 22] INFO [2022-01-10 05:42:17,831] jepsen worker 13 - jepsen.util 2263 :info :write [4 23] :RequestTimeoutException INFO [2022-01-10 05:42:17,832] jepsen worker 13 - jepsen.util 2353 :invoke :read [4 nil] INFO [2022-01-10 05:42:17,965] jepsen worker 53 - jepsen.util 2573 :info :write [17 22] :RequestTimeoutException INFO [2022-01-10 05:42:17,966] jepsen worker 53 - jepsen.util 2663 :invoke :read [17 nil] INFO [2022-01-10 05:42:17,976] jepsen worker 81 - jepsen.util 2421 :info :write [27 36] :RequestTimeoutException INFO [2022-01-10 05:42:17,977] jepsen worker 81 - jepsen.util 2511 :invoke :write [27 20] INFO [2022-01-10 05:42:18,081] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:18,081] jepsen worker nemesis - couchbase.util Rebalance check count: 291 INFO [2022-01-10 05:42:18,082] jepsen worker nemesis - couchbase.util Rebalance stuck count: 229 INFO [2022-01-10 05:42:18,187] jepsen worker 83 - jepsen.util 2603 :ok :read [27 30] INFO [2022-01-10 05:42:18,187] jepsen worker 83 - jepsen.util 2603 :invoke :write [27 11] INFO [2022-01-10 05:42:18,221] jepsen worker 19 - jepsen.util 2359 :fail :read [6 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:18,222] jepsen worker 19 - jepsen.util 2359 :invoke :write [6 48] INFO [2022-01-10 05:42:18,236] jepsen worker 73 - jepsen.util 2053 :ok :read [24 18] INFO [2022-01-10 05:42:18,237] jepsen worker 73 - jepsen.util 2053 :invoke :write [24 28] INFO [2022-01-10 05:42:18,251] jepsen worker 22 - jepsen.util 2002 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:18,252] jepsen worker 22 - jepsen.util 2002 :invoke :read [7 nil] INFO [2022-01-10 05:42:18,264] jepsen worker 74 - jepsen.util 2774 :ok :read [24 18] INFO [2022-01-10 05:42:18,265] jepsen worker 74 - jepsen.util 2774 :invoke :write [24 23] INFO [2022-01-10 05:42:18,269] jepsen worker 64 - jepsen.util 2224 :ok :read [21 7] INFO [2022-01-10 05:42:18,269] jepsen worker 64 - jepsen.util 2224 :invoke :write [21 47] INFO [2022-01-10 05:42:18,272] jepsen worker 42 - jepsen.util 2202 :ok :read [14 14] INFO [2022-01-10 05:42:18,273] jepsen worker 42 - jepsen.util 2202 :invoke :read [14 nil] INFO [2022-01-10 05:42:18,655] jepsen worker 24 - jepsen.util 2004 :ok :read [8 27] INFO [2022-01-10 05:42:18,656] jepsen worker 24 - jepsen.util 2004 :invoke :read [8 nil] INFO [2022-01-10 05:42:18,672] jepsen worker 53 - jepsen.util 2663 :ok :read [17 41] INFO [2022-01-10 05:42:18,672] jepsen worker 53 - jepsen.util 2663 :invoke :write [17 22] INFO [2022-01-10 05:42:19,089] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:19,089] jepsen worker nemesis - couchbase.util Rebalance check count: 292 INFO [2022-01-10 05:42:19,090] jepsen worker nemesis - couchbase.util Rebalance stuck count: 230 INFO [2022-01-10 05:42:19,127] jepsen worker 13 - jepsen.util 2353 :ok :read [4 3] INFO [2022-01-10 05:42:19,128] jepsen worker 13 - jepsen.util 2353 :invoke :read [4 nil] INFO [2022-01-10 05:42:19,176] jepsen worker 79 - jepsen.util 2509 :info :write [26 17] :RequestTimeoutException INFO [2022-01-10 05:42:19,176] jepsen worker 79 - jepsen.util 2599 :invoke :write [26 29] INFO [2022-01-10 05:42:19,328] jepsen worker 32 - jepsen.util 2372 :ok :read [10 37] INFO [2022-01-10 05:42:19,329] jepsen worker 32 - jepsen.util 2372 :invoke :read [10 nil] INFO [2022-01-10 05:42:19,522] jepsen worker 65 - jepsen.util 2135 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:19,523] jepsen worker 65 - jepsen.util 2135 :invoke :write [21 29] INFO [2022-01-10 05:42:19,537] jepsen worker 22 - jepsen.util 2002 :ok :read [7 :nil] INFO [2022-01-10 05:42:19,538] jepsen worker 22 - jepsen.util 2002 :invoke :read [7 nil] INFO [2022-01-10 05:42:19,920] jepsen worker 1 - jepsen.util 2341 :ok :read [0 46] INFO [2022-01-10 05:42:19,921] jepsen worker 1 - jepsen.util 2341 :invoke :write [0 15] INFO [2022-01-10 05:42:19,928] jepsen worker 50 - jepsen.util 2390 :ok :read [16 38] INFO [2022-01-10 05:42:19,929] jepsen worker 50 - jepsen.util 2390 :invoke :write [16 19] INFO [2022-01-10 05:42:19,941] jepsen worker 21 - jepsen.util 2181 :info :write [7 45] :RequestTimeoutException INFO [2022-01-10 05:42:19,941] jepsen worker 23 - jepsen.util 2363 :info :write [7 38] :RequestTimeoutException INFO [2022-01-10 05:42:19,941] jepsen worker 25 - jepsen.util 2005 :info :write [8 23] :RequestTimeoutException INFO [2022-01-10 05:42:19,942] jepsen worker 25 - jepsen.util 2095 :invoke :write [8 22] INFO [2022-01-10 05:42:19,942] jepsen worker 21 - jepsen.util 2271 :invoke :write [7 20] INFO [2022-01-10 05:42:19,943] jepsen worker 23 - jepsen.util 2453 :invoke :read [7 nil] INFO [2022-01-10 05:42:20,000] jepsen worker 31 - jepsen.util 2551 :info :write [10 3] :RequestTimeoutException INFO [2022-01-10 05:42:20,001] jepsen worker 31 - jepsen.util 2641 :invoke :write [10 31] INFO [2022-01-10 05:42:20,018] jepsen worker 2 - jepsen.util 2252 :ok :read [0 46] WARN [2022-01-10 05:42:20,022] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18377556,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":19139974,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9215044,"encode_duration_us":16958,"operation_id":"0x445b","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10003224}],"total_count":1}} INFO [2022-01-10 05:42:20,041] jepsen worker 48 - jepsen.util 1938 :info :write [16 17] :RequestTimeoutException INFO [2022-01-10 05:42:20,087] jepsen worker 48 - jepsen.util 2028 :invoke :read [16 nil] WARN [2022-01-10 05:42:20,087] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":23321474,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":23321474,"total_server_duration_us":43,"operation_id":"0x44b5","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":23342868},{"operation_name":"get","last_dispatch_duration_us":20884505,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":20884505,"total_server_duration_us":19,"operation_id":"0x44c3","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":22478122},{"operation_name":"get","last_dispatch_duration_us":14656271,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":14656271,"total_server_duration_us":38,"operation_id":"0x45bf","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":17784166},{"operation_name":"get","last_dispatch_duration_us":14423411,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":14423411,"total_server_duration_us":19,"operation_id":"0x461f","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":16020373},{"operation_name":"get","last_dispatch_duration_us":15985410,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":15985410,"total_server_duration_us":32,"operation_id":"0x44e9","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":15987356},{"operation_name":"get","last_dispatch_duration_us":14995134,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14995134,"total_server_duration_us":416679,"operation_id":"0x4515","timeout_ms":30000,"last_server_duration_us":416679,"total_duration_us":14996973},{"operation_name":"get","last_dispatch_duration_us":13344360,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13344360,"total_server_duration_us":619447,"operation_id":"0x458c","timeout_ms":30000,"last_server_duration_us":619447,"total_duration_us":13346582},{"operation_name":"get","last_dispatch_duration_us":12524825,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12524825,"total_server_duration_us":619447,"operation_id":"0x458b","timeout_ms":30000,"last_server_duration_us":619447,"total_duration_us":12526871},{"operation_name":"get","last_dispatch_duration_us":12152995,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":12152995,"total_server_duration_us":19,"operation_id":"0x46d9","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":12153288},{"operation_name":"upsert","last_dispatch_duration_us":10150610,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10150610,"total_server_duration_us":19966,"encode_duration_us":16757,"operation_id":"0x45ed","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10010686}],"total_count":20}} INFO [2022-01-10 05:42:20,088] jepsen worker 2 - jepsen.util 2252 :invoke :write [0 13] INFO [2022-01-10 05:42:20,090] jepsen worker 13 - jepsen.util 2353 :ok :read [4 3] INFO [2022-01-10 05:42:20,091] jepsen worker 13 - jepsen.util 2353 :invoke :read [4 nil] INFO [2022-01-10 05:42:20,104] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:20,104] jepsen worker nemesis - couchbase.util Rebalance check count: 293 INFO [2022-01-10 05:42:20,104] jepsen worker nemesis - couchbase.util Rebalance stuck count: 231 INFO [2022-01-10 05:42:20,506] jepsen worker 13 - jepsen.util 2353 :ok :read [4 3] INFO [2022-01-10 05:42:20,507] jepsen worker 13 - jepsen.util 2353 :invoke :write [4 0] INFO [2022-01-10 05:42:20,610] jepsen worker 29 - jepsen.util 2279 :info :write [9 33] :RequestTimeoutException INFO [2022-01-10 05:42:20,611] jepsen worker 29 - jepsen.util 2369 :invoke :read [9 nil] INFO [2022-01-10 05:42:20,772] jepsen worker 22 - jepsen.util 2002 :ok :read [7 :nil] INFO [2022-01-10 05:42:20,773] jepsen worker 22 - jepsen.util 2002 :invoke :write [7 34] INFO [2022-01-10 05:42:20,796] jepsen worker 34 - jepsen.util 2104 :info :write [11 25] :RequestTimeoutException INFO [2022-01-10 05:42:20,797] jepsen worker 34 - jepsen.util 2194 :invoke :read [11 nil] INFO [2022-01-10 05:42:20,821] jepsen worker 85 - jepsen.util 2605 :info :write [28 45] :RequestTimeoutException INFO [2022-01-10 05:42:20,822] jepsen worker 85 - jepsen.util 2695 :invoke :read [28 nil] INFO [2022-01-10 05:42:20,976] jepsen worker 61 - jepsen.util 2221 :info :write [20 20] :RequestTimeoutException INFO [2022-01-10 05:42:20,976] jepsen worker 76 - jepsen.util 1606 :info :write [25 39] :RequestTimeoutException INFO [2022-01-10 05:42:20,977] jepsen worker 76 - jepsen.util 1696 :invoke :write [25 33] INFO [2022-01-10 05:42:20,977] jepsen worker 61 - jepsen.util 2311 :invoke :read [20 nil] INFO [2022-01-10 05:42:20,986] jepsen worker 75 - jepsen.util 1425 :info :write [25 13] :RequestTimeoutException INFO [2022-01-10 05:42:20,987] jepsen worker 75 - jepsen.util 1515 :invoke :write [25 9] INFO [2022-01-10 05:42:20,996] jepsen worker 43 - jepsen.util 2023 :info :write [14 23] :RequestTimeoutException INFO [2022-01-10 05:42:20,996] jepsen worker 43 - jepsen.util 2113 :invoke :read [14 nil] INFO [2022-01-10 05:42:21,111] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:21,111] jepsen worker nemesis - couchbase.util Rebalance check count: 294 INFO [2022-01-10 05:42:21,111] jepsen worker nemesis - couchbase.util Rebalance stuck count: 232 INFO [2022-01-10 05:42:21,245] jepsen worker 78 - jepsen.util 2508 :info :write [26 37] :RequestTimeoutException INFO [2022-01-10 05:42:21,246] jepsen worker 78 - jepsen.util 2598 :invoke :write [26 11] INFO [2022-01-10 05:42:21,352] jepsen worker 82 - jepsen.util 2152 :info :write [27 48] :RequestTimeoutException INFO [2022-01-10 05:42:21,353] jepsen worker 82 - jepsen.util 2242 :invoke :write [27 40] INFO [2022-01-10 05:42:21,425] jepsen worker 27 - jepsen.util 2097 :info :write [9 38] :RequestTimeoutException INFO [2022-01-10 05:42:21,426] jepsen worker 27 - jepsen.util 2187 :invoke :write [9 2] INFO [2022-01-10 05:42:21,465] jepsen worker 59 - jepsen.util 2309 :info :write [19 15] :RequestTimeoutException INFO [2022-01-10 05:42:21,466] jepsen worker 59 - jepsen.util 2399 :invoke :write [19 43] INFO [2022-01-10 05:42:21,496] jepsen worker 67 - jepsen.util 2047 :info :write [22 20] :RequestTimeoutException INFO [2022-01-10 05:42:21,497] jepsen worker 67 - jepsen.util 2137 :invoke :write [22 44] WARN [2022-01-10 05:42:21,524] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9494938,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13859189,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":10177626,"encode_duration_us":40650,"operation_id":"0x4655","timeout_ms":10000,"last_server_duration_us":9230339,"total_duration_us":10005126}],"total_count":1}} WARN [2022-01-10 05:42:21,524] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":3975343,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9280866,"total_server_duration_us":762449,"encode_duration_us":17387,"operation_id":"0x46d7","timeout_ms":10000,"last_server_duration_us":376530,"total_duration_us":10010471},{"operation_name":"upsert","last_dispatch_duration_us":3738893,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10937755,"total_server_duration_us":579806,"encode_duration_us":16308,"operation_id":"0x45ec","timeout_ms":10000,"last_server_duration_us":377905,"total_duration_us":10009062},{"operation_name":"upsert","last_dispatch_duration_us":1411471,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1411471,"total_server_duration_us":202533,"encode_duration_us":16626,"operation_id":"0x4793","timeout_ms":10000,"last_server_duration_us":202533,"total_duration_us":10007018},{"operation_name":"upsert","last_dispatch_duration_us":4126561,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11752858,"total_server_duration_us":762196,"encode_duration_us":16837,"operation_id":"0x4625","timeout_ms":10000,"last_server_duration_us":587887,"total_duration_us":10006890},{"operation_name":"upsert","last_dispatch_duration_us":794496,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9764861,"total_server_duration_us":4085095,"encode_duration_us":18953,"operation_id":"0x4737","timeout_ms":10000,"last_server_duration_us":114454,"total_duration_us":10006446},{"operation_name":"upsert","last_dispatch_duration_us":9494938,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":13859189,"total_server_duration_us":10177626,"encode_duration_us":40650,"operation_id":"0x4655","timeout_ms":10000,"last_server_duration_us":9230339,"total_duration_us":10005126},{"operation_name":"upsert","last_dispatch_duration_us":5016497,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5016497,"total_server_duration_us":405275,"encode_duration_us":16853,"operation_id":"0x4794","timeout_ms":10000,"last_server_duration_us":405275,"total_duration_us":10004156},{"operation_name":"upsert","last_dispatch_duration_us":980089,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":8655264,"total_server_duration_us":5490949,"encode_duration_us":20038,"operation_id":"0x46e8","timeout_ms":10000,"last_server_duration_us":410105,"total_duration_us":10003728},{"operation_name":"upsert","last_dispatch_duration_us":3572442,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7885726,"total_server_duration_us":589964,"encode_duration_us":17131,"operation_id":"0x4716","timeout_ms":10000,"last_server_duration_us":205709,"total_duration_us":10003596},{"operation_name":"upsert","last_dispatch_duration_us":351597,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":4891883,"total_server_duration_us":1361238,"encode_duration_us":18374,"operation_id":"0x4628","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10003337}],"total_count":16}} WARN [2022-01-10 05:42:21,624] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16704353,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16704353,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":10586148,"encode_duration_us":16311,"operation_id":"0x4567","timeout_ms":10000,"last_server_duration_us":10586148,"total_duration_us":10001045},{"operation_name":"upsert","last_dispatch_duration_us":12473498,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12473498,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9207861,"encode_duration_us":16890,"operation_id":"0x45fb","timeout_ms":10000,"last_server_duration_us":9207861,"total_duration_us":10005341}],"total_count":2}} WARN [2022-01-10 05:42:21,625] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":778716,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9704787,"total_server_duration_us":2465706,"encode_duration_us":17166,"operation_id":"0x47ce","timeout_ms":10000,"last_server_duration_us":214713,"total_duration_us":10010224},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":18948,"operation_id":"0x47c7","timeout_ms":10000,"total_duration_us":10009239},{"operation_name":"upsert","last_dispatch_duration_us":1585453,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9456133,"total_server_duration_us":4056306,"encode_duration_us":16698,"operation_id":"0x470a","timeout_ms":10000,"last_server_duration_us":215363,"total_duration_us":10008679},{"operation_name":"upsert","last_dispatch_duration_us":5848989,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5848989,"total_server_duration_us":396248,"encode_duration_us":16199,"operation_id":"0x4620","timeout_ms":10000,"last_server_duration_us":396248,"total_duration_us":10008401},{"operation_name":"upsert","last_dispatch_duration_us":2838119,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":2838119,"total_server_duration_us":228525,"encode_duration_us":16738,"operation_id":"0x47d3","timeout_ms":10000,"last_server_duration_us":228525,"total_duration_us":10006477},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":22124,"operation_id":"0x47c6","timeout_ms":10000,"total_duration_us":10005941},{"operation_name":"upsert","last_dispatch_duration_us":7235209,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7235209,"total_server_duration_us":400469,"encode_duration_us":16526,"operation_id":"0x47c4","timeout_ms":10000,"last_server_duration_us":400469,"total_duration_us":10005735},{"operation_name":"upsert","last_dispatch_duration_us":913547,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":10681210,"total_server_duration_us":2955438,"encode_duration_us":16254,"operation_id":"0x476d","timeout_ms":10000,"last_server_duration_us":72726,"total_duration_us":10003567},{"operation_name":"upsert","last_dispatch_duration_us":2120284,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2120284,"total_server_duration_us":1547203,"encode_duration_us":19059,"operation_id":"0x47bb","timeout_ms":10000,"last_server_duration_us":1547203,"total_duration_us":10003180},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":17830,"operation_id":"0x47c5","timeout_ms":10000,"total_duration_us":10002441}],"total_count":21}} WARN [2022-01-10 05:42:21,630] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15616145,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":15616145,"total_server_duration_us":11552,"operation_id":"0x4660","timeout_ms":30000,"last_server_duration_us":11552,"total_duration_us":15618049},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":18666,"operation_id":"0x464f","timeout_ms":10000,"total_duration_us":10009457},{"operation_name":"upsert","last_dispatch_duration_us":6110473,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6110473,"total_server_duration_us":191687,"encode_duration_us":22362,"operation_id":"0x46af","timeout_ms":10000,"last_server_duration_us":191687,"total_duration_us":10009241},{"operation_name":"upsert","last_dispatch_duration_us":4781394,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":4781394,"total_server_duration_us":1291425,"encode_duration_us":16645,"operation_id":"0x46b6","timeout_ms":10000,"last_server_duration_us":1291425,"total_duration_us":10008973},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":17136,"operation_id":"0x47c2","timeout_ms":10000,"total_duration_us":10008430},{"operation_name":"upsert","last_dispatch_duration_us":6712125,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6712125,"total_server_duration_us":401315,"encode_duration_us":17401,"operation_id":"0x4709","timeout_ms":10000,"last_server_duration_us":401315,"total_duration_us":10006394},{"operation_name":"upsert","last_dispatch_duration_us":4512817,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":4512817,"total_server_duration_us":199169,"encode_duration_us":17317,"operation_id":"0x4717","timeout_ms":10000,"last_server_duration_us":199169,"total_duration_us":10002825},{"operation_name":"upsert","last_dispatch_duration_us":10868199,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":16855008,"total_server_duration_us":424941,"encode_duration_us":16435,"operation_id":"0x4689","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10001671},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":17350,"operation_id":"0x47d0","timeout_ms":10000,"total_duration_us":10001395},{"operation_name":"get","last_dispatch_duration_us":8921081,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8921081,"total_server_duration_us":1429916,"operation_id":"0x473a","timeout_ms":30000,"last_server_duration_us":1429916,"total_duration_us":8923101}],"total_count":16}} WARN [2022-01-10 05:42:21,635] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":29066071,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":29066071,"total_server_duration_us":1020055,"operation_id":"0x42b0","timeout_ms":30000,"last_server_duration_us":1020055,"total_duration_us":29068137},{"operation_name":"get","last_dispatch_duration_us":11205866,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11205866,"total_server_duration_us":43,"operation_id":"0x4458","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":22488610},{"operation_name":"get","last_dispatch_duration_us":4045163,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4045163,"total_server_duration_us":15,"operation_id":"0x463a","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":14307784},{"operation_name":"get","last_dispatch_duration_us":11449541,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11449541,"total_server_duration_us":49,"operation_id":"0x4667","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":11449856},{"operation_name":"get","last_dispatch_duration_us":11404160,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11404160,"total_server_duration_us":86622,"operation_id":"0x4746","timeout_ms":30000,"last_server_duration_us":86622,"total_duration_us":11406510},{"operation_name":"upsert","last_dispatch_duration_us":13956075,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":17618232,"total_server_duration_us":9570066,"encode_duration_us":16850,"operation_id":"0x461d","timeout_ms":10000,"last_server_duration_us":9189682,"total_duration_us":10008476},{"operation_name":"upsert","last_dispatch_duration_us":156961,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9413064,"total_server_duration_us":294548,"encode_duration_us":18565,"operation_id":"0x47d4","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10004488},{"operation_name":"upsert","last_dispatch_duration_us":11582510,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11582510,"total_server_duration_us":393445,"encode_duration_us":17094,"operation_id":"0x4651","timeout_ms":10000,"last_server_duration_us":393445,"total_duration_us":10002721},{"operation_name":"upsert","last_dispatch_duration_us":1339037,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":6029937,"total_server_duration_us":3918332,"encode_duration_us":17242,"operation_id":"0x4656","timeout_ms":10000,"last_server_duration_us":1326892,"total_duration_us":10002098},{"operation_name":"upsert","last_dispatch_duration_us":1428437,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9784020,"total_server_duration_us":2107850,"encode_duration_us":18985,"operation_id":"0x46e4","timeout_ms":10000,"last_server_duration_us":369962,"total_duration_us":10001746}],"total_count":25}} WARN [2022-01-10 05:42:21,730] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17448624,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":20193389,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":11065410,"encode_duration_us":16844,"operation_id":"0x4485","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10008517}],"total_count":1}} INFO [2022-01-10 05:42:21,745] jepsen worker 69 - jepsen.util 2319 :info :write [23 18] :RequestTimeoutException INFO [2022-01-10 05:42:21,746] jepsen worker 69 - jepsen.util 2409 :invoke :read [23 nil] INFO [2022-01-10 05:42:21,764] jepsen worker 54 - jepsen.util 2214 :info :write [18 46] :RequestTimeoutException INFO [2022-01-10 05:42:21,765] jepsen worker 54 - jepsen.util 2304 :invoke :write [18 25] INFO [2022-01-10 05:42:21,795] jepsen worker 5 - jepsen.util 2345 :info :write [1 36] :RequestTimeoutException INFO [2022-01-10 05:42:21,795] jepsen worker 5 - jepsen.util 2435 :invoke :read [1 nil] INFO [2022-01-10 05:42:21,815] jepsen worker 36 - jepsen.util 1926 :info :write [12 35] :RequestTimeoutException INFO [2022-01-10 05:42:21,816] jepsen worker 36 - jepsen.util 2016 :invoke :read [12 nil] WARN [2022-01-10 05:42:21,825] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17002963,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":17002963,"total_server_duration_us":7856,"operation_id":"0x4624","timeout_ms":30000,"last_server_duration_us":7856,"total_duration_us":17003342},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","encode_duration_us":16734,"operation_id":"0x46ad","timeout_ms":10000,"total_duration_us":10009192},{"operation_name":"upsert","last_dispatch_duration_us":1046796,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":7415761,"total_server_duration_us":4289635,"encode_duration_us":17184,"operation_id":"0x4648","timeout_ms":10000,"last_server_duration_us":202744,"total_duration_us":10008873},{"operation_name":"upsert","last_dispatch_duration_us":235767,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":7070608,"total_server_duration_us":1213431,"encode_duration_us":19403,"operation_id":"0x4797","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10008700},{"operation_name":"upsert","last_dispatch_duration_us":1429267,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":11139036,"total_server_duration_us":500135,"encode_duration_us":16552,"operation_id":"0x468c","timeout_ms":10000,"last_server_duration_us":481182,"total_duration_us":10008461},{"operation_name":"upsert","last_dispatch_duration_us":746829,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10265414,"total_server_duration_us":745954,"encode_duration_us":16519,"operation_id":"0x46e7","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10007054},{"operation_name":"upsert","last_dispatch_duration_us":4652062,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":4652062,"total_server_duration_us":192926,"encode_duration_us":17107,"operation_id":"0x47a1","timeout_ms":10000,"last_server_duration_us":192926,"total_duration_us":10006928},{"operation_name":"upsert","last_dispatch_duration_us":5549680,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":5549680,"total_server_duration_us":587887,"encode_duration_us":26324,"operation_id":"0x470c","timeout_ms":10000,"last_server_duration_us":587887,"total_duration_us":10006355},{"operation_name":"upsert","last_dispatch_duration_us":12473498,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":12473498,"total_server_duration_us":9207861,"encode_duration_us":16890,"operation_id":"0x45fb","timeout_ms":10000,"last_server_duration_us":9207861,"total_duration_us":10005341},{"operation_name":"upsert","last_dispatch_duration_us":7653168,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7653168,"total_server_duration_us":1435250,"encode_duration_us":16085,"operation_id":"0x4621","timeout_ms":10000,"last_server_duration_us":1435250,"total_duration_us":10003640}],"total_count":17}} INFO [2022-01-10 05:42:21,825] jepsen worker 42 - jepsen.util 2202 :ok :read [14 14] INFO [2022-01-10 05:42:21,826] jepsen worker 42 - jepsen.util 2202 :invoke :read [14 nil] INFO [2022-01-10 05:42:21,835] jepsen worker 38 - jepsen.util 2378 :info :write [12 39] :RequestTimeoutException INFO [2022-01-10 05:42:21,835] jepsen worker 38 - jepsen.util 2468 :invoke :write [12 36] WARN [2022-01-10 05:42:21,836] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":13956075,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17618232,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9570066,"encode_duration_us":16850,"operation_id":"0x461d","timeout_ms":10000,"last_server_duration_us":9189682,"total_duration_us":10008476},{"operation_name":"upsert","last_dispatch_duration_us":30959377,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":30959377,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":10418705,"encode_duration_us":18770,"operation_id":"0x42a1","timeout_ms":10000,"last_server_duration_us":10418705,"total_duration_us":10008706}],"total_count":2}} INFO [2022-01-10 05:42:21,995] jepsen worker 70 - jepsen.util 2500 :info :write [23 17] :RequestTimeoutException INFO [2022-01-10 05:42:21,996] jepsen worker 70 - jepsen.util 2590 :invoke :read [23 nil] INFO [2022-01-10 05:42:22,117] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:22,117] jepsen worker nemesis - couchbase.util Rebalance check count: 295 INFO [2022-01-10 05:42:22,117] jepsen worker nemesis - couchbase.util Rebalance stuck count: 233 INFO [2022-01-10 05:42:22,170] jepsen worker 49 - jepsen.util 2119 :info :write [16 47] :RequestTimeoutException INFO [2022-01-10 05:42:22,170] jepsen worker 49 - jepsen.util 2209 :invoke :write [16 35] INFO [2022-01-10 05:42:22,312] jepsen worker 6 - jepsen.util 2166 :info :write [2 42] :RequestTimeoutException INFO [2022-01-10 05:42:22,312] jepsen worker 6 - jepsen.util 2256 :invoke :write [2 34] INFO [2022-01-10 05:42:22,316] jepsen worker 41 - jepsen.util 2111 :info :write [13 40] :RequestTimeoutException INFO [2022-01-10 05:42:22,316] jepsen worker 41 - jepsen.util 2201 :invoke :read [13 nil] INFO [2022-01-10 05:42:22,458] jepsen worker 20 - jepsen.util 2000 :ok :read [6 25] INFO [2022-01-10 05:42:22,459] jepsen worker 20 - jepsen.util 2000 :invoke :read [6 nil] INFO [2022-01-10 05:42:22,551] jepsen worker 57 - jepsen.util 2577 :info :write [19 42] :RequestTimeoutException INFO [2022-01-10 05:42:22,552] jepsen worker 57 - jepsen.util 2667 :invoke :write [19 31] INFO [2022-01-10 05:42:22,645] jepsen worker 9 - jepsen.util 2619 :info :write [3 18] :RequestTimeoutException INFO [2022-01-10 05:42:22,646] jepsen worker 9 - jepsen.util 2709 :invoke :read [3 nil] INFO [2022-01-10 05:42:22,672] jepsen worker 9 - jepsen.util 2709 :ok :read [3 2] INFO [2022-01-10 05:42:22,673] jepsen worker 9 - jepsen.util 2709 :invoke :write [3 38] INFO [2022-01-10 05:42:22,817] jepsen worker 61 - jepsen.util 2311 :ok :read [20 :nil] INFO [2022-01-10 05:42:22,817] jepsen worker 61 - jepsen.util 2311 :invoke :write [20 26] INFO [2022-01-10 05:42:22,836] jepsen worker 51 - jepsen.util 2481 :info :write [17 25] :RequestTimeoutException INFO [2022-01-10 05:42:22,837] jepsen worker 51 - jepsen.util 2571 :invoke :read [17 nil] INFO [2022-01-10 05:42:22,900] jepsen worker 51 - jepsen.util 2571 :ok :read [17 41] INFO [2022-01-10 05:42:22,900] jepsen worker 51 - jepsen.util 2571 :invoke :write [17 14] INFO [2022-01-10 05:42:22,946] jepsen worker 7 - jepsen.util 2347 :info :write [2 43] :RequestTimeoutException INFO [2022-01-10 05:42:22,947] jepsen worker 7 - jepsen.util 2437 :invoke :read [2 nil] INFO [2022-01-10 05:42:23,042] jepsen worker 45 - jepsen.util 2565 :info :write [15 32] :RequestTimeoutException INFO [2022-01-10 05:42:23,042] jepsen worker 45 - jepsen.util 2655 :invoke :write [15 41] INFO [2022-01-10 05:42:23,061] jepsen worker 11 - jepsen.util 2171 :info :write [3 37] :RequestTimeoutException INFO [2022-01-10 05:42:23,061] jepsen worker 11 - jepsen.util 2261 :invoke :write [3 16] INFO [2022-01-10 05:42:23,075] jepsen worker 55 - jepsen.util 2485 :info :write [18 31] :RequestTimeoutException INFO [2022-01-10 05:42:23,075] jepsen worker 55 - jepsen.util 2575 :invoke :write [18 37] INFO [2022-01-10 05:42:23,123] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:23,123] jepsen worker nemesis - couchbase.util Rebalance check count: 296 INFO [2022-01-10 05:42:23,124] jepsen worker nemesis - couchbase.util Rebalance stuck count: 234 INFO [2022-01-10 05:42:23,124] jepsen worker 58 - jepsen.util 2218 :ok :read [19 31] INFO [2022-01-10 05:42:23,125] jepsen worker 58 - jepsen.util 2218 :invoke :write [19 30] INFO [2022-01-10 05:42:23,161] jepsen worker 84 - jepsen.util 2244 :ok :read [28 13] INFO [2022-01-10 05:42:23,162] jepsen worker 84 - jepsen.util 2244 :invoke :read [28 nil] INFO [2022-01-10 05:42:23,170] jepsen worker 86 - jepsen.util 1796 :ok :read [28 13] INFO [2022-01-10 05:42:23,171] jepsen worker 86 - jepsen.util 1796 :invoke :read [28 nil] INFO [2022-01-10 05:42:23,177] jepsen worker 28 - jepsen.util 2188 :ok :read [9 22] INFO [2022-01-10 05:42:23,177] jepsen worker 28 - jepsen.util 2188 :invoke :write [9 24] INFO [2022-01-10 05:42:23,250] jepsen worker 30 - jepsen.util 2100 :ok :read [10 37] INFO [2022-01-10 05:42:23,251] jepsen worker 30 - jepsen.util 2100 :invoke :write [10 8] INFO [2022-01-10 05:42:23,254] jepsen worker 71 - jepsen.util 1871 :ok :read [23 26] INFO [2022-01-10 05:42:23,254] jepsen worker 71 - jepsen.util 1871 :invoke :read [23 nil] INFO [2022-01-10 05:42:23,310] jepsen worker 72 - jepsen.util 2322 :info :write [24 25] :RequestTimeoutException INFO [2022-01-10 05:42:23,310] jepsen worker 72 - jepsen.util 2412 :invoke :write [24 36] INFO [2022-01-10 05:42:23,423] jepsen worker 8 - jepsen.util 2618 :ok :read [2 14] INFO [2022-01-10 05:42:23,423] jepsen worker 8 - jepsen.util 2618 :invoke :read [2 nil] INFO [2022-01-10 05:42:23,438] jepsen worker 88 - jepsen.util 2248 :ok :read [29 0] INFO [2022-01-10 05:42:23,438] jepsen worker 88 - jepsen.util 2248 :invoke :read [29 nil] INFO [2022-01-10 05:42:23,724] jepsen worker 0 - jepsen.util 2160 :info :write [0 30] :RequestTimeoutException INFO [2022-01-10 05:42:23,725] jepsen worker 0 - jepsen.util 2250 :invoke :write [0 12] INFO [2022-01-10 05:42:23,762] jepsen worker 39 - jepsen.util 2559 :info :write [13 44] :RequestTimeoutException INFO [2022-01-10 05:42:23,763] jepsen worker 39 - jepsen.util 2649 :invoke :write [13 7] INFO [2022-01-10 05:42:23,817] jepsen worker 24 - jepsen.util 2004 :ok :read [8 27] INFO [2022-01-10 05:42:23,818] jepsen worker 24 - jepsen.util 2004 :invoke :read [8 nil] INFO [2022-01-10 05:42:23,832] jepsen worker 52 - jepsen.util 2752 :info :write [17 10] :RequestTimeoutException INFO [2022-01-10 05:42:23,833] jepsen worker 52 - jepsen.util 2842 :invoke :write [17 37] INFO [2022-01-10 05:42:24,130] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:24,130] jepsen worker nemesis - couchbase.util Rebalance check count: 297 INFO [2022-01-10 05:42:24,130] jepsen worker nemesis - couchbase.util Rebalance stuck count: 235 INFO [2022-01-10 05:42:24,676] jepsen worker 37 - jepsen.util 2287 :info :write [12 29] :RequestTimeoutException INFO [2022-01-10 05:42:24,677] jepsen worker 37 - jepsen.util 2377 :invoke :write [12 35] INFO [2022-01-10 05:42:25,072] jepsen worker 60 - jepsen.util 2580 :info :write [20 34] :RequestTimeoutException INFO [2022-01-10 05:42:25,073] jepsen worker 60 - jepsen.util 2670 :invoke :write [20 49] INFO [2022-01-10 05:42:25,135] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:25,136] jepsen worker nemesis - couchbase.util Rebalance check count: 298 INFO [2022-01-10 05:42:25,136] jepsen worker nemesis - couchbase.util Rebalance stuck count: 236 INFO [2022-01-10 05:42:25,219] jepsen worker 89 - jepsen.util 2609 :info :write [29 10] :RequestTimeoutException INFO [2022-01-10 05:42:25,221] jepsen worker 89 - jepsen.util 2699 :invoke :read [29 nil] INFO [2022-01-10 05:42:25,243] jepsen worker 42 - jepsen.util 2202 :ok :read [14 14] INFO [2022-01-10 05:42:25,243] jepsen worker 42 - jepsen.util 2202 :invoke :read [14 nil] INFO [2022-01-10 05:42:25,245] jepsen worker 89 - jepsen.util 2699 :ok :read [29 0] INFO [2022-01-10 05:42:25,245] jepsen worker 89 - jepsen.util 2699 :invoke :write [29 16] INFO [2022-01-10 05:42:25,376] jepsen worker 68 - jepsen.util 2408 :ok :read [22 41] INFO [2022-01-10 05:42:25,377] jepsen worker 68 - jepsen.util 2408 :invoke :write [22 27] INFO [2022-01-10 05:42:25,683] jepsen worker 56 - jepsen.util 2486 :ok :read [18 14] INFO [2022-01-10 05:42:25,684] jepsen worker 56 - jepsen.util 2486 :invoke :read [18 nil] INFO [2022-01-10 05:42:25,716] jepsen worker 66 - jepsen.util 1956 :info :write [22 10] :RequestTimeoutException INFO [2022-01-10 05:42:25,717] jepsen worker 66 - jepsen.util 2046 :invoke :write [22 0] INFO [2022-01-10 05:42:26,141] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:26,141] jepsen worker nemesis - couchbase.util Rebalance check count: 299 INFO [2022-01-10 05:42:26,141] jepsen worker nemesis - couchbase.util Rebalance stuck count: 237 INFO [2022-01-10 05:42:26,336] jepsen worker 12 - jepsen.util 2262 :info :write [4 31] :RequestTimeoutException INFO [2022-01-10 05:42:26,336] jepsen worker 12 - jepsen.util 2352 :invoke :write [4 9] INFO [2022-01-10 05:42:26,540] jepsen worker 4 - jepsen.util 2524 :info :write [1 25] :RequestTimeoutException INFO [2022-01-10 05:42:26,540] jepsen worker 4 - jepsen.util 2614 :invoke :read [1 nil] INFO [2022-01-10 05:42:26,847] jepsen worker 18 - jepsen.util 1908 :info :write [6 20] :RequestTimeoutException INFO [2022-01-10 05:42:26,847] jepsen worker 18 - jepsen.util 1998 :invoke :write [6 35] INFO [2022-01-10 05:42:26,971] jepsen worker 14 - jepsen.util 2624 :info :write [4 38] :RequestTimeoutException INFO [2022-01-10 05:42:26,972] jepsen worker 14 - jepsen.util 2714 :invoke :write [4 14] INFO [2022-01-10 05:42:27,062] jepsen worker 47 - jepsen.util 2207 :info :write [15 42] :RequestTimeoutException INFO [2022-01-10 05:42:27,062] jepsen worker 63 - jepsen.util 2133 :info :write [21 36] :RequestTimeoutException INFO [2022-01-10 05:42:27,062] jepsen worker 47 - jepsen.util 2297 :invoke :read [15 nil] INFO [2022-01-10 05:42:27,063] jepsen worker 63 - jepsen.util 2223 :invoke :read [21 nil] INFO [2022-01-10 05:42:27,066] jepsen worker 33 - jepsen.util 2463 :info :write [11 25] :RequestTimeoutException INFO [2022-01-10 05:42:27,066] jepsen worker 77 - jepsen.util 2597 :info :write [25 5] :RequestTimeoutException INFO [2022-01-10 05:42:27,066] jepsen worker 33 - jepsen.util 2553 :invoke :write [11 21] INFO [2022-01-10 05:42:27,067] jepsen worker 77 - jepsen.util 2687 :invoke :write [25 48] INFO [2022-01-10 05:42:27,089] jepsen worker 3 - jepsen.util 2343 :info :write [1 10] :RequestTimeoutException INFO [2022-01-10 05:42:27,090] jepsen worker 3 - jepsen.util 2433 :invoke :read [1 nil] INFO [2022-01-10 05:42:27,095] jepsen worker 40 - jepsen.util 1750 :info :write [13 0] :RequestTimeoutException INFO [2022-01-10 05:42:27,095] jepsen worker 40 - jepsen.util 1840 :invoke :read [13 nil] INFO [2022-01-10 05:42:27,147] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:27,147] jepsen worker nemesis - couchbase.util Rebalance check count: 300 INFO [2022-01-10 05:42:27,147] jepsen worker nemesis - couchbase.util Rebalance stuck count: 238 INFO [2022-01-10 05:42:27,149] jepsen worker 10 - jepsen.util 2440 :info :write [3 2] :RequestTimeoutException INFO [2022-01-10 05:42:27,150] jepsen worker 10 - jepsen.util 2530 :invoke :read [3 nil] INFO [2022-01-10 05:42:27,207] jepsen worker 10 - jepsen.util 2530 :ok :read [3 31] INFO [2022-01-10 05:42:27,208] jepsen worker 10 - jepsen.util 2530 :invoke :write [3 45] INFO [2022-01-10 05:42:27,215] jepsen worker 26 - jepsen.util 1826 :info :write [8 49] :RequestTimeoutException INFO [2022-01-10 05:42:27,215] jepsen worker 62 - jepsen.util 2402 :info :write [20 8] :RequestTimeoutException INFO [2022-01-10 05:42:27,216] jepsen worker 62 - jepsen.util 2492 :invoke :write [20 8] INFO [2022-01-10 05:42:27,216] jepsen worker 26 - jepsen.util 1916 :invoke :write [8 6] INFO [2022-01-10 05:42:27,381] jepsen worker 80 - jepsen.util 2600 :info :write [26 17] :RequestTimeoutException INFO [2022-01-10 05:42:27,382] jepsen worker 80 - jepsen.util 2690 :invoke :write [26 8] INFO [2022-01-10 05:42:27,436] jepsen worker 44 - jepsen.util 2204 :info :write [14 13] :RequestTimeoutException INFO [2022-01-10 05:42:27,437] jepsen worker 44 - jepsen.util 2294 :invoke :read [14 nil] INFO [2022-01-10 05:42:27,446] jepsen worker 46 - jepsen.util 2566 :info :write [15 0] :RequestTimeoutException INFO [2022-01-10 05:42:27,447] jepsen worker 46 - jepsen.util 2656 :invoke :read [15 nil] INFO [2022-01-10 05:42:27,520] jepsen worker 17 - jepsen.util 2357 :info :write [5 22] :RequestTimeoutException INFO [2022-01-10 05:42:27,520] jepsen worker 17 - jepsen.util 2447 :invoke :read [5 nil] INFO [2022-01-10 05:42:27,545] jepsen worker 43 - jepsen.util 2113 :ok :read [14 14] INFO [2022-01-10 05:42:27,545] jepsen worker 43 - jepsen.util 2113 :invoke :read [14 nil] INFO [2022-01-10 05:42:27,566] jepsen worker 48 - jepsen.util 2028 :ok :read [16 38] INFO [2022-01-10 05:42:27,567] jepsen worker 48 - jepsen.util 2028 :invoke :write [16 47] INFO [2022-01-10 05:42:28,001] jepsen worker 81 - jepsen.util 2511 :info :write [27 20] :RequestTimeoutException INFO [2022-01-10 05:42:28,002] jepsen worker 81 - jepsen.util 2601 :invoke :read [27 nil] INFO [2022-01-10 05:42:28,152] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:28,153] jepsen worker nemesis - couchbase.util Rebalance check count: 301 INFO [2022-01-10 05:42:28,153] jepsen worker nemesis - couchbase.util Rebalance stuck count: 239 INFO [2022-01-10 05:42:28,211] jepsen worker 83 - jepsen.util 2603 :info :write [27 11] :RequestTimeoutException INFO [2022-01-10 05:42:28,211] jepsen worker 83 - jepsen.util 2693 :invoke :write [27 47] INFO [2022-01-10 05:42:28,245] jepsen worker 19 - jepsen.util 2359 :info :write [6 48] :RequestTimeoutException INFO [2022-01-10 05:42:28,246] jepsen worker 19 - jepsen.util 2449 :invoke :write [6 35] INFO [2022-01-10 05:42:28,256] jepsen worker 73 - jepsen.util 2053 :info :write [24 28] :RequestTimeoutException INFO [2022-01-10 05:42:28,257] jepsen worker 73 - jepsen.util 2143 :invoke :read [24 nil] INFO [2022-01-10 05:42:28,286] jepsen worker 74 - jepsen.util 2774 :info :write [24 23] :RequestTimeoutException INFO [2022-01-10 05:42:28,286] jepsen worker 74 - jepsen.util 2864 :invoke :write [24 27] INFO [2022-01-10 05:42:28,295] jepsen worker 64 - jepsen.util 2224 :info :write [21 47] :RequestTimeoutException INFO [2022-01-10 05:42:28,295] jepsen worker 64 - jepsen.util 2314 :invoke :read [21 nil] INFO [2022-01-10 05:42:28,365] jepsen worker 32 - jepsen.util 2372 :ok :read [10 37] INFO [2022-01-10 05:42:28,365] jepsen worker 32 - jepsen.util 2372 :invoke :write [10 38] INFO [2022-01-10 05:42:28,374] jepsen worker 36 - jepsen.util 2016 :ok :read [12 42] INFO [2022-01-10 05:42:28,374] jepsen worker 36 - jepsen.util 2016 :invoke :write [12 32] INFO [2022-01-10 05:42:28,697] jepsen worker 53 - jepsen.util 2663 :info :write [17 22] :RequestTimeoutException INFO [2022-01-10 05:42:28,697] jepsen worker 53 - jepsen.util 2753 :invoke :read [17 nil] INFO [2022-01-10 05:42:28,728] jepsen worker 53 - jepsen.util 2753 :ok :read [17 41] INFO [2022-01-10 05:42:28,729] jepsen worker 53 - jepsen.util 2753 :invoke :read [17 nil] INFO [2022-01-10 05:42:28,754] jepsen worker 53 - jepsen.util 2753 :ok :read [17 41] INFO [2022-01-10 05:42:28,754] jepsen worker 53 - jepsen.util 2753 :invoke :read [17 nil] INFO [2022-01-10 05:42:28,781] jepsen worker 53 - jepsen.util 2753 :ok :read [17 41] INFO [2022-01-10 05:42:28,782] jepsen worker 53 - jepsen.util 2753 :invoke :write [17 27] INFO [2022-01-10 05:42:29,160] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:29,160] jepsen worker nemesis - couchbase.util Rebalance check count: 302 INFO [2022-01-10 05:42:29,160] jepsen worker nemesis - couchbase.util Rebalance stuck count: 240 INFO [2022-01-10 05:42:29,195] jepsen worker 79 - jepsen.util 2599 :info :write [26 29] :RequestTimeoutException INFO [2022-01-10 05:42:29,196] jepsen worker 79 - jepsen.util 2689 :invoke :read [26 nil] INFO [2022-01-10 05:42:29,549] jepsen worker 65 - jepsen.util 2135 :info :write [21 29] :RequestTimeoutException INFO [2022-01-10 05:42:29,550] jepsen worker 65 - jepsen.util 2225 :invoke :read [21 nil] INFO [2022-01-10 05:42:29,721] jepsen worker 79 - jepsen.util 2689 :ok :read [26 10] INFO [2022-01-10 05:42:29,722] jepsen worker 79 - jepsen.util 2689 :invoke :read [26 nil] INFO [2022-01-10 05:42:29,792] jepsen worker 79 - jepsen.util 2689 :ok :read [26 10] INFO [2022-01-10 05:42:29,793] jepsen worker 79 - jepsen.util 2689 :invoke :write [26 44] INFO [2022-01-10 05:42:29,913] jepsen worker 50 - jepsen.util 2390 :info :write [16 19] :SyncWriteAmbiguous INFO [2022-01-10 05:42:29,914] jepsen worker 50 - jepsen.util 2480 :invoke :write [16 20] INFO [2022-01-10 05:42:29,946] jepsen worker 1 - jepsen.util 2341 :info :write [0 15] :RequestTimeoutException INFO [2022-01-10 05:42:29,947] jepsen worker 1 - jepsen.util 2431 :invoke :read [0 nil] INFO [2022-01-10 05:42:29,965] jepsen worker 25 - jepsen.util 2095 :info :write [8 22] :RequestTimeoutException INFO [2022-01-10 05:42:29,966] jepsen worker 25 - jepsen.util 2185 :invoke :write [8 47] INFO [2022-01-10 05:42:29,969] jepsen worker 21 - jepsen.util 2271 :info :write [7 20] :RequestTimeoutException INFO [2022-01-10 05:42:29,970] jepsen worker 21 - jepsen.util 2361 :invoke :write [7 28] WARN [2022-01-10 05:42:30,000] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11236078,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11371716,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9052492,"encode_duration_us":18284,"operation_id":"0x47f4","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10003470},{"operation_name":"upsert","last_dispatch_duration_us":11667899,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11667899,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9221773,"encode_duration_us":16695,"operation_id":"0x47d1","timeout_ms":10000,"last_server_duration_us":9221773,"total_duration_us":10007687},{"operation_name":"upsert","last_dispatch_duration_us":11273928,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11273928,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9041612,"encode_duration_us":17887,"operation_id":"0x47ff","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":10009650}],"total_count":3}} WARN [2022-01-10 05:42:30,100] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10872140,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10872140,"total_server_duration_us":8612,"operation_id":"0x4807","timeout_ms":30000,"last_server_duration_us":8612,"total_duration_us":10874695},{"operation_name":"get","last_dispatch_duration_us":10496715,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10496715,"total_server_duration_us":14919,"operation_id":"0x480c","timeout_ms":30000,"last_server_duration_us":14919,"total_duration_us":10498805},{"operation_name":"upsert","last_dispatch_duration_us":11273928,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":11273928,"total_server_duration_us":9041612,"encode_duration_us":17887,"operation_id":"0x47ff","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":10009650},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":21937,"operation_id":"0x4832","timeout_ms":10000,"total_duration_us":10008362},{"operation_name":"upsert","last_dispatch_duration_us":5639180,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8299093,"total_server_duration_us":798908,"encode_duration_us":17031,"operation_id":"0x4931","timeout_ms":10000,"last_server_duration_us":220807,"total_duration_us":10007792},{"operation_name":"upsert","last_dispatch_duration_us":11667899,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11667899,"total_server_duration_us":9221773,"encode_duration_us":16695,"operation_id":"0x47d1","timeout_ms":10000,"last_server_duration_us":9221773,"total_duration_us":10007687},{"operation_name":"upsert","last_dispatch_duration_us":6300819,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10430370,"total_server_duration_us":1654077,"encode_duration_us":16720,"operation_id":"0x48fd","timeout_ms":10000,"last_server_duration_us":32065,"total_duration_us":10007340},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":17024,"operation_id":"0x48f5","timeout_ms":10000,"total_duration_us":10006521},{"operation_name":"upsert","last_dispatch_duration_us":1225830,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":4549618,"total_server_duration_us":388042,"encode_duration_us":17710,"operation_id":"0x48fe","timeout_ms":10000,"last_server_duration_us":185749,"total_duration_us":10005469},{"operation_name":"upsert","last_dispatch_duration_us":9293394,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":11607995,"total_server_duration_us":206419,"encode_duration_us":27339,"operation_id":"0x47ed","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10004878}],"total_count":22}} INFO [2022-01-10 05:42:30,116] jepsen worker 31 - jepsen.util 2641 :info :write [10 31] :RequestTimeoutException INFO [2022-01-10 05:42:30,116] jepsen worker 2 - jepsen.util 2252 :info :write [0 13] :RequestTimeoutException INFO [2022-01-10 05:42:30,117] jepsen worker 2 - jepsen.util 2342 :invoke :write [0 30] INFO [2022-01-10 05:42:30,118] jepsen worker 31 - jepsen.util 2731 :invoke :read [10 nil] INFO [2022-01-10 05:42:30,165] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:30,165] jepsen worker nemesis - couchbase.util Rebalance check count: 303 INFO [2022-01-10 05:42:30,166] jepsen worker nemesis - couchbase.util Rebalance stuck count: 241 INFO [2022-01-10 05:42:30,209] jepsen worker 17 - jepsen.util 2447 :ok :read [5 42] INFO [2022-01-10 05:42:30,209] jepsen worker 17 - jepsen.util 2447 :invoke :write [5 44] INFO [2022-01-10 05:42:30,532] jepsen worker 13 - jepsen.util 2353 :info :write [4 0] :RequestTimeoutException INFO [2022-01-10 05:42:30,533] jepsen worker 13 - jepsen.util 2443 :invoke :read [4 nil] INFO [2022-01-10 05:42:30,554] jepsen worker 13 - jepsen.util 2443 :ok :read [4 3] INFO [2022-01-10 05:42:30,555] jepsen worker 13 - jepsen.util 2443 :invoke :write [4 19] INFO [2022-01-10 05:42:30,795] jepsen worker 22 - jepsen.util 2002 :info :write [7 34] :RequestTimeoutException INFO [2022-01-10 05:42:30,796] jepsen worker 22 - jepsen.util 2092 :invoke :write [7 0] INFO [2022-01-10 05:42:31,002] jepsen worker 76 - jepsen.util 1696 :info :write [25 33] :RequestTimeoutException INFO [2022-01-10 05:42:31,002] jepsen worker 76 - jepsen.util 1786 :invoke :write [25 15] INFO [2022-01-10 05:42:31,011] jepsen worker 75 - jepsen.util 1515 :info :write [25 9] :RequestTimeoutException INFO [2022-01-10 05:42:31,012] jepsen worker 75 - jepsen.util 1605 :invoke :write [25 12] INFO [2022-01-10 05:42:31,171] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:31,171] jepsen worker nemesis - couchbase.util Rebalance check count: 304 INFO [2022-01-10 05:42:31,171] jepsen worker nemesis - couchbase.util Rebalance stuck count: 242 INFO [2022-01-10 05:42:31,266] jepsen worker 78 - jepsen.util 2598 :info :write [26 11] :RequestTimeoutException INFO [2022-01-10 05:42:31,267] jepsen worker 78 - jepsen.util 2688 :invoke :read [26 nil] INFO [2022-01-10 05:42:31,289] jepsen worker 78 - jepsen.util 2688 :ok :read [26 10] INFO [2022-01-10 05:42:31,289] jepsen worker 78 - jepsen.util 2688 :invoke :read [26 nil] INFO [2022-01-10 05:42:31,311] jepsen worker 78 - jepsen.util 2688 :ok :read [26 10] INFO [2022-01-10 05:42:31,311] jepsen worker 78 - jepsen.util 2688 :invoke :read [26 nil] INFO [2022-01-10 05:42:31,338] jepsen worker 78 - jepsen.util 2688 :ok :read [26 10] INFO [2022-01-10 05:42:31,339] jepsen worker 78 - jepsen.util 2688 :invoke :write [26 13] INFO [2022-01-10 05:42:31,379] jepsen worker 82 - jepsen.util 2242 :info :write [27 40] :RequestTimeoutException INFO [2022-01-10 05:42:31,380] jepsen worker 82 - jepsen.util 2332 :invoke :write [27 12] INFO [2022-01-10 05:42:31,452] jepsen worker 27 - jepsen.util 2187 :info :write [9 2] :RequestTimeoutException INFO [2022-01-10 05:42:31,452] jepsen worker 27 - jepsen.util 2277 :invoke :write [9 24] INFO [2022-01-10 05:42:31,487] jepsen worker 59 - jepsen.util 2399 :info :write [19 43] :RequestTimeoutException INFO [2022-01-10 05:42:31,487] jepsen worker 59 - jepsen.util 2489 :invoke :write [19 18] INFO [2022-01-10 05:42:31,521] jepsen worker 67 - jepsen.util 2137 :info :write [22 44] :RequestTimeoutException INFO [2022-01-10 05:42:31,521] jepsen worker 67 - jepsen.util 2227 :invoke :write [22 48] WARN [2022-01-10 05:42:31,537] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14125458,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":14125458,"total_server_duration_us":38,"operation_id":"0x47d5","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":14127376},{"operation_name":"upsert","last_dispatch_duration_us":9432176,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9432176,"total_server_duration_us":23644,"encode_duration_us":16874,"operation_id":"0x4924","timeout_ms":10000,"last_server_duration_us":23644,"total_duration_us":10007320},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":16723,"operation_id":"0x4831","timeout_ms":10000,"total_duration_us":10006970},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":16796,"operation_id":"0x49b0","timeout_ms":10000,"total_duration_us":10006679},{"operation_name":"upsert","last_dispatch_duration_us":52588,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":5829875,"total_server_duration_us":4641519,"encode_duration_us":17934,"operation_id":"0x48ea","timeout_ms":10000,"last_server_duration_us":37963,"total_duration_us":10005945},{"operation_name":"upsert","last_dispatch_duration_us":9253766,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9253766,"total_server_duration_us":19185,"encode_duration_us":16781,"operation_id":"0x492b","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10005872},{"operation_name":"upsert","last_dispatch_duration_us":2386183,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":5140807,"total_server_duration_us":360480,"encode_duration_us":16890,"operation_id":"0x480a","timeout_ms":10000,"last_server_duration_us":191481,"total_duration_us":10001526},{"operation_name":"get","last_dispatch_duration_us":7462480,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7462480,"total_server_duration_us":27,"operation_id":"0x4980","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":7464397}],"total_count":8}} WARN [2022-01-10 05:42:31,538] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17020743,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":21912626,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9336722,"encode_duration_us":18374,"operation_id":"0x4628","timeout_ms":10000,"last_server_duration_us":7975484,"total_duration_us":10003337},{"operation_name":"upsert","last_dispatch_duration_us":11263358,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20544225,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10154022,"encode_duration_us":17387,"operation_id":"0x46d7","timeout_ms":10000,"last_server_duration_us":9391573,"total_duration_us":10010471},{"operation_name":"upsert","last_dispatch_duration_us":11432717,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19318443,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9801035,"encode_duration_us":17131,"operation_id":"0x4716","timeout_ms":10000,"last_server_duration_us":9211071,"total_duration_us":10003596}],"total_count":3}} WARN [2022-01-10 05:42:31,637] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12767192,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20420361,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":10474739,"encode_duration_us":16085,"operation_id":"0x4621","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10003640},{"operation_name":"upsert","last_dispatch_duration_us":9657412,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16728021,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":10240187,"encode_duration_us":19403,"operation_id":"0x4797","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10008700}],"total_count":2}} WARN [2022-01-10 05:42:31,638] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9792800,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11399535,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":10497450,"encode_duration_us":19977,"operation_id":"0x47fb","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10001358},{"operation_name":"upsert","last_dispatch_duration_us":10975443,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13813563,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9808615,"encode_duration_us":16738,"operation_id":"0x47d3","timeout_ms":10000,"last_server_duration_us":9580090,"total_duration_us":10006477},{"operation_name":"upsert","last_dispatch_duration_us":15535474,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16360883,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":8066473,"encode_duration_us":16865,"operation_id":"0x486f","timeout_ms":10000,"last_server_duration_us":7753394,"total_duration_us":10010219}],"total_count":3}} WARN [2022-01-10 05:42:31,638] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":15535474,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":16360883,"total_server_duration_us":8066473,"encode_duration_us":16865,"operation_id":"0x486f","timeout_ms":10000,"last_server_duration_us":7753394,"total_duration_us":10010219},{"operation_name":"upsert","last_dispatch_duration_us":515453,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":3371071,"total_server_duration_us":306115,"encode_duration_us":17071,"operation_id":"0x47e6","timeout_ms":10000,"last_server_duration_us":110022,"total_duration_us":10008493},{"operation_name":"upsert","encode_duration_us":17786,"operation_id":"0x482e","timeout_ms":10000,"total_duration_us":10007865},{"operation_name":"upsert","last_dispatch_duration_us":619272,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":10076854,"total_server_duration_us":1729928,"encode_duration_us":16461,"operation_id":"0x482c","timeout_ms":10000,"last_server_duration_us":200008,"total_duration_us":10007517},{"operation_name":"upsert","last_dispatch_duration_us":33927,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9892872,"total_server_duration_us":143947,"encode_duration_us":17533,"operation_id":"0x48a4","timeout_ms":10000,"last_server_duration_us":21163,"total_duration_us":10004513},{"operation_name":"upsert","last_dispatch_duration_us":392942,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":5464888,"total_server_duration_us":1905597,"encode_duration_us":17355,"operation_id":"0x49b7","timeout_ms":10000,"last_server_duration_us":44168,"total_duration_us":10001836},{"operation_name":"upsert","encode_duration_us":20992,"operation_id":"0x497f","timeout_ms":10000,"total_duration_us":10001387},{"operation_name":"upsert","last_dispatch_duration_us":9792800,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11399535,"total_server_duration_us":10497450,"encode_duration_us":19977,"operation_id":"0x47fb","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10001358},{"operation_name":"upsert","last_dispatch_duration_us":505567,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":8880088,"total_server_duration_us":2734171,"encode_duration_us":16732,"operation_id":"0x48c5","timeout_ms":10000,"last_server_duration_us":170173,"total_duration_us":10001199},{"operation_name":"get","last_dispatch_duration_us":8471712,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8471712,"total_server_duration_us":32,"operation_id":"0x4882","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":8474306}],"total_count":12}} WARN [2022-01-10 05:42:31,642] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":20707,"operation_id":"0x48f0","timeout_ms":10000,"total_duration_us":10010131},{"operation_name":"upsert","encode_duration_us":20805,"operation_id":"0x4892","timeout_ms":10000,"total_duration_us":10008793},{"operation_name":"upsert","last_dispatch_duration_us":275870,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":7254290,"total_server_duration_us":2612521,"encode_duration_us":16609,"operation_id":"0x48ff","timeout_ms":10000,"last_server_duration_us":92740,"total_duration_us":10007586},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":27104,"operation_id":"0x4808","timeout_ms":10000,"total_duration_us":10007567},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":17539,"operation_id":"0x49b9","timeout_ms":10000,"total_duration_us":10007440},{"operation_name":"upsert","last_dispatch_duration_us":88530,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":6175141,"total_server_duration_us":2770955,"encode_duration_us":18692,"operation_id":"0x498a","timeout_ms":10000,"last_server_duration_us":30633,"total_duration_us":10005807},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":17987,"operation_id":"0x4830","timeout_ms":10000,"total_duration_us":10005749},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":17677,"operation_id":"0x4852","timeout_ms":10000,"total_duration_us":10005657},{"operation_name":"upsert","encode_duration_us":19237,"operation_id":"0x49ae","timeout_ms":10000,"total_duration_us":10004821},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16401,"operation_id":"0x48f3","timeout_ms":10000,"total_duration_us":10002863}],"total_count":13}} WARN [2022-01-10 05:42:31,649] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1573622,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1573622,"total_server_duration_us":19966,"encode_duration_us":22455,"operation_id":"0x48f1","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10010414},{"operation_name":"upsert","last_dispatch_duration_us":2332758,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":3903131,"total_server_duration_us":1108706,"encode_duration_us":17056,"operation_id":"0x48f2","timeout_ms":10000,"last_server_duration_us":749312,"total_duration_us":10009784},{"operation_name":"upsert","last_dispatch_duration_us":3321423,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9048325,"total_server_duration_us":976573,"encode_duration_us":16239,"operation_id":"0x48e8","timeout_ms":10000,"last_server_duration_us":23898,"total_duration_us":10007611},{"operation_name":"upsert","last_dispatch_duration_us":9120557,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":17631213,"total_server_duration_us":79594,"encode_duration_us":27418,"operation_id":"0x4809","timeout_ms":10000,"last_server_duration_us":20441,"total_duration_us":10007268},{"operation_name":"upsert","last_dispatch_duration_us":33057,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":791235,"total_server_duration_us":442668,"encode_duration_us":21071,"operation_id":"0x4982","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10006907},{"operation_name":"upsert","last_dispatch_duration_us":9121875,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10039581,"total_server_duration_us":250614,"encode_duration_us":18850,"operation_id":"0x497b","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10006421},{"operation_name":"upsert","last_dispatch_duration_us":64174,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10041731,"total_server_duration_us":82454,"encode_duration_us":21596,"operation_id":"0x4983","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10006366},{"operation_name":"upsert","last_dispatch_duration_us":981059,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":5513897,"total_server_duration_us":2900001,"encode_duration_us":17531,"operation_id":"0x4933","timeout_ms":10000,"last_server_duration_us":49951,"total_duration_us":10006299},{"operation_name":"upsert","last_dispatch_duration_us":11943842,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":14028768,"total_server_duration_us":10530009,"encode_duration_us":16587,"operation_id":"0x4900","timeout_ms":10000,"last_server_duration_us":9653104,"total_duration_us":10004843},{"operation_name":"upsert","last_dispatch_duration_us":1412471,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9133609,"total_server_duration_us":1968679,"encode_duration_us":16729,"operation_id":"0x4901","timeout_ms":10000,"last_server_duration_us":212770,"total_duration_us":10003547}],"total_count":19}} WARN [2022-01-10 05:42:31,743] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16069661,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16069661,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":4196708,"encode_duration_us":17136,"operation_id":"0x47c2","timeout_ms":10000,"last_server_duration_us":4196708,"total_duration_us":10008430},{"operation_name":"upsert","last_dispatch_duration_us":12688244,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17469639,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10518551,"encode_duration_us":16645,"operation_id":"0x46b6","timeout_ms":10000,"last_server_duration_us":9227126,"total_duration_us":10008973},{"operation_name":"upsert","last_dispatch_duration_us":25410188,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":25410188,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9040551,"encode_duration_us":18666,"operation_id":"0x464f","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10009457}],"total_count":3}} INFO [2022-01-10 05:42:31,787] jepsen worker 54 - jepsen.util 2304 :info :write [18 25] :RequestTimeoutException INFO [2022-01-10 05:42:31,788] jepsen worker 54 - jepsen.util 2394 :invoke :write [18 15] WARN [2022-01-10 05:42:31,838] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14182220,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":14182220,"total_server_duration_us":10692,"operation_id":"0x4741","timeout_ms":30000,"last_server_duration_us":10692,"total_duration_us":14184099},{"operation_name":"upsert","last_dispatch_duration_us":9028482,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10313164,"total_server_duration_us":99291,"encode_duration_us":18198,"operation_id":"0x48c7","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10010348},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":16450,"operation_id":"0x4903","timeout_ms":10000,"total_duration_us":10010329},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","encode_duration_us":16865,"operation_id":"0x495c","timeout_ms":10000,"total_duration_us":10009231},{"operation_name":"upsert","last_dispatch_duration_us":9026903,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":11613646,"total_server_duration_us":535423,"encode_duration_us":16379,"operation_id":"0x489c","timeout_ms":10000,"last_server_duration_us":22059,"total_duration_us":10009198},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":16935,"operation_id":"0x49b8","timeout_ms":10000,"total_duration_us":10008700},{"operation_name":"upsert","last_dispatch_duration_us":645133,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":645133,"total_server_duration_us":18646,"encode_duration_us":16730,"operation_id":"0x48f4","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10008530},{"operation_name":"upsert","last_dispatch_duration_us":9032227,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9681617,"total_server_duration_us":39148,"encode_duration_us":16955,"operation_id":"0x48fc","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10006083},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","encode_duration_us":20261,"operation_id":"0x497e","timeout_ms":10000,"total_duration_us":10005783},{"operation_name":"upsert","last_dispatch_duration_us":10574626,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10574626,"total_server_duration_us":18953,"encode_duration_us":16407,"operation_id":"0x4806","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10005114}],"total_count":12}} WARN [2022-01-10 05:42:31,849] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11943842,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14028768,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":10530009,"encode_duration_us":16587,"operation_id":"0x4900","timeout_ms":10000,"last_server_duration_us":9653104,"total_duration_us":10004843}],"total_count":1}} INFO [2022-01-10 05:42:31,857] jepsen worker 38 - jepsen.util 2468 :info :write [12 36] :RequestTimeoutException INFO [2022-01-10 05:42:31,858] jepsen worker 38 - jepsen.util 2558 :invoke :write [12 24] INFO [2022-01-10 05:42:32,177] jepsen worker 16 - jepsen.util 2446 :ok :read [5 42] INFO [2022-01-10 05:42:32,177] jepsen worker 16 - jepsen.util 2446 :invoke :write [5 29] INFO [2022-01-10 05:42:32,177] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:32,177] jepsen worker nemesis - couchbase.util Rebalance check count: 305 INFO [2022-01-10 05:42:32,177] jepsen worker nemesis - couchbase.util Rebalance stuck count: 243 INFO [2022-01-10 05:42:32,184] jepsen worker 15 - jepsen.util 1995 :ok :read [5 42] INFO [2022-01-10 05:42:32,184] jepsen worker 15 - jepsen.util 1995 :invoke :write [5 17] INFO [2022-01-10 05:42:32,195] jepsen worker 49 - jepsen.util 2209 :info :write [16 35] :RequestTimeoutException INFO [2022-01-10 05:42:32,196] jepsen worker 49 - jepsen.util 2299 :invoke :read [16 nil] INFO [2022-01-10 05:42:32,215] jepsen worker 29 - jepsen.util 2369 :ok :read [9 22] INFO [2022-01-10 05:42:32,216] jepsen worker 29 - jepsen.util 2369 :invoke :read [9 nil] INFO [2022-01-10 05:42:32,329] jepsen worker 71 - jepsen.util 1871 :ok :read [23 26] INFO [2022-01-10 05:42:32,330] jepsen worker 71 - jepsen.util 1871 :invoke :write [23 4] INFO [2022-01-10 05:42:32,336] jepsen worker 6 - jepsen.util 2256 :info :write [2 34] :RequestTimeoutException INFO [2022-01-10 05:42:32,336] jepsen worker 6 - jepsen.util 2346 :invoke :read [2 nil] INFO [2022-01-10 05:42:32,369] jepsen worker 4 - jepsen.util 2614 :ok :read [1 41] INFO [2022-01-10 05:42:32,371] jepsen worker 4 - jepsen.util 2614 :invoke :read [1 nil] INFO [2022-01-10 05:42:32,378] jepsen worker 31 - jepsen.util 2731 :ok :read [10 37] INFO [2022-01-10 05:42:32,379] jepsen worker 31 - jepsen.util 2731 :invoke :write [10 7] INFO [2022-01-10 05:42:32,418] jepsen worker 69 - jepsen.util 2409 :ok :read [23 26] INFO [2022-01-10 05:42:32,418] jepsen worker 69 - jepsen.util 2409 :invoke :read [23 nil] INFO [2022-01-10 05:42:32,426] jepsen worker 88 - jepsen.util 2248 :ok :read [29 0] INFO [2022-01-10 05:42:32,426] jepsen worker 88 - jepsen.util 2248 :invoke :read [29 nil] INFO [2022-01-10 05:42:32,487] jepsen worker 69 - jepsen.util 2409 :ok :read [23 26] INFO [2022-01-10 05:42:32,487] jepsen worker 69 - jepsen.util 2409 :invoke :write [23 8] INFO [2022-01-10 05:42:32,496] jepsen worker 88 - jepsen.util 2248 :ok :read [29 0] INFO [2022-01-10 05:42:32,497] jepsen worker 88 - jepsen.util 2248 :invoke :read [29 nil] INFO [2022-01-10 05:42:32,498] jepsen worker 4 - jepsen.util 2614 :ok :read [1 41] INFO [2022-01-10 05:42:32,498] jepsen worker 4 - jepsen.util 2614 :invoke :write [1 17] INFO [2022-01-10 05:42:32,533] jepsen worker 20 - jepsen.util 2000 :ok :read [6 25] INFO [2022-01-10 05:42:32,535] jepsen worker 20 - jepsen.util 2000 :invoke :read [6 nil] INFO [2022-01-10 05:42:32,559] jepsen worker 88 - jepsen.util 2248 :ok :read [29 0] INFO [2022-01-10 05:42:32,559] jepsen worker 88 - jepsen.util 2248 :invoke :write [29 27] INFO [2022-01-10 05:42:32,571] jepsen worker 57 - jepsen.util 2667 :info :write [19 31] :RequestTimeoutException INFO [2022-01-10 05:42:32,572] jepsen worker 57 - jepsen.util 2757 :invoke :write [19 9] INFO [2022-01-10 05:42:32,696] jepsen worker 9 - jepsen.util 2709 :info :write [3 38] :RequestTimeoutException INFO [2022-01-10 05:42:32,697] jepsen worker 9 - jepsen.util 2799 :invoke :write [3 8] INFO [2022-01-10 05:42:32,736] jepsen worker 8 - jepsen.util 2618 :ok :read [2 14] INFO [2022-01-10 05:42:32,737] jepsen worker 8 - jepsen.util 2618 :invoke :read [2 nil] INFO [2022-01-10 05:42:32,846] jepsen worker 61 - jepsen.util 2311 :info :write [20 26] :RequestTimeoutException INFO [2022-01-10 05:42:32,847] jepsen worker 61 - jepsen.util 2401 :invoke :read [20 nil] INFO [2022-01-10 05:42:32,921] jepsen worker 51 - jepsen.util 2571 :info :write [17 14] :RequestTimeoutException INFO [2022-01-10 05:42:32,922] jepsen worker 51 - jepsen.util 2661 :invoke :write [17 21] INFO [2022-01-10 05:42:33,065] jepsen worker 45 - jepsen.util 2655 :info :write [15 41] :RequestTimeoutException INFO [2022-01-10 05:42:33,066] jepsen worker 45 - jepsen.util 2745 :invoke :write [15 20] INFO [2022-01-10 05:42:33,090] jepsen worker 11 - jepsen.util 2261 :info :write [3 16] :RequestTimeoutException INFO [2022-01-10 05:42:33,091] jepsen worker 11 - jepsen.util 2351 :invoke :read [3 nil] INFO [2022-01-10 05:42:33,101] jepsen worker 55 - jepsen.util 2575 :info :write [18 37] :RequestTimeoutException INFO [2022-01-10 05:42:33,102] jepsen worker 55 - jepsen.util 2665 :invoke :write [18 17] INFO [2022-01-10 05:42:33,145] jepsen worker 58 - jepsen.util 2218 :info :write [19 30] :RequestTimeoutException INFO [2022-01-10 05:42:33,145] jepsen worker 58 - jepsen.util 2308 :invoke :write [19 1] INFO [2022-01-10 05:42:33,153] jepsen worker 44 - jepsen.util 2294 :ok :read [14 14] INFO [2022-01-10 05:42:33,153] jepsen worker 44 - jepsen.util 2294 :invoke :write [14 11] INFO [2022-01-10 05:42:33,184] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:33,184] jepsen worker nemesis - couchbase.util Rebalance check count: 306 INFO [2022-01-10 05:42:33,185] jepsen worker nemesis - couchbase.util Rebalance stuck count: 244 INFO [2022-01-10 05:42:33,205] jepsen worker 28 - jepsen.util 2188 :info :write [9 24] :RequestTimeoutException INFO [2022-01-10 05:42:33,205] jepsen worker 28 - jepsen.util 2278 :invoke :read [9 nil] INFO [2022-01-10 05:42:33,276] jepsen worker 30 - jepsen.util 2100 :info :write [10 8] :RequestTimeoutException INFO [2022-01-10 05:42:33,277] jepsen worker 30 - jepsen.util 2190 :invoke :write [10 49] INFO [2022-01-10 05:42:33,336] jepsen worker 72 - jepsen.util 2412 :info :write [24 36] :RequestTimeoutException INFO [2022-01-10 05:42:33,337] jepsen worker 72 - jepsen.util 2502 :invoke :write [24 23] INFO [2022-01-10 05:42:33,342] jepsen worker 73 - jepsen.util 2143 :ok :read [24 18] INFO [2022-01-10 05:42:33,343] jepsen worker 73 - jepsen.util 2143 :invoke :write [24 5] INFO [2022-01-10 05:42:33,349] jepsen worker 6 - jepsen.util 2346 :ok :read [2 14] INFO [2022-01-10 05:42:33,350] jepsen worker 6 - jepsen.util 2346 :invoke :read [2 nil] INFO [2022-01-10 05:42:33,412] jepsen worker 20 - jepsen.util 2000 :ok :read [6 25] INFO [2022-01-10 05:42:33,413] jepsen worker 20 - jepsen.util 2000 :invoke :write [6 15] INFO [2022-01-10 05:42:33,432] jepsen worker 8 - jepsen.util 2618 :ok :read [2 14] INFO [2022-01-10 05:42:33,433] jepsen worker 8 - jepsen.util 2618 :invoke :write [2 1] INFO [2022-01-10 05:42:33,751] jepsen worker 0 - jepsen.util 2250 :info :write [0 12] :RequestTimeoutException INFO [2022-01-10 05:42:33,751] jepsen worker 0 - jepsen.util 2340 :invoke :write [0 39] INFO [2022-01-10 05:42:33,784] jepsen worker 39 - jepsen.util 2649 :info :write [13 7] :RequestTimeoutException INFO [2022-01-10 05:42:33,785] jepsen worker 39 - jepsen.util 2739 :invoke :write [13 26] INFO [2022-01-10 05:42:33,856] jepsen worker 52 - jepsen.util 2842 :info :write [17 37] :RequestTimeoutException INFO [2022-01-10 05:42:33,856] jepsen worker 52 - jepsen.util 2932 :invoke :read [17 nil] INFO [2022-01-10 05:42:33,881] jepsen worker 52 - jepsen.util 2932 :ok :read [17 41] INFO [2022-01-10 05:42:33,881] jepsen worker 52 - jepsen.util 2932 :invoke :read [17 nil] INFO [2022-01-10 05:42:33,912] jepsen worker 52 - jepsen.util 2932 :ok :read [17 41] INFO [2022-01-10 05:42:33,912] jepsen worker 52 - jepsen.util 2932 :invoke :write [17 8] INFO [2022-01-10 05:42:34,035] jepsen worker 5 - jepsen.util 2435 :ok :read [1 10] INFO [2022-01-10 05:42:34,036] jepsen worker 5 - jepsen.util 2435 :invoke :write [1 44] INFO [2022-01-10 05:42:34,153] jepsen worker 3 - jepsen.util 2433 :ok :read [1 10] INFO [2022-01-10 05:42:34,153] jepsen worker 3 - jepsen.util 2433 :invoke :write [1 18] INFO [2022-01-10 05:42:34,161] jepsen worker 11 - jepsen.util 2351 :ok :read [3 31] INFO [2022-01-10 05:42:34,161] jepsen worker 11 - jepsen.util 2351 :invoke :write [3 11] INFO [2022-01-10 05:42:34,197] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:34,197] jepsen worker nemesis - couchbase.util Rebalance check count: 307 INFO [2022-01-10 05:42:34,197] jepsen worker nemesis - couchbase.util Rebalance stuck count: 245 INFO [2022-01-10 05:42:34,303] jepsen worker 6 - jepsen.util 2346 :ok :read [2 14] INFO [2022-01-10 05:42:34,304] jepsen worker 6 - jepsen.util 2346 :invoke :write [2 25] INFO [2022-01-10 05:42:34,700] jepsen worker 37 - jepsen.util 2377 :info :write [12 35] :RequestTimeoutException INFO [2022-01-10 05:42:34,701] jepsen worker 37 - jepsen.util 2467 :invoke :write [12 34] INFO [2022-01-10 05:42:34,741] jepsen worker 87 - jepsen.util 2517 :ok :read [29 0] INFO [2022-01-10 05:42:34,742] jepsen worker 87 - jepsen.util 2517 :invoke :read [29 nil] INFO [2022-01-10 05:42:34,751] jepsen worker 70 - jepsen.util 2590 :ok :read [23 26] INFO [2022-01-10 05:42:34,751] jepsen worker 70 - jepsen.util 2590 :invoke :read [23 nil] INFO [2022-01-10 05:42:34,787] jepsen worker 56 - jepsen.util 2486 :ok :read [18 14] INFO [2022-01-10 05:42:34,787] jepsen worker 56 - jepsen.util 2486 :invoke :write [18 25] INFO [2022-01-10 05:42:35,102] jepsen worker 60 - jepsen.util 2670 :info :write [20 49] :RequestTimeoutException INFO [2022-01-10 05:42:35,102] jepsen worker 60 - jepsen.util 2760 :invoke :read [20 nil] INFO [2022-01-10 05:42:35,205] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:35,205] jepsen worker nemesis - couchbase.util Rebalance check count: 308 INFO [2022-01-10 05:42:35,205] jepsen worker nemesis - couchbase.util Rebalance stuck count: 246 INFO [2022-01-10 05:42:35,266] jepsen worker 89 - jepsen.util 2699 :info :write [29 16] :RequestTimeoutException INFO [2022-01-10 05:42:35,267] jepsen worker 89 - jepsen.util 2789 :invoke :read [29 nil] INFO [2022-01-10 05:42:35,396] jepsen worker 68 - jepsen.util 2408 :info :write [22 27] :RequestTimeoutException INFO [2022-01-10 05:42:35,396] jepsen worker 68 - jepsen.util 2498 :invoke :read [22 nil] INFO [2022-01-10 05:42:35,741] jepsen worker 66 - jepsen.util 2046 :info :write [22 0] :RequestTimeoutException INFO [2022-01-10 05:42:35,742] jepsen worker 66 - jepsen.util 2136 :invoke :read [22 nil] INFO [2022-01-10 05:42:35,957] jepsen worker 43 - jepsen.util 2113 :ok :read [14 14] INFO [2022-01-10 05:42:35,958] jepsen worker 43 - jepsen.util 2113 :invoke :write [14 41] INFO [2022-01-10 05:42:36,073] jepsen worker 42 - jepsen.util 2202 :ok :read [14 14] INFO [2022-01-10 05:42:36,073] jepsen worker 42 - jepsen.util 2202 :invoke :write [14 39] INFO [2022-01-10 05:42:36,082] jepsen worker 24 - jepsen.util 2004 :ok :read [8 27] INFO [2022-01-10 05:42:36,083] jepsen worker 24 - jepsen.util 2004 :invoke :write [8 8] INFO [2022-01-10 05:42:36,212] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:36,212] jepsen worker nemesis - couchbase.util Rebalance check count: 309 INFO [2022-01-10 05:42:36,212] jepsen worker nemesis - couchbase.util Rebalance stuck count: 247 INFO [2022-01-10 05:42:36,365] jepsen worker 12 - jepsen.util 2352 :info :write [4 9] :RequestTimeoutException INFO [2022-01-10 05:42:36,365] jepsen worker 12 - jepsen.util 2442 :invoke :read [4 nil] INFO [2022-01-10 05:42:36,393] jepsen worker 12 - jepsen.util 2442 :ok :read [4 3] INFO [2022-01-10 05:42:36,394] jepsen worker 12 - jepsen.util 2442 :invoke :read [4 nil] INFO [2022-01-10 05:42:36,427] jepsen worker 12 - jepsen.util 2442 :ok :read [4 3] INFO [2022-01-10 05:42:36,428] jepsen worker 12 - jepsen.util 2442 :invoke :write [4 19] INFO [2022-01-10 05:42:36,767] jepsen worker 81 - jepsen.util 2601 :ok :read [27 30] INFO [2022-01-10 05:42:36,768] jepsen worker 81 - jepsen.util 2601 :invoke :write [27 17] INFO [2022-01-10 05:42:36,870] jepsen worker 18 - jepsen.util 1998 :info :write [6 35] :RequestTimeoutException INFO [2022-01-10 05:42:36,871] jepsen worker 18 - jepsen.util 2088 :invoke :write [6 1] INFO [2022-01-10 05:42:36,992] jepsen worker 14 - jepsen.util 2714 :info :write [4 14] :RequestTimeoutException INFO [2022-01-10 05:42:36,992] jepsen worker 14 - jepsen.util 2804 :invoke :read [4 nil] INFO [2022-01-10 05:42:37,028] jepsen worker 14 - jepsen.util 2804 :ok :read [4 3] INFO [2022-01-10 05:42:37,029] jepsen worker 14 - jepsen.util 2804 :invoke :read [4 nil] INFO [2022-01-10 05:42:37,049] jepsen worker 14 - jepsen.util 2804 :ok :read [4 3] INFO [2022-01-10 05:42:37,049] jepsen worker 14 - jepsen.util 2804 :invoke :write [4 49] INFO [2022-01-10 05:42:37,084] jepsen worker 33 - jepsen.util 2553 :info :write [11 21] :RequestTimeoutException INFO [2022-01-10 05:42:37,085] jepsen worker 77 - jepsen.util 2687 :info :write [25 48] :RequestTimeoutException INFO [2022-01-10 05:42:37,085] jepsen worker 33 - jepsen.util 2643 :invoke :write [11 16] INFO [2022-01-10 05:42:37,086] jepsen worker 77 - jepsen.util 2777 :invoke :read [25 nil] INFO [2022-01-10 05:42:37,217] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:37,217] jepsen worker nemesis - couchbase.util Rebalance check count: 310 INFO [2022-01-10 05:42:37,218] jepsen worker nemesis - couchbase.util Rebalance stuck count: 248 INFO [2022-01-10 05:42:37,226] jepsen worker 10 - jepsen.util 2530 :info :write [3 45] :RequestTimeoutException INFO [2022-01-10 05:42:37,226] jepsen worker 10 - jepsen.util 2620 :invoke :write [3 48] INFO [2022-01-10 05:42:37,241] jepsen worker 62 - jepsen.util 2492 :info :write [20 8] :RequestTimeoutException INFO [2022-01-10 05:42:37,241] jepsen worker 62 - jepsen.util 2582 :invoke :read [20 nil] INFO [2022-01-10 05:42:37,245] jepsen worker 26 - jepsen.util 1916 :info :write [8 6] :RequestTimeoutException INFO [2022-01-10 05:42:37,246] jepsen worker 26 - jepsen.util 2006 :invoke :read [8 nil] INFO [2022-01-10 05:42:37,406] jepsen worker 80 - jepsen.util 2690 :info :write [26 8] :RequestTimeoutException INFO [2022-01-10 05:42:37,407] jepsen worker 80 - jepsen.util 2780 :invoke :read [26 nil] INFO [2022-01-10 05:42:37,441] jepsen worker 80 - jepsen.util 2780 :ok :read [26 10] INFO [2022-01-10 05:42:37,441] jepsen worker 80 - jepsen.util 2780 :invoke :write [26 33] INFO [2022-01-10 05:42:37,591] jepsen worker 48 - jepsen.util 2028 :info :write [16 47] :RequestTimeoutException INFO [2022-01-10 05:42:37,591] jepsen worker 48 - jepsen.util 2118 :invoke :read [16 nil] INFO [2022-01-10 05:42:37,787] jepsen worker 64 - jepsen.util 2314 :ok :read [21 7] INFO [2022-01-10 05:42:37,787] jepsen worker 64 - jepsen.util 2314 :invoke :read [21 nil] INFO [2022-01-10 05:42:38,100] jepsen worker 47 - jepsen.util 2297 :ok :read [15 23] INFO [2022-01-10 05:42:38,101] jepsen worker 47 - jepsen.util 2297 :invoke :read [15 nil] INFO [2022-01-10 05:42:38,225] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3508771929824561}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3411764705882353}} INFO [2022-01-10 05:42:38,225] jepsen worker nemesis - couchbase.util Rebalance check count: 311 INFO [2022-01-10 05:42:38,225] jepsen worker nemesis - couchbase.util Rebalance stuck count: 249 INFO [2022-01-10 05:42:38,234] jepsen worker 83 - jepsen.util 2693 :info :write [27 47] :RequestTimeoutException INFO [2022-01-10 05:42:38,235] jepsen worker 83 - jepsen.util 2783 :invoke :write [27 14] INFO [2022-01-10 05:42:38,267] jepsen worker 19 - jepsen.util 2449 :info :write [6 35] :RequestTimeoutException INFO [2022-01-10 05:42:38,268] jepsen worker 19 - jepsen.util 2539 :invoke :read [6 nil] INFO [2022-01-10 05:42:38,312] jepsen worker 74 - jepsen.util 2864 :info :write [24 27] :RequestTimeoutException INFO [2022-01-10 05:42:38,313] jepsen worker 74 - jepsen.util 2954 :invoke :read [24 nil] INFO [2022-01-10 05:42:38,320] jepsen worker 62 - jepsen.util 2582 :ok :read [20 :nil] INFO [2022-01-10 05:42:38,321] jepsen worker 62 - jepsen.util 2582 :invoke :read [20 nil] INFO [2022-01-10 05:42:38,333] jepsen worker 64 - jepsen.util 2314 :ok :read [21 7] INFO [2022-01-10 05:42:38,334] jepsen worker 64 - jepsen.util 2314 :invoke :write [21 17] INFO [2022-01-10 05:42:38,392] jepsen worker 32 - jepsen.util 2372 :info :write [10 38] :RequestTimeoutException INFO [2022-01-10 05:42:38,392] jepsen worker 32 - jepsen.util 2462 :invoke :write [10 45] INFO [2022-01-10 05:42:38,402] jepsen worker 36 - jepsen.util 2016 :info :write [12 32] :RequestTimeoutException INFO [2022-01-10 05:42:38,403] jepsen worker 36 - jepsen.util 2106 :invoke :read [12 nil] INFO [2022-01-10 05:42:38,801] jepsen worker 53 - jepsen.util 2753 :info :write [17 27] :RequestTimeoutException INFO [2022-01-10 05:42:38,802] jepsen worker 53 - jepsen.util 2843 :invoke :read [17 nil] INFO [2022-01-10 05:42:38,828] jepsen worker 53 - jepsen.util 2843 :ok :read [17 41] INFO [2022-01-10 05:42:38,828] jepsen worker 53 - jepsen.util 2843 :invoke :read [17 nil] INFO [2022-01-10 05:42:38,854] jepsen worker 53 - jepsen.util 2843 :ok :read [17 41] INFO [2022-01-10 05:42:38,854] jepsen worker 53 - jepsen.util 2843 :invoke :read [17 nil] INFO [2022-01-10 05:42:38,878] jepsen worker 53 - jepsen.util 2843 :ok :read [17 41] INFO [2022-01-10 05:42:38,879] jepsen worker 53 - jepsen.util 2843 :invoke :write [17 32] INFO [2022-01-10 05:42:39,231] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:39,231] jepsen worker nemesis - couchbase.util Rebalance check count: 312 INFO [2022-01-10 05:42:39,231] jepsen worker nemesis - couchbase.util Rebalance stuck count: 249 INFO [2022-01-10 05:42:39,293] jepsen worker 17 - jepsen.util 2447 :info :write [5 44] :SyncWriteAmbiguous INFO [2022-01-10 05:42:39,293] jepsen worker 17 - jepsen.util 2537 :invoke :read [5 nil] INFO [2022-01-10 05:42:39,814] jepsen worker 79 - jepsen.util 2689 :info :write [26 44] :RequestTimeoutException INFO [2022-01-10 05:42:39,815] jepsen worker 79 - jepsen.util 2779 :invoke :read [26 nil] INFO [2022-01-10 05:42:39,840] jepsen worker 79 - jepsen.util 2779 :ok :read [26 10] INFO [2022-01-10 05:42:39,840] jepsen worker 79 - jepsen.util 2779 :invoke :read [26 nil] INFO [2022-01-10 05:42:39,873] jepsen worker 79 - jepsen.util 2779 :ok :read [26 10] INFO [2022-01-10 05:42:39,873] jepsen worker 79 - jepsen.util 2779 :invoke :read [26 nil] INFO [2022-01-10 05:42:39,896] jepsen worker 79 - jepsen.util 2779 :ok :read [26 10] INFO [2022-01-10 05:42:39,896] jepsen worker 79 - jepsen.util 2779 :invoke :write [26 41] INFO [2022-01-10 05:42:39,940] jepsen worker 50 - jepsen.util 2480 :info :write [16 20] :RequestTimeoutException INFO [2022-01-10 05:42:39,941] jepsen worker 50 - jepsen.util 2570 :invoke :read [16 nil] INFO [2022-01-10 05:42:39,986] jepsen worker 25 - jepsen.util 2185 :info :write [8 47] :RequestTimeoutException INFO [2022-01-10 05:42:39,987] jepsen worker 25 - jepsen.util 2275 :invoke :read [8 nil] INFO [2022-01-10 05:42:39,991] jepsen worker 21 - jepsen.util 2361 :info :write [7 28] :RequestTimeoutException INFO [2022-01-10 05:42:39,991] jepsen worker 21 - jepsen.util 2451 :invoke :write [7 4] WARN [2022-01-10 05:42:40,013] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18419764,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18419764,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9037366,"encode_duration_us":28997,"operation_id":"0x484b","timeout_ms":10000,"last_server_duration_us":9037366,"total_duration_us":10000601},{"operation_name":"upsert","last_dispatch_duration_us":19301069,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":19301069,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9035244,"encode_duration_us":21937,"operation_id":"0x4832","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10008362}],"total_count":2}} INFO [2022-01-10 05:42:40,098] jepsen worker 63 - jepsen.util 2223 :ok :read [21 7] INFO [2022-01-10 05:42:40,099] jepsen worker 63 - jepsen.util 2223 :invoke :write [21 18] WARN [2022-01-10 05:42:40,114] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17814354,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":17814354,"total_server_duration_us":38,"operation_id":"0x486a","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":17816205},{"operation_name":"get","last_dispatch_duration_us":16281569,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":16281569,"total_server_duration_us":27,"operation_id":"0x48be","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":16283540},{"operation_name":"get","last_dispatch_duration_us":11588470,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11588470,"total_server_duration_us":27,"operation_id":"0x498b","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":11590329},{"operation_name":"get","last_dispatch_duration_us":10653928,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10653928,"total_server_duration_us":43,"operation_id":"0x49ce","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":10655282},{"operation_name":"upsert","last_dispatch_duration_us":31783,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":2801092,"total_server_duration_us":1807769,"encode_duration_us":16673,"operation_id":"0x4a9a","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10010545},{"operation_name":"upsert","last_dispatch_duration_us":8379473,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8379473,"total_server_duration_us":221683,"encode_duration_us":18043,"operation_id":"0x4acb","timeout_ms":10000,"last_server_duration_us":221683,"total_duration_us":10009934},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":17366,"operation_id":"0x4a1c","timeout_ms":10000,"total_duration_us":10009585},{"operation_name":"upsert","encode_duration_us":17167,"operation_id":"0x4af7","timeout_ms":10000,"total_duration_us":10004808},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":19696,"operation_id":"0x49e7","timeout_ms":10000,"total_duration_us":10004152},{"operation_name":"upsert","last_dispatch_duration_us":81962,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":996806,"total_server_duration_us":560730,"encode_duration_us":16739,"operation_id":"0x4b47","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10004085}],"total_count":16}} INFO [2022-01-10 05:42:40,145] jepsen worker 2 - jepsen.util 2342 :info :write [0 30] :RequestTimeoutException INFO [2022-01-10 05:42:40,157] jepsen worker 2 - jepsen.util 2432 :invoke :write [0 44] INFO [2022-01-10 05:42:40,237] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:40,237] jepsen worker nemesis - couchbase.util Rebalance check count: 313 INFO [2022-01-10 05:42:40,237] jepsen worker nemesis - couchbase.util Rebalance stuck count: 250 INFO [2022-01-10 05:42:40,579] jepsen worker 13 - jepsen.util 2443 :info :write [4 19] :RequestTimeoutException INFO [2022-01-10 05:42:40,580] jepsen worker 13 - jepsen.util 2533 :invoke :write [4 9] INFO [2022-01-10 05:42:40,822] jepsen worker 22 - jepsen.util 2092 :info :write [7 0] :RequestTimeoutException INFO [2022-01-10 05:42:40,822] jepsen worker 22 - jepsen.util 2182 :invoke :read [7 nil] INFO [2022-01-10 05:42:40,932] jepsen worker 40 - jepsen.util 1840 :ok :read [13 15] INFO [2022-01-10 05:42:40,933] jepsen worker 40 - jepsen.util 1840 :invoke :write [13 20] INFO [2022-01-10 05:42:40,941] jepsen worker 1 - jepsen.util 2431 :ok :read [0 46] INFO [2022-01-10 05:42:40,941] jepsen worker 1 - jepsen.util 2431 :invoke :read [0 nil] INFO [2022-01-10 05:42:41,027] jepsen worker 76 - jepsen.util 1786 :info :write [25 15] :RequestTimeoutException INFO [2022-01-10 05:42:41,028] jepsen worker 76 - jepsen.util 1876 :invoke :read [25 nil] INFO [2022-01-10 05:42:41,041] jepsen worker 75 - jepsen.util 1605 :info :write [25 12] :RequestTimeoutException INFO [2022-01-10 05:42:41,042] jepsen worker 75 - jepsen.util 1695 :invoke :write [25 49] INFO [2022-01-10 05:42:41,055] jepsen worker 66 - jepsen.util 2136 :ok :read [22 41] INFO [2022-01-10 05:42:41,055] jepsen worker 66 - jepsen.util 2136 :invoke :write [22 13] INFO [2022-01-10 05:42:41,064] jepsen worker 1 - jepsen.util 2431 :ok :read [0 46] INFO [2022-01-10 05:42:41,065] jepsen worker 1 - jepsen.util 2431 :invoke :write [0 6] INFO [2022-01-10 05:42:41,247] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:41,247] jepsen worker nemesis - couchbase.util Rebalance check count: 314 INFO [2022-01-10 05:42:41,247] jepsen worker nemesis - couchbase.util Rebalance stuck count: 251 INFO [2022-01-10 05:42:41,299] jepsen worker 84 - jepsen.util 2244 :ok :read [28 45] INFO [2022-01-10 05:42:41,300] jepsen worker 84 - jepsen.util 2244 :invoke :read [28 nil] INFO [2022-01-10 05:42:41,307] jepsen worker 86 - jepsen.util 1796 :ok :read [28 45] INFO [2022-01-10 05:42:41,308] jepsen worker 86 - jepsen.util 1796 :invoke :write [28 18] INFO [2022-01-10 05:42:41,364] jepsen worker 78 - jepsen.util 2688 :info :write [26 13] :RequestTimeoutException INFO [2022-01-10 05:42:41,365] jepsen worker 78 - jepsen.util 2778 :invoke :read [26 nil] INFO [2022-01-10 05:42:41,406] jepsen worker 82 - jepsen.util 2332 :info :write [27 12] :RequestTimeoutException INFO [2022-01-10 05:42:41,406] jepsen worker 82 - jepsen.util 2422 :invoke :read [27 nil] INFO [2022-01-10 05:42:41,471] jepsen worker 71 - jepsen.util 1871 :info :write [23 4] :SyncWriteAmbiguous INFO [2022-01-10 05:42:41,472] jepsen worker 71 - jepsen.util 1961 :invoke :write [23 6] INFO [2022-01-10 05:42:41,480] jepsen worker 27 - jepsen.util 2277 :info :write [9 24] :RequestTimeoutException INFO [2022-01-10 05:42:41,482] jepsen worker 27 - jepsen.util 2367 :invoke :read [9 nil] INFO [2022-01-10 05:42:41,511] jepsen worker 59 - jepsen.util 2489 :info :write [19 18] :RequestTimeoutException INFO [2022-01-10 05:42:41,512] jepsen worker 59 - jepsen.util 2579 :invoke :read [19 nil] INFO [2022-01-10 05:42:41,515] jepsen worker 84 - jepsen.util 2244 :ok :read [28 45] INFO [2022-01-10 05:42:41,516] jepsen worker 84 - jepsen.util 2244 :invoke :write [28 0] INFO [2022-01-10 05:42:41,547] jepsen worker 67 - jepsen.util 2227 :info :write [22 48] :RequestTimeoutException INFO [2022-01-10 05:42:41,548] jepsen worker 67 - jepsen.util 2317 :invoke :read [22 nil] INFO [2022-01-10 05:42:41,579] jepsen worker 29 - jepsen.util 2369 :ok :read [9 22] INFO [2022-01-10 05:42:41,580] jepsen worker 29 - jepsen.util 2369 :invoke :read [9 nil] WARN [2022-01-10 05:42:41,609] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":20283115,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":20283115,"total_server_duration_us":43,"operation_id":"0x486c","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":20284478},{"operation_name":"get","last_dispatch_duration_us":13016511,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":13016511,"total_server_duration_us":32,"operation_id":"0x4abd","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":13018419},{"operation_name":"get","last_dispatch_duration_us":12739162,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":12739162,"total_server_duration_us":32,"operation_id":"0x49e0","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":12740693},{"operation_name":"upsert","encode_duration_us":16286,"operation_id":"0x4a2e","timeout_ms":10000,"total_duration_us":10008977},{"operation_name":"upsert","last_dispatch_duration_us":13395791,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":13395791,"total_server_duration_us":146617,"encode_duration_us":16756,"operation_id":"0x4aca","timeout_ms":10000,"last_server_duration_us":146617,"total_duration_us":10007639},{"operation_name":"upsert","last_dispatch_duration_us":15025559,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":15025559,"total_server_duration_us":19652,"encode_duration_us":16238,"operation_id":"0x49bb","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10006956},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":16699,"operation_id":"0x4a81","timeout_ms":10000,"total_duration_us":10006758},{"operation_name":"upsert","last_dispatch_duration_us":50420,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9224354,"total_server_duration_us":83186,"encode_duration_us":17301,"operation_id":"0x4ad1","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10005672},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":16636,"operation_id":"0x4b4b","timeout_ms":10000,"total_duration_us":10003703},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":24864,"operation_id":"0x4b77","timeout_ms":10000,"total_duration_us":10003689}],"total_count":14}} WARN [2022-01-10 05:42:41,610] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9055829,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9526523,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9281937,"encode_duration_us":16986,"operation_id":"0x4a73","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10001169},{"operation_name":"upsert","last_dispatch_duration_us":17114839,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":20485911,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9338175,"encode_duration_us":17071,"operation_id":"0x47e6","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10008493},{"operation_name":"upsert","last_dispatch_duration_us":9284161,"last_local_id":"EFE9906200000002/00000000598DAEAE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10089515,"last_local_socket":"172.23.104.79:44342","total_server_duration_us":9772017,"encode_duration_us":17811,"operation_id":"0x4acc","timeout_ms":10000,"last_server_duration_us":9073483,"total_duration_us":10005048}],"total_count":3}} INFO [2022-01-10 05:42:41,616] jepsen worker 89 - jepsen.util 2789 :ok :read [29 0] INFO [2022-01-10 05:42:41,616] jepsen worker 89 - jepsen.util 2789 :invoke :read [29 nil] INFO [2022-01-10 05:42:41,685] jepsen worker 27 - jepsen.util 2367 :ok :read [9 22] INFO [2022-01-10 05:42:41,686] jepsen worker 27 - jepsen.util 2367 :invoke :write [9 11] INFO [2022-01-10 05:42:41,696] jepsen worker 89 - jepsen.util 2789 :ok :read [29 0] INFO [2022-01-10 05:42:41,697] jepsen worker 89 - jepsen.util 2789 :invoke :read [29 nil] WARN [2022-01-10 05:42:41,709] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17215351,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":22356158,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9396785,"encode_duration_us":16890,"operation_id":"0x480a","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10001526},{"operation_name":"upsert","last_dispatch_duration_us":21660536,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":21660536,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9049045,"encode_duration_us":16723,"operation_id":"0x4831","timeout_ms":10000,"last_server_duration_us":9049045,"total_duration_us":10006970}],"total_count":2}} WARN [2022-01-10 05:42:41,710] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13818816,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":13818816,"total_server_duration_us":7856,"operation_id":"0x4ac1","timeout_ms":30000,"last_server_duration_us":7856,"total_duration_us":13820925},{"operation_name":"get","last_dispatch_duration_us":10977250,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10977250,"total_server_duration_us":19,"operation_id":"0x4b49","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":10979583},{"operation_name":"upsert","last_dispatch_duration_us":9045549,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9884224,"total_server_duration_us":9322359,"encode_duration_us":16828,"operation_id":"0x4afd","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10010302},{"operation_name":"upsert","last_dispatch_duration_us":8639513,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9458756,"total_server_duration_us":301879,"encode_duration_us":16630,"operation_id":"0x4afc","timeout_ms":10000,"last_server_duration_us":20921,"total_duration_us":10009222},{"operation_name":"upsert","encode_duration_us":17119,"operation_id":"0x4a69","timeout_ms":10000,"total_duration_us":10008173},{"operation_name":"upsert","encode_duration_us":16443,"operation_id":"0x4afb","timeout_ms":10000,"total_duration_us":10007838},{"operation_name":"upsert","last_dispatch_duration_us":8686242,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":14723963,"total_server_duration_us":302671,"encode_duration_us":18037,"operation_id":"0x4a12","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10007272},{"operation_name":"upsert","last_dispatch_duration_us":170318,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9956894,"total_server_duration_us":95399,"encode_duration_us":16494,"operation_id":"0x4b84","timeout_ms":10000,"last_server_duration_us":30069,"total_duration_us":10007024},{"operation_name":"upsert","encode_duration_us":22088,"operation_id":"0x4b75","timeout_ms":10000,"total_duration_us":10003287},{"operation_name":"upsert","last_dispatch_duration_us":18150959,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":18150959,"total_server_duration_us":27402,"encode_duration_us":16148,"operation_id":"0x49eb","timeout_ms":10000,"last_server_duration_us":27402,"total_duration_us":10002861}],"total_count":12}} WARN [2022-01-10 05:42:41,710] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18756767,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18756767,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9613843,"encode_duration_us":16378,"operation_id":"0x483b","timeout_ms":10000,"last_server_duration_us":9613843,"total_duration_us":10001729},{"operation_name":"upsert","last_dispatch_duration_us":19156980,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":24706661,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9615703,"encode_duration_us":26324,"operation_id":"0x470c","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10006355},{"operation_name":"upsert","last_dispatch_duration_us":16253014,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16898147,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":7231429,"encode_duration_us":16730,"operation_id":"0x48f4","timeout_ms":10000,"last_server_duration_us":7212783,"total_duration_us":10008530},{"operation_name":"upsert","last_dispatch_duration_us":25604334,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":30256396,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9215439,"encode_duration_us":17107,"operation_id":"0x47a1","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10006928}],"total_count":4}} WARN [2022-01-10 05:42:41,710] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12792230,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12792230,"total_server_duration_us":139698,"encode_duration_us":16980,"operation_id":"0x49e9","timeout_ms":10000,"last_server_duration_us":139698,"total_duration_us":10005444},{"operation_name":"upsert","last_dispatch_duration_us":8747641,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":18176284,"total_server_duration_us":102903,"encode_duration_us":18941,"operation_id":"0x4a1e","timeout_ms":10000,"last_server_duration_us":26241,"total_duration_us":10005435},{"operation_name":"upsert","encode_duration_us":17004,"operation_id":"0x4a10","timeout_ms":10000,"total_duration_us":10005105},{"operation_name":"upsert","last_dispatch_duration_us":9284161,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":10089515,"total_server_duration_us":9772017,"encode_duration_us":17811,"operation_id":"0x4acc","timeout_ms":10000,"last_server_duration_us":9073483,"total_duration_us":10005048},{"operation_name":"upsert","last_dispatch_duration_us":10517482,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10517482,"total_server_duration_us":19185,"encode_duration_us":16482,"operation_id":"0x49cf","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10004888},{"operation_name":"upsert","last_dispatch_duration_us":8751117,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":12592783,"total_server_duration_us":1296216,"encode_duration_us":18130,"operation_id":"0x4a43","timeout_ms":10000,"last_server_duration_us":42743,"total_duration_us":10004309},{"operation_name":"upsert","encode_duration_us":16887,"operation_id":"0x4af9","timeout_ms":10000,"total_duration_us":10003029},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":23219,"operation_id":"0x4b82","timeout_ms":10000,"total_duration_us":10001469},{"operation_name":"upsert","last_dispatch_duration_us":9055829,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9526523,"total_server_duration_us":9281937,"encode_duration_us":16986,"operation_id":"0x4a73","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10001169},{"operation_name":"upsert","encode_duration_us":16702,"operation_id":"0x4abf","timeout_ms":10000,"total_duration_us":10001097}],"total_count":15}} INFO [2022-01-10 05:42:41,753] jepsen worker 89 - jepsen.util 2789 :ok :read [29 0] INFO [2022-01-10 05:42:41,753] jepsen worker 89 - jepsen.util 2789 :invoke :write [29 34] WARN [2022-01-10 05:42:41,809] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10138246,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10138246,"total_server_duration_us":411245,"operation_id":"0x4a42","timeout_ms":30000,"last_server_duration_us":411245,"total_duration_us":12248371},{"operation_name":"get","last_dispatch_duration_us":1266547,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1266547,"total_server_duration_us":6977,"operation_id":"0x4abc","timeout_ms":30000,"last_server_duration_us":6977,"total_duration_us":11022088},{"operation_name":"get","last_dispatch_duration_us":10250762,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10250762,"total_server_duration_us":411531,"operation_id":"0x4a71","timeout_ms":30000,"last_server_duration_us":411531,"total_duration_us":10814652},{"operation_name":"upsert","last_dispatch_duration_us":12791390,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12791390,"total_server_duration_us":9988457,"encode_duration_us":17878,"operation_id":"0x4a0c","timeout_ms":10000,"last_server_duration_us":9988457,"total_duration_us":10010182},{"operation_name":"upsert","last_dispatch_duration_us":7459994,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":7685755,"total_server_duration_us":151044,"encode_duration_us":16755,"operation_id":"0x4b85","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10007656},{"operation_name":"upsert","last_dispatch_duration_us":1986570,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1986570,"total_server_duration_us":179088,"encode_duration_us":17145,"operation_id":"0x4a1f","timeout_ms":10000,"last_server_duration_us":179088,"total_duration_us":10007604},{"operation_name":"upsert","last_dispatch_duration_us":1261425,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1261425,"total_server_duration_us":189220,"encode_duration_us":17003,"operation_id":"0x4b76","timeout_ms":10000,"last_server_duration_us":189220,"total_duration_us":10006427},{"operation_name":"upsert","last_dispatch_duration_us":8406985,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9145312,"total_server_duration_us":429362,"encode_duration_us":17505,"operation_id":"0x49ed","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10004660},{"operation_name":"upsert","last_dispatch_duration_us":9043591,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9578786,"total_server_duration_us":9326535,"encode_duration_us":17174,"operation_id":"0x4a72","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10003066},{"operation_name":"upsert","last_dispatch_duration_us":8311532,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9053067,"total_server_duration_us":462891,"encode_duration_us":17953,"operation_id":"0x49df","timeout_ms":10000,"last_server_duration_us":21001,"total_duration_us":10002692}],"total_count":16}} WARN [2022-01-10 05:42:41,810] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":20416187,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20416187,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9027816,"encode_duration_us":17350,"operation_id":"0x47d0","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10001395},{"operation_name":"upsert","last_dispatch_duration_us":9045549,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9884224,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9322359,"encode_duration_us":16828,"operation_id":"0x4afd","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10010302},{"operation_name":"upsert","last_dispatch_duration_us":27055361,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":27055361,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9028877,"encode_duration_us":17677,"operation_id":"0x4852","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10005657}],"total_count":3}} INFO [2022-01-10 05:42:41,811] jepsen worker 54 - jepsen.util 2394 :info :write [18 15] :RequestTimeoutException INFO [2022-01-10 05:42:41,811] jepsen worker 54 - jepsen.util 2484 :invoke :write [18 10] INFO [2022-01-10 05:42:41,884] jepsen worker 38 - jepsen.util 2558 :info :write [12 24] :RequestTimeoutException INFO [2022-01-10 05:42:41,885] jepsen worker 38 - jepsen.util 2648 :invoke :read [12 nil] WARN [2022-01-10 05:42:41,909] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9043591,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9578786,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9326535,"encode_duration_us":17174,"operation_id":"0x4a72","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10003066},{"operation_name":"upsert","last_dispatch_duration_us":9040441,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9831677,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9470484,"encode_duration_us":21071,"operation_id":"0x4982","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10006907},{"operation_name":"upsert","last_dispatch_duration_us":12791390,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12791390,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9988457,"encode_duration_us":17878,"operation_id":"0x4a0c","timeout_ms":10000,"last_server_duration_us":9988457,"total_duration_us":10010182},{"operation_name":"upsert","last_dispatch_duration_us":10820598,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12394220,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9466629,"encode_duration_us":22455,"operation_id":"0x48f1","timeout_ms":10000,"last_server_duration_us":9446663,"total_duration_us":10010414}],"total_count":4}} WARN [2022-01-10 05:42:41,911] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12219911,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":12219911,"total_server_duration_us":23,"operation_id":"0x49d6","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":12221049},{"operation_name":"get","last_dispatch_duration_us":10057611,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10057611,"total_server_duration_us":6191,"operation_id":"0x49ea","timeout_ms":30000,"last_server_duration_us":6191,"total_duration_us":10060053},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":16386,"operation_id":"0x4b83","timeout_ms":10000,"total_duration_us":10009916},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":16369,"operation_id":"0x4b48","timeout_ms":10000,"total_duration_us":10009360},{"operation_name":"upsert","last_dispatch_duration_us":43821,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":1104925,"total_server_duration_us":672355,"encode_duration_us":16221,"operation_id":"0x4b56","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10007868},{"operation_name":"upsert","last_dispatch_duration_us":10980350,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10980350,"total_server_duration_us":40376,"encode_duration_us":19605,"operation_id":"0x4a11","timeout_ms":10000,"last_server_duration_us":40376,"total_duration_us":10007667},{"operation_name":"upsert","last_dispatch_duration_us":33449,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":7333582,"total_server_duration_us":41017,"encode_duration_us":16848,"operation_id":"0x4a4a","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10005777},{"operation_name":"upsert","last_dispatch_duration_us":4514787,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12151968,"total_server_duration_us":637797,"encode_duration_us":16571,"operation_id":"0x4aba","timeout_ms":10000,"last_server_duration_us":21163,"total_duration_us":10005093},{"operation_name":"get","last_dispatch_duration_us":9295885,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9295885,"total_server_duration_us":9686,"operation_id":"0x4a20","timeout_ms":30000,"last_server_duration_us":9686,"total_duration_us":9298381},{"operation_name":"get","last_dispatch_duration_us":7046534,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":7046534,"total_server_duration_us":43,"operation_id":"0x4ac0","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":7047657}],"total_count":17}} INFO [2022-01-10 05:42:41,914] jepsen worker 38 - jepsen.util 2648 :ok :read [12 29] INFO [2022-01-10 05:42:41,914] jepsen worker 38 - jepsen.util 2648 :invoke :write [12 7] INFO [2022-01-10 05:42:42,204] jepsen worker 16 - jepsen.util 2446 :info :write [5 29] :RequestTimeoutException INFO [2022-01-10 05:42:42,205] jepsen worker 16 - jepsen.util 2536 :invoke :read [5 nil] INFO [2022-01-10 05:42:42,214] jepsen worker 15 - jepsen.util 1995 :info :write [5 17] :RequestTimeoutException INFO [2022-01-10 05:42:42,215] jepsen worker 15 - jepsen.util 2085 :invoke :write [5 45] INFO [2022-01-10 05:42:42,256] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:42,257] jepsen worker nemesis - couchbase.util Rebalance check count: 315 INFO [2022-01-10 05:42:42,257] jepsen worker nemesis - couchbase.util Rebalance stuck count: 252 INFO [2022-01-10 05:42:42,273] jepsen worker 34 - jepsen.util 2194 :ok :read [11 21] INFO [2022-01-10 05:42:42,273] jepsen worker 34 - jepsen.util 2194 :invoke :read [11 nil] INFO [2022-01-10 05:42:42,379] jepsen worker 29 - jepsen.util 2369 :ok :read [9 22] INFO [2022-01-10 05:42:42,380] jepsen worker 29 - jepsen.util 2369 :invoke :read [9 nil] INFO [2022-01-10 05:42:42,406] jepsen worker 31 - jepsen.util 2731 :info :write [10 7] :RequestTimeoutException INFO [2022-01-10 05:42:42,406] jepsen worker 31 - jepsen.util 2821 :invoke :read [10 nil] INFO [2022-01-10 05:42:42,418] jepsen worker 77 - jepsen.util 2777 :ok :read [25 :nil] INFO [2022-01-10 05:42:42,420] jepsen worker 77 - jepsen.util 2777 :invoke :read [25 nil] INFO [2022-01-10 05:42:42,450] jepsen worker 31 - jepsen.util 2821 :ok :read [10 37] INFO [2022-01-10 05:42:42,451] jepsen worker 31 - jepsen.util 2821 :invoke :read [10 nil] INFO [2022-01-10 05:42:42,480] jepsen worker 31 - jepsen.util 2821 :ok :read [10 37] INFO [2022-01-10 05:42:42,480] jepsen worker 31 - jepsen.util 2821 :invoke :read [10 nil] INFO [2022-01-10 05:42:42,504] jepsen worker 31 - jepsen.util 2821 :ok :read [10 37] INFO [2022-01-10 05:42:42,506] jepsen worker 31 - jepsen.util 2821 :invoke :write [10 15] INFO [2022-01-10 05:42:42,514] jepsen worker 69 - jepsen.util 2409 :info :write [23 8] :RequestTimeoutException INFO [2022-01-10 05:42:42,515] jepsen worker 69 - jepsen.util 2499 :invoke :read [23 nil] INFO [2022-01-10 05:42:42,524] jepsen worker 26 - jepsen.util 2006 :ok :read [8 27] INFO [2022-01-10 05:42:42,525] jepsen worker 26 - jepsen.util 2006 :invoke :read [8 nil] INFO [2022-01-10 05:42:42,525] jepsen worker 4 - jepsen.util 2614 :info :write [1 17] :RequestTimeoutException INFO [2022-01-10 05:42:42,526] jepsen worker 4 - jepsen.util 2704 :invoke :read [1 nil] INFO [2022-01-10 05:42:42,584] jepsen worker 88 - jepsen.util 2248 :info :write [29 27] :RequestTimeoutException INFO [2022-01-10 05:42:42,585] jepsen worker 88 - jepsen.util 2338 :invoke :read [29 nil] INFO [2022-01-10 05:42:42,591] jepsen worker 57 - jepsen.util 2757 :info :write [19 9] :RequestTimeoutException INFO [2022-01-10 05:42:42,592] jepsen worker 57 - jepsen.util 2847 :invoke :write [19 33] INFO [2022-01-10 05:42:42,612] jepsen worker 88 - jepsen.util 2338 :ok :read [29 0] INFO [2022-01-10 05:42:42,612] jepsen worker 88 - jepsen.util 2338 :invoke :read [29 nil] INFO [2022-01-10 05:42:42,639] jepsen worker 88 - jepsen.util 2338 :ok :read [29 0] INFO [2022-01-10 05:42:42,639] jepsen worker 88 - jepsen.util 2338 :invoke :write [29 34] INFO [2022-01-10 05:42:42,716] jepsen worker 9 - jepsen.util 2799 :info :write [3 8] :RequestTimeoutException INFO [2022-01-10 05:42:42,717] jepsen worker 9 - jepsen.util 2889 :invoke :read [3 nil] INFO [2022-01-10 05:42:42,742] jepsen worker 9 - jepsen.util 2889 :ok :read [3 31] INFO [2022-01-10 05:42:42,743] jepsen worker 9 - jepsen.util 2889 :invoke :write [3 12] INFO [2022-01-10 05:42:42,944] jepsen worker 51 - jepsen.util 2661 :info :write [17 21] :RequestTimeoutException INFO [2022-01-10 05:42:42,945] jepsen worker 51 - jepsen.util 2751 :invoke :read [17 nil] INFO [2022-01-10 05:42:42,972] jepsen worker 51 - jepsen.util 2751 :ok :read [17 41] INFO [2022-01-10 05:42:42,972] jepsen worker 51 - jepsen.util 2751 :invoke :write [17 46] INFO [2022-01-10 05:42:43,086] jepsen worker 45 - jepsen.util 2745 :info :write [15 20] :RequestTimeoutException INFO [2022-01-10 05:42:43,087] jepsen worker 45 - jepsen.util 2835 :invoke :write [15 31] INFO [2022-01-10 05:42:43,121] jepsen worker 55 - jepsen.util 2665 :info :write [18 17] :RequestTimeoutException INFO [2022-01-10 05:42:43,122] jepsen worker 55 - jepsen.util 2755 :invoke :write [18 37] INFO [2022-01-10 05:42:43,166] jepsen worker 58 - jepsen.util 2308 :info :write [19 1] :RequestTimeoutException INFO [2022-01-10 05:42:43,167] jepsen worker 58 - jepsen.util 2398 :invoke :write [19 26] INFO [2022-01-10 05:42:43,179] jepsen worker 44 - jepsen.util 2294 :info :write [14 11] :RequestTimeoutException INFO [2022-01-10 05:42:43,180] jepsen worker 44 - jepsen.util 2384 :invoke :write [14 31] INFO [2022-01-10 05:42:43,197] jepsen worker 5 - jepsen.util 2435 :info :write [1 44] :SyncWriteAmbiguous INFO [2022-01-10 05:42:43,198] jepsen worker 5 - jepsen.util 2525 :invoke :read [1 nil] INFO [2022-01-10 05:42:43,243] jepsen worker 5 - jepsen.util 2525 :ok :read [1 10] INFO [2022-01-10 05:42:43,244] jepsen worker 5 - jepsen.util 2525 :invoke :write [1 10] INFO [2022-01-10 05:42:43,264] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:43,264] jepsen worker nemesis - couchbase.util Rebalance check count: 316 INFO [2022-01-10 05:42:43,264] jepsen worker nemesis - couchbase.util Rebalance stuck count: 253 INFO [2022-01-10 05:42:43,304] jepsen worker 30 - jepsen.util 2190 :info :write [10 49] :RequestTimeoutException INFO [2022-01-10 05:42:43,305] jepsen worker 30 - jepsen.util 2280 :invoke :write [10 43] INFO [2022-01-10 05:42:43,356] jepsen worker 72 - jepsen.util 2502 :info :write [24 23] :RequestTimeoutException INFO [2022-01-10 05:42:43,356] jepsen worker 72 - jepsen.util 2592 :invoke :write [24 49] INFO [2022-01-10 05:42:43,369] jepsen worker 73 - jepsen.util 2143 :info :write [24 5] :RequestTimeoutException INFO [2022-01-10 05:42:43,370] jepsen worker 73 - jepsen.util 2233 :invoke :read [24 nil] INFO [2022-01-10 05:42:43,440] jepsen worker 20 - jepsen.util 2000 :info :write [6 15] :RequestTimeoutException INFO [2022-01-10 05:42:43,440] jepsen worker 20 - jepsen.util 2090 :invoke :read [6 nil] INFO [2022-01-10 05:42:43,459] jepsen worker 8 - jepsen.util 2618 :info :write [2 1] :RequestTimeoutException INFO [2022-01-10 05:42:43,460] jepsen worker 8 - jepsen.util 2708 :invoke :write [2 34] INFO [2022-01-10 05:42:43,616] jepsen worker 65 - jepsen.util 2225 :ok :read [21 7] INFO [2022-01-10 05:42:43,616] jepsen worker 65 - jepsen.util 2225 :invoke :write [21 41] INFO [2022-01-10 05:42:43,728] jepsen worker 11 - jepsen.util 2351 :info :write [3 11] :SyncWriteAmbiguous INFO [2022-01-10 05:42:43,728] jepsen worker 11 - jepsen.util 2441 :invoke :write [3 17] INFO [2022-01-10 05:42:43,781] jepsen worker 0 - jepsen.util 2340 :info :write [0 39] :RequestTimeoutException INFO [2022-01-10 05:42:43,782] jepsen worker 0 - jepsen.util 2430 :invoke :write [0 25] INFO [2022-01-10 05:42:43,812] jepsen worker 39 - jepsen.util 2739 :info :write [13 26] :RequestTimeoutException INFO [2022-01-10 05:42:43,812] jepsen worker 39 - jepsen.util 2829 :invoke :read [13 nil] INFO [2022-01-10 05:42:43,873] jepsen worker 87 - jepsen.util 2517 :ok :read [29 0] INFO [2022-01-10 05:42:43,873] jepsen worker 87 - jepsen.util 2517 :invoke :write [29 11] INFO [2022-01-10 05:42:43,918] jepsen worker 70 - jepsen.util 2590 :ok :read [23 26] INFO [2022-01-10 05:42:43,919] jepsen worker 70 - jepsen.util 2590 :invoke :read [23 nil] INFO [2022-01-10 05:42:43,946] jepsen worker 52 - jepsen.util 2932 :info :write [17 8] :RequestTimeoutException INFO [2022-01-10 05:42:43,947] jepsen worker 52 - jepsen.util 3022 :invoke :write [17 38] INFO [2022-01-10 05:42:44,179] jepsen worker 3 - jepsen.util 2433 :info :write [1 18] :RequestTimeoutException INFO [2022-01-10 05:42:44,180] jepsen worker 3 - jepsen.util 2523 :invoke :read [1 nil] INFO [2022-01-10 05:42:44,272] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:44,272] jepsen worker nemesis - couchbase.util Rebalance check count: 317 INFO [2022-01-10 05:42:44,272] jepsen worker nemesis - couchbase.util Rebalance stuck count: 254 INFO [2022-01-10 05:42:44,289] jepsen worker 20 - jepsen.util 2090 :ok :read [6 25] INFO [2022-01-10 05:42:44,289] jepsen worker 20 - jepsen.util 2090 :invoke :write [6 4] INFO [2022-01-10 05:42:44,319] jepsen worker 26 - jepsen.util 2006 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:44,320] jepsen worker 61 - jepsen.util 2401 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:44,320] jepsen worker 25 - jepsen.util 2275 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:44,320] jepsen worker 26 - jepsen.util 2006 :invoke :write [8 17] INFO [2022-01-10 05:42:44,322] jepsen worker 61 - jepsen.util 2401 :invoke :write [20 39] INFO [2022-01-10 05:42:44,330] jepsen worker 6 - jepsen.util 2346 :info :write [2 25] :RequestTimeoutException INFO [2022-01-10 05:42:44,331] jepsen worker 6 - jepsen.util 2436 :invoke :write [2 5] INFO [2022-01-10 05:42:44,334] jepsen worker 25 - jepsen.util 2275 :invoke :write [8 43] INFO [2022-01-10 05:42:44,352] jepsen worker 34 - jepsen.util 2194 :ok :read [11 21] INFO [2022-01-10 05:42:44,353] jepsen worker 34 - jepsen.util 2194 :invoke :read [11 nil] INFO [2022-01-10 05:42:44,666] jepsen worker 7 - jepsen.util 2437 :ok :read [2 14] INFO [2022-01-10 05:42:44,666] jepsen worker 7 - jepsen.util 2437 :invoke :read [2 nil] INFO [2022-01-10 05:42:44,732] jepsen worker 37 - jepsen.util 2467 :info :write [12 34] :RequestTimeoutException INFO [2022-01-10 05:42:44,733] jepsen worker 37 - jepsen.util 2557 :invoke :write [12 31] INFO [2022-01-10 05:42:44,762] jepsen worker 34 - jepsen.util 2194 :ok :read [11 21] INFO [2022-01-10 05:42:44,763] jepsen worker 34 - jepsen.util 2194 :invoke :write [11 8] INFO [2022-01-10 05:42:44,811] jepsen worker 56 - jepsen.util 2486 :info :write [18 25] :RequestTimeoutException INFO [2022-01-10 05:42:44,811] jepsen worker 56 - jepsen.util 2576 :invoke :write [18 26] INFO [2022-01-10 05:42:44,934] jepsen worker 60 - jepsen.util 2760 :ok :read [20 :nil] INFO [2022-01-10 05:42:44,934] jepsen worker 60 - jepsen.util 2760 :invoke :write [20 31] INFO [2022-01-10 05:42:45,281] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:45,281] jepsen worker nemesis - couchbase.util Rebalance check count: 318 INFO [2022-01-10 05:42:45,329] jepsen worker 36 - jepsen.util 2106 :ok :read [12 29] INFO [2022-01-10 05:42:45,368] jepsen worker 4 - jepsen.util 2704 :ok :read [1 10] INFO [2022-01-10 05:42:45,380] jepsen worker nemesis - couchbase.util Rebalance stuck count: 255 INFO [2022-01-10 05:42:45,380] jepsen worker 36 - jepsen.util 2106 :invoke :write [12 3] INFO [2022-01-10 05:42:45,380] jepsen worker 4 - jepsen.util 2704 :invoke :read [1 nil] INFO [2022-01-10 05:42:45,443] jepsen worker 4 - jepsen.util 2704 :ok :read [1 10] INFO [2022-01-10 05:42:45,444] jepsen worker 4 - jepsen.util 2704 :invoke :write [1 46] INFO [2022-01-10 05:42:45,512] jepsen worker 7 - jepsen.util 2437 :ok :read [2 14] INFO [2022-01-10 05:42:45,513] jepsen worker 7 - jepsen.util 2437 :invoke :write [2 44] INFO [2022-01-10 05:42:45,630] jepsen worker 49 - jepsen.util 2299 :ok :read [16 38] INFO [2022-01-10 05:42:45,630] jepsen worker 49 - jepsen.util 2299 :invoke :write [16 18] INFO [2022-01-10 05:42:45,643] jepsen worker 48 - jepsen.util 2118 :ok :read [16 38] INFO [2022-01-10 05:42:45,643] jepsen worker 48 - jepsen.util 2118 :invoke :read [16 nil] INFO [2022-01-10 05:42:45,685] jepsen worker 82 - jepsen.util 2422 :ok :read [27 30] INFO [2022-01-10 05:42:45,686] jepsen worker 82 - jepsen.util 2422 :invoke :write [27 26] INFO [2022-01-10 05:42:45,735] jepsen worker 48 - jepsen.util 2118 :ok :read [16 38] INFO [2022-01-10 05:42:45,735] jepsen worker 48 - jepsen.util 2118 :invoke :read [16 nil] INFO [2022-01-10 05:42:45,772] jepsen worker 28 - jepsen.util 2278 :ok :read [9 22] INFO [2022-01-10 05:42:45,773] jepsen worker 28 - jepsen.util 2278 :invoke :write [9 37] INFO [2022-01-10 05:42:45,985] jepsen worker 43 - jepsen.util 2113 :info :write [14 41] :RequestTimeoutException INFO [2022-01-10 05:42:45,986] jepsen worker 43 - jepsen.util 2203 :invoke :read [14 nil] INFO [2022-01-10 05:42:46,096] jepsen worker 42 - jepsen.util 2202 :info :write [14 39] :RequestTimeoutException INFO [2022-01-10 05:42:46,097] jepsen worker 42 - jepsen.util 2292 :invoke :read [14 nil] INFO [2022-01-10 05:42:46,106] jepsen worker 24 - jepsen.util 2004 :info :write [8 8] :RequestTimeoutException INFO [2022-01-10 05:42:46,107] jepsen worker 24 - jepsen.util 2094 :invoke :read [8 nil] INFO [2022-01-10 05:42:46,247] jepsen worker 73 - jepsen.util 2233 :ok :read [24 18] INFO [2022-01-10 05:42:46,248] jepsen worker 73 - jepsen.util 2233 :invoke :write [24 47] INFO [2022-01-10 05:42:46,397] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:46,397] jepsen worker nemesis - couchbase.util Rebalance check count: 319 INFO [2022-01-10 05:42:46,397] jepsen worker nemesis - couchbase.util Rebalance stuck count: 256 INFO [2022-01-10 05:42:46,447] jepsen worker 12 - jepsen.util 2442 :info :write [4 19] :RequestTimeoutException INFO [2022-01-10 05:42:46,447] jepsen worker 12 - jepsen.util 2532 :invoke :read [4 nil] INFO [2022-01-10 05:42:46,591] jepsen worker 42 - jepsen.util 2292 :ok :read [14 14] INFO [2022-01-10 05:42:46,591] jepsen worker 42 - jepsen.util 2292 :invoke :read [14 nil] INFO [2022-01-10 05:42:46,640] jepsen worker 12 - jepsen.util 2532 :ok :read [4 3] INFO [2022-01-10 05:42:46,641] jepsen worker 12 - jepsen.util 2532 :invoke :write [4 17] INFO [2022-01-10 05:42:46,791] jepsen worker 81 - jepsen.util 2601 :info :write [27 17] :RequestTimeoutException INFO [2022-01-10 05:42:46,792] jepsen worker 81 - jepsen.util 2691 :invoke :read [27 nil] INFO [2022-01-10 05:42:46,891] jepsen worker 18 - jepsen.util 2088 :info :write [6 1] :RequestTimeoutException INFO [2022-01-10 05:42:46,891] jepsen worker 18 - jepsen.util 2178 :invoke :write [6 2] INFO [2022-01-10 05:42:47,043] jepsen worker 35 - jepsen.util 2015 :fail :read [11 nil] :UnambiguousTimeoutException INFO [2022-01-10 05:42:47,043] jepsen worker 35 - jepsen.util 2015 :invoke :write [11 1] INFO [2022-01-10 05:42:47,074] jepsen worker 14 - jepsen.util 2804 :info :write [4 49] :RequestTimeoutException INFO [2022-01-10 05:42:47,075] jepsen worker 14 - jepsen.util 2894 :invoke :read [4 nil] INFO [2022-01-10 05:42:47,103] jepsen worker 14 - jepsen.util 2894 :ok :read [4 3] INFO [2022-01-10 05:42:47,104] jepsen worker 14 - jepsen.util 2894 :invoke :write [4 40] INFO [2022-01-10 05:42:47,106] jepsen worker 33 - jepsen.util 2643 :info :write [11 16] :RequestTimeoutException INFO [2022-01-10 05:42:47,106] jepsen worker 33 - jepsen.util 2733 :invoke :read [11 nil] INFO [2022-01-10 05:42:47,252] jepsen worker 10 - jepsen.util 2620 :info :write [3 48] :RequestTimeoutException INFO [2022-01-10 05:42:47,253] jepsen worker 10 - jepsen.util 2710 :invoke :write [3 28] INFO [2022-01-10 05:42:47,405] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:47,405] jepsen worker nemesis - couchbase.util Rebalance check count: 320 INFO [2022-01-10 05:42:47,405] jepsen worker nemesis - couchbase.util Rebalance stuck count: 257 INFO [2022-01-10 05:42:47,432] jepsen worker 47 - jepsen.util 2297 :ok :read [15 23] INFO [2022-01-10 05:42:47,433] jepsen worker 47 - jepsen.util 2297 :invoke :write [15 27] INFO [2022-01-10 05:42:47,437] jepsen worker 74 - jepsen.util 2954 :ok :read [24 18] INFO [2022-01-10 05:42:47,438] jepsen worker 74 - jepsen.util 2954 :invoke :read [24 nil] INFO [2022-01-10 05:42:47,465] jepsen worker 80 - jepsen.util 2780 :info :write [26 33] :RequestTimeoutException INFO [2022-01-10 05:42:47,465] jepsen worker 80 - jepsen.util 2870 :invoke :read [26 nil] INFO [2022-01-10 05:42:47,503] jepsen worker 62 - jepsen.util 2582 :ok :read [20 :nil] INFO [2022-01-10 05:42:47,504] jepsen worker 62 - jepsen.util 2582 :invoke :read [20 nil] INFO [2022-01-10 05:42:47,575] jepsen worker 80 - jepsen.util 2870 :ok :read [26 10] INFO [2022-01-10 05:42:47,575] jepsen worker 80 - jepsen.util 2870 :invoke :read [26 nil] INFO [2022-01-10 05:42:47,591] jepsen worker 78 - jepsen.util 2778 :ok :read [26 10] INFO [2022-01-10 05:42:47,593] jepsen worker 78 - jepsen.util 2778 :invoke :read [26 nil] INFO [2022-01-10 05:42:47,601] jepsen worker 42 - jepsen.util 2292 :ok :read [14 14] INFO [2022-01-10 05:42:47,601] jepsen worker 42 - jepsen.util 2292 :invoke :read [14 nil] INFO [2022-01-10 05:42:47,622] jepsen worker 74 - jepsen.util 2954 :ok :read [24 18] INFO [2022-01-10 05:42:47,623] jepsen worker 74 - jepsen.util 2954 :invoke :read [24 nil] INFO [2022-01-10 05:42:47,660] jepsen worker 80 - jepsen.util 2870 :ok :read [26 10] INFO [2022-01-10 05:42:47,661] jepsen worker 80 - jepsen.util 2870 :invoke :read [26 nil] INFO [2022-01-10 05:42:47,715] jepsen worker 43 - jepsen.util 2203 :ok :read [14 14] INFO [2022-01-10 05:42:47,716] jepsen worker 43 - jepsen.util 2203 :invoke :write [14 19] INFO [2022-01-10 05:42:47,716] jepsen worker 80 - jepsen.util 2870 :ok :read [26 10] INFO [2022-01-10 05:42:47,717] jepsen worker 80 - jepsen.util 2870 :invoke :read [26 nil] INFO [2022-01-10 05:42:47,748] jepsen worker 78 - jepsen.util 2778 :ok :read [26 10] INFO [2022-01-10 05:42:47,748] jepsen worker 78 - jepsen.util 2778 :invoke :write [26 24] INFO [2022-01-10 05:42:47,843] jepsen worker 80 - jepsen.util 2870 :ok :read [26 10] INFO [2022-01-10 05:42:47,844] jepsen worker 80 - jepsen.util 2870 :invoke :write [26 36] INFO [2022-01-10 05:42:47,957] jepsen worker 29 - jepsen.util 2369 :ok :read [9 24] INFO [2022-01-10 05:42:47,958] jepsen worker 29 - jepsen.util 2369 :invoke :write [9 26] INFO [2022-01-10 05:42:47,966] jepsen worker 19 - jepsen.util 2539 :fail :read [6 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:47,967] jepsen worker 22 - jepsen.util 2182 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:47,966] jepsen worker 23 - jepsen.util 2453 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:47,970] jepsen worker 46 - jepsen.util 2656 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:47,971] jepsen worker 46 - jepsen.util 2656 :invoke :write [15 26] INFO [2022-01-10 05:42:47,971] jepsen worker 19 - jepsen.util 2539 :invoke :write [6 7] INFO [2022-01-10 05:42:47,972] jepsen worker 22 - jepsen.util 2182 :invoke :write [7 42] INFO [2022-01-10 05:42:47,977] jepsen worker 23 - jepsen.util 2453 :invoke :write [7 30] INFO [2022-01-10 05:42:48,139] jepsen worker 61 - jepsen.util 2401 :fail :write [20 39] :ServerOutOfMemoryException INFO [2022-01-10 05:42:48,139] jepsen worker 61 - jepsen.util 2401 :invoke :read [20 nil] INFO [2022-01-10 05:42:48,151] jepsen worker 62 - jepsen.util 2582 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:48,151] jepsen worker 64 - jepsen.util 2314 :fail :write [21 17] :ServerOutOfMemoryException INFO [2022-01-10 05:42:48,151] jepsen worker 76 - jepsen.util 1876 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:48,151] jepsen worker 74 - jepsen.util 2954 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:48,152] jepsen worker 62 - jepsen.util 2582 :invoke :write [20 37] INFO [2022-01-10 05:42:48,153] jepsen worker 74 - jepsen.util 2954 :invoke :write [24 35] INFO [2022-01-10 05:42:48,153] jepsen worker 64 - jepsen.util 2314 :invoke :write [21 32] INFO [2022-01-10 05:42:48,155] jepsen worker 76 - jepsen.util 1876 :invoke :read [25 nil] INFO [2022-01-10 05:42:48,259] jepsen worker 83 - jepsen.util 2783 :info :write [27 14] :RequestTimeoutException INFO [2022-01-10 05:42:48,260] jepsen worker 83 - jepsen.util 2873 :invoke :read [27 nil] INFO [2022-01-10 05:42:48,298] jepsen worker 24 - jepsen.util 2094 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:48,299] jepsen worker 24 - jepsen.util 2094 :invoke :read [8 nil] INFO [2022-01-10 05:42:48,350] jepsen worker 42 - jepsen.util 2292 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:48,351] jepsen worker 42 - jepsen.util 2292 :invoke :read [14 nil] INFO [2022-01-10 05:42:48,412] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:48,412] jepsen worker nemesis - couchbase.util Rebalance check count: 321 INFO [2022-01-10 05:42:48,412] jepsen worker nemesis - couchbase.util Rebalance stuck count: 258 INFO [2022-01-10 05:42:48,421] jepsen worker 32 - jepsen.util 2462 :info :write [10 45] :RequestTimeoutException INFO [2022-01-10 05:42:48,422] jepsen worker 32 - jepsen.util 2552 :invoke :write [10 2] INFO [2022-01-10 05:42:48,482] jepsen worker 76 - jepsen.util 1876 :ok :read [25 :nil] INFO [2022-01-10 05:42:48,482] jepsen worker 76 - jepsen.util 1876 :invoke :read [25 nil] INFO [2022-01-10 05:42:48,637] jepsen worker 33 - jepsen.util 2733 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:48,637] jepsen worker 33 - jepsen.util 2733 :invoke :read [11 nil] INFO [2022-01-10 05:42:48,841] jepsen worker 42 - jepsen.util 2292 :ok :read [14 14] INFO [2022-01-10 05:42:48,842] jepsen worker 42 - jepsen.util 2292 :invoke :read [14 nil] INFO [2022-01-10 05:42:48,906] jepsen worker 53 - jepsen.util 2843 :info :write [17 32] :RequestTimeoutException INFO [2022-01-10 05:42:48,907] jepsen worker 53 - jepsen.util 2933 :invoke :write [17 16] INFO [2022-01-10 05:42:48,919] jepsen worker 76 - jepsen.util 1876 :ok :read [25 :nil] INFO [2022-01-10 05:42:48,919] jepsen worker 76 - jepsen.util 1876 :invoke :write [25 22] INFO [2022-01-10 05:42:49,079] jepsen worker 33 - jepsen.util 2733 :ok :read [11 21] INFO [2022-01-10 05:42:49,080] jepsen worker 33 - jepsen.util 2733 :invoke :read [11 nil] INFO [2022-01-10 05:42:49,141] jepsen worker 42 - jepsen.util 2292 :ok :read [14 14] INFO [2022-01-10 05:42:49,142] jepsen worker 42 - jepsen.util 2292 :invoke :write [14 10] INFO [2022-01-10 05:42:49,211] jepsen worker 33 - jepsen.util 2733 :ok :read [11 21] INFO [2022-01-10 05:42:49,212] jepsen worker 33 - jepsen.util 2733 :invoke :write [11 47] INFO [2022-01-10 05:42:49,419] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:49,419] jepsen worker nemesis - couchbase.util Rebalance check count: 322 INFO [2022-01-10 05:42:49,419] jepsen worker nemesis - couchbase.util Rebalance stuck count: 259 INFO [2022-01-10 05:42:49,922] jepsen worker 79 - jepsen.util 2779 :info :write [26 41] :RequestTimeoutException INFO [2022-01-10 05:42:49,922] jepsen worker 79 - jepsen.util 2869 :invoke :read [26 nil] INFO [2022-01-10 05:42:49,957] jepsen worker 85 - jepsen.util 2695 :ok :read [28 45] INFO [2022-01-10 05:42:49,957] jepsen worker 85 - jepsen.util 2695 :invoke :write [28 15] INFO [2022-01-10 05:42:50,002] jepsen worker 41 - jepsen.util 2201 :ok :read [13 15] INFO [2022-01-10 05:42:50,003] jepsen worker 41 - jepsen.util 2201 :invoke :write [13 16] INFO [2022-01-10 05:42:50,015] jepsen worker 21 - jepsen.util 2451 :info :write [7 4] :RequestTimeoutException INFO [2022-01-10 05:42:50,016] jepsen worker 21 - jepsen.util 2541 :invoke :write [7 5] WARN [2022-01-10 05:42:50,027] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9393990,"last_local_id":"EFE9906200000001/000000007CF9F49A","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10142211,"last_local_socket":"172.23.104.79:44306","total_server_duration_us":9405414,"encode_duration_us":19522,"operation_id":"0x4bdd","timeout_ms":10000,"last_server_duration_us":9058604,"total_duration_us":10001862},{"operation_name":"upsert","last_dispatch_duration_us":19335450,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":22732708,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9331535,"encode_duration_us":18671,"operation_id":"0x49a4","timeout_ms":10000,"last_server_duration_us":9137368,"total_duration_us":10002593},{"operation_name":"upsert","last_dispatch_duration_us":9037812,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9678635,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9291088,"encode_duration_us":21688,"operation_id":"0x4bea","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10004799},{"operation_name":"upsert","last_dispatch_duration_us":19046660,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19046660,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9033121,"encode_duration_us":19696,"operation_id":"0x49e7","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10004152},{"operation_name":"upsert","last_dispatch_duration_us":9244545,"last_local_id":"EFE9906200000001/000000007CF9F49A","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10241351,"last_local_socket":"172.23.104.79:44306","total_server_duration_us":9588546,"encode_duration_us":16739,"operation_id":"0x4b47","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10004085},{"operation_name":"upsert","last_dispatch_duration_us":24691668,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":24691668,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":6596699,"encode_duration_us":17366,"operation_id":"0x4a1c","timeout_ms":10000,"last_server_duration_us":6596699,"total_duration_us":10009585}],"total_count":6}} INFO [2022-01-10 05:42:50,121] jepsen worker 63 - jepsen.util 2223 :info :write [21 18] :RequestTimeoutException INFO [2022-01-10 05:42:50,122] jepsen worker 63 - jepsen.util 2313 :invoke :read [21 nil] WARN [2022-01-10 05:42:50,128] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14493968,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":14493968,"total_server_duration_us":5628,"operation_id":"0x4a0f","timeout_ms":30000,"last_server_duration_us":5628,"total_duration_us":21704712},{"operation_name":"get","last_dispatch_duration_us":18120360,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":18120360,"total_server_duration_us":27,"operation_id":"0x4a1a","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":18121879},{"operation_name":"get","last_dispatch_duration_us":18117594,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":18117594,"total_server_duration_us":11,"operation_id":"0x4a1b","timeout_ms":30000,"last_server_duration_us":11,"total_duration_us":18118766},{"operation_name":"upsert","last_dispatch_duration_us":780648,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10118033,"total_server_duration_us":82977,"encode_duration_us":16525,"operation_id":"0x4bac","timeout_ms":10000,"last_server_duration_us":35211,"total_duration_us":10009816},{"operation_name":"upsert","last_dispatch_duration_us":931081,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10050302,"total_server_duration_us":316006,"encode_duration_us":17239,"operation_id":"0x4c67","timeout_ms":10000,"last_server_duration_us":165691,"total_duration_us":10009290},{"operation_name":"upsert","last_dispatch_duration_us":7946259,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9116117,"total_server_duration_us":345908,"encode_duration_us":17359,"operation_id":"0x4bb7","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10008908},{"operation_name":"upsert","last_dispatch_duration_us":8083431,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9691875,"total_server_duration_us":358491,"encode_duration_us":17227,"operation_id":"0x4ba5","timeout_ms":10000,"last_server_duration_us":21001,"total_duration_us":10008751},{"operation_name":"upsert","last_dispatch_duration_us":8067020,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9047581,"total_server_duration_us":316438,"encode_duration_us":16509,"operation_id":"0x4bba","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10008302},{"operation_name":"upsert","encode_duration_us":18774,"operation_id":"0x4b53","timeout_ms":10000,"total_duration_us":10007974},{"operation_name":"upsert","last_dispatch_duration_us":766752,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10161582,"total_server_duration_us":97190,"encode_duration_us":22075,"operation_id":"0x4bae","timeout_ms":10000,"last_server_duration_us":25015,"total_duration_us":10007791}],"total_count":28}} INFO [2022-01-10 05:42:50,186] jepsen worker 2 - jepsen.util 2432 :info :write [0 44] :RequestTimeoutException INFO [2022-01-10 05:42:50,187] jepsen worker 2 - jepsen.util 2522 :invoke :write [0 6] INFO [2022-01-10 05:42:50,248] jepsen worker 1 - jepsen.util 2431 :info :write [0 6] :SyncWriteAmbiguous INFO [2022-01-10 05:42:50,250] jepsen worker 1 - jepsen.util 2521 :invoke :write [0 41] INFO [2022-01-10 05:42:50,425] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:50,425] jepsen worker nemesis - couchbase.util Rebalance check count: 323 INFO [2022-01-10 05:42:50,427] jepsen worker nemesis - couchbase.util Rebalance stuck count: 260 INFO [2022-01-10 05:42:50,480] jepsen worker 83 - jepsen.util 2873 :ok :read [27 30] INFO [2022-01-10 05:42:50,481] jepsen worker 83 - jepsen.util 2873 :invoke :write [27 31] INFO [2022-01-10 05:42:50,526] jepsen worker 24 - jepsen.util 2094 :ok :read [8 27] INFO [2022-01-10 05:42:50,526] jepsen worker 24 - jepsen.util 2094 :invoke :read [8 nil] INFO [2022-01-10 05:42:50,552] jepsen worker 81 - jepsen.util 2691 :ok :read [27 30] INFO [2022-01-10 05:42:50,553] jepsen worker 81 - jepsen.util 2691 :invoke :write [27 35] INFO [2022-01-10 05:42:50,600] jepsen worker 13 - jepsen.util 2533 :info :write [4 9] :RequestTimeoutException INFO [2022-01-10 05:42:50,600] jepsen worker 13 - jepsen.util 2623 :invoke :write [4 7] INFO [2022-01-10 05:42:50,844] jepsen worker 89 - jepsen.util 2789 :info :write [29 34] :SyncWriteAmbiguous INFO [2022-01-10 05:42:50,844] jepsen worker 89 - jepsen.util 2879 :invoke :write [29 0] INFO [2022-01-10 05:42:50,962] jepsen worker 40 - jepsen.util 1840 :info :write [13 20] :RequestTimeoutException INFO [2022-01-10 05:42:50,963] jepsen worker 40 - jepsen.util 1930 :invoke :write [13 20] INFO [2022-01-10 05:42:50,985] jepsen worker 38 - jepsen.util 2648 :info :write [12 7] :SyncWriteAmbiguous INFO [2022-01-10 05:42:50,986] jepsen worker 38 - jepsen.util 2738 :invoke :write [12 11] INFO [2022-01-10 05:42:51,042] jepsen worker 3 - jepsen.util 2523 :ok :read [1 10] INFO [2022-01-10 05:42:51,043] jepsen worker 3 - jepsen.util 2523 :invoke :read [1 nil] INFO [2022-01-10 05:42:51,071] jepsen worker 75 - jepsen.util 1695 :info :write [25 49] :RequestTimeoutException INFO [2022-01-10 05:42:51,071] jepsen worker 75 - jepsen.util 1785 :invoke :write [25 41] INFO [2022-01-10 05:42:51,082] jepsen worker 66 - jepsen.util 2136 :info :write [22 13] :RequestTimeoutException INFO [2022-01-10 05:42:51,082] jepsen worker 66 - jepsen.util 2226 :invoke :read [22 nil] INFO [2022-01-10 05:42:51,117] jepsen worker 3 - jepsen.util 2523 :ok :read [1 10] INFO [2022-01-10 05:42:51,118] jepsen worker 3 - jepsen.util 2523 :invoke :write [1 25] INFO [2022-01-10 05:42:51,335] jepsen worker 86 - jepsen.util 1796 :info :write [28 18] :RequestTimeoutException INFO [2022-01-10 05:42:51,335] jepsen worker 86 - jepsen.util 1886 :invoke :read [28 nil] INFO [2022-01-10 05:42:51,434] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:51,435] jepsen worker nemesis - couchbase.util Rebalance check count: 324 INFO [2022-01-10 05:42:51,435] jepsen worker nemesis - couchbase.util Rebalance stuck count: 261 INFO [2022-01-10 05:42:51,500] jepsen worker 71 - jepsen.util 1961 :info :write [23 6] :RequestTimeoutException INFO [2022-01-10 05:42:51,501] jepsen worker 71 - jepsen.util 2051 :invoke :write [23 39] INFO [2022-01-10 05:42:51,545] jepsen worker 84 - jepsen.util 2244 :info :write [28 0] :RequestTimeoutException INFO [2022-01-10 05:42:51,545] jepsen worker 84 - jepsen.util 2334 :invoke :write [28 22] WARN [2022-01-10 05:42:51,623] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14186875,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14186875,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9035244,"encode_duration_us":23219,"operation_id":"0x4b82","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10001469}],"total_count":1}} WARN [2022-01-10 05:42:51,623] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11967637,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11967637,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9038428,"encode_duration_us":16700,"operation_id":"0x4b9e","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":10004781},{"operation_name":"upsert","last_dispatch_duration_us":17860812,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17860812,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9023573,"encode_duration_us":16699,"operation_id":"0x4a81","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10006758}],"total_count":2}} INFO [2022-01-10 05:42:51,712] jepsen worker 27 - jepsen.util 2367 :info :write [9 11] :RequestTimeoutException INFO [2022-01-10 05:42:51,712] jepsen worker 27 - jepsen.util 2457 :invoke :write [9 47] WARN [2022-01-10 05:42:51,724] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":8915020,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":8915020,"total_server_duration_us":1536,"operation_id":"0x4be9","timeout_ms":30000,"last_server_duration_us":1536,"total_duration_us":12551702},{"operation_name":"get","last_dispatch_duration_us":698695,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":698695,"total_server_duration_us":15,"operation_id":"0x4bdc","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":11471971},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":17177,"operation_id":"0x4cd0","timeout_ms":10000,"total_duration_us":10010573},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":18898,"operation_id":"0x4d4a","timeout_ms":10000,"total_duration_us":10008825},{"operation_name":"upsert","last_dispatch_duration_us":36693,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":36693,"total_server_duration_us":25712,"encode_duration_us":16374,"operation_id":"0x4c85","timeout_ms":10000,"last_server_duration_us":25712,"total_duration_us":10006173},{"operation_name":"upsert","last_dispatch_duration_us":11967637,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":11967637,"total_server_duration_us":9038428,"encode_duration_us":16700,"operation_id":"0x4b9e","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":10004781},{"operation_name":"upsert","last_dispatch_duration_us":9119013,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9119013,"total_server_duration_us":23475,"encode_duration_us":25145,"operation_id":"0x4c29","timeout_ms":10000,"last_server_duration_us":23475,"total_duration_us":10004650},{"operation_name":"upsert","last_dispatch_duration_us":2009304,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4914145,"total_server_duration_us":206146,"encode_duration_us":17478,"operation_id":"0x4d1f","timeout_ms":10000,"last_server_duration_us":130113,"total_duration_us":10004282},{"operation_name":"upsert","last_dispatch_duration_us":4853114,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4853114,"total_server_duration_us":153114,"encode_duration_us":16584,"operation_id":"0x4c92","timeout_ms":10000,"last_server_duration_us":153114,"total_duration_us":10002361},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":22353,"operation_id":"0x4c3e","timeout_ms":10000,"total_duration_us":10000513}],"total_count":17}} WARN [2022-01-10 05:42:51,724] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":2304505,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2304505,"total_server_duration_us":27,"operation_id":"0x48eb","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":30010829},{"operation_name":"get","last_dispatch_duration_us":2454018,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2454018,"total_server_duration_us":12055,"operation_id":"0x497c","timeout_ms":30000,"last_server_duration_us":12055,"total_duration_us":28022680},{"operation_name":"get","last_dispatch_duration_us":2095032,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2095032,"total_server_duration_us":15,"operation_id":"0x4ace","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":20518176},{"operation_name":"upsert","last_dispatch_duration_us":114163,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9609159,"total_server_duration_us":181333,"encode_duration_us":17074,"operation_id":"0x4d48","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10010995},{"operation_name":"upsert","last_dispatch_duration_us":139167,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9605385,"total_server_duration_us":202567,"encode_duration_us":16584,"operation_id":"0x4d4b","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10009168},{"operation_name":"upsert","last_dispatch_duration_us":29705,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":3044477,"total_server_duration_us":2617076,"encode_duration_us":16468,"operation_id":"0x4d1b","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10008639},{"operation_name":"upsert","last_dispatch_duration_us":32514,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":3848225,"total_server_duration_us":306340,"encode_duration_us":16332,"operation_id":"0x4c9f","timeout_ms":10000,"last_server_duration_us":20680,"total_duration_us":10008261},{"operation_name":"upsert","last_dispatch_duration_us":16217151,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":16217151,"total_server_duration_us":9028877,"encode_duration_us":18877,"operation_id":"0x4c09","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10006673},{"operation_name":"upsert","last_dispatch_duration_us":31878,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":7879699,"total_server_duration_us":266852,"encode_duration_us":16385,"operation_id":"0x4be0","timeout_ms":10000,"last_server_duration_us":20841,"total_duration_us":10001950},{"operation_name":"upsert","last_dispatch_duration_us":7964414,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":17669358,"total_server_duration_us":193602,"encode_duration_us":16552,"operation_id":"0x4bbb","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10001944}],"total_count":15}} WARN [2022-01-10 05:42:51,823] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":608940,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":2319818,"total_server_duration_us":678704,"encode_duration_us":26868,"operation_id":"0x4c12","timeout_ms":10000,"last_server_duration_us":24583,"total_duration_us":10006438},{"operation_name":"upsert","last_dispatch_duration_us":9031707,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10915571,"total_server_duration_us":265347,"encode_duration_us":16968,"operation_id":"0x4c68","timeout_ms":10000,"last_server_duration_us":40482,"total_duration_us":10005164},{"operation_name":"upsert","last_dispatch_duration_us":9238059,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10617882,"total_server_duration_us":144983,"encode_duration_us":18334,"operation_id":"0x4c69","timeout_ms":10000,"last_server_duration_us":111164,"total_duration_us":10004966},{"operation_name":"upsert","last_dispatch_duration_us":15193517,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":15193517,"total_server_duration_us":9029938,"encode_duration_us":16511,"operation_id":"0x4be7","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10003683},{"operation_name":"upsert","last_dispatch_duration_us":1296704,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":3523043,"total_server_duration_us":2064744,"encode_duration_us":16435,"operation_id":"0x4c72","timeout_ms":10000,"last_server_duration_us":606604,"total_duration_us":10001864},{"operation_name":"upsert","last_dispatch_duration_us":32718,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":5519602,"total_server_duration_us":372529,"encode_duration_us":16947,"operation_id":"0x4bc2","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10001559},{"operation_name":"upsert","last_dispatch_duration_us":630658,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9779222,"total_server_duration_us":196733,"encode_duration_us":16374,"operation_id":"0x4cce","timeout_ms":10000,"last_server_duration_us":120142,"total_duration_us":9800446},{"operation_name":"get","last_dispatch_duration_us":9314259,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9314259,"total_server_duration_us":351086,"operation_id":"0x4cc3","timeout_ms":30000,"last_server_duration_us":351086,"total_duration_us":9316502},{"operation_name":"get","last_dispatch_duration_us":9181546,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9181546,"total_server_duration_us":113463,"operation_id":"0x4ccd","timeout_ms":30000,"last_server_duration_us":113463,"total_duration_us":9181865},{"operation_name":"get","last_dispatch_duration_us":9107933,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9107933,"total_server_duration_us":7180,"operation_id":"0x4ccc","timeout_ms":30000,"last_server_duration_us":7180,"total_duration_us":9110338}],"total_count":13}} WARN [2022-01-10 05:42:51,823] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12366764,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17997262,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9136182,"encode_duration_us":18706,"operation_id":"0x4bed","timeout_ms":10000,"last_server_duration_us":9058604,"total_duration_us":10007137},{"operation_name":"upsert","last_dispatch_duration_us":32420611,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":32420611,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9041612,"encode_duration_us":16450,"operation_id":"0x4903","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":10010329},{"operation_name":"upsert","last_dispatch_duration_us":10714441,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":15829567,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":10175803,"encode_duration_us":17454,"operation_id":"0x4be8","timeout_ms":10000,"last_server_duration_us":9237837,"total_duration_us":10007995}],"total_count":3}} WARN [2022-01-10 05:42:51,823] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14247500,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":14247500,"total_server_duration_us":346831,"operation_id":"0x49a5","timeout_ms":30000,"last_server_duration_us":346831,"total_duration_us":21460420},{"operation_name":"get","last_dispatch_duration_us":3273871,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3273871,"total_server_duration_us":87950,"operation_id":"0x4b27","timeout_ms":30000,"last_server_duration_us":87950,"total_duration_us":14051183},{"operation_name":"get","last_dispatch_duration_us":13416343,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":13416343,"total_server_duration_us":6777,"operation_id":"0x4bad","timeout_ms":30000,"last_server_duration_us":6777,"total_duration_us":13418386},{"operation_name":"upsert","last_dispatch_duration_us":116744,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":5589276,"total_server_duration_us":41301,"encode_duration_us":17726,"operation_id":"0x4d26","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10010546},{"operation_name":"upsert","last_dispatch_duration_us":623872,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":3452917,"total_server_duration_us":1351266,"encode_duration_us":16861,"operation_id":"0x4cf2","timeout_ms":10000,"last_server_duration_us":24411,"total_duration_us":10009501},{"operation_name":"upsert","last_dispatch_duration_us":33514,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9444174,"total_server_duration_us":203462,"encode_duration_us":16935,"operation_id":"0x4bb3","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10009124},{"operation_name":"upsert","last_dispatch_duration_us":2423161,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7674812,"total_server_duration_us":602983,"encode_duration_us":16177,"operation_id":"0x4c97","timeout_ms":10000,"last_server_duration_us":53135,"total_duration_us":10003607},{"operation_name":"upsert","last_dispatch_duration_us":35094,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9447704,"total_server_duration_us":204669,"encode_duration_us":23549,"operation_id":"0x4bb8","timeout_ms":10000,"last_server_duration_us":23054,"total_duration_us":10002813},{"operation_name":"upsert","last_dispatch_duration_us":10079224,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10079224,"total_server_duration_us":194167,"encode_duration_us":17586,"operation_id":"0x4bde","timeout_ms":10000,"last_server_duration_us":194167,"total_duration_us":10001436},{"operation_name":"upsert","last_dispatch_duration_us":4473005,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4473005,"total_server_duration_us":107108,"encode_duration_us":17048,"operation_id":"0x4bec","timeout_ms":10000,"last_server_duration_us":107108,"total_duration_us":10000934}],"total_count":19}} WARN [2022-01-10 05:42:51,824] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":20251455,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":29385065,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":12161798,"encode_duration_us":16729,"operation_id":"0x4901","timeout_ms":10000,"last_server_duration_us":10193119,"total_duration_us":10003547},{"operation_name":"upsert","last_dispatch_duration_us":15193517,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15193517,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9029938,"encode_duration_us":16511,"operation_id":"0x4be7","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10003683}],"total_count":2}} WARN [2022-01-10 05:42:51,825] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16217151,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16217151,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9028877,"encode_duration_us":18877,"operation_id":"0x4c09","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10006673},{"operation_name":"get","last_dispatch_duration_us":2304505,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":2304505,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":27,"operation_id":"0x48eb","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":30010829},{"operation_name":"upsert","last_dispatch_duration_us":33184076,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":33184076,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9449908,"encode_duration_us":27104,"operation_id":"0x4808","timeout_ms":10000,"last_server_duration_us":9449908,"total_duration_us":10007567}],"total_count":3}} INFO [2022-01-10 05:42:51,825] jepsen worker 77 - jepsen.util 2777 :ok :read [25 :nil] INFO [2022-01-10 05:42:51,825] jepsen worker 77 - jepsen.util 2777 :invoke :write [25 36] INFO [2022-01-10 05:42:51,834] jepsen worker 54 - jepsen.util 2484 :info :write [18 10] :RequestTimeoutException INFO [2022-01-10 05:42:51,835] jepsen worker 54 - jepsen.util 2574 :invoke :write [18 3] WARN [2022-01-10 05:42:51,924] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":29119318,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":29119318,"total_server_duration_us":56,"operation_id":"0x49ac","timeout_ms":30000,"last_server_duration_us":56,"total_duration_us":29120338},{"operation_name":"get","last_dispatch_duration_us":27107021,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":27107021,"total_server_duration_us":6096,"operation_id":"0x49e8","timeout_ms":30000,"last_server_duration_us":6096,"total_duration_us":27671082},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":17092,"operation_id":"0x4c02","timeout_ms":10000,"total_duration_us":10010757},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":17560,"operation_id":"0x4d58","timeout_ms":10000,"total_duration_us":10009410},{"operation_name":"upsert","last_dispatch_duration_us":3312538,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8947662,"total_server_duration_us":362620,"encode_duration_us":17434,"operation_id":"0x4bee","timeout_ms":10000,"last_server_duration_us":25888,"total_duration_us":10009183},{"operation_name":"upsert","last_dispatch_duration_us":10514715,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":17332034,"total_server_duration_us":451658,"encode_duration_us":16597,"operation_id":"0x4c1c","timeout_ms":10000,"last_server_duration_us":66037,"total_duration_us":10008917},{"operation_name":"upsert","last_dispatch_duration_us":11610917,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":15986325,"total_server_duration_us":1033368,"encode_duration_us":17330,"operation_id":"0x4bef","timeout_ms":10000,"last_server_duration_us":87064,"total_duration_us":10008240},{"operation_name":"upsert","last_dispatch_duration_us":10714441,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":15829567,"total_server_duration_us":10175803,"encode_duration_us":17454,"operation_id":"0x4be8","timeout_ms":10000,"last_server_duration_us":9237837,"total_duration_us":10007995},{"operation_name":"upsert","last_dispatch_duration_us":12366764,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":17997262,"total_server_duration_us":9136182,"encode_duration_us":18706,"operation_id":"0x4bed","timeout_ms":10000,"last_server_duration_us":9058604,"total_duration_us":10007137},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":17293,"operation_id":"0x4cca","timeout_ms":10000,"total_duration_us":10006234}],"total_count":15}} INFO [2022-01-10 05:42:52,241] jepsen worker 15 - jepsen.util 2085 :info :write [5 45] :RequestTimeoutException INFO [2022-01-10 05:42:52,243] jepsen worker 15 - jepsen.util 2175 :invoke :read [5 nil] INFO [2022-01-10 05:42:52,320] jepsen worker 5 - jepsen.util 2525 :info :write [1 10] :SyncWriteAmbiguous INFO [2022-01-10 05:42:52,321] jepsen worker 5 - jepsen.util 2615 :invoke :read [1 nil] INFO [2022-01-10 05:42:52,442] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:52,442] jepsen worker nemesis - couchbase.util Rebalance check count: 325 INFO [2022-01-10 05:42:52,442] jepsen worker nemesis - couchbase.util Rebalance stuck count: 262 INFO [2022-01-10 05:42:52,526] jepsen worker 31 - jepsen.util 2821 :info :write [10 15] :RequestTimeoutException INFO [2022-01-10 05:42:52,527] jepsen worker 31 - jepsen.util 2911 :invoke :read [10 nil] INFO [2022-01-10 05:42:52,620] jepsen worker 57 - jepsen.util 2847 :info :write [19 33] :RequestTimeoutException INFO [2022-01-10 05:42:52,621] jepsen worker 57 - jepsen.util 2937 :invoke :write [19 42] INFO [2022-01-10 05:42:52,665] jepsen worker 88 - jepsen.util 2338 :info :write [29 34] :RequestTimeoutException INFO [2022-01-10 05:42:52,666] jepsen worker 88 - jepsen.util 2428 :invoke :read [29 nil] INFO [2022-01-10 05:42:52,744] jepsen worker 9 - jepsen.util 2889 :info :write [3 12] :SyncWriteAmbiguous INFO [2022-01-10 05:42:52,745] jepsen worker 9 - jepsen.util 2979 :invoke :write [3 37] INFO [2022-01-10 05:42:52,780] jepsen worker 61 - jepsen.util 2401 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:52,781] jepsen worker 61 - jepsen.util 2401 :invoke :write [20 32] INFO [2022-01-10 05:42:52,934] jepsen worker 88 - jepsen.util 2428 :ok :read [29 0] INFO [2022-01-10 05:42:52,934] jepsen worker 88 - jepsen.util 2428 :invoke :write [29 34] INFO [2022-01-10 05:42:52,980] jepsen worker 69 - jepsen.util 2499 :ok :read [23 26] INFO [2022-01-10 05:42:52,980] jepsen worker 69 - jepsen.util 2499 :invoke :read [23 nil] INFO [2022-01-10 05:42:52,996] jepsen worker 51 - jepsen.util 2751 :info :write [17 46] :RequestTimeoutException INFO [2022-01-10 05:42:52,997] jepsen worker 51 - jepsen.util 2841 :invoke :read [17 nil] INFO [2022-01-10 05:42:53,097] jepsen worker 70 - jepsen.util 2590 :ok :read [23 26] INFO [2022-01-10 05:42:53,098] jepsen worker 70 - jepsen.util 2590 :invoke :read [23 nil] INFO [2022-01-10 05:42:53,111] jepsen worker 45 - jepsen.util 2835 :info :write [15 31] :RequestTimeoutException INFO [2022-01-10 05:42:53,112] jepsen worker 45 - jepsen.util 2925 :invoke :read [15 nil] INFO [2022-01-10 05:42:53,126] jepsen worker 5 - jepsen.util 2615 :ok :read [1 10] INFO [2022-01-10 05:42:53,126] jepsen worker 5 - jepsen.util 2615 :invoke :read [1 nil] INFO [2022-01-10 05:42:53,133] jepsen worker 69 - jepsen.util 2499 :ok :read [23 26] INFO [2022-01-10 05:42:53,133] jepsen worker 69 - jepsen.util 2499 :invoke :read [23 nil] INFO [2022-01-10 05:42:53,144] jepsen worker 55 - jepsen.util 2755 :info :write [18 37] :RequestTimeoutException INFO [2022-01-10 05:42:53,145] jepsen worker 55 - jepsen.util 2845 :invoke :read [18 nil] INFO [2022-01-10 05:42:53,193] jepsen worker 70 - jepsen.util 2590 :ok :read [23 26] INFO [2022-01-10 05:42:53,193] jepsen worker 70 - jepsen.util 2590 :invoke :write [23 28] INFO [2022-01-10 05:42:53,195] jepsen worker 58 - jepsen.util 2398 :info :write [19 26] :RequestTimeoutException INFO [2022-01-10 05:42:53,196] jepsen worker 58 - jepsen.util 2488 :invoke :read [19 nil] INFO [2022-01-10 05:42:53,209] jepsen worker 44 - jepsen.util 2384 :info :write [14 31] :RequestTimeoutException INFO [2022-01-10 05:42:53,210] jepsen worker 44 - jepsen.util 2474 :invoke :read [14 nil] INFO [2022-01-10 05:42:53,222] jepsen worker 5 - jepsen.util 2615 :ok :read [1 10] INFO [2022-01-10 05:42:53,222] jepsen worker 5 - jepsen.util 2615 :invoke :write [1 49] INFO [2022-01-10 05:42:53,235] jepsen worker 69 - jepsen.util 2499 :ok :read [23 26] INFO [2022-01-10 05:42:53,235] jepsen worker 69 - jepsen.util 2499 :invoke :read [23 nil] INFO [2022-01-10 05:42:53,336] jepsen worker 30 - jepsen.util 2280 :info :write [10 43] :RequestTimeoutException INFO [2022-01-10 05:42:53,337] jepsen worker 30 - jepsen.util 2370 :invoke :write [10 38] INFO [2022-01-10 05:42:53,347] jepsen worker 69 - jepsen.util 2499 :ok :read [23 26] INFO [2022-01-10 05:42:53,347] jepsen worker 69 - jepsen.util 2499 :invoke :read [23 nil] INFO [2022-01-10 05:42:53,381] jepsen worker 72 - jepsen.util 2592 :info :write [24 49] :RequestTimeoutException INFO [2022-01-10 05:42:53,381] jepsen worker 72 - jepsen.util 2682 :invoke :read [24 nil] INFO [2022-01-10 05:42:53,448] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:53,448] jepsen worker nemesis - couchbase.util Rebalance check count: 326 INFO [2022-01-10 05:42:53,448] jepsen worker nemesis - couchbase.util Rebalance stuck count: 263 INFO [2022-01-10 05:42:53,467] jepsen worker 69 - jepsen.util 2499 :ok :read [23 26] INFO [2022-01-10 05:42:53,468] jepsen worker 69 - jepsen.util 2499 :invoke :read [23 nil] INFO [2022-01-10 05:42:53,480] jepsen worker 8 - jepsen.util 2708 :info :write [2 34] :RequestTimeoutException INFO [2022-01-10 05:42:53,480] jepsen worker 8 - jepsen.util 2798 :invoke :write [2 13] INFO [2022-01-10 05:42:53,583] jepsen worker 69 - jepsen.util 2499 :ok :read [23 26] INFO [2022-01-10 05:42:53,584] jepsen worker 69 - jepsen.util 2499 :invoke :read [23 nil] INFO [2022-01-10 05:42:53,636] jepsen worker 65 - jepsen.util 2225 :info :write [21 41] :RequestTimeoutException INFO [2022-01-10 05:42:53,636] jepsen worker 65 - jepsen.util 2315 :invoke :read [21 nil] INFO [2022-01-10 05:42:53,701] jepsen worker 69 - jepsen.util 2499 :ok :read [23 26] INFO [2022-01-10 05:42:53,701] jepsen worker 69 - jepsen.util 2499 :invoke :read [23 nil] INFO [2022-01-10 05:42:53,751] jepsen worker 11 - jepsen.util 2441 :info :write [3 17] :RequestTimeoutException INFO [2022-01-10 05:42:53,752] jepsen worker 11 - jepsen.util 2531 :invoke :write [3 3] INFO [2022-01-10 05:42:53,805] jepsen worker 69 - jepsen.util 2499 :ok :read [23 26] INFO [2022-01-10 05:42:53,806] jepsen worker 0 - jepsen.util 2430 :info :write [0 25] :RequestTimeoutException INFO [2022-01-10 05:42:53,806] jepsen worker 69 - jepsen.util 2499 :invoke :read [23 nil] INFO [2022-01-10 05:42:53,806] jepsen worker 0 - jepsen.util 2520 :invoke :read [0 nil] INFO [2022-01-10 05:42:53,867] jepsen worker 51 - jepsen.util 2841 :ok :read [17 41] INFO [2022-01-10 05:42:53,868] jepsen worker 51 - jepsen.util 2841 :invoke :write [17 25] INFO [2022-01-10 05:42:53,871] jepsen worker 69 - jepsen.util 2499 :ok :read [23 26] INFO [2022-01-10 05:42:53,872] jepsen worker 69 - jepsen.util 2499 :invoke :write [23 7] INFO [2022-01-10 05:42:53,891] jepsen worker 87 - jepsen.util 2517 :info :write [29 11] :RequestTimeoutException INFO [2022-01-10 05:42:53,892] jepsen worker 87 - jepsen.util 2607 :invoke :write [29 42] INFO [2022-01-10 05:42:53,975] jepsen worker 52 - jepsen.util 3022 :info :write [17 38] :RequestTimeoutException INFO [2022-01-10 05:42:53,975] jepsen worker 52 - jepsen.util 3112 :invoke :write [17 35] INFO [2022-01-10 05:42:54,259] jepsen worker 65 - jepsen.util 2315 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:54,260] jepsen worker 65 - jepsen.util 2315 :invoke :read [21 nil] INFO [2022-01-10 05:42:54,261] jepsen worker 65 - jepsen.util 2315 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:54,262] jepsen worker 65 - jepsen.util 2315 :invoke :write [21 24] INFO [2022-01-10 05:42:54,279] jepsen worker 24 - jepsen.util 2094 :ok :read [8 27] INFO [2022-01-10 05:42:54,280] jepsen worker 24 - jepsen.util 2094 :invoke :read [8 nil] INFO [2022-01-10 05:42:54,316] jepsen worker 20 - jepsen.util 2090 :info :write [6 4] :RequestTimeoutException INFO [2022-01-10 05:42:54,317] jepsen worker 20 - jepsen.util 2180 :invoke :write [6 5] INFO [2022-01-10 05:42:54,343] jepsen worker 72 - jepsen.util 2682 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:54,344] jepsen worker 72 - jepsen.util 2682 :invoke :write [24 32] INFO [2022-01-10 05:42:54,350] jepsen worker 6 - jepsen.util 2436 :info :write [2 5] :RequestTimeoutException INFO [2022-01-10 05:42:54,351] jepsen worker 6 - jepsen.util 2526 :invoke :read [2 nil] INFO [2022-01-10 05:42:54,351] jepsen worker 26 - jepsen.util 2006 :info :write [8 17] :RequestTimeoutException INFO [2022-01-10 05:42:54,352] jepsen worker 26 - jepsen.util 2096 :invoke :read [8 nil] INFO [2022-01-10 05:42:54,361] jepsen worker 25 - jepsen.util 2275 :info :write [8 43] :RequestTimeoutException INFO [2022-01-10 05:42:54,362] jepsen worker 25 - jepsen.util 2365 :invoke :read [8 nil] INFO [2022-01-10 05:42:54,393] jepsen worker 6 - jepsen.util 2526 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:54,393] jepsen worker 6 - jepsen.util 2526 :invoke :write [2 27] INFO [2022-01-10 05:42:54,454] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3625730994152047}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3470588235294118}} INFO [2022-01-10 05:42:54,455] jepsen worker nemesis - couchbase.util Rebalance check count: 327 INFO [2022-01-10 05:42:54,455] jepsen worker nemesis - couchbase.util Rebalance stuck count: 264 INFO [2022-01-10 05:42:54,744] jepsen worker 49 - jepsen.util 2299 :info :write [16 18] :SyncWriteAmbiguous INFO [2022-01-10 05:42:54,745] jepsen worker 49 - jepsen.util 2389 :invoke :write [16 3] INFO [2022-01-10 05:42:54,772] jepsen worker 37 - jepsen.util 2557 :info :write [12 31] :RequestTimeoutException INFO [2022-01-10 05:42:54,772] jepsen worker 37 - jepsen.util 2647 :invoke :read [12 nil] INFO [2022-01-10 05:42:54,782] jepsen worker 67 - jepsen.util 2317 :ok :read [22 41] INFO [2022-01-10 05:42:54,783] jepsen worker 67 - jepsen.util 2317 :invoke :write [22 5] INFO [2022-01-10 05:42:54,786] jepsen worker 34 - jepsen.util 2194 :info :write [11 8] :RequestTimeoutException INFO [2022-01-10 05:42:54,787] jepsen worker 34 - jepsen.util 2284 :invoke :read [11 nil] INFO [2022-01-10 05:42:54,790] jepsen worker 39 - jepsen.util 2829 :ok :read [13 15] INFO [2022-01-10 05:42:54,791] jepsen worker 39 - jepsen.util 2829 :invoke :read [13 nil] INFO [2022-01-10 05:42:54,837] jepsen worker 56 - jepsen.util 2576 :info :write [18 26] :RequestTimeoutException INFO [2022-01-10 05:42:54,838] jepsen worker 56 - jepsen.util 2666 :invoke :write [18 1] INFO [2022-01-10 05:42:54,840] jepsen worker 86 - jepsen.util 1886 :ok :read [28 45] INFO [2022-01-10 05:42:54,841] jepsen worker 86 - jepsen.util 1886 :invoke :write [28 15] INFO [2022-01-10 05:42:54,858] jepsen worker 39 - jepsen.util 2829 :ok :read [13 15] INFO [2022-01-10 05:42:54,859] jepsen worker 39 - jepsen.util 2829 :invoke :read [13 nil] INFO [2022-01-10 05:42:54,891] jepsen worker 48 - jepsen.util 2118 :ok :read [16 38] INFO [2022-01-10 05:42:54,892] jepsen worker 48 - jepsen.util 2118 :invoke :read [16 nil] INFO [2022-01-10 05:42:54,965] jepsen worker 60 - jepsen.util 2760 :info :write [20 31] :RequestTimeoutException INFO [2022-01-10 05:42:54,966] jepsen worker 60 - jepsen.util 2850 :invoke :write [20 28] INFO [2022-01-10 05:42:54,981] jepsen worker 39 - jepsen.util 2829 :ok :read [13 15] INFO [2022-01-10 05:42:54,981] jepsen worker 39 - jepsen.util 2829 :invoke :read [13 nil] INFO [2022-01-10 05:42:55,086] jepsen worker 39 - jepsen.util 2829 :ok :read [13 15] INFO [2022-01-10 05:42:55,087] jepsen worker 39 - jepsen.util 2829 :invoke :read [13 nil] INFO [2022-01-10 05:42:55,203] jepsen worker 39 - jepsen.util 2829 :ok :read [13 15] INFO [2022-01-10 05:42:55,204] jepsen worker 39 - jepsen.util 2829 :invoke :write [13 28] INFO [2022-01-10 05:42:55,405] jepsen worker 36 - jepsen.util 2106 :info :write [12 3] :RequestTimeoutException INFO [2022-01-10 05:42:55,405] jepsen worker 36 - jepsen.util 2196 :invoke :read [12 nil] INFO [2022-01-10 05:42:55,449] jepsen worker 36 - jepsen.util 2196 :ok :read [12 29] INFO [2022-01-10 05:42:55,450] jepsen worker 36 - jepsen.util 2196 :invoke :read [12 nil] INFO [2022-01-10 05:42:55,472] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.368421052631579}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3529411764705882}} INFO [2022-01-10 05:42:55,474] jepsen worker 4 - jepsen.util 2704 :info :write [1 46] :RequestTimeoutException INFO [2022-01-10 05:42:55,534] jepsen worker 7 - jepsen.util 2437 :info :write [2 44] :RequestTimeoutException INFO [2022-01-10 05:42:55,612] jepsen worker nemesis - couchbase.util Rebalance check count: 328 INFO [2022-01-10 05:42:55,612] jepsen worker 4 - jepsen.util 2794 :invoke :write [1 6] INFO [2022-01-10 05:42:55,612] jepsen worker nemesis - couchbase.util Rebalance stuck count: 264 INFO [2022-01-10 05:42:55,612] jepsen worker 7 - jepsen.util 2527 :invoke :write [2 15] INFO [2022-01-10 05:42:55,636] jepsen worker 36 - jepsen.util 2196 :ok :read [12 29] INFO [2022-01-10 05:42:55,637] jepsen worker 36 - jepsen.util 2196 :invoke :read [12 nil] INFO [2022-01-10 05:42:55,661] jepsen worker 36 - jepsen.util 2196 :ok :read [12 29] INFO [2022-01-10 05:42:55,662] jepsen worker 36 - jepsen.util 2196 :invoke :write [12 35] INFO [2022-01-10 05:42:55,716] jepsen worker 82 - jepsen.util 2422 :info :write [27 26] :RequestTimeoutException INFO [2022-01-10 05:42:55,717] jepsen worker 82 - jepsen.util 2512 :invoke :write [27 24] INFO [2022-01-10 05:42:55,801] jepsen worker 28 - jepsen.util 2278 :info :write [9 37] :RequestTimeoutException INFO [2022-01-10 05:42:55,802] jepsen worker 28 - jepsen.util 2368 :invoke :write [9 18] INFO [2022-01-10 05:42:56,214] jepsen worker 34 - jepsen.util 2284 :ok :read [11 21] INFO [2022-01-10 05:42:56,214] jepsen worker 34 - jepsen.util 2284 :invoke :write [11 20] INFO [2022-01-10 05:42:56,275] jepsen worker 73 - jepsen.util 2233 :info :write [24 47] :RequestTimeoutException INFO [2022-01-10 05:42:56,275] jepsen worker 73 - jepsen.util 2323 :invoke :read [24 nil] INFO [2022-01-10 05:42:56,618] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.368421052631579}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3529411764705882}} INFO [2022-01-10 05:42:56,618] jepsen worker nemesis - couchbase.util Rebalance check count: 329 INFO [2022-01-10 05:42:56,618] jepsen worker nemesis - couchbase.util Rebalance stuck count: 265 INFO [2022-01-10 05:42:56,660] jepsen worker 12 - jepsen.util 2532 :info :write [4 17] :RequestTimeoutException INFO [2022-01-10 05:42:56,661] jepsen worker 12 - jepsen.util 2622 :invoke :read [4 nil] INFO [2022-01-10 05:42:56,685] jepsen worker 12 - jepsen.util 2622 :ok :read [4 3] INFO [2022-01-10 05:42:56,685] jepsen worker 12 - jepsen.util 2622 :invoke :read [4 nil] INFO [2022-01-10 05:42:56,709] jepsen worker 12 - jepsen.util 2622 :ok :read [4 3] INFO [2022-01-10 05:42:56,709] jepsen worker 12 - jepsen.util 2622 :invoke :read [4 nil] INFO [2022-01-10 05:42:56,737] jepsen worker 12 - jepsen.util 2622 :ok :read [4 3] INFO [2022-01-10 05:42:56,737] jepsen worker 12 - jepsen.util 2622 :invoke :read [4 nil] INFO [2022-01-10 05:42:56,761] jepsen worker 12 - jepsen.util 2622 :ok :read [4 3] INFO [2022-01-10 05:42:56,762] jepsen worker 12 - jepsen.util 2622 :invoke :write [4 17] INFO [2022-01-10 05:42:56,917] jepsen worker 18 - jepsen.util 2178 :info :write [6 2] :RequestTimeoutException INFO [2022-01-10 05:42:56,918] jepsen worker 18 - jepsen.util 2268 :invoke :write [6 8] INFO [2022-01-10 05:42:57,062] jepsen worker 35 - jepsen.util 2015 :info :write [11 1] :RequestTimeoutException INFO [2022-01-10 05:42:57,062] jepsen worker 35 - jepsen.util 2105 :invoke :read [11 nil] INFO [2022-01-10 05:42:57,117] jepsen worker 45 - jepsen.util 2925 :ok :read [15 23] INFO [2022-01-10 05:42:57,117] jepsen worker 45 - jepsen.util 2925 :invoke :read [15 nil] INFO [2022-01-10 05:42:57,131] jepsen worker 14 - jepsen.util 2894 :info :write [4 40] :RequestTimeoutException INFO [2022-01-10 05:42:57,132] jepsen worker 14 - jepsen.util 2984 :invoke :read [4 nil] INFO [2022-01-10 05:42:57,275] jepsen worker 10 - jepsen.util 2710 :info :write [3 28] :RequestTimeoutException INFO [2022-01-10 05:42:57,276] jepsen worker 10 - jepsen.util 2800 :invoke :write [3 16] INFO [2022-01-10 05:42:57,457] jepsen worker 47 - jepsen.util 2297 :info :write [15 27] :RequestTimeoutException INFO [2022-01-10 05:42:57,457] jepsen worker 47 - jepsen.util 2387 :invoke :write [15 10] INFO [2022-01-10 05:42:57,496] jepsen worker 78 - jepsen.util 2778 :info :write [26 24] :SyncWriteAmbiguous INFO [2022-01-10 05:42:57,496] jepsen worker 78 - jepsen.util 2868 :invoke :read [26 nil] INFO [2022-01-10 05:42:57,519] jepsen worker 78 - jepsen.util 2868 :ok :read [26 10] INFO [2022-01-10 05:42:57,520] jepsen worker 78 - jepsen.util 2868 :invoke :read [26 nil] INFO [2022-01-10 05:42:57,542] jepsen worker 78 - jepsen.util 2868 :ok :read [26 10] INFO [2022-01-10 05:42:57,543] jepsen worker 78 - jepsen.util 2868 :invoke :write [26 26] INFO [2022-01-10 05:42:57,551] jepsen worker 79 - jepsen.util 2869 :ok :read [26 10] INFO [2022-01-10 05:42:57,552] jepsen worker 79 - jepsen.util 2869 :invoke :write [26 43] INFO [2022-01-10 05:42:57,624] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.368421052631579}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3529411764705882}} INFO [2022-01-10 05:42:57,624] jepsen worker nemesis - couchbase.util Rebalance check count: 330 INFO [2022-01-10 05:42:57,624] jepsen worker nemesis - couchbase.util Rebalance stuck count: 266 INFO [2022-01-10 05:42:57,647] jepsen worker 35 - jepsen.util 2105 :ok :read [11 21] INFO [2022-01-10 05:42:57,648] jepsen worker 35 - jepsen.util 2105 :invoke :read [11 nil] INFO [2022-01-10 05:42:57,741] jepsen worker 43 - jepsen.util 2203 :info :write [14 19] :RequestTimeoutException INFO [2022-01-10 05:42:57,742] jepsen worker 43 - jepsen.util 2293 :invoke :read [14 nil] INFO [2022-01-10 05:42:57,801] jepsen worker 50 - jepsen.util 2570 :ok :read [16 38] INFO [2022-01-10 05:42:57,801] jepsen worker 50 - jepsen.util 2570 :invoke :read [16 nil] INFO [2022-01-10 05:42:57,815] jepsen worker 59 - jepsen.util 2579 :ok :read [19 31] INFO [2022-01-10 05:42:57,816] jepsen worker 59 - jepsen.util 2579 :invoke :write [19 41] INFO [2022-01-10 05:42:57,870] jepsen worker 80 - jepsen.util 2870 :info :write [26 36] :RequestTimeoutException INFO [2022-01-10 05:42:57,872] jepsen worker 80 - jepsen.util 2960 :invoke :write [26 10] INFO [2022-01-10 05:42:57,891] jepsen worker 66 - jepsen.util 2226 :ok :read [22 41] INFO [2022-01-10 05:42:57,892] jepsen worker 66 - jepsen.util 2226 :invoke :read [22 nil] INFO [2022-01-10 05:42:57,931] jepsen worker 15 - jepsen.util 2175 :ok :read [5 42] INFO [2022-01-10 05:42:57,931] jepsen worker 15 - jepsen.util 2175 :invoke :write [5 13] INFO [2022-01-10 05:42:57,970] jepsen worker 50 - jepsen.util 2570 :ok :read [16 38] INFO [2022-01-10 05:42:57,971] jepsen worker 50 - jepsen.util 2570 :invoke :write [16 34] INFO [2022-01-10 05:42:57,985] jepsen worker 66 - jepsen.util 2226 :ok :read [22 41] INFO [2022-01-10 05:42:57,985] jepsen worker 29 - jepsen.util 2369 :info :write [9 26] :RequestTimeoutException INFO [2022-01-10 05:42:57,985] jepsen worker 66 - jepsen.util 2226 :invoke :write [22 17] INFO [2022-01-10 05:42:57,986] jepsen worker 29 - jepsen.util 2459 :invoke :read [9 nil] INFO [2022-01-10 05:42:57,991] jepsen worker 46 - jepsen.util 2656 :info :write [15 26] :RequestTimeoutException INFO [2022-01-10 05:42:57,991] jepsen worker 22 - jepsen.util 2182 :info :write [7 42] :RequestTimeoutException INFO [2022-01-10 05:42:57,992] jepsen worker 19 - jepsen.util 2539 :info :write [6 7] :RequestTimeoutException INFO [2022-01-10 05:42:57,992] jepsen worker 19 - jepsen.util 2629 :invoke :read [6 nil] INFO [2022-01-10 05:42:57,993] jepsen worker 22 - jepsen.util 2272 :invoke :read [7 nil] INFO [2022-01-10 05:42:57,993] jepsen worker 46 - jepsen.util 2746 :invoke :read [15 nil] INFO [2022-01-10 05:42:58,002] jepsen worker 23 - jepsen.util 2453 :info :write [7 30] :RequestTimeoutException INFO [2022-01-10 05:42:58,003] jepsen worker 23 - jepsen.util 2543 :invoke :read [7 nil] INFO [2022-01-10 05:42:58,082] jepsen worker 35 - jepsen.util 2105 :ok :read [11 21] INFO [2022-01-10 05:42:58,083] jepsen worker 35 - jepsen.util 2105 :invoke :read [11 nil] INFO [2022-01-10 05:42:58,176] jepsen worker 62 - jepsen.util 2582 :info :write [20 37] :RequestTimeoutException INFO [2022-01-10 05:42:58,176] jepsen worker 64 - jepsen.util 2314 :info :write [21 32] :RequestTimeoutException INFO [2022-01-10 05:42:58,176] jepsen worker 74 - jepsen.util 2954 :info :write [24 35] :RequestTimeoutException INFO [2022-01-10 05:42:58,177] jepsen worker 64 - jepsen.util 2404 :invoke :read [21 nil] INFO [2022-01-10 05:42:58,177] jepsen worker 62 - jepsen.util 2672 :invoke :write [20 25] INFO [2022-01-10 05:42:58,178] jepsen worker 74 - jepsen.util 3044 :invoke :read [24 nil] INFO [2022-01-10 05:42:58,344] jepsen worker 43 - jepsen.util 2293 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:58,345] jepsen worker 43 - jepsen.util 2293 :invoke :write [14 27] INFO [2022-01-10 05:42:58,398] jepsen worker 35 - jepsen.util 2105 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:58,398] jepsen worker 64 - jepsen.util 2404 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:58,399] jepsen worker 64 - jepsen.util 2404 :invoke :write [21 41] INFO [2022-01-10 05:42:58,400] jepsen worker 35 - jepsen.util 2105 :invoke :write [11 22] INFO [2022-01-10 05:42:58,430] jepsen worker 19 - jepsen.util 2629 :ok :read [6 25] INFO [2022-01-10 05:42:58,430] jepsen worker 19 - jepsen.util 2629 :invoke :write [6 1] INFO [2022-01-10 05:42:58,446] jepsen worker 32 - jepsen.util 2552 :info :write [10 2] :RequestTimeoutException INFO [2022-01-10 05:42:58,447] jepsen worker 32 - jepsen.util 2642 :invoke :write [10 2] INFO [2022-01-10 05:42:58,483] jepsen worker 74 - jepsen.util 3044 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:58,483] jepsen worker 74 - jepsen.util 3044 :invoke :read [24 nil] INFO [2022-01-10 05:42:58,581] jepsen worker 23 - jepsen.util 2543 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:58,583] jepsen worker 23 - jepsen.util 2543 :invoke :write [7 16] INFO [2022-01-10 05:42:58,640] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.368421052631579}, :ns_1@172.23.107.227 {:progress 0.2404692082111437}, :ns_1@172.23.105.187 {:progress 0.2404692082111437}, :ns_1@172.23.107.228 {:progress 0.3529411764705882}} INFO [2022-01-10 05:42:58,640] jepsen worker nemesis - couchbase.util Rebalance check count: 331 INFO [2022-01-10 05:42:58,640] jepsen worker nemesis - couchbase.util Rebalance stuck count: 267 INFO [2022-01-10 05:42:58,706] jepsen worker 74 - jepsen.util 3044 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:58,707] jepsen worker 74 - jepsen.util 3044 :invoke :read [24 nil] INFO [2022-01-10 05:42:58,708] jepsen worker 74 - jepsen.util 3044 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:58,709] jepsen worker 74 - jepsen.util 3044 :invoke :read [24 nil] INFO [2022-01-10 05:42:58,750] jepsen worker 74 - jepsen.util 3044 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:58,751] jepsen worker 74 - jepsen.util 3044 :invoke :write [24 30] INFO [2022-01-10 05:42:58,932] jepsen worker 53 - jepsen.util 2933 :info :write [17 16] :RequestTimeoutException INFO [2022-01-10 05:42:58,933] jepsen worker 53 - jepsen.util 3023 :invoke :read [17 nil] INFO [2022-01-10 05:42:58,947] jepsen worker 76 - jepsen.util 1876 :info :write [25 22] :RequestTimeoutException INFO [2022-01-10 05:42:58,948] jepsen worker 76 - jepsen.util 1966 :invoke :write [25 10] INFO [2022-01-10 05:42:58,973] jepsen worker 53 - jepsen.util 3023 :ok :read [17 41] INFO [2022-01-10 05:42:58,973] jepsen worker 53 - jepsen.util 3023 :invoke :write [17 20] INFO [2022-01-10 05:42:59,154] jepsen worker 68 - jepsen.util 2498 :ok :read [22 41] INFO [2022-01-10 05:42:59,155] jepsen worker 68 - jepsen.util 2498 :invoke :read [22 nil] INFO [2022-01-10 05:42:59,161] jepsen worker 17 - jepsen.util 2537 :ok :read [5 42] INFO [2022-01-10 05:42:59,161] jepsen worker 17 - jepsen.util 2537 :invoke :read [5 nil] INFO [2022-01-10 05:42:59,164] jepsen worker 42 - jepsen.util 2292 :info :write [14 10] :RequestTimeoutException INFO [2022-01-10 05:42:59,166] jepsen worker 42 - jepsen.util 2382 :invoke :read [14 nil] INFO [2022-01-10 05:42:59,235] jepsen worker 33 - jepsen.util 2733 :info :write [11 47] :RequestTimeoutException INFO [2022-01-10 05:42:59,236] jepsen worker 33 - jepsen.util 2823 :invoke :write [11 44] INFO [2022-01-10 05:42:59,423] jepsen worker 16 - jepsen.util 2536 :ok :read [5 42] INFO [2022-01-10 05:42:59,424] jepsen worker 16 - jepsen.util 2536 :invoke :write [5 14] INFO [2022-01-10 05:42:59,436] jepsen worker 68 - jepsen.util 2498 :ok :read [22 41] INFO [2022-01-10 05:42:59,436] jepsen worker 68 - jepsen.util 2498 :invoke :write [22 3] INFO [2022-01-10 05:42:59,457] jepsen worker 17 - jepsen.util 2537 :ok :read [5 42] INFO [2022-01-10 05:42:59,457] jepsen worker 17 - jepsen.util 2537 :invoke :write [5 4] INFO [2022-01-10 05:42:59,646] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.368421052631579}, :ns_1@172.23.107.227 {:progress 0.2434017595307918}, :ns_1@172.23.105.187 {:progress 0.2434017595307918}, :ns_1@172.23.107.228 {:progress 0.3529411764705882}} INFO [2022-01-10 05:42:59,646] jepsen worker nemesis - couchbase.util Rebalance check count: 332 INFO [2022-01-10 05:42:59,647] jepsen worker nemesis - couchbase.util Rebalance stuck count: 267 INFO [2022-01-10 05:42:59,668] jepsen worker 13 - jepsen.util 2623 :info :write [4 7] :SyncWriteAmbiguous INFO [2022-01-10 05:42:59,668] jepsen worker 13 - jepsen.util 2713 :invoke :read [4 nil] INFO [2022-01-10 05:42:59,693] jepsen worker 13 - jepsen.util 2713 :ok :read [4 3] INFO [2022-01-10 05:42:59,694] jepsen worker 13 - jepsen.util 2713 :invoke :write [4 20] INFO [2022-01-10 05:42:59,703] jepsen worker 42 - jepsen.util 2382 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:59,703] jepsen worker 42 - jepsen.util 2382 :invoke :read [14 nil] INFO [2022-01-10 05:42:59,734] jepsen worker 14 - jepsen.util 2984 :ok :read [4 3] INFO [2022-01-10 05:42:59,735] jepsen worker 14 - jepsen.util 2984 :invoke :read [4 nil] INFO [2022-01-10 05:42:59,761] jepsen worker 14 - jepsen.util 2984 :ok :read [4 3] INFO [2022-01-10 05:42:59,762] jepsen worker 14 - jepsen.util 2984 :invoke :read [4 nil] INFO [2022-01-10 05:42:59,785] jepsen worker 0 - jepsen.util 2520 :ok :read [0 46] INFO [2022-01-10 05:42:59,786] jepsen worker 14 - jepsen.util 2984 :ok :read [4 3] INFO [2022-01-10 05:42:59,786] jepsen worker 0 - jepsen.util 2520 :invoke :read [0 nil] INFO [2022-01-10 05:42:59,786] jepsen worker 14 - jepsen.util 2984 :invoke :write [4 43] INFO [2022-01-10 05:42:59,913] jepsen worker 89 - jepsen.util 2879 :info :write [29 0] :SyncWriteAmbiguous INFO [2022-01-10 05:42:59,914] jepsen worker 89 - jepsen.util 2969 :invoke :write [29 24] INFO [2022-01-10 05:42:59,952] jepsen worker 42 - jepsen.util 2382 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:42:59,953] jepsen worker 42 - jepsen.util 2382 :invoke :write [14 19] INFO [2022-01-10 05:42:59,978] jepsen worker 0 - jepsen.util 2520 :ok :read [0 46] INFO [2022-01-10 05:42:59,978] jepsen worker 0 - jepsen.util 2520 :invoke :write [0 6] INFO [2022-01-10 05:42:59,979] jepsen worker 85 - jepsen.util 2695 :info :write [28 15] :RequestTimeoutException INFO [2022-01-10 05:42:59,980] jepsen worker 85 - jepsen.util 2785 :invoke :read [28 nil] INFO [2022-01-10 05:43:00,029] jepsen worker 41 - jepsen.util 2201 :info :write [13 16] :RequestTimeoutException INFO [2022-01-10 05:43:00,030] jepsen worker 41 - jepsen.util 2291 :invoke :write [13 9] WARN [2022-01-10 05:43:00,040] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14675045,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15387803,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9591126,"encode_duration_us":16932,"operation_id":"0x4d5a","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10010257}],"total_count":1}} INFO [2022-01-10 05:43:00,046] jepsen worker 21 - jepsen.util 2541 :info :write [7 5] :RequestTimeoutException INFO [2022-01-10 05:43:00,047] jepsen worker 21 - jepsen.util 2631 :invoke :write [7 23] WARN [2022-01-10 05:43:00,140] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14675045,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":15387803,"total_server_duration_us":9591126,"encode_duration_us":16932,"operation_id":"0x4d5a","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10010257},{"operation_name":"upsert","last_dispatch_duration_us":8120980,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":12938018,"total_server_duration_us":402640,"encode_duration_us":16291,"operation_id":"0x4def","timeout_ms":10000,"last_server_duration_us":22720,"total_duration_us":10010078},{"operation_name":"upsert","last_dispatch_duration_us":361902,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7039074,"total_server_duration_us":1718555,"encode_duration_us":16244,"operation_id":"0x4e56","timeout_ms":10000,"last_server_duration_us":263580,"total_duration_us":10009880},{"operation_name":"upsert","last_dispatch_duration_us":693395,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6105783,"total_server_duration_us":1967481,"encode_duration_us":20805,"operation_id":"0x4e1f","timeout_ms":10000,"last_server_duration_us":114620,"total_duration_us":10008789},{"operation_name":"upsert","last_dispatch_duration_us":8715395,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9891432,"total_server_duration_us":402150,"encode_duration_us":16184,"operation_id":"0x4e27","timeout_ms":10000,"last_server_duration_us":26864,"total_duration_us":10008096},{"operation_name":"upsert","last_dispatch_duration_us":79898,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9995385,"total_server_duration_us":809868,"encode_duration_us":18880,"operation_id":"0x4eb0","timeout_ms":10000,"last_server_duration_us":37859,"total_duration_us":10007866},{"operation_name":"upsert","last_dispatch_duration_us":9010098,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":15609436,"total_server_duration_us":114853,"encode_duration_us":19627,"operation_id":"0x4d54","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10006589},{"operation_name":"upsert","last_dispatch_duration_us":8690077,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9863137,"total_server_duration_us":385463,"encode_duration_us":24006,"operation_id":"0x4e28","timeout_ms":10000,"last_server_duration_us":21649,"total_duration_us":10005882},{"operation_name":"upsert","last_dispatch_duration_us":8459898,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":13070002,"total_server_duration_us":409467,"encode_duration_us":16217,"operation_id":"0x4d7f","timeout_ms":10000,"last_server_duration_us":22637,"total_duration_us":10005783},{"operation_name":"upsert","last_dispatch_duration_us":574657,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5992688,"total_server_duration_us":763711,"encode_duration_us":16973,"operation_id":"0x4eef","timeout_ms":10000,"last_server_duration_us":87506,"total_duration_us":10004950}],"total_count":17}} INFO [2022-01-10 05:43:00,212] jepsen worker 2 - jepsen.util 2522 :info :write [0 6] :RequestTimeoutException INFO [2022-01-10 05:43:00,213] jepsen worker 2 - jepsen.util 2612 :invoke :write [0 10] INFO [2022-01-10 05:43:00,277] jepsen worker 1 - jepsen.util 2521 :info :write [0 41] :RequestTimeoutException INFO [2022-01-10 05:43:00,277] jepsen worker 1 - jepsen.util 2611 :invoke :write [0 20] INFO [2022-01-10 05:43:00,282] jepsen worker 31 - jepsen.util 2911 :ok :read [10 43] INFO [2022-01-10 05:43:00,283] jepsen worker 31 - jepsen.util 2911 :invoke :write [10 35] INFO [2022-01-10 05:43:00,290] jepsen worker 85 - jepsen.util 2785 :ok :read [28 45] INFO [2022-01-10 05:43:00,291] jepsen worker 85 - jepsen.util 2785 :invoke :read [28 nil] INFO [2022-01-10 05:43:00,408] jepsen worker 55 - jepsen.util 2845 :ok :read [18 14] INFO [2022-01-10 05:43:00,408] jepsen worker 55 - jepsen.util 2845 :invoke :write [18 48] INFO [2022-01-10 05:43:00,502] jepsen worker 83 - jepsen.util 2873 :info :write [27 31] :RequestTimeoutException INFO [2022-01-10 05:43:00,504] jepsen worker 83 - jepsen.util 2963 :invoke :write [27 48] INFO [2022-01-10 05:43:00,549] jepsen worker 85 - jepsen.util 2785 :ok :read [28 45] INFO [2022-01-10 05:43:00,551] jepsen worker 85 - jepsen.util 2785 :invoke :write [28 16] INFO [2022-01-10 05:43:00,572] jepsen worker 81 - jepsen.util 2691 :info :write [27 35] :RequestTimeoutException INFO [2022-01-10 05:43:00,573] jepsen worker 81 - jepsen.util 2781 :invoke :read [27 nil] INFO [2022-01-10 05:43:00,653] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.368421052631579}, :ns_1@172.23.107.227 {:progress 0.2434017595307918}, :ns_1@172.23.105.187 {:progress 0.2434017595307918}, :ns_1@172.23.107.228 {:progress 0.3529411764705882}} INFO [2022-01-10 05:43:00,653] jepsen worker nemesis - couchbase.util Rebalance check count: 333 INFO [2022-01-10 05:43:00,654] jepsen worker nemesis - couchbase.util Rebalance stuck count: 268 INFO [2022-01-10 05:43:00,703] jepsen worker 81 - jepsen.util 2781 :ok :read [27 30] INFO [2022-01-10 05:43:00,704] jepsen worker 81 - jepsen.util 2781 :invoke :write [27 0] INFO [2022-01-10 05:43:00,986] jepsen worker 40 - jepsen.util 1930 :info :write [13 20] :RequestTimeoutException INFO [2022-01-10 05:43:00,987] jepsen worker 40 - jepsen.util 2020 :invoke :write [13 35] INFO [2022-01-10 05:43:01,010] jepsen worker 38 - jepsen.util 2738 :info :write [12 11] :RequestTimeoutException INFO [2022-01-10 05:43:01,010] jepsen worker 38 - jepsen.util 2828 :invoke :read [12 nil] INFO [2022-01-10 05:43:01,092] jepsen worker 75 - jepsen.util 1785 :info :write [25 41] :RequestTimeoutException INFO [2022-01-10 05:43:01,093] jepsen worker 75 - jepsen.util 1875 :invoke :write [25 26] INFO [2022-01-10 05:43:01,146] jepsen worker 3 - jepsen.util 2523 :info :write [1 25] :RequestTimeoutException INFO [2022-01-10 05:43:01,147] jepsen worker 3 - jepsen.util 2613 :invoke :read [1 nil] INFO [2022-01-10 05:43:01,232] jepsen worker 63 - jepsen.util 2313 :ok :read [21 7] INFO [2022-01-10 05:43:01,233] jepsen worker 63 - jepsen.util 2313 :invoke :write [21 27] INFO [2022-01-10 05:43:01,356] jepsen worker 25 - jepsen.util 2365 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:01,357] jepsen worker 25 - jepsen.util 2365 :invoke :read [8 nil] INFO [2022-01-10 05:43:01,535] jepsen worker 71 - jepsen.util 2051 :info :write [23 39] :RequestTimeoutException INFO [2022-01-10 05:43:01,536] jepsen worker 71 - jepsen.util 2141 :invoke :read [23 nil] INFO [2022-01-10 05:43:01,576] jepsen worker 84 - jepsen.util 2334 :info :write [28 22] :RequestTimeoutException INFO [2022-01-10 05:43:01,577] jepsen worker 84 - jepsen.util 2424 :invoke :write [28 36] INFO [2022-01-10 05:43:01,628] jepsen worker 71 - jepsen.util 2141 :ok :read [23 26] INFO [2022-01-10 05:43:01,631] jepsen worker 71 - jepsen.util 2141 :invoke :read [23 nil] WARN [2022-01-10 05:43:01,640] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18083397,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18083397,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9030999,"encode_duration_us":22353,"operation_id":"0x4c3e","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10000513},{"operation_name":"upsert","last_dispatch_duration_us":9047191,"last_local_id":"EFE9906200000006/0000000021E2C07D","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":9479971,"last_local_socket":"172.23.104.79:44302","total_server_duration_us":9277791,"encode_duration_us":18854,"operation_id":"0x4ee5","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10004774},{"operation_name":"upsert","last_dispatch_duration_us":17919295,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17955988,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9060956,"encode_duration_us":16374,"operation_id":"0x4c85","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10006173},{"operation_name":"upsert","last_dispatch_duration_us":10483472,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10483472,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9595327,"encode_duration_us":18898,"operation_id":"0x4d4a","timeout_ms":10000,"last_server_duration_us":9595327,"total_duration_us":10008825}],"total_count":4}} WARN [2022-01-10 05:43:01,641] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9035764,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9982654,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9190216,"encode_duration_us":16917,"operation_id":"0x4f49","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10010981}],"total_count":1}} INFO [2022-01-10 05:43:01,669] jepsen worker 44 - jepsen.util 2474 :ok :read [14 14] INFO [2022-01-10 05:43:01,670] jepsen worker 44 - jepsen.util 2474 :invoke :read [14 nil] INFO [2022-01-10 05:43:01,683] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.368421052631579}, :ns_1@172.23.107.227 {:progress 0.2434017595307918}, :ns_1@172.23.105.187 {:progress 0.2434017595307918}, :ns_1@172.23.107.228 {:progress 0.3529411764705882}} INFO [2022-01-10 05:43:01,683] jepsen worker nemesis - couchbase.util Rebalance check count: 334 INFO [2022-01-10 05:43:01,683] jepsen worker nemesis - couchbase.util Rebalance stuck count: 269 INFO [2022-01-10 05:43:01,724] jepsen worker 71 - jepsen.util 2141 :ok :read [23 26] INFO [2022-01-10 05:43:01,724] jepsen worker 71 - jepsen.util 2141 :invoke :read [23 nil] INFO [2022-01-10 05:43:01,726] jepsen worker 46 - jepsen.util 2746 :ok :read [15 23] INFO [2022-01-10 05:43:01,727] jepsen worker 46 - jepsen.util 2746 :invoke :read [15 nil] WARN [2022-01-10 05:43:01,738] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17051265,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":17051265,"total_server_duration_us":9027816,"encode_duration_us":17524,"operation_id":"0x4d89","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10008210},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":31405,"operation_id":"0x4e10","timeout_ms":10000,"total_duration_us":10006839},{"operation_name":"upsert","last_dispatch_duration_us":54732,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":737860,"total_server_duration_us":96566,"encode_duration_us":17647,"operation_id":"0x4f19","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10006234},{"operation_name":"upsert","last_dispatch_duration_us":623428,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7959085,"total_server_duration_us":416584,"encode_duration_us":18752,"operation_id":"0x4eb4","timeout_ms":10000,"last_server_duration_us":162796,"total_duration_us":10005508},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":18542,"operation_id":"0x4dde","timeout_ms":10000,"total_duration_us":10003964},{"operation_name":"upsert","last_dispatch_duration_us":52392,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9582085,"total_server_duration_us":259753,"encode_duration_us":17782,"operation_id":"0x4f1b","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10003307},{"operation_name":"upsert","last_dispatch_duration_us":1008561,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7503681,"total_server_duration_us":1061774,"encode_duration_us":16565,"operation_id":"0x4f42","timeout_ms":10000,"last_server_duration_us":342334,"total_duration_us":10003289},{"operation_name":"upsert","last_dispatch_duration_us":138579,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9102861,"total_server_duration_us":101162,"encode_duration_us":21552,"operation_id":"0x4d65","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10003215},{"operation_name":"upsert","last_dispatch_duration_us":643359,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9533501,"total_server_duration_us":562668,"encode_duration_us":18342,"operation_id":"0x4eb1","timeout_ms":10000,"last_server_duration_us":162796,"total_duration_us":10001729},{"operation_name":"upsert","last_dispatch_duration_us":744176,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9852314,"total_server_duration_us":548237,"encode_duration_us":17768,"operation_id":"0x4eb2","timeout_ms":10000,"last_server_duration_us":77290,"total_duration_us":10001451}],"total_count":17}} INFO [2022-01-10 05:43:01,739] jepsen worker 27 - jepsen.util 2457 :info :write [9 47] :RequestTimeoutException INFO [2022-01-10 05:43:01,740] jepsen worker 27 - jepsen.util 2547 :invoke :write [9 48] WARN [2022-01-10 05:43:01,741] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13216677,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":13216677,"total_server_duration_us":6678,"operation_id":"0x4d5b","timeout_ms":30000,"last_server_duration_us":6678,"total_duration_us":13218575},{"operation_name":"get","last_dispatch_duration_us":10960866,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10960866,"total_server_duration_us":19,"operation_id":"0x4dec","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":10963037},{"operation_name":"get","last_dispatch_duration_us":10444225,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":10444225,"total_server_duration_us":43,"operation_id":"0x4d90","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":10445341},{"operation_name":"upsert","last_dispatch_duration_us":2428844,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9540713,"total_server_duration_us":1392250,"encode_duration_us":16604,"operation_id":"0x4dfb","timeout_ms":10000,"last_server_duration_us":1080820,"total_duration_us":10009641},{"operation_name":"upsert","last_dispatch_duration_us":7134524,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":8927199,"total_server_duration_us":1273413,"encode_duration_us":16822,"operation_id":"0x4e7d","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10009604},{"operation_name":"upsert","last_dispatch_duration_us":2622264,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":6094701,"total_server_duration_us":500056,"encode_duration_us":16641,"operation_id":"0x4dc8","timeout_ms":10000,"last_server_duration_us":338124,"total_duration_us":10006835},{"operation_name":"upsert","last_dispatch_duration_us":2717974,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5223559,"total_server_duration_us":312600,"encode_duration_us":17194,"operation_id":"0x4dba","timeout_ms":10000,"last_server_duration_us":96865,"total_duration_us":10006248},{"operation_name":"upsert","last_dispatch_duration_us":41326,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9694670,"total_server_duration_us":214060,"encode_duration_us":21915,"operation_id":"0x4e46","timeout_ms":10000,"last_server_duration_us":22887,"total_duration_us":10006051},{"operation_name":"upsert","last_dispatch_duration_us":9047191,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9479971,"total_server_duration_us":9277791,"encode_duration_us":18854,"operation_id":"0x4ee5","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10004774},{"operation_name":"upsert","last_dispatch_duration_us":2095158,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":6877239,"total_server_duration_us":346479,"encode_duration_us":25604,"operation_id":"0x4df9","timeout_ms":10000,"last_server_duration_us":59374,"total_duration_us":10004725}],"total_count":20}} WARN [2022-01-10 05:43:01,741] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9035764,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9982654,"total_server_duration_us":9190216,"encode_duration_us":16917,"operation_id":"0x4f49","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10010981},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":18832,"operation_id":"0x4e32","timeout_ms":10000,"total_duration_us":10010594},{"operation_name":"upsert","last_dispatch_duration_us":2741448,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2741448,"total_server_duration_us":43838,"encode_duration_us":18609,"operation_id":"0x4f11","timeout_ms":10000,"last_server_duration_us":43838,"total_duration_us":10010583},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":20135,"operation_id":"0x4f4c","timeout_ms":10000,"total_duration_us":10009534},{"operation_name":"upsert","last_dispatch_duration_us":33560,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9277256,"total_server_duration_us":237550,"encode_duration_us":17018,"operation_id":"0x4d97","timeout_ms":10000,"last_server_duration_us":18570,"total_duration_us":10008818},{"operation_name":"upsert","last_dispatch_duration_us":3015153,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10721824,"total_server_duration_us":982863,"encode_duration_us":19724,"operation_id":"0x4df7","timeout_ms":10000,"last_server_duration_us":69481,"total_duration_us":10006465},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":16276,"operation_id":"0x4f3e","timeout_ms":10000,"total_duration_us":10006013},{"operation_name":"upsert","last_dispatch_duration_us":8184778,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":12956708,"total_server_duration_us":258853,"encode_duration_us":16304,"operation_id":"0x4e84","timeout_ms":10000,"last_server_duration_us":18570,"total_duration_us":10005686},{"operation_name":"upsert","last_dispatch_duration_us":2749668,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9603674,"total_server_duration_us":290742,"encode_duration_us":17685,"operation_id":"0x4e11","timeout_ms":10000,"last_server_duration_us":56034,"total_duration_us":10005155},{"operation_name":"upsert","last_dispatch_duration_us":2500100,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":2500100,"total_server_duration_us":21731,"encode_duration_us":23929,"operation_id":"0x4dc2","timeout_ms":10000,"last_server_duration_us":21731,"total_duration_us":10003558}],"total_count":14}} INFO [2022-01-10 05:43:01,824] jepsen worker 71 - jepsen.util 2141 :ok :read [23 26] INFO [2022-01-10 05:43:01,825] jepsen worker 71 - jepsen.util 2141 :invoke :read [23 nil] WARN [2022-01-10 05:43:01,837] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18432770,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":27380432,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9519190,"encode_duration_us":17434,"operation_id":"0x4bee","timeout_ms":10000,"last_server_duration_us":9156570,"total_duration_us":10009183},{"operation_name":"upsert","last_dispatch_duration_us":27580654,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":27580654,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9035244,"encode_duration_us":16386,"operation_id":"0x4b83","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10009916},{"operation_name":"upsert","last_dispatch_duration_us":11328600,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11328600,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9022513,"encode_duration_us":17560,"operation_id":"0x4d58","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10009410}],"total_count":3}} WARN [2022-01-10 05:43:01,838] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17835771,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":17835771,"total_server_duration_us":32,"operation_id":"0x4d1c","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":17838204},{"operation_name":"get","last_dispatch_duration_us":16282428,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":16282428,"total_server_duration_us":11614,"operation_id":"0x4d59","timeout_ms":30000,"last_server_duration_us":11614,"total_duration_us":16285480},{"operation_name":"upsert","last_dispatch_duration_us":14002992,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":14509620,"total_server_duration_us":6018704,"encode_duration_us":20518,"operation_id":"0x4dc6","timeout_ms":10000,"last_server_duration_us":5697192,"total_duration_us":10009661},{"operation_name":"upsert","last_dispatch_duration_us":151032,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8901540,"total_server_duration_us":220722,"encode_duration_us":17641,"operation_id":"0x4f1a","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10008058},{"operation_name":"upsert","last_dispatch_duration_us":31125,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":383569,"total_server_duration_us":226766,"encode_duration_us":16640,"operation_id":"0x4e1e","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10007809},{"operation_name":"upsert","last_dispatch_duration_us":546321,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7083545,"total_server_duration_us":771572,"encode_duration_us":18979,"operation_id":"0x4ee6","timeout_ms":10000,"last_server_duration_us":86475,"total_duration_us":10007565},{"operation_name":"upsert","last_dispatch_duration_us":808567,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6134630,"total_server_duration_us":1689405,"encode_duration_us":17552,"operation_id":"0x4e79","timeout_ms":10000,"last_server_duration_us":121837,"total_duration_us":10007130},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":17365,"operation_id":"0x4ec3","timeout_ms":10000,"total_duration_us":10006578},{"operation_name":"upsert","last_dispatch_duration_us":317631,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7938786,"total_server_duration_us":2139041,"encode_duration_us":16584,"operation_id":"0x4ebd","timeout_ms":10000,"last_server_duration_us":78414,"total_duration_us":10006366},{"operation_name":"upsert","last_dispatch_duration_us":262390,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6057462,"total_server_duration_us":1408667,"encode_duration_us":17183,"operation_id":"0x4e86","timeout_ms":10000,"last_server_duration_us":29323,"total_duration_us":10005838}],"total_count":21}} WARN [2022-01-10 05:43:01,838] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17051265,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17051265,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9027816,"encode_duration_us":17524,"operation_id":"0x4d89","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10008210}],"total_count":1}} WARN [2022-01-10 05:43:01,838] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9056995,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9561421,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9337095,"encode_duration_us":16867,"operation_id":"0x4d91","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10002719},{"operation_name":"upsert","last_dispatch_duration_us":18402845,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":26088601,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9542617,"encode_duration_us":16755,"operation_id":"0x4b85","timeout_ms":10000,"last_server_duration_us":9391573,"total_duration_us":10007656},{"operation_name":"upsert","last_dispatch_duration_us":14002992,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14509620,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":6018704,"encode_duration_us":20518,"operation_id":"0x4dc6","timeout_ms":10000,"last_server_duration_us":5697192,"total_duration_us":10009661}],"total_count":3}} INFO [2022-01-10 05:43:01,850] jepsen worker 77 - jepsen.util 2777 :info :write [25 36] :RequestTimeoutException INFO [2022-01-10 05:43:01,851] jepsen worker 77 - jepsen.util 2867 :invoke :read [25 nil] INFO [2022-01-10 05:43:01,862] jepsen worker 54 - jepsen.util 2574 :info :write [18 3] :RequestTimeoutException INFO [2022-01-10 05:43:01,863] jepsen worker 25 - jepsen.util 2365 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:01,863] jepsen worker 54 - jepsen.util 2664 :invoke :read [18 nil] INFO [2022-01-10 05:43:01,863] jepsen worker 25 - jepsen.util 2365 :invoke :write [8 3] INFO [2022-01-10 05:43:01,895] jepsen worker 44 - jepsen.util 2474 :ok :read [14 14] INFO [2022-01-10 05:43:01,896] jepsen worker 44 - jepsen.util 2474 :invoke :write [14 4] INFO [2022-01-10 05:43:01,915] jepsen worker 46 - jepsen.util 2746 :ok :read [15 23] INFO [2022-01-10 05:43:01,916] jepsen worker 46 - jepsen.util 2746 :invoke :write [15 4] INFO [2022-01-10 05:43:01,927] jepsen worker 71 - jepsen.util 2141 :ok :read [23 26] INFO [2022-01-10 05:43:01,928] jepsen worker 71 - jepsen.util 2141 :invoke :read [23 nil] WARN [2022-01-10 05:43:01,937] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18065544,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":18065544,"total_server_duration_us":27,"operation_id":"0x4c47","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":23741043},{"operation_name":"get","last_dispatch_duration_us":18254500,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":18254500,"total_server_duration_us":6827,"operation_id":"0x4cf9","timeout_ms":30000,"last_server_duration_us":6827,"total_duration_us":19852280},{"operation_name":"get","last_dispatch_duration_us":305963,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":305963,"total_server_duration_us":27,"operation_id":"0x4d88","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":17202135},{"operation_name":"get","last_dispatch_duration_us":9496293,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9496293,"total_server_duration_us":18341,"operation_id":"0x4f12","timeout_ms":30000,"last_server_duration_us":18341,"total_duration_us":11095458},{"operation_name":"upsert","last_dispatch_duration_us":9061548,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9061548,"total_server_duration_us":22059,"encode_duration_us":16432,"operation_id":"0x4d95","timeout_ms":10000,"last_server_duration_us":22059,"total_duration_us":10010356},{"operation_name":"upsert","last_dispatch_duration_us":270211,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10121938,"total_server_duration_us":142762,"encode_duration_us":16809,"operation_id":"0x4f10","timeout_ms":10000,"last_server_duration_us":33035,"total_duration_us":10009286},{"operation_name":"upsert","last_dispatch_duration_us":17733984,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":17733984,"total_server_duration_us":47990,"encode_duration_us":21522,"operation_id":"0x4dc4","timeout_ms":10000,"last_server_duration_us":47990,"total_duration_us":10007015},{"operation_name":"upsert","last_dispatch_duration_us":29233,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":4292493,"total_server_duration_us":1024107,"encode_duration_us":19081,"operation_id":"0x4eaf","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10006592},{"operation_name":"upsert","last_dispatch_duration_us":34213,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":3130644,"total_server_duration_us":482765,"encode_duration_us":16577,"operation_id":"0x4f3f","timeout_ms":10000,"last_server_duration_us":21568,"total_duration_us":10006464},{"operation_name":"upsert","last_dispatch_duration_us":209339,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":490726,"total_server_duration_us":63453,"encode_duration_us":20187,"operation_id":"0x4f59","timeout_ms":10000,"last_server_duration_us":39214,"total_duration_us":10005971}],"total_count":19}} INFO [2022-01-10 05:43:01,986] jepsen worker 71 - jepsen.util 2141 :ok :read [23 26] INFO [2022-01-10 05:43:01,987] jepsen worker 71 - jepsen.util 2141 :invoke :write [23 25] INFO [2022-01-10 05:43:02,316] jepsen worker 70 - jepsen.util 2590 :info :write [23 28] :SyncWriteAmbiguous INFO [2022-01-10 05:43:02,318] jepsen worker 70 - jepsen.util 2680 :invoke :read [23 nil] INFO [2022-01-10 05:43:02,646] jepsen worker 57 - jepsen.util 2937 :info :write [19 42] :RequestTimeoutException INFO [2022-01-10 05:43:02,647] jepsen worker 57 - jepsen.util 3027 :invoke :write [19 35] INFO [2022-01-10 05:43:02,688] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3742690058479532}, :ns_1@172.23.107.227 {:progress 0.2434017595307918}, :ns_1@172.23.105.187 {:progress 0.2434017595307918}, :ns_1@172.23.107.228 {:progress 0.3529411764705882}} INFO [2022-01-10 05:43:02,688] jepsen worker nemesis - couchbase.util Rebalance check count: 335 INFO [2022-01-10 05:43:02,688] jepsen worker nemesis - couchbase.util Rebalance stuck count: 269 INFO [2022-01-10 05:43:02,772] jepsen worker 9 - jepsen.util 2979 :info :write [3 37] :RequestTimeoutException INFO [2022-01-10 05:43:02,773] jepsen worker 9 - jepsen.util 3069 :invoke :write [3 5] INFO [2022-01-10 05:43:02,811] jepsen worker 61 - jepsen.util 2401 :info :write [20 32] :RequestTimeoutException INFO [2022-01-10 05:43:02,811] jepsen worker 61 - jepsen.util 2491 :invoke :read [20 nil] INFO [2022-01-10 05:43:02,959] jepsen worker 88 - jepsen.util 2428 :info :write [29 34] :RequestTimeoutException INFO [2022-01-10 05:43:02,960] jepsen worker 88 - jepsen.util 2518 :invoke :read [29 nil] INFO [2022-01-10 05:43:03,000] jepsen worker 77 - jepsen.util 2867 :ok :read [25 :nil] INFO [2022-01-10 05:43:03,001] jepsen worker 77 - jepsen.util 2867 :invoke :read [25 nil] INFO [2022-01-10 05:43:03,241] jepsen worker 5 - jepsen.util 2615 :info :write [1 49] :RequestTimeoutException INFO [2022-01-10 05:43:03,242] jepsen worker 5 - jepsen.util 2705 :invoke :write [1 10] INFO [2022-01-10 05:43:03,333] jepsen worker 37 - jepsen.util 2647 :ok :read [12 29] INFO [2022-01-10 05:43:03,333] jepsen worker 37 - jepsen.util 2647 :invoke :read [12 nil] INFO [2022-01-10 05:43:03,362] jepsen worker 30 - jepsen.util 2370 :info :write [10 38] :RequestTimeoutException INFO [2022-01-10 05:43:03,363] jepsen worker 30 - jepsen.util 2460 :invoke :read [10 nil] INFO [2022-01-10 05:43:03,366] jepsen worker 61 - jepsen.util 2491 :ok :read [20 :nil] INFO [2022-01-10 05:43:03,368] jepsen worker 61 - jepsen.util 2491 :invoke :read [20 nil] INFO [2022-01-10 05:43:03,374] jepsen worker 38 - jepsen.util 2828 :ok :read [12 29] INFO [2022-01-10 05:43:03,375] jepsen worker 77 - jepsen.util 2867 :ok :read [25 :nil] INFO [2022-01-10 05:43:03,376] jepsen worker 38 - jepsen.util 2828 :invoke :read [12 nil] INFO [2022-01-10 05:43:03,376] jepsen worker 77 - jepsen.util 2867 :invoke :write [25 17] INFO [2022-01-10 05:43:03,383] jepsen worker 88 - jepsen.util 2518 :ok :read [29 0] INFO [2022-01-10 05:43:03,384] jepsen worker 88 - jepsen.util 2518 :invoke :write [29 6] INFO [2022-01-10 05:43:03,399] jepsen worker 37 - jepsen.util 2647 :ok :read [12 29] INFO [2022-01-10 05:43:03,400] jepsen worker 37 - jepsen.util 2647 :invoke :write [12 40] INFO [2022-01-10 05:43:03,437] jepsen worker 38 - jepsen.util 2828 :ok :read [12 29] INFO [2022-01-10 05:43:03,438] jepsen worker 38 - jepsen.util 2828 :invoke :write [12 43] INFO [2022-01-10 05:43:03,501] jepsen worker 8 - jepsen.util 2798 :info :write [2 13] :RequestTimeoutException INFO [2022-01-10 05:43:03,502] jepsen worker 8 - jepsen.util 2888 :invoke :write [2 41] INFO [2022-01-10 05:43:03,694] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3742690058479532}, :ns_1@172.23.107.227 {:progress 0.2434017595307918}, :ns_1@172.23.105.187 {:progress 0.2434017595307918}, :ns_1@172.23.107.228 {:progress 0.3529411764705882}} INFO [2022-01-10 05:43:03,694] jepsen worker nemesis - couchbase.util Rebalance check count: 336 INFO [2022-01-10 05:43:03,694] jepsen worker nemesis - couchbase.util Rebalance stuck count: 270 INFO [2022-01-10 05:43:03,729] jepsen worker 61 - jepsen.util 2491 :ok :read [20 :nil] INFO [2022-01-10 05:43:03,730] jepsen worker 61 - jepsen.util 2491 :invoke :write [20 13] INFO [2022-01-10 05:43:03,776] jepsen worker 11 - jepsen.util 2531 :info :write [3 3] :RequestTimeoutException INFO [2022-01-10 05:43:03,777] jepsen worker 11 - jepsen.util 2621 :invoke :write [3 25] INFO [2022-01-10 05:43:03,886] jepsen worker 51 - jepsen.util 2841 :info :write [17 25] :RequestTimeoutException INFO [2022-01-10 05:43:03,887] jepsen worker 51 - jepsen.util 2931 :invoke :write [17 42] INFO [2022-01-10 05:43:03,891] jepsen worker 69 - jepsen.util 2499 :info :write [23 7] :RequestTimeoutException INFO [2022-01-10 05:43:03,892] jepsen worker 69 - jepsen.util 2589 :invoke :write [23 26] INFO [2022-01-10 05:43:03,922] jepsen worker 87 - jepsen.util 2607 :info :write [29 42] :RequestTimeoutException INFO [2022-01-10 05:43:03,923] jepsen worker 87 - jepsen.util 2697 :invoke :write [29 18] INFO [2022-01-10 05:43:03,996] jepsen worker 52 - jepsen.util 3112 :info :write [17 35] :RequestTimeoutException INFO [2022-01-10 05:43:03,996] jepsen worker 52 - jepsen.util 3202 :invoke :read [17 nil] INFO [2022-01-10 05:43:04,033] jepsen worker 58 - jepsen.util 2488 :ok :read [19 31] INFO [2022-01-10 05:43:04,033] jepsen worker 58 - jepsen.util 2488 :invoke :read [19 nil] INFO [2022-01-10 05:43:04,184] jepsen worker 48 - jepsen.util 2118 :ok :read [16 38] INFO [2022-01-10 05:43:04,185] jepsen worker 48 - jepsen.util 2118 :invoke :write [16 24] INFO [2022-01-10 05:43:04,269] jepsen worker 58 - jepsen.util 2488 :ok :read [19 31] INFO [2022-01-10 05:43:04,269] jepsen worker 58 - jepsen.util 2488 :invoke :read [19 nil] INFO [2022-01-10 05:43:04,285] jepsen worker 65 - jepsen.util 2315 :info :write [21 24] :RequestTimeoutException INFO [2022-01-10 05:43:04,285] jepsen worker 65 - jepsen.util 2405 :invoke :read [21 nil] INFO [2022-01-10 05:43:04,341] jepsen worker 20 - jepsen.util 2180 :info :write [6 5] :RequestTimeoutException INFO [2022-01-10 05:43:04,341] jepsen worker 20 - jepsen.util 2270 :invoke :write [6 41] INFO [2022-01-10 05:43:04,366] jepsen worker 72 - jepsen.util 2682 :info :write [24 32] :RequestTimeoutException INFO [2022-01-10 05:43:04,367] jepsen worker 72 - jepsen.util 2772 :invoke :write [24 3] INFO [2022-01-10 05:43:04,400] jepsen worker 39 - jepsen.util 2829 :info :write [13 28] :SyncWriteAmbiguous INFO [2022-01-10 05:43:04,401] jepsen worker 39 - jepsen.util 2919 :invoke :write [13 34] INFO [2022-01-10 05:43:04,415] jepsen worker 6 - jepsen.util 2526 :info :write [2 27] :RequestTimeoutException INFO [2022-01-10 05:43:04,415] jepsen worker 6 - jepsen.util 2616 :invoke :write [2 0] INFO [2022-01-10 05:43:04,484] jepsen worker 58 - jepsen.util 2488 :ok :read [19 31] INFO [2022-01-10 05:43:04,485] jepsen worker 58 - jepsen.util 2488 :invoke :write [19 35] INFO [2022-01-10 05:43:04,701] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3742690058479532}, :ns_1@172.23.107.227 {:progress 0.2434017595307918}, :ns_1@172.23.105.187 {:progress 0.2434017595307918}, :ns_1@172.23.107.228 {:progress 0.3529411764705882}} INFO [2022-01-10 05:43:04,701] jepsen worker nemesis - couchbase.util Rebalance check count: 337 INFO [2022-01-10 05:43:04,701] jepsen worker nemesis - couchbase.util Rebalance stuck count: 271 INFO [2022-01-10 05:43:04,772] jepsen worker 49 - jepsen.util 2389 :info :write [16 3] :RequestTimeoutException INFO [2022-01-10 05:43:04,773] jepsen worker 49 - jepsen.util 2479 :invoke :write [16 39] INFO [2022-01-10 05:43:04,805] jepsen worker 29 - jepsen.util 2459 :ok :read [9 24] INFO [2022-01-10 05:43:04,806] jepsen worker 29 - jepsen.util 2459 :invoke :read [9 nil] INFO [2022-01-10 05:43:04,811] jepsen worker 67 - jepsen.util 2317 :info :write [22 5] :RequestTimeoutException INFO [2022-01-10 05:43:04,813] jepsen worker 67 - jepsen.util 2407 :invoke :read [22 nil] INFO [2022-01-10 05:43:04,861] jepsen worker 86 - jepsen.util 1886 :info :write [28 15] :RequestTimeoutException INFO [2022-01-10 05:43:04,862] jepsen worker 86 - jepsen.util 1976 :invoke :write [28 32] INFO [2022-01-10 05:43:04,865] jepsen worker 56 - jepsen.util 2666 :info :write [18 1] :RequestTimeoutException INFO [2022-01-10 05:43:04,866] jepsen worker 56 - jepsen.util 2756 :invoke :write [18 28] INFO [2022-01-10 05:43:04,996] jepsen worker 60 - jepsen.util 2850 :info :write [20 28] :RequestTimeoutException INFO [2022-01-10 05:43:04,997] jepsen worker 60 - jepsen.util 2940 :invoke :write [20 17] INFO [2022-01-10 05:43:05,104] jepsen worker 24 - jepsen.util 2094 :ok :read [8 27] INFO [2022-01-10 05:43:05,104] jepsen worker 24 - jepsen.util 2094 :invoke :read [8 nil] INFO [2022-01-10 05:43:05,632] jepsen worker 7 - jepsen.util 2527 :info :write [2 15] :RequestTimeoutException INFO [2022-01-10 05:43:05,632] jepsen worker 7 - jepsen.util 2617 :invoke :write [2 32] INFO [2022-01-10 05:43:05,646] jepsen worker 4 - jepsen.util 2794 :info :write [1 6] :RequestTimeoutException INFO [2022-01-10 05:43:05,671] jepsen worker 4 - jepsen.util 2884 :invoke :read [1 nil] INFO [2022-01-10 05:43:05,690] jepsen worker 36 - jepsen.util 2196 :info :write [12 35] :RequestTimeoutException INFO [2022-01-10 05:43:05,691] jepsen worker 36 - jepsen.util 2286 :invoke :write [12 29] INFO [2022-01-10 05:43:05,726] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3801169590643275}, :ns_1@172.23.107.227 {:progress 0.2551319648093842}, :ns_1@172.23.105.187 {:progress 0.2551319648093842}, :ns_1@172.23.107.228 {:progress 0.3647058823529412}} INFO [2022-01-10 05:43:05,726] jepsen worker nemesis - couchbase.util Rebalance check count: 338 INFO [2022-01-10 05:43:05,726] jepsen worker nemesis - couchbase.util Rebalance stuck count: 271 INFO [2022-01-10 05:43:05,741] jepsen worker 82 - jepsen.util 2512 :info :write [27 24] :RequestTimeoutException INFO [2022-01-10 05:43:05,742] jepsen worker 82 - jepsen.util 2602 :invoke :write [27 12] INFO [2022-01-10 05:43:05,825] jepsen worker 28 - jepsen.util 2368 :info :write [9 18] :RequestTimeoutException INFO [2022-01-10 05:43:05,826] jepsen worker 28 - jepsen.util 2458 :invoke :read [9 nil] INFO [2022-01-10 05:43:05,831] jepsen worker 26 - jepsen.util 2096 :ok :read [8 27] INFO [2022-01-10 05:43:05,831] jepsen worker 26 - jepsen.util 2096 :invoke :read [8 nil] INFO [2022-01-10 05:43:05,838] jepsen worker 29 - jepsen.util 2459 :ok :read [9 24] INFO [2022-01-10 05:43:05,839] jepsen worker 29 - jepsen.util 2459 :invoke :read [9 nil] INFO [2022-01-10 05:43:05,918] jepsen worker 29 - jepsen.util 2459 :ok :read [9 24] INFO [2022-01-10 05:43:05,918] jepsen worker 29 - jepsen.util 2459 :invoke :read [9 nil] INFO [2022-01-10 05:43:05,947] jepsen worker 34 - jepsen.util 2284 :info :write [11 20] :SyncWriteAmbiguous INFO [2022-01-10 05:43:05,948] jepsen worker 34 - jepsen.util 2374 :invoke :write [11 17] INFO [2022-01-10 05:43:05,985] jepsen worker 73 - jepsen.util 2323 :ok :read [24 18] INFO [2022-01-10 05:43:05,986] jepsen worker 73 - jepsen.util 2323 :invoke :write [24 22] INFO [2022-01-10 05:43:06,041] jepsen worker 29 - jepsen.util 2459 :ok :read [9 24] INFO [2022-01-10 05:43:06,042] jepsen worker 29 - jepsen.util 2459 :invoke :write [9 30] INFO [2022-01-10 05:43:06,043] jepsen worker 67 - jepsen.util 2407 :ok :read [22 41] INFO [2022-01-10 05:43:06,044] jepsen worker 67 - jepsen.util 2407 :invoke :read [22 nil] INFO [2022-01-10 05:43:06,442] jepsen worker 22 - jepsen.util 2272 :ok :read [7 :nil] INFO [2022-01-10 05:43:06,443] jepsen worker 22 - jepsen.util 2272 :invoke :read [7 nil] INFO [2022-01-10 05:43:06,732] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3801169590643275}, :ns_1@172.23.107.227 {:progress 0.2551319648093842}, :ns_1@172.23.105.187 {:progress 0.2551319648093842}, :ns_1@172.23.107.228 {:progress 0.3647058823529412}} INFO [2022-01-10 05:43:06,732] jepsen worker nemesis - couchbase.util Rebalance check count: 339 INFO [2022-01-10 05:43:06,732] jepsen worker nemesis - couchbase.util Rebalance stuck count: 272 INFO [2022-01-10 05:43:06,790] jepsen worker 12 - jepsen.util 2622 :info :write [4 17] :RequestTimeoutException INFO [2022-01-10 05:43:06,790] jepsen worker 12 - jepsen.util 2712 :invoke :read [4 nil] INFO [2022-01-10 05:43:06,816] jepsen worker 12 - jepsen.util 2712 :ok :read [4 3] INFO [2022-01-10 05:43:06,817] jepsen worker 12 - jepsen.util 2712 :invoke :read [4 nil] INFO [2022-01-10 05:43:06,825] jepsen worker 24 - jepsen.util 2094 :ok :read [8 27] INFO [2022-01-10 05:43:06,826] jepsen worker 24 - jepsen.util 2094 :invoke :read [8 nil] INFO [2022-01-10 05:43:06,834] jepsen worker 65 - jepsen.util 2405 :ok :read [21 7] INFO [2022-01-10 05:43:06,835] jepsen worker 65 - jepsen.util 2405 :invoke :read [21 nil] INFO [2022-01-10 05:43:06,844] jepsen worker 12 - jepsen.util 2712 :ok :read [4 3] INFO [2022-01-10 05:43:06,844] jepsen worker 12 - jepsen.util 2712 :invoke :write [4 47] INFO [2022-01-10 05:43:06,902] jepsen worker 26 - jepsen.util 2096 :ok :read [8 27] INFO [2022-01-10 05:43:06,903] jepsen worker 26 - jepsen.util 2096 :invoke :read [8 nil] INFO [2022-01-10 05:43:06,942] jepsen worker 18 - jepsen.util 2268 :info :write [6 8] :RequestTimeoutException INFO [2022-01-10 05:43:06,942] jepsen worker 18 - jepsen.util 2358 :invoke :read [6 nil] INFO [2022-01-10 05:43:06,967] jepsen worker 18 - jepsen.util 2358 :fail :read [6 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:06,967] jepsen worker 18 - jepsen.util 2358 :invoke :write [6 12] INFO [2022-01-10 05:43:07,234] jepsen worker 22 - jepsen.util 2272 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:07,236] jepsen worker 22 - jepsen.util 2272 :invoke :write [7 41] INFO [2022-01-10 05:43:07,305] jepsen worker 10 - jepsen.util 2800 :info :write [3 16] :RequestTimeoutException INFO [2022-01-10 05:43:07,307] jepsen worker 10 - jepsen.util 2890 :invoke :write [3 49] INFO [2022-01-10 05:43:07,485] jepsen worker 47 - jepsen.util 2387 :info :write [15 10] :RequestTimeoutException INFO [2022-01-10 05:43:07,488] jepsen worker 47 - jepsen.util 2477 :invoke :read [15 nil] INFO [2022-01-10 05:43:07,522] jepsen worker 80 - jepsen.util 2960 :info :write [26 10] :SyncWriteAmbiguous INFO [2022-01-10 05:43:07,522] jepsen worker 80 - jepsen.util 3050 :invoke :read [26 nil] INFO [2022-01-10 05:43:07,545] jepsen worker 80 - jepsen.util 3050 :ok :read [26 10] INFO [2022-01-10 05:43:07,546] jepsen worker 80 - jepsen.util 3050 :invoke :write [26 15] INFO [2022-01-10 05:43:07,570] jepsen worker 78 - jepsen.util 2868 :info :write [26 26] :RequestTimeoutException INFO [2022-01-10 05:43:07,571] jepsen worker 78 - jepsen.util 2958 :invoke :write [26 10] INFO [2022-01-10 05:43:07,585] jepsen worker 79 - jepsen.util 2869 :info :write [26 43] :RequestTimeoutException INFO [2022-01-10 05:43:07,585] jepsen worker 79 - jepsen.util 2959 :invoke :write [26 0] INFO [2022-01-10 05:43:07,743] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3801169590643275}, :ns_1@172.23.107.227 {:progress 0.2551319648093842}, :ns_1@172.23.105.187 {:progress 0.2551319648093842}, :ns_1@172.23.107.228 {:progress 0.3647058823529412}} INFO [2022-01-10 05:43:07,743] jepsen worker nemesis - couchbase.util Rebalance check count: 340 INFO [2022-01-10 05:43:07,743] jepsen worker nemesis - couchbase.util Rebalance stuck count: 273 INFO [2022-01-10 05:43:07,760] jepsen worker 65 - jepsen.util 2405 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:07,760] jepsen worker 65 - jepsen.util 2405 :invoke :write [21 17] INFO [2022-01-10 05:43:07,767] jepsen worker 24 - jepsen.util 2094 :ok :read [8 27] INFO [2022-01-10 05:43:07,767] jepsen worker 24 - jepsen.util 2094 :invoke :write [8 1] INFO [2022-01-10 05:43:07,786] jepsen worker 26 - jepsen.util 2096 :ok :read [8 27] INFO [2022-01-10 05:43:07,787] jepsen worker 26 - jepsen.util 2096 :invoke :write [8 11] INFO [2022-01-10 05:43:07,836] jepsen worker 59 - jepsen.util 2579 :info :write [19 41] :RequestTimeoutException INFO [2022-01-10 05:43:07,836] jepsen worker 59 - jepsen.util 2669 :invoke :read [19 nil] INFO [2022-01-10 05:43:07,915] jepsen worker 47 - jepsen.util 2477 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:07,916] jepsen worker 47 - jepsen.util 2477 :invoke :write [15 28] INFO [2022-01-10 05:43:07,957] jepsen worker 15 - jepsen.util 2175 :info :write [5 13] :RequestTimeoutException INFO [2022-01-10 05:43:07,958] jepsen worker 15 - jepsen.util 2265 :invoke :read [5 nil] INFO [2022-01-10 05:43:07,996] jepsen worker 50 - jepsen.util 2570 :info :write [16 34] :RequestTimeoutException INFO [2022-01-10 05:43:07,997] jepsen worker 50 - jepsen.util 2660 :invoke :write [16 1] INFO [2022-01-10 05:43:08,006] jepsen worker 66 - jepsen.util 2226 :info :write [22 17] :RequestTimeoutException INFO [2022-01-10 05:43:08,007] jepsen worker 66 - jepsen.util 2316 :invoke :read [22 nil] INFO [2022-01-10 05:43:08,201] jepsen worker 62 - jepsen.util 2672 :info :write [20 25] :RequestTimeoutException INFO [2022-01-10 05:43:08,201] jepsen worker 62 - jepsen.util 2762 :invoke :write [20 12] INFO [2022-01-10 05:43:08,372] jepsen worker 43 - jepsen.util 2293 :info :write [14 27] :RequestTimeoutException INFO [2022-01-10 05:43:08,372] jepsen worker 43 - jepsen.util 2383 :invoke :read [14 nil] INFO [2022-01-10 05:43:08,426] jepsen worker 64 - jepsen.util 2404 :info :write [21 41] :RequestTimeoutException INFO [2022-01-10 05:43:08,427] jepsen worker 64 - jepsen.util 2494 :invoke :write [21 13] INFO [2022-01-10 05:43:08,430] jepsen worker 35 - jepsen.util 2105 :info :write [11 22] :RequestTimeoutException INFO [2022-01-10 05:43:08,431] jepsen worker 35 - jepsen.util 2195 :invoke :read [11 nil] INFO [2022-01-10 05:43:08,455] jepsen worker 19 - jepsen.util 2629 :info :write [6 1] :RequestTimeoutException INFO [2022-01-10 05:43:08,456] jepsen worker 19 - jepsen.util 2719 :invoke :read [6 nil] INFO [2022-01-10 05:43:08,477] jepsen worker 32 - jepsen.util 2642 :info :write [10 2] :RequestTimeoutException INFO [2022-01-10 05:43:08,478] jepsen worker 32 - jepsen.util 2732 :invoke :write [10 1] INFO [2022-01-10 05:43:08,500] jepsen worker 45 - jepsen.util 2925 :ok :read [15 23] INFO [2022-01-10 05:43:08,501] jepsen worker 45 - jepsen.util 2925 :invoke :read [15 nil] INFO [2022-01-10 05:43:08,602] jepsen worker 23 - jepsen.util 2543 :info :write [7 16] :RequestTimeoutException INFO [2022-01-10 05:43:08,603] jepsen worker 23 - jepsen.util 2633 :invoke :write [7 47] INFO [2022-01-10 05:43:08,749] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3801169590643275}, :ns_1@172.23.107.227 {:progress 0.2551319648093842}, :ns_1@172.23.105.187 {:progress 0.2551319648093842}, :ns_1@172.23.107.228 {:progress 0.3647058823529412}} INFO [2022-01-10 05:43:08,749] jepsen worker nemesis - couchbase.util Rebalance check count: 341 INFO [2022-01-10 05:43:08,749] jepsen worker nemesis - couchbase.util Rebalance stuck count: 274 INFO [2022-01-10 05:43:08,774] jepsen worker 74 - jepsen.util 3044 :info :write [24 30] :RequestTimeoutException INFO [2022-01-10 05:43:08,775] jepsen worker 74 - jepsen.util 3134 :invoke :read [24 nil] INFO [2022-01-10 05:43:08,869] jepsen worker 28 - jepsen.util 2458 :ok :read [9 24] INFO [2022-01-10 05:43:08,870] jepsen worker 28 - jepsen.util 2458 :invoke :write [9 15] INFO [2022-01-10 05:43:08,882] jepsen worker 66 - jepsen.util 2316 :ok :read [22 41] INFO [2022-01-10 05:43:08,882] jepsen worker 66 - jepsen.util 2316 :invoke :write [22 47] INFO [2022-01-10 05:43:08,972] jepsen worker 76 - jepsen.util 1966 :info :write [25 10] :RequestTimeoutException INFO [2022-01-10 05:43:08,973] jepsen worker 76 - jepsen.util 2056 :invoke :read [25 nil] INFO [2022-01-10 05:43:09,000] jepsen worker 53 - jepsen.util 3023 :info :write [17 20] :RequestTimeoutException INFO [2022-01-10 05:43:09,001] jepsen worker 53 - jepsen.util 3113 :invoke :read [17 nil] INFO [2022-01-10 05:43:09,023] jepsen worker 53 - jepsen.util 3113 :ok :read [17 41] INFO [2022-01-10 05:43:09,024] jepsen worker 53 - jepsen.util 3113 :invoke :write [17 43] INFO [2022-01-10 05:43:09,210] jepsen worker 76 - jepsen.util 2056 :ok :read [25 :nil] INFO [2022-01-10 05:43:09,214] jepsen worker 76 - jepsen.util 2056 :invoke :write [25 15] INFO [2022-01-10 05:43:09,261] jepsen worker 33 - jepsen.util 2823 :info :write [11 44] :RequestTimeoutException INFO [2022-01-10 05:43:09,262] jepsen worker 33 - jepsen.util 2913 :invoke :write [11 38] INFO [2022-01-10 05:43:09,280] jepsen worker 17 - jepsen.util 2537 :info :write [5 4] :SyncWriteAmbiguous INFO [2022-01-10 05:43:09,281] jepsen worker 17 - jepsen.util 2627 :invoke :write [5 41] INFO [2022-01-10 05:43:09,337] jepsen worker 74 - jepsen.util 3134 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:09,337] jepsen worker 45 - jepsen.util 2925 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:09,339] jepsen worker 74 - jepsen.util 3134 :invoke :read [24 nil] INFO [2022-01-10 05:43:09,339] jepsen worker 45 - jepsen.util 2925 :invoke :write [15 10] INFO [2022-01-10 05:43:09,356] jepsen worker 43 - jepsen.util 2383 :ok :read [14 14] INFO [2022-01-10 05:43:09,359] jepsen worker 31 - jepsen.util 2911 :info :write [10 35] :SyncWriteAmbiguous INFO [2022-01-10 05:43:09,361] jepsen worker 43 - jepsen.util 2383 :invoke :write [14 38] INFO [2022-01-10 05:43:09,363] jepsen worker 31 - jepsen.util 3001 :invoke :write [10 17] INFO [2022-01-10 05:43:09,396] jepsen worker 3 - jepsen.util 2613 :ok :read [1 10] INFO [2022-01-10 05:43:09,396] jepsen worker 3 - jepsen.util 2613 :invoke :write [1 29] INFO [2022-01-10 05:43:09,450] jepsen worker 16 - jepsen.util 2536 :info :write [5 14] :RequestTimeoutException INFO [2022-01-10 05:43:09,451] jepsen worker 16 - jepsen.util 2626 :invoke :read [5 nil] INFO [2022-01-10 05:43:09,460] jepsen worker 68 - jepsen.util 2498 :info :write [22 3] :RequestTimeoutException INFO [2022-01-10 05:43:09,461] jepsen worker 68 - jepsen.util 2588 :invoke :read [22 nil] INFO [2022-01-10 05:43:09,493] jepsen worker 19 - jepsen.util 2719 :ok :read [6 25] INFO [2022-01-10 05:43:09,494] jepsen worker 19 - jepsen.util 2719 :invoke :read [6 nil] INFO [2022-01-10 05:43:09,503] jepsen worker 35 - jepsen.util 2195 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:09,503] jepsen worker 35 - jepsen.util 2195 :invoke :read [11 nil] INFO [2022-01-10 05:43:09,527] jepsen worker 70 - jepsen.util 2680 :ok :read [23 26] INFO [2022-01-10 05:43:09,528] jepsen worker 70 - jepsen.util 2680 :invoke :read [23 nil] INFO [2022-01-10 05:43:09,532] jepsen worker 30 - jepsen.util 2460 :ok :read [10 43] INFO [2022-01-10 05:43:09,532] jepsen worker 30 - jepsen.util 2460 :invoke :read [10 nil] INFO [2022-01-10 05:43:09,557] jepsen worker 13 - jepsen.util 2713 :info :write [4 20] :SyncWriteAmbiguous INFO [2022-01-10 05:43:09,558] jepsen worker 13 - jepsen.util 2803 :invoke :read [4 nil] INFO [2022-01-10 05:43:09,573] jepsen worker 4 - jepsen.util 2884 :ok :read [1 10] INFO [2022-01-10 05:43:09,574] jepsen worker 4 - jepsen.util 2884 :invoke :read [1 nil] INFO [2022-01-10 05:43:09,585] jepsen worker 13 - jepsen.util 2803 :ok :read [4 3] INFO [2022-01-10 05:43:09,586] jepsen worker 13 - jepsen.util 2803 :invoke :write [4 40] INFO [2022-01-10 05:43:09,589] jepsen worker 70 - jepsen.util 2680 :ok :read [23 26] INFO [2022-01-10 05:43:09,589] jepsen worker 70 - jepsen.util 2680 :invoke :write [23 15] INFO [2022-01-10 05:43:09,598] jepsen worker 30 - jepsen.util 2460 :ok :read [10 43] INFO [2022-01-10 05:43:09,599] jepsen worker 30 - jepsen.util 2460 :invoke :write [10 16] INFO [2022-01-10 05:43:09,619] jepsen worker 52 - jepsen.util 3202 :ok :read [17 41] INFO [2022-01-10 05:43:09,621] jepsen worker 52 - jepsen.util 3202 :invoke :write [17 7] INFO [2022-01-10 05:43:09,632] jepsen worker 4 - jepsen.util 2884 :ok :read [1 10] INFO [2022-01-10 05:43:09,633] jepsen worker 4 - jepsen.util 2884 :invoke :write [1 40] INFO [2022-01-10 05:43:09,693] jepsen worker 74 - jepsen.util 3134 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:09,694] jepsen worker 74 - jepsen.util 3134 :invoke :write [24 31] INFO [2022-01-10 05:43:09,713] jepsen worker 16 - jepsen.util 2626 :ok :read [5 42] INFO [2022-01-10 05:43:09,715] jepsen worker 16 - jepsen.util 2626 :invoke :read [5 nil] INFO [2022-01-10 05:43:09,755] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3859649122807017}, :ns_1@172.23.107.227 {:progress 0.2580645161290323}, :ns_1@172.23.105.187 {:progress 0.2580645161290323}, :ns_1@172.23.107.228 {:progress 0.3705882352941177}} INFO [2022-01-10 05:43:09,755] jepsen worker nemesis - couchbase.util Rebalance check count: 342 INFO [2022-01-10 05:43:09,755] jepsen worker nemesis - couchbase.util Rebalance stuck count: 274 INFO [2022-01-10 05:43:09,811] jepsen worker 14 - jepsen.util 2984 :info :write [4 43] :RequestTimeoutException INFO [2022-01-10 05:43:09,812] jepsen worker 14 - jepsen.util 3074 :invoke :write [4 34] INFO [2022-01-10 05:43:09,945] jepsen worker 89 - jepsen.util 2969 :info :write [29 24] :RequestTimeoutException INFO [2022-01-10 05:43:09,946] jepsen worker 89 - jepsen.util 3059 :invoke :read [29 nil] INFO [2022-01-10 05:43:09,996] jepsen worker 42 - jepsen.util 2382 :info :write [14 19] :RequestTimeoutException INFO [2022-01-10 05:43:09,997] jepsen worker 42 - jepsen.util 2472 :invoke :read [14 nil] INFO [2022-01-10 05:43:10,010] jepsen worker 0 - jepsen.util 2520 :info :write [0 6] :RequestTimeoutException INFO [2022-01-10 05:43:10,010] jepsen worker 0 - jepsen.util 2610 :invoke :write [0 20] INFO [2022-01-10 05:43:10,061] jepsen worker 41 - jepsen.util 2291 :info :write [13 9] :RequestTimeoutException INFO [2022-01-10 05:43:10,062] jepsen worker 41 - jepsen.util 2381 :invoke :write [13 22] INFO [2022-01-10 05:43:10,076] jepsen worker 21 - jepsen.util 2631 :info :write [7 23] :RequestTimeoutException INFO [2022-01-10 05:43:10,077] jepsen worker 21 - jepsen.util 2721 :invoke :write [7 6] WARN [2022-01-10 05:43:10,158] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7969299,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8041727,"total_server_duration_us":69177,"encode_duration_us":19697,"operation_id":"0x5074","timeout_ms":10000,"last_server_duration_us":30256,"total_duration_us":10008535},{"operation_name":"upsert","last_dispatch_duration_us":8193028,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8365498,"total_server_duration_us":64259,"encode_duration_us":19324,"operation_id":"0x500b","timeout_ms":10000,"last_server_duration_us":25188,"total_duration_us":10007453},{"operation_name":"upsert","last_dispatch_duration_us":205565,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":4982969,"total_server_duration_us":2773948,"encode_duration_us":16483,"operation_id":"0x50b3","timeout_ms":10000,"last_server_duration_us":53374,"total_duration_us":10007198},{"operation_name":"upsert","last_dispatch_duration_us":7924006,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":8836035,"total_server_duration_us":382838,"encode_duration_us":26158,"operation_id":"0x5079","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10006248},{"operation_name":"upsert","last_dispatch_duration_us":907640,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4388104,"total_server_duration_us":1040568,"encode_duration_us":17268,"operation_id":"0x503e","timeout_ms":10000,"last_server_duration_us":89137,"total_duration_us":10004956},{"operation_name":"upsert","last_dispatch_duration_us":45479,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9026511,"total_server_duration_us":262549,"encode_duration_us":29171,"operation_id":"0x4f4b","timeout_ms":10000,"last_server_duration_us":30728,"total_duration_us":10004236},{"operation_name":"upsert","last_dispatch_duration_us":501887,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5489770,"total_server_duration_us":2004025,"encode_duration_us":18938,"operation_id":"0x50cb","timeout_ms":10000,"last_server_duration_us":414674,"total_duration_us":10003925},{"operation_name":"upsert","last_dispatch_duration_us":350395,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6060591,"total_server_duration_us":1877821,"encode_duration_us":17646,"operation_id":"0x4fe7","timeout_ms":10000,"last_server_duration_us":54697,"total_duration_us":10002983},{"operation_name":"upsert","last_dispatch_duration_us":228557,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5675739,"total_server_duration_us":1920778,"encode_duration_us":20091,"operation_id":"0x4fe1","timeout_ms":10000,"last_server_duration_us":147722,"total_duration_us":10001951},{"operation_name":"upsert","last_dispatch_duration_us":9515055,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":9760449,"total_server_duration_us":9637075,"encode_duration_us":17810,"operation_id":"0x50f0","timeout_ms":10000,"last_server_duration_us":9500805,"total_duration_us":9842451}],"total_count":18}} INFO [2022-01-10 05:43:10,241] jepsen worker 2 - jepsen.util 2612 :info :write [0 10] :RequestTimeoutException INFO [2022-01-10 05:43:10,242] jepsen worker 2 - jepsen.util 2702 :invoke :read [0 nil] INFO [2022-01-10 05:43:10,305] jepsen worker 1 - jepsen.util 2611 :info :write [0 20] :RequestTimeoutException INFO [2022-01-10 05:43:10,306] jepsen worker 1 - jepsen.util 2701 :invoke :write [0 4] INFO [2022-01-10 05:43:10,380] jepsen worker 16 - jepsen.util 2626 :ok :read [5 42] INFO [2022-01-10 05:43:10,380] jepsen worker 16 - jepsen.util 2626 :invoke :read [5 nil] INFO [2022-01-10 05:43:10,435] jepsen worker 55 - jepsen.util 2845 :info :write [18 48] :RequestTimeoutException INFO [2022-01-10 05:43:10,435] jepsen worker 55 - jepsen.util 2935 :invoke :write [18 11] INFO [2022-01-10 05:43:10,525] jepsen worker 83 - jepsen.util 2963 :info :write [27 48] :RequestTimeoutException INFO [2022-01-10 05:43:10,526] jepsen worker 83 - jepsen.util 3053 :invoke :write [27 18] INFO [2022-01-10 05:43:10,580] jepsen worker 85 - jepsen.util 2785 :info :write [28 16] :RequestTimeoutException INFO [2022-01-10 05:43:10,580] jepsen worker 85 - jepsen.util 2875 :invoke :read [28 nil] INFO [2022-01-10 05:43:10,689] jepsen worker 16 - jepsen.util 2626 :ok :read [5 42] INFO [2022-01-10 05:43:10,689] jepsen worker 16 - jepsen.util 2626 :invoke :read [5 nil] INFO [2022-01-10 05:43:10,730] jepsen worker 81 - jepsen.util 2781 :info :write [27 0] :RequestTimeoutException INFO [2022-01-10 05:43:10,731] jepsen worker 81 - jepsen.util 2871 :invoke :read [27 nil] INFO [2022-01-10 05:43:10,761] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3859649122807017}, :ns_1@172.23.107.227 {:progress 0.2580645161290323}, :ns_1@172.23.105.187 {:progress 0.2580645161290323}, :ns_1@172.23.107.228 {:progress 0.3705882352941177}} INFO [2022-01-10 05:43:10,761] jepsen worker nemesis - couchbase.util Rebalance check count: 343 INFO [2022-01-10 05:43:10,761] jepsen worker nemesis - couchbase.util Rebalance stuck count: 275 INFO [2022-01-10 05:43:10,874] jepsen worker 19 - jepsen.util 2719 :ok :read [6 25] INFO [2022-01-10 05:43:10,875] jepsen worker 19 - jepsen.util 2719 :invoke :read [6 nil] INFO [2022-01-10 05:43:10,922] jepsen worker 42 - jepsen.util 2472 :ok :read [14 14] INFO [2022-01-10 05:43:10,923] jepsen worker 42 - jepsen.util 2472 :invoke :read [14 nil] INFO [2022-01-10 05:43:11,012] jepsen worker 40 - jepsen.util 2020 :info :write [13 35] :RequestTimeoutException INFO [2022-01-10 05:43:11,013] jepsen worker 40 - jepsen.util 2110 :invoke :read [13 nil] INFO [2022-01-10 05:43:11,121] jepsen worker 75 - jepsen.util 1875 :info :write [25 26] :RequestTimeoutException INFO [2022-01-10 05:43:11,121] jepsen worker 75 - jepsen.util 1965 :invoke :read [25 nil] INFO [2022-01-10 05:43:11,199] jepsen worker 16 - jepsen.util 2626 :ok :read [5 42] INFO [2022-01-10 05:43:11,199] jepsen worker 16 - jepsen.util 2626 :invoke :write [5 6] INFO [2022-01-10 05:43:11,260] jepsen worker 63 - jepsen.util 2313 :info :write [21 27] :RequestTimeoutException INFO [2022-01-10 05:43:11,260] jepsen worker 63 - jepsen.util 2403 :invoke :read [21 nil] INFO [2022-01-10 05:43:11,420] jepsen worker 42 - jepsen.util 2472 :ok :read [14 14] INFO [2022-01-10 05:43:11,421] jepsen worker 42 - jepsen.util 2472 :invoke :write [14 11] INFO [2022-01-10 05:43:11,461] jepsen worker 71 - jepsen.util 2141 :info :write [23 25] :SyncWriteAmbiguous INFO [2022-01-10 05:43:11,462] jepsen worker 71 - jepsen.util 2231 :invoke :read [23 nil] INFO [2022-01-10 05:43:11,479] jepsen worker 63 - jepsen.util 2403 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:11,479] jepsen worker 63 - jepsen.util 2403 :invoke :read [21 nil] INFO [2022-01-10 05:43:11,484] jepsen worker 54 - jepsen.util 2664 :ok :read [18 14] INFO [2022-01-10 05:43:11,484] jepsen worker 54 - jepsen.util 2664 :invoke :write [18 38] INFO [2022-01-10 05:43:11,523] jepsen worker 63 - jepsen.util 2403 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:11,524] jepsen worker 63 - jepsen.util 2403 :invoke :write [21 15] INFO [2022-01-10 05:43:11,563] jepsen worker 19 - jepsen.util 2719 :ok :read [6 25] INFO [2022-01-10 05:43:11,564] jepsen worker 19 - jepsen.util 2719 :invoke :read [6 nil] INFO [2022-01-10 05:43:11,600] jepsen worker 84 - jepsen.util 2424 :info :write [28 36] :RequestTimeoutException INFO [2022-01-10 05:43:11,600] jepsen worker 84 - jepsen.util 2514 :invoke :write [28 12] WARN [2022-01-10 05:43:11,655] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17601630,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18137912,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9151968,"encode_duration_us":17189,"operation_id":"0x4fab","timeout_ms":10000,"last_server_duration_us":9056479,"total_duration_us":10000872}],"total_count":1}} WARN [2022-01-10 05:43:11,655] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":1099472,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1099472,"total_server_duration_us":451400,"operation_id":"0x506c","timeout_ms":30000,"last_server_duration_us":451400,"total_duration_us":11364712},{"operation_name":"upsert","last_dispatch_duration_us":7350391,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":8209844,"total_server_duration_us":330453,"encode_duration_us":17414,"operation_id":"0x5004","timeout_ms":10000,"last_server_duration_us":74372,"total_duration_us":10010441},{"operation_name":"upsert","last_dispatch_duration_us":9730662,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10254928,"total_server_duration_us":169020,"encode_duration_us":19083,"operation_id":"0x4f92","timeout_ms":10000,"last_server_duration_us":145882,"total_duration_us":10009934},{"operation_name":"upsert","last_dispatch_duration_us":993345,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2193151,"total_server_duration_us":157410,"encode_duration_us":17299,"operation_id":"0x5142","timeout_ms":10000,"last_server_duration_us":60759,"total_duration_us":10009720},{"operation_name":"upsert","last_dispatch_duration_us":507776,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7029065,"total_server_duration_us":1418366,"encode_duration_us":18849,"operation_id":"0x5119","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10008626},{"operation_name":"upsert","last_dispatch_duration_us":616981,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1696948,"total_server_duration_us":134382,"encode_duration_us":16411,"operation_id":"0x50e4","timeout_ms":10000,"last_server_duration_us":70556,"total_duration_us":10008505},{"operation_name":"upsert","last_dispatch_duration_us":44801,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":1442199,"total_server_duration_us":1046173,"encode_duration_us":17374,"operation_id":"0x5106","timeout_ms":10000,"last_server_duration_us":21895,"total_duration_us":10007140},{"operation_name":"upsert","last_dispatch_duration_us":9189171,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10679740,"total_server_duration_us":223680,"encode_duration_us":17241,"operation_id":"0x4fe2","timeout_ms":10000,"last_server_duration_us":40163,"total_duration_us":10005739},{"operation_name":"upsert","last_dispatch_duration_us":584212,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":8471472,"total_server_duration_us":300228,"encode_duration_us":18178,"operation_id":"0x502b","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10004942},{"operation_name":"upsert","last_dispatch_duration_us":789695,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":789695,"total_server_duration_us":121327,"encode_duration_us":18635,"operation_id":"0x50af","timeout_ms":10000,"last_server_duration_us":121327,"total_duration_us":10004026}],"total_count":24}} WARN [2022-01-10 05:43:11,667] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18148859,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20648959,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9061220,"encode_duration_us":23929,"operation_id":"0x4dc2","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10003558},{"operation_name":"upsert","last_dispatch_duration_us":9093923,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17047912,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9133953,"encode_duration_us":17132,"operation_id":"0x4fc6","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10005407},{"operation_name":"upsert","last_dispatch_duration_us":18134828,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":23724105,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9078667,"encode_duration_us":17726,"operation_id":"0x4d26","timeout_ms":10000,"last_server_duration_us":9037366,"total_duration_us":10010546},{"operation_name":"upsert","last_dispatch_duration_us":18080445,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18080445,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9030999,"encode_duration_us":18832,"operation_id":"0x4e32","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10010594},{"operation_name":"upsert","last_dispatch_duration_us":12243438,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14984886,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9413844,"encode_duration_us":18609,"operation_id":"0x4f11","timeout_ms":10000,"last_server_duration_us":9370006,"total_duration_us":10010583}],"total_count":5}} WARN [2022-01-10 05:43:11,752] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":30646,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8848537,"total_server_duration_us":269467,"encode_duration_us":18001,"operation_id":"0x4fd7","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10010325},{"operation_name":"upsert","last_dispatch_duration_us":32100,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9829996,"total_server_duration_us":190755,"encode_duration_us":17162,"operation_id":"0x4f8b","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10009365},{"operation_name":"upsert","last_dispatch_duration_us":10056119,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10056119,"total_server_duration_us":26064,"encode_duration_us":17192,"operation_id":"0x4f6d","timeout_ms":10000,"last_server_duration_us":26064,"total_duration_us":10008724},{"operation_name":"upsert","last_dispatch_duration_us":711267,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9412012,"total_server_duration_us":486943,"encode_duration_us":22157,"operation_id":"0x5111","timeout_ms":10000,"last_server_duration_us":114123,"total_duration_us":10008479},{"operation_name":"upsert","last_dispatch_duration_us":1292087,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9393880,"total_server_duration_us":109980,"encode_duration_us":16500,"operation_id":"0x5140","timeout_ms":10000,"last_server_duration_us":25450,"total_duration_us":10007814},{"operation_name":"upsert","last_dispatch_duration_us":7736438,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8142815,"total_server_duration_us":124400,"encode_duration_us":18740,"operation_id":"0x4ffa","timeout_ms":10000,"last_server_duration_us":33721,"total_duration_us":10006895},{"operation_name":"upsert","last_dispatch_duration_us":370284,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6413369,"total_server_duration_us":1118729,"encode_duration_us":16444,"operation_id":"0x506a","timeout_ms":10000,"last_server_duration_us":119466,"total_duration_us":10006634},{"operation_name":"upsert","last_dispatch_duration_us":32226,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9328715,"total_server_duration_us":244360,"encode_duration_us":17899,"operation_id":"0x4fad","timeout_ms":10000,"last_server_duration_us":21163,"total_duration_us":10006052},{"operation_name":"upsert","last_dispatch_duration_us":269749,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":4335678,"total_server_duration_us":438641,"encode_duration_us":18644,"operation_id":"0x50d9","timeout_ms":10000,"last_server_duration_us":28036,"total_duration_us":10004546},{"operation_name":"upsert","last_dispatch_duration_us":229727,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":4754864,"total_server_duration_us":894433,"encode_duration_us":23632,"operation_id":"0x50b0","timeout_ms":10000,"last_server_duration_us":66957,"total_duration_us":10002585}],"total_count":15}} WARN [2022-01-10 05:43:11,755] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":32325,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8463996,"total_server_duration_us":243168,"encode_duration_us":23305,"operation_id":"0x501c","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10009692},{"operation_name":"upsert","last_dispatch_duration_us":926489,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8002489,"total_server_duration_us":1583901,"encode_duration_us":17304,"operation_id":"0x50b1","timeout_ms":10000,"last_server_duration_us":80826,"total_duration_us":10007937},{"operation_name":"upsert","last_dispatch_duration_us":1331179,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9190390,"total_server_duration_us":1259238,"encode_duration_us":35705,"operation_id":"0x510f","timeout_ms":10000,"last_server_duration_us":21977,"total_duration_us":10006057},{"operation_name":"upsert","last_dispatch_duration_us":1284454,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8030865,"total_server_duration_us":1671744,"encode_duration_us":22924,"operation_id":"0x5112","timeout_ms":10000,"last_server_duration_us":81971,"total_duration_us":10005855},{"operation_name":"upsert","last_dispatch_duration_us":741067,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7579279,"total_server_duration_us":1566427,"encode_duration_us":21343,"operation_id":"0x50b2","timeout_ms":10000,"last_server_duration_us":122517,"total_duration_us":10004824},{"operation_name":"upsert","last_dispatch_duration_us":8435548,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9969828,"total_server_duration_us":283014,"encode_duration_us":20276,"operation_id":"0x509e","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10004286},{"operation_name":"upsert","last_dispatch_duration_us":489953,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7758379,"total_server_duration_us":1562401,"encode_duration_us":17827,"operation_id":"0x4fe5","timeout_ms":10000,"last_server_duration_us":60885,"total_duration_us":10004081},{"operation_name":"upsert","last_dispatch_duration_us":8606312,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9402678,"total_server_duration_us":425046,"encode_duration_us":17159,"operation_id":"0x4fd8","timeout_ms":10000,"last_server_duration_us":23138,"total_duration_us":10002740},{"operation_name":"upsert","last_dispatch_duration_us":9152238,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9905580,"total_server_duration_us":9144281,"encode_duration_us":22916,"operation_id":"0x50a3","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10002164},{"operation_name":"upsert","last_dispatch_duration_us":8403972,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9846386,"total_server_duration_us":243108,"encode_duration_us":17705,"operation_id":"0x509b","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10001635}],"total_count":17}} INFO [2022-01-10 05:43:11,762] jepsen worker 27 - jepsen.util 2547 :info :write [9 48] :RequestTimeoutException INFO [2022-01-10 05:43:11,763] jepsen worker 27 - jepsen.util 2637 :invoke :read [9 nil] WARN [2022-01-10 05:43:11,767] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11455465,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11455465,"total_server_duration_us":29508,"operation_id":"0x4fe4","timeout_ms":30000,"last_server_duration_us":29508,"total_duration_us":11459999},{"operation_name":"get","last_dispatch_duration_us":9218216,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9218216,"total_server_duration_us":5954,"operation_id":"0x4fa7","timeout_ms":30000,"last_server_duration_us":5954,"total_duration_us":10820173},{"operation_name":"get","last_dispatch_duration_us":10807900,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10807900,"total_server_duration_us":11303,"operation_id":"0x4fe0","timeout_ms":30000,"last_server_duration_us":11303,"total_duration_us":10809991},{"operation_name":"upsert","last_dispatch_duration_us":10946637,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10946637,"total_server_duration_us":90481,"encode_duration_us":18889,"operation_id":"0x5010","timeout_ms":10000,"last_server_duration_us":90481,"total_duration_us":10010732},{"operation_name":"upsert","last_dispatch_duration_us":8339249,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11929328,"total_server_duration_us":265154,"encode_duration_us":17649,"operation_id":"0x50b4","timeout_ms":10000,"last_server_duration_us":26596,"total_duration_us":10009902},{"operation_name":"upsert","last_dispatch_duration_us":1215233,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10963887,"total_server_duration_us":851695,"encode_duration_us":18661,"operation_id":"0x5075","timeout_ms":10000,"last_server_duration_us":427638,"total_duration_us":10008889},{"operation_name":"upsert","last_dispatch_duration_us":8792703,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8792703,"total_server_duration_us":18953,"encode_duration_us":17425,"operation_id":"0x4f77","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10007357},{"operation_name":"upsert","last_dispatch_duration_us":146795,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":2280851,"total_server_duration_us":275268,"encode_duration_us":23178,"operation_id":"0x510d","timeout_ms":10000,"last_server_duration_us":70960,"total_duration_us":10007067},{"operation_name":"upsert","last_dispatch_duration_us":9093923,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":17047912,"total_server_duration_us":9133953,"encode_duration_us":17132,"operation_id":"0x4fc6","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10005407},{"operation_name":"upsert","last_dispatch_duration_us":1232986,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":5325915,"total_server_duration_us":211795,"encode_duration_us":17441,"operation_id":"0x511e","timeout_ms":10000,"last_server_duration_us":97483,"total_duration_us":10001680}],"total_count":23}} INFO [2022-01-10 05:43:11,770] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3918128654970761}, :ns_1@172.23.107.227 {:progress 0.2639296187683284}, :ns_1@172.23.105.187 {:progress 0.2639296187683284}, :ns_1@172.23.107.228 {:progress 0.3764705882352941}} INFO [2022-01-10 05:43:11,770] jepsen worker nemesis - couchbase.util Rebalance check count: 344 INFO [2022-01-10 05:43:11,770] jepsen worker nemesis - couchbase.util Rebalance stuck count: 275 INFO [2022-01-10 05:43:11,810] jepsen worker 35 - jepsen.util 2195 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:11,811] jepsen worker 35 - jepsen.util 2195 :invoke :read [11 nil] WARN [2022-01-10 05:43:11,850] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9044378,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":9932728,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":9735292,"encode_duration_us":24518,"operation_id":"0x50da","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10001539},{"operation_name":"upsert","last_dispatch_duration_us":9056888,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10290501,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9452848,"encode_duration_us":17416,"operation_id":"0x4f99","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10007067},{"operation_name":"upsert","last_dispatch_duration_us":18293865,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18293865,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9039489,"encode_duration_us":17385,"operation_id":"0x4f0f","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10003856}],"total_count":3}} WARN [2022-01-10 05:43:11,852] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17558109,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":26660971,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9152330,"encode_duration_us":21552,"operation_id":"0x4d65","timeout_ms":10000,"last_server_duration_us":9051168,"total_duration_us":10003215},{"operation_name":"upsert","last_dispatch_duration_us":17094706,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17094706,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9028877,"encode_duration_us":31405,"operation_id":"0x4e10","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10006839},{"operation_name":"upsert","last_dispatch_duration_us":18031684,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18031684,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9028877,"encode_duration_us":18542,"operation_id":"0x4dde","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10003964}],"total_count":3}} WARN [2022-01-10 05:43:11,856] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9149726,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9970928,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9142295,"encode_duration_us":18539,"operation_id":"0x50a5","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10001265},{"operation_name":"upsert","last_dispatch_duration_us":17457976,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17841546,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9256704,"encode_duration_us":16640,"operation_id":"0x4e1e","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10007809},{"operation_name":"upsert","last_dispatch_duration_us":9152238,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9905580,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9144281,"encode_duration_us":22916,"operation_id":"0x50a3","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10002164}],"total_count":3}} INFO [2022-01-10 05:43:11,890] jepsen worker 25 - jepsen.util 2365 :info :write [8 3] :RequestTimeoutException INFO [2022-01-10 05:43:11,890] jepsen worker 25 - jepsen.util 2455 :invoke :read [8 nil] INFO [2022-01-10 05:43:11,919] jepsen worker 44 - jepsen.util 2474 :info :write [14 4] :RequestTimeoutException INFO [2022-01-10 05:43:11,920] jepsen worker 44 - jepsen.util 2564 :invoke :write [14 30] INFO [2022-01-10 05:43:11,950] jepsen worker 46 - jepsen.util 2746 :info :write [15 4] :RequestTimeoutException INFO [2022-01-10 05:43:11,951] jepsen worker 46 - jepsen.util 2836 :invoke :read [15 nil] WARN [2022-01-10 05:43:12,051] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":6814488,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":7612478,"total_server_duration_us":499645,"encode_duration_us":16386,"operation_id":"0x501e","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10010785},{"operation_name":"upsert","last_dispatch_duration_us":7898047,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":8866448,"total_server_duration_us":387437,"encode_duration_us":16216,"operation_id":"0x5078","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10010326},{"operation_name":"upsert","last_dispatch_duration_us":7919232,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9108232,"total_server_duration_us":437481,"encode_duration_us":17513,"operation_id":"0x505d","timeout_ms":10000,"last_server_duration_us":20921,"total_duration_us":10009664},{"operation_name":"upsert","last_dispatch_duration_us":1266799,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8696093,"total_server_duration_us":1869411,"encode_duration_us":16771,"operation_id":"0x514a","timeout_ms":10000,"last_server_duration_us":69080,"total_duration_us":10009075},{"operation_name":"upsert","last_dispatch_duration_us":509346,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10003773,"total_server_duration_us":185149,"encode_duration_us":16599,"operation_id":"0x4f6c","timeout_ms":10000,"last_server_duration_us":61265,"total_duration_us":10007248},{"operation_name":"upsert","last_dispatch_duration_us":9631211,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9888345,"total_server_duration_us":113324,"encode_duration_us":21225,"operation_id":"0x511f","timeout_ms":10000,"last_server_duration_us":69883,"total_duration_us":10007231},{"operation_name":"upsert","last_dispatch_duration_us":9056888,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":10290501,"total_server_duration_us":9452848,"encode_duration_us":17416,"operation_id":"0x4f99","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10007067},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","encode_duration_us":18509,"operation_id":"0x512d","timeout_ms":10000,"total_duration_us":10006923},{"operation_name":"upsert","last_dispatch_duration_us":112919,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":1135450,"total_server_duration_us":155133,"encode_duration_us":17702,"operation_id":"0x50e6","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10005525},{"operation_name":"upsert","last_dispatch_duration_us":9551846,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10308697,"total_server_duration_us":700112,"encode_duration_us":25154,"operation_id":"0x5110","timeout_ms":10000,"last_server_duration_us":590878,"total_duration_us":10005491}],"total_count":18}} INFO [2022-01-10 05:43:12,524] jepsen worker 37 - jepsen.util 2647 :info :write [12 40] :SyncWriteAmbiguous INFO [2022-01-10 05:43:12,525] jepsen worker 37 - jepsen.util 2737 :invoke :read [12 nil] INFO [2022-01-10 05:43:12,676] jepsen worker 57 - jepsen.util 3027 :info :write [19 35] :RequestTimeoutException INFO [2022-01-10 05:43:12,676] jepsen worker 57 - jepsen.util 3117 :invoke :write [19 41] INFO [2022-01-10 05:43:12,775] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3918128654970761}, :ns_1@172.23.107.227 {:progress 0.2639296187683284}, :ns_1@172.23.105.187 {:progress 0.2639296187683284}, :ns_1@172.23.107.228 {:progress 0.3764705882352941}} INFO [2022-01-10 05:43:12,775] jepsen worker nemesis - couchbase.util Rebalance check count: 345 INFO [2022-01-10 05:43:12,775] jepsen worker nemesis - couchbase.util Rebalance stuck count: 276 INFO [2022-01-10 05:43:12,799] jepsen worker 9 - jepsen.util 3069 :info :write [3 5] :RequestTimeoutException INFO [2022-01-10 05:43:12,800] jepsen worker 9 - jepsen.util 3159 :invoke :read [3 nil] INFO [2022-01-10 05:43:13,271] jepsen worker 5 - jepsen.util 2705 :info :write [1 10] :RequestTimeoutException INFO [2022-01-10 05:43:13,272] jepsen worker 5 - jepsen.util 2795 :invoke :write [1 31] INFO [2022-01-10 05:43:13,406] jepsen worker 88 - jepsen.util 2518 :info :write [29 6] :RequestTimeoutException INFO [2022-01-10 05:43:13,406] jepsen worker 77 - jepsen.util 2867 :info :write [25 17] :RequestTimeoutException INFO [2022-01-10 05:43:13,407] jepsen worker 77 - jepsen.util 2957 :invoke :read [25 nil] INFO [2022-01-10 05:43:13,408] jepsen worker 88 - jepsen.util 2608 :invoke :write [29 3] INFO [2022-01-10 05:43:13,456] jepsen worker 38 - jepsen.util 2828 :info :write [12 43] :RequestTimeoutException INFO [2022-01-10 05:43:13,457] jepsen worker 38 - jepsen.util 2918 :invoke :write [12 32] INFO [2022-01-10 05:43:13,526] jepsen worker 8 - jepsen.util 2888 :info :write [2 41] :RequestTimeoutException INFO [2022-01-10 05:43:13,527] jepsen worker 8 - jepsen.util 2978 :invoke :read [2 nil] INFO [2022-01-10 05:43:13,655] jepsen worker 19 - jepsen.util 2719 :ok :read [6 25] INFO [2022-01-10 05:43:13,655] jepsen worker 19 - jepsen.util 2719 :invoke :write [6 5] INFO [2022-01-10 05:43:13,752] jepsen worker 61 - jepsen.util 2491 :info :write [20 13] :RequestTimeoutException INFO [2022-01-10 05:43:13,752] jepsen worker 61 - jepsen.util 2581 :invoke :read [20 nil] INFO [2022-01-10 05:43:13,781] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3918128654970761}, :ns_1@172.23.107.227 {:progress 0.2639296187683284}, :ns_1@172.23.105.187 {:progress 0.2639296187683284}, :ns_1@172.23.107.228 {:progress 0.3764705882352941}} INFO [2022-01-10 05:43:13,781] jepsen worker nemesis - couchbase.util Rebalance check count: 346 INFO [2022-01-10 05:43:13,781] jepsen worker nemesis - couchbase.util Rebalance stuck count: 277 INFO [2022-01-10 05:43:13,799] jepsen worker 11 - jepsen.util 2621 :info :write [3 25] :RequestTimeoutException INFO [2022-01-10 05:43:13,800] jepsen worker 11 - jepsen.util 2711 :invoke :read [3 nil] INFO [2022-01-10 05:43:13,808] jepsen worker 8 - jepsen.util 2978 :ok :read [2 14] INFO [2022-01-10 05:43:13,809] jepsen worker 8 - jepsen.util 2978 :invoke :read [2 nil] INFO [2022-01-10 05:43:13,838] jepsen worker 46 - jepsen.util 2836 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:13,838] jepsen worker 46 - jepsen.util 2836 :invoke :read [15 nil] INFO [2022-01-10 05:43:13,912] jepsen worker 51 - jepsen.util 2931 :info :write [17 42] :RequestTimeoutException INFO [2022-01-10 05:43:13,913] jepsen worker 51 - jepsen.util 3021 :invoke :write [17 10] INFO [2022-01-10 05:43:13,916] jepsen worker 69 - jepsen.util 2589 :info :write [23 26] :RequestTimeoutException INFO [2022-01-10 05:43:13,916] jepsen worker 69 - jepsen.util 2679 :invoke :read [23 nil] INFO [2022-01-10 05:43:13,932] jepsen worker 56 - jepsen.util 2756 :info :write [18 28] :SyncWriteAmbiguous INFO [2022-01-10 05:43:13,933] jepsen worker 56 - jepsen.util 2846 :invoke :write [18 8] INFO [2022-01-10 05:43:13,951] jepsen worker 87 - jepsen.util 2697 :info :write [29 18] :RequestTimeoutException INFO [2022-01-10 05:43:13,952] jepsen worker 87 - jepsen.util 2787 :invoke :read [29 nil] INFO [2022-01-10 05:43:13,982] jepsen worker 25 - jepsen.util 2455 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:13,983] jepsen worker 25 - jepsen.util 2455 :invoke :write [8 16] INFO [2022-01-10 05:43:13,999] jepsen worker 89 - jepsen.util 3059 :ok :read [29 0] INFO [2022-01-10 05:43:13,999] jepsen worker 89 - jepsen.util 3059 :invoke :read [29 nil] INFO [2022-01-10 05:43:14,053] jepsen worker 89 - jepsen.util 3059 :ok :read [29 0] INFO [2022-01-10 05:43:14,053] jepsen worker 89 - jepsen.util 3059 :invoke :read [29 nil] INFO [2022-01-10 05:43:14,106] jepsen worker 89 - jepsen.util 3059 :ok :read [29 0] INFO [2022-01-10 05:43:14,107] jepsen worker 89 - jepsen.util 3059 :invoke :write [29 8] INFO [2022-01-10 05:43:14,158] jepsen worker 8 - jepsen.util 2978 :ok :read [2 14] INFO [2022-01-10 05:43:14,158] jepsen worker 8 - jepsen.util 2978 :invoke :read [2 nil] INFO [2022-01-10 05:43:14,206] jepsen worker 48 - jepsen.util 2118 :info :write [16 24] :RequestTimeoutException INFO [2022-01-10 05:43:14,207] jepsen worker 48 - jepsen.util 2208 :invoke :write [16 31] INFO [2022-01-10 05:43:14,313] jepsen worker 8 - jepsen.util 2978 :ok :read [2 14] INFO [2022-01-10 05:43:14,314] jepsen worker 8 - jepsen.util 2978 :invoke :write [2 27] INFO [2022-01-10 05:43:14,370] jepsen worker 20 - jepsen.util 2270 :info :write [6 41] :RequestTimeoutException INFO [2022-01-10 05:43:14,371] jepsen worker 20 - jepsen.util 2360 :invoke :write [6 30] INFO [2022-01-10 05:43:14,392] jepsen worker 72 - jepsen.util 2772 :info :write [24 3] :RequestTimeoutException INFO [2022-01-10 05:43:14,393] jepsen worker 72 - jepsen.util 2862 :invoke :read [24 nil] INFO [2022-01-10 05:43:14,426] jepsen worker 39 - jepsen.util 2919 :info :write [13 34] :RequestTimeoutException INFO [2022-01-10 05:43:14,427] jepsen worker 39 - jepsen.util 3009 :invoke :read [13 nil] INFO [2022-01-10 05:43:14,433] jepsen worker 46 - jepsen.util 2836 :ok :read [15 23] INFO [2022-01-10 05:43:14,434] jepsen worker 46 - jepsen.util 2836 :invoke :read [15 nil] INFO [2022-01-10 05:43:14,441] jepsen worker 6 - jepsen.util 2616 :info :write [2 0] :RequestTimeoutException INFO [2022-01-10 05:43:14,442] jepsen worker 6 - jepsen.util 2706 :invoke :write [2 32] INFO [2022-01-10 05:43:14,451] jepsen worker 59 - jepsen.util 2669 :ok :read [19 31] INFO [2022-01-10 05:43:14,452] jepsen worker 59 - jepsen.util 2669 :invoke :read [19 nil] INFO [2022-01-10 05:43:14,506] jepsen worker 58 - jepsen.util 2488 :info :write [19 35] :RequestTimeoutException INFO [2022-01-10 05:43:14,507] jepsen worker 58 - jepsen.util 2578 :invoke :read [19 nil] INFO [2022-01-10 05:43:14,619] jepsen worker 39 - jepsen.util 3009 :ok :read [13 15] INFO [2022-01-10 05:43:14,620] jepsen worker 39 - jepsen.util 3009 :invoke :write [13 14] INFO [2022-01-10 05:43:14,627] jepsen worker 68 - jepsen.util 2588 :ok :read [22 41] INFO [2022-01-10 05:43:14,628] jepsen worker 68 - jepsen.util 2588 :invoke :read [22 nil] INFO [2022-01-10 05:43:14,688] jepsen worker 61 - jepsen.util 2581 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:14,689] jepsen worker 61 - jepsen.util 2581 :invoke :write [20 20] INFO [2022-01-10 05:43:14,718] jepsen worker 85 - jepsen.util 2875 :ok :read [28 45] INFO [2022-01-10 05:43:14,718] jepsen worker 85 - jepsen.util 2875 :invoke :write [28 1] INFO [2022-01-10 05:43:14,791] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3976608187134503}, :ns_1@172.23.107.227 {:progress 0.2697947214076246}, :ns_1@172.23.105.187 {:progress 0.2697947214076246}, :ns_1@172.23.107.228 {:progress 0.3823529411764706}} INFO [2022-01-10 05:43:14,791] jepsen worker nemesis - couchbase.util Rebalance check count: 347 INFO [2022-01-10 05:43:14,791] jepsen worker nemesis - couchbase.util Rebalance stuck count: 277 INFO [2022-01-10 05:43:14,805] jepsen worker 49 - jepsen.util 2479 :info :write [16 39] :RequestTimeoutException INFO [2022-01-10 05:43:14,806] jepsen worker 49 - jepsen.util 2569 :invoke :read [16 nil] INFO [2022-01-10 05:43:14,890] jepsen worker 86 - jepsen.util 1976 :info :write [28 32] :RequestTimeoutException INFO [2022-01-10 05:43:14,890] jepsen worker 86 - jepsen.util 2066 :invoke :read [28 nil] INFO [2022-01-10 05:43:14,898] jepsen worker 75 - jepsen.util 1965 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:14,898] jepsen worker 77 - jepsen.util 2957 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:14,899] jepsen worker 75 - jepsen.util 1965 :invoke :read [25 nil] INFO [2022-01-10 05:43:14,899] jepsen worker 77 - jepsen.util 2957 :invoke :read [25 nil] INFO [2022-01-10 05:43:14,944] jepsen worker 46 - jepsen.util 2836 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:14,945] jepsen worker 46 - jepsen.util 2836 :invoke :read [15 nil] INFO [2022-01-10 05:43:15,017] jepsen worker 60 - jepsen.util 2940 :info :write [20 17] :RequestTimeoutException INFO [2022-01-10 05:43:15,017] jepsen worker 60 - jepsen.util 3030 :invoke :write [20 46] INFO [2022-01-10 05:43:15,643] jepsen worker 75 - jepsen.util 1965 :ok :read [25 :nil] INFO [2022-01-10 05:43:15,643] jepsen worker 77 - jepsen.util 2957 :ok :read [25 :nil] INFO [2022-01-10 05:43:15,643] jepsen worker 77 - jepsen.util 2957 :invoke :write [25 44] INFO [2022-01-10 05:43:15,644] jepsen worker 75 - jepsen.util 1965 :invoke :write [25 13] INFO [2022-01-10 05:43:15,656] jepsen worker 72 - jepsen.util 2862 :ok :read [24 18] INFO [2022-01-10 05:43:15,657] jepsen worker 72 - jepsen.util 2862 :invoke :read [24 nil] INFO [2022-01-10 05:43:15,701] jepsen worker 7 - jepsen.util 2617 :info :write [2 32] :RequestTimeoutException INFO [2022-01-10 05:43:15,702] jepsen worker 7 - jepsen.util 2707 :invoke :write [2 9] INFO [2022-01-10 05:43:15,726] jepsen worker 36 - jepsen.util 2286 :info :write [12 29] :RequestTimeoutException INFO [2022-01-10 05:43:15,727] jepsen worker 36 - jepsen.util 2376 :invoke :read [12 nil] INFO [2022-01-10 05:43:15,769] jepsen worker 82 - jepsen.util 2602 :info :write [27 12] :RequestTimeoutException INFO [2022-01-10 05:43:15,770] jepsen worker 82 - jepsen.util 2692 :invoke :read [27 nil] INFO [2022-01-10 05:43:15,792] jepsen worker 59 - jepsen.util 2669 :ok :read [19 31] INFO [2022-01-10 05:43:15,792] jepsen worker 59 - jepsen.util 2669 :invoke :read [19 nil] INFO [2022-01-10 05:43:15,798] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.3976608187134503}, :ns_1@172.23.107.227 {:progress 0.2697947214076246}, :ns_1@172.23.105.187 {:progress 0.2697947214076246}, :ns_1@172.23.107.228 {:progress 0.3823529411764706}} INFO [2022-01-10 05:43:15,798] jepsen worker nemesis - couchbase.util Rebalance check count: 348 INFO [2022-01-10 05:43:15,798] jepsen worker nemesis - couchbase.util Rebalance stuck count: 278 INFO [2022-01-10 05:43:15,819] jepsen worker 58 - jepsen.util 2578 :ok :read [19 31] INFO [2022-01-10 05:43:15,820] jepsen worker 58 - jepsen.util 2578 :invoke :write [19 31] INFO [2022-01-10 05:43:15,822] jepsen worker 46 - jepsen.util 2836 :ok :read [15 23] INFO [2022-01-10 05:43:15,823] jepsen worker 46 - jepsen.util 2836 :invoke :read [15 nil] INFO [2022-01-10 05:43:15,977] jepsen worker 34 - jepsen.util 2374 :info :write [11 17] :RequestTimeoutException INFO [2022-01-10 05:43:15,977] jepsen worker 34 - jepsen.util 2464 :invoke :write [11 45] INFO [2022-01-10 05:43:16,006] jepsen worker 73 - jepsen.util 2323 :info :write [24 22] :RequestTimeoutException INFO [2022-01-10 05:43:16,007] jepsen worker 73 - jepsen.util 2413 :invoke :read [24 nil] INFO [2022-01-10 05:43:16,071] jepsen worker 29 - jepsen.util 2459 :info :write [9 30] :RequestTimeoutException INFO [2022-01-10 05:43:16,072] jepsen worker 29 - jepsen.util 2549 :invoke :read [9 nil] INFO [2022-01-10 05:43:16,273] jepsen worker 35 - jepsen.util 2195 :ok :read [11 21] INFO [2022-01-10 05:43:16,274] jepsen worker 35 - jepsen.util 2195 :invoke :read [11 nil] INFO [2022-01-10 05:43:16,285] jepsen worker 72 - jepsen.util 2862 :ok :read [24 18] INFO [2022-01-10 05:43:16,286] jepsen worker 72 - jepsen.util 2862 :invoke :read [24 nil] INFO [2022-01-10 05:43:16,446] jepsen worker 67 - jepsen.util 2407 :ok :read [22 41] INFO [2022-01-10 05:43:16,447] jepsen worker 67 - jepsen.util 2407 :invoke :read [22 nil] INFO [2022-01-10 05:43:16,467] jepsen worker 15 - jepsen.util 2265 :ok :read [5 42] INFO [2022-01-10 05:43:16,467] jepsen worker 15 - jepsen.util 2265 :invoke :write [5 22] INFO [2022-01-10 05:43:16,474] jepsen worker 81 - jepsen.util 2871 :ok :read [27 30] INFO [2022-01-10 05:43:16,475] jepsen worker 81 - jepsen.util 2871 :invoke :write [27 44] INFO [2022-01-10 05:43:16,497] jepsen worker 68 - jepsen.util 2588 :ok :read [22 41] INFO [2022-01-10 05:43:16,499] jepsen worker 68 - jepsen.util 2588 :invoke :read [22 nil] INFO [2022-01-10 05:43:16,502] jepsen worker 27 - jepsen.util 2637 :ok :read [9 24] INFO [2022-01-10 05:43:16,503] jepsen worker 27 - jepsen.util 2637 :invoke :write [9 38] INFO [2022-01-10 05:43:16,602] jepsen worker 80 - jepsen.util 3050 :info :write [26 15] :SyncWriteAmbiguous INFO [2022-01-10 05:43:16,603] jepsen worker 80 - jepsen.util 3140 :invoke :read [26 nil] INFO [2022-01-10 05:43:16,629] jepsen worker 80 - jepsen.util 3140 :ok :read [26 10] INFO [2022-01-10 05:43:16,629] jepsen worker 80 - jepsen.util 3140 :invoke :read [26 nil] INFO [2022-01-10 05:43:16,655] jepsen worker 80 - jepsen.util 3140 :ok :read [26 10] INFO [2022-01-10 05:43:16,656] jepsen worker 80 - jepsen.util 3140 :invoke :write [26 23] INFO [2022-01-10 05:43:16,769] jepsen worker 49 - jepsen.util 2569 :ok :read [16 38] INFO [2022-01-10 05:43:16,770] jepsen worker 49 - jepsen.util 2569 :invoke :read [16 nil] INFO [2022-01-10 05:43:16,775] jepsen worker 67 - jepsen.util 2407 :ok :read [22 41] INFO [2022-01-10 05:43:16,776] jepsen worker 67 - jepsen.util 2407 :invoke :read [22 nil] INFO [2022-01-10 05:43:16,808] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4035087719298246}, :ns_1@172.23.107.227 {:progress 0.2727272727272727}, :ns_1@172.23.105.187 {:progress 0.2727272727272727}, :ns_1@172.23.107.228 {:progress 0.388235294117647}} INFO [2022-01-10 05:43:16,809] jepsen worker nemesis - couchbase.util Rebalance check count: 349 INFO [2022-01-10 05:43:16,809] jepsen worker nemesis - couchbase.util Rebalance stuck count: 278 INFO [2022-01-10 05:43:16,814] jepsen worker 82 - jepsen.util 2692 :ok :read [27 30] INFO [2022-01-10 05:43:16,816] jepsen worker 82 - jepsen.util 2692 :invoke :write [27 15] INFO [2022-01-10 05:43:16,829] jepsen worker 59 - jepsen.util 2669 :ok :read [19 31] INFO [2022-01-10 05:43:16,829] jepsen worker 59 - jepsen.util 2669 :invoke :read [19 nil] INFO [2022-01-10 05:43:16,871] jepsen worker 12 - jepsen.util 2712 :info :write [4 47] :RequestTimeoutException INFO [2022-01-10 05:43:16,872] jepsen worker 12 - jepsen.util 2802 :invoke :write [4 16] INFO [2022-01-10 05:43:16,994] jepsen worker 18 - jepsen.util 2358 :info :write [6 12] :RequestTimeoutException INFO [2022-01-10 05:43:16,995] jepsen worker 18 - jepsen.util 2448 :invoke :read [6 nil] INFO [2022-01-10 05:43:17,256] jepsen worker 22 - jepsen.util 2272 :info :write [7 41] :RequestTimeoutException INFO [2022-01-10 05:43:17,257] jepsen worker 22 - jepsen.util 2362 :invoke :read [7 nil] INFO [2022-01-10 05:43:17,344] jepsen worker 10 - jepsen.util 2890 :info :write [3 49] :RequestTimeoutException INFO [2022-01-10 05:43:17,344] jepsen worker 10 - jepsen.util 2980 :invoke :read [3 nil] INFO [2022-01-10 05:43:17,368] jepsen worker 73 - jepsen.util 2413 :ok :read [24 18] INFO [2022-01-10 05:43:17,369] jepsen worker 73 - jepsen.util 2413 :invoke :read [24 nil] INFO [2022-01-10 05:43:17,478] jepsen worker 46 - jepsen.util 2836 :ok :read [15 23] INFO [2022-01-10 05:43:17,479] jepsen worker 46 - jepsen.util 2836 :invoke :write [15 15] INFO [2022-01-10 05:43:17,500] jepsen worker 49 - jepsen.util 2569 :ok :read [16 38] INFO [2022-01-10 05:43:17,501] jepsen worker 49 - jepsen.util 2569 :invoke :read [16 nil] INFO [2022-01-10 05:43:17,513] jepsen worker 67 - jepsen.util 2407 :ok :read [22 41] INFO [2022-01-10 05:43:17,514] jepsen worker 67 - jepsen.util 2407 :invoke :read [22 nil] INFO [2022-01-10 05:43:17,533] jepsen worker 68 - jepsen.util 2588 :ok :read [22 41] INFO [2022-01-10 05:43:17,533] jepsen worker 68 - jepsen.util 2588 :invoke :read [22 nil] INFO [2022-01-10 05:43:17,541] jepsen worker 18 - jepsen.util 2448 :ok :read [6 25] INFO [2022-01-10 05:43:17,542] jepsen worker 18 - jepsen.util 2448 :invoke :write [6 13] INFO [2022-01-10 05:43:17,596] jepsen worker 78 - jepsen.util 2958 :info :write [26 10] :RequestTimeoutException INFO [2022-01-10 05:43:17,597] jepsen worker 78 - jepsen.util 3048 :invoke :read [26 nil] INFO [2022-01-10 05:43:17,611] jepsen worker 79 - jepsen.util 2959 :info :write [26 0] :RequestTimeoutException INFO [2022-01-10 05:43:17,611] jepsen worker 79 - jepsen.util 3049 :invoke :read [26 nil] INFO [2022-01-10 05:43:17,622] jepsen worker 78 - jepsen.util 3048 :ok :read [26 10] INFO [2022-01-10 05:43:17,623] jepsen worker 78 - jepsen.util 3048 :invoke :read [26 nil] INFO [2022-01-10 05:43:17,647] jepsen worker 78 - jepsen.util 3048 :ok :read [26 10] INFO [2022-01-10 05:43:17,648] jepsen worker 78 - jepsen.util 3048 :invoke :write [26 40] INFO [2022-01-10 05:43:17,785] jepsen worker 65 - jepsen.util 2405 :info :write [21 17] :RequestTimeoutException INFO [2022-01-10 05:43:17,786] jepsen worker 65 - jepsen.util 2495 :invoke :read [21 nil] INFO [2022-01-10 05:43:17,796] jepsen worker 24 - jepsen.util 2094 :info :write [8 1] :RequestTimeoutException INFO [2022-01-10 05:43:17,796] jepsen worker 24 - jepsen.util 2184 :invoke :write [8 16] INFO [2022-01-10 05:43:17,815] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4035087719298246}, :ns_1@172.23.107.227 {:progress 0.2727272727272727}, :ns_1@172.23.105.187 {:progress 0.2727272727272727}, :ns_1@172.23.107.228 {:progress 0.388235294117647}} INFO [2022-01-10 05:43:17,815] jepsen worker nemesis - couchbase.util Rebalance check count: 350 INFO [2022-01-10 05:43:17,815] jepsen worker nemesis - couchbase.util Rebalance stuck count: 279 INFO [2022-01-10 05:43:17,826] jepsen worker 26 - jepsen.util 2096 :info :write [8 11] :RequestTimeoutException INFO [2022-01-10 05:43:17,827] jepsen worker 26 - jepsen.util 2186 :invoke :read [8 nil] INFO [2022-01-10 05:43:17,940] jepsen worker 47 - jepsen.util 2477 :info :write [15 28] :RequestTimeoutException INFO [2022-01-10 05:43:17,940] jepsen worker 47 - jepsen.util 2567 :invoke :read [15 nil] INFO [2022-01-10 05:43:17,999] jepsen worker 49 - jepsen.util 2569 :ok :read [16 38] INFO [2022-01-10 05:43:18,000] jepsen worker 49 - jepsen.util 2569 :invoke :read [16 nil] INFO [2022-01-10 05:43:18,019] jepsen worker 50 - jepsen.util 2660 :info :write [16 1] :RequestTimeoutException INFO [2022-01-10 05:43:18,020] jepsen worker 50 - jepsen.util 2750 :invoke :write [16 11] INFO [2022-01-10 05:43:18,220] jepsen worker 67 - jepsen.util 2407 :ok :read [22 41] INFO [2022-01-10 05:43:18,221] jepsen worker 67 - jepsen.util 2407 :invoke :write [22 10] INFO [2022-01-10 05:43:18,227] jepsen worker 62 - jepsen.util 2762 :info :write [20 12] :RequestTimeoutException INFO [2022-01-10 05:43:18,227] jepsen worker 62 - jepsen.util 2852 :invoke :read [20 nil] INFO [2022-01-10 05:43:18,431] jepsen worker 59 - jepsen.util 2669 :ok :read [19 31] INFO [2022-01-10 05:43:18,431] jepsen worker 59 - jepsen.util 2669 :invoke :write [19 39] INFO [2022-01-10 05:43:18,455] jepsen worker 64 - jepsen.util 2494 :info :write [21 13] :RequestTimeoutException INFO [2022-01-10 05:43:18,456] jepsen worker 64 - jepsen.util 2584 :invoke :write [21 31] INFO [2022-01-10 05:43:18,512] jepsen worker 32 - jepsen.util 2732 :info :write [10 1] :RequestTimeoutException INFO [2022-01-10 05:43:18,512] jepsen worker 32 - jepsen.util 2822 :invoke :read [10 nil] INFO [2022-01-10 05:43:18,522] jepsen worker 73 - jepsen.util 2413 :ok :read [24 18] INFO [2022-01-10 05:43:18,522] jepsen worker 73 - jepsen.util 2413 :invoke :write [24 19] INFO [2022-01-10 05:43:18,623] jepsen worker 62 - jepsen.util 2852 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:18,624] jepsen worker 62 - jepsen.util 2852 :invoke :read [20 nil] INFO [2022-01-10 05:43:18,626] jepsen worker 23 - jepsen.util 2633 :info :write [7 47] :RequestTimeoutException INFO [2022-01-10 05:43:18,626] jepsen worker 23 - jepsen.util 2723 :invoke :write [7 7] INFO [2022-01-10 05:43:18,680] jepsen worker 49 - jepsen.util 2569 :ok :read [16 38] INFO [2022-01-10 05:43:18,681] jepsen worker 49 - jepsen.util 2569 :invoke :write [16 13] INFO [2022-01-10 05:43:18,684] jepsen worker 13 - jepsen.util 2803 :info :write [4 40] :SyncWriteAmbiguous INFO [2022-01-10 05:43:18,685] jepsen worker 13 - jepsen.util 2893 :invoke :read [4 nil] INFO [2022-01-10 05:43:18,689] jepsen worker 47 - jepsen.util 2567 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:18,690] jepsen worker 47 - jepsen.util 2567 :invoke :write [15 31] INFO [2022-01-10 05:43:18,717] jepsen worker 13 - jepsen.util 2893 :ok :read [4 3] INFO [2022-01-10 05:43:18,718] jepsen worker 13 - jepsen.util 2893 :invoke :read [4 nil] INFO [2022-01-10 05:43:18,750] jepsen worker 13 - jepsen.util 2893 :ok :read [4 3] INFO [2022-01-10 05:43:18,751] jepsen worker 13 - jepsen.util 2893 :invoke :write [4 12] INFO [2022-01-10 05:43:18,758] jepsen worker 30 - jepsen.util 2460 :info :write [10 16] :SyncWriteAmbiguous INFO [2022-01-10 05:43:18,759] jepsen worker 30 - jepsen.util 2550 :invoke :read [10 nil] INFO [2022-01-10 05:43:18,786] jepsen worker 30 - jepsen.util 2550 :ok :read [10 43] INFO [2022-01-10 05:43:18,786] jepsen worker 30 - jepsen.util 2550 :invoke :write [10 15] INFO [2022-01-10 05:43:18,821] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4093567251461988}, :ns_1@172.23.107.227 {:progress 0.2785923753665689}, :ns_1@172.23.105.187 {:progress 0.2785923753665689}, :ns_1@172.23.107.228 {:progress 0.3941176470588236}} INFO [2022-01-10 05:43:18,821] jepsen worker nemesis - couchbase.util Rebalance check count: 351 INFO [2022-01-10 05:43:18,821] jepsen worker nemesis - couchbase.util Rebalance stuck count: 279 INFO [2022-01-10 05:43:18,874] jepsen worker 68 - jepsen.util 2588 :ok :read [22 41] INFO [2022-01-10 05:43:18,877] jepsen worker 68 - jepsen.util 2588 :invoke :write [22 25] INFO [2022-01-10 05:43:18,882] jepsen worker 72 - jepsen.util 2862 :ok :read [24 18] INFO [2022-01-10 05:43:18,883] jepsen worker 72 - jepsen.util 2862 :invoke :read [24 nil] INFO [2022-01-10 05:43:18,892] jepsen worker 28 - jepsen.util 2458 :info :write [9 15] :RequestTimeoutException INFO [2022-01-10 05:43:18,893] jepsen worker 28 - jepsen.util 2548 :invoke :read [9 nil] INFO [2022-01-10 05:43:18,902] jepsen worker 35 - jepsen.util 2195 :ok :read [11 21] INFO [2022-01-10 05:43:18,902] jepsen worker 66 - jepsen.util 2316 :info :write [22 47] :RequestTimeoutException INFO [2022-01-10 05:43:18,903] jepsen worker 35 - jepsen.util 2195 :invoke :read [11 nil] INFO [2022-01-10 05:43:18,903] jepsen worker 65 - jepsen.util 2495 :ok :read [21 7] INFO [2022-01-10 05:43:18,905] jepsen worker 65 - jepsen.util 2495 :invoke :read [21 nil] INFO [2022-01-10 05:43:18,906] jepsen worker 66 - jepsen.util 2406 :invoke :read [22 nil] INFO [2022-01-10 05:43:18,908] jepsen worker 87 - jepsen.util 2787 :ok :read [29 0] INFO [2022-01-10 05:43:18,910] jepsen worker 87 - jepsen.util 2787 :invoke :write [29 30] INFO [2022-01-10 05:43:18,922] jepsen worker 32 - jepsen.util 2822 :ok :read [10 43] INFO [2022-01-10 05:43:18,923] jepsen worker 32 - jepsen.util 2822 :invoke :read [10 nil] INFO [2022-01-10 05:43:18,952] jepsen worker 32 - jepsen.util 2822 :ok :read [10 43] INFO [2022-01-10 05:43:18,952] jepsen worker 32 - jepsen.util 2822 :invoke :read [10 nil] INFO [2022-01-10 05:43:18,988] jepsen worker 32 - jepsen.util 2822 :ok :read [10 43] INFO [2022-01-10 05:43:18,988] jepsen worker 32 - jepsen.util 2822 :invoke :read [10 nil] INFO [2022-01-10 05:43:19,036] jepsen worker 32 - jepsen.util 2822 :ok :read [10 43] INFO [2022-01-10 05:43:19,037] jepsen worker 32 - jepsen.util 2822 :invoke :write [10 11] INFO [2022-01-10 05:43:19,045] jepsen worker 53 - jepsen.util 3113 :info :write [17 43] :RequestTimeoutException INFO [2022-01-10 05:43:19,046] jepsen worker 53 - jepsen.util 3203 :invoke :read [17 nil] INFO [2022-01-10 05:43:19,097] jepsen worker 52 - jepsen.util 3202 :info :write [17 7] :SyncWriteAmbiguous INFO [2022-01-10 05:43:19,097] jepsen worker 52 - jepsen.util 3292 :invoke :read [17 nil] INFO [2022-01-10 05:43:19,125] jepsen worker 53 - jepsen.util 3203 :ok :read [17 41] INFO [2022-01-10 05:43:19,126] jepsen worker 53 - jepsen.util 3203 :invoke :read [17 nil] INFO [2022-01-10 05:43:19,128] jepsen worker 52 - jepsen.util 3292 :ok :read [17 41] INFO [2022-01-10 05:43:19,129] jepsen worker 52 - jepsen.util 3292 :invoke :write [17 36] INFO [2022-01-10 05:43:19,156] jepsen worker 53 - jepsen.util 3203 :ok :read [17 41] INFO [2022-01-10 05:43:19,157] jepsen worker 53 - jepsen.util 3203 :invoke :write [17 43] INFO [2022-01-10 05:43:19,235] jepsen worker 76 - jepsen.util 2056 :info :write [25 15] :RequestTimeoutException INFO [2022-01-10 05:43:19,236] jepsen worker 76 - jepsen.util 2146 :invoke :write [25 1] INFO [2022-01-10 05:43:19,239] jepsen worker 2 - jepsen.util 2702 :ok :read [0 46] INFO [2022-01-10 05:43:19,240] jepsen worker 2 - jepsen.util 2702 :invoke :write [0 43] INFO [2022-01-10 05:43:19,290] jepsen worker 33 - jepsen.util 2913 :info :write [11 38] :RequestTimeoutException INFO [2022-01-10 05:43:19,290] jepsen worker 33 - jepsen.util 3003 :invoke :write [11 0] INFO [2022-01-10 05:43:19,301] jepsen worker 17 - jepsen.util 2627 :info :write [5 41] :RequestTimeoutException INFO [2022-01-10 05:43:19,302] jepsen worker 17 - jepsen.util 2717 :invoke :read [5 nil] INFO [2022-01-10 05:43:19,361] jepsen worker 45 - jepsen.util 2925 :info :write [15 10] :RequestTimeoutException INFO [2022-01-10 05:43:19,362] jepsen worker 45 - jepsen.util 3015 :invoke :read [15 nil] INFO [2022-01-10 05:43:19,391] jepsen worker 43 - jepsen.util 2383 :info :write [14 38] :RequestTimeoutException INFO [2022-01-10 05:43:19,391] jepsen worker 43 - jepsen.util 2473 :invoke :write [14 7] INFO [2022-01-10 05:43:19,406] jepsen worker 31 - jepsen.util 3001 :info :write [10 17] :RequestTimeoutException INFO [2022-01-10 05:43:19,406] jepsen worker 31 - jepsen.util 3091 :invoke :read [10 nil] INFO [2022-01-10 05:43:19,424] jepsen worker 3 - jepsen.util 2613 :info :write [1 29] :RequestTimeoutException INFO [2022-01-10 05:43:19,425] jepsen worker 3 - jepsen.util 2703 :invoke :read [1 nil] INFO [2022-01-10 05:43:19,615] jepsen worker 40 - jepsen.util 2110 :ok :read [13 15] INFO [2022-01-10 05:43:19,615] jepsen worker 70 - jepsen.util 2680 :info :write [23 15] :RequestTimeoutException INFO [2022-01-10 05:43:19,616] jepsen worker 70 - jepsen.util 2770 :invoke :write [23 15] INFO [2022-01-10 05:43:19,616] jepsen worker 40 - jepsen.util 2110 :invoke :write [13 45] INFO [2022-01-10 05:43:19,655] jepsen worker 4 - jepsen.util 2884 :info :write [1 40] :RequestTimeoutException INFO [2022-01-10 05:43:19,656] jepsen worker 4 - jepsen.util 2974 :invoke :write [1 25] INFO [2022-01-10 05:43:19,721] jepsen worker 74 - jepsen.util 3134 :info :write [24 31] :RequestTimeoutException INFO [2022-01-10 05:43:19,722] jepsen worker 74 - jepsen.util 3224 :invoke :write [24 1] INFO [2022-01-10 05:43:19,827] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4093567251461988}, :ns_1@172.23.107.227 {:progress 0.2785923753665689}, :ns_1@172.23.105.187 {:progress 0.2785923753665689}, :ns_1@172.23.107.228 {:progress 0.3941176470588236}} INFO [2022-01-10 05:43:19,827] jepsen worker nemesis - couchbase.util Rebalance check count: 352 INFO [2022-01-10 05:43:19,827] jepsen worker nemesis - couchbase.util Rebalance stuck count: 280 INFO [2022-01-10 05:43:19,840] jepsen worker 14 - jepsen.util 3074 :info :write [4 34] :RequestTimeoutException INFO [2022-01-10 05:43:19,841] jepsen worker 14 - jepsen.util 3164 :invoke :write [4 46] INFO [2022-01-10 05:43:19,846] jepsen worker 62 - jepsen.util 2852 :ok :read [20 :nil] INFO [2022-01-10 05:43:19,847] jepsen worker 62 - jepsen.util 2852 :invoke :write [20 45] INFO [2022-01-10 05:43:19,862] jepsen worker 22 - jepsen.util 2362 :ok :read [7 :nil] INFO [2022-01-10 05:43:19,862] jepsen worker 22 - jepsen.util 2362 :invoke :read [7 nil] INFO [2022-01-10 05:43:20,041] jepsen worker 0 - jepsen.util 2610 :info :write [0 20] :RequestTimeoutException INFO [2022-01-10 05:43:20,042] jepsen worker 0 - jepsen.util 2700 :invoke :read [0 nil] WARN [2022-01-10 05:43:20,071] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9816117,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10872908,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":10515472,"encode_duration_us":18970,"operation_id":"0x511b","timeout_ms":10000,"last_server_duration_us":9761422,"total_duration_us":10007935}],"total_count":1}} INFO [2022-01-10 05:43:20,084] jepsen worker 41 - jepsen.util 2381 :info :write [13 22] :RequestTimeoutException INFO [2022-01-10 05:43:20,085] jepsen worker 41 - jepsen.util 2471 :invoke :write [13 20] INFO [2022-01-10 05:43:20,106] jepsen worker 21 - jepsen.util 2721 :info :write [7 6] :RequestTimeoutException INFO [2022-01-10 05:43:20,106] jepsen worker 21 - jepsen.util 2811 :invoke :write [7 23] WARN [2022-01-10 05:43:20,172] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":65843,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8835468,"total_server_duration_us":251684,"encode_duration_us":16275,"operation_id":"0x511d","timeout_ms":10000,"last_server_duration_us":21568,"total_duration_us":10009505},{"operation_name":"upsert","last_dispatch_duration_us":394393,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7052400,"total_server_duration_us":3078371,"encode_duration_us":17470,"operation_id":"0x52a4","timeout_ms":10000,"last_server_duration_us":73273,"total_duration_us":10008636},{"operation_name":"upsert","last_dispatch_duration_us":1316393,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10841798,"total_server_duration_us":1270315,"encode_duration_us":21709,"operation_id":"0x51fe","timeout_ms":10000,"last_server_duration_us":194167,"total_duration_us":10008632},{"operation_name":"upsert","last_dispatch_duration_us":9816117,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10872908,"total_server_duration_us":10515472,"encode_duration_us":18970,"operation_id":"0x511b","timeout_ms":10000,"last_server_duration_us":9761422,"total_duration_us":10007935},{"operation_name":"upsert","last_dispatch_duration_us":40331,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8257347,"total_server_duration_us":265285,"encode_duration_us":20782,"operation_id":"0x52da","timeout_ms":10000,"last_server_duration_us":20045,"total_duration_us":10006958},{"operation_name":"upsert","last_dispatch_duration_us":88303,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8174047,"total_server_duration_us":298006,"encode_duration_us":19097,"operation_id":"0x52e4","timeout_ms":10000,"last_server_duration_us":21649,"total_duration_us":10006195},{"operation_name":"upsert","last_dispatch_duration_us":471824,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6919902,"total_server_duration_us":2313433,"encode_duration_us":21695,"operation_id":"0x5267","timeout_ms":10000,"last_server_duration_us":135224,"total_duration_us":10004718},{"operation_name":"upsert","last_dispatch_duration_us":766377,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8463377,"total_server_duration_us":1775248,"encode_duration_us":18905,"operation_id":"0x52d3","timeout_ms":10000,"last_server_duration_us":247968,"total_duration_us":10001817},{"operation_name":"upsert","last_dispatch_duration_us":82583,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8175073,"total_server_duration_us":285347,"encode_duration_us":19630,"operation_id":"0x52e7","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10001546},{"operation_name":"upsert","last_dispatch_duration_us":9047629,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":9310667,"total_server_duration_us":9188397,"encode_duration_us":18231,"operation_id":"0x52e6","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":9455309}],"total_count":24}} INFO [2022-01-10 05:43:20,299] jepsen worker 26 - jepsen.util 2186 :ok :read [8 27] INFO [2022-01-10 05:43:20,300] jepsen worker 26 - jepsen.util 2186 :invoke :write [8 8] INFO [2022-01-10 05:43:20,331] jepsen worker 1 - jepsen.util 2701 :info :write [0 4] :RequestTimeoutException INFO [2022-01-10 05:43:20,332] jepsen worker 1 - jepsen.util 2791 :invoke :read [0 nil] INFO [2022-01-10 05:43:20,392] jepsen worker 86 - jepsen.util 2066 :ok :read [28 45] INFO [2022-01-10 05:43:20,393] jepsen worker 86 - jepsen.util 2066 :invoke :write [28 11] INFO [2022-01-10 05:43:20,456] jepsen worker 55 - jepsen.util 2935 :info :write [18 11] :RequestTimeoutException INFO [2022-01-10 05:43:20,457] jepsen worker 55 - jepsen.util 3025 :invoke :read [18 nil] INFO [2022-01-10 05:43:20,552] jepsen worker 83 - jepsen.util 3053 :info :write [27 18] :RequestTimeoutException INFO [2022-01-10 05:43:20,553] jepsen worker 83 - jepsen.util 3143 :invoke :read [27 nil] INFO [2022-01-10 05:43:20,627] jepsen worker 9 - jepsen.util 3159 :ok :read [3 31] INFO [2022-01-10 05:43:20,628] jepsen worker 9 - jepsen.util 3159 :invoke :write [3 45] INFO [2022-01-10 05:43:20,630] jepsen worker 31 - jepsen.util 3091 :ok :read [10 43] INFO [2022-01-10 05:43:20,631] jepsen worker 31 - jepsen.util 3091 :invoke :read [10 nil] INFO [2022-01-10 05:43:20,660] jepsen worker 31 - jepsen.util 3091 :ok :read [10 43] INFO [2022-01-10 05:43:20,660] jepsen worker 31 - jepsen.util 3091 :invoke :write [10 19] INFO [2022-01-10 05:43:20,715] jepsen worker 29 - jepsen.util 2549 :ok :read [9 24] INFO [2022-01-10 05:43:20,717] jepsen worker 29 - jepsen.util 2549 :invoke :write [9 12] INFO [2022-01-10 05:43:20,818] jepsen worker 65 - jepsen.util 2495 :ok :read [21 7] INFO [2022-01-10 05:43:20,833] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4093567251461988}, :ns_1@172.23.107.227 {:progress 0.281524926686217}, :ns_1@172.23.105.187 {:progress 0.281524926686217}, :ns_1@172.23.107.228 {:progress 0.4}} INFO [2022-01-10 05:43:20,861] jepsen worker 65 - jepsen.util 2495 :invoke :read [21 nil] INFO [2022-01-10 05:43:20,861] jepsen worker nemesis - couchbase.util Rebalance check count: 353 INFO [2022-01-10 05:43:20,861] jepsen worker nemesis - couchbase.util Rebalance stuck count: 280 INFO [2022-01-10 05:43:21,231] jepsen worker 16 - jepsen.util 2626 :info :write [5 6] :RequestTimeoutException INFO [2022-01-10 05:43:21,235] jepsen worker 16 - jepsen.util 2716 :invoke :write [5 19] INFO [2022-01-10 05:43:21,337] jepsen worker 35 - jepsen.util 2195 :ok :read [11 21] INFO [2022-01-10 05:43:21,337] jepsen worker 35 - jepsen.util 2195 :invoke :write [11 42] INFO [2022-01-10 05:43:21,338] jepsen worker 72 - jepsen.util 2862 :ok :read [24 18] INFO [2022-01-10 05:43:21,338] jepsen worker 72 - jepsen.util 2862 :invoke :write [24 21] INFO [2022-01-10 05:43:21,446] jepsen worker 42 - jepsen.util 2472 :info :write [14 11] :RequestTimeoutException INFO [2022-01-10 05:43:21,447] jepsen worker 42 - jepsen.util 2562 :invoke :write [14 9] INFO [2022-01-10 05:43:21,511] jepsen worker 54 - jepsen.util 2664 :info :write [18 38] :RequestTimeoutException INFO [2022-01-10 05:43:21,511] jepsen worker 54 - jepsen.util 2754 :invoke :write [18 34] INFO [2022-01-10 05:43:21,556] jepsen worker 63 - jepsen.util 2403 :info :write [21 15] :RequestTimeoutException INFO [2022-01-10 05:43:21,557] jepsen worker 63 - jepsen.util 2493 :invoke :write [21 13] INFO [2022-01-10 05:43:21,625] jepsen worker 84 - jepsen.util 2514 :info :write [28 12] :RequestTimeoutException INFO [2022-01-10 05:43:21,626] jepsen worker 84 - jepsen.util 2604 :invoke :read [28 nil] INFO [2022-01-10 05:43:21,644] jepsen worker 45 - jepsen.util 3015 :ok :read [15 23] INFO [2022-01-10 05:43:21,644] jepsen worker 45 - jepsen.util 3015 :invoke :read [15 nil] WARN [2022-01-10 05:43:21,675] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17968500,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":26606929,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9108460,"encode_duration_us":16206,"operation_id":"0x4fd6","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10002316}],"total_count":1}} WARN [2022-01-10 05:43:21,675] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17286193,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":17286193,"total_server_duration_us":22389,"encode_duration_us":17833,"operation_id":"0x51a9","timeout_ms":10000,"last_server_duration_us":22389,"total_duration_us":10010978},{"operation_name":"upsert","last_dispatch_duration_us":37579,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":976371,"total_server_duration_us":621601,"encode_duration_us":16293,"operation_id":"0x5264","timeout_ms":10000,"last_server_duration_us":22223,"total_duration_us":10010095},{"operation_name":"upsert","last_dispatch_duration_us":1552982,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7844694,"total_server_duration_us":468322,"encode_duration_us":19073,"operation_id":"0x52d8","timeout_ms":10000,"last_server_duration_us":58874,"total_duration_us":10009709},{"operation_name":"upsert","last_dispatch_duration_us":498636,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":3932803,"total_server_duration_us":636543,"encode_duration_us":21404,"operation_id":"0x5344","timeout_ms":10000,"last_server_duration_us":120142,"total_duration_us":10009592},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":18593,"operation_id":"0x51d5","timeout_ms":10000,"total_duration_us":10009063},{"operation_name":"upsert","last_dispatch_duration_us":1406940,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":4240725,"total_server_duration_us":585231,"encode_duration_us":19633,"operation_id":"0x523a","timeout_ms":10000,"last_server_duration_us":70152,"total_duration_us":10008677},{"operation_name":"upsert","last_dispatch_duration_us":2109067,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3914400,"total_server_duration_us":1886986,"encode_duration_us":20272,"operation_id":"0x5212","timeout_ms":10000,"last_server_duration_us":1438163,"total_duration_us":10008630},{"operation_name":"upsert","last_dispatch_duration_us":1340822,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":4306727,"total_server_duration_us":1394040,"encode_duration_us":21264,"operation_id":"0x530b","timeout_ms":10000,"last_server_duration_us":870618,"total_duration_us":10008433},{"operation_name":"upsert","last_dispatch_duration_us":946657,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3083030,"total_server_duration_us":211619,"encode_duration_us":17771,"operation_id":"0x51e4","timeout_ms":10000,"last_server_duration_us":56034,"total_duration_us":10006881},{"operation_name":"upsert","last_dispatch_duration_us":1765563,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":6502634,"total_server_duration_us":697429,"encode_duration_us":20698,"operation_id":"0x52ee","timeout_ms":10000,"last_server_duration_us":89734,"total_duration_us":10005842}],"total_count":16}} WARN [2022-01-10 05:43:21,752] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17698093,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":19978945,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9319003,"encode_duration_us":23178,"operation_id":"0x510d","timeout_ms":10000,"last_server_duration_us":9043735,"total_duration_us":10007067},{"operation_name":"upsert","last_dispatch_duration_us":10473520,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12513666,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9442340,"encode_duration_us":18412,"operation_id":"0x5180","timeout_ms":10000,"last_server_duration_us":9265708,"total_duration_us":10009300}],"total_count":2}} WARN [2022-01-10 05:43:21,765] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":34600,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":6082301,"total_server_duration_us":347753,"encode_duration_us":24013,"operation_id":"0x52a7","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10008412},{"operation_name":"upsert","last_dispatch_duration_us":8817549,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10114902,"total_server_duration_us":325942,"encode_duration_us":16645,"operation_id":"0x5315","timeout_ms":10000,"last_server_duration_us":130113,"total_duration_us":10008349},{"operation_name":"upsert","last_dispatch_duration_us":10509491,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10509491,"total_server_duration_us":128369,"encode_duration_us":16945,"operation_id":"0x51e2","timeout_ms":10000,"last_server_duration_us":128369,"total_duration_us":10007432},{"operation_name":"upsert","last_dispatch_duration_us":9196348,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11330786,"total_server_duration_us":400300,"encode_duration_us":16984,"operation_id":"0x52c8","timeout_ms":10000,"last_server_duration_us":122177,"total_duration_us":10004693},{"operation_name":"upsert","last_dispatch_duration_us":10211547,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10211547,"total_server_duration_us":116782,"encode_duration_us":17367,"operation_id":"0x51c5","timeout_ms":10000,"last_server_duration_us":116782,"total_duration_us":10004135},{"operation_name":"upsert","last_dispatch_duration_us":8477368,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":9043088,"total_server_duration_us":380373,"encode_duration_us":21266,"operation_id":"0x51d3","timeout_ms":10000,"last_server_duration_us":22720,"total_duration_us":10003497},{"operation_name":"upsert","last_dispatch_duration_us":8938426,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11272118,"total_server_duration_us":300163,"encode_duration_us":16423,"operation_id":"0x52c9","timeout_ms":10000,"last_server_duration_us":80968,"total_duration_us":10002856},{"operation_name":"upsert","last_dispatch_duration_us":10399815,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":17818514,"total_server_duration_us":9250152,"encode_duration_us":18579,"operation_id":"0x5164","timeout_ms":10000,"last_server_duration_us":9128839,"total_duration_us":10002433},{"operation_name":"upsert","last_dispatch_duration_us":9310821,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10809663,"total_server_duration_us":618390,"encode_duration_us":17186,"operation_id":"0x52d5","timeout_ms":10000,"last_server_duration_us":499933,"total_duration_us":10002114},{"operation_name":"upsert","last_dispatch_duration_us":9042711,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9042711,"total_server_duration_us":9030999,"encode_duration_us":17193,"operation_id":"0x5208","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":9046661}],"total_count":21}} WARN [2022-01-10 05:43:21,770] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":880600,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6638173,"total_server_duration_us":1351152,"encode_duration_us":17500,"operation_id":"0x5237","timeout_ms":10000,"last_server_duration_us":158588,"total_duration_us":10010363},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":31824,"operation_id":"0x52d9","timeout_ms":10000,"total_duration_us":10010301},{"operation_name":"upsert","last_dispatch_duration_us":875593,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7251503,"total_server_duration_us":813814,"encode_duration_us":16149,"operation_id":"0x52a1","timeout_ms":10000,"last_server_duration_us":55061,"total_duration_us":10008284},{"operation_name":"upsert","last_dispatch_duration_us":84576,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":933982,"total_server_duration_us":560304,"encode_duration_us":16721,"operation_id":"0x5273","timeout_ms":10000,"last_server_duration_us":40270,"total_duration_us":10007437},{"operation_name":"upsert","last_dispatch_duration_us":860346,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":7341500,"total_server_duration_us":2207826,"encode_duration_us":17095,"operation_id":"0x51e3","timeout_ms":10000,"last_server_duration_us":848764,"total_duration_us":10007101},{"operation_name":"upsert","last_dispatch_duration_us":1833961,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6374618,"total_server_duration_us":557518,"encode_duration_us":22203,"operation_id":"0x51ae","timeout_ms":10000,"last_server_duration_us":61774,"total_duration_us":10006814},{"operation_name":"upsert","last_dispatch_duration_us":622377,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9049216,"total_server_duration_us":1484495,"encode_duration_us":25040,"operation_id":"0x534a","timeout_ms":10000,"last_server_duration_us":154240,"total_duration_us":10006021},{"operation_name":"upsert","last_dispatch_duration_us":675193,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6665911,"total_server_duration_us":2327864,"encode_duration_us":18907,"operation_id":"0x5346","timeout_ms":10000,"last_server_duration_us":172334,"total_duration_us":10005943},{"operation_name":"upsert","last_dispatch_duration_us":7720544,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8356586,"total_server_duration_us":203281,"encode_duration_us":18314,"operation_id":"0x51d4","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10005120},{"operation_name":"upsert","last_dispatch_duration_us":7745782,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8522144,"total_server_duration_us":232213,"encode_duration_us":18892,"operation_id":"0x51af","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10002898}],"total_count":21}} WARN [2022-01-10 05:43:21,853] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10384880,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10384880,"total_server_duration_us":43,"operation_id":"0x5239","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":10387043},{"operation_name":"upsert","last_dispatch_duration_us":13274216,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":13274216,"total_server_duration_us":23054,"encode_duration_us":16529,"operation_id":"0x526f","timeout_ms":10000,"last_server_duration_us":23054,"total_duration_us":10019202},{"operation_name":"upsert","last_dispatch_duration_us":2661631,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8136026,"total_server_duration_us":841606,"encode_duration_us":18168,"operation_id":"0x530d","timeout_ms":10000,"last_server_duration_us":422721,"total_duration_us":10010144},{"operation_name":"upsert","last_dispatch_duration_us":2334881,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4569236,"total_server_duration_us":366356,"encode_duration_us":17798,"operation_id":"0x5288","timeout_ms":10000,"last_server_duration_us":96557,"total_duration_us":10009951},{"operation_name":"upsert","last_dispatch_duration_us":2502628,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7282335,"total_server_duration_us":356856,"encode_duration_us":28669,"operation_id":"0x5295","timeout_ms":10000,"last_server_duration_us":91532,"total_duration_us":10009789},{"operation_name":"upsert","last_dispatch_duration_us":10473520,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12513666,"total_server_duration_us":9442340,"encode_duration_us":18412,"operation_id":"0x5180","timeout_ms":10000,"last_server_duration_us":9265708,"total_duration_us":10009300},{"operation_name":"upsert","last_dispatch_duration_us":14842395,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":14842395,"total_server_duration_us":20760,"encode_duration_us":27261,"operation_id":"0x521f","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10006896},{"operation_name":"upsert","last_dispatch_duration_us":2409500,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8842365,"total_server_duration_us":825503,"encode_duration_us":18355,"operation_id":"0x5287","timeout_ms":10000,"last_server_duration_us":158779,"total_duration_us":10006111},{"operation_name":"upsert","last_dispatch_duration_us":41263,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9349252,"total_server_duration_us":256953,"encode_duration_us":16585,"operation_id":"0x52cc","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10004562},{"operation_name":"upsert","last_dispatch_duration_us":2492931,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10073920,"total_server_duration_us":495667,"encode_duration_us":18453,"operation_id":"0x51b2","timeout_ms":10000,"last_server_duration_us":141143,"total_duration_us":10004395}],"total_count":32}} WARN [2022-01-10 05:43:21,865] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17842737,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18978187,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9346953,"encode_duration_us":17702,"operation_id":"0x50e6","timeout_ms":10000,"last_server_duration_us":9191820,"total_duration_us":10005525},{"operation_name":"upsert","last_dispatch_duration_us":17821652,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17821652,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9250696,"encode_duration_us":18509,"operation_id":"0x512d","timeout_ms":10000,"last_server_duration_us":9250696,"total_duration_us":10006923},{"operation_name":"upsert","last_dispatch_duration_us":9052236,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16588761,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9254856,"encode_duration_us":16665,"operation_id":"0x5199","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":10009250}],"total_count":3}} WARN [2022-01-10 05:43:21,866] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10399815,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17818514,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9250152,"encode_duration_us":18579,"operation_id":"0x5164","timeout_ms":10000,"last_server_duration_us":9128839,"total_duration_us":10002433},{"operation_name":"upsert","last_dispatch_duration_us":9912227,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14247906,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":10005679,"encode_duration_us":18644,"operation_id":"0x50d9","timeout_ms":10000,"last_server_duration_us":9567038,"total_duration_us":10004546},{"operation_name":"upsert","last_dispatch_duration_us":9830975,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14585840,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":10250433,"encode_duration_us":23632,"operation_id":"0x50b0","timeout_ms":10000,"last_server_duration_us":9356000,"total_duration_us":10002585}],"total_count":3}} INFO [2022-01-10 05:43:21,868] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4093567251461988}, :ns_1@172.23.107.227 {:progress 0.281524926686217}, :ns_1@172.23.105.187 {:progress 0.281524926686217}, :ns_1@172.23.107.228 {:progress 0.4}} INFO [2022-01-10 05:43:21,868] jepsen worker nemesis - couchbase.util Rebalance check count: 354 INFO [2022-01-10 05:43:21,868] jepsen worker nemesis - couchbase.util Rebalance stuck count: 281 WARN [2022-01-10 05:43:21,870] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16850587,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17690378,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9379636,"encode_duration_us":16710,"operation_id":"0x51b1","timeout_ms":10000,"last_server_duration_us":9180063,"total_duration_us":10000730},{"operation_name":"upsert","last_dispatch_duration_us":16531517,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":24995514,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9351764,"encode_duration_us":23305,"operation_id":"0x501c","timeout_ms":10000,"last_server_duration_us":9108596,"total_duration_us":10009692}],"total_count":2}} INFO [2022-01-10 05:43:21,942] jepsen worker 44 - jepsen.util 2564 :info :write [14 30] :RequestTimeoutException INFO [2022-01-10 05:43:21,943] jepsen worker 44 - jepsen.util 2654 :invoke :write [14 40] WARN [2022-01-10 05:43:22,066] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8685975,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10123973,"total_server_duration_us":208833,"encode_duration_us":16560,"operation_id":"0x5207","timeout_ms":10000,"last_server_duration_us":60002,"total_duration_us":10010473},{"operation_name":"upsert","last_dispatch_duration_us":629743,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7118828,"total_server_duration_us":1239769,"encode_duration_us":23771,"operation_id":"0x5177","timeout_ms":10000,"last_server_duration_us":61265,"total_duration_us":10009895},{"operation_name":"upsert","last_dispatch_duration_us":8998391,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8998391,"total_server_duration_us":72862,"encode_duration_us":16817,"operation_id":"0x5226","timeout_ms":10000,"last_server_duration_us":72862,"total_duration_us":10009776},{"operation_name":"upsert","last_dispatch_duration_us":9052236,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":16588761,"total_server_duration_us":9254856,"encode_duration_us":16665,"operation_id":"0x5199","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":10009250},{"operation_name":"upsert","last_dispatch_duration_us":37111,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":1168110,"total_server_duration_us":548450,"encode_duration_us":17356,"operation_id":"0x5308","timeout_ms":10000,"last_server_duration_us":20921,"total_duration_us":10009232},{"operation_name":"upsert","last_dispatch_duration_us":559172,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9670950,"total_server_duration_us":1436725,"encode_duration_us":18944,"operation_id":"0x51e1","timeout_ms":10000,"last_server_duration_us":85011,"total_duration_us":10008826},{"operation_name":"upsert","last_dispatch_duration_us":815952,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8790328,"total_server_duration_us":1221945,"encode_duration_us":20425,"operation_id":"0x52d4","timeout_ms":10000,"last_server_duration_us":76312,"total_duration_us":10006356},{"operation_name":"upsert","last_dispatch_duration_us":1372956,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9329458,"total_server_duration_us":1114903,"encode_duration_us":17633,"operation_id":"0x5297","timeout_ms":10000,"last_server_duration_us":133099,"total_duration_us":10005893},{"operation_name":"upsert","last_dispatch_duration_us":2041668,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10307433,"total_server_duration_us":2972910,"encode_duration_us":21116,"operation_id":"0x5174","timeout_ms":10000,"last_server_duration_us":1520744,"total_duration_us":10004524},{"operation_name":"upsert","last_dispatch_duration_us":38682,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":6629844,"total_server_duration_us":259278,"encode_duration_us":17948,"operation_id":"0x51cc","timeout_ms":10000,"last_server_duration_us":22389,"total_duration_us":10004112}],"total_count":20}} INFO [2022-01-10 05:43:22,704] jepsen worker 57 - jepsen.util 3117 :info :write [19 41] :RequestTimeoutException INFO [2022-01-10 05:43:22,705] jepsen worker 57 - jepsen.util 3207 :invoke :write [19 11] INFO [2022-01-10 05:43:22,780] jepsen worker 65 - jepsen.util 2495 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:22,781] jepsen worker 65 - jepsen.util 2495 :invoke :read [21 nil] INFO [2022-01-10 05:43:22,873] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4093567251461988}, :ns_1@172.23.107.227 {:progress 0.281524926686217}, :ns_1@172.23.105.187 {:progress 0.281524926686217}, :ns_1@172.23.107.228 {:progress 0.4}} INFO [2022-01-10 05:43:22,873] jepsen worker nemesis - couchbase.util Rebalance check count: 355 INFO [2022-01-10 05:43:22,874] jepsen worker nemesis - couchbase.util Rebalance stuck count: 282 INFO [2022-01-10 05:43:23,300] jepsen worker 5 - jepsen.util 2795 :info :write [1 31] :RequestTimeoutException INFO [2022-01-10 05:43:23,300] jepsen worker 5 - jepsen.util 2885 :invoke :read [1 nil] INFO [2022-01-10 05:43:23,426] jepsen worker 88 - jepsen.util 2608 :info :write [29 3] :RequestTimeoutException INFO [2022-01-10 05:43:23,426] jepsen worker 88 - jepsen.util 2698 :invoke :write [29 32] INFO [2022-01-10 05:43:23,481] jepsen worker 38 - jepsen.util 2918 :info :write [12 32] :RequestTimeoutException INFO [2022-01-10 05:43:23,481] jepsen worker 38 - jepsen.util 3008 :invoke :write [12 7] INFO [2022-01-10 05:43:23,680] jepsen worker 19 - jepsen.util 2719 :info :write [6 5] :RequestTimeoutException INFO [2022-01-10 05:43:23,680] jepsen worker 19 - jepsen.util 2809 :invoke :write [6 20] INFO [2022-01-10 05:43:23,880] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4093567251461988}, :ns_1@172.23.107.227 {:progress 0.281524926686217}, :ns_1@172.23.105.187 {:progress 0.281524926686217}, :ns_1@172.23.107.228 {:progress 0.4}} INFO [2022-01-10 05:43:23,880] jepsen worker nemesis - couchbase.util Rebalance check count: 356 INFO [2022-01-10 05:43:23,880] jepsen worker nemesis - couchbase.util Rebalance stuck count: 283 INFO [2022-01-10 05:43:23,931] jepsen worker 51 - jepsen.util 3021 :info :write [17 10] :RequestTimeoutException INFO [2022-01-10 05:43:23,932] jepsen worker 51 - jepsen.util 3111 :invoke :read [17 nil] INFO [2022-01-10 05:43:23,955] jepsen worker 51 - jepsen.util 3111 :ok :read [17 41] INFO [2022-01-10 05:43:23,956] jepsen worker 51 - jepsen.util 3111 :invoke :read [17 nil] INFO [2022-01-10 05:43:23,961] jepsen worker 56 - jepsen.util 2846 :info :write [18 8] :RequestTimeoutException INFO [2022-01-10 05:43:23,961] jepsen worker 56 - jepsen.util 2936 :invoke :read [18 nil] INFO [2022-01-10 05:43:23,987] jepsen worker 56 - jepsen.util 2936 :ok :read [18 14] INFO [2022-01-10 05:43:23,988] jepsen worker 56 - jepsen.util 2936 :invoke :write [18 36] INFO [2022-01-10 05:43:24,010] jepsen worker 25 - jepsen.util 2455 :info :write [8 16] :RequestTimeoutException INFO [2022-01-10 05:43:24,010] jepsen worker 25 - jepsen.util 2545 :invoke :write [8 1] INFO [2022-01-10 05:43:24,018] jepsen worker 51 - jepsen.util 3111 :ok :read [17 41] INFO [2022-01-10 05:43:24,019] jepsen worker 51 - jepsen.util 3111 :invoke :read [17 nil] INFO [2022-01-10 05:43:24,044] jepsen worker 51 - jepsen.util 3111 :ok :read [17 41] INFO [2022-01-10 05:43:24,044] jepsen worker 51 - jepsen.util 3111 :invoke :read [17 nil] INFO [2022-01-10 05:43:24,082] jepsen worker 51 - jepsen.util 3111 :ok :read [17 41] INFO [2022-01-10 05:43:24,082] jepsen worker 51 - jepsen.util 3111 :invoke :read [17 nil] INFO [2022-01-10 05:43:24,130] jepsen worker 22 - jepsen.util 2362 :ok :read [7 :nil] INFO [2022-01-10 05:43:24,131] jepsen worker 22 - jepsen.util 2362 :invoke :read [7 nil] INFO [2022-01-10 05:43:24,132] jepsen worker 89 - jepsen.util 3059 :info :write [29 8] :RequestTimeoutException INFO [2022-01-10 05:43:24,133] jepsen worker 89 - jepsen.util 3149 :invoke :write [29 48] INFO [2022-01-10 05:43:24,146] jepsen worker 51 - jepsen.util 3111 :ok :read [17 41] INFO [2022-01-10 05:43:24,147] jepsen worker 51 - jepsen.util 3111 :invoke :write [17 36] INFO [2022-01-10 05:43:24,226] jepsen worker 48 - jepsen.util 2208 :info :write [16 31] :RequestTimeoutException INFO [2022-01-10 05:43:24,227] jepsen worker 48 - jepsen.util 2298 :invoke :read [16 nil] INFO [2022-01-10 05:43:24,326] jepsen worker 39 - jepsen.util 3009 :info :write [13 14] :SyncWriteAmbiguous INFO [2022-01-10 05:43:24,327] jepsen worker 39 - jepsen.util 3099 :invoke :read [13 nil] INFO [2022-01-10 05:43:24,345] jepsen worker 8 - jepsen.util 2978 :info :write [2 27] :RequestTimeoutException INFO [2022-01-10 05:43:24,346] jepsen worker 8 - jepsen.util 3068 :invoke :write [2 0] INFO [2022-01-10 05:43:24,358] jepsen worker 0 - jepsen.util 2700 :ok :read [0 46] INFO [2022-01-10 05:43:24,358] jepsen worker 0 - jepsen.util 2700 :invoke :read [0 nil] INFO [2022-01-10 05:43:24,400] jepsen worker 20 - jepsen.util 2360 :info :write [6 30] :RequestTimeoutException INFO [2022-01-10 05:43:24,400] jepsen worker 20 - jepsen.util 2450 :invoke :write [6 14] INFO [2022-01-10 05:43:24,471] jepsen worker 6 - jepsen.util 2706 :info :write [2 32] :RequestTimeoutException INFO [2022-01-10 05:43:24,472] jepsen worker 6 - jepsen.util 2796 :invoke :read [2 nil] INFO [2022-01-10 05:43:24,716] jepsen worker 61 - jepsen.util 2581 :info :write [20 20] :RequestTimeoutException INFO [2022-01-10 05:43:24,717] jepsen worker 61 - jepsen.util 2671 :invoke :write [20 3] INFO [2022-01-10 05:43:24,744] jepsen worker 85 - jepsen.util 2875 :info :write [28 1] :RequestTimeoutException INFO [2022-01-10 05:43:24,745] jepsen worker 85 - jepsen.util 2965 :invoke :write [28 21] INFO [2022-01-10 05:43:24,752] jepsen worker 28 - jepsen.util 2548 :ok :read [9 24] INFO [2022-01-10 05:43:24,753] jepsen worker 28 - jepsen.util 2548 :invoke :read [9 nil] INFO [2022-01-10 05:43:24,881] jepsen worker 45 - jepsen.util 3015 :ok :read [15 23] INFO [2022-01-10 05:43:24,882] jepsen worker 45 - jepsen.util 3015 :invoke :read [15 nil] INFO [2022-01-10 05:43:24,886] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4093567251461988}, :ns_1@172.23.107.227 {:progress 0.281524926686217}, :ns_1@172.23.105.187 {:progress 0.281524926686217}, :ns_1@172.23.107.228 {:progress 0.4}} INFO [2022-01-10 05:43:24,886] jepsen worker nemesis - couchbase.util Rebalance check count: 357 INFO [2022-01-10 05:43:24,886] jepsen worker nemesis - couchbase.util Rebalance stuck count: 284 INFO [2022-01-10 05:43:24,892] jepsen worker 65 - jepsen.util 2495 :ok :read [21 7] INFO [2022-01-10 05:43:24,892] jepsen worker 65 - jepsen.util 2495 :invoke :read [21 nil] INFO [2022-01-10 05:43:25,042] jepsen worker 60 - jepsen.util 3030 :info :write [20 46] :RequestTimeoutException INFO [2022-01-10 05:43:25,042] jepsen worker 60 - jepsen.util 3120 :invoke :read [20 nil] INFO [2022-01-10 05:43:25,065] jepsen worker 17 - jepsen.util 2717 :ok :read [5 42] INFO [2022-01-10 05:43:25,066] jepsen worker 17 - jepsen.util 2717 :invoke :read [5 nil] INFO [2022-01-10 05:43:25,629] jepsen worker 1 - jepsen.util 2791 :ok :read [0 46] INFO [2022-01-10 05:43:25,629] jepsen worker 1 - jepsen.util 2791 :invoke :read [0 nil] INFO [2022-01-10 05:43:25,634] jepsen worker 66 - jepsen.util 2406 :ok :read [22 41] INFO [2022-01-10 05:43:25,634] jepsen worker 66 - jepsen.util 2406 :invoke :read [22 nil] INFO [2022-01-10 05:43:25,672] jepsen worker 77 - jepsen.util 2957 :info :write [25 44] :RequestTimeoutException INFO [2022-01-10 05:43:25,672] jepsen worker 75 - jepsen.util 1965 :info :write [25 13] :RequestTimeoutException INFO [2022-01-10 05:43:25,673] jepsen worker 75 - jepsen.util 2055 :invoke :write [25 34] INFO [2022-01-10 05:43:25,673] jepsen worker 77 - jepsen.util 3047 :invoke :write [25 44] INFO [2022-01-10 05:43:25,726] jepsen worker 80 - jepsen.util 3140 :info :write [26 23] :SyncWriteAmbiguous INFO [2022-01-10 05:43:25,727] jepsen worker 80 - jepsen.util 3230 :invoke :write [26 21] INFO [2022-01-10 05:43:25,746] jepsen worker 7 - jepsen.util 2707 :info :write [2 9] :RequestTimeoutException INFO [2022-01-10 05:43:25,829] jepsen worker 79 - jepsen.util 3049 :ok :read [26 10] INFO [2022-01-10 05:43:25,844] jepsen worker 58 - jepsen.util 2578 :info :write [19 31] :RequestTimeoutException INFO [2022-01-10 05:43:26,221] jepsen worker 58 - jepsen.util 2668 :invoke :write [19 11] INFO [2022-01-10 05:43:25,891] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4093567251461988}, :ns_1@172.23.107.227 {:progress 0.281524926686217}, :ns_1@172.23.105.187 {:progress 0.281524926686217}, :ns_1@172.23.107.228 {:progress 0.4}} INFO [2022-01-10 05:43:26,223] jepsen worker nemesis - couchbase.util Rebalance check count: 358 INFO [2022-01-10 05:43:26,223] jepsen worker nemesis - couchbase.util Rebalance stuck count: 285 INFO [2022-01-10 05:43:25,996] jepsen worker 34 - jepsen.util 2464 :info :write [11 45] :RequestTimeoutException INFO [2022-01-10 05:43:26,223] jepsen worker 79 - jepsen.util 3049 :invoke :write [26 10] INFO [2022-01-10 05:43:26,223] jepsen worker 34 - jepsen.util 2554 :invoke :write [11 34] INFO [2022-01-10 05:43:26,220] jepsen worker 7 - jepsen.util 2797 :invoke :read [2 nil] INFO [2022-01-10 05:43:26,375] jepsen worker 6 - jepsen.util 2796 :ok :read [2 14] INFO [2022-01-10 05:43:26,376] jepsen worker 6 - jepsen.util 2796 :invoke :write [2 19] INFO [2022-01-10 05:43:26,496] jepsen worker 15 - jepsen.util 2265 :info :write [5 22] :RequestTimeoutException INFO [2022-01-10 05:43:26,496] jepsen worker 81 - jepsen.util 2871 :info :write [27 44] :RequestTimeoutException INFO [2022-01-10 05:43:26,497] jepsen worker 81 - jepsen.util 2961 :invoke :read [27 nil] INFO [2022-01-10 05:43:26,497] jepsen worker 15 - jepsen.util 2355 :invoke :read [5 nil] INFO [2022-01-10 05:43:26,526] jepsen worker 27 - jepsen.util 2637 :info :write [9 38] :RequestTimeoutException INFO [2022-01-10 05:43:26,527] jepsen worker 27 - jepsen.util 2727 :invoke :write [9 46] INFO [2022-01-10 05:43:26,835] jepsen worker 82 - jepsen.util 2692 :info :write [27 15] :RequestTimeoutException INFO [2022-01-10 05:43:26,835] jepsen worker 82 - jepsen.util 2782 :invoke :read [27 nil] INFO [2022-01-10 05:43:26,891] jepsen worker 12 - jepsen.util 2802 :info :write [4 16] :RequestTimeoutException INFO [2022-01-10 05:43:26,892] jepsen worker 12 - jepsen.util 2892 :invoke :read [4 nil] INFO [2022-01-10 05:43:27,228] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4093567251461988}, :ns_1@172.23.107.227 {:progress 0.281524926686217}, :ns_1@172.23.105.187 {:progress 0.281524926686217}, :ns_1@172.23.107.228 {:progress 0.4}} INFO [2022-01-10 05:43:27,228] jepsen worker nemesis - couchbase.util Rebalance check count: 359 INFO [2022-01-10 05:43:27,228] jepsen worker nemesis - couchbase.util Rebalance stuck count: 286 INFO [2022-01-10 05:43:27,506] jepsen worker 46 - jepsen.util 2836 :info :write [15 15] :RequestTimeoutException INFO [2022-01-10 05:43:27,507] jepsen worker 46 - jepsen.util 2926 :invoke :read [15 nil] INFO [2022-01-10 05:43:27,565] jepsen worker 18 - jepsen.util 2448 :info :write [6 13] :RequestTimeoutException INFO [2022-01-10 05:43:27,565] jepsen worker 18 - jepsen.util 2538 :invoke :write [6 12] INFO [2022-01-10 05:43:27,671] jepsen worker 78 - jepsen.util 3048 :info :write [26 40] :RequestTimeoutException INFO [2022-01-10 05:43:27,672] jepsen worker 78 - jepsen.util 3138 :invoke :write [26 20] INFO [2022-01-10 05:43:27,801] jepsen worker 22 - jepsen.util 2362 :ok :read [7 :nil] INFO [2022-01-10 05:43:27,802] jepsen worker 22 - jepsen.util 2362 :invoke :write [7 36] INFO [2022-01-10 05:43:27,824] jepsen worker 24 - jepsen.util 2184 :info :write [8 16] :RequestTimeoutException INFO [2022-01-10 05:43:27,825] jepsen worker 24 - jepsen.util 2274 :invoke :write [8 26] INFO [2022-01-10 05:43:27,861] jepsen worker 30 - jepsen.util 2550 :info :write [10 15] :SyncWriteAmbiguous INFO [2022-01-10 05:43:27,863] jepsen worker 30 - jepsen.util 2640 :invoke :write [10 38] INFO [2022-01-10 05:43:27,964] jepsen worker 60 - jepsen.util 3120 :ok :read [20 :nil] INFO [2022-01-10 05:43:27,965] jepsen worker 60 - jepsen.util 3120 :invoke :write [20 23] INFO [2022-01-10 05:43:27,977] jepsen worker 45 - jepsen.util 3015 :ok :read [15 23] INFO [2022-01-10 05:43:27,978] jepsen worker 45 - jepsen.util 3015 :invoke :read [15 nil] INFO [2022-01-10 05:43:28,042] jepsen worker 50 - jepsen.util 2750 :info :write [16 11] :RequestTimeoutException INFO [2022-01-10 05:43:28,043] jepsen worker 50 - jepsen.util 2840 :invoke :write [16 15] INFO [2022-01-10 05:43:28,131] jepsen worker 7 - jepsen.util 2797 :ok :read [2 14] INFO [2022-01-10 05:43:28,131] jepsen worker 7 - jepsen.util 2797 :invoke :read [2 nil] INFO [2022-01-10 05:43:28,206] jepsen worker 52 - jepsen.util 3292 :info :write [17 36] :SyncWriteAmbiguous INFO [2022-01-10 05:43:28,207] jepsen worker 52 - jepsen.util 3382 :invoke :read [17 nil] INFO [2022-01-10 05:43:28,230] jepsen worker 52 - jepsen.util 3382 :ok :read [17 41] INFO [2022-01-10 05:43:28,231] jepsen worker 52 - jepsen.util 3382 :invoke :read [17 nil] INFO [2022-01-10 05:43:28,234] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4093567251461988}, :ns_1@172.23.107.227 {:progress 0.281524926686217}, :ns_1@172.23.105.187 {:progress 0.281524926686217}, :ns_1@172.23.107.228 {:progress 0.4}} INFO [2022-01-10 05:43:28,234] jepsen worker nemesis - couchbase.util Rebalance check count: 360 INFO [2022-01-10 05:43:28,234] jepsen worker nemesis - couchbase.util Rebalance stuck count: 287 INFO [2022-01-10 05:43:28,246] jepsen worker 67 - jepsen.util 2407 :info :write [22 10] :RequestTimeoutException INFO [2022-01-10 05:43:28,247] jepsen worker 67 - jepsen.util 2497 :invoke :write [22 32] INFO [2022-01-10 05:43:28,253] jepsen worker 52 - jepsen.util 3382 :ok :read [17 41] INFO [2022-01-10 05:43:28,254] jepsen worker 52 - jepsen.util 3382 :invoke :write [17 49] INFO [2022-01-10 05:43:28,455] jepsen worker 59 - jepsen.util 2669 :info :write [19 39] :RequestTimeoutException INFO [2022-01-10 05:43:28,455] jepsen worker 59 - jepsen.util 2759 :invoke :read [19 nil] INFO [2022-01-10 05:43:28,480] jepsen worker 64 - jepsen.util 2584 :info :write [21 31] :RequestTimeoutException INFO [2022-01-10 05:43:28,481] jepsen worker 64 - jepsen.util 2674 :invoke :read [21 nil] INFO [2022-01-10 05:43:28,550] jepsen worker 73 - jepsen.util 2413 :info :write [24 19] :RequestTimeoutException INFO [2022-01-10 05:43:28,551] jepsen worker 73 - jepsen.util 2503 :invoke :read [24 nil] INFO [2022-01-10 05:43:28,646] jepsen worker 23 - jepsen.util 2723 :info :write [7 7] :RequestTimeoutException INFO [2022-01-10 05:43:28,646] jepsen worker 23 - jepsen.util 2813 :invoke :write [7 12] INFO [2022-01-10 05:43:28,706] jepsen worker 49 - jepsen.util 2569 :info :write [16 13] :RequestTimeoutException INFO [2022-01-10 05:43:28,706] jepsen worker 49 - jepsen.util 2659 :invoke :read [16 nil] INFO [2022-01-10 05:43:28,719] jepsen worker 47 - jepsen.util 2567 :info :write [15 31] :RequestTimeoutException INFO [2022-01-10 05:43:28,720] jepsen worker 47 - jepsen.util 2657 :invoke :write [15 7] INFO [2022-01-10 05:43:28,739] jepsen worker 84 - jepsen.util 2604 :ok :read [28 45] INFO [2022-01-10 05:43:28,739] jepsen worker 84 - jepsen.util 2604 :invoke :read [28 nil] INFO [2022-01-10 05:43:28,779] jepsen worker 13 - jepsen.util 2893 :info :write [4 12] :RequestTimeoutException INFO [2022-01-10 05:43:28,780] jepsen worker 13 - jepsen.util 2983 :invoke :write [4 11] INFO [2022-01-10 05:43:28,905] jepsen worker 68 - jepsen.util 2588 :info :write [22 25] :RequestTimeoutException INFO [2022-01-10 05:43:28,905] jepsen worker 68 - jepsen.util 2678 :invoke :read [22 nil] INFO [2022-01-10 05:43:28,935] jepsen worker 87 - jepsen.util 2787 :info :write [29 30] :RequestTimeoutException INFO [2022-01-10 05:43:28,936] jepsen worker 87 - jepsen.util 2877 :invoke :write [29 36] INFO [2022-01-10 05:43:28,950] jepsen worker 39 - jepsen.util 3099 :ok :read [13 15] INFO [2022-01-10 05:43:28,951] jepsen worker 39 - jepsen.util 3099 :invoke :write [13 27] INFO [2022-01-10 05:43:28,962] jepsen worker 66 - jepsen.util 2406 :ok :read [22 41] INFO [2022-01-10 05:43:28,963] jepsen worker 66 - jepsen.util 2406 :invoke :read [22 nil] INFO [2022-01-10 05:43:29,056] jepsen worker 10 - jepsen.util 2980 :ok :read [3 31] INFO [2022-01-10 05:43:29,057] jepsen worker 10 - jepsen.util 2980 :invoke :read [3 nil] INFO [2022-01-10 05:43:29,061] jepsen worker 48 - jepsen.util 2298 :ok :read [16 38] INFO [2022-01-10 05:43:29,061] jepsen worker 11 - jepsen.util 2711 :ok :read [3 31] INFO [2022-01-10 05:43:29,062] jepsen worker 48 - jepsen.util 2298 :invoke :write [16 26] INFO [2022-01-10 05:43:29,062] jepsen worker 11 - jepsen.util 2711 :invoke :write [3 11] INFO [2022-01-10 05:43:29,064] jepsen worker 32 - jepsen.util 2822 :info :write [10 11] :RequestTimeoutException INFO [2022-01-10 05:43:29,065] jepsen worker 32 - jepsen.util 2912 :invoke :write [10 20] INFO [2022-01-10 05:43:29,094] jepsen worker 3 - jepsen.util 2703 :ok :read [1 10] INFO [2022-01-10 05:43:29,095] jepsen worker 3 - jepsen.util 2703 :invoke :write [1 29] INFO [2022-01-10 05:43:29,136] jepsen worker 84 - jepsen.util 2604 :ok :read [28 45] INFO [2022-01-10 05:43:29,137] jepsen worker 84 - jepsen.util 2604 :invoke :read [28 nil] INFO [2022-01-10 05:43:29,185] jepsen worker 53 - jepsen.util 3203 :info :write [17 43] :RequestTimeoutException INFO [2022-01-10 05:43:29,186] jepsen worker 53 - jepsen.util 3293 :invoke :read [17 nil] INFO [2022-01-10 05:43:29,233] jepsen worker 49 - jepsen.util 2659 :ok :read [16 38] INFO [2022-01-10 05:43:29,233] jepsen worker 49 - jepsen.util 2659 :invoke :read [16 nil] INFO [2022-01-10 05:43:29,235] jepsen worker 82 - jepsen.util 2782 :ok :read [27 30] INFO [2022-01-10 05:43:29,236] jepsen worker 82 - jepsen.util 2782 :invoke :read [27 nil] INFO [2022-01-10 05:43:29,240] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4093567251461988}, :ns_1@172.23.107.227 {:progress 0.281524926686217}, :ns_1@172.23.105.187 {:progress 0.281524926686217}, :ns_1@172.23.107.228 {:progress 0.4}} INFO [2022-01-10 05:43:29,240] jepsen worker nemesis - couchbase.util Rebalance check count: 361 INFO [2022-01-10 05:43:29,240] jepsen worker nemesis - couchbase.util Rebalance stuck count: 288 INFO [2022-01-10 05:43:29,244] jepsen worker 66 - jepsen.util 2406 :ok :read [22 41] INFO [2022-01-10 05:43:29,245] jepsen worker 66 - jepsen.util 2406 :invoke :read [22 nil] INFO [2022-01-10 05:43:29,260] jepsen worker 76 - jepsen.util 2146 :info :write [25 1] :RequestTimeoutException INFO [2022-01-10 05:43:29,260] jepsen worker 2 - jepsen.util 2702 :info :write [0 43] :RequestTimeoutException INFO [2022-01-10 05:43:29,260] jepsen worker 2 - jepsen.util 2792 :invoke :write [0 29] INFO [2022-01-10 05:43:29,261] jepsen worker 76 - jepsen.util 2236 :invoke :write [25 38] INFO [2022-01-10 05:43:29,312] jepsen worker 33 - jepsen.util 3003 :info :write [11 0] :RequestTimeoutException INFO [2022-01-10 05:43:29,312] jepsen worker 33 - jepsen.util 3093 :invoke :read [11 nil] INFO [2022-01-10 05:43:29,342] jepsen worker 28 - jepsen.util 2548 :ok :read [9 24] INFO [2022-01-10 05:43:29,342] jepsen worker 28 - jepsen.util 2548 :invoke :read [9 nil] INFO [2022-01-10 05:43:29,424] jepsen worker 43 - jepsen.util 2473 :info :write [14 7] :RequestTimeoutException INFO [2022-01-10 05:43:29,425] jepsen worker 43 - jepsen.util 2563 :invoke :write [14 33] INFO [2022-01-10 05:43:29,456] jepsen worker 17 - jepsen.util 2717 :ok :read [5 42] INFO [2022-01-10 05:43:29,456] jepsen worker 17 - jepsen.util 2717 :invoke :read [5 nil] INFO [2022-01-10 05:43:29,619] jepsen worker 71 - jepsen.util 2231 :ok :read [23 26] INFO [2022-01-10 05:43:29,620] jepsen worker 71 - jepsen.util 2231 :invoke :read [23 nil] INFO [2022-01-10 05:43:29,640] jepsen worker 40 - jepsen.util 2110 :info :write [13 45] :RequestTimeoutException INFO [2022-01-10 05:43:29,641] jepsen worker 40 - jepsen.util 2200 :invoke :read [13 nil] INFO [2022-01-10 05:43:29,642] jepsen worker 70 - jepsen.util 2770 :info :write [23 15] :RequestTimeoutException INFO [2022-01-10 05:43:29,643] jepsen worker 70 - jepsen.util 2860 :invoke :write [23 1] INFO [2022-01-10 05:43:29,655] jepsen worker 37 - jepsen.util 2737 :ok :read [12 29] INFO [2022-01-10 05:43:29,656] jepsen worker 37 - jepsen.util 2737 :invoke :read [12 nil] INFO [2022-01-10 05:43:29,676] jepsen worker 4 - jepsen.util 2974 :info :write [1 25] :RequestTimeoutException INFO [2022-01-10 05:43:29,677] jepsen worker 4 - jepsen.util 3064 :invoke :read [1 nil] INFO [2022-01-10 05:43:29,681] jepsen worker 84 - jepsen.util 2604 :ok :read [28 45] INFO [2022-01-10 05:43:29,682] jepsen worker 84 - jepsen.util 2604 :invoke :read [28 nil] INFO [2022-01-10 05:43:29,701] jepsen worker 49 - jepsen.util 2659 :ok :read [16 38] INFO [2022-01-10 05:43:29,701] jepsen worker 49 - jepsen.util 2659 :invoke :write [16 25] INFO [2022-01-10 05:43:29,703] jepsen worker 4 - jepsen.util 3064 :ok :read [1 10] INFO [2022-01-10 05:43:29,704] jepsen worker 4 - jepsen.util 3064 :invoke :read [1 nil] INFO [2022-01-10 05:43:29,716] jepsen worker 66 - jepsen.util 2406 :ok :read [22 41] INFO [2022-01-10 05:43:29,717] jepsen worker 66 - jepsen.util 2406 :invoke :write [22 30] INFO [2022-01-10 05:43:29,730] jepsen worker 4 - jepsen.util 3064 :ok :read [1 10] INFO [2022-01-10 05:43:29,731] jepsen worker 4 - jepsen.util 3064 :invoke :read [1 nil] INFO [2022-01-10 05:43:29,741] jepsen worker 74 - jepsen.util 3224 :info :write [24 1] :RequestTimeoutException INFO [2022-01-10 05:43:29,741] jepsen worker 74 - jepsen.util 3314 :invoke :write [24 15] INFO [2022-01-10 05:43:29,756] jepsen worker 4 - jepsen.util 3064 :ok :read [1 10] INFO [2022-01-10 05:43:29,757] jepsen worker 4 - jepsen.util 3064 :invoke :write [1 0] INFO [2022-01-10 05:43:29,865] jepsen worker 14 - jepsen.util 3164 :info :write [4 46] :RequestTimeoutException INFO [2022-01-10 05:43:29,866] jepsen worker 65 - jepsen.util 2495 :ok :read [21 7] INFO [2022-01-10 05:43:29,867] jepsen worker 14 - jepsen.util 3254 :invoke :write [4 31] INFO [2022-01-10 05:43:29,868] jepsen worker 65 - jepsen.util 2495 :invoke :write [21 41] INFO [2022-01-10 05:43:29,876] jepsen worker 62 - jepsen.util 2852 :info :write [20 45] :RequestTimeoutException INFO [2022-01-10 05:43:29,877] jepsen worker 62 - jepsen.util 2942 :invoke :write [20 10] INFO [2022-01-10 05:43:29,902] jepsen worker 46 - jepsen.util 2926 :ok :read [15 23] INFO [2022-01-10 05:43:29,903] jepsen worker 46 - jepsen.util 2926 :invoke :read [15 nil] INFO [2022-01-10 05:43:30,109] jepsen worker 41 - jepsen.util 2471 :info :write [13 20] :RequestTimeoutException INFO [2022-01-10 05:43:30,110] jepsen worker 41 - jepsen.util 2561 :invoke :write [13 43] INFO [2022-01-10 05:43:30,132] jepsen worker 21 - jepsen.util 2811 :info :write [7 23] :RequestTimeoutException INFO [2022-01-10 05:43:30,132] jepsen worker 21 - jepsen.util 2901 :invoke :write [7 40] WARN [2022-01-10 05:43:30,185] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2190812,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8142230,"total_server_duration_us":3048661,"encode_duration_us":20602,"operation_id":"0x53d5","timeout_ms":10000,"last_server_duration_us":1021734,"total_duration_us":10010415},{"operation_name":"upsert","last_dispatch_duration_us":1480049,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7647879,"total_server_duration_us":1349823,"encode_duration_us":17715,"operation_id":"0x549a","timeout_ms":10000,"last_server_duration_us":98413,"total_duration_us":10009972},{"operation_name":"upsert","last_dispatch_duration_us":38168,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":1152156,"total_server_duration_us":613753,"encode_duration_us":18090,"operation_id":"0x54e7","timeout_ms":10000,"last_server_duration_us":22554,"total_duration_us":10009644},{"operation_name":"upsert","last_dispatch_duration_us":33159,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":1273930,"total_server_duration_us":593119,"encode_duration_us":17511,"operation_id":"0x54dd","timeout_ms":10000,"last_server_duration_us":21568,"total_duration_us":10009436},{"operation_name":"upsert","last_dispatch_duration_us":2147881,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10052132,"total_server_duration_us":890540,"encode_duration_us":18486,"operation_id":"0x5377","timeout_ms":10000,"last_server_duration_us":46062,"total_duration_us":10008931},{"operation_name":"upsert","last_dispatch_duration_us":950583,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4148841,"total_server_duration_us":592971,"encode_duration_us":16559,"operation_id":"0x53ec","timeout_ms":10000,"last_server_duration_us":36119,"total_duration_us":10008795},{"operation_name":"upsert","last_dispatch_duration_us":30964,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":1159045,"total_server_duration_us":518653,"encode_duration_us":16397,"operation_id":"0x5512","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10007330},{"operation_name":"upsert","last_dispatch_duration_us":1836476,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9491828,"total_server_duration_us":2909478,"encode_duration_us":25368,"operation_id":"0x54ee","timeout_ms":10000,"last_server_duration_us":199169,"total_duration_us":10006969},{"operation_name":"upsert","last_dispatch_duration_us":2258216,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10813439,"total_server_duration_us":1615796,"encode_duration_us":18536,"operation_id":"0x534c","timeout_ms":10000,"last_server_duration_us":732825,"total_duration_us":10005579},{"operation_name":"upsert","last_dispatch_duration_us":1351470,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":1351470,"total_server_duration_us":132042,"encode_duration_us":17349,"operation_id":"0x54a7","timeout_ms":10000,"last_server_duration_us":132042,"total_duration_us":10005241}],"total_count":16}} INFO [2022-01-10 05:43:30,245] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2873900293255132}, :ns_1@172.23.105.187 {:progress 0.2873900293255132}, :ns_1@172.23.107.228 {:progress 0.4058823529411765}} INFO [2022-01-10 05:43:30,245] jepsen worker nemesis - couchbase.util Rebalance check count: 362 INFO [2022-01-10 05:43:30,245] jepsen worker nemesis - couchbase.util Rebalance stuck count: 288 INFO [2022-01-10 05:43:30,326] jepsen worker 26 - jepsen.util 2186 :info :write [8 8] :RequestTimeoutException INFO [2022-01-10 05:43:30,327] jepsen worker 26 - jepsen.util 2276 :invoke :read [8 nil] INFO [2022-01-10 05:43:30,420] jepsen worker 86 - jepsen.util 2066 :info :write [28 11] :RequestTimeoutException INFO [2022-01-10 05:43:30,421] jepsen worker 86 - jepsen.util 2156 :invoke :read [28 nil] INFO [2022-01-10 05:43:30,655] jepsen worker 9 - jepsen.util 3159 :info :write [3 45] :RequestTimeoutException INFO [2022-01-10 05:43:30,656] jepsen worker 9 - jepsen.util 3249 :invoke :write [3 30] INFO [2022-01-10 05:43:30,686] jepsen worker 31 - jepsen.util 3091 :info :write [10 19] :RequestTimeoutException INFO [2022-01-10 05:43:30,687] jepsen worker 31 - jepsen.util 3181 :invoke :read [10 nil] INFO [2022-01-10 05:43:30,881] jepsen worker 29 - jepsen.util 2549 :info :write [9 12] :RequestTimeoutException INFO [2022-01-10 05:43:30,882] jepsen worker 29 - jepsen.util 2639 :invoke :write [9 28] INFO [2022-01-10 05:43:30,886] jepsen worker 1 - jepsen.util 2791 :ok :read [0 46] INFO [2022-01-10 05:43:30,987] jepsen worker 83 - jepsen.util 3143 :ok :read [27 30] INFO [2022-01-10 05:43:31,388] jepsen worker 83 - jepsen.util 3143 :invoke :write [27 19] INFO [2022-01-10 05:43:31,166] jepsen worker 45 - jepsen.util 3015 :ok :read [15 23] INFO [2022-01-10 05:43:31,253] jepsen worker 84 - jepsen.util 2604 :ok :read [28 45] INFO [2022-01-10 05:43:31,256] jepsen worker 16 - jepsen.util 2716 :info :write [5 19] :RequestTimeoutException INFO [2022-01-10 05:43:31,258] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2873900293255132}, :ns_1@172.23.105.187 {:progress 0.2873900293255132}, :ns_1@172.23.107.228 {:progress 0.4058823529411765}} INFO [2022-01-10 05:43:31,395] jepsen worker 16 - jepsen.util 2806 :invoke :read [5 nil] INFO [2022-01-10 05:43:31,395] jepsen worker nemesis - couchbase.util Rebalance check count: 363 INFO [2022-01-10 05:43:31,395] jepsen worker nemesis - couchbase.util Rebalance stuck count: 289 INFO [2022-01-10 05:43:31,268] jepsen worker 82 - jepsen.util 2782 :ok :read [27 30] INFO [2022-01-10 05:43:31,356] jepsen worker 35 - jepsen.util 2195 :info :write [11 42] :RequestTimeoutException INFO [2022-01-10 05:43:31,362] jepsen worker 72 - jepsen.util 2862 :info :write [24 21] :RequestTimeoutException INFO [2022-01-10 05:43:31,395] jepsen worker 82 - jepsen.util 2782 :invoke :read [27 nil] INFO [2022-01-10 05:43:31,395] jepsen worker 35 - jepsen.util 2285 :invoke :read [11 nil] INFO [2022-01-10 05:43:31,395] jepsen worker 45 - jepsen.util 3015 :invoke :read [15 nil] INFO [2022-01-10 05:43:31,388] jepsen worker 1 - jepsen.util 2791 :invoke :write [0 25] INFO [2022-01-10 05:43:31,395] jepsen worker 84 - jepsen.util 2604 :invoke :write [28 34] INFO [2022-01-10 05:43:31,395] jepsen worker 72 - jepsen.util 2952 :invoke :write [24 21] INFO [2022-01-10 05:43:31,470] jepsen worker 42 - jepsen.util 2562 :info :write [14 9] :RequestTimeoutException INFO [2022-01-10 05:43:31,470] jepsen worker 42 - jepsen.util 2652 :invoke :read [14 nil] INFO [2022-01-10 05:43:31,531] jepsen worker 54 - jepsen.util 2754 :info :write [18 34] :RequestTimeoutException INFO [2022-01-10 05:43:31,532] jepsen worker 54 - jepsen.util 2844 :invoke :write [18 49] INFO [2022-01-10 05:43:31,576] jepsen worker 63 - jepsen.util 2493 :info :write [21 13] :RequestTimeoutException INFO [2022-01-10 05:43:31,577] jepsen worker 63 - jepsen.util 2583 :invoke :write [21 2] INFO [2022-01-10 05:43:31,652] jepsen worker 81 - jepsen.util 2961 :ok :read [27 30] INFO [2022-01-10 05:43:31,653] jepsen worker 81 - jepsen.util 2961 :invoke :write [27 30] WARN [2022-01-10 05:43:31,688] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9686908,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13993635,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10477094,"encode_duration_us":21264,"operation_id":"0x530b","timeout_ms":10000,"last_server_duration_us":9083054,"total_duration_us":10008433},{"operation_name":"upsert","last_dispatch_duration_us":25648444,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":25648444,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9032060,"encode_duration_us":18593,"operation_id":"0x51d5","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10009063}],"total_count":2}} WARN [2022-01-10 05:43:31,688] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18142840,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":18142840,"total_server_duration_us":27,"operation_id":"0x5347","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":18143769},{"operation_name":"get","last_dispatch_duration_us":17115176,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":17115176,"total_server_duration_us":43,"operation_id":"0x5376","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":17116155},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":17233,"operation_id":"0x53c7","timeout_ms":10000,"total_duration_us":10009786},{"operation_name":"upsert","last_dispatch_duration_us":3015278,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9095772,"total_server_duration_us":433913,"encode_duration_us":21797,"operation_id":"0x53db","timeout_ms":10000,"last_server_duration_us":78132,"total_duration_us":10006998},{"operation_name":"upsert","last_dispatch_duration_us":3069131,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9763430,"total_server_duration_us":1835203,"encode_duration_us":17296,"operation_id":"0x54a8","timeout_ms":10000,"last_server_duration_us":1043673,"total_duration_us":10006281},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":17148,"operation_id":"0x53a0","timeout_ms":10000,"total_duration_us":10006190},{"operation_name":"upsert","last_dispatch_duration_us":3577518,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7941506,"total_server_duration_us":207658,"encode_duration_us":16059,"operation_id":"0x54ff","timeout_ms":10000,"last_server_duration_us":41337,"total_duration_us":10002370},{"operation_name":"upsert","last_dispatch_duration_us":33137,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":1221380,"total_server_duration_us":705649,"encode_duration_us":16303,"operation_id":"0x5467","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10002358},{"operation_name":"get","last_dispatch_duration_us":7095959,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7095959,"total_server_duration_us":10874,"operation_id":"0x5552","timeout_ms":30000,"last_server_duration_us":10874,"total_duration_us":7098134},{"operation_name":"get","last_dispatch_duration_us":6712005,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":6712005,"total_server_duration_us":6927,"operation_id":"0x54d7","timeout_ms":30000,"last_server_duration_us":6927,"total_duration_us":6713922}],"total_count":25}} INFO [2022-01-10 05:43:31,699] jepsen worker 33 - jepsen.util 3093 :ok :read [11 21] INFO [2022-01-10 05:43:31,699] jepsen worker 33 - jepsen.util 3093 :invoke :read [11 nil] WARN [2022-01-10 05:43:31,779] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":5137816,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10668152,"total_server_duration_us":366853,"encode_duration_us":16995,"operation_id":"0x540c","timeout_ms":10000,"last_server_duration_us":96711,"total_duration_us":10010663},{"operation_name":"upsert","last_dispatch_duration_us":6890838,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13994579,"total_server_duration_us":784333,"encode_duration_us":19632,"operation_id":"0x540d","timeout_ms":10000,"last_server_duration_us":75062,"total_duration_us":10008617},{"operation_name":"upsert","last_dispatch_duration_us":30613,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":92479,"total_server_duration_us":58648,"encode_duration_us":17055,"operation_id":"0x54f0","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10008467},{"operation_name":"upsert","last_dispatch_duration_us":4075246,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":4075246,"total_server_duration_us":68946,"encode_duration_us":16573,"operation_id":"0x5517","timeout_ms":10000,"last_server_duration_us":68946,"total_duration_us":10008144},{"operation_name":"upsert","last_dispatch_duration_us":2275992,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":5708687,"total_server_duration_us":676397,"encode_duration_us":16289,"operation_id":"0x536e","timeout_ms":10000,"last_server_duration_us":236131,"total_duration_us":10005161},{"operation_name":"upsert","last_dispatch_duration_us":6215251,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6215251,"total_server_duration_us":47192,"encode_duration_us":18308,"operation_id":"0x554e","timeout_ms":10000,"last_server_duration_us":47192,"total_duration_us":10004771},{"operation_name":"upsert","last_dispatch_duration_us":29080,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":1019852,"total_server_duration_us":604181,"encode_duration_us":18297,"operation_id":"0x547a","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10004645},{"operation_name":"upsert","last_dispatch_duration_us":7844851,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8595368,"total_server_duration_us":441791,"encode_duration_us":20363,"operation_id":"0x53cf","timeout_ms":10000,"last_server_duration_us":23898,"total_duration_us":10004080},{"operation_name":"upsert","last_dispatch_duration_us":4514799,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9629827,"total_server_duration_us":492812,"encode_duration_us":20465,"operation_id":"0x5404","timeout_ms":10000,"last_server_duration_us":88543,"total_duration_us":10003171},{"operation_name":"upsert","last_dispatch_duration_us":3403681,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10522270,"total_server_duration_us":2019607,"encode_duration_us":17611,"operation_id":"0x549e","timeout_ms":10000,"last_server_duration_us":134337,"total_duration_us":10003163}],"total_count":17}} WARN [2022-01-10 05:43:31,783] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1652669,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10219635,"total_server_duration_us":1533721,"encode_duration_us":18941,"operation_id":"0x5514","timeout_ms":10000,"last_server_duration_us":361551,"total_duration_us":10009832},{"operation_name":"upsert","last_dispatch_duration_us":2015216,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11649859,"total_server_duration_us":2768576,"encode_duration_us":17118,"operation_id":"0x541b","timeout_ms":10000,"last_server_duration_us":97793,"total_duration_us":10009679},{"operation_name":"upsert","last_dispatch_duration_us":1751458,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10356912,"total_server_duration_us":1658148,"encode_duration_us":19363,"operation_id":"0x5472","timeout_ms":10000,"last_server_duration_us":155369,"total_duration_us":10007621},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":17104,"operation_id":"0x54f2","timeout_ms":10000,"total_duration_us":10002663},{"operation_name":"upsert","last_dispatch_duration_us":1937369,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9158641,"total_server_duration_us":829241,"encode_duration_us":16339,"operation_id":"0x5551","timeout_ms":10000,"last_server_duration_us":191687,"total_duration_us":10002279},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":16824,"operation_id":"0x539f","timeout_ms":10000,"total_duration_us":10000657},{"operation_name":"upsert","last_dispatch_duration_us":9295509,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9673693,"total_server_duration_us":9632120,"encode_duration_us":17308,"operation_id":"0x53de","timeout_ms":10000,"last_server_duration_us":9266781,"total_duration_us":9687021},{"operation_name":"upsert","last_dispatch_duration_us":9046975,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9046975,"total_server_duration_us":9025695,"encode_duration_us":16943,"operation_id":"0x5444","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":9050881},{"operation_name":"get","last_dispatch_duration_us":8201981,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":8201981,"total_server_duration_us":56,"operation_id":"0x5478","timeout_ms":30000,"last_server_duration_us":56,"total_duration_us":8204080},{"operation_name":"get","last_dispatch_duration_us":5842705,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":5842705,"total_server_duration_us":195621,"operation_id":"0x54d3","timeout_ms":30000,"last_server_duration_us":195621,"total_duration_us":5844663}],"total_count":16}} WARN [2022-01-10 05:43:31,867] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7649392,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":14860317,"total_server_duration_us":996225,"encode_duration_us":18544,"operation_id":"0x543e","timeout_ms":10000,"last_server_duration_us":247508,"total_duration_us":10009571},{"operation_name":"upsert","last_dispatch_duration_us":4351259,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8068586,"total_server_duration_us":1600227,"encode_duration_us":16344,"operation_id":"0x551e","timeout_ms":10000,"last_server_duration_us":573027,"total_duration_us":10009523},{"operation_name":"upsert","last_dispatch_duration_us":4671602,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6694088,"total_server_duration_us":179390,"encode_duration_us":17365,"operation_id":"0x53e5","timeout_ms":10000,"last_server_duration_us":101539,"total_duration_us":10008683},{"operation_name":"upsert","last_dispatch_duration_us":28638,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":1007918,"total_server_duration_us":430023,"encode_duration_us":17068,"operation_id":"0x5525","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10007996},{"operation_name":"upsert","last_dispatch_duration_us":32677,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":966011,"total_server_duration_us":499543,"encode_duration_us":20468,"operation_id":"0x5524","timeout_ms":10000,"last_server_duration_us":21731,"total_duration_us":10006622},{"operation_name":"upsert","last_dispatch_duration_us":3832540,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":4420285,"total_server_duration_us":877812,"encode_duration_us":18440,"operation_id":"0x54a6","timeout_ms":10000,"last_server_duration_us":723008,"total_duration_us":10006201},{"operation_name":"upsert","last_dispatch_duration_us":4369665,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7335930,"total_server_duration_us":1169815,"encode_duration_us":17035,"operation_id":"0x543f","timeout_ms":10000,"last_server_duration_us":145515,"total_duration_us":10003591},{"operation_name":"upsert","last_dispatch_duration_us":6365903,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10132957,"total_server_duration_us":541815,"encode_duration_us":19991,"operation_id":"0x5441","timeout_ms":10000,"last_server_duration_us":114454,"total_duration_us":10002683},{"operation_name":"upsert","last_dispatch_duration_us":3683613,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8406238,"total_server_duration_us":502761,"encode_duration_us":16489,"operation_id":"0x53d4","timeout_ms":10000,"last_server_duration_us":49603,"total_duration_us":10002304},{"operation_name":"upsert","last_dispatch_duration_us":4225438,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4225438,"total_server_duration_us":694253,"encode_duration_us":16359,"operation_id":"0x554d","timeout_ms":10000,"last_server_duration_us":694253,"total_duration_us":10001878}],"total_count":18}} WARN [2022-01-10 05:43:31,878] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16942701,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":23572545,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9288155,"encode_duration_us":17948,"operation_id":"0x51cc","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10004112},{"operation_name":"upsert","last_dispatch_duration_us":9048775,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10216886,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":9579449,"encode_duration_us":17356,"operation_id":"0x5308","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10009232}],"total_count":2}} WARN [2022-01-10 05:43:31,883] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18278931,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18278931,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9024634,"encode_duration_us":17884,"operation_id":"0x5317","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10002001}],"total_count":1}} INFO [2022-01-10 05:43:31,965] jepsen worker 44 - jepsen.util 2654 :info :write [14 40] :RequestTimeoutException INFO [2022-01-10 05:43:31,966] jepsen worker 44 - jepsen.util 2744 :invoke :read [14 nil] INFO [2022-01-10 05:43:32,034] jepsen worker 26 - jepsen.util 2276 :ok :read [8 27] INFO [2022-01-10 05:43:32,034] jepsen worker 26 - jepsen.util 2276 :invoke :write [8 23] INFO [2022-01-10 05:43:32,039] jepsen worker 64 - jepsen.util 2674 :ok :read [21 7] INFO [2022-01-10 05:43:32,040] jepsen worker 64 - jepsen.util 2674 :invoke :write [21 17] WARN [2022-01-10 05:43:32,079] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15232287,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":15232287,"total_server_duration_us":38,"operation_id":"0x53bc","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":15233402},{"operation_name":"get","last_dispatch_duration_us":11693259,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":11693259,"total_server_duration_us":49,"operation_id":"0x5470","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":11694523},{"operation_name":"upsert","last_dispatch_duration_us":5247650,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8602804,"total_server_duration_us":258892,"encode_duration_us":16872,"operation_id":"0x5520","timeout_ms":10000,"last_server_duration_us":96096,"total_duration_us":10009931},{"operation_name":"upsert","last_dispatch_duration_us":2735634,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10104355,"total_server_duration_us":1939634,"encode_duration_us":17052,"operation_id":"0x54aa","timeout_ms":10000,"last_server_duration_us":1353287,"total_duration_us":10009931},{"operation_name":"upsert","last_dispatch_duration_us":6757240,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":6757240,"total_server_duration_us":22223,"encode_duration_us":17696,"operation_id":"0x539d","timeout_ms":10000,"last_server_duration_us":22223,"total_duration_us":10009338},{"operation_name":"upsert","last_dispatch_duration_us":1680410,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10047645,"total_server_duration_us":1495410,"encode_duration_us":19202,"operation_id":"0x53d6","timeout_ms":10000,"last_server_duration_us":132394,"total_duration_us":10008963},{"operation_name":"upsert","last_dispatch_duration_us":2531402,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11110025,"total_server_duration_us":2901269,"encode_duration_us":20378,"operation_id":"0x54b5","timeout_ms":10000,"last_server_duration_us":509892,"total_duration_us":10007859},{"operation_name":"upsert","last_dispatch_duration_us":1877509,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9827059,"total_server_duration_us":1141703,"encode_duration_us":16512,"operation_id":"0x53a2","timeout_ms":10000,"last_server_duration_us":263108,"total_duration_us":10007381},{"operation_name":"upsert","last_dispatch_duration_us":7417449,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11991726,"total_server_duration_us":321128,"encode_duration_us":17546,"operation_id":"0x54f1","timeout_ms":10000,"last_server_duration_us":170957,"total_duration_us":10005801},{"operation_name":"upsert","last_dispatch_duration_us":1634377,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8738485,"total_server_duration_us":1582306,"encode_duration_us":22178,"operation_id":"0x53ca","timeout_ms":10000,"last_server_duration_us":45838,"total_duration_us":10004099}],"total_count":18}} INFO [2022-01-10 05:43:32,400] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2873900293255132}, :ns_1@172.23.105.187 {:progress 0.2873900293255132}, :ns_1@172.23.107.228 {:progress 0.4058823529411765}} INFO [2022-01-10 05:43:32,400] jepsen worker nemesis - couchbase.util Rebalance check count: 364 INFO [2022-01-10 05:43:32,401] jepsen worker nemesis - couchbase.util Rebalance stuck count: 290 INFO [2022-01-10 05:43:32,537] jepsen worker 0 - jepsen.util 2700 :ok :read [0 46] INFO [2022-01-10 05:43:32,537] jepsen worker 0 - jepsen.util 2700 :invoke :read [0 nil] INFO [2022-01-10 05:43:32,725] jepsen worker 57 - jepsen.util 3207 :info :write [19 11] :RequestTimeoutException INFO [2022-01-10 05:43:32,726] jepsen worker 57 - jepsen.util 3297 :invoke :read [19 nil] INFO [2022-01-10 05:43:33,406] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2873900293255132}, :ns_1@172.23.105.187 {:progress 0.2873900293255132}, :ns_1@172.23.107.228 {:progress 0.4058823529411765}} INFO [2022-01-10 05:43:33,406] jepsen worker nemesis - couchbase.util Rebalance check count: 365 INFO [2022-01-10 05:43:33,406] jepsen worker nemesis - couchbase.util Rebalance stuck count: 291 INFO [2022-01-10 05:43:33,452] jepsen worker 88 - jepsen.util 2698 :info :write [29 32] :RequestTimeoutException INFO [2022-01-10 05:43:33,453] jepsen worker 88 - jepsen.util 2788 :invoke :write [29 2] INFO [2022-01-10 05:43:33,506] jepsen worker 38 - jepsen.util 3008 :info :write [12 7] :RequestTimeoutException INFO [2022-01-10 05:43:33,507] jepsen worker 38 - jepsen.util 3098 :invoke :read [12 nil] INFO [2022-01-10 05:43:33,701] jepsen worker 19 - jepsen.util 2809 :info :write [6 20] :RequestTimeoutException INFO [2022-01-10 05:43:33,701] jepsen worker 19 - jepsen.util 2899 :invoke :read [6 nil] INFO [2022-01-10 05:43:33,771] jepsen worker 28 - jepsen.util 2548 :ok :read [9 24] INFO [2022-01-10 05:43:33,771] jepsen worker 28 - jepsen.util 2548 :invoke :read [9 nil] INFO [2022-01-10 05:43:33,789] jepsen worker 17 - jepsen.util 2717 :ok :read [5 42] INFO [2022-01-10 05:43:33,790] jepsen worker 17 - jepsen.util 2717 :invoke :read [5 nil] INFO [2022-01-10 05:43:33,794] jepsen worker 82 - jepsen.util 2782 :ok :read [27 30] INFO [2022-01-10 05:43:33,794] jepsen worker 82 - jepsen.util 2782 :invoke :write [27 2] INFO [2022-01-10 05:43:33,964] jepsen worker 40 - jepsen.util 2200 :ok :read [13 15] INFO [2022-01-10 05:43:33,965] jepsen worker 40 - jepsen.util 2200 :invoke :read [13 nil] INFO [2022-01-10 05:43:33,971] jepsen worker 68 - jepsen.util 2678 :ok :read [22 41] INFO [2022-01-10 05:43:33,971] jepsen worker 68 - jepsen.util 2678 :invoke :write [22 25] INFO [2022-01-10 05:43:34,015] jepsen worker 56 - jepsen.util 2936 :info :write [18 36] :RequestTimeoutException INFO [2022-01-10 05:43:34,016] jepsen worker 56 - jepsen.util 3026 :invoke :read [18 nil] INFO [2022-01-10 05:43:34,032] jepsen worker 46 - jepsen.util 2926 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:34,033] jepsen worker 46 - jepsen.util 2926 :invoke :read [15 nil] INFO [2022-01-10 05:43:34,040] jepsen worker 25 - jepsen.util 2545 :info :write [8 1] :RequestTimeoutException INFO [2022-01-10 05:43:34,041] jepsen worker 25 - jepsen.util 2635 :invoke :write [8 0] INFO [2022-01-10 05:43:34,064] jepsen worker 35 - jepsen.util 2285 :ok :read [11 21] INFO [2022-01-10 05:43:34,065] jepsen worker 35 - jepsen.util 2285 :invoke :write [11 26] INFO [2022-01-10 05:43:34,158] jepsen worker 42 - jepsen.util 2652 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:34,159] jepsen worker 42 - jepsen.util 2652 :invoke :write [14 29] INFO [2022-01-10 05:43:34,163] jepsen worker 89 - jepsen.util 3149 :info :write [29 48] :RequestTimeoutException INFO [2022-01-10 05:43:34,163] jepsen worker 89 - jepsen.util 3239 :invoke :read [29 nil] INFO [2022-01-10 05:43:34,175] jepsen worker 51 - jepsen.util 3111 :info :write [17 36] :RequestTimeoutException INFO [2022-01-10 05:43:34,176] jepsen worker 51 - jepsen.util 3201 :invoke :write [17 34] INFO [2022-01-10 05:43:34,190] jepsen worker 33 - jepsen.util 3093 :ok :read [11 21] INFO [2022-01-10 05:43:34,191] jepsen worker 33 - jepsen.util 3093 :invoke :read [11 nil] INFO [2022-01-10 05:43:34,198] jepsen worker 89 - jepsen.util 3239 :ok :read [29 0] INFO [2022-01-10 05:43:34,198] jepsen worker 89 - jepsen.util 3239 :invoke :read [29 nil] INFO [2022-01-10 05:43:34,222] jepsen worker 89 - jepsen.util 3239 :ok :read [29 0] INFO [2022-01-10 05:43:34,222] jepsen worker 89 - jepsen.util 3239 :invoke :write [29 15] INFO [2022-01-10 05:43:34,375] jepsen worker 8 - jepsen.util 3068 :info :write [2 0] :RequestTimeoutException INFO [2022-01-10 05:43:34,375] jepsen worker 8 - jepsen.util 3158 :invoke :write [2 43] INFO [2022-01-10 05:43:34,412] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2873900293255132}, :ns_1@172.23.105.187 {:progress 0.2873900293255132}, :ns_1@172.23.107.228 {:progress 0.4058823529411765}} INFO [2022-01-10 05:43:34,412] jepsen worker nemesis - couchbase.util Rebalance check count: 366 INFO [2022-01-10 05:43:34,412] jepsen worker nemesis - couchbase.util Rebalance stuck count: 292 INFO [2022-01-10 05:43:34,426] jepsen worker 20 - jepsen.util 2450 :info :write [6 14] :RequestTimeoutException INFO [2022-01-10 05:43:34,426] jepsen worker 20 - jepsen.util 2540 :invoke :write [6 42] INFO [2022-01-10 05:43:34,742] jepsen worker 61 - jepsen.util 2671 :info :write [20 3] :RequestTimeoutException INFO [2022-01-10 05:43:34,743] jepsen worker 61 - jepsen.util 2761 :invoke :write [20 12] INFO [2022-01-10 05:43:34,766] jepsen worker 85 - jepsen.util 2965 :info :write [28 21] :RequestTimeoutException INFO [2022-01-10 05:43:34,767] jepsen worker 85 - jepsen.util 3055 :invoke :read [28 nil] INFO [2022-01-10 05:43:34,804] jepsen worker 69 - jepsen.util 2679 :ok :read [23 26] INFO [2022-01-10 05:43:34,804] jepsen worker 69 - jepsen.util 2679 :invoke :write [23 4] INFO [2022-01-10 05:43:34,841] jepsen worker 12 - jepsen.util 2892 :ok :read [4 3] INFO [2022-01-10 05:43:34,842] jepsen worker 12 - jepsen.util 2892 :invoke :read [4 nil] INFO [2022-01-10 05:43:34,870] jepsen worker 56 - jepsen.util 3026 :ok :read [18 14] INFO [2022-01-10 05:43:34,870] jepsen worker 56 - jepsen.util 3026 :invoke :write [18 11] INFO [2022-01-10 05:43:34,899] jepsen worker 12 - jepsen.util 2892 :ok :read [4 3] INFO [2022-01-10 05:43:34,900] jepsen worker 12 - jepsen.util 2892 :invoke :read [4 nil] INFO [2022-01-10 05:43:34,947] jepsen worker 12 - jepsen.util 2892 :ok :read [4 3] INFO [2022-01-10 05:43:34,947] jepsen worker 12 - jepsen.util 2892 :invoke :read [4 nil] INFO [2022-01-10 05:43:35,002] jepsen worker 12 - jepsen.util 2892 :ok :read [4 3] INFO [2022-01-10 05:43:35,003] jepsen worker 12 - jepsen.util 2892 :invoke :write [4 8] INFO [2022-01-10 05:43:35,403] jepsen worker 16 - jepsen.util 2806 :ok :read [5 42] INFO [2022-01-10 05:43:35,404] jepsen worker 16 - jepsen.util 2806 :invoke :read [5 nil] INFO [2022-01-10 05:43:35,417] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2873900293255132}, :ns_1@172.23.105.187 {:progress 0.2873900293255132}, :ns_1@172.23.107.228 {:progress 0.4058823529411765}} INFO [2022-01-10 05:43:35,417] jepsen worker nemesis - couchbase.util Rebalance check count: 367 INFO [2022-01-10 05:43:35,417] jepsen worker nemesis - couchbase.util Rebalance stuck count: 293 INFO [2022-01-10 05:43:35,695] jepsen worker 75 - jepsen.util 2055 :info :write [25 34] :RequestTimeoutException INFO [2022-01-10 05:43:35,695] jepsen worker 77 - jepsen.util 3047 :info :write [25 44] :RequestTimeoutException INFO [2022-01-10 05:43:35,696] jepsen worker 75 - jepsen.util 2145 :invoke :write [25 12] INFO [2022-01-10 05:43:35,696] jepsen worker 77 - jepsen.util 3137 :invoke :write [25 35] INFO [2022-01-10 05:43:36,227] jepsen worker 15 - jepsen.util 2355 :ok :read [5 42] INFO [2022-01-10 05:43:36,227] jepsen worker 15 - jepsen.util 2355 :invoke :read [5 nil] INFO [2022-01-10 05:43:36,235] jepsen worker 57 - jepsen.util 3297 :ok :read [19 31] INFO [2022-01-10 05:43:36,239] jepsen worker 80 - jepsen.util 3230 :info :write [26 21] :RequestTimeoutException INFO [2022-01-10 05:43:36,556] jepsen worker 57 - jepsen.util 3297 :invoke :write [19 7] INFO [2022-01-10 05:43:36,242] jepsen worker 58 - jepsen.util 2668 :info :write [19 11] :RequestTimeoutException INFO [2022-01-10 05:43:36,556] jepsen worker 80 - jepsen.util 3320 :invoke :read [26 nil] INFO [2022-01-10 05:43:36,246] jepsen worker 34 - jepsen.util 2554 :info :write [11 34] :RequestTimeoutException INFO [2022-01-10 05:43:36,246] jepsen worker 79 - jepsen.util 3049 :info :write [26 10] :RequestTimeoutException INFO [2022-01-10 05:43:36,556] jepsen worker 34 - jepsen.util 2644 :invoke :write [11 0] INFO [2022-01-10 05:43:36,556] jepsen worker 79 - jepsen.util 3139 :invoke :read [26 nil] INFO [2022-01-10 05:43:36,247] jepsen worker 59 - jepsen.util 2759 :ok :read [19 31] INFO [2022-01-10 05:43:36,399] jepsen worker 6 - jepsen.util 2796 :info :write [2 19] :RequestTimeoutException INFO [2022-01-10 05:43:36,557] jepsen worker 59 - jepsen.util 2759 :invoke :read [19 nil] INFO [2022-01-10 05:43:36,557] jepsen worker 6 - jepsen.util 2886 :invoke :write [2 7] INFO [2022-01-10 05:43:36,423] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2873900293255132}, :ns_1@172.23.105.187 {:progress 0.2873900293255132}, :ns_1@172.23.107.228 {:progress 0.4058823529411765}} INFO [2022-01-10 05:43:36,550] jepsen worker 27 - jepsen.util 2727 :info :write [9 46] :RequestTimeoutException INFO [2022-01-10 05:43:36,557] jepsen worker nemesis - couchbase.util Rebalance check count: 368 INFO [2022-01-10 05:43:36,557] jepsen worker 27 - jepsen.util 2817 :invoke :read [9 nil] INFO [2022-01-10 05:43:36,556] jepsen worker 58 - jepsen.util 2758 :invoke :read [19 nil] INFO [2022-01-10 05:43:36,557] jepsen worker nemesis - couchbase.util Rebalance stuck count: 294 INFO [2022-01-10 05:43:37,108] jepsen worker 33 - jepsen.util 3093 :ok :read [11 21] INFO [2022-01-10 05:43:37,109] jepsen worker 33 - jepsen.util 3093 :invoke :write [11 46] INFO [2022-01-10 05:43:37,131] jepsen worker 44 - jepsen.util 2744 :ok :read [14 14] INFO [2022-01-10 05:43:37,131] jepsen worker 44 - jepsen.util 2744 :invoke :read [14 nil] INFO [2022-01-10 05:43:37,157] jepsen worker 7 - jepsen.util 2797 :ok :read [2 14] INFO [2022-01-10 05:43:37,158] jepsen worker 7 - jepsen.util 2797 :invoke :read [2 nil] INFO [2022-01-10 05:43:37,245] jepsen worker 80 - jepsen.util 3320 :ok :read [26 10] INFO [2022-01-10 05:43:37,246] jepsen worker 80 - jepsen.util 3320 :invoke :read [26 nil] INFO [2022-01-10 05:43:37,403] jepsen worker 31 - jepsen.util 3181 :ok :read [10 43] INFO [2022-01-10 05:43:37,404] jepsen worker 31 - jepsen.util 3181 :invoke :read [10 nil] INFO [2022-01-10 05:43:37,427] jepsen worker 52 - jepsen.util 3382 :info :write [17 49] :SyncWriteAmbiguous INFO [2022-01-10 05:43:37,428] jepsen worker 52 - jepsen.util 3472 :invoke :read [17 nil] INFO [2022-01-10 05:43:37,451] jepsen worker 79 - jepsen.util 3139 :ok :read [26 10] INFO [2022-01-10 05:43:37,451] jepsen worker 79 - jepsen.util 3139 :invoke :write [26 13] INFO [2022-01-10 05:43:37,458] jepsen worker 31 - jepsen.util 3181 :ok :read [10 43] INFO [2022-01-10 05:43:37,458] jepsen worker 31 - jepsen.util 3181 :invoke :read [10 nil] INFO [2022-01-10 05:43:37,480] jepsen worker 31 - jepsen.util 3181 :ok :read [10 43] INFO [2022-01-10 05:43:37,480] jepsen worker 31 - jepsen.util 3181 :invoke :write [10 27] INFO [2022-01-10 05:43:37,564] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2873900293255132}, :ns_1@172.23.105.187 {:progress 0.2873900293255132}, :ns_1@172.23.107.228 {:progress 0.4058823529411765}} INFO [2022-01-10 05:43:37,564] jepsen worker nemesis - couchbase.util Rebalance check count: 369 INFO [2022-01-10 05:43:37,564] jepsen worker nemesis - couchbase.util Rebalance stuck count: 295 INFO [2022-01-10 05:43:37,585] jepsen worker 18 - jepsen.util 2538 :info :write [6 12] :RequestTimeoutException INFO [2022-01-10 05:43:37,586] jepsen worker 18 - jepsen.util 2628 :invoke :write [6 7] INFO [2022-01-10 05:43:37,661] jepsen worker 53 - jepsen.util 3293 :ok :read [17 41] INFO [2022-01-10 05:43:37,662] jepsen worker 53 - jepsen.util 3293 :invoke :read [17 nil] INFO [2022-01-10 05:43:37,663] jepsen worker 80 - jepsen.util 3320 :ok :read [26 10] INFO [2022-01-10 05:43:37,664] jepsen worker 80 - jepsen.util 3320 :invoke :read [26 nil] INFO [2022-01-10 05:43:37,696] jepsen worker 78 - jepsen.util 3138 :info :write [26 20] :RequestTimeoutException INFO [2022-01-10 05:43:37,697] jepsen worker 78 - jepsen.util 3228 :invoke :read [26 nil] INFO [2022-01-10 05:43:37,724] jepsen worker 73 - jepsen.util 2503 :ok :read [24 18] INFO [2022-01-10 05:43:37,725] jepsen worker 73 - jepsen.util 2503 :invoke :read [24 nil] INFO [2022-01-10 05:43:37,825] jepsen worker 22 - jepsen.util 2362 :info :write [7 36] :RequestTimeoutException INFO [2022-01-10 05:43:37,826] jepsen worker 22 - jepsen.util 2452 :invoke :read [7 nil] INFO [2022-01-10 05:43:37,849] jepsen worker 24 - jepsen.util 2274 :info :write [8 26] :RequestTimeoutException INFO [2022-01-10 05:43:37,850] jepsen worker 24 - jepsen.util 2364 :invoke :write [8 24] INFO [2022-01-10 05:43:37,892] jepsen worker 30 - jepsen.util 2640 :info :write [10 38] :RequestTimeoutException INFO [2022-01-10 05:43:37,893] jepsen worker 30 - jepsen.util 2730 :invoke :read [10 nil] INFO [2022-01-10 05:43:38,001] jepsen worker 60 - jepsen.util 3120 :info :write [20 23] :RequestTimeoutException INFO [2022-01-10 05:43:38,001] jepsen worker 60 - jepsen.util 3210 :invoke :read [20 nil] INFO [2022-01-10 05:43:38,066] jepsen worker 50 - jepsen.util 2840 :info :write [16 15] :RequestTimeoutException INFO [2022-01-10 05:43:38,067] jepsen worker 50 - jepsen.util 2930 :invoke :read [16 nil] INFO [2022-01-10 05:43:38,076] jepsen worker 53 - jepsen.util 3293 :ok :read [17 41] INFO [2022-01-10 05:43:38,077] jepsen worker 53 - jepsen.util 3293 :invoke :read [17 nil] INFO [2022-01-10 05:43:38,080] jepsen worker 78 - jepsen.util 3228 :ok :read [26 10] INFO [2022-01-10 05:43:38,081] jepsen worker 78 - jepsen.util 3228 :invoke :write [26 2] INFO [2022-01-10 05:43:38,105] jepsen worker 5 - jepsen.util 2885 :ok :read [1 10] INFO [2022-01-10 05:43:38,106] jepsen worker 5 - jepsen.util 2885 :invoke :read [1 nil] INFO [2022-01-10 05:43:38,111] jepsen worker 10 - jepsen.util 2980 :ok :read [3 31] INFO [2022-01-10 05:43:38,111] jepsen worker 10 - jepsen.util 2980 :invoke :write [3 47] INFO [2022-01-10 05:43:38,268] jepsen worker 53 - jepsen.util 3293 :ok :read [17 41] INFO [2022-01-10 05:43:38,268] jepsen worker 53 - jepsen.util 3293 :invoke :read [17 nil] INFO [2022-01-10 05:43:38,274] jepsen worker 67 - jepsen.util 2497 :info :write [22 32] :RequestTimeoutException INFO [2022-01-10 05:43:38,275] jepsen worker 67 - jepsen.util 2587 :invoke :read [22 nil] INFO [2022-01-10 05:43:38,569] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2873900293255132}, :ns_1@172.23.105.187 {:progress 0.2873900293255132}, :ns_1@172.23.107.228 {:progress 0.4058823529411765}} INFO [2022-01-10 05:43:38,570] jepsen worker nemesis - couchbase.util Rebalance check count: 370 INFO [2022-01-10 05:43:38,570] jepsen worker nemesis - couchbase.util Rebalance stuck count: 296 INFO [2022-01-10 05:43:38,667] jepsen worker 23 - jepsen.util 2813 :info :write [7 12] :RequestTimeoutException INFO [2022-01-10 05:43:38,667] jepsen worker 23 - jepsen.util 2903 :invoke :write [7 16] INFO [2022-01-10 05:43:38,676] jepsen worker 53 - jepsen.util 3293 :ok :read [17 41] INFO [2022-01-10 05:43:38,677] jepsen worker 53 - jepsen.util 3293 :invoke :read [17 nil] INFO [2022-01-10 05:43:38,745] jepsen worker 47 - jepsen.util 2657 :info :write [15 7] :RequestTimeoutException INFO [2022-01-10 05:43:38,748] jepsen worker 47 - jepsen.util 2747 :invoke :write [15 47] INFO [2022-01-10 05:43:38,748] jepsen worker 55 - jepsen.util 3025 :ok :read [18 14] INFO [2022-01-10 05:43:38,749] jepsen worker 0 - jepsen.util 2700 :ok :read [0 46] INFO [2022-01-10 05:43:38,750] jepsen worker 0 - jepsen.util 2700 :invoke :write [0 5] INFO [2022-01-10 05:43:38,750] jepsen worker 27 - jepsen.util 2817 :ok :read [9 24] INFO [2022-01-10 05:43:38,750] jepsen worker 55 - jepsen.util 3025 :invoke :write [18 29] INFO [2022-01-10 05:43:38,751] jepsen worker 27 - jepsen.util 2817 :invoke :read [9 nil] INFO [2022-01-10 05:43:38,764] jepsen worker 36 - jepsen.util 2376 :ok :read [12 29] INFO [2022-01-10 05:43:38,765] jepsen worker 36 - jepsen.util 2376 :invoke :write [12 17] INFO [2022-01-10 05:43:38,805] jepsen worker 86 - jepsen.util 2156 :ok :read [28 45] INFO [2022-01-10 05:43:38,806] jepsen worker 13 - jepsen.util 2983 :info :write [4 11] :RequestTimeoutException INFO [2022-01-10 05:43:38,806] jepsen worker 86 - jepsen.util 2156 :invoke :write [28 5] INFO [2022-01-10 05:43:38,807] jepsen worker 13 - jepsen.util 3073 :invoke :read [4 nil] INFO [2022-01-10 05:43:38,812] jepsen worker 71 - jepsen.util 2231 :ok :read [23 26] INFO [2022-01-10 05:43:38,813] jepsen worker 71 - jepsen.util 2231 :invoke :write [23 48] INFO [2022-01-10 05:43:38,817] jepsen worker 37 - jepsen.util 2737 :ok :read [12 29] INFO [2022-01-10 05:43:38,818] jepsen worker 37 - jepsen.util 2737 :invoke :write [12 29] INFO [2022-01-10 05:43:38,819] jepsen worker 15 - jepsen.util 2355 :ok :read [5 42] INFO [2022-01-10 05:43:38,820] jepsen worker 15 - jepsen.util 2355 :invoke :write [5 25] INFO [2022-01-10 05:43:38,891] jepsen worker 53 - jepsen.util 3293 :ok :read [17 41] INFO [2022-01-10 05:43:38,892] jepsen worker 53 - jepsen.util 3293 :invoke :write [17 17] INFO [2022-01-10 05:43:38,894] jepsen worker 52 - jepsen.util 3472 :ok :read [17 41] INFO [2022-01-10 05:43:38,895] jepsen worker 52 - jepsen.util 3472 :invoke :write [17 31] INFO [2022-01-10 05:43:38,910] jepsen worker 80 - jepsen.util 3320 :ok :read [26 10] INFO [2022-01-10 05:43:38,910] jepsen worker 80 - jepsen.util 3320 :invoke :write [26 16] INFO [2022-01-10 05:43:38,962] jepsen worker 87 - jepsen.util 2877 :info :write [29 36] :RequestTimeoutException INFO [2022-01-10 05:43:38,963] jepsen worker 87 - jepsen.util 2967 :invoke :read [29 nil] INFO [2022-01-10 05:43:38,981] jepsen worker 39 - jepsen.util 3099 :info :write [13 27] :RequestTimeoutException INFO [2022-01-10 05:43:38,982] jepsen worker 39 - jepsen.util 3189 :invoke :write [13 12] INFO [2022-01-10 05:43:39,086] jepsen worker 48 - jepsen.util 2298 :info :write [16 26] :RequestTimeoutException INFO [2022-01-10 05:43:39,087] jepsen worker 48 - jepsen.util 2388 :invoke :read [16 nil] INFO [2022-01-10 05:43:39,089] jepsen worker 11 - jepsen.util 2711 :info :write [3 11] :RequestTimeoutException INFO [2022-01-10 05:43:39,090] jepsen worker 11 - jepsen.util 2801 :invoke :read [3 nil] INFO [2022-01-10 05:43:39,096] jepsen worker 32 - jepsen.util 2912 :info :write [10 20] :RequestTimeoutException INFO [2022-01-10 05:43:39,097] jepsen worker 32 - jepsen.util 3002 :invoke :write [10 21] INFO [2022-01-10 05:43:39,120] jepsen worker 3 - jepsen.util 2703 :info :write [1 29] :RequestTimeoutException INFO [2022-01-10 05:43:39,120] jepsen worker 3 - jepsen.util 2793 :invoke :write [1 17] INFO [2022-01-10 05:43:39,284] jepsen worker 2 - jepsen.util 2792 :info :write [0 29] :RequestTimeoutException INFO [2022-01-10 05:43:39,285] jepsen worker 2 - jepsen.util 2882 :invoke :write [0 2] INFO [2022-01-10 05:43:39,286] jepsen worker 76 - jepsen.util 2236 :info :write [25 38] :RequestTimeoutException INFO [2022-01-10 05:43:39,286] jepsen worker 76 - jepsen.util 2326 :invoke :read [25 nil] INFO [2022-01-10 05:43:39,452] jepsen worker 43 - jepsen.util 2563 :info :write [14 33] :RequestTimeoutException INFO [2022-01-10 05:43:39,453] jepsen worker 43 - jepsen.util 2653 :invoke :read [14 nil] INFO [2022-01-10 05:43:39,575] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2932551319648093}, :ns_1@172.23.105.187 {:progress 0.2932551319648093}, :ns_1@172.23.107.228 {:progress 0.4117647058823529}} INFO [2022-01-10 05:43:39,575] jepsen worker nemesis - couchbase.util Rebalance check count: 371 INFO [2022-01-10 05:43:39,575] jepsen worker nemesis - couchbase.util Rebalance stuck count: 296 INFO [2022-01-10 05:43:39,671] jepsen worker 70 - jepsen.util 2860 :info :write [23 1] :RequestTimeoutException INFO [2022-01-10 05:43:39,671] jepsen worker 70 - jepsen.util 2950 :invoke :write [23 9] INFO [2022-01-10 05:43:39,721] jepsen worker 49 - jepsen.util 2659 :info :write [16 25] :RequestTimeoutException INFO [2022-01-10 05:43:39,722] jepsen worker 49 - jepsen.util 2749 :invoke :write [16 36] INFO [2022-01-10 05:43:39,728] jepsen worker 46 - jepsen.util 2926 :ok :read [15 23] INFO [2022-01-10 05:43:39,729] jepsen worker 46 - jepsen.util 2926 :invoke :read [15 nil] INFO [2022-01-10 05:43:39,741] jepsen worker 66 - jepsen.util 2406 :info :write [22 30] :RequestTimeoutException INFO [2022-01-10 05:43:39,742] jepsen worker 66 - jepsen.util 2496 :invoke :read [22 nil] INFO [2022-01-10 05:43:39,765] jepsen worker 74 - jepsen.util 3314 :info :write [24 15] :RequestTimeoutException INFO [2022-01-10 05:43:39,766] jepsen worker 74 - jepsen.util 3404 :invoke :read [24 nil] INFO [2022-01-10 05:43:39,784] jepsen worker 4 - jepsen.util 3064 :info :write [1 0] :RequestTimeoutException INFO [2022-01-10 05:43:39,785] jepsen worker 4 - jepsen.util 3154 :invoke :write [1 6] INFO [2022-01-10 05:43:39,890] jepsen worker 14 - jepsen.util 3254 :info :write [4 31] :RequestTimeoutException INFO [2022-01-10 05:43:39,890] jepsen worker 14 - jepsen.util 3344 :invoke :read [4 nil] INFO [2022-01-10 05:43:39,891] jepsen worker 65 - jepsen.util 2495 :info :write [21 41] :RequestTimeoutException INFO [2022-01-10 05:43:39,892] jepsen worker 65 - jepsen.util 2585 :invoke :read [21 nil] INFO [2022-01-10 05:43:39,902] jepsen worker 62 - jepsen.util 2942 :info :write [20 10] :RequestTimeoutException INFO [2022-01-10 05:43:39,902] jepsen worker 62 - jepsen.util 3032 :invoke :write [20 32] INFO [2022-01-10 05:43:39,985] jepsen worker 85 - jepsen.util 3055 :ok :read [28 45] INFO [2022-01-10 05:43:39,985] jepsen worker 85 - jepsen.util 3055 :invoke :read [28 nil] WARN [2022-01-10 05:43:40,096] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10516710,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14453288,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":10568648,"encode_duration_us":18731,"operation_id":"0x5431","timeout_ms":10000,"last_server_duration_us":9930938,"total_duration_us":10005197},{"operation_name":"upsert","last_dispatch_duration_us":10461808,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14610649,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9715415,"encode_duration_us":16559,"operation_id":"0x53ec","timeout_ms":10000,"last_server_duration_us":9122444,"total_duration_us":10008795},{"operation_name":"upsert","last_dispatch_duration_us":9662293,"last_local_id":"EFE9906200000001/000000007CF9F49A","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10821338,"last_local_socket":"172.23.104.79:44306","total_server_duration_us":10164118,"encode_duration_us":16397,"operation_id":"0x5512","timeout_ms":10000,"last_server_duration_us":9645465,"total_duration_us":10007330}],"total_count":3}} INFO [2022-01-10 05:43:40,136] jepsen worker 41 - jepsen.util 2561 :info :write [13 43] :RequestTimeoutException INFO [2022-01-10 05:43:40,136] jepsen worker 41 - jepsen.util 2651 :invoke :read [13 nil] INFO [2022-01-10 05:43:40,151] jepsen worker 21 - jepsen.util 2901 :info :write [7 40] :RequestTimeoutException INFO [2022-01-10 05:43:40,152] jepsen worker 21 - jepsen.util 2991 :invoke :read [7 nil] WARN [2022-01-10 05:43:40,197] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":8824703,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8824703,"total_server_duration_us":32,"operation_id":"0x5522","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":10418295},{"operation_name":"upsert","last_dispatch_duration_us":2569750,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7400087,"total_server_duration_us":2609377,"encode_duration_us":17672,"operation_id":"0x55d5","timeout_ms":10000,"last_server_duration_us":333936,"total_duration_us":10010444},{"operation_name":"upsert","last_dispatch_duration_us":2036689,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8732556,"total_server_duration_us":1305662,"encode_duration_us":16768,"operation_id":"0x568a","timeout_ms":10000,"last_server_duration_us":396248,"total_duration_us":10010140},{"operation_name":"upsert","last_dispatch_duration_us":35889,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":607659,"total_server_duration_us":379930,"encode_duration_us":17900,"operation_id":"0x56e9","timeout_ms":10000,"last_server_duration_us":21163,"total_duration_us":10009130},{"operation_name":"upsert","last_dispatch_duration_us":8280836,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":8969160,"total_server_duration_us":814226,"encode_duration_us":19394,"operation_id":"0x55d1","timeout_ms":10000,"last_server_duration_us":392606,"total_duration_us":10008375},{"operation_name":"upsert","last_dispatch_duration_us":2729309,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10854611,"total_server_duration_us":1711689,"encode_duration_us":16597,"operation_id":"0x56c6","timeout_ms":10000,"last_server_duration_us":196453,"total_duration_us":10006487},{"operation_name":"upsert","last_dispatch_duration_us":3042429,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5636968,"total_server_duration_us":2773277,"encode_duration_us":16551,"operation_id":"0x5605","timeout_ms":10000,"last_server_duration_us":2295293,"total_duration_us":10004978},{"operation_name":"upsert","last_dispatch_duration_us":3030467,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5252779,"total_server_duration_us":723468,"encode_duration_us":20792,"operation_id":"0x569e","timeout_ms":10000,"last_server_duration_us":194997,"total_duration_us":10003445},{"operation_name":"upsert","last_dispatch_duration_us":1705482,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5337282,"total_server_duration_us":960016,"encode_duration_us":17927,"operation_id":"0x5572","timeout_ms":10000,"last_server_duration_us":508330,"total_duration_us":10003433},{"operation_name":"upsert","last_dispatch_duration_us":3114198,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5627206,"total_server_duration_us":2387081,"encode_duration_us":17480,"operation_id":"0x565b","timeout_ms":10000,"last_server_duration_us":1761571,"total_duration_us":10001336}],"total_count":15}} INFO [2022-01-10 05:43:40,262] jepsen worker 19 - jepsen.util 2899 :ok :read [6 25] INFO [2022-01-10 05:43:40,263] jepsen worker 19 - jepsen.util 2899 :invoke :read [6 nil] INFO [2022-01-10 05:43:40,265] jepsen worker 44 - jepsen.util 2744 :ok :read [14 14] INFO [2022-01-10 05:43:40,265] jepsen worker 44 - jepsen.util 2744 :invoke :write [14 41] INFO [2022-01-10 05:43:40,278] jepsen worker 7 - jepsen.util 2797 :ok :read [2 14] INFO [2022-01-10 05:43:40,278] jepsen worker 7 - jepsen.util 2797 :invoke :write [2 38] INFO [2022-01-10 05:43:40,465] jepsen worker 22 - jepsen.util 2452 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:40,465] jepsen worker 45 - jepsen.util 3015 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:40,466] jepsen worker 22 - jepsen.util 2452 :invoke :write [7 47] INFO [2022-01-10 05:43:40,467] jepsen worker 45 - jepsen.util 3015 :invoke :write [15 21] INFO [2022-01-10 05:43:40,509] jepsen worker 14 - jepsen.util 3344 :ok :read [4 3] INFO [2022-01-10 05:43:40,510] jepsen worker 14 - jepsen.util 3344 :invoke :read [4 nil] INFO [2022-01-10 05:43:40,525] jepsen worker 50 - jepsen.util 2930 :ok :read [16 38] INFO [2022-01-10 05:43:40,525] jepsen worker 50 - jepsen.util 2930 :invoke :write [16 13] INFO [2022-01-10 05:43:40,581] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2932551319648093}, :ns_1@172.23.105.187 {:progress 0.2932551319648093}, :ns_1@172.23.107.228 {:progress 0.4117647058823529}} INFO [2022-01-10 05:43:40,581] jepsen worker nemesis - couchbase.util Rebalance check count: 372 INFO [2022-01-10 05:43:40,581] jepsen worker nemesis - couchbase.util Rebalance stuck count: 297 INFO [2022-01-10 05:43:40,680] jepsen worker 9 - jepsen.util 3249 :info :write [3 30] :RequestTimeoutException INFO [2022-01-10 05:43:40,681] jepsen worker 9 - jepsen.util 3339 :invoke :write [3 49] INFO [2022-01-10 05:43:40,696] jepsen worker 59 - jepsen.util 2759 :ok :read [19 31] INFO [2022-01-10 05:43:40,697] jepsen worker 59 - jepsen.util 2759 :invoke :write [19 14] INFO [2022-01-10 05:43:40,902] jepsen worker 46 - jepsen.util 2926 :ok :read [15 23] INFO [2022-01-10 05:43:40,902] jepsen worker 46 - jepsen.util 2926 :invoke :read [15 nil] INFO [2022-01-10 05:43:41,110] jepsen worker 13 - jepsen.util 3073 :ok :read [4 3] INFO [2022-01-10 05:43:41,111] jepsen worker 13 - jepsen.util 3073 :invoke :write [4 44] INFO [2022-01-10 05:43:41,139] jepsen worker 58 - jepsen.util 2758 :ok :read [19 31] INFO [2022-01-10 05:43:41,139] jepsen worker 58 - jepsen.util 2758 :invoke :read [19 nil] INFO [2022-01-10 05:43:41,407] jepsen worker 29 - jepsen.util 2639 :info :write [9 28] :RequestTimeoutException INFO [2022-01-10 05:43:41,415] jepsen worker 83 - jepsen.util 3143 :info :write [27 19] :RequestTimeoutException INFO [2022-01-10 05:43:41,419] jepsen worker 29 - jepsen.util 2729 :invoke :read [9 nil] INFO [2022-01-10 05:43:41,416] jepsen worker 1 - jepsen.util 2791 :info :write [0 25] :RequestTimeoutException INFO [2022-01-10 05:43:41,419] jepsen worker 83 - jepsen.util 3233 :invoke :write [27 41] INFO [2022-01-10 05:43:41,416] jepsen worker 72 - jepsen.util 2952 :info :write [24 21] :RequestTimeoutException INFO [2022-01-10 05:43:41,419] jepsen worker 1 - jepsen.util 2881 :invoke :read [0 nil] INFO [2022-01-10 05:43:41,420] jepsen worker 72 - jepsen.util 3042 :invoke :write [24 47] INFO [2022-01-10 05:43:41,424] jepsen worker 84 - jepsen.util 2604 :info :write [28 34] :RequestTimeoutException INFO [2022-01-10 05:43:41,425] jepsen worker 84 - jepsen.util 2694 :invoke :write [28 42] INFO [2022-01-10 05:43:41,552] jepsen worker 54 - jepsen.util 2844 :info :write [18 49] :RequestTimeoutException INFO [2022-01-10 05:43:41,553] jepsen worker 54 - jepsen.util 2934 :invoke :read [18 nil] INFO [2022-01-10 05:43:41,591] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2932551319648093}, :ns_1@172.23.105.187 {:progress 0.2932551319648093}, :ns_1@172.23.107.228 {:progress 0.4117647058823529}} INFO [2022-01-10 05:43:41,591] jepsen worker nemesis - couchbase.util Rebalance check count: 373 INFO [2022-01-10 05:43:41,591] jepsen worker nemesis - couchbase.util Rebalance stuck count: 298 INFO [2022-01-10 05:43:41,596] jepsen worker 63 - jepsen.util 2583 :info :write [21 2] :RequestTimeoutException INFO [2022-01-10 05:43:41,597] jepsen worker 63 - jepsen.util 2673 :invoke :read [21 nil] INFO [2022-01-10 05:43:41,602] jepsen worker 14 - jepsen.util 3344 :ok :read [4 3] INFO [2022-01-10 05:43:41,603] jepsen worker 14 - jepsen.util 3344 :invoke :read [4 nil] INFO [2022-01-10 05:43:41,619] jepsen worker 46 - jepsen.util 2926 :ok :read [15 23] INFO [2022-01-10 05:43:41,619] jepsen worker 46 - jepsen.util 2926 :invoke :write [15 17] INFO [2022-01-10 05:43:41,675] jepsen worker 81 - jepsen.util 2961 :info :write [27 30] :RequestTimeoutException INFO [2022-01-10 05:43:41,675] jepsen worker 81 - jepsen.util 3051 :invoke :write [27 11] WARN [2022-01-10 05:43:41,703] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17830422,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":17830422,"total_server_duration_us":15,"operation_id":"0x541a","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":23015084},{"operation_name":"get","last_dispatch_duration_us":18167875,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":18167875,"total_server_duration_us":43,"operation_id":"0x5521","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":18273402},{"operation_name":"upsert","last_dispatch_duration_us":3840430,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":5763434,"total_server_duration_us":2268576,"encode_duration_us":19243,"operation_id":"0x56b5","timeout_ms":10000,"last_server_duration_us":971454,"total_duration_us":10010007},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":17483,"operation_id":"0x56cf","timeout_ms":10000,"total_duration_us":10009679},{"operation_name":"upsert","last_dispatch_duration_us":5368093,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10841322,"total_server_duration_us":122372,"encode_duration_us":27804,"operation_id":"0x567f","timeout_ms":10000,"last_server_duration_us":102169,"total_duration_us":10007516},{"operation_name":"upsert","last_dispatch_duration_us":4361940,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":4361940,"total_server_duration_us":386475,"encode_duration_us":22351,"operation_id":"0x5732","timeout_ms":10000,"last_server_duration_us":386475,"total_duration_us":10005891},{"operation_name":"upsert","last_dispatch_duration_us":2667969,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":6763133,"total_server_duration_us":296094,"encode_duration_us":16618,"operation_id":"0x56f6","timeout_ms":10000,"last_server_duration_us":71773,"total_duration_us":10005874},{"operation_name":"upsert","last_dispatch_duration_us":6630061,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9600971,"total_server_duration_us":1131520,"encode_duration_us":18622,"operation_id":"0x56e1","timeout_ms":10000,"last_server_duration_us":582254,"total_duration_us":10004772},{"operation_name":"upsert","last_dispatch_duration_us":5133192,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":12246405,"total_server_duration_us":1635072,"encode_duration_us":16480,"operation_id":"0x55a2","timeout_ms":10000,"last_server_duration_us":209976,"total_duration_us":10003344},{"operation_name":"upsert","last_dispatch_duration_us":5372585,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7279297,"total_server_duration_us":1158762,"encode_duration_us":17380,"operation_id":"0x56df","timeout_ms":10000,"last_server_duration_us":789890,"total_duration_us":10001996}],"total_count":26}} WARN [2022-01-10 05:43:41,703] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":25201513,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":25201513,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9030999,"encode_duration_us":17148,"operation_id":"0x53a0","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10006190}],"total_count":1}} WARN [2022-01-10 05:43:41,780] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9043929,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10051848,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9456779,"encode_duration_us":17068,"operation_id":"0x5525","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10007996}],"total_count":1}} WARN [2022-01-10 05:43:41,792] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12808538,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":12808538,"total_server_duration_us":18799,"encode_duration_us":17429,"operation_id":"0x55d0","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10011511},{"operation_name":"upsert","last_dispatch_duration_us":8051171,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8051171,"total_server_duration_us":81398,"encode_duration_us":16266,"operation_id":"0x56ca","timeout_ms":10000,"last_server_duration_us":81398,"total_duration_us":10010164},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":19484,"operation_id":"0x567d","timeout_ms":10000,"total_duration_us":10008955},{"operation_name":"upsert","last_dispatch_duration_us":7659364,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7659364,"total_server_duration_us":24411,"encode_duration_us":17475,"operation_id":"0x56f8","timeout_ms":10000,"last_server_duration_us":24411,"total_duration_us":10006775},{"operation_name":"upsert","last_dispatch_duration_us":58557,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8450398,"total_server_duration_us":178687,"encode_duration_us":20620,"operation_id":"0x56b3","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10004684},{"operation_name":"upsert","last_dispatch_duration_us":8436322,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":14623477,"total_server_duration_us":912700,"encode_duration_us":20192,"operation_id":"0x55e4","timeout_ms":10000,"last_server_duration_us":642714,"total_duration_us":10004511},{"operation_name":"upsert","last_dispatch_duration_us":3297135,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":12402358,"total_server_duration_us":2667777,"encode_duration_us":17088,"operation_id":"0x562d","timeout_ms":10000,"last_server_duration_us":1666848,"total_duration_us":10002987},{"operation_name":"upsert","last_dispatch_duration_us":13479479,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":13479479,"total_server_duration_us":19030,"encode_duration_us":21865,"operation_id":"0x55a0","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10002896},{"operation_name":"upsert","last_dispatch_duration_us":34701,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":34701,"total_server_duration_us":21001,"encode_duration_us":16953,"operation_id":"0x5734","timeout_ms":10000,"last_server_duration_us":21001,"total_duration_us":10002534},{"operation_name":"get","last_dispatch_duration_us":6535952,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6535952,"total_server_duration_us":93346,"operation_id":"0x568e","timeout_ms":30000,"last_server_duration_us":93346,"total_duration_us":9158319}],"total_count":14}} WARN [2022-01-10 05:43:41,796] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12632671,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12632671,"total_server_duration_us":32,"operation_id":"0x53c5","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":20873011},{"operation_name":"upsert","last_dispatch_duration_us":30249,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":3747335,"total_server_duration_us":79640,"encode_duration_us":20405,"operation_id":"0x56b9","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10010289},{"operation_name":"upsert","last_dispatch_duration_us":4841921,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":4841921,"total_server_duration_us":202533,"encode_duration_us":16296,"operation_id":"0x56fd","timeout_ms":10000,"last_server_duration_us":202533,"total_duration_us":10008954},{"operation_name":"upsert","last_dispatch_duration_us":11311723,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":11311723,"total_server_duration_us":19030,"encode_duration_us":16267,"operation_id":"0x55a1","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10008118},{"operation_name":"upsert","last_dispatch_duration_us":5232730,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8120154,"total_server_duration_us":221996,"encode_duration_us":16620,"operation_id":"0x5680","timeout_ms":10000,"last_server_duration_us":200428,"total_duration_us":10006113},{"operation_name":"upsert","last_dispatch_duration_us":2526280,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10899694,"total_server_duration_us":3562556,"encode_duration_us":16913,"operation_id":"0x562e","timeout_ms":10000,"last_server_duration_us":105664,"total_duration_us":10005499},{"operation_name":"upsert","last_dispatch_duration_us":36946,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":8972663,"total_server_duration_us":235440,"encode_duration_us":18361,"operation_id":"0x562f","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10004430},{"operation_name":"upsert","last_dispatch_duration_us":3152664,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8286552,"total_server_duration_us":3231625,"encode_duration_us":16677,"operation_id":"0x568f","timeout_ms":10000,"last_server_duration_us":2277546,"total_duration_us":10002907},{"operation_name":"upsert","last_dispatch_duration_us":3570673,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8131364,"total_server_duration_us":530508,"encode_duration_us":17683,"operation_id":"0x55ec","timeout_ms":10000,"last_server_duration_us":384810,"total_duration_us":10002335},{"operation_name":"upsert","last_dispatch_duration_us":2563096,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":2563096,"total_server_duration_us":332372,"encode_duration_us":16247,"operation_id":"0x5735","timeout_ms":10000,"last_server_duration_us":332372,"total_duration_us":10002323}],"total_count":21}} WARN [2022-01-10 05:43:41,880] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":6628055,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11872110,"total_server_duration_us":1476495,"encode_duration_us":16780,"operation_id":"0x55d6","timeout_ms":10000,"last_server_duration_us":1178993,"total_duration_us":10007982},{"operation_name":"upsert","last_dispatch_duration_us":8131649,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8842399,"total_server_duration_us":382494,"encode_duration_us":18688,"operation_id":"0x55c4","timeout_ms":10000,"last_server_duration_us":21649,"total_duration_us":10007934},{"operation_name":"upsert","last_dispatch_duration_us":2535542,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":4642578,"total_server_duration_us":1450349,"encode_duration_us":17370,"operation_id":"0x56ab","timeout_ms":10000,"last_server_duration_us":384255,"total_duration_us":10007601},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":16912,"operation_id":"0x56c7","timeout_ms":10000,"total_duration_us":10006863},{"operation_name":"upsert","last_dispatch_duration_us":5676919,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5676919,"total_server_duration_us":212555,"encode_duration_us":16733,"operation_id":"0x567b","timeout_ms":10000,"last_server_duration_us":212555,"total_duration_us":10005809},{"operation_name":"upsert","last_dispatch_duration_us":5911374,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11565135,"total_server_duration_us":1554808,"encode_duration_us":17566,"operation_id":"0x5606","timeout_ms":10000,"last_server_duration_us":101067,"total_duration_us":10004205},{"operation_name":"upsert","last_dispatch_duration_us":4947031,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11067433,"total_server_duration_us":315535,"encode_duration_us":16459,"operation_id":"0x5585","timeout_ms":10000,"last_server_duration_us":173123,"total_duration_us":10003041},{"operation_name":"upsert","last_dispatch_duration_us":1752435,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":3914264,"total_server_duration_us":1807311,"encode_duration_us":20453,"operation_id":"0x565c","timeout_ms":10000,"last_server_duration_us":999994,"total_duration_us":10002624},{"operation_name":"upsert","last_dispatch_duration_us":1112857,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1112857,"total_server_duration_us":59249,"encode_duration_us":17775,"operation_id":"0x5731","timeout_ms":10000,"last_server_duration_us":59249,"total_duration_us":10002182},{"operation_name":"upsert","last_dispatch_duration_us":3153615,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7615200,"total_server_duration_us":319987,"encode_duration_us":21367,"operation_id":"0x56e8","timeout_ms":10000,"last_server_duration_us":120818,"total_duration_us":10001845}],"total_count":19}} WARN [2022-01-10 05:43:41,891] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18016667,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":24773908,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9057467,"encode_duration_us":17696,"operation_id":"0x539d","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10009338}],"total_count":1}} WARN [2022-01-10 05:43:41,892] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17145942,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17238421,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9084343,"encode_duration_us":17055,"operation_id":"0x54f0","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10008467}],"total_count":1}} WARN [2022-01-10 05:43:41,996] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17433865,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":18367847,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":9584938,"encode_duration_us":16721,"operation_id":"0x5273","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10007437}],"total_count":1}} INFO [2022-01-10 05:43:42,055] jepsen worker 26 - jepsen.util 2276 :info :write [8 23] :RequestTimeoutException INFO [2022-01-10 05:43:42,056] jepsen worker 26 - jepsen.util 2366 :invoke :write [8 38] INFO [2022-01-10 05:43:42,059] jepsen worker 64 - jepsen.util 2674 :info :write [21 17] :RequestTimeoutException INFO [2022-01-10 05:43:42,060] jepsen worker 64 - jepsen.util 2764 :invoke :read [21 nil] WARN [2022-01-10 05:43:42,092] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14788637,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":14788637,"total_server_duration_us":27,"operation_id":"0x559f","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":14789719},{"operation_name":"upsert","last_dispatch_duration_us":3115258,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7128195,"total_server_duration_us":2792412,"encode_duration_us":18071,"operation_id":"0x55c6","timeout_ms":10000,"last_server_duration_us":1530207,"total_duration_us":10010297},{"operation_name":"upsert","last_dispatch_duration_us":9034628,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9034628,"total_server_duration_us":19030,"encode_duration_us":16652,"operation_id":"0x56b7","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10010137},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":16565,"operation_id":"0x5704","timeout_ms":10000,"total_duration_us":10006726},{"operation_name":"upsert","last_dispatch_duration_us":3156712,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9691255,"total_server_duration_us":820420,"encode_duration_us":16886,"operation_id":"0x567c","timeout_ms":10000,"last_server_duration_us":217751,"total_duration_us":10006417},{"operation_name":"upsert","last_dispatch_duration_us":2807184,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10553413,"total_server_duration_us":3246723,"encode_duration_us":17050,"operation_id":"0x5630","timeout_ms":10000,"last_server_duration_us":2279318,"total_duration_us":10005621},{"operation_name":"upsert","last_dispatch_duration_us":3548699,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":12774502,"total_server_duration_us":770012,"encode_duration_us":16724,"operation_id":"0x5633","timeout_ms":10000,"last_server_duration_us":540907,"total_duration_us":10005386},{"operation_name":"upsert","last_dispatch_duration_us":2442938,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":4036980,"total_server_duration_us":1124077,"encode_duration_us":18217,"operation_id":"0x56f7","timeout_ms":10000,"last_server_duration_us":504280,"total_duration_us":10004127},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":22087,"operation_id":"0x56ba","timeout_ms":10000,"total_duration_us":10002537},{"operation_name":"upsert","last_dispatch_duration_us":313989,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":2722798,"total_server_duration_us":488252,"encode_duration_us":16470,"operation_id":"0x562c","timeout_ms":10000,"last_server_duration_us":26774,"total_duration_us":10001910}],"total_count":19}} INFO [2022-01-10 05:43:42,189] jepsen worker 40 - jepsen.util 2200 :ok :read [13 15] INFO [2022-01-10 05:43:42,189] jepsen worker 40 - jepsen.util 2200 :invoke :read [13 nil] INFO [2022-01-10 05:43:42,204] jepsen worker 17 - jepsen.util 2717 :ok :read [5 42] INFO [2022-01-10 05:43:42,205] jepsen worker 17 - jepsen.util 2717 :invoke :write [5 38] INFO [2022-01-10 05:43:42,373] jepsen worker 14 - jepsen.util 3344 :ok :read [4 3] INFO [2022-01-10 05:43:42,374] jepsen worker 14 - jepsen.util 3344 :invoke :read [4 nil] INFO [2022-01-10 05:43:42,599] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2961876832844574}, :ns_1@172.23.105.187 {:progress 0.2961876832844574}, :ns_1@172.23.107.228 {:progress 0.4117647058823529}} INFO [2022-01-10 05:43:42,599] jepsen worker nemesis - couchbase.util Rebalance check count: 374 INFO [2022-01-10 05:43:42,599] jepsen worker nemesis - couchbase.util Rebalance stuck count: 298 INFO [2022-01-10 05:43:42,933] jepsen worker 14 - jepsen.util 3344 :ok :read [4 3] INFO [2022-01-10 05:43:42,935] jepsen worker 14 - jepsen.util 3344 :invoke :write [4 15] INFO [2022-01-10 05:43:43,050] jepsen worker 41 - jepsen.util 2651 :ok :read [13 15] INFO [2022-01-10 05:43:43,051] jepsen worker 41 - jepsen.util 2651 :invoke :write [13 35] INFO [2022-01-10 05:43:43,067] jepsen worker 66 - jepsen.util 2496 :ok :read [22 41] INFO [2022-01-10 05:43:43,068] jepsen worker 66 - jepsen.util 2496 :invoke :read [22 nil] INFO [2022-01-10 05:43:43,278] jepsen worker 89 - jepsen.util 3239 :info :write [29 15] :SyncWriteAmbiguous INFO [2022-01-10 05:43:43,279] jepsen worker 89 - jepsen.util 3329 :invoke :write [29 40] INFO [2022-01-10 05:43:43,423] jepsen worker 28 - jepsen.util 2548 :ok :read [9 24] INFO [2022-01-10 05:43:43,424] jepsen worker 28 - jepsen.util 2548 :invoke :read [9 nil] INFO [2022-01-10 05:43:43,476] jepsen worker 88 - jepsen.util 2788 :info :write [29 2] :RequestTimeoutException INFO [2022-01-10 05:43:43,477] jepsen worker 88 - jepsen.util 2878 :invoke :write [29 4] INFO [2022-01-10 05:43:43,605] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2961876832844574}, :ns_1@172.23.105.187 {:progress 0.2961876832844574}, :ns_1@172.23.107.228 {:progress 0.4117647058823529}} INFO [2022-01-10 05:43:43,605] jepsen worker nemesis - couchbase.util Rebalance check count: 375 INFO [2022-01-10 05:43:43,605] jepsen worker nemesis - couchbase.util Rebalance stuck count: 299 INFO [2022-01-10 05:43:43,822] jepsen worker 82 - jepsen.util 2782 :info :write [27 2] :RequestTimeoutException INFO [2022-01-10 05:43:43,822] jepsen worker 82 - jepsen.util 2872 :invoke :read [27 nil] INFO [2022-01-10 05:43:43,964] jepsen worker 56 - jepsen.util 3026 :info :write [18 11] :SyncWriteAmbiguous INFO [2022-01-10 05:43:43,965] jepsen worker 56 - jepsen.util 3116 :invoke :write [18 2] INFO [2022-01-10 05:43:43,990] jepsen worker 68 - jepsen.util 2678 :info :write [22 25] :RequestTimeoutException INFO [2022-01-10 05:43:43,990] jepsen worker 68 - jepsen.util 2768 :invoke :write [22 16] INFO [2022-01-10 05:43:44,036] jepsen worker 58 - jepsen.util 2758 :ok :read [19 31] INFO [2022-01-10 05:43:44,037] jepsen worker 58 - jepsen.util 2758 :invoke :write [19 27] INFO [2022-01-10 05:43:44,061] jepsen worker 25 - jepsen.util 2635 :info :write [8 0] :RequestTimeoutException INFO [2022-01-10 05:43:44,062] jepsen worker 25 - jepsen.util 2725 :invoke :read [8 nil] INFO [2022-01-10 05:43:44,085] jepsen worker 35 - jepsen.util 2285 :info :write [11 26] :RequestTimeoutException INFO [2022-01-10 05:43:44,085] jepsen worker 35 - jepsen.util 2375 :invoke :write [11 23] INFO [2022-01-10 05:43:44,189] jepsen worker 42 - jepsen.util 2652 :info :write [14 29] :RequestTimeoutException INFO [2022-01-10 05:43:44,190] jepsen worker 42 - jepsen.util 2742 :invoke :read [14 nil] INFO [2022-01-10 05:43:44,196] jepsen worker 51 - jepsen.util 3201 :info :write [17 34] :RequestTimeoutException INFO [2022-01-10 05:43:44,197] jepsen worker 51 - jepsen.util 3291 :invoke :read [17 nil] INFO [2022-01-10 05:43:44,305] jepsen worker 65 - jepsen.util 2585 :ok :read [21 7] INFO [2022-01-10 05:43:44,306] jepsen worker 65 - jepsen.util 2585 :invoke :write [21 46] INFO [2022-01-10 05:43:44,400] jepsen worker 8 - jepsen.util 3158 :info :write [2 43] :RequestTimeoutException INFO [2022-01-10 05:43:44,400] jepsen worker 8 - jepsen.util 3248 :invoke :write [2 33] INFO [2022-01-10 05:43:44,452] jepsen worker 20 - jepsen.util 2540 :info :write [6 42] :RequestTimeoutException INFO [2022-01-10 05:43:44,452] jepsen worker 20 - jepsen.util 2630 :invoke :write [6 4] INFO [2022-01-10 05:43:44,575] jepsen worker 67 - jepsen.util 2587 :ok :read [22 41] INFO [2022-01-10 05:43:44,575] jepsen worker 67 - jepsen.util 2587 :invoke :write [22 37] INFO [2022-01-10 05:43:44,624] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4210526315789473}, :ns_1@172.23.107.227 {:progress 0.2961876832844574}, :ns_1@172.23.105.187 {:progress 0.2961876832844574}, :ns_1@172.23.107.228 {:progress 0.4117647058823529}} INFO [2022-01-10 05:43:44,624] jepsen worker nemesis - couchbase.util Rebalance check count: 376 INFO [2022-01-10 05:43:44,624] jepsen worker nemesis - couchbase.util Rebalance stuck count: 300 INFO [2022-01-10 05:43:44,677] jepsen worker 29 - jepsen.util 2729 :ok :read [9 24] INFO [2022-01-10 05:43:44,678] jepsen worker 29 - jepsen.util 2729 :invoke :read [9 nil] INFO [2022-01-10 05:43:44,685] jepsen worker 48 - jepsen.util 2388 :ok :read [16 38] INFO [2022-01-10 05:43:44,686] jepsen worker 48 - jepsen.util 2388 :invoke :read [16 nil] INFO [2022-01-10 05:43:44,766] jepsen worker 61 - jepsen.util 2761 :info :write [20 12] :RequestTimeoutException INFO [2022-01-10 05:43:44,767] jepsen worker 61 - jepsen.util 2851 :invoke :write [20 19] INFO [2022-01-10 05:43:44,827] jepsen worker 69 - jepsen.util 2679 :info :write [23 4] :RequestTimeoutException INFO [2022-01-10 05:43:44,828] jepsen worker 69 - jepsen.util 2769 :invoke :read [23 nil] INFO [2022-01-10 05:43:45,026] jepsen worker 12 - jepsen.util 2892 :info :write [4 8] :RequestTimeoutException INFO [2022-01-10 05:43:45,027] jepsen worker 12 - jepsen.util 2982 :invoke :write [4 4] INFO [2022-01-10 05:43:45,495] jepsen worker 51 - jepsen.util 3291 :ok :read [17 41] INFO [2022-01-10 05:43:45,496] jepsen worker 51 - jepsen.util 3291 :invoke :read [17 nil] INFO [2022-01-10 05:43:45,555] jepsen worker 40 - jepsen.util 2200 :ok :read [13 15] INFO [2022-01-10 05:43:45,555] jepsen worker 40 - jepsen.util 2200 :invoke :read [13 nil] INFO [2022-01-10 05:43:45,631] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4327485380116959}, :ns_1@172.23.107.227 {:progress 0.2991202346041055}, :ns_1@172.23.105.187 {:progress 0.2991202346041055}, :ns_1@172.23.107.228 {:progress 0.4176470588235294}} INFO [2022-01-10 05:43:45,631] jepsen worker nemesis - couchbase.util Rebalance check count: 377 INFO [2022-01-10 05:43:45,631] jepsen worker nemesis - couchbase.util Rebalance stuck count: 300 INFO [2022-01-10 05:43:45,714] jepsen worker 75 - jepsen.util 2145 :info :write [25 12] :RequestTimeoutException INFO [2022-01-10 05:43:45,715] jepsen worker 75 - jepsen.util 2235 :invoke :write [25 39] INFO [2022-01-10 05:43:45,716] jepsen worker 77 - jepsen.util 3137 :info :write [25 35] :RequestTimeoutException INFO [2022-01-10 05:43:45,716] jepsen worker 77 - jepsen.util 3227 :invoke :write [25 16] INFO [2022-01-10 05:43:45,830] jepsen worker 28 - jepsen.util 2548 :ok :read [9 24] INFO [2022-01-10 05:43:45,831] jepsen worker 28 - jepsen.util 2548 :invoke :read [9 nil] INFO [2022-01-10 05:43:46,048] jepsen worker 51 - jepsen.util 3291 :ok :read [17 41] INFO [2022-01-10 05:43:46,048] jepsen worker 51 - jepsen.util 3291 :invoke :read [17 nil] INFO [2022-01-10 05:43:46,189] jepsen worker 51 - jepsen.util 3291 :ok :read [17 41] INFO [2022-01-10 05:43:46,190] jepsen worker 51 - jepsen.util 3291 :invoke :write [17 15] INFO [2022-01-10 05:43:46,365] jepsen worker 82 - jepsen.util 2872 :ok :read [27 30] INFO [2022-01-10 05:43:46,365] jepsen worker 82 - jepsen.util 2872 :invoke :read [27 nil] INFO [2022-01-10 05:43:46,379] jepsen worker 43 - jepsen.util 2653 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:46,380] jepsen worker 63 - jepsen.util 2673 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:46,380] jepsen worker 43 - jepsen.util 2653 :invoke :read [14 nil] INFO [2022-01-10 05:43:46,381] jepsen worker 63 - jepsen.util 2673 :invoke :write [21 11] INFO [2022-01-10 05:43:46,466] jepsen worker 29 - jepsen.util 2729 :ok :read [9 24] INFO [2022-01-10 05:43:46,467] jepsen worker 29 - jepsen.util 2729 :invoke :read [9 nil] INFO [2022-01-10 05:43:46,493] jepsen worker 48 - jepsen.util 2388 :ok :read [16 38] INFO [2022-01-10 05:43:46,494] jepsen worker 48 - jepsen.util 2388 :invoke :write [16 6] INFO [2022-01-10 05:43:46,581] jepsen worker 6 - jepsen.util 2886 :info :write [2 7] :RequestTimeoutException INFO [2022-01-10 05:43:46,581] jepsen worker 34 - jepsen.util 2644 :info :write [11 0] :RequestTimeoutException INFO [2022-01-10 05:43:46,582] jepsen worker 6 - jepsen.util 2976 :invoke :write [2 9] INFO [2022-01-10 05:43:46,582] jepsen worker 34 - jepsen.util 2734 :invoke :write [11 29] INFO [2022-01-10 05:43:46,585] jepsen worker 57 - jepsen.util 3297 :info :write [19 7] :RequestTimeoutException INFO [2022-01-10 05:43:46,585] jepsen worker 57 - jepsen.util 3387 :invoke :write [19 5] INFO [2022-01-10 05:43:46,636] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4327485380116959}, :ns_1@172.23.107.227 {:progress 0.2991202346041055}, :ns_1@172.23.105.187 {:progress 0.2991202346041055}, :ns_1@172.23.107.228 {:progress 0.4176470588235294}} INFO [2022-01-10 05:43:46,637] jepsen worker nemesis - couchbase.util Rebalance check count: 378 INFO [2022-01-10 05:43:46,637] jepsen worker nemesis - couchbase.util Rebalance stuck count: 301 INFO [2022-01-10 05:43:46,905] jepsen worker 60 - jepsen.util 3210 :ok :read [20 :nil] INFO [2022-01-10 05:43:46,905] jepsen worker 60 - jepsen.util 3210 :invoke :read [20 nil] INFO [2022-01-10 05:43:46,960] jepsen worker 43 - jepsen.util 2653 :ok :read [14 14] INFO [2022-01-10 05:43:46,961] jepsen worker 43 - jepsen.util 2653 :invoke :write [14 21] INFO [2022-01-10 05:43:47,071] jepsen worker 40 - jepsen.util 2200 :ok :read [13 15] INFO [2022-01-10 05:43:47,072] jepsen worker 40 - jepsen.util 2200 :invoke :read [13 nil] INFO [2022-01-10 05:43:47,088] jepsen worker 28 - jepsen.util 2548 :ok :read [9 24] INFO [2022-01-10 05:43:47,089] jepsen worker 28 - jepsen.util 2548 :invoke :write [9 2] INFO [2022-01-10 05:43:47,130] jepsen worker 33 - jepsen.util 3093 :info :write [11 46] :RequestTimeoutException INFO [2022-01-10 05:43:47,130] jepsen worker 33 - jepsen.util 3183 :invoke :read [11 nil] INFO [2022-01-10 05:43:47,166] jepsen worker 21 - jepsen.util 2991 :ok :read [7 :nil] INFO [2022-01-10 05:43:47,167] jepsen worker 21 - jepsen.util 2991 :invoke :write [7 0] INFO [2022-01-10 05:43:47,185] jepsen worker 74 - jepsen.util 3404 :ok :read [24 18] INFO [2022-01-10 05:43:47,185] jepsen worker 74 - jepsen.util 3404 :invoke :read [24 nil] INFO [2022-01-10 05:43:47,235] jepsen worker 82 - jepsen.util 2872 :ok :read [27 30] INFO [2022-01-10 05:43:47,235] jepsen worker 82 - jepsen.util 2872 :invoke :write [27 10] INFO [2022-01-10 05:43:47,260] jepsen worker 73 - jepsen.util 2503 :ok :read [24 18] INFO [2022-01-10 05:43:47,261] jepsen worker 73 - jepsen.util 2503 :invoke :read [24 nil] INFO [2022-01-10 05:43:47,413] jepsen worker 29 - jepsen.util 2729 :ok :read [9 24] INFO [2022-01-10 05:43:47,414] jepsen worker 29 - jepsen.util 2729 :invoke :read [9 nil] INFO [2022-01-10 05:43:47,460] jepsen worker 60 - jepsen.util 3210 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:47,461] jepsen worker 60 - jepsen.util 3210 :invoke :read [20 nil] INFO [2022-01-10 05:43:47,476] jepsen worker 79 - jepsen.util 3139 :info :write [26 13] :RequestTimeoutException INFO [2022-01-10 05:43:47,477] jepsen worker 79 - jepsen.util 3229 :invoke :read [26 nil] INFO [2022-01-10 05:43:47,483] jepsen worker 74 - jepsen.util 3404 :ok :read [24 18] INFO [2022-01-10 05:43:47,484] jepsen worker 74 - jepsen.util 3404 :invoke :write [24 26] INFO [2022-01-10 05:43:47,502] jepsen worker 31 - jepsen.util 3181 :info :write [10 27] :RequestTimeoutException INFO [2022-01-10 05:43:47,503] jepsen worker 31 - jepsen.util 3271 :invoke :read [10 nil] INFO [2022-01-10 05:43:47,554] jepsen worker 76 - jepsen.util 2326 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:47,554] jepsen worker 19 - jepsen.util 2899 :fail :read [6 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:47,555] jepsen worker 76 - jepsen.util 2326 :invoke :read [25 nil] INFO [2022-01-10 05:43:47,555] jepsen worker 19 - jepsen.util 2899 :invoke :read [6 nil] INFO [2022-01-10 05:43:47,611] jepsen worker 18 - jepsen.util 2628 :info :write [6 7] :RequestTimeoutException INFO [2022-01-10 05:43:47,612] jepsen worker 18 - jepsen.util 2718 :invoke :write [6 13] INFO [2022-01-10 05:43:47,644] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4385964912280702}, :ns_1@172.23.107.227 {:progress 0.2991202346041055}, :ns_1@172.23.105.187 {:progress 0.2991202346041055}, :ns_1@172.23.107.228 {:progress 0.4176470588235294}} INFO [2022-01-10 05:43:47,644] jepsen worker nemesis - couchbase.util Rebalance check count: 379 INFO [2022-01-10 05:43:47,644] jepsen worker nemesis - couchbase.util Rebalance stuck count: 301 INFO [2022-01-10 05:43:47,748] jepsen worker 60 - jepsen.util 3210 :ok :read [20 :nil] INFO [2022-01-10 05:43:47,749] jepsen worker 60 - jepsen.util 3210 :invoke :read [20 nil] INFO [2022-01-10 05:43:47,781] jepsen worker 40 - jepsen.util 2200 :ok :read [13 15] INFO [2022-01-10 05:43:47,782] jepsen worker 40 - jepsen.util 2200 :invoke :read [13 nil] INFO [2022-01-10 05:43:47,875] jepsen worker 24 - jepsen.util 2364 :info :write [8 24] :RequestTimeoutException INFO [2022-01-10 05:43:47,876] jepsen worker 24 - jepsen.util 2454 :invoke :read [8 nil] INFO [2022-01-10 05:43:48,106] jepsen worker 78 - jepsen.util 3228 :info :write [26 2] :RequestTimeoutException INFO [2022-01-10 05:43:48,106] jepsen worker 78 - jepsen.util 3318 :invoke :read [26 nil] INFO [2022-01-10 05:43:48,121] jepsen worker 60 - jepsen.util 3210 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:48,121] jepsen worker 60 - jepsen.util 3210 :invoke :read [20 nil] INFO [2022-01-10 05:43:48,130] jepsen worker 10 - jepsen.util 2980 :info :write [3 47] :RequestTimeoutException INFO [2022-01-10 05:43:48,130] jepsen worker 10 - jepsen.util 3070 :invoke :write [3 17] INFO [2022-01-10 05:43:48,360] jepsen worker 42 - jepsen.util 2742 :ok :read [14 14] INFO [2022-01-10 05:43:48,360] jepsen worker 42 - jepsen.util 2742 :invoke :write [14 35] INFO [2022-01-10 05:43:48,477] jepsen worker 79 - jepsen.util 3229 :ok :read [26 10] INFO [2022-01-10 05:43:48,478] jepsen worker 79 - jepsen.util 3229 :invoke :read [26 nil] INFO [2022-01-10 05:43:48,559] jepsen worker 78 - jepsen.util 3318 :ok :read [26 10] INFO [2022-01-10 05:43:48,559] jepsen worker 78 - jepsen.util 3318 :invoke :write [26 18] INFO [2022-01-10 05:43:48,649] jepsen worker 60 - jepsen.util 3210 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:48,649] jepsen worker 60 - jepsen.util 3210 :invoke :read [20 nil] INFO [2022-01-10 05:43:48,691] jepsen worker 23 - jepsen.util 2903 :info :write [7 16] :RequestTimeoutException INFO [2022-01-10 05:43:48,692] jepsen worker 23 - jepsen.util 2993 :invoke :write [7 13] INFO [2022-01-10 05:43:48,699] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4385964912280702}, :ns_1@172.23.107.227 {:progress 0.2991202346041055}, :ns_1@172.23.105.187 {:progress 0.2991202346041055}, :ns_1@172.23.107.228 {:progress 0.4176470588235294}} INFO [2022-01-10 05:43:48,700] jepsen worker nemesis - couchbase.util Rebalance check count: 380 INFO [2022-01-10 05:43:48,700] jepsen worker nemesis - couchbase.util Rebalance stuck count: 302 INFO [2022-01-10 05:43:48,739] jepsen worker 79 - jepsen.util 3229 :ok :read [26 10] INFO [2022-01-10 05:43:48,740] jepsen worker 79 - jepsen.util 3229 :invoke :read [26 nil] INFO [2022-01-10 05:43:48,771] jepsen worker 55 - jepsen.util 3025 :info :write [18 29] :RequestTimeoutException INFO [2022-01-10 05:43:48,771] jepsen worker 55 - jepsen.util 3115 :invoke :write [18 45] INFO [2022-01-10 05:43:48,774] jepsen worker 47 - jepsen.util 2747 :info :write [15 47] :RequestTimeoutException INFO [2022-01-10 05:43:48,775] jepsen worker 47 - jepsen.util 2837 :invoke :write [15 27] INFO [2022-01-10 05:43:48,776] jepsen worker 0 - jepsen.util 2700 :info :write [0 5] :RequestTimeoutException INFO [2022-01-10 05:43:48,776] jepsen worker 0 - jepsen.util 2790 :invoke :read [0 nil] INFO [2022-01-10 05:43:48,780] jepsen worker 40 - jepsen.util 2200 :ok :read [13 15] INFO [2022-01-10 05:43:48,781] jepsen worker 40 - jepsen.util 2200 :invoke :write [13 6] INFO [2022-01-10 05:43:48,791] jepsen worker 36 - jepsen.util 2376 :info :write [12 17] :RequestTimeoutException INFO [2022-01-10 05:43:48,794] jepsen worker 29 - jepsen.util 2729 :ok :read [9 24] INFO [2022-01-10 05:43:48,794] jepsen worker 36 - jepsen.util 2466 :invoke :read [12 nil] INFO [2022-01-10 05:43:48,795] jepsen worker 29 - jepsen.util 2729 :invoke :read [9 nil] INFO [2022-01-10 05:43:48,817] jepsen worker 73 - jepsen.util 2503 :ok :read [24 18] INFO [2022-01-10 05:43:48,819] jepsen worker 73 - jepsen.util 2503 :invoke :read [24 nil] INFO [2022-01-10 05:43:48,831] jepsen worker 76 - jepsen.util 2326 :ok :read [25 :nil] INFO [2022-01-10 05:43:48,832] jepsen worker 76 - jepsen.util 2326 :invoke :write [25 7] INFO [2022-01-10 05:43:48,837] jepsen worker 86 - jepsen.util 2156 :info :write [28 5] :RequestTimeoutException INFO [2022-01-10 05:43:48,838] jepsen worker 86 - jepsen.util 2246 :invoke :write [28 23] INFO [2022-01-10 05:43:48,841] jepsen worker 71 - jepsen.util 2231 :info :write [23 48] :RequestTimeoutException INFO [2022-01-10 05:43:48,841] jepsen worker 37 - jepsen.util 2737 :info :write [12 29] :RequestTimeoutException INFO [2022-01-10 05:43:48,842] jepsen worker 71 - jepsen.util 2321 :invoke :read [23 nil] INFO [2022-01-10 05:43:48,843] jepsen worker 37 - jepsen.util 2827 :invoke :read [12 nil] INFO [2022-01-10 05:43:48,846] jepsen worker 33 - jepsen.util 3183 :ok :read [11 21] INFO [2022-01-10 05:43:48,846] jepsen worker 15 - jepsen.util 2355 :info :write [5 25] :RequestTimeoutException INFO [2022-01-10 05:43:48,849] jepsen worker 15 - jepsen.util 2445 :invoke :read [5 nil] INFO [2022-01-10 05:43:48,855] jepsen worker 33 - jepsen.util 3183 :invoke :write [11 30] INFO [2022-01-10 05:43:48,859] jepsen worker 19 - jepsen.util 2899 :ok :read [6 25] INFO [2022-01-10 05:43:48,860] jepsen worker 19 - jepsen.util 2899 :invoke :write [6 9] INFO [2022-01-10 05:43:48,865] jepsen worker 60 - jepsen.util 3210 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:48,866] jepsen worker 60 - jepsen.util 3210 :invoke :write [20 24] INFO [2022-01-10 05:43:48,867] jepsen worker 71 - jepsen.util 2321 :ok :read [23 26] INFO [2022-01-10 05:43:48,868] jepsen worker 71 - jepsen.util 2321 :invoke :write [23 36] INFO [2022-01-10 05:43:48,920] jepsen worker 52 - jepsen.util 3472 :info :write [17 31] :RequestTimeoutException INFO [2022-01-10 05:43:48,921] jepsen worker 53 - jepsen.util 3293 :info :write [17 17] :RequestTimeoutException INFO [2022-01-10 05:43:48,921] jepsen worker 52 - jepsen.util 3562 :invoke :read [17 nil] INFO [2022-01-10 05:43:48,923] jepsen worker 53 - jepsen.util 3383 :invoke :read [17 nil] INFO [2022-01-10 05:43:48,930] jepsen worker 80 - jepsen.util 3320 :info :write [26 16] :RequestTimeoutException INFO [2022-01-10 05:43:48,931] jepsen worker 80 - jepsen.util 3410 :invoke :read [26 nil] INFO [2022-01-10 05:43:48,957] jepsen worker 80 - jepsen.util 3410 :ok :read [26 10] INFO [2022-01-10 05:43:48,958] jepsen worker 80 - jepsen.util 3410 :invoke :read [26 nil] INFO [2022-01-10 05:43:49,002] jepsen worker 39 - jepsen.util 3189 :info :write [13 12] :RequestTimeoutException INFO [2022-01-10 05:43:49,005] jepsen worker 39 - jepsen.util 3279 :invoke :write [13 22] INFO [2022-01-10 05:43:49,034] jepsen worker 80 - jepsen.util 3410 :ok :read [26 10] INFO [2022-01-10 05:43:49,035] jepsen worker 80 - jepsen.util 3410 :invoke :read [26 nil] INFO [2022-01-10 05:43:49,041] jepsen worker 52 - jepsen.util 3562 :ok :read [17 41] INFO [2022-01-10 05:43:49,042] jepsen worker 52 - jepsen.util 3562 :invoke :write [17 26] INFO [2022-01-10 05:43:49,044] jepsen worker 79 - jepsen.util 3229 :ok :read [26 10] INFO [2022-01-10 05:43:49,045] jepsen worker 79 - jepsen.util 3229 :invoke :read [26 nil] INFO [2022-01-10 05:43:49,065] jepsen worker 80 - jepsen.util 3410 :ok :read [26 10] INFO [2022-01-10 05:43:49,067] jepsen worker 80 - jepsen.util 3410 :invoke :read [26 nil] INFO [2022-01-10 05:43:49,085] jepsen worker 73 - jepsen.util 2503 :ok :read [24 18] INFO [2022-01-10 05:43:49,086] jepsen worker 73 - jepsen.util 2503 :invoke :write [24 3] INFO [2022-01-10 05:43:49,115] jepsen worker 32 - jepsen.util 3002 :info :write [10 21] :RequestTimeoutException INFO [2022-01-10 05:43:49,115] jepsen worker 32 - jepsen.util 3092 :invoke :write [10 27] INFO [2022-01-10 05:43:49,141] jepsen worker 80 - jepsen.util 3410 :ok :read [26 10] INFO [2022-01-10 05:43:49,142] jepsen worker 80 - jepsen.util 3410 :invoke :read [26 nil] INFO [2022-01-10 05:43:49,146] jepsen worker 3 - jepsen.util 2793 :info :write [1 17] :RequestTimeoutException INFO [2022-01-10 05:43:49,147] jepsen worker 3 - jepsen.util 2883 :invoke :write [1 47] INFO [2022-01-10 05:43:49,203] jepsen worker 80 - jepsen.util 3410 :ok :read [26 10] INFO [2022-01-10 05:43:49,204] jepsen worker 80 - jepsen.util 3410 :invoke :write [26 5] INFO [2022-01-10 05:43:49,308] jepsen worker 27 - jepsen.util 2817 :ok :read [9 24] INFO [2022-01-10 05:43:49,308] jepsen worker 27 - jepsen.util 2817 :invoke :write [9 5] INFO [2022-01-10 05:43:49,310] jepsen worker 2 - jepsen.util 2882 :info :write [0 2] :RequestTimeoutException INFO [2022-01-10 05:43:49,311] jepsen worker 2 - jepsen.util 2972 :invoke :read [0 nil] INFO [2022-01-10 05:43:49,331] jepsen worker 79 - jepsen.util 3229 :ok :read [26 10] INFO [2022-01-10 05:43:49,331] jepsen worker 79 - jepsen.util 3229 :invoke :read [26 nil] INFO [2022-01-10 05:43:49,542] jepsen worker 64 - jepsen.util 2764 :ok :read [21 7] INFO [2022-01-10 05:43:49,543] jepsen worker 64 - jepsen.util 2764 :invoke :write [21 45] INFO [2022-01-10 05:43:49,575] jepsen worker 53 - jepsen.util 3383 :ok :read [17 41] INFO [2022-01-10 05:43:49,576] jepsen worker 53 - jepsen.util 3383 :invoke :read [17 nil] INFO [2022-01-10 05:43:49,584] jepsen worker 79 - jepsen.util 3229 :ok :read [26 10] INFO [2022-01-10 05:43:49,585] jepsen worker 79 - jepsen.util 3229 :invoke :read [26 nil] INFO [2022-01-10 05:43:49,585] jepsen worker 25 - jepsen.util 2725 :ok :read [8 27] INFO [2022-01-10 05:43:49,585] jepsen worker 25 - jepsen.util 2725 :invoke :read [8 nil] INFO [2022-01-10 05:43:49,641] jepsen worker 79 - jepsen.util 3229 :ok :read [26 10] INFO [2022-01-10 05:43:49,642] jepsen worker 79 - jepsen.util 3229 :invoke :write [26 31] INFO [2022-01-10 05:43:49,691] jepsen worker 70 - jepsen.util 2950 :info :write [23 9] :RequestTimeoutException INFO [2022-01-10 05:43:49,692] jepsen worker 70 - jepsen.util 3040 :invoke :read [23 nil] INFO [2022-01-10 05:43:49,705] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4385964912280702}, :ns_1@172.23.107.227 {:progress 0.2991202346041055}, :ns_1@172.23.105.187 {:progress 0.2991202346041055}, :ns_1@172.23.107.228 {:progress 0.4176470588235294}} INFO [2022-01-10 05:43:49,705] jepsen worker nemesis - couchbase.util Rebalance check count: 381 INFO [2022-01-10 05:43:49,705] jepsen worker nemesis - couchbase.util Rebalance stuck count: 303 INFO [2022-01-10 05:43:49,745] jepsen worker 49 - jepsen.util 2749 :info :write [16 36] :RequestTimeoutException INFO [2022-01-10 05:43:49,746] jepsen worker 49 - jepsen.util 2839 :invoke :write [16 11] INFO [2022-01-10 05:43:49,812] jepsen worker 4 - jepsen.util 3154 :info :write [1 6] :RequestTimeoutException INFO [2022-01-10 05:43:49,813] jepsen worker 4 - jepsen.util 3244 :invoke :write [1 9] INFO [2022-01-10 05:43:49,820] jepsen worker 53 - jepsen.util 3383 :ok :read [17 41] INFO [2022-01-10 05:43:49,821] jepsen worker 53 - jepsen.util 3383 :invoke :read [17 nil] INFO [2022-01-10 05:43:49,910] jepsen worker 1 - jepsen.util 2881 :ok :read [0 46] INFO [2022-01-10 05:43:49,911] jepsen worker 1 - jepsen.util 2881 :invoke :read [0 nil] INFO [2022-01-10 05:43:49,925] jepsen worker 62 - jepsen.util 3032 :info :write [20 32] :RequestTimeoutException INFO [2022-01-10 05:43:49,926] jepsen worker 62 - jepsen.util 3122 :invoke :write [20 38] INFO [2022-01-10 05:43:49,927] jepsen worker 16 - jepsen.util 2806 :ok :read [5 42] INFO [2022-01-10 05:43:49,927] jepsen worker 16 - jepsen.util 2806 :invoke :write [5 6] INFO [2022-01-10 05:43:50,001] jepsen worker 15 - jepsen.util 2445 :ok :read [5 42] INFO [2022-01-10 05:43:50,002] jepsen worker 15 - jepsen.util 2445 :invoke :read [5 nil] WARN [2022-01-10 05:43:50,111] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12149708,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17402488,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9903531,"encode_duration_us":20792,"operation_id":"0x569e","timeout_ms":10000,"last_server_duration_us":9180063,"total_duration_us":10003445},{"operation_name":"upsert","last_dispatch_duration_us":11877223,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20609779,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":11258706,"encode_duration_us":16768,"operation_id":"0x568a","timeout_ms":10000,"last_server_duration_us":9953044,"total_duration_us":10010140}],"total_count":2}} INFO [2022-01-10 05:43:50,132] jepsen worker 53 - jepsen.util 3383 :ok :read [17 41] INFO [2022-01-10 05:43:50,132] jepsen worker 53 - jepsen.util 3383 :invoke :read [17 nil] INFO [2022-01-10 05:43:50,175] jepsen worker 85 - jepsen.util 3055 :ok :read [28 45] INFO [2022-01-10 05:43:50,176] jepsen worker 85 - jepsen.util 3055 :invoke :write [28 9] WARN [2022-01-10 05:43:50,211] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10538889,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10538889,"total_server_duration_us":49,"operation_id":"0x588e","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":10540900},{"operation_name":"upsert","last_dispatch_duration_us":10560700,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12692642,"total_server_duration_us":228194,"encode_duration_us":19146,"operation_id":"0x5818","timeout_ms":10000,"last_server_duration_us":202744,"total_duration_us":10009188},{"operation_name":"upsert","last_dispatch_duration_us":10377697,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":17615164,"total_server_duration_us":2537211,"encode_duration_us":18106,"operation_id":"0x572f","timeout_ms":10000,"last_server_duration_us":138798,"total_duration_us":10008390},{"operation_name":"upsert","last_dispatch_duration_us":944376,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":944376,"total_server_duration_us":162796,"encode_duration_us":17692,"operation_id":"0x586a","timeout_ms":10000,"last_server_duration_us":162796,"total_duration_us":10007009},{"operation_name":"upsert","last_dispatch_duration_us":254743,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1324947,"total_server_duration_us":455125,"encode_duration_us":19329,"operation_id":"0x588f","timeout_ms":10000,"last_server_duration_us":161835,"total_duration_us":10006467},{"operation_name":"upsert","last_dispatch_duration_us":229670,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":825001,"total_server_duration_us":111158,"encode_duration_us":16457,"operation_id":"0x58dc","timeout_ms":10000,"last_server_duration_us":50652,"total_duration_us":10005141},{"operation_name":"upsert","last_dispatch_duration_us":9848151,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9848151,"total_server_duration_us":103434,"encode_duration_us":17640,"operation_id":"0x58c4","timeout_ms":10000,"last_server_duration_us":103434,"total_duration_us":10004005},{"operation_name":"upsert","last_dispatch_duration_us":10562318,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":17747707,"total_server_duration_us":1489150,"encode_duration_us":17077,"operation_id":"0x5736","timeout_ms":10000,"last_server_duration_us":161260,"total_duration_us":10003895},{"operation_name":"upsert","last_dispatch_duration_us":9526868,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":15118617,"total_server_duration_us":800334,"encode_duration_us":16771,"operation_id":"0x5757","timeout_ms":10000,"last_server_duration_us":216013,"total_duration_us":10003053},{"operation_name":"upsert","last_dispatch_duration_us":9150796,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9150796,"total_server_duration_us":214930,"encode_duration_us":16379,"operation_id":"0x57f0","timeout_ms":10000,"last_server_duration_us":214930,"total_duration_us":10001452}],"total_count":15}} INFO [2022-01-10 05:43:50,283] jepsen worker 24 - jepsen.util 2454 :ok :read [8 27] INFO [2022-01-10 05:43:50,283] jepsen worker 24 - jepsen.util 2454 :invoke :read [8 nil] INFO [2022-01-10 05:43:50,291] jepsen worker 44 - jepsen.util 2744 :info :write [14 41] :RequestTimeoutException INFO [2022-01-10 05:43:50,292] jepsen worker 44 - jepsen.util 2834 :invoke :write [14 49] INFO [2022-01-10 05:43:50,293] jepsen worker 1 - jepsen.util 2881 :ok :read [0 46] INFO [2022-01-10 05:43:50,293] jepsen worker 1 - jepsen.util 2881 :invoke :write [0 34] INFO [2022-01-10 05:43:50,297] jepsen worker 15 - jepsen.util 2445 :ok :read [5 42] INFO [2022-01-10 05:43:50,297] jepsen worker 15 - jepsen.util 2445 :invoke :read [5 nil] INFO [2022-01-10 05:43:50,301] jepsen worker 7 - jepsen.util 2797 :info :write [2 38] :RequestTimeoutException INFO [2022-01-10 05:43:50,301] jepsen worker 7 - jepsen.util 2887 :invoke :read [2 nil] INFO [2022-01-10 05:43:50,491] jepsen worker 45 - jepsen.util 3015 :info :write [15 21] :RequestTimeoutException INFO [2022-01-10 05:43:50,491] jepsen worker 22 - jepsen.util 2452 :info :write [7 47] :RequestTimeoutException INFO [2022-01-10 05:43:50,492] jepsen worker 22 - jepsen.util 2542 :invoke :read [7 nil] INFO [2022-01-10 05:43:50,493] jepsen worker 45 - jepsen.util 3105 :invoke :read [15 nil] INFO [2022-01-10 05:43:50,494] jepsen worker 22 - jepsen.util 2542 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:50,494] jepsen worker 22 - jepsen.util 2542 :invoke :write [7 17] INFO [2022-01-10 05:43:50,552] jepsen worker 50 - jepsen.util 2930 :info :write [16 13] :RequestTimeoutException INFO [2022-01-10 05:43:50,552] jepsen worker 50 - jepsen.util 3020 :invoke :read [16 nil] INFO [2022-01-10 05:43:50,702] jepsen worker 9 - jepsen.util 3339 :info :write [3 49] :RequestTimeoutException INFO [2022-01-10 05:43:50,703] jepsen worker 9 - jepsen.util 3429 :invoke :read [3 nil] INFO [2022-01-10 05:43:50,714] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4444444444444444}, :ns_1@172.23.107.227 {:progress 0.3049853372434017}, :ns_1@172.23.105.187 {:progress 0.3049853372434017}, :ns_1@172.23.107.228 {:progress 0.4235294117647059}} INFO [2022-01-10 05:43:50,714] jepsen worker nemesis - couchbase.util Rebalance check count: 382 INFO [2022-01-10 05:43:50,714] jepsen worker nemesis - couchbase.util Rebalance stuck count: 303 INFO [2022-01-10 05:43:50,715] jepsen worker 59 - jepsen.util 2759 :info :write [19 14] :RequestTimeoutException INFO [2022-01-10 05:43:50,716] jepsen worker 59 - jepsen.util 2849 :invoke :read [19 nil] INFO [2022-01-10 05:43:50,810] jepsen worker 7 - jepsen.util 2887 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:50,811] jepsen worker 7 - jepsen.util 2887 :invoke :read [2 nil] INFO [2022-01-10 05:43:50,826] jepsen worker 24 - jepsen.util 2454 :ok :read [8 27] INFO [2022-01-10 05:43:50,827] jepsen worker 24 - jepsen.util 2454 :invoke :write [8 45] INFO [2022-01-10 05:43:51,124] jepsen worker 53 - jepsen.util 3383 :ok :read [17 41] INFO [2022-01-10 05:43:51,124] jepsen worker 53 - jepsen.util 3383 :invoke :write [17 37] INFO [2022-01-10 05:43:51,136] jepsen worker 13 - jepsen.util 3073 :info :write [4 44] :RequestTimeoutException INFO [2022-01-10 05:43:51,137] jepsen worker 13 - jepsen.util 3163 :invoke :write [4 2] INFO [2022-01-10 05:43:51,188] jepsen worker 45 - jepsen.util 3105 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:43:51,189] jepsen worker 45 - jepsen.util 3105 :invoke :read [15 nil] INFO [2022-01-10 05:43:51,441] jepsen worker 83 - jepsen.util 3233 :info :write [27 41] :RequestTimeoutException INFO [2022-01-10 05:43:51,441] jepsen worker 83 - jepsen.util 3323 :invoke :write [27 7] INFO [2022-01-10 05:43:51,446] jepsen worker 72 - jepsen.util 3042 :info :write [24 47] :RequestTimeoutException INFO [2022-01-10 05:43:51,446] jepsen worker 72 - jepsen.util 3132 :invoke :read [24 nil] INFO [2022-01-10 05:43:51,449] jepsen worker 84 - jepsen.util 2694 :info :write [28 42] :RequestTimeoutException INFO [2022-01-10 05:43:51,450] jepsen worker 84 - jepsen.util 2784 :invoke :write [28 46] INFO [2022-01-10 05:43:51,509] jepsen worker 26 - jepsen.util 2366 :fail :write [8 38] :ServerOutOfMemoryException INFO [2022-01-10 05:43:51,510] jepsen worker 26 - jepsen.util 2366 :invoke :read [8 nil] INFO [2022-01-10 05:43:51,550] jepsen worker 6 - jepsen.util 2976 :fail :write [2 9] :ServerOutOfMemoryException INFO [2022-01-10 05:43:51,551] jepsen worker 6 - jepsen.util 2976 :invoke :read [2 nil] INFO [2022-01-10 05:43:51,641] jepsen worker 46 - jepsen.util 2926 :info :write [15 17] :RequestTimeoutException INFO [2022-01-10 05:43:51,642] jepsen worker 46 - jepsen.util 3016 :invoke :read [15 nil] INFO [2022-01-10 05:43:51,701] jepsen worker 81 - jepsen.util 3051 :info :write [27 11] :RequestTimeoutException INFO [2022-01-10 05:43:51,702] jepsen worker 81 - jepsen.util 3141 :invoke :read [27 nil] INFO [2022-01-10 05:43:51,714] jepsen worker 60 - jepsen.util 3210 :fail :write [20 24] :ServerOutOfMemoryException INFO [2022-01-10 05:43:51,716] jepsen worker 60 - jepsen.util 3210 :invoke :write [20 9] WARN [2022-01-10 05:43:51,719] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18130309,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18130309,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9039489,"encode_duration_us":17483,"operation_id":"0x56cf","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10009679},{"operation_name":"upsert","last_dispatch_duration_us":14819551,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20582986,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":11880240,"encode_duration_us":19243,"operation_id":"0x56b5","timeout_ms":10000,"last_server_duration_us":9611664,"total_duration_us":10010007}],"total_count":2}} WARN [2022-01-10 05:43:51,720] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14504095,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":14504095,"total_server_duration_us":302490,"operation_id":"0x57e9","timeout_ms":30000,"last_server_duration_us":302490,"total_duration_us":14508313},{"operation_name":"get","last_dispatch_duration_us":10173293,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10173293,"total_server_duration_us":10213,"operation_id":"0x58dd","timeout_ms":30000,"last_server_duration_us":10213,"total_duration_us":10176090},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":17068,"operation_id":"0x58a1","timeout_ms":10000,"total_duration_us":10010052},{"operation_name":"upsert","last_dispatch_duration_us":406631,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":6926126,"total_server_duration_us":4333342,"encode_duration_us":16689,"operation_id":"0x58a4","timeout_ms":10000,"last_server_duration_us":394005,"total_duration_us":10009326},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":18072,"operation_id":"0x5892","timeout_ms":10000,"total_duration_us":10007476},{"operation_name":"upsert","last_dispatch_duration_us":799579,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2636110,"total_server_duration_us":680857,"encode_duration_us":16683,"operation_id":"0x5816","timeout_ms":10000,"last_server_duration_us":141143,"total_duration_us":10006789},{"operation_name":"upsert","last_dispatch_duration_us":1226602,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2633885,"total_server_duration_us":289800,"encode_duration_us":17279,"operation_id":"0x587b","timeout_ms":10000,"last_server_duration_us":135936,"total_duration_us":10006210},{"operation_name":"upsert","last_dispatch_duration_us":849593,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3490627,"total_server_duration_us":430653,"encode_duration_us":19325,"operation_id":"0x58e7","timeout_ms":10000,"last_server_duration_us":69883,"total_duration_us":10005971},{"operation_name":"upsert","last_dispatch_duration_us":686313,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3673437,"total_server_duration_us":515066,"encode_duration_us":19131,"operation_id":"0x58e9","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10005730},{"operation_name":"upsert","last_dispatch_duration_us":394009,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3911731,"total_server_duration_us":393458,"encode_duration_us":18410,"operation_id":"0x58ea","timeout_ms":10000,"last_server_duration_us":24669,"total_duration_us":10005587}],"total_count":18}} INFO [2022-01-10 05:43:51,720] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4444444444444444}, :ns_1@172.23.107.227 {:progress 0.3049853372434017}, :ns_1@172.23.105.187 {:progress 0.3049853372434017}, :ns_1@172.23.107.228 {:progress 0.4235294117647059}} INFO [2022-01-10 05:43:51,720] jepsen worker nemesis - couchbase.util Rebalance check count: 383 INFO [2022-01-10 05:43:51,720] jepsen worker nemesis - couchbase.util Rebalance stuck count: 304 WARN [2022-01-10 05:43:51,794] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17142174,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18108186,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9533726,"encode_duration_us":20468,"operation_id":"0x5524","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10006622},{"operation_name":"upsert","last_dispatch_duration_us":15025557,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":15025557,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9217491,"encode_duration_us":16912,"operation_id":"0x56c7","timeout_ms":10000,"last_server_duration_us":9217491,"total_duration_us":10006863}],"total_count":2}} WARN [2022-01-10 05:43:51,810] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9345879,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10859069,"total_server_duration_us":576986,"encode_duration_us":16565,"operation_id":"0x58ec","timeout_ms":10000,"last_server_duration_us":452588,"total_duration_us":10009256},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":16687,"operation_id":"0x58d3","timeout_ms":10000,"total_duration_us":10009027},{"operation_name":"upsert","last_dispatch_duration_us":10822708,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":16955347,"total_server_duration_us":10127949,"encode_duration_us":17738,"operation_id":"0x57a8","timeout_ms":10000,"last_server_duration_us":9137368,"total_duration_us":10008847},{"operation_name":"upsert","last_dispatch_duration_us":7523455,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13624319,"total_server_duration_us":296351,"encode_duration_us":16324,"operation_id":"0x57bc","timeout_ms":10000,"last_server_duration_us":216664,"total_duration_us":10008304},{"operation_name":"upsert","last_dispatch_duration_us":9768093,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9768093,"total_server_duration_us":290824,"encode_duration_us":18140,"operation_id":"0x5846","timeout_ms":10000,"last_server_duration_us":290824,"total_duration_us":10007056},{"operation_name":"upsert","last_dispatch_duration_us":2275137,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12082182,"total_server_duration_us":320472,"encode_duration_us":17914,"operation_id":"0x5817","timeout_ms":10000,"last_server_duration_us":196245,"total_duration_us":10006706},{"operation_name":"upsert","last_dispatch_duration_us":37528,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":186148,"total_server_duration_us":100419,"encode_duration_us":16522,"operation_id":"0x5845","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10004610},{"operation_name":"upsert","last_dispatch_duration_us":9761550,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11866277,"total_server_duration_us":238012,"encode_duration_us":16796,"operation_id":"0x58a8","timeout_ms":10000,"last_server_duration_us":25888,"total_duration_us":10002666},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":17623,"operation_id":"0x58f3","timeout_ms":10000,"total_duration_us":10002641},{"operation_name":"get","last_dispatch_duration_us":9518532,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9518532,"total_server_duration_us":10752,"operation_id":"0x5856","timeout_ms":30000,"last_server_duration_us":10752,"total_duration_us":9520807}],"total_count":18}} WARN [2022-01-10 05:43:51,816] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":626165,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9421007,"total_server_duration_us":2816373,"encode_duration_us":17467,"operation_id":"0x5844","timeout_ms":10000,"last_server_duration_us":200008,"total_duration_us":10006787},{"operation_name":"upsert","last_dispatch_duration_us":1022879,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":6517901,"total_server_duration_us":3356652,"encode_duration_us":16964,"operation_id":"0x57e2","timeout_ms":10000,"last_server_duration_us":386197,"total_duration_us":10005512},{"operation_name":"upsert","last_dispatch_duration_us":1042206,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9747902,"total_server_duration_us":3593615,"encode_duration_us":18958,"operation_id":"0x586f","timeout_ms":10000,"last_server_duration_us":426479,"total_duration_us":10005142},{"operation_name":"upsert","last_dispatch_duration_us":33055,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":7550973,"total_server_duration_us":271011,"encode_duration_us":18180,"operation_id":"0x57dd","timeout_ms":10000,"last_server_duration_us":20521,"total_duration_us":10003714},{"operation_name":"upsert","last_dispatch_duration_us":9042598,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11592896,"total_server_duration_us":241764,"encode_duration_us":20158,"operation_id":"0x57cf","timeout_ms":10000,"last_server_duration_us":141324,"total_duration_us":10002550},{"operation_name":"get","last_dispatch_duration_us":6001394,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":6001394,"total_server_duration_us":8502,"operation_id":"0x579a","timeout_ms":30000,"last_server_duration_us":8502,"total_duration_us":9637721},{"operation_name":"upsert","last_dispatch_duration_us":9069087,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9069087,"total_server_duration_us":9027816,"encode_duration_us":18250,"operation_id":"0x57df","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":9073916},{"operation_name":"get","last_dispatch_duration_us":8220716,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8220716,"total_server_duration_us":372966,"operation_id":"0x57a1","timeout_ms":30000,"last_server_duration_us":372966,"total_duration_us":8398692},{"operation_name":"get","last_dispatch_duration_us":8206153,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8206153,"total_server_duration_us":28310,"operation_id":"0x57a9","timeout_ms":30000,"last_server_duration_us":28310,"total_duration_us":8208217},{"operation_name":"get","last_dispatch_duration_us":6925280,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6925280,"total_server_duration_us":43,"operation_id":"0x58b5","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":6925651}],"total_count":32}} INFO [2022-01-10 05:43:51,846] jepsen worker 35 - jepsen.util 2375 :fail :write [11 23] :ServerOutOfMemoryException INFO [2022-01-10 05:43:51,847] jepsen worker 35 - jepsen.util 2375 :invoke :write [11 40] WARN [2022-01-10 05:43:51,894] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":3338784,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":3338784,"total_server_duration_us":134514,"encode_duration_us":18519,"operation_id":"0x58a0","timeout_ms":10000,"last_server_duration_us":134514,"total_duration_us":10011035},{"operation_name":"upsert","last_dispatch_duration_us":4072454,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":4072454,"total_server_duration_us":206985,"encode_duration_us":16649,"operation_id":"0x58a3","timeout_ms":10000,"last_server_duration_us":206985,"total_duration_us":10009039},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","encode_duration_us":16986,"operation_id":"0x58b2","timeout_ms":10000,"total_duration_us":10007691},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","encode_duration_us":16373,"operation_id":"0x5785","timeout_ms":10000,"total_duration_us":10006137},{"operation_name":"upsert","last_dispatch_duration_us":518103,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9308685,"total_server_duration_us":106977,"encode_duration_us":19476,"operation_id":"0x5919","timeout_ms":10000,"last_server_duration_us":43289,"total_duration_us":10005888},{"operation_name":"upsert","last_dispatch_duration_us":2871466,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":5219900,"total_server_duration_us":340618,"encode_duration_us":19377,"operation_id":"0x5891","timeout_ms":10000,"last_server_duration_us":117618,"total_duration_us":10005882},{"operation_name":"upsert","last_dispatch_duration_us":7984500,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11404643,"total_server_duration_us":192088,"encode_duration_us":16436,"operation_id":"0x57f1","timeout_ms":10000,"last_server_duration_us":47306,"total_duration_us":10002548},{"operation_name":"upsert","last_dispatch_duration_us":775512,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":6297952,"total_server_duration_us":2046224,"encode_duration_us":16708,"operation_id":"0x57b8","timeout_ms":10000,"last_server_duration_us":142231,"total_duration_us":10002459},{"operation_name":"upsert","last_dispatch_duration_us":1013183,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":8812595,"total_server_duration_us":1563854,"encode_duration_us":23025,"operation_id":"0x5914","timeout_ms":10000,"last_server_duration_us":153864,"total_duration_us":10001199},{"operation_name":"upsert","last_dispatch_duration_us":2788994,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":2788994,"total_server_duration_us":95635,"encode_duration_us":16552,"operation_id":"0x58f4","timeout_ms":10000,"last_server_duration_us":95635,"total_duration_us":10001075}],"total_count":18}} WARN [2022-01-10 05:43:51,905] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18042678,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18042678,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9035244,"encode_duration_us":22087,"operation_id":"0x56ba","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10002537},{"operation_name":"upsert","last_dispatch_duration_us":11956841,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11956841,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9121379,"encode_duration_us":24315,"operation_id":"0x57b6","timeout_ms":10000,"last_server_duration_us":9121379,"total_duration_us":10005474}],"total_count":2}} WARN [2022-01-10 05:43:51,910] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17975535,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":26526744,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":11514123,"encode_duration_us":19586,"operation_id":"0x54eb","timeout_ms":10000,"last_server_duration_us":11351300,"total_duration_us":10001209},{"operation_name":"upsert","last_dispatch_duration_us":18115631,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18115631,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9029938,"encode_duration_us":19484,"operation_id":"0x567d","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10008955},{"operation_name":"upsert","last_dispatch_duration_us":10822708,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16955347,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10127949,"encode_duration_us":17738,"operation_id":"0x57a8","timeout_ms":10000,"last_server_duration_us":9137368,"total_duration_us":10008847}],"total_count":3}} WARN [2022-01-10 05:43:51,916] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11974285,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14537381,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9456948,"encode_duration_us":16247,"operation_id":"0x5735","timeout_ms":10000,"last_server_duration_us":9124576,"total_duration_us":10002323}],"total_count":1}} INFO [2022-01-10 05:43:51,926] jepsen worker 62 - jepsen.util 3122 :fail :write [20 38] :ServerOutOfMemoryException INFO [2022-01-10 05:43:51,927] jepsen worker 62 - jepsen.util 3122 :invoke :read [20 nil] INFO [2022-01-10 05:43:51,962] jepsen worker 26 - jepsen.util 2366 :ok :read [8 27] INFO [2022-01-10 05:43:51,963] jepsen worker 26 - jepsen.util 2366 :invoke :write [8 33] INFO [2022-01-10 05:43:51,972] jepsen worker 6 - jepsen.util 2976 :ok :read [2 14] INFO [2022-01-10 05:43:51,972] jepsen worker 6 - jepsen.util 2976 :invoke :read [2 nil] WARN [2022-01-10 05:43:52,105] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1046588,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8394232,"total_server_duration_us":712647,"encode_duration_us":16830,"operation_id":"0x58b4","timeout_ms":10000,"last_server_duration_us":143869,"total_duration_us":10007652},{"operation_name":"upsert","last_dispatch_duration_us":9047030,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11973648,"total_server_duration_us":268965,"encode_duration_us":16419,"operation_id":"0x57bb","timeout_ms":10000,"last_server_duration_us":73136,"total_duration_us":10007193},{"operation_name":"upsert","last_dispatch_duration_us":445190,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9840534,"total_server_duration_us":1802436,"encode_duration_us":17149,"operation_id":"0x58a5","timeout_ms":10000,"last_server_duration_us":93802,"total_duration_us":10006833},{"operation_name":"upsert","last_dispatch_duration_us":2468502,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":5830536,"total_server_duration_us":363230,"encode_duration_us":16528,"operation_id":"0x591a","timeout_ms":10000,"last_server_duration_us":72999,"total_duration_us":10006724},{"operation_name":"upsert","last_dispatch_duration_us":11956841,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11956841,"total_server_duration_us":9121379,"encode_duration_us":24315,"operation_id":"0x57b6","timeout_ms":10000,"last_server_duration_us":9121379,"total_duration_us":10005474},{"operation_name":"upsert","last_dispatch_duration_us":453523,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9938641,"total_server_duration_us":1815068,"encode_duration_us":16708,"operation_id":"0x58a6","timeout_ms":10000,"last_server_duration_us":63568,"total_duration_us":10002281},{"operation_name":"upsert","last_dispatch_duration_us":8190836,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":15048322,"total_server_duration_us":1753888,"encode_duration_us":16454,"operation_id":"0x5758","timeout_ms":10000,"last_server_duration_us":166467,"total_duration_us":10002139},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":16522,"operation_id":"0x5874","timeout_ms":10000,"total_duration_us":10001524},{"operation_name":"upsert","last_dispatch_duration_us":1154556,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10316409,"total_server_duration_us":280935,"encode_duration_us":19240,"operation_id":"0x583a","timeout_ms":10000,"last_server_duration_us":158588,"total_duration_us":10001352},{"operation_name":"upsert","last_dispatch_duration_us":2639614,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":12380562,"total_server_duration_us":1217748,"encode_duration_us":16914,"operation_id":"0x57aa","timeout_ms":10000,"last_server_duration_us":41337,"total_duration_us":10000686}],"total_count":19}} INFO [2022-01-10 05:43:52,146] jepsen worker 21 - jepsen.util 2991 :fail :write [7 0] :ServerOutOfMemoryException INFO [2022-01-10 05:43:52,147] jepsen worker 21 - jepsen.util 2991 :invoke :write [7 34] INFO [2022-01-10 05:43:52,194] jepsen worker 44 - jepsen.util 2834 :fail :write [14 49] :ServerOutOfMemoryException INFO [2022-01-10 05:43:52,194] jepsen worker 44 - jepsen.util 2834 :invoke :write [14 26] INFO [2022-01-10 05:43:52,227] jepsen worker 17 - jepsen.util 2717 :info :write [5 38] :RequestTimeoutException INFO [2022-01-10 05:43:52,227] jepsen worker 17 - jepsen.util 2807 :invoke :read [5 nil] INFO [2022-01-10 05:43:52,249] jepsen worker 7 - jepsen.util 2887 :ok :read [2 14] INFO [2022-01-10 05:43:52,251] jepsen worker 7 - jepsen.util 2887 :invoke :read [2 nil] INFO [2022-01-10 05:43:52,282] jepsen worker 22 - jepsen.util 2542 :fail :write [7 17] :ServerOutOfMemoryException INFO [2022-01-10 05:43:52,283] jepsen worker 22 - jepsen.util 2542 :invoke :read [7 nil] INFO [2022-01-10 05:43:52,726] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4502923976608187}, :ns_1@172.23.107.227 {:progress 0.3049853372434017}, :ns_1@172.23.105.187 {:progress 0.3049853372434017}, :ns_1@172.23.107.228 {:progress 0.4235294117647059}} INFO [2022-01-10 05:43:52,726] jepsen worker nemesis - couchbase.util Rebalance check count: 384 INFO [2022-01-10 05:43:52,726] jepsen worker nemesis - couchbase.util Rebalance stuck count: 304 INFO [2022-01-10 05:43:52,956] jepsen worker 14 - jepsen.util 3344 :info :write [4 15] :RequestTimeoutException INFO [2022-01-10 05:43:52,957] jepsen worker 14 - jepsen.util 3434 :invoke :write [4 9] INFO [2022-01-10 05:43:52,999] jepsen worker 6 - jepsen.util 2976 :ok :read [2 14] INFO [2022-01-10 05:43:53,000] jepsen worker 6 - jepsen.util 2976 :invoke :read [2 nil] INFO [2022-01-10 05:43:53,076] jepsen worker 41 - jepsen.util 2651 :info :write [13 35] :RequestTimeoutException INFO [2022-01-10 05:43:53,076] jepsen worker 41 - jepsen.util 2741 :invoke :read [13 nil] INFO [2022-01-10 05:43:53,142] jepsen worker 41 - jepsen.util 2741 :ok :read [13 15] INFO [2022-01-10 05:43:53,143] jepsen worker 41 - jepsen.util 2741 :invoke :write [13 34] INFO [2022-01-10 05:43:53,299] jepsen worker 89 - jepsen.util 3329 :info :write [29 40] :RequestTimeoutException INFO [2022-01-10 05:43:53,300] jepsen worker 89 - jepsen.util 3419 :invoke :read [29 nil] INFO [2022-01-10 05:43:53,383] jepsen worker 6 - jepsen.util 2976 :ok :read [2 14] INFO [2022-01-10 05:43:53,385] jepsen worker 6 - jepsen.util 2976 :invoke :read [2 nil] INFO [2022-01-10 05:43:53,502] jepsen worker 88 - jepsen.util 2878 :info :write [29 4] :RequestTimeoutException INFO [2022-01-10 05:43:53,502] jepsen worker 88 - jepsen.util 2968 :invoke :read [29 nil] INFO [2022-01-10 05:43:53,604] jepsen worker 22 - jepsen.util 2542 :ok :read [7 :nil] INFO [2022-01-10 05:43:53,605] jepsen worker 22 - jepsen.util 2542 :invoke :read [7 nil] INFO [2022-01-10 05:43:53,736] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4502923976608187}, :ns_1@172.23.107.227 {:progress 0.3049853372434017}, :ns_1@172.23.105.187 {:progress 0.3049853372434017}, :ns_1@172.23.107.228 {:progress 0.4235294117647059}} INFO [2022-01-10 05:43:53,736] jepsen worker nemesis - couchbase.util Rebalance check count: 385 INFO [2022-01-10 05:43:53,736] jepsen worker nemesis - couchbase.util Rebalance stuck count: 305 INFO [2022-01-10 05:43:53,751] jepsen worker 66 - jepsen.util 2496 :ok :read [22 41] INFO [2022-01-10 05:43:53,752] jepsen worker 66 - jepsen.util 2496 :invoke :read [22 nil] INFO [2022-01-10 05:43:53,844] jepsen worker 66 - jepsen.util 2496 :ok :read [22 41] INFO [2022-01-10 05:43:53,844] jepsen worker 66 - jepsen.util 2496 :invoke :write [22 31] INFO [2022-01-10 05:43:53,848] jepsen worker 17 - jepsen.util 2807 :ok :read [5 42] INFO [2022-01-10 05:43:53,849] jepsen worker 17 - jepsen.util 2807 :invoke :write [5 19] INFO [2022-01-10 05:43:53,988] jepsen worker 45 - jepsen.util 3105 :ok :read [15 23] INFO [2022-01-10 05:43:53,989] jepsen worker 45 - jepsen.util 3105 :invoke :write [15 41] INFO [2022-01-10 05:43:53,991] jepsen worker 56 - jepsen.util 3116 :info :write [18 2] :RequestTimeoutException INFO [2022-01-10 05:43:53,992] jepsen worker 56 - jepsen.util 3206 :invoke :read [18 nil] INFO [2022-01-10 05:43:54,015] jepsen worker 68 - jepsen.util 2768 :info :write [22 16] :RequestTimeoutException INFO [2022-01-10 05:43:54,016] jepsen worker 68 - jepsen.util 2858 :invoke :write [22 31] INFO [2022-01-10 05:43:54,038] jepsen worker 2 - jepsen.util 2972 :ok :read [0 46] INFO [2022-01-10 05:43:54,039] jepsen worker 2 - jepsen.util 2972 :invoke :write [0 9] INFO [2022-01-10 05:43:54,050] jepsen worker 0 - jepsen.util 2790 :ok :read [0 46] INFO [2022-01-10 05:43:54,050] jepsen worker 0 - jepsen.util 2790 :invoke :write [0 24] INFO [2022-01-10 05:43:54,066] jepsen worker 58 - jepsen.util 2758 :info :write [19 27] :RequestTimeoutException INFO [2022-01-10 05:43:54,067] jepsen worker 58 - jepsen.util 2848 :invoke :read [19 nil] INFO [2022-01-10 05:43:54,209] jepsen worker 22 - jepsen.util 2542 :ok :read [7 :nil] INFO [2022-01-10 05:43:54,210] jepsen worker 22 - jepsen.util 2542 :invoke :read [7 nil] INFO [2022-01-10 05:43:54,331] jepsen worker 65 - jepsen.util 2585 :info :write [21 46] :RequestTimeoutException INFO [2022-01-10 05:43:54,331] jepsen worker 65 - jepsen.util 2675 :invoke :read [21 nil] INFO [2022-01-10 05:43:54,421] jepsen worker 8 - jepsen.util 3248 :info :write [2 33] :RequestTimeoutException INFO [2022-01-10 05:43:54,421] jepsen worker 8 - jepsen.util 3338 :invoke :read [2 nil] INFO [2022-01-10 05:43:54,475] jepsen worker 20 - jepsen.util 2630 :info :write [6 4] :RequestTimeoutException INFO [2022-01-10 05:43:54,475] jepsen worker 20 - jepsen.util 2720 :invoke :read [6 nil] INFO [2022-01-10 05:43:54,506] jepsen worker 62 - jepsen.util 3122 :ok :read [20 :nil] INFO [2022-01-10 05:43:54,506] jepsen worker 62 - jepsen.util 3122 :invoke :read [20 nil] INFO [2022-01-10 05:43:54,520] jepsen worker 46 - jepsen.util 3016 :ok :read [15 23] INFO [2022-01-10 05:43:54,521] jepsen worker 46 - jepsen.util 3016 :invoke :write [15 28] INFO [2022-01-10 05:43:54,526] jepsen worker 72 - jepsen.util 3132 :ok :read [24 18] INFO [2022-01-10 05:43:54,526] jepsen worker 72 - jepsen.util 3132 :invoke :write [24 46] INFO [2022-01-10 05:43:54,596] jepsen worker 67 - jepsen.util 2587 :info :write [22 37] :RequestTimeoutException INFO [2022-01-10 05:43:54,597] jepsen worker 67 - jepsen.util 2677 :invoke :write [22 26] INFO [2022-01-10 05:43:54,744] jepsen worker 22 - jepsen.util 2542 :ok :read [7 :nil] INFO [2022-01-10 05:43:54,745] jepsen worker 22 - jepsen.util 2542 :invoke :write [7 0] INFO [2022-01-10 05:43:54,745] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.456140350877193}, :ns_1@172.23.107.227 {:progress 0.3108504398826979}, :ns_1@172.23.105.187 {:progress 0.3108504398826979}, :ns_1@172.23.107.228 {:progress 0.4352941176470588}} INFO [2022-01-10 05:43:54,745] jepsen worker nemesis - couchbase.util Rebalance check count: 386 INFO [2022-01-10 05:43:54,745] jepsen worker nemesis - couchbase.util Rebalance stuck count: 305 INFO [2022-01-10 05:43:54,796] jepsen worker 61 - jepsen.util 2851 :info :write [20 19] :RequestTimeoutException INFO [2022-01-10 05:43:54,796] jepsen worker 61 - jepsen.util 2941 :invoke :write [20 33] INFO [2022-01-10 05:43:54,948] jepsen worker 7 - jepsen.util 2887 :ok :read [2 14] INFO [2022-01-10 05:43:54,949] jepsen worker 7 - jepsen.util 2887 :invoke :read [2 nil] INFO [2022-01-10 05:43:54,964] jepsen worker 25 - jepsen.util 2725 :ok :read [8 27] INFO [2022-01-10 05:43:54,964] jepsen worker 25 - jepsen.util 2725 :invoke :write [8 18] INFO [2022-01-10 05:43:55,051] jepsen worker 12 - jepsen.util 2982 :info :write [4 4] :RequestTimeoutException INFO [2022-01-10 05:43:55,052] jepsen worker 12 - jepsen.util 3072 :invoke :write [4 9] INFO [2022-01-10 05:43:55,097] jepsen worker 30 - jepsen.util 2730 :ok :read [10 43] INFO [2022-01-10 05:43:55,098] jepsen worker 30 - jepsen.util 2730 :invoke :write [10 36] INFO [2022-01-10 05:43:55,243] jepsen worker 54 - jepsen.util 2934 :ok :read [18 14] INFO [2022-01-10 05:43:55,244] jepsen worker 54 - jepsen.util 2934 :invoke :write [18 8] INFO [2022-01-10 05:43:55,737] jepsen worker 75 - jepsen.util 2235 :info :write [25 39] :RequestTimeoutException INFO [2022-01-10 05:43:55,737] jepsen worker 75 - jepsen.util 2325 :invoke :read [25 nil] INFO [2022-01-10 05:43:55,741] jepsen worker 77 - jepsen.util 3227 :info :write [25 16] :RequestTimeoutException INFO [2022-01-10 05:43:55,742] jepsen worker 77 - jepsen.util 3317 :invoke :write [25 29] INFO [2022-01-10 05:43:55,743] jepsen worker 62 - jepsen.util 3122 :ok :read [20 :nil] INFO [2022-01-10 05:43:55,744] jepsen worker 62 - jepsen.util 3122 :invoke :write [20 8] INFO [2022-01-10 05:43:55,763] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.456140350877193}, :ns_1@172.23.107.227 {:progress 0.3108504398826979}, :ns_1@172.23.105.187 {:progress 0.3108504398826979}, :ns_1@172.23.107.228 {:progress 0.4352941176470588}} INFO [2022-01-10 05:43:55,763] jepsen worker nemesis - couchbase.util Rebalance check count: 387 INFO [2022-01-10 05:43:55,763] jepsen worker nemesis - couchbase.util Rebalance stuck count: 306 INFO [2022-01-10 05:43:55,824] jepsen worker 20 - jepsen.util 2720 :ok :read [6 25] INFO [2022-01-10 05:43:55,825] jepsen worker 20 - jepsen.util 2720 :invoke :read [6 nil] INFO [2022-01-10 05:43:56,217] jepsen worker 51 - jepsen.util 3291 :info :write [17 15] :RequestTimeoutException INFO [2022-01-10 05:43:56,217] jepsen worker 51 - jepsen.util 3381 :invoke :write [17 29] INFO [2022-01-10 05:43:56,234] jepsen worker 38 - jepsen.util 3098 :ok :read [12 29] INFO [2022-01-10 05:43:56,235] jepsen worker 38 - jepsen.util 3098 :invoke :write [12 12] INFO [2022-01-10 05:43:56,242] jepsen worker 5 - jepsen.util 2885 :ok :read [1 10] INFO [2022-01-10 05:43:56,243] jepsen worker 5 - jepsen.util 2885 :invoke :read [1 nil] INFO [2022-01-10 05:43:56,406] jepsen worker 63 - jepsen.util 2673 :info :write [21 11] :RequestTimeoutException INFO [2022-01-10 05:43:56,407] jepsen worker 63 - jepsen.util 2763 :invoke :write [21 24] INFO [2022-01-10 05:43:56,507] jepsen worker 57 - jepsen.util 3387 :info :write [19 5] :SyncWriteAmbiguous INFO [2022-01-10 05:43:56,508] jepsen worker 57 - jepsen.util 3477 :invoke :write [19 36] INFO [2022-01-10 05:43:56,516] jepsen worker 48 - jepsen.util 2388 :info :write [16 6] :RequestTimeoutException INFO [2022-01-10 05:43:56,517] jepsen worker 48 - jepsen.util 2478 :invoke :write [16 8] INFO [2022-01-10 05:43:56,606] jepsen worker 34 - jepsen.util 2734 :info :write [11 29] :RequestTimeoutException INFO [2022-01-10 05:43:56,606] jepsen worker 34 - jepsen.util 2824 :invoke :write [11 11] INFO [2022-01-10 05:43:56,770] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.456140350877193}, :ns_1@172.23.107.227 {:progress 0.3108504398826979}, :ns_1@172.23.105.187 {:progress 0.3108504398826979}, :ns_1@172.23.107.228 {:progress 0.4352941176470588}} INFO [2022-01-10 05:43:56,770] jepsen worker nemesis - couchbase.util Rebalance check count: 388 INFO [2022-01-10 05:43:56,770] jepsen worker nemesis - couchbase.util Rebalance stuck count: 307 INFO [2022-01-10 05:43:56,974] jepsen worker 11 - jepsen.util 2801 :ok :read [3 31] INFO [2022-01-10 05:43:56,974] jepsen worker 11 - jepsen.util 2801 :invoke :write [3 27] INFO [2022-01-10 05:43:56,986] jepsen worker 43 - jepsen.util 2653 :info :write [14 21] :RequestTimeoutException INFO [2022-01-10 05:43:56,987] jepsen worker 43 - jepsen.util 2743 :invoke :write [14 9] INFO [2022-01-10 05:43:57,053] jepsen worker 58 - jepsen.util 2848 :ok :read [19 31] INFO [2022-01-10 05:43:57,053] jepsen worker 58 - jepsen.util 2848 :invoke :write [19 5] INFO [2022-01-10 05:43:57,116] jepsen worker 28 - jepsen.util 2548 :info :write [9 2] :RequestTimeoutException INFO [2022-01-10 05:43:57,117] jepsen worker 28 - jepsen.util 2638 :invoke :read [9 nil] INFO [2022-01-10 05:43:57,256] jepsen worker 82 - jepsen.util 2872 :info :write [27 10] :RequestTimeoutException INFO [2022-01-10 05:43:57,257] jepsen worker 82 - jepsen.util 2962 :invoke :read [27 nil] INFO [2022-01-10 05:43:57,299] jepsen worker 7 - jepsen.util 2887 :ok :read [2 14] INFO [2022-01-10 05:43:57,299] jepsen worker 7 - jepsen.util 2887 :invoke :read [2 nil] INFO [2022-01-10 05:43:57,510] jepsen worker 74 - jepsen.util 3404 :info :write [24 26] :RequestTimeoutException INFO [2022-01-10 05:43:57,510] jepsen worker 74 - jepsen.util 3494 :invoke :write [24 36] INFO [2022-01-10 05:43:57,635] jepsen worker 18 - jepsen.util 2718 :info :write [6 13] :RequestTimeoutException INFO [2022-01-10 05:43:57,638] jepsen worker 18 - jepsen.util 2808 :invoke :write [6 49] INFO [2022-01-10 05:43:57,776] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.456140350877193}, :ns_1@172.23.107.227 {:progress 0.3108504398826979}, :ns_1@172.23.105.187 {:progress 0.3108504398826979}, :ns_1@172.23.107.228 {:progress 0.4352941176470588}} INFO [2022-01-10 05:43:57,776] jepsen worker nemesis - couchbase.util Rebalance check count: 389 INFO [2022-01-10 05:43:57,776] jepsen worker nemesis - couchbase.util Rebalance stuck count: 308 INFO [2022-01-10 05:43:57,878] jepsen worker 65 - jepsen.util 2675 :ok :read [21 7] INFO [2022-01-10 05:43:57,878] jepsen worker 65 - jepsen.util 2675 :invoke :write [21 14] INFO [2022-01-10 05:43:57,879] jepsen worker 20 - jepsen.util 2720 :ok :read [6 25] INFO [2022-01-10 05:43:57,879] jepsen worker 20 - jepsen.util 2720 :invoke :read [6 nil] INFO [2022-01-10 05:43:57,931] jepsen worker 71 - jepsen.util 2321 :info :write [23 36] :SyncWriteAmbiguous INFO [2022-01-10 05:43:57,931] jepsen worker 71 - jepsen.util 2411 :invoke :read [23 nil] INFO [2022-01-10 05:43:57,992] jepsen worker 89 - jepsen.util 3419 :ok :read [29 0] INFO [2022-01-10 05:43:57,992] jepsen worker 89 - jepsen.util 3419 :invoke :write [29 33] INFO [2022-01-10 05:43:58,152] jepsen worker 10 - jepsen.util 3070 :info :write [3 17] :RequestTimeoutException INFO [2022-01-10 05:43:58,152] jepsen worker 10 - jepsen.util 3160 :invoke :read [3 nil] INFO [2022-01-10 05:43:58,202] jepsen worker 10 - jepsen.util 3160 :ok :read [3 31] INFO [2022-01-10 05:43:58,202] jepsen worker 10 - jepsen.util 3160 :invoke :write [3 47] INFO [2022-01-10 05:43:58,211] jepsen worker 3 - jepsen.util 2883 :info :write [1 47] :SyncWriteAmbiguous INFO [2022-01-10 05:43:58,212] jepsen worker 3 - jepsen.util 2973 :invoke :write [1 40] INFO [2022-01-10 05:43:58,215] jepsen worker 75 - jepsen.util 2325 :ok :read [25 :nil] INFO [2022-01-10 05:43:58,216] jepsen worker 75 - jepsen.util 2325 :invoke :read [25 nil] INFO [2022-01-10 05:43:58,236] jepsen worker 56 - jepsen.util 3206 :ok :read [18 14] INFO [2022-01-10 05:43:58,237] jepsen worker 56 - jepsen.util 3206 :invoke :write [18 17] INFO [2022-01-10 05:43:58,333] jepsen worker 23 - jepsen.util 2993 :info :write [7 13] :SyncWriteAmbiguous INFO [2022-01-10 05:43:58,334] jepsen worker 23 - jepsen.util 3083 :invoke :read [7 nil] INFO [2022-01-10 05:43:58,382] jepsen worker 42 - jepsen.util 2742 :info :write [14 35] :RequestTimeoutException INFO [2022-01-10 05:43:58,382] jepsen worker 42 - jepsen.util 2832 :invoke :write [14 2] INFO [2022-01-10 05:43:58,580] jepsen worker 78 - jepsen.util 3318 :info :write [26 18] :RequestTimeoutException INFO [2022-01-10 05:43:58,581] jepsen worker 78 - jepsen.util 3408 :invoke :write [26 19] INFO [2022-01-10 05:43:58,714] jepsen worker 29 - jepsen.util 2729 :ok :read [9 24] INFO [2022-01-10 05:43:58,714] jepsen worker 29 - jepsen.util 2729 :invoke :read [9 nil] INFO [2022-01-10 05:43:58,786] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4619883040935673}, :ns_1@172.23.107.227 {:progress 0.3167155425219942}, :ns_1@172.23.105.187 {:progress 0.3167155425219942}, :ns_1@172.23.107.228 {:progress 0.4411764705882353}} INFO [2022-01-10 05:43:58,786] jepsen worker nemesis - couchbase.util Rebalance check count: 390 INFO [2022-01-10 05:43:58,786] jepsen worker nemesis - couchbase.util Rebalance stuck count: 308 INFO [2022-01-10 05:43:58,801] jepsen worker 55 - jepsen.util 3115 :info :write [18 45] :RequestTimeoutException INFO [2022-01-10 05:43:58,801] jepsen worker 55 - jepsen.util 3205 :invoke :read [18 nil] INFO [2022-01-10 05:43:58,804] jepsen worker 47 - jepsen.util 2837 :info :write [15 27] :RequestTimeoutException INFO [2022-01-10 05:43:58,805] jepsen worker 47 - jepsen.util 2927 :invoke :write [15 45] INFO [2022-01-10 05:43:58,806] jepsen worker 40 - jepsen.util 2200 :info :write [13 6] :RequestTimeoutException INFO [2022-01-10 05:43:58,807] jepsen worker 40 - jepsen.util 2290 :invoke :read [13 nil] INFO [2022-01-10 05:43:58,862] jepsen worker 76 - jepsen.util 2326 :info :write [25 7] :RequestTimeoutException INFO [2022-01-10 05:43:58,862] jepsen worker 86 - jepsen.util 2246 :info :write [28 23] :RequestTimeoutException INFO [2022-01-10 05:43:58,863] jepsen worker 76 - jepsen.util 2416 :invoke :write [25 7] INFO [2022-01-10 05:43:58,863] jepsen worker 86 - jepsen.util 2336 :invoke :write [28 8] INFO [2022-01-10 05:43:58,882] jepsen worker 33 - jepsen.util 3183 :info :write [11 30] :RequestTimeoutException INFO [2022-01-10 05:43:58,882] jepsen worker 33 - jepsen.util 3273 :invoke :read [11 nil] INFO [2022-01-10 05:43:58,892] jepsen worker 19 - jepsen.util 2899 :info :write [6 9] :RequestTimeoutException INFO [2022-01-10 05:43:58,893] jepsen worker 19 - jepsen.util 2989 :invoke :write [6 15] INFO [2022-01-10 05:43:59,026] jepsen worker 39 - jepsen.util 3279 :info :write [13 22] :RequestTimeoutException INFO [2022-01-10 05:43:59,027] jepsen worker 39 - jepsen.util 3369 :invoke :read [13 nil] INFO [2022-01-10 05:43:59,066] jepsen worker 52 - jepsen.util 3562 :info :write [17 26] :RequestTimeoutException INFO [2022-01-10 05:43:59,067] jepsen worker 52 - jepsen.util 3652 :invoke :write [17 49] INFO [2022-01-10 05:43:59,112] jepsen worker 73 - jepsen.util 2503 :info :write [24 3] :RequestTimeoutException INFO [2022-01-10 05:43:59,113] jepsen worker 73 - jepsen.util 2593 :invoke :write [24 44] INFO [2022-01-10 05:43:59,119] jepsen worker 8 - jepsen.util 3338 :ok :read [2 14] INFO [2022-01-10 05:43:59,120] jepsen worker 8 - jepsen.util 3338 :invoke :write [2 19] INFO [2022-01-10 05:43:59,141] jepsen worker 32 - jepsen.util 3092 :info :write [10 27] :RequestTimeoutException INFO [2022-01-10 05:43:59,142] jepsen worker 32 - jepsen.util 3182 :invoke :read [10 nil] INFO [2022-01-10 05:43:59,232] jepsen worker 80 - jepsen.util 3410 :info :write [26 5] :RequestTimeoutException INFO [2022-01-10 05:43:59,232] jepsen worker 80 - jepsen.util 3500 :invoke :write [26 48] INFO [2022-01-10 05:43:59,335] jepsen worker 27 - jepsen.util 2817 :info :write [9 5] :RequestTimeoutException INFO [2022-01-10 05:43:59,335] jepsen worker 27 - jepsen.util 2907 :invoke :read [9 nil] INFO [2022-01-10 05:43:59,566] jepsen worker 64 - jepsen.util 2764 :info :write [21 45] :RequestTimeoutException INFO [2022-01-10 05:43:59,567] jepsen worker 64 - jepsen.util 2854 :invoke :write [21 18] INFO [2022-01-10 05:43:59,626] jepsen worker 15 - jepsen.util 2445 :ok :read [5 42] INFO [2022-01-10 05:43:59,627] jepsen worker 15 - jepsen.util 2445 :invoke :read [5 nil] INFO [2022-01-10 05:43:59,637] jepsen worker 59 - jepsen.util 2849 :ok :read [19 31] INFO [2022-01-10 05:43:59,637] jepsen worker 59 - jepsen.util 2849 :invoke :read [19 nil] INFO [2022-01-10 05:43:59,663] jepsen worker 81 - jepsen.util 3141 :ok :read [27 30] INFO [2022-01-10 05:43:59,663] jepsen worker 81 - jepsen.util 3141 :invoke :read [27 nil] INFO [2022-01-10 05:43:59,676] jepsen worker 79 - jepsen.util 3229 :info :write [26 31] :RequestTimeoutException INFO [2022-01-10 05:43:59,676] jepsen worker 79 - jepsen.util 3319 :invoke :write [26 6] INFO [2022-01-10 05:43:59,679] jepsen worker 82 - jepsen.util 2962 :ok :read [27 30] INFO [2022-01-10 05:43:59,679] jepsen worker 82 - jepsen.util 2962 :invoke :write [27 46] INFO [2022-01-10 05:43:59,716] jepsen worker 29 - jepsen.util 2729 :ok :read [9 24] INFO [2022-01-10 05:43:59,717] jepsen worker 29 - jepsen.util 2729 :invoke :read [9 nil] INFO [2022-01-10 05:43:59,739] jepsen worker 15 - jepsen.util 2445 :ok :read [5 42] INFO [2022-01-10 05:43:59,740] jepsen worker 15 - jepsen.util 2445 :invoke :write [5 23] INFO [2022-01-10 05:43:59,770] jepsen worker 50 - jepsen.util 3020 :ok :read [16 38] INFO [2022-01-10 05:43:59,770] jepsen worker 50 - jepsen.util 3020 :invoke :read [16 nil] INFO [2022-01-10 05:43:59,772] jepsen worker 49 - jepsen.util 2839 :info :write [16 11] :RequestTimeoutException INFO [2022-01-10 05:43:59,773] jepsen worker 49 - jepsen.util 2929 :invoke :read [16 nil] INFO [2022-01-10 05:43:59,792] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4678362573099415}, :ns_1@172.23.107.227 {:progress 0.3225806451612904}, :ns_1@172.23.105.187 {:progress 0.3225806451612904}, :ns_1@172.23.107.228 {:progress 0.4470588235294117}} INFO [2022-01-10 05:43:59,792] jepsen worker nemesis - couchbase.util Rebalance check count: 391 INFO [2022-01-10 05:43:59,792] jepsen worker nemesis - couchbase.util Rebalance stuck count: 308 INFO [2022-01-10 05:43:59,830] jepsen worker 39 - jepsen.util 3369 :ok :read [13 15] INFO [2022-01-10 05:43:59,831] jepsen worker 39 - jepsen.util 3369 :invoke :write [13 18] INFO [2022-01-10 05:43:59,835] jepsen worker 29 - jepsen.util 2729 :ok :read [9 24] INFO [2022-01-10 05:43:59,836] jepsen worker 4 - jepsen.util 3244 :info :write [1 9] :RequestTimeoutException INFO [2022-01-10 05:43:59,839] jepsen worker 59 - jepsen.util 2849 :ok :read [19 31] INFO [2022-01-10 05:43:59,840] jepsen worker 29 - jepsen.util 2729 :invoke :read [9 nil] INFO [2022-01-10 05:43:59,844] jepsen worker 59 - jepsen.util 2849 :invoke :write [19 29] INFO [2022-01-10 05:43:59,845] jepsen worker 4 - jepsen.util 3334 :invoke :write [1 31] INFO [2022-01-10 05:43:59,848] jepsen worker 50 - jepsen.util 3020 :ok :read [16 38] INFO [2022-01-10 05:43:59,851] jepsen worker 50 - jepsen.util 3020 :invoke :read [16 nil] INFO [2022-01-10 05:43:59,861] jepsen worker 81 - jepsen.util 3141 :ok :read [27 30] INFO [2022-01-10 05:43:59,863] jepsen worker 81 - jepsen.util 3141 :invoke :write [27 30] INFO [2022-01-10 05:43:59,940] jepsen worker 20 - jepsen.util 2720 :ok :read [6 25] INFO [2022-01-10 05:43:59,941] jepsen worker 20 - jepsen.util 2720 :invoke :write [6 34] INFO [2022-01-10 05:43:59,951] jepsen worker 16 - jepsen.util 2806 :info :write [5 6] :RequestTimeoutException INFO [2022-01-10 05:43:59,952] jepsen worker 16 - jepsen.util 2896 :invoke :write [5 5] WARN [2022-01-10 05:44:00,124] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14961322,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14961322,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9069231,"encode_duration_us":19334,"operation_id":"0x5993","timeout_ms":10000,"last_server_duration_us":9069231,"total_duration_us":10004512},{"operation_name":"upsert","last_dispatch_duration_us":10207302,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10207302,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9545300,"encode_duration_us":19016,"operation_id":"0x5aa7","timeout_ms":10000,"last_server_duration_us":9545300,"total_duration_us":10007164}],"total_count":2}} INFO [2022-01-10 05:44:00,179] jepsen worker 7 - jepsen.util 2887 :ok :read [2 14] INFO [2022-01-10 05:44:00,179] jepsen worker 7 - jepsen.util 2887 :invoke :write [2 25] INFO [2022-01-10 05:44:00,201] jepsen worker 85 - jepsen.util 3055 :info :write [28 9] :RequestTimeoutException INFO [2022-01-10 05:44:00,202] jepsen worker 85 - jepsen.util 3145 :invoke :read [28 nil] WARN [2022-01-10 05:44:00,224] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":542373,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2445092,"total_server_duration_us":617983,"encode_duration_us":19107,"operation_id":"0x5a7d","timeout_ms":10000,"last_server_duration_us":151059,"total_duration_us":10009563},{"operation_name":"upsert","last_dispatch_duration_us":10207302,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10207302,"total_server_duration_us":9545300,"encode_duration_us":19016,"operation_id":"0x5aa7","timeout_ms":10000,"last_server_duration_us":9545300,"total_duration_us":10007164},{"operation_name":"upsert","last_dispatch_duration_us":1005722,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3321888,"total_server_duration_us":979277,"encode_duration_us":16803,"operation_id":"0x59a1","timeout_ms":10000,"last_server_duration_us":540907,"total_duration_us":10004715},{"operation_name":"upsert","last_dispatch_duration_us":14961322,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14961322,"total_server_duration_us":9069231,"encode_duration_us":19334,"operation_id":"0x5993","timeout_ms":10000,"last_server_duration_us":9069231,"total_duration_us":10004512},{"operation_name":"upsert","last_dispatch_duration_us":603845,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3573598,"total_server_duration_us":787098,"encode_duration_us":17577,"operation_id":"0x5a33","timeout_ms":10000,"last_server_duration_us":68148,"total_duration_us":10003931},{"operation_name":"upsert","last_dispatch_duration_us":290980,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1198936,"total_server_duration_us":315652,"encode_duration_us":17840,"operation_id":"0x593f","timeout_ms":10000,"last_server_duration_us":177689,"total_duration_us":9434199},{"operation_name":"get","last_dispatch_duration_us":9312232,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9312232,"total_server_duration_us":13223,"operation_id":"0x5add","timeout_ms":30000,"last_server_duration_us":13223,"total_duration_us":9314599},{"operation_name":"upsert","last_dispatch_duration_us":9038077,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9038077,"total_server_duration_us":9025695,"encode_duration_us":17948,"operation_id":"0x5aa4","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":9044012},{"operation_name":"get","last_dispatch_duration_us":8904090,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8904090,"total_server_duration_us":32,"operation_id":"0x5af2","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":8906277},{"operation_name":"get","last_dispatch_duration_us":7945149,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":7945149,"total_server_duration_us":8612,"operation_id":"0x5b1c","timeout_ms":30000,"last_server_duration_us":8612,"total_duration_us":7947102}],"total_count":15}} INFO [2022-01-10 05:44:00,314] jepsen worker 1 - jepsen.util 2881 :info :write [0 34] :RequestTimeoutException INFO [2022-01-10 05:44:00,315] jepsen worker 1 - jepsen.util 2971 :invoke :read [0 nil] INFO [2022-01-10 05:44:00,350] jepsen worker 29 - jepsen.util 2729 :ok :read [9 24] INFO [2022-01-10 05:44:00,350] jepsen worker 29 - jepsen.util 2729 :invoke :read [9 nil] INFO [2022-01-10 05:44:00,420] jepsen worker 50 - jepsen.util 3020 :ok :read [16 38] INFO [2022-01-10 05:44:00,421] jepsen worker 50 - jepsen.util 3020 :invoke :read [16 nil] INFO [2022-01-10 05:44:00,485] jepsen worker 29 - jepsen.util 2729 :ok :read [9 24] INFO [2022-01-10 05:44:00,485] jepsen worker 29 - jepsen.util 2729 :invoke :write [9 0] INFO [2022-01-10 05:44:00,489] jepsen worker 49 - jepsen.util 2929 :ok :read [16 38] INFO [2022-01-10 05:44:00,490] jepsen worker 49 - jepsen.util 2929 :invoke :read [16 nil] INFO [2022-01-10 05:44:00,494] jepsen worker 1 - jepsen.util 2971 :ok :read [0 46] INFO [2022-01-10 05:44:00,495] jepsen worker 1 - jepsen.util 2971 :invoke :read [0 nil] INFO [2022-01-10 05:44:00,547] jepsen worker 50 - jepsen.util 3020 :ok :read [16 38] INFO [2022-01-10 05:44:00,548] jepsen worker 50 - jepsen.util 3020 :invoke :write [16 31] INFO [2022-01-10 05:44:00,558] jepsen worker 1 - jepsen.util 2971 :ok :read [0 46] INFO [2022-01-10 05:44:00,558] jepsen worker 1 - jepsen.util 2971 :invoke :read [0 nil] INFO [2022-01-10 05:44:00,776] jepsen worker 28 - jepsen.util 2638 :ok :read [9 24] INFO [2022-01-10 05:44:00,776] jepsen worker 28 - jepsen.util 2638 :invoke :write [9 30] INFO [2022-01-10 05:44:00,784] jepsen worker 1 - jepsen.util 2971 :ok :read [0 46] INFO [2022-01-10 05:44:00,784] jepsen worker 40 - jepsen.util 2290 :ok :read [13 15] INFO [2022-01-10 05:44:00,785] jepsen worker 40 - jepsen.util 2290 :invoke :write [13 35] INFO [2022-01-10 05:44:00,786] jepsen worker 1 - jepsen.util 2971 :invoke :read [0 nil] INFO [2022-01-10 05:44:00,797] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4678362573099415}, :ns_1@172.23.107.227 {:progress 0.3225806451612904}, :ns_1@172.23.105.187 {:progress 0.3225806451612904}, :ns_1@172.23.107.228 {:progress 0.4470588235294117}} INFO [2022-01-10 05:44:00,797] jepsen worker nemesis - couchbase.util Rebalance check count: 392 INFO [2022-01-10 05:44:00,797] jepsen worker nemesis - couchbase.util Rebalance stuck count: 309 INFO [2022-01-10 05:44:00,856] jepsen worker 24 - jepsen.util 2454 :info :write [8 45] :RequestTimeoutException INFO [2022-01-10 05:44:00,856] jepsen worker 24 - jepsen.util 2544 :invoke :read [8 nil] INFO [2022-01-10 05:44:01,150] jepsen worker 53 - jepsen.util 3383 :info :write [17 37] :RequestTimeoutException INFO [2022-01-10 05:44:01,150] jepsen worker 53 - jepsen.util 3473 :invoke :read [17 nil] INFO [2022-01-10 05:44:01,166] jepsen worker 13 - jepsen.util 3163 :info :write [4 2] :RequestTimeoutException INFO [2022-01-10 05:44:01,166] jepsen worker 13 - jepsen.util 3253 :invoke :write [4 7] INFO [2022-01-10 05:44:01,180] jepsen worker 53 - jepsen.util 3473 :ok :read [17 41] INFO [2022-01-10 05:44:01,182] jepsen worker 53 - jepsen.util 3473 :invoke :write [17 7] INFO [2022-01-10 05:44:01,228] jepsen worker 49 - jepsen.util 2929 :ok :read [16 38] INFO [2022-01-10 05:44:01,229] jepsen worker 49 - jepsen.util 2929 :invoke :read [16 nil] INFO [2022-01-10 05:44:01,275] jepsen worker 1 - jepsen.util 2971 :ok :read [0 46] INFO [2022-01-10 05:44:01,275] jepsen worker 1 - jepsen.util 2971 :invoke :read [0 nil] INFO [2022-01-10 05:44:01,460] jepsen worker 83 - jepsen.util 3323 :info :write [27 7] :RequestTimeoutException INFO [2022-01-10 05:44:01,460] jepsen worker 83 - jepsen.util 3413 :invoke :write [27 31] INFO [2022-01-10 05:44:01,476] jepsen worker 84 - jepsen.util 2784 :info :write [28 46] :RequestTimeoutException INFO [2022-01-10 05:44:01,477] jepsen worker 84 - jepsen.util 2874 :invoke :write [28 0] INFO [2022-01-10 05:44:01,511] jepsen worker 75 - jepsen.util 2325 :ok :read [25 :nil] INFO [2022-01-10 05:44:01,512] jepsen worker 75 - jepsen.util 2325 :invoke :write [25 18] WARN [2022-01-10 05:44:01,733] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9713966,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19033892,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9190999,"encode_duration_us":16808,"operation_id":"0x5918","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10003642},{"operation_name":"upsert","last_dispatch_duration_us":18087018,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18087018,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9033121,"encode_duration_us":18072,"operation_id":"0x5892","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10007476}],"total_count":2}} WARN [2022-01-10 05:44:01,733] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17867829,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":17867829,"total_server_duration_us":49,"operation_id":"0x58aa","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":17868915},{"operation_name":"upsert","encode_duration_us":16435,"operation_id":"0x5a9f","timeout_ms":10000,"total_duration_us":10009089},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":16351,"operation_id":"0x5992","timeout_ms":10000,"total_duration_us":10008794},{"operation_name":"upsert","last_dispatch_duration_us":9744701,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":18846682,"total_server_duration_us":155157,"encode_duration_us":16297,"operation_id":"0x591f","timeout_ms":10000,"last_server_duration_us":41767,"total_duration_us":10008427},{"operation_name":"upsert","last_dispatch_duration_us":9281869,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10497481,"total_server_duration_us":213250,"encode_duration_us":16222,"operation_id":"0x5adb","timeout_ms":10000,"last_server_duration_us":47079,"total_duration_us":10008152},{"operation_name":"upsert","last_dispatch_duration_us":1147208,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7684429,"total_server_duration_us":807066,"encode_duration_us":17078,"operation_id":"0x59db","timeout_ms":10000,"last_server_duration_us":128022,"total_duration_us":10007163},{"operation_name":"upsert","last_dispatch_duration_us":1947957,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5722625,"total_server_duration_us":721740,"encode_duration_us":17207,"operation_id":"0x599f","timeout_ms":10000,"last_server_duration_us":71502,"total_duration_us":10007126},{"operation_name":"upsert","last_dispatch_duration_us":2023871,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5746431,"total_server_duration_us":555553,"encode_duration_us":16477,"operation_id":"0x591e","timeout_ms":10000,"last_server_duration_us":83124,"total_duration_us":10004570},{"operation_name":"upsert","last_dispatch_duration_us":2022014,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8675898,"total_server_duration_us":1175953,"encode_duration_us":16332,"operation_id":"0x59a0","timeout_ms":10000,"last_server_duration_us":593209,"total_duration_us":10003605},{"operation_name":"upsert","last_dispatch_duration_us":9736208,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10678090,"total_server_duration_us":182371,"encode_duration_us":20755,"operation_id":"0x5ad2","timeout_ms":10000,"last_server_duration_us":70286,"total_duration_us":10002753}],"total_count":20}} INFO [2022-01-10 05:44:01,739] jepsen worker 60 - jepsen.util 3210 :info :write [20 9] :RequestTimeoutException INFO [2022-01-10 05:44:01,740] jepsen worker 60 - jepsen.util 3300 :invoke :write [20 44] INFO [2022-01-10 05:44:01,803] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4678362573099415}, :ns_1@172.23.107.227 {:progress 0.3225806451612904}, :ns_1@172.23.105.187 {:progress 0.3225806451612904}, :ns_1@172.23.107.228 {:progress 0.4470588235294117}} INFO [2022-01-10 05:44:01,803] jepsen worker nemesis - couchbase.util Rebalance check count: 393 INFO [2022-01-10 05:44:01,803] jepsen worker nemesis - couchbase.util Rebalance stuck count: 310 WARN [2022-01-10 05:44:01,806] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1413905,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10722590,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":377918,"encode_duration_us":19476,"operation_id":"0x5919","timeout_ms":10000,"last_server_duration_us":270941,"total_duration_us":10005888},{"operation_name":"upsert","last_dispatch_duration_us":11470411,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14809196,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9219695,"encode_duration_us":18519,"operation_id":"0x58a0","timeout_ms":10000,"last_server_duration_us":9085181,"total_duration_us":10011035},{"operation_name":"upsert","last_dispatch_duration_us":21601131,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":21601131,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9034183,"encode_duration_us":16373,"operation_id":"0x5785","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10006137}],"total_count":3}} WARN [2022-01-10 05:44:01,823] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9015819,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9015819,"total_server_duration_us":603,"operation_id":"0x591b","timeout_ms":30000,"last_server_duration_us":603,"total_duration_us":13676017},{"operation_name":"upsert","last_dispatch_duration_us":3007193,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9821106,"total_server_duration_us":337667,"encode_duration_us":16729,"operation_id":"0x5a9b","timeout_ms":10000,"last_server_duration_us":117786,"total_duration_us":10008921},{"operation_name":"upsert","last_dispatch_duration_us":2843292,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":5632534,"total_server_duration_us":468587,"encode_duration_us":23334,"operation_id":"0x5a8f","timeout_ms":10000,"last_server_duration_us":43618,"total_duration_us":10008575},{"operation_name":"upsert","last_dispatch_duration_us":34298,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":4058239,"total_server_duration_us":2777320,"encode_duration_us":18994,"operation_id":"0x5aa5","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10008353},{"operation_name":"upsert","last_dispatch_duration_us":175983,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":4292710,"total_server_duration_us":3442095,"encode_duration_us":16546,"operation_id":"0x59c5","timeout_ms":10000,"last_server_duration_us":163951,"total_duration_us":10007369},{"operation_name":"upsert","last_dispatch_duration_us":69874,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":2344867,"total_server_duration_us":299589,"encode_duration_us":18307,"operation_id":"0x5ac1","timeout_ms":10000,"last_server_duration_us":46287,"total_duration_us":10007191},{"operation_name":"upsert","last_dispatch_duration_us":2776727,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7397401,"total_server_duration_us":327741,"encode_duration_us":22485,"operation_id":"0x5a8c","timeout_ms":10000,"last_server_duration_us":106465,"total_duration_us":10006424},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":17861,"operation_id":"0x5972","timeout_ms":10000,"total_duration_us":10006306},{"operation_name":"upsert","last_dispatch_duration_us":34422,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":3150817,"total_server_duration_us":264088,"encode_duration_us":17503,"operation_id":"0x5a8d","timeout_ms":10000,"last_server_duration_us":22059,"total_duration_us":10005421},{"operation_name":"upsert","last_dispatch_duration_us":3157975,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9775564,"total_server_duration_us":626453,"encode_duration_us":16242,"operation_id":"0x5a5e","timeout_ms":10000,"last_server_duration_us":306254,"total_duration_us":10004475}],"total_count":17}} WARN [2022-01-10 05:44:01,833] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":33066,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":3950504,"total_server_duration_us":1974059,"encode_duration_us":23837,"operation_id":"0x5aad","timeout_ms":10000,"last_server_duration_us":21895,"total_duration_us":10009495},{"operation_name":"upsert","last_dispatch_duration_us":118967,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":6179279,"total_server_duration_us":2197079,"encode_duration_us":16956,"operation_id":"0x59f6","timeout_ms":10000,"last_server_duration_us":80398,"total_duration_us":10009336},{"operation_name":"upsert","last_dispatch_duration_us":10153923,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11935368,"total_server_duration_us":388327,"encode_duration_us":18478,"operation_id":"0x5a20","timeout_ms":10000,"last_server_duration_us":89286,"total_duration_us":10008759},{"operation_name":"upsert","last_dispatch_duration_us":8913348,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11658495,"total_server_duration_us":808871,"encode_duration_us":16572,"operation_id":"0x5a1e","timeout_ms":10000,"last_server_duration_us":85596,"total_duration_us":10008377},{"operation_name":"upsert","last_dispatch_duration_us":8919745,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12091831,"total_server_duration_us":1197108,"encode_duration_us":17403,"operation_id":"0x59f9","timeout_ms":10000,"last_server_duration_us":362902,"total_duration_us":10007159},{"operation_name":"upsert","last_dispatch_duration_us":10067606,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10067606,"total_server_duration_us":92589,"encode_duration_us":17982,"operation_id":"0x5a81","timeout_ms":10000,"last_server_duration_us":92589,"total_duration_us":10007041},{"operation_name":"upsert","last_dispatch_duration_us":56114,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8270045,"total_server_duration_us":85821,"encode_duration_us":16884,"operation_id":"0x5acf","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10005701},{"operation_name":"upsert","last_dispatch_duration_us":564814,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":564814,"total_server_duration_us":25450,"encode_duration_us":19920,"operation_id":"0x5b12","timeout_ms":10000,"last_server_duration_us":25450,"total_duration_us":10005284},{"operation_name":"upsert","last_dispatch_duration_us":617117,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":4575123,"total_server_duration_us":325307,"encode_duration_us":16261,"operation_id":"0x59a2","timeout_ms":10000,"last_server_duration_us":120649,"total_duration_us":10004215},{"operation_name":"upsert","last_dispatch_duration_us":10026893,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":12104521,"total_server_duration_us":9306217,"encode_duration_us":18903,"operation_id":"0x59fb","timeout_ms":10000,"last_server_duration_us":9218562,"total_duration_us":10003276}],"total_count":24}} INFO [2022-01-10 05:44:01,836] jepsen worker 49 - jepsen.util 2929 :ok :read [16 38] INFO [2022-01-10 05:44:01,837] jepsen worker 49 - jepsen.util 2929 :invoke :read [16 nil] INFO [2022-01-10 05:44:01,849] jepsen worker 24 - jepsen.util 2544 :ok :read [8 27] INFO [2022-01-10 05:44:01,849] jepsen worker 24 - jepsen.util 2544 :invoke :write [8 14] INFO [2022-01-10 05:44:01,869] jepsen worker 35 - jepsen.util 2375 :info :write [11 40] :RequestTimeoutException INFO [2022-01-10 05:44:01,870] jepsen worker 35 - jepsen.util 2465 :invoke :write [11 8] INFO [2022-01-10 05:44:01,878] jepsen worker 30 - jepsen.util 2730 :ok :write [10 36] INFO [2022-01-10 05:44:01,879] jepsen worker 30 - jepsen.util 2730 :invoke :read [10 nil] INFO [2022-01-10 05:44:01,906] jepsen worker 32 - jepsen.util 3182 :ok :read [10 36] WARN [2022-01-10 05:44:01,907] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17189643,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":17189643,"total_server_duration_us":38,"operation_id":"0x586b","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":17190660},{"operation_name":"get","last_dispatch_duration_us":10664130,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10664130,"total_server_duration_us":15339,"operation_id":"0x5969","timeout_ms":30000,"last_server_duration_us":15339,"total_duration_us":10668850},{"operation_name":"upsert","last_dispatch_duration_us":31900,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":5151759,"total_server_duration_us":1111234,"encode_duration_us":17153,"operation_id":"0x5b0e","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10010660},{"operation_name":"upsert","last_dispatch_duration_us":2505348,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11825480,"total_server_duration_us":283654,"encode_duration_us":17371,"operation_id":"0x59b7","timeout_ms":10000,"last_server_duration_us":60759,"total_duration_us":10010468},{"operation_name":"upsert","last_dispatch_duration_us":9453795,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9453795,"total_server_duration_us":19730,"encode_duration_us":18187,"operation_id":"0x5994","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10009957},{"operation_name":"upsert","last_dispatch_duration_us":2845649,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7088787,"total_server_duration_us":540779,"encode_duration_us":20211,"operation_id":"0x5b06","timeout_ms":10000,"last_server_duration_us":31681,"total_duration_us":10008211},{"operation_name":"upsert","last_dispatch_duration_us":10656597,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10656597,"total_server_duration_us":20124,"encode_duration_us":16450,"operation_id":"0x5968","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10007859},{"operation_name":"upsert","last_dispatch_duration_us":2749472,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9780159,"total_server_duration_us":345186,"encode_duration_us":16561,"operation_id":"0x59fd","timeout_ms":10000,"last_server_duration_us":123200,"total_duration_us":10005948},{"operation_name":"upsert","last_dispatch_duration_us":2807169,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9206737,"total_server_duration_us":382834,"encode_duration_us":17610,"operation_id":"0x5aa9","timeout_ms":10000,"last_server_duration_us":111819,"total_duration_us":10004769},{"operation_name":"upsert","last_dispatch_duration_us":28593,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":5810391,"total_server_duration_us":1798927,"encode_duration_us":20910,"operation_id":"0x5a99","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10002189}],"total_count":18}} INFO [2022-01-10 05:44:01,908] jepsen worker 32 - jepsen.util 3182 :invoke :write [10 19] WARN [2022-01-10 05:44:01,922] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":25532640,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":25532640,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9030999,"encode_duration_us":16565,"operation_id":"0x5704","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10006726}],"total_count":1}} WARN [2022-01-10 05:44:01,923] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17340621,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17375323,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9055184,"encode_duration_us":16953,"operation_id":"0x5734","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10002534},{"operation_name":"upsert","last_dispatch_duration_us":17307918,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17494066,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9145216,"encode_duration_us":16522,"operation_id":"0x5845","timeout_ms":10000,"last_server_duration_us":9044797,"total_duration_us":10004610}],"total_count":2}} WARN [2022-01-10 05:44:01,933] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10026893,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12104521,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9306217,"encode_duration_us":18903,"operation_id":"0x59fb","timeout_ms":10000,"last_server_duration_us":9218562,"total_duration_us":10003276}],"total_count":1}} INFO [2022-01-10 05:44:01,955] jepsen worker 30 - jepsen.util 2730 :ok :read [10 36] INFO [2022-01-10 05:44:01,956] jepsen worker 30 - jepsen.util 2730 :invoke :write [10 12] INFO [2022-01-10 05:44:01,985] jepsen worker 26 - jepsen.util 2366 :info :write [8 33] :RequestTimeoutException INFO [2022-01-10 05:44:01,986] jepsen worker 26 - jepsen.util 2456 :invoke :write [8 47] WARN [2022-01-10 05:44:02,022] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":22710350,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":22710350,"total_server_duration_us":43,"operation_id":"0x5786","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":22711633},{"operation_name":"get","last_dispatch_duration_us":18118230,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":18118230,"total_server_duration_us":32,"operation_id":"0x5870","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":18119375},{"operation_name":"upsert","last_dispatch_duration_us":1229037,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":6934911,"total_server_duration_us":743363,"encode_duration_us":19083,"operation_id":"0x5a2f","timeout_ms":10000,"last_server_duration_us":20441,"total_duration_us":10006562},{"operation_name":"upsert","last_dispatch_duration_us":436889,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":4424334,"total_server_duration_us":816858,"encode_duration_us":18110,"operation_id":"0x5b0d","timeout_ms":10000,"last_server_duration_us":82979,"total_duration_us":10006292},{"operation_name":"upsert","last_dispatch_duration_us":891788,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7206004,"total_server_duration_us":1307589,"encode_duration_us":16786,"operation_id":"0x5b36","timeout_ms":10000,"last_server_duration_us":95482,"total_duration_us":10005091},{"operation_name":"upsert","last_dispatch_duration_us":1108655,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7199168,"total_server_duration_us":983057,"encode_duration_us":18342,"operation_id":"0x5b23","timeout_ms":10000,"last_server_duration_us":130287,"total_duration_us":10004782},{"operation_name":"upsert","last_dispatch_duration_us":549295,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":6167305,"total_server_duration_us":1172384,"encode_duration_us":16446,"operation_id":"0x5a60","timeout_ms":10000,"last_server_duration_us":97483,"total_duration_us":10003389},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":16545,"operation_id":"0x5970","timeout_ms":10000,"total_duration_us":10002937},{"operation_name":"upsert","last_dispatch_duration_us":715108,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7523140,"total_server_duration_us":515930,"encode_duration_us":16434,"operation_id":"0x5b11","timeout_ms":10000,"last_server_duration_us":89584,"total_duration_us":10000928},{"operation_name":"upsert","last_dispatch_duration_us":9899905,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9899905,"total_server_duration_us":9101143,"encode_duration_us":16449,"operation_id":"0x59fe","timeout_ms":10000,"last_server_duration_us":9101143,"total_duration_us":9903416}],"total_count":16}} INFO [2022-01-10 05:44:02,073] jepsen worker 49 - jepsen.util 2929 :ok :read [16 38] INFO [2022-01-10 05:44:02,074] jepsen worker 49 - jepsen.util 2929 :invoke :read [16 nil] INFO [2022-01-10 05:44:02,170] jepsen worker 21 - jepsen.util 2991 :info :write [7 34] :RequestTimeoutException INFO [2022-01-10 05:44:02,171] jepsen worker 21 - jepsen.util 3081 :invoke :write [7 2] INFO [2022-01-10 05:44:02,225] jepsen worker 44 - jepsen.util 2834 :info :write [14 26] :RequestTimeoutException INFO [2022-01-10 05:44:02,226] jepsen worker 44 - jepsen.util 2924 :invoke :write [14 13] INFO [2022-01-10 05:44:02,439] jepsen worker 41 - jepsen.util 2741 :info :write [13 34] :SyncWriteAmbiguous INFO [2022-01-10 05:44:02,440] jepsen worker 41 - jepsen.util 2831 :invoke :read [13 nil] INFO [2022-01-10 05:44:02,468] jepsen worker 49 - jepsen.util 2929 :ok :read [16 38] INFO [2022-01-10 05:44:02,469] jepsen worker 49 - jepsen.util 2929 :invoke :read [16 nil] INFO [2022-01-10 05:44:02,809] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4736842105263158}, :ns_1@172.23.107.227 {:progress 0.3284457478005866}, :ns_1@172.23.105.187 {:progress 0.3284457478005866}, :ns_1@172.23.107.228 {:progress 0.4529411764705882}} INFO [2022-01-10 05:44:02,809] jepsen worker nemesis - couchbase.util Rebalance check count: 394 INFO [2022-01-10 05:44:02,809] jepsen worker nemesis - couchbase.util Rebalance stuck count: 310 INFO [2022-01-10 05:44:02,980] jepsen worker 14 - jepsen.util 3434 :info :write [4 9] :RequestTimeoutException INFO [2022-01-10 05:44:02,980] jepsen worker 14 - jepsen.util 3524 :invoke :write [4 17] INFO [2022-01-10 05:44:03,149] jepsen worker 17 - jepsen.util 2807 :info :write [5 19] :SyncWriteAmbiguous INFO [2022-01-10 05:44:03,150] jepsen worker 17 - jepsen.util 2897 :invoke :read [5 nil] INFO [2022-01-10 05:44:03,171] jepsen worker 27 - jepsen.util 2907 :ok :read [9 24] INFO [2022-01-10 05:44:03,172] jepsen worker 27 - jepsen.util 2907 :invoke :write [9 42] INFO [2022-01-10 05:44:03,233] jepsen worker 6 - jepsen.util 2976 :ok :read [2 14] INFO [2022-01-10 05:44:03,233] jepsen worker 6 - jepsen.util 2976 :invoke :read [2 nil] INFO [2022-01-10 05:44:03,238] jepsen worker 85 - jepsen.util 3145 :ok :read [28 45] INFO [2022-01-10 05:44:03,239] jepsen worker 85 - jepsen.util 3145 :invoke :read [28 nil] INFO [2022-01-10 05:44:03,282] jepsen worker 2 - jepsen.util 2972 :info :write [0 9] :SyncWriteAmbiguous INFO [2022-01-10 05:44:03,283] jepsen worker 2 - jepsen.util 3062 :invoke :write [0 37] INFO [2022-01-10 05:44:03,805] jepsen worker 41 - jepsen.util 2831 :ok :read [13 15] INFO [2022-01-10 05:44:03,806] jepsen worker 41 - jepsen.util 2831 :invoke :write [13 44] INFO [2022-01-10 05:44:03,815] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4736842105263158}, :ns_1@172.23.107.227 {:progress 0.3284457478005866}, :ns_1@172.23.105.187 {:progress 0.3284457478005866}, :ns_1@172.23.107.228 {:progress 0.4529411764705882}} INFO [2022-01-10 05:44:03,815] jepsen worker nemesis - couchbase.util Rebalance check count: 395 INFO [2022-01-10 05:44:03,815] jepsen worker nemesis - couchbase.util Rebalance stuck count: 311 INFO [2022-01-10 05:44:03,817] jepsen worker 17 - jepsen.util 2897 :ok :read [5 42] INFO [2022-01-10 05:44:03,818] jepsen worker 17 - jepsen.util 2897 :invoke :read [5 nil] INFO [2022-01-10 05:44:03,834] jepsen worker 23 - jepsen.util 3083 :ok :read [7 :nil] INFO [2022-01-10 05:44:03,835] jepsen worker 23 - jepsen.util 3083 :invoke :read [7 nil] INFO [2022-01-10 05:44:03,865] jepsen worker 66 - jepsen.util 2496 :info :write [22 31] :RequestTimeoutException INFO [2022-01-10 05:44:03,866] jepsen worker 66 - jepsen.util 2586 :invoke :read [22 nil] INFO [2022-01-10 05:44:04,012] jepsen worker 45 - jepsen.util 3105 :info :write [15 41] :RequestTimeoutException INFO [2022-01-10 05:44:04,012] jepsen worker 45 - jepsen.util 3195 :invoke :read [15 nil] INFO [2022-01-10 05:44:04,030] jepsen worker 6 - jepsen.util 2976 :ok :read [2 14] INFO [2022-01-10 05:44:04,031] jepsen worker 6 - jepsen.util 2976 :invoke :write [2 36] INFO [2022-01-10 05:44:04,044] jepsen worker 85 - jepsen.util 3145 :ok :read [28 45] INFO [2022-01-10 05:44:04,045] jepsen worker 85 - jepsen.util 3145 :invoke :write [28 1] INFO [2022-01-10 05:44:04,046] jepsen worker 68 - jepsen.util 2858 :info :write [22 31] :RequestTimeoutException INFO [2022-01-10 05:44:04,047] jepsen worker 68 - jepsen.util 2948 :invoke :read [22 nil] INFO [2022-01-10 05:44:04,055] jepsen worker 17 - jepsen.util 2897 :ok :read [5 42] INFO [2022-01-10 05:44:04,055] jepsen worker 17 - jepsen.util 2897 :invoke :write [5 5] INFO [2022-01-10 05:44:04,076] jepsen worker 0 - jepsen.util 2790 :info :write [0 24] :RequestTimeoutException INFO [2022-01-10 05:44:04,077] jepsen worker 0 - jepsen.util 2880 :invoke :write [0 30] INFO [2022-01-10 05:44:04,310] jepsen worker 71 - jepsen.util 2411 :ok :read [23 26] INFO [2022-01-10 05:44:04,310] jepsen worker 71 - jepsen.util 2411 :invoke :read [23 nil] INFO [2022-01-10 05:44:04,320] jepsen worker 55 - jepsen.util 3205 :ok :read [18 14] INFO [2022-01-10 05:44:04,321] jepsen worker 55 - jepsen.util 3205 :invoke :write [18 25] INFO [2022-01-10 05:44:04,358] jepsen worker 71 - jepsen.util 2411 :ok :read [23 26] INFO [2022-01-10 05:44:04,358] jepsen worker 71 - jepsen.util 2411 :invoke :read [23 nil] INFO [2022-01-10 05:44:04,396] jepsen worker 23 - jepsen.util 3083 :ok :read [7 :nil] INFO [2022-01-10 05:44:04,397] jepsen worker 23 - jepsen.util 3083 :invoke :read [7 nil] INFO [2022-01-10 05:44:04,429] jepsen worker 71 - jepsen.util 2411 :ok :read [23 26] INFO [2022-01-10 05:44:04,430] jepsen worker 71 - jepsen.util 2411 :invoke :read [23 nil] INFO [2022-01-10 05:44:04,497] jepsen worker 71 - jepsen.util 2411 :ok :read [23 26] INFO [2022-01-10 05:44:04,498] jepsen worker 71 - jepsen.util 2411 :invoke :read [23 nil] INFO [2022-01-10 05:44:04,542] jepsen worker 46 - jepsen.util 3016 :info :write [15 28] :RequestTimeoutException INFO [2022-01-10 05:44:04,542] jepsen worker 46 - jepsen.util 3106 :invoke :read [15 nil] INFO [2022-01-10 05:44:04,552] jepsen worker 72 - jepsen.util 3132 :info :write [24 46] :RequestTimeoutException INFO [2022-01-10 05:44:04,552] jepsen worker 72 - jepsen.util 3222 :invoke :write [24 15] INFO [2022-01-10 05:44:04,563] jepsen worker 71 - jepsen.util 2411 :ok :read [23 26] INFO [2022-01-10 05:44:04,563] jepsen worker 71 - jepsen.util 2411 :invoke :write [23 46] INFO [2022-01-10 05:44:04,615] jepsen worker 67 - jepsen.util 2677 :info :write [22 26] :RequestTimeoutException INFO [2022-01-10 05:44:04,616] jepsen worker 67 - jepsen.util 2767 :invoke :write [22 4] INFO [2022-01-10 05:44:04,718] jepsen worker 66 - jepsen.util 2586 :ok :read [22 41] INFO [2022-01-10 05:44:04,719] jepsen worker 66 - jepsen.util 2586 :invoke :write [22 32] INFO [2022-01-10 05:44:04,759] jepsen worker 23 - jepsen.util 3083 :ok :read [7 :nil] INFO [2022-01-10 05:44:04,760] jepsen worker 23 - jepsen.util 3083 :invoke :read [7 nil] INFO [2022-01-10 05:44:04,769] jepsen worker 22 - jepsen.util 2542 :info :write [7 0] :RequestTimeoutException INFO [2022-01-10 05:44:04,770] jepsen worker 22 - jepsen.util 2632 :invoke :write [7 19] INFO [2022-01-10 05:44:04,822] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4736842105263158}, :ns_1@172.23.107.227 {:progress 0.3284457478005866}, :ns_1@172.23.105.187 {:progress 0.3284457478005866}, :ns_1@172.23.107.228 {:progress 0.4529411764705882}} INFO [2022-01-10 05:44:04,822] jepsen worker nemesis - couchbase.util Rebalance check count: 396 INFO [2022-01-10 05:44:04,822] jepsen worker nemesis - couchbase.util Rebalance stuck count: 312 INFO [2022-01-10 05:44:04,826] jepsen worker 61 - jepsen.util 2941 :info :write [20 33] :RequestTimeoutException INFO [2022-01-10 05:44:04,827] jepsen worker 61 - jepsen.util 3031 :invoke :write [20 24] INFO [2022-01-10 05:44:04,909] jepsen worker 68 - jepsen.util 2948 :ok :read [22 41] INFO [2022-01-10 05:44:04,909] jepsen worker 68 - jepsen.util 2948 :invoke :write [22 22] INFO [2022-01-10 05:44:04,986] jepsen worker 25 - jepsen.util 2725 :info :write [8 18] :RequestTimeoutException INFO [2022-01-10 05:44:04,987] jepsen worker 25 - jepsen.util 2815 :invoke :write [8 33] INFO [2022-01-10 05:44:05,080] jepsen worker 12 - jepsen.util 3072 :info :write [4 9] :RequestTimeoutException INFO [2022-01-10 05:44:05,081] jepsen worker 12 - jepsen.util 3162 :invoke :read [4 nil] INFO [2022-01-10 05:44:05,150] jepsen worker 12 - jepsen.util 3162 :ok :read [4 3] INFO [2022-01-10 05:44:05,150] jepsen worker 12 - jepsen.util 3162 :invoke :read [4 nil] INFO [2022-01-10 05:44:05,275] jepsen worker 54 - jepsen.util 2934 :info :write [18 8] :RequestTimeoutException INFO [2022-01-10 05:44:05,276] jepsen worker 54 - jepsen.util 3024 :invoke :read [18 nil] INFO [2022-01-10 05:44:05,295] jepsen worker 12 - jepsen.util 3162 :ok :read [4 3] INFO [2022-01-10 05:44:05,295] jepsen worker 12 - jepsen.util 3162 :invoke :write [4 8] INFO [2022-01-10 05:44:05,298] jepsen worker 87 - jepsen.util 2967 :ok :read [29 0] INFO [2022-01-10 05:44:05,298] jepsen worker 87 - jepsen.util 2967 :invoke :write [29 0] INFO [2022-01-10 05:44:05,766] jepsen worker 77 - jepsen.util 3317 :info :write [25 29] :RequestTimeoutException INFO [2022-01-10 05:44:05,767] jepsen worker 77 - jepsen.util 3407 :invoke :read [25 nil] INFO [2022-01-10 05:44:05,771] jepsen worker 62 - jepsen.util 3122 :info :write [20 8] :RequestTimeoutException INFO [2022-01-10 05:44:05,771] jepsen worker 62 - jepsen.util 3212 :invoke :write [20 43] INFO [2022-01-10 05:44:05,828] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4736842105263158}, :ns_1@172.23.107.227 {:progress 0.3284457478005866}, :ns_1@172.23.105.187 {:progress 0.3284457478005866}, :ns_1@172.23.107.228 {:progress 0.4529411764705882}} INFO [2022-01-10 05:44:05,828] jepsen worker nemesis - couchbase.util Rebalance check count: 397 INFO [2022-01-10 05:44:05,828] jepsen worker nemesis - couchbase.util Rebalance stuck count: 313 INFO [2022-01-10 05:44:05,986] jepsen worker 31 - jepsen.util 3271 :ok :read [10 36] INFO [2022-01-10 05:44:05,987] jepsen worker 31 - jepsen.util 3271 :invoke :read [10 nil] INFO [2022-01-10 05:44:06,023] jepsen worker 23 - jepsen.util 3083 :ok :read [7 :nil] INFO [2022-01-10 05:44:06,023] jepsen worker 23 - jepsen.util 3083 :invoke :read [7 nil] INFO [2022-01-10 05:44:06,062] jepsen worker 9 - jepsen.util 3429 :ok :read [3 31] INFO [2022-01-10 05:44:06,063] jepsen worker 9 - jepsen.util 3429 :invoke :write [3 0] INFO [2022-01-10 05:44:06,126] jepsen worker 31 - jepsen.util 3271 :ok :read [10 36] INFO [2022-01-10 05:44:06,127] jepsen worker 31 - jepsen.util 3271 :invoke :read [10 nil] INFO [2022-01-10 05:44:06,164] jepsen worker 31 - jepsen.util 3271 :ok :read [10 36] INFO [2022-01-10 05:44:06,165] jepsen worker 31 - jepsen.util 3271 :invoke :read [10 nil] INFO [2022-01-10 05:44:06,206] jepsen worker 31 - jepsen.util 3271 :ok :read [10 36] INFO [2022-01-10 05:44:06,207] jepsen worker 31 - jepsen.util 3271 :invoke :write [10 20] INFO [2022-01-10 05:44:06,241] jepsen worker 51 - jepsen.util 3381 :info :write [17 29] :RequestTimeoutException INFO [2022-01-10 05:44:06,242] jepsen worker 51 - jepsen.util 3471 :invoke :write [17 13] INFO [2022-01-10 05:44:06,259] jepsen worker 38 - jepsen.util 3098 :info :write [12 12] :RequestTimeoutException INFO [2022-01-10 05:44:06,260] jepsen worker 38 - jepsen.util 3188 :invoke :write [12 19] INFO [2022-01-10 05:44:06,354] jepsen worker 37 - jepsen.util 2827 :ok :read [12 29] INFO [2022-01-10 05:44:06,355] jepsen worker 37 - jepsen.util 2827 :invoke :write [12 24] INFO [2022-01-10 05:44:06,381] jepsen worker 88 - jepsen.util 2968 :ok :read [29 0] INFO [2022-01-10 05:44:06,381] jepsen worker 88 - jepsen.util 2968 :invoke :read [29 nil] INFO [2022-01-10 05:44:06,393] jepsen worker 46 - jepsen.util 3106 :ok :read [15 23] INFO [2022-01-10 05:44:06,394] jepsen worker 46 - jepsen.util 3106 :invoke :write [15 15] INFO [2022-01-10 05:44:06,435] jepsen worker 63 - jepsen.util 2763 :info :write [21 24] :RequestTimeoutException INFO [2022-01-10 05:44:06,435] jepsen worker 63 - jepsen.util 2853 :invoke :read [21 nil] INFO [2022-01-10 05:44:06,494] jepsen worker 88 - jepsen.util 2968 :ok :read [29 0] INFO [2022-01-10 05:44:06,494] jepsen worker 88 - jepsen.util 2968 :invoke :write [29 41] INFO [2022-01-10 05:44:06,535] jepsen worker 23 - jepsen.util 3083 :ok :read [7 :nil] INFO [2022-01-10 05:44:06,537] jepsen worker 57 - jepsen.util 3477 :info :write [19 36] :RequestTimeoutException INFO [2022-01-10 05:44:06,924] jepsen worker 23 - jepsen.util 3083 :invoke :write [7 14] INFO [2022-01-10 05:44:06,924] jepsen worker 57 - jepsen.util 3567 :invoke :write [19 13] INFO [2022-01-10 05:44:06,539] jepsen worker 48 - jepsen.util 2478 :info :write [16 8] :RequestTimeoutException INFO [2022-01-10 05:44:06,632] jepsen worker 34 - jepsen.util 2824 :info :write [11 11] :RequestTimeoutException INFO [2022-01-10 05:44:06,931] jepsen worker 48 - jepsen.util 2568 :invoke :write [16 0] INFO [2022-01-10 05:44:06,931] jepsen worker 34 - jepsen.util 2914 :invoke :write [11 12] INFO [2022-01-10 05:44:06,834] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4736842105263158}, :ns_1@172.23.107.227 {:progress 0.3284457478005866}, :ns_1@172.23.105.187 {:progress 0.3284457478005866}, :ns_1@172.23.107.228 {:progress 0.4529411764705882}} INFO [2022-01-10 05:44:06,932] jepsen worker nemesis - couchbase.util Rebalance check count: 398 INFO [2022-01-10 05:44:06,932] jepsen worker nemesis - couchbase.util Rebalance stuck count: 314 INFO [2022-01-10 05:44:07,001] jepsen worker 11 - jepsen.util 2801 :info :write [3 27] :RequestTimeoutException INFO [2022-01-10 05:44:07,001] jepsen worker 11 - jepsen.util 2891 :invoke :write [3 5] INFO [2022-01-10 05:44:07,006] jepsen worker 43 - jepsen.util 2743 :info :write [14 9] :RequestTimeoutException INFO [2022-01-10 05:44:07,007] jepsen worker 43 - jepsen.util 2833 :invoke :read [14 nil] INFO [2022-01-10 05:44:07,080] jepsen worker 58 - jepsen.util 2848 :info :write [19 5] :RequestTimeoutException INFO [2022-01-10 05:44:07,080] jepsen worker 58 - jepsen.util 2938 :invoke :read [19 nil] INFO [2022-01-10 05:44:07,308] jepsen worker 56 - jepsen.util 3206 :info :write [18 17] :SyncWriteAmbiguous INFO [2022-01-10 05:44:07,308] jepsen worker 56 - jepsen.util 3296 :invoke :read [18 nil] INFO [2022-01-10 05:44:07,335] jepsen worker 54 - jepsen.util 3024 :ok :read [18 14] INFO [2022-01-10 05:44:07,337] jepsen worker 54 - jepsen.util 3024 :invoke :read [18 nil] INFO [2022-01-10 05:44:07,339] jepsen worker 56 - jepsen.util 3296 :ok :read [18 14] INFO [2022-01-10 05:44:07,340] jepsen worker 56 - jepsen.util 3296 :invoke :read [18 nil] INFO [2022-01-10 05:44:07,365] jepsen worker 54 - jepsen.util 3024 :ok :read [18 14] INFO [2022-01-10 05:44:07,365] jepsen worker 54 - jepsen.util 3024 :invoke :write [18 20] INFO [2022-01-10 05:44:07,372] jepsen worker 56 - jepsen.util 3296 :ok :read [18 14] INFO [2022-01-10 05:44:07,372] jepsen worker 56 - jepsen.util 3296 :invoke :read [18 nil] INFO [2022-01-10 05:44:07,397] jepsen worker 56 - jepsen.util 3296 :ok :read [18 14] INFO [2022-01-10 05:44:07,398] jepsen worker 56 - jepsen.util 3296 :invoke :write [18 48] INFO [2022-01-10 05:44:07,536] jepsen worker 74 - jepsen.util 3494 :info :write [24 36] :RequestTimeoutException INFO [2022-01-10 05:44:07,537] jepsen worker 74 - jepsen.util 3584 :invoke :read [24 nil] INFO [2022-01-10 05:44:07,602] jepsen worker 43 - jepsen.util 2833 :ok :read [14 14] INFO [2022-01-10 05:44:07,603] jepsen worker 43 - jepsen.util 2833 :invoke :write [14 5] INFO [2022-01-10 05:44:07,659] jepsen worker 18 - jepsen.util 2808 :info :write [6 49] :RequestTimeoutException INFO [2022-01-10 05:44:07,662] jepsen worker 18 - jepsen.util 2898 :invoke :write [6 46] INFO [2022-01-10 05:44:07,727] jepsen worker 63 - jepsen.util 2853 :ok :read [21 7] INFO [2022-01-10 05:44:07,728] jepsen worker 63 - jepsen.util 2853 :invoke :write [21 28] INFO [2022-01-10 05:44:07,902] jepsen worker 65 - jepsen.util 2675 :info :write [21 14] :RequestTimeoutException INFO [2022-01-10 05:44:07,903] jepsen worker 65 - jepsen.util 2765 :invoke :write [21 30] INFO [2022-01-10 05:44:07,937] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3343108504398827}, :ns_1@172.23.105.187 {:progress 0.3343108504398827}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:07,938] jepsen worker nemesis - couchbase.util Rebalance check count: 399 INFO [2022-01-10 05:44:07,938] jepsen worker nemesis - couchbase.util Rebalance stuck count: 314 INFO [2022-01-10 05:44:08,016] jepsen worker 89 - jepsen.util 3419 :info :write [29 33] :RequestTimeoutException INFO [2022-01-10 05:44:08,017] jepsen worker 89 - jepsen.util 3509 :invoke :read [29 nil] INFO [2022-01-10 05:44:08,065] jepsen worker 89 - jepsen.util 3509 :ok :read [29 0] INFO [2022-01-10 05:44:08,066] jepsen worker 89 - jepsen.util 3509 :invoke :read [29 nil] INFO [2022-01-10 05:44:08,093] jepsen worker 89 - jepsen.util 3509 :ok :read [29 0] INFO [2022-01-10 05:44:08,094] jepsen worker 89 - jepsen.util 3509 :invoke :write [29 30] INFO [2022-01-10 05:44:08,226] jepsen worker 10 - jepsen.util 3160 :info :write [3 47] :RequestTimeoutException INFO [2022-01-10 05:44:08,227] jepsen worker 10 - jepsen.util 3250 :invoke :read [3 nil] INFO [2022-01-10 05:44:08,242] jepsen worker 3 - jepsen.util 2973 :info :write [1 40] :RequestTimeoutException INFO [2022-01-10 05:44:08,243] jepsen worker 3 - jepsen.util 3063 :invoke :read [1 nil] INFO [2022-01-10 05:44:08,405] jepsen worker 42 - jepsen.util 2832 :info :write [14 2] :RequestTimeoutException INFO [2022-01-10 05:44:08,406] jepsen worker 42 - jepsen.util 2922 :invoke :read [14 nil] INFO [2022-01-10 05:44:08,610] jepsen worker 78 - jepsen.util 3408 :info :write [26 19] :RequestTimeoutException INFO [2022-01-10 05:44:08,610] jepsen worker 78 - jepsen.util 3498 :invoke :read [26 nil] INFO [2022-01-10 05:44:08,633] jepsen worker 78 - jepsen.util 3498 :ok :read [26 10] INFO [2022-01-10 05:44:08,633] jepsen worker 78 - jepsen.util 3498 :invoke :read [26 nil] INFO [2022-01-10 05:44:08,658] jepsen worker 78 - jepsen.util 3498 :ok :read [26 10] INFO [2022-01-10 05:44:08,659] jepsen worker 78 - jepsen.util 3498 :invoke :read [26 nil] INFO [2022-01-10 05:44:08,718] jepsen worker 78 - jepsen.util 3498 :ok :read [26 10] INFO [2022-01-10 05:44:08,719] jepsen worker 78 - jepsen.util 3498 :invoke :read [26 nil] INFO [2022-01-10 05:44:08,821] jepsen worker 78 - jepsen.util 3498 :ok :read [26 10] INFO [2022-01-10 05:44:08,822] jepsen worker 78 - jepsen.util 3498 :invoke :write [26 26] INFO [2022-01-10 05:44:08,826] jepsen worker 47 - jepsen.util 2927 :info :write [15 45] :RequestTimeoutException INFO [2022-01-10 05:44:08,827] jepsen worker 47 - jepsen.util 3017 :invoke :write [15 22] INFO [2022-01-10 05:44:08,884] jepsen worker 76 - jepsen.util 2416 :info :write [25 7] :RequestTimeoutException INFO [2022-01-10 05:44:08,885] jepsen worker 76 - jepsen.util 2506 :invoke :write [25 5] INFO [2022-01-10 05:44:08,887] jepsen worker 86 - jepsen.util 2336 :info :write [28 8] :RequestTimeoutException INFO [2022-01-10 05:44:08,887] jepsen worker 86 - jepsen.util 2426 :invoke :write [28 45] INFO [2022-01-10 05:44:08,911] jepsen worker 19 - jepsen.util 2989 :info :write [6 15] :RequestTimeoutException INFO [2022-01-10 05:44:08,912] jepsen worker 19 - jepsen.util 3079 :invoke :read [6 nil] INFO [2022-01-10 05:44:08,943] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3343108504398827}, :ns_1@172.23.105.187 {:progress 0.3343108504398827}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:08,943] jepsen worker nemesis - couchbase.util Rebalance check count: 400 INFO [2022-01-10 05:44:08,943] jepsen worker nemesis - couchbase.util Rebalance stuck count: 315 INFO [2022-01-10 05:44:09,091] jepsen worker 52 - jepsen.util 3652 :info :write [17 49] :RequestTimeoutException INFO [2022-01-10 05:44:09,094] jepsen worker 52 - jepsen.util 3742 :invoke :read [17 nil] INFO [2022-01-10 05:44:09,136] jepsen worker 73 - jepsen.util 2593 :info :write [24 44] :RequestTimeoutException INFO [2022-01-10 05:44:09,136] jepsen worker 73 - jepsen.util 2683 :invoke :write [24 47] INFO [2022-01-10 05:44:09,146] jepsen worker 8 - jepsen.util 3338 :info :write [2 19] :RequestTimeoutException INFO [2022-01-10 05:44:09,147] jepsen worker 8 - jepsen.util 3428 :invoke :read [2 nil] INFO [2022-01-10 05:44:09,253] jepsen worker 52 - jepsen.util 3742 :ok :read [17 41] INFO [2022-01-10 05:44:09,254] jepsen worker 52 - jepsen.util 3742 :invoke :write [17 14] INFO [2022-01-10 05:44:09,260] jepsen worker 80 - jepsen.util 3500 :info :write [26 48] :RequestTimeoutException INFO [2022-01-10 05:44:09,260] jepsen worker 80 - jepsen.util 3590 :invoke :write [26 9] INFO [2022-01-10 05:44:09,587] jepsen worker 64 - jepsen.util 2854 :info :write [21 18] :RequestTimeoutException INFO [2022-01-10 05:44:09,588] jepsen worker 64 - jepsen.util 2944 :invoke :read [21 nil] INFO [2022-01-10 05:44:09,700] jepsen worker 79 - jepsen.util 3319 :info :write [26 6] :RequestTimeoutException INFO [2022-01-10 05:44:09,701] jepsen worker 79 - jepsen.util 3409 :invoke :read [26 nil] INFO [2022-01-10 05:44:09,704] jepsen worker 82 - jepsen.util 2962 :info :write [27 46] :RequestTimeoutException INFO [2022-01-10 05:44:09,705] jepsen worker 82 - jepsen.util 3052 :invoke :read [27 nil] INFO [2022-01-10 05:44:09,764] jepsen worker 15 - jepsen.util 2445 :info :write [5 23] :RequestTimeoutException INFO [2022-01-10 05:44:09,765] jepsen worker 15 - jepsen.util 2535 :invoke :write [5 4] INFO [2022-01-10 05:44:09,814] jepsen worker 33 - jepsen.util 3273 :ok :read [11 21] INFO [2022-01-10 05:44:09,815] jepsen worker 33 - jepsen.util 3273 :invoke :read [11 nil] INFO [2022-01-10 05:44:09,856] jepsen worker 39 - jepsen.util 3369 :info :write [13 18] :RequestTimeoutException INFO [2022-01-10 05:44:09,857] jepsen worker 39 - jepsen.util 3459 :invoke :read [13 nil] INFO [2022-01-10 05:44:09,865] jepsen worker 4 - jepsen.util 3334 :info :write [1 31] :RequestTimeoutException INFO [2022-01-10 05:44:09,866] jepsen worker 4 - jepsen.util 3424 :invoke :write [1 0] INFO [2022-01-10 05:44:09,875] jepsen worker 59 - jepsen.util 2849 :info :write [19 29] :RequestTimeoutException INFO [2022-01-10 05:44:09,876] jepsen worker 59 - jepsen.util 2939 :invoke :read [19 nil] INFO [2022-01-10 05:44:09,895] jepsen worker 81 - jepsen.util 3141 :info :write [27 30] :RequestTimeoutException INFO [2022-01-10 05:44:09,895] jepsen worker 81 - jepsen.util 3231 :invoke :read [27 nil] INFO [2022-01-10 05:44:09,971] jepsen worker 20 - jepsen.util 2720 :info :write [6 34] :RequestTimeoutException INFO [2022-01-10 05:44:09,971] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3343108504398827}, :ns_1@172.23.105.187 {:progress 0.3343108504398827}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:09,971] jepsen worker nemesis - couchbase.util Rebalance check count: 401 INFO [2022-01-10 05:44:09,971] jepsen worker nemesis - couchbase.util Rebalance stuck count: 316 INFO [2022-01-10 05:44:09,972] jepsen worker 20 - jepsen.util 2810 :invoke :write [6 26] INFO [2022-01-10 05:44:09,979] jepsen worker 16 - jepsen.util 2896 :info :write [5 5] :RequestTimeoutException INFO [2022-01-10 05:44:09,980] jepsen worker 16 - jepsen.util 2986 :invoke :write [5 31] INFO [2022-01-10 05:44:10,206] jepsen worker 7 - jepsen.util 2887 :info :write [2 25] :RequestTimeoutException INFO [2022-01-10 05:44:10,207] jepsen worker 7 - jepsen.util 2977 :invoke :write [2 22] WARN [2022-01-10 05:44:10,280] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9753157,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14940033,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9180228,"encode_duration_us":15546,"operation_id":"0x5ba5","timeout_ms":10000,"last_server_duration_us":9091564,"total_duration_us":10001896},{"operation_name":"upsert","last_dispatch_duration_us":9420877,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10522731,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9192534,"encode_duration_us":18554,"operation_id":"0x5b39","timeout_ms":10000,"last_server_duration_us":9148034,"total_duration_us":10003016}],"total_count":2}} WARN [2022-01-10 05:44:10,280] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9451923,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10506056,"total_server_duration_us":310634,"encode_duration_us":19310,"operation_id":"0x5b41","timeout_ms":10000,"last_server_duration_us":177888,"total_duration_us":10010838},{"operation_name":"upsert","last_dispatch_duration_us":1263025,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1874637,"total_server_duration_us":342019,"encode_duration_us":16363,"operation_id":"0x5bfc","timeout_ms":10000,"last_server_duration_us":111491,"total_duration_us":10010623},{"operation_name":"upsert","last_dispatch_duration_us":849204,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":849204,"total_server_duration_us":123884,"encode_duration_us":21242,"operation_id":"0x5cbc","timeout_ms":10000,"last_server_duration_us":123884,"total_duration_us":10009877},{"operation_name":"upsert","last_dispatch_duration_us":271058,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":2479460,"total_server_duration_us":1044550,"encode_duration_us":16275,"operation_id":"0x5c94","timeout_ms":10000,"last_server_duration_us":97793,"total_duration_us":10008224},{"operation_name":"upsert","last_dispatch_duration_us":388019,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":2624414,"total_server_duration_us":561852,"encode_duration_us":17188,"operation_id":"0x5ca8","timeout_ms":10000,"last_server_duration_us":99502,"total_duration_us":10006908},{"operation_name":"upsert","last_dispatch_duration_us":557101,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2448020,"total_server_duration_us":897172,"encode_duration_us":17686,"operation_id":"0x5c87","timeout_ms":10000,"last_server_duration_us":118961,"total_duration_us":10004154},{"operation_name":"upsert","last_dispatch_duration_us":1905429,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3593985,"total_server_duration_us":291287,"encode_duration_us":16800,"operation_id":"0x5c7e","timeout_ms":10000,"last_server_duration_us":58999,"total_duration_us":10004084},{"operation_name":"upsert","last_dispatch_duration_us":696930,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10081516,"total_server_duration_us":102816,"encode_duration_us":17227,"operation_id":"0x5adf","timeout_ms":10000,"last_server_duration_us":38482,"total_duration_us":10003468},{"operation_name":"upsert","last_dispatch_duration_us":9420877,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10522731,"total_server_duration_us":9192534,"encode_duration_us":18554,"operation_id":"0x5b39","timeout_ms":10000,"last_server_duration_us":9148034,"total_duration_us":10003016},{"operation_name":"upsert","last_dispatch_duration_us":9753157,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14940033,"total_server_duration_us":9180228,"encode_duration_us":15546,"operation_id":"0x5ba5","timeout_ms":10000,"last_server_duration_us":9091564,"total_duration_us":10001896}],"total_count":22}} INFO [2022-01-10 05:44:10,384] jepsen worker 45 - jepsen.util 3195 :ok :read [15 23] INFO [2022-01-10 05:44:10,384] jepsen worker 45 - jepsen.util 3195 :invoke :read [15 nil] INFO [2022-01-10 05:44:10,506] jepsen worker 29 - jepsen.util 2729 :info :write [9 0] :RequestTimeoutException INFO [2022-01-10 05:44:10,507] jepsen worker 29 - jepsen.util 2819 :invoke :read [9 nil] INFO [2022-01-10 05:44:10,577] jepsen worker 50 - jepsen.util 3020 :info :write [16 31] :RequestTimeoutException INFO [2022-01-10 05:44:10,577] jepsen worker 50 - jepsen.util 3110 :invoke :read [16 nil] INFO [2022-01-10 05:44:10,588] jepsen worker 19 - jepsen.util 3079 :ok :read [6 25] INFO [2022-01-10 05:44:10,589] jepsen worker 19 - jepsen.util 3079 :invoke :read [6 nil] INFO [2022-01-10 05:44:10,771] jepsen worker 33 - jepsen.util 3273 :ok :read [11 21] INFO [2022-01-10 05:44:10,772] jepsen worker 33 - jepsen.util 3273 :invoke :read [11 nil] INFO [2022-01-10 05:44:10,801] jepsen worker 28 - jepsen.util 2638 :info :write [9 30] :RequestTimeoutException INFO [2022-01-10 05:44:10,801] jepsen worker 28 - jepsen.util 2728 :invoke :read [9 nil] INFO [2022-01-10 05:44:10,811] jepsen worker 40 - jepsen.util 2290 :info :write [13 35] :RequestTimeoutException INFO [2022-01-10 05:44:10,812] jepsen worker 40 - jepsen.util 2380 :invoke :write [13 40] INFO [2022-01-10 05:44:10,979] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3343108504398827}, :ns_1@172.23.105.187 {:progress 0.3343108504398827}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:10,979] jepsen worker nemesis - couchbase.util Rebalance check count: 402 INFO [2022-01-10 05:44:10,979] jepsen worker nemesis - couchbase.util Rebalance stuck count: 317 INFO [2022-01-10 05:44:10,984] jepsen worker 32 - jepsen.util 3182 :info :write [10 19] :SyncWriteAmbiguous INFO [2022-01-10 05:44:10,985] jepsen worker 32 - jepsen.util 3272 :invoke :read [10 nil] INFO [2022-01-10 05:44:11,120] jepsen worker 79 - jepsen.util 3409 :ok :read [26 10] INFO [2022-01-10 05:44:11,121] jepsen worker 79 - jepsen.util 3409 :invoke :write [26 23] INFO [2022-01-10 05:44:11,205] jepsen worker 13 - jepsen.util 3253 :info :write [4 7] :RequestTimeoutException INFO [2022-01-10 05:44:11,206] jepsen worker 13 - jepsen.util 3343 :invoke :read [4 nil] INFO [2022-01-10 05:44:11,214] jepsen worker 53 - jepsen.util 3473 :info :write [17 7] :RequestTimeoutException INFO [2022-01-10 05:44:11,215] jepsen worker 53 - jepsen.util 3563 :invoke :read [17 nil] INFO [2022-01-10 05:44:11,281] jepsen worker 13 - jepsen.util 3343 :ok :read [4 3] INFO [2022-01-10 05:44:11,281] jepsen worker 13 - jepsen.util 3343 :invoke :read [4 nil] INFO [2022-01-10 05:44:11,294] jepsen worker 39 - jepsen.util 3459 :ok :read [13 15] INFO [2022-01-10 05:44:11,295] jepsen worker 39 - jepsen.util 3459 :invoke :write [13 49] INFO [2022-01-10 05:44:11,347] jepsen worker 13 - jepsen.util 3343 :ok :read [4 3] INFO [2022-01-10 05:44:11,347] jepsen worker 13 - jepsen.util 3343 :invoke :read [4 nil] INFO [2022-01-10 05:44:11,398] jepsen worker 13 - jepsen.util 3343 :ok :read [4 3] INFO [2022-01-10 05:44:11,398] jepsen worker 13 - jepsen.util 3343 :invoke :write [4 20] INFO [2022-01-10 05:44:11,490] jepsen worker 83 - jepsen.util 3413 :info :write [27 31] :RequestTimeoutException INFO [2022-01-10 05:44:11,490] jepsen worker 83 - jepsen.util 3503 :invoke :write [27 27] INFO [2022-01-10 05:44:11,502] jepsen worker 84 - jepsen.util 2874 :info :write [28 0] :RequestTimeoutException INFO [2022-01-10 05:44:11,502] jepsen worker 84 - jepsen.util 2964 :invoke :write [28 24] INFO [2022-01-10 05:44:11,529] jepsen worker 53 - jepsen.util 3563 :ok :read [17 41] INFO [2022-01-10 05:44:11,530] jepsen worker 53 - jepsen.util 3563 :invoke :read [17 nil] INFO [2022-01-10 05:44:11,552] jepsen worker 75 - jepsen.util 2325 :info :write [25 18] :RequestTimeoutException INFO [2022-01-10 05:44:11,552] jepsen worker 75 - jepsen.util 2415 :invoke :read [25 nil] INFO [2022-01-10 05:44:11,767] jepsen worker 60 - jepsen.util 3300 :info :write [20 44] :RequestTimeoutException WARN [2022-01-10 05:44:11,783] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12462995,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12462995,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9204652,"encode_duration_us":16511,"operation_id":"0x5c2a","timeout_ms":10000,"last_server_duration_us":9204652,"total_duration_us":10008421}],"total_count":1}} WARN [2022-01-10 05:44:11,783] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":27114637,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":27114637,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9032060,"encode_duration_us":17736,"operation_id":"0x58a2","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10004948}],"total_count":1}} INFO [2022-01-10 05:44:11,878] jepsen worker 60 - jepsen.util 3390 :invoke :read [20 nil] WARN [2022-01-10 05:44:11,878] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18159069,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":18159069,"total_server_duration_us":43,"operation_id":"0x5a30","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":18467441},{"operation_name":"get","last_dispatch_duration_us":40547,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":40547,"total_server_duration_us":43,"operation_id":"0x5a8a","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":17488563},{"operation_name":"get","last_dispatch_duration_us":9155408,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9155408,"total_server_duration_us":49,"operation_id":"0x5aeb","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":15343258},{"operation_name":"get","last_dispatch_duration_us":42557,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":42557,"total_server_duration_us":43,"operation_id":"0x5b6f","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":12862000},{"operation_name":"upsert","last_dispatch_duration_us":3497256,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7896038,"total_server_duration_us":666863,"encode_duration_us":19953,"operation_id":"0x5cbd","timeout_ms":10000,"last_server_duration_us":171940,"total_duration_us":10009963},{"operation_name":"upsert","last_dispatch_duration_us":1599669,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10206446,"total_server_duration_us":721459,"encode_duration_us":16612,"operation_id":"0x5be4","timeout_ms":10000,"last_server_duration_us":79687,"total_duration_us":10009573},{"operation_name":"upsert","last_dispatch_duration_us":106091,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":106091,"total_server_duration_us":19574,"encode_duration_us":19278,"operation_id":"0x5c57","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10009204},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":16863,"operation_id":"0x5c1e","timeout_ms":10000,"total_duration_us":10009031},{"operation_name":"upsert","last_dispatch_duration_us":1405112,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":4403807,"total_server_duration_us":1953744,"encode_duration_us":16682,"operation_id":"0x5ce9","timeout_ms":10000,"last_server_duration_us":892708,"total_duration_us":10008734},{"operation_name":"upsert","last_dispatch_duration_us":81986,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":2787425,"total_server_duration_us":877619,"encode_duration_us":17351,"operation_id":"0x5c83","timeout_ms":10000,"last_server_duration_us":48449,"total_duration_us":10005721}],"total_count":18}} INFO [2022-01-10 05:44:11,880] jepsen worker 24 - jepsen.util 2544 :info :write [8 14] :RequestTimeoutException INFO [2022-01-10 05:44:11,881] jepsen worker 24 - jepsen.util 2634 :invoke :read [8 nil] WARN [2022-01-10 05:44:11,882] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":42764,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9425991,"total_server_duration_us":202049,"encode_duration_us":16722,"operation_id":"0x5bd0","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10013918},{"operation_name":"upsert","last_dispatch_duration_us":4145015,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12936623,"total_server_duration_us":391981,"encode_duration_us":16762,"operation_id":"0x5c4b","timeout_ms":10000,"last_server_duration_us":53374,"total_duration_us":10006480},{"operation_name":"upsert","last_dispatch_duration_us":844567,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8251142,"total_server_duration_us":1053157,"encode_duration_us":17707,"operation_id":"0x5cfd","timeout_ms":10000,"last_server_duration_us":191687,"total_duration_us":10006269},{"operation_name":"upsert","last_dispatch_duration_us":2068347,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11883457,"total_server_duration_us":479374,"encode_duration_us":18614,"operation_id":"0x5ba4","timeout_ms":10000,"last_server_duration_us":86182,"total_duration_us":10006097},{"operation_name":"upsert","last_dispatch_duration_us":3844573,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8040882,"total_server_duration_us":398815,"encode_duration_us":19052,"operation_id":"0x5bff","timeout_ms":10000,"last_server_duration_us":109859,"total_duration_us":10005542},{"operation_name":"upsert","last_dispatch_duration_us":3703533,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7034113,"total_server_duration_us":2090198,"encode_duration_us":18091,"operation_id":"0x5cfe","timeout_ms":10000,"last_server_duration_us":1891679,"total_duration_us":10003411},{"operation_name":"upsert","last_dispatch_duration_us":1479890,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9988159,"total_server_duration_us":312973,"encode_duration_us":19336,"operation_id":"0x5b93","timeout_ms":10000,"last_server_duration_us":121667,"total_duration_us":10002767},{"operation_name":"upsert","last_dispatch_duration_us":2233761,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6756708,"total_server_duration_us":488938,"encode_duration_us":16919,"operation_id":"0x5c81","timeout_ms":10000,"last_server_duration_us":78837,"total_duration_us":10001315},{"operation_name":"upsert","last_dispatch_duration_us":9274236,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9274236,"total_server_duration_us":9210001,"encode_duration_us":16602,"operation_id":"0x5b6c","timeout_ms":10000,"last_server_duration_us":9210001,"total_duration_us":9278006},{"operation_name":"get","last_dispatch_duration_us":6357703,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":6357703,"total_server_duration_us":43,"operation_id":"0x5c4c","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":6359042}],"total_count":17}} WARN [2022-01-10 05:44:11,882] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9461345,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9735217,"total_server_duration_us":293496,"encode_duration_us":16910,"operation_id":"0x5cce","timeout_ms":10000,"last_server_duration_us":223881,"total_duration_us":10011141},{"operation_name":"upsert","last_dispatch_duration_us":2389292,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7867815,"total_server_duration_us":771842,"encode_duration_us":16965,"operation_id":"0x5d11","timeout_ms":10000,"last_server_duration_us":108076,"total_duration_us":10009060},{"operation_name":"upsert","last_dispatch_duration_us":2280528,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7918286,"total_server_duration_us":1092387,"encode_duration_us":16632,"operation_id":"0x5c8b","timeout_ms":10000,"last_server_duration_us":73410,"total_duration_us":10008932},{"operation_name":"upsert","last_dispatch_duration_us":494791,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":1361434,"total_server_duration_us":383388,"encode_duration_us":17453,"operation_id":"0x5cb8","timeout_ms":10000,"last_server_duration_us":194997,"total_duration_us":10007246},{"operation_name":"upsert","last_dispatch_duration_us":62788,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8950536,"total_server_duration_us":244799,"encode_duration_us":18984,"operation_id":"0x5c4d","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10004425},{"operation_name":"upsert","last_dispatch_duration_us":2433606,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10126554,"total_server_duration_us":1135705,"encode_duration_us":16784,"operation_id":"0x5c7d","timeout_ms":10000,"last_server_duration_us":282993,"total_duration_us":10004093},{"operation_name":"upsert","last_dispatch_duration_us":44600,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8927497,"total_server_duration_us":239585,"encode_duration_us":19423,"operation_id":"0x5c56","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10003776},{"operation_name":"upsert","last_dispatch_duration_us":832896,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8559911,"total_server_duration_us":1663234,"encode_duration_us":20546,"operation_id":"0x5be5","timeout_ms":10000,"last_server_duration_us":88098,"total_duration_us":10002893},{"operation_name":"upsert","last_dispatch_duration_us":9801354,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10300463,"total_server_duration_us":310133,"encode_duration_us":17554,"operation_id":"0x5ccc","timeout_ms":10000,"last_server_duration_us":132042,"total_duration_us":10002862},{"operation_name":"upsert","last_dispatch_duration_us":2142074,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8135655,"total_server_duration_us":1120047,"encode_duration_us":16654,"operation_id":"0x5c8f","timeout_ms":10000,"last_server_duration_us":97020,"total_duration_us":10002318}],"total_count":12}} INFO [2022-01-10 05:44:11,891] jepsen worker 35 - jepsen.util 2465 :info :write [11 8] :RequestTimeoutException INFO [2022-01-10 05:44:11,891] jepsen worker 35 - jepsen.util 2555 :invoke :read [11 nil] INFO [2022-01-10 05:44:11,923] jepsen worker 19 - jepsen.util 3079 :ok :read [6 25] INFO [2022-01-10 05:44:11,923] jepsen worker 19 - jepsen.util 3079 :invoke :write [6 7] INFO [2022-01-10 05:44:11,976] jepsen worker 30 - jepsen.util 2730 :info :write [10 12] :RequestTimeoutException INFO [2022-01-10 05:44:11,977] jepsen worker 30 - jepsen.util 2820 :invoke :write [10 33] WARN [2022-01-10 05:44:11,978] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2711886,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5091346,"total_server_duration_us":66951,"encode_duration_us":19207,"operation_id":"0x5bc5","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10009710},{"operation_name":"upsert","last_dispatch_duration_us":500486,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10032126,"total_server_duration_us":588897,"encode_duration_us":17588,"operation_id":"0x5bfd","timeout_ms":10000,"last_server_duration_us":182708,"total_duration_us":10009209},{"operation_name":"upsert","last_dispatch_duration_us":1946409,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1946409,"total_server_duration_us":67882,"encode_duration_us":17296,"operation_id":"0x5cc5","timeout_ms":10000,"last_server_duration_us":67882,"total_duration_us":10008713},{"operation_name":"upsert","last_dispatch_duration_us":12462995,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12462995,"total_server_duration_us":9204652,"encode_duration_us":16511,"operation_id":"0x5c2a","timeout_ms":10000,"last_server_duration_us":9204652,"total_duration_us":10008421},{"operation_name":"upsert","last_dispatch_duration_us":1494868,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":4187211,"total_server_duration_us":1862813,"encode_duration_us":31296,"operation_id":"0x5cf8","timeout_ms":10000,"last_server_duration_us":88543,"total_duration_us":10006803},{"operation_name":"upsert","last_dispatch_duration_us":890093,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10270980,"total_server_duration_us":553290,"encode_duration_us":22630,"operation_id":"0x5b94","timeout_ms":10000,"last_server_duration_us":258175,"total_duration_us":10006570},{"operation_name":"upsert","last_dispatch_duration_us":1680803,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1680803,"total_server_duration_us":51003,"encode_duration_us":16207,"operation_id":"0x5c5a","timeout_ms":10000,"last_server_duration_us":51003,"total_duration_us":10006218},{"operation_name":"upsert","last_dispatch_duration_us":892026,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10229201,"total_server_duration_us":2009208,"encode_duration_us":17003,"operation_id":"0x5c7f","timeout_ms":10000,"last_server_duration_us":184530,"total_duration_us":10005419},{"operation_name":"upsert","last_dispatch_duration_us":8780264,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9437028,"total_server_duration_us":246841,"encode_duration_us":16312,"operation_id":"0x5b90","timeout_ms":10000,"last_server_duration_us":58501,"total_duration_us":10004157},{"operation_name":"upsert","last_dispatch_duration_us":742709,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10017870,"total_server_duration_us":430384,"encode_duration_us":20644,"operation_id":"0x5b97","timeout_ms":10000,"last_server_duration_us":185545,"total_duration_us":10004144}],"total_count":18}} WARN [2022-01-10 05:44:11,982] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":27144645,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":27144645,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9046921,"encode_duration_us":16522,"operation_id":"0x5874","timeout_ms":10000,"last_server_duration_us":9046921,"total_duration_us":10001524},{"operation_name":"upsert","last_dispatch_duration_us":9730270,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":12305854,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":9413897,"encode_duration_us":20173,"operation_id":"0x5b63","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10001605},{"operation_name":"upsert","last_dispatch_duration_us":9794899,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10911185,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9509265,"encode_duration_us":18198,"operation_id":"0x5c20","timeout_ms":10000,"last_server_duration_us":9183269,"total_duration_us":10007580},{"operation_name":"upsert","last_dispatch_duration_us":11906158,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11906158,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":10849761,"encode_duration_us":17730,"operation_id":"0x5c2b","timeout_ms":10000,"last_server_duration_us":10849761,"total_duration_us":10002860}],"total_count":4}} WARN [2022-01-10 05:44:11,982] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18107415,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18107415,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9025695,"encode_duration_us":17861,"operation_id":"0x5972","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10006306}],"total_count":1}} INFO [2022-01-10 05:44:11,986] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3343108504398827}, :ns_1@172.23.105.187 {:progress 0.3343108504398827}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:11,986] jepsen worker nemesis - couchbase.util Rebalance check count: 403 INFO [2022-01-10 05:44:11,986] jepsen worker nemesis - couchbase.util Rebalance stuck count: 318 INFO [2022-01-10 05:44:12,015] jepsen worker 26 - jepsen.util 2456 :info :write [8 47] :RequestTimeoutException INFO [2022-01-10 05:44:12,016] jepsen worker 26 - jepsen.util 2546 :invoke :write [8 24] INFO [2022-01-10 05:44:12,020] jepsen worker 82 - jepsen.util 3052 :ok :read [27 30] INFO [2022-01-10 05:44:12,021] jepsen worker 82 - jepsen.util 3052 :invoke :write [27 37] INFO [2022-01-10 05:44:12,022] jepsen worker 49 - jepsen.util 2929 :ok :read [16 38] INFO [2022-01-10 05:44:12,023] jepsen worker 49 - jepsen.util 2929 :invoke :read [16 nil] WARN [2022-01-10 05:44:12,083] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9411825,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9976639,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9286868,"encode_duration_us":19920,"operation_id":"0x5b12","timeout_ms":10000,"last_server_duration_us":9261418,"total_duration_us":10005284},{"operation_name":"upsert","last_dispatch_duration_us":9232951,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":15412230,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":11260996,"encode_duration_us":16956,"operation_id":"0x59f6","timeout_ms":10000,"last_server_duration_us":9063917,"total_duration_us":10009336},{"operation_name":"upsert","last_dispatch_duration_us":9073525,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17343571,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9121065,"encode_duration_us":16884,"operation_id":"0x5acf","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10005701}],"total_count":3}} INFO [2022-01-10 05:44:12,181] jepsen worker 45 - jepsen.util 3195 :ok :read [15 23] INFO [2022-01-10 05:44:12,182] jepsen worker 45 - jepsen.util 3195 :invoke :read [15 nil] WARN [2022-01-10 05:44:12,183] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":26320134,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":26320134,"total_server_duration_us":32,"operation_id":"0x58a7","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":26321211},{"operation_name":"get","last_dispatch_duration_us":10906401,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10906401,"total_server_duration_us":14573,"operation_id":"0x5c80","timeout_ms":30000,"last_server_duration_us":14573,"total_duration_us":10909670},{"operation_name":"upsert","last_dispatch_duration_us":9297264,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10480806,"total_server_duration_us":586765,"encode_duration_us":16530,"operation_id":"0x5bc8","timeout_ms":10000,"last_server_duration_us":103434,"total_duration_us":10010477},{"operation_name":"upsert","last_dispatch_duration_us":89101,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9807657,"total_server_duration_us":689320,"encode_duration_us":18964,"operation_id":"0x5c5b","timeout_ms":10000,"last_server_duration_us":70018,"total_duration_us":10009243},{"operation_name":"upsert","last_dispatch_duration_us":9794899,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10911185,"total_server_duration_us":9509265,"encode_duration_us":18198,"operation_id":"0x5c20","timeout_ms":10000,"last_server_duration_us":9183269,"total_duration_us":10007580},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":16793,"operation_id":"0x5bfb","timeout_ms":10000,"total_duration_us":10007186},{"operation_name":"upsert","last_dispatch_duration_us":212326,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9544400,"total_server_duration_us":791264,"encode_duration_us":19198,"operation_id":"0x5c8d","timeout_ms":10000,"last_server_duration_us":75201,"total_duration_us":10007094},{"operation_name":"upsert","last_dispatch_duration_us":1017081,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9185562,"total_server_duration_us":349737,"encode_duration_us":21259,"operation_id":"0x5cfc","timeout_ms":10000,"last_server_duration_us":128891,"total_duration_us":10007068},{"operation_name":"upsert","last_dispatch_duration_us":11002839,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":17438822,"total_server_duration_us":1364981,"encode_duration_us":16562,"operation_id":"0x5b40","timeout_ms":10000,"last_server_duration_us":128022,"total_duration_us":10006088},{"operation_name":"upsert","last_dispatch_duration_us":10934626,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":15058469,"total_server_duration_us":423615,"encode_duration_us":19502,"operation_id":"0x5bb8","timeout_ms":10000,"last_server_duration_us":84283,"total_duration_us":10004852}],"total_count":16}} INFO [2022-01-10 05:44:12,196] jepsen worker 21 - jepsen.util 3081 :info :write [7 2] :RequestTimeoutException INFO [2022-01-10 05:44:12,197] jepsen worker 21 - jepsen.util 3171 :invoke :read [7 nil] INFO [2022-01-10 05:44:12,249] jepsen worker 44 - jepsen.util 2924 :info :write [14 13] :RequestTimeoutException INFO [2022-01-10 05:44:12,250] jepsen worker 44 - jepsen.util 3014 :invoke :read [14 nil] INFO [2022-01-10 05:44:12,263] jepsen worker 35 - jepsen.util 2555 :ok :read [11 21] INFO [2022-01-10 05:44:12,264] jepsen worker 35 - jepsen.util 2555 :invoke :read [11 nil] INFO [2022-01-10 05:44:12,288] jepsen worker 33 - jepsen.util 3273 :ok :read [11 21] INFO [2022-01-10 05:44:12,288] jepsen worker 33 - jepsen.util 3273 :invoke :read [11 nil] INFO [2022-01-10 05:44:12,557] jepsen worker 64 - jepsen.util 2944 :ok :read [21 7] INFO [2022-01-10 05:44:12,558] jepsen worker 64 - jepsen.util 2944 :invoke :read [21 nil] INFO [2022-01-10 05:44:12,603] jepsen worker 35 - jepsen.util 2555 :ok :read [11 21] INFO [2022-01-10 05:44:12,603] jepsen worker 35 - jepsen.util 2555 :invoke :read [11 nil] INFO [2022-01-10 05:44:12,612] jepsen worker 60 - jepsen.util 3390 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:44:12,613] jepsen worker 60 - jepsen.util 3390 :invoke :write [20 14] INFO [2022-01-10 05:44:12,641] jepsen worker 1 - jepsen.util 2971 :ok :read [0 46] INFO [2022-01-10 05:44:12,642] jepsen worker 1 - jepsen.util 2971 :invoke :write [0 47] INFO [2022-01-10 05:44:12,696] jepsen worker 44 - jepsen.util 3014 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:44:12,697] jepsen worker 44 - jepsen.util 3014 :invoke :read [14 nil] INFO [2022-01-10 05:44:12,709] jepsen worker 45 - jepsen.util 3195 :ok :read [15 23] INFO [2022-01-10 05:44:12,710] jepsen worker 45 - jepsen.util 3195 :invoke :write [15 18] INFO [2022-01-10 05:44:12,719] jepsen worker 33 - jepsen.util 3273 :ok :read [11 21] INFO [2022-01-10 05:44:12,720] jepsen worker 33 - jepsen.util 3273 :invoke :read [11 nil] INFO [2022-01-10 05:44:12,991] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3343108504398827}, :ns_1@172.23.105.187 {:progress 0.3343108504398827}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:12,992] jepsen worker nemesis - couchbase.util Rebalance check count: 404 INFO [2022-01-10 05:44:12,992] jepsen worker nemesis - couchbase.util Rebalance stuck count: 319 INFO [2022-01-10 05:44:13,007] jepsen worker 14 - jepsen.util 3524 :info :write [4 17] :RequestTimeoutException INFO [2022-01-10 05:44:13,007] jepsen worker 14 - jepsen.util 3614 :invoke :read [4 nil] INFO [2022-01-10 05:44:13,191] jepsen worker 27 - jepsen.util 2907 :info :write [9 42] :RequestTimeoutException INFO [2022-01-10 05:44:13,193] jepsen worker 27 - jepsen.util 2997 :invoke :write [9 22] INFO [2022-01-10 05:44:13,233] jepsen worker 53 - jepsen.util 3563 :ok :read [17 41] INFO [2022-01-10 05:44:13,234] jepsen worker 53 - jepsen.util 3563 :invoke :write [17 41] INFO [2022-01-10 05:44:13,305] jepsen worker 2 - jepsen.util 3062 :info :write [0 37] :RequestTimeoutException INFO [2022-01-10 05:44:13,306] jepsen worker 2 - jepsen.util 3152 :invoke :write [0 7] INFO [2022-01-10 05:44:13,403] jepsen worker 14 - jepsen.util 3614 :ok :read [4 3] INFO [2022-01-10 05:44:13,403] jepsen worker 14 - jepsen.util 3614 :invoke :read [4 nil] INFO [2022-01-10 05:44:13,438] jepsen worker 81 - jepsen.util 3231 :ok :read [27 30] INFO [2022-01-10 05:44:13,439] jepsen worker 81 - jepsen.util 3231 :invoke :read [27 nil] INFO [2022-01-10 05:44:13,448] jepsen worker 28 - jepsen.util 2728 :ok :read [9 24] INFO [2022-01-10 05:44:13,449] jepsen worker 28 - jepsen.util 2728 :invoke :write [9 27] INFO [2022-01-10 05:44:13,466] jepsen worker 35 - jepsen.util 2555 :ok :read [11 21] INFO [2022-01-10 05:44:13,466] jepsen worker 35 - jepsen.util 2555 :invoke :read [11 nil] INFO [2022-01-10 05:44:13,566] jepsen worker 35 - jepsen.util 2555 :ok :read [11 21] INFO [2022-01-10 05:44:13,567] jepsen worker 35 - jepsen.util 2555 :invoke :write [11 28] INFO [2022-01-10 05:44:13,585] jepsen worker 14 - jepsen.util 3614 :ok :read [4 3] INFO [2022-01-10 05:44:13,586] jepsen worker 14 - jepsen.util 3614 :invoke :read [4 nil] INFO [2022-01-10 05:44:13,618] jepsen worker 44 - jepsen.util 3014 :ok :read [14 14] INFO [2022-01-10 05:44:13,618] jepsen worker 44 - jepsen.util 3014 :invoke :write [14 30] INFO [2022-01-10 05:44:13,639] jepsen worker 71 - jepsen.util 2411 :info :write [23 46] :SyncWriteAmbiguous INFO [2022-01-10 05:44:13,640] jepsen worker 71 - jepsen.util 2501 :invoke :read [23 nil] INFO [2022-01-10 05:44:13,682] jepsen worker 75 - jepsen.util 2415 :ok :read [25 :nil] INFO [2022-01-10 05:44:13,683] jepsen worker 75 - jepsen.util 2415 :invoke :write [25 30] INFO [2022-01-10 05:44:13,694] jepsen worker 42 - jepsen.util 2922 :ok :read [14 14] INFO [2022-01-10 05:44:13,695] jepsen worker 42 - jepsen.util 2922 :invoke :write [14 30] INFO [2022-01-10 05:44:13,702] jepsen worker 14 - jepsen.util 3614 :ok :read [4 3] INFO [2022-01-10 05:44:13,702] jepsen worker 14 - jepsen.util 3614 :invoke :write [4 5] INFO [2022-01-10 05:44:13,831] jepsen worker 41 - jepsen.util 2831 :info :write [13 44] :RequestTimeoutException INFO [2022-01-10 05:44:13,832] jepsen worker 41 - jepsen.util 2921 :invoke :read [13 nil] INFO [2022-01-10 05:44:13,998] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:13,999] jepsen worker nemesis - couchbase.util Rebalance check count: 405 INFO [2022-01-10 05:44:13,999] jepsen worker nemesis - couchbase.util Rebalance stuck count: 319 INFO [2022-01-10 05:44:14,055] jepsen worker 6 - jepsen.util 2976 :info :write [2 36] :RequestTimeoutException INFO [2022-01-10 05:44:14,055] jepsen worker 6 - jepsen.util 3066 :invoke :read [2 nil] INFO [2022-01-10 05:44:14,072] jepsen worker 85 - jepsen.util 3145 :info :write [28 1] :RequestTimeoutException INFO [2022-01-10 05:44:14,073] jepsen worker 85 - jepsen.util 3235 :invoke :write [28 3] INFO [2022-01-10 05:44:14,081] jepsen worker 17 - jepsen.util 2897 :info :write [5 5] :RequestTimeoutException INFO [2022-01-10 05:44:14,082] jepsen worker 17 - jepsen.util 2987 :invoke :read [5 nil] INFO [2022-01-10 05:44:14,106] jepsen worker 0 - jepsen.util 2880 :info :write [0 30] :RequestTimeoutException INFO [2022-01-10 05:44:14,106] jepsen worker 0 - jepsen.util 2970 :invoke :read [0 nil] INFO [2022-01-10 05:44:14,349] jepsen worker 69 - jepsen.util 2769 :ok :read [23 26] INFO [2022-01-10 05:44:14,351] jepsen worker 69 - jepsen.util 2769 :invoke :read [23 nil] INFO [2022-01-10 05:44:14,351] jepsen worker 55 - jepsen.util 3205 :info :write [18 25] :RequestTimeoutException INFO [2022-01-10 05:44:14,352] jepsen worker 55 - jepsen.util 3295 :invoke :write [18 2] INFO [2022-01-10 05:44:14,362] jepsen worker 36 - jepsen.util 2466 :ok :read [12 29] INFO [2022-01-10 05:44:14,363] jepsen worker 77 - jepsen.util 3407 :ok :read [25 :nil] INFO [2022-01-10 05:44:14,363] jepsen worker 36 - jepsen.util 2466 :invoke :write [12 13] INFO [2022-01-10 05:44:14,363] jepsen worker 77 - jepsen.util 3407 :invoke :read [25 nil] INFO [2022-01-10 05:44:14,365] jepsen worker 21 - jepsen.util 3171 :ok :read [7 :nil] INFO [2022-01-10 05:44:14,366] jepsen worker 21 - jepsen.util 3171 :invoke :write [7 11] INFO [2022-01-10 05:44:14,398] jepsen worker 70 - jepsen.util 3040 :ok :read [23 26] INFO [2022-01-10 05:44:14,400] jepsen worker 70 - jepsen.util 3040 :invoke :write [23 26] INFO [2022-01-10 05:44:14,420] jepsen worker 8 - jepsen.util 3428 :ok :read [2 14] INFO [2022-01-10 05:44:14,421] jepsen worker 8 - jepsen.util 3428 :invoke :write [2 8] INFO [2022-01-10 05:44:14,432] jepsen worker 74 - jepsen.util 3584 :ok :read [24 18] INFO [2022-01-10 05:44:14,432] jepsen worker 64 - jepsen.util 2944 :ok :read [21 7] INFO [2022-01-10 05:44:14,433] jepsen worker 74 - jepsen.util 3584 :invoke :write [24 15] INFO [2022-01-10 05:44:14,433] jepsen worker 5 - jepsen.util 2885 :ok :read [1 10] INFO [2022-01-10 05:44:14,433] jepsen worker 64 - jepsen.util 2944 :invoke :write [21 5] INFO [2022-01-10 05:44:14,433] jepsen worker 5 - jepsen.util 2885 :invoke :write [1 12] INFO [2022-01-10 05:44:14,585] jepsen worker 72 - jepsen.util 3222 :info :write [24 15] :RequestTimeoutException INFO [2022-01-10 05:44:14,586] jepsen worker 72 - jepsen.util 3312 :invoke :write [24 35] INFO [2022-01-10 05:44:14,646] jepsen worker 67 - jepsen.util 2767 :info :write [22 4] :RequestTimeoutException INFO [2022-01-10 05:44:14,646] jepsen worker 67 - jepsen.util 2857 :invoke :write [22 34] INFO [2022-01-10 05:44:14,746] jepsen worker 66 - jepsen.util 2586 :info :write [22 32] :RequestTimeoutException INFO [2022-01-10 05:44:14,747] jepsen worker 66 - jepsen.util 2676 :invoke :write [22 3] INFO [2022-01-10 05:44:14,800] jepsen worker 22 - jepsen.util 2632 :info :write [7 19] :RequestTimeoutException INFO [2022-01-10 05:44:14,800] jepsen worker 22 - jepsen.util 2722 :invoke :write [7 40] INFO [2022-01-10 05:44:14,852] jepsen worker 61 - jepsen.util 3031 :info :write [20 24] :RequestTimeoutException INFO [2022-01-10 05:44:14,853] jepsen worker 61 - jepsen.util 3121 :invoke :read [20 nil] INFO [2022-01-10 05:44:14,941] jepsen worker 68 - jepsen.util 2948 :info :write [22 22] :RequestTimeoutException INFO [2022-01-10 05:44:14,942] jepsen worker 68 - jepsen.util 3038 :invoke :read [22 nil] INFO [2022-01-10 05:44:15,005] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:15,005] jepsen worker nemesis - couchbase.util Rebalance check count: 406 INFO [2022-01-10 05:44:15,005] jepsen worker nemesis - couchbase.util Rebalance stuck count: 320 INFO [2022-01-10 05:44:15,006] jepsen worker 25 - jepsen.util 2815 :info :write [8 33] :RequestTimeoutException INFO [2022-01-10 05:44:15,006] jepsen worker 25 - jepsen.util 2905 :invoke :read [8 nil] INFO [2022-01-10 05:44:15,320] jepsen worker 12 - jepsen.util 3162 :info :write [4 8] :RequestTimeoutException INFO [2022-01-10 05:44:15,321] jepsen worker 87 - jepsen.util 2967 :info :write [29 0] :RequestTimeoutException INFO [2022-01-10 05:44:15,321] jepsen worker 12 - jepsen.util 3252 :invoke :read [4 nil] INFO [2022-01-10 05:44:15,321] jepsen worker 87 - jepsen.util 3057 :invoke :read [29 nil] INFO [2022-01-10 05:44:15,343] jepsen worker 87 - jepsen.util 3057 :ok :read [29 0] INFO [2022-01-10 05:44:15,344] jepsen worker 87 - jepsen.util 3057 :invoke :write [29 7] INFO [2022-01-10 05:44:15,471] jepsen worker 37 - jepsen.util 2827 :info :write [12 24] :SyncWriteAmbiguous INFO [2022-01-10 05:44:15,472] jepsen worker 37 - jepsen.util 2917 :invoke :read [12 nil] INFO [2022-01-10 05:44:15,630] jepsen worker 10 - jepsen.util 3250 :ok :read [3 31] INFO [2022-01-10 05:44:15,631] jepsen worker 10 - jepsen.util 3250 :invoke :write [3 27] INFO [2022-01-10 05:44:15,725] jepsen worker 12 - jepsen.util 3252 :ok :read [4 3] INFO [2022-01-10 05:44:15,725] jepsen worker 12 - jepsen.util 3252 :invoke :write [4 31] INFO [2022-01-10 05:44:15,790] jepsen worker 62 - jepsen.util 3212 :info :write [20 43] :RequestTimeoutException INFO [2022-01-10 05:44:15,791] jepsen worker 62 - jepsen.util 3302 :invoke :write [20 36] INFO [2022-01-10 05:44:16,011] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:16,011] jepsen worker nemesis - couchbase.util Rebalance check count: 407 INFO [2022-01-10 05:44:16,011] jepsen worker nemesis - couchbase.util Rebalance stuck count: 321 INFO [2022-01-10 05:44:16,091] jepsen worker 9 - jepsen.util 3429 :info :write [3 0] :RequestTimeoutException INFO [2022-01-10 05:44:16,092] jepsen worker 9 - jepsen.util 3519 :invoke :write [3 34] INFO [2022-01-10 05:44:16,231] jepsen worker 31 - jepsen.util 3271 :info :write [10 20] :RequestTimeoutException INFO [2022-01-10 05:44:16,232] jepsen worker 31 - jepsen.util 3361 :invoke :write [10 30] INFO [2022-01-10 05:44:16,272] jepsen worker 51 - jepsen.util 3471 :info :write [17 13] :RequestTimeoutException INFO [2022-01-10 05:44:16,273] jepsen worker 51 - jepsen.util 3561 :invoke :write [17 22] INFO [2022-01-10 05:44:16,287] jepsen worker 38 - jepsen.util 3188 :info :write [12 19] :RequestTimeoutException INFO [2022-01-10 05:44:16,287] jepsen worker 38 - jepsen.util 3278 :invoke :write [12 26] INFO [2022-01-10 05:44:16,421] jepsen worker 46 - jepsen.util 3106 :info :write [15 15] :RequestTimeoutException INFO [2022-01-10 05:44:16,422] jepsen worker 46 - jepsen.util 3196 :invoke :read [15 nil] INFO [2022-01-10 05:44:16,451] jepsen worker 54 - jepsen.util 3024 :info :write [18 20] :SyncWriteAmbiguous INFO [2022-01-10 05:44:16,451] jepsen worker 54 - jepsen.util 3114 :invoke :read [18 nil] INFO [2022-01-10 05:44:16,486] jepsen worker 54 - jepsen.util 3114 :ok :read [18 14] INFO [2022-01-10 05:44:16,487] jepsen worker 54 - jepsen.util 3114 :invoke :read [18 nil] INFO [2022-01-10 05:44:16,515] jepsen worker 3 - jepsen.util 3063 :ok :read [1 10] INFO [2022-01-10 05:44:16,516] jepsen worker 3 - jepsen.util 3063 :invoke :write [1 13] INFO [2022-01-10 05:44:16,521] jepsen worker 88 - jepsen.util 2968 :info :write [29 41] :RequestTimeoutException INFO [2022-01-10 05:44:16,521] jepsen worker 54 - jepsen.util 3114 :ok :read [18 14] INFO [2022-01-10 05:44:16,522] jepsen worker 88 - jepsen.util 3058 :invoke :read [29 nil] INFO [2022-01-10 05:44:16,522] jepsen worker 54 - jepsen.util 3114 :invoke :read [18 nil] INFO [2022-01-10 05:44:16,543] jepsen worker 54 - jepsen.util 3114 :ok :read [18 14] INFO [2022-01-10 05:44:16,544] jepsen worker 54 - jepsen.util 3114 :invoke :write [18 47] INFO [2022-01-10 05:44:16,944] jepsen worker 23 - jepsen.util 3083 :info :write [7 14] :RequestTimeoutException INFO [2022-01-10 05:44:16,947] jepsen worker 23 - jepsen.util 3173 :invoke :read [7 nil] INFO [2022-01-10 05:44:16,955] jepsen worker 57 - jepsen.util 3567 :info :write [19 13] :RequestTimeoutException INFO [2022-01-10 05:44:16,955] jepsen worker 57 - jepsen.util 3657 :invoke :read [19 nil] INFO [2022-01-10 05:44:16,960] jepsen worker 34 - jepsen.util 2914 :info :write [11 12] :RequestTimeoutException INFO [2022-01-10 05:44:16,960] jepsen worker 34 - jepsen.util 3004 :invoke :read [11 nil] INFO [2022-01-10 05:44:16,966] jepsen worker 48 - jepsen.util 2568 :info :write [16 0] :RequestTimeoutException INFO [2022-01-10 05:44:16,966] jepsen worker 48 - jepsen.util 2658 :invoke :write [16 0] INFO [2022-01-10 05:44:17,021] jepsen worker 11 - jepsen.util 2891 :info :write [3 5] :RequestTimeoutException INFO [2022-01-10 05:44:17,022] jepsen worker 11 - jepsen.util 2981 :invoke :read [3 nil] INFO [2022-01-10 05:44:17,034] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:17,034] jepsen worker nemesis - couchbase.util Rebalance check count: 408 INFO [2022-01-10 05:44:17,034] jepsen worker nemesis - couchbase.util Rebalance stuck count: 322 INFO [2022-01-10 05:44:17,078] jepsen worker 11 - jepsen.util 2981 :ok :read [3 31] INFO [2022-01-10 05:44:17,079] jepsen worker 11 - jepsen.util 2981 :invoke :write [3 28] INFO [2022-01-10 05:44:17,416] jepsen worker 49 - jepsen.util 2929 :ok :read [16 38] INFO [2022-01-10 05:44:17,417] jepsen worker 49 - jepsen.util 2929 :invoke :read [16 nil] INFO [2022-01-10 05:44:17,425] jepsen worker 56 - jepsen.util 3296 :info :write [18 48] :RequestTimeoutException INFO [2022-01-10 05:44:17,425] jepsen worker 56 - jepsen.util 3386 :invoke :write [18 19] INFO [2022-01-10 05:44:17,637] jepsen worker 43 - jepsen.util 2833 :info :write [14 5] :RequestTimeoutException INFO [2022-01-10 05:44:17,637] jepsen worker 43 - jepsen.util 2923 :invoke :read [14 nil] INFO [2022-01-10 05:44:17,689] jepsen worker 18 - jepsen.util 2898 :info :write [6 46] :RequestTimeoutException INFO [2022-01-10 05:44:17,692] jepsen worker 18 - jepsen.util 2988 :invoke :read [6 nil] INFO [2022-01-10 05:44:17,751] jepsen worker 63 - jepsen.util 2853 :info :write [21 28] :RequestTimeoutException INFO [2022-01-10 05:44:17,752] jepsen worker 63 - jepsen.util 2943 :invoke :write [21 26] INFO [2022-01-10 05:44:17,922] jepsen worker 65 - jepsen.util 2765 :info :write [21 30] :RequestTimeoutException INFO [2022-01-10 05:44:17,922] jepsen worker 65 - jepsen.util 2855 :invoke :read [21 nil] INFO [2022-01-10 05:44:17,956] jepsen worker 46 - jepsen.util 3196 :ok :read [15 23] INFO [2022-01-10 05:44:17,957] jepsen worker 46 - jepsen.util 3196 :invoke :read [15 nil] INFO [2022-01-10 05:44:18,041] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:18,041] jepsen worker nemesis - couchbase.util Rebalance check count: 409 INFO [2022-01-10 05:44:18,041] jepsen worker nemesis - couchbase.util Rebalance stuck count: 323 INFO [2022-01-10 05:44:18,116] jepsen worker 89 - jepsen.util 3509 :info :write [29 30] :RequestTimeoutException INFO [2022-01-10 05:44:18,117] jepsen worker 89 - jepsen.util 3599 :invoke :write [29 32] INFO [2022-01-10 05:44:18,441] jepsen worker 24 - jepsen.util 2634 :ok :read [8 27] INFO [2022-01-10 05:44:18,442] jepsen worker 24 - jepsen.util 2634 :invoke :write [8 0] INFO [2022-01-10 05:44:18,849] jepsen worker 78 - jepsen.util 3498 :info :write [26 26] :RequestTimeoutException INFO [2022-01-10 05:44:18,850] jepsen worker 78 - jepsen.util 3588 :invoke :read [26 nil] INFO [2022-01-10 05:44:18,852] jepsen worker 47 - jepsen.util 3017 :info :write [15 22] :RequestTimeoutException INFO [2022-01-10 05:44:18,852] jepsen worker 47 - jepsen.util 3107 :invoke :read [15 nil] INFO [2022-01-10 05:44:18,888] jepsen worker 81 - jepsen.util 3231 :ok :read [27 30] INFO [2022-01-10 05:44:18,888] jepsen worker 81 - jepsen.util 3231 :invoke :write [27 12] INFO [2022-01-10 05:44:18,906] jepsen worker 76 - jepsen.util 2506 :info :write [25 5] :RequestTimeoutException INFO [2022-01-10 05:44:18,907] jepsen worker 76 - jepsen.util 2596 :invoke :read [25 nil] INFO [2022-01-10 05:44:18,911] jepsen worker 86 - jepsen.util 2426 :info :write [28 45] :RequestTimeoutException INFO [2022-01-10 05:44:18,911] jepsen worker 86 - jepsen.util 2516 :invoke :read [28 nil] INFO [2022-01-10 05:44:18,934] jepsen worker 4 - jepsen.util 3424 :info :write [1 0] :SyncWriteAmbiguous INFO [2022-01-10 05:44:18,934] jepsen worker 4 - jepsen.util 3514 :invoke :write [1 31] INFO [2022-01-10 05:44:19,046] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:19,046] jepsen worker nemesis - couchbase.util Rebalance check count: 410 INFO [2022-01-10 05:44:19,046] jepsen worker nemesis - couchbase.util Rebalance stuck count: 324 INFO [2022-01-10 05:44:19,102] jepsen worker 58 - jepsen.util 2938 :ok :read [19 31] INFO [2022-01-10 05:44:19,102] jepsen worker 58 - jepsen.util 2938 :invoke :read [19 nil] INFO [2022-01-10 05:44:19,160] jepsen worker 73 - jepsen.util 2683 :info :write [24 47] :RequestTimeoutException INFO [2022-01-10 05:44:19,160] jepsen worker 73 - jepsen.util 2773 :invoke :read [24 nil] INFO [2022-01-10 05:44:19,275] jepsen worker 52 - jepsen.util 3742 :info :write [17 14] :RequestTimeoutException INFO [2022-01-10 05:44:19,276] jepsen worker 52 - jepsen.util 3832 :invoke :read [17 nil] INFO [2022-01-10 05:44:19,286] jepsen worker 80 - jepsen.util 3590 :info :write [26 9] :RequestTimeoutException INFO [2022-01-10 05:44:19,286] jepsen worker 80 - jepsen.util 3680 :invoke :read [26 nil] INFO [2022-01-10 05:44:19,790] jepsen worker 15 - jepsen.util 2535 :info :write [5 4] :RequestTimeoutException INFO [2022-01-10 05:44:19,790] jepsen worker 15 - jepsen.util 2625 :invoke :read [5 nil] INFO [2022-01-10 05:44:19,996] jepsen worker 20 - jepsen.util 2810 :info :write [6 26] :RequestTimeoutException INFO [2022-01-10 05:44:19,997] jepsen worker 20 - jepsen.util 2900 :invoke :read [6 nil] INFO [2022-01-10 05:44:19,999] jepsen worker 16 - jepsen.util 2986 :info :write [5 31] :RequestTimeoutException INFO [2022-01-10 05:44:20,000] jepsen worker 16 - jepsen.util 3076 :invoke :write [5 42] INFO [2022-01-10 05:44:20,052] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:20,052] jepsen worker nemesis - couchbase.util Rebalance check count: 411 INFO [2022-01-10 05:44:20,052] jepsen worker nemesis - couchbase.util Rebalance stuck count: 325 INFO [2022-01-10 05:44:20,135] jepsen worker 29 - jepsen.util 2819 :ok :read [9 24] INFO [2022-01-10 05:44:20,135] jepsen worker 29 - jepsen.util 2819 :invoke :write [9 17] INFO [2022-01-10 05:44:20,232] jepsen worker 7 - jepsen.util 2977 :info :write [2 22] :RequestTimeoutException INFO [2022-01-10 05:44:20,233] jepsen worker 7 - jepsen.util 3067 :invoke :write [2 25] INFO [2022-01-10 05:44:20,279] jepsen worker 78 - jepsen.util 3588 :ok :read [26 10] INFO [2022-01-10 05:44:20,280] jepsen worker 78 - jepsen.util 3588 :invoke :read [26 nil] WARN [2022-01-10 05:44:20,292] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9808805,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10658009,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9373508,"encode_duration_us":21242,"operation_id":"0x5cbc","timeout_ms":10000,"last_server_duration_us":9249624,"total_duration_us":10009877}],"total_count":1}} INFO [2022-01-10 05:44:20,320] jepsen worker 52 - jepsen.util 3832 :ok :read [17 41] INFO [2022-01-10 05:44:20,321] jepsen worker 52 - jepsen.util 3832 :invoke :write [17 13] INFO [2022-01-10 05:44:20,354] jepsen worker 78 - jepsen.util 3588 :ok :read [26 10] INFO [2022-01-10 05:44:20,354] jepsen worker 78 - jepsen.util 3588 :invoke :read [26 nil] WARN [2022-01-10 05:44:20,392] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":5632151,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11900550,"total_server_duration_us":1394615,"encode_duration_us":19735,"operation_id":"0x5e13","timeout_ms":10000,"last_server_duration_us":578951,"total_duration_us":10010572},{"operation_name":"upsert","last_dispatch_duration_us":849237,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7692290,"total_server_duration_us":1445513,"encode_duration_us":21550,"operation_id":"0x5d8f","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10010003},{"operation_name":"upsert","last_dispatch_duration_us":1177077,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":3810551,"total_server_duration_us":2098198,"encode_duration_us":22497,"operation_id":"0x5cf9","timeout_ms":10000,"last_server_duration_us":65775,"total_duration_us":10009204},{"operation_name":"upsert","last_dispatch_duration_us":46550,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":407524,"total_server_duration_us":250102,"encode_duration_us":18694,"operation_id":"0x5e24","timeout_ms":10000,"last_server_duration_us":34314,"total_duration_us":10007580},{"operation_name":"upsert","last_dispatch_duration_us":4145827,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9202128,"total_server_duration_us":1029786,"encode_duration_us":17911,"operation_id":"0x5d7e","timeout_ms":10000,"last_server_duration_us":237936,"total_duration_us":10004935},{"operation_name":"upsert","last_dispatch_duration_us":5848947,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14932200,"total_server_duration_us":1206552,"encode_duration_us":17964,"operation_id":"0x5d59","timeout_ms":10000,"last_server_duration_us":835226,"total_duration_us":10003437},{"operation_name":"upsert","last_dispatch_duration_us":1437492,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6231619,"total_server_duration_us":1560244,"encode_duration_us":25153,"operation_id":"0x5d24","timeout_ms":10000,"last_server_duration_us":1056846,"total_duration_us":10002858},{"operation_name":"upsert","last_dispatch_duration_us":3710421,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9983518,"total_server_duration_us":4620168,"encode_duration_us":18755,"operation_id":"0x5e12","timeout_ms":10000,"last_server_duration_us":2212391,"total_duration_us":10001417},{"operation_name":"get","last_dispatch_duration_us":9536752,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9536752,"total_server_duration_us":413530,"operation_id":"0x5d2f","timeout_ms":30000,"last_server_duration_us":413530,"total_duration_us":9538954},{"operation_name":"upsert","last_dispatch_duration_us":9059507,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9059507,"total_server_duration_us":9041612,"encode_duration_us":19611,"operation_id":"0x5e23","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":9063822}],"total_count":18}} INFO [2022-01-10 05:44:20,836] jepsen worker 40 - jepsen.util 2380 :info :write [13 40] :RequestTimeoutException INFO [2022-01-10 05:44:20,836] jepsen worker 40 - jepsen.util 2470 :invoke :read [13 nil] INFO [2022-01-10 05:44:20,906] jepsen worker 46 - jepsen.util 3196 :ok :read [15 23] INFO [2022-01-10 05:44:20,906] jepsen worker 46 - jepsen.util 3196 :invoke :read [15 nil] INFO [2022-01-10 05:44:20,919] jepsen worker 65 - jepsen.util 2855 :ok :read [21 7] INFO [2022-01-10 05:44:20,920] jepsen worker 65 - jepsen.util 2855 :invoke :read [21 nil] INFO [2022-01-10 05:44:20,938] jepsen worker 78 - jepsen.util 3588 :ok :read [26 10] INFO [2022-01-10 05:44:20,939] jepsen worker 78 - jepsen.util 3588 :invoke :write [26 6] INFO [2022-01-10 05:44:21,059] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:21,059] jepsen worker nemesis - couchbase.util Rebalance check count: 412 INFO [2022-01-10 05:44:21,059] jepsen worker nemesis - couchbase.util Rebalance stuck count: 326 INFO [2022-01-10 05:44:21,145] jepsen worker 79 - jepsen.util 3409 :info :write [26 23] :RequestTimeoutException INFO [2022-01-10 05:44:21,145] jepsen worker 79 - jepsen.util 3499 :invoke :read [26 nil] INFO [2022-01-10 05:44:21,145] jepsen worker 61 - jepsen.util 3121 :ok :read [20 :nil] INFO [2022-01-10 05:44:21,145] jepsen worker 77 - jepsen.util 3407 :ok :read [25 :nil] INFO [2022-01-10 05:44:21,146] jepsen worker 77 - jepsen.util 3407 :invoke :read [25 nil] INFO [2022-01-10 05:44:21,146] jepsen worker 61 - jepsen.util 3121 :invoke :write [20 9] INFO [2022-01-10 05:44:21,150] jepsen worker 59 - jepsen.util 2939 :ok :read [19 31] INFO [2022-01-10 05:44:21,150] jepsen worker 59 - jepsen.util 2939 :invoke :read [19 nil] INFO [2022-01-10 05:44:21,263] jepsen worker 50 - jepsen.util 3110 :ok :read [16 38] INFO [2022-01-10 05:44:21,263] jepsen worker 50 - jepsen.util 3110 :invoke :write [16 49] INFO [2022-01-10 05:44:21,322] jepsen worker 39 - jepsen.util 3459 :info :write [13 49] :RequestTimeoutException INFO [2022-01-10 05:44:21,323] jepsen worker 39 - jepsen.util 3549 :invoke :read [13 nil] INFO [2022-01-10 05:44:21,356] jepsen worker 80 - jepsen.util 3680 :ok :read [26 10] INFO [2022-01-10 05:44:21,356] jepsen worker 80 - jepsen.util 3680 :invoke :read [26 nil] INFO [2022-01-10 05:44:21,373] jepsen worker 79 - jepsen.util 3499 :ok :read [26 10] INFO [2022-01-10 05:44:21,373] jepsen worker 79 - jepsen.util 3499 :invoke :read [26 nil] INFO [2022-01-10 05:44:21,422] jepsen worker 13 - jepsen.util 3343 :info :write [4 20] :RequestTimeoutException INFO [2022-01-10 05:44:21,423] jepsen worker 13 - jepsen.util 3433 :invoke :read [4 nil] INFO [2022-01-10 05:44:21,511] jepsen worker 83 - jepsen.util 3503 :info :write [27 27] :RequestTimeoutException INFO [2022-01-10 05:44:21,512] jepsen worker 83 - jepsen.util 3593 :invoke :read [27 nil] INFO [2022-01-10 05:44:21,524] jepsen worker 84 - jepsen.util 2964 :info :write [28 24] :RequestTimeoutException INFO [2022-01-10 05:44:21,525] jepsen worker 84 - jepsen.util 3054 :invoke :read [28 nil] INFO [2022-01-10 05:44:21,546] jepsen worker 57 - jepsen.util 3657 :ok :read [19 31] INFO [2022-01-10 05:44:21,546] jepsen worker 57 - jepsen.util 3657 :invoke :write [19 49] WARN [2022-01-10 05:44:21,890] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12843581,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13293426,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":11536550,"encode_duration_us":17675,"operation_id":"0x5d80","timeout_ms":10000,"last_server_duration_us":11487295,"total_duration_us":10007407},{"operation_name":"upsert","last_dispatch_duration_us":18084767,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18084767,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9028877,"encode_duration_us":16863,"operation_id":"0x5c1e","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10009031}],"total_count":2}} INFO [2022-01-10 05:44:21,891] jepsen worker 65 - jepsen.util 2855 :ok :read [21 7] WARN [2022-01-10 05:44:21,891] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12001676,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":12001676,"total_server_duration_us":207412,"operation_id":"0x5e18","timeout_ms":30000,"last_server_duration_us":207412,"total_duration_us":12003664},{"operation_name":"get","last_dispatch_duration_us":11248247,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11248247,"total_server_duration_us":38,"operation_id":"0x5eaa","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":11250374},{"operation_name":"get","last_dispatch_duration_us":10668825,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10668825,"total_server_duration_us":26864,"operation_id":"0x5eb8","timeout_ms":30000,"last_server_duration_us":26864,"total_duration_us":10671490},{"operation_name":"upsert","last_dispatch_duration_us":2451280,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4324965,"total_server_duration_us":386219,"encode_duration_us":20166,"operation_id":"0x5d1f","timeout_ms":10000,"last_server_duration_us":67617,"total_duration_us":10010321},{"operation_name":"upsert","last_dispatch_duration_us":66569,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9220516,"total_server_duration_us":225399,"encode_duration_us":17419,"operation_id":"0x5de3","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10010167},{"operation_name":"upsert","last_dispatch_duration_us":61165,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9427602,"total_server_duration_us":203467,"encode_duration_us":16104,"operation_id":"0x5df3","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10009500},{"operation_name":"upsert","last_dispatch_duration_us":124884,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":6159826,"total_server_duration_us":5089115,"encode_duration_us":16396,"operation_id":"0x5dbf","timeout_ms":10000,"last_server_duration_us":97174,"total_duration_us":10008316},{"operation_name":"upsert","last_dispatch_duration_us":4901998,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":12372470,"total_server_duration_us":862352,"encode_duration_us":18046,"operation_id":"0x5df1","timeout_ms":10000,"last_server_duration_us":489444,"total_duration_us":10008184},{"operation_name":"upsert","last_dispatch_duration_us":36290,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9400435,"total_server_duration_us":227519,"encode_duration_us":16273,"operation_id":"0x5dec","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10007581},{"operation_name":"upsert","last_dispatch_duration_us":12843581,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":13293426,"total_server_duration_us":11536550,"encode_duration_us":17675,"operation_id":"0x5d80","timeout_ms":10000,"last_server_duration_us":11487295,"total_duration_us":10007407}],"total_count":19}} WARN [2022-01-10 05:44:21,891] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17213428,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17714655,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9231848,"encode_duration_us":16602,"operation_id":"0x5d23","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10003296}],"total_count":1}} INFO [2022-01-10 05:44:21,892] jepsen worker 65 - jepsen.util 2855 :invoke :write [21 21] WARN [2022-01-10 05:44:21,894] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11347549,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11347549,"total_server_duration_us":410960,"operation_id":"0x5cfb","timeout_ms":30000,"last_server_duration_us":410960,"total_duration_us":11349360},{"operation_name":"upsert","last_dispatch_duration_us":14553378,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":14553378,"total_server_duration_us":1246295,"encode_duration_us":18944,"operation_id":"0x5d81","timeout_ms":10000,"last_server_duration_us":1246295,"total_duration_us":10009723},{"operation_name":"upsert","last_dispatch_duration_us":34224,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":1575469,"total_server_duration_us":335797,"encode_duration_us":16481,"operation_id":"0x5dee","timeout_ms":10000,"last_server_duration_us":22306,"total_duration_us":10008909},{"operation_name":"upsert","last_dispatch_duration_us":1579148,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":8183825,"total_server_duration_us":1149304,"encode_duration_us":16584,"operation_id":"0x5d4e","timeout_ms":10000,"last_server_duration_us":52777,"total_duration_us":10008838},{"operation_name":"upsert","last_dispatch_duration_us":1502707,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10842807,"total_server_duration_us":1676834,"encode_duration_us":25535,"operation_id":"0x5e26","timeout_ms":10000,"last_server_duration_us":309279,"total_duration_us":10007744},{"operation_name":"upsert","last_dispatch_duration_us":1591222,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":8864498,"total_server_duration_us":4816488,"encode_duration_us":18751,"operation_id":"0x5e85","timeout_ms":10000,"last_server_duration_us":20441,"total_duration_us":10005928},{"operation_name":"upsert","last_dispatch_duration_us":7744093,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8215587,"total_server_duration_us":311422,"encode_duration_us":17801,"operation_id":"0x5e4a","timeout_ms":10000,"last_server_duration_us":20680,"total_duration_us":10003778},{"operation_name":"upsert","last_dispatch_duration_us":1250366,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6737116,"total_server_duration_us":4676060,"encode_duration_us":17034,"operation_id":"0x5e78","timeout_ms":10000,"last_server_duration_us":520879,"total_duration_us":10003669},{"operation_name":"upsert","last_dispatch_duration_us":2828299,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10643089,"total_server_duration_us":2525040,"encode_duration_us":16588,"operation_id":"0x5db5","timeout_ms":10000,"last_server_duration_us":108399,"total_duration_us":10001780},{"operation_name":"get","last_dispatch_duration_us":9610808,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9610808,"total_server_duration_us":49,"operation_id":"0x5eb7","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":9613023}],"total_count":16}} WARN [2022-01-10 05:44:21,895] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":424644,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":6705379,"total_server_duration_us":4001396,"encode_duration_us":18227,"operation_id":"0x5ded","timeout_ms":10000,"last_server_duration_us":114454,"total_duration_us":10010979},{"operation_name":"upsert","last_dispatch_duration_us":3122560,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8757357,"total_server_duration_us":4685347,"encode_duration_us":16183,"operation_id":"0x5ee6","timeout_ms":10000,"last_server_duration_us":997083,"total_duration_us":10010503},{"operation_name":"upsert","last_dispatch_duration_us":33440,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8235025,"total_server_duration_us":268124,"encode_duration_us":19913,"operation_id":"0x5d89","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10010442},{"operation_name":"upsert","last_dispatch_duration_us":5093460,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":13169770,"total_server_duration_us":2497495,"encode_duration_us":21290,"operation_id":"0x5db4","timeout_ms":10000,"last_server_duration_us":193132,"total_duration_us":10010276},{"operation_name":"upsert","last_dispatch_duration_us":7918319,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":16269617,"total_server_duration_us":1006114,"encode_duration_us":16408,"operation_id":"0x5db3","timeout_ms":10000,"last_server_duration_us":130462,"total_duration_us":10007935},{"operation_name":"upsert","last_dispatch_duration_us":6620649,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6620649,"total_server_duration_us":170565,"encode_duration_us":16527,"operation_id":"0x5e77","timeout_ms":10000,"last_server_duration_us":170565,"total_duration_us":10007852},{"operation_name":"upsert","last_dispatch_duration_us":5617328,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":12597814,"total_server_duration_us":1113135,"encode_duration_us":19149,"operation_id":"0x5d7f","timeout_ms":10000,"last_server_duration_us":33035,"total_duration_us":10007461},{"operation_name":"upsert","last_dispatch_duration_us":625931,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":7246446,"total_server_duration_us":5554696,"encode_duration_us":16488,"operation_id":"0x5ee8","timeout_ms":10000,"last_server_duration_us":598552,"total_duration_us":10006951},{"operation_name":"upsert","last_dispatch_duration_us":6215771,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6215771,"total_server_duration_us":2216476,"encode_duration_us":19127,"operation_id":"0x5eb5","timeout_ms":10000,"last_server_duration_us":2216476,"total_duration_us":10005420},{"operation_name":"upsert","last_dispatch_duration_us":5542739,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":12925838,"total_server_duration_us":1277316,"encode_duration_us":19845,"operation_id":"0x5d79","timeout_ms":10000,"last_server_duration_us":336289,"total_duration_us":10005119}],"total_count":16}} INFO [2022-01-10 05:44:21,944] jepsen worker 19 - jepsen.util 3079 :info :write [6 7] :RequestTimeoutException WARN [2022-01-10 05:44:21,992] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8362309,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":14059959,"total_server_duration_us":1225451,"encode_duration_us":19422,"operation_id":"0x5d91","timeout_ms":10000,"last_server_duration_us":356973,"total_duration_us":10009857},{"operation_name":"upsert","last_dispatch_duration_us":9952718,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":14351668,"total_server_duration_us":793161,"encode_duration_us":24660,"operation_id":"0x5e15","timeout_ms":10000,"last_server_duration_us":226751,"total_duration_us":10009393},{"operation_name":"upsert","last_dispatch_duration_us":10887480,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10887480,"total_server_duration_us":34612,"encode_duration_us":18652,"operation_id":"0x5d2c","timeout_ms":10000,"last_server_duration_us":34612,"total_duration_us":10005816},{"operation_name":"upsert","last_dispatch_duration_us":2263605,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2263605,"total_server_duration_us":272615,"encode_duration_us":18328,"operation_id":"0x5ead","timeout_ms":10000,"last_server_duration_us":272615,"total_duration_us":10005409},{"operation_name":"upsert","last_dispatch_duration_us":9670392,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9670392,"total_server_duration_us":371326,"encode_duration_us":17992,"operation_id":"0x5eda","timeout_ms":10000,"last_server_duration_us":371326,"total_duration_us":10005381},{"operation_name":"upsert","last_dispatch_duration_us":3476504,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8459619,"total_server_duration_us":1560892,"encode_duration_us":21678,"operation_id":"0x5d9e","timeout_ms":10000,"last_server_duration_us":760760,"total_duration_us":10005256},{"operation_name":"upsert","last_dispatch_duration_us":2041507,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9183289,"total_server_duration_us":3909339,"encode_duration_us":17486,"operation_id":"0x5e84","timeout_ms":10000,"last_server_duration_us":203378,"total_duration_us":10003455},{"operation_name":"upsert","last_dispatch_duration_us":17213428,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":17714655,"total_server_duration_us":9231848,"encode_duration_us":16602,"operation_id":"0x5d23","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10003296},{"operation_name":"get","last_dispatch_duration_us":1896832,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1896832,"total_server_duration_us":3310,"operation_id":"0x5dd3","timeout_ms":30000,"last_server_duration_us":3310,"total_duration_us":8595530},{"operation_name":"get","last_dispatch_duration_us":1704601,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1704601,"total_server_duration_us":7909,"operation_id":"0x5e25","timeout_ms":30000,"last_server_duration_us":7909,"total_duration_us":6874299}],"total_count":16}} WARN [2022-01-10 05:44:21,994] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":31014049,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":31014049,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9023573,"encode_duration_us":16545,"operation_id":"0x5970","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10002937}],"total_count":1}} INFO [2022-01-10 05:44:22,071] jepsen worker 19 - jepsen.util 3169 :invoke :read [6 nil] INFO [2022-01-10 05:44:22,001] jepsen worker 30 - jepsen.util 2820 :info :write [10 33] :RequestTimeoutException INFO [2022-01-10 05:44:22,045] jepsen worker 26 - jepsen.util 2546 :info :write [8 24] :RequestTimeoutException INFO [2022-01-10 05:44:22,071] jepsen worker 30 - jepsen.util 2910 :invoke :write [10 21] INFO [2022-01-10 05:44:22,045] jepsen worker 82 - jepsen.util 3052 :info :write [27 37] :RequestTimeoutException INFO [2022-01-10 05:44:22,064] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:22,071] jepsen worker 26 - jepsen.util 2636 :invoke :write [8 34] INFO [2022-01-10 05:44:22,071] jepsen worker 82 - jepsen.util 3142 :invoke :read [27 nil] INFO [2022-01-10 05:44:22,071] jepsen worker nemesis - couchbase.util Rebalance check count: 413 INFO [2022-01-10 05:44:22,071] jepsen worker nemesis - couchbase.util Rebalance stuck count: 327 WARN [2022-01-10 05:44:22,171] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":29504908,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":29504908,"total_server_duration_us":32,"operation_id":"0x59c4","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":29506160},{"operation_name":"get","last_dispatch_duration_us":25549783,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":25549783,"total_server_duration_us":15,"operation_id":"0x5a7e","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":25551118},{"operation_name":"get","last_dispatch_duration_us":24682214,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":24682214,"total_server_duration_us":11,"operation_id":"0x5aae","timeout_ms":30000,"last_server_duration_us":11,"total_duration_us":24684112},{"operation_name":"get","last_dispatch_duration_us":18169161,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":18169161,"total_server_duration_us":19,"operation_id":"0x5bfa","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":18171491},{"operation_name":"upsert","last_dispatch_duration_us":706066,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2614519,"total_server_duration_us":1597404,"encode_duration_us":17489,"operation_id":"0x5e14","timeout_ms":10000,"last_server_duration_us":501484,"total_duration_us":10010362},{"operation_name":"upsert","last_dispatch_duration_us":1292232,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2239527,"total_server_duration_us":316919,"encode_duration_us":17017,"operation_id":"0x5e27","timeout_ms":10000,"last_server_duration_us":142412,"total_duration_us":10010199},{"operation_name":"upsert","last_dispatch_duration_us":2381161,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9417486,"total_server_duration_us":3849141,"encode_duration_us":17193,"operation_id":"0x5e76","timeout_ms":10000,"last_server_duration_us":547655,"total_duration_us":10010083},{"operation_name":"upsert","last_dispatch_duration_us":1779926,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":1779926,"total_server_duration_us":57880,"encode_duration_us":19325,"operation_id":"0x5dac","timeout_ms":10000,"last_server_duration_us":57880,"total_duration_us":10009233},{"operation_name":"upsert","last_dispatch_duration_us":8924979,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10910434,"total_server_duration_us":569290,"encode_duration_us":16540,"operation_id":"0x5e9b","timeout_ms":10000,"last_server_duration_us":202322,"total_duration_us":10007424},{"operation_name":"upsert","last_dispatch_duration_us":542668,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10424705,"total_server_duration_us":233145,"encode_duration_us":17564,"operation_id":"0x5d2d","timeout_ms":10000,"last_server_duration_us":60506,"total_duration_us":10005211}],"total_count":20}} INFO [2022-01-10 05:44:22,188] jepsen worker 80 - jepsen.util 3680 :ok :read [26 10] INFO [2022-01-10 05:44:22,189] jepsen worker 80 - jepsen.util 3680 :invoke :write [26 18] INFO [2022-01-10 05:44:22,208] jepsen worker 79 - jepsen.util 3499 :ok :read [26 10] INFO [2022-01-10 05:44:22,208] jepsen worker 79 - jepsen.util 3499 :invoke :read [26 nil] INFO [2022-01-10 05:44:22,486] jepsen worker 33 - jepsen.util 3273 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:44:22,487] jepsen worker 33 - jepsen.util 3273 :invoke :read [11 nil] INFO [2022-01-10 05:44:22,565] jepsen worker 13 - jepsen.util 3433 :ok :read [4 3] INFO [2022-01-10 05:44:22,569] jepsen worker 13 - jepsen.util 3433 :invoke :read [4 nil] INFO [2022-01-10 05:44:22,640] jepsen worker 60 - jepsen.util 3390 :info :write [20 14] :RequestTimeoutException INFO [2022-01-10 05:44:22,640] jepsen worker 60 - jepsen.util 3480 :invoke :read [20 nil] INFO [2022-01-10 05:44:22,666] jepsen worker 1 - jepsen.util 2971 :info :write [0 47] :RequestTimeoutException INFO [2022-01-10 05:44:22,667] jepsen worker 1 - jepsen.util 3061 :invoke :read [0 nil] INFO [2022-01-10 05:44:22,676] jepsen worker 86 - jepsen.util 2516 :ok :read [28 45] INFO [2022-01-10 05:44:22,676] jepsen worker 86 - jepsen.util 2516 :invoke :read [28 nil] INFO [2022-01-10 05:44:22,730] jepsen worker 45 - jepsen.util 3195 :info :write [15 18] :RequestTimeoutException INFO [2022-01-10 05:44:22,731] jepsen worker 45 - jepsen.util 3285 :invoke :read [15 nil] INFO [2022-01-10 05:44:22,786] jepsen worker 13 - jepsen.util 3433 :ok :read [4 3] INFO [2022-01-10 05:44:22,787] jepsen worker 13 - jepsen.util 3433 :invoke :read [4 nil] INFO [2022-01-10 05:44:23,000] jepsen worker 17 - jepsen.util 2987 :ok :read [5 42] INFO [2022-01-10 05:44:23,001] jepsen worker 17 - jepsen.util 2987 :invoke :read [5 nil] INFO [2022-01-10 05:44:23,001] jepsen worker 15 - jepsen.util 2625 :ok :read [5 42] INFO [2022-01-10 05:44:23,001] jepsen worker 15 - jepsen.util 2625 :invoke :read [5 nil] INFO [2022-01-10 05:44:23,076] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:23,076] jepsen worker nemesis - couchbase.util Rebalance check count: 414 INFO [2022-01-10 05:44:23,076] jepsen worker nemesis - couchbase.util Rebalance stuck count: 328 INFO [2022-01-10 05:44:23,126] jepsen worker 46 - jepsen.util 3196 :ok :read [15 23] INFO [2022-01-10 05:44:23,126] jepsen worker 46 - jepsen.util 3196 :invoke :read [15 nil] INFO [2022-01-10 05:44:23,195] jepsen worker 79 - jepsen.util 3499 :ok :read [26 10] INFO [2022-01-10 05:44:23,195] jepsen worker 79 - jepsen.util 3499 :invoke :write [26 19] INFO [2022-01-10 05:44:23,205] jepsen worker 13 - jepsen.util 3433 :ok :read [4 3] INFO [2022-01-10 05:44:23,206] jepsen worker 13 - jepsen.util 3433 :invoke :read [4 nil] INFO [2022-01-10 05:44:23,216] jepsen worker 27 - jepsen.util 2997 :info :write [9 22] :RequestTimeoutException INFO [2022-01-10 05:44:23,217] jepsen worker 27 - jepsen.util 3087 :invoke :write [9 3] INFO [2022-01-10 05:44:23,256] jepsen worker 53 - jepsen.util 3563 :info :write [17 41] :RequestTimeoutException INFO [2022-01-10 05:44:23,258] jepsen worker 53 - jepsen.util 3653 :invoke :write [17 27] INFO [2022-01-10 05:44:23,296] jepsen worker 0 - jepsen.util 2970 :ok :read [0 46] INFO [2022-01-10 05:44:23,297] jepsen worker 0 - jepsen.util 2970 :invoke :read [0 nil] INFO [2022-01-10 05:44:23,314] jepsen worker 58 - jepsen.util 2938 :ok :read [19 31] INFO [2022-01-10 05:44:23,315] jepsen worker 58 - jepsen.util 2938 :invoke :write [19 11] INFO [2022-01-10 05:44:23,331] jepsen worker 2 - jepsen.util 3152 :info :write [0 7] :RequestTimeoutException INFO [2022-01-10 05:44:23,331] jepsen worker 2 - jepsen.util 3242 :invoke :write [0 44] INFO [2022-01-10 05:44:23,397] jepsen worker 13 - jepsen.util 3433 :ok :read [4 3] INFO [2022-01-10 05:44:23,397] jepsen worker 13 - jepsen.util 3433 :invoke :write [4 33] INFO [2022-01-10 05:44:23,454] jepsen worker 32 - jepsen.util 3272 :ok :read [10 36] INFO [2022-01-10 05:44:23,455] jepsen worker 32 - jepsen.util 3272 :invoke :write [10 47] INFO [2022-01-10 05:44:23,475] jepsen worker 28 - jepsen.util 2728 :info :write [9 27] :RequestTimeoutException INFO [2022-01-10 05:44:23,476] jepsen worker 69 - jepsen.util 2769 :ok :read [23 26] INFO [2022-01-10 05:44:23,476] jepsen worker 28 - jepsen.util 2818 :invoke :read [9 nil] INFO [2022-01-10 05:44:23,477] jepsen worker 69 - jepsen.util 2769 :invoke :write [23 18] INFO [2022-01-10 05:44:23,586] jepsen worker 35 - jepsen.util 2555 :info :write [11 28] :RequestTimeoutException INFO [2022-01-10 05:44:23,587] jepsen worker 35 - jepsen.util 2645 :invoke :write [11 46] INFO [2022-01-10 05:44:23,601] jepsen worker 59 - jepsen.util 2939 :ok :read [19 31] INFO [2022-01-10 05:44:23,601] jepsen worker 59 - jepsen.util 2939 :invoke :read [19 nil] INFO [2022-01-10 05:44:23,640] jepsen worker 44 - jepsen.util 3014 :info :write [14 30] :RequestTimeoutException INFO [2022-01-10 05:44:23,641] jepsen worker 44 - jepsen.util 3104 :invoke :write [14 30] INFO [2022-01-10 05:44:23,716] jepsen worker 75 - jepsen.util 2415 :info :write [25 30] :RequestTimeoutException INFO [2022-01-10 05:44:23,716] jepsen worker 75 - jepsen.util 2505 :invoke :write [25 22] INFO [2022-01-10 05:44:23,722] jepsen worker 14 - jepsen.util 3614 :info :write [4 5] :RequestTimeoutException INFO [2022-01-10 05:44:23,722] jepsen worker 14 - jepsen.util 3704 :invoke :write [4 38] INFO [2022-01-10 05:44:23,726] jepsen worker 42 - jepsen.util 2922 :info :write [14 30] :RequestTimeoutException INFO [2022-01-10 05:44:23,727] jepsen worker 42 - jepsen.util 3012 :invoke :write [14 24] INFO [2022-01-10 05:44:23,792] jepsen worker 84 - jepsen.util 3054 :ok :read [28 45] INFO [2022-01-10 05:44:23,793] jepsen worker 84 - jepsen.util 3054 :invoke :write [28 48] INFO [2022-01-10 05:44:23,799] jepsen worker 1 - jepsen.util 3061 :ok :read [0 46] INFO [2022-01-10 05:44:23,799] jepsen worker 1 - jepsen.util 3061 :invoke :write [0 10] INFO [2022-01-10 05:44:23,914] jepsen worker 41 - jepsen.util 2921 :ok :read [13 15] INFO [2022-01-10 05:44:23,915] jepsen worker 41 - jepsen.util 2921 :invoke :write [13 47] INFO [2022-01-10 05:44:23,914] jepsen worker 39 - jepsen.util 3549 :ok :read [13 15] INFO [2022-01-10 05:44:23,915] jepsen worker 39 - jepsen.util 3549 :invoke :read [13 nil] INFO [2022-01-10 05:44:23,941] jepsen worker 73 - jepsen.util 2773 :ok :read [24 18] INFO [2022-01-10 05:44:23,942] jepsen worker 73 - jepsen.util 2773 :invoke :read [24 nil] INFO [2022-01-10 05:44:24,082] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:24,082] jepsen worker nemesis - couchbase.util Rebalance check count: 415 INFO [2022-01-10 05:44:24,082] jepsen worker nemesis - couchbase.util Rebalance stuck count: 329 INFO [2022-01-10 05:44:24,101] jepsen worker 85 - jepsen.util 3235 :info :write [28 3] :RequestTimeoutException INFO [2022-01-10 05:44:24,102] jepsen worker 85 - jepsen.util 3325 :invoke :write [28 23] INFO [2022-01-10 05:44:24,385] jepsen worker 55 - jepsen.util 3295 :info :write [18 2] :RequestTimeoutException INFO [2022-01-10 05:44:24,386] jepsen worker 55 - jepsen.util 3385 :invoke :read [18 nil] INFO [2022-01-10 05:44:24,400] jepsen worker 36 - jepsen.util 2466 :info :write [12 13] :RequestTimeoutException INFO [2022-01-10 05:44:24,400] jepsen worker 36 - jepsen.util 2556 :invoke :read [12 nil] INFO [2022-01-10 05:44:24,405] jepsen worker 21 - jepsen.util 3171 :info :write [7 11] :RequestTimeoutException INFO [2022-01-10 05:44:24,406] jepsen worker 21 - jepsen.util 3261 :invoke :read [7 nil] INFO [2022-01-10 05:44:24,429] jepsen worker 70 - jepsen.util 3040 :info :write [23 26] :RequestTimeoutException INFO [2022-01-10 05:44:24,430] jepsen worker 70 - jepsen.util 3130 :invoke :read [23 nil] INFO [2022-01-10 05:44:24,442] jepsen worker 8 - jepsen.util 3428 :info :write [2 8] :RequestTimeoutException INFO [2022-01-10 05:44:24,443] jepsen worker 8 - jepsen.util 3518 :invoke :write [2 47] INFO [2022-01-10 05:44:24,455] jepsen worker 74 - jepsen.util 3584 :info :write [24 15] :RequestTimeoutException INFO [2022-01-10 05:44:24,456] jepsen worker 74 - jepsen.util 3674 :invoke :read [24 nil] INFO [2022-01-10 05:44:24,460] jepsen worker 5 - jepsen.util 2885 :info :write [1 12] :RequestTimeoutException INFO [2022-01-10 05:44:24,461] jepsen worker 5 - jepsen.util 2975 :invoke :write [1 24] INFO [2022-01-10 05:44:24,465] jepsen worker 64 - jepsen.util 2944 :info :write [21 5] :RequestTimeoutException INFO [2022-01-10 05:44:24,465] jepsen worker 64 - jepsen.util 3034 :invoke :write [21 40] INFO [2022-01-10 05:44:24,616] jepsen worker 72 - jepsen.util 3312 :info :write [24 35] :RequestTimeoutException INFO [2022-01-10 05:44:24,616] jepsen worker 72 - jepsen.util 3402 :invoke :write [24 11] INFO [2022-01-10 05:44:24,670] jepsen worker 67 - jepsen.util 2857 :info :write [22 34] :RequestTimeoutException INFO [2022-01-10 05:44:24,670] jepsen worker 67 - jepsen.util 2947 :invoke :read [22 nil] INFO [2022-01-10 05:44:24,772] jepsen worker 66 - jepsen.util 2676 :info :write [22 3] :RequestTimeoutException INFO [2022-01-10 05:44:24,772] jepsen worker 66 - jepsen.util 2766 :invoke :write [22 27] INFO [2022-01-10 05:44:24,826] jepsen worker 22 - jepsen.util 2722 :info :write [7 40] :RequestTimeoutException INFO [2022-01-10 05:44:24,826] jepsen worker 22 - jepsen.util 2812 :invoke :read [7 nil] INFO [2022-01-10 05:44:24,959] jepsen worker 0 - jepsen.util 2970 :ok :read [0 46] INFO [2022-01-10 05:44:24,959] jepsen worker 0 - jepsen.util 2970 :invoke :read [0 nil] INFO [2022-01-10 05:44:25,096] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:25,096] jepsen worker nemesis - couchbase.util Rebalance check count: 416 INFO [2022-01-10 05:44:25,096] jepsen worker nemesis - couchbase.util Rebalance stuck count: 330 INFO [2022-01-10 05:44:25,138] jepsen worker 82 - jepsen.util 3142 :ok :read [27 30] INFO [2022-01-10 05:44:25,139] jepsen worker 82 - jepsen.util 3142 :invoke :write [27 37] INFO [2022-01-10 05:44:25,264] jepsen worker 15 - jepsen.util 2625 :ok :read [5 42] INFO [2022-01-10 05:44:25,265] jepsen worker 15 - jepsen.util 2625 :invoke :write [5 48] INFO [2022-01-10 05:44:25,287] jepsen worker 68 - jepsen.util 3038 :ok :read [22 41] INFO [2022-01-10 05:44:25,288] jepsen worker 68 - jepsen.util 3038 :invoke :read [22 nil] INFO [2022-01-10 05:44:25,303] jepsen worker 86 - jepsen.util 2516 :ok :read [28 45] INFO [2022-01-10 05:44:25,303] jepsen worker 86 - jepsen.util 2516 :invoke :read [28 nil] INFO [2022-01-10 05:44:25,371] jepsen worker 87 - jepsen.util 3057 :info :write [29 7] :RequestTimeoutException INFO [2022-01-10 05:44:25,372] jepsen worker 87 - jepsen.util 3147 :invoke :write [29 23] INFO [2022-01-10 05:44:25,488] jepsen worker 60 - jepsen.util 3480 :ok :read [20 :nil] INFO [2022-01-10 05:44:25,488] jepsen worker 60 - jepsen.util 3480 :invoke :read [20 nil] INFO [2022-01-10 05:44:25,523] jepsen worker 71 - jepsen.util 2501 :ok :read [23 26] INFO [2022-01-10 05:44:25,523] jepsen worker 71 - jepsen.util 2501 :invoke :write [23 14] INFO [2022-01-10 05:44:25,524] jepsen worker 59 - jepsen.util 2939 :ok :read [19 31] INFO [2022-01-10 05:44:25,525] jepsen worker 59 - jepsen.util 2939 :invoke :read [19 nil] INFO [2022-01-10 05:44:25,610] jepsen worker 34 - jepsen.util 3004 :ok :read [11 21] INFO [2022-01-10 05:44:25,611] jepsen worker 34 - jepsen.util 3004 :invoke :write [11 35] INFO [2022-01-10 05:44:25,626] jepsen worker 49 - jepsen.util 2929 :ok :read [16 38] INFO [2022-01-10 05:44:25,627] jepsen worker 49 - jepsen.util 2929 :invoke :write [16 28] INFO [2022-01-10 05:44:25,639] jepsen worker 33 - jepsen.util 3273 :ok :read [11 21] INFO [2022-01-10 05:44:25,639] jepsen worker 33 - jepsen.util 3273 :invoke :read [11 nil] INFO [2022-01-10 05:44:25,651] jepsen worker 10 - jepsen.util 3250 :info :write [3 27] :RequestTimeoutException INFO [2022-01-10 05:44:25,652] jepsen worker 10 - jepsen.util 3340 :invoke :read [3 nil] INFO [2022-01-10 05:44:25,748] jepsen worker 76 - jepsen.util 2596 :ok :read [25 :nil] INFO [2022-01-10 05:44:25,749] jepsen worker 76 - jepsen.util 2596 :invoke :read [25 nil] INFO [2022-01-10 05:44:25,749] jepsen worker 12 - jepsen.util 3252 :info :write [4 31] :RequestTimeoutException INFO [2022-01-10 05:44:25,750] jepsen worker 12 - jepsen.util 3342 :invoke :read [4 nil] INFO [2022-01-10 05:44:25,790] jepsen worker 6 - jepsen.util 3066 :ok :read [2 14] INFO [2022-01-10 05:44:25,790] jepsen worker 6 - jepsen.util 3066 :invoke :write [2 25] INFO [2022-01-10 05:44:25,791] jepsen worker 46 - jepsen.util 3196 :ok :read [15 23] INFO [2022-01-10 05:44:25,792] jepsen worker 46 - jepsen.util 3196 :invoke :read [15 nil] INFO [2022-01-10 05:44:25,813] jepsen worker 62 - jepsen.util 3302 :info :write [20 36] :RequestTimeoutException INFO [2022-01-10 05:44:25,814] jepsen worker 62 - jepsen.util 3392 :invoke :write [20 17] INFO [2022-01-10 05:44:25,823] jepsen worker 76 - jepsen.util 2596 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:44:25,824] jepsen worker 76 - jepsen.util 2596 :invoke :write [25 33] INFO [2022-01-10 05:44:25,824] jepsen worker 18 - jepsen.util 2988 :ok :read [6 25] INFO [2022-01-10 05:44:25,825] jepsen worker 18 - jepsen.util 2988 :invoke :write [6 13] INFO [2022-01-10 05:44:25,841] jepsen worker 33 - jepsen.util 3273 :ok :read [11 21] INFO [2022-01-10 05:44:25,842] jepsen worker 33 - jepsen.util 3273 :invoke :write [11 30] INFO [2022-01-10 05:44:25,979] jepsen worker 67 - jepsen.util 2947 :ok :read [22 41] INFO [2022-01-10 05:44:25,980] jepsen worker 67 - jepsen.util 2947 :invoke :read [22 nil] INFO [2022-01-10 05:44:26,070] jepsen worker 40 - jepsen.util 2470 :ok :read [13 15] INFO [2022-01-10 05:44:26,071] jepsen worker 40 - jepsen.util 2470 :invoke :write [13 24] INFO [2022-01-10 05:44:26,102] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:26,102] jepsen worker nemesis - couchbase.util Rebalance check count: 417 INFO [2022-01-10 05:44:26,102] jepsen worker nemesis - couchbase.util Rebalance stuck count: 331 INFO [2022-01-10 05:44:26,114] jepsen worker 9 - jepsen.util 3519 :info :write [3 34] :RequestTimeoutException INFO [2022-01-10 05:44:26,116] jepsen worker 9 - jepsen.util 3609 :invoke :read [3 nil] INFO [2022-01-10 05:44:26,127] jepsen worker 39 - jepsen.util 3549 :ok :read [13 15] INFO [2022-01-10 05:44:26,135] jepsen worker 39 - jepsen.util 3549 :invoke :write [13 45] INFO [2022-01-10 05:44:26,140] jepsen worker 86 - jepsen.util 2516 :ok :read [28 45] INFO [2022-01-10 05:44:26,140] jepsen worker 86 - jepsen.util 2516 :invoke :write [28 10] INFO [2022-01-10 05:44:26,144] jepsen worker 9 - jepsen.util 3609 :ok :read [3 31] INFO [2022-01-10 05:44:26,144] jepsen worker 9 - jepsen.util 3609 :invoke :write [3 12] INFO [2022-01-10 05:44:26,211] jepsen worker 68 - jepsen.util 3038 :ok :read [22 41] INFO [2022-01-10 05:44:26,212] jepsen worker 68 - jepsen.util 3038 :invoke :read [22 nil] INFO [2022-01-10 05:44:26,256] jepsen worker 31 - jepsen.util 3361 :info :write [10 30] :RequestTimeoutException INFO [2022-01-10 05:44:26,256] jepsen worker 31 - jepsen.util 3451 :invoke :read [10 nil] INFO [2022-01-10 05:44:26,300] jepsen worker 51 - jepsen.util 3561 :info :write [17 22] :RequestTimeoutException INFO [2022-01-10 05:44:26,301] jepsen worker 51 - jepsen.util 3651 :invoke :write [17 25] INFO [2022-01-10 05:44:26,311] jepsen worker 38 - jepsen.util 3278 :info :write [12 26] :RequestTimeoutException INFO [2022-01-10 05:44:26,312] jepsen worker 38 - jepsen.util 3368 :invoke :write [12 11] INFO [2022-01-10 05:44:26,479] jepsen worker 12 - jepsen.util 3342 :ok :read [4 3] INFO [2022-01-10 05:44:26,480] jepsen worker 12 - jepsen.util 3342 :invoke :write [4 21] INFO [2022-01-10 05:44:26,545] jepsen worker 3 - jepsen.util 3063 :info :write [1 13] :RequestTimeoutException INFO [2022-01-10 05:44:26,546] jepsen worker 3 - jepsen.util 3153 :invoke :read [1 nil] INFO [2022-01-10 05:44:26,571] jepsen worker 54 - jepsen.util 3114 :info :write [18 47] :RequestTimeoutException INFO [2022-01-10 05:44:26,572] jepsen worker 54 - jepsen.util 3204 :invoke :write [18 1] INFO [2022-01-10 05:44:26,980] jepsen worker 0 - jepsen.util 2970 :ok :read [0 46] INFO [2022-01-10 05:44:26,980] jepsen worker 0 - jepsen.util 2970 :invoke :read [0 nil] INFO [2022-01-10 05:44:26,986] jepsen worker 48 - jepsen.util 2658 :info :write [16 0] :RequestTimeoutException INFO [2022-01-10 05:44:26,988] jepsen worker 48 - jepsen.util 2748 :invoke :read [16 nil] INFO [2022-01-10 05:44:27,101] jepsen worker 11 - jepsen.util 2981 :info :write [3 28] :RequestTimeoutException INFO [2022-01-10 05:44:27,102] jepsen worker 11 - jepsen.util 3071 :invoke :read [3 nil] INFO [2022-01-10 05:44:27,110] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:27,110] jepsen worker nemesis - couchbase.util Rebalance check count: 418 INFO [2022-01-10 05:44:27,110] jepsen worker nemesis - couchbase.util Rebalance stuck count: 332 INFO [2022-01-10 05:44:27,233] jepsen worker 68 - jepsen.util 3038 :ok :read [22 41] INFO [2022-01-10 05:44:27,234] jepsen worker 68 - jepsen.util 3038 :invoke :write [22 32] INFO [2022-01-10 05:44:27,322] jepsen worker 83 - jepsen.util 3593 :ok :read [27 30] INFO [2022-01-10 05:44:27,323] jepsen worker 83 - jepsen.util 3593 :invoke :read [27 nil] INFO [2022-01-10 05:44:27,445] jepsen worker 56 - jepsen.util 3386 :info :write [18 19] :RequestTimeoutException INFO [2022-01-10 05:44:27,445] jepsen worker 56 - jepsen.util 3476 :invoke :write [18 45] INFO [2022-01-10 05:44:27,674] jepsen worker 19 - jepsen.util 3169 :ok :read [6 25] INFO [2022-01-10 05:44:27,675] jepsen worker 19 - jepsen.util 3169 :invoke :read [6 nil] INFO [2022-01-10 05:44:27,745] jepsen worker 59 - jepsen.util 2939 :ok :read [19 31] INFO [2022-01-10 05:44:27,745] jepsen worker 59 - jepsen.util 2939 :invoke :write [19 46] INFO [2022-01-10 05:44:27,772] jepsen worker 63 - jepsen.util 2943 :info :write [21 26] :RequestTimeoutException INFO [2022-01-10 05:44:27,772] jepsen worker 63 - jepsen.util 3033 :invoke :read [21 nil] INFO [2022-01-10 05:44:27,992] jepsen worker 37 - jepsen.util 2917 :ok :read [12 29] INFO [2022-01-10 05:44:27,993] jepsen worker 37 - jepsen.util 2917 :invoke :write [12 38] INFO [2022-01-10 05:44:28,041] jepsen worker 31 - jepsen.util 3451 :ok :read [10 36] INFO [2022-01-10 05:44:28,042] jepsen worker 31 - jepsen.util 3451 :invoke :write [10 23] INFO [2022-01-10 05:44:28,050] jepsen worker 11 - jepsen.util 3071 :ok :read [3 31] INFO [2022-01-10 05:44:28,050] jepsen worker 11 - jepsen.util 3071 :invoke :read [3 nil] INFO [2022-01-10 05:44:28,053] jepsen worker 77 - jepsen.util 3407 :ok :read [25 :nil] INFO [2022-01-10 05:44:28,053] jepsen worker 45 - jepsen.util 3285 :ok :read [15 23] INFO [2022-01-10 05:44:28,054] jepsen worker 77 - jepsen.util 3407 :invoke :read [25 nil] INFO [2022-01-10 05:44:28,055] jepsen worker 45 - jepsen.util 3285 :invoke :read [15 nil] INFO [2022-01-10 05:44:28,118] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:28,118] jepsen worker nemesis - couchbase.util Rebalance check count: 419 INFO [2022-01-10 05:44:28,118] jepsen worker nemesis - couchbase.util Rebalance stuck count: 333 INFO [2022-01-10 05:44:28,128] jepsen worker 11 - jepsen.util 3071 :ok :read [3 31] INFO [2022-01-10 05:44:28,129] jepsen worker 11 - jepsen.util 3071 :invoke :read [3 nil] INFO [2022-01-10 05:44:28,142] jepsen worker 89 - jepsen.util 3599 :info :write [29 32] :RequestTimeoutException INFO [2022-01-10 05:44:28,142] jepsen worker 89 - jepsen.util 3689 :invoke :read [29 nil] INFO [2022-01-10 05:44:28,176] jepsen worker 73 - jepsen.util 2773 :ok :read [24 18] INFO [2022-01-10 05:44:28,177] jepsen worker 73 - jepsen.util 2773 :invoke :write [24 19] INFO [2022-01-10 05:44:28,202] jepsen worker 11 - jepsen.util 3071 :ok :read [3 31] INFO [2022-01-10 05:44:28,202] jepsen worker 11 - jepsen.util 3071 :invoke :write [3 21] INFO [2022-01-10 05:44:28,431] jepsen worker 28 - jepsen.util 2818 :ok :read [9 27] INFO [2022-01-10 05:44:28,432] jepsen worker 28 - jepsen.util 2818 :invoke :read [9 nil] INFO [2022-01-10 05:44:28,472] jepsen worker 24 - jepsen.util 2634 :info :write [8 0] :RequestTimeoutException INFO [2022-01-10 05:44:28,473] jepsen worker 24 - jepsen.util 2724 :invoke :read [8 nil] INFO [2022-01-10 05:44:28,524] jepsen worker 48 - jepsen.util 2748 :ok :read [16 38] INFO [2022-01-10 05:44:28,525] jepsen worker 48 - jepsen.util 2748 :invoke :read [16 nil] INFO [2022-01-10 05:44:28,531] jepsen worker 17 - jepsen.util 2987 :ok :read [5 42] INFO [2022-01-10 05:44:28,532] jepsen worker 17 - jepsen.util 2987 :invoke :read [5 nil] INFO [2022-01-10 05:44:28,880] jepsen worker 19 - jepsen.util 3169 :ok :read [6 25] INFO [2022-01-10 05:44:28,881] jepsen worker 19 - jepsen.util 3169 :invoke :write [6 32] INFO [2022-01-10 05:44:28,908] jepsen worker 45 - jepsen.util 3285 :ok :read [15 23] INFO [2022-01-10 05:44:28,909] jepsen worker 45 - jepsen.util 3285 :invoke :read [15 nil] INFO [2022-01-10 05:44:28,913] jepsen worker 63 - jepsen.util 3033 :ok :read [21 7] INFO [2022-01-10 05:44:28,913] jepsen worker 63 - jepsen.util 3033 :invoke :read [21 nil] INFO [2022-01-10 05:44:28,915] jepsen worker 81 - jepsen.util 3231 :info :write [27 12] :RequestTimeoutException INFO [2022-01-10 05:44:28,915] jepsen worker 81 - jepsen.util 3321 :invoke :read [27 nil] INFO [2022-01-10 05:44:28,956] jepsen worker 4 - jepsen.util 3514 :info :write [1 31] :RequestTimeoutException INFO [2022-01-10 05:44:28,956] jepsen worker 4 - jepsen.util 3604 :invoke :read [1 nil] INFO [2022-01-10 05:44:28,988] jepsen worker 67 - jepsen.util 2947 :ok :read [22 41] INFO [2022-01-10 05:44:28,988] jepsen worker 67 - jepsen.util 2947 :invoke :write [22 14] INFO [2022-01-10 05:44:29,125] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:29,125] jepsen worker nemesis - couchbase.util Rebalance check count: 420 INFO [2022-01-10 05:44:29,125] jepsen worker nemesis - couchbase.util Rebalance stuck count: 334 INFO [2022-01-10 05:44:29,562] jepsen worker 0 - jepsen.util 2970 :ok :read [0 46] INFO [2022-01-10 05:44:29,563] jepsen worker 0 - jepsen.util 2970 :invoke :read [0 nil] INFO [2022-01-10 05:44:29,673] jepsen worker 23 - jepsen.util 3173 :ok :read [7 :nil] INFO [2022-01-10 05:44:29,674] jepsen worker 23 - jepsen.util 3173 :invoke :read [7 nil] INFO [2022-01-10 05:44:30,026] jepsen worker 16 - jepsen.util 3076 :info :write [5 42] :RequestTimeoutException INFO [2022-01-10 05:44:30,026] jepsen worker 45 - jepsen.util 3285 :ok :read [15 23] INFO [2022-01-10 05:44:30,027] jepsen worker 16 - jepsen.util 3166 :invoke :write [5 14] INFO [2022-01-10 05:44:30,027] jepsen worker 21 - jepsen.util 3261 :ok :read [7 :nil] INFO [2022-01-10 05:44:30,027] jepsen worker 45 - jepsen.util 3285 :invoke :write [15 44] INFO [2022-01-10 05:44:30,028] jepsen worker 21 - jepsen.util 3261 :invoke :write [7 17] INFO [2022-01-10 05:44:30,036] jepsen worker 63 - jepsen.util 3033 :ok :read [21 7] INFO [2022-01-10 05:44:30,037] jepsen worker 63 - jepsen.util 3033 :invoke :write [21 27] INFO [2022-01-10 05:44:30,042] jepsen worker 47 - jepsen.util 3107 :ok :read [15 23] INFO [2022-01-10 05:44:30,043] jepsen worker 47 - jepsen.util 3107 :invoke :write [15 14] INFO [2022-01-10 05:44:30,132] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:30,132] jepsen worker nemesis - couchbase.util Rebalance check count: 421 INFO [2022-01-10 05:44:30,132] jepsen worker nemesis - couchbase.util Rebalance stuck count: 335 INFO [2022-01-10 05:44:30,156] jepsen worker 29 - jepsen.util 2819 :info :write [9 17] :RequestTimeoutException INFO [2022-01-10 05:44:30,157] jepsen worker 29 - jepsen.util 2909 :invoke :write [9 44] INFO [2022-01-10 05:44:30,261] jepsen worker 7 - jepsen.util 3067 :info :write [2 25] :RequestTimeoutException INFO [2022-01-10 05:44:30,261] jepsen worker 7 - jepsen.util 3157 :invoke :read [2 nil] WARN [2022-01-10 05:44:30,306] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17316099,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17723624,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9284285,"encode_duration_us":18694,"operation_id":"0x5e24","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10007580},{"operation_name":"upsert","last_dispatch_duration_us":11623159,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11623159,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":6213512,"encode_duration_us":16938,"operation_id":"0x5f4f","timeout_ms":10000,"last_server_duration_us":6213512,"total_duration_us":10008498}],"total_count":2}} WARN [2022-01-10 05:44:30,307] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11867172,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":11867172,"total_server_duration_us":56,"operation_id":"0x5f54","timeout_ms":30000,"last_server_duration_us":56,"total_duration_us":11868420},{"operation_name":"get","last_dispatch_duration_us":10314727,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10314727,"total_server_duration_us":21405,"operation_id":"0x5fae","timeout_ms":30000,"last_server_duration_us":21405,"total_duration_us":10316908},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":18356,"operation_id":"0x5fea","timeout_ms":10000,"total_duration_us":10009984},{"operation_name":"upsert","last_dispatch_duration_us":303638,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10261809,"total_server_duration_us":1344228,"encode_duration_us":16541,"operation_id":"0x6063","timeout_ms":10000,"last_server_duration_us":135047,"total_duration_us":10009432},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":22895,"operation_id":"0x5f83","timeout_ms":10000,"total_duration_us":10009335},{"operation_name":"upsert","last_dispatch_duration_us":11623159,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11623159,"total_server_duration_us":6213512,"encode_duration_us":16938,"operation_id":"0x5f4f","timeout_ms":10000,"last_server_duration_us":6213512,"total_duration_us":10008498},{"operation_name":"upsert","last_dispatch_duration_us":7613574,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":13198282,"total_server_duration_us":508383,"encode_duration_us":20059,"operation_id":"0x5f11","timeout_ms":10000,"last_server_duration_us":209548,"total_duration_us":10008114},{"operation_name":"upsert","last_dispatch_duration_us":277893,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":7814028,"total_server_duration_us":2345598,"encode_duration_us":16229,"operation_id":"0x5ee2","timeout_ms":10000,"last_server_duration_us":184733,"total_duration_us":10006975},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":16144,"operation_id":"0x5fdb","timeout_ms":10000,"total_duration_us":10005915},{"operation_name":"upsert","last_dispatch_duration_us":8200424,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14047896,"total_server_duration_us":4243861,"encode_duration_us":16768,"operation_id":"0x5eea","timeout_ms":10000,"last_server_duration_us":312317,"total_duration_us":10004565}],"total_count":26}} INFO [2022-01-10 05:44:30,314] jepsen worker 83 - jepsen.util 3593 :ok :read [27 30] INFO [2022-01-10 05:44:30,314] jepsen worker 83 - jepsen.util 3593 :invoke :read [27 nil] INFO [2022-01-10 05:44:30,345] jepsen worker 52 - jepsen.util 3832 :info :write [17 13] :RequestTimeoutException INFO [2022-01-10 05:44:30,346] jepsen worker 52 - jepsen.util 3922 :invoke :read [17 nil] INFO [2022-01-10 05:44:30,389] jepsen worker 28 - jepsen.util 2818 :ok :read [9 27] INFO [2022-01-10 05:44:30,390] jepsen worker 28 - jepsen.util 2818 :invoke :write [9 49] INFO [2022-01-10 05:44:30,393] jepsen worker 48 - jepsen.util 2748 :ok :read [16 38] INFO [2022-01-10 05:44:30,393] jepsen worker 48 - jepsen.util 2748 :invoke :read [16 nil] INFO [2022-01-10 05:44:30,410] jepsen worker 17 - jepsen.util 2987 :ok :read [5 42] INFO [2022-01-10 05:44:30,411] jepsen worker 17 - jepsen.util 2987 :invoke :write [5 3] INFO [2022-01-10 05:44:30,580] jepsen worker 81 - jepsen.util 3321 :ok :read [27 30] INFO [2022-01-10 05:44:30,581] jepsen worker 81 - jepsen.util 3321 :invoke :read [27 nil] INFO [2022-01-10 05:44:30,598] jepsen worker 83 - jepsen.util 3593 :ok :read [27 30] INFO [2022-01-10 05:44:30,599] jepsen worker 83 - jepsen.util 3593 :invoke :write [27 16] INFO [2022-01-10 05:44:30,808] jepsen worker 25 - jepsen.util 2905 :ok :read [8 27] INFO [2022-01-10 05:44:30,809] jepsen worker 25 - jepsen.util 2905 :invoke :read [8 nil] INFO [2022-01-10 05:44:30,943] jepsen worker 52 - jepsen.util 3922 :ok :read [17 41] INFO [2022-01-10 05:44:30,943] jepsen worker 52 - jepsen.util 3922 :invoke :read [17 nil] INFO [2022-01-10 05:44:30,959] jepsen worker 77 - jepsen.util 3407 :ok :read [25 :nil] INFO [2022-01-10 05:44:30,960] jepsen worker 77 - jepsen.util 3407 :invoke :read [25 nil] INFO [2022-01-10 05:44:30,965] jepsen worker 78 - jepsen.util 3588 :info :write [26 6] :RequestTimeoutException INFO [2022-01-10 05:44:30,965] jepsen worker 78 - jepsen.util 3678 :invoke :read [26 nil] INFO [2022-01-10 05:44:30,983] jepsen worker 23 - jepsen.util 3173 :ok :read [7 :nil] INFO [2022-01-10 05:44:30,984] jepsen worker 23 - jepsen.util 3173 :invoke :write [7 33] INFO [2022-01-10 05:44:30,998] jepsen worker 43 - jepsen.util 2923 :ok :read [14 14] INFO [2022-01-10 05:44:30,998] jepsen worker 43 - jepsen.util 2923 :invoke :write [14 45] INFO [2022-01-10 05:44:31,141] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:31,141] jepsen worker nemesis - couchbase.util Rebalance check count: 422 INFO [2022-01-10 05:44:31,141] jepsen worker nemesis - couchbase.util Rebalance stuck count: 336 INFO [2022-01-10 05:44:31,171] jepsen worker 61 - jepsen.util 3121 :info :write [20 9] :RequestTimeoutException INFO [2022-01-10 05:44:31,172] jepsen worker 61 - jepsen.util 3211 :invoke :write [20 32] INFO [2022-01-10 05:44:31,291] jepsen worker 50 - jepsen.util 3110 :info :write [16 49] :RequestTimeoutException INFO [2022-01-10 05:44:31,292] jepsen worker 50 - jepsen.util 3200 :invoke :read [16 nil] INFO [2022-01-10 05:44:31,311] jepsen worker 20 - jepsen.util 2900 :ok :read [6 25] INFO [2022-01-10 05:44:31,311] jepsen worker 20 - jepsen.util 2900 :invoke :read [6 nil] INFO [2022-01-10 05:44:31,570] jepsen worker 57 - jepsen.util 3657 :info :write [19 49] :RequestTimeoutException INFO [2022-01-10 05:44:31,571] jepsen worker 57 - jepsen.util 3747 :invoke :write [19 9] INFO [2022-01-10 05:44:31,639] jepsen worker 48 - jepsen.util 2748 :ok :read [16 38] INFO [2022-01-10 05:44:31,640] jepsen worker 48 - jepsen.util 2748 :invoke :write [16 43] INFO [2022-01-10 05:44:31,655] jepsen worker 0 - jepsen.util 2970 :ok :read [0 46] INFO [2022-01-10 05:44:31,655] jepsen worker 0 - jepsen.util 2970 :invoke :read [0 nil] INFO [2022-01-10 05:44:31,661] jepsen worker 81 - jepsen.util 3321 :ok :read [27 30] INFO [2022-01-10 05:44:31,662] jepsen worker 81 - jepsen.util 3321 :invoke :read [27 nil] INFO [2022-01-10 05:44:31,807] jepsen worker 78 - jepsen.util 3678 :ok :read [26 10] INFO [2022-01-10 05:44:31,808] jepsen worker 78 - jepsen.util 3678 :invoke :read [26 nil] INFO [2022-01-10 05:44:31,827] jepsen worker 52 - jepsen.util 3922 :ok :read [17 41] INFO [2022-01-10 05:44:31,828] jepsen worker 52 - jepsen.util 3922 :invoke :read [17 nil] INFO [2022-01-10 05:44:31,835] jepsen worker 78 - jepsen.util 3678 :ok :read [26 10] INFO [2022-01-10 05:44:31,836] jepsen worker 78 - jepsen.util 3678 :invoke :write [26 8] WARN [2022-01-10 05:44:31,884] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12805091,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12805091,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9021452,"encode_duration_us":16521,"operation_id":"0x5fe2","timeout_ms":10000,"last_server_duration_us":9021452,"total_duration_us":10006345}],"total_count":1}} WARN [2022-01-10 05:44:31,905] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":5066438,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11622266,"total_server_duration_us":1815791,"encode_duration_us":18289,"operation_id":"0x5f8a","timeout_ms":10000,"last_server_duration_us":291563,"total_duration_us":10013169},{"operation_name":"upsert","last_dispatch_duration_us":2672538,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9765166,"total_server_duration_us":1055538,"encode_duration_us":18824,"operation_id":"0x60c9","timeout_ms":10000,"last_server_duration_us":22141,"total_duration_us":10008714},{"operation_name":"upsert","last_dispatch_duration_us":3509697,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8333074,"total_server_duration_us":359004,"encode_duration_us":18599,"operation_id":"0x6099","timeout_ms":10000,"last_server_duration_us":99658,"total_duration_us":10008441},{"operation_name":"upsert","last_dispatch_duration_us":138179,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":904048,"total_server_duration_us":423956,"encode_duration_us":18157,"operation_id":"0x5feb","timeout_ms":10000,"last_server_duration_us":18493,"total_duration_us":10008413},{"operation_name":"upsert","last_dispatch_duration_us":31033,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":620597,"total_server_duration_us":368918,"encode_duration_us":16349,"operation_id":"0x6041","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10007442},{"operation_name":"upsert","last_dispatch_duration_us":1709623,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11346682,"total_server_duration_us":471745,"encode_duration_us":17621,"operation_id":"0x5f4c","timeout_ms":10000,"last_server_duration_us":179689,"total_duration_us":10006604},{"operation_name":"upsert","last_dispatch_duration_us":2319800,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8503635,"total_server_duration_us":459630,"encode_duration_us":17717,"operation_id":"0x60c8","timeout_ms":10000,"last_server_duration_us":110674,"total_duration_us":10006302},{"operation_name":"upsert","last_dispatch_duration_us":31405,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":4581337,"total_server_duration_us":405182,"encode_duration_us":18089,"operation_id":"0x5f0f","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10005107},{"operation_name":"upsert","last_dispatch_duration_us":111825,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":808898,"total_server_duration_us":423744,"encode_duration_us":17238,"operation_id":"0x600f","timeout_ms":10000,"last_server_duration_us":24325,"total_duration_us":10004461},{"operation_name":"upsert","last_dispatch_duration_us":4612047,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11686022,"total_server_duration_us":382748,"encode_duration_us":16999,"operation_id":"0x5fd4","timeout_ms":10000,"last_server_duration_us":204860,"total_duration_us":10004215}],"total_count":24}} WARN [2022-01-10 05:44:31,907] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":3072596,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":3072596,"total_server_duration_us":32,"operation_id":"0x6090","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":11299489},{"operation_name":"upsert","last_dispatch_duration_us":3476745,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12930030,"total_server_duration_us":422447,"encode_duration_us":18512,"operation_id":"0x6042","timeout_ms":10000,"last_server_duration_us":291809,"total_duration_us":10011018},{"operation_name":"upsert","last_dispatch_duration_us":2557585,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11110329,"total_server_duration_us":534068,"encode_duration_us":18969,"operation_id":"0x5f79","timeout_ms":10000,"last_server_duration_us":281050,"total_duration_us":10009065},{"operation_name":"upsert","last_dispatch_duration_us":2477009,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":6598220,"total_server_duration_us":947526,"encode_duration_us":16369,"operation_id":"0x5f9f","timeout_ms":10000,"last_server_duration_us":701395,"total_duration_us":10007442},{"operation_name":"upsert","last_dispatch_duration_us":32632,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8811035,"total_server_duration_us":243520,"encode_duration_us":21022,"operation_id":"0x5fb8","timeout_ms":10000,"last_server_duration_us":20441,"total_duration_us":10005577},{"operation_name":"upsert","last_dispatch_duration_us":32488,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8378657,"total_server_duration_us":254376,"encode_duration_us":17864,"operation_id":"0x5fe4","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10005380},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16781,"operation_id":"0x602c","timeout_ms":10000,"total_duration_us":10002633},{"operation_name":"upsert","last_dispatch_duration_us":707255,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":7987002,"total_server_duration_us":4216983,"encode_duration_us":17967,"operation_id":"0x5f63","timeout_ms":10000,"last_server_duration_us":379006,"total_duration_us":10001118},{"operation_name":"upsert","last_dispatch_duration_us":10373482,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10373482,"total_server_duration_us":835997,"encode_duration_us":19596,"operation_id":"0x5f87","timeout_ms":10000,"last_server_duration_us":835997,"total_duration_us":10001094},{"operation_name":"get","last_dispatch_duration_us":8627688,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8627688,"total_server_duration_us":277663,"operation_id":"0x600c","timeout_ms":30000,"last_server_duration_us":277663,"total_duration_us":8631350}],"total_count":20}} WARN [2022-01-10 05:44:31,910] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12504075,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12504075,"total_server_duration_us":27,"operation_id":"0x5fb9","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":12505336},{"operation_name":"upsert","last_dispatch_duration_us":4645059,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9549214,"total_server_duration_us":539298,"encode_duration_us":16321,"operation_id":"0x6091","timeout_ms":10000,"last_server_duration_us":109047,"total_duration_us":10008816},{"operation_name":"upsert","last_dispatch_duration_us":2880345,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9659179,"total_server_duration_us":1795211,"encode_duration_us":16870,"operation_id":"0x5fac","timeout_ms":10000,"last_server_duration_us":800460,"total_duration_us":10007860},{"operation_name":"upsert","last_dispatch_duration_us":5821805,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":14272136,"total_server_duration_us":543204,"encode_duration_us":16278,"operation_id":"0x5f21","timeout_ms":10000,"last_server_duration_us":152553,"total_duration_us":10007104},{"operation_name":"upsert","last_dispatch_duration_us":5850961,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":15159361,"total_server_duration_us":1057614,"encode_duration_us":18739,"operation_id":"0x5f4a","timeout_ms":10000,"last_server_duration_us":63439,"total_duration_us":10003952},{"operation_name":"upsert","last_dispatch_duration_us":38610,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9587009,"total_server_duration_us":186985,"encode_duration_us":17220,"operation_id":"0x6066","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10003428},{"operation_name":"upsert","last_dispatch_duration_us":4126775,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9550130,"total_server_duration_us":376621,"encode_duration_us":16797,"operation_id":"0x6098","timeout_ms":10000,"last_server_duration_us":67089,"total_duration_us":10002945},{"operation_name":"upsert","last_dispatch_duration_us":1750300,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":8914474,"total_server_duration_us":2673854,"encode_duration_us":18619,"operation_id":"0x5f4b","timeout_ms":10000,"last_server_duration_us":410675,"total_duration_us":10002843},{"operation_name":"upsert","last_dispatch_duration_us":5344075,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":12218456,"total_server_duration_us":372953,"encode_duration_us":16785,"operation_id":"0x600d","timeout_ms":10000,"last_server_duration_us":235231,"total_duration_us":10001848},{"operation_name":"upsert","last_dispatch_duration_us":1780790,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9421954,"total_server_duration_us":1939178,"encode_duration_us":17216,"operation_id":"0x5f51","timeout_ms":10000,"last_server_duration_us":789890,"total_duration_us":10001224}],"total_count":23}} INFO [2022-01-10 05:44:31,917] jepsen worker 65 - jepsen.util 2855 :info :write [21 21] :RequestTimeoutException INFO [2022-01-10 05:44:31,917] jepsen worker 65 - jepsen.util 2945 :invoke :read [21 nil] WARN [2022-01-10 05:44:31,984] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":2967468,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2967468,"total_server_duration_us":15410,"operation_id":"0x5faf","timeout_ms":30000,"last_server_duration_us":15410,"total_duration_us":15787803},{"operation_name":"get","last_dispatch_duration_us":2565521,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2565521,"total_server_duration_us":33918,"operation_id":"0x6018","timeout_ms":30000,"last_server_duration_us":33918,"total_duration_us":13345277},{"operation_name":"get","last_dispatch_duration_us":3987341,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3987341,"total_server_duration_us":2814,"operation_id":"0x600a","timeout_ms":30000,"last_server_duration_us":2814,"total_duration_us":12725540},{"operation_name":"get","last_dispatch_duration_us":2437243,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2437243,"total_server_duration_us":27,"operation_id":"0x6062","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":11172587},{"operation_name":"get","last_dispatch_duration_us":4901671,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":4901671,"total_server_duration_us":301739,"operation_id":"0x5f76","timeout_ms":30000,"last_server_duration_us":301739,"total_duration_us":10066814},{"operation_name":"upsert","last_dispatch_duration_us":10877084,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10877084,"total_server_duration_us":381212,"encode_duration_us":19115,"operation_id":"0x5f8b","timeout_ms":10000,"last_server_duration_us":381212,"total_duration_us":10010198},{"operation_name":"upsert","last_dispatch_duration_us":5737868,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5737868,"total_server_duration_us":338649,"encode_duration_us":20743,"operation_id":"0x5f5f","timeout_ms":10000,"last_server_duration_us":338649,"total_duration_us":10009980},{"operation_name":"upsert","last_dispatch_duration_us":9607912,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":14293235,"total_server_duration_us":706376,"encode_duration_us":24545,"operation_id":"0x5f5b","timeout_ms":10000,"last_server_duration_us":120480,"total_duration_us":10007854},{"operation_name":"upsert","last_dispatch_duration_us":10758399,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10758399,"total_server_duration_us":221902,"encode_duration_us":16948,"operation_id":"0x5f12","timeout_ms":10000,"last_server_duration_us":221902,"total_duration_us":10007377},{"operation_name":"upsert","last_dispatch_duration_us":12805091,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":12805091,"total_server_duration_us":9021452,"encode_duration_us":16521,"operation_id":"0x5fe2","timeout_ms":10000,"last_server_duration_us":9021452,"total_duration_us":10006345}],"total_count":20}} WARN [2022-01-10 05:44:31,989] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10687319,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11627192,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9619564,"encode_duration_us":16598,"operation_id":"0x5dd7","timeout_ms":10000,"last_server_duration_us":9511650,"total_duration_us":10001490},{"operation_name":"upsert","last_dispatch_duration_us":18105395,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18105395,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9037366,"encode_duration_us":18090,"operation_id":"0x5dc0","timeout_ms":10000,"last_server_duration_us":9037366,"total_duration_us":10003058}],"total_count":2}} WARN [2022-01-10 05:44:32,011] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16927148,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18502617,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9360431,"encode_duration_us":16481,"operation_id":"0x5dee","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10008909}],"total_count":1}} INFO [2022-01-10 05:44:32,013] jepsen worker 52 - jepsen.util 3922 :ok :read [17 41] INFO [2022-01-10 05:44:32,014] jepsen worker 52 - jepsen.util 3922 :invoke :read [17 nil] INFO [2022-01-10 05:44:32,096] jepsen worker 30 - jepsen.util 2910 :info :write [10 21] :RequestTimeoutException INFO [2022-01-10 05:44:32,097] jepsen worker 30 - jepsen.util 3000 :invoke :read [10 nil] INFO [2022-01-10 05:44:32,102] jepsen worker 26 - jepsen.util 2636 :info :write [8 34] :RequestTimeoutException INFO [2022-01-10 05:44:32,103] jepsen worker 26 - jepsen.util 2726 :invoke :write [8 10] WARN [2022-01-10 05:44:32,108] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9036712,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16072579,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9261697,"encode_duration_us":17498,"operation_id":"0x5e17","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10001911}],"total_count":1}} INFO [2022-01-10 05:44:32,147] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:32,147] jepsen worker nemesis - couchbase.util Rebalance check count: 423 INFO [2022-01-10 05:44:32,147] jepsen worker nemesis - couchbase.util Rebalance stuck count: 337 WARN [2022-01-10 05:44:32,204] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12453421,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":12453421,"total_server_duration_us":49,"operation_id":"0x5edb","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":12454370},{"operation_name":"get","last_dispatch_duration_us":2984124,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2984124,"total_server_duration_us":16629,"operation_id":"0x5f77","timeout_ms":30000,"last_server_duration_us":16629,"total_duration_us":11719818},{"operation_name":"upsert","last_dispatch_duration_us":9092322,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9092322,"total_server_duration_us":23729,"encode_duration_us":25348,"operation_id":"0x5f84","timeout_ms":10000,"last_server_duration_us":23729,"total_duration_us":10010751},{"operation_name":"upsert","last_dispatch_duration_us":2357831,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10389156,"total_server_duration_us":2730713,"encode_duration_us":17075,"operation_id":"0x5fe3","timeout_ms":10000,"last_server_duration_us":307513,"total_duration_us":10009433},{"operation_name":"upsert","last_dispatch_duration_us":12057548,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12962365,"total_server_duration_us":933647,"encode_duration_us":17203,"operation_id":"0x5f20","timeout_ms":10000,"last_server_duration_us":891122,"total_duration_us":10009033},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":17056,"operation_id":"0x5f89","timeout_ms":10000,"total_duration_us":10009024},{"operation_name":"upsert","last_dispatch_duration_us":2467020,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10220955,"total_server_duration_us":3413493,"encode_duration_us":16196,"operation_id":"0x5fc7","timeout_ms":10000,"last_server_duration_us":238388,"total_duration_us":10007301},{"operation_name":"upsert","last_dispatch_duration_us":1535585,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9951967,"total_server_duration_us":541339,"encode_duration_us":16229,"operation_id":"0x60d0","timeout_ms":10000,"last_server_duration_us":47192,"total_duration_us":10006891},{"operation_name":"upsert","last_dispatch_duration_us":2013613,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11199810,"total_server_duration_us":593621,"encode_duration_us":16457,"operation_id":"0x5f8c","timeout_ms":10000,"last_server_duration_us":304746,"total_duration_us":10006103},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":23751,"operation_id":"0x5f86","timeout_ms":10000,"total_duration_us":10005523}],"total_count":26}} INFO [2022-01-10 05:44:32,215] jepsen worker 80 - jepsen.util 3680 :info :write [26 18] :RequestTimeoutException INFO [2022-01-10 05:44:32,216] jepsen worker 80 - jepsen.util 3770 :invoke :read [26 nil] INFO [2022-01-10 05:44:32,281] jepsen worker 80 - jepsen.util 3770 :ok :read [26 10] INFO [2022-01-10 05:44:32,281] jepsen worker 80 - jepsen.util 3770 :invoke :read [26 nil] INFO [2022-01-10 05:44:32,353] jepsen worker 80 - jepsen.util 3770 :ok :read [26 10] INFO [2022-01-10 05:44:32,353] jepsen worker 80 - jepsen.util 3770 :invoke :write [26 27] INFO [2022-01-10 05:44:32,376] jepsen worker 52 - jepsen.util 3922 :ok :read [17 41] INFO [2022-01-10 05:44:32,377] jepsen worker 52 - jepsen.util 3922 :invoke :write [17 17] INFO [2022-01-10 05:44:32,427] jepsen worker 7 - jepsen.util 3157 :ok :read [2 14] INFO [2022-01-10 05:44:32,427] jepsen worker 7 - jepsen.util 3157 :invoke :write [2 5] INFO [2022-01-10 05:44:32,664] jepsen worker 55 - jepsen.util 3385 :ok :read [18 14] INFO [2022-01-10 05:44:32,665] jepsen worker 55 - jepsen.util 3385 :invoke :read [18 nil] INFO [2022-01-10 05:44:32,856] jepsen worker 55 - jepsen.util 3385 :ok :read [18 14] INFO [2022-01-10 05:44:32,856] jepsen worker 55 - jepsen.util 3385 :invoke :write [18 11] INFO [2022-01-10 05:44:33,153] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:33,153] jepsen worker nemesis - couchbase.util Rebalance check count: 424 INFO [2022-01-10 05:44:33,153] jepsen worker nemesis - couchbase.util Rebalance stuck count: 338 INFO [2022-01-10 05:44:33,216] jepsen worker 79 - jepsen.util 3499 :info :write [26 19] :RequestTimeoutException INFO [2022-01-10 05:44:33,217] jepsen worker 79 - jepsen.util 3589 :invoke :write [26 39] INFO [2022-01-10 05:44:33,236] jepsen worker 27 - jepsen.util 3087 :info :write [9 3] :RequestTimeoutException INFO [2022-01-10 05:44:33,237] jepsen worker 27 - jepsen.util 3177 :invoke :read [9 nil] INFO [2022-01-10 05:44:33,279] jepsen worker 53 - jepsen.util 3653 :info :write [17 27] :RequestTimeoutException INFO [2022-01-10 05:44:33,280] jepsen worker 53 - jepsen.util 3743 :invoke :write [17 12] INFO [2022-01-10 05:44:33,341] jepsen worker 58 - jepsen.util 2938 :info :write [19 11] :RequestTimeoutException INFO [2022-01-10 05:44:33,341] jepsen worker 58 - jepsen.util 3028 :invoke :write [19 17] INFO [2022-01-10 05:44:33,352] jepsen worker 2 - jepsen.util 3242 :info :write [0 44] :RequestTimeoutException INFO [2022-01-10 05:44:33,352] jepsen worker 2 - jepsen.util 3332 :invoke :read [0 nil] INFO [2022-01-10 05:44:33,422] jepsen worker 13 - jepsen.util 3433 :info :write [4 33] :RequestTimeoutException INFO [2022-01-10 05:44:33,423] jepsen worker 13 - jepsen.util 3523 :invoke :write [4 12] INFO [2022-01-10 05:44:33,467] jepsen worker 81 - jepsen.util 3321 :ok :read [27 30] INFO [2022-01-10 05:44:33,468] jepsen worker 81 - jepsen.util 3321 :invoke :write [27 40] INFO [2022-01-10 05:44:33,480] jepsen worker 32 - jepsen.util 3272 :info :write [10 47] :RequestTimeoutException INFO [2022-01-10 05:44:33,485] jepsen worker 32 - jepsen.util 3362 :invoke :write [10 7] INFO [2022-01-10 05:44:33,499] jepsen worker 69 - jepsen.util 2769 :info :write [23 18] :RequestTimeoutException INFO [2022-01-10 05:44:33,500] jepsen worker 69 - jepsen.util 2859 :invoke :read [23 nil] INFO [2022-01-10 05:44:33,531] jepsen worker 77 - jepsen.util 3407 :ok :read [25 :nil] INFO [2022-01-10 05:44:33,532] jepsen worker 77 - jepsen.util 3407 :invoke :read [25 nil] INFO [2022-01-10 05:44:33,546] jepsen worker 25 - jepsen.util 2905 :ok :read [8 27] INFO [2022-01-10 05:44:33,547] jepsen worker 25 - jepsen.util 2905 :invoke :write [8 35] INFO [2022-01-10 05:44:33,611] jepsen worker 35 - jepsen.util 2645 :info :write [11 46] :RequestTimeoutException INFO [2022-01-10 05:44:33,612] jepsen worker 35 - jepsen.util 2735 :invoke :read [11 nil] INFO [2022-01-10 05:44:33,664] jepsen worker 44 - jepsen.util 3104 :info :write [14 30] :RequestTimeoutException INFO [2022-01-10 05:44:33,665] jepsen worker 44 - jepsen.util 3194 :invoke :write [14 32] INFO [2022-01-10 05:44:33,735] jepsen worker 75 - jepsen.util 2505 :info :write [25 22] :RequestTimeoutException INFO [2022-01-10 05:44:33,736] jepsen worker 75 - jepsen.util 2595 :invoke :write [25 49] INFO [2022-01-10 05:44:33,750] jepsen worker 14 - jepsen.util 3704 :info :write [4 38] :RequestTimeoutException INFO [2022-01-10 05:44:33,751] jepsen worker 14 - jepsen.util 3794 :invoke :read [4 nil] INFO [2022-01-10 05:44:33,751] jepsen worker 42 - jepsen.util 3012 :info :write [14 24] :RequestTimeoutException INFO [2022-01-10 05:44:33,752] jepsen worker 42 - jepsen.util 3102 :invoke :write [14 1] INFO [2022-01-10 05:44:33,811] jepsen worker 84 - jepsen.util 3054 :info :write [28 48] :RequestTimeoutException INFO [2022-01-10 05:44:33,812] jepsen worker 84 - jepsen.util 3144 :invoke :read [28 nil] INFO [2022-01-10 05:44:33,821] jepsen worker 1 - jepsen.util 3061 :info :write [0 10] :RequestTimeoutException INFO [2022-01-10 05:44:33,822] jepsen worker 1 - jepsen.util 3151 :invoke :write [0 0] INFO [2022-01-10 05:44:33,945] jepsen worker 0 - jepsen.util 2970 :ok :read [0 46] INFO [2022-01-10 05:44:33,945] jepsen worker 41 - jepsen.util 2921 :info :write [13 47] :RequestTimeoutException INFO [2022-01-10 05:44:33,946] jepsen worker 0 - jepsen.util 2970 :invoke :write [0 19] INFO [2022-01-10 05:44:33,946] jepsen worker 41 - jepsen.util 3011 :invoke :read [13 nil] INFO [2022-01-10 05:44:34,004] jepsen worker 27 - jepsen.util 3177 :ok :read [9 27] INFO [2022-01-10 05:44:34,005] jepsen worker 27 - jepsen.util 3177 :invoke :write [9 0] INFO [2022-01-10 05:44:34,126] jepsen worker 85 - jepsen.util 3325 :info :write [28 23] :RequestTimeoutException INFO [2022-01-10 05:44:34,127] jepsen worker 85 - jepsen.util 3415 :invoke :write [28 26] INFO [2022-01-10 05:44:34,146] jepsen worker 14 - jepsen.util 3794 :ok :read [4 3] INFO [2022-01-10 05:44:34,146] jepsen worker 14 - jepsen.util 3794 :invoke :read [4 nil] INFO [2022-01-10 05:44:34,158] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:34,159] jepsen worker nemesis - couchbase.util Rebalance check count: 425 INFO [2022-01-10 05:44:34,159] jepsen worker nemesis - couchbase.util Rebalance stuck count: 339 INFO [2022-01-10 05:44:34,174] jepsen worker 14 - jepsen.util 3794 :ok :read [4 3] INFO [2022-01-10 05:44:34,174] jepsen worker 14 - jepsen.util 3794 :invoke :read [4 nil] INFO [2022-01-10 05:44:34,227] jepsen worker 14 - jepsen.util 3794 :ok :read [4 3] INFO [2022-01-10 05:44:34,228] jepsen worker 14 - jepsen.util 3794 :invoke :write [4 3] INFO [2022-01-10 05:44:34,279] jepsen worker 2 - jepsen.util 3332 :ok :read [0 46] INFO [2022-01-10 05:44:34,280] jepsen worker 2 - jepsen.util 3332 :invoke :read [0 nil] INFO [2022-01-10 05:44:34,445] jepsen worker 50 - jepsen.util 3200 :ok :read [16 38] INFO [2022-01-10 05:44:34,446] jepsen worker 50 - jepsen.util 3200 :invoke :write [16 41] INFO [2022-01-10 05:44:34,472] jepsen worker 8 - jepsen.util 3518 :info :write [2 47] :RequestTimeoutException INFO [2022-01-10 05:44:34,473] jepsen worker 8 - jepsen.util 3608 :invoke :read [2 nil] INFO [2022-01-10 05:44:34,484] jepsen worker 5 - jepsen.util 2975 :info :write [1 24] :RequestTimeoutException INFO [2022-01-10 05:44:34,484] jepsen worker 64 - jepsen.util 3034 :info :write [21 40] :RequestTimeoutException INFO [2022-01-10 05:44:34,485] jepsen worker 5 - jepsen.util 3065 :invoke :write [1 6] INFO [2022-01-10 05:44:34,486] jepsen worker 64 - jepsen.util 3124 :invoke :read [21 nil] INFO [2022-01-10 05:44:34,512] jepsen worker 84 - jepsen.util 3144 :ok :read [28 45] INFO [2022-01-10 05:44:34,513] jepsen worker 84 - jepsen.util 3144 :invoke :write [28 49] INFO [2022-01-10 05:44:34,599] jepsen worker 65 - jepsen.util 2945 :ok :read [21 7] INFO [2022-01-10 05:44:34,599] jepsen worker 65 - jepsen.util 2945 :invoke :write [21 29] INFO [2022-01-10 05:44:34,601] jepsen worker 88 - jepsen.util 3058 :ok :read [29 0] INFO [2022-01-10 05:44:34,602] jepsen worker 88 - jepsen.util 3058 :invoke :write [29 28] INFO [2022-01-10 05:44:34,636] jepsen worker 72 - jepsen.util 3402 :info :write [24 11] :RequestTimeoutException INFO [2022-01-10 05:44:34,636] jepsen worker 72 - jepsen.util 3492 :invoke :write [24 28] INFO [2022-01-10 05:44:34,792] jepsen worker 66 - jepsen.util 2766 :info :write [22 27] :RequestTimeoutException INFO [2022-01-10 05:44:34,792] jepsen worker 66 - jepsen.util 2856 :invoke :write [22 12] INFO [2022-01-10 05:44:35,138] jepsen worker 64 - jepsen.util 3124 :ok :read [21 7] INFO [2022-01-10 05:44:35,139] jepsen worker 64 - jepsen.util 3124 :invoke :write [21 49] INFO [2022-01-10 05:44:35,160] jepsen worker 82 - jepsen.util 3142 :info :write [27 37] :RequestTimeoutException INFO [2022-01-10 05:44:35,161] jepsen worker 82 - jepsen.util 3232 :invoke :write [27 48] INFO [2022-01-10 05:44:35,164] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:35,164] jepsen worker nemesis - couchbase.util Rebalance check count: 426 INFO [2022-01-10 05:44:35,164] jepsen worker nemesis - couchbase.util Rebalance stuck count: 340 INFO [2022-01-10 05:44:35,181] jepsen worker 22 - jepsen.util 2812 :ok :read [7 :nil] INFO [2022-01-10 05:44:35,182] jepsen worker 22 - jepsen.util 2812 :invoke :write [7 29] INFO [2022-01-10 05:44:35,193] jepsen worker 74 - jepsen.util 3674 :ok :read [24 18] INFO [2022-01-10 05:44:35,193] jepsen worker 74 - jepsen.util 3674 :invoke :read [24 nil] INFO [2022-01-10 05:44:35,203] jepsen worker 9 - jepsen.util 3609 :info :write [3 12] :SyncWriteAmbiguous INFO [2022-01-10 05:44:35,204] jepsen worker 9 - jepsen.util 3699 :invoke :read [3 nil] INFO [2022-01-10 05:44:35,290] jepsen worker 15 - jepsen.util 2625 :info :write [5 48] :RequestTimeoutException INFO [2022-01-10 05:44:35,291] jepsen worker 15 - jepsen.util 2715 :invoke :read [5 nil] INFO [2022-01-10 05:44:35,391] jepsen worker 87 - jepsen.util 3147 :info :write [29 23] :RequestTimeoutException INFO [2022-01-10 05:44:35,392] jepsen worker 87 - jepsen.util 3237 :invoke :write [29 39] INFO [2022-01-10 05:44:35,545] jepsen worker 71 - jepsen.util 2501 :info :write [23 14] :RequestTimeoutException INFO [2022-01-10 05:44:35,545] jepsen worker 71 - jepsen.util 2591 :invoke :write [23 26] INFO [2022-01-10 05:44:35,642] jepsen worker 34 - jepsen.util 3004 :info :write [11 35] :RequestTimeoutException INFO [2022-01-10 05:44:35,642] jepsen worker 34 - jepsen.util 3094 :invoke :write [11 10] INFO [2022-01-10 05:44:35,655] jepsen worker 49 - jepsen.util 2929 :info :write [16 28] :RequestTimeoutException INFO [2022-01-10 05:44:35,655] jepsen worker 49 - jepsen.util 3019 :invoke :read [16 nil] INFO [2022-01-10 05:44:35,722] jepsen worker 60 - jepsen.util 3480 :ok :read [20 :nil] INFO [2022-01-10 05:44:35,723] jepsen worker 60 - jepsen.util 3480 :invoke :read [20 nil] INFO [2022-01-10 05:44:35,774] jepsen worker 46 - jepsen.util 3196 :ok :read [15 23] INFO [2022-01-10 05:44:35,775] jepsen worker 6 - jepsen.util 3066 :info :write [2 25] :SyncWriteAmbiguous INFO [2022-01-10 05:44:35,775] jepsen worker 46 - jepsen.util 3196 :invoke :read [15 nil] INFO [2022-01-10 05:44:35,779] jepsen worker 6 - jepsen.util 3156 :invoke :read [2 nil] INFO [2022-01-10 05:44:35,803] jepsen worker 2 - jepsen.util 3332 :ok :read [0 46] INFO [2022-01-10 05:44:35,804] jepsen worker 2 - jepsen.util 3332 :invoke :write [0 41] INFO [2022-01-10 05:44:35,850] jepsen worker 18 - jepsen.util 2988 :info :write [6 13] :RequestTimeoutException INFO [2022-01-10 05:44:35,850] jepsen worker 62 - jepsen.util 3392 :info :write [20 17] :RequestTimeoutException INFO [2022-01-10 05:44:35,850] jepsen worker 76 - jepsen.util 2596 :info :write [25 33] :RequestTimeoutException INFO [2022-01-10 05:44:35,852] jepsen worker 18 - jepsen.util 3078 :invoke :read [6 nil] INFO [2022-01-10 05:44:35,852] jepsen worker 76 - jepsen.util 2686 :invoke :read [25 nil] INFO [2022-01-10 05:44:35,853] jepsen worker 62 - jepsen.util 3482 :invoke :write [20 38] INFO [2022-01-10 05:44:35,870] jepsen worker 33 - jepsen.util 3273 :info :write [11 30] :RequestTimeoutException INFO [2022-01-10 05:44:35,871] jepsen worker 33 - jepsen.util 3363 :invoke :write [11 23] INFO [2022-01-10 05:44:35,995] jepsen worker 77 - jepsen.util 3407 :ok :read [25 :nil] INFO [2022-01-10 05:44:35,996] jepsen worker 77 - jepsen.util 3407 :invoke :write [25 5] INFO [2022-01-10 05:44:36,032] jepsen worker 8 - jepsen.util 3608 :ok :read [2 14] INFO [2022-01-10 05:44:36,033] jepsen worker 8 - jepsen.util 3608 :invoke :write [2 18] INFO [2022-01-10 05:44:36,039] jepsen worker 74 - jepsen.util 3674 :ok :read [24 18] INFO [2022-01-10 05:44:36,039] jepsen worker 74 - jepsen.util 3674 :invoke :read [24 nil] INFO [2022-01-10 05:44:36,095] jepsen worker 40 - jepsen.util 2470 :info :write [13 24] :RequestTimeoutException INFO [2022-01-10 05:44:36,095] jepsen worker 40 - jepsen.util 2560 :invoke :read [13 nil] INFO [2022-01-10 05:44:36,159] jepsen worker 39 - jepsen.util 3549 :info :write [13 45] :RequestTimeoutException INFO [2022-01-10 05:44:36,160] jepsen worker 39 - jepsen.util 3639 :invoke :read [13 nil] INFO [2022-01-10 05:44:36,161] jepsen worker 86 - jepsen.util 2516 :info :write [28 10] :RequestTimeoutException INFO [2022-01-10 05:44:36,162] jepsen worker 86 - jepsen.util 2606 :invoke :read [28 nil] INFO [2022-01-10 05:44:36,171] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:36,171] jepsen worker nemesis - couchbase.util Rebalance check count: 427 INFO [2022-01-10 05:44:36,171] jepsen worker nemesis - couchbase.util Rebalance stuck count: 341 INFO [2022-01-10 05:44:36,186] jepsen worker 60 - jepsen.util 3480 :ok :read [20 :nil] INFO [2022-01-10 05:44:36,188] jepsen worker 60 - jepsen.util 3480 :invoke :write [20 25] INFO [2022-01-10 05:44:36,188] jepsen worker 46 - jepsen.util 3196 :ok :read [15 23] INFO [2022-01-10 05:44:36,189] jepsen worker 46 - jepsen.util 3196 :invoke :read [15 nil] INFO [2022-01-10 05:44:36,200] jepsen worker 6 - jepsen.util 3156 :ok :read [2 14] INFO [2022-01-10 05:44:36,201] jepsen worker 6 - jepsen.util 3156 :invoke :write [2 43] INFO [2022-01-10 05:44:36,296] jepsen worker 76 - jepsen.util 2686 :ok :read [25 :nil] INFO [2022-01-10 05:44:36,298] jepsen worker 76 - jepsen.util 2686 :invoke :read [25 nil] INFO [2022-01-10 05:44:36,320] jepsen worker 15 - jepsen.util 2715 :ok :read [5 42] INFO [2022-01-10 05:44:36,320] jepsen worker 15 - jepsen.util 2715 :invoke :read [5 nil] INFO [2022-01-10 05:44:36,321] jepsen worker 51 - jepsen.util 3651 :info :write [17 25] :RequestTimeoutException INFO [2022-01-10 05:44:36,322] jepsen worker 51 - jepsen.util 3741 :invoke :write [17 0] INFO [2022-01-10 05:44:36,335] jepsen worker 38 - jepsen.util 3368 :info :write [12 11] :RequestTimeoutException INFO [2022-01-10 05:44:36,336] jepsen worker 38 - jepsen.util 3458 :invoke :write [12 5] INFO [2022-01-10 05:44:36,374] jepsen worker 76 - jepsen.util 2686 :ok :read [25 :nil] INFO [2022-01-10 05:44:36,375] jepsen worker 76 - jepsen.util 2686 :invoke :read [25 nil] INFO [2022-01-10 05:44:36,403] jepsen worker 24 - jepsen.util 2724 :ok :read [8 27] INFO [2022-01-10 05:44:36,404] jepsen worker 24 - jepsen.util 2724 :invoke :read [8 nil] INFO [2022-01-10 05:44:36,506] jepsen worker 12 - jepsen.util 3342 :info :write [4 21] :RequestTimeoutException INFO [2022-01-10 05:44:36,507] jepsen worker 12 - jepsen.util 3432 :invoke :read [4 nil] INFO [2022-01-10 05:44:36,533] jepsen worker 12 - jepsen.util 3432 :ok :read [4 3] INFO [2022-01-10 05:44:36,533] jepsen worker 12 - jepsen.util 3432 :invoke :read [4 nil] INFO [2022-01-10 05:44:36,557] jepsen worker 12 - jepsen.util 3432 :ok :read [4 3] INFO [2022-01-10 05:44:36,559] jepsen worker 12 - jepsen.util 3432 :invoke :read [4 nil] INFO [2022-01-10 05:44:36,583] jepsen worker 12 - jepsen.util 3432 :ok :read [4 3] INFO [2022-01-10 05:44:36,584] jepsen worker 12 - jepsen.util 3432 :invoke :read [4 nil] INFO [2022-01-10 05:44:36,590] jepsen worker 54 - jepsen.util 3204 :info :write [18 1] :RequestTimeoutException INFO [2022-01-10 05:44:36,591] jepsen worker 54 - jepsen.util 3294 :invoke :write [18 36] INFO [2022-01-10 05:44:36,601] jepsen worker 35 - jepsen.util 2735 :ok :read [11 21] INFO [2022-01-10 05:44:36,602] jepsen worker 35 - jepsen.util 2735 :invoke :write [11 25] INFO [2022-01-10 05:44:36,617] jepsen worker 12 - jepsen.util 3432 :ok :read [4 3] INFO [2022-01-10 05:44:36,618] jepsen worker 12 - jepsen.util 3432 :invoke :write [4 44] INFO [2022-01-10 05:44:36,784] jepsen worker 39 - jepsen.util 3639 :ok :read [13 15] INFO [2022-01-10 05:44:36,784] jepsen worker 39 - jepsen.util 3639 :invoke :write [13 34] INFO [2022-01-10 05:44:36,915] jepsen worker 86 - jepsen.util 2606 :ok :read [28 45] INFO [2022-01-10 05:44:36,915] jepsen worker 86 - jepsen.util 2606 :invoke :read [28 nil] INFO [2022-01-10 05:44:37,092] jepsen worker 37 - jepsen.util 2917 :info :write [12 38] :SyncWriteAmbiguous INFO [2022-01-10 05:44:37,093] jepsen worker 37 - jepsen.util 3007 :invoke :read [12 nil] INFO [2022-01-10 05:44:37,177] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:37,339] jepsen worker nemesis - couchbase.util Rebalance check count: 428 INFO [2022-01-10 05:44:37,340] jepsen worker nemesis - couchbase.util Rebalance stuck count: 342 INFO [2022-01-10 05:44:37,182] jepsen worker 4 - jepsen.util 3604 :ok :read [1 10] INFO [2022-01-10 05:44:37,239] jepsen worker 15 - jepsen.util 2715 :ok :read [5 42] INFO [2022-01-10 05:44:37,340] jepsen worker 4 - jepsen.util 3604 :invoke :write [1 49] INFO [2022-01-10 05:44:37,239] jepsen worker 49 - jepsen.util 3019 :ok :read [16 38] INFO [2022-01-10 05:44:37,254] jepsen worker 68 - jepsen.util 3038 :info :write [22 32] :RequestTimeoutException INFO [2022-01-10 05:44:37,340] jepsen worker 49 - jepsen.util 3019 :invoke :read [16 nil] INFO [2022-01-10 05:44:37,340] jepsen worker 68 - jepsen.util 3128 :invoke :write [22 5] INFO [2022-01-10 05:44:37,346] jepsen worker 76 - jepsen.util 2686 :ok :read [25 :nil] INFO [2022-01-10 05:44:37,347] jepsen worker 76 - jepsen.util 2686 :invoke :write [25 24] INFO [2022-01-10 05:44:37,352] jepsen worker 15 - jepsen.util 2715 :invoke :write [5 42] INFO [2022-01-10 05:44:37,419] jepsen worker 74 - jepsen.util 3674 :ok :read [24 18] INFO [2022-01-10 05:44:37,420] jepsen worker 74 - jepsen.util 3674 :invoke :write [24 26] INFO [2022-01-10 05:44:37,470] jepsen worker 56 - jepsen.util 3476 :info :write [18 45] :RequestTimeoutException INFO [2022-01-10 05:44:37,471] jepsen worker 56 - jepsen.util 3566 :invoke :write [18 6] INFO [2022-01-10 05:44:37,594] jepsen worker 46 - jepsen.util 3196 :ok :read [15 23] INFO [2022-01-10 05:44:37,595] jepsen worker 46 - jepsen.util 3196 :invoke :write [15 3] INFO [2022-01-10 05:44:37,640] jepsen worker 24 - jepsen.util 2724 :ok :read [8 27] INFO [2022-01-10 05:44:37,641] jepsen worker 24 - jepsen.util 2724 :invoke :read [8 nil] INFO [2022-01-10 05:44:37,771] jepsen worker 59 - jepsen.util 2939 :info :write [19 46] :RequestTimeoutException INFO [2022-01-10 05:44:37,772] jepsen worker 59 - jepsen.util 3029 :invoke :write [19 15] INFO [2022-01-10 05:44:37,890] jepsen worker 86 - jepsen.util 2606 :ok :read [28 45] INFO [2022-01-10 05:44:37,890] jepsen worker 86 - jepsen.util 2606 :invoke :read [28 nil] INFO [2022-01-10 05:44:37,965] jepsen worker 24 - jepsen.util 2724 :ok :read [8 27] INFO [2022-01-10 05:44:37,965] jepsen worker 24 - jepsen.util 2724 :invoke :read [8 nil] INFO [2022-01-10 05:44:38,066] jepsen worker 31 - jepsen.util 3451 :info :write [10 23] :RequestTimeoutException INFO [2022-01-10 05:44:38,067] jepsen worker 31 - jepsen.util 3541 :invoke :read [10 nil] INFO [2022-01-10 05:44:38,136] jepsen worker 89 - jepsen.util 3689 :ok :read [29 0] INFO [2022-01-10 05:44:38,137] jepsen worker 89 - jepsen.util 3689 :invoke :read [29 nil] INFO [2022-01-10 05:44:38,139] jepsen worker 3 - jepsen.util 3153 :ok :read [1 10] INFO [2022-01-10 05:44:38,139] jepsen worker 3 - jepsen.util 3153 :invoke :read [1 nil] INFO [2022-01-10 05:44:38,201] jepsen worker 73 - jepsen.util 2773 :info :write [24 19] :RequestTimeoutException INFO [2022-01-10 05:44:38,202] jepsen worker 73 - jepsen.util 2863 :invoke :write [24 0] INFO [2022-01-10 05:44:38,214] jepsen worker 24 - jepsen.util 2724 :ok :read [8 27] INFO [2022-01-10 05:44:38,215] jepsen worker 24 - jepsen.util 2724 :invoke :read [8 nil] INFO [2022-01-10 05:44:38,226] jepsen worker 11 - jepsen.util 3071 :info :write [3 21] :RequestTimeoutException INFO [2022-01-10 05:44:38,226] jepsen worker 11 - jepsen.util 3161 :invoke :read [3 nil] INFO [2022-01-10 05:44:38,346] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:38,346] jepsen worker nemesis - couchbase.util Rebalance check count: 429 INFO [2022-01-10 05:44:38,346] jepsen worker nemesis - couchbase.util Rebalance stuck count: 343 INFO [2022-01-10 05:44:38,420] jepsen worker 49 - jepsen.util 3019 :ok :read [16 38] INFO [2022-01-10 05:44:38,420] jepsen worker 49 - jepsen.util 3019 :invoke :read [16 nil] INFO [2022-01-10 05:44:38,720] jepsen worker 40 - jepsen.util 2560 :ok :read [13 15] INFO [2022-01-10 05:44:38,721] jepsen worker 40 - jepsen.util 2560 :invoke :write [13 29] INFO [2022-01-10 05:44:38,734] jepsen worker 24 - jepsen.util 2724 :ok :read [8 27] INFO [2022-01-10 05:44:38,735] jepsen worker 24 - jepsen.util 2724 :invoke :write [8 33] INFO [2022-01-10 05:44:38,899] jepsen worker 86 - jepsen.util 2606 :ok :read [28 45] INFO [2022-01-10 05:44:38,900] jepsen worker 86 - jepsen.util 2606 :invoke :write [28 17] INFO [2022-01-10 05:44:38,905] jepsen worker 19 - jepsen.util 3169 :info :write [6 32] :RequestTimeoutException INFO [2022-01-10 05:44:38,906] jepsen worker 19 - jepsen.util 3259 :invoke :read [6 nil] INFO [2022-01-10 05:44:39,010] jepsen worker 67 - jepsen.util 2947 :info :write [22 14] :RequestTimeoutException INFO [2022-01-10 05:44:39,011] jepsen worker 67 - jepsen.util 3037 :invoke :write [22 30] INFO [2022-01-10 05:44:39,351] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:39,351] jepsen worker nemesis - couchbase.util Rebalance check count: 430 INFO [2022-01-10 05:44:39,351] jepsen worker nemesis - couchbase.util Rebalance stuck count: 344 INFO [2022-01-10 05:44:39,696] jepsen worker 49 - jepsen.util 3019 :ok :read [16 38] INFO [2022-01-10 05:44:39,696] jepsen worker 49 - jepsen.util 3019 :invoke :read [16 nil] INFO [2022-01-10 05:44:40,051] jepsen worker 16 - jepsen.util 3166 :info :write [5 14] :RequestTimeoutException INFO [2022-01-10 05:44:40,052] jepsen worker 16 - jepsen.util 3256 :invoke :read [5 nil] INFO [2022-01-10 05:44:40,055] jepsen worker 45 - jepsen.util 3285 :info :write [15 44] :RequestTimeoutException INFO [2022-01-10 05:44:40,055] jepsen worker 63 - jepsen.util 3033 :info :write [21 27] :RequestTimeoutException INFO [2022-01-10 05:44:40,055] jepsen worker 21 - jepsen.util 3261 :info :write [7 17] :RequestTimeoutException INFO [2022-01-10 05:44:40,056] jepsen worker 63 - jepsen.util 3123 :invoke :read [21 nil] INFO [2022-01-10 05:44:40,056] jepsen worker 21 - jepsen.util 3351 :invoke :read [7 nil] INFO [2022-01-10 05:44:40,057] jepsen worker 45 - jepsen.util 3375 :invoke :write [15 12] INFO [2022-01-10 05:44:40,066] jepsen worker 47 - jepsen.util 3107 :info :write [15 14] :RequestTimeoutException INFO [2022-01-10 05:44:40,066] jepsen worker 47 - jepsen.util 3197 :invoke :read [15 nil] INFO [2022-01-10 05:44:40,185] jepsen worker 29 - jepsen.util 2909 :info :write [9 44] :RequestTimeoutException INFO [2022-01-10 05:44:40,186] jepsen worker 29 - jepsen.util 2999 :invoke :write [9 5] WARN [2022-01-10 05:44:40,319] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":20156252,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":20156252,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9024634,"encode_duration_us":22895,"operation_id":"0x5f83","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10009335}],"total_count":1}} WARN [2022-01-10 05:44:40,320] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18053933,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":18053933,"total_server_duration_us":27,"operation_id":"0x5fe8","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":18055296},{"operation_name":"upsert","last_dispatch_duration_us":284357,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8091690,"total_server_duration_us":1379313,"encode_duration_us":16510,"operation_id":"0x628e","timeout_ms":10000,"last_server_duration_us":76451,"total_duration_us":10010910},{"operation_name":"upsert","last_dispatch_duration_us":945512,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9043373,"total_server_duration_us":1662835,"encode_duration_us":17682,"operation_id":"0x61a0","timeout_ms":10000,"last_server_duration_us":75755,"total_duration_us":10009945},{"operation_name":"upsert","last_dispatch_duration_us":1192196,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8092276,"total_server_duration_us":1498798,"encode_duration_us":17678,"operation_id":"0x6283","timeout_ms":10000,"last_server_duration_us":93650,"total_duration_us":10009547},{"operation_name":"upsert","last_dispatch_duration_us":847681,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":9171508,"total_server_duration_us":5025911,"encode_duration_us":17077,"operation_id":"0x60be","timeout_ms":10000,"last_server_duration_us":797434,"total_duration_us":10007962},{"operation_name":"upsert","last_dispatch_duration_us":674289,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9326954,"total_server_duration_us":1728962,"encode_duration_us":16086,"operation_id":"0x61c9","timeout_ms":10000,"last_server_duration_us":111327,"total_duration_us":10007057},{"operation_name":"upsert","last_dispatch_duration_us":594611,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":8944029,"total_server_duration_us":4206200,"encode_duration_us":17144,"operation_id":"0x609b","timeout_ms":10000,"last_server_duration_us":504280,"total_duration_us":10006651},{"operation_name":"upsert","last_dispatch_duration_us":971603,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8173507,"total_server_duration_us":795538,"encode_duration_us":16511,"operation_id":"0x6137","timeout_ms":10000,"last_server_duration_us":217098,"total_duration_us":10006154},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":17631,"operation_id":"0x61d7","timeout_ms":10000,"total_duration_us":10004821},{"operation_name":"upsert","last_dispatch_duration_us":343237,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6622693,"total_server_duration_us":1603178,"encode_duration_us":18079,"operation_id":"0x6257","timeout_ms":10000,"last_server_duration_us":105984,"total_duration_us":10004456}],"total_count":18}} INFO [2022-01-10 05:44:40,365] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:40,365] jepsen worker nemesis - couchbase.util Rebalance check count: 431 INFO [2022-01-10 05:44:40,365] jepsen worker nemesis - couchbase.util Rebalance stuck count: 345 INFO [2022-01-10 05:44:40,416] jepsen worker 28 - jepsen.util 2818 :info :write [9 49] :RequestTimeoutException INFO [2022-01-10 05:44:40,416] jepsen worker 28 - jepsen.util 2908 :invoke :write [9 34] INFO [2022-01-10 05:44:40,436] jepsen worker 17 - jepsen.util 2987 :info :write [5 3] :RequestTimeoutException INFO [2022-01-10 05:44:40,437] jepsen worker 17 - jepsen.util 3077 :invoke :read [5 nil] INFO [2022-01-10 05:44:40,626] jepsen worker 83 - jepsen.util 3593 :info :write [27 16] :RequestTimeoutException INFO [2022-01-10 05:44:40,627] jepsen worker 83 - jepsen.util 3683 :invoke :read [27 nil] INFO [2022-01-10 05:44:40,647] jepsen worker 63 - jepsen.util 3123 :ok :read [21 7] INFO [2022-01-10 05:44:40,648] jepsen worker 63 - jepsen.util 3123 :invoke :read [21 nil] INFO [2022-01-10 05:44:40,751] jepsen worker 49 - jepsen.util 3019 :ok :read [16 38] INFO [2022-01-10 05:44:40,752] jepsen worker 49 - jepsen.util 3019 :invoke :read [16 nil] INFO [2022-01-10 05:44:41,006] jepsen worker 23 - jepsen.util 3173 :info :write [7 33] :RequestTimeoutException INFO [2022-01-10 05:44:41,006] jepsen worker 23 - jepsen.util 3263 :invoke :read [7 nil] INFO [2022-01-10 05:44:41,025] jepsen worker 43 - jepsen.util 2923 :info :write [14 45] :RequestTimeoutException INFO [2022-01-10 05:44:41,026] jepsen worker 43 - jepsen.util 3013 :invoke :write [14 22] INFO [2022-01-10 05:44:41,142] jepsen worker 63 - jepsen.util 3123 :ok :read [21 7] INFO [2022-01-10 05:44:41,143] jepsen worker 63 - jepsen.util 3123 :invoke :read [21 nil] INFO [2022-01-10 05:44:41,196] jepsen worker 61 - jepsen.util 3211 :info :write [20 32] :RequestTimeoutException INFO [2022-01-10 05:44:41,196] jepsen worker 61 - jepsen.util 3301 :invoke :read [20 nil] INFO [2022-01-10 05:44:41,371] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:41,371] jepsen worker nemesis - couchbase.util Rebalance check count: 432 INFO [2022-01-10 05:44:41,371] jepsen worker nemesis - couchbase.util Rebalance stuck count: 346 INFO [2022-01-10 05:44:41,594] jepsen worker 57 - jepsen.util 3747 :info :write [19 9] :RequestTimeoutException INFO [2022-01-10 05:44:41,595] jepsen worker 57 - jepsen.util 3837 :invoke :write [19 10] INFO [2022-01-10 05:44:41,605] jepsen worker 23 - jepsen.util 3263 :ok :read [7 :nil] INFO [2022-01-10 05:44:41,607] jepsen worker 23 - jepsen.util 3263 :invoke :read [7 nil] INFO [2022-01-10 05:44:41,661] jepsen worker 48 - jepsen.util 2748 :info :write [16 43] :RequestTimeoutException INFO [2022-01-10 05:44:41,662] jepsen worker 48 - jepsen.util 2838 :invoke :read [16 nil] INFO [2022-01-10 05:44:41,800] jepsen worker 20 - jepsen.util 2900 :ok :read [6 25] INFO [2022-01-10 05:44:41,800] jepsen worker 20 - jepsen.util 2900 :invoke :write [6 1] INFO [2022-01-10 05:44:41,833] jepsen worker 70 - jepsen.util 3130 :ok :read [23 26] INFO [2022-01-10 05:44:41,833] jepsen worker 70 - jepsen.util 3130 :invoke :read [23 nil] INFO [2022-01-10 05:44:41,862] jepsen worker 78 - jepsen.util 3678 :info :write [26 8] :RequestTimeoutException INFO [2022-01-10 05:44:41,862] jepsen worker 78 - jepsen.util 3768 :invoke :write [26 16] INFO [2022-01-10 05:44:41,898] jepsen worker 30 - jepsen.util 3000 :ok :read [10 36] INFO [2022-01-10 05:44:41,899] jepsen worker 30 - jepsen.util 3000 :invoke :write [10 8] WARN [2022-01-10 05:44:41,917] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":13650520,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":23320913,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9597382,"encode_duration_us":17992,"operation_id":"0x5eda","timeout_ms":10000,"last_server_duration_us":9226056,"total_duration_us":10005381},{"operation_name":"upsert","last_dispatch_duration_us":16008611,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16008611,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9032060,"encode_duration_us":17512,"operation_id":"0x60f2","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10006758}],"total_count":2}} WARN [2022-01-10 05:44:41,920] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17208123,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17208123,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9026756,"encode_duration_us":16437,"operation_id":"0x616b","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10003134},{"operation_name":"upsert","last_dispatch_duration_us":9089780,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9710377,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9402039,"encode_duration_us":16349,"operation_id":"0x6041","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10007442}],"total_count":2}} WARN [2022-01-10 05:44:41,920] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2202119,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10706952,"total_server_duration_us":522294,"encode_duration_us":16549,"operation_id":"0x612f","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10008484},{"operation_name":"upsert","last_dispatch_duration_us":1194537,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":8143833,"total_server_duration_us":505329,"encode_duration_us":17512,"operation_id":"0x6218","timeout_ms":10000,"last_server_duration_us":208906,"total_duration_us":10007700},{"operation_name":"upsert","last_dispatch_duration_us":32315,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":203849,"total_server_duration_us":41061,"encode_duration_us":17218,"operation_id":"0x6204","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10007102},{"operation_name":"upsert","last_dispatch_duration_us":1024303,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":6555848,"total_server_duration_us":831897,"encode_duration_us":18116,"operation_id":"0x6233","timeout_ms":10000,"last_server_duration_us":302490,"total_duration_us":10006096},{"operation_name":"upsert","last_dispatch_duration_us":734817,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5298111,"total_server_duration_us":435811,"encode_duration_us":18243,"operation_id":"0x6136","timeout_ms":10000,"last_server_duration_us":72181,"total_duration_us":10005136},{"operation_name":"upsert","last_dispatch_duration_us":2440004,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10155826,"total_server_duration_us":679860,"encode_duration_us":16923,"operation_id":"0x615a","timeout_ms":10000,"last_server_duration_us":95023,"total_duration_us":10004837},{"operation_name":"upsert","last_dispatch_duration_us":1128943,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":8082454,"total_server_duration_us":1445814,"encode_duration_us":20032,"operation_id":"0x6284","timeout_ms":10000,"last_server_duration_us":49371,"total_duration_us":10003561},{"operation_name":"upsert","last_dispatch_duration_us":17208123,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":17208123,"total_server_duration_us":9026756,"encode_duration_us":16437,"operation_id":"0x616b","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10003134},{"operation_name":"upsert","last_dispatch_duration_us":147060,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":4831449,"total_server_duration_us":4522997,"encode_duration_us":17269,"operation_id":"0x61d6","timeout_ms":10000,"last_server_duration_us":132923,"total_duration_us":10002470},{"operation_name":"upsert","last_dispatch_duration_us":2670941,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9976881,"total_server_duration_us":520933,"encode_duration_us":16284,"operation_id":"0x6159","timeout_ms":10000,"last_server_duration_us":127155,"total_duration_us":10001557}],"total_count":19}} WARN [2022-01-10 05:44:41,922] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10470680,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10470680,"total_server_duration_us":6579,"operation_id":"0x62c3","timeout_ms":30000,"last_server_duration_us":6579,"total_duration_us":10473138},{"operation_name":"upsert","last_dispatch_duration_us":12656442,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":19611646,"total_server_duration_us":9451980,"encode_duration_us":19365,"operation_id":"0x60f3","timeout_ms":10000,"last_server_duration_us":9308658,"total_duration_us":10010795},{"operation_name":"upsert","last_dispatch_duration_us":10959397,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":16116074,"total_server_duration_us":518136,"encode_duration_us":19940,"operation_id":"0x619e","timeout_ms":10000,"last_server_duration_us":20680,"total_duration_us":10009528},{"operation_name":"upsert","last_dispatch_duration_us":10199282,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13756936,"total_server_duration_us":111644,"encode_duration_us":21381,"operation_id":"0x616a","timeout_ms":10000,"last_server_duration_us":78414,"total_duration_us":10007184},{"operation_name":"upsert","last_dispatch_duration_us":3926803,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9594800,"total_server_duration_us":108817,"encode_duration_us":17009,"operation_id":"0x614c","timeout_ms":10000,"last_server_duration_us":87654,"total_duration_us":10007131},{"operation_name":"upsert","last_dispatch_duration_us":370920,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":5555967,"total_server_duration_us":4628736,"encode_duration_us":16897,"operation_id":"0x6131","timeout_ms":10000,"last_server_duration_us":307261,"total_duration_us":10006876},{"operation_name":"upsert","last_dispatch_duration_us":1499731,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9788663,"total_server_duration_us":257099,"encode_duration_us":17463,"operation_id":"0x62c6","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10003935},{"operation_name":"upsert","last_dispatch_duration_us":978978,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10778999,"total_server_duration_us":726284,"encode_duration_us":16298,"operation_id":"0x6130","timeout_ms":10000,"last_server_duration_us":28036,"total_duration_us":10003045},{"operation_name":"upsert","last_dispatch_duration_us":1298271,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10318080,"total_server_duration_us":591559,"encode_duration_us":16501,"operation_id":"0x6187","timeout_ms":10000,"last_server_duration_us":61901,"total_duration_us":10002399},{"operation_name":"upsert","last_dispatch_duration_us":1434631,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10978079,"total_server_duration_us":935003,"encode_duration_us":19595,"operation_id":"0x61d2","timeout_ms":10000,"last_server_duration_us":144965,"total_duration_us":10000980}],"total_count":19}} WARN [2022-01-10 05:44:41,924] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10720797,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10720797,"total_server_duration_us":71231,"operation_id":"0x6169","timeout_ms":30000,"last_server_duration_us":71231,"total_duration_us":10722729},{"operation_name":"get","last_dispatch_duration_us":10353654,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10353654,"total_server_duration_us":585,"operation_id":"0x618e","timeout_ms":30000,"last_server_duration_us":585,"total_duration_us":10354022},{"operation_name":"get","last_dispatch_duration_us":10232692,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10232692,"total_server_duration_us":13689,"operation_id":"0x619b","timeout_ms":30000,"last_server_duration_us":13689,"total_duration_us":10233054},{"operation_name":"upsert","last_dispatch_duration_us":240238,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8056230,"total_server_duration_us":1906063,"encode_duration_us":16678,"operation_id":"0x6296","timeout_ms":10000,"last_server_duration_us":84574,"total_duration_us":10010131},{"operation_name":"upsert","last_dispatch_duration_us":10618385,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":13786491,"total_server_duration_us":1015600,"encode_duration_us":16481,"operation_id":"0x60ef","timeout_ms":10000,"last_server_duration_us":522773,"total_duration_us":10007956},{"operation_name":"upsert","last_dispatch_duration_us":77954,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":7686234,"total_server_duration_us":962111,"encode_duration_us":16864,"operation_id":"0x6293","timeout_ms":10000,"last_server_duration_us":24497,"total_duration_us":10007703},{"operation_name":"upsert","last_dispatch_duration_us":187374,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8161418,"total_server_duration_us":1059310,"encode_duration_us":16918,"operation_id":"0x6294","timeout_ms":10000,"last_server_duration_us":140600,"total_duration_us":10007436},{"operation_name":"upsert","last_dispatch_duration_us":8290664,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8851105,"total_server_duration_us":184074,"encode_duration_us":16998,"operation_id":"0x622a","timeout_ms":10000,"last_server_duration_us":18570,"total_duration_us":10006734},{"operation_name":"upsert","last_dispatch_duration_us":740693,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9980474,"total_server_duration_us":450084,"encode_duration_us":16250,"operation_id":"0x615e","timeout_ms":10000,"last_server_duration_us":88394,"total_duration_us":10006549},{"operation_name":"upsert","last_dispatch_duration_us":1794806,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7033423,"total_server_duration_us":623206,"encode_duration_us":16729,"operation_id":"0x62c1","timeout_ms":10000,"last_server_duration_us":232539,"total_duration_us":10006256}],"total_count":20}} INFO [2022-01-10 05:44:41,951] jepsen worker 49 - jepsen.util 3019 :ok :read [16 38] INFO [2022-01-10 05:44:41,951] jepsen worker 49 - jepsen.util 3019 :invoke :read [16 nil] INFO [2022-01-10 05:44:41,972] jepsen worker 31 - jepsen.util 3541 :ok :read [10 36] INFO [2022-01-10 05:44:41,973] jepsen worker 31 - jepsen.util 3541 :invoke :write [10 25] INFO [2022-01-10 05:44:41,982] jepsen worker 70 - jepsen.util 3130 :ok :read [23 26] INFO [2022-01-10 05:44:41,983] jepsen worker 70 - jepsen.util 3130 :invoke :read [23 nil] WARN [2022-01-10 05:44:42,018] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11219785,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11219785,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9642192,"encode_duration_us":17520,"operation_id":"0x625c","timeout_ms":10000,"last_server_duration_us":9642192,"total_duration_us":10003462},{"operation_name":"upsert","last_dispatch_duration_us":10215451,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10215451,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9188613,"encode_duration_us":29253,"operation_id":"0x61c6","timeout_ms":10000,"last_server_duration_us":9188613,"total_duration_us":10006521},{"operation_name":"upsert","last_dispatch_duration_us":18140078,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18140078,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9032060,"encode_duration_us":23751,"operation_id":"0x5f86","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10005523},{"operation_name":"upsert","last_dispatch_duration_us":18005853,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18005853,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9027816,"encode_duration_us":17056,"operation_id":"0x5f89","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10009024},{"operation_name":"upsert","last_dispatch_duration_us":9371954,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10222910,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9470131,"encode_duration_us":16602,"operation_id":"0x61c4","timeout_ms":10000,"last_server_duration_us":9129905,"total_duration_us":10008233},{"operation_name":"upsert","last_dispatch_duration_us":10223369,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10223369,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9364618,"encode_duration_us":17340,"operation_id":"0x61c5","timeout_ms":10000,"last_server_duration_us":9364618,"total_duration_us":10008381},{"operation_name":"upsert","last_dispatch_duration_us":18283164,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18283164,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9038428,"encode_duration_us":16434,"operation_id":"0x6134","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":10005567}],"total_count":7}} WARN [2022-01-10 05:44:42,018] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11565010,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11565010,"total_server_duration_us":27,"operation_id":"0x61d9","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":11566067},{"operation_name":"upsert","last_dispatch_duration_us":552880,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3388241,"total_server_duration_us":305856,"encode_duration_us":16743,"operation_id":"0x62ba","timeout_ms":10000,"last_server_duration_us":239294,"total_duration_us":10009990},{"operation_name":"upsert","last_dispatch_duration_us":930620,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":10343620,"total_server_duration_us":1590761,"encode_duration_us":16604,"operation_id":"0x60fa","timeout_ms":10000,"last_server_duration_us":119635,"total_duration_us":10009159},{"operation_name":"upsert","last_dispatch_duration_us":11261721,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11261721,"total_server_duration_us":615379,"encode_duration_us":20845,"operation_id":"0x615c","timeout_ms":10000,"last_server_duration_us":615379,"total_duration_us":10009131},{"operation_name":"upsert","last_dispatch_duration_us":1778992,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3120945,"total_server_duration_us":712060,"encode_duration_us":19667,"operation_id":"0x6285","timeout_ms":10000,"last_server_duration_us":631377,"total_duration_us":10007562},{"operation_name":"upsert","last_dispatch_duration_us":16008611,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":16008611,"total_server_duration_us":9032060,"encode_duration_us":17512,"operation_id":"0x60f2","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10006758},{"operation_name":"upsert","last_dispatch_duration_us":2219993,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2219993,"total_server_duration_us":201479,"encode_duration_us":16668,"operation_id":"0x6287","timeout_ms":10000,"last_server_duration_us":201479,"total_duration_us":10005735},{"operation_name":"upsert","last_dispatch_duration_us":57854,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":8234595,"total_server_duration_us":1870485,"encode_duration_us":21291,"operation_id":"0x61d8","timeout_ms":10000,"last_server_duration_us":47192,"total_duration_us":10003982},{"operation_name":"upsert","last_dispatch_duration_us":1052730,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9897214,"total_server_duration_us":933033,"encode_duration_us":16270,"operation_id":"0x612b","timeout_ms":10000,"last_server_duration_us":79403,"total_duration_us":10003961},{"operation_name":"upsert","last_dispatch_duration_us":2150273,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5208298,"total_server_duration_us":171347,"encode_duration_us":16460,"operation_id":"0x6144","timeout_ms":10000,"last_server_duration_us":31108,"total_duration_us":10001973}],"total_count":20}} WARN [2022-01-10 05:44:42,022] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17579968,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17579968,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9268927,"encode_duration_us":16781,"operation_id":"0x602c","timeout_ms":10000,"last_server_duration_us":9268927,"total_duration_us":10002633},{"operation_name":"upsert","last_dispatch_duration_us":12656442,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19611646,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9451980,"encode_duration_us":19365,"operation_id":"0x60f3","timeout_ms":10000,"last_server_duration_us":9308658,"total_duration_us":10010795}],"total_count":2}} WARN [2022-01-10 05:44:42,025] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11948366,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":21370321,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":11537772,"encode_duration_us":17216,"operation_id":"0x5f51","timeout_ms":10000,"last_server_duration_us":9598594,"total_duration_us":10001224}],"total_count":1}} INFO [2022-01-10 05:44:42,053] jepsen worker 70 - jepsen.util 3130 :ok :read [23 26] INFO [2022-01-10 05:44:42,054] jepsen worker 70 - jepsen.util 3130 :invoke :read [23 nil] INFO [2022-01-10 05:44:42,081] jepsen worker 63 - jepsen.util 3123 :ok :read [21 7] INFO [2022-01-10 05:44:42,081] jepsen worker 63 - jepsen.util 3123 :invoke :read [21 nil] INFO [2022-01-10 05:44:42,125] jepsen worker 70 - jepsen.util 3130 :ok :read [23 26] INFO [2022-01-10 05:44:42,125] jepsen worker 70 - jepsen.util 3130 :invoke :read [23 nil] INFO [2022-01-10 05:44:42,132] jepsen worker 26 - jepsen.util 2726 :info :write [8 10] :RequestTimeoutException INFO [2022-01-10 05:44:42,166] jepsen worker 83 - jepsen.util 3683 :ok :read [27 30] INFO [2022-01-10 05:44:42,176] jepsen worker 83 - jepsen.util 3683 :invoke :write [27 11] INFO [2022-01-10 05:44:42,176] jepsen worker 26 - jepsen.util 2816 :invoke :read [8 nil] WARN [2022-01-10 05:44:42,218] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17380787,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":17380787,"total_server_duration_us":69,"operation_id":"0x6168","timeout_ms":30000,"last_server_duration_us":69,"total_duration_us":17382329},{"operation_name":"upsert","last_dispatch_duration_us":497341,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9825556,"total_server_duration_us":1747313,"encode_duration_us":17597,"operation_id":"0x6145","timeout_ms":10000,"last_server_duration_us":33524,"total_duration_us":10009152},{"operation_name":"upsert","last_dispatch_duration_us":10223369,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10223369,"total_server_duration_us":9364618,"encode_duration_us":17340,"operation_id":"0x61c5","timeout_ms":10000,"last_server_duration_us":9364618,"total_duration_us":10008381},{"operation_name":"upsert","last_dispatch_duration_us":9371954,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10222910,"total_server_duration_us":9470131,"encode_duration_us":16602,"operation_id":"0x61c4","timeout_ms":10000,"last_server_duration_us":9129905,"total_duration_us":10008233},{"operation_name":"upsert","last_dispatch_duration_us":2694525,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7769118,"total_server_duration_us":460656,"encode_duration_us":18285,"operation_id":"0x6197","timeout_ms":10000,"last_server_duration_us":98879,"total_duration_us":10006613},{"operation_name":"upsert","last_dispatch_duration_us":10215451,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10215451,"total_server_duration_us":9188613,"encode_duration_us":29253,"operation_id":"0x61c6","timeout_ms":10000,"last_server_duration_us":9188613,"total_duration_us":10006521},{"operation_name":"upsert","last_dispatch_duration_us":9288714,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10250960,"total_server_duration_us":75709,"encode_duration_us":20446,"operation_id":"0x61c7","timeout_ms":10000,"last_server_duration_us":56524,"total_duration_us":10006518},{"operation_name":"upsert","last_dispatch_duration_us":9224584,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":18449498,"total_server_duration_us":40590,"encode_duration_us":18189,"operation_id":"0x6132","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10006291},{"operation_name":"upsert","last_dispatch_duration_us":18283164,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":18283164,"total_server_duration_us":9038428,"encode_duration_us":16434,"operation_id":"0x6134","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":10005567},{"operation_name":"upsert","last_dispatch_duration_us":1743122,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":5273140,"total_server_duration_us":507734,"encode_duration_us":18853,"operation_id":"0x61d1","timeout_ms":10000,"last_server_duration_us":216664,"total_duration_us":10004506}],"total_count":21}} INFO [2022-01-10 05:44:42,256] jepsen worker 70 - jepsen.util 3130 :ok :read [23 26] INFO [2022-01-10 05:44:42,256] jepsen worker 70 - jepsen.util 3130 :invoke :write [23 24] INFO [2022-01-10 05:44:42,278] jepsen worker 23 - jepsen.util 3263 :ok :read [7 :nil] INFO [2022-01-10 05:44:42,280] jepsen worker 23 - jepsen.util 3263 :invoke :write [7 23] INFO [2022-01-10 05:44:42,376] jepsen worker 80 - jepsen.util 3770 :info :write [26 27] :RequestTimeoutException INFO [2022-01-10 05:44:42,377] jepsen worker 80 - jepsen.util 3860 :invoke :read [26 nil] INFO [2022-01-10 05:44:42,379] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:42,379] jepsen worker nemesis - couchbase.util Rebalance check count: 433 INFO [2022-01-10 05:44:42,379] jepsen worker nemesis - couchbase.util Rebalance stuck count: 347 INFO [2022-01-10 05:44:42,386] jepsen worker 79 - jepsen.util 3589 :info :write [26 39] :SyncWriteAmbiguous INFO [2022-01-10 05:44:42,387] jepsen worker 79 - jepsen.util 3679 :invoke :write [26 31] INFO [2022-01-10 05:44:42,399] jepsen worker 52 - jepsen.util 3922 :info :write [17 17] :RequestTimeoutException INFO [2022-01-10 05:44:42,400] jepsen worker 52 - jepsen.util 4012 :invoke :read [17 nil] INFO [2022-01-10 05:44:42,434] jepsen worker 49 - jepsen.util 3019 :ok :read [16 38] INFO [2022-01-10 05:44:42,435] jepsen worker 49 - jepsen.util 3019 :invoke :write [16 21] INFO [2022-01-10 05:44:42,446] jepsen worker 7 - jepsen.util 3157 :info :write [2 5] :RequestTimeoutException INFO [2022-01-10 05:44:42,446] jepsen worker 7 - jepsen.util 3247 :invoke :read [2 nil] INFO [2022-01-10 05:44:42,577] jepsen worker 80 - jepsen.util 3860 :ok :read [26 10] INFO [2022-01-10 05:44:42,577] jepsen worker 80 - jepsen.util 3860 :invoke :read [26 nil] INFO [2022-01-10 05:44:42,614] jepsen worker 80 - jepsen.util 3860 :ok :read [26 10] INFO [2022-01-10 05:44:42,614] jepsen worker 80 - jepsen.util 3860 :invoke :read [26 nil] INFO [2022-01-10 05:44:42,635] jepsen worker 21 - jepsen.util 3351 :ok :read [7 :nil] INFO [2022-01-10 05:44:42,635] jepsen worker 21 - jepsen.util 3351 :invoke :read [7 nil] INFO [2022-01-10 05:44:42,658] jepsen worker 80 - jepsen.util 3860 :ok :read [26 10] INFO [2022-01-10 05:44:42,659] jepsen worker 80 - jepsen.util 3860 :invoke :write [26 42] INFO [2022-01-10 05:44:42,851] jepsen worker 63 - jepsen.util 3123 :ok :read [21 7] INFO [2022-01-10 05:44:42,852] jepsen worker 63 - jepsen.util 3123 :invoke :write [21 49] INFO [2022-01-10 05:44:42,881] jepsen worker 55 - jepsen.util 3385 :info :write [18 11] :RequestTimeoutException INFO [2022-01-10 05:44:42,882] jepsen worker 55 - jepsen.util 3475 :invoke :write [18 49] INFO [2022-01-10 05:44:43,017] jepsen worker 21 - jepsen.util 3351 :ok :read [7 :nil] INFO [2022-01-10 05:44:43,017] jepsen worker 21 - jepsen.util 3351 :invoke :read [7 nil] INFO [2022-01-10 05:44:43,059] jepsen worker 7 - jepsen.util 3247 :ok :read [2 14] INFO [2022-01-10 05:44:43,060] jepsen worker 7 - jepsen.util 3247 :invoke :read [2 nil] INFO [2022-01-10 05:44:43,267] jepsen worker 21 - jepsen.util 3351 :ok :read [7 :nil] INFO [2022-01-10 05:44:43,267] jepsen worker 21 - jepsen.util 3351 :invoke :read [7 nil] INFO [2022-01-10 05:44:43,306] jepsen worker 53 - jepsen.util 3743 :info :write [17 12] :RequestTimeoutException INFO [2022-01-10 05:44:43,307] jepsen worker 53 - jepsen.util 3833 :invoke :write [17 27] INFO [2022-01-10 05:44:43,370] jepsen worker 58 - jepsen.util 3028 :info :write [19 17] :RequestTimeoutException INFO [2022-01-10 05:44:43,371] jepsen worker 58 - jepsen.util 3118 :invoke :write [19 20] INFO [2022-01-10 05:44:43,385] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:43,385] jepsen worker nemesis - couchbase.util Rebalance check count: 434 INFO [2022-01-10 05:44:43,385] jepsen worker nemesis - couchbase.util Rebalance stuck count: 348 INFO [2022-01-10 05:44:43,435] jepsen worker 21 - jepsen.util 3351 :ok :read [7 :nil] INFO [2022-01-10 05:44:43,436] jepsen worker 21 - jepsen.util 3351 :invoke :read [7 nil] INFO [2022-01-10 05:44:43,447] jepsen worker 13 - jepsen.util 3523 :info :write [4 12] :RequestTimeoutException INFO [2022-01-10 05:44:43,447] jepsen worker 13 - jepsen.util 3613 :invoke :write [4 32] INFO [2022-01-10 05:44:43,492] jepsen worker 81 - jepsen.util 3321 :info :write [27 40] :RequestTimeoutException INFO [2022-01-10 05:44:43,493] jepsen worker 81 - jepsen.util 3411 :invoke :read [27 nil] INFO [2022-01-10 05:44:43,511] jepsen worker 32 - jepsen.util 3362 :info :write [10 7] :RequestTimeoutException INFO [2022-01-10 05:44:43,512] jepsen worker 32 - jepsen.util 3452 :invoke :write [10 6] INFO [2022-01-10 05:44:43,565] jepsen worker 25 - jepsen.util 2905 :info :write [8 35] :RequestTimeoutException INFO [2022-01-10 05:44:43,566] jepsen worker 25 - jepsen.util 2995 :invoke :read [8 nil] INFO [2022-01-10 05:44:43,690] jepsen worker 44 - jepsen.util 3194 :info :write [14 32] :RequestTimeoutException INFO [2022-01-10 05:44:43,691] jepsen worker 44 - jepsen.util 3284 :invoke :read [14 nil] INFO [2022-01-10 05:44:43,698] jepsen worker 36 - jepsen.util 2556 :ok :read [12 29] INFO [2022-01-10 05:44:43,699] jepsen worker 36 - jepsen.util 2556 :invoke :read [12 nil] INFO [2022-01-10 05:44:43,726] jepsen worker 10 - jepsen.util 3340 :ok :read [3 31] INFO [2022-01-10 05:44:43,727] jepsen worker 10 - jepsen.util 3340 :invoke :read [3 nil] INFO [2022-01-10 05:44:43,761] jepsen worker 69 - jepsen.util 2859 :ok :read [23 26] INFO [2022-01-10 05:44:43,761] jepsen worker 75 - jepsen.util 2595 :info :write [25 49] :RequestTimeoutException INFO [2022-01-10 05:44:43,761] jepsen worker 69 - jepsen.util 2859 :invoke :read [23 nil] INFO [2022-01-10 05:44:43,762] jepsen worker 75 - jepsen.util 2685 :invoke :write [25 29] INFO [2022-01-10 05:44:43,781] jepsen worker 42 - jepsen.util 3102 :info :write [14 1] :RequestTimeoutException INFO [2022-01-10 05:44:43,782] jepsen worker 42 - jepsen.util 3192 :invoke :read [14 nil] INFO [2022-01-10 05:44:43,846] jepsen worker 1 - jepsen.util 3151 :info :write [0 0] :RequestTimeoutException INFO [2022-01-10 05:44:43,847] jepsen worker 1 - jepsen.util 3241 :invoke :read [0 nil] INFO [2022-01-10 05:44:43,971] jepsen worker 0 - jepsen.util 2970 :info :write [0 19] :RequestTimeoutException INFO [2022-01-10 05:44:43,972] jepsen worker 0 - jepsen.util 3060 :invoke :read [0 nil] INFO [2022-01-10 05:44:44,025] jepsen worker 27 - jepsen.util 3177 :info :write [9 0] :RequestTimeoutException INFO [2022-01-10 05:44:44,026] jepsen worker 27 - jepsen.util 3267 :invoke :read [9 nil] INFO [2022-01-10 05:44:44,075] jepsen worker 21 - jepsen.util 3351 :ok :read [7 :nil] INFO [2022-01-10 05:44:44,075] jepsen worker 21 - jepsen.util 3351 :invoke :write [7 14] INFO [2022-01-10 05:44:44,085] jepsen worker 7 - jepsen.util 3247 :ok :read [2 14] INFO [2022-01-10 05:44:44,086] jepsen worker 7 - jepsen.util 3247 :invoke :read [2 nil] INFO [2022-01-10 05:44:44,152] jepsen worker 85 - jepsen.util 3415 :info :write [28 26] :RequestTimeoutException INFO [2022-01-10 05:44:44,153] jepsen worker 85 - jepsen.util 3505 :invoke :write [28 27] INFO [2022-01-10 05:44:44,225] jepsen worker 18 - jepsen.util 3078 :ok :read [6 25] INFO [2022-01-10 05:44:44,226] jepsen worker 18 - jepsen.util 3078 :invoke :write [6 38] INFO [2022-01-10 05:44:44,246] jepsen worker 14 - jepsen.util 3794 :info :write [4 3] :RequestTimeoutException INFO [2022-01-10 05:44:44,247] jepsen worker 14 - jepsen.util 3884 :invoke :write [4 35] INFO [2022-01-10 05:44:44,295] jepsen worker 19 - jepsen.util 3259 :ok :read [6 25] INFO [2022-01-10 05:44:44,295] jepsen worker 27 - jepsen.util 3267 :ok :read [9 27] INFO [2022-01-10 05:44:44,295] jepsen worker 19 - jepsen.util 3259 :invoke :read [6 nil] INFO [2022-01-10 05:44:44,296] jepsen worker 27 - jepsen.util 3267 :invoke :read [9 nil] INFO [2022-01-10 05:44:44,381] jepsen worker 47 - jepsen.util 3197 :ok :read [15 23] INFO [2022-01-10 05:44:44,381] jepsen worker 47 - jepsen.util 3197 :invoke :read [15 nil] INFO [2022-01-10 05:44:44,391] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:44,391] jepsen worker nemesis - couchbase.util Rebalance check count: 435 INFO [2022-01-10 05:44:44,391] jepsen worker nemesis - couchbase.util Rebalance stuck count: 349 INFO [2022-01-10 05:44:44,471] jepsen worker 50 - jepsen.util 3200 :info :write [16 41] :RequestTimeoutException INFO [2022-01-10 05:44:44,471] jepsen worker 50 - jepsen.util 3290 :invoke :write [16 29] INFO [2022-01-10 05:44:44,511] jepsen worker 5 - jepsen.util 3065 :info :write [1 6] :RequestTimeoutException INFO [2022-01-10 05:44:44,512] jepsen worker 5 - jepsen.util 3155 :invoke :read [1 nil] INFO [2022-01-10 05:44:44,522] jepsen worker 25 - jepsen.util 2995 :ok :read [8 27] INFO [2022-01-10 05:44:44,523] jepsen worker 25 - jepsen.util 2995 :invoke :write [8 17] INFO [2022-01-10 05:44:44,535] jepsen worker 84 - jepsen.util 3144 :info :write [28 49] :RequestTimeoutException INFO [2022-01-10 05:44:44,536] jepsen worker 84 - jepsen.util 3234 :invoke :write [28 0] INFO [2022-01-10 05:44:44,554] jepsen worker 7 - jepsen.util 3247 :ok :read [2 14] INFO [2022-01-10 05:44:44,555] jepsen worker 7 - jepsen.util 3247 :invoke :write [2 4] INFO [2022-01-10 05:44:44,556] jepsen worker 27 - jepsen.util 3267 :ok :read [9 27] INFO [2022-01-10 05:44:44,557] jepsen worker 27 - jepsen.util 3267 :invoke :read [9 nil] INFO [2022-01-10 05:44:44,625] jepsen worker 88 - jepsen.util 3058 :info :write [29 28] :RequestTimeoutException INFO [2022-01-10 05:44:44,625] jepsen worker 65 - jepsen.util 2945 :info :write [21 29] :RequestTimeoutException INFO [2022-01-10 05:44:44,626] jepsen worker 88 - jepsen.util 3148 :invoke :write [29 49] INFO [2022-01-10 05:44:44,627] jepsen worker 65 - jepsen.util 3035 :invoke :write [21 46] INFO [2022-01-10 05:44:44,656] jepsen worker 72 - jepsen.util 3492 :info :write [24 28] :RequestTimeoutException INFO [2022-01-10 05:44:44,657] jepsen worker 72 - jepsen.util 3582 :invoke :write [24 0] INFO [2022-01-10 05:44:44,802] jepsen worker 52 - jepsen.util 4012 :ok :read [17 41] INFO [2022-01-10 05:44:44,802] jepsen worker 52 - jepsen.util 4012 :invoke :write [17 9] INFO [2022-01-10 05:44:44,820] jepsen worker 66 - jepsen.util 2856 :info :write [22 12] :RequestTimeoutException INFO [2022-01-10 05:44:44,821] jepsen worker 66 - jepsen.util 2946 :invoke :write [22 33] INFO [2022-01-10 05:44:44,943] jepsen worker 47 - jepsen.util 3197 :ok :read [15 23] INFO [2022-01-10 05:44:44,944] jepsen worker 47 - jepsen.util 3197 :invoke :read [15 nil] INFO [2022-01-10 05:44:45,031] jepsen worker 19 - jepsen.util 3259 :ok :read [6 25] INFO [2022-01-10 05:44:45,033] jepsen worker 19 - jepsen.util 3259 :invoke :write [6 11] INFO [2022-01-10 05:44:45,085] jepsen worker 27 - jepsen.util 3267 :ok :read [9 27] INFO [2022-01-10 05:44:45,086] jepsen worker 27 - jepsen.util 3267 :invoke :read [9 nil] INFO [2022-01-10 05:44:45,165] jepsen worker 64 - jepsen.util 3124 :info :write [21 49] :RequestTimeoutException INFO [2022-01-10 05:44:45,165] jepsen worker 64 - jepsen.util 3214 :invoke :write [21 49] INFO [2022-01-10 05:44:45,182] jepsen worker 82 - jepsen.util 3232 :info :write [27 48] :RequestTimeoutException INFO [2022-01-10 05:44:45,183] jepsen worker 82 - jepsen.util 3322 :invoke :write [27 28] INFO [2022-01-10 05:44:45,207] jepsen worker 22 - jepsen.util 2812 :info :write [7 29] :RequestTimeoutException INFO [2022-01-10 05:44:45,207] jepsen worker 22 - jepsen.util 2902 :invoke :read [7 nil] INFO [2022-01-10 05:44:45,397] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:45,397] jepsen worker nemesis - couchbase.util Rebalance check count: 436 INFO [2022-01-10 05:44:45,397] jepsen worker nemesis - couchbase.util Rebalance stuck count: 350 INFO [2022-01-10 05:44:45,415] jepsen worker 87 - jepsen.util 3237 :info :write [29 39] :RequestTimeoutException INFO [2022-01-10 05:44:45,415] jepsen worker 87 - jepsen.util 3327 :invoke :read [29 nil] INFO [2022-01-10 05:44:45,566] jepsen worker 71 - jepsen.util 2591 :info :write [23 26] :RequestTimeoutException INFO [2022-01-10 05:44:45,566] jepsen worker 71 - jepsen.util 2681 :invoke :read [23 nil] INFO [2022-01-10 05:44:45,669] jepsen worker 34 - jepsen.util 3094 :info :write [11 10] :RequestTimeoutException INFO [2022-01-10 05:44:45,670] jepsen worker 34 - jepsen.util 3184 :invoke :read [11 nil] INFO [2022-01-10 05:44:45,831] jepsen worker 2 - jepsen.util 3332 :info :write [0 41] :RequestTimeoutException INFO [2022-01-10 05:44:45,832] jepsen worker 2 - jepsen.util 3422 :invoke :write [0 14] INFO [2022-01-10 05:44:45,875] jepsen worker 62 - jepsen.util 3482 :info :write [20 38] :RequestTimeoutException INFO [2022-01-10 05:44:45,876] jepsen worker 62 - jepsen.util 3572 :invoke :read [20 nil] INFO [2022-01-10 05:44:45,890] jepsen worker 33 - jepsen.util 3363 :info :write [11 23] :RequestTimeoutException INFO [2022-01-10 05:44:45,891] jepsen worker 33 - jepsen.util 3453 :invoke :read [11 nil] INFO [2022-01-10 05:44:45,943] jepsen worker 22 - jepsen.util 2902 :ok :read [7 :nil] INFO [2022-01-10 05:44:45,943] jepsen worker 22 - jepsen.util 2902 :invoke :write [7 38] INFO [2022-01-10 05:44:45,962] jepsen worker 16 - jepsen.util 3256 :ok :read [5 42] INFO [2022-01-10 05:44:45,963] jepsen worker 16 - jepsen.util 3256 :invoke :read [5 nil] INFO [2022-01-10 05:44:46,026] jepsen worker 77 - jepsen.util 3407 :info :write [25 5] :RequestTimeoutException INFO [2022-01-10 05:44:46,026] jepsen worker 77 - jepsen.util 3497 :invoke :read [25 nil] INFO [2022-01-10 05:44:46,056] jepsen worker 8 - jepsen.util 3608 :info :write [2 18] :RequestTimeoutException INFO [2022-01-10 05:44:46,057] jepsen worker 8 - jepsen.util 3698 :invoke :write [2 23] INFO [2022-01-10 05:44:46,189] jepsen worker 9 - jepsen.util 3699 :ok :read [3 31] INFO [2022-01-10 05:44:46,189] jepsen worker 9 - jepsen.util 3699 :invoke :write [3 42] INFO [2022-01-10 05:44:46,216] jepsen worker 60 - jepsen.util 3480 :info :write [20 25] :RequestTimeoutException INFO [2022-01-10 05:44:46,220] jepsen worker 60 - jepsen.util 3570 :invoke :read [20 nil] INFO [2022-01-10 05:44:46,227] jepsen worker 6 - jepsen.util 3156 :info :write [2 43] :RequestTimeoutException INFO [2022-01-10 05:44:46,227] jepsen worker 6 - jepsen.util 3246 :invoke :write [2 27] INFO [2022-01-10 05:44:46,231] jepsen worker 37 - jepsen.util 3007 :ok :read [12 29] INFO [2022-01-10 05:44:46,232] jepsen worker 37 - jepsen.util 3007 :invoke :read [12 nil] INFO [2022-01-10 05:44:46,261] jepsen worker 11 - jepsen.util 3161 :ok :read [3 31] INFO [2022-01-10 05:44:46,261] jepsen worker 11 - jepsen.util 3161 :invoke :write [3 16] INFO [2022-01-10 05:44:46,269] jepsen worker 62 - jepsen.util 3572 :ok :read [20 :nil] INFO [2022-01-10 05:44:46,270] jepsen worker 62 - jepsen.util 3572 :invoke :read [20 nil] INFO [2022-01-10 05:44:46,299] jepsen worker 87 - jepsen.util 3327 :ok :read [29 0] INFO [2022-01-10 05:44:46,300] jepsen worker 87 - jepsen.util 3327 :invoke :write [29 49] INFO [2022-01-10 05:44:46,339] jepsen worker 37 - jepsen.util 3007 :ok :read [12 29] INFO [2022-01-10 05:44:46,339] jepsen worker 37 - jepsen.util 3007 :invoke :write [12 5] INFO [2022-01-10 05:44:46,355] jepsen worker 51 - jepsen.util 3741 :info :write [17 0] :RequestTimeoutException INFO [2022-01-10 05:44:46,356] jepsen worker 51 - jepsen.util 3831 :invoke :read [17 nil] INFO [2022-01-10 05:44:46,366] jepsen worker 38 - jepsen.util 3458 :info :write [12 5] :RequestTimeoutException INFO [2022-01-10 05:44:46,366] jepsen worker 38 - jepsen.util 3548 :invoke :read [12 nil] INFO [2022-01-10 05:44:46,387] jepsen worker 51 - jepsen.util 3831 :ok :read [17 41] INFO [2022-01-10 05:44:46,388] jepsen worker 51 - jepsen.util 3831 :invoke :read [17 nil] INFO [2022-01-10 05:44:46,404] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.47953216374269}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:46,404] jepsen worker nemesis - couchbase.util Rebalance check count: 437 INFO [2022-01-10 05:44:46,404] jepsen worker nemesis - couchbase.util Rebalance stuck count: 351 INFO [2022-01-10 05:44:46,433] jepsen worker 51 - jepsen.util 3831 :ok :read [17 41] INFO [2022-01-10 05:44:46,434] jepsen worker 51 - jepsen.util 3831 :invoke :write [17 7] INFO [2022-01-10 05:44:46,497] jepsen worker 81 - jepsen.util 3411 :ok :read [27 30] INFO [2022-01-10 05:44:46,498] jepsen worker 81 - jepsen.util 3411 :invoke :write [27 40] INFO [2022-01-10 05:44:46,534] jepsen worker 1 - jepsen.util 3241 :ok :read [0 46] INFO [2022-01-10 05:44:46,534] jepsen worker 1 - jepsen.util 3241 :invoke :read [0 nil] INFO [2022-01-10 05:44:46,570] jepsen worker 62 - jepsen.util 3572 :ok :read [20 :nil] INFO [2022-01-10 05:44:46,571] jepsen worker 62 - jepsen.util 3572 :invoke :read [20 nil] INFO [2022-01-10 05:44:46,612] jepsen worker 54 - jepsen.util 3294 :info :write [18 36] :RequestTimeoutException INFO [2022-01-10 05:44:46,612] jepsen worker 54 - jepsen.util 3384 :invoke :read [18 nil] INFO [2022-01-10 05:44:46,625] jepsen worker 35 - jepsen.util 2735 :info :write [11 25] :RequestTimeoutException INFO [2022-01-10 05:44:46,626] jepsen worker 35 - jepsen.util 2825 :invoke :write [11 24] INFO [2022-01-10 05:44:46,649] jepsen worker 12 - jepsen.util 3432 :info :write [4 44] :RequestTimeoutException INFO [2022-01-10 05:44:46,651] jepsen worker 12 - jepsen.util 3522 :invoke :read [4 nil] INFO [2022-01-10 05:44:46,651] jepsen worker 33 - jepsen.util 3453 :ok :read [11 21] INFO [2022-01-10 05:44:46,652] jepsen worker 33 - jepsen.util 3453 :invoke :write [11 19] INFO [2022-01-10 05:44:46,760] jepsen worker 12 - jepsen.util 3522 :ok :read [4 3] INFO [2022-01-10 05:44:46,761] jepsen worker 12 - jepsen.util 3522 :invoke :write [4 47] INFO [2022-01-10 05:44:46,772] jepsen worker 16 - jepsen.util 3256 :ok :read [5 42] INFO [2022-01-10 05:44:46,773] jepsen worker 16 - jepsen.util 3256 :invoke :write [5 17] INFO [2022-01-10 05:44:46,785] jepsen worker 54 - jepsen.util 3384 :ok :read [18 14] INFO [2022-01-10 05:44:46,785] jepsen worker 54 - jepsen.util 3384 :invoke :read [18 nil] INFO [2022-01-10 05:44:46,807] jepsen worker 39 - jepsen.util 3639 :info :write [13 34] :RequestTimeoutException INFO [2022-01-10 05:44:46,807] jepsen worker 39 - jepsen.util 3729 :invoke :read [13 nil] INFO [2022-01-10 05:44:46,921] jepsen worker 54 - jepsen.util 3384 :ok :read [18 14] INFO [2022-01-10 05:44:46,921] jepsen worker 54 - jepsen.util 3384 :invoke :write [18 18] INFO [2022-01-10 05:44:46,988] jepsen worker 62 - jepsen.util 3572 :ok :read [20 :nil] INFO [2022-01-10 05:44:46,988] jepsen worker 62 - jepsen.util 3572 :invoke :read [20 nil] INFO [2022-01-10 05:44:47,060] jepsen worker 1 - jepsen.util 3241 :ok :read [0 46] INFO [2022-01-10 05:44:47,061] jepsen worker 1 - jepsen.util 3241 :invoke :read [0 nil] INFO [2022-01-10 05:44:47,178] jepsen worker 62 - jepsen.util 3572 :ok :read [20 :nil] INFO [2022-01-10 05:44:47,179] jepsen worker 62 - jepsen.util 3572 :invoke :read [20 nil] INFO [2022-01-10 05:44:47,221] jepsen worker 3 - jepsen.util 3153 :ok :read [1 10] INFO [2022-01-10 05:44:47,221] jepsen worker 89 - jepsen.util 3689 :ok :read [29 0] INFO [2022-01-10 05:44:47,255] jepsen worker 5 - jepsen.util 3155 :ok :read [1 10] INFO [2022-01-10 05:44:47,275] jepsen worker 3 - jepsen.util 3153 :invoke :write [1 44] INFO [2022-01-10 05:44:47,275] jepsen worker 89 - jepsen.util 3689 :invoke :read [29 nil] INFO [2022-01-10 05:44:47,276] jepsen worker 5 - jepsen.util 3155 :invoke :read [1 nil] INFO [2022-01-10 05:44:47,296] jepsen worker 47 - jepsen.util 3197 :ok :read [15 23] INFO [2022-01-10 05:44:47,297] jepsen worker 47 - jepsen.util 3197 :invoke :read [15 nil] INFO [2022-01-10 05:44:47,306] jepsen worker 5 - jepsen.util 3155 :ok :read [1 10] INFO [2022-01-10 05:44:47,307] jepsen worker 5 - jepsen.util 3155 :invoke :read [1 nil] INFO [2022-01-10 05:44:47,310] jepsen worker 89 - jepsen.util 3689 :ok :read [29 0] INFO [2022-01-10 05:44:47,311] jepsen worker 89 - jepsen.util 3689 :invoke :write [29 3] INFO [2022-01-10 05:44:47,362] jepsen worker 68 - jepsen.util 3128 :info :write [22 5] :RequestTimeoutException INFO [2022-01-10 05:44:47,362] jepsen worker 68 - jepsen.util 3218 :invoke :read [22 nil] INFO [2022-01-10 05:44:47,363] jepsen worker 5 - jepsen.util 3155 :ok :read [1 10] INFO [2022-01-10 05:44:47,363] jepsen worker 5 - jepsen.util 3155 :invoke :write [1 27] INFO [2022-01-10 05:44:47,365] jepsen worker 39 - jepsen.util 3729 :ok :read [13 15] INFO [2022-01-10 05:44:47,365] jepsen worker 39 - jepsen.util 3729 :invoke :read [13 nil] INFO [2022-01-10 05:44:47,366] jepsen worker 4 - jepsen.util 3604 :info :write [1 49] :RequestTimeoutException INFO [2022-01-10 05:44:47,367] jepsen worker 4 - jepsen.util 3694 :invoke :write [1 47] INFO [2022-01-10 05:44:47,375] jepsen worker 76 - jepsen.util 2686 :info :write [25 24] :RequestTimeoutException INFO [2022-01-10 05:44:47,376] jepsen worker 76 - jepsen.util 2776 :invoke :write [25 1] INFO [2022-01-10 05:44:47,391] jepsen worker 15 - jepsen.util 2715 :info :write [5 42] :RequestTimeoutException INFO [2022-01-10 05:44:47,392] jepsen worker 15 - jepsen.util 2805 :invoke :read [5 nil] INFO [2022-01-10 05:44:47,412] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:47,413] jepsen worker nemesis - couchbase.util Rebalance check count: 438 INFO [2022-01-10 05:44:47,413] jepsen worker nemesis - couchbase.util Rebalance stuck count: 351 INFO [2022-01-10 05:44:47,418] jepsen worker 62 - jepsen.util 3572 :ok :read [20 :nil] INFO [2022-01-10 05:44:47,419] jepsen worker 62 - jepsen.util 3572 :invoke :write [20 19] INFO [2022-01-10 05:44:47,446] jepsen worker 74 - jepsen.util 3674 :info :write [24 26] :RequestTimeoutException INFO [2022-01-10 05:44:47,446] jepsen worker 74 - jepsen.util 3764 :invoke :read [24 nil] INFO [2022-01-10 05:44:47,494] jepsen worker 56 - jepsen.util 3566 :info :write [18 6] :RequestTimeoutException INFO [2022-01-10 05:44:47,495] jepsen worker 56 - jepsen.util 3656 :invoke :read [18 nil] INFO [2022-01-10 05:44:47,593] jepsen worker 56 - jepsen.util 3656 :ok :read [18 14] INFO [2022-01-10 05:44:47,593] jepsen worker 56 - jepsen.util 3656 :invoke :read [18 nil] INFO [2022-01-10 05:44:47,604] jepsen worker 1 - jepsen.util 3241 :ok :read [0 46] INFO [2022-01-10 05:44:47,604] jepsen worker 1 - jepsen.util 3241 :invoke :read [0 nil] INFO [2022-01-10 05:44:47,616] jepsen worker 46 - jepsen.util 3196 :info :write [15 3] :RequestTimeoutException INFO [2022-01-10 05:44:47,617] jepsen worker 46 - jepsen.util 3286 :invoke :read [15 nil] INFO [2022-01-10 05:44:47,680] jepsen worker 56 - jepsen.util 3656 :ok :read [18 14] INFO [2022-01-10 05:44:47,681] jepsen worker 56 - jepsen.util 3656 :invoke :write [18 39] INFO [2022-01-10 05:44:47,796] jepsen worker 59 - jepsen.util 3029 :info :write [19 15] :RequestTimeoutException INFO [2022-01-10 05:44:47,796] jepsen worker 59 - jepsen.util 3119 :invoke :write [19 3] INFO [2022-01-10 05:44:47,890] jepsen worker 46 - jepsen.util 3286 :ok :read [15 23] INFO [2022-01-10 05:44:47,890] jepsen worker 46 - jepsen.util 3286 :invoke :read [15 nil] INFO [2022-01-10 05:44:47,943] jepsen worker 39 - jepsen.util 3729 :ok :read [13 15] INFO [2022-01-10 05:44:47,943] jepsen worker 39 - jepsen.util 3729 :invoke :read [13 nil] INFO [2022-01-10 05:44:47,971] jepsen worker 47 - jepsen.util 3197 :ok :read [15 23] INFO [2022-01-10 05:44:47,972] jepsen worker 47 - jepsen.util 3197 :invoke :write [15 2] INFO [2022-01-10 05:44:48,137] jepsen worker 1 - jepsen.util 3241 :ok :read [0 46] INFO [2022-01-10 05:44:48,138] jepsen worker 1 - jepsen.util 3241 :invoke :read [0 nil] INFO [2022-01-10 05:44:48,157] jepsen worker 46 - jepsen.util 3286 :ok :read [15 23] INFO [2022-01-10 05:44:48,157] jepsen worker 46 - jepsen.util 3286 :invoke :read [15 nil] INFO [2022-01-10 05:44:48,222] jepsen worker 73 - jepsen.util 2863 :info :write [24 0] :RequestTimeoutException INFO [2022-01-10 05:44:48,222] jepsen worker 73 - jepsen.util 2953 :invoke :write [24 3] INFO [2022-01-10 05:44:48,419] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:48,419] jepsen worker nemesis - couchbase.util Rebalance check count: 439 INFO [2022-01-10 05:44:48,419] jepsen worker nemesis - couchbase.util Rebalance stuck count: 352 INFO [2022-01-10 05:44:48,423] jepsen worker 39 - jepsen.util 3729 :ok :read [13 15] INFO [2022-01-10 05:44:48,424] jepsen worker 39 - jepsen.util 3729 :invoke :write [13 26] INFO [2022-01-10 05:44:48,439] jepsen worker 46 - jepsen.util 3286 :ok :read [15 23] INFO [2022-01-10 05:44:48,440] jepsen worker 46 - jepsen.util 3286 :invoke :read [15 nil] INFO [2022-01-10 05:44:48,638] jepsen worker 1 - jepsen.util 3241 :ok :read [0 46] INFO [2022-01-10 05:44:48,639] jepsen worker 1 - jepsen.util 3241 :invoke :read [0 nil] INFO [2022-01-10 05:44:48,742] jepsen worker 40 - jepsen.util 2560 :info :write [13 29] :RequestTimeoutException INFO [2022-01-10 05:44:48,743] jepsen worker 40 - jepsen.util 2650 :invoke :read [13 nil] INFO [2022-01-10 05:44:48,760] jepsen worker 24 - jepsen.util 2724 :info :write [8 33] :RequestTimeoutException INFO [2022-01-10 05:44:48,761] jepsen worker 24 - jepsen.util 2814 :invoke :write [8 41] INFO [2022-01-10 05:44:48,921] jepsen worker 86 - jepsen.util 2606 :info :write [28 17] :RequestTimeoutException INFO [2022-01-10 05:44:48,922] jepsen worker 86 - jepsen.util 2696 :invoke :read [28 nil] INFO [2022-01-10 05:44:49,035] jepsen worker 67 - jepsen.util 3037 :info :write [22 30] :RequestTimeoutException INFO [2022-01-10 05:44:49,036] jepsen worker 67 - jepsen.util 3127 :invoke :write [22 22] INFO [2022-01-10 05:44:49,141] jepsen worker 46 - jepsen.util 3286 :ok :read [15 23] INFO [2022-01-10 05:44:49,142] jepsen worker 46 - jepsen.util 3286 :invoke :write [15 45] INFO [2022-01-10 05:44:49,259] jepsen worker 1 - jepsen.util 3241 :ok :read [0 46] INFO [2022-01-10 05:44:49,260] jepsen worker 1 - jepsen.util 3241 :invoke :write [0 21] INFO [2022-01-10 05:44:49,425] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:49,426] jepsen worker nemesis - couchbase.util Rebalance check count: 440 INFO [2022-01-10 05:44:49,426] jepsen worker nemesis - couchbase.util Rebalance stuck count: 353 INFO [2022-01-10 05:44:49,647] jepsen worker 61 - jepsen.util 3301 :ok :read [20 :nil] INFO [2022-01-10 05:44:49,648] jepsen worker 61 - jepsen.util 3301 :invoke :read [20 nil] INFO [2022-01-10 05:44:49,768] jepsen worker 17 - jepsen.util 3077 :ok :read [5 42] INFO [2022-01-10 05:44:49,770] jepsen worker 17 - jepsen.util 3077 :invoke :write [5 9] INFO [2022-01-10 05:44:49,885] jepsen worker 42 - jepsen.util 3192 :ok :read [14 14] INFO [2022-01-10 05:44:49,885] jepsen worker 42 - jepsen.util 3192 :invoke :write [14 42] INFO [2022-01-10 05:44:49,919] jepsen worker 60 - jepsen.util 3570 :ok :read [20 :nil] INFO [2022-01-10 05:44:49,920] jepsen worker 60 - jepsen.util 3570 :invoke :read [20 nil] INFO [2022-01-10 05:44:49,938] jepsen worker 34 - jepsen.util 3184 :ok :read [11 21] INFO [2022-01-10 05:44:49,939] jepsen worker 34 - jepsen.util 3184 :invoke :read [11 nil] INFO [2022-01-10 05:44:49,944] jepsen worker 74 - jepsen.util 3764 :ok :read [24 18] INFO [2022-01-10 05:44:49,945] jepsen worker 74 - jepsen.util 3764 :invoke :read [24 nil] INFO [2022-01-10 05:44:49,975] jepsen worker 86 - jepsen.util 2696 :ok :read [28 45] INFO [2022-01-10 05:44:49,976] jepsen worker 86 - jepsen.util 2696 :invoke :read [28 nil] INFO [2022-01-10 05:44:49,992] jepsen worker 41 - jepsen.util 3011 :ok :read [13 15] INFO [2022-01-10 05:44:49,993] jepsen worker 41 - jepsen.util 3011 :invoke :write [13 33] INFO [2022-01-10 05:44:50,015] jepsen worker 48 - jepsen.util 2838 :ok :read [16 38] INFO [2022-01-10 05:44:50,016] jepsen worker 48 - jepsen.util 2838 :invoke :read [16 nil] INFO [2022-01-10 05:44:50,035] jepsen worker 0 - jepsen.util 3060 :ok :read [0 46] INFO [2022-01-10 05:44:50,036] jepsen worker 0 - jepsen.util 3060 :invoke :write [0 5] INFO [2022-01-10 05:44:50,054] jepsen worker 61 - jepsen.util 3301 :ok :read [20 :nil] INFO [2022-01-10 05:44:50,054] jepsen worker 61 - jepsen.util 3301 :invoke :write [20 30] INFO [2022-01-10 05:44:50,086] jepsen worker 45 - jepsen.util 3375 :info :write [15 12] :RequestTimeoutException INFO [2022-01-10 05:44:50,087] jepsen worker 45 - jepsen.util 3465 :invoke :read [15 nil] INFO [2022-01-10 05:44:50,216] jepsen worker 29 - jepsen.util 2999 :info :write [9 5] :RequestTimeoutException INFO [2022-01-10 05:44:50,216] jepsen worker 29 - jepsen.util 3089 :invoke :write [9 19] INFO [2022-01-10 05:44:50,218] jepsen worker 60 - jepsen.util 3570 :ok :read [20 :nil] INFO [2022-01-10 05:44:50,219] jepsen worker 60 - jepsen.util 3570 :invoke :write [20 4] INFO [2022-01-10 05:44:50,234] jepsen worker 34 - jepsen.util 3184 :ok :read [11 21] INFO [2022-01-10 05:44:50,235] jepsen worker 34 - jepsen.util 3184 :invoke :read [11 nil] INFO [2022-01-10 05:44:50,236] jepsen worker 74 - jepsen.util 3764 :ok :read [24 18] INFO [2022-01-10 05:44:50,236] jepsen worker 74 - jepsen.util 3764 :invoke :read [24 nil] WARN [2022-01-10 05:44:50,368] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":26220015,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":26220015,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9033121,"encode_duration_us":16928,"operation_id":"0x6017","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10001912},{"operation_name":"upsert","last_dispatch_duration_us":10058638,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13385525,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10541182,"encode_duration_us":16595,"operation_id":"0x6389","timeout_ms":10000,"last_server_duration_us":9136301,"total_duration_us":10008647}],"total_count":2}} WARN [2022-01-10 05:44:50,368] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":19277942,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":19277942,"total_server_duration_us":27,"operation_id":"0x6166","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":19279179},{"operation_name":"get","last_dispatch_duration_us":18057615,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":18057615,"total_server_duration_us":27,"operation_id":"0x61a1","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":18058859},{"operation_name":"get","last_dispatch_duration_us":9160905,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9160905,"total_server_duration_us":32,"operation_id":"0x6325","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":10246250},{"operation_name":"upsert","last_dispatch_duration_us":182484,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4222638,"total_server_duration_us":2176617,"encode_duration_us":17630,"operation_id":"0x6486","timeout_ms":10000,"last_server_duration_us":72317,"total_duration_us":10009686},{"operation_name":"upsert","last_dispatch_duration_us":158485,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":3803650,"total_server_duration_us":1974518,"encode_duration_us":23077,"operation_id":"0x63ca","timeout_ms":10000,"last_server_duration_us":79971,"total_duration_us":10009208},{"operation_name":"upsert","last_dispatch_duration_us":8528798,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11830188,"total_server_duration_us":1692916,"encode_duration_us":18206,"operation_id":"0x63ff","timeout_ms":10000,"last_server_duration_us":131866,"total_duration_us":10008912},{"operation_name":"upsert","last_dispatch_duration_us":10058638,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":13385525,"total_server_duration_us":10541182,"encode_duration_us":16595,"operation_id":"0x6389","timeout_ms":10000,"last_server_duration_us":9136301,"total_duration_us":10008647},{"operation_name":"upsert","last_dispatch_duration_us":29004,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4377904,"total_server_duration_us":1234705,"encode_duration_us":16515,"operation_id":"0x6455","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10006920},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":16342,"operation_id":"0x6402","timeout_ms":10000,"total_duration_us":10006061},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":16755,"operation_id":"0x638f","timeout_ms":10000,"total_duration_us":10005878}],"total_count":19}} INFO [2022-01-10 05:44:50,431] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:50,431] jepsen worker nemesis - couchbase.util Rebalance check count: 441 INFO [2022-01-10 05:44:50,431] jepsen worker nemesis - couchbase.util Rebalance stuck count: 354 INFO [2022-01-10 05:44:50,446] jepsen worker 28 - jepsen.util 2908 :info :write [9 34] :RequestTimeoutException INFO [2022-01-10 05:44:50,447] jepsen worker 28 - jepsen.util 2998 :invoke :write [9 25] INFO [2022-01-10 05:44:50,500] jepsen worker 68 - jepsen.util 3218 :ok :read [22 41] INFO [2022-01-10 05:44:50,501] jepsen worker 68 - jepsen.util 3218 :invoke :write [22 46] INFO [2022-01-10 05:44:50,557] jepsen worker 34 - jepsen.util 3184 :ok :read [11 21] INFO [2022-01-10 05:44:50,558] jepsen worker 34 - jepsen.util 3184 :invoke :write [11 31] INFO [2022-01-10 05:44:50,568] jepsen worker 74 - jepsen.util 3764 :ok :read [24 18] INFO [2022-01-10 05:44:50,569] jepsen worker 74 - jepsen.util 3764 :invoke :read [24 nil] INFO [2022-01-10 05:44:50,629] jepsen worker 26 - jepsen.util 2816 :ok :read [8 27] INFO [2022-01-10 05:44:50,630] jepsen worker 26 - jepsen.util 2816 :invoke :read [8 nil] INFO [2022-01-10 05:44:51,051] jepsen worker 43 - jepsen.util 3013 :info :write [14 22] :RequestTimeoutException INFO [2022-01-10 05:44:51,052] jepsen worker 43 - jepsen.util 3103 :invoke :read [14 nil] INFO [2022-01-10 05:44:51,068] jepsen worker 71 - jepsen.util 2681 :ok :read [23 26] INFO [2022-01-10 05:44:51,070] jepsen worker 71 - jepsen.util 2681 :invoke :write [23 30] INFO [2022-01-10 05:44:51,201] jepsen worker 77 - jepsen.util 3497 :ok :read [25 :nil] INFO [2022-01-10 05:44:51,201] jepsen worker 77 - jepsen.util 3497 :invoke :write [25 46] INFO [2022-01-10 05:44:51,221] jepsen worker 26 - jepsen.util 2816 :ok :read [8 27] INFO [2022-01-10 05:44:51,222] jepsen worker 26 - jepsen.util 2816 :invoke :read [8 nil] INFO [2022-01-10 05:44:51,368] jepsen worker 70 - jepsen.util 3130 :info :write [23 24] :SyncWriteAmbiguous INFO [2022-01-10 05:44:51,369] jepsen worker 70 - jepsen.util 3220 :invoke :read [23 nil] INFO [2022-01-10 05:44:51,396] jepsen worker 70 - jepsen.util 3220 :ok :read [23 26] INFO [2022-01-10 05:44:51,397] jepsen worker 70 - jepsen.util 3220 :invoke :read [23 nil] INFO [2022-01-10 05:44:51,437] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:51,437] jepsen worker nemesis - couchbase.util Rebalance check count: 442 INFO [2022-01-10 05:44:51,437] jepsen worker nemesis - couchbase.util Rebalance stuck count: 355 INFO [2022-01-10 05:44:51,441] jepsen worker 70 - jepsen.util 3220 :ok :read [23 26] INFO [2022-01-10 05:44:51,441] jepsen worker 70 - jepsen.util 3220 :invoke :read [23 nil] INFO [2022-01-10 05:44:51,474] jepsen worker 70 - jepsen.util 3220 :ok :read [23 26] INFO [2022-01-10 05:44:51,475] jepsen worker 70 - jepsen.util 3220 :invoke :read [23 nil] INFO [2022-01-10 05:44:51,496] jepsen worker 70 - jepsen.util 3220 :ok :read [23 26] INFO [2022-01-10 05:44:51,497] jepsen worker 70 - jepsen.util 3220 :invoke :write [23 2] INFO [2022-01-10 05:44:51,579] jepsen worker 43 - jepsen.util 3103 :ok :read [14 14] INFO [2022-01-10 05:44:51,579] jepsen worker 43 - jepsen.util 3103 :invoke :write [14 22] INFO [2022-01-10 05:44:51,616] jepsen worker 57 - jepsen.util 3837 :info :write [19 10] :RequestTimeoutException INFO [2022-01-10 05:44:51,616] jepsen worker 57 - jepsen.util 3927 :invoke :read [19 nil] INFO [2022-01-10 05:44:51,685] jepsen worker 44 - jepsen.util 3284 :ok :read [14 14] INFO [2022-01-10 05:44:51,686] jepsen worker 44 - jepsen.util 3284 :invoke :write [14 18] INFO [2022-01-10 05:44:51,791] jepsen worker 26 - jepsen.util 2816 :ok :read [8 27] INFO [2022-01-10 05:44:51,792] jepsen worker 26 - jepsen.util 2816 :invoke :write [8 30] INFO [2022-01-10 05:44:51,827] jepsen worker 48 - jepsen.util 2838 :ok :read [16 38] INFO [2022-01-10 05:44:51,828] jepsen worker 48 - jepsen.util 2838 :invoke :read [16 nil] INFO [2022-01-10 05:44:51,831] jepsen worker 20 - jepsen.util 2900 :info :write [6 1] :RequestTimeoutException INFO [2022-01-10 05:44:51,832] jepsen worker 20 - jepsen.util 2990 :invoke :read [6 nil] WARN [2022-01-10 05:44:51,871] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10376828,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":15346180,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9796766,"encode_duration_us":19391,"operation_id":"0x62b9","timeout_ms":10000,"last_server_duration_us":9638919,"total_duration_us":10001681},{"operation_name":"upsert","last_dispatch_duration_us":11685970,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13905963,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9575797,"encode_duration_us":16668,"operation_id":"0x6287","timeout_ms":10000,"last_server_duration_us":9374318,"total_duration_us":10005735},{"operation_name":"upsert","last_dispatch_duration_us":9431363,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11801242,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9682406,"encode_duration_us":16412,"operation_id":"0x634d","timeout_ms":10000,"last_server_duration_us":9111791,"total_duration_us":10006940},{"operation_name":"upsert","last_dispatch_duration_us":9616061,"last_local_id":"EFE9906200000002/00000000598DAEAE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":11304807,"last_local_socket":"172.23.104.79:44342","total_server_duration_us":10180310,"encode_duration_us":16478,"operation_id":"0x6320","timeout_ms":10000,"last_server_duration_us":9583354,"total_duration_us":10008526},{"operation_name":"upsert","last_dispatch_duration_us":10829887,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10829887,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9028877,"encode_duration_us":19789,"operation_id":"0x63cb","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10009219}],"total_count":5}} WARN [2022-01-10 05:44:51,871] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8415589,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":12728266,"total_server_duration_us":370296,"encode_duration_us":28616,"operation_id":"0x6400","timeout_ms":10000,"last_server_duration_us":47647,"total_duration_us":10010442},{"operation_name":"upsert","last_dispatch_duration_us":10200026,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10200026,"total_server_duration_us":9696807,"encode_duration_us":18609,"operation_id":"0x633f","timeout_ms":10000,"last_server_duration_us":9696807,"total_duration_us":10009626},{"operation_name":"upsert","last_dispatch_duration_us":1181391,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7241644,"total_server_duration_us":608502,"encode_duration_us":16394,"operation_id":"0x634f","timeout_ms":10000,"last_server_duration_us":66957,"total_duration_us":10008949},{"operation_name":"upsert","last_dispatch_duration_us":16975347,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":17188376,"total_server_duration_us":9159108,"encode_duration_us":17254,"operation_id":"0x6326","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10008192},{"operation_name":"upsert","last_dispatch_duration_us":8673728,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":17052015,"total_server_duration_us":266126,"encode_duration_us":16324,"operation_id":"0x6317","timeout_ms":10000,"last_server_duration_us":34711,"total_duration_us":10008052},{"operation_name":"upsert","last_dispatch_duration_us":506317,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4080794,"total_server_duration_us":167718,"encode_duration_us":16421,"operation_id":"0x64ab","timeout_ms":10000,"last_server_duration_us":116949,"total_duration_us":10008001},{"operation_name":"upsert","last_dispatch_duration_us":16495321,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":16495321,"total_server_duration_us":9027816,"encode_duration_us":17031,"operation_id":"0x635f","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10007498},{"operation_name":"upsert","last_dispatch_duration_us":9038885,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":15466937,"total_server_duration_us":512800,"encode_duration_us":16647,"operation_id":"0x635c","timeout_ms":10000,"last_server_duration_us":46852,"total_duration_us":10007286},{"operation_name":"upsert","last_dispatch_duration_us":1361280,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":2439883,"total_server_duration_us":114543,"encode_duration_us":16969,"operation_id":"0x6454","timeout_ms":10000,"last_server_duration_us":42634,"total_duration_us":10003877},{"operation_name":"upsert","last_dispatch_duration_us":10195853,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":12006715,"total_server_duration_us":9328963,"encode_duration_us":17887,"operation_id":"0x63fd","timeout_ms":10000,"last_server_duration_us":9175790,"total_duration_us":10002914}],"total_count":19}} WARN [2022-01-10 05:44:51,872] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16350197,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16350197,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9024634,"encode_duration_us":17016,"operation_id":"0x619a","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10001070},{"operation_name":"upsert","last_dispatch_duration_us":16495321,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16495321,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9027816,"encode_duration_us":17031,"operation_id":"0x635f","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10007498},{"operation_name":"upsert","last_dispatch_duration_us":10195853,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12006715,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9328963,"encode_duration_us":17887,"operation_id":"0x63fd","timeout_ms":10000,"last_server_duration_us":9175790,"total_duration_us":10002914},{"operation_name":"upsert","last_dispatch_duration_us":10200026,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10200026,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9696807,"encode_duration_us":18609,"operation_id":"0x633f","timeout_ms":10000,"last_server_duration_us":9696807,"total_duration_us":10009626},{"operation_name":"upsert","last_dispatch_duration_us":16975347,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17188376,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9159108,"encode_duration_us":17254,"operation_id":"0x6326","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10008192}],"total_count":5}} INFO [2022-01-10 05:44:51,882] jepsen worker 78 - jepsen.util 3768 :info :write [26 16] :RequestTimeoutException INFO [2022-01-10 05:44:51,883] jepsen worker 78 - jepsen.util 3858 :invoke :read [26 nil] INFO [2022-01-10 05:44:51,920] jepsen worker 30 - jepsen.util 3000 :info :write [10 8] :RequestTimeoutException INFO [2022-01-10 05:44:51,920] jepsen worker 30 - jepsen.util 3090 :invoke :read [10 nil] INFO [2022-01-10 05:44:51,954] jepsen worker 45 - jepsen.util 3465 :ok :read [15 23] INFO [2022-01-10 05:44:51,955] jepsen worker 45 - jepsen.util 3465 :invoke :write [15 33] WARN [2022-01-10 05:44:51,970] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1406655,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8741921,"total_server_duration_us":1408740,"encode_duration_us":21537,"operation_id":"0x64d5","timeout_ms":10000,"last_server_duration_us":95635,"total_duration_us":10009033},{"operation_name":"upsert","last_dispatch_duration_us":546717,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10389914,"total_server_duration_us":197311,"encode_duration_us":19574,"operation_id":"0x62ec","timeout_ms":10000,"last_server_duration_us":105025,"total_duration_us":10008159},{"operation_name":"upsert","last_dispatch_duration_us":678608,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10264211,"total_server_duration_us":466959,"encode_duration_us":16605,"operation_id":"0x633e","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10007642},{"operation_name":"upsert","last_dispatch_duration_us":424984,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8682305,"total_server_duration_us":1061770,"encode_duration_us":16438,"operation_id":"0x6324","timeout_ms":10000,"last_server_duration_us":99814,"total_duration_us":10007603},{"operation_name":"upsert","last_dispatch_duration_us":286193,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8175468,"total_server_duration_us":777184,"encode_duration_us":19541,"operation_id":"0x63b4","timeout_ms":10000,"last_server_duration_us":70556,"total_duration_us":10007297},{"operation_name":"upsert","last_dispatch_duration_us":381622,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":7906491,"total_server_duration_us":565989,"encode_duration_us":17202,"operation_id":"0x6357","timeout_ms":10000,"last_server_duration_us":59124,"total_duration_us":10006881},{"operation_name":"upsert","last_dispatch_duration_us":8615671,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":10299700,"total_server_duration_us":1118178,"encode_duration_us":18518,"operation_id":"0x62e8","timeout_ms":10000,"last_server_duration_us":105344,"total_duration_us":10005958},{"operation_name":"upsert","last_dispatch_duration_us":442061,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8517666,"total_server_duration_us":817473,"encode_duration_us":16371,"operation_id":"0x638a","timeout_ms":10000,"last_server_duration_us":84429,"total_duration_us":10004158},{"operation_name":"upsert","last_dispatch_duration_us":654244,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":5904836,"total_server_duration_us":693032,"encode_duration_us":17319,"operation_id":"0x643f","timeout_ms":10000,"last_server_duration_us":173123,"total_duration_us":10003631},{"operation_name":"upsert","last_dispatch_duration_us":42096,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":925487,"total_server_duration_us":545139,"encode_duration_us":17616,"operation_id":"0x63d2","timeout_ms":10000,"last_server_duration_us":27945,"total_duration_us":10002194}],"total_count":19}} WARN [2022-01-10 05:44:51,971] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":938898,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9355624,"total_server_duration_us":535545,"encode_duration_us":19417,"operation_id":"0x6487","timeout_ms":10000,"last_server_duration_us":86475,"total_duration_us":10009429},{"operation_name":"upsert","last_dispatch_duration_us":10829887,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10829887,"total_server_duration_us":9028877,"encode_duration_us":19789,"operation_id":"0x63cb","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10009219},{"operation_name":"upsert","last_dispatch_duration_us":9616061,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":11304807,"total_server_duration_us":10180310,"encode_duration_us":16478,"operation_id":"0x6320","timeout_ms":10000,"last_server_duration_us":9583354,"total_duration_us":10008526},{"operation_name":"upsert","last_dispatch_duration_us":9003076,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12101702,"total_server_duration_us":282286,"encode_duration_us":17454,"operation_id":"0x6361","timeout_ms":10000,"last_server_duration_us":48679,"total_duration_us":10008100},{"operation_name":"upsert","last_dispatch_duration_us":9431363,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11801242,"total_server_duration_us":9682406,"encode_duration_us":16412,"operation_id":"0x634d","timeout_ms":10000,"last_server_duration_us":9111791,"total_duration_us":10006940},{"operation_name":"upsert","last_dispatch_duration_us":497160,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10355024,"total_server_duration_us":510966,"encode_duration_us":16877,"operation_id":"0x641d","timeout_ms":10000,"last_server_duration_us":82259,"total_duration_us":10006378},{"operation_name":"upsert","last_dispatch_duration_us":9070059,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11653445,"total_server_duration_us":47735,"encode_duration_us":16438,"operation_id":"0x6390","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10003368},{"operation_name":"upsert","last_dispatch_duration_us":701261,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7484897,"total_server_duration_us":666614,"encode_duration_us":16449,"operation_id":"0x64b5","timeout_ms":10000,"last_server_duration_us":91081,"total_duration_us":10003203},{"operation_name":"upsert","last_dispatch_duration_us":8910197,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8910197,"total_server_duration_us":39529,"encode_duration_us":19627,"operation_id":"0x63d3","timeout_ms":10000,"last_server_duration_us":39529,"total_duration_us":10003192},{"operation_name":"upsert","last_dispatch_duration_us":899584,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":899584,"total_server_duration_us":24669,"encode_duration_us":26134,"operation_id":"0x63b9","timeout_ms":10000,"last_server_duration_us":24669,"total_duration_us":10002456}],"total_count":29}} WARN [2022-01-10 05:44:51,972] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10968320,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10968320,"total_server_duration_us":62,"operation_id":"0x638d","timeout_ms":30000,"last_server_duration_us":62,"total_duration_us":10970339},{"operation_name":"upsert","last_dispatch_duration_us":8375919,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":13561400,"total_server_duration_us":702732,"encode_duration_us":17992,"operation_id":"0x63c6","timeout_ms":10000,"last_server_duration_us":132394,"total_duration_us":10009865},{"operation_name":"upsert","last_dispatch_duration_us":8419469,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12818750,"total_server_duration_us":1261867,"encode_duration_us":16373,"operation_id":"0x6401","timeout_ms":10000,"last_server_duration_us":73821,"total_duration_us":10008948},{"operation_name":"upsert","last_dispatch_duration_us":777961,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":2269400,"total_server_duration_us":232329,"encode_duration_us":19987,"operation_id":"0x647f","timeout_ms":10000,"last_server_duration_us":111000,"total_duration_us":10008591},{"operation_name":"upsert","last_dispatch_duration_us":8489350,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":13019158,"total_server_duration_us":705575,"encode_duration_us":16557,"operation_id":"0x638c","timeout_ms":10000,"last_server_duration_us":83413,"total_duration_us":10007572},{"operation_name":"upsert","last_dispatch_duration_us":9950515,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":13811822,"total_server_duration_us":9525320,"encode_duration_us":18408,"operation_id":"0x63c7","timeout_ms":10000,"last_server_duration_us":9172586,"total_duration_us":10006511},{"operation_name":"upsert","last_dispatch_duration_us":624296,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":4981142,"total_server_duration_us":1901980,"encode_duration_us":16616,"operation_id":"0x62f5","timeout_ms":10000,"last_server_duration_us":32840,"total_duration_us":10006069},{"operation_name":"upsert","last_dispatch_duration_us":682180,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":5851440,"total_server_duration_us":1562236,"encode_duration_us":16695,"operation_id":"0x63bb","timeout_ms":10000,"last_server_duration_us":159731,"total_duration_us":10005756},{"operation_name":"upsert","last_dispatch_duration_us":128949,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9382462,"total_server_duration_us":126281,"encode_duration_us":22856,"operation_id":"0x63fe","timeout_ms":10000,"last_server_duration_us":24154,"total_duration_us":10003253},{"operation_name":"upsert","last_dispatch_duration_us":8413787,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10237266,"total_server_duration_us":204379,"encode_duration_us":17252,"operation_id":"0x6403","timeout_ms":10000,"last_server_duration_us":84574,"total_duration_us":10003086}],"total_count":22}} INFO [2022-01-10 05:44:52,000] jepsen worker 31 - jepsen.util 3541 :info :write [10 25] :RequestTimeoutException INFO [2022-01-10 05:44:52,003] jepsen worker 31 - jepsen.util 3631 :invoke :write [10 10] WARN [2022-01-10 05:44:52,071] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17852540,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":23125680,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9552531,"encode_duration_us":18853,"operation_id":"0x61d1","timeout_ms":10000,"last_server_duration_us":9044797,"total_duration_us":10004506}],"total_count":1}} WARN [2022-01-10 05:44:52,072] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17326940,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17802851,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9150722,"encode_duration_us":17223,"operation_id":"0x6234","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10005913},{"operation_name":"upsert","last_dispatch_duration_us":9254629,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16940864,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":10172112,"encode_duration_us":16864,"operation_id":"0x6293","timeout_ms":10000,"last_server_duration_us":9210001,"total_duration_us":10007703},{"operation_name":"upsert","last_dispatch_duration_us":9950515,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13811822,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9525320,"encode_duration_us":18408,"operation_id":"0x63c7","timeout_ms":10000,"last_server_duration_us":9172586,"total_duration_us":10006511}],"total_count":3}} INFO [2022-01-10 05:44:52,140] jepsen worker 78 - jepsen.util 3858 :ok :read [26 10] INFO [2022-01-10 05:44:52,141] jepsen worker 78 - jepsen.util 3858 :invoke :read [26 nil] WARN [2022-01-10 05:44:52,171] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9227345,"last_local_id":"EFE9906200000004/000000008B42EFB5","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":14783313,"last_local_socket":"172.23.104.79:44292","total_server_duration_us":13750115,"encode_duration_us":16897,"operation_id":"0x6131","timeout_ms":10000,"last_server_duration_us":9121379,"total_duration_us":10006876}],"total_count":1}} INFO [2022-01-10 05:44:52,202] jepsen worker 83 - jepsen.util 3683 :info :write [27 11] :RequestTimeoutException INFO [2022-01-10 05:44:52,202] jepsen worker 83 - jepsen.util 3773 :invoke :read [27 nil] INFO [2022-01-10 05:44:52,207] jepsen worker 48 - jepsen.util 2838 :ok :read [16 38] INFO [2022-01-10 05:44:52,215] jepsen worker 78 - jepsen.util 3858 :ok :read [26 10] INFO [2022-01-10 05:44:52,314] jepsen worker 78 - jepsen.util 3858 :invoke :read [26 nil] WARN [2022-01-10 05:44:52,272] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9329380,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9329380,"total_server_duration_us":1406,"operation_id":"0x634e","timeout_ms":30000,"last_server_duration_us":1406,"total_duration_us":16024465},{"operation_name":"upsert","last_dispatch_duration_us":16260909,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":16260909,"total_server_duration_us":30539,"encode_duration_us":17568,"operation_id":"0x6322","timeout_ms":10000,"last_server_duration_us":30539,"total_duration_us":10010562},{"operation_name":"upsert","last_dispatch_duration_us":9429619,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9429619,"total_server_duration_us":128369,"encode_duration_us":16862,"operation_id":"0x6382","timeout_ms":10000,"last_server_duration_us":128369,"total_duration_us":10010258},{"operation_name":"upsert","last_dispatch_duration_us":615125,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8982644,"total_server_duration_us":2585908,"encode_duration_us":16973,"operation_id":"0x6392","timeout_ms":10000,"last_server_duration_us":77010,"total_duration_us":10009429},{"operation_name":"upsert","last_dispatch_duration_us":8645285,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9119748,"total_server_duration_us":205525,"encode_duration_us":18521,"operation_id":"0x64db","timeout_ms":10000,"last_server_duration_us":22554,"total_duration_us":10007756},{"operation_name":"upsert","last_dispatch_duration_us":872921,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9836029,"total_server_duration_us":1262794,"encode_duration_us":17088,"operation_id":"0x632a","timeout_ms":10000,"last_server_duration_us":62925,"total_duration_us":10007492},{"operation_name":"upsert","last_dispatch_duration_us":341313,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7717083,"total_server_duration_us":1988273,"encode_duration_us":18283,"operation_id":"0x6446","timeout_ms":10000,"last_server_duration_us":87211,"total_duration_us":10006006},{"operation_name":"upsert","last_dispatch_duration_us":388308,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":4900652,"total_server_duration_us":967599,"encode_duration_us":16778,"operation_id":"0x62f6","timeout_ms":10000,"last_server_duration_us":54818,"total_duration_us":10005032},{"operation_name":"upsert","last_dispatch_duration_us":35246,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":516528,"total_server_duration_us":182360,"encode_duration_us":17555,"operation_id":"0x64d8","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10002508},{"operation_name":"upsert","last_dispatch_duration_us":692543,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8746259,"total_server_duration_us":2244228,"encode_duration_us":16763,"operation_id":"0x63b8","timeout_ms":10000,"last_server_duration_us":359124,"total_duration_us":10001751}],"total_count":22}} INFO [2022-01-10 05:44:52,306] jepsen worker 23 - jepsen.util 3263 :info :write [7 23] :RequestTimeoutException INFO [2022-01-10 05:44:52,314] jepsen worker 48 - jepsen.util 2838 :invoke :write [16 9] INFO [2022-01-10 05:44:52,315] jepsen worker 23 - jepsen.util 3353 :invoke :read [7 nil] INFO [2022-01-10 05:44:52,395] jepsen worker 78 - jepsen.util 3858 :ok :read [26 10] INFO [2022-01-10 05:44:52,396] jepsen worker 78 - jepsen.util 3858 :invoke :read [26 nil] INFO [2022-01-10 05:44:52,415] jepsen worker 79 - jepsen.util 3679 :info :write [26 31] :RequestTimeoutException INFO [2022-01-10 05:44:52,416] jepsen worker 79 - jepsen.util 3769 :invoke :write [26 25] INFO [2022-01-10 05:44:52,440] jepsen worker 78 - jepsen.util 3858 :ok :read [26 10] INFO [2022-01-10 05:44:52,441] jepsen worker 78 - jepsen.util 3858 :invoke :read [26 nil] INFO [2022-01-10 05:44:52,444] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:52,444] jepsen worker nemesis - couchbase.util Rebalance check count: 443 INFO [2022-01-10 05:44:52,444] jepsen worker nemesis - couchbase.util Rebalance stuck count: 356 INFO [2022-01-10 05:44:52,461] jepsen worker 49 - jepsen.util 3019 :info :write [16 21] :RequestTimeoutException INFO [2022-01-10 05:44:52,462] jepsen worker 49 - jepsen.util 3109 :invoke :write [16 29] INFO [2022-01-10 05:44:52,489] jepsen worker 78 - jepsen.util 3858 :ok :read [26 10] INFO [2022-01-10 05:44:52,490] jepsen worker 78 - jepsen.util 3858 :invoke :write [26 4] INFO [2022-01-10 05:44:52,644] jepsen worker 57 - jepsen.util 3927 :ok :read [19 31] INFO [2022-01-10 05:44:52,644] jepsen worker 57 - jepsen.util 3927 :invoke :read [19 nil] INFO [2022-01-10 05:44:52,680] jepsen worker 80 - jepsen.util 3860 :info :write [26 42] :RequestTimeoutException INFO [2022-01-10 05:44:52,681] jepsen worker 80 - jepsen.util 3950 :invoke :write [26 32] INFO [2022-01-10 05:44:52,879] jepsen worker 63 - jepsen.util 3123 :info :write [21 49] :RequestTimeoutException INFO [2022-01-10 05:44:52,880] jepsen worker 63 - jepsen.util 3213 :invoke :write [21 31] INFO [2022-01-10 05:44:52,911] jepsen worker 55 - jepsen.util 3475 :info :write [18 49] :RequestTimeoutException INFO [2022-01-10 05:44:52,912] jepsen worker 55 - jepsen.util 3565 :invoke :write [18 47] INFO [2022-01-10 05:44:53,011] jepsen worker 38 - jepsen.util 3548 :ok :read [12 29] INFO [2022-01-10 05:44:53,011] jepsen worker 38 - jepsen.util 3548 :invoke :read [12 nil] INFO [2022-01-10 05:44:53,326] jepsen worker 53 - jepsen.util 3833 :info :write [17 27] :RequestTimeoutException INFO [2022-01-10 05:44:53,327] jepsen worker 53 - jepsen.util 3923 :invoke :write [17 42] INFO [2022-01-10 05:44:53,391] jepsen worker 58 - jepsen.util 3118 :info :write [19 20] :RequestTimeoutException INFO [2022-01-10 05:44:53,392] jepsen worker 58 - jepsen.util 3208 :invoke :write [19 5] INFO [2022-01-10 05:44:53,450] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:53,450] jepsen worker nemesis - couchbase.util Rebalance check count: 444 INFO [2022-01-10 05:44:53,450] jepsen worker nemesis - couchbase.util Rebalance stuck count: 357 INFO [2022-01-10 05:44:53,475] jepsen worker 13 - jepsen.util 3613 :info :write [4 32] :RequestTimeoutException INFO [2022-01-10 05:44:53,476] jepsen worker 13 - jepsen.util 3703 :invoke :read [4 nil] INFO [2022-01-10 05:44:53,530] jepsen worker 32 - jepsen.util 3452 :info :write [10 6] :RequestTimeoutException INFO [2022-01-10 05:44:53,531] jepsen worker 32 - jepsen.util 3542 :invoke :write [10 30] INFO [2022-01-10 05:44:53,636] jepsen worker 27 - jepsen.util 3267 :ok :read [9 27] INFO [2022-01-10 05:44:53,637] jepsen worker 27 - jepsen.util 3267 :invoke :read [9 nil] INFO [2022-01-10 05:44:53,664] jepsen worker 13 - jepsen.util 3703 :ok :read [4 3] INFO [2022-01-10 05:44:53,665] jepsen worker 13 - jepsen.util 3703 :invoke :read [4 nil] INFO [2022-01-10 05:44:53,669] jepsen worker 74 - jepsen.util 3764 :ok :read [24 18] INFO [2022-01-10 05:44:53,670] jepsen worker 74 - jepsen.util 3764 :invoke :read [24 nil] INFO [2022-01-10 05:44:53,718] jepsen worker 13 - jepsen.util 3703 :ok :read [4 3] INFO [2022-01-10 05:44:53,718] jepsen worker 13 - jepsen.util 3703 :invoke :read [4 nil] INFO [2022-01-10 05:44:53,781] jepsen worker 75 - jepsen.util 2685 :info :write [25 29] :RequestTimeoutException INFO [2022-01-10 05:44:53,781] jepsen worker 75 - jepsen.util 2775 :invoke :read [25 nil] INFO [2022-01-10 05:44:53,864] jepsen worker 27 - jepsen.util 3267 :ok :read [9 27] INFO [2022-01-10 05:44:53,864] jepsen worker 27 - jepsen.util 3267 :invoke :read [9 nil] INFO [2022-01-10 05:44:53,884] jepsen worker 74 - jepsen.util 3764 :ok :read [24 18] INFO [2022-01-10 05:44:53,884] jepsen worker 74 - jepsen.util 3764 :invoke :write [24 24] INFO [2022-01-10 05:44:53,956] jepsen worker 75 - jepsen.util 2775 :ok :read [25 :nil] INFO [2022-01-10 05:44:53,957] jepsen worker 75 - jepsen.util 2775 :invoke :write [25 49] INFO [2022-01-10 05:44:54,102] jepsen worker 21 - jepsen.util 3351 :info :write [7 14] :RequestTimeoutException INFO [2022-01-10 05:44:54,103] jepsen worker 21 - jepsen.util 3441 :invoke :read [7 nil] INFO [2022-01-10 05:44:54,176] jepsen worker 85 - jepsen.util 3505 :info :write [28 27] :RequestTimeoutException INFO [2022-01-10 05:44:54,177] jepsen worker 85 - jepsen.util 3595 :invoke :read [28 nil] INFO [2022-01-10 05:44:54,250] jepsen worker 13 - jepsen.util 3703 :ok :read [4 3] INFO [2022-01-10 05:44:54,251] jepsen worker 13 - jepsen.util 3703 :invoke :write [4 38] INFO [2022-01-10 05:44:54,251] jepsen worker 18 - jepsen.util 3078 :info :write [6 38] :RequestTimeoutException INFO [2022-01-10 05:44:54,252] jepsen worker 18 - jepsen.util 3168 :invoke :write [6 6] INFO [2022-01-10 05:44:54,271] jepsen worker 14 - jepsen.util 3884 :info :write [4 35] :RequestTimeoutException INFO [2022-01-10 05:44:54,271] jepsen worker 14 - jepsen.util 3974 :invoke :read [4 nil] INFO [2022-01-10 05:44:54,301] jepsen worker 21 - jepsen.util 3441 :ok :read [7 :nil] INFO [2022-01-10 05:44:54,302] jepsen worker 21 - jepsen.util 3441 :invoke :write [7 49] INFO [2022-01-10 05:44:54,456] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:54,456] jepsen worker nemesis - couchbase.util Rebalance check count: 445 INFO [2022-01-10 05:44:54,456] jepsen worker nemesis - couchbase.util Rebalance stuck count: 358 INFO [2022-01-10 05:44:54,495] jepsen worker 50 - jepsen.util 3290 :info :write [16 29] :RequestTimeoutException INFO [2022-01-10 05:44:54,496] jepsen worker 50 - jepsen.util 3380 :invoke :read [16 nil] INFO [2022-01-10 05:44:54,551] jepsen worker 25 - jepsen.util 2995 :info :write [8 17] :RequestTimeoutException INFO [2022-01-10 05:44:54,552] jepsen worker 25 - jepsen.util 3085 :invoke :read [8 nil] INFO [2022-01-10 05:44:54,560] jepsen worker 84 - jepsen.util 3234 :info :write [28 0] :RequestTimeoutException INFO [2022-01-10 05:44:54,560] jepsen worker 84 - jepsen.util 3324 :invoke :read [28 nil] INFO [2022-01-10 05:44:54,562] jepsen worker 27 - jepsen.util 3267 :ok :read [9 27] INFO [2022-01-10 05:44:54,562] jepsen worker 27 - jepsen.util 3267 :invoke :read [9 nil] INFO [2022-01-10 05:44:54,590] jepsen worker 7 - jepsen.util 3247 :info :write [2 4] :RequestTimeoutException INFO [2022-01-10 05:44:54,590] jepsen worker 7 - jepsen.util 3337 :invoke :write [2 43] INFO [2022-01-10 05:44:54,647] jepsen worker 65 - jepsen.util 3035 :info :write [21 46] :RequestTimeoutException INFO [2022-01-10 05:44:54,648] jepsen worker 65 - jepsen.util 3125 :invoke :read [21 nil] INFO [2022-01-10 05:44:54,652] jepsen worker 88 - jepsen.util 3148 :info :write [29 49] :RequestTimeoutException INFO [2022-01-10 05:44:54,653] jepsen worker 88 - jepsen.util 3238 :invoke :write [29 43] INFO [2022-01-10 05:44:54,669] jepsen worker 14 - jepsen.util 3974 :ok :read [4 3] INFO [2022-01-10 05:44:54,670] jepsen worker 57 - jepsen.util 3927 :ok :read [19 31] INFO [2022-01-10 05:44:54,670] jepsen worker 14 - jepsen.util 3974 :invoke :write [4 3] INFO [2022-01-10 05:44:54,671] jepsen worker 57 - jepsen.util 3927 :invoke :write [19 41] INFO [2022-01-10 05:44:54,681] jepsen worker 72 - jepsen.util 3582 :info :write [24 0] :RequestTimeoutException INFO [2022-01-10 05:44:54,682] jepsen worker 72 - jepsen.util 3672 :invoke :write [24 2] INFO [2022-01-10 05:44:54,768] jepsen worker 27 - jepsen.util 3267 :ok :read [9 27] INFO [2022-01-10 05:44:54,768] jepsen worker 27 - jepsen.util 3267 :invoke :read [9 nil] INFO [2022-01-10 05:44:54,825] jepsen worker 52 - jepsen.util 4012 :info :write [17 9] :RequestTimeoutException INFO [2022-01-10 05:44:54,826] jepsen worker 52 - jepsen.util 4102 :invoke :read [17 nil] INFO [2022-01-10 05:44:54,845] jepsen worker 66 - jepsen.util 2946 :info :write [22 33] :RequestTimeoutException INFO [2022-01-10 05:44:54,845] jepsen worker 66 - jepsen.util 3036 :invoke :read [22 nil] INFO [2022-01-10 05:44:54,916] jepsen worker 27 - jepsen.util 3267 :ok :read [9 27] INFO [2022-01-10 05:44:54,917] jepsen worker 27 - jepsen.util 3267 :invoke :read [9 nil] INFO [2022-01-10 05:44:54,935] jepsen worker 25 - jepsen.util 3085 :ok :read [8 27] INFO [2022-01-10 05:44:54,936] jepsen worker 25 - jepsen.util 3085 :invoke :write [8 4] INFO [2022-01-10 05:44:55,023] jepsen worker 52 - jepsen.util 4102 :ok :read [17 41] INFO [2022-01-10 05:44:55,023] jepsen worker 52 - jepsen.util 4102 :invoke :write [17 32] INFO [2022-01-10 05:44:55,061] jepsen worker 19 - jepsen.util 3259 :info :write [6 11] :RequestTimeoutException INFO [2022-01-10 05:44:55,062] jepsen worker 19 - jepsen.util 3349 :invoke :read [6 nil] INFO [2022-01-10 05:44:55,155] jepsen worker 27 - jepsen.util 3267 :ok :read [9 27] INFO [2022-01-10 05:44:55,156] jepsen worker 27 - jepsen.util 3267 :invoke :write [9 28] INFO [2022-01-10 05:44:55,185] jepsen worker 64 - jepsen.util 3214 :info :write [21 49] :RequestTimeoutException INFO [2022-01-10 05:44:55,186] jepsen worker 64 - jepsen.util 3304 :invoke :read [21 nil] INFO [2022-01-10 05:44:55,209] jepsen worker 82 - jepsen.util 3322 :info :write [27 28] :RequestTimeoutException INFO [2022-01-10 05:44:55,210] jepsen worker 82 - jepsen.util 3412 :invoke :read [27 nil] INFO [2022-01-10 05:44:55,280] jepsen worker 9 - jepsen.util 3699 :info :write [3 42] :SyncWriteAmbiguous INFO [2022-01-10 05:44:55,280] jepsen worker 9 - jepsen.util 3789 :invoke :write [3 24] INFO [2022-01-10 05:44:55,482] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:55,482] jepsen worker nemesis - couchbase.util Rebalance check count: 446 INFO [2022-01-10 05:44:55,482] jepsen worker nemesis - couchbase.util Rebalance stuck count: 359 INFO [2022-01-10 05:44:55,857] jepsen worker 2 - jepsen.util 3422 :info :write [0 14] :RequestTimeoutException INFO [2022-01-10 05:44:55,857] jepsen worker 2 - jepsen.util 3512 :invoke :write [0 8] INFO [2022-01-10 05:44:55,962] jepsen worker 22 - jepsen.util 2902 :info :write [7 38] :RequestTimeoutException INFO [2022-01-10 05:44:55,963] jepsen worker 22 - jepsen.util 2992 :invoke :read [7 nil] INFO [2022-01-10 05:44:55,968] jepsen worker 22 - jepsen.util 2992 :ok :read [7 :nil] INFO [2022-01-10 05:44:55,968] jepsen worker 22 - jepsen.util 2992 :invoke :read [7 nil] INFO [2022-01-10 05:44:56,008] jepsen worker 22 - jepsen.util 2992 :ok :read [7 :nil] INFO [2022-01-10 05:44:56,009] jepsen worker 22 - jepsen.util 2992 :invoke :write [7 36] INFO [2022-01-10 05:44:56,081] jepsen worker 8 - jepsen.util 3698 :info :write [2 23] :RequestTimeoutException INFO [2022-01-10 05:44:56,081] jepsen worker 8 - jepsen.util 3788 :invoke :write [2 17] INFO [2022-01-10 05:44:56,256] jepsen worker 6 - jepsen.util 3246 :info :write [2 27] :RequestTimeoutException INFO [2022-01-10 05:44:56,257] jepsen worker 6 - jepsen.util 3336 :invoke :write [2 22] INFO [2022-01-10 05:44:56,286] jepsen worker 11 - jepsen.util 3161 :info :write [3 16] :RequestTimeoutException INFO [2022-01-10 05:44:56,287] jepsen worker 11 - jepsen.util 3251 :invoke :write [3 47] INFO [2022-01-10 05:44:56,327] jepsen worker 87 - jepsen.util 3327 :info :write [29 49] :RequestTimeoutException INFO [2022-01-10 05:44:56,328] jepsen worker 87 - jepsen.util 3417 :invoke :read [29 nil] INFO [2022-01-10 05:44:56,362] jepsen worker 82 - jepsen.util 3412 :ok :read [27 30] INFO [2022-01-10 05:44:56,362] jepsen worker 82 - jepsen.util 3412 :invoke :read [27 nil] INFO [2022-01-10 05:44:56,367] jepsen worker 37 - jepsen.util 3007 :info :write [12 5] :RequestTimeoutException INFO [2022-01-10 05:44:56,367] jepsen worker 37 - jepsen.util 3097 :invoke :read [12 nil] INFO [2022-01-10 05:44:56,460] jepsen worker 51 - jepsen.util 3831 :info :write [17 7] :RequestTimeoutException INFO [2022-01-10 05:44:56,460] jepsen worker 51 - jepsen.util 3921 :invoke :read [17 nil] INFO [2022-01-10 05:44:56,488] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:56,488] jepsen worker nemesis - couchbase.util Rebalance check count: 447 INFO [2022-01-10 05:44:56,488] jepsen worker nemesis - couchbase.util Rebalance stuck count: 360 INFO [2022-01-10 05:44:56,526] jepsen worker 81 - jepsen.util 3411 :info :write [27 40] :RequestTimeoutException INFO [2022-01-10 05:44:56,526] jepsen worker 81 - jepsen.util 3501 :invoke :write [27 41] INFO [2022-01-10 05:44:56,619] jepsen worker 51 - jepsen.util 3921 :ok :read [17 41] INFO [2022-01-10 05:44:56,619] jepsen worker 51 - jepsen.util 3921 :invoke :write [17 39] INFO [2022-01-10 05:44:56,651] jepsen worker 35 - jepsen.util 2825 :info :write [11 24] :RequestTimeoutException INFO [2022-01-10 05:44:56,651] jepsen worker 35 - jepsen.util 2915 :invoke :read [11 nil] INFO [2022-01-10 05:44:56,665] jepsen worker 82 - jepsen.util 3412 :ok :read [27 30] INFO [2022-01-10 05:44:56,665] jepsen worker 82 - jepsen.util 3412 :invoke :write [27 45] INFO [2022-01-10 05:44:56,672] jepsen worker 33 - jepsen.util 3453 :info :write [11 19] :RequestTimeoutException INFO [2022-01-10 05:44:56,673] jepsen worker 33 - jepsen.util 3543 :invoke :write [11 13] INFO [2022-01-10 05:44:56,785] jepsen worker 12 - jepsen.util 3522 :info :write [4 47] :RequestTimeoutException INFO [2022-01-10 05:44:56,785] jepsen worker 12 - jepsen.util 3612 :invoke :write [4 31] INFO [2022-01-10 05:44:56,806] jepsen worker 16 - jepsen.util 3256 :info :write [5 17] :RequestTimeoutException INFO [2022-01-10 05:44:56,807] jepsen worker 16 - jepsen.util 3346 :invoke :write [5 2] INFO [2022-01-10 05:44:56,947] jepsen worker 54 - jepsen.util 3384 :info :write [18 18] :RequestTimeoutException INFO [2022-01-10 05:44:56,947] jepsen worker 54 - jepsen.util 3474 :invoke :read [18 nil] INFO [2022-01-10 05:44:57,306] jepsen worker 3 - jepsen.util 3153 :info :write [1 44] :RequestTimeoutException INFO [2022-01-10 05:44:57,306] jepsen worker 3 - jepsen.util 3243 :invoke :write [1 33] INFO [2022-01-10 05:44:57,335] jepsen worker 89 - jepsen.util 3689 :info :write [29 3] :RequestTimeoutException INFO [2022-01-10 05:44:57,336] jepsen worker 89 - jepsen.util 3779 :invoke :read [29 nil] INFO [2022-01-10 05:44:57,386] jepsen worker 5 - jepsen.util 3155 :info :write [1 27] :RequestTimeoutException INFO [2022-01-10 05:44:57,387] jepsen worker 5 - jepsen.util 3245 :invoke :read [1 nil] INFO [2022-01-10 05:44:57,391] jepsen worker 4 - jepsen.util 3694 :info :write [1 47] :RequestTimeoutException INFO [2022-01-10 05:44:57,392] jepsen worker 4 - jepsen.util 3784 :invoke :read [1 nil] INFO [2022-01-10 05:44:57,396] jepsen worker 76 - jepsen.util 2776 :info :write [25 1] :RequestTimeoutException INFO [2022-01-10 05:44:57,397] jepsen worker 76 - jepsen.util 2866 :invoke :read [25 nil] INFO [2022-01-10 05:44:57,450] jepsen worker 62 - jepsen.util 3572 :info :write [20 19] :RequestTimeoutException INFO [2022-01-10 05:44:57,451] jepsen worker 62 - jepsen.util 3662 :invoke :write [20 2] INFO [2022-01-10 05:44:57,461] jepsen worker 85 - jepsen.util 3595 :ok :read [28 45] INFO [2022-01-10 05:44:57,461] jepsen worker 85 - jepsen.util 3595 :invoke :write [28 27] INFO [2022-01-10 05:44:57,495] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:57,495] jepsen worker nemesis - couchbase.util Rebalance check count: 448 INFO [2022-01-10 05:44:57,495] jepsen worker nemesis - couchbase.util Rebalance stuck count: 361 INFO [2022-01-10 05:44:57,705] jepsen worker 56 - jepsen.util 3656 :info :write [18 39] :RequestTimeoutException INFO [2022-01-10 05:44:57,706] jepsen worker 56 - jepsen.util 3746 :invoke :write [18 45] INFO [2022-01-10 05:44:57,765] jepsen worker 76 - jepsen.util 2866 :ok :read [25 :nil] INFO [2022-01-10 05:44:57,766] jepsen worker 76 - jepsen.util 2866 :invoke :read [25 nil] INFO [2022-01-10 05:44:57,821] jepsen worker 59 - jepsen.util 3119 :info :write [19 3] :RequestTimeoutException INFO [2022-01-10 05:44:57,822] jepsen worker 59 - jepsen.util 3209 :invoke :read [19 nil] INFO [2022-01-10 05:44:57,847] jepsen worker 59 - jepsen.util 3209 :ok :read [19 31] INFO [2022-01-10 05:44:57,848] jepsen worker 59 - jepsen.util 3209 :invoke :read [19 nil] INFO [2022-01-10 05:44:57,881] jepsen worker 59 - jepsen.util 3209 :ok :read [19 31] INFO [2022-01-10 05:44:57,881] jepsen worker 59 - jepsen.util 3209 :invoke :write [19 34] INFO [2022-01-10 05:44:57,991] jepsen worker 47 - jepsen.util 3197 :info :write [15 2] :RequestTimeoutException INFO [2022-01-10 05:44:57,991] jepsen worker 47 - jepsen.util 3287 :invoke :write [15 37] INFO [2022-01-10 05:44:58,129] jepsen worker 76 - jepsen.util 2866 :ok :read [25 :nil] INFO [2022-01-10 05:44:58,131] jepsen worker 76 - jepsen.util 2866 :invoke :write [25 36] INFO [2022-01-10 05:44:58,247] jepsen worker 73 - jepsen.util 2953 :info :write [24 3] :RequestTimeoutException INFO [2022-01-10 05:44:58,248] jepsen worker 73 - jepsen.util 3043 :invoke :read [24 nil] INFO [2022-01-10 05:44:58,446] jepsen worker 39 - jepsen.util 3729 :info :write [13 26] :RequestTimeoutException INFO [2022-01-10 05:44:58,447] jepsen worker 39 - jepsen.util 3819 :invoke :read [13 nil] INFO [2022-01-10 05:44:58,511] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:58,511] jepsen worker nemesis - couchbase.util Rebalance check count: 449 INFO [2022-01-10 05:44:58,511] jepsen worker nemesis - couchbase.util Rebalance stuck count: 362 INFO [2022-01-10 05:44:58,785] jepsen worker 24 - jepsen.util 2814 :info :write [8 41] :RequestTimeoutException INFO [2022-01-10 05:44:58,786] jepsen worker 24 - jepsen.util 2904 :invoke :read [8 nil] INFO [2022-01-10 05:44:58,986] jepsen worker 23 - jepsen.util 3353 :ok :read [7 :nil] INFO [2022-01-10 05:44:58,987] jepsen worker 23 - jepsen.util 3353 :invoke :write [7 48] INFO [2022-01-10 05:44:59,060] jepsen worker 67 - jepsen.util 3127 :info :write [22 22] :RequestTimeoutException INFO [2022-01-10 05:44:59,060] jepsen worker 67 - jepsen.util 3217 :invoke :write [22 49] INFO [2022-01-10 05:44:59,094] jepsen worker 24 - jepsen.util 2904 :ok :read [8 27] INFO [2022-01-10 05:44:59,095] jepsen worker 24 - jepsen.util 2904 :invoke :write [8 41] INFO [2022-01-10 05:44:59,168] jepsen worker 39 - jepsen.util 3819 :ok :read [13 15] INFO [2022-01-10 05:44:59,168] jepsen worker 39 - jepsen.util 3819 :invoke :write [13 2] INFO [2022-01-10 05:44:59,170] jepsen worker 46 - jepsen.util 3286 :info :write [15 45] :RequestTimeoutException INFO [2022-01-10 05:44:59,173] jepsen worker 46 - jepsen.util 3376 :invoke :read [15 nil] INFO [2022-01-10 05:44:59,226] jepsen worker 40 - jepsen.util 2650 :ok :read [13 15] INFO [2022-01-10 05:44:59,226] jepsen worker 40 - jepsen.util 2650 :invoke :write [13 49] INFO [2022-01-10 05:44:59,268] jepsen worker 15 - jepsen.util 2805 :ok :read [5 42] INFO [2022-01-10 05:44:59,269] jepsen worker 15 - jepsen.util 2805 :invoke :write [5 30] INFO [2022-01-10 05:44:59,286] jepsen worker 1 - jepsen.util 3241 :info :write [0 21] :RequestTimeoutException INFO [2022-01-10 05:44:59,287] jepsen worker 1 - jepsen.util 3331 :invoke :read [0 nil] INFO [2022-01-10 05:44:59,313] jepsen worker 19 - jepsen.util 3349 :ok :read [6 25] INFO [2022-01-10 05:44:59,314] jepsen worker 19 - jepsen.util 3349 :invoke :write [6 0] INFO [2022-01-10 05:44:59,467] jepsen worker 50 - jepsen.util 3380 :ok :read [16 38] INFO [2022-01-10 05:44:59,468] jepsen worker 50 - jepsen.util 3380 :invoke :read [16 nil] INFO [2022-01-10 05:44:59,518] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:44:59,518] jepsen worker nemesis - couchbase.util Rebalance check count: 450 INFO [2022-01-10 05:44:59,518] jepsen worker nemesis - couchbase.util Rebalance stuck count: 363 INFO [2022-01-10 05:44:59,626] jepsen worker 86 - jepsen.util 2696 :ok :read [28 45] INFO [2022-01-10 05:44:59,627] jepsen worker 86 - jepsen.util 2696 :invoke :read [28 nil] INFO [2022-01-10 05:44:59,800] jepsen worker 17 - jepsen.util 3077 :info :write [5 9] :RequestTimeoutException INFO [2022-01-10 05:44:59,801] jepsen worker 17 - jepsen.util 3167 :invoke :read [5 nil] INFO [2022-01-10 05:44:59,914] jepsen worker 42 - jepsen.util 3192 :info :write [14 42] :RequestTimeoutException INFO [2022-01-10 05:44:59,915] jepsen worker 42 - jepsen.util 3282 :invoke :read [14 nil] INFO [2022-01-10 05:44:59,948] jepsen worker 50 - jepsen.util 3380 :ok :read [16 38] INFO [2022-01-10 05:44:59,949] jepsen worker 50 - jepsen.util 3380 :invoke :read [16 nil] INFO [2022-01-10 05:44:59,998] jepsen worker 83 - jepsen.util 3773 :ok :read [27 30] INFO [2022-01-10 05:44:59,999] jepsen worker 83 - jepsen.util 3773 :invoke :read [27 nil] INFO [2022-01-10 05:45:00,031] jepsen worker 41 - jepsen.util 3011 :info :write [13 33] :RequestTimeoutException INFO [2022-01-10 05:45:00,032] jepsen worker 41 - jepsen.util 3101 :invoke :read [13 nil] INFO [2022-01-10 05:45:00,049] jepsen worker 17 - jepsen.util 3167 :ok :read [5 42] INFO [2022-01-10 05:45:00,049] jepsen worker 17 - jepsen.util 3167 :invoke :write [5 23] INFO [2022-01-10 05:45:00,060] jepsen worker 0 - jepsen.util 3060 :info :write [0 5] :RequestTimeoutException INFO [2022-01-10 05:45:00,060] jepsen worker 0 - jepsen.util 3150 :invoke :read [0 nil] INFO [2022-01-10 05:45:00,085] jepsen worker 61 - jepsen.util 3301 :info :write [20 30] :RequestTimeoutException INFO [2022-01-10 05:45:00,086] jepsen worker 61 - jepsen.util 3391 :invoke :write [20 23] INFO [2022-01-10 05:45:00,094] jepsen worker 86 - jepsen.util 2696 :ok :read [28 45] INFO [2022-01-10 05:45:00,095] jepsen worker 86 - jepsen.util 2696 :invoke :read [28 nil] INFO [2022-01-10 05:45:00,111] jepsen worker 84 - jepsen.util 3324 :ok :read [28 45] INFO [2022-01-10 05:45:00,112] jepsen worker 84 - jepsen.util 3324 :invoke :read [28 nil] INFO [2022-01-10 05:45:00,169] jepsen worker 83 - jepsen.util 3773 :ok :read [27 30] INFO [2022-01-10 05:45:00,170] jepsen worker 83 - jepsen.util 3773 :invoke :read [27 nil] INFO [2022-01-10 05:45:00,196] jepsen worker 66 - jepsen.util 3036 :ok :read [22 41] INFO [2022-01-10 05:45:00,197] jepsen worker 66 - jepsen.util 3036 :invoke :write [22 23] INFO [2022-01-10 05:45:00,223] jepsen worker 86 - jepsen.util 2696 :ok :read [28 45] INFO [2022-01-10 05:45:00,223] jepsen worker 86 - jepsen.util 2696 :invoke :write [28 9] INFO [2022-01-10 05:45:00,227] jepsen worker 84 - jepsen.util 3324 :ok :read [28 45] INFO [2022-01-10 05:45:00,228] jepsen worker 84 - jepsen.util 3324 :invoke :read [28 nil] INFO [2022-01-10 05:45:00,245] jepsen worker 60 - jepsen.util 3570 :info :write [20 4] :RequestTimeoutException INFO [2022-01-10 05:45:00,245] jepsen worker 60 - jepsen.util 3660 :invoke :read [20 nil] INFO [2022-01-10 05:45:00,248] jepsen worker 41 - jepsen.util 3101 :ok :read [13 15] INFO [2022-01-10 05:45:00,249] jepsen worker 41 - jepsen.util 3101 :invoke :read [13 nil] INFO [2022-01-10 05:45:00,256] jepsen worker 29 - jepsen.util 3089 :info :write [9 19] :RequestTimeoutException INFO [2022-01-10 05:45:00,257] jepsen worker 29 - jepsen.util 3179 :invoke :write [9 38] WARN [2022-01-10 05:45:00,282] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18149790,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18149790,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9024634,"encode_duration_us":20031,"operation_id":"0x6362","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10002367},{"operation_name":"upsert","last_dispatch_duration_us":9099614,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13322253,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":11252226,"encode_duration_us":17630,"operation_id":"0x6486","timeout_ms":10000,"last_server_duration_us":9075609,"total_duration_us":10009686},{"operation_name":"upsert","last_dispatch_duration_us":9157792,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11374796,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":10058597,"encode_duration_us":16831,"operation_id":"0x6557","timeout_ms":10000,"last_server_duration_us":9058604,"total_duration_us":10006005}],"total_count":3}} INFO [2022-01-10 05:45:00,289] jepsen worker 83 - jepsen.util 3773 :ok :read [27 30] INFO [2022-01-10 05:45:00,289] jepsen worker 83 - jepsen.util 3773 :invoke :write [27 23] INFO [2022-01-10 05:45:00,384] jepsen worker 50 - jepsen.util 3380 :ok :read [16 38] INFO [2022-01-10 05:45:00,385] jepsen worker 50 - jepsen.util 3380 :invoke :write [16 5] INFO [2022-01-10 05:45:00,406] jepsen worker 84 - jepsen.util 3324 :ok :read [28 45] INFO [2022-01-10 05:45:00,408] jepsen worker 84 - jepsen.util 3324 :invoke :write [28 43] INFO [2022-01-10 05:45:00,444] jepsen worker 20 - jepsen.util 2990 :ok :read [6 25] INFO [2022-01-10 05:45:00,445] jepsen worker 20 - jepsen.util 2990 :invoke :read [6 nil] INFO [2022-01-10 05:45:00,471] jepsen worker 28 - jepsen.util 2998 :info :write [9 25] :RequestTimeoutException INFO [2022-01-10 05:45:00,471] jepsen worker 28 - jepsen.util 3088 :invoke :read [9 nil] WARN [2022-01-10 05:45:00,483] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":349514,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":4535252,"total_server_duration_us":1385149,"encode_duration_us":16629,"operation_id":"0x64eb","timeout_ms":10000,"last_server_duration_us":58004,"total_duration_us":10010999},{"operation_name":"upsert","last_dispatch_duration_us":941044,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":4582035,"total_server_duration_us":737612,"encode_duration_us":20501,"operation_id":"0x668c","timeout_ms":10000,"last_server_duration_us":79971,"total_duration_us":10009369},{"operation_name":"upsert","last_dispatch_duration_us":111163,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3359933,"total_server_duration_us":806507,"encode_duration_us":19021,"operation_id":"0x6684","timeout_ms":10000,"last_server_duration_us":33035,"total_duration_us":10009110},{"operation_name":"upsert","last_dispatch_duration_us":382744,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5830549,"total_server_duration_us":1326126,"encode_duration_us":16696,"operation_id":"0x664d","timeout_ms":10000,"last_server_duration_us":88246,"total_duration_us":10006936},{"operation_name":"upsert","last_dispatch_duration_us":9157792,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11374796,"total_server_duration_us":10058597,"encode_duration_us":16831,"operation_id":"0x6557","timeout_ms":10000,"last_server_duration_us":9058604,"total_duration_us":10006005},{"operation_name":"upsert","last_dispatch_duration_us":475663,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5191184,"total_server_duration_us":860394,"encode_duration_us":19808,"operation_id":"0x6696","timeout_ms":10000,"last_server_duration_us":80398,"total_duration_us":10005137},{"operation_name":"upsert","last_dispatch_duration_us":709047,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":5360283,"total_server_duration_us":2374213,"encode_duration_us":21468,"operation_id":"0x651e","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10004728},{"operation_name":"upsert","last_dispatch_duration_us":132922,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":4966285,"total_server_duration_us":1789288,"encode_duration_us":21998,"operation_id":"0x65df","timeout_ms":10000,"last_server_duration_us":36729,"total_duration_us":10001358},{"operation_name":"upsert","last_dispatch_duration_us":8898601,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11541449,"total_server_duration_us":1139067,"encode_duration_us":21081,"operation_id":"0x657f","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10001322},{"operation_name":"get","last_dispatch_duration_us":461827,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":461827,"total_server_duration_us":7803,"operation_id":"0x669a","timeout_ms":30000,"last_server_duration_us":7803,"total_duration_us":3079655}],"total_count":12}} INFO [2022-01-10 05:45:00,497] jepsen worker 71 - jepsen.util 2681 :info :write [23 30] :SyncWriteAmbiguous INFO [2022-01-10 05:45:00,499] jepsen worker 71 - jepsen.util 2771 :invoke :write [23 45] INFO [2022-01-10 05:45:00,525] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:00,525] jepsen worker nemesis - couchbase.util Rebalance check count: 451 INFO [2022-01-10 05:45:00,525] jepsen worker nemesis - couchbase.util Rebalance stuck count: 364 INFO [2022-01-10 05:45:00,525] jepsen worker 41 - jepsen.util 3101 :ok :read [13 15] INFO [2022-01-10 05:45:00,526] jepsen worker 41 - jepsen.util 3101 :invoke :write [13 5] INFO [2022-01-10 05:45:00,528] jepsen worker 30 - jepsen.util 3090 :ok :read [10 36] INFO [2022-01-10 05:45:00,528] jepsen worker 30 - jepsen.util 3090 :invoke :read [10 nil] INFO [2022-01-10 05:45:00,530] jepsen worker 68 - jepsen.util 3218 :info :write [22 46] :RequestTimeoutException INFO [2022-01-10 05:45:00,531] jepsen worker 68 - jepsen.util 3308 :invoke :write [22 21] INFO [2022-01-10 05:45:00,586] jepsen worker 34 - jepsen.util 3184 :info :write [11 31] :RequestTimeoutException INFO [2022-01-10 05:45:00,587] jepsen worker 34 - jepsen.util 3274 :invoke :read [11 nil] INFO [2022-01-10 05:45:00,596] jepsen worker 87 - jepsen.util 3417 :ok :read [29 0] INFO [2022-01-10 05:45:00,597] jepsen worker 87 - jepsen.util 3417 :invoke :read [29 nil] INFO [2022-01-10 05:45:00,611] jepsen worker 4 - jepsen.util 3784 :ok :read [1 10] INFO [2022-01-10 05:45:00,612] jepsen worker 4 - jepsen.util 3784 :invoke :write [1 36] INFO [2022-01-10 05:45:00,642] jepsen worker 30 - jepsen.util 3090 :ok :read [10 36] INFO [2022-01-10 05:45:00,643] jepsen worker 30 - jepsen.util 3090 :invoke :read [10 nil] INFO [2022-01-10 05:45:00,687] jepsen worker 87 - jepsen.util 3417 :ok :read [29 0] INFO [2022-01-10 05:45:00,688] jepsen worker 87 - jepsen.util 3417 :invoke :write [29 19] INFO [2022-01-10 05:45:00,771] jepsen worker 30 - jepsen.util 3090 :ok :read [10 36] INFO [2022-01-10 05:45:00,771] jepsen worker 30 - jepsen.util 3090 :invoke :read [10 nil] INFO [2022-01-10 05:45:00,922] jepsen worker 30 - jepsen.util 3090 :ok :read [10 36] INFO [2022-01-10 05:45:00,922] jepsen worker 30 - jepsen.util 3090 :invoke :read [10 nil] INFO [2022-01-10 05:45:00,968] jepsen worker 46 - jepsen.util 3376 :ok :read [15 23] INFO [2022-01-10 05:45:00,969] jepsen worker 46 - jepsen.util 3376 :invoke :write [15 5] INFO [2022-01-10 05:45:00,995] jepsen worker 65 - jepsen.util 3125 :ok :read [21 7] INFO [2022-01-10 05:45:00,995] jepsen worker 65 - jepsen.util 3125 :invoke :read [21 nil] INFO [2022-01-10 05:45:01,070] jepsen worker 30 - jepsen.util 3090 :ok :read [10 36] INFO [2022-01-10 05:45:01,071] jepsen worker 30 - jepsen.util 3090 :invoke :read [10 nil] INFO [2022-01-10 05:45:01,076] jepsen worker 42 - jepsen.util 3282 :ok :read [14 14] INFO [2022-01-10 05:45:01,077] jepsen worker 42 - jepsen.util 3282 :invoke :read [14 nil] INFO [2022-01-10 05:45:01,200] jepsen worker 30 - jepsen.util 3090 :ok :read [10 36] INFO [2022-01-10 05:45:01,201] jepsen worker 30 - jepsen.util 3090 :invoke :read [10 nil] INFO [2022-01-10 05:45:01,204] jepsen worker 64 - jepsen.util 3304 :ok :read [21 7] INFO [2022-01-10 05:45:01,206] jepsen worker 64 - jepsen.util 3304 :invoke :read [21 nil] INFO [2022-01-10 05:45:01,226] jepsen worker 77 - jepsen.util 3497 :info :write [25 46] :RequestTimeoutException INFO [2022-01-10 05:45:01,227] jepsen worker 77 - jepsen.util 3587 :invoke :write [25 1] INFO [2022-01-10 05:45:01,252] jepsen worker 35 - jepsen.util 2915 :ok :read [11 21] INFO [2022-01-10 05:45:01,253] jepsen worker 35 - jepsen.util 2915 :invoke :write [11 1] INFO [2022-01-10 05:45:01,316] jepsen worker 30 - jepsen.util 3090 :ok :read [10 36] INFO [2022-01-10 05:45:01,317] jepsen worker 30 - jepsen.util 3090 :invoke :read [10 nil] INFO [2022-01-10 05:45:01,337] jepsen worker 42 - jepsen.util 3282 :ok :read [14 14] INFO [2022-01-10 05:45:01,337] jepsen worker 42 - jepsen.util 3282 :invoke :write [14 41] INFO [2022-01-10 05:45:01,472] jepsen worker 30 - jepsen.util 3090 :ok :read [10 36] INFO [2022-01-10 05:45:01,472] jepsen worker 30 - jepsen.util 3090 :invoke :write [10 44] INFO [2022-01-10 05:45:01,525] jepsen worker 20 - jepsen.util 2990 :ok :read [6 25] INFO [2022-01-10 05:45:01,528] jepsen worker 20 - jepsen.util 2990 :invoke :write [6 19] INFO [2022-01-10 05:45:01,528] jepsen worker 70 - jepsen.util 3220 :info :write [23 2] :RequestTimeoutException INFO [2022-01-10 05:45:01,528] jepsen worker 70 - jepsen.util 3310 :invoke :write [23 17] INFO [2022-01-10 05:45:01,540] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:01,540] jepsen worker nemesis - couchbase.util Rebalance check count: 452 INFO [2022-01-10 05:45:01,540] jepsen worker nemesis - couchbase.util Rebalance stuck count: 365 INFO [2022-01-10 05:45:01,542] jepsen worker 34 - jepsen.util 3274 :ok :read [11 21] INFO [2022-01-10 05:45:01,543] jepsen worker 34 - jepsen.util 3274 :invoke :read [11 nil] INFO [2022-01-10 05:45:01,604] jepsen worker 43 - jepsen.util 3103 :info :write [14 22] :RequestTimeoutException INFO [2022-01-10 05:45:01,605] jepsen worker 43 - jepsen.util 3193 :invoke :read [14 nil] INFO [2022-01-10 05:45:01,707] jepsen worker 44 - jepsen.util 3284 :info :write [14 18] :RequestTimeoutException INFO [2022-01-10 05:45:01,707] jepsen worker 44 - jepsen.util 3374 :invoke :write [14 15] INFO [2022-01-10 05:45:01,816] jepsen worker 26 - jepsen.util 2816 :info :write [8 30] :RequestTimeoutException INFO [2022-01-10 05:45:01,817] jepsen worker 26 - jepsen.util 2906 :invoke :write [8 43] INFO [2022-01-10 05:45:01,873] jepsen worker 36 - jepsen.util 2556 :ok :read [12 29] INFO [2022-01-10 05:45:01,874] jepsen worker 36 - jepsen.util 2556 :invoke :read [12 nil] INFO [2022-01-10 05:45:01,884] jepsen worker 10 - jepsen.util 3340 :ok :read [3 31] INFO [2022-01-10 05:45:01,885] jepsen worker 10 - jepsen.util 3340 :invoke :write [3 44] INFO [2022-01-10 05:45:01,918] jepsen worker 69 - jepsen.util 2859 :ok :read [23 26] INFO [2022-01-10 05:45:01,919] jepsen worker 69 - jepsen.util 2859 :invoke :read [23 nil] INFO [2022-01-10 05:45:01,943] jepsen worker 37 - jepsen.util 3097 :ok :read [12 29] INFO [2022-01-10 05:45:01,944] jepsen worker 37 - jepsen.util 3097 :invoke :write [12 2] INFO [2022-01-10 05:45:01,974] jepsen worker 36 - jepsen.util 2556 :ok :read [12 29] INFO [2022-01-10 05:45:01,975] jepsen worker 36 - jepsen.util 2556 :invoke :write [12 19] INFO [2022-01-10 05:45:01,982] jepsen worker 45 - jepsen.util 3465 :info :write [15 33] :RequestTimeoutException INFO [2022-01-10 05:45:01,983] jepsen worker 45 - jepsen.util 3555 :invoke :write [15 10] WARN [2022-01-10 05:45:01,985] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17722239,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20162122,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9161464,"encode_duration_us":16969,"operation_id":"0x6454","timeout_ms":10000,"last_server_duration_us":9046921,"total_duration_us":10003877},{"operation_name":"upsert","last_dispatch_duration_us":9630799,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13711594,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9341372,"encode_duration_us":16421,"operation_id":"0x64ab","timeout_ms":10000,"last_server_duration_us":9173654,"total_duration_us":10008001},{"operation_name":"upsert","last_dispatch_duration_us":9659359,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11151879,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9355253,"encode_duration_us":19067,"operation_id":"0x655f","timeout_ms":10000,"last_server_duration_us":9223914,"total_duration_us":10003980},{"operation_name":"upsert","last_dispatch_duration_us":9865609,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10477461,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9186847,"encode_duration_us":18715,"operation_id":"0x6581","timeout_ms":10000,"last_server_duration_us":9164043,"total_duration_us":10009090}],"total_count":4}} WARN [2022-01-10 05:45:01,985] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9240692,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9240692,"total_server_duration_us":7437,"operation_id":"0x6603","timeout_ms":30000,"last_server_duration_us":7437,"total_duration_us":11861356},{"operation_name":"get","last_dispatch_duration_us":8865700,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":8865700,"total_server_duration_us":23,"operation_id":"0x664c","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":10464607},{"operation_name":"upsert","last_dispatch_duration_us":9865609,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10477461,"total_server_duration_us":9186847,"encode_duration_us":18715,"operation_id":"0x6581","timeout_ms":10000,"last_server_duration_us":9164043,"total_duration_us":10009090},{"operation_name":"upsert","last_dispatch_duration_us":909539,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10733939,"total_server_duration_us":170083,"encode_duration_us":16194,"operation_id":"0x6698","timeout_ms":10000,"last_server_duration_us":21977,"total_duration_us":10007062},{"operation_name":"upsert","last_dispatch_duration_us":325998,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5366602,"total_server_duration_us":833192,"encode_duration_us":18288,"operation_id":"0x6552","timeout_ms":10000,"last_server_duration_us":232986,"total_duration_us":10006659},{"operation_name":"upsert","last_dispatch_duration_us":387204,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":3510009,"total_server_duration_us":1626632,"encode_duration_us":17397,"operation_id":"0x6553","timeout_ms":10000,"last_server_duration_us":67353,"total_duration_us":10005312},{"operation_name":"upsert","last_dispatch_duration_us":9734842,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":12658453,"total_server_duration_us":9579859,"encode_duration_us":18188,"operation_id":"0x65b3","timeout_ms":10000,"last_server_duration_us":9318333,"total_duration_us":10005147},{"operation_name":"upsert","last_dispatch_duration_us":331563,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":3971961,"total_server_duration_us":2470228,"encode_duration_us":16193,"operation_id":"0x64f1","timeout_ms":10000,"last_server_duration_us":319968,"total_duration_us":10004837},{"operation_name":"upsert","last_dispatch_duration_us":9659359,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11151879,"total_server_duration_us":9355253,"encode_duration_us":19067,"operation_id":"0x655f","timeout_ms":10000,"last_server_duration_us":9223914,"total_duration_us":10003980},{"operation_name":"upsert","last_dispatch_duration_us":107962,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4593187,"total_server_duration_us":1422531,"encode_duration_us":20765,"operation_id":"0x65ca","timeout_ms":10000,"last_server_duration_us":36933,"total_duration_us":10002980}],"total_count":18}} WARN [2022-01-10 05:45:01,986] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":111918,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":6145205,"total_server_duration_us":740881,"encode_duration_us":28574,"operation_id":"0x6697","timeout_ms":10000,"last_server_duration_us":32258,"total_duration_us":10009905},{"operation_name":"upsert","last_dispatch_duration_us":34381,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9307560,"total_server_duration_us":234789,"encode_duration_us":16948,"operation_id":"0x65c0","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10008829},{"operation_name":"upsert","last_dispatch_duration_us":122079,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":497243,"total_server_duration_us":85829,"encode_duration_us":18396,"operation_id":"0x65c2","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10008155},{"operation_name":"upsert","last_dispatch_duration_us":133155,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":532526,"total_server_duration_us":106878,"encode_duration_us":18217,"operation_id":"0x65bf","timeout_ms":10000,"last_server_duration_us":23222,"total_duration_us":10006036},{"operation_name":"upsert","last_dispatch_duration_us":1072852,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10658429,"total_server_duration_us":811257,"encode_duration_us":19222,"operation_id":"0x64e8","timeout_ms":10000,"last_server_duration_us":363172,"total_duration_us":10006022},{"operation_name":"upsert","last_dispatch_duration_us":31041,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9008569,"total_server_duration_us":231279,"encode_duration_us":18642,"operation_id":"0x65ef","timeout_ms":10000,"last_server_duration_us":20521,"total_duration_us":10005893},{"operation_name":"upsert","last_dispatch_duration_us":153416,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":2501834,"total_server_duration_us":263563,"encode_duration_us":19527,"operation_id":"0x6636","timeout_ms":10000,"last_server_duration_us":36017,"total_duration_us":10004065},{"operation_name":"upsert","last_dispatch_duration_us":1051270,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9121582,"total_server_duration_us":428835,"encode_duration_us":19639,"operation_id":"0x6551","timeout_ms":10000,"last_server_duration_us":44611,"total_duration_us":10003525},{"operation_name":"upsert","last_dispatch_duration_us":528094,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":2682751,"total_server_duration_us":435958,"encode_duration_us":20652,"operation_id":"0x66c4","timeout_ms":10000,"last_server_duration_us":116782,"total_duration_us":10003178},{"operation_name":"upsert","last_dispatch_duration_us":258459,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":3263732,"total_server_duration_us":817285,"encode_duration_us":16540,"operation_id":"0x66d2","timeout_ms":10000,"last_server_duration_us":123884,"total_duration_us":10003096}],"total_count":17}} WARN [2022-01-10 05:45:01,987] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9054914,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10009156,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9328675,"encode_duration_us":19663,"operation_id":"0x6601","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10002612},{"operation_name":"upsert","last_dispatch_duration_us":10140257,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10140257,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9044797,"encode_duration_us":17616,"operation_id":"0x6663","timeout_ms":10000,"last_server_duration_us":9044797,"total_duration_us":10008218},{"operation_name":"upsert","last_dispatch_duration_us":10114353,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17599251,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9696552,"encode_duration_us":16449,"operation_id":"0x64b5","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10003203}],"total_count":3}} INFO [2022-01-10 05:45:02,025] jepsen worker 31 - jepsen.util 3631 :info :write [10 10] :RequestTimeoutException INFO [2022-01-10 05:45:02,026] jepsen worker 31 - jepsen.util 3721 :invoke :write [10 16] INFO [2022-01-10 05:45:02,026] jepsen worker 69 - jepsen.util 2859 :ok :read [23 26] INFO [2022-01-10 05:45:02,027] jepsen worker 69 - jepsen.util 2859 :invoke :write [23 4] WARN [2022-01-10 05:45:02,029] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17632264,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18148793,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9216543,"encode_duration_us":17555,"operation_id":"0x64d8","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10002508},{"operation_name":"upsert","last_dispatch_duration_us":9890886,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17607970,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":11298006,"encode_duration_us":18283,"operation_id":"0x6446","timeout_ms":10000,"last_server_duration_us":9309733,"total_duration_us":10006006}],"total_count":2}} INFO [2022-01-10 05:45:02,057] jepsen worker 38 - jepsen.util 3548 :ok :read [12 29] INFO [2022-01-10 05:45:02,058] jepsen worker 38 - jepsen.util 3548 :invoke :read [12 nil] WARN [2022-01-10 05:45:02,085] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8665700,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":15973354,"total_server_duration_us":626473,"encode_duration_us":18921,"operation_id":"0x6559","timeout_ms":10000,"last_server_duration_us":48794,"total_duration_us":10008968},{"operation_name":"upsert","last_dispatch_duration_us":8470207,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10999699,"total_server_duration_us":262047,"encode_duration_us":16829,"operation_id":"0x64ed","timeout_ms":10000,"last_server_duration_us":41767,"total_duration_us":10008956},{"operation_name":"upsert","last_dispatch_duration_us":9437436,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11211826,"total_server_duration_us":9622276,"encode_duration_us":16440,"operation_id":"0x64e6","timeout_ms":10000,"last_server_duration_us":9365696,"total_duration_us":10008529},{"operation_name":"upsert","last_dispatch_duration_us":55805,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8647774,"total_server_duration_us":784986,"encode_duration_us":16508,"operation_id":"0x662a","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10007948},{"operation_name":"upsert","last_dispatch_duration_us":9043629,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9414515,"total_server_duration_us":9252663,"encode_duration_us":20894,"operation_id":"0x6511","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10007794},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":16927,"operation_id":"0x6602","timeout_ms":10000,"total_duration_us":10006350},{"operation_name":"upsert","last_dispatch_duration_us":6763780,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":7180503,"total_server_duration_us":264263,"encode_duration_us":19529,"operation_id":"0x655e","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10006008},{"operation_name":"upsert","last_dispatch_duration_us":8870711,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13404808,"total_server_duration_us":447319,"encode_duration_us":16168,"operation_id":"0x65cc","timeout_ms":10000,"last_server_duration_us":84138,"total_duration_us":10003076},{"operation_name":"upsert","last_dispatch_duration_us":8678843,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":16630936,"total_server_duration_us":490878,"encode_duration_us":24443,"operation_id":"0x654a","timeout_ms":10000,"last_server_duration_us":21977,"total_duration_us":10001856},{"operation_name":"upsert","last_dispatch_duration_us":64706,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":3311757,"total_server_duration_us":2289287,"encode_duration_us":17737,"operation_id":"0x64d7","timeout_ms":10000,"last_server_duration_us":51356,"total_duration_us":10001426}],"total_count":22}} WARN [2022-01-10 05:45:02,087] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7659440,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12445642,"total_server_duration_us":431383,"encode_duration_us":22350,"operation_id":"0x65e7","timeout_ms":10000,"last_server_duration_us":152553,"total_duration_us":10009650},{"operation_name":"upsert","last_dispatch_duration_us":99161,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4929483,"total_server_duration_us":944344,"encode_duration_us":18500,"operation_id":"0x65bd","timeout_ms":10000,"last_server_duration_us":72045,"total_duration_us":10009056},{"operation_name":"upsert","last_dispatch_duration_us":10140257,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10140257,"total_server_duration_us":9044797,"encode_duration_us":17616,"operation_id":"0x6663","timeout_ms":10000,"last_server_duration_us":9044797,"total_duration_us":10008218},{"operation_name":"upsert","last_dispatch_duration_us":7726451,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12347357,"total_server_duration_us":346526,"encode_duration_us":18879,"operation_id":"0x65c6","timeout_ms":10000,"last_server_duration_us":44833,"total_duration_us":10007742},{"operation_name":"upsert","last_dispatch_duration_us":8339969,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8515757,"total_server_duration_us":134038,"encode_duration_us":21172,"operation_id":"0x66c9","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10006832},{"operation_name":"upsert","last_dispatch_duration_us":7163118,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12505502,"total_server_duration_us":374989,"encode_duration_us":16946,"operation_id":"0x65ad","timeout_ms":10000,"last_server_duration_us":60254,"total_duration_us":10006695},{"operation_name":"upsert","last_dispatch_duration_us":8242846,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9145854,"total_server_duration_us":338342,"encode_duration_us":17167,"operation_id":"0x65fe","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10006408},{"operation_name":"upsert","last_dispatch_duration_us":42769,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":920924,"total_server_duration_us":314968,"encode_duration_us":23251,"operation_id":"0x65fc","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10006392},{"operation_name":"upsert","last_dispatch_duration_us":8930575,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10183312,"total_server_duration_us":218853,"encode_duration_us":18173,"operation_id":"0x6638","timeout_ms":10000,"last_server_duration_us":101853,"total_duration_us":10003966},{"operation_name":"upsert","last_dispatch_duration_us":182585,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":3820217,"total_server_duration_us":1753797,"encode_duration_us":19021,"operation_id":"0x657e","timeout_ms":10000,"last_server_duration_us":84138,"total_duration_us":10003304}],"total_count":17}} WARN [2022-01-10 05:45:02,186] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9994438,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12885619,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9320189,"encode_duration_us":16947,"operation_id":"0x65b0","timeout_ms":10000,"last_server_duration_us":9253912,"total_duration_us":10000838},{"operation_name":"upsert","last_dispatch_duration_us":9437436,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11211826,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9622276,"encode_duration_us":16440,"operation_id":"0x64e6","timeout_ms":10000,"last_server_duration_us":9365696,"total_duration_us":10008529},{"operation_name":"upsert","last_dispatch_duration_us":9043629,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9414515,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9252663,"encode_duration_us":20894,"operation_id":"0x6511","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10007794}],"total_count":3}} INFO [2022-01-10 05:45:02,198] jepsen worker 43 - jepsen.util 3193 :ok :read [14 14] INFO [2022-01-10 05:45:02,198] jepsen worker 43 - jepsen.util 3193 :invoke :read [14 nil] INFO [2022-01-10 05:45:02,226] jepsen worker 80 - jepsen.util 3950 :info :write [26 32] :SyncWriteAmbiguous INFO [2022-01-10 05:45:02,227] jepsen worker 80 - jepsen.util 4040 :invoke :read [26 nil] WARN [2022-01-10 05:45:02,230] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9629851,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11240211,"total_server_duration_us":970355,"encode_duration_us":20552,"operation_id":"0x6605","timeout_ms":10000,"last_server_duration_us":80683,"total_duration_us":10009551},{"operation_name":"upsert","last_dispatch_duration_us":39537,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7461854,"total_server_duration_us":223248,"encode_duration_us":20036,"operation_id":"0x667d","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10009226},{"operation_name":"upsert","last_dispatch_duration_us":9205150,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":13194585,"total_server_duration_us":1118560,"encode_duration_us":24976,"operation_id":"0x655c","timeout_ms":10000,"last_server_duration_us":122347,"total_duration_us":10009163},{"operation_name":"upsert","last_dispatch_duration_us":584171,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10252906,"total_server_duration_us":201734,"encode_duration_us":19329,"operation_id":"0x6662","timeout_ms":10000,"last_server_duration_us":120480,"total_duration_us":10008525},{"operation_name":"upsert","last_dispatch_duration_us":9689201,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":14614886,"total_server_duration_us":913414,"encode_duration_us":18859,"operation_id":"0x6510","timeout_ms":10000,"last_server_duration_us":57756,"total_duration_us":10008444},{"operation_name":"upsert","last_dispatch_duration_us":1353284,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":1353284,"total_server_duration_us":26418,"encode_duration_us":20660,"operation_id":"0x6699","timeout_ms":10000,"last_server_duration_us":26418,"total_duration_us":10008034},{"operation_name":"upsert","last_dispatch_duration_us":1775950,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10330054,"total_server_duration_us":158773,"encode_duration_us":17844,"operation_id":"0x658a","timeout_ms":10000,"last_server_duration_us":54094,"total_duration_us":10007566},{"operation_name":"upsert","last_dispatch_duration_us":32733,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":4322988,"total_server_duration_us":213615,"encode_duration_us":16145,"operation_id":"0x665b","timeout_ms":10000,"last_server_duration_us":20680,"total_duration_us":10007494},{"operation_name":"upsert","last_dispatch_duration_us":31391,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":6886274,"total_server_duration_us":234793,"encode_duration_us":16337,"operation_id":"0x668b","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10007392},{"operation_name":"upsert","last_dispatch_duration_us":31625,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":6594453,"total_server_duration_us":536654,"encode_duration_us":29335,"operation_id":"0x668a","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10007042}],"total_count":18}} INFO [2022-01-10 05:45:02,340] jepsen worker 48 - jepsen.util 2838 :info :write [16 9] :RequestTimeoutException INFO [2022-01-10 05:45:02,340] jepsen worker 48 - jepsen.util 2928 :invoke :read [16 nil] INFO [2022-01-10 05:45:02,434] jepsen worker 80 - jepsen.util 4040 :ok :read [26 10] INFO [2022-01-10 05:45:02,442] jepsen worker 79 - jepsen.util 3769 :info :write [26 25] :RequestTimeoutException INFO [2022-01-10 05:45:02,564] jepsen worker 79 - jepsen.util 3859 :invoke :read [26 nil] INFO [2022-01-10 05:45:02,486] jepsen worker 49 - jepsen.util 3109 :info :write [16 29] :RequestTimeoutException INFO [2022-01-10 05:45:02,516] jepsen worker 78 - jepsen.util 3858 :info :write [26 4] :RequestTimeoutException INFO [2022-01-10 05:45:02,564] jepsen worker 49 - jepsen.util 3199 :invoke :write [16 0] INFO [2022-01-10 05:45:02,546] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:02,564] jepsen worker nemesis - couchbase.util Rebalance check count: 453 INFO [2022-01-10 05:45:02,564] jepsen worker 78 - jepsen.util 3948 :invoke :write [26 47] INFO [2022-01-10 05:45:02,564] jepsen worker 80 - jepsen.util 4040 :invoke :write [26 41] INFO [2022-01-10 05:45:02,564] jepsen worker nemesis - couchbase.util Rebalance stuck count: 366 INFO [2022-01-10 05:45:02,759] jepsen worker 43 - jepsen.util 3193 :ok :read [14 14] INFO [2022-01-10 05:45:02,760] jepsen worker 43 - jepsen.util 3193 :invoke :read [14 nil] INFO [2022-01-10 05:45:02,905] jepsen worker 63 - jepsen.util 3213 :info :write [21 31] :RequestTimeoutException INFO [2022-01-10 05:45:02,906] jepsen worker 63 - jepsen.util 3303 :invoke :write [21 33] INFO [2022-01-10 05:45:02,936] jepsen worker 55 - jepsen.util 3565 :info :write [18 47] :RequestTimeoutException INFO [2022-01-10 05:45:02,936] jepsen worker 55 - jepsen.util 3655 :invoke :write [18 1] INFO [2022-01-10 05:45:02,986] jepsen worker 73 - jepsen.util 3043 :ok :read [24 18] INFO [2022-01-10 05:45:02,987] jepsen worker 73 - jepsen.util 3043 :invoke :write [24 27] INFO [2022-01-10 05:45:03,061] jepsen worker 43 - jepsen.util 3193 :ok :read [14 14] INFO [2022-01-10 05:45:03,062] jepsen worker 43 - jepsen.util 3193 :invoke :read [14 nil] INFO [2022-01-10 05:45:03,099] jepsen worker 79 - jepsen.util 3859 :ok :read [26 10] INFO [2022-01-10 05:45:03,100] jepsen worker 79 - jepsen.util 3859 :invoke :write [26 15] INFO [2022-01-10 05:45:03,349] jepsen worker 53 - jepsen.util 3923 :info :write [17 42] :RequestTimeoutException INFO [2022-01-10 05:45:03,350] jepsen worker 53 - jepsen.util 4013 :invoke :read [17 nil] INFO [2022-01-10 05:45:03,411] jepsen worker 58 - jepsen.util 3208 :info :write [19 5] :RequestTimeoutException INFO [2022-01-10 05:45:03,412] jepsen worker 58 - jepsen.util 3298 :invoke :read [19 nil] INFO [2022-01-10 05:45:03,551] jepsen worker 32 - jepsen.util 3542 :info :write [10 30] :RequestTimeoutException INFO [2022-01-10 05:45:03,552] jepsen worker 32 - jepsen.util 3632 :invoke :write [10 31] INFO [2022-01-10 05:45:03,569] jepsen worker 43 - jepsen.util 3193 :ok :read [14 14] INFO [2022-01-10 05:45:03,570] jepsen worker 43 - jepsen.util 3193 :invoke :write [14 14] INFO [2022-01-10 05:45:03,574] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:03,574] jepsen worker nemesis - couchbase.util Rebalance check count: 454 INFO [2022-01-10 05:45:03,574] jepsen worker nemesis - couchbase.util Rebalance stuck count: 367 INFO [2022-01-10 05:45:03,905] jepsen worker 74 - jepsen.util 3764 :info :write [24 24] :RequestTimeoutException INFO [2022-01-10 05:45:03,906] jepsen worker 74 - jepsen.util 3854 :invoke :write [24 20] INFO [2022-01-10 05:45:03,985] jepsen worker 75 - jepsen.util 2775 :info :write [25 49] :RequestTimeoutException INFO [2022-01-10 05:45:03,985] jepsen worker 75 - jepsen.util 2865 :invoke :write [25 45] INFO [2022-01-10 05:45:04,133] jepsen worker 53 - jepsen.util 4013 :ok :read [17 41] INFO [2022-01-10 05:45:04,134] jepsen worker 53 - jepsen.util 4013 :invoke :write [17 27] INFO [2022-01-10 05:45:04,272] jepsen worker 18 - jepsen.util 3168 :info :write [6 6] :RequestTimeoutException INFO [2022-01-10 05:45:04,272] jepsen worker 18 - jepsen.util 3258 :invoke :write [6 30] INFO [2022-01-10 05:45:04,276] jepsen worker 13 - jepsen.util 3703 :info :write [4 38] :RequestTimeoutException INFO [2022-01-10 05:45:04,277] jepsen worker 13 - jepsen.util 3793 :invoke :write [4 21] INFO [2022-01-10 05:45:04,326] jepsen worker 21 - jepsen.util 3441 :info :write [7 49] :RequestTimeoutException INFO [2022-01-10 05:45:04,326] jepsen worker 21 - jepsen.util 3531 :invoke :read [7 nil] INFO [2022-01-10 05:45:04,540] jepsen worker 5 - jepsen.util 3245 :ok :read [1 10] INFO [2022-01-10 05:45:04,541] jepsen worker 5 - jepsen.util 3245 :invoke :read [1 nil] INFO [2022-01-10 05:45:04,571] jepsen worker 21 - jepsen.util 3531 :ok :read [7 :nil] INFO [2022-01-10 05:45:04,571] jepsen worker 21 - jepsen.util 3531 :invoke :read [7 nil] INFO [2022-01-10 05:45:04,581] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:04,581] jepsen worker nemesis - couchbase.util Rebalance check count: 455 INFO [2022-01-10 05:45:04,581] jepsen worker nemesis - couchbase.util Rebalance stuck count: 368 INFO [2022-01-10 05:45:04,609] jepsen worker 7 - jepsen.util 3337 :info :write [2 43] :RequestTimeoutException INFO [2022-01-10 05:45:04,610] jepsen worker 7 - jepsen.util 3427 :invoke :write [2 4] INFO [2022-01-10 05:45:04,676] jepsen worker 88 - jepsen.util 3238 :info :write [29 43] :RequestTimeoutException INFO [2022-01-10 05:45:04,677] jepsen worker 88 - jepsen.util 3328 :invoke :write [29 13] INFO [2022-01-10 05:45:04,696] jepsen worker 14 - jepsen.util 3974 :info :write [4 3] :RequestTimeoutException INFO [2022-01-10 05:45:04,696] jepsen worker 14 - jepsen.util 4064 :invoke :read [4 nil] INFO [2022-01-10 05:45:04,710] jepsen worker 57 - jepsen.util 3927 :info :write [19 41] :RequestTimeoutException INFO [2022-01-10 05:45:04,711] jepsen worker 72 - jepsen.util 3672 :info :write [24 2] :RequestTimeoutException INFO [2022-01-10 05:45:04,711] jepsen worker 57 - jepsen.util 4017 :invoke :read [19 nil] INFO [2022-01-10 05:45:04,712] jepsen worker 72 - jepsen.util 3762 :invoke :read [24 nil] INFO [2022-01-10 05:45:04,790] jepsen worker 57 - jepsen.util 4017 :ok :read [19 31] INFO [2022-01-10 05:45:04,791] jepsen worker 57 - jepsen.util 4017 :invoke :write [19 48] INFO [2022-01-10 05:45:04,875] jepsen worker 14 - jepsen.util 4064 :ok :read [4 3] INFO [2022-01-10 05:45:04,876] jepsen worker 14 - jepsen.util 4064 :invoke :read [4 nil] INFO [2022-01-10 05:45:04,951] jepsen worker 14 - jepsen.util 4064 :ok :read [4 3] INFO [2022-01-10 05:45:04,952] jepsen worker 14 - jepsen.util 4064 :invoke :read [4 nil] INFO [2022-01-10 05:45:04,954] jepsen worker 25 - jepsen.util 3085 :info :write [8 4] :RequestTimeoutException INFO [2022-01-10 05:45:04,955] jepsen worker 25 - jepsen.util 3175 :invoke :read [8 nil] INFO [2022-01-10 05:45:05,045] jepsen worker 52 - jepsen.util 4102 :info :write [17 32] :RequestTimeoutException INFO [2022-01-10 05:45:05,046] jepsen worker 52 - jepsen.util 4192 :invoke :read [17 nil] INFO [2022-01-10 05:45:05,051] jepsen worker 14 - jepsen.util 4064 :ok :read [4 3] INFO [2022-01-10 05:45:05,052] jepsen worker 14 - jepsen.util 4064 :invoke :read [4 nil] INFO [2022-01-10 05:45:05,063] jepsen worker 21 - jepsen.util 3531 :ok :read [7 :nil] INFO [2022-01-10 05:45:05,063] jepsen worker 21 - jepsen.util 3531 :invoke :read [7 nil] INFO [2022-01-10 05:45:05,089] jepsen worker 14 - jepsen.util 4064 :ok :read [4 3] INFO [2022-01-10 05:45:05,089] jepsen worker 14 - jepsen.util 4064 :invoke :read [4 nil] INFO [2022-01-10 05:45:05,182] jepsen worker 27 - jepsen.util 3267 :info :write [9 28] :RequestTimeoutException INFO [2022-01-10 05:45:05,182] jepsen worker 27 - jepsen.util 3357 :invoke :write [9 23] INFO [2022-01-10 05:45:05,195] jepsen worker 14 - jepsen.util 4064 :ok :read [4 3] INFO [2022-01-10 05:45:05,195] jepsen worker 14 - jepsen.util 4064 :invoke :write [4 7] INFO [2022-01-10 05:45:05,300] jepsen worker 52 - jepsen.util 4192 :ok :read [17 41] INFO [2022-01-10 05:45:05,301] jepsen worker 9 - jepsen.util 3789 :info :write [3 24] :RequestTimeoutException INFO [2022-01-10 05:45:05,301] jepsen worker 52 - jepsen.util 4192 :invoke :write [17 13] INFO [2022-01-10 05:45:05,302] jepsen worker 9 - jepsen.util 3879 :invoke :read [3 nil] INFO [2022-01-10 05:45:05,587] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:05,587] jepsen worker nemesis - couchbase.util Rebalance check count: 456 INFO [2022-01-10 05:45:05,587] jepsen worker nemesis - couchbase.util Rebalance stuck count: 369 INFO [2022-01-10 05:45:05,884] jepsen worker 2 - jepsen.util 3512 :info :write [0 8] :RequestTimeoutException INFO [2022-01-10 05:45:05,885] jepsen worker 2 - jepsen.util 3602 :invoke :write [0 36] INFO [2022-01-10 05:45:06,035] jepsen worker 82 - jepsen.util 3412 :info :write [27 45] :SyncWriteAmbiguous INFO [2022-01-10 05:45:06,035] jepsen worker 22 - jepsen.util 2992 :info :write [7 36] :RequestTimeoutException INFO [2022-01-10 05:45:06,036] jepsen worker 82 - jepsen.util 3502 :invoke :read [27 nil] INFO [2022-01-10 05:45:06,037] jepsen worker 22 - jepsen.util 3082 :invoke :read [7 nil] INFO [2022-01-10 05:45:06,100] jepsen worker 8 - jepsen.util 3788 :info :write [2 17] :RequestTimeoutException INFO [2022-01-10 05:45:06,100] jepsen worker 8 - jepsen.util 3878 :invoke :read [2 nil] INFO [2022-01-10 05:45:06,282] jepsen worker 6 - jepsen.util 3336 :info :write [2 22] :RequestTimeoutException INFO [2022-01-10 05:45:06,282] jepsen worker 6 - jepsen.util 3426 :invoke :read [2 nil] INFO [2022-01-10 05:45:06,316] jepsen worker 11 - jepsen.util 3251 :info :write [3 47] :RequestTimeoutException INFO [2022-01-10 05:45:06,317] jepsen worker 11 - jepsen.util 3341 :invoke :write [3 32] INFO [2022-01-10 05:45:06,550] jepsen worker 81 - jepsen.util 3501 :info :write [27 41] :RequestTimeoutException INFO [2022-01-10 05:45:06,550] jepsen worker 81 - jepsen.util 3591 :invoke :write [27 18] INFO [2022-01-10 05:45:06,594] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:06,594] jepsen worker nemesis - couchbase.util Rebalance check count: 457 INFO [2022-01-10 05:45:06,594] jepsen worker nemesis - couchbase.util Rebalance stuck count: 370 INFO [2022-01-10 05:45:06,645] jepsen worker 51 - jepsen.util 3921 :info :write [17 39] :RequestTimeoutException INFO [2022-01-10 05:45:06,646] jepsen worker 51 - jepsen.util 4011 :invoke :read [17 nil] INFO [2022-01-10 05:45:06,680] jepsen worker 1 - jepsen.util 3331 :ok :read [0 46] INFO [2022-01-10 05:45:06,680] jepsen worker 1 - jepsen.util 3331 :invoke :read [0 nil] INFO [2022-01-10 05:45:06,696] jepsen worker 33 - jepsen.util 3543 :info :write [11 13] :RequestTimeoutException INFO [2022-01-10 05:45:06,696] jepsen worker 33 - jepsen.util 3633 :invoke :write [11 43] INFO [2022-01-10 05:45:06,811] jepsen worker 12 - jepsen.util 3612 :info :write [4 31] :RequestTimeoutException INFO [2022-01-10 05:45:06,811] jepsen worker 1 - jepsen.util 3331 :ok :read [0 46] INFO [2022-01-10 05:45:06,812] jepsen worker 1 - jepsen.util 3331 :invoke :write [0 18] INFO [2022-01-10 05:45:06,812] jepsen worker 12 - jepsen.util 3702 :invoke :read [4 nil] INFO [2022-01-10 05:45:06,834] jepsen worker 16 - jepsen.util 3346 :info :write [5 2] :RequestTimeoutException INFO [2022-01-10 05:45:06,835] jepsen worker 16 - jepsen.util 3436 :invoke :read [5 nil] INFO [2022-01-10 05:45:06,904] jepsen worker 12 - jepsen.util 3702 :ok :read [4 3] INFO [2022-01-10 05:45:06,905] jepsen worker 12 - jepsen.util 3702 :invoke :read [4 nil] INFO [2022-01-10 05:45:07,025] jepsen worker 51 - jepsen.util 4011 :ok :read [17 41] INFO [2022-01-10 05:45:07,026] jepsen worker 51 - jepsen.util 4011 :invoke :read [17 nil] INFO [2022-01-10 05:45:07,048] jepsen worker 12 - jepsen.util 3702 :ok :read [4 3] INFO [2022-01-10 05:45:07,049] jepsen worker 12 - jepsen.util 3702 :invoke :read [4 nil] INFO [2022-01-10 05:45:07,098] jepsen worker 12 - jepsen.util 3702 :ok :read [4 3] INFO [2022-01-10 05:45:07,098] jepsen worker 12 - jepsen.util 3702 :invoke :write [4 3] INFO [2022-01-10 05:45:07,197] jepsen worker 51 - jepsen.util 4011 :ok :read [17 41] INFO [2022-01-10 05:45:07,197] jepsen worker 51 - jepsen.util 4011 :invoke :write [17 48] INFO [2022-01-10 05:45:07,349] jepsen worker 3 - jepsen.util 3243 :info :write [1 33] :RequestTimeoutException INFO [2022-01-10 05:45:07,350] jepsen worker 3 - jepsen.util 3333 :invoke :read [1 nil] INFO [2022-01-10 05:45:07,475] jepsen worker 62 - jepsen.util 3662 :info :write [20 2] :RequestTimeoutException INFO [2022-01-10 05:45:07,479] jepsen worker 62 - jepsen.util 3752 :invoke :write [20 22] INFO [2022-01-10 05:45:07,489] jepsen worker 85 - jepsen.util 3595 :info :write [28 27] :RequestTimeoutException INFO [2022-01-10 05:45:07,490] jepsen worker 85 - jepsen.util 3685 :invoke :write [28 39] INFO [2022-01-10 05:45:07,601] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:07,601] jepsen worker nemesis - couchbase.util Rebalance check count: 458 INFO [2022-01-10 05:45:07,601] jepsen worker nemesis - couchbase.util Rebalance stuck count: 371 INFO [2022-01-10 05:45:07,730] jepsen worker 56 - jepsen.util 3746 :info :write [18 45] :RequestTimeoutException INFO [2022-01-10 05:45:07,730] jepsen worker 56 - jepsen.util 3836 :invoke :read [18 nil] INFO [2022-01-10 05:45:07,902] jepsen worker 59 - jepsen.util 3209 :info :write [19 34] :RequestTimeoutException INFO [2022-01-10 05:45:07,903] jepsen worker 59 - jepsen.util 3299 :invoke :write [19 18] INFO [2022-01-10 05:45:08,016] jepsen worker 47 - jepsen.util 3287 :info :write [15 37] :RequestTimeoutException INFO [2022-01-10 05:45:08,017] jepsen worker 47 - jepsen.util 3377 :invoke :read [15 nil] INFO [2022-01-10 05:45:08,164] jepsen worker 76 - jepsen.util 2866 :info :write [25 36] :RequestTimeoutException INFO [2022-01-10 05:45:08,165] jepsen worker 76 - jepsen.util 2956 :invoke :read [25 nil] INFO [2022-01-10 05:45:08,608] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:08,608] jepsen worker nemesis - couchbase.util Rebalance check count: 459 INFO [2022-01-10 05:45:08,608] jepsen worker nemesis - couchbase.util Rebalance stuck count: 372 INFO [2022-01-10 05:45:08,874] jepsen worker 39 - jepsen.util 3819 :info :write [13 2] :SyncWriteAmbiguous INFO [2022-01-10 05:45:08,875] jepsen worker 39 - jepsen.util 3909 :invoke :write [13 17] INFO [2022-01-10 05:45:09,010] jepsen worker 23 - jepsen.util 3353 :info :write [7 48] :RequestTimeoutException INFO [2022-01-10 05:45:09,010] jepsen worker 23 - jepsen.util 3443 :invoke :read [7 nil] INFO [2022-01-10 05:45:09,079] jepsen worker 67 - jepsen.util 3217 :info :write [22 49] :RequestTimeoutException INFO [2022-01-10 05:45:09,080] jepsen worker 67 - jepsen.util 3307 :invoke :read [22 nil] INFO [2022-01-10 05:45:09,116] jepsen worker 24 - jepsen.util 2904 :info :write [8 41] :RequestTimeoutException INFO [2022-01-10 05:45:09,116] jepsen worker 24 - jepsen.util 2994 :invoke :read [8 nil] INFO [2022-01-10 05:45:09,251] jepsen worker 40 - jepsen.util 2650 :info :write [13 49] :RequestTimeoutException INFO [2022-01-10 05:45:09,251] jepsen worker 40 - jepsen.util 2740 :invoke :read [13 nil] INFO [2022-01-10 05:45:09,291] jepsen worker 15 - jepsen.util 2805 :info :write [5 30] :RequestTimeoutException INFO [2022-01-10 05:45:09,291] jepsen worker 15 - jepsen.util 2895 :invoke :write [5 23] INFO [2022-01-10 05:45:09,340] jepsen worker 19 - jepsen.util 3349 :info :write [6 0] :RequestTimeoutException INFO [2022-01-10 05:45:09,341] jepsen worker 19 - jepsen.util 3439 :invoke :write [6 44] INFO [2022-01-10 05:45:09,342] jepsen worker 54 - jepsen.util 3474 :ok :read [18 14] INFO [2022-01-10 05:45:09,343] jepsen worker 54 - jepsen.util 3474 :invoke :write [18 45] INFO [2022-01-10 05:45:09,615] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:09,615] jepsen worker nemesis - couchbase.util Rebalance check count: 460 INFO [2022-01-10 05:45:09,615] jepsen worker nemesis - couchbase.util Rebalance stuck count: 373 INFO [2022-01-10 05:45:09,802] jepsen worker 0 - jepsen.util 3150 :ok :read [0 46] INFO [2022-01-10 05:45:09,803] jepsen worker 0 - jepsen.util 3150 :invoke :write [0 37] INFO [2022-01-10 05:45:10,071] jepsen worker 17 - jepsen.util 3167 :info :write [5 23] :RequestTimeoutException INFO [2022-01-10 05:45:10,072] jepsen worker 17 - jepsen.util 3257 :invoke :write [5 34] INFO [2022-01-10 05:45:10,111] jepsen worker 61 - jepsen.util 3391 :info :write [20 23] :RequestTimeoutException INFO [2022-01-10 05:45:10,112] jepsen worker 61 - jepsen.util 3481 :invoke :write [20 8] INFO [2022-01-10 05:45:10,225] jepsen worker 66 - jepsen.util 3036 :info :write [22 23] :RequestTimeoutException INFO [2022-01-10 05:45:10,226] jepsen worker 66 - jepsen.util 3126 :invoke :read [22 nil] INFO [2022-01-10 05:45:10,246] jepsen worker 86 - jepsen.util 2696 :info :write [28 9] :RequestTimeoutException INFO [2022-01-10 05:45:10,247] jepsen worker 86 - jepsen.util 2786 :invoke :read [28 nil] INFO [2022-01-10 05:45:10,291] jepsen worker 29 - jepsen.util 3179 :info :write [9 38] :RequestTimeoutException INFO [2022-01-10 05:45:10,292] jepsen worker 29 - jepsen.util 3269 :invoke :write [9 37] WARN [2022-01-10 05:45:10,298] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":26404595,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":26404595,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9024634,"encode_duration_us":16755,"operation_id":"0x638f","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10005878}],"total_count":1}} INFO [2022-01-10 05:45:10,316] jepsen worker 83 - jepsen.util 3773 :info :write [27 23] :RequestTimeoutException INFO [2022-01-10 05:45:10,316] jepsen worker 83 - jepsen.util 3863 :invoke :read [27 nil] WARN [2022-01-10 05:45:10,399] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18158399,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":18158399,"total_server_duration_us":135,"operation_id":"0x6529","timeout_ms":30000,"last_server_duration_us":135,"total_duration_us":18159333},{"operation_name":"get","last_dispatch_duration_us":18141499,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":18141499,"total_server_duration_us":19,"operation_id":"0x6530","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":18142740},{"operation_name":"get","last_dispatch_duration_us":18139661,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":18139661,"total_server_duration_us":15,"operation_id":"0x6537","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":18141488},{"operation_name":"upsert","last_dispatch_duration_us":574613,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5146733,"total_server_duration_us":772847,"encode_duration_us":16905,"operation_id":"0x6754","timeout_ms":10000,"last_server_duration_us":86328,"total_duration_us":10010729},{"operation_name":"upsert","last_dispatch_duration_us":159630,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3372278,"total_server_duration_us":1403542,"encode_duration_us":22731,"operation_id":"0x6826","timeout_ms":10000,"last_server_duration_us":116281,"total_duration_us":10010405},{"operation_name":"upsert","last_dispatch_duration_us":647773,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1721217,"total_server_duration_us":507375,"encode_duration_us":17809,"operation_id":"0x669b","timeout_ms":10000,"last_server_duration_us":360471,"total_duration_us":10009333},{"operation_name":"upsert","last_dispatch_duration_us":9813798,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13042824,"total_server_duration_us":933171,"encode_duration_us":17991,"operation_id":"0x67ed","timeout_ms":10000,"last_server_duration_us":582254,"total_duration_us":10007766},{"operation_name":"upsert","last_dispatch_duration_us":410397,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2882911,"total_server_duration_us":846111,"encode_duration_us":16182,"operation_id":"0x6722","timeout_ms":10000,"last_server_duration_us":113298,"total_duration_us":10007710},{"operation_name":"upsert","last_dispatch_duration_us":253844,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3479892,"total_server_duration_us":526617,"encode_duration_us":18842,"operation_id":"0x66ca","timeout_ms":10000,"last_server_duration_us":53734,"total_duration_us":10005859},{"operation_name":"upsert","last_dispatch_duration_us":9319461,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13375833,"total_server_duration_us":1077098,"encode_duration_us":22184,"operation_id":"0x67b8","timeout_ms":10000,"last_server_duration_us":712167,"total_duration_us":10004140}],"total_count":20}} INFO [2022-01-10 05:45:10,411] jepsen worker 50 - jepsen.util 3380 :info :write [16 5] :RequestTimeoutException INFO [2022-01-10 05:45:10,411] jepsen worker 50 - jepsen.util 3470 :invoke :read [16 nil] INFO [2022-01-10 05:45:10,435] jepsen worker 84 - jepsen.util 3324 :info :write [28 43] :RequestTimeoutException INFO [2022-01-10 05:45:10,436] jepsen worker 84 - jepsen.util 3414 :invoke :read [28 nil] INFO [2022-01-10 05:45:10,526] jepsen worker 71 - jepsen.util 2771 :info :write [23 45] :RequestTimeoutException INFO [2022-01-10 05:45:10,526] jepsen worker 71 - jepsen.util 2861 :invoke :write [23 3] INFO [2022-01-10 05:45:10,546] jepsen worker 41 - jepsen.util 3101 :info :write [13 5] :RequestTimeoutException INFO [2022-01-10 05:45:10,547] jepsen worker 41 - jepsen.util 3191 :invoke :read [13 nil] INFO [2022-01-10 05:45:10,549] jepsen worker 68 - jepsen.util 3308 :info :write [22 21] :RequestTimeoutException INFO [2022-01-10 05:45:10,550] jepsen worker 68 - jepsen.util 3398 :invoke :write [22 2] INFO [2022-01-10 05:45:10,620] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:10,620] jepsen worker nemesis - couchbase.util Rebalance check count: 461 INFO [2022-01-10 05:45:10,620] jepsen worker nemesis - couchbase.util Rebalance stuck count: 374 INFO [2022-01-10 05:45:10,631] jepsen worker 4 - jepsen.util 3784 :info :write [1 36] :RequestTimeoutException INFO [2022-01-10 05:45:10,632] jepsen worker 4 - jepsen.util 3874 :invoke :write [1 7] INFO [2022-01-10 05:45:10,661] jepsen worker 65 - jepsen.util 3125 :ok :read [21 7] INFO [2022-01-10 05:45:10,661] jepsen worker 70 - jepsen.util 3310 :info :write [23 17] :SyncWriteAmbiguous INFO [2022-01-10 05:45:10,664] jepsen worker 70 - jepsen.util 3400 :invoke :write [23 12] INFO [2022-01-10 05:45:10,666] jepsen worker 65 - jepsen.util 3125 :invoke :read [21 nil] INFO [2022-01-10 05:45:10,674] jepsen worker 64 - jepsen.util 3304 :ok :read [21 7] INFO [2022-01-10 05:45:10,674] jepsen worker 64 - jepsen.util 3304 :invoke :read [21 nil] INFO [2022-01-10 05:45:10,707] jepsen worker 34 - jepsen.util 3274 :ok :read [11 21] INFO [2022-01-10 05:45:10,707] jepsen worker 34 - jepsen.util 3274 :invoke :write [11 27] INFO [2022-01-10 05:45:10,711] jepsen worker 87 - jepsen.util 3417 :info :write [29 19] :RequestTimeoutException INFO [2022-01-10 05:45:10,712] jepsen worker 87 - jepsen.util 3507 :invoke :read [29 nil] INFO [2022-01-10 05:45:10,781] jepsen worker 47 - jepsen.util 3377 :ok :read [15 23] INFO [2022-01-10 05:45:10,781] jepsen worker 47 - jepsen.util 3377 :invoke :write [15 37] INFO [2022-01-10 05:45:10,827] jepsen worker 65 - jepsen.util 3125 :ok :read [21 7] INFO [2022-01-10 05:45:10,828] jepsen worker 65 - jepsen.util 3125 :invoke :read [21 nil] INFO [2022-01-10 05:45:10,836] jepsen worker 28 - jepsen.util 3088 :ok :read [9 27] INFO [2022-01-10 05:45:10,837] jepsen worker 28 - jepsen.util 3088 :invoke :write [9 17] INFO [2022-01-10 05:45:10,858] jepsen worker 64 - jepsen.util 3304 :ok :read [21 7] INFO [2022-01-10 05:45:10,859] jepsen worker 64 - jepsen.util 3304 :invoke :read [21 nil] INFO [2022-01-10 05:45:10,992] jepsen worker 46 - jepsen.util 3376 :info :write [15 5] :RequestTimeoutException INFO [2022-01-10 05:45:10,993] jepsen worker 46 - jepsen.util 3466 :invoke :write [15 43] INFO [2022-01-10 05:45:11,027] jepsen worker 65 - jepsen.util 3125 :ok :read [21 7] INFO [2022-01-10 05:45:11,027] jepsen worker 65 - jepsen.util 3125 :invoke :read [21 nil] INFO [2022-01-10 05:45:11,065] jepsen worker 64 - jepsen.util 3304 :ok :read [21 7] INFO [2022-01-10 05:45:11,065] jepsen worker 64 - jepsen.util 3304 :invoke :read [21 nil] INFO [2022-01-10 05:45:11,109] jepsen worker 89 - jepsen.util 3779 :ok :read [29 0] INFO [2022-01-10 05:45:11,109] jepsen worker 89 - jepsen.util 3779 :invoke :read [29 nil] INFO [2022-01-10 05:45:11,118] jepsen worker 38 - jepsen.util 3548 :ok :read [12 29] INFO [2022-01-10 05:45:11,119] jepsen worker 38 - jepsen.util 3548 :invoke :read [12 nil] INFO [2022-01-10 05:45:11,192] jepsen worker 65 - jepsen.util 3125 :ok :read [21 7] INFO [2022-01-10 05:45:11,192] jepsen worker 65 - jepsen.util 3125 :invoke :write [21 8] INFO [2022-01-10 05:45:11,228] jepsen worker 48 - jepsen.util 2928 :ok :read [16 38] INFO [2022-01-10 05:45:11,229] jepsen worker 48 - jepsen.util 2928 :invoke :write [16 0] INFO [2022-01-10 05:45:11,229] jepsen worker 64 - jepsen.util 3304 :ok :read [21 7] INFO [2022-01-10 05:45:11,230] jepsen worker 64 - jepsen.util 3304 :invoke :read [21 nil] INFO [2022-01-10 05:45:11,266] jepsen worker 77 - jepsen.util 3587 :info :write [25 1] :RequestTimeoutException INFO [2022-01-10 05:45:11,266] jepsen worker 77 - jepsen.util 3677 :invoke :write [25 10] INFO [2022-01-10 05:45:11,282] jepsen worker 35 - jepsen.util 2915 :info :write [11 1] :RequestTimeoutException INFO [2022-01-10 05:45:11,283] jepsen worker 35 - jepsen.util 3005 :invoke :write [11 43] INFO [2022-01-10 05:45:11,364] jepsen worker 42 - jepsen.util 3282 :info :write [14 41] :RequestTimeoutException INFO [2022-01-10 05:45:11,365] jepsen worker 42 - jepsen.util 3372 :invoke :read [14 nil] INFO [2022-01-10 05:45:11,418] jepsen worker 64 - jepsen.util 3304 :ok :read [21 7] INFO [2022-01-10 05:45:11,418] jepsen worker 64 - jepsen.util 3304 :invoke :read [21 nil] INFO [2022-01-10 05:45:11,501] jepsen worker 30 - jepsen.util 3090 :info :write [10 44] :RequestTimeoutException INFO [2022-01-10 05:45:11,502] jepsen worker 30 - jepsen.util 3180 :invoke :read [10 nil] INFO [2022-01-10 05:45:11,552] jepsen worker 20 - jepsen.util 2990 :info :write [6 19] :RequestTimeoutException INFO [2022-01-10 05:45:11,552] jepsen worker 20 - jepsen.util 3080 :invoke :read [6 nil] INFO [2022-01-10 05:45:11,579] jepsen worker 64 - jepsen.util 3304 :ok :read [21 7] INFO [2022-01-10 05:45:11,580] jepsen worker 64 - jepsen.util 3304 :invoke :read [21 nil] INFO [2022-01-10 05:45:11,590] jepsen worker 6 - jepsen.util 3426 :ok :read [2 14] INFO [2022-01-10 05:45:11,590] jepsen worker 6 - jepsen.util 3426 :invoke :write [2 6] INFO [2022-01-10 05:45:11,636] jepsen worker 82 - jepsen.util 3502 :ok :read [27 30] INFO [2022-01-10 05:45:11,636] jepsen worker 82 - jepsen.util 3502 :invoke :write [27 24] INFO [2022-01-10 05:45:11,642] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:11,642] jepsen worker nemesis - couchbase.util Rebalance check count: 462 INFO [2022-01-10 05:45:11,642] jepsen worker nemesis - couchbase.util Rebalance stuck count: 375 INFO [2022-01-10 05:45:11,736] jepsen worker 44 - jepsen.util 3374 :info :write [14 15] :RequestTimeoutException INFO [2022-01-10 05:45:11,737] jepsen worker 44 - jepsen.util 3464 :invoke :read [14 nil] INFO [2022-01-10 05:45:11,787] jepsen worker 64 - jepsen.util 3304 :ok :read [21 7] INFO [2022-01-10 05:45:11,788] jepsen worker 64 - jepsen.util 3304 :invoke :write [21 28] INFO [2022-01-10 05:45:11,816] jepsen worker 21 - jepsen.util 3531 :ok :read [7 :nil] INFO [2022-01-10 05:45:11,817] jepsen worker 21 - jepsen.util 3531 :invoke :write [7 26] INFO [2022-01-10 05:45:11,839] jepsen worker 26 - jepsen.util 2906 :info :write [8 43] :RequestTimeoutException INFO [2022-01-10 05:45:11,840] jepsen worker 26 - jepsen.util 2996 :invoke :read [8 nil] INFO [2022-01-10 05:45:11,841] jepsen worker 50 - jepsen.util 3470 :ok :read [16 38] INFO [2022-01-10 05:45:11,841] jepsen worker 50 - jepsen.util 3470 :invoke :read [16 nil] INFO [2022-01-10 05:45:11,865] jepsen worker 25 - jepsen.util 3175 :ok :read [8 27] INFO [2022-01-10 05:45:11,865] jepsen worker 25 - jepsen.util 3175 :invoke :write [8 1] WARN [2022-01-10 05:45:11,900] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17255816,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18176741,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9345967,"encode_duration_us":23251,"operation_id":"0x65fc","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10006392}],"total_count":1}} INFO [2022-01-10 05:45:11,905] jepsen worker 10 - jepsen.util 3340 :info :write [3 44] :RequestTimeoutException INFO [2022-01-10 05:45:11,906] jepsen worker 10 - jepsen.util 3430 :invoke :write [3 19] INFO [2022-01-10 05:45:11,975] jepsen worker 37 - jepsen.util 3097 :info :write [12 2] :RequestTimeoutException INFO [2022-01-10 05:45:11,976] jepsen worker 37 - jepsen.util 3187 :invoke :read [12 nil] INFO [2022-01-10 05:45:11,994] jepsen worker 36 - jepsen.util 2556 :info :write [12 19] :RequestTimeoutException INFO [2022-01-10 05:45:11,995] jepsen worker 36 - jepsen.util 2646 :invoke :read [12 nil] WARN [2022-01-10 05:45:11,998] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9734842,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12658453,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9579859,"encode_duration_us":18188,"operation_id":"0x65b3","timeout_ms":10000,"last_server_duration_us":9318333,"total_duration_us":10005147}],"total_count":1}} WARN [2022-01-10 05:45:11,999] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":111813,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9353212,"total_server_duration_us":78944,"encode_duration_us":19001,"operation_id":"0x68cc","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10009284},{"operation_name":"upsert","last_dispatch_duration_us":630100,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10296150,"total_server_duration_us":762187,"encode_duration_us":19301,"operation_id":"0x6770","timeout_ms":10000,"last_server_duration_us":27583,"total_duration_us":10008822},{"operation_name":"upsert","last_dispatch_duration_us":236276,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9093700,"total_server_duration_us":1608010,"encode_duration_us":17101,"operation_id":"0x67e6","timeout_ms":10000,"last_server_duration_us":137901,"total_duration_us":10008066},{"operation_name":"upsert","last_dispatch_duration_us":852621,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":6940619,"total_server_duration_us":1278011,"encode_duration_us":16294,"operation_id":"0x6853","timeout_ms":10000,"last_server_duration_us":136649,"total_duration_us":10007797},{"operation_name":"upsert","last_dispatch_duration_us":203794,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":4616678,"total_server_duration_us":911365,"encode_duration_us":27713,"operation_id":"0x688d","timeout_ms":10000,"last_server_duration_us":43398,"total_duration_us":10005669},{"operation_name":"upsert","last_dispatch_duration_us":1374083,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7063017,"total_server_duration_us":2212906,"encode_duration_us":16469,"operation_id":"0x687d","timeout_ms":10000,"last_server_duration_us":366425,"total_duration_us":10004617},{"operation_name":"upsert","last_dispatch_duration_us":885238,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":6766945,"total_server_duration_us":1089858,"encode_duration_us":21272,"operation_id":"0x688f","timeout_ms":10000,"last_server_duration_us":201269,"total_duration_us":10004008},{"operation_name":"upsert","last_dispatch_duration_us":8915522,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9897154,"total_server_duration_us":172638,"encode_duration_us":18893,"operation_id":"0x68a1","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10003782},{"operation_name":"upsert","last_dispatch_duration_us":8858882,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":16790656,"total_server_duration_us":234936,"encode_duration_us":16378,"operation_id":"0x672e","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10003499},{"operation_name":"upsert","last_dispatch_duration_us":8702037,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":14903436,"total_server_duration_us":215509,"encode_duration_us":17534,"operation_id":"0x6799","timeout_ms":10000,"last_server_duration_us":22059,"total_duration_us":10002262}],"total_count":15}} INFO [2022-01-10 05:45:12,011] jepsen worker 45 - jepsen.util 3555 :info :write [15 10] :RequestTimeoutException INFO [2022-01-10 05:45:12,012] jepsen worker 45 - jepsen.util 3645 :invoke :write [15 42] WARN [2022-01-10 05:45:12,045] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17921742,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":25918800,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9059284,"encode_duration_us":16351,"operation_id":"0x6520","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10001121},{"operation_name":"upsert","last_dispatch_duration_us":9619863,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10973147,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9213962,"encode_duration_us":20660,"operation_id":"0x6699","timeout_ms":10000,"last_server_duration_us":9187544,"total_duration_us":10008034}],"total_count":2}} INFO [2022-01-10 05:45:12,054] jepsen worker 22 - jepsen.util 3082 :ok :read [7 :nil] INFO [2022-01-10 05:45:12,054] jepsen worker 69 - jepsen.util 2859 :info :write [23 4] :RequestTimeoutException INFO [2022-01-10 05:45:12,055] jepsen worker 69 - jepsen.util 2949 :invoke :read [23 nil] INFO [2022-01-10 05:45:12,056] jepsen worker 31 - jepsen.util 3721 :info :write [10 16] :RequestTimeoutException INFO [2022-01-10 05:45:12,056] jepsen worker 22 - jepsen.util 3082 :invoke :write [7 12] INFO [2022-01-10 05:45:12,057] jepsen worker 31 - jepsen.util 3811 :invoke :read [10 nil] WARN [2022-01-10 05:45:12,099] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13756226,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":13756226,"total_server_duration_us":23,"operation_id":"0x67f5","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":13757468},{"operation_name":"upsert","last_dispatch_duration_us":9764146,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":18539380,"total_server_duration_us":66530,"encode_duration_us":18146,"operation_id":"0x66f0","timeout_ms":10000,"last_server_duration_us":22141,"total_duration_us":10008595},{"operation_name":"upsert","last_dispatch_duration_us":120791,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9949587,"total_server_duration_us":95704,"encode_duration_us":19558,"operation_id":"0x68c9","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10008505},{"operation_name":"upsert","last_dispatch_duration_us":165692,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9927623,"total_server_duration_us":124563,"encode_duration_us":17230,"operation_id":"0x68ce","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10006450},{"operation_name":"upsert","last_dispatch_duration_us":1320299,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":1320299,"total_server_duration_us":80255,"encode_duration_us":18165,"operation_id":"0x67c3","timeout_ms":10000,"last_server_duration_us":80255,"total_duration_us":10006221},{"operation_name":"upsert","last_dispatch_duration_us":191781,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10051980,"total_server_duration_us":78227,"encode_duration_us":16567,"operation_id":"0x68bc","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10005862},{"operation_name":"upsert","last_dispatch_duration_us":427961,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":4493265,"total_server_duration_us":4081829,"encode_duration_us":19261,"operation_id":"0x66fe","timeout_ms":10000,"last_server_duration_us":416106,"total_duration_us":10005580},{"operation_name":"upsert","last_dispatch_duration_us":10202851,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10202851,"total_server_duration_us":9086245,"encode_duration_us":21148,"operation_id":"0x687f","timeout_ms":10000,"last_server_duration_us":9086245,"total_duration_us":10003370},{"operation_name":"upsert","last_dispatch_duration_us":9570184,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10940965,"total_server_duration_us":542838,"encode_duration_us":17278,"operation_id":"0x681e","timeout_ms":10000,"last_server_duration_us":202322,"total_duration_us":10003012},{"operation_name":"upsert","last_dispatch_duration_us":10035649,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":13396910,"total_server_duration_us":11466236,"encode_duration_us":16234,"operation_id":"0x672c","timeout_ms":10000,"last_server_duration_us":9826220,"total_duration_us":10002823}],"total_count":19}} WARN [2022-01-10 05:45:12,099] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1091454,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":6653935,"total_server_duration_us":2582749,"encode_duration_us":16405,"operation_id":"0x675d","timeout_ms":10000,"last_server_duration_us":803871,"total_duration_us":10008458},{"operation_name":"upsert","last_dispatch_duration_us":430683,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":6184095,"total_server_duration_us":2126398,"encode_duration_us":17723,"operation_id":"0x6701","timeout_ms":10000,"last_server_duration_us":71231,"total_duration_us":10008146},{"operation_name":"upsert","last_dispatch_duration_us":605373,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":6519699,"total_server_duration_us":1472236,"encode_duration_us":17252,"operation_id":"0x6768","timeout_ms":10000,"last_server_duration_us":76312,"total_duration_us":10007462},{"operation_name":"upsert","last_dispatch_duration_us":46353,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":326197,"total_server_duration_us":174194,"encode_duration_us":20674,"operation_id":"0x67c4","timeout_ms":10000,"last_server_duration_us":27945,"total_duration_us":10007385},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":31101,"operation_id":"0x68c8","timeout_ms":10000,"total_duration_us":10006828},{"operation_name":"upsert","last_dispatch_duration_us":7424783,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7424783,"total_server_duration_us":191687,"encode_duration_us":16378,"operation_id":"0x67cd","timeout_ms":10000,"last_server_duration_us":191687,"total_duration_us":10005872},{"operation_name":"upsert","last_dispatch_duration_us":9294341,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":13580526,"total_server_duration_us":1462989,"encode_duration_us":18957,"operation_id":"0x6700","timeout_ms":10000,"last_server_duration_us":104865,"total_duration_us":10004604},{"operation_name":"upsert","last_dispatch_duration_us":7653485,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":7739821,"total_server_duration_us":59762,"encode_duration_us":20136,"operation_id":"0x6767","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10002504},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":22280,"operation_id":"0x681f","timeout_ms":10000,"total_duration_us":10001944},{"operation_name":"upsert","last_dispatch_duration_us":9345870,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9345870,"total_server_duration_us":9154436,"encode_duration_us":17967,"operation_id":"0x67cc","timeout_ms":10000,"last_server_duration_us":9154436,"total_duration_us":9350435}],"total_count":12}} WARN [2022-01-10 05:45:12,100] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":8741402,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8741402,"total_server_duration_us":108,"operation_id":"0x67ee","timeout_ms":30000,"last_server_duration_us":108,"total_duration_us":12381174},{"operation_name":"upsert","last_dispatch_duration_us":9730167,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":13637712,"total_server_duration_us":10804983,"encode_duration_us":16631,"operation_id":"0x67bb","timeout_ms":10000,"last_server_duration_us":9050106,"total_duration_us":10009426},{"operation_name":"upsert","last_dispatch_duration_us":449674,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":1431083,"total_server_duration_us":503232,"encode_duration_us":17215,"operation_id":"0x688e","timeout_ms":10000,"last_server_duration_us":32840,"total_duration_us":10008640},{"operation_name":"upsert","last_dispatch_duration_us":6834859,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10115100,"total_server_duration_us":388184,"encode_duration_us":20075,"operation_id":"0x68dd","timeout_ms":10000,"last_server_duration_us":36525,"total_duration_us":10007429},{"operation_name":"upsert","last_dispatch_duration_us":333828,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":1026598,"total_server_duration_us":109732,"encode_duration_us":19776,"operation_id":"0x6890","timeout_ms":10000,"last_server_duration_us":62540,"total_duration_us":10006981},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","encode_duration_us":16518,"operation_id":"0x6723","timeout_ms":10000,"total_duration_us":10006098},{"operation_name":"upsert","last_dispatch_duration_us":289300,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":4873806,"total_server_duration_us":2806308,"encode_duration_us":19945,"operation_id":"0x67ca","timeout_ms":10000,"last_server_duration_us":53254,"total_duration_us":10005782},{"operation_name":"upsert","last_dispatch_duration_us":8791344,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8791344,"total_server_duration_us":21082,"encode_duration_us":20200,"operation_id":"0x6893","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10005712},{"operation_name":"upsert","last_dispatch_duration_us":765228,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":1903608,"total_server_duration_us":553862,"encode_duration_us":21653,"operation_id":"0x6888","timeout_ms":10000,"last_server_duration_us":392886,"total_duration_us":10005661},{"operation_name":"upsert","last_dispatch_duration_us":1171945,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4344157,"total_server_duration_us":1233005,"encode_duration_us":19182,"operation_id":"0x6760","timeout_ms":10000,"last_server_duration_us":109534,"total_duration_us":10003832}],"total_count":17}} INFO [2022-01-10 05:45:12,157] jepsen worker 20 - jepsen.util 3080 :ok :read [6 25] INFO [2022-01-10 05:45:12,157] jepsen worker 20 - jepsen.util 3080 :invoke :write [6 48] WARN [2022-01-10 05:45:12,199] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":15696772,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16622260,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9570834,"encode_duration_us":17616,"operation_id":"0x63d2","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10002194},{"operation_name":"upsert","last_dispatch_duration_us":10035649,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13396910,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":11466236,"encode_duration_us":16234,"operation_id":"0x672c","timeout_ms":10000,"last_server_duration_us":9826220,"total_duration_us":10002823},{"operation_name":"upsert","last_dispatch_duration_us":10202851,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10202851,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9086245,"encode_duration_us":21148,"operation_id":"0x687f","timeout_ms":10000,"last_server_duration_us":9086245,"total_duration_us":10003370},{"operation_name":"upsert","last_dispatch_duration_us":17620654,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":24801157,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9303752,"encode_duration_us":19529,"operation_id":"0x655e","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10006008}],"total_count":4}} WARN [2022-01-10 05:45:12,199] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9518361,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12201112,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9612816,"encode_duration_us":20652,"operation_id":"0x66c4","timeout_ms":10000,"last_server_duration_us":9176858,"total_duration_us":10003178},{"operation_name":"upsert","last_dispatch_duration_us":17516286,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18048812,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9137877,"encode_duration_us":18217,"operation_id":"0x65bf","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10006036},{"operation_name":"upsert","last_dispatch_duration_us":17536655,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18033899,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9125318,"encode_duration_us":18396,"operation_id":"0x65c2","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10008155}],"total_count":3}} INFO [2022-01-10 05:45:12,213] jepsen worker 26 - jepsen.util 2996 :ok :read [8 27] INFO [2022-01-10 05:45:12,214] jepsen worker 26 - jepsen.util 2996 :invoke :write [8 26] WARN [2022-01-10 05:45:12,246] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8993409,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11660464,"total_server_duration_us":660860,"encode_duration_us":27050,"operation_id":"0x676f","timeout_ms":10000,"last_server_duration_us":419552,"total_duration_us":10011116},{"operation_name":"upsert","last_dispatch_duration_us":173375,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4939268,"total_server_duration_us":892061,"encode_duration_us":16464,"operation_id":"0x6856","timeout_ms":10000,"last_server_duration_us":76730,"total_duration_us":10009035},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":19374,"operation_id":"0x67f6","timeout_ms":10000,"total_duration_us":10005594},{"operation_name":"upsert","last_dispatch_duration_us":1162211,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":5992458,"total_server_duration_us":3411354,"encode_duration_us":16294,"operation_id":"0x672b","timeout_ms":10000,"last_server_duration_us":812233,"total_duration_us":10005379},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":17753,"operation_id":"0x6808","timeout_ms":10000,"total_duration_us":10005108},{"operation_name":"upsert","last_dispatch_duration_us":7963069,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8614402,"total_server_duration_us":205028,"encode_duration_us":17526,"operation_id":"0x67c9","timeout_ms":10000,"last_server_duration_us":102169,"total_duration_us":10005026},{"operation_name":"upsert","last_dispatch_duration_us":521145,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2566633,"total_server_duration_us":366279,"encode_duration_us":17074,"operation_id":"0x68f0","timeout_ms":10000,"last_server_duration_us":108399,"total_duration_us":10004710},{"operation_name":"upsert","last_dispatch_duration_us":214590,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5148374,"total_server_duration_us":1434632,"encode_duration_us":18616,"operation_id":"0x6848","timeout_ms":10000,"last_server_duration_us":71909,"total_duration_us":10003703},{"operation_name":"upsert","last_dispatch_duration_us":8105251,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9485421,"total_server_duration_us":281113,"encode_duration_us":21663,"operation_id":"0x66fc","timeout_ms":10000,"last_server_duration_us":184733,"total_duration_us":10002464},{"operation_name":"upsert","last_dispatch_duration_us":3496864,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":3496864,"total_server_duration_us":401315,"encode_duration_us":16695,"operation_id":"0x6849","timeout_ms":10000,"last_server_duration_us":401315,"total_duration_us":10001657}],"total_count":15}} INFO [2022-01-10 05:45:12,590] jepsen worker 49 - jepsen.util 3199 :info :write [16 0] :RequestTimeoutException INFO [2022-01-10 05:45:12,590] jepsen worker 49 - jepsen.util 3289 :invoke :read [16 nil] INFO [2022-01-10 05:45:12,591] jepsen worker 78 - jepsen.util 3948 :info :write [26 47] :RequestTimeoutException INFO [2022-01-10 05:45:12,601] jepsen worker 80 - jepsen.util 4040 :info :write [26 41] :RequestTimeoutException INFO [2022-01-10 05:45:12,604] jepsen worker 80 - jepsen.util 4130 :invoke :read [26 nil] INFO [2022-01-10 05:45:12,604] jepsen worker 78 - jepsen.util 4038 :invoke :write [26 49] INFO [2022-01-10 05:45:12,647] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:12,647] jepsen worker nemesis - couchbase.util Rebalance check count: 463 INFO [2022-01-10 05:45:12,648] jepsen worker nemesis - couchbase.util Rebalance stuck count: 376 INFO [2022-01-10 05:45:12,783] jepsen worker 80 - jepsen.util 4130 :ok :read [26 10] INFO [2022-01-10 05:45:12,784] jepsen worker 80 - jepsen.util 4130 :invoke :write [26 13] INFO [2022-01-10 05:45:12,926] jepsen worker 63 - jepsen.util 3303 :info :write [21 33] :RequestTimeoutException INFO [2022-01-10 05:45:12,927] jepsen worker 63 - jepsen.util 3393 :invoke :read [21 nil] INFO [2022-01-10 05:45:12,961] jepsen worker 55 - jepsen.util 3655 :info :write [18 1] :RequestTimeoutException INFO [2022-01-10 05:45:12,961] jepsen worker 55 - jepsen.util 3745 :invoke :read [18 nil] INFO [2022-01-10 05:45:13,011] jepsen worker 73 - jepsen.util 3043 :info :write [24 27] :RequestTimeoutException INFO [2022-01-10 05:45:13,012] jepsen worker 73 - jepsen.util 3133 :invoke :read [24 nil] INFO [2022-01-10 05:45:13,125] jepsen worker 79 - jepsen.util 3859 :info :write [26 15] :RequestTimeoutException INFO [2022-01-10 05:45:13,126] jepsen worker 79 - jepsen.util 3949 :invoke :read [26 nil] INFO [2022-01-10 05:45:13,238] jepsen worker 40 - jepsen.util 2740 :fail :read [13 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:45:13,238] jepsen worker 86 - jepsen.util 2786 :fail :read [28 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:45:13,238] jepsen worker 41 - jepsen.util 3191 :fail :read [13 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:45:13,238] jepsen worker 86 - jepsen.util 2786 :invoke :read [28 nil] INFO [2022-01-10 05:45:13,239] jepsen worker 40 - jepsen.util 2740 :invoke :read [13 nil] INFO [2022-01-10 05:45:13,239] jepsen worker 41 - jepsen.util 3191 :invoke :write [13 23] INFO [2022-01-10 05:45:13,253] jepsen worker 16 - jepsen.util 3436 :ok :read [5 42] INFO [2022-01-10 05:45:13,254] jepsen worker 16 - jepsen.util 3436 :invoke :read [5 nil] INFO [2022-01-10 05:45:13,346] jepsen worker 79 - jepsen.util 3949 :ok :read [26 10] INFO [2022-01-10 05:45:13,346] jepsen worker 79 - jepsen.util 3949 :invoke :read [26 nil] INFO [2022-01-10 05:45:13,542] jepsen worker 5 - jepsen.util 3245 :ok :read [1 10] INFO [2022-01-10 05:45:13,543] jepsen worker 5 - jepsen.util 3245 :invoke :write [1 18] INFO [2022-01-10 05:45:13,557] jepsen worker 9 - jepsen.util 3879 :ok :read [3 31] INFO [2022-01-10 05:45:13,558] jepsen worker 9 - jepsen.util 3879 :invoke :write [3 25] INFO [2022-01-10 05:45:13,561] jepsen worker 79 - jepsen.util 3949 :ok :read [26 10] INFO [2022-01-10 05:45:13,562] jepsen worker 79 - jepsen.util 3949 :invoke :write [26 39] INFO [2022-01-10 05:45:13,580] jepsen worker 32 - jepsen.util 3632 :info :write [10 31] :RequestTimeoutException INFO [2022-01-10 05:45:13,580] jepsen worker 32 - jepsen.util 3722 :invoke :read [10 nil] INFO [2022-01-10 05:45:13,594] jepsen worker 43 - jepsen.util 3193 :info :write [14 14] :RequestTimeoutException INFO [2022-01-10 05:45:13,595] jepsen worker 43 - jepsen.util 3283 :invoke :write [14 6] INFO [2022-01-10 05:45:13,657] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:13,657] jepsen worker nemesis - couchbase.util Rebalance check count: 464 INFO [2022-01-10 05:45:13,657] jepsen worker nemesis - couchbase.util Rebalance stuck count: 377 INFO [2022-01-10 05:45:13,865] jepsen worker 40 - jepsen.util 2740 :fail :read [13 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:45:13,868] jepsen worker 40 - jepsen.util 2740 :invoke :read [13 nil] INFO [2022-01-10 05:45:13,887] jepsen worker 50 - jepsen.util 3470 :ok :read [16 38] INFO [2022-01-10 05:45:13,888] jepsen worker 50 - jepsen.util 3470 :invoke :write [16 19] INFO [2022-01-10 05:45:13,897] jepsen worker 16 - jepsen.util 3436 :ok :read [5 42] INFO [2022-01-10 05:45:13,898] jepsen worker 16 - jepsen.util 3436 :invoke :read [5 nil] INFO [2022-01-10 05:45:13,907] jepsen worker 86 - jepsen.util 2786 :ok :read [28 45] INFO [2022-01-10 05:45:13,908] jepsen worker 86 - jepsen.util 2786 :invoke :write [28 27] INFO [2022-01-10 05:45:13,932] jepsen worker 74 - jepsen.util 3854 :info :write [24 20] :RequestTimeoutException INFO [2022-01-10 05:45:13,932] jepsen worker 74 - jepsen.util 3944 :invoke :read [24 nil] INFO [2022-01-10 05:45:14,006] jepsen worker 75 - jepsen.util 2865 :info :write [25 45] :RequestTimeoutException INFO [2022-01-10 05:45:14,007] jepsen worker 75 - jepsen.util 2955 :invoke :write [25 43] INFO [2022-01-10 05:45:14,154] jepsen worker 53 - jepsen.util 4013 :info :write [17 27] :RequestTimeoutException INFO [2022-01-10 05:45:14,156] jepsen worker 53 - jepsen.util 4103 :invoke :write [17 25] INFO [2022-01-10 05:45:14,291] jepsen worker 18 - jepsen.util 3258 :info :write [6 30] :RequestTimeoutException INFO [2022-01-10 05:45:14,292] jepsen worker 18 - jepsen.util 3348 :invoke :read [6 nil] INFO [2022-01-10 05:45:14,304] jepsen worker 13 - jepsen.util 3793 :info :write [4 21] :RequestTimeoutException INFO [2022-01-10 05:45:14,305] jepsen worker 13 - jepsen.util 3883 :invoke :write [4 28] INFO [2022-01-10 05:45:14,630] jepsen worker 7 - jepsen.util 3427 :info :write [2 4] :RequestTimeoutException INFO [2022-01-10 05:45:14,631] jepsen worker 7 - jepsen.util 3517 :invoke :write [2 48] INFO [2022-01-10 05:45:14,663] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:14,663] jepsen worker nemesis - couchbase.util Rebalance check count: 465 INFO [2022-01-10 05:45:14,663] jepsen worker nemesis - couchbase.util Rebalance stuck count: 378 INFO [2022-01-10 05:45:14,701] jepsen worker 88 - jepsen.util 3328 :info :write [29 13] :RequestTimeoutException INFO [2022-01-10 05:45:14,702] jepsen worker 88 - jepsen.util 3418 :invoke :write [29 11] INFO [2022-01-10 05:45:14,811] jepsen worker 57 - jepsen.util 4017 :info :write [19 48] :RequestTimeoutException INFO [2022-01-10 05:45:14,812] jepsen worker 57 - jepsen.util 4107 :invoke :read [19 nil] INFO [2022-01-10 05:45:15,088] jepsen worker 40 - jepsen.util 2740 :ok :read [13 15] INFO [2022-01-10 05:45:15,089] jepsen worker 40 - jepsen.util 2740 :invoke :write [13 21] INFO [2022-01-10 05:45:15,216] jepsen worker 14 - jepsen.util 4064 :info :write [4 7] :RequestTimeoutException INFO [2022-01-10 05:45:15,217] jepsen worker 14 - jepsen.util 4154 :invoke :read [4 nil] INFO [2022-01-10 05:45:15,221] jepsen worker 27 - jepsen.util 3357 :info :write [9 23] :RequestTimeoutException INFO [2022-01-10 05:45:15,222] jepsen worker 27 - jepsen.util 3447 :invoke :write [9 22] INFO [2022-01-10 05:45:15,329] jepsen worker 52 - jepsen.util 4192 :info :write [17 13] :RequestTimeoutException INFO [2022-01-10 05:45:15,330] jepsen worker 52 - jepsen.util 4282 :invoke :write [17 41] INFO [2022-01-10 05:45:15,514] jepsen worker 60 - jepsen.util 3660 :ok :read [20 :nil] INFO [2022-01-10 05:45:15,515] jepsen worker 60 - jepsen.util 3660 :invoke :write [20 6] INFO [2022-01-10 05:45:15,669] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:15,669] jepsen worker nemesis - couchbase.util Rebalance check count: 466 INFO [2022-01-10 05:45:15,669] jepsen worker nemesis - couchbase.util Rebalance stuck count: 379 INFO [2022-01-10 05:45:15,911] jepsen worker 2 - jepsen.util 3602 :info :write [0 36] :RequestTimeoutException INFO [2022-01-10 05:45:15,911] jepsen worker 2 - jepsen.util 3692 :invoke :write [0 9] INFO [2022-01-10 05:45:16,072] jepsen worker 14 - jepsen.util 4154 :ok :read [4 3] INFO [2022-01-10 05:45:16,073] jepsen worker 14 - jepsen.util 4154 :invoke :write [4 28] INFO [2022-01-10 05:45:16,158] jepsen worker 8 - jepsen.util 3878 :ok :read [2 14] INFO [2022-01-10 05:45:16,160] jepsen worker 8 - jepsen.util 3878 :invoke :read [2 nil] INFO [2022-01-10 05:45:16,168] jepsen worker 24 - jepsen.util 2994 :ok :read [8 27] INFO [2022-01-10 05:45:16,169] jepsen worker 24 - jepsen.util 2994 :invoke :write [8 16] INFO [2022-01-10 05:45:16,185] jepsen worker 16 - jepsen.util 3436 :ok :read [5 42] INFO [2022-01-10 05:45:16,185] jepsen worker 16 - jepsen.util 3436 :invoke :write [5 43] INFO [2022-01-10 05:45:16,195] jepsen worker 23 - jepsen.util 3443 :ok :read [7 :nil] INFO [2022-01-10 05:45:16,195] jepsen worker 23 - jepsen.util 3443 :invoke :read [7 nil] INFO [2022-01-10 05:45:16,346] jepsen worker 11 - jepsen.util 3341 :info :write [3 32] :RequestTimeoutException INFO [2022-01-10 05:45:16,346] jepsen worker 11 - jepsen.util 3431 :invoke :read [3 nil] INFO [2022-01-10 05:45:16,379] jepsen worker 23 - jepsen.util 3443 :ok :read [7 :nil] INFO [2022-01-10 05:45:16,380] jepsen worker 23 - jepsen.util 3443 :invoke :write [7 23] INFO [2022-01-10 05:45:16,392] jepsen worker 72 - jepsen.util 3762 :ok :read [24 18] INFO [2022-01-10 05:45:16,393] jepsen worker 72 - jepsen.util 3762 :invoke :write [24 14] INFO [2022-01-10 05:45:16,479] jepsen worker 63 - jepsen.util 3393 :ok :read [21 7] INFO [2022-01-10 05:45:16,479] jepsen worker 63 - jepsen.util 3393 :invoke :read [21 nil] INFO [2022-01-10 05:45:16,518] jepsen worker 8 - jepsen.util 3878 :ok :read [2 14] INFO [2022-01-10 05:45:16,518] jepsen worker 8 - jepsen.util 3878 :invoke :read [2 nil] INFO [2022-01-10 05:45:16,525] jepsen worker 44 - jepsen.util 3464 :ok :read [14 14] INFO [2022-01-10 05:45:16,525] jepsen worker 44 - jepsen.util 3464 :invoke :write [14 27] INFO [2022-01-10 05:45:16,571] jepsen worker 81 - jepsen.util 3591 :info :write [27 18] :RequestTimeoutException INFO [2022-01-10 05:45:16,572] jepsen worker 81 - jepsen.util 3681 :invoke :read [27 nil] INFO [2022-01-10 05:45:16,675] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:16,675] jepsen worker nemesis - couchbase.util Rebalance check count: 467 INFO [2022-01-10 05:45:16,675] jepsen worker nemesis - couchbase.util Rebalance stuck count: 380 INFO [2022-01-10 05:45:16,715] jepsen worker 33 - jepsen.util 3633 :info :write [11 43] :RequestTimeoutException INFO [2022-01-10 05:45:16,716] jepsen worker 33 - jepsen.util 3723 :invoke :read [11 nil] INFO [2022-01-10 05:45:16,836] jepsen worker 1 - jepsen.util 3331 :info :write [0 18] :RequestTimeoutException INFO [2022-01-10 05:45:16,836] jepsen worker 1 - jepsen.util 3421 :invoke :write [0 37] INFO [2022-01-10 05:45:17,120] jepsen worker 12 - jepsen.util 3702 :info :write [4 3] :RequestTimeoutException INFO [2022-01-10 05:45:17,120] jepsen worker 12 - jepsen.util 3792 :invoke :write [4 16] INFO [2022-01-10 05:45:17,225] jepsen worker 51 - jepsen.util 4011 :info :write [17 48] :RequestTimeoutException INFO [2022-01-10 05:45:17,225] jepsen worker 51 - jepsen.util 4101 :invoke :write [17 45] INFO [2022-01-10 05:45:17,501] jepsen worker 62 - jepsen.util 3752 :info :write [20 22] :RequestTimeoutException INFO [2022-01-10 05:45:17,502] jepsen worker 62 - jepsen.util 3842 :invoke :write [20 15] INFO [2022-01-10 05:45:17,514] jepsen worker 85 - jepsen.util 3685 :info :write [28 39] :RequestTimeoutException INFO [2022-01-10 05:45:17,515] jepsen worker 85 - jepsen.util 3775 :invoke :read [28 nil] INFO [2022-01-10 05:45:17,680] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:17,680] jepsen worker nemesis - couchbase.util Rebalance check count: 468 INFO [2022-01-10 05:45:17,781] jepsen worker nemesis - couchbase.util Rebalance stuck count: 381 INFO [2022-01-10 05:45:17,930] jepsen worker 59 - jepsen.util 3299 :info :write [19 18] :RequestTimeoutException INFO [2022-01-10 05:45:17,930] jepsen worker 59 - jepsen.util 3389 :invoke :write [19 44] INFO [2022-01-10 05:45:18,068] jepsen worker 49 - jepsen.util 3289 :ok :read [16 38] INFO [2022-01-10 05:45:18,069] jepsen worker 49 - jepsen.util 3289 :invoke :write [16 30] INFO [2022-01-10 05:45:18,069] jepsen worker 8 - jepsen.util 3878 :ok :read [2 14] INFO [2022-01-10 05:45:18,070] jepsen worker 8 - jepsen.util 3878 :invoke :read [2 nil] INFO [2022-01-10 05:45:18,094] jepsen worker 42 - jepsen.util 3372 :ok :read [14 14] INFO [2022-01-10 05:45:18,095] jepsen worker 42 - jepsen.util 3372 :invoke :write [14 3] INFO [2022-01-10 05:45:18,282] jepsen worker 56 - jepsen.util 3836 :ok :read [18 14] INFO [2022-01-10 05:45:18,283] jepsen worker 56 - jepsen.util 3836 :invoke :write [18 16] INFO [2022-01-10 05:45:18,584] jepsen worker 8 - jepsen.util 3878 :ok :read [2 14] INFO [2022-01-10 05:45:18,584] jepsen worker 8 - jepsen.util 3878 :invoke :read [2 nil] INFO [2022-01-10 05:45:18,723] jepsen worker 58 - jepsen.util 3298 :ok :read [19 31] INFO [2022-01-10 05:45:18,724] jepsen worker 58 - jepsen.util 3298 :invoke :write [19 7] INFO [2022-01-10 05:45:18,728] jepsen worker 66 - jepsen.util 3126 :ok :read [22 41] INFO [2022-01-10 05:45:18,728] jepsen worker 66 - jepsen.util 3126 :invoke :write [22 0] INFO [2022-01-10 05:45:18,787] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:18,787] jepsen worker nemesis - couchbase.util Rebalance check count: 469 INFO [2022-01-10 05:45:18,787] jepsen worker nemesis - couchbase.util Rebalance stuck count: 382 INFO [2022-01-10 05:45:18,901] jepsen worker 39 - jepsen.util 3909 :info :write [13 17] :RequestTimeoutException INFO [2022-01-10 05:45:18,901] jepsen worker 39 - jepsen.util 3999 :invoke :write [13 42] INFO [2022-01-10 05:45:19,170] jepsen worker 8 - jepsen.util 3878 :ok :read [2 14] INFO [2022-01-10 05:45:19,171] jepsen worker 8 - jepsen.util 3878 :invoke :write [2 10] INFO [2022-01-10 05:45:19,284] jepsen worker 57 - jepsen.util 4107 :ok :read [19 31] INFO [2022-01-10 05:45:19,285] jepsen worker 57 - jepsen.util 4107 :invoke :write [19 48] INFO [2022-01-10 05:45:19,292] jepsen worker 81 - jepsen.util 3681 :ok :read [27 30] INFO [2022-01-10 05:45:19,293] jepsen worker 81 - jepsen.util 3681 :invoke :write [27 15] INFO [2022-01-10 05:45:19,298] jepsen worker 83 - jepsen.util 3863 :ok :read [27 30] INFO [2022-01-10 05:45:19,298] jepsen worker 83 - jepsen.util 3863 :invoke :read [27 nil] INFO [2022-01-10 05:45:19,317] jepsen worker 15 - jepsen.util 2895 :info :write [5 23] :RequestTimeoutException INFO [2022-01-10 05:45:19,317] jepsen worker 15 - jepsen.util 2985 :invoke :read [5 nil] INFO [2022-01-10 05:45:19,361] jepsen worker 19 - jepsen.util 3439 :info :write [6 44] :RequestTimeoutException INFO [2022-01-10 05:45:19,362] jepsen worker 19 - jepsen.util 3529 :invoke :read [6 nil] INFO [2022-01-10 05:45:19,366] jepsen worker 54 - jepsen.util 3474 :info :write [18 45] :RequestTimeoutException INFO [2022-01-10 05:45:19,367] jepsen worker 54 - jepsen.util 3564 :invoke :read [18 nil] INFO [2022-01-10 05:45:19,793] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:19,793] jepsen worker nemesis - couchbase.util Rebalance check count: 470 INFO [2022-01-10 05:45:19,793] jepsen worker nemesis - couchbase.util Rebalance stuck count: 383 INFO [2022-01-10 05:45:19,830] jepsen worker 0 - jepsen.util 3150 :info :write [0 37] :RequestTimeoutException INFO [2022-01-10 05:45:19,831] jepsen worker 0 - jepsen.util 3240 :invoke :read [0 nil] INFO [2022-01-10 05:45:19,873] jepsen worker 85 - jepsen.util 3775 :ok :read [28 45] INFO [2022-01-10 05:45:19,874] jepsen worker 85 - jepsen.util 3775 :invoke :write [28 0] INFO [2022-01-10 05:45:20,094] jepsen worker 17 - jepsen.util 3257 :info :write [5 34] :RequestTimeoutException INFO [2022-01-10 05:45:20,095] jepsen worker 17 - jepsen.util 3347 :invoke :read [5 nil] INFO [2022-01-10 05:45:20,136] jepsen worker 61 - jepsen.util 3481 :info :write [20 8] :RequestTimeoutException INFO [2022-01-10 05:45:20,137] jepsen worker 61 - jepsen.util 3571 :invoke :read [20 nil] INFO [2022-01-10 05:45:20,165] jepsen worker 89 - jepsen.util 3779 :ok :read [29 0] INFO [2022-01-10 05:45:20,166] jepsen worker 89 - jepsen.util 3779 :invoke :read [29 nil] INFO [2022-01-10 05:45:20,180] jepsen worker 38 - jepsen.util 3548 :ok :read [12 29] INFO [2022-01-10 05:45:20,181] jepsen worker 31 - jepsen.util 3811 :ok :read [10 36] INFO [2022-01-10 05:45:20,181] jepsen worker 38 - jepsen.util 3548 :invoke :read [12 nil] INFO [2022-01-10 05:45:20,181] jepsen worker 31 - jepsen.util 3811 :invoke :write [10 48] INFO [2022-01-10 05:45:20,206] jepsen worker 38 - jepsen.util 3548 :ok :read [12 29] INFO [2022-01-10 05:45:20,207] jepsen worker 38 - jepsen.util 3548 :invoke :read [12 nil] INFO [2022-01-10 05:45:20,210] jepsen worker 89 - jepsen.util 3779 :ok :read [29 0] INFO [2022-01-10 05:45:20,216] jepsen worker 89 - jepsen.util 3779 :invoke :read [29 nil] INFO [2022-01-10 05:45:20,270] jepsen worker 38 - jepsen.util 3548 :ok :read [12 29] INFO [2022-01-10 05:45:20,271] jepsen worker 38 - jepsen.util 3548 :invoke :write [12 11] INFO [2022-01-10 05:45:20,290] jepsen worker 89 - jepsen.util 3779 :ok :read [29 0] INFO [2022-01-10 05:45:20,290] jepsen worker 19 - jepsen.util 3529 :ok :read [6 25] INFO [2022-01-10 05:45:20,290] jepsen worker 89 - jepsen.util 3779 :invoke :read [29 nil] INFO [2022-01-10 05:45:20,291] jepsen worker 19 - jepsen.util 3529 :invoke :read [6 nil] WARN [2022-01-10 05:45:20,312] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9039805,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10078443,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9274964,"encode_duration_us":16596,"operation_id":"0x68f5","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10002698},{"operation_name":"upsert","last_dispatch_duration_us":9516273,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12760612,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10107512,"encode_duration_us":16214,"operation_id":"0x6753","timeout_ms":10000,"last_server_duration_us":9088372,"total_duration_us":10003739},{"operation_name":"upsert","last_dispatch_duration_us":9606745,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12058735,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10583156,"encode_duration_us":19129,"operation_id":"0x68cb","timeout_ms":10000,"last_server_duration_us":9483465,"total_duration_us":10007213},{"operation_name":"upsert","last_dispatch_duration_us":9476235,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12359147,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9958967,"encode_duration_us":16182,"operation_id":"0x6722","timeout_ms":10000,"last_server_duration_us":9112856,"total_duration_us":10007710}],"total_count":4}} INFO [2022-01-10 05:45:20,316] jepsen worker 29 - jepsen.util 3269 :info :write [9 37] :RequestTimeoutException INFO [2022-01-10 05:45:20,317] jepsen worker 29 - jepsen.util 3359 :invoke :read [9 nil] INFO [2022-01-10 05:45:20,363] jepsen worker 89 - jepsen.util 3779 :ok :read [29 0] INFO [2022-01-10 05:45:20,363] jepsen worker 89 - jepsen.util 3779 :invoke :read [29 nil] WARN [2022-01-10 05:45:20,412] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":381150,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":381150,"total_server_duration_us":170173,"operation_id":"0x696b","timeout_ms":30000,"last_server_duration_us":170173,"total_duration_us":11664377},{"operation_name":"get","last_dispatch_duration_us":10342476,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10342476,"total_server_duration_us":5907,"operation_id":"0x6892","timeout_ms":30000,"last_server_duration_us":5907,"total_duration_us":10345347},{"operation_name":"upsert","last_dispatch_duration_us":38593,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":1144167,"total_server_duration_us":290937,"encode_duration_us":20524,"operation_id":"0x68f4","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10010309},{"operation_name":"upsert","last_dispatch_duration_us":2436109,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":9216838,"total_server_duration_us":3787123,"encode_duration_us":16963,"operation_id":"0x69ef","timeout_ms":10000,"last_server_duration_us":589548,"total_duration_us":10010046},{"operation_name":"upsert","last_dispatch_duration_us":36509,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":1160799,"total_server_duration_us":290945,"encode_duration_us":17017,"operation_id":"0x68f7","timeout_ms":10000,"last_server_duration_us":24583,"total_duration_us":10009955},{"operation_name":"upsert","last_dispatch_duration_us":600384,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":6371314,"total_server_duration_us":1459228,"encode_duration_us":18907,"operation_id":"0x695d","timeout_ms":10000,"last_server_duration_us":160495,"total_duration_us":10007356},{"operation_name":"upsert","last_dispatch_duration_us":9606745,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":12058735,"total_server_duration_us":10583156,"encode_duration_us":19129,"operation_id":"0x68cb","timeout_ms":10000,"last_server_duration_us":9483465,"total_duration_us":10007213},{"operation_name":"upsert","last_dispatch_duration_us":1588922,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":5692548,"total_server_duration_us":416499,"encode_duration_us":17697,"operation_id":"0x69f2","timeout_ms":10000,"last_server_duration_us":26685,"total_duration_us":10005291},{"operation_name":"upsert","last_dispatch_duration_us":1719792,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":1719792,"total_server_duration_us":188605,"encode_duration_us":16781,"operation_id":"0x6a66","timeout_ms":10000,"last_server_duration_us":188605,"total_duration_us":10004815},{"operation_name":"upsert","last_dispatch_duration_us":9039805,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10078443,"total_server_duration_us":9274964,"encode_duration_us":16596,"operation_id":"0x68f5","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10002698}],"total_count":19}} INFO [2022-01-10 05:45:20,427] jepsen worker 89 - jepsen.util 3779 :ok :read [29 0] INFO [2022-01-10 05:45:20,428] jepsen worker 89 - jepsen.util 3779 :invoke :read [29 nil] INFO [2022-01-10 05:45:20,497] jepsen worker 89 - jepsen.util 3779 :ok :read [29 0] INFO [2022-01-10 05:45:20,498] jepsen worker 89 - jepsen.util 3779 :invoke :read [29 nil] INFO [2022-01-10 05:45:20,550] jepsen worker 71 - jepsen.util 2861 :info :write [23 3] :RequestTimeoutException INFO [2022-01-10 05:45:20,551] jepsen worker 71 - jepsen.util 2951 :invoke :write [23 42] INFO [2022-01-10 05:45:20,572] jepsen worker 89 - jepsen.util 3779 :ok :read [29 0] INFO [2022-01-10 05:45:20,576] jepsen worker 89 - jepsen.util 3779 :invoke :read [29 nil] INFO [2022-01-10 05:45:20,576] jepsen worker 68 - jepsen.util 3398 :info :write [22 2] :RequestTimeoutException INFO [2022-01-10 05:45:20,577] jepsen worker 68 - jepsen.util 3488 :invoke :write [22 21] INFO [2022-01-10 05:45:20,634] jepsen worker 89 - jepsen.util 3779 :ok :read [29 0] INFO [2022-01-10 05:45:20,634] jepsen worker 89 - jepsen.util 3779 :invoke :write [29 6] INFO [2022-01-10 05:45:20,661] jepsen worker 4 - jepsen.util 3874 :info :write [1 7] :RequestTimeoutException INFO [2022-01-10 05:45:20,662] jepsen worker 4 - jepsen.util 3964 :invoke :write [1 20] INFO [2022-01-10 05:45:20,685] jepsen worker 70 - jepsen.util 3400 :info :write [23 12] :RequestTimeoutException INFO [2022-01-10 05:45:20,686] jepsen worker 70 - jepsen.util 3490 :invoke :read [23 nil] INFO [2022-01-10 05:45:20,690] jepsen worker 63 - jepsen.util 3393 :ok :read [21 7] INFO [2022-01-10 05:45:20,691] jepsen worker 73 - jepsen.util 3133 :ok :read [24 18] INFO [2022-01-10 05:45:20,691] jepsen worker 73 - jepsen.util 3133 :invoke :read [24 nil] INFO [2022-01-10 05:45:20,691] jepsen worker 63 - jepsen.util 3393 :invoke :read [21 nil] INFO [2022-01-10 05:45:20,732] jepsen worker 34 - jepsen.util 3274 :info :write [11 27] :RequestTimeoutException INFO [2022-01-10 05:45:20,733] jepsen worker 34 - jepsen.util 3364 :invoke :read [11 nil] INFO [2022-01-10 05:45:20,777] jepsen worker 70 - jepsen.util 3490 :ok :read [23 26] INFO [2022-01-10 05:45:20,777] jepsen worker 70 - jepsen.util 3490 :invoke :read [23 nil] INFO [2022-01-10 05:45:20,799] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:20,799] jepsen worker nemesis - couchbase.util Rebalance check count: 471 INFO [2022-01-10 05:45:20,799] jepsen worker nemesis - couchbase.util Rebalance stuck count: 384 INFO [2022-01-10 05:45:20,801] jepsen worker 47 - jepsen.util 3377 :info :write [15 37] :RequestTimeoutException INFO [2022-01-10 05:45:20,802] jepsen worker 47 - jepsen.util 3467 :invoke :write [15 22] INFO [2022-01-10 05:45:20,811] jepsen worker 70 - jepsen.util 3490 :ok :read [23 26] INFO [2022-01-10 05:45:20,812] jepsen worker 70 - jepsen.util 3490 :invoke :write [23 33] INFO [2022-01-10 05:45:20,865] jepsen worker 28 - jepsen.util 3088 :info :write [9 17] :RequestTimeoutException INFO [2022-01-10 05:45:20,866] jepsen worker 28 - jepsen.util 3178 :invoke :write [9 45] INFO [2022-01-10 05:45:21,019] jepsen worker 46 - jepsen.util 3466 :info :write [15 43] :RequestTimeoutException INFO [2022-01-10 05:45:21,020] jepsen worker 46 - jepsen.util 3556 :invoke :write [15 0] INFO [2022-01-10 05:45:21,085] jepsen worker 19 - jepsen.util 3529 :ok :read [6 25] INFO [2022-01-10 05:45:21,086] jepsen worker 19 - jepsen.util 3529 :invoke :write [6 23] INFO [2022-01-10 05:45:21,177] jepsen worker 17 - jepsen.util 3347 :ok :read [5 42] INFO [2022-01-10 05:45:21,178] jepsen worker 17 - jepsen.util 3347 :invoke :read [5 nil] INFO [2022-01-10 05:45:21,211] jepsen worker 65 - jepsen.util 3125 :info :write [21 8] :RequestTimeoutException INFO [2022-01-10 05:45:21,212] jepsen worker 65 - jepsen.util 3215 :invoke :read [21 nil] INFO [2022-01-10 05:45:21,254] jepsen worker 48 - jepsen.util 2928 :info :write [16 0] :RequestTimeoutException INFO [2022-01-10 05:45:21,255] jepsen worker 48 - jepsen.util 3018 :invoke :write [16 20] INFO [2022-01-10 05:45:21,292] jepsen worker 77 - jepsen.util 3677 :info :write [25 10] :RequestTimeoutException INFO [2022-01-10 05:45:21,292] jepsen worker 77 - jepsen.util 3767 :invoke :write [25 44] INFO [2022-01-10 05:45:21,299] jepsen worker 0 - jepsen.util 3240 :ok :read [0 46] INFO [2022-01-10 05:45:21,299] jepsen worker 0 - jepsen.util 3240 :invoke :read [0 nil] INFO [2022-01-10 05:45:21,306] jepsen worker 35 - jepsen.util 3005 :info :write [11 43] :RequestTimeoutException INFO [2022-01-10 05:45:21,307] jepsen worker 35 - jepsen.util 3095 :invoke :write [11 36] INFO [2022-01-10 05:45:21,612] jepsen worker 6 - jepsen.util 3426 :info :write [2 6] :RequestTimeoutException INFO [2022-01-10 05:45:21,613] jepsen worker 6 - jepsen.util 3516 :invoke :read [2 nil] INFO [2022-01-10 05:45:21,665] jepsen worker 82 - jepsen.util 3502 :info :write [27 24] :RequestTimeoutException INFO [2022-01-10 05:45:21,665] jepsen worker 82 - jepsen.util 3592 :invoke :write [27 35] INFO [2022-01-10 05:45:21,805] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:21,805] jepsen worker nemesis - couchbase.util Rebalance check count: 472 INFO [2022-01-10 05:45:21,805] jepsen worker nemesis - couchbase.util Rebalance stuck count: 385 INFO [2022-01-10 05:45:21,812] jepsen worker 64 - jepsen.util 3304 :info :write [21 28] :RequestTimeoutException INFO [2022-01-10 05:45:21,812] jepsen worker 64 - jepsen.util 3394 :invoke :read [21 nil] INFO [2022-01-10 05:45:21,846] jepsen worker 21 - jepsen.util 3531 :info :write [7 26] :RequestTimeoutException INFO [2022-01-10 05:45:21,846] jepsen worker 21 - jepsen.util 3621 :invoke :write [7 30] INFO [2022-01-10 05:45:21,885] jepsen worker 25 - jepsen.util 3175 :info :write [8 1] :RequestTimeoutException INFO [2022-01-10 05:45:21,886] jepsen worker 25 - jepsen.util 3265 :invoke :write [8 12] WARN [2022-01-10 05:45:21,912] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":25322321,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":25322321,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9027816,"encode_duration_us":16518,"operation_id":"0x6723","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10006098},{"operation_name":"upsert","last_dispatch_duration_us":9730167,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13637712,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":10804983,"encode_duration_us":16631,"operation_id":"0x67bb","timeout_ms":10000,"last_server_duration_us":9050106,"total_duration_us":10009426},{"operation_name":"upsert","last_dispatch_duration_us":11728353,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12754952,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":11127526,"encode_duration_us":19776,"operation_id":"0x6890","timeout_ms":10000,"last_server_duration_us":11017794,"total_duration_us":10006981}],"total_count":3}} INFO [2022-01-10 05:45:21,926] jepsen worker 10 - jepsen.util 3430 :info :write [3 19] :RequestTimeoutException INFO [2022-01-10 05:45:21,927] jepsen worker 10 - jepsen.util 3520 :invoke :write [3 0] WARN [2022-01-10 05:45:22,011] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10068118,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11376968,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":10185991,"encode_duration_us":16667,"operation_id":"0x695c","timeout_ms":10000,"last_server_duration_us":9990672,"total_duration_us":10001400},{"operation_name":"upsert","last_dispatch_duration_us":9939294,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14555972,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10335327,"encode_duration_us":27713,"operation_id":"0x688d","timeout_ms":10000,"last_server_duration_us":9423962,"total_duration_us":10005669},{"operation_name":"upsert","last_dispatch_duration_us":17889283,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18892262,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9187081,"encode_duration_us":16479,"operation_id":"0x6899","timeout_ms":10000,"last_server_duration_us":9020392,"total_duration_us":10002253}],"total_count":3}} WARN [2022-01-10 05:45:22,011] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9171606,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11068504,"total_server_duration_us":553423,"encode_duration_us":16163,"operation_id":"0x6a39","timeout_ms":10000,"last_server_duration_us":164724,"total_duration_us":10008648},{"operation_name":"upsert","last_dispatch_duration_us":8980926,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":13153812,"total_server_duration_us":947595,"encode_duration_us":16673,"operation_id":"0x69b4","timeout_ms":10000,"last_server_duration_us":145331,"total_duration_us":10008387},{"operation_name":"upsert","last_dispatch_duration_us":8459595,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":8644228,"total_server_duration_us":42245,"encode_duration_us":17286,"operation_id":"0x6927","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10006353},{"operation_name":"upsert","last_dispatch_duration_us":774007,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":6049164,"total_server_duration_us":4151264,"encode_duration_us":30102,"operation_id":"0x6906","timeout_ms":10000,"last_server_duration_us":760760,"total_duration_us":10006129},{"operation_name":"upsert","last_dispatch_duration_us":8897841,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11894266,"total_server_duration_us":709996,"encode_duration_us":17732,"operation_id":"0x6928","timeout_ms":10000,"last_server_duration_us":127328,"total_duration_us":10005757},{"operation_name":"upsert","last_dispatch_duration_us":211831,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":211831,"total_server_duration_us":21324,"encode_duration_us":23108,"operation_id":"0x6a77","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10005343},{"operation_name":"upsert","last_dispatch_duration_us":7812091,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":13540987,"total_server_duration_us":2142524,"encode_duration_us":16212,"operation_id":"0x69c0","timeout_ms":10000,"last_server_duration_us":164917,"total_duration_us":10003930},{"operation_name":"upsert","last_dispatch_duration_us":8712528,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":13543913,"total_server_duration_us":1028412,"encode_duration_us":16344,"operation_id":"0x697e","timeout_ms":10000,"last_server_duration_us":254676,"total_duration_us":10003547},{"operation_name":"upsert","last_dispatch_duration_us":2355055,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11292177,"total_server_duration_us":465390,"encode_duration_us":18235,"operation_id":"0x69f1","timeout_ms":10000,"last_server_duration_us":394565,"total_duration_us":10003262},{"operation_name":"upsert","last_dispatch_duration_us":1490181,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10154091,"total_server_duration_us":1326485,"encode_duration_us":16694,"operation_id":"0x6a45","timeout_ms":10000,"last_server_duration_us":98413,"total_duration_us":10002584}],"total_count":13}} INFO [2022-01-10 05:45:22,036] jepsen worker 45 - jepsen.util 3645 :info :write [15 42] :RequestTimeoutException INFO [2022-01-10 05:45:22,037] jepsen worker 45 - jepsen.util 3735 :invoke :write [15 4] INFO [2022-01-10 05:45:22,042] jepsen worker 65 - jepsen.util 3215 :ok :read [21 7] INFO [2022-01-10 05:45:22,043] jepsen worker 65 - jepsen.util 3215 :invoke :write [21 4] WARN [2022-01-10 05:45:22,060] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10987725,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13554358,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":11186232,"encode_duration_us":17074,"operation_id":"0x68f0","timeout_ms":10000,"last_server_duration_us":10819953,"total_duration_us":10004710},{"operation_name":"upsert","last_dispatch_duration_us":26781727,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":35901476,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9234402,"encode_duration_us":18521,"operation_id":"0x64db","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10007756}],"total_count":2}} INFO [2022-01-10 05:45:22,076] jepsen worker 22 - jepsen.util 3082 :info :write [7 12] :RequestTimeoutException INFO [2022-01-10 05:45:22,077] jepsen worker 22 - jepsen.util 3172 :invoke :read [7 nil] WARN [2022-01-10 05:45:22,111] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11627045,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11627045,"total_server_duration_us":43618,"operation_id":"0x688b","timeout_ms":30000,"last_server_duration_us":43618,"total_duration_us":15267490},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":19505,"operation_id":"0x69bf","timeout_ms":10000,"total_duration_us":10008728},{"operation_name":"upsert","last_dispatch_duration_us":1730164,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11298582,"total_server_duration_us":836167,"encode_duration_us":17070,"operation_id":"0x6a76","timeout_ms":10000,"last_server_duration_us":173518,"total_duration_us":10008371},{"operation_name":"upsert","last_dispatch_duration_us":2299169,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11664241,"total_server_duration_us":1097915,"encode_duration_us":17092,"operation_id":"0x6a44","timeout_ms":10000,"last_server_duration_us":162796,"total_duration_us":10007591},{"operation_name":"upsert","last_dispatch_duration_us":2152884,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11088506,"total_server_duration_us":388570,"encode_duration_us":16748,"operation_id":"0x6a70","timeout_ms":10000,"last_server_duration_us":137901,"total_duration_us":10007211},{"operation_name":"upsert","last_dispatch_duration_us":6164548,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10108689,"total_server_duration_us":1476943,"encode_duration_us":16406,"operation_id":"0x69e3","timeout_ms":10000,"last_server_duration_us":206347,"total_duration_us":10006623},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":17406,"operation_id":"0x6aae","timeout_ms":10000,"total_duration_us":10004851},{"operation_name":"upsert","last_dispatch_duration_us":11459725,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":11459725,"total_server_duration_us":19966,"encode_duration_us":25033,"operation_id":"0x68f8","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10004553},{"operation_name":"upsert","last_dispatch_duration_us":1574797,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":6843283,"total_server_duration_us":1800955,"encode_duration_us":16908,"operation_id":"0x6993","timeout_ms":10000,"last_server_duration_us":610647,"total_duration_us":10003636},{"operation_name":"upsert","last_dispatch_duration_us":11692070,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11692070,"total_server_duration_us":158779,"encode_duration_us":16634,"operation_id":"0x6954","timeout_ms":10000,"last_server_duration_us":158779,"total_duration_us":10003452}],"total_count":14}} WARN [2022-01-10 05:45:22,112] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":820969,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":5067415,"total_server_duration_us":4424852,"encode_duration_us":16318,"operation_id":"0x6904","timeout_ms":10000,"last_server_duration_us":793658,"total_duration_us":10009732},{"operation_name":"upsert","last_dispatch_duration_us":143485,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":340548,"total_server_duration_us":38760,"encode_duration_us":18717,"operation_id":"0x68f6","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10009052},{"operation_name":"upsert","last_dispatch_duration_us":2264875,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10906618,"total_server_duration_us":2425653,"encode_duration_us":29096,"operation_id":"0x6992","timeout_ms":10000,"last_server_duration_us":1061964,"total_duration_us":10008969},{"operation_name":"upsert","last_dispatch_duration_us":139909,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":265616,"total_server_duration_us":39077,"encode_duration_us":17567,"operation_id":"0x6953","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10007298},{"operation_name":"upsert","last_dispatch_duration_us":5223398,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6277429,"total_server_duration_us":266382,"encode_duration_us":16932,"operation_id":"0x6acf","timeout_ms":10000,"last_server_duration_us":156881,"total_duration_us":10006304},{"operation_name":"upsert","last_dispatch_duration_us":15436062,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":15436062,"total_server_duration_us":9032060,"encode_duration_us":19197,"operation_id":"0x6962","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10004501},{"operation_name":"upsert","last_dispatch_duration_us":3479041,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":4945634,"total_server_duration_us":355351,"encode_duration_us":20143,"operation_id":"0x6a88","timeout_ms":10000,"last_server_duration_us":185749,"total_duration_us":10003889},{"operation_name":"upsert","last_dispatch_duration_us":5476545,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7307878,"total_server_duration_us":523732,"encode_duration_us":17047,"operation_id":"0x6ab4","timeout_ms":10000,"last_server_duration_us":372419,"total_duration_us":10003739},{"operation_name":"upsert","last_dispatch_duration_us":4377947,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8854286,"total_server_duration_us":763329,"encode_duration_us":21887,"operation_id":"0x6aad","timeout_ms":10000,"last_server_duration_us":559304,"total_duration_us":10002504},{"operation_name":"upsert","last_dispatch_duration_us":4913102,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9213292,"total_server_duration_us":2253858,"encode_duration_us":18270,"operation_id":"0x6a95","timeout_ms":10000,"last_server_duration_us":244757,"total_duration_us":10001083}],"total_count":26}} WARN [2022-01-10 05:45:22,113] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15294009,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":15294009,"total_server_duration_us":156881,"operation_id":"0x6932","timeout_ms":30000,"last_server_duration_us":156881,"total_duration_us":15296299},{"operation_name":"get","last_dispatch_duration_us":10534827,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10534827,"total_server_duration_us":27,"operation_id":"0x69ff","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":10537290},{"operation_name":"upsert","last_dispatch_duration_us":215861,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":5745989,"total_server_duration_us":3308972,"encode_duration_us":17482,"operation_id":"0x692a","timeout_ms":10000,"last_server_duration_us":197705,"total_duration_us":10007893},{"operation_name":"upsert","last_dispatch_duration_us":3616805,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3616805,"total_server_duration_us":181498,"encode_duration_us":16617,"operation_id":"0x6a6a","timeout_ms":10000,"last_server_duration_us":181498,"total_duration_us":10006249},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","encode_duration_us":17105,"operation_id":"0x6a46","timeout_ms":10000,"total_duration_us":10005717},{"operation_name":"upsert","last_dispatch_duration_us":377973,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":377973,"total_server_duration_us":80683,"encode_duration_us":18233,"operation_id":"0x6ad9","timeout_ms":10000,"last_server_duration_us":80683,"total_duration_us":10001511},{"operation_name":"upsert","last_dispatch_duration_us":4649355,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9096625,"total_server_duration_us":358735,"encode_duration_us":26768,"operation_id":"0x6ad8","timeout_ms":10000,"last_server_duration_us":203167,"total_duration_us":10001109},{"operation_name":"upsert","last_dispatch_duration_us":3118821,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9134602,"total_server_duration_us":138643,"encode_duration_us":17183,"operation_id":"0x69c9","timeout_ms":10000,"last_server_duration_us":64993,"total_duration_us":10001015},{"operation_name":"get","last_dispatch_duration_us":6016341,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6016341,"total_server_duration_us":3273,"operation_id":"0x69b6","timeout_ms":30000,"last_server_duration_us":3273,"total_duration_us":6016636},{"operation_name":"get","last_dispatch_duration_us":4191868,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4191868,"total_server_duration_us":358855,"operation_id":"0x6ba5","timeout_ms":30000,"last_server_duration_us":358855,"total_duration_us":4194069}],"total_count":12}} INFO [2022-01-10 05:45:22,175] jepsen worker 20 - jepsen.util 3080 :info :write [6 48] :RequestTimeoutException INFO [2022-01-10 05:45:22,176] jepsen worker 20 - jepsen.util 3170 :invoke :read [6 nil] WARN [2022-01-10 05:45:22,211] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9947432,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9947432,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9482382,"encode_duration_us":22280,"operation_id":"0x681f","timeout_ms":10000,"last_server_duration_us":9482382,"total_duration_us":10001944},{"operation_name":"upsert","last_dispatch_duration_us":16651477,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16977675,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9195646,"encode_duration_us":20674,"operation_id":"0x67c4","timeout_ms":10000,"last_server_duration_us":9021452,"total_duration_us":10007385}],"total_count":2}} WARN [2022-01-10 05:45:22,212] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":15436062,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15436062,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9032060,"encode_duration_us":19197,"operation_id":"0x6962","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10004501}],"total_count":1}} INFO [2022-01-10 05:45:22,242] jepsen worker 26 - jepsen.util 2996 :info :write [8 26] :RequestTimeoutException INFO [2022-01-10 05:45:22,242] jepsen worker 26 - jepsen.util 3086 :invoke :write [8 42] WARN [2022-01-10 05:45:22,260] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10039753,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10039753,"total_server_duration_us":32,"operation_id":"0x69b7","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":10043614},{"operation_name":"upsert","last_dispatch_duration_us":229036,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":4844721,"total_server_duration_us":2104215,"encode_duration_us":17388,"operation_id":"0x6995","timeout_ms":10000,"last_server_duration_us":217098,"total_duration_us":10010053},{"operation_name":"upsert","last_dispatch_duration_us":1676952,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11407957,"total_server_duration_us":1050988,"encode_duration_us":16503,"operation_id":"0x6a65","timeout_ms":10000,"last_server_duration_us":199379,"total_duration_us":10009973},{"operation_name":"upsert","last_dispatch_duration_us":5426032,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7922716,"total_server_duration_us":756913,"encode_duration_us":17113,"operation_id":"0x6905","timeout_ms":10000,"last_server_duration_us":380108,"total_duration_us":10007566},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":16764,"operation_id":"0x6a74","timeout_ms":10000,"total_duration_us":10005944},{"operation_name":"upsert","last_dispatch_duration_us":1291664,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9963817,"total_server_duration_us":2445569,"encode_duration_us":20546,"operation_id":"0x6a9f","timeout_ms":10000,"last_server_duration_us":269748,"total_duration_us":10005060},{"operation_name":"upsert","last_dispatch_duration_us":1094409,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":6244295,"total_server_duration_us":3618244,"encode_duration_us":16434,"operation_id":"0x69e8","timeout_ms":10000,"last_server_duration_us":1079530,"total_duration_us":10004387},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":22863,"operation_id":"0x6933","timeout_ms":10000,"total_duration_us":10003560},{"operation_name":"upsert","last_dispatch_duration_us":5368441,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11313677,"total_server_duration_us":842239,"encode_duration_us":22913,"operation_id":"0x6a12","timeout_ms":10000,"last_server_duration_us":279354,"total_duration_us":10003269},{"operation_name":"upsert","last_dispatch_duration_us":10658167,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11256895,"total_server_duration_us":245347,"encode_duration_us":16723,"operation_id":"0x6961","timeout_ms":10000,"last_server_duration_us":186971,"total_duration_us":10002626}],"total_count":20}} INFO [2022-01-10 05:45:22,306] jepsen worker 64 - jepsen.util 3394 :ok :read [21 7] INFO [2022-01-10 05:45:22,306] jepsen worker 64 - jepsen.util 3394 :invoke :read [21 nil] INFO [2022-01-10 05:45:22,402] jepsen worker 6 - jepsen.util 3516 :ok :read [2 14] INFO [2022-01-10 05:45:22,403] jepsen worker 6 - jepsen.util 3516 :invoke :write [2 21] INFO [2022-01-10 05:45:22,589] jepsen worker 3 - jepsen.util 3333 :ok :read [1 10] INFO [2022-01-10 05:45:22,589] jepsen worker 3 - jepsen.util 3333 :invoke :read [1 nil] INFO [2022-01-10 05:45:22,625] jepsen worker 78 - jepsen.util 4038 :info :write [26 49] :RequestTimeoutException INFO [2022-01-10 05:45:22,628] jepsen worker 78 - jepsen.util 4128 :invoke :write [26 19] INFO [2022-01-10 05:45:22,641] jepsen worker 17 - jepsen.util 3347 :ok :read [5 42] INFO [2022-01-10 05:45:22,641] jepsen worker 17 - jepsen.util 3347 :invoke :write [5 46] INFO [2022-01-10 05:45:22,811] jepsen worker 20 - jepsen.util 3170 :ok :read [6 25] INFO [2022-01-10 05:45:22,811] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:22,811] jepsen worker 80 - jepsen.util 4130 :info :write [26 13] :RequestTimeoutException INFO [2022-01-10 05:45:23,086] jepsen worker nemesis - couchbase.util Rebalance check count: 473 INFO [2022-01-10 05:45:22,815] jepsen worker 20 - jepsen.util 3170 :invoke :read [6 nil] INFO [2022-01-10 05:45:23,086] jepsen worker nemesis - couchbase.util Rebalance stuck count: 386 INFO [2022-01-10 05:45:23,086] jepsen worker 80 - jepsen.util 4220 :invoke :read [26 nil] INFO [2022-01-10 05:45:22,868] jepsen worker 0 - jepsen.util 3240 :ok :read [0 46] INFO [2022-01-10 05:45:23,086] jepsen worker 0 - jepsen.util 3240 :invoke :read [0 nil] INFO [2022-01-10 05:45:23,171] jepsen worker 64 - jepsen.util 3394 :ok :read [21 7] INFO [2022-01-10 05:45:23,171] jepsen worker 64 - jepsen.util 3394 :invoke :read [21 nil] INFO [2022-01-10 05:45:23,262] jepsen worker 41 - jepsen.util 3191 :info :write [13 23] :RequestTimeoutException INFO [2022-01-10 05:45:23,263] jepsen worker 41 - jepsen.util 3281 :invoke :read [13 nil] INFO [2022-01-10 05:45:23,566] jepsen worker 5 - jepsen.util 3245 :info :write [1 18] :RequestTimeoutException INFO [2022-01-10 05:45:23,567] jepsen worker 5 - jepsen.util 3335 :invoke :read [1 nil] INFO [2022-01-10 05:45:23,583] jepsen worker 63 - jepsen.util 3393 :ok :read [21 7] INFO [2022-01-10 05:45:23,584] jepsen worker 63 - jepsen.util 3393 :invoke :write [21 24] INFO [2022-01-10 05:45:23,586] jepsen worker 33 - jepsen.util 3723 :ok :read [11 21] INFO [2022-01-10 05:45:23,587] jepsen worker 33 - jepsen.util 3723 :invoke :write [11 20] INFO [2022-01-10 05:45:23,596] jepsen worker 79 - jepsen.util 3949 :info :write [26 39] :RequestTimeoutException INFO [2022-01-10 05:45:23,596] jepsen worker 9 - jepsen.util 3879 :info :write [3 25] :RequestTimeoutException INFO [2022-01-10 05:45:23,597] jepsen worker 9 - jepsen.util 3969 :invoke :read [3 nil] INFO [2022-01-10 05:45:23,597] jepsen worker 79 - jepsen.util 4039 :invoke :write [26 16] INFO [2022-01-10 05:45:23,625] jepsen worker 43 - jepsen.util 3283 :info :write [14 6] :RequestTimeoutException INFO [2022-01-10 05:45:23,626] jepsen worker 43 - jepsen.util 3373 :invoke :write [14 26] INFO [2022-01-10 05:45:23,914] jepsen worker 50 - jepsen.util 3470 :info :write [16 19] :RequestTimeoutException INFO [2022-01-10 05:45:23,915] jepsen worker 50 - jepsen.util 3560 :invoke :write [16 5] INFO [2022-01-10 05:45:23,932] jepsen worker 86 - jepsen.util 2786 :info :write [28 27] :RequestTimeoutException INFO [2022-01-10 05:45:23,933] jepsen worker 86 - jepsen.util 2876 :invoke :write [28 8] INFO [2022-01-10 05:45:24,029] jepsen worker 75 - jepsen.util 2955 :info :write [25 43] :RequestTimeoutException INFO [2022-01-10 05:45:24,030] jepsen worker 75 - jepsen.util 3045 :invoke :read [25 nil] INFO [2022-01-10 05:45:24,095] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:24,095] jepsen worker nemesis - couchbase.util Rebalance check count: 474 INFO [2022-01-10 05:45:24,095] jepsen worker nemesis - couchbase.util Rebalance stuck count: 387 INFO [2022-01-10 05:45:24,182] jepsen worker 53 - jepsen.util 4103 :info :write [17 25] :RequestTimeoutException INFO [2022-01-10 05:45:24,183] jepsen worker 53 - jepsen.util 4193 :invoke :write [17 10] INFO [2022-01-10 05:45:24,331] jepsen worker 13 - jepsen.util 3883 :info :write [4 28] :RequestTimeoutException INFO [2022-01-10 05:45:24,332] jepsen worker 13 - jepsen.util 3973 :invoke :read [4 nil] INFO [2022-01-10 05:45:24,655] jepsen worker 7 - jepsen.util 3517 :info :write [2 48] :RequestTimeoutException INFO [2022-01-10 05:45:24,655] jepsen worker 7 - jepsen.util 3607 :invoke :read [2 nil] INFO [2022-01-10 05:45:24,725] jepsen worker 88 - jepsen.util 3418 :info :write [29 11] :RequestTimeoutException INFO [2022-01-10 05:45:24,726] jepsen worker 88 - jepsen.util 3508 :invoke :read [29 nil] INFO [2022-01-10 05:45:24,781] jepsen worker 41 - jepsen.util 3281 :ok :read [13 15] INFO [2022-01-10 05:45:24,782] jepsen worker 41 - jepsen.util 3281 :invoke :read [13 nil] INFO [2022-01-10 05:45:25,086] jepsen worker 41 - jepsen.util 3281 :ok :read [13 15] INFO [2022-01-10 05:45:25,087] jepsen worker 41 - jepsen.util 3281 :invoke :read [13 nil] INFO [2022-01-10 05:45:25,100] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:25,100] jepsen worker nemesis - couchbase.util Rebalance check count: 475 INFO [2022-01-10 05:45:25,100] jepsen worker nemesis - couchbase.util Rebalance stuck count: 388 INFO [2022-01-10 05:45:25,112] jepsen worker 40 - jepsen.util 2740 :info :write [13 21] :RequestTimeoutException INFO [2022-01-10 05:45:25,113] jepsen worker 40 - jepsen.util 2830 :invoke :read [13 nil] INFO [2022-01-10 05:45:25,246] jepsen worker 27 - jepsen.util 3447 :info :write [9 22] :RequestTimeoutException INFO [2022-01-10 05:45:25,247] jepsen worker 27 - jepsen.util 3537 :invoke :write [9 46] INFO [2022-01-10 05:45:25,341] jepsen worker 41 - jepsen.util 3281 :ok :read [13 15] INFO [2022-01-10 05:45:25,342] jepsen worker 41 - jepsen.util 3281 :invoke :write [13 10] INFO [2022-01-10 05:45:25,348] jepsen worker 40 - jepsen.util 2830 :ok :read [13 15] INFO [2022-01-10 05:45:25,348] jepsen worker 40 - jepsen.util 2830 :invoke :write [13 44] INFO [2022-01-10 05:45:25,351] jepsen worker 52 - jepsen.util 4282 :info :write [17 41] :RequestTimeoutException INFO [2022-01-10 05:45:25,352] jepsen worker 52 - jepsen.util 4372 :invoke :read [17 nil] INFO [2022-01-10 05:45:25,354] jepsen worker 0 - jepsen.util 3240 :ok :read [0 46] INFO [2022-01-10 05:45:25,354] jepsen worker 0 - jepsen.util 3240 :invoke :read [0 nil] INFO [2022-01-10 05:45:25,546] jepsen worker 60 - jepsen.util 3660 :info :write [20 6] :RequestTimeoutException INFO [2022-01-10 05:45:25,547] jepsen worker 60 - jepsen.util 3750 :invoke :write [20 2] INFO [2022-01-10 05:45:25,631] jepsen worker 13 - jepsen.util 3973 :ok :read [4 3] INFO [2022-01-10 05:45:25,631] jepsen worker 13 - jepsen.util 3973 :invoke :read [4 nil] INFO [2022-01-10 05:45:25,727] jepsen worker 76 - jepsen.util 2956 :ok :read [25 :nil] INFO [2022-01-10 05:45:25,728] jepsen worker 76 - jepsen.util 2956 :invoke :read [25 nil] INFO [2022-01-10 05:45:25,747] jepsen worker 20 - jepsen.util 3170 :ok :read [6 25] INFO [2022-01-10 05:45:25,747] jepsen worker 20 - jepsen.util 3170 :invoke :write [6 31] INFO [2022-01-10 05:45:25,935] jepsen worker 2 - jepsen.util 3692 :info :write [0 9] :RequestTimeoutException INFO [2022-01-10 05:45:25,935] jepsen worker 2 - jepsen.util 3782 :invoke :read [0 nil] INFO [2022-01-10 05:45:25,962] jepsen worker 22 - jepsen.util 3172 :ok :read [7 :nil] INFO [2022-01-10 05:45:25,964] jepsen worker 22 - jepsen.util 3172 :invoke :read [7 nil] INFO [2022-01-10 05:45:25,976] jepsen worker 74 - jepsen.util 3944 :ok :read [24 18] INFO [2022-01-10 05:45:25,977] jepsen worker 74 - jepsen.util 3944 :invoke :read [24 nil] INFO [2022-01-10 05:45:25,982] jepsen worker 73 - jepsen.util 3133 :ok :read [24 18] INFO [2022-01-10 05:45:25,983] jepsen worker 73 - jepsen.util 3133 :invoke :write [24 32] INFO [2022-01-10 05:45:26,092] jepsen worker 14 - jepsen.util 4154 :info :write [4 28] :RequestTimeoutException INFO [2022-01-10 05:45:26,092] jepsen worker 14 - jepsen.util 4244 :invoke :read [4 nil] INFO [2022-01-10 05:45:26,105] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:26,105] jepsen worker nemesis - couchbase.util Rebalance check count: 476 INFO [2022-01-10 05:45:26,105] jepsen worker nemesis - couchbase.util Rebalance stuck count: 389 INFO [2022-01-10 05:45:26,168] jepsen worker 64 - jepsen.util 3394 :ok :read [21 7] INFO [2022-01-10 05:45:26,169] jepsen worker 64 - jepsen.util 3394 :invoke :write [21 2] INFO [2022-01-10 05:45:26,178] jepsen worker 18 - jepsen.util 3348 :ok :read [6 25] INFO [2022-01-10 05:45:26,179] jepsen worker 18 - jepsen.util 3348 :invoke :read [6 nil] INFO [2022-01-10 05:45:26,189] jepsen worker 24 - jepsen.util 2994 :info :write [8 16] :RequestTimeoutException INFO [2022-01-10 05:45:26,190] jepsen worker 24 - jepsen.util 3084 :invoke :write [8 11] INFO [2022-01-10 05:45:26,211] jepsen worker 16 - jepsen.util 3436 :info :write [5 43] :RequestTimeoutException INFO [2022-01-10 05:45:26,212] jepsen worker 16 - jepsen.util 3526 :invoke :read [5 nil] INFO [2022-01-10 05:45:26,403] jepsen worker 80 - jepsen.util 4220 :ok :read [26 10] INFO [2022-01-10 05:45:26,404] jepsen worker 80 - jepsen.util 4220 :invoke :write [26 48] INFO [2022-01-10 05:45:26,404] jepsen worker 23 - jepsen.util 3443 :info :write [7 23] :RequestTimeoutException INFO [2022-01-10 05:45:26,405] jepsen worker 23 - jepsen.util 3533 :invoke :write [7 45] INFO [2022-01-10 05:45:26,414] jepsen worker 72 - jepsen.util 3762 :info :write [24 14] :RequestTimeoutException INFO [2022-01-10 05:45:26,415] jepsen worker 72 - jepsen.util 3852 :invoke :read [24 nil] INFO [2022-01-10 05:45:26,549] jepsen worker 44 - jepsen.util 3464 :info :write [14 27] :RequestTimeoutException INFO [2022-01-10 05:45:26,550] jepsen worker 44 - jepsen.util 3554 :invoke :read [14 nil] INFO [2022-01-10 05:45:26,577] jepsen worker 61 - jepsen.util 3571 :ok :read [20 :nil] INFO [2022-01-10 05:45:26,577] jepsen worker 61 - jepsen.util 3571 :invoke :write [20 25] INFO [2022-01-10 05:45:26,593] jepsen worker 34 - jepsen.util 3364 :ok :read [11 21] INFO [2022-01-10 05:45:26,594] jepsen worker 34 - jepsen.util 3364 :invoke :read [11 nil] INFO [2022-01-10 05:45:26,653] jepsen worker 52 - jepsen.util 4372 :ok :read [17 41] INFO [2022-01-10 05:45:26,654] jepsen worker 52 - jepsen.util 4372 :invoke :write [17 43] INFO [2022-01-10 05:45:26,755] jepsen worker 0 - jepsen.util 3240 :ok :read [0 46] INFO [2022-01-10 05:45:26,755] jepsen worker 0 - jepsen.util 3240 :invoke :write [0 40] INFO [2022-01-10 05:45:26,856] jepsen worker 1 - jepsen.util 3421 :info :write [0 37] :RequestTimeoutException INFO [2022-01-10 05:45:26,857] jepsen worker 1 - jepsen.util 3511 :invoke :write [0 7] INFO [2022-01-10 05:45:26,917] jepsen worker 2 - jepsen.util 3782 :ok :read [0 46] INFO [2022-01-10 05:45:26,918] jepsen worker 2 - jepsen.util 3782 :invoke :read [0 nil] INFO [2022-01-10 05:45:26,991] jepsen worker 2 - jepsen.util 3782 :ok :read [0 46] INFO [2022-01-10 05:45:26,992] jepsen worker 2 - jepsen.util 3782 :invoke :write [0 24] INFO [2022-01-10 05:45:27,111] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:27,111] jepsen worker nemesis - couchbase.util Rebalance check count: 477 INFO [2022-01-10 05:45:27,111] jepsen worker nemesis - couchbase.util Rebalance stuck count: 390 INFO [2022-01-10 05:45:27,141] jepsen worker 12 - jepsen.util 3792 :info :write [4 16] :RequestTimeoutException INFO [2022-01-10 05:45:27,141] jepsen worker 12 - jepsen.util 3882 :invoke :read [4 nil] INFO [2022-01-10 05:45:27,223] jepsen worker 12 - jepsen.util 3882 :ok :read [4 3] INFO [2022-01-10 05:45:27,224] jepsen worker 12 - jepsen.util 3882 :invoke :read [4 nil] INFO [2022-01-10 05:45:27,245] jepsen worker 51 - jepsen.util 4101 :info :write [17 45] :RequestTimeoutException INFO [2022-01-10 05:45:27,245] jepsen worker 51 - jepsen.util 4191 :invoke :read [17 nil] INFO [2022-01-10 05:45:27,351] jepsen worker 87 - jepsen.util 3507 :ok :read [29 0] INFO [2022-01-10 05:45:27,351] jepsen worker 87 - jepsen.util 3507 :invoke :read [29 nil] INFO [2022-01-10 05:45:27,361] jepsen worker 83 - jepsen.util 3863 :ok :read [27 30] INFO [2022-01-10 05:45:27,362] jepsen worker 83 - jepsen.util 3863 :invoke :write [27 4] INFO [2022-01-10 05:45:27,390] jepsen worker 11 - jepsen.util 3431 :ok :read [3 31] INFO [2022-01-10 05:45:27,391] jepsen worker 11 - jepsen.util 3431 :invoke :read [3 nil] INFO [2022-01-10 05:45:27,427] jepsen worker 15 - jepsen.util 2985 :ok :read [5 42] INFO [2022-01-10 05:45:27,427] jepsen worker 15 - jepsen.util 2985 :invoke :write [5 36] INFO [2022-01-10 05:45:27,442] jepsen worker 13 - jepsen.util 3973 :ok :read [4 3] INFO [2022-01-10 05:45:27,442] jepsen worker 13 - jepsen.util 3973 :invoke :read [4 nil] INFO [2022-01-10 05:45:27,444] jepsen worker 14 - jepsen.util 4244 :ok :read [4 3] INFO [2022-01-10 05:45:27,445] jepsen worker 14 - jepsen.util 4244 :invoke :read [4 nil] INFO [2022-01-10 05:45:27,490] jepsen worker 29 - jepsen.util 3359 :ok :read [9 27] INFO [2022-01-10 05:45:27,491] jepsen worker 29 - jepsen.util 3359 :invoke :write [9 26] INFO [2022-01-10 05:45:27,526] jepsen worker 62 - jepsen.util 3842 :info :write [20 15] :RequestTimeoutException INFO [2022-01-10 05:45:27,527] jepsen worker 62 - jepsen.util 3932 :invoke :read [20 nil] INFO [2022-01-10 05:45:27,670] jepsen worker 69 - jepsen.util 2949 :ok :read [23 26] INFO [2022-01-10 05:45:27,671] jepsen worker 69 - jepsen.util 2949 :invoke :read [23 nil] INFO [2022-01-10 05:45:27,691] jepsen worker 55 - jepsen.util 3745 :ok :read [18 14] INFO [2022-01-10 05:45:27,692] jepsen worker 55 - jepsen.util 3745 :invoke :write [18 27] INFO [2022-01-10 05:45:27,716] jepsen worker 67 - jepsen.util 3307 :ok :read [22 41] INFO [2022-01-10 05:45:27,717] jepsen worker 67 - jepsen.util 3307 :invoke :read [22 nil] INFO [2022-01-10 05:45:27,718] jepsen worker 9 - jepsen.util 3969 :ok :read [3 31] INFO [2022-01-10 05:45:27,718] jepsen worker 9 - jepsen.util 3969 :invoke :write [3 35] INFO [2022-01-10 05:45:27,728] jepsen worker 69 - jepsen.util 2949 :ok :read [23 26] INFO [2022-01-10 05:45:27,729] jepsen worker 69 - jepsen.util 2949 :invoke :read [23 nil] INFO [2022-01-10 05:45:27,752] jepsen worker 84 - jepsen.util 3414 :ok :read [28 45] INFO [2022-01-10 05:45:27,753] jepsen worker 84 - jepsen.util 3414 :invoke :read [28 nil] INFO [2022-01-10 05:45:27,804] jepsen worker 69 - jepsen.util 2949 :ok :read [23 26] INFO [2022-01-10 05:45:27,804] jepsen worker 69 - jepsen.util 2949 :invoke :write [23 41] INFO [2022-01-10 05:45:27,952] jepsen worker 59 - jepsen.util 3389 :info :write [19 44] :RequestTimeoutException INFO [2022-01-10 05:45:27,953] jepsen worker 59 - jepsen.util 3479 :invoke :read [19 nil] INFO [2022-01-10 05:45:27,956] jepsen worker 51 - jepsen.util 4191 :ok :read [17 41] INFO [2022-01-10 05:45:27,971] jepsen worker 51 - jepsen.util 4191 :invoke :read [17 nil] INFO [2022-01-10 05:45:28,097] jepsen worker 49 - jepsen.util 3289 :info :write [16 30] :RequestTimeoutException INFO [2022-01-10 05:45:28,098] jepsen worker 49 - jepsen.util 3379 :invoke :write [16 37] INFO [2022-01-10 05:45:28,116] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:28,116] jepsen worker nemesis - couchbase.util Rebalance check count: 478 INFO [2022-01-10 05:45:28,116] jepsen worker nemesis - couchbase.util Rebalance stuck count: 391 INFO [2022-01-10 05:45:28,121] jepsen worker 42 - jepsen.util 3372 :info :write [14 3] :RequestTimeoutException INFO [2022-01-10 05:45:28,122] jepsen worker 42 - jepsen.util 3462 :invoke :read [14 nil] INFO [2022-01-10 05:45:28,149] jepsen worker 51 - jepsen.util 4191 :ok :read [17 41] INFO [2022-01-10 05:45:28,150] jepsen worker 51 - jepsen.util 4191 :invoke :read [17 nil] INFO [2022-01-10 05:45:28,160] jepsen worker 14 - jepsen.util 4244 :ok :read [4 3] INFO [2022-01-10 05:45:28,161] jepsen worker 14 - jepsen.util 4244 :invoke :write [4 33] INFO [2022-01-10 05:45:28,217] jepsen worker 18 - jepsen.util 3348 :ok :read [6 25] INFO [2022-01-10 05:45:28,218] jepsen worker 18 - jepsen.util 3348 :invoke :read [6 nil] INFO [2022-01-10 05:45:28,226] jepsen worker 76 - jepsen.util 2956 :ok :read [25 :nil] INFO [2022-01-10 05:45:28,227] jepsen worker 76 - jepsen.util 2956 :invoke :write [25 35] INFO [2022-01-10 05:45:28,234] jepsen worker 34 - jepsen.util 3364 :ok :read [11 21] INFO [2022-01-10 05:45:28,235] jepsen worker 34 - jepsen.util 3364 :invoke :read [11 nil] INFO [2022-01-10 05:45:28,238] jepsen worker 7 - jepsen.util 3607 :ok :read [2 14] INFO [2022-01-10 05:45:28,239] jepsen worker 7 - jepsen.util 3607 :invoke :write [2 25] INFO [2022-01-10 05:45:28,261] jepsen worker 16 - jepsen.util 3526 :ok :read [5 42] INFO [2022-01-10 05:45:28,262] jepsen worker 16 - jepsen.util 3526 :invoke :write [5 48] INFO [2022-01-10 05:45:28,268] jepsen worker 67 - jepsen.util 3307 :ok :read [22 41] INFO [2022-01-10 05:45:28,268] jepsen worker 67 - jepsen.util 3307 :invoke :write [22 27] INFO [2022-01-10 05:45:28,306] jepsen worker 56 - jepsen.util 3836 :info :write [18 16] :RequestTimeoutException INFO [2022-01-10 05:45:28,307] jepsen worker 56 - jepsen.util 3926 :invoke :read [18 nil] INFO [2022-01-10 05:45:28,392] jepsen worker 84 - jepsen.util 3414 :ok :read [28 45] INFO [2022-01-10 05:45:28,392] jepsen worker 84 - jepsen.util 3414 :invoke :write [28 45] INFO [2022-01-10 05:45:28,555] jepsen worker 12 - jepsen.util 3882 :ok :read [4 3] INFO [2022-01-10 05:45:28,555] jepsen worker 12 - jepsen.util 3882 :invoke :write [4 26] INFO [2022-01-10 05:45:28,650] jepsen worker 44 - jepsen.util 3554 :ok :read [14 14] INFO [2022-01-10 05:45:28,651] jepsen worker 44 - jepsen.util 3554 :invoke :write [14 24] INFO [2022-01-10 05:45:28,746] jepsen worker 58 - jepsen.util 3298 :info :write [19 7] :RequestTimeoutException INFO [2022-01-10 05:45:28,747] jepsen worker 58 - jepsen.util 3388 :invoke :read [19 nil] INFO [2022-01-10 05:45:28,750] jepsen worker 66 - jepsen.util 3126 :info :write [22 0] :RequestTimeoutException INFO [2022-01-10 05:45:28,750] jepsen worker 66 - jepsen.util 3216 :invoke :write [22 19] INFO [2022-01-10 05:45:28,777] jepsen worker 13 - jepsen.util 3973 :ok :read [4 3] INFO [2022-01-10 05:45:28,778] jepsen worker 13 - jepsen.util 3973 :invoke :write [4 18] INFO [2022-01-10 05:45:28,820] jepsen worker 37 - jepsen.util 3187 :ok :read [12 29] INFO [2022-01-10 05:45:28,820] jepsen worker 37 - jepsen.util 3187 :invoke :write [12 9] INFO [2022-01-10 05:45:28,837] jepsen worker 32 - jepsen.util 3722 :ok :read [10 36] INFO [2022-01-10 05:45:28,838] jepsen worker 32 - jepsen.util 3722 :invoke :read [10 nil] INFO [2022-01-10 05:45:28,906] jepsen worker 88 - jepsen.util 3508 :ok :read [29 0] INFO [2022-01-10 05:45:28,907] jepsen worker 88 - jepsen.util 3508 :invoke :write [29 0] INFO [2022-01-10 05:45:28,922] jepsen worker 32 - jepsen.util 3722 :ok :read [10 36] INFO [2022-01-10 05:45:28,923] jepsen worker 32 - jepsen.util 3722 :invoke :read [10 nil] INFO [2022-01-10 05:45:28,926] jepsen worker 39 - jepsen.util 3999 :info :write [13 42] :RequestTimeoutException INFO [2022-01-10 05:45:28,927] jepsen worker 39 - jepsen.util 4089 :invoke :read [13 nil] INFO [2022-01-10 05:45:28,989] jepsen worker 51 - jepsen.util 4191 :ok :read [17 41] INFO [2022-01-10 05:45:28,989] jepsen worker 51 - jepsen.util 4191 :invoke :read [17 nil] INFO [2022-01-10 05:45:29,045] jepsen worker 32 - jepsen.util 3722 :ok :read [10 36] INFO [2022-01-10 05:45:29,046] jepsen worker 32 - jepsen.util 3722 :invoke :read [10 nil] INFO [2022-01-10 05:45:29,122] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:29,122] jepsen worker nemesis - couchbase.util Rebalance check count: 479 INFO [2022-01-10 05:45:29,122] jepsen worker nemesis - couchbase.util Rebalance stuck count: 392 INFO [2022-01-10 05:45:29,167] jepsen worker 32 - jepsen.util 3722 :ok :read [10 36] INFO [2022-01-10 05:45:29,168] jepsen worker 32 - jepsen.util 3722 :invoke :write [10 42] INFO [2022-01-10 05:45:29,190] jepsen worker 8 - jepsen.util 3878 :info :write [2 10] :RequestTimeoutException INFO [2022-01-10 05:45:29,191] jepsen worker 8 - jepsen.util 3968 :invoke :read [2 nil] INFO [2022-01-10 05:45:29,310] jepsen worker 57 - jepsen.util 4107 :info :write [19 48] :RequestTimeoutException INFO [2022-01-10 05:45:29,310] jepsen worker 57 - jepsen.util 4197 :invoke :write [19 49] INFO [2022-01-10 05:45:29,320] jepsen worker 81 - jepsen.util 3681 :info :write [27 15] :RequestTimeoutException INFO [2022-01-10 05:45:29,321] jepsen worker 81 - jepsen.util 3771 :invoke :write [27 3] INFO [2022-01-10 05:45:29,622] jepsen worker 51 - jepsen.util 4191 :ok :read [17 41] INFO [2022-01-10 05:45:29,624] jepsen worker 51 - jepsen.util 4191 :invoke :read [17 nil] INFO [2022-01-10 05:45:29,720] jepsen worker 89 - jepsen.util 3779 :info :write [29 6] :SyncWriteAmbiguous INFO [2022-01-10 05:45:29,722] jepsen worker 89 - jepsen.util 3869 :invoke :write [29 25] INFO [2022-01-10 05:45:29,900] jepsen worker 85 - jepsen.util 3775 :info :write [28 0] :RequestTimeoutException INFO [2022-01-10 05:45:29,901] jepsen worker 85 - jepsen.util 3865 :invoke :write [28 9] INFO [2022-01-10 05:45:30,051] jepsen worker 58 - jepsen.util 3388 :ok :read [19 31] INFO [2022-01-10 05:45:30,052] jepsen worker 58 - jepsen.util 3388 :invoke :write [19 49] INFO [2022-01-10 05:45:30,128] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:30,128] jepsen worker nemesis - couchbase.util Rebalance check count: 480 INFO [2022-01-10 05:45:30,128] jepsen worker nemesis - couchbase.util Rebalance stuck count: 393 INFO [2022-01-10 05:45:30,203] jepsen worker 31 - jepsen.util 3811 :info :write [10 48] :RequestTimeoutException INFO [2022-01-10 05:45:30,203] jepsen worker 31 - jepsen.util 3901 :invoke :write [10 40] INFO [2022-01-10 05:45:30,292] jepsen worker 38 - jepsen.util 3548 :info :write [12 11] :RequestTimeoutException INFO [2022-01-10 05:45:30,293] jepsen worker 38 - jepsen.util 3638 :invoke :read [12 nil] WARN [2022-01-10 05:45:30,324] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11141560,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12861353,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9721959,"encode_duration_us":16781,"operation_id":"0x6a66","timeout_ms":10000,"last_server_duration_us":9533354,"total_duration_us":10004815},{"operation_name":"upsert","last_dispatch_duration_us":17530630,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18674797,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9316632,"encode_duration_us":20524,"operation_id":"0x68f4","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10010309},{"operation_name":"upsert","last_dispatch_duration_us":17459192,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18619991,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9314518,"encode_duration_us":17017,"operation_id":"0x68f7","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10009955}],"total_count":3}} WARN [2022-01-10 05:45:30,424] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1375912,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8822886,"total_server_duration_us":1776026,"encode_duration_us":18582,"operation_id":"0x6b1e","timeout_ms":10000,"last_server_duration_us":379006,"total_duration_us":10010655},{"operation_name":"upsert","last_dispatch_duration_us":10179801,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10179801,"total_server_duration_us":291809,"encode_duration_us":16469,"operation_id":"0x6b3f","timeout_ms":10000,"last_server_duration_us":291809,"total_duration_us":10009524},{"operation_name":"upsert","last_dispatch_duration_us":11005731,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12366622,"total_server_duration_us":641752,"encode_duration_us":18476,"operation_id":"0x6a9d","timeout_ms":10000,"last_server_duration_us":251659,"total_duration_us":10008947},{"operation_name":"upsert","last_dispatch_duration_us":669707,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":4145885,"total_server_duration_us":926753,"encode_duration_us":16627,"operation_id":"0x6b4c","timeout_ms":10000,"last_server_duration_us":64993,"total_duration_us":10006907},{"operation_name":"upsert","last_dispatch_duration_us":2274160,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7612151,"total_server_duration_us":1455428,"encode_duration_us":17467,"operation_id":"0x6ba3","timeout_ms":10000,"last_server_duration_us":406977,"total_duration_us":10006136},{"operation_name":"upsert","last_dispatch_duration_us":10209764,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12134055,"total_server_duration_us":1096007,"encode_duration_us":19957,"operation_id":"0x6aac","timeout_ms":10000,"last_server_duration_us":152740,"total_duration_us":10004687},{"operation_name":"upsert","last_dispatch_duration_us":1203437,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":10371706,"total_server_duration_us":2659569,"encode_duration_us":17343,"operation_id":"0x6aeb","timeout_ms":10000,"last_server_duration_us":599221,"total_duration_us":10003023},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":17332,"operation_id":"0x6a7a","timeout_ms":10000,"total_duration_us":10002849},{"operation_name":"upsert","last_dispatch_duration_us":1042269,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6121716,"total_server_duration_us":2474767,"encode_duration_us":18099,"operation_id":"0x6ba4","timeout_ms":10000,"last_server_duration_us":282507,"total_duration_us":10002656},{"operation_name":"upsert","last_dispatch_duration_us":1001130,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":2227853,"total_server_duration_us":249281,"encode_duration_us":20083,"operation_id":"0x6b96","timeout_ms":10000,"last_server_duration_us":144416,"total_duration_us":10002539}],"total_count":16}} INFO [2022-01-10 05:45:30,569] jepsen worker 18 - jepsen.util 3348 :ok :read [6 25] INFO [2022-01-10 05:45:30,569] jepsen worker 18 - jepsen.util 3348 :invoke :read [6 nil] INFO [2022-01-10 05:45:30,575] jepsen worker 71 - jepsen.util 2951 :info :write [23 42] :RequestTimeoutException INFO [2022-01-10 05:45:30,575] jepsen worker 34 - jepsen.util 3364 :ok :read [11 21] INFO [2022-01-10 05:45:30,575] jepsen worker 71 - jepsen.util 3041 :invoke :read [23 nil] INFO [2022-01-10 05:45:30,575] jepsen worker 34 - jepsen.util 3364 :invoke :read [11 nil] INFO [2022-01-10 05:45:30,596] jepsen worker 68 - jepsen.util 3488 :info :write [22 21] :RequestTimeoutException INFO [2022-01-10 05:45:30,597] jepsen worker 68 - jepsen.util 3578 :invoke :read [22 nil] INFO [2022-01-10 05:45:30,658] jepsen worker 51 - jepsen.util 4191 :ok :read [17 41] INFO [2022-01-10 05:45:30,658] jepsen worker 51 - jepsen.util 4191 :invoke :read [17 nil] INFO [2022-01-10 05:45:30,700] jepsen worker 4 - jepsen.util 3964 :info :write [1 20] :RequestTimeoutException INFO [2022-01-10 05:45:30,701] jepsen worker 4 - jepsen.util 4054 :invoke :write [1 17] INFO [2022-01-10 05:45:30,820] jepsen worker 47 - jepsen.util 3467 :info :write [15 22] :RequestTimeoutException INFO [2022-01-10 05:45:30,821] jepsen worker 47 - jepsen.util 3557 :invoke :write [15 36] INFO [2022-01-10 05:45:30,832] jepsen worker 70 - jepsen.util 3490 :info :write [23 33] :RequestTimeoutException INFO [2022-01-10 05:45:30,832] jepsen worker 70 - jepsen.util 3580 :invoke :read [23 nil] INFO [2022-01-10 05:45:30,856] jepsen worker 70 - jepsen.util 3580 :ok :read [23 26] INFO [2022-01-10 05:45:30,857] jepsen worker 70 - jepsen.util 3580 :invoke :read [23 nil] INFO [2022-01-10 05:45:30,878] jepsen worker 70 - jepsen.util 3580 :ok :read [23 26] INFO [2022-01-10 05:45:30,878] jepsen worker 70 - jepsen.util 3580 :invoke :write [23 36] INFO [2022-01-10 05:45:30,895] jepsen worker 28 - jepsen.util 3178 :info :write [9 45] :RequestTimeoutException INFO [2022-01-10 05:45:30,895] jepsen worker 28 - jepsen.util 3268 :invoke :write [9 13] INFO [2022-01-10 05:45:31,045] jepsen worker 46 - jepsen.util 3556 :info :write [15 0] :RequestTimeoutException INFO [2022-01-10 05:45:31,046] jepsen worker 46 - jepsen.util 3646 :invoke :read [15 nil] INFO [2022-01-10 05:45:31,104] jepsen worker 19 - jepsen.util 3529 :info :write [6 23] :RequestTimeoutException INFO [2022-01-10 05:45:31,105] jepsen worker 19 - jepsen.util 3619 :invoke :write [6 46] INFO [2022-01-10 05:45:31,134] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:31,134] jepsen worker nemesis - couchbase.util Rebalance check count: 481 INFO [2022-01-10 05:45:31,134] jepsen worker nemesis - couchbase.util Rebalance stuck count: 394 INFO [2022-01-10 05:45:31,192] jepsen worker 75 - jepsen.util 3045 :ok :read [25 :nil] INFO [2022-01-10 05:45:31,192] jepsen worker 22 - jepsen.util 3172 :ok :read [7 :nil] INFO [2022-01-10 05:45:31,194] jepsen worker 22 - jepsen.util 3172 :invoke :write [7 46] INFO [2022-01-10 05:45:31,195] jepsen worker 75 - jepsen.util 3045 :invoke :write [25 21] INFO [2022-01-10 05:45:31,282] jepsen worker 48 - jepsen.util 3018 :info :write [16 20] :RequestTimeoutException INFO [2022-01-10 05:45:31,284] jepsen worker 48 - jepsen.util 3108 :invoke :write [16 7] INFO [2022-01-10 05:45:31,317] jepsen worker 77 - jepsen.util 3767 :info :write [25 44] :RequestTimeoutException INFO [2022-01-10 05:45:31,317] jepsen worker 77 - jepsen.util 3857 :invoke :write [25 13] INFO [2022-01-10 05:45:31,331] jepsen worker 35 - jepsen.util 3095 :info :write [11 36] :RequestTimeoutException INFO [2022-01-10 05:45:31,331] jepsen worker 35 - jepsen.util 3185 :invoke :write [11 33] INFO [2022-01-10 05:45:31,365] jepsen worker 74 - jepsen.util 3944 :ok :read [24 18] INFO [2022-01-10 05:45:31,366] jepsen worker 74 - jepsen.util 3944 :invoke :write [24 37] INFO [2022-01-10 05:45:31,686] jepsen worker 82 - jepsen.util 3592 :info :write [27 35] :RequestTimeoutException INFO [2022-01-10 05:45:31,687] jepsen worker 82 - jepsen.util 3682 :invoke :write [27 34] INFO [2022-01-10 05:45:31,695] jepsen worker 51 - jepsen.util 4191 :ok :read [17 41] INFO [2022-01-10 05:45:31,696] jepsen worker 51 - jepsen.util 4191 :invoke :write [17 48] INFO [2022-01-10 05:45:31,872] jepsen worker 21 - jepsen.util 3621 :info :write [7 30] :RequestTimeoutException INFO [2022-01-10 05:45:31,874] jepsen worker 21 - jepsen.util 3711 :invoke :read [7 nil] INFO [2022-01-10 05:45:31,881] jepsen worker 30 - jepsen.util 3180 :ok :read [10 36] INFO [2022-01-10 05:45:31,882] jepsen worker 30 - jepsen.util 3180 :invoke :write [10 24] INFO [2022-01-10 05:45:31,890] jepsen worker 36 - jepsen.util 2646 :ok :read [12 29] INFO [2022-01-10 05:45:31,890] jepsen worker 36 - jepsen.util 2646 :invoke :write [12 14] INFO [2022-01-10 05:45:31,906] jepsen worker 25 - jepsen.util 3265 :info :write [8 12] :RequestTimeoutException INFO [2022-01-10 05:45:31,907] jepsen worker 25 - jepsen.util 3355 :invoke :write [8 44] WARN [2022-01-10 05:45:31,926] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":24772555,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":27096363,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9971020,"encode_duration_us":16377,"operation_id":"0x6895","timeout_ms":10000,"last_server_duration_us":9423962,"total_duration_us":10002869},{"operation_name":"upsert","last_dispatch_duration_us":17958284,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17958284,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9020392,"encode_duration_us":17105,"operation_id":"0x6a46","timeout_ms":10000,"last_server_duration_us":9020392,"total_duration_us":10005717}],"total_count":2}} INFO [2022-01-10 05:45:31,951] jepsen worker 10 - jepsen.util 3520 :info :write [3 0] :RequestTimeoutException INFO [2022-01-10 05:45:31,953] jepsen worker 10 - jepsen.util 3610 :invoke :write [3 27] WARN [2022-01-10 05:45:32,026] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17468154,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":24235100,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10170784,"encode_duration_us":21272,"operation_id":"0x688f","timeout_ms":10000,"last_server_duration_us":9080926,"total_duration_us":10004008},{"operation_name":"upsert","last_dispatch_duration_us":17882030,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18093861,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9067183,"encode_duration_us":23108,"operation_id":"0x6a77","timeout_ms":10000,"last_server_duration_us":9045859,"total_duration_us":10005343}],"total_count":2}} WARN [2022-01-10 05:45:32,026] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16825499,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":16825499,"total_server_duration_us":32,"operation_id":"0x6adb","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":16826666},{"operation_name":"get","last_dispatch_duration_us":15234417,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":15234417,"total_server_duration_us":15,"operation_id":"0x6b1b","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":15235805},{"operation_name":"upsert","last_dispatch_duration_us":3173313,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9359366,"total_server_duration_us":4313865,"encode_duration_us":16565,"operation_id":"0x6b04","timeout_ms":10000,"last_server_duration_us":783501,"total_duration_us":10010598},{"operation_name":"upsert","last_dispatch_duration_us":3515715,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10800287,"total_server_duration_us":4460843,"encode_duration_us":16636,"operation_id":"0x6b4b","timeout_ms":10000,"last_server_duration_us":1049613,"total_duration_us":10008936},{"operation_name":"upsert","last_dispatch_duration_us":6777575,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":12167931,"total_server_duration_us":1582325,"encode_duration_us":18740,"operation_id":"0x6bf4","timeout_ms":10000,"last_server_duration_us":411531,"total_duration_us":10006704},{"operation_name":"upsert","last_dispatch_duration_us":4480013,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":6038731,"total_server_duration_us":698540,"encode_duration_us":16523,"operation_id":"0x6c97","timeout_ms":10000,"last_server_duration_us":592876,"total_duration_us":10006554},{"operation_name":"upsert","last_dispatch_duration_us":3108442,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10216948,"total_server_duration_us":2010871,"encode_duration_us":16290,"operation_id":"0x6b76","timeout_ms":10000,"last_server_duration_us":376256,"total_duration_us":10004454},{"operation_name":"upsert","last_dispatch_duration_us":3267659,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10236648,"total_server_duration_us":2383712,"encode_duration_us":16988,"operation_id":"0x6bcf","timeout_ms":10000,"last_server_duration_us":1001659,"total_duration_us":10002486},{"operation_name":"upsert","last_dispatch_duration_us":5245868,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10345934,"total_server_duration_us":1731773,"encode_duration_us":16864,"operation_id":"0x6c87","timeout_ms":10000,"last_server_duration_us":22389,"total_duration_us":10000884},{"operation_name":"get","last_dispatch_duration_us":8080997,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":8080997,"total_server_duration_us":30256,"operation_id":"0x6c28","timeout_ms":30000,"last_server_duration_us":30256,"total_duration_us":8085884}],"total_count":16}} INFO [2022-01-10 05:45:32,064] jepsen worker 45 - jepsen.util 3735 :info :write [15 4] :RequestTimeoutException INFO [2022-01-10 05:45:32,065] jepsen worker 45 - jepsen.util 3825 :invoke :read [15 nil] INFO [2022-01-10 05:45:32,069] jepsen worker 65 - jepsen.util 3215 :info :write [21 4] :RequestTimeoutException INFO [2022-01-10 05:45:32,070] jepsen worker 65 - jepsen.util 3305 :invoke :write [21 0] WARN [2022-01-10 05:45:32,077] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":24029078,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":24029078,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9030999,"encode_duration_us":22863,"operation_id":"0x6933","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10003560}],"total_count":1}} WARN [2022-01-10 05:45:32,124] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":957221,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":9476711,"total_server_duration_us":3892963,"encode_duration_us":16320,"operation_id":"0x6b49","timeout_ms":10000,"last_server_duration_us":237033,"total_duration_us":10009056},{"operation_name":"upsert","last_dispatch_duration_us":609331,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9032653,"total_server_duration_us":905595,"encode_duration_us":17544,"operation_id":"0x6ba1","timeout_ms":10000,"last_server_duration_us":140059,"total_duration_us":10007949},{"operation_name":"upsert","last_dispatch_duration_us":5918580,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":5918580,"total_server_duration_us":365610,"encode_duration_us":17006,"operation_id":"0x6cb9","timeout_ms":10000,"last_server_duration_us":365610,"total_duration_us":10007849},{"operation_name":"upsert","last_dispatch_duration_us":1018740,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10024693,"total_server_duration_us":1279250,"encode_duration_us":16308,"operation_id":"0x6b6d","timeout_ms":10000,"last_server_duration_us":128717,"total_duration_us":10006339},{"operation_name":"upsert","last_dispatch_duration_us":984024,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":5192972,"total_server_duration_us":1172267,"encode_duration_us":20646,"operation_id":"0x6c94","timeout_ms":10000,"last_server_duration_us":546367,"total_duration_us":10005332},{"operation_name":"upsert","last_dispatch_duration_us":843938,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10703871,"total_server_duration_us":1926055,"encode_duration_us":18453,"operation_id":"0x6b40","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10005084},{"operation_name":"upsert","last_dispatch_duration_us":1865106,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9910224,"total_server_duration_us":2809361,"encode_duration_us":17558,"operation_id":"0x6b18","timeout_ms":10000,"last_server_duration_us":364797,"total_duration_us":10004721},{"operation_name":"upsert","last_dispatch_duration_us":1073345,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":5605950,"total_server_duration_us":1109671,"encode_duration_us":16663,"operation_id":"0x6bf1","timeout_ms":10000,"last_server_duration_us":481486,"total_duration_us":10004208},{"operation_name":"upsert","last_dispatch_duration_us":7822609,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8387953,"total_server_duration_us":247916,"encode_duration_us":17248,"operation_id":"0x6c58","timeout_ms":10000,"last_server_duration_us":20680,"total_duration_us":10003417},{"operation_name":"upsert","last_dispatch_duration_us":7899396,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8426896,"total_server_duration_us":246007,"encode_duration_us":17369,"operation_id":"0x6c55","timeout_ms":10000,"last_server_duration_us":21649,"total_duration_us":10003411}],"total_count":20}} WARN [2022-01-10 05:45:32,126] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11879745,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11879745,"total_server_duration_us":2416,"operation_id":"0x6a1a","timeout_ms":30000,"last_server_duration_us":2416,"total_duration_us":17560683},{"operation_name":"get","last_dispatch_duration_us":15093342,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":15093342,"total_server_duration_us":43,"operation_id":"0x69f0","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":15094521},{"operation_name":"get","last_dispatch_duration_us":11868680,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11868680,"total_server_duration_us":49,"operation_id":"0x6b4a","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":11871260},{"operation_name":"upsert","last_dispatch_duration_us":11014948,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11014948,"total_server_duration_us":379557,"encode_duration_us":20500,"operation_id":"0x6b77","timeout_ms":10000,"last_server_duration_us":379557,"total_duration_us":10010320},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":26646,"operation_id":"0x6b1c","timeout_ms":10000,"total_duration_us":10010195},{"operation_name":"upsert","last_dispatch_duration_us":2040868,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":10857504,"total_server_duration_us":4152537,"encode_duration_us":24453,"operation_id":"0x6b1d","timeout_ms":10000,"last_server_duration_us":622846,"total_duration_us":10009348},{"operation_name":"upsert","last_dispatch_duration_us":178272,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8206030,"total_server_duration_us":659425,"encode_duration_us":17683,"operation_id":"0x6bf3","timeout_ms":10000,"last_server_duration_us":134514,"total_duration_us":10009049},{"operation_name":"upsert","last_dispatch_duration_us":2868826,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8523064,"total_server_duration_us":1590447,"encode_duration_us":16233,"operation_id":"0x6c96","timeout_ms":10000,"last_server_duration_us":1209511,"total_duration_us":10007264},{"operation_name":"upsert","last_dispatch_duration_us":341021,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":4601119,"total_server_duration_us":853960,"encode_duration_us":17109,"operation_id":"0x6c1d","timeout_ms":10000,"last_server_duration_us":137364,"total_duration_us":10007028},{"operation_name":"upsert","last_dispatch_duration_us":1732019,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10416458,"total_server_duration_us":813142,"encode_duration_us":17212,"operation_id":"0x6b75","timeout_ms":10000,"last_server_duration_us":116615,"total_duration_us":10006409}],"total_count":24}} WARN [2022-01-10 05:45:32,127] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18613522,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":18613522,"total_server_duration_us":11614,"operation_id":"0x6a3b","timeout_ms":30000,"last_server_duration_us":11614,"total_duration_us":18615485},{"operation_name":"get","last_dispatch_duration_us":17300120,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":17300120,"total_server_duration_us":14230,"operation_id":"0x6a73","timeout_ms":30000,"last_server_duration_us":14230,"total_duration_us":17302037},{"operation_name":"get","last_dispatch_duration_us":16620664,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":16620664,"total_server_duration_us":32,"operation_id":"0x6a81","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":16621754},{"operation_name":"get","last_dispatch_duration_us":4314577,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4314577,"total_server_duration_us":49,"operation_id":"0x6b41","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":12029145},{"operation_name":"get","last_dispatch_duration_us":11024437,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":11024437,"total_server_duration_us":56,"operation_id":"0x6ba2","timeout_ms":30000,"last_server_duration_us":56,"total_duration_us":11025463},{"operation_name":"upsert","last_dispatch_duration_us":3665678,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11345748,"total_server_duration_us":648217,"encode_duration_us":16401,"operation_id":"0x6add","timeout_ms":10000,"last_server_duration_us":378731,"total_duration_us":10007115},{"operation_name":"upsert","last_dispatch_duration_us":7415394,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11251672,"total_server_duration_us":2231433,"encode_duration_us":16647,"operation_id":"0x6bd2","timeout_ms":10000,"last_server_duration_us":2066903,"total_duration_us":10007017},{"operation_name":"upsert","last_dispatch_duration_us":12642003,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":12642003,"total_server_duration_us":19887,"encode_duration_us":16659,"operation_id":"0x6b59","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10006371},{"operation_name":"upsert","last_dispatch_duration_us":4636016,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4636016,"total_server_duration_us":202956,"encode_duration_us":18410,"operation_id":"0x6c8a","timeout_ms":10000,"last_server_duration_us":202956,"total_duration_us":10006338},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","encode_duration_us":16251,"operation_id":"0x6bfb","timeout_ms":10000,"total_duration_us":10006170}],"total_count":29}} INFO [2022-01-10 05:45:32,140] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:32,141] jepsen worker nemesis - couchbase.util Rebalance check count: 482 INFO [2022-01-10 05:45:32,141] jepsen worker nemesis - couchbase.util Rebalance stuck count: 395 WARN [2022-01-10 05:45:32,226] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":19686185,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":19686185,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9029938,"encode_duration_us":16430,"operation_id":"0x6ada","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10003187},{"operation_name":"upsert","last_dispatch_duration_us":16221043,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16221043,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9024634,"encode_duration_us":19505,"operation_id":"0x69bf","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10008728},{"operation_name":"upsert","last_dispatch_duration_us":21844126,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":21844126,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":12068138,"encode_duration_us":31101,"operation_id":"0x68c8","timeout_ms":10000,"last_server_duration_us":12068138,"total_duration_us":10006828}],"total_count":3}} INFO [2022-01-10 05:45:32,272] jepsen worker 26 - jepsen.util 3086 :info :write [8 42] :RequestTimeoutException INFO [2022-01-10 05:45:32,273] jepsen worker 26 - jepsen.util 3176 :invoke :read [8 nil] WARN [2022-01-10 05:45:32,278] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15595137,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":15595137,"total_server_duration_us":23,"operation_id":"0x6ade","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":15596101},{"operation_name":"get","last_dispatch_duration_us":14699191,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":14699191,"total_server_duration_us":15,"operation_id":"0x6b0c","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":14701599},{"operation_name":"upsert","last_dispatch_duration_us":62165,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":7748161,"total_server_duration_us":203085,"encode_duration_us":25793,"operation_id":"0x6c64","timeout_ms":10000,"last_server_duration_us":21813,"total_duration_us":10010935},{"operation_name":"upsert","last_dispatch_duration_us":2481553,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":5561148,"total_server_duration_us":286660,"encode_duration_us":16654,"operation_id":"0x6c27","timeout_ms":10000,"last_server_duration_us":84429,"total_duration_us":10009943},{"operation_name":"upsert","last_dispatch_duration_us":1400006,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":6788002,"total_server_duration_us":395120,"encode_duration_us":17059,"operation_id":"0x6c4a","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10008346},{"operation_name":"upsert","last_dispatch_duration_us":4136857,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":6129368,"total_server_duration_us":1169528,"encode_duration_us":17845,"operation_id":"0x6cbd","timeout_ms":10000,"last_server_duration_us":606268,"total_duration_us":10007886},{"operation_name":"upsert","last_dispatch_duration_us":1719762,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":6641904,"total_server_duration_us":711564,"encode_duration_us":17132,"operation_id":"0x6c26","timeout_ms":10000,"last_server_duration_us":198750,"total_duration_us":10007139},{"operation_name":"upsert","last_dispatch_duration_us":3338669,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10085435,"total_server_duration_us":1685735,"encode_duration_us":16667,"operation_id":"0x6ba7","timeout_ms":10000,"last_server_duration_us":601901,"total_duration_us":10007070},{"operation_name":"upsert","last_dispatch_duration_us":2884488,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7511643,"total_server_duration_us":2064304,"encode_duration_us":16519,"operation_id":"0x6b42","timeout_ms":10000,"last_server_duration_us":1565789,"total_duration_us":10005054},{"operation_name":"upsert","last_dispatch_duration_us":3825099,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11169521,"total_server_duration_us":2015859,"encode_duration_us":17004,"operation_id":"0x6b9f","timeout_ms":10000,"last_server_duration_us":797434,"total_duration_us":10002250}],"total_count":21}} INFO [2022-01-10 05:45:32,425] jepsen worker 6 - jepsen.util 3516 :info :write [2 21] :RequestTimeoutException INFO [2022-01-10 05:45:32,425] jepsen worker 6 - jepsen.util 3606 :invoke :read [2 nil] INFO [2022-01-10 05:45:32,656] jepsen worker 78 - jepsen.util 4128 :info :write [26 19] :RequestTimeoutException INFO [2022-01-10 05:45:32,657] jepsen worker 78 - jepsen.util 4218 :invoke :write [26 47] INFO [2022-01-10 05:45:32,661] jepsen worker 17 - jepsen.util 3347 :info :write [5 46] :RequestTimeoutException INFO [2022-01-10 05:45:32,662] jepsen worker 17 - jepsen.util 3437 :invoke :read [5 nil] INFO [2022-01-10 05:45:32,898] jepsen worker 62 - jepsen.util 3932 :ok :read [20 :nil] INFO [2022-01-10 05:45:32,899] jepsen worker 62 - jepsen.util 3932 :invoke :read [20 nil] INFO [2022-01-10 05:45:33,147] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:33,147] jepsen worker nemesis - couchbase.util Rebalance check count: 483 INFO [2022-01-10 05:45:33,492] jepsen worker nemesis - couchbase.util Rebalance stuck count: 396 INFO [2022-01-10 05:45:33,606] jepsen worker 63 - jepsen.util 3393 :info :write [21 24] :RequestTimeoutException INFO [2022-01-10 05:45:33,607] jepsen worker 63 - jepsen.util 3483 :invoke :read [21 nil] INFO [2022-01-10 05:45:33,611] jepsen worker 33 - jepsen.util 3723 :info :write [11 20] :RequestTimeoutException INFO [2022-01-10 05:45:33,612] jepsen worker 33 - jepsen.util 3813 :invoke :read [11 nil] INFO [2022-01-10 05:45:33,621] jepsen worker 79 - jepsen.util 4039 :info :write [26 16] :RequestTimeoutException INFO [2022-01-10 05:45:33,622] jepsen worker 79 - jepsen.util 4129 :invoke :read [26 nil] INFO [2022-01-10 05:45:33,646] jepsen worker 43 - jepsen.util 3373 :info :write [14 26] :RequestTimeoutException INFO [2022-01-10 05:45:33,647] jepsen worker 43 - jepsen.util 3463 :invoke :write [14 37] INFO [2022-01-10 05:45:33,752] jepsen worker 17 - jepsen.util 3437 :ok :read [5 42] INFO [2022-01-10 05:45:33,753] jepsen worker 17 - jepsen.util 3437 :invoke :write [5 14] INFO [2022-01-10 05:45:33,941] jepsen worker 50 - jepsen.util 3560 :info :write [16 5] :RequestTimeoutException INFO [2022-01-10 05:45:33,942] jepsen worker 50 - jepsen.util 3650 :invoke :write [16 4] INFO [2022-01-10 05:45:33,954] jepsen worker 86 - jepsen.util 2876 :info :write [28 8] :RequestTimeoutException INFO [2022-01-10 05:45:33,955] jepsen worker 86 - jepsen.util 2966 :invoke :read [28 nil] INFO [2022-01-10 05:45:34,148] jepsen worker 72 - jepsen.util 3852 :ok :read [24 18] INFO [2022-01-10 05:45:34,148] jepsen worker 72 - jepsen.util 3852 :invoke :write [24 32] INFO [2022-01-10 05:45:34,201] jepsen worker 79 - jepsen.util 4129 :ok :read [26 10] INFO [2022-01-10 05:45:34,202] jepsen worker 79 - jepsen.util 4129 :invoke :read [26 nil] INFO [2022-01-10 05:45:34,209] jepsen worker 53 - jepsen.util 4193 :info :write [17 10] :RequestTimeoutException INFO [2022-01-10 05:45:34,210] jepsen worker 53 - jepsen.util 4283 :invoke :write [17 43] INFO [2022-01-10 05:45:34,498] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:34,498] jepsen worker nemesis - couchbase.util Rebalance check count: 484 INFO [2022-01-10 05:45:34,498] jepsen worker nemesis - couchbase.util Rebalance stuck count: 397 INFO [2022-01-10 05:45:34,806] jepsen worker 18 - jepsen.util 3348 :ok :read [6 25] INFO [2022-01-10 05:45:34,807] jepsen worker 18 - jepsen.util 3348 :invoke :read [6 nil] INFO [2022-01-10 05:45:34,907] jepsen worker 79 - jepsen.util 4129 :ok :read [26 10] INFO [2022-01-10 05:45:34,908] jepsen worker 79 - jepsen.util 4129 :invoke :write [26 12] INFO [2022-01-10 05:45:35,266] jepsen worker 27 - jepsen.util 3537 :info :write [9 46] :RequestTimeoutException INFO [2022-01-10 05:45:35,266] jepsen worker 27 - jepsen.util 3627 :invoke :write [9 49] INFO [2022-01-10 05:45:35,365] jepsen worker 41 - jepsen.util 3281 :info :write [13 10] :RequestTimeoutException INFO [2022-01-10 05:45:35,365] jepsen worker 41 - jepsen.util 3371 :invoke :read [13 nil] INFO [2022-01-10 05:45:35,375] jepsen worker 86 - jepsen.util 2966 :ok :read [28 45] INFO [2022-01-10 05:45:35,375] jepsen worker 40 - jepsen.util 2830 :info :write [13 44] :RequestTimeoutException INFO [2022-01-10 05:45:35,375] jepsen worker 86 - jepsen.util 2966 :invoke :read [28 nil] INFO [2022-01-10 05:45:35,376] jepsen worker 40 - jepsen.util 2920 :invoke :write [13 13] INFO [2022-01-10 05:45:35,505] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:35,505] jepsen worker nemesis - couchbase.util Rebalance check count: 485 INFO [2022-01-10 05:45:35,505] jepsen worker nemesis - couchbase.util Rebalance stuck count: 398 INFO [2022-01-10 05:45:35,506] jepsen worker 34 - jepsen.util 3364 :ok :read [11 21] INFO [2022-01-10 05:45:35,507] jepsen worker 34 - jepsen.util 3364 :invoke :read [11 nil] INFO [2022-01-10 05:45:35,571] jepsen worker 60 - jepsen.util 3750 :info :write [20 2] :RequestTimeoutException INFO [2022-01-10 05:45:35,572] jepsen worker 60 - jepsen.util 3840 :invoke :read [20 nil] INFO [2022-01-10 05:45:35,634] jepsen worker 59 - jepsen.util 3479 :ok :read [19 31] INFO [2022-01-10 05:45:35,634] jepsen worker 59 - jepsen.util 3479 :invoke :write [19 37] INFO [2022-01-10 05:45:35,658] jepsen worker 39 - jepsen.util 4089 :ok :read [13 15] INFO [2022-01-10 05:45:35,658] jepsen worker 39 - jepsen.util 4089 :invoke :write [13 14] INFO [2022-01-10 05:45:35,660] jepsen worker 26 - jepsen.util 3176 :ok :read [8 27] INFO [2022-01-10 05:45:35,660] jepsen worker 26 - jepsen.util 3176 :invoke :write [8 17] INFO [2022-01-10 05:45:35,667] jepsen worker 8 - jepsen.util 3968 :ok :read [2 14] INFO [2022-01-10 05:45:35,669] jepsen worker 8 - jepsen.util 3968 :invoke :write [2 37] INFO [2022-01-10 05:45:35,770] jepsen worker 20 - jepsen.util 3170 :info :write [6 31] :RequestTimeoutException INFO [2022-01-10 05:45:35,771] jepsen worker 20 - jepsen.util 3260 :invoke :read [6 nil] INFO [2022-01-10 05:45:36,012] jepsen worker 73 - jepsen.util 3133 :info :write [24 32] :RequestTimeoutException INFO [2022-01-10 05:45:36,013] jepsen worker 73 - jepsen.util 3223 :invoke :read [24 nil] INFO [2022-01-10 05:45:36,189] jepsen worker 64 - jepsen.util 3394 :info :write [21 2] :RequestTimeoutException INFO [2022-01-10 05:45:36,190] jepsen worker 64 - jepsen.util 3484 :invoke :write [21 40] INFO [2022-01-10 05:45:36,215] jepsen worker 24 - jepsen.util 3084 :info :write [8 11] :RequestTimeoutException INFO [2022-01-10 05:45:36,216] jepsen worker 24 - jepsen.util 3174 :invoke :write [8 23] INFO [2022-01-10 05:45:36,429] jepsen worker 23 - jepsen.util 3533 :info :write [7 45] :RequestTimeoutException INFO [2022-01-10 05:45:36,430] jepsen worker 23 - jepsen.util 3623 :invoke :read [7 nil] INFO [2022-01-10 05:45:36,431] jepsen worker 80 - jepsen.util 4220 :info :write [26 48] :RequestTimeoutException INFO [2022-01-10 05:45:36,432] jepsen worker 80 - jepsen.util 4310 :invoke :write [26 47] INFO [2022-01-10 05:45:36,437] jepsen worker 54 - jepsen.util 3564 :ok :read [18 14] INFO [2022-01-10 05:45:36,437] jepsen worker 54 - jepsen.util 3564 :invoke :write [18 41] INFO [2022-01-10 05:45:36,443] jepsen worker 5 - jepsen.util 3335 :ok :read [1 10] INFO [2022-01-10 05:45:36,443] jepsen worker 5 - jepsen.util 3335 :invoke :read [1 nil] INFO [2022-01-10 05:45:36,480] jepsen worker 87 - jepsen.util 3507 :ok :read [29 0] INFO [2022-01-10 05:45:36,481] jepsen worker 87 - jepsen.util 3507 :invoke :read [29 nil] INFO [2022-01-10 05:45:36,485] jepsen worker 11 - jepsen.util 3431 :ok :read [3 31] INFO [2022-01-10 05:45:36,486] jepsen worker 11 - jepsen.util 3431 :invoke :read [3 nil] INFO [2022-01-10 05:45:36,511] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:36,511] jepsen worker nemesis - couchbase.util Rebalance check count: 486 INFO [2022-01-10 05:45:36,511] jepsen worker nemesis - couchbase.util Rebalance stuck count: 399 INFO [2022-01-10 05:45:36,547] jepsen worker 5 - jepsen.util 3335 :ok :read [1 10] INFO [2022-01-10 05:45:36,548] jepsen worker 5 - jepsen.util 3335 :invoke :write [1 29] INFO [2022-01-10 05:45:36,571] jepsen worker 60 - jepsen.util 3840 :ok :read [20 :nil] INFO [2022-01-10 05:45:36,572] jepsen worker 60 - jepsen.util 3840 :invoke :write [20 14] INFO [2022-01-10 05:45:36,607] jepsen worker 61 - jepsen.util 3571 :info :write [20 25] :RequestTimeoutException INFO [2022-01-10 05:45:36,607] jepsen worker 61 - jepsen.util 3661 :invoke :write [20 6] INFO [2022-01-10 05:45:36,622] jepsen worker 87 - jepsen.util 3507 :ok :read [29 0] INFO [2022-01-10 05:45:36,623] jepsen worker 87 - jepsen.util 3507 :invoke :write [29 5] INFO [2022-01-10 05:45:36,628] jepsen worker 11 - jepsen.util 3431 :ok :read [3 31] INFO [2022-01-10 05:45:36,629] jepsen worker 11 - jepsen.util 3431 :invoke :read [3 nil] INFO [2022-01-10 05:45:36,633] jepsen worker 2 - jepsen.util 3782 :info :write [0 24] :SyncWriteAmbiguous INFO [2022-01-10 05:45:36,635] jepsen worker 2 - jepsen.util 3872 :invoke :write [0 4] INFO [2022-01-10 05:45:36,680] jepsen worker 52 - jepsen.util 4372 :info :write [17 43] :RequestTimeoutException INFO [2022-01-10 05:45:36,681] jepsen worker 52 - jepsen.util 4462 :invoke :write [17 11] INFO [2022-01-10 05:45:36,707] jepsen worker 18 - jepsen.util 3348 :ok :read [6 25] INFO [2022-01-10 05:45:36,708] jepsen worker 18 - jepsen.util 3348 :invoke :write [6 37] INFO [2022-01-10 05:45:36,762] jepsen worker 11 - jepsen.util 3431 :ok :read [3 31] INFO [2022-01-10 05:45:36,763] jepsen worker 11 - jepsen.util 3431 :invoke :write [3 25] INFO [2022-01-10 05:45:36,780] jepsen worker 0 - jepsen.util 3240 :info :write [0 40] :RequestTimeoutException INFO [2022-01-10 05:45:36,780] jepsen worker 0 - jepsen.util 3330 :invoke :write [0 27] INFO [2022-01-10 05:45:36,881] jepsen worker 1 - jepsen.util 3511 :info :write [0 7] :RequestTimeoutException INFO [2022-01-10 05:45:36,881] jepsen worker 1 - jepsen.util 3601 :invoke :read [0 nil] INFO [2022-01-10 05:45:36,893] jepsen worker 34 - jepsen.util 3364 :ok :read [11 21] INFO [2022-01-10 05:45:36,894] jepsen worker 34 - jepsen.util 3364 :invoke :read [11 nil] INFO [2022-01-10 05:45:37,055] jepsen worker 33 - jepsen.util 3813 :ok :read [11 21] INFO [2022-01-10 05:45:37,055] jepsen worker 20 - jepsen.util 3260 :ok :read [6 25] INFO [2022-01-10 05:45:37,056] jepsen worker 20 - jepsen.util 3260 :invoke :write [6 18] INFO [2022-01-10 05:45:37,056] jepsen worker 33 - jepsen.util 3813 :invoke :read [11 nil] INFO [2022-01-10 05:45:37,391] jepsen worker 83 - jepsen.util 3863 :info :write [27 4] :RequestTimeoutException INFO [2022-01-10 05:45:37,391] jepsen worker 83 - jepsen.util 3953 :invoke :write [27 26] INFO [2022-01-10 05:45:37,451] jepsen worker 15 - jepsen.util 2985 :info :write [5 36] :RequestTimeoutException INFO [2022-01-10 05:45:37,451] jepsen worker 15 - jepsen.util 3075 :invoke :read [5 nil] INFO [2022-01-10 05:45:37,511] jepsen worker 29 - jepsen.util 3359 :info :write [9 26] :RequestTimeoutException INFO [2022-01-10 05:45:37,511] jepsen worker 29 - jepsen.util 3449 :invoke :write [9 46] INFO [2022-01-10 05:45:37,516] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:37,516] jepsen worker nemesis - couchbase.util Rebalance check count: 487 INFO [2022-01-10 05:45:37,516] jepsen worker nemesis - couchbase.util Rebalance stuck count: 400 INFO [2022-01-10 05:45:37,557] jepsen worker 68 - jepsen.util 3578 :ok :read [22 41] INFO [2022-01-10 05:45:37,558] jepsen worker 68 - jepsen.util 3578 :invoke :write [22 8] INFO [2022-01-10 05:45:37,711] jepsen worker 86 - jepsen.util 2966 :ok :read [28 45] INFO [2022-01-10 05:45:37,712] jepsen worker 86 - jepsen.util 2966 :invoke :write [28 43] INFO [2022-01-10 05:45:37,719] jepsen worker 55 - jepsen.util 3745 :info :write [18 27] :RequestTimeoutException INFO [2022-01-10 05:45:37,720] jepsen worker 55 - jepsen.util 3835 :invoke :read [18 nil] INFO [2022-01-10 05:45:37,746] jepsen worker 55 - jepsen.util 3835 :ok :read [18 14] INFO [2022-01-10 05:45:37,746] jepsen worker 55 - jepsen.util 3835 :invoke :write [18 34] INFO [2022-01-10 05:45:37,750] jepsen worker 9 - jepsen.util 3969 :info :write [3 35] :RequestTimeoutException INFO [2022-01-10 05:45:37,751] jepsen worker 9 - jepsen.util 4059 :invoke :write [3 35] INFO [2022-01-10 05:45:37,829] jepsen worker 69 - jepsen.util 2949 :info :write [23 41] :RequestTimeoutException INFO [2022-01-10 05:45:37,830] jepsen worker 69 - jepsen.util 3039 :invoke :read [23 nil] INFO [2022-01-10 05:45:37,972] jepsen worker 37 - jepsen.util 3187 :info :write [12 9] :SyncWriteAmbiguous INFO [2022-01-10 05:45:37,973] jepsen worker 37 - jepsen.util 3277 :invoke :write [12 39] INFO [2022-01-10 05:45:38,126] jepsen worker 49 - jepsen.util 3379 :info :write [16 37] :RequestTimeoutException INFO [2022-01-10 05:45:38,127] jepsen worker 49 - jepsen.util 3469 :invoke :write [16 41] INFO [2022-01-10 05:45:38,218] jepsen worker 14 - jepsen.util 4244 :info :write [4 33] :RequestTimeoutException INFO [2022-01-10 05:45:38,219] jepsen worker 14 - jepsen.util 4334 :invoke :read [4 nil] INFO [2022-01-10 05:45:38,256] jepsen worker 76 - jepsen.util 2956 :info :write [25 35] :RequestTimeoutException INFO [2022-01-10 05:45:38,266] jepsen worker 7 - jepsen.util 3607 :info :write [2 25] :RequestTimeoutException INFO [2022-01-10 05:45:38,271] jepsen worker 32 - jepsen.util 3722 :info :write [10 42] :SyncWriteAmbiguous INFO [2022-01-10 05:45:38,631] jepsen worker 7 - jepsen.util 3697 :invoke :read [2 nil] INFO [2022-01-10 05:45:38,286] jepsen worker 16 - jepsen.util 3526 :info :write [5 48] :RequestTimeoutException INFO [2022-01-10 05:45:38,295] jepsen worker 67 - jepsen.util 3307 :info :write [22 27] :RequestTimeoutException INFO [2022-01-10 05:45:38,631] jepsen worker 16 - jepsen.util 3616 :invoke :write [5 17] INFO [2022-01-10 05:45:38,631] jepsen worker 67 - jepsen.util 3397 :invoke :write [22 46] INFO [2022-01-10 05:45:38,416] jepsen worker 84 - jepsen.util 3414 :info :write [28 45] :RequestTimeoutException INFO [2022-01-10 05:45:38,524] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:38,631] jepsen worker 84 - jepsen.util 3504 :invoke :read [28 nil] INFO [2022-01-10 05:45:38,631] jepsen worker nemesis - couchbase.util Rebalance check count: 488 INFO [2022-01-10 05:45:38,631] jepsen worker nemesis - couchbase.util Rebalance stuck count: 401 INFO [2022-01-10 05:45:38,575] jepsen worker 12 - jepsen.util 3882 :info :write [4 26] :RequestTimeoutException INFO [2022-01-10 05:45:38,631] jepsen worker 76 - jepsen.util 3046 :invoke :write [25 2] INFO [2022-01-10 05:45:38,632] jepsen worker 12 - jepsen.util 3972 :invoke :read [4 nil] INFO [2022-01-10 05:45:38,631] jepsen worker 32 - jepsen.util 3812 :invoke :read [10 nil] INFO [2022-01-10 05:45:38,660] jepsen worker 32 - jepsen.util 3812 :ok :read [10 36] INFO [2022-01-10 05:45:38,660] jepsen worker 32 - jepsen.util 3812 :invoke :write [10 46] INFO [2022-01-10 05:45:38,676] jepsen worker 44 - jepsen.util 3554 :info :write [14 24] :RequestTimeoutException INFO [2022-01-10 05:45:38,676] jepsen worker 44 - jepsen.util 3644 :invoke :read [14 nil] INFO [2022-01-10 05:45:38,719] jepsen worker 1 - jepsen.util 3601 :ok :read [0 46] INFO [2022-01-10 05:45:38,719] jepsen worker 1 - jepsen.util 3601 :invoke :read [0 nil] INFO [2022-01-10 05:45:38,770] jepsen worker 88 - jepsen.util 3508 :info :write [29 0] :SyncWriteAmbiguous INFO [2022-01-10 05:45:38,770] jepsen worker 66 - jepsen.util 3216 :info :write [22 19] :RequestTimeoutException INFO [2022-01-10 05:45:38,771] jepsen worker 66 - jepsen.util 3306 :invoke :read [22 nil] INFO [2022-01-10 05:45:38,771] jepsen worker 88 - jepsen.util 3598 :invoke :read [29 nil] INFO [2022-01-10 05:45:38,802] jepsen worker 13 - jepsen.util 3973 :info :write [4 18] :RequestTimeoutException INFO [2022-01-10 05:45:38,803] jepsen worker 13 - jepsen.util 4063 :invoke :write [4 44] INFO [2022-01-10 05:45:38,825] jepsen worker 71 - jepsen.util 3041 :ok :read [23 26] INFO [2022-01-10 05:45:38,825] jepsen worker 71 - jepsen.util 3041 :invoke :write [23 36] INFO [2022-01-10 05:45:38,836] jepsen worker 69 - jepsen.util 3039 :ok :read [23 26] INFO [2022-01-10 05:45:38,837] jepsen worker 69 - jepsen.util 3039 :invoke :read [23 nil] INFO [2022-01-10 05:45:38,859] jepsen worker 69 - jepsen.util 3039 :ok :read [23 26] INFO [2022-01-10 05:45:38,859] jepsen worker 69 - jepsen.util 3039 :invoke :read [23 nil] INFO [2022-01-10 05:45:38,908] jepsen worker 69 - jepsen.util 3039 :ok :read [23 26] INFO [2022-01-10 05:45:38,909] jepsen worker 69 - jepsen.util 3039 :invoke :write [23 17] INFO [2022-01-10 05:45:39,237] cb-events - com.couchbase.metrics [com.couchbase.metrics][LatencyMetricsAggregatedEvent][600s] Aggregated Latency Metrics: {"operations":{"kv":{"get":{"total_count":628,"percentiles_us":{"50.0":922746.879,"90.0":1.0938744831E7,"99.0":2.2548578303E7,"99.9":2.4024973311E7,"100.0":2.4561844223E7}},"upsert":{"total_count":602,"percentiles_us":{"50.0":1.0066329599E7,"90.0":1.0066329599E7,"99.0":1.0066329599E7,"99.9":1.0066329599E7,"100.0":1.0066329599E7}}}},"meta":{"emit_interval_s":600}} INFO [2022-01-10 05:45:39,331] jepsen worker 57 - jepsen.util 4197 :info :write [19 49] :RequestTimeoutException INFO [2022-01-10 05:45:39,331] jepsen worker 57 - jepsen.util 4287 :invoke :write [19 30] INFO [2022-01-10 05:45:39,344] jepsen worker 81 - jepsen.util 3771 :info :write [27 3] :RequestTimeoutException INFO [2022-01-10 05:45:39,345] jepsen worker 81 - jepsen.util 3861 :invoke :read [27 nil] INFO [2022-01-10 05:45:39,637] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:39,637] jepsen worker nemesis - couchbase.util Rebalance check count: 489 INFO [2022-01-10 05:45:39,637] jepsen worker nemesis - couchbase.util Rebalance stuck count: 402 INFO [2022-01-10 05:45:39,746] jepsen worker 89 - jepsen.util 3869 :info :write [29 25] :RequestTimeoutException INFO [2022-01-10 05:45:39,746] jepsen worker 89 - jepsen.util 3959 :invoke :write [29 20] INFO [2022-01-10 05:45:39,820] jepsen worker 41 - jepsen.util 3371 :ok :read [13 15] INFO [2022-01-10 05:45:39,820] jepsen worker 41 - jepsen.util 3371 :invoke :write [13 39] INFO [2022-01-10 05:45:39,821] jepsen worker 15 - jepsen.util 3075 :ok :read [5 42] INFO [2022-01-10 05:45:39,821] jepsen worker 15 - jepsen.util 3075 :invoke :read [5 nil] INFO [2022-01-10 05:45:39,930] jepsen worker 85 - jepsen.util 3865 :info :write [28 9] :RequestTimeoutException INFO [2022-01-10 05:45:39,930] jepsen worker 85 - jepsen.util 3955 :invoke :write [28 27] INFO [2022-01-10 05:45:40,076] jepsen worker 58 - jepsen.util 3388 :info :write [19 49] :RequestTimeoutException INFO [2022-01-10 05:45:40,076] jepsen worker 58 - jepsen.util 3478 :invoke :read [19 nil] INFO [2022-01-10 05:45:40,222] jepsen worker 31 - jepsen.util 3901 :info :write [10 40] :RequestTimeoutException INFO [2022-01-10 05:45:40,223] jepsen worker 31 - jepsen.util 3991 :invoke :read [10 nil] WARN [2022-01-10 05:45:40,339] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9895059,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12619317,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9843018,"encode_duration_us":17624,"operation_id":"0x6d1a","timeout_ms":10000,"last_server_duration_us":9261418,"total_duration_us":10002172},{"operation_name":"upsert","last_dispatch_duration_us":21171172,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":21171172,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9027816,"encode_duration_us":17332,"operation_id":"0x6a7a","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10002849},{"operation_name":"upsert","last_dispatch_duration_us":10440185,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10440185,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9605128,"encode_duration_us":18673,"operation_id":"0x6cc8","timeout_ms":10000,"last_server_duration_us":9605128,"total_duration_us":10002196},{"operation_name":"upsert","last_dispatch_duration_us":9890734,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11548158,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9792370,"encode_duration_us":18286,"operation_id":"0x6d50","timeout_ms":10000,"last_server_duration_us":9447744,"total_duration_us":10004001}],"total_count":4}} INFO [2022-01-10 05:45:40,420] jepsen worker 14 - jepsen.util 4334 :ok :read [4 3] INFO [2022-01-10 05:45:40,421] jepsen worker 14 - jepsen.util 4334 :invoke :read [4 nil] WARN [2022-01-10 05:45:40,439] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":20362999,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":20362999,"total_server_duration_us":32,"operation_id":"0x6ab1","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":20364031},{"operation_name":"get","last_dispatch_duration_us":19879140,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":19879140,"total_server_duration_us":15,"operation_id":"0x6adc","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":19880284},{"operation_name":"upsert","last_dispatch_duration_us":9676940,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12826164,"total_server_duration_us":1112175,"encode_duration_us":17807,"operation_id":"0x6c89","timeout_ms":10000,"last_server_duration_us":128891,"total_duration_us":10009879},{"operation_name":"upsert","last_dispatch_duration_us":9868530,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11803127,"total_server_duration_us":966847,"encode_duration_us":16518,"operation_id":"0x6d51","timeout_ms":10000,"last_server_duration_us":240655,"total_duration_us":10009609},{"operation_name":"upsert","last_dispatch_duration_us":8233744,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11866412,"total_server_duration_us":763362,"encode_duration_us":17342,"operation_id":"0x6cbc","timeout_ms":10000,"last_server_duration_us":431997,"total_duration_us":10009127},{"operation_name":"upsert","last_dispatch_duration_us":1288549,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3072408,"total_server_duration_us":663798,"encode_duration_us":16745,"operation_id":"0x6d81","timeout_ms":10000,"last_server_duration_us":242930,"total_duration_us":10007642},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":17722,"operation_id":"0x6c5f","timeout_ms":10000,"total_duration_us":10005096},{"operation_name":"upsert","last_dispatch_duration_us":8178538,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8178538,"total_server_duration_us":142776,"encode_duration_us":17905,"operation_id":"0x6e29","timeout_ms":10000,"last_server_duration_us":142776,"total_duration_us":10004897},{"operation_name":"upsert","last_dispatch_duration_us":9890734,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11548158,"total_server_duration_us":9792370,"encode_duration_us":18286,"operation_id":"0x6d50","timeout_ms":10000,"last_server_duration_us":9447744,"total_duration_us":10004001},{"operation_name":"upsert","last_dispatch_duration_us":10440185,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10440185,"total_server_duration_us":9605128,"encode_duration_us":18673,"operation_id":"0x6cc8","timeout_ms":10000,"last_server_duration_us":9605128,"total_duration_us":10002196}],"total_count":15}} INFO [2022-01-10 05:45:40,454] jepsen worker 12 - jepsen.util 3972 :ok :read [4 3] INFO [2022-01-10 05:45:40,455] jepsen worker 12 - jepsen.util 3972 :invoke :write [4 30] INFO [2022-01-10 05:45:40,488] jepsen worker 1 - jepsen.util 3601 :ok :read [0 46] INFO [2022-01-10 05:45:40,489] jepsen worker 1 - jepsen.util 3601 :invoke :read [0 nil] INFO [2022-01-10 05:45:40,614] jepsen worker 14 - jepsen.util 4334 :ok :read [4 3] INFO [2022-01-10 05:45:40,615] jepsen worker 14 - jepsen.util 4334 :invoke :write [4 6] INFO [2022-01-10 05:45:40,654] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:40,655] jepsen worker nemesis - couchbase.util Rebalance check count: 490 INFO [2022-01-10 05:45:40,655] jepsen worker nemesis - couchbase.util Rebalance stuck count: 403 INFO [2022-01-10 05:45:40,708] jepsen worker 3 - jepsen.util 3333 :ok :read [1 10] INFO [2022-01-10 05:45:40,709] jepsen worker 3 - jepsen.util 3333 :invoke :write [1 26] INFO [2022-01-10 05:45:40,718] jepsen worker 38 - jepsen.util 3638 :ok :read [12 29] INFO [2022-01-10 05:45:40,719] jepsen worker 38 - jepsen.util 3638 :invoke :read [12 nil] INFO [2022-01-10 05:45:40,736] jepsen worker 4 - jepsen.util 4054 :info :write [1 17] :RequestTimeoutException INFO [2022-01-10 05:45:40,736] jepsen worker 4 - jepsen.util 4144 :invoke :write [1 16] INFO [2022-01-10 05:45:40,739] cb-events - com.couchbase.metrics [com.couchbase.metrics][LatencyMetricsAggregatedEvent][600s] Aggregated Latency Metrics: {"operations":{"kv":{"get":{"total_count":637,"percentiles_us":{"50.0":2097151.999,"90.0":1.0871635967E7,"99.0":1.9058917375E7,"99.9":2.4964497407E7,"100.0":2.8454158335E7}},"upsert":{"total_count":601,"percentiles_us":{"50.0":1.0066329599E7,"90.0":1.0066329599E7,"99.0":1.0066329599E7,"99.9":1.0066329599E7,"100.0":1.0066329599E7}}}},"meta":{"emit_interval_s":600}} INFO [2022-01-10 05:45:40,777] jepsen worker 66 - jepsen.util 3306 :ok :read [22 41] INFO [2022-01-10 05:45:40,778] jepsen worker 66 - jepsen.util 3306 :invoke :read [22 nil] INFO [2022-01-10 05:45:40,822] cb-events - com.couchbase.metrics [com.couchbase.metrics][LatencyMetricsAggregatedEvent][600s] Aggregated Latency Metrics: {"operations":{"kv":{"get":{"total_count":623,"percentiles_us":{"50.0":1090519.039,"90.0":9865003.007,"99.0":2.4964497407E7,"99.9":2.9527900159E7,"100.0":3.0064771071E7}},"upsert":{"total_count":618,"percentiles_us":{"50.0":1.0066329599E7,"90.0":1.0066329599E7,"99.0":1.0066329599E7,"99.9":1.0066329599E7,"100.0":1.0066329599E7}}}},"meta":{"emit_interval_s":600}} INFO [2022-01-10 05:45:40,822] cb-events - com.couchbase.metrics [com.couchbase.metrics][LatencyMetricsAggregatedEvent][600s] Aggregated Latency Metrics: {"operations":{"kv":{"get":{"total_count":650,"percentiles_us":{"50.0":1283457.023,"90.0":1.0334765055E7,"99.0":2.2145925119E7,"99.9":3.0064771071E7,"100.0":3.0064771071E7}},"upsert":{"total_count":607,"percentiles_us":{"50.0":1.0066329599E7,"90.0":1.0066329599E7,"99.0":1.0066329599E7,"99.9":1.0066329599E7,"100.0":1.0066329599E7}}}},"meta":{"emit_interval_s":600}} INFO [2022-01-10 05:45:40,836] cb-events - com.couchbase.metrics [com.couchbase.metrics][LatencyMetricsAggregatedEvent][600s] Aggregated Latency Metrics: {"operations":{"kv":{"get":{"total_count":564,"percentiles_us":{"50.0":1451229.183,"90.0":1.0133438463E7,"99.0":2.4696061951E7,"99.9":3.0064771071E7,"100.0":3.0064771071E7}},"upsert":{"total_count":596,"percentiles_us":{"50.0":1.0066329599E7,"90.0":1.0066329599E7,"99.0":1.0066329599E7,"99.9":1.0066329599E7,"100.0":1.0066329599E7}}}},"meta":{"emit_interval_s":600}} INFO [2022-01-10 05:45:40,840] cb-events - com.couchbase.metrics [com.couchbase.metrics][LatencyMetricsAggregatedEvent][600s] Aggregated Latency Metrics: {"operations":{"kv":{"get":{"total_count":709,"percentiles_us":{"50.0":1224736.767,"90.0":1.0066329599E7,"99.0":2.3622320127E7,"99.9":3.0064771071E7,"100.0":3.0064771071E7}},"upsert":{"total_count":627,"percentiles_us":{"50.0":1.0066329599E7,"90.0":1.0066329599E7,"99.0":1.0066329599E7,"99.9":1.0066329599E7,"100.0":1.0066329599E7}}}},"meta":{"emit_interval_s":600}} INFO [2022-01-10 05:45:40,850] jepsen worker 47 - jepsen.util 3557 :info :write [15 36] :RequestTimeoutException INFO [2022-01-10 05:45:40,851] jepsen worker 47 - jepsen.util 3647 :invoke :read [15 nil] INFO [2022-01-10 05:45:40,902] jepsen worker 70 - jepsen.util 3580 :info :write [23 36] :RequestTimeoutException INFO [2022-01-10 05:45:40,902] jepsen worker 70 - jepsen.util 3670 :invoke :read [23 nil] INFO [2022-01-10 05:45:40,926] jepsen worker 28 - jepsen.util 3268 :info :write [9 13] :RequestTimeoutException INFO [2022-01-10 05:45:40,927] jepsen worker 28 - jepsen.util 3358 :invoke :read [9 nil] INFO [2022-01-10 05:45:40,943] jepsen worker 56 - jepsen.util 3926 :ok :read [18 14] INFO [2022-01-10 05:45:40,944] jepsen worker 56 - jepsen.util 3926 :invoke :write [18 22] INFO [2022-01-10 05:45:40,947] jepsen worker 33 - jepsen.util 3813 :ok :read [11 21] INFO [2022-01-10 05:45:40,948] jepsen worker 33 - jepsen.util 3813 :invoke :read [11 nil] INFO [2022-01-10 05:45:41,053] jepsen worker 81 - jepsen.util 3861 :ok :read [27 30] INFO [2022-01-10 05:45:41,053] jepsen worker 81 - jepsen.util 3861 :invoke :write [27 4] INFO [2022-01-10 05:45:41,135] jepsen worker 19 - jepsen.util 3619 :info :write [6 46] :RequestTimeoutException INFO [2022-01-10 05:45:41,135] jepsen worker 19 - jepsen.util 3709 :invoke :read [6 nil] INFO [2022-01-10 05:45:41,226] jepsen worker 75 - jepsen.util 3045 :info :write [25 21] :RequestTimeoutException INFO [2022-01-10 05:45:41,226] jepsen worker 22 - jepsen.util 3172 :info :write [7 46] :RequestTimeoutException INFO [2022-01-10 05:45:41,227] jepsen worker 75 - jepsen.util 3135 :invoke :write [25 47] INFO [2022-01-10 05:45:41,227] jepsen worker 22 - jepsen.util 3262 :invoke :read [7 nil] INFO [2022-01-10 05:45:41,306] jepsen worker 48 - jepsen.util 3108 :info :write [16 7] :RequestTimeoutException INFO [2022-01-10 05:45:41,307] jepsen worker 48 - jepsen.util 3198 :invoke :write [16 10] INFO [2022-01-10 05:45:41,337] jepsen worker 34 - jepsen.util 3364 :ok :read [11 21] INFO [2022-01-10 05:45:41,337] jepsen worker 34 - jepsen.util 3364 :invoke :read [11 nil] INFO [2022-01-10 05:45:41,340] jepsen worker 77 - jepsen.util 3857 :info :write [25 13] :RequestTimeoutException INFO [2022-01-10 05:45:41,341] jepsen worker 77 - jepsen.util 3947 :invoke :read [25 nil] INFO [2022-01-10 05:45:41,343] jepsen worker 44 - jepsen.util 3644 :ok :read [14 14] INFO [2022-01-10 05:45:41,343] jepsen worker 44 - jepsen.util 3644 :invoke :read [14 nil] INFO [2022-01-10 05:45:41,351] jepsen worker 35 - jepsen.util 3185 :info :write [11 33] :RequestTimeoutException INFO [2022-01-10 05:45:41,352] jepsen worker 35 - jepsen.util 3275 :invoke :write [11 28] INFO [2022-01-10 05:45:41,385] jepsen worker 74 - jepsen.util 3944 :info :write [24 37] :RequestTimeoutException INFO [2022-01-10 05:45:41,386] jepsen worker 74 - jepsen.util 4034 :invoke :read [24 nil] INFO [2022-01-10 05:45:41,426] jepsen worker 19 - jepsen.util 3709 :ok :read [6 25] INFO [2022-01-10 05:45:41,426] jepsen worker 19 - jepsen.util 3709 :invoke :write [6 15] INFO [2022-01-10 05:45:41,592] jepsen worker 15 - jepsen.util 3075 :ok :read [5 42] INFO [2022-01-10 05:45:41,593] jepsen worker 15 - jepsen.util 3075 :invoke :read [5 nil] INFO [2022-01-10 05:45:41,661] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:41,661] jepsen worker nemesis - couchbase.util Rebalance check count: 491 INFO [2022-01-10 05:45:41,661] jepsen worker nemesis - couchbase.util Rebalance stuck count: 404 INFO [2022-01-10 05:45:41,716] jepsen worker 82 - jepsen.util 3682 :info :write [27 34] :RequestTimeoutException INFO [2022-01-10 05:45:41,716] jepsen worker 51 - jepsen.util 4191 :info :write [17 48] :RequestTimeoutException INFO [2022-01-10 05:45:41,716] jepsen worker 51 - jepsen.util 4281 :invoke :read [17 nil] INFO [2022-01-10 05:45:41,717] jepsen worker 82 - jepsen.util 3772 :invoke :write [27 15] INFO [2022-01-10 05:45:41,775] jepsen worker 74 - jepsen.util 4034 :ok :read [24 18] INFO [2022-01-10 05:45:41,777] jepsen worker 74 - jepsen.util 4034 :invoke :write [24 44] INFO [2022-01-10 05:45:41,778] jepsen worker 77 - jepsen.util 3947 :ok :read [25 :nil] INFO [2022-01-10 05:45:41,779] jepsen worker 77 - jepsen.util 3947 :invoke :read [25 nil] INFO [2022-01-10 05:45:41,779] jepsen worker 34 - jepsen.util 3364 :ok :read [11 21] INFO [2022-01-10 05:45:41,779] jepsen worker 34 - jepsen.util 3364 :invoke :write [11 4] INFO [2022-01-10 05:45:41,794] jepsen worker 44 - jepsen.util 3644 :ok :read [14 14] INFO [2022-01-10 05:45:41,794] jepsen worker 44 - jepsen.util 3644 :invoke :read [14 nil] INFO [2022-01-10 05:45:41,799] jepsen worker 1 - jepsen.util 3601 :ok :read [0 46] INFO [2022-01-10 05:45:41,800] jepsen worker 1 - jepsen.util 3601 :invoke :write [0 20] INFO [2022-01-10 05:45:41,905] jepsen worker 30 - jepsen.util 3180 :info :write [10 24] :RequestTimeoutException INFO [2022-01-10 05:45:41,905] jepsen worker 30 - jepsen.util 3270 :invoke :read [10 nil] INFO [2022-01-10 05:45:41,915] jepsen worker 36 - jepsen.util 2646 :info :write [12 14] :RequestTimeoutException INFO [2022-01-10 05:45:41,915] jepsen worker 36 - jepsen.util 2736 :invoke :read [12 nil] INFO [2022-01-10 05:45:41,934] jepsen worker 25 - jepsen.util 3355 :info :write [8 44] :RequestTimeoutException INFO [2022-01-10 05:45:41,935] jepsen worker 25 - jepsen.util 3445 :invoke :write [8 1] WARN [2022-01-10 05:45:41,941] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18107788,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18107788,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9030999,"encode_duration_us":16251,"operation_id":"0x6bfb","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10006170}],"total_count":1}} INFO [2022-01-10 05:45:41,976] jepsen worker 10 - jepsen.util 3610 :info :write [3 27] :RequestTimeoutException INFO [2022-01-10 05:45:41,976] jepsen worker 10 - jepsen.util 3700 :invoke :read [3 nil] INFO [2022-01-10 05:45:42,013] jepsen worker 66 - jepsen.util 3306 :ok :read [22 41] INFO [2022-01-10 05:45:42,014] jepsen worker 66 - jepsen.util 3306 :invoke :read [22 nil] WARN [2022-01-10 05:45:42,024] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2006624,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10522657,"total_server_duration_us":3578057,"encode_duration_us":17797,"operation_id":"0x6ded","timeout_ms":10000,"last_server_duration_us":252354,"total_duration_us":10038291},{"operation_name":"upsert","last_dispatch_duration_us":10133802,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10133802,"total_server_duration_us":173320,"encode_duration_us":19321,"operation_id":"0x6db6","timeout_ms":10000,"last_server_duration_us":173320,"total_duration_us":10009167},{"operation_name":"upsert","last_dispatch_duration_us":12985978,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":12985978,"total_server_duration_us":9435850,"encode_duration_us":16729,"operation_id":"0x6d19","timeout_ms":10000,"last_server_duration_us":9435850,"total_duration_us":10008702},{"operation_name":"upsert","last_dispatch_duration_us":8937685,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":16521516,"total_server_duration_us":210094,"encode_duration_us":16741,"operation_id":"0x6cbb","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10007079},{"operation_name":"upsert","last_dispatch_duration_us":10591424,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10591424,"total_server_duration_us":256306,"encode_duration_us":17779,"operation_id":"0x6dba","timeout_ms":10000,"last_server_duration_us":256306,"total_duration_us":10005397},{"operation_name":"upsert","last_dispatch_duration_us":630113,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9896157,"total_server_duration_us":4124254,"encode_duration_us":21552,"operation_id":"0x6d85","timeout_ms":10000,"last_server_duration_us":362902,"total_duration_us":10004911},{"operation_name":"upsert","last_dispatch_duration_us":10307066,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10307066,"total_server_duration_us":86328,"encode_duration_us":18542,"operation_id":"0x6da9","timeout_ms":10000,"last_server_duration_us":86328,"total_duration_us":10004486},{"operation_name":"upsert","last_dispatch_duration_us":1596642,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10935141,"total_server_duration_us":435575,"encode_duration_us":16578,"operation_id":"0x6e28","timeout_ms":10000,"last_server_duration_us":295516,"total_duration_us":10003273},{"operation_name":"upsert","last_dispatch_duration_us":10829519,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10829519,"total_server_duration_us":19030,"encode_duration_us":16592,"operation_id":"0x6dbb","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10002838},{"operation_name":"upsert","last_dispatch_duration_us":9044751,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9693160,"total_server_duration_us":9174822,"encode_duration_us":16822,"operation_id":"0x6e1c","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":9844492}],"total_count":18}} WARN [2022-01-10 05:45:42,024] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12985978,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12985978,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9435850,"encode_duration_us":16729,"operation_id":"0x6d19","timeout_ms":10000,"last_server_duration_us":9435850,"total_duration_us":10008702}],"total_count":1}} WARN [2022-01-10 05:45:42,044] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":18096453,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":18096453,"total_server_duration_us":43,"operation_id":"0x6cc9","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":18097621},{"operation_name":"get","last_dispatch_duration_us":10406143,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10406143,"total_server_duration_us":49,"operation_id":"0x6e53","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":10407506},{"operation_name":"upsert","last_dispatch_duration_us":2460542,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":12021349,"total_server_duration_us":238696,"encode_duration_us":17160,"operation_id":"0x6de4","timeout_ms":10000,"last_server_duration_us":83703,"total_duration_us":10010262},{"operation_name":"upsert","last_dispatch_duration_us":2374090,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7963271,"total_server_duration_us":903513,"encode_duration_us":18610,"operation_id":"0x6d88","timeout_ms":10000,"last_server_duration_us":74786,"total_duration_us":10009965},{"operation_name":"upsert","last_dispatch_duration_us":1951745,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8287372,"total_server_duration_us":1938802,"encode_duration_us":16419,"operation_id":"0x6df1","timeout_ms":10000,"last_server_duration_us":156502,"total_duration_us":10009827},{"operation_name":"upsert","last_dispatch_duration_us":1585160,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9111476,"total_server_duration_us":166265,"encode_duration_us":19062,"operation_id":"0x6e91","timeout_ms":10000,"last_server_duration_us":81254,"total_duration_us":10009005},{"operation_name":"upsert","last_dispatch_duration_us":950519,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":10682409,"total_server_duration_us":5435002,"encode_duration_us":18591,"operation_id":"0x6cca","timeout_ms":10000,"last_server_duration_us":326920,"total_duration_us":10008280},{"operation_name":"upsert","last_dispatch_duration_us":1850749,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9522127,"total_server_duration_us":153371,"encode_duration_us":23554,"operation_id":"0x6e7c","timeout_ms":10000,"last_server_duration_us":73684,"total_duration_us":10006778},{"operation_name":"upsert","last_dispatch_duration_us":2842827,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9713295,"total_server_duration_us":1312224,"encode_duration_us":16631,"operation_id":"0x6cfa","timeout_ms":10000,"last_server_duration_us":301489,"total_duration_us":10003092},{"operation_name":"upsert","last_dispatch_duration_us":3196101,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11452293,"total_server_duration_us":2160414,"encode_duration_us":27259,"operation_id":"0x6df0","timeout_ms":10000,"last_server_duration_us":193339,"total_duration_us":10001479}],"total_count":23}} INFO [2022-01-10 05:45:42,092] jepsen worker 65 - jepsen.util 3305 :info :write [21 0] :RequestTimeoutException INFO [2022-01-10 05:45:42,093] jepsen worker 65 - jepsen.util 3395 :invoke :read [21 nil] WARN [2022-01-10 05:45:42,126] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14714800,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14714800,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9698995,"encode_duration_us":17656,"operation_id":"0x6d80","timeout_ms":10000,"last_server_duration_us":9698995,"total_duration_us":10002275},{"operation_name":"upsert","last_dispatch_duration_us":9154934,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16903096,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9231962,"encode_duration_us":25793,"operation_id":"0x6c64","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10010935},{"operation_name":"upsert","last_dispatch_duration_us":11251497,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16812646,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9873279,"encode_duration_us":16654,"operation_id":"0x6c27","timeout_ms":10000,"last_server_duration_us":9586619,"total_duration_us":10009943}],"total_count":3}} WARN [2022-01-10 05:45:42,138] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":4167078,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13064107,"total_server_duration_us":597923,"encode_duration_us":18611,"operation_id":"0x6cc6","timeout_ms":10000,"last_server_duration_us":227416,"total_duration_us":10010146},{"operation_name":"upsert","last_dispatch_duration_us":4351343,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12293932,"total_server_duration_us":572117,"encode_duration_us":18679,"operation_id":"0x6d77","timeout_ms":10000,"last_server_duration_us":281293,"total_duration_us":10009603},{"operation_name":"upsert","last_dispatch_duration_us":3233266,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":11462493,"total_server_duration_us":3920368,"encode_duration_us":16904,"operation_id":"0x6e12","timeout_ms":10000,"last_server_duration_us":898664,"total_duration_us":10006932},{"operation_name":"upsert","last_dispatch_duration_us":3501346,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11261852,"total_server_duration_us":586055,"encode_duration_us":17137,"operation_id":"0x6d52","timeout_ms":10000,"last_server_duration_us":343125,"total_duration_us":10006771},{"operation_name":"upsert","last_dispatch_duration_us":31212,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":806617,"total_server_duration_us":465368,"encode_duration_us":16656,"operation_id":"0x6e7b","timeout_ms":10000,"last_server_duration_us":20680,"total_duration_us":10006491},{"operation_name":"upsert","last_dispatch_duration_us":8107558,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8107558,"total_server_duration_us":336027,"encode_duration_us":18092,"operation_id":"0x6cf8","timeout_ms":10000,"last_server_duration_us":336027,"total_duration_us":10005927},{"operation_name":"upsert","last_dispatch_duration_us":1073790,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":10290265,"total_server_duration_us":5643507,"encode_duration_us":18628,"operation_id":"0x6cf9","timeout_ms":10000,"last_server_duration_us":502725,"total_duration_us":10005197},{"operation_name":"upsert","last_dispatch_duration_us":4070938,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9237801,"total_server_duration_us":1225317,"encode_duration_us":17199,"operation_id":"0x6e89","timeout_ms":10000,"last_server_duration_us":202322,"total_duration_us":10002086},{"operation_name":"upsert","last_dispatch_duration_us":9230338,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":12528566,"total_server_duration_us":760684,"encode_duration_us":17680,"operation_id":"0x6ccb","timeout_ms":10000,"last_server_duration_us":337599,"total_duration_us":10002061},{"operation_name":"upsert","last_dispatch_duration_us":54743,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":887086,"total_server_duration_us":502013,"encode_duration_us":16241,"operation_id":"0x6e52","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10001754}],"total_count":16}} WARN [2022-01-10 05:45:42,142] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17052153,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":17052153,"total_server_duration_us":32,"operation_id":"0x6c2a","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":17054081},{"operation_name":"get","last_dispatch_duration_us":12860184,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":12860184,"total_server_duration_us":15,"operation_id":"0x6cf5","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":12861356},{"operation_name":"upsert","last_dispatch_duration_us":2541567,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10860224,"total_server_duration_us":1122582,"encode_duration_us":16422,"operation_id":"0x6df3","timeout_ms":10000,"last_server_duration_us":102642,"total_duration_us":10009851},{"operation_name":"upsert","last_dispatch_duration_us":2920403,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8297428,"total_server_duration_us":1959076,"encode_duration_us":21013,"operation_id":"0x6e85","timeout_ms":10000,"last_server_duration_us":1740446,"total_duration_us":10009384},{"operation_name":"upsert","last_dispatch_duration_us":3503881,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9277294,"total_server_duration_us":666693,"encode_duration_us":23026,"operation_id":"0x6e86","timeout_ms":10000,"last_server_duration_us":116448,"total_duration_us":10008668},{"operation_name":"upsert","last_dispatch_duration_us":2349358,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7958347,"total_server_duration_us":1698189,"encode_duration_us":16615,"operation_id":"0x6e4b","timeout_ms":10000,"last_server_duration_us":356436,"total_duration_us":10007174},{"operation_name":"upsert","last_dispatch_duration_us":137627,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":6972046,"total_server_duration_us":59071,"encode_duration_us":17024,"operation_id":"0x6e37","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10006357},{"operation_name":"upsert","last_dispatch_duration_us":2266767,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":5949720,"total_server_duration_us":808490,"encode_duration_us":16997,"operation_id":"0x6df5","timeout_ms":10000,"last_server_duration_us":239520,"total_duration_us":10006160},{"operation_name":"upsert","last_dispatch_duration_us":2458495,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8200354,"total_server_duration_us":1075902,"encode_duration_us":17507,"operation_id":"0x6df2","timeout_ms":10000,"last_server_duration_us":161069,"total_duration_us":10005708},{"operation_name":"upsert","last_dispatch_duration_us":1867535,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7508430,"total_server_duration_us":797179,"encode_duration_us":16521,"operation_id":"0x6e87","timeout_ms":10000,"last_server_duration_us":98568,"total_duration_us":10005053}],"total_count":23}} INFO [2022-01-10 05:45:42,187] jepsen worker 77 - jepsen.util 3947 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:45:42,187] jepsen worker 77 - jepsen.util 3947 :invoke :write [25 31] INFO [2022-01-10 05:45:42,209] jepsen worker 44 - jepsen.util 3644 :ok :read [14 14] INFO [2022-01-10 05:45:42,210] jepsen worker 44 - jepsen.util 3644 :invoke :write [14 10] INFO [2022-01-10 05:45:42,229] jepsen worker 28 - jepsen.util 3358 :ok :read [9 27] INFO [2022-01-10 05:45:42,230] jepsen worker 28 - jepsen.util 3358 :invoke :read [9 nil] WARN [2022-01-10 05:45:42,238] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9711941,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15317892,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10412955,"encode_duration_us":16663,"operation_id":"0x6bf1","timeout_ms":10000,"last_server_duration_us":9303284,"total_duration_us":10004208},{"operation_name":"upsert","last_dispatch_duration_us":9452590,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18485243,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10134863,"encode_duration_us":17544,"operation_id":"0x6ba1","timeout_ms":10000,"last_server_duration_us":9229268,"total_duration_us":10007949}],"total_count":2}} WARN [2022-01-10 05:45:42,244] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":27086263,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":27086263,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9023573,"encode_duration_us":26646,"operation_id":"0x6b1c","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10010195}],"total_count":1}} WARN [2022-01-10 05:45:42,327] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2140049,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":2140049,"total_server_duration_us":133983,"encode_duration_us":17701,"operation_id":"0x6e4a","timeout_ms":10000,"last_server_duration_us":133983,"total_duration_us":10010634},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","encode_duration_us":19661,"operation_id":"0x6e79","timeout_ms":10000,"total_duration_us":10007916},{"operation_name":"upsert","last_dispatch_duration_us":7831043,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9074731,"total_server_duration_us":250066,"encode_duration_us":16570,"operation_id":"0x6de1","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10007807},{"operation_name":"upsert","last_dispatch_duration_us":7941136,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9118906,"total_server_duration_us":240476,"encode_duration_us":23913,"operation_id":"0x6dcd","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10007423},{"operation_name":"upsert","last_dispatch_duration_us":15218400,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":15218400,"total_server_duration_us":10686247,"encode_duration_us":16992,"operation_id":"0x6d84","timeout_ms":10000,"last_server_duration_us":10686247,"total_duration_us":10007008},{"operation_name":"upsert","last_dispatch_duration_us":10488421,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10488421,"total_server_duration_us":43289,"encode_duration_us":17405,"operation_id":"0x6d9c","timeout_ms":10000,"last_server_duration_us":43289,"total_duration_us":10006130},{"operation_name":"upsert","last_dispatch_duration_us":285002,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9761950,"total_server_duration_us":4920091,"encode_duration_us":19694,"operation_id":"0x6d22","timeout_ms":10000,"last_server_duration_us":149201,"total_duration_us":10006072},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","encode_duration_us":16649,"operation_id":"0x6e88","timeout_ms":10000,"total_duration_us":10005466},{"operation_name":"upsert","last_dispatch_duration_us":2930448,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":12810146,"total_server_duration_us":3548960,"encode_duration_us":18475,"operation_id":"0x6d89","timeout_ms":10000,"last_server_duration_us":214281,"total_duration_us":10005418},{"operation_name":"upsert","last_dispatch_duration_us":4342600,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4342600,"total_server_duration_us":1885135,"encode_duration_us":18662,"operation_id":"0x6d67","timeout_ms":10000,"last_server_duration_us":1885135,"total_duration_us":10003273}],"total_count":16}} INFO [2022-01-10 05:45:42,579] jepsen worker 7 - jepsen.util 3697 :ok :read [2 14] INFO [2022-01-10 05:45:42,579] jepsen worker 7 - jepsen.util 3697 :invoke :write [2 6] INFO [2022-01-10 05:45:42,582] jepsen worker 21 - jepsen.util 3711 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:45:42,583] jepsen worker 21 - jepsen.util 3711 :invoke :read [7 nil] INFO [2022-01-10 05:45:42,586] jepsen worker 42 - jepsen.util 3462 :ok :read [14 14] INFO [2022-01-10 05:45:42,587] jepsen worker 42 - jepsen.util 3462 :invoke :read [14 nil] INFO [2022-01-10 05:45:42,599] jepsen worker 46 - jepsen.util 3646 :ok :read [15 23] INFO [2022-01-10 05:45:42,599] jepsen worker 46 - jepsen.util 3646 :invoke :write [15 45] INFO [2022-01-10 05:45:42,607] jepsen worker 6 - jepsen.util 3606 :ok :read [2 14] INFO [2022-01-10 05:45:42,607] jepsen worker 6 - jepsen.util 3606 :invoke :read [2 nil] INFO [2022-01-10 05:45:42,677] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:42,677] jepsen worker nemesis - couchbase.util Rebalance check count: 492 INFO [2022-01-10 05:45:42,677] jepsen worker nemesis - couchbase.util Rebalance stuck count: 405 INFO [2022-01-10 05:45:42,684] jepsen worker 78 - jepsen.util 4218 :info :write [26 47] :RequestTimeoutException INFO [2022-01-10 05:45:42,685] jepsen worker 78 - jepsen.util 4308 :invoke :write [26 36] INFO [2022-01-10 05:45:43,276] jepsen worker 51 - jepsen.util 4281 :ok :read [17 41] INFO [2022-01-10 05:45:43,277] jepsen worker 51 - jepsen.util 4281 :invoke :write [17 35] INFO [2022-01-10 05:45:43,346] jepsen worker 15 - jepsen.util 3075 :ok :read [5 42] INFO [2022-01-10 05:45:43,523] jepsen worker 66 - jepsen.util 3306 :ok :read [22 41] INFO [2022-01-10 05:45:43,907] jepsen worker 66 - jepsen.util 3306 :invoke :write [22 49] INFO [2022-01-10 05:45:43,671] jepsen worker 43 - jepsen.util 3463 :info :write [14 37] :RequestTimeoutException INFO [2022-01-10 05:45:43,907] jepsen worker 43 - jepsen.util 3553 :invoke :write [14 42] INFO [2022-01-10 05:45:43,683] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:43,737] jepsen worker 28 - jepsen.util 3358 :ok :read [9 27] INFO [2022-01-10 05:45:43,911] jepsen worker nemesis - couchbase.util Rebalance check count: 493 INFO [2022-01-10 05:45:43,775] jepsen worker 17 - jepsen.util 3437 :info :write [5 14] :RequestTimeoutException INFO [2022-01-10 05:45:43,907] jepsen worker 15 - jepsen.util 3075 :invoke :write [5 25] INFO [2022-01-10 05:45:43,911] jepsen worker 28 - jepsen.util 3358 :invoke :write [9 17] INFO [2022-01-10 05:45:43,911] jepsen worker nemesis - couchbase.util Rebalance stuck count: 406 INFO [2022-01-10 05:45:43,911] jepsen worker 17 - jepsen.util 3527 :invoke :read [5 nil] INFO [2022-01-10 05:45:43,923] jepsen worker 23 - jepsen.util 3623 :ok :read [7 :nil] INFO [2022-01-10 05:45:43,924] jepsen worker 23 - jepsen.util 3623 :invoke :read [7 nil] INFO [2022-01-10 05:45:43,965] jepsen worker 50 - jepsen.util 3650 :info :write [16 4] :RequestTimeoutException INFO [2022-01-10 05:45:43,966] jepsen worker 50 - jepsen.util 3740 :invoke :write [16 2] INFO [2022-01-10 05:45:43,967] jepsen worker 47 - jepsen.util 3647 :ok :read [15 23] INFO [2022-01-10 05:45:43,968] jepsen worker 47 - jepsen.util 3647 :invoke :read [15 nil] INFO [2022-01-10 05:45:44,172] jepsen worker 72 - jepsen.util 3852 :info :write [24 32] :RequestTimeoutException INFO [2022-01-10 05:45:44,173] jepsen worker 72 - jepsen.util 3942 :invoke :write [24 36] INFO [2022-01-10 05:45:44,230] jepsen worker 53 - jepsen.util 4283 :info :write [17 43] :RequestTimeoutException INFO [2022-01-10 05:45:44,231] jepsen worker 53 - jepsen.util 4373 :invoke :write [17 30] INFO [2022-01-10 05:45:44,277] jepsen worker 62 - jepsen.util 3932 :ok :read [20 :nil] INFO [2022-01-10 05:45:44,278] jepsen worker 62 - jepsen.util 3932 :invoke :read [20 nil] INFO [2022-01-10 05:45:44,298] jepsen worker 45 - jepsen.util 3825 :ok :read [15 23] INFO [2022-01-10 05:45:44,299] jepsen worker 45 - jepsen.util 3825 :invoke :write [15 20] INFO [2022-01-10 05:45:44,305] jepsen worker 63 - jepsen.util 3483 :ok :read [21 7] INFO [2022-01-10 05:45:44,305] jepsen worker 63 - jepsen.util 3483 :invoke :write [21 41] INFO [2022-01-10 05:45:44,752] jepsen worker 22 - jepsen.util 3262 :ok :read [7 :nil] INFO [2022-01-10 05:45:44,752] jepsen worker 22 - jepsen.util 3262 :invoke :read [7 nil] INFO [2022-01-10 05:45:44,756] jepsen worker 62 - jepsen.util 3932 :ok :read [20 :nil] INFO [2022-01-10 05:45:44,757] jepsen worker 62 - jepsen.util 3932 :invoke :read [20 nil] INFO [2022-01-10 05:45:44,766] jepsen worker 73 - jepsen.util 3223 :ok :read [24 18] INFO [2022-01-10 05:45:44,767] jepsen worker 73 - jepsen.util 3223 :invoke :write [24 36] INFO [2022-01-10 05:45:44,925] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:44,925] jepsen worker nemesis - couchbase.util Rebalance check count: 494 INFO [2022-01-10 05:45:44,925] jepsen worker nemesis - couchbase.util Rebalance stuck count: 407 INFO [2022-01-10 05:45:44,936] jepsen worker 79 - jepsen.util 4129 :info :write [26 12] :RequestTimeoutException INFO [2022-01-10 05:45:44,936] jepsen worker 79 - jepsen.util 4219 :invoke :read [26 nil] INFO [2022-01-10 05:45:45,076] jepsen worker 17 - jepsen.util 3527 :ok :read [5 42] INFO [2022-01-10 05:45:45,077] jepsen worker 17 - jepsen.util 3527 :invoke :read [5 nil] INFO [2022-01-10 05:45:45,278] jepsen worker 33 - jepsen.util 3813 :ok :read [11 21] INFO [2022-01-10 05:45:45,278] jepsen worker 33 - jepsen.util 3813 :invoke :read [11 nil] INFO [2022-01-10 05:45:45,292] jepsen worker 27 - jepsen.util 3627 :info :write [9 49] :RequestTimeoutException INFO [2022-01-10 05:45:45,292] jepsen worker 27 - jepsen.util 3717 :invoke :read [9 nil] INFO [2022-01-10 05:45:45,401] jepsen worker 40 - jepsen.util 2920 :info :write [13 13] :RequestTimeoutException INFO [2022-01-10 05:45:45,402] jepsen worker 40 - jepsen.util 3010 :invoke :write [13 8] INFO [2022-01-10 05:45:45,562] jepsen worker 54 - jepsen.util 3564 :info :write [18 41] :SyncWriteAmbiguous INFO [2022-01-10 05:45:45,563] jepsen worker 54 - jepsen.util 3654 :invoke :write [18 26] INFO [2022-01-10 05:45:45,662] jepsen worker 59 - jepsen.util 3479 :info :write [19 37] :RequestTimeoutException INFO [2022-01-10 05:45:45,662] jepsen worker 59 - jepsen.util 3569 :invoke :write [19 44] INFO [2022-01-10 05:45:45,681] jepsen worker 39 - jepsen.util 4089 :info :write [13 14] :RequestTimeoutException INFO [2022-01-10 05:45:45,682] jepsen worker 39 - jepsen.util 4179 :invoke :write [13 36] INFO [2022-01-10 05:45:45,686] jepsen worker 26 - jepsen.util 3176 :info :write [8 17] :RequestTimeoutException INFO [2022-01-10 05:45:45,687] jepsen worker 26 - jepsen.util 3266 :invoke :read [8 nil] INFO [2022-01-10 05:45:45,696] jepsen worker 8 - jepsen.util 3968 :info :write [2 37] :RequestTimeoutException INFO [2022-01-10 05:45:45,697] jepsen worker 8 - jepsen.util 4058 :invoke :read [2 nil] INFO [2022-01-10 05:45:45,931] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:45,931] jepsen worker nemesis - couchbase.util Rebalance check count: 495 INFO [2022-01-10 05:45:45,931] jepsen worker nemesis - couchbase.util Rebalance stuck count: 408 INFO [2022-01-10 05:45:45,975] jepsen worker 17 - jepsen.util 3527 :ok :read [5 42] INFO [2022-01-10 05:45:45,976] jepsen worker 17 - jepsen.util 3527 :invoke :read [5 nil] INFO [2022-01-10 05:45:45,977] jepsen worker 31 - jepsen.util 3991 :ok :read [10 36] INFO [2022-01-10 05:45:45,978] jepsen worker 31 - jepsen.util 3991 :invoke :read [10 nil] INFO [2022-01-10 05:45:46,011] jepsen worker 22 - jepsen.util 3262 :ok :read [7 :nil] INFO [2022-01-10 05:45:46,011] jepsen worker 62 - jepsen.util 3932 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:45:46,011] jepsen worker 22 - jepsen.util 3262 :invoke :read [7 nil] INFO [2022-01-10 05:45:46,012] jepsen worker 62 - jepsen.util 3932 :invoke :write [20 35] INFO [2022-01-10 05:45:46,079] jepsen worker 31 - jepsen.util 3991 :ok :read [10 36] INFO [2022-01-10 05:45:46,080] jepsen worker 31 - jepsen.util 3991 :invoke :write [10 45] INFO [2022-01-10 05:45:46,212] jepsen worker 64 - jepsen.util 3484 :info :write [21 40] :RequestTimeoutException INFO [2022-01-10 05:45:46,213] jepsen worker 64 - jepsen.util 3574 :invoke :read [21 nil] INFO [2022-01-10 05:45:46,236] jepsen worker 24 - jepsen.util 3174 :info :write [8 23] :RequestTimeoutException INFO [2022-01-10 05:45:46,237] jepsen worker 24 - jepsen.util 3264 :invoke :read [8 nil] INFO [2022-01-10 05:45:46,362] jepsen worker 79 - jepsen.util 4219 :ok :read [26 10] INFO [2022-01-10 05:45:46,363] jepsen worker 79 - jepsen.util 4219 :invoke :write [26 48] INFO [2022-01-10 05:45:46,454] jepsen worker 80 - jepsen.util 4310 :info :write [26 47] :RequestTimeoutException INFO [2022-01-10 05:45:46,455] jepsen worker 80 - jepsen.util 4400 :invoke :read [26 nil] INFO [2022-01-10 05:45:46,575] jepsen worker 5 - jepsen.util 3335 :info :write [1 29] :RequestTimeoutException INFO [2022-01-10 05:45:46,576] jepsen worker 5 - jepsen.util 3425 :invoke :write [1 22] INFO [2022-01-10 05:45:46,582] jepsen worker 80 - jepsen.util 4400 :ok :read [26 10] INFO [2022-01-10 05:45:46,583] jepsen worker 80 - jepsen.util 4400 :invoke :read [26 nil] INFO [2022-01-10 05:45:46,594] jepsen worker 60 - jepsen.util 3840 :info :write [20 14] :RequestTimeoutException INFO [2022-01-10 05:45:46,595] jepsen worker 60 - jepsen.util 3930 :invoke :read [20 nil] INFO [2022-01-10 05:45:46,635] jepsen worker 61 - jepsen.util 3661 :info :write [20 6] :RequestTimeoutException INFO [2022-01-10 05:45:46,636] jepsen worker 61 - jepsen.util 3751 :invoke :read [20 nil] INFO [2022-01-10 05:45:46,646] jepsen worker 87 - jepsen.util 3507 :info :write [29 5] :RequestTimeoutException INFO [2022-01-10 05:45:46,646] jepsen worker 87 - jepsen.util 3597 :invoke :write [29 23] INFO [2022-01-10 05:45:46,660] jepsen worker 2 - jepsen.util 3872 :info :write [0 4] :RequestTimeoutException INFO [2022-01-10 05:45:46,661] jepsen worker 2 - jepsen.util 3962 :invoke :write [0 26] INFO [2022-01-10 05:45:46,687] jepsen worker 80 - jepsen.util 4400 :ok :read [26 10] INFO [2022-01-10 05:45:46,688] jepsen worker 80 - jepsen.util 4400 :invoke :write [26 44] INFO [2022-01-10 05:45:46,701] jepsen worker 52 - jepsen.util 4462 :info :write [17 11] :RequestTimeoutException INFO [2022-01-10 05:45:46,702] jepsen worker 52 - jepsen.util 4552 :invoke :read [17 nil] INFO [2022-01-10 05:45:46,736] jepsen worker 18 - jepsen.util 3348 :info :write [6 37] :RequestTimeoutException INFO [2022-01-10 05:45:46,737] jepsen worker 18 - jepsen.util 3438 :invoke :write [6 5] INFO [2022-01-10 05:45:46,786] jepsen worker 11 - jepsen.util 3431 :info :write [3 25] :RequestTimeoutException INFO [2022-01-10 05:45:46,787] jepsen worker 11 - jepsen.util 3521 :invoke :write [3 35] INFO [2022-01-10 05:45:46,806] jepsen worker 0 - jepsen.util 3330 :info :write [0 27] :RequestTimeoutException INFO [2022-01-10 05:45:46,807] jepsen worker 0 - jepsen.util 3420 :invoke :write [0 36] INFO [2022-01-10 05:45:46,936] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:46,936] jepsen worker nemesis - couchbase.util Rebalance check count: 496 INFO [2022-01-10 05:45:46,936] jepsen worker nemesis - couchbase.util Rebalance stuck count: 409 INFO [2022-01-10 05:45:47,006] jepsen worker 52 - jepsen.util 4552 :ok :read [17 41] INFO [2022-01-10 05:45:47,006] jepsen worker 52 - jepsen.util 4552 :invoke :write [17 10] INFO [2022-01-10 05:45:47,072] jepsen worker 26 - jepsen.util 3266 :ok :read [8 27] INFO [2022-01-10 05:45:47,073] jepsen worker 26 - jepsen.util 3266 :invoke :write [8 32] INFO [2022-01-10 05:45:47,076] jepsen worker 20 - jepsen.util 3260 :info :write [6 18] :RequestTimeoutException INFO [2022-01-10 05:45:47,077] jepsen worker 20 - jepsen.util 3350 :invoke :write [6 21] INFO [2022-01-10 05:45:47,082] jepsen worker 17 - jepsen.util 3527 :ok :read [5 42] INFO [2022-01-10 05:45:47,082] jepsen worker 17 - jepsen.util 3527 :invoke :read [5 nil] INFO [2022-01-10 05:45:47,415] jepsen worker 83 - jepsen.util 3953 :info :write [27 26] :RequestTimeoutException INFO [2022-01-10 05:45:47,416] jepsen worker 83 - jepsen.util 4043 :invoke :write [27 25] INFO [2022-01-10 05:45:47,529] jepsen worker 29 - jepsen.util 3449 :info :write [9 46] :RequestTimeoutException INFO [2022-01-10 05:45:47,530] jepsen worker 29 - jepsen.util 3539 :invoke :write [9 14] INFO [2022-01-10 05:45:47,584] jepsen worker 68 - jepsen.util 3578 :info :write [22 8] :RequestTimeoutException INFO [2022-01-10 05:45:47,585] jepsen worker 68 - jepsen.util 3668 :invoke :write [22 38] INFO [2022-01-10 05:45:47,598] jepsen worker 21 - jepsen.util 3711 :ok :read [7 :nil] INFO [2022-01-10 05:45:47,599] jepsen worker 21 - jepsen.util 3711 :invoke :read [7 nil] INFO [2022-01-10 05:45:47,613] jepsen worker 42 - jepsen.util 3462 :ok :read [14 14] INFO [2022-01-10 05:45:47,613] jepsen worker 42 - jepsen.util 3462 :invoke :read [14 nil] INFO [2022-01-10 05:45:47,621] jepsen worker 23 - jepsen.util 3623 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:45:47,622] jepsen worker 23 - jepsen.util 3623 :invoke :write [7 40] INFO [2022-01-10 05:45:47,628] jepsen worker 65 - jepsen.util 3395 :ok :read [21 7] INFO [2022-01-10 05:45:47,629] jepsen worker 65 - jepsen.util 3395 :invoke :write [21 0] INFO [2022-01-10 05:45:47,638] jepsen worker 6 - jepsen.util 3606 :ok :read [2 14] INFO [2022-01-10 05:45:47,638] jepsen worker 6 - jepsen.util 3606 :invoke :write [2 48] INFO [2022-01-10 05:45:47,719] jepsen worker 22 - jepsen.util 3262 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:45:47,719] jepsen worker 22 - jepsen.util 3262 :invoke :write [7 16] INFO [2022-01-10 05:45:47,736] jepsen worker 86 - jepsen.util 2966 :info :write [28 43] :RequestTimeoutException INFO [2022-01-10 05:45:47,737] jepsen worker 86 - jepsen.util 3056 :invoke :read [28 nil] INFO [2022-01-10 05:45:47,769] jepsen worker 84 - jepsen.util 3504 :ok :read [28 45] INFO [2022-01-10 05:45:47,770] jepsen worker 84 - jepsen.util 3504 :invoke :read [28 nil] INFO [2022-01-10 05:45:47,772] jepsen worker 55 - jepsen.util 3835 :info :write [18 34] :RequestTimeoutException INFO [2022-01-10 05:45:47,772] jepsen worker 55 - jepsen.util 3925 :invoke :write [18 16] INFO [2022-01-10 05:45:47,776] jepsen worker 9 - jepsen.util 4059 :info :write [3 35] :RequestTimeoutException INFO [2022-01-10 05:45:47,776] jepsen worker 9 - jepsen.util 4149 :invoke :read [3 nil] INFO [2022-01-10 05:45:47,905] jepsen worker 58 - jepsen.util 3478 :ok :read [19 31] INFO [2022-01-10 05:45:47,906] jepsen worker 58 - jepsen.util 3478 :invoke :read [19 nil] INFO [2022-01-10 05:45:47,945] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:47,945] jepsen worker nemesis - couchbase.util Rebalance check count: 497 INFO [2022-01-10 05:45:47,945] jepsen worker nemesis - couchbase.util Rebalance stuck count: 410 INFO [2022-01-10 05:45:47,995] jepsen worker 37 - jepsen.util 3277 :info :write [12 39] :RequestTimeoutException INFO [2022-01-10 05:45:47,995] jepsen worker 37 - jepsen.util 3367 :invoke :read [12 nil] INFO [2022-01-10 05:45:48,038] jepsen worker 64 - jepsen.util 3574 :ok :read [21 7] INFO [2022-01-10 05:45:48,038] jepsen worker 64 - jepsen.util 3574 :invoke :write [21 26] INFO [2022-01-10 05:45:48,166] jepsen worker 60 - jepsen.util 3930 :ok :read [20 :nil] INFO [2022-01-10 05:45:48,167] jepsen worker 60 - jepsen.util 3930 :invoke :read [20 nil] INFO [2022-01-10 05:45:48,226] jepsen worker 49 - jepsen.util 3469 :info :write [16 41] :RequestTimeoutException INFO [2022-01-10 05:45:48,227] jepsen worker 49 - jepsen.util 3559 :invoke :write [16 16] INFO [2022-01-10 05:45:48,383] jepsen worker 84 - jepsen.util 3504 :ok :read [28 45] INFO [2022-01-10 05:45:48,383] jepsen worker 84 - jepsen.util 3504 :invoke :read [28 nil] INFO [2022-01-10 05:45:48,391] jepsen worker 58 - jepsen.util 3478 :ok :read [19 31] INFO [2022-01-10 05:45:48,479] jepsen worker 27 - jepsen.util 3717 :ok :read [9 27] INFO [2022-01-10 05:45:48,607] jepsen worker 17 - jepsen.util 3527 :ok :read [5 42] INFO [2022-01-10 05:45:48,638] jepsen worker 58 - jepsen.util 3478 :invoke :write [19 40] INFO [2022-01-10 05:45:48,638] jepsen worker 27 - jepsen.util 3717 :invoke :read [9 nil] INFO [2022-01-10 05:45:48,638] jepsen worker 17 - jepsen.util 3527 :invoke :write [5 44] INFO [2022-01-10 05:45:48,651] jepsen worker 16 - jepsen.util 3616 :info :write [5 17] :RequestTimeoutException INFO [2022-01-10 05:45:48,652] jepsen worker 16 - jepsen.util 3706 :invoke :write [5 38] INFO [2022-01-10 05:45:48,656] jepsen worker 67 - jepsen.util 3397 :info :write [22 46] :RequestTimeoutException INFO [2022-01-10 05:45:48,657] jepsen worker 67 - jepsen.util 3487 :invoke :write [22 16] INFO [2022-01-10 05:45:48,666] jepsen worker 76 - jepsen.util 3046 :info :write [25 2] :RequestTimeoutException INFO [2022-01-10 05:45:48,666] jepsen worker 76 - jepsen.util 3136 :invoke :read [25 nil] INFO [2022-01-10 05:45:48,692] jepsen worker 32 - jepsen.util 3812 :info :write [10 46] :RequestTimeoutException INFO [2022-01-10 05:45:48,692] jepsen worker 32 - jepsen.util 3902 :invoke :read [10 nil] INFO [2022-01-10 05:45:48,827] jepsen worker 13 - jepsen.util 4063 :info :write [4 44] :RequestTimeoutException INFO [2022-01-10 05:45:48,827] jepsen worker 13 - jepsen.util 4153 :invoke :write [4 26] INFO [2022-01-10 05:45:48,851] jepsen worker 71 - jepsen.util 3041 :info :write [23 36] :RequestTimeoutException INFO [2022-01-10 05:45:48,852] jepsen worker 71 - jepsen.util 3131 :invoke :write [23 33] INFO [2022-01-10 05:45:48,931] jepsen worker 69 - jepsen.util 3039 :info :write [23 17] :RequestTimeoutException INFO [2022-01-10 05:45:48,932] jepsen worker 69 - jepsen.util 3129 :invoke :read [23 nil] INFO [2022-01-10 05:45:48,951] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:48,951] jepsen worker nemesis - couchbase.util Rebalance check count: 498 INFO [2022-01-10 05:45:48,951] jepsen worker nemesis - couchbase.util Rebalance stuck count: 411 INFO [2022-01-10 05:45:49,321] jepsen worker 84 - jepsen.util 3504 :ok :read [28 45] INFO [2022-01-10 05:45:49,322] jepsen worker 84 - jepsen.util 3504 :invoke :read [28 nil] INFO [2022-01-10 05:45:49,349] jepsen worker 57 - jepsen.util 4287 :info :write [19 30] :RequestTimeoutException INFO [2022-01-10 05:45:49,350] jepsen worker 57 - jepsen.util 4377 :invoke :read [19 nil] INFO [2022-01-10 05:45:49,723] jepsen worker 27 - jepsen.util 3717 :ok :read [9 27] INFO [2022-01-10 05:45:49,724] jepsen worker 27 - jepsen.util 3717 :invoke :write [9 25] INFO [2022-01-10 05:45:49,766] jepsen worker 89 - jepsen.util 3959 :info :write [29 20] :RequestTimeoutException INFO [2022-01-10 05:45:49,767] jepsen worker 89 - jepsen.util 4049 :invoke :write [29 30] INFO [2022-01-10 05:45:49,792] jepsen worker 24 - jepsen.util 3264 :ok :read [8 27] INFO [2022-01-10 05:45:49,792] jepsen worker 24 - jepsen.util 3264 :invoke :write [8 8] INFO [2022-01-10 05:45:49,799] jepsen worker 38 - jepsen.util 3638 :ok :read [12 29] INFO [2022-01-10 05:45:49,800] jepsen worker 38 - jepsen.util 3638 :invoke :read [12 nil] INFO [2022-01-10 05:45:49,803] jepsen worker 4 - jepsen.util 4144 :info :write [1 16] :SyncWriteAmbiguous INFO [2022-01-10 05:45:49,803] jepsen worker 4 - jepsen.util 4234 :invoke :write [1 18] INFO [2022-01-10 05:45:49,846] jepsen worker 41 - jepsen.util 3371 :info :write [13 39] :RequestTimeoutException INFO [2022-01-10 05:45:49,847] jepsen worker 41 - jepsen.util 3461 :invoke :write [13 32] INFO [2022-01-10 05:45:49,894] jepsen worker 60 - jepsen.util 3930 :ok :read [20 :nil] INFO [2022-01-10 05:45:49,895] jepsen worker 60 - jepsen.util 3930 :invoke :read [20 nil] INFO [2022-01-10 05:45:49,951] jepsen worker 85 - jepsen.util 3955 :info :write [28 27] :RequestTimeoutException INFO [2022-01-10 05:45:49,952] jepsen worker 85 - jepsen.util 4045 :invoke :read [28 nil] INFO [2022-01-10 05:45:49,956] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:49,956] jepsen worker nemesis - couchbase.util Rebalance check count: 499 INFO [2022-01-10 05:45:49,956] jepsen worker nemesis - couchbase.util Rebalance stuck count: 412 INFO [2022-01-10 05:45:50,043] jepsen worker 88 - jepsen.util 3598 :ok :read [29 0] INFO [2022-01-10 05:45:50,044] jepsen worker 88 - jepsen.util 3598 :invoke :write [29 34] INFO [2022-01-10 05:45:50,099] jepsen worker 37 - jepsen.util 3367 :ok :read [12 29] INFO [2022-01-10 05:45:50,099] jepsen worker 37 - jepsen.util 3367 :invoke :write [12 33] INFO [2022-01-10 05:45:50,229] jepsen worker 84 - jepsen.util 3504 :ok :read [28 45] INFO [2022-01-10 05:45:50,230] jepsen worker 84 - jepsen.util 3504 :invoke :write [28 11] INFO [2022-01-10 05:45:50,312] jepsen worker 33 - jepsen.util 3813 :ok :read [11 21] INFO [2022-01-10 05:45:50,313] jepsen worker 85 - jepsen.util 4045 :ok :read [28 45] INFO [2022-01-10 05:45:50,313] jepsen worker 33 - jepsen.util 3813 :invoke :write [11 27] INFO [2022-01-10 05:45:50,314] jepsen worker 85 - jepsen.util 4045 :invoke :read [28 nil] WARN [2022-01-10 05:45:50,352] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9992999,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18171538,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9601339,"encode_duration_us":17905,"operation_id":"0x6e29","timeout_ms":10000,"last_server_duration_us":9458563,"total_duration_us":10004897},{"operation_name":"upsert","last_dispatch_duration_us":20336385,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":20336385,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9028877,"encode_duration_us":17722,"operation_id":"0x6c5f","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10005096},{"operation_name":"upsert","last_dispatch_duration_us":18069551,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18069551,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9023573,"encode_duration_us":17333,"operation_id":"0x6ead","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10006800}],"total_count":3}} WARN [2022-01-10 05:45:50,452] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12611550,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":12611550,"total_server_duration_us":49,"operation_id":"0x6df4","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":12614020},{"operation_name":"get","last_dispatch_duration_us":11251215,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":11251215,"total_server_duration_us":62,"operation_id":"0x6fec","timeout_ms":30000,"last_server_duration_us":62,"total_duration_us":11252188},{"operation_name":"upsert","last_dispatch_duration_us":344654,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6178817,"total_server_duration_us":792523,"encode_duration_us":16552,"operation_id":"0x6eae","timeout_ms":10000,"last_server_duration_us":162603,"total_duration_us":10010088},{"operation_name":"upsert","last_dispatch_duration_us":1612025,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":10918550,"total_server_duration_us":1662604,"encode_duration_us":17275,"operation_id":"0x6ebb","timeout_ms":10000,"last_server_duration_us":275013,"total_duration_us":10009823},{"operation_name":"upsert","last_dispatch_duration_us":9075365,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10002511,"total_server_duration_us":297480,"encode_duration_us":16542,"operation_id":"0x6fa2","timeout_ms":10000,"last_server_duration_us":54094,"total_duration_us":10009546},{"operation_name":"upsert","last_dispatch_duration_us":586076,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5866143,"total_server_duration_us":3913446,"encode_duration_us":19619,"operation_id":"0x6e7e","timeout_ms":10000,"last_server_duration_us":99658,"total_duration_us":10009186},{"operation_name":"upsert","last_dispatch_duration_us":18069551,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":18069551,"total_server_duration_us":9023573,"encode_duration_us":17333,"operation_id":"0x6ead","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10006800},{"operation_name":"upsert","last_dispatch_duration_us":8880534,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10411850,"total_server_duration_us":660772,"encode_duration_us":16781,"operation_id":"0x6f9f","timeout_ms":10000,"last_server_duration_us":85157,"total_duration_us":10006048},{"operation_name":"upsert","last_dispatch_duration_us":9044347,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9044347,"total_server_duration_us":21243,"encode_duration_us":16732,"operation_id":"0x6eac","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10005280},{"operation_name":"upsert","last_dispatch_duration_us":186542,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":9657693,"total_server_duration_us":2780179,"encode_duration_us":16754,"operation_id":"0x6f69","timeout_ms":10000,"last_server_duration_us":174309,"total_duration_us":10005275}],"total_count":18}} INFO [2022-01-10 05:45:50,480] jepsen worker 12 - jepsen.util 3972 :info :write [4 30] :RequestTimeoutException INFO [2022-01-10 05:45:50,481] jepsen worker 12 - jepsen.util 4062 :invoke :read [4 nil] INFO [2022-01-10 05:45:50,611] jepsen worker 61 - jepsen.util 3751 :ok :read [20 :nil] INFO [2022-01-10 05:45:50,611] jepsen worker 61 - jepsen.util 3751 :invoke :read [20 nil] INFO [2022-01-10 05:45:50,621] jepsen worker 47 - jepsen.util 3647 :ok :read [15 23] INFO [2022-01-10 05:45:50,622] jepsen worker 47 - jepsen.util 3647 :invoke :write [15 35] INFO [2022-01-10 05:45:50,639] jepsen worker 14 - jepsen.util 4334 :info :write [4 6] :RequestTimeoutException INFO [2022-01-10 05:45:50,640] jepsen worker 14 - jepsen.util 4424 :invoke :write [4 1] INFO [2022-01-10 05:45:50,667] jepsen worker 8 - jepsen.util 4058 :ok :read [2 14] INFO [2022-01-10 05:45:50,668] jepsen worker 8 - jepsen.util 4058 :invoke :write [2 25] INFO [2022-01-10 05:45:50,736] jepsen worker 3 - jepsen.util 3333 :info :write [1 26] :RequestTimeoutException INFO [2022-01-10 05:45:50,737] jepsen worker 3 - jepsen.util 3423 :invoke :write [1 5] INFO [2022-01-10 05:45:50,876] jepsen worker 76 - jepsen.util 3136 :ok :read [25 :nil] INFO [2022-01-10 05:45:50,877] jepsen worker 76 - jepsen.util 3136 :invoke :write [25 30] INFO [2022-01-10 05:45:50,962] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:50,962] jepsen worker nemesis - couchbase.util Rebalance check count: 500 INFO [2022-01-10 05:45:50,962] jepsen worker nemesis - couchbase.util Rebalance stuck count: 413 INFO [2022-01-10 05:45:50,964] jepsen worker 56 - jepsen.util 3926 :info :write [18 22] :RequestTimeoutException INFO [2022-01-10 05:45:50,965] jepsen worker 56 - jepsen.util 4016 :invoke :write [18 30] INFO [2022-01-10 05:45:51,081] jepsen worker 81 - jepsen.util 3861 :info :write [27 4] :RequestTimeoutException INFO [2022-01-10 05:45:51,082] jepsen worker 81 - jepsen.util 3951 :invoke :read [27 nil] INFO [2022-01-10 05:45:51,212] jepsen worker 12 - jepsen.util 4062 :ok :read [4 3] INFO [2022-01-10 05:45:51,213] jepsen worker 12 - jepsen.util 4062 :invoke :read [4 nil] INFO [2022-01-10 05:45:51,256] jepsen worker 75 - jepsen.util 3135 :info :write [25 47] :RequestTimeoutException INFO [2022-01-10 05:45:51,257] jepsen worker 75 - jepsen.util 3225 :invoke :read [25 nil] INFO [2022-01-10 05:45:51,332] jepsen worker 48 - jepsen.util 3198 :info :write [16 10] :RequestTimeoutException INFO [2022-01-10 05:45:51,332] jepsen worker 48 - jepsen.util 3288 :invoke :write [16 23] INFO [2022-01-10 05:45:51,359] jepsen worker 61 - jepsen.util 3751 :ok :read [20 :nil] INFO [2022-01-10 05:45:51,360] jepsen worker 61 - jepsen.util 3751 :invoke :read [20 nil] INFO [2022-01-10 05:45:51,371] jepsen worker 35 - jepsen.util 3275 :info :write [11 28] :RequestTimeoutException INFO [2022-01-10 05:45:51,371] jepsen worker 35 - jepsen.util 3365 :invoke :read [11 nil] INFO [2022-01-10 05:45:51,407] jepsen worker 86 - jepsen.util 3056 :ok :read [28 45] INFO [2022-01-10 05:45:51,408] jepsen worker 86 - jepsen.util 3056 :invoke :read [28 nil] INFO [2022-01-10 05:45:51,455] jepsen worker 19 - jepsen.util 3709 :info :write [6 15] :RequestTimeoutException INFO [2022-01-10 05:45:51,456] jepsen worker 19 - jepsen.util 3799 :invoke :write [6 6] INFO [2022-01-10 05:45:51,739] jepsen worker 82 - jepsen.util 3772 :info :write [27 15] :RequestTimeoutException INFO [2022-01-10 05:45:51,740] jepsen worker 82 - jepsen.util 3862 :invoke :read [27 nil] INFO [2022-01-10 05:45:51,805] jepsen worker 74 - jepsen.util 4034 :info :write [24 44] :RequestTimeoutException INFO [2022-01-10 05:45:51,805] jepsen worker 74 - jepsen.util 4124 :invoke :write [24 1] INFO [2022-01-10 05:45:51,816] jepsen worker 34 - jepsen.util 3364 :info :write [11 4] :RequestTimeoutException INFO [2022-01-10 05:45:51,816] jepsen worker 34 - jepsen.util 3454 :invoke :write [11 23] INFO [2022-01-10 05:45:51,831] jepsen worker 1 - jepsen.util 3601 :info :write [0 20] :RequestTimeoutException INFO [2022-01-10 05:45:51,832] jepsen worker 1 - jepsen.util 3691 :invoke :read [0 nil] INFO [2022-01-10 05:45:51,842] jepsen worker 75 - jepsen.util 3225 :ok :read [25 :nil] INFO [2022-01-10 05:45:51,843] jepsen worker 75 - jepsen.util 3225 :invoke :read [25 nil] INFO [2022-01-10 05:45:51,961] jepsen worker 25 - jepsen.util 3445 :info :write [8 1] :RequestTimeoutException INFO [2022-01-10 05:45:51,961] jepsen worker 25 - jepsen.util 3535 :invoke :read [8 nil] INFO [2022-01-10 05:45:51,969] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:51,969] jepsen worker nemesis - couchbase.util Rebalance check count: 501 INFO [2022-01-10 05:45:51,969] jepsen worker nemesis - couchbase.util Rebalance stuck count: 414 WARN [2022-01-10 05:45:52,037] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":15136346,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":21175077,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":10388786,"encode_duration_us":16523,"operation_id":"0x6c97","timeout_ms":10000,"last_server_duration_us":9690246,"total_duration_us":10006554},{"operation_name":"upsert","last_dispatch_duration_us":9037806,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9522972,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9274521,"encode_duration_us":18131,"operation_id":"0x6ffb","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10006986}],"total_count":2}} WARN [2022-01-10 05:45:52,038] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14446288,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":14446288,"total_server_duration_us":43,"operation_id":"0x6de5","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":14448197},{"operation_name":"get","last_dispatch_duration_us":11531856,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11531856,"total_server_duration_us":7489,"operation_id":"0x6e7d","timeout_ms":30000,"last_server_duration_us":7489,"total_duration_us":11534013},{"operation_name":"get","last_dispatch_duration_us":10706358,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10706358,"total_server_duration_us":76,"operation_id":"0x6eab","timeout_ms":30000,"last_server_duration_us":76,"total_duration_us":10706682},{"operation_name":"get","last_dispatch_duration_us":10162170,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10162170,"total_server_duration_us":32,"operation_id":"0x6eb9","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":10164285},{"operation_name":"upsert","last_dispatch_duration_us":2206004,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":8796538,"total_server_duration_us":458684,"encode_duration_us":22157,"operation_id":"0x7091","timeout_ms":10000,"last_server_duration_us":40695,"total_duration_us":10008893},{"operation_name":"upsert","last_dispatch_duration_us":1721227,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":6391792,"total_server_duration_us":960797,"encode_duration_us":16371,"operation_id":"0x6f38","timeout_ms":10000,"last_server_duration_us":239067,"total_duration_us":10008503},{"operation_name":"upsert","last_dispatch_duration_us":9037806,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9522972,"total_server_duration_us":9274521,"encode_duration_us":18131,"operation_id":"0x6ffb","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10006986},{"operation_name":"upsert","last_dispatch_duration_us":10246012,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10246012,"total_server_duration_us":308269,"encode_duration_us":17159,"operation_id":"0x6ee4","timeout_ms":10000,"last_server_duration_us":308269,"total_duration_us":10005408},{"operation_name":"upsert","last_dispatch_duration_us":7913593,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":8474742,"total_server_duration_us":311421,"encode_duration_us":17019,"operation_id":"0x6ffd","timeout_ms":10000,"last_server_duration_us":23983,"total_duration_us":10004574},{"operation_name":"upsert","last_dispatch_duration_us":2459585,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9940405,"total_server_duration_us":1151301,"encode_duration_us":16934,"operation_id":"0x6fd3","timeout_ms":10000,"last_server_duration_us":190452,"total_duration_us":10002670}],"total_count":23}} WARN [2022-01-10 05:45:52,061] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2150699,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11903561,"total_server_duration_us":3605073,"encode_duration_us":16515,"operation_id":"0x6f3e","timeout_ms":10000,"last_server_duration_us":239974,"total_duration_us":10010578},{"operation_name":"upsert","last_dispatch_duration_us":2240738,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11310023,"total_server_duration_us":1838376,"encode_duration_us":17104,"operation_id":"0x6f80","timeout_ms":10000,"last_server_duration_us":256073,"total_duration_us":10010525},{"operation_name":"upsert","last_dispatch_duration_us":1588638,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":5386078,"total_server_duration_us":756739,"encode_duration_us":26499,"operation_id":"0x708a","timeout_ms":10000,"last_server_duration_us":122688,"total_duration_us":10009060},{"operation_name":"upsert","last_dispatch_duration_us":2099636,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8163922,"total_server_duration_us":1524149,"encode_duration_us":16518,"operation_id":"0x7026","timeout_ms":10000,"last_server_duration_us":619107,"total_duration_us":10008402},{"operation_name":"upsert","last_dispatch_duration_us":1043920,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9954417,"total_server_duration_us":4523416,"encode_duration_us":19264,"operation_id":"0x6f2e","timeout_ms":10000,"last_server_duration_us":942455,"total_duration_us":10008093},{"operation_name":"upsert","last_dispatch_duration_us":1082523,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10062422,"total_server_duration_us":1061273,"encode_duration_us":17156,"operation_id":"0x6f8e","timeout_ms":10000,"last_server_duration_us":213201,"total_duration_us":10007907},{"operation_name":"upsert","last_dispatch_duration_us":11957911,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":11957911,"total_server_duration_us":9028877,"encode_duration_us":16693,"operation_id":"0x6fb8","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10007517},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":21617,"operation_id":"0x7041","timeout_ms":10000,"total_duration_us":10005257},{"operation_name":"upsert","last_dispatch_duration_us":2735907,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10742368,"total_server_duration_us":2474931,"encode_duration_us":20677,"operation_id":"0x6f3d","timeout_ms":10000,"last_server_duration_us":813376,"total_duration_us":10004544},{"operation_name":"upsert","last_dispatch_duration_us":2243876,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8651542,"total_server_duration_us":1015625,"encode_duration_us":16580,"operation_id":"0x6f66","timeout_ms":10000,"last_server_duration_us":273573,"total_duration_us":10003600}],"total_count":18}} INFO [2022-01-10 05:45:52,112] jepsen worker 12 - jepsen.util 4062 :ok :read [4 3] INFO [2022-01-10 05:45:52,112] jepsen worker 12 - jepsen.util 4062 :invoke :write [4 7] INFO [2022-01-10 05:45:52,136] jepsen worker 81 - jepsen.util 3951 :ok :read [27 30] INFO [2022-01-10 05:45:52,137] jepsen worker 81 - jepsen.util 3951 :invoke :write [27 35] WARN [2022-01-10 05:45:52,140] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18804352,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18804352,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9521414,"encode_duration_us":17568,"operation_id":"0x6e0b","timeout_ms":10000,"last_server_duration_us":9521414,"total_duration_us":10001519},{"operation_name":"upsert","last_dispatch_duration_us":15218400,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":15218400,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":10686247,"encode_duration_us":16992,"operation_id":"0x6d84","timeout_ms":10000,"last_server_duration_us":10686247,"total_duration_us":10007008}],"total_count":2}} WARN [2022-01-10 05:45:52,152] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1382826,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9972281,"total_server_duration_us":775331,"encode_duration_us":17128,"operation_id":"0x705b","timeout_ms":10000,"last_server_duration_us":79829,"total_duration_us":10010298},{"operation_name":"upsert","last_dispatch_duration_us":30351,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":354568,"total_server_duration_us":196580,"encode_duration_us":16254,"operation_id":"0x6fb7","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10008513},{"operation_name":"upsert","last_dispatch_duration_us":3097566,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11750435,"total_server_duration_us":1847381,"encode_duration_us":16067,"operation_id":"0x6f35","timeout_ms":10000,"last_server_duration_us":355096,"total_duration_us":10008459},{"operation_name":"upsert","last_dispatch_duration_us":4807575,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12667215,"total_server_duration_us":1457025,"encode_duration_us":17115,"operation_id":"0x6f0c","timeout_ms":10000,"last_server_duration_us":635833,"total_duration_us":10006229},{"operation_name":"upsert","last_dispatch_duration_us":2931977,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10665547,"total_server_duration_us":1540583,"encode_duration_us":16339,"operation_id":"0x6f3c","timeout_ms":10000,"last_server_duration_us":245901,"total_duration_us":10005960},{"operation_name":"upsert","last_dispatch_duration_us":3141653,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":12540236,"total_server_duration_us":1741180,"encode_duration_us":21159,"operation_id":"0x6f3b","timeout_ms":10000,"last_server_duration_us":179288,"total_duration_us":10005728},{"operation_name":"upsert","last_dispatch_duration_us":1020357,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9985555,"total_server_duration_us":842641,"encode_duration_us":18898,"operation_id":"0x7065","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10005027},{"operation_name":"upsert","last_dispatch_duration_us":5839319,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12878392,"total_server_duration_us":1347974,"encode_duration_us":18256,"operation_id":"0x6f65","timeout_ms":10000,"last_server_duration_us":612673,"total_duration_us":10003147},{"operation_name":"upsert","last_dispatch_duration_us":3824399,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11503645,"total_server_duration_us":892523,"encode_duration_us":18721,"operation_id":"0x6eb1","timeout_ms":10000,"last_server_duration_us":407545,"total_duration_us":10003112},{"operation_name":"upsert","last_dispatch_duration_us":29477,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":61931,"total_server_duration_us":38615,"encode_duration_us":28348,"operation_id":"0x6fd7","timeout_ms":10000,"last_server_duration_us":18570,"total_duration_us":10002822}],"total_count":22}} WARN [2022-01-10 05:45:52,154] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":5354782,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5354782,"total_server_duration_us":158018,"encode_duration_us":23009,"operation_id":"0x6fd5","timeout_ms":10000,"last_server_duration_us":158018,"total_duration_us":10010521},{"operation_name":"upsert","last_dispatch_duration_us":2069834,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7697178,"total_server_duration_us":485678,"encode_duration_us":18257,"operation_id":"0x7064","timeout_ms":10000,"last_server_duration_us":170565,"total_duration_us":10009873},{"operation_name":"upsert","last_dispatch_duration_us":85871,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9663007,"total_server_duration_us":207764,"encode_duration_us":17390,"operation_id":"0x6f6d","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10009021},{"operation_name":"upsert","last_dispatch_duration_us":5176424,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12099055,"total_server_duration_us":604365,"encode_duration_us":20032,"operation_id":"0x6f6f","timeout_ms":10000,"last_server_duration_us":171940,"total_duration_us":10006920},{"operation_name":"upsert","last_dispatch_duration_us":1538748,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10063867,"total_server_duration_us":694334,"encode_duration_us":17177,"operation_id":"0x6fb0","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10006602},{"operation_name":"upsert","last_dispatch_duration_us":1858501,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7417379,"total_server_duration_us":376128,"encode_duration_us":16420,"operation_id":"0x6ef7","timeout_ms":10000,"last_server_duration_us":24325,"total_duration_us":10005643},{"operation_name":"upsert","last_dispatch_duration_us":63849,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9592245,"total_server_duration_us":201920,"encode_duration_us":18032,"operation_id":"0x6f87","timeout_ms":10000,"last_server_duration_us":21977,"total_duration_us":10004763},{"operation_name":"upsert","last_dispatch_duration_us":1127509,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9767552,"total_server_duration_us":1318006,"encode_duration_us":18345,"operation_id":"0x6ff9","timeout_ms":10000,"last_server_duration_us":220369,"total_duration_us":10004619},{"operation_name":"upsert","last_dispatch_duration_us":3114418,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11876450,"total_server_duration_us":626612,"encode_duration_us":94682,"operation_id":"0x6fcd","timeout_ms":10000,"last_server_duration_us":51003,"total_duration_us":10003615},{"operation_name":"upsert","last_dispatch_duration_us":4303401,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":13032498,"total_server_duration_us":387399,"encode_duration_us":16315,"operation_id":"0x6f98","timeout_ms":10000,"last_server_duration_us":93954,"total_duration_us":10003421}],"total_count":18}} INFO [2022-01-10 05:45:52,217] jepsen worker 77 - jepsen.util 3947 :info :write [25 31] :RequestTimeoutException INFO [2022-01-10 05:45:52,218] jepsen worker 77 - jepsen.util 4037 :invoke :read [25 nil] INFO [2022-01-10 05:45:52,237] jepsen worker 44 - jepsen.util 3644 :info :write [14 10] :RequestTimeoutException INFO [2022-01-10 05:45:52,237] jepsen worker 44 - jepsen.util 3734 :invoke :read [14 nil] WARN [2022-01-10 05:45:52,252] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9041869,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9928955,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9522405,"encode_duration_us":16241,"operation_id":"0x6e52","timeout_ms":10000,"last_server_duration_us":9020392,"total_duration_us":10001754}],"total_count":1}} INFO [2022-01-10 05:45:52,258] jepsen worker 57 - jepsen.util 4377 :ok :read [19 31] INFO [2022-01-10 05:45:52,259] jepsen worker 57 - jepsen.util 4377 :invoke :write [19 10] WARN [2022-01-10 05:45:52,262] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11957911,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11957911,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9028877,"encode_duration_us":16693,"operation_id":"0x6fb8","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10007517}],"total_count":1}} WARN [2022-01-10 05:45:52,340] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12206920,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12206920,"total_server_duration_us":17737,"operation_id":"0x6eb0","timeout_ms":30000,"last_server_duration_us":17737,"total_duration_us":12209128},{"operation_name":"get","last_dispatch_duration_us":11375819,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11375819,"total_server_duration_us":4741,"operation_id":"0x6eda","timeout_ms":30000,"last_server_duration_us":4741,"total_duration_us":11376326},{"operation_name":"get","last_dispatch_duration_us":10678310,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10678310,"total_server_duration_us":14162,"operation_id":"0x6ee1","timeout_ms":30000,"last_server_duration_us":14162,"total_duration_us":10680776},{"operation_name":"upsert","last_dispatch_duration_us":2060540,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":11052980,"total_server_duration_us":1498354,"encode_duration_us":16030,"operation_id":"0x7033","timeout_ms":10000,"last_server_duration_us":592209,"total_duration_us":10008031},{"operation_name":"upsert","last_dispatch_duration_us":1683180,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10324833,"total_server_duration_us":3067075,"encode_duration_us":17141,"operation_id":"0x7031","timeout_ms":10000,"last_server_duration_us":547655,"total_duration_us":10007581},{"operation_name":"upsert","last_dispatch_duration_us":9874887,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":12634192,"total_server_duration_us":317367,"encode_duration_us":18030,"operation_id":"0x6f72","timeout_ms":10000,"last_server_duration_us":158779,"total_duration_us":10005905},{"operation_name":"upsert","last_dispatch_duration_us":9602934,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9602934,"total_server_duration_us":115117,"encode_duration_us":17254,"operation_id":"0x707a","timeout_ms":10000,"last_server_duration_us":115117,"total_duration_us":10004580},{"operation_name":"upsert","last_dispatch_duration_us":1733455,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10745068,"total_server_duration_us":3192281,"encode_duration_us":16396,"operation_id":"0x6f0e","timeout_ms":10000,"last_server_duration_us":1060257,"total_duration_us":10002706},{"operation_name":"upsert","last_dispatch_duration_us":3260043,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":12475762,"total_server_duration_us":344030,"encode_duration_us":16168,"operation_id":"0x7005","timeout_ms":10000,"last_server_duration_us":232539,"total_duration_us":10000848},{"operation_name":"upsert","last_dispatch_duration_us":10251918,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":12777925,"total_server_duration_us":346682,"encode_duration_us":16613,"operation_id":"0x6fa1","timeout_ms":10000,"last_server_duration_us":95023,"total_duration_us":10000602}],"total_count":21}} INFO [2022-01-10 05:45:52,427] jepsen worker 75 - jepsen.util 3225 :ok :read [25 :nil] INFO [2022-01-10 05:45:52,429] jepsen worker 75 - jepsen.util 3225 :invoke :write [25 14] INFO [2022-01-10 05:45:52,602] jepsen worker 7 - jepsen.util 3697 :info :write [2 6] :RequestTimeoutException INFO [2022-01-10 05:45:52,603] jepsen worker 7 - jepsen.util 3787 :invoke :read [2 nil] INFO [2022-01-10 05:45:52,621] jepsen worker 46 - jepsen.util 3646 :info :write [15 45] :RequestTimeoutException INFO [2022-01-10 05:45:52,621] jepsen worker 46 - jepsen.util 3736 :invoke :read [15 nil] INFO [2022-01-10 05:45:52,709] jepsen worker 78 - jepsen.util 4308 :info :write [26 36] :RequestTimeoutException INFO [2022-01-10 05:45:52,710] jepsen worker 78 - jepsen.util 4398 :invoke :write [26 0] INFO [2022-01-10 05:45:52,949] jepsen worker 82 - jepsen.util 3862 :ok :read [27 30] INFO [2022-01-10 05:45:52,950] jepsen worker 82 - jepsen.util 3862 :invoke :read [27 nil] INFO [2022-01-10 05:45:52,959] jepsen worker 86 - jepsen.util 3056 :ok :read [28 45] INFO [2022-01-10 05:45:52,960] jepsen worker 86 - jepsen.util 3056 :invoke :read [28 nil] INFO [2022-01-10 05:45:52,975] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:52,975] jepsen worker nemesis - couchbase.util Rebalance check count: 502 INFO [2022-01-10 05:45:52,975] jepsen worker nemesis - couchbase.util Rebalance stuck count: 415 INFO [2022-01-10 05:45:53,081] jepsen worker 61 - jepsen.util 3751 :ok :read [20 :nil] INFO [2022-01-10 05:45:53,081] jepsen worker 61 - jepsen.util 3751 :invoke :read [20 nil] INFO [2022-01-10 05:45:53,113] jepsen worker 46 - jepsen.util 3736 :ok :read [15 23] INFO [2022-01-10 05:45:53,114] jepsen worker 46 - jepsen.util 3736 :invoke :write [15 48] INFO [2022-01-10 05:45:53,741] jepsen worker 82 - jepsen.util 3862 :ok :read [27 30] INFO [2022-01-10 05:45:53,741] jepsen worker 82 - jepsen.util 3862 :invoke :read [27 nil] INFO [2022-01-10 05:45:53,745] jepsen worker 86 - jepsen.util 3056 :ok :read [28 45] INFO [2022-01-10 05:45:53,749] jepsen worker 86 - jepsen.util 3056 :invoke :read [28 nil] INFO [2022-01-10 05:45:53,936] jepsen worker 15 - jepsen.util 3075 :info :write [5 25] :RequestTimeoutException INFO [2022-01-10 05:45:53,936] jepsen worker 51 - jepsen.util 4281 :info :write [17 35] :RequestTimeoutException INFO [2022-01-10 05:45:53,937] jepsen worker 51 - jepsen.util 4371 :invoke :read [17 nil] INFO [2022-01-10 05:45:53,937] jepsen worker 15 - jepsen.util 3165 :invoke :read [5 nil] INFO [2022-01-10 05:45:53,940] jepsen worker 66 - jepsen.util 3306 :info :write [22 49] :RequestTimeoutException INFO [2022-01-10 05:45:53,940] jepsen worker 28 - jepsen.util 3358 :info :write [9 17] :RequestTimeoutException INFO [2022-01-10 05:45:53,941] jepsen worker 66 - jepsen.util 3396 :invoke :read [22 nil] INFO [2022-01-10 05:45:53,941] jepsen worker 28 - jepsen.util 3448 :invoke :read [9 nil] INFO [2022-01-10 05:45:53,941] jepsen worker 43 - jepsen.util 3553 :info :write [14 42] :RequestTimeoutException INFO [2022-01-10 05:45:53,942] jepsen worker 43 - jepsen.util 3643 :invoke :write [14 5] INFO [2022-01-10 05:45:53,981] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:53,981] jepsen worker nemesis - couchbase.util Rebalance check count: 503 INFO [2022-01-10 05:45:53,981] jepsen worker nemesis - couchbase.util Rebalance stuck count: 416 INFO [2022-01-10 05:45:53,991] jepsen worker 50 - jepsen.util 3740 :info :write [16 2] :RequestTimeoutException INFO [2022-01-10 05:45:53,991] jepsen worker 50 - jepsen.util 3830 :invoke :write [16 17] INFO [2022-01-10 05:45:54,015] jepsen worker 1 - jepsen.util 3691 :ok :read [0 46] INFO [2022-01-10 05:45:54,015] jepsen worker 1 - jepsen.util 3691 :invoke :read [0 nil] INFO [2022-01-10 05:45:54,195] jepsen worker 72 - jepsen.util 3942 :info :write [24 36] :RequestTimeoutException INFO [2022-01-10 05:45:54,196] jepsen worker 72 - jepsen.util 4032 :invoke :write [24 44] INFO [2022-01-10 05:45:54,256] jepsen worker 53 - jepsen.util 4373 :info :write [17 30] :RequestTimeoutException INFO [2022-01-10 05:45:54,256] jepsen worker 53 - jepsen.util 4463 :invoke :write [17 38] INFO [2022-01-10 05:45:54,295] jepsen worker 82 - jepsen.util 3862 :ok :read [27 30] INFO [2022-01-10 05:45:54,295] jepsen worker 82 - jepsen.util 3862 :invoke :read [27 nil] INFO [2022-01-10 05:45:54,301] jepsen worker 86 - jepsen.util 3056 :ok :read [28 45] INFO [2022-01-10 05:45:54,302] jepsen worker 86 - jepsen.util 3056 :invoke :read [28 nil] INFO [2022-01-10 05:45:54,320] jepsen worker 45 - jepsen.util 3825 :info :write [15 20] :RequestTimeoutException INFO [2022-01-10 05:45:54,320] jepsen worker 45 - jepsen.util 3915 :invoke :write [15 41] INFO [2022-01-10 05:45:54,329] jepsen worker 63 - jepsen.util 3483 :info :write [21 41] :RequestTimeoutException INFO [2022-01-10 05:45:54,330] jepsen worker 63 - jepsen.util 3573 :invoke :write [21 39] INFO [2022-01-10 05:45:54,350] jepsen worker 15 - jepsen.util 3165 :ok :read [5 42] INFO [2022-01-10 05:45:54,350] jepsen worker 15 - jepsen.util 3165 :invoke :read [5 nil] INFO [2022-01-10 05:45:54,569] jepsen worker 15 - jepsen.util 3165 :ok :read [5 42] INFO [2022-01-10 05:45:54,569] jepsen worker 15 - jepsen.util 3165 :invoke :write [5 9] INFO [2022-01-10 05:45:54,586] jepsen worker 86 - jepsen.util 3056 :ok :read [28 45] INFO [2022-01-10 05:45:54,586] jepsen worker 86 - jepsen.util 3056 :invoke :read [28 nil] INFO [2022-01-10 05:45:54,620] jepsen worker 51 - jepsen.util 4371 :ok :read [17 41] INFO [2022-01-10 05:45:54,621] jepsen worker 51 - jepsen.util 4371 :invoke :write [17 33] INFO [2022-01-10 05:45:54,664] jepsen worker 60 - jepsen.util 3930 :ok :read [20 :nil] INFO [2022-01-10 05:45:54,664] jepsen worker 60 - jepsen.util 3930 :invoke :read [20 nil] INFO [2022-01-10 05:45:54,746] jepsen worker 69 - jepsen.util 3129 :ok :read [23 26] INFO [2022-01-10 05:45:54,746] jepsen worker 69 - jepsen.util 3129 :invoke :read [23 nil] INFO [2022-01-10 05:45:54,786] jepsen worker 69 - jepsen.util 3129 :ok :read [23 26] INFO [2022-01-10 05:45:54,786] jepsen worker 69 - jepsen.util 3129 :invoke :read [23 nil] INFO [2022-01-10 05:45:54,790] jepsen worker 73 - jepsen.util 3223 :info :write [24 36] :RequestTimeoutException INFO [2022-01-10 05:45:54,791] jepsen worker 73 - jepsen.util 3313 :invoke :read [24 nil] INFO [2022-01-10 05:45:54,838] jepsen worker 69 - jepsen.util 3129 :ok :read [23 26] INFO [2022-01-10 05:45:54,839] jepsen worker 69 - jepsen.util 3129 :invoke :read [23 nil] INFO [2022-01-10 05:45:54,903] jepsen worker 69 - jepsen.util 3129 :ok :read [23 26] INFO [2022-01-10 05:45:54,903] jepsen worker 69 - jepsen.util 3129 :invoke :write [23 2] INFO [2022-01-10 05:45:54,988] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:54,989] jepsen worker nemesis - couchbase.util Rebalance check count: 504 INFO [2022-01-10 05:45:54,989] jepsen worker nemesis - couchbase.util Rebalance stuck count: 417 INFO [2022-01-10 05:45:55,006] jepsen worker 1 - jepsen.util 3691 :ok :read [0 46] INFO [2022-01-10 05:45:55,007] jepsen worker 1 - jepsen.util 3691 :invoke :read [0 nil] INFO [2022-01-10 05:45:55,015] jepsen worker 86 - jepsen.util 3056 :ok :read [28 45] INFO [2022-01-10 05:45:55,016] jepsen worker 86 - jepsen.util 3056 :invoke :read [28 nil] INFO [2022-01-10 05:45:55,154] jepsen worker 61 - jepsen.util 3751 :ok :read [20 :nil] INFO [2022-01-10 05:45:55,154] jepsen worker 61 - jepsen.util 3751 :invoke :write [20 0] INFO [2022-01-10 05:45:55,423] jepsen worker 86 - jepsen.util 3056 :ok :read [28 45] INFO [2022-01-10 05:45:55,423] jepsen worker 86 - jepsen.util 3056 :invoke :read [28 nil] INFO [2022-01-10 05:45:55,426] jepsen worker 40 - jepsen.util 3010 :info :write [13 8] :RequestTimeoutException INFO [2022-01-10 05:45:55,427] jepsen worker 40 - jepsen.util 3100 :invoke :write [13 37] INFO [2022-01-10 05:45:55,493] jepsen worker 60 - jepsen.util 3930 :ok :read [20 :nil] INFO [2022-01-10 05:45:55,494] jepsen worker 60 - jepsen.util 3930 :invoke :write [20 35] INFO [2022-01-10 05:45:55,591] jepsen worker 54 - jepsen.util 3654 :info :write [18 26] :RequestTimeoutException INFO [2022-01-10 05:45:55,591] jepsen worker 54 - jepsen.util 3744 :invoke :read [18 nil] INFO [2022-01-10 05:45:55,685] jepsen worker 59 - jepsen.util 3569 :info :write [19 44] :RequestTimeoutException INFO [2022-01-10 05:45:55,686] jepsen worker 59 - jepsen.util 3659 :invoke :read [19 nil] INFO [2022-01-10 05:45:55,706] jepsen worker 39 - jepsen.util 4179 :info :write [13 36] :RequestTimeoutException INFO [2022-01-10 05:45:55,707] jepsen worker 39 - jepsen.util 4269 :invoke :read [13 nil] INFO [2022-01-10 05:45:55,757] jepsen worker 77 - jepsen.util 4037 :ok :read [25 :nil] INFO [2022-01-10 05:45:55,757] jepsen worker 77 - jepsen.util 4037 :invoke :write [25 26] INFO [2022-01-10 05:45:55,910] jepsen worker 70 - jepsen.util 3670 :ok :read [23 26] INFO [2022-01-10 05:45:55,911] jepsen worker 70 - jepsen.util 3670 :invoke :read [23 nil] INFO [2022-01-10 05:45:55,926] jepsen worker 36 - jepsen.util 2736 :ok :read [12 29] INFO [2022-01-10 05:45:55,926] jepsen worker 86 - jepsen.util 3056 :ok :read [28 45] INFO [2022-01-10 05:45:55,927] jepsen worker 36 - jepsen.util 2736 :invoke :write [12 33] INFO [2022-01-10 05:45:55,928] jepsen worker 86 - jepsen.util 3056 :invoke :write [28 20] INFO [2022-01-10 05:45:55,946] jepsen worker 35 - jepsen.util 3365 :ok :read [11 21] INFO [2022-01-10 05:45:55,946] jepsen worker 25 - jepsen.util 3535 :ok :read [8 27] INFO [2022-01-10 05:45:55,947] jepsen worker 35 - jepsen.util 3365 :invoke :read [11 nil] INFO [2022-01-10 05:45:55,947] jepsen worker 25 - jepsen.util 3535 :invoke :read [8 nil] INFO [2022-01-10 05:45:55,995] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:55,995] jepsen worker nemesis - couchbase.util Rebalance check count: 505 INFO [2022-01-10 05:45:55,995] jepsen worker nemesis - couchbase.util Rebalance stuck count: 418 INFO [2022-01-10 05:45:56,005] jepsen worker 59 - jepsen.util 3659 :ok :read [19 31] INFO [2022-01-10 05:45:56,006] jepsen worker 59 - jepsen.util 3659 :invoke :write [19 4] INFO [2022-01-10 05:45:56,040] jepsen worker 62 - jepsen.util 3932 :info :write [20 35] :RequestTimeoutException INFO [2022-01-10 05:45:56,040] jepsen worker 62 - jepsen.util 4022 :invoke :write [20 29] INFO [2022-01-10 05:45:56,083] jepsen worker 1 - jepsen.util 3691 :ok :read [0 46] INFO [2022-01-10 05:45:56,083] jepsen worker 1 - jepsen.util 3691 :invoke :read [0 nil] INFO [2022-01-10 05:45:56,105] jepsen worker 31 - jepsen.util 3991 :info :write [10 45] :RequestTimeoutException INFO [2022-01-10 05:45:56,106] jepsen worker 31 - jepsen.util 4081 :invoke :write [10 11] INFO [2022-01-10 05:45:56,390] jepsen worker 79 - jepsen.util 4219 :info :write [26 48] :RequestTimeoutException INFO [2022-01-10 05:45:56,391] jepsen worker 79 - jepsen.util 4309 :invoke :read [26 nil] INFO [2022-01-10 05:45:56,399] jepsen worker 35 - jepsen.util 3365 :ok :read [11 21] INFO [2022-01-10 05:45:56,400] jepsen worker 35 - jepsen.util 3365 :invoke :write [11 29] INFO [2022-01-10 05:45:56,408] jepsen worker 25 - jepsen.util 3535 :ok :read [8 27] INFO [2022-01-10 05:45:56,408] jepsen worker 25 - jepsen.util 3535 :invoke :write [8 30] INFO [2022-01-10 05:45:56,491] jepsen worker 30 - jepsen.util 3270 :ok :read [10 36] INFO [2022-01-10 05:45:56,491] jepsen worker 30 - jepsen.util 3270 :invoke :write [10 23] INFO [2022-01-10 05:45:56,563] jepsen worker 79 - jepsen.util 4309 :ok :read [26 10] INFO [2022-01-10 05:45:56,564] jepsen worker 79 - jepsen.util 4309 :invoke :read [26 nil] INFO [2022-01-10 05:45:56,606] jepsen worker 5 - jepsen.util 3425 :info :write [1 22] :RequestTimeoutException INFO [2022-01-10 05:45:56,606] jepsen worker 5 - jepsen.util 3515 :invoke :read [1 nil] INFO [2022-01-10 05:45:56,666] jepsen worker 87 - jepsen.util 3597 :info :write [29 23] :RequestTimeoutException INFO [2022-01-10 05:45:56,666] jepsen worker 87 - jepsen.util 3687 :invoke :read [29 nil] INFO [2022-01-10 05:45:56,681] jepsen worker 2 - jepsen.util 3962 :info :write [0 26] :RequestTimeoutException INFO [2022-01-10 05:45:56,682] jepsen worker 2 - jepsen.util 4052 :invoke :read [0 nil] INFO [2022-01-10 05:45:56,715] jepsen worker 80 - jepsen.util 4400 :info :write [26 44] :RequestTimeoutException INFO [2022-01-10 05:45:56,715] jepsen worker 80 - jepsen.util 4490 :invoke :read [26 nil] INFO [2022-01-10 05:45:56,745] jepsen worker 80 - jepsen.util 4490 :ok :read [26 10] INFO [2022-01-10 05:45:56,746] jepsen worker 80 - jepsen.util 4490 :invoke :write [26 33] INFO [2022-01-10 05:45:56,756] jepsen worker 18 - jepsen.util 3438 :info :write [6 5] :RequestTimeoutException INFO [2022-01-10 05:45:56,757] jepsen worker 18 - jepsen.util 3528 :invoke :write [6 38] INFO [2022-01-10 05:45:56,774] jepsen worker 1 - jepsen.util 3691 :ok :read [0 46] INFO [2022-01-10 05:45:56,775] jepsen worker 1 - jepsen.util 3691 :invoke :read [0 nil] INFO [2022-01-10 05:45:56,810] jepsen worker 11 - jepsen.util 3521 :info :write [3 35] :RequestTimeoutException INFO [2022-01-10 05:45:56,811] jepsen worker 11 - jepsen.util 3611 :invoke :read [3 nil] INFO [2022-01-10 05:45:56,832] jepsen worker 0 - jepsen.util 3420 :info :write [0 36] :RequestTimeoutException INFO [2022-01-10 05:45:56,832] jepsen worker 0 - jepsen.util 3510 :invoke :write [0 25] INFO [2022-01-10 05:45:56,996] jepsen worker 79 - jepsen.util 4309 :ok :read [26 10] INFO [2022-01-10 05:45:56,997] jepsen worker 79 - jepsen.util 4309 :invoke :write [26 2] INFO [2022-01-10 05:45:57,001] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:57,001] jepsen worker nemesis - couchbase.util Rebalance check count: 506 INFO [2022-01-10 05:45:57,001] jepsen worker nemesis - couchbase.util Rebalance stuck count: 419 INFO [2022-01-10 05:45:57,025] jepsen worker 52 - jepsen.util 4552 :info :write [17 10] :RequestTimeoutException INFO [2022-01-10 05:45:57,026] jepsen worker 52 - jepsen.util 4642 :invoke :read [17 nil] INFO [2022-01-10 05:45:57,093] jepsen worker 1 - jepsen.util 3691 :ok :read [0 46] INFO [2022-01-10 05:45:57,094] jepsen worker 1 - jepsen.util 3691 :invoke :write [0 5] INFO [2022-01-10 05:45:57,096] jepsen worker 20 - jepsen.util 3350 :info :write [6 21] :RequestTimeoutException INFO [2022-01-10 05:45:57,096] jepsen worker 20 - jepsen.util 3440 :invoke :write [6 0] INFO [2022-01-10 05:45:57,100] jepsen worker 26 - jepsen.util 3266 :info :write [8 32] :RequestTimeoutException INFO [2022-01-10 05:45:57,100] jepsen worker 26 - jepsen.util 3356 :invoke :write [8 37] INFO [2022-01-10 05:45:57,225] jepsen worker 52 - jepsen.util 4642 :ok :read [17 41] INFO [2022-01-10 05:45:57,226] jepsen worker 52 - jepsen.util 4642 :invoke :write [17 4] INFO [2022-01-10 05:45:57,441] jepsen worker 83 - jepsen.util 4043 :info :write [27 25] :RequestTimeoutException INFO [2022-01-10 05:45:57,442] jepsen worker 83 - jepsen.util 4133 :invoke :write [27 5] INFO [2022-01-10 05:45:57,554] jepsen worker 29 - jepsen.util 3539 :info :write [9 14] :RequestTimeoutException INFO [2022-01-10 05:45:57,555] jepsen worker 29 - jepsen.util 3629 :invoke :read [9 nil] INFO [2022-01-10 05:45:57,616] jepsen worker 68 - jepsen.util 3668 :info :write [22 38] :RequestTimeoutException INFO [2022-01-10 05:45:57,616] jepsen worker 68 - jepsen.util 3758 :invoke :write [22 25] INFO [2022-01-10 05:45:57,631] jepsen worker 29 - jepsen.util 3629 :ok :read [9 27] INFO [2022-01-10 05:45:57,631] jepsen worker 29 - jepsen.util 3629 :invoke :read [9 nil] INFO [2022-01-10 05:45:57,641] jepsen worker 23 - jepsen.util 3623 :info :write [7 40] :RequestTimeoutException INFO [2022-01-10 05:45:57,642] jepsen worker 23 - jepsen.util 3713 :invoke :write [7 4] INFO [2022-01-10 05:45:57,655] jepsen worker 65 - jepsen.util 3395 :info :write [21 0] :RequestTimeoutException INFO [2022-01-10 05:45:57,656] jepsen worker 65 - jepsen.util 3485 :invoke :read [21 nil] INFO [2022-01-10 05:45:57,661] jepsen worker 6 - jepsen.util 3606 :info :write [2 48] :RequestTimeoutException INFO [2022-01-10 05:45:57,661] jepsen worker 6 - jepsen.util 3696 :invoke :read [2 nil] INFO [2022-01-10 05:45:57,720] jepsen worker 29 - jepsen.util 3629 :ok :read [9 27] INFO [2022-01-10 05:45:57,721] jepsen worker 29 - jepsen.util 3629 :invoke :read [9 nil] INFO [2022-01-10 05:45:57,740] jepsen worker 22 - jepsen.util 3262 :info :write [7 16] :RequestTimeoutException INFO [2022-01-10 05:45:57,740] jepsen worker 22 - jepsen.util 3352 :invoke :write [7 7] INFO [2022-01-10 05:45:57,802] jepsen worker 55 - jepsen.util 3925 :info :write [18 16] :RequestTimeoutException INFO [2022-01-10 05:45:57,802] jepsen worker 55 - jepsen.util 4015 :invoke :write [18 29] INFO [2022-01-10 05:45:58,007] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:58,007] jepsen worker nemesis - couchbase.util Rebalance check count: 507 INFO [2022-01-10 05:45:58,007] jepsen worker nemesis - couchbase.util Rebalance stuck count: 420 INFO [2022-01-10 05:45:58,067] jepsen worker 64 - jepsen.util 3574 :info :write [21 26] :RequestTimeoutException INFO [2022-01-10 05:45:58,067] jepsen worker 64 - jepsen.util 3664 :invoke :write [21 46] INFO [2022-01-10 05:45:58,255] jepsen worker 49 - jepsen.util 3559 :info :write [16 16] :RequestTimeoutException INFO [2022-01-10 05:45:58,256] jepsen worker 49 - jepsen.util 3649 :invoke :read [16 nil] INFO [2022-01-10 05:45:58,447] jepsen worker 21 - jepsen.util 3711 :ok :read [7 :nil] INFO [2022-01-10 05:45:58,448] jepsen worker 21 - jepsen.util 3711 :invoke :read [7 nil] INFO [2022-01-10 05:45:58,526] jepsen worker 42 - jepsen.util 3462 :ok :read [14 14] INFO [2022-01-10 05:45:58,527] jepsen worker 42 - jepsen.util 3462 :invoke :write [14 8] INFO [2022-01-10 05:45:58,664] jepsen worker 58 - jepsen.util 3478 :info :write [19 40] :RequestTimeoutException INFO [2022-01-10 05:45:58,665] jepsen worker 58 - jepsen.util 3568 :invoke :read [19 nil] INFO [2022-01-10 05:45:58,666] jepsen worker 17 - jepsen.util 3527 :info :write [5 44] :RequestTimeoutException INFO [2022-01-10 05:45:58,667] jepsen worker 17 - jepsen.util 3617 :invoke :read [5 nil] INFO [2022-01-10 05:45:58,676] jepsen worker 16 - jepsen.util 3706 :info :write [5 38] :RequestTimeoutException INFO [2022-01-10 05:45:58,676] jepsen worker 16 - jepsen.util 3796 :invoke :read [5 nil] INFO [2022-01-10 05:45:58,679] jepsen worker 67 - jepsen.util 3487 :info :write [22 16] :RequestTimeoutException INFO [2022-01-10 05:45:58,680] jepsen worker 67 - jepsen.util 3577 :invoke :read [22 nil] INFO [2022-01-10 05:45:58,851] jepsen worker 13 - jepsen.util 4153 :info :write [4 26] :RequestTimeoutException INFO [2022-01-10 05:45:58,851] jepsen worker 13 - jepsen.util 4243 :invoke :read [4 nil] INFO [2022-01-10 05:45:58,880] jepsen worker 5 - jepsen.util 3515 :ok :read [1 26] INFO [2022-01-10 05:45:58,880] jepsen worker 5 - jepsen.util 3515 :invoke :read [1 nil] INFO [2022-01-10 05:45:58,881] jepsen worker 54 - jepsen.util 3744 :ok :read [18 14] INFO [2022-01-10 05:45:58,881] jepsen worker 54 - jepsen.util 3744 :invoke :read [18 nil] INFO [2022-01-10 05:45:58,884] jepsen worker 71 - jepsen.util 3131 :info :write [23 33] :RequestTimeoutException INFO [2022-01-10 05:45:58,885] jepsen worker 71 - jepsen.util 3221 :invoke :write [23 43] INFO [2022-01-10 05:45:58,904] jepsen worker 5 - jepsen.util 3515 :ok :read [1 26] INFO [2022-01-10 05:45:58,905] jepsen worker 5 - jepsen.util 3515 :invoke :read [1 nil] INFO [2022-01-10 05:45:58,910] jepsen worker 54 - jepsen.util 3744 :ok :read [18 14] INFO [2022-01-10 05:45:58,911] jepsen worker 54 - jepsen.util 3744 :invoke :write [18 4] INFO [2022-01-10 05:45:58,927] jepsen worker 5 - jepsen.util 3515 :ok :read [1 26] INFO [2022-01-10 05:45:58,928] jepsen worker 5 - jepsen.util 3515 :invoke :write [1 5] INFO [2022-01-10 05:45:59,013] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:45:59,013] jepsen worker nemesis - couchbase.util Rebalance check count: 508 INFO [2022-01-10 05:45:59,013] jepsen worker nemesis - couchbase.util Rebalance stuck count: 421 INFO [2022-01-10 05:45:59,077] jepsen worker 13 - jepsen.util 4243 :ok :read [4 3] INFO [2022-01-10 05:45:59,078] jepsen worker 13 - jepsen.util 4243 :invoke :read [4 nil] INFO [2022-01-10 05:45:59,217] jepsen worker 21 - jepsen.util 3711 :ok :read [7 :nil] INFO [2022-01-10 05:45:59,217] jepsen worker 21 - jepsen.util 3711 :invoke :read [7 nil] INFO [2022-01-10 05:45:59,373] jepsen worker 85 - jepsen.util 4045 :ok :read [28 45] INFO [2022-01-10 05:45:59,374] jepsen worker 85 - jepsen.util 4045 :invoke :read [28 nil] INFO [2022-01-10 05:45:59,582] jepsen worker 6 - jepsen.util 3696 :ok :read [2 14] INFO [2022-01-10 05:45:59,583] jepsen worker 6 - jepsen.util 3696 :invoke :write [2 49] INFO [2022-01-10 05:45:59,595] jepsen worker 7 - jepsen.util 3787 :ok :read [2 14] INFO [2022-01-10 05:45:59,595] jepsen worker 7 - jepsen.util 3787 :invoke :read [2 nil] INFO [2022-01-10 05:45:59,752] jepsen worker 27 - jepsen.util 3717 :info :write [9 25] :RequestTimeoutException INFO [2022-01-10 05:45:59,752] jepsen worker 44 - jepsen.util 3734 :ok :read [14 14] INFO [2022-01-10 05:45:59,752] jepsen worker 27 - jepsen.util 3807 :invoke :write [9 26] INFO [2022-01-10 05:45:59,753] jepsen worker 44 - jepsen.util 3734 :invoke :write [14 43] INFO [2022-01-10 05:45:59,792] jepsen worker 89 - jepsen.util 4049 :info :write [29 30] :RequestTimeoutException INFO [2022-01-10 05:45:59,792] jepsen worker 89 - jepsen.util 4139 :invoke :read [29 nil] INFO [2022-01-10 05:45:59,815] jepsen worker 89 - jepsen.util 4139 :ok :read [29 0] INFO [2022-01-10 05:45:59,816] jepsen worker 89 - jepsen.util 4139 :invoke :write [29 14] INFO [2022-01-10 05:45:59,821] jepsen worker 24 - jepsen.util 3264 :info :write [8 8] :RequestTimeoutException INFO [2022-01-10 05:45:59,821] jepsen worker 24 - jepsen.util 3354 :invoke :write [8 39] INFO [2022-01-10 05:45:59,826] jepsen worker 4 - jepsen.util 4234 :info :write [1 18] :RequestTimeoutException INFO [2022-01-10 05:45:59,827] jepsen worker 4 - jepsen.util 4324 :invoke :read [1 nil] INFO [2022-01-10 05:45:59,871] jepsen worker 41 - jepsen.util 3461 :info :write [13 32] :RequestTimeoutException INFO [2022-01-10 05:45:59,872] jepsen worker 41 - jepsen.util 3551 :invoke :read [13 nil] INFO [2022-01-10 05:45:59,883] jepsen worker 13 - jepsen.util 4243 :ok :read [4 3] INFO [2022-01-10 05:45:59,884] jepsen worker 13 - jepsen.util 4243 :invoke :read [4 nil] INFO [2022-01-10 05:45:59,986] jepsen worker 13 - jepsen.util 4243 :ok :read [4 3] INFO [2022-01-10 05:45:59,986] jepsen worker 13 - jepsen.util 4243 :invoke :write [4 3] INFO [2022-01-10 05:46:00,019] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:00,019] jepsen worker nemesis - couchbase.util Rebalance check count: 509 INFO [2022-01-10 05:46:00,019] jepsen worker nemesis - couchbase.util Rebalance stuck count: 422 INFO [2022-01-10 05:46:00,065] jepsen worker 88 - jepsen.util 3598 :info :write [29 34] :RequestTimeoutException INFO [2022-01-10 05:46:00,066] jepsen worker 88 - jepsen.util 3688 :invoke :read [29 nil] INFO [2022-01-10 05:46:00,131] jepsen worker 37 - jepsen.util 3367 :info :write [12 33] :RequestTimeoutException INFO [2022-01-10 05:46:00,132] jepsen worker 37 - jepsen.util 3457 :invoke :read [12 nil] INFO [2022-01-10 05:46:00,166] jepsen worker 21 - jepsen.util 3711 :ok :read [7 :nil] INFO [2022-01-10 05:46:00,167] jepsen worker 21 - jepsen.util 3711 :invoke :write [7 23] INFO [2022-01-10 05:46:00,255] jepsen worker 84 - jepsen.util 3504 :info :write [28 11] :RequestTimeoutException INFO [2022-01-10 05:46:00,256] jepsen worker 84 - jepsen.util 3594 :invoke :write [28 35] INFO [2022-01-10 05:46:00,353] jepsen worker 33 - jepsen.util 3813 :info :write [11 27] :RequestTimeoutException INFO [2022-01-10 05:46:00,353] jepsen worker 33 - jepsen.util 3903 :invoke :write [11 17] WARN [2022-01-10 05:46:00,366] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":21025063,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":21025063,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9022513,"encode_duration_us":16553,"operation_id":"0x6fc6","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10004325},{"operation_name":"upsert","last_dispatch_duration_us":9558817,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10570734,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9324755,"encode_duration_us":17935,"operation_id":"0x71c7","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10007695}],"total_count":2}} INFO [2022-01-10 05:46:00,390] jepsen worker 7 - jepsen.util 3787 :ok :read [2 14] INFO [2022-01-10 05:46:00,391] jepsen worker 7 - jepsen.util 3787 :invoke :write [2 7] WARN [2022-01-10 05:46:00,466] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9100908,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11012851,"total_server_duration_us":229799,"encode_duration_us":16589,"operation_id":"0x71c3","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10010460},{"operation_name":"upsert","last_dispatch_duration_us":704565,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9648870,"total_server_duration_us":2834970,"encode_duration_us":18147,"operation_id":"0x706b","timeout_ms":10000,"last_server_duration_us":431415,"total_duration_us":10010039},{"operation_name":"upsert","last_dispatch_duration_us":191439,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":5796701,"total_server_duration_us":3867266,"encode_duration_us":18008,"operation_id":"0x7168","timeout_ms":10000,"last_server_duration_us":107914,"total_duration_us":10008435},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":16979,"operation_id":"0x7223","timeout_ms":10000,"total_duration_us":10007865},{"operation_name":"upsert","last_dispatch_duration_us":9558817,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10570734,"total_server_duration_us":9324755,"encode_duration_us":17935,"operation_id":"0x71c7","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10007695},{"operation_name":"upsert","last_dispatch_duration_us":9092412,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11728340,"total_server_duration_us":529366,"encode_duration_us":16235,"operation_id":"0x718f","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10007359},{"operation_name":"upsert","last_dispatch_duration_us":9015219,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13651076,"total_server_duration_us":1053215,"encode_duration_us":16138,"operation_id":"0x712a","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10005522},{"operation_name":"upsert","last_dispatch_duration_us":258375,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8271896,"total_server_duration_us":1963852,"encode_duration_us":16261,"operation_id":"0x70f8","timeout_ms":10000,"last_server_duration_us":65253,"total_duration_us":10005403},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":16311,"operation_id":"0x721b","timeout_ms":10000,"total_duration_us":10004660},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":27367,"operation_id":"0x71eb","timeout_ms":10000,"total_duration_us":10004233}],"total_count":16}} INFO [2022-01-10 05:46:00,571] jepsen worker 65 - jepsen.util 3485 :ok :read [21 7] INFO [2022-01-10 05:46:00,572] jepsen worker 65 - jepsen.util 3485 :invoke :read [21 nil] INFO [2022-01-10 05:46:00,652] jepsen worker 47 - jepsen.util 3647 :info :write [15 35] :RequestTimeoutException INFO [2022-01-10 05:46:00,653] jepsen worker 47 - jepsen.util 3737 :invoke :read [15 nil] INFO [2022-01-10 05:46:00,659] jepsen worker 14 - jepsen.util 4424 :info :write [4 1] :RequestTimeoutException INFO [2022-01-10 05:46:00,660] jepsen worker 14 - jepsen.util 4514 :invoke :read [4 nil] INFO [2022-01-10 05:46:00,692] jepsen worker 8 - jepsen.util 4058 :info :write [2 25] :RequestTimeoutException INFO [2022-01-10 05:46:00,692] jepsen worker 8 - jepsen.util 4148 :invoke :read [2 nil] INFO [2022-01-10 05:46:00,762] jepsen worker 3 - jepsen.util 3423 :info :write [1 5] :RequestTimeoutException INFO [2022-01-10 05:46:00,762] jepsen worker 3 - jepsen.util 3513 :invoke :write [1 30] INFO [2022-01-10 05:46:00,883] jepsen worker 65 - jepsen.util 3485 :ok :read [21 7] INFO [2022-01-10 05:46:00,883] jepsen worker 65 - jepsen.util 3485 :invoke :write [21 11] INFO [2022-01-10 05:46:00,902] jepsen worker 76 - jepsen.util 3136 :info :write [25 30] :RequestTimeoutException INFO [2022-01-10 05:46:00,902] jepsen worker 76 - jepsen.util 3226 :invoke :read [25 nil] INFO [2022-01-10 05:46:00,986] jepsen worker 56 - jepsen.util 4016 :info :write [18 30] :RequestTimeoutException INFO [2022-01-10 05:46:00,988] jepsen worker 56 - jepsen.util 4106 :invoke :read [18 nil] INFO [2022-01-10 05:46:01,025] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:01,025] jepsen worker nemesis - couchbase.util Rebalance check count: 510 INFO [2022-01-10 05:46:01,025] jepsen worker nemesis - couchbase.util Rebalance stuck count: 423 INFO [2022-01-10 05:46:01,030] jepsen worker 14 - jepsen.util 4514 :ok :read [4 3] INFO [2022-01-10 05:46:01,031] jepsen worker 14 - jepsen.util 4514 :invoke :read [4 nil] INFO [2022-01-10 05:46:01,195] jepsen worker 14 - jepsen.util 4514 :ok :read [4 3] INFO [2022-01-10 05:46:01,195] jepsen worker 14 - jepsen.util 4514 :invoke :write [4 29] INFO [2022-01-10 05:46:01,356] jepsen worker 48 - jepsen.util 3288 :info :write [16 23] :RequestTimeoutException INFO [2022-01-10 05:46:01,356] jepsen worker 48 - jepsen.util 3378 :invoke :write [16 37] INFO [2022-01-10 05:46:01,482] jepsen worker 19 - jepsen.util 3799 :info :write [6 6] :RequestTimeoutException INFO [2022-01-10 05:46:01,482] jepsen worker 19 - jepsen.util 3889 :invoke :write [6 32] INFO [2022-01-10 05:46:01,694] jepsen worker 76 - jepsen.util 3226 :ok :read [25 :nil] INFO [2022-01-10 05:46:01,695] jepsen worker 76 - jepsen.util 3226 :invoke :write [25 22] INFO [2022-01-10 05:46:01,831] jepsen worker 74 - jepsen.util 4124 :info :write [24 1] :RequestTimeoutException INFO [2022-01-10 05:46:01,831] jepsen worker 74 - jepsen.util 4214 :invoke :write [24 10] INFO [2022-01-10 05:46:01,844] jepsen worker 34 - jepsen.util 3454 :info :write [11 23] :RequestTimeoutException INFO [2022-01-10 05:46:01,845] jepsen worker 34 - jepsen.util 3544 :invoke :read [11 nil] INFO [2022-01-10 05:46:01,961] jepsen worker 8 - jepsen.util 4148 :ok :read [2 14] INFO [2022-01-10 05:46:01,962] jepsen worker 8 - jepsen.util 4148 :invoke :write [2 35] WARN [2022-01-10 05:46:01,967] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9890617,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17506380,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":8343179,"encode_duration_us":18465,"operation_id":"0x70ec","timeout_ms":10000,"last_server_duration_us":7722594,"total_duration_us":10006244},{"operation_name":"upsert","last_dispatch_duration_us":17886900,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":24858947,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9089009,"encode_duration_us":17024,"operation_id":"0x6e37","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10006357}],"total_count":2}} INFO [2022-01-10 05:46:02,032] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:02,032] jepsen worker nemesis - couchbase.util Rebalance check count: 511 INFO [2022-01-10 05:46:02,032] jepsen worker nemesis - couchbase.util Rebalance stuck count: 424 INFO [2022-01-10 05:46:02,042] jepsen worker 75 - jepsen.util 3225 :info :write [25 14] :SyncWriteAmbiguous INFO [2022-01-10 05:46:02,043] jepsen worker 75 - jepsen.util 3315 :invoke :write [25 38] WARN [2022-01-10 05:46:02,052] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10896527,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10896527,"total_server_duration_us":6827,"operation_id":"0x7192","timeout_ms":30000,"last_server_duration_us":6827,"total_duration_us":10898488},{"operation_name":"get","last_dispatch_duration_us":10846767,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10846767,"total_server_duration_us":1161,"operation_id":"0x7190","timeout_ms":30000,"last_server_duration_us":1161,"total_duration_us":10847065},{"operation_name":"upsert","last_dispatch_duration_us":1151667,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":6879202,"total_server_duration_us":560702,"encode_duration_us":23008,"operation_id":"0x70ed","timeout_ms":10000,"last_server_duration_us":111000,"total_duration_us":10010245},{"operation_name":"upsert","last_dispatch_duration_us":33156,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":963039,"total_server_duration_us":528791,"encode_duration_us":21140,"operation_id":"0x721c","timeout_ms":10000,"last_server_duration_us":20841,"total_duration_us":10010185},{"operation_name":"upsert","last_dispatch_duration_us":35383,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":5155964,"total_server_duration_us":414051,"encode_duration_us":17135,"operation_id":"0x7129","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10009580},{"operation_name":"upsert","last_dispatch_duration_us":769156,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":769156,"total_server_duration_us":131690,"encode_duration_us":18878,"operation_id":"0x7216","timeout_ms":10000,"last_server_duration_us":131690,"total_duration_us":10009029},{"operation_name":"upsert","last_dispatch_duration_us":776485,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":2999109,"total_server_duration_us":211603,"encode_duration_us":16390,"operation_id":"0x718e","timeout_ms":10000,"last_server_duration_us":117953,"total_duration_us":10008500},{"operation_name":"upsert","last_dispatch_duration_us":1093533,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7699638,"total_server_duration_us":909909,"encode_duration_us":21031,"operation_id":"0x70ee","timeout_ms":10000,"last_server_duration_us":104387,"total_duration_us":10008279},{"operation_name":"upsert","last_dispatch_duration_us":12250453,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":16480170,"total_server_duration_us":9450529,"encode_duration_us":17203,"operation_id":"0x7094","timeout_ms":10000,"last_server_duration_us":9126707,"total_duration_us":10007670},{"operation_name":"upsert","last_dispatch_duration_us":1128835,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1128835,"total_server_duration_us":19262,"encode_duration_us":16866,"operation_id":"0x7279","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10007486}],"total_count":24}} WARN [2022-01-10 05:46:02,052] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12250453,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16480170,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9450529,"encode_duration_us":17203,"operation_id":"0x7094","timeout_ms":10000,"last_server_duration_us":9126707,"total_duration_us":10007670}],"total_count":1}} WARN [2022-01-10 05:46:02,076] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14982323,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":14982323,"total_server_duration_us":27,"operation_id":"0x7057","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":14983677},{"operation_name":"get","last_dispatch_duration_us":13976641,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":13976641,"total_server_duration_us":15,"operation_id":"0x7093","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":13977847},{"operation_name":"upsert","last_dispatch_duration_us":9034611,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":14803209,"total_server_duration_us":948564,"encode_duration_us":18145,"operation_id":"0x709e","timeout_ms":10000,"last_server_duration_us":77851,"total_duration_us":10010961},{"operation_name":"upsert","last_dispatch_duration_us":9203621,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11424274,"total_server_duration_us":603128,"encode_duration_us":17061,"operation_id":"0x71bb","timeout_ms":10000,"last_server_duration_us":216230,"total_duration_us":10010444},{"operation_name":"upsert","last_dispatch_duration_us":545915,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":5471896,"total_server_duration_us":345676,"encode_duration_us":17539,"operation_id":"0x71c6","timeout_ms":10000,"last_server_duration_us":41123,"total_duration_us":10009943},{"operation_name":"upsert","last_dispatch_duration_us":1033873,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":7622411,"total_server_duration_us":832318,"encode_duration_us":16061,"operation_id":"0x7128","timeout_ms":10000,"last_server_duration_us":128022,"total_duration_us":10007442},{"operation_name":"upsert","last_dispatch_duration_us":10684372,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":13802649,"total_server_duration_us":9997456,"encode_duration_us":18512,"operation_id":"0x709f","timeout_ms":10000,"last_server_duration_us":9583354,"total_duration_us":10007107},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":18986,"operation_id":"0x7217","timeout_ms":10000,"total_duration_us":10002746},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":16384,"operation_id":"0x715f","timeout_ms":10000,"total_duration_us":10002394},{"operation_name":"upsert","last_dispatch_duration_us":8717005,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11020238,"total_server_duration_us":715289,"encode_duration_us":16951,"operation_id":"0x7187","timeout_ms":10000,"last_server_duration_us":66957,"total_duration_us":10001430}],"total_count":16}} INFO [2022-01-10 05:46:02,135] jepsen worker 12 - jepsen.util 4062 :info :write [4 7] :RequestTimeoutException INFO [2022-01-10 05:46:02,136] jepsen worker 12 - jepsen.util 4152 :invoke :read [4 nil] WARN [2022-01-10 05:46:02,154] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12040904,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12040904,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9034183,"encode_duration_us":16369,"operation_id":"0x717c","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10005748}],"total_count":1}} INFO [2022-01-10 05:46:02,161] jepsen worker 81 - jepsen.util 3951 :info :write [27 35] :RequestTimeoutException INFO [2022-01-10 05:46:02,161] jepsen worker 81 - jepsen.util 4041 :invoke :write [27 30] WARN [2022-01-10 05:46:02,165] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14569966,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":14569966,"total_server_duration_us":43,"operation_id":"0x7092","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":14570958},{"operation_name":"upsert","last_dispatch_duration_us":1301519,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9610339,"total_server_duration_us":738401,"encode_duration_us":28919,"operation_id":"0x7225","timeout_ms":10000,"last_server_duration_us":112804,"total_duration_us":10009444},{"operation_name":"upsert","last_dispatch_duration_us":1245165,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8514131,"total_server_duration_us":830656,"encode_duration_us":20132,"operation_id":"0x7228","timeout_ms":10000,"last_server_duration_us":169390,"total_duration_us":10009231},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":20738,"operation_id":"0x71b2","timeout_ms":10000,"total_duration_us":10007773},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":16687,"operation_id":"0x723d","timeout_ms":10000,"total_duration_us":10007306},{"operation_name":"upsert","last_dispatch_duration_us":1604633,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10647829,"total_server_duration_us":1269034,"encode_duration_us":16896,"operation_id":"0x724a","timeout_ms":10000,"last_server_duration_us":163951,"total_duration_us":10007228},{"operation_name":"upsert","last_dispatch_duration_us":260640,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8056192,"total_server_duration_us":947348,"encode_duration_us":17126,"operation_id":"0x7183","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10007084},{"operation_name":"upsert","last_dispatch_duration_us":182166,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":5321780,"total_server_duration_us":646533,"encode_duration_us":20414,"operation_id":"0x7207","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10006979},{"operation_name":"upsert","last_dispatch_duration_us":1182062,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6120826,"total_server_duration_us":217095,"encode_duration_us":19753,"operation_id":"0x7259","timeout_ms":10000,"last_server_duration_us":60506,"total_duration_us":10005436},{"operation_name":"upsert","last_dispatch_duration_us":1890740,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11349980,"total_server_duration_us":622436,"encode_duration_us":16624,"operation_id":"0x70a3","timeout_ms":10000,"last_server_duration_us":74372,"total_duration_us":10005422}],"total_count":17}} WARN [2022-01-10 05:46:02,167] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1081672,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9968242,"total_server_duration_us":1039501,"encode_duration_us":21093,"operation_id":"0x70eb","timeout_ms":10000,"last_server_duration_us":162796,"total_duration_us":10009351},{"operation_name":"upsert","last_dispatch_duration_us":33365,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8358549,"total_server_duration_us":255284,"encode_duration_us":19602,"operation_id":"0x715d","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10008979},{"operation_name":"upsert","last_dispatch_duration_us":461874,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8159554,"total_server_duration_us":1297030,"encode_duration_us":17441,"operation_id":"0x7194","timeout_ms":10000,"last_server_duration_us":232763,"total_duration_us":10008695},{"operation_name":"upsert","last_dispatch_duration_us":31898,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8540480,"total_server_duration_us":236762,"encode_duration_us":17259,"operation_id":"0x7150","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10007884},{"operation_name":"upsert","last_dispatch_duration_us":66302,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9397271,"total_server_duration_us":380899,"encode_duration_us":22129,"operation_id":"0x7191","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10007670},{"operation_name":"upsert","last_dispatch_duration_us":917484,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":10002649,"total_server_duration_us":1457878,"encode_duration_us":16460,"operation_id":"0x70f9","timeout_ms":10000,"last_server_duration_us":33918,"total_duration_us":10007237},{"operation_name":"upsert","last_dispatch_duration_us":9890617,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":17506380,"total_server_duration_us":8343179,"encode_duration_us":18465,"operation_id":"0x70ec","timeout_ms":10000,"last_server_duration_us":7722594,"total_duration_us":10006244},{"operation_name":"upsert","last_dispatch_duration_us":84556,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":5803474,"total_server_duration_us":533655,"encode_duration_us":16838,"operation_id":"0x7255","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10005861},{"operation_name":"upsert","last_dispatch_duration_us":86934,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8678077,"total_server_duration_us":516532,"encode_duration_us":17026,"operation_id":"0x71c9","timeout_ms":10000,"last_server_duration_us":20045,"total_duration_us":10005791},{"operation_name":"upsert","last_dispatch_duration_us":1029229,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10507931,"total_server_duration_us":481174,"encode_duration_us":20842,"operation_id":"0x7133","timeout_ms":10000,"last_server_duration_us":21568,"total_duration_us":10002507}],"total_count":17}} INFO [2022-01-10 05:46:02,170] jepsen worker 12 - jepsen.util 4152 :ok :read [4 3] INFO [2022-01-10 05:46:02,170] jepsen worker 12 - jepsen.util 4152 :invoke :write [4 18] WARN [2022-01-10 05:46:02,265] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17650084,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17712015,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9067492,"encode_duration_us":28348,"operation_id":"0x6fd7","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10002822},{"operation_name":"upsert","last_dispatch_duration_us":17669417,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18023986,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9227579,"encode_duration_us":16254,"operation_id":"0x6fb7","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10008513}],"total_count":2}} WARN [2022-01-10 05:46:02,277] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":15142105,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15142105,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":6088255,"encode_duration_us":21617,"operation_id":"0x7041","timeout_ms":10000,"last_server_duration_us":6088255,"total_duration_us":10005257},{"operation_name":"upsert","last_dispatch_duration_us":10684372,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13802649,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9997456,"encode_duration_us":18512,"operation_id":"0x709f","timeout_ms":10000,"last_server_duration_us":9583354,"total_duration_us":10007107}],"total_count":2}} INFO [2022-01-10 05:46:02,280] jepsen worker 57 - jepsen.util 4377 :info :write [19 10] :RequestTimeoutException INFO [2022-01-10 05:46:02,281] jepsen worker 57 - jepsen.util 4467 :invoke :read [19 nil] WARN [2022-01-10 05:46:02,354] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":546725,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10006455,"total_server_duration_us":1191111,"encode_duration_us":16932,"operation_id":"0x714f","timeout_ms":10000,"last_server_duration_us":126291,"total_duration_us":10010033},{"operation_name":"upsert","last_dispatch_duration_us":637066,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9119674,"total_server_duration_us":1040745,"encode_duration_us":19342,"operation_id":"0x7186","timeout_ms":10000,"last_server_duration_us":125257,"total_duration_us":10007527},{"operation_name":"upsert","last_dispatch_duration_us":1133650,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10418374,"total_server_duration_us":1886638,"encode_duration_us":17417,"operation_id":"0x70fc","timeout_ms":10000,"last_server_duration_us":127155,"total_duration_us":10006210},{"operation_name":"upsert","last_dispatch_duration_us":1253700,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10387936,"total_server_duration_us":1961353,"encode_duration_us":16245,"operation_id":"0x7111","timeout_ms":10000,"last_server_duration_us":124227,"total_duration_us":10006078},{"operation_name":"upsert","last_dispatch_duration_us":12040904,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":12040904,"total_server_duration_us":9034183,"encode_duration_us":16369,"operation_id":"0x717c","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10005748},{"operation_name":"upsert","last_dispatch_duration_us":911964,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10780999,"total_server_duration_us":1087441,"encode_duration_us":17973,"operation_id":"0x70ac","timeout_ms":10000,"last_server_duration_us":52896,"total_duration_us":10005542},{"operation_name":"upsert","last_dispatch_duration_us":483197,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7232984,"total_server_duration_us":835246,"encode_duration_us":19508,"operation_id":"0x71ca","timeout_ms":10000,"last_server_duration_us":67353,"total_duration_us":10002269},{"operation_name":"upsert","last_dispatch_duration_us":463351,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9345928,"total_server_duration_us":1921665,"encode_duration_us":18161,"operation_id":"0x71a3","timeout_ms":10000,"last_server_duration_us":66168,"total_duration_us":10001944},{"operation_name":"upsert","last_dispatch_duration_us":310768,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":8344121,"total_server_duration_us":1450482,"encode_duration_us":16807,"operation_id":"0x7229","timeout_ms":10000,"last_server_duration_us":58750,"total_duration_us":10001926},{"operation_name":"upsert","last_dispatch_duration_us":249939,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9885412,"total_server_duration_us":1128719,"encode_duration_us":24684,"operation_id":"0x7159","timeout_ms":10000,"last_server_duration_us":82691,"total_duration_us":10001817}],"total_count":23}} INFO [2022-01-10 05:46:02,480] jepsen worker 34 - jepsen.util 3544 :ok :read [11 21] INFO [2022-01-10 05:46:02,481] jepsen worker 34 - jepsen.util 3544 :invoke :write [11 5] INFO [2022-01-10 05:46:02,735] jepsen worker 78 - jepsen.util 4398 :info :write [26 0] :RequestTimeoutException INFO [2022-01-10 05:46:02,736] jepsen worker 78 - jepsen.util 4488 :invoke :read [26 nil] INFO [2022-01-10 05:46:02,828] jepsen worker 66 - jepsen.util 3396 :ok :read [22 41] INFO [2022-01-10 05:46:02,828] jepsen worker 66 - jepsen.util 3396 :invoke :write [22 36] INFO [2022-01-10 05:46:02,888] jepsen worker 82 - jepsen.util 3862 :ok :read [27 30] INFO [2022-01-10 05:46:02,889] jepsen worker 82 - jepsen.util 3862 :invoke :read [27 nil] INFO [2022-01-10 05:46:02,895] jepsen worker 39 - jepsen.util 4269 :ok :read [13 15] INFO [2022-01-10 05:46:02,895] jepsen worker 39 - jepsen.util 4269 :invoke :write [13 13] INFO [2022-01-10 05:46:02,920] jepsen worker 73 - jepsen.util 3313 :ok :read [24 18] INFO [2022-01-10 05:46:02,921] jepsen worker 73 - jepsen.util 3313 :invoke :read [24 nil] INFO [2022-01-10 05:46:02,953] jepsen worker 78 - jepsen.util 4488 :ok :read [26 10] INFO [2022-01-10 05:46:02,954] jepsen worker 78 - jepsen.util 4488 :invoke :write [26 6] INFO [2022-01-10 05:46:02,959] jepsen worker 28 - jepsen.util 3448 :ok :read [9 27] INFO [2022-01-10 05:46:02,959] jepsen worker 28 - jepsen.util 3448 :invoke :write [9 15] INFO [2022-01-10 05:46:03,037] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:03,037] jepsen worker nemesis - couchbase.util Rebalance check count: 512 INFO [2022-01-10 05:46:03,037] jepsen worker nemesis - couchbase.util Rebalance stuck count: 425 INFO [2022-01-10 05:46:03,134] jepsen worker 46 - jepsen.util 3736 :info :write [15 48] :RequestTimeoutException INFO [2022-01-10 05:46:03,135] jepsen worker 46 - jepsen.util 3826 :invoke :read [15 nil] INFO [2022-01-10 05:46:03,617] jepsen worker 73 - jepsen.util 3313 :ok :read [24 18] INFO [2022-01-10 05:46:03,617] jepsen worker 73 - jepsen.util 3313 :invoke :write [24 21] INFO [2022-01-10 05:46:03,965] jepsen worker 43 - jepsen.util 3643 :info :write [14 5] :RequestTimeoutException INFO [2022-01-10 05:46:03,965] jepsen worker 43 - jepsen.util 3733 :invoke :read [14 nil] INFO [2022-01-10 05:46:04,015] jepsen worker 50 - jepsen.util 3830 :info :write [16 17] :RequestTimeoutException INFO [2022-01-10 05:46:04,022] jepsen worker 50 - jepsen.util 3920 :invoke :read [16 nil] INFO [2022-01-10 05:46:04,046] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:04,046] jepsen worker nemesis - couchbase.util Rebalance check count: 513 INFO [2022-01-10 05:46:04,046] jepsen worker nemesis - couchbase.util Rebalance stuck count: 426 INFO [2022-01-10 05:46:04,194] jepsen worker 46 - jepsen.util 3826 :ok :read [15 23] INFO [2022-01-10 05:46:04,195] jepsen worker 46 - jepsen.util 3826 :invoke :write [15 49] INFO [2022-01-10 05:46:04,220] jepsen worker 72 - jepsen.util 4032 :info :write [24 44] :RequestTimeoutException INFO [2022-01-10 05:46:04,220] jepsen worker 72 - jepsen.util 4122 :invoke :write [24 14] INFO [2022-01-10 05:46:04,282] jepsen worker 53 - jepsen.util 4463 :info :write [17 38] :RequestTimeoutException INFO [2022-01-10 05:46:04,282] jepsen worker 53 - jepsen.util 4553 :invoke :read [17 nil] INFO [2022-01-10 05:46:04,343] jepsen worker 53 - jepsen.util 4553 :ok :read [17 41] INFO [2022-01-10 05:46:04,344] jepsen worker 53 - jepsen.util 4553 :invoke :read [17 nil] INFO [2022-01-10 05:46:04,346] jepsen worker 45 - jepsen.util 3915 :info :write [15 41] :RequestTimeoutException INFO [2022-01-10 05:46:04,347] jepsen worker 45 - jepsen.util 4005 :invoke :read [15 nil] INFO [2022-01-10 05:46:04,351] jepsen worker 63 - jepsen.util 3573 :info :write [21 39] :RequestTimeoutException INFO [2022-01-10 05:46:04,352] jepsen worker 63 - jepsen.util 3663 :invoke :read [21 nil] INFO [2022-01-10 05:46:04,400] jepsen worker 53 - jepsen.util 4553 :ok :read [17 41] INFO [2022-01-10 05:46:04,401] jepsen worker 53 - jepsen.util 4553 :invoke :write [17 35] INFO [2022-01-10 05:46:04,502] jepsen worker 43 - jepsen.util 3733 :ok :read [14 14] INFO [2022-01-10 05:46:04,502] jepsen worker 43 - jepsen.util 3733 :invoke :read [14 nil] INFO [2022-01-10 05:46:04,592] jepsen worker 15 - jepsen.util 3165 :info :write [5 9] :RequestTimeoutException INFO [2022-01-10 05:46:04,592] jepsen worker 15 - jepsen.util 3255 :invoke :read [5 nil] INFO [2022-01-10 05:46:04,640] jepsen worker 51 - jepsen.util 4371 :info :write [17 33] :RequestTimeoutException INFO [2022-01-10 05:46:04,641] jepsen worker 51 - jepsen.util 4461 :invoke :read [17 nil] INFO [2022-01-10 05:46:04,729] jepsen worker 51 - jepsen.util 4461 :ok :read [17 41] INFO [2022-01-10 05:46:04,729] jepsen worker 51 - jepsen.util 4461 :invoke :write [17 24] INFO [2022-01-10 05:46:04,926] jepsen worker 69 - jepsen.util 3129 :info :write [23 2] :RequestTimeoutException INFO [2022-01-10 05:46:04,927] jepsen worker 69 - jepsen.util 3219 :invoke :write [23 14] INFO [2022-01-10 05:46:04,973] jepsen worker 9 - jepsen.util 4149 :ok :read [3 31] INFO [2022-01-10 05:46:04,975] jepsen worker 9 - jepsen.util 4149 :invoke :write [3 23] INFO [2022-01-10 05:46:04,987] jepsen worker 32 - jepsen.util 3902 :ok :read [10 36] INFO [2022-01-10 05:46:04,988] jepsen worker 32 - jepsen.util 3902 :invoke :write [10 19] INFO [2022-01-10 05:46:04,999] jepsen worker 38 - jepsen.util 3638 :ok :read [12 29] INFO [2022-01-10 05:46:05,000] jepsen worker 38 - jepsen.util 3638 :invoke :write [12 13] INFO [2022-01-10 05:46:05,053] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:05,054] jepsen worker nemesis - couchbase.util Rebalance check count: 514 INFO [2022-01-10 05:46:05,054] jepsen worker nemesis - couchbase.util Rebalance stuck count: 427 INFO [2022-01-10 05:46:05,081] jepsen worker 70 - jepsen.util 3670 :ok :read [23 26] INFO [2022-01-10 05:46:05,082] jepsen worker 70 - jepsen.util 3670 :invoke :write [23 3] INFO [2022-01-10 05:46:05,131] jepsen worker 37 - jepsen.util 3457 :ok :read [12 29] INFO [2022-01-10 05:46:05,131] jepsen worker 37 - jepsen.util 3457 :invoke :write [12 14] INFO [2022-01-10 05:46:05,155] jepsen worker 47 - jepsen.util 3737 :ok :read [15 23] INFO [2022-01-10 05:46:05,156] jepsen worker 47 - jepsen.util 3737 :invoke :read [15 nil] INFO [2022-01-10 05:46:05,162] jepsen worker 43 - jepsen.util 3733 :ok :read [14 14] INFO [2022-01-10 05:46:05,162] jepsen worker 43 - jepsen.util 3733 :invoke :read [14 nil] INFO [2022-01-10 05:46:05,181] jepsen worker 11 - jepsen.util 3611 :ok :read [3 31] INFO [2022-01-10 05:46:05,182] jepsen worker 61 - jepsen.util 3751 :info :write [20 0] :RequestTimeoutException INFO [2022-01-10 05:46:05,182] jepsen worker 11 - jepsen.util 3611 :invoke :write [3 16] INFO [2022-01-10 05:46:05,182] jepsen worker 61 - jepsen.util 3841 :invoke :write [20 31] INFO [2022-01-10 05:46:05,277] jepsen worker 4 - jepsen.util 4324 :ok :read [1 26] INFO [2022-01-10 05:46:05,278] jepsen worker 4 - jepsen.util 4324 :invoke :write [1 42] INFO [2022-01-10 05:46:05,338] jepsen worker 29 - jepsen.util 3629 :ok :read [9 27] INFO [2022-01-10 05:46:05,341] jepsen worker 29 - jepsen.util 3629 :invoke :write [9 20] INFO [2022-01-10 05:46:05,377] jepsen worker 47 - jepsen.util 3737 :ok :read [15 23] INFO [2022-01-10 05:46:05,378] jepsen worker 47 - jepsen.util 3737 :invoke :read [15 nil] INFO [2022-01-10 05:46:05,446] jepsen worker 40 - jepsen.util 3100 :info :write [13 37] :RequestTimeoutException INFO [2022-01-10 05:46:05,447] jepsen worker 40 - jepsen.util 3190 :invoke :read [13 nil] INFO [2022-01-10 05:46:05,520] jepsen worker 60 - jepsen.util 3930 :info :write [20 35] :RequestTimeoutException INFO [2022-01-10 05:46:05,520] jepsen worker 60 - jepsen.util 4020 :invoke :read [20 nil] INFO [2022-01-10 05:46:05,522] jepsen worker 63 - jepsen.util 3663 :ok :read [21 7] INFO [2022-01-10 05:46:05,523] jepsen worker 63 - jepsen.util 3663 :invoke :read [21 nil] INFO [2022-01-10 05:46:05,549] jepsen worker 47 - jepsen.util 3737 :ok :read [15 23] INFO [2022-01-10 05:46:05,550] jepsen worker 47 - jepsen.util 3737 :invoke :read [15 nil] INFO [2022-01-10 05:46:05,612] jepsen worker 43 - jepsen.util 3733 :ok :read [14 14] INFO [2022-01-10 05:46:05,613] jepsen worker 43 - jepsen.util 3733 :invoke :read [14 nil] INFO [2022-01-10 05:46:05,782] jepsen worker 77 - jepsen.util 4037 :info :write [25 26] :RequestTimeoutException INFO [2022-01-10 05:46:05,783] jepsen worker 77 - jepsen.util 4127 :invoke :write [25 38] INFO [2022-01-10 05:46:05,799] jepsen worker 49 - jepsen.util 3649 :ok :read [16 38] INFO [2022-01-10 05:46:05,800] jepsen worker 49 - jepsen.util 3649 :invoke :write [16 7] INFO [2022-01-10 05:46:05,824] jepsen worker 40 - jepsen.util 3190 :ok :read [13 15] INFO [2022-01-10 05:46:05,824] jepsen worker 40 - jepsen.util 3190 :invoke :write [13 44] INFO [2022-01-10 05:46:05,859] jepsen worker 67 - jepsen.util 3577 :ok :read [22 41] INFO [2022-01-10 05:46:05,860] jepsen worker 67 - jepsen.util 3577 :invoke :write [22 5] INFO [2022-01-10 05:46:05,927] jepsen worker 47 - jepsen.util 3737 :ok :read [15 23] INFO [2022-01-10 05:46:05,928] jepsen worker 47 - jepsen.util 3737 :invoke :read [15 nil] INFO [2022-01-10 05:46:05,946] jepsen worker 36 - jepsen.util 2736 :info :write [12 33] :RequestTimeoutException INFO [2022-01-10 05:46:05,948] jepsen worker 36 - jepsen.util 2826 :invoke :write [12 48] INFO [2022-01-10 05:46:05,960] jepsen worker 86 - jepsen.util 3056 :info :write [28 20] :RequestTimeoutException INFO [2022-01-10 05:46:05,961] jepsen worker 86 - jepsen.util 3146 :invoke :read [28 nil] INFO [2022-01-10 05:46:06,032] jepsen worker 59 - jepsen.util 3659 :info :write [19 4] :RequestTimeoutException INFO [2022-01-10 05:46:06,033] jepsen worker 59 - jepsen.util 3749 :invoke :write [19 43] INFO [2022-01-10 05:46:06,060] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:06,060] jepsen worker nemesis - couchbase.util Rebalance check count: 515 INFO [2022-01-10 05:46:06,060] jepsen worker nemesis - couchbase.util Rebalance stuck count: 428 INFO [2022-01-10 05:46:06,061] jepsen worker 62 - jepsen.util 4022 :info :write [20 29] :RequestTimeoutException INFO [2022-01-10 05:46:06,061] jepsen worker 62 - jepsen.util 4112 :invoke :write [20 11] INFO [2022-01-10 05:46:06,134] jepsen worker 31 - jepsen.util 4081 :info :write [10 11] :RequestTimeoutException INFO [2022-01-10 05:46:06,135] jepsen worker 31 - jepsen.util 4171 :invoke :read [10 nil] INFO [2022-01-10 05:46:06,227] jepsen worker 41 - jepsen.util 3551 :ok :read [13 15] INFO [2022-01-10 05:46:06,228] jepsen worker 41 - jepsen.util 3551 :invoke :write [13 3] INFO [2022-01-10 05:46:06,249] jepsen worker 57 - jepsen.util 4467 :ok :read [19 31] INFO [2022-01-10 05:46:06,250] jepsen worker 57 - jepsen.util 4467 :invoke :read [19 nil] INFO [2022-01-10 05:46:06,378] jepsen worker 60 - jepsen.util 4020 :ok :read [20 :nil] INFO [2022-01-10 05:46:06,379] jepsen worker 60 - jepsen.util 4020 :invoke :read [20 nil] INFO [2022-01-10 05:46:06,397] jepsen worker 63 - jepsen.util 3663 :ok :read [21 7] INFO [2022-01-10 05:46:06,397] jepsen worker 63 - jepsen.util 3663 :invoke :write [21 22] INFO [2022-01-10 05:46:06,420] jepsen worker 35 - jepsen.util 3365 :info :write [11 29] :RequestTimeoutException INFO [2022-01-10 05:46:06,420] jepsen worker 35 - jepsen.util 3455 :invoke :write [11 37] INFO [2022-01-10 05:46:06,423] jepsen worker 47 - jepsen.util 3737 :ok :read [15 23] INFO [2022-01-10 05:46:06,425] jepsen worker 47 - jepsen.util 3737 :invoke :write [15 11] INFO [2022-01-10 05:46:06,430] jepsen worker 25 - jepsen.util 3535 :info :write [8 30] :RequestTimeoutException INFO [2022-01-10 05:46:06,431] jepsen worker 25 - jepsen.util 3625 :invoke :read [8 nil] INFO [2022-01-10 05:46:06,502] jepsen worker 43 - jepsen.util 3733 :ok :read [14 14] INFO [2022-01-10 05:46:06,502] jepsen worker 43 - jepsen.util 3733 :invoke :read [14 nil] INFO [2022-01-10 05:46:06,512] jepsen worker 30 - jepsen.util 3270 :info :write [10 23] :RequestTimeoutException INFO [2022-01-10 05:46:06,513] jepsen worker 30 - jepsen.util 3360 :invoke :write [10 9] INFO [2022-01-10 05:46:06,771] jepsen worker 80 - jepsen.util 4490 :info :write [26 33] :RequestTimeoutException INFO [2022-01-10 05:46:06,771] jepsen worker 80 - jepsen.util 4580 :invoke :write [26 14] INFO [2022-01-10 05:46:06,775] jepsen worker 18 - jepsen.util 3528 :info :write [6 38] :RequestTimeoutException INFO [2022-01-10 05:46:06,775] jepsen worker 18 - jepsen.util 3618 :invoke :write [6 45] INFO [2022-01-10 05:46:06,832] jepsen worker 86 - jepsen.util 3146 :ok :read [28 45] INFO [2022-01-10 05:46:06,832] jepsen worker 86 - jepsen.util 3146 :invoke :read [28 nil] INFO [2022-01-10 05:46:06,859] jepsen worker 0 - jepsen.util 3510 :info :write [0 25] :RequestTimeoutException INFO [2022-01-10 05:46:06,860] jepsen worker 0 - jepsen.util 3600 :invoke :read [0 nil] INFO [2022-01-10 05:46:06,915] jepsen worker 60 - jepsen.util 4020 :ok :read [20 :nil] INFO [2022-01-10 05:46:06,915] jepsen worker 60 - jepsen.util 4020 :invoke :read [20 nil] INFO [2022-01-10 05:46:06,949] jepsen worker 57 - jepsen.util 4467 :ok :read [19 31] INFO [2022-01-10 05:46:06,949] jepsen worker 57 - jepsen.util 4467 :invoke :read [19 nil] INFO [2022-01-10 05:46:07,026] jepsen worker 79 - jepsen.util 4309 :info :write [26 2] :RequestTimeoutException INFO [2022-01-10 05:46:07,026] jepsen worker 79 - jepsen.util 4399 :invoke :read [26 nil] INFO [2022-01-10 05:46:07,065] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:07,065] jepsen worker nemesis - couchbase.util Rebalance check count: 516 INFO [2022-01-10 05:46:07,065] jepsen worker nemesis - couchbase.util Rebalance stuck count: 429 INFO [2022-01-10 05:46:07,116] jepsen worker 1 - jepsen.util 3691 :info :write [0 5] :RequestTimeoutException INFO [2022-01-10 05:46:07,116] jepsen worker 20 - jepsen.util 3440 :info :write [6 0] :RequestTimeoutException INFO [2022-01-10 05:46:07,117] jepsen worker 20 - jepsen.util 3530 :invoke :read [6 nil] INFO [2022-01-10 05:46:07,117] jepsen worker 1 - jepsen.util 3781 :invoke :read [0 nil] INFO [2022-01-10 05:46:07,131] jepsen worker 26 - jepsen.util 3356 :info :write [8 37] :RequestTimeoutException INFO [2022-01-10 05:46:07,132] jepsen worker 26 - jepsen.util 3446 :invoke :write [8 43] INFO [2022-01-10 05:46:07,252] jepsen worker 52 - jepsen.util 4642 :info :write [17 4] :RequestTimeoutException INFO [2022-01-10 05:46:07,253] jepsen worker 52 - jepsen.util 4732 :invoke :read [17 nil] INFO [2022-01-10 05:46:07,304] jepsen worker 79 - jepsen.util 4399 :ok :read [26 10] INFO [2022-01-10 05:46:07,305] jepsen worker 79 - jepsen.util 4399 :invoke :write [26 40] INFO [2022-01-10 05:46:07,408] jepsen worker 86 - jepsen.util 3146 :ok :read [28 45] INFO [2022-01-10 05:46:07,409] jepsen worker 86 - jepsen.util 3146 :invoke :write [28 26] INFO [2022-01-10 05:46:07,419] jepsen worker 52 - jepsen.util 4732 :ok :read [17 41] INFO [2022-01-10 05:46:07,420] jepsen worker 52 - jepsen.util 4732 :invoke :write [17 16] INFO [2022-01-10 05:46:07,422] jepsen worker 57 - jepsen.util 4467 :ok :read [19 31] INFO [2022-01-10 05:46:07,423] jepsen worker 57 - jepsen.util 4467 :invoke :write [19 6] INFO [2022-01-10 05:46:07,464] jepsen worker 83 - jepsen.util 4133 :info :write [27 5] :RequestTimeoutException INFO [2022-01-10 05:46:07,466] jepsen worker 83 - jepsen.util 4223 :invoke :write [27 32] INFO [2022-01-10 05:46:07,523] jepsen worker 1 - jepsen.util 3781 :ok :read [0 46] INFO [2022-01-10 05:46:07,524] jepsen worker 1 - jepsen.util 3781 :invoke :write [0 23] INFO [2022-01-10 05:46:07,640] jepsen worker 68 - jepsen.util 3758 :info :write [22 25] :RequestTimeoutException INFO [2022-01-10 05:46:07,640] jepsen worker 68 - jepsen.util 3848 :invoke :read [22 nil] INFO [2022-01-10 05:46:07,662] jepsen worker 23 - jepsen.util 3713 :info :write [7 4] :RequestTimeoutException INFO [2022-01-10 05:46:07,662] jepsen worker 23 - jepsen.util 3803 :invoke :write [7 44] INFO [2022-01-10 05:46:07,765] jepsen worker 22 - jepsen.util 3352 :info :write [7 7] :RequestTimeoutException INFO [2022-01-10 05:46:07,765] jepsen worker 22 - jepsen.util 3442 :invoke :write [7 47] INFO [2022-01-10 05:46:07,826] jepsen worker 55 - jepsen.util 4015 :info :write [18 29] :RequestTimeoutException INFO [2022-01-10 05:46:07,826] jepsen worker 55 - jepsen.util 4105 :invoke :write [18 24] INFO [2022-01-10 05:46:08,072] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:08,072] jepsen worker nemesis - couchbase.util Rebalance check count: 517 INFO [2022-01-10 05:46:08,072] jepsen worker nemesis - couchbase.util Rebalance stuck count: 430 INFO [2022-01-10 05:46:08,086] jepsen worker 10 - jepsen.util 3700 :ok :read [3 31] INFO [2022-01-10 05:46:08,087] jepsen worker 10 - jepsen.util 3700 :invoke :read [3 nil] INFO [2022-01-10 05:46:08,089] jepsen worker 64 - jepsen.util 3664 :info :write [21 46] :RequestTimeoutException INFO [2022-01-10 05:46:08,090] jepsen worker 64 - jepsen.util 3754 :invoke :read [21 nil] INFO [2022-01-10 05:46:08,551] jepsen worker 42 - jepsen.util 3462 :info :write [14 8] :RequestTimeoutException INFO [2022-01-10 05:46:08,551] jepsen worker 42 - jepsen.util 3552 :invoke :read [14 nil] INFO [2022-01-10 05:46:08,578] jepsen worker 16 - jepsen.util 3796 :ok :read [5 25] INFO [2022-01-10 05:46:08,579] jepsen worker 16 - jepsen.util 3796 :invoke :read [5 nil] INFO [2022-01-10 05:46:08,585] jepsen worker 85 - jepsen.util 4045 :ok :read [28 45] INFO [2022-01-10 05:46:08,586] jepsen worker 85 - jepsen.util 4045 :invoke :write [28 27] INFO [2022-01-10 05:46:08,605] jepsen worker 50 - jepsen.util 3920 :ok :read [16 38] INFO [2022-01-10 05:46:08,606] jepsen worker 50 - jepsen.util 3920 :invoke :write [16 3] INFO [2022-01-10 05:46:08,618] jepsen worker 16 - jepsen.util 3796 :ok :read [5 25] INFO [2022-01-10 05:46:08,618] jepsen worker 16 - jepsen.util 3796 :invoke :write [5 18] INFO [2022-01-10 05:46:08,912] jepsen worker 71 - jepsen.util 3221 :info :write [23 43] :RequestTimeoutException INFO [2022-01-10 05:46:08,913] jepsen worker 71 - jepsen.util 3311 :invoke :write [23 9] INFO [2022-01-10 05:46:08,940] jepsen worker 54 - jepsen.util 3744 :info :write [18 4] :RequestTimeoutException INFO [2022-01-10 05:46:08,940] jepsen worker 54 - jepsen.util 3834 :invoke :read [18 nil] INFO [2022-01-10 05:46:08,950] jepsen worker 5 - jepsen.util 3515 :info :write [1 5] :RequestTimeoutException INFO [2022-01-10 05:46:08,951] jepsen worker 5 - jepsen.util 3605 :invoke :read [1 nil] INFO [2022-01-10 05:46:09,077] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:09,078] jepsen worker nemesis - couchbase.util Rebalance check count: 518 INFO [2022-01-10 05:46:09,126] jepsen worker nemesis - couchbase.util Rebalance stuck count: 431 INFO [2022-01-10 05:46:09,564] jepsen worker 45 - jepsen.util 4005 :ok :read [15 23] INFO [2022-01-10 05:46:09,564] jepsen worker 45 - jepsen.util 4005 :invoke :read [15 nil] INFO [2022-01-10 05:46:09,611] jepsen worker 6 - jepsen.util 3696 :info :write [2 49] :RequestTimeoutException INFO [2022-01-10 05:46:09,612] jepsen worker 6 - jepsen.util 3786 :invoke :write [2 19] INFO [2022-01-10 05:46:09,777] jepsen worker 44 - jepsen.util 3734 :info :write [14 43] :RequestTimeoutException INFO [2022-01-10 05:46:09,777] jepsen worker 27 - jepsen.util 3807 :info :write [9 26] :RequestTimeoutException INFO [2022-01-10 05:46:09,777] jepsen worker 44 - jepsen.util 3824 :invoke :read [14 nil] INFO [2022-01-10 05:46:09,778] jepsen worker 27 - jepsen.util 3897 :invoke :write [9 3] INFO [2022-01-10 05:46:09,841] jepsen worker 89 - jepsen.util 4139 :info :write [29 14] :RequestTimeoutException INFO [2022-01-10 05:46:09,842] jepsen worker 89 - jepsen.util 4229 :invoke :read [29 nil] INFO [2022-01-10 05:46:09,845] jepsen worker 24 - jepsen.util 3354 :info :write [8 39] :RequestTimeoutException INFO [2022-01-10 05:46:09,845] jepsen worker 24 - jepsen.util 3444 :invoke :read [8 nil] INFO [2022-01-10 05:46:09,865] jepsen worker 89 - jepsen.util 4229 :ok :read [29 0] INFO [2022-01-10 05:46:09,866] jepsen worker 89 - jepsen.util 4229 :invoke :read [29 nil] INFO [2022-01-10 05:46:09,889] jepsen worker 89 - jepsen.util 4229 :ok :read [29 0] INFO [2022-01-10 05:46:09,889] jepsen worker 89 - jepsen.util 4229 :invoke :write [29 24] INFO [2022-01-10 05:46:10,010] jepsen worker 13 - jepsen.util 4243 :info :write [4 3] :RequestTimeoutException INFO [2022-01-10 05:46:10,010] jepsen worker 13 - jepsen.util 4333 :invoke :read [4 nil] INFO [2022-01-10 05:46:10,113] jepsen worker 13 - jepsen.util 4333 :ok :read [4 3] INFO [2022-01-10 05:46:10,114] jepsen worker 13 - jepsen.util 4333 :invoke :read [4 nil] INFO [2022-01-10 05:46:10,132] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:10,132] jepsen worker nemesis - couchbase.util Rebalance check count: 519 INFO [2022-01-10 05:46:10,132] jepsen worker nemesis - couchbase.util Rebalance stuck count: 432 INFO [2022-01-10 05:46:10,156] jepsen worker 45 - jepsen.util 4005 :ok :read [15 23] INFO [2022-01-10 05:46:10,157] jepsen worker 45 - jepsen.util 4005 :invoke :write [15 16] INFO [2022-01-10 05:46:10,159] jepsen worker 13 - jepsen.util 4333 :ok :read [4 3] INFO [2022-01-10 05:46:10,160] jepsen worker 13 - jepsen.util 4333 :invoke :write [4 6] INFO [2022-01-10 05:46:10,182] jepsen worker 64 - jepsen.util 3754 :ok :read [21 7] INFO [2022-01-10 05:46:10,182] jepsen worker 64 - jepsen.util 3754 :invoke :read [21 nil] INFO [2022-01-10 05:46:10,191] jepsen worker 21 - jepsen.util 3711 :info :write [7 23] :RequestTimeoutException INFO [2022-01-10 05:46:10,191] jepsen worker 21 - jepsen.util 3801 :invoke :read [7 nil] INFO [2022-01-10 05:46:10,281] jepsen worker 84 - jepsen.util 3594 :info :write [28 35] :RequestTimeoutException INFO [2022-01-10 05:46:10,282] jepsen worker 84 - jepsen.util 3684 :invoke :read [28 nil] INFO [2022-01-10 05:46:10,317] jepsen worker 84 - jepsen.util 3684 :ok :read [28 45] INFO [2022-01-10 05:46:10,318] jepsen worker 84 - jepsen.util 3684 :invoke :write [28 15] INFO [2022-01-10 05:46:10,330] jepsen worker 44 - jepsen.util 3824 :ok :read [14 14] INFO [2022-01-10 05:46:10,330] jepsen worker 44 - jepsen.util 3824 :invoke :write [14 18] INFO [2022-01-10 05:46:10,375] jepsen worker 33 - jepsen.util 3903 :info :write [11 17] :RequestTimeoutException INFO [2022-01-10 05:46:10,376] jepsen worker 33 - jepsen.util 3993 :invoke :read [11 nil] WARN [2022-01-10 05:46:10,382] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":27104339,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":36148687,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9050120,"encode_duration_us":16732,"operation_id":"0x6eac","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10005280},{"operation_name":"upsert","last_dispatch_duration_us":9499024,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10087201,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9443799,"encode_duration_us":16999,"operation_id":"0x727b","timeout_ms":10000,"last_server_duration_us":9372162,"total_duration_us":10010076},{"operation_name":"upsert","last_dispatch_duration_us":18175815,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18175815,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9105402,"encode_duration_us":16979,"operation_id":"0x7223","timeout_ms":10000,"last_server_duration_us":9105402,"total_duration_us":10007865}],"total_count":3}} INFO [2022-01-10 05:46:10,411] jepsen worker 7 - jepsen.util 3787 :info :write [2 7] :RequestTimeoutException INFO [2022-01-10 05:46:10,412] jepsen worker 7 - jepsen.util 3877 :invoke :write [2 37] WARN [2022-01-10 05:46:10,482] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":26094962,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":26094962,"total_server_duration_us":32,"operation_id":"0x7095","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":26096223},{"operation_name":"upsert","last_dispatch_duration_us":9499024,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10087201,"total_server_duration_us":9443799,"encode_duration_us":16999,"operation_id":"0x727b","timeout_ms":10000,"last_server_duration_us":9372162,"total_duration_us":10010076},{"operation_name":"upsert","last_dispatch_duration_us":400994,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1896230,"total_server_duration_us":346483,"encode_duration_us":16663,"operation_id":"0x739f","timeout_ms":10000,"last_server_duration_us":21977,"total_duration_us":10007005},{"operation_name":"upsert","last_dispatch_duration_us":994452,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1730317,"total_server_duration_us":149648,"encode_duration_us":17793,"operation_id":"0x740e","timeout_ms":10000,"last_server_duration_us":66957,"total_duration_us":10004705},{"operation_name":"upsert","last_dispatch_duration_us":11072370,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11072370,"total_server_duration_us":77851,"encode_duration_us":17084,"operation_id":"0x7385","timeout_ms":10000,"last_server_duration_us":77851,"total_duration_us":10004295},{"operation_name":"upsert","last_dispatch_duration_us":838630,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9986131,"total_server_duration_us":408660,"encode_duration_us":18103,"operation_id":"0x72d9","timeout_ms":10000,"last_server_duration_us":113463,"total_duration_us":10004077},{"operation_name":"upsert","last_dispatch_duration_us":165276,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":722303,"total_server_duration_us":223348,"encode_duration_us":17141,"operation_id":"0x741c","timeout_ms":10000,"last_server_duration_us":70421,"total_duration_us":10003658},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":24828,"operation_id":"0x7349","timeout_ms":10000,"total_duration_us":10002319},{"operation_name":"upsert","last_dispatch_duration_us":805322,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10420555,"total_server_duration_us":726691,"encode_duration_us":18508,"operation_id":"0x72b1","timeout_ms":10000,"last_server_duration_us":420703,"total_duration_us":10001503},{"operation_name":"upsert","last_dispatch_duration_us":432215,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1762232,"total_server_duration_us":439669,"encode_duration_us":16645,"operation_id":"0x7369","timeout_ms":10000,"last_server_duration_us":282264,"total_duration_us":10000257}],"total_count":16}} INFO [2022-01-10 05:46:10,787] jepsen worker 3 - jepsen.util 3513 :info :write [1 30] :RequestTimeoutException INFO [2022-01-10 05:46:10,787] jepsen worker 3 - jepsen.util 3603 :invoke :read [1 nil] INFO [2022-01-10 05:46:10,906] jepsen worker 65 - jepsen.util 3485 :info :write [21 11] :RequestTimeoutException INFO [2022-01-10 05:46:10,907] jepsen worker 65 - jepsen.util 3575 :invoke :read [21 nil] INFO [2022-01-10 05:46:10,932] jepsen worker 64 - jepsen.util 3754 :ok :read [21 7] INFO [2022-01-10 05:46:10,933] jepsen worker 64 - jepsen.util 3754 :invoke :write [21 43] INFO [2022-01-10 05:46:11,137] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:11,138] jepsen worker nemesis - couchbase.util Rebalance check count: 520 INFO [2022-01-10 05:46:11,138] jepsen worker nemesis - couchbase.util Rebalance stuck count: 433 INFO [2022-01-10 05:46:11,220] jepsen worker 14 - jepsen.util 4514 :info :write [4 29] :RequestTimeoutException INFO [2022-01-10 05:46:11,221] jepsen worker 14 - jepsen.util 4604 :invoke :read [4 nil] INFO [2022-01-10 05:46:11,380] jepsen worker 48 - jepsen.util 3378 :info :write [16 37] :RequestTimeoutException INFO [2022-01-10 05:46:11,381] jepsen worker 48 - jepsen.util 3468 :invoke :write [16 8] INFO [2022-01-10 05:46:11,383] jepsen worker 14 - jepsen.util 4604 :ok :read [4 3] INFO [2022-01-10 05:46:11,383] jepsen worker 14 - jepsen.util 4604 :invoke :read [4 nil] INFO [2022-01-10 05:46:11,472] jepsen worker 14 - jepsen.util 4604 :ok :read [4 3] INFO [2022-01-10 05:46:11,472] jepsen worker 14 - jepsen.util 4604 :invoke :write [4 32] INFO [2022-01-10 05:46:11,510] jepsen worker 19 - jepsen.util 3889 :info :write [6 32] :RequestTimeoutException INFO [2022-01-10 05:46:11,510] jepsen worker 19 - jepsen.util 3979 :invoke :read [6 nil] INFO [2022-01-10 05:46:11,721] jepsen worker 76 - jepsen.util 3226 :info :write [25 22] :RequestTimeoutException INFO [2022-01-10 05:46:11,721] jepsen worker 76 - jepsen.util 3316 :invoke :write [25 43] INFO [2022-01-10 05:46:11,851] jepsen worker 74 - jepsen.util 4214 :info :write [24 10] :RequestTimeoutException INFO [2022-01-10 05:46:11,855] jepsen worker 74 - jepsen.util 4304 :invoke :read [24 nil] INFO [2022-01-10 05:46:11,900] jepsen worker 2 - jepsen.util 4052 :ok :read [0 46] INFO [2022-01-10 05:46:11,900] jepsen worker 2 - jepsen.util 4052 :invoke :read [0 nil] INFO [2022-01-10 05:46:11,920] jepsen worker 58 - jepsen.util 3568 :ok :read [19 31] INFO [2022-01-10 05:46:11,920] jepsen worker 58 - jepsen.util 3568 :invoke :read [19 nil] INFO [2022-01-10 05:46:11,981] jepsen worker 8 - jepsen.util 4148 :info :write [2 35] :RequestTimeoutException WARN [2022-01-10 05:46:11,981] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9654357,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18051509,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":11141776,"encode_duration_us":17360,"operation_id":"0x7175","timeout_ms":10000,"last_server_duration_us":9440174,"total_duration_us":10001068},{"operation_name":"upsert","last_dispatch_duration_us":9042127,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17400677,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9278857,"encode_duration_us":19602,"operation_id":"0x715d","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10008979},{"operation_name":"upsert","last_dispatch_duration_us":10303073,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11170400,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9402866,"encode_duration_us":17144,"operation_id":"0x72da","timeout_ms":10000,"last_server_duration_us":9288246,"total_duration_us":10006081}],"total_count":3}} INFO [2022-01-10 05:46:11,982] jepsen worker 8 - jepsen.util 4238 :invoke :read [2 nil] WARN [2022-01-10 05:46:12,066] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":733987,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":6415637,"total_server_duration_us":1071068,"encode_duration_us":16759,"operation_id":"0x73ea","timeout_ms":10000,"last_server_duration_us":30069,"total_duration_us":10010949},{"operation_name":"upsert","last_dispatch_duration_us":872125,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1444091,"total_server_duration_us":399063,"encode_duration_us":19026,"operation_id":"0x737d","timeout_ms":10000,"last_server_duration_us":334720,"total_duration_us":10010782},{"operation_name":"upsert","last_dispatch_duration_us":33970,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9030532,"total_server_duration_us":220479,"encode_duration_us":17266,"operation_id":"0x740d","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10007826},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":17118,"operation_id":"0x741b","timeout_ms":10000,"total_duration_us":10007527},{"operation_name":"upsert","last_dispatch_duration_us":629911,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2002061,"total_server_duration_us":448761,"encode_duration_us":17851,"operation_id":"0x7454","timeout_ms":10000,"last_server_duration_us":135758,"total_duration_us":10007250},{"operation_name":"upsert","last_dispatch_duration_us":909885,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3270469,"total_server_duration_us":246874,"encode_duration_us":16560,"operation_id":"0x73b6","timeout_ms":10000,"last_server_duration_us":52064,"total_duration_us":10006614},{"operation_name":"upsert","last_dispatch_duration_us":8432531,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10858209,"total_server_duration_us":347496,"encode_duration_us":16927,"operation_id":"0x7285","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10005974},{"operation_name":"upsert","last_dispatch_duration_us":846987,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":846987,"total_server_duration_us":446368,"encode_duration_us":16311,"operation_id":"0x7344","timeout_ms":10000,"last_server_duration_us":446368,"total_duration_us":10003210},{"operation_name":"upsert","last_dispatch_duration_us":807399,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1821452,"total_server_duration_us":181114,"encode_duration_us":17041,"operation_id":"0x72e7","timeout_ms":10000,"last_server_duration_us":50652,"total_duration_us":10003133},{"operation_name":"upsert","last_dispatch_duration_us":1000765,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4958704,"total_server_duration_us":553464,"encode_duration_us":16735,"operation_id":"0x741d","timeout_ms":10000,"last_server_duration_us":27132,"total_duration_us":10003126}],"total_count":15}} WARN [2022-01-10 05:46:12,067] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10378381,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13377490,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9330851,"encode_duration_us":16390,"operation_id":"0x718e","timeout_ms":10000,"last_server_duration_us":9119248,"total_duration_us":10008500},{"operation_name":"upsert","last_dispatch_duration_us":9091921,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10054960,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9556607,"encode_duration_us":21140,"operation_id":"0x721c","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10010185}],"total_count":2}} INFO [2022-01-10 05:46:12,071] jepsen worker 75 - jepsen.util 3315 :info :write [25 38] :RequestTimeoutException INFO [2022-01-10 05:46:12,072] jepsen worker 75 - jepsen.util 3405 :invoke :write [25 14] INFO [2022-01-10 05:46:12,075] jepsen worker 17 - jepsen.util 3617 :ok :read [5 25] INFO [2022-01-10 05:46:12,075] jepsen worker 17 - jepsen.util 3617 :invoke :write [5 35] WARN [2022-01-10 05:46:12,092] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17177268,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":17177268,"total_server_duration_us":56,"operation_id":"0x71ab","timeout_ms":30000,"last_server_duration_us":56,"total_duration_us":17178342},{"operation_name":"get","last_dispatch_duration_us":16274791,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":16274791,"total_server_duration_us":43,"operation_id":"0x71cb","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":16275982},{"operation_name":"get","last_dispatch_duration_us":15195390,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":15195390,"total_server_duration_us":13488,"operation_id":"0x7354","timeout_ms":30000,"last_server_duration_us":13488,"total_duration_us":15197550},{"operation_name":"get","last_dispatch_duration_us":15182499,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":15182499,"total_server_duration_us":56,"operation_id":"0x720f","timeout_ms":30000,"last_server_duration_us":56,"total_duration_us":15183753},{"operation_name":"get","last_dispatch_duration_us":13238761,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":13238761,"total_server_duration_us":18723,"operation_id":"0x73b7","timeout_ms":30000,"last_server_duration_us":18723,"total_duration_us":13240930},{"operation_name":"upsert","last_dispatch_duration_us":4353698,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":4353698,"total_server_duration_us":21895,"encode_duration_us":16296,"operation_id":"0x7434","timeout_ms":10000,"last_server_duration_us":21895,"total_duration_us":10007346},{"operation_name":"upsert","last_dispatch_duration_us":616836,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6177076,"total_server_duration_us":1120058,"encode_duration_us":16401,"operation_id":"0x73fd","timeout_ms":10000,"last_server_duration_us":64993,"total_duration_us":10006515},{"operation_name":"upsert","last_dispatch_duration_us":12178527,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":12178527,"total_server_duration_us":41767,"encode_duration_us":19416,"operation_id":"0x73fe","timeout_ms":10000,"last_server_duration_us":41767,"total_duration_us":10004130},{"operation_name":"upsert","last_dispatch_duration_us":544846,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":5896836,"total_server_duration_us":1370086,"encode_duration_us":18784,"operation_id":"0x7441","timeout_ms":10000,"last_server_duration_us":165885,"total_duration_us":10003874},{"operation_name":"upsert","last_dispatch_duration_us":701012,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4148454,"total_server_duration_us":509875,"encode_duration_us":16698,"operation_id":"0x737c","timeout_ms":10000,"last_server_duration_us":47306,"total_duration_us":10002561}],"total_count":23}} INFO [2022-01-10 05:46:12,095] jepsen worker 82 - jepsen.util 3862 :ok :read [27 30] INFO [2022-01-10 05:46:12,095] jepsen worker 82 - jepsen.util 3862 :invoke :write [27 31] INFO [2022-01-10 05:46:12,143] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:12,143] jepsen worker nemesis - couchbase.util Rebalance check count: 521 INFO [2022-01-10 05:46:12,143] jepsen worker nemesis - couchbase.util Rebalance stuck count: 434 WARN [2022-01-10 05:46:12,177] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":16386,"operation_id":"0x73dc","timeout_ms":10000,"total_duration_us":10010120},{"operation_name":"upsert","last_dispatch_duration_us":10386987,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13867933,"total_server_duration_us":9642508,"encode_duration_us":16754,"operation_id":"0x7318","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10010065},{"operation_name":"upsert","last_dispatch_duration_us":82215,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":3942810,"total_server_duration_us":3117122,"encode_duration_us":16926,"operation_id":"0x7384","timeout_ms":10000,"last_server_duration_us":67485,"total_duration_us":10008420},{"operation_name":"upsert","last_dispatch_duration_us":146793,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9798778,"total_server_duration_us":569698,"encode_duration_us":17441,"operation_id":"0x7343","timeout_ms":10000,"last_server_duration_us":18570,"total_duration_us":10007833},{"operation_name":"upsert","last_dispatch_duration_us":8244722,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13428238,"total_server_duration_us":769064,"encode_duration_us":17613,"operation_id":"0x7331","timeout_ms":10000,"last_server_duration_us":78414,"total_duration_us":10006158},{"operation_name":"upsert","last_dispatch_duration_us":140691,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":3661471,"total_server_duration_us":2655904,"encode_duration_us":19227,"operation_id":"0x72e3","timeout_ms":10000,"last_server_duration_us":85011,"total_duration_us":10005385},{"operation_name":"upsert","last_dispatch_duration_us":8486037,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10527355,"total_server_duration_us":985267,"encode_duration_us":16795,"operation_id":"0x72e9","timeout_ms":10000,"last_server_duration_us":38586,"total_duration_us":10005176},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","encode_duration_us":16890,"operation_id":"0x7350","timeout_ms":10000,"total_duration_us":10003075},{"operation_name":"upsert","last_dispatch_duration_us":10426232,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10426232,"total_server_duration_us":9107531,"encode_duration_us":16607,"operation_id":"0x738a","timeout_ms":10000,"last_server_duration_us":9107531,"total_duration_us":10002601},{"operation_name":"upsert","last_dispatch_duration_us":426189,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9841134,"total_server_duration_us":416146,"encode_duration_us":17489,"operation_id":"0x746e","timeout_ms":10000,"last_server_duration_us":98879,"total_duration_us":10002054}],"total_count":16}} WARN [2022-01-10 05:46:12,182] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":169648,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":5414883,"total_server_duration_us":1128801,"encode_duration_us":18642,"operation_id":"0x7379","timeout_ms":10000,"last_server_duration_us":34612,"total_duration_us":10009625},{"operation_name":"upsert","last_dispatch_duration_us":243642,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":7380945,"total_server_duration_us":779455,"encode_duration_us":17040,"operation_id":"0x7298","timeout_ms":10000,"last_server_duration_us":25015,"total_duration_us":10007604},{"operation_name":"upsert","last_dispatch_duration_us":419146,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8211690,"total_server_duration_us":414569,"encode_duration_us":16683,"operation_id":"0x744c","timeout_ms":10000,"last_server_duration_us":21731,"total_duration_us":10006603},{"operation_name":"upsert","last_dispatch_duration_us":10303073,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11170400,"total_server_duration_us":9402866,"encode_duration_us":17144,"operation_id":"0x72da","timeout_ms":10000,"last_server_duration_us":9288246,"total_duration_us":10006081},{"operation_name":"upsert","last_dispatch_duration_us":126348,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":7544761,"total_server_duration_us":54626,"encode_duration_us":16888,"operation_id":"0x73ac","timeout_ms":10000,"last_server_duration_us":23898,"total_duration_us":10006009},{"operation_name":"upsert","last_dispatch_duration_us":8138631,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8638818,"total_server_duration_us":279896,"encode_duration_us":16539,"operation_id":"0x730f","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10005901},{"operation_name":"upsert","last_dispatch_duration_us":407217,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9538825,"total_server_duration_us":317136,"encode_duration_us":18153,"operation_id":"0x737b","timeout_ms":10000,"last_server_duration_us":23222,"total_duration_us":10003982},{"operation_name":"upsert","last_dispatch_duration_us":283400,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":7878245,"total_server_duration_us":1020238,"encode_duration_us":19556,"operation_id":"0x7281","timeout_ms":10000,"last_server_duration_us":28218,"total_duration_us":10003269},{"operation_name":"upsert","last_dispatch_duration_us":7572668,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9630034,"total_server_duration_us":279419,"encode_duration_us":16323,"operation_id":"0x731a","timeout_ms":10000,"last_server_duration_us":62925,"total_duration_us":10002256},{"operation_name":"get","last_dispatch_duration_us":8353497,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8353497,"total_server_duration_us":62,"operation_id":"0x7377","timeout_ms":30000,"last_server_duration_us":62,"total_duration_us":8354726}],"total_count":16}} INFO [2022-01-10 05:46:12,186] jepsen worker 81 - jepsen.util 4041 :info :write [27 30] :RequestTimeoutException INFO [2022-01-10 05:46:12,187] jepsen worker 81 - jepsen.util 4131 :invoke :read [27 nil] INFO [2022-01-10 05:46:12,195] jepsen worker 12 - jepsen.util 4152 :info :write [4 18] :RequestTimeoutException INFO [2022-01-10 05:46:12,196] jepsen worker 12 - jepsen.util 4242 :invoke :write [4 0] INFO [2022-01-10 05:46:12,201] jepsen worker 0 - jepsen.util 3600 :ok :read [0 46] INFO [2022-01-10 05:46:12,201] jepsen worker 0 - jepsen.util 3600 :invoke :write [0 9] WARN [2022-01-10 05:46:12,268] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9369204,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16602189,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9900226,"encode_duration_us":19508,"operation_id":"0x71ca","timeout_ms":10000,"last_server_duration_us":9064980,"total_duration_us":10002269}],"total_count":1}} WARN [2022-01-10 05:46:12,278] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10426232,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10426232,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9107531,"encode_duration_us":16607,"operation_id":"0x738a","timeout_ms":10000,"last_server_duration_us":9107531,"total_duration_us":10002601},{"operation_name":"upsert","last_dispatch_duration_us":9511899,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14833680,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9925117,"encode_duration_us":20414,"operation_id":"0x7207","timeout_ms":10000,"last_server_duration_us":9278584,"total_duration_us":10006979},{"operation_name":"upsert","last_dispatch_duration_us":9120949,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17177141,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9970921,"encode_duration_us":17126,"operation_id":"0x7183","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10007084},{"operation_name":"upsert","last_dispatch_duration_us":17707794,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17707794,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9022513,"encode_duration_us":20738,"operation_id":"0x71b2","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10007773},{"operation_name":"upsert","last_dispatch_duration_us":10386987,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13867933,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9642508,"encode_duration_us":16754,"operation_id":"0x7318","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10010065}],"total_count":5}} WARN [2022-01-10 05:46:12,292] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18263718,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18263718,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9033121,"encode_duration_us":16384,"operation_id":"0x715f","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10002394},{"operation_name":"upsert","last_dispatch_duration_us":9678982,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17301394,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9876053,"encode_duration_us":16061,"operation_id":"0x7128","timeout_ms":10000,"last_server_duration_us":9043735,"total_duration_us":10007442},{"operation_name":"upsert","last_dispatch_duration_us":17661627,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":23133524,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9377736,"encode_duration_us":17539,"operation_id":"0x71c6","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10009943}],"total_count":3}} WARN [2022-01-10 05:46:12,368] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":36025,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":1101600,"total_server_duration_us":645090,"encode_duration_us":17581,"operation_id":"0x73de","timeout_ms":10000,"last_server_duration_us":22141,"total_duration_us":10010825},{"operation_name":"upsert","last_dispatch_duration_us":678284,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10337432,"total_server_duration_us":108722,"encode_duration_us":21984,"operation_id":"0x7342","timeout_ms":10000,"last_server_duration_us":43618,"total_duration_us":10009731},{"operation_name":"upsert","last_dispatch_duration_us":511819,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10155319,"total_server_duration_us":85809,"encode_duration_us":16515,"operation_id":"0x7378","timeout_ms":10000,"last_server_duration_us":28676,"total_duration_us":10009532},{"operation_name":"upsert","last_dispatch_duration_us":982384,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8296988,"total_server_duration_us":1812459,"encode_duration_us":16185,"operation_id":"0x731b","timeout_ms":10000,"last_server_duration_us":120818,"total_duration_us":10008914},{"operation_name":"upsert","last_dispatch_duration_us":589611,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2838964,"total_server_duration_us":384620,"encode_duration_us":17741,"operation_id":"0x744d","timeout_ms":10000,"last_server_duration_us":157260,"total_duration_us":10008792},{"operation_name":"upsert","last_dispatch_duration_us":270655,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":4362007,"total_server_duration_us":1454650,"encode_duration_us":16341,"operation_id":"0x744b","timeout_ms":10000,"last_server_duration_us":75062,"total_duration_us":10007649},{"operation_name":"upsert","last_dispatch_duration_us":461137,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5168834,"total_server_duration_us":1116181,"encode_duration_us":16959,"operation_id":"0x72e2","timeout_ms":10000,"last_server_duration_us":133452,"total_duration_us":10006540},{"operation_name":"upsert","last_dispatch_duration_us":303383,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":4544754,"total_server_duration_us":2103058,"encode_duration_us":16546,"operation_id":"0x7411","timeout_ms":10000,"last_server_duration_us":284699,"total_duration_us":10006196},{"operation_name":"upsert","last_dispatch_duration_us":342686,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10250671,"total_server_duration_us":183687,"encode_duration_us":16660,"operation_id":"0x7388","timeout_ms":10000,"last_server_duration_us":23898,"total_duration_us":10006034},{"operation_name":"upsert","last_dispatch_duration_us":61705,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":1198309,"total_server_duration_us":576802,"encode_duration_us":16739,"operation_id":"0x73df","timeout_ms":10000,"last_server_duration_us":20521,"total_duration_us":10005149}],"total_count":18}} INFO [2022-01-10 05:46:12,406] jepsen worker 2 - jepsen.util 4052 :ok :read [0 46] INFO [2022-01-10 05:46:12,407] jepsen worker 2 - jepsen.util 4052 :invoke :read [0 nil] INFO [2022-01-10 05:46:12,440] jepsen worker 15 - jepsen.util 3255 :ok :read [5 25] INFO [2022-01-10 05:46:12,441] jepsen worker 15 - jepsen.util 3255 :invoke :write [5 9] INFO [2022-01-10 05:46:12,450] jepsen worker 58 - jepsen.util 3568 :ok :read [19 31] INFO [2022-01-10 05:46:12,451] jepsen worker 58 - jepsen.util 3568 :invoke :read [19 nil] INFO [2022-01-10 05:46:12,487] jepsen worker 68 - jepsen.util 3848 :ok :read [22 41] INFO [2022-01-10 05:46:12,488] jepsen worker 68 - jepsen.util 3848 :invoke :write [22 20] INFO [2022-01-10 05:46:12,506] jepsen worker 34 - jepsen.util 3544 :info :write [11 5] :RequestTimeoutException INFO [2022-01-10 05:46:12,507] jepsen worker 34 - jepsen.util 3634 :invoke :write [11 5] INFO [2022-01-10 05:46:12,613] jepsen worker 81 - jepsen.util 4131 :ok :read [27 30] INFO [2022-01-10 05:46:12,614] jepsen worker 81 - jepsen.util 4131 :invoke :read [27 nil] INFO [2022-01-10 05:46:12,856] jepsen worker 66 - jepsen.util 3396 :info :write [22 36] :RequestTimeoutException INFO [2022-01-10 05:46:12,857] jepsen worker 66 - jepsen.util 3486 :invoke :write [22 45] INFO [2022-01-10 05:46:12,916] jepsen worker 39 - jepsen.util 4269 :info :write [13 13] :RequestTimeoutException INFO [2022-01-10 05:46:12,917] jepsen worker 39 - jepsen.util 4359 :invoke :read [13 nil] INFO [2022-01-10 05:46:12,980] jepsen worker 2 - jepsen.util 4052 :ok :read [0 46] INFO [2022-01-10 05:46:12,980] jepsen worker 2 - jepsen.util 4052 :invoke :read [0 nil] INFO [2022-01-10 05:46:12,981] jepsen worker 28 - jepsen.util 3448 :info :write [9 15] :RequestTimeoutException INFO [2022-01-10 05:46:12,981] jepsen worker 78 - jepsen.util 4488 :info :write [26 6] :RequestTimeoutException INFO [2022-01-10 05:46:12,982] jepsen worker 28 - jepsen.util 3538 :invoke :write [9 21] INFO [2022-01-10 05:46:12,983] jepsen worker 78 - jepsen.util 4578 :invoke :write [26 22] INFO [2022-01-10 05:46:12,988] jepsen worker 58 - jepsen.util 3568 :ok :read [19 31] INFO [2022-01-10 05:46:12,988] jepsen worker 58 - jepsen.util 3568 :invoke :read [19 nil] INFO [2022-01-10 05:46:13,149] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:13,150] jepsen worker nemesis - couchbase.util Rebalance check count: 522 INFO [2022-01-10 05:46:13,150] jepsen worker nemesis - couchbase.util Rebalance stuck count: 435 INFO [2022-01-10 05:46:13,164] jepsen worker 81 - jepsen.util 4131 :ok :read [27 30] INFO [2022-01-10 05:46:13,164] jepsen worker 81 - jepsen.util 4131 :invoke :read [27 nil] INFO [2022-01-10 05:46:13,532] jepsen worker 2 - jepsen.util 4052 :ok :read [0 46] INFO [2022-01-10 05:46:13,532] jepsen worker 2 - jepsen.util 4052 :invoke :read [0 nil] INFO [2022-01-10 05:46:13,614] jepsen worker 58 - jepsen.util 3568 :ok :read [19 31] INFO [2022-01-10 05:46:13,615] jepsen worker 58 - jepsen.util 3568 :invoke :read [19 nil] INFO [2022-01-10 05:46:13,645] jepsen worker 73 - jepsen.util 3313 :info :write [24 21] :RequestTimeoutException INFO [2022-01-10 05:46:13,646] jepsen worker 73 - jepsen.util 3403 :invoke :read [24 nil] INFO [2022-01-10 05:46:13,816] jepsen worker 2 - jepsen.util 4052 :ok :read [0 46] INFO [2022-01-10 05:46:13,817] jepsen worker 2 - jepsen.util 4052 :invoke :write [0 15] INFO [2022-01-10 05:46:13,928] jepsen worker 58 - jepsen.util 3568 :ok :read [19 31] INFO [2022-01-10 05:46:13,928] jepsen worker 58 - jepsen.util 3568 :invoke :read [19 nil] INFO [2022-01-10 05:46:13,945] jepsen worker 81 - jepsen.util 4131 :ok :read [27 30] INFO [2022-01-10 05:46:13,946] jepsen worker 81 - jepsen.util 4131 :invoke :read [27 nil] INFO [2022-01-10 05:46:14,155] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:14,155] jepsen worker nemesis - couchbase.util Rebalance check count: 523 INFO [2022-01-10 05:46:14,226] jepsen worker 46 - jepsen.util 3826 :info :write [15 49] :RequestTimeoutException INFO [2022-01-10 05:46:14,233] jepsen worker 9 - jepsen.util 4149 :info :write [3 23] :SyncWriteAmbiguous INFO [2022-01-10 05:46:14,245] jepsen worker 72 - jepsen.util 4122 :info :write [24 14] :RequestTimeoutException INFO [2022-01-10 05:46:14,253] jepsen worker nemesis - couchbase.util Rebalance stuck count: 436 INFO [2022-01-10 05:46:14,253] jepsen worker 46 - jepsen.util 3916 :invoke :write [15 4] INFO [2022-01-10 05:46:14,253] jepsen worker 9 - jepsen.util 4239 :invoke :read [3 nil] INFO [2022-01-10 05:46:14,254] jepsen worker 72 - jepsen.util 4212 :invoke :write [24 17] INFO [2022-01-10 05:46:14,350] jepsen worker 81 - jepsen.util 4131 :ok :read [27 30] INFO [2022-01-10 05:46:14,351] jepsen worker 81 - jepsen.util 4131 :invoke :write [27 30] INFO [2022-01-10 05:46:14,420] jepsen worker 53 - jepsen.util 4553 :info :write [17 35] :RequestTimeoutException INFO [2022-01-10 05:46:14,421] jepsen worker 53 - jepsen.util 4643 :invoke :read [17 nil] INFO [2022-01-10 05:46:14,469] jepsen worker 53 - jepsen.util 4643 :ok :read [17 41] INFO [2022-01-10 05:46:14,470] jepsen worker 53 - jepsen.util 4643 :invoke :write [17 38] INFO [2022-01-10 05:46:14,755] jepsen worker 51 - jepsen.util 4461 :info :write [17 24] :RequestTimeoutException INFO [2022-01-10 05:46:14,755] jepsen worker 51 - jepsen.util 4551 :invoke :read [17 nil] INFO [2022-01-10 05:46:14,858] jepsen worker 51 - jepsen.util 4551 :ok :read [17 41] INFO [2022-01-10 05:46:14,858] jepsen worker 51 - jepsen.util 4551 :invoke :read [17 nil] INFO [2022-01-10 05:46:14,907] jepsen worker 51 - jepsen.util 4551 :ok :read [17 41] INFO [2022-01-10 05:46:14,907] jepsen worker 51 - jepsen.util 4551 :invoke :write [17 7] INFO [2022-01-10 05:46:14,946] jepsen worker 69 - jepsen.util 3219 :info :write [23 14] :RequestTimeoutException INFO [2022-01-10 05:46:14,947] jepsen worker 69 - jepsen.util 3309 :invoke :write [23 42] INFO [2022-01-10 05:46:14,986] jepsen worker 25 - jepsen.util 3625 :ok :read [8 39] INFO [2022-01-10 05:46:14,987] jepsen worker 25 - jepsen.util 3625 :invoke :read [8 nil] INFO [2022-01-10 05:46:15,016] jepsen worker 32 - jepsen.util 3902 :info :write [10 19] :RequestTimeoutException INFO [2022-01-10 05:46:15,017] jepsen worker 32 - jepsen.util 3992 :invoke :write [10 20] INFO [2022-01-10 05:46:15,026] jepsen worker 38 - jepsen.util 3638 :info :write [12 13] :RequestTimeoutException INFO [2022-01-10 05:46:15,026] jepsen worker 38 - jepsen.util 3728 :invoke :read [12 nil] INFO [2022-01-10 05:46:15,116] jepsen worker 70 - jepsen.util 3670 :info :write [23 3] :RequestTimeoutException INFO [2022-01-10 05:46:15,117] jepsen worker 70 - jepsen.util 3760 :invoke :read [23 nil] INFO [2022-01-10 05:46:15,166] jepsen worker 37 - jepsen.util 3457 :info :write [12 14] :RequestTimeoutException INFO [2022-01-10 05:46:15,167] jepsen worker 37 - jepsen.util 3547 :invoke :write [12 11] INFO [2022-01-10 05:46:15,206] jepsen worker 11 - jepsen.util 3611 :info :write [3 16] :RequestTimeoutException INFO [2022-01-10 05:46:15,206] jepsen worker 11 - jepsen.util 3701 :invoke :read [3 nil] INFO [2022-01-10 05:46:15,210] jepsen worker 61 - jepsen.util 3841 :info :write [20 31] :RequestTimeoutException INFO [2022-01-10 05:46:15,211] jepsen worker 61 - jepsen.util 3931 :invoke :write [20 0] INFO [2022-01-10 05:46:15,259] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:15,259] jepsen worker nemesis - couchbase.util Rebalance check count: 524 INFO [2022-01-10 05:46:15,259] jepsen worker nemesis - couchbase.util Rebalance stuck count: 437 INFO [2022-01-10 05:46:15,303] jepsen worker 21 - jepsen.util 3801 :ok :read [7 :nil] INFO [2022-01-10 05:46:15,304] jepsen worker 21 - jepsen.util 3801 :invoke :read [7 nil] INFO [2022-01-10 05:46:15,305] jepsen worker 4 - jepsen.util 4324 :info :write [1 42] :RequestTimeoutException INFO [2022-01-10 05:46:15,306] jepsen worker 4 - jepsen.util 4414 :invoke :read [1 nil] INFO [2022-01-10 05:46:15,323] jepsen worker 20 - jepsen.util 3530 :ok :read [6 25] INFO [2022-01-10 05:46:15,324] jepsen worker 20 - jepsen.util 3530 :invoke :write [6 14] INFO [2022-01-10 05:46:15,376] jepsen worker 29 - jepsen.util 3629 :info :write [9 20] :RequestTimeoutException INFO [2022-01-10 05:46:15,376] jepsen worker 29 - jepsen.util 3719 :invoke :write [9 35] INFO [2022-01-10 05:46:15,425] jepsen worker 21 - jepsen.util 3801 :ok :read [7 :nil] INFO [2022-01-10 05:46:15,426] jepsen worker 21 - jepsen.util 3801 :invoke :write [7 17] INFO [2022-01-10 05:46:15,518] jepsen worker 25 - jepsen.util 3625 :ok :read [8 39] INFO [2022-01-10 05:46:15,519] jepsen worker 25 - jepsen.util 3625 :invoke :write [8 41] INFO [2022-01-10 05:46:15,566] jepsen worker 24 - jepsen.util 3444 :ok :read [8 39] INFO [2022-01-10 05:46:15,567] jepsen worker 24 - jepsen.util 3444 :invoke :write [8 44] INFO [2022-01-10 05:46:15,574] jepsen worker 74 - jepsen.util 4304 :ok :read [24 18] INFO [2022-01-10 05:46:15,574] jepsen worker 74 - jepsen.util 4304 :invoke :read [24 nil] INFO [2022-01-10 05:46:15,813] jepsen worker 77 - jepsen.util 4127 :info :write [25 38] :RequestTimeoutException INFO [2022-01-10 05:46:15,814] jepsen worker 77 - jepsen.util 4217 :invoke :read [25 nil] INFO [2022-01-10 05:46:15,832] jepsen worker 49 - jepsen.util 3649 :info :write [16 7] :RequestTimeoutException INFO [2022-01-10 05:46:15,833] jepsen worker 49 - jepsen.util 3739 :invoke :write [16 17] INFO [2022-01-10 05:46:15,837] jepsen worker 74 - jepsen.util 4304 :ok :read [24 18] INFO [2022-01-10 05:46:15,837] jepsen worker 74 - jepsen.util 4304 :invoke :read [24 nil] INFO [2022-01-10 05:46:15,852] jepsen worker 40 - jepsen.util 3190 :info :write [13 44] :RequestTimeoutException INFO [2022-01-10 05:46:15,852] jepsen worker 40 - jepsen.util 3280 :invoke :write [13 25] INFO [2022-01-10 05:46:15,886] jepsen worker 67 - jepsen.util 3577 :info :write [22 5] :RequestTimeoutException INFO [2022-01-10 05:46:15,886] jepsen worker 67 - jepsen.util 3667 :invoke :read [22 nil] INFO [2022-01-10 05:46:15,975] jepsen worker 36 - jepsen.util 2826 :info :write [12 48] :RequestTimeoutException INFO [2022-01-10 05:46:15,975] jepsen worker 36 - jepsen.util 2916 :invoke :write [12 26] INFO [2022-01-10 05:46:16,060] jepsen worker 59 - jepsen.util 3749 :info :write [19 43] :RequestTimeoutException INFO [2022-01-10 05:46:16,061] jepsen worker 59 - jepsen.util 3839 :invoke :write [19 47] INFO [2022-01-10 05:46:16,086] jepsen worker 60 - jepsen.util 4020 :ok :read [20 :nil] INFO [2022-01-10 05:46:16,087] jepsen worker 60 - jepsen.util 4020 :invoke :write [20 44] INFO [2022-01-10 05:46:16,091] jepsen worker 62 - jepsen.util 4112 :info :write [20 11] :RequestTimeoutException INFO [2022-01-10 05:46:16,092] jepsen worker 62 - jepsen.util 4202 :invoke :read [20 nil] INFO [2022-01-10 05:46:16,216] jepsen worker 43 - jepsen.util 3733 :ok :read [14 14] INFO [2022-01-10 05:46:16,217] jepsen worker 43 - jepsen.util 3733 :invoke :read [14 nil] INFO [2022-01-10 05:46:16,250] jepsen worker 41 - jepsen.util 3551 :info :write [13 3] :RequestTimeoutException INFO [2022-01-10 05:46:16,250] jepsen worker 41 - jepsen.util 3641 :invoke :write [13 12] INFO [2022-01-10 05:46:16,267] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:16,268] jepsen worker nemesis - couchbase.util Rebalance check count: 525 INFO [2022-01-10 05:46:16,268] jepsen worker nemesis - couchbase.util Rebalance stuck count: 438 INFO [2022-01-10 05:46:16,427] jepsen worker 63 - jepsen.util 3663 :info :write [21 22] :RequestTimeoutException INFO [2022-01-10 05:46:16,427] jepsen worker 63 - jepsen.util 3753 :invoke :read [21 nil] INFO [2022-01-10 05:46:16,445] jepsen worker 47 - jepsen.util 3737 :info :write [15 11] :RequestTimeoutException INFO [2022-01-10 05:46:16,446] jepsen worker 47 - jepsen.util 3827 :invoke :read [15 nil] INFO [2022-01-10 05:46:16,450] jepsen worker 74 - jepsen.util 4304 :ok :read [24 18] INFO [2022-01-10 05:46:16,450] jepsen worker 74 - jepsen.util 4304 :invoke :read [24 nil] INFO [2022-01-10 05:46:16,451] jepsen worker 35 - jepsen.util 3455 :info :write [11 37] :RequestTimeoutException INFO [2022-01-10 05:46:16,451] jepsen worker 35 - jepsen.util 3545 :invoke :write [11 22] INFO [2022-01-10 05:46:16,536] jepsen worker 30 - jepsen.util 3360 :info :write [10 9] :RequestTimeoutException INFO [2022-01-10 05:46:16,537] jepsen worker 30 - jepsen.util 3450 :invoke :read [10 nil] INFO [2022-01-10 05:46:16,619] jepsen worker 42 - jepsen.util 3552 :ok :read [14 14] INFO [2022-01-10 05:46:16,620] jepsen worker 42 - jepsen.util 3552 :invoke :read [14 nil] INFO [2022-01-10 05:46:16,647] jepsen worker 33 - jepsen.util 3993 :ok :read [11 21] INFO [2022-01-10 05:46:16,647] jepsen worker 33 - jepsen.util 3993 :invoke :write [11 25] INFO [2022-01-10 05:46:16,678] jepsen worker 43 - jepsen.util 3733 :ok :read [14 14] INFO [2022-01-10 05:46:16,678] jepsen worker 43 - jepsen.util 3733 :invoke :read [14 nil] INFO [2022-01-10 05:46:16,785] jepsen worker 19 - jepsen.util 3979 :ok :read [6 25] INFO [2022-01-10 05:46:16,787] jepsen worker 19 - jepsen.util 3979 :invoke :read [6 nil] INFO [2022-01-10 05:46:16,790] jepsen worker 80 - jepsen.util 4580 :info :write [26 14] :RequestTimeoutException INFO [2022-01-10 05:46:16,791] jepsen worker 80 - jepsen.util 4670 :invoke :write [26 39] INFO [2022-01-10 05:46:16,802] jepsen worker 18 - jepsen.util 3618 :info :write [6 45] :RequestTimeoutException INFO [2022-01-10 05:46:16,803] jepsen worker 18 - jepsen.util 3708 :invoke :write [6 45] INFO [2022-01-10 05:46:16,811] jepsen worker 73 - jepsen.util 3403 :ok :read [24 18] INFO [2022-01-10 05:46:16,812] jepsen worker 73 - jepsen.util 3403 :invoke :write [24 27] INFO [2022-01-10 05:46:16,917] jepsen worker 42 - jepsen.util 3552 :ok :read [14 14] INFO [2022-01-10 05:46:16,918] jepsen worker 42 - jepsen.util 3552 :invoke :read [14 nil] INFO [2022-01-10 05:46:16,989] jepsen worker 74 - jepsen.util 4304 :ok :read [24 18] INFO [2022-01-10 05:46:16,989] jepsen worker 74 - jepsen.util 4304 :invoke :read [24 nil] INFO [2022-01-10 05:46:17,042] jepsen worker 19 - jepsen.util 3979 :ok :read [6 25] INFO [2022-01-10 05:46:17,043] jepsen worker 19 - jepsen.util 3979 :invoke :write [6 9] INFO [2022-01-10 05:46:17,160] jepsen worker 26 - jepsen.util 3446 :info :write [8 43] :RequestTimeoutException INFO [2022-01-10 05:46:17,161] jepsen worker 26 - jepsen.util 3536 :invoke :write [8 3] INFO [2022-01-10 05:46:17,194] jepsen worker 56 - jepsen.util 4106 :ok :read [18 14] INFO [2022-01-10 05:46:17,195] jepsen worker 56 - jepsen.util 4106 :invoke :write [18 26] INFO [2022-01-10 05:46:17,211] jepsen worker 57 - jepsen.util 4467 :info :write [19 6] :SyncWriteAmbiguous INFO [2022-01-10 05:46:17,212] jepsen worker 57 - jepsen.util 4557 :invoke :write [19 23] INFO [2022-01-10 05:46:17,228] jepsen worker 10 - jepsen.util 3700 :ok :read [3 31] INFO [2022-01-10 05:46:17,230] jepsen worker 10 - jepsen.util 3700 :invoke :read [3 nil] INFO [2022-01-10 05:46:17,236] jepsen worker 54 - jepsen.util 3834 :ok :read [18 14] INFO [2022-01-10 05:46:17,236] jepsen worker 54 - jepsen.util 3834 :invoke :read [18 nil] INFO [2022-01-10 05:46:17,238] jepsen worker 47 - jepsen.util 3827 :ok :read [15 23] INFO [2022-01-10 05:46:17,239] jepsen worker 47 - jepsen.util 3827 :invoke :read [15 nil] INFO [2022-01-10 05:46:17,252] jepsen worker 3 - jepsen.util 3603 :ok :read [1 26] INFO [2022-01-10 05:46:17,253] jepsen worker 3 - jepsen.util 3603 :invoke :read [1 nil] INFO [2022-01-10 05:46:17,291] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:17,291] jepsen worker nemesis - couchbase.util Rebalance check count: 526 INFO [2022-01-10 05:46:17,291] jepsen worker nemesis - couchbase.util Rebalance stuck count: 439 INFO [2022-01-10 05:46:17,295] jepsen worker 9 - jepsen.util 4239 :ok :read [3 31] INFO [2022-01-10 05:46:17,295] jepsen worker 9 - jepsen.util 4239 :invoke :write [3 12] INFO [2022-01-10 05:46:17,312] jepsen worker 38 - jepsen.util 3728 :ok :read [12 29] INFO [2022-01-10 05:46:17,315] jepsen worker 30 - jepsen.util 3450 :ok :read [10 36] INFO [2022-01-10 05:46:17,315] jepsen worker 38 - jepsen.util 3728 :invoke :write [12 13] INFO [2022-01-10 05:46:17,317] jepsen worker 30 - jepsen.util 3450 :invoke :read [10 nil] INFO [2022-01-10 05:46:17,331] jepsen worker 79 - jepsen.util 4399 :info :write [26 40] :RequestTimeoutException INFO [2022-01-10 05:46:17,331] jepsen worker 54 - jepsen.util 3834 :ok :read [18 14] INFO [2022-01-10 05:46:17,333] jepsen worker 54 - jepsen.util 3834 :invoke :write [18 26] INFO [2022-01-10 05:46:17,333] jepsen worker 79 - jepsen.util 4489 :invoke :write [26 36] INFO [2022-01-10 05:46:17,338] jepsen worker 10 - jepsen.util 3700 :ok :read [3 31] INFO [2022-01-10 05:46:17,340] jepsen worker 10 - jepsen.util 3700 :invoke :write [3 30] INFO [2022-01-10 05:46:17,380] jepsen worker 3 - jepsen.util 3603 :ok :read [1 26] INFO [2022-01-10 05:46:17,380] jepsen worker 3 - jepsen.util 3603 :invoke :write [1 30] INFO [2022-01-10 05:46:17,430] jepsen worker 30 - jepsen.util 3450 :ok :read [10 36] INFO [2022-01-10 05:46:17,431] jepsen worker 30 - jepsen.util 3450 :invoke :read [10 nil] INFO [2022-01-10 05:46:17,436] jepsen worker 86 - jepsen.util 3146 :info :write [28 26] :RequestTimeoutException INFO [2022-01-10 05:46:17,436] jepsen worker 86 - jepsen.util 3236 :invoke :write [28 31] INFO [2022-01-10 05:46:17,442] jepsen worker 52 - jepsen.util 4732 :info :write [17 16] :RequestTimeoutException INFO [2022-01-10 05:46:17,443] jepsen worker 52 - jepsen.util 4822 :invoke :read [17 nil] INFO [2022-01-10 05:46:17,487] jepsen worker 83 - jepsen.util 4223 :info :write [27 32] :RequestTimeoutException INFO [2022-01-10 05:46:17,487] jepsen worker 83 - jepsen.util 4313 :invoke :read [27 nil] INFO [2022-01-10 05:46:17,508] jepsen worker 65 - jepsen.util 3575 :ok :read [21 7] INFO [2022-01-10 05:46:17,509] jepsen worker 65 - jepsen.util 3575 :invoke :read [21 nil] INFO [2022-01-10 05:46:17,527] jepsen worker 42 - jepsen.util 3552 :ok :read [14 14] INFO [2022-01-10 05:46:17,534] jepsen worker 42 - jepsen.util 3552 :invoke :write [14 44] INFO [2022-01-10 05:46:17,546] jepsen worker 1 - jepsen.util 3781 :info :write [0 23] :RequestTimeoutException INFO [2022-01-10 05:46:17,547] jepsen worker 1 - jepsen.util 3871 :invoke :read [0 nil] INFO [2022-01-10 05:46:17,547] jepsen worker 8 - jepsen.util 4238 :ok :read [2 14] INFO [2022-01-10 05:46:17,548] jepsen worker 8 - jepsen.util 4238 :invoke :read [2 nil] INFO [2022-01-10 05:46:17,555] jepsen worker 52 - jepsen.util 4822 :ok :read [17 41] INFO [2022-01-10 05:46:17,555] jepsen worker 52 - jepsen.util 4822 :invoke :read [17 nil] INFO [2022-01-10 05:46:17,597] jepsen worker 74 - jepsen.util 4304 :ok :read [24 18] INFO [2022-01-10 05:46:17,598] jepsen worker 74 - jepsen.util 4304 :invoke :write [24 13] INFO [2022-01-10 05:46:17,638] jepsen worker 30 - jepsen.util 3450 :ok :read [10 36] INFO [2022-01-10 05:46:17,639] jepsen worker 30 - jepsen.util 3450 :invoke :write [10 5] INFO [2022-01-10 05:46:17,694] jepsen worker 23 - jepsen.util 3803 :info :write [7 44] :RequestTimeoutException INFO [2022-01-10 05:46:17,695] jepsen worker 23 - jepsen.util 3893 :invoke :write [7 16] INFO [2022-01-10 05:46:17,796] jepsen worker 22 - jepsen.util 3442 :info :write [7 47] :RequestTimeoutException INFO [2022-01-10 05:46:17,797] jepsen worker 22 - jepsen.util 3532 :invoke :read [7 nil] INFO [2022-01-10 05:46:17,850] jepsen worker 55 - jepsen.util 4105 :info :write [18 24] :RequestTimeoutException INFO [2022-01-10 05:46:17,851] jepsen worker 55 - jepsen.util 4195 :invoke :write [18 39] INFO [2022-01-10 05:46:17,934] jepsen worker 62 - jepsen.util 4202 :ok :read [20 :nil] INFO [2022-01-10 05:46:17,934] jepsen worker 77 - jepsen.util 4217 :ok :read [25 :nil] INFO [2022-01-10 05:46:17,935] jepsen worker 62 - jepsen.util 4202 :invoke :write [20 29] INFO [2022-01-10 05:46:17,936] jepsen worker 77 - jepsen.util 4217 :invoke :write [25 38] INFO [2022-01-10 05:46:18,023] jepsen worker 52 - jepsen.util 4822 :ok :read [17 41] INFO [2022-01-10 05:46:18,024] jepsen worker 52 - jepsen.util 4822 :invoke :read [17 nil] INFO [2022-01-10 05:46:18,129] jepsen worker 52 - jepsen.util 4822 :ok :read [17 41] INFO [2022-01-10 05:46:18,130] jepsen worker 52 - jepsen.util 4822 :invoke :read [17 nil] INFO [2022-01-10 05:46:18,264] jepsen worker 52 - jepsen.util 4822 :ok :read [17 41] INFO [2022-01-10 05:46:18,265] jepsen worker 52 - jepsen.util 4822 :invoke :read [17 nil] INFO [2022-01-10 05:46:18,296] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:18,296] jepsen worker nemesis - couchbase.util Rebalance check count: 527 INFO [2022-01-10 05:46:18,296] jepsen worker nemesis - couchbase.util Rebalance stuck count: 440 INFO [2022-01-10 05:46:18,337] jepsen worker 1 - jepsen.util 3871 :ok :read [0 46] INFO [2022-01-10 05:46:18,338] jepsen worker 1 - jepsen.util 3871 :invoke :write [0 1] INFO [2022-01-10 05:46:18,373] jepsen worker 50 - jepsen.util 3920 :info :write [16 3] :SyncWriteAmbiguous INFO [2022-01-10 05:46:18,374] jepsen worker 50 - jepsen.util 4010 :invoke :write [16 49] INFO [2022-01-10 05:46:18,495] jepsen worker 8 - jepsen.util 4238 :ok :read [2 14] INFO [2022-01-10 05:46:18,495] jepsen worker 8 - jepsen.util 4238 :invoke :write [2 7] INFO [2022-01-10 05:46:18,521] jepsen worker 65 - jepsen.util 3575 :ok :read [21 7] INFO [2022-01-10 05:46:18,522] jepsen worker 65 - jepsen.util 3575 :invoke :read [21 nil] INFO [2022-01-10 05:46:18,523] jepsen worker 52 - jepsen.util 4822 :ok :read [17 41] INFO [2022-01-10 05:46:18,524] jepsen worker 52 - jepsen.util 4822 :invoke :read [17 nil] INFO [2022-01-10 05:46:18,614] jepsen worker 85 - jepsen.util 4045 :info :write [28 27] :RequestTimeoutException INFO [2022-01-10 05:46:18,615] jepsen worker 85 - jepsen.util 4135 :invoke :read [28 nil] INFO [2022-01-10 05:46:18,620] jepsen worker 52 - jepsen.util 4822 :ok :read [17 41] INFO [2022-01-10 05:46:18,620] jepsen worker 52 - jepsen.util 4822 :invoke :read [17 nil] INFO [2022-01-10 05:46:18,645] jepsen worker 16 - jepsen.util 3796 :info :write [5 18] :RequestTimeoutException INFO [2022-01-10 05:46:18,645] jepsen worker 16 - jepsen.util 3886 :invoke :read [5 nil] INFO [2022-01-10 05:46:18,695] jepsen worker 52 - jepsen.util 4822 :ok :read [17 41] INFO [2022-01-10 05:46:18,695] jepsen worker 52 - jepsen.util 4822 :invoke :read [17 nil] INFO [2022-01-10 05:46:18,720] jepsen worker 52 - jepsen.util 4822 :ok :read [17 41] INFO [2022-01-10 05:46:18,721] jepsen worker 52 - jepsen.util 4822 :invoke :read [17 nil] INFO [2022-01-10 05:46:18,750] jepsen worker 52 - jepsen.util 4822 :ok :read [17 41] INFO [2022-01-10 05:46:18,751] jepsen worker 52 - jepsen.util 4822 :invoke :read [17 nil] INFO [2022-01-10 05:46:18,782] jepsen worker 47 - jepsen.util 3827 :ok :read [15 23] INFO [2022-01-10 05:46:18,783] jepsen worker 47 - jepsen.util 3827 :invoke :read [15 nil] INFO [2022-01-10 05:46:18,783] jepsen worker 52 - jepsen.util 4822 :ok :read [17 41] INFO [2022-01-10 05:46:18,784] jepsen worker 52 - jepsen.util 4822 :invoke :read [17 nil] INFO [2022-01-10 05:46:18,820] jepsen worker 52 - jepsen.util 4822 :ok :read [17 41] INFO [2022-01-10 05:46:18,821] jepsen worker 52 - jepsen.util 4822 :invoke :write [17 24] INFO [2022-01-10 05:46:18,941] jepsen worker 71 - jepsen.util 3311 :info :write [23 9] :RequestTimeoutException INFO [2022-01-10 05:46:18,942] jepsen worker 83 - jepsen.util 4313 :ok :read [27 30] INFO [2022-01-10 05:46:18,942] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:18,943] jepsen worker 83 - jepsen.util 4313 :invoke :read [27 nil] INFO [2022-01-10 05:46:19,302] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:19,302] jepsen worker nemesis - couchbase.util Rebalance check count: 528 INFO [2022-01-10 05:46:19,334] jepsen worker nemesis - couchbase.util Rebalance stuck count: 441 INFO [2022-01-10 05:46:19,405] jepsen worker 16 - jepsen.util 3886 :ok :read [5 25] INFO [2022-01-10 05:46:19,405] jepsen worker 16 - jepsen.util 3886 :invoke :write [5 44] INFO [2022-01-10 05:46:19,640] jepsen worker 6 - jepsen.util 3786 :info :write [2 19] :RequestTimeoutException INFO [2022-01-10 05:46:19,640] jepsen worker 6 - jepsen.util 3876 :invoke :read [2 nil] INFO [2022-01-10 05:46:19,739] jepsen worker 22 - jepsen.util 3532 :ok :read [7 :nil] INFO [2022-01-10 05:46:19,740] jepsen worker 22 - jepsen.util 3532 :invoke :read [7 nil] INFO [2022-01-10 05:46:19,770] jepsen worker 47 - jepsen.util 3827 :ok :read [15 23] INFO [2022-01-10 05:46:19,770] jepsen worker 47 - jepsen.util 3827 :invoke :read [15 nil] INFO [2022-01-10 05:46:19,772] jepsen worker 85 - jepsen.util 4135 :ok :read [28 45] INFO [2022-01-10 05:46:19,773] jepsen worker 85 - jepsen.util 4135 :invoke :write [28 36] INFO [2022-01-10 05:46:19,779] jepsen worker 65 - jepsen.util 3575 :ok :read [21 7] INFO [2022-01-10 05:46:19,779] jepsen worker 65 - jepsen.util 3575 :invoke :write [21 9] INFO [2022-01-10 05:46:19,807] jepsen worker 27 - jepsen.util 3897 :info :write [9 3] :RequestTimeoutException INFO [2022-01-10 05:46:19,808] jepsen worker 27 - jepsen.util 3987 :invoke :read [9 nil] INFO [2022-01-10 05:46:19,911] jepsen worker 89 - jepsen.util 4229 :info :write [29 24] :RequestTimeoutException INFO [2022-01-10 05:46:19,912] jepsen worker 89 - jepsen.util 4319 :invoke :write [29 29] INFO [2022-01-10 05:46:20,031] jepsen worker 27 - jepsen.util 3987 :ok :read [9 27] INFO [2022-01-10 05:46:20,031] jepsen worker 27 - jepsen.util 3987 :invoke :write [9 3] INFO [2022-01-10 05:46:20,034] jepsen worker 83 - jepsen.util 4313 :ok :read [27 30] INFO [2022-01-10 05:46:20,035] jepsen worker 83 - jepsen.util 4313 :invoke :read [27 nil] INFO [2022-01-10 05:46:20,181] jepsen worker 45 - jepsen.util 4005 :info :write [15 16] :RequestTimeoutException INFO [2022-01-10 05:46:20,183] jepsen worker 45 - jepsen.util 4095 :invoke :write [15 32] INFO [2022-01-10 05:46:20,186] jepsen worker 13 - jepsen.util 4333 :info :write [4 6] :RequestTimeoutException INFO [2022-01-10 05:46:20,187] jepsen worker 13 - jepsen.util 4423 :invoke :write [4 29] INFO [2022-01-10 05:46:20,299] jepsen worker 22 - jepsen.util 3532 :ok :read [7 :nil] INFO [2022-01-10 05:46:20,300] jepsen worker 22 - jepsen.util 3532 :invoke :write [7 4] INFO [2022-01-10 05:46:20,323] jepsen worker 6 - jepsen.util 3876 :ok :read [2 14] INFO [2022-01-10 05:46:20,324] jepsen worker 6 - jepsen.util 3876 :invoke :write [2 26] INFO [2022-01-10 05:46:20,341] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:20,341] jepsen worker nemesis - couchbase.util Rebalance check count: 529 INFO [2022-01-10 05:46:20,341] jepsen worker nemesis - couchbase.util Rebalance stuck count: 442 INFO [2022-01-10 05:46:20,342] jepsen worker 84 - jepsen.util 3684 :info :write [28 15] :RequestTimeoutException INFO [2022-01-10 05:46:20,343] jepsen worker 84 - jepsen.util 3774 :invoke :read [28 nil] INFO [2022-01-10 05:46:20,351] jepsen worker 44 - jepsen.util 3824 :info :write [14 18] :RequestTimeoutException INFO [2022-01-10 05:46:20,352] jepsen worker 44 - jepsen.util 3914 :invoke :read [14 nil] WARN [2022-01-10 05:46:20,398] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":21030998,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":21030998,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9020392,"encode_duration_us":24828,"operation_id":"0x7349","timeout_ms":10000,"last_server_duration_us":9020392,"total_duration_us":10002319},{"operation_name":"upsert","last_dispatch_duration_us":8990906,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10721223,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":8940111,"encode_duration_us":17793,"operation_id":"0x740e","timeout_ms":10000,"last_server_duration_us":8790463,"total_duration_us":10004705}],"total_count":2}} INFO [2022-01-10 05:46:20,441] jepsen worker 7 - jepsen.util 3877 :info :write [2 37] :RequestTimeoutException INFO [2022-01-10 05:46:20,442] jepsen worker 7 - jepsen.util 3967 :invoke :write [2 31] WARN [2022-01-10 05:46:20,499] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16187608,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":16187608,"total_server_duration_us":49,"operation_id":"0x7443","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":16189736},{"operation_name":"upsert","last_dispatch_duration_us":8523153,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9818502,"total_server_duration_us":842183,"encode_duration_us":18595,"operation_id":"0x75a8","timeout_ms":10000,"last_server_duration_us":351353,"total_duration_us":10009069},{"operation_name":"upsert","last_dispatch_duration_us":194090,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":3333569,"total_server_duration_us":1844744,"encode_duration_us":16291,"operation_id":"0x74eb","timeout_ms":10000,"last_server_duration_us":162603,"total_duration_us":10008630},{"operation_name":"upsert","last_dispatch_duration_us":8604350,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11183364,"total_server_duration_us":961506,"encode_duration_us":18476,"operation_id":"0x74a8","timeout_ms":10000,"last_server_duration_us":121497,"total_duration_us":10008416},{"operation_name":"upsert","last_dispatch_duration_us":11233370,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":11233370,"total_server_duration_us":19108,"encode_duration_us":19242,"operation_id":"0x7534","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10007083},{"operation_name":"upsert","last_dispatch_duration_us":442905,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1647241,"total_server_duration_us":418471,"encode_duration_us":26959,"operation_id":"0x757d","timeout_ms":10000,"last_server_duration_us":89584,"total_duration_us":10003990},{"operation_name":"upsert","last_dispatch_duration_us":124783,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":3148065,"total_server_duration_us":2441395,"encode_duration_us":21436,"operation_id":"0x7479","timeout_ms":10000,"last_server_duration_us":114123,"total_duration_us":10002477},{"operation_name":"upsert","last_dispatch_duration_us":8532136,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9829587,"total_server_duration_us":480222,"encode_duration_us":23415,"operation_id":"0x75aa","timeout_ms":10000,"last_server_duration_us":23813,"total_duration_us":10002399},{"operation_name":"upsert","last_dispatch_duration_us":9141018,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9716089,"total_server_duration_us":9170451,"encode_duration_us":17098,"operation_id":"0x75a9","timeout_ms":10000,"last_server_duration_us":9049045,"total_duration_us":9748663},{"operation_name":"get","last_dispatch_duration_us":9123927,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9123927,"total_server_duration_us":43,"operation_id":"0x759e","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":9124970}],"total_count":23}} INFO [2022-01-10 05:46:20,697] jepsen worker 47 - jepsen.util 3827 :ok :read [15 23] INFO [2022-01-10 05:46:20,698] jepsen worker 47 - jepsen.util 3827 :invoke :read [15 nil] INFO [2022-01-10 05:46:20,951] jepsen worker 64 - jepsen.util 3754 :info :write [21 43] :RequestTimeoutException INFO [2022-01-10 05:46:20,952] jepsen worker 64 - jepsen.util 3844 :invoke :write [21 39] INFO [2022-01-10 05:46:20,964] jepsen worker 39 - jepsen.util 4359 :ok :read [13 15] INFO [2022-01-10 05:46:20,965] jepsen worker 39 - jepsen.util 4359 :invoke :read [13 nil] INFO [2022-01-10 05:46:21,041] jepsen worker 39 - jepsen.util 4359 :ok :read [13 15] INFO [2022-01-10 05:46:21,042] jepsen worker 39 - jepsen.util 4359 :invoke :read [13 nil] INFO [2022-01-10 05:46:21,154] jepsen worker 39 - jepsen.util 4359 :ok :read [13 15] INFO [2022-01-10 05:46:21,155] jepsen worker 39 - jepsen.util 4359 :invoke :read [13 nil] INFO [2022-01-10 05:46:21,218] jepsen worker 39 - jepsen.util 4359 :ok :read [13 15] INFO [2022-01-10 05:46:21,218] jepsen worker 39 - jepsen.util 4359 :invoke :read [13 nil] INFO [2022-01-10 05:46:21,221] jepsen worker 83 - jepsen.util 4313 :ok :read [27 30] INFO [2022-01-10 05:46:21,222] jepsen worker 83 - jepsen.util 4313 :invoke :write [27 24] INFO [2022-01-10 05:46:21,342] jepsen worker 47 - jepsen.util 3827 :ok :read [15 23] INFO [2022-01-10 05:46:21,342] jepsen worker 47 - jepsen.util 3827 :invoke :read [15 nil] INFO [2022-01-10 05:46:21,346] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:21,347] jepsen worker nemesis - couchbase.util Rebalance check count: 530 INFO [2022-01-10 05:46:21,347] jepsen worker nemesis - couchbase.util Rebalance stuck count: 443 INFO [2022-01-10 05:46:21,402] jepsen worker 48 - jepsen.util 3468 :info :write [16 8] :RequestTimeoutException INFO [2022-01-10 05:46:21,403] jepsen worker 48 - jepsen.util 3558 :invoke :read [16 nil] INFO [2022-01-10 05:46:21,409] jepsen worker 39 - jepsen.util 4359 :ok :read [13 15] INFO [2022-01-10 05:46:21,409] jepsen worker 39 - jepsen.util 4359 :invoke :write [13 27] INFO [2022-01-10 05:46:21,499] jepsen worker 14 - jepsen.util 4604 :info :write [4 32] :RequestTimeoutException INFO [2022-01-10 05:46:21,500] jepsen worker 14 - jepsen.util 4694 :invoke :write [4 6] INFO [2022-01-10 05:46:21,751] jepsen worker 76 - jepsen.util 3316 :info :write [25 43] :RequestTimeoutException INFO [2022-01-10 05:46:21,751] jepsen worker 76 - jepsen.util 3406 :invoke :write [25 43] WARN [2022-01-10 05:46:21,996] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9158740,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16703501,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9089870,"encode_duration_us":16888,"operation_id":"0x73ac","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10006009},{"operation_name":"upsert","last_dispatch_duration_us":9241681,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12237014,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":10015236,"encode_duration_us":17953,"operation_id":"0x74d0","timeout_ms":10000,"last_server_duration_us":9221773,"total_duration_us":10006111},{"operation_name":"upsert","last_dispatch_duration_us":10024901,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11387405,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9558388,"encode_duration_us":17852,"operation_id":"0x7577","timeout_ms":10000,"last_server_duration_us":9484549,"total_duration_us":10008245}],"total_count":3}} INFO [2022-01-10 05:46:22,004] jepsen worker 47 - jepsen.util 3827 :ok :read [15 23] INFO [2022-01-10 05:46:22,005] jepsen worker 47 - jepsen.util 3827 :invoke :read [15 nil] INFO [2022-01-10 05:46:22,057] jepsen worker 43 - jepsen.util 3733 :ok :read [14 14] INFO [2022-01-10 05:46:22,057] jepsen worker 43 - jepsen.util 3733 :invoke :read [14 nil] WARN [2022-01-10 05:46:22,083] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13391929,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":13391929,"total_server_duration_us":27,"operation_id":"0x73b8","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":13392982},{"operation_name":"upsert","last_dispatch_duration_us":10229484,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10229484,"total_server_duration_us":9195027,"encode_duration_us":18066,"operation_id":"0x7528","timeout_ms":10000,"last_server_duration_us":9195027,"total_duration_us":10011163},{"operation_name":"upsert","last_dispatch_duration_us":907461,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2749048,"total_server_duration_us":373627,"encode_duration_us":17652,"operation_id":"0x7608","timeout_ms":10000,"last_server_duration_us":49487,"total_duration_us":10010976},{"operation_name":"upsert","encode_duration_us":19547,"operation_id":"0x7543","timeout_ms":10000,"total_duration_us":10010255},{"operation_name":"upsert","last_dispatch_duration_us":43103,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":791772,"total_server_duration_us":381460,"encode_duration_us":17205,"operation_id":"0x75c9","timeout_ms":10000,"last_server_duration_us":21001,"total_duration_us":10010095},{"operation_name":"upsert","last_dispatch_duration_us":850983,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2680373,"total_server_duration_us":310459,"encode_duration_us":21449,"operation_id":"0x7647","timeout_ms":10000,"last_server_duration_us":77711,"total_duration_us":10006781},{"operation_name":"upsert","last_dispatch_duration_us":9205884,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10769873,"total_server_duration_us":278536,"encode_duration_us":20817,"operation_id":"0x7470","timeout_ms":10000,"last_server_duration_us":152366,"total_duration_us":10006327},{"operation_name":"upsert","last_dispatch_duration_us":1049758,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10796801,"total_server_duration_us":298007,"encode_duration_us":17283,"operation_id":"0x74a0","timeout_ms":10000,"last_server_duration_us":162411,"total_duration_us":10004297},{"operation_name":"upsert","last_dispatch_duration_us":32348,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":653141,"total_server_duration_us":358118,"encode_duration_us":17119,"operation_id":"0x75f7","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10003807},{"operation_name":"upsert","last_dispatch_duration_us":156436,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":3401184,"total_server_duration_us":2791910,"encode_duration_us":22394,"operation_id":"0x7576","timeout_ms":10000,"last_server_duration_us":138439,"total_duration_us":10003230}],"total_count":16}} WARN [2022-01-10 05:46:22,083] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17997955,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20557307,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9151539,"encode_duration_us":21195,"operation_id":"0x7218","timeout_ms":10000,"last_server_duration_us":9072420,"total_duration_us":10001811},{"operation_name":"upsert","last_dispatch_duration_us":10229484,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10229484,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9195027,"encode_duration_us":18066,"operation_id":"0x7528","timeout_ms":10000,"last_server_duration_us":9195027,"total_duration_us":10011163}],"total_count":2}} INFO [2022-01-10 05:46:22,099] jepsen worker 75 - jepsen.util 3405 :info :write [25 14] :RequestTimeoutException INFO [2022-01-10 05:46:22,100] jepsen worker 75 - jepsen.util 3495 :invoke :write [25 48] INFO [2022-01-10 05:46:22,101] jepsen worker 17 - jepsen.util 3617 :info :write [5 35] :RequestTimeoutException INFO [2022-01-10 05:46:22,102] jepsen worker 17 - jepsen.util 3707 :invoke :read [5 nil] INFO [2022-01-10 05:46:22,116] jepsen worker 82 - jepsen.util 3862 :info :write [27 31] :RequestTimeoutException INFO [2022-01-10 05:46:22,117] jepsen worker 82 - jepsen.util 3952 :invoke :write [27 47] WARN [2022-01-10 05:46:22,185] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":527829,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":3663751,"total_server_duration_us":172408,"encode_duration_us":17181,"operation_id":"0x75ed","timeout_ms":10000,"last_server_duration_us":41767,"total_duration_us":10011345},{"operation_name":"upsert","last_dispatch_duration_us":40692,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9649939,"total_server_duration_us":219132,"encode_duration_us":17302,"operation_id":"0x74fa","timeout_ms":10000,"last_server_duration_us":23390,"total_duration_us":10010622},{"operation_name":"upsert","last_dispatch_duration_us":8670458,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9061940,"total_server_duration_us":61926,"encode_duration_us":23795,"operation_id":"0x7503","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10010484},{"operation_name":"upsert","last_dispatch_duration_us":9438796,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9979081,"total_server_duration_us":164431,"encode_duration_us":19741,"operation_id":"0x7541","timeout_ms":10000,"last_server_duration_us":124055,"total_duration_us":10008878},{"operation_name":"upsert","last_dispatch_duration_us":9680245,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11803585,"total_server_duration_us":9327190,"encode_duration_us":23049,"operation_id":"0x74cf","timeout_ms":10000,"last_server_duration_us":9166178,"total_duration_us":10007444},{"operation_name":"upsert","last_dispatch_duration_us":9633216,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":13330405,"total_server_duration_us":10456746,"encode_duration_us":17457,"operation_id":"0x747b","timeout_ms":10000,"last_server_duration_us":9340925,"total_duration_us":10007399},{"operation_name":"upsert","last_dispatch_duration_us":176915,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":751694,"total_server_duration_us":70988,"encode_duration_us":19052,"operation_id":"0x74fb","timeout_ms":10000,"last_server_duration_us":22389,"total_duration_us":10006555},{"operation_name":"upsert","last_dispatch_duration_us":62009,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9619156,"total_server_duration_us":159289,"encode_duration_us":27596,"operation_id":"0x74fc","timeout_ms":10000,"last_server_duration_us":30445,"total_duration_us":10006162},{"operation_name":"upsert","last_dispatch_duration_us":576000,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10231783,"total_server_duration_us":228920,"encode_duration_us":23447,"operation_id":"0x749b","timeout_ms":10000,"last_server_duration_us":78837,"total_duration_us":10003568},{"operation_name":"upsert","last_dispatch_duration_us":283157,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":6309034,"total_server_duration_us":295792,"encode_duration_us":17290,"operation_id":"0x757a","timeout_ms":10000,"last_server_duration_us":99502,"total_duration_us":10002906}],"total_count":31}} WARN [2022-01-10 05:46:22,191] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2484438,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":5990331,"total_server_duration_us":1264100,"encode_duration_us":20761,"operation_id":"0x752f","timeout_ms":10000,"last_server_duration_us":68547,"total_duration_us":10010359},{"operation_name":"upsert","last_dispatch_duration_us":103420,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":3805320,"total_server_duration_us":1745183,"encode_duration_us":17220,"operation_id":"0x749f","timeout_ms":10000,"last_server_duration_us":73410,"total_duration_us":10009433},{"operation_name":"upsert","last_dispatch_duration_us":128427,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6885073,"total_server_duration_us":732393,"encode_duration_us":17287,"operation_id":"0x7560","timeout_ms":10000,"last_server_duration_us":115782,"total_duration_us":10009158},{"operation_name":"upsert","last_dispatch_duration_us":310198,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":4131881,"total_server_duration_us":2954363,"encode_duration_us":16372,"operation_id":"0x7578","timeout_ms":10000,"last_server_duration_us":137722,"total_duration_us":10005457},{"operation_name":"upsert","last_dispatch_duration_us":338057,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2547373,"total_server_duration_us":789880,"encode_duration_us":18683,"operation_id":"0x7600","timeout_ms":10000,"last_server_duration_us":188605,"total_duration_us":10005387},{"operation_name":"upsert","last_dispatch_duration_us":714560,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":5569163,"total_server_duration_us":1407266,"encode_duration_us":18756,"operation_id":"0x7605","timeout_ms":10000,"last_server_duration_us":40270,"total_duration_us":10004434},{"operation_name":"upsert","last_dispatch_duration_us":2566074,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":5884042,"total_server_duration_us":1152414,"encode_duration_us":23301,"operation_id":"0x7530","timeout_ms":10000,"last_server_duration_us":71502,"total_duration_us":10003114},{"operation_name":"upsert","last_dispatch_duration_us":583836,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9203355,"total_server_duration_us":248215,"encode_duration_us":16525,"operation_id":"0x746f","timeout_ms":10000,"last_server_duration_us":44611,"total_duration_us":10002659},{"operation_name":"upsert","last_dispatch_duration_us":349043,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8296966,"total_server_duration_us":685318,"encode_duration_us":26833,"operation_id":"0x7536","timeout_ms":10000,"last_server_duration_us":44944,"total_duration_us":10002309},{"operation_name":"upsert","last_dispatch_duration_us":739953,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":6580850,"total_server_duration_us":802581,"encode_duration_us":17979,"operation_id":"0x7632","timeout_ms":10000,"last_server_duration_us":45278,"total_duration_us":10002305}],"total_count":16}} WARN [2022-01-10 05:46:22,196] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":934739,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10423835,"total_server_duration_us":268940,"encode_duration_us":19179,"operation_id":"0x7590","timeout_ms":10000,"last_server_duration_us":109209,"total_duration_us":10010617},{"operation_name":"upsert","last_dispatch_duration_us":9623194,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":13832477,"total_server_duration_us":419004,"encode_duration_us":23399,"operation_id":"0x7509","timeout_ms":10000,"last_server_duration_us":38690,"total_duration_us":10010412},{"operation_name":"upsert","last_dispatch_duration_us":9856361,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":13656022,"total_server_duration_us":371204,"encode_duration_us":17232,"operation_id":"0x7531","timeout_ms":10000,"last_server_duration_us":119804,"total_duration_us":10008251},{"operation_name":"upsert","last_dispatch_duration_us":10024901,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11387405,"total_server_duration_us":9558388,"encode_duration_us":17852,"operation_id":"0x7577","timeout_ms":10000,"last_server_duration_us":9484549,"total_duration_us":10008245},{"operation_name":"upsert","last_dispatch_duration_us":7366520,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8265350,"total_server_duration_us":271797,"encode_duration_us":19252,"operation_id":"0x7508","timeout_ms":10000,"last_server_duration_us":31012,"total_duration_us":10007368},{"operation_name":"upsert","last_dispatch_duration_us":10264990,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12636363,"total_server_duration_us":324015,"encode_duration_us":17396,"operation_id":"0x7478","timeout_ms":10000,"last_server_duration_us":161644,"total_duration_us":10006208},{"operation_name":"upsert","last_dispatch_duration_us":9241681,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12237014,"total_server_duration_us":10015236,"encode_duration_us":17953,"operation_id":"0x74d0","timeout_ms":10000,"last_server_duration_us":9221773,"total_duration_us":10006111},{"operation_name":"upsert","last_dispatch_duration_us":7876810,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8043959,"total_server_duration_us":104660,"encode_duration_us":17544,"operation_id":"0x7546","timeout_ms":10000,"last_server_duration_us":21568,"total_duration_us":10004568},{"operation_name":"upsert","last_dispatch_duration_us":100403,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":3351441,"total_server_duration_us":1569421,"encode_duration_us":22166,"operation_id":"0x7601","timeout_ms":10000,"last_server_duration_us":21001,"total_duration_us":10003754},{"operation_name":"upsert","last_dispatch_duration_us":9262470,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10445229,"total_server_duration_us":357871,"encode_duration_us":17916,"operation_id":"0x7544","timeout_ms":10000,"last_server_duration_us":47762,"total_duration_us":10002003}],"total_count":21}} INFO [2022-01-10 05:46:22,202] jepsen worker 44 - jepsen.util 3914 :ok :read [14 14] INFO [2022-01-10 05:46:22,203] jepsen worker 44 - jepsen.util 3914 :invoke :write [14 13] INFO [2022-01-10 05:46:22,226] jepsen worker 0 - jepsen.util 3600 :info :write [0 9] :RequestTimeoutException INFO [2022-01-10 05:46:22,227] jepsen worker 0 - jepsen.util 3690 :invoke :write [0 15] INFO [2022-01-10 05:46:22,236] jepsen worker 12 - jepsen.util 4242 :info :write [4 0] :RequestTimeoutException INFO [2022-01-10 05:46:22,237] jepsen worker 12 - jepsen.util 4332 :invoke :write [4 43] WARN [2022-01-10 05:46:22,285] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9284460,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13638159,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9056078,"encode_duration_us":16296,"operation_id":"0x7434","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10007346},{"operation_name":"upsert","last_dispatch_duration_us":9633216,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13330405,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":10456746,"encode_duration_us":17457,"operation_id":"0x747b","timeout_ms":10000,"last_server_duration_us":9340925,"total_duration_us":10007399},{"operation_name":"upsert","last_dispatch_duration_us":9680245,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11803585,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9327190,"encode_duration_us":23049,"operation_id":"0x74cf","timeout_ms":10000,"last_server_duration_us":9166178,"total_duration_us":10007444}],"total_count":3}} WARN [2022-01-10 05:46:22,285] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10112739,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12951703,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9687904,"encode_duration_us":17741,"operation_id":"0x744d","timeout_ms":10000,"last_server_duration_us":9303284,"total_duration_us":10008792}],"total_count":1}} WARN [2022-01-10 05:46:22,291] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":24820207,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":24820207,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9066042,"encode_duration_us":18954,"operation_id":"0x7208","timeout_ms":10000,"last_server_duration_us":9066042,"total_duration_us":10004835}],"total_count":1}} INFO [2022-01-10 05:46:22,352] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:22,352] jepsen worker nemesis - couchbase.util Rebalance check count: 531 INFO [2022-01-10 05:46:22,352] jepsen worker nemesis - couchbase.util Rebalance stuck count: 444 WARN [2022-01-10 05:46:22,386] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":322729,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2539093,"total_server_duration_us":954834,"encode_duration_us":16608,"operation_id":"0x75d3","timeout_ms":10000,"last_server_duration_us":131339,"total_duration_us":10010419},{"operation_name":"upsert","last_dispatch_duration_us":342119,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9872654,"total_server_duration_us":651966,"encode_duration_us":17246,"operation_id":"0x7574","timeout_ms":10000,"last_server_duration_us":95023,"total_duration_us":10009808},{"operation_name":"upsert","last_dispatch_duration_us":400875,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10197397,"total_server_duration_us":249531,"encode_duration_us":16942,"operation_id":"0x7506","timeout_ms":10000,"last_server_duration_us":100910,"total_duration_us":10009655},{"operation_name":"upsert","last_dispatch_duration_us":32902,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":3258297,"total_server_duration_us":1580118,"encode_duration_us":16938,"operation_id":"0x7633","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10009501},{"operation_name":"upsert","last_dispatch_duration_us":8687448,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11427012,"total_server_duration_us":548556,"encode_duration_us":18822,"operation_id":"0x7535","timeout_ms":10000,"last_server_duration_us":279354,"total_duration_us":10007853},{"operation_name":"upsert","last_dispatch_duration_us":861211,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":3678304,"total_server_duration_us":554380,"encode_duration_us":19578,"operation_id":"0x7658","timeout_ms":10000,"last_server_duration_us":172531,"total_duration_us":10007027},{"operation_name":"upsert","last_dispatch_duration_us":32555,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":814570,"total_server_duration_us":434905,"encode_duration_us":16632,"operation_id":"0x759d","timeout_ms":10000,"last_server_duration_us":20045,"total_duration_us":10006264},{"operation_name":"upsert","last_dispatch_duration_us":60620,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":3927154,"total_server_duration_us":1320144,"encode_duration_us":16414,"operation_id":"0x74d5","timeout_ms":10000,"last_server_duration_us":31585,"total_duration_us":10002332},{"operation_name":"upsert","last_dispatch_duration_us":8568418,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10998377,"total_server_duration_us":494078,"encode_duration_us":19206,"operation_id":"0x753e","timeout_ms":10000,"last_server_duration_us":160304,"total_duration_us":10001739},{"operation_name":"upsert","last_dispatch_duration_us":571588,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":4356412,"total_server_duration_us":1771501,"encode_duration_us":16599,"operation_id":"0x755a","timeout_ms":10000,"last_server_duration_us":58128,"total_duration_us":10001647}],"total_count":16}} INFO [2022-01-10 05:46:22,462] jepsen worker 15 - jepsen.util 3255 :info :write [5 9] :RequestTimeoutException INFO [2022-01-10 05:46:22,466] jepsen worker 15 - jepsen.util 3345 :invoke :write [5 9] INFO [2022-01-10 05:46:22,511] jepsen worker 68 - jepsen.util 3848 :info :write [22 20] :RequestTimeoutException INFO [2022-01-10 05:46:22,512] jepsen worker 68 - jepsen.util 3938 :invoke :write [22 27] INFO [2022-01-10 05:46:22,531] jepsen worker 34 - jepsen.util 3634 :info :write [11 5] :RequestTimeoutException INFO [2022-01-10 05:46:22,531] jepsen worker 34 - jepsen.util 3724 :invoke :write [11 15] INFO [2022-01-10 05:46:22,598] jepsen worker 47 - jepsen.util 3827 :ok :read [15 23] INFO [2022-01-10 05:46:22,599] jepsen worker 47 - jepsen.util 3827 :invoke :write [15 47] INFO [2022-01-10 05:46:22,879] jepsen worker 43 - jepsen.util 3733 :ok :read [14 14] INFO [2022-01-10 05:46:22,879] jepsen worker 43 - jepsen.util 3733 :invoke :read [14 nil] INFO [2022-01-10 05:46:22,885] jepsen worker 66 - jepsen.util 3486 :info :write [22 45] :RequestTimeoutException INFO [2022-01-10 05:46:22,885] jepsen worker 66 - jepsen.util 3576 :invoke :write [22 23] INFO [2022-01-10 05:46:23,001] jepsen worker 78 - jepsen.util 4578 :info :write [26 22] :RequestTimeoutException INFO [2022-01-10 05:46:23,002] jepsen worker 78 - jepsen.util 4668 :invoke :read [26 nil] INFO [2022-01-10 05:46:23,010] jepsen worker 28 - jepsen.util 3538 :info :write [9 21] :RequestTimeoutException INFO [2022-01-10 05:46:23,011] jepsen worker 28 - jepsen.util 3628 :invoke :write [9 7] INFO [2022-01-10 05:46:23,038] jepsen worker 78 - jepsen.util 4668 :ok :read [26 10] INFO [2022-01-10 05:46:23,039] jepsen worker 78 - jepsen.util 4668 :invoke :read [26 nil] INFO [2022-01-10 05:46:23,070] jepsen worker 78 - jepsen.util 4668 :ok :read [26 10] INFO [2022-01-10 05:46:23,071] jepsen worker 78 - jepsen.util 4668 :invoke :write [26 8] INFO [2022-01-10 05:46:23,223] jepsen worker 17 - jepsen.util 3707 :ok :read [5 25] INFO [2022-01-10 05:46:23,223] jepsen worker 17 - jepsen.util 3707 :invoke :write [5 40] INFO [2022-01-10 05:46:23,357] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:23,358] jepsen worker nemesis - couchbase.util Rebalance check count: 532 INFO [2022-01-10 05:46:23,358] jepsen worker nemesis - couchbase.util Rebalance stuck count: 445 INFO [2022-01-10 05:46:23,417] jepsen worker 43 - jepsen.util 3733 :ok :read [14 14] INFO [2022-01-10 05:46:23,417] jepsen worker 43 - jepsen.util 3733 :invoke :read [14 nil] INFO [2022-01-10 05:46:23,609] jepsen worker 48 - jepsen.util 3558 :ok :read [16 38] INFO [2022-01-10 05:46:23,610] jepsen worker 48 - jepsen.util 3558 :invoke :read [16 nil] INFO [2022-01-10 05:46:23,619] jepsen worker 5 - jepsen.util 3605 :ok :read [1 26] INFO [2022-01-10 05:46:23,620] jepsen worker 5 - jepsen.util 3605 :invoke :write [1 27] INFO [2022-01-10 05:46:23,672] jepsen worker 31 - jepsen.util 4171 :ok :read [10 36] INFO [2022-01-10 05:46:23,673] jepsen worker 31 - jepsen.util 4171 :invoke :write [10 42] INFO [2022-01-10 05:46:23,674] jepsen worker 70 - jepsen.util 3760 :ok :read [23 26] INFO [2022-01-10 05:46:23,676] jepsen worker 70 - jepsen.util 3760 :invoke :write [23 29] INFO [2022-01-10 05:46:23,731] jepsen worker 87 - jepsen.util 3687 :ok :read [29 0] INFO [2022-01-10 05:46:23,731] jepsen worker 87 - jepsen.util 3687 :invoke :read [29 nil] INFO [2022-01-10 05:46:23,780] jepsen worker 88 - jepsen.util 3688 :ok :read [29 0] INFO [2022-01-10 05:46:23,780] jepsen worker 88 - jepsen.util 3688 :invoke :read [29 nil] INFO [2022-01-10 05:46:23,791] jepsen worker 11 - jepsen.util 3701 :ok :read [3 31] INFO [2022-01-10 05:46:23,792] jepsen worker 11 - jepsen.util 3701 :invoke :write [3 12] INFO [2022-01-10 05:46:23,797] jepsen worker 87 - jepsen.util 3687 :ok :read [29 0] INFO [2022-01-10 05:46:23,798] jepsen worker 87 - jepsen.util 3687 :invoke :write [29 23] INFO [2022-01-10 05:46:23,806] jepsen worker 88 - jepsen.util 3688 :ok :read [29 0] INFO [2022-01-10 05:46:23,807] jepsen worker 88 - jepsen.util 3688 :invoke :read [29 nil] INFO [2022-01-10 05:46:23,842] jepsen worker 88 - jepsen.util 3688 :ok :read [29 0] INFO [2022-01-10 05:46:23,842] jepsen worker 88 - jepsen.util 3688 :invoke :read [29 nil] INFO [2022-01-10 05:46:23,846] jepsen worker 2 - jepsen.util 4052 :info :write [0 15] :RequestTimeoutException INFO [2022-01-10 05:46:23,847] jepsen worker 2 - jepsen.util 4142 :invoke :read [0 nil] INFO [2022-01-10 05:46:23,917] jepsen worker 4 - jepsen.util 4414 :ok :read [1 26] INFO [2022-01-10 05:46:23,918] jepsen worker 4 - jepsen.util 4414 :invoke :write [1 37] INFO [2022-01-10 05:46:23,931] jepsen worker 88 - jepsen.util 3688 :ok :read [29 0] INFO [2022-01-10 05:46:23,932] jepsen worker 88 - jepsen.util 3688 :invoke :write [29 5] INFO [2022-01-10 05:46:24,276] jepsen worker 72 - jepsen.util 4212 :info :write [24 17] :RequestTimeoutException INFO [2022-01-10 05:46:24,277] jepsen worker 72 - jepsen.util 4302 :invoke :read [24 nil] INFO [2022-01-10 05:46:24,281] jepsen worker 46 - jepsen.util 3916 :info :write [15 4] :RequestTimeoutException INFO [2022-01-10 05:46:24,281] jepsen worker 46 - jepsen.util 4006 :invoke :read [15 nil] INFO [2022-01-10 05:46:24,364] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:24,364] jepsen worker nemesis - couchbase.util Rebalance check count: 533 INFO [2022-01-10 05:46:24,372] jepsen worker 81 - jepsen.util 4131 :info :write [27 30] :RequestTimeoutException INFO [2022-01-10 05:46:24,388] jepsen worker nemesis - couchbase.util Rebalance stuck count: 446 INFO [2022-01-10 05:46:24,388] jepsen worker 81 - jepsen.util 4221 :invoke :write [27 0] INFO [2022-01-10 05:46:24,447] jepsen worker 43 - jepsen.util 3733 :ok :read [14 14] INFO [2022-01-10 05:46:24,448] jepsen worker 43 - jepsen.util 3733 :invoke :write [14 3] INFO [2022-01-10 05:46:24,489] jepsen worker 53 - jepsen.util 4643 :info :write [17 38] :RequestTimeoutException INFO [2022-01-10 05:46:24,490] jepsen worker 53 - jepsen.util 4733 :invoke :read [17 nil] INFO [2022-01-10 05:46:24,942] jepsen worker 51 - jepsen.util 4551 :info :write [17 7] :RequestTimeoutException INFO [2022-01-10 05:46:24,943] jepsen worker 51 - jepsen.util 4641 :invoke :write [17 0] INFO [2022-01-10 05:46:24,976] jepsen worker 69 - jepsen.util 3309 :info :write [23 42] :RequestTimeoutException INFO [2022-01-10 05:46:24,977] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 38] INFO [2022-01-10 05:46:25,051] jepsen worker 32 - jepsen.util 3992 :info :write [10 20] :RequestTimeoutException INFO [2022-01-10 05:46:25,052] jepsen worker 32 - jepsen.util 4082 :invoke :write [10 7] INFO [2022-01-10 05:46:25,189] jepsen worker 37 - jepsen.util 3547 :info :write [12 11] :RequestTimeoutException INFO [2022-01-10 05:46:25,190] jepsen worker 37 - jepsen.util 3637 :invoke :write [12 47] INFO [2022-01-10 05:46:25,236] jepsen worker 61 - jepsen.util 3931 :info :write [20 0] :RequestTimeoutException INFO [2022-01-10 05:46:25,238] jepsen worker 61 - jepsen.util 4021 :invoke :write [20 38] INFO [2022-01-10 05:46:25,345] jepsen worker 20 - jepsen.util 3530 :info :write [6 14] :RequestTimeoutException INFO [2022-01-10 05:46:25,345] jepsen worker 20 - jepsen.util 3620 :invoke :write [6 14] INFO [2022-01-10 05:46:25,393] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:25,394] jepsen worker nemesis - couchbase.util Rebalance check count: 534 INFO [2022-01-10 05:46:25,394] jepsen worker nemesis - couchbase.util Rebalance stuck count: 447 INFO [2022-01-10 05:46:25,405] jepsen worker 29 - jepsen.util 3719 :info :write [9 35] :RequestTimeoutException INFO [2022-01-10 05:46:25,406] jepsen worker 29 - jepsen.util 3809 :invoke :write [9 21] INFO [2022-01-10 05:46:25,450] jepsen worker 21 - jepsen.util 3801 :info :write [7 17] :RequestTimeoutException INFO [2022-01-10 05:46:25,451] jepsen worker 21 - jepsen.util 3891 :invoke :write [7 28] INFO [2022-01-10 05:46:25,545] jepsen worker 25 - jepsen.util 3625 :info :write [8 41] :RequestTimeoutException INFO [2022-01-10 05:46:25,546] jepsen worker 25 - jepsen.util 3715 :invoke :write [8 11] INFO [2022-01-10 05:46:25,595] jepsen worker 24 - jepsen.util 3444 :info :write [8 44] :RequestTimeoutException INFO [2022-01-10 05:46:25,596] jepsen worker 24 - jepsen.util 3534 :invoke :read [8 nil] INFO [2022-01-10 05:46:25,631] jepsen worker 58 - jepsen.util 3568 :ok :read [19 31] INFO [2022-01-10 05:46:25,632] jepsen worker 58 - jepsen.util 3568 :invoke :read [19 nil] INFO [2022-01-10 05:46:25,859] jepsen worker 49 - jepsen.util 3739 :info :write [16 17] :RequestTimeoutException INFO [2022-01-10 05:46:25,860] jepsen worker 49 - jepsen.util 3829 :invoke :read [16 nil] INFO [2022-01-10 05:46:25,872] jepsen worker 40 - jepsen.util 3280 :info :write [13 25] :RequestTimeoutException INFO [2022-01-10 05:46:25,872] jepsen worker 40 - jepsen.util 3370 :invoke :write [13 21] INFO [2022-01-10 05:46:26,001] jepsen worker 36 - jepsen.util 2916 :info :write [12 26] :RequestTimeoutException INFO [2022-01-10 05:46:26,002] jepsen worker 36 - jepsen.util 3006 :invoke :write [12 14] INFO [2022-01-10 05:46:26,084] jepsen worker 59 - jepsen.util 3839 :info :write [19 47] :RequestTimeoutException INFO [2022-01-10 05:46:26,085] jepsen worker 59 - jepsen.util 3929 :invoke :read [19 nil] INFO [2022-01-10 05:46:26,116] jepsen worker 60 - jepsen.util 4020 :info :write [20 44] :RequestTimeoutException INFO [2022-01-10 05:46:26,117] jepsen worker 60 - jepsen.util 4110 :invoke :read [20 nil] INFO [2022-01-10 05:46:26,191] jepsen worker 63 - jepsen.util 3753 :ok :read [21 7] INFO [2022-01-10 05:46:26,196] jepsen worker 63 - jepsen.util 3753 :invoke :read [21 nil] INFO [2022-01-10 05:46:26,270] jepsen worker 41 - jepsen.util 3641 :info :write [13 12] :RequestTimeoutException INFO [2022-01-10 05:46:26,270] jepsen worker 41 - jepsen.util 3731 :invoke :write [13 33] INFO [2022-01-10 05:46:26,402] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:26,402] jepsen worker nemesis - couchbase.util Rebalance check count: 535 INFO [2022-01-10 05:46:26,402] jepsen worker nemesis - couchbase.util Rebalance stuck count: 448 INFO [2022-01-10 05:46:26,441] jepsen worker 63 - jepsen.util 3753 :ok :read [21 7] INFO [2022-01-10 05:46:26,442] jepsen worker 63 - jepsen.util 3753 :invoke :read [21 nil] INFO [2022-01-10 05:46:26,471] jepsen worker 35 - jepsen.util 3545 :info :write [11 22] :RequestTimeoutException INFO [2022-01-10 05:46:26,472] jepsen worker 35 - jepsen.util 3635 :invoke :read [11 nil] INFO [2022-01-10 05:46:26,676] jepsen worker 33 - jepsen.util 3993 :info :write [11 25] :RequestTimeoutException INFO [2022-01-10 05:46:26,677] jepsen worker 33 - jepsen.util 4083 :invoke :read [11 nil] INFO [2022-01-10 05:46:26,815] jepsen worker 80 - jepsen.util 4670 :info :write [26 39] :RequestTimeoutException INFO [2022-01-10 05:46:26,816] jepsen worker 80 - jepsen.util 4760 :invoke :read [26 nil] INFO [2022-01-10 05:46:26,819] jepsen worker 60 - jepsen.util 4110 :ok :read [20 :nil] INFO [2022-01-10 05:46:26,820] jepsen worker 60 - jepsen.util 4110 :invoke :read [20 nil] INFO [2022-01-10 05:46:26,829] jepsen worker 18 - jepsen.util 3708 :info :write [6 45] :RequestTimeoutException INFO [2022-01-10 05:46:26,830] jepsen worker 18 - jepsen.util 3798 :invoke :write [6 47] INFO [2022-01-10 05:46:26,836] jepsen worker 73 - jepsen.util 3403 :info :write [24 27] :RequestTimeoutException INFO [2022-01-10 05:46:26,837] jepsen worker 73 - jepsen.util 3493 :invoke :read [24 nil] INFO [2022-01-10 05:46:26,868] jepsen worker 30 - jepsen.util 3450 :info :write [10 5] :SyncWriteAmbiguous INFO [2022-01-10 05:46:26,869] jepsen worker 30 - jepsen.util 3540 :invoke :write [10 32] INFO [2022-01-10 05:46:26,894] jepsen worker 63 - jepsen.util 3753 :ok :read [21 7] INFO [2022-01-10 05:46:26,894] jepsen worker 63 - jepsen.util 3753 :invoke :write [21 27] INFO [2022-01-10 05:46:27,067] jepsen worker 19 - jepsen.util 3979 :info :write [6 9] :RequestTimeoutException INFO [2022-01-10 05:46:27,067] jepsen worker 19 - jepsen.util 4069 :invoke :read [6 nil] INFO [2022-01-10 05:46:27,192] jepsen worker 26 - jepsen.util 3536 :info :write [8 3] :RequestTimeoutException INFO [2022-01-10 05:46:27,193] jepsen worker 26 - jepsen.util 3626 :invoke :read [8 nil] INFO [2022-01-10 05:46:27,225] jepsen worker 56 - jepsen.util 4106 :info :write [18 26] :RequestTimeoutException INFO [2022-01-10 05:46:27,225] jepsen worker 56 - jepsen.util 4196 :invoke :read [18 nil] INFO [2022-01-10 05:46:27,226] jepsen worker 67 - jepsen.util 3667 :ok :read [22 41] INFO [2022-01-10 05:46:27,226] jepsen worker 67 - jepsen.util 3667 :invoke :write [22 47] INFO [2022-01-10 05:46:27,236] jepsen worker 57 - jepsen.util 4557 :info :write [19 23] :RequestTimeoutException INFO [2022-01-10 05:46:27,236] jepsen worker 57 - jepsen.util 4647 :invoke :write [19 26] INFO [2022-01-10 05:46:27,295] jepsen worker 69 - jepsen.util 3399 :fail :write [23 38] :DurabilityImpossible INFO [2022-01-10 05:46:27,295] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 45] INFO [2022-01-10 05:46:27,324] jepsen worker 9 - jepsen.util 4239 :info :write [3 12] :RequestTimeoutException INFO [2022-01-10 05:46:27,325] jepsen worker 9 - jepsen.util 4329 :invoke :read [3 nil] INFO [2022-01-10 05:46:27,335] jepsen worker 38 - jepsen.util 3728 :info :write [12 13] :RequestTimeoutException INFO [2022-01-10 05:46:27,336] jepsen worker 38 - jepsen.util 3818 :invoke :read [12 nil] INFO [2022-01-10 05:46:27,349] jepsen worker 9 - jepsen.util 4329 :ok :read [3 31] INFO [2022-01-10 05:46:27,349] jepsen worker 9 - jepsen.util 4329 :invoke :read [3 nil] INFO [2022-01-10 05:46:27,354] jepsen worker 54 - jepsen.util 3834 :info :write [18 26] :RequestTimeoutException INFO [2022-01-10 05:46:27,355] jepsen worker 54 - jepsen.util 3924 :invoke :read [18 nil] INFO [2022-01-10 05:46:27,361] jepsen worker 79 - jepsen.util 4489 :info :write [26 36] :RequestTimeoutException INFO [2022-01-10 05:46:27,362] jepsen worker 79 - jepsen.util 4579 :invoke :read [26 nil] INFO [2022-01-10 05:46:27,365] jepsen worker 10 - jepsen.util 3700 :info :write [3 30] :RequestTimeoutException INFO [2022-01-10 05:46:27,365] jepsen worker 10 - jepsen.util 3790 :invoke :read [3 nil] INFO [2022-01-10 05:46:27,372] jepsen worker 9 - jepsen.util 4329 :ok :read [3 31] INFO [2022-01-10 05:46:27,372] jepsen worker 9 - jepsen.util 4329 :invoke :read [3 nil] INFO [2022-01-10 05:46:27,396] jepsen worker 9 - jepsen.util 4329 :ok :read [3 31] INFO [2022-01-10 05:46:27,397] jepsen worker 9 - jepsen.util 4329 :invoke :read [3 nil] INFO [2022-01-10 05:46:27,408] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:27,408] jepsen worker nemesis - couchbase.util Rebalance check count: 536 INFO [2022-01-10 05:46:27,408] jepsen worker nemesis - couchbase.util Rebalance stuck count: 449 INFO [2022-01-10 05:46:27,414] jepsen worker 69 - jepsen.util 3399 :fail :write [23 45] :DurabilityImpossible INFO [2022-01-10 05:46:27,415] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 48] INFO [2022-01-10 05:46:27,415] jepsen worker 3 - jepsen.util 3603 :info :write [1 30] :RequestTimeoutException INFO [2022-01-10 05:46:27,415] jepsen worker 3 - jepsen.util 3693 :invoke :write [1 10] INFO [2022-01-10 05:46:27,422] jepsen worker 9 - jepsen.util 4329 :ok :read [3 31] INFO [2022-01-10 05:46:27,423] jepsen worker 9 - jepsen.util 4329 :invoke :write [3 23] INFO [2022-01-10 05:46:27,465] jepsen worker 86 - jepsen.util 3236 :info :write [28 31] :RequestTimeoutException INFO [2022-01-10 05:46:27,465] jepsen worker 86 - jepsen.util 3326 :invoke :write [28 23] INFO [2022-01-10 05:46:27,521] jepsen worker 48 - jepsen.util 3558 :ok :read [16 38] INFO [2022-01-10 05:46:27,523] jepsen worker 48 - jepsen.util 3558 :invoke :write [16 33] INFO [2022-01-10 05:46:27,554] jepsen worker 2 - jepsen.util 4142 :ok :read [0 46] INFO [2022-01-10 05:46:27,555] jepsen worker 2 - jepsen.util 4142 :invoke :read [0 nil] INFO [2022-01-10 05:46:27,558] jepsen worker 69 - jepsen.util 3399 :fail :write [23 48] :DurabilityImpossible INFO [2022-01-10 05:46:27,558] jepsen worker 69 - jepsen.util 3399 :invoke :read [23 nil] INFO [2022-01-10 05:46:27,566] jepsen worker 84 - jepsen.util 3774 :ok :read [28 45] INFO [2022-01-10 05:46:27,566] jepsen worker 84 - jepsen.util 3774 :invoke :write [28 34] INFO [2022-01-10 05:46:27,567] jepsen worker 42 - jepsen.util 3552 :info :write [14 44] :RequestTimeoutException INFO [2022-01-10 05:46:27,567] jepsen worker 42 - jepsen.util 3642 :invoke :read [14 nil] INFO [2022-01-10 05:46:27,625] jepsen worker 74 - jepsen.util 4304 :info :write [24 13] :RequestTimeoutException INFO [2022-01-10 05:46:27,625] jepsen worker 74 - jepsen.util 4394 :invoke :read [24 nil] INFO [2022-01-10 05:46:27,628] jepsen worker 35 - jepsen.util 3635 :ok :read [11 21] INFO [2022-01-10 05:46:27,629] jepsen worker 35 - jepsen.util 3635 :invoke :write [11 27] INFO [2022-01-10 05:46:27,638] jepsen worker 72 - jepsen.util 4302 :ok :read [24 18] INFO [2022-01-10 05:46:27,638] jepsen worker 72 - jepsen.util 4302 :invoke :read [24 nil] INFO [2022-01-10 05:46:27,716] jepsen worker 23 - jepsen.util 3893 :info :write [7 16] :RequestTimeoutException INFO [2022-01-10 05:46:27,717] jepsen worker 23 - jepsen.util 3983 :invoke :write [7 23] INFO [2022-01-10 05:46:27,865] jepsen worker 58 - jepsen.util 3568 :ok :read [19 31] INFO [2022-01-10 05:46:27,866] jepsen worker 58 - jepsen.util 3568 :invoke :write [19 38] INFO [2022-01-10 05:46:27,874] jepsen worker 55 - jepsen.util 4195 :info :write [18 39] :RequestTimeoutException INFO [2022-01-10 05:46:27,875] jepsen worker 55 - jepsen.util 4285 :invoke :read [18 nil] INFO [2022-01-10 05:46:27,902] jepsen worker 60 - jepsen.util 4110 :ok :read [20 :nil] INFO [2022-01-10 05:46:27,903] jepsen worker 60 - jepsen.util 4110 :invoke :read [20 nil] INFO [2022-01-10 05:46:27,918] jepsen worker 19 - jepsen.util 4069 :ok :read [6 25] INFO [2022-01-10 05:46:27,918] jepsen worker 73 - jepsen.util 3493 :ok :read [24 18] INFO [2022-01-10 05:46:27,919] jepsen worker 19 - jepsen.util 4069 :invoke :write [6 1] INFO [2022-01-10 05:46:27,919] jepsen worker 73 - jepsen.util 3493 :invoke :write [24 25] INFO [2022-01-10 05:46:27,956] jepsen worker 62 - jepsen.util 4202 :info :write [20 29] :RequestTimeoutException INFO [2022-01-10 05:46:27,961] jepsen worker 62 - jepsen.util 4292 :invoke :write [20 28] INFO [2022-01-10 05:46:27,966] jepsen worker 77 - jepsen.util 4217 :info :write [25 38] :RequestTimeoutException INFO [2022-01-10 05:46:27,967] jepsen worker 77 - jepsen.util 4307 :invoke :write [25 42] INFO [2022-01-10 05:46:28,167] jepsen worker 74 - jepsen.util 4394 :ok :read [24 18] INFO [2022-01-10 05:46:28,168] jepsen worker 74 - jepsen.util 4394 :invoke :write [24 7] INFO [2022-01-10 05:46:28,327] jepsen worker 59 - jepsen.util 3929 :ok :read [19 31] INFO [2022-01-10 05:46:28,328] jepsen worker 59 - jepsen.util 3929 :invoke :write [19 28] INFO [2022-01-10 05:46:28,333] jepsen worker 49 - jepsen.util 3829 :ok :read [16 38] INFO [2022-01-10 05:46:28,333] jepsen worker 49 - jepsen.util 3829 :invoke :read [16 nil] INFO [2022-01-10 05:46:28,362] jepsen worker 1 - jepsen.util 3871 :info :write [0 1] :RequestTimeoutException INFO [2022-01-10 05:46:28,362] jepsen worker 1 - jepsen.util 3961 :invoke :read [0 nil] INFO [2022-01-10 05:46:28,374] jepsen worker 72 - jepsen.util 4302 :ok :read [24 18] INFO [2022-01-10 05:46:28,374] jepsen worker 72 - jepsen.util 4302 :invoke :read [24 nil] INFO [2022-01-10 05:46:28,396] jepsen worker 50 - jepsen.util 4010 :info :write [16 49] :RequestTimeoutException INFO [2022-01-10 05:46:28,397] jepsen worker 50 - jepsen.util 4100 :invoke :read [16 nil] INFO [2022-01-10 05:46:28,413] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:28,413] jepsen worker nemesis - couchbase.util Rebalance check count: 537 INFO [2022-01-10 05:46:28,413] jepsen worker nemesis - couchbase.util Rebalance stuck count: 450 INFO [2022-01-10 05:46:28,526] jepsen worker 8 - jepsen.util 4238 :info :write [2 7] :RequestTimeoutException INFO [2022-01-10 05:46:28,527] jepsen worker 8 - jepsen.util 4328 :invoke :read [2 nil] INFO [2022-01-10 05:46:28,629] jepsen worker 2 - jepsen.util 4142 :ok :read [0 46] INFO [2022-01-10 05:46:28,630] jepsen worker 2 - jepsen.util 4142 :invoke :read [0 nil] INFO [2022-01-10 05:46:28,845] jepsen worker 52 - jepsen.util 4822 :info :write [17 24] :RequestTimeoutException INFO [2022-01-10 05:46:28,846] jepsen worker 52 - jepsen.util 4912 :invoke :read [17 nil] INFO [2022-01-10 05:46:28,956] jepsen worker 60 - jepsen.util 4110 :ok :read [20 :nil] INFO [2022-01-10 05:46:28,957] jepsen worker 60 - jepsen.util 4110 :invoke :write [20 21] INFO [2022-01-10 05:46:28,973] jepsen worker 72 - jepsen.util 4302 :ok :read [24 18] INFO [2022-01-10 05:46:28,974] jepsen worker 72 - jepsen.util 4302 :invoke :write [24 10] INFO [2022-01-10 05:46:29,424] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4853801169590644}, :ns_1@172.23.107.227 {:progress 0.3401759530791789}, :ns_1@172.23.105.187 {:progress 0.3401759530791789}, :ns_1@172.23.107.228 {:progress 0.4588235294117647}} INFO [2022-01-10 05:46:29,424] jepsen worker nemesis - couchbase.util Rebalance check count: 538 INFO [2022-01-10 05:46:29,424] jepsen worker nemesis - couchbase.util Rebalance stuck count: 451 INFO [2022-01-10 05:46:29,432] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:29,433] jepsen worker 71 - jepsen.util 3401 :invoke :write [23 31] INFO [2022-01-10 05:46:29,441] jepsen worker 16 - jepsen.util 3886 :info :write [5 44] :RequestTimeoutException INFO [2022-01-10 05:46:29,442] jepsen worker 16 - jepsen.util 3976 :invoke :write [5 40] INFO [2022-01-10 05:46:29,445] jepsen worker 69 - jepsen.util 3399 :ok :read [23 26] INFO [2022-01-10 05:46:29,445] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 39] INFO [2022-01-10 05:46:29,511] jepsen worker 13 - jepsen.util 4423 :info :write [4 29] :SyncWriteAmbiguous INFO [2022-01-10 05:46:29,512] jepsen worker 13 - jepsen.util 4513 :invoke :read [4 nil] INFO [2022-01-10 05:46:29,542] jepsen worker 13 - jepsen.util 4513 :ok :read [4 3] INFO [2022-01-10 05:46:29,543] jepsen worker 13 - jepsen.util 4513 :invoke :read [4 nil] INFO [2022-01-10 05:46:29,547] jepsen worker 71 - jepsen.util 3401 :fail :write [23 31] :DurabilityImpossible INFO [2022-01-10 05:46:29,548] jepsen worker 71 - jepsen.util 3401 :invoke :write [23 6] INFO [2022-01-10 05:46:29,572] jepsen worker 13 - jepsen.util 4513 :ok :read [4 3] INFO [2022-01-10 05:46:29,572] jepsen worker 13 - jepsen.util 4513 :invoke :write [4 41] INFO [2022-01-10 05:46:29,587] jepsen worker 69 - jepsen.util 3399 :fail :write [23 39] :DurabilityImpossible INFO [2022-01-10 05:46:29,588] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 15] INFO [2022-01-10 05:46:29,640] jepsen worker 49 - jepsen.util 3829 :ok :read [16 38] INFO [2022-01-10 05:46:29,640] jepsen worker 49 - jepsen.util 3829 :invoke :read [16 nil] INFO [2022-01-10 05:46:29,656] jepsen worker 71 - jepsen.util 3401 :fail :write [23 6] :DurabilityImpossible INFO [2022-01-10 05:46:29,657] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:29,689] jepsen worker 69 - jepsen.util 3399 :fail :write [23 15] :DurabilityImpossible INFO [2022-01-10 05:46:29,689] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 28] INFO [2022-01-10 05:46:29,750] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:29,751] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:29,755] jepsen worker 50 - jepsen.util 4100 :ok :read [16 38] INFO [2022-01-10 05:46:29,758] jepsen worker 50 - jepsen.util 4100 :invoke :read [16 nil] INFO [2022-01-10 05:46:29,800] jepsen worker 85 - jepsen.util 4135 :info :write [28 36] :RequestTimeoutException INFO [2022-01-10 05:46:29,800] jepsen worker 85 - jepsen.util 4225 :invoke :write [28 30] INFO [2022-01-10 05:46:29,806] jepsen worker 65 - jepsen.util 3575 :info :write [21 9] :RequestTimeoutException INFO [2022-01-10 05:46:29,807] jepsen worker 65 - jepsen.util 3665 :invoke :write [21 41] INFO [2022-01-10 05:46:29,858] jepsen worker 24 - jepsen.util 3534 :ok :read [8 39] INFO [2022-01-10 05:46:29,859] jepsen worker 24 - jepsen.util 3534 :invoke :read [8 nil] INFO [2022-01-10 05:46:29,871] jepsen worker 69 - jepsen.util 3399 :fail :write [23 28] :DurabilityImpossible INFO [2022-01-10 05:46:29,872] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 21] INFO [2022-01-10 05:46:29,901] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:29,902] jepsen worker 71 - jepsen.util 3401 :invoke :write [23 16] INFO [2022-01-10 05:46:29,936] jepsen worker 89 - jepsen.util 4319 :info :write [29 29] :RequestTimeoutException INFO [2022-01-10 05:46:29,937] jepsen worker 89 - jepsen.util 4409 :invoke :read [29 nil] INFO [2022-01-10 05:46:29,986] jepsen worker 89 - jepsen.util 4409 :ok :read [29 0] INFO [2022-01-10 05:46:29,987] jepsen worker 89 - jepsen.util 4409 :invoke :read [29 nil] INFO [2022-01-10 05:46:29,996] jepsen worker 42 - jepsen.util 3642 :ok :read [14 14] INFO [2022-01-10 05:46:29,997] jepsen worker 42 - jepsen.util 3642 :invoke :read [14 nil] INFO [2022-01-10 05:46:30,005] jepsen worker 69 - jepsen.util 3399 :fail :write [23 21] :DurabilityImpossible INFO [2022-01-10 05:46:30,006] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 3] INFO [2022-01-10 05:46:30,011] jepsen worker 80 - jepsen.util 4760 :ok :read [26 10] INFO [2022-01-10 05:46:30,012] jepsen worker 80 - jepsen.util 4760 :invoke :read [26 nil] INFO [2022-01-10 05:46:30,018] jepsen worker 89 - jepsen.util 4409 :ok :read [29 0] INFO [2022-01-10 05:46:30,019] jepsen worker 89 - jepsen.util 4409 :invoke :read [29 nil] INFO [2022-01-10 05:46:30,046] jepsen worker 89 - jepsen.util 4409 :ok :read [29 0] INFO [2022-01-10 05:46:30,046] jepsen worker 89 - jepsen.util 4409 :invoke :write [29 20] INFO [2022-01-10 05:46:30,055] jepsen worker 27 - jepsen.util 3987 :info :write [9 3] :RequestTimeoutException INFO [2022-01-10 05:46:30,056] jepsen worker 27 - jepsen.util 4077 :invoke :write [9 33] INFO [2022-01-10 05:46:30,085] jepsen worker 52 - jepsen.util 4912 :ok :read [17 41] INFO [2022-01-10 05:46:30,085] jepsen worker 52 - jepsen.util 4912 :invoke :read [17 nil] INFO [2022-01-10 05:46:30,100] jepsen worker 2 - jepsen.util 4142 :ok :read [0 46] INFO [2022-01-10 05:46:30,100] jepsen worker 2 - jepsen.util 4142 :invoke :read [0 nil] INFO [2022-01-10 05:46:30,166] jepsen worker 71 - jepsen.util 3401 :fail :write [23 16] :DurabilityImpossible INFO [2022-01-10 05:46:30,166] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:30,183] jepsen worker 24 - jepsen.util 3534 :ok :read [8 39] INFO [2022-01-10 05:46:30,184] jepsen worker 24 - jepsen.util 3534 :invoke :write [8 13] INFO [2022-01-10 05:46:30,191] jepsen worker 42 - jepsen.util 3642 :ok :read [14 14] INFO [2022-01-10 05:46:30,192] jepsen worker 42 - jepsen.util 3642 :invoke :read [14 nil] INFO [2022-01-10 05:46:30,204] jepsen worker 46 - jepsen.util 4006 :ok :read [15 23] INFO [2022-01-10 05:46:30,204] jepsen worker 46 - jepsen.util 4006 :invoke :write [15 37] INFO [2022-01-10 05:46:30,210] jepsen worker 45 - jepsen.util 4095 :info :write [15 32] :RequestTimeoutException INFO [2022-01-10 05:46:30,211] jepsen worker 45 - jepsen.util 4185 :invoke :read [15 nil] INFO [2022-01-10 05:46:30,277] jepsen worker 69 - jepsen.util 3399 :fail :write [23 3] :DurabilityImpossible INFO [2022-01-10 05:46:30,279] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 34] INFO [2022-01-10 05:46:30,299] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:30,300] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:30,321] jepsen worker 22 - jepsen.util 3532 :info :write [7 4] :RequestTimeoutException INFO [2022-01-10 05:46:30,322] jepsen worker 22 - jepsen.util 3622 :invoke :write [7 22] INFO [2022-01-10 05:46:30,351] jepsen worker 6 - jepsen.util 3876 :info :write [2 26] :RequestTimeoutException INFO [2022-01-10 05:46:30,352] jepsen worker 6 - jepsen.util 3966 :invoke :write [2 39] INFO [2022-01-10 05:46:30,404] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:30,405] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:30,407] jepsen worker 80 - jepsen.util 4760 :ok :read [26 10] INFO [2022-01-10 05:46:30,408] jepsen worker 80 - jepsen.util 4760 :invoke :read [26 nil] INFO [2022-01-10 05:46:30,430] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4970760233918129}, :ns_1@172.23.107.227 {:progress 0.343108504398827}, :ns_1@172.23.105.187 {:progress 0.343108504398827}, :ns_1@172.23.107.228 {:progress 0.4705882352941176}} INFO [2022-01-10 05:46:30,430] jepsen worker nemesis - couchbase.util Rebalance check count: 539 INFO [2022-01-10 05:46:30,430] jepsen worker nemesis - couchbase.util Rebalance stuck count: 451 INFO [2022-01-10 05:46:30,465] jepsen worker 7 - jepsen.util 3967 :info :write [2 31] :RequestTimeoutException INFO [2022-01-10 05:46:30,466] jepsen worker 7 - jepsen.util 4057 :invoke :write [2 16] WARN [2022-01-10 05:46:30,511] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9643352,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13451166,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10304288,"encode_duration_us":17037,"operation_id":"0x76f9","timeout_ms":10000,"last_server_duration_us":9232481,"total_duration_us":10008641}],"total_count":1}} WARN [2022-01-10 05:46:30,512] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":718130,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":4409167,"total_server_duration_us":926155,"encode_duration_us":16935,"operation_id":"0x76fb","timeout_ms":10000,"last_server_duration_us":58252,"total_duration_us":10010648},{"operation_name":"upsert","last_dispatch_duration_us":870380,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9814551,"total_server_duration_us":1339480,"encode_duration_us":17562,"operation_id":"0x76f7","timeout_ms":10000,"last_server_duration_us":148646,"total_duration_us":10010249},{"operation_name":"upsert","last_dispatch_duration_us":9643352,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":13451166,"total_server_duration_us":10304288,"encode_duration_us":17037,"operation_id":"0x76f9","timeout_ms":10000,"last_server_duration_us":9232481,"total_duration_us":10008641},{"operation_name":"upsert","last_dispatch_duration_us":7683318,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8923412,"total_server_duration_us":182589,"encode_duration_us":16896,"operation_id":"0x776a","timeout_ms":10000,"last_server_duration_us":21001,"total_duration_us":10007565},{"operation_name":"upsert","last_dispatch_duration_us":7624737,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9028448,"total_server_duration_us":239821,"encode_duration_us":21324,"operation_id":"0x7765","timeout_ms":10000,"last_server_duration_us":22141,"total_duration_us":10007151},{"operation_name":"upsert","last_dispatch_duration_us":1062500,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10005596,"total_server_duration_us":701608,"encode_duration_us":20314,"operation_id":"0x7688","timeout_ms":10000,"last_server_duration_us":178488,"total_duration_us":10007126},{"operation_name":"upsert","last_dispatch_duration_us":9389616,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11989618,"total_server_duration_us":700647,"encode_duration_us":19551,"operation_id":"0x7775","timeout_ms":10000,"last_server_duration_us":91081,"total_duration_us":10006846},{"operation_name":"upsert","last_dispatch_duration_us":979106,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":5930776,"total_server_duration_us":1338649,"encode_duration_us":17434,"operation_id":"0x77fe","timeout_ms":10000,"last_server_duration_us":440772,"total_duration_us":10005712},{"operation_name":"upsert","last_dispatch_duration_us":354634,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":5271344,"total_server_duration_us":1191103,"encode_duration_us":22230,"operation_id":"0x776e","timeout_ms":10000,"last_server_duration_us":175300,"total_duration_us":10005662},{"operation_name":"upsert","last_dispatch_duration_us":53092,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":1056689,"total_server_duration_us":173878,"encode_duration_us":16983,"operation_id":"0x7797","timeout_ms":10000,"last_server_duration_us":22223,"total_duration_us":10005639}],"total_count":21}} INFO [2022-01-10 05:46:30,535] jepsen worker 52 - jepsen.util 4912 :ok :read [17 41] INFO [2022-01-10 05:46:30,536] jepsen worker 52 - jepsen.util 4912 :invoke :read [17 nil] INFO [2022-01-10 05:46:30,640] jepsen worker 69 - jepsen.util 3399 :fail :write [23 34] :DurabilityImpossible INFO [2022-01-10 05:46:30,641] jepsen worker 69 - jepsen.util 3399 :invoke :read [23 nil] INFO [2022-01-10 05:46:30,677] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:30,677] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:30,683] jepsen worker 69 - jepsen.util 3399 :ok :read [23 26] INFO [2022-01-10 05:46:30,683] jepsen worker 69 - jepsen.util 3399 :invoke :read [23 nil] INFO [2022-01-10 05:46:30,976] jepsen worker 64 - jepsen.util 3844 :info :write [21 39] :RequestTimeoutException INFO [2022-01-10 05:46:30,976] jepsen worker 64 - jepsen.util 3934 :invoke :write [21 1] INFO [2022-01-10 05:46:31,001] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:31,002] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:31,022] jepsen worker 69 - jepsen.util 3399 :ok :read [23 26] INFO [2022-01-10 05:46:31,023] jepsen worker 69 - jepsen.util 3399 :invoke :read [23 nil] INFO [2022-01-10 05:46:31,031] jepsen worker 80 - jepsen.util 4760 :ok :read [26 10] INFO [2022-01-10 05:46:31,032] jepsen worker 80 - jepsen.util 4760 :invoke :write [26 1] INFO [2022-01-10 05:46:31,038] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:31,038] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:31,063] jepsen worker 69 - jepsen.util 3399 :ok :read [23 26] INFO [2022-01-10 05:46:31,064] jepsen worker 69 - jepsen.util 3399 :invoke :read [23 nil] INFO [2022-01-10 05:46:31,072] jepsen worker 49 - jepsen.util 3829 :ok :read [16 38] INFO [2022-01-10 05:46:31,072] jepsen worker 49 - jepsen.util 3829 :invoke :read [16 nil] INFO [2022-01-10 05:46:31,112] jepsen worker 53 - jepsen.util 4733 :ok :read [17 41] INFO [2022-01-10 05:46:31,112] jepsen worker 53 - jepsen.util 4733 :invoke :read [17 nil] INFO [2022-01-10 05:46:31,118] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:31,120] jepsen worker 71 - jepsen.util 3401 :invoke :write [23 6] INFO [2022-01-10 05:46:31,120] jepsen worker 69 - jepsen.util 3399 :ok :read [23 26] INFO [2022-01-10 05:46:31,120] jepsen worker 69 - jepsen.util 3399 :invoke :read [23 nil] INFO [2022-01-10 05:46:31,157] jepsen worker 69 - jepsen.util 3399 :ok :read [23 26] INFO [2022-01-10 05:46:31,158] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 34] INFO [2022-01-10 05:46:31,170] jepsen worker 79 - jepsen.util 4579 :ok :read [26 10] INFO [2022-01-10 05:46:31,171] jepsen worker 79 - jepsen.util 4579 :invoke :read [26 nil] INFO [2022-01-10 05:46:31,176] jepsen worker 52 - jepsen.util 4912 :ok :read [17 41] INFO [2022-01-10 05:46:31,177] jepsen worker 52 - jepsen.util 4912 :invoke :write [17 21] INFO [2022-01-10 05:46:31,194] jepsen worker 53 - jepsen.util 4733 :ok :read [17 41] INFO [2022-01-10 05:46:31,194] jepsen worker 53 - jepsen.util 4733 :invoke :write [17 49] INFO [2022-01-10 05:46:31,234] jepsen worker 79 - jepsen.util 4579 :ok :read [26 10] INFO [2022-01-10 05:46:31,234] jepsen worker 79 - jepsen.util 4579 :invoke :read [26 nil] INFO [2022-01-10 05:46:31,251] jepsen worker 83 - jepsen.util 4313 :info :write [27 24] :RequestTimeoutException INFO [2022-01-10 05:46:31,251] jepsen worker 83 - jepsen.util 4403 :invoke :read [27 nil] INFO [2022-01-10 05:46:31,404] jepsen worker 71 - jepsen.util 3401 :fail :write [23 6] :DurabilityImpossible INFO [2022-01-10 05:46:31,405] jepsen worker 71 - jepsen.util 3401 :invoke :write [23 22] INFO [2022-01-10 05:46:31,436] jepsen worker 39 - jepsen.util 4359 :info :write [13 27] :RequestTimeoutException INFO [2022-01-10 05:46:31,436] jepsen worker 39 - jepsen.util 4449 :invoke :read [13 nil] INFO [2022-01-10 05:46:31,438] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4970760233918129}, :ns_1@172.23.107.227 {:progress 0.343108504398827}, :ns_1@172.23.105.187 {:progress 0.343108504398827}, :ns_1@172.23.107.228 {:progress 0.4705882352941176}} INFO [2022-01-10 05:46:31,438] jepsen worker nemesis - couchbase.util Rebalance check count: 540 INFO [2022-01-10 05:46:31,439] jepsen worker nemesis - couchbase.util Rebalance stuck count: 452 INFO [2022-01-10 05:46:31,443] jepsen worker 79 - jepsen.util 4579 :ok :read [26 10] INFO [2022-01-10 05:46:31,443] jepsen worker 79 - jepsen.util 4579 :invoke :write [26 46] INFO [2022-01-10 05:46:31,518] jepsen worker 69 - jepsen.util 3399 :fail :write [23 34] :DurabilityImpossible INFO [2022-01-10 05:46:31,518] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 25] INFO [2022-01-10 05:46:31,522] jepsen worker 14 - jepsen.util 4694 :info :write [4 6] :RequestTimeoutException INFO [2022-01-10 05:46:31,522] jepsen worker 14 - jepsen.util 4784 :invoke :write [4 19] INFO [2022-01-10 05:46:31,776] jepsen worker 76 - jepsen.util 3406 :info :write [25 43] :RequestTimeoutException INFO [2022-01-10 05:46:31,777] jepsen worker 76 - jepsen.util 3496 :invoke :read [25 nil] INFO [2022-01-10 05:46:31,845] jepsen worker 71 - jepsen.util 3401 :fail :write [23 22] :DurabilityImpossible INFO [2022-01-10 05:46:31,846] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:31,867] jepsen worker 2 - jepsen.util 4142 :ok :read [0 46] INFO [2022-01-10 05:46:31,867] jepsen worker 2 - jepsen.util 4142 :invoke :write [0 26] INFO [2022-01-10 05:46:31,913] jepsen worker 69 - jepsen.util 3399 :fail :write [23 25] :DurabilityImpossible INFO [2022-01-10 05:46:31,914] jepsen worker 69 - jepsen.util 3399 :invoke :read [23 nil] INFO [2022-01-10 05:46:31,958] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:31,959] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:31,960] jepsen worker 39 - jepsen.util 4449 :ok :read [13 15] INFO [2022-01-10 05:46:31,960] jepsen worker 39 - jepsen.util 4449 :invoke :write [13 13] INFO [2022-01-10 05:46:31,965] jepsen worker 69 - jepsen.util 3399 :ok :read [23 26] INFO [2022-01-10 05:46:31,966] jepsen worker 69 - jepsen.util 3399 :invoke :read [23 nil] WARN [2022-01-10 05:46:32,009] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16845097,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17739709,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9256954,"encode_duration_us":21506,"operation_id":"0x7507","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10001675}],"total_count":1}} INFO [2022-01-10 05:46:32,021] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:32,021] jepsen worker 71 - jepsen.util 3401 :invoke :write [23 49] INFO [2022-01-10 05:46:32,028] jepsen worker 11 - jepsen.util 3701 :ok :write [3 12] INFO [2022-01-10 05:46:32,029] jepsen worker 11 - jepsen.util 3701 :invoke :read [3 nil] INFO [2022-01-10 05:46:32,030] jepsen worker 69 - jepsen.util 3399 :ok :read [23 26] INFO [2022-01-10 05:46:32,031] jepsen worker 69 - jepsen.util 3399 :invoke :read [23 nil] INFO [2022-01-10 05:46:32,072] jepsen worker 69 - jepsen.util 3399 :ok :read [23 26] INFO [2022-01-10 05:46:32,072] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 38] WARN [2022-01-10 05:46:32,097] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":58682,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":1966394,"total_server_duration_us":1579848,"encode_duration_us":17728,"operation_id":"0x7769","timeout_ms":10000,"last_server_duration_us":18570,"total_duration_us":10009165},{"operation_name":"upsert","last_dispatch_duration_us":1080534,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4457681,"total_server_duration_us":775549,"encode_duration_us":18666,"operation_id":"0x76c1","timeout_ms":10000,"last_server_duration_us":48219,"total_duration_us":10008250},{"operation_name":"upsert","last_dispatch_duration_us":8163619,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":12908152,"total_server_duration_us":292115,"encode_duration_us":17995,"operation_id":"0x7659","timeout_ms":10000,"last_server_duration_us":59500,"total_duration_us":10007523},{"operation_name":"upsert","last_dispatch_duration_us":34656,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":8707504,"total_server_duration_us":247822,"encode_duration_us":17131,"operation_id":"0x771f","timeout_ms":10000,"last_server_duration_us":23390,"total_duration_us":10007516},{"operation_name":"upsert","last_dispatch_duration_us":9630257,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11673241,"total_server_duration_us":9467751,"encode_duration_us":17606,"operation_id":"0x7667","timeout_ms":10000,"last_server_duration_us":9293616,"total_duration_us":10005191},{"operation_name":"upsert","last_dispatch_duration_us":1150335,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3690832,"total_server_duration_us":312452,"encode_duration_us":18750,"operation_id":"0x7668","timeout_ms":10000,"last_server_duration_us":47533,"total_duration_us":10004575},{"operation_name":"upsert","last_dispatch_duration_us":9751250,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11717087,"total_server_duration_us":9576498,"encode_duration_us":16666,"operation_id":"0x7666","timeout_ms":10000,"last_server_duration_us":9431526,"total_duration_us":10004337},{"operation_name":"upsert","last_dispatch_duration_us":769446,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3050394,"total_server_duration_us":357296,"encode_duration_us":22426,"operation_id":"0x7809","timeout_ms":10000,"last_server_duration_us":242246,"total_duration_us":10003721},{"operation_name":"upsert","last_dispatch_duration_us":49956,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9544035,"total_server_duration_us":207032,"encode_duration_us":29551,"operation_id":"0x76e9","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10003589},{"operation_name":"upsert","last_dispatch_duration_us":857013,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3236601,"total_server_duration_us":568960,"encode_duration_us":16633,"operation_id":"0x772e","timeout_ms":10000,"last_server_duration_us":253514,"total_duration_us":10002226}],"total_count":24}} WARN [2022-01-10 05:46:32,097] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9051156,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9704297,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9391239,"encode_duration_us":17119,"operation_id":"0x75f7","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10003807},{"operation_name":"upsert","last_dispatch_duration_us":9751250,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11717087,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9576498,"encode_duration_us":16666,"operation_id":"0x7666","timeout_ms":10000,"last_server_duration_us":9431526,"total_duration_us":10004337},{"operation_name":"upsert","last_dispatch_duration_us":9630257,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11673241,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9467751,"encode_duration_us":17606,"operation_id":"0x7667","timeout_ms":10000,"last_server_duration_us":9293616,"total_duration_us":10005191}],"total_count":3}} INFO [2022-01-10 05:46:32,121] jepsen worker 75 - jepsen.util 3495 :info :write [25 48] :RequestTimeoutException INFO [2022-01-10 05:46:32,121] jepsen worker 75 - jepsen.util 3585 :invoke :write [25 18] INFO [2022-01-10 05:46:32,146] jepsen worker 82 - jepsen.util 3952 :info :write [27 47] :RequestTimeoutException INFO [2022-01-10 05:46:32,147] jepsen worker 82 - jepsen.util 4042 :invoke :read [27 nil] WARN [2022-01-10 05:46:32,200] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":8268620,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8268620,"total_server_duration_us":76,"operation_id":"0x753d","timeout_ms":30000,"last_server_duration_us":76,"total_duration_us":17520342},{"operation_name":"get","last_dispatch_duration_us":11686591,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11686591,"total_server_duration_us":6432,"operation_id":"0x76b7","timeout_ms":30000,"last_server_duration_us":6432,"total_duration_us":11688534},{"operation_name":"get","last_dispatch_duration_us":11322956,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11322956,"total_server_duration_us":8123,"operation_id":"0x771e","timeout_ms":30000,"last_server_duration_us":8123,"total_duration_us":11324923},{"operation_name":"get","last_dispatch_duration_us":2277136,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":10466556,"total_server_duration_us":2268137,"operation_id":"0x77cf","timeout_ms":30000,"last_server_duration_us":2268105,"total_duration_us":10473177},{"operation_name":"upsert","last_dispatch_duration_us":495004,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6336231,"total_server_duration_us":1184914,"encode_duration_us":18762,"operation_id":"0x7731","timeout_ms":10000,"last_server_duration_us":104228,"total_duration_us":10009684},{"operation_name":"upsert","last_dispatch_duration_us":11740159,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11740159,"total_server_duration_us":221464,"encode_duration_us":19445,"operation_id":"0x76b6","timeout_ms":10000,"last_server_duration_us":221464,"total_duration_us":10009374},{"operation_name":"upsert","last_dispatch_duration_us":491319,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":5477462,"total_server_duration_us":1100939,"encode_duration_us":19546,"operation_id":"0x7722","timeout_ms":10000,"last_server_duration_us":91985,"total_duration_us":10009080},{"operation_name":"upsert","last_dispatch_duration_us":606076,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4814047,"total_server_duration_us":510473,"encode_duration_us":24094,"operation_id":"0x7774","timeout_ms":10000,"last_server_duration_us":46626,"total_duration_us":10008416},{"operation_name":"upsert","last_dispatch_duration_us":40687,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":2743417,"total_server_duration_us":2189334,"encode_duration_us":31617,"operation_id":"0x7663","timeout_ms":10000,"last_server_duration_us":24069,"total_duration_us":10008187},{"operation_name":"upsert","last_dispatch_duration_us":10568689,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":12102881,"total_server_duration_us":368777,"encode_duration_us":16798,"operation_id":"0x7662","timeout_ms":10000,"last_server_duration_us":141868,"total_duration_us":10007440}],"total_count":29}} INFO [2022-01-10 05:46:32,200] jepsen worker 71 - jepsen.util 3401 :fail :write [23 49] :DurabilityImpossible INFO [2022-01-10 05:46:32,200] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] WARN [2022-01-10 05:46:32,202] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9258181,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":11815791,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":10777429,"encode_duration_us":17618,"operation_id":"0x76c4","timeout_ms":10000,"last_server_duration_us":9234623,"total_duration_us":10001290},{"operation_name":"upsert","last_dispatch_duration_us":9035987,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9850557,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9457418,"encode_duration_us":16632,"operation_id":"0x759d","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10006264},{"operation_name":"upsert","last_dispatch_duration_us":9437900,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14872158,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":10644376,"encode_duration_us":16835,"operation_id":"0x76f8","timeout_ms":10000,"last_server_duration_us":9118182,"total_duration_us":10006832}],"total_count":3}} WARN [2022-01-10 05:46:32,210] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14652165,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":14652165,"total_server_duration_us":27,"operation_id":"0x75cb","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":14653492},{"operation_name":"upsert","last_dispatch_duration_us":1214671,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5078327,"total_server_duration_us":681955,"encode_duration_us":20246,"operation_id":"0x7799","timeout_ms":10000,"last_server_duration_us":202533,"total_duration_us":10009591},{"operation_name":"upsert","last_dispatch_duration_us":1041500,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6065454,"total_server_duration_us":1014395,"encode_duration_us":16733,"operation_id":"0x77f4","timeout_ms":10000,"last_server_duration_us":80826,"total_duration_us":10009587},{"operation_name":"upsert","last_dispatch_duration_us":955812,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8120475,"total_server_duration_us":556013,"encode_duration_us":20987,"operation_id":"0x77a5","timeout_ms":10000,"last_server_duration_us":114786,"total_duration_us":10009136},{"operation_name":"upsert","last_dispatch_duration_us":1421646,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8438585,"total_server_duration_us":599995,"encode_duration_us":18834,"operation_id":"0x7839","timeout_ms":10000,"last_server_duration_us":62157,"total_duration_us":10007143},{"operation_name":"upsert","last_dispatch_duration_us":3769467,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":3769467,"total_server_duration_us":19262,"encode_duration_us":16743,"operation_id":"0x77fc","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10006784},{"operation_name":"upsert","last_dispatch_duration_us":1071158,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4647329,"total_server_duration_us":465498,"encode_duration_us":16524,"operation_id":"0x782c","timeout_ms":10000,"last_server_duration_us":96250,"total_duration_us":10006495},{"operation_name":"upsert","last_dispatch_duration_us":118319,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":2547797,"total_server_duration_us":1213590,"encode_duration_us":17730,"operation_id":"0x7752","timeout_ms":10000,"last_server_duration_us":78132,"total_duration_us":10005330},{"operation_name":"upsert","last_dispatch_duration_us":1138593,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9506158,"total_server_duration_us":498357,"encode_duration_us":18075,"operation_id":"0x76c0","timeout_ms":10000,"last_server_duration_us":163951,"total_duration_us":10003343},{"operation_name":"upsert","last_dispatch_duration_us":1077026,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3096825,"total_server_duration_us":248025,"encode_duration_us":16848,"operation_id":"0x7798","timeout_ms":10000,"last_server_duration_us":111655,"total_duration_us":10003270}],"total_count":20}} WARN [2022-01-10 05:46:32,211] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":27047664,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":27047664,"total_server_duration_us":69,"operation_id":"0x7353","timeout_ms":30000,"last_server_duration_us":69,"total_duration_us":27049476},{"operation_name":"get","last_dispatch_duration_us":23692352,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":23692352,"total_server_duration_us":76,"operation_id":"0x7412","timeout_ms":30000,"last_server_duration_us":76,"total_duration_us":23693679},{"operation_name":"upsert","last_dispatch_duration_us":900414,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":4746675,"total_server_duration_us":648039,"encode_duration_us":28223,"operation_id":"0x77d7","timeout_ms":10000,"last_server_duration_us":394846,"total_duration_us":10007481},{"operation_name":"upsert","last_dispatch_duration_us":8786613,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":11083851,"total_server_duration_us":1057571,"encode_duration_us":26482,"operation_id":"0x76e5","timeout_ms":10000,"last_server_duration_us":53974,"total_duration_us":10007471},{"operation_name":"upsert","last_dispatch_duration_us":493119,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10383719,"total_server_duration_us":232468,"encode_duration_us":23181,"operation_id":"0x775e","timeout_ms":10000,"last_server_duration_us":50301,"total_duration_us":10007255},{"operation_name":"upsert","last_dispatch_duration_us":2267302,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":6610763,"total_server_duration_us":902049,"encode_duration_us":16913,"operation_id":"0x77a3","timeout_ms":10000,"last_server_duration_us":181700,"total_duration_us":10006430},{"operation_name":"upsert","last_dispatch_duration_us":483811,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":10059938,"total_server_duration_us":357631,"encode_duration_us":16354,"operation_id":"0x783a","timeout_ms":10000,"last_server_duration_us":95942,"total_duration_us":10004386},{"operation_name":"upsert","last_dispatch_duration_us":1650515,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10030450,"total_server_duration_us":1641624,"encode_duration_us":16467,"operation_id":"0x76c2","timeout_ms":10000,"last_server_duration_us":757059,"total_duration_us":10003692},{"operation_name":"upsert","last_dispatch_duration_us":2161480,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":7594195,"total_server_duration_us":1000855,"encode_duration_us":17116,"operation_id":"0x771d","timeout_ms":10000,"last_server_duration_us":627617,"total_duration_us":10001852},{"operation_name":"upsert","last_dispatch_duration_us":9045227,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":11932433,"total_server_duration_us":10831072,"encode_duration_us":17425,"operation_id":"0x768d","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10000769}],"total_count":17}} INFO [2022-01-10 05:46:32,230] jepsen worker 44 - jepsen.util 3914 :info :write [14 13] :RequestTimeoutException INFO [2022-01-10 05:46:32,231] jepsen worker 44 - jepsen.util 4004 :invoke :write [14 40] INFO [2022-01-10 05:46:32,250] jepsen worker 0 - jepsen.util 3690 :info :write [0 15] :RequestTimeoutException INFO [2022-01-10 05:46:32,250] jepsen worker 0 - jepsen.util 3780 :invoke :write [0 34] INFO [2022-01-10 05:46:32,256] jepsen worker 69 - jepsen.util 3399 :fail :write [23 38] :DurabilityImpossible INFO [2022-01-10 05:46:32,257] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 49] INFO [2022-01-10 05:46:32,261] jepsen worker 12 - jepsen.util 4332 :info :write [4 43] :RequestTimeoutException INFO [2022-01-10 05:46:32,262] jepsen worker 12 - jepsen.util 4422 :invoke :read [4 nil] INFO [2022-01-10 05:46:32,296] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:32,296] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] WARN [2022-01-10 05:46:32,300] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17657192,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18408887,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9112600,"encode_duration_us":19052,"operation_id":"0x74fb","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":10006555},{"operation_name":"upsert","last_dispatch_duration_us":9641753,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13305504,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9330046,"encode_duration_us":17181,"operation_id":"0x75ed","timeout_ms":10000,"last_server_duration_us":9157638,"total_duration_us":10011345},{"operation_name":"upsert","last_dispatch_duration_us":12310760,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":21372701,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":3689506,"encode_duration_us":23795,"operation_id":"0x7503","timeout_ms":10000,"last_server_duration_us":3627580,"total_duration_us":10010484}],"total_count":3}} WARN [2022-01-10 05:46:32,311] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9045227,"last_local_id":"EFE9906200000004/000000008B42EFB5","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":11932433,"last_local_socket":"172.23.104.79:44292","total_server_duration_us":10831072,"encode_duration_us":17425,"operation_id":"0x768d","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10000769},{"operation_name":"upsert","last_dispatch_duration_us":9374438,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16259511,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9869761,"encode_duration_us":17287,"operation_id":"0x7560","timeout_ms":10000,"last_server_duration_us":9137368,"total_duration_us":10009158},{"operation_name":"upsert","last_dispatch_duration_us":32917482,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":32917482,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9053293,"encode_duration_us":16687,"operation_id":"0x723d","timeout_ms":10000,"last_server_duration_us":9053293,"total_duration_us":10007306}],"total_count":3}} INFO [2022-01-10 05:46:32,385] jepsen worker 33 - jepsen.util 4083 :ok :read [11 21] INFO [2022-01-10 05:46:32,386] jepsen worker 33 - jepsen.util 4083 :invoke :write [11 25] WARN [2022-01-10 05:46:32,403] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1947314,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10277644,"total_server_duration_us":1141634,"encode_duration_us":16756,"operation_id":"0x768c","timeout_ms":10000,"last_server_duration_us":204225,"total_duration_us":10010776},{"operation_name":"upsert","last_dispatch_duration_us":8755978,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11269708,"total_server_duration_us":634018,"encode_duration_us":16263,"operation_id":"0x7805","timeout_ms":10000,"last_server_duration_us":137364,"total_duration_us":10010227},{"operation_name":"upsert","last_dispatch_duration_us":1255975,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9709848,"total_server_duration_us":861094,"encode_duration_us":17796,"operation_id":"0x77f3","timeout_ms":10000,"last_server_duration_us":170957,"total_duration_us":10008549},{"operation_name":"upsert","last_dispatch_duration_us":3767099,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11807156,"total_server_duration_us":780902,"encode_duration_us":17312,"operation_id":"0x771c","timeout_ms":10000,"last_server_duration_us":70960,"total_duration_us":10008202},{"operation_name":"upsert","last_dispatch_duration_us":8735760,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":14594695,"total_server_duration_us":2028051,"encode_duration_us":18114,"operation_id":"0x7753","timeout_ms":10000,"last_server_duration_us":420415,"total_duration_us":10007183},{"operation_name":"upsert","last_dispatch_duration_us":1652892,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7658653,"total_server_duration_us":239085,"encode_duration_us":20698,"operation_id":"0x7836","timeout_ms":10000,"last_server_duration_us":29230,"total_duration_us":10007112},{"operation_name":"upsert","last_dispatch_duration_us":9437900,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":14872158,"total_server_duration_us":10644376,"encode_duration_us":16835,"operation_id":"0x76f8","timeout_ms":10000,"last_server_duration_us":9118182,"total_duration_us":10006832},{"operation_name":"upsert","last_dispatch_duration_us":31678,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8559390,"total_server_duration_us":265770,"encode_duration_us":17197,"operation_id":"0x76f1","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10004464},{"operation_name":"upsert","last_dispatch_duration_us":4492813,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10088147,"total_server_duration_us":786413,"encode_duration_us":17131,"operation_id":"0x772a","timeout_ms":10000,"last_server_duration_us":122347,"total_duration_us":10001421},{"operation_name":"upsert","last_dispatch_duration_us":9258181,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":11815791,"total_server_duration_us":10777429,"encode_duration_us":17618,"operation_id":"0x76c4","timeout_ms":10000,"last_server_duration_us":9234623,"total_duration_us":10001290}],"total_count":14}} INFO [2022-01-10 05:46:32,404] jepsen worker 26 - jepsen.util 3626 :ok :read [8 39] INFO [2022-01-10 05:46:32,405] jepsen worker 26 - jepsen.util 3626 :invoke :write [8 34] INFO [2022-01-10 05:46:32,447] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4970760233918129}, :ns_1@172.23.107.227 {:progress 0.343108504398827}, :ns_1@172.23.105.187 {:progress 0.343108504398827}, :ns_1@172.23.107.228 {:progress 0.4764705882352941}} INFO [2022-01-10 05:46:32,448] jepsen worker nemesis - couchbase.util Rebalance check count: 541 INFO [2022-01-10 05:46:32,448] jepsen worker nemesis - couchbase.util Rebalance stuck count: 452 INFO [2022-01-10 05:46:32,486] jepsen worker 15 - jepsen.util 3345 :info :write [5 9] :RequestTimeoutException INFO [2022-01-10 05:46:32,487] jepsen worker 15 - jepsen.util 3435 :invoke :write [5 47] INFO [2022-01-10 05:46:32,531] jepsen worker 68 - jepsen.util 3938 :info :write [22 27] :RequestTimeoutException INFO [2022-01-10 05:46:32,532] jepsen worker 68 - jepsen.util 4028 :invoke :write [22 3] INFO [2022-01-10 05:46:32,554] jepsen worker 34 - jepsen.util 3724 :info :write [11 15] :RequestTimeoutException INFO [2022-01-10 05:46:32,555] jepsen worker 34 - jepsen.util 3814 :invoke :write [11 45] INFO [2022-01-10 05:46:32,626] jepsen worker 47 - jepsen.util 3827 :info :write [15 47] :RequestTimeoutException INFO [2022-01-10 05:46:32,627] jepsen worker 47 - jepsen.util 3917 :invoke :write [15 46] INFO [2022-01-10 05:46:32,676] jepsen worker 8 - jepsen.util 4328 :ok :read [2 14] INFO [2022-01-10 05:46:32,676] jepsen worker 8 - jepsen.util 4328 :invoke :write [2 36] INFO [2022-01-10 05:46:32,762] jepsen worker 5 - jepsen.util 3605 :info :write [1 27] :SyncWriteAmbiguous INFO [2022-01-10 05:46:32,762] jepsen worker 5 - jepsen.util 3695 :invoke :write [1 15] INFO [2022-01-10 05:46:32,859] jepsen worker 54 - jepsen.util 3924 :ok :read [18 14] INFO [2022-01-10 05:46:32,860] jepsen worker 54 - jepsen.util 3924 :invoke :read [18 nil] INFO [2022-01-10 05:46:32,889] jepsen worker 54 - jepsen.util 3924 :ok :read [18 14] INFO [2022-01-10 05:46:32,890] jepsen worker 54 - jepsen.util 3924 :invoke :read [18 nil] INFO [2022-01-10 05:46:32,897] jepsen worker 69 - jepsen.util 3399 :fail :write [23 49] :DurabilityImpossible INFO [2022-01-10 05:46:32,900] jepsen worker 69 - jepsen.util 3399 :invoke :write [23 34] INFO [2022-01-10 05:46:32,906] jepsen worker 66 - jepsen.util 3576 :info :write [22 23] :RequestTimeoutException INFO [2022-01-10 05:46:32,907] jepsen worker 66 - jepsen.util 3666 :invoke :read [22 nil] INFO [2022-01-10 05:46:32,915] jepsen worker 87 - jepsen.util 3687 :info :write [29 23] :SyncWriteAmbiguous INFO [2022-01-10 05:46:32,916] jepsen worker 87 - jepsen.util 3777 :invoke :read [29 nil] INFO [2022-01-10 05:46:32,927] jepsen worker 54 - jepsen.util 3924 :ok :read [18 14] INFO [2022-01-10 05:46:32,928] jepsen worker 54 - jepsen.util 3924 :invoke :write [18 36] INFO [2022-01-10 05:46:32,941] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:32,942] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:32,982] jepsen worker 12 - jepsen.util 4422 :ok :read [4 3] INFO [2022-01-10 05:46:32,982] jepsen worker 12 - jepsen.util 4422 :invoke :read [4 nil] INFO [2022-01-10 05:46:33,029] jepsen worker 56 - jepsen.util 4196 :ok :read [18 14] INFO [2022-01-10 05:46:33,030] jepsen worker 56 - jepsen.util 4196 :invoke :read [18 nil] INFO [2022-01-10 05:46:33,032] jepsen worker 28 - jepsen.util 3628 :info :write [9 7] :RequestTimeoutException INFO [2022-01-10 05:46:33,032] jepsen worker 28 - jepsen.util 3718 :invoke :write [9 13] INFO [2022-01-10 05:46:33,037] jepsen worker 10 - jepsen.util 3790 :ok :read [3 12] INFO [2022-01-10 05:46:33,038] jepsen worker 10 - jepsen.util 3790 :invoke :write [3 3] INFO [2022-01-10 05:46:33,055] jepsen worker 55 - jepsen.util 4285 :ok :read [18 14] INFO [2022-01-10 05:46:33,058] jepsen worker 55 - jepsen.util 4285 :invoke :read [18 nil] INFO [2022-01-10 05:46:33,062] jepsen worker 11 - jepsen.util 3701 :ok :read [3 12] INFO [2022-01-10 05:46:33,063] jepsen worker 11 - jepsen.util 3701 :invoke :read [3 nil] INFO [2022-01-10 05:46:33,069] jepsen worker 56 - jepsen.util 4196 :ok :read [18 14] INFO [2022-01-10 05:46:33,070] jepsen worker 56 - jepsen.util 4196 :invoke :read [18 nil] INFO [2022-01-10 05:46:33,090] jepsen worker 78 - jepsen.util 4668 :info :write [26 8] :RequestTimeoutException INFO [2022-01-10 05:46:33,091] jepsen worker 78 - jepsen.util 4758 :invoke :write [26 47] INFO [2022-01-10 05:46:33,131] jepsen worker 55 - jepsen.util 4285 :ok :read [18 14] INFO [2022-01-10 05:46:33,132] jepsen worker 55 - jepsen.util 4285 :invoke :read [18 nil] INFO [2022-01-10 05:46:33,135] jepsen worker 11 - jepsen.util 3701 :ok :read [3 12] INFO [2022-01-10 05:46:33,136] jepsen worker 11 - jepsen.util 3701 :invoke :write [3 2] INFO [2022-01-10 05:46:33,143] jepsen worker 56 - jepsen.util 4196 :ok :read [18 14] INFO [2022-01-10 05:46:33,144] jepsen worker 56 - jepsen.util 4196 :invoke :write [18 48] INFO [2022-01-10 05:46:33,178] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:33,179] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:33,195] jepsen worker 55 - jepsen.util 4285 :ok :read [18 14] INFO [2022-01-10 05:46:33,196] jepsen worker 55 - jepsen.util 4285 :invoke :read [18 nil] INFO [2022-01-10 05:46:33,245] jepsen worker 17 - jepsen.util 3707 :info :write [5 40] :RequestTimeoutException INFO [2022-01-10 05:46:33,246] jepsen worker 17 - jepsen.util 3797 :invoke :write [5 36] INFO [2022-01-10 05:46:33,315] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:33,315] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:33,344] jepsen worker 55 - jepsen.util 4285 :ok :read [18 14] INFO [2022-01-10 05:46:33,344] jepsen worker 55 - jepsen.util 4285 :invoke :write [18 12] INFO [2022-01-10 05:46:33,455] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4970760233918129}, :ns_1@172.23.107.227 {:progress 0.343108504398827}, :ns_1@172.23.105.187 {:progress 0.343108504398827}, :ns_1@172.23.107.228 {:progress 0.4764705882352941}} INFO [2022-01-10 05:46:33,455] jepsen worker nemesis - couchbase.util Rebalance check count: 542 INFO [2022-01-10 05:46:33,455] jepsen worker nemesis - couchbase.util Rebalance stuck count: 453 INFO [2022-01-10 05:46:33,523] jepsen worker 12 - jepsen.util 4422 :ok :read [4 3] INFO [2022-01-10 05:46:33,524] jepsen worker 12 - jepsen.util 4422 :invoke :read [4 nil] INFO [2022-01-10 05:46:33,591] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:33,592] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:33,696] jepsen worker 31 - jepsen.util 4171 :info :write [10 42] :RequestTimeoutException INFO [2022-01-10 05:46:33,696] jepsen worker 31 - jepsen.util 4261 :invoke :read [10 nil] INFO [2022-01-10 05:46:33,706] jepsen worker 70 - jepsen.util 3760 :info :write [23 29] :RequestTimeoutException INFO [2022-01-10 05:46:33,706] jepsen worker 70 - jepsen.util 3850 :invoke :read [23 nil] INFO [2022-01-10 05:46:33,766] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:33,767] jepsen worker 71 - jepsen.util 3401 :invoke :read [23 nil] INFO [2022-01-10 05:46:33,797] jepsen worker 12 - jepsen.util 4422 :ok :read [4 3] INFO [2022-01-10 05:46:33,797] jepsen worker 12 - jepsen.util 4422 :invoke :read [4 nil] INFO [2022-01-10 05:46:33,941] jepsen worker 4 - jepsen.util 4414 :info :write [1 37] :RequestTimeoutException INFO [2022-01-10 05:46:33,942] jepsen worker 4 - jepsen.util 4504 :invoke :read [1 nil] INFO [2022-01-10 05:46:33,962] jepsen worker 88 - jepsen.util 3688 :info :write [29 5] :RequestTimeoutException INFO [2022-01-10 05:46:33,962] jepsen worker 88 - jepsen.util 3778 :invoke :read [29 nil] INFO [2022-01-10 05:46:33,998] jepsen worker 88 - jepsen.util 3778 :ok :read [29 0] INFO [2022-01-10 05:46:33,999] jepsen worker 88 - jepsen.util 3778 :invoke :read [29 nil] INFO [2022-01-10 05:46:34,022] jepsen worker 88 - jepsen.util 3778 :ok :read [29 0] INFO [2022-01-10 05:46:34,022] jepsen worker 88 - jepsen.util 3778 :invoke :read [29 nil] INFO [2022-01-10 05:46:34,044] jepsen worker 88 - jepsen.util 3778 :ok :read [29 0] INFO [2022-01-10 05:46:34,045] jepsen worker 88 - jepsen.util 3778 :invoke :write [29 26] INFO [2022-01-10 05:46:34,416] jepsen worker 81 - jepsen.util 4221 :info :write [27 0] :RequestTimeoutException INFO [2022-01-10 05:46:34,417] jepsen worker 81 - jepsen.util 4311 :invoke :read [27 nil] INFO [2022-01-10 05:46:34,463] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4970760233918129}, :ns_1@172.23.107.227 {:progress 0.343108504398827}, :ns_1@172.23.105.187 {:progress 0.343108504398827}, :ns_1@172.23.107.228 {:progress 0.4764705882352941}} INFO [2022-01-10 05:46:34,463] jepsen worker nemesis - couchbase.util Rebalance check count: 543 INFO [2022-01-10 05:46:34,471] jepsen worker 43 - jepsen.util 3733 :info :write [14 3] :RequestTimeoutException INFO [2022-01-10 05:46:34,525] jepsen worker nemesis - couchbase.util Rebalance stuck count: 454 INFO [2022-01-10 05:46:34,525] jepsen worker 43 - jepsen.util 3823 :invoke :read [14 nil] INFO [2022-01-10 05:46:34,534] jepsen worker 71 - jepsen.util 3401 :ok :read [23 26] INFO [2022-01-10 05:46:34,535] jepsen worker 71 - jepsen.util 3401 :invoke :write [23 20] INFO [2022-01-10 05:46:34,735] jepsen worker 70 - jepsen.util 3850 :ok :read [23 26] INFO [2022-01-10 05:46:34,736] jepsen worker 70 - jepsen.util 3850 :invoke :read [23 nil] INFO [2022-01-10 05:46:34,754] jepsen worker 12 - jepsen.util 4422 :ok :read [4 3] INFO [2022-01-10 05:46:34,754] jepsen worker 12 - jepsen.util 4422 :invoke :write [4 8] INFO [2022-01-10 05:46:34,929] jepsen worker 70 - jepsen.util 3850 :ok :read [23 26] INFO [2022-01-10 05:46:34,929] jepsen worker 70 - jepsen.util 3850 :invoke :write [23 36] INFO [2022-01-10 05:46:34,962] jepsen worker 51 - jepsen.util 4641 :info :write [17 0] :RequestTimeoutException INFO [2022-01-10 05:46:34,963] jepsen worker 51 - jepsen.util 4731 :invoke :read [17 nil] INFO [2022-01-10 05:46:35,081] jepsen worker 32 - jepsen.util 4082 :info :write [10 7] :RequestTimeoutException INFO [2022-01-10 05:46:35,082] jepsen worker 32 - jepsen.util 4172 :invoke :read [10 nil] INFO [2022-01-10 05:46:35,215] jepsen worker 37 - jepsen.util 3637 :info :write [12 47] :RequestTimeoutException INFO [2022-01-10 05:46:35,217] jepsen worker 37 - jepsen.util 3727 :invoke :write [12 42] INFO [2022-01-10 05:46:35,241] jepsen worker 43 - jepsen.util 3823 :ok :read [14 14] INFO [2022-01-10 05:46:35,243] jepsen worker 43 - jepsen.util 3823 :invoke :read [14 nil] INFO [2022-01-10 05:46:35,266] jepsen worker 61 - jepsen.util 4021 :info :write [20 38] :RequestTimeoutException INFO [2022-01-10 05:46:35,266] jepsen worker 61 - jepsen.util 4111 :invoke :read [20 nil] INFO [2022-01-10 05:46:35,370] jepsen worker 20 - jepsen.util 3620 :info :write [6 14] :RequestTimeoutException INFO [2022-01-10 05:46:35,370] jepsen worker 20 - jepsen.util 3710 :invoke :read [6 nil] INFO [2022-01-10 05:46:35,372] jepsen worker 45 - jepsen.util 4185 :ok :read [15 23] INFO [2022-01-10 05:46:35,372] jepsen worker 45 - jepsen.util 4185 :invoke :read [15 nil] INFO [2022-01-10 05:46:35,431] jepsen worker 29 - jepsen.util 3809 :info :write [9 21] :RequestTimeoutException INFO [2022-01-10 05:46:35,432] jepsen worker 29 - jepsen.util 3899 :invoke :read [9 nil] INFO [2022-01-10 05:46:35,476] jepsen worker 21 - jepsen.util 3891 :info :write [7 28] :RequestTimeoutException INFO [2022-01-10 05:46:35,477] jepsen worker 21 - jepsen.util 3981 :invoke :write [7 5] INFO [2022-01-10 05:46:35,530] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4970760233918129}, :ns_1@172.23.107.227 {:progress 0.343108504398827}, :ns_1@172.23.105.187 {:progress 0.343108504398827}, :ns_1@172.23.107.228 {:progress 0.4764705882352941}} INFO [2022-01-10 05:46:35,530] jepsen worker nemesis - couchbase.util Rebalance check count: 544 INFO [2022-01-10 05:46:35,530] jepsen worker nemesis - couchbase.util Rebalance stuck count: 455 INFO [2022-01-10 05:46:35,571] jepsen worker 25 - jepsen.util 3715 :info :write [8 11] :RequestTimeoutException INFO [2022-01-10 05:46:35,572] jepsen worker 25 - jepsen.util 3805 :invoke :read [8 nil] INFO [2022-01-10 05:46:35,638] jepsen worker 45 - jepsen.util 4185 :ok :read [15 23] INFO [2022-01-10 05:46:35,638] jepsen worker 45 - jepsen.util 4185 :invoke :write [15 5] INFO [2022-01-10 05:46:35,900] jepsen worker 40 - jepsen.util 3370 :info :write [13 21] :RequestTimeoutException INFO [2022-01-10 05:46:35,900] jepsen worker 40 - jepsen.util 3460 :invoke :write [13 39] INFO [2022-01-10 05:46:36,021] jepsen worker 36 - jepsen.util 3006 :info :write [12 14] :RequestTimeoutException INFO [2022-01-10 05:46:36,022] jepsen worker 36 - jepsen.util 3096 :invoke :read [12 nil] INFO [2022-01-10 05:46:36,265] jepsen worker 51 - jepsen.util 4731 :ok :read [17 41] INFO [2022-01-10 05:46:36,265] jepsen worker 51 - jepsen.util 4731 :invoke :read [17 nil] INFO [2022-01-10 05:46:36,285] jepsen worker 83 - jepsen.util 4403 :ok :read [27 30] INFO [2022-01-10 05:46:36,286] jepsen worker 83 - jepsen.util 4403 :invoke :read [27 nil] INFO [2022-01-10 05:46:36,294] jepsen worker 41 - jepsen.util 3731 :info :write [13 33] :RequestTimeoutException INFO [2022-01-10 05:46:36,295] jepsen worker 41 - jepsen.util 3821 :invoke :read [13 nil] INFO [2022-01-10 05:46:36,366] jepsen worker 43 - jepsen.util 3823 :ok :read [14 14] INFO [2022-01-10 05:46:36,367] jepsen worker 43 - jepsen.util 3823 :invoke :read [14 nil] INFO [2022-01-10 05:46:36,461] jepsen worker 30 - jepsen.util 3540 :info :write [10 32] :SyncWriteAmbiguous INFO [2022-01-10 05:46:36,462] jepsen worker 30 - jepsen.util 3630 :invoke :write [10 27] INFO [2022-01-10 05:46:36,538] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.4970760233918129}, :ns_1@172.23.107.227 {:progress 0.343108504398827}, :ns_1@172.23.105.187 {:progress 0.343108504398827}, :ns_1@172.23.107.228 {:progress 0.4764705882352941}} INFO [2022-01-10 05:46:36,538] jepsen worker nemesis - couchbase.util Rebalance check count: 545 INFO [2022-01-10 05:46:36,538] jepsen worker nemesis - couchbase.util Rebalance stuck count: 456 INFO [2022-01-10 05:46:36,551] jepsen worker 83 - jepsen.util 4403 :ok :read [27 30] INFO [2022-01-10 05:46:36,552] jepsen worker 83 - jepsen.util 4403 :invoke :read [27 nil] INFO [2022-01-10 05:46:36,563] jepsen worker 29 - jepsen.util 3899 :ok :read [9 27] INFO [2022-01-10 05:46:36,563] jepsen worker 29 - jepsen.util 3899 :invoke :write [9 10] INFO [2022-01-10 05:46:36,636] jepsen worker 83 - jepsen.util 4403 :ok :read [27 30] INFO [2022-01-10 05:46:36,636] jepsen worker 83 - jepsen.util 4403 :invoke :read [27 nil] INFO [2022-01-10 05:46:36,728] jepsen worker 87 - jepsen.util 3777 :ok :read [29 0] INFO [2022-01-10 05:46:36,729] jepsen worker 87 - jepsen.util 3777 :invoke :write [29 15] INFO [2022-01-10 05:46:36,735] jepsen worker 4 - jepsen.util 4504 :ok :read [1 26] INFO [2022-01-10 05:46:36,736] jepsen worker 4 - jepsen.util 4504 :invoke :write [1 31] INFO [2022-01-10 05:46:36,856] jepsen worker 18 - jepsen.util 3798 :info :write [6 47] :RequestTimeoutException INFO [2022-01-10 05:46:36,857] jepsen worker 18 - jepsen.util 3888 :invoke :write [6 13] INFO [2022-01-10 05:46:36,872] jepsen worker 83 - jepsen.util 4403 :ok :read [27 30] INFO [2022-01-10 05:46:36,873] jepsen worker 83 - jepsen.util 4403 :invoke :write [27 45] INFO [2022-01-10 05:46:36,922] jepsen worker 63 - jepsen.util 3753 :info :write [21 27] :RequestTimeoutException INFO [2022-01-10 05:46:36,923] jepsen worker 63 - jepsen.util 3843 :invoke :read [21 nil] INFO [2022-01-10 05:46:37,044] jepsen worker 43 - jepsen.util 3823 :ok :read [14 14] INFO [2022-01-10 05:46:37,044] jepsen worker 43 - jepsen.util 3823 :invoke :read [14 nil] INFO [2022-01-10 05:46:37,185] jepsen worker 51 - jepsen.util 4731 :ok :read [17 41] INFO [2022-01-10 05:46:37,191] jepsen worker 51 - jepsen.util 4731 :invoke :read [17 nil] INFO [2022-01-10 05:46:37,247] jepsen worker 67 - jepsen.util 3667 :info :write [22 47] :RequestTimeoutException INFO [2022-01-10 05:46:37,247] jepsen worker 67 - jepsen.util 3757 :invoke :write [22 33] INFO [2022-01-10 05:46:37,256] jepsen worker 57 - jepsen.util 4647 :info :write [19 26] :RequestTimeoutException INFO [2022-01-10 05:46:37,257] jepsen worker 57 - jepsen.util 4737 :invoke :write [19 14] INFO [2022-01-10 05:46:37,437] jepsen worker 3 - jepsen.util 3693 :info :write [1 10] :RequestTimeoutException INFO [2022-01-10 05:46:37,437] jepsen worker 3 - jepsen.util 3783 :invoke :read [1 nil] INFO [2022-01-10 05:46:37,451] jepsen worker 9 - jepsen.util 4329 :info :write [3 23] :RequestTimeoutException INFO [2022-01-10 05:46:37,452] jepsen worker 9 - jepsen.util 4419 :invoke :write [3 28] INFO [2022-01-10 05:46:37,491] jepsen worker 86 - jepsen.util 3326 :info :write [28 23] :RequestTimeoutException INFO [2022-01-10 05:46:37,491] jepsen worker 86 - jepsen.util 3416 :invoke :read [28 nil] INFO [2022-01-10 05:46:37,544] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:37,544] jepsen worker nemesis - couchbase.util Rebalance check count: 546 INFO [2022-01-10 05:46:37,544] jepsen worker nemesis - couchbase.util Rebalance stuck count: 456 INFO [2022-01-10 05:46:37,570] jepsen worker 48 - jepsen.util 3558 :info :write [16 33] :RequestTimeoutException INFO [2022-01-10 05:46:37,571] jepsen worker 48 - jepsen.util 3648 :invoke :read [16 nil] INFO [2022-01-10 05:46:37,586] jepsen worker 84 - jepsen.util 3774 :info :write [28 34] :RequestTimeoutException INFO [2022-01-10 05:46:37,586] jepsen worker 84 - jepsen.util 3864 :invoke :write [28 18] INFO [2022-01-10 05:46:37,656] jepsen worker 35 - jepsen.util 3635 :info :write [11 27] :RequestTimeoutException INFO [2022-01-10 05:46:37,657] jepsen worker 35 - jepsen.util 3725 :invoke :write [11 10] INFO [2022-01-10 05:46:37,705] jepsen worker 43 - jepsen.util 3823 :ok :read [14 14] INFO [2022-01-10 05:46:37,705] jepsen worker 43 - jepsen.util 3823 :invoke :read [14 nil] INFO [2022-01-10 05:46:37,740] jepsen worker 23 - jepsen.util 3983 :info :write [7 23] :RequestTimeoutException INFO [2022-01-10 05:46:37,740] jepsen worker 23 - jepsen.util 4073 :invoke :read [7 nil] INFO [2022-01-10 05:46:37,744] jepsen worker 51 - jepsen.util 4731 :ok :read [17 41] INFO [2022-01-10 05:46:37,745] jepsen worker 51 - jepsen.util 4731 :invoke :write [17 24] INFO [2022-01-10 05:46:37,886] jepsen worker 58 - jepsen.util 3568 :info :write [19 38] :RequestTimeoutException INFO [2022-01-10 05:46:37,887] jepsen worker 58 - jepsen.util 3658 :invoke :write [19 37] INFO [2022-01-10 05:46:37,947] jepsen worker 19 - jepsen.util 4069 :info :write [6 1] :RequestTimeoutException INFO [2022-01-10 05:46:37,948] jepsen worker 19 - jepsen.util 4159 :invoke :read [6 nil] INFO [2022-01-10 05:46:37,951] jepsen worker 73 - jepsen.util 3493 :info :write [24 25] :RequestTimeoutException INFO [2022-01-10 05:46:37,952] jepsen worker 73 - jepsen.util 3583 :invoke :write [24 5] INFO [2022-01-10 05:46:37,986] jepsen worker 62 - jepsen.util 4292 :info :write [20 28] :RequestTimeoutException INFO [2022-01-10 05:46:37,987] jepsen worker 62 - jepsen.util 4382 :invoke :write [20 37] INFO [2022-01-10 05:46:37,991] jepsen worker 77 - jepsen.util 4307 :info :write [25 42] :RequestTimeoutException INFO [2022-01-10 05:46:37,992] jepsen worker 77 - jepsen.util 4397 :invoke :read [25 nil] INFO [2022-01-10 05:46:38,171] jepsen worker 43 - jepsen.util 3823 :ok :read [14 14] INFO [2022-01-10 05:46:38,172] jepsen worker 43 - jepsen.util 3823 :invoke :read [14 nil] INFO [2022-01-10 05:46:38,196] jepsen worker 74 - jepsen.util 4394 :info :write [24 7] :RequestTimeoutException INFO [2022-01-10 05:46:38,196] jepsen worker 74 - jepsen.util 4484 :invoke :write [24 7] INFO [2022-01-10 05:46:38,332] jepsen worker 61 - jepsen.util 4111 :ok :read [20 :nil] INFO [2022-01-10 05:46:38,333] jepsen worker 61 - jepsen.util 4111 :invoke :write [20 33] INFO [2022-01-10 05:46:38,340] jepsen worker 82 - jepsen.util 4042 :ok :read [27 30] INFO [2022-01-10 05:46:38,340] jepsen worker 82 - jepsen.util 4042 :invoke :read [27 nil] INFO [2022-01-10 05:46:38,346] jepsen worker 81 - jepsen.util 4311 :ok :read [27 30] INFO [2022-01-10 05:46:38,347] jepsen worker 81 - jepsen.util 4311 :invoke :read [27 nil] INFO [2022-01-10 05:46:38,356] jepsen worker 59 - jepsen.util 3929 :info :write [19 28] :RequestTimeoutException INFO [2022-01-10 05:46:38,357] jepsen worker 59 - jepsen.util 4019 :invoke :write [19 25] INFO [2022-01-10 05:46:38,408] jepsen worker 41 - jepsen.util 3821 :ok :read [13 15] INFO [2022-01-10 05:46:38,409] jepsen worker 41 - jepsen.util 3821 :invoke :read [13 nil] INFO [2022-01-10 05:46:38,424] jepsen worker 82 - jepsen.util 4042 :ok :read [27 30] INFO [2022-01-10 05:46:38,425] jepsen worker 82 - jepsen.util 4042 :invoke :read [27 nil] INFO [2022-01-10 05:46:38,469] jepsen worker 81 - jepsen.util 4311 :ok :read [27 30] INFO [2022-01-10 05:46:38,470] jepsen worker 81 - jepsen.util 4311 :invoke :read [27 nil] INFO [2022-01-10 05:46:38,472] jepsen worker 41 - jepsen.util 3821 :ok :read [13 15] INFO [2022-01-10 05:46:38,472] jepsen worker 41 - jepsen.util 3821 :invoke :write [13 18] INFO [2022-01-10 05:46:38,478] jepsen worker 82 - jepsen.util 4042 :ok :read [27 30] INFO [2022-01-10 05:46:38,479] jepsen worker 82 - jepsen.util 4042 :invoke :write [27 40] INFO [2022-01-10 05:46:38,526] jepsen worker 81 - jepsen.util 4311 :ok :read [27 30] INFO [2022-01-10 05:46:38,528] jepsen worker 81 - jepsen.util 4311 :invoke :read [27 nil] INFO [2022-01-10 05:46:38,551] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:38,551] jepsen worker nemesis - couchbase.util Rebalance check count: 547 INFO [2022-01-10 05:46:38,551] jepsen worker nemesis - couchbase.util Rebalance stuck count: 457 INFO [2022-01-10 05:46:38,552] jepsen worker 43 - jepsen.util 3823 :ok :read [14 14] INFO [2022-01-10 05:46:38,553] jepsen worker 43 - jepsen.util 3823 :invoke :read [14 nil] INFO [2022-01-10 05:46:38,716] jepsen worker 77 - jepsen.util 4397 :ok :read [25 :nil] INFO [2022-01-10 05:46:38,716] jepsen worker 77 - jepsen.util 4397 :invoke :read [25 nil] INFO [2022-01-10 05:46:38,928] jepsen worker 43 - jepsen.util 3823 :ok :read [14 14] INFO [2022-01-10 05:46:38,928] jepsen worker 43 - jepsen.util 3823 :invoke :read [14 nil] INFO [2022-01-10 05:46:38,981] jepsen worker 60 - jepsen.util 4110 :info :write [20 21] :RequestTimeoutException INFO [2022-01-10 05:46:38,981] jepsen worker 60 - jepsen.util 4200 :invoke :read [20 nil] INFO [2022-01-10 05:46:38,998] jepsen worker 42 - jepsen.util 3642 :ok :read [14 14] INFO [2022-01-10 05:46:38,999] jepsen worker 42 - jepsen.util 3642 :invoke :write [14 32] INFO [2022-01-10 05:46:39,001] jepsen worker 72 - jepsen.util 4302 :info :write [24 10] :RequestTimeoutException INFO [2022-01-10 05:46:39,002] jepsen worker 72 - jepsen.util 4392 :invoke :read [24 nil] INFO [2022-01-10 05:46:39,195] jepsen worker 43 - jepsen.util 3823 :ok :read [14 14] INFO [2022-01-10 05:46:39,196] jepsen worker 43 - jepsen.util 3823 :invoke :write [14 49] INFO [2022-01-10 05:46:39,411] jepsen worker 66 - jepsen.util 3666 :ok :read [22 41] INFO [2022-01-10 05:46:39,412] jepsen worker 66 - jepsen.util 3666 :invoke :read [22 nil] INFO [2022-01-10 05:46:39,431] jepsen worker 50 - jepsen.util 4100 :ok :read [16 38] INFO [2022-01-10 05:46:39,432] jepsen worker 50 - jepsen.util 4100 :invoke :read [16 nil] INFO [2022-01-10 05:46:39,434] jepsen worker 48 - jepsen.util 3648 :ok :read [16 38] INFO [2022-01-10 05:46:39,435] jepsen worker 48 - jepsen.util 3648 :invoke :read [16 nil] INFO [2022-01-10 05:46:39,459] jepsen worker 66 - jepsen.util 3666 :ok :read [22 41] INFO [2022-01-10 05:46:39,459] jepsen worker 66 - jepsen.util 3666 :invoke :read [22 nil] INFO [2022-01-10 05:46:39,466] jepsen worker 16 - jepsen.util 3976 :info :write [5 40] :RequestTimeoutException INFO [2022-01-10 05:46:39,466] jepsen worker 16 - jepsen.util 4066 :invoke :read [5 nil] INFO [2022-01-10 05:46:39,487] jepsen worker 48 - jepsen.util 3648 :ok :read [16 38] INFO [2022-01-10 05:46:39,494] jepsen worker 48 - jepsen.util 3648 :invoke :write [16 15] INFO [2022-01-10 05:46:39,509] jepsen worker 66 - jepsen.util 3666 :ok :read [22 41] INFO [2022-01-10 05:46:39,509] jepsen worker 66 - jepsen.util 3666 :invoke :read [22 nil] INFO [2022-01-10 05:46:39,538] jepsen worker 66 - jepsen.util 3666 :ok :read [22 41] INFO [2022-01-10 05:46:39,539] jepsen worker 66 - jepsen.util 3666 :invoke :read [22 nil] INFO [2022-01-10 05:46:39,557] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:39,557] jepsen worker nemesis - couchbase.util Rebalance check count: 548 INFO [2022-01-10 05:46:39,557] jepsen worker nemesis - couchbase.util Rebalance stuck count: 458 INFO [2022-01-10 05:46:39,591] jepsen worker 13 - jepsen.util 4513 :info :write [4 41] :RequestTimeoutException INFO [2022-01-10 05:46:39,592] jepsen worker 13 - jepsen.util 4603 :invoke :read [4 nil] INFO [2022-01-10 05:46:39,626] jepsen worker 13 - jepsen.util 4603 :ok :read [4 3] INFO [2022-01-10 05:46:39,627] jepsen worker 13 - jepsen.util 4603 :invoke :write [4 11] INFO [2022-01-10 05:46:39,825] jepsen worker 85 - jepsen.util 4225 :info :write [28 30] :RequestTimeoutException INFO [2022-01-10 05:46:39,825] jepsen worker 85 - jepsen.util 4315 :invoke :read [28 nil] INFO [2022-01-10 05:46:39,826] jepsen worker 65 - jepsen.util 3665 :info :write [21 41] :RequestTimeoutException INFO [2022-01-10 05:46:39,826] jepsen worker 65 - jepsen.util 3755 :invoke :write [21 21] INFO [2022-01-10 05:46:39,994] jepsen worker 77 - jepsen.util 4397 :ok :read [25 :nil] INFO [2022-01-10 05:46:39,995] jepsen worker 77 - jepsen.util 4397 :invoke :write [25 26] INFO [2022-01-10 05:46:40,079] jepsen worker 89 - jepsen.util 4409 :info :write [29 20] :RequestTimeoutException INFO [2022-01-10 05:46:40,080] jepsen worker 89 - jepsen.util 4499 :invoke :read [29 nil] INFO [2022-01-10 05:46:40,091] jepsen worker 27 - jepsen.util 4077 :info :write [9 33] :RequestTimeoutException INFO [2022-01-10 05:46:40,092] jepsen worker 27 - jepsen.util 4167 :invoke :read [9 nil] INFO [2022-01-10 05:46:40,205] jepsen worker 24 - jepsen.util 3534 :info :write [8 13] :RequestTimeoutException INFO [2022-01-10 05:46:40,206] jepsen worker 24 - jepsen.util 3624 :invoke :write [8 15] INFO [2022-01-10 05:46:40,225] jepsen worker 46 - jepsen.util 4006 :info :write [15 37] :RequestTimeoutException INFO [2022-01-10 05:46:40,225] jepsen worker 46 - jepsen.util 4096 :invoke :write [15 41] INFO [2022-01-10 05:46:40,341] jepsen worker 22 - jepsen.util 3622 :info :write [7 22] :RequestTimeoutException INFO [2022-01-10 05:46:40,341] jepsen worker 22 - jepsen.util 3712 :invoke :write [7 46] INFO [2022-01-10 05:46:40,375] jepsen worker 6 - jepsen.util 3966 :info :write [2 39] :RequestTimeoutException INFO [2022-01-10 05:46:40,375] jepsen worker 6 - jepsen.util 4056 :invoke :read [2 nil] INFO [2022-01-10 05:46:40,486] jepsen worker 7 - jepsen.util 4057 :info :write [2 16] :RequestTimeoutException INFO [2022-01-10 05:46:40,486] jepsen worker 7 - jepsen.util 4147 :invoke :read [2 nil] INFO [2022-01-10 05:46:40,503] jepsen worker 81 - jepsen.util 4311 :ok :read [27 30] INFO [2022-01-10 05:46:40,503] jepsen worker 81 - jepsen.util 4311 :invoke :read [27 nil] WARN [2022-01-10 05:46:40,526] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18504781,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18504781,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9192889,"encode_duration_us":18041,"operation_id":"0x780b","timeout_ms":10000,"last_server_duration_us":9192889,"total_duration_us":10004327},{"operation_name":"upsert","last_dispatch_duration_us":9528140,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15458917,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":10501624,"encode_duration_us":17434,"operation_id":"0x77fe","timeout_ms":10000,"last_server_duration_us":9162975,"total_duration_us":10005712},{"operation_name":"upsert","last_dispatch_duration_us":17171889,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18228578,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9719178,"encode_duration_us":16983,"operation_id":"0x7797","timeout_ms":10000,"last_server_duration_us":9545300,"total_duration_us":10005639}],"total_count":3}} WARN [2022-01-10 05:46:40,526] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9383222,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9383222,"total_server_duration_us":19418,"encode_duration_us":16442,"operation_id":"0x78f5","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10007880},{"operation_name":"upsert","last_dispatch_duration_us":9626837,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9626837,"total_server_duration_us":197705,"encode_duration_us":16307,"operation_id":"0x7927","timeout_ms":10000,"last_server_duration_us":197705,"total_duration_us":10006955},{"operation_name":"upsert","last_dispatch_duration_us":200665,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":200665,"total_server_duration_us":112311,"encode_duration_us":16214,"operation_id":"0x786a","timeout_ms":10000,"last_server_duration_us":112311,"total_duration_us":10006030},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":18507,"operation_id":"0x79f5","timeout_ms":10000,"total_duration_us":10005436},{"operation_name":"upsert","last_dispatch_duration_us":860699,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":2862047,"total_server_duration_us":314907,"encode_duration_us":16519,"operation_id":"0x7896","timeout_ms":10000,"last_server_duration_us":28952,"total_duration_us":10004465},{"operation_name":"upsert","last_dispatch_duration_us":18504781,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":18504781,"total_server_duration_us":9192889,"encode_duration_us":18041,"operation_id":"0x780b","timeout_ms":10000,"last_server_duration_us":9192889,"total_duration_us":10004327},{"operation_name":"upsert","last_dispatch_duration_us":9083984,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9083984,"total_server_duration_us":299493,"encode_duration_us":17861,"operation_id":"0x7a0c","timeout_ms":10000,"last_server_duration_us":299493,"total_duration_us":10002923},{"operation_name":"upsert","last_dispatch_duration_us":9725177,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9725177,"total_server_duration_us":51121,"encode_duration_us":20734,"operation_id":"0x7a12","timeout_ms":10000,"last_server_duration_us":51121,"total_duration_us":10001440},{"operation_name":"upsert","last_dispatch_duration_us":9746438,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9746438,"total_server_duration_us":719387,"encode_duration_us":18757,"operation_id":"0x7a0e","timeout_ms":10000,"last_server_duration_us":719387,"total_duration_us":10001023},{"operation_name":"upsert","last_dispatch_duration_us":9568224,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9568224,"total_server_duration_us":9405602,"encode_duration_us":17892,"operation_id":"0x794d","timeout_ms":10000,"last_server_duration_us":9405602,"total_duration_us":9572337}],"total_count":13}} INFO [2022-01-10 05:46:40,563] jepsen worker 66 - jepsen.util 3666 :ok :read [22 41] INFO [2022-01-10 05:46:40,564] jepsen worker 66 - jepsen.util 3666 :invoke :write [22 26] INFO [2022-01-10 05:46:40,565] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:40,565] jepsen worker nemesis - couchbase.util Rebalance check count: 549 INFO [2022-01-10 05:46:40,565] jepsen worker nemesis - couchbase.util Rebalance stuck count: 459 INFO [2022-01-10 05:46:40,592] jepsen worker 27 - jepsen.util 4167 :ok :read [9 27] INFO [2022-01-10 05:46:40,593] jepsen worker 27 - jepsen.util 4167 :invoke :write [9 1] INFO [2022-01-10 05:46:40,719] jepsen worker 49 - jepsen.util 3829 :ok :read [16 38] INFO [2022-01-10 05:46:40,720] jepsen worker 49 - jepsen.util 3829 :invoke :write [16 24] INFO [2022-01-10 05:46:40,725] jepsen worker 81 - jepsen.util 4311 :ok :read [27 30] INFO [2022-01-10 05:46:40,726] jepsen worker 81 - jepsen.util 4311 :invoke :write [27 24] INFO [2022-01-10 05:46:40,743] jepsen worker 85 - jepsen.util 4315 :ok :read [28 45] INFO [2022-01-10 05:46:40,744] jepsen worker 85 - jepsen.util 4315 :invoke :read [28 nil] INFO [2022-01-10 05:46:41,000] jepsen worker 64 - jepsen.util 3934 :info :write [21 1] :RequestTimeoutException INFO [2022-01-10 05:46:41,000] jepsen worker 64 - jepsen.util 4024 :invoke :read [21 nil] INFO [2022-01-10 05:46:41,062] jepsen worker 80 - jepsen.util 4760 :info :write [26 1] :RequestTimeoutException INFO [2022-01-10 05:46:41,063] jepsen worker 80 - jepsen.util 4850 :invoke :write [26 5] INFO [2022-01-10 05:46:41,125] jepsen worker 6 - jepsen.util 4056 :ok :read [2 14] INFO [2022-01-10 05:46:41,126] jepsen worker 6 - jepsen.util 4056 :invoke :read [2 nil] INFO [2022-01-10 05:46:41,212] jepsen worker 52 - jepsen.util 4912 :info :write [17 21] :RequestTimeoutException INFO [2022-01-10 05:46:41,213] jepsen worker 52 - jepsen.util 5002 :invoke :write [17 49] INFO [2022-01-10 05:46:41,220] jepsen worker 53 - jepsen.util 4733 :info :write [17 49] :RequestTimeoutException INFO [2022-01-10 05:46:41,220] jepsen worker 53 - jepsen.util 4823 :invoke :read [17 nil] INFO [2022-01-10 05:46:41,352] jepsen worker 32 - jepsen.util 4172 :ok :read [10 36] INFO [2022-01-10 05:46:41,352] jepsen worker 32 - jepsen.util 4172 :invoke :write [10 27] INFO [2022-01-10 05:46:41,469] jepsen worker 6 - jepsen.util 4056 :ok :read [2 14] INFO [2022-01-10 05:46:41,470] jepsen worker 79 - jepsen.util 4579 :info :write [26 46] :RequestTimeoutException INFO [2022-01-10 05:46:41,471] jepsen worker 6 - jepsen.util 4056 :invoke :read [2 nil] INFO [2022-01-10 05:46:41,472] jepsen worker 79 - jepsen.util 4669 :invoke :write [26 10] INFO [2022-01-10 05:46:41,551] jepsen worker 14 - jepsen.util 4784 :info :write [4 19] :RequestTimeoutException INFO [2022-01-10 05:46:41,552] jepsen worker 14 - jepsen.util 4874 :invoke :write [4 28] INFO [2022-01-10 05:46:41,572] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:41,572] jepsen worker nemesis - couchbase.util Rebalance check count: 550 INFO [2022-01-10 05:46:41,572] jepsen worker nemesis - couchbase.util Rebalance stuck count: 460 INFO [2022-01-10 05:46:41,578] jepsen worker 50 - jepsen.util 4100 :ok :read [16 38] INFO [2022-01-10 05:46:41,578] jepsen worker 50 - jepsen.util 4100 :invoke :write [16 26] INFO [2022-01-10 05:46:41,811] jepsen worker 6 - jepsen.util 4056 :ok :read [2 14] INFO [2022-01-10 05:46:41,811] jepsen worker 6 - jepsen.util 4056 :invoke :read [2 nil] INFO [2022-01-10 05:46:41,845] jepsen worker 5 - jepsen.util 3695 :info :write [1 15] :SyncWriteAmbiguous INFO [2022-01-10 05:46:41,846] jepsen worker 5 - jepsen.util 3785 :invoke :write [1 47] INFO [2022-01-10 05:46:41,896] jepsen worker 2 - jepsen.util 4142 :info :write [0 26] :RequestTimeoutException INFO [2022-01-10 05:46:41,896] jepsen worker 2 - jepsen.util 4232 :invoke :write [0 8] INFO [2022-01-10 05:46:41,945] jepsen worker 31 - jepsen.util 4261 :ok :read [10 36] INFO [2022-01-10 05:46:41,946] jepsen worker 31 - jepsen.util 4261 :invoke :write [10 26] INFO [2022-01-10 05:46:41,960] jepsen worker 3 - jepsen.util 3783 :ok :read [1 26] INFO [2022-01-10 05:46:41,960] jepsen worker 3 - jepsen.util 3783 :invoke :write [1 3] INFO [2022-01-10 05:46:41,986] jepsen worker 39 - jepsen.util 4449 :info :write [13 13] :RequestTimeoutException INFO [2022-01-10 05:46:41,987] jepsen worker 39 - jepsen.util 4539 :invoke :read [13 nil] WARN [2022-01-10 05:46:42,023] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10856632,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13953457,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9764014,"encode_duration_us":16848,"operation_id":"0x7798","timeout_ms":10000,"last_server_duration_us":9515989,"total_duration_us":10003270},{"operation_name":"upsert","last_dispatch_duration_us":11154405,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16232732,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":10551107,"encode_duration_us":20246,"operation_id":"0x7799","timeout_ms":10000,"last_server_duration_us":9869152,"total_duration_us":10009591},{"operation_name":"upsert","last_dispatch_duration_us":10625050,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17219545,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9644430,"encode_duration_us":17727,"operation_id":"0x7864","timeout_ms":10000,"last_server_duration_us":9319408,"total_duration_us":10010715}],"total_count":3}} WARN [2022-01-10 05:46:42,110] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":998820,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7759776,"total_server_duration_us":913791,"encode_duration_us":17114,"operation_id":"0x7865","timeout_ms":10000,"last_server_duration_us":118289,"total_duration_us":10010110},{"operation_name":"upsert","last_dispatch_duration_us":9054359,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9661818,"total_server_duration_us":9399344,"encode_duration_us":17835,"operation_id":"0x7965","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":10009919},{"operation_name":"upsert","last_dispatch_duration_us":6983287,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":7971288,"total_server_duration_us":578585,"encode_duration_us":19239,"operation_id":"0x78ee","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10009522},{"operation_name":"upsert","last_dispatch_duration_us":909532,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3381809,"total_server_duration_us":735780,"encode_duration_us":22911,"operation_id":"0x7992","timeout_ms":10000,"last_server_duration_us":116949,"total_duration_us":10008948},{"operation_name":"upsert","last_dispatch_duration_us":1275581,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":4538709,"total_server_duration_us":2803229,"encode_duration_us":18925,"operation_id":"0x7a52","timeout_ms":10000,"last_server_duration_us":1035212,"total_duration_us":10008732},{"operation_name":"upsert","last_dispatch_duration_us":8773517,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11842831,"total_server_duration_us":208432,"encode_duration_us":16429,"operation_id":"0x7966","timeout_ms":10000,"last_server_duration_us":40482,"total_duration_us":10008150},{"operation_name":"upsert","last_dispatch_duration_us":1675892,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4677729,"total_server_duration_us":442575,"encode_duration_us":16194,"operation_id":"0x7994","timeout_ms":10000,"last_server_duration_us":106626,"total_duration_us":10007459},{"operation_name":"upsert","last_dispatch_duration_us":1326922,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3625329,"total_server_duration_us":366870,"encode_duration_us":18279,"operation_id":"0x78fa","timeout_ms":10000,"last_server_duration_us":75616,"total_duration_us":10006438},{"operation_name":"upsert","last_dispatch_duration_us":1080325,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1961017,"total_server_duration_us":404504,"encode_duration_us":19299,"operation_id":"0x79c2","timeout_ms":10000,"last_server_duration_us":271897,"total_duration_us":10004095},{"operation_name":"upsert","last_dispatch_duration_us":995438,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5153351,"total_server_duration_us":420983,"encode_duration_us":18027,"operation_id":"0x78cc","timeout_ms":10000,"last_server_duration_us":78414,"total_duration_us":10003744}],"total_count":24}} WARN [2022-01-10 05:46:42,111] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10188656,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10188656,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9242122,"encode_duration_us":16520,"operation_id":"0x7a11","timeout_ms":10000,"last_server_duration_us":9242122,"total_duration_us":10001810},{"operation_name":"upsert","last_dispatch_duration_us":9054359,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9661818,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9399344,"encode_duration_us":17835,"operation_id":"0x7965","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":10009919},{"operation_name":"upsert","last_dispatch_duration_us":10103133,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16702341,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10015583,"encode_duration_us":16334,"operation_id":"0x7869","timeout_ms":10000,"last_server_duration_us":9169382,"total_duration_us":10002337}],"total_count":3}} INFO [2022-01-10 05:46:42,146] jepsen worker 75 - jepsen.util 3585 :info :write [25 18] :RequestTimeoutException INFO [2022-01-10 05:46:42,146] jepsen worker 75 - jepsen.util 3675 :invoke :read [25 nil] WARN [2022-01-10 05:46:42,215] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10482486,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14334026,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9116964,"encode_duration_us":16909,"operation_id":"0x7866","timeout_ms":10000,"last_server_duration_us":9052231,"total_duration_us":10005230},{"operation_name":"upsert","last_dispatch_duration_us":9075597,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9665299,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9365738,"encode_duration_us":25337,"operation_id":"0x7a00","timeout_ms":10000,"last_server_duration_us":9062854,"total_duration_us":10006949}],"total_count":2}} WARN [2022-01-10 05:46:42,216] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":18346,"operation_id":"0x7a73","timeout_ms":10000,"total_duration_us":10009630},{"operation_name":"upsert","last_dispatch_duration_us":631973,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1340369,"total_server_duration_us":140564,"encode_duration_us":16445,"operation_id":"0x794c","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10009003},{"operation_name":"upsert","last_dispatch_duration_us":9577429,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":12468993,"total_server_duration_us":478455,"encode_duration_us":19231,"operation_id":"0x799d","timeout_ms":10000,"last_server_duration_us":58750,"total_duration_us":10008312},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":17231,"operation_id":"0x7a77","timeout_ms":10000,"total_duration_us":10007995},{"operation_name":"upsert","last_dispatch_duration_us":9793396,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11514573,"total_server_duration_us":9251793,"encode_duration_us":19920,"operation_id":"0x786b","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10006821},{"operation_name":"upsert","last_dispatch_duration_us":860502,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1680164,"total_server_duration_us":239702,"encode_duration_us":18764,"operation_id":"0x7993","timeout_ms":10000,"last_server_duration_us":102484,"total_duration_us":10005815},{"operation_name":"upsert","last_dispatch_duration_us":8692813,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":13007131,"total_server_duration_us":446001,"encode_duration_us":16631,"operation_id":"0x789a","timeout_ms":10000,"last_server_duration_us":25101,"total_duration_us":10005613},{"operation_name":"upsert","last_dispatch_duration_us":652420,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":2355526,"total_server_duration_us":186996,"encode_duration_us":18404,"operation_id":"0x79cb","timeout_ms":10000,"last_server_duration_us":78273,"total_duration_us":10004471},{"operation_name":"upsert","last_dispatch_duration_us":10265497,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":13194417,"total_server_duration_us":9616771,"encode_duration_us":16383,"operation_id":"0x798f","timeout_ms":10000,"last_server_duration_us":9432607,"total_duration_us":10003853},{"operation_name":"upsert","last_dispatch_duration_us":250659,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10102494,"total_server_duration_us":383153,"encode_duration_us":20847,"operation_id":"0x78f9","timeout_ms":10000,"last_server_duration_us":183112,"total_duration_us":10003564}],"total_count":24}} WARN [2022-01-10 05:46:42,224] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10625050,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":17219545,"total_server_duration_us":9644430,"encode_duration_us":17727,"operation_id":"0x7864","timeout_ms":10000,"last_server_duration_us":9319408,"total_duration_us":10010715},{"operation_name":"upsert","last_dispatch_duration_us":1284345,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10171688,"total_server_duration_us":117727,"encode_duration_us":17474,"operation_id":"0x7991","timeout_ms":10000,"last_server_duration_us":53254,"total_duration_us":10009981},{"operation_name":"upsert","last_dispatch_duration_us":938988,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9760805,"total_server_duration_us":120164,"encode_duration_us":17777,"operation_id":"0x799b","timeout_ms":10000,"last_server_duration_us":62284,"total_duration_us":10009524},{"operation_name":"upsert","last_dispatch_duration_us":898568,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2242998,"total_server_duration_us":71128,"encode_duration_us":17744,"operation_id":"0x78f6","timeout_ms":10000,"last_server_duration_us":51710,"total_duration_us":10009327},{"operation_name":"upsert","last_dispatch_duration_us":970985,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10657682,"total_server_duration_us":166954,"encode_duration_us":17789,"operation_id":"0x796e","timeout_ms":10000,"last_server_duration_us":62668,"total_duration_us":10008683},{"operation_name":"upsert","last_dispatch_duration_us":533532,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":10123095,"total_server_duration_us":366467,"encode_duration_us":20591,"operation_id":"0x789b","timeout_ms":10000,"last_server_duration_us":94259,"total_duration_us":10008186},{"operation_name":"upsert","last_dispatch_duration_us":9851452,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":16094805,"total_server_duration_us":303979,"encode_duration_us":23538,"operation_id":"0x78cb","timeout_ms":10000,"last_server_duration_us":22637,"total_duration_us":10003506},{"operation_name":"upsert","last_dispatch_duration_us":9663157,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":17707820,"total_server_duration_us":358107,"encode_duration_us":16796,"operation_id":"0x788b","timeout_ms":10000,"last_server_duration_us":36423,"total_duration_us":10003061},{"operation_name":"upsert","last_dispatch_duration_us":1205308,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2486078,"total_server_duration_us":127709,"encode_duration_us":17643,"operation_id":"0x7a15","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10001627},{"operation_name":"upsert","last_dispatch_duration_us":908230,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2783711,"total_server_duration_us":290925,"encode_duration_us":16733,"operation_id":"0x79f6","timeout_ms":10000,"last_server_duration_us":110511,"total_duration_us":10001018}],"total_count":18}} WARN [2022-01-10 05:46:42,224] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":141875,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":7504378,"total_server_duration_us":290158,"encode_duration_us":19101,"operation_id":"0x78c2","timeout_ms":10000,"last_server_duration_us":22887,"total_duration_us":10010496},{"operation_name":"upsert","last_dispatch_duration_us":1212777,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6933795,"total_server_duration_us":1580033,"encode_duration_us":17010,"operation_id":"0x794e","timeout_ms":10000,"last_server_duration_us":97174,"total_duration_us":10010111},{"operation_name":"upsert","last_dispatch_duration_us":33631,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":7399519,"total_server_duration_us":3198790,"encode_duration_us":24598,"operation_id":"0x7a4a","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10009893},{"operation_name":"upsert","last_dispatch_duration_us":10549207,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10549207,"total_server_duration_us":204648,"encode_duration_us":17114,"operation_id":"0x78f8","timeout_ms":10000,"last_server_duration_us":204648,"total_duration_us":10007756},{"operation_name":"upsert","last_dispatch_duration_us":60861,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":7390577,"total_server_duration_us":2096103,"encode_duration_us":24882,"operation_id":"0x7a3c","timeout_ms":10000,"last_server_duration_us":37859,"total_duration_us":10004971},{"operation_name":"upsert","last_dispatch_duration_us":440857,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":9992903,"total_server_duration_us":505814,"encode_duration_us":19275,"operation_id":"0x7867","timeout_ms":10000,"last_server_duration_us":87064,"total_duration_us":10003944},{"operation_name":"upsert","last_dispatch_duration_us":6107424,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":6107424,"total_server_duration_us":97329,"encode_duration_us":30687,"operation_id":"0x7a02","timeout_ms":10000,"last_server_duration_us":97329,"total_duration_us":10003443},{"operation_name":"upsert","last_dispatch_duration_us":10414754,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":12704646,"total_server_duration_us":502557,"encode_duration_us":16936,"operation_id":"0x788d","timeout_ms":10000,"last_server_duration_us":66168,"total_duration_us":10003357},{"operation_name":"upsert","last_dispatch_duration_us":1095610,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":9873907,"total_server_duration_us":871397,"encode_duration_us":16417,"operation_id":"0x78ec","timeout_ms":10000,"last_server_duration_us":178888,"total_duration_us":10002256},{"operation_name":"upsert","last_dispatch_duration_us":1695484,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":10354168,"total_server_duration_us":314761,"encode_duration_us":17936,"operation_id":"0x791d","timeout_ms":10000,"last_server_duration_us":81111,"total_duration_us":10001275}],"total_count":23}} INFO [2022-01-10 05:46:42,252] jepsen worker 44 - jepsen.util 4004 :info :write [14 40] :RequestTimeoutException INFO [2022-01-10 05:46:42,253] jepsen worker 44 - jepsen.util 4094 :invoke :write [14 15] INFO [2022-01-10 05:46:42,276] jepsen worker 0 - jepsen.util 3780 :info :write [0 34] :RequestTimeoutException INFO [2022-01-10 05:46:42,276] jepsen worker 0 - jepsen.util 3870 :invoke :write [0 43] WARN [2022-01-10 05:46:42,316] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10154973,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13074523,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9212468,"encode_duration_us":16807,"operation_id":"0x796b","timeout_ms":10000,"last_server_duration_us":9102208,"total_duration_us":10001491},{"operation_name":"upsert","last_dispatch_duration_us":9793396,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11514573,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9251793,"encode_duration_us":19920,"operation_id":"0x786b","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10006821},{"operation_name":"upsert","last_dispatch_duration_us":10265497,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13194417,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9616771,"encode_duration_us":16383,"operation_id":"0x798f","timeout_ms":10000,"last_server_duration_us":9432607,"total_duration_us":10003853}],"total_count":3}} WARN [2022-01-10 05:46:42,325] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11417829,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16164505,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10088213,"encode_duration_us":28223,"operation_id":"0x77d7","timeout_ms":10000,"last_server_duration_us":9440174,"total_duration_us":10007481}],"total_count":1}} INFO [2022-01-10 05:46:42,409] jepsen worker 33 - jepsen.util 4083 :info :write [11 25] :RequestTimeoutException INFO [2022-01-10 05:46:42,410] jepsen worker 33 - jepsen.util 4173 :invoke :read [11 nil] WARN [2022-01-10 05:46:42,415] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7942784,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11317858,"total_server_duration_us":282943,"encode_duration_us":17381,"operation_id":"0x791c","timeout_ms":10000,"last_server_duration_us":118289,"total_duration_us":10009344},{"operation_name":"upsert","last_dispatch_duration_us":8872697,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10551564,"total_server_duration_us":351454,"encode_duration_us":37791,"operation_id":"0x7969","timeout_ms":10000,"last_server_duration_us":235006,"total_duration_us":10008897},{"operation_name":"upsert","last_dispatch_duration_us":594417,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":5659134,"total_server_duration_us":1686477,"encode_duration_us":17330,"operation_id":"0x7a50","timeout_ms":10000,"last_server_duration_us":52896,"total_duration_us":10008527},{"operation_name":"upsert","last_dispatch_duration_us":9075597,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9665299,"total_server_duration_us":9365738,"encode_duration_us":25337,"operation_id":"0x7a00","timeout_ms":10000,"last_server_duration_us":9062854,"total_duration_us":10006949},{"operation_name":"upsert","last_dispatch_duration_us":10482486,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":14334026,"total_server_duration_us":9116964,"encode_duration_us":16909,"operation_id":"0x7866","timeout_ms":10000,"last_server_duration_us":9052231,"total_duration_us":10005230},{"operation_name":"upsert","last_dispatch_duration_us":421074,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9519497,"total_server_duration_us":349009,"encode_duration_us":18761,"operation_id":"0x78f7","timeout_ms":10000,"last_server_duration_us":171350,"total_duration_us":10004841},{"operation_name":"upsert","last_dispatch_duration_us":381184,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":6959026,"total_server_duration_us":869156,"encode_duration_us":17639,"operation_id":"0x797b","timeout_ms":10000,"last_server_duration_us":60885,"total_duration_us":10004357},{"operation_name":"upsert","last_dispatch_duration_us":1866823,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":1866823,"total_server_duration_us":203801,"encode_duration_us":18289,"operation_id":"0x7a38","timeout_ms":10000,"last_server_duration_us":203801,"total_duration_us":10004356},{"operation_name":"upsert","last_dispatch_duration_us":1429031,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":5059861,"total_server_duration_us":809375,"encode_duration_us":21504,"operation_id":"0x7a4b","timeout_ms":10000,"last_server_duration_us":112969,"total_duration_us":10003362},{"operation_name":"upsert","last_dispatch_duration_us":822713,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":8774505,"total_server_duration_us":1114730,"encode_duration_us":17115,"operation_id":"0x788f","timeout_ms":10000,"last_server_duration_us":549266,"total_duration_us":10001265}],"total_count":19}} INFO [2022-01-10 05:46:42,430] jepsen worker 26 - jepsen.util 3626 :info :write [8 34] :RequestTimeoutException INFO [2022-01-10 05:46:42,431] jepsen worker 26 - jepsen.util 3716 :invoke :write [8 40] INFO [2022-01-10 05:46:42,514] jepsen worker 15 - jepsen.util 3435 :info :write [5 47] :RequestTimeoutException INFO [2022-01-10 05:46:42,515] jepsen worker 15 - jepsen.util 3525 :invoke :read [5 nil] INFO [2022-01-10 05:46:42,555] jepsen worker 85 - jepsen.util 4315 :ok :read [28 45] INFO [2022-01-10 05:46:42,555] jepsen worker 76 - jepsen.util 3496 :ok :read [25 :nil] INFO [2022-01-10 05:46:42,555] jepsen worker 85 - jepsen.util 4315 :invoke :write [28 8] INFO [2022-01-10 05:46:42,556] jepsen worker 68 - jepsen.util 4028 :info :write [22 3] :RequestTimeoutException INFO [2022-01-10 05:46:42,556] jepsen worker 76 - jepsen.util 3496 :invoke :write [25 4] INFO [2022-01-10 05:46:42,557] jepsen worker 68 - jepsen.util 4118 :invoke :read [22 nil] INFO [2022-01-10 05:46:42,577] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:42,578] jepsen worker nemesis - couchbase.util Rebalance check count: 551 INFO [2022-01-10 05:46:42,578] jepsen worker nemesis - couchbase.util Rebalance stuck count: 461 INFO [2022-01-10 05:46:42,580] jepsen worker 34 - jepsen.util 3814 :info :write [11 45] :RequestTimeoutException INFO [2022-01-10 05:46:42,581] jepsen worker 34 - jepsen.util 3904 :invoke :write [11 5] INFO [2022-01-10 05:46:42,626] jepsen worker 6 - jepsen.util 4056 :ok :read [2 14] INFO [2022-01-10 05:46:42,627] jepsen worker 6 - jepsen.util 4056 :invoke :write [2 15] INFO [2022-01-10 05:46:42,653] jepsen worker 47 - jepsen.util 3917 :info :write [15 46] :RequestTimeoutException INFO [2022-01-10 05:46:42,653] jepsen worker 47 - jepsen.util 4007 :invoke :write [15 34] INFO [2022-01-10 05:46:42,700] jepsen worker 8 - jepsen.util 4328 :info :write [2 36] :RequestTimeoutException INFO [2022-01-10 05:46:42,701] jepsen worker 8 - jepsen.util 4418 :invoke :read [2 nil] INFO [2022-01-10 05:46:42,936] jepsen worker 69 - jepsen.util 3399 :info :write [23 34] :RequestTimeoutException INFO [2022-01-10 05:46:42,937] jepsen worker 69 - jepsen.util 3489 :invoke :read [23 nil] INFO [2022-01-10 05:46:42,955] jepsen worker 54 - jepsen.util 3924 :info :write [18 36] :RequestTimeoutException INFO [2022-01-10 05:46:42,956] jepsen worker 54 - jepsen.util 4014 :invoke :write [18 38] INFO [2022-01-10 05:46:43,062] jepsen worker 10 - jepsen.util 3790 :info :write [3 3] :RequestTimeoutException INFO [2022-01-10 05:46:43,063] jepsen worker 10 - jepsen.util 3880 :invoke :read [3 nil] INFO [2022-01-10 05:46:43,066] jepsen worker 28 - jepsen.util 3718 :info :write [9 13] :RequestTimeoutException INFO [2022-01-10 05:46:43,067] jepsen worker 28 - jepsen.util 3808 :invoke :read [9 nil] INFO [2022-01-10 05:46:43,085] jepsen worker 10 - jepsen.util 3880 :ok :read [3 12] INFO [2022-01-10 05:46:43,086] jepsen worker 10 - jepsen.util 3880 :invoke :read [3 nil] INFO [2022-01-10 05:46:43,129] jepsen worker 78 - jepsen.util 4758 :info :write [26 47] :RequestTimeoutException INFO [2022-01-10 05:46:43,130] jepsen worker 78 - jepsen.util 4848 :invoke :read [26 nil] INFO [2022-01-10 05:46:43,135] jepsen worker 10 - jepsen.util 3880 :ok :read [3 12] INFO [2022-01-10 05:46:43,136] jepsen worker 10 - jepsen.util 3880 :invoke :read [3 nil] INFO [2022-01-10 05:46:43,157] jepsen worker 10 - jepsen.util 3880 :ok :read [3 12] INFO [2022-01-10 05:46:43,157] jepsen worker 10 - jepsen.util 3880 :invoke :write [3 1] INFO [2022-01-10 05:46:43,162] jepsen worker 11 - jepsen.util 3701 :info :write [3 2] :RequestTimeoutException INFO [2022-01-10 05:46:43,163] jepsen worker 11 - jepsen.util 3791 :invoke :write [3 5] INFO [2022-01-10 05:46:43,171] jepsen worker 56 - jepsen.util 4196 :info :write [18 48] :RequestTimeoutException INFO [2022-01-10 05:46:43,176] jepsen worker 56 - jepsen.util 4286 :invoke :read [18 nil] INFO [2022-01-10 05:46:43,202] jepsen worker 56 - jepsen.util 4286 :ok :read [18 14] INFO [2022-01-10 05:46:43,203] jepsen worker 56 - jepsen.util 4286 :invoke :read [18 nil] INFO [2022-01-10 05:46:43,226] jepsen worker 69 - jepsen.util 3489 :ok :read [23 26] INFO [2022-01-10 05:46:43,226] jepsen worker 69 - jepsen.util 3489 :invoke :write [23 21] INFO [2022-01-10 05:46:43,227] jepsen worker 56 - jepsen.util 4286 :ok :read [18 14] INFO [2022-01-10 05:46:43,227] jepsen worker 56 - jepsen.util 4286 :invoke :write [18 10] INFO [2022-01-10 05:46:43,272] jepsen worker 17 - jepsen.util 3797 :info :write [5 36] :RequestTimeoutException INFO [2022-01-10 05:46:43,272] jepsen worker 17 - jepsen.util 3887 :invoke :write [5 32] INFO [2022-01-10 05:46:43,371] jepsen worker 55 - jepsen.util 4285 :info :write [18 12] :RequestTimeoutException INFO [2022-01-10 05:46:43,372] jepsen worker 55 - jepsen.util 4375 :invoke :read [18 nil] INFO [2022-01-10 05:46:43,399] jepsen worker 55 - jepsen.util 4375 :ok :read [18 14] INFO [2022-01-10 05:46:43,400] jepsen worker 55 - jepsen.util 4375 :invoke :write [18 36] INFO [2022-01-10 05:46:43,585] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:43,585] jepsen worker nemesis - couchbase.util Rebalance check count: 552 INFO [2022-01-10 05:46:43,585] jepsen worker nemesis - couchbase.util Rebalance stuck count: 462 INFO [2022-01-10 05:46:43,717] jepsen worker 25 - jepsen.util 3805 :ok :read [8 39] INFO [2022-01-10 05:46:43,718] jepsen worker 25 - jepsen.util 3805 :invoke :write [8 23] INFO [2022-01-10 05:46:43,797] jepsen worker 78 - jepsen.util 4848 :ok :read [26 10] INFO [2022-01-10 05:46:43,797] jepsen worker 78 - jepsen.util 4848 :invoke :write [26 15] INFO [2022-01-10 05:46:43,899] jepsen worker 86 - jepsen.util 3416 :ok :read [28 45] INFO [2022-01-10 05:46:43,899] jepsen worker 68 - jepsen.util 4118 :ok :read [22 41] INFO [2022-01-10 05:46:43,899] jepsen worker 86 - jepsen.util 3416 :invoke :read [28 nil] INFO [2022-01-10 05:46:43,900] jepsen worker 68 - jepsen.util 4118 :invoke :write [22 24] INFO [2022-01-10 05:46:44,036] jepsen worker 39 - jepsen.util 4539 :ok :read [13 15] INFO [2022-01-10 05:46:44,036] jepsen worker 39 - jepsen.util 4539 :invoke :read [13 nil] INFO [2022-01-10 05:46:44,043] jepsen worker 86 - jepsen.util 3416 :ok :read [28 45] INFO [2022-01-10 05:46:44,043] jepsen worker 86 - jepsen.util 3416 :invoke :write [28 25] INFO [2022-01-10 05:46:44,066] jepsen worker 88 - jepsen.util 3778 :info :write [29 26] :RequestTimeoutException INFO [2022-01-10 05:46:44,067] jepsen worker 88 - jepsen.util 3868 :invoke :read [29 nil] INFO [2022-01-10 05:46:44,101] jepsen worker 15 - jepsen.util 3525 :ok :read [5 25] INFO [2022-01-10 05:46:44,101] jepsen worker 15 - jepsen.util 3525 :invoke :read [5 nil] INFO [2022-01-10 05:46:44,556] jepsen worker 71 - jepsen.util 3401 :info :write [23 20] :RequestTimeoutException INFO [2022-01-10 05:46:44,556] jepsen worker 71 - jepsen.util 3491 :invoke :read [23 nil] INFO [2022-01-10 05:46:44,577] jepsen worker 75 - jepsen.util 3675 :ok :read [25 :nil] INFO [2022-01-10 05:46:44,589] jepsen worker 20 - jepsen.util 3710 :ok :read [6 25] INFO [2022-01-10 05:46:44,591] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:44,632] jepsen worker 20 - jepsen.util 3710 :invoke :write [6 10] INFO [2022-01-10 05:46:44,632] jepsen worker 75 - jepsen.util 3675 :invoke :read [25 nil] INFO [2022-01-10 05:46:44,632] jepsen worker nemesis - couchbase.util Rebalance check count: 553 INFO [2022-01-10 05:46:44,632] jepsen worker nemesis - couchbase.util Rebalance stuck count: 463 INFO [2022-01-10 05:46:44,696] jepsen worker 23 - jepsen.util 4073 :ok :read [7 :nil] INFO [2022-01-10 05:46:44,697] jepsen worker 75 - jepsen.util 3675 :ok :read [25 :nil] INFO [2022-01-10 05:46:44,697] jepsen worker 23 - jepsen.util 4073 :invoke :read [7 nil] INFO [2022-01-10 05:46:44,698] jepsen worker 75 - jepsen.util 3675 :invoke :read [25 nil] INFO [2022-01-10 05:46:44,781] jepsen worker 12 - jepsen.util 4422 :info :write [4 8] :RequestTimeoutException INFO [2022-01-10 05:46:44,782] jepsen worker 12 - jepsen.util 4512 :invoke :write [4 39] INFO [2022-01-10 05:46:44,800] jepsen worker 1 - jepsen.util 3961 :ok :read [0 46] INFO [2022-01-10 05:46:44,801] jepsen worker 1 - jepsen.util 3961 :invoke :write [0 41] INFO [2022-01-10 05:46:44,951] jepsen worker 70 - jepsen.util 3850 :info :write [23 36] :RequestTimeoutException INFO [2022-01-10 05:46:44,952] jepsen worker 70 - jepsen.util 3940 :invoke :read [23 nil] INFO [2022-01-10 05:46:44,994] jepsen worker 75 - jepsen.util 3675 :ok :read [25 :nil] INFO [2022-01-10 05:46:44,994] jepsen worker 23 - jepsen.util 4073 :ok :read [7 :nil] INFO [2022-01-10 05:46:44,995] jepsen worker 75 - jepsen.util 3675 :invoke :write [25 1] INFO [2022-01-10 05:46:44,995] jepsen worker 23 - jepsen.util 4073 :invoke :write [7 39] INFO [2022-01-10 05:46:45,198] jepsen worker 28 - jepsen.util 3808 :ok :read [9 33] INFO [2022-01-10 05:46:45,198] jepsen worker 28 - jepsen.util 3808 :invoke :write [9 42] INFO [2022-01-10 05:46:45,244] jepsen worker 37 - jepsen.util 3727 :info :write [12 42] :RequestTimeoutException INFO [2022-01-10 05:46:45,245] jepsen worker 37 - jepsen.util 3817 :invoke :read [12 nil] INFO [2022-01-10 05:46:45,349] jepsen worker 64 - jepsen.util 4024 :ok :read [21 7] INFO [2022-01-10 05:46:45,350] jepsen worker 64 - jepsen.util 4024 :invoke :write [21 3] INFO [2022-01-10 05:46:45,359] jepsen worker 63 - jepsen.util 3843 :ok :read [21 7] INFO [2022-01-10 05:46:45,360] jepsen worker 63 - jepsen.util 3843 :invoke :read [21 nil] INFO [2022-01-10 05:46:45,439] jepsen worker 39 - jepsen.util 4539 :ok :read [13 15] INFO [2022-01-10 05:46:45,440] jepsen worker 39 - jepsen.util 4539 :invoke :read [13 nil] INFO [2022-01-10 05:46:45,456] jepsen worker 15 - jepsen.util 3525 :ok :read [5 25] INFO [2022-01-10 05:46:45,457] jepsen worker 15 - jepsen.util 3525 :invoke :write [5 0] INFO [2022-01-10 05:46:45,506] jepsen worker 21 - jepsen.util 3981 :info :write [7 5] :RequestTimeoutException INFO [2022-01-10 05:46:45,506] jepsen worker 21 - jepsen.util 4071 :invoke :write [7 33] INFO [2022-01-10 05:46:45,542] jepsen worker 30 - jepsen.util 3630 :info :write [10 27] :SyncWriteAmbiguous INFO [2022-01-10 05:46:45,543] jepsen worker 30 - jepsen.util 3720 :invoke :read [10 nil] INFO [2022-01-10 05:46:45,614] jepsen worker 16 - jepsen.util 4066 :ok :read [5 25] INFO [2022-01-10 05:46:45,615] jepsen worker 16 - jepsen.util 4066 :invoke :write [5 4] INFO [2022-01-10 05:46:45,638] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:45,638] jepsen worker nemesis - couchbase.util Rebalance check count: 554 INFO [2022-01-10 05:46:45,638] jepsen worker nemesis - couchbase.util Rebalance stuck count: 464 INFO [2022-01-10 05:46:45,666] jepsen worker 45 - jepsen.util 4185 :info :write [15 5] :RequestTimeoutException INFO [2022-01-10 05:46:45,667] jepsen worker 45 - jepsen.util 4275 :invoke :write [15 27] INFO [2022-01-10 05:46:45,687] jepsen worker 88 - jepsen.util 3868 :ok :read [29 0] INFO [2022-01-10 05:46:45,689] jepsen worker 88 - jepsen.util 3868 :invoke :write [29 10] INFO [2022-01-10 05:46:45,696] jepsen worker 30 - jepsen.util 3720 :ok :read [10 36] INFO [2022-01-10 05:46:45,696] jepsen worker 30 - jepsen.util 3720 :invoke :write [10 11] INFO [2022-01-10 05:46:45,924] jepsen worker 40 - jepsen.util 3460 :info :write [13 39] :RequestTimeoutException INFO [2022-01-10 05:46:45,925] jepsen worker 40 - jepsen.util 3550 :invoke :read [13 nil] INFO [2022-01-10 05:46:46,591] jepsen worker 29 - jepsen.util 3899 :info :write [9 10] :RequestTimeoutException INFO [2022-01-10 05:46:46,592] jepsen worker 29 - jepsen.util 3989 :invoke :read [9 nil] INFO [2022-01-10 05:46:46,644] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:46,644] jepsen worker nemesis - couchbase.util Rebalance check count: 555 INFO [2022-01-10 05:46:46,644] jepsen worker nemesis - couchbase.util Rebalance stuck count: 465 INFO [2022-01-10 05:46:46,757] jepsen worker 71 - jepsen.util 3491 :ok :read [23 26] INFO [2022-01-10 05:46:46,757] jepsen worker 18 - jepsen.util 3888 :info :write [6 13] :SyncWriteAmbiguous INFO [2022-01-10 05:46:46,758] jepsen worker 71 - jepsen.util 3491 :invoke :write [23 40] INFO [2022-01-10 05:46:46,758] jepsen worker 87 - jepsen.util 3777 :info :write [29 15] :RequestTimeoutException INFO [2022-01-10 05:46:46,759] jepsen worker 18 - jepsen.util 3978 :invoke :write [6 12] INFO [2022-01-10 05:46:46,760] jepsen worker 87 - jepsen.util 3867 :invoke :write [29 48] INFO [2022-01-10 05:46:46,769] jepsen worker 4 - jepsen.util 4504 :info :write [1 31] :RequestTimeoutException INFO [2022-01-10 05:46:46,769] jepsen worker 4 - jepsen.util 4594 :invoke :write [1 26] INFO [2022-01-10 05:46:46,798] jepsen worker 72 - jepsen.util 4392 :ok :read [24 18] INFO [2022-01-10 05:46:46,798] jepsen worker 72 - jepsen.util 4392 :invoke :read [24 nil] INFO [2022-01-10 05:46:46,817] jepsen worker 39 - jepsen.util 4539 :ok :read [13 15] INFO [2022-01-10 05:46:46,817] jepsen worker 39 - jepsen.util 4539 :invoke :write [13 12] INFO [2022-01-10 05:46:46,892] jepsen worker 83 - jepsen.util 4403 :info :write [27 45] :RequestTimeoutException INFO [2022-01-10 05:46:46,893] jepsen worker 83 - jepsen.util 4493 :invoke :write [27 2] INFO [2022-01-10 05:46:46,960] jepsen worker 63 - jepsen.util 3843 :ok :read [21 7] INFO [2022-01-10 05:46:46,960] jepsen worker 63 - jepsen.util 3843 :invoke :read [21 nil] INFO [2022-01-10 05:46:46,987] jepsen worker 19 - jepsen.util 4159 :ok :read [6 25] INFO [2022-01-10 05:46:46,988] jepsen worker 19 - jepsen.util 4159 :invoke :write [6 45] INFO [2022-01-10 05:46:47,005] jepsen worker 53 - jepsen.util 4823 :ok :read [17 41] INFO [2022-01-10 05:46:47,006] jepsen worker 53 - jepsen.util 4823 :invoke :read [17 nil] INFO [2022-01-10 05:46:47,008] jepsen worker 70 - jepsen.util 3940 :ok :read [23 26] INFO [2022-01-10 05:46:47,009] jepsen worker 70 - jepsen.util 3940 :invoke :write [23 10] INFO [2022-01-10 05:46:47,069] jepsen worker 29 - jepsen.util 3989 :ok :read [9 33] INFO [2022-01-10 05:46:47,069] jepsen worker 29 - jepsen.util 3989 :invoke :read [9 nil] INFO [2022-01-10 05:46:47,228] jepsen worker 53 - jepsen.util 4823 :ok :read [17 41] INFO [2022-01-10 05:46:47,229] jepsen worker 53 - jepsen.util 4823 :invoke :write [17 8] INFO [2022-01-10 05:46:47,275] jepsen worker 67 - jepsen.util 3757 :info :write [22 33] :RequestTimeoutException INFO [2022-01-10 05:46:47,276] jepsen worker 67 - jepsen.util 3847 :invoke :write [22 25] INFO [2022-01-10 05:46:47,286] jepsen worker 57 - jepsen.util 4737 :info :write [19 14] :RequestTimeoutException INFO [2022-01-10 05:46:47,286] jepsen worker 57 - jepsen.util 4827 :invoke :write [19 7] INFO [2022-01-10 05:46:47,316] jepsen worker 29 - jepsen.util 3989 :ok :read [9 33] INFO [2022-01-10 05:46:47,316] jepsen worker 29 - jepsen.util 3989 :invoke :read [9 nil] INFO [2022-01-10 05:46:47,472] jepsen worker 9 - jepsen.util 4419 :info :write [3 28] :RequestTimeoutException INFO [2022-01-10 05:46:47,472] jepsen worker 9 - jepsen.util 4509 :invoke :write [3 1] INFO [2022-01-10 05:46:47,614] jepsen worker 84 - jepsen.util 3864 :info :write [28 18] :RequestTimeoutException INFO [2022-01-10 05:46:47,615] jepsen worker 84 - jepsen.util 3954 :invoke :write [28 1] INFO [2022-01-10 05:46:47,650] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:47,650] jepsen worker nemesis - couchbase.util Rebalance check count: 556 INFO [2022-01-10 05:46:47,650] jepsen worker nemesis - couchbase.util Rebalance stuck count: 466 INFO [2022-01-10 05:46:47,685] jepsen worker 35 - jepsen.util 3725 :info :write [11 10] :RequestTimeoutException INFO [2022-01-10 05:46:47,686] jepsen worker 35 - jepsen.util 3815 :invoke :read [11 nil] INFO [2022-01-10 05:46:47,767] jepsen worker 51 - jepsen.util 4731 :info :write [17 24] :RequestTimeoutException INFO [2022-01-10 05:46:47,767] jepsen worker 51 - jepsen.util 4821 :invoke :write [17 49] INFO [2022-01-10 05:46:47,912] jepsen worker 58 - jepsen.util 3658 :info :write [19 37] :RequestTimeoutException INFO [2022-01-10 05:46:47,912] jepsen worker 58 - jepsen.util 3748 :invoke :write [19 11] INFO [2022-01-10 05:46:47,971] jepsen worker 73 - jepsen.util 3583 :info :write [24 5] :RequestTimeoutException INFO [2022-01-10 05:46:47,972] jepsen worker 73 - jepsen.util 3673 :invoke :read [24 nil] INFO [2022-01-10 05:46:48,015] jepsen worker 62 - jepsen.util 4382 :info :write [20 37] :RequestTimeoutException INFO [2022-01-10 05:46:48,016] jepsen worker 62 - jepsen.util 4472 :invoke :write [20 9] INFO [2022-01-10 05:46:48,220] jepsen worker 74 - jepsen.util 4484 :info :write [24 7] :RequestTimeoutException INFO [2022-01-10 05:46:48,220] jepsen worker 74 - jepsen.util 4574 :invoke :write [24 44] INFO [2022-01-10 05:46:48,356] jepsen worker 61 - jepsen.util 4111 :info :write [20 33] :RequestTimeoutException INFO [2022-01-10 05:46:48,357] jepsen worker 61 - jepsen.util 4201 :invoke :read [20 nil] INFO [2022-01-10 05:46:48,382] jepsen worker 59 - jepsen.util 4019 :info :write [19 25] :RequestTimeoutException INFO [2022-01-10 05:46:48,383] jepsen worker 59 - jepsen.util 4109 :invoke :read [19 nil] INFO [2022-01-10 05:46:48,499] jepsen worker 41 - jepsen.util 3821 :info :write [13 18] :RequestTimeoutException INFO [2022-01-10 05:46:48,500] jepsen worker 41 - jepsen.util 3911 :invoke :write [13 0] INFO [2022-01-10 05:46:48,510] jepsen worker 82 - jepsen.util 4042 :info :write [27 40] :RequestTimeoutException INFO [2022-01-10 05:46:48,511] jepsen worker 82 - jepsen.util 4132 :invoke :write [27 26] INFO [2022-01-10 05:46:48,540] jepsen worker 29 - jepsen.util 3989 :ok :read [9 33] INFO [2022-01-10 05:46:48,541] jepsen worker 29 - jepsen.util 3989 :invoke :read [9 nil] INFO [2022-01-10 05:46:48,655] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:48,655] jepsen worker nemesis - couchbase.util Rebalance check count: 557 INFO [2022-01-10 05:46:48,655] jepsen worker nemesis - couchbase.util Rebalance stuck count: 467 INFO [2022-01-10 05:46:49,014] jepsen worker 59 - jepsen.util 4109 :ok :read [19 31] INFO [2022-01-10 05:46:49,014] jepsen worker 59 - jepsen.util 4109 :invoke :write [19 32] INFO [2022-01-10 05:46:49,024] jepsen worker 42 - jepsen.util 3642 :info :write [14 32] :RequestTimeoutException INFO [2022-01-10 05:46:49,025] jepsen worker 42 - jepsen.util 3732 :invoke :write [14 31] INFO [2022-01-10 05:46:49,222] jepsen worker 43 - jepsen.util 3823 :info :write [14 49] :RequestTimeoutException INFO [2022-01-10 05:46:49,222] jepsen worker 43 - jepsen.util 3913 :invoke :read [14 nil] INFO [2022-01-10 05:46:49,327] jepsen worker 29 - jepsen.util 3989 :ok :read [9 33] INFO [2022-01-10 05:46:49,327] jepsen worker 29 - jepsen.util 3989 :invoke :read [9 nil] INFO [2022-01-10 05:46:49,521] jepsen worker 48 - jepsen.util 3648 :info :write [16 15] :RequestTimeoutException INFO [2022-01-10 05:46:49,521] jepsen worker 48 - jepsen.util 3738 :invoke :read [16 nil] INFO [2022-01-10 05:46:49,646] jepsen worker 13 - jepsen.util 4603 :info :write [4 11] :RequestTimeoutException INFO [2022-01-10 05:46:49,647] jepsen worker 13 - jepsen.util 4693 :invoke :read [4 nil] INFO [2022-01-10 05:46:49,661] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:49,661] jepsen worker nemesis - couchbase.util Rebalance check count: 558 INFO [2022-01-10 05:46:49,661] jepsen worker nemesis - couchbase.util Rebalance stuck count: 468 INFO [2022-01-10 05:46:49,852] jepsen worker 65 - jepsen.util 3755 :info :write [21 21] :RequestTimeoutException INFO [2022-01-10 05:46:49,852] jepsen worker 65 - jepsen.util 3845 :invoke :read [21 nil] INFO [2022-01-10 05:46:50,016] jepsen worker 77 - jepsen.util 4397 :info :write [25 26] :RequestTimeoutException INFO [2022-01-10 05:46:50,017] jepsen worker 77 - jepsen.util 4487 :invoke :write [25 39] INFO [2022-01-10 05:46:50,224] jepsen worker 24 - jepsen.util 3624 :info :write [8 15] :RequestTimeoutException INFO [2022-01-10 05:46:50,225] jepsen worker 24 - jepsen.util 3714 :invoke :write [8 13] INFO [2022-01-10 05:46:50,246] jepsen worker 46 - jepsen.util 4096 :info :write [15 41] :RequestTimeoutException INFO [2022-01-10 05:46:50,246] jepsen worker 46 - jepsen.util 4186 :invoke :write [15 32] INFO [2022-01-10 05:46:50,359] jepsen worker 48 - jepsen.util 3738 :ok :read [16 38] INFO [2022-01-10 05:46:50,359] jepsen worker 8 - jepsen.util 4418 :ok :read [2 14] INFO [2022-01-10 05:46:50,360] jepsen worker 48 - jepsen.util 3738 :invoke :write [16 5] INFO [2022-01-10 05:46:50,360] jepsen worker 22 - jepsen.util 3712 :info :write [7 46] :RequestTimeoutException INFO [2022-01-10 05:46:50,360] jepsen worker 8 - jepsen.util 4418 :invoke :read [2 nil] INFO [2022-01-10 05:46:50,361] jepsen worker 22 - jepsen.util 3802 :invoke :write [7 19] INFO [2022-01-10 05:46:50,477] jepsen worker 60 - jepsen.util 4200 :ok :read [20 :nil] INFO [2022-01-10 05:46:50,478] jepsen worker 60 - jepsen.util 4200 :invoke :read [20 nil] INFO [2022-01-10 05:46:50,510] jepsen worker 7 - jepsen.util 4147 :ok :read [2 14] INFO [2022-01-10 05:46:50,510] jepsen worker 7 - jepsen.util 4147 :invoke :write [2 1] WARN [2022-01-10 05:46:50,539] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18612832,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":21474879,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9388390,"encode_duration_us":16519,"operation_id":"0x7896","timeout_ms":10000,"last_server_duration_us":9073483,"total_duration_us":10004465}],"total_count":1}} WARN [2022-01-10 05:46:50,539] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16419349,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":16419349,"total_server_duration_us":12311,"operation_id":"0x799e","timeout_ms":30000,"last_server_duration_us":12311,"total_duration_us":16421695},{"operation_name":"get","last_dispatch_duration_us":1019827,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1019827,"total_server_duration_us":484,"operation_id":"0x7a65","timeout_ms":30000,"last_server_duration_us":484,"total_duration_us":10777244},{"operation_name":"upsert","last_dispatch_duration_us":35637,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":1169001,"total_server_duration_us":615876,"encode_duration_us":17849,"operation_id":"0x7b72","timeout_ms":10000,"last_server_duration_us":23390,"total_duration_us":10014271},{"operation_name":"upsert","last_dispatch_duration_us":222208,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2663691,"total_server_duration_us":334068,"encode_duration_us":16342,"operation_id":"0x7bb7","timeout_ms":10000,"last_server_duration_us":84574,"total_duration_us":10010997},{"operation_name":"upsert","last_dispatch_duration_us":699795,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":1922108,"total_server_duration_us":293516,"encode_duration_us":16502,"operation_id":"0x7b8a","timeout_ms":10000,"last_server_duration_us":171350,"total_duration_us":10010510},{"operation_name":"upsert","last_dispatch_duration_us":406975,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":1701497,"total_server_duration_us":366742,"encode_duration_us":17459,"operation_id":"0x7b90","timeout_ms":10000,"last_server_duration_us":237259,"total_duration_us":10009482},{"operation_name":"upsert","last_dispatch_duration_us":934355,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":4436812,"total_server_duration_us":1127350,"encode_duration_us":16915,"operation_id":"0x7bed","timeout_ms":10000,"last_server_duration_us":53374,"total_duration_us":10007975},{"operation_name":"upsert","last_dispatch_duration_us":12595800,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12595800,"total_server_duration_us":140420,"encode_duration_us":19099,"operation_id":"0x7a8c","timeout_ms":10000,"last_server_duration_us":140420,"total_duration_us":10007539},{"operation_name":"upsert","last_dispatch_duration_us":778160,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10486600,"total_server_duration_us":316398,"encode_duration_us":16978,"operation_id":"0x7b4f","timeout_ms":10000,"last_server_duration_us":191275,"total_duration_us":10006022},{"operation_name":"upsert","last_dispatch_duration_us":2125809,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":7545501,"total_server_duration_us":1100135,"encode_duration_us":21579,"operation_id":"0x7b28","timeout_ms":10000,"last_server_duration_us":70690,"total_duration_us":10004967}],"total_count":16}} INFO [2022-01-10 05:46:50,591] jepsen worker 66 - jepsen.util 3666 :info :write [22 26] :RequestTimeoutException INFO [2022-01-10 05:46:50,591] jepsen worker 66 - jepsen.util 3756 :invoke :read [22 nil] INFO [2022-01-10 05:46:50,621] jepsen worker 27 - jepsen.util 4167 :info :write [9 1] :RequestTimeoutException INFO [2022-01-10 05:46:50,622] jepsen worker 27 - jepsen.util 4257 :invoke :read [9 nil] INFO [2022-01-10 05:46:50,666] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:50,666] jepsen worker nemesis - couchbase.util Rebalance check count: 559 INFO [2022-01-10 05:46:50,666] jepsen worker nemesis - couchbase.util Rebalance stuck count: 469 INFO [2022-01-10 05:46:50,746] jepsen worker 49 - jepsen.util 3829 :info :write [16 24] :RequestTimeoutException INFO [2022-01-10 05:46:50,747] jepsen worker 49 - jepsen.util 3919 :invoke :read [16 nil] INFO [2022-01-10 05:46:50,747] jepsen worker 29 - jepsen.util 3989 :ok :read [9 33] INFO [2022-01-10 05:46:50,748] jepsen worker 29 - jepsen.util 3989 :invoke :read [9 nil] INFO [2022-01-10 05:46:50,749] jepsen worker 81 - jepsen.util 4311 :info :write [27 24] :RequestTimeoutException INFO [2022-01-10 05:46:50,750] jepsen worker 81 - jepsen.util 4401 :invoke :write [27 43] INFO [2022-01-10 05:46:50,907] jepsen worker 89 - jepsen.util 4499 :ok :read [29 0] INFO [2022-01-10 05:46:50,907] jepsen worker 89 - jepsen.util 4499 :invoke :write [29 47] INFO [2022-01-10 05:46:50,953] jepsen worker 33 - jepsen.util 4173 :ok :read [11 21] INFO [2022-01-10 05:46:50,954] jepsen worker 33 - jepsen.util 4173 :invoke :write [11 4] INFO [2022-01-10 05:46:50,992] jepsen worker 40 - jepsen.util 3550 :ok :read [13 15] INFO [2022-01-10 05:46:50,993] jepsen worker 40 - jepsen.util 3550 :invoke :write [13 12] INFO [2022-01-10 05:46:51,084] jepsen worker 80 - jepsen.util 4850 :info :write [26 5] :RequestTimeoutException INFO [2022-01-10 05:46:51,085] jepsen worker 80 - jepsen.util 4940 :invoke :read [26 nil] INFO [2022-01-10 05:46:51,236] jepsen worker 52 - jepsen.util 5002 :info :write [17 49] :RequestTimeoutException INFO [2022-01-10 05:46:51,236] jepsen worker 52 - jepsen.util 5092 :invoke :read [17 nil] INFO [2022-01-10 05:46:51,371] jepsen worker 32 - jepsen.util 4172 :info :write [10 27] :RequestTimeoutException INFO [2022-01-10 05:46:51,372] jepsen worker 32 - jepsen.util 4262 :invoke :read [10 nil] INFO [2022-01-10 05:46:51,390] jepsen worker 52 - jepsen.util 5092 :ok :read [17 41] INFO [2022-01-10 05:46:51,391] jepsen worker 52 - jepsen.util 5092 :invoke :write [17 0] INFO [2022-01-10 05:46:51,444] jepsen worker 32 - jepsen.util 4262 :ok :read [10 36] INFO [2022-01-10 05:46:51,444] jepsen worker 32 - jepsen.util 4262 :invoke :write [10 21] INFO [2022-01-10 05:46:51,492] jepsen worker 79 - jepsen.util 4669 :info :write [26 10] :RequestTimeoutException INFO [2022-01-10 05:46:51,493] jepsen worker 79 - jepsen.util 4759 :invoke :read [26 nil] INFO [2022-01-10 05:46:51,516] jepsen worker 60 - jepsen.util 4200 :ok :read [20 :nil] INFO [2022-01-10 05:46:51,517] jepsen worker 60 - jepsen.util 4200 :invoke :write [20 40] INFO [2022-01-10 05:46:51,577] jepsen worker 14 - jepsen.util 4874 :info :write [4 28] :RequestTimeoutException INFO [2022-01-10 05:46:51,578] jepsen worker 14 - jepsen.util 4964 :invoke :read [4 nil] INFO [2022-01-10 05:46:51,601] jepsen worker 80 - jepsen.util 4940 :ok :read [26 10] INFO [2022-01-10 05:46:51,602] jepsen worker 80 - jepsen.util 4940 :invoke :write [26 34] INFO [2022-01-10 05:46:51,606] jepsen worker 50 - jepsen.util 4100 :info :write [16 26] :RequestTimeoutException INFO [2022-01-10 05:46:51,607] jepsen worker 50 - jepsen.util 4190 :invoke :write [16 33] INFO [2022-01-10 05:46:51,673] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:51,673] jepsen worker nemesis - couchbase.util Rebalance check count: 560 INFO [2022-01-10 05:46:51,673] jepsen worker nemesis - couchbase.util Rebalance stuck count: 470 INFO [2022-01-10 05:46:51,796] jepsen worker 66 - jepsen.util 3756 :ok :read [22 41] INFO [2022-01-10 05:46:51,797] jepsen worker 66 - jepsen.util 3756 :invoke :read [22 nil] INFO [2022-01-10 05:46:51,871] jepsen worker 5 - jepsen.util 3785 :info :write [1 47] :RequestTimeoutException INFO [2022-01-10 05:46:51,872] jepsen worker 5 - jepsen.util 3875 :invoke :write [1 35] INFO [2022-01-10 05:46:51,925] jepsen worker 2 - jepsen.util 4232 :info :write [0 8] :RequestTimeoutException INFO [2022-01-10 05:46:51,925] jepsen worker 2 - jepsen.util 4322 :invoke :write [0 33] INFO [2022-01-10 05:46:51,966] jepsen worker 31 - jepsen.util 4261 :info :write [10 26] :RequestTimeoutException INFO [2022-01-10 05:46:51,966] jepsen worker 31 - jepsen.util 4351 :invoke :read [10 nil] INFO [2022-01-10 05:46:51,980] jepsen worker 3 - jepsen.util 3783 :info :write [1 3] :RequestTimeoutException INFO [2022-01-10 05:46:51,980] jepsen worker 3 - jepsen.util 3873 :invoke :write [1 10] INFO [2022-01-10 05:46:52,018] jepsen worker 31 - jepsen.util 4351 :ok :read [10 36] INFO [2022-01-10 05:46:52,019] jepsen worker 31 - jepsen.util 4351 :invoke :read [10 nil] WARN [2022-01-10 05:46:52,037] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9036870,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10786028,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9499460,"encode_duration_us":17177,"operation_id":"0x7af5","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10003487},{"operation_name":"upsert","last_dispatch_duration_us":12014362,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12014362,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":11422729,"encode_duration_us":19483,"operation_id":"0x7a82","timeout_ms":10000,"last_server_duration_us":11422729,"total_duration_us":10004138}],"total_count":2}} INFO [2022-01-10 05:46:52,039] jepsen worker 31 - jepsen.util 4351 :ok :read [10 36] INFO [2022-01-10 05:46:52,040] jepsen worker 31 - jepsen.util 4351 :invoke :read [10 nil] INFO [2022-01-10 05:46:52,060] jepsen worker 31 - jepsen.util 4351 :ok :read [10 36] INFO [2022-01-10 05:46:52,061] jepsen worker 31 - jepsen.util 4351 :invoke :write [10 44] WARN [2022-01-10 05:46:52,124] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10844864,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10844864,"total_server_duration_us":10825682,"encode_duration_us":17289,"operation_id":"0x7bfb","timeout_ms":10000,"last_server_duration_us":10825682,"total_duration_us":10009084},{"operation_name":"upsert","last_dispatch_duration_us":41842,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":1110699,"total_server_duration_us":543930,"encode_duration_us":16372,"operation_id":"0x7c80","timeout_ms":10000,"last_server_duration_us":26507,"total_duration_us":10008329},{"operation_name":"upsert","last_dispatch_duration_us":9420443,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":11767165,"total_server_duration_us":441033,"encode_duration_us":17756,"operation_id":"0x7b12","timeout_ms":10000,"last_server_duration_us":224321,"total_duration_us":10008201},{"operation_name":"upsert","last_dispatch_duration_us":9484535,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10094381,"total_server_duration_us":236844,"encode_duration_us":17641,"operation_id":"0x7c2e","timeout_ms":10000,"last_server_duration_us":70960,"total_duration_us":10008019},{"operation_name":"upsert","last_dispatch_duration_us":9335008,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9821995,"total_server_duration_us":421639,"encode_duration_us":23674,"operation_id":"0x7c2f","timeout_ms":10000,"last_server_duration_us":54818,"total_duration_us":10004258},{"operation_name":"upsert","last_dispatch_duration_us":9247076,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":11351094,"total_server_duration_us":635790,"encode_duration_us":18753,"operation_id":"0x7b1f","timeout_ms":10000,"last_server_duration_us":209548,"total_duration_us":10002679},{"operation_name":"upsert","last_dispatch_duration_us":2082222,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2082222,"total_server_duration_us":161069,"encode_duration_us":16189,"operation_id":"0x7bb5","timeout_ms":10000,"last_server_duration_us":161069,"total_duration_us":10001868},{"operation_name":"upsert","last_dispatch_duration_us":66194,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":1189458,"total_server_duration_us":567543,"encode_duration_us":16359,"operation_id":"0x7c5c","timeout_ms":10000,"last_server_duration_us":21977,"total_duration_us":10001640},{"operation_name":"get","last_dispatch_duration_us":5900561,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5900561,"total_server_duration_us":198123,"operation_id":"0x7c8e","timeout_ms":30000,"last_server_duration_us":198123,"total_duration_us":8523615},{"operation_name":"get","last_dispatch_duration_us":1722727,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1722727,"total_server_duration_us":74510,"operation_id":"0x7b81","timeout_ms":30000,"last_server_duration_us":74510,"total_duration_us":8421568}],"total_count":16}} WARN [2022-01-10 05:46:52,124] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10453931,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10870433,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9152768,"encode_duration_us":24495,"operation_id":"0x79c3","timeout_ms":10000,"last_server_duration_us":9097950,"total_duration_us":10002150},{"operation_name":"upsert","last_dispatch_duration_us":9803094,"last_local_id":"EFE9906200000006/0000000021E2C07D","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":14341803,"last_local_socket":"172.23.104.79:44302","total_server_duration_us":12207752,"encode_duration_us":18925,"operation_id":"0x7a52","timeout_ms":10000,"last_server_duration_us":9404523,"total_duration_us":10008732},{"operation_name":"upsert","last_dispatch_duration_us":10844864,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10844864,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10825682,"encode_duration_us":17289,"operation_id":"0x7bfb","timeout_ms":10000,"last_server_duration_us":10825682,"total_duration_us":10009084}],"total_count":3}} INFO [2022-01-10 05:46:52,218] jepsen worker 27 - jepsen.util 4257 :ok :read [9 33] INFO [2022-01-10 05:46:52,218] jepsen worker 27 - jepsen.util 4257 :invoke :read [9 nil] INFO [2022-01-10 05:46:52,221] jepsen worker 10 - jepsen.util 3880 :info :write [3 1] :SyncWriteAmbiguous INFO [2022-01-10 05:46:52,222] jepsen worker 10 - jepsen.util 3970 :invoke :read [3 nil] WARN [2022-01-10 05:46:52,228] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9705421,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":14765282,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":10295007,"encode_duration_us":21504,"operation_id":"0x7a4b","timeout_ms":10000,"last_server_duration_us":9485632,"total_duration_us":10003362},{"operation_name":"upsert","last_dispatch_duration_us":10846740,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10846740,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9066042,"encode_duration_us":16632,"operation_id":"0x7a8e","timeout_ms":10000,"last_server_duration_us":9066042,"total_duration_us":10007790},{"operation_name":"upsert","last_dispatch_duration_us":10548589,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12415412,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9230557,"encode_duration_us":18289,"operation_id":"0x7a38","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10004356}],"total_count":3}} WARN [2022-01-10 05:46:52,230] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11494461,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11494461,"total_server_duration_us":551,"operation_id":"0x7beb","timeout_ms":30000,"last_server_duration_us":551,"total_duration_us":11494801},{"operation_name":"get","last_dispatch_duration_us":10811671,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10811671,"total_server_duration_us":43,"operation_id":"0x7c20","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":10812655},{"operation_name":"upsert","last_dispatch_duration_us":8967377,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11245108,"total_server_duration_us":1452949,"encode_duration_us":17086,"operation_id":"0x7b30","timeout_ms":10000,"last_server_duration_us":182103,"total_duration_us":10010279},{"operation_name":"upsert","last_dispatch_duration_us":609267,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":6585785,"total_server_duration_us":1870193,"encode_duration_us":25376,"operation_id":"0x7ab4","timeout_ms":10000,"last_server_duration_us":167439,"total_duration_us":10010069},{"operation_name":"upsert","last_dispatch_duration_us":12382595,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":14905041,"total_server_duration_us":4040052,"encode_duration_us":18406,"operation_id":"0x7b2e","timeout_ms":10000,"last_server_duration_us":3511046,"total_duration_us":10009888},{"operation_name":"get","last_dispatch_duration_us":10005990,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10005990,"total_server_duration_us":38,"operation_id":"0x7c2c","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":10008424},{"operation_name":"upsert","last_dispatch_duration_us":11583126,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11583126,"total_server_duration_us":18953,"encode_duration_us":19731,"operation_id":"0x7a87","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10005131},{"operation_name":"upsert","last_dispatch_duration_us":3244541,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9642582,"total_server_duration_us":3070735,"encode_duration_us":19586,"operation_id":"0x7c5f","timeout_ms":10000,"last_server_duration_us":384532,"total_duration_us":10004417},{"operation_name":"upsert","last_dispatch_duration_us":31497,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9393629,"total_server_duration_us":238337,"encode_duration_us":17571,"operation_id":"0x7c82","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10002052},{"operation_name":"upsert","last_dispatch_duration_us":1477329,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":6909216,"total_server_duration_us":1653977,"encode_duration_us":18709,"operation_id":"0x7ba6","timeout_ms":10000,"last_server_duration_us":134692,"total_duration_us":10002034}],"total_count":19}} WARN [2022-01-10 05:46:52,237] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7652034,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8749631,"total_server_duration_us":339794,"encode_duration_us":16965,"operation_id":"0x7ac7","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10010156},{"operation_name":"upsert","last_dispatch_duration_us":783752,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10356466,"total_server_duration_us":618092,"encode_duration_us":17855,"operation_id":"0x7b5d","timeout_ms":10000,"last_server_duration_us":164917,"total_duration_us":10009859},{"operation_name":"upsert","last_dispatch_duration_us":6883831,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11227864,"total_server_duration_us":812685,"encode_duration_us":16411,"operation_id":"0x7bf4","timeout_ms":10000,"last_server_duration_us":199379,"total_duration_us":10009050},{"operation_name":"upsert","last_dispatch_duration_us":7572916,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8784670,"total_server_duration_us":374486,"encode_duration_us":16744,"operation_id":"0x7ac6","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10008694},{"operation_name":"upsert","last_dispatch_duration_us":185076,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9993585,"total_server_duration_us":752998,"encode_duration_us":16170,"operation_id":"0x7bc3","timeout_ms":10000,"last_server_duration_us":65123,"total_duration_us":10008504},{"operation_name":"upsert","last_dispatch_duration_us":2115431,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":5621275,"total_server_duration_us":1525315,"encode_duration_us":16543,"operation_id":"0x7c1e","timeout_ms":10000,"last_server_duration_us":148646,"total_duration_us":10008280},{"operation_name":"upsert","last_dispatch_duration_us":7770674,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11263212,"total_server_duration_us":212614,"encode_duration_us":16387,"operation_id":"0x7aca","timeout_ms":10000,"last_server_duration_us":26685,"total_duration_us":10008279},{"operation_name":"upsert","last_dispatch_duration_us":9036697,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10004041,"total_server_duration_us":9281068,"encode_duration_us":18664,"operation_id":"0x7acc","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10007811},{"operation_name":"upsert","last_dispatch_duration_us":800410,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8390566,"total_server_duration_us":2250860,"encode_duration_us":16997,"operation_id":"0x7a8f","timeout_ms":10000,"last_server_duration_us":29137,"total_duration_us":10007408},{"operation_name":"upsert","last_dispatch_duration_us":309627,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10211564,"total_server_duration_us":882029,"encode_duration_us":16747,"operation_id":"0x7bb3","timeout_ms":10000,"last_server_duration_us":192926,"total_duration_us":10006801}],"total_count":20}} WARN [2022-01-10 05:46:52,238] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2177073,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8426628,"total_server_duration_us":1344257,"encode_duration_us":16389,"operation_id":"0x7c60","timeout_ms":10000,"last_server_duration_us":921855,"total_duration_us":10011071},{"operation_name":"upsert","last_dispatch_duration_us":1702232,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7410503,"total_server_duration_us":401071,"encode_duration_us":17489,"operation_id":"0x7b8b","timeout_ms":10000,"last_server_duration_us":133276,"total_duration_us":10010815},{"operation_name":"upsert","last_dispatch_duration_us":42732,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":1993381,"total_server_duration_us":578533,"encode_duration_us":17561,"operation_id":"0x7ab6","timeout_ms":10000,"last_server_duration_us":25800,"total_duration_us":10009715},{"operation_name":"upsert","last_dispatch_duration_us":2183838,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10316967,"total_server_duration_us":514068,"encode_duration_us":16445,"operation_id":"0x7c3c","timeout_ms":10000,"last_server_duration_us":134692,"total_duration_us":10008929},{"operation_name":"upsert","last_dispatch_duration_us":2645410,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11197822,"total_server_duration_us":71233,"encode_duration_us":16615,"operation_id":"0x7a8d","timeout_ms":10000,"last_server_duration_us":24154,"total_duration_us":10006460},{"operation_name":"upsert","last_dispatch_duration_us":12148525,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":13779438,"total_server_duration_us":496711,"encode_duration_us":16671,"operation_id":"0x7bc2","timeout_ms":10000,"last_server_duration_us":406977,"total_duration_us":10005951},{"operation_name":"upsert","last_dispatch_duration_us":539821,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":5611799,"total_server_duration_us":2965543,"encode_duration_us":17256,"operation_id":"0x7c5b","timeout_ms":10000,"last_server_duration_us":397934,"total_duration_us":10004780},{"operation_name":"upsert","last_dispatch_duration_us":12014362,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12014362,"total_server_duration_us":11422729,"encode_duration_us":19483,"operation_id":"0x7a82","timeout_ms":10000,"last_server_duration_us":11422729,"total_duration_us":10004138},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":16461,"operation_id":"0x7c1f","timeout_ms":10000,"total_duration_us":10003742},{"operation_name":"upsert","last_dispatch_duration_us":9036870,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10786028,"total_server_duration_us":9499460,"encode_duration_us":17177,"operation_id":"0x7af5","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10003487}],"total_count":19}} INFO [2022-01-10 05:46:52,272] jepsen worker 44 - jepsen.util 4094 :info :write [14 15] :RequestTimeoutException INFO [2022-01-10 05:46:52,272] jepsen worker 44 - jepsen.util 4184 :invoke :write [14 19] INFO [2022-01-10 05:46:52,296] jepsen worker 0 - jepsen.util 3870 :info :write [0 43] :RequestTimeoutException INFO [2022-01-10 05:46:52,297] jepsen worker 0 - jepsen.util 3960 :invoke :read [0 nil] WARN [2022-01-10 05:46:52,330] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17844637,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18694211,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9423663,"encode_duration_us":18465,"operation_id":"0x7964","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10001907}],"total_count":1}} WARN [2022-01-10 05:46:52,337] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8281128,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14388553,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":8154509,"encode_duration_us":30687,"operation_id":"0x7a02","timeout_ms":10000,"last_server_duration_us":8057180,"total_duration_us":10003443},{"operation_name":"upsert","last_dispatch_duration_us":9036697,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10004041,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9281068,"encode_duration_us":18664,"operation_id":"0x7acc","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10007811}],"total_count":2}} INFO [2022-01-10 05:46:52,376] jepsen worker 43 - jepsen.util 3913 :ok :read [14 14] INFO [2022-01-10 05:46:52,376] jepsen worker 43 - jepsen.util 3913 :invoke :read [14 nil] WARN [2022-01-10 05:46:52,429] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2463031,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":5165772,"total_server_duration_us":122405,"encode_duration_us":19442,"operation_id":"0x7bc8","timeout_ms":10000,"last_server_duration_us":22306,"total_duration_us":10010893},{"operation_name":"upsert","last_dispatch_duration_us":9508913,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":13486570,"total_server_duration_us":906735,"encode_duration_us":27724,"operation_id":"0x7abe","timeout_ms":10000,"last_server_duration_us":154993,"total_duration_us":10009890},{"operation_name":"upsert","last_dispatch_duration_us":1499441,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":7033839,"total_server_duration_us":1503624,"encode_duration_us":17026,"operation_id":"0x7bc7","timeout_ms":10000,"last_server_duration_us":146617,"total_duration_us":10009607},{"operation_name":"upsert","last_dispatch_duration_us":9207074,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12028454,"total_server_duration_us":319433,"encode_duration_us":16569,"operation_id":"0x7a8b","timeout_ms":10000,"last_server_duration_us":22554,"total_duration_us":10008454},{"operation_name":"upsert","last_dispatch_duration_us":10846740,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10846740,"total_server_duration_us":9066042,"encode_duration_us":16632,"operation_id":"0x7a8e","timeout_ms":10000,"last_server_duration_us":9066042,"total_duration_us":10007790},{"operation_name":"upsert","last_dispatch_duration_us":937077,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":3993794,"total_server_duration_us":177934,"encode_duration_us":16636,"operation_id":"0x7c41","timeout_ms":10000,"last_server_duration_us":54456,"total_duration_us":10006259},{"operation_name":"upsert","last_dispatch_duration_us":3800720,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5396670,"total_server_duration_us":3588459,"encode_duration_us":16452,"operation_id":"0x7bbf","timeout_ms":10000,"last_server_duration_us":3243626,"total_duration_us":10006232},{"operation_name":"upsert","last_dispatch_duration_us":8784716,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8784716,"total_server_duration_us":41337,"encode_duration_us":17558,"operation_id":"0x7a90","timeout_ms":10000,"last_server_duration_us":41337,"total_duration_us":10005707},{"operation_name":"upsert","last_dispatch_duration_us":9383987,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12005341,"total_server_duration_us":1486890,"encode_duration_us":17648,"operation_id":"0x7a8a","timeout_ms":10000,"last_server_duration_us":737206,"total_duration_us":10005415},{"operation_name":"upsert","last_dispatch_duration_us":8014025,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8660133,"total_server_duration_us":264568,"encode_duration_us":16402,"operation_id":"0x7c83","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10002525}],"total_count":19}} INFO [2022-01-10 05:46:52,455] jepsen worker 26 - jepsen.util 3716 :info :write [8 40] :RequestTimeoutException INFO [2022-01-10 05:46:52,456] jepsen worker 26 - jepsen.util 3806 :invoke :read [8 nil] INFO [2022-01-10 05:46:52,580] jepsen worker 85 - jepsen.util 4315 :info :write [28 8] :RequestTimeoutException INFO [2022-01-10 05:46:52,581] jepsen worker 85 - jepsen.util 4405 :invoke :read [28 nil] INFO [2022-01-10 05:46:52,585] jepsen worker 76 - jepsen.util 3496 :info :write [25 4] :RequestTimeoutException INFO [2022-01-10 05:46:52,586] jepsen worker 76 - jepsen.util 3586 :invoke :read [25 nil] INFO [2022-01-10 05:46:52,612] jepsen worker 34 - jepsen.util 3904 :info :write [11 5] :RequestTimeoutException INFO [2022-01-10 05:46:52,612] jepsen worker 34 - jepsen.util 3994 :invoke :read [11 nil] INFO [2022-01-10 05:46:52,651] jepsen worker 6 - jepsen.util 4056 :info :write [2 15] :RequestTimeoutException INFO [2022-01-10 05:46:52,652] jepsen worker 6 - jepsen.util 4146 :invoke :read [2 nil] INFO [2022-01-10 05:46:52,676] jepsen worker 47 - jepsen.util 4007 :info :write [15 34] :RequestTimeoutException INFO [2022-01-10 05:46:52,677] jepsen worker 47 - jepsen.util 4097 :invoke :write [15 15] INFO [2022-01-10 05:46:52,679] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:52,679] jepsen worker nemesis - couchbase.util Rebalance check count: 561 INFO [2022-01-10 05:46:52,679] jepsen worker nemesis - couchbase.util Rebalance stuck count: 471 INFO [2022-01-10 05:46:52,839] jepsen worker 14 - jepsen.util 4964 :ok :read [4 3] INFO [2022-01-10 05:46:52,840] jepsen worker 13 - jepsen.util 4693 :ok :read [4 3] INFO [2022-01-10 05:46:52,840] jepsen worker 14 - jepsen.util 4964 :invoke :read [4 nil] INFO [2022-01-10 05:46:52,840] jepsen worker 13 - jepsen.util 4693 :invoke :write [4 32] INFO [2022-01-10 05:46:52,976] jepsen worker 54 - jepsen.util 4014 :info :write [18 38] :RequestTimeoutException INFO [2022-01-10 05:46:52,976] jepsen worker 54 - jepsen.util 4104 :invoke :read [18 nil] INFO [2022-01-10 05:46:53,191] jepsen worker 11 - jepsen.util 3791 :info :write [3 5] :RequestTimeoutException INFO [2022-01-10 05:46:53,192] jepsen worker 11 - jepsen.util 3881 :invoke :write [3 26] INFO [2022-01-10 05:46:53,228] jepsen worker 66 - jepsen.util 3756 :ok :read [22 41] INFO [2022-01-10 05:46:53,229] jepsen worker 66 - jepsen.util 3756 :invoke :write [22 48] INFO [2022-01-10 05:46:53,244] jepsen worker 69 - jepsen.util 3489 :info :write [23 21] :RequestTimeoutException INFO [2022-01-10 05:46:53,245] jepsen worker 69 - jepsen.util 3579 :invoke :read [23 nil] INFO [2022-01-10 05:46:53,255] jepsen worker 56 - jepsen.util 4286 :info :write [18 10] :RequestTimeoutException INFO [2022-01-10 05:46:53,255] jepsen worker 56 - jepsen.util 4376 :invoke :read [18 nil] INFO [2022-01-10 05:46:53,296] jepsen worker 17 - jepsen.util 3887 :info :write [5 32] :RequestTimeoutException INFO [2022-01-10 05:46:53,297] jepsen worker 17 - jepsen.util 3977 :invoke :read [5 nil] INFO [2022-01-10 05:46:53,429] jepsen worker 55 - jepsen.util 4375 :info :write [18 36] :RequestTimeoutException INFO [2022-01-10 05:46:53,430] jepsen worker 55 - jepsen.util 4465 :invoke :read [18 nil] INFO [2022-01-10 05:46:53,497] jepsen worker 55 - jepsen.util 4465 :ok :read [18 14] INFO [2022-01-10 05:46:53,497] jepsen worker 55 - jepsen.util 4465 :invoke :read [18 nil] INFO [2022-01-10 05:46:53,529] jepsen worker 55 - jepsen.util 4465 :ok :read [18 14] INFO [2022-01-10 05:46:53,529] jepsen worker 55 - jepsen.util 4465 :invoke :write [18 4] INFO [2022-01-10 05:46:53,685] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:53,685] jepsen worker nemesis - couchbase.util Rebalance check count: 562 INFO [2022-01-10 05:46:53,685] jepsen worker nemesis - couchbase.util Rebalance stuck count: 472 INFO [2022-01-10 05:46:53,741] jepsen worker 25 - jepsen.util 3805 :info :write [8 23] :RequestTimeoutException INFO [2022-01-10 05:46:53,742] jepsen worker 25 - jepsen.util 3895 :invoke :read [8 nil] INFO [2022-01-10 05:46:53,826] jepsen worker 78 - jepsen.util 4848 :info :write [26 15] :RequestTimeoutException INFO [2022-01-10 05:46:53,827] jepsen worker 78 - jepsen.util 4938 :invoke :write [26 48] INFO [2022-01-10 05:46:53,903] jepsen worker 27 - jepsen.util 4257 :ok :read [9 33] INFO [2022-01-10 05:46:53,903] jepsen worker 27 - jepsen.util 4257 :invoke :write [9 34] INFO [2022-01-10 05:46:53,912] jepsen worker 85 - jepsen.util 4405 :ok :read [28 45] INFO [2022-01-10 05:46:53,912] jepsen worker 85 - jepsen.util 4405 :invoke :read [28 nil] INFO [2022-01-10 05:46:53,929] jepsen worker 68 - jepsen.util 4118 :info :write [22 24] :RequestTimeoutException INFO [2022-01-10 05:46:53,930] jepsen worker 68 - jepsen.util 4208 :invoke :read [22 nil] INFO [2022-01-10 05:46:54,066] jepsen worker 86 - jepsen.util 3416 :info :write [28 25] :RequestTimeoutException INFO [2022-01-10 05:46:54,067] jepsen worker 86 - jepsen.util 3506 :invoke :read [28 nil] INFO [2022-01-10 05:46:54,392] jepsen worker 10 - jepsen.util 3970 :ok :read [3 12] INFO [2022-01-10 05:46:54,393] jepsen worker 10 - jepsen.util 3970 :invoke :read [3 nil] INFO [2022-01-10 05:46:54,399] jepsen worker 56 - jepsen.util 4376 :ok :read [18 14] INFO [2022-01-10 05:46:54,399] jepsen worker 56 - jepsen.util 4376 :invoke :read [18 nil] INFO [2022-01-10 05:46:54,415] jepsen worker 10 - jepsen.util 3970 :ok :read [3 12] INFO [2022-01-10 05:46:54,416] jepsen worker 10 - jepsen.util 3970 :invoke :read [3 nil] INFO [2022-01-10 05:46:54,427] jepsen worker 56 - jepsen.util 4376 :ok :read [18 14] INFO [2022-01-10 05:46:54,427] jepsen worker 56 - jepsen.util 4376 :invoke :read [18 nil] INFO [2022-01-10 05:46:54,437] jepsen worker 10 - jepsen.util 3970 :ok :read [3 12] INFO [2022-01-10 05:46:54,438] jepsen worker 10 - jepsen.util 3970 :invoke :write [3 37] INFO [2022-01-10 05:46:54,449] jepsen worker 56 - jepsen.util 4376 :ok :read [18 14] INFO [2022-01-10 05:46:54,449] jepsen worker 56 - jepsen.util 4376 :invoke :write [18 17] INFO [2022-01-10 05:46:54,615] jepsen worker 17 - jepsen.util 3977 :ok :read [5 25] INFO [2022-01-10 05:46:54,616] jepsen worker 17 - jepsen.util 3977 :invoke :write [5 27] INFO [2022-01-10 05:46:54,650] jepsen worker 20 - jepsen.util 3710 :info :write [6 10] :RequestTimeoutException INFO [2022-01-10 05:46:54,688] jepsen worker 20 - jepsen.util 3800 :invoke :read [6 nil] INFO [2022-01-10 05:46:54,691] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:54,691] jepsen worker nemesis - couchbase.util Rebalance check count: 563 INFO [2022-01-10 05:46:54,691] jepsen worker nemesis - couchbase.util Rebalance stuck count: 473 INFO [2022-01-10 05:46:54,742] jepsen worker 43 - jepsen.util 3913 :ok :read [14 14] INFO [2022-01-10 05:46:54,743] jepsen worker 43 - jepsen.util 3913 :invoke :read [14 nil] INFO [2022-01-10 05:46:54,787] jepsen worker 30 - jepsen.util 3720 :info :write [10 11] :SyncWriteAmbiguous INFO [2022-01-10 05:46:54,789] jepsen worker 30 - jepsen.util 3810 :invoke :read [10 nil] INFO [2022-01-10 05:46:54,811] jepsen worker 12 - jepsen.util 4512 :info :write [4 39] :RequestTimeoutException INFO [2022-01-10 05:46:54,811] jepsen worker 12 - jepsen.util 4602 :invoke :read [4 nil] INFO [2022-01-10 05:46:54,820] jepsen worker 30 - jepsen.util 3810 :ok :read [10 36] INFO [2022-01-10 05:46:54,820] jepsen worker 30 - jepsen.util 3810 :invoke :write [10 17] INFO [2022-01-10 05:46:54,824] jepsen worker 1 - jepsen.util 3961 :info :write [0 41] :RequestTimeoutException INFO [2022-01-10 05:46:54,825] jepsen worker 1 - jepsen.util 4051 :invoke :read [0 nil] INFO [2022-01-10 05:46:54,936] jepsen worker 54 - jepsen.util 4104 :ok :read [18 14] INFO [2022-01-10 05:46:54,936] jepsen worker 54 - jepsen.util 4104 :invoke :write [18 49] INFO [2022-01-10 05:46:55,019] jepsen worker 23 - jepsen.util 4073 :info :write [7 39] :RequestTimeoutException INFO [2022-01-10 05:46:55,020] jepsen worker 23 - jepsen.util 4163 :invoke :write [7 36] INFO [2022-01-10 05:46:55,030] jepsen worker 75 - jepsen.util 3675 :info :write [25 1] :RequestTimeoutException INFO [2022-01-10 05:46:55,030] jepsen worker 75 - jepsen.util 3765 :invoke :write [25 13] INFO [2022-01-10 05:46:55,104] jepsen worker 12 - jepsen.util 4602 :ok :read [4 3] INFO [2022-01-10 05:46:55,105] jepsen worker 12 - jepsen.util 4602 :invoke :read [4 nil] INFO [2022-01-10 05:46:55,222] jepsen worker 28 - jepsen.util 3808 :info :write [9 42] :RequestTimeoutException INFO [2022-01-10 05:46:55,223] jepsen worker 28 - jepsen.util 3898 :invoke :write [9 43] INFO [2022-01-10 05:46:55,370] jepsen worker 64 - jepsen.util 4024 :info :write [21 3] :RequestTimeoutException INFO [2022-01-10 05:46:55,371] jepsen worker 64 - jepsen.util 4114 :invoke :write [21 43] INFO [2022-01-10 05:46:55,475] jepsen worker 15 - jepsen.util 3525 :info :write [5 0] :RequestTimeoutException INFO [2022-01-10 05:46:55,476] jepsen worker 15 - jepsen.util 3615 :invoke :write [5 46] INFO [2022-01-10 05:46:55,530] jepsen worker 21 - jepsen.util 4071 :info :write [7 33] :RequestTimeoutException INFO [2022-01-10 05:46:55,530] jepsen worker 21 - jepsen.util 4161 :invoke :read [7 nil] INFO [2022-01-10 05:46:55,635] jepsen worker 16 - jepsen.util 4066 :info :write [5 4] :RequestTimeoutException INFO [2022-01-10 05:46:55,636] jepsen worker 16 - jepsen.util 4156 :invoke :read [5 nil] INFO [2022-01-10 05:46:55,686] jepsen worker 45 - jepsen.util 4275 :info :write [15 27] :RequestTimeoutException INFO [2022-01-10 05:46:55,687] jepsen worker 45 - jepsen.util 4365 :invoke :write [15 35] INFO [2022-01-10 05:46:55,701] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:55,701] jepsen worker nemesis - couchbase.util Rebalance check count: 564 INFO [2022-01-10 05:46:55,701] jepsen worker nemesis - couchbase.util Rebalance stuck count: 474 INFO [2022-01-10 05:46:55,712] jepsen worker 88 - jepsen.util 3868 :info :write [29 10] :RequestTimeoutException INFO [2022-01-10 05:46:55,712] jepsen worker 88 - jepsen.util 3958 :invoke :write [29 4] INFO [2022-01-10 05:46:56,122] jepsen worker 12 - jepsen.util 4602 :ok :read [4 3] INFO [2022-01-10 05:46:56,122] jepsen worker 12 - jepsen.util 4602 :invoke :write [4 1] INFO [2022-01-10 05:46:56,341] jepsen worker 14 - jepsen.util 4964 :ok :read [4 3] INFO [2022-01-10 05:46:56,341] jepsen worker 14 - jepsen.util 4964 :invoke :read [4 nil] INFO [2022-01-10 05:46:56,352] jepsen worker 68 - jepsen.util 4208 :ok :read [22 41] INFO [2022-01-10 05:46:56,352] jepsen worker 68 - jepsen.util 4208 :invoke :read [22 nil] INFO [2022-01-10 05:46:56,706] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:56,706] jepsen worker nemesis - couchbase.util Rebalance check count: 565 INFO [2022-01-10 05:46:56,706] jepsen worker nemesis - couchbase.util Rebalance stuck count: 475 INFO [2022-01-10 05:46:56,780] jepsen worker 73 - jepsen.util 3673 :ok :read [24 18] INFO [2022-01-10 05:46:56,781] jepsen worker 73 - jepsen.util 3673 :invoke :write [24 31] INFO [2022-01-10 05:46:56,786] jepsen worker 18 - jepsen.util 3978 :info :write [6 12] :RequestTimeoutException INFO [2022-01-10 05:46:56,786] jepsen worker 71 - jepsen.util 3491 :info :write [23 40] :RequestTimeoutException INFO [2022-01-10 05:46:56,787] jepsen worker 18 - jepsen.util 4068 :invoke :read [6 nil] INFO [2022-01-10 05:46:56,787] jepsen worker 71 - jepsen.util 3581 :invoke :write [23 27] INFO [2022-01-10 05:46:56,791] jepsen worker 87 - jepsen.util 3867 :info :write [29 48] :RequestTimeoutException INFO [2022-01-10 05:46:56,792] jepsen worker 87 - jepsen.util 3957 :invoke :write [29 35] INFO [2022-01-10 05:46:56,802] jepsen worker 4 - jepsen.util 4594 :info :write [1 26] :RequestTimeoutException INFO [2022-01-10 05:46:56,803] jepsen worker 4 - jepsen.util 4684 :invoke :read [1 nil] INFO [2022-01-10 05:46:56,846] jepsen worker 39 - jepsen.util 4539 :info :write [13 12] :RequestTimeoutException INFO [2022-01-10 05:46:56,846] jepsen worker 39 - jepsen.util 4629 :invoke :read [13 nil] INFO [2022-01-10 05:46:56,916] jepsen worker 83 - jepsen.util 4493 :info :write [27 2] :RequestTimeoutException INFO [2022-01-10 05:46:56,917] jepsen worker 83 - jepsen.util 4583 :invoke :read [27 nil] INFO [2022-01-10 05:46:57,017] jepsen worker 19 - jepsen.util 4159 :info :write [6 45] :RequestTimeoutException INFO [2022-01-10 05:46:57,017] jepsen worker 19 - jepsen.util 4249 :invoke :read [6 nil] INFO [2022-01-10 05:46:57,035] jepsen worker 70 - jepsen.util 3940 :info :write [23 10] :RequestTimeoutException INFO [2022-01-10 05:46:57,036] jepsen worker 70 - jepsen.util 4030 :invoke :write [23 22] INFO [2022-01-10 05:46:57,195] jepsen worker 43 - jepsen.util 3913 :ok :read [14 14] INFO [2022-01-10 05:46:57,196] jepsen worker 43 - jepsen.util 3913 :invoke :write [14 48] INFO [2022-01-10 05:46:57,250] jepsen worker 53 - jepsen.util 4823 :info :write [17 8] :RequestTimeoutException INFO [2022-01-10 05:46:57,251] jepsen worker 53 - jepsen.util 4913 :invoke :read [17 nil] INFO [2022-01-10 05:46:57,301] jepsen worker 67 - jepsen.util 3847 :info :write [22 25] :RequestTimeoutException INFO [2022-01-10 05:46:57,302] jepsen worker 67 - jepsen.util 3937 :invoke :write [22 30] INFO [2022-01-10 05:46:57,314] jepsen worker 57 - jepsen.util 4827 :info :write [19 7] :RequestTimeoutException INFO [2022-01-10 05:46:57,315] jepsen worker 57 - jepsen.util 4917 :invoke :read [19 nil] INFO [2022-01-10 05:46:57,346] jepsen worker 38 - jepsen.util 3818 :fail :read [12 nil] :UnambiguousTimeoutException INFO [2022-01-10 05:46:57,346] jepsen worker 38 - jepsen.util 3818 :invoke :write [12 47] INFO [2022-01-10 05:46:57,384] jepsen worker 65 - jepsen.util 3845 :ok :read [21 7] INFO [2022-01-10 05:46:57,385] jepsen worker 65 - jepsen.util 3845 :invoke :write [21 14] INFO [2022-01-10 05:46:57,496] jepsen worker 9 - jepsen.util 4509 :info :write [3 1] :RequestTimeoutException INFO [2022-01-10 05:46:57,496] jepsen worker 9 - jepsen.util 4599 :invoke :read [3 nil] INFO [2022-01-10 05:46:57,572] jepsen worker 75 - jepsen.util 3765 :fail :write [25 13] :ServerOutOfMemoryException INFO [2022-01-10 05:46:57,572] jepsen worker 34 - jepsen.util 3994 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:46:57,572] jepsen worker 75 - jepsen.util 3765 :invoke :write [25 17] INFO [2022-01-10 05:46:57,573] jepsen worker 64 - jepsen.util 4114 :fail :write [21 43] :ServerOutOfMemoryException INFO [2022-01-10 05:46:57,574] jepsen worker 64 - jepsen.util 4114 :invoke :read [21 nil] INFO [2022-01-10 05:46:57,574] jepsen worker 34 - jepsen.util 3994 :invoke :write [11 14] INFO [2022-01-10 05:46:57,607] jepsen worker 16 - jepsen.util 4156 :ok :read [5 25] INFO [2022-01-10 05:46:57,608] jepsen worker 16 - jepsen.util 4156 :invoke :read [5 nil] INFO [2022-01-10 05:46:57,621] jepsen worker 14 - jepsen.util 4964 :ok :read [4 3] INFO [2022-01-10 05:46:57,622] jepsen worker 14 - jepsen.util 4964 :invoke :read [4 nil] INFO [2022-01-10 05:46:57,640] jepsen worker 84 - jepsen.util 3954 :info :write [28 1] :RequestTimeoutException INFO [2022-01-10 05:46:57,640] jepsen worker 84 - jepsen.util 4044 :invoke :read [28 nil] INFO [2022-01-10 05:46:57,712] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:57,712] jepsen worker nemesis - couchbase.util Rebalance check count: 566 INFO [2022-01-10 05:46:57,712] jepsen worker nemesis - couchbase.util Rebalance stuck count: 476 INFO [2022-01-10 05:46:57,796] jepsen worker 51 - jepsen.util 4821 :info :write [17 49] :RequestTimeoutException INFO [2022-01-10 05:46:57,797] jepsen worker 51 - jepsen.util 4911 :invoke :read [17 nil] INFO [2022-01-10 05:46:57,931] jepsen worker 58 - jepsen.util 3748 :info :write [19 11] :RequestTimeoutException INFO [2022-01-10 05:46:57,932] jepsen worker 58 - jepsen.util 3838 :invoke :read [19 nil] INFO [2022-01-10 05:46:58,028] jepsen worker 79 - jepsen.util 4759 :ok :read [26 10] INFO [2022-01-10 05:46:58,029] jepsen worker 79 - jepsen.util 4759 :invoke :read [26 nil] INFO [2022-01-10 05:46:58,036] jepsen worker 62 - jepsen.util 4472 :info :write [20 9] :RequestTimeoutException INFO [2022-01-10 05:46:58,037] jepsen worker 62 - jepsen.util 4562 :invoke :read [20 nil] INFO [2022-01-10 05:46:58,044] jepsen worker 69 - jepsen.util 3579 :ok :read [23 26] INFO [2022-01-10 05:46:58,044] jepsen worker 69 - jepsen.util 3579 :invoke :write [23 39] INFO [2022-01-10 05:46:58,055] jepsen worker 79 - jepsen.util 4759 :ok :read [26 10] INFO [2022-01-10 05:46:58,056] jepsen worker 79 - jepsen.util 4759 :invoke :write [26 39] INFO [2022-01-10 05:46:58,239] jepsen worker 74 - jepsen.util 4574 :info :write [24 44] :RequestTimeoutException INFO [2022-01-10 05:46:58,240] jepsen worker 74 - jepsen.util 4664 :invoke :read [24 nil] INFO [2022-01-10 05:46:58,354] jepsen worker 68 - jepsen.util 4208 :ok :read [22 41] INFO [2022-01-10 05:46:58,354] jepsen worker 68 - jepsen.util 4208 :invoke :read [22 nil] INFO [2022-01-10 05:46:58,439] jepsen worker 14 - jepsen.util 4964 :ok :read [4 3] INFO [2022-01-10 05:46:58,440] jepsen worker 14 - jepsen.util 4964 :invoke :write [4 43] INFO [2022-01-10 05:46:58,531] jepsen worker 41 - jepsen.util 3911 :info :write [13 0] :RequestTimeoutException INFO [2022-01-10 05:46:58,532] jepsen worker 41 - jepsen.util 4001 :invoke :write [13 42] INFO [2022-01-10 05:46:58,534] jepsen worker 82 - jepsen.util 4132 :info :write [27 26] :RequestTimeoutException INFO [2022-01-10 05:46:58,535] jepsen worker 82 - jepsen.util 4222 :invoke :write [27 10] INFO [2022-01-10 05:46:58,718] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:58,718] jepsen worker nemesis - couchbase.util Rebalance check count: 567 INFO [2022-01-10 05:46:58,718] jepsen worker nemesis - couchbase.util Rebalance stuck count: 477 INFO [2022-01-10 05:46:59,016] jepsen worker 63 - jepsen.util 3843 :ok :read [21 7] INFO [2022-01-10 05:46:59,017] jepsen worker 63 - jepsen.util 3843 :invoke :write [21 4] INFO [2022-01-10 05:46:59,036] jepsen worker 59 - jepsen.util 4109 :info :write [19 32] :RequestTimeoutException INFO [2022-01-10 05:46:59,037] jepsen worker 59 - jepsen.util 4199 :invoke :write [19 19] INFO [2022-01-10 05:46:59,045] jepsen worker 42 - jepsen.util 3732 :info :write [14 31] :RequestTimeoutException INFO [2022-01-10 05:46:59,047] jepsen worker 42 - jepsen.util 3822 :invoke :read [14 nil] INFO [2022-01-10 05:46:59,049] jepsen worker 72 - jepsen.util 4392 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:46:59,050] jepsen worker 72 - jepsen.util 4392 :invoke :read [24 nil] INFO [2022-01-10 05:46:59,067] jepsen worker 51 - jepsen.util 4911 :ok :read [17 41] INFO [2022-01-10 05:46:59,067] jepsen worker 51 - jepsen.util 4911 :invoke :write [17 19] INFO [2022-01-10 05:46:59,075] jepsen worker 8 - jepsen.util 4418 :ok :read [2 14] INFO [2022-01-10 05:46:59,076] jepsen worker 8 - jepsen.util 4418 :invoke :read [2 nil] INFO [2022-01-10 05:46:59,702] jepsen worker 85 - jepsen.util 4405 :ok :read [28 45] INFO [2022-01-10 05:46:59,703] jepsen worker 85 - jepsen.util 4405 :invoke :write [28 13] INFO [2022-01-10 05:46:59,728] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:46:59,734] jepsen worker 16 - jepsen.util 4156 :ok :read [5 25] INFO [2022-01-10 05:47:00,035] jepsen worker 77 - jepsen.util 4487 :info :write [25 39] :RequestTimeoutException INFO [2022-01-10 05:47:00,105] jepsen worker 89 - jepsen.util 4499 :info :write [29 47] :SyncWriteAmbiguous INFO [2022-01-10 05:47:00,162] jepsen worker 29 - jepsen.util 3989 :ok :read [9 33] INFO [2022-01-10 05:47:00,235] jepsen worker 89 - jepsen.util 4589 :invoke :read [29 nil] INFO [2022-01-10 05:47:00,175] jepsen worker 49 - jepsen.util 3919 :ok :read [16 38] INFO [2022-01-10 05:47:00,181] jepsen worker 68 - jepsen.util 4208 :ok :read [22 41] INFO [2022-01-10 05:47:00,235] jepsen worker nemesis - couchbase.util Rebalance check count: 568 INFO [2022-01-10 05:47:00,235] jepsen worker 49 - jepsen.util 3919 :invoke :read [16 nil] INFO [2022-01-10 05:47:00,235] jepsen worker 77 - jepsen.util 4577 :invoke :read [25 nil] INFO [2022-01-10 05:47:00,235] jepsen worker 68 - jepsen.util 4208 :invoke :read [22 nil] INFO [2022-01-10 05:47:00,235] jepsen worker 29 - jepsen.util 3989 :invoke :write [9 23] INFO [2022-01-10 05:47:00,235] jepsen worker nemesis - couchbase.util Rebalance stuck count: 478 INFO [2022-01-10 05:47:00,235] jepsen worker 16 - jepsen.util 4156 :invoke :read [5 nil] INFO [2022-01-10 05:47:00,246] jepsen worker 24 - jepsen.util 3714 :info :write [8 13] :RequestTimeoutException INFO [2022-01-10 05:47:00,247] jepsen worker 24 - jepsen.util 3804 :invoke :read [8 nil] INFO [2022-01-10 05:47:00,270] jepsen worker 46 - jepsen.util 4186 :info :write [15 32] :RequestTimeoutException INFO [2022-01-10 05:47:00,271] jepsen worker 46 - jepsen.util 4276 :invoke :read [15 nil] INFO [2022-01-10 05:47:00,381] jepsen worker 48 - jepsen.util 3738 :info :write [16 5] :RequestTimeoutException INFO [2022-01-10 05:47:00,382] jepsen worker 48 - jepsen.util 3828 :invoke :write [16 28] INFO [2022-01-10 05:47:00,392] jepsen worker 22 - jepsen.util 3802 :info :write [7 19] :RequestTimeoutException INFO [2022-01-10 05:47:00,392] jepsen worker 22 - jepsen.util 3892 :invoke :read [7 nil] INFO [2022-01-10 05:47:00,470] jepsen worker 1 - jepsen.util 4051 :ok :read [0 46] INFO [2022-01-10 05:47:00,471] jepsen worker 1 - jepsen.util 4051 :invoke :read [0 nil] INFO [2022-01-10 05:47:00,475] jepsen worker 0 - jepsen.util 3960 :ok :read [0 46] INFO [2022-01-10 05:47:00,475] jepsen worker 0 - jepsen.util 3960 :invoke :read [0 nil] INFO [2022-01-10 05:47:00,537] jepsen worker 7 - jepsen.util 4147 :info :write [2 1] :RequestTimeoutException INFO [2022-01-10 05:47:00,537] jepsen worker 7 - jepsen.util 4237 :invoke :write [2 31] WARN [2022-01-10 05:47:00,551] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10512281,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14721046,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":10988652,"encode_duration_us":18184,"operation_id":"0x7d1c","timeout_ms":10000,"last_server_duration_us":10217708,"total_duration_us":10004805},{"operation_name":"upsert","last_dispatch_duration_us":9038272,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9488591,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9278810,"encode_duration_us":16990,"operation_id":"0x7cc8","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10009848},{"operation_name":"upsert","last_dispatch_duration_us":10511193,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14948005,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10281786,"encode_duration_us":16915,"operation_id":"0x7bed","timeout_ms":10000,"last_server_duration_us":9154436,"total_duration_us":10007975}],"total_count":3}} WARN [2022-01-10 05:47:00,552] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":773302,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":22095658,"total_server_duration_us":1175,"operation_id":"0x795c","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":30007896},{"operation_name":"upsert","last_dispatch_duration_us":9952245,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":12482067,"total_server_duration_us":1799570,"encode_duration_us":17913,"operation_id":"0x7c92","timeout_ms":10000,"last_server_duration_us":801975,"total_duration_us":10010492},{"operation_name":"upsert","last_dispatch_duration_us":1766443,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":3906944,"total_server_duration_us":720780,"encode_duration_us":17157,"operation_id":"0x7d8a","timeout_ms":10000,"last_server_duration_us":140962,"total_duration_us":10009917},{"operation_name":"upsert","last_dispatch_duration_us":9415271,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12283410,"total_server_duration_us":626442,"encode_duration_us":18013,"operation_id":"0x7c81","timeout_ms":10000,"last_server_duration_us":222561,"total_duration_us":10009871},{"operation_name":"upsert","last_dispatch_duration_us":9038272,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9488591,"total_server_duration_us":9278810,"encode_duration_us":16990,"operation_id":"0x7cc8","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10009848},{"operation_name":"upsert","last_dispatch_duration_us":11064985,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":13763836,"total_server_duration_us":1337096,"encode_duration_us":17430,"operation_id":"0x7c8f","timeout_ms":10000,"last_server_duration_us":577633,"total_duration_us":10005867},{"operation_name":"upsert","last_dispatch_duration_us":10512281,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14721046,"total_server_duration_us":10988652,"encode_duration_us":18184,"operation_id":"0x7d1c","timeout_ms":10000,"last_server_duration_us":10217708,"total_duration_us":10004805},{"operation_name":"upsert","last_dispatch_duration_us":2289315,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":10492748,"total_server_duration_us":1186194,"encode_duration_us":21071,"operation_id":"0x7d80","timeout_ms":10000,"last_server_duration_us":200218,"total_duration_us":10004577},{"operation_name":"upsert","last_dispatch_duration_us":2363241,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":7385631,"total_server_duration_us":784494,"encode_duration_us":16983,"operation_id":"0x7c52","timeout_ms":10000,"last_server_duration_us":210405,"total_duration_us":10003292},{"operation_name":"upsert","last_dispatch_duration_us":9375201,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14740254,"total_server_duration_us":1200761,"encode_duration_us":17114,"operation_id":"0x7d2e","timeout_ms":10000,"last_server_duration_us":97947,"total_duration_us":10002583}],"total_count":25}} INFO [2022-01-10 05:47:00,619] jepsen worker 61 - jepsen.util 4201 :ok :read [20 :nil] INFO [2022-01-10 05:47:00,619] jepsen worker 61 - jepsen.util 4201 :invoke :write [20 6] INFO [2022-01-10 05:47:00,631] jepsen worker 35 - jepsen.util 3815 :ok :read [11 21] INFO [2022-01-10 05:47:00,631] jepsen worker 35 - jepsen.util 3815 :invoke :read [11 nil] INFO [2022-01-10 05:47:00,763] jepsen worker 53 - jepsen.util 4913 :ok :read [17 41] INFO [2022-01-10 05:47:00,763] jepsen worker 53 - jepsen.util 4913 :invoke :read [17 nil] INFO [2022-01-10 05:47:00,776] jepsen worker 81 - jepsen.util 4401 :info :write [27 43] :RequestTimeoutException INFO [2022-01-10 05:47:00,776] jepsen worker 81 - jepsen.util 4491 :invoke :write [27 19] INFO [2022-01-10 05:47:00,840] jepsen worker 84 - jepsen.util 4044 :ok :read [28 45] INFO [2022-01-10 05:47:00,841] jepsen worker 84 - jepsen.util 4044 :invoke :write [28 49] INFO [2022-01-10 05:47:00,981] jepsen worker 33 - jepsen.util 4173 :info :write [11 4] :RequestTimeoutException INFO [2022-01-10 05:47:00,982] jepsen worker 33 - jepsen.util 4263 :invoke :write [11 33] INFO [2022-01-10 05:47:01,021] jepsen worker 40 - jepsen.util 3550 :info :write [13 12] :RequestTimeoutException INFO [2022-01-10 05:47:01,021] jepsen worker 40 - jepsen.util 3640 :invoke :write [13 18] INFO [2022-01-10 05:47:01,026] jepsen worker 68 - jepsen.util 4208 :ok :read [22 41] INFO [2022-01-10 05:47:01,027] jepsen worker 68 - jepsen.util 4208 :invoke :write [22 38] INFO [2022-01-10 05:47:01,185] jepsen worker 16 - jepsen.util 4156 :ok :read [5 25] INFO [2022-01-10 05:47:01,185] jepsen worker 16 - jepsen.util 4156 :invoke :read [5 nil] INFO [2022-01-10 05:47:01,244] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:01,244] jepsen worker nemesis - couchbase.util Rebalance check count: 569 INFO [2022-01-10 05:47:01,244] jepsen worker nemesis - couchbase.util Rebalance stuck count: 479 INFO [2022-01-10 05:47:01,412] jepsen worker 52 - jepsen.util 5092 :info :write [17 0] :RequestTimeoutException INFO [2022-01-10 05:47:01,413] jepsen worker 52 - jepsen.util 5182 :invoke :write [17 34] INFO [2022-01-10 05:47:01,466] jepsen worker 32 - jepsen.util 4262 :info :write [10 21] :RequestTimeoutException INFO [2022-01-10 05:47:01,466] jepsen worker 32 - jepsen.util 4352 :invoke :read [10 nil] INFO [2022-01-10 05:47:01,546] jepsen worker 60 - jepsen.util 4200 :info :write [20 40] :RequestTimeoutException INFO [2022-01-10 05:47:01,547] jepsen worker 60 - jepsen.util 4290 :invoke :write [20 34] INFO [2022-01-10 05:47:01,630] jepsen worker 80 - jepsen.util 4940 :info :write [26 34] :RequestTimeoutException INFO [2022-01-10 05:47:01,631] jepsen worker 80 - jepsen.util 5030 :invoke :read [26 nil] INFO [2022-01-10 05:47:01,635] jepsen worker 50 - jepsen.util 4190 :info :write [16 33] :RequestTimeoutException INFO [2022-01-10 05:47:01,636] jepsen worker 50 - jepsen.util 4280 :invoke :write [16 34] INFO [2022-01-10 05:47:01,663] jepsen worker 18 - jepsen.util 4068 :ok :read [6 25] INFO [2022-01-10 05:47:01,663] jepsen worker 18 - jepsen.util 4068 :invoke :write [6 42] INFO [2022-01-10 05:47:01,679] jepsen worker 19 - jepsen.util 4249 :ok :read [6 25] INFO [2022-01-10 05:47:01,680] jepsen worker 19 - jepsen.util 4249 :invoke :write [6 8] INFO [2022-01-10 05:47:01,836] jepsen worker 9 - jepsen.util 4599 :ok :read [3 12] INFO [2022-01-10 05:47:01,836] jepsen worker 9 - jepsen.util 4599 :invoke :read [3 nil] INFO [2022-01-10 05:47:01,873] jepsen worker 9 - jepsen.util 4599 :ok :read [3 12] INFO [2022-01-10 05:47:01,873] jepsen worker 9 - jepsen.util 4599 :invoke :read [3 nil] INFO [2022-01-10 05:47:01,893] jepsen worker 58 - jepsen.util 3838 :ok :read [19 31] INFO [2022-01-10 05:47:01,894] jepsen worker 58 - jepsen.util 3838 :invoke :write [19 39] INFO [2022-01-10 05:47:01,896] jepsen worker 0 - jepsen.util 3960 :ok :read [0 46] INFO [2022-01-10 05:47:01,896] jepsen worker 0 - jepsen.util 3960 :invoke :write [0 42] INFO [2022-01-10 05:47:01,899] jepsen worker 5 - jepsen.util 3875 :info :write [1 35] :RequestTimeoutException INFO [2022-01-10 05:47:01,900] jepsen worker 5 - jepsen.util 3965 :invoke :write [1 29] INFO [2022-01-10 05:47:01,916] jepsen worker 9 - jepsen.util 4599 :ok :read [3 12] INFO [2022-01-10 05:47:01,916] jepsen worker 9 - jepsen.util 4599 :invoke :read [3 nil] INFO [2022-01-10 05:47:01,952] jepsen worker 2 - jepsen.util 4322 :info :write [0 33] :RequestTimeoutException INFO [2022-01-10 05:47:01,952] jepsen worker 2 - jepsen.util 4412 :invoke :read [0 nil] INFO [2022-01-10 05:47:01,965] jepsen worker 9 - jepsen.util 4599 :ok :read [3 12] INFO [2022-01-10 05:47:01,966] jepsen worker 9 - jepsen.util 4599 :invoke :write [3 35] INFO [2022-01-10 05:47:02,011] jepsen worker 3 - jepsen.util 3873 :info :write [1 10] :RequestTimeoutException INFO [2022-01-10 05:47:02,012] jepsen worker 3 - jepsen.util 3963 :invoke :write [1 13] WARN [2022-01-10 05:47:02,054] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9060665,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13154657,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9276403,"encode_duration_us":16466,"operation_id":"0x7d8c","timeout_ms":10000,"last_server_duration_us":9047983,"total_duration_us":10006384}],"total_count":1}} INFO [2022-01-10 05:47:02,086] jepsen worker 16 - jepsen.util 4156 :ok :read [5 25] INFO [2022-01-10 05:47:02,086] jepsen worker 31 - jepsen.util 4351 :info :write [10 44] :RequestTimeoutException INFO [2022-01-10 05:47:02,086] jepsen worker 16 - jepsen.util 4156 :invoke :read [5 nil] INFO [2022-01-10 05:47:02,087] jepsen worker 31 - jepsen.util 4441 :invoke :read [10 nil] INFO [2022-01-10 05:47:02,110] jepsen worker 64 - jepsen.util 4114 :ok :read [21 7] INFO [2022-01-10 05:47:02,110] jepsen worker 64 - jepsen.util 4114 :invoke :read [21 nil] WARN [2022-01-10 05:47:02,137] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":8617581,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8617581,"total_server_duration_us":191893,"operation_id":"0x7d74","timeout_ms":30000,"last_server_duration_us":191893,"total_duration_us":12249576},{"operation_name":"get","last_dispatch_duration_us":8399558,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8399558,"total_server_duration_us":13223,"operation_id":"0x7d7d","timeout_ms":30000,"last_server_duration_us":13223,"total_duration_us":12041565},{"operation_name":"upsert","last_dispatch_duration_us":2006628,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11304123,"total_server_duration_us":1063548,"encode_duration_us":20198,"operation_id":"0x7dba","timeout_ms":10000,"last_server_duration_us":247048,"total_duration_us":10009946},{"operation_name":"upsert","last_dispatch_duration_us":8405581,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9435079,"total_server_duration_us":563270,"encode_duration_us":18123,"operation_id":"0x7cc5","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10009583},{"operation_name":"upsert","last_dispatch_duration_us":366578,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9745426,"total_server_duration_us":1521366,"encode_duration_us":17971,"operation_id":"0x7e39","timeout_ms":10000,"last_server_duration_us":163566,"total_duration_us":10009269},{"operation_name":"upsert","last_dispatch_duration_us":9870925,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":12719462,"total_server_duration_us":10713509,"encode_duration_us":21193,"operation_id":"0x7d15","timeout_ms":10000,"last_server_duration_us":9859238,"total_duration_us":10006629},{"operation_name":"upsert","last_dispatch_duration_us":1867801,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11517030,"total_server_duration_us":911644,"encode_duration_us":18459,"operation_id":"0x7d89","timeout_ms":10000,"last_server_duration_us":124227,"total_duration_us":10005993},{"operation_name":"upsert","last_dispatch_duration_us":2811100,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2811100,"total_server_duration_us":177489,"encode_duration_us":17991,"operation_id":"0x7cd8","timeout_ms":10000,"last_server_duration_us":177489,"total_duration_us":10004829},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":22052,"operation_id":"0x7e38","timeout_ms":10000,"total_duration_us":10004481},{"operation_name":"upsert","last_dispatch_duration_us":5561941,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5561941,"total_server_duration_us":226530,"encode_duration_us":18122,"operation_id":"0x7d29","timeout_ms":10000,"last_server_duration_us":226530,"total_duration_us":10001711}],"total_count":24}} WARN [2022-01-10 05:47:02,137] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9870925,"last_local_id":"EFE9906200000006/0000000021E2C07D","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":12719462,"last_local_socket":"172.23.104.79:44302","total_server_duration_us":10713509,"encode_duration_us":21193,"operation_id":"0x7d15","timeout_ms":10000,"last_server_duration_us":9859238,"total_duration_us":10006629},{"operation_name":"upsert","last_dispatch_duration_us":9064231,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10174931,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9591913,"encode_duration_us":16372,"operation_id":"0x7c80","timeout_ms":10000,"last_server_duration_us":9047983,"total_duration_us":10008329}],"total_count":2}} WARN [2022-01-10 05:47:02,243] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":3897210,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":12418197,"total_server_duration_us":1994463,"encode_duration_us":18212,"operation_id":"0x7ceb","timeout_ms":10000,"last_server_duration_us":1002076,"total_duration_us":10010166},{"operation_name":"upsert","last_dispatch_duration_us":1170329,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1170329,"total_server_duration_us":170761,"encode_duration_us":16973,"operation_id":"0x7e13","timeout_ms":10000,"last_server_duration_us":170761,"total_duration_us":10008832},{"operation_name":"upsert","last_dispatch_duration_us":4561492,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":11189107,"total_server_duration_us":1484159,"encode_duration_us":20197,"operation_id":"0x7d6b","timeout_ms":10000,"last_server_duration_us":377630,"total_duration_us":10007579},{"operation_name":"upsert","last_dispatch_duration_us":5441250,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":12920743,"total_server_duration_us":2863963,"encode_duration_us":21275,"operation_id":"0x7da7","timeout_ms":10000,"last_server_duration_us":1077381,"total_duration_us":10007152},{"operation_name":"upsert","last_dispatch_duration_us":3059840,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":12175020,"total_server_duration_us":1125195,"encode_duration_us":16380,"operation_id":"0x7cef","timeout_ms":10000,"last_server_duration_us":248198,"total_duration_us":10005875},{"operation_name":"upsert","last_dispatch_duration_us":5144880,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9949363,"total_server_duration_us":1162599,"encode_duration_us":16293,"operation_id":"0x7ddb","timeout_ms":10000,"last_server_duration_us":414388,"total_duration_us":10005183},{"operation_name":"upsert","last_dispatch_duration_us":5876343,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":13139590,"total_server_duration_us":1048144,"encode_duration_us":17127,"operation_id":"0x7d7c","timeout_ms":10000,"last_server_duration_us":356168,"total_duration_us":10004989},{"operation_name":"upsert","last_dispatch_duration_us":10557693,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":15055378,"total_server_duration_us":691170,"encode_duration_us":24166,"operation_id":"0x7d7f","timeout_ms":10000,"last_server_duration_us":403293,"total_duration_us":10003883},{"operation_name":"upsert","last_dispatch_duration_us":7136425,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":7554158,"total_server_duration_us":242805,"encode_duration_us":17052,"operation_id":"0x7e44","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10003696},{"operation_name":"upsert","last_dispatch_duration_us":10526514,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":18545104,"total_server_duration_us":9720246,"encode_duration_us":19635,"operation_id":"0x7c96","timeout_ms":10000,"last_server_duration_us":9523585,"total_duration_us":10002409}],"total_count":21}} WARN [2022-01-10 05:47:02,250] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12928514,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12928514,"total_server_duration_us":6530,"operation_id":"0x7d8e","timeout_ms":30000,"last_server_duration_us":6530,"total_duration_us":12930348},{"operation_name":"get","last_dispatch_duration_us":12260508,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12260508,"total_server_duration_us":568,"operation_id":"0x7db8","timeout_ms":30000,"last_server_duration_us":568,"total_duration_us":12260835},{"operation_name":"upsert","last_dispatch_duration_us":39562,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":8842013,"total_server_duration_us":275205,"encode_duration_us":22146,"operation_id":"0x7d73","timeout_ms":10000,"last_server_duration_us":26064,"total_duration_us":10009965},{"operation_name":"upsert","last_dispatch_duration_us":10947563,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13100638,"total_server_duration_us":369621,"encode_duration_us":19773,"operation_id":"0x7c94","timeout_ms":10000,"last_server_duration_us":199169,"total_duration_us":10009946},{"operation_name":"upsert","last_dispatch_duration_us":1164968,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":6723147,"total_server_duration_us":885556,"encode_duration_us":16514,"operation_id":"0x7e6a","timeout_ms":10000,"last_server_duration_us":144416,"total_duration_us":10009033},{"operation_name":"upsert","last_dispatch_duration_us":1325714,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8338879,"total_server_duration_us":2770315,"encode_duration_us":17137,"operation_id":"0x7d26","timeout_ms":10000,"last_server_duration_us":138619,"total_duration_us":10005904},{"operation_name":"upsert","last_dispatch_duration_us":11654892,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11654892,"total_server_duration_us":377355,"encode_duration_us":24111,"operation_id":"0x7e11","timeout_ms":10000,"last_server_duration_us":377355,"total_duration_us":10005694},{"operation_name":"upsert","last_dispatch_duration_us":14212992,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":17930116,"total_server_duration_us":834862,"encode_duration_us":19056,"operation_id":"0x7c95","timeout_ms":10000,"last_server_duration_us":404425,"total_duration_us":10004965},{"operation_name":"upsert","last_dispatch_duration_us":1130792,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":7724822,"total_server_duration_us":1446684,"encode_duration_us":16539,"operation_id":"0x7e10","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10004449},{"operation_name":"upsert","last_dispatch_duration_us":1400821,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":7269232,"total_server_duration_us":4917227,"encode_duration_us":16436,"operation_id":"0x7e43","timeout_ms":10000,"last_server_duration_us":391488,"total_duration_us":10004447}],"total_count":16}} INFO [2022-01-10 05:47:02,251] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:02,251] jepsen worker nemesis - couchbase.util Rebalance check count: 570 INFO [2022-01-10 05:47:02,251] jepsen worker nemesis - couchbase.util Rebalance stuck count: 480 WARN [2022-01-10 05:47:02,254] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2380876,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8055011,"total_server_duration_us":955803,"encode_duration_us":17133,"operation_id":"0x7d7b","timeout_ms":10000,"last_server_duration_us":538346,"total_duration_us":10010808},{"operation_name":"upsert","encode_duration_us":17141,"operation_id":"0x7d6a","timeout_ms":10000,"total_duration_us":10008278},{"operation_name":"upsert","last_dispatch_duration_us":6129503,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9922695,"total_server_duration_us":585767,"encode_duration_us":16891,"operation_id":"0x7e2a","timeout_ms":10000,"last_server_duration_us":239294,"total_duration_us":10008084},{"operation_name":"upsert","last_dispatch_duration_us":5999565,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8112357,"total_server_duration_us":248760,"encode_duration_us":20132,"operation_id":"0x7e49","timeout_ms":10000,"last_server_duration_us":42200,"total_duration_us":10007508},{"operation_name":"upsert","last_dispatch_duration_us":2116186,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10230042,"total_server_duration_us":919113,"encode_duration_us":16551,"operation_id":"0x7cc9","timeout_ms":10000,"last_server_duration_us":203378,"total_duration_us":10006465},{"operation_name":"upsert","last_dispatch_duration_us":9060665,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":13154657,"total_server_duration_us":9276403,"encode_duration_us":16466,"operation_id":"0x7d8c","timeout_ms":10000,"last_server_duration_us":9047983,"total_duration_us":10006384},{"operation_name":"upsert","last_dispatch_duration_us":10106439,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10106439,"total_server_duration_us":414960,"encode_duration_us":16435,"operation_id":"0x7e0d","timeout_ms":10000,"last_server_duration_us":414960,"total_duration_us":10004165},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","encode_duration_us":16519,"operation_id":"0x7ddc","timeout_ms":10000,"total_duration_us":10003400},{"operation_name":"upsert","encode_duration_us":16660,"operation_id":"0x7db0","timeout_ms":10000,"total_duration_us":10002696},{"operation_name":"upsert","last_dispatch_duration_us":4502216,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12110679,"total_server_duration_us":721114,"encode_duration_us":16275,"operation_id":"0x7d2b","timeout_ms":10000,"last_server_duration_us":139878,"total_duration_us":10001885}],"total_count":13}} INFO [2022-01-10 05:47:02,296] jepsen worker 44 - jepsen.util 4184 :info :write [14 19] :RequestTimeoutException INFO [2022-01-10 05:47:02,297] jepsen worker 44 - jepsen.util 4274 :invoke :read [14 nil] WARN [2022-01-10 05:47:02,343] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10526514,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18545104,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9720246,"encode_duration_us":19635,"operation_id":"0x7c96","timeout_ms":10000,"last_server_duration_us":9523585,"total_duration_us":10002409}],"total_count":1}} WARN [2022-01-10 05:47:02,345] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10526234,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16164050,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9985773,"encode_duration_us":16559,"operation_id":"0x7d1e","timeout_ms":10000,"last_server_duration_us":9680409,"total_duration_us":10007158},{"operation_name":"upsert","last_dispatch_duration_us":10155185,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15320957,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9783151,"encode_duration_us":19442,"operation_id":"0x7bc8","timeout_ms":10000,"last_server_duration_us":9660746,"total_duration_us":10010893}],"total_count":2}} INFO [2022-01-10 05:47:02,401] jepsen worker 53 - jepsen.util 4913 :ok :read [17 41] INFO [2022-01-10 05:47:02,401] jepsen worker 53 - jepsen.util 4913 :invoke :write [17 11] WARN [2022-01-10 05:47:02,445] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2830133,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12017399,"total_server_duration_us":1284583,"encode_duration_us":23285,"operation_id":"0x7d1f","timeout_ms":10000,"last_server_duration_us":197078,"total_duration_us":10010873},{"operation_name":"upsert","last_dispatch_duration_us":125422,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":1228476,"total_server_duration_us":295659,"encode_duration_us":16641,"operation_id":"0x7e69","timeout_ms":10000,"last_server_duration_us":38795,"total_duration_us":10009887},{"operation_name":"upsert","last_dispatch_duration_us":36912,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9045325,"total_server_duration_us":242201,"encode_duration_us":19991,"operation_id":"0x7ccb","timeout_ms":10000,"last_server_duration_us":22059,"total_duration_us":10009255},{"operation_name":"upsert","last_dispatch_duration_us":12768811,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":12768811,"total_server_duration_us":99190,"encode_duration_us":16684,"operation_id":"0x7d8d","timeout_ms":10000,"last_server_duration_us":99190,"total_duration_us":10007379},{"operation_name":"upsert","last_dispatch_duration_us":10526234,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":16164050,"total_server_duration_us":9985773,"encode_duration_us":16559,"operation_id":"0x7d1e","timeout_ms":10000,"last_server_duration_us":9680409,"total_duration_us":10007158},{"operation_name":"upsert","last_dispatch_duration_us":491648,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10043224,"total_server_duration_us":7915060,"encode_duration_us":20590,"operation_id":"0x7e48","timeout_ms":10000,"last_server_duration_us":481486,"total_duration_us":10006949},{"operation_name":"upsert","last_dispatch_duration_us":3263963,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10959290,"total_server_duration_us":1338278,"encode_duration_us":16393,"operation_id":"0x7d2c","timeout_ms":10000,"last_server_duration_us":399623,"total_duration_us":10006386},{"operation_name":"upsert","last_dispatch_duration_us":10465191,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11487259,"total_server_duration_us":631218,"encode_duration_us":22273,"operation_id":"0x7ced","timeout_ms":10000,"last_server_duration_us":540587,"total_duration_us":10006376},{"operation_name":"upsert","last_dispatch_duration_us":67278,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":6270705,"total_server_duration_us":360990,"encode_duration_us":25425,"operation_id":"0x7d6c","timeout_ms":10000,"last_server_duration_us":46739,"total_duration_us":10005063},{"operation_name":"upsert","last_dispatch_duration_us":4571241,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11077778,"total_server_duration_us":1210395,"encode_duration_us":18603,"operation_id":"0x7e0e","timeout_ms":10000,"last_server_duration_us":207199,"total_duration_us":10004480}],"total_count":24}} INFO [2022-01-10 05:47:02,456] jepsen worker 1 - jepsen.util 4051 :ok :read [0 46] INFO [2022-01-10 05:47:02,457] jepsen worker 1 - jepsen.util 4051 :invoke :write [0 15] INFO [2022-01-10 05:47:02,655] jepsen worker 22 - jepsen.util 3892 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:02,656] jepsen worker 21 - jepsen.util 4161 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:02,656] jepsen worker 42 - jepsen.util 3822 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:02,656] jepsen worker 22 - jepsen.util 3892 :invoke :read [7 nil] INFO [2022-01-10 05:47:02,657] jepsen worker 21 - jepsen.util 4161 :invoke :read [7 nil] INFO [2022-01-10 05:47:02,657] jepsen worker 42 - jepsen.util 3822 :invoke :read [14 nil] INFO [2022-01-10 05:47:02,710] jepsen worker 47 - jepsen.util 4097 :info :write [15 15] :RequestTimeoutException INFO [2022-01-10 05:47:02,711] jepsen worker 47 - jepsen.util 4187 :invoke :write [15 37] INFO [2022-01-10 05:47:02,866] jepsen worker 13 - jepsen.util 4693 :info :write [4 32] :RequestTimeoutException INFO [2022-01-10 05:47:02,867] jepsen worker 13 - jepsen.util 4783 :invoke :write [4 16] INFO [2022-01-10 05:47:02,875] jepsen worker 22 - jepsen.util 3892 :ok :read [7 :nil] INFO [2022-01-10 05:47:02,875] jepsen worker 21 - jepsen.util 4161 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:02,875] jepsen worker 22 - jepsen.util 3892 :invoke :write [7 13] INFO [2022-01-10 05:47:02,876] jepsen worker 21 - jepsen.util 4161 :invoke :write [7 29] INFO [2022-01-10 05:47:02,886] jepsen worker 46 - jepsen.util 4276 :ok :read [15 23] INFO [2022-01-10 05:47:02,887] jepsen worker 46 - jepsen.util 4276 :invoke :write [15 27] INFO [2022-01-10 05:47:03,100] jepsen worker 83 - jepsen.util 4583 :ok :read [27 30] INFO [2022-01-10 05:47:03,101] jepsen worker 83 - jepsen.util 4583 :invoke :write [27 1] INFO [2022-01-10 05:47:03,111] jepsen worker 42 - jepsen.util 3822 :ok :read [14 14] INFO [2022-01-10 05:47:03,111] jepsen worker 42 - jepsen.util 3822 :invoke :write [14 42] INFO [2022-01-10 05:47:03,216] jepsen worker 11 - jepsen.util 3881 :info :write [3 26] :RequestTimeoutException INFO [2022-01-10 05:47:03,216] jepsen worker 11 - jepsen.util 3971 :invoke :write [3 5] INFO [2022-01-10 05:47:03,256] jepsen worker 66 - jepsen.util 3756 :info :write [22 48] :RequestTimeoutException INFO [2022-01-10 05:47:03,256] jepsen worker 66 - jepsen.util 3846 :invoke :read [22 nil] INFO [2022-01-10 05:47:03,256] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:03,256] jepsen worker nemesis - couchbase.util Rebalance check count: 571 INFO [2022-01-10 05:47:03,256] jepsen worker nemesis - couchbase.util Rebalance stuck count: 481 INFO [2022-01-10 05:47:03,477] jepsen worker 16 - jepsen.util 4156 :ok :read [5 25] INFO [2022-01-10 05:47:03,478] jepsen worker 16 - jepsen.util 4156 :invoke :read [5 nil] INFO [2022-01-10 05:47:03,535] jepsen worker 56 - jepsen.util 4376 :info :write [18 17] :SyncWriteAmbiguous INFO [2022-01-10 05:47:03,536] jepsen worker 56 - jepsen.util 4466 :invoke :write [18 47] INFO [2022-01-10 05:47:03,550] jepsen worker 55 - jepsen.util 4465 :info :write [18 4] :RequestTimeoutException INFO [2022-01-10 05:47:03,551] jepsen worker 55 - jepsen.util 4555 :invoke :write [18 49] INFO [2022-01-10 05:47:03,856] jepsen worker 78 - jepsen.util 4938 :info :write [26 48] :RequestTimeoutException INFO [2022-01-10 05:47:03,857] jepsen worker 78 - jepsen.util 5028 :invoke :write [26 13] INFO [2022-01-10 05:47:03,878] jepsen worker 80 - jepsen.util 5030 :ok :read [26 10] INFO [2022-01-10 05:47:03,879] jepsen worker 80 - jepsen.util 5030 :invoke :read [26 nil] INFO [2022-01-10 05:47:03,916] jepsen worker 30 - jepsen.util 3810 :info :write [10 17] :SyncWriteAmbiguous INFO [2022-01-10 05:47:03,917] jepsen worker 30 - jepsen.util 3900 :invoke :write [10 3] INFO [2022-01-10 05:47:03,931] jepsen worker 27 - jepsen.util 4257 :info :write [9 34] :RequestTimeoutException INFO [2022-01-10 05:47:03,932] jepsen worker 27 - jepsen.util 4347 :invoke :write [9 35] INFO [2022-01-10 05:47:04,262] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:04,262] jepsen worker nemesis - couchbase.util Rebalance check count: 572 INFO [2022-01-10 05:47:04,262] jepsen worker nemesis - couchbase.util Rebalance stuck count: 482 INFO [2022-01-10 05:47:04,464] jepsen worker 10 - jepsen.util 3970 :info :write [3 37] :RequestTimeoutException INFO [2022-01-10 05:47:04,465] jepsen worker 10 - jepsen.util 4060 :invoke :read [3 nil] INFO [2022-01-10 05:47:04,636] jepsen worker 17 - jepsen.util 3977 :info :write [5 27] :RequestTimeoutException INFO [2022-01-10 05:47:04,637] jepsen worker 17 - jepsen.util 4067 :invoke :read [5 nil] INFO [2022-01-10 05:47:04,734] jepsen worker 16 - jepsen.util 4156 :ok :read [5 25] INFO [2022-01-10 05:47:04,735] jepsen worker 16 - jepsen.util 4156 :invoke :write [5 2] INFO [2022-01-10 05:47:04,753] jepsen worker 17 - jepsen.util 4067 :ok :read [5 25] INFO [2022-01-10 05:47:04,856] jepsen worker 17 - jepsen.util 4067 :invoke :read [5 nil] INFO [2022-01-10 05:47:04,962] jepsen worker 54 - jepsen.util 4104 :info :write [18 49] :RequestTimeoutException INFO [2022-01-10 05:47:04,962] jepsen worker 54 - jepsen.util 4194 :invoke :write [18 23] INFO [2022-01-10 05:47:04,967] jepsen worker 64 - jepsen.util 4114 :ok :read [21 7] INFO [2022-01-10 05:47:04,968] jepsen worker 64 - jepsen.util 4114 :invoke :read [21 nil] INFO [2022-01-10 05:47:05,040] jepsen worker 23 - jepsen.util 4163 :info :write [7 36] :RequestTimeoutException INFO [2022-01-10 05:47:05,041] jepsen worker 23 - jepsen.util 4253 :invoke :read [7 nil] INFO [2022-01-10 05:47:05,174] jepsen worker 26 - jepsen.util 3806 :ok :read [8 39] INFO [2022-01-10 05:47:05,175] jepsen worker 26 - jepsen.util 3806 :invoke :write [8 21] INFO [2022-01-10 05:47:05,255] jepsen worker 28 - jepsen.util 3898 :info :write [9 43] :RequestTimeoutException INFO [2022-01-10 05:47:05,256] jepsen worker 28 - jepsen.util 3988 :invoke :read [9 nil] INFO [2022-01-10 05:47:05,268] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:05,268] jepsen worker nemesis - couchbase.util Rebalance check count: 573 INFO [2022-01-10 05:47:05,268] jepsen worker nemesis - couchbase.util Rebalance stuck count: 483 INFO [2022-01-10 05:47:05,502] jepsen worker 15 - jepsen.util 3615 :info :write [5 46] :RequestTimeoutException INFO [2022-01-10 05:47:05,503] jepsen worker 15 - jepsen.util 3705 :invoke :read [5 nil] INFO [2022-01-10 05:47:05,715] jepsen worker 45 - jepsen.util 4365 :info :write [15 35] :RequestTimeoutException INFO [2022-01-10 05:47:05,715] jepsen worker 45 - jepsen.util 4455 :invoke :read [15 nil] INFO [2022-01-10 05:47:05,732] jepsen worker 80 - jepsen.util 5030 :ok :read [26 10] INFO [2022-01-10 05:47:05,732] jepsen worker 80 - jepsen.util 5030 :invoke :write [26 6] INFO [2022-01-10 05:47:05,735] jepsen worker 88 - jepsen.util 3958 :info :write [29 4] :RequestTimeoutException INFO [2022-01-10 05:47:05,736] jepsen worker 88 - jepsen.util 4048 :invoke :write [29 6] INFO [2022-01-10 05:47:05,982] jepsen worker 66 - jepsen.util 3846 :ok :read [22 41] INFO [2022-01-10 05:47:05,983] jepsen worker 66 - jepsen.util 3846 :invoke :read [22 nil] INFO [2022-01-10 05:47:05,997] jepsen worker 25 - jepsen.util 3895 :ok :read [8 39] INFO [2022-01-10 05:47:05,997] jepsen worker 25 - jepsen.util 3895 :invoke :write [8 25] INFO [2022-01-10 05:47:06,026] jepsen worker 36 - jepsen.util 3096 :fail :read [12 nil] :UnambiguousTimeoutException INFO [2022-01-10 05:47:06,027] jepsen worker 36 - jepsen.util 3096 :invoke :read [12 nil] INFO [2022-01-10 05:47:06,152] jepsen worker 12 - jepsen.util 4602 :info :write [4 1] :RequestTimeoutException INFO [2022-01-10 05:47:06,153] jepsen worker 12 - jepsen.util 4692 :invoke :write [4 49] INFO [2022-01-10 05:47:06,273] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:06,273] jepsen worker nemesis - couchbase.util Rebalance check count: 574 INFO [2022-01-10 05:47:06,273] jepsen worker nemesis - couchbase.util Rebalance stuck count: 484 INFO [2022-01-10 05:47:06,804] jepsen worker 73 - jepsen.util 3673 :info :write [24 31] :RequestTimeoutException INFO [2022-01-10 05:47:06,805] jepsen worker 73 - jepsen.util 3763 :invoke :write [24 45] INFO [2022-01-10 05:47:06,816] jepsen worker 87 - jepsen.util 3957 :info :write [29 35] :RequestTimeoutException INFO [2022-01-10 05:47:06,817] jepsen worker 87 - jepsen.util 4047 :invoke :read [29 nil] INFO [2022-01-10 05:47:06,822] jepsen worker 71 - jepsen.util 3581 :info :write [23 27] :RequestTimeoutException INFO [2022-01-10 05:47:06,823] jepsen worker 71 - jepsen.util 3671 :invoke :write [23 46] INFO [2022-01-10 05:47:06,991] jepsen worker 20 - jepsen.util 3800 :fail :read [6 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:06,991] jepsen worker 20 - jepsen.util 3800 :invoke :read [6 nil] INFO [2022-01-10 05:47:07,062] jepsen worker 70 - jepsen.util 4030 :info :write [23 22] :RequestTimeoutException INFO [2022-01-10 05:47:07,062] jepsen worker 70 - jepsen.util 4120 :invoke :read [23 nil] INFO [2022-01-10 05:47:07,210] jepsen worker 45 - jepsen.util 4455 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:07,211] jepsen worker 45 - jepsen.util 4455 :invoke :write [15 10] INFO [2022-01-10 05:47:07,220] jepsen worker 43 - jepsen.util 3913 :info :write [14 48] :RequestTimeoutException INFO [2022-01-10 05:47:07,221] jepsen worker 43 - jepsen.util 4003 :invoke :write [14 47] INFO [2022-01-10 05:47:07,250] jepsen worker 35 - jepsen.util 3815 :ok :read [11 21] INFO [2022-01-10 05:47:07,250] jepsen worker 35 - jepsen.util 3815 :invoke :write [11 39] INFO [2022-01-10 05:47:07,258] jepsen worker 6 - jepsen.util 4146 :ok :read [2 14] INFO [2022-01-10 05:47:07,259] jepsen worker 6 - jepsen.util 4146 :invoke :read [2 nil] INFO [2022-01-10 05:47:07,279] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:07,279] jepsen worker nemesis - couchbase.util Rebalance check count: 575 INFO [2022-01-10 05:47:07,279] jepsen worker nemesis - couchbase.util Rebalance stuck count: 485 INFO [2022-01-10 05:47:07,320] jepsen worker 67 - jepsen.util 3937 :info :write [22 30] :RequestTimeoutException INFO [2022-01-10 05:47:07,321] jepsen worker 67 - jepsen.util 4027 :invoke :write [22 24] INFO [2022-01-10 05:47:07,364] jepsen worker 38 - jepsen.util 3818 :info :write [12 47] :RequestTimeoutException INFO [2022-01-10 05:47:07,365] jepsen worker 38 - jepsen.util 3908 :invoke :read [12 nil] INFO [2022-01-10 05:47:07,411] jepsen worker 65 - jepsen.util 3845 :info :write [21 14] :RequestTimeoutException INFO [2022-01-10 05:47:07,412] jepsen worker 65 - jepsen.util 3935 :invoke :read [21 nil] INFO [2022-01-10 05:47:07,595] jepsen worker 34 - jepsen.util 3994 :info :write [11 14] :RequestTimeoutException INFO [2022-01-10 05:47:07,596] jepsen worker 34 - jepsen.util 4084 :invoke :write [11 2] INFO [2022-01-10 05:47:07,604] jepsen worker 75 - jepsen.util 3765 :info :write [25 17] :RequestTimeoutException INFO [2022-01-10 05:47:07,605] jepsen worker 75 - jepsen.util 3855 :invoke :write [25 22] INFO [2022-01-10 05:47:07,620] jepsen worker 77 - jepsen.util 4577 :ok :read [25 :nil] INFO [2022-01-10 05:47:07,621] jepsen worker 77 - jepsen.util 4577 :invoke :write [25 44] INFO [2022-01-10 05:47:07,826] jepsen worker 44 - jepsen.util 4274 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:07,827] jepsen worker 44 - jepsen.util 4274 :invoke :read [14 nil] INFO [2022-01-10 05:47:08,071] jepsen worker 69 - jepsen.util 3579 :info :write [23 39] :RequestTimeoutException INFO [2022-01-10 05:47:08,071] jepsen worker 69 - jepsen.util 3669 :invoke :read [23 nil] INFO [2022-01-10 05:47:08,081] jepsen worker 79 - jepsen.util 4759 :info :write [26 39] :RequestTimeoutException INFO [2022-01-10 05:47:08,082] jepsen worker 79 - jepsen.util 4849 :invoke :write [26 14] INFO [2022-01-10 05:47:08,284] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:08,285] jepsen worker nemesis - couchbase.util Rebalance check count: 576 INFO [2022-01-10 05:47:08,285] jepsen worker nemesis - couchbase.util Rebalance stuck count: 486 INFO [2022-01-10 05:47:08,464] jepsen worker 14 - jepsen.util 4964 :info :write [4 43] :RequestTimeoutException INFO [2022-01-10 05:47:08,465] jepsen worker 14 - jepsen.util 5054 :invoke :write [4 18] INFO [2022-01-10 05:47:08,475] jepsen worker 70 - jepsen.util 4120 :ok :read [23 26] INFO [2022-01-10 05:47:08,475] jepsen worker 70 - jepsen.util 4120 :invoke :read [23 nil] INFO [2022-01-10 05:47:08,486] jepsen worker 69 - jepsen.util 3669 :ok :read [23 26] INFO [2022-01-10 05:47:08,486] jepsen worker 69 - jepsen.util 3669 :invoke :read [23 nil] INFO [2022-01-10 05:47:08,555] jepsen worker 41 - jepsen.util 4001 :info :write [13 42] :RequestTimeoutException INFO [2022-01-10 05:47:08,556] jepsen worker 41 - jepsen.util 4091 :invoke :write [13 11] INFO [2022-01-10 05:47:08,559] jepsen worker 82 - jepsen.util 4222 :info :write [27 10] :RequestTimeoutException INFO [2022-01-10 05:47:08,560] jepsen worker 82 - jepsen.util 4312 :invoke :read [27 nil] INFO [2022-01-10 05:47:08,845] jepsen worker 28 - jepsen.util 3988 :ok :read [9 33] INFO [2022-01-10 05:47:08,845] jepsen worker 28 - jepsen.util 3988 :invoke :read [9 nil] INFO [2022-01-10 05:47:08,982] jepsen worker 69 - jepsen.util 3669 :ok :read [23 26] INFO [2022-01-10 05:47:08,982] jepsen worker 69 - jepsen.util 3669 :invoke :read [23 nil] INFO [2022-01-10 05:47:09,041] jepsen worker 63 - jepsen.util 3843 :info :write [21 4] :RequestTimeoutException INFO [2022-01-10 05:47:09,042] jepsen worker 63 - jepsen.util 3933 :invoke :write [21 12] INFO [2022-01-10 05:47:09,062] jepsen worker 59 - jepsen.util 4199 :info :write [19 19] :RequestTimeoutException INFO [2022-01-10 05:47:09,063] jepsen worker 59 - jepsen.util 4289 :invoke :read [19 nil] INFO [2022-01-10 05:47:09,092] jepsen worker 51 - jepsen.util 4911 :info :write [17 19] :RequestTimeoutException INFO [2022-01-10 05:47:09,092] jepsen worker 51 - jepsen.util 5001 :invoke :write [17 29] INFO [2022-01-10 05:47:09,156] jepsen worker 69 - jepsen.util 3669 :ok :read [23 26] INFO [2022-01-10 05:47:09,156] jepsen worker 69 - jepsen.util 3669 :invoke :read [23 nil] INFO [2022-01-10 05:47:09,291] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:09,291] jepsen worker nemesis - couchbase.util Rebalance check count: 577 INFO [2022-01-10 05:47:09,291] jepsen worker nemesis - couchbase.util Rebalance stuck count: 487 INFO [2022-01-10 05:47:09,384] jepsen worker 32 - jepsen.util 4352 :ok :read [10 36] INFO [2022-01-10 05:47:09,385] jepsen worker 32 - jepsen.util 4352 :invoke :read [10 nil] INFO [2022-01-10 05:47:09,402] jepsen worker 4 - jepsen.util 4684 :ok :read [1 26] INFO [2022-01-10 05:47:09,403] jepsen worker 4 - jepsen.util 4684 :invoke :read [1 nil] INFO [2022-01-10 05:47:09,409] jepsen worker 89 - jepsen.util 4589 :ok :read [29 0] INFO [2022-01-10 05:47:09,410] jepsen worker 89 - jepsen.util 4589 :invoke :write [29 48] INFO [2022-01-10 05:47:09,450] jepsen worker 31 - jepsen.util 4441 :ok :read [10 36] INFO [2022-01-10 05:47:09,450] jepsen worker 31 - jepsen.util 4441 :invoke :read [10 nil] INFO [2022-01-10 05:47:09,482] jepsen worker 69 - jepsen.util 3669 :ok :read [23 26] INFO [2022-01-10 05:47:09,482] jepsen worker 69 - jepsen.util 3669 :invoke :write [23 36] INFO [2022-01-10 05:47:10,150] jepsen worker 70 - jepsen.util 4120 :ok :read [23 26] INFO [2022-01-10 05:47:10,255] jepsen worker 29 - jepsen.util 3989 :info :write [9 23] :RequestTimeoutException INFO [2022-01-10 05:47:10,256] jepsen worker 85 - jepsen.util 4405 :info :write [28 13] :RequestTimeoutException INFO [2022-01-10 05:47:10,297] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:10,401] jepsen worker 48 - jepsen.util 3828 :info :write [16 28] :RequestTimeoutException INFO [2022-01-10 05:47:10,402] jepsen worker nemesis - couchbase.util Rebalance check count: 578 INFO [2022-01-10 05:47:10,402] jepsen worker 70 - jepsen.util 4120 :invoke :read [23 nil] INFO [2022-01-10 05:47:10,403] jepsen worker nemesis - couchbase.util Rebalance stuck count: 488 INFO [2022-01-10 05:47:10,402] jepsen worker 29 - jepsen.util 4079 :invoke :read [9 nil] INFO [2022-01-10 05:47:10,402] jepsen worker 85 - jepsen.util 4495 :invoke :read [28 nil] INFO [2022-01-10 05:47:10,403] jepsen worker 48 - jepsen.util 3918 :invoke :write [16 29] INFO [2022-01-10 05:47:10,413] jepsen worker 24 - jepsen.util 3804 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:10,414] jepsen worker 24 - jepsen.util 3804 :invoke :write [8 47] INFO [2022-01-10 05:47:10,481] jepsen worker 74 - jepsen.util 4664 :ok :read [24 18] INFO [2022-01-10 05:47:10,482] jepsen worker 74 - jepsen.util 4664 :invoke :write [24 13] INFO [2022-01-10 05:47:10,561] jepsen worker 7 - jepsen.util 4237 :info :write [2 31] :RequestTimeoutException INFO [2022-01-10 05:47:10,561] jepsen worker 7 - jepsen.util 4327 :invoke :write [2 16] WARN [2022-01-10 05:47:10,564] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":40783,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8815618,"total_server_duration_us":284714,"encode_duration_us":16605,"operation_id":"0x7ede","timeout_ms":10000,"last_server_duration_us":27854,"total_duration_us":10009327},{"operation_name":"upsert","last_dispatch_duration_us":1798185,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5417268,"total_server_duration_us":1358615,"encode_duration_us":19212,"operation_id":"0x7f1a","timeout_ms":10000,"last_server_duration_us":385919,"total_duration_us":10008075},{"operation_name":"upsert","last_dispatch_duration_us":1634663,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":8837234,"total_server_duration_us":962817,"encode_duration_us":16448,"operation_id":"0x7fa3","timeout_ms":10000,"last_server_duration_us":375981,"total_duration_us":10007329},{"operation_name":"upsert","last_dispatch_duration_us":1823398,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10296433,"total_server_duration_us":2417604,"encode_duration_us":17439,"operation_id":"0x7f56","timeout_ms":10000,"last_server_duration_us":390651,"total_duration_us":10005137},{"operation_name":"upsert","last_dispatch_duration_us":3972015,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":12690829,"total_server_duration_us":2767702,"encode_duration_us":28328,"operation_id":"0x7f73","timeout_ms":10000,"last_server_duration_us":807667,"total_duration_us":10003013},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":16857,"operation_id":"0x7ffb","timeout_ms":10000,"total_duration_us":10002239},{"operation_name":"upsert","last_dispatch_duration_us":1809745,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9903667,"total_server_duration_us":1886994,"encode_duration_us":18761,"operation_id":"0x7f72","timeout_ms":10000,"last_server_duration_us":195621,"total_duration_us":10001288},{"operation_name":"upsert","last_dispatch_duration_us":1816281,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":9298663,"total_server_duration_us":2591321,"encode_duration_us":16719,"operation_id":"0x7f6e","timeout_ms":10000,"last_server_duration_us":379833,"total_duration_us":10000809},{"operation_name":"upsert","last_dispatch_duration_us":9063812,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9063812,"total_server_duration_us":9032060,"encode_duration_us":16855,"operation_id":"0x7edf","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":9067473},{"operation_name":"get","last_dispatch_duration_us":1194995,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1194995,"total_server_duration_us":189220,"operation_id":"0x7ff8","timeout_ms":30000,"last_server_duration_us":189220,"total_duration_us":7383538}],"total_count":13}} INFO [2022-01-10 05:47:10,642] jepsen worker 61 - jepsen.util 4201 :info :write [20 6] :RequestTimeoutException INFO [2022-01-10 05:47:10,643] jepsen worker 61 - jepsen.util 4291 :invoke :write [20 33] INFO [2022-01-10 05:47:10,804] jepsen worker 81 - jepsen.util 4491 :info :write [27 19] :RequestTimeoutException INFO [2022-01-10 05:47:10,805] jepsen worker 81 - jepsen.util 4581 :invoke :write [27 27] INFO [2022-01-10 05:47:10,869] jepsen worker 84 - jepsen.util 4044 :info :write [28 49] :RequestTimeoutException INFO [2022-01-10 05:47:10,870] jepsen worker 84 - jepsen.util 4134 :invoke :read [28 nil] INFO [2022-01-10 05:47:10,872] jepsen worker 66 - jepsen.util 3846 :ok :read [22 41] INFO [2022-01-10 05:47:10,873] jepsen worker 66 - jepsen.util 3846 :invoke :read [22 nil] INFO [2022-01-10 05:47:10,876] jepsen worker 44 - jepsen.util 4274 :ok :read [14 14] INFO [2022-01-10 05:47:10,877] jepsen worker 44 - jepsen.util 4274 :invoke :write [14 1] INFO [2022-01-10 05:47:11,006] jepsen worker 33 - jepsen.util 4263 :info :write [11 33] :RequestTimeoutException INFO [2022-01-10 05:47:11,007] jepsen worker 33 - jepsen.util 4353 :invoke :write [11 5] INFO [2022-01-10 05:47:11,027] jepsen worker 9 - jepsen.util 4599 :info :write [3 35] :SyncWriteAmbiguous INFO [2022-01-10 05:47:11,027] jepsen worker 9 - jepsen.util 4689 :invoke :write [3 36] INFO [2022-01-10 05:47:11,050] jepsen worker 40 - jepsen.util 3640 :info :write [13 18] :RequestTimeoutException INFO [2022-01-10 05:47:11,051] jepsen worker 40 - jepsen.util 3730 :invoke :read [13 nil] INFO [2022-01-10 05:47:11,051] jepsen worker 68 - jepsen.util 4208 :info :write [22 38] :RequestTimeoutException INFO [2022-01-10 05:47:11,052] jepsen worker 68 - jepsen.util 4298 :invoke :write [22 19] INFO [2022-01-10 05:47:11,275] jepsen worker 86 - jepsen.util 3506 :ok :read [28 45] INFO [2022-01-10 05:47:11,275] jepsen worker 86 - jepsen.util 3506 :invoke :read [28 nil] INFO [2022-01-10 05:47:11,408] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:11,408] jepsen worker nemesis - couchbase.util Rebalance check count: 579 INFO [2022-01-10 05:47:11,408] jepsen worker nemesis - couchbase.util Rebalance stuck count: 489 INFO [2022-01-10 05:47:11,434] jepsen worker 6 - jepsen.util 4146 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:11,434] jepsen worker 20 - jepsen.util 3800 :fail :read [6 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:11,435] jepsen worker 20 - jepsen.util 3800 :invoke :read [6 nil] INFO [2022-01-10 05:47:11,436] jepsen worker 6 - jepsen.util 4146 :invoke :read [2 nil] INFO [2022-01-10 05:47:11,442] jepsen worker 52 - jepsen.util 5182 :info :write [17 34] :RequestTimeoutException INFO [2022-01-10 05:47:11,443] jepsen worker 52 - jepsen.util 5272 :invoke :write [17 1] INFO [2022-01-10 05:47:11,571] jepsen worker 60 - jepsen.util 4290 :info :write [20 34] :RequestTimeoutException INFO [2022-01-10 05:47:11,571] jepsen worker 60 - jepsen.util 4380 :invoke :read [20 nil] INFO [2022-01-10 05:47:11,656] jepsen worker 50 - jepsen.util 4280 :info :write [16 34] :RequestTimeoutException INFO [2022-01-10 05:47:11,656] jepsen worker 50 - jepsen.util 4370 :invoke :write [16 12] INFO [2022-01-10 05:47:11,689] jepsen worker 18 - jepsen.util 4068 :info :write [6 42] :RequestTimeoutException INFO [2022-01-10 05:47:11,690] jepsen worker 18 - jepsen.util 4158 :invoke :read [6 nil] INFO [2022-01-10 05:47:11,700] jepsen worker 19 - jepsen.util 4249 :info :write [6 8] :RequestTimeoutException INFO [2022-01-10 05:47:11,701] jepsen worker 19 - jepsen.util 4339 :invoke :write [6 17] INFO [2022-01-10 05:47:11,916] jepsen worker 58 - jepsen.util 3838 :info :write [19 39] :RequestTimeoutException INFO [2022-01-10 05:47:11,917] jepsen worker 58 - jepsen.util 3928 :invoke :write [19 5] INFO [2022-01-10 05:47:11,920] jepsen worker 0 - jepsen.util 3960 :info :write [0 42] :RequestTimeoutException INFO [2022-01-10 05:47:11,920] jepsen worker 0 - jepsen.util 4050 :invoke :write [0 48] INFO [2022-01-10 05:47:11,930] jepsen worker 5 - jepsen.util 3965 :info :write [1 29] :RequestTimeoutException INFO [2022-01-10 05:47:11,931] jepsen worker 5 - jepsen.util 4055 :invoke :write [1 40] INFO [2022-01-10 05:47:12,036] jepsen worker 3 - jepsen.util 3963 :info :write [1 13] :RequestTimeoutException INFO [2022-01-10 05:47:12,037] jepsen worker 3 - jepsen.util 4053 :invoke :read [1 nil] INFO [2022-01-10 05:47:12,060] jepsen worker 57 - jepsen.util 4917 :fail :read [19 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:12,061] jepsen worker 49 - jepsen.util 3919 :fail :read [16 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:12,061] jepsen worker 57 - jepsen.util 4917 :invoke :write [19 11] INFO [2022-01-10 05:47:12,062] jepsen worker 49 - jepsen.util 3919 :invoke :write [16 44] INFO [2022-01-10 05:47:12,060] jepsen worker 39 - jepsen.util 4629 :fail :read [13 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:12,063] jepsen worker 39 - jepsen.util 4629 :invoke :read [13 nil] WARN [2022-01-10 05:47:12,162] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12580585,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":12580585,"total_server_duration_us":27,"operation_id":"0x7f57","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":12582066},{"operation_name":"upsert","last_dispatch_duration_us":1335034,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9631425,"total_server_duration_us":1216736,"encode_duration_us":16346,"operation_id":"0x7ed1","timeout_ms":10000,"last_server_duration_us":101381,"total_duration_us":10011033},{"operation_name":"upsert","last_dispatch_duration_us":17367618,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":17367618,"total_server_duration_us":9023573,"encode_duration_us":20344,"operation_id":"0x7e6c","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10009469},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":16803,"operation_id":"0x7f6f","timeout_ms":10000,"total_duration_us":10009213},{"operation_name":"upsert","last_dispatch_duration_us":528784,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":8776650,"total_server_duration_us":5954121,"encode_duration_us":18262,"operation_id":"0x7f99","timeout_ms":10000,"last_server_duration_us":515530,"total_duration_us":10007884},{"operation_name":"upsert","last_dispatch_duration_us":585029,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9249116,"total_server_duration_us":3675919,"encode_duration_us":17414,"operation_id":"0x7f9a","timeout_ms":10000,"last_server_duration_us":330550,"total_duration_us":10007327},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":16412,"operation_id":"0x8031","timeout_ms":10000,"total_duration_us":10006759},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":16266,"operation_id":"0x8001","timeout_ms":10000,"total_duration_us":10006387},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","encode_duration_us":18720,"operation_id":"0x7fc4","timeout_ms":10000,"total_duration_us":10004728},{"operation_name":"upsert","last_dispatch_duration_us":319126,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":3937481,"total_server_duration_us":1358861,"encode_duration_us":20475,"operation_id":"0x8027","timeout_ms":10000,"last_server_duration_us":306002,"total_duration_us":10003657}],"total_count":17}} WARN [2022-01-10 05:47:12,163] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":19209118,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":22020218,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9818590,"encode_duration_us":17991,"operation_id":"0x7cd8","timeout_ms":10000,"last_server_duration_us":9641101,"total_duration_us":10004829},{"operation_name":"upsert","last_dispatch_duration_us":17367618,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17367618,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9023573,"encode_duration_us":20344,"operation_id":"0x7e6c","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10009469}],"total_count":2}} WARN [2022-01-10 05:47:12,169] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":916112,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":13966282,"total_server_duration_us":941180,"operation_id":"0x7b50","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":30002613},{"operation_name":"get","last_dispatch_duration_us":7897158,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7897158,"total_server_duration_us":620126,"operation_id":"0x7e7f","timeout_ms":30000,"last_server_duration_us":620126,"total_duration_us":14591905},{"operation_name":"get","last_dispatch_duration_us":7644630,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7644630,"total_server_duration_us":402162,"operation_id":"0x7ee1","timeout_ms":30000,"last_server_duration_us":402162,"total_duration_us":12301866},{"operation_name":"get","last_dispatch_duration_us":10165326,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10165326,"total_server_duration_us":43,"operation_id":"0x7ff9","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":10165565},{"operation_name":"upsert","last_dispatch_duration_us":6520773,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":14297332,"total_server_duration_us":2355043,"encode_duration_us":16427,"operation_id":"0x7f0f","timeout_ms":10000,"last_server_duration_us":1793461,"total_duration_us":10015095},{"operation_name":"upsert","last_dispatch_duration_us":10913619,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10913619,"total_server_duration_us":1939414,"encode_duration_us":16850,"operation_id":"0x7e78","timeout_ms":10000,"last_server_duration_us":1939414,"total_duration_us":10006025},{"operation_name":"upsert","last_dispatch_duration_us":8327252,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":14715123,"total_server_duration_us":233071,"encode_duration_us":16610,"operation_id":"0x7f21","timeout_ms":10000,"last_server_duration_us":28493,"total_duration_us":10005872},{"operation_name":"upsert","last_dispatch_duration_us":6501147,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11496273,"total_server_duration_us":587538,"encode_duration_us":17248,"operation_id":"0x7fa4","timeout_ms":10000,"last_server_duration_us":382041,"total_duration_us":10005653},{"operation_name":"upsert","last_dispatch_duration_us":9837981,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9837981,"total_server_duration_us":210405,"encode_duration_us":18510,"operation_id":"0x8025","timeout_ms":10000,"last_server_duration_us":210405,"total_duration_us":10005390},{"operation_name":"upsert","last_dispatch_duration_us":4339232,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11767400,"total_server_duration_us":767902,"encode_duration_us":16768,"operation_id":"0x7ee0","timeout_ms":10000,"last_server_duration_us":590213,"total_duration_us":10003331}],"total_count":15}} INFO [2022-01-10 05:47:12,261] jepsen worker 70 - jepsen.util 4120 :ok :read [23 26] INFO [2022-01-10 05:47:12,261] jepsen worker 70 - jepsen.util 4120 :invoke :write [23 8] WARN [2022-01-10 05:47:12,262] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12702407,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12702407,"total_server_duration_us":434330,"operation_id":"0x7e7b","timeout_ms":30000,"last_server_duration_us":434330,"total_duration_us":12704219},{"operation_name":"get","last_dispatch_duration_us":12235943,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12235943,"total_server_duration_us":38,"operation_id":"0x7ebf","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":12238590},{"operation_name":"upsert","last_dispatch_duration_us":3009900,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":10264634,"total_server_duration_us":655723,"encode_duration_us":18399,"operation_id":"0x802f","timeout_ms":10000,"last_server_duration_us":112804,"total_duration_us":10009802},{"operation_name":"upsert","last_dispatch_duration_us":2853783,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":10909784,"total_server_duration_us":1451199,"encode_duration_us":25168,"operation_id":"0x7f5f","timeout_ms":10000,"last_server_duration_us":198332,"total_duration_us":10009196},{"operation_name":"upsert","last_dispatch_duration_us":3391156,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":12197505,"total_server_duration_us":2364707,"encode_duration_us":16318,"operation_id":"0x7f63","timeout_ms":10000,"last_server_duration_us":323050,"total_duration_us":10008937},{"operation_name":"upsert","last_dispatch_duration_us":984801,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":3508281,"total_server_duration_us":439008,"encode_duration_us":16348,"operation_id":"0x7f11","timeout_ms":10000,"last_server_duration_us":146617,"total_duration_us":10008761},{"operation_name":"upsert","last_dispatch_duration_us":3086026,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":10696804,"total_server_duration_us":1952340,"encode_duration_us":16559,"operation_id":"0x7fc9","timeout_ms":10000,"last_server_duration_us":313079,"total_duration_us":10007629},{"operation_name":"upsert","last_dispatch_duration_us":30796,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":1321952,"total_server_duration_us":582007,"encode_duration_us":17241,"operation_id":"0x7f05","timeout_ms":10000,"last_server_duration_us":20680,"total_duration_us":10007451},{"operation_name":"upsert","last_dispatch_duration_us":3493667,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":11149051,"total_server_duration_us":2128549,"encode_duration_us":21882,"operation_id":"0x7f3a","timeout_ms":10000,"last_server_duration_us":580271,"total_duration_us":10007378},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":20941,"operation_id":"0x7fc7","timeout_ms":10000,"total_duration_us":10003049}],"total_count":13}} WARN [2022-01-10 05:47:12,262] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2151171,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10047380,"total_server_duration_us":461062,"encode_duration_us":16524,"operation_id":"0x7eaa","timeout_ms":10000,"last_server_duration_us":114123,"total_duration_us":10009567},{"operation_name":"upsert","last_dispatch_duration_us":2904135,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":11399671,"total_server_duration_us":1431196,"encode_duration_us":18744,"operation_id":"0x7ecf","timeout_ms":10000,"last_server_duration_us":296012,"total_duration_us":10009229},{"operation_name":"upsert","last_dispatch_duration_us":8651940,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":16163359,"total_server_duration_us":263169,"encode_duration_us":16344,"operation_id":"0x7e6f","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10008445},{"operation_name":"upsert","last_dispatch_duration_us":2429264,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":10943366,"total_server_duration_us":2231227,"encode_duration_us":16843,"operation_id":"0x7ea0","timeout_ms":10000,"last_server_duration_us":600560,"total_duration_us":10007996},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","encode_duration_us":16211,"operation_id":"0x805c","timeout_ms":10000,"total_duration_us":10007212},{"operation_name":"upsert","last_dispatch_duration_us":12447427,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12447427,"total_server_duration_us":9030999,"encode_duration_us":17237,"operation_id":"0x7f5e","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10006114},{"operation_name":"upsert","last_dispatch_duration_us":9061270,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":16067922,"total_server_duration_us":38605,"encode_duration_us":17579,"operation_id":"0x7ea8","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10005896},{"operation_name":"upsert","last_dispatch_duration_us":3506844,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8134206,"total_server_duration_us":479930,"encode_duration_us":17144,"operation_id":"0x7ffa","timeout_ms":10000,"last_server_duration_us":202322,"total_duration_us":10003895},{"operation_name":"upsert","last_dispatch_duration_us":4463667,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9910205,"total_server_duration_us":510083,"encode_duration_us":18771,"operation_id":"0x8056","timeout_ms":10000,"last_server_duration_us":200008,"total_duration_us":10002495},{"operation_name":"get","last_dispatch_duration_us":7902205,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":7902205,"total_server_duration_us":43,"operation_id":"0x8030","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":7903148}],"total_count":20}} WARN [2022-01-10 05:47:12,362] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12167732,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19892554,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":12431014,"encode_duration_us":16539,"operation_id":"0x7e10","timeout_ms":10000,"last_server_duration_us":10984330,"total_duration_us":10004449}],"total_count":1}} INFO [2022-01-10 05:47:12,413] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:12,414] jepsen worker nemesis - couchbase.util Rebalance check count: 580 INFO [2022-01-10 05:47:12,414] jepsen worker nemesis - couchbase.util Rebalance stuck count: 490 INFO [2022-01-10 05:47:12,426] jepsen worker 53 - jepsen.util 4913 :info :write [17 11] :RequestTimeoutException INFO [2022-01-10 05:47:12,427] jepsen worker 53 - jepsen.util 5003 :invoke :write [17 41] WARN [2022-01-10 05:47:12,463] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12447427,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12447427,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9030999,"encode_duration_us":17237,"operation_id":"0x7f5e","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10006114}],"total_count":1}} WARN [2022-01-10 05:47:12,463] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":5678869,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10359162,"total_server_duration_us":846026,"encode_duration_us":17245,"operation_id":"0x8024","timeout_ms":10000,"last_server_duration_us":375432,"total_duration_us":10010519},{"operation_name":"upsert","last_dispatch_duration_us":30756,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":2482532,"total_server_duration_us":239665,"encode_duration_us":20102,"operation_id":"0x8059","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10009402},{"operation_name":"upsert","last_dispatch_duration_us":1581314,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":1581314,"total_server_duration_us":378731,"encode_duration_us":17374,"operation_id":"0x8034","timeout_ms":10000,"last_server_duration_us":378731,"total_duration_us":10008184},{"operation_name":"upsert","last_dispatch_duration_us":4009233,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9224328,"total_server_duration_us":700704,"encode_duration_us":16799,"operation_id":"0x7fa5","timeout_ms":10000,"last_server_duration_us":394005,"total_duration_us":10007264},{"operation_name":"upsert","last_dispatch_duration_us":2871448,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7697261,"total_server_duration_us":393880,"encode_duration_us":16292,"operation_id":"0x7f6a","timeout_ms":10000,"last_server_duration_us":22554,"total_duration_us":10007204},{"operation_name":"upsert","last_dispatch_duration_us":5896896,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11199216,"total_server_duration_us":885365,"encode_duration_us":20687,"operation_id":"0x8026","timeout_ms":10000,"last_server_duration_us":389256,"total_duration_us":10007056},{"operation_name":"upsert","last_dispatch_duration_us":4077961,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":6882833,"total_server_duration_us":636323,"encode_duration_us":17474,"operation_id":"0x8057","timeout_ms":10000,"last_server_duration_us":390651,"total_duration_us":10005269},{"operation_name":"upsert","last_dispatch_duration_us":7715014,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8275622,"total_server_duration_us":317651,"encode_duration_us":16735,"operation_id":"0x7eaf","timeout_ms":10000,"last_server_duration_us":18570,"total_duration_us":10003703},{"operation_name":"upsert","last_dispatch_duration_us":3185049,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":3185049,"total_server_duration_us":1608843,"encode_duration_us":16167,"operation_id":"0x8035","timeout_ms":10000,"last_server_duration_us":1608843,"total_duration_us":10003070},{"operation_name":"upsert","last_dispatch_duration_us":2677042,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12586266,"total_server_duration_us":3095613,"encode_duration_us":29444,"operation_id":"0x7e86","timeout_ms":10000,"last_server_duration_us":389814,"total_duration_us":10002694}],"total_count":14}} INFO [2022-01-10 05:47:12,476] jepsen worker 1 - jepsen.util 4051 :info :write [0 15] :RequestTimeoutException INFO [2022-01-10 05:47:12,476] jepsen worker 1 - jepsen.util 4141 :invoke :write [0 7] INFO [2022-01-10 05:47:12,620] jepsen worker 55 - jepsen.util 4555 :info :write [18 49] :SyncWriteAmbiguous INFO [2022-01-10 05:47:12,621] jepsen worker 55 - jepsen.util 4645 :invoke :write [18 1] INFO [2022-01-10 05:47:12,735] jepsen worker 47 - jepsen.util 4187 :info :write [15 37] :RequestTimeoutException INFO [2022-01-10 05:47:12,736] jepsen worker 47 - jepsen.util 4277 :invoke :read [15 nil] INFO [2022-01-10 05:47:12,896] jepsen worker 22 - jepsen.util 3892 :info :write [7 13] :RequestTimeoutException INFO [2022-01-10 05:47:12,897] jepsen worker 22 - jepsen.util 3982 :invoke :read [7 nil] INFO [2022-01-10 05:47:12,900] jepsen worker 13 - jepsen.util 4783 :info :write [4 16] :RequestTimeoutException INFO [2022-01-10 05:47:12,900] jepsen worker 13 - jepsen.util 4873 :invoke :read [4 nil] INFO [2022-01-10 05:47:12,906] jepsen worker 21 - jepsen.util 4161 :info :write [7 29] :RequestTimeoutException INFO [2022-01-10 05:47:12,907] jepsen worker 21 - jepsen.util 4251 :invoke :read [7 nil] INFO [2022-01-10 05:47:12,910] jepsen worker 46 - jepsen.util 4276 :info :write [15 27] :RequestTimeoutException INFO [2022-01-10 05:47:12,910] jepsen worker 46 - jepsen.util 4366 :invoke :read [15 nil] INFO [2022-01-10 05:47:13,080] jepsen worker 30 - jepsen.util 3900 :info :write [10 3] :SyncWriteAmbiguous INFO [2022-01-10 05:47:13,081] jepsen worker 30 - jepsen.util 3990 :invoke :read [10 nil] INFO [2022-01-10 05:47:13,125] jepsen worker 83 - jepsen.util 4583 :info :write [27 1] :RequestTimeoutException INFO [2022-01-10 05:47:13,126] jepsen worker 83 - jepsen.util 4673 :invoke :write [27 42] INFO [2022-01-10 05:47:13,132] jepsen worker 87 - jepsen.util 4047 :ok :read [29 0] INFO [2022-01-10 05:47:13,132] jepsen worker 87 - jepsen.util 4047 :invoke :write [29 44] INFO [2022-01-10 05:47:13,137] jepsen worker 42 - jepsen.util 3822 :info :write [14 42] :RequestTimeoutException INFO [2022-01-10 05:47:13,137] jepsen worker 42 - jepsen.util 3912 :invoke :read [14 nil] INFO [2022-01-10 05:47:13,138] jepsen worker 3 - jepsen.util 4053 :ok :read [1 26] INFO [2022-01-10 05:47:13,138] jepsen worker 3 - jepsen.util 4053 :invoke :write [1 39] INFO [2022-01-10 05:47:13,242] jepsen worker 11 - jepsen.util 3971 :info :write [3 5] :RequestTimeoutException INFO [2022-01-10 05:47:13,242] jepsen worker 11 - jepsen.util 4061 :invoke :write [3 35] INFO [2022-01-10 05:47:13,420] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:13,420] jepsen worker nemesis - couchbase.util Rebalance check count: 581 INFO [2022-01-10 05:47:13,420] jepsen worker nemesis - couchbase.util Rebalance stuck count: 491 INFO [2022-01-10 05:47:13,561] jepsen worker 56 - jepsen.util 4466 :info :write [18 47] :RequestTimeoutException INFO [2022-01-10 05:47:13,562] jepsen worker 56 - jepsen.util 4556 :invoke :read [18 nil] INFO [2022-01-10 05:47:13,886] jepsen worker 78 - jepsen.util 5028 :info :write [26 13] :RequestTimeoutException INFO [2022-01-10 05:47:13,886] jepsen worker 78 - jepsen.util 5118 :invoke :write [26 31] INFO [2022-01-10 05:47:13,962] jepsen worker 27 - jepsen.util 4347 :info :write [9 35] :RequestTimeoutException INFO [2022-01-10 05:47:13,962] jepsen worker 27 - jepsen.util 4437 :invoke :write [9 9] INFO [2022-01-10 05:47:14,129] jepsen worker 47 - jepsen.util 4277 :ok :read [15 23] INFO [2022-01-10 05:47:14,129] jepsen worker 47 - jepsen.util 4277 :invoke :read [15 nil] INFO [2022-01-10 05:47:14,263] jepsen worker 86 - jepsen.util 3506 :ok :read [28 45] INFO [2022-01-10 05:47:14,263] jepsen worker 86 - jepsen.util 3506 :invoke :write [28 48] INFO [2022-01-10 05:47:14,426] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:14,426] jepsen worker nemesis - couchbase.util Rebalance check count: 582 INFO [2022-01-10 05:47:14,426] jepsen worker nemesis - couchbase.util Rebalance stuck count: 492 INFO [2022-01-10 05:47:14,721] jepsen worker 23 - jepsen.util 4253 :ok :read [7 :nil] INFO [2022-01-10 05:47:14,722] jepsen worker 23 - jepsen.util 4253 :invoke :read [7 nil] INFO [2022-01-10 05:47:14,760] jepsen worker 20 - jepsen.util 3800 :ok :read [6 25] INFO [2022-01-10 05:47:14,761] jepsen worker 20 - jepsen.util 3800 :invoke :read [6 nil] INFO [2022-01-10 05:47:14,881] jepsen worker 16 - jepsen.util 4156 :info :write [5 2] :RequestTimeoutException INFO [2022-01-10 05:47:14,881] jepsen worker 16 - jepsen.util 4246 :invoke :write [5 4] INFO [2022-01-10 05:47:14,948] jepsen worker 47 - jepsen.util 4277 :ok :read [15 23] INFO [2022-01-10 05:47:14,949] jepsen worker 47 - jepsen.util 4277 :invoke :write [15 13] INFO [2022-01-10 05:47:14,985] jepsen worker 54 - jepsen.util 4194 :info :write [18 23] :RequestTimeoutException INFO [2022-01-10 05:47:14,985] jepsen worker 54 - jepsen.util 4284 :invoke :write [18 3] INFO [2022-01-10 05:47:15,257] jepsen worker 37 - jepsen.util 3817 :fail :read [12 nil] :UnambiguousTimeoutException INFO [2022-01-10 05:47:15,257] jepsen worker 37 - jepsen.util 3817 :invoke :read [12 nil] INFO [2022-01-10 05:47:15,262] jepsen worker 26 - jepsen.util 3806 :info :write [8 21] :RequestTimeoutException INFO [2022-01-10 05:47:15,297] jepsen worker 26 - jepsen.util 3896 :invoke :read [8 nil] INFO [2022-01-10 05:47:15,432] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:15,432] jepsen worker nemesis - couchbase.util Rebalance check count: 583 INFO [2022-01-10 05:47:15,432] jepsen worker nemesis - couchbase.util Rebalance stuck count: 493 INFO [2022-01-10 05:47:15,502] jepsen worker 17 - jepsen.util 4067 :ok :read [5 25] INFO [2022-01-10 05:47:15,502] jepsen worker 17 - jepsen.util 4067 :invoke :read [5 nil] INFO [2022-01-10 05:47:15,755] jepsen worker 80 - jepsen.util 5030 :info :write [26 6] :RequestTimeoutException INFO [2022-01-10 05:47:15,756] jepsen worker 80 - jepsen.util 5120 :invoke :write [26 37] INFO [2022-01-10 05:47:15,761] jepsen worker 88 - jepsen.util 4048 :info :write [29 6] :RequestTimeoutException INFO [2022-01-10 05:47:15,762] jepsen worker 88 - jepsen.util 4138 :invoke :read [29 nil] INFO [2022-01-10 05:47:15,926] jepsen worker 13 - jepsen.util 4873 :ok :read [4 3] INFO [2022-01-10 05:47:15,926] jepsen worker 13 - jepsen.util 4873 :invoke :write [4 29] INFO [2022-01-10 05:47:16,022] jepsen worker 25 - jepsen.util 3895 :info :write [8 25] :RequestTimeoutException INFO [2022-01-10 05:47:16,022] jepsen worker 25 - jepsen.util 3985 :invoke :read [8 nil] INFO [2022-01-10 05:47:16,174] jepsen worker 12 - jepsen.util 4692 :info :write [4 49] :RequestTimeoutException INFO [2022-01-10 05:47:16,175] jepsen worker 12 - jepsen.util 4782 :invoke :read [4 nil] INFO [2022-01-10 05:47:16,438] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:16,438] jepsen worker nemesis - couchbase.util Rebalance check count: 584 INFO [2022-01-10 05:47:16,438] jepsen worker nemesis - couchbase.util Rebalance stuck count: 494 INFO [2022-01-10 05:47:16,826] jepsen worker 73 - jepsen.util 3763 :info :write [24 45] :RequestTimeoutException INFO [2022-01-10 05:47:16,826] jepsen worker 73 - jepsen.util 3853 :invoke :write [24 23] INFO [2022-01-10 05:47:16,841] jepsen worker 71 - jepsen.util 3671 :info :write [23 46] :RequestTimeoutException INFO [2022-01-10 05:47:16,842] jepsen worker 71 - jepsen.util 3761 :invoke :write [23 39] INFO [2022-01-10 05:47:17,107] jepsen worker 39 - jepsen.util 4629 :ok :read [13 15] INFO [2022-01-10 05:47:17,110] jepsen worker 39 - jepsen.util 4629 :invoke :read [13 nil] INFO [2022-01-10 05:47:17,115] jepsen worker 2 - jepsen.util 4412 :ok :read [0 46] INFO [2022-01-10 05:47:17,117] jepsen worker 2 - jepsen.util 4412 :invoke :read [0 nil] INFO [2022-01-10 05:47:17,236] jepsen worker 45 - jepsen.util 4455 :info :write [15 10] :RequestTimeoutException INFO [2022-01-10 05:47:17,237] jepsen worker 45 - jepsen.util 4545 :invoke :read [15 nil] INFO [2022-01-10 05:47:17,251] jepsen worker 43 - jepsen.util 4003 :info :write [14 47] :RequestTimeoutException INFO [2022-01-10 05:47:17,252] jepsen worker 43 - jepsen.util 4093 :invoke :read [14 nil] INFO [2022-01-10 05:47:17,272] jepsen worker 35 - jepsen.util 3815 :info :write [11 39] :RequestTimeoutException INFO [2022-01-10 05:47:17,273] jepsen worker 35 - jepsen.util 3905 :invoke :write [11 49] INFO [2022-01-10 05:47:17,325] jepsen worker 85 - jepsen.util 4495 :ok :read [28 45] INFO [2022-01-10 05:47:17,326] jepsen worker 85 - jepsen.util 4495 :invoke :read [28 nil] INFO [2022-01-10 05:47:17,327] jepsen worker 40 - jepsen.util 3730 :ok :read [13 15] INFO [2022-01-10 05:47:17,328] jepsen worker 40 - jepsen.util 3730 :invoke :write [13 16] INFO [2022-01-10 05:47:17,344] jepsen worker 67 - jepsen.util 4027 :info :write [22 24] :RequestTimeoutException INFO [2022-01-10 05:47:17,345] jepsen worker 67 - jepsen.util 4117 :invoke :read [22 nil] INFO [2022-01-10 05:47:17,443] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:17,443] jepsen worker nemesis - couchbase.util Rebalance check count: 585 INFO [2022-01-10 05:47:17,444] jepsen worker nemesis - couchbase.util Rebalance stuck count: 495 INFO [2022-01-10 05:47:17,622] jepsen worker 34 - jepsen.util 4084 :info :write [11 2] :RequestTimeoutException INFO [2022-01-10 05:47:17,623] jepsen worker 34 - jepsen.util 4174 :invoke :read [11 nil] INFO [2022-01-10 05:47:17,626] jepsen worker 75 - jepsen.util 3855 :info :write [25 22] :RequestTimeoutException INFO [2022-01-10 05:47:17,627] jepsen worker 75 - jepsen.util 3945 :invoke :write [25 33] INFO [2022-01-10 05:47:17,644] jepsen worker 77 - jepsen.util 4577 :info :write [25 44] :RequestTimeoutException INFO [2022-01-10 05:47:17,645] jepsen worker 77 - jepsen.util 4667 :invoke :read [25 nil] INFO [2022-01-10 05:47:17,940] jepsen worker 84 - jepsen.util 4134 :ok :read [28 45] INFO [2022-01-10 05:47:17,941] jepsen worker 84 - jepsen.util 4134 :invoke :read [28 nil] INFO [2022-01-10 05:47:18,045] jepsen worker 64 - jepsen.util 4114 :ok :read [21 7] INFO [2022-01-10 05:47:18,045] jepsen worker 64 - jepsen.util 4114 :invoke :write [21 23] INFO [2022-01-10 05:47:18,051] jepsen worker 65 - jepsen.util 3935 :ok :read [21 7] INFO [2022-01-10 05:47:18,051] jepsen worker 65 - jepsen.util 3935 :invoke :read [21 nil] INFO [2022-01-10 05:47:18,105] jepsen worker 79 - jepsen.util 4849 :info :write [26 14] :RequestTimeoutException INFO [2022-01-10 05:47:18,105] jepsen worker 79 - jepsen.util 4939 :invoke :write [26 31] INFO [2022-01-10 05:47:18,178] jepsen worker 66 - jepsen.util 3846 :ok :read [22 41] INFO [2022-01-10 05:47:18,179] jepsen worker 66 - jepsen.util 3846 :invoke :write [22 30] INFO [2022-01-10 05:47:18,188] jepsen worker 85 - jepsen.util 4495 :ok :read [28 45] INFO [2022-01-10 05:47:18,188] jepsen worker 85 - jepsen.util 4495 :invoke :write [28 5] INFO [2022-01-10 05:47:18,271] jepsen worker 60 - jepsen.util 4380 :ok :read [20 :nil] INFO [2022-01-10 05:47:18,272] jepsen worker 60 - jepsen.util 4380 :invoke :write [20 5] INFO [2022-01-10 05:47:18,283] jepsen worker 18 - jepsen.util 4158 :ok :read [6 25] INFO [2022-01-10 05:47:18,284] jepsen worker 18 - jepsen.util 4158 :invoke :write [6 4] INFO [2022-01-10 05:47:18,284] jepsen worker 6 - jepsen.util 4146 :ok :read [2 14] INFO [2022-01-10 05:47:18,285] jepsen worker 6 - jepsen.util 4146 :invoke :write [2 5] INFO [2022-01-10 05:47:18,384] jepsen worker 84 - jepsen.util 4134 :ok :read [28 45] INFO [2022-01-10 05:47:18,385] jepsen worker 84 - jepsen.util 4134 :invoke :read [28 nil] INFO [2022-01-10 05:47:18,430] jepsen worker 10 - jepsen.util 4060 :ok :read [3 12] INFO [2022-01-10 05:47:18,431] jepsen worker 10 - jepsen.util 4060 :invoke :write [3 36] INFO [2022-01-10 05:47:18,437] jepsen worker 32 - jepsen.util 4352 :ok :read [10 36] INFO [2022-01-10 05:47:18,437] jepsen worker 32 - jepsen.util 4352 :invoke :write [10 8] INFO [2022-01-10 05:47:18,442] jepsen worker 88 - jepsen.util 4138 :ok :read [29 0] INFO [2022-01-10 05:47:18,443] jepsen worker 88 - jepsen.util 4138 :invoke :write [29 9] INFO [2022-01-10 05:47:18,449] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:18,449] jepsen worker nemesis - couchbase.util Rebalance check count: 586 INFO [2022-01-10 05:47:18,450] jepsen worker nemesis - couchbase.util Rebalance stuck count: 496 INFO [2022-01-10 05:47:18,486] jepsen worker 14 - jepsen.util 5054 :info :write [4 18] :RequestTimeoutException INFO [2022-01-10 05:47:18,486] jepsen worker 14 - jepsen.util 5144 :invoke :write [4 16] INFO [2022-01-10 05:47:18,502] jepsen worker 4 - jepsen.util 4684 :ok :read [1 26] INFO [2022-01-10 05:47:18,503] jepsen worker 4 - jepsen.util 4684 :invoke :read [1 nil] INFO [2022-01-10 05:47:18,565] jepsen worker 31 - jepsen.util 4441 :ok :read [10 36] INFO [2022-01-10 05:47:18,565] jepsen worker 31 - jepsen.util 4441 :invoke :read [10 nil] INFO [2022-01-10 05:47:18,579] jepsen worker 41 - jepsen.util 4091 :info :write [13 11] :RequestTimeoutException INFO [2022-01-10 05:47:18,580] jepsen worker 41 - jepsen.util 4181 :invoke :write [13 28] INFO [2022-01-10 05:47:18,772] jepsen worker 82 - jepsen.util 4312 :ok :read [27 30] INFO [2022-01-10 05:47:18,772] jepsen worker 82 - jepsen.util 4312 :invoke :write [27 46] INFO [2022-01-10 05:47:18,815] jepsen worker 43 - jepsen.util 4093 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:18,815] jepsen worker 43 - jepsen.util 4093 :invoke :write [14 10] INFO [2022-01-10 05:47:19,026] jepsen worker 42 - jepsen.util 3912 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:19,026] jepsen worker 22 - jepsen.util 3982 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:19,027] jepsen worker 22 - jepsen.util 3982 :invoke :write [7 19] INFO [2022-01-10 05:47:19,027] jepsen worker 42 - jepsen.util 3912 :invoke :write [14 46] INFO [2022-01-10 05:47:19,066] jepsen worker 63 - jepsen.util 3933 :info :write [21 12] :RequestTimeoutException INFO [2022-01-10 05:47:19,067] jepsen worker 63 - jepsen.util 4023 :invoke :read [21 nil] INFO [2022-01-10 05:47:19,072] jepsen worker 26 - jepsen.util 3896 :ok :read [8 39] INFO [2022-01-10 05:47:19,072] jepsen worker 26 - jepsen.util 3896 :invoke :read [8 nil] INFO [2022-01-10 05:47:19,115] jepsen worker 51 - jepsen.util 5001 :info :write [17 29] :RequestTimeoutException INFO [2022-01-10 05:47:19,115] jepsen worker 51 - jepsen.util 5091 :invoke :read [17 nil] INFO [2022-01-10 05:47:19,226] jepsen worker 46 - jepsen.util 4366 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:19,227] jepsen worker 46 - jepsen.util 4366 :invoke :write [15 24] INFO [2022-01-10 05:47:19,388] jepsen worker 2 - jepsen.util 4412 :fail :read [0 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:19,389] jepsen worker 2 - jepsen.util 4412 :invoke :read [0 nil] INFO [2022-01-10 05:47:19,401] jepsen worker 15 - jepsen.util 3705 :ok :read [5 25] INFO [2022-01-10 05:47:19,402] jepsen worker 15 - jepsen.util 3705 :invoke :read [5 nil] INFO [2022-01-10 05:47:19,431] jepsen worker 89 - jepsen.util 4589 :info :write [29 48] :RequestTimeoutException INFO [2022-01-10 05:47:19,431] jepsen worker 89 - jepsen.util 4679 :invoke :write [29 26] INFO [2022-01-10 05:47:19,456] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:19,456] jepsen worker nemesis - couchbase.util Rebalance check count: 587 INFO [2022-01-10 05:47:19,456] jepsen worker nemesis - couchbase.util Rebalance stuck count: 497 INFO [2022-01-10 05:47:19,468] jepsen worker 76 - jepsen.util 3586 :ok :read [25 :nil] INFO [2022-01-10 05:47:19,468] jepsen worker 21 - jepsen.util 4251 :ok :read [7 :nil] INFO [2022-01-10 05:47:19,468] jepsen worker 76 - jepsen.util 3586 :invoke :read [25 nil] INFO [2022-01-10 05:47:19,469] jepsen worker 21 - jepsen.util 4251 :invoke :read [7 nil] INFO [2022-01-10 05:47:19,501] jepsen worker 69 - jepsen.util 3669 :info :write [23 36] :RequestTimeoutException INFO [2022-01-10 05:47:19,502] jepsen worker 65 - jepsen.util 3935 :ok :read [21 7] INFO [2022-01-10 05:47:19,502] jepsen worker 69 - jepsen.util 3759 :invoke :write [23 42] INFO [2022-01-10 05:47:19,502] jepsen worker 65 - jepsen.util 3935 :invoke :write [21 45] INFO [2022-01-10 05:47:19,609] jepsen worker 23 - jepsen.util 4253 :ok :read [7 :nil] INFO [2022-01-10 05:47:19,609] jepsen worker 23 - jepsen.util 4253 :invoke :read [7 nil] INFO [2022-01-10 05:47:19,627] jepsen worker 28 - jepsen.util 3988 :ok :read [9 33] INFO [2022-01-10 05:47:19,632] jepsen worker 28 - jepsen.util 3988 :invoke :read [9 nil] INFO [2022-01-10 05:47:19,640] jepsen worker 29 - jepsen.util 4079 :ok :read [9 33] INFO [2022-01-10 05:47:19,641] jepsen worker 29 - jepsen.util 4079 :invoke :write [9 35] INFO [2022-01-10 05:47:19,697] jepsen worker 77 - jepsen.util 4667 :ok :read [25 :nil] INFO [2022-01-10 05:47:19,699] jepsen worker 77 - jepsen.util 4667 :invoke :write [25 47] INFO [2022-01-10 05:47:19,711] jepsen worker 45 - jepsen.util 4545 :ok :read [15 23] INFO [2022-01-10 05:47:19,712] jepsen worker 45 - jepsen.util 4545 :invoke :read [15 nil] INFO [2022-01-10 05:47:19,714] jepsen worker 20 - jepsen.util 3800 :ok :read [6 25] INFO [2022-01-10 05:47:19,715] jepsen worker 20 - jepsen.util 3800 :invoke :read [6 nil] INFO [2022-01-10 05:47:19,819] jepsen worker 21 - jepsen.util 4251 :ok :read [7 :nil] INFO [2022-01-10 05:47:19,819] jepsen worker 21 - jepsen.util 4251 :invoke :write [7 13] INFO [2022-01-10 05:47:19,832] jepsen worker 45 - jepsen.util 4545 :ok :read [15 23] INFO [2022-01-10 05:47:19,833] jepsen worker 45 - jepsen.util 4545 :invoke :write [15 21] INFO [2022-01-10 05:47:20,031] jepsen worker 23 - jepsen.util 4253 :ok :read [7 :nil] INFO [2022-01-10 05:47:20,031] jepsen worker 23 - jepsen.util 4253 :invoke :write [7 23] INFO [2022-01-10 05:47:20,070] jepsen worker 62 - jepsen.util 4562 :ok :read [20 :nil] INFO [2022-01-10 05:47:20,070] jepsen worker 76 - jepsen.util 3586 :ok :read [25 :nil] INFO [2022-01-10 05:47:20,071] jepsen worker 62 - jepsen.util 4562 :invoke :read [20 nil] INFO [2022-01-10 05:47:20,071] jepsen worker 76 - jepsen.util 3586 :invoke :read [25 nil] INFO [2022-01-10 05:47:20,095] jepsen worker 9 - jepsen.util 4689 :info :write [3 36] :SyncWriteAmbiguous INFO [2022-01-10 05:47:20,095] jepsen worker 9 - jepsen.util 4779 :invoke :write [3 42] INFO [2022-01-10 05:47:20,113] jepsen worker 72 - jepsen.util 4392 :ok :read [24 18] INFO [2022-01-10 05:47:20,113] jepsen worker 72 - jepsen.util 4392 :invoke :write [24 21] INFO [2022-01-10 05:47:20,149] jepsen worker 56 - jepsen.util 4556 :ok :read [18 14] INFO [2022-01-10 05:47:20,150] jepsen worker 56 - jepsen.util 4556 :invoke :read [18 nil] INFO [2022-01-10 05:47:20,266] jepsen worker 15 - jepsen.util 3705 :ok :read [5 25] INFO [2022-01-10 05:47:20,266] jepsen worker 15 - jepsen.util 3705 :invoke :write [5 17] INFO [2022-01-10 05:47:20,291] jepsen worker 2 - jepsen.util 4412 :ok :read [0 46] INFO [2022-01-10 05:47:20,292] jepsen worker 2 - jepsen.util 4412 :invoke :read [0 nil] INFO [2022-01-10 05:47:20,346] jepsen worker 76 - jepsen.util 3586 :ok :read [25 :nil] INFO [2022-01-10 05:47:20,346] jepsen worker 62 - jepsen.util 4562 :ok :read [20 :nil] INFO [2022-01-10 05:47:20,349] jepsen worker 76 - jepsen.util 3586 :invoke :write [25 5] INFO [2022-01-10 05:47:20,349] jepsen worker 62 - jepsen.util 4562 :invoke :write [20 8] INFO [2022-01-10 05:47:20,359] jepsen worker 8 - jepsen.util 4418 :ok :read [2 14] INFO [2022-01-10 05:47:20,360] jepsen worker 8 - jepsen.util 4418 :invoke :read [2 nil] INFO [2022-01-10 05:47:20,432] jepsen worker 48 - jepsen.util 3918 :info :write [16 29] :RequestTimeoutException INFO [2022-01-10 05:47:20,432] jepsen worker 48 - jepsen.util 4008 :invoke :write [16 31] INFO [2022-01-10 05:47:20,436] jepsen worker 24 - jepsen.util 3804 :info :write [8 47] :RequestTimeoutException INFO [2022-01-10 05:47:20,437] jepsen worker 24 - jepsen.util 3894 :invoke :read [8 nil] INFO [2022-01-10 05:47:20,462] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:20,462] jepsen worker nemesis - couchbase.util Rebalance check count: 588 INFO [2022-01-10 05:47:20,462] jepsen worker nemesis - couchbase.util Rebalance stuck count: 498 INFO [2022-01-10 05:47:20,505] jepsen worker 74 - jepsen.util 4664 :info :write [24 13] :RequestTimeoutException INFO [2022-01-10 05:47:20,505] jepsen worker 74 - jepsen.util 4754 :invoke :read [24 nil] INFO [2022-01-10 05:47:20,516] jepsen worker 8 - jepsen.util 4418 :ok :read [2 14] INFO [2022-01-10 05:47:20,517] jepsen worker 8 - jepsen.util 4418 :invoke :read [2 nil] WARN [2022-01-10 05:47:20,577] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12674630,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":15669507,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10608357,"encode_duration_us":16334,"operation_id":"0x7e06","timeout_ms":10000,"last_server_duration_us":10215471,"total_duration_us":10001152},{"operation_name":"upsert","last_dispatch_duration_us":19379703,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":23286647,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":10841398,"encode_duration_us":17157,"operation_id":"0x7d8a","timeout_ms":10000,"last_server_duration_us":10120618,"total_duration_us":10009917}],"total_count":2}} WARN [2022-01-10 05:47:20,577] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17190471,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":17190471,"total_server_duration_us":421567,"operation_id":"0x7ed3","timeout_ms":30000,"last_server_duration_us":421567,"total_duration_us":17192369},{"operation_name":"get","last_dispatch_duration_us":15213126,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":15213126,"total_server_duration_us":819485,"operation_id":"0x7f60","timeout_ms":30000,"last_server_duration_us":819485,"total_duration_us":15213282},{"operation_name":"get","last_dispatch_duration_us":14744772,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14744772,"total_server_duration_us":815664,"operation_id":"0x7f6c","timeout_ms":30000,"last_server_duration_us":815664,"total_duration_us":14745102},{"operation_name":"get","last_dispatch_duration_us":3490266,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":3490266,"total_server_duration_us":25976,"operation_id":"0x7fa1","timeout_ms":30000,"last_server_duration_us":25976,"total_duration_us":12226640},{"operation_name":"get","last_dispatch_duration_us":11824991,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11824991,"total_server_duration_us":23,"operation_id":"0x7ff5","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":11825309},{"operation_name":"upsert","last_dispatch_duration_us":12291321,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12291321,"total_server_duration_us":609298,"encode_duration_us":17610,"operation_id":"0x801d","timeout_ms":10000,"last_server_duration_us":609298,"total_duration_us":10009471},{"operation_name":"upsert","last_dispatch_duration_us":7660910,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":7660910,"total_server_duration_us":501794,"encode_duration_us":16230,"operation_id":"0x8170","timeout_ms":10000,"last_server_duration_us":501794,"total_duration_us":10006114},{"operation_name":"upsert","last_dispatch_duration_us":5446487,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11599761,"total_server_duration_us":1192247,"encode_duration_us":16480,"operation_id":"0x814a","timeout_ms":10000,"last_server_duration_us":801596,"total_duration_us":10005942},{"operation_name":"upsert","last_dispatch_duration_us":4953437,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":6960846,"total_server_duration_us":835486,"encode_duration_us":16250,"operation_id":"0x8105","timeout_ms":10000,"last_server_duration_us":456755,"total_duration_us":10005700},{"operation_name":"upsert","last_dispatch_duration_us":8007782,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":8007782,"total_server_duration_us":440185,"encode_duration_us":16270,"operation_id":"0x819e","timeout_ms":10000,"last_server_duration_us":440185,"total_duration_us":10005570}],"total_count":19}} INFO [2022-01-10 05:47:20,586] jepsen worker 7 - jepsen.util 4327 :info :write [2 16] :RequestTimeoutException INFO [2022-01-10 05:47:20,587] jepsen worker 7 - jepsen.util 4417 :invoke :write [2 23] INFO [2022-01-10 05:47:20,671] jepsen worker 61 - jepsen.util 4291 :info :write [20 33] :RequestTimeoutException INFO [2022-01-10 05:47:20,671] jepsen worker 61 - jepsen.util 4381 :invoke :write [20 40] INFO [2022-01-10 05:47:20,723] jepsen worker 20 - jepsen.util 3800 :ok :read [6 25] INFO [2022-01-10 05:47:20,723] jepsen worker 20 - jepsen.util 3800 :invoke :write [6 18] INFO [2022-01-10 05:47:20,824] jepsen worker 81 - jepsen.util 4581 :info :write [27 27] :RequestTimeoutException INFO [2022-01-10 05:47:20,825] jepsen worker 81 - jepsen.util 4671 :invoke :write [27 5] INFO [2022-01-10 05:47:20,895] jepsen worker 44 - jepsen.util 4274 :info :write [14 1] :RequestTimeoutException INFO [2022-01-10 05:47:20,895] jepsen worker 44 - jepsen.util 4364 :invoke :read [14 nil] INFO [2022-01-10 05:47:21,029] jepsen worker 33 - jepsen.util 4353 :info :write [11 5] :RequestTimeoutException INFO [2022-01-10 05:47:21,030] jepsen worker 33 - jepsen.util 4443 :invoke :write [11 3] INFO [2022-01-10 05:47:21,081] jepsen worker 68 - jepsen.util 4298 :info :write [22 19] :RequestTimeoutException INFO [2022-01-10 05:47:21,082] jepsen worker 68 - jepsen.util 4388 :invoke :write [22 41] INFO [2022-01-10 05:47:21,301] jepsen worker 34 - jepsen.util 4174 :ok :read [11 21] INFO [2022-01-10 05:47:21,302] jepsen worker 34 - jepsen.util 4174 :invoke :read [11 nil] INFO [2022-01-10 05:47:21,313] jepsen worker 67 - jepsen.util 4117 :fail :read [22 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:21,314] jepsen worker 67 - jepsen.util 4117 :invoke :write [22 37] INFO [2022-01-10 05:47:21,465] jepsen worker 52 - jepsen.util 5272 :info :write [17 1] :RequestTimeoutException INFO [2022-01-10 05:47:21,466] jepsen worker 52 - jepsen.util 5362 :invoke :read [17 nil] INFO [2022-01-10 05:47:21,467] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:21,467] jepsen worker nemesis - couchbase.util Rebalance check count: 589 INFO [2022-01-10 05:47:21,467] jepsen worker nemesis - couchbase.util Rebalance stuck count: 499 INFO [2022-01-10 05:47:21,574] jepsen worker 84 - jepsen.util 4134 :ok :read [28 45] INFO [2022-01-10 05:47:21,574] jepsen worker 84 - jepsen.util 4134 :invoke :read [28 nil] INFO [2022-01-10 05:47:21,680] jepsen worker 50 - jepsen.util 4370 :info :write [16 12] :RequestTimeoutException INFO [2022-01-10 05:47:21,680] jepsen worker 50 - jepsen.util 4460 :invoke :write [16 17] INFO [2022-01-10 05:47:21,726] jepsen worker 19 - jepsen.util 4339 :info :write [6 17] :RequestTimeoutException INFO [2022-01-10 05:47:21,726] jepsen worker 19 - jepsen.util 4429 :invoke :read [6 nil] INFO [2022-01-10 05:47:21,873] jepsen worker 51 - jepsen.util 5091 :ok :read [17 41] INFO [2022-01-10 05:47:21,873] jepsen worker 52 - jepsen.util 5362 :ok :read [17 41] INFO [2022-01-10 05:47:21,873] jepsen worker 51 - jepsen.util 5091 :invoke :read [17 nil] INFO [2022-01-10 05:47:21,874] jepsen worker 52 - jepsen.util 5362 :invoke :read [17 nil] INFO [2022-01-10 05:47:21,941] jepsen worker 58 - jepsen.util 3928 :info :write [19 5] :RequestTimeoutException INFO [2022-01-10 05:47:21,942] jepsen worker 58 - jepsen.util 4018 :invoke :write [19 28] INFO [2022-01-10 05:47:21,946] jepsen worker 0 - jepsen.util 4050 :info :write [0 48] :RequestTimeoutException INFO [2022-01-10 05:47:21,946] jepsen worker 0 - jepsen.util 4140 :invoke :write [0 6] INFO [2022-01-10 05:47:21,955] jepsen worker 5 - jepsen.util 4055 :info :write [1 40] :RequestTimeoutException INFO [2022-01-10 05:47:21,956] jepsen worker 5 - jepsen.util 4145 :invoke :write [1 8] INFO [2022-01-10 05:47:21,996] jepsen worker 2 - jepsen.util 4412 :ok :read [0 46] INFO [2022-01-10 05:47:21,997] jepsen worker 2 - jepsen.util 4412 :invoke :read [0 nil] INFO [2022-01-10 05:47:22,070] jepsen worker 52 - jepsen.util 5362 :ok :read [17 41] INFO [2022-01-10 05:47:22,070] jepsen worker 52 - jepsen.util 5362 :invoke :write [17 44] INFO [2022-01-10 05:47:22,085] jepsen worker 57 - jepsen.util 4917 :info :write [19 11] :RequestTimeoutException INFO [2022-01-10 05:47:22,085] jepsen worker 49 - jepsen.util 3919 :info :write [16 44] :RequestTimeoutException INFO [2022-01-10 05:47:22,085] jepsen worker 57 - jepsen.util 5007 :invoke :read [19 nil] INFO [2022-01-10 05:47:22,086] jepsen worker 49 - jepsen.util 4009 :invoke :read [16 nil] WARN [2022-01-10 05:47:22,175] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12536410,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":12536410,"total_server_duration_us":2678,"operation_id":"0x7e7d","timeout_ms":30000,"last_server_duration_us":2678,"total_duration_us":26881227},{"operation_name":"get","last_dispatch_duration_us":11749120,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11749120,"total_server_duration_us":5221,"operation_id":"0x7f98","timeout_ms":30000,"last_server_duration_us":5221,"total_duration_us":22032914},{"operation_name":"get","last_dispatch_duration_us":292196,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":292196,"total_server_duration_us":10095,"operation_id":"0x7fc8","timeout_ms":30000,"last_server_duration_us":10095,"total_duration_us":21268797},{"operation_name":"get","last_dispatch_duration_us":11774833,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11774833,"total_server_duration_us":16050,"operation_id":"0x7fc6","timeout_ms":30000,"last_server_duration_us":16050,"total_duration_us":21047363},{"operation_name":"get","last_dispatch_duration_us":10629563,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10629563,"total_server_duration_us":746737,"operation_id":"0x80e4","timeout_ms":30000,"last_server_duration_us":746737,"total_duration_us":10631668},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","encode_duration_us":18282,"operation_id":"0x81fc","timeout_ms":10000,"total_duration_us":10010415},{"operation_name":"upsert","encode_duration_us":19681,"operation_id":"0x8147","timeout_ms":10000,"total_duration_us":10009728},{"operation_name":"upsert","last_dispatch_duration_us":398632,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":398632,"total_server_duration_us":30445,"encode_duration_us":17551,"operation_id":"0x81d6","timeout_ms":10000,"last_server_duration_us":30445,"total_duration_us":10009427},{"operation_name":"upsert","last_dispatch_duration_us":9100224,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":14979503,"total_server_duration_us":41057,"encode_duration_us":16512,"operation_id":"0x809a","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10008320},{"operation_name":"upsert","last_dispatch_duration_us":12004356,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":12004356,"total_server_duration_us":795168,"encode_duration_us":16984,"operation_id":"0x80e5","timeout_ms":10000,"last_server_duration_us":795168,"total_duration_us":10007686}],"total_count":17}} WARN [2022-01-10 05:47:22,175] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10646631,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16341854,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10270730,"encode_duration_us":16886,"operation_id":"0x7f6d","timeout_ms":10000,"last_server_duration_us":9798741,"total_duration_us":10001243},{"operation_name":"upsert","last_dispatch_duration_us":27263950,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":32825892,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":10583299,"encode_duration_us":18122,"operation_id":"0x7d29","timeout_ms":10000,"last_server_duration_us":10356769,"total_duration_us":10001711},{"operation_name":"upsert","last_dispatch_duration_us":12718425,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12718425,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9027816,"encode_duration_us":20189,"operation_id":"0x8106","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10004528}],"total_count":3}} WARN [2022-01-10 05:47:22,184] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":3309755,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":19073482,"total_server_duration_us":1634360,"operation_id":"0x7d27","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":30009989},{"operation_name":"upsert","last_dispatch_duration_us":3765330,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":12027877,"total_server_duration_us":1962169,"encode_duration_us":18492,"operation_id":"0x80ba","timeout_ms":10000,"last_server_duration_us":120142,"total_duration_us":10008971},{"operation_name":"upsert","last_dispatch_duration_us":32411,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":859289,"total_server_duration_us":541431,"encode_duration_us":16578,"operation_id":"0x822b","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10007535},{"operation_name":"upsert","last_dispatch_duration_us":6322186,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":15024829,"total_server_duration_us":1001226,"encode_duration_us":16608,"operation_id":"0x8078","timeout_ms":10000,"last_server_duration_us":606941,"total_duration_us":10007385},{"operation_name":"upsert","last_dispatch_duration_us":7593738,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12361194,"total_server_duration_us":1608678,"encode_duration_us":17446,"operation_id":"0x808f","timeout_ms":10000,"last_server_duration_us":1220814,"total_duration_us":10006691},{"operation_name":"upsert","last_dispatch_duration_us":752704,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9911846,"total_server_duration_us":439718,"encode_duration_us":17459,"operation_id":"0x81d3","timeout_ms":10000,"last_server_duration_us":25015,"total_duration_us":10004428},{"operation_name":"upsert","last_dispatch_duration_us":6276874,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11112855,"total_server_duration_us":1108631,"encode_duration_us":17919,"operation_id":"0x813c","timeout_ms":10000,"last_server_duration_us":738669,"total_duration_us":10002161},{"operation_name":"upsert","last_dispatch_duration_us":1364816,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9943225,"total_server_duration_us":873610,"encode_duration_us":18112,"operation_id":"0x8178","timeout_ms":10000,"last_server_duration_us":172531,"total_duration_us":10001753},{"operation_name":"get","last_dispatch_duration_us":3454560,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3454560,"total_server_duration_us":11992,"operation_id":"0x80e8","timeout_ms":30000,"last_server_duration_us":11992,"total_duration_us":9678875},{"operation_name":"get","last_dispatch_duration_us":7051513,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7051513,"total_server_duration_us":23,"operation_id":"0x81f6","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":7053625}],"total_count":22}} INFO [2022-01-10 05:47:22,268] jepsen worker 12 - jepsen.util 4782 :ok :read [4 3] INFO [2022-01-10 05:47:22,269] jepsen worker 12 - jepsen.util 4782 :invoke :write [4 30] WARN [2022-01-10 05:47:22,275] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15146585,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":15146585,"total_server_duration_us":808047,"operation_id":"0x805a","timeout_ms":30000,"last_server_duration_us":808047,"total_duration_us":15148563},{"operation_name":"get","last_dispatch_duration_us":8712723,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8712723,"total_server_duration_us":827147,"operation_id":"0x80f1","timeout_ms":30000,"last_server_duration_us":827147,"total_duration_us":13883967},{"operation_name":"get","last_dispatch_duration_us":8093326,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8093326,"total_server_duration_us":10692,"operation_id":"0x817a","timeout_ms":30000,"last_server_duration_us":10692,"total_duration_us":10196947},{"operation_name":"upsert","last_dispatch_duration_us":14390257,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":14390257,"total_server_duration_us":1020055,"encode_duration_us":19144,"operation_id":"0x80bd","timeout_ms":10000,"last_server_duration_us":1020055,"total_duration_us":10010116},{"operation_name":"upsert","last_dispatch_duration_us":6544715,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12013430,"total_server_duration_us":59841,"encode_duration_us":16125,"operation_id":"0x814d","timeout_ms":10000,"last_server_duration_us":27583,"total_duration_us":10009505},{"operation_name":"upsert","last_dispatch_duration_us":1186899,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9624226,"total_server_duration_us":752560,"encode_duration_us":19355,"operation_id":"0x81d2","timeout_ms":10000,"last_server_duration_us":344447,"total_duration_us":10008704},{"operation_name":"upsert","last_dispatch_duration_us":33447,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":776438,"total_server_duration_us":480809,"encode_duration_us":16198,"operation_id":"0x8097","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10008161},{"operation_name":"upsert","last_dispatch_duration_us":7172972,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12182352,"total_server_duration_us":375219,"encode_duration_us":16301,"operation_id":"0x8114","timeout_ms":10000,"last_server_duration_us":209334,"total_duration_us":10007642},{"operation_name":"upsert","last_dispatch_duration_us":6285302,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11644059,"total_server_duration_us":2351723,"encode_duration_us":80224,"operation_id":"0x80ef","timeout_ms":10000,"last_server_duration_us":2161887,"total_duration_us":10006250},{"operation_name":"upsert","last_dispatch_duration_us":7394541,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11960332,"total_server_duration_us":612036,"encode_duration_us":17926,"operation_id":"0x8149","timeout_ms":10000,"last_server_duration_us":217751,"total_duration_us":10003052}],"total_count":21}} WARN [2022-01-10 05:47:22,275] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13948439,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":13948439,"total_server_duration_us":43,"operation_id":"0x80c4","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":13949748},{"operation_name":"upsert","last_dispatch_duration_us":11172961,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":14836529,"total_server_duration_us":818676,"encode_duration_us":16559,"operation_id":"0x8093","timeout_ms":10000,"last_server_duration_us":232539,"total_duration_us":10008270},{"operation_name":"upsert","last_dispatch_duration_us":10966495,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10966495,"total_server_duration_us":192719,"encode_duration_us":16390,"operation_id":"0x8146","timeout_ms":10000,"last_server_duration_us":192719,"total_duration_us":10008143},{"operation_name":"upsert","last_dispatch_duration_us":131922,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1434166,"total_server_duration_us":281496,"encode_duration_us":16410,"operation_id":"0x81d5","timeout_ms":10000,"last_server_duration_us":73959,"total_duration_us":10007785},{"operation_name":"upsert","last_dispatch_duration_us":807694,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10375150,"total_server_duration_us":394208,"encode_duration_us":16430,"operation_id":"0x819c","timeout_ms":10000,"last_server_duration_us":186156,"total_duration_us":10007517},{"operation_name":"upsert","last_dispatch_duration_us":79612,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1533181,"total_server_duration_us":465747,"encode_duration_us":17562,"operation_id":"0x8216","timeout_ms":10000,"last_server_duration_us":22637,"total_duration_us":10006644},{"operation_name":"upsert","last_dispatch_duration_us":3284703,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":12712071,"total_server_duration_us":4290164,"encode_duration_us":18611,"operation_id":"0x8067","timeout_ms":10000,"last_server_duration_us":2480261,"total_duration_us":10005650},{"operation_name":"upsert","last_dispatch_duration_us":11994884,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":14433299,"total_server_duration_us":11550624,"encode_duration_us":25003,"operation_id":"0x808d","timeout_ms":10000,"last_server_duration_us":11161368,"total_duration_us":10005052},{"operation_name":"upsert","last_dispatch_duration_us":10780561,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10780561,"total_server_duration_us":184733,"encode_duration_us":16992,"operation_id":"0x814e","timeout_ms":10000,"last_server_duration_us":184733,"total_duration_us":10003849},{"operation_name":"upsert","last_dispatch_duration_us":11560462,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":14824036,"total_server_duration_us":2541158,"encode_duration_us":17471,"operation_id":"0x808c","timeout_ms":10000,"last_server_duration_us":382318,"total_duration_us":10002700}],"total_count":18}} INFO [2022-01-10 05:47:22,284] jepsen worker 51 - jepsen.util 5091 :ok :read [17 41] INFO [2022-01-10 05:47:22,284] jepsen worker 51 - jepsen.util 5091 :invoke :write [17 39] INFO [2022-01-10 05:47:22,286] jepsen worker 70 - jepsen.util 4120 :info :write [23 8] :RequestTimeoutException INFO [2022-01-10 05:47:22,287] jepsen worker 70 - jepsen.util 4210 :invoke :read [23 nil] WARN [2022-01-10 05:47:22,375] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12073457,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13654771,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9927291,"encode_duration_us":17374,"operation_id":"0x8034","timeout_ms":10000,"last_server_duration_us":9548560,"total_duration_us":10008184}],"total_count":1}} INFO [2022-01-10 05:47:22,418] jepsen worker 2 - jepsen.util 4412 :ok :read [0 46] INFO [2022-01-10 05:47:22,418] jepsen worker 2 - jepsen.util 4412 :invoke :read [0 nil] INFO [2022-01-10 05:47:22,452] jepsen worker 53 - jepsen.util 5003 :info :write [17 41] :RequestTimeoutException INFO [2022-01-10 05:47:22,452] jepsen worker 53 - jepsen.util 5093 :invoke :read [17 nil] INFO [2022-01-10 05:47:22,473] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:22,473] jepsen worker nemesis - couchbase.util Rebalance check count: 590 INFO [2022-01-10 05:47:22,473] jepsen worker nemesis - couchbase.util Rebalance stuck count: 500 WARN [2022-01-10 05:47:22,475] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11994884,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14433299,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":11550624,"encode_duration_us":25003,"operation_id":"0x808d","timeout_ms":10000,"last_server_duration_us":11161368,"total_duration_us":10005052},{"operation_name":"upsert","last_dispatch_duration_us":16374131,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16374131,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9022513,"encode_duration_us":16211,"operation_id":"0x805c","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10007212}],"total_count":2}} WARN [2022-01-10 05:47:22,476] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13061533,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":13061533,"total_server_duration_us":660054,"operation_id":"0x80e6","timeout_ms":30000,"last_server_duration_us":660054,"total_duration_us":13063590},{"operation_name":"get","last_dispatch_duration_us":8660072,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8660072,"total_server_duration_us":194375,"operation_id":"0x819a","timeout_ms":30000,"last_server_duration_us":194375,"total_duration_us":10765451},{"operation_name":"get","last_dispatch_duration_us":10623300,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10623300,"total_server_duration_us":23,"operation_id":"0x814c","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":10625228},{"operation_name":"upsert","last_dispatch_duration_us":3579638,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":12540965,"total_server_duration_us":3461795,"encode_duration_us":24252,"operation_id":"0x808b","timeout_ms":10000,"last_server_duration_us":525619,"total_duration_us":10007977},{"operation_name":"upsert","last_dispatch_duration_us":4466987,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13606738,"total_server_duration_us":765963,"encode_duration_us":16500,"operation_id":"0x817b","timeout_ms":10000,"last_server_duration_us":368872,"total_duration_us":10006062},{"operation_name":"upsert","last_dispatch_duration_us":8947733,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8947733,"total_server_duration_us":394846,"encode_duration_us":17552,"operation_id":"0x8208","timeout_ms":10000,"last_server_duration_us":394846,"total_duration_us":10005112},{"operation_name":"upsert","last_dispatch_duration_us":12228534,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":15081810,"total_server_duration_us":1034877,"encode_duration_us":18744,"operation_id":"0x808e","timeout_ms":10000,"last_server_duration_us":644784,"total_duration_us":10004066},{"operation_name":"upsert","last_dispatch_duration_us":7588298,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7588298,"total_server_duration_us":576974,"encode_duration_us":17559,"operation_id":"0x81f9","timeout_ms":10000,"last_server_duration_us":576974,"total_duration_us":10003957},{"operation_name":"get","last_dispatch_duration_us":8655557,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8655557,"total_server_duration_us":126,"operation_id":"0x81d1","timeout_ms":30000,"last_server_duration_us":126,"total_duration_us":9223234},{"operation_name":"upsert","last_dispatch_duration_us":9139164,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9139164,"total_server_duration_us":9024634,"encode_duration_us":17242,"operation_id":"0x80bc","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":9143976}],"total_count":16}} INFO [2022-01-10 05:47:22,516] jepsen worker 1 - jepsen.util 4141 :info :write [0 7] :RequestTimeoutException INFO [2022-01-10 05:47:22,516] jepsen worker 1 - jepsen.util 4231 :invoke :read [0 nil] INFO [2022-01-10 05:47:22,635] jepsen worker 70 - jepsen.util 4210 :ok :read [23 26] INFO [2022-01-10 05:47:22,636] jepsen worker 70 - jepsen.util 4210 :invoke :write [23 28] INFO [2022-01-10 05:47:22,641] jepsen worker 55 - jepsen.util 4645 :info :write [18 1] :RequestTimeoutException INFO [2022-01-10 05:47:22,642] jepsen worker 55 - jepsen.util 4735 :invoke :read [18 nil] INFO [2022-01-10 05:47:22,662] jepsen worker 28 - jepsen.util 3988 :ok :read [9 33] INFO [2022-01-10 05:47:22,662] jepsen worker 28 - jepsen.util 3988 :invoke :write [9 6] INFO [2022-01-10 05:47:23,144] jepsen worker 83 - jepsen.util 4673 :info :write [27 42] :RequestTimeoutException INFO [2022-01-10 05:47:23,145] jepsen worker 83 - jepsen.util 4763 :invoke :write [27 16] INFO [2022-01-10 05:47:23,160] jepsen worker 87 - jepsen.util 4047 :info :write [29 44] :RequestTimeoutException INFO [2022-01-10 05:47:23,160] jepsen worker 3 - jepsen.util 4053 :info :write [1 39] :RequestTimeoutException INFO [2022-01-10 05:47:23,160] jepsen worker 3 - jepsen.util 4143 :invoke :read [1 nil] INFO [2022-01-10 05:47:23,161] jepsen worker 87 - jepsen.util 4137 :invoke :write [29 41] INFO [2022-01-10 05:47:23,270] jepsen worker 11 - jepsen.util 4061 :info :write [3 35] :RequestTimeoutException INFO [2022-01-10 05:47:23,270] jepsen worker 11 - jepsen.util 4151 :invoke :write [3 25] INFO [2022-01-10 05:47:23,479] jepsen worker 53 - jepsen.util 5093 :ok :read [17 41] INFO [2022-01-10 05:47:23,480] jepsen worker 53 - jepsen.util 5093 :invoke :write [17 8] INFO [2022-01-10 05:47:23,482] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:23,482] jepsen worker nemesis - couchbase.util Rebalance check count: 591 INFO [2022-01-10 05:47:23,482] jepsen worker nemesis - couchbase.util Rebalance stuck count: 501 INFO [2022-01-10 05:47:23,514] jepsen worker 84 - jepsen.util 4134 :ok :read [28 45] INFO [2022-01-10 05:47:23,515] jepsen worker 84 - jepsen.util 4134 :invoke :read [28 nil] INFO [2022-01-10 05:47:23,660] jepsen worker 2 - jepsen.util 4412 :ok :read [0 46] INFO [2022-01-10 05:47:23,661] jepsen worker 2 - jepsen.util 4412 :invoke :write [0 34] INFO [2022-01-10 05:47:23,668] jepsen worker 57 - jepsen.util 5007 :ok :read [19 31] INFO [2022-01-10 05:47:23,669] jepsen worker 57 - jepsen.util 5007 :invoke :write [19 30] INFO [2022-01-10 05:47:23,855] jepsen worker 44 - jepsen.util 4364 :ok :read [14 14] INFO [2022-01-10 05:47:23,856] jepsen worker 44 - jepsen.util 4364 :invoke :write [14 21] INFO [2022-01-10 05:47:23,858] jepsen worker 26 - jepsen.util 3896 :ok :read [8 39] INFO [2022-01-10 05:47:23,859] jepsen worker 26 - jepsen.util 3896 :invoke :write [8 26] INFO [2022-01-10 05:47:23,906] jepsen worker 78 - jepsen.util 5118 :info :write [26 31] :RequestTimeoutException INFO [2022-01-10 05:47:23,907] jepsen worker 78 - jepsen.util 5208 :invoke :read [26 nil] INFO [2022-01-10 05:47:23,991] jepsen worker 27 - jepsen.util 4437 :info :write [9 9] :RequestTimeoutException INFO [2022-01-10 05:47:23,992] jepsen worker 27 - jepsen.util 4527 :invoke :read [9 nil] INFO [2022-01-10 05:47:24,295] jepsen worker 86 - jepsen.util 3506 :info :write [28 48] :RequestTimeoutException INFO [2022-01-10 05:47:24,296] jepsen worker 86 - jepsen.util 3596 :invoke :write [28 48] INFO [2022-01-10 05:47:24,313] jepsen worker 78 - jepsen.util 5208 :ok :read [26 10] INFO [2022-01-10 05:47:24,314] jepsen worker 78 - jepsen.util 5208 :invoke :read [26 nil] INFO [2022-01-10 05:47:24,357] jepsen worker 39 - jepsen.util 4629 :ok :read [13 15] INFO [2022-01-10 05:47:24,358] jepsen worker 39 - jepsen.util 4629 :invoke :read [13 nil] INFO [2022-01-10 05:47:24,402] jepsen worker 78 - jepsen.util 5208 :ok :read [26 10] INFO [2022-01-10 05:47:24,402] jepsen worker 78 - jepsen.util 5208 :invoke :write [26 6] INFO [2022-01-10 05:47:24,488] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:24,489] jepsen worker nemesis - couchbase.util Rebalance check count: 592 INFO [2022-01-10 05:47:24,489] jepsen worker nemesis - couchbase.util Rebalance stuck count: 502 INFO [2022-01-10 05:47:24,713] jepsen worker 24 - jepsen.util 3894 :ok :read [8 39] INFO [2022-01-10 05:47:24,714] jepsen worker 24 - jepsen.util 3894 :invoke :read [8 nil] INFO [2022-01-10 05:47:24,904] jepsen worker 16 - jepsen.util 4246 :info :write [5 4] :RequestTimeoutException INFO [2022-01-10 05:47:24,905] jepsen worker 16 - jepsen.util 4336 :invoke :write [5 6] INFO [2022-01-10 05:47:24,974] jepsen worker 47 - jepsen.util 4277 :info :write [15 13] :RequestTimeoutException INFO [2022-01-10 05:47:24,975] jepsen worker 47 - jepsen.util 4367 :invoke :write [15 34] INFO [2022-01-10 05:47:25,006] jepsen worker 54 - jepsen.util 4284 :info :write [18 3] :RequestTimeoutException INFO [2022-01-10 05:47:25,007] jepsen worker 54 - jepsen.util 4374 :invoke :write [18 2] INFO [2022-01-10 05:47:25,496] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:25,496] jepsen worker nemesis - couchbase.util Rebalance check count: 593 INFO [2022-01-10 05:47:25,587] jepsen worker nemesis - couchbase.util Rebalance stuck count: 503 INFO [2022-01-10 05:47:25,634] jepsen worker 25 - jepsen.util 3985 :ok :read [8 39] INFO [2022-01-10 05:47:25,635] jepsen worker 25 - jepsen.util 3985 :invoke :read [8 nil] INFO [2022-01-10 05:47:25,781] jepsen worker 80 - jepsen.util 5120 :info :write [26 37] :RequestTimeoutException INFO [2022-01-10 05:47:25,782] jepsen worker 80 - jepsen.util 5210 :invoke :write [26 5] INFO [2022-01-10 05:47:25,827] jepsen worker 8 - jepsen.util 4418 :ok :read [2 14] INFO [2022-01-10 05:47:25,828] jepsen worker 8 - jepsen.util 4418 :invoke :read [2 nil] INFO [2022-01-10 05:47:25,949] jepsen worker 13 - jepsen.util 4873 :info :write [4 29] :RequestTimeoutException INFO [2022-01-10 05:47:25,951] jepsen worker 13 - jepsen.util 4963 :invoke :write [4 13] INFO [2022-01-10 05:47:26,227] jepsen worker 39 - jepsen.util 4629 :ok :read [13 15] INFO [2022-01-10 05:47:26,227] jepsen worker 39 - jepsen.util 4629 :invoke :write [13 39] INFO [2022-01-10 05:47:26,228] jepsen worker 34 - jepsen.util 4174 :ok :read [11 21] INFO [2022-01-10 05:47:26,228] jepsen worker 34 - jepsen.util 4174 :invoke :read [11 nil] INFO [2022-01-10 05:47:26,331] jepsen worker 49 - jepsen.util 4009 :ok :read [16 38] INFO [2022-01-10 05:47:26,332] jepsen worker 49 - jepsen.util 4009 :invoke :write [16 37] INFO [2022-01-10 05:47:26,511] jepsen worker 1 - jepsen.util 4231 :ok :read [0 46] INFO [2022-01-10 05:47:26,512] jepsen worker 1 - jepsen.util 4231 :invoke :write [0 13] INFO [2022-01-10 05:47:26,553] jepsen worker 84 - jepsen.util 4134 :ok :read [28 45] INFO [2022-01-10 05:47:26,553] jepsen worker 84 - jepsen.util 4134 :invoke :read [28 nil] INFO [2022-01-10 05:47:26,596] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:26,596] jepsen worker nemesis - couchbase.util Rebalance check count: 594 INFO [2022-01-10 05:47:26,596] jepsen worker nemesis - couchbase.util Rebalance stuck count: 504 INFO [2022-01-10 05:47:26,845] jepsen worker 73 - jepsen.util 3853 :info :write [24 23] :RequestTimeoutException INFO [2022-01-10 05:47:26,846] jepsen worker 73 - jepsen.util 3943 :invoke :write [24 22] INFO [2022-01-10 05:47:26,870] jepsen worker 71 - jepsen.util 3761 :info :write [23 39] :RequestTimeoutException INFO [2022-01-10 05:47:26,871] jepsen worker 71 - jepsen.util 3851 :invoke :read [23 nil] INFO [2022-01-10 05:47:26,913] jepsen worker 59 - jepsen.util 4289 :ok :read [19 31] INFO [2022-01-10 05:47:26,914] jepsen worker 59 - jepsen.util 4289 :invoke :read [19 nil] INFO [2022-01-10 05:47:27,301] jepsen worker 35 - jepsen.util 3905 :info :write [11 49] :RequestTimeoutException INFO [2022-01-10 05:47:27,301] jepsen worker 35 - jepsen.util 3995 :invoke :write [11 4] INFO [2022-01-10 05:47:27,346] jepsen worker 40 - jepsen.util 3730 :info :write [13 16] :RequestTimeoutException INFO [2022-01-10 05:47:27,347] jepsen worker 40 - jepsen.util 3820 :invoke :write [13 9] INFO [2022-01-10 05:47:27,534] jepsen worker 32 - jepsen.util 4352 :info :write [10 8] :SyncWriteAmbiguous INFO [2022-01-10 05:47:27,535] jepsen worker 32 - jepsen.util 4442 :invoke :read [10 nil] INFO [2022-01-10 05:47:27,594] jepsen worker 30 - jepsen.util 3990 :ok :read [10 36] INFO [2022-01-10 05:47:27,594] jepsen worker 30 - jepsen.util 3990 :invoke :read [10 nil] INFO [2022-01-10 05:47:27,602] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:27,602] jepsen worker nemesis - couchbase.util Rebalance check count: 595 INFO [2022-01-10 05:47:27,602] jepsen worker nemesis - couchbase.util Rebalance stuck count: 505 INFO [2022-01-10 05:47:27,616] jepsen worker 4 - jepsen.util 4684 :ok :read [1 26] INFO [2022-01-10 05:47:27,617] jepsen worker 4 - jepsen.util 4684 :invoke :read [1 nil] INFO [2022-01-10 05:47:27,626] jepsen worker 31 - jepsen.util 4441 :ok :read [10 36] INFO [2022-01-10 05:47:27,627] jepsen worker 31 - jepsen.util 4441 :invoke :write [10 16] INFO [2022-01-10 05:47:27,650] jepsen worker 75 - jepsen.util 3945 :info :write [25 33] :RequestTimeoutException INFO [2022-01-10 05:47:27,650] jepsen worker 75 - jepsen.util 4035 :invoke :read [25 nil] INFO [2022-01-10 05:47:27,699] jepsen worker 30 - jepsen.util 3990 :ok :read [10 36] INFO [2022-01-10 05:47:27,700] jepsen worker 30 - jepsen.util 3990 :invoke :read [10 nil] INFO [2022-01-10 05:47:27,708] jepsen worker 4 - jepsen.util 4684 :ok :read [1 26] INFO [2022-01-10 05:47:27,709] jepsen worker 4 - jepsen.util 4684 :invoke :read [1 nil] INFO [2022-01-10 05:47:27,806] jepsen worker 30 - jepsen.util 3990 :ok :read [10 36] INFO [2022-01-10 05:47:27,806] jepsen worker 30 - jepsen.util 3990 :invoke :write [10 11] INFO [2022-01-10 05:47:27,815] jepsen worker 4 - jepsen.util 4684 :ok :read [1 26] INFO [2022-01-10 05:47:27,816] jepsen worker 4 - jepsen.util 4684 :invoke :write [1 46] INFO [2022-01-10 05:47:27,967] jepsen worker 55 - jepsen.util 4735 :ok :read [18 14] INFO [2022-01-10 05:47:27,967] jepsen worker 55 - jepsen.util 4735 :invoke :write [18 40] INFO [2022-01-10 05:47:28,069] jepsen worker 64 - jepsen.util 4114 :info :write [21 23] :RequestTimeoutException INFO [2022-01-10 05:47:28,070] jepsen worker 64 - jepsen.util 4204 :invoke :write [21 30] INFO [2022-01-10 05:47:28,127] jepsen worker 79 - jepsen.util 4939 :info :write [26 31] :RequestTimeoutException INFO [2022-01-10 05:47:28,127] jepsen worker 79 - jepsen.util 5029 :invoke :write [26 36] INFO [2022-01-10 05:47:28,206] jepsen worker 66 - jepsen.util 3846 :info :write [22 30] :RequestTimeoutException INFO [2022-01-10 05:47:28,207] jepsen worker 66 - jepsen.util 3936 :invoke :write [22 24] INFO [2022-01-10 05:47:28,215] jepsen worker 85 - jepsen.util 4495 :info :write [28 5] :RequestTimeoutException INFO [2022-01-10 05:47:28,216] jepsen worker 85 - jepsen.util 4585 :invoke :read [28 nil] INFO [2022-01-10 05:47:28,296] jepsen worker 60 - jepsen.util 4380 :info :write [20 5] :RequestTimeoutException INFO [2022-01-10 05:47:28,296] jepsen worker 60 - jepsen.util 4470 :invoke :write [20 10] INFO [2022-01-10 05:47:28,305] jepsen worker 6 - jepsen.util 4146 :info :write [2 5] :RequestTimeoutException INFO [2022-01-10 05:47:28,306] jepsen worker 6 - jepsen.util 4236 :invoke :read [2 nil] INFO [2022-01-10 05:47:28,312] jepsen worker 18 - jepsen.util 4158 :info :write [6 4] :RequestTimeoutException INFO [2022-01-10 05:47:28,312] jepsen worker 18 - jepsen.util 4248 :invoke :read [6 nil] INFO [2022-01-10 05:47:28,325] jepsen worker 63 - jepsen.util 4023 :ok :read [21 7] INFO [2022-01-10 05:47:28,326] jepsen worker 63 - jepsen.util 4023 :invoke :read [21 nil] INFO [2022-01-10 05:47:28,456] jepsen worker 10 - jepsen.util 4060 :info :write [3 36] :RequestTimeoutException INFO [2022-01-10 05:47:28,457] jepsen worker 10 - jepsen.util 4150 :invoke :write [3 40] INFO [2022-01-10 05:47:28,466] jepsen worker 88 - jepsen.util 4138 :info :write [29 9] :RequestTimeoutException INFO [2022-01-10 05:47:28,467] jepsen worker 88 - jepsen.util 4228 :invoke :write [29 17] INFO [2022-01-10 05:47:28,511] jepsen worker 14 - jepsen.util 5144 :info :write [4 16] :RequestTimeoutException INFO [2022-01-10 05:47:28,511] jepsen worker 14 - jepsen.util 5234 :invoke :write [4 7] INFO [2022-01-10 05:47:28,605] jepsen worker 41 - jepsen.util 4181 :info :write [13 28] :RequestTimeoutException INFO [2022-01-10 05:47:28,605] jepsen worker 41 - jepsen.util 4271 :invoke :write [13 42] INFO [2022-01-10 05:47:28,608] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:28,608] jepsen worker nemesis - couchbase.util Rebalance check count: 596 INFO [2022-01-10 05:47:28,608] jepsen worker nemesis - couchbase.util Rebalance stuck count: 506 INFO [2022-01-10 05:47:28,771] jepsen worker 34 - jepsen.util 4174 :ok :read [11 21] INFO [2022-01-10 05:47:28,771] jepsen worker 34 - jepsen.util 4174 :invoke :read [11 nil] INFO [2022-01-10 05:47:28,792] jepsen worker 82 - jepsen.util 4312 :info :write [27 46] :RequestTimeoutException INFO [2022-01-10 05:47:28,792] jepsen worker 82 - jepsen.util 4402 :invoke :read [27 nil] INFO [2022-01-10 05:47:28,836] jepsen worker 43 - jepsen.util 4093 :info :write [14 10] :RequestTimeoutException INFO [2022-01-10 05:47:28,837] jepsen worker 43 - jepsen.util 4183 :invoke :write [14 23] INFO [2022-01-10 05:47:29,056] jepsen worker 42 - jepsen.util 3912 :info :write [14 46] :RequestTimeoutException INFO [2022-01-10 05:47:29,056] jepsen worker 22 - jepsen.util 3982 :info :write [7 19] :RequestTimeoutException INFO [2022-01-10 05:47:29,057] jepsen worker 42 - jepsen.util 4002 :invoke :read [14 nil] INFO [2022-01-10 05:47:29,057] jepsen worker 22 - jepsen.util 4072 :invoke :write [7 47] INFO [2022-01-10 05:47:29,197] jepsen worker 56 - jepsen.util 4556 :ok :read [18 14] INFO [2022-01-10 05:47:29,197] jepsen worker 56 - jepsen.util 4556 :invoke :write [18 33] INFO [2022-01-10 05:47:29,246] jepsen worker 46 - jepsen.util 4366 :info :write [15 24] :RequestTimeoutException INFO [2022-01-10 05:47:29,247] jepsen worker 46 - jepsen.util 4456 :invoke :write [15 20] INFO [2022-01-10 05:47:29,326] jepsen worker 85 - jepsen.util 4585 :ok :read [28 45] INFO [2022-01-10 05:47:29,326] jepsen worker 85 - jepsen.util 4585 :invoke :read [28 nil] INFO [2022-01-10 05:47:29,406] jepsen worker 42 - jepsen.util 4002 :ok :read [14 14] INFO [2022-01-10 05:47:29,407] jepsen worker 42 - jepsen.util 4002 :invoke :read [14 nil] INFO [2022-01-10 05:47:29,463] jepsen worker 89 - jepsen.util 4679 :info :write [29 26] :RequestTimeoutException INFO [2022-01-10 05:47:29,463] jepsen worker 89 - jepsen.util 4769 :invoke :read [29 nil] INFO [2022-01-10 05:47:29,501] jepsen worker 84 - jepsen.util 4134 :ok :read [28 45] INFO [2022-01-10 05:47:29,501] jepsen worker 84 - jepsen.util 4134 :invoke :read [28 nil] INFO [2022-01-10 05:47:29,526] jepsen worker 69 - jepsen.util 3759 :info :write [23 42] :RequestTimeoutException INFO [2022-01-10 05:47:29,526] jepsen worker 69 - jepsen.util 3849 :invoke :read [23 nil] INFO [2022-01-10 05:47:29,530] jepsen worker 65 - jepsen.util 3935 :info :write [21 45] :RequestTimeoutException INFO [2022-01-10 05:47:29,530] jepsen worker 65 - jepsen.util 4025 :invoke :read [21 nil] INFO [2022-01-10 05:47:29,557] jepsen worker 69 - jepsen.util 3849 :ok :read [23 26] INFO [2022-01-10 05:47:29,558] jepsen worker 69 - jepsen.util 3849 :invoke :read [23 nil] INFO [2022-01-10 05:47:29,582] jepsen worker 69 - jepsen.util 3849 :ok :read [23 26] INFO [2022-01-10 05:47:29,583] jepsen worker 69 - jepsen.util 3849 :invoke :read [23 nil] INFO [2022-01-10 05:47:29,611] jepsen worker 69 - jepsen.util 3849 :ok :read [23 26] INFO [2022-01-10 05:47:29,612] jepsen worker 69 - jepsen.util 3849 :invoke :write [23 0] INFO [2022-01-10 05:47:29,613] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:29,613] jepsen worker nemesis - couchbase.util Rebalance check count: 597 INFO [2022-01-10 05:47:29,613] jepsen worker nemesis - couchbase.util Rebalance stuck count: 507 INFO [2022-01-10 05:47:29,660] jepsen worker 29 - jepsen.util 4079 :info :write [9 35] :RequestTimeoutException INFO [2022-01-10 05:47:29,660] jepsen worker 29 - jepsen.util 4169 :invoke :read [9 nil] INFO [2022-01-10 05:47:29,722] jepsen worker 77 - jepsen.util 4667 :info :write [25 47] :RequestTimeoutException INFO [2022-01-10 05:47:29,723] jepsen worker 77 - jepsen.util 4757 :invoke :read [25 nil] INFO [2022-01-10 05:47:29,785] jepsen worker 42 - jepsen.util 4002 :ok :read [14 14] INFO [2022-01-10 05:47:29,785] jepsen worker 42 - jepsen.util 4002 :invoke :read [14 nil] INFO [2022-01-10 05:47:29,842] jepsen worker 21 - jepsen.util 4251 :info :write [7 13] :RequestTimeoutException INFO [2022-01-10 05:47:29,843] jepsen worker 21 - jepsen.util 4341 :invoke :read [7 nil] INFO [2022-01-10 05:47:29,851] jepsen worker 45 - jepsen.util 4545 :info :write [15 21] :RequestTimeoutException INFO [2022-01-10 05:47:29,852] jepsen worker 45 - jepsen.util 4635 :invoke :write [15 27] INFO [2022-01-10 05:47:29,907] jepsen worker 42 - jepsen.util 4002 :ok :read [14 14] INFO [2022-01-10 05:47:29,908] jepsen worker 42 - jepsen.util 4002 :invoke :read [14 nil] INFO [2022-01-10 05:47:29,942] jepsen worker 17 - jepsen.util 4067 :ok :read [5 25] INFO [2022-01-10 05:47:29,943] jepsen worker 17 - jepsen.util 4067 :invoke :write [5 33] INFO [2022-01-10 05:47:30,056] jepsen worker 23 - jepsen.util 4253 :info :write [7 23] :RequestTimeoutException INFO [2022-01-10 05:47:30,057] jepsen worker 23 - jepsen.util 4343 :invoke :write [7 45] INFO [2022-01-10 05:47:30,121] jepsen worker 9 - jepsen.util 4779 :info :write [3 42] :RequestTimeoutException INFO [2022-01-10 05:47:30,122] jepsen worker 9 - jepsen.util 4869 :invoke :write [3 20] INFO [2022-01-10 05:47:30,141] jepsen worker 72 - jepsen.util 4392 :info :write [24 21] :RequestTimeoutException INFO [2022-01-10 05:47:30,141] jepsen worker 72 - jepsen.util 4482 :invoke :read [24 nil] INFO [2022-01-10 05:47:30,292] jepsen worker 15 - jepsen.util 3705 :info :write [5 17] :RequestTimeoutException INFO [2022-01-10 05:47:30,293] jepsen worker 15 - jepsen.util 3795 :invoke :read [5 nil] INFO [2022-01-10 05:47:30,372] jepsen worker 76 - jepsen.util 3586 :info :write [25 5] :RequestTimeoutException INFO [2022-01-10 05:47:30,372] jepsen worker 76 - jepsen.util 3676 :invoke :read [25 nil] INFO [2022-01-10 05:47:30,381] jepsen worker 62 - jepsen.util 4562 :info :write [20 8] :RequestTimeoutException INFO [2022-01-10 05:47:30,381] jepsen worker 62 - jepsen.util 4652 :invoke :read [20 nil] INFO [2022-01-10 05:47:30,455] jepsen worker 48 - jepsen.util 4008 :info :write [16 31] :RequestTimeoutException INFO [2022-01-10 05:47:30,455] jepsen worker 48 - jepsen.util 4098 :invoke :read [16 nil] INFO [2022-01-10 05:47:30,542] jepsen worker 42 - jepsen.util 4002 :ok :read [14 14] INFO [2022-01-10 05:47:30,550] jepsen worker 42 - jepsen.util 4002 :invoke :read [14 nil] WARN [2022-01-10 05:47:30,592] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16707025,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16707025,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9027816,"encode_duration_us":17472,"operation_id":"0x80e7","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10004204}],"total_count":1}} WARN [2022-01-10 05:47:30,592] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1336389,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4951027,"total_server_duration_us":592347,"encode_duration_us":22471,"operation_id":"0x8291","timeout_ms":10000,"last_server_duration_us":389814,"total_duration_us":10008402},{"operation_name":"upsert","last_dispatch_duration_us":1894522,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7772307,"total_server_duration_us":1233361,"encode_duration_us":17015,"operation_id":"0x82b3","timeout_ms":10000,"last_server_duration_us":387864,"total_duration_us":10007849},{"operation_name":"upsert","last_dispatch_duration_us":956048,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4934869,"total_server_duration_us":1468450,"encode_duration_us":16257,"operation_id":"0x82b2","timeout_ms":10000,"last_server_duration_us":660403,"total_duration_us":10005941},{"operation_name":"upsert","last_dispatch_duration_us":1293659,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4844440,"total_server_duration_us":263277,"encode_duration_us":18250,"operation_id":"0x834c","timeout_ms":10000,"last_server_duration_us":90931,"total_duration_us":10005662},{"operation_name":"upsert","last_dispatch_duration_us":2642049,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11511251,"total_server_duration_us":4715071,"encode_duration_us":17413,"operation_id":"0x81d4","timeout_ms":10000,"last_server_duration_us":821780,"total_duration_us":10005115},{"operation_name":"upsert","last_dispatch_duration_us":5005038,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":5005038,"total_server_duration_us":1004160,"encode_duration_us":17678,"operation_id":"0x822e","timeout_ms":10000,"last_server_duration_us":1004160,"total_duration_us":10004858},{"operation_name":"upsert","last_dispatch_duration_us":3836365,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":11050934,"total_server_duration_us":737578,"encode_duration_us":16485,"operation_id":"0x8206","timeout_ms":10000,"last_server_duration_us":460639,"total_duration_us":10004297},{"operation_name":"upsert","last_dispatch_duration_us":2790586,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12001419,"total_server_duration_us":2097431,"encode_duration_us":18034,"operation_id":"0x822f","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10003847},{"operation_name":"upsert","last_dispatch_duration_us":4241894,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12076158,"total_server_duration_us":2188306,"encode_duration_us":20562,"operation_id":"0x822a","timeout_ms":10000,"last_server_duration_us":382871,"total_duration_us":10003348},{"operation_name":"upsert","last_dispatch_duration_us":4236294,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":12751642,"total_server_duration_us":1363716,"encode_duration_us":16503,"operation_id":"0x81f5","timeout_ms":10000,"last_server_duration_us":583246,"total_duration_us":10002541}],"total_count":20}} INFO [2022-01-10 05:47:30,611] jepsen worker 7 - jepsen.util 4417 :info :write [2 23] :RequestTimeoutException INFO [2022-01-10 05:47:30,612] jepsen worker 7 - jepsen.util 4507 :invoke :write [2 38] INFO [2022-01-10 05:47:30,620] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:30,620] jepsen worker nemesis - couchbase.util Rebalance check count: 598 INFO [2022-01-10 05:47:30,620] jepsen worker nemesis - couchbase.util Rebalance stuck count: 508 INFO [2022-01-10 05:47:30,628] jepsen worker 59 - jepsen.util 4289 :ok :read [19 31] INFO [2022-01-10 05:47:30,629] jepsen worker 59 - jepsen.util 4289 :invoke :read [19 nil] INFO [2022-01-10 05:47:30,696] jepsen worker 61 - jepsen.util 4381 :info :write [20 40] :RequestTimeoutException INFO [2022-01-10 05:47:30,696] jepsen worker 61 - jepsen.util 4471 :invoke :read [20 nil] INFO [2022-01-10 05:47:30,724] jepsen worker 42 - jepsen.util 4002 :ok :read [14 14] INFO [2022-01-10 05:47:30,728] jepsen worker 42 - jepsen.util 4002 :invoke :read [14 nil] INFO [2022-01-10 05:47:30,739] jepsen worker 84 - jepsen.util 4134 :ok :read [28 45] INFO [2022-01-10 05:47:30,740] jepsen worker 84 - jepsen.util 4134 :invoke :read [28 nil] INFO [2022-01-10 05:47:30,746] jepsen worker 20 - jepsen.util 3800 :info :write [6 18] :RequestTimeoutException INFO [2022-01-10 05:47:30,747] jepsen worker 20 - jepsen.util 3890 :invoke :write [6 41] INFO [2022-01-10 05:47:30,806] jepsen worker 61 - jepsen.util 4471 :ok :read [20 :nil] INFO [2022-01-10 05:47:30,807] jepsen worker 61 - jepsen.util 4471 :invoke :write [20 6] INFO [2022-01-10 05:47:30,821] jepsen worker 42 - jepsen.util 4002 :ok :read [14 14] INFO [2022-01-10 05:47:30,822] jepsen worker 42 - jepsen.util 4002 :invoke :read [14 nil] INFO [2022-01-10 05:47:30,836] jepsen worker 72 - jepsen.util 4482 :ok :read [24 18] INFO [2022-01-10 05:47:30,837] jepsen worker 72 - jepsen.util 4482 :invoke :read [24 nil] INFO [2022-01-10 05:47:30,851] jepsen worker 81 - jepsen.util 4671 :info :write [27 5] :RequestTimeoutException INFO [2022-01-10 05:47:30,852] jepsen worker 81 - jepsen.util 4761 :invoke :write [27 41] INFO [2022-01-10 05:47:30,987] jepsen worker 85 - jepsen.util 4585 :ok :read [28 45] INFO [2022-01-10 05:47:30,988] jepsen worker 85 - jepsen.util 4585 :invoke :write [28 10] INFO [2022-01-10 05:47:31,010] jepsen worker 32 - jepsen.util 4442 :ok :read [10 36] INFO [2022-01-10 05:47:31,011] jepsen worker 32 - jepsen.util 4442 :invoke :read [10 nil] INFO [2022-01-10 05:47:31,018] jepsen worker 89 - jepsen.util 4769 :ok :read [29 0] INFO [2022-01-10 05:47:31,019] jepsen worker 89 - jepsen.util 4769 :invoke :read [29 nil] INFO [2022-01-10 05:47:31,036] jepsen worker 32 - jepsen.util 4442 :ok :read [10 36] INFO [2022-01-10 05:47:31,037] jepsen worker 32 - jepsen.util 4442 :invoke :write [10 30] INFO [2022-01-10 05:47:31,040] jepsen worker 42 - jepsen.util 4002 :ok :read [14 14] INFO [2022-01-10 05:47:31,041] jepsen worker 42 - jepsen.util 4002 :invoke :write [14 31] INFO [2022-01-10 05:47:31,051] jepsen worker 89 - jepsen.util 4769 :ok :read [29 0] INFO [2022-01-10 05:47:31,051] jepsen worker 89 - jepsen.util 4769 :invoke :write [29 36] INFO [2022-01-10 05:47:31,056] jepsen worker 33 - jepsen.util 4443 :info :write [11 3] :RequestTimeoutException INFO [2022-01-10 05:47:31,059] jepsen worker 33 - jepsen.util 4533 :invoke :read [11 nil] INFO [2022-01-10 05:47:31,110] jepsen worker 68 - jepsen.util 4388 :info :write [22 41] :RequestTimeoutException INFO [2022-01-10 05:47:31,110] jepsen worker 68 - jepsen.util 4478 :invoke :write [22 29] INFO [2022-01-10 05:47:31,271] jepsen worker 72 - jepsen.util 4482 :ok :read [24 18] INFO [2022-01-10 05:47:31,271] jepsen worker 72 - jepsen.util 4482 :invoke :read [24 nil] INFO [2022-01-10 05:47:31,335] jepsen worker 67 - jepsen.util 4117 :info :write [22 37] :RequestTimeoutException INFO [2022-01-10 05:47:31,335] jepsen worker 67 - jepsen.util 4207 :invoke :read [22 nil] INFO [2022-01-10 05:47:31,625] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:31,626] jepsen worker nemesis - couchbase.util Rebalance check count: 599 INFO [2022-01-10 05:47:31,626] jepsen worker nemesis - couchbase.util Rebalance stuck count: 509 INFO [2022-01-10 05:47:31,651] jepsen worker 72 - jepsen.util 4482 :ok :read [24 18] INFO [2022-01-10 05:47:31,651] jepsen worker 72 - jepsen.util 4482 :invoke :read [24 nil] INFO [2022-01-10 05:47:31,707] jepsen worker 50 - jepsen.util 4460 :info :write [16 17] :RequestTimeoutException INFO [2022-01-10 05:47:31,708] jepsen worker 50 - jepsen.util 4550 :invoke :write [16 14] INFO [2022-01-10 05:47:31,877] jepsen worker 84 - jepsen.util 4134 :ok :read [28 45] INFO [2022-01-10 05:47:31,878] jepsen worker 84 - jepsen.util 4134 :invoke :read [28 nil] INFO [2022-01-10 05:47:31,965] jepsen worker 58 - jepsen.util 4018 :info :write [19 28] :RequestTimeoutException INFO [2022-01-10 05:47:31,965] jepsen worker 58 - jepsen.util 4108 :invoke :write [19 24] INFO [2022-01-10 05:47:31,966] jepsen worker 0 - jepsen.util 4140 :info :write [0 6] :RequestTimeoutException INFO [2022-01-10 05:47:31,967] jepsen worker 0 - jepsen.util 4230 :invoke :write [0 33] INFO [2022-01-10 05:47:31,979] jepsen worker 5 - jepsen.util 4145 :info :write [1 8] :RequestTimeoutException INFO [2022-01-10 05:47:31,980] jepsen worker 5 - jepsen.util 4235 :invoke :read [1 nil] INFO [2022-01-10 05:47:32,008] jepsen worker 5 - jepsen.util 4235 :ok :read [1 26] INFO [2022-01-10 05:47:32,009] jepsen worker 5 - jepsen.util 4235 :invoke :write [1 6] INFO [2022-01-10 05:47:32,038] jepsen worker 72 - jepsen.util 4482 :ok :read [24 18] INFO [2022-01-10 05:47:32,039] jepsen worker 72 - jepsen.util 4482 :invoke :write [24 1] INFO [2022-01-10 05:47:32,092] jepsen worker 52 - jepsen.util 5362 :info :write [17 44] :RequestTimeoutException INFO [2022-01-10 05:47:32,093] jepsen worker 52 - jepsen.util 5452 :invoke :read [17 nil] WARN [2022-01-10 05:47:32,097] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9046252,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9773034,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9482698,"encode_duration_us":16223,"operation_id":"0x82b4","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10003492}],"total_count":1}} WARN [2022-01-10 05:47:32,192] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17833055,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":17833055,"total_server_duration_us":34811,"operation_id":"0x819d","timeout_ms":30000,"last_server_duration_us":34811,"total_duration_us":17835249},{"operation_name":"get","last_dispatch_duration_us":14497296,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":14497296,"total_server_duration_us":43,"operation_id":"0x825d","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":14498854},{"operation_name":"get","last_dispatch_duration_us":14422327,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":14422327,"total_server_duration_us":307513,"operation_id":"0x82bd","timeout_ms":30000,"last_server_duration_us":307513,"total_duration_us":14425615},{"operation_name":"upsert","last_dispatch_duration_us":15606398,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":15606398,"total_server_duration_us":134869,"encode_duration_us":17223,"operation_id":"0x8289","timeout_ms":10000,"last_server_duration_us":134869,"total_duration_us":10011010},{"operation_name":"upsert","last_dispatch_duration_us":278561,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":278561,"total_server_duration_us":28036,"encode_duration_us":16964,"operation_id":"0x8311","timeout_ms":10000,"last_server_duration_us":28036,"total_duration_us":10010145},{"operation_name":"upsert","last_dispatch_duration_us":9449824,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9449824,"total_server_duration_us":136471,"encode_duration_us":16290,"operation_id":"0x83dc","timeout_ms":10000,"last_server_duration_us":136471,"total_duration_us":10009282},{"operation_name":"upsert","last_dispatch_duration_us":4288700,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5193762,"total_server_duration_us":768492,"encode_duration_us":23217,"operation_id":"0x83b6","timeout_ms":10000,"last_server_duration_us":201690,"total_duration_us":10007440},{"operation_name":"upsert","last_dispatch_duration_us":2914161,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3256129,"total_server_duration_us":395615,"encode_duration_us":19390,"operation_id":"0x83ae","timeout_ms":10000,"last_server_duration_us":362091,"total_duration_us":10007336},{"operation_name":"upsert","last_dispatch_duration_us":29864,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":6621675,"total_server_duration_us":3489523,"encode_duration_us":16493,"operation_id":"0x834a","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10007063},{"operation_name":"upsert","last_dispatch_duration_us":108428,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":7691588,"total_server_duration_us":58722,"encode_duration_us":19899,"operation_id":"0x83a9","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10005119}],"total_count":17}} WARN [2022-01-10 05:47:32,192] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":21593530,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25531012,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10915027,"encode_duration_us":20475,"operation_id":"0x8027","timeout_ms":10000,"last_server_duration_us":9556166,"total_duration_us":10003657},{"operation_name":"upsert","last_dispatch_duration_us":18091225,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18091225,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9026756,"encode_duration_us":16217,"operation_id":"0x81a8","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10003896}],"total_count":2}} WARN [2022-01-10 05:47:32,197] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":4186330,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4186330,"total_server_duration_us":373240,"encode_duration_us":16387,"operation_id":"0x83de","timeout_ms":10000,"last_server_duration_us":373240,"total_duration_us":10009217},{"operation_name":"upsert","last_dispatch_duration_us":3021039,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3569752,"total_server_duration_us":424352,"encode_duration_us":16230,"operation_id":"0x83a6","timeout_ms":10000,"last_server_duration_us":397934,"total_duration_us":10008107},{"operation_name":"upsert","last_dispatch_duration_us":2591298,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10651191,"total_server_duration_us":1625914,"encode_duration_us":20095,"operation_id":"0x8341","timeout_ms":10000,"last_server_duration_us":794035,"total_duration_us":10006567},{"operation_name":"upsert","last_dispatch_duration_us":5491279,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5491279,"total_server_duration_us":373514,"encode_duration_us":16355,"operation_id":"0x83c9","timeout_ms":10000,"last_server_duration_us":373514,"total_duration_us":10006028},{"operation_name":"upsert","last_dispatch_duration_us":9046252,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9773034,"total_server_duration_us":9482698,"encode_duration_us":16223,"operation_id":"0x82b4","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10003492},{"operation_name":"upsert","last_dispatch_duration_us":504833,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":504833,"total_server_duration_us":20441,"encode_duration_us":25273,"operation_id":"0x836d","timeout_ms":10000,"last_server_duration_us":20441,"total_duration_us":10002969},{"operation_name":"upsert","last_dispatch_duration_us":3107853,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8235484,"total_server_duration_us":1656830,"encode_duration_us":16220,"operation_id":"0x8305","timeout_ms":10000,"last_server_duration_us":404708,"total_duration_us":10002280},{"operation_name":"upsert","last_dispatch_duration_us":2768774,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8403911,"total_server_duration_us":1202223,"encode_duration_us":17413,"operation_id":"0x8344","timeout_ms":10000,"last_server_duration_us":581262,"total_duration_us":10001993},{"operation_name":"upsert","last_dispatch_duration_us":531678,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2110139,"total_server_duration_us":39304,"encode_duration_us":21276,"operation_id":"0x8342","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10001822},{"operation_name":"upsert","last_dispatch_duration_us":607883,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":607883,"total_server_duration_us":21163,"encode_duration_us":26402,"operation_id":"0x836e","timeout_ms":10000,"last_server_duration_us":21163,"total_duration_us":10001670}],"total_count":23}} WARN [2022-01-10 05:47:32,287] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":515017,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2228399,"total_server_duration_us":78218,"encode_duration_us":16900,"operation_id":"0x8343","timeout_ms":10000,"last_server_duration_us":37138,"total_duration_us":10010057},{"operation_name":"upsert","last_dispatch_duration_us":38588,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9946966,"total_server_duration_us":127792,"encode_duration_us":21256,"operation_id":"0x837b","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10008719},{"operation_name":"upsert","last_dispatch_duration_us":9316675,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":18572420,"total_server_duration_us":11509267,"encode_duration_us":16221,"operation_id":"0x8237","timeout_ms":10000,"last_server_duration_us":9145900,"total_duration_us":10008137},{"operation_name":"upsert","last_dispatch_duration_us":5577176,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":5577176,"total_server_duration_us":371326,"encode_duration_us":16247,"operation_id":"0x83bb","timeout_ms":10000,"last_server_duration_us":371326,"total_duration_us":10007744},{"operation_name":"upsert","last_dispatch_duration_us":7415062,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":14891859,"total_server_duration_us":2506526,"encode_duration_us":19004,"operation_id":"0x82d1","timeout_ms":10000,"last_server_duration_us":36627,"total_duration_us":10005866},{"operation_name":"upsert","last_dispatch_duration_us":4047631,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":4725617,"total_server_duration_us":263234,"encode_duration_us":18233,"operation_id":"0x8391","timeout_ms":10000,"last_server_duration_us":201059,"total_duration_us":10004385},{"operation_name":"upsert","last_dispatch_duration_us":3679526,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8154063,"total_server_duration_us":2123136,"encode_duration_us":17368,"operation_id":"0x83b2","timeout_ms":10000,"last_server_duration_us":494985,"total_duration_us":10003973},{"operation_name":"upsert","last_dispatch_duration_us":6744610,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7565728,"total_server_duration_us":667101,"encode_duration_us":19713,"operation_id":"0x83a4","timeout_ms":10000,"last_server_duration_us":623526,"total_duration_us":10002486},{"operation_name":"upsert","last_dispatch_duration_us":33991,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":9375890,"total_server_duration_us":1200264,"encode_duration_us":18958,"operation_id":"0x8412","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10001841},{"operation_name":"upsert","last_dispatch_duration_us":3949364,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11416426,"total_server_duration_us":2611538,"encode_duration_us":17133,"operation_id":"0x8365","timeout_ms":10000,"last_server_duration_us":213849,"total_duration_us":10001816}],"total_count":14}} WARN [2022-01-10 05:47:32,288] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2731337,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11770462,"total_server_duration_us":1136273,"encode_duration_us":29136,"operation_id":"0x8238","timeout_ms":10000,"last_server_duration_us":225645,"total_duration_us":10009331},{"operation_name":"upsert","last_dispatch_duration_us":4491005,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":4491005,"total_server_duration_us":211049,"encode_duration_us":16461,"operation_id":"0x83ea","timeout_ms":10000,"last_server_duration_us":211049,"total_duration_us":10009092},{"operation_name":"upsert","last_dispatch_duration_us":8744485,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9045059,"total_server_duration_us":133455,"encode_duration_us":18218,"operation_id":"0x8346","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10006579},{"operation_name":"upsert","last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","encode_duration_us":17016,"operation_id":"0x837e","timeout_ms":10000,"total_duration_us":10006117},{"operation_name":"upsert","last_dispatch_duration_us":51111,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":326921,"total_server_duration_us":105663,"encode_duration_us":17234,"operation_id":"0x8348","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10005908},{"operation_name":"upsert","last_dispatch_duration_us":12661726,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":18897229,"total_server_duration_us":9604325,"encode_duration_us":18658,"operation_id":"0x8236","timeout_ms":10000,"last_server_duration_us":9149101,"total_duration_us":10005696},{"operation_name":"upsert","last_dispatch_duration_us":3817983,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":3817983,"total_server_duration_us":1325954,"encode_duration_us":16619,"operation_id":"0x8339","timeout_ms":10000,"last_server_duration_us":1325954,"total_duration_us":10003972},{"operation_name":"upsert","last_dispatch_duration_us":4568667,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9773575,"total_server_duration_us":401154,"encode_duration_us":22881,"operation_id":"0x8340","timeout_ms":10000,"last_server_duration_us":35815,"total_duration_us":10003206},{"operation_name":"upsert","last_dispatch_duration_us":668798,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9404969,"total_server_duration_us":4306498,"encode_duration_us":21114,"operation_id":"0x83bc","timeout_ms":10000,"last_server_duration_us":41660,"total_duration_us":10002796},{"operation_name":"upsert","last_dispatch_duration_us":2909118,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":6411260,"total_server_duration_us":1488251,"encode_duration_us":16697,"operation_id":"0x8288","timeout_ms":10000,"last_server_duration_us":1343835,"total_duration_us":10002477}],"total_count":17}} INFO [2022-01-10 05:47:32,295] jepsen worker 12 - jepsen.util 4782 :info :write [4 30] :RequestTimeoutException INFO [2022-01-10 05:47:32,295] jepsen worker 12 - jepsen.util 4872 :invoke :write [4 39] INFO [2022-01-10 05:47:32,310] jepsen worker 51 - jepsen.util 5091 :info :write [17 39] :RequestTimeoutException INFO [2022-01-10 05:47:32,311] jepsen worker 51 - jepsen.util 5181 :invoke :read [17 nil] WARN [2022-01-10 05:47:32,388] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9316675,"last_local_id":"EFE9906200000004/000000008B42EFB5","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":18572420,"last_local_socket":"172.23.104.79:44292","total_server_duration_us":11509267,"encode_duration_us":16221,"operation_id":"0x8237","timeout_ms":10000,"last_server_duration_us":9145900,"total_duration_us":10008137},{"operation_name":"upsert","last_dispatch_duration_us":17375976,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18152414,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9506504,"encode_duration_us":16198,"operation_id":"0x8097","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10008161}],"total_count":2}} WARN [2022-01-10 05:47:32,392] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9034612,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9927210,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9375534,"encode_duration_us":16697,"operation_id":"0x8267","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10004832},{"operation_name":"upsert","last_dispatch_duration_us":12635644,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14438370,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9474847,"encode_duration_us":16457,"operation_id":"0x8316","timeout_ms":10000,"last_server_duration_us":9088372,"total_duration_us":10005754}],"total_count":2}} WARN [2022-01-10 05:47:32,392] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":702585,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9679522,"total_server_duration_us":2060713,"encode_duration_us":16177,"operation_id":"0x8306","timeout_ms":10000,"last_server_duration_us":23475,"total_duration_us":10010963},{"operation_name":"upsert","last_dispatch_duration_us":7930420,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8928822,"total_server_duration_us":436477,"encode_duration_us":16241,"operation_id":"0x8268","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10010475},{"operation_name":"upsert","last_dispatch_duration_us":8083064,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9004479,"total_server_duration_us":452294,"encode_duration_us":16926,"operation_id":"0x8266","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10009980},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","encode_duration_us":18692,"operation_id":"0x837f","timeout_ms":10000,"total_duration_us":10008175},{"operation_name":"upsert","last_dispatch_duration_us":4118996,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":4118996,"total_server_duration_us":357510,"encode_duration_us":19445,"operation_id":"0x83df","timeout_ms":10000,"last_server_duration_us":357510,"total_duration_us":10007808},{"operation_name":"upsert","last_dispatch_duration_us":4479321,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4479321,"total_server_duration_us":408682,"encode_duration_us":16270,"operation_id":"0x8338","timeout_ms":10000,"last_server_duration_us":408682,"total_duration_us":10007395},{"operation_name":"upsert","last_dispatch_duration_us":1089706,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10129930,"total_server_duration_us":3947416,"encode_duration_us":16479,"operation_id":"0x82de","timeout_ms":10000,"last_server_duration_us":169781,"total_duration_us":10005964},{"operation_name":"upsert","last_dispatch_duration_us":12635644,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":14438370,"total_server_duration_us":9474847,"encode_duration_us":16457,"operation_id":"0x8316","timeout_ms":10000,"last_server_duration_us":9088372,"total_duration_us":10005754},{"operation_name":"upsert","last_dispatch_duration_us":9034612,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9927210,"total_server_duration_us":9375534,"encode_duration_us":16697,"operation_id":"0x8267","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10004832},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","encode_duration_us":17619,"operation_id":"0x840e","timeout_ms":10000,"total_duration_us":10004546}],"total_count":14}} WARN [2022-01-10 05:47:32,488] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12661726,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":18897229,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":9604325,"encode_duration_us":18658,"operation_id":"0x8236","timeout_ms":10000,"last_server_duration_us":9149101,"total_duration_us":10005696}],"total_count":1}} INFO [2022-01-10 05:47:32,631] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:32,632] jepsen worker nemesis - couchbase.util Rebalance check count: 600 INFO [2022-01-10 05:47:32,632] jepsen worker nemesis - couchbase.util Rebalance stuck count: 510 INFO [2022-01-10 05:47:32,635] jepsen worker 84 - jepsen.util 4134 :ok :read [28 45] INFO [2022-01-10 05:47:32,635] jepsen worker 84 - jepsen.util 4134 :invoke :write [28 3] INFO [2022-01-10 05:47:32,662] jepsen worker 70 - jepsen.util 4210 :info :write [23 28] :RequestTimeoutException INFO [2022-01-10 05:47:32,662] jepsen worker 70 - jepsen.util 4300 :invoke :read [23 nil] INFO [2022-01-10 05:47:32,687] jepsen worker 70 - jepsen.util 4300 :ok :read [23 26] INFO [2022-01-10 05:47:32,687] jepsen worker 70 - jepsen.util 4300 :invoke :read [23 nil] INFO [2022-01-10 05:47:32,690] jepsen worker 28 - jepsen.util 3988 :info :write [9 6] :RequestTimeoutException INFO [2022-01-10 05:47:32,690] jepsen worker 28 - jepsen.util 4078 :invoke :read [9 nil] INFO [2022-01-10 05:47:32,714] jepsen worker 70 - jepsen.util 4300 :ok :read [23 26] INFO [2022-01-10 05:47:32,714] jepsen worker 70 - jepsen.util 4300 :invoke :read [23 nil] INFO [2022-01-10 05:47:32,745] jepsen worker 70 - jepsen.util 4300 :ok :read [23 26] INFO [2022-01-10 05:47:32,746] jepsen worker 70 - jepsen.util 4300 :invoke :write [23 6] INFO [2022-01-10 05:47:33,172] jepsen worker 83 - jepsen.util 4763 :info :write [27 16] :RequestTimeoutException INFO [2022-01-10 05:47:33,173] jepsen worker 83 - jepsen.util 4853 :invoke :write [27 41] INFO [2022-01-10 05:47:33,191] jepsen worker 87 - jepsen.util 4137 :info :write [29 41] :RequestTimeoutException INFO [2022-01-10 05:47:33,192] jepsen worker 87 - jepsen.util 4227 :invoke :read [29 nil] INFO [2022-01-10 05:47:33,294] jepsen worker 11 - jepsen.util 4151 :info :write [3 25] :RequestTimeoutException INFO [2022-01-10 05:47:33,295] jepsen worker 11 - jepsen.util 4241 :invoke :read [3 nil] INFO [2022-01-10 05:47:33,489] jepsen worker 74 - jepsen.util 4754 :ok :read [24 18] INFO [2022-01-10 05:47:33,489] jepsen worker 74 - jepsen.util 4754 :invoke :write [24 8] INFO [2022-01-10 05:47:33,505] jepsen worker 53 - jepsen.util 5093 :info :write [17 8] :RequestTimeoutException INFO [2022-01-10 05:47:33,506] jepsen worker 53 - jepsen.util 5183 :invoke :write [17 0] INFO [2022-01-10 05:47:33,637] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:33,637] jepsen worker nemesis - couchbase.util Rebalance check count: 601 INFO [2022-01-10 05:47:33,637] jepsen worker nemesis - couchbase.util Rebalance stuck count: 511 INFO [2022-01-10 05:47:33,681] jepsen worker 2 - jepsen.util 4412 :info :write [0 34] :RequestTimeoutException INFO [2022-01-10 05:47:33,682] jepsen worker 2 - jepsen.util 4502 :invoke :write [0 43] INFO [2022-01-10 05:47:33,692] jepsen worker 57 - jepsen.util 5007 :info :write [19 30] :RequestTimeoutException INFO [2022-01-10 05:47:33,692] jepsen worker 57 - jepsen.util 5097 :invoke :read [19 nil] INFO [2022-01-10 05:47:33,696] jepsen worker 44 - jepsen.util 4364 :ok :write [14 21] INFO [2022-01-10 05:47:33,696] jepsen worker 44 - jepsen.util 4364 :invoke :read [14 nil] INFO [2022-01-10 05:47:33,880] jepsen worker 26 - jepsen.util 3896 :info :write [8 26] :RequestTimeoutException INFO [2022-01-10 05:47:33,880] jepsen worker 26 - jepsen.util 3986 :invoke :write [8 30] INFO [2022-01-10 05:47:33,887] jepsen worker 75 - jepsen.util 4035 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:33,887] jepsen worker 6 - jepsen.util 4236 :fail :read [2 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:33,887] jepsen worker 75 - jepsen.util 4035 :invoke :write [25 19] INFO [2022-01-10 05:47:33,887] jepsen worker 6 - jepsen.util 4236 :invoke :read [2 nil] INFO [2022-01-10 05:47:34,079] jepsen worker 28 - jepsen.util 4078 :ok :read [9 33] INFO [2022-01-10 05:47:34,079] jepsen worker 28 - jepsen.util 4078 :invoke :read [9 nil] INFO [2022-01-10 05:47:34,270] jepsen worker 21 - jepsen.util 4341 :ok :read [7 :nil] INFO [2022-01-10 05:47:34,271] jepsen worker 21 - jepsen.util 4341 :invoke :read [7 nil] INFO [2022-01-10 05:47:34,322] jepsen worker 86 - jepsen.util 3596 :info :write [28 48] :RequestTimeoutException INFO [2022-01-10 05:47:34,323] jepsen worker 86 - jepsen.util 3686 :invoke :write [28 2] INFO [2022-01-10 05:47:34,347] jepsen worker 76 - jepsen.util 3676 :ok :read [25 :nil] INFO [2022-01-10 05:47:34,348] jepsen worker 76 - jepsen.util 3676 :invoke :write [25 31] INFO [2022-01-10 05:47:34,380] jepsen worker 6 - jepsen.util 4236 :ok :read [2 14] INFO [2022-01-10 05:47:34,380] jepsen worker 6 - jepsen.util 4236 :invoke :write [2 49] INFO [2022-01-10 05:47:34,382] jepsen worker 44 - jepsen.util 4364 :ok :read [14 21] INFO [2022-01-10 05:47:34,382] jepsen worker 44 - jepsen.util 4364 :invoke :write [14 9] INFO [2022-01-10 05:47:34,426] jepsen worker 78 - jepsen.util 5208 :info :write [26 6] :RequestTimeoutException INFO [2022-01-10 05:47:34,426] jepsen worker 78 - jepsen.util 5298 :invoke :read [26 nil] INFO [2022-01-10 05:47:34,460] jepsen worker 78 - jepsen.util 5298 :ok :read [26 10] INFO [2022-01-10 05:47:34,461] jepsen worker 78 - jepsen.util 5298 :invoke :read [26 nil] INFO [2022-01-10 05:47:34,483] jepsen worker 78 - jepsen.util 5298 :ok :read [26 10] INFO [2022-01-10 05:47:34,483] jepsen worker 78 - jepsen.util 5298 :invoke :read [26 nil] INFO [2022-01-10 05:47:34,508] jepsen worker 78 - jepsen.util 5298 :ok :read [26 10] INFO [2022-01-10 05:47:34,509] jepsen worker 78 - jepsen.util 5298 :invoke :read [26 nil] INFO [2022-01-10 05:47:34,532] jepsen worker 78 - jepsen.util 5298 :ok :read [26 10] INFO [2022-01-10 05:47:34,532] jepsen worker 78 - jepsen.util 5298 :invoke :write [26 48] INFO [2022-01-10 05:47:34,643] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:34,643] jepsen worker nemesis - couchbase.util Rebalance check count: 602 INFO [2022-01-10 05:47:34,643] jepsen worker nemesis - couchbase.util Rebalance stuck count: 512 INFO [2022-01-10 05:47:34,757] jepsen worker 57 - jepsen.util 5097 :ok :read [19 31] INFO [2022-01-10 05:47:34,757] jepsen worker 57 - jepsen.util 5097 :invoke :write [19 28] INFO [2022-01-10 05:47:34,762] jepsen worker 21 - jepsen.util 4341 :ok :read [7 :nil] INFO [2022-01-10 05:47:34,763] jepsen worker 21 - jepsen.util 4341 :invoke :read [7 nil] INFO [2022-01-10 05:47:34,926] jepsen worker 16 - jepsen.util 4336 :info :write [5 6] :RequestTimeoutException INFO [2022-01-10 05:47:34,927] jepsen worker 16 - jepsen.util 4426 :invoke :read [5 nil] INFO [2022-01-10 05:47:35,001] jepsen worker 47 - jepsen.util 4367 :info :write [15 34] :RequestTimeoutException INFO [2022-01-10 05:47:35,001] jepsen worker 47 - jepsen.util 4457 :invoke :write [15 15] INFO [2022-01-10 05:47:35,025] jepsen worker 54 - jepsen.util 4374 :info :write [18 2] :RequestTimeoutException INFO [2022-01-10 05:47:35,025] jepsen worker 54 - jepsen.util 4464 :invoke :write [18 33] INFO [2022-01-10 05:47:35,058] jepsen worker 28 - jepsen.util 4078 :ok :read [9 33] INFO [2022-01-10 05:47:35,059] jepsen worker 28 - jepsen.util 4078 :invoke :read [9 nil] INFO [2022-01-10 05:47:35,433] jepsen worker 21 - jepsen.util 4341 :ok :read [7 :nil] INFO [2022-01-10 05:47:35,433] jepsen worker 21 - jepsen.util 4341 :invoke :write [7 23] INFO [2022-01-10 05:47:35,587] jepsen worker 19 - jepsen.util 4429 :ok :read [6 25] INFO [2022-01-10 05:47:35,587] jepsen worker 19 - jepsen.util 4429 :invoke :read [6 nil] INFO [2022-01-10 05:47:35,652] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:35,660] jepsen worker nemesis - couchbase.util Rebalance check count: 603 INFO [2022-01-10 05:47:35,660] jepsen worker nemesis - couchbase.util Rebalance stuck count: 513 INFO [2022-01-10 05:47:35,764] jepsen worker 80 - jepsen.util 5210 :info :write [26 5] :SyncWriteAmbiguous INFO [2022-01-10 05:47:35,764] jepsen worker 80 - jepsen.util 5300 :invoke :read [26 nil] INFO [2022-01-10 05:47:35,838] jepsen worker 36 - jepsen.util 3096 :ok :read [12 14] INFO [2022-01-10 05:47:35,838] jepsen worker 36 - jepsen.util 3096 :invoke :read [12 nil] INFO [2022-01-10 05:47:35,841] jepsen worker 38 - jepsen.util 3908 :ok :read [12 14] INFO [2022-01-10 05:47:35,842] jepsen worker 38 - jepsen.util 3908 :invoke :write [12 3] INFO [2022-01-10 05:47:35,847] jepsen worker 37 - jepsen.util 3817 :ok :read [12 14] INFO [2022-01-10 05:47:35,847] jepsen worker 37 - jepsen.util 3817 :invoke :write [12 24] INFO [2022-01-10 05:47:35,854] jepsen worker 80 - jepsen.util 5300 :ok :read [26 10] INFO [2022-01-10 05:47:35,854] jepsen worker 80 - jepsen.util 5300 :invoke :write [26 28] INFO [2022-01-10 05:47:35,869] jepsen worker 36 - jepsen.util 3096 :ok :read [12 14] INFO [2022-01-10 05:47:35,870] jepsen worker 36 - jepsen.util 3096 :invoke :read [12 nil] INFO [2022-01-10 05:47:35,969] jepsen worker 13 - jepsen.util 4963 :info :write [4 13] :RequestTimeoutException INFO [2022-01-10 05:47:35,970] jepsen worker 13 - jepsen.util 5053 :invoke :read [4 nil] INFO [2022-01-10 05:47:36,033] jepsen worker 36 - jepsen.util 3096 :ok :read [12 14] INFO [2022-01-10 05:47:36,034] jepsen worker 36 - jepsen.util 3096 :invoke :write [12 11] INFO [2022-01-10 05:47:36,135] jepsen worker 27 - jepsen.util 4527 :ok :read [9 6] INFO [2022-01-10 05:47:36,136] jepsen worker 27 - jepsen.util 4527 :invoke :write [9 22] INFO [2022-01-10 05:47:36,255] jepsen worker 39 - jepsen.util 4629 :info :write [13 39] :RequestTimeoutException INFO [2022-01-10 05:47:36,255] jepsen worker 39 - jepsen.util 4719 :invoke :read [13 nil] INFO [2022-01-10 05:47:36,357] jepsen worker 49 - jepsen.util 4009 :info :write [16 37] :RequestTimeoutException INFO [2022-01-10 05:47:36,357] jepsen worker 49 - jepsen.util 4099 :invoke :read [16 nil] INFO [2022-01-10 05:47:36,399] jepsen worker 28 - jepsen.util 4078 :ok :read [9 6] INFO [2022-01-10 05:47:36,400] jepsen worker 28 - jepsen.util 4078 :invoke :read [9 nil] INFO [2022-01-10 05:47:36,536] jepsen worker 1 - jepsen.util 4231 :info :write [0 13] :RequestTimeoutException INFO [2022-01-10 05:47:36,537] jepsen worker 1 - jepsen.util 4321 :invoke :write [0 29] INFO [2022-01-10 05:47:36,641] jepsen worker 82 - jepsen.util 4402 :ok :read [27 30] INFO [2022-01-10 05:47:36,642] jepsen worker 82 - jepsen.util 4402 :invoke :write [27 38] INFO [2022-01-10 05:47:36,650] jepsen worker 3 - jepsen.util 4143 :ok :read [1 26] INFO [2022-01-10 05:47:36,650] jepsen worker 3 - jepsen.util 4143 :invoke :read [1 nil] INFO [2022-01-10 05:47:36,665] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:36,665] jepsen worker nemesis - couchbase.util Rebalance check count: 604 INFO [2022-01-10 05:47:36,665] jepsen worker nemesis - couchbase.util Rebalance stuck count: 514 INFO [2022-01-10 05:47:36,725] jepsen worker 3 - jepsen.util 4143 :ok :read [1 26] INFO [2022-01-10 05:47:36,726] jepsen worker 3 - jepsen.util 4143 :invoke :write [1 49] INFO [2022-01-10 05:47:36,758] jepsen worker 31 - jepsen.util 4441 :info :write [10 16] :SyncWriteAmbiguous INFO [2022-01-10 05:47:36,759] jepsen worker 31 - jepsen.util 4531 :invoke :write [10 7] INFO [2022-01-10 05:47:36,872] jepsen worker 73 - jepsen.util 3943 :info :write [24 22] :RequestTimeoutException INFO [2022-01-10 05:47:36,873] jepsen worker 73 - jepsen.util 4033 :invoke :read [24 nil] INFO [2022-01-10 05:47:36,996] jepsen worker 48 - jepsen.util 4098 :ok :read [16 38] INFO [2022-01-10 05:47:36,997] jepsen worker 48 - jepsen.util 4098 :invoke :write [16 21] INFO [2022-01-10 05:47:37,044] jepsen worker 8 - jepsen.util 4418 :ok :read [2 14] INFO [2022-01-10 05:47:37,045] jepsen worker 8 - jepsen.util 4418 :invoke :read [2 nil] INFO [2022-01-10 05:47:37,050] jepsen worker 34 - jepsen.util 4174 :ok :read [11 21] INFO [2022-01-10 05:47:37,050] jepsen worker 34 - jepsen.util 4174 :invoke :read [11 nil] INFO [2022-01-10 05:47:37,069] jepsen worker 87 - jepsen.util 4227 :ok :read [29 0] INFO [2022-01-10 05:47:37,070] jepsen worker 87 - jepsen.util 4227 :invoke :write [29 14] INFO [2022-01-10 05:47:37,321] jepsen worker 35 - jepsen.util 3995 :info :write [11 4] :RequestTimeoutException INFO [2022-01-10 05:47:37,323] jepsen worker 24 - jepsen.util 3894 :ok :read [8 39] INFO [2022-01-10 05:47:37,324] jepsen worker 24 - jepsen.util 3894 :invoke :write [8 32] INFO [2022-01-10 05:47:37,325] jepsen worker 35 - jepsen.util 4085 :invoke :read [11 nil] INFO [2022-01-10 05:47:37,374] jepsen worker 40 - jepsen.util 3820 :info :write [13 9] :RequestTimeoutException INFO [2022-01-10 05:47:37,375] jepsen worker 40 - jepsen.util 3910 :invoke :write [13 4] INFO [2022-01-10 05:47:37,513] jepsen worker 62 - jepsen.util 4652 :ok :read [20 :nil] INFO [2022-01-10 05:47:37,513] jepsen worker 62 - jepsen.util 4652 :invoke :write [20 14] INFO [2022-01-10 05:47:37,630] jepsen worker 63 - jepsen.util 4023 :ok :read [21 7] INFO [2022-01-10 05:47:37,630] jepsen worker 63 - jepsen.util 4023 :invoke :read [21 nil] INFO [2022-01-10 05:47:37,668] jepsen worker 19 - jepsen.util 4429 :ok :read [6 25] INFO [2022-01-10 05:47:37,668] jepsen worker 19 - jepsen.util 4429 :invoke :write [6 28] INFO [2022-01-10 05:47:37,671] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:37,671] jepsen worker nemesis - couchbase.util Rebalance check count: 605 INFO [2022-01-10 05:47:37,671] jepsen worker nemesis - couchbase.util Rebalance stuck count: 515 INFO [2022-01-10 05:47:37,831] jepsen worker 30 - jepsen.util 3990 :info :write [10 11] :RequestTimeoutException INFO [2022-01-10 05:47:37,831] jepsen worker 30 - jepsen.util 4080 :invoke :read [10 nil] INFO [2022-01-10 05:47:37,841] jepsen worker 4 - jepsen.util 4684 :info :write [1 46] :RequestTimeoutException INFO [2022-01-10 05:47:37,842] jepsen worker 4 - jepsen.util 4774 :invoke :read [1 nil] INFO [2022-01-10 05:47:37,919] jepsen worker 28 - jepsen.util 4078 :ok :read [9 6] INFO [2022-01-10 05:47:37,919] jepsen worker 28 - jepsen.util 4078 :invoke :write [9 44] INFO [2022-01-10 05:47:37,989] jepsen worker 55 - jepsen.util 4735 :info :write [18 40] :RequestTimeoutException INFO [2022-01-10 05:47:37,990] jepsen worker 55 - jepsen.util 4825 :invoke :write [18 3] INFO [2022-01-10 05:47:38,091] jepsen worker 64 - jepsen.util 4204 :info :write [21 30] :RequestTimeoutException INFO [2022-01-10 05:47:38,092] jepsen worker 64 - jepsen.util 4294 :invoke :write [21 38] INFO [2022-01-10 05:47:38,157] jepsen worker 79 - jepsen.util 5029 :info :write [26 36] :RequestTimeoutException INFO [2022-01-10 05:47:38,157] jepsen worker 79 - jepsen.util 5119 :invoke :write [26 5] INFO [2022-01-10 05:47:38,190] jepsen worker 33 - jepsen.util 4533 :ok :read [11 21] INFO [2022-01-10 05:47:38,191] jepsen worker 33 - jepsen.util 4533 :invoke :write [11 22] INFO [2022-01-10 05:47:38,231] jepsen worker 66 - jepsen.util 3936 :info :write [22 24] :RequestTimeoutException INFO [2022-01-10 05:47:38,232] jepsen worker 66 - jepsen.util 4026 :invoke :write [22 48] INFO [2022-01-10 05:47:38,236] jepsen worker 8 - jepsen.util 4418 :ok :read [2 14] INFO [2022-01-10 05:47:38,238] jepsen worker 8 - jepsen.util 4418 :invoke :write [2 48] INFO [2022-01-10 05:47:38,244] jepsen worker 77 - jepsen.util 4757 :ok :read [25 :nil] INFO [2022-01-10 05:47:38,247] jepsen worker 77 - jepsen.util 4757 :invoke :write [25 22] INFO [2022-01-10 05:47:38,260] jepsen worker 25 - jepsen.util 3985 :ok :read [8 39] INFO [2022-01-10 05:47:38,260] jepsen worker 25 - jepsen.util 3985 :invoke :read [8 nil] INFO [2022-01-10 05:47:38,327] jepsen worker 60 - jepsen.util 4470 :info :write [20 10] :RequestTimeoutException INFO [2022-01-10 05:47:38,327] jepsen worker 60 - jepsen.util 4560 :invoke :write [20 33] INFO [2022-01-10 05:47:38,339] jepsen worker 73 - jepsen.util 4033 :ok :read [24 18] INFO [2022-01-10 05:47:38,340] jepsen worker 73 - jepsen.util 4033 :invoke :write [24 46] INFO [2022-01-10 05:47:38,357] jepsen worker 34 - jepsen.util 4174 :ok :read [11 21] INFO [2022-01-10 05:47:38,357] jepsen worker 34 - jepsen.util 4174 :invoke :write [11 14] INFO [2022-01-10 05:47:38,476] jepsen worker 10 - jepsen.util 4150 :info :write [3 40] :RequestTimeoutException INFO [2022-01-10 05:47:38,477] jepsen worker 10 - jepsen.util 4240 :invoke :read [3 nil] INFO [2022-01-10 05:47:38,491] jepsen worker 88 - jepsen.util 4228 :info :write [29 17] :RequestTimeoutException INFO [2022-01-10 05:47:38,492] jepsen worker 88 - jepsen.util 4318 :invoke :read [29 nil] INFO [2022-01-10 05:47:38,515] jepsen worker 35 - jepsen.util 4085 :ok :read [11 21] INFO [2022-01-10 05:47:38,516] jepsen worker 35 - jepsen.util 4085 :invoke :read [11 nil] INFO [2022-01-10 05:47:38,534] jepsen worker 14 - jepsen.util 5234 :info :write [4 7] :RequestTimeoutException INFO [2022-01-10 05:47:38,535] jepsen worker 14 - jepsen.util 5324 :invoke :read [4 nil] INFO [2022-01-10 05:47:38,599] jepsen worker 88 - jepsen.util 4318 :ok :read [29 0] INFO [2022-01-10 05:47:38,599] jepsen worker 88 - jepsen.util 4318 :invoke :read [29 nil] INFO [2022-01-10 05:47:38,626] jepsen worker 41 - jepsen.util 4271 :info :write [13 42] :RequestTimeoutException INFO [2022-01-10 05:47:38,627] jepsen worker 41 - jepsen.util 4361 :invoke :write [13 41] INFO [2022-01-10 05:47:38,657] jepsen worker 63 - jepsen.util 4023 :ok :read [21 7] INFO [2022-01-10 05:47:38,658] jepsen worker 63 - jepsen.util 4023 :invoke :write [21 9] INFO [2022-01-10 05:47:38,690] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:38,690] jepsen worker nemesis - couchbase.util Rebalance check count: 606 INFO [2022-01-10 05:47:38,690] jepsen worker nemesis - couchbase.util Rebalance stuck count: 516 INFO [2022-01-10 05:47:38,782] jepsen worker 88 - jepsen.util 4318 :ok :read [29 0] INFO [2022-01-10 05:47:38,782] jepsen worker 88 - jepsen.util 4318 :invoke :read [29 nil] INFO [2022-01-10 05:47:38,798] jepsen worker 35 - jepsen.util 4085 :ok :read [11 21] INFO [2022-01-10 05:47:38,799] jepsen worker 35 - jepsen.util 4085 :invoke :read [11 nil] INFO [2022-01-10 05:47:38,862] jepsen worker 43 - jepsen.util 4183 :info :write [14 23] :RequestTimeoutException INFO [2022-01-10 05:47:38,863] jepsen worker 43 - jepsen.util 4273 :invoke :write [14 35] INFO [2022-01-10 05:47:38,919] jepsen worker 88 - jepsen.util 4318 :ok :read [29 0] INFO [2022-01-10 05:47:38,920] jepsen worker 88 - jepsen.util 4318 :invoke :write [29 12] INFO [2022-01-10 05:47:39,081] jepsen worker 22 - jepsen.util 4072 :info :write [7 47] :RequestTimeoutException INFO [2022-01-10 05:47:39,081] jepsen worker 22 - jepsen.util 4162 :invoke :write [7 49] INFO [2022-01-10 05:47:39,222] jepsen worker 56 - jepsen.util 4556 :info :write [18 33] :RequestTimeoutException INFO [2022-01-10 05:47:39,222] jepsen worker 56 - jepsen.util 4646 :invoke :read [18 nil] INFO [2022-01-10 05:47:39,275] jepsen worker 46 - jepsen.util 4456 :info :write [15 20] :RequestTimeoutException INFO [2022-01-10 05:47:39,276] jepsen worker 46 - jepsen.util 4546 :invoke :read [15 nil] INFO [2022-01-10 05:47:39,356] jepsen worker 25 - jepsen.util 3985 :ok :read [8 39] INFO [2022-01-10 05:47:39,357] jepsen worker 25 - jepsen.util 3985 :invoke :read [8 nil] INFO [2022-01-10 05:47:39,628] jepsen worker 25 - jepsen.util 3985 :ok :read [8 39] INFO [2022-01-10 05:47:39,628] jepsen worker 25 - jepsen.util 3985 :invoke :read [8 nil] INFO [2022-01-10 05:47:39,629] jepsen worker 69 - jepsen.util 3849 :info :write [23 0] :RequestTimeoutException INFO [2022-01-10 05:47:39,630] jepsen worker 69 - jepsen.util 3939 :invoke :write [23 40] INFO [2022-01-10 05:47:39,695] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5029239766081872}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:39,695] jepsen worker nemesis - couchbase.util Rebalance check count: 607 INFO [2022-01-10 05:47:39,695] jepsen worker nemesis - couchbase.util Rebalance stuck count: 517 INFO [2022-01-10 05:47:39,753] jepsen worker 25 - jepsen.util 3985 :ok :read [8 39] INFO [2022-01-10 05:47:39,753] jepsen worker 25 - jepsen.util 3985 :invoke :read [8 nil] INFO [2022-01-10 05:47:39,876] jepsen worker 45 - jepsen.util 4635 :info :write [15 27] :RequestTimeoutException INFO [2022-01-10 05:47:39,877] jepsen worker 45 - jepsen.util 4725 :invoke :write [15 2] INFO [2022-01-10 05:47:39,881] jepsen worker 25 - jepsen.util 3985 :ok :read [8 39] INFO [2022-01-10 05:47:39,882] jepsen worker 25 - jepsen.util 3985 :invoke :write [8 20] INFO [2022-01-10 05:47:39,971] jepsen worker 17 - jepsen.util 4067 :info :write [5 33] :RequestTimeoutException INFO [2022-01-10 05:47:39,972] jepsen worker 17 - jepsen.util 4157 :invoke :read [5 nil] INFO [2022-01-10 05:47:40,079] jepsen worker 23 - jepsen.util 4343 :info :write [7 45] :RequestTimeoutException INFO [2022-01-10 05:47:40,080] jepsen worker 23 - jepsen.util 4433 :invoke :read [7 nil] INFO [2022-01-10 05:47:40,142] jepsen worker 9 - jepsen.util 4869 :info :write [3 20] :RequestTimeoutException INFO [2022-01-10 05:47:40,143] jepsen worker 9 - jepsen.util 4959 :invoke :read [3 nil] INFO [2022-01-10 05:47:40,329] jepsen worker 52 - jepsen.util 5452 :ok :read [17 41] INFO [2022-01-10 05:47:40,329] jepsen worker 52 - jepsen.util 5452 :invoke :read [17 nil] INFO [2022-01-10 05:47:40,351] jepsen worker 71 - jepsen.util 3851 :ok :read [23 26] INFO [2022-01-10 05:47:40,352] jepsen worker 71 - jepsen.util 3851 :invoke :write [23 11] INFO [2022-01-10 05:47:40,403] jepsen worker 15 - jepsen.util 3795 :ok :read [5 25] INFO [2022-01-10 05:47:40,405] jepsen worker 3 - jepsen.util 4143 :info :write [1 49] :SyncWriteAmbiguous INFO [2022-01-10 05:47:40,406] jepsen worker 3 - jepsen.util 4233 :invoke :write [1 38] INFO [2022-01-10 05:47:40,407] jepsen worker 15 - jepsen.util 3795 :invoke :write [5 31] INFO [2022-01-10 05:47:40,422] jepsen worker 13 - jepsen.util 5053 :ok :read [4 3] INFO [2022-01-10 05:47:40,422] jepsen worker 13 - jepsen.util 5053 :invoke :read [4 nil] INFO [2022-01-10 05:47:40,440] jepsen worker 30 - jepsen.util 4080 :ok :read [10 36] INFO [2022-01-10 05:47:40,441] jepsen worker 30 - jepsen.util 4080 :invoke :read [10 nil] INFO [2022-01-10 05:47:40,461] jepsen worker 51 - jepsen.util 5181 :ok :read [17 41] INFO [2022-01-10 05:47:40,462] jepsen worker 51 - jepsen.util 5181 :invoke :write [17 44] INFO [2022-01-10 05:47:40,491] jepsen worker 11 - jepsen.util 4241 :ok :read [3 12] INFO [2022-01-10 05:47:40,493] jepsen worker 11 - jepsen.util 4241 :invoke :read [3 nil] INFO [2022-01-10 05:47:40,501] jepsen worker 59 - jepsen.util 4289 :ok :read [19 31] INFO [2022-01-10 05:47:40,502] jepsen worker 59 - jepsen.util 4289 :invoke :read [19 nil] INFO [2022-01-10 05:47:40,524] jepsen worker 52 - jepsen.util 5452 :ok :read [17 41] INFO [2022-01-10 05:47:40,525] jepsen worker 52 - jepsen.util 5452 :invoke :read [17 nil] INFO [2022-01-10 05:47:40,532] jepsen worker 13 - jepsen.util 5053 :ok :read [4 3] INFO [2022-01-10 05:47:40,533] jepsen worker 13 - jepsen.util 5053 :invoke :write [4 6] INFO [2022-01-10 05:47:40,564] jepsen worker 52 - jepsen.util 5452 :ok :read [17 41] INFO [2022-01-10 05:47:40,564] jepsen worker 52 - jepsen.util 5452 :invoke :read [17 nil] WARN [2022-01-10 05:47:40,607] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9058141,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9681538,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9419274,"encode_duration_us":16455,"operation_id":"0x8449","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":10006896},{"operation_name":"upsert","last_dispatch_duration_us":9125490,"last_local_id":"EFE9906200000001/000000007CF9F49A","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":13137566,"last_local_socket":"172.23.104.79:44306","total_server_duration_us":11008531,"encode_duration_us":16855,"operation_id":"0x8419","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10008071},{"operation_name":"upsert","last_dispatch_duration_us":10724631,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18496939,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10444432,"encode_duration_us":17015,"operation_id":"0x82b3","timeout_ms":10000,"last_server_duration_us":9211071,"total_duration_us":10007849}],"total_count":3}} WARN [2022-01-10 05:47:40,607] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1684152,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9982390,"total_server_duration_us":1759401,"encode_duration_us":16557,"operation_id":"0x84f7","timeout_ms":10000,"last_server_duration_us":250503,"total_duration_us":10010317},{"operation_name":"upsert","last_dispatch_duration_us":760758,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7842378,"total_server_duration_us":1561891,"encode_duration_us":18628,"operation_id":"0x855c","timeout_ms":10000,"last_server_duration_us":292549,"total_duration_us":10008536},{"operation_name":"upsert","last_dispatch_duration_us":9125490,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":13137566,"total_server_duration_us":11008531,"encode_duration_us":16855,"operation_id":"0x8419","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10008071},{"operation_name":"upsert","last_dispatch_duration_us":1547980,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10412969,"total_server_duration_us":1678487,"encode_duration_us":19503,"operation_id":"0x84cb","timeout_ms":10000,"last_server_duration_us":174111,"total_duration_us":10007103},{"operation_name":"upsert","last_dispatch_duration_us":9058141,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9681538,"total_server_duration_us":9419274,"encode_duration_us":16455,"operation_id":"0x8449","timeout_ms":10000,"last_server_duration_us":9041612,"total_duration_us":10006896},{"operation_name":"upsert","last_dispatch_duration_us":30194,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":7696006,"total_server_duration_us":283216,"encode_duration_us":17040,"operation_id":"0x8530","timeout_ms":10000,"last_server_duration_us":18723,"total_duration_us":10005382},{"operation_name":"upsert","last_dispatch_duration_us":8469954,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":11959879,"total_server_duration_us":1687151,"encode_duration_us":20797,"operation_id":"0x844a","timeout_ms":10000,"last_server_duration_us":24755,"total_duration_us":10004183},{"operation_name":"upsert","last_dispatch_duration_us":1454591,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":5588837,"total_server_duration_us":921733,"encode_duration_us":17710,"operation_id":"0x840c","timeout_ms":10000,"last_server_duration_us":428799,"total_duration_us":10003637},{"operation_name":"upsert","last_dispatch_duration_us":1193205,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10256303,"total_server_duration_us":2642477,"encode_duration_us":17704,"operation_id":"0x83e7","timeout_ms":10000,"last_server_duration_us":200428,"total_duration_us":10002423},{"operation_name":"upsert","last_dispatch_duration_us":8964554,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9490702,"total_server_duration_us":350813,"encode_duration_us":16299,"operation_id":"0x849b","timeout_ms":10000,"last_server_duration_us":22389,"total_duration_us":10001171}],"total_count":21}} INFO [2022-01-10 05:47:40,632] jepsen worker 7 - jepsen.util 4507 :info :write [2 38] :RequestTimeoutException INFO [2022-01-10 05:47:40,633] jepsen worker 7 - jepsen.util 4597 :invoke :write [2 33] INFO [2022-01-10 05:47:40,643] jepsen worker 52 - jepsen.util 5452 :ok :read [17 41] INFO [2022-01-10 05:47:40,644] jepsen worker 52 - jepsen.util 5452 :invoke :write [17 28] INFO [2022-01-10 05:47:40,700] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5087719298245614}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:40,701] jepsen worker nemesis - couchbase.util Rebalance check count: 608 INFO [2022-01-10 05:47:40,701] jepsen worker nemesis - couchbase.util Rebalance stuck count: 517 INFO [2022-01-10 05:47:40,772] jepsen worker 20 - jepsen.util 3890 :info :write [6 41] :RequestTimeoutException INFO [2022-01-10 05:47:40,773] jepsen worker 20 - jepsen.util 3980 :invoke :write [6 6] INFO [2022-01-10 05:47:40,836] jepsen worker 61 - jepsen.util 4471 :info :write [20 6] :RequestTimeoutException INFO [2022-01-10 05:47:40,837] jepsen worker 61 - jepsen.util 4561 :invoke :write [20 34] INFO [2022-01-10 05:47:40,875] jepsen worker 81 - jepsen.util 4761 :info :write [27 41] :RequestTimeoutException INFO [2022-01-10 05:47:40,876] jepsen worker 81 - jepsen.util 4851 :invoke :read [27 nil] INFO [2022-01-10 05:47:41,015] jepsen worker 85 - jepsen.util 4585 :info :write [28 10] :RequestTimeoutException INFO [2022-01-10 05:47:41,015] jepsen worker 85 - jepsen.util 4675 :invoke :write [28 20] INFO [2022-01-10 05:47:41,056] jepsen worker 32 - jepsen.util 4442 :info :write [10 30] :RequestTimeoutException INFO [2022-01-10 05:47:41,057] jepsen worker 32 - jepsen.util 4532 :invoke :read [10 nil] INFO [2022-01-10 05:47:41,066] jepsen worker 42 - jepsen.util 4002 :info :write [14 31] :RequestTimeoutException INFO [2022-01-10 05:47:41,067] jepsen worker 42 - jepsen.util 4092 :invoke :read [14 nil] INFO [2022-01-10 05:47:41,075] jepsen worker 89 - jepsen.util 4769 :info :write [29 36] :RequestTimeoutException INFO [2022-01-10 05:47:41,076] jepsen worker 89 - jepsen.util 4859 :invoke :read [29 nil] INFO [2022-01-10 05:47:41,084] jepsen worker 32 - jepsen.util 4532 :ok :read [10 36] INFO [2022-01-10 05:47:41,084] jepsen worker 32 - jepsen.util 4532 :invoke :write [10 21] INFO [2022-01-10 05:47:41,099] jepsen worker 89 - jepsen.util 4859 :ok :read [29 0] INFO [2022-01-10 05:47:41,099] jepsen worker 89 - jepsen.util 4859 :invoke :read [29 nil] INFO [2022-01-10 05:47:41,129] jepsen worker 89 - jepsen.util 4859 :ok :read [29 0] INFO [2022-01-10 05:47:41,129] jepsen worker 89 - jepsen.util 4859 :invoke :read [29 nil] INFO [2022-01-10 05:47:41,130] jepsen worker 68 - jepsen.util 4478 :info :write [22 29] :RequestTimeoutException INFO [2022-01-10 05:47:41,131] jepsen worker 68 - jepsen.util 4568 :invoke :write [22 31] INFO [2022-01-10 05:47:41,153] jepsen worker 89 - jepsen.util 4859 :ok :read [29 0] INFO [2022-01-10 05:47:41,154] jepsen worker 89 - jepsen.util 4859 :invoke :write [29 8] INFO [2022-01-10 05:47:41,174] jepsen worker 30 - jepsen.util 4080 :ok :read [10 36] INFO [2022-01-10 05:47:41,174] jepsen worker 30 - jepsen.util 4080 :invoke :write [10 21] INFO [2022-01-10 05:47:41,580] jepsen worker 67 - jepsen.util 4207 :ok :read [22 41] INFO [2022-01-10 05:47:41,581] jepsen worker 67 - jepsen.util 4207 :invoke :write [22 31] INFO [2022-01-10 05:47:41,588] jepsen worker 59 - jepsen.util 4289 :ok :read [19 31] INFO [2022-01-10 05:47:41,588] jepsen worker 59 - jepsen.util 4289 :invoke :read [19 nil] INFO [2022-01-10 05:47:41,706] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5087719298245614}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:41,706] jepsen worker nemesis - couchbase.util Rebalance check count: 609 INFO [2022-01-10 05:47:41,706] jepsen worker nemesis - couchbase.util Rebalance stuck count: 518 INFO [2022-01-10 05:47:41,736] jepsen worker 50 - jepsen.util 4550 :info :write [16 14] :RequestTimeoutException INFO [2022-01-10 05:47:41,736] jepsen worker 50 - jepsen.util 4640 :invoke :read [16 nil] INFO [2022-01-10 05:47:42,043] jepsen worker 58 - jepsen.util 4108 :info :write [19 24] :RequestTimeoutException INFO [2022-01-10 05:47:42,043] jepsen worker 5 - jepsen.util 4235 :info :write [1 6] :RequestTimeoutException INFO [2022-01-10 05:47:42,043] jepsen worker 0 - jepsen.util 4230 :info :write [0 33] :RequestTimeoutException INFO [2022-01-10 05:47:42,044] jepsen worker 5 - jepsen.util 4325 :invoke :write [1 44] INFO [2022-01-10 05:47:42,047] jepsen worker 58 - jepsen.util 4198 :invoke :read [19 nil] INFO [2022-01-10 05:47:42,047] jepsen worker 0 - jepsen.util 4320 :invoke :write [0 34] INFO [2022-01-10 05:47:42,066] jepsen worker 72 - jepsen.util 4482 :info :write [24 1] :RequestTimeoutException INFO [2022-01-10 05:47:42,067] jepsen worker 72 - jepsen.util 4572 :invoke :write [24 29] WARN [2022-01-10 05:47:42,114] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":13755678,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":15865817,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9263218,"encode_duration_us":21276,"operation_id":"0x8342","timeout_ms":10000,"last_server_duration_us":9223914,"total_duration_us":10001822},{"operation_name":"upsert","last_dispatch_duration_us":14414743,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14919577,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9451967,"encode_duration_us":25273,"operation_id":"0x836d","timeout_ms":10000,"last_server_duration_us":9431526,"total_duration_us":10002969}],"total_count":2}} WARN [2022-01-10 05:47:42,205] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10718877,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10718877,"total_server_duration_us":7282,"operation_id":"0x83f7","timeout_ms":30000,"last_server_duration_us":7282,"total_duration_us":13845065},{"operation_name":"get","last_dispatch_duration_us":808773,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":808773,"total_server_duration_us":15,"operation_id":"0x84a2","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":12608093},{"operation_name":"get","last_dispatch_duration_us":1446401,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1446401,"total_server_duration_us":15764,"operation_id":"0x84c2","timeout_ms":30000,"last_server_duration_us":15764,"total_duration_us":11201350},{"operation_name":"get","last_dispatch_duration_us":10091749,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10091749,"total_server_duration_us":6579,"operation_id":"0x8594","timeout_ms":30000,"last_server_duration_us":6579,"total_duration_us":10094245},{"operation_name":"upsert","last_dispatch_duration_us":10404164,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10404164,"total_server_duration_us":169585,"encode_duration_us":16848,"operation_id":"0x858a","timeout_ms":10000,"last_server_duration_us":169585,"total_duration_us":10010799},{"operation_name":"upsert","last_dispatch_duration_us":106486,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":4689608,"total_server_duration_us":60313,"encode_duration_us":19921,"operation_id":"0x8448","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10009729},{"operation_name":"upsert","last_dispatch_duration_us":11972701,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11972701,"total_server_duration_us":189220,"encode_duration_us":16596,"operation_id":"0x849a","timeout_ms":10000,"last_server_duration_us":189220,"total_duration_us":10008275},{"operation_name":"upsert","last_dispatch_duration_us":852730,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":852730,"total_server_duration_us":19340,"encode_duration_us":16523,"operation_id":"0x85b0","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10008247},{"operation_name":"upsert","last_dispatch_duration_us":35051,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":4256595,"total_server_duration_us":2870842,"encode_duration_us":16713,"operation_id":"0x8420","timeout_ms":10000,"last_server_duration_us":22804,"total_duration_us":10007588},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","encode_duration_us":16520,"operation_id":"0x851d","timeout_ms":10000,"total_duration_us":10007431}],"total_count":22}} WARN [2022-01-10 05:47:42,205] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11943051,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11943051,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9350615,"encode_duration_us":16826,"operation_id":"0x8529","timeout_ms":10000,"last_server_duration_us":9350615,"total_duration_us":10007179},{"operation_name":"upsert","last_dispatch_duration_us":12279710,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12558272,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9340993,"encode_duration_us":16964,"operation_id":"0x8311","timeout_ms":10000,"last_server_duration_us":9312957,"total_duration_us":10010145}],"total_count":2}} WARN [2022-01-10 05:47:42,214] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":8892567,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":24624849,"total_server_duration_us":171989,"operation_id":"0x8115","timeout_ms":30000,"last_server_duration_us":62,"total_duration_us":29792354},{"operation_name":"get","last_dispatch_duration_us":8898586,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":23805270,"total_server_duration_us":8091,"operation_id":"0x814b","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":28460043},{"operation_name":"get","last_dispatch_duration_us":8904303,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":16412182,"total_server_duration_us":139933,"operation_id":"0x82bb","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":20533765},{"operation_name":"get","last_dispatch_duration_us":1815763,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1815763,"total_server_duration_us":9398,"operation_id":"0x8480","timeout_ms":30000,"last_server_duration_us":9398,"total_duration_us":12589155},{"operation_name":"upsert","last_dispatch_duration_us":33534,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":1047033,"total_server_duration_us":571009,"encode_duration_us":19347,"operation_id":"0x85f5","timeout_ms":10000,"last_server_duration_us":20921,"total_duration_us":10014722},{"operation_name":"upsert","last_dispatch_duration_us":1479644,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9931429,"total_server_duration_us":262733,"encode_duration_us":20040,"operation_id":"0x852c","timeout_ms":10000,"last_server_duration_us":22306,"total_duration_us":10009228},{"operation_name":"upsert","last_dispatch_duration_us":1160234,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1160234,"total_server_duration_us":73959,"encode_duration_us":16603,"operation_id":"0x8588","timeout_ms":10000,"last_server_duration_us":73959,"total_duration_us":10006854},{"operation_name":"upsert","last_dispatch_duration_us":9096460,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":11329292,"total_server_duration_us":719080,"encode_duration_us":16456,"operation_id":"0x8479","timeout_ms":10000,"last_server_duration_us":22804,"total_duration_us":10006535},{"operation_name":"upsert","last_dispatch_duration_us":1010989,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9626779,"total_server_duration_us":2499299,"encode_duration_us":19190,"operation_id":"0x84cd","timeout_ms":10000,"last_server_duration_us":222781,"total_duration_us":10004652},{"operation_name":"upsert","last_dispatch_duration_us":1179908,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10124300,"total_server_duration_us":1617818,"encode_duration_us":17044,"operation_id":"0x8531","timeout_ms":10000,"last_server_duration_us":100910,"total_duration_us":10002903}],"total_count":21}} WARN [2022-01-10 05:47:42,301] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10226233,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10226233,"total_server_duration_us":77010,"operation_id":"0x85d0","timeout_ms":30000,"last_server_duration_us":77010,"total_duration_us":10228541},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16145,"operation_id":"0x8552","timeout_ms":10000,"total_duration_us":10008317},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16908,"operation_id":"0x84ed","timeout_ms":10000,"total_duration_us":10008050},{"operation_name":"upsert","last_dispatch_duration_us":3084033,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":6671337,"total_server_duration_us":1120850,"encode_duration_us":16482,"operation_id":"0x8476","timeout_ms":10000,"last_server_duration_us":125429,"total_duration_us":10007941},{"operation_name":"upsert","last_dispatch_duration_us":9046433,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9508714,"total_server_duration_us":9275848,"encode_duration_us":16859,"operation_id":"0x855b","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10006587},{"operation_name":"upsert","last_dispatch_duration_us":12692028,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":15191498,"total_server_duration_us":10533811,"encode_duration_us":16260,"operation_id":"0x844d","timeout_ms":10000,"last_server_duration_us":9389415,"total_duration_us":10006275},{"operation_name":"upsert","last_dispatch_duration_us":10226708,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":17709989,"total_server_duration_us":1121752,"encode_duration_us":20325,"operation_id":"0x8447","timeout_ms":10000,"last_server_duration_us":171350,"total_duration_us":10005939},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16201,"operation_id":"0x8520","timeout_ms":10000,"total_duration_us":10004431},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":17020,"operation_id":"0x859a","timeout_ms":10000,"total_duration_us":10002547},{"operation_name":"upsert","last_dispatch_duration_us":11055108,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":16781803,"total_server_duration_us":9850392,"encode_duration_us":17164,"operation_id":"0x844c","timeout_ms":10000,"last_server_duration_us":9371084,"total_duration_us":10002476}],"total_count":12}} WARN [2022-01-10 05:47:42,301] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13472963,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":13472963,"total_server_duration_us":62,"operation_id":"0x8446","timeout_ms":30000,"last_server_duration_us":62,"total_duration_us":13474170},{"operation_name":"get","last_dispatch_duration_us":13464612,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":13464612,"total_server_duration_us":100,"operation_id":"0x84ee","timeout_ms":30000,"last_server_duration_us":100,"total_duration_us":13466416},{"operation_name":"upsert","last_dispatch_duration_us":129094,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4517376,"total_server_duration_us":1491380,"encode_duration_us":16180,"operation_id":"0x85f6","timeout_ms":10000,"last_server_duration_us":107108,"total_duration_us":10010488},{"operation_name":"upsert","last_dispatch_duration_us":12216991,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":12216991,"total_server_duration_us":20601,"encode_duration_us":18516,"operation_id":"0x8527","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10009832},{"operation_name":"upsert","last_dispatch_duration_us":485138,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":5467236,"total_server_duration_us":2509712,"encode_duration_us":16600,"operation_id":"0x85c5","timeout_ms":10000,"last_server_duration_us":415247,"total_duration_us":10007929},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":16516,"operation_id":"0x84cc","timeout_ms":10000,"total_duration_us":10007617},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":19478,"operation_id":"0x85de","timeout_ms":10000,"total_duration_us":10007386},{"operation_name":"upsert","last_dispatch_duration_us":791969,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4893452,"total_server_duration_us":2004761,"encode_duration_us":22279,"operation_id":"0x85be","timeout_ms":10000,"last_server_duration_us":397091,"total_duration_us":10005777},{"operation_name":"upsert","last_dispatch_duration_us":12035590,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":12035590,"total_server_duration_us":141324,"encode_duration_us":16094,"operation_id":"0x8499","timeout_ms":10000,"last_server_duration_us":141324,"total_duration_us":10004577},{"operation_name":"upsert","last_dispatch_duration_us":2179878,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":10878629,"total_server_duration_us":350594,"encode_duration_us":16688,"operation_id":"0x852e","timeout_ms":10000,"last_server_duration_us":63697,"total_duration_us":10002003}],"total_count":14}} WARN [2022-01-10 05:47:42,305] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14976738,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14976738,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9033121,"encode_duration_us":17619,"operation_id":"0x840e","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10004546},{"operation_name":"upsert","last_dispatch_duration_us":9035637,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":14277632,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":10539652,"encode_duration_us":16385,"operation_id":"0x841b","timeout_ms":10000,"last_server_duration_us":9020392,"total_duration_us":10008660},{"operation_name":"upsert","last_dispatch_duration_us":13939907,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13939907,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9212141,"encode_duration_us":18692,"operation_id":"0x837f","timeout_ms":10000,"last_server_duration_us":9212141,"total_duration_us":10008175},{"operation_name":"upsert","last_dispatch_duration_us":13106380,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13106380,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9245337,"encode_duration_us":16459,"operation_id":"0x8421","timeout_ms":10000,"last_server_duration_us":9245337,"total_duration_us":10010667}],"total_count":4}} INFO [2022-01-10 05:47:42,322] jepsen worker 12 - jepsen.util 4872 :info :write [4 39] :RequestTimeoutException INFO [2022-01-10 05:47:42,322] jepsen worker 12 - jepsen.util 4962 :invoke :write [4 1] INFO [2022-01-10 05:47:42,351] jepsen worker 58 - jepsen.util 4198 :ok :read [19 31] INFO [2022-01-10 05:47:42,352] jepsen worker 58 - jepsen.util 4198 :invoke :write [19 33] WARN [2022-01-10 05:47:42,401] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11055108,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16781803,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9850392,"encode_duration_us":17164,"operation_id":"0x844c","timeout_ms":10000,"last_server_duration_us":9371084,"total_duration_us":10002476},{"operation_name":"upsert","last_dispatch_duration_us":12692028,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15191498,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10533811,"encode_duration_us":16260,"operation_id":"0x844d","timeout_ms":10000,"last_server_duration_us":9389415,"total_duration_us":10006275},{"operation_name":"upsert","last_dispatch_duration_us":9046433,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9508714,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9275848,"encode_duration_us":16859,"operation_id":"0x855b","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10006587},{"operation_name":"upsert","last_dispatch_duration_us":14175266,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16403665,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9495698,"encode_duration_us":16900,"operation_id":"0x8343","timeout_ms":10000,"last_server_duration_us":9417480,"total_duration_us":10010057}],"total_count":4}} WARN [2022-01-10 05:47:42,501] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":13893167,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19153689,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":10045840,"encode_duration_us":16057,"operation_id":"0x8313","timeout_ms":10000,"last_server_duration_us":9628013,"total_duration_us":10001745},{"operation_name":"upsert","last_dispatch_duration_us":17772451,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18099372,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9152584,"encode_duration_us":17234,"operation_id":"0x8348","timeout_ms":10000,"last_server_duration_us":9046921,"total_duration_us":10005908},{"operation_name":"upsert","last_dispatch_duration_us":20725518,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":20725518,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":9026756,"encode_duration_us":17016,"operation_id":"0x837e","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10006117}],"total_count":3}} WARN [2022-01-10 05:47:42,506] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12966154,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12966154,"total_server_duration_us":32,"operation_id":"0x83b9","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":12968994},{"operation_name":"get","last_dispatch_duration_us":12126657,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":12126657,"total_server_duration_us":100,"operation_id":"0x846f","timeout_ms":30000,"last_server_duration_us":100,"total_duration_us":12127860},{"operation_name":"upsert","last_dispatch_duration_us":13106380,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13106380,"total_server_duration_us":9245337,"encode_duration_us":16459,"operation_id":"0x8421","timeout_ms":10000,"last_server_duration_us":9245337,"total_duration_us":10010667},{"operation_name":"upsert","last_dispatch_duration_us":9035637,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":14277632,"total_server_duration_us":10539652,"encode_duration_us":16385,"operation_id":"0x841b","timeout_ms":10000,"last_server_duration_us":9020392,"total_duration_us":10008660},{"operation_name":"upsert","last_dispatch_duration_us":519920,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8619824,"total_server_duration_us":926183,"encode_duration_us":16168,"operation_id":"0x858b","timeout_ms":10000,"last_server_duration_us":20680,"total_duration_us":10005681},{"operation_name":"upsert","last_dispatch_duration_us":50448,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":9813087,"total_server_duration_us":191879,"encode_duration_us":16395,"operation_id":"0x851f","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10005150},{"operation_name":"upsert","last_dispatch_duration_us":10005037,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10005037,"total_server_duration_us":257006,"encode_duration_us":18525,"operation_id":"0x846d","timeout_ms":10000,"last_server_duration_us":257006,"total_duration_us":10002114},{"operation_name":"upsert","last_dispatch_duration_us":6244467,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":6244467,"total_server_duration_us":235906,"encode_duration_us":17225,"operation_id":"0x85cb","timeout_ms":10000,"last_server_duration_us":235906,"total_duration_us":10001551},{"operation_name":"upsert","last_dispatch_duration_us":8165319,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":8606386,"total_server_duration_us":316149,"encode_duration_us":16272,"operation_id":"0x8565","timeout_ms":10000,"last_server_duration_us":49603,"total_duration_us":10000922},{"operation_name":"upsert","last_dispatch_duration_us":8336032,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10153559,"total_server_duration_us":448588,"encode_duration_us":17010,"operation_id":"0x84c3","timeout_ms":10000,"last_server_duration_us":18570,"total_duration_us":10000827}],"total_count":18}} INFO [2022-01-10 05:47:42,656] jepsen worker 84 - jepsen.util 4134 :info :write [28 3] :RequestTimeoutException INFO [2022-01-10 05:47:42,656] jepsen worker 84 - jepsen.util 4224 :invoke :write [28 46] INFO [2022-01-10 05:47:42,713] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5087719298245614}, :ns_1@172.23.107.227 {:progress 0.3548387096774194}, :ns_1@172.23.105.187 {:progress 0.3548387096774194}, :ns_1@172.23.107.228 {:progress 0.4823529411764705}} INFO [2022-01-10 05:47:42,713] jepsen worker nemesis - couchbase.util Rebalance check count: 610 INFO [2022-01-10 05:47:42,713] jepsen worker nemesis - couchbase.util Rebalance stuck count: 519 INFO [2022-01-10 05:47:42,771] jepsen worker 70 - jepsen.util 4300 :info :write [23 6] :RequestTimeoutException INFO [2022-01-10 05:47:42,772] jepsen worker 70 - jepsen.util 4390 :invoke :read [23 nil] INFO [2022-01-10 05:47:43,195] jepsen worker 83 - jepsen.util 4853 :info :write [27 41] :RequestTimeoutException INFO [2022-01-10 05:47:43,196] jepsen worker 83 - jepsen.util 4943 :invoke :write [27 40] INFO [2022-01-10 05:47:43,509] jepsen worker 74 - jepsen.util 4754 :info :write [24 8] :RequestTimeoutException INFO [2022-01-10 05:47:43,510] jepsen worker 74 - jepsen.util 4844 :invoke :write [24 36] INFO [2022-01-10 05:47:43,526] jepsen worker 53 - jepsen.util 5183 :info :write [17 0] :RequestTimeoutException INFO [2022-01-10 05:47:43,527] jepsen worker 53 - jepsen.util 5273 :invoke :write [17 15] INFO [2022-01-10 05:47:43,701] jepsen worker 2 - jepsen.util 4502 :info :write [0 43] :RequestTimeoutException INFO [2022-01-10 05:47:43,701] jepsen worker 2 - jepsen.util 4592 :invoke :read [0 nil] INFO [2022-01-10 05:47:43,720] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5087719298245614}, :ns_1@172.23.107.227 {:progress 0.3695014662756598}, :ns_1@172.23.105.187 {:progress 0.3695014662756598}, :ns_1@172.23.107.228 {:progress 0.4941176470588236}} INFO [2022-01-10 05:47:43,720] jepsen worker nemesis - couchbase.util Rebalance check count: 611 INFO [2022-01-10 05:47:43,720] jepsen worker nemesis - couchbase.util Rebalance stuck count: 519 INFO [2022-01-10 05:47:43,906] jepsen worker 26 - jepsen.util 3986 :info :write [8 30] :RequestTimeoutException INFO [2022-01-10 05:47:43,906] jepsen worker 26 - jepsen.util 4076 :invoke :write [8 42] INFO [2022-01-10 05:47:43,910] jepsen worker 75 - jepsen.util 4035 :info :write [25 19] :RequestTimeoutException INFO [2022-01-10 05:47:43,910] jepsen worker 75 - jepsen.util 4125 :invoke :read [25 nil] INFO [2022-01-10 05:47:44,350] jepsen worker 86 - jepsen.util 3686 :info :write [28 2] :RequestTimeoutException INFO [2022-01-10 05:47:44,350] jepsen worker 86 - jepsen.util 3776 :invoke :write [28 7] INFO [2022-01-10 05:47:44,375] jepsen worker 76 - jepsen.util 3676 :info :write [25 31] :RequestTimeoutException INFO [2022-01-10 05:47:44,376] jepsen worker 76 - jepsen.util 3766 :invoke :read [25 nil] INFO [2022-01-10 05:47:44,399] jepsen worker 6 - jepsen.util 4236 :info :write [2 49] :RequestTimeoutException INFO [2022-01-10 05:47:44,400] jepsen worker 6 - jepsen.util 4326 :invoke :read [2 nil] INFO [2022-01-10 05:47:44,404] jepsen worker 44 - jepsen.util 4364 :info :write [14 9] :RequestTimeoutException INFO [2022-01-10 05:47:44,405] jepsen worker 44 - jepsen.util 4454 :invoke :read [14 nil] INFO [2022-01-10 05:47:44,552] jepsen worker 78 - jepsen.util 5298 :info :write [26 48] :RequestTimeoutException INFO [2022-01-10 05:47:44,553] jepsen worker 78 - jepsen.util 5388 :invoke :write [26 11] INFO [2022-01-10 05:47:44,725] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5087719298245614}, :ns_1@172.23.107.227 {:progress 0.3695014662756598}, :ns_1@172.23.105.187 {:progress 0.3695014662756598}, :ns_1@172.23.107.228 {:progress 0.4941176470588236}} INFO [2022-01-10 05:47:44,725] jepsen worker nemesis - couchbase.util Rebalance check count: 612 INFO [2022-01-10 05:47:44,725] jepsen worker nemesis - couchbase.util Rebalance stuck count: 520 INFO [2022-01-10 05:47:44,784] jepsen worker 57 - jepsen.util 5097 :info :write [19 28] :RequestTimeoutException INFO [2022-01-10 05:47:44,785] jepsen worker 57 - jepsen.util 5187 :invoke :read [19 nil] INFO [2022-01-10 05:47:45,031] jepsen worker 47 - jepsen.util 4457 :info :write [15 15] :RequestTimeoutException INFO [2022-01-10 05:47:45,031] jepsen worker 47 - jepsen.util 4547 :invoke :write [15 33] INFO [2022-01-10 05:47:45,044] jepsen worker 54 - jepsen.util 4464 :info :write [18 33] :RequestTimeoutException INFO [2022-01-10 05:47:45,045] jepsen worker 54 - jepsen.util 4554 :invoke :read [18 nil] INFO [2022-01-10 05:47:45,068] jepsen worker 54 - jepsen.util 4554 :ok :read [18 14] INFO [2022-01-10 05:47:45,068] jepsen worker 54 - jepsen.util 4554 :invoke :read [18 nil] INFO [2022-01-10 05:47:45,090] jepsen worker 54 - jepsen.util 4554 :ok :read [18 14] INFO [2022-01-10 05:47:45,091] jepsen worker 54 - jepsen.util 4554 :invoke :read [18 nil] INFO [2022-01-10 05:47:45,114] jepsen worker 54 - jepsen.util 4554 :ok :read [18 14] INFO [2022-01-10 05:47:45,115] jepsen worker 54 - jepsen.util 4554 :invoke :write [18 11] INFO [2022-01-10 05:47:45,137] jepsen worker 37 - jepsen.util 3817 :info :write [12 24] :SyncWriteAmbiguous INFO [2022-01-10 05:47:45,138] jepsen worker 37 - jepsen.util 3907 :invoke :write [12 46] INFO [2022-01-10 05:47:45,360] jepsen worker 14 - jepsen.util 5324 :ok :read [4 3] INFO [2022-01-10 05:47:45,360] jepsen worker 14 - jepsen.util 5324 :invoke :read [4 nil] INFO [2022-01-10 05:47:45,454] jepsen worker 21 - jepsen.util 4341 :info :write [7 23] :RequestTimeoutException INFO [2022-01-10 05:47:45,455] jepsen worker 21 - jepsen.util 4431 :invoke :write [7 35] INFO [2022-01-10 05:47:45,479] jepsen worker 65 - jepsen.util 4025 :ok :read [21 7] INFO [2022-01-10 05:47:45,480] jepsen worker 65 - jepsen.util 4025 :invoke :read [21 nil] INFO [2022-01-10 05:47:45,514] jepsen worker 3 - jepsen.util 4233 :fail :write [1 38] :DurabilityImpossible INFO [2022-01-10 05:47:45,514] jepsen worker 3 - jepsen.util 4233 :invoke :write [1 39] INFO [2022-01-10 05:47:45,540] jepsen worker 14 - jepsen.util 5324 :ok :read [4 3] INFO [2022-01-10 05:47:45,541] jepsen worker 14 - jepsen.util 5324 :invoke :read [4 nil] INFO [2022-01-10 05:47:45,606] jepsen worker 14 - jepsen.util 5324 :ok :read [4 3] INFO [2022-01-10 05:47:45,607] jepsen worker 14 - jepsen.util 5324 :invoke :write [4 44] INFO [2022-01-10 05:47:45,653] jepsen worker 65 - jepsen.util 4025 :ok :read [21 7] INFO [2022-01-10 05:47:45,653] jepsen worker 65 - jepsen.util 4025 :invoke :write [21 34] INFO [2022-01-10 05:47:45,686] jepsen worker 44 - jepsen.util 4454 :ok :read [14 21] INFO [2022-01-10 05:47:45,687] jepsen worker 44 - jepsen.util 4454 :invoke :write [14 21] INFO [2022-01-10 05:47:45,731] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5087719298245614}, :ns_1@172.23.107.227 {:progress 0.3695014662756598}, :ns_1@172.23.105.187 {:progress 0.3695014662756598}, :ns_1@172.23.107.228 {:progress 0.4941176470588236}} INFO [2022-01-10 05:47:45,731] jepsen worker nemesis - couchbase.util Rebalance check count: 613 INFO [2022-01-10 05:47:45,731] jepsen worker nemesis - couchbase.util Rebalance stuck count: 521 INFO [2022-01-10 05:47:45,852] jepsen worker 18 - jepsen.util 4248 :ok :read [6 25] INFO [2022-01-10 05:47:45,853] jepsen worker 18 - jepsen.util 4248 :invoke :write [6 49] INFO [2022-01-10 05:47:45,866] jepsen worker 38 - jepsen.util 3908 :info :write [12 3] :RequestTimeoutException INFO [2022-01-10 05:47:45,866] jepsen worker 38 - jepsen.util 3998 :invoke :read [12 nil] INFO [2022-01-10 05:47:45,886] jepsen worker 80 - jepsen.util 5300 :info :write [26 28] :RequestTimeoutException INFO [2022-01-10 05:47:45,886] jepsen worker 80 - jepsen.util 5390 :invoke :write [26 20] INFO [2022-01-10 05:47:45,979] jepsen worker 10 - jepsen.util 4240 :ok :read [3 12] INFO [2022-01-10 05:47:45,980] jepsen worker 10 - jepsen.util 4240 :invoke :read [3 nil] INFO [2022-01-10 05:47:46,056] jepsen worker 36 - jepsen.util 3096 :info :write [12 11] :RequestTimeoutException INFO [2022-01-10 05:47:46,057] jepsen worker 36 - jepsen.util 3186 :invoke :read [12 nil] INFO [2022-01-10 05:47:46,080] jepsen worker 36 - jepsen.util 3186 :ok :read [12 14] INFO [2022-01-10 05:47:46,080] jepsen worker 36 - jepsen.util 3186 :invoke :read [12 nil] INFO [2022-01-10 05:47:46,102] jepsen worker 36 - jepsen.util 3186 :ok :read [12 14] INFO [2022-01-10 05:47:46,103] jepsen worker 36 - jepsen.util 3186 :invoke :read [12 nil] INFO [2022-01-10 05:47:46,129] jepsen worker 36 - jepsen.util 3186 :ok :read [12 14] INFO [2022-01-10 05:47:46,129] jepsen worker 36 - jepsen.util 3186 :invoke :read [12 nil] INFO [2022-01-10 05:47:46,160] jepsen worker 27 - jepsen.util 4527 :info :write [9 22] :RequestTimeoutException INFO [2022-01-10 05:47:46,161] jepsen worker 27 - jepsen.util 4617 :invoke :write [9 7] INFO [2022-01-10 05:47:46,169] jepsen worker 36 - jepsen.util 3186 :ok :read [12 14] INFO [2022-01-10 05:47:46,170] jepsen worker 36 - jepsen.util 3186 :invoke :write [12 35] INFO [2022-01-10 05:47:46,561] jepsen worker 1 - jepsen.util 4321 :info :write [0 29] :RequestTimeoutException INFO [2022-01-10 05:47:46,562] jepsen worker 1 - jepsen.util 4411 :invoke :read [0 nil] INFO [2022-01-10 05:47:46,659] jepsen worker 82 - jepsen.util 4402 :info :write [27 38] :RequestTimeoutException INFO [2022-01-10 05:47:46,660] jepsen worker 82 - jepsen.util 4492 :invoke :read [27 nil] INFO [2022-01-10 05:47:46,736] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5146198830409356}, :ns_1@172.23.107.227 {:progress 0.3695014662756598}, :ns_1@172.23.105.187 {:progress 0.3695014662756598}, :ns_1@172.23.107.228 {:progress 0.4941176470588236}} INFO [2022-01-10 05:47:46,736] jepsen worker nemesis - couchbase.util Rebalance check count: 614 INFO [2022-01-10 05:47:46,736] jepsen worker nemesis - couchbase.util Rebalance stuck count: 521 INFO [2022-01-10 05:47:46,805] jepsen worker 31 - jepsen.util 4531 :info :write [10 7] :RequestTimeoutException INFO [2022-01-10 05:47:46,806] jepsen worker 31 - jepsen.util 4621 :invoke :read [10 nil] INFO [2022-01-10 05:47:46,887] jepsen worker 23 - jepsen.util 4433 :ok :read [7 :nil] INFO [2022-01-10 05:47:46,887] jepsen worker 23 - jepsen.util 4433 :invoke :write [7 7] INFO [2022-01-10 05:47:47,019] jepsen worker 48 - jepsen.util 4098 :info :write [16 21] :RequestTimeoutException INFO [2022-01-10 05:47:47,020] jepsen worker 48 - jepsen.util 4188 :invoke :write [16 41] INFO [2022-01-10 05:47:47,089] jepsen worker 87 - jepsen.util 4227 :info :write [29 14] :RequestTimeoutException INFO [2022-01-10 05:47:47,090] jepsen worker 87 - jepsen.util 4317 :invoke :write [29 44] INFO [2022-01-10 05:47:47,355] jepsen worker 24 - jepsen.util 3894 :info :write [8 32] :RequestTimeoutException INFO [2022-01-10 05:47:47,442] jepsen worker 40 - jepsen.util 3910 :info :write [13 4] :RequestTimeoutException INFO [2022-01-10 05:47:47,443] jepsen worker 24 - jepsen.util 3984 :invoke :read [8 nil] INFO [2022-01-10 05:47:47,443] jepsen worker 40 - jepsen.util 4000 :invoke :read [13 nil] INFO [2022-01-10 05:47:47,458] jepsen worker 29 - jepsen.util 4169 :ok :read [9 6] INFO [2022-01-10 05:47:47,458] jepsen worker 29 - jepsen.util 4169 :invoke :write [9 24] INFO [2022-01-10 05:47:47,536] jepsen worker 62 - jepsen.util 4652 :info :write [20 14] :RequestTimeoutException INFO [2022-01-10 05:47:47,537] jepsen worker 62 - jepsen.util 4742 :invoke :write [20 10] INFO [2022-01-10 05:47:47,546] jepsen worker 19 - jepsen.util 4429 :info :write [6 28] :SyncWriteAmbiguous INFO [2022-01-10 05:47:47,546] jepsen worker 19 - jepsen.util 4519 :invoke :write [6 20] INFO [2022-01-10 05:47:47,602] jepsen worker 49 - jepsen.util 4099 :ok :read [16 38] INFO [2022-01-10 05:47:47,603] jepsen worker 49 - jepsen.util 4099 :invoke :write [16 48] INFO [2022-01-10 05:47:47,741] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5146198830409356}, :ns_1@172.23.107.227 {:progress 0.3695014662756598}, :ns_1@172.23.105.187 {:progress 0.3695014662756598}, :ns_1@172.23.107.228 {:progress 0.4941176470588236}} INFO [2022-01-10 05:47:47,741] jepsen worker nemesis - couchbase.util Rebalance check count: 615 INFO [2022-01-10 05:47:47,742] jepsen worker nemesis - couchbase.util Rebalance stuck count: 522 INFO [2022-01-10 05:47:47,796] jepsen worker 16 - jepsen.util 4426 :ok :read [5 25] INFO [2022-01-10 05:47:47,797] jepsen worker 16 - jepsen.util 4426 :invoke :write [5 1] INFO [2022-01-10 05:47:47,863] jepsen worker 50 - jepsen.util 4640 :ok :read [16 38] INFO [2022-01-10 05:47:47,864] jepsen worker 50 - jepsen.util 4640 :invoke :read [16 nil] INFO [2022-01-10 05:47:47,895] jepsen worker 35 - jepsen.util 4085 :ok :read [11 21] INFO [2022-01-10 05:47:47,896] jepsen worker 35 - jepsen.util 4085 :invoke :read [11 nil] INFO [2022-01-10 05:47:47,945] jepsen worker 28 - jepsen.util 4078 :info :write [9 44] :RequestTimeoutException INFO [2022-01-10 05:47:47,946] jepsen worker 28 - jepsen.util 4168 :invoke :read [9 nil] INFO [2022-01-10 05:47:47,956] jepsen worker 81 - jepsen.util 4851 :ok :read [27 30] INFO [2022-01-10 05:47:47,956] jepsen worker 81 - jepsen.util 4851 :invoke :read [27 nil] INFO [2022-01-10 05:47:48,015] jepsen worker 55 - jepsen.util 4825 :info :write [18 3] :RequestTimeoutException INFO [2022-01-10 05:47:48,016] jepsen worker 55 - jepsen.util 4915 :invoke :write [18 26] INFO [2022-01-10 05:47:48,116] jepsen worker 64 - jepsen.util 4294 :info :write [21 38] :RequestTimeoutException INFO [2022-01-10 05:47:48,116] jepsen worker 64 - jepsen.util 4384 :invoke :read [21 nil] INFO [2022-01-10 05:47:48,150] jepsen worker 75 - jepsen.util 4125 :ok :read [25 :nil] INFO [2022-01-10 05:47:48,151] jepsen worker 75 - jepsen.util 4125 :invoke :read [25 nil] INFO [2022-01-10 05:47:48,180] jepsen worker 79 - jepsen.util 5119 :info :write [26 5] :RequestTimeoutException INFO [2022-01-10 05:47:48,180] jepsen worker 79 - jepsen.util 5209 :invoke :read [26 nil] INFO [2022-01-10 05:47:48,215] jepsen worker 33 - jepsen.util 4533 :info :write [11 22] :RequestTimeoutException INFO [2022-01-10 05:47:48,216] jepsen worker 33 - jepsen.util 4623 :invoke :read [11 nil] INFO [2022-01-10 05:47:48,261] jepsen worker 66 - jepsen.util 4026 :info :write [22 48] :RequestTimeoutException INFO [2022-01-10 05:47:48,261] jepsen worker 8 - jepsen.util 4418 :info :write [2 48] :RequestTimeoutException INFO [2022-01-10 05:47:48,262] jepsen worker 66 - jepsen.util 4116 :invoke :write [22 23] INFO [2022-01-10 05:47:48,262] jepsen worker 8 - jepsen.util 4508 :invoke :read [2 nil] INFO [2022-01-10 05:47:48,271] jepsen worker 77 - jepsen.util 4757 :info :write [25 22] :RequestTimeoutException INFO [2022-01-10 05:47:48,272] jepsen worker 77 - jepsen.util 4847 :invoke :write [25 1] INFO [2022-01-10 05:47:48,290] jepsen worker 35 - jepsen.util 4085 :ok :read [11 21] INFO [2022-01-10 05:47:48,294] jepsen worker 35 - jepsen.util 4085 :invoke :write [11 19] INFO [2022-01-10 05:47:48,332] jepsen worker 75 - jepsen.util 4125 :ok :read [25 :nil] INFO [2022-01-10 05:47:48,332] jepsen worker 75 - jepsen.util 4125 :invoke :write [25 47] INFO [2022-01-10 05:47:48,349] jepsen worker 6 - jepsen.util 4326 :ok :read [2 14] INFO [2022-01-10 05:47:48,349] jepsen worker 6 - jepsen.util 4326 :invoke :write [2 43] INFO [2022-01-10 05:47:48,356] jepsen worker 60 - jepsen.util 4560 :info :write [20 33] :RequestTimeoutException INFO [2022-01-10 05:47:48,357] jepsen worker 60 - jepsen.util 4650 :invoke :write [20 0] INFO [2022-01-10 05:47:48,366] jepsen worker 73 - jepsen.util 4033 :info :write [24 46] :RequestTimeoutException INFO [2022-01-10 05:47:48,367] jepsen worker 73 - jepsen.util 4123 :invoke :write [24 24] INFO [2022-01-10 05:47:48,376] jepsen worker 34 - jepsen.util 4174 :info :write [11 14] :RequestTimeoutException INFO [2022-01-10 05:47:48,376] jepsen worker 34 - jepsen.util 4264 :invoke :read [11 nil] INFO [2022-01-10 05:47:48,455] jepsen worker 50 - jepsen.util 4640 :ok :read [16 38] INFO [2022-01-10 05:47:48,455] jepsen worker 50 - jepsen.util 4640 :invoke :read [16 nil] INFO [2022-01-10 05:47:48,533] jepsen worker 8 - jepsen.util 4508 :ok :read [2 14] INFO [2022-01-10 05:47:48,534] jepsen worker 8 - jepsen.util 4508 :invoke :read [2 nil] INFO [2022-01-10 05:47:48,660] jepsen worker 41 - jepsen.util 4361 :info :write [13 41] :RequestTimeoutException INFO [2022-01-10 05:47:48,661] jepsen worker 41 - jepsen.util 4451 :invoke :read [13 nil] INFO [2022-01-10 05:47:48,686] jepsen worker 63 - jepsen.util 4023 :info :write [21 9] :RequestTimeoutException INFO [2022-01-10 05:47:48,686] jepsen worker 63 - jepsen.util 4113 :invoke :write [21 13] INFO [2022-01-10 05:47:48,743] jepsen worker 40 - jepsen.util 4000 :ok :read [13 15] INFO [2022-01-10 05:47:48,744] jepsen worker 40 - jepsen.util 4000 :invoke :read [13 nil] INFO [2022-01-10 05:47:48,748] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.52046783625731}, :ns_1@172.23.107.227 {:progress 0.375366568914956}, :ns_1@172.23.105.187 {:progress 0.375366568914956}, :ns_1@172.23.107.228 {:progress 0.5058823529411764}} INFO [2022-01-10 05:47:48,748] jepsen worker nemesis - couchbase.util Rebalance check count: 616 INFO [2022-01-10 05:47:48,748] jepsen worker nemesis - couchbase.util Rebalance stuck count: 522 INFO [2022-01-10 05:47:48,761] jepsen worker 1 - jepsen.util 4411 :ok :read [0 46] INFO [2022-01-10 05:47:48,761] jepsen worker 1 - jepsen.util 4411 :invoke :read [0 nil] INFO [2022-01-10 05:47:48,871] jepsen worker 8 - jepsen.util 4508 :ok :read [2 14] INFO [2022-01-10 05:47:48,871] jepsen worker 8 - jepsen.util 4508 :invoke :write [2 18] INFO [2022-01-10 05:47:48,881] jepsen worker 43 - jepsen.util 4273 :info :write [14 35] :RequestTimeoutException INFO [2022-01-10 05:47:48,883] jepsen worker 43 - jepsen.util 4363 :invoke :write [14 12] INFO [2022-01-10 05:47:48,944] jepsen worker 88 - jepsen.util 4318 :info :write [29 12] :RequestTimeoutException INFO [2022-01-10 05:47:48,945] jepsen worker 88 - jepsen.util 4408 :invoke :write [29 15] INFO [2022-01-10 05:47:49,013] jepsen worker 40 - jepsen.util 4000 :ok :read [13 15] INFO [2022-01-10 05:47:49,014] jepsen worker 40 - jepsen.util 4000 :invoke :write [13 15] INFO [2022-01-10 05:47:49,031] jepsen worker 81 - jepsen.util 4851 :ok :read [27 30] INFO [2022-01-10 05:47:49,031] jepsen worker 81 - jepsen.util 4851 :invoke :write [27 8] INFO [2022-01-10 05:47:49,035] jepsen worker 1 - jepsen.util 4411 :ok :read [0 46] INFO [2022-01-10 05:47:49,036] jepsen worker 1 - jepsen.util 4411 :invoke :write [0 6] INFO [2022-01-10 05:47:49,106] jepsen worker 22 - jepsen.util 4162 :info :write [7 49] :RequestTimeoutException INFO [2022-01-10 05:47:49,107] jepsen worker 22 - jepsen.util 4252 :invoke :read [7 nil] INFO [2022-01-10 05:47:49,167] jepsen worker 64 - jepsen.util 4384 :ok :read [21 7] INFO [2022-01-10 05:47:49,167] jepsen worker 64 - jepsen.util 4384 :invoke :read [21 nil] INFO [2022-01-10 05:47:49,355] jepsen worker 76 - jepsen.util 3766 :ok :read [25 :nil] INFO [2022-01-10 05:47:49,356] jepsen worker 76 - jepsen.util 3766 :invoke :read [25 nil] INFO [2022-01-10 05:47:49,368] jepsen worker 64 - jepsen.util 4384 :ok :read [21 7] INFO [2022-01-10 05:47:49,369] jepsen worker 64 - jepsen.util 4384 :invoke :write [21 25] INFO [2022-01-10 05:47:49,403] jepsen worker 70 - jepsen.util 4390 :ok :read [23 26] INFO [2022-01-10 05:47:49,404] jepsen worker 70 - jepsen.util 4390 :invoke :read [23 nil] INFO [2022-01-10 05:47:49,446] jepsen worker 70 - jepsen.util 4390 :ok :read [23 26] INFO [2022-01-10 05:47:49,447] jepsen worker 70 - jepsen.util 4390 :invoke :read [23 nil] INFO [2022-01-10 05:47:49,456] jepsen worker 24 - jepsen.util 3984 :ok :read [8 39] INFO [2022-01-10 05:47:49,457] jepsen worker 24 - jepsen.util 3984 :invoke :write [8 18] INFO [2022-01-10 05:47:49,495] jepsen worker 11 - jepsen.util 4241 :ok :read [3 12] INFO [2022-01-10 05:47:49,496] jepsen worker 11 - jepsen.util 4241 :invoke :write [3 12] INFO [2022-01-10 05:47:49,514] jepsen worker 70 - jepsen.util 4390 :ok :read [23 26] INFO [2022-01-10 05:47:49,514] jepsen worker 70 - jepsen.util 4390 :invoke :read [23 nil] INFO [2022-01-10 05:47:49,524] jepsen worker 76 - jepsen.util 3766 :ok :read [25 :nil] INFO [2022-01-10 05:47:49,525] jepsen worker 76 - jepsen.util 3766 :invoke :read [25 nil] INFO [2022-01-10 05:47:49,542] jepsen worker 42 - jepsen.util 4092 :ok :read [14 21] INFO [2022-01-10 05:47:49,542] jepsen worker 42 - jepsen.util 4092 :invoke :read [14 nil] INFO [2022-01-10 05:47:49,559] jepsen worker 70 - jepsen.util 4390 :ok :read [23 26] INFO [2022-01-10 05:47:49,562] jepsen worker 70 - jepsen.util 4390 :invoke :read [23 nil] INFO [2022-01-10 05:47:49,620] jepsen worker 70 - jepsen.util 4390 :ok :read [23 26] INFO [2022-01-10 05:47:49,621] jepsen worker 70 - jepsen.util 4390 :invoke :write [23 30] INFO [2022-01-10 05:47:49,635] jepsen worker 50 - jepsen.util 4640 :ok :read [16 38] INFO [2022-01-10 05:47:49,635] jepsen worker 50 - jepsen.util 4640 :invoke :read [16 nil] INFO [2022-01-10 05:47:49,652] jepsen worker 41 - jepsen.util 4451 :ok :read [13 15] INFO [2022-01-10 05:47:49,653] jepsen worker 41 - jepsen.util 4451 :invoke :read [13 nil] INFO [2022-01-10 05:47:49,656] jepsen worker 69 - jepsen.util 3939 :info :write [23 40] :RequestTimeoutException INFO [2022-01-10 05:47:49,656] jepsen worker 69 - jepsen.util 4029 :invoke :write [23 21] INFO [2022-01-10 05:47:49,762] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.52046783625731}, :ns_1@172.23.107.227 {:progress 0.375366568914956}, :ns_1@172.23.105.187 {:progress 0.375366568914956}, :ns_1@172.23.107.228 {:progress 0.5058823529411764}} INFO [2022-01-10 05:47:49,762] jepsen worker nemesis - couchbase.util Rebalance check count: 617 INFO [2022-01-10 05:47:49,762] jepsen worker nemesis - couchbase.util Rebalance stuck count: 523 INFO [2022-01-10 05:47:49,900] jepsen worker 45 - jepsen.util 4725 :info :write [15 2] :RequestTimeoutException INFO [2022-01-10 05:47:49,901] jepsen worker 45 - jepsen.util 4815 :invoke :write [15 1] INFO [2022-01-10 05:47:49,911] jepsen worker 25 - jepsen.util 3985 :info :write [8 20] :RequestTimeoutException INFO [2022-01-10 05:47:49,911] jepsen worker 25 - jepsen.util 4075 :invoke :write [8 14] INFO [2022-01-10 05:47:50,376] jepsen worker 71 - jepsen.util 3851 :info :write [23 11] :RequestTimeoutException INFO [2022-01-10 05:47:50,377] jepsen worker 71 - jepsen.util 3941 :invoke :write [23 7] INFO [2022-01-10 05:47:50,441] jepsen worker 15 - jepsen.util 3795 :info :write [5 31] :RequestTimeoutException INFO [2022-01-10 05:47:50,441] jepsen worker 15 - jepsen.util 3885 :invoke :read [5 nil] INFO [2022-01-10 05:47:50,486] jepsen worker 51 - jepsen.util 5181 :info :write [17 44] :RequestTimeoutException INFO [2022-01-10 05:47:50,487] jepsen worker 51 - jepsen.util 5271 :invoke :read [17 nil] INFO [2022-01-10 05:47:50,559] jepsen worker 39 - jepsen.util 4719 :ok :read [13 15] INFO [2022-01-10 05:47:50,559] jepsen worker 39 - jepsen.util 4719 :invoke :write [13 3] INFO [2022-01-10 05:47:50,560] jepsen worker 13 - jepsen.util 5053 :info :write [4 6] :RequestTimeoutException INFO [2022-01-10 05:47:50,561] jepsen worker 13 - jepsen.util 5143 :invoke :write [4 31] INFO [2022-01-10 05:47:50,576] jepsen worker 22 - jepsen.util 4252 :ok :read [7 :nil] INFO [2022-01-10 05:47:50,576] jepsen worker 22 - jepsen.util 4252 :invoke :read [7 nil] WARN [2022-01-10 05:47:50,625] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9116819,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13363262,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9293398,"encode_duration_us":17179,"operation_id":"0x8686","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10001206},{"operation_name":"upsert","last_dispatch_duration_us":9739822,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12187049,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9320700,"encode_duration_us":22350,"operation_id":"0x865c","timeout_ms":10000,"last_server_duration_us":9042674,"total_duration_us":10003706},{"operation_name":"upsert","last_dispatch_duration_us":10219930,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10862804,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9779079,"encode_duration_us":17060,"operation_id":"0x865e","timeout_ms":10000,"last_server_duration_us":9618202,"total_duration_us":10004559},{"operation_name":"upsert","last_dispatch_duration_us":9051509,"last_local_id":"EFE9906200000001/000000007CF9F49A","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":16747516,"last_local_socket":"172.23.104.79:44306","total_server_duration_us":9321644,"encode_duration_us":17040,"operation_id":"0x8530","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":10005382}],"total_count":4}} WARN [2022-01-10 05:47:50,625] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9578926,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":17900649,"total_server_duration_us":11138457,"encode_duration_us":16620,"operation_id":"0x85f4","timeout_ms":10000,"last_server_duration_us":9474800,"total_duration_us":10059234},{"operation_name":"upsert","last_dispatch_duration_us":622082,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":6808040,"total_server_duration_us":1451007,"encode_duration_us":16631,"operation_id":"0x8677","timeout_ms":10000,"last_server_duration_us":549589,"total_duration_us":10009860},{"operation_name":"upsert","last_dispatch_duration_us":735199,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4228853,"total_server_duration_us":675017,"encode_duration_us":16155,"operation_id":"0x8714","timeout_ms":10000,"last_server_duration_us":72726,"total_duration_us":10009189},{"operation_name":"upsert","last_dispatch_duration_us":107236,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":1489909,"total_server_duration_us":252778,"encode_duration_us":16276,"operation_id":"0x8715","timeout_ms":10000,"last_server_duration_us":21405,"total_duration_us":10007967},{"operation_name":"upsert","last_dispatch_duration_us":375346,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9704098,"total_server_duration_us":1849634,"encode_duration_us":18974,"operation_id":"0x85c1","timeout_ms":10000,"last_server_duration_us":21731,"total_duration_us":10007250},{"operation_name":"upsert","last_dispatch_duration_us":8212768,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9089304,"total_server_duration_us":282208,"encode_duration_us":17154,"operation_id":"0x874e","timeout_ms":10000,"last_server_duration_us":29137,"total_duration_us":10006780},{"operation_name":"upsert","last_dispatch_duration_us":8222396,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10796245,"total_server_duration_us":279664,"encode_duration_us":38973,"operation_id":"0x86dc","timeout_ms":10000,"last_server_duration_us":20045,"total_duration_us":10006287},{"operation_name":"upsert","last_dispatch_duration_us":10219930,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10862804,"total_server_duration_us":9779079,"encode_duration_us":17060,"operation_id":"0x865e","timeout_ms":10000,"last_server_duration_us":9618202,"total_duration_us":10004559},{"operation_name":"upsert","last_dispatch_duration_us":9739822,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":12187049,"total_server_duration_us":9320700,"encode_duration_us":22350,"operation_id":"0x865c","timeout_ms":10000,"last_server_duration_us":9042674,"total_duration_us":10003706},{"operation_name":"upsert","last_dispatch_duration_us":935740,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9107681,"total_server_duration_us":1393391,"encode_duration_us":19646,"operation_id":"0x85c0","timeout_ms":10000,"last_server_duration_us":45502,"total_duration_us":10002912}],"total_count":13}} INFO [2022-01-10 05:47:50,656] jepsen worker 7 - jepsen.util 4597 :info :write [2 33] :RequestTimeoutException INFO [2022-01-10 05:47:50,658] jepsen worker 7 - jepsen.util 4687 :invoke :read [2 nil] INFO [2022-01-10 05:47:50,670] jepsen worker 52 - jepsen.util 5452 :info :write [17 28] :RequestTimeoutException INFO [2022-01-10 05:47:50,671] jepsen worker 52 - jepsen.util 5542 :invoke :write [17 28] INFO [2022-01-10 05:47:50,764] jepsen worker 17 - jepsen.util 4157 :ok :read [5 25] INFO [2022-01-10 05:47:50,765] jepsen worker 17 - jepsen.util 4157 :invoke :write [5 33] INFO [2022-01-10 05:47:50,785] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.52046783625731}, :ns_1@172.23.107.227 {:progress 0.375366568914956}, :ns_1@172.23.105.187 {:progress 0.375366568914956}, :ns_1@172.23.107.228 {:progress 0.5058823529411764}} INFO [2022-01-10 05:47:50,785] jepsen worker nemesis - couchbase.util Rebalance check count: 618 INFO [2022-01-10 05:47:50,785] jepsen worker nemesis - couchbase.util Rebalance stuck count: 524 INFO [2022-01-10 05:47:50,799] jepsen worker 20 - jepsen.util 3980 :info :write [6 6] :RequestTimeoutException INFO [2022-01-10 05:47:50,801] jepsen worker 20 - jepsen.util 4070 :invoke :write [6 16] INFO [2022-01-10 05:47:50,821] jepsen worker 76 - jepsen.util 3766 :ok :read [25 :nil] INFO [2022-01-10 05:47:50,821] jepsen worker 76 - jepsen.util 3766 :invoke :read [25 nil] INFO [2022-01-10 05:47:50,838] jepsen worker 42 - jepsen.util 4092 :ok :read [14 21] INFO [2022-01-10 05:47:50,839] jepsen worker 42 - jepsen.util 4092 :invoke :read [14 nil] INFO [2022-01-10 05:47:50,858] jepsen worker 15 - jepsen.util 3885 :ok :read [5 25] INFO [2022-01-10 05:47:50,858] jepsen worker 15 - jepsen.util 3885 :invoke :read [5 nil] INFO [2022-01-10 05:47:50,862] jepsen worker 61 - jepsen.util 4561 :info :write [20 34] :RequestTimeoutException INFO [2022-01-10 05:47:50,863] jepsen worker 61 - jepsen.util 4651 :invoke :write [20 46] INFO [2022-01-10 05:47:50,934] jepsen worker 15 - jepsen.util 3885 :ok :read [5 25] INFO [2022-01-10 05:47:50,934] jepsen worker 15 - jepsen.util 3885 :invoke :write [5 46] INFO [2022-01-10 05:47:50,966] jepsen worker 22 - jepsen.util 4252 :ok :read [7 :nil] INFO [2022-01-10 05:47:50,966] jepsen worker 22 - jepsen.util 4252 :invoke :read [7 nil] INFO [2022-01-10 05:47:50,998] jepsen worker 2 - jepsen.util 4592 :ok :read [0 46] INFO [2022-01-10 05:47:50,999] jepsen worker 2 - jepsen.util 4592 :invoke :read [0 nil] INFO [2022-01-10 05:47:51,044] jepsen worker 85 - jepsen.util 4675 :info :write [28 20] :RequestTimeoutException INFO [2022-01-10 05:47:51,045] jepsen worker 85 - jepsen.util 4765 :invoke :read [28 nil] INFO [2022-01-10 05:47:51,054] jepsen worker 82 - jepsen.util 4492 :ok :read [27 30] INFO [2022-01-10 05:47:51,055] jepsen worker 82 - jepsen.util 4492 :invoke :read [27 nil] INFO [2022-01-10 05:47:51,074] jepsen worker 7 - jepsen.util 4687 :ok :read [2 14] INFO [2022-01-10 05:47:51,074] jepsen worker 7 - jepsen.util 4687 :invoke :read [2 nil] INFO [2022-01-10 05:47:51,105] jepsen worker 32 - jepsen.util 4532 :info :write [10 21] :RequestTimeoutException INFO [2022-01-10 05:47:51,107] jepsen worker 32 - jepsen.util 4622 :invoke :write [10 46] INFO [2022-01-10 05:47:51,134] jepsen worker 50 - jepsen.util 4640 :ok :read [16 38] INFO [2022-01-10 05:47:51,135] jepsen worker 50 - jepsen.util 4640 :invoke :write [16 8] INFO [2022-01-10 05:47:51,151] jepsen worker 68 - jepsen.util 4568 :info :write [22 31] :RequestTimeoutException INFO [2022-01-10 05:47:51,152] jepsen worker 68 - jepsen.util 4658 :invoke :read [22 nil] INFO [2022-01-10 05:47:51,180] jepsen worker 89 - jepsen.util 4859 :info :write [29 8] :RequestTimeoutException INFO [2022-01-10 05:47:51,181] jepsen worker 89 - jepsen.util 4949 :invoke :read [29 nil] INFO [2022-01-10 05:47:51,188] jepsen worker 41 - jepsen.util 4451 :ok :read [13 15] INFO [2022-01-10 05:47:51,189] jepsen worker 41 - jepsen.util 4451 :invoke :read [13 nil] INFO [2022-01-10 05:47:51,196] jepsen worker 30 - jepsen.util 4080 :info :write [10 21] :RequestTimeoutException INFO [2022-01-10 05:47:51,197] jepsen worker 30 - jepsen.util 4170 :invoke :read [10 nil] INFO [2022-01-10 05:47:51,301] jepsen worker 76 - jepsen.util 3766 :ok :read [25 :nil] INFO [2022-01-10 05:47:51,301] jepsen worker 76 - jepsen.util 3766 :invoke :read [25 nil] INFO [2022-01-10 05:47:51,358] jepsen worker 42 - jepsen.util 4092 :ok :read [14 21] INFO [2022-01-10 05:47:51,358] jepsen worker 42 - jepsen.util 4092 :invoke :read [14 nil] INFO [2022-01-10 05:47:51,366] jepsen worker 22 - jepsen.util 4252 :ok :read [7 :nil] INFO [2022-01-10 05:47:51,367] jepsen worker 7 - jepsen.util 4687 :ok :read [2 14] INFO [2022-01-10 05:47:51,368] jepsen worker 22 - jepsen.util 4252 :invoke :write [7 41] INFO [2022-01-10 05:47:51,368] jepsen worker 7 - jepsen.util 4687 :invoke :write [2 49] INFO [2022-01-10 05:47:51,612] jepsen worker 67 - jepsen.util 4207 :info :write [22 31] :RequestTimeoutException INFO [2022-01-10 05:47:51,612] jepsen worker 67 - jepsen.util 4297 :invoke :read [22 nil] INFO [2022-01-10 05:47:51,666] jepsen worker 2 - jepsen.util 4592 :ok :read [0 46] INFO [2022-01-10 05:47:51,667] jepsen worker 2 - jepsen.util 4592 :invoke :read [0 nil] INFO [2022-01-10 05:47:51,677] jepsen worker 82 - jepsen.util 4492 :ok :read [27 30] INFO [2022-01-10 05:47:51,678] jepsen worker 82 - jepsen.util 4492 :invoke :read [27 nil] INFO [2022-01-10 05:47:51,791] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5321637426900585}, :ns_1@172.23.107.227 {:progress 0.375366568914956}, :ns_1@172.23.105.187 {:progress 0.375366568914956}, :ns_1@172.23.107.228 {:progress 0.5117647058823529}} INFO [2022-01-10 05:47:51,791] jepsen worker nemesis - couchbase.util Rebalance check count: 619 INFO [2022-01-10 05:47:51,791] jepsen worker nemesis - couchbase.util Rebalance stuck count: 524 INFO [2022-01-10 05:47:51,974] jepsen worker 76 - jepsen.util 3766 :ok :read [25 :nil] INFO [2022-01-10 05:47:51,975] jepsen worker 76 - jepsen.util 3766 :invoke :read [25 nil] INFO [2022-01-10 05:47:52,037] jepsen worker 42 - jepsen.util 4092 :ok :read [14 21] INFO [2022-01-10 05:47:52,038] jepsen worker 42 - jepsen.util 4092 :invoke :read [14 nil] INFO [2022-01-10 05:47:52,066] jepsen worker 0 - jepsen.util 4320 :info :write [0 34] :RequestTimeoutException INFO [2022-01-10 05:47:52,066] jepsen worker 0 - jepsen.util 4410 :invoke :write [0 10] INFO [2022-01-10 05:47:52,071] jepsen worker 5 - jepsen.util 4325 :info :write [1 44] :RequestTimeoutException INFO [2022-01-10 05:47:52,076] jepsen worker 5 - jepsen.util 4415 :invoke :write [1 16] INFO [2022-01-10 05:47:52,089] jepsen worker 72 - jepsen.util 4572 :info :write [24 29] :RequestTimeoutException INFO [2022-01-10 05:47:52,090] jepsen worker 72 - jepsen.util 4662 :invoke :read [24 nil] WARN [2022-01-10 05:47:52,128] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10263444,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10263444,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9114986,"encode_duration_us":18639,"operation_id":"0x871e","timeout_ms":10000,"last_server_duration_us":9114986,"total_duration_us":10005650},{"operation_name":"upsert","last_dispatch_duration_us":3581792,"last_local_id":"EFE9906200000002/00000000598DAEAE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":4628826,"last_local_socket":"172.23.104.79:44342","total_server_duration_us":593398,"encode_duration_us":19347,"operation_id":"0x85f5","timeout_ms":10000,"last_server_duration_us":22389,"total_duration_us":10014722}],"total_count":2}} WARN [2022-01-10 05:47:52,217] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10728298,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":19369914,"total_server_duration_us":9907482,"encode_duration_us":16621,"operation_id":"0x85f3","timeout_ms":10000,"last_server_duration_us":9788857,"total_duration_us":10060514},{"operation_name":"upsert","last_dispatch_duration_us":122281,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2623004,"total_server_duration_us":95024,"encode_duration_us":19192,"operation_id":"0x8685","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10009205},{"operation_name":"upsert","last_dispatch_duration_us":438733,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9421386,"total_server_duration_us":341698,"encode_duration_us":19154,"operation_id":"0x877b","timeout_ms":10000,"last_server_duration_us":48679,"total_duration_us":10008819},{"operation_name":"upsert","last_dispatch_duration_us":1092315,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":1092315,"total_server_duration_us":69347,"encode_duration_us":25013,"operation_id":"0x8789","timeout_ms":10000,"last_server_duration_us":69347,"total_duration_us":10008372},{"operation_name":"upsert","last_dispatch_duration_us":9058446,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9869040,"total_server_duration_us":9540200,"encode_duration_us":18148,"operation_id":"0x8617","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10006703},{"operation_name":"upsert","last_dispatch_duration_us":10598078,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10598078,"total_server_duration_us":48679,"encode_duration_us":18065,"operation_id":"0x87b4","timeout_ms":10000,"last_server_duration_us":48679,"total_duration_us":10006665},{"operation_name":"upsert","last_dispatch_duration_us":4686380,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":4686380,"total_server_duration_us":40802,"encode_duration_us":16341,"operation_id":"0x86c0","timeout_ms":10000,"last_server_duration_us":40802,"total_duration_us":10006477},{"operation_name":"upsert","last_dispatch_duration_us":249327,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10042194,"total_server_duration_us":438601,"encode_duration_us":17151,"operation_id":"0x871f","timeout_ms":10000,"last_server_duration_us":53494,"total_duration_us":10006173},{"operation_name":"upsert","last_dispatch_duration_us":206706,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1514069,"total_server_duration_us":96149,"encode_duration_us":19932,"operation_id":"0x8722","timeout_ms":10000,"last_server_duration_us":38899,"total_duration_us":10003427},{"operation_name":"upsert","last_dispatch_duration_us":10505887,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":17793144,"total_server_duration_us":194478,"encode_duration_us":16730,"operation_id":"0x8637","timeout_ms":10000,"last_server_duration_us":32451,"total_duration_us":10001610}],"total_count":14}} WARN [2022-01-10 05:47:52,218] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9058446,"last_local_id":"EFE9906200000006/0000000021E2C07D","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":9869040,"last_local_socket":"172.23.104.79:44302","total_server_duration_us":9540200,"encode_duration_us":18148,"operation_id":"0x8617","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10006703},{"operation_name":"upsert","last_dispatch_duration_us":18018366,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":22707975,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9110419,"encode_duration_us":19921,"operation_id":"0x8448","timeout_ms":10000,"last_server_duration_us":9050106,"total_duration_us":10009729},{"operation_name":"upsert","last_dispatch_duration_us":17969030,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17969030,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9030999,"encode_duration_us":16520,"operation_id":"0x851d","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10007431},{"operation_name":"upsert","last_dispatch_duration_us":10728298,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19369914,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9907482,"encode_duration_us":16621,"operation_id":"0x85f3","timeout_ms":10000,"last_server_duration_us":9788857,"total_duration_us":10060514}],"total_count":4}} WARN [2022-01-10 05:47:52,229] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":56521,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9446416,"total_server_duration_us":218619,"encode_duration_us":20949,"operation_id":"0x86b3","timeout_ms":10000,"last_server_duration_us":22887,"total_duration_us":10009931},{"operation_name":"upsert","last_dispatch_duration_us":1294442,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3299326,"total_server_duration_us":712865,"encode_duration_us":16584,"operation_id":"0x8651","timeout_ms":10000,"last_server_duration_us":524669,"total_duration_us":10007872},{"operation_name":"upsert","last_dispatch_duration_us":32491,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":7405487,"total_server_duration_us":1097264,"encode_duration_us":17254,"operation_id":"0x875a","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10007757},{"operation_name":"upsert","last_dispatch_duration_us":8961497,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8961497,"total_server_duration_us":59625,"encode_duration_us":16275,"operation_id":"0x8716","timeout_ms":10000,"last_server_duration_us":59625,"total_duration_us":10006565},{"operation_name":"upsert","last_dispatch_duration_us":9215908,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9215908,"total_server_duration_us":41767,"encode_duration_us":20925,"operation_id":"0x872c","timeout_ms":10000,"last_server_duration_us":41767,"total_duration_us":10006198},{"operation_name":"upsert","last_dispatch_duration_us":30987,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7951181,"total_server_duration_us":1794308,"encode_duration_us":16305,"operation_id":"0x862a","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10005806},{"operation_name":"upsert","last_dispatch_duration_us":10263444,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10263444,"total_server_duration_us":9114986,"encode_duration_us":18639,"operation_id":"0x871e","timeout_ms":10000,"last_server_duration_us":9114986,"total_duration_us":10005650},{"operation_name":"upsert","last_dispatch_duration_us":70811,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9438864,"total_server_duration_us":254941,"encode_duration_us":19038,"operation_id":"0x86b0","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10004717},{"operation_name":"upsert","last_dispatch_duration_us":266278,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1409385,"total_server_duration_us":43143,"encode_duration_us":21092,"operation_id":"0x8724","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10004146},{"operation_name":"upsert","last_dispatch_duration_us":121797,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9392469,"total_server_duration_us":284559,"encode_duration_us":17793,"operation_id":"0x86b5","timeout_ms":10000,"last_server_duration_us":33426,"total_duration_us":10003501}],"total_count":27}} WARN [2022-01-10 05:47:52,314] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17760081,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":17760081,"total_server_duration_us":27,"operation_id":"0x8566","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":17782751},{"operation_name":"get","last_dispatch_duration_us":10740099,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10740099,"total_server_duration_us":31969,"operation_id":"0x8684","timeout_ms":30000,"last_server_duration_us":31969,"total_duration_us":12850667},{"operation_name":"get","last_dispatch_duration_us":10921056,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10921056,"total_server_duration_us":12183,"operation_id":"0x86be","timeout_ms":30000,"last_server_duration_us":12183,"total_duration_us":11230695},{"operation_name":"upsert","last_dispatch_duration_us":9121811,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9121811,"total_server_duration_us":33622,"encode_duration_us":16814,"operation_id":"0x8786","timeout_ms":10000,"last_server_duration_us":33622,"total_duration_us":10006944},{"operation_name":"upsert","last_dispatch_duration_us":670216,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":9944787,"total_server_duration_us":378162,"encode_duration_us":21603,"operation_id":"0x8723","timeout_ms":10000,"last_server_duration_us":138798,"total_duration_us":10005790},{"operation_name":"upsert","last_dispatch_duration_us":512491,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10168353,"total_server_duration_us":570043,"encode_duration_us":18568,"operation_id":"0x874f","timeout_ms":10000,"last_server_duration_us":140239,"total_duration_us":10005541},{"operation_name":"upsert","last_dispatch_duration_us":9053022,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9053022,"total_server_duration_us":33328,"encode_duration_us":19824,"operation_id":"0x878b","timeout_ms":10000,"last_server_duration_us":33328,"total_duration_us":10003985},{"operation_name":"upsert","last_dispatch_duration_us":60493,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":881554,"total_server_duration_us":387347,"encode_duration_us":17666,"operation_id":"0x87c3","timeout_ms":10000,"last_server_duration_us":42634,"total_duration_us":10003581},{"operation_name":"upsert","last_dispatch_duration_us":15918844,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":15918844,"total_server_duration_us":9046921,"encode_duration_us":16705,"operation_id":"0x862e","timeout_ms":10000,"last_server_duration_us":9046921,"total_duration_us":10002970},{"operation_name":"get","last_dispatch_duration_us":7062561,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":7062561,"total_server_duration_us":10571,"operation_id":"0x87b3","timeout_ms":30000,"last_server_duration_us":10571,"total_duration_us":7064413}],"total_count":18}} WARN [2022-01-10 05:47:52,314] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":112235,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":112235,"total_server_duration_us":5582,"operation_id":"0x852b","timeout_ms":30000,"last_server_duration_us":5582,"total_duration_us":17523242},{"operation_name":"get","last_dispatch_duration_us":10261099,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10261099,"total_server_duration_us":17888,"operation_id":"0x8562","timeout_ms":30000,"last_server_duration_us":17888,"total_duration_us":15934742},{"operation_name":"get","last_dispatch_duration_us":14283271,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":14283271,"total_server_duration_us":27,"operation_id":"0x86bd","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":14285842},{"operation_name":"get","last_dispatch_duration_us":9686667,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9686667,"total_server_duration_us":6628,"operation_id":"0x877c","timeout_ms":30000,"last_server_duration_us":6628,"total_duration_us":10777489},{"operation_name":"upsert","last_dispatch_duration_us":648345,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9964656,"total_server_duration_us":130813,"encode_duration_us":19695,"operation_id":"0x87c5","timeout_ms":10000,"last_server_duration_us":46965,"total_duration_us":10010089},{"operation_name":"upsert","last_dispatch_duration_us":13264785,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":13264785,"total_server_duration_us":54094,"encode_duration_us":17194,"operation_id":"0x86f0","timeout_ms":10000,"last_server_duration_us":54094,"total_duration_us":10009285},{"operation_name":"upsert","last_dispatch_duration_us":132712,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":282666,"total_server_duration_us":40095,"encode_duration_us":17502,"operation_id":"0x87b2","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10007198},{"operation_name":"upsert","last_dispatch_duration_us":8212496,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":8896182,"total_server_duration_us":442652,"encode_duration_us":19899,"operation_id":"0x85ff","timeout_ms":10000,"last_server_duration_us":24154,"total_duration_us":10005557},{"operation_name":"upsert","last_dispatch_duration_us":12473042,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":12473042,"total_server_duration_us":39004,"encode_duration_us":24428,"operation_id":"0x8720","timeout_ms":10000,"last_server_duration_us":39004,"total_duration_us":10003464},{"operation_name":"upsert","last_dispatch_duration_us":9053856,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":9647632,"total_server_duration_us":9412617,"encode_duration_us":17405,"operation_id":"0x8663","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10002186}],"total_count":18}} WARN [2022-01-10 05:47:52,319] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10372972,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13980048,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9865963,"encode_duration_us":17688,"operation_id":"0x862d","timeout_ms":10000,"last_server_duration_us":9137368,"total_duration_us":10001639},{"operation_name":"upsert","last_dispatch_duration_us":11427400,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11427400,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9092628,"encode_duration_us":16574,"operation_id":"0x86bc","timeout_ms":10000,"last_server_duration_us":9092628,"total_duration_us":10006516}],"total_count":2}} INFO [2022-01-10 05:47:52,352] jepsen worker 12 - jepsen.util 4962 :info :write [4 1] :RequestTimeoutException INFO [2022-01-10 05:47:52,353] jepsen worker 12 - jepsen.util 5052 :invoke :read [4 nil] INFO [2022-01-10 05:47:52,374] jepsen worker 58 - jepsen.util 4198 :info :write [19 33] :RequestTimeoutException INFO [2022-01-10 05:47:52,375] jepsen worker 58 - jepsen.util 4288 :invoke :write [19 45] WARN [2022-01-10 05:47:52,414] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9053856,"last_local_id":"EFE9906200000004/000000008B42EFB5","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":9647632,"last_local_socket":"172.23.104.79:44292","total_server_duration_us":9412617,"encode_duration_us":17405,"operation_id":"0x8663","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10002186},{"operation_name":"upsert","last_dispatch_duration_us":19084225,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":24661402,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":10071414,"encode_duration_us":16247,"operation_id":"0x83bb","timeout_ms":10000,"last_server_duration_us":9700088,"total_duration_us":10007744},{"operation_name":"upsert","last_dispatch_duration_us":17222167,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17672364,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9312909,"encode_duration_us":17095,"operation_id":"0x8592","timeout_ms":10000,"last_server_duration_us":9067105,"total_duration_us":10002450},{"operation_name":"upsert","last_dispatch_duration_us":19321861,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25993199,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10576167,"encode_duration_us":16482,"operation_id":"0x8476","timeout_ms":10000,"last_server_duration_us":9455317,"total_duration_us":10007941}],"total_count":4}} WARN [2022-01-10 05:47:52,514] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":15918844,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":15918844,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":9046921,"encode_duration_us":16705,"operation_id":"0x862e","timeout_ms":10000,"last_server_duration_us":9046921,"total_duration_us":10002970},{"operation_name":"upsert","last_dispatch_duration_us":20906253,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20906253,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9360308,"encode_duration_us":16516,"operation_id":"0x84cc","timeout_ms":10000,"last_server_duration_us":9360308,"total_duration_us":10007617},{"operation_name":"upsert","last_dispatch_duration_us":9231662,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14125115,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":11221182,"encode_duration_us":22279,"operation_id":"0x85be","timeout_ms":10000,"last_server_duration_us":9216421,"total_duration_us":10005777}],"total_count":3}} WARN [2022-01-10 05:47:52,519] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10409810,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13888653,"total_server_duration_us":296325,"encode_duration_us":16248,"operation_id":"0x865b","timeout_ms":10000,"last_server_duration_us":83124,"total_duration_us":10009860},{"operation_name":"upsert","last_dispatch_duration_us":62947,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":960832,"total_server_duration_us":377438,"encode_duration_us":16533,"operation_id":"0x8792","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10008806},{"operation_name":"upsert","last_dispatch_duration_us":109835,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":1039974,"total_server_duration_us":402835,"encode_duration_us":18549,"operation_id":"0x878f","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10008173},{"operation_name":"upsert","last_dispatch_duration_us":1003530,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9785131,"total_server_duration_us":331132,"encode_duration_us":17505,"operation_id":"0x87ab","timeout_ms":10000,"last_server_duration_us":27764,"total_duration_us":10007824},{"operation_name":"upsert","last_dispatch_duration_us":445614,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10164104,"total_server_duration_us":131551,"encode_duration_us":25168,"operation_id":"0x872b","timeout_ms":10000,"last_server_duration_us":70286,"total_duration_us":10007529},{"operation_name":"upsert","last_dispatch_duration_us":11427400,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11427400,"total_server_duration_us":9092628,"encode_duration_us":16574,"operation_id":"0x86bc","timeout_ms":10000,"last_server_duration_us":9092628,"total_duration_us":10006516},{"operation_name":"upsert","last_dispatch_duration_us":30815,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":2551883,"total_server_duration_us":499885,"encode_duration_us":19171,"operation_id":"0x87c2","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10006424},{"operation_name":"upsert","last_dispatch_duration_us":882428,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10032841,"total_server_duration_us":477993,"encode_duration_us":17969,"operation_id":"0x877a","timeout_ms":10000,"last_server_duration_us":81111,"total_duration_us":10004628},{"operation_name":"upsert","last_dispatch_duration_us":709128,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2885433,"total_server_duration_us":289526,"encode_duration_us":16904,"operation_id":"0x87e8","timeout_ms":10000,"last_server_duration_us":139518,"total_duration_us":10004527},{"operation_name":"upsert","last_dispatch_duration_us":63475,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":3108966,"total_server_duration_us":388340,"encode_duration_us":17529,"operation_id":"0x871d","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10004229}],"total_count":21}} INFO [2022-01-10 05:47:52,676] jepsen worker 84 - jepsen.util 4224 :info :write [28 46] :RequestTimeoutException INFO [2022-01-10 05:47:52,677] jepsen worker 84 - jepsen.util 4314 :invoke :read [28 nil] INFO [2022-01-10 05:47:52,797] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5321637426900585}, :ns_1@172.23.107.227 {:progress 0.375366568914956}, :ns_1@172.23.105.187 {:progress 0.375366568914956}, :ns_1@172.23.107.228 {:progress 0.5117647058823529}} INFO [2022-01-10 05:47:52,797] jepsen worker nemesis - couchbase.util Rebalance check count: 620 INFO [2022-01-10 05:47:52,797] jepsen worker nemesis - couchbase.util Rebalance stuck count: 525 INFO [2022-01-10 05:47:52,808] jepsen worker 85 - jepsen.util 4765 :ok :read [28 45] INFO [2022-01-10 05:47:52,809] jepsen worker 85 - jepsen.util 4765 :invoke :read [28 nil] INFO [2022-01-10 05:47:52,848] jepsen worker 59 - jepsen.util 4289 :ok :read [19 31] INFO [2022-01-10 05:47:52,849] jepsen worker 59 - jepsen.util 4289 :invoke :write [19 7] INFO [2022-01-10 05:47:52,858] jepsen worker 57 - jepsen.util 5187 :ok :read [19 31] INFO [2022-01-10 05:47:52,859] jepsen worker 57 - jepsen.util 5187 :invoke :read [19 nil] INFO [2022-01-10 05:47:53,009] jepsen worker 72 - jepsen.util 4662 :ok :read [24 18] INFO [2022-01-10 05:47:53,009] jepsen worker 72 - jepsen.util 4662 :invoke :read [24 nil] INFO [2022-01-10 05:47:53,135] jepsen worker 76 - jepsen.util 3766 :ok :read [25 :nil] INFO [2022-01-10 05:47:53,136] jepsen worker 76 - jepsen.util 3766 :invoke :read [25 nil] INFO [2022-01-10 05:47:53,149] jepsen worker 42 - jepsen.util 4092 :ok :read [14 21] INFO [2022-01-10 05:47:53,149] jepsen worker 42 - jepsen.util 4092 :invoke :write [14 48] INFO [2022-01-10 05:47:53,215] jepsen worker 83 - jepsen.util 4943 :info :write [27 40] :RequestTimeoutException INFO [2022-01-10 05:47:53,215] jepsen worker 83 - jepsen.util 5033 :invoke :read [27 nil] INFO [2022-01-10 05:47:53,221] jepsen worker 72 - jepsen.util 4662 :ok :read [24 18] INFO [2022-01-10 05:47:53,221] jepsen worker 72 - jepsen.util 4662 :invoke :read [24 nil] INFO [2022-01-10 05:47:53,270] jepsen worker 76 - jepsen.util 3766 :ok :read [25 :nil] INFO [2022-01-10 05:47:53,270] jepsen worker 76 - jepsen.util 3766 :invoke :write [25 4] INFO [2022-01-10 05:47:53,458] jepsen worker 2 - jepsen.util 4592 :ok :read [0 46] INFO [2022-01-10 05:47:53,459] jepsen worker 2 - jepsen.util 4592 :invoke :read [0 nil] INFO [2022-01-10 05:47:53,471] jepsen worker 67 - jepsen.util 4297 :ok :read [22 41] INFO [2022-01-10 05:47:53,472] jepsen worker 67 - jepsen.util 4297 :invoke :read [22 nil] INFO [2022-01-10 05:47:53,478] jepsen worker 72 - jepsen.util 4662 :ok :read [24 18] INFO [2022-01-10 05:47:53,481] jepsen worker 72 - jepsen.util 4662 :invoke :write [24 39] INFO [2022-01-10 05:47:53,498] jepsen worker 28 - jepsen.util 4168 :ok :read [9 22] INFO [2022-01-10 05:47:53,499] jepsen worker 28 - jepsen.util 4168 :invoke :write [9 33] INFO [2022-01-10 05:47:53,508] jepsen worker 82 - jepsen.util 4492 :ok :read [27 30] INFO [2022-01-10 05:47:53,509] jepsen worker 82 - jepsen.util 4492 :invoke :read [27 nil] INFO [2022-01-10 05:47:53,514] jepsen worker 57 - jepsen.util 5187 :ok :read [19 31] INFO [2022-01-10 05:47:53,514] jepsen worker 57 - jepsen.util 5187 :invoke :read [19 nil] INFO [2022-01-10 05:47:53,536] jepsen worker 74 - jepsen.util 4844 :info :write [24 36] :RequestTimeoutException INFO [2022-01-10 05:47:53,536] jepsen worker 74 - jepsen.util 4934 :invoke :read [24 nil] INFO [2022-01-10 05:47:53,549] jepsen worker 53 - jepsen.util 5273 :info :write [17 15] :RequestTimeoutException INFO [2022-01-10 05:47:53,550] jepsen worker 53 - jepsen.util 5363 :invoke :write [17 49] INFO [2022-01-10 05:47:53,632] jepsen worker 85 - jepsen.util 4765 :ok :read [28 45] INFO [2022-01-10 05:47:53,634] jepsen worker 85 - jepsen.util 4765 :invoke :write [28 2] INFO [2022-01-10 05:47:53,666] jepsen worker 2 - jepsen.util 4592 :ok :read [0 46] INFO [2022-01-10 05:47:53,667] jepsen worker 2 - jepsen.util 4592 :invoke :write [0 32] INFO [2022-01-10 05:47:53,680] jepsen worker 57 - jepsen.util 5187 :ok :read [19 31] INFO [2022-01-10 05:47:53,681] jepsen worker 57 - jepsen.util 5187 :invoke :read [19 nil] INFO [2022-01-10 05:47:53,686] jepsen worker 82 - jepsen.util 4492 :ok :read [27 30] INFO [2022-01-10 05:47:53,687] jepsen worker 82 - jepsen.util 4492 :invoke :write [27 15] INFO [2022-01-10 05:47:53,802] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.543859649122807}, :ns_1@172.23.107.227 {:progress 0.375366568914956}, :ns_1@172.23.105.187 {:progress 0.375366568914956}, :ns_1@172.23.107.228 {:progress 0.5176470588235293}} INFO [2022-01-10 05:47:53,803] jepsen worker nemesis - couchbase.util Rebalance check count: 621 INFO [2022-01-10 05:47:53,803] jepsen worker nemesis - couchbase.util Rebalance stuck count: 525 INFO [2022-01-10 05:47:53,926] jepsen worker 26 - jepsen.util 4076 :info :write [8 42] :RequestTimeoutException INFO [2022-01-10 05:47:53,926] jepsen worker 26 - jepsen.util 4166 :invoke :write [8 5] INFO [2022-01-10 05:47:54,018] jepsen worker 83 - jepsen.util 5033 :ok :read [27 30] INFO [2022-01-10 05:47:54,019] jepsen worker 68 - jepsen.util 4658 :ok :read [22 41] INFO [2022-01-10 05:47:54,019] jepsen worker 83 - jepsen.util 5033 :invoke :write [27 20] INFO [2022-01-10 05:47:54,019] jepsen worker 68 - jepsen.util 4658 :invoke :read [22 nil] INFO [2022-01-10 05:47:54,157] jepsen worker 57 - jepsen.util 5187 :ok :read [19 31] INFO [2022-01-10 05:47:54,158] jepsen worker 57 - jepsen.util 5187 :invoke :read [19 nil] INFO [2022-01-10 05:47:54,238] jepsen worker 41 - jepsen.util 4451 :ok :read [13 15] INFO [2022-01-10 05:47:54,239] jepsen worker 41 - jepsen.util 4451 :invoke :write [13 34] INFO [2022-01-10 05:47:54,248] jepsen worker 37 - jepsen.util 3907 :info :write [12 46] :SyncWriteAmbiguous INFO [2022-01-10 05:47:54,249] jepsen worker 37 - jepsen.util 3997 :invoke :write [12 3] INFO [2022-01-10 05:47:54,375] jepsen worker 86 - jepsen.util 3776 :info :write [28 7] :RequestTimeoutException INFO [2022-01-10 05:47:54,376] jepsen worker 86 - jepsen.util 3866 :invoke :read [28 nil] INFO [2022-01-10 05:47:54,576] jepsen worker 78 - jepsen.util 5388 :info :write [26 11] :RequestTimeoutException INFO [2022-01-10 05:47:54,577] jepsen worker 78 - jepsen.util 5478 :invoke :write [26 38] INFO [2022-01-10 05:47:54,742] jepsen worker 57 - jepsen.util 5187 :ok :read [19 31] INFO [2022-01-10 05:47:54,742] jepsen worker 57 - jepsen.util 5187 :invoke :write [19 24] INFO [2022-01-10 05:47:54,808] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5555555555555556}, :ns_1@172.23.107.227 {:progress 0.375366568914956}, :ns_1@172.23.105.187 {:progress 0.375366568914956}, :ns_1@172.23.107.228 {:progress 0.5294117647058824}} INFO [2022-01-10 05:47:54,808] jepsen worker nemesis - couchbase.util Rebalance check count: 622 INFO [2022-01-10 05:47:54,808] jepsen worker nemesis - couchbase.util Rebalance stuck count: 525 INFO [2022-01-10 05:47:54,904] jepsen worker 68 - jepsen.util 4658 :ok :read [22 41] INFO [2022-01-10 05:47:54,905] jepsen worker 68 - jepsen.util 4658 :invoke :write [22 41] INFO [2022-01-10 05:47:54,988] jepsen worker 56 - jepsen.util 4646 :ok :read [18 14] INFO [2022-01-10 05:47:54,989] jepsen worker 56 - jepsen.util 4646 :invoke :read [18 nil] INFO [2022-01-10 05:47:54,998] jepsen worker 9 - jepsen.util 4959 :ok :read [3 12] INFO [2022-01-10 05:47:54,999] jepsen worker 9 - jepsen.util 4959 :invoke :read [3 nil] INFO [2022-01-10 05:47:55,037] jepsen worker 10 - jepsen.util 4240 :ok :read [3 12] INFO [2022-01-10 05:47:55,039] jepsen worker 10 - jepsen.util 4240 :invoke :write [3 46] INFO [2022-01-10 05:47:55,049] jepsen worker 89 - jepsen.util 4949 :ok :read [29 0] INFO [2022-01-10 05:47:55,050] jepsen worker 89 - jepsen.util 4949 :invoke :read [29 nil] INFO [2022-01-10 05:47:55,054] jepsen worker 47 - jepsen.util 4547 :info :write [15 33] :RequestTimeoutException INFO [2022-01-10 05:47:55,056] jepsen worker 47 - jepsen.util 4637 :invoke :read [15 nil] INFO [2022-01-10 05:47:55,060] jepsen worker 56 - jepsen.util 4646 :ok :read [18 14] INFO [2022-01-10 05:47:55,060] jepsen worker 56 - jepsen.util 4646 :invoke :write [18 42] INFO [2022-01-10 05:47:55,066] jepsen worker 9 - jepsen.util 4959 :ok :read [3 12] INFO [2022-01-10 05:47:55,067] jepsen worker 9 - jepsen.util 4959 :invoke :write [3 37] INFO [2022-01-10 05:47:55,089] jepsen worker 89 - jepsen.util 4949 :ok :read [29 0] INFO [2022-01-10 05:47:55,089] jepsen worker 89 - jepsen.util 4949 :invoke :write [29 23] INFO [2022-01-10 05:47:55,136] jepsen worker 54 - jepsen.util 4554 :info :write [18 11] :RequestTimeoutException INFO [2022-01-10 05:47:55,136] jepsen worker 54 - jepsen.util 4644 :invoke :write [18 28] INFO [2022-01-10 05:47:55,409] jepsen worker 31 - jepsen.util 4621 :ok :read [10 36] INFO [2022-01-10 05:47:55,410] jepsen worker 31 - jepsen.util 4621 :invoke :read [10 nil] INFO [2022-01-10 05:47:55,441] jepsen worker 31 - jepsen.util 4621 :ok :read [10 36] INFO [2022-01-10 05:47:55,441] jepsen worker 31 - jepsen.util 4621 :invoke :write [10 12] INFO [2022-01-10 05:47:55,482] jepsen worker 21 - jepsen.util 4431 :info :write [7 35] :RequestTimeoutException INFO [2022-01-10 05:47:55,482] jepsen worker 21 - jepsen.util 4521 :invoke :write [7 14] INFO [2022-01-10 05:47:55,535] jepsen worker 3 - jepsen.util 4233 :info :write [1 39] :RequestTimeoutException INFO [2022-01-10 05:47:55,535] jepsen worker 3 - jepsen.util 4323 :invoke :write [1 35] INFO [2022-01-10 05:47:55,626] jepsen worker 14 - jepsen.util 5324 :info :write [4 44] :RequestTimeoutException INFO [2022-01-10 05:47:55,627] jepsen worker 14 - jepsen.util 5414 :invoke :read [4 nil] INFO [2022-01-10 05:47:55,656] jepsen worker 14 - jepsen.util 5414 :ok :read [4 3] INFO [2022-01-10 05:47:55,656] jepsen worker 14 - jepsen.util 5414 :invoke :read [4 nil] INFO [2022-01-10 05:47:55,671] jepsen worker 65 - jepsen.util 4025 :info :write [21 34] :RequestTimeoutException INFO [2022-01-10 05:47:55,672] jepsen worker 65 - jepsen.util 4115 :invoke :read [21 nil] INFO [2022-01-10 05:47:55,680] jepsen worker 14 - jepsen.util 5414 :ok :read [4 3] INFO [2022-01-10 05:47:55,681] jepsen worker 14 - jepsen.util 5414 :invoke :read [4 nil] INFO [2022-01-10 05:47:55,711] jepsen worker 44 - jepsen.util 4454 :info :write [14 21] :RequestTimeoutException INFO [2022-01-10 05:47:55,712] jepsen worker 44 - jepsen.util 4544 :invoke :read [14 nil] INFO [2022-01-10 05:47:55,741] jepsen worker 14 - jepsen.util 5414 :ok :read [4 3] INFO [2022-01-10 05:47:55,742] jepsen worker 14 - jepsen.util 5414 :invoke :read [4 nil] INFO [2022-01-10 05:47:55,770] jepsen worker 14 - jepsen.util 5414 :ok :read [4 3] INFO [2022-01-10 05:47:55,770] jepsen worker 14 - jepsen.util 5414 :invoke :read [4 nil] INFO [2022-01-10 05:47:55,802] jepsen worker 14 - jepsen.util 5414 :ok :read [4 3] INFO [2022-01-10 05:47:55,803] jepsen worker 14 - jepsen.util 5414 :invoke :read [4 nil] INFO [2022-01-10 05:47:55,815] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5555555555555556}, :ns_1@172.23.107.227 {:progress 0.375366568914956}, :ns_1@172.23.105.187 {:progress 0.375366568914956}, :ns_1@172.23.107.228 {:progress 0.5294117647058824}} INFO [2022-01-10 05:47:55,815] jepsen worker nemesis - couchbase.util Rebalance check count: 623 INFO [2022-01-10 05:47:55,815] jepsen worker nemesis - couchbase.util Rebalance stuck count: 526 INFO [2022-01-10 05:47:55,830] jepsen worker 14 - jepsen.util 5414 :ok :read [4 3] INFO [2022-01-10 05:47:55,831] jepsen worker 14 - jepsen.util 5414 :invoke :read [4 nil] INFO [2022-01-10 05:47:55,854] jepsen worker 14 - jepsen.util 5414 :ok :read [4 3] INFO [2022-01-10 05:47:55,854] jepsen worker 14 - jepsen.util 5414 :invoke :read [4 nil] INFO [2022-01-10 05:47:55,872] jepsen worker 18 - jepsen.util 4248 :info :write [6 49] :RequestTimeoutException INFO [2022-01-10 05:47:55,872] jepsen worker 18 - jepsen.util 4338 :invoke :read [6 nil] INFO [2022-01-10 05:47:55,879] jepsen worker 14 - jepsen.util 5414 :ok :read [4 3] INFO [2022-01-10 05:47:55,880] jepsen worker 14 - jepsen.util 5414 :invoke :read [4 nil] INFO [2022-01-10 05:47:55,904] jepsen worker 14 - jepsen.util 5414 :ok :read [4 3] INFO [2022-01-10 05:47:55,904] jepsen worker 14 - jepsen.util 5414 :invoke :write [4 33] INFO [2022-01-10 05:47:55,911] jepsen worker 80 - jepsen.util 5390 :info :write [26 20] :RequestTimeoutException INFO [2022-01-10 05:47:55,912] jepsen worker 80 - jepsen.util 5480 :invoke :write [26 2] INFO [2022-01-10 05:47:56,195] jepsen worker 36 - jepsen.util 3186 :info :write [12 35] :RequestTimeoutException INFO [2022-01-10 05:47:56,195] jepsen worker 27 - jepsen.util 4617 :info :write [9 7] :RequestTimeoutException INFO [2022-01-10 05:47:56,196] jepsen worker 36 - jepsen.util 3276 :invoke :write [12 30] INFO [2022-01-10 05:47:56,197] jepsen worker 27 - jepsen.util 4707 :invoke :write [9 22] INFO [2022-01-10 05:47:56,822] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5614035087719298}, :ns_1@172.23.107.227 {:progress 0.3812316715542522}, :ns_1@172.23.105.187 {:progress 0.3812316715542522}, :ns_1@172.23.107.228 {:progress 0.5352941176470588}} INFO [2022-01-10 05:47:56,822] jepsen worker nemesis - couchbase.util Rebalance check count: 624 INFO [2022-01-10 05:47:56,822] jepsen worker nemesis - couchbase.util Rebalance stuck count: 526 INFO [2022-01-10 05:47:56,830] jepsen worker 65 - jepsen.util 4115 :ok :read [21 7] INFO [2022-01-10 05:47:56,830] jepsen worker 65 - jepsen.util 4115 :invoke :read [21 nil] INFO [2022-01-10 05:47:56,916] jepsen worker 23 - jepsen.util 4433 :info :write [7 7] :RequestTimeoutException INFO [2022-01-10 05:47:56,921] jepsen worker 23 - jepsen.util 4523 :invoke :write [7 3] INFO [2022-01-10 05:47:56,953] jepsen worker 46 - jepsen.util 4546 :ok :read [15 23] INFO [2022-01-10 05:47:56,953] jepsen worker 46 - jepsen.util 4546 :invoke :write [15 3] INFO [2022-01-10 05:47:57,038] jepsen worker 18 - jepsen.util 4338 :ok :read [6 25] INFO [2022-01-10 05:47:57,040] jepsen worker 18 - jepsen.util 4338 :invoke :write [6 42] INFO [2022-01-10 05:47:57,041] jepsen worker 48 - jepsen.util 4188 :info :write [16 41] :RequestTimeoutException INFO [2022-01-10 05:47:57,042] jepsen worker 48 - jepsen.util 4278 :invoke :read [16 nil] INFO [2022-01-10 05:47:57,059] jepsen worker 47 - jepsen.util 4637 :ok :read [15 23] INFO [2022-01-10 05:47:57,060] jepsen worker 47 - jepsen.util 4637 :invoke :read [15 nil] INFO [2022-01-10 05:47:57,094] jepsen worker 33 - jepsen.util 4623 :ok :read [11 21] INFO [2022-01-10 05:47:57,094] jepsen worker 33 - jepsen.util 4623 :invoke :read [11 nil] INFO [2022-01-10 05:47:57,115] jepsen worker 87 - jepsen.util 4317 :info :write [29 44] :RequestTimeoutException INFO [2022-01-10 05:47:57,116] jepsen worker 87 - jepsen.util 4407 :invoke :read [29 nil] INFO [2022-01-10 05:47:57,129] jepsen worker 34 - jepsen.util 4264 :ok :read [11 21] INFO [2022-01-10 05:47:57,130] jepsen worker 34 - jepsen.util 4264 :invoke :read [11 nil] INFO [2022-01-10 05:47:57,391] jepsen worker 65 - jepsen.util 4115 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:57,391] jepsen worker 65 - jepsen.util 4115 :invoke :write [21 6] INFO [2022-01-10 05:47:57,486] jepsen worker 29 - jepsen.util 4169 :info :write [9 24] :RequestTimeoutException INFO [2022-01-10 05:47:57,487] jepsen worker 29 - jepsen.util 4259 :invoke :write [9 0] INFO [2022-01-10 05:47:57,541] jepsen worker 74 - jepsen.util 4934 :ok :read [24 18] INFO [2022-01-10 05:47:57,541] jepsen worker 74 - jepsen.util 4934 :invoke :read [24 nil] INFO [2022-01-10 05:47:57,572] jepsen worker 62 - jepsen.util 4742 :info :write [20 10] :RequestTimeoutException INFO [2022-01-10 05:47:57,572] jepsen worker 62 - jepsen.util 4832 :invoke :write [20 38] INFO [2022-01-10 05:47:57,576] jepsen worker 19 - jepsen.util 4519 :info :write [6 20] :RequestTimeoutException INFO [2022-01-10 05:47:57,576] jepsen worker 19 - jepsen.util 4609 :invoke :read [6 nil] INFO [2022-01-10 05:47:57,626] jepsen worker 49 - jepsen.util 4099 :info :write [16 48] :RequestTimeoutException INFO [2022-01-10 05:47:57,626] jepsen worker 49 - jepsen.util 4189 :invoke :write [16 21] INFO [2022-01-10 05:47:57,816] jepsen worker 16 - jepsen.util 4426 :info :write [5 1] :RequestTimeoutException INFO [2022-01-10 05:47:57,817] jepsen worker 16 - jepsen.util 4516 :invoke :read [5 nil] INFO [2022-01-10 05:47:57,827] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5614035087719298}, :ns_1@172.23.107.227 {:progress 0.3812316715542522}, :ns_1@172.23.105.187 {:progress 0.3812316715542522}, :ns_1@172.23.107.228 {:progress 0.5352941176470588}} INFO [2022-01-10 05:47:57,828] jepsen worker nemesis - couchbase.util Rebalance check count: 625 INFO [2022-01-10 05:47:57,828] jepsen worker nemesis - couchbase.util Rebalance stuck count: 527 INFO [2022-01-10 05:47:57,929] jepsen worker 47 - jepsen.util 4637 :fail :read [15 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:57,929] jepsen worker 33 - jepsen.util 4623 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:57,929] jepsen worker 44 - jepsen.util 4544 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:57,930] jepsen worker 47 - jepsen.util 4637 :invoke :write [15 22] INFO [2022-01-10 05:47:57,930] jepsen worker 34 - jepsen.util 4264 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:47:57,931] jepsen worker 33 - jepsen.util 4623 :invoke :write [11 1] INFO [2022-01-10 05:47:57,931] jepsen worker 44 - jepsen.util 4544 :invoke :write [14 5] INFO [2022-01-10 05:47:57,932] jepsen worker 34 - jepsen.util 4264 :invoke :read [11 nil] INFO [2022-01-10 05:47:57,964] jepsen worker 74 - jepsen.util 4934 :ok :read [24 18] INFO [2022-01-10 05:47:57,965] jepsen worker 74 - jepsen.util 4934 :invoke :write [24 41] INFO [2022-01-10 05:47:57,997] jepsen worker 34 - jepsen.util 4264 :ok :read [11 21] INFO [2022-01-10 05:47:57,998] jepsen worker 34 - jepsen.util 4264 :invoke :read [11 nil] INFO [2022-01-10 05:47:58,044] jepsen worker 55 - jepsen.util 4915 :info :write [18 26] :RequestTimeoutException INFO [2022-01-10 05:47:58,045] jepsen worker 55 - jepsen.util 5005 :invoke :read [18 nil] INFO [2022-01-10 05:47:58,286] jepsen worker 66 - jepsen.util 4116 :info :write [22 23] :RequestTimeoutException INFO [2022-01-10 05:47:58,287] jepsen worker 66 - jepsen.util 4206 :invoke :write [22 24] INFO [2022-01-10 05:47:58,295] jepsen worker 77 - jepsen.util 4847 :info :write [25 1] :RequestTimeoutException INFO [2022-01-10 05:47:58,295] jepsen worker 77 - jepsen.util 4937 :invoke :write [25 22] INFO [2022-01-10 05:47:58,320] jepsen worker 35 - jepsen.util 4085 :info :write [11 19] :RequestTimeoutException INFO [2022-01-10 05:47:58,321] jepsen worker 35 - jepsen.util 4175 :invoke :write [11 48] INFO [2022-01-10 05:47:58,361] jepsen worker 75 - jepsen.util 4125 :info :write [25 47] :RequestTimeoutException INFO [2022-01-10 05:47:58,362] jepsen worker 75 - jepsen.util 4215 :invoke :write [25 40] INFO [2022-01-10 05:47:58,370] jepsen worker 6 - jepsen.util 4326 :info :write [2 43] :RequestTimeoutException INFO [2022-01-10 05:47:58,371] jepsen worker 6 - jepsen.util 4416 :invoke :read [2 nil] INFO [2022-01-10 05:47:58,386] jepsen worker 60 - jepsen.util 4650 :info :write [20 0] :RequestTimeoutException INFO [2022-01-10 05:47:58,386] jepsen worker 60 - jepsen.util 4740 :invoke :write [20 38] INFO [2022-01-10 05:47:58,389] jepsen worker 73 - jepsen.util 4123 :info :write [24 24] :RequestTimeoutException INFO [2022-01-10 05:47:58,390] jepsen worker 73 - jepsen.util 4213 :invoke :read [24 nil] INFO [2022-01-10 05:47:58,584] jepsen worker 11 - jepsen.util 4241 :info :write [3 12] :SyncWriteAmbiguous INFO [2022-01-10 05:47:58,585] jepsen worker 11 - jepsen.util 4331 :invoke :read [3 nil] INFO [2022-01-10 05:47:58,691] jepsen worker 38 - jepsen.util 3998 :ok :read [12 14] INFO [2022-01-10 05:47:58,692] jepsen worker 38 - jepsen.util 3998 :invoke :read [12 nil] INFO [2022-01-10 05:47:58,711] jepsen worker 63 - jepsen.util 4113 :info :write [21 13] :RequestTimeoutException INFO [2022-01-10 05:47:58,711] jepsen worker 63 - jepsen.util 4203 :invoke :read [21 nil] INFO [2022-01-10 05:47:58,834] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5614035087719298}, :ns_1@172.23.107.227 {:progress 0.3812316715542522}, :ns_1@172.23.105.187 {:progress 0.3812316715542522}, :ns_1@172.23.107.228 {:progress 0.5352941176470588}} INFO [2022-01-10 05:47:58,834] jepsen worker nemesis - couchbase.util Rebalance check count: 626 INFO [2022-01-10 05:47:58,834] jepsen worker nemesis - couchbase.util Rebalance stuck count: 528 INFO [2022-01-10 05:47:58,891] jepsen worker 8 - jepsen.util 4508 :info :write [2 18] :RequestTimeoutException INFO [2022-01-10 05:47:58,892] jepsen worker 8 - jepsen.util 4598 :invoke :write [2 43] INFO [2022-01-10 05:47:58,905] jepsen worker 43 - jepsen.util 4363 :info :write [14 12] :RequestTimeoutException INFO [2022-01-10 05:47:58,906] jepsen worker 43 - jepsen.util 4453 :invoke :write [14 23] INFO [2022-01-10 05:47:58,966] jepsen worker 88 - jepsen.util 4408 :info :write [29 15] :RequestTimeoutException INFO [2022-01-10 05:47:58,966] jepsen worker 88 - jepsen.util 4498 :invoke :write [29 15] INFO [2022-01-10 05:47:59,040] jepsen worker 40 - jepsen.util 4000 :info :write [13 15] :RequestTimeoutException INFO [2022-01-10 05:47:59,040] jepsen worker 40 - jepsen.util 4090 :invoke :write [13 46] INFO [2022-01-10 05:47:59,057] jepsen worker 81 - jepsen.util 4851 :info :write [27 8] :RequestTimeoutException INFO [2022-01-10 05:47:59,057] jepsen worker 81 - jepsen.util 4941 :invoke :write [27 17] INFO [2022-01-10 05:47:59,059] jepsen worker 1 - jepsen.util 4411 :info :write [0 6] :RequestTimeoutException INFO [2022-01-10 05:47:59,060] jepsen worker 1 - jepsen.util 4501 :invoke :read [0 nil] INFO [2022-01-10 05:47:59,070] jepsen worker 34 - jepsen.util 4264 :ok :read [11 21] INFO [2022-01-10 05:47:59,071] jepsen worker 34 - jepsen.util 4264 :invoke :read [11 nil] INFO [2022-01-10 05:47:59,249] jepsen worker 6 - jepsen.util 4416 :ok :read [2 14] INFO [2022-01-10 05:47:59,249] jepsen worker 6 - jepsen.util 4416 :invoke :read [2 nil] INFO [2022-01-10 05:47:59,396] jepsen worker 64 - jepsen.util 4384 :info :write [21 25] :RequestTimeoutException INFO [2022-01-10 05:47:59,397] jepsen worker 64 - jepsen.util 4474 :invoke :write [21 21] INFO [2022-01-10 05:47:59,477] jepsen worker 34 - jepsen.util 4264 :ok :read [11 21] INFO [2022-01-10 05:47:59,478] jepsen worker 34 - jepsen.util 4264 :invoke :write [11 13] INFO [2022-01-10 05:47:59,496] jepsen worker 24 - jepsen.util 3984 :info :write [8 18] :RequestTimeoutException INFO [2022-01-10 05:47:59,496] jepsen worker 24 - jepsen.util 4074 :invoke :write [8 27] INFO [2022-01-10 05:47:59,651] jepsen worker 70 - jepsen.util 4390 :info :write [23 30] :RequestTimeoutException INFO [2022-01-10 05:47:59,652] jepsen worker 70 - jepsen.util 4480 :invoke :write [23 34] INFO [2022-01-10 05:47:59,682] jepsen worker 6 - jepsen.util 4416 :ok :read [2 14] INFO [2022-01-10 05:47:59,682] jepsen worker 6 - jepsen.util 4416 :invoke :write [2 3] INFO [2022-01-10 05:47:59,692] jepsen worker 69 - jepsen.util 4029 :info :write [23 21] :RequestTimeoutException INFO [2022-01-10 05:47:59,693] jepsen worker 69 - jepsen.util 4119 :invoke :read [23 nil] INFO [2022-01-10 05:47:59,839] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5672514619883041}, :ns_1@172.23.107.227 {:progress 0.3870967741935484}, :ns_1@172.23.105.187 {:progress 0.3870967741935484}, :ns_1@172.23.107.228 {:progress 0.5411764705882354}} INFO [2022-01-10 05:47:59,839] jepsen worker nemesis - couchbase.util Rebalance check count: 627 INFO [2022-01-10 05:47:59,839] jepsen worker nemesis - couchbase.util Rebalance stuck count: 528 INFO [2022-01-10 05:47:59,927] jepsen worker 45 - jepsen.util 4815 :info :write [15 1] :RequestTimeoutException INFO [2022-01-10 05:47:59,927] jepsen worker 45 - jepsen.util 4905 :invoke :read [15 nil] INFO [2022-01-10 05:47:59,940] jepsen worker 25 - jepsen.util 4075 :info :write [8 14] :RequestTimeoutException INFO [2022-01-10 05:47:59,941] jepsen worker 25 - jepsen.util 4165 :invoke :read [8 nil] INFO [2022-01-10 05:48:00,253] jepsen worker 38 - jepsen.util 3998 :ok :read [12 14] INFO [2022-01-10 05:48:00,253] jepsen worker 38 - jepsen.util 3998 :invoke :write [12 24] INFO [2022-01-10 05:48:00,395] jepsen worker 71 - jepsen.util 3941 :info :write [23 7] :RequestTimeoutException INFO [2022-01-10 05:48:00,396] jepsen worker 71 - jepsen.util 4031 :invoke :read [23 nil] INFO [2022-01-10 05:48:00,447] jepsen worker 45 - jepsen.util 4905 :ok :read [15 23] INFO [2022-01-10 05:48:00,448] jepsen worker 45 - jepsen.util 4905 :invoke :write [15 3] INFO [2022-01-10 05:48:00,582] jepsen worker 39 - jepsen.util 4719 :info :write [13 3] :RequestTimeoutException INFO [2022-01-10 05:48:00,582] jepsen worker 13 - jepsen.util 5143 :info :write [4 31] :RequestTimeoutException INFO [2022-01-10 05:48:00,582] jepsen worker 13 - jepsen.util 5233 :invoke :read [4 nil] INFO [2022-01-10 05:48:00,583] jepsen worker 39 - jepsen.util 4809 :invoke :write [13 36] INFO [2022-01-10 05:48:00,667] jepsen worker 12 - jepsen.util 5052 :ok :read [4 3] INFO [2022-01-10 05:48:00,667] jepsen worker 12 - jepsen.util 5052 :invoke :read [4 nil] INFO [2022-01-10 05:48:00,691] jepsen worker 52 - jepsen.util 5542 :info :write [17 28] :RequestTimeoutException INFO [2022-01-10 05:48:00,691] jepsen worker 52 - jepsen.util 5632 :invoke :read [17 nil] WARN [2022-01-10 05:48:00,715] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17474289,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18964198,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9280594,"encode_duration_us":16276,"operation_id":"0x8715","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10007967},{"operation_name":"upsert","last_dispatch_duration_us":9578926,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17900649,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":11138457,"encode_duration_us":16620,"operation_id":"0x85f4","timeout_ms":10000,"last_server_duration_us":9474800,"total_duration_us":10059234},{"operation_name":"upsert","last_dispatch_duration_us":18487201,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":26329580,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":10606688,"encode_duration_us":18628,"operation_id":"0x855c","timeout_ms":10000,"last_server_duration_us":9044797,"total_duration_us":10008536}],"total_count":3}} WARN [2022-01-10 05:48:00,715] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17661121,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":17661121,"total_server_duration_us":1249039,"operation_id":"0x8757","timeout_ms":30000,"last_server_duration_us":1249039,"total_duration_us":17663351},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":18034,"operation_id":"0x88fa","timeout_ms":10000,"total_duration_us":10009876},{"operation_name":"upsert","last_dispatch_duration_us":9037323,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9037323,"total_server_duration_us":20282,"encode_duration_us":17082,"operation_id":"0x88ca","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10006422},{"operation_name":"upsert","last_dispatch_duration_us":1247201,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9417200,"total_server_duration_us":383975,"encode_duration_us":18398,"operation_id":"0x8841","timeout_ms":10000,"last_server_duration_us":74510,"total_duration_us":10005533},{"operation_name":"upsert","last_dispatch_duration_us":1129302,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10317799,"total_server_duration_us":431825,"encode_duration_us":17083,"operation_id":"0x87f0","timeout_ms":10000,"last_server_duration_us":99346,"total_duration_us":10004643},{"operation_name":"upsert","last_dispatch_duration_us":37425,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":1590553,"total_server_duration_us":1179985,"encode_duration_us":21014,"operation_id":"0x88a6","timeout_ms":10000,"last_server_duration_us":25625,"total_duration_us":10003816},{"operation_name":"upsert","last_dispatch_duration_us":1012562,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10633362,"total_server_duration_us":559981,"encode_duration_us":18654,"operation_id":"0x8907","timeout_ms":10000,"last_server_duration_us":80255,"total_duration_us":10003516},{"operation_name":"upsert","last_dispatch_duration_us":9102204,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":17575734,"total_server_duration_us":68526,"encode_duration_us":16790,"operation_id":"0x87b9","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10003441},{"operation_name":"upsert","last_dispatch_duration_us":12014143,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":12014143,"total_server_duration_us":85157,"encode_duration_us":19059,"operation_id":"0x8866","timeout_ms":10000,"last_server_duration_us":85157,"total_duration_us":10003374},{"operation_name":"upsert","last_dispatch_duration_us":504680,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10267610,"total_server_duration_us":864284,"encode_duration_us":18188,"operation_id":"0x8933","timeout_ms":10000,"last_server_duration_us":165304,"total_duration_us":10003127}],"total_count":14}} INFO [2022-01-10 05:48:00,792] jepsen worker 17 - jepsen.util 4157 :info :write [5 33] :RequestTimeoutException INFO [2022-01-10 05:48:00,792] jepsen worker 17 - jepsen.util 4247 :invoke :write [5 38] INFO [2022-01-10 05:48:00,825] jepsen worker 20 - jepsen.util 4070 :info :write [6 16] :RequestTimeoutException INFO [2022-01-10 05:48:00,825] jepsen worker 20 - jepsen.util 4160 :invoke :read [6 nil] INFO [2022-01-10 05:48:00,844] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5672514619883041}, :ns_1@172.23.107.227 {:progress 0.3870967741935484}, :ns_1@172.23.105.187 {:progress 0.3870967741935484}, :ns_1@172.23.107.228 {:progress 0.5411764705882354}} INFO [2022-01-10 05:48:00,844] jepsen worker nemesis - couchbase.util Rebalance check count: 628 INFO [2022-01-10 05:48:00,844] jepsen worker nemesis - couchbase.util Rebalance stuck count: 529 INFO [2022-01-10 05:48:00,885] jepsen worker 61 - jepsen.util 4651 :info :write [20 46] :RequestTimeoutException INFO [2022-01-10 05:48:00,886] jepsen worker 61 - jepsen.util 4741 :invoke :write [20 32] INFO [2022-01-10 05:48:00,956] jepsen worker 15 - jepsen.util 3885 :info :write [5 46] :RequestTimeoutException INFO [2022-01-10 05:48:00,956] jepsen worker 15 - jepsen.util 3975 :invoke :write [5 20] INFO [2022-01-10 05:48:01,030] jepsen worker 73 - jepsen.util 4213 :ok :read [24 18] INFO [2022-01-10 05:48:01,031] jepsen worker 73 - jepsen.util 4213 :invoke :read [24 nil] INFO [2022-01-10 05:48:01,132] jepsen worker 32 - jepsen.util 4622 :info :write [10 46] :RequestTimeoutException INFO [2022-01-10 05:48:01,132] jepsen worker 32 - jepsen.util 4712 :invoke :read [10 nil] INFO [2022-01-10 05:48:01,156] jepsen worker 50 - jepsen.util 4640 :info :write [16 8] :RequestTimeoutException INFO [2022-01-10 05:48:01,157] jepsen worker 50 - jepsen.util 4730 :invoke :write [16 16] INFO [2022-01-10 05:48:01,390] jepsen worker 22 - jepsen.util 4252 :info :write [7 41] :RequestTimeoutException INFO [2022-01-10 05:48:01,390] jepsen worker 22 - jepsen.util 4342 :invoke :read [7 nil] INFO [2022-01-10 05:48:01,396] jepsen worker 7 - jepsen.util 4687 :info :write [2 49] :RequestTimeoutException INFO [2022-01-10 05:48:01,397] jepsen worker 7 - jepsen.util 4777 :invoke :read [2 nil] INFO [2022-01-10 05:48:01,537] jepsen worker 73 - jepsen.util 4213 :ok :read [24 18] INFO [2022-01-10 05:48:01,537] jepsen worker 73 - jepsen.util 4213 :invoke :read [24 nil] INFO [2022-01-10 05:48:01,598] jepsen worker 22 - jepsen.util 4342 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:48:01,599] jepsen worker 22 - jepsen.util 4342 :invoke :read [7 nil] INFO [2022-01-10 05:48:01,716] jepsen worker 73 - jepsen.util 4213 :ok :read [24 18] INFO [2022-01-10 05:48:01,717] jepsen worker 73 - jepsen.util 4213 :invoke :read [24 nil] INFO [2022-01-10 05:48:01,850] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5672514619883041}, :ns_1@172.23.107.227 {:progress 0.3870967741935484}, :ns_1@172.23.105.187 {:progress 0.3870967741935484}, :ns_1@172.23.107.228 {:progress 0.5411764705882354}} INFO [2022-01-10 05:48:01,850] jepsen worker nemesis - couchbase.util Rebalance check count: 629 INFO [2022-01-10 05:48:01,850] jepsen worker nemesis - couchbase.util Rebalance stuck count: 530 INFO [2022-01-10 05:48:02,011] jepsen worker 73 - jepsen.util 4213 :ok :read [24 18] INFO [2022-01-10 05:48:02,012] jepsen worker 73 - jepsen.util 4213 :invoke :write [24 49] INFO [2022-01-10 05:48:02,090] jepsen worker 0 - jepsen.util 4410 :info :write [0 10] :RequestTimeoutException INFO [2022-01-10 05:48:02,090] jepsen worker 0 - jepsen.util 4500 :invoke :write [0 2] INFO [2022-01-10 05:48:02,102] jepsen worker 5 - jepsen.util 4415 :info :write [1 16] :RequestTimeoutException INFO [2022-01-10 05:48:02,103] jepsen worker 5 - jepsen.util 4505 :invoke :read [1 nil] WARN [2022-01-10 05:48:02,232] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15748230,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":15748230,"total_server_duration_us":27,"operation_id":"0x8756","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":15750340},{"operation_name":"get","last_dispatch_duration_us":14837687,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":14837687,"total_server_duration_us":19,"operation_id":"0x8784","timeout_ms":30000,"last_server_duration_us":19,"total_duration_us":14839076},{"operation_name":"get","last_dispatch_duration_us":11242819,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11242819,"total_server_duration_us":9284,"operation_id":"0x87c4","timeout_ms":30000,"last_server_duration_us":9284,"total_duration_us":11244780},{"operation_name":"upsert","last_dispatch_duration_us":10137047,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10137047,"total_server_duration_us":9101143,"encode_duration_us":19798,"operation_id":"0x896d","timeout_ms":10000,"last_server_duration_us":9101143,"total_duration_us":10008820},{"operation_name":"upsert","last_dispatch_duration_us":12942860,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":12942860,"total_server_duration_us":20362,"encode_duration_us":17815,"operation_id":"0x87e6","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10007864},{"operation_name":"upsert","last_dispatch_duration_us":8571533,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11777956,"total_server_duration_us":443725,"encode_duration_us":21124,"operation_id":"0x8909","timeout_ms":10000,"last_server_duration_us":74648,"total_duration_us":10007118},{"operation_name":"upsert","last_dispatch_duration_us":10233325,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10499675,"total_server_duration_us":10305364,"encode_duration_us":22452,"operation_id":"0x894b","timeout_ms":10000,"last_server_duration_us":10198705,"total_duration_us":10006605},{"operation_name":"upsert","last_dispatch_duration_us":8600690,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":12014235,"total_server_duration_us":639757,"encode_duration_us":17025,"operation_id":"0x8917","timeout_ms":10000,"last_server_duration_us":89734,"total_duration_us":10006246},{"operation_name":"upsert","last_dispatch_duration_us":8238003,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":12252591,"total_server_duration_us":388443,"encode_duration_us":17906,"operation_id":"0x8899","timeout_ms":10000,"last_server_duration_us":67882,"total_duration_us":10005718},{"operation_name":"upsert","last_dispatch_duration_us":568017,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7602019,"total_server_duration_us":144060,"encode_duration_us":16531,"operation_id":"0x88c9","timeout_ms":10000,"last_server_duration_us":27673,"total_duration_us":10004137}],"total_count":20}} WARN [2022-01-10 05:48:02,233] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17028202,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":26279229,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9208330,"encode_duration_us":17477,"operation_id":"0x852f","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10006217},{"operation_name":"upsert","last_dispatch_duration_us":10137047,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10137047,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9101143,"encode_duration_us":19798,"operation_id":"0x896d","timeout_ms":10000,"last_server_duration_us":9101143,"total_duration_us":10008820},{"operation_name":"upsert","last_dispatch_duration_us":10233325,"last_local_id":"EFE9906200000006/0000000021E2C07D","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10499675,"last_local_socket":"172.23.104.79:44302","total_server_duration_us":10305364,"encode_duration_us":22452,"operation_id":"0x894b","timeout_ms":10000,"last_server_duration_us":10198705,"total_duration_us":10006605}],"total_count":3}} WARN [2022-01-10 05:48:02,317] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12809593,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":12809593,"total_server_duration_us":32,"operation_id":"0x8897","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":12810722},{"operation_name":"upsert","last_dispatch_duration_us":1122678,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":6083779,"total_server_duration_us":252613,"encode_duration_us":17162,"operation_id":"0x88d3","timeout_ms":10000,"last_server_duration_us":50068,"total_duration_us":10010156},{"operation_name":"upsert","last_dispatch_duration_us":549003,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7557700,"total_server_duration_us":2576905,"encode_duration_us":18443,"operation_id":"0x88d5","timeout_ms":10000,"last_server_duration_us":114454,"total_duration_us":10010154},{"operation_name":"upsert","last_dispatch_duration_us":976840,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6857537,"total_server_duration_us":1507169,"encode_duration_us":18530,"operation_id":"0x88c8","timeout_ms":10000,"last_server_duration_us":108723,"total_duration_us":10009749},{"operation_name":"upsert","last_dispatch_duration_us":308544,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":5849659,"total_server_duration_us":960834,"encode_duration_us":18364,"operation_id":"0x896c","timeout_ms":10000,"last_server_duration_us":116448,"total_duration_us":10006314},{"operation_name":"upsert","last_dispatch_duration_us":1058611,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4901749,"total_server_duration_us":973730,"encode_duration_us":21494,"operation_id":"0x89b5","timeout_ms":10000,"last_server_duration_us":250965,"total_duration_us":10006083},{"operation_name":"upsert","last_dispatch_duration_us":800890,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":5665755,"total_server_duration_us":357685,"encode_duration_us":16443,"operation_id":"0x88d6","timeout_ms":10000,"last_server_duration_us":96557,"total_duration_us":10006016},{"operation_name":"upsert","last_dispatch_duration_us":1250025,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":6247146,"total_server_duration_us":275582,"encode_duration_us":18234,"operation_id":"0x8906","timeout_ms":10000,"last_server_duration_us":82835,"total_duration_us":10005116},{"operation_name":"upsert","last_dispatch_duration_us":1357958,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":6671978,"total_server_duration_us":155837,"encode_duration_us":19764,"operation_id":"0x8936","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10004822},{"operation_name":"upsert","last_dispatch_duration_us":1565899,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":3893624,"total_server_duration_us":162681,"encode_duration_us":17656,"operation_id":"0x89ad","timeout_ms":10000,"last_server_duration_us":32743,"total_duration_us":10003204}],"total_count":21}} WARN [2022-01-10 05:48:02,317] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1465784,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":3715585,"total_server_duration_us":809065,"encode_duration_us":23927,"operation_id":"0x88a5","timeout_ms":10000,"last_server_duration_us":53254,"total_duration_us":10010355},{"operation_name":"upsert","last_dispatch_duration_us":807638,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4622272,"total_server_duration_us":366850,"encode_duration_us":28471,"operation_id":"0x8944","timeout_ms":10000,"last_server_duration_us":195205,"total_duration_us":10009892},{"operation_name":"upsert","last_dispatch_duration_us":1438115,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3317488,"total_server_duration_us":98288,"encode_duration_us":16255,"operation_id":"0x8818","timeout_ms":10000,"last_server_duration_us":27312,"total_duration_us":10009182},{"operation_name":"upsert","last_dispatch_duration_us":1204619,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5871050,"total_server_duration_us":545677,"encode_duration_us":17409,"operation_id":"0x8941","timeout_ms":10000,"last_server_duration_us":158018,"total_duration_us":10009022},{"operation_name":"upsert","last_dispatch_duration_us":1412405,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2475898,"total_server_duration_us":790639,"encode_duration_us":22053,"operation_id":"0x890c","timeout_ms":10000,"last_server_duration_us":657963,"total_duration_us":10006273},{"operation_name":"upsert","last_dispatch_duration_us":1921119,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5898207,"total_server_duration_us":666806,"encode_duration_us":17237,"operation_id":"0x899f","timeout_ms":10000,"last_server_duration_us":257941,"total_duration_us":10004667},{"operation_name":"upsert","last_dispatch_duration_us":28708,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":957058,"total_server_duration_us":577797,"encode_duration_us":16710,"operation_id":"0x886c","timeout_ms":10000,"last_server_duration_us":18417,"total_duration_us":10003902},{"operation_name":"upsert","last_dispatch_duration_us":33979,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":706095,"total_server_duration_us":404363,"encode_duration_us":16739,"operation_id":"0x8934","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10002927},{"operation_name":"upsert","last_dispatch_duration_us":32220,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":1886249,"total_server_duration_us":565800,"encode_duration_us":16830,"operation_id":"0x8876","timeout_ms":10000,"last_server_duration_us":20045,"total_duration_us":10002106},{"operation_name":"upsert","last_dispatch_duration_us":610166,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":610166,"total_server_duration_us":80255,"encode_duration_us":19692,"operation_id":"0x89a0","timeout_ms":10000,"last_server_duration_us":80255,"total_duration_us":10001237}],"total_count":13}} WARN [2022-01-10 05:48:02,327] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9040700,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":13304046,"total_server_duration_us":9143802,"encode_duration_us":18620,"operation_id":"0x87ef","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10010249},{"operation_name":"upsert","last_dispatch_duration_us":675550,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":4437482,"total_server_duration_us":407019,"encode_duration_us":16753,"operation_id":"0x898e","timeout_ms":10000,"last_server_duration_us":74234,"total_duration_us":10009750},{"operation_name":"upsert","last_dispatch_duration_us":1640622,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":7815329,"total_server_duration_us":877601,"encode_duration_us":16798,"operation_id":"0x89db","timeout_ms":10000,"last_server_duration_us":351620,"total_duration_us":10008518},{"operation_name":"upsert","last_dispatch_duration_us":8751326,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9828844,"total_server_duration_us":348619,"encode_duration_us":18143,"operation_id":"0x8872","timeout_ms":10000,"last_server_duration_us":130813,"total_duration_us":10007723},{"operation_name":"upsert","last_dispatch_duration_us":42304,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":7922014,"total_server_duration_us":291473,"encode_duration_us":18055,"operation_id":"0x89a6","timeout_ms":10000,"last_server_duration_us":26685,"total_duration_us":10005994},{"operation_name":"upsert","last_dispatch_duration_us":2642097,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2642097,"total_server_duration_us":85157,"encode_duration_us":27802,"operation_id":"0x88d4","timeout_ms":10000,"last_server_duration_us":85157,"total_duration_us":10005905},{"operation_name":"upsert","last_dispatch_duration_us":10008722,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11083905,"total_server_duration_us":259066,"encode_duration_us":18839,"operation_id":"0x887e","timeout_ms":10000,"last_server_duration_us":81828,"total_duration_us":10005236},{"operation_name":"upsert","last_dispatch_duration_us":1564859,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":9734565,"total_server_duration_us":1611476,"encode_duration_us":29990,"operation_id":"0x894d","timeout_ms":10000,"last_server_duration_us":269271,"total_duration_us":10004465},{"operation_name":"upsert","last_dispatch_duration_us":471467,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":7400822,"total_server_duration_us":705033,"encode_duration_us":16771,"operation_id":"0x8978","timeout_ms":10000,"last_server_duration_us":112969,"total_duration_us":10003138},{"operation_name":"upsert","last_dispatch_duration_us":2342588,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":3346678,"total_server_duration_us":285936,"encode_duration_us":17356,"operation_id":"0x8898","timeout_ms":10000,"last_server_duration_us":84574,"total_duration_us":10001488}],"total_count":25}} INFO [2022-01-10 05:48:02,394] jepsen worker 58 - jepsen.util 4288 :info :write [19 45] :RequestTimeoutException INFO [2022-01-10 05:48:02,395] jepsen worker 58 - jepsen.util 4378 :invoke :write [19 25] INFO [2022-01-10 05:48:02,399] jepsen worker 31 - jepsen.util 4621 :ok :write [10 12] INFO [2022-01-10 05:48:02,400] jepsen worker 31 - jepsen.util 4621 :invoke :write [10 24] WARN [2022-01-10 05:48:02,427] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":27068967,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":34634696,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9847164,"encode_duration_us":19713,"operation_id":"0x83a4","timeout_ms":10000,"last_server_duration_us":9180063,"total_duration_us":10002486},{"operation_name":"upsert","last_dispatch_duration_us":19729649,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19729649,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9125641,"encode_duration_us":16201,"operation_id":"0x8520","timeout_ms":10000,"last_server_duration_us":9125641,"total_duration_us":10004431},{"operation_name":"upsert","last_dispatch_duration_us":19475889,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19475889,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9279658,"encode_duration_us":17020,"operation_id":"0x859a","timeout_ms":10000,"last_server_duration_us":9279658,"total_duration_us":10002547},{"operation_name":"upsert","last_dispatch_duration_us":19204432,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19204432,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9238908,"encode_duration_us":16145,"operation_id":"0x8552","timeout_ms":10000,"last_server_duration_us":9238908,"total_duration_us":10008317},{"operation_name":"upsert","last_dispatch_duration_us":9040700,"last_local_id":"EFE9906200000004/000000008B42EFB5","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":13304046,"last_local_socket":"172.23.104.79:44292","total_server_duration_us":9143802,"encode_duration_us":18620,"operation_id":"0x87ef","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10010249}],"total_count":5}} INFO [2022-01-10 05:48:02,463] jepsen worker 87 - jepsen.util 4407 :ok :read [29 0] INFO [2022-01-10 05:48:02,463] jepsen worker 87 - jepsen.util 4407 :invoke :read [29 nil] INFO [2022-01-10 05:48:02,464] jepsen worker 22 - jepsen.util 4342 :ok :read [7 :nil] INFO [2022-01-10 05:48:02,465] jepsen worker 22 - jepsen.util 4342 :invoke :write [7 31] INFO [2022-01-10 05:48:02,478] jepsen worker 55 - jepsen.util 5005 :ok :read [18 14] INFO [2022-01-10 05:48:02,481] jepsen worker 55 - jepsen.util 5005 :invoke :write [18 3] INFO [2022-01-10 05:48:02,493] jepsen worker 84 - jepsen.util 4314 :ok :read [28 45] INFO [2022-01-10 05:48:02,498] jepsen worker 84 - jepsen.util 4314 :invoke :read [28 nil] INFO [2022-01-10 05:48:02,511] jepsen worker 67 - jepsen.util 4297 :ok :read [22 41] INFO [2022-01-10 05:48:02,512] jepsen worker 67 - jepsen.util 4297 :invoke :read [22 nil] WARN [2022-01-10 05:48:02,517] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14087610,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":14087610,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":9072420,"encode_duration_us":19980,"operation_id":"0x8845","timeout_ms":10000,"last_server_duration_us":9072420,"total_duration_us":10002558},{"operation_name":"upsert","last_dispatch_duration_us":9066098,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9947652,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9424713,"encode_duration_us":17666,"operation_id":"0x87c3","timeout_ms":10000,"last_server_duration_us":9037366,"total_duration_us":10003581}],"total_count":2}} INFO [2022-01-10 05:48:02,522] jepsen worker 32 - jepsen.util 4712 :ok :read [10 12] INFO [2022-01-10 05:48:02,522] jepsen worker 32 - jepsen.util 4712 :invoke :write [10 12] WARN [2022-01-10 05:48:02,534] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":716191,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":5223200,"total_server_duration_us":753936,"encode_duration_us":17079,"operation_id":"0x8935","timeout_ms":10000,"last_server_duration_us":66694,"total_duration_us":10007770},{"operation_name":"upsert","last_dispatch_duration_us":871233,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":2661723,"total_server_duration_us":226343,"encode_duration_us":16399,"operation_id":"0x89d9","timeout_ms":10000,"last_server_duration_us":100753,"total_duration_us":10006169},{"operation_name":"upsert","last_dispatch_duration_us":1114247,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":6924137,"total_server_duration_us":1464236,"encode_duration_us":20038,"operation_id":"0x8908","timeout_ms":10000,"last_server_duration_us":56770,"total_duration_us":10005798},{"operation_name":"upsert","last_dispatch_duration_us":8131228,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":8669202,"total_server_duration_us":297871,"encode_duration_us":16293,"operation_id":"0x8819","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10005589},{"operation_name":"upsert","last_dispatch_duration_us":1106788,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":6164263,"total_server_duration_us":870510,"encode_duration_us":17242,"operation_id":"0x8937","timeout_ms":10000,"last_server_duration_us":89435,"total_duration_us":10005362},{"operation_name":"upsert","last_dispatch_duration_us":727927,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":6930010,"total_server_duration_us":1807872,"encode_duration_us":16640,"operation_id":"0x89b4","timeout_ms":10000,"last_server_duration_us":227859,"total_duration_us":10005021},{"operation_name":"upsert","last_dispatch_duration_us":1204257,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8071614,"total_server_duration_us":1585766,"encode_duration_us":19538,"operation_id":"0x890d","timeout_ms":10000,"last_server_duration_us":150873,"total_duration_us":10002086},{"operation_name":"upsert","last_dispatch_duration_us":1206492,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8293374,"total_server_duration_us":1546356,"encode_duration_us":18001,"operation_id":"0x890a","timeout_ms":10000,"last_server_duration_us":74510,"total_duration_us":10002006},{"operation_name":"upsert","last_dispatch_duration_us":9088893,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9088893,"total_server_duration_us":9038428,"encode_duration_us":16475,"operation_id":"0x8870","timeout_ms":10000,"last_server_duration_us":9038428,"total_duration_us":9092468},{"operation_name":"get","last_dispatch_duration_us":1645945,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":1645945,"total_server_duration_us":12311,"operation_id":"0x89a3","timeout_ms":30000,"last_server_duration_us":12311,"total_duration_us":1747978}],"total_count":14}} INFO [2022-01-10 05:48:02,573] jepsen worker 87 - jepsen.util 4407 :ok :read [29 0] INFO [2022-01-10 05:48:02,573] jepsen worker 87 - jepsen.util 4407 :invoke :write [29 21] INFO [2022-01-10 05:48:02,855] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5730994152046784}, :ns_1@172.23.107.227 {:progress 0.3929618768328446}, :ns_1@172.23.105.187 {:progress 0.3929618768328446}, :ns_1@172.23.107.228 {:progress 0.5470588235294118}} INFO [2022-01-10 05:48:02,855] jepsen worker nemesis - couchbase.util Rebalance check count: 630 INFO [2022-01-10 05:48:02,855] jepsen worker nemesis - couchbase.util Rebalance stuck count: 530 INFO [2022-01-10 05:48:02,870] jepsen worker 59 - jepsen.util 4289 :info :write [19 7] :RequestTimeoutException INFO [2022-01-10 05:48:02,871] jepsen worker 59 - jepsen.util 4379 :invoke :read [19 nil] INFO [2022-01-10 05:48:03,176] jepsen worker 42 - jepsen.util 4092 :info :write [14 48] :RequestTimeoutException INFO [2022-01-10 05:48:03,177] jepsen worker 42 - jepsen.util 4182 :invoke :read [14 nil] INFO [2022-01-10 05:48:03,295] jepsen worker 76 - jepsen.util 3766 :info :write [25 4] :RequestTimeoutException INFO [2022-01-10 05:48:03,296] jepsen worker 76 - jepsen.util 3856 :invoke :read [25 nil] INFO [2022-01-10 05:48:03,314] jepsen worker 79 - jepsen.util 5209 :ok :read [26 10] INFO [2022-01-10 05:48:03,315] jepsen worker 79 - jepsen.util 5209 :invoke :read [26 nil] INFO [2022-01-10 05:48:03,318] jepsen worker 51 - jepsen.util 5271 :ok :read [17 41] INFO [2022-01-10 05:48:03,318] jepsen worker 51 - jepsen.util 5271 :invoke :write [17 24] INFO [2022-01-10 05:48:03,341] jepsen worker 37 - jepsen.util 3997 :info :write [12 3] :SyncWriteAmbiguous INFO [2022-01-10 05:48:03,342] jepsen worker 37 - jepsen.util 4087 :invoke :write [12 35] INFO [2022-01-10 05:48:03,350] jepsen worker 71 - jepsen.util 4031 :ok :read [23 26] INFO [2022-01-10 05:48:03,350] jepsen worker 71 - jepsen.util 4031 :invoke :write [23 18] INFO [2022-01-10 05:48:03,370] jepsen worker 79 - jepsen.util 5209 :ok :read [26 10] INFO [2022-01-10 05:48:03,375] jepsen worker 79 - jepsen.util 5209 :invoke :read [26 nil] INFO [2022-01-10 05:48:03,506] jepsen worker 72 - jepsen.util 4662 :info :write [24 39] :RequestTimeoutException INFO [2022-01-10 05:48:03,506] jepsen worker 72 - jepsen.util 4752 :invoke :read [24 nil] INFO [2022-01-10 05:48:03,521] jepsen worker 28 - jepsen.util 4168 :info :write [9 33] :RequestTimeoutException INFO [2022-01-10 05:48:03,522] jepsen worker 28 - jepsen.util 4258 :invoke :write [9 36] INFO [2022-01-10 05:48:03,576] jepsen worker 53 - jepsen.util 5363 :info :write [17 49] :RequestTimeoutException INFO [2022-01-10 05:48:03,577] jepsen worker 53 - jepsen.util 5453 :invoke :read [17 nil] INFO [2022-01-10 05:48:03,646] jepsen worker 79 - jepsen.util 5209 :ok :read [26 10] INFO [2022-01-10 05:48:03,647] jepsen worker 79 - jepsen.util 5209 :invoke :write [26 10] INFO [2022-01-10 05:48:03,670] jepsen worker 85 - jepsen.util 4765 :info :write [28 2] :RequestTimeoutException INFO [2022-01-10 05:48:03,671] jepsen worker 85 - jepsen.util 4855 :invoke :read [28 nil] INFO [2022-01-10 05:48:03,692] jepsen worker 2 - jepsen.util 4592 :info :write [0 32] :RequestTimeoutException INFO [2022-01-10 05:48:03,692] jepsen worker 2 - jepsen.util 4682 :invoke :read [0 nil] INFO [2022-01-10 05:48:03,712] jepsen worker 82 - jepsen.util 4492 :info :write [27 15] :RequestTimeoutException INFO [2022-01-10 05:48:03,712] jepsen worker 82 - jepsen.util 4582 :invoke :write [27 9] INFO [2022-01-10 05:48:03,740] jepsen worker 85 - jepsen.util 4855 :ok :read [28 45] INFO [2022-01-10 05:48:03,741] jepsen worker 85 - jepsen.util 4855 :invoke :read [28 nil] INFO [2022-01-10 05:48:03,838] jepsen worker 85 - jepsen.util 4855 :ok :read [28 45] INFO [2022-01-10 05:48:03,839] jepsen worker 85 - jepsen.util 4855 :invoke :write [28 32] INFO [2022-01-10 05:48:03,861] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5730994152046784}, :ns_1@172.23.107.227 {:progress 0.3929618768328446}, :ns_1@172.23.105.187 {:progress 0.3929618768328446}, :ns_1@172.23.107.228 {:progress 0.5470588235294118}} INFO [2022-01-10 05:48:03,861] jepsen worker nemesis - couchbase.util Rebalance check count: 631 INFO [2022-01-10 05:48:03,861] jepsen worker nemesis - couchbase.util Rebalance stuck count: 531 INFO [2022-01-10 05:48:03,951] jepsen worker 26 - jepsen.util 4166 :info :write [8 5] :RequestTimeoutException INFO [2022-01-10 05:48:03,951] jepsen worker 26 - jepsen.util 4256 :invoke :read [8 nil] INFO [2022-01-10 05:48:04,040] jepsen worker 83 - jepsen.util 5033 :info :write [27 20] :RequestTimeoutException INFO [2022-01-10 05:48:04,040] jepsen worker 83 - jepsen.util 5123 :invoke :read [27 nil] INFO [2022-01-10 05:48:04,266] jepsen worker 41 - jepsen.util 4451 :info :write [13 34] :RequestTimeoutException INFO [2022-01-10 05:48:04,267] jepsen worker 41 - jepsen.util 4541 :invoke :read [13 nil] INFO [2022-01-10 05:48:04,400] jepsen worker 48 - jepsen.util 4278 :ok :read [16 38] INFO [2022-01-10 05:48:04,401] jepsen worker 48 - jepsen.util 4278 :invoke :write [16 19] INFO [2022-01-10 05:48:04,495] jepsen worker 16 - jepsen.util 4516 :ok :read [5 25] INFO [2022-01-10 05:48:04,496] jepsen worker 16 - jepsen.util 4516 :invoke :write [5 8] INFO [2022-01-10 05:48:04,586] jepsen worker 42 - jepsen.util 4182 :ok :read [14 21] INFO [2022-01-10 05:48:04,587] jepsen worker 42 - jepsen.util 4182 :invoke :read [14 nil] INFO [2022-01-10 05:48:04,600] jepsen worker 78 - jepsen.util 5478 :info :write [26 38] :RequestTimeoutException INFO [2022-01-10 05:48:04,600] jepsen worker 78 - jepsen.util 5568 :invoke :write [26 25] INFO [2022-01-10 05:48:04,771] jepsen worker 57 - jepsen.util 5187 :info :write [19 24] :RequestTimeoutException INFO [2022-01-10 05:48:04,772] jepsen worker 57 - jepsen.util 5277 :invoke :read [19 nil] INFO [2022-01-10 05:48:04,867] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5730994152046784}, :ns_1@172.23.107.227 {:progress 0.3929618768328446}, :ns_1@172.23.105.187 {:progress 0.3929618768328446}, :ns_1@172.23.107.228 {:progress 0.5470588235294118}} INFO [2022-01-10 05:48:04,867] jepsen worker nemesis - couchbase.util Rebalance check count: 632 INFO [2022-01-10 05:48:04,867] jepsen worker nemesis - couchbase.util Rebalance stuck count: 532 INFO [2022-01-10 05:48:04,887] jepsen worker 19 - jepsen.util 4609 :ok :read [6 25] INFO [2022-01-10 05:48:04,887] jepsen worker 19 - jepsen.util 4609 :invoke :read [6 nil] INFO [2022-01-10 05:48:04,936] jepsen worker 68 - jepsen.util 4658 :info :write [22 41] :RequestTimeoutException INFO [2022-01-10 05:48:04,937] jepsen worker 68 - jepsen.util 4748 :invoke :read [22 nil] INFO [2022-01-10 05:48:04,954] jepsen worker 42 - jepsen.util 4182 :ok :read [14 21] INFO [2022-01-10 05:48:04,955] jepsen worker 42 - jepsen.util 4182 :invoke :write [14 29] INFO [2022-01-10 05:48:05,062] jepsen worker 10 - jepsen.util 4240 :info :write [3 46] :RequestTimeoutException INFO [2022-01-10 05:48:05,062] jepsen worker 10 - jepsen.util 4330 :invoke :write [3 38] INFO [2022-01-10 05:48:05,091] jepsen worker 9 - jepsen.util 4959 :info :write [3 37] :RequestTimeoutException INFO [2022-01-10 05:48:05,091] jepsen worker 56 - jepsen.util 4646 :info :write [18 42] :RequestTimeoutException INFO [2022-01-10 05:48:05,092] jepsen worker 9 - jepsen.util 5049 :invoke :read [3 nil] INFO [2022-01-10 05:48:05,093] jepsen worker 56 - jepsen.util 4736 :invoke :write [18 1] INFO [2022-01-10 05:48:05,111] jepsen worker 89 - jepsen.util 4949 :info :write [29 23] :RequestTimeoutException INFO [2022-01-10 05:48:05,112] jepsen worker 89 - jepsen.util 5039 :invoke :read [29 nil] INFO [2022-01-10 05:48:05,157] jepsen worker 54 - jepsen.util 4644 :info :write [18 28] :RequestTimeoutException INFO [2022-01-10 05:48:05,157] jepsen worker 54 - jepsen.util 4734 :invoke :write [18 0] INFO [2022-01-10 05:48:05,245] jepsen worker 41 - jepsen.util 4541 :ok :read [13 15] INFO [2022-01-10 05:48:05,246] jepsen worker 41 - jepsen.util 4541 :invoke :read [13 nil] INFO [2022-01-10 05:48:05,274] jepsen worker 59 - jepsen.util 4379 :ok :read [19 31] INFO [2022-01-10 05:48:05,274] jepsen worker 59 - jepsen.util 4379 :invoke :write [19 40] INFO [2022-01-10 05:48:05,364] jepsen worker 76 - jepsen.util 3856 :ok :read [25 :nil] INFO [2022-01-10 05:48:05,365] jepsen worker 76 - jepsen.util 3856 :invoke :write [25 25] INFO [2022-01-10 05:48:05,382] jepsen worker 20 - jepsen.util 4160 :ok :read [6 25] INFO [2022-01-10 05:48:05,382] jepsen worker 20 - jepsen.util 4160 :invoke :read [6 nil] INFO [2022-01-10 05:48:05,393] jepsen worker 7 - jepsen.util 4777 :ok :read [2 14] INFO [2022-01-10 05:48:05,394] jepsen worker 7 - jepsen.util 4777 :invoke :write [2 48] INFO [2022-01-10 05:48:05,511] jepsen worker 21 - jepsen.util 4521 :info :write [7 14] :RequestTimeoutException INFO [2022-01-10 05:48:05,512] jepsen worker 21 - jepsen.util 4611 :invoke :read [7 nil] INFO [2022-01-10 05:48:05,554] jepsen worker 3 - jepsen.util 4323 :info :write [1 35] :RequestTimeoutException INFO [2022-01-10 05:48:05,555] jepsen worker 3 - jepsen.util 4413 :invoke :read [1 nil] INFO [2022-01-10 05:48:05,821] jepsen worker 25 - jepsen.util 4165 :ok :read [8 39] INFO [2022-01-10 05:48:05,822] jepsen worker 25 - jepsen.util 4165 :invoke :write [8 24] INFO [2022-01-10 05:48:05,878] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5730994152046784}, :ns_1@172.23.107.227 {:progress 0.3929618768328446}, :ns_1@172.23.105.187 {:progress 0.3929618768328446}, :ns_1@172.23.107.228 {:progress 0.5470588235294118}} INFO [2022-01-10 05:48:05,878] jepsen worker nemesis - couchbase.util Rebalance check count: 633 INFO [2022-01-10 05:48:05,878] jepsen worker nemesis - couchbase.util Rebalance stuck count: 533 INFO [2022-01-10 05:48:05,926] jepsen worker 14 - jepsen.util 5414 :info :write [4 33] :RequestTimeoutException INFO [2022-01-10 05:48:05,926] jepsen worker 14 - jepsen.util 5504 :invoke :read [4 nil] INFO [2022-01-10 05:48:05,941] jepsen worker 80 - jepsen.util 5480 :info :write [26 2] :RequestTimeoutException INFO [2022-01-10 05:48:05,941] jepsen worker 80 - jepsen.util 5570 :invoke :read [26 nil] INFO [2022-01-10 05:48:05,976] jepsen worker 63 - jepsen.util 4203 :ok :read [21 7] INFO [2022-01-10 05:48:05,977] jepsen worker 26 - jepsen.util 4256 :ok :read [8 39] INFO [2022-01-10 05:48:05,977] jepsen worker 63 - jepsen.util 4203 :invoke :write [21 14] INFO [2022-01-10 05:48:05,978] jepsen worker 26 - jepsen.util 4256 :invoke :read [8 nil] INFO [2022-01-10 05:48:05,979] jepsen worker 19 - jepsen.util 4609 :ok :read [6 25] INFO [2022-01-10 05:48:05,980] jepsen worker 19 - jepsen.util 4609 :invoke :write [6 37] INFO [2022-01-10 05:48:06,215] jepsen worker 36 - jepsen.util 3276 :info :write [12 30] :RequestTimeoutException INFO [2022-01-10 05:48:06,215] jepsen worker 27 - jepsen.util 4707 :info :write [9 22] :RequestTimeoutException INFO [2022-01-10 05:48:06,216] jepsen worker 36 - jepsen.util 3366 :invoke :write [12 35] INFO [2022-01-10 05:48:06,216] jepsen worker 27 - jepsen.util 4797 :invoke :read [9 nil] INFO [2022-01-10 05:48:06,254] jepsen worker 41 - jepsen.util 4541 :ok :read [13 15] INFO [2022-01-10 05:48:06,254] jepsen worker 41 - jepsen.util 4541 :invoke :read [13 nil] INFO [2022-01-10 05:48:06,281] jepsen worker 68 - jepsen.util 4748 :ok :read [22 41] INFO [2022-01-10 05:48:06,281] jepsen worker 68 - jepsen.util 4748 :invoke :write [22 36] INFO [2022-01-10 05:48:06,303] jepsen worker 20 - jepsen.util 4160 :ok :read [6 25] INFO [2022-01-10 05:48:06,304] jepsen worker 20 - jepsen.util 4160 :invoke :write [6 11] INFO [2022-01-10 05:48:06,884] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5789473684210527}, :ns_1@172.23.107.227 {:progress 0.3988269794721407}, :ns_1@172.23.105.187 {:progress 0.3988269794721407}, :ns_1@172.23.107.228 {:progress 0.5529411764705883}} INFO [2022-01-10 05:48:06,884] jepsen worker nemesis - couchbase.util Rebalance check count: 634 INFO [2022-01-10 05:48:06,884] jepsen worker nemesis - couchbase.util Rebalance stuck count: 533 INFO [2022-01-10 05:48:06,923] jepsen worker 41 - jepsen.util 4541 :ok :read [13 15] INFO [2022-01-10 05:48:06,924] jepsen worker 41 - jepsen.util 4541 :invoke :write [13 17] INFO [2022-01-10 05:48:06,940] jepsen worker 23 - jepsen.util 4523 :info :write [7 3] :RequestTimeoutException INFO [2022-01-10 05:48:06,941] jepsen worker 23 - jepsen.util 4613 :invoke :write [7 30] INFO [2022-01-10 05:48:06,975] jepsen worker 46 - jepsen.util 4546 :info :write [15 3] :RequestTimeoutException INFO [2022-01-10 05:48:06,975] jepsen worker 46 - jepsen.util 4636 :invoke :read [15 nil] INFO [2022-01-10 05:48:07,066] jepsen worker 18 - jepsen.util 4338 :info :write [6 42] :RequestTimeoutException INFO [2022-01-10 05:48:07,067] jepsen worker 18 - jepsen.util 4428 :invoke :read [6 nil] INFO [2022-01-10 05:48:07,096] jepsen worker 83 - jepsen.util 5123 :ok :read [27 30] INFO [2022-01-10 05:48:07,097] jepsen worker 83 - jepsen.util 5123 :invoke :write [27 49] INFO [2022-01-10 05:48:07,410] jepsen worker 65 - jepsen.util 4115 :info :write [21 6] :RequestTimeoutException INFO [2022-01-10 05:48:07,410] jepsen worker 65 - jepsen.util 4205 :invoke :write [21 5] INFO [2022-01-10 05:48:07,511] jepsen worker 29 - jepsen.util 4259 :info :write [9 0] :RequestTimeoutException INFO [2022-01-10 05:48:07,511] jepsen worker 29 - jepsen.util 4349 :invoke :write [9 37] INFO [2022-01-10 05:48:07,604] jepsen worker 62 - jepsen.util 4832 :info :write [20 38] :RequestTimeoutException INFO [2022-01-10 05:48:07,605] jepsen worker 62 - jepsen.util 4922 :invoke :read [20 nil] INFO [2022-01-10 05:48:07,649] jepsen worker 49 - jepsen.util 4189 :info :write [16 21] :RequestTimeoutException INFO [2022-01-10 05:48:07,650] jepsen worker 49 - jepsen.util 4279 :invoke :write [16 26] INFO [2022-01-10 05:48:07,770] jepsen worker 30 - jepsen.util 4170 :ok :read [10 12] INFO [2022-01-10 05:48:07,771] jepsen worker 30 - jepsen.util 4170 :invoke :write [10 41] INFO [2022-01-10 05:48:07,779] jepsen worker 11 - jepsen.util 4331 :ok :read [3 12] INFO [2022-01-10 05:48:07,780] jepsen worker 11 - jepsen.util 4331 :invoke :read [3 nil] INFO [2022-01-10 05:48:07,784] jepsen worker 89 - jepsen.util 5039 :ok :read [29 0] INFO [2022-01-10 05:48:07,784] jepsen worker 89 - jepsen.util 5039 :invoke :write [29 47] INFO [2022-01-10 05:48:07,818] jepsen worker 11 - jepsen.util 4331 :ok :read [3 12] INFO [2022-01-10 05:48:07,818] jepsen worker 11 - jepsen.util 4331 :invoke :read [3 nil] INFO [2022-01-10 05:48:07,852] jepsen worker 4 - jepsen.util 4774 :fail :read [1 nil] :UnambiguousTimeoutException INFO [2022-01-10 05:48:07,852] jepsen worker 4 - jepsen.util 4774 :invoke :read [1 nil] INFO [2022-01-10 05:48:07,895] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5789473684210527}, :ns_1@172.23.107.227 {:progress 0.3988269794721407}, :ns_1@172.23.105.187 {:progress 0.3988269794721407}, :ns_1@172.23.107.228 {:progress 0.5529411764705883}} INFO [2022-01-10 05:48:07,895] jepsen worker nemesis - couchbase.util Rebalance check count: 635 INFO [2022-01-10 05:48:07,895] jepsen worker nemesis - couchbase.util Rebalance stuck count: 534 INFO [2022-01-10 05:48:07,912] jepsen worker 11 - jepsen.util 4331 :ok :read [3 12] INFO [2022-01-10 05:48:07,912] jepsen worker 11 - jepsen.util 4331 :invoke :read [3 nil] INFO [2022-01-10 05:48:07,951] jepsen worker 11 - jepsen.util 4331 :ok :read [3 12] INFO [2022-01-10 05:48:07,952] jepsen worker 11 - jepsen.util 4331 :invoke :write [3 8] INFO [2022-01-10 05:48:07,952] jepsen worker 47 - jepsen.util 4637 :info :write [15 22] :RequestTimeoutException INFO [2022-01-10 05:48:07,952] jepsen worker 33 - jepsen.util 4623 :info :write [11 1] :RequestTimeoutException INFO [2022-01-10 05:48:07,953] jepsen worker 47 - jepsen.util 4727 :invoke :read [15 nil] INFO [2022-01-10 05:48:07,954] jepsen worker 33 - jepsen.util 4713 :invoke :write [11 42] INFO [2022-01-10 05:48:07,962] jepsen worker 44 - jepsen.util 4544 :info :write [14 5] :RequestTimeoutException INFO [2022-01-10 05:48:07,963] jepsen worker 44 - jepsen.util 4634 :invoke :write [14 45] INFO [2022-01-10 05:48:07,992] jepsen worker 74 - jepsen.util 4934 :info :write [24 41] :RequestTimeoutException INFO [2022-01-10 05:48:07,993] jepsen worker 74 - jepsen.util 5024 :invoke :read [24 nil] INFO [2022-01-10 05:48:08,311] jepsen worker 66 - jepsen.util 4206 :info :write [22 24] :RequestTimeoutException INFO [2022-01-10 05:48:08,311] jepsen worker 66 - jepsen.util 4296 :invoke :write [22 16] INFO [2022-01-10 05:48:08,325] jepsen worker 77 - jepsen.util 4937 :info :write [25 22] :RequestTimeoutException INFO [2022-01-10 05:48:08,326] jepsen worker 77 - jepsen.util 5027 :invoke :read [25 nil] INFO [2022-01-10 05:48:08,345] jepsen worker 35 - jepsen.util 4175 :info :write [11 48] :RequestTimeoutException INFO [2022-01-10 05:48:08,346] jepsen worker 35 - jepsen.util 4265 :invoke :write [11 36] INFO [2022-01-10 05:48:08,389] jepsen worker 72 - jepsen.util 4752 :ok :read [24 18] INFO [2022-01-10 05:48:08,390] jepsen worker 72 - jepsen.util 4752 :invoke :write [24 24] INFO [2022-01-10 05:48:08,400] jepsen worker 75 - jepsen.util 4215 :info :write [25 40] :RequestTimeoutException INFO [2022-01-10 05:48:08,400] jepsen worker 75 - jepsen.util 4305 :invoke :write [25 30] INFO [2022-01-10 05:48:08,406] jepsen worker 60 - jepsen.util 4740 :info :write [20 38] :RequestTimeoutException INFO [2022-01-10 05:48:08,407] jepsen worker 60 - jepsen.util 4830 :invoke :read [20 nil] INFO [2022-01-10 05:48:08,900] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5789473684210527}, :ns_1@172.23.107.227 {:progress 0.3988269794721407}, :ns_1@172.23.105.187 {:progress 0.3988269794721407}, :ns_1@172.23.107.228 {:progress 0.5529411764705883}} INFO [2022-01-10 05:48:08,900] jepsen worker nemesis - couchbase.util Rebalance check count: 636 INFO [2022-01-10 05:48:08,900] jepsen worker nemesis - couchbase.util Rebalance stuck count: 535 INFO [2022-01-10 05:48:08,920] jepsen worker 8 - jepsen.util 4598 :info :write [2 43] :RequestTimeoutException INFO [2022-01-10 05:48:08,920] jepsen worker 8 - jepsen.util 4688 :invoke :read [2 nil] INFO [2022-01-10 05:48:08,932] jepsen worker 43 - jepsen.util 4453 :info :write [14 23] :RequestTimeoutException INFO [2022-01-10 05:48:08,932] jepsen worker 43 - jepsen.util 4543 :invoke :write [14 2] INFO [2022-01-10 05:48:08,996] jepsen worker 88 - jepsen.util 4498 :info :write [29 15] :RequestTimeoutException INFO [2022-01-10 05:48:08,996] jepsen worker 88 - jepsen.util 4588 :invoke :write [29 14] INFO [2022-01-10 05:48:09,061] jepsen worker 40 - jepsen.util 4090 :info :write [13 46] :RequestTimeoutException INFO [2022-01-10 05:48:09,061] jepsen worker 40 - jepsen.util 4180 :invoke :write [13 36] INFO [2022-01-10 05:48:09,085] jepsen worker 81 - jepsen.util 4941 :info :write [27 17] :RequestTimeoutException INFO [2022-01-10 05:48:09,086] jepsen worker 81 - jepsen.util 5031 :invoke :write [27 32] INFO [2022-01-10 05:48:09,420] jepsen worker 64 - jepsen.util 4474 :info :write [21 21] :RequestTimeoutException INFO [2022-01-10 05:48:09,420] jepsen worker 64 - jepsen.util 4564 :invoke :read [21 nil] INFO [2022-01-10 05:48:09,502] jepsen worker 34 - jepsen.util 4264 :info :write [11 13] :RequestTimeoutException INFO [2022-01-10 05:48:09,503] jepsen worker 34 - jepsen.util 4354 :invoke :write [11 41] INFO [2022-01-10 05:48:09,522] jepsen worker 24 - jepsen.util 4074 :info :write [8 27] :RequestTimeoutException INFO [2022-01-10 05:48:09,523] jepsen worker 24 - jepsen.util 4164 :invoke :write [8 48] INFO [2022-01-10 05:48:09,545] jepsen worker 26 - jepsen.util 4256 :ok :read [8 39] INFO [2022-01-10 05:48:09,545] jepsen worker 26 - jepsen.util 4256 :invoke :write [8 18] INFO [2022-01-10 05:48:09,677] jepsen worker 70 - jepsen.util 4480 :info :write [23 34] :RequestTimeoutException INFO [2022-01-10 05:48:09,677] jepsen worker 70 - jepsen.util 4570 :invoke :read [23 nil] INFO [2022-01-10 05:48:09,712] jepsen worker 6 - jepsen.util 4416 :info :write [2 3] :RequestTimeoutException INFO [2022-01-10 05:48:09,713] jepsen worker 6 - jepsen.util 4506 :invoke :read [2 nil] INFO [2022-01-10 05:48:09,736] jepsen worker 69 - jepsen.util 4119 :ok :read [23 26] INFO [2022-01-10 05:48:09,738] jepsen worker 69 - jepsen.util 4119 :invoke :read [23 nil] INFO [2022-01-10 05:48:09,741] jepsen worker 13 - jepsen.util 5233 :ok :read [4 3] INFO [2022-01-10 05:48:09,742] jepsen worker 13 - jepsen.util 5233 :invoke :write [4 8] INFO [2022-01-10 05:48:09,743] jepsen worker 12 - jepsen.util 5052 :ok :read [4 3] INFO [2022-01-10 05:48:09,744] jepsen worker 12 - jepsen.util 5052 :invoke :read [4 nil] INFO [2022-01-10 05:48:09,749] jepsen worker 52 - jepsen.util 5632 :ok :read [17 41] INFO [2022-01-10 05:48:09,750] jepsen worker 52 - jepsen.util 5632 :invoke :write [17 20] INFO [2022-01-10 05:48:09,770] jepsen worker 77 - jepsen.util 5027 :ok :read [25 :nil] INFO [2022-01-10 05:48:09,771] jepsen worker 77 - jepsen.util 5027 :invoke :write [25 4] INFO [2022-01-10 05:48:09,799] jepsen worker 18 - jepsen.util 4428 :ok :read [6 25] INFO [2022-01-10 05:48:09,800] jepsen worker 18 - jepsen.util 4428 :invoke :write [6 45] INFO [2022-01-10 05:48:09,906] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5789473684210527}, :ns_1@172.23.107.227 {:progress 0.3988269794721407}, :ns_1@172.23.105.187 {:progress 0.3988269794721407}, :ns_1@172.23.107.228 {:progress 0.5529411764705883}} INFO [2022-01-10 05:48:09,906] jepsen worker nemesis - couchbase.util Rebalance check count: 637 INFO [2022-01-10 05:48:09,906] jepsen worker nemesis - couchbase.util Rebalance stuck count: 536 INFO [2022-01-10 05:48:09,929] jepsen worker 53 - jepsen.util 5453 :ok :read [17 41] INFO [2022-01-10 05:48:09,930] jepsen worker 53 - jepsen.util 5453 :invoke :write [17 38] INFO [2022-01-10 05:48:09,949] jepsen worker 80 - jepsen.util 5570 :ok :read [26 10] INFO [2022-01-10 05:48:09,950] jepsen worker 80 - jepsen.util 5570 :invoke :write [26 15] INFO [2022-01-10 05:48:10,097] jepsen worker 60 - jepsen.util 4830 :ok :read [20 :nil] INFO [2022-01-10 05:48:10,098] jepsen worker 60 - jepsen.util 4830 :invoke :write [20 7] INFO [2022-01-10 05:48:10,101] jepsen worker 5 - jepsen.util 4505 :ok :read [1 49] INFO [2022-01-10 05:48:10,102] jepsen worker 5 - jepsen.util 4505 :invoke :write [1 0] INFO [2022-01-10 05:48:10,112] jepsen worker 3 - jepsen.util 4413 :ok :read [1 49] INFO [2022-01-10 05:48:10,113] jepsen worker 3 - jepsen.util 4413 :invoke :read [1 nil] INFO [2022-01-10 05:48:10,277] jepsen worker 38 - jepsen.util 3998 :info :write [12 24] :RequestTimeoutException INFO [2022-01-10 05:48:10,277] jepsen worker 38 - jepsen.util 4088 :invoke :write [12 32] INFO [2022-01-10 05:48:10,475] jepsen worker 45 - jepsen.util 4905 :info :write [15 3] :RequestTimeoutException INFO [2022-01-10 05:48:10,476] jepsen worker 45 - jepsen.util 4995 :invoke :write [15 45] INFO [2022-01-10 05:48:10,499] jepsen worker 4 - jepsen.util 4774 :ok :read [1 49] INFO [2022-01-10 05:48:10,500] jepsen worker 4 - jepsen.util 4774 :invoke :write [1 31] INFO [2022-01-10 05:48:10,508] jepsen worker 3 - jepsen.util 4413 :ok :read [1 49] INFO [2022-01-10 05:48:10,509] jepsen worker 3 - jepsen.util 4413 :invoke :write [1 26] INFO [2022-01-10 05:48:10,510] jepsen worker 69 - jepsen.util 4119 :ok :read [23 26] INFO [2022-01-10 05:48:10,510] jepsen worker 69 - jepsen.util 4119 :invoke :read [23 nil] INFO [2022-01-10 05:48:10,606] jepsen worker 39 - jepsen.util 4809 :info :write [13 36] :RequestTimeoutException INFO [2022-01-10 05:48:10,606] jepsen worker 39 - jepsen.util 4899 :invoke :write [13 24] WARN [2022-01-10 05:48:10,728] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9365925,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18783125,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9462775,"encode_duration_us":18398,"operation_id":"0x8841","timeout_ms":10000,"last_server_duration_us":9078800,"total_duration_us":10005533},{"operation_name":"upsert","last_dispatch_duration_us":18105864,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18105864,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9033121,"encode_duration_us":18034,"operation_id":"0x88fa","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10009876}],"total_count":2}} WARN [2022-01-10 05:48:10,729] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":16557796,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":16557796,"total_server_duration_us":32,"operation_id":"0x89b1","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":16559092},{"operation_name":"upsert","last_dispatch_duration_us":856793,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":4529897,"total_server_duration_us":1179982,"encode_duration_us":20573,"operation_id":"0x8ae2","timeout_ms":10000,"last_server_duration_us":189015,"total_duration_us":10010603},{"operation_name":"upsert","last_dispatch_duration_us":697757,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5290059,"total_server_duration_us":1448660,"encode_duration_us":18433,"operation_id":"0x8b74","timeout_ms":10000,"last_server_duration_us":124398,"total_duration_us":10008163},{"operation_name":"upsert","last_dispatch_duration_us":387895,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":996782,"total_server_duration_us":303288,"encode_duration_us":20792,"operation_id":"0x8b11","timeout_ms":10000,"last_server_duration_us":68946,"total_duration_us":10008133},{"operation_name":"upsert","last_dispatch_duration_us":758689,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":6571765,"total_server_duration_us":1021916,"encode_duration_us":22212,"operation_id":"0x8b3e","timeout_ms":10000,"last_server_duration_us":58004,"total_duration_us":10004606},{"operation_name":"upsert","last_dispatch_duration_us":334141,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1358225,"total_server_duration_us":212008,"encode_duration_us":18671,"operation_id":"0x899b","timeout_ms":10000,"last_server_duration_us":119804,"total_duration_us":10004318},{"operation_name":"upsert","last_dispatch_duration_us":505540,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6822590,"total_server_duration_us":1823731,"encode_duration_us":17160,"operation_id":"0x8ad1","timeout_ms":10000,"last_server_duration_us":134160,"total_duration_us":10003369},{"operation_name":"upsert","last_dispatch_duration_us":8875567,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10810452,"total_server_duration_us":682857,"encode_duration_us":16170,"operation_id":"0x89e3","timeout_ms":10000,"last_server_duration_us":21977,"total_duration_us":10002563},{"operation_name":"upsert","last_dispatch_duration_us":164089,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":8883636,"total_server_duration_us":311082,"encode_duration_us":16633,"operation_id":"0x8a7c","timeout_ms":10000,"last_server_duration_us":27492,"total_duration_us":10001725},{"operation_name":"upsert","last_dispatch_duration_us":71805,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":8380180,"total_server_duration_us":618449,"encode_duration_us":16515,"operation_id":"0x8aaf","timeout_ms":10000,"last_server_duration_us":56401,"total_duration_us":10001163}],"total_count":15}} INFO [2022-01-10 05:48:10,814] jepsen worker 17 - jepsen.util 4247 :info :write [5 38] :RequestTimeoutException INFO [2022-01-10 05:48:10,815] jepsen worker 17 - jepsen.util 4337 :invoke :write [5 33] INFO [2022-01-10 05:48:10,909] jepsen worker 61 - jepsen.util 4741 :info :write [20 32] :RequestTimeoutException INFO [2022-01-10 05:48:10,910] jepsen worker 61 - jepsen.util 4831 :invoke :write [20 45] INFO [2022-01-10 05:48:10,915] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5789473684210527}, :ns_1@172.23.107.227 {:progress 0.3988269794721407}, :ns_1@172.23.105.187 {:progress 0.3988269794721407}, :ns_1@172.23.107.228 {:progress 0.5529411764705883}} INFO [2022-01-10 05:48:10,982] jepsen worker 15 - jepsen.util 3975 :info :write [5 20] :RequestTimeoutException INFO [2022-01-10 05:48:11,019] jepsen worker 12 - jepsen.util 5052 :ok :read [4 3] INFO [2022-01-10 05:48:11,471] jepsen worker nemesis - couchbase.util Rebalance check count: 638 INFO [2022-01-10 05:48:11,181] jepsen worker 50 - jepsen.util 4730 :info :write [16 16] :RequestTimeoutException INFO [2022-01-10 05:48:11,471] jepsen worker 15 - jepsen.util 4065 :invoke :write [5 12] INFO [2022-01-10 05:48:11,471] jepsen worker 12 - jepsen.util 5052 :invoke :read [4 nil] INFO [2022-01-10 05:48:11,471] jepsen worker 50 - jepsen.util 4820 :invoke :read [16 nil] INFO [2022-01-10 05:48:11,471] jepsen worker nemesis - couchbase.util Rebalance stuck count: 537 INFO [2022-01-10 05:48:11,541] jepsen worker 31 - jepsen.util 4621 :info :write [10 24] :SyncWriteAmbiguous INFO [2022-01-10 05:48:11,542] jepsen worker 31 - jepsen.util 4711 :invoke :read [10 nil] INFO [2022-01-10 05:48:11,640] jepsen worker 69 - jepsen.util 4119 :ok :read [23 26] INFO [2022-01-10 05:48:11,641] jepsen worker 69 - jepsen.util 4119 :invoke :write [23 27] INFO [2022-01-10 05:48:11,729] jepsen worker 86 - jepsen.util 3866 :ok :read [28 45] INFO [2022-01-10 05:48:11,730] jepsen worker 86 - jepsen.util 3866 :invoke :write [28 31] INFO [2022-01-10 05:48:11,824] jepsen worker 9 - jepsen.util 5049 :ok :read [3 12] INFO [2022-01-10 05:48:11,825] jepsen worker 9 - jepsen.util 5049 :invoke :read [3 nil] INFO [2022-01-10 05:48:11,839] jepsen worker 21 - jepsen.util 4611 :ok :read [7 :nil] INFO [2022-01-10 05:48:11,839] jepsen worker 21 - jepsen.util 4611 :invoke :read [7 nil] INFO [2022-01-10 05:48:11,892] jepsen worker 9 - jepsen.util 5049 :ok :read [3 12] INFO [2022-01-10 05:48:11,892] jepsen worker 9 - jepsen.util 5049 :invoke :read [3 nil] INFO [2022-01-10 05:48:11,925] jepsen worker 9 - jepsen.util 5049 :ok :read [3 12] INFO [2022-01-10 05:48:11,925] jepsen worker 9 - jepsen.util 5049 :invoke :read [3 nil] INFO [2022-01-10 05:48:11,961] jepsen worker 9 - jepsen.util 5049 :ok :read [3 12] INFO [2022-01-10 05:48:11,961] jepsen worker 9 - jepsen.util 5049 :invoke :read [3 nil] INFO [2022-01-10 05:48:12,002] jepsen worker 9 - jepsen.util 5049 :ok :read [3 12] INFO [2022-01-10 05:48:12,003] jepsen worker 9 - jepsen.util 5049 :invoke :write [3 46] INFO [2022-01-10 05:48:12,031] jepsen worker 73 - jepsen.util 4213 :info :write [24 49] :RequestTimeoutException INFO [2022-01-10 05:48:12,032] jepsen worker 73 - jepsen.util 4303 :invoke :write [24 6] INFO [2022-01-10 05:48:12,120] jepsen worker 0 - jepsen.util 4500 :info :write [0 2] :RequestTimeoutException INFO [2022-01-10 05:48:12,121] jepsen worker 0 - jepsen.util 4590 :invoke :read [0 nil] WARN [2022-01-10 05:48:12,229] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10830571,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11440738,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9139922,"encode_duration_us":19692,"operation_id":"0x89a0","timeout_ms":10000,"last_server_duration_us":9059667,"total_duration_us":10001237},{"operation_name":"upsert","last_dispatch_duration_us":9894573,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12370472,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":10249202,"encode_duration_us":22053,"operation_id":"0x890c","timeout_ms":10000,"last_server_duration_us":9458563,"total_duration_us":10006273},{"operation_name":"upsert","last_dispatch_duration_us":9043286,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9749382,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9436423,"encode_duration_us":16739,"operation_id":"0x8934","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10002927},{"operation_name":"upsert","last_dispatch_duration_us":18485821,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":22201407,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9944300,"encode_duration_us":23927,"operation_id":"0x88a5","timeout_ms":10000,"last_server_duration_us":9135235,"total_duration_us":10010355}],"total_count":4}} WARN [2022-01-10 05:48:12,246] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9183817,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":29446092,"total_server_duration_us":6472,"operation_id":"0x8713","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":30007925},{"operation_name":"get","last_dispatch_duration_us":10021110,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10021110,"total_server_duration_us":32,"operation_id":"0x8b4a","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":10023756},{"operation_name":"upsert","last_dispatch_duration_us":1384933,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3069557,"total_server_duration_us":464442,"encode_duration_us":16999,"operation_id":"0x8a7b","timeout_ms":10000,"last_server_duration_us":267606,"total_duration_us":10010942},{"operation_name":"upsert","last_dispatch_duration_us":743706,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10566240,"total_server_duration_us":163582,"encode_duration_us":17644,"operation_id":"0x8a5c","timeout_ms":10000,"last_server_duration_us":73273,"total_duration_us":10010379},{"operation_name":"upsert","last_dispatch_duration_us":13772391,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":13772391,"total_server_duration_us":9034183,"encode_duration_us":18603,"operation_id":"0x8aa8","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10009868},{"operation_name":"upsert","last_dispatch_duration_us":198378,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":4381565,"total_server_duration_us":346939,"encode_duration_us":16488,"operation_id":"0x8ba3","timeout_ms":10000,"last_server_duration_us":73959,"total_duration_us":10007153},{"operation_name":"upsert","last_dispatch_duration_us":31603,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":1193714,"total_server_duration_us":353308,"encode_duration_us":16689,"operation_id":"0x8a6f","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10006904},{"operation_name":"upsert","last_dispatch_duration_us":986180,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2186011,"total_server_duration_us":576632,"encode_duration_us":16985,"operation_id":"0x8a3c","timeout_ms":10000,"last_server_duration_us":306254,"total_duration_us":10006758},{"operation_name":"upsert","last_dispatch_duration_us":1702980,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10749897,"total_server_duration_us":1335647,"encode_duration_us":16455,"operation_id":"0x8b10","timeout_ms":10000,"last_server_duration_us":731732,"total_duration_us":10006569},{"operation_name":"upsert","last_dispatch_duration_us":9445764,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10815159,"total_server_duration_us":279114,"encode_duration_us":16455,"operation_id":"0x8a17","timeout_ms":10000,"last_server_duration_us":69080,"total_duration_us":10004942}],"total_count":30}} WARN [2022-01-10 05:48:12,247] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9507557,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17109577,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9202664,"encode_duration_us":16531,"operation_id":"0x88c9","timeout_ms":10000,"last_server_duration_us":9058604,"total_duration_us":10004137},{"operation_name":"upsert","last_dispatch_duration_us":9035901,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10076475,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9320745,"encode_duration_us":17775,"operation_id":"0x8a6e","timeout_ms":10000,"last_server_duration_us":9023573,"total_duration_us":10004560},{"operation_name":"upsert","last_dispatch_duration_us":13772391,"last_local_id":"EFE9906200000006/0000000021E2C07D","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":13772391,"last_local_socket":"172.23.104.79:44302","total_server_duration_us":9034183,"encode_duration_us":18603,"operation_id":"0x8aa8","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10009868},{"operation_name":"get","last_dispatch_duration_us":9183817,"last_local_id":"EFE9906200000006/0000000021E2C07D","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":29446092,"last_local_socket":"172.23.104.79:44302","total_server_duration_us":6472,"operation_id":"0x8713","timeout_ms":30000,"last_server_duration_us":23,"total_duration_us":30007925}],"total_count":4}} INFO [2022-01-10 05:48:12,264] jepsen worker 21 - jepsen.util 4611 :ok :read [7 :nil] INFO [2022-01-10 05:48:12,264] jepsen worker 21 - jepsen.util 4611 :invoke :read [7 nil] WARN [2022-01-10 05:48:12,330] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17335828,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":17335828,"total_server_duration_us":114786,"operation_id":"0x8a48","timeout_ms":30000,"last_server_duration_us":114786,"total_duration_us":17338106},{"operation_name":"upsert","last_dispatch_duration_us":7661154,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11286078,"total_server_duration_us":594602,"encode_duration_us":18714,"operation_id":"0x8a0e","timeout_ms":10000,"last_server_duration_us":134337,"total_duration_us":10007205},{"operation_name":"upsert","last_dispatch_duration_us":7607944,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11953859,"total_server_duration_us":513796,"encode_duration_us":18421,"operation_id":"0x8a11","timeout_ms":10000,"last_server_duration_us":61520,"total_duration_us":10005963},{"operation_name":"upsert","last_dispatch_duration_us":2115207,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2115207,"total_server_duration_us":62796,"encode_duration_us":18277,"operation_id":"0x8b12","timeout_ms":10000,"last_server_duration_us":62796,"total_duration_us":10005470},{"operation_name":"upsert","last_dispatch_duration_us":196008,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":3827712,"total_server_duration_us":2822446,"encode_duration_us":17759,"operation_id":"0x8aa7","timeout_ms":10000,"last_server_duration_us":72045,"total_duration_us":10002928},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":16562,"operation_id":"0x8ab0","timeout_ms":10000,"total_duration_us":10001410},{"operation_name":"get","last_dispatch_duration_us":9797264,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9797264,"total_server_duration_us":42308,"operation_id":"0x89e4","timeout_ms":30000,"last_server_duration_us":42308,"total_duration_us":9800386},{"operation_name":"get","last_dispatch_duration_us":9015711,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9015711,"total_server_duration_us":21731,"operation_id":"0x8a13","timeout_ms":30000,"last_server_duration_us":21731,"total_duration_us":9021086},{"operation_name":"get","last_dispatch_duration_us":7288133,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7288133,"total_server_duration_us":7231,"operation_id":"0x8ae1","timeout_ms":30000,"last_server_duration_us":7231,"total_duration_us":7290126},{"operation_name":"get","last_dispatch_duration_us":1569152,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1569152,"total_server_duration_us":234557,"operation_id":"0x8b1f","timeout_ms":30000,"last_server_duration_us":234557,"total_duration_us":7245893}],"total_count":16}} WARN [2022-01-10 05:48:12,330] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10821869,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10821869,"total_server_duration_us":10294990,"encode_duration_us":16694,"operation_id":"0x8ad3","timeout_ms":10000,"last_server_duration_us":10294990,"total_duration_us":10009578},{"operation_name":"upsert","last_dispatch_duration_us":9764035,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10643963,"total_server_duration_us":95387,"encode_duration_us":21400,"operation_id":"0x8a69","timeout_ms":10000,"last_server_duration_us":74786,"total_duration_us":10009107},{"operation_name":"upsert","last_dispatch_duration_us":8756529,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":12605563,"total_server_duration_us":109859,"encode_duration_us":20589,"operation_id":"0x8b3a","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10008104},{"operation_name":"upsert","last_dispatch_duration_us":9682138,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11451676,"total_server_duration_us":101573,"encode_duration_us":19113,"operation_id":"0x8a46","timeout_ms":10000,"last_server_duration_us":46287,"total_duration_us":10007949},{"operation_name":"upsert","last_dispatch_duration_us":34679,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":8050985,"total_server_duration_us":1575853,"encode_duration_us":18232,"operation_id":"0x8a19","timeout_ms":10000,"last_server_duration_us":22059,"total_duration_us":10007041},{"operation_name":"upsert","last_dispatch_duration_us":359571,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":1778491,"total_server_duration_us":249831,"encode_duration_us":16399,"operation_id":"0x8b72","timeout_ms":10000,"last_server_duration_us":39635,"total_duration_us":10006124},{"operation_name":"upsert","last_dispatch_duration_us":180312,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":1617325,"total_server_duration_us":159667,"encode_duration_us":17308,"operation_id":"0x8b76","timeout_ms":10000,"last_server_duration_us":62029,"total_duration_us":10004892},{"operation_name":"upsert","last_dispatch_duration_us":35376,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":1943385,"total_server_duration_us":832900,"encode_duration_us":20723,"operation_id":"0x8b49","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10003329},{"operation_name":"upsert","last_dispatch_duration_us":8763082,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":14908622,"total_server_duration_us":295453,"encode_duration_us":16261,"operation_id":"0x8a78","timeout_ms":10000,"last_server_duration_us":22804,"total_duration_us":10003190},{"operation_name":"upsert","last_dispatch_duration_us":10451037,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10451037,"total_server_duration_us":19262,"encode_duration_us":16391,"operation_id":"0x8b16","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10002410}],"total_count":19}} WARN [2022-01-10 05:48:12,343] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1075739,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":8367078,"total_server_duration_us":525126,"encode_duration_us":16257,"operation_id":"0x8b9a","timeout_ms":10000,"last_server_duration_us":44389,"total_duration_us":10008834},{"operation_name":"upsert","last_dispatch_duration_us":3788311,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6612060,"total_server_duration_us":573112,"encode_duration_us":16835,"operation_id":"0x8b48","timeout_ms":10000,"last_server_duration_us":223881,"total_duration_us":10007665},{"operation_name":"upsert","last_dispatch_duration_us":2881443,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":5474704,"total_server_duration_us":1356725,"encode_duration_us":21184,"operation_id":"0x8b51","timeout_ms":10000,"last_server_duration_us":1138233,"total_duration_us":10007500},{"operation_name":"upsert","last_dispatch_duration_us":2194221,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7635548,"total_server_duration_us":820832,"encode_duration_us":23348,"operation_id":"0x8b06","timeout_ms":10000,"last_server_duration_us":217533,"total_duration_us":10006946},{"operation_name":"upsert","last_dispatch_duration_us":1954549,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7269989,"total_server_duration_us":592169,"encode_duration_us":18886,"operation_id":"0x8b39","timeout_ms":10000,"last_server_duration_us":82691,"total_duration_us":10006033},{"operation_name":"upsert","last_dispatch_duration_us":3607549,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10174097,"total_server_duration_us":744612,"encode_duration_us":20519,"operation_id":"0x8b07","timeout_ms":10000,"last_server_duration_us":184936,"total_duration_us":10005809},{"operation_name":"upsert","last_dispatch_duration_us":9270929,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10372062,"total_server_duration_us":185862,"encode_duration_us":19491,"operation_id":"0x8a1c","timeout_ms":10000,"last_server_duration_us":59249,"total_duration_us":10004949},{"operation_name":"upsert","last_dispatch_duration_us":2043529,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":3658801,"total_server_duration_us":325355,"encode_duration_us":19384,"operation_id":"0x8b47","timeout_ms":10000,"last_server_duration_us":113463,"total_duration_us":10004557},{"operation_name":"upsert","last_dispatch_duration_us":5353085,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13124804,"total_server_duration_us":2637181,"encode_duration_us":17940,"operation_id":"0x8b05","timeout_ms":10000,"last_server_duration_us":1331589,"total_duration_us":10004334},{"operation_name":"upsert","last_dispatch_duration_us":3092740,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6114587,"total_server_duration_us":744904,"encode_duration_us":16969,"operation_id":"0x8b04","timeout_ms":10000,"last_server_duration_us":224101,"total_duration_us":10004170}],"total_count":19}} WARN [2022-01-10 05:48:12,348] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10408009,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11475722,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9596934,"encode_duration_us":16430,"operation_id":"0x8ad0","timeout_ms":10000,"last_server_duration_us":9323710,"total_duration_us":10001741},{"operation_name":"upsert","last_dispatch_duration_us":10488239,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17412376,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":11826629,"encode_duration_us":20038,"operation_id":"0x8908","timeout_ms":10000,"last_server_duration_us":10362393,"total_duration_us":10005798},{"operation_name":"upsert","last_dispatch_duration_us":10282894,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15506094,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9781752,"encode_duration_us":17079,"operation_id":"0x8935","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10007770},{"operation_name":"upsert","last_dispatch_duration_us":17475281,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":18436113,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":9407376,"encode_duration_us":16533,"operation_id":"0x8792","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10008806}],"total_count":4}} INFO [2022-01-10 05:48:12,426] jepsen worker 58 - jepsen.util 4378 :info :write [19 25] :RequestTimeoutException INFO [2022-01-10 05:48:12,427] jepsen worker 58 - jepsen.util 4468 :invoke :write [19 49] WARN [2022-01-10 05:48:12,443] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9719185,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10280646,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9174589,"encode_duration_us":22495,"operation_id":"0x8a23","timeout_ms":10000,"last_server_duration_us":9072420,"total_duration_us":10001414},{"operation_name":"upsert","last_dispatch_duration_us":10585446,"last_local_id":"EFE9906200000004/000000008B42EFB5","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":17986268,"last_local_socket":"172.23.104.79:44292","total_server_duration_us":9801918,"encode_duration_us":16771,"operation_id":"0x8978","timeout_ms":10000,"last_server_duration_us":9096885,"total_duration_us":10003138}],"total_count":2}} INFO [2022-01-10 05:48:12,445] jepsen worker 21 - jepsen.util 4611 :ok :read [7 :nil] INFO [2022-01-10 05:48:12,446] jepsen worker 21 - jepsen.util 4611 :invoke :read [7 nil] INFO [2022-01-10 05:48:12,477] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5789473684210527}, :ns_1@172.23.107.227 {:progress 0.3988269794721407}, :ns_1@172.23.105.187 {:progress 0.3988269794721407}, :ns_1@172.23.107.228 {:progress 0.5529411764705883}} INFO [2022-01-10 05:48:12,478] jepsen worker nemesis - couchbase.util Rebalance check count: 639 INFO [2022-01-10 05:48:12,478] jepsen worker nemesis - couchbase.util Rebalance stuck count: 538 INFO [2022-01-10 05:48:12,484] jepsen worker 22 - jepsen.util 4342 :info :write [7 31] :RequestTimeoutException INFO [2022-01-10 05:48:12,485] jepsen worker 22 - jepsen.util 4432 :invoke :read [7 nil] INFO [2022-01-10 05:48:12,490] jepsen worker 1 - jepsen.util 4501 :ok :read [0 46] INFO [2022-01-10 05:48:12,490] jepsen worker 1 - jepsen.util 4501 :invoke :write [0 43] INFO [2022-01-10 05:48:12,506] jepsen worker 55 - jepsen.util 5005 :info :write [18 3] :RequestTimeoutException INFO [2022-01-10 05:48:12,507] jepsen worker 55 - jepsen.util 5095 :invoke :write [18 23] WARN [2022-01-10 05:48:12,531] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10013844,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16260990,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9477025,"encode_duration_us":18234,"operation_id":"0x8906","timeout_ms":10000,"last_server_duration_us":9201443,"total_duration_us":10005116},{"operation_name":"upsert","last_dispatch_duration_us":10821869,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10821869,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":10294990,"encode_duration_us":16694,"operation_id":"0x8ad3","timeout_ms":10000,"last_server_duration_us":10294990,"total_duration_us":10009578}],"total_count":2}} INFO [2022-01-10 05:48:12,546] jepsen worker 32 - jepsen.util 4712 :info :write [10 12] :RequestTimeoutException INFO [2022-01-10 05:48:12,547] jepsen worker 32 - jepsen.util 4802 :invoke :read [10 nil] WARN [2022-01-10 05:48:12,548] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15113299,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":15113299,"total_server_duration_us":92,"operation_id":"0x8903","timeout_ms":30000,"last_server_duration_us":92,"total_duration_us":15115370},{"operation_name":"get","last_dispatch_duration_us":12813579,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":12813579,"total_server_duration_us":32,"operation_id":"0x8976","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":12815668},{"operation_name":"upsert","last_dispatch_duration_us":962912,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":4700029,"total_server_duration_us":563051,"encode_duration_us":18388,"operation_id":"0x8bc8","timeout_ms":10000,"last_server_duration_us":227194,"total_duration_us":10010558},{"operation_name":"upsert","last_dispatch_duration_us":9022612,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10694940,"total_server_duration_us":374167,"encode_duration_us":17225,"operation_id":"0x8b38","timeout_ms":10000,"last_server_duration_us":119129,"total_duration_us":10010194},{"operation_name":"upsert","last_dispatch_duration_us":8532656,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10523009,"total_server_duration_us":497148,"encode_duration_us":28269,"operation_id":"0x8b15","timeout_ms":10000,"last_server_duration_us":202112,"total_duration_us":10008217},{"operation_name":"upsert","last_dispatch_duration_us":9552832,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11634378,"total_server_duration_us":208322,"encode_duration_us":27824,"operation_id":"0x8a1a","timeout_ms":10000,"last_server_duration_us":81541,"total_duration_us":10007232},{"operation_name":"upsert","last_dispatch_duration_us":2741814,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12048009,"total_server_duration_us":1888738,"encode_duration_us":16376,"operation_id":"0x8b46","timeout_ms":10000,"last_server_duration_us":1367518,"total_duration_us":10006051},{"operation_name":"upsert","last_dispatch_duration_us":422417,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9912857,"total_server_duration_us":148247,"encode_duration_us":16385,"operation_id":"0x8a49","timeout_ms":10000,"last_server_duration_us":21001,"total_duration_us":10005945},{"operation_name":"upsert","last_dispatch_duration_us":2704483,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2704483,"total_server_duration_us":304746,"encode_duration_us":16644,"operation_id":"0x8b99","timeout_ms":10000,"last_server_duration_us":304746,"total_duration_us":10005880},{"operation_name":"upsert","last_dispatch_duration_us":375012,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10025691,"total_server_duration_us":253740,"encode_duration_us":16769,"operation_id":"0x8ae3","timeout_ms":10000,"last_server_duration_us":55668,"total_duration_us":10005464}],"total_count":17}} INFO [2022-01-10 05:48:12,596] jepsen worker 87 - jepsen.util 4407 :info :write [29 21] :RequestTimeoutException INFO [2022-01-10 05:48:12,597] jepsen worker 87 - jepsen.util 4497 :invoke :read [29 nil] INFO [2022-01-10 05:48:12,604] jepsen worker 84 - jepsen.util 4314 :ok :read [28 45] INFO [2022-01-10 05:48:12,605] jepsen worker 84 - jepsen.util 4314 :invoke :read [28 nil] INFO [2022-01-10 05:48:12,695] jepsen worker 67 - jepsen.util 4297 :ok :read [22 41] INFO [2022-01-10 05:48:12,695] jepsen worker 67 - jepsen.util 4297 :invoke :read [22 nil] INFO [2022-01-10 05:48:12,711] jepsen worker 2 - jepsen.util 4682 :ok :read [0 46] INFO [2022-01-10 05:48:12,711] jepsen worker 2 - jepsen.util 4682 :invoke :read [0 nil] INFO [2022-01-10 05:48:12,920] jepsen worker 57 - jepsen.util 5277 :ok :read [19 31] INFO [2022-01-10 05:48:12,921] jepsen worker 57 - jepsen.util 5277 :invoke :read [19 nil] INFO [2022-01-10 05:48:12,933] jepsen worker 27 - jepsen.util 4797 :ok :read [9 22] INFO [2022-01-10 05:48:12,933] jepsen worker 27 - jepsen.util 4797 :invoke :write [9 30] INFO [2022-01-10 05:48:13,340] jepsen worker 51 - jepsen.util 5271 :info :write [17 24] :RequestTimeoutException INFO [2022-01-10 05:48:13,341] jepsen worker 51 - jepsen.util 5361 :invoke :read [17 nil] INFO [2022-01-10 05:48:13,370] jepsen worker 37 - jepsen.util 4087 :info :write [12 35] :RequestTimeoutException INFO [2022-01-10 05:48:13,370] jepsen worker 37 - jepsen.util 4177 :invoke :write [12 20] INFO [2022-01-10 05:48:13,379] jepsen worker 71 - jepsen.util 4031 :info :write [23 18] :RequestTimeoutException INFO [2022-01-10 05:48:13,380] jepsen worker 71 - jepsen.util 4121 :invoke :write [23 47] INFO [2022-01-10 05:48:13,457] jepsen worker 51 - jepsen.util 5361 :ok :read [17 41] INFO [2022-01-10 05:48:13,458] jepsen worker 51 - jepsen.util 5361 :invoke :write [17 15] INFO [2022-01-10 05:48:13,483] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5789473684210527}, :ns_1@172.23.107.227 {:progress 0.3988269794721407}, :ns_1@172.23.105.187 {:progress 0.3988269794721407}, :ns_1@172.23.107.228 {:progress 0.5529411764705883}} INFO [2022-01-10 05:48:13,484] jepsen worker nemesis - couchbase.util Rebalance check count: 640 INFO [2022-01-10 05:48:13,484] jepsen worker nemesis - couchbase.util Rebalance stuck count: 539 INFO [2022-01-10 05:48:13,547] jepsen worker 28 - jepsen.util 4258 :info :write [9 36] :RequestTimeoutException INFO [2022-01-10 05:48:13,547] jepsen worker 28 - jepsen.util 4348 :invoke :read [9 nil] INFO [2022-01-10 05:48:13,562] jepsen worker 12 - jepsen.util 5052 :ok :read [4 3] INFO [2022-01-10 05:48:13,563] jepsen worker 12 - jepsen.util 5052 :invoke :write [4 23] INFO [2022-01-10 05:48:13,669] jepsen worker 79 - jepsen.util 5209 :info :write [26 10] :RequestTimeoutException INFO [2022-01-10 05:48:13,670] jepsen worker 79 - jepsen.util 5299 :invoke :write [26 13] INFO [2022-01-10 05:48:13,739] jepsen worker 82 - jepsen.util 4582 :info :write [27 9] :RequestTimeoutException INFO [2022-01-10 05:48:13,740] jepsen worker 82 - jepsen.util 4672 :invoke :read [27 nil] INFO [2022-01-10 05:48:13,847] jepsen worker 0 - jepsen.util 4590 :ok :read [0 46] INFO [2022-01-10 05:48:13,847] jepsen worker 0 - jepsen.util 4590 :invoke :write [0 30] INFO [2022-01-10 05:48:13,865] jepsen worker 85 - jepsen.util 4855 :info :write [28 32] :RequestTimeoutException INFO [2022-01-10 05:48:13,866] jepsen worker 85 - jepsen.util 4945 :invoke :write [28 32] INFO [2022-01-10 05:48:14,114] jepsen worker 67 - jepsen.util 4297 :ok :read [22 41] INFO [2022-01-10 05:48:14,114] jepsen worker 67 - jepsen.util 4297 :invoke :read [22 nil] INFO [2022-01-10 05:48:14,422] jepsen worker 48 - jepsen.util 4278 :info :write [16 19] :RequestTimeoutException INFO [2022-01-10 05:48:14,422] jepsen worker 48 - jepsen.util 4368 :invoke :write [16 43] INFO [2022-01-10 05:48:14,489] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5789473684210527}, :ns_1@172.23.107.227 {:progress 0.3988269794721407}, :ns_1@172.23.105.187 {:progress 0.3988269794721407}, :ns_1@172.23.107.228 {:progress 0.5529411764705883}} INFO [2022-01-10 05:48:14,489] jepsen worker nemesis - couchbase.util Rebalance check count: 641 INFO [2022-01-10 05:48:14,489] jepsen worker nemesis - couchbase.util Rebalance stuck count: 540 INFO [2022-01-10 05:48:14,522] jepsen worker 16 - jepsen.util 4516 :info :write [5 8] :RequestTimeoutException INFO [2022-01-10 05:48:14,523] jepsen worker 16 - jepsen.util 4606 :invoke :write [5 32] INFO [2022-01-10 05:48:14,625] jepsen worker 78 - jepsen.util 5568 :info :write [26 25] :RequestTimeoutException INFO [2022-01-10 05:48:14,625] jepsen worker 78 - jepsen.util 5658 :invoke :read [26 nil] INFO [2022-01-10 05:48:14,651] jepsen worker 21 - jepsen.util 4611 :ok :read [7 :nil] INFO [2022-01-10 05:48:14,651] jepsen worker 21 - jepsen.util 4611 :invoke :write [7 16] INFO [2022-01-10 05:48:14,736] jepsen worker 2 - jepsen.util 4682 :ok :read [0 46] INFO [2022-01-10 05:48:14,737] jepsen worker 2 - jepsen.util 4682 :invoke :write [0 1] INFO [2022-01-10 05:48:14,753] jepsen worker 57 - jepsen.util 5277 :ok :read [19 31] INFO [2022-01-10 05:48:14,754] jepsen worker 57 - jepsen.util 5277 :invoke :write [19 44] INFO [2022-01-10 05:48:14,812] jepsen worker 64 - jepsen.util 4564 :ok :read [21 7] INFO [2022-01-10 05:48:14,813] jepsen worker 64 - jepsen.util 4564 :invoke :read [21 nil] INFO [2022-01-10 05:48:14,937] jepsen worker 84 - jepsen.util 4314 :ok :read [28 45] INFO [2022-01-10 05:48:14,938] jepsen worker 84 - jepsen.util 4314 :invoke :write [28 43] INFO [2022-01-10 05:48:14,975] jepsen worker 42 - jepsen.util 4182 :info :write [14 29] :RequestTimeoutException INFO [2022-01-10 05:48:14,976] jepsen worker 42 - jepsen.util 4272 :invoke :read [14 nil] INFO [2022-01-10 05:48:15,092] jepsen worker 10 - jepsen.util 4330 :info :write [3 38] :RequestTimeoutException INFO [2022-01-10 05:48:15,092] jepsen worker 10 - jepsen.util 4420 :invoke :write [3 37] INFO [2022-01-10 05:48:15,121] jepsen worker 56 - jepsen.util 4736 :info :write [18 1] :RequestTimeoutException INFO [2022-01-10 05:48:15,121] jepsen worker 56 - jepsen.util 4826 :invoke :write [18 7] INFO [2022-01-10 05:48:15,186] jepsen worker 54 - jepsen.util 4734 :info :write [18 0] :RequestTimeoutException INFO [2022-01-10 05:48:15,187] jepsen worker 54 - jepsen.util 4824 :invoke :read [18 nil] INFO [2022-01-10 05:48:15,187] jepsen worker 14 - jepsen.util 5504 :ok :read [4 3] INFO [2022-01-10 05:48:15,188] jepsen worker 14 - jepsen.util 5504 :invoke :write [4 44] INFO [2022-01-10 05:48:15,212] jepsen worker 54 - jepsen.util 4824 :ok :read [18 14] INFO [2022-01-10 05:48:15,214] jepsen worker 54 - jepsen.util 4824 :invoke :read [18 nil] INFO [2022-01-10 05:48:15,239] jepsen worker 54 - jepsen.util 4824 :ok :read [18 14] INFO [2022-01-10 05:48:15,239] jepsen worker 54 - jepsen.util 4824 :invoke :read [18 nil] INFO [2022-01-10 05:48:15,265] jepsen worker 54 - jepsen.util 4824 :ok :read [18 14] INFO [2022-01-10 05:48:15,266] jepsen worker 54 - jepsen.util 4824 :invoke :write [18 34] INFO [2022-01-10 05:48:15,301] jepsen worker 59 - jepsen.util 4379 :info :write [19 40] :RequestTimeoutException INFO [2022-01-10 05:48:15,301] jepsen worker 59 - jepsen.util 4469 :invoke :read [19 nil] INFO [2022-01-10 05:48:15,396] jepsen worker 76 - jepsen.util 3856 :info :write [25 25] :RequestTimeoutException INFO [2022-01-10 05:48:15,396] jepsen worker 76 - jepsen.util 3946 :invoke :read [25 nil] INFO [2022-01-10 05:48:15,415] jepsen worker 7 - jepsen.util 4777 :info :write [2 48] :RequestTimeoutException INFO [2022-01-10 05:48:15,416] jepsen worker 7 - jepsen.util 4867 :invoke :write [2 1] INFO [2022-01-10 05:48:15,497] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5789473684210527}, :ns_1@172.23.107.227 {:progress 0.3988269794721407}, :ns_1@172.23.105.187 {:progress 0.3988269794721407}, :ns_1@172.23.107.228 {:progress 0.5529411764705883}} INFO [2022-01-10 05:48:15,498] jepsen worker nemesis - couchbase.util Rebalance check count: 642 INFO [2022-01-10 05:48:15,498] jepsen worker nemesis - couchbase.util Rebalance stuck count: 541 INFO [2022-01-10 05:48:15,517] jepsen worker 78 - jepsen.util 5658 :ok :read [26 10] INFO [2022-01-10 05:48:15,518] jepsen worker 78 - jepsen.util 5658 :invoke :read [26 nil] INFO [2022-01-10 05:48:15,866] jepsen worker 25 - jepsen.util 4165 :info :write [8 24] :RequestTimeoutException INFO [2022-01-10 05:48:15,867] jepsen worker 25 - jepsen.util 4255 :invoke :read [8 nil] INFO [2022-01-10 05:48:15,950] jepsen worker 70 - jepsen.util 4570 :ok :read [23 26] INFO [2022-01-10 05:48:15,950] jepsen worker 70 - jepsen.util 4570 :invoke :read [23 nil] INFO [2022-01-10 05:48:15,997] jepsen worker 63 - jepsen.util 4203 :info :write [21 14] :RequestTimeoutException INFO [2022-01-10 05:48:15,997] jepsen worker 63 - jepsen.util 4293 :invoke :read [21 nil] INFO [2022-01-10 05:48:16,006] jepsen worker 19 - jepsen.util 4609 :info :write [6 37] :RequestTimeoutException INFO [2022-01-10 05:48:16,006] jepsen worker 19 - jepsen.util 4699 :invoke :read [6 nil] INFO [2022-01-10 05:48:16,084] jepsen worker 78 - jepsen.util 5658 :ok :read [26 10] INFO [2022-01-10 05:48:16,085] jepsen worker 78 - jepsen.util 5658 :invoke :write [26 49] INFO [2022-01-10 05:48:16,245] jepsen worker 36 - jepsen.util 3366 :info :write [12 35] :RequestTimeoutException INFO [2022-01-10 05:48:16,246] jepsen worker 36 - jepsen.util 3456 :invoke :write [12 16] INFO [2022-01-10 05:48:16,290] jepsen worker 47 - jepsen.util 4727 :ok :read [15 23] INFO [2022-01-10 05:48:16,291] jepsen worker 47 - jepsen.util 4727 :invoke :read [15 nil] INFO [2022-01-10 05:48:16,309] jepsen worker 68 - jepsen.util 4748 :info :write [22 36] :RequestTimeoutException INFO [2022-01-10 05:48:16,310] jepsen worker 68 - jepsen.util 4838 :invoke :write [22 2] INFO [2022-01-10 05:48:16,321] jepsen worker 74 - jepsen.util 5024 :ok :read [24 18] INFO [2022-01-10 05:48:16,322] jepsen worker 74 - jepsen.util 5024 :invoke :read [24 nil] INFO [2022-01-10 05:48:16,326] jepsen worker 20 - jepsen.util 4160 :info :write [6 11] :RequestTimeoutException INFO [2022-01-10 05:48:16,327] jepsen worker 20 - jepsen.util 4250 :invoke :read [6 nil] INFO [2022-01-10 05:48:16,425] jepsen worker 70 - jepsen.util 4570 :ok :read [23 26] INFO [2022-01-10 05:48:16,425] jepsen worker 70 - jepsen.util 4570 :invoke :read [23 nil] INFO [2022-01-10 05:48:16,504] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5789473684210527}, :ns_1@172.23.107.227 {:progress 0.3988269794721407}, :ns_1@172.23.105.187 {:progress 0.3988269794721407}, :ns_1@172.23.107.228 {:progress 0.5529411764705883}} INFO [2022-01-10 05:48:16,504] jepsen worker nemesis - couchbase.util Rebalance check count: 643 INFO [2022-01-10 05:48:16,504] jepsen worker nemesis - couchbase.util Rebalance stuck count: 542 INFO [2022-01-10 05:48:16,806] jepsen worker 19 - jepsen.util 4699 :ok :read [6 25] INFO [2022-01-10 05:48:16,807] jepsen worker 19 - jepsen.util 4699 :invoke :read [6 nil] INFO [2022-01-10 05:48:16,895] jepsen worker 25 - jepsen.util 4255 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:48:16,895] jepsen worker 89 - jepsen.util 5039 :info :write [29 47] :SyncWriteAmbiguous INFO [2022-01-10 05:48:16,896] jepsen worker 25 - jepsen.util 4255 :invoke :write [8 47] INFO [2022-01-10 05:48:16,897] jepsen worker 89 - jepsen.util 5129 :invoke :read [29 nil] INFO [2022-01-10 05:48:16,930] jepsen worker 74 - jepsen.util 5024 :ok :read [24 18] INFO [2022-01-10 05:48:16,930] jepsen worker 74 - jepsen.util 5024 :invoke :read [24 nil] INFO [2022-01-10 05:48:16,932] jepsen worker 8 - jepsen.util 4688 :ok :read [2 14] INFO [2022-01-10 05:48:16,933] jepsen worker 8 - jepsen.util 4688 :invoke :write [2 5] INFO [2022-01-10 05:48:16,951] jepsen worker 41 - jepsen.util 4541 :info :write [13 17] :RequestTimeoutException INFO [2022-01-10 05:48:16,951] jepsen worker 41 - jepsen.util 4631 :invoke :read [13 nil] INFO [2022-01-10 05:48:16,960] jepsen worker 23 - jepsen.util 4613 :info :write [7 30] :RequestTimeoutException INFO [2022-01-10 05:48:16,961] jepsen worker 23 - jepsen.util 4703 :invoke :read [7 nil] INFO [2022-01-10 05:48:16,989] jepsen worker 87 - jepsen.util 4497 :ok :read [29 0] INFO [2022-01-10 05:48:16,990] jepsen worker 87 - jepsen.util 4497 :invoke :read [29 nil] INFO [2022-01-10 05:48:17,007] jepsen worker 31 - jepsen.util 4711 :ok :read [10 12] INFO [2022-01-10 05:48:17,007] jepsen worker 31 - jepsen.util 4711 :invoke :read [10 nil] INFO [2022-01-10 05:48:17,013] jepsen worker 32 - jepsen.util 4802 :ok :read [10 12] INFO [2022-01-10 05:48:17,013] jepsen worker 32 - jepsen.util 4802 :invoke :write [10 19] INFO [2022-01-10 05:48:17,015] jepsen worker 42 - jepsen.util 4272 :fail :read [14 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:48:17,015] jepsen worker 64 - jepsen.util 4564 :fail :read [21 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:48:17,017] jepsen worker 42 - jepsen.util 4272 :invoke :write [14 13] INFO [2022-01-10 05:48:17,017] jepsen worker 64 - jepsen.util 4564 :invoke :read [21 nil] INFO [2022-01-10 05:48:17,037] jepsen worker 31 - jepsen.util 4711 :ok :read [10 12] INFO [2022-01-10 05:48:17,037] jepsen worker 31 - jepsen.util 4711 :invoke :read [10 nil] INFO [2022-01-10 05:48:17,060] jepsen worker 6 - jepsen.util 4506 :ok :read [2 14] INFO [2022-01-10 05:48:17,061] jepsen worker 6 - jepsen.util 4506 :invoke :read [2 nil] INFO [2022-01-10 05:48:17,068] jepsen worker 46 - jepsen.util 4636 :ok :read [15 23] INFO [2022-01-10 05:48:17,068] jepsen worker 46 - jepsen.util 4636 :invoke :write [15 11] INFO [2022-01-10 05:48:17,126] jepsen worker 83 - jepsen.util 5123 :info :write [27 49] :RequestTimeoutException INFO [2022-01-10 05:48:17,127] jepsen worker 83 - jepsen.util 5213 :invoke :write [27 18] INFO [2022-01-10 05:48:17,138] jepsen worker 31 - jepsen.util 4711 :ok :read [10 12] INFO [2022-01-10 05:48:17,139] jepsen worker 31 - jepsen.util 4711 :invoke :read [10 nil] INFO [2022-01-10 05:48:17,170] jepsen worker 31 - jepsen.util 4711 :ok :read [10 12] INFO [2022-01-10 05:48:17,170] jepsen worker 31 - jepsen.util 4711 :invoke :write [10 23] INFO [2022-01-10 05:48:17,179] jepsen worker 74 - jepsen.util 5024 :ok :read [24 18] INFO [2022-01-10 05:48:17,180] jepsen worker 74 - jepsen.util 5024 :invoke :write [24 10] INFO [2022-01-10 05:48:17,431] jepsen worker 65 - jepsen.util 4205 :info :write [21 5] :RequestTimeoutException INFO [2022-01-10 05:48:17,432] jepsen worker 65 - jepsen.util 4295 :invoke :write [21 9] INFO [2022-01-10 05:48:17,497] jepsen worker 70 - jepsen.util 4570 :ok :read [23 26] INFO [2022-01-10 05:48:17,497] jepsen worker 70 - jepsen.util 4570 :invoke :read [23 nil] INFO [2022-01-10 05:48:17,509] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5847953216374269}, :ns_1@172.23.107.227 {:progress 0.4046920821114369}, :ns_1@172.23.105.187 {:progress 0.4046920821114369}, :ns_1@172.23.107.228 {:progress 0.5588235294117647}} INFO [2022-01-10 05:48:17,509] jepsen worker nemesis - couchbase.util Rebalance check count: 644 INFO [2022-01-10 05:48:17,509] jepsen worker nemesis - couchbase.util Rebalance stuck count: 542 INFO [2022-01-10 05:48:17,535] jepsen worker 29 - jepsen.util 4349 :info :write [9 37] :RequestTimeoutException INFO [2022-01-10 05:48:17,536] jepsen worker 29 - jepsen.util 4439 :invoke :write [9 27] INFO [2022-01-10 05:48:17,638] jepsen worker 70 - jepsen.util 4570 :ok :read [23 26] INFO [2022-01-10 05:48:17,638] jepsen worker 70 - jepsen.util 4570 :invoke :read [23 nil] INFO [2022-01-10 05:48:17,669] jepsen worker 49 - jepsen.util 4279 :info :write [16 26] :RequestTimeoutException INFO [2022-01-10 05:48:17,670] jepsen worker 49 - jepsen.util 4369 :invoke :write [16 22] INFO [2022-01-10 05:48:17,794] jepsen worker 30 - jepsen.util 4170 :info :write [10 41] :RequestTimeoutException INFO [2022-01-10 05:48:17,795] jepsen worker 30 - jepsen.util 4260 :invoke :read [10 nil] INFO [2022-01-10 05:48:17,891] jepsen worker 23 - jepsen.util 4703 :fail :read [7 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:48:17,891] jepsen worker 23 - jepsen.util 4703 :invoke :write [7 6] INFO [2022-01-10 05:48:17,906] jepsen worker 19 - jepsen.util 4699 :ok :read [6 25] INFO [2022-01-10 05:48:17,907] jepsen worker 19 - jepsen.util 4699 :invoke :write [6 29] INFO [2022-01-10 05:48:17,928] jepsen worker 20 - jepsen.util 4250 :ok :read [6 25] INFO [2022-01-10 05:48:17,929] jepsen worker 20 - jepsen.util 4250 :invoke :write [6 29] INFO [2022-01-10 05:48:17,975] jepsen worker 11 - jepsen.util 4331 :info :write [3 8] :RequestTimeoutException INFO [2022-01-10 05:48:17,975] jepsen worker 11 - jepsen.util 4421 :invoke :write [3 39] INFO [2022-01-10 05:48:17,986] jepsen worker 33 - jepsen.util 4713 :info :write [11 42] :RequestTimeoutException INFO [2022-01-10 05:48:17,987] jepsen worker 33 - jepsen.util 4803 :invoke :write [11 37] INFO [2022-01-10 05:48:17,991] jepsen worker 44 - jepsen.util 4634 :info :write [14 45] :RequestTimeoutException INFO [2022-01-10 05:48:17,991] jepsen worker 44 - jepsen.util 4724 :invoke :read [14 nil] INFO [2022-01-10 05:48:18,336] jepsen worker 66 - jepsen.util 4296 :info :write [22 16] :RequestTimeoutException INFO [2022-01-10 05:48:18,336] jepsen worker 66 - jepsen.util 4386 :invoke :write [22 31] INFO [2022-01-10 05:48:18,341] jepsen worker 70 - jepsen.util 4570 :ok :read [23 26] INFO [2022-01-10 05:48:18,341] jepsen worker 70 - jepsen.util 4570 :invoke :write [23 41] INFO [2022-01-10 05:48:18,372] jepsen worker 35 - jepsen.util 4265 :info :write [11 36] :RequestTimeoutException INFO [2022-01-10 05:48:18,372] jepsen worker 35 - jepsen.util 4355 :invoke :read [11 nil] INFO [2022-01-10 05:48:18,422] jepsen worker 72 - jepsen.util 4752 :info :write [24 24] :RequestTimeoutException INFO [2022-01-10 05:48:18,422] jepsen worker 72 - jepsen.util 4842 :invoke :write [24 15] INFO [2022-01-10 05:48:18,426] jepsen worker 75 - jepsen.util 4305 :info :write [25 30] :RequestTimeoutException INFO [2022-01-10 05:48:18,427] jepsen worker 75 - jepsen.util 4395 :invoke :write [25 26] INFO [2022-01-10 05:48:18,515] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5847953216374269}, :ns_1@172.23.107.227 {:progress 0.4046920821114369}, :ns_1@172.23.105.187 {:progress 0.4046920821114369}, :ns_1@172.23.107.228 {:progress 0.5588235294117647}} INFO [2022-01-10 05:48:18,515] jepsen worker nemesis - couchbase.util Rebalance check count: 645 INFO [2022-01-10 05:48:18,515] jepsen worker nemesis - couchbase.util Rebalance stuck count: 543 INFO [2022-01-10 05:48:18,956] jepsen worker 43 - jepsen.util 4543 :info :write [14 2] :RequestTimeoutException INFO [2022-01-10 05:48:18,957] jepsen worker 43 - jepsen.util 4633 :invoke :write [14 9] INFO [2022-01-10 05:48:19,020] jepsen worker 88 - jepsen.util 4588 :info :write [29 14] :RequestTimeoutException INFO [2022-01-10 05:48:19,021] jepsen worker 88 - jepsen.util 4678 :invoke :read [29 nil] INFO [2022-01-10 05:48:19,046] jepsen worker 64 - jepsen.util 4564 :ok :read [21 7] INFO [2022-01-10 05:48:19,047] jepsen worker 64 - jepsen.util 4564 :invoke :write [21 28] INFO [2022-01-10 05:48:19,051] jepsen worker 63 - jepsen.util 4293 :ok :read [21 7] INFO [2022-01-10 05:48:19,052] jepsen worker 63 - jepsen.util 4293 :invoke :read [21 nil] INFO [2022-01-10 05:48:19,082] jepsen worker 40 - jepsen.util 4180 :info :write [13 36] :RequestTimeoutException INFO [2022-01-10 05:48:19,083] jepsen worker 40 - jepsen.util 4270 :invoke :read [13 nil] INFO [2022-01-10 05:48:19,106] jepsen worker 81 - jepsen.util 5031 :info :write [27 32] :RequestTimeoutException INFO [2022-01-10 05:48:19,107] jepsen worker 81 - jepsen.util 5121 :invoke :write [27 42] INFO [2022-01-10 05:48:19,520] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5847953216374269}, :ns_1@172.23.107.227 {:progress 0.4046920821114369}, :ns_1@172.23.105.187 {:progress 0.4046920821114369}, :ns_1@172.23.107.228 {:progress 0.5588235294117647}} INFO [2022-01-10 05:48:19,520] jepsen worker nemesis - couchbase.util Rebalance check count: 646 INFO [2022-01-10 05:48:19,520] jepsen worker nemesis - couchbase.util Rebalance stuck count: 544 INFO [2022-01-10 05:48:19,535] jepsen worker 34 - jepsen.util 4354 :info :write [11 41] :RequestTimeoutException INFO [2022-01-10 05:48:19,536] jepsen worker 34 - jepsen.util 4444 :invoke :read [11 nil] INFO [2022-01-10 05:48:19,545] jepsen worker 24 - jepsen.util 4164 :info :write [8 48] :RequestTimeoutException INFO [2022-01-10 05:48:19,546] jepsen worker 24 - jepsen.util 4254 :invoke :read [8 nil] INFO [2022-01-10 05:48:19,572] jepsen worker 26 - jepsen.util 4256 :info :write [8 18] :RequestTimeoutException INFO [2022-01-10 05:48:19,572] jepsen worker 26 - jepsen.util 4346 :invoke :write [8 46] INFO [2022-01-10 05:48:19,762] jepsen worker 13 - jepsen.util 5233 :info :write [4 8] :RequestTimeoutException INFO [2022-01-10 05:48:19,763] jepsen worker 13 - jepsen.util 5323 :invoke :write [4 37] INFO [2022-01-10 05:48:19,771] jepsen worker 52 - jepsen.util 5632 :info :write [17 20] :RequestTimeoutException INFO [2022-01-10 05:48:19,771] jepsen worker 52 - jepsen.util 5722 :invoke :read [17 nil] INFO [2022-01-10 05:48:19,800] jepsen worker 77 - jepsen.util 5027 :info :write [25 4] :RequestTimeoutException INFO [2022-01-10 05:48:19,801] jepsen worker 77 - jepsen.util 5117 :invoke :read [25 nil] INFO [2022-01-10 05:48:19,827] jepsen worker 18 - jepsen.util 4428 :info :write [6 45] :RequestTimeoutException INFO [2022-01-10 05:48:19,827] jepsen worker 18 - jepsen.util 4518 :invoke :read [6 nil] INFO [2022-01-10 05:48:19,961] jepsen worker 53 - jepsen.util 5453 :info :write [17 38] :RequestTimeoutException INFO [2022-01-10 05:48:19,962] jepsen worker 53 - jepsen.util 5543 :invoke :write [17 44] INFO [2022-01-10 05:48:19,970] jepsen worker 80 - jepsen.util 5570 :info :write [26 15] :RequestTimeoutException INFO [2022-01-10 05:48:19,971] jepsen worker 80 - jepsen.util 5660 :invoke :write [26 48] INFO [2022-01-10 05:48:20,121] jepsen worker 60 - jepsen.util 4830 :info :write [20 7] :RequestTimeoutException INFO [2022-01-10 05:48:20,121] jepsen worker 60 - jepsen.util 4920 :invoke :read [20 nil] INFO [2022-01-10 05:48:20,122] jepsen worker 5 - jepsen.util 4505 :info :write [1 0] :RequestTimeoutException INFO [2022-01-10 05:48:20,122] jepsen worker 5 - jepsen.util 4595 :invoke :read [1 nil] INFO [2022-01-10 05:48:20,307] jepsen worker 38 - jepsen.util 4088 :info :write [12 32] :RequestTimeoutException INFO [2022-01-10 05:48:20,307] jepsen worker 38 - jepsen.util 4178 :invoke :write [12 32] INFO [2022-01-10 05:48:20,437] jepsen worker 62 - jepsen.util 4922 :ok :read [20 :nil] INFO [2022-01-10 05:48:20,438] jepsen worker 62 - jepsen.util 4922 :invoke :read [20 nil] INFO [2022-01-10 05:48:20,501] jepsen worker 45 - jepsen.util 4995 :info :write [15 45] :RequestTimeoutException INFO [2022-01-10 05:48:20,501] jepsen worker 45 - jepsen.util 5085 :invoke :read [15 nil] INFO [2022-01-10 05:48:20,521] jepsen worker 4 - jepsen.util 4774 :info :write [1 31] :RequestTimeoutException INFO [2022-01-10 05:48:20,522] jepsen worker 4 - jepsen.util 4864 :invoke :read [1 nil] INFO [2022-01-10 05:48:20,528] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5847953216374269}, :ns_1@172.23.107.227 {:progress 0.4105571847507331}, :ns_1@172.23.105.187 {:progress 0.4105571847507331}, :ns_1@172.23.107.228 {:progress 0.5588235294117647}} INFO [2022-01-10 05:48:20,528] jepsen worker nemesis - couchbase.util Rebalance check count: 647 INFO [2022-01-10 05:48:20,528] jepsen worker nemesis - couchbase.util Rebalance stuck count: 544 INFO [2022-01-10 05:48:20,532] jepsen worker 3 - jepsen.util 4413 :info :write [1 26] :RequestTimeoutException INFO [2022-01-10 05:48:20,533] jepsen worker 3 - jepsen.util 4503 :invoke :write [1 8] INFO [2022-01-10 05:48:20,624] jepsen worker 39 - jepsen.util 4899 :info :write [13 24] :RequestTimeoutException INFO [2022-01-10 05:48:20,625] jepsen worker 39 - jepsen.util 4989 :invoke :write [13 47] INFO [2022-01-10 05:48:20,667] jepsen worker 88 - jepsen.util 4678 :ok :read [29 0] INFO [2022-01-10 05:48:20,668] jepsen worker 88 - jepsen.util 4678 :invoke :write [29 11] WARN [2022-01-10 05:48:20,744] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10733889,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15038984,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":10575066,"encode_duration_us":16581,"operation_id":"0x8b97","timeout_ms":10000,"last_server_duration_us":9844924,"total_duration_us":10004728},{"operation_name":"upsert","last_dispatch_duration_us":12726205,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13722988,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":12064638,"encode_duration_us":20792,"operation_id":"0x8b11","timeout_ms":10000,"last_server_duration_us":11761350,"total_duration_us":10008133}],"total_count":2}} WARN [2022-01-10 05:48:20,744] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":662951,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":3588877,"total_server_duration_us":1103364,"encode_duration_us":16227,"operation_id":"0x8c2a","timeout_ms":10000,"last_server_duration_us":107269,"total_duration_us":10009233},{"operation_name":"upsert","last_dispatch_duration_us":510807,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":5568512,"total_server_duration_us":2693313,"encode_duration_us":19220,"operation_id":"0x8c97","timeout_ms":10000,"last_server_duration_us":242474,"total_duration_us":10009168},{"operation_name":"upsert","last_dispatch_duration_us":1012247,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2998681,"total_server_duration_us":544895,"encode_duration_us":22043,"operation_id":"0x8d27","timeout_ms":10000,"last_server_duration_us":288122,"total_duration_us":10009097},{"operation_name":"upsert","last_dispatch_duration_us":8779265,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9083378,"total_server_duration_us":146824,"encode_duration_us":16810,"operation_id":"0x8cdb","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10006529},{"operation_name":"upsert","last_dispatch_duration_us":8880666,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11573026,"total_server_duration_us":2228568,"encode_duration_us":16638,"operation_id":"0x8cc5","timeout_ms":10000,"last_server_duration_us":393725,"total_duration_us":10006179},{"operation_name":"upsert","last_dispatch_duration_us":8788930,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":8927775,"total_server_duration_us":88080,"encode_duration_us":17703,"operation_id":"0x8ced","timeout_ms":10000,"last_server_duration_us":27043,"total_duration_us":10004999},{"operation_name":"upsert","last_dispatch_duration_us":10733889,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":15038984,"total_server_duration_us":10575066,"encode_duration_us":16581,"operation_id":"0x8b97","timeout_ms":10000,"last_server_duration_us":9844924,"total_duration_us":10004728},{"operation_name":"upsert","last_dispatch_duration_us":1835031,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":5309708,"total_server_duration_us":3836020,"encode_duration_us":19834,"operation_id":"0x8c37","timeout_ms":10000,"last_server_duration_us":1144837,"total_duration_us":10003746},{"operation_name":"upsert","last_dispatch_duration_us":9557439,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11249643,"total_server_duration_us":431065,"encode_duration_us":16914,"operation_id":"0x8c59","timeout_ms":10000,"last_server_duration_us":68414,"total_duration_us":10002723},{"operation_name":"upsert","last_dispatch_duration_us":9798437,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":13510950,"total_server_duration_us":1835504,"encode_duration_us":17025,"operation_id":"0x8bd2","timeout_ms":10000,"last_server_duration_us":1010421,"total_duration_us":10001968}],"total_count":16}} INFO [2022-01-10 05:48:20,836] jepsen worker 17 - jepsen.util 4337 :info :write [5 33] :RequestTimeoutException INFO [2022-01-10 05:48:20,837] jepsen worker 17 - jepsen.util 4427 :invoke :write [5 3] INFO [2022-01-10 05:48:21,490] jepsen worker 61 - jepsen.util 4831 :info :write [20 45] :RequestTimeoutException INFO [2022-01-10 05:48:21,490] jepsen worker 61 - jepsen.util 4921 :invoke :read [20 nil] INFO [2022-01-10 05:48:21,502] jepsen worker 15 - jepsen.util 4065 :info :write [5 12] :RequestTimeoutException INFO [2022-01-10 05:48:21,532] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5847953216374269}, :ns_1@172.23.107.227 {:progress 0.4105571847507331}, :ns_1@172.23.105.187 {:progress 0.4105571847507331}, :ns_1@172.23.107.228 {:progress 0.5588235294117647}} INFO [2022-01-10 05:48:21,661] jepsen worker 69 - jepsen.util 4119 :info :write [23 27] :RequestTimeoutException INFO [2022-01-10 05:48:21,905] jepsen worker nemesis - couchbase.util Rebalance check count: 648 INFO [2022-01-10 05:48:21,756] jepsen worker 86 - jepsen.util 3866 :info :write [28 31] :RequestTimeoutException INFO [2022-01-10 05:48:21,905] jepsen worker nemesis - couchbase.util Rebalance stuck count: 545 INFO [2022-01-10 05:48:21,905] jepsen worker 86 - jepsen.util 3956 :invoke :read [28 nil] INFO [2022-01-10 05:48:21,905] jepsen worker 69 - jepsen.util 4209 :invoke :read [23 nil] INFO [2022-01-10 05:48:21,905] jepsen worker 15 - jepsen.util 4155 :invoke :write [5 0] INFO [2022-01-10 05:48:21,992] jepsen worker 82 - jepsen.util 4672 :ok :read [27 30] INFO [2022-01-10 05:48:21,993] jepsen worker 82 - jepsen.util 4672 :invoke :write [27 38] INFO [2022-01-10 05:48:22,026] jepsen worker 9 - jepsen.util 5049 :info :write [3 46] :RequestTimeoutException INFO [2022-01-10 05:48:22,027] jepsen worker 9 - jepsen.util 5139 :invoke :read [3 nil] INFO [2022-01-10 05:48:22,046] jepsen worker 9 - jepsen.util 5139 :ok :read [3 12] INFO [2022-01-10 05:48:22,047] jepsen worker 9 - jepsen.util 5139 :invoke :write [3 15] INFO [2022-01-10 05:48:22,055] jepsen worker 73 - jepsen.util 4303 :info :write [24 6] :RequestTimeoutException INFO [2022-01-10 05:48:22,056] jepsen worker 73 - jepsen.util 4393 :invoke :write [24 35] INFO [2022-01-10 05:48:22,068] jepsen worker 4 - jepsen.util 4864 :ok :read [1 49] INFO [2022-01-10 05:48:22,069] jepsen worker 4 - jepsen.util 4864 :invoke :read [1 nil] INFO [2022-01-10 05:48:22,074] jepsen worker 52 - jepsen.util 5722 :ok :read [17 41] INFO [2022-01-10 05:48:22,076] jepsen worker 52 - jepsen.util 5722 :invoke :write [17 13] WARN [2022-01-10 05:48:22,246] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9048962,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9489305,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9293714,"encode_duration_us":17057,"operation_id":"0x8c64","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10011135}],"total_count":1}} WARN [2022-01-10 05:48:22,260] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":502199,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":6734870,"total_server_duration_us":2445892,"encode_duration_us":16896,"operation_id":"0x8cb9","timeout_ms":10000,"last_server_duration_us":488216,"total_duration_us":10009962},{"operation_name":"upsert","last_dispatch_duration_us":2985506,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":11959108,"total_server_duration_us":1949289,"encode_duration_us":20766,"operation_id":"0x8d50","timeout_ms":10000,"last_server_duration_us":396529,"total_duration_us":10009774},{"operation_name":"upsert","last_dispatch_duration_us":11867409,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":11867409,"total_server_duration_us":21977,"encode_duration_us":17576,"operation_id":"0x8c5c","timeout_ms":10000,"last_server_duration_us":21977,"total_duration_us":10009476},{"operation_name":"upsert","last_dispatch_duration_us":2408737,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9055298,"total_server_duration_us":2257806,"encode_duration_us":18465,"operation_id":"0x8cef","timeout_ms":10000,"last_server_duration_us":157070,"total_duration_us":10008933},{"operation_name":"upsert","last_dispatch_duration_us":2128782,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2128782,"total_server_duration_us":219713,"encode_duration_us":17509,"operation_id":"0x8d5c","timeout_ms":10000,"last_server_duration_us":219713,"total_duration_us":10006740},{"operation_name":"upsert","last_dispatch_duration_us":200973,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7462353,"total_server_duration_us":2691387,"encode_duration_us":19820,"operation_id":"0x8c66","timeout_ms":10000,"last_server_duration_us":190041,"total_duration_us":10005456},{"operation_name":"upsert","last_dispatch_duration_us":2966969,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10505492,"total_server_duration_us":1754799,"encode_duration_us":16270,"operation_id":"0x8d51","timeout_ms":10000,"last_server_duration_us":604251,"total_duration_us":10003929},{"operation_name":"upsert","last_dispatch_duration_us":2725710,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10635079,"total_server_duration_us":2218752,"encode_duration_us":16766,"operation_id":"0x8d5e","timeout_ms":10000,"last_server_duration_us":366425,"total_duration_us":10003522},{"operation_name":"upsert","last_dispatch_duration_us":2849169,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":11727743,"total_server_duration_us":1408314,"encode_duration_us":17486,"operation_id":"0x8d41","timeout_ms":10000,"last_server_duration_us":353222,"total_duration_us":10003024},{"operation_name":"upsert","last_dispatch_duration_us":3075372,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9096147,"total_server_duration_us":1704563,"encode_duration_us":17144,"operation_id":"0x8d8b","timeout_ms":10000,"last_server_duration_us":231198,"total_duration_us":10002249}],"total_count":21}} WARN [2022-01-10 05:48:22,260] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16961666,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18155380,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9383246,"encode_duration_us":16689,"operation_id":"0x8a6f","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10006904}],"total_count":1}} WARN [2022-01-10 05:48:22,343] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11946425,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":14354385,"total_server_duration_us":405386,"encode_duration_us":18424,"operation_id":"0x8cbd","timeout_ms":10000,"last_server_duration_us":201269,"total_duration_us":10010462},{"operation_name":"upsert","last_dispatch_duration_us":1264973,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":2837810,"total_server_duration_us":137693,"encode_duration_us":21096,"operation_id":"0x8bd0","timeout_ms":10000,"last_server_duration_us":55061,"total_duration_us":10009006},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","encode_duration_us":18008,"operation_id":"0x8d4f","timeout_ms":10000,"total_duration_us":10008470},{"operation_name":"upsert","last_dispatch_duration_us":1540896,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":11066117,"total_server_duration_us":1948633,"encode_duration_us":20684,"operation_id":"0x8d5b","timeout_ms":10000,"last_server_duration_us":199379,"total_duration_us":10007855},{"operation_name":"upsert","last_dispatch_duration_us":2177687,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4447701,"total_server_duration_us":541989,"encode_duration_us":24476,"operation_id":"0x8cee","timeout_ms":10000,"last_server_duration_us":390930,"total_duration_us":10007030},{"operation_name":"upsert","last_dispatch_duration_us":1239798,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":3019309,"total_server_duration_us":122176,"encode_duration_us":17575,"operation_id":"0x8c05","timeout_ms":10000,"last_server_duration_us":49255,"total_duration_us":10006670},{"operation_name":"upsert","last_dispatch_duration_us":17757944,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":18013211,"total_server_duration_us":9064725,"encode_duration_us":17018,"operation_id":"0x8bd6","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10006387},{"operation_name":"upsert","last_dispatch_duration_us":3062497,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":3062497,"total_server_duration_us":1041555,"encode_duration_us":19873,"operation_id":"0x8cfc","timeout_ms":10000,"last_server_duration_us":1041555,"total_duration_us":10005707},{"operation_name":"upsert","last_dispatch_duration_us":31215,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9413076,"total_server_duration_us":198539,"encode_duration_us":19604,"operation_id":"0x8bd4","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10005534},{"operation_name":"upsert","last_dispatch_duration_us":70727,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9453928,"total_server_duration_us":252205,"encode_duration_us":17125,"operation_id":"0x8bd7","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10004979}],"total_count":16}} WARN [2022-01-10 05:48:22,346] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13412650,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":13412650,"total_server_duration_us":6977,"operation_id":"0x8b3c","timeout_ms":30000,"last_server_duration_us":6977,"total_duration_us":13414962},{"operation_name":"get","last_dispatch_duration_us":2559956,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2559956,"total_server_duration_us":5674,"operation_id":"0x8cc6","timeout_ms":30000,"last_server_duration_us":5674,"total_duration_us":12830510},{"operation_name":"get","last_dispatch_duration_us":10165204,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10165204,"total_server_duration_us":12247,"operation_id":"0x8bd5","timeout_ms":30000,"last_server_duration_us":12247,"total_duration_us":10167331},{"operation_name":"get","last_dispatch_duration_us":10082018,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10082018,"total_server_duration_us":49,"operation_id":"0x8bd3","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":10084080},{"operation_name":"upsert","last_dispatch_duration_us":9048962,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9489305,"total_server_duration_us":9293714,"encode_duration_us":17057,"operation_id":"0x8c64","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10011135},{"operation_name":"upsert","last_dispatch_duration_us":3357004,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":5423113,"total_server_duration_us":510419,"encode_duration_us":16634,"operation_id":"0x8d9e","timeout_ms":10000,"last_server_duration_us":320994,"total_duration_us":10009069},{"operation_name":"upsert","last_dispatch_duration_us":3494158,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3494158,"total_server_duration_us":231421,"encode_duration_us":17072,"operation_id":"0x8c8f","timeout_ms":10000,"last_server_duration_us":231421,"total_duration_us":10008551},{"operation_name":"upsert","last_dispatch_duration_us":6722307,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":11394874,"total_server_duration_us":357203,"encode_duration_us":21990,"operation_id":"0x8c68","timeout_ms":10000,"last_server_duration_us":103434,"total_duration_us":10008265},{"operation_name":"upsert","last_dispatch_duration_us":2520338,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7300628,"total_server_duration_us":592563,"encode_duration_us":16681,"operation_id":"0x8cf8","timeout_ms":10000,"last_server_duration_us":402727,"total_duration_us":10006814},{"operation_name":"upsert","last_dispatch_duration_us":3074640,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5320727,"total_server_duration_us":343381,"encode_duration_us":19199,"operation_id":"0x8c8a","timeout_ms":10000,"last_server_duration_us":268319,"total_duration_us":10003892}],"total_count":25}} WARN [2022-01-10 05:48:22,355] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":6438201,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6438201,"total_server_duration_us":119129,"operation_id":"0x8cbb","timeout_ms":30000,"last_server_duration_us":119129,"total_duration_us":10077913},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":19707,"operation_id":"0x8d89","timeout_ms":10000,"total_duration_us":10010545},{"operation_name":"upsert","last_dispatch_duration_us":11220306,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":15992807,"total_server_duration_us":1235624,"encode_duration_us":16162,"operation_id":"0x8c35","timeout_ms":10000,"last_server_duration_us":791396,"total_duration_us":10009512},{"operation_name":"upsert","last_dispatch_duration_us":7138971,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7138971,"total_server_duration_us":975569,"encode_duration_us":21887,"operation_id":"0x8cfb","timeout_ms":10000,"last_server_duration_us":975569,"total_duration_us":10009356},{"operation_name":"upsert","last_dispatch_duration_us":37179,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":947144,"total_server_duration_us":586900,"encode_duration_us":19182,"operation_id":"0x8c5a","timeout_ms":10000,"last_server_duration_us":25800,"total_duration_us":10009216},{"operation_name":"upsert","last_dispatch_duration_us":7230173,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7230173,"total_server_duration_us":375706,"encode_duration_us":17532,"operation_id":"0x8d29","timeout_ms":10000,"last_server_duration_us":375706,"total_duration_us":10008638},{"operation_name":"upsert","last_dispatch_duration_us":6571578,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6571578,"total_server_duration_us":81398,"encode_duration_us":17361,"operation_id":"0x8cf9","timeout_ms":10000,"last_server_duration_us":81398,"total_duration_us":10007898},{"operation_name":"upsert","last_dispatch_duration_us":2315997,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":10570001,"total_server_duration_us":2207964,"encode_duration_us":16765,"operation_id":"0x8d5f","timeout_ms":10000,"last_server_duration_us":376256,"total_duration_us":10006149},{"operation_name":"upsert","last_dispatch_duration_us":11353568,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":14892481,"total_server_duration_us":10506633,"encode_duration_us":16544,"operation_id":"0x8c34","timeout_ms":10000,"last_server_duration_us":10048338,"total_duration_us":10004086},{"operation_name":"upsert","last_dispatch_duration_us":10818255,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10818255,"total_server_duration_us":425032,"encode_duration_us":16259,"operation_id":"0x8d1e","timeout_ms":10000,"last_server_duration_us":425032,"total_duration_us":10003765}],"total_count":14}} WARN [2022-01-10 05:48:22,362] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9225419,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":11312110,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":9465388,"encode_duration_us":18130,"operation_id":"0x8c00","timeout_ms":10000,"last_server_duration_us":9214281,"total_duration_us":10002279},{"operation_name":"upsert","last_dispatch_duration_us":9126821,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":11316452,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":9634361,"encode_duration_us":19138,"operation_id":"0x8c01","timeout_ms":10000,"last_server_duration_us":9113921,"total_duration_us":10007879}],"total_count":2}} INFO [2022-01-10 05:48:22,450] jepsen worker 58 - jepsen.util 4468 :info :write [19 49] :RequestTimeoutException INFO [2022-01-10 05:48:22,451] jepsen worker 58 - jepsen.util 4558 :invoke :write [19 34] WARN [2022-01-10 05:48:22,455] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11353568,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14892481,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10506633,"encode_duration_us":16544,"operation_id":"0x8c34","timeout_ms":10000,"last_server_duration_us":10048338,"total_duration_us":10004086}],"total_count":1}} INFO [2022-01-10 05:48:22,516] jepsen worker 1 - jepsen.util 4501 :info :write [0 43] :RequestTimeoutException INFO [2022-01-10 05:48:22,516] jepsen worker 1 - jepsen.util 4591 :invoke :write [0 28] INFO [2022-01-10 05:48:22,526] jepsen worker 55 - jepsen.util 5095 :info :write [18 23] :RequestTimeoutException INFO [2022-01-10 05:48:22,526] jepsen worker 55 - jepsen.util 5185 :invoke :read [18 nil] WARN [2022-01-10 05:48:22,544] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10564323,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":12507708,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":11385015,"encode_duration_us":20723,"operation_id":"0x8b49","timeout_ms":10000,"last_server_duration_us":10552115,"total_duration_us":10003329},{"operation_name":"upsert","last_dispatch_duration_us":11139240,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12756566,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":10055279,"encode_duration_us":17308,"operation_id":"0x8b76","timeout_ms":10000,"last_server_duration_us":9895612,"total_duration_us":10004892},{"operation_name":"upsert","last_dispatch_duration_us":17757944,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18013211,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9064725,"encode_duration_us":17018,"operation_id":"0x8bd6","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10006387}],"total_count":3}} WARN [2022-01-10 05:48:22,562] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1157382,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8746310,"total_server_duration_us":1530923,"encode_duration_us":18719,"operation_id":"0x8d48","timeout_ms":10000,"last_server_duration_us":363984,"total_duration_us":10009868},{"operation_name":"upsert","last_dispatch_duration_us":9126821,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":11316452,"total_server_duration_us":9634361,"encode_duration_us":19138,"operation_id":"0x8c01","timeout_ms":10000,"last_server_duration_us":9113921,"total_duration_us":10007879},{"operation_name":"upsert","last_dispatch_duration_us":1586938,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":6788942,"total_server_duration_us":1237848,"encode_duration_us":19229,"operation_id":"0x8c16","timeout_ms":10000,"last_server_duration_us":567129,"total_duration_us":10006629},{"operation_name":"upsert","last_dispatch_duration_us":32311,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":757452,"total_server_duration_us":455676,"encode_duration_us":16384,"operation_id":"0x8d1d","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10006360},{"operation_name":"upsert","last_dispatch_duration_us":7740946,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10097706,"total_server_duration_us":920345,"encode_duration_us":16447,"operation_id":"0x8c07","timeout_ms":10000,"last_server_duration_us":306505,"total_duration_us":10005376},{"operation_name":"upsert","last_dispatch_duration_us":2679035,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9921326,"total_server_duration_us":1218888,"encode_duration_us":22758,"operation_id":"0x8c99","timeout_ms":10000,"last_server_duration_us":265471,"total_duration_us":10004548},{"operation_name":"upsert","last_dispatch_duration_us":7699604,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10062174,"total_server_duration_us":561164,"encode_duration_us":24341,"operation_id":"0x8c02","timeout_ms":10000,"last_server_duration_us":99970,"total_duration_us":10003977},{"operation_name":"upsert","last_dispatch_duration_us":3786927,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11241263,"total_server_duration_us":2535254,"encode_duration_us":16166,"operation_id":"0x8cc7","timeout_ms":10000,"last_server_duration_us":618090,"total_duration_us":10002384},{"operation_name":"upsert","last_dispatch_duration_us":9225419,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":11312110,"total_server_duration_us":9465388,"encode_duration_us":18130,"operation_id":"0x8c00","timeout_ms":10000,"last_server_duration_us":9214281,"total_duration_us":10002279},{"operation_name":"upsert","last_dispatch_duration_us":1103342,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7656359,"total_server_duration_us":1797372,"encode_duration_us":16542,"operation_id":"0x8cba","timeout_ms":10000,"last_server_duration_us":287142,"total_duration_us":10002140}],"total_count":19}} INFO [2022-01-10 05:48:22,676] jepsen worker 5 - jepsen.util 4595 :ok :read [1 49] INFO [2022-01-10 05:48:22,676] jepsen worker 5 - jepsen.util 4595 :invoke :write [1 49] INFO [2022-01-10 05:48:22,680] jepsen worker 35 - jepsen.util 4355 :ok :read [11 21] INFO [2022-01-10 05:48:22,681] jepsen worker 35 - jepsen.util 4355 :invoke :read [11 nil] INFO [2022-01-10 05:48:22,910] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5847953216374269}, :ns_1@172.23.107.227 {:progress 0.4105571847507331}, :ns_1@172.23.105.187 {:progress 0.4105571847507331}, :ns_1@172.23.107.228 {:progress 0.5588235294117647}} INFO [2022-01-10 05:48:22,910] jepsen worker nemesis - couchbase.util Rebalance check count: 649 INFO [2022-01-10 05:48:22,910] jepsen worker nemesis - couchbase.util Rebalance stuck count: 546 INFO [2022-01-10 05:48:22,955] jepsen worker 27 - jepsen.util 4797 :info :write [9 30] :RequestTimeoutException INFO [2022-01-10 05:48:22,956] jepsen worker 27 - jepsen.util 4887 :invoke :write [9 25] INFO [2022-01-10 05:48:23,065] jepsen worker 22 - jepsen.util 4432 :ok :read [7 :nil] INFO [2022-01-10 05:48:23,067] jepsen worker 22 - jepsen.util 4432 :invoke :read [7 nil] INFO [2022-01-10 05:48:23,089] jepsen worker 62 - jepsen.util 4922 :ok :read [20 :nil] INFO [2022-01-10 05:48:23,090] jepsen worker 62 - jepsen.util 4922 :invoke :read [20 nil] INFO [2022-01-10 05:48:23,100] jepsen worker 34 - jepsen.util 4444 :ok :read [11 21] INFO [2022-01-10 05:48:23,100] jepsen worker 34 - jepsen.util 4444 :invoke :write [11 19] INFO [2022-01-10 05:48:23,105] jepsen worker 63 - jepsen.util 4293 :ok :read [21 7] INFO [2022-01-10 05:48:23,105] jepsen worker 63 - jepsen.util 4293 :invoke :write [21 41] INFO [2022-01-10 05:48:23,258] jepsen worker 77 - jepsen.util 5117 :ok :read [25 :nil] INFO [2022-01-10 05:48:23,259] jepsen worker 77 - jepsen.util 5117 :invoke :write [25 8] INFO [2022-01-10 05:48:23,308] jepsen worker 69 - jepsen.util 4209 :ok :read [23 26] INFO [2022-01-10 05:48:23,308] jepsen worker 69 - jepsen.util 4209 :invoke :read [23 nil] INFO [2022-01-10 05:48:23,390] jepsen worker 37 - jepsen.util 4177 :info :write [12 20] :RequestTimeoutException INFO [2022-01-10 05:48:23,391] jepsen worker 37 - jepsen.util 4267 :invoke :write [12 27] INFO [2022-01-10 05:48:23,402] jepsen worker 71 - jepsen.util 4121 :info :write [23 47] :RequestTimeoutException INFO [2022-01-10 05:48:23,402] jepsen worker 71 - jepsen.util 4211 :invoke :write [23 15] INFO [2022-01-10 05:48:23,486] jepsen worker 51 - jepsen.util 5361 :info :write [17 15] :RequestTimeoutException INFO [2022-01-10 05:48:23,487] jepsen worker 51 - jepsen.util 5451 :invoke :read [17 nil] INFO [2022-01-10 05:48:23,581] jepsen worker 12 - jepsen.util 5052 :info :write [4 23] :RequestTimeoutException INFO [2022-01-10 05:48:23,581] jepsen worker 12 - jepsen.util 5142 :invoke :write [4 19] INFO [2022-01-10 05:48:23,691] jepsen worker 79 - jepsen.util 5299 :info :write [26 13] :RequestTimeoutException INFO [2022-01-10 05:48:23,692] jepsen worker 79 - jepsen.util 5389 :invoke :write [26 19] INFO [2022-01-10 05:48:23,866] jepsen worker 0 - jepsen.util 4590 :info :write [0 30] :RequestTimeoutException INFO [2022-01-10 05:48:23,866] jepsen worker 0 - jepsen.util 4680 :invoke :write [0 5] INFO [2022-01-10 05:48:23,867] jepsen worker 76 - jepsen.util 3946 :ok :read [25 :nil] INFO [2022-01-10 05:48:23,868] jepsen worker 76 - jepsen.util 3946 :invoke :read [25 nil] INFO [2022-01-10 05:48:23,886] jepsen worker 85 - jepsen.util 4945 :info :write [28 32] :RequestTimeoutException INFO [2022-01-10 05:48:23,887] jepsen worker 85 - jepsen.util 5035 :invoke :read [28 nil] INFO [2022-01-10 05:48:23,924] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5847953216374269}, :ns_1@172.23.107.227 {:progress 0.4105571847507331}, :ns_1@172.23.105.187 {:progress 0.4105571847507331}, :ns_1@172.23.107.228 {:progress 0.5588235294117647}} INFO [2022-01-10 05:48:23,924] jepsen worker nemesis - couchbase.util Rebalance check count: 650 INFO [2022-01-10 05:48:23,925] jepsen worker nemesis - couchbase.util Rebalance stuck count: 547 INFO [2022-01-10 05:48:23,930] jepsen worker 4 - jepsen.util 4864 :ok :read [1 49] INFO [2022-01-10 05:48:23,930] jepsen worker 4 - jepsen.util 4864 :invoke :read [1 nil] INFO [2022-01-10 05:48:23,933] jepsen worker 51 - jepsen.util 5451 :ok :read [17 41] INFO [2022-01-10 05:48:23,934] jepsen worker 51 - jepsen.util 5451 :invoke :write [17 29] INFO [2022-01-10 05:48:24,109] jepsen worker 60 - jepsen.util 4920 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:48:24,109] jepsen worker 60 - jepsen.util 4920 :invoke :write [20 24] INFO [2022-01-10 05:48:24,112] jepsen worker 76 - jepsen.util 3946 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:48:24,112] jepsen worker 76 - jepsen.util 3946 :invoke :write [25 19] INFO [2022-01-10 05:48:24,122] jepsen worker 44 - jepsen.util 4724 :ok :read [14 21] INFO [2022-01-10 05:48:24,122] jepsen worker 44 - jepsen.util 4724 :invoke :read [14 nil] INFO [2022-01-10 05:48:24,284] jepsen worker 18 - jepsen.util 4518 :ok :read [6 25] INFO [2022-01-10 05:48:24,284] jepsen worker 18 - jepsen.util 4518 :invoke :read [6 nil] INFO [2022-01-10 05:48:24,284] jepsen worker 47 - jepsen.util 4727 :ok :read [15 23] INFO [2022-01-10 05:48:24,285] jepsen worker 47 - jepsen.util 4727 :invoke :write [15 18] INFO [2022-01-10 05:48:24,292] jepsen worker 45 - jepsen.util 5085 :ok :read [15 23] INFO [2022-01-10 05:48:24,292] jepsen worker 45 - jepsen.util 5085 :invoke :read [15 nil] INFO [2022-01-10 05:48:24,450] jepsen worker 48 - jepsen.util 4368 :info :write [16 43] :RequestTimeoutException INFO [2022-01-10 05:48:24,450] jepsen worker 48 - jepsen.util 4458 :invoke :read [16 nil] INFO [2022-01-10 05:48:24,534] jepsen worker 44 - jepsen.util 4724 :ok :read [14 21] INFO [2022-01-10 05:48:24,535] jepsen worker 44 - jepsen.util 4724 :invoke :write [14 37] INFO [2022-01-10 05:48:24,551] jepsen worker 16 - jepsen.util 4606 :info :write [5 32] :RequestTimeoutException INFO [2022-01-10 05:48:24,552] jepsen worker 16 - jepsen.util 4696 :invoke :read [5 nil] INFO [2022-01-10 05:48:24,679] jepsen worker 21 - jepsen.util 4611 :info :write [7 16] :RequestTimeoutException INFO [2022-01-10 05:48:24,680] jepsen worker 21 - jepsen.util 4701 :invoke :write [7 0] INFO [2022-01-10 05:48:24,702] jepsen worker 62 - jepsen.util 4922 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:48:24,703] jepsen worker 62 - jepsen.util 4922 :invoke :write [20 4] INFO [2022-01-10 05:48:24,755] jepsen worker 2 - jepsen.util 4682 :info :write [0 1] :RequestTimeoutException INFO [2022-01-10 05:48:24,756] jepsen worker 2 - jepsen.util 4772 :invoke :write [0 38] INFO [2022-01-10 05:48:24,776] jepsen worker 57 - jepsen.util 5277 :info :write [19 44] :RequestTimeoutException INFO [2022-01-10 05:48:24,777] jepsen worker 57 - jepsen.util 5367 :invoke :write [19 19] INFO [2022-01-10 05:48:24,935] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5847953216374269}, :ns_1@172.23.107.227 {:progress 0.4105571847507331}, :ns_1@172.23.105.187 {:progress 0.4105571847507331}, :ns_1@172.23.107.228 {:progress 0.5588235294117647}} INFO [2022-01-10 05:48:24,935] jepsen worker nemesis - couchbase.util Rebalance check count: 651 INFO [2022-01-10 05:48:24,935] jepsen worker nemesis - couchbase.util Rebalance stuck count: 548 INFO [2022-01-10 05:48:24,965] jepsen worker 84 - jepsen.util 4314 :info :write [28 43] :RequestTimeoutException INFO [2022-01-10 05:48:24,966] jepsen worker 84 - jepsen.util 4404 :invoke :write [28 46] INFO [2022-01-10 05:48:25,114] jepsen worker 10 - jepsen.util 4420 :info :write [3 37] :RequestTimeoutException INFO [2022-01-10 05:48:25,115] jepsen worker 10 - jepsen.util 4510 :invoke :read [3 nil] INFO [2022-01-10 05:48:25,146] jepsen worker 56 - jepsen.util 4826 :info :write [18 7] :RequestTimeoutException INFO [2022-01-10 05:48:25,147] jepsen worker 56 - jepsen.util 4916 :invoke :read [18 nil] INFO [2022-01-10 05:48:25,216] jepsen worker 14 - jepsen.util 5504 :info :write [4 44] :RequestTimeoutException INFO [2022-01-10 05:48:25,217] jepsen worker 14 - jepsen.util 5594 :invoke :write [4 33] INFO [2022-01-10 05:48:25,290] jepsen worker 54 - jepsen.util 4824 :info :write [18 34] :RequestTimeoutException INFO [2022-01-10 05:48:25,290] jepsen worker 54 - jepsen.util 4914 :invoke :read [18 nil] INFO [2022-01-10 05:48:25,306] jepsen worker 24 - jepsen.util 4254 :fail :read [8 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:48:25,307] jepsen worker 24 - jepsen.util 4254 :invoke :write [8 15] INFO [2022-01-10 05:48:25,441] jepsen worker 7 - jepsen.util 4867 :info :write [2 1] :RequestTimeoutException INFO [2022-01-10 05:48:25,442] jepsen worker 7 - jepsen.util 4957 :invoke :write [2 13] INFO [2022-01-10 05:48:25,897] jepsen worker 30 - jepsen.util 4260 :ok :read [10 12] INFO [2022-01-10 05:48:25,898] jepsen worker 30 - jepsen.util 4260 :invoke :read [10 nil] INFO [2022-01-10 05:48:25,918] jepsen worker 30 - jepsen.util 4260 :ok :read [10 12] INFO [2022-01-10 05:48:25,919] jepsen worker 30 - jepsen.util 4260 :invoke :read [10 nil] INFO [2022-01-10 05:48:25,941] jepsen worker 30 - jepsen.util 4260 :ok :read [10 12] INFO [2022-01-10 05:48:25,942] jepsen worker 30 - jepsen.util 4260 :invoke :write [10 48] INFO [2022-01-10 05:48:25,942] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5964912280701755}, :ns_1@172.23.107.227 {:progress 0.4164222873900293}, :ns_1@172.23.105.187 {:progress 0.4164222873900293}, :ns_1@172.23.107.228 {:progress 0.5705882352941176}} INFO [2022-01-10 05:48:25,942] jepsen worker nemesis - couchbase.util Rebalance check count: 652 INFO [2022-01-10 05:48:25,942] jepsen worker nemesis - couchbase.util Rebalance stuck count: 548 INFO [2022-01-10 05:48:26,059] jepsen worker 89 - jepsen.util 5129 :ok :read [29 0] INFO [2022-01-10 05:48:26,059] jepsen worker 89 - jepsen.util 5129 :invoke :write [29 4] INFO [2022-01-10 05:48:26,101] jepsen worker 87 - jepsen.util 4497 :ok :read [29 0] INFO [2022-01-10 05:48:26,102] jepsen worker 87 - jepsen.util 4497 :invoke :read [29 nil] INFO [2022-01-10 05:48:26,107] jepsen worker 78 - jepsen.util 5658 :info :write [26 49] :RequestTimeoutException INFO [2022-01-10 05:48:26,107] jepsen worker 78 - jepsen.util 5748 :invoke :write [26 47] INFO [2022-01-10 05:48:26,271] jepsen worker 36 - jepsen.util 3456 :info :write [12 16] :RequestTimeoutException INFO [2022-01-10 05:48:26,272] jepsen worker 36 - jepsen.util 3546 :invoke :write [12 20] INFO [2022-01-10 05:48:26,336] jepsen worker 68 - jepsen.util 4838 :info :write [22 2] :RequestTimeoutException INFO [2022-01-10 05:48:26,337] jepsen worker 68 - jepsen.util 4928 :invoke :read [22 nil] INFO [2022-01-10 05:48:26,790] jepsen worker 16 - jepsen.util 4696 :ok :read [5 25] INFO [2022-01-10 05:48:26,791] jepsen worker 16 - jepsen.util 4696 :invoke :write [5 16] INFO [2022-01-10 05:48:26,924] jepsen worker 25 - jepsen.util 4255 :info :write [8 47] :RequestTimeoutException INFO [2022-01-10 05:48:26,928] jepsen worker 25 - jepsen.util 4345 :invoke :write [8 47] INFO [2022-01-10 05:48:26,948] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5964912280701755}, :ns_1@172.23.107.227 {:progress 0.4164222873900293}, :ns_1@172.23.105.187 {:progress 0.4164222873900293}, :ns_1@172.23.107.228 {:progress 0.5705882352941176}} INFO [2022-01-10 05:48:26,948] jepsen worker nemesis - couchbase.util Rebalance check count: 653 INFO [2022-01-10 05:48:26,948] jepsen worker nemesis - couchbase.util Rebalance stuck count: 549 INFO [2022-01-10 05:48:26,955] jepsen worker 8 - jepsen.util 4688 :info :write [2 5] :RequestTimeoutException INFO [2022-01-10 05:48:26,956] jepsen worker 8 - jepsen.util 4778 :invoke :read [2 nil] INFO [2022-01-10 05:48:27,041] jepsen worker 32 - jepsen.util 4802 :info :write [10 19] :RequestTimeoutException INFO [2022-01-10 05:48:27,041] jepsen worker 42 - jepsen.util 4272 :info :write [14 13] :RequestTimeoutException INFO [2022-01-10 05:48:27,042] jepsen worker 42 - jepsen.util 4362 :invoke :write [14 43] INFO [2022-01-10 05:48:27,043] jepsen worker 32 - jepsen.util 4892 :invoke :write [10 23] INFO [2022-01-10 05:48:27,092] jepsen worker 46 - jepsen.util 4636 :info :write [15 11] :RequestTimeoutException INFO [2022-01-10 05:48:27,092] jepsen worker 46 - jepsen.util 4726 :invoke :write [15 21] INFO [2022-01-10 05:48:27,152] jepsen worker 83 - jepsen.util 5213 :info :write [27 18] :RequestTimeoutException INFO [2022-01-10 05:48:27,152] jepsen worker 83 - jepsen.util 5303 :invoke :write [27 38] INFO [2022-01-10 05:48:27,191] jepsen worker 31 - jepsen.util 4711 :info :write [10 23] :RequestTimeoutException INFO [2022-01-10 05:48:27,191] jepsen worker 31 - jepsen.util 4801 :invoke :read [10 nil] INFO [2022-01-10 05:48:27,205] jepsen worker 74 - jepsen.util 5024 :info :write [24 10] :RequestTimeoutException INFO [2022-01-10 05:48:27,205] jepsen worker 74 - jepsen.util 5114 :invoke :write [24 5] INFO [2022-01-10 05:48:27,212] jepsen worker 31 - jepsen.util 4801 :ok :read [10 12] INFO [2022-01-10 05:48:27,213] jepsen worker 31 - jepsen.util 4801 :invoke :read [10 nil] INFO [2022-01-10 05:48:27,234] jepsen worker 31 - jepsen.util 4801 :ok :read [10 12] INFO [2022-01-10 05:48:27,234] jepsen worker 31 - jepsen.util 4801 :invoke :write [10 31] INFO [2022-01-10 05:48:27,456] jepsen worker 65 - jepsen.util 4295 :info :write [21 9] :RequestTimeoutException INFO [2022-01-10 05:48:27,457] jepsen worker 65 - jepsen.util 4385 :invoke :write [21 47] INFO [2022-01-10 05:48:27,561] jepsen worker 29 - jepsen.util 4439 :info :write [9 27] :RequestTimeoutException INFO [2022-01-10 05:48:27,562] jepsen worker 29 - jepsen.util 4529 :invoke :read [9 nil] INFO [2022-01-10 05:48:27,695] jepsen worker 49 - jepsen.util 4369 :info :write [16 22] :RequestTimeoutException INFO [2022-01-10 05:48:27,695] jepsen worker 49 - jepsen.util 4459 :invoke :read [16 nil] INFO [2022-01-10 05:48:27,920] jepsen worker 23 - jepsen.util 4703 :info :write [7 6] :RequestTimeoutException INFO [2022-01-10 05:48:27,921] jepsen worker 23 - jepsen.util 4793 :invoke :read [7 nil] INFO [2022-01-10 05:48:27,929] jepsen worker 19 - jepsen.util 4699 :info :write [6 29] :RequestTimeoutException INFO [2022-01-10 05:48:27,930] jepsen worker 19 - jepsen.util 4789 :invoke :write [6 3] INFO [2022-01-10 05:48:27,951] jepsen worker 20 - jepsen.util 4250 :info :write [6 29] :RequestTimeoutException INFO [2022-01-10 05:48:27,951] jepsen worker 20 - jepsen.util 4340 :invoke :write [6 25] INFO [2022-01-10 05:48:27,954] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5964912280701755}, :ns_1@172.23.107.227 {:progress 0.4164222873900293}, :ns_1@172.23.105.187 {:progress 0.4164222873900293}, :ns_1@172.23.107.228 {:progress 0.5705882352941176}} INFO [2022-01-10 05:48:27,954] jepsen worker nemesis - couchbase.util Rebalance check count: 654 INFO [2022-01-10 05:48:27,954] jepsen worker nemesis - couchbase.util Rebalance stuck count: 550 INFO [2022-01-10 05:48:27,999] jepsen worker 11 - jepsen.util 4421 :info :write [3 39] :RequestTimeoutException INFO [2022-01-10 05:48:28,000] jepsen worker 11 - jepsen.util 4511 :invoke :write [3 30] INFO [2022-01-10 05:48:28,012] jepsen worker 33 - jepsen.util 4803 :info :write [11 37] :RequestTimeoutException INFO [2022-01-10 05:48:28,012] jepsen worker 33 - jepsen.util 4893 :invoke :write [11 17] INFO [2022-01-10 05:48:28,171] jepsen worker 61 - jepsen.util 4921 :ok :read [20 :nil] INFO [2022-01-10 05:48:28,172] jepsen worker 61 - jepsen.util 4921 :invoke :read [20 nil] INFO [2022-01-10 05:48:28,359] jepsen worker 70 - jepsen.util 4570 :info :write [23 41] :RequestTimeoutException INFO [2022-01-10 05:48:28,360] jepsen worker 70 - jepsen.util 4660 :invoke :write [23 25] INFO [2022-01-10 05:48:28,361] jepsen worker 66 - jepsen.util 4386 :info :write [22 31] :RequestTimeoutException INFO [2022-01-10 05:48:28,362] jepsen worker 66 - jepsen.util 4476 :invoke :write [22 6] INFO [2022-01-10 05:48:28,376] jepsen worker 69 - jepsen.util 4209 :ok :read [23 26] INFO [2022-01-10 05:48:28,382] jepsen worker 69 - jepsen.util 4209 :invoke :write [23 40] INFO [2022-01-10 05:48:28,418] jepsen worker 35 - jepsen.util 4355 :ok :read [11 21] INFO [2022-01-10 05:48:28,419] jepsen worker 35 - jepsen.util 4355 :invoke :write [11 31] INFO [2022-01-10 05:48:28,446] jepsen worker 72 - jepsen.util 4842 :info :write [24 15] :RequestTimeoutException INFO [2022-01-10 05:48:28,446] jepsen worker 72 - jepsen.util 4932 :invoke :write [24 4] INFO [2022-01-10 05:48:28,449] jepsen worker 75 - jepsen.util 4395 :info :write [25 26] :RequestTimeoutException INFO [2022-01-10 05:48:28,450] jepsen worker 75 - jepsen.util 4485 :invoke :write [25 5] INFO [2022-01-10 05:48:28,642] jepsen worker 67 - jepsen.util 4297 :ok :read [22 41] INFO [2022-01-10 05:48:28,642] jepsen worker 50 - jepsen.util 4820 :ok :read [16 38] INFO [2022-01-10 05:48:28,643] jepsen worker 67 - jepsen.util 4297 :invoke :read [22 nil] INFO [2022-01-10 05:48:28,644] jepsen worker 50 - jepsen.util 4820 :invoke :read [16 nil] INFO [2022-01-10 05:48:28,676] jepsen worker 6 - jepsen.util 4506 :ok :read [2 14] INFO [2022-01-10 05:48:28,677] jepsen worker 6 - jepsen.util 4506 :invoke :write [2 37] INFO [2022-01-10 05:48:28,797] jepsen worker 40 - jepsen.util 4270 :fail :read [13 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:48:28,798] jepsen worker 40 - jepsen.util 4270 :invoke :write [13 26] INFO [2022-01-10 05:48:28,830] jepsen worker 68 - jepsen.util 4928 :ok :read [22 41] INFO [2022-01-10 05:48:28,831] jepsen worker 68 - jepsen.util 4928 :invoke :read [22 nil] INFO [2022-01-10 05:48:28,960] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5964912280701755}, :ns_1@172.23.107.227 {:progress 0.4164222873900293}, :ns_1@172.23.105.187 {:progress 0.4164222873900293}, :ns_1@172.23.107.228 {:progress 0.5705882352941176}} INFO [2022-01-10 05:48:28,960] jepsen worker nemesis - couchbase.util Rebalance check count: 655 INFO [2022-01-10 05:48:28,960] jepsen worker nemesis - couchbase.util Rebalance stuck count: 551 INFO [2022-01-10 05:48:28,982] jepsen worker 43 - jepsen.util 4633 :info :write [14 9] :RequestTimeoutException INFO [2022-01-10 05:48:28,983] jepsen worker 43 - jepsen.util 4723 :invoke :read [14 nil] INFO [2022-01-10 05:48:29,071] jepsen worker 64 - jepsen.util 4564 :info :write [21 28] :RequestTimeoutException INFO [2022-01-10 05:48:29,072] jepsen worker 64 - jepsen.util 4654 :invoke :write [21 13] INFO [2022-01-10 05:48:29,125] jepsen worker 81 - jepsen.util 5121 :info :write [27 42] :RequestTimeoutException INFO [2022-01-10 05:48:29,126] jepsen worker 81 - jepsen.util 5211 :invoke :read [27 nil] INFO [2022-01-10 05:48:29,437] jepsen worker 28 - jepsen.util 4348 :ok :read [9 22] INFO [2022-01-10 05:48:29,438] jepsen worker 28 - jepsen.util 4348 :invoke :read [9 nil] INFO [2022-01-10 05:48:29,592] jepsen worker 26 - jepsen.util 4346 :info :write [8 46] :RequestTimeoutException INFO [2022-01-10 05:48:29,592] jepsen worker 26 - jepsen.util 4436 :invoke :write [8 33] INFO [2022-01-10 05:48:29,646] jepsen worker 48 - jepsen.util 4458 :ok :read [16 38] INFO [2022-01-10 05:48:29,647] jepsen worker 48 - jepsen.util 4458 :invoke :write [16 35] INFO [2022-01-10 05:48:29,786] jepsen worker 13 - jepsen.util 5323 :info :write [4 37] :RequestTimeoutException INFO [2022-01-10 05:48:29,787] jepsen worker 13 - jepsen.util 5413 :invoke :write [4 32] INFO [2022-01-10 05:48:29,965] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5964912280701755}, :ns_1@172.23.107.227 {:progress 0.4164222873900293}, :ns_1@172.23.105.187 {:progress 0.4164222873900293}, :ns_1@172.23.107.228 {:progress 0.5705882352941176}} INFO [2022-01-10 05:48:29,965] jepsen worker nemesis - couchbase.util Rebalance check count: 656 INFO [2022-01-10 05:48:29,965] jepsen worker nemesis - couchbase.util Rebalance stuck count: 552 INFO [2022-01-10 05:48:29,990] jepsen worker 53 - jepsen.util 5543 :info :write [17 44] :RequestTimeoutException INFO [2022-01-10 05:48:29,991] jepsen worker 53 - jepsen.util 5633 :invoke :read [17 nil] INFO [2022-01-10 05:48:29,992] jepsen worker 80 - jepsen.util 5660 :info :write [26 48] :RequestTimeoutException INFO [2022-01-10 05:48:29,993] jepsen worker 80 - jepsen.util 5750 :invoke :write [26 11] INFO [2022-01-10 05:48:30,086] jepsen worker 49 - jepsen.util 4459 :ok :read [16 38] INFO [2022-01-10 05:48:30,087] jepsen worker 49 - jepsen.util 4459 :invoke :write [16 13] INFO [2022-01-10 05:48:30,334] jepsen worker 38 - jepsen.util 4178 :info :write [12 32] :RequestTimeoutException INFO [2022-01-10 05:48:30,335] jepsen worker 38 - jepsen.util 4268 :invoke :write [12 25] INFO [2022-01-10 05:48:30,445] jepsen worker 41 - jepsen.util 4631 :fail :read [13 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:48:30,446] jepsen worker 41 - jepsen.util 4631 :invoke :read [13 nil] INFO [2022-01-10 05:48:30,551] jepsen worker 3 - jepsen.util 4503 :info :write [1 8] :RequestTimeoutException INFO [2022-01-10 05:48:30,552] jepsen worker 3 - jepsen.util 4593 :invoke :write [1 43] INFO [2022-01-10 05:48:30,646] jepsen worker 39 - jepsen.util 4989 :info :write [13 47] :RequestTimeoutException INFO [2022-01-10 05:48:30,647] jepsen worker 39 - jepsen.util 5079 :invoke :read [13 nil] WARN [2022-01-10 05:48:30,665] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10900332,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10900332,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9036305,"encode_duration_us":18926,"operation_id":"0x8e40","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10002092}],"total_count":1}} INFO [2022-01-10 05:48:30,686] jepsen worker 88 - jepsen.util 4678 :info :write [29 11] :RequestTimeoutException INFO [2022-01-10 05:48:30,686] jepsen worker 88 - jepsen.util 4768 :invoke :read [29 nil] INFO [2022-01-10 05:48:30,710] jepsen worker 88 - jepsen.util 4768 :ok :read [29 0] INFO [2022-01-10 05:48:30,711] jepsen worker 88 - jepsen.util 4768 :invoke :read [29 nil] INFO [2022-01-10 05:48:30,730] jepsen worker 4 - jepsen.util 4864 :ok :read [1 49] INFO [2022-01-10 05:48:30,731] jepsen worker 4 - jepsen.util 4864 :invoke :read [1 nil] INFO [2022-01-10 05:48:30,735] jepsen worker 88 - jepsen.util 4768 :ok :read [29 0] INFO [2022-01-10 05:48:30,736] jepsen worker 88 - jepsen.util 4768 :invoke :write [29 8] WARN [2022-01-10 05:48:30,765] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":3848018,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10551184,"total_server_duration_us":2581198,"encode_duration_us":19175,"operation_id":"0x8ea0","timeout_ms":10000,"last_server_duration_us":1594606,"total_duration_us":10008455},{"operation_name":"upsert","last_dispatch_duration_us":967300,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":8876685,"total_server_duration_us":4508311,"encode_duration_us":18856,"operation_id":"0x8f45","timeout_ms":10000,"last_server_duration_us":403293,"total_duration_us":10007430},{"operation_name":"upsert","last_dispatch_duration_us":1620949,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10269258,"total_server_duration_us":2292349,"encode_duration_us":16569,"operation_id":"0x8ebb","timeout_ms":10000,"last_server_duration_us":389256,"total_duration_us":10007407},{"operation_name":"upsert","last_dispatch_duration_us":3586433,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6800728,"total_server_duration_us":793801,"encode_duration_us":17841,"operation_id":"0x8db1","timeout_ms":10000,"last_server_duration_us":241109,"total_duration_us":10004912},{"operation_name":"upsert","last_dispatch_duration_us":4250764,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":5441056,"total_server_duration_us":2364886,"encode_duration_us":19538,"operation_id":"0x8eb6","timeout_ms":10000,"last_server_duration_us":2196083,"total_duration_us":10004771},{"operation_name":"upsert","last_dispatch_duration_us":3402905,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11024350,"total_server_duration_us":2161740,"encode_duration_us":16733,"operation_id":"0x8ea2","timeout_ms":10000,"last_server_duration_us":388978,"total_duration_us":10004666},{"operation_name":"upsert","last_dispatch_duration_us":10900332,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":10900332,"total_server_duration_us":9036305,"encode_duration_us":18926,"operation_id":"0x8e40","timeout_ms":10000,"last_server_duration_us":9036305,"total_duration_us":10002092},{"operation_name":"upsert","last_dispatch_duration_us":2038081,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9658536,"total_server_duration_us":1394172,"encode_duration_us":16269,"operation_id":"0x8f0f","timeout_ms":10000,"last_server_duration_us":596880,"total_duration_us":10001274},{"operation_name":"upsert","last_dispatch_duration_us":4798692,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":14480844,"total_server_duration_us":1010051,"encode_duration_us":16329,"operation_id":"0x8d60","timeout_ms":10000,"last_server_duration_us":380384,"total_duration_us":10001129},{"operation_name":"get","last_dispatch_duration_us":9146635,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9146635,"total_server_duration_us":49,"operation_id":"0x8e9f","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":9147744}],"total_count":18}} INFO [2022-01-10 05:48:30,861] jepsen worker 17 - jepsen.util 4427 :info :write [5 3] :RequestTimeoutException INFO [2022-01-10 05:48:30,862] jepsen worker 17 - jepsen.util 4517 :invoke :write [5 25] INFO [2022-01-10 05:48:30,880] jepsen worker 8 - jepsen.util 4778 :ok :read [2 14] INFO [2022-01-10 05:48:30,880] jepsen worker 8 - jepsen.util 4778 :invoke :write [2 0] INFO [2022-01-10 05:48:30,971] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5964912280701755}, :ns_1@172.23.107.227 {:progress 0.4164222873900293}, :ns_1@172.23.105.187 {:progress 0.4164222873900293}, :ns_1@172.23.107.228 {:progress 0.5705882352941176}} INFO [2022-01-10 05:48:30,971] jepsen worker nemesis - couchbase.util Rebalance check count: 657 INFO [2022-01-10 05:48:30,971] jepsen worker nemesis - couchbase.util Rebalance stuck count: 553 INFO [2022-01-10 05:48:31,115] jepsen worker 81 - jepsen.util 5211 :ok :read [27 30] INFO [2022-01-10 05:48:31,116] jepsen worker 81 - jepsen.util 5211 :invoke :read [27 nil] INFO [2022-01-10 05:48:31,922] jepsen worker 15 - jepsen.util 4155 :info :write [5 0] :RequestTimeoutException INFO [2022-01-10 05:48:31,922] jepsen worker 15 - jepsen.util 4245 :invoke :read [5 nil] INFO [2022-01-10 05:48:31,976] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5964912280701755}, :ns_1@172.23.107.227 {:progress 0.4164222873900293}, :ns_1@172.23.105.187 {:progress 0.4164222873900293}, :ns_1@172.23.107.228 {:progress 0.5705882352941176}} INFO [2022-01-10 05:48:31,976] jepsen worker nemesis - couchbase.util Rebalance check count: 658 INFO [2022-01-10 05:48:31,976] jepsen worker nemesis - couchbase.util Rebalance stuck count: 554 INFO [2022-01-10 05:48:32,015] jepsen worker 82 - jepsen.util 4672 :info :write [27 38] :RequestTimeoutException INFO [2022-01-10 05:48:32,016] jepsen worker 82 - jepsen.util 4762 :invoke :write [27 44] INFO [2022-01-10 05:48:32,066] jepsen worker 9 - jepsen.util 5139 :info :write [3 15] :RequestTimeoutException INFO [2022-01-10 05:48:32,067] jepsen worker 9 - jepsen.util 5229 :invoke :read [3 nil] INFO [2022-01-10 05:48:32,068] jepsen worker 85 - jepsen.util 5035 :ok :read [28 45] INFO [2022-01-10 05:48:32,068] jepsen worker 85 - jepsen.util 5035 :invoke :write [28 25] INFO [2022-01-10 05:48:32,081] jepsen worker 73 - jepsen.util 4393 :info :write [24 35] :RequestTimeoutException INFO [2022-01-10 05:48:32,082] jepsen worker 73 - jepsen.util 4483 :invoke :write [24 20] INFO [2022-01-10 05:48:32,101] jepsen worker 52 - jepsen.util 5722 :info :write [17 13] :RequestTimeoutException INFO [2022-01-10 05:48:32,102] jepsen worker 52 - jepsen.util 5812 :invoke :write [17 39] WARN [2022-01-10 05:48:32,177] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9047169,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9610394,"total_server_duration_us":9322156,"encode_duration_us":19451,"operation_id":"0x8ea8","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10007825},{"operation_name":"upsert","last_dispatch_duration_us":9551914,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10424913,"total_server_duration_us":989184,"encode_duration_us":16475,"operation_id":"0x8ee7","timeout_ms":10000,"last_server_duration_us":581923,"total_duration_us":10007575},{"operation_name":"upsert","last_dispatch_duration_us":7435097,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9913478,"total_server_duration_us":782931,"encode_duration_us":17150,"operation_id":"0x8e4f","timeout_ms":10000,"last_server_duration_us":406126,"total_duration_us":10007305},{"operation_name":"upsert","last_dispatch_duration_us":4992794,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4992794,"total_server_duration_us":610647,"encode_duration_us":16663,"operation_id":"0x8f0d","timeout_ms":10000,"last_server_duration_us":610647,"total_duration_us":10006951},{"operation_name":"upsert","last_dispatch_duration_us":5200078,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":7553195,"total_server_duration_us":570725,"encode_duration_us":19013,"operation_id":"0x8ea9","timeout_ms":10000,"last_server_duration_us":200218,"total_duration_us":10005343},{"operation_name":"upsert","last_dispatch_duration_us":1458313,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10216828,"total_server_duration_us":1792690,"encode_duration_us":17481,"operation_id":"0x8f6a","timeout_ms":10000,"last_server_duration_us":989199,"total_duration_us":10005182},{"operation_name":"upsert","last_dispatch_duration_us":2794311,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":2794311,"total_server_duration_us":390930,"encode_duration_us":16455,"operation_id":"0x8df0","timeout_ms":10000,"last_server_duration_us":390930,"total_duration_us":10004397},{"operation_name":"upsert","last_dispatch_duration_us":10184015,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10616230,"total_server_duration_us":10194139,"encode_duration_us":20153,"operation_id":"0x8eb9","timeout_ms":10000,"last_server_duration_us":9774587,"total_duration_us":10004046},{"operation_name":"upsert","last_dispatch_duration_us":30410,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":647730,"total_server_duration_us":336406,"encode_duration_us":16635,"operation_id":"0x8eb5","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10003251},{"operation_name":"upsert","last_dispatch_duration_us":4690512,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4690512,"total_server_duration_us":2179826,"encode_duration_us":18822,"operation_id":"0x8edd","timeout_ms":10000,"last_server_duration_us":2179826,"total_duration_us":10002096}],"total_count":16}} WARN [2022-01-10 05:48:32,258] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":15480028,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15480028,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":10152938,"encode_duration_us":16148,"operation_id":"0x8e11","timeout_ms":10000,"last_server_duration_us":10152938,"total_duration_us":10003676},{"operation_name":"upsert","last_dispatch_duration_us":10734046,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10734046,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9037366,"encode_duration_us":17017,"operation_id":"0x8e47","timeout_ms":10000,"last_server_duration_us":9037366,"total_duration_us":10006665}],"total_count":2}} WARN [2022-01-10 05:48:32,277] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10184015,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10616230,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":10194139,"encode_duration_us":20153,"operation_id":"0x8eb9","timeout_ms":10000,"last_server_duration_us":9774587,"total_duration_us":10004046},{"operation_name":"upsert","last_dispatch_duration_us":9047169,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9610394,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9322156,"encode_duration_us":19451,"operation_id":"0x8ea8","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10007825}],"total_count":2}} WARN [2022-01-10 05:48:32,357] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17152671,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":17152671,"total_server_duration_us":836768,"operation_id":"0x8d87","timeout_ms":30000,"last_server_duration_us":836768,"total_duration_us":17155095},{"operation_name":"get","last_dispatch_duration_us":15871034,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":15871034,"total_server_duration_us":611998,"operation_id":"0x8dee","timeout_ms":30000,"last_server_duration_us":611998,"total_duration_us":15873270},{"operation_name":"upsert","last_dispatch_duration_us":2658349,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":7995534,"total_server_duration_us":1008458,"encode_duration_us":16921,"operation_id":"0x8e49","timeout_ms":10000,"last_server_duration_us":200638,"total_duration_us":10010735},{"operation_name":"upsert","last_dispatch_duration_us":14112538,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":14112538,"total_server_duration_us":391209,"encode_duration_us":18121,"operation_id":"0x8e7e","timeout_ms":10000,"last_server_duration_us":391209,"total_duration_us":10007069},{"operation_name":"upsert","last_dispatch_duration_us":11340872,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":12384087,"total_server_duration_us":315152,"encode_duration_us":16511,"operation_id":"0x8f2c","timeout_ms":10000,"last_server_duration_us":94564,"total_duration_us":10006602},{"operation_name":"upsert","last_dispatch_duration_us":1429913,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11310350,"total_server_duration_us":2160469,"encode_duration_us":17043,"operation_id":"0x8eb7","timeout_ms":10000,"last_server_duration_us":175300,"total_duration_us":10006301},{"operation_name":"upsert","last_dispatch_duration_us":30377,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":528680,"total_server_duration_us":308918,"encode_duration_us":12423,"operation_id":"0x8f95","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10005791},{"operation_name":"upsert","last_dispatch_duration_us":10863735,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":15252422,"total_server_duration_us":10541513,"encode_duration_us":17113,"operation_id":"0x8e73","timeout_ms":10000,"last_server_duration_us":9738403,"total_duration_us":10004222},{"operation_name":"upsert","last_dispatch_duration_us":11151559,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11151559,"total_server_duration_us":212124,"encode_duration_us":16395,"operation_id":"0x8f4a","timeout_ms":10000,"last_server_duration_us":212124,"total_duration_us":10004097},{"operation_name":"upsert","last_dispatch_duration_us":21017,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8282955,"total_server_duration_us":256485,"encode_duration_us":16425,"operation_id":"0x8dbe","timeout_ms":10000,"last_server_duration_us":12894,"total_duration_us":10001855}],"total_count":15}} WARN [2022-01-10 05:48:32,359] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9349355,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9349355,"total_server_duration_us":49,"operation_id":"0x8e12","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":14512149},{"operation_name":"get","last_dispatch_duration_us":9859247,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9859247,"total_server_duration_us":399341,"operation_id":"0x8ea3","timeout_ms":30000,"last_server_duration_us":399341,"total_duration_us":13493054},{"operation_name":"upsert","last_dispatch_duration_us":1115284,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":8668548,"total_server_duration_us":4709292,"encode_duration_us":16357,"operation_id":"0x8ded","timeout_ms":10000,"last_server_duration_us":411816,"total_duration_us":10010906},{"operation_name":"upsert","last_dispatch_duration_us":6517881,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":6517881,"total_server_duration_us":1519253,"encode_duration_us":19320,"operation_id":"0x8e3e","timeout_ms":10000,"last_server_duration_us":1519253,"total_duration_us":10008015},{"operation_name":"upsert","last_dispatch_duration_us":10583657,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12161945,"total_server_duration_us":236702,"encode_duration_us":18095,"operation_id":"0x8dbd","timeout_ms":10000,"last_server_duration_us":73136,"total_duration_us":10007216},{"operation_name":"upsert","last_dispatch_duration_us":10734046,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10734046,"total_server_duration_us":9037366,"encode_duration_us":17017,"operation_id":"0x8e47","timeout_ms":10000,"last_server_duration_us":9037366,"total_duration_us":10006665},{"operation_name":"upsert","last_dispatch_duration_us":1642497,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6254034,"total_server_duration_us":609828,"encode_duration_us":16980,"operation_id":"0x8eea","timeout_ms":10000,"last_server_duration_us":412958,"total_duration_us":10005768},{"operation_name":"upsert","last_dispatch_duration_us":5684865,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7652507,"total_server_duration_us":553515,"encode_duration_us":16474,"operation_id":"0x8de3","timeout_ms":10000,"last_server_duration_us":391488,"total_duration_us":10005262},{"operation_name":"upsert","last_dispatch_duration_us":8240867,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8240867,"total_server_duration_us":393165,"encode_duration_us":17415,"operation_id":"0x8e30","timeout_ms":10000,"last_server_duration_us":393165,"total_duration_us":10003923},{"operation_name":"upsert","last_dispatch_duration_us":15480028,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":15480028,"total_server_duration_us":10152938,"encode_duration_us":16148,"operation_id":"0x8e11","timeout_ms":10000,"last_server_duration_us":10152938,"total_duration_us":10003676}],"total_count":19}} WARN [2022-01-10 05:48:32,368] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11597766,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11597766,"total_server_duration_us":213417,"operation_id":"0x8eab","timeout_ms":30000,"last_server_duration_us":213417,"total_duration_us":11599641},{"operation_name":"get","last_dispatch_duration_us":10578548,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10578548,"total_server_duration_us":9002,"operation_id":"0x8dbc","timeout_ms":30000,"last_server_duration_us":9002,"total_duration_us":10578922},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":18274,"operation_id":"0x8e1a","timeout_ms":10000,"total_duration_us":10009557},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16485,"operation_id":"0x8f0a","timeout_ms":10000,"total_duration_us":10007752},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":16712,"operation_id":"0x8eb4","timeout_ms":10000,"total_duration_us":10007205},{"operation_name":"upsert","last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","encode_duration_us":16457,"operation_id":"0x8eac","timeout_ms":10000,"total_duration_us":10006439},{"operation_name":"upsert","last_dispatch_duration_us":6223222,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":15223632,"total_server_duration_us":2065893,"encode_duration_us":18549,"operation_id":"0x8e7a","timeout_ms":10000,"last_server_duration_us":148461,"total_duration_us":10006424},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","encode_duration_us":9383,"operation_id":"0x8f92","timeout_ms":10000,"total_duration_us":10006341},{"operation_name":"upsert","last_dispatch_duration_us":4219652,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":11587172,"total_server_duration_us":2168700,"encode_duration_us":16301,"operation_id":"0x8dec","timeout_ms":10000,"last_server_duration_us":598217,"total_duration_us":10004423},{"operation_name":"upsert","last_dispatch_duration_us":6133896,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":11828595,"total_server_duration_us":1461676,"encode_duration_us":16257,"operation_id":"0x8f3c","timeout_ms":10000,"last_server_duration_us":297253,"total_duration_us":10004067}],"total_count":14}} WARN [2022-01-10 05:48:32,374] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9053608,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9811060,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9477128,"encode_duration_us":16384,"operation_id":"0x8d1d","timeout_ms":10000,"last_server_duration_us":9021452,"total_duration_us":10006360}],"total_count":1}} INFO [2022-01-10 05:48:32,388] jepsen worker 18 - jepsen.util 4518 :ok :read [6 25] INFO [2022-01-10 05:48:32,388] jepsen worker 18 - jepsen.util 4518 :invoke :write [6 23] INFO [2022-01-10 05:48:32,424] jepsen worker 45 - jepsen.util 5085 :ok :read [15 23] INFO [2022-01-10 05:48:32,425] jepsen worker 45 - jepsen.util 5085 :invoke :write [15 42] INFO [2022-01-10 05:48:32,457] jepsen worker 81 - jepsen.util 5211 :ok :read [27 30] INFO [2022-01-10 05:48:32,457] jepsen worker 81 - jepsen.util 5211 :invoke :write [27 44] WARN [2022-01-10 05:48:32,469] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11364872,"last_local_id":"EFE9906200000004/000000008B42EFB5","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":21039210,"last_local_socket":"172.23.104.79:44292","total_server_duration_us":11495919,"encode_duration_us":17116,"operation_id":"0x8d54","timeout_ms":10000,"last_server_duration_us":9512734,"total_duration_us":10002188},{"operation_name":"upsert","last_dispatch_duration_us":21651848,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":30018926,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":11099925,"encode_duration_us":16257,"operation_id":"0x8b9a","timeout_ms":10000,"last_server_duration_us":10574799,"total_duration_us":10008834},{"operation_name":"upsert","last_dispatch_duration_us":16339129,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":23974678,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":11557278,"encode_duration_us":23348,"operation_id":"0x8b06","timeout_ms":10000,"last_server_duration_us":10736446,"total_duration_us":10006946}],"total_count":3}} INFO [2022-01-10 05:48:32,474] jepsen worker 58 - jepsen.util 4558 :info :write [19 34] :RequestTimeoutException INFO [2022-01-10 05:48:32,475] jepsen worker 58 - jepsen.util 4648 :invoke :write [19 17] INFO [2022-01-10 05:48:32,535] jepsen worker 1 - jepsen.util 4591 :info :write [0 28] :RequestTimeoutException INFO [2022-01-10 05:48:32,536] jepsen worker 1 - jepsen.util 4681 :invoke :write [0 33] WARN [2022-01-10 05:48:32,557] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10863735,"last_local_id":"EFE9906200000005/000000009D86D9DF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":15252422,"last_local_socket":"172.23.104.79:44284","total_server_duration_us":10541513,"encode_duration_us":17113,"operation_id":"0x8e73","timeout_ms":10000,"last_server_duration_us":9738403,"total_duration_us":10004222},{"operation_name":"upsert","last_dispatch_duration_us":13032509,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13032509,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":11353639,"encode_duration_us":18008,"operation_id":"0x8d4f","timeout_ms":10000,"last_server_duration_us":11353639,"total_duration_us":10008470}],"total_count":2}} WARN [2022-01-10 05:48:32,575] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":6592363,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10495171,"total_server_duration_us":1608945,"encode_duration_us":16937,"operation_id":"0x8f3b","timeout_ms":10000,"last_server_duration_us":616395,"total_duration_us":10010741},{"operation_name":"upsert","last_dispatch_duration_us":4190818,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11276680,"total_server_duration_us":982047,"encode_duration_us":16278,"operation_id":"0x8e19","timeout_ms":10000,"last_server_duration_us":202322,"total_duration_us":10010149},{"operation_name":"upsert","last_dispatch_duration_us":3682697,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10015335,"total_server_duration_us":1143214,"encode_duration_us":18804,"operation_id":"0x8edb","timeout_ms":10000,"last_server_duration_us":583246,"total_duration_us":10009533},{"operation_name":"upsert","last_dispatch_duration_us":8157363,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":8157363,"total_server_duration_us":19887,"encode_duration_us":16645,"operation_id":"0x8edf","timeout_ms":10000,"last_server_duration_us":19887,"total_duration_us":10006511},{"operation_name":"upsert","last_dispatch_duration_us":4189037,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11825426,"total_server_duration_us":2024989,"encode_duration_us":16327,"operation_id":"0x8dba","timeout_ms":10000,"last_server_duration_us":792150,"total_duration_us":10006449},{"operation_name":"upsert","last_dispatch_duration_us":3463914,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10421788,"total_server_duration_us":1356855,"encode_duration_us":16584,"operation_id":"0x8edc","timeout_ms":10000,"last_server_duration_us":388978,"total_duration_us":10005492},{"operation_name":"upsert","last_dispatch_duration_us":4023804,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12900212,"total_server_duration_us":4177498,"encode_duration_us":16504,"operation_id":"0x8eeb","timeout_ms":10000,"last_server_duration_us":149016,"total_duration_us":10005471},{"operation_name":"upsert","last_dispatch_duration_us":72879,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":474626,"total_server_duration_us":262769,"encode_duration_us":18102,"operation_id":"0x8e4c","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10005283},{"operation_name":"upsert","last_dispatch_duration_us":4652004,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10574859,"total_server_duration_us":1677432,"encode_duration_us":24173,"operation_id":"0x8e1c","timeout_ms":10000,"last_server_duration_us":373240,"total_duration_us":10003281},{"operation_name":"upsert","last_dispatch_duration_us":4445429,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":13810610,"total_server_duration_us":1850245,"encode_duration_us":16568,"operation_id":"0x8deb","timeout_ms":10000,"last_server_duration_us":614025,"total_duration_us":10002301}],"total_count":17}} INFO [2022-01-10 05:48:32,701] jepsen worker 5 - jepsen.util 4595 :info :write [1 49] :RequestTimeoutException INFO [2022-01-10 05:48:32,702] jepsen worker 5 - jepsen.util 4685 :invoke :write [1 38] INFO [2022-01-10 05:48:32,846] jepsen worker 53 - jepsen.util 5633 :ok :read [17 41] INFO [2022-01-10 05:48:32,847] jepsen worker 53 - jepsen.util 5633 :invoke :read [17 nil] INFO [2022-01-10 05:48:32,856] jepsen worker 4 - jepsen.util 4864 :ok :read [1 49] INFO [2022-01-10 05:48:32,857] jepsen worker 4 - jepsen.util 4864 :invoke :write [1 39] INFO [2022-01-10 05:48:32,981] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.5964912280701755}, :ns_1@172.23.107.227 {:progress 0.4164222873900293}, :ns_1@172.23.105.187 {:progress 0.4164222873900293}, :ns_1@172.23.107.228 {:progress 0.5705882352941176}} INFO [2022-01-10 05:48:32,982] jepsen worker nemesis - couchbase.util Rebalance check count: 659 INFO [2022-01-10 05:48:32,982] jepsen worker nemesis - couchbase.util Rebalance stuck count: 555 INFO [2022-01-10 05:48:32,982] jepsen worker 27 - jepsen.util 4887 :info :write [9 25] :RequestTimeoutException INFO [2022-01-10 05:48:32,982] jepsen worker 27 - jepsen.util 4977 :invoke :write [9 17] INFO [2022-01-10 05:48:33,126] jepsen worker 34 - jepsen.util 4444 :info :write [11 19] :RequestTimeoutException INFO [2022-01-10 05:48:33,126] jepsen worker 63 - jepsen.util 4293 :info :write [21 41] :RequestTimeoutException INFO [2022-01-10 05:48:33,127] jepsen worker 34 - jepsen.util 4534 :invoke :read [11 nil] INFO [2022-01-10 05:48:33,127] jepsen worker 63 - jepsen.util 4383 :invoke :read [21 nil] INFO [2022-01-10 05:48:33,285] jepsen worker 77 - jepsen.util 5117 :info :write [25 8] :RequestTimeoutException INFO [2022-01-10 05:48:33,285] jepsen worker 77 - jepsen.util 5207 :invoke :read [25 nil] INFO [2022-01-10 05:48:33,416] jepsen worker 37 - jepsen.util 4267 :info :write [12 27] :RequestTimeoutException INFO [2022-01-10 05:48:33,417] jepsen worker 37 - jepsen.util 4357 :invoke :read [12 nil] INFO [2022-01-10 05:48:33,421] jepsen worker 71 - jepsen.util 4211 :info :write [23 15] :RequestTimeoutException INFO [2022-01-10 05:48:33,421] jepsen worker 71 - jepsen.util 4301 :invoke :read [23 nil] INFO [2022-01-10 05:48:33,491] jepsen worker 22 - jepsen.util 4432 :ok :read [7 :nil] INFO [2022-01-10 05:48:33,491] jepsen worker 22 - jepsen.util 4432 :invoke :write [7 3] INFO [2022-01-10 05:48:33,606] jepsen worker 12 - jepsen.util 5142 :info :write [4 19] :RequestTimeoutException INFO [2022-01-10 05:48:33,607] jepsen worker 12 - jepsen.util 5232 :invoke :write [4 44] INFO [2022-01-10 05:48:33,719] jepsen worker 53 - jepsen.util 5633 :ok :read [17 41] INFO [2022-01-10 05:48:33,720] jepsen worker 79 - jepsen.util 5389 :info :write [26 19] :RequestTimeoutException INFO [2022-01-10 05:48:33,720] jepsen worker 53 - jepsen.util 5633 :invoke :write [17 3] INFO [2022-01-10 05:48:33,721] jepsen worker 79 - jepsen.util 5479 :invoke :write [26 43] INFO [2022-01-10 05:48:33,799] jepsen worker 23 - jepsen.util 4793 :ok :read [7 :nil] INFO [2022-01-10 05:48:33,799] jepsen worker 23 - jepsen.util 4793 :invoke :read [7 nil] INFO [2022-01-10 05:48:33,808] jepsen worker 61 - jepsen.util 4921 :ok :read [20 :nil] INFO [2022-01-10 05:48:33,809] jepsen worker 61 - jepsen.util 4921 :invoke :write [20 2] INFO [2022-01-10 05:48:33,891] jepsen worker 0 - jepsen.util 4680 :info :write [0 5] :RequestTimeoutException INFO [2022-01-10 05:48:33,892] jepsen worker 0 - jepsen.util 4770 :invoke :write [0 47] INFO [2022-01-10 05:48:33,910] jepsen worker 86 - jepsen.util 3956 :ok :read [28 45] INFO [2022-01-10 05:48:33,911] jepsen worker 86 - jepsen.util 3956 :invoke :read [28 nil] INFO [2022-01-10 05:48:33,926] jepsen worker 37 - jepsen.util 4357 :ok :read [12 14] INFO [2022-01-10 05:48:33,927] jepsen worker 37 - jepsen.util 4357 :invoke :read [12 nil] INFO [2022-01-10 05:48:33,940] jepsen worker 29 - jepsen.util 4529 :ok :read [9 22] INFO [2022-01-10 05:48:33,941] jepsen worker 29 - jepsen.util 4529 :invoke :write [9 28] INFO [2022-01-10 05:48:33,955] jepsen worker 51 - jepsen.util 5451 :info :write [17 29] :RequestTimeoutException INFO [2022-01-10 05:48:33,956] jepsen worker 51 - jepsen.util 5541 :invoke :read [17 nil] INFO [2022-01-10 05:48:33,957] jepsen worker 50 - jepsen.util 4820 :ok :read [16 38] INFO [2022-01-10 05:48:33,958] jepsen worker 50 - jepsen.util 4820 :invoke :write [16 0] INFO [2022-01-10 05:48:33,987] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6023391812865497}, :ns_1@172.23.107.227 {:progress 0.4164222873900293}, :ns_1@172.23.105.187 {:progress 0.4164222873900293}, :ns_1@172.23.107.228 {:progress 0.5705882352941176}} INFO [2022-01-10 05:48:33,987] jepsen worker nemesis - couchbase.util Rebalance check count: 660 INFO [2022-01-10 05:48:33,987] jepsen worker nemesis - couchbase.util Rebalance stuck count: 555 INFO [2022-01-10 05:48:34,102] jepsen worker 37 - jepsen.util 4357 :ok :read [12 14] INFO [2022-01-10 05:48:34,103] jepsen worker 37 - jepsen.util 4357 :invoke :write [12 24] INFO [2022-01-10 05:48:34,136] jepsen worker 76 - jepsen.util 3946 :info :write [25 19] :RequestTimeoutException INFO [2022-01-10 05:48:34,136] jepsen worker 60 - jepsen.util 4920 :info :write [20 24] :RequestTimeoutException INFO [2022-01-10 05:48:34,137] jepsen worker 60 - jepsen.util 5010 :invoke :write [20 48] INFO [2022-01-10 05:48:34,137] jepsen worker 76 - jepsen.util 4036 :invoke :write [25 23] INFO [2022-01-10 05:48:34,203] jepsen worker 71 - jepsen.util 4301 :ok :read [23 26] INFO [2022-01-10 05:48:34,203] jepsen worker 71 - jepsen.util 4301 :invoke :write [23 5] INFO [2022-01-10 05:48:34,207] jepsen worker 59 - jepsen.util 4469 :ok :read [19 31] INFO [2022-01-10 05:48:34,208] jepsen worker 59 - jepsen.util 4469 :invoke :write [19 9] INFO [2022-01-10 05:48:34,312] jepsen worker 47 - jepsen.util 4727 :info :write [15 18] :RequestTimeoutException INFO [2022-01-10 05:48:34,313] jepsen worker 47 - jepsen.util 4817 :invoke :read [15 nil] INFO [2022-01-10 05:48:34,367] jepsen worker 28 - jepsen.util 4348 :ok :read [9 22] INFO [2022-01-10 05:48:34,368] jepsen worker 28 - jepsen.util 4348 :invoke :read [9 nil] INFO [2022-01-10 05:48:34,377] jepsen worker 47 - jepsen.util 4817 :ok :read [15 23] INFO [2022-01-10 05:48:34,379] jepsen worker 47 - jepsen.util 4817 :invoke :read [15 nil] INFO [2022-01-10 05:48:34,450] jepsen worker 34 - jepsen.util 4534 :ok :read [11 21] INFO [2022-01-10 05:48:34,450] jepsen worker 34 - jepsen.util 4534 :invoke :write [11 32] INFO [2022-01-10 05:48:34,556] jepsen worker 44 - jepsen.util 4724 :info :write [14 37] :RequestTimeoutException INFO [2022-01-10 05:48:34,557] jepsen worker 44 - jepsen.util 4814 :invoke :read [14 nil] INFO [2022-01-10 05:48:34,626] jepsen worker 41 - jepsen.util 4631 :fail :read [13 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:48:34,627] jepsen worker 41 - jepsen.util 4631 :invoke :read [13 nil] INFO [2022-01-10 05:48:34,640] jepsen worker 67 - jepsen.util 4297 :ok :read [22 41] INFO [2022-01-10 05:48:34,641] jepsen worker 67 - jepsen.util 4297 :invoke :write [22 29] INFO [2022-01-10 05:48:34,706] jepsen worker 21 - jepsen.util 4701 :info :write [7 0] :RequestTimeoutException INFO [2022-01-10 05:48:34,707] jepsen worker 21 - jepsen.util 4791 :invoke :read [7 nil] INFO [2022-01-10 05:48:34,726] jepsen worker 62 - jepsen.util 4922 :info :write [20 4] :RequestTimeoutException INFO [2022-01-10 05:48:34,726] jepsen worker 62 - jepsen.util 5012 :invoke :read [20 nil] INFO [2022-01-10 05:48:34,779] jepsen worker 2 - jepsen.util 4772 :info :write [0 38] :RequestTimeoutException INFO [2022-01-10 05:48:34,780] jepsen worker 2 - jepsen.util 4862 :invoke :read [0 nil] INFO [2022-01-10 05:48:34,802] jepsen worker 57 - jepsen.util 5367 :info :write [19 19] :RequestTimeoutException INFO [2022-01-10 05:48:34,803] jepsen worker 57 - jepsen.util 5457 :invoke :write [19 24] INFO [2022-01-10 05:48:34,948] jepsen worker 41 - jepsen.util 4631 :ok :read [13 15] INFO [2022-01-10 05:48:34,948] jepsen worker 41 - jepsen.util 4631 :invoke :read [13 nil] INFO [2022-01-10 05:48:34,986] jepsen worker 84 - jepsen.util 4404 :info :write [28 46] :RequestTimeoutException INFO [2022-01-10 05:48:34,986] jepsen worker 84 - jepsen.util 4494 :invoke :read [28 nil] INFO [2022-01-10 05:48:34,992] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6023391812865497}, :ns_1@172.23.107.227 {:progress 0.4164222873900293}, :ns_1@172.23.105.187 {:progress 0.4164222873900293}, :ns_1@172.23.107.228 {:progress 0.5705882352941176}} INFO [2022-01-10 05:48:34,993] jepsen worker nemesis - couchbase.util Rebalance check count: 661 INFO [2022-01-10 05:48:34,993] jepsen worker nemesis - couchbase.util Rebalance stuck count: 556 INFO [2022-01-10 05:48:35,055] jepsen worker 15 - jepsen.util 4245 :ok :read [5 25] INFO [2022-01-10 05:48:35,056] jepsen worker 15 - jepsen.util 4245 :invoke :write [5 19] INFO [2022-01-10 05:48:35,065] jepsen worker 47 - jepsen.util 4817 :ok :read [15 23] INFO [2022-01-10 05:48:35,066] jepsen worker 47 - jepsen.util 4817 :invoke :read [15 nil] INFO [2022-01-10 05:48:35,067] jepsen worker 86 - jepsen.util 3956 :ok :read [28 45] INFO [2022-01-10 05:48:35,067] jepsen worker 86 - jepsen.util 3956 :invoke :write [28 40] INFO [2022-01-10 05:48:35,113] jepsen worker 56 - jepsen.util 4916 :ok :read [18 14] INFO [2022-01-10 05:48:35,114] jepsen worker 56 - jepsen.util 4916 :invoke :read [18 nil] INFO [2022-01-10 05:48:35,159] jepsen worker 87 - jepsen.util 4497 :ok :read [29 0] INFO [2022-01-10 05:48:35,160] jepsen worker 87 - jepsen.util 4497 :invoke :write [29 29] INFO [2022-01-10 05:48:35,167] jepsen worker 55 - jepsen.util 5185 :ok :read [18 14] INFO [2022-01-10 05:48:35,168] jepsen worker 55 - jepsen.util 5185 :invoke :write [18 30] INFO [2022-01-10 05:48:35,178] jepsen worker 54 - jepsen.util 4914 :ok :read [18 14] INFO [2022-01-10 05:48:35,179] jepsen worker 54 - jepsen.util 4914 :invoke :write [18 34] INFO [2022-01-10 05:48:35,179] jepsen worker 56 - jepsen.util 4916 :ok :read [18 14] INFO [2022-01-10 05:48:35,180] jepsen worker 56 - jepsen.util 4916 :invoke :read [18 nil] INFO [2022-01-10 05:48:35,236] jepsen worker 14 - jepsen.util 5594 :info :write [4 33] :RequestTimeoutException INFO [2022-01-10 05:48:35,237] jepsen worker 14 - jepsen.util 5684 :invoke :read [4 nil] INFO [2022-01-10 05:48:35,243] jepsen worker 56 - jepsen.util 4916 :ok :read [18 14] INFO [2022-01-10 05:48:35,244] jepsen worker 56 - jepsen.util 4916 :invoke :read [18 nil] INFO [2022-01-10 05:48:35,316] jepsen worker 77 - jepsen.util 5207 :ok :read [25 :nil] INFO [2022-01-10 05:48:35,317] jepsen worker 77 - jepsen.util 5207 :invoke :read [25 nil] INFO [2022-01-10 05:48:35,329] jepsen worker 24 - jepsen.util 4254 :info :write [8 15] :RequestTimeoutException INFO [2022-01-10 05:48:35,330] jepsen worker 24 - jepsen.util 4344 :invoke :read [8 nil] INFO [2022-01-10 05:48:35,344] jepsen worker 41 - jepsen.util 4631 :ok :read [13 15] INFO [2022-01-10 05:48:35,346] jepsen worker 41 - jepsen.util 4631 :invoke :write [13 47] INFO [2022-01-10 05:48:35,368] jepsen worker 56 - jepsen.util 4916 :ok :read [18 14] INFO [2022-01-10 05:48:35,369] jepsen worker 56 - jepsen.util 4916 :invoke :write [18 41] INFO [2022-01-10 05:48:35,462] jepsen worker 7 - jepsen.util 4957 :info :write [2 13] :RequestTimeoutException INFO [2022-01-10 05:48:35,462] jepsen worker 7 - jepsen.util 5047 :invoke :write [2 6] INFO [2022-01-10 05:48:35,555] jepsen worker 2 - jepsen.util 4862 :ok :read [0 46] INFO [2022-01-10 05:48:35,555] jepsen worker 2 - jepsen.util 4862 :invoke :write [0 7] INFO [2022-01-10 05:48:35,895] jepsen worker 44 - jepsen.util 4814 :ok :read [14 21] INFO [2022-01-10 05:48:35,895] jepsen worker 44 - jepsen.util 4814 :invoke :read [14 nil] INFO [2022-01-10 05:48:35,965] jepsen worker 30 - jepsen.util 4260 :info :write [10 48] :RequestTimeoutException INFO [2022-01-10 05:48:35,966] jepsen worker 30 - jepsen.util 4350 :invoke :write [10 4] INFO [2022-01-10 05:48:35,968] jepsen worker 43 - jepsen.util 4723 :ok :read [14 21] INFO [2022-01-10 05:48:35,969] jepsen worker 43 - jepsen.util 4723 :invoke :read [14 nil] INFO [2022-01-10 05:48:35,977] jepsen worker 47 - jepsen.util 4817 :ok :read [15 23] INFO [2022-01-10 05:48:35,977] jepsen worker 47 - jepsen.util 4817 :invoke :write [15 40] INFO [2022-01-10 05:48:35,998] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6081871345029239}, :ns_1@172.23.107.227 {:progress 0.4193548387096774}, :ns_1@172.23.105.187 {:progress 0.4193548387096774}, :ns_1@172.23.107.228 {:progress 0.5823529411764705}} INFO [2022-01-10 05:48:35,998] jepsen worker nemesis - couchbase.util Rebalance check count: 662 INFO [2022-01-10 05:48:35,998] jepsen worker nemesis - couchbase.util Rebalance stuck count: 556 INFO [2022-01-10 05:48:36,085] jepsen worker 89 - jepsen.util 5129 :info :write [29 4] :RequestTimeoutException INFO [2022-01-10 05:48:36,085] jepsen worker 89 - jepsen.util 5219 :invoke :write [29 41] INFO [2022-01-10 05:48:36,126] jepsen worker 78 - jepsen.util 5748 :info :write [26 47] :RequestTimeoutException INFO [2022-01-10 05:48:36,127] jepsen worker 78 - jepsen.util 5838 :invoke :read [26 nil] INFO [2022-01-10 05:48:36,291] jepsen worker 36 - jepsen.util 3546 :info :write [12 20] :RequestTimeoutException INFO [2022-01-10 05:48:36,292] jepsen worker 36 - jepsen.util 3636 :invoke :read [12 nil] INFO [2022-01-10 05:48:36,838] jepsen worker 84 - jepsen.util 4494 :ok :read [28 45] INFO [2022-01-10 05:48:36,838] jepsen worker 84 - jepsen.util 4494 :invoke :read [28 nil] INFO [2022-01-10 05:48:36,955] jepsen worker 16 - jepsen.util 4696 :info :write [5 16] :RequestTimeoutException INFO [2022-01-10 05:48:36,955] jepsen worker 16 - jepsen.util 4786 :invoke :write [5 14] INFO [2022-01-10 05:48:36,956] jepsen worker 25 - jepsen.util 4345 :info :write [8 47] :RequestTimeoutException INFO [2022-01-10 05:48:36,968] jepsen worker 25 - jepsen.util 4435 :invoke :read [8 nil] INFO [2022-01-10 05:48:37,003] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6081871345029239}, :ns_1@172.23.107.227 {:progress 0.4193548387096774}, :ns_1@172.23.105.187 {:progress 0.4193548387096774}, :ns_1@172.23.107.228 {:progress 0.5823529411764705}} INFO [2022-01-10 05:48:37,003] jepsen worker nemesis - couchbase.util Rebalance check count: 663 INFO [2022-01-10 05:48:37,003] jepsen worker nemesis - couchbase.util Rebalance stuck count: 557 INFO [2022-01-10 05:48:37,061] jepsen worker 62 - jepsen.util 5012 :ok :read [20 :nil] INFO [2022-01-10 05:48:37,061] jepsen worker 77 - jepsen.util 5207 :ok :read [25 :nil] INFO [2022-01-10 05:48:37,062] jepsen worker 62 - jepsen.util 5012 :invoke :write [20 31] INFO [2022-01-10 05:48:37,065] jepsen worker 42 - jepsen.util 4362 :info :write [14 43] :RequestTimeoutException INFO [2022-01-10 05:48:37,066] jepsen worker 32 - jepsen.util 4892 :info :write [10 23] :RequestTimeoutException INFO [2022-01-10 05:48:37,067] jepsen worker 77 - jepsen.util 5207 :invoke :write [25 47] INFO [2022-01-10 05:48:37,072] jepsen worker 32 - jepsen.util 4982 :invoke :read [10 nil] INFO [2022-01-10 05:48:37,073] jepsen worker 42 - jepsen.util 4452 :invoke :write [14 28] INFO [2022-01-10 05:48:37,101] jepsen worker 32 - jepsen.util 4982 :ok :read [10 12] INFO [2022-01-10 05:48:37,102] jepsen worker 32 - jepsen.util 4982 :invoke :write [10 46] INFO [2022-01-10 05:48:37,112] jepsen worker 46 - jepsen.util 4726 :info :write [15 21] :RequestTimeoutException INFO [2022-01-10 05:48:37,112] jepsen worker 46 - jepsen.util 4816 :invoke :read [15 nil] INFO [2022-01-10 05:48:37,175] jepsen worker 83 - jepsen.util 5303 :info :write [27 38] :RequestTimeoutException INFO [2022-01-10 05:48:37,175] jepsen worker 83 - jepsen.util 5393 :invoke :write [27 16] INFO [2022-01-10 05:48:37,191] jepsen worker 44 - jepsen.util 4814 :ok :read [14 21] INFO [2022-01-10 05:48:37,192] jepsen worker 44 - jepsen.util 4814 :invoke :read [14 nil] INFO [2022-01-10 05:48:37,229] jepsen worker 74 - jepsen.util 5114 :info :write [24 5] :RequestTimeoutException INFO [2022-01-10 05:48:37,230] jepsen worker 74 - jepsen.util 5204 :invoke :write [24 21] INFO [2022-01-10 05:48:37,255] jepsen worker 31 - jepsen.util 4801 :info :write [10 31] :RequestTimeoutException INFO [2022-01-10 05:48:37,256] jepsen worker 31 - jepsen.util 4891 :invoke :write [10 47] INFO [2022-01-10 05:48:37,482] jepsen worker 65 - jepsen.util 4385 :info :write [21 47] :RequestTimeoutException INFO [2022-01-10 05:48:37,483] jepsen worker 65 - jepsen.util 4475 :invoke :read [21 nil] INFO [2022-01-10 05:48:37,956] jepsen worker 19 - jepsen.util 4789 :info :write [6 3] :RequestTimeoutException INFO [2022-01-10 05:48:37,957] jepsen worker 19 - jepsen.util 4879 :invoke :read [6 nil] INFO [2022-01-10 05:48:37,980] jepsen worker 20 - jepsen.util 4340 :info :write [6 25] :RequestTimeoutException INFO [2022-01-10 05:48:37,980] jepsen worker 20 - jepsen.util 4430 :invoke :read [6 nil] INFO [2022-01-10 05:48:38,009] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6140350877192983}, :ns_1@172.23.107.227 {:progress 0.4193548387096774}, :ns_1@172.23.105.187 {:progress 0.4193548387096774}, :ns_1@172.23.107.228 {:progress 0.5823529411764705}} INFO [2022-01-10 05:48:38,010] jepsen worker nemesis - couchbase.util Rebalance check count: 664 INFO [2022-01-10 05:48:38,010] jepsen worker nemesis - couchbase.util Rebalance stuck count: 557 INFO [2022-01-10 05:48:38,022] jepsen worker 11 - jepsen.util 4511 :info :write [3 30] :RequestTimeoutException INFO [2022-01-10 05:48:38,022] jepsen worker 11 - jepsen.util 4601 :invoke :write [3 37] INFO [2022-01-10 05:48:38,036] jepsen worker 33 - jepsen.util 4893 :info :write [11 17] :RequestTimeoutException INFO [2022-01-10 05:48:38,037] jepsen worker 33 - jepsen.util 4983 :invoke :write [11 43] INFO [2022-01-10 05:48:38,381] jepsen worker 70 - jepsen.util 4660 :info :write [23 25] :RequestTimeoutException INFO [2022-01-10 05:48:38,381] jepsen worker 70 - jepsen.util 4750 :invoke :write [23 29] INFO [2022-01-10 05:48:38,384] jepsen worker 66 - jepsen.util 4476 :info :write [22 6] :RequestTimeoutException INFO [2022-01-10 05:48:38,385] jepsen worker 66 - jepsen.util 4566 :invoke :write [22 4] INFO [2022-01-10 05:48:38,410] jepsen worker 69 - jepsen.util 4209 :info :write [23 40] :RequestTimeoutException INFO [2022-01-10 05:48:38,410] jepsen worker 69 - jepsen.util 4299 :invoke :read [23 nil] INFO [2022-01-10 05:48:38,441] jepsen worker 69 - jepsen.util 4299 :ok :read [23 26] INFO [2022-01-10 05:48:38,442] jepsen worker 69 - jepsen.util 4299 :invoke :read [23 nil] INFO [2022-01-10 05:48:38,444] jepsen worker 35 - jepsen.util 4355 :info :write [11 31] :RequestTimeoutException INFO [2022-01-10 05:48:38,445] jepsen worker 35 - jepsen.util 4445 :invoke :write [11 18] INFO [2022-01-10 05:48:38,465] jepsen worker 72 - jepsen.util 4932 :info :write [24 4] :RequestTimeoutException INFO [2022-01-10 05:48:38,467] jepsen worker 72 - jepsen.util 5022 :invoke :write [24 18] INFO [2022-01-10 05:48:38,470] jepsen worker 75 - jepsen.util 4485 :info :write [25 5] :RequestTimeoutException INFO [2022-01-10 05:48:38,470] jepsen worker 75 - jepsen.util 4575 :invoke :write [25 24] INFO [2022-01-10 05:48:38,509] jepsen worker 69 - jepsen.util 4299 :ok :read [23 26] INFO [2022-01-10 05:48:38,509] jepsen worker 69 - jepsen.util 4299 :invoke :write [23 31] INFO [2022-01-10 05:48:38,549] jepsen worker 46 - jepsen.util 4816 :ok :read [15 23] INFO [2022-01-10 05:48:38,549] jepsen worker 46 - jepsen.util 4816 :invoke :read [15 nil] INFO [2022-01-10 05:48:38,681] jepsen worker 84 - jepsen.util 4494 :ok :read [28 45] INFO [2022-01-10 05:48:38,681] jepsen worker 84 - jepsen.util 4494 :invoke :read [28 nil] INFO [2022-01-10 05:48:38,688] jepsen worker 20 - jepsen.util 4430 :ok :read [6 25] INFO [2022-01-10 05:48:38,689] jepsen worker 20 - jepsen.util 4430 :invoke :write [6 20] INFO [2022-01-10 05:48:38,702] jepsen worker 6 - jepsen.util 4506 :info :write [2 37] :RequestTimeoutException INFO [2022-01-10 05:48:38,702] jepsen worker 6 - jepsen.util 4596 :invoke :write [2 12] INFO [2022-01-10 05:48:38,821] jepsen worker 40 - jepsen.util 4270 :info :write [13 26] :RequestTimeoutException INFO [2022-01-10 05:48:38,822] jepsen worker 40 - jepsen.util 4360 :invoke :write [13 23] INFO [2022-01-10 05:48:38,864] jepsen worker 10 - jepsen.util 4510 :ok :read [3 12] INFO [2022-01-10 05:48:38,865] jepsen worker 10 - jepsen.util 4510 :invoke :write [3 28] INFO [2022-01-10 05:48:38,870] jepsen worker 9 - jepsen.util 5229 :ok :read [3 12] INFO [2022-01-10 05:48:38,870] jepsen worker 9 - jepsen.util 5229 :invoke :write [3 3] INFO [2022-01-10 05:48:39,015] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6140350877192983}, :ns_1@172.23.107.227 {:progress 0.4193548387096774}, :ns_1@172.23.105.187 {:progress 0.4193548387096774}, :ns_1@172.23.107.228 {:progress 0.5823529411764705}} INFO [2022-01-10 05:48:39,015] jepsen worker nemesis - couchbase.util Rebalance check count: 665 INFO [2022-01-10 05:48:39,015] jepsen worker nemesis - couchbase.util Rebalance stuck count: 558 INFO [2022-01-10 05:48:39,096] jepsen worker 64 - jepsen.util 4654 :info :write [21 13] :RequestTimeoutException INFO [2022-01-10 05:48:39,097] jepsen worker 64 - jepsen.util 4744 :invoke :write [21 42] INFO [2022-01-10 05:48:39,592] jepsen worker 44 - jepsen.util 4814 :ok :read [14 21] INFO [2022-01-10 05:48:39,593] jepsen worker 44 - jepsen.util 4814 :invoke :read [14 nil] INFO [2022-01-10 05:48:39,614] jepsen worker 26 - jepsen.util 4436 :info :write [8 33] :RequestTimeoutException INFO [2022-01-10 05:48:39,616] jepsen worker 26 - jepsen.util 4526 :invoke :write [8 23] INFO [2022-01-10 05:48:39,642] jepsen worker 23 - jepsen.util 4793 :ok :read [7 :nil] INFO [2022-01-10 05:48:39,643] jepsen worker 23 - jepsen.util 4793 :invoke :write [7 36] INFO [2022-01-10 05:48:39,672] jepsen worker 48 - jepsen.util 4458 :info :write [16 35] :RequestTimeoutException INFO [2022-01-10 05:48:39,673] jepsen worker 48 - jepsen.util 4548 :invoke :write [16 3] INFO [2022-01-10 05:48:39,678] jepsen worker 43 - jepsen.util 4723 :ok :read [14 21] INFO [2022-01-10 05:48:39,679] jepsen worker 43 - jepsen.util 4723 :invoke :read [14 nil] INFO [2022-01-10 05:48:39,806] jepsen worker 13 - jepsen.util 5413 :info :write [4 32] :RequestTimeoutException INFO [2022-01-10 05:48:39,807] jepsen worker 13 - jepsen.util 5503 :invoke :write [4 42] INFO [2022-01-10 05:48:40,012] jepsen worker 80 - jepsen.util 5750 :info :write [26 11] :RequestTimeoutException INFO [2022-01-10 05:48:40,012] jepsen worker 80 - jepsen.util 5840 :invoke :write [26 40] INFO [2022-01-10 05:48:40,021] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6198830409356726}, :ns_1@172.23.107.227 {:progress 0.4369501466275659}, :ns_1@172.23.105.187 {:progress 0.4369501466275659}, :ns_1@172.23.107.228 {:progress 0.5941176470588235}} INFO [2022-01-10 05:48:40,021] jepsen worker nemesis - couchbase.util Rebalance check count: 666 INFO [2022-01-10 05:48:40,021] jepsen worker nemesis - couchbase.util Rebalance stuck count: 558 INFO [2022-01-10 05:48:40,111] jepsen worker 49 - jepsen.util 4459 :info :write [16 13] :RequestTimeoutException INFO [2022-01-10 05:48:40,112] jepsen worker 49 - jepsen.util 4549 :invoke :write [16 24] INFO [2022-01-10 05:48:40,356] jepsen worker 38 - jepsen.util 4268 :info :write [12 25] :RequestTimeoutException INFO [2022-01-10 05:48:40,357] jepsen worker 38 - jepsen.util 4358 :invoke :read [12 nil] INFO [2022-01-10 05:48:40,424] jepsen worker 38 - jepsen.util 4358 :ok :read [12 14] INFO [2022-01-10 05:48:40,425] jepsen worker 38 - jepsen.util 4358 :invoke :write [12 4] INFO [2022-01-10 05:48:40,538] jepsen worker 14 - jepsen.util 5684 :ok :read [4 3] INFO [2022-01-10 05:48:40,539] jepsen worker 14 - jepsen.util 5684 :invoke :read [4 nil] INFO [2022-01-10 05:48:40,558] jepsen worker 36 - jepsen.util 3636 :ok :read [12 14] INFO [2022-01-10 05:48:40,558] jepsen worker 36 - jepsen.util 3636 :invoke :write [12 3] INFO [2022-01-10 05:48:40,571] jepsen worker 3 - jepsen.util 4593 :info :write [1 43] :RequestTimeoutException INFO [2022-01-10 05:48:40,572] jepsen worker 3 - jepsen.util 4683 :invoke :read [1 nil] INFO [2022-01-10 05:48:40,591] jepsen worker 14 - jepsen.util 5684 :ok :read [4 3] INFO [2022-01-10 05:48:40,592] jepsen worker 14 - jepsen.util 5684 :invoke :write [4 44] WARN [2022-01-10 05:48:40,679] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10826208,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16267265,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":12161430,"encode_duration_us":19538,"operation_id":"0x8eb6","timeout_ms":10000,"last_server_duration_us":9796544,"total_duration_us":10004771},{"operation_name":"upsert","last_dispatch_duration_us":18134992,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":27062767,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9119079,"encode_duration_us":17703,"operation_id":"0x8ced","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10004999},{"operation_name":"upsert","last_dispatch_duration_us":10868759,"last_local_id":"EFE9906200000001/000000007CF9F49A","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":19745444,"last_local_socket":"172.23.104.79:44306","total_server_duration_us":13967956,"encode_duration_us":18856,"operation_id":"0x8f45","timeout_ms":10000,"last_server_duration_us":9459645,"total_duration_us":10007430}],"total_count":3}} INFO [2022-01-10 05:48:40,711] jepsen worker 3 - jepsen.util 4683 :ok :read [1 49] INFO [2022-01-10 05:48:40,712] jepsen worker 3 - jepsen.util 4683 :invoke :write [1 1] INFO [2022-01-10 05:48:40,756] jepsen worker 88 - jepsen.util 4768 :info :write [29 8] :RequestTimeoutException INFO [2022-01-10 05:48:40,756] jepsen worker 88 - jepsen.util 4858 :invoke :read [29 nil] WARN [2022-01-10 05:48:40,780] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2552731,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4813278,"total_server_duration_us":796687,"encode_duration_us":16454,"operation_id":"0x907f","timeout_ms":10000,"last_server_duration_us":413816,"total_duration_us":10009561},{"operation_name":"upsert","last_dispatch_duration_us":1871849,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9847650,"total_server_duration_us":1573230,"encode_duration_us":16416,"operation_id":"0x90c0","timeout_ms":10000,"last_server_duration_us":212124,"total_duration_us":10008261},{"operation_name":"upsert","last_dispatch_duration_us":153380,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9350947,"total_server_duration_us":108987,"encode_duration_us":17649,"operation_id":"0x9056","timeout_ms":10000,"last_server_duration_us":31872,"total_duration_us":10007247},{"operation_name":"upsert","last_dispatch_duration_us":7610929,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9949510,"total_server_duration_us":1052272,"encode_duration_us":19203,"operation_id":"0x8fcd","timeout_ms":10000,"last_server_duration_us":266894,"total_duration_us":10006388},{"operation_name":"upsert","last_dispatch_duration_us":2356721,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9944129,"total_server_duration_us":1340289,"encode_duration_us":16929,"operation_id":"0x8f93","timeout_ms":10000,"last_server_duration_us":190246,"total_duration_us":10004787},{"operation_name":"upsert","last_dispatch_duration_us":1265129,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":7003297,"total_server_duration_us":297213,"encode_duration_us":16921,"operation_id":"0x90f0","timeout_ms":10000,"last_server_duration_us":92589,"total_duration_us":10004339},{"operation_name":"upsert","last_dispatch_duration_us":2854854,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":2854854,"total_server_duration_us":391488,"encode_duration_us":16925,"operation_id":"0x908b","timeout_ms":10000,"last_server_duration_us":391488,"total_duration_us":10004256},{"operation_name":"upsert","last_dispatch_duration_us":9705100,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":16411171,"total_server_duration_us":1830934,"encode_duration_us":16565,"operation_id":"0x8ff6","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10004044},{"operation_name":"upsert","last_dispatch_duration_us":2378316,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10103577,"total_server_duration_us":1385557,"encode_duration_us":18707,"operation_id":"0x8f9f","timeout_ms":10000,"last_server_duration_us":399341,"total_duration_us":10003584},{"operation_name":"upsert","last_dispatch_duration_us":2470618,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":2470618,"total_server_duration_us":599891,"encode_duration_us":18422,"operation_id":"0x90be","timeout_ms":10000,"last_server_duration_us":599891,"total_duration_us":10003351}],"total_count":21}} INFO [2022-01-10 05:48:40,803] jepsen worker 78 - jepsen.util 5838 :ok :read [26 10] INFO [2022-01-10 05:48:40,804] jepsen worker 78 - jepsen.util 5838 :invoke :read [26 nil] INFO [2022-01-10 05:48:40,864] jepsen worker 78 - jepsen.util 5838 :ok :read [26 10] INFO [2022-01-10 05:48:40,865] jepsen worker 78 - jepsen.util 5838 :invoke :read [26 nil] INFO [2022-01-10 05:48:40,889] jepsen worker 17 - jepsen.util 4517 :info :write [5 25] :RequestTimeoutException INFO [2022-01-10 05:48:40,890] jepsen worker 17 - jepsen.util 4607 :invoke :read [5 nil] INFO [2022-01-10 05:48:40,910] jepsen worker 8 - jepsen.util 4778 :info :write [2 0] :RequestTimeoutException INFO [2022-01-10 05:48:40,912] jepsen worker 8 - jepsen.util 4868 :invoke :write [2 21] INFO [2022-01-10 05:48:40,941] jepsen worker 78 - jepsen.util 5838 :ok :read [26 10] INFO [2022-01-10 05:48:40,942] jepsen worker 78 - jepsen.util 5838 :invoke :write [26 46] INFO [2022-01-10 05:48:41,028] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6198830409356726}, :ns_1@172.23.107.227 {:progress 0.4369501466275659}, :ns_1@172.23.105.187 {:progress 0.4369501466275659}, :ns_1@172.23.107.228 {:progress 0.5941176470588235}} INFO [2022-01-10 05:48:41,028] jepsen worker nemesis - couchbase.util Rebalance check count: 667 INFO [2022-01-10 05:48:41,028] jepsen worker nemesis - couchbase.util Rebalance stuck count: 559 INFO [2022-01-10 05:48:41,100] jepsen worker 44 - jepsen.util 4814 :ok :read [14 21] INFO [2022-01-10 05:48:41,100] jepsen worker 44 - jepsen.util 4814 :invoke :read [14 nil] INFO [2022-01-10 05:48:41,189] jepsen worker 17 - jepsen.util 4607 :ok :read [5 25] INFO [2022-01-10 05:48:41,190] jepsen worker 17 - jepsen.util 4607 :invoke :write [5 43] INFO [2022-01-10 05:48:41,303] jepsen worker 46 - jepsen.util 4816 :ok :read [15 23] INFO [2022-01-10 05:48:41,303] jepsen worker 46 - jepsen.util 4816 :invoke :write [15 28] INFO [2022-01-10 05:48:41,778] jepsen worker 68 - jepsen.util 4928 :ok :read [22 41] INFO [2022-01-10 05:48:41,781] jepsen worker 68 - jepsen.util 4928 :invoke :read [22 nil] INFO [2022-01-10 05:48:41,795] jepsen worker 39 - jepsen.util 5079 :ok :read [13 15] INFO [2022-01-10 05:48:41,795] jepsen worker 39 - jepsen.util 5079 :invoke :read [13 nil] INFO [2022-01-10 05:48:42,036] jepsen worker 82 - jepsen.util 4762 :info :write [27 44] :RequestTimeoutException INFO [2022-01-10 05:48:42,037] jepsen worker 82 - jepsen.util 4852 :invoke :write [27 0] INFO [2022-01-10 05:48:42,055] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6257309941520468}, :ns_1@172.23.107.227 {:progress 0.4428152492668622}, :ns_1@172.23.105.187 {:progress 0.4428152492668622}, :ns_1@172.23.107.228 {:progress 0.5941176470588235}} INFO [2022-01-10 05:48:42,055] jepsen worker nemesis - couchbase.util Rebalance check count: 668 INFO [2022-01-10 05:48:42,055] jepsen worker nemesis - couchbase.util Rebalance stuck count: 559 INFO [2022-01-10 05:48:42,097] jepsen worker 85 - jepsen.util 5035 :info :write [28 25] :RequestTimeoutException INFO [2022-01-10 05:48:42,097] jepsen worker 85 - jepsen.util 5125 :invoke :read [28 nil] INFO [2022-01-10 05:48:42,110] jepsen worker 44 - jepsen.util 4814 :ok :read [14 21] INFO [2022-01-10 05:48:42,110] jepsen worker 44 - jepsen.util 4814 :invoke :write [14 8] INFO [2022-01-10 05:48:42,115] jepsen worker 73 - jepsen.util 4483 :info :write [24 20] :RequestTimeoutException INFO [2022-01-10 05:48:42,116] jepsen worker 73 - jepsen.util 4573 :invoke :write [24 15] INFO [2022-01-10 05:48:42,126] jepsen worker 52 - jepsen.util 5812 :info :write [17 39] :RequestTimeoutException INFO [2022-01-10 05:48:42,127] jepsen worker 52 - jepsen.util 5902 :invoke :read [17 nil] WARN [2022-01-10 05:48:42,190] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":13732923,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":13732923,"total_server_duration_us":32,"operation_id":"0x9029","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":13733971},{"operation_name":"upsert","last_dispatch_duration_us":2572102,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":8130399,"total_server_duration_us":1893168,"encode_duration_us":16352,"operation_id":"0x90f1","timeout_ms":10000,"last_server_duration_us":208052,"total_duration_us":10007621},{"operation_name":"upsert","last_dispatch_duration_us":8792017,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8792017,"total_server_duration_us":377355,"encode_duration_us":16606,"operation_id":"0x8f97","timeout_ms":10000,"last_server_duration_us":377355,"total_duration_us":10007425},{"operation_name":"upsert","last_dispatch_duration_us":11387540,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":17662757,"total_server_duration_us":9599834,"encode_duration_us":16480,"operation_id":"0x8fa3","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10007347},{"operation_name":"upsert","last_dispatch_duration_us":9222561,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":18442763,"total_server_duration_us":3655640,"encode_duration_us":17791,"operation_id":"0x8f98","timeout_ms":10000,"last_server_duration_us":52539,"total_duration_us":10007147},{"operation_name":"upsert","last_dispatch_duration_us":815852,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":5428608,"total_server_duration_us":459562,"encode_duration_us":16392,"operation_id":"0x9113","timeout_ms":10000,"last_server_duration_us":18799,"total_duration_us":10007014},{"operation_name":"upsert","last_dispatch_duration_us":2975066,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9573011,"total_server_duration_us":962127,"encode_duration_us":16282,"operation_id":"0x90e4","timeout_ms":10000,"last_server_duration_us":44611,"total_duration_us":10005978},{"operation_name":"upsert","last_dispatch_duration_us":9954406,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":12068828,"total_server_duration_us":641709,"encode_duration_us":16547,"operation_id":"0x90bd","timeout_ms":10000,"last_server_duration_us":26330,"total_duration_us":10003627},{"operation_name":"upsert","last_dispatch_duration_us":1611888,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8286314,"total_server_duration_us":311235,"encode_duration_us":16173,"operation_id":"0x9084","timeout_ms":10000,"last_server_duration_us":40802,"total_duration_us":10002418},{"operation_name":"upsert","last_dispatch_duration_us":8929847,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":14320575,"total_server_duration_us":721430,"encode_duration_us":16484,"operation_id":"0x905f","timeout_ms":10000,"last_server_duration_us":37653,"total_duration_us":10001762}],"total_count":22}} WARN [2022-01-10 05:48:42,272] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18010996,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18010996,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":12462116,"encode_duration_us":18166,"operation_id":"0x8fc6","timeout_ms":10000,"last_server_duration_us":12462116,"total_duration_us":10002058}],"total_count":1}} WARN [2022-01-10 05:48:42,291] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16805554,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17453284,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9368466,"encode_duration_us":16635,"operation_id":"0x8eb5","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10003251},{"operation_name":"upsert","last_dispatch_duration_us":11387540,"last_local_id":"EFE9906200000006/0000000021E2C07D","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":17662757,"last_local_socket":"172.23.104.79:44302","total_server_duration_us":9599834,"encode_duration_us":16480,"operation_id":"0x8fa3","timeout_ms":10000,"last_server_duration_us":9024634,"total_duration_us":10007347}],"total_count":2}} WARN [2022-01-10 05:48:42,372] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":8851678,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":8851678,"total_server_duration_us":38,"operation_id":"0x8f90","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":11983813},{"operation_name":"upsert","encode_duration_us":16853,"operation_id":"0x9080","timeout_ms":10000,"total_duration_us":10010344},{"operation_name":"upsert","encode_duration_us":16267,"operation_id":"0x90b2","timeout_ms":10000,"total_duration_us":10009446},{"operation_name":"upsert","last_dispatch_duration_us":8486068,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":8486068,"total_server_duration_us":761501,"encode_duration_us":16795,"operation_id":"0x9006","timeout_ms":10000,"last_server_duration_us":761501,"total_duration_us":10008331},{"operation_name":"upsert","last_dispatch_duration_us":1607666,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":10057213,"total_server_duration_us":1338643,"encode_duration_us":16150,"operation_id":"0x8fd2","timeout_ms":10000,"last_server_duration_us":208906,"total_duration_us":10007862},{"operation_name":"upsert","last_dispatch_duration_us":4429460,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4429460,"total_server_duration_us":1794529,"encode_duration_us":18659,"operation_id":"0x8fc5","timeout_ms":10000,"last_server_duration_us":1794529,"total_duration_us":10005980},{"operation_name":"upsert","last_dispatch_duration_us":1949982,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":1949982,"total_server_duration_us":30445,"encode_duration_us":17455,"operation_id":"0x916c","timeout_ms":10000,"last_server_duration_us":30445,"total_duration_us":10005513},{"operation_name":"upsert","last_dispatch_duration_us":31382,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":1027797,"total_server_duration_us":560608,"encode_duration_us":18817,"operation_id":"0x9055","timeout_ms":10000,"last_server_duration_us":20921,"total_duration_us":10004398},{"operation_name":"upsert","last_dispatch_duration_us":32813,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":1127220,"total_server_duration_us":577518,"encode_duration_us":16286,"operation_id":"0x908f","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10004024},{"operation_name":"upsert","last_dispatch_duration_us":2102460,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":7557086,"total_server_duration_us":1105689,"encode_duration_us":16168,"operation_id":"0x9031","timeout_ms":10000,"last_server_duration_us":112475,"total_duration_us":10002312}],"total_count":21}} WARN [2022-01-10 05:48:42,375] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","encode_duration_us":16828,"operation_id":"0x916a","timeout_ms":10000,"total_duration_us":10010903},{"operation_name":"upsert","last_dispatch_duration_us":1782168,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11572866,"total_server_duration_us":1477449,"encode_duration_us":18016,"operation_id":"0x8ffb","timeout_ms":10000,"last_server_duration_us":342070,"total_duration_us":10007895},{"operation_name":"upsert","last_dispatch_duration_us":974399,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8911653,"total_server_duration_us":1723069,"encode_duration_us":16283,"operation_id":"0x90e7","timeout_ms":10000,"last_server_duration_us":589548,"total_duration_us":10007331},{"operation_name":"upsert","last_dispatch_duration_us":647093,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":10389927,"total_server_duration_us":552010,"encode_duration_us":17464,"operation_id":"0x8fcf","timeout_ms":10000,"last_server_duration_us":132746,"total_duration_us":10006967},{"operation_name":"upsert","last_dispatch_duration_us":2051341,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11261773,"total_server_duration_us":2314910,"encode_duration_us":16117,"operation_id":"0x90b5","timeout_ms":10000,"last_server_duration_us":944891,"total_duration_us":10006878},{"operation_name":"upsert","last_dispatch_duration_us":60277,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9028904,"total_server_duration_us":256407,"encode_duration_us":16205,"operation_id":"0x9083","timeout_ms":10000,"last_server_duration_us":24154,"total_duration_us":10006678},{"operation_name":"upsert","last_dispatch_duration_us":31176,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":5091959,"total_server_duration_us":611822,"encode_duration_us":16224,"operation_id":"0x911a","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10004443},{"operation_name":"upsert","last_dispatch_duration_us":1589093,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10814268,"total_server_duration_us":2807845,"encode_duration_us":16655,"operation_id":"0x9004","timeout_ms":10000,"last_server_duration_us":88989,"total_duration_us":10004390},{"operation_name":"upsert","last_dispatch_duration_us":1805763,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10861407,"total_server_duration_us":2616028,"encode_duration_us":18768,"operation_id":"0x8ffc","timeout_ms":10000,"last_server_duration_us":563211,"total_duration_us":10004298},{"operation_name":"upsert","last_dispatch_duration_us":8894355,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8894355,"total_server_duration_us":1031414,"encode_duration_us":16306,"operation_id":"0x9028","timeout_ms":10000,"last_server_duration_us":1031414,"total_duration_us":10002458}],"total_count":16}} WARN [2022-01-10 05:48:42,382] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":14226718,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":14226718,"total_server_duration_us":21001,"operation_id":"0x8e4d","timeout_ms":30000,"last_server_duration_us":21001,"total_duration_us":18887829},{"operation_name":"get","last_dispatch_duration_us":10388202,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10388202,"total_server_duration_us":16339,"operation_id":"0x8fc3","timeout_ms":30000,"last_server_duration_us":16339,"total_duration_us":10422568},{"operation_name":"upsert","last_dispatch_duration_us":10039206,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10039206,"total_server_duration_us":19652,"encode_duration_us":16879,"operation_id":"0x9002","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10009480},{"operation_name":"upsert","last_dispatch_duration_us":11827429,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11827429,"total_server_duration_us":40270,"encode_duration_us":16417,"operation_id":"0x8fc2","timeout_ms":10000,"last_server_duration_us":40270,"total_duration_us":10008513},{"operation_name":"upsert","last_dispatch_duration_us":5318180,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8863300,"total_server_duration_us":935688,"encode_duration_us":17039,"operation_id":"0x9090","timeout_ms":10000,"last_server_duration_us":711807,"total_duration_us":10007689},{"operation_name":"upsert","last_dispatch_duration_us":3519783,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":3519783,"total_server_duration_us":42091,"encode_duration_us":17031,"operation_id":"0x9021","timeout_ms":10000,"last_server_duration_us":42091,"total_duration_us":10007639},{"operation_name":"upsert","last_dispatch_duration_us":3050932,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":3050932,"total_server_duration_us":42634,"encode_duration_us":16528,"operation_id":"0x8ff3","timeout_ms":10000,"last_server_duration_us":42634,"total_duration_us":10007548},{"operation_name":"upsert","last_dispatch_duration_us":5071947,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12131501,"total_server_duration_us":1938241,"encode_duration_us":18171,"operation_id":"0x90c5","timeout_ms":10000,"last_server_duration_us":1238992,"total_duration_us":10006240},{"operation_name":"upsert","last_dispatch_duration_us":3498290,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":3498290,"total_server_duration_us":190863,"encode_duration_us":16377,"operation_id":"0x9082","timeout_ms":10000,"last_server_duration_us":190863,"total_duration_us":10005856},{"operation_name":"upsert","last_dispatch_duration_us":32263,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":951514,"total_server_duration_us":588789,"encode_duration_us":18058,"operation_id":"0x90b4","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10002754}],"total_count":16}} WARN [2022-01-10 05:48:42,388] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":17231759,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17706386,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9296952,"encode_duration_us":18102,"operation_id":"0x8e4c","timeout_ms":10000,"last_server_duration_us":9034183,"total_duration_us":10005283}],"total_count":1}} INFO [2022-01-10 05:48:42,416] jepsen worker 18 - jepsen.util 4518 :info :write [6 23] :RequestTimeoutException INFO [2022-01-10 05:48:42,416] jepsen worker 18 - jepsen.util 4608 :invoke :read [6 nil] INFO [2022-01-10 05:48:42,445] jepsen worker 45 - jepsen.util 5085 :info :write [15 42] :RequestTimeoutException INFO [2022-01-10 05:48:42,446] jepsen worker 45 - jepsen.util 5175 :invoke :read [15 nil] INFO [2022-01-10 05:48:42,481] jepsen worker 81 - jepsen.util 5211 :info :write [27 44] :RequestTimeoutException INFO [2022-01-10 05:48:42,482] jepsen worker 81 - jepsen.util 5301 :invoke :write [27 39] INFO [2022-01-10 05:48:42,500] jepsen worker 58 - jepsen.util 4648 :info :write [19 17] :RequestTimeoutException INFO [2022-01-10 05:48:42,501] jepsen worker 58 - jepsen.util 4738 :invoke :write [19 42] INFO [2022-01-10 05:48:42,517] jepsen worker 52 - jepsen.util 5902 :ok :read [17 41] INFO [2022-01-10 05:48:42,518] jepsen worker 52 - jepsen.util 5902 :invoke :read [17 nil] INFO [2022-01-10 05:48:42,562] jepsen worker 1 - jepsen.util 4681 :info :write [0 33] :RequestTimeoutException INFO [2022-01-10 05:48:42,562] jepsen worker 1 - jepsen.util 4771 :invoke :write [0 6] WARN [2022-01-10 05:48:42,576] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9037734,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9739273,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9419268,"encode_duration_us":16816,"operation_id":"0x8f4b","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10000583}],"total_count":1}} WARN [2022-01-10 05:48:42,588] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12620202,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":12620202,"total_server_duration_us":32,"operation_id":"0x8fa0","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":12622869},{"operation_name":"upsert","last_dispatch_duration_us":2112473,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9426675,"total_server_duration_us":1473198,"encode_duration_us":17318,"operation_id":"0x90b3","timeout_ms":10000,"last_server_duration_us":316643,"total_duration_us":10010205},{"operation_name":"upsert","last_dispatch_duration_us":590278,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9054460,"total_server_duration_us":718962,"encode_duration_us":16770,"operation_id":"0x9140","timeout_ms":10000,"last_server_duration_us":21731,"total_duration_us":10010022},{"operation_name":"upsert","last_dispatch_duration_us":879478,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10623219,"total_server_duration_us":1068534,"encode_duration_us":17446,"operation_id":"0x8fd9","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10009866},{"operation_name":"upsert","last_dispatch_duration_us":1449803,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":4877973,"total_server_duration_us":155012,"encode_duration_us":16653,"operation_id":"0x90bf","timeout_ms":10000,"last_server_duration_us":35714,"total_duration_us":10009834},{"operation_name":"upsert","last_dispatch_duration_us":2340077,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5540698,"total_server_duration_us":579774,"encode_duration_us":22058,"operation_id":"0x9141","timeout_ms":10000,"last_server_duration_us":117116,"total_duration_us":10006972},{"operation_name":"upsert","last_dispatch_duration_us":1053851,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10401510,"total_server_duration_us":1632692,"encode_duration_us":16286,"operation_id":"0x901a","timeout_ms":10000,"last_server_duration_us":63568,"total_duration_us":10006252},{"operation_name":"upsert","last_dispatch_duration_us":2395831,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9783023,"total_server_duration_us":2090150,"encode_duration_us":18483,"operation_id":"0x908e","timeout_ms":10000,"last_server_duration_us":443713,"total_duration_us":10004713},{"operation_name":"upsert","last_dispatch_duration_us":2493204,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10966130,"total_server_duration_us":2580936,"encode_duration_us":17217,"operation_id":"0x9033","timeout_ms":10000,"last_server_duration_us":132394,"total_duration_us":10004563},{"operation_name":"upsert","last_dispatch_duration_us":1866909,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9249077,"total_server_duration_us":1147907,"encode_duration_us":16147,"operation_id":"0x90c2","timeout_ms":10000,"last_server_duration_us":197914,"total_duration_us":10002583}],"total_count":17}} INFO [2022-01-10 05:48:42,664] jepsen worker 52 - jepsen.util 5902 :ok :read [17 41] INFO [2022-01-10 05:48:42,664] jepsen worker 52 - jepsen.util 5902 :invoke :read [17 nil] INFO [2022-01-10 05:48:42,692] jepsen worker 52 - jepsen.util 5902 :ok :read [17 41] INFO [2022-01-10 05:48:42,693] jepsen worker 52 - jepsen.util 5902 :invoke :write [17 13] INFO [2022-01-10 05:48:42,726] jepsen worker 5 - jepsen.util 4685 :info :write [1 38] :RequestTimeoutException INFO [2022-01-10 05:48:42,727] jepsen worker 5 - jepsen.util 4775 :invoke :read [1 nil] INFO [2022-01-10 05:48:42,783] jepsen worker 5 - jepsen.util 4775 :ok :read [1 49] INFO [2022-01-10 05:48:42,783] jepsen worker 5 - jepsen.util 4775 :invoke :write [1 26] INFO [2022-01-10 05:48:42,880] jepsen worker 4 - jepsen.util 4864 :info :write [1 39] :RequestTimeoutException INFO [2022-01-10 05:48:42,880] jepsen worker 4 - jepsen.util 4954 :invoke :read [1 nil] INFO [2022-01-10 05:48:42,954] jepsen worker 65 - jepsen.util 4475 :ok :read [21 7] INFO [2022-01-10 05:48:42,954] jepsen worker 65 - jepsen.util 4475 :invoke :read [21 nil] INFO [2022-01-10 05:48:43,011] jepsen worker 27 - jepsen.util 4977 :info :write [9 17] :RequestTimeoutException INFO [2022-01-10 05:48:43,012] jepsen worker 27 - jepsen.util 5067 :invoke :write [9 4] INFO [2022-01-10 05:48:43,070] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6257309941520468}, :ns_1@172.23.107.227 {:progress 0.4428152492668622}, :ns_1@172.23.105.187 {:progress 0.4428152492668622}, :ns_1@172.23.107.228 {:progress 0.5941176470588235}} INFO [2022-01-10 05:48:43,070] jepsen worker nemesis - couchbase.util Rebalance check count: 669 INFO [2022-01-10 05:48:43,070] jepsen worker nemesis - couchbase.util Rebalance stuck count: 560 INFO [2022-01-10 05:48:43,428] jepsen worker 51 - jepsen.util 5541 :ok :read [17 41] INFO [2022-01-10 05:48:43,428] jepsen worker 51 - jepsen.util 5541 :invoke :write [17 41] INFO [2022-01-10 05:48:43,511] jepsen worker 4 - jepsen.util 4954 :ok :read [1 49] INFO [2022-01-10 05:48:43,511] jepsen worker 4 - jepsen.util 4954 :invoke :write [1 4] INFO [2022-01-10 05:48:43,522] jepsen worker 22 - jepsen.util 4432 :info :write [7 3] :RequestTimeoutException INFO [2022-01-10 05:48:43,523] jepsen worker 22 - jepsen.util 4522 :invoke :read [7 nil] INFO [2022-01-10 05:48:43,631] jepsen worker 12 - jepsen.util 5232 :info :write [4 44] :RequestTimeoutException INFO [2022-01-10 05:48:43,632] jepsen worker 12 - jepsen.util 5322 :invoke :write [4 3] INFO [2022-01-10 05:48:43,744] jepsen worker 79 - jepsen.util 5479 :info :write [26 43] :RequestTimeoutException INFO [2022-01-10 05:48:43,745] jepsen worker 79 - jepsen.util 5569 :invoke :read [26 nil] INFO [2022-01-10 05:48:43,750] jepsen worker 53 - jepsen.util 5633 :info :write [17 3] :RequestTimeoutException INFO [2022-01-10 05:48:43,750] jepsen worker 53 - jepsen.util 5723 :invoke :write [17 8] INFO [2022-01-10 05:48:43,834] jepsen worker 61 - jepsen.util 4921 :info :write [20 2] :RequestTimeoutException INFO [2022-01-10 05:48:43,835] jepsen worker 61 - jepsen.util 5011 :invoke :write [20 37] INFO [2022-01-10 05:48:43,916] jepsen worker 0 - jepsen.util 4770 :info :write [0 47] :RequestTimeoutException INFO [2022-01-10 05:48:43,917] jepsen worker 0 - jepsen.util 4860 :invoke :read [0 nil] INFO [2022-01-10 05:48:43,966] jepsen worker 29 - jepsen.util 4529 :info :write [9 28] :RequestTimeoutException INFO [2022-01-10 05:48:43,966] jepsen worker 29 - jepsen.util 4619 :invoke :read [9 nil] INFO [2022-01-10 05:48:43,986] jepsen worker 50 - jepsen.util 4820 :info :write [16 0] :RequestTimeoutException INFO [2022-01-10 05:48:43,987] jepsen worker 50 - jepsen.util 4910 :invoke :write [16 18] INFO [2022-01-10 05:48:44,082] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6257309941520468}, :ns_1@172.23.107.227 {:progress 0.4428152492668622}, :ns_1@172.23.105.187 {:progress 0.4428152492668622}, :ns_1@172.23.107.228 {:progress 0.5941176470588235}} INFO [2022-01-10 05:48:44,082] jepsen worker nemesis - couchbase.util Rebalance check count: 670 INFO [2022-01-10 05:48:44,082] jepsen worker nemesis - couchbase.util Rebalance stuck count: 561 INFO [2022-01-10 05:48:44,119] jepsen worker 79 - jepsen.util 5569 :ok :read [26 10] INFO [2022-01-10 05:48:44,120] jepsen worker 79 - jepsen.util 5569 :invoke :write [26 42] INFO [2022-01-10 05:48:44,131] jepsen worker 37 - jepsen.util 4357 :info :write [12 24] :RequestTimeoutException INFO [2022-01-10 05:48:44,132] jepsen worker 37 - jepsen.util 4447 :invoke :read [12 nil] INFO [2022-01-10 05:48:44,162] jepsen worker 60 - jepsen.util 5010 :info :write [20 48] :RequestTimeoutException INFO [2022-01-10 05:48:44,163] jepsen worker 60 - jepsen.util 5100 :invoke :write [20 7] INFO [2022-01-10 05:48:44,166] jepsen worker 76 - jepsen.util 4036 :info :write [25 23] :RequestTimeoutException INFO [2022-01-10 05:48:44,167] jepsen worker 76 - jepsen.util 4126 :invoke :write [25 45] INFO [2022-01-10 05:48:44,226] jepsen worker 71 - jepsen.util 4301 :info :write [23 5] :RequestTimeoutException INFO [2022-01-10 05:48:44,227] jepsen worker 71 - jepsen.util 4391 :invoke :read [23 nil] INFO [2022-01-10 05:48:44,231] jepsen worker 59 - jepsen.util 4469 :info :write [19 9] :RequestTimeoutException INFO [2022-01-10 05:48:44,232] jepsen worker 59 - jepsen.util 4559 :invoke :read [19 nil] INFO [2022-01-10 05:48:44,317] jepsen worker 37 - jepsen.util 4447 :ok :read [12 14] INFO [2022-01-10 05:48:44,318] jepsen worker 37 - jepsen.util 4447 :invoke :write [12 25] INFO [2022-01-10 05:48:44,475] jepsen worker 34 - jepsen.util 4534 :info :write [11 32] :RequestTimeoutException INFO [2022-01-10 05:48:44,476] jepsen worker 34 - jepsen.util 4624 :invoke :write [11 1] INFO [2022-01-10 05:48:44,524] jepsen worker 56 - jepsen.util 4916 :info :write [18 41] :SyncWriteAmbiguous INFO [2022-01-10 05:48:44,525] jepsen worker 56 - jepsen.util 5006 :invoke :write [18 40] INFO [2022-01-10 05:48:44,532] jepsen worker 71 - jepsen.util 4391 :ok :read [23 26] INFO [2022-01-10 05:48:44,532] jepsen worker 71 - jepsen.util 4391 :invoke :read [23 nil] INFO [2022-01-10 05:48:44,666] jepsen worker 67 - jepsen.util 4297 :info :write [22 29] :RequestTimeoutException INFO [2022-01-10 05:48:44,666] jepsen worker 67 - jepsen.util 4387 :invoke :read [22 nil] INFO [2022-01-10 05:48:44,826] jepsen worker 57 - jepsen.util 5457 :info :write [19 24] :RequestTimeoutException INFO [2022-01-10 05:48:44,827] jepsen worker 57 - jepsen.util 5547 :invoke :read [19 nil] INFO [2022-01-10 05:48:44,834] jepsen worker 0 - jepsen.util 4860 :ok :read [0 46] INFO [2022-01-10 05:48:44,834] jepsen worker 0 - jepsen.util 4860 :invoke :write [0 20] INFO [2022-01-10 05:48:45,082] jepsen worker 15 - jepsen.util 4245 :info :write [5 19] :RequestTimeoutException INFO [2022-01-10 05:48:45,083] jepsen worker 15 - jepsen.util 4335 :invoke :read [5 nil] INFO [2022-01-10 05:48:45,089] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.631578947368421}, :ns_1@172.23.107.227 {:progress 0.4486803519061584}, :ns_1@172.23.105.187 {:progress 0.4486803519061584}, :ns_1@172.23.107.228 {:progress 0.6}} INFO [2022-01-10 05:48:45,089] jepsen worker nemesis - couchbase.util Rebalance check count: 671 INFO [2022-01-10 05:48:45,089] jepsen worker nemesis - couchbase.util Rebalance stuck count: 561 INFO [2022-01-10 05:48:45,095] jepsen worker 86 - jepsen.util 3956 :info :write [28 40] :RequestTimeoutException INFO [2022-01-10 05:48:45,096] jepsen worker 86 - jepsen.util 4046 :invoke :write [28 0] INFO [2022-01-10 05:48:45,189] jepsen worker 55 - jepsen.util 5185 :info :write [18 30] :RequestTimeoutException INFO [2022-01-10 05:48:45,190] jepsen worker 55 - jepsen.util 5275 :invoke :write [18 47] INFO [2022-01-10 05:48:45,195] jepsen worker 87 - jepsen.util 4497 :info :write [29 29] :RequestTimeoutException INFO [2022-01-10 05:48:45,196] jepsen worker 87 - jepsen.util 4587 :invoke :read [29 nil] INFO [2022-01-10 05:48:45,200] jepsen worker 54 - jepsen.util 4914 :info :write [18 34] :RequestTimeoutException INFO [2022-01-10 05:48:45,200] jepsen worker 54 - jepsen.util 5004 :invoke :write [18 7] INFO [2022-01-10 05:48:45,256] jepsen worker 87 - jepsen.util 4587 :ok :read [29 0] INFO [2022-01-10 05:48:45,257] jepsen worker 87 - jepsen.util 4587 :invoke :read [29 nil] INFO [2022-01-10 05:48:45,271] jepsen worker 63 - jepsen.util 4383 :ok :read [21 7] INFO [2022-01-10 05:48:45,272] jepsen worker 63 - jepsen.util 4383 :invoke :write [21 31] INFO [2022-01-10 05:48:45,282] jepsen worker 87 - jepsen.util 4587 :ok :read [29 0] INFO [2022-01-10 05:48:45,283] jepsen worker 87 - jepsen.util 4587 :invoke :write [29 40] INFO [2022-01-10 05:48:45,325] jepsen worker 29 - jepsen.util 4619 :ok :read [9 22] INFO [2022-01-10 05:48:45,327] jepsen worker 29 - jepsen.util 4619 :invoke :read [9 nil] INFO [2022-01-10 05:48:45,366] jepsen worker 41 - jepsen.util 4631 :info :write [13 47] :RequestTimeoutException INFO [2022-01-10 05:48:45,367] jepsen worker 41 - jepsen.util 4721 :invoke :read [13 nil] INFO [2022-01-10 05:48:45,456] jepsen worker 71 - jepsen.util 4391 :ok :read [23 26] INFO [2022-01-10 05:48:45,458] jepsen worker 71 - jepsen.util 4391 :invoke :write [23 47] INFO [2022-01-10 05:48:45,490] jepsen worker 7 - jepsen.util 5047 :info :write [2 6] :RequestTimeoutException INFO [2022-01-10 05:48:45,491] jepsen worker 7 - jepsen.util 5137 :invoke :read [2 nil] INFO [2022-01-10 05:48:45,581] jepsen worker 2 - jepsen.util 4862 :info :write [0 7] :RequestTimeoutException INFO [2022-01-10 05:48:45,582] jepsen worker 2 - jepsen.util 4952 :invoke :write [0 37] INFO [2022-01-10 05:48:45,727] jepsen worker 29 - jepsen.util 4619 :ok :read [9 22] INFO [2022-01-10 05:48:45,727] jepsen worker 29 - jepsen.util 4619 :invoke :write [9 28] INFO [2022-01-10 05:48:45,734] jepsen worker 28 - jepsen.util 4348 :ok :read [9 22] INFO [2022-01-10 05:48:45,735] jepsen worker 28 - jepsen.util 4348 :invoke :write [9 23] INFO [2022-01-10 05:48:45,739] jepsen worker 59 - jepsen.util 4559 :ok :read [19 31] INFO [2022-01-10 05:48:45,739] jepsen worker 59 - jepsen.util 4559 :invoke :write [19 49] INFO [2022-01-10 05:48:45,782] jepsen worker 85 - jepsen.util 5125 :ok :read [28 45] INFO [2022-01-10 05:48:45,782] jepsen worker 85 - jepsen.util 5125 :invoke :read [28 nil] INFO [2022-01-10 05:48:45,991] jepsen worker 30 - jepsen.util 4350 :info :write [10 4] :RequestTimeoutException INFO [2022-01-10 05:48:45,992] jepsen worker 30 - jepsen.util 4440 :invoke :write [10 12] INFO [2022-01-10 05:48:46,011] jepsen worker 47 - jepsen.util 4817 :info :write [15 40] :RequestTimeoutException INFO [2022-01-10 05:48:46,012] jepsen worker 47 - jepsen.util 4907 :invoke :read [15 nil] INFO [2022-01-10 05:48:46,098] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.631578947368421}, :ns_1@172.23.107.227 {:progress 0.4486803519061584}, :ns_1@172.23.105.187 {:progress 0.4486803519061584}, :ns_1@172.23.107.228 {:progress 0.6}} INFO [2022-01-10 05:48:46,098] jepsen worker nemesis - couchbase.util Rebalance check count: 672 INFO [2022-01-10 05:48:46,098] jepsen worker nemesis - couchbase.util Rebalance stuck count: 562 INFO [2022-01-10 05:48:46,106] jepsen worker 89 - jepsen.util 5219 :info :write [29 41] :RequestTimeoutException INFO [2022-01-10 05:48:46,107] jepsen worker 89 - jepsen.util 5309 :invoke :read [29 nil] INFO [2022-01-10 05:48:46,180] jepsen worker 85 - jepsen.util 5125 :ok :read [28 45] INFO [2022-01-10 05:48:46,181] jepsen worker 85 - jepsen.util 5125 :invoke :write [28 38] INFO [2022-01-10 05:48:46,996] jepsen worker 16 - jepsen.util 4786 :info :write [5 14] :RequestTimeoutException INFO [2022-01-10 05:48:46,997] jepsen worker 16 - jepsen.util 4876 :invoke :read [5 nil] INFO [2022-01-10 05:48:47,089] jepsen worker 62 - jepsen.util 5012 :info :write [20 31] :RequestTimeoutException INFO [2022-01-10 05:48:47,090] jepsen worker 62 - jepsen.util 5102 :invoke :write [20 43] INFO [2022-01-10 05:48:47,096] jepsen worker 42 - jepsen.util 4452 :info :write [14 28] :RequestTimeoutException INFO [2022-01-10 05:48:47,096] jepsen worker 42 - jepsen.util 4542 :invoke :read [14 nil] INFO [2022-01-10 05:48:47,100] jepsen worker 77 - jepsen.util 5207 :info :write [25 47] :RequestTimeoutException INFO [2022-01-10 05:48:47,100] jepsen worker 77 - jepsen.util 5297 :invoke :write [25 2] INFO [2022-01-10 05:48:47,104] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6374269005847953}, :ns_1@172.23.107.227 {:progress 0.4516129032258065}, :ns_1@172.23.105.187 {:progress 0.4516129032258065}, :ns_1@172.23.107.228 {:progress 0.6058823529411765}} INFO [2022-01-10 05:48:47,104] jepsen worker nemesis - couchbase.util Rebalance check count: 673 INFO [2022-01-10 05:48:47,104] jepsen worker nemesis - couchbase.util Rebalance stuck count: 562 INFO [2022-01-10 05:48:47,132] jepsen worker 32 - jepsen.util 4982 :info :write [10 46] :RequestTimeoutException INFO [2022-01-10 05:48:47,132] jepsen worker 32 - jepsen.util 5072 :invoke :write [10 2] INFO [2022-01-10 05:48:47,194] jepsen worker 83 - jepsen.util 5393 :info :write [27 16] :RequestTimeoutException INFO [2022-01-10 05:48:47,195] jepsen worker 83 - jepsen.util 5483 :invoke :read [27 nil] INFO [2022-01-10 05:48:47,255] jepsen worker 74 - jepsen.util 5204 :info :write [24 21] :RequestTimeoutException INFO [2022-01-10 05:48:47,256] jepsen worker 74 - jepsen.util 5294 :invoke :read [24 nil] INFO [2022-01-10 05:48:47,280] jepsen worker 31 - jepsen.util 4891 :info :write [10 47] :RequestTimeoutException INFO [2022-01-10 05:48:47,281] jepsen worker 31 - jepsen.util 4981 :invoke :read [10 nil] INFO [2022-01-10 05:48:47,331] jepsen worker 18 - jepsen.util 4608 :ok :read [6 25] INFO [2022-01-10 05:48:47,335] jepsen worker 18 - jepsen.util 4608 :invoke :write [6 11] INFO [2022-01-10 05:48:47,344] jepsen worker 47 - jepsen.util 4907 :ok :read [15 23] INFO [2022-01-10 05:48:47,345] jepsen worker 47 - jepsen.util 4907 :invoke :read [15 nil] INFO [2022-01-10 05:48:47,346] jepsen worker 24 - jepsen.util 4344 :ok :read [8 39] INFO [2022-01-10 05:48:47,347] jepsen worker 24 - jepsen.util 4344 :invoke :write [8 32] INFO [2022-01-10 05:48:47,734] jepsen worker 25 - jepsen.util 4435 :ok :read [8 39] INFO [2022-01-10 05:48:47,736] jepsen worker 25 - jepsen.util 4435 :invoke :write [8 38] INFO [2022-01-10 05:48:47,751] jepsen worker 19 - jepsen.util 4879 :ok :read [6 25] INFO [2022-01-10 05:48:47,753] jepsen worker 19 - jepsen.util 4879 :invoke :write [6 38] INFO [2022-01-10 05:48:47,843] jepsen worker 84 - jepsen.util 4494 :ok :read [28 45] INFO [2022-01-10 05:48:47,843] jepsen worker 84 - jepsen.util 4494 :invoke :read [28 nil] INFO [2022-01-10 05:48:47,968] jepsen worker 84 - jepsen.util 4494 :ok :read [28 45] INFO [2022-01-10 05:48:47,969] jepsen worker 84 - jepsen.util 4494 :invoke :read [28 nil] INFO [2022-01-10 05:48:48,045] jepsen worker 11 - jepsen.util 4601 :info :write [3 37] :RequestTimeoutException INFO [2022-01-10 05:48:48,045] jepsen worker 11 - jepsen.util 4691 :invoke :write [3 19] INFO [2022-01-10 05:48:48,061] jepsen worker 84 - jepsen.util 4494 :ok :read [28 45] INFO [2022-01-10 05:48:48,062] jepsen worker 84 - jepsen.util 4494 :invoke :read [28 nil] INFO [2022-01-10 05:48:48,066] jepsen worker 33 - jepsen.util 4983 :info :write [11 43] :RequestTimeoutException INFO [2022-01-10 05:48:48,067] jepsen worker 33 - jepsen.util 5073 :invoke :write [11 42] INFO [2022-01-10 05:48:48,113] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6432748538011697}, :ns_1@172.23.107.227 {:progress 0.4545454545454546}, :ns_1@172.23.105.187 {:progress 0.4545454545454546}, :ns_1@172.23.107.228 {:progress 0.611764705882353}} INFO [2022-01-10 05:48:48,114] jepsen worker nemesis - couchbase.util Rebalance check count: 674 INFO [2022-01-10 05:48:48,114] jepsen worker nemesis - couchbase.util Rebalance stuck count: 562 INFO [2022-01-10 05:48:48,135] jepsen worker 84 - jepsen.util 4494 :ok :read [28 45] INFO [2022-01-10 05:48:48,136] jepsen worker 84 - jepsen.util 4494 :invoke :read [28 nil] INFO [2022-01-10 05:48:48,248] jepsen worker 84 - jepsen.util 4494 :ok :read [28 45] INFO [2022-01-10 05:48:48,248] jepsen worker 84 - jepsen.util 4494 :invoke :write [28 35] INFO [2022-01-10 05:48:48,400] jepsen worker 70 - jepsen.util 4750 :info :write [23 29] :RequestTimeoutException INFO [2022-01-10 05:48:48,400] jepsen worker 70 - jepsen.util 4840 :invoke :write [23 31] INFO [2022-01-10 05:48:48,412] jepsen worker 66 - jepsen.util 4566 :info :write [22 4] :RequestTimeoutException INFO [2022-01-10 05:48:48,413] jepsen worker 66 - jepsen.util 4656 :invoke :write [22 48] INFO [2022-01-10 05:48:48,471] jepsen worker 35 - jepsen.util 4445 :info :write [11 18] :RequestTimeoutException INFO [2022-01-10 05:48:48,472] jepsen worker 35 - jepsen.util 4535 :invoke :write [11 32] INFO [2022-01-10 05:48:48,495] jepsen worker 72 - jepsen.util 5022 :info :write [24 18] :RequestTimeoutException INFO [2022-01-10 05:48:48,495] jepsen worker 72 - jepsen.util 5112 :invoke :read [24 nil] INFO [2022-01-10 05:48:48,495] jepsen worker 75 - jepsen.util 4575 :info :write [25 24] :RequestTimeoutException INFO [2022-01-10 05:48:48,496] jepsen worker 75 - jepsen.util 4665 :invoke :read [25 nil] INFO [2022-01-10 05:48:48,536] jepsen worker 69 - jepsen.util 4299 :info :write [23 31] :RequestTimeoutException INFO [2022-01-10 05:48:48,536] jepsen worker 69 - jepsen.util 4389 :invoke :write [23 44] INFO [2022-01-10 05:48:48,711] jepsen worker 20 - jepsen.util 4430 :info :write [6 20] :RequestTimeoutException INFO [2022-01-10 05:48:48,711] jepsen worker 20 - jepsen.util 4520 :invoke :write [6 17] INFO [2022-01-10 05:48:48,740] jepsen worker 6 - jepsen.util 4596 :info :write [2 12] :RequestTimeoutException INFO [2022-01-10 05:48:48,740] jepsen worker 6 - jepsen.util 4686 :invoke :write [2 6] INFO [2022-01-10 05:48:48,842] jepsen worker 40 - jepsen.util 4360 :info :write [13 23] :RequestTimeoutException INFO [2022-01-10 05:48:48,842] jepsen worker 40 - jepsen.util 4450 :invoke :read [13 nil] INFO [2022-01-10 05:48:48,891] jepsen worker 10 - jepsen.util 4510 :info :write [3 28] :RequestTimeoutException INFO [2022-01-10 05:48:48,891] jepsen worker 9 - jepsen.util 5229 :info :write [3 3] :RequestTimeoutException INFO [2022-01-10 05:48:48,892] jepsen worker 9 - jepsen.util 5319 :invoke :write [3 9] INFO [2022-01-10 05:48:48,892] jepsen worker 10 - jepsen.util 4600 :invoke :read [3 nil] INFO [2022-01-10 05:48:49,116] jepsen worker 64 - jepsen.util 4744 :info :write [21 42] :RequestTimeoutException INFO [2022-01-10 05:48:49,117] jepsen worker 64 - jepsen.util 4834 :invoke :write [21 6] INFO [2022-01-10 05:48:49,127] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6432748538011697}, :ns_1@172.23.107.227 {:progress 0.4545454545454546}, :ns_1@172.23.105.187 {:progress 0.4545454545454546}, :ns_1@172.23.107.228 {:progress 0.611764705882353}} INFO [2022-01-10 05:48:49,127] jepsen worker nemesis - couchbase.util Rebalance check count: 675 INFO [2022-01-10 05:48:49,127] jepsen worker nemesis - couchbase.util Rebalance stuck count: 563 INFO [2022-01-10 05:48:49,252] jepsen worker 74 - jepsen.util 5294 :ok :read [24 18] INFO [2022-01-10 05:48:49,252] jepsen worker 74 - jepsen.util 5294 :invoke :read [24 nil] INFO [2022-01-10 05:48:49,289] jepsen worker 47 - jepsen.util 4907 :ok :read [15 23] INFO [2022-01-10 05:48:49,289] jepsen worker 47 - jepsen.util 4907 :invoke :write [15 24] INFO [2022-01-10 05:48:49,411] jepsen worker 45 - jepsen.util 5175 :ok :read [15 23] INFO [2022-01-10 05:48:49,412] jepsen worker 45 - jepsen.util 5175 :invoke :write [15 46] INFO [2022-01-10 05:48:49,418] jepsen worker 7 - jepsen.util 5137 :ok :read [2 14] INFO [2022-01-10 05:48:49,419] jepsen worker 7 - jepsen.util 5137 :invoke :read [2 nil] INFO [2022-01-10 05:48:49,641] jepsen worker 26 - jepsen.util 4526 :info :write [8 23] :RequestTimeoutException INFO [2022-01-10 05:48:49,642] jepsen worker 26 - jepsen.util 4616 :invoke :write [8 35] INFO [2022-01-10 05:48:49,674] jepsen worker 23 - jepsen.util 4793 :info :write [7 36] :RequestTimeoutException INFO [2022-01-10 05:48:49,675] jepsen worker 23 - jepsen.util 4883 :invoke :write [7 33] INFO [2022-01-10 05:48:49,695] jepsen worker 48 - jepsen.util 4548 :info :write [16 3] :RequestTimeoutException INFO [2022-01-10 05:48:49,696] jepsen worker 48 - jepsen.util 4638 :invoke :write [16 25] INFO [2022-01-10 05:48:49,826] jepsen worker 13 - jepsen.util 5503 :info :write [4 42] :RequestTimeoutException INFO [2022-01-10 05:48:49,827] jepsen worker 13 - jepsen.util 5593 :invoke :write [4 45] INFO [2022-01-10 05:48:50,040] jepsen worker 80 - jepsen.util 5840 :info :write [26 40] :RequestTimeoutException INFO [2022-01-10 05:48:50,041] jepsen worker 80 - jepsen.util 5930 :invoke :read [26 nil] INFO [2022-01-10 05:48:50,063] jepsen worker 72 - jepsen.util 5112 :ok :read [24 18] INFO [2022-01-10 05:48:50,064] jepsen worker 72 - jepsen.util 5112 :invoke :read [24 nil] INFO [2022-01-10 05:48:50,132] jepsen worker 49 - jepsen.util 4549 :info :write [16 24] :RequestTimeoutException INFO [2022-01-10 05:48:50,132] jepsen worker 49 - jepsen.util 4639 :invoke :read [16 nil] INFO [2022-01-10 05:48:50,134] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6491228070175439}, :ns_1@172.23.107.227 {:progress 0.4574780058651027}, :ns_1@172.23.105.187 {:progress 0.4574780058651027}, :ns_1@172.23.107.228 {:progress 0.6176470588235294}} INFO [2022-01-10 05:48:50,134] jepsen worker nemesis - couchbase.util Rebalance check count: 676 INFO [2022-01-10 05:48:50,134] jepsen worker nemesis - couchbase.util Rebalance stuck count: 563 INFO [2022-01-10 05:48:50,187] jepsen worker 80 - jepsen.util 5930 :ok :read [26 10] INFO [2022-01-10 05:48:50,187] jepsen worker 80 - jepsen.util 5930 :invoke :read [26 nil] INFO [2022-01-10 05:48:50,220] jepsen worker 80 - jepsen.util 5930 :ok :read [26 10] INFO [2022-01-10 05:48:50,220] jepsen worker 80 - jepsen.util 5930 :invoke :read [26 nil] INFO [2022-01-10 05:48:50,315] jepsen worker 7 - jepsen.util 5137 :ok :read [2 14] INFO [2022-01-10 05:48:50,315] jepsen worker 7 - jepsen.util 5137 :invoke :write [2 16] INFO [2022-01-10 05:48:50,446] jepsen worker 38 - jepsen.util 4358 :info :write [12 4] :RequestTimeoutException INFO [2022-01-10 05:48:50,447] jepsen worker 38 - jepsen.util 4448 :invoke :write [12 20] INFO [2022-01-10 05:48:50,498] jepsen worker 80 - jepsen.util 5930 :ok :read [26 10] INFO [2022-01-10 05:48:50,499] jepsen worker 80 - jepsen.util 5930 :invoke :read [26 nil] INFO [2022-01-10 05:48:50,522] jepsen worker 80 - jepsen.util 5930 :ok :read [26 10] INFO [2022-01-10 05:48:50,523] jepsen worker 80 - jepsen.util 5930 :invoke :read [26 nil] INFO [2022-01-10 05:48:50,546] jepsen worker 80 - jepsen.util 5930 :ok :read [26 10] INFO [2022-01-10 05:48:50,547] jepsen worker 80 - jepsen.util 5930 :invoke :read [26 nil] INFO [2022-01-10 05:48:50,562] jepsen worker 21 - jepsen.util 4791 :ok :read [7 :nil] INFO [2022-01-10 05:48:50,563] jepsen worker 21 - jepsen.util 4791 :invoke :read [7 nil] INFO [2022-01-10 05:48:50,573] jepsen worker 80 - jepsen.util 5930 :ok :read [26 10] INFO [2022-01-10 05:48:50,573] jepsen worker 80 - jepsen.util 5930 :invoke :write [26 30] INFO [2022-01-10 05:48:50,585] jepsen worker 36 - jepsen.util 3636 :info :write [12 3] :RequestTimeoutException INFO [2022-01-10 05:48:50,586] jepsen worker 36 - jepsen.util 3726 :invoke :read [12 nil] INFO [2022-01-10 05:48:50,615] jepsen worker 14 - jepsen.util 5684 :info :write [4 44] :RequestTimeoutException INFO [2022-01-10 05:48:50,615] jepsen worker 14 - jepsen.util 5774 :invoke :write [4 48] INFO [2022-01-10 05:48:50,631] jepsen worker 72 - jepsen.util 5112 :ok :read [24 18] INFO [2022-01-10 05:48:50,632] jepsen worker 72 - jepsen.util 5112 :invoke :write [24 29] WARN [2022-01-10 05:48:50,695] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11148484,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14003339,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9582239,"encode_duration_us":16925,"operation_id":"0x908b","timeout_ms":10000,"last_server_duration_us":9190751,"total_duration_us":10004256},{"operation_name":"upsert","last_dispatch_duration_us":9123335,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18474283,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9135743,"encode_duration_us":17649,"operation_id":"0x9056","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10007247}],"total_count":2}} INFO [2022-01-10 05:48:50,731] jepsen worker 3 - jepsen.util 4683 :info :write [1 1] :RequestTimeoutException INFO [2022-01-10 05:48:50,732] jepsen worker 3 - jepsen.util 4773 :invoke :read [1 nil] WARN [2022-01-10 05:48:50,796] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10318040,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10318040,"total_server_duration_us":91532,"operation_id":"0x90e5","timeout_ms":30000,"last_server_duration_us":91532,"total_duration_us":12932798},{"operation_name":"get","last_dispatch_duration_us":10044588,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":10044588,"total_server_duration_us":147353,"operation_id":"0x911d","timeout_ms":30000,"last_server_duration_us":147353,"total_duration_us":11133830},{"operation_name":"upsert","last_dispatch_duration_us":50997,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":632883,"total_server_duration_us":245066,"encode_duration_us":24429,"operation_id":"0x9210","timeout_ms":10000,"last_server_duration_us":29975,"total_duration_us":10009809},{"operation_name":"upsert","last_dispatch_duration_us":1988971,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":2927692,"total_server_duration_us":287109,"encode_duration_us":22119,"operation_id":"0x916b","timeout_ms":10000,"last_server_duration_us":103592,"total_duration_us":10009749},{"operation_name":"upsert","last_dispatch_duration_us":1087294,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8283535,"total_server_duration_us":2435087,"encode_duration_us":16399,"operation_id":"0x91dd","timeout_ms":10000,"last_server_duration_us":540267,"total_duration_us":10007652},{"operation_name":"upsert","last_dispatch_duration_us":203865,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":6455532,"total_server_duration_us":1169670,"encode_duration_us":18713,"operation_id":"0x9304","timeout_ms":10000,"last_server_duration_us":39952,"total_duration_us":10006990},{"operation_name":"upsert","last_dispatch_duration_us":722565,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":6163843,"total_server_duration_us":858959,"encode_duration_us":26574,"operation_id":"0x92d6","timeout_ms":10000,"last_server_duration_us":47306,"total_duration_us":10004115},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":17686,"operation_id":"0x9298","timeout_ms":10000,"total_duration_us":10004084},{"operation_name":"upsert","last_dispatch_duration_us":1266593,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":9703893,"total_server_duration_us":410240,"encode_duration_us":18919,"operation_id":"0x91b4","timeout_ms":10000,"last_server_duration_us":44722,"total_duration_us":10003947},{"operation_name":"upsert","last_dispatch_duration_us":1114615,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8972280,"total_server_duration_us":2258410,"encode_duration_us":21179,"operation_id":"0x91c8","timeout_ms":10000,"last_server_duration_us":293784,"total_duration_us":10003903}],"total_count":16}} INFO [2022-01-10 05:48:50,830] jepsen worker 74 - jepsen.util 5294 :ok :read [24 18] INFO [2022-01-10 05:48:50,831] jepsen worker 74 - jepsen.util 5294 :invoke :write [24 29] INFO [2022-01-10 05:48:50,884] jepsen worker 17 - jepsen.util 4607 :info :write [5 43] :SyncWriteAmbiguous INFO [2022-01-10 05:48:50,885] jepsen worker 17 - jepsen.util 4697 :invoke :read [5 nil] INFO [2022-01-10 05:48:50,939] jepsen worker 8 - jepsen.util 4868 :info :write [2 21] :RequestTimeoutException INFO [2022-01-10 05:48:50,940] jepsen worker 8 - jepsen.util 4958 :invoke :read [2 nil] INFO [2022-01-10 05:48:50,961] jepsen worker 78 - jepsen.util 5838 :info :write [26 46] :RequestTimeoutException INFO [2022-01-10 05:48:50,963] jepsen worker 78 - jepsen.util 5928 :invoke :write [26 36] INFO [2022-01-10 05:48:50,976] jepsen worker 68 - jepsen.util 4928 :ok :read [22 41] INFO [2022-01-10 05:48:50,976] jepsen worker 68 - jepsen.util 4928 :invoke :write [22 1] INFO [2022-01-10 05:48:50,990] jepsen worker 67 - jepsen.util 4387 :ok :read [22 41] INFO [2022-01-10 05:48:50,991] jepsen worker 67 - jepsen.util 4387 :invoke :write [22 0] INFO [2022-01-10 05:48:50,993] jepsen worker 39 - jepsen.util 5079 :ok :read [13 15] INFO [2022-01-10 05:48:50,994] jepsen worker 39 - jepsen.util 5079 :invoke :write [13 39] INFO [2022-01-10 05:48:51,025] jepsen worker 83 - jepsen.util 5483 :ok :read [27 30] INFO [2022-01-10 05:48:51,026] jepsen worker 83 - jepsen.util 5483 :invoke :read [27 nil] INFO [2022-01-10 05:48:51,141] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6491228070175439}, :ns_1@172.23.107.227 {:progress 0.4574780058651027}, :ns_1@172.23.105.187 {:progress 0.4574780058651027}, :ns_1@172.23.107.228 {:progress 0.6176470588235294}} INFO [2022-01-10 05:48:51,141] jepsen worker nemesis - couchbase.util Rebalance check count: 677 INFO [2022-01-10 05:48:51,141] jepsen worker nemesis - couchbase.util Rebalance stuck count: 564 INFO [2022-01-10 05:48:51,160] jepsen worker 3 - jepsen.util 4773 :ok :read [1 49] INFO [2022-01-10 05:48:51,161] jepsen worker 3 - jepsen.util 4773 :invoke :write [1 5] INFO [2022-01-10 05:48:51,197] jepsen worker 83 - jepsen.util 5483 :ok :read [27 30] INFO [2022-01-10 05:48:51,198] jepsen worker 83 - jepsen.util 5483 :invoke :read [27 nil] INFO [2022-01-10 05:48:51,321] jepsen worker 46 - jepsen.util 4816 :info :write [15 28] :RequestTimeoutException INFO [2022-01-10 05:48:51,323] jepsen worker 46 - jepsen.util 4906 :invoke :write [15 9] INFO [2022-01-10 05:48:51,324] jepsen worker 83 - jepsen.util 5483 :ok :read [27 30] INFO [2022-01-10 05:48:51,325] jepsen worker 83 - jepsen.util 5483 :invoke :write [27 20] INFO [2022-01-10 05:48:51,686] jepsen worker 8 - jepsen.util 4958 :ok :read [2 14] INFO [2022-01-10 05:48:51,686] jepsen worker 8 - jepsen.util 4958 :invoke :write [2 19] INFO [2022-01-10 05:48:51,814] jepsen worker 41 - jepsen.util 4721 :ok :read [13 15] INFO [2022-01-10 05:48:51,815] jepsen worker 41 - jepsen.util 4721 :invoke :write [13 36] INFO [2022-01-10 05:48:51,824] jepsen worker 57 - jepsen.util 5547 :ok :read [19 31] INFO [2022-01-10 05:48:51,825] jepsen worker 57 - jepsen.util 5547 :invoke :read [19 nil] INFO [2022-01-10 05:48:51,863] jepsen worker 16 - jepsen.util 4876 :ok :read [5 25] INFO [2022-01-10 05:48:51,863] jepsen worker 16 - jepsen.util 4876 :invoke :read [5 nil] INFO [2022-01-10 05:48:51,871] jepsen worker 40 - jepsen.util 4450 :ok :read [13 15] INFO [2022-01-10 05:48:51,872] jepsen worker 40 - jepsen.util 4450 :invoke :read [13 nil] INFO [2022-01-10 05:48:51,910] jepsen worker 57 - jepsen.util 5547 :ok :read [19 31] INFO [2022-01-10 05:48:51,910] jepsen worker 57 - jepsen.util 5547 :invoke :read [19 nil] INFO [2022-01-10 05:48:51,959] jepsen worker 16 - jepsen.util 4876 :ok :read [5 25] INFO [2022-01-10 05:48:51,959] jepsen worker 16 - jepsen.util 4876 :invoke :read [5 nil] INFO [2022-01-10 05:48:52,012] jepsen worker 40 - jepsen.util 4450 :ok :read [13 15] INFO [2022-01-10 05:48:52,013] jepsen worker 40 - jepsen.util 4450 :invoke :read [13 nil] INFO [2022-01-10 05:48:52,029] jepsen worker 57 - jepsen.util 5547 :ok :read [19 31] INFO [2022-01-10 05:48:52,030] jepsen worker 57 - jepsen.util 5547 :invoke :write [19 12] INFO [2022-01-10 05:48:52,072] jepsen worker 82 - jepsen.util 4852 :info :write [27 0] :RequestTimeoutException INFO [2022-01-10 05:48:52,073] jepsen worker 82 - jepsen.util 4942 :invoke :read [27 nil] INFO [2022-01-10 05:48:52,080] jepsen worker 16 - jepsen.util 4876 :ok :read [5 25] INFO [2022-01-10 05:48:52,081] jepsen worker 16 - jepsen.util 4876 :invoke :read [5 nil] INFO [2022-01-10 05:48:52,123] jepsen worker 27 - jepsen.util 5067 :info :write [9 4] :SyncWriteAmbiguous INFO [2022-01-10 05:48:52,124] jepsen worker 27 - jepsen.util 5157 :invoke :write [9 11] INFO [2022-01-10 05:48:52,134] jepsen worker 44 - jepsen.util 4814 :info :write [14 8] :RequestTimeoutException INFO [2022-01-10 05:48:52,135] jepsen worker 44 - jepsen.util 4904 :invoke :read [14 nil] INFO [2022-01-10 05:48:52,141] jepsen worker 73 - jepsen.util 4573 :info :write [24 15] :RequestTimeoutException INFO [2022-01-10 05:48:52,141] jepsen worker 73 - jepsen.util 4663 :invoke :write [24 25] INFO [2022-01-10 05:48:52,148] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6491228070175439}, :ns_1@172.23.107.227 {:progress 0.4574780058651027}, :ns_1@172.23.105.187 {:progress 0.4574780058651027}, :ns_1@172.23.107.228 {:progress 0.6176470588235294}} INFO [2022-01-10 05:48:52,148] jepsen worker nemesis - couchbase.util Rebalance check count: 678 INFO [2022-01-10 05:48:52,148] jepsen worker nemesis - couchbase.util Rebalance stuck count: 565 INFO [2022-01-10 05:48:52,150] jepsen worker 40 - jepsen.util 4450 :ok :read [13 15] INFO [2022-01-10 05:48:52,150] jepsen worker 40 - jepsen.util 4450 :invoke :read [13 nil] WARN [2022-01-10 05:48:52,205] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1329880,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":3723208,"total_server_duration_us":770085,"encode_duration_us":22956,"operation_id":"0x923c","timeout_ms":10000,"last_server_duration_us":75616,"total_duration_us":10009688},{"operation_name":"upsert","last_dispatch_duration_us":9147288,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":14815555,"total_server_duration_us":174244,"encode_duration_us":19980,"operation_id":"0x9199","timeout_ms":10000,"last_server_duration_us":20601,"total_duration_us":10008321},{"operation_name":"upsert","last_dispatch_duration_us":9118055,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":15247395,"total_server_duration_us":982986,"encode_duration_us":16578,"operation_id":"0x9175","timeout_ms":10000,"last_server_duration_us":27854,"total_duration_us":10006734},{"operation_name":"upsert","last_dispatch_duration_us":1990964,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9062971,"total_server_duration_us":507312,"encode_duration_us":18761,"operation_id":"0x92a4","timeout_ms":10000,"last_server_duration_us":122688,"total_duration_us":10006245},{"operation_name":"upsert","last_dispatch_duration_us":62741,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":361217,"total_server_duration_us":192965,"encode_duration_us":17922,"operation_id":"0x91a6","timeout_ms":10000,"last_server_duration_us":49951,"total_duration_us":10005972},{"operation_name":"upsert","last_dispatch_duration_us":33273,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":1134563,"total_server_duration_us":384476,"encode_duration_us":21804,"operation_id":"0x92cb","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10004030},{"operation_name":"upsert","last_dispatch_duration_us":1480002,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9918474,"total_server_duration_us":744367,"encode_duration_us":17376,"operation_id":"0x92b0","timeout_ms":10000,"last_server_duration_us":140239,"total_duration_us":10003739},{"operation_name":"upsert","last_dispatch_duration_us":49681,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":1377700,"total_server_duration_us":405514,"encode_duration_us":16861,"operation_id":"0x92cc","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10003700},{"operation_name":"upsert","last_dispatch_duration_us":2335662,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":8840606,"total_server_duration_us":250969,"encode_duration_us":21898,"operation_id":"0x92d5","timeout_ms":10000,"last_server_duration_us":22223,"total_duration_us":10003068},{"operation_name":"upsert","last_dispatch_duration_us":373260,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":4057915,"total_server_duration_us":1398351,"encode_duration_us":16209,"operation_id":"0x9312","timeout_ms":10000,"last_server_duration_us":118457,"total_duration_us":10002352}],"total_count":15}} INFO [2022-01-10 05:48:52,261] jepsen worker 15 - jepsen.util 4335 :ok :read [5 25] INFO [2022-01-10 05:48:52,262] jepsen worker 15 - jepsen.util 4335 :invoke :read [5 nil] WARN [2022-01-10 05:48:52,297] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10410907,"last_local_id":"EFE9906200000002/00000000598DAEAE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":17967993,"last_local_socket":"172.23.104.79:44342","total_server_duration_us":10165356,"encode_duration_us":16168,"operation_id":"0x9031","timeout_ms":10000,"last_server_duration_us":9059667,"total_duration_us":10002312},{"operation_name":"upsert","last_dispatch_duration_us":9043440,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16146161,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9743404,"encode_duration_us":16731,"operation_id":"0x91e0","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10007143}],"total_count":2}} WARN [2022-01-10 05:48:52,306] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11966886,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17395494,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9510730,"encode_duration_us":16392,"operation_id":"0x9113","timeout_ms":10000,"last_server_duration_us":9051168,"total_duration_us":10007014}],"total_count":1}} INFO [2022-01-10 05:48:52,348] jepsen worker 82 - jepsen.util 4942 :ok :read [27 30] INFO [2022-01-10 05:48:52,349] jepsen worker 82 - jepsen.util 4942 :invoke :write [27 3] INFO [2022-01-10 05:48:52,358] jepsen worker 16 - jepsen.util 4876 :ok :read [5 25] INFO [2022-01-10 05:48:52,359] jepsen worker 16 - jepsen.util 4876 :invoke :write [5 3] INFO [2022-01-10 05:48:52,388] jepsen worker 36 - jepsen.util 3726 :ok :read [12 14] INFO [2022-01-10 05:48:52,388] jepsen worker 36 - jepsen.util 3726 :invoke :read [12 nil] WARN [2022-01-10 05:48:52,397] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":5549835,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":9065431,"total_server_duration_us":299620,"encode_duration_us":19155,"operation_id":"0x91a5","timeout_ms":10000,"last_server_duration_us":44944,"total_duration_us":10010853},{"operation_name":"upsert","last_dispatch_duration_us":36421,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":504278,"total_server_duration_us":279050,"encode_duration_us":18643,"operation_id":"0x926c","timeout_ms":10000,"last_server_duration_us":21324,"total_duration_us":10010743},{"operation_name":"upsert","last_dispatch_duration_us":12205849,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":12205849,"total_server_duration_us":9498637,"encode_duration_us":16399,"operation_id":"0x9177","timeout_ms":10000,"last_server_duration_us":9498637,"total_duration_us":10008364},{"operation_name":"upsert","last_dispatch_duration_us":264531,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":1017424,"total_server_duration_us":108624,"encode_duration_us":21949,"operation_id":"0x9354","timeout_ms":10000,"last_server_duration_us":21243,"total_duration_us":10008286},{"operation_name":"upsert","last_dispatch_duration_us":214167,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10091816,"total_server_duration_us":416676,"encode_duration_us":18221,"operation_id":"0x9206","timeout_ms":10000,"last_server_duration_us":38170,"total_duration_us":10007681},{"operation_name":"upsert","last_dispatch_duration_us":9038738,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9584639,"total_server_duration_us":9354401,"encode_duration_us":16763,"operation_id":"0x923b","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10007601},{"operation_name":"upsert","last_dispatch_duration_us":427668,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9973782,"total_server_duration_us":782467,"encode_duration_us":17716,"operation_id":"0x921a","timeout_ms":10000,"last_server_duration_us":37756,"total_duration_us":10007485},{"operation_name":"upsert","last_dispatch_duration_us":291908,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":8364520,"total_server_duration_us":709426,"encode_duration_us":21250,"operation_id":"0x91cf","timeout_ms":10000,"last_server_duration_us":279838,"total_duration_us":10006871},{"operation_name":"upsert","last_dispatch_duration_us":504266,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9638604,"total_server_duration_us":649656,"encode_duration_us":19670,"operation_id":"0x92a1","timeout_ms":10000,"last_server_duration_us":347096,"total_duration_us":10006870},{"operation_name":"upsert","last_dispatch_duration_us":8041256,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10596338,"total_server_duration_us":103307,"encode_duration_us":16814,"operation_id":"0x91d9","timeout_ms":10000,"last_server_duration_us":26596,"total_duration_us":10006418}],"total_count":16}} WARN [2022-01-10 05:48:52,398] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":571950,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":571950,"total_server_duration_us":43,"operation_id":"0x91db","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":11351598},{"operation_name":"upsert","last_dispatch_duration_us":577034,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":4306379,"total_server_duration_us":856233,"encode_duration_us":17003,"operation_id":"0x92a7","timeout_ms":10000,"last_server_duration_us":37963,"total_duration_us":10009037},{"operation_name":"upsert","last_dispatch_duration_us":10377622,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":13504021,"total_server_duration_us":9734537,"encode_duration_us":19270,"operation_id":"0x91d6","timeout_ms":10000,"last_server_duration_us":9381865,"total_duration_us":10008895},{"operation_name":"upsert","last_dispatch_duration_us":11225391,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11225391,"total_server_duration_us":59876,"encode_duration_us":20033,"operation_id":"0x91ce","timeout_ms":10000,"last_server_duration_us":59876,"total_duration_us":10008008},{"operation_name":"upsert","last_dispatch_duration_us":11136099,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11136099,"total_server_duration_us":89286,"encode_duration_us":16747,"operation_id":"0x91cd","timeout_ms":10000,"last_server_duration_us":89286,"total_duration_us":10007557},{"operation_name":"upsert","last_dispatch_duration_us":379085,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":3525885,"total_server_duration_us":844878,"encode_duration_us":17286,"operation_id":"0x9184","timeout_ms":10000,"last_server_duration_us":29323,"total_duration_us":10006298},{"operation_name":"upsert","last_dispatch_duration_us":809942,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":3217151,"total_server_duration_us":735346,"encode_duration_us":17210,"operation_id":"0x91d8","timeout_ms":10000,"last_server_duration_us":133099,"total_duration_us":10004771},{"operation_name":"upsert","last_dispatch_duration_us":10586844,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10586844,"total_server_duration_us":39214,"encode_duration_us":17730,"operation_id":"0x9202","timeout_ms":10000,"last_server_duration_us":39214,"total_duration_us":10004636},{"operation_name":"upsert","last_dispatch_duration_us":39514,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":1115761,"total_server_duration_us":604605,"encode_duration_us":17293,"operation_id":"0x923d","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10002874},{"operation_name":"upsert","last_dispatch_duration_us":698182,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":5825505,"total_server_duration_us":972937,"encode_duration_us":17803,"operation_id":"0x9301","timeout_ms":10000,"last_server_duration_us":347096,"total_duration_us":10002576}],"total_count":15}} WARN [2022-01-10 05:48:52,398] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":1506828,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1506828,"total_server_duration_us":4118,"operation_id":"0x91e7","timeout_ms":30000,"last_server_duration_us":4118,"total_duration_us":15855032},{"operation_name":"get","last_dispatch_duration_us":3895974,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3895974,"total_server_duration_us":13157,"operation_id":"0x919e","timeout_ms":30000,"last_server_duration_us":13157,"total_duration_us":12126232},{"operation_name":"get","last_dispatch_duration_us":1215688,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":1215688,"total_server_duration_us":35916,"operation_id":"0x9216","timeout_ms":30000,"last_server_duration_us":35916,"total_duration_us":11993060},{"operation_name":"upsert","last_dispatch_duration_us":3280516,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":3280516,"total_server_duration_us":20362,"encode_duration_us":17800,"operation_id":"0x9299","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10010904},{"operation_name":"upsert","last_dispatch_duration_us":12635121,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12635121,"total_server_duration_us":125257,"encode_duration_us":16191,"operation_id":"0x9173","timeout_ms":10000,"last_server_duration_us":125257,"total_duration_us":10010504},{"operation_name":"upsert","last_dispatch_duration_us":8655227,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":15514146,"total_server_duration_us":1558046,"encode_duration_us":17927,"operation_id":"0x9207","timeout_ms":10000,"last_server_duration_us":45838,"total_duration_us":10009574},{"operation_name":"upsert","last_dispatch_duration_us":8758623,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":13925179,"total_server_duration_us":471851,"encode_duration_us":18107,"operation_id":"0x9266","timeout_ms":10000,"last_server_duration_us":85888,"total_duration_us":10009529},{"operation_name":"upsert","last_dispatch_duration_us":1271946,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5913713,"total_server_duration_us":84570,"encode_duration_us":16286,"operation_id":"0x9275","timeout_ms":10000,"last_server_duration_us":46400,"total_duration_us":10008584},{"operation_name":"upsert","last_dispatch_duration_us":2085502,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5783166,"total_server_duration_us":937187,"encode_duration_us":17222,"operation_id":"0x92a5","timeout_ms":10000,"last_server_duration_us":781252,"total_duration_us":10007344},{"operation_name":"upsert","last_dispatch_duration_us":9043440,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":16146161,"total_server_duration_us":9743404,"encode_duration_us":16731,"operation_id":"0x91e0","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10007143}],"total_count":23}} WARN [2022-01-10 05:48:52,404] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18033073,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":26190436,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9050886,"encode_duration_us":16645,"operation_id":"0x8edf","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10006511},{"operation_name":"upsert","last_dispatch_duration_us":9917102,"last_local_id":"EFE9906200000003/00000000E0DA3632","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":14795076,"last_local_socket":"172.23.104.79:44296","total_server_duration_us":9258284,"encode_duration_us":16653,"operation_id":"0x90bf","timeout_ms":10000,"last_server_duration_us":9103272,"total_duration_us":10009834},{"operation_name":"upsert","last_dispatch_duration_us":11206604,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16747302,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9951936,"encode_duration_us":22058,"operation_id":"0x9141","timeout_ms":10000,"last_server_duration_us":9372162,"total_duration_us":10006972}],"total_count":3}} WARN [2022-01-10 05:48:52,498] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9118843,"last_local_id":"EFE9906200000004/000000008B42EFB5","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":13101633,"last_local_socket":"172.23.104.79:44292","total_server_duration_us":10349134,"encode_duration_us":16439,"operation_id":"0x9112","timeout_ms":10000,"last_server_duration_us":9101143,"total_duration_us":10001870},{"operation_name":"upsert","last_dispatch_duration_us":14208329,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18007825,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":10032326,"encode_duration_us":21107,"operation_id":"0x91d7","timeout_ms":10000,"last_server_duration_us":9184338,"total_duration_us":10003580},{"operation_name":"upsert","last_dispatch_duration_us":9038738,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9584639,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9354401,"encode_duration_us":16763,"operation_id":"0x923b","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10007601},{"operation_name":"upsert","last_dispatch_duration_us":12205849,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12205849,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9498637,"encode_duration_us":16399,"operation_id":"0x9177","timeout_ms":10000,"last_server_duration_us":9498637,"total_duration_us":10008364}],"total_count":4}} INFO [2022-01-10 05:48:52,501] jepsen worker 81 - jepsen.util 5301 :info :write [27 39] :RequestTimeoutException INFO [2022-01-10 05:48:52,502] jepsen worker 81 - jepsen.util 5391 :invoke :write [27 49] INFO [2022-01-10 05:48:52,526] jepsen worker 58 - jepsen.util 4738 :info :write [19 42] :RequestTimeoutException INFO [2022-01-10 05:48:52,527] jepsen worker 58 - jepsen.util 4828 :invoke :write [19 13] INFO [2022-01-10 05:48:52,581] jepsen worker 1 - jepsen.util 4771 :info :write [0 6] :RequestTimeoutException INFO [2022-01-10 05:48:52,581] jepsen worker 1 - jepsen.util 4861 :invoke :write [0 10] WARN [2022-01-10 05:48:52,598] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10377622,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13504021,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9734537,"encode_duration_us":19270,"operation_id":"0x91d6","timeout_ms":10000,"last_server_duration_us":9381865,"total_duration_us":10008895},{"operation_name":"upsert","last_dispatch_duration_us":12679739,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12679739,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9898921,"encode_duration_us":16828,"operation_id":"0x916a","timeout_ms":10000,"last_server_duration_us":9898921,"total_duration_us":10010903}],"total_count":2}} WARN [2022-01-10 05:48:52,604] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10747667,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10747667,"total_server_duration_us":27,"operation_id":"0x9265","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":10750603},{"operation_name":"upsert","last_dispatch_duration_us":909457,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10462958,"total_server_duration_us":76763,"encode_duration_us":18624,"operation_id":"0x91b3","timeout_ms":10000,"last_server_duration_us":56401,"total_duration_us":10010746},{"operation_name":"upsert","last_dispatch_duration_us":403061,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9995629,"total_server_duration_us":626619,"encode_duration_us":16432,"operation_id":"0x92f8","timeout_ms":10000,"last_server_duration_us":80968,"total_duration_us":10010331},{"operation_name":"upsert","last_dispatch_duration_us":10300890,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10300890,"total_server_duration_us":39004,"encode_duration_us":22854,"operation_id":"0x9269","timeout_ms":10000,"last_server_duration_us":39004,"total_duration_us":10009603},{"operation_name":"upsert","last_dispatch_duration_us":1354647,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":1354647,"total_server_duration_us":217751,"encode_duration_us":27368,"operation_id":"0x92b7","timeout_ms":10000,"last_server_duration_us":217751,"total_duration_us":10008948},{"operation_name":"upsert","last_dispatch_duration_us":10827217,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":13380426,"total_server_duration_us":1364873,"encode_duration_us":17923,"operation_id":"0x9219","timeout_ms":10000,"last_server_duration_us":781627,"total_duration_us":10008705},{"operation_name":"upsert","last_dispatch_duration_us":617781,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":2271302,"total_server_duration_us":494168,"encode_duration_us":18439,"operation_id":"0x9329","timeout_ms":10000,"last_server_duration_us":102959,"total_duration_us":10008348},{"operation_name":"upsert","last_dispatch_duration_us":33911,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":8297258,"total_server_duration_us":659157,"encode_duration_us":17144,"operation_id":"0x9176","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10007115},{"operation_name":"upsert","last_dispatch_duration_us":9470195,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10489889,"total_server_duration_us":201622,"encode_duration_us":16434,"operation_id":"0x9198","timeout_ms":10000,"last_server_duration_us":20124,"total_duration_us":10005940},{"operation_name":"upsert","last_dispatch_duration_us":49240,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":7852927,"total_server_duration_us":230826,"encode_duration_us":17722,"operation_id":"0x9276","timeout_ms":10000,"last_server_duration_us":26953,"total_duration_us":10005672}],"total_count":24}} INFO [2022-01-10 05:48:52,675] jepsen worker 43 - jepsen.util 4723 :ok :read [14 21] INFO [2022-01-10 05:48:52,676] jepsen worker 43 - jepsen.util 4723 :invoke :read [14 nil] INFO [2022-01-10 05:48:52,714] jepsen worker 52 - jepsen.util 5902 :info :write [17 13] :RequestTimeoutException INFO [2022-01-10 05:48:52,715] jepsen worker 52 - jepsen.util 5992 :invoke :write [17 27] INFO [2022-01-10 05:48:52,805] jepsen worker 5 - jepsen.util 4775 :info :write [1 26] :RequestTimeoutException INFO [2022-01-10 05:48:52,805] jepsen worker 5 - jepsen.util 4865 :invoke :write [1 44] INFO [2022-01-10 05:48:52,849] jepsen worker 21 - jepsen.util 4791 :ok :read [7 :nil] INFO [2022-01-10 05:48:52,850] jepsen worker 21 - jepsen.util 4791 :invoke :read [7 nil] INFO [2022-01-10 05:48:53,154] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6549707602339181}, :ns_1@172.23.107.227 {:progress 0.4604105571847508}, :ns_1@172.23.105.187 {:progress 0.4604105571847508}, :ns_1@172.23.107.228 {:progress 0.6235294117647059}} INFO [2022-01-10 05:48:53,154] jepsen worker nemesis - couchbase.util Rebalance check count: 679 INFO [2022-01-10 05:48:53,154] jepsen worker nemesis - couchbase.util Rebalance stuck count: 565 INFO [2022-01-10 05:48:53,450] jepsen worker 51 - jepsen.util 5541 :info :write [17 41] :RequestTimeoutException INFO [2022-01-10 05:48:53,450] jepsen worker 51 - jepsen.util 5631 :invoke :read [17 nil] INFO [2022-01-10 05:48:53,536] jepsen worker 4 - jepsen.util 4954 :info :write [1 4] :RequestTimeoutException INFO [2022-01-10 05:48:53,536] jepsen worker 4 - jepsen.util 5044 :invoke :read [1 nil] INFO [2022-01-10 05:48:53,539] jepsen worker 22 - jepsen.util 4522 :ok :read [7 :nil] INFO [2022-01-10 05:48:53,539] jepsen worker 22 - jepsen.util 4522 :invoke :read [7 nil] INFO [2022-01-10 05:48:53,621] jepsen worker 15 - jepsen.util 4335 :ok :read [5 25] INFO [2022-01-10 05:48:53,625] jepsen worker 15 - jepsen.util 4335 :invoke :write [5 3] INFO [2022-01-10 05:48:53,628] jepsen worker 40 - jepsen.util 4450 :ok :read [13 15] INFO [2022-01-10 05:48:53,629] jepsen worker 40 - jepsen.util 4450 :invoke :write [13 29] INFO [2022-01-10 05:48:53,641] jepsen worker 31 - jepsen.util 4981 :ok :read [10 12] INFO [2022-01-10 05:48:53,641] jepsen worker 31 - jepsen.util 4981 :invoke :read [10 nil] INFO [2022-01-10 05:48:53,656] jepsen worker 12 - jepsen.util 5322 :info :write [4 3] :RequestTimeoutException INFO [2022-01-10 05:48:53,657] jepsen worker 12 - jepsen.util 5412 :invoke :write [4 25] INFO [2022-01-10 05:48:53,696] jepsen worker 31 - jepsen.util 4981 :ok :read [10 12] INFO [2022-01-10 05:48:53,697] jepsen worker 31 - jepsen.util 4981 :invoke :read [10 nil] INFO [2022-01-10 05:48:53,700] jepsen worker 65 - jepsen.util 4475 :ok :read [21 7] INFO [2022-01-10 05:48:53,701] jepsen worker 65 - jepsen.util 4475 :invoke :write [21 21] INFO [2022-01-10 05:48:53,730] jepsen worker 88 - jepsen.util 4858 :ok :read [29 0] INFO [2022-01-10 05:48:53,731] jepsen worker 88 - jepsen.util 4858 :invoke :read [29 nil] INFO [2022-01-10 05:48:53,740] jepsen worker 31 - jepsen.util 4981 :ok :read [10 12] INFO [2022-01-10 05:48:53,740] jepsen worker 31 - jepsen.util 4981 :invoke :write [10 37] INFO [2022-01-10 05:48:53,774] jepsen worker 53 - jepsen.util 5723 :info :write [17 8] :RequestTimeoutException INFO [2022-01-10 05:48:53,775] jepsen worker 53 - jepsen.util 5813 :invoke :write [17 28] INFO [2022-01-10 05:48:53,783] jepsen worker 89 - jepsen.util 5309 :ok :read [29 0] INFO [2022-01-10 05:48:53,783] jepsen worker 89 - jepsen.util 5309 :invoke :write [29 2] INFO [2022-01-10 05:48:53,856] jepsen worker 61 - jepsen.util 5011 :info :write [20 37] :RequestTimeoutException INFO [2022-01-10 05:48:53,857] jepsen worker 61 - jepsen.util 5101 :invoke :write [20 41] INFO [2022-01-10 05:48:54,006] jepsen worker 50 - jepsen.util 4910 :info :write [16 18] :RequestTimeoutException INFO [2022-01-10 05:48:54,007] jepsen worker 50 - jepsen.util 5000 :invoke :write [16 24] INFO [2022-01-10 05:48:54,141] jepsen worker 79 - jepsen.util 5569 :info :write [26 42] :RequestTimeoutException INFO [2022-01-10 05:48:54,142] jepsen worker 79 - jepsen.util 5659 :invoke :read [26 nil] INFO [2022-01-10 05:48:54,160] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6549707602339181}, :ns_1@172.23.107.227 {:progress 0.4604105571847508}, :ns_1@172.23.105.187 {:progress 0.4604105571847508}, :ns_1@172.23.107.228 {:progress 0.6235294117647059}} INFO [2022-01-10 05:48:54,161] jepsen worker nemesis - couchbase.util Rebalance check count: 680 INFO [2022-01-10 05:48:54,161] jepsen worker nemesis - couchbase.util Rebalance stuck count: 566 INFO [2022-01-10 05:48:54,168] jepsen worker 36 - jepsen.util 3726 :ok :read [12 14] INFO [2022-01-10 05:48:54,168] jepsen worker 36 - jepsen.util 3726 :invoke :read [12 nil] INFO [2022-01-10 05:48:54,171] jepsen worker 51 - jepsen.util 5631 :ok :read [17 41] INFO [2022-01-10 05:48:54,172] jepsen worker 51 - jepsen.util 5631 :invoke :write [17 18] INFO [2022-01-10 05:48:54,195] jepsen worker 60 - jepsen.util 5100 :info :write [20 7] :RequestTimeoutException INFO [2022-01-10 05:48:54,195] jepsen worker 76 - jepsen.util 4126 :info :write [25 45] :RequestTimeoutException INFO [2022-01-10 05:48:54,196] jepsen worker 60 - jepsen.util 5190 :invoke :write [20 29] INFO [2022-01-10 05:48:54,197] jepsen worker 76 - jepsen.util 4216 :invoke :read [25 nil] INFO [2022-01-10 05:48:54,256] jepsen worker 4 - jepsen.util 5044 :ok :read [1 49] INFO [2022-01-10 05:48:54,257] jepsen worker 4 - jepsen.util 5044 :invoke :write [1 43] INFO [2022-01-10 05:48:54,341] jepsen worker 37 - jepsen.util 4447 :info :write [12 25] :RequestTimeoutException INFO [2022-01-10 05:48:54,342] jepsen worker 37 - jepsen.util 4537 :invoke :write [12 14] INFO [2022-01-10 05:48:54,420] jepsen worker 36 - jepsen.util 3726 :ok :read [12 14] INFO [2022-01-10 05:48:54,420] jepsen worker 36 - jepsen.util 3726 :invoke :read [12 nil] INFO [2022-01-10 05:48:54,420] jepsen worker 79 - jepsen.util 5659 :ok :read [26 10] INFO [2022-01-10 05:48:54,421] jepsen worker 79 - jepsen.util 5659 :invoke :read [26 nil] INFO [2022-01-10 05:48:54,496] jepsen worker 34 - jepsen.util 4624 :info :write [11 1] :RequestTimeoutException INFO [2022-01-10 05:48:54,497] jepsen worker 34 - jepsen.util 4714 :invoke :read [11 nil] INFO [2022-01-10 05:48:54,552] jepsen worker 56 - jepsen.util 5006 :info :write [18 40] :RequestTimeoutException INFO [2022-01-10 05:48:54,552] jepsen worker 56 - jepsen.util 5096 :invoke :write [18 3] INFO [2022-01-10 05:48:54,731] jepsen worker 36 - jepsen.util 3726 :ok :read [12 14] INFO [2022-01-10 05:48:54,731] jepsen worker 79 - jepsen.util 5659 :ok :read [26 10] INFO [2022-01-10 05:48:54,732] jepsen worker 79 - jepsen.util 5659 :invoke :write [26 24] INFO [2022-01-10 05:48:54,732] jepsen worker 36 - jepsen.util 3726 :invoke :write [12 42] INFO [2022-01-10 05:48:54,859] jepsen worker 0 - jepsen.util 4860 :info :write [0 20] :RequestTimeoutException INFO [2022-01-10 05:48:54,860] jepsen worker 0 - jepsen.util 4950 :invoke :read [0 nil] INFO [2022-01-10 05:48:55,053] jepsen worker 21 - jepsen.util 4791 :ok :read [7 :nil] INFO [2022-01-10 05:48:55,053] jepsen worker 21 - jepsen.util 4791 :invoke :write [7 18] INFO [2022-01-10 05:48:55,122] jepsen worker 86 - jepsen.util 4046 :info :write [28 0] :RequestTimeoutException INFO [2022-01-10 05:48:55,123] jepsen worker 86 - jepsen.util 4136 :invoke :read [28 nil] INFO [2022-01-10 05:48:55,168] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6549707602339181}, :ns_1@172.23.107.227 {:progress 0.4604105571847508}, :ns_1@172.23.105.187 {:progress 0.4604105571847508}, :ns_1@172.23.107.228 {:progress 0.6235294117647059}} INFO [2022-01-10 05:48:55,168] jepsen worker nemesis - couchbase.util Rebalance check count: 681 INFO [2022-01-10 05:48:55,168] jepsen worker nemesis - couchbase.util Rebalance stuck count: 567 INFO [2022-01-10 05:48:55,216] jepsen worker 55 - jepsen.util 5275 :info :write [18 47] :RequestTimeoutException INFO [2022-01-10 05:48:55,216] jepsen worker 55 - jepsen.util 5365 :invoke :read [18 nil] INFO [2022-01-10 05:48:55,235] jepsen worker 54 - jepsen.util 5004 :info :write [18 7] :RequestTimeoutException INFO [2022-01-10 05:48:55,236] jepsen worker 54 - jepsen.util 5094 :invoke :write [18 13] INFO [2022-01-10 05:48:55,306] jepsen worker 63 - jepsen.util 4383 :info :write [21 31] :RequestTimeoutException INFO [2022-01-10 05:48:55,307] jepsen worker 63 - jepsen.util 4473 :invoke :write [21 5] INFO [2022-01-10 05:48:55,310] jepsen worker 87 - jepsen.util 4587 :info :write [29 40] :RequestTimeoutException INFO [2022-01-10 05:48:55,311] jepsen worker 87 - jepsen.util 4677 :invoke :read [29 nil] INFO [2022-01-10 05:48:55,479] jepsen worker 71 - jepsen.util 4391 :info :write [23 47] :RequestTimeoutException INFO [2022-01-10 05:48:55,480] jepsen worker 71 - jepsen.util 4481 :invoke :read [23 nil] INFO [2022-01-10 05:48:55,602] jepsen worker 2 - jepsen.util 4952 :info :write [0 37] :RequestTimeoutException INFO [2022-01-10 05:48:55,603] jepsen worker 2 - jepsen.util 5042 :invoke :write [0 14] INFO [2022-01-10 05:48:55,752] jepsen worker 29 - jepsen.util 4619 :info :write [9 28] :RequestTimeoutException INFO [2022-01-10 05:48:55,753] jepsen worker 29 - jepsen.util 4709 :invoke :write [9 33] INFO [2022-01-10 05:48:55,756] jepsen worker 28 - jepsen.util 4348 :info :write [9 23] :RequestTimeoutException INFO [2022-01-10 05:48:55,756] jepsen worker 28 - jepsen.util 4438 :invoke :write [9 35] INFO [2022-01-10 05:48:55,761] jepsen worker 59 - jepsen.util 4559 :info :write [19 49] :RequestTimeoutException INFO [2022-01-10 05:48:55,762] jepsen worker 59 - jepsen.util 4649 :invoke :read [19 nil] INFO [2022-01-10 05:48:56,016] jepsen worker 30 - jepsen.util 4440 :info :write [10 12] :RequestTimeoutException INFO [2022-01-10 05:48:56,017] jepsen worker 30 - jepsen.util 4530 :invoke :write [10 31] INFO [2022-01-10 05:48:56,071] jepsen worker 85 - jepsen.util 5125 :info :write [28 38] :SyncWriteAmbiguous INFO [2022-01-10 05:48:56,072] jepsen worker 85 - jepsen.util 5215 :invoke :write [28 49] INFO [2022-01-10 05:48:56,174] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6549707602339181}, :ns_1@172.23.107.227 {:progress 0.4604105571847508}, :ns_1@172.23.105.187 {:progress 0.4604105571847508}, :ns_1@172.23.107.228 {:progress 0.6235294117647059}} INFO [2022-01-10 05:48:56,174] jepsen worker nemesis - couchbase.util Rebalance check count: 682 INFO [2022-01-10 05:48:56,174] jepsen worker nemesis - couchbase.util Rebalance stuck count: 568 INFO [2022-01-10 05:48:56,437] jepsen worker 49 - jepsen.util 4639 :ok :read [16 38] INFO [2022-01-10 05:48:56,438] jepsen worker 49 - jepsen.util 4639 :invoke :read [16 nil] INFO [2022-01-10 05:48:56,450] jepsen worker 0 - jepsen.util 4950 :ok :read [0 46] INFO [2022-01-10 05:48:56,451] jepsen worker 0 - jepsen.util 4950 :invoke :write [0 43] INFO [2022-01-10 05:48:56,533] jepsen worker 49 - jepsen.util 4639 :ok :read [16 38] INFO [2022-01-10 05:48:56,533] jepsen worker 49 - jepsen.util 4639 :invoke :write [16 26] INFO [2022-01-10 05:48:56,816] jepsen worker 76 - jepsen.util 4216 :ok :read [25 :nil] INFO [2022-01-10 05:48:56,817] jepsen worker 76 - jepsen.util 4216 :invoke :read [25 nil] INFO [2022-01-10 05:48:56,819] jepsen worker 76 - jepsen.util 4216 :ok :read [25 :nil] INFO [2022-01-10 05:48:56,819] jepsen worker 76 - jepsen.util 4216 :invoke :write [25 44] INFO [2022-01-10 05:48:57,104] jepsen worker 71 - jepsen.util 4481 :ok :read [23 26] INFO [2022-01-10 05:48:57,104] jepsen worker 71 - jepsen.util 4481 :invoke :write [23 3] INFO [2022-01-10 05:48:57,120] jepsen worker 62 - jepsen.util 5102 :info :write [20 43] :RequestTimeoutException INFO [2022-01-10 05:48:57,120] jepsen worker 62 - jepsen.util 5192 :invoke :read [20 nil] INFO [2022-01-10 05:48:57,126] jepsen worker 77 - jepsen.util 5297 :info :write [25 2] :RequestTimeoutException INFO [2022-01-10 05:48:57,161] jepsen worker 32 - jepsen.util 5072 :info :write [10 2] :RequestTimeoutException INFO [2022-01-10 05:48:57,165] jepsen worker 62 - jepsen.util 5192 :ok :read [20 :nil] INFO [2022-01-10 05:48:57,181] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6608187134502924}, :ns_1@172.23.107.227 {:progress 0.4633431085043989}, :ns_1@172.23.105.187 {:progress 0.4633431085043989}, :ns_1@172.23.107.228 {:progress 0.6294117647058823}} INFO [2022-01-10 05:48:57,190] jepsen worker 62 - jepsen.util 5192 :invoke :write [20 29] INFO [2022-01-10 05:48:57,190] jepsen worker nemesis - couchbase.util Rebalance check count: 683 INFO [2022-01-10 05:48:57,190] jepsen worker 32 - jepsen.util 5162 :invoke :write [10 26] INFO [2022-01-10 05:48:57,190] jepsen worker 77 - jepsen.util 5387 :invoke :read [25 nil] INFO [2022-01-10 05:48:57,190] jepsen worker nemesis - couchbase.util Rebalance stuck count: 568 INFO [2022-01-10 05:48:57,330] jepsen worker 10 - jepsen.util 4600 :ok :read [3 12] INFO [2022-01-10 05:48:57,331] jepsen worker 10 - jepsen.util 4600 :invoke :read [3 nil] INFO [2022-01-10 05:48:57,361] jepsen worker 18 - jepsen.util 4608 :info :write [6 11] :RequestTimeoutException INFO [2022-01-10 05:48:57,364] jepsen worker 18 - jepsen.util 4698 :invoke :write [6 3] INFO [2022-01-10 05:48:57,365] jepsen worker 87 - jepsen.util 4677 :ok :read [29 0] INFO [2022-01-10 05:48:57,366] jepsen worker 87 - jepsen.util 4677 :invoke :write [29 39] INFO [2022-01-10 05:48:57,376] jepsen worker 24 - jepsen.util 4344 :info :write [8 32] :RequestTimeoutException INFO [2022-01-10 05:48:57,377] jepsen worker 24 - jepsen.util 4434 :invoke :write [8 7] INFO [2022-01-10 05:48:57,453] jepsen worker 10 - jepsen.util 4600 :ok :read [3 12] INFO [2022-01-10 05:48:57,454] jepsen worker 10 - jepsen.util 4600 :invoke :write [3 33] INFO [2022-01-10 05:48:57,615] jepsen worker 77 - jepsen.util 5387 :ok :read [25 :nil] INFO [2022-01-10 05:48:57,615] jepsen worker 77 - jepsen.util 5387 :invoke :read [25 nil] INFO [2022-01-10 05:48:57,759] jepsen worker 25 - jepsen.util 4435 :info :write [8 38] :RequestTimeoutException INFO [2022-01-10 05:48:57,760] jepsen worker 25 - jepsen.util 4525 :invoke :read [8 nil] INFO [2022-01-10 05:48:57,777] jepsen worker 19 - jepsen.util 4879 :info :write [6 38] :RequestTimeoutException INFO [2022-01-10 05:48:57,777] jepsen worker 19 - jepsen.util 4969 :invoke :read [6 nil] INFO [2022-01-10 05:48:58,070] jepsen worker 17 - jepsen.util 4697 :ok :read [5 25] INFO [2022-01-10 05:48:58,071] jepsen worker 11 - jepsen.util 4691 :info :write [3 19] :RequestTimeoutException INFO [2022-01-10 05:48:58,071] jepsen worker 17 - jepsen.util 4697 :invoke :read [5 nil] INFO [2022-01-10 05:48:58,072] jepsen worker 11 - jepsen.util 4781 :invoke :write [3 16] INFO [2022-01-10 05:48:58,097] jepsen worker 33 - jepsen.util 5073 :info :write [11 42] :RequestTimeoutException INFO [2022-01-10 05:48:58,097] jepsen worker 33 - jepsen.util 5163 :invoke :read [11 nil] INFO [2022-01-10 05:48:58,123] jepsen worker 17 - jepsen.util 4697 :ok :read [5 25] INFO [2022-01-10 05:48:58,124] jepsen worker 17 - jepsen.util 4697 :invoke :write [5 26] INFO [2022-01-10 05:48:58,198] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6608187134502924}, :ns_1@172.23.107.227 {:progress 0.4633431085043989}, :ns_1@172.23.105.187 {:progress 0.4633431085043989}, :ns_1@172.23.107.228 {:progress 0.6294117647058823}} INFO [2022-01-10 05:48:58,198] jepsen worker nemesis - couchbase.util Rebalance check count: 684 INFO [2022-01-10 05:48:58,198] jepsen worker nemesis - couchbase.util Rebalance stuck count: 569 INFO [2022-01-10 05:48:58,271] jepsen worker 84 - jepsen.util 4494 :info :write [28 35] :RequestTimeoutException INFO [2022-01-10 05:48:58,271] jepsen worker 84 - jepsen.util 4584 :invoke :write [28 16] INFO [2022-01-10 05:48:58,421] jepsen worker 70 - jepsen.util 4840 :info :write [23 31] :RequestTimeoutException INFO [2022-01-10 05:48:58,421] jepsen worker 70 - jepsen.util 4930 :invoke :read [23 nil] INFO [2022-01-10 05:48:58,434] jepsen worker 66 - jepsen.util 4656 :info :write [22 48] :RequestTimeoutException INFO [2022-01-10 05:48:58,435] jepsen worker 66 - jepsen.util 4746 :invoke :write [22 10] INFO [2022-01-10 05:48:58,495] jepsen worker 35 - jepsen.util 4535 :info :write [11 32] :RequestTimeoutException INFO [2022-01-10 05:48:58,496] jepsen worker 35 - jepsen.util 4625 :invoke :read [11 nil] INFO [2022-01-10 05:48:58,557] jepsen worker 69 - jepsen.util 4389 :info :write [23 44] :RequestTimeoutException INFO [2022-01-10 05:48:58,557] jepsen worker 69 - jepsen.util 4479 :invoke :read [23 nil] INFO [2022-01-10 05:48:58,666] jepsen worker 77 - jepsen.util 5387 :ok :read [25 :nil] INFO [2022-01-10 05:48:58,666] jepsen worker 77 - jepsen.util 5387 :invoke :write [25 45] INFO [2022-01-10 05:48:58,676] jepsen worker 70 - jepsen.util 4930 :ok :read [23 26] INFO [2022-01-10 05:48:58,676] jepsen worker 70 - jepsen.util 4930 :invoke :read [23 nil] INFO [2022-01-10 05:48:58,708] jepsen worker 70 - jepsen.util 4930 :ok :read [23 26] INFO [2022-01-10 05:48:58,708] jepsen worker 70 - jepsen.util 4930 :invoke :read [23 nil] INFO [2022-01-10 05:48:58,731] jepsen worker 20 - jepsen.util 4520 :info :write [6 17] :RequestTimeoutException INFO [2022-01-10 05:48:58,732] jepsen worker 20 - jepsen.util 4610 :invoke :read [6 nil] INFO [2022-01-10 05:48:58,743] jepsen worker 70 - jepsen.util 4930 :ok :read [23 26] INFO [2022-01-10 05:48:58,743] jepsen worker 70 - jepsen.util 4930 :invoke :write [23 0] INFO [2022-01-10 05:48:58,765] jepsen worker 6 - jepsen.util 4686 :info :write [2 6] :RequestTimeoutException INFO [2022-01-10 05:48:58,766] jepsen worker 6 - jepsen.util 4776 :invoke :write [2 39] INFO [2022-01-10 05:48:58,919] jepsen worker 9 - jepsen.util 5319 :info :write [3 9] :RequestTimeoutException INFO [2022-01-10 05:48:58,920] jepsen worker 9 - jepsen.util 5409 :invoke :read [3 nil] INFO [2022-01-10 05:48:59,136] jepsen worker 64 - jepsen.util 4834 :info :write [21 6] :RequestTimeoutException INFO [2022-01-10 05:48:59,137] jepsen worker 64 - jepsen.util 4924 :invoke :read [21 nil] INFO [2022-01-10 05:48:59,203] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6666666666666667}, :ns_1@172.23.107.227 {:progress 0.466275659824047}, :ns_1@172.23.105.187 {:progress 0.466275659824047}, :ns_1@172.23.107.228 {:progress 0.6352941176470588}} INFO [2022-01-10 05:48:59,204] jepsen worker nemesis - couchbase.util Rebalance check count: 685 INFO [2022-01-10 05:48:59,204] jepsen worker nemesis - couchbase.util Rebalance stuck count: 569 INFO [2022-01-10 05:48:59,311] jepsen worker 47 - jepsen.util 4907 :info :write [15 24] :RequestTimeoutException INFO [2022-01-10 05:48:59,312] jepsen worker 47 - jepsen.util 4997 :invoke :write [15 27] INFO [2022-01-10 05:48:59,407] jepsen worker 69 - jepsen.util 4479 :ok :read [23 26] INFO [2022-01-10 05:48:59,408] jepsen worker 69 - jepsen.util 4479 :invoke :write [23 1] INFO [2022-01-10 05:48:59,413] jepsen worker 35 - jepsen.util 4625 :ok :read [11 21] INFO [2022-01-10 05:48:59,413] jepsen worker 35 - jepsen.util 4625 :invoke :write [11 14] INFO [2022-01-10 05:48:59,439] jepsen worker 45 - jepsen.util 5175 :info :write [15 46] :RequestTimeoutException INFO [2022-01-10 05:48:59,440] jepsen worker 45 - jepsen.util 5265 :invoke :write [15 11] INFO [2022-01-10 05:48:59,661] jepsen worker 26 - jepsen.util 4616 :info :write [8 35] :RequestTimeoutException INFO [2022-01-10 05:48:59,662] jepsen worker 26 - jepsen.util 4706 :invoke :write [8 37] INFO [2022-01-10 05:48:59,694] jepsen worker 23 - jepsen.util 4883 :info :write [7 33] :RequestTimeoutException INFO [2022-01-10 05:48:59,695] jepsen worker 23 - jepsen.util 4973 :invoke :write [7 49] INFO [2022-01-10 05:48:59,716] jepsen worker 48 - jepsen.util 4638 :info :write [16 25] :RequestTimeoutException INFO [2022-01-10 05:48:59,717] jepsen worker 48 - jepsen.util 4728 :invoke :read [16 nil] INFO [2022-01-10 05:48:59,759] jepsen worker 48 - jepsen.util 4728 :ok :read [16 38] INFO [2022-01-10 05:48:59,760] jepsen worker 48 - jepsen.util 4728 :invoke :write [16 27] INFO [2022-01-10 05:48:59,860] jepsen worker 13 - jepsen.util 5593 :info :write [4 45] :RequestTimeoutException INFO [2022-01-10 05:48:59,860] jepsen worker 13 - jepsen.util 5683 :invoke :read [4 nil] INFO [2022-01-10 05:49:00,209] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6666666666666667}, :ns_1@172.23.107.227 {:progress 0.466275659824047}, :ns_1@172.23.105.187 {:progress 0.466275659824047}, :ns_1@172.23.107.228 {:progress 0.6352941176470588}} INFO [2022-01-10 05:49:00,209] jepsen worker nemesis - couchbase.util Rebalance check count: 686 INFO [2022-01-10 05:49:00,209] jepsen worker nemesis - couchbase.util Rebalance stuck count: 570 INFO [2022-01-10 05:49:00,340] jepsen worker 7 - jepsen.util 5137 :info :write [2 16] :RequestTimeoutException INFO [2022-01-10 05:49:00,341] jepsen worker 7 - jepsen.util 5227 :invoke :write [2 42] INFO [2022-01-10 05:49:00,471] jepsen worker 38 - jepsen.util 4448 :info :write [12 20] :RequestTimeoutException INFO [2022-01-10 05:49:00,472] jepsen worker 38 - jepsen.util 4538 :invoke :write [12 48] INFO [2022-01-10 05:49:00,591] jepsen worker 80 - jepsen.util 5930 :info :write [26 30] :RequestTimeoutException INFO [2022-01-10 05:49:00,592] jepsen worker 80 - jepsen.util 6020 :invoke :read [26 nil] INFO [2022-01-10 05:49:00,636] jepsen worker 14 - jepsen.util 5774 :info :write [4 48] :RequestTimeoutException INFO [2022-01-10 05:49:00,636] jepsen worker 14 - jepsen.util 5864 :invoke :read [4 nil] INFO [2022-01-10 05:49:00,659] jepsen worker 72 - jepsen.util 5112 :info :write [24 29] :RequestTimeoutException INFO [2022-01-10 05:49:00,660] jepsen worker 72 - jepsen.util 5202 :invoke :write [24 22] INFO [2022-01-10 05:49:00,795] jepsen worker 80 - jepsen.util 6020 :ok :read [26 10] INFO [2022-01-10 05:49:00,795] jepsen worker 80 - jepsen.util 6020 :invoke :write [26 28] WARN [2022-01-10 05:49:00,810] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":13696684,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13696684,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9046921,"encode_duration_us":17137,"operation_id":"0x9274","timeout_ms":10000,"last_server_duration_us":9046921,"total_duration_us":10001004},{"operation_name":"upsert","last_dispatch_duration_us":17220124,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17853007,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9277126,"encode_duration_us":24429,"operation_id":"0x9210","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10009809},{"operation_name":"upsert","last_dispatch_duration_us":10303083,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14068004,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":11136668,"encode_duration_us":19739,"operation_id":"0x935a","timeout_ms":10000,"last_server_duration_us":10112824,"total_duration_us":10003603}],"total_count":3}} WARN [2022-01-10 05:49:00,810] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12951903,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":12951903,"total_server_duration_us":43,"operation_id":"0x9319","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":12953072},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","encode_duration_us":23291,"operation_id":"0x93f3","timeout_ms":10000,"total_duration_us":10010884},{"operation_name":"upsert","last_dispatch_duration_us":384022,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":624103,"total_server_duration_us":108909,"encode_duration_us":17585,"operation_id":"0x949d","timeout_ms":10000,"last_server_duration_us":23898,"total_duration_us":10006233},{"operation_name":"upsert","last_dispatch_duration_us":1840452,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":10181672,"total_server_duration_us":973412,"encode_duration_us":17371,"operation_id":"0x93a7","timeout_ms":10000,"last_server_duration_us":57509,"total_duration_us":10005793},{"operation_name":"upsert","last_dispatch_duration_us":31720,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":6788696,"total_server_duration_us":723624,"encode_duration_us":16236,"operation_id":"0x9485","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10004653},{"operation_name":"upsert","last_dispatch_duration_us":1136841,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":7573768,"total_server_duration_us":700190,"encode_duration_us":18113,"operation_id":"0x9305","timeout_ms":10000,"last_server_duration_us":147353,"total_duration_us":10003858},{"operation_name":"upsert","last_dispatch_duration_us":10303083,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":14068004,"total_server_duration_us":11136668,"encode_duration_us":19739,"operation_id":"0x935a","timeout_ms":10000,"last_server_duration_us":10112824,"total_duration_us":10003603},{"operation_name":"upsert","last_dispatch_duration_us":9999558,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11610667,"total_server_duration_us":77461,"encode_duration_us":27115,"operation_id":"0x93c1","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10003498},{"operation_name":"upsert","last_dispatch_duration_us":2240190,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":10310642,"total_server_duration_us":2010327,"encode_duration_us":18745,"operation_id":"0x936c","timeout_ms":10000,"last_server_duration_us":1165191,"total_duration_us":10002291},{"operation_name":"upsert","last_dispatch_duration_us":639360,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":1422511,"total_server_duration_us":300605,"encode_duration_us":16367,"operation_id":"0x94b9","timeout_ms":10000,"last_server_duration_us":22637,"total_duration_us":10002124}],"total_count":20}} INFO [2022-01-10 05:49:00,855] jepsen worker 74 - jepsen.util 5294 :info :write [24 29] :RequestTimeoutException INFO [2022-01-10 05:49:00,856] jepsen worker 74 - jepsen.util 5384 :invoke :read [24 nil] INFO [2022-01-10 05:49:00,895] jepsen worker 14 - jepsen.util 5864 :ok :read [4 3] INFO [2022-01-10 05:49:00,896] jepsen worker 14 - jepsen.util 5864 :invoke :read [4 nil] INFO [2022-01-10 05:49:00,994] jepsen worker 78 - jepsen.util 5928 :info :write [26 36] :RequestTimeoutException INFO [2022-01-10 05:49:00,994] jepsen worker 78 - jepsen.util 6018 :invoke :write [26 17] INFO [2022-01-10 05:49:01,003] jepsen worker 13 - jepsen.util 5683 :ok :read [4 3] INFO [2022-01-10 05:49:01,004] jepsen worker 13 - jepsen.util 5683 :invoke :write [4 46] INFO [2022-01-10 05:49:01,005] jepsen worker 68 - jepsen.util 4928 :info :write [22 1] :RequestTimeoutException INFO [2022-01-10 05:49:01,005] jepsen worker 68 - jepsen.util 5018 :invoke :read [22 nil] INFO [2022-01-10 05:49:01,024] jepsen worker 67 - jepsen.util 4387 :info :write [22 0] :RequestTimeoutException INFO [2022-01-10 05:49:01,024] jepsen worker 39 - jepsen.util 5079 :info :write [13 39] :RequestTimeoutException INFO [2022-01-10 05:49:01,025] jepsen worker 67 - jepsen.util 4477 :invoke :write [22 46] INFO [2022-01-10 05:49:01,025] jepsen worker 39 - jepsen.util 5169 :invoke :read [13 nil] INFO [2022-01-10 05:49:01,189] jepsen worker 3 - jepsen.util 4773 :info :write [1 5] :RequestTimeoutException INFO [2022-01-10 05:49:01,190] jepsen worker 3 - jepsen.util 4863 :invoke :read [1 nil] INFO [2022-01-10 05:49:01,215] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6666666666666667}, :ns_1@172.23.107.227 {:progress 0.466275659824047}, :ns_1@172.23.105.187 {:progress 0.466275659824047}, :ns_1@172.23.107.228 {:progress 0.6352941176470588}} INFO [2022-01-10 05:49:01,215] jepsen worker nemesis - couchbase.util Rebalance check count: 687 INFO [2022-01-10 05:49:01,215] jepsen worker nemesis - couchbase.util Rebalance stuck count: 571 INFO [2022-01-10 05:49:01,293] jepsen worker 14 - jepsen.util 5864 :ok :read [4 3] INFO [2022-01-10 05:49:01,294] jepsen worker 14 - jepsen.util 5864 :invoke :write [4 26] INFO [2022-01-10 05:49:01,345] jepsen worker 46 - jepsen.util 4906 :info :write [15 9] :RequestTimeoutException INFO [2022-01-10 05:49:01,346] jepsen worker 46 - jepsen.util 4996 :invoke :write [15 31] INFO [2022-01-10 05:49:01,354] jepsen worker 83 - jepsen.util 5483 :info :write [27 20] :RequestTimeoutException INFO [2022-01-10 05:49:01,355] jepsen worker 83 - jepsen.util 5573 :invoke :read [27 nil] INFO [2022-01-10 05:49:01,403] jepsen worker 3 - jepsen.util 4863 :ok :read [1 49] INFO [2022-01-10 05:49:01,403] jepsen worker 3 - jepsen.util 4863 :invoke :write [1 39] INFO [2022-01-10 05:49:01,706] jepsen worker 8 - jepsen.util 4958 :info :write [2 19] :RequestTimeoutException INFO [2022-01-10 05:49:01,706] jepsen worker 8 - jepsen.util 5048 :invoke :read [2 nil] INFO [2022-01-10 05:49:01,825] jepsen worker 64 - jepsen.util 4924 :ok :read [21 7] INFO [2022-01-10 05:49:01,826] jepsen worker 64 - jepsen.util 4924 :invoke :write [21 24] INFO [2022-01-10 05:49:01,836] jepsen worker 41 - jepsen.util 4721 :info :write [13 36] :RequestTimeoutException INFO [2022-01-10 05:49:01,836] jepsen worker 41 - jepsen.util 4811 :invoke :read [13 nil] INFO [2022-01-10 05:49:02,013] jepsen worker 25 - jepsen.util 4525 :ok :read [8 39] INFO [2022-01-10 05:49:02,013] jepsen worker 25 - jepsen.util 4525 :invoke :read [8 nil] INFO [2022-01-10 05:49:02,056] jepsen worker 57 - jepsen.util 5547 :info :write [19 12] :RequestTimeoutException INFO [2022-01-10 05:49:02,057] jepsen worker 57 - jepsen.util 5637 :invoke :read [19 nil] INFO [2022-01-10 05:49:02,080] jepsen worker 74 - jepsen.util 5384 :ok :read [24 18] INFO [2022-01-10 05:49:02,081] jepsen worker 74 - jepsen.util 5384 :invoke :read [24 nil] INFO [2022-01-10 05:49:02,149] jepsen worker 27 - jepsen.util 5157 :info :write [9 11] :RequestTimeoutException INFO [2022-01-10 05:49:02,150] jepsen worker 27 - jepsen.util 5247 :invoke :write [9 1] INFO [2022-01-10 05:49:02,166] jepsen worker 73 - jepsen.util 4663 :info :write [24 25] :RequestTimeoutException INFO [2022-01-10 05:49:02,166] jepsen worker 73 - jepsen.util 4753 :invoke :write [24 30] INFO [2022-01-10 05:49:02,220] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.672514619883041}, :ns_1@172.23.107.227 {:progress 0.469208211143695}, :ns_1@172.23.105.187 {:progress 0.469208211143695}, :ns_1@172.23.107.228 {:progress 0.6411764705882352}} INFO [2022-01-10 05:49:02,221] jepsen worker nemesis - couchbase.util Rebalance check count: 688 INFO [2022-01-10 05:49:02,221] jepsen worker nemesis - couchbase.util Rebalance stuck count: 571 WARN [2022-01-10 05:49:02,222] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":136085,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":7874140,"total_server_duration_us":2394989,"encode_duration_us":18064,"operation_id":"0x9511","timeout_ms":10000,"last_server_duration_us":97329,"total_duration_us":10009601},{"operation_name":"upsert","last_dispatch_duration_us":13130060,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":13130060,"total_server_duration_us":19030,"encode_duration_us":18221,"operation_id":"0x9450","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10009402},{"operation_name":"upsert","last_dispatch_duration_us":6887676,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":14813213,"total_server_duration_us":200364,"encode_duration_us":18411,"operation_id":"0x93f6","timeout_ms":10000,"last_server_duration_us":25362,"total_duration_us":10008303},{"operation_name":"upsert","last_dispatch_duration_us":6532356,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":14625709,"total_server_duration_us":253723,"encode_duration_us":19323,"operation_id":"0x93c7","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10006676},{"operation_name":"upsert","last_dispatch_duration_us":2734645,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9861864,"total_server_duration_us":826750,"encode_duration_us":18158,"operation_id":"0x935e","timeout_ms":10000,"last_server_duration_us":41875,"total_duration_us":10005932},{"operation_name":"upsert","last_dispatch_duration_us":445119,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":6599436,"total_server_duration_us":3159024,"encode_duration_us":17526,"operation_id":"0x93c4","timeout_ms":10000,"last_server_duration_us":84283,"total_duration_us":10004999},{"operation_name":"upsert","last_dispatch_duration_us":1686622,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":6089095,"total_server_duration_us":1384519,"encode_duration_us":17413,"operation_id":"0x93bf","timeout_ms":10000,"last_server_duration_us":130462,"total_duration_us":10003701},{"operation_name":"upsert","last_dispatch_duration_us":1007731,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":2205819,"total_server_duration_us":433664,"encode_duration_us":18531,"operation_id":"0x9483","timeout_ms":10000,"last_server_duration_us":58999,"total_duration_us":10003291},{"operation_name":"upsert","last_dispatch_duration_us":925883,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":8011980,"total_server_duration_us":2063016,"encode_duration_us":16181,"operation_id":"0x9484","timeout_ms":10000,"last_server_duration_us":366968,"total_duration_us":10003250},{"operation_name":"upsert","last_dispatch_duration_us":144077,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":7886701,"total_server_duration_us":1223694,"encode_duration_us":20470,"operation_id":"0x94e6","timeout_ms":10000,"last_server_duration_us":40695,"total_duration_us":10002946}],"total_count":16}} WARN [2022-01-10 05:49:02,322] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":14259396,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15637097,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":7278492,"encode_duration_us":16861,"operation_id":"0x92cc","timeout_ms":10000,"last_server_duration_us":6872978,"total_duration_us":10003700},{"operation_name":"upsert","last_dispatch_duration_us":9074788,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10209351,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9415475,"encode_duration_us":21804,"operation_id":"0x92cb","timeout_ms":10000,"last_server_duration_us":9030999,"total_duration_us":10004030}],"total_count":2}} INFO [2022-01-10 05:49:02,350] jepsen worker 25 - jepsen.util 4525 :ok :read [8 39] INFO [2022-01-10 05:49:02,350] jepsen worker 25 - jepsen.util 4525 :invoke :read [8 nil] INFO [2022-01-10 05:49:02,376] jepsen worker 82 - jepsen.util 4942 :info :write [27 3] :RequestTimeoutException INFO [2022-01-10 05:49:02,377] jepsen worker 82 - jepsen.util 5032 :invoke :read [27 nil] INFO [2022-01-10 05:49:02,386] jepsen worker 16 - jepsen.util 4876 :info :write [5 3] :RequestTimeoutException INFO [2022-01-10 05:49:02,387] jepsen worker 16 - jepsen.util 4966 :invoke :read [5 nil] INFO [2022-01-10 05:49:02,390] jepsen worker 74 - jepsen.util 5384 :ok :read [24 18] INFO [2022-01-10 05:49:02,390] jepsen worker 74 - jepsen.util 5384 :invoke :read [24 nil] WARN [2022-01-10 05:49:02,411] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":850353,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6297675,"total_server_duration_us":1465794,"encode_duration_us":18948,"operation_id":"0x947b","timeout_ms":10000,"last_server_duration_us":113793,"total_duration_us":10010200},{"operation_name":"upsert","last_dispatch_duration_us":33134,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":1026276,"total_server_duration_us":599779,"encode_duration_us":16879,"operation_id":"0x93f2","timeout_ms":10000,"last_server_duration_us":20203,"total_duration_us":10008032},{"operation_name":"upsert","last_dispatch_duration_us":8275022,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8976531,"total_server_duration_us":212286,"encode_duration_us":16952,"operation_id":"0x9362","timeout_ms":10000,"last_server_duration_us":32258,"total_duration_us":10007670},{"operation_name":"upsert","last_dispatch_duration_us":31824,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":942624,"total_server_duration_us":568507,"encode_duration_us":16400,"operation_id":"0x941e","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10007016},{"operation_name":"upsert","last_dispatch_duration_us":7262759,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":13873185,"total_server_duration_us":613222,"encode_duration_us":17038,"operation_id":"0x946a","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10006115},{"operation_name":"upsert","last_dispatch_duration_us":1249302,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":8792055,"total_server_duration_us":1156384,"encode_duration_us":17335,"operation_id":"0x9393","timeout_ms":10000,"last_server_duration_us":75755,"total_duration_us":10005926},{"operation_name":"upsert","last_dispatch_duration_us":8054887,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9517159,"total_server_duration_us":751375,"encode_duration_us":16937,"operation_id":"0x940f","timeout_ms":10000,"last_server_duration_us":29415,"total_duration_us":10003806},{"operation_name":"upsert","last_dispatch_duration_us":2030924,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9048897,"total_server_duration_us":1569858,"encode_duration_us":16564,"operation_id":"0x9489","timeout_ms":10000,"last_server_duration_us":47647,"total_duration_us":10003123},{"operation_name":"upsert","last_dispatch_duration_us":7765665,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":16506709,"total_server_duration_us":318548,"encode_duration_us":16463,"operation_id":"0x93c5","timeout_ms":10000,"last_server_duration_us":21082,"total_duration_us":10002898},{"operation_name":"upsert","last_dispatch_duration_us":7330389,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":12458883,"total_server_duration_us":336277,"encode_duration_us":16418,"operation_id":"0x94b3","timeout_ms":10000,"last_server_duration_us":22804,"total_duration_us":10002476}],"total_count":23}} WARN [2022-01-10 05:49:02,412] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1329245,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4053553,"total_server_duration_us":527353,"encode_duration_us":22686,"operation_id":"0x93f5","timeout_ms":10000,"last_server_duration_us":49487,"total_duration_us":10011035},{"operation_name":"upsert","last_dispatch_duration_us":1784648,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7394173,"total_server_duration_us":372630,"encode_duration_us":19813,"operation_id":"0x9456","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10009027},{"operation_name":"upsert","last_dispatch_duration_us":2277002,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6997472,"total_server_duration_us":576029,"encode_duration_us":19312,"operation_id":"0x93c0","timeout_ms":10000,"last_server_duration_us":85596,"total_duration_us":10008877},{"operation_name":"upsert","last_dispatch_duration_us":284013,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":1941847,"total_server_duration_us":109416,"encode_duration_us":17251,"operation_id":"0x9546","timeout_ms":10000,"last_server_duration_us":22223,"total_duration_us":10008506},{"operation_name":"upsert","last_dispatch_duration_us":306720,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":6850384,"total_server_duration_us":834385,"encode_duration_us":16848,"operation_id":"0x9455","timeout_ms":10000,"last_server_duration_us":177290,"total_duration_us":10007934},{"operation_name":"upsert","last_dispatch_duration_us":383562,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4536836,"total_server_duration_us":154944,"encode_duration_us":19149,"operation_id":"0x950e","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10005040},{"operation_name":"upsert","last_dispatch_duration_us":1036098,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":8312172,"total_server_duration_us":1917195,"encode_duration_us":19386,"operation_id":"0x9392","timeout_ms":10000,"last_server_duration_us":74786,"total_duration_us":10004653},{"operation_name":"upsert","last_dispatch_duration_us":624846,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5941803,"total_server_duration_us":230662,"encode_duration_us":18876,"operation_id":"0x9486","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":10004021},{"operation_name":"upsert","last_dispatch_duration_us":749794,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":4085757,"total_server_duration_us":270639,"encode_duration_us":17066,"operation_id":"0x9540","timeout_ms":10000,"last_server_duration_us":61774,"total_duration_us":10003183},{"operation_name":"upsert","last_dispatch_duration_us":967471,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9238042,"total_server_duration_us":2088159,"encode_duration_us":17096,"operation_id":"0x94ed","timeout_ms":10000,"last_server_duration_us":58004,"total_duration_us":10002452}],"total_count":16}} WARN [2022-01-10 05:49:02,418] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9079914,"last_local_id":"EFE9906200000003/00000000135F5DA5","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18339362,"last_local_socket":"172.23.104.79:34164","total_server_duration_us":9087031,"encode_duration_us":18231,"operation_id":"0x9211","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10002793},{"operation_name":"upsert","last_dispatch_duration_us":10061948,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12088802,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9552783,"encode_duration_us":20052,"operation_id":"0x94b7","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10006974}],"total_count":2}} WARN [2022-01-10 05:49:02,511] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":12978748,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12978748,"total_server_duration_us":17888,"operation_id":"0x92d7","timeout_ms":30000,"last_server_duration_us":17888,"total_duration_us":12981036},{"operation_name":"get","last_dispatch_duration_us":10726644,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10726644,"total_server_duration_us":92892,"operation_id":"0x9388","timeout_ms":30000,"last_server_duration_us":92892,"total_duration_us":10728611},{"operation_name":"get","last_dispatch_duration_us":10015094,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10015094,"total_server_duration_us":4404,"operation_id":"0x9391","timeout_ms":30000,"last_server_duration_us":4404,"total_duration_us":10015306},{"operation_name":"upsert","last_dispatch_duration_us":8562438,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11272646,"total_server_duration_us":621291,"encode_duration_us":17273,"operation_id":"0x93ea","timeout_ms":10000,"last_server_duration_us":18953,"total_duration_us":10008150},{"operation_name":"upsert","last_dispatch_duration_us":68287,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9673795,"total_server_duration_us":130642,"encode_duration_us":17903,"operation_id":"0x947a","timeout_ms":10000,"last_server_duration_us":21731,"total_duration_us":10007324},{"operation_name":"upsert","last_dispatch_duration_us":9245200,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9972594,"total_server_duration_us":9091513,"encode_duration_us":16569,"operation_id":"0x9410","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10004999},{"operation_name":"upsert","last_dispatch_duration_us":8669884,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11107314,"total_server_duration_us":228583,"encode_duration_us":19933,"operation_id":"0x940e","timeout_ms":10000,"last_server_duration_us":32548,"total_duration_us":10004273},{"operation_name":"upsert","last_dispatch_duration_us":9509098,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11784470,"total_server_duration_us":9239499,"encode_duration_us":16887,"operation_id":"0x93fb","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10002547},{"operation_name":"upsert","last_dispatch_duration_us":8663611,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9833878,"total_server_duration_us":127661,"encode_duration_us":17116,"operation_id":"0x9361","timeout_ms":10000,"last_server_duration_us":23306,"total_duration_us":10001583},{"operation_name":"upsert","last_dispatch_duration_us":474786,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":5030326,"total_server_duration_us":3342599,"encode_duration_us":16574,"operation_id":"0x94ec","timeout_ms":10000,"last_server_duration_us":55061,"total_duration_us":10001065}],"total_count":13}} WARN [2022-01-10 05:49:02,511] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9513733,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17108983,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9272815,"encode_duration_us":18127,"operation_id":"0x92ca","timeout_ms":10000,"last_server_duration_us":9112856,"total_duration_us":10000959},{"operation_name":"upsert","last_dispatch_duration_us":9210093,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9489870,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9079847,"encode_duration_us":17479,"operation_id":"0x92fc","timeout_ms":10000,"last_server_duration_us":9058604,"total_duration_us":10001416},{"operation_name":"upsert","last_dispatch_duration_us":17482889,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17987167,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9312171,"encode_duration_us":18643,"operation_id":"0x926c","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10010743},{"operation_name":"upsert","last_dispatch_duration_us":9509098,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11784470,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9239499,"encode_duration_us":16887,"operation_id":"0x93fb","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":10002547},{"operation_name":"upsert","last_dispatch_duration_us":9245200,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9972594,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9091513,"encode_duration_us":16569,"operation_id":"0x9410","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10004999}],"total_count":5}} INFO [2022-01-10 05:49:02,521] jepsen worker 81 - jepsen.util 5391 :info :write [27 49] :RequestTimeoutException INFO [2022-01-10 05:49:02,522] jepsen worker 81 - jepsen.util 5481 :invoke :write [27 13] INFO [2022-01-10 05:49:02,545] jepsen worker 58 - jepsen.util 4828 :info :write [19 13] :RequestTimeoutException INFO [2022-01-10 05:49:02,545] jepsen worker 58 - jepsen.util 4918 :invoke :write [19 19] INFO [2022-01-10 05:49:02,552] jepsen worker 25 - jepsen.util 4525 :ok :read [8 39] INFO [2022-01-10 05:49:02,552] jepsen worker 25 - jepsen.util 4525 :invoke :read [8 nil] INFO [2022-01-10 05:49:02,606] jepsen worker 1 - jepsen.util 4861 :info :write [0 10] :RequestTimeoutException INFO [2022-01-10 05:49:02,606] jepsen worker 1 - jepsen.util 4951 :invoke :read [0 nil] WARN [2022-01-10 05:49:02,611] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10172978,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14386226,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9742208,"encode_duration_us":22896,"operation_id":"0x9449","timeout_ms":10000,"last_server_duration_us":9232481,"total_duration_us":10001801}],"total_count":1}} WARN [2022-01-10 05:49:02,618] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":978014,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":3432435,"total_server_duration_us":264382,"encode_duration_us":17170,"operation_id":"0x9448","timeout_ms":10000,"last_server_duration_us":22887,"total_duration_us":10010866},{"operation_name":"upsert","last_dispatch_duration_us":9056624,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11014813,"total_server_duration_us":224721,"encode_duration_us":18284,"operation_id":"0x94ee","timeout_ms":10000,"last_server_duration_us":19496,"total_duration_us":10008380},{"operation_name":"upsert","last_dispatch_duration_us":9167385,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":11299992,"total_server_duration_us":298412,"encode_duration_us":16289,"operation_id":"0x94e5","timeout_ms":10000,"last_server_duration_us":27222,"total_duration_us":10007861},{"operation_name":"upsert","last_dispatch_duration_us":34858,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":5395641,"total_server_duration_us":375946,"encode_duration_us":18724,"operation_id":"0x94ac","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10007708},{"operation_name":"upsert","last_dispatch_duration_us":191654,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9894650,"total_server_duration_us":143822,"encode_duration_us":16930,"operation_id":"0x93e8","timeout_ms":10000,"last_server_duration_us":34116,"total_duration_us":10007496},{"operation_name":"upsert","last_dispatch_duration_us":397395,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9861573,"total_server_duration_us":1574272,"encode_duration_us":20276,"operation_id":"0x951c","timeout_ms":10000,"last_server_duration_us":101381,"total_duration_us":10007480},{"operation_name":"upsert","last_dispatch_duration_us":10061948,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":12088802,"total_server_duration_us":9552783,"encode_duration_us":20052,"operation_id":"0x94b7","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10006974},{"operation_name":"upsert","last_dispatch_duration_us":1112848,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10499523,"total_server_duration_us":877658,"encode_duration_us":25694,"operation_id":"0x94d9","timeout_ms":10000,"last_server_duration_us":23138,"total_duration_us":10006555},{"operation_name":"upsert","last_dispatch_duration_us":1394426,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":4425790,"total_server_duration_us":1019476,"encode_duration_us":16560,"operation_id":"0x9469","timeout_ms":10000,"last_server_duration_us":562233,"total_duration_us":10006332},{"operation_name":"upsert","last_dispatch_duration_us":1746326,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":11482208,"total_server_duration_us":1064617,"encode_duration_us":16414,"operation_id":"0x93f9","timeout_ms":10000,"last_server_duration_us":177689,"total_duration_us":10004989}],"total_count":14}} INFO [2022-01-10 05:49:02,640] jepsen worker 86 - jepsen.util 4136 :ok :read [28 45] INFO [2022-01-10 05:49:02,641] jepsen worker 86 - jepsen.util 4136 :invoke :read [28 nil] INFO [2022-01-10 05:49:02,659] jepsen worker 74 - jepsen.util 5384 :ok :read [24 18] INFO [2022-01-10 05:49:02,659] jepsen worker 74 - jepsen.util 5384 :invoke :write [24 14] INFO [2022-01-10 05:49:02,714] jepsen worker 42 - jepsen.util 4542 :ok :read [14 21] INFO [2022-01-10 05:49:02,715] jepsen worker 42 - jepsen.util 4542 :invoke :write [14 11] INFO [2022-01-10 05:49:02,740] jepsen worker 52 - jepsen.util 5992 :info :write [17 27] :RequestTimeoutException INFO [2022-01-10 05:49:02,740] jepsen worker 52 - jepsen.util 6082 :invoke :read [17 nil] INFO [2022-01-10 05:49:02,753] jepsen worker 75 - jepsen.util 4665 :ok :read [25 :nil] INFO [2022-01-10 05:49:02,754] jepsen worker 75 - jepsen.util 4665 :invoke :read [25 nil] INFO [2022-01-10 05:49:02,779] jepsen worker 25 - jepsen.util 4525 :ok :read [8 39] INFO [2022-01-10 05:49:02,780] jepsen worker 25 - jepsen.util 4525 :invoke :write [8 26] INFO [2022-01-10 05:49:02,792] jepsen worker 44 - jepsen.util 4904 :ok :read [14 21] INFO [2022-01-10 05:49:02,792] jepsen worker 44 - jepsen.util 4904 :invoke :read [14 nil] INFO [2022-01-10 05:49:02,796] jepsen worker 43 - jepsen.util 4723 :ok :read [14 21] INFO [2022-01-10 05:49:02,797] jepsen worker 43 - jepsen.util 4723 :invoke :write [14 45] INFO [2022-01-10 05:49:02,798] jepsen worker 88 - jepsen.util 4858 :ok :read [29 0] INFO [2022-01-10 05:49:02,799] jepsen worker 88 - jepsen.util 4858 :invoke :write [29 34] INFO [2022-01-10 05:49:02,831] jepsen worker 5 - jepsen.util 4865 :info :write [1 44] :RequestTimeoutException INFO [2022-01-10 05:49:02,832] jepsen worker 5 - jepsen.util 4955 :invoke :read [1 nil] INFO [2022-01-10 05:49:02,863] jepsen worker 55 - jepsen.util 5365 :ok :read [18 14] INFO [2022-01-10 05:49:02,863] jepsen worker 55 - jepsen.util 5365 :invoke :write [18 10] INFO [2022-01-10 05:49:03,002] jepsen worker 22 - jepsen.util 4522 :ok :read [7 :nil] INFO [2022-01-10 05:49:03,003] jepsen worker 22 - jepsen.util 4522 :invoke :read [7 nil] INFO [2022-01-10 05:49:03,226] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.672514619883041}, :ns_1@172.23.107.227 {:progress 0.469208211143695}, :ns_1@172.23.105.187 {:progress 0.469208211143695}, :ns_1@172.23.107.228 {:progress 0.6411764705882352}} INFO [2022-01-10 05:49:03,226] jepsen worker nemesis - couchbase.util Rebalance check count: 689 INFO [2022-01-10 05:49:03,226] jepsen worker nemesis - couchbase.util Rebalance stuck count: 572 INFO [2022-01-10 05:49:03,521] jepsen worker 75 - jepsen.util 4665 :ok :read [25 :nil] INFO [2022-01-10 05:49:03,521] jepsen worker 75 - jepsen.util 4665 :invoke :write [25 29] INFO [2022-01-10 05:49:03,533] jepsen worker 44 - jepsen.util 4904 :ok :read [14 21] INFO [2022-01-10 05:49:03,534] jepsen worker 44 - jepsen.util 4904 :invoke :read [14 nil] INFO [2022-01-10 05:49:03,586] jepsen worker 52 - jepsen.util 6082 :ok :read [17 41] INFO [2022-01-10 05:49:03,587] jepsen worker 52 - jepsen.util 6082 :invoke :write [17 40] INFO [2022-01-10 05:49:03,630] jepsen worker 59 - jepsen.util 4649 :ok :read [19 31] INFO [2022-01-10 05:49:03,631] jepsen worker 59 - jepsen.util 4649 :invoke :read [19 nil] INFO [2022-01-10 05:49:03,650] jepsen worker 15 - jepsen.util 4335 :info :write [5 3] :RequestTimeoutException INFO [2022-01-10 05:49:03,651] jepsen worker 15 - jepsen.util 4425 :invoke :write [5 24] INFO [2022-01-10 05:49:03,652] jepsen worker 39 - jepsen.util 5169 :ok :read [13 15] INFO [2022-01-10 05:49:03,653] jepsen worker 39 - jepsen.util 5169 :invoke :write [13 14] INFO [2022-01-10 05:49:03,655] jepsen worker 40 - jepsen.util 4450 :info :write [13 29] :RequestTimeoutException INFO [2022-01-10 05:49:03,656] jepsen worker 40 - jepsen.util 4540 :invoke :read [13 nil] INFO [2022-01-10 05:49:03,662] jepsen worker 22 - jepsen.util 4522 :ok :read [7 :nil] INFO [2022-01-10 05:49:03,664] jepsen worker 22 - jepsen.util 4522 :invoke :write [7 4] INFO [2022-01-10 05:49:03,683] jepsen worker 57 - jepsen.util 5637 :ok :read [19 31] INFO [2022-01-10 05:49:03,688] jepsen worker 57 - jepsen.util 5637 :invoke :read [19 nil] INFO [2022-01-10 05:49:03,695] jepsen worker 12 - jepsen.util 5412 :info :write [4 25] :RequestTimeoutException INFO [2022-01-10 05:49:03,696] jepsen worker 12 - jepsen.util 5502 :invoke :read [4 nil] INFO [2022-01-10 05:49:03,742] jepsen worker 65 - jepsen.util 4475 :info :write [21 21] :RequestTimeoutException INFO [2022-01-10 05:49:03,743] jepsen worker 65 - jepsen.util 4565 :invoke :read [21 nil] INFO [2022-01-10 05:49:03,770] jepsen worker 31 - jepsen.util 4981 :info :write [10 37] :RequestTimeoutException INFO [2022-01-10 05:49:03,771] jepsen worker 31 - jepsen.util 5071 :invoke :write [10 46] INFO [2022-01-10 05:49:03,773] jepsen worker 59 - jepsen.util 4649 :ok :read [19 31] INFO [2022-01-10 05:49:03,773] jepsen worker 59 - jepsen.util 4649 :invoke :write [19 48] INFO [2022-01-10 05:49:03,806] jepsen worker 53 - jepsen.util 5813 :info :write [17 28] :RequestTimeoutException INFO [2022-01-10 05:49:03,806] jepsen worker 53 - jepsen.util 5903 :invoke :write [17 18] INFO [2022-01-10 05:49:03,815] jepsen worker 89 - jepsen.util 5309 :info :write [29 2] :RequestTimeoutException INFO [2022-01-10 05:49:03,815] jepsen worker 89 - jepsen.util 5399 :invoke :read [29 nil] INFO [2022-01-10 05:49:03,849] jepsen worker 57 - jepsen.util 5637 :ok :read [19 31] INFO [2022-01-10 05:49:03,850] jepsen worker 57 - jepsen.util 5637 :invoke :read [19 nil] INFO [2022-01-10 05:49:03,880] jepsen worker 61 - jepsen.util 5101 :info :write [20 41] :RequestTimeoutException INFO [2022-01-10 05:49:03,881] jepsen worker 61 - jepsen.util 5191 :invoke :read [20 nil] INFO [2022-01-10 05:49:03,895] jepsen worker 12 - jepsen.util 5502 :ok :read [4 3] INFO [2022-01-10 05:49:03,896] jepsen worker 12 - jepsen.util 5502 :invoke :write [4 27] INFO [2022-01-10 05:49:04,004] jepsen worker 57 - jepsen.util 5637 :ok :read [19 31] INFO [2022-01-10 05:49:04,005] jepsen worker 57 - jepsen.util 5637 :invoke :read [19 nil] INFO [2022-01-10 05:49:04,032] jepsen worker 50 - jepsen.util 5000 :info :write [16 24] :RequestTimeoutException INFO [2022-01-10 05:49:04,032] jepsen worker 50 - jepsen.util 5090 :invoke :write [16 17] INFO [2022-01-10 05:49:04,196] jepsen worker 51 - jepsen.util 5631 :info :write [17 18] :RequestTimeoutException INFO [2022-01-10 05:49:04,197] jepsen worker 51 - jepsen.util 5721 :invoke :write [17 48] INFO [2022-01-10 05:49:04,221] jepsen worker 60 - jepsen.util 5190 :info :write [20 29] :RequestTimeoutException INFO [2022-01-10 05:49:04,222] jepsen worker 60 - jepsen.util 5280 :invoke :write [20 4] INFO [2022-01-10 05:49:04,232] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.672514619883041}, :ns_1@172.23.107.227 {:progress 0.469208211143695}, :ns_1@172.23.105.187 {:progress 0.469208211143695}, :ns_1@172.23.107.228 {:progress 0.6411764705882352}} INFO [2022-01-10 05:49:04,232] jepsen worker nemesis - couchbase.util Rebalance check count: 690 INFO [2022-01-10 05:49:04,232] jepsen worker nemesis - couchbase.util Rebalance stuck count: 573 INFO [2022-01-10 05:49:04,281] jepsen worker 4 - jepsen.util 5044 :info :write [1 43] :RequestTimeoutException INFO [2022-01-10 05:49:04,282] jepsen worker 4 - jepsen.util 5134 :invoke :read [1 nil] INFO [2022-01-10 05:49:04,359] jepsen worker 44 - jepsen.util 4904 :ok :read [14 21] INFO [2022-01-10 05:49:04,359] jepsen worker 44 - jepsen.util 4904 :invoke :read [14 nil] INFO [2022-01-10 05:49:04,375] jepsen worker 37 - jepsen.util 4537 :info :write [12 14] :RequestTimeoutException INFO [2022-01-10 05:49:04,377] jepsen worker 37 - jepsen.util 4627 :invoke :read [12 nil] INFO [2022-01-10 05:49:04,572] jepsen worker 56 - jepsen.util 5096 :info :write [18 3] :RequestTimeoutException INFO [2022-01-10 05:49:04,573] jepsen worker 56 - jepsen.util 5186 :invoke :read [18 nil] INFO [2022-01-10 05:49:04,584] jepsen worker 57 - jepsen.util 5637 :ok :read [19 31] INFO [2022-01-10 05:49:04,585] jepsen worker 57 - jepsen.util 5637 :invoke :read [19 nil] INFO [2022-01-10 05:49:04,755] jepsen worker 36 - jepsen.util 3726 :info :write [12 42] :RequestTimeoutException INFO [2022-01-10 05:49:04,756] jepsen worker 79 - jepsen.util 5659 :info :write [26 24] :RequestTimeoutException INFO [2022-01-10 05:49:04,756] jepsen worker 36 - jepsen.util 3816 :invoke :read [12 nil] INFO [2022-01-10 05:49:04,757] jepsen worker 79 - jepsen.util 5749 :invoke :write [26 23] INFO [2022-01-10 05:49:04,802] jepsen worker 57 - jepsen.util 5637 :ok :read [19 31] INFO [2022-01-10 05:49:04,803] jepsen worker 57 - jepsen.util 5637 :invoke :write [19 31] INFO [2022-01-10 05:49:05,075] jepsen worker 21 - jepsen.util 4791 :info :write [7 18] :RequestTimeoutException INFO [2022-01-10 05:49:05,076] jepsen worker 21 - jepsen.util 4881 :invoke :read [7 nil] INFO [2022-01-10 05:49:05,239] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6783625730994152}, :ns_1@172.23.107.227 {:progress 0.4721407624633431}, :ns_1@172.23.105.187 {:progress 0.4721407624633431}, :ns_1@172.23.107.228 {:progress 0.6470588235294117}} INFO [2022-01-10 05:49:05,239] jepsen worker nemesis - couchbase.util Rebalance check count: 691 INFO [2022-01-10 05:49:05,239] jepsen worker nemesis - couchbase.util Rebalance stuck count: 573 INFO [2022-01-10 05:49:05,256] jepsen worker 54 - jepsen.util 5094 :info :write [18 13] :RequestTimeoutException INFO [2022-01-10 05:49:05,256] jepsen worker 54 - jepsen.util 5184 :invoke :write [18 12] INFO [2022-01-10 05:49:05,301] jepsen worker 44 - jepsen.util 4904 :ok :read [14 21] INFO [2022-01-10 05:49:05,302] jepsen worker 44 - jepsen.util 4904 :invoke :write [14 10] INFO [2022-01-10 05:49:05,330] jepsen worker 63 - jepsen.util 4473 :info :write [21 5] :RequestTimeoutException INFO [2022-01-10 05:49:05,331] jepsen worker 63 - jepsen.util 4563 :invoke :read [21 nil] INFO [2022-01-10 05:49:05,362] jepsen worker 40 - jepsen.util 4540 :ok :read [13 15] INFO [2022-01-10 05:49:05,363] jepsen worker 40 - jepsen.util 4540 :invoke :read [13 nil] INFO [2022-01-10 05:49:05,387] jepsen worker 40 - jepsen.util 4540 :ok :read [13 15] INFO [2022-01-10 05:49:05,388] jepsen worker 40 - jepsen.util 4540 :invoke :write [13 25] INFO [2022-01-10 05:49:05,631] jepsen worker 2 - jepsen.util 5042 :info :write [0 14] :RequestTimeoutException INFO [2022-01-10 05:49:05,631] jepsen worker 2 - jepsen.util 5132 :invoke :write [0 16] INFO [2022-01-10 05:49:05,775] jepsen worker 29 - jepsen.util 4709 :info :write [9 33] :RequestTimeoutException INFO [2022-01-10 05:49:05,775] jepsen worker 29 - jepsen.util 4799 :invoke :write [9 18] INFO [2022-01-10 05:49:05,776] jepsen worker 28 - jepsen.util 4438 :info :write [9 35] :RequestTimeoutException INFO [2022-01-10 05:49:05,776] jepsen worker 28 - jepsen.util 4528 :invoke :read [9 nil] INFO [2022-01-10 05:49:05,839] jepsen worker 36 - jepsen.util 3816 :ok :read [12 14] INFO [2022-01-10 05:49:05,839] jepsen worker 36 - jepsen.util 3816 :invoke :read [12 nil] INFO [2022-01-10 05:49:05,978] jepsen worker 85 - jepsen.util 5215 :info :write [28 49] :SyncWriteAmbiguous INFO [2022-01-10 05:49:05,979] jepsen worker 85 - jepsen.util 5305 :invoke :read [28 nil] INFO [2022-01-10 05:49:06,013] jepsen worker 41 - jepsen.util 4811 :ok :read [13 15] INFO [2022-01-10 05:49:06,014] jepsen worker 41 - jepsen.util 4811 :invoke :read [13 nil] INFO [2022-01-10 05:49:06,042] jepsen worker 30 - jepsen.util 4530 :info :write [10 31] :RequestTimeoutException INFO [2022-01-10 05:49:06,043] jepsen worker 30 - jepsen.util 4620 :invoke :write [10 46] INFO [2022-01-10 05:49:06,052] jepsen worker 36 - jepsen.util 3816 :ok :read [12 14] INFO [2022-01-10 05:49:06,053] jepsen worker 36 - jepsen.util 3816 :invoke :write [12 40] INFO [2022-01-10 05:49:06,154] jepsen worker 16 - jepsen.util 4966 :ok :read [5 25] INFO [2022-01-10 05:49:06,154] jepsen worker 16 - jepsen.util 4966 :invoke :write [5 35] INFO [2022-01-10 05:49:06,243] jepsen worker 21 - jepsen.util 4881 :ok :read [7 :nil] INFO [2022-01-10 05:49:06,243] jepsen worker 21 - jepsen.util 4881 :invoke :write [7 0] INFO [2022-01-10 05:49:06,245] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6783625730994152}, :ns_1@172.23.107.227 {:progress 0.4721407624633431}, :ns_1@172.23.105.187 {:progress 0.4721407624633431}, :ns_1@172.23.107.228 {:progress 0.6470588235294117}} INFO [2022-01-10 05:49:06,245] jepsen worker nemesis - couchbase.util Rebalance check count: 692 INFO [2022-01-10 05:49:06,245] jepsen worker nemesis - couchbase.util Rebalance stuck count: 574 INFO [2022-01-10 05:49:06,366] jepsen worker 85 - jepsen.util 5305 :ok :read [28 45] INFO [2022-01-10 05:49:06,366] jepsen worker 85 - jepsen.util 5305 :invoke :write [28 23] INFO [2022-01-10 05:49:06,477] jepsen worker 0 - jepsen.util 4950 :info :write [0 43] :RequestTimeoutException INFO [2022-01-10 05:49:06,477] jepsen worker 0 - jepsen.util 5040 :invoke :write [0 36] INFO [2022-01-10 05:49:06,505] jepsen worker 87 - jepsen.util 4677 :info :write [29 39] :SyncWriteAmbiguous INFO [2022-01-10 05:49:06,506] jepsen worker 87 - jepsen.util 4767 :invoke :read [29 nil] INFO [2022-01-10 05:49:06,557] jepsen worker 49 - jepsen.util 4639 :info :write [16 26] :RequestTimeoutException INFO [2022-01-10 05:49:06,557] jepsen worker 49 - jepsen.util 4729 :invoke :write [16 1] INFO [2022-01-10 05:49:06,608] jepsen worker 41 - jepsen.util 4811 :ok :read [13 15] INFO [2022-01-10 05:49:06,609] jepsen worker 41 - jepsen.util 4811 :invoke :write [13 12] INFO [2022-01-10 05:49:06,625] jepsen worker 9 - jepsen.util 5409 :ok :read [3 3] INFO [2022-01-10 05:49:06,626] jepsen worker 9 - jepsen.util 5409 :invoke :write [3 31] INFO [2022-01-10 05:49:06,855] jepsen worker 76 - jepsen.util 4216 :info :write [25 44] :RequestTimeoutException INFO [2022-01-10 05:49:06,856] jepsen worker 76 - jepsen.util 4306 :invoke :read [25 nil] INFO [2022-01-10 05:49:07,126] jepsen worker 71 - jepsen.util 4481 :info :write [23 3] :RequestTimeoutException INFO [2022-01-10 05:49:07,127] jepsen worker 71 - jepsen.util 4571 :invoke :write [23 28] INFO [2022-01-10 05:49:07,211] jepsen worker 62 - jepsen.util 5192 :info :write [20 29] :RequestTimeoutException INFO [2022-01-10 05:49:07,211] jepsen worker 62 - jepsen.util 5282 :invoke :read [20 nil] INFO [2022-01-10 05:49:07,215] jepsen worker 32 - jepsen.util 5162 :info :write [10 26] :RequestTimeoutException INFO [2022-01-10 05:49:07,252] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6783625730994152}, :ns_1@172.23.107.227 {:progress 0.4721407624633431}, :ns_1@172.23.105.187 {:progress 0.4721407624633431}, :ns_1@172.23.107.228 {:progress 0.6470588235294117}} INFO [2022-01-10 05:49:07,284] jepsen worker 32 - jepsen.util 5252 :invoke :read [10 nil] INFO [2022-01-10 05:49:07,285] jepsen worker nemesis - couchbase.util Rebalance check count: 693 INFO [2022-01-10 05:49:07,285] jepsen worker nemesis - couchbase.util Rebalance stuck count: 575 INFO [2022-01-10 05:49:07,390] jepsen worker 18 - jepsen.util 4698 :info :write [6 3] :RequestTimeoutException INFO [2022-01-10 05:49:07,391] jepsen worker 18 - jepsen.util 4788 :invoke :read [6 nil] INFO [2022-01-10 05:49:07,402] jepsen worker 24 - jepsen.util 4434 :info :write [8 7] :RequestTimeoutException INFO [2022-01-10 05:49:07,402] jepsen worker 24 - jepsen.util 4524 :invoke :write [8 23] INFO [2022-01-10 05:49:07,481] jepsen worker 10 - jepsen.util 4600 :info :write [3 33] :RequestTimeoutException INFO [2022-01-10 05:49:07,482] jepsen worker 10 - jepsen.util 4690 :invoke :write [3 18] INFO [2022-01-10 05:49:07,482] jepsen worker 33 - jepsen.util 5163 :ok :read [11 21] INFO [2022-01-10 05:49:07,483] jepsen worker 33 - jepsen.util 5163 :invoke :read [11 nil] INFO [2022-01-10 05:49:07,778] jepsen worker 65 - jepsen.util 4565 :ok :read [21 7] INFO [2022-01-10 05:49:07,779] jepsen worker 65 - jepsen.util 4565 :invoke :write [21 15] INFO [2022-01-10 05:49:07,925] jepsen worker 20 - jepsen.util 4610 :ok :read [6 25] INFO [2022-01-10 05:49:07,926] jepsen worker 19 - jepsen.util 4969 :ok :read [6 25] INFO [2022-01-10 05:49:07,926] jepsen worker 20 - jepsen.util 4610 :invoke :read [6 nil] INFO [2022-01-10 05:49:07,926] jepsen worker 19 - jepsen.util 4969 :invoke :read [6 nil] INFO [2022-01-10 05:49:07,968] jepsen worker 63 - jepsen.util 4563 :ok :read [21 7] INFO [2022-01-10 05:49:07,969] jepsen worker 63 - jepsen.util 4563 :invoke :read [21 nil] INFO [2022-01-10 05:49:08,039] jepsen worker 8 - jepsen.util 5048 :ok :read [2 14] INFO [2022-01-10 05:49:08,039] jepsen worker 8 - jepsen.util 5048 :invoke :read [2 nil] INFO [2022-01-10 05:49:08,060] jepsen worker 63 - jepsen.util 4563 :ok :read [21 7] INFO [2022-01-10 05:49:08,061] jepsen worker 63 - jepsen.util 4563 :invoke :read [21 nil] INFO [2022-01-10 05:49:08,095] jepsen worker 11 - jepsen.util 4781 :info :write [3 16] :RequestTimeoutException INFO [2022-01-10 05:49:08,096] jepsen worker 11 - jepsen.util 4871 :invoke :read [3 nil] INFO [2022-01-10 05:49:08,119] jepsen worker 11 - jepsen.util 4871 :ok :read [3 3] INFO [2022-01-10 05:49:08,119] jepsen worker 11 - jepsen.util 4871 :invoke :read [3 nil] INFO [2022-01-10 05:49:08,140] jepsen worker 11 - jepsen.util 4871 :ok :read [3 3] INFO [2022-01-10 05:49:08,141] jepsen worker 11 - jepsen.util 4871 :invoke :write [3 27] INFO [2022-01-10 05:49:08,150] jepsen worker 61 - jepsen.util 5191 :ok :read [20 :nil] INFO [2022-01-10 05:49:08,151] jepsen worker 61 - jepsen.util 5191 :invoke :read [20 nil] INFO [2022-01-10 05:49:08,152] jepsen worker 17 - jepsen.util 4697 :info :write [5 26] :RequestTimeoutException INFO [2022-01-10 05:49:08,153] jepsen worker 17 - jepsen.util 4787 :invoke :write [5 43] INFO [2022-01-10 05:49:08,197] jepsen worker 8 - jepsen.util 5048 :ok :read [2 14] INFO [2022-01-10 05:49:08,198] jepsen worker 8 - jepsen.util 5048 :invoke :read [2 nil] INFO [2022-01-10 05:49:08,204] jepsen worker 63 - jepsen.util 4563 :ok :read [21 7] INFO [2022-01-10 05:49:08,204] jepsen worker 63 - jepsen.util 4563 :invoke :write [21 16] INFO [2022-01-10 05:49:08,231] jepsen worker 76 - jepsen.util 4306 :ok :read [25 :nil] INFO [2022-01-10 05:49:08,231] jepsen worker 18 - jepsen.util 4788 :ok :read [6 25] INFO [2022-01-10 05:49:08,231] jepsen worker 76 - jepsen.util 4306 :invoke :read [25 nil] INFO [2022-01-10 05:49:08,232] jepsen worker 18 - jepsen.util 4788 :invoke :write [6 15] INFO [2022-01-10 05:49:08,270] jepsen worker 8 - jepsen.util 5048 :ok :read [2 14] INFO [2022-01-10 05:49:08,270] jepsen worker 8 - jepsen.util 5048 :invoke :write [2 22] INFO [2022-01-10 05:49:08,275] jepsen worker 33 - jepsen.util 5163 :ok :read [11 21] INFO [2022-01-10 05:49:08,275] jepsen worker 33 - jepsen.util 5163 :invoke :write [11 45] INFO [2022-01-10 05:49:08,290] jepsen worker 84 - jepsen.util 4584 :info :write [28 16] :RequestTimeoutException INFO [2022-01-10 05:49:08,290] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6842105263157895}, :ns_1@172.23.107.227 {:progress 0.4750733137829912}, :ns_1@172.23.105.187 {:progress 0.4750733137829912}, :ns_1@172.23.107.228 {:progress 0.6529411764705882}} INFO [2022-01-10 05:49:08,290] jepsen worker nemesis - couchbase.util Rebalance check count: 694 INFO [2022-01-10 05:49:08,290] jepsen worker nemesis - couchbase.util Rebalance stuck count: 575 INFO [2022-01-10 05:49:08,291] jepsen worker 84 - jepsen.util 4674 :invoke :read [28 nil] INFO [2022-01-10 05:49:08,456] jepsen worker 66 - jepsen.util 4746 :info :write [22 10] :RequestTimeoutException INFO [2022-01-10 05:49:08,457] jepsen worker 66 - jepsen.util 4836 :invoke :write [22 5] INFO [2022-01-10 05:49:08,696] jepsen worker 77 - jepsen.util 5387 :info :write [25 45] :RequestTimeoutException INFO [2022-01-10 05:49:08,697] jepsen worker 77 - jepsen.util 5477 :invoke :read [25 nil] INFO [2022-01-10 05:49:08,748] jepsen worker 20 - jepsen.util 4610 :ok :read [6 25] INFO [2022-01-10 05:49:08,749] jepsen worker 20 - jepsen.util 4610 :invoke :read [6 nil] INFO [2022-01-10 05:49:08,771] jepsen worker 70 - jepsen.util 4930 :info :write [23 0] :RequestTimeoutException INFO [2022-01-10 05:49:08,772] jepsen worker 70 - jepsen.util 5020 :invoke :read [23 nil] INFO [2022-01-10 05:49:08,790] jepsen worker 6 - jepsen.util 4776 :info :write [2 39] :RequestTimeoutException INFO [2022-01-10 05:49:08,791] jepsen worker 6 - jepsen.util 4866 :invoke :read [2 nil] INFO [2022-01-10 05:49:08,921] jepsen worker 70 - jepsen.util 5020 :ok :read [23 26] INFO [2022-01-10 05:49:08,922] jepsen worker 70 - jepsen.util 5020 :invoke :read [23 nil] INFO [2022-01-10 05:49:09,068] jepsen worker 61 - jepsen.util 5191 :ok :read [20 :nil] INFO [2022-01-10 05:49:09,069] jepsen worker 61 - jepsen.util 5191 :invoke :write [20 12] INFO [2022-01-10 05:49:09,070] jepsen worker 19 - jepsen.util 4969 :ok :read [6 25] INFO [2022-01-10 05:49:09,070] jepsen worker 19 - jepsen.util 4969 :invoke :read [6 nil] INFO [2022-01-10 05:49:09,076] jepsen worker 70 - jepsen.util 5020 :ok :read [23 26] INFO [2022-01-10 05:49:09,077] jepsen worker 70 - jepsen.util 5020 :invoke :write [23 33] INFO [2022-01-10 05:49:09,176] jepsen worker 62 - jepsen.util 5282 :ok :read [20 :nil] INFO [2022-01-10 05:49:09,176] jepsen worker 62 - jepsen.util 5282 :invoke :read [20 nil] INFO [2022-01-10 05:49:09,217] jepsen worker 6 - jepsen.util 4866 :ok :read [2 14] INFO [2022-01-10 05:49:09,218] jepsen worker 6 - jepsen.util 4866 :invoke :read [2 nil] INFO [2022-01-10 05:49:09,305] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6842105263157895}, :ns_1@172.23.107.227 {:progress 0.4750733137829912}, :ns_1@172.23.105.187 {:progress 0.4750733137829912}, :ns_1@172.23.107.228 {:progress 0.6529411764705882}} INFO [2022-01-10 05:49:09,305] jepsen worker nemesis - couchbase.util Rebalance check count: 695 INFO [2022-01-10 05:49:09,305] jepsen worker nemesis - couchbase.util Rebalance stuck count: 576 INFO [2022-01-10 05:49:09,331] jepsen worker 47 - jepsen.util 4997 :info :write [15 27] :RequestTimeoutException INFO [2022-01-10 05:49:09,331] jepsen worker 47 - jepsen.util 5087 :invoke :read [15 nil] INFO [2022-01-10 05:49:09,434] jepsen worker 69 - jepsen.util 4479 :info :write [23 1] :RequestTimeoutException INFO [2022-01-10 05:49:09,435] jepsen worker 69 - jepsen.util 4569 :invoke :read [23 nil] INFO [2022-01-10 05:49:09,441] jepsen worker 35 - jepsen.util 4625 :info :write [11 14] :RequestTimeoutException INFO [2022-01-10 05:49:09,442] jepsen worker 35 - jepsen.util 4715 :invoke :write [11 38] INFO [2022-01-10 05:49:09,457] jepsen worker 48 - jepsen.util 4728 :info :write [16 27] :SyncWriteAmbiguous INFO [2022-01-10 05:49:09,458] jepsen worker 48 - jepsen.util 4818 :invoke :write [16 44] INFO [2022-01-10 05:49:09,465] jepsen worker 45 - jepsen.util 5265 :info :write [15 11] :RequestTimeoutException INFO [2022-01-10 05:49:09,466] jepsen worker 45 - jepsen.util 5355 :invoke :write [15 22] INFO [2022-01-10 05:49:09,620] jepsen worker 1 - jepsen.util 4951 :ok :read [0 46] INFO [2022-01-10 05:49:09,620] jepsen worker 1 - jepsen.util 4951 :invoke :read [0 nil] INFO [2022-01-10 05:49:09,686] jepsen worker 26 - jepsen.util 4706 :info :write [8 37] :RequestTimeoutException INFO [2022-01-10 05:49:09,687] jepsen worker 26 - jepsen.util 4796 :invoke :read [8 nil] INFO [2022-01-10 05:49:09,703] jepsen worker 62 - jepsen.util 5282 :ok :read [20 :nil] INFO [2022-01-10 05:49:09,703] jepsen worker 62 - jepsen.util 5282 :invoke :write [20 35] INFO [2022-01-10 05:49:09,720] jepsen worker 23 - jepsen.util 4973 :info :write [7 49] :RequestTimeoutException INFO [2022-01-10 05:49:09,721] jepsen worker 23 - jepsen.util 5063 :invoke :write [7 9] INFO [2022-01-10 05:49:09,766] jepsen worker 19 - jepsen.util 4969 :ok :read [6 25] INFO [2022-01-10 05:49:09,766] jepsen worker 19 - jepsen.util 4969 :invoke :write [6 26] INFO [2022-01-10 05:49:09,876] jepsen worker 20 - jepsen.util 4610 :ok :read [6 25] INFO [2022-01-10 05:49:09,877] jepsen worker 20 - jepsen.util 4610 :invoke :read [6 nil] INFO [2022-01-10 05:49:09,883] jepsen worker 47 - jepsen.util 5087 :ok :read [15 23] INFO [2022-01-10 05:49:09,883] jepsen worker 47 - jepsen.util 5087 :invoke :read [15 nil] INFO [2022-01-10 05:49:10,181] jepsen worker 6 - jepsen.util 4866 :ok :read [2 14] INFO [2022-01-10 05:49:10,181] jepsen worker 6 - jepsen.util 4866 :invoke :read [2 nil] INFO [2022-01-10 05:49:10,282] jepsen worker 47 - jepsen.util 5087 :ok :read [15 23] INFO [2022-01-10 05:49:10,282] jepsen worker 47 - jepsen.util 5087 :invoke :write [15 6] INFO [2022-01-10 05:49:10,315] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6900584795321638}, :ns_1@172.23.107.227 {:progress 0.4780058651026393}, :ns_1@172.23.105.187 {:progress 0.4780058651026393}, :ns_1@172.23.107.228 {:progress 0.6588235294117647}} INFO [2022-01-10 05:49:10,315] jepsen worker nemesis - couchbase.util Rebalance check count: 696 INFO [2022-01-10 05:49:10,315] jepsen worker nemesis - couchbase.util Rebalance stuck count: 576 INFO [2022-01-10 05:49:10,357] jepsen worker 28 - jepsen.util 4528 :ok :read [9 22] INFO [2022-01-10 05:49:10,358] jepsen worker 28 - jepsen.util 4528 :invoke :write [9 16] INFO [2022-01-10 05:49:10,371] jepsen worker 7 - jepsen.util 5227 :info :write [2 42] :RequestTimeoutException INFO [2022-01-10 05:49:10,372] jepsen worker 7 - jepsen.util 5317 :invoke :write [2 39] INFO [2022-01-10 05:49:10,446] jepsen worker 1 - jepsen.util 4951 :ok :read [0 46] INFO [2022-01-10 05:49:10,446] jepsen worker 1 - jepsen.util 4951 :invoke :write [0 12] INFO [2022-01-10 05:49:10,495] jepsen worker 38 - jepsen.util 4538 :info :write [12 48] :RequestTimeoutException INFO [2022-01-10 05:49:10,496] jepsen worker 38 - jepsen.util 4628 :invoke :read [12 nil] INFO [2022-01-10 05:49:10,523] jepsen worker 26 - jepsen.util 4796 :ok :read [8 39] INFO [2022-01-10 05:49:10,524] jepsen worker 26 - jepsen.util 4796 :invoke :write [8 38] INFO [2022-01-10 05:49:10,651] jepsen worker 20 - jepsen.util 4610 :ok :read [6 25] INFO [2022-01-10 05:49:10,651] jepsen worker 20 - jepsen.util 4610 :invoke :write [6 45] INFO [2022-01-10 05:49:10,681] jepsen worker 72 - jepsen.util 5202 :info :write [24 22] :RequestTimeoutException INFO [2022-01-10 05:49:10,682] jepsen worker 72 - jepsen.util 5292 :invoke :read [24 nil] INFO [2022-01-10 05:49:10,696] jepsen worker 78 - jepsen.util 6018 :info :write [26 17] :SyncWriteAmbiguous INFO [2022-01-10 05:49:10,696] jepsen worker 78 - jepsen.util 6108 :invoke :write [26 1] INFO [2022-01-10 05:49:10,796] jepsen worker 6 - jepsen.util 4866 :ok :read [2 14] INFO [2022-01-10 05:49:10,796] jepsen worker 6 - jepsen.util 4866 :invoke :write [2 39] INFO [2022-01-10 05:49:10,816] jepsen worker 80 - jepsen.util 6020 :info :write [26 28] :RequestTimeoutException INFO [2022-01-10 05:49:10,817] jepsen worker 80 - jepsen.util 6110 :invoke :read [26 nil] WARN [2022-01-10 05:49:10,823] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9426626,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10050730,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9419716,"encode_duration_us":17585,"operation_id":"0x949d","timeout_ms":10000,"last_server_duration_us":9310807,"total_duration_us":10006233},{"operation_name":"upsert","last_dispatch_duration_us":17524664,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17524664,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9032060,"encode_duration_us":23291,"operation_id":"0x93f3","timeout_ms":10000,"last_server_duration_us":9032060,"total_duration_us":10010884},{"operation_name":"upsert","last_dispatch_duration_us":9401927,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13245173,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9643716,"encode_duration_us":22744,"operation_id":"0x9513","timeout_ms":10000,"last_server_duration_us":9281805,"total_duration_us":10010497}],"total_count":3}} WARN [2022-01-10 05:49:10,823] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":986094,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":7052545,"total_server_duration_us":1488801,"encode_duration_us":26573,"operation_id":"0x9588","timeout_ms":10000,"last_server_duration_us":88098,"total_duration_us":10010528},{"operation_name":"upsert","last_dispatch_duration_us":9401927,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":13245173,"total_server_duration_us":9643716,"encode_duration_us":22744,"operation_id":"0x9513","timeout_ms":10000,"last_server_duration_us":9281805,"total_duration_us":10010497},{"operation_name":"upsert","last_dispatch_duration_us":7616136,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11839427,"total_server_duration_us":793335,"encode_duration_us":18057,"operation_id":"0x9512","timeout_ms":10000,"last_server_duration_us":34911,"total_duration_us":10009922},{"operation_name":"upsert","last_dispatch_duration_us":7405280,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9826703,"total_server_duration_us":1442024,"encode_duration_us":25695,"operation_id":"0x963a","timeout_ms":10000,"last_server_duration_us":57880,"total_duration_us":10009256},{"operation_name":"upsert","last_dispatch_duration_us":715882,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":1768552,"total_server_duration_us":189156,"encode_duration_us":17042,"operation_id":"0x96a8","timeout_ms":10000,"last_server_duration_us":77992,"total_duration_us":10008692},{"operation_name":"upsert","last_dispatch_duration_us":369594,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8468227,"total_server_duration_us":642257,"encode_duration_us":16568,"operation_id":"0x96aa","timeout_ms":10000,"last_server_duration_us":161260,"total_duration_us":10007680},{"operation_name":"upsert","last_dispatch_duration_us":5637260,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":5637260,"total_server_duration_us":19418,"encode_duration_us":17375,"operation_id":"0x9644","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10007102},{"operation_name":"upsert","last_dispatch_duration_us":39050,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":4629322,"total_server_duration_us":545113,"encode_duration_us":22302,"operation_id":"0x951f","timeout_ms":10000,"last_server_duration_us":24669,"total_duration_us":10006344},{"operation_name":"upsert","last_dispatch_duration_us":7450447,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8678966,"total_server_duration_us":345853,"encode_duration_us":16370,"operation_id":"0x9602","timeout_ms":10000,"last_server_duration_us":28218,"total_duration_us":10005220},{"operation_name":"upsert","last_dispatch_duration_us":7616690,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":11748274,"total_server_duration_us":1191344,"encode_duration_us":24975,"operation_id":"0x9514","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10004914}],"total_count":21}} INFO [2022-01-10 05:49:10,862] jepsen worker 38 - jepsen.util 4628 :ok :read [12 14] INFO [2022-01-10 05:49:10,863] jepsen worker 38 - jepsen.util 4628 :invoke :read [12 nil] INFO [2022-01-10 05:49:10,897] jepsen worker 5 - jepsen.util 4955 :ok :read [1 49] INFO [2022-01-10 05:49:10,898] jepsen worker 5 - jepsen.util 4955 :invoke :write [1 11] INFO [2022-01-10 05:49:10,911] jepsen worker 38 - jepsen.util 4628 :ok :read [12 14] INFO [2022-01-10 05:49:10,912] jepsen worker 38 - jepsen.util 4628 :invoke :write [12 36] INFO [2022-01-10 05:49:11,026] jepsen worker 13 - jepsen.util 5683 :info :write [4 46] :RequestTimeoutException INFO [2022-01-10 05:49:11,027] jepsen worker 13 - jepsen.util 5773 :invoke :write [4 9] INFO [2022-01-10 05:49:11,039] jepsen worker 4 - jepsen.util 5134 :ok :read [1 49] INFO [2022-01-10 05:49:11,039] jepsen worker 4 - jepsen.util 5134 :invoke :read [1 nil] INFO [2022-01-10 05:49:11,045] jepsen worker 67 - jepsen.util 4477 :info :write [22 46] :RequestTimeoutException INFO [2022-01-10 05:49:11,046] jepsen worker 67 - jepsen.util 4567 :invoke :write [22 24] INFO [2022-01-10 05:49:11,105] jepsen worker 34 - jepsen.util 4714 :ok :read [11 21] INFO [2022-01-10 05:49:11,105] jepsen worker 34 - jepsen.util 4714 :invoke :write [11 13] INFO [2022-01-10 05:49:11,259] jepsen worker 4 - jepsen.util 5134 :ok :read [1 49] INFO [2022-01-10 05:49:11,259] jepsen worker 4 - jepsen.util 5134 :invoke :write [1 44] INFO [2022-01-10 05:49:11,320] jepsen worker 14 - jepsen.util 5864 :info :write [4 26] :RequestTimeoutException INFO [2022-01-10 05:49:11,320] jepsen worker 14 - jepsen.util 5954 :invoke :read [4 nil] INFO [2022-01-10 05:49:11,321] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6900584795321638}, :ns_1@172.23.107.227 {:progress 0.4780058651026393}, :ns_1@172.23.105.187 {:progress 0.4780058651026393}, :ns_1@172.23.107.228 {:progress 0.6588235294117647}} INFO [2022-01-10 05:49:11,321] jepsen worker nemesis - couchbase.util Rebalance check count: 697 INFO [2022-01-10 05:49:11,322] jepsen worker nemesis - couchbase.util Rebalance stuck count: 577 INFO [2022-01-10 05:49:11,366] jepsen worker 46 - jepsen.util 4996 :info :write [15 31] :RequestTimeoutException INFO [2022-01-10 05:49:11,366] jepsen worker 46 - jepsen.util 5086 :invoke :read [15 nil] INFO [2022-01-10 05:49:11,432] jepsen worker 3 - jepsen.util 4863 :info :write [1 39] :RequestTimeoutException INFO [2022-01-10 05:49:11,433] jepsen worker 3 - jepsen.util 4953 :invoke :write [1 10] INFO [2022-01-10 05:49:11,615] jepsen worker 77 - jepsen.util 5477 :ok :read [25 :nil] INFO [2022-01-10 05:49:11,615] jepsen worker 77 - jepsen.util 5477 :invoke :read [25 nil] INFO [2022-01-10 05:49:11,630] jepsen worker 72 - jepsen.util 5292 :ok :read [24 18] INFO [2022-01-10 05:49:11,630] jepsen worker 72 - jepsen.util 5292 :invoke :read [24 nil] INFO [2022-01-10 05:49:11,650] jepsen worker 83 - jepsen.util 5573 :ok :read [27 30] INFO [2022-01-10 05:49:11,650] jepsen worker 83 - jepsen.util 5573 :invoke :write [27 25] INFO [2022-01-10 05:49:11,655] jepsen worker 82 - jepsen.util 5032 :ok :read [27 30] INFO [2022-01-10 05:49:11,655] jepsen worker 82 - jepsen.util 5032 :invoke :read [27 nil] INFO [2022-01-10 05:49:11,692] jepsen worker 86 - jepsen.util 4136 :ok :read [28 45] INFO [2022-01-10 05:49:11,693] jepsen worker 86 - jepsen.util 4136 :invoke :read [28 nil] INFO [2022-01-10 05:49:11,823] jepsen worker 80 - jepsen.util 6110 :ok :read [26 10] INFO [2022-01-10 05:49:11,824] jepsen worker 80 - jepsen.util 6110 :invoke :read [26 nil] INFO [2022-01-10 05:49:11,827] jepsen worker 14 - jepsen.util 5954 :ok :read [4 3] INFO [2022-01-10 05:49:11,828] jepsen worker 14 - jepsen.util 5954 :invoke :write [4 2] INFO [2022-01-10 05:49:11,846] jepsen worker 64 - jepsen.util 4924 :info :write [21 24] :RequestTimeoutException INFO [2022-01-10 05:49:11,847] jepsen worker 64 - jepsen.util 5014 :invoke :read [21 nil] INFO [2022-01-10 05:49:11,856] jepsen worker 69 - jepsen.util 4569 :fail :read [23 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:49:11,857] jepsen worker 69 - jepsen.util 4569 :invoke :write [23 46] INFO [2022-01-10 05:49:11,865] jepsen worker 37 - jepsen.util 4627 :ok :read [12 14] INFO [2022-01-10 05:49:11,865] jepsen worker 37 - jepsen.util 4627 :invoke :read [12 nil] INFO [2022-01-10 05:49:11,928] jepsen worker 77 - jepsen.util 5477 :ok :read [25 :nil] INFO [2022-01-10 05:49:11,929] jepsen worker 77 - jepsen.util 5477 :invoke :write [25 23] INFO [2022-01-10 05:49:11,949] jepsen worker 72 - jepsen.util 5292 :ok :read [24 18] INFO [2022-01-10 05:49:11,951] jepsen worker 72 - jepsen.util 5292 :invoke :write [24 47] INFO [2022-01-10 05:49:11,954] jepsen worker 46 - jepsen.util 5086 :ok :read [15 23] INFO [2022-01-10 05:49:11,955] jepsen worker 46 - jepsen.util 5086 :invoke :read [15 nil] INFO [2022-01-10 05:49:11,998] jepsen worker 37 - jepsen.util 4627 :ok :read [12 14] INFO [2022-01-10 05:49:11,999] jepsen worker 37 - jepsen.util 4627 :invoke :read [12 nil] INFO [2022-01-10 05:49:12,105] jepsen worker 37 - jepsen.util 4627 :ok :read [12 14] INFO [2022-01-10 05:49:12,105] jepsen worker 80 - jepsen.util 6110 :ok :read [26 10] INFO [2022-01-10 05:49:12,106] jepsen worker 80 - jepsen.util 6110 :invoke :read [26 nil] INFO [2022-01-10 05:49:12,106] jepsen worker 37 - jepsen.util 4627 :invoke :read [12 nil] INFO [2022-01-10 05:49:12,132] jepsen worker 56 - jepsen.util 5186 :ok :read [18 14] INFO [2022-01-10 05:49:12,132] jepsen worker 56 - jepsen.util 5186 :invoke :read [18 nil] INFO [2022-01-10 05:49:12,136] jepsen worker 87 - jepsen.util 4767 :ok :read [29 0] INFO [2022-01-10 05:49:12,136] jepsen worker 87 - jepsen.util 4767 :invoke :read [29 nil] INFO [2022-01-10 05:49:12,152] jepsen worker 46 - jepsen.util 5086 :ok :read [15 23] INFO [2022-01-10 05:49:12,153] jepsen worker 46 - jepsen.util 5086 :invoke :write [15 19] INFO [2022-01-10 05:49:12,158] jepsen worker 56 - jepsen.util 5186 :ok :read [18 14] INFO [2022-01-10 05:49:12,159] jepsen worker 56 - jepsen.util 5186 :invoke :write [18 24] INFO [2022-01-10 05:49:12,164] jepsen worker 87 - jepsen.util 4767 :ok :read [29 0] INFO [2022-01-10 05:49:12,164] jepsen worker 87 - jepsen.util 4767 :invoke :read [29 nil] INFO [2022-01-10 05:49:12,187] jepsen worker 73 - jepsen.util 4753 :info :write [24 30] :RequestTimeoutException INFO [2022-01-10 05:49:12,187] jepsen worker 73 - jepsen.util 4843 :invoke :read [24 nil] INFO [2022-01-10 05:49:12,192] jepsen worker 87 - jepsen.util 4767 :ok :read [29 0] INFO [2022-01-10 05:49:12,192] jepsen worker 27 - jepsen.util 5247 :info :write [9 1] :RequestTimeoutException INFO [2022-01-10 05:49:12,193] jepsen worker 27 - jepsen.util 5337 :invoke :write [9 10] INFO [2022-01-10 05:49:12,193] jepsen worker 87 - jepsen.util 4767 :invoke :write [29 7] WARN [2022-01-10 05:49:12,236] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10276407,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10276407,"total_server_duration_us":8502,"operation_id":"0x9707","timeout_ms":30000,"last_server_duration_us":8502,"total_duration_us":10278260},{"operation_name":"get","last_dispatch_duration_us":9045449,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":9045449,"total_server_duration_us":122177,"operation_id":"0x965f","timeout_ms":30000,"last_server_duration_us":122177,"total_duration_us":10131152},{"operation_name":"upsert","last_dispatch_duration_us":829382,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":829382,"total_server_duration_us":199588,"encode_duration_us":16502,"operation_id":"0x96a9","timeout_ms":10000,"last_server_duration_us":199588,"total_duration_us":10010769},{"operation_name":"upsert","last_dispatch_duration_us":15992648,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":15992648,"total_server_duration_us":9028877,"encode_duration_us":16702,"operation_id":"0x95ee","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10010674},{"operation_name":"upsert","last_dispatch_duration_us":261526,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":1347113,"total_server_duration_us":141113,"encode_duration_us":19152,"operation_id":"0x96d4","timeout_ms":10000,"last_server_duration_us":60002,"total_duration_us":10010033},{"operation_name":"upsert","last_dispatch_duration_us":8802041,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":14932194,"total_server_duration_us":1740341,"encode_duration_us":16964,"operation_id":"0x95ba","timeout_ms":10000,"last_server_duration_us":121667,"total_duration_us":10007095},{"operation_name":"upsert","last_dispatch_duration_us":10122633,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":13456782,"total_server_duration_us":9554818,"encode_duration_us":18098,"operation_id":"0x95b9","timeout_ms":10000,"last_server_duration_us":9210001,"total_duration_us":10006430},{"operation_name":"upsert","last_dispatch_duration_us":1277306,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":2138885,"total_server_duration_us":42107,"encode_duration_us":18876,"operation_id":"0x9645","timeout_ms":10000,"last_server_duration_us":22141,"total_duration_us":10001397},{"operation_name":"upsert","last_dispatch_duration_us":10023502,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10023502,"total_server_duration_us":9808630,"encode_duration_us":16214,"operation_id":"0x9557","timeout_ms":10000,"last_server_duration_us":9808630,"total_duration_us":10001392},{"operation_name":"upsert","last_dispatch_duration_us":775985,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":10291543,"total_server_duration_us":200533,"encode_duration_us":16892,"operation_id":"0x96a7","timeout_ms":10000,"last_server_duration_us":78696,"total_duration_us":10001067}],"total_count":21}} INFO [2022-01-10 05:49:12,239] jepsen worker 80 - jepsen.util 6110 :ok :read [26 10] INFO [2022-01-10 05:49:12,249] jepsen worker 37 - jepsen.util 4627 :ok :read [12 14] INFO [2022-01-10 05:49:12,263] jepsen worker 80 - jepsen.util 6110 :invoke :read [26 nil] INFO [2022-01-10 05:49:12,263] jepsen worker 37 - jepsen.util 4627 :invoke :read [12 nil] WARN [2022-01-10 05:49:12,326] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9589958,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11284186,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9657662,"encode_duration_us":25077,"operation_id":"0x9587","timeout_ms":10000,"last_server_duration_us":9553993,"total_duration_us":10000820},{"operation_name":"upsert","last_dispatch_duration_us":9792979,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14329816,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9356387,"encode_duration_us":19149,"operation_id":"0x950e","timeout_ms":10000,"last_server_duration_us":9201443,"total_duration_us":10005040},{"operation_name":"upsert","last_dispatch_duration_us":9045985,"last_local_id":"EFE9906200000002/00000000A7E0A91C","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10623698,"last_local_socket":"172.23.104.79:34084","total_server_duration_us":9455089,"encode_duration_us":16478,"operation_id":"0x95ea","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10002849}],"total_count":3}} INFO [2022-01-10 05:49:12,333] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.6900584795321638}, :ns_1@172.23.107.227 {:progress 0.4780058651026393}, :ns_1@172.23.105.187 {:progress 0.4780058651026393}, :ns_1@172.23.107.228 {:progress 0.6588235294117647}} INFO [2022-01-10 05:49:12,333] jepsen worker nemesis - couchbase.util Rebalance check count: 698 INFO [2022-01-10 05:49:12,333] jepsen worker nemesis - couchbase.util Rebalance stuck count: 578 WARN [2022-01-10 05:49:12,363] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10023502,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10023502,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9808630,"encode_duration_us":16214,"operation_id":"0x9557","timeout_ms":10000,"last_server_duration_us":9808630,"total_duration_us":10001392},{"operation_name":"upsert","last_dispatch_duration_us":10122633,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13456782,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":9554818,"encode_duration_us":18098,"operation_id":"0x95b9","timeout_ms":10000,"last_server_duration_us":9210001,"total_duration_us":10006430},{"operation_name":"upsert","last_dispatch_duration_us":15992648,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15992648,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9028877,"encode_duration_us":16702,"operation_id":"0x95ee","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10010674}],"total_count":3}} INFO [2022-01-10 05:49:12,368] jepsen worker 37 - jepsen.util 4627 :ok :read [12 14] INFO [2022-01-10 05:49:12,369] jepsen worker 37 - jepsen.util 4627 :invoke :read [12 nil] INFO [2022-01-10 05:49:12,370] jepsen worker 80 - jepsen.util 6110 :ok :read [26 10] INFO [2022-01-10 05:49:12,371] jepsen worker 80 - jepsen.util 6110 :invoke :read [26 nil] WARN [2022-01-10 05:49:12,424] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":15602496,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":15602496,"total_server_duration_us":18341,"operation_id":"0x9447","timeout_ms":30000,"last_server_duration_us":18341,"total_duration_us":15604426},{"operation_name":"get","last_dispatch_duration_us":14255698,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":14255698,"total_server_duration_us":62412,"operation_id":"0x9488","timeout_ms":30000,"last_server_duration_us":62412,"total_duration_us":14255992},{"operation_name":"get","last_dispatch_duration_us":10636245,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10636245,"total_server_duration_us":27,"operation_id":"0x954f","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":10640064},{"operation_name":"get","last_dispatch_duration_us":10104000,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10104000,"total_server_duration_us":11,"operation_id":"0x955a","timeout_ms":30000,"last_server_duration_us":11,"total_duration_us":10106253},{"operation_name":"upsert","last_dispatch_duration_us":1397342,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":6479240,"total_server_duration_us":3469892,"encode_duration_us":17726,"operation_id":"0x9709","timeout_ms":10000,"last_server_duration_us":77290,"total_duration_us":10010635},{"operation_name":"upsert","last_dispatch_duration_us":613579,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":6338120,"total_server_duration_us":2070794,"encode_duration_us":17146,"operation_id":"0x9691","timeout_ms":10000,"last_server_duration_us":80540,"total_duration_us":10010275},{"operation_name":"upsert","last_dispatch_duration_us":754950,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10019912,"total_server_duration_us":314828,"encode_duration_us":30665,"operation_id":"0x9597","timeout_ms":10000,"last_server_duration_us":97947,"total_duration_us":10009945},{"operation_name":"upsert","last_dispatch_duration_us":35515,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9403448,"total_server_duration_us":236712,"encode_duration_us":17225,"operation_id":"0x964a","timeout_ms":10000,"last_server_duration_us":22887,"total_duration_us":10009733},{"operation_name":"upsert","last_dispatch_duration_us":7366101,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":7366101,"total_server_duration_us":19966,"encode_duration_us":16730,"operation_id":"0x9734","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10009212},{"operation_name":"upsert","last_dispatch_duration_us":10044141,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":10044141,"total_server_duration_us":8718214,"encode_duration_us":16801,"operation_id":"0x9610","timeout_ms":10000,"last_server_duration_us":8718214,"total_duration_us":10007388}],"total_count":30}} WARN [2022-01-10 05:49:12,424] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":315845,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10024705,"total_server_duration_us":180839,"encode_duration_us":16272,"operation_id":"0x9619","timeout_ms":10000,"last_server_duration_us":38899,"total_duration_us":10009099},{"operation_name":"upsert","last_dispatch_duration_us":772189,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9511984,"total_server_duration_us":1453138,"encode_duration_us":16623,"operation_id":"0x95d2","timeout_ms":10000,"last_server_duration_us":53135,"total_duration_us":10006792},{"operation_name":"upsert","last_dispatch_duration_us":610829,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9585308,"total_server_duration_us":1196710,"encode_duration_us":17561,"operation_id":"0x95b7","timeout_ms":10000,"last_server_duration_us":88098,"total_duration_us":10006289},{"operation_name":"upsert","last_dispatch_duration_us":1387376,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":8028585,"total_server_duration_us":1194893,"encode_duration_us":16469,"operation_id":"0x96fc","timeout_ms":10000,"last_server_duration_us":108237,"total_duration_us":10005048},{"operation_name":"upsert","last_dispatch_duration_us":408223,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9112419,"total_server_duration_us":2693504,"encode_duration_us":16640,"operation_id":"0x963c","timeout_ms":10000,"last_server_duration_us":93195,"total_duration_us":10004938},{"operation_name":"upsert","last_dispatch_duration_us":394769,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10278141,"total_server_duration_us":169478,"encode_duration_us":17956,"operation_id":"0x961a","timeout_ms":10000,"last_server_duration_us":44389,"total_duration_us":10004820},{"operation_name":"upsert","last_dispatch_duration_us":7877220,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":7877220,"total_server_duration_us":18646,"encode_duration_us":16316,"operation_id":"0x9678","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10004268},{"operation_name":"upsert","last_dispatch_duration_us":863323,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7412116,"total_server_duration_us":1060211,"encode_duration_us":16602,"operation_id":"0x9735","timeout_ms":10000,"last_server_duration_us":49951,"total_duration_us":10003143},{"operation_name":"upsert","last_dispatch_duration_us":920447,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":5507925,"total_server_duration_us":844862,"encode_duration_us":16623,"operation_id":"0x9708","timeout_ms":10000,"last_server_duration_us":162219,"total_duration_us":10002830},{"operation_name":"upsert","last_dispatch_duration_us":647588,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6711944,"total_server_duration_us":1686288,"encode_duration_us":17263,"operation_id":"0x972a","timeout_ms":10000,"last_server_duration_us":102011,"total_duration_us":10002162}],"total_count":17}} WARN [2022-01-10 05:49:12,426] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1041884,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10986515,"total_server_duration_us":154585,"encode_duration_us":18232,"operation_id":"0x967c","timeout_ms":10000,"last_server_duration_us":27492,"total_duration_us":10011055},{"operation_name":"upsert","last_dispatch_duration_us":8979904,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12652033,"total_server_duration_us":207627,"encode_duration_us":16535,"operation_id":"0x9555","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10009777},{"operation_name":"upsert","last_dispatch_duration_us":826157,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":3640756,"total_server_duration_us":175865,"encode_duration_us":16297,"operation_id":"0x9559","timeout_ms":10000,"last_server_duration_us":44944,"total_duration_us":10007747},{"operation_name":"upsert","last_dispatch_duration_us":9374507,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11116860,"total_server_duration_us":76366,"encode_duration_us":20354,"operation_id":"0x9554","timeout_ms":10000,"last_server_duration_us":18570,"total_duration_us":10006832},{"operation_name":"upsert","last_dispatch_duration_us":412807,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9133267,"total_server_duration_us":1948476,"encode_duration_us":26564,"operation_id":"0x95bb","timeout_ms":10000,"last_server_duration_us":149201,"total_duration_us":10006252},{"operation_name":"upsert","last_dispatch_duration_us":892039,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":7369288,"total_server_duration_us":1717269,"encode_duration_us":16907,"operation_id":"0x96f7","timeout_ms":10000,"last_server_duration_us":72045,"total_duration_us":10003452},{"operation_name":"upsert","last_dispatch_duration_us":1684180,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":2461655,"total_server_duration_us":1475889,"encode_duration_us":16375,"operation_id":"0x96fe","timeout_ms":10000,"last_server_duration_us":1387069,"total_duration_us":10003280},{"operation_name":"upsert","last_dispatch_duration_us":9045985,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":10623698,"total_server_duration_us":9455089,"encode_duration_us":16478,"operation_id":"0x95ea","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10002849},{"operation_name":"upsert","last_dispatch_duration_us":514720,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9375593,"total_server_duration_us":117930,"encode_duration_us":16344,"operation_id":"0x9603","timeout_ms":10000,"last_server_duration_us":97329,"total_duration_us":10002331},{"operation_name":"upsert","last_dispatch_duration_us":408574,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":8893082,"total_server_duration_us":1391579,"encode_duration_us":27716,"operation_id":"0x95a5","timeout_ms":10000,"last_server_duration_us":34215,"total_duration_us":10002168}],"total_count":27}} WARN [2022-01-10 05:49:12,433] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9709836,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11305285,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9086048,"encode_duration_us":18462,"operation_id":"0x9550","timeout_ms":10000,"last_server_duration_us":9033121,"total_duration_us":10001330},{"operation_name":"upsert","last_dispatch_duration_us":18501687,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":22927478,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":10201676,"encode_duration_us":16560,"operation_id":"0x9469","timeout_ms":10000,"last_server_duration_us":9182200,"total_duration_us":10006332}],"total_count":2}} WARN [2022-01-10 05:49:12,525] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9300069,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10702974,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":10119456,"encode_duration_us":21033,"operation_id":"0x9675","timeout_ms":10000,"last_server_duration_us":9148034,"total_duration_us":10006381},{"operation_name":"upsert","last_dispatch_duration_us":19873624,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":28939055,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":10078597,"encode_duration_us":19155,"operation_id":"0x91a5","timeout_ms":10000,"last_server_duration_us":9778977,"total_duration_us":10010853},{"operation_name":"upsert","last_dispatch_duration_us":10044141,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10044141,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":8718214,"encode_duration_us":16801,"operation_id":"0x9610","timeout_ms":10000,"last_server_duration_us":8718214,"total_duration_us":10007388}],"total_count":3}} INFO [2022-01-10 05:49:12,545] jepsen worker 81 - jepsen.util 5481 :info :write [27 13] :RequestTimeoutException INFO [2022-01-10 05:49:12,546] jepsen worker 81 - jepsen.util 5571 :invoke :read [27 nil] INFO [2022-01-10 05:49:12,570] jepsen worker 58 - jepsen.util 4918 :info :write [19 19] :RequestTimeoutException INFO [2022-01-10 05:49:12,571] jepsen worker 58 - jepsen.util 5008 :invoke :read [19 nil] WARN [2022-01-10 05:49:12,634] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9390017,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9390017,"total_server_duration_us":6530,"operation_id":"0x95be","timeout_ms":30000,"last_server_duration_us":6530,"total_duration_us":16593919},{"operation_name":"upsert","last_dispatch_duration_us":848464,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":6903070,"total_server_duration_us":1188442,"encode_duration_us":16467,"operation_id":"0x9706","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10009014},{"operation_name":"upsert","encode_duration_us":17109,"operation_id":"0x9647","timeout_ms":10000,"total_duration_us":10008396},{"operation_name":"upsert","last_dispatch_duration_us":135527,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10007726,"total_server_duration_us":499697,"encode_duration_us":16967,"operation_id":"0x9586","timeout_ms":10000,"last_server_duration_us":20521,"total_duration_us":10006790},{"operation_name":"upsert","last_dispatch_duration_us":697812,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10070883,"total_server_duration_us":1422779,"encode_duration_us":17450,"operation_id":"0x9567","timeout_ms":10000,"last_server_duration_us":521510,"total_duration_us":10006473},{"operation_name":"upsert","last_dispatch_duration_us":9412962,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9412962,"total_server_duration_us":141143,"encode_duration_us":18387,"operation_id":"0x96b2","timeout_ms":10000,"last_server_duration_us":141143,"total_duration_us":10005944},{"operation_name":"upsert","last_dispatch_duration_us":9435588,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9435588,"total_server_duration_us":77290,"encode_duration_us":16890,"operation_id":"0x9692","timeout_ms":10000,"last_server_duration_us":77290,"total_duration_us":10005804},{"operation_name":"upsert","last_dispatch_duration_us":17175026,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":17175026,"total_server_duration_us":163758,"encode_duration_us":16873,"operation_id":"0x95ad","timeout_ms":10000,"last_server_duration_us":163758,"total_duration_us":10005618},{"operation_name":"upsert","last_dispatch_duration_us":33818,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":1104284,"total_server_duration_us":574122,"encode_duration_us":24709,"operation_id":"0x959e","timeout_ms":10000,"last_server_duration_us":21731,"total_duration_us":10004363},{"operation_name":"upsert","last_dispatch_duration_us":9577201,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9577201,"total_server_duration_us":235456,"encode_duration_us":19516,"operation_id":"0x95ec","timeout_ms":10000,"last_server_duration_us":235456,"total_duration_us":10002799}],"total_count":19}} INFO [2022-01-10 05:49:12,686] jepsen worker 74 - jepsen.util 5384 :info :write [24 14] :RequestTimeoutException INFO [2022-01-10 05:49:12,687] jepsen worker 74 - jepsen.util 5474 :invoke :write [24 34] INFO [2022-01-10 05:49:12,742] jepsen worker 42 - jepsen.util 4542 :info :write [14 11] :RequestTimeoutException INFO [2022-01-10 05:49:12,743] jepsen worker 42 - jepsen.util 4632 :invoke :read [14 nil] INFO [2022-01-10 05:49:12,806] jepsen worker 25 - jepsen.util 4525 :info :write [8 26] :RequestTimeoutException INFO [2022-01-10 05:49:12,806] jepsen worker 25 - jepsen.util 4615 :invoke :read [8 nil] INFO [2022-01-10 05:49:12,821] jepsen worker 43 - jepsen.util 4723 :info :write [14 45] :RequestTimeoutException INFO [2022-01-10 05:49:12,822] jepsen worker 43 - jepsen.util 4813 :invoke :write [14 48] INFO [2022-01-10 05:49:12,824] jepsen worker 88 - jepsen.util 4858 :info :write [29 34] :RequestTimeoutException INFO [2022-01-10 05:49:12,825] jepsen worker 88 - jepsen.util 4948 :invoke :write [29 3] INFO [2022-01-10 05:49:12,863] jepsen worker 80 - jepsen.util 6110 :ok :read [26 10] INFO [2022-01-10 05:49:12,863] jepsen worker 80 - jepsen.util 6110 :invoke :read [26 nil] INFO [2022-01-10 05:49:12,876] jepsen worker 37 - jepsen.util 4627 :ok :read [12 14] INFO [2022-01-10 05:49:12,877] jepsen worker 37 - jepsen.util 4627 :invoke :write [12 33] INFO [2022-01-10 05:49:12,885] jepsen worker 55 - jepsen.util 5365 :info :write [18 10] :RequestTimeoutException INFO [2022-01-10 05:49:12,885] jepsen worker 55 - jepsen.util 5455 :invoke :read [18 nil] INFO [2022-01-10 05:49:12,909] jepsen worker 55 - jepsen.util 5455 :ok :read [18 14] INFO [2022-01-10 05:49:12,909] jepsen worker 55 - jepsen.util 5455 :invoke :read [18 nil] INFO [2022-01-10 05:49:12,935] jepsen worker 55 - jepsen.util 5455 :ok :read [18 14] INFO [2022-01-10 05:49:12,936] jepsen worker 55 - jepsen.util 5455 :invoke :write [18 30] INFO [2022-01-10 05:49:12,965] jepsen worker 80 - jepsen.util 6110 :fail :read [26 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:49:12,966] jepsen worker 80 - jepsen.util 6110 :invoke :write [26 28] INFO [2022-01-10 05:49:13,275] jepsen worker 64 - jepsen.util 5014 :ok :read [21 7] INFO [2022-01-10 05:49:13,276] jepsen worker 64 - jepsen.util 5014 :invoke :write [21 12] INFO [2022-01-10 05:49:13,338] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:13,338] jepsen worker nemesis - couchbase.util Rebalance check count: 699 INFO [2022-01-10 05:49:13,338] jepsen worker nemesis - couchbase.util Rebalance stuck count: 578 INFO [2022-01-10 05:49:13,435] jepsen worker 73 - jepsen.util 4843 :ok :read [24 18] INFO [2022-01-10 05:49:13,436] jepsen worker 73 - jepsen.util 4843 :invoke :read [24 nil] INFO [2022-01-10 05:49:13,542] jepsen worker 75 - jepsen.util 4665 :info :write [25 29] :RequestTimeoutException INFO [2022-01-10 05:49:13,543] jepsen worker 75 - jepsen.util 4755 :invoke :write [25 45] INFO [2022-01-10 05:49:13,606] jepsen worker 52 - jepsen.util 6082 :info :write [17 40] :RequestTimeoutException INFO [2022-01-10 05:49:13,607] jepsen worker 52 - jepsen.util 6172 :invoke :write [17 7] INFO [2022-01-10 05:49:13,674] jepsen worker 15 - jepsen.util 4425 :info :write [5 24] :RequestTimeoutException INFO [2022-01-10 05:49:13,675] jepsen worker 15 - jepsen.util 4515 :invoke :write [5 17] INFO [2022-01-10 05:49:13,680] jepsen worker 39 - jepsen.util 5169 :info :write [13 14] :RequestTimeoutException INFO [2022-01-10 05:49:13,680] jepsen worker 39 - jepsen.util 5259 :invoke :write [13 35] INFO [2022-01-10 05:49:13,692] jepsen worker 22 - jepsen.util 4522 :info :write [7 4] :RequestTimeoutException INFO [2022-01-10 05:49:13,692] jepsen worker 22 - jepsen.util 4612 :invoke :read [7 nil] INFO [2022-01-10 05:49:13,797] jepsen worker 31 - jepsen.util 5071 :info :write [10 46] :RequestTimeoutException INFO [2022-01-10 05:49:13,797] jepsen worker 31 - jepsen.util 5161 :invoke :write [10 26] INFO [2022-01-10 05:49:13,799] jepsen worker 59 - jepsen.util 4649 :info :write [19 48] :RequestTimeoutException INFO [2022-01-10 05:49:13,800] jepsen worker 59 - jepsen.util 4739 :invoke :read [19 nil] INFO [2022-01-10 05:49:13,831] jepsen worker 53 - jepsen.util 5903 :info :write [17 18] :RequestTimeoutException INFO [2022-01-10 05:49:13,832] jepsen worker 53 - jepsen.util 5993 :invoke :write [17 25] INFO [2022-01-10 05:49:13,919] jepsen worker 12 - jepsen.util 5502 :info :write [4 27] :RequestTimeoutException INFO [2022-01-10 05:49:13,920] jepsen worker 12 - jepsen.util 5592 :invoke :write [4 18] INFO [2022-01-10 05:49:14,060] jepsen worker 50 - jepsen.util 5090 :info :write [16 17] :RequestTimeoutException INFO [2022-01-10 05:49:14,060] jepsen worker 50 - jepsen.util 5180 :invoke :write [16 39] INFO [2022-01-10 05:49:14,191] jepsen worker 58 - jepsen.util 5008 :ok :read [19 31] INFO [2022-01-10 05:49:14,191] jepsen worker 58 - jepsen.util 5008 :invoke :read [19 nil] INFO [2022-01-10 05:49:14,219] jepsen worker 58 - jepsen.util 5008 :ok :read [19 31] INFO [2022-01-10 05:49:14,220] jepsen worker 58 - jepsen.util 5008 :invoke :read [19 nil] INFO [2022-01-10 05:49:14,222] jepsen worker 51 - jepsen.util 5721 :info :write [17 48] :RequestTimeoutException INFO [2022-01-10 05:49:14,223] jepsen worker 51 - jepsen.util 5811 :invoke :read [17 nil] INFO [2022-01-10 05:49:14,246] jepsen worker 60 - jepsen.util 5280 :info :write [20 4] :RequestTimeoutException INFO [2022-01-10 05:49:14,247] jepsen worker 60 - jepsen.util 5370 :invoke :read [20 nil] INFO [2022-01-10 05:49:14,248] jepsen worker 58 - jepsen.util 5008 :ok :read [19 31] INFO [2022-01-10 05:49:14,248] jepsen worker 58 - jepsen.util 5008 :invoke :read [19 nil] INFO [2022-01-10 05:49:14,273] jepsen worker 58 - jepsen.util 5008 :ok :read [19 31] INFO [2022-01-10 05:49:14,273] jepsen worker 58 - jepsen.util 5008 :invoke :write [19 24] INFO [2022-01-10 05:49:14,344] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:14,344] jepsen worker nemesis - couchbase.util Rebalance check count: 700 INFO [2022-01-10 05:49:14,344] jepsen worker nemesis - couchbase.util Rebalance stuck count: 579 INFO [2022-01-10 05:49:14,378] jepsen worker 42 - jepsen.util 4632 :ok :read [14 21] INFO [2022-01-10 05:49:14,378] jepsen worker 42 - jepsen.util 4632 :invoke :write [14 34] INFO [2022-01-10 05:49:14,379] jepsen worker 73 - jepsen.util 4843 :ok :read [24 18] INFO [2022-01-10 05:49:14,380] jepsen worker 73 - jepsen.util 4843 :invoke :read [24 nil] INFO [2022-01-10 05:49:14,772] jepsen worker 73 - jepsen.util 4843 :ok :read [24 18] INFO [2022-01-10 05:49:14,773] jepsen worker 73 - jepsen.util 4843 :invoke :write [24 6] INFO [2022-01-10 05:49:14,774] jepsen worker 25 - jepsen.util 4615 :ok :read [8 39] INFO [2022-01-10 05:49:14,774] jepsen worker 25 - jepsen.util 4615 :invoke :read [8 nil] INFO [2022-01-10 05:49:14,781] jepsen worker 79 - jepsen.util 5749 :info :write [26 23] :RequestTimeoutException INFO [2022-01-10 05:49:14,782] jepsen worker 79 - jepsen.util 5839 :invoke :read [26 nil] INFO [2022-01-10 05:49:14,839] jepsen worker 57 - jepsen.util 5637 :info :write [19 31] :RequestTimeoutException INFO [2022-01-10 05:49:14,841] jepsen worker 57 - jepsen.util 5727 :invoke :write [19 2] INFO [2022-01-10 05:49:15,127] jepsen worker 30 - jepsen.util 4620 :info :write [10 46] :SyncWriteAmbiguous INFO [2022-01-10 05:49:15,128] jepsen worker 30 - jepsen.util 4710 :invoke :write [10 48] INFO [2022-01-10 05:49:15,272] jepsen worker 68 - jepsen.util 5018 :ok :read [22 41] INFO [2022-01-10 05:49:15,272] jepsen worker 68 - jepsen.util 5018 :invoke :read [22 nil] INFO [2022-01-10 05:49:15,281] jepsen worker 54 - jepsen.util 5184 :info :write [18 12] :RequestTimeoutException INFO [2022-01-10 05:49:15,281] jepsen worker 54 - jepsen.util 5274 :invoke :read [18 nil] INFO [2022-01-10 05:49:15,322] jepsen worker 44 - jepsen.util 4904 :info :write [14 10] :RequestTimeoutException INFO [2022-01-10 05:49:15,323] jepsen worker 44 - jepsen.util 4994 :invoke :write [14 38] INFO [2022-01-10 05:49:15,350] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:15,350] jepsen worker nemesis - couchbase.util Rebalance check count: 701 INFO [2022-01-10 05:49:15,350] jepsen worker nemesis - couchbase.util Rebalance stuck count: 580 INFO [2022-01-10 05:49:15,406] jepsen worker 40 - jepsen.util 4540 :info :write [13 25] :RequestTimeoutException INFO [2022-01-10 05:49:15,407] jepsen worker 40 - jepsen.util 4630 :invoke :write [13 41] INFO [2022-01-10 05:49:15,550] jepsen worker 60 - jepsen.util 5370 :ok :read [20 :nil] INFO [2022-01-10 05:49:15,551] jepsen worker 60 - jepsen.util 5370 :invoke :read [20 nil] INFO [2022-01-10 05:49:15,656] jepsen worker 2 - jepsen.util 5132 :info :write [0 16] :RequestTimeoutException INFO [2022-01-10 05:49:15,657] jepsen worker 2 - jepsen.util 5222 :invoke :read [0 nil] INFO [2022-01-10 05:49:15,695] jepsen worker 60 - jepsen.util 5370 :ok :read [20 :nil] INFO [2022-01-10 05:49:15,696] jepsen worker 60 - jepsen.util 5370 :invoke :write [20 8] INFO [2022-01-10 05:49:15,799] jepsen worker 29 - jepsen.util 4799 :info :write [9 18] :RequestTimeoutException INFO [2022-01-10 05:49:15,800] jepsen worker 29 - jepsen.util 4889 :invoke :write [9 39] INFO [2022-01-10 05:49:15,818] jepsen worker 25 - jepsen.util 4615 :ok :read [8 39] INFO [2022-01-10 05:49:15,818] jepsen worker 25 - jepsen.util 4615 :invoke :write [8 38] INFO [2022-01-10 05:49:16,079] jepsen worker 36 - jepsen.util 3816 :info :write [12 40] :RequestTimeoutException INFO [2022-01-10 05:49:16,080] jepsen worker 36 - jepsen.util 3906 :invoke :write [12 3] INFO [2022-01-10 05:49:16,175] jepsen worker 16 - jepsen.util 4966 :info :write [5 35] :RequestTimeoutException INFO [2022-01-10 05:49:16,175] jepsen worker 16 - jepsen.util 5056 :invoke :read [5 nil] INFO [2022-01-10 05:49:16,276] jepsen worker 21 - jepsen.util 4881 :info :write [7 0] :RequestTimeoutException INFO [2022-01-10 05:49:16,277] jepsen worker 21 - jepsen.util 4971 :invoke :write [7 23] INFO [2022-01-10 05:49:16,355] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:16,355] jepsen worker nemesis - couchbase.util Rebalance check count: 702 INFO [2022-01-10 05:49:16,355] jepsen worker nemesis - couchbase.util Rebalance stuck count: 581 INFO [2022-01-10 05:49:16,386] jepsen worker 85 - jepsen.util 5305 :info :write [28 23] :RequestTimeoutException INFO [2022-01-10 05:49:16,387] jepsen worker 85 - jepsen.util 5395 :invoke :write [28 49] INFO [2022-01-10 05:49:16,505] jepsen worker 0 - jepsen.util 5040 :info :write [0 36] :RequestTimeoutException INFO [2022-01-10 05:49:16,505] jepsen worker 0 - jepsen.util 5130 :invoke :write [0 35] INFO [2022-01-10 05:49:16,579] jepsen worker 49 - jepsen.util 4729 :info :write [16 1] :RequestTimeoutException INFO [2022-01-10 05:49:16,580] jepsen worker 49 - jepsen.util 4819 :invoke :write [16 10] INFO [2022-01-10 05:49:16,635] jepsen worker 41 - jepsen.util 4811 :info :write [13 12] :RequestTimeoutException INFO [2022-01-10 05:49:16,636] jepsen worker 41 - jepsen.util 4901 :invoke :read [13 nil] INFO [2022-01-10 05:49:16,652] jepsen worker 9 - jepsen.util 5409 :info :write [3 31] :RequestTimeoutException INFO [2022-01-10 05:49:16,653] jepsen worker 9 - jepsen.util 5499 :invoke :write [3 6] INFO [2022-01-10 05:49:16,754] jepsen worker 76 - jepsen.util 4306 :ok :read [25 :nil] INFO [2022-01-10 05:49:16,755] jepsen worker 76 - jepsen.util 4306 :invoke :read [25 nil] INFO [2022-01-10 05:49:17,146] jepsen worker 71 - jepsen.util 4571 :info :write [23 28] :RequestTimeoutException INFO [2022-01-10 05:49:17,147] jepsen worker 71 - jepsen.util 4661 :invoke :write [23 11] INFO [2022-01-10 05:49:17,360] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:17,360] jepsen worker nemesis - couchbase.util Rebalance check count: 703 INFO [2022-01-10 05:49:17,360] jepsen worker nemesis - couchbase.util Rebalance stuck count: 582 INFO [2022-01-10 05:49:17,430] jepsen worker 24 - jepsen.util 4524 :info :write [8 23] :RequestTimeoutException INFO [2022-01-10 05:49:17,431] jepsen worker 24 - jepsen.util 4614 :invoke :write [8 6] INFO [2022-01-10 05:49:17,522] jepsen worker 10 - jepsen.util 4690 :info :write [3 18] :RequestTimeoutException INFO [2022-01-10 05:49:17,523] jepsen worker 10 - jepsen.util 4780 :invoke :write [3 41] INFO [2022-01-10 05:49:17,802] jepsen worker 65 - jepsen.util 4565 :info :write [21 15] :RequestTimeoutException INFO [2022-01-10 05:49:17,803] jepsen worker 65 - jepsen.util 4655 :invoke :read [21 nil] INFO [2022-01-10 05:49:17,887] jepsen worker 79 - jepsen.util 5839 :ok :read [26 10] INFO [2022-01-10 05:49:17,887] jepsen worker 79 - jepsen.util 5839 :invoke :read [26 nil] INFO [2022-01-10 05:49:18,096] jepsen worker 51 - jepsen.util 5811 :ok :read [17 41] INFO [2022-01-10 05:49:18,096] jepsen worker 51 - jepsen.util 5811 :invoke :write [17 30] INFO [2022-01-10 05:49:18,166] jepsen worker 11 - jepsen.util 4871 :info :write [3 27] :RequestTimeoutException INFO [2022-01-10 05:49:18,167] jepsen worker 11 - jepsen.util 4961 :invoke :read [3 nil] INFO [2022-01-10 05:49:18,181] jepsen worker 17 - jepsen.util 4787 :info :write [5 43] :RequestTimeoutException INFO [2022-01-10 05:49:18,182] jepsen worker 17 - jepsen.util 4877 :invoke :write [5 46] INFO [2022-01-10 05:49:18,225] jepsen worker 63 - jepsen.util 4563 :info :write [21 16] :RequestTimeoutException INFO [2022-01-10 05:49:18,226] jepsen worker 63 - jepsen.util 4653 :invoke :read [21 nil] INFO [2022-01-10 05:49:18,256] jepsen worker 18 - jepsen.util 4788 :info :write [6 15] :RequestTimeoutException INFO [2022-01-10 05:49:18,257] jepsen worker 18 - jepsen.util 4878 :invoke :write [6 10] INFO [2022-01-10 05:49:18,295] jepsen worker 8 - jepsen.util 5048 :info :write [2 22] :RequestTimeoutException INFO [2022-01-10 05:49:18,296] jepsen worker 8 - jepsen.util 5138 :invoke :write [2 2] INFO [2022-01-10 05:49:18,296] jepsen worker 33 - jepsen.util 5163 :info :write [11 45] :RequestTimeoutException INFO [2022-01-10 05:49:18,296] jepsen worker 33 - jepsen.util 5253 :invoke :read [11 nil] INFO [2022-01-10 05:49:18,366] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:18,366] jepsen worker nemesis - couchbase.util Rebalance check count: 704 INFO [2022-01-10 05:49:18,366] jepsen worker nemesis - couchbase.util Rebalance stuck count: 583 INFO [2022-01-10 05:49:18,450] jepsen worker 76 - jepsen.util 4306 :ok :read [25 :nil] INFO [2022-01-10 05:49:18,450] jepsen worker 76 - jepsen.util 4306 :invoke :read [25 nil] INFO [2022-01-10 05:49:18,486] jepsen worker 66 - jepsen.util 4836 :info :write [22 5] :RequestTimeoutException INFO [2022-01-10 05:49:18,487] jepsen worker 66 - jepsen.util 4926 :invoke :write [22 45] INFO [2022-01-10 05:49:18,514] jepsen worker 79 - jepsen.util 5839 :ok :read [26 10] INFO [2022-01-10 05:49:18,514] jepsen worker 79 - jepsen.util 5839 :invoke :read [26 nil] INFO [2022-01-10 05:49:19,011] jepsen worker 81 - jepsen.util 5571 :ok :read [27 30] INFO [2022-01-10 05:49:19,012] jepsen worker 81 - jepsen.util 5571 :invoke :write [27 27] INFO [2022-01-10 05:49:19,095] jepsen worker 61 - jepsen.util 5191 :info :write [20 12] :RequestTimeoutException INFO [2022-01-10 05:49:19,096] jepsen worker 61 - jepsen.util 5281 :invoke :read [20 nil] INFO [2022-01-10 05:49:19,102] jepsen worker 70 - jepsen.util 5020 :info :write [23 33] :RequestTimeoutException INFO [2022-01-10 05:49:19,102] jepsen worker 70 - jepsen.util 5110 :invoke :read [23 nil] INFO [2022-01-10 05:49:19,371] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:19,371] jepsen worker nemesis - couchbase.util Rebalance check count: 705 INFO [2022-01-10 05:49:19,371] jepsen worker nemesis - couchbase.util Rebalance stuck count: 584 INFO [2022-01-10 05:49:19,466] jepsen worker 35 - jepsen.util 4715 :info :write [11 38] :RequestTimeoutException INFO [2022-01-10 05:49:19,466] jepsen worker 35 - jepsen.util 4805 :invoke :write [11 22] INFO [2022-01-10 05:49:19,479] jepsen worker 48 - jepsen.util 4818 :info :write [16 44] :RequestTimeoutException INFO [2022-01-10 05:49:19,480] jepsen worker 48 - jepsen.util 4908 :invoke :write [16 3] INFO [2022-01-10 05:49:19,491] jepsen worker 45 - jepsen.util 5355 :info :write [15 22] :RequestTimeoutException INFO [2022-01-10 05:49:19,492] jepsen worker 45 - jepsen.util 5445 :invoke :read [15 nil] INFO [2022-01-10 05:49:19,741] jepsen worker 62 - jepsen.util 5282 :info :write [20 35] :RequestTimeoutException INFO [2022-01-10 05:49:19,741] jepsen worker 62 - jepsen.util 5372 :invoke :write [20 12] INFO [2022-01-10 05:49:19,751] jepsen worker 23 - jepsen.util 5063 :info :write [7 9] :RequestTimeoutException INFO [2022-01-10 05:49:19,751] jepsen worker 23 - jepsen.util 5153 :invoke :write [7 6] INFO [2022-01-10 05:49:19,791] jepsen worker 19 - jepsen.util 4969 :info :write [6 26] :RequestTimeoutException INFO [2022-01-10 05:49:19,792] jepsen worker 19 - jepsen.util 5059 :invoke :read [6 nil] INFO [2022-01-10 05:49:19,892] jepsen worker 22 - jepsen.util 4612 :ok :read [7 :nil] INFO [2022-01-10 05:49:19,893] jepsen worker 76 - jepsen.util 4306 :ok :read [25 :nil] INFO [2022-01-10 05:49:19,894] jepsen worker 22 - jepsen.util 4612 :invoke :write [7 21] INFO [2022-01-10 05:49:19,895] jepsen worker 76 - jepsen.util 4306 :invoke :read [25 nil] INFO [2022-01-10 05:49:20,154] jepsen worker 61 - jepsen.util 5281 :ok :read [20 :nil] INFO [2022-01-10 05:49:20,155] jepsen worker 61 - jepsen.util 5281 :invoke :read [20 nil] INFO [2022-01-10 05:49:20,311] jepsen worker 47 - jepsen.util 5087 :info :write [15 6] :RequestTimeoutException INFO [2022-01-10 05:49:20,312] jepsen worker 47 - jepsen.util 5177 :invoke :read [15 nil] INFO [2022-01-10 05:49:20,379] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:20,379] jepsen worker nemesis - couchbase.util Rebalance check count: 706 INFO [2022-01-10 05:49:20,379] jepsen worker nemesis - couchbase.util Rebalance stuck count: 585 INFO [2022-01-10 05:49:20,382] jepsen worker 28 - jepsen.util 4528 :info :write [9 16] :RequestTimeoutException INFO [2022-01-10 05:49:20,382] jepsen worker 28 - jepsen.util 4618 :invoke :read [9 nil] INFO [2022-01-10 05:49:20,394] jepsen worker 7 - jepsen.util 5317 :info :write [2 39] :RequestTimeoutException INFO [2022-01-10 05:49:20,395] jepsen worker 7 - jepsen.util 5407 :invoke :read [2 nil] INFO [2022-01-10 05:49:20,472] jepsen worker 1 - jepsen.util 4951 :info :write [0 12] :RequestTimeoutException INFO [2022-01-10 05:49:20,473] jepsen worker 1 - jepsen.util 5041 :invoke :write [0 25] INFO [2022-01-10 05:49:20,546] jepsen worker 26 - jepsen.util 4796 :info :write [8 38] :RequestTimeoutException INFO [2022-01-10 05:49:20,547] jepsen worker 26 - jepsen.util 4886 :invoke :read [8 nil] INFO [2022-01-10 05:49:20,596] jepsen worker 79 - jepsen.util 5839 :ok :read [26 10] INFO [2022-01-10 05:49:20,597] jepsen worker 79 - jepsen.util 5839 :invoke :read [26 nil] INFO [2022-01-10 05:49:20,630] jepsen worker 61 - jepsen.util 5281 :ok :read [20 :nil] INFO [2022-01-10 05:49:20,631] jepsen worker 61 - jepsen.util 5281 :invoke :read [20 nil] INFO [2022-01-10 05:49:20,676] jepsen worker 20 - jepsen.util 4610 :info :write [6 45] :RequestTimeoutException INFO [2022-01-10 05:49:20,677] jepsen worker 20 - jepsen.util 4700 :invoke :write [6 48] INFO [2022-01-10 05:49:20,680] jepsen worker 59 - jepsen.util 4739 :ok :read [19 31] INFO [2022-01-10 05:49:20,681] jepsen worker 59 - jepsen.util 4739 :invoke :write [19 6] INFO [2022-01-10 05:49:20,697] jepsen worker 2 - jepsen.util 5222 :ok :read [0 46] INFO [2022-01-10 05:49:20,698] jepsen worker 2 - jepsen.util 5222 :invoke :read [0 nil] INFO [2022-01-10 05:49:20,721] jepsen worker 78 - jepsen.util 6108 :info :write [26 1] :RequestTimeoutException INFO [2022-01-10 05:49:20,722] jepsen worker 78 - jepsen.util 6198 :invoke :write [26 24] INFO [2022-01-10 05:49:20,817] jepsen worker 6 - jepsen.util 4866 :info :write [2 39] :RequestTimeoutException INFO [2022-01-10 05:49:20,817] jepsen worker 6 - jepsen.util 4956 :invoke :write [2 7] INFO [2022-01-10 05:49:20,820] jepsen worker 82 - jepsen.util 5032 :ok :read [27 30] INFO [2022-01-10 05:49:20,821] jepsen worker 82 - jepsen.util 5032 :invoke :write [27 11] WARN [2022-01-10 05:49:20,836] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9960658,"last_local_id":"EFE9906200000001/00000000E26F4444","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10601562,"last_local_socket":"172.23.104.79:38414","total_server_duration_us":9862732,"encode_duration_us":16690,"operation_id":"0x986a","timeout_ms":10000,"last_server_duration_us":9527926,"total_duration_us":10003185},{"operation_name":"upsert","last_dispatch_duration_us":13796829,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13796829,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9029938,"encode_duration_us":17509,"operation_id":"0x966d","timeout_ms":10000,"last_server_duration_us":9029938,"total_duration_us":10004688},{"operation_name":"upsert","last_dispatch_duration_us":16447471,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":21076793,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9585664,"encode_duration_us":22302,"operation_id":"0x951f","timeout_ms":10000,"last_server_duration_us":9040551,"total_duration_us":10006344},{"operation_name":"upsert","last_dispatch_duration_us":10242863,"last_local_id":"EFE9906200000001/000000007CF9F49A","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10242863,"last_local_socket":"172.23.104.79:44306","total_server_duration_us":9429365,"encode_duration_us":17073,"operation_id":"0x96d5","timeout_ms":10000,"last_server_duration_us":9429365,"total_duration_us":10004908}],"total_count":4}} WARN [2022-01-10 05:49:20,836] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9079406,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":9079406,"total_server_duration_us":136292,"operation_id":"0x96fa","timeout_ms":30000,"last_server_duration_us":136292,"total_duration_us":14251557},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":16978,"operation_id":"0x9809","timeout_ms":10000,"total_duration_us":10009712},{"operation_name":"upsert","last_dispatch_duration_us":9141000,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9141000,"total_server_duration_us":19652,"encode_duration_us":18035,"operation_id":"0x9764","timeout_ms":10000,"last_server_duration_us":19652,"total_duration_us":10006950},{"operation_name":"upsert","last_dispatch_duration_us":10331943,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11318979,"total_server_duration_us":1154697,"encode_duration_us":18202,"operation_id":"0x986d","timeout_ms":10000,"last_server_duration_us":750048,"total_duration_us":10005853},{"operation_name":"upsert","last_dispatch_duration_us":9138173,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":9138173,"total_server_duration_us":19108,"encode_duration_us":17168,"operation_id":"0x9766","timeout_ms":10000,"last_server_duration_us":19108,"total_duration_us":10003735},{"operation_name":"upsert","last_dispatch_duration_us":9960658,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10601562,"total_server_duration_us":9862732,"encode_duration_us":16690,"operation_id":"0x986a","timeout_ms":10000,"last_server_duration_us":9527926,"total_duration_us":10003185},{"operation_name":"upsert","last_dispatch_duration_us":630048,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":9949936,"total_server_duration_us":203708,"encode_duration_us":18814,"operation_id":"0x98d2","timeout_ms":10000,"last_server_duration_us":159540,"total_duration_us":10003172},{"operation_name":"upsert","last_dispatch_duration_us":8858923,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8858923,"total_server_duration_us":19185,"encode_duration_us":20842,"operation_id":"0x9774","timeout_ms":10000,"last_server_duration_us":19185,"total_duration_us":10002040},{"operation_name":"upsert","last_dispatch_duration_us":8937712,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":8937712,"total_server_duration_us":20282,"encode_duration_us":18096,"operation_id":"0x9806","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10001535},{"operation_name":"get","last_dispatch_duration_us":7467736,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":7467736,"total_server_duration_us":284211,"operation_id":"0x97aa","timeout_ms":30000,"last_server_duration_us":284211,"total_duration_us":7470332}],"total_count":12}} INFO [2022-01-10 05:49:20,857] jepsen worker 70 - jepsen.util 5110 :fail :read [23 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:49:20,858] jepsen worker 70 - jepsen.util 5110 :invoke :read [23 nil] INFO [2022-01-10 05:49:20,921] jepsen worker 5 - jepsen.util 4955 :info :write [1 11] :RequestTimeoutException INFO [2022-01-10 05:49:20,921] jepsen worker 5 - jepsen.util 5045 :invoke :read [1 nil] INFO [2022-01-10 05:49:20,936] jepsen worker 38 - jepsen.util 4628 :info :write [12 36] :RequestTimeoutException INFO [2022-01-10 05:49:20,939] jepsen worker 38 - jepsen.util 4718 :invoke :write [12 12] INFO [2022-01-10 05:49:20,942] jepsen worker 61 - jepsen.util 5281 :ok :read [20 :nil] INFO [2022-01-10 05:49:20,945] jepsen worker 61 - jepsen.util 5281 :invoke :read [20 nil] INFO [2022-01-10 05:49:20,960] jepsen worker 26 - jepsen.util 4886 :ok :read [8 39] INFO [2022-01-10 05:49:20,961] jepsen worker 26 - jepsen.util 4886 :invoke :write [8 28] INFO [2022-01-10 05:49:20,965] jepsen worker 89 - jepsen.util 5399 :ok :read [29 0] INFO [2022-01-10 05:49:20,965] jepsen worker 89 - jepsen.util 5399 :invoke :write [29 3] INFO [2022-01-10 05:49:20,993] jepsen worker 32 - jepsen.util 5252 :ok :read [10 31] INFO [2022-01-10 05:49:20,998] jepsen worker 32 - jepsen.util 5252 :invoke :write [10 43] INFO [2022-01-10 05:49:21,051] jepsen worker 13 - jepsen.util 5773 :info :write [4 9] :RequestTimeoutException INFO [2022-01-10 05:49:21,052] jepsen worker 13 - jepsen.util 5863 :invoke :write [4 10] INFO [2022-01-10 05:49:21,065] jepsen worker 67 - jepsen.util 4567 :info :write [22 24] :RequestTimeoutException INFO [2022-01-10 05:49:21,065] jepsen worker 67 - jepsen.util 4657 :invoke :read [22 nil] INFO [2022-01-10 05:49:21,130] jepsen worker 34 - jepsen.util 4714 :info :write [11 13] :RequestTimeoutException INFO [2022-01-10 05:49:21,130] jepsen worker 34 - jepsen.util 4804 :invoke :write [11 39] INFO [2022-01-10 05:49:21,230] jepsen worker 56 - jepsen.util 5186 :info :write [18 24] :SyncWriteAmbiguous INFO [2022-01-10 05:49:21,230] jepsen worker 56 - jepsen.util 5276 :invoke :read [18 nil] INFO [2022-01-10 05:49:21,281] jepsen worker 4 - jepsen.util 5134 :info :write [1 44] :RequestTimeoutException INFO [2022-01-10 05:49:21,282] jepsen worker 4 - jepsen.util 5224 :invoke :read [1 nil] INFO [2022-01-10 05:49:21,288] jepsen worker 56 - jepsen.util 5276 :ok :read [18 14] INFO [2022-01-10 05:49:21,289] jepsen worker 56 - jepsen.util 5276 :invoke :read [18 nil] INFO [2022-01-10 05:49:21,315] jepsen worker 56 - jepsen.util 5276 :ok :read [18 14] INFO [2022-01-10 05:49:21,316] jepsen worker 56 - jepsen.util 5276 :invoke :read [18 nil] INFO [2022-01-10 05:49:21,355] jepsen worker 67 - jepsen.util 4657 :ok :read [22 41] INFO [2022-01-10 05:49:21,356] jepsen worker 67 - jepsen.util 4657 :invoke :write [22 23] INFO [2022-01-10 05:49:21,360] jepsen worker 56 - jepsen.util 5276 :ok :read [18 14] INFO [2022-01-10 05:49:21,360] jepsen worker 56 - jepsen.util 5276 :invoke :read [18 nil] INFO [2022-01-10 05:49:21,364] jepsen worker 86 - jepsen.util 4136 :ok :read [28 45] INFO [2022-01-10 05:49:21,365] jepsen worker 86 - jepsen.util 4136 :invoke :write [28 48] INFO [2022-01-10 05:49:21,386] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:21,386] jepsen worker nemesis - couchbase.util Rebalance check count: 707 INFO [2022-01-10 05:49:21,386] jepsen worker nemesis - couchbase.util Rebalance stuck count: 586 INFO [2022-01-10 05:49:21,388] jepsen worker 56 - jepsen.util 5276 :ok :read [18 14] INFO [2022-01-10 05:49:21,388] jepsen worker 56 - jepsen.util 5276 :invoke :write [18 39] INFO [2022-01-10 05:49:21,429] jepsen worker 79 - jepsen.util 5839 :ok :read [26 10] INFO [2022-01-10 05:49:21,430] jepsen worker 79 - jepsen.util 5839 :invoke :write [26 12] INFO [2022-01-10 05:49:21,451] jepsen worker 3 - jepsen.util 4953 :info :write [1 10] :RequestTimeoutException INFO [2022-01-10 05:49:21,452] jepsen worker 3 - jepsen.util 5043 :invoke :read [1 nil] INFO [2022-01-10 05:49:21,611] jepsen worker 76 - jepsen.util 4306 :ok :read [25 :nil] INFO [2022-01-10 05:49:21,612] jepsen worker 76 - jepsen.util 4306 :invoke :read [25 nil] INFO [2022-01-10 05:49:21,614] jepsen worker 61 - jepsen.util 5281 :ok :read [20 :nil] INFO [2022-01-10 05:49:21,619] jepsen worker 61 - jepsen.util 5281 :invoke :read [20 nil] INFO [2022-01-10 05:49:21,671] jepsen worker 83 - jepsen.util 5573 :info :write [27 25] :RequestTimeoutException INFO [2022-01-10 05:49:21,672] jepsen worker 83 - jepsen.util 5663 :invoke :read [27 nil] INFO [2022-01-10 05:49:21,787] jepsen worker 70 - jepsen.util 5110 :fail :read [23 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:49:21,787] jepsen worker 70 - jepsen.util 5110 :invoke :read [23 nil] INFO [2022-01-10 05:49:21,850] jepsen worker 2 - jepsen.util 5222 :ok :read [0 46] INFO [2022-01-10 05:49:21,851] jepsen worker 2 - jepsen.util 5222 :invoke :write [0 9] INFO [2022-01-10 05:49:21,856] jepsen worker 14 - jepsen.util 5954 :info :write [4 2] :RequestTimeoutException INFO [2022-01-10 05:49:21,857] jepsen worker 41 - jepsen.util 4901 :ok :read [13 15] INFO [2022-01-10 05:49:21,857] jepsen worker 14 - jepsen.util 6044 :invoke :write [4 49] INFO [2022-01-10 05:49:21,857] jepsen worker 41 - jepsen.util 4901 :invoke :write [13 46] INFO [2022-01-10 05:49:21,871] jepsen worker 28 - jepsen.util 4618 :ok :read [9 22] INFO [2022-01-10 05:49:21,872] jepsen worker 28 - jepsen.util 4618 :invoke :write [9 35] INFO [2022-01-10 05:49:21,885] jepsen worker 69 - jepsen.util 4569 :info :write [23 46] :RequestTimeoutException INFO [2022-01-10 05:49:21,886] jepsen worker 69 - jepsen.util 4659 :invoke :read [23 nil] INFO [2022-01-10 05:49:21,921] jepsen worker 69 - jepsen.util 4659 :fail :read [23 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:49:21,921] jepsen worker 69 - jepsen.util 4659 :invoke :read [23 nil] INFO [2022-01-10 05:49:21,959] jepsen worker 77 - jepsen.util 5477 :info :write [25 23] :RequestTimeoutException INFO [2022-01-10 05:49:21,960] jepsen worker 77 - jepsen.util 5567 :invoke :read [25 nil] INFO [2022-01-10 05:49:21,990] jepsen worker 72 - jepsen.util 5292 :info :write [24 47] :RequestTimeoutException INFO [2022-01-10 05:49:21,990] jepsen worker 72 - jepsen.util 5382 :invoke :read [24 nil] INFO [2022-01-10 05:49:22,031] jepsen worker 3 - jepsen.util 5043 :ok :read [1 49] INFO [2022-01-10 05:49:22,031] jepsen worker 3 - jepsen.util 5043 :invoke :write [1 49] INFO [2022-01-10 05:49:22,048] jepsen worker 5 - jepsen.util 5045 :ok :read [1 49] INFO [2022-01-10 05:49:22,048] jepsen worker 5 - jepsen.util 5045 :invoke :read [1 nil] INFO [2022-01-10 05:49:22,180] jepsen worker 46 - jepsen.util 5086 :info :write [15 19] :RequestTimeoutException INFO [2022-01-10 05:49:22,180] jepsen worker 46 - jepsen.util 5176 :invoke :read [15 nil] INFO [2022-01-10 05:49:22,215] jepsen worker 27 - jepsen.util 5337 :info :write [9 10] :RequestTimeoutException INFO [2022-01-10 05:49:22,215] jepsen worker 87 - jepsen.util 4767 :info :write [29 7] :RequestTimeoutException INFO [2022-01-10 05:49:22,216] jepsen worker 87 - jepsen.util 4857 :invoke :read [29 nil] INFO [2022-01-10 05:49:22,216] jepsen worker 27 - jepsen.util 5427 :invoke :write [9 6] INFO [2022-01-10 05:49:22,246] jepsen worker 69 - jepsen.util 4659 :ok :read [23 26] INFO [2022-01-10 05:49:22,246] jepsen worker 69 - jepsen.util 4659 :invoke :write [23 8] WARN [2022-01-10 05:49:22,277] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":937119,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":5578933,"total_server_duration_us":559104,"encode_duration_us":17362,"operation_id":"0x98d0","timeout_ms":10000,"last_server_duration_us":112804,"total_duration_us":10010774},{"operation_name":"upsert","last_dispatch_duration_us":489239,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":6942439,"total_server_duration_us":693610,"encode_duration_us":18889,"operation_id":"0x98b3","timeout_ms":10000,"last_server_duration_us":123200,"total_duration_us":10010286},{"operation_name":"upsert","last_dispatch_duration_us":941164,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":6884141,"total_server_duration_us":1559399,"encode_duration_us":26657,"operation_id":"0x98b2","timeout_ms":10000,"last_server_duration_us":282507,"total_duration_us":10009962},{"operation_name":"upsert","last_dispatch_duration_us":7831053,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":8223210,"total_server_duration_us":259321,"encode_duration_us":16266,"operation_id":"0x97d5","timeout_ms":10000,"last_server_duration_us":19574,"total_duration_us":10007799},{"operation_name":"upsert","last_dispatch_duration_us":1704899,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":6369774,"total_server_duration_us":457313,"encode_duration_us":17055,"operation_id":"0x98ba","timeout_ms":10000,"last_server_duration_us":168999,"total_duration_us":10006906},{"operation_name":"upsert","last_dispatch_duration_us":811673,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10107642,"total_server_duration_us":929410,"encode_duration_us":17202,"operation_id":"0x98fd","timeout_ms":10000,"last_server_duration_us":154428,"total_duration_us":10006561},{"operation_name":"upsert","last_dispatch_duration_us":2276029,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":11232652,"total_server_duration_us":1038000,"encode_duration_us":18281,"operation_id":"0x979a","timeout_ms":10000,"last_server_duration_us":223440,"total_duration_us":10005233},{"operation_name":"upsert","last_dispatch_duration_us":640624,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10436014,"total_server_duration_us":2552893,"encode_duration_us":16547,"operation_id":"0x9906","timeout_ms":10000,"last_server_duration_us":221683,"total_duration_us":10004831},{"operation_name":"upsert","last_dispatch_duration_us":12603182,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":12603182,"total_server_duration_us":9139501,"encode_duration_us":22490,"operation_id":"0x9868","timeout_ms":10000,"last_server_duration_us":9139501,"total_duration_us":10004554},{"operation_name":"upsert","last_dispatch_duration_us":1980692,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9809851,"total_server_duration_us":2357852,"encode_duration_us":19417,"operation_id":"0x98fa","timeout_ms":10000,"last_server_duration_us":222781,"total_duration_us":10002944}],"total_count":20}} WARN [2022-01-10 05:49:22,340] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10883841,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12486427,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9854260,"encode_duration_us":16391,"operation_id":"0x97d9","timeout_ms":10000,"last_server_duration_us":9170449,"total_duration_us":10001347},{"operation_name":"upsert","last_dispatch_duration_us":10570907,"last_local_id":"EFE9906200000002/000000000DD1E359","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10570907,"last_local_socket":"172.23.104.79:38440","total_server_duration_us":9334468,"encode_duration_us":17796,"operation_id":"0x986f","timeout_ms":10000,"last_server_duration_us":9334468,"total_duration_us":10002149}],"total_count":2}} WARN [2022-01-10 05:49:22,377] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12603182,"last_local_id":"EFE9906200000006/0000000095B39CD8","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12603182,"last_local_socket":"172.23.104.79:52090","total_server_duration_us":9139501,"encode_duration_us":22490,"operation_id":"0x9868","timeout_ms":10000,"last_server_duration_us":9139501,"total_duration_us":10004554}],"total_count":1}} INFO [2022-01-10 05:49:22,392] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} WARN [2022-01-10 05:49:22,437] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":961378,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8181365,"total_server_duration_us":1281717,"encode_duration_us":16851,"operation_id":"0x973f","timeout_ms":10000,"last_server_duration_us":49371,"total_duration_us":10009573},{"operation_name":"upsert","last_dispatch_duration_us":9041806,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":9579820,"total_server_duration_us":9323935,"encode_duration_us":16399,"operation_id":"0x9867","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10008347},{"operation_name":"upsert","last_dispatch_duration_us":953929,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":8172169,"total_server_duration_us":1307004,"encode_duration_us":17433,"operation_id":"0x975c","timeout_ms":10000,"last_server_duration_us":72862,"total_duration_us":10008258},{"operation_name":"upsert","last_dispatch_duration_us":745866,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1307844,"total_server_duration_us":152657,"encode_duration_us":16355,"operation_id":"0x98d5","timeout_ms":10000,"last_server_duration_us":110674,"total_duration_us":10005505},{"operation_name":"upsert","last_dispatch_duration_us":8031398,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000005/0000000094AEA561","last_local_socket":"172.23.104.79:34136","total_dispatch_duration_us":8896467,"total_server_duration_us":565305,"encode_duration_us":19666,"operation_id":"0x9792","timeout_ms":10000,"last_server_duration_us":29137,"total_duration_us":10004553},{"operation_name":"upsert","last_dispatch_duration_us":939287,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":4981060,"total_server_duration_us":747896,"encode_duration_us":28703,"operation_id":"0x9807","timeout_ms":10000,"last_server_duration_us":36423,"total_duration_us":10003732},{"operation_name":"upsert","last_dispatch_duration_us":1074420,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":6943335,"total_server_duration_us":1116233,"encode_duration_us":19898,"operation_id":"0x97a7","timeout_ms":10000,"last_server_duration_us":52896,"total_duration_us":10003470},{"operation_name":"upsert","last_dispatch_duration_us":1141937,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":3726380,"total_server_duration_us":442732,"encode_duration_us":16708,"operation_id":"0x98f7","timeout_ms":10000,"last_server_duration_us":162411,"total_duration_us":10003183},{"operation_name":"upsert","last_dispatch_duration_us":10702023,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11512706,"total_server_duration_us":9386877,"encode_duration_us":17437,"operation_id":"0x9808","timeout_ms":10000,"last_server_duration_us":9277511,"total_duration_us":10001707},{"operation_name":"upsert","last_dispatch_duration_us":2903577,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":9684300,"total_server_duration_us":2866567,"encode_duration_us":16657,"operation_id":"0x9833","timeout_ms":10000,"last_server_duration_us":224101,"total_duration_us":10001585}],"total_count":14}} INFO [2022-01-10 05:49:22,501] jepsen worker nemesis - couchbase.util Rebalance check count: 708 WARN [2022-01-10 05:49:22,441] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1235339,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":10205697,"total_server_duration_us":685831,"encode_duration_us":18178,"operation_id":"0x992e","timeout_ms":10000,"last_server_duration_us":180894,"total_duration_us":10009725},{"operation_name":"upsert","last_dispatch_duration_us":6545738,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9112022,"total_server_duration_us":1087437,"encode_duration_us":16455,"operation_id":"0x98d6","timeout_ms":10000,"last_server_duration_us":115616,"total_duration_us":10008027},{"operation_name":"upsert","last_dispatch_duration_us":9475563,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":12748676,"total_server_duration_us":229577,"encode_duration_us":17128,"operation_id":"0x97da","timeout_ms":10000,"last_server_duration_us":94717,"total_duration_us":10006756},{"operation_name":"upsert","last_dispatch_duration_us":870990,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":10484948,"total_server_duration_us":117182,"encode_duration_us":22030,"operation_id":"0x9870","timeout_ms":10000,"last_server_duration_us":55790,"total_duration_us":10006344},{"operation_name":"upsert","last_dispatch_duration_us":9318485,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11616660,"total_server_duration_us":216861,"encode_duration_us":20874,"operation_id":"0x980c","timeout_ms":10000,"last_server_duration_us":87211,"total_duration_us":10005236},{"operation_name":"upsert","last_dispatch_duration_us":7006873,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":12865330,"total_server_duration_us":1166455,"encode_duration_us":18900,"operation_id":"0x986c","timeout_ms":10000,"last_server_duration_us":197287,"total_duration_us":10004950},{"operation_name":"upsert","last_dispatch_duration_us":9280390,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11601274,"total_server_duration_us":157967,"encode_duration_us":18117,"operation_id":"0x973a","timeout_ms":10000,"last_server_duration_us":54335,"total_duration_us":10004875},{"operation_name":"upsert","last_dispatch_duration_us":6287560,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":9594694,"total_server_duration_us":342857,"encode_duration_us":18673,"operation_id":"0x98a1","timeout_ms":10000,"last_server_duration_us":84574,"total_duration_us":10004775},{"operation_name":"upsert","last_dispatch_duration_us":6614782,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10423382,"total_server_duration_us":458283,"encode_duration_us":21144,"operation_id":"0x9895","timeout_ms":10000,"last_server_duration_us":92135,"total_duration_us":10004768},{"operation_name":"upsert","last_dispatch_duration_us":30336,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8803694,"total_server_duration_us":253436,"encode_duration_us":16983,"operation_id":"0x9944","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10004297}],"total_count":19}} WARN [2022-01-10 05:49:22,447] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9614460,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11049080,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9108975,"encode_duration_us":17790,"operation_id":"0x973c","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10005987}],"total_count":1}} WARN [2022-01-10 05:49:22,451] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8619206,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":12918967,"total_server_duration_us":716277,"encode_duration_us":16590,"operation_id":"0x974c","timeout_ms":10000,"last_server_duration_us":61012,"total_duration_us":10010098},{"operation_name":"upsert","last_dispatch_duration_us":8267462,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13682942,"total_server_duration_us":436176,"encode_duration_us":18246,"operation_id":"0x9775","timeout_ms":10000,"last_server_duration_us":69749,"total_duration_us":10009325},{"operation_name":"upsert","last_dispatch_duration_us":30644,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":921475,"total_server_duration_us":549496,"encode_duration_us":31150,"operation_id":"0x9839","timeout_ms":10000,"last_server_duration_us":18876,"total_duration_us":10008146},{"operation_name":"upsert","last_dispatch_duration_us":10084919,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10084919,"total_server_duration_us":212770,"encode_duration_us":17148,"operation_id":"0x98d3","timeout_ms":10000,"last_server_duration_us":212770,"total_duration_us":10008066},{"operation_name":"upsert","last_dispatch_duration_us":1572622,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":2815517,"total_server_duration_us":245703,"encode_duration_us":17149,"operation_id":"0x98a3","timeout_ms":10000,"last_server_duration_us":92589,"total_duration_us":10007417},{"operation_name":"upsert","last_dispatch_duration_us":729779,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":10359062,"total_server_duration_us":1282371,"encode_duration_us":17059,"operation_id":"0x98de","timeout_ms":10000,"last_server_duration_us":234557,"total_duration_us":10007371},{"operation_name":"upsert","last_dispatch_duration_us":29495,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":992732,"total_server_duration_us":552922,"encode_duration_us":18236,"operation_id":"0x980d","timeout_ms":10000,"last_server_duration_us":18646,"total_duration_us":10007286},{"operation_name":"upsert","last_dispatch_duration_us":1334912,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":11234544,"total_server_duration_us":125811,"encode_duration_us":16460,"operation_id":"0x98d1","timeout_ms":10000,"last_server_duration_us":106626,"total_duration_us":10007100},{"operation_name":"upsert","last_dispatch_duration_us":8763112,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13241668,"total_server_duration_us":545677,"encode_duration_us":16909,"operation_id":"0x9763","timeout_ms":10000,"last_server_duration_us":119804,"total_duration_us":10007088},{"operation_name":"upsert","last_dispatch_duration_us":2215991,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":8822383,"total_server_duration_us":1297111,"encode_duration_us":18055,"operation_id":"0x97a6","timeout_ms":10000,"last_server_duration_us":122688,"total_duration_us":10006118}],"total_count":28}} INFO [2022-01-10 05:49:22,466] jepsen worker 4 - jepsen.util 5224 :ok :read [1 49] INFO [2022-01-10 05:49:22,501] jepsen worker nemesis - couchbase.util Rebalance stuck count: 587 INFO [2022-01-10 05:49:22,502] jepsen worker 4 - jepsen.util 5224 :invoke :write [1 26] INFO [2022-01-10 05:49:22,554] jepsen worker 61 - jepsen.util 5281 :ok :read [20 :nil] INFO [2022-01-10 05:49:22,555] jepsen worker 61 - jepsen.util 5281 :invoke :read [20 nil] WARN [2022-01-10 05:49:22,601] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10702023,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11512706,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9386877,"encode_duration_us":17437,"operation_id":"0x9808","timeout_ms":10000,"last_server_duration_us":9277511,"total_duration_us":10001707},{"operation_name":"upsert","last_dispatch_duration_us":9313829,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17191049,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9053890,"encode_duration_us":16316,"operation_id":"0x9678","timeout_ms":10000,"last_server_duration_us":9035244,"total_duration_us":10004268},{"operation_name":"upsert","last_dispatch_duration_us":9041806,"last_local_id":"EFE9906200000005/0000000094AEA561","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9579820,"last_local_socket":"172.23.104.79:34136","total_server_duration_us":9323935,"encode_duration_us":16399,"operation_id":"0x9867","timeout_ms":10000,"last_server_duration_us":9025695,"total_duration_us":10008347}],"total_count":3}} WARN [2022-01-10 05:49:22,602] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10289514,"last_local_id":"EFE9906200000004/000000000C8781B9","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12469935,"last_local_socket":"172.23.104.79:38374","total_server_duration_us":9785780,"encode_duration_us":18491,"operation_id":"0x976f","timeout_ms":10000,"last_server_duration_us":9612754,"total_duration_us":10001374},{"operation_name":"upsert","last_dispatch_duration_us":9895660,"last_local_id":"EFE9906200000004/00000000FBB6977C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17261761,"last_local_socket":"172.23.104.79:52118","total_server_duration_us":9131757,"encode_duration_us":16730,"operation_id":"0x9734","timeout_ms":10000,"last_server_duration_us":9111791,"total_duration_us":10009212}],"total_count":2}} WARN [2022-01-10 05:49:22,702] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1447789,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5235833,"total_server_duration_us":691813,"encode_duration_us":16414,"operation_id":"0x9837","timeout_ms":10000,"last_server_duration_us":227859,"total_duration_us":10010374},{"operation_name":"upsert","last_dispatch_duration_us":45850,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9136290,"total_server_duration_us":58424,"encode_duration_us":26175,"operation_id":"0x97cd","timeout_ms":10000,"last_server_duration_us":20441,"total_duration_us":10009865},{"operation_name":"upsert","last_dispatch_duration_us":1272721,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":8807785,"total_server_duration_us":2917499,"encode_duration_us":28936,"operation_id":"0x9934","timeout_ms":10000,"last_server_duration_us":257006,"total_duration_us":10009685},{"operation_name":"upsert","last_dispatch_duration_us":5572956,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10333669,"total_server_duration_us":2173079,"encode_duration_us":16370,"operation_id":"0x97fe","timeout_ms":10000,"last_server_duration_us":117283,"total_duration_us":10009282},{"operation_name":"upsert","last_dispatch_duration_us":8954525,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10013229,"total_server_duration_us":528737,"encode_duration_us":17675,"operation_id":"0x9773","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10008613},{"operation_name":"upsert","last_dispatch_duration_us":134219,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":10005575,"total_server_duration_us":115629,"encode_duration_us":17228,"operation_id":"0x9791","timeout_ms":10000,"last_server_duration_us":19730,"total_duration_us":10008230},{"operation_name":"upsert","last_dispatch_duration_us":1508859,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9893256,"total_server_duration_us":2014429,"encode_duration_us":16820,"operation_id":"0x98ff","timeout_ms":10000,"last_server_duration_us":343918,"total_duration_us":10007150},{"operation_name":"upsert","last_dispatch_duration_us":1420721,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9368559,"total_server_duration_us":1049806,"encode_duration_us":19073,"operation_id":"0x9941","timeout_ms":10000,"last_server_duration_us":105344,"total_duration_us":10007024},{"operation_name":"upsert","last_dispatch_duration_us":1472569,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9444297,"total_server_duration_us":1334434,"encode_duration_us":23771,"operation_id":"0x9932","timeout_ms":10000,"last_server_duration_us":106144,"total_duration_us":10006194},{"operation_name":"upsert","last_dispatch_duration_us":9614460,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":11049080,"total_server_duration_us":9108975,"encode_duration_us":17790,"operation_id":"0x973c","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10005987}],"total_count":17}} INFO [2022-01-10 05:49:22,712] jepsen worker 74 - jepsen.util 5474 :info :write [24 34] :RequestTimeoutException INFO [2022-01-10 05:49:22,712] jepsen worker 74 - jepsen.util 5564 :invoke :read [24 nil] INFO [2022-01-10 05:49:22,845] jepsen worker 88 - jepsen.util 4948 :info :write [29 3] :RequestTimeoutException INFO [2022-01-10 05:49:22,846] jepsen worker 88 - jepsen.util 5038 :invoke :read [29 nil] INFO [2022-01-10 05:49:22,854] jepsen worker 43 - jepsen.util 4813 :info :write [14 48] :RequestTimeoutException INFO [2022-01-10 05:49:22,855] jepsen worker 43 - jepsen.util 4903 :invoke :read [14 nil] INFO [2022-01-10 05:49:22,904] jepsen worker 37 - jepsen.util 4627 :info :write [12 33] :RequestTimeoutException INFO [2022-01-10 05:49:22,905] jepsen worker 37 - jepsen.util 4717 :invoke :read [12 nil] INFO [2022-01-10 05:49:22,948] jepsen worker 70 - jepsen.util 5110 :fail :read [23 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:49:22,948] jepsen worker 70 - jepsen.util 5110 :invoke :write [23 27] INFO [2022-01-10 05:49:22,959] jepsen worker 55 - jepsen.util 5455 :info :write [18 30] :RequestTimeoutException INFO [2022-01-10 05:49:22,960] jepsen worker 55 - jepsen.util 5545 :invoke :read [18 nil] INFO [2022-01-10 05:49:22,986] jepsen worker 80 - jepsen.util 6110 :info :write [26 28] :RequestTimeoutException INFO [2022-01-10 05:49:22,987] jepsen worker 80 - jepsen.util 6200 :invoke :read [26 nil] INFO [2022-01-10 05:49:23,286] jepsen worker 5 - jepsen.util 5045 :ok :read [1 49] INFO [2022-01-10 05:49:23,287] jepsen worker 5 - jepsen.util 5045 :invoke :write [1 47] INFO [2022-01-10 05:49:23,296] jepsen worker 64 - jepsen.util 5014 :info :write [21 12] :RequestTimeoutException INFO [2022-01-10 05:49:23,297] jepsen worker 64 - jepsen.util 5104 :invoke :read [21 nil] INFO [2022-01-10 05:49:23,507] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:23,507] jepsen worker nemesis - couchbase.util Rebalance check count: 709 INFO [2022-01-10 05:49:23,507] jepsen worker nemesis - couchbase.util Rebalance stuck count: 588 INFO [2022-01-10 05:49:23,562] jepsen worker 75 - jepsen.util 4755 :info :write [25 45] :RequestTimeoutException INFO [2022-01-10 05:49:23,562] jepsen worker 75 - jepsen.util 4845 :invoke :write [25 45] INFO [2022-01-10 05:49:23,636] jepsen worker 52 - jepsen.util 6172 :info :write [17 7] :RequestTimeoutException INFO [2022-01-10 05:49:23,637] jepsen worker 52 - jepsen.util 6262 :invoke :write [17 48] INFO [2022-01-10 05:49:23,701] jepsen worker 15 - jepsen.util 4515 :info :write [5 17] :RequestTimeoutException INFO [2022-01-10 05:49:23,701] jepsen worker 15 - jepsen.util 4605 :invoke :read [5 nil] INFO [2022-01-10 05:49:23,704] jepsen worker 39 - jepsen.util 5259 :info :write [13 35] :RequestTimeoutException INFO [2022-01-10 05:49:23,705] jepsen worker 39 - jepsen.util 5349 :invoke :read [13 nil] INFO [2022-01-10 05:49:23,820] jepsen worker 31 - jepsen.util 5161 :info :write [10 26] :RequestTimeoutException INFO [2022-01-10 05:49:23,820] jepsen worker 31 - jepsen.util 5251 :invoke :read [10 nil] INFO [2022-01-10 05:49:23,843] jepsen worker 31 - jepsen.util 5251 :ok :read [10 31] INFO [2022-01-10 05:49:23,844] jepsen worker 31 - jepsen.util 5251 :invoke :read [10 nil] INFO [2022-01-10 05:49:23,856] jepsen worker 53 - jepsen.util 5993 :info :write [17 25] :RequestTimeoutException INFO [2022-01-10 05:49:23,857] jepsen worker 53 - jepsen.util 6083 :invoke :write [17 36] INFO [2022-01-10 05:49:23,869] jepsen worker 31 - jepsen.util 5251 :ok :read [10 31] INFO [2022-01-10 05:49:23,869] jepsen worker 31 - jepsen.util 5251 :invoke :write [10 21] INFO [2022-01-10 05:49:23,897] jepsen worker 76 - jepsen.util 4306 :ok :read [25 :nil] INFO [2022-01-10 05:49:23,898] jepsen worker 76 - jepsen.util 4306 :invoke :write [25 38] INFO [2022-01-10 05:49:23,929] jepsen worker 19 - jepsen.util 5059 :ok :read [6 25] INFO [2022-01-10 05:49:23,929] jepsen worker 19 - jepsen.util 5059 :invoke :write [6 42] INFO [2022-01-10 05:49:23,932] jepsen worker 61 - jepsen.util 5281 :ok :read [20 :nil] INFO [2022-01-10 05:49:23,932] jepsen worker 61 - jepsen.util 5281 :invoke :read [20 nil] INFO [2022-01-10 05:49:23,941] jepsen worker 12 - jepsen.util 5592 :info :write [4 18] :RequestTimeoutException INFO [2022-01-10 05:49:23,942] jepsen worker 12 - jepsen.util 5682 :invoke :read [4 nil] INFO [2022-01-10 05:49:23,976] jepsen worker 47 - jepsen.util 5177 :ok :read [15 23] INFO [2022-01-10 05:49:23,976] jepsen worker 47 - jepsen.util 5177 :invoke :write [15 24] INFO [2022-01-10 05:49:23,980] jepsen worker 63 - jepsen.util 4653 :ok :read [21 7] INFO [2022-01-10 05:49:23,981] jepsen worker 63 - jepsen.util 4653 :invoke :write [21 21] INFO [2022-01-10 05:49:24,008] jepsen worker 45 - jepsen.util 5445 :ok :read [15 23] INFO [2022-01-10 05:49:24,008] jepsen worker 45 - jepsen.util 5445 :invoke :read [15 nil] INFO [2022-01-10 05:49:24,047] jepsen worker 65 - jepsen.util 4655 :ok :read [21 7] INFO [2022-01-10 05:49:24,047] jepsen worker 65 - jepsen.util 4655 :invoke :read [21 nil] INFO [2022-01-10 05:49:24,064] jepsen worker 77 - jepsen.util 5567 :fail :read [25 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:49:24,065] jepsen worker 77 - jepsen.util 5567 :invoke :write [25 46] INFO [2022-01-10 05:49:24,084] jepsen worker 50 - jepsen.util 5180 :info :write [16 39] :RequestTimeoutException INFO [2022-01-10 05:49:24,085] jepsen worker 50 - jepsen.util 5270 :invoke :write [16 2] INFO [2022-01-10 05:49:24,191] jepsen worker 30 - jepsen.util 4710 :info :write [10 48] :SyncWriteAmbiguous INFO [2022-01-10 05:49:24,192] jepsen worker 30 - jepsen.util 4800 :invoke :read [10 nil] INFO [2022-01-10 05:49:24,211] jepsen worker 11 - jepsen.util 4961 :ok :read [3 3] INFO [2022-01-10 05:49:24,212] jepsen worker 11 - jepsen.util 4961 :invoke :write [3 45] INFO [2022-01-10 05:49:24,225] jepsen worker 30 - jepsen.util 4800 :ok :read [10 31] INFO [2022-01-10 05:49:24,226] jepsen worker 30 - jepsen.util 4800 :invoke :read [10 nil] INFO [2022-01-10 05:49:24,253] jepsen worker 30 - jepsen.util 4800 :ok :read [10 31] INFO [2022-01-10 05:49:24,253] jepsen worker 30 - jepsen.util 4800 :invoke :read [10 nil] INFO [2022-01-10 05:49:24,260] jepsen worker 33 - jepsen.util 5253 :fail :read [11 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:49:24,260] jepsen worker 72 - jepsen.util 5382 :fail :read [24 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:49:24,261] jepsen worker 33 - jepsen.util 5253 :invoke :read [11 nil] INFO [2022-01-10 05:49:24,261] jepsen worker 72 - jepsen.util 5382 :invoke :read [24 nil] INFO [2022-01-10 05:49:24,278] jepsen worker 43 - jepsen.util 4903 :ok :read [14 21] INFO [2022-01-10 05:49:24,279] jepsen worker 43 - jepsen.util 4903 :invoke :read [14 nil] INFO [2022-01-10 05:49:24,300] jepsen worker 30 - jepsen.util 4800 :ok :read [10 31] INFO [2022-01-10 05:49:24,300] jepsen worker 58 - jepsen.util 5008 :info :write [19 24] :RequestTimeoutException INFO [2022-01-10 05:49:24,300] jepsen worker 30 - jepsen.util 4800 :invoke :write [10 1] INFO [2022-01-10 05:49:24,301] jepsen worker 58 - jepsen.util 5098 :invoke :read [19 nil] INFO [2022-01-10 05:49:24,315] jepsen worker 7 - jepsen.util 5407 :ok :read [2 14] INFO [2022-01-10 05:49:24,315] jepsen worker 7 - jepsen.util 5407 :invoke :read [2 nil] INFO [2022-01-10 05:49:24,396] jepsen worker 42 - jepsen.util 4632 :info :write [14 34] :RequestTimeoutException INFO [2022-01-10 05:49:24,397] jepsen worker 42 - jepsen.util 4722 :invoke :write [14 6] INFO [2022-01-10 05:49:24,486] jepsen worker 45 - jepsen.util 5445 :ok :read [15 23] INFO [2022-01-10 05:49:24,487] jepsen worker 45 - jepsen.util 5445 :invoke :read [15 nil] INFO [2022-01-10 05:49:24,513] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:24,513] jepsen worker nemesis - couchbase.util Rebalance check count: 710 INFO [2022-01-10 05:49:24,513] jepsen worker nemesis - couchbase.util Rebalance stuck count: 589 INFO [2022-01-10 05:49:24,543] jepsen worker 65 - jepsen.util 4655 :ok :read [21 7] INFO [2022-01-10 05:49:24,544] jepsen worker 65 - jepsen.util 4655 :invoke :write [21 33] INFO [2022-01-10 05:49:24,551] jepsen worker 43 - jepsen.util 4903 :ok :read [14 21] INFO [2022-01-10 05:49:24,552] jepsen worker 43 - jepsen.util 4903 :invoke :read [14 nil] INFO [2022-01-10 05:49:24,562] jepsen worker 7 - jepsen.util 5407 :ok :read [2 14] INFO [2022-01-10 05:49:24,563] jepsen worker 7 - jepsen.util 5407 :invoke :write [2 12] INFO [2022-01-10 05:49:24,571] jepsen worker 33 - jepsen.util 5253 :ok :read [11 21] INFO [2022-01-10 05:49:24,572] jepsen worker 33 - jepsen.util 5253 :invoke :read [11 nil] INFO [2022-01-10 05:49:24,696] jepsen worker 61 - jepsen.util 5281 :fail :read [20 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:49:24,696] jepsen worker 61 - jepsen.util 5281 :invoke :write [20 39] INFO [2022-01-10 05:49:24,717] jepsen worker 37 - jepsen.util 4717 :ok :read [12 14] INFO [2022-01-10 05:49:24,717] jepsen worker 37 - jepsen.util 4717 :invoke :write [12 8] INFO [2022-01-10 05:49:24,727] jepsen worker 74 - jepsen.util 5564 :ok :read [24 18] INFO [2022-01-10 05:49:24,727] jepsen worker 74 - jepsen.util 5564 :invoke :read [24 nil] INFO [2022-01-10 05:49:24,762] jepsen worker 72 - jepsen.util 5382 :ok :read [24 18] INFO [2022-01-10 05:49:24,762] jepsen worker 72 - jepsen.util 5382 :invoke :read [24 nil] INFO [2022-01-10 05:49:24,769] jepsen worker 64 - jepsen.util 5104 :ok :read [21 7] INFO [2022-01-10 05:49:24,769] jepsen worker 64 - jepsen.util 5104 :invoke :write [21 27] INFO [2022-01-10 05:49:24,781] jepsen worker 45 - jepsen.util 5445 :ok :read [15 23] INFO [2022-01-10 05:49:24,786] jepsen worker 45 - jepsen.util 5445 :invoke :read [15 nil] INFO [2022-01-10 05:49:24,800] jepsen worker 73 - jepsen.util 4843 :info :write [24 6] :RequestTimeoutException INFO [2022-01-10 05:49:24,801] jepsen worker 73 - jepsen.util 4933 :invoke :write [24 43] INFO [2022-01-10 05:49:24,811] jepsen worker 12 - jepsen.util 5682 :ok :read [4 3] INFO [2022-01-10 05:49:24,812] jepsen worker 12 - jepsen.util 5682 :invoke :write [4 5] INFO [2022-01-10 05:49:24,867] jepsen worker 57 - jepsen.util 5727 :info :write [19 2] :RequestTimeoutException INFO [2022-01-10 05:49:24,868] jepsen worker 57 - jepsen.util 5817 :invoke :read [19 nil] INFO [2022-01-10 05:49:24,879] jepsen worker 43 - jepsen.util 4903 :ok :read [14 21] INFO [2022-01-10 05:49:24,879] jepsen worker 43 - jepsen.util 4903 :invoke :read [14 nil] INFO [2022-01-10 05:49:25,325] jepsen worker 84 - jepsen.util 4674 :ok :read [28 45] INFO [2022-01-10 05:49:25,325] jepsen worker 84 - jepsen.util 4674 :invoke :write [28 12] INFO [2022-01-10 05:49:25,345] jepsen worker 44 - jepsen.util 4994 :info :write [14 38] :RequestTimeoutException INFO [2022-01-10 05:49:25,346] jepsen worker 44 - jepsen.util 5084 :invoke :write [14 9] INFO [2022-01-10 05:49:25,430] jepsen worker 40 - jepsen.util 4630 :info :write [13 41] :RequestTimeoutException INFO [2022-01-10 05:49:25,430] jepsen worker 40 - jepsen.util 4720 :invoke :write [13 28] INFO [2022-01-10 05:49:25,518] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:25,518] jepsen worker nemesis - couchbase.util Rebalance check count: 711 INFO [2022-01-10 05:49:25,518] jepsen worker nemesis - couchbase.util Rebalance stuck count: 590 INFO [2022-01-10 05:49:25,720] jepsen worker 60 - jepsen.util 5370 :info :write [20 8] :RequestTimeoutException INFO [2022-01-10 05:49:25,721] jepsen worker 60 - jepsen.util 5460 :invoke :write [20 15] INFO [2022-01-10 05:49:25,773] jepsen worker 33 - jepsen.util 5253 :ok :read [11 21] INFO [2022-01-10 05:49:25,774] jepsen worker 33 - jepsen.util 5253 :invoke :write [11 13] INFO [2022-01-10 05:49:25,818] jepsen worker 72 - jepsen.util 5382 :ok :read [24 18] INFO [2022-01-10 05:49:25,819] jepsen worker 72 - jepsen.util 5382 :invoke :write [24 4] INFO [2022-01-10 05:49:25,826] jepsen worker 29 - jepsen.util 4889 :info :write [9 39] :RequestTimeoutException INFO [2022-01-10 05:49:25,826] jepsen worker 29 - jepsen.util 4979 :invoke :read [9 nil] INFO [2022-01-10 05:49:25,841] jepsen worker 25 - jepsen.util 4615 :info :write [8 38] :RequestTimeoutException INFO [2022-01-10 05:49:25,841] jepsen worker 25 - jepsen.util 4705 :invoke :write [8 29] INFO [2022-01-10 05:49:25,858] jepsen worker 45 - jepsen.util 5445 :ok :read [15 23] INFO [2022-01-10 05:49:25,859] jepsen worker 45 - jepsen.util 5445 :invoke :write [15 4] INFO [2022-01-10 05:49:25,910] jepsen worker 80 - jepsen.util 6200 :ok :read [26 10] INFO [2022-01-10 05:49:25,911] jepsen worker 80 - jepsen.util 6200 :invoke :write [26 34] INFO [2022-01-10 05:49:26,076] jepsen worker 43 - jepsen.util 4903 :ok :read [14 21] INFO [2022-01-10 05:49:26,077] jepsen worker 43 - jepsen.util 4903 :invoke :write [14 39] INFO [2022-01-10 05:49:26,101] jepsen worker 36 - jepsen.util 3906 :info :write [12 3] :RequestTimeoutException INFO [2022-01-10 05:49:26,101] jepsen worker 36 - jepsen.util 3996 :invoke :read [12 nil] INFO [2022-01-10 05:49:26,296] jepsen worker 21 - jepsen.util 4971 :info :write [7 23] :RequestTimeoutException INFO [2022-01-10 05:49:26,296] jepsen worker 21 - jepsen.util 5061 :invoke :read [7 nil] INFO [2022-01-10 05:49:26,399] jepsen worker 74 - jepsen.util 5564 :ok :read [24 18] INFO [2022-01-10 05:49:26,399] jepsen worker 74 - jepsen.util 5564 :invoke :write [24 18] INFO [2022-01-10 05:49:26,410] jepsen worker 85 - jepsen.util 5395 :info :write [28 49] :RequestTimeoutException INFO [2022-01-10 05:49:26,411] jepsen worker 85 - jepsen.util 5485 :invoke :read [28 nil] INFO [2022-01-10 05:49:26,525] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:26,525] jepsen worker nemesis - couchbase.util Rebalance check count: 712 INFO [2022-01-10 05:49:26,525] jepsen worker nemesis - couchbase.util Rebalance stuck count: 591 INFO [2022-01-10 05:49:26,532] jepsen worker 0 - jepsen.util 5130 :info :write [0 35] :RequestTimeoutException INFO [2022-01-10 05:49:26,533] jepsen worker 0 - jepsen.util 5220 :invoke :read [0 nil] INFO [2022-01-10 05:49:26,606] jepsen worker 49 - jepsen.util 4819 :info :write [16 10] :RequestTimeoutException INFO [2022-01-10 05:49:26,606] jepsen worker 49 - jepsen.util 4909 :invoke :read [16 nil] INFO [2022-01-10 05:49:26,674] jepsen worker 9 - jepsen.util 5499 :info :write [3 6] :RequestTimeoutException INFO [2022-01-10 05:49:26,675] jepsen worker 9 - jepsen.util 5589 :invoke :write [3 24] INFO [2022-01-10 05:49:27,166] jepsen worker 71 - jepsen.util 4661 :info :write [23 11] :RequestTimeoutException INFO [2022-01-10 05:49:27,166] jepsen worker 71 - jepsen.util 4751 :invoke :read [23 nil] INFO [2022-01-10 05:49:27,449] jepsen worker 24 - jepsen.util 4614 :info :write [8 6] :RequestTimeoutException INFO [2022-01-10 05:49:27,450] jepsen worker 24 - jepsen.util 4704 :invoke :write [8 11] INFO [2022-01-10 05:49:27,530] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:27,688] jepsen worker nemesis - couchbase.util Rebalance check count: 713 INFO [2022-01-10 05:49:27,688] jepsen worker nemesis - couchbase.util Rebalance stuck count: 592 INFO [2022-01-10 05:49:27,552] jepsen worker 10 - jepsen.util 4780 :info :write [3 41] :RequestTimeoutException INFO [2022-01-10 05:49:27,688] jepsen worker 10 - jepsen.util 4870 :invoke :write [3 17] INFO [2022-01-10 05:49:27,799] jepsen worker 36 - jepsen.util 3996 :ok :read [12 14] INFO [2022-01-10 05:49:27,800] jepsen worker 36 - jepsen.util 3996 :invoke :write [12 41] INFO [2022-01-10 05:49:27,847] jepsen worker 71 - jepsen.util 4751 :ok :read [23 26] INFO [2022-01-10 05:49:27,847] jepsen worker 71 - jepsen.util 4751 :invoke :read [23 nil] INFO [2022-01-10 05:49:28,126] jepsen worker 51 - jepsen.util 5811 :info :write [17 30] :RequestTimeoutException INFO [2022-01-10 05:49:28,127] jepsen worker 51 - jepsen.util 5901 :invoke :write [17 27] INFO [2022-01-10 05:49:28,204] jepsen worker 17 - jepsen.util 4877 :info :write [5 46] :RequestTimeoutException INFO [2022-01-10 05:49:28,205] jepsen worker 17 - jepsen.util 4967 :invoke :write [5 40] INFO [2022-01-10 05:49:28,280] jepsen worker 18 - jepsen.util 4878 :info :write [6 10] :RequestTimeoutException INFO [2022-01-10 05:49:28,280] jepsen worker 18 - jepsen.util 4968 :invoke :write [6 21] INFO [2022-01-10 05:49:28,322] jepsen worker 8 - jepsen.util 5138 :info :write [2 2] :RequestTimeoutException INFO [2022-01-10 05:49:28,322] jepsen worker 8 - jepsen.util 5228 :invoke :write [2 15] INFO [2022-01-10 05:49:28,354] jepsen worker 71 - jepsen.util 4751 :ok :read [23 26] INFO [2022-01-10 05:49:28,355] jepsen worker 71 - jepsen.util 4751 :invoke :read [23 nil] INFO [2022-01-10 05:49:28,511] jepsen worker 66 - jepsen.util 4926 :info :write [22 45] :RequestTimeoutException INFO [2022-01-10 05:49:28,512] jepsen worker 66 - jepsen.util 5016 :invoke :write [22 16] INFO [2022-01-10 05:49:28,693] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:28,693] jepsen worker nemesis - couchbase.util Rebalance check count: 714 INFO [2022-01-10 05:49:28,693] jepsen worker nemesis - couchbase.util Rebalance stuck count: 593 INFO [2022-01-10 05:49:29,036] jepsen worker 81 - jepsen.util 5571 :info :write [27 27] :RequestTimeoutException INFO [2022-01-10 05:49:29,036] jepsen worker 81 - jepsen.util 5661 :invoke :read [27 nil] INFO [2022-01-10 05:49:29,490] jepsen worker 35 - jepsen.util 4805 :info :write [11 22] :RequestTimeoutException INFO [2022-01-10 05:49:29,491] jepsen worker 35 - jepsen.util 4895 :invoke :write [11 46] INFO [2022-01-10 05:49:29,501] jepsen worker 48 - jepsen.util 4908 :info :write [16 3] :RequestTimeoutException INFO [2022-01-10 05:49:29,502] jepsen worker 48 - jepsen.util 4998 :invoke :write [16 41] INFO [2022-01-10 05:49:29,699] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:29,699] jepsen worker nemesis - couchbase.util Rebalance check count: 715 INFO [2022-01-10 05:49:29,699] jepsen worker nemesis - couchbase.util Rebalance stuck count: 594 INFO [2022-01-10 05:49:29,762] jepsen worker 62 - jepsen.util 5372 :info :write [20 12] :RequestTimeoutException INFO [2022-01-10 05:49:29,762] jepsen worker 62 - jepsen.util 5462 :invoke :read [20 nil] INFO [2022-01-10 05:49:29,775] jepsen worker 23 - jepsen.util 5153 :info :write [7 6] :RequestTimeoutException INFO [2022-01-10 05:49:29,775] jepsen worker 23 - jepsen.util 5243 :invoke :write [7 3] INFO [2022-01-10 05:49:29,926] jepsen worker 22 - jepsen.util 4612 :info :write [7 21] :RequestTimeoutException INFO [2022-01-10 05:49:29,926] jepsen worker 22 - jepsen.util 4702 :invoke :write [7 33] INFO [2022-01-10 05:49:30,002] jepsen worker 54 - jepsen.util 5274 :ok :read [18 14] INFO [2022-01-10 05:49:30,002] jepsen worker 54 - jepsen.util 5274 :invoke :read [18 nil] INFO [2022-01-10 05:49:30,454] jepsen worker 56 - jepsen.util 5276 :info :write [18 39] :SyncWriteAmbiguous INFO [2022-01-10 05:49:30,456] jepsen worker 56 - jepsen.util 5366 :invoke :read [18 nil] INFO [2022-01-10 05:49:30,551] jepsen worker 1 - jepsen.util 5041 :info :write [0 25] :RequestTimeoutException INFO [2022-01-10 05:49:30,552] jepsen worker 1 - jepsen.util 5131 :invoke :read [0 nil] INFO [2022-01-10 05:49:30,553] jepsen worker 88 - jepsen.util 5038 :ok :read [29 0] INFO [2022-01-10 05:49:30,553] jepsen worker 88 - jepsen.util 5038 :invoke :read [29 nil] INFO [2022-01-10 05:49:30,569] jepsen worker 56 - jepsen.util 5366 :ok :read [18 14] INFO [2022-01-10 05:49:30,570] jepsen worker 56 - jepsen.util 5366 :invoke :read [18 nil] INFO [2022-01-10 05:49:30,576] jepsen worker 88 - jepsen.util 5038 :ok :read [29 0] INFO [2022-01-10 05:49:30,577] jepsen worker 88 - jepsen.util 5038 :invoke :read [29 nil] INFO [2022-01-10 05:49:30,593] jepsen worker 56 - jepsen.util 5366 :ok :read [18 14] INFO [2022-01-10 05:49:30,593] jepsen worker 56 - jepsen.util 5366 :invoke :write [18 9] INFO [2022-01-10 05:49:30,599] jepsen worker 88 - jepsen.util 5038 :ok :read [29 0] INFO [2022-01-10 05:49:30,600] jepsen worker 88 - jepsen.util 5038 :invoke :write [29 27] INFO [2022-01-10 05:49:30,702] jepsen worker 59 - jepsen.util 4739 :info :write [19 6] :RequestTimeoutException INFO [2022-01-10 05:49:30,702] jepsen worker 59 - jepsen.util 4829 :invoke :read [19 nil] INFO [2022-01-10 05:49:30,705] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:30,705] jepsen worker nemesis - couchbase.util Rebalance check count: 716 INFO [2022-01-10 05:49:30,705] jepsen worker nemesis - couchbase.util Rebalance stuck count: 595 INFO [2022-01-10 05:49:30,716] jepsen worker 20 - jepsen.util 4700 :info :write [6 48] :RequestTimeoutException INFO [2022-01-10 05:49:30,717] jepsen worker 20 - jepsen.util 4790 :invoke :read [6 nil] INFO [2022-01-10 05:49:30,750] jepsen worker 78 - jepsen.util 6198 :info :write [26 24] :RequestTimeoutException INFO [2022-01-10 05:49:30,751] jepsen worker 78 - jepsen.util 6288 :invoke :read [26 nil] INFO [2022-01-10 05:49:30,841] jepsen worker 82 - jepsen.util 5032 :info :write [27 11] :RequestTimeoutException INFO [2022-01-10 05:49:30,842] jepsen worker 6 - jepsen.util 4956 :info :write [2 7] :RequestTimeoutException INFO [2022-01-10 05:49:30,842] jepsen worker 82 - jepsen.util 5122 :invoke :write [27 8] INFO [2022-01-10 05:49:30,844] jepsen worker 6 - jepsen.util 5046 :invoke :read [2 nil] WARN [2022-01-10 05:49:30,850] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18116987,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":27108725,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9049239,"encode_duration_us":27743,"operation_id":"0x95ac","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":10002964},{"operation_name":"upsert","last_dispatch_duration_us":18359140,"last_local_id":"EFE9906200000001/00000000CC853C83","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18359140,"last_local_socket":"172.23.104.79:52130","total_server_duration_us":9700088,"encode_duration_us":16978,"operation_id":"0x9809","timeout_ms":10000,"last_server_duration_us":9700088,"total_duration_us":10009712},{"operation_name":"upsert","last_dispatch_duration_us":18005555,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":27146556,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9041104,"encode_duration_us":18035,"operation_id":"0x9764","timeout_ms":10000,"last_server_duration_us":9021452,"total_duration_us":10006950}],"total_count":3}} WARN [2022-01-10 05:49:30,850] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":17129396,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":17129396,"total_server_duration_us":38,"operation_id":"0x9799","timeout_ms":30000,"last_server_duration_us":38,"total_duration_us":17131412},{"operation_name":"get","last_dispatch_duration_us":17017079,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":17017079,"total_server_duration_us":219495,"operation_id":"0x986e","timeout_ms":30000,"last_server_duration_us":219495,"total_duration_us":17019077},{"operation_name":"get","last_dispatch_duration_us":14702209,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":14702209,"total_server_duration_us":43,"operation_id":"0x99dd","timeout_ms":30000,"last_server_duration_us":43,"total_duration_us":14704997},{"operation_name":"get","last_dispatch_duration_us":13691281,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":13691281,"total_server_duration_us":32,"operation_id":"0x9835","timeout_ms":30000,"last_server_duration_us":32,"total_duration_us":13692442},{"operation_name":"upsert","last_dispatch_duration_us":1491516,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":11165276,"total_server_duration_us":887233,"encode_duration_us":21872,"operation_id":"0x996e","timeout_ms":10000,"last_server_duration_us":66168,"total_duration_us":10009458},{"operation_name":"upsert","last_dispatch_duration_us":1296617,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":9940709,"total_server_duration_us":1666969,"encode_duration_us":18919,"operation_id":"0x9931","timeout_ms":10000,"last_server_duration_us":292056,"total_duration_us":10008584},{"operation_name":"upsert","last_dispatch_duration_us":989089,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":10803538,"total_server_duration_us":1445651,"encode_duration_us":19544,"operation_id":"0x9971","timeout_ms":10000,"last_server_duration_us":326662,"total_duration_us":10007307},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","encode_duration_us":16295,"operation_id":"0x9981","timeout_ms":10000,"total_duration_us":10007170},{"operation_name":"upsert","last_dispatch_duration_us":15456902,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":15456902,"total_server_duration_us":803871,"encode_duration_us":16981,"operation_id":"0x99a4","timeout_ms":10000,"last_server_duration_us":803871,"total_duration_us":10006409},{"operation_name":"upsert","last_dispatch_duration_us":3191426,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":10139182,"total_server_duration_us":4035398,"encode_duration_us":16654,"operation_id":"0x9aa6","timeout_ms":10000,"last_server_duration_us":1092893,"total_duration_us":10005891}],"total_count":18}} INFO [2022-01-10 05:49:30,886] jepsen worker 71 - jepsen.util 4751 :ok :read [23 26] INFO [2022-01-10 05:49:30,887] jepsen worker 71 - jepsen.util 4751 :invoke :read [23 nil] INFO [2022-01-10 05:49:30,972] jepsen worker 38 - jepsen.util 4718 :info :write [12 12] :RequestTimeoutException INFO [2022-01-10 05:49:30,973] jepsen worker 38 - jepsen.util 4808 :invoke :write [12 25] INFO [2022-01-10 05:49:30,984] jepsen worker 26 - jepsen.util 4886 :info :write [8 28] :RequestTimeoutException INFO [2022-01-10 05:49:30,985] jepsen worker 26 - jepsen.util 4976 :invoke :read [8 nil] INFO [2022-01-10 05:49:30,995] jepsen worker 89 - jepsen.util 5399 :info :write [29 3] :RequestTimeoutException INFO [2022-01-10 05:49:30,995] jepsen worker 89 - jepsen.util 5489 :invoke :read [29 nil] INFO [2022-01-10 05:49:31,025] jepsen worker 32 - jepsen.util 5252 :info :write [10 43] :RequestTimeoutException INFO [2022-01-10 05:49:31,025] jepsen worker 32 - jepsen.util 5342 :invoke :write [10 15] INFO [2022-01-10 05:49:31,032] jepsen worker 89 - jepsen.util 5489 :ok :read [29 0] INFO [2022-01-10 05:49:31,032] jepsen worker 89 - jepsen.util 5489 :invoke :read [29 nil] INFO [2022-01-10 05:49:31,055] jepsen worker 89 - jepsen.util 5489 :ok :read [29 0] INFO [2022-01-10 05:49:31,056] jepsen worker 89 - jepsen.util 5489 :invoke :write [29 32] INFO [2022-01-10 05:49:31,075] jepsen worker 13 - jepsen.util 5863 :info :write [4 10] :RequestTimeoutException INFO [2022-01-10 05:49:31,076] jepsen worker 13 - jepsen.util 5953 :invoke :write [4 10] INFO [2022-01-10 05:49:31,150] jepsen worker 34 - jepsen.util 4804 :info :write [11 39] :RequestTimeoutException INFO [2022-01-10 05:49:31,150] jepsen worker 34 - jepsen.util 4894 :invoke :write [11 40] INFO [2022-01-10 05:49:31,285] jepsen worker 15 - jepsen.util 4605 :ok :read [5 25] INFO [2022-01-10 05:49:31,286] jepsen worker 15 - jepsen.util 4605 :invoke :write [5 17] INFO [2022-01-10 05:49:31,286] jepsen worker 57 - jepsen.util 5817 :ok :read [19 31] INFO [2022-01-10 05:49:31,287] jepsen worker 57 - jepsen.util 5817 :invoke :read [19 nil] INFO [2022-01-10 05:49:31,375] jepsen worker 67 - jepsen.util 4657 :info :write [22 23] :RequestTimeoutException INFO [2022-01-10 05:49:31,376] jepsen worker 67 - jepsen.util 4747 :invoke :read [22 nil] INFO [2022-01-10 05:49:31,391] jepsen worker 86 - jepsen.util 4136 :info :write [28 48] :RequestTimeoutException INFO [2022-01-10 05:49:31,392] jepsen worker 86 - jepsen.util 4226 :invoke :write [28 1] INFO [2022-01-10 05:49:31,453] jepsen worker 79 - jepsen.util 5839 :info :write [26 12] :RequestTimeoutException INFO [2022-01-10 05:49:31,453] jepsen worker 79 - jepsen.util 5929 :invoke :write [26 33] INFO [2022-01-10 05:49:31,478] jepsen worker 68 - jepsen.util 5018 :ok :read [22 41] INFO [2022-01-10 05:49:31,479] jepsen worker 68 - jepsen.util 5018 :invoke :read [22 nil] INFO [2022-01-10 05:49:31,591] jepsen worker 71 - jepsen.util 4751 :fail :read [23 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:49:31,591] jepsen worker 71 - jepsen.util 4751 :invoke :write [23 35] INFO [2022-01-10 05:49:31,609] jepsen worker 78 - jepsen.util 6288 :ok :read [26 10] INFO [2022-01-10 05:49:31,610] jepsen worker 78 - jepsen.util 6288 :invoke :write [26 14] INFO [2022-01-10 05:49:31,711] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:31,711] jepsen worker nemesis - couchbase.util Rebalance check count: 717 INFO [2022-01-10 05:49:31,711] jepsen worker nemesis - couchbase.util Rebalance stuck count: 596 INFO [2022-01-10 05:49:31,872] jepsen worker 2 - jepsen.util 5222 :info :write [0 9] :RequestTimeoutException INFO [2022-01-10 05:49:31,873] jepsen worker 2 - jepsen.util 5312 :invoke :read [0 nil] INFO [2022-01-10 05:49:31,881] jepsen worker 41 - jepsen.util 4901 :info :write [13 46] :RequestTimeoutException INFO [2022-01-10 05:49:31,882] jepsen worker 41 - jepsen.util 4991 :invoke :write [13 7] INFO [2022-01-10 05:49:31,885] jepsen worker 58 - jepsen.util 5098 :ok :read [19 31] INFO [2022-01-10 05:49:31,886] jepsen worker 58 - jepsen.util 5098 :invoke :read [19 nil] INFO [2022-01-10 05:49:31,886] jepsen worker 14 - jepsen.util 6044 :info :write [4 49] :RequestTimeoutException INFO [2022-01-10 05:49:31,886] jepsen worker 14 - jepsen.util 6134 :invoke :read [4 nil] INFO [2022-01-10 05:49:31,892] jepsen worker 28 - jepsen.util 4618 :info :write [9 35] :RequestTimeoutException INFO [2022-01-10 05:49:31,892] jepsen worker 28 - jepsen.util 4708 :invoke :read [9 nil] INFO [2022-01-10 05:49:32,051] jepsen worker 3 - jepsen.util 5043 :info :write [1 49] :RequestTimeoutException INFO [2022-01-10 05:49:32,052] jepsen worker 3 - jepsen.util 5133 :invoke :write [1 19] INFO [2022-01-10 05:49:32,145] jepsen worker 62 - jepsen.util 5462 :ok :read [20 :nil] INFO [2022-01-10 05:49:32,146] jepsen worker 62 - jepsen.util 5462 :invoke :read [20 nil] INFO [2022-01-10 05:49:32,236] jepsen worker 27 - jepsen.util 5427 :info :write [9 6] :RequestTimeoutException INFO [2022-01-10 05:49:32,237] jepsen worker 27 - jepsen.util 5517 :invoke :write [9 35] WARN [2022-01-10 05:49:32,252] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":6772547,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":9701357,"total_server_duration_us":801085,"encode_duration_us":19582,"operation_id":"0x9ac0","timeout_ms":10000,"last_server_duration_us":598552,"total_duration_us":10057467},{"operation_name":"upsert","last_dispatch_duration_us":1597896,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":6122376,"total_server_duration_us":1450570,"encode_duration_us":17073,"operation_id":"0x99cd","timeout_ms":10000,"last_server_duration_us":352153,"total_duration_us":10010024},{"operation_name":"upsert","last_dispatch_duration_us":3806424,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":11922115,"total_server_duration_us":385434,"encode_duration_us":16401,"operation_id":"0x997f","timeout_ms":10000,"last_server_duration_us":202322,"total_duration_us":10008603},{"operation_name":"upsert","last_dispatch_duration_us":3874521,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":3874521,"total_server_duration_us":201901,"encode_duration_us":20853,"operation_id":"0x9afe","timeout_ms":10000,"last_server_duration_us":201901,"total_duration_us":10005053},{"operation_name":"upsert","last_dispatch_duration_us":740246,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10633492,"total_server_duration_us":3371025,"encode_duration_us":26689,"operation_id":"0x9aeb","timeout_ms":10000,"last_server_duration_us":282993,"total_duration_us":10004879},{"operation_name":"upsert","last_dispatch_duration_us":6861139,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":6861139,"total_server_duration_us":214281,"encode_duration_us":18780,"operation_id":"0x9b26","timeout_ms":10000,"last_server_duration_us":214281,"total_duration_us":10004549},{"operation_name":"upsert","last_dispatch_duration_us":2071006,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":10618337,"total_server_duration_us":3563870,"encode_duration_us":16262,"operation_id":"0x99a3","timeout_ms":10000,"last_server_duration_us":772281,"total_duration_us":10004027},{"operation_name":"upsert","last_dispatch_duration_us":2065483,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":11745189,"total_server_duration_us":1496251,"encode_duration_us":16395,"operation_id":"0x9a03","timeout_ms":10000,"last_server_duration_us":139698,"total_duration_us":10003610},{"operation_name":"upsert","last_dispatch_duration_us":9872946,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":12923729,"total_server_duration_us":539149,"encode_duration_us":18697,"operation_id":"0x9a02","timeout_ms":10000,"last_server_duration_us":256539,"total_duration_us":10002896},{"operation_name":"upsert","last_dispatch_duration_us":623732,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9689243,"total_server_duration_us":2186483,"encode_duration_us":16538,"operation_id":"0x9b2c","timeout_ms":10000,"last_server_duration_us":114786,"total_duration_us":10002618}],"total_count":17}} INFO [2022-01-10 05:49:32,270] jepsen worker 69 - jepsen.util 4659 :info :write [23 8] :RequestTimeoutException INFO [2022-01-10 05:49:32,271] jepsen worker 69 - jepsen.util 4749 :invoke :read [23 nil] WARN [2022-01-10 05:49:32,353] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9662692,"last_local_id":"EFE9906200000002/00000000B5693B6C","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11157113,"last_local_socket":"172.23.104.79:52136","total_server_duration_us":9636023,"encode_duration_us":16265,"operation_id":"0x9a87","timeout_ms":10000,"last_server_duration_us":9439093,"total_duration_us":10007436}],"total_count":1}} WARN [2022-01-10 05:49:32,353] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":11658950,"last_local_id":"EFE9906200000006/00000000A12A8681","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17237883,"last_local_socket":"172.23.104.79:38420","total_server_duration_us":10515466,"encode_duration_us":17362,"operation_id":"0x98d0","timeout_ms":10000,"last_server_duration_us":9956362,"total_duration_us":10010774}],"total_count":1}} WARN [2022-01-10 05:49:32,453] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8883192,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":13558111,"total_server_duration_us":1021866,"encode_duration_us":16194,"operation_id":"0x9a0e","timeout_ms":10000,"last_server_duration_us":377630,"total_duration_us":10008849},{"operation_name":"upsert","last_dispatch_duration_us":9484845,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":13961139,"total_server_duration_us":9446905,"encode_duration_us":16940,"operation_id":"0x9a01","timeout_ms":10000,"last_server_duration_us":9139501,"total_duration_us":10008304},{"operation_name":"upsert","last_dispatch_duration_us":8795816,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":15677298,"total_server_duration_us":1759897,"encode_duration_us":17498,"operation_id":"0x99d4","timeout_ms":10000,"last_server_duration_us":358586,"total_duration_us":10007580},{"operation_name":"upsert","last_dispatch_duration_us":2211032,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":11289837,"total_server_duration_us":507730,"encode_duration_us":16380,"operation_id":"0x99a2","timeout_ms":10000,"last_server_duration_us":133099,"total_duration_us":10007451},{"operation_name":"upsert","last_dispatch_duration_us":1238395,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10942873,"total_server_duration_us":1011676,"encode_duration_us":17923,"operation_id":"0x997b","timeout_ms":10000,"last_server_duration_us":203167,"total_duration_us":10001969},{"operation_name":"upsert","last_dispatch_duration_us":2076719,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":11203817,"total_server_duration_us":4035608,"encode_duration_us":26637,"operation_id":"0x997e","timeout_ms":10000,"last_server_duration_us":201059,"total_duration_us":10001850},{"operation_name":"upsert","last_dispatch_duration_us":1388057,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":10804312,"total_server_duration_us":2197720,"encode_duration_us":17871,"operation_id":"0x9974","timeout_ms":10000,"last_server_duration_us":210834,"total_duration_us":10001506},{"operation_name":"upsert","last_dispatch_duration_us":1290228,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10041108,"total_server_duration_us":245909,"encode_duration_us":16524,"operation_id":"0x99b2","timeout_ms":10000,"last_server_duration_us":125429,"total_duration_us":10001099},{"operation_name":"get","last_dispatch_duration_us":1059161,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1059161,"total_server_duration_us":27,"operation_id":"0x9a51","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":6224141},{"operation_name":"get","last_dispatch_duration_us":1303256,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":1303256,"total_server_duration_us":63310,"operation_id":"0x9a64","timeout_ms":30000,"last_server_duration_us":63310,"total_duration_us":5962948}],"total_count":20}} WARN [2022-01-10 05:49:32,453] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":3606250,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":12014292,"total_server_duration_us":2799041,"encode_duration_us":18758,"operation_id":"0x9a59","timeout_ms":10000,"last_server_duration_us":164917,"total_duration_us":10010372},{"operation_name":"upsert","last_dispatch_duration_us":1792765,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":9900991,"total_server_duration_us":2519696,"encode_duration_us":18716,"operation_id":"0x9b25","timeout_ms":10000,"last_server_duration_us":217316,"total_duration_us":10009402},{"operation_name":"upsert","last_dispatch_duration_us":4415806,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7950286,"total_server_duration_us":384369,"encode_duration_us":30165,"operation_id":"0x9acc","timeout_ms":10000,"last_server_duration_us":211049,"total_duration_us":10008170},{"operation_name":"upsert","last_dispatch_duration_us":5487865,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10309333,"total_server_duration_us":248191,"encode_duration_us":23720,"operation_id":"0x9ab5","timeout_ms":10000,"last_server_duration_us":84720,"total_duration_us":10007824},{"operation_name":"upsert","last_dispatch_duration_us":9662692,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":11157113,"total_server_duration_us":9636023,"encode_duration_us":16265,"operation_id":"0x9a87","timeout_ms":10000,"last_server_duration_us":9439093,"total_duration_us":10007436},{"operation_name":"upsert","last_dispatch_duration_us":2374929,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":5028520,"total_server_duration_us":62857,"encode_duration_us":16616,"operation_id":"0x99de","timeout_ms":10000,"last_server_duration_us":44058,"total_duration_us":10005530},{"operation_name":"upsert","last_dispatch_duration_us":3846453,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":13575132,"total_server_duration_us":2064860,"encode_duration_us":16526,"operation_id":"0x9a36","timeout_ms":10000,"last_server_duration_us":484846,"total_duration_us":10001808},{"operation_name":"upsert","last_dispatch_duration_us":2873873,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":2873873,"total_server_duration_us":40695,"encode_duration_us":16661,"operation_id":"0x9a0b","timeout_ms":10000,"last_server_duration_us":40695,"total_duration_us":10001377},{"operation_name":"upsert","last_dispatch_duration_us":9823955,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":9823955,"total_server_duration_us":386197,"encode_duration_us":17920,"operation_id":"0x9b36","timeout_ms":10000,"last_server_duration_us":386197,"total_duration_us":10001318},{"operation_name":"upsert","last_dispatch_duration_us":35401,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":8841626,"total_server_duration_us":256147,"encode_duration_us":18069,"operation_id":"0x996d","timeout_ms":10000,"last_server_duration_us":21895,"total_duration_us":10001311}],"total_count":15}} WARN [2022-01-10 05:49:32,455] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9700662,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9700662,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9688060,"encode_duration_us":16190,"operation_id":"0x98a2","timeout_ms":10000,"last_server_duration_us":9688060,"total_duration_us":10004610},{"operation_name":"upsert","last_dispatch_duration_us":9830443,"last_local_id":"EFE9906200000003/000000005A8DE651","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18966734,"last_local_socket":"172.23.104.79:52078","total_server_duration_us":9875847,"encode_duration_us":26175,"operation_id":"0x97cd","timeout_ms":10000,"last_server_duration_us":9817423,"total_duration_us":10009865}],"total_count":2}} WARN [2022-01-10 05:49:32,456] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":6172694,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":15704174,"total_server_duration_us":1218366,"encode_duration_us":16416,"operation_id":"0x9a0d","timeout_ms":10000,"last_server_duration_us":355900,"total_duration_us":10009231},{"operation_name":"upsert","last_dispatch_duration_us":6077554,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":13760090,"total_server_duration_us":909156,"encode_duration_us":16792,"operation_id":"0x9a65","timeout_ms":10000,"last_server_duration_us":530059,"total_duration_us":10008384},{"operation_name":"upsert","last_dispatch_duration_us":9051491,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":9535253,"total_server_duration_us":9326034,"encode_duration_us":21139,"operation_id":"0x9a38","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10007161},{"operation_name":"upsert","last_dispatch_duration_us":4371754,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":7814208,"total_server_duration_us":497974,"encode_duration_us":16157,"operation_id":"0x9ae6","timeout_ms":10000,"last_server_duration_us":231421,"total_duration_us":10006974},{"operation_name":"upsert","last_dispatch_duration_us":1758147,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6076588,"total_server_duration_us":393558,"encode_duration_us":17331,"operation_id":"0x9952","timeout_ms":10000,"last_server_duration_us":88394,"total_duration_us":10006756},{"operation_name":"upsert","last_dispatch_duration_us":761066,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":7753004,"total_server_duration_us":3969688,"encode_duration_us":16756,"operation_id":"0x9b00","timeout_ms":10000,"last_server_duration_us":163758,"total_duration_us":10005597},{"operation_name":"upsert","last_dispatch_duration_us":5766528,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10091037,"total_server_duration_us":1282062,"encode_duration_us":16265,"operation_id":"0x9a92","timeout_ms":10000,"last_server_duration_us":671958,"total_duration_us":10004207},{"operation_name":"upsert","last_dispatch_duration_us":5913178,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":10374549,"total_server_duration_us":1302424,"encode_duration_us":17373,"operation_id":"0x9acb","timeout_ms":10000,"last_server_duration_us":380936,"total_duration_us":10003251},{"operation_name":"upsert","last_dispatch_duration_us":2175100,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6023207,"total_server_duration_us":1130476,"encode_duration_us":16555,"operation_id":"0x997d","timeout_ms":10000,"last_server_duration_us":688916,"total_duration_us":10001871},{"operation_name":"upsert","last_dispatch_duration_us":6171043,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000004/00000000FBB6977C","last_local_socket":"172.23.104.79:52118","total_dispatch_duration_us":9335913,"total_server_duration_us":570761,"encode_duration_us":17678,"operation_id":"0x9b28","timeout_ms":10000,"last_server_duration_us":192306,"total_duration_us":10001742}],"total_count":22}} INFO [2022-01-10 05:49:32,526] jepsen worker 4 - jepsen.util 5224 :info :write [1 26] :RequestTimeoutException WARN [2022-01-10 05:49:32,553] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":10678725,"last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11986569,"last_local_socket":"172.23.104.79:38378","total_server_duration_us":9535601,"encode_duration_us":16355,"operation_id":"0x98d5","timeout_ms":10000,"last_server_duration_us":9382944,"total_duration_us":10005505},{"operation_name":"upsert","last_dispatch_duration_us":9484845,"last_local_id":"EFE9906200000005/000000001CDA61C4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13961139,"last_local_socket":"172.23.104.79:52098","total_server_duration_us":9446905,"encode_duration_us":16940,"operation_id":"0x9a01","timeout_ms":10000,"last_server_duration_us":9139501,"total_duration_us":10008304}],"total_count":2}} WARN [2022-01-10 05:49:32,608] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9051491,"last_local_id":"EFE9906200000004/000000008D35FDEF","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9535253,"last_local_socket":"172.23.104.79:34126","total_server_duration_us":9326034,"encode_duration_us":21139,"operation_id":"0x9a38","timeout_ms":10000,"last_server_duration_us":9026756,"total_duration_us":10007161}],"total_count":1}} INFO [2022-01-10 05:49:32,608] jepsen worker 4 - jepsen.util 5314 :invoke :write [1 9] INFO [2022-01-10 05:49:32,698] jepsen worker 14 - jepsen.util 6134 :ok :read [4 3] INFO [2022-01-10 05:49:32,698] jepsen worker 14 - jepsen.util 6134 :invoke :read [4 nil] WARN [2022-01-10 05:49:32,708] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2445282,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9910423,"total_server_duration_us":1006954,"encode_duration_us":16520,"operation_id":"0x9ad3","timeout_ms":10000,"last_server_duration_us":20760,"total_duration_us":10010652},{"operation_name":"upsert","last_dispatch_duration_us":9527892,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":9804035,"total_server_duration_us":538212,"encode_duration_us":16768,"operation_id":"0x99b0","timeout_ms":10000,"last_server_duration_us":396810,"total_duration_us":10009340},{"operation_name":"upsert","last_dispatch_duration_us":13199310,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13199310,"total_server_duration_us":592876,"encode_duration_us":16582,"operation_id":"0x9a0c","timeout_ms":10000,"last_server_duration_us":592876,"total_duration_us":10006292},{"operation_name":"upsert","last_dispatch_duration_us":2484773,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":5726094,"total_server_duration_us":826942,"encode_duration_us":16782,"operation_id":"0x9a90","timeout_ms":10000,"last_server_duration_us":314859,"total_duration_us":10006244},{"operation_name":"upsert","last_dispatch_duration_us":1226947,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":9232932,"total_server_duration_us":865734,"encode_duration_us":16988,"operation_id":"0x9b37","timeout_ms":10000,"last_server_duration_us":104069,"total_duration_us":10006198},{"operation_name":"upsert","last_dispatch_duration_us":32227,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":908524,"total_server_duration_us":546732,"encode_duration_us":17312,"operation_id":"0x9973","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10005752},{"operation_name":"upsert","last_dispatch_duration_us":3420084,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":7256807,"total_server_duration_us":382052,"encode_duration_us":16595,"operation_id":"0x9a63","timeout_ms":10000,"last_server_duration_us":65906,"total_duration_us":10005625},{"operation_name":"upsert","last_dispatch_duration_us":30639,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":950104,"total_server_duration_us":560337,"encode_duration_us":16602,"operation_id":"0x99a1","timeout_ms":10000,"last_server_duration_us":19262,"total_duration_us":10005190},{"operation_name":"upsert","last_dispatch_duration_us":1790615,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":10009524,"total_server_duration_us":1082529,"encode_duration_us":18919,"operation_id":"0x99e8","timeout_ms":10000,"last_server_duration_us":80398,"total_duration_us":10004618},{"operation_name":"upsert","last_dispatch_duration_us":13566741,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":13566741,"total_server_duration_us":2904895,"encode_duration_us":18518,"operation_id":"0x99df","timeout_ms":10000,"last_server_duration_us":2904895,"total_duration_us":10003609}],"total_count":17}} INFO [2022-01-10 05:49:32,717] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:32,717] jepsen worker nemesis - couchbase.util Rebalance check count: 718 INFO [2022-01-10 05:49:32,717] jepsen worker nemesis - couchbase.util Rebalance stuck count: 597 INFO [2022-01-10 05:49:32,822] jepsen worker 14 - jepsen.util 6134 :ok :read [4 3] INFO [2022-01-10 05:49:32,823] jepsen worker 14 - jepsen.util 6134 :invoke :read [4 nil] INFO [2022-01-10 05:49:32,827] jepsen worker 20 - jepsen.util 4790 :ok :read [6 25] INFO [2022-01-10 05:49:32,828] jepsen worker 20 - jepsen.util 4790 :invoke :read [6 nil] INFO [2022-01-10 05:49:32,969] jepsen worker 85 - jepsen.util 5485 :ok :read [28 45] INFO [2022-01-10 05:49:32,970] jepsen worker 85 - jepsen.util 5485 :invoke :read [28 nil] INFO [2022-01-10 05:49:32,976] jepsen worker 70 - jepsen.util 5110 :info :write [23 27] :RequestTimeoutException INFO [2022-01-10 05:49:32,977] jepsen worker 70 - jepsen.util 5200 :invoke :read [23 nil] INFO [2022-01-10 05:49:33,254] jepsen worker 31 - jepsen.util 5251 :info :write [10 21] :SyncWriteAmbiguous INFO [2022-01-10 05:49:33,255] jepsen worker 31 - jepsen.util 5341 :invoke :read [10 nil] INFO [2022-01-10 05:49:33,306] jepsen worker 5 - jepsen.util 5045 :info :write [1 47] :RequestTimeoutException INFO [2022-01-10 05:49:33,307] jepsen worker 5 - jepsen.util 5135 :invoke :write [1 26] INFO [2022-01-10 05:49:33,338] jepsen worker 81 - jepsen.util 5661 :ok :read [27 30] INFO [2022-01-10 05:49:33,339] jepsen worker 81 - jepsen.util 5661 :invoke :read [27 nil] INFO [2022-01-10 05:49:33,340] jepsen worker 57 - jepsen.util 5817 :ok :read [19 31] INFO [2022-01-10 05:49:33,340] jepsen worker 57 - jepsen.util 5817 :invoke :write [19 6] INFO [2022-01-10 05:49:33,343] jepsen worker 2 - jepsen.util 5312 :ok :read [0 46] INFO [2022-01-10 05:49:33,343] jepsen worker 2 - jepsen.util 5312 :invoke :write [0 30] INFO [2022-01-10 05:49:33,518] jepsen worker 69 - jepsen.util 4749 :ok :read [23 26] INFO [2022-01-10 05:49:33,519] jepsen worker 69 - jepsen.util 4749 :invoke :write [23 25] INFO [2022-01-10 05:49:33,519] jepsen worker 14 - jepsen.util 6134 :ok :read [4 3] INFO [2022-01-10 05:49:33,520] jepsen worker 14 - jepsen.util 6134 :invoke :read [4 nil] INFO [2022-01-10 05:49:33,540] jepsen worker 70 - jepsen.util 5200 :ok :read [23 26] INFO [2022-01-10 05:49:33,541] jepsen worker 70 - jepsen.util 5200 :invoke :write [23 39] INFO [2022-01-10 05:49:33,581] jepsen worker 75 - jepsen.util 4845 :info :write [25 45] :RequestTimeoutException INFO [2022-01-10 05:49:33,582] jepsen worker 75 - jepsen.util 4935 :invoke :read [25 nil] INFO [2022-01-10 05:49:33,656] jepsen worker 52 - jepsen.util 6262 :info :write [17 48] :RequestTimeoutException INFO [2022-01-10 05:49:33,657] jepsen worker 52 - jepsen.util 6352 :invoke :write [17 41] INFO [2022-01-10 05:49:33,723] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:33,723] jepsen worker nemesis - couchbase.util Rebalance check count: 719 INFO [2022-01-10 05:49:33,723] jepsen worker nemesis - couchbase.util Rebalance stuck count: 598 INFO [2022-01-10 05:49:33,748] jepsen worker 16 - jepsen.util 5056 :ok :read [5 25] INFO [2022-01-10 05:49:33,749] jepsen worker 16 - jepsen.util 5056 :invoke :read [5 nil] INFO [2022-01-10 05:49:33,765] jepsen worker 83 - jepsen.util 5663 :ok :read [27 30] INFO [2022-01-10 05:49:33,766] jepsen worker 83 - jepsen.util 5663 :invoke :write [27 12] INFO [2022-01-10 05:49:33,770] jepsen worker 0 - jepsen.util 5220 :ok :read [0 46] INFO [2022-01-10 05:49:33,770] jepsen worker 0 - jepsen.util 5220 :invoke :write [0 37] INFO [2022-01-10 05:49:33,880] jepsen worker 53 - jepsen.util 6083 :info :write [17 36] :RequestTimeoutException INFO [2022-01-10 05:49:33,880] jepsen worker 53 - jepsen.util 6173 :invoke :read [17 nil] INFO [2022-01-10 05:49:33,905] jepsen worker 39 - jepsen.util 5349 :fail :read [13 nil] :ServerOutOfMemoryException INFO [2022-01-10 05:49:33,907] jepsen worker 39 - jepsen.util 5349 :invoke :read [13 nil] INFO [2022-01-10 05:49:33,925] jepsen worker 46 - jepsen.util 5176 :ok :read [15 23] INFO [2022-01-10 05:49:33,925] jepsen worker 76 - jepsen.util 4306 :info :write [25 38] :RequestTimeoutException INFO [2022-01-10 05:49:33,925] jepsen worker 76 - jepsen.util 4396 :invoke :read [25 nil] INFO [2022-01-10 05:49:33,926] jepsen worker 46 - jepsen.util 5176 :invoke :write [15 0] INFO [2022-01-10 05:49:33,954] jepsen worker 6 - jepsen.util 5046 :ok :read [2 14] INFO [2022-01-10 05:49:33,955] jepsen worker 6 - jepsen.util 5046 :invoke :write [2 21] INFO [2022-01-10 05:49:33,956] jepsen worker 19 - jepsen.util 5059 :info :write [6 42] :RequestTimeoutException INFO [2022-01-10 05:49:33,957] jepsen worker 19 - jepsen.util 5149 :invoke :read [6 nil] INFO [2022-01-10 05:49:33,959] jepsen worker 26 - jepsen.util 4976 :ok :read [8 39] INFO [2022-01-10 05:49:33,960] jepsen worker 26 - jepsen.util 4976 :invoke :write [8 31] INFO [2022-01-10 05:49:33,964] jepsen worker 1 - jepsen.util 5131 :ok :read [0 46] INFO [2022-01-10 05:49:33,965] jepsen worker 1 - jepsen.util 5131 :invoke :write [0 42] INFO [2022-01-10 05:49:33,977] jepsen worker 29 - jepsen.util 4979 :ok :read [9 22] INFO [2022-01-10 05:49:33,980] jepsen worker 29 - jepsen.util 4979 :invoke :write [9 11] INFO [2022-01-10 05:49:33,991] jepsen worker 67 - jepsen.util 4747 :ok :read [22 41] INFO [2022-01-10 05:49:33,991] jepsen worker 67 - jepsen.util 4747 :invoke :read [22 nil] INFO [2022-01-10 05:49:33,996] jepsen worker 49 - jepsen.util 4909 :ok :read [16 38] INFO [2022-01-10 05:49:33,997] jepsen worker 49 - jepsen.util 4909 :invoke :read [16 nil] INFO [2022-01-10 05:49:34,000] jepsen worker 47 - jepsen.util 5177 :info :write [15 24] :RequestTimeoutException INFO [2022-01-10 05:49:34,000] jepsen worker 47 - jepsen.util 5267 :invoke :write [15 41] INFO [2022-01-10 05:49:34,005] jepsen worker 63 - jepsen.util 4653 :info :write [21 21] :RequestTimeoutException INFO [2022-01-10 05:49:34,006] jepsen worker 63 - jepsen.util 4743 :invoke :write [21 1] INFO [2022-01-10 05:49:34,091] jepsen worker 77 - jepsen.util 5567 :info :write [25 46] :RequestTimeoutException INFO [2022-01-10 05:49:34,092] jepsen worker 77 - jepsen.util 5657 :invoke :write [25 10] INFO [2022-01-10 05:49:34,106] jepsen worker 50 - jepsen.util 5270 :info :write [16 2] :RequestTimeoutException INFO [2022-01-10 05:49:34,106] jepsen worker 50 - jepsen.util 5360 :invoke :write [16 34] INFO [2022-01-10 05:49:34,232] jepsen worker 11 - jepsen.util 4961 :info :write [3 45] :RequestTimeoutException INFO [2022-01-10 05:49:34,232] jepsen worker 11 - jepsen.util 5051 :invoke :write [3 49] INFO [2022-01-10 05:49:34,323] jepsen worker 30 - jepsen.util 4800 :info :write [10 1] :RequestTimeoutException INFO [2022-01-10 05:49:34,324] jepsen worker 30 - jepsen.util 4890 :invoke :write [10 46] INFO [2022-01-10 05:49:34,333] jepsen worker 85 - jepsen.util 5485 :ok :read [28 45] INFO [2022-01-10 05:49:34,334] jepsen worker 85 - jepsen.util 5485 :invoke :read [28 nil] INFO [2022-01-10 05:49:34,382] jepsen worker 14 - jepsen.util 6134 :ok :read [4 3] INFO [2022-01-10 05:49:34,383] jepsen worker 14 - jepsen.util 6134 :invoke :write [4 19] INFO [2022-01-10 05:49:34,416] jepsen worker 42 - jepsen.util 4722 :info :write [14 6] :RequestTimeoutException INFO [2022-01-10 05:49:34,417] jepsen worker 42 - jepsen.util 4812 :invoke :read [14 nil] INFO [2022-01-10 05:49:34,429] jepsen worker 76 - jepsen.util 4396 :ok :read [25 :nil] INFO [2022-01-10 05:49:34,429] jepsen worker 76 - jepsen.util 4396 :invoke :read [25 nil] INFO [2022-01-10 05:49:34,576] jepsen worker 65 - jepsen.util 4655 :info :write [21 33] :RequestTimeoutException INFO [2022-01-10 05:49:34,577] jepsen worker 65 - jepsen.util 4745 :invoke :write [21 20] INFO [2022-01-10 05:49:34,586] jepsen worker 7 - jepsen.util 5407 :info :write [2 12] :RequestTimeoutException INFO [2022-01-10 05:49:34,587] jepsen worker 7 - jepsen.util 5497 :invoke :write [2 44] INFO [2022-01-10 05:49:34,602] jepsen worker 19 - jepsen.util 5149 :ok :read [6 25] INFO [2022-01-10 05:49:34,603] jepsen worker 19 - jepsen.util 5149 :invoke :read [6 nil] INFO [2022-01-10 05:49:34,725] jepsen worker 61 - jepsen.util 5281 :info :write [20 39] :RequestTimeoutException INFO [2022-01-10 05:49:34,725] jepsen worker 61 - jepsen.util 5371 :invoke :write [20 13] INFO [2022-01-10 05:49:34,729] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:34,729] jepsen worker nemesis - couchbase.util Rebalance check count: 720 INFO [2022-01-10 05:49:34,729] jepsen worker nemesis - couchbase.util Rebalance stuck count: 599 INFO [2022-01-10 05:49:34,741] jepsen worker 37 - jepsen.util 4717 :info :write [12 8] :RequestTimeoutException INFO [2022-01-10 05:49:34,741] jepsen worker 37 - jepsen.util 4807 :invoke :read [12 nil] INFO [2022-01-10 05:49:34,800] jepsen worker 64 - jepsen.util 5104 :info :write [21 27] :RequestTimeoutException INFO [2022-01-10 05:49:34,801] jepsen worker 64 - jepsen.util 5194 :invoke :write [21 1] INFO [2022-01-10 05:49:34,830] jepsen worker 73 - jepsen.util 4933 :info :write [24 43] :RequestTimeoutException INFO [2022-01-10 05:49:34,830] jepsen worker 73 - jepsen.util 5023 :invoke :read [24 nil] INFO [2022-01-10 05:49:34,832] jepsen worker 12 - jepsen.util 5682 :info :write [4 5] :RequestTimeoutException INFO [2022-01-10 05:49:34,832] jepsen worker 12 - jepsen.util 5772 :invoke :read [4 nil] INFO [2022-01-10 05:49:34,869] jepsen worker 68 - jepsen.util 5018 :ok :read [22 41] INFO [2022-01-10 05:49:34,870] jepsen worker 68 - jepsen.util 5018 :invoke :read [22 nil] INFO [2022-01-10 05:49:34,895] jepsen worker 58 - jepsen.util 5098 :ok :read [19 31] INFO [2022-01-10 05:49:34,895] jepsen worker 58 - jepsen.util 5098 :invoke :read [19 nil] INFO [2022-01-10 05:49:34,901] jepsen worker 28 - jepsen.util 4708 :ok :read [9 22] INFO [2022-01-10 05:49:34,902] jepsen worker 28 - jepsen.util 4708 :invoke :write [9 16] INFO [2022-01-10 05:49:35,085] jepsen worker 12 - jepsen.util 5772 :ok :read [4 3] INFO [2022-01-10 05:49:35,085] jepsen worker 12 - jepsen.util 5772 :invoke :read [4 nil] INFO [2022-01-10 05:49:35,093] jepsen worker 81 - jepsen.util 5661 :ok :read [27 30] INFO [2022-01-10 05:49:35,094] jepsen worker 81 - jepsen.util 5661 :invoke :read [27 nil] INFO [2022-01-10 05:49:35,174] jepsen worker 37 - jepsen.util 4807 :ok :read [12 14] INFO [2022-01-10 05:49:35,175] jepsen worker 37 - jepsen.util 4807 :invoke :write [12 18] INFO [2022-01-10 05:49:35,284] jepsen worker 53 - jepsen.util 6173 :ok :read [17 41] INFO [2022-01-10 05:49:35,285] jepsen worker 53 - jepsen.util 6173 :invoke :read [17 nil] INFO [2022-01-10 05:49:35,300] jepsen worker 21 - jepsen.util 5061 :ok :read [7 :nil] INFO [2022-01-10 05:49:35,301] jepsen worker 21 - jepsen.util 5061 :invoke :read [7 nil] INFO [2022-01-10 05:49:35,345] jepsen worker 84 - jepsen.util 4674 :info :write [28 12] :RequestTimeoutException INFO [2022-01-10 05:49:35,346] jepsen worker 84 - jepsen.util 4764 :invoke :write [28 39] INFO [2022-01-10 05:49:35,366] jepsen worker 44 - jepsen.util 5084 :info :write [14 9] :RequestTimeoutException INFO [2022-01-10 05:49:35,367] jepsen worker 44 - jepsen.util 5174 :invoke :read [14 nil] INFO [2022-01-10 05:49:35,451] jepsen worker 40 - jepsen.util 4720 :info :write [13 28] :RequestTimeoutException INFO [2022-01-10 05:49:35,452] jepsen worker 40 - jepsen.util 4810 :invoke :read [13 nil] INFO [2022-01-10 05:49:35,494] jepsen worker 16 - jepsen.util 5056 :ok :read [5 25] INFO [2022-01-10 05:49:35,494] jepsen worker 16 - jepsen.util 5056 :invoke :write [5 41] INFO [2022-01-10 05:49:35,657] jepsen worker 53 - jepsen.util 6173 :ok :read [17 41] INFO [2022-01-10 05:49:35,658] jepsen worker 53 - jepsen.util 6173 :invoke :write [17 16] INFO [2022-01-10 05:49:35,730] jepsen worker 12 - jepsen.util 5772 :ok :read [4 3] INFO [2022-01-10 05:49:35,730] jepsen worker 12 - jepsen.util 5772 :invoke :write [4 41] INFO [2022-01-10 05:49:35,735] jepsen worker nemesis - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0.695906432748538}, :ns_1@172.23.107.227 {:progress 0.4809384164222874}, :ns_1@172.23.105.187 {:progress 0.4809384164222874}, :ns_1@172.23.107.228 {:progress 0.6647058823529413}} INFO [2022-01-10 05:49:35,735] jepsen worker nemesis - couchbase.util Rebalance check count: 721 INFO [2022-01-10 05:49:35,735] jepsen worker nemesis - couchbase.util Rebalance stuck count: 600 INFO [2022-01-10 05:49:35,735] jepsen worker nemesis - couchbase.util Rebalance stuck WARN [2022-01-10 05:49:35,738] jepsen worker nemesis - jepsen.generator.interpreter Process :nemesis crashed java.lang.RuntimeException: Rebalance stuck at couchbase.util$wait_for_rebalance_complete.invokeStatic(util.clj:218) at couchbase.util$wait_for_rebalance_complete.invoke(util.clj:204) at couchbase.util$wait_for_rebalance_complete.invokeStatic(util.clj:205) at couchbase.util$wait_for_rebalance_complete.invoke(util.clj:204) at couchbase.util$rebalance.invokeStatic(util.clj:253) at couchbase.util$rebalance.invoke(util.clj:227) at couchbase.nemesis$swap_rebalance$fn__3895.invoke(nemesis.clj:237) at couchbase.nemesis$swap_rebalance.invokeStatic(nemesis.clj:235) at couchbase.nemesis$swap_rebalance.invoke(nemesis.clj:227) at couchbase.nemesis$couchbase$reify__3984.invoke_BANG_(nemesis.clj:410) at jepsen.generator.interpreter.NemesisWorker.invoke_BANG_(interpreter.clj:74) at jepsen.generator.interpreter$spawn_worker$fn__9837$fn__9838.invoke(interpreter.clj:137) at jepsen.generator.interpreter$spawn_worker$fn__9837.invoke(interpreter.clj:120) at clojure.core$binding_conveyor_fn$fn__5772.invoke(core.clj:2034) at clojure.lang.AFn.call(AFn.java:18) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) INFO [2022-01-10 05:49:35,743] jepsen worker nemesis - jepsen.generator.interpreter Aborting test due to nemesis failure. INFO [2022-01-10 05:49:35,745] jepsen worker 60 - jepsen.util 5460 :info :write [20 15] :RequestTimeoutException INFO [2022-01-10 05:49:35,796] jepsen worker 33 - jepsen.util 5253 :info :write [11 13] :RequestTimeoutException INFO [2022-01-10 05:49:35,846] jepsen worker 72 - jepsen.util 5382 :info :write [24 4] :RequestTimeoutException INFO [2022-01-10 05:49:35,870] jepsen worker 25 - jepsen.util 4705 :info :write [8 29] :RequestTimeoutException INFO [2022-01-10 05:49:35,886] jepsen worker 45 - jepsen.util 5445 :info :write [15 4] :RequestTimeoutException INFO [2022-01-10 05:49:35,936] jepsen worker 80 - jepsen.util 6200 :info :write [26 34] :RequestTimeoutException INFO [2022-01-10 05:49:35,952] jepsen worker 76 - jepsen.util 4396 :ok :read [25 :nil] INFO [2022-01-10 05:49:36,097] jepsen worker 43 - jepsen.util 4903 :info :write [14 39] :RequestTimeoutException INFO [2022-01-10 05:49:36,333] jepsen worker 85 - jepsen.util 5485 :ok :read [28 45] INFO [2022-01-10 05:49:36,351] jepsen worker 81 - jepsen.util 5661 :ok :read [27 30] INFO [2022-01-10 05:49:36,420] jepsen worker 74 - jepsen.util 5564 :info :write [24 18] :RequestTimeoutException INFO [2022-01-10 05:49:36,699] jepsen worker 9 - jepsen.util 5589 :info :write [3 24] :RequestTimeoutException INFO [2022-01-10 05:49:36,834] jepsen worker 21 - jepsen.util 5061 :ok :read [7 :nil] INFO [2022-01-10 05:49:36,939] jepsen worker 40 - jepsen.util 4810 :ok :read [13 15] INFO [2022-01-10 05:49:37,012] jepsen worker 44 - jepsen.util 5174 :ok :read [14 21] INFO [2022-01-10 05:49:37,638] jepsen worker 39 - jepsen.util 5349 :ok :read [13 15] INFO [2022-01-10 05:49:37,665] jepsen worker 59 - jepsen.util 4829 :ok :read [19 31] INFO [2022-01-10 05:49:37,706] jepsen worker 24 - jepsen.util 4704 :info :write [8 11] :RequestTimeoutException INFO [2022-01-10 05:49:37,711] jepsen worker 10 - jepsen.util 4870 :info :write [3 17] :RequestTimeoutException INFO [2022-01-10 05:49:37,822] jepsen worker 36 - jepsen.util 3996 :info :write [12 41] :RequestTimeoutException INFO [2022-01-10 05:49:37,865] jepsen worker 67 - jepsen.util 4747 :ok :read [22 41] INFO [2022-01-10 05:49:38,155] jepsen worker 51 - jepsen.util 5901 :info :write [17 27] :RequestTimeoutException INFO [2022-01-10 05:49:38,226] jepsen worker 17 - jepsen.util 4967 :info :write [5 40] :RequestTimeoutException INFO [2022-01-10 05:49:38,300] jepsen worker 18 - jepsen.util 4968 :info :write [6 21] :RequestTimeoutException INFO [2022-01-10 05:49:38,341] jepsen worker 8 - jepsen.util 5228 :info :write [2 15] :RequestTimeoutException INFO [2022-01-10 05:49:38,536] jepsen worker 66 - jepsen.util 5016 :info :write [22 16] :RequestTimeoutException INFO [2022-01-10 05:49:39,078] jepsen worker 87 - jepsen.util 4857 :ok :read [29 0] INFO [2022-01-10 05:49:39,091] jepsen worker 55 - jepsen.util 5545 :ok :read [18 14] INFO [2022-01-10 05:49:39,104] jepsen worker 54 - jepsen.util 5274 :ok :read [18 14] INFO [2022-01-10 05:49:39,143] jepsen worker 31 - jepsen.util 5341 :ok :read [10 31] INFO [2022-01-10 05:49:39,514] jepsen worker 35 - jepsen.util 4895 :info :write [11 46] :RequestTimeoutException INFO [2022-01-10 05:49:39,526] jepsen worker 48 - jepsen.util 4998 :info :write [16 41] :RequestTimeoutException INFO [2022-01-10 05:49:39,664] jepsen worker 56 - jepsen.util 5366 :info :write [18 9] :SyncWriteAmbiguous INFO [2022-01-10 05:49:39,802] jepsen worker 23 - jepsen.util 5243 :info :write [7 3] :RequestTimeoutException INFO [2022-01-10 05:49:39,946] jepsen worker 22 - jepsen.util 4702 :info :write [7 33] :RequestTimeoutException INFO [2022-01-10 05:49:40,621] jepsen worker 88 - jepsen.util 5038 :info :write [29 27] :RequestTimeoutException WARN [2022-01-10 05:49:40,765] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":18060849,"last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18060849,"last_local_socket":"172.23.104.79:34130","total_server_duration_us":9027816,"encode_duration_us":20521,"operation_id":"0x9aed","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10008576}],"total_count":1}} INFO [2022-01-10 05:49:40,824] jepsen worker 62 - jepsen.util 5462 :ok :read [20 :nil] WARN [2022-01-10 05:49:40,866] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":3716946,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":3716946,"total_server_duration_us":49,"operation_id":"0x9a0a","timeout_ms":30000,"last_server_duration_us":49,"total_duration_us":17557551},{"operation_name":"get","last_dispatch_duration_us":16847244,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":16847244,"total_server_duration_us":27,"operation_id":"0x9b35","timeout_ms":30000,"last_server_duration_us":27,"total_duration_us":16848231},{"operation_name":"get","last_dispatch_duration_us":16185740,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":16185740,"total_server_duration_us":17963,"operation_id":"0x99dc","timeout_ms":30000,"last_server_duration_us":17963,"total_duration_us":16187770},{"operation_name":"get","last_dispatch_duration_us":16114036,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":16114036,"total_server_duration_us":15,"operation_id":"0x9b5c","timeout_ms":30000,"last_server_duration_us":15,"total_duration_us":16116278},{"operation_name":"get","last_dispatch_duration_us":7416688,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":7416688,"total_server_duration_us":1839594,"operation_id":"0x9b04","timeout_ms":30000,"last_server_duration_us":1839594,"total_duration_us":12071707},{"operation_name":"get","last_dispatch_duration_us":7579971,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000001/00000000CC853C83","last_local_socket":"172.23.104.79:52130","total_dispatch_duration_us":7579971,"total_server_duration_us":2046544,"operation_id":"0x9b70","timeout_ms":30000,"last_server_duration_us":2046544,"total_duration_us":10199645},{"operation_name":"upsert","last_dispatch_duration_us":305180,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000001/000000007CF9F49A","last_local_socket":"172.23.104.79:44306","total_dispatch_duration_us":8511134,"total_server_duration_us":1713953,"encode_duration_us":16216,"operation_id":"0x9c5e","timeout_ms":10000,"last_server_duration_us":21895,"total_duration_us":10010812},{"operation_name":"upsert","last_dispatch_duration_us":9026600,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":18090255,"total_server_duration_us":46446,"encode_duration_us":16363,"operation_id":"0x9aee","timeout_ms":10000,"last_server_duration_us":22887,"total_duration_us":10010166},{"operation_name":"upsert","last_dispatch_duration_us":18060849,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000001/00000000FCD5F8E8","last_local_socket":"172.23.104.79:34130","total_dispatch_duration_us":18060849,"total_server_duration_us":9027816,"encode_duration_us":20521,"operation_id":"0x9aed","timeout_ms":10000,"last_server_duration_us":9027816,"total_duration_us":10008576},{"operation_name":"upsert","last_dispatch_duration_us":846203,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000001/00000000E26F4444","last_local_socket":"172.23.104.79:38414","total_dispatch_duration_us":8502359,"total_server_duration_us":2032839,"encode_duration_us":16915,"operation_id":"0x9bea","timeout_ms":10000,"last_server_duration_us":307765,"total_duration_us":10006430}],"total_count":28}} INFO [2022-01-10 05:49:40,866] jepsen worker 82 - jepsen.util 5122 :info :write [27 8] :RequestTimeoutException INFO [2022-01-10 05:49:40,956] jepsen worker 20 - jepsen.util 4790 :ok :read [6 25] INFO [2022-01-10 05:49:40,995] jepsen worker 38 - jepsen.util 4808 :info :write [12 25] :RequestTimeoutException INFO [2022-01-10 05:49:41,051] jepsen worker 32 - jepsen.util 5342 :info :write [10 15] :RequestTimeoutException INFO [2022-01-10 05:49:41,080] jepsen worker 89 - jepsen.util 5489 :info :write [29 32] :RequestTimeoutException INFO [2022-01-10 05:49:41,096] jepsen worker 13 - jepsen.util 5953 :info :write [4 10] :RequestTimeoutException INFO [2022-01-10 05:49:41,171] jepsen worker 34 - jepsen.util 4894 :info :write [11 40] :RequestTimeoutException INFO [2022-01-10 05:49:41,311] jepsen worker 15 - jepsen.util 4605 :info :write [5 17] :RequestTimeoutException INFO [2022-01-10 05:49:41,416] jepsen worker 86 - jepsen.util 4226 :info :write [28 1] :RequestTimeoutException INFO [2022-01-10 05:49:41,480] jepsen worker 79 - jepsen.util 5929 :info :write [26 33] :RequestTimeoutException INFO [2022-01-10 05:49:41,612] jepsen worker 71 - jepsen.util 4751 :info :write [23 35] :RequestTimeoutException INFO [2022-01-10 05:49:41,632] jepsen worker 78 - jepsen.util 6288 :info :write [26 14] :RequestTimeoutException INFO [2022-01-10 05:49:41,878] jepsen worker 75 - jepsen.util 4935 :ok :read [25 :nil] INFO [2022-01-10 05:49:41,906] jepsen worker 41 - jepsen.util 4991 :info :write [13 7] :RequestTimeoutException INFO [2022-01-10 05:49:42,076] jepsen worker 3 - jepsen.util 5133 :info :write [1 19] :RequestTimeoutException INFO [2022-01-10 05:49:42,259] jepsen worker 27 - jepsen.util 5517 :info :write [9 35] :RequestTimeoutException WARN [2022-01-10 05:49:42,267] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":11720260,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":11720260,"total_server_duration_us":1055142,"operation_id":"0x9b34","timeout_ms":30000,"last_server_duration_us":1055142,"total_duration_us":11722999},{"operation_name":"upsert","last_dispatch_duration_us":1092516,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":7374914,"total_server_duration_us":1555453,"encode_duration_us":17327,"operation_id":"0x9ced","timeout_ms":10000,"last_server_duration_us":123371,"total_duration_us":10007763},{"operation_name":"upsert","last_dispatch_duration_us":8788292,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9643110,"total_server_duration_us":434029,"encode_duration_us":17132,"operation_id":"0x9ba6","timeout_ms":10000,"last_server_duration_us":21163,"total_duration_us":10005267},{"operation_name":"upsert","last_dispatch_duration_us":9073806,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9500246,"total_server_duration_us":9294804,"encode_duration_us":16812,"operation_id":"0x9c3d","timeout_ms":10000,"last_server_duration_us":9044797,"total_duration_us":10004810},{"operation_name":"upsert","last_dispatch_duration_us":1320566,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":10151347,"total_server_duration_us":903935,"encode_duration_us":17355,"operation_id":"0x9bdd","timeout_ms":10000,"last_server_duration_us":359124,"total_duration_us":10002773},{"operation_name":"upsert","last_dispatch_duration_us":1505037,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000006/00000000A12A8681","last_local_socket":"172.23.104.79:38420","total_dispatch_duration_us":4760926,"total_server_duration_us":154601,"encode_duration_us":17548,"operation_id":"0x9ceb","timeout_ms":10000,"last_server_duration_us":134160,"total_duration_us":10002540},{"operation_name":"upsert","last_dispatch_duration_us":30988,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":760973,"total_server_duration_us":423862,"encode_duration_us":17444,"operation_id":"0x9b9f","timeout_ms":10000,"last_server_duration_us":19340,"total_duration_us":10001745},{"operation_name":"upsert","last_dispatch_duration_us":295783,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000006/0000000021E2C07D","last_local_socket":"172.23.104.79:44302","total_dispatch_duration_us":9757903,"total_server_duration_us":3215844,"encode_duration_us":21749,"operation_id":"0x9bbb","timeout_ms":10000,"last_server_duration_us":89584,"total_duration_us":10001184},{"operation_name":"upsert","last_dispatch_duration_us":9046612,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000006/0000000044652E4D","last_local_socket":"172.23.104.79:34134","total_dispatch_duration_us":9046612,"total_server_duration_us":9028877,"encode_duration_us":17951,"operation_id":"0x9cb8","timeout_ms":10000,"last_server_duration_us":9028877,"total_duration_us":9050144},{"operation_name":"get","last_dispatch_duration_us":3923269,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000006/0000000095B39CD8","last_local_socket":"172.23.104.79:52090","total_dispatch_duration_us":3923269,"total_server_duration_us":8230,"operation_id":"0x9c0c","timeout_ms":30000,"last_server_duration_us":8230,"total_duration_us":6544880}],"total_count":23}} WARN [2022-01-10 05:49:42,367] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9073806,"last_local_id":"EFE9906200000006/0000000044652E4D","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9500246,"last_local_socket":"172.23.104.79:34134","total_server_duration_us":9294804,"encode_duration_us":16812,"operation_id":"0x9c3d","timeout_ms":10000,"last_server_duration_us":9044797,"total_duration_us":10004810}],"total_count":1}} WARN [2022-01-10 05:49:42,421] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1958847,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7467821,"total_server_duration_us":685733,"encode_duration_us":21342,"operation_id":"0x9baa","timeout_ms":10000,"last_server_duration_us":444892,"total_duration_us":10010724},{"operation_name":"upsert","last_dispatch_duration_us":1956378,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11707463,"total_server_duration_us":1491368,"encode_duration_us":17442,"operation_id":"0x9c01","timeout_ms":10000,"last_server_duration_us":341016,"total_duration_us":10008988},{"operation_name":"upsert","last_dispatch_duration_us":1394045,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":10799037,"total_server_duration_us":2245980,"encode_duration_us":18617,"operation_id":"0x9bff","timeout_ms":10000,"last_server_duration_us":158588,"total_duration_us":10008110},{"operation_name":"upsert","last_dispatch_duration_us":2170858,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000005/000000009D86D9DF","last_local_socket":"172.23.104.79:44284","total_dispatch_duration_us":10785934,"total_server_duration_us":1268026,"encode_duration_us":16422,"operation_id":"0x9b38","timeout_ms":10000,"last_server_duration_us":341016,"total_duration_us":10007588},{"operation_name":"upsert","last_dispatch_duration_us":1201338,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":8937238,"total_server_duration_us":839490,"encode_duration_us":16709,"operation_id":"0x9d16","timeout_ms":10000,"last_server_duration_us":261695,"total_duration_us":10006548},{"operation_name":"upsert","last_dispatch_duration_us":1818496,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":9667163,"total_server_duration_us":1364969,"encode_duration_us":16999,"operation_id":"0x9cdc","timeout_ms":10000,"last_server_duration_us":517415,"total_duration_us":10005623},{"operation_name":"upsert","last_dispatch_duration_us":2172012,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":11876416,"total_server_duration_us":2175947,"encode_duration_us":17220,"operation_id":"0x9bac","timeout_ms":10000,"last_server_duration_us":135402,"total_duration_us":10005303},{"operation_name":"upsert","last_dispatch_duration_us":2847596,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":10394503,"total_server_duration_us":1098200,"encode_duration_us":16829,"operation_id":"0x9b96","timeout_ms":10000,"last_server_duration_us":513961,"total_duration_us":10003219},{"operation_name":"upsert","last_dispatch_duration_us":1969482,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000005/000000001CDA61C4","last_local_socket":"172.23.104.79:52098","total_dispatch_duration_us":11459501,"total_server_duration_us":1701160,"encode_duration_us":20038,"operation_id":"0x9c63","timeout_ms":10000,"last_server_duration_us":349222,"total_duration_us":10003200},{"operation_name":"upsert","last_dispatch_duration_us":2336355,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000005/00000000A0BDDEAB","last_local_socket":"172.23.104.79:38378","total_dispatch_duration_us":7382958,"total_server_duration_us":1227343,"encode_duration_us":18401,"operation_id":"0x9bf7","timeout_ms":10000,"last_server_duration_us":830605,"total_duration_us":10003065}],"total_count":21}} WARN [2022-01-10 05:49:42,520] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1777932,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":6195402,"total_server_duration_us":510516,"encode_duration_us":17342,"operation_id":"0x9ca4","timeout_ms":10000,"last_server_duration_us":254443,"total_duration_us":10008503},{"operation_name":"upsert","last_dispatch_duration_us":36663,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":510887,"total_server_duration_us":300019,"encode_duration_us":18010,"operation_id":"0x9ce2","timeout_ms":10000,"last_server_duration_us":24841,"total_duration_us":10007447},{"operation_name":"upsert","last_dispatch_duration_us":5724562,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":8407777,"total_server_duration_us":739487,"encode_duration_us":18968,"operation_id":"0x9b95","timeout_ms":10000,"last_server_duration_us":313587,"total_duration_us":10006899},{"operation_name":"upsert","last_dispatch_duration_us":36519,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000004/000000008D35FDEF","last_local_socket":"172.23.104.79:34126","total_dispatch_duration_us":493228,"total_server_duration_us":285426,"encode_duration_us":16505,"operation_id":"0x9cb9","timeout_ms":10000,"last_server_duration_us":20282,"total_duration_us":10004464},{"operation_name":"upsert","last_dispatch_duration_us":697504,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":9363736,"total_server_duration_us":2265450,"encode_duration_us":17338,"operation_id":"0x9c56","timeout_ms":10000,"last_server_duration_us":104228,"total_duration_us":10004129},{"operation_name":"upsert","last_dispatch_duration_us":89549,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":8312286,"total_server_duration_us":1740884,"encode_duration_us":16540,"operation_id":"0x9cf2","timeout_ms":10000,"last_server_duration_us":74372,"total_duration_us":10003621},{"operation_name":"upsert","last_dispatch_duration_us":3114376,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":10046592,"total_server_duration_us":462905,"encode_duration_us":20599,"operation_id":"0x9b8d","timeout_ms":10000,"last_server_duration_us":125773,"total_duration_us":10003477},{"operation_name":"upsert","last_dispatch_duration_us":1481294,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":9897804,"total_server_duration_us":4897397,"encode_duration_us":16375,"operation_id":"0x9bd2","timeout_ms":10000,"last_server_duration_us":361011,"total_duration_us":10002915},{"operation_name":"upsert","last_dispatch_duration_us":267630,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000004/000000008B42EFB5","last_local_socket":"172.23.104.79:44292","total_dispatch_duration_us":8414342,"total_server_duration_us":1191383,"encode_duration_us":18287,"operation_id":"0x9cf3","timeout_ms":10000,"last_server_duration_us":174309,"total_duration_us":10002643},{"operation_name":"upsert","last_dispatch_duration_us":5008071,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000004/000000000C8781B9","last_local_socket":"172.23.104.79:38374","total_dispatch_duration_us":11595432,"total_server_duration_us":234769,"encode_duration_us":16122,"operation_id":"0x9b67","timeout_ms":10000,"last_server_duration_us":21487,"total_duration_us":10002286}],"total_count":14}} WARN [2022-01-10 05:49:42,520] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12735053,"last_local_id":"EFE9906200000003/0000000019B074E8","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13828343,"last_local_socket":"172.23.104.79:38448","total_server_duration_us":9589185,"encode_duration_us":17037,"operation_id":"0x9c00","timeout_ms":10000,"last_server_duration_us":9388337,"total_duration_us":10010282}],"total_count":1}} WARN [2022-01-10 05:49:42,522] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1780082,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":11403678,"total_server_duration_us":2152985,"encode_duration_us":16493,"operation_id":"0x9b5d","timeout_ms":10000,"last_server_duration_us":778257,"total_duration_us":10010690},{"operation_name":"upsert","last_dispatch_duration_us":252751,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":10117156,"total_server_duration_us":746826,"encode_duration_us":16744,"operation_id":"0x9c02","timeout_ms":10000,"last_server_duration_us":58252,"total_duration_us":10007714},{"operation_name":"upsert","last_dispatch_duration_us":433297,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000002/00000000598DAEAE","last_local_socket":"172.23.104.79:44342","total_dispatch_duration_us":8203369,"total_server_duration_us":815352,"encode_duration_us":16512,"operation_id":"0x9d17","timeout_ms":10000,"last_server_duration_us":65253,"total_duration_us":10007208},{"operation_name":"upsert","last_dispatch_duration_us":3402556,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":10512576,"total_server_duration_us":686615,"encode_duration_us":16857,"operation_id":"0x9b92","timeout_ms":10000,"last_server_duration_us":128717,"total_duration_us":10007078},{"operation_name":"upsert","last_dispatch_duration_us":7213443,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":7213443,"total_server_duration_us":227637,"encode_duration_us":16594,"operation_id":"0x9c8a","timeout_ms":10000,"last_server_duration_us":227637,"total_duration_us":10006775},{"operation_name":"upsert","last_dispatch_duration_us":5595118,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":7406539,"total_server_duration_us":1235866,"encode_duration_us":19913,"operation_id":"0x9b90","timeout_ms":10000,"last_server_duration_us":289349,"total_duration_us":10006229},{"operation_name":"upsert","last_dispatch_duration_us":6831916,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","total_dispatch_duration_us":6831916,"total_server_duration_us":546045,"encode_duration_us":16384,"operation_id":"0x9c5f","timeout_ms":10000,"last_server_duration_us":546045,"total_duration_us":10003888},{"operation_name":"upsert","last_dispatch_duration_us":4067363,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000002/000000000DD1E359","last_local_socket":"172.23.104.79:38440","total_dispatch_duration_us":4067363,"total_server_duration_us":1514284,"encode_duration_us":16306,"operation_id":"0x9ba7","timeout_ms":10000,"last_server_duration_us":1514284,"total_duration_us":10002025},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000002/00000000B5693B6C","last_local_socket":"172.23.104.79:52136","encode_duration_us":20582,"operation_id":"0x9cef","timeout_ms":10000,"total_duration_us":10001898},{"operation_name":"upsert","last_dispatch_duration_us":9053145,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000002/00000000A7E0A91C","last_local_socket":"172.23.104.79:34084","total_dispatch_duration_us":9253995,"total_server_duration_us":9161366,"encode_duration_us":17378,"operation_id":"0x9b8c","timeout_ms":10000,"last_server_duration_us":9039489,"total_duration_us":9365400}],"total_count":14}} INFO [2022-01-10 05:49:42,637] jepsen worker 4 - jepsen.util 5314 :info :write [1 9] :RequestTimeoutException WARN [2022-01-10 05:49:42,721] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8224704,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":15198998,"total_server_duration_us":1779507,"encode_duration_us":19837,"operation_id":"0x9bc7","timeout_ms":10000,"last_server_duration_us":226973,"total_duration_us":10010403},{"operation_name":"upsert","last_dispatch_duration_us":12735053,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":13828343,"total_server_duration_us":9589185,"encode_duration_us":17037,"operation_id":"0x9c00","timeout_ms":10000,"last_server_duration_us":9388337,"total_duration_us":10010282},{"operation_name":"upsert","last_dispatch_duration_us":8658729,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":15932330,"total_server_duration_us":518607,"encode_duration_us":19336,"operation_id":"0x9bd1","timeout_ms":10000,"last_server_duration_us":25275,"total_duration_us":10008746},{"operation_name":"upsert","last_dispatch_duration_us":32219,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":980559,"total_server_duration_us":564617,"encode_duration_us":16104,"operation_id":"0x9c10","timeout_ms":10000,"last_server_duration_us":19966,"total_duration_us":10007552},{"operation_name":"upsert","last_dispatch_duration_us":258356,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":7599343,"total_server_duration_us":1317054,"encode_duration_us":18083,"operation_id":"0x9cf0","timeout_ms":10000,"last_server_duration_us":178088,"total_duration_us":10007427},{"operation_name":"upsert","last_dispatch_duration_us":29917,"last_remote_socket":"172.23.107.227:11210","last_local_id":"EFE9906200000003/00000000135F5DA5","last_local_socket":"172.23.104.79:34164","total_dispatch_duration_us":774971,"total_server_duration_us":501573,"encode_duration_us":16904,"operation_id":"0x9ce3","timeout_ms":10000,"last_server_duration_us":19418,"total_duration_us":10006094},{"operation_name":"upsert","last_dispatch_duration_us":754355,"last_remote_socket":"172.23.106.61:11210","last_local_id":"EFE9906200000003/000000005A8DE651","last_local_socket":"172.23.104.79:52078","total_dispatch_duration_us":6870573,"total_server_duration_us":1932139,"encode_duration_us":16333,"operation_id":"0x9d1f","timeout_ms":10000,"last_server_duration_us":208906,"total_duration_us":10005263},{"operation_name":"upsert","last_dispatch_duration_us":1136991,"last_remote_socket":"172.23.105.187:11210","last_local_id":"EFE9906200000003/00000000E0DA3632","last_local_socket":"172.23.104.79:44296","total_dispatch_duration_us":10556261,"total_server_duration_us":1449605,"encode_duration_us":16919,"operation_id":"0x9b8b","timeout_ms":10000,"last_server_duration_us":80398,"total_duration_us":10005030},{"operation_name":"upsert","last_dispatch_duration_us":8660812,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":15643382,"total_server_duration_us":2680003,"encode_duration_us":18565,"operation_id":"0x9b91","timeout_ms":10000,"last_server_duration_us":623867,"total_duration_us":10004501},{"operation_name":"upsert","last_dispatch_duration_us":8090265,"last_remote_socket":"172.23.107.228:11210","last_local_id":"EFE9906200000003/0000000019B074E8","last_local_socket":"172.23.104.79:38448","total_dispatch_duration_us":9629470,"total_server_duration_us":381678,"encode_duration_us":16137,"operation_id":"0x9c0d","timeout_ms":10000,"last_server_duration_us":223660,"total_duration_us":10003417}],"total_count":16}} INFO [2022-01-10 05:49:42,939] jepsen worker 19 - jepsen.util 5149 :ok :read [6 25] INFO [2022-01-10 05:49:43,335] jepsen worker 5 - jepsen.util 5135 :info :write [1 26] :RequestTimeoutException INFO [2022-01-10 05:49:43,362] jepsen worker 2 - jepsen.util 5312 :info :write [0 30] :RequestTimeoutException INFO [2022-01-10 05:49:43,371] jepsen worker 57 - jepsen.util 5817 :info :write [19 6] :RequestTimeoutException INFO [2022-01-10 05:49:43,542] jepsen worker 69 - jepsen.util 4749 :info :write [23 25] :RequestTimeoutException INFO [2022-01-10 05:49:43,561] jepsen worker 70 - jepsen.util 5200 :info :write [23 39] :RequestTimeoutException INFO [2022-01-10 05:49:43,596] jepsen worker 73 - jepsen.util 5023 :ok :read [24 18] INFO [2022-01-10 05:49:43,604] jepsen worker 42 - jepsen.util 4812 :ok :read [14 21] INFO [2022-01-10 05:49:43,682] jepsen worker 52 - jepsen.util 6352 :info :write [17 41] :RequestTimeoutException INFO [2022-01-10 05:49:43,795] jepsen worker 83 - jepsen.util 5663 :info :write [27 12] :RequestTimeoutException INFO [2022-01-10 05:49:43,795] jepsen worker 0 - jepsen.util 5220 :info :write [0 37] :RequestTimeoutException INFO [2022-01-10 05:49:43,961] jepsen worker 46 - jepsen.util 5176 :info :write [15 0] :RequestTimeoutException INFO [2022-01-10 05:49:43,981] jepsen worker 6 - jepsen.util 5046 :info :write [2 21] :RequestTimeoutException INFO [2022-01-10 05:49:43,986] jepsen worker 26 - jepsen.util 4976 :info :write [8 31] :RequestTimeoutException INFO [2022-01-10 05:49:43,994] jepsen worker 1 - jepsen.util 5131 :info :write [0 42] :RequestTimeoutException INFO [2022-01-10 05:49:44,006] jepsen worker 29 - jepsen.util 4979 :info :write [9 11] :RequestTimeoutException INFO [2022-01-10 05:49:44,025] jepsen worker 47 - jepsen.util 5267 :info :write [15 41] :RequestTimeoutException INFO [2022-01-10 05:49:44,029] jepsen worker 63 - jepsen.util 4743 :info :write [21 1] :RequestTimeoutException INFO [2022-01-10 05:49:44,112] jepsen worker 77 - jepsen.util 5657 :info :write [25 10] :RequestTimeoutException INFO [2022-01-10 05:49:44,126] jepsen worker 50 - jepsen.util 5360 :info :write [16 34] :RequestTimeoutException INFO [2022-01-10 05:49:44,261] jepsen worker 11 - jepsen.util 5051 :info :write [3 49] :RequestTimeoutException INFO [2022-01-10 05:49:44,345] jepsen worker 30 - jepsen.util 4890 :info :write [10 46] :RequestTimeoutException INFO [2022-01-10 05:49:44,411] jepsen worker 14 - jepsen.util 6134 :info :write [4 19] :RequestTimeoutException INFO [2022-01-10 05:49:44,600] jepsen worker 65 - jepsen.util 4745 :info :write [21 20] :RequestTimeoutException INFO [2022-01-10 05:49:44,612] jepsen worker 7 - jepsen.util 5497 :info :write [2 44] :RequestTimeoutException INFO [2022-01-10 05:49:44,746] jepsen worker 61 - jepsen.util 5371 :info :write [20 13] :RequestTimeoutException INFO [2022-01-10 05:49:44,825] jepsen worker 64 - jepsen.util 5194 :info :write [21 1] :RequestTimeoutException INFO [2022-01-10 05:49:44,925] jepsen worker 28 - jepsen.util 4708 :info :write [9 16] :RequestTimeoutException INFO [2022-01-10 05:49:45,201] jepsen worker 37 - jepsen.util 4807 :info :write [12 18] :RequestTimeoutException INFO [2022-01-10 05:49:45,372] jepsen worker 84 - jepsen.util 4764 :info :write [28 39] :RequestTimeoutException INFO [2022-01-10 05:49:45,515] jepsen worker 16 - jepsen.util 5056 :info :write [5 41] :RequestTimeoutException INFO [2022-01-10 05:49:45,686] jepsen worker 53 - jepsen.util 6173 :info :write [17 16] :RequestTimeoutException INFO [2022-01-10 05:49:45,759] jepsen worker 12 - jepsen.util 5772 :info :write [4 41] :RequestTimeoutException INFO [2022-01-10 05:49:46,172] jepsen worker 58 - jepsen.util 5098 :ok :read [19 31] INFO [2022-01-10 05:49:46,336] jepsen worker 68 - jepsen.util 5018 :ok :read [22 41] INFO [2022-01-10 05:49:48,030] jepsen worker 49 - jepsen.util 4909 :ok :read [16 38] INFO [2022-01-10 05:49:48,075] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][4515us] Closed bucket "default" {"coreId":"0xefe9906200000001"} INFO [2022-01-10 05:49:48,127] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][253us] Closed bucket "default" {"coreId":"0xefe9906200000004"} INFO [2022-01-10 05:49:48,130] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][363us] Closed bucket "default" {"coreId":"0xefe9906200000002"} INFO [2022-01-10 05:49:48,151] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][294us] Closed bucket "default" {"coreId":"0xefe9906200000003"} INFO [2022-01-10 05:49:48,161] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][140us] Node disconnected {"coreId":"0xefe9906200000002","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 05:49:48,161] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][216us] Node disconnected {"coreId":"0xefe9906200000001","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 05:49:48,161] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][99us] Node disconnected {"coreId":"0xefe9906200000004","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][87us] Node disconnected {"coreId":"0xefe9906200000001","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][89us] Node disconnected {"coreId":"0xefe9906200000004","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][90us] Node disconnected {"coreId":"0xefe9906200000003","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][91us] Node disconnected {"coreId":"0xefe9906200000001","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][141us] Node disconnected {"coreId":"0xefe9906200000002","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][165us] Node disconnected {"coreId":"0xefe9906200000001","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][95us] Node disconnected {"coreId":"0xefe9906200000004","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][84us] Node disconnected {"coreId":"0xefe9906200000003","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][142us] Node disconnected {"coreId":"0xefe9906200000002","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][86us] Node disconnected {"coreId":"0xefe9906200000004","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][35ms] Completed shutdown and closed all open buckets {"coreId":"0xefe9906200000001"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][17ms] Completed shutdown and closed all open buckets {"coreId":"0xefe9906200000004"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][97us] Node disconnected {"coreId":"0xefe9906200000002","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][142us] Node disconnected {"coreId":"0xefe9906200000003","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][17ms] Completed shutdown and closed all open buckets {"coreId":"0xefe9906200000002"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][89us] Node disconnected {"coreId":"0xefe9906200000003","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:49:48,162] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][15ms] Completed shutdown and closed all open buckets {"coreId":"0xefe9906200000003"} INFO [2022-01-10 05:49:48,174] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][2369us] Closed bucket "default" {"coreId":"0xefe9906200000006"} INFO [2022-01-10 05:49:48,174] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][96us] Node disconnected {"coreId":"0xefe9906200000006","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 05:49:48,174] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][77us] Node disconnected {"coreId":"0xefe9906200000006","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 05:49:48,174] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][90us] Node disconnected {"coreId":"0xefe9906200000006","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 05:49:48,174] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][284us] Node disconnected {"coreId":"0xefe9906200000006","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:49:48,175] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][21ms] Completed shutdown and closed all open buckets {"coreId":"0xefe9906200000006"} INFO [2022-01-10 05:49:48,229] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][235us] Closed bucket "default" {"coreId":"0xefe9906200000005"} INFO [2022-01-10 05:49:48,229] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][84us] Node disconnected {"coreId":"0xefe9906200000005","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 05:49:48,229] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][83us] Node disconnected {"coreId":"0xefe9906200000005","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 05:49:48,229] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][85us] Node disconnected {"coreId":"0xefe9906200000005","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 05:49:48,229] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][126us] Node disconnected {"coreId":"0xefe9906200000005","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 05:49:48,229] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][16ms] Completed shutdown and closed all open buckets {"coreId":"0xefe9906200000005"} INFO [2022-01-10 05:49:48,358] jepsen test runner - jepsen.core Run complete, writing INFO [2022-01-10 05:49:48,362] jepsen fressian - couchbase.core Not writing fressian INFO [2022-01-10 05:49:50,301] jepsen test runner - jepsen.core Snarfing log files INFO [2022-01-10 05:49:56,718] jepsen node 172.23.107.227 - couchbase.util Running cbcollect_info on 172.23.107.227 INFO [2022-01-10 05:49:57,554] jepsen node 172.23.107.228 - couchbase.util Running cbcollect_info on 172.23.107.228 INFO [2022-01-10 05:49:59,649] jepsen node 172.23.106.61 - couchbase.util Running cbcollect_info on 172.23.106.61 INFO [2022-01-10 05:50:01,424] jepsen node 172.23.105.187 - couchbase.util Running cbcollect_info on 172.23.105.187 INFO [2022-01-10 05:55:32,476] jepsen node 172.23.107.227 - couchbase.util Creating hashtable dump on 172.23.107.227 INFO [2022-01-10 05:58:46,102] jepsen node 172.23.107.227 - couchbase.util Collecting tcp packet capture INFO [2022-01-10 05:59:39,779] jepsen node 172.23.105.187 - couchbase.util Creating hashtable dump on 172.23.105.187 INFO [2022-01-10 06:00:47,862] jepsen node 172.23.107.228 - couchbase.util Creating hashtable dump on 172.23.107.228 INFO [2022-01-10 06:01:58,048] jepsen node 172.23.106.61 - couchbase.util Creating hashtable dump on 172.23.106.61 INFO [2022-01-10 06:02:59,236] jepsen node 172.23.105.187 - couchbase.util Collecting tcp packet capture INFO [2022-01-10 06:04:06,303] jepsen node 172.23.107.228 - couchbase.util Collecting tcp packet capture INFO [2022-01-10 06:05:15,894] jepsen node 172.23.106.61 - couchbase.util Collecting tcp packet capture INFO [2022-01-10 06:05:45,543] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/users.dets to cbcollect_info_ns_1@172.23.107.227_20220110-135029/users.dets INFO [2022-01-10 06:05:49,165] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ini.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ini.log INFO [2022-01-10 06:05:50,489] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_access.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_access.log INFO [2022-01-10 06:05:51,801] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_dcpdebug.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_dcpdebug.log INFO [2022-01-10 06:05:53,115] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.goxdcr.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.goxdcr.log INFO [2022-01-10 06:05:54,492] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/goxdcr_replications.json to cbcollect_info_ns_1@172.23.107.227_20220110-135029/goxdcr_replications.json INFO [2022-01-10 06:05:55,778] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/stats_snapshot/01FS24AZ00TV32DKEM6C8GCM1S/meta.json to cbcollect_info_ns_1@172.23.107.227_20220110-135029/stats_snapshot/01FS24AZ00TV32DKEM6C8GCM1S/meta.json INFO [2022-01-10 06:05:56,886] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/goxdcr_mprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/goxdcr_mprof.log INFO [2022-01-10 06:05:58,118] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.stats.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.stats.log INFO [2022-01-10 06:05:59,480] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.indexer_stats.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.indexer_stats.log INFO [2022-01-10 06:06:00,794] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/syslog.tar.gz to cbcollect_info_ns_1@172.23.107.227_20220110-135029/syslog.tar.gz INFO [2022-01-10 06:06:02,135] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/diag.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/diag.log INFO [2022-01-10 06:06:03,535] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/systemd_journal.gz to cbcollect_info_ns_1@172.23.107.227_20220110-135029/systemd_journal.gz INFO [2022-01-10 06:06:07,333] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/projector_pprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/projector_pprof.log INFO [2022-01-10 06:06:08,450] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.projector.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.projector.log INFO [2022-01-10 06:06:09,789] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_trace.json to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_trace.json INFO [2022-01-10 06:06:11,097] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.couchdb.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.couchdb.log INFO [2022-01-10 06:06:12,398] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_info.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_info.log INFO [2022-01-10 06:06:12,679] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/projector_mprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/projector_mprof.log INFO [2022-01-10 06:06:13,710] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/projector_mprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/projector_mprof.log INFO [2022-01-10 06:06:15,046] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_cbas_debug.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_cbas_debug.log INFO [2022-01-10 06:06:16,383] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_log INFO [2022-01-10 06:06:17,371] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_access.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_access.log INFO [2022-01-10 06:06:17,681] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/kv_trace.json to cbcollect_info_ns_1@172.23.107.227_20220110-135029/kv_trace.json INFO [2022-01-10 06:06:18,724] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.babysitter.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.babysitter.log INFO [2022-01-10 06:06:19,015] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/Couchbase-172.23.107.227.pcap.1641821702.gz to Couchbase-172.23.107.227.pcap.1641821702.gz INFO [2022-01-10 06:06:20,084] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.http_access_internal.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.http_access_internal.log INFO [2022-01-10 06:06:21,243] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/rebalance_report_20220110T133619.json to cbcollect_info_ns_1@172.23.105.187_20220110-135039/rebalance_report_20220110T133619.json INFO [2022-01-10 06:06:22,709] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_error.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_error.log INFO [2022-01-10 06:06:24,098] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_trace.json to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_trace.json INFO [2022-01-10 06:06:25,238] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.couchdb.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.couchdb.log INFO [2022-01-10 06:06:26,573] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/projector_pprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/projector_pprof.log INFO [2022-01-10 06:06:27,918] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/systemd_journal.gz to cbcollect_info_ns_1@172.23.105.187_20220110-135039/systemd_journal.gz INFO [2022-01-10 06:06:31,418] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/event_log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/event_log INFO [2022-01-10 06:06:32,729] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.backup_service.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.backup_service.log INFO [2022-01-10 06:06:34,039] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.xdcr_target.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.xdcr_target.log INFO [2022-01-10 06:06:35,388] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/diag.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/diag.log INFO [2022-01-10 06:06:36,611] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/syslog.tar.gz to cbcollect_info_ns_1@172.23.105.187_20220110-135039/syslog.tar.gz INFO [2022-01-10 06:06:38,170] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.indexer_stats.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.indexer_stats.log INFO [2022-01-10 06:06:39,520] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/mdocs.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/mdocs.log INFO [2022-01-10 06:06:40,908] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/stats.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/stats.log INFO [2022-01-10 06:06:42,479] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.fts.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.fts.log INFO [2022-01-10 06:06:43,741] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_info.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_info.log INFO [2022-01-10 06:06:44,980] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_warn.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_warn.log INFO [2022-01-10 06:06:46,394] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.metakv.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.metakv.log INFO [2022-01-10 06:06:47,776] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/goxdcr_replications.json to cbcollect_info_ns_1@172.23.105.187_20220110-135039/goxdcr_replications.json INFO [2022-01-10 06:06:49,165] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/users.dets to cbcollect_info_ns_1@172.23.105.187_20220110-135039/users.dets INFO [2022-01-10 06:06:50,472] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/goxdcr_cprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/goxdcr_cprof.log INFO [2022-01-10 06:06:51,809] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.error.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.error.log INFO [2022-01-10 06:06:53,153] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/stats_snapshot/01FS24HB1J2J6VA14SXN7E4F0N/meta.json to cbcollect_info_ns_1@172.23.105.187_20220110-135039/stats_snapshot/01FS24HB1J2J6VA14SXN7E4F0N/meta.json INFO [2022-01-10 06:06:54,502] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_dcpdebug.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_dcpdebug.log INFO [2022-01-10 06:06:55,867] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/couchbase.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/couchbase.log INFO [2022-01-10 06:06:57,462] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.json_rpc.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.json_rpc.log INFO [2022-01-10 06:06:58,845] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/goxdcr_remote_clusters.json to cbcollect_info_ns_1@172.23.105.187_20220110-135039/goxdcr_remote_clusters.json INFO [2022-01-10 06:07:00,186] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.indexer.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.indexer.log INFO [2022-01-10 06:07:01,516] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/Couchbase-172.23.105.187.pcap.1641821702.gz to Couchbase-172.23.105.187.pcap.1641821702.gz INFO [2022-01-10 06:07:42,178] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.http_access_internal.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.http_access_internal.log INFO [2022-01-10 06:07:45,462] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.babysitter.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.babysitter.log INFO [2022-01-10 06:07:46,830] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_info.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_info.log INFO [2022-01-10 06:07:48,187] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.http_access.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.http_access.log INFO [2022-01-10 06:07:49,567] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.debug.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.debug.log INFO [2022-01-10 06:07:51,157] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/memcached.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/memcached.log INFO [2022-01-10 06:08:34,862] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.couchdb.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.couchdb.log INFO [2022-01-10 06:08:36,306] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.error.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.error.log INFO [2022-01-10 06:08:37,699] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.stats.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.stats.log INFO [2022-01-10 06:08:39,112] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/master_events.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/master_events.log INFO [2022-01-10 06:08:40,516] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/projector_cprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/projector_cprof.log INFO [2022-01-10 06:08:41,873] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.mapreduce_errors.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.mapreduce_errors.log INFO [2022-01-10 06:08:43,151] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.ns_couchdb.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.ns_couchdb.log INFO [2022-01-10 06:08:44,468] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.indexer_stats.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.indexer_stats.log INFO [2022-01-10 06:08:45,716] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.goxdcr.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.goxdcr.log INFO [2022-01-10 06:08:46,916] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_error.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_error.log INFO [2022-01-10 06:08:48,313] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/syslog.tar.gz to cbcollect_info_ns_1@172.23.107.228_20220110-135031/syslog.tar.gz INFO [2022-01-10 06:08:49,920] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.prometheus.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.prometheus.log INFO [2022-01-10 06:08:51,281] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/rebalance_report_20220110T135111.json to cbcollect_info_ns_1@172.23.107.228_20220110-135031/rebalance_report_20220110T135111.json INFO [2022-01-10 06:08:52,700] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_trace.json to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_trace.json INFO [2022-01-10 06:08:54,112] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/stats_snapshot/01FS24HBCCZ0JEPSZVH2STHHS7/meta.json to cbcollect_info_ns_1@172.23.107.228_20220110-135031/stats_snapshot/01FS24HBCCZ0JEPSZVH2STHHS7/meta.json INFO [2022-01-10 06:08:55,477] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/stats_snapshot/01FS24HBCCZ0JEPSZVH2STHHS7/tombstones to cbcollect_info_ns_1@172.23.107.228_20220110-135031/stats_snapshot/01FS24HBCCZ0JEPSZVH2STHHS7/tombstones INFO [2022-01-10 06:08:56,835] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_warn.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_warn.log INFO [2022-01-10 06:08:58,195] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/Couchbase-172.23.107.228.pcap.1641821702.gz to Couchbase-172.23.107.228.pcap.1641821702.gz INFO [2022-01-10 06:09:25,308] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/kv_trace.json to cbcollect_info_ns_1@172.23.105.187_20220110-135039/kv_trace.json INFO [2022-01-10 06:09:29,246] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_debug.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_debug.log INFO [2022-01-10 06:09:30,625] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/goxdcr_pprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/goxdcr_pprof.log INFO [2022-01-10 06:09:32,059] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.prometheus.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.prometheus.log INFO [2022-01-10 06:09:33,105] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.debug.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.debug.log INFO [2022-01-10 06:09:35,157] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ini.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ini.log INFO [2022-01-10 06:09:36,574] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.views.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.views.log INFO [2022-01-10 06:09:37,960] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ddocs.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ddocs.log INFO [2022-01-10 06:09:39,348] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.http_access.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.http_access.log INFO [2022-01-10 06:09:40,752] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.stats.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.stats.log INFO [2022-01-10 06:09:42,197] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.query.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.query.log INFO [2022-01-10 06:09:43,562] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/rebalance_report_20220110T133537.json to cbcollect_info_ns_1@172.23.105.187_20220110-135039/rebalance_report_20220110T133537.json INFO [2022-01-10 06:09:44,884] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_shutdown.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_shutdown.log INFO [2022-01-10 06:09:46,262] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/rebalance_report_20220110T135111.json to cbcollect_info_ns_1@172.23.105.187_20220110-135039/rebalance_report_20220110T135111.json INFO [2022-01-10 06:09:47,691] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/stats_snapshot/01FS24HB1J2J6VA14SXN7E4F0N/index to cbcollect_info_ns_1@172.23.105.187_20220110-135039/stats_snapshot/01FS24HB1J2J6VA14SXN7E4F0N/index INFO [2022-01-10 06:09:49,093] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.info.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.info.log INFO [2022-01-10 06:09:50,562] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.ns_couchdb.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.ns_couchdb.log INFO [2022-01-10 06:09:51,953] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/master_events.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/master_events.log INFO [2022-01-10 06:09:53,320] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_log INFO [2022-01-10 06:09:54,720] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_cbas_debug.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.analytics_cbas_debug.log INFO [2022-01-10 06:09:55,983] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/stats_snapshot/01FS24HB1J2J6VA14SXN7E4F0N/chunks/000001 to cbcollect_info_ns_1@172.23.105.187_20220110-135039/stats_snapshot/01FS24HB1J2J6VA14SXN7E4F0N/chunks/000001 INFO [2022-01-10 06:09:57,374] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/hashtable_dump.txt to hashtable_dump.txt INFO [2022-01-10 06:09:58,864] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.projector.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.projector.log INFO [2022-01-10 06:10:00,218] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.mapreduce_errors.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.mapreduce_errors.log INFO [2022-01-10 06:10:01,494] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/stats_snapshot/01FS24HB1J2J6VA14SXN7E4F0N/tombstones to cbcollect_info_ns_1@172.23.105.187_20220110-135039/stats_snapshot/01FS24HB1J2J6VA14SXN7E4F0N/tombstones INFO [2022-01-10 06:10:02,669] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_warn.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_warn.log INFO [2022-01-10 06:10:02,827] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/couchstore_local.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/couchstore_local.log INFO [2022-01-10 06:10:04,220] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/cbcollect_info.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/cbcollect_info.log INFO [2022-01-10 06:10:04,348] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.indexer.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.indexer.log INFO [2022-01-10 06:10:04,859] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/stats_snapshot/01FS24HJPFXJC166ZCRDX51YP9/meta.json to cbcollect_info_ns_1@172.23.106.61_20220110-135033/stats_snapshot/01FS24HJPFXJC166ZCRDX51YP9/meta.json INFO [2022-01-10 06:10:05,574] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/.ignore to .ignore INFO [2022-01-10 06:10:05,745] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.debug.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.debug.log INFO [2022-01-10 06:10:06,229] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.indexer.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.indexer.log INFO [2022-01-10 06:10:07,079] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/memcached.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/memcached.log INFO [2022-01-10 06:10:07,332] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/memcached.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/memcached.log INFO [2022-01-10 06:10:07,597] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/Couchbase-172.23.106.61.pcap.1641821702.gz to Couchbase-172.23.106.61.pcap.1641821702.gz INFO [2022-01-10 06:10:19,059] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.babysitter.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.babysitter.log INFO [2022-01-10 06:10:20,479] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/cbcollect_info.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/cbcollect_info.log INFO [2022-01-10 06:10:21,891] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/master_events.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/master_events.log INFO [2022-01-10 06:10:23,386] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_debug.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_debug.log INFO [2022-01-10 06:10:24,952] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ddocs.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ddocs.log INFO [2022-01-10 06:10:26,419] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.prometheus.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.prometheus.log INFO [2022-01-10 06:10:27,862] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.mapreduce_errors.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.mapreduce_errors.log INFO [2022-01-10 06:10:29,373] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.fts.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.fts.log INFO [2022-01-10 06:10:30,781] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_error.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_error.log INFO [2022-01-10 06:10:32,193] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.info.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.info.log INFO [2022-01-10 06:10:33,639] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/rebalance_report_20220110T133619.json to cbcollect_info_ns_1@172.23.107.227_20220110-135029/rebalance_report_20220110T133619.json INFO [2022-01-10 06:10:35,098] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_shutdown.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_shutdown.log INFO [2022-01-10 06:10:36,517] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/event_log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/event_log INFO [2022-01-10 06:10:38,027] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/rebalance_report_20220110T133537.json to cbcollect_info_ns_1@172.23.107.227_20220110-135029/rebalance_report_20220110T133537.json INFO [2022-01-10 06:10:38,289] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.reports.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.reports.log INFO [2022-01-10 06:10:39,361] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_warn.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.analytics_warn.log INFO [2022-01-10 06:10:39,759] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/chronicle_logs.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/chronicle_logs.log INFO [2022-01-10 06:10:40,709] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/couchbase.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/couchbase.log INFO [2022-01-10 06:10:41,342] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/goxdcr_mprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/goxdcr_mprof.log INFO [2022-01-10 06:10:42,274] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.error.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.error.log INFO [2022-01-10 06:10:42,754] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/projector_cprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/projector_cprof.log INFO [2022-01-10 06:10:43,672] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/projector_cprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/projector_cprof.log INFO [2022-01-10 06:10:44,067] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.eventing.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.eventing.log INFO [2022-01-10 06:10:44,920] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.http_access.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.http_access.log INFO [2022-01-10 06:10:45,425] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.goxdcr.log to cbcollect_info_ns_1@172.23.105.187_20220110-135039/ns_server.goxdcr.log INFO [2022-01-10 06:10:46,362] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.eventing.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.eventing.log INFO [2022-01-10 06:10:47,668] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/hashtable_dump.txt to hashtable_dump.txt INFO [2022-01-10 06:10:48,986] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/mdocs.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/mdocs.log INFO [2022-01-10 06:10:50,355] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/stats_snapshot/01FS24AZ00TV32DKEM6C8GCM1S/index to cbcollect_info_ns_1@172.23.107.227_20220110-135029/stats_snapshot/01FS24AZ00TV32DKEM6C8GCM1S/index INFO [2022-01-10 06:10:51,708] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.views.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.views.log INFO [2022-01-10 06:10:53,098] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/chronicle_logs.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/chronicle_logs.log INFO [2022-01-10 06:10:54,460] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/stats_snapshot/01FS24AZ00TV32DKEM6C8GCM1S/tombstones to cbcollect_info_ns_1@172.23.107.227_20220110-135029/stats_snapshot/01FS24AZ00TV32DKEM6C8GCM1S/tombstones INFO [2022-01-10 06:10:55,887] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/goxdcr_pprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/goxdcr_pprof.log INFO [2022-01-10 06:10:57,244] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/.ignore to .ignore INFO [2022-01-10 06:10:58,627] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.backup_service.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.backup_service.log INFO [2022-01-10 06:10:59,983] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.query.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.query.log INFO [2022-01-10 06:11:01,319] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/stats.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/stats.log INFO [2022-01-10 06:11:02,660] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.json_rpc.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.json_rpc.log INFO [2022-01-10 06:11:03,993] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.ns_couchdb.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.ns_couchdb.log INFO [2022-01-10 06:11:05,346] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/stats_snapshot/01FS24AZ00TV32DKEM6C8GCM1S/chunks/000001 to cbcollect_info_ns_1@172.23.107.227_20220110-135029/stats_snapshot/01FS24AZ00TV32DKEM6C8GCM1S/chunks/000001 INFO [2022-01-10 06:11:06,675] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/goxdcr_cprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/goxdcr_cprof.log INFO [2022-01-10 06:11:07,995] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.metakv.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.metakv.log INFO [2022-01-10 06:11:09,343] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/couchstore_local.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/couchstore_local.log INFO [2022-01-10 06:11:10,658] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/goxdcr_remote_clusters.json to cbcollect_info_ns_1@172.23.107.227_20220110-135029/goxdcr_remote_clusters.json INFO [2022-01-10 06:11:12,015] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.reports.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.reports.log INFO [2022-01-10 06:11:13,392] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.xdcr_target.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.xdcr_target.log INFO [2022-01-10 06:11:14,779] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.http_access_internal.log to cbcollect_info_ns_1@172.23.107.227_20220110-135029/ns_server.http_access_internal.log INFO [2022-01-10 06:11:28,266] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/mdocs.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/mdocs.log INFO [2022-01-10 06:11:29,559] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/goxdcr_mprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/goxdcr_mprof.log INFO [2022-01-10 06:11:30,866] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/projector_mprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/projector_mprof.log INFO [2022-01-10 06:11:32,046] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.xdcr_target.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.xdcr_target.log INFO [2022-01-10 06:11:33,363] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.fts.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.fts.log INFO [2022-01-10 06:11:34,683] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_debug.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_debug.log INFO [2022-01-10 06:11:36,007] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_dcpdebug.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_dcpdebug.log INFO [2022-01-10 06:11:37,325] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.eventing.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.eventing.log INFO [2022-01-10 06:11:38,678] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/goxdcr_replications.json to cbcollect_info_ns_1@172.23.107.228_20220110-135031/goxdcr_replications.json INFO [2022-01-10 06:11:39,975] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.metakv.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.metakv.log INFO [2022-01-10 06:11:41,266] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_access.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_access.log INFO [2022-01-10 06:11:42,564] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/goxdcr_remote_clusters.json to cbcollect_info_ns_1@172.23.107.228_20220110-135031/goxdcr_remote_clusters.json INFO [2022-01-10 06:11:43,887] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.json_rpc.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.json_rpc.log INFO [2022-01-10 06:11:45,037] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/goxdcr_cprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/goxdcr_cprof.log INFO [2022-01-10 06:11:46,292] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/event_log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/event_log INFO [2022-01-10 06:11:47,255] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.query.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.query.log INFO [2022-01-10 06:11:48,555] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/hashtable_dump.txt to hashtable_dump.txt INFO [2022-01-10 06:11:49,928] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.projector.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.projector.log INFO [2022-01-10 06:11:51,246] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/stats.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/stats.log INFO [2022-01-10 06:11:52,782] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_log INFO [2022-01-10 06:11:54,098] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/rebalance_report_20220110T133619.json to cbcollect_info_ns_1@172.23.107.228_20220110-135031/rebalance_report_20220110T133619.json INFO [2022-01-10 06:11:55,394] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.views.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.views.log INFO [2022-01-10 06:11:56,675] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/goxdcr_pprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/goxdcr_pprof.log INFO [2022-01-10 06:11:57,858] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.info.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.info.log INFO [2022-01-10 06:11:59,196] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/.ignore to .ignore INFO [2022-01-10 06:12:00,365] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ini.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ini.log INFO [2022-01-10 06:12:01,671] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/projector_pprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/projector_pprof.log INFO [2022-01-10 06:12:02,957] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_shutdown.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_shutdown.log INFO [2022-01-10 06:12:04,291] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.indexer.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.indexer.log INFO [2022-01-10 06:12:05,496] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/cbcollect_info.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/cbcollect_info.log INFO [2022-01-10 06:12:06,825] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/systemd_journal.gz to cbcollect_info_ns_1@172.23.107.228_20220110-135031/systemd_journal.gz INFO [2022-01-10 06:12:10,064] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/diag.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/diag.log INFO [2022-01-10 06:12:11,273] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.reports.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.reports.log INFO [2022-01-10 06:12:12,592] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.backup_service.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.backup_service.log INFO [2022-01-10 06:12:13,747] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/couchstore_local.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/couchstore_local.log INFO [2022-01-10 06:12:15,050] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/stats_snapshot/01FS24HBCCZ0JEPSZVH2STHHS7/chunks/000001 to cbcollect_info_ns_1@172.23.107.228_20220110-135031/stats_snapshot/01FS24HBCCZ0JEPSZVH2STHHS7/chunks/000001 INFO [2022-01-10 06:12:16,407] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/kv_trace.json to cbcollect_info_ns_1@172.23.107.228_20220110-135031/kv_trace.json INFO [2022-01-10 06:12:20,212] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_cbas_debug.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ns_server.analytics_cbas_debug.log INFO [2022-01-10 06:12:21,519] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/couchbase.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/couchbase.log INFO [2022-01-10 06:12:23,083] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/ddocs.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/ddocs.log INFO [2022-01-10 06:12:24,422] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/chronicle_logs.log to cbcollect_info_ns_1@172.23.107.228_20220110-135031/chronicle_logs.log INFO [2022-01-10 06:12:25,910] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/rebalance_report_20220110T133537.json to cbcollect_info_ns_1@172.23.107.228_20220110-135031/rebalance_report_20220110T133537.json INFO [2022-01-10 06:12:27,226] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/users.dets to cbcollect_info_ns_1@172.23.107.228_20220110-135031/users.dets INFO [2022-01-10 06:12:28,515] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-135031/stats_snapshot/01FS24HBCCZ0JEPSZVH2STHHS7/index to cbcollect_info_ns_1@172.23.107.228_20220110-135031/stats_snapshot/01FS24HBCCZ0JEPSZVH2STHHS7/index INFO [2022-01-10 06:13:22,586] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/diag.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/diag.log INFO [2022-01-10 06:13:23,932] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.indexer_stats.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.indexer_stats.log INFO [2022-01-10 06:13:25,227] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_dcpdebug.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_dcpdebug.log INFO [2022-01-10 06:13:26,478] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/rebalance_report_20220110T133537.json to cbcollect_info_ns_1@172.23.106.61_20220110-135033/rebalance_report_20220110T133537.json INFO [2022-01-10 06:13:27,775] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_shutdown.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_shutdown.log INFO [2022-01-10 06:13:29,037] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/stats.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/stats.log INFO [2022-01-10 06:13:30,562] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/goxdcr_replications.json to cbcollect_info_ns_1@172.23.106.61_20220110-135033/goxdcr_replications.json INFO [2022-01-10 06:13:31,843] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/cbcollect_info.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/cbcollect_info.log INFO [2022-01-10 06:13:32,916] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/couchbase.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/couchbase.log INFO [2022-01-10 06:13:34,468] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.error.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.error.log INFO [2022-01-10 06:13:35,788] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.query.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.query.log INFO [2022-01-10 06:13:37,075] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.info.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.info.log INFO [2022-01-10 06:13:38,500] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/projector_pprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/projector_pprof.log INFO [2022-01-10 06:13:39,800] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/goxdcr_pprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/goxdcr_pprof.log INFO [2022-01-10 06:13:41,104] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/chronicle_logs.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/chronicle_logs.log INFO [2022-01-10 06:13:42,562] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/kv_trace.json to cbcollect_info_ns_1@172.23.106.61_20220110-135033/kv_trace.json INFO [2022-01-10 06:13:46,353] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_debug.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_debug.log INFO [2022-01-10 06:13:47,659] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ini.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ini.log INFO [2022-01-10 06:13:48,944] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.babysitter.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.babysitter.log INFO [2022-01-10 06:13:50,233] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/systemd_journal.gz to cbcollect_info_ns_1@172.23.106.61_20220110-135033/systemd_journal.gz INFO [2022-01-10 06:13:53,760] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/projector_cprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/projector_cprof.log INFO [2022-01-10 06:13:55,048] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/mdocs.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/mdocs.log INFO [2022-01-10 06:13:56,346] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.views.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.views.log INFO [2022-01-10 06:13:57,620] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/users.dets to cbcollect_info_ns_1@172.23.106.61_20220110-135033/users.dets INFO [2022-01-10 06:13:58,910] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.reports.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.reports.log INFO [2022-01-10 06:14:00,208] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.mapreduce_errors.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.mapreduce_errors.log INFO [2022-01-10 06:14:01,499] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.eventing.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.eventing.log INFO [2022-01-10 06:14:02,790] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.http_access_internal.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.http_access_internal.log INFO [2022-01-10 06:14:04,081] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.http_access.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.http_access.log INFO [2022-01-10 06:14:05,371] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.fts.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.fts.log INFO [2022-01-10 06:14:06,668] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.ns_couchdb.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.ns_couchdb.log INFO [2022-01-10 06:14:07,986] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_cbas_debug.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_cbas_debug.log INFO [2022-01-10 06:14:09,291] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/rebalance_report_20220110T133619.json to cbcollect_info_ns_1@172.23.106.61_20220110-135033/rebalance_report_20220110T133619.json INFO [2022-01-10 06:14:10,597] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/event_log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/event_log INFO [2022-01-10 06:14:11,879] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_access.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_access.log INFO [2022-01-10 06:14:13,119] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.goxdcr.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.goxdcr.log INFO [2022-01-10 06:14:14,419] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.metakv.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.metakv.log INFO [2022-01-10 06:14:15,721] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/goxdcr_cprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/goxdcr_cprof.log INFO [2022-01-10 06:14:17,012] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/rebalance_report_20220110T135111.json to cbcollect_info_ns_1@172.23.106.61_20220110-135033/rebalance_report_20220110T135111.json INFO [2022-01-10 06:14:18,322] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/memcached.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/memcached.log INFO [2022-01-10 06:14:59,422] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.debug.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.debug.log INFO [2022-01-10 06:15:03,508] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/couchstore_local.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/couchstore_local.log INFO [2022-01-10 06:15:04,830] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/hashtable_dump.txt to hashtable_dump.txt INFO [2022-01-10 06:15:06,220] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ddocs.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ddocs.log INFO [2022-01-10 06:15:07,527] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/stats_snapshot/01FS24HJPFXJC166ZCRDX51YP9/tombstones to cbcollect_info_ns_1@172.23.106.61_20220110-135033/stats_snapshot/01FS24HJPFXJC166ZCRDX51YP9/tombstones INFO [2022-01-10 06:15:08,847] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.json_rpc.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.json_rpc.log INFO [2022-01-10 06:15:10,149] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_info.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_info.log INFO [2022-01-10 06:15:11,487] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_log INFO [2022-01-10 06:15:12,779] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/goxdcr_mprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/goxdcr_mprof.log INFO [2022-01-10 06:15:14,068] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.backup_service.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.backup_service.log INFO [2022-01-10 06:15:15,347] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/syslog.tar.gz to cbcollect_info_ns_1@172.23.106.61_20220110-135033/syslog.tar.gz INFO [2022-01-10 06:15:16,772] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.couchdb.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.couchdb.log INFO [2022-01-10 06:15:18,043] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/.ignore to .ignore INFO [2022-01-10 06:15:19,338] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/projector_mprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/projector_mprof.log INFO [2022-01-10 06:15:20,637] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.xdcr_target.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.xdcr_target.log INFO [2022-01-10 06:15:21,926] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/stats_snapshot/01FS24HJPFXJC166ZCRDX51YP9/chunks/000001 to cbcollect_info_ns_1@172.23.106.61_20220110-135033/stats_snapshot/01FS24HJPFXJC166ZCRDX51YP9/chunks/000001 INFO [2022-01-10 06:15:23,241] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_error.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_error.log INFO [2022-01-10 06:15:24,515] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_trace.json to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.analytics_trace.json INFO [2022-01-10 06:15:25,798] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.prometheus.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.prometheus.log INFO [2022-01-10 06:15:27,084] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.projector.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.projector.log INFO [2022-01-10 06:15:28,376] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/goxdcr_remote_clusters.json to cbcollect_info_ns_1@172.23.106.61_20220110-135033/goxdcr_remote_clusters.json INFO [2022-01-10 06:15:29,698] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.stats.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/ns_server.stats.log INFO [2022-01-10 06:15:31,003] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/stats_snapshot/01FS24HJPFXJC166ZCRDX51YP9/index to cbcollect_info_ns_1@172.23.106.61_20220110-135033/stats_snapshot/01FS24HJPFXJC166ZCRDX51YP9/index INFO [2022-01-10 06:15:32,334] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-135033/master_events.log to cbcollect_info_ns_1@172.23.106.61_20220110-135033/master_events.log INFO [2022-01-10 06:15:33,576] jepsen test runner - jepsen.core Snarfing log files WARN [2022-01-10 06:15:33,578] jepsen node 172.23.105.187 - couchbase.core Ignoring duplicate log collection request WARN [2022-01-10 06:15:33,578] jepsen node 172.23.107.227 - couchbase.core Ignoring duplicate log collection request WARN [2022-01-10 06:15:33,578] jepsen node 172.23.107.228 - couchbase.core Ignoring duplicate log collection request WARN [2022-01-10 06:15:33,578] jepsen node 172.23.106.61 - couchbase.core Ignoring duplicate log collection request INFO [2022-01-10 06:15:33,581] jepsen node 172.23.107.228 - couchbase.util Tearing down couchbase node INFO [2022-01-10 06:15:33,581] jepsen node 172.23.105.187 - couchbase.util Tearing down couchbase node INFO [2022-01-10 06:15:33,581] jepsen node 172.23.106.61 - couchbase.util Tearing down couchbase node INFO [2022-01-10 06:15:33,582] jepsen node 172.23.107.227 - couchbase.util Tearing down couchbase node INFO [2022-01-10 06:15:37,604] jepsen node 172.23.107.227 - couchbase.util Teardown Complete INFO [2022-01-10 06:15:37,902] jepsen node 172.23.106.61 - couchbase.util Teardown Complete INFO [2022-01-10 06:15:38,107] jepsen node 172.23.105.187 - couchbase.util Teardown Complete INFO [2022-01-10 06:15:38,312] jepsen node 172.23.107.228 - couchbase.util Teardown Complete INFO [2022-01-10 06:15:38,322] jepsen test runner - jepsen.core Analyzing... INFO [2022-01-10 06:15:44,325] knossos reporter - knossos.search {:linear {:space 4, :cost 1.34E+15, :op {:process 531, :type :ok, :f :read, :value 30, :index 2165, :time 112562713395, :cas 1641821774473723904}}, :wgl [:WGL]} INFO [2022-01-10 06:15:44,327] knossos reporter - knossos.search {:linear {:space 1, :cost 6.05E+15, :op {:process 464, :type :ok, :f :read, :value :nil, :index 1993, :time 105864908587}}, :wgl [:WGL]} INFO [2022-01-10 06:15:44,327] knossos reporter - knossos.search {:linear {:space 1, :cost 1.96E+13, :op {:process 421, :type :ok, :f :read, :value :nil, :index 1461, :time 85761119620}}, :wgl [:WGL]} INFO [2022-01-10 06:15:44,327] knossos reporter - knossos.search {:linear {:space 8, :cost 9.76E+12, :op {:process 425, :type :ok, :f :read, :value 7, :index 1673, :time 93163989938, :cas 1641821772992020480}}, :wgl [:WGL]} INFO [2022-01-10 06:15:44,327] knossos reporter - knossos.search {:linear {:space 1, :cost 1.22E+12, :op {:process 382, :type :ok, :f :read, :value :nil, :index 1715, :time 94778993602}}, :wgl [:WGL]} INFO [2022-01-10 06:15:44,340] knossos reporter - knossos.search {:linear {:space 4, :cost 7.85E+13, :op {:process 343, :type :ok, :f :read, :value 33, :index 1675, :time 93170247122, :cas 1641821772925239296}}, :wgl [:WGL]} INFO [2022-01-10 06:15:44,363] knossos reporter - knossos.search {:linear {:space 1, :cost 8.10E+10, :op {:process 272, :type :ok, :f :read, :value :nil, :index 1785, :time 97605012959}}, :wgl [:WGL]} INFO [2022-01-10 06:15:44,368] knossos reporter - knossos.search {:linear {:space 4, :cost 7.85E+13, :op {:process 365, :type :ok, :f :read, :value 41, :index 1641, :time 92060440074, :cas 1641821774618951680}}, :wgl [:WGL]} INFO [2022-01-10 06:15:44,368] knossos reporter - knossos.search {:linear {:space 1, :cost 1.22E+12, :op {:process 316, :type :ok, :f :read, :value :nil, :index 1727, :time 95298690612}}, :wgl [:WGL]} INFO [2022-01-10 06:15:44,381] knossos reporter - knossos.search {:linear {:space 256, :cost 8.57E+16, :op {:process 489, :type :ok, :f :read, :value 23, :index 2183, :time 113013363595, :cas 1641821809248043008}}, :wgl [:WGL]} INFO [2022-01-10 06:15:49,615] knossos reporter - knossos.search {:linear {:space 8, :cost 1.57E+14, :op {:process 365, :type :ok, :f :read, :value 41, :index 1661, :time 92549991712, :cas 1641821774618951680}}, :wgl [:WGL]} INFO [2022-01-10 06:15:49,617] knossos reporter - knossos.search {:linear {:space 128, :cost 2.96E+20, :op {:process 491, :type :ok, :f :read, :value 23, :index 2599, :time 127260596005, :cas 1641821809248043008}}, :wgl [:WGL]} INFO [2022-01-10 06:15:49,618] knossos reporter - knossos.search {:linear {:space 1, :cost 3.35E+14, :op {:process 383, :type :ok, :f :read, :value :nil, :index 1815, :time 98468310726}}, :wgl [:WGL]} INFO [2022-01-10 06:15:49,618] knossos reporter - knossos.search {:linear {:space 1, :cost 3.35E+14, :op {:process 512, :type :ok, :f :read, :value :nil, :index 1607, :time 91354803377}}, :wgl [:WGL]} INFO [2022-01-10 06:15:49,618] knossos reporter - knossos.search {:linear {:space 8, :cost 9.76E+12, :op {:process 423, :type :ok, :f :read, :value 7, :index 1707, :time 94329057902, :cas 1641821772992020480}}, :wgl [:WGL]} INFO [2022-01-10 06:15:49,615] knossos reporter - knossos.search {:linear {:space 1, :cost 3.35E+14, :op {:process 495, :type :ok, :f :read, :value :nil, :index 1869, :time 100647364203}}, :wgl [:WGL]} INFO [2022-01-10 06:15:49,615] knossos reporter - knossos.search {:linear {:space 1, :cost 1.96E+13, :op {:process 451, :type :ok, :f :read, :value :nil, :index 2001, :time 106344079384}}, :wgl [:WGL]} INFO [2022-01-10 06:15:49,615] knossos reporter - knossos.search {:linear {:space 4, :cost 4.61E+17, :op {:process 433, :type :ok, :f :read, :value 33, :index 1933, :time 103673194960, :cas 1641821772925239296}}, :wgl [:WGL]} INFO [2022-01-10 06:15:54,706] knossos reporter - knossos.search {:linear {:space 1, :cost 2.47E+22, :op {:process 540, :type :ok, :f :read, :value :nil, :index 2633, :time 128116015545}}, :wgl [:WGL]} INFO [2022-01-10 06:15:54,707] knossos reporter - knossos.search {:linear {:space 4, :cost 1.95E+20, :op {:process 442, :type :ok, :f :read, :value 30, :index 2549, :time 125960150793, :cas 1641821774473723904}}, :wgl [:WGL]} INFO [2022-01-10 06:15:54,707] knossos reporter - knossos.search {:linear {:space 4, :cost 7.85E+13, :op {:process 424, :type :ok, :f :read, :value 7, :index 1901, :time 101807503563, :cas 1641821772992020480}}, :wgl [:WGL]} INFO [2022-01-10 06:15:54,732] knossos reporter - knossos.search {:linear {:space 8, :cost 1.57E+14, :op {:process 453, :type :ok, :f :read, :value 41, :index 1695, :time 93862068646, :cas 1641821774618951680}}, :wgl [:WGL]} INFO [2022-01-10 06:15:54,737] knossos reporter - knossos.search {:linear {:space 256, :cost 5.92E+20, :op {:process 490, :type :ok, :f :read, :value 23, :index 2603, :time 127337571569, :cas 1641821809248043008}}, :wgl [:WGL]} INFO [2022-01-10 06:16:00,038] knossos reporter - knossos.search {:linear {:space 1, :cost 1.15E+17, :op {:process 472, :type :ok, :f :read, :value :nil, :index 1973, :time 105419838848}}, :wgl [:WGL]} INFO [2022-01-10 06:16:00,039] knossos reporter - knossos.search {:linear {:space 1, :cost 1.15E+17, :op {:process 510, :type :ok, :f :read, :value :nil, :index 1755, :time 96437911213}}, :wgl [:WGL]} INFO [2022-01-10 06:16:00,051] knossos reporter - knossos.search {:linear {:space 4, :cost 7.85E+13, :op {:process 424, :type :ok, :f :read, :value 7, :index 1927, :time 103096538125, :cas 1641821772992020480}}, :wgl [:WGL]} INFO [2022-01-10 06:16:00,052] knossos reporter - knossos.search {:linear {:space 4, :cost 4.29E+21, :op {:process 545, :type :ok, :f :read, :value 41, :index 2239, :time 115416604033, :cas 1641821774618951680}}, :wgl [:WGL]} INFO [2022-01-10 06:16:00,054] knossos reporter - knossos.search {:linear {:space 1, :cost 1.15E+17, :op {:process 587, :type :ok, :f :read, :value 2, :index 2093, :time 110407420813, :cas 1641821836586254336}}, :wgl [:WGL]} INFO [2022-01-10 06:16:05,052] knossos reporter - knossos.search {:linear {:space 4, :cost 1.95E+20, :op {:process 603, :type :ok, :f :read, :value 7, :index 2387, :time 120658101312, :cas 1641821772992020480}}, :wgl [:WGL]} INFO [2022-01-10 06:16:10,135] knossos reporter - knossos.search {:linear {:space 2, :cost 1.21E+16, :op {:process 463, :type :ok, :f :read, :value :nil, :index 1995, :time 105926644422}}, :wgl [:WGL]} INFO [2022-01-10 06:16:10,139] knossos reporter - knossos.search {:linear {:space 256, :cost 5.92E+20, :op {:process 491, :type :ok, :f :read, :value 23, :index 2605, :time 127438307172, :cas 1641821809248043008}}, :wgl [:WGL]} INFO [2022-01-10 06:16:20,144] knossos reporter - knossos.search {:linear {:space 128, :cost 2.96E+20, :op {:process 491, :type :ok, :f :read, :value 23, :index 2615, :time 127662519476, :cas 1641821809248043008}}, :wgl [:WGL]} INFO [2022-01-10 06:16:25,138] knossos reporter - knossos.search {:linear {:space 2, :cost 1.21E+16, :op {:process 463, :type :ok, :f :read, :value :nil, :index 2073, :time 109489587482}}, :wgl [:WGL]} INFO [2022-01-10 06:16:35,148] knossos reporter - knossos.search {:linear {:space 128, :cost 1.91E+27, :op {:process 580, :type :ok, :f :read, :value 23, :index 3127, :time 148978857634, :cas 1641821809248043008}}, :wgl [:WGL]} INFO [2022-01-10 06:16:45,755] knossos reporter - knossos.search {:linear {:space 4, :cost 9.25E+18, :op {:process 614, :type :ok, :f :read, :value 33, :index 2055, :time 108475472515, :cas 1641821772925239296}}, :wgl [:WGL]} INFO [2022-01-10 06:16:51,173] knossos reporter - knossos.search {:linear {:space 2, :cost 2.30E+17, :op {:process 464, :type :ok, :f :read, :value :nil, :index 2079, :time 109682819802}}, :wgl [:WGL]} INFO [2022-01-10 06:16:56,371] knossos reporter - knossos.search {:linear {:space 1, :cost 1.15E+17, :op {:process 510, :type :ok, :f :read, :value :nil, :index 1793, :time 97714323441}}, :wgl [:WGL]} INFO [2022-01-10 06:16:56,372] knossos reporter - knossos.search {:linear {:space 2, :cost 2.30E+17, :op {:process 472, :type :ok, :f :read, :value :nil, :index 1983, :time 105629277905}}, :wgl [:WGL]} INFO [2022-01-10 06:17:23,167] knossos reporter - knossos.search {:linear {:space 98304, :cost 1.13E+22, :op {:persist-to 0, :index 2112, :value 6, :time 110680112590, :process 585, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}}, :wgl [:WGL]} INFO [2022-01-10 06:17:29,083] knossos reporter - knossos.search {:linear {:space 98304, :cost 2.43E+27, :op {:process 675, :type :ok, :f :read, :value 2, :index 2491, :time 125230864390, :cas 1641821836586254336}}, :wgl [:WGL]} Exception in thread "Thread-175" java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at clojure.core$deref_future.invokeStatic(core.clj:2304) at clojure.core$future_call$reify__8477.deref(core.clj:6976) at clojure.core$deref.invokeStatic(core.clj:2324) at clojure.core$pmap$step__8490$fn__8494.invoke(core.clj:7027) at clojure.lang.LazySeq.sval(LazySeq.java:42) at clojure.lang.LazySeq.seq(LazySeq.java:51) at clojure.lang.RT.seq(RT.java:535) at clojure.core$seq__5419.invokeStatic(core.clj:139) at clojure.core$apply.invokeStatic(core.clj:662) at clojure.core$apply.invoke(core.clj:662) at knossos.linear$step.invokeStatic(linear.clj:199) at knossos.linear$step.invoke(linear.clj:169) at clojure.core$partial$fn__5861.invoke(core.clj:2643) at clojure.lang.PersistentVector.reduce(PersistentVector.java:343) at clojure.core$reduce.invokeStatic(core.clj:6829) at clojure.core$reduce.invoke(core.clj:6812) at knossos.linear$check.invokeStatic(linear.clj:228) at knossos.linear$check.invoke(linear.clj:225) at knossos.linear$start_analysis$fn__3319$fn__3320.invoke(linear.clj:277) at knossos.linear$start_analysis$fn__3319.invoke(linear.clj:277) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: Java heap space at org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.resize(NonBlockingHashMap.java:863) at org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.access$200(NonBlockingHashMap.java:713) at org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:649) at org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:354) at org.cliffc.high_scale_lib.NonBlockingHashMap.putIfAbsent(NonBlockingHashMap.java:321) at org.cliffc.high_scale_lib.NonBlockingHashSet.add(NonBlockingHashSet.java:32) at knossos.weak_cache_set.NBHSCacheSet.add_BANG_(weak_cache_set.clj:66) at knossos.linear$jit_linearizations.invokeStatic(linear.clj:78) at knossos.linear$jit_linearizations.invoke(linear.clj:66) at knossos.linear$jit_linearizations$recurrence__3268.invoke(linear.clj:98) at knossos.util$rkeep$fn__670$fn__671.invoke(util.clj:42) at clojure.core.reducers$filter$fn__587$fn__588.invoke(reducers.clj:161) at knossos.linear.config.ArrayProcesses$reify__3142.reduce(config.clj:231) at clojure.core.protocols$fn__8164.invokeStatic(protocols.clj:89) at clojure.core.protocols$fn__8164.invoke(protocols.clj:75) at clojure.core.protocols$fn__8110$G__8105__8123.invoke(protocols.clj:13) at clojure.core.reducers$folder$reify__554.coll_reduce(reducers.clj:93) at clojure.core.reducers$folder$reify__554.coll_reduce(reducers.clj:93) at clojure.core$reduce.invokeStatic(core.clj:6830) at clojure.core$reduce.invoke(core.clj:6812) at knossos.linear$jit_linearizations.invokeStatic(linear.clj:97) at knossos.linear$jit_linearizations.invoke(linear.clj:66) at knossos.linear$jit_linearizations$recurrence__3268.invoke(linear.clj:98) at knossos.util$rkeep$fn__670$fn__671.invoke(util.clj:42) at clojure.core.reducers$filter$fn__587$fn__588.invoke(reducers.clj:161) at knossos.linear.config.ArrayProcesses$reify__3142.reduce(config.clj:231) at clojure.core.protocols$fn__8164.invokeStatic(protocols.clj:89) at clojure.core.protocols$fn__8164.invoke(protocols.clj:75) at clojure.core.protocols$fn__8110$G__8105__8123.invoke(protocols.clj:13) at clojure.core.reducers$folder$reify__554.coll_reduce(reducers.clj:93) at clojure.core.reducers$folder$reify__554.coll_reduce(reducers.clj:93) at clojure.core$reduce.invokeStatic(core.clj:6830) WARN [2022-01-10 06:22:28,075] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:22:28,076] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:22:28,076] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:22:28,076] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:22:28,076] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:22:28,077] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:22:28,077] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:22:28,077] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:22:28,077] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:22:28,077] Service Thread - knossos.search Out of memory; aborting search INFO [2022-01-10 06:22:28,720] clojure-agent-send-pool-8 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/7/results.edn INFO [2022-01-10 06:22:28,734] clojure-agent-send-pool-5 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/13/results.edn INFO [2022-01-10 06:22:49,055] clojure-agent-send-pool-1 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/4/results.edn INFO [2022-01-10 06:22:49,058] clojure-agent-send-pool-2 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/15/results.edn INFO [2022-01-10 06:22:49,071] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/24/results.edn INFO [2022-01-10 06:22:49,081] clojure-agent-send-pool-6 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/21/results.edn INFO [2022-01-10 06:22:49,188] clojure-agent-send-pool-6 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/21/history.edn INFO [2022-01-10 06:22:49,197] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/24/history.edn INFO [2022-01-10 06:22:49,327] clojure-agent-send-pool-5 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/13/history.edn INFO [2022-01-10 06:22:49,335] clojure-agent-send-pool-8 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/7/history.edn INFO [2022-01-10 06:22:49,368] clojure-agent-send-pool-2 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/15/history.edn INFO [2022-01-10 06:22:49,474] clojure-agent-send-pool-1 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/4/history.edn INFO [2022-01-10 06:22:49,980] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/20/results.edn INFO [2022-01-10 06:22:50,183] clojure-agent-send-pool-4 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/0/results.edn INFO [2022-01-10 06:22:50,217] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/20/history.edn INFO [2022-01-10 06:22:50,422] clojure-agent-send-pool-4 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/0/history.edn INFO [2022-01-10 06:22:53,832] clojure-agent-send-pool-0 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/1/results.edn INFO [2022-01-10 06:22:53,863] clojure-agent-send-pool-9 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/27/results.edn INFO [2022-01-10 06:22:54,067] clojure-agent-send-pool-0 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/1/history.edn INFO [2022-01-10 06:22:54,157] clojure-agent-send-pool-9 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/27/history.edn INFO [2022-01-10 06:22:54,341] knossos reporter - knossos.search {:linear {:space 512, :cost 3.10E+18, :op {:process 426, :type :ok, :f :read, :value 41, :index 2659, :time 129180467661, :cas 1641821832787394560}}, :wgl [:WGL]} INFO [2022-01-10 06:22:54,803] knossos reporter - knossos.search {:linear {:space 4, :cost 1.34E+15, :op {:process 290, :type :ok, :f :read, :value 25, :index 2181, :time 112972728288, :cas 1641821773106577408}}, :wgl [:WGL]} INFO [2022-01-10 06:22:54,806] knossos reporter - knossos.search {:linear {:space 1024, :cost 2.53E+25, :op {:process 628, :type :ok, :f :read, :value 33, :index 2303, :time 116690848143, :cas 1641821804809355264}}, :wgl [:WGL]} INFO [2022-01-10 06:22:54,819] knossos reporter - knossos.search {:linear {:space 1, :cost 8.10E+10, :op {:process 345, :type :ok, :f :read, :value :nil, :index 1535, :time 89491110288}}, :wgl [:WGL]} INFO [2022-01-10 06:22:54,855] knossos reporter - knossos.search {:linear {:space 1, :cost 1.22E+12, :op {:process 445, :type :ok, :f :read, :value :nil, :index 1873, :time 100650526750}}, :wgl [:WGL]} INFO [2022-01-10 06:22:55,166] knossos reporter - knossos.search {:linear {:space 4, :cost 7.85E+13, :op {:process 413, :type :ok, :f :read, :value 37, :index 1981, :time 105603122625, :cas 1641821776248307712}}, :wgl [:WGL]} INFO [2022-01-10 06:22:55,360] knossos reporter - knossos.search {:linear {:space 4096, :cost 8.03E+16, :op {:process 281, :type :ok, :f :read, :value 2, :index 1703, :time 94059793879, :cas 1641821809848090624}}, :wgl [:WGL]} INFO [2022-01-10 06:22:55,911] knossos reporter - knossos.search {:linear {:space 1, :cost 3.35E+14, :op {:process 216, :type :ok, :f :read, :value :nil, :index 1955, :time 104537807243}}, :wgl [:WGL]} INFO [2022-01-10 06:23:00,066] knossos reporter - knossos.search {:linear {:space 1, :cost 1.22E+12, :op {:process 527, :type :ok, :f :read, :value :nil, :index 1845, :time 99923804586}}, :wgl [:WGL]} INFO [2022-01-10 06:23:00,072] knossos reporter - knossos.search {:linear {:space 1, :cost 3.35E+14, :op {:process 446, :type :ok, :f :read, :value :nil, :index 2081, :time 109917779306}}, :wgl [:WGL]} INFO [2022-01-10 06:23:00,072] knossos reporter - knossos.search {:linear {:space 8, :cost 9.76E+12, :op {:process 366, :type :ok, :f :read, :value 21, :index 1891, :time 101595744800, :cas 1641821774528118784}}, :wgl [:WGL]} INFO [2022-01-10 06:23:00,087] knossos reporter - knossos.search {:linear {:space 2, :cost 2.44E+12, :op {:process 429, :type :ok, :f :read, :value :nil, :index 1457, :time 85689465250}}, :wgl [:WGL]} INFO [2022-01-10 06:23:00,167] knossos reporter - knossos.search {:linear {:space 4, :cost 7.85E+13, :op {:process 413, :type :ok, :f :read, :value 41, :index 2051, :time 108333928135, :cas 1641821836906201088}}, :wgl [:WGL]} INFO [2022-01-10 06:23:00,365] knossos reporter - knossos.search {:linear {:space 2048, :cost 2.36E+20, :op {:process 549, :type :ok, :f :read, :value 2, :index 2029, :time 107465913858, :cas 1641821809848090624}}, :wgl [:WGL]} INFO [2022-01-10 06:23:04,419] knossos reporter - knossos.search {:linear {:space 512, :cost 5.90E+19, :op {:process 426, :type :ok, :f :read, :value 41, :index 2679, :time 130160878449, :cas 1641821832787394560}}, :wgl [:WGL]} INFO [2022-01-10 06:23:05,067] knossos reporter - knossos.search {:linear {:space 1, :cost 1.15E+17, :op {:process 617, :type :ok, :f :read, :value :nil, :index 2089, :time 109986497392}}, :wgl [:WGL]} INFO [2022-01-10 06:23:05,072] knossos reporter - knossos.search {:linear {:space 4, :cost 1.34E+15, :op {:process 367, :type :ok, :f :read, :value 21, :index 2071, :time 109451526245, :cas 1641821774528118784}}, :wgl [:WGL]} INFO [2022-01-10 06:23:05,092] knossos reporter - knossos.search {:linear {:space 1, :cost 1.96E+13, :op {:process 520, :type :ok, :f :read, :value :nil, :index 1693, :time 93838223037}}, :wgl [:WGL]} INFO [2022-01-10 06:23:10,093] knossos reporter - knossos.search {:linear {:space 1, :cost 1.15E+17, :op {:process 519, :type :ok, :f :read, :value :nil, :index 1953, :time 104532954333}}, :wgl [:WGL]} INFO [2022-01-10 06:23:10,111] knossos reporter - knossos.search {:linear {:space 4, :cost 1.34E+15, :op {:process 290, :type :ok, :f :read, :value 25, :index 2187, :time 113208564221, :cas 1641821773106577408}}, :wgl [:WGL]} INFO [2022-01-10 06:23:10,874] knossos reporter - knossos.search {:linear {:space 16384, :cost 5.48E+18, :op {:process 592, :type :invoke, :f :read, :value 41, :index 2178, :time 112781109011}}, :wgl [:WGL]} INFO [2022-01-10 06:23:15,788] knossos reporter - knossos.search {:linear {:space 2048, :cost 2.20E+24, :op {:process 550, :type :ok, :f :read, :value 2, :index 2431, :time 121568242475, :cas 1641821809848090624}}, :wgl [:WGL]} INFO [2022-01-10 06:23:16,023] knossos reporter - knossos.search {:linear {:space 16384, :cost 9.91E+19, :op {:process 413, :type :ok, :f :read, :value 41, :index 2199, :time 113814093042, :cas 1641821836906201088}}, :wgl [:WGL]} INFO [2022-01-10 06:23:20,580] knossos reporter - knossos.search {:linear {:space 1, :cost 1.15E+17, :op {:process 624, :type :ok, :f :read, :value :nil, :index 2259, :time 115865560791}}, :wgl [:WGL]} INFO [2022-01-10 06:23:21,517] knossos reporter - knossos.search {:linear {:space 2, :cost 6.70E+14, :op {:process 577, :type :ok, :f :read, :value :nil, :index 1979, :time 105589975344}}, :wgl [:WGL]} INFO [2022-01-10 06:23:21,794] knossos reporter - knossos.search {:linear {:space 32768, :cost 1.98E+20, :op {:process 411, :type :ok, :f :read, :value 41, :index 2215, :time 114438578093, :cas 1641821836906201088}}, :wgl [:WGL]} INFO [2022-01-10 06:23:25,931] knossos reporter - knossos.search {:linear {:space 4, :cost 2.42E+16, :op {:process 456, :type :ok, :f :read, :value 21, :index 2133, :time 111784751938, :cas 1641821774528118784}}, :wgl [:WGL]} INFO [2022-01-10 06:23:36,519] knossos reporter - knossos.search {:linear {:space 2, :cost 6.70E+14, :op {:process 216, :type :ok, :f :read, :value :nil, :index 2049, :time 108319397184}}, :wgl [:WGL]} INFO [2022-01-10 06:23:43,578] knossos reporter - knossos.search {:linear {:space 16384, :cost 9.91E+19, :op {:process 592, :type :ok, :f :read, :value 41, :index 2257, :time 115843586231, :cas 1641821836906201088}}, :wgl [:WGL]} INFO [2022-01-10 06:23:44,726] knossos reporter - knossos.search {:linear {:space 512, :cost 1.18E+21, :op {:process 426, :type :ok, :f :read, :value 41, :index 2713, :time 131327927590, :cas 1641821832787394560}}, :wgl [:WGL]} INFO [2022-01-10 06:23:46,157] knossos reporter - knossos.search {:linear {:space 4, :cost 4.61E+17, :op {:process 558, :type :ok, :f :read, :value 25, :index 2425, :time 121116783631, :cas 1641821773106577408}}, :wgl [:WGL]} INFO [2022-01-10 06:23:48,735] knossos reporter - knossos.search {:linear {:space 16384, :cost 9.91E+19, :op {:process 411, :type :ok, :f :read, :value 41, :index 2285, :time 116173336748, :cas 1641821836906201088}}, :wgl [:WGL]} INFO [2022-01-10 06:23:56,628] knossos reporter - knossos.search {:linear {:space 1, :cost 6.05E+15, :op {:process 216, :type :ok, :f :read, :value :nil, :index 2143, :time 111928473078}}, :wgl [:WGL]} INFO [2022-01-10 06:23:59,248] knossos reporter - knossos.search {:linear {:space 8192, :cost 3.99E+23, :op {:process 501, :type :ok, :f :read, :value 41, :index 2623, :time 127768586799, :cas 1641821836906201088}}, :wgl [:WGL]} INFO [2022-01-10 06:24:16,241] knossos reporter - knossos.search {:linear {:space 4, :cost 2.42E+16, :op {:process 456, :type :ok, :f :read, :value 21, :index 2147, :time 111945156466, :cas 1641821774528118784}}, :wgl [:WGL]} INFO [2022-01-10 06:24:32,142] knossos reporter - knossos.search {:linear {:space 2, :cost 1.21E+16, :op {:process 216, :type :ok, :f :read, :value :nil, :index 2197, :time 113799691801}}, :wgl [:WGL]} INFO [2022-01-10 06:24:35,364] knossos reporter - knossos.search {:linear {:space 512, :cost 1.18E+21, :op {:process 426, :type :ok, :f :read, :value 41, :index 2715, :time 131370180726, :cas 1641821832787394560}}, :wgl [:WGL]} INFO [2022-01-10 06:24:47,143] knossos reporter - knossos.search {:linear {:space 4, :cost 2.42E+16, :op {:process 456, :type :ok, :f :read, :value 21, :index 2151, :time 111985462640, :cas 1641821774528118784}}, :wgl [:WGL]} INFO [2022-01-10 06:25:02,156] knossos reporter - knossos.search {:linear {:space 2, :cost 4.62E+18, :op {:process 526, :type :ok, :f :read, :value :nil, :index 2267, :time 115917943663}}, :wgl [:WGL]} INFO [2022-01-10 06:25:18,227] knossos reporter - knossos.search {:linear {:space 1, :cost 1.07E+21, :op {:process 521, :type :ok, :f :read, :value :nil, :index 2241, :time 115465052417}}, :wgl [:WGL]} INFO [2022-01-10 06:25:19,551] knossos reporter - knossos.search {:linear {:space 2, :cost 2.30E+17, :op {:process 446, :type :ok, :f :read, :value :nil, :index 2307, :time 116765776901}}, :wgl [:WGL]} INFO [2022-01-10 06:25:19,520] knossos reporter - knossos.search {:linear {:space 2, :cost 2.30E+17, :op {:process 578, :type :ok, :f :read, :value :nil, :index 2295, :time 116459562107}}, :wgl [:WGL]} INFO [2022-01-10 06:25:29,585] knossos reporter - knossos.search {:linear {:space 4, :cost 4.61E+17, :op {:process 548, :type :ok, :f :read, :value 21, :index 2227, :time 115228765306, :cas 1641821774528118784}}, :wgl [:WGL]} INFO [2022-01-10 06:25:31,435] knossos reporter - knossos.search {:linear {:space 512, :cost 1.99E+29, :op {:process 787, :type :ok, :f :read, :value 41, :index 3475, :time 166513958950, :cas 1641821832787394560}}, :wgl [:WGL]} INFO [2022-01-10 06:25:33,229] knossos reporter - knossos.search {:linear {:space 4, :cost 2.38E+24, :op {:process 648, :type :ok, :f :read, :value 25, :index 2883, :time 138849819466, :cas 1641821773106577408}}, :wgl [:WGL]} INFO [2022-01-10 06:25:52,710] knossos reporter - knossos.search {:linear {:space 32768, :cost 1.59E+24, :op {:process 503, :type :ok, :f :read, :value 41, :index 2639, :time 128384265014, :cas 1641821836906201088}}, :wgl [:WGL]} INFO [2022-01-10 06:37:10,215] knossos reporter - knossos.search {:linear {:space 2, :cost 2.30E+17, :op {:process 578, :type :ok, :f :read, :value :nil, :index 2324, :time 117459744980}}, :wgl [:WGL]} INFO [2022-01-10 06:38:38,982] knossos reporter - knossos.search {:linear {:space 1, :cost 2.31E+18, :op {:process 625, :type :ok, :f :read, :value :nil, :index 2527, :time 125610531893}}, :wgl [:WGL]} WARN [2022-01-10 06:39:52,827] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:39:52,827] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:39:52,827] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:39:52,827] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:39:52,827] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:39:52,827] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:39:52,827] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:39:52,828] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:39:52,828] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:39:52,828] Service Thread - knossos.search Out of memory; aborting search INFO [2022-01-10 06:39:52,891] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/29/results.edn INFO [2022-01-10 06:39:52,891] clojure-agent-send-pool-0 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/2/results.edn INFO [2022-01-10 06:39:52,895] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/3/results.edn INFO [2022-01-10 06:39:52,900] clojure-agent-send-pool-5 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/6/results.edn INFO [2022-01-10 06:39:52,911] clojure-agent-send-pool-6 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/22/results.edn INFO [2022-01-10 06:39:52,964] clojure-agent-send-pool-5 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/6/history.edn INFO [2022-01-10 06:39:53,058] clojure-agent-send-pool-6 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/22/history.edn INFO [2022-01-10 06:39:53,060] clojure-agent-send-pool-0 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/2/history.edn INFO [2022-01-10 06:39:53,129] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/3/history.edn INFO [2022-01-10 06:39:53,147] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/29/history.edn INFO [2022-01-10 06:39:53,544] clojure-agent-send-pool-9 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/23/results.edn INFO [2022-01-10 06:39:53,573] clojure-agent-send-pool-8 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/28/results.edn INFO [2022-01-10 06:39:53,708] clojure-agent-send-pool-8 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/28/history.edn INFO [2022-01-10 06:39:53,736] clojure-agent-send-pool-9 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/23/history.edn INFO [2022-01-10 06:39:54,126] clojure-agent-send-pool-4 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/12/results.edn INFO [2022-01-10 06:39:54,130] clojure-agent-send-pool-2 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/25/results.edn INFO [2022-01-10 06:39:54,335] clojure-agent-send-pool-4 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/12/history.edn INFO [2022-01-10 06:39:54,366] clojure-agent-send-pool-2 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/25/history.edn INFO [2022-01-10 06:41:01,999] knossos reporter - knossos.search {:linear {:space 8, :cost 6.48E+11, :op {:process 391, :type :ok, :f :read, :value 36, :index 1279, :time 79257989632, :cas 1641821773851262976}}, :wgl [:WGL]} INFO [2022-01-10 06:41:02,139] knossos reporter - knossos.search {:linear {:space 8, :cost 9.76E+12, :op {:process 312, :type :ok, :f :read, :value 14, :index 1721, :time 95150314395, :cas 1641821779375292416}}, :wgl [:WGL]} INFO [2022-01-10 06:41:02,139] knossos reporter - knossos.search {:linear {:space 4, :cost 1.34E+15, :op {:process 418, :type :ok, :f :read, :value 31, :index 1827, :time 99039537728, :cas 1641821829376573440}}, :wgl [:WGL]} INFO [2022-01-10 06:41:02,139] knossos reporter - knossos.search {:linear {:space 8, :cost 1.57E+14, :op {:process 439, :type :ok, :f :read, :value 10, :index 1865, :time 100603958709, :cas 1641821776055042048}}, :wgl [:WGL]} INFO [2022-01-10 06:41:02,139] knossos reporter - knossos.search {:linear {:space 4, :cost 7.85E+13, :op {:process 485, :type :ok, :f :read, :value 21, :index 1753, :time 96434655023, :cas 1641821775052931072}}, :wgl [:WGL]} INFO [2022-01-10 06:41:02,142] knossos reporter - knossos.search {:linear {:space 1280, :cost 4.28E+17, :op {:process 375, :type :ok, :f :read, :value 42, :index 2163, :time 112433905532, :cas 1641821818897498112}}, :wgl [:WGL]} INFO [2022-01-10 06:41:02,142] knossos reporter - knossos.search {:linear {:space 2, :cost 2.44E+12, :op {:process 409, :type :ok, :f :read, :value :nil, :index 2003, :time 106363606900}}, :wgl [:WGL]} INFO [2022-01-10 06:41:02,149] knossos reporter - knossos.search {:linear {:space 2052, :cost 1.66E+14, :op {:process 415, :type :ok, :f :read, :value 14, :index 1875, :time 100851742460, :cas 1641821774075199488}}, :wgl [:WGL]} INFO [2022-01-10 06:41:02,168] knossos reporter - knossos.search {:linear {:space 2052, :cost 4.74E+21, :op {:process 479, :type :ok, :f :read, :value 22, :index 2333, :time 117527542538, :cas 1641821778870796288}}, :wgl [:WGL]} INFO [2022-01-10 06:41:02,369] clojure-agent-send-pool-1 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/17/results.edn INFO [2022-01-10 06:41:02,405] clojure-agent-send-pool-1 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/17/history.edn INFO [2022-01-10 06:41:07,000] knossos reporter - knossos.search {:linear {:space 8, :cost 1.57E+14, :op {:process 392, :type :ok, :f :read, :value 36, :index 1531, :time 89305534114, :cas 1641821773851262976}}, :wgl [:WGL]} INFO [2022-01-10 06:41:07,140] knossos reporter - knossos.search {:linear {:space 4, :cost 2.42E+16, :op {:process 402, :type :ok, :f :read, :value 14, :index 2011, :time 106581093663, :cas 1641821779375292416}}, :wgl [:WGL]} INFO [2022-01-10 06:41:07,140] knossos reporter - knossos.search {:linear {:space 8, :cost 1.57E+14, :op {:process 303, :type :ok, :f :read, :value 21, :index 1789, :time 97688418782, :cas 1641821775052931072}}, :wgl [:WGL]} INFO [2022-01-10 06:41:07,147] knossos reporter - knossos.search {:linear {:space 1, :cost 1.96E+13, :op {:process 318, :type :ok, :f :read, :value :nil, :index 2015, :time 106614352799}}, :wgl [:WGL]} INFO [2022-01-10 06:41:07,158] knossos reporter - knossos.search {:linear {:space 4098, :cost 8.04E+16, :op {:process 324, :type :ok, :f :read, :value 14, :index 2061, :time 109049939613, :cas 1641821774075199488}}, :wgl [:WGL]} INFO [2022-01-10 06:41:07,427] knossos reporter - knossos.search {:linear {:space 16896, :cost 5.66E+18, :op {:process 375, :type :ok, :f :read, :value 42, :index 2179, :time 112856433943, :cas 1641821818897498112}}, :wgl [:WGL]} INFO [2022-01-10 06:41:07,767] knossos reporter - knossos.search {:linear {:space 1, :cost 1.22E+12, :op {:process 384, :type :ok, :f :read, :value :nil, :index 1819, :time 98668986141}}, :wgl [:WGL]} INFO [2022-01-10 06:41:12,141] knossos reporter - knossos.search {:linear {:space 4, :cost 4.61E+17, :op {:process 530, :type :ok, :f :read, :value 10, :index 2131, :time 111719495833, :cas 1641821776055042048}}, :wgl [:WGL]} INFO [2022-01-10 06:41:12,148] knossos reporter - knossos.search {:linear {:space 2, :cost 3.92E+13, :op {:process 318, :type :ok, :f :read, :value :nil, :index 2017, :time 106619238476}}, :wgl [:WGL]} INFO [2022-01-10 06:41:12,204] knossos reporter - knossos.search {:linear {:space 8194, :cost 3.99E+23, :op {:process 595, :type :ok, :f :read, :value 14, :index 2625, :time 127783485021, :cas 1641821774075199488}}, :wgl [:WGL]} INFO [2022-01-10 06:41:12,899] knossos reporter - knossos.search {:linear {:space 1, :cost 1.22E+12, :op {:process 384, :type :ok, :f :read, :value :nil, :index 1843, :time 99870615384}}, :wgl [:WGL]} INFO [2022-01-10 06:41:17,001] knossos reporter - knossos.search {:linear {:space 8, :cost 1.57E+14, :op {:process 300, :type :ok, :f :read, :value 36, :index 1545, :time 89749157916, :cas 1641821773851262976}}, :wgl [:WGL]} INFO [2022-01-10 06:41:17,142] knossos reporter - knossos.search {:linear {:space 4, :cost 9.25E+18, :op {:process 393, :type :ok, :f :read, :value 21, :index 2103, :time 110623843750, :cas 1641821775052931072}}, :wgl [:WGL]} INFO [2022-01-10 06:41:17,148] knossos reporter - knossos.search {:linear {:space 2, :cost 3.92E+13, :op {:process 318, :type :ok, :f :read, :value :nil, :index 2019, :time 106621603335}}, :wgl [:WGL]} INFO [2022-01-10 06:41:17,903] knossos reporter - knossos.search {:linear {:space 1, :cost 6.05E+15, :op {:process 474, :type :ok, :f :read, :value :nil, :index 2107, :time 110669465912}}, :wgl [:WGL]} INFO [2022-01-10 06:41:22,002] knossos reporter - knossos.search {:linear {:space 4, :cost 7.85E+13, :op {:process 300, :type :ok, :f :read, :value 36, :index 1567, :time 90183182780, :cas 1641821773851262976}}, :wgl [:WGL]} INFO [2022-01-10 06:41:23,481] knossos reporter - knossos.search {:linear {:space 16896, :cost 1.02E+20, :op {:process 377, :type :ok, :f :read, :value 42, :index 2311, :time 116811383351, :cas 1641821818897498112}}, :wgl [:WGL]} INFO [2022-01-10 06:41:27,150] knossos reporter - knossos.search {:linear {:space 2, :cost 6.70E+14, :op {:process 409, :type :ok, :f :read, :value :nil, :index 2249, :time 115710083663}}, :wgl [:WGL]} INFO [2022-01-10 06:41:27,345] knossos reporter - knossos.search {:linear {:space 16384, :cost 5.48E+18, :op {:process 418, :type :ok, :f :read, :value 31, :index 1829, :time 99078127432, :cas 1641821829376573440}}, :wgl [:WGL]} INFO [2022-01-10 06:41:32,003] knossos reporter - knossos.search {:linear {:space 4, :cost 7.85E+13, :op {:persist-to 0, :mutation-token Optional[mt{vbID=148, vbUUID=148830943203494, seqno=15, bucket=default}], :index 1817, :value 37, :time 98664258611, :process 300, :replicate-to 0, :type :ok, :cas 1641821828653973504, :f :write, :durability-level 3}}, :wgl [:WGL]} INFO [2022-01-10 06:41:32,456] knossos reporter - knossos.search {:linear {:space 16384, :cost 7.97E+23, :op {:process 689, :type :ok, :f :read, :value 31, :index 2219, :time 114775894550, :cas 1641821829376573440}}, :wgl [:WGL]} INFO [2022-01-10 06:42:42,145] knossos reporter - knossos.search {:linear {:space 16384, :cost 9.91E+19, :op {:process 300, :type :ok, :f :read, :value 37, :index 1897, :time 101769572953, :cas 1641821828653973504}}, :wgl [:WGL]} INFO [2022-01-10 06:42:46,456] knossos reporter - knossos.search {:linear {:space 1, :cost 3.35E+14, :op {:process 409, :type :ok, :f :read, :value :nil, :index 2261, :time 115879680051}}, :wgl [:WGL]} INFO [2022-01-10 06:43:03,409] knossos reporter - knossos.search {:linear {:space 49664, :cost 1.66E+19, :op {:persist-to 0, :index 2312, :value 46, :time 116811418683, :process 377, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}}, :wgl [:WGL]} INFO [2022-01-10 06:43:06,808] knossos reporter - knossos.search {:linear {:space 8, :cost 4.84E+16, :op {:process 494, :type :ok, :f :read, :value 14, :index 2013, :time 106598144765, :cas 1641821779375292416}}, :wgl [:WGL]} INFO [2022-01-10 06:43:08,827] knossos reporter - knossos.search {:linear {:space 32768, :cost 1.98E+20, :op {:process 482, :type :ok, :f :read, :value 37, :index 2021, :time 106769202968, :cas 1641821828653973504}}, :wgl [:WGL]} INFO [2022-01-10 06:43:09,062] knossos reporter - knossos.search {:linear {:space 49664, :cost 5.72E+21, :op {:process 556, :type :ok, :f :read, :value 42, :index 2531, :time 125652502733, :cas 1641821818897498112}}, :wgl [:WGL]} INFO [2022-01-10 06:44:18,334] knossos reporter - knossos.search {:linear {:space 1, :cost 3.35E+14, :op {:process 409, :type :ok, :f :read, :value :nil, :index 2289, :time 116257253183}}, :wgl [:WGL]} INFO [2022-01-10 06:44:28,579] knossos reporter - knossos.search {:linear {:space 1, :cost 2.31E+18, :op {:process 475, :type :ok, :f :read, :value :nil, :index 2265, :time 115891411700}}, :wgl [:WGL]} INFO [2022-01-10 06:44:33,565] knossos reporter - knossos.search {:linear {:space 1, :cost 6.05E+15, :op {:process 409, :type :ok, :f :read, :value :nil, :index 2309, :time 116801690661}}, :wgl [:WGL]} WARN [2022-01-10 06:45:46,087] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:45:46,087] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:45:46,087] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:45:46,087] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:45:46,088] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:45:46,088] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:45:46,088] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:45:46,088] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:45:46,088] Service Thread - knossos.search Out of memory; aborting search WARN [2022-01-10 06:45:46,088] Service Thread - knossos.search Out of memory; aborting search INFO [2022-01-10 06:45:46,168] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/5/results.edn INFO [2022-01-10 06:45:46,169] clojure-agent-send-pool-0 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/9/results.edn INFO [2022-01-10 06:45:46,170] clojure-agent-send-pool-9 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/16/results.edn INFO [2022-01-10 06:45:46,170] clojure-agent-send-pool-2 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/18/results.edn INFO [2022-01-10 06:45:46,171] clojure-agent-send-pool-8 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/26/results.edn INFO [2022-01-10 06:45:46,173] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/14/results.edn INFO [2022-01-10 06:45:46,175] clojure-agent-send-pool-5 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/19/results.edn INFO [2022-01-10 06:45:46,177] clojure-agent-send-pool-4 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/10/results.edn INFO [2022-01-10 06:45:46,244] clojure-agent-send-pool-0 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/9/history.edn INFO [2022-01-10 06:45:46,315] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/5/history.edn INFO [2022-01-10 06:45:46,320] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/14/history.edn INFO [2022-01-10 06:45:46,366] clojure-agent-send-pool-2 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/18/history.edn INFO [2022-01-10 06:45:46,389] clojure-agent-send-pool-4 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/10/history.edn INFO [2022-01-10 06:45:46,389] clojure-agent-send-pool-5 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/19/history.edn INFO [2022-01-10 06:45:46,404] clojure-agent-send-pool-9 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/16/history.edn INFO [2022-01-10 06:45:46,449] clojure-agent-send-pool-8 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/26/history.edn INFO [2022-01-10 06:45:46,512] clojure-agent-send-pool-1 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/8/results.edn INFO [2022-01-10 06:45:46,667] clojure-agent-send-pool-1 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/8/history.edn INFO [2022-01-10 06:45:47,127] clojure-agent-send-pool-6 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/11/results.edn INFO [2022-01-10 06:45:47,188] clojure-agent-send-pool-6 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/independent/11/history.edn INFO [2022-01-10 06:45:47,189] jepsen test runner - jepsen.core Analysis complete INFO [2022-01-10 06:45:47,191] jepsen fressian - couchbase.core Not writing fressian INFO [2022-01-10 07:17:27,520] jepsen results - jepsen.store Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T053441.000-0800/results.edn INFO [2022-01-10 07:17:28,566] jepsen test runner - jepsen.core {:indep {:valid? :unknown, :results {0 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :configs ({:model #knossos.model.CASRegister{:value :nil}, :last-op {:process 451, :type :ok, :f :read, :value :nil, :index 2001, :time 106344079384}, :pending [{:persist-to 0, :index 183, :value 20, :time 12553176197, :process 0, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 177, :value 31, :time 12509873349, :process 1, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 113, :value 10, :time 10686973406, :process 2, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 612, :value 34, :time 51656043820, :process 90, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 678, :value 48, :time 52162679004, :process 91, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 931, :value 18, :time 66466908462, :process 92, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 864, :value 14, :time 61680159767, :process 180, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 896, :value 21, :time 62195661301, :process 181, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1374, :value 22, :time 82361065448, :process 182, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1612, :value 35, :time 91517484363, :process 270, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1270, :value 49, :time 78635842099, :process 271, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1806, :value 45, :time 97965774558, :process 272, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1886, :value 31, :time 101544143792, :process 360, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1520, :value 1, :time 88659216485, :process 361, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2048, :value 35, :time 107990335500, :process 362, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2124, :value 8, :time 111569303449, :process 450, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2002, :value 43, :time 106344118891, :process 451, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2340, :value 22, :time 118011019780, :process 452, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 540, :type :invoke, :f :read, :value :nil, :index 2436, :time 121606339711} {:persist-to 0, :index 2294, :value 2, :time 116370642822, :process 541, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 542, :type :invoke, :f :read, :value :nil, :index 2630, :time 128036637975} {:persist-to 0, :index 2572, :value 3, :time 126399622129, :process 631, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]}), :previous-ok {:process 451, :type :ok, :f :read, :value :nil, :index 2001, :time 106344079384}, :last-op nil, :op {:process 540, :type :ok, :f :read, :value :nil, :index 2633, :time 128116015545}, :final-paths ()}, :valid? :unknown}, 7 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 20 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :configs ({:model #knossos.model.CASRegister{:value :nil}, :last-op {:process 510, :type :ok, :f :read, :value :nil, :index 1755, :time 96437911213}, :pending [{:persist-to 0, :index 226, :value 2, :time 13055360132, :process 60, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 218, :value 18, :time 13010784405, :process 61, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 45, :value 46, :time 8365687646, :process 62, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 600, :value 10, :time 51560958723, :process 150, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 664, :value 39, :time 52064830912, :process 151, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 463, :value 29, :time 46638073401, :process 152, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 858, :value 30, :time 61590258453, :process 240, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 888, :value 33, :time 62086296355, :process 241, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 806, :value 33, :time 59948540211, :process 242, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1136, :value 42, :time 72629129617, :process 330, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1164, :value 6, :time 73813834592, :process 331, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1024, :value 18, :time 70371242503, :process 332, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1382, :value 17, :time 82650853549, :process 420, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1462, :value 39, :time 85761164731, :process 421, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1292, :value 32, :time 80394164546, :process 422, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 510, :type :invoke, :f :read, :value :nil, :index 1756, :time 96437933340} {:persist-to 0, :index 1748, :value 32, :time 95790320665, :process 511, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1608, :value 23, :time 91354834476, :process 512, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]}), :previous-ok {:process 510, :type :ok, :f :read, :value :nil, :index 1755, :time 96437911213}, :last-op nil, :op {:process 510, :type :ok, :f :read, :value :nil, :index 1793, :time 97714323441}, :final-paths ()}, :valid? :unknown}, 27 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :configs ({:model #knossos.model.CASRegister{:value 30}, :last-op {:process 531, :type :ok, :f :read, :value 30, :index 2165, :time 112562713395, :cas 1641821774473723904}, :pending [{:persist-to 0, :index 429, :value 19, :time 40581763360, :process 83, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 512, :value 49, :time 49426588102, :process 171, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1590, :value 32, :time 90951653555, :process 172, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 550, :value 37, :time 50712793758, :process 173, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 790, :value 0, :time 59460084093, :process 261, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1880, :value 18, :time 100976162718, :process 262, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1204, :value 31, :time 75609149142, :process 263, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1008, :value 1, :time 69495872020, :process 351, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2120, :value 0, :time 111006045886, :process 352, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1436, :value 35, :time 84733622536, :process 353, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1744, :value 27, :time 95713220439, :process 441, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 442, :type :invoke, :f :read, :value 30, :index 2420, :time 121030674857} {:persist-to 0, :index 1714, :value 28, :time 94765361756, :process 443, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2166, :value 41, :time 112562761020, :process 531, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1958, :value 45, :time 104786178994, :process 533, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2456, :value 9, :time 122594086743, :process 621, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2222, :value 12, :time 114810805048, :process 623, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2486, :value 48, :time 124840543517, :process 713, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 30}, :last-op {:process 531, :type :ok, :f :read, :value 30, :index 2165, :time 112562713395, :cas 1641821774473723904}, :pending [{:persist-to 0, :index 131, :value 2, :time 11181243019, :process 82, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 429, :value 19, :time 40581763360, :process 83, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 512, :value 49, :time 49426588102, :process 171, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1590, :value 32, :time 90951653555, :process 172, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 550, :value 37, :time 50712793758, :process 173, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 790, :value 0, :time 59460084093, :process 261, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1880, :value 18, :time 100976162718, :process 262, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1204, :value 31, :time 75609149142, :process 263, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1008, :value 1, :time 69495872020, :process 351, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2120, :value 0, :time 111006045886, :process 352, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1436, :value 35, :time 84733622536, :process 353, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1744, :value 27, :time 95713220439, :process 441, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 442, :type :invoke, :f :read, :value 30, :index 2420, :time 121030674857} {:persist-to 0, :index 1714, :value 28, :time 94765361756, :process 443, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2166, :value 41, :time 112562761020, :process 531, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1958, :value 45, :time 104786178994, :process 533, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2456, :value 9, :time 122594086743, :process 621, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2222, :value 12, :time 114810805048, :process 623, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2486, :value 48, :time 124840543517, :process 713, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 30}, :last-op {:process 531, :type :ok, :f :read, :value 30, :index 2165, :time 112562713395, :cas 1641821774473723904}, :pending [{:persist-to 0, :index 107, :value 1, :time 10471085342, :process 81, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 429, :value 19, :time 40581763360, :process 83, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 512, :value 49, :time 49426588102, :process 171, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1590, :value 32, :time 90951653555, :process 172, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 550, :value 37, :time 50712793758, :process 173, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 790, :value 0, :time 59460084093, :process 261, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1880, :value 18, :time 100976162718, :process 262, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1204, :value 31, :time 75609149142, :process 263, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1008, :value 1, :time 69495872020, :process 351, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2120, :value 0, :time 111006045886, :process 352, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1436, :value 35, :time 84733622536, :process 353, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1744, :value 27, :time 95713220439, :process 441, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 442, :type :invoke, :f :read, :value 30, :index 2420, :time 121030674857} {:persist-to 0, :index 1714, :value 28, :time 94765361756, :process 443, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2166, :value 41, :time 112562761020, :process 531, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1958, :value 45, :time 104786178994, :process 533, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2456, :value 9, :time 122594086743, :process 621, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2222, :value 12, :time 114810805048, :process 623, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2486, :value 48, :time 124840543517, :process 713, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 30}, :last-op {:process 531, :type :ok, :f :read, :value 30, :index 2165, :time 112562713395, :cas 1641821774473723904}, :pending [{:persist-to 0, :index 107, :value 1, :time 10471085342, :process 81, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 131, :value 2, :time 11181243019, :process 82, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 429, :value 19, :time 40581763360, :process 83, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 512, :value 49, :time 49426588102, :process 171, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1590, :value 32, :time 90951653555, :process 172, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 550, :value 37, :time 50712793758, :process 173, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 790, :value 0, :time 59460084093, :process 261, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1880, :value 18, :time 100976162718, :process 262, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1204, :value 31, :time 75609149142, :process 263, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1008, :value 1, :time 69495872020, :process 351, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2120, :value 0, :time 111006045886, :process 352, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1436, :value 35, :time 84733622536, :process 353, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1744, :value 27, :time 95713220439, :process 441, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 442, :type :invoke, :f :read, :value 30, :index 2420, :time 121030674857} {:persist-to 0, :index 1714, :value 28, :time 94765361756, :process 443, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2166, :value 41, :time 112562761020, :process 531, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1958, :value 45, :time 104786178994, :process 533, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2456, :value 9, :time 122594086743, :process 621, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2222, :value 12, :time 114810805048, :process 623, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2486, :value 48, :time 124840543517, :process 713, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]}), :previous-ok {:process 531, :type :ok, :f :read, :value 30, :index 2165, :time 112562713395, :cas 1641821774473723904}, :last-op nil, :op {:process 442, :type :ok, :f :read, :value 30, :index 2549, :time 125960150793, :cas 1641821774473723904}, :final-paths ()}, :valid? :unknown}, 1 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :configs ({:model #knossos.model.CASRegister{:value 41}, :last-op {:process 453, :type :ok, :f :read, :value 41, :index 1695, :time 93862068646, :cas 1641821774618951680}, :pending [{:persist-to 0, :index 114, :value 40, :time 10703303379, :process 3, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 436, :value 10, :time 40850734667, :process 4, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 568, :value 4, :time 51042469588, :process 93, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 580, :value 27, :time 51227700033, :process 94, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 465, :value 17, :time 47160494132, :process 95, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 850, :value 13, :time 61291865296, :process 183, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1104, :value 0, :time 71660775755, :process 184, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 708, :value 7, :time 57180795992, :process 185, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1064, :value 6, :time 71316143432, :process 273, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1584, :value 2, :time 90878127328, :process 274, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1324, :value 20, :time 81468761743, :process 275, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1320, :value 39, :time 81346311752, :process 363, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1878, :value 23, :time 100904142073, :process 364, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1662, :value 18, :time 92550029078, :process 365, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1696, :value 0, :time 93862112068, :process 453, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 454, :type :invoke, :f :read, :value 41, :index 2118, :time 110924868787} {:persist-to 0, :index 1922, :value 26, :time 102570406137, :process 455, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1942, :value 31, :time 103886184555, :process 543, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 545, :type :invoke, :f :read, :value 41, :index 2172, :time 112593806174} {:process 633, :type :invoke, :f :read, :value 41, :index 2204, :time 113911269850}]} {:model #knossos.model.CASRegister{:value 41}, :last-op {:process 453, :type :ok, :f :read, :value 41, :index 1695, :time 93862068646, :cas 1641821774618951680}, :pending [{:persist-to 0, :index 436, :value 10, :time 40850734667, :process 4, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 81, :value 3, :time 9825985223, :process 5, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 568, :value 4, :time 51042469588, :process 93, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 580, :value 27, :time 51227700033, :process 94, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 465, :value 17, :time 47160494132, :process 95, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 850, :value 13, :time 61291865296, :process 183, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1104, :value 0, :time 71660775755, :process 184, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 708, :value 7, :time 57180795992, :process 185, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1064, :value 6, :time 71316143432, :process 273, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1584, :value 2, :time 90878127328, :process 274, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1324, :value 20, :time 81468761743, :process 275, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1320, :value 39, :time 81346311752, :process 363, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1878, :value 23, :time 100904142073, :process 364, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1662, :value 18, :time 92550029078, :process 365, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1696, :value 0, :time 93862112068, :process 453, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 454, :type :invoke, :f :read, :value 41, :index 2118, :time 110924868787} {:persist-to 0, :index 1922, :value 26, :time 102570406137, :process 455, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1942, :value 31, :time 103886184555, :process 543, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 545, :type :invoke, :f :read, :value 41, :index 2172, :time 112593806174} {:process 633, :type :invoke, :f :read, :value 41, :index 2204, :time 113911269850}]} {:model #knossos.model.CASRegister{:value 41}, :last-op {:process 453, :type :ok, :f :read, :value 41, :index 1695, :time 93862068646, :cas 1641821774618951680}, :pending [{:persist-to 0, :index 114, :value 40, :time 10703303379, :process 3, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 436, :value 10, :time 40850734667, :process 4, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 81, :value 3, :time 9825985223, :process 5, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 568, :value 4, :time 51042469588, :process 93, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 580, :value 27, :time 51227700033, :process 94, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 465, :value 17, :time 47160494132, :process 95, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 850, :value 13, :time 61291865296, :process 183, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1104, :value 0, :time 71660775755, :process 184, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 708, :value 7, :time 57180795992, :process 185, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1064, :value 6, :time 71316143432, :process 273, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1584, :value 2, :time 90878127328, :process 274, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1324, :value 20, :time 81468761743, :process 275, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1320, :value 39, :time 81346311752, :process 363, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1878, :value 23, :time 100904142073, :process 364, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1662, :value 18, :time 92550029078, :process 365, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1696, :value 0, :time 93862112068, :process 453, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 454, :type :invoke, :f :read, :value 41, :index 2118, :time 110924868787} {:persist-to 0, :index 1922, :value 26, :time 102570406137, :process 455, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1942, :value 31, :time 103886184555, :process 543, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 545, :type :invoke, :f :read, :value 41, :index 2172, :time 112593806174} {:process 633, :type :invoke, :f :read, :value 41, :index 2204, :time 113911269850}]} {:model #knossos.model.CASRegister{:value 41}, :last-op {:process 453, :type :ok, :f :read, :value 41, :index 1695, :time 93862068646, :cas 1641821774618951680}, :pending [{:persist-to 0, :index 436, :value 10, :time 40850734667, :process 4, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 568, :value 4, :time 51042469588, :process 93, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 580, :value 27, :time 51227700033, :process 94, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 465, :value 17, :time 47160494132, :process 95, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 850, :value 13, :time 61291865296, :process 183, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1104, :value 0, :time 71660775755, :process 184, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 708, :value 7, :time 57180795992, :process 185, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1064, :value 6, :time 71316143432, :process 273, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1584, :value 2, :time 90878127328, :process 274, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1324, :value 20, :time 81468761743, :process 275, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1320, :value 39, :time 81346311752, :process 363, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1878, :value 23, :time 100904142073, :process 364, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1662, :value 18, :time 92550029078, :process 365, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1696, :value 0, :time 93862112068, :process 453, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 454, :type :invoke, :f :read, :value 41, :index 2118, :time 110924868787} {:persist-to 0, :index 1922, :value 26, :time 102570406137, :process 455, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1942, :value 31, :time 103886184555, :process 543, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 545, :type :invoke, :f :read, :value 41, :index 2172, :time 112593806174} {:process 633, :type :invoke, :f :read, :value 41, :index 2204, :time 113911269850}]}), :previous-ok {:process 453, :type :ok, :f :read, :value 41, :index 1695, :time 93862068646, :cas 1641821774618951680}, :last-op nil, :op {:process 545, :type :ok, :f :read, :value 41, :index 2239, :time 115416604033, :cas 1641821774618951680}, :final-paths ()}, :valid? :unknown}, 24 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 4 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 15 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 21 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 13 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 22 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 29 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 6 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 28 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :configs ({:model #knossos.model.CASRegister{:value :nil}, :last-op {:process 624, :type :ok, :f :read, :value :nil, :index 2259, :time 115865560791}, :pending [{:persist-to 0, :index 95, :value 41, :time 10282795465, :process 84, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 156, :value 31, :time 12135754814, :process 85, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 145, :value 21, :time 11828783548, :process 86, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 624, :value 25, :time 51783599802, :process 174, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 524, :value 19, :time 49841946761, :process 175, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 618, :value 30, :time 51721296084, :process 176, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 932, :value 7, :time 66467475081, :process 264, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1288, :value 18, :time 80004052525, :process 265, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 866, :value 41, :time 61750285061, :process 266, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1352, :value 40, :time 81922275193, :process 354, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1556, :value 4, :time 90024252495, :process 355, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1602, :value 17, :time 91130135619, :process 356, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1636, :value 37, :time 91945965166, :process 444, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1874, :value 16, :time 100650548953, :process 445, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 446, :type :invoke, :f :read, :value :nil, :index 2308, :time 116765807920} {:persist-to 0, :index 1908, :value 39, :time 101965811427, :process 534, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2110, :value 41, :time 110674182422, :process 535, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2260, :value 1, :time 115865594137, :process 624, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 625, :type :invoke, :f :read, :value :nil, :index 2394, :time 120696630035}]}), :previous-ok {:process 446, :type :ok, :f :read, :value :nil, :index 2307, :time 116765776901}, :last-op nil, :op {:process 625, :type :ok, :f :read, :value :nil, :index 2527, :time 125610531893}, :final-paths ()}, :valid? :unknown}, 25 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :configs ({:model #knossos.model.CASRegister{:value :nil}, :last-op {:process 617, :type :ok, :f :read, :value :nil, :index 2089, :time 109986497392}, :pending [{:persist-to 0, :index 196, :value 26, :time 12734704110, :process 75, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 190, :value 32, :time 12633005285, :process 76, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 225, :value 5, :time 13054635457, :process 77, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 652, :value 41, :time 51969134229, :process 165, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 620, :value 43, :time 51739152985, :process 166, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 590, :value 8, :time 51455480914, :process 167, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 882, :value 13, :time 61990539788, :process 255, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1156, :value 19, :time 73520614672, :process 256, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 852, :value 38, :time 61480562304, :process 257, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1574, :value 37, :time 90354359500, :process 345, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1570, :value 48, :time 90215469613, :process 346, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1076, :value 15, :time 71504303507, :process 347, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1854, :value 20, :time 100379245978, :process 435, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1848, :value 26, :time 99924893316, :process 436, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1328, :value 33, :time 81530809086, :process 437, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2096, :value 32, :time 110410262144, :process 525, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 526, :type :invoke, :f :read, :value :nil, :index 2088, :time 109950176021} {:persist-to 0, :index 1847, :value 15, :time 99924172694, :process 527, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2090, :value 10, :time 109986522288, :process 617, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value :nil}, :last-op {:process 617, :type :ok, :f :read, :value :nil, :index 2089, :time 109986497392}, :pending [{:persist-to 0, :index 196, :value 26, :time 12734704110, :process 75, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 190, :value 32, :time 12633005285, :process 76, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 225, :value 5, :time 13054635457, :process 77, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 652, :value 41, :time 51969134229, :process 165, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 620, :value 43, :time 51739152985, :process 166, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 590, :value 8, :time 51455480914, :process 167, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 882, :value 13, :time 61990539788, :process 255, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1156, :value 19, :time 73520614672, :process 256, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 852, :value 38, :time 61480562304, :process 257, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1574, :value 37, :time 90354359500, :process 345, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1570, :value 48, :time 90215469613, :process 346, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1076, :value 15, :time 71504303507, :process 347, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1854, :value 20, :time 100379245978, :process 435, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1848, :value 26, :time 99924893316, :process 436, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1328, :value 33, :time 81530809086, :process 437, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2096, :value 32, :time 110410262144, :process 525, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1847, :value 15, :time 99924172694, :process 527, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2090, :value 10, :time 109986522288, :process 617, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]}), :previous-ok {:process 617, :type :ok, :f :read, :value :nil, :index 2089, :time 109986497392}, :last-op nil, :op {:process 526, :type :ok, :f :read, :value :nil, :index 2267, :time 115917943663}, :final-paths ()}, :valid? :unknown}, 17 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :configs ({:model #knossos.model.CASRegister{:value 41}, :last-op {:process 501, :type :ok, :f :read, :value 41, :index 2623, :time 127768586799, :cas 1641821836906201088}, :pending [{:persist-to 0, :index 500, :value 11, :time 48882400427, :process 141, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 566, :value 30, :time 50929620244, :process 142, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 834, :value 20, :time 60834233661, :process 143, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1558, :value 11, :time 90059351234, :process 231, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1042, :value 44, :time 70861406755, :process 233, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1852, :value 22, :time 100080568363, :process 321, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1274, :value 43, :time 78726189616, :process 322, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2286, :value 26, :time 116173371558, :process 411, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2200, :value 48, :time 113814131394, :process 413, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2624, :value 28, :time 127768619624, :process 501, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 503, :type :invoke, :f :read, :value 41, :index 2476, :time 123844999679} {:persist-to 0, :index 2258, :value 14, :time 115843622048, :process 592, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 41}, :last-op {:process 501, :type :ok, :f :read, :value 41, :index 2623, :time 127768586799, :cas 1641821836906201088}, :pending [{:persist-to 0, :index 408, :value 6, :time 39367506228, :process 51, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 138, :value 27, :time 11417641396, :process 52, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 500, :value 11, :time 48882400427, :process 141, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 566, :value 30, :time 50929620244, :process 142, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1274, :value 43, :time 78726189616, :process 322, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1308, :value 12, :time 80895768296, :process 323, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2286, :value 26, :time 116173371558, :process 411, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2200, :value 48, :time 113814131394, :process 413, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2624, :value 28, :time 127768619624, :process 501, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2258, :value 14, :time 115843622048, :process 592, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 682, :type :invoke, :f :read, :value 41, :index 2544, :time 125865710599}]} {:model #knossos.model.CASRegister{:value 41}, :last-op {:process 501, :type :ok, :f :read, :value 41, :index 2623, :time 127768586799, :cas 1641821836906201088}, :pending [{:persist-to 0, :index 408, :value 6, :time 39367506228, :process 51, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 138, :value 27, :time 11417641396, :process 52, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 834, :value 20, :time 60834233661, :process 143, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1558, :value 11, :time 90059351234, :process 231, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 838, :value 45, :time 60955807524, :process 232, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1042, :value 44, :time 70861406755, :process 233, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1852, :value 22, :time 100080568363, :process 321, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1308, :value 12, :time 80895768296, :process 323, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2286, :value 26, :time 116173371558, :process 411, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2200, :value 48, :time 113814131394, :process 413, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2624, :value 28, :time 127768619624, :process 501, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1926, :value 36, :time 102751004908, :process 502, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 503, :type :invoke, :f :read, :value 41, :index 2476, :time 123844999679} {:persist-to 0, :index 2258, :value 14, :time 115843622048, :process 592, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 41}, :last-op {:process 501, :type :ok, :f :read, :value 41, :index 2623, :time 127768586799, :cas 1641821836906201088}, :pending [{:persist-to 0, :index 408, :value 6, :time 39367506228, :process 51, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 200, :value 25, :time 12884449469, :process 53, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 500, :value 11, :time 48882400427, :process 141, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1558, :value 11, :time 90059351234, :process 231, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1042, :value 44, :time 70861406755, :process 233, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1852, :value 22, :time 100080568363, :process 321, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1274, :value 43, :time 78726189616, :process 322, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1308, :value 12, :time 80895768296, :process 323, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2286, :value 26, :time 116173371558, :process 411, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2200, :value 48, :time 113814131394, :process 413, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2624, :value 28, :time 127768619624, :process 501, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1926, :value 36, :time 102751004908, :process 502, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 503, :type :invoke, :f :read, :value 41, :index 2476, :time 123844999679} {:persist-to 0, :index 2258, :value 14, :time 115843622048, :process 592, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 41}, :last-op {:process 501, :type :ok, :f :read, :value 41, :index 2623, :time 127768586799, :cas 1641821836906201088}, :pending [{:persist-to 0, :index 200, :value 25, :time 12884449469, :process 53, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 566, :value 30, :time 50929620244, :process 142, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 834, :value 20, :time 60834233661, :process 143, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1558, :value 11, :time 90059351234, :process 231, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 838, :value 45, :time 60955807524, :process 232, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1042, :value 44, :time 70861406755, :process 233, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2286, :value 26, :time 116173371558, :process 411, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2200, :value 48, :time 113814131394, :process 413, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2624, :value 28, :time 127768619624, :process 501, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2258, :value 14, :time 115843622048, :process 592, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 41}, :last-op {:process 501, :type :ok, :f :read, :value 41, :index 2623, :time 127768586799, :cas 1641821836906201088}, :pending [{:persist-to 0, :index 408, :value 6, :time 39367506228, :process 51, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 138, :value 27, :time 11417641396, :process 52, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1558, :value 11, :time 90059351234, :process 231, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1042, :value 44, :time 70861406755, :process 233, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1274, :value 43, :time 78726189616, :process 322, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2286, :value 26, :time 116173371558, :process 411, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2200, :value 48, :time 113814131394, :process 413, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2624, :value 28, :time 127768619624, :process 501, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1926, :value 36, :time 102751004908, :process 502, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2258, :value 14, :time 115843622048, :process 592, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 41}, :last-op {:process 501, :type :ok, :f :read, :value 41, :index 2623, :time 127768586799, :cas 1641821836906201088}, :pending [{:persist-to 0, :index 408, :value 6, :time 39367506228, :process 51, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 138, :value 27, :time 11417641396, :process 52, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 200, :value 25, :time 12884449469, :process 53, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 500, :value 11, :time 48882400427, :process 141, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 566, :value 30, :time 50929620244, :process 142, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 834, :value 20, :time 60834233661, :process 143, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 838, :value 45, :time 60955807524, :process 232, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1042, :value 44, :time 70861406755, :process 233, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2286, :value 26, :time 116173371558, :process 411, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2200, :value 48, :time 113814131394, :process 413, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2624, :value 28, :time 127768619624, :process 501, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1926, :value 36, :time 102751004908, :process 502, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2258, :value 14, :time 115843622048, :process 592, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 41}, :last-op {:process 501, :type :ok, :f :read, :value 41, :index 2623, :time 127768586799, :cas 1641821836906201088}, :pending [{:persist-to 0, :index 408, :value 6, :time 39367506228, :process 51, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 138, :value 27, :time 11417641396, :process 52, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 500, :value 11, :time 48882400427, :process 141, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 834, :value 20, :time 60834233661, :process 143, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1558, :value 11, :time 90059351234, :process 231, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1042, :value 44, :time 70861406755, :process 233, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1274, :value 43, :time 78726189616, :process 322, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1308, :value 12, :time 80895768296, :process 323, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2286, :value 26, :time 116173371558, :process 411, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2200, :value 48, :time 113814131394, :process 413, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2624, :value 28, :time 127768619624, :process 501, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 503, :type :invoke, :f :read, :value 41, :index 2476, :time 123844999679} {:persist-to 0, :index 2258, :value 14, :time 115843622048, :process 592, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 41}, :last-op {:process 501, :type :ok, :f :read, :value 41, :index 2623, :time 127768586799, :cas 1641821836906201088}, :pending [{:persist-to 0, :index 566, :value 30, :time 50929620244, :process 142, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 834, :value 20, :time 60834233661, :process 143, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 838, :value 45, :time 60955807524, :process 232, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1042, :value 44, :time 70861406755, :process 233, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1852, :value 22, :time 100080568363, :process 321, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1274, :value 43, :time 78726189616, :process 322, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2286, :value 26, :time 116173371558, :process 411, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2200, :value 48, :time 113814131394, :process 413, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2624, :value 28, :time 127768619624, :process 501, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1926, :value 36, :time 102751004908, :process 502, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2258, :value 14, :time 115843622048, :process 592, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 682, :type :invoke, :f :read, :value 41, :index 2544, :time 125865710599}]} {:model #knossos.model.CASRegister{:value 41}, :last-op {:process 501, :type :ok, :f :read, :value 41, :index 2623, :time 127768586799, :cas 1641821836906201088}, :pending [{:persist-to 0, :index 408, :value 6, :time 39367506228, :process 51, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 200, :value 25, :time 12884449469, :process 53, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 834, :value 20, :time 60834233661, :process 143, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1042, :value 44, :time 70861406755, :process 233, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1308, :value 12, :time 80895768296, :process 323, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2286, :value 26, :time 116173371558, :process 411, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2200, :value 48, :time 113814131394, :process 413, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2624, :value 28, :time 127768619624, :process 501, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 503, :type :invoke, :f :read, :value 41, :index 2476, :time 123844999679} {:persist-to 0, :index 2258, :value 14, :time 115843622048, :process 592, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 682, :type :invoke, :f :read, :value 41, :index 2544, :time 125865710599}]}), :previous-ok {:process 501, :type :ok, :f :read, :value 41, :index 2623, :time 127768586799, :cas 1641821836906201088}, :last-op nil, :op {:process 503, :type :ok, :f :read, :value 41, :index 2639, :time 128384265014, :cas 1641821836906201088}, :final-paths ()}, :valid? :unknown}, 3 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 12 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :configs ({:model #knossos.model.CASRegister{:value :nil}, :last-op {:process 578, :type :ok, :f :read, :value :nil, :index 2295, :time 116459562107}, :pending [{:persist-to 0, :index 146, :value 30, :time 11857179142, :process 36, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 154, :value 47, :time 12076056882, :process 37, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 172, :value 11, :time 12433918505, :process 38, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1174, :value 29, :time 74171360434, :process 126, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 672, :value 27, :time 52139771914, :process 127, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 562, :value 49, :time 50908118536, :process 128, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2198, :value 16, :time 113799746758, :process 216, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 892, :value 7, :time 62169012667, :process 217, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1084, :value 34, :time 71509861156, :process 218, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1128, :value 43, :time 72190802985, :process 307, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1330, :value 23, :time 81540102592, :process 308, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1360, :value 23, :time 82215290555, :process 397, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1620, :value 41, :time 91560922998, :process 398, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1646, :value 38, :time 92255914078, :process 487, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1890, :value 9, :time 101580647374, :process 488, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1980, :value 8, :time 105590011103, :process 577, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 578, :type :invoke, :f :read, :value :nil, :index 2296, :time 116459680888}]} {:model #knossos.model.CASRegister{:value :nil}, :last-op {:process 578, :type :ok, :f :read, :value :nil, :index 2295, :time 116459562107}, :pending [{:persist-to 0, :index 146, :value 30, :time 11857179142, :process 36, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 154, :value 47, :time 12076056882, :process 37, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 172, :value 11, :time 12433918505, :process 38, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1174, :value 29, :time 74171360434, :process 126, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 672, :value 27, :time 52139771914, :process 127, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 562, :value 49, :time 50908118536, :process 128, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2198, :value 16, :time 113799746758, :process 216, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 892, :value 7, :time 62169012667, :process 217, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1084, :value 34, :time 71509861156, :process 218, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1128, :value 43, :time 72190802985, :process 307, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1330, :value 23, :time 81540102592, :process 308, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1360, :value 23, :time 82215290555, :process 397, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1620, :value 41, :time 91560922998, :process 398, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1646, :value 38, :time 92255914078, :process 487, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1890, :value 9, :time 101580647374, :process 488, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1980, :value 8, :time 105590011103, :process 577, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 578, :type :invoke, :f :read, :value :nil, :index 2296, :time 116459680888} {:process 667, :type :invoke, :f :read, :value :nil, :index 2246, :time 115616085338}]}), :previous-ok {:process 578, :type :ok, :f :read, :value :nil, :index 2295, :time 116459562107}, :last-op nil, :op {:process 578, :type :ok, :f :read, :value :nil, :index 2324, :time 117459744980}, :final-paths ()}, :valid? :unknown}, 2 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 23 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :configs ({:model #knossos.model.CASRegister{:value :nil}, :last-op {:process 519, :type :ok, :f :read, :value :nil, :index 1953, :time 104532954333}, :pending [{:persist-to 0, :index 220, :value 31, :time 13012178746, :process 69, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 195, :value 10, :time 12638198345, :process 70, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 99, :value 42, :time 10314621459, :process 71, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 598, :value 34, :time 51549404407, :process 159, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 670, :value 27, :time 52126070566, :process 160, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 514, :value 27, :time 49452843853, :process 161, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 856, :value 37, :time 61575943731, :process 249, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 906, :value 27, :time 63330466602, :process 250, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 926, :value 14, :time 66128963780, :process 251, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1098, :value 36, :time 71600268170, :process 339, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1142, :value 12, :time 73356949367, :process 340, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1446, :value 9, :time 85380865103, :process 341, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1484, :value 6, :time 87066822985, :process 429, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1400, :value 30, :time 83686665141, :process 430, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1732, :value 24, :time 95410163803, :process 431, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1954, :value 15, :time 104532987338, :process 519, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1694, :value 27, :time 93838259367, :process 520, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 521, :type :invoke, :f :read, :value :nil, :index 1976, :time 105436374022} {:process 609, :type :invoke, :f :read, :value :nil, :index 2218, :time 114556281707} {:persist-to 0, :index 1940, :value 32, :time 103868994794, :process 610, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 700, :type :invoke, :f :read, :value :nil, :index 2202, :time 113896039090}]}), :previous-ok {:process 519, :type :ok, :f :read, :value :nil, :index 1953, :time 104532954333}, :last-op nil, :op {:process 521, :type :ok, :f :read, :value :nil, :index 2241, :time 115465052417}, :final-paths ()}, :valid? :unknown}, 19 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 11 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :configs ({:model #knossos.model.CASRegister{:value 21}, :last-op {:process 485, :type :ok, :f :read, :value 21, :index 1753, :time 96434655023, :cas 1641821775052931072}, :pending [{:persist-to 0, :index 164, :value 13, :time 12317708805, :process 33, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 209, :value 30, :time 12944009846, :process 34, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 439, :value 0, :time 41082764045, :process 35, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 606, :value 5, :time 51596576179, :process 123, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 642, :value 11, :time 51896351135, :process 124, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 574, :value 15, :time 51143163313, :process 125, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1432, :value 12, :time 84544901810, :process 213, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 960, :value 16, :time 67622508371, :process 214, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 844, :value 23, :time 61174360769, :process 215, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1790, :value 11, :time 97688484156, :process 303, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1250, :value 48, :time 77643939957, :process 304, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1056, :value 5, :time 71195986986, :process 305, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 393, :type :invoke, :f :read, :value 21, :index 2040, :time 107711033229} {:persist-to 0, :index 1494, :value 16, :time 87670200876, :process 394, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1318, :value 9, :time 81220470102, :process 395, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1792, :value 35, :time 97701500490, :process 484, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1754, :value 2, :time 96434693182, :process 485, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2042, :value 13, :time 107729539549, :process 574, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2008, :value 34, :time 106454072994, :process 575, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 21}, :last-op {:process 485, :type :ok, :f :read, :value 21, :index 1753, :time 96434655023, :cas 1641821775052931072}, :pending [{:persist-to 0, :index 164, :value 13, :time 12317708805, :process 33, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 439, :value 0, :time 41082764045, :process 35, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 606, :value 5, :time 51596576179, :process 123, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 642, :value 11, :time 51896351135, :process 124, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 574, :value 15, :time 51143163313, :process 125, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1432, :value 12, :time 84544901810, :process 213, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 960, :value 16, :time 67622508371, :process 214, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 844, :value 23, :time 61174360769, :process 215, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1790, :value 11, :time 97688484156, :process 303, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1250, :value 48, :time 77643939957, :process 304, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1056, :value 5, :time 71195986986, :process 305, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 393, :type :invoke, :f :read, :value 21, :index 2040, :time 107711033229} {:persist-to 0, :index 1494, :value 16, :time 87670200876, :process 394, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1318, :value 9, :time 81220470102, :process 395, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1792, :value 35, :time 97701500490, :process 484, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1754, :value 2, :time 96434693182, :process 485, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2042, :value 13, :time 107729539549, :process 574, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2008, :value 34, :time 106454072994, :process 575, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 21}, :last-op {:process 303, :type :ok, :f :read, :value 21, :index 1789, :time 97688418782, :cas 1641821775052931072}, :pending [{:persist-to 0, :index 209, :value 30, :time 12944009846, :process 34, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 439, :value 0, :time 41082764045, :process 35, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 606, :value 5, :time 51596576179, :process 123, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 642, :value 11, :time 51896351135, :process 124, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 574, :value 15, :time 51143163313, :process 125, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1432, :value 12, :time 84544901810, :process 213, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 960, :value 16, :time 67622508371, :process 214, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 844, :value 23, :time 61174360769, :process 215, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1790, :value 11, :time 97688484156, :process 303, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1250, :value 48, :time 77643939957, :process 304, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1056, :value 5, :time 71195986986, :process 305, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 393, :type :invoke, :f :read, :value 21, :index 2040, :time 107711033229} {:persist-to 0, :index 1494, :value 16, :time 87670200876, :process 394, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1318, :value 9, :time 81220470102, :process 395, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1792, :value 35, :time 97701500490, :process 484, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1754, :value 2, :time 96434693182, :process 485, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2042, :value 13, :time 107729539549, :process 574, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2008, :value 34, :time 106454072994, :process 575, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 21}, :last-op {:process 485, :type :ok, :f :read, :value 21, :index 1753, :time 96434655023, :cas 1641821775052931072}, :pending [{:persist-to 0, :index 439, :value 0, :time 41082764045, :process 35, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 606, :value 5, :time 51596576179, :process 123, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 642, :value 11, :time 51896351135, :process 124, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 574, :value 15, :time 51143163313, :process 125, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1432, :value 12, :time 84544901810, :process 213, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 960, :value 16, :time 67622508371, :process 214, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 844, :value 23, :time 61174360769, :process 215, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1790, :value 11, :time 97688484156, :process 303, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1250, :value 48, :time 77643939957, :process 304, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1056, :value 5, :time 71195986986, :process 305, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 393, :type :invoke, :f :read, :value 21, :index 2040, :time 107711033229} {:persist-to 0, :index 1494, :value 16, :time 87670200876, :process 394, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1318, :value 9, :time 81220470102, :process 395, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1792, :value 35, :time 97701500490, :process 484, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1754, :value 2, :time 96434693182, :process 485, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2042, :value 13, :time 107729539549, :process 574, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2008, :value 34, :time 106454072994, :process 575, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]}), :previous-ok {:process 303, :type :ok, :f :read, :value 21, :index 1789, :time 97688418782, :cas 1641821775052931072}, :last-op nil, :op {:process 393, :type :ok, :f :read, :value 21, :index 2103, :time 110623843750, :cas 1641821775052931072}, :final-paths ()}, :valid? :unknown}, 9 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 5 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 14 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 26 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 16 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 10 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 18 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :final-paths (), :configs ()}, :valid? :unknown}, 8 {:timeline {:valid? true}, :linear {:valid? :unknown, :cause :out-of-memory, :configs ({:model #knossos.model.CASRegister{:value :nil}, :last-op {:process 474, :type :ok, :f :read, :value :nil, :index 2107, :time 110669465912}, :pending [{:persist-to 0, :index 186, :value 49, :time 12589013369, :process 24, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 144, :value 9, :time 11796365823, :process 25, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 208, :value 44, :time 12943144689, :process 26, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 776, :value 16, :time 59066732861, :process 114, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 916, :value 36, :time 65110891049, :process 115, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 660, :value 2, :time 52030253333, :process 116, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1146, :value 48, :time 73384464616, :process 204, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1198, :value 22, :time 75395123732, :process 205, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 886, :value 46, :time 62054526578, :process 206, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1492, :value 28, :time 87402831253, :process 294, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1628, :value 11, :time 91774765963, :process 295, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1326, :value 30, :time 81502853617, :process 296, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1860, :value 21, :time 100540613279, :process 384, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1900, :value 15, :time 101795193047, :process 385, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 1616, :value 47, :time 91526261393, :process 386, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2108, :value 18, :time 110669512128, :process 474, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:process 475, :type :invoke, :f :read, :value :nil, :index 2138, :time 111825819180} {:persist-to 0, :index 1888, :value 23, :time 101556090717, :process 476, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 2126, :value 38, :time 111580112752, :process 566, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]}), :previous-ok {:process 474, :type :ok, :f :read, :value :nil, :index 2107, :time 110669465912}, :last-op nil, :op {:process 475, :type :ok, :f :read, :value :nil, :index 2265, :time 115891411700}, :final-paths ()}, :valid? :unknown}}, :failures []}, :sanity {:valid? :unknown, :error "Test aborted due to nemesis exception"}, :valid? :unknown} Errors occurred during analysis, but no anomalies found. ಠ~ಠ Jepsen failed with unknown failure Test crashed! ###### Current Test Report ######### pass: 7 unknown: 0 fail: 0 crash: 1 memcached errors (not reported as total): 0 7/8 = 87.50% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #9 Test params: workload=partition,no-autofailover,replicas=1,disrupt-count=1,disrupt-time=10,kv-timeout=15,durability=0:0:100:0,cas Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=partition --no-autofailover --replicas=1 --disrupt-count=1 --disrupt-time=10 --kv-timeout=15 --durability=0:0:100:0 --cas --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-9.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 8 unknown: 0 fail: 0 crash: 1 memcached errors (not reported as total): 0 8/9 = 88.88% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #10 Test params: workload=kill,scenario=kill-ns-server,replicas=2,no-autofailover,durability=0:0:0:100,eviction-policy=value Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=kill --scenario=kill-ns-server --replicas=2 --no-autofailover --durability=0:0:0:100 --eviction-policy=value --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-10.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 9 unknown: 0 fail: 0 crash: 1 memcached errors (not reported as total): 0 9/10 = 90.00% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #11 Test params: workload=kill,scopes=1,collections=3,scenario=kill-babysitter,replicas=2,no-autofailover,durability=0:0:0:100,eviction-policy=value,cas Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=kill --scopes=1 --collections=3 --scenario=kill-babysitter --replicas=2 --no-autofailover --durability=0:0:0:100 --eviction-policy=value --cas --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-11.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 10 unknown: 0 fail: 0 crash: 1 memcached errors (not reported as total): 0 10/11 = 90.90% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #12 Test params: workload=kill,scenario=kill-memcached,replicas=2,no-autofailover,durability=0:0:0:100,eviction-policy=value,use-json-docs,doc-padding-size=4096 Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=kill --scenario=kill-memcached --replicas=2 --no-autofailover --durability=0:0:0:100 --eviction-policy=value --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-12.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 11 unknown: 0 fail: 0 crash: 1 memcached errors (not reported as total): 0 11/12 = 91.66% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #13 Test params: workload=kill,scenario=hard-reboot,replicas=2,no-autofailover,durability=0:0:0:100,eviction-policy=full,cas Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=kill --scenario=hard-reboot --replicas=2 --no-autofailover --durability=0:0:0:100 --eviction-policy=full --cas --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-13.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 12 unknown: 0 fail: 0 crash: 1 memcached errors (not reported as total): 0 12/13 = 92.30% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #14 Test params: workload=kill,scenario=suspend-process,process-to-suspend=memcached,process-suspend-time=35,replicas=1,no-autofailover,durability=0:0:100:0,eviction-policy=full,cas Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=kill --scenario=suspend-process --process-to-suspend=memcached --process-suspend-time=35 --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --cas --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-14.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 13 unknown: 0 fail: 0 crash: 1 memcached errors (not reported as total): 0 13/14 = 92.85% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #15 Test params: workload=set-kill,scenario=kill-memcached,replicas=1,no-autofailover,durability=0:0:100:0,eviction-policy=full,use-json-docs,doc-padding-size=64 Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-15.log file to use error_msg_from_memcached.log 08:38:05.543 [main] INFO jepsen.cli - Test options: {:scopes nil, :hashdump true, :cycles 1, :ssh-private-key "./resources/my.key", :enable-memcached-debug-log-level true, :bucket-type :couchbase, :persist-to 0, :collect-core-files false, :manipulate-disks false, :concurrency 4, :pool-size 6, :autoreprovision-maxnodes 1, :leave-db-running? false, :use-checker nil, :skip-teardown false, :logging-json? false, :dcp-set-read false, :replicate-to 0, :cluster-run false, :cas false, :ssh {:dummy? false, :username "root", :password "root", :strict-host-key-checking false, :private-key-path "./resources/my.key"}, :net-interface "eth0", :collections nil, :doc-padding-size 64, :use-subdoc false, :argv ("test" "--nodes-file" "./nodes" "--username" "root" "--ssh-private-key=./resources/my.key" "--workload=set-kill" "--scenario=kill-memcached" "--replicas=1" "--no-autofailover" "--durability=0:0:100:0" "--eviction-policy=full" "--use-json-docs" "--doc-padding-size=64" "--enable-tcp-capture" "--hashdump" "--enable-memcached-debug-log-level" "--net-interface=eth0"), :durability (0 0 100 0), :disable-out-of-order-execution false, :nodes ["172.23.105.187" "172.23.107.228" "172.23.106.61" "172.23.107.227"], :test-count 1, :install-path "/opt/couchbase", :doc-count 30, :process-suspend-time 10, :perf-graphs false, :replicas 1, :autofailover false, :storage-backend :couchstore, :cbcollect true, :connect-timeout 10, :autoreprovision true, :disrupt-count 1, :collect-data-files false, :doc-threads 3, :kv-timeout #object[java.time.Duration 0x478cebdb "PT2S"], :disable-auto-compaction false, :time-limit 60, :use-json-docs true, :eviction-policy "fullEviction", :scenario :kill-memcached, :workload "set-kill", :recovery-type :delta, :enable-tcp-capture true} INFO [2022-01-10 08:38:05,811] jepsen test runner - jepsen.core Test version 7a53fac81fdd4c21e7b4a3d63887911ad58baa52 (plus uncommitted changes) INFO [2022-01-10 08:38:05,813] jepsen test runner - jepsen.core Command line: lein run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 INFO [2022-01-10 08:38:05,931] jepsen test runner - jepsen.core Running test: {:scopes nil :remote #jepsen.control.retry.Remote{:remote #jepsen.control.scp.Remote{:cmd-remote #jepsen.control.clj_ssh.Remote{:concurrency-limit 8, :conn-spec nil, :session nil, :semaphore nil}, :conn-spec nil}, :conn nil} :hashdump true :cycles 1 :ssh-private-key "./resources/my.key" :enable-memcached-debug-log-level true :custom-vbucket-count 64 :bucket-type :couchbase :persist-to 0 :collect-core-files false :manipulate-disks false :concurrency 1000 :pool-size 6 :autoreprovision-maxnodes 1 :db #object[couchbase.core$couchbase_remote$reify__3667 "0x6e96f3cd" "couchbase.core$couchbase_remote$reify__3667@6e96f3cd"] :leave-db-running? false :use-checker nil :skip-teardown false :name "Couchbase" :logging-json? false :dcp-set-read false :start-time #object[org.joda.time.DateTime "0x2569afda" "2022-01-10T08:38:05.000-08:00"] :net #object[jepsen.net$reify__9402 "0x247a8411" "jepsen.net$reify__9402@247a8411"] :client #couchbase.clients.NewSetClient{:cluster nil, :bucket nil, :collection nil, :dcpClient nil} :replicate-to 0 :barrier #object[java.util.concurrent.CyclicBarrier "0x820b78a" "java.util.concurrent.CyclicBarrier@820b78a"] :cluster-run false :cas false :pure-generators true :ssh {:dummy? false, :username "root", :password "root", :strict-host-key-checking false, :private-key-path "./resources/my.key"} :net-interface "eth0" :collections nil :doc-padding-size 64 :checker #object[jepsen.checker$compose$reify__8903 "0x7683ea43" "jepsen.checker$compose$reify__8903@7683ea43"] :use-subdoc false :argv ("test" "--nodes-file" "./nodes" "--username" "root" "--ssh-private-key=./resources/my.key" "--workload=set-kill" "--scenario=kill-memcached" "--replicas=1" "--no-autofailover" "--durability=0:0:100:0" "--eviction-policy=full" "--use-json-docs" "--doc-padding-size=64" "--enable-tcp-capture" "--hashdump" "--enable-memcached-debug-log-level" "--net-interface=eth0") :db-intialized #object[clojure.lang.Atom "0xf1e1d66" {:status :ready, :val false}] :disrupt-time 30 :nemesis #unprintable "couchbase.nemesis$couchbase$reify__3984@75312c94" :durability (0 0 100 0) :disable-out-of-order-execution false :workload-type :set :nodes ["172.23.105.187" "172.23.107.228" "172.23.106.61" "172.23.107.227"] :test-count 1 :install-path "/opt/couchbase" :doc-count 30 :process-suspend-time 10 :perf-graphs false :replicas 1 :autofailover false :storage-backend :couchstore :cbcollect true :connect-timeout 10 :autoreprovision true :disrupt-count 1 :generator (couchbase.workload.SetWorkloadGenWrapper {:gen ((jepsen.generator.Synchronize {:gen (couchbase.workload.Stopable {:gen (jepsen.generator.Any {:gens [(jepsen.generator.OnThreads {:f #{:nemesis}, :gen [([{:type :sleep, :value 10} {:type :info, :f :kill-process, :kill-process :memcached, :targeter #object[couchbase.nemesis$basic_nodes_targeter "0x359a2c96" "couchbase.nemesis$basic_nodes_targeter@359a2c96"], :target-count 1} {:type :sleep, :value 20} {:type :info, :f :rebalance-cluster}]) {:type :sleep, :value 10} {:type :stop-test}]}) (jepsen.generator.OnThreads {:f #object[clojure.core$complement$fn__5686 "0x23ee70a7" "clojure.core$complement$fn__5686@23ee70a7"], :gen ({:f :add, :value 0, :replicate-to 0, :persist-to 0, :durability-level 2, :json true} {:f :add, :value 1, :replicate-to 0, :persist-to 0, :durability-level 2, :json true} {:f :add, :value 2, :replicate-to 0, :persist-to 0, :durability-level 2, :json true} {:f :add, :value 3, :replicate-to 0, :persist-to 0, :durability-level 2, :json true} {:f :add, :value 4, :replicate-to 0, :persist-to 0, :durability-level 2, :json true} {:f :add, :value 5, :replicate-to 0, :persist-to 0, :durability-level 2, :json true} {:f :add, :value 6, :replicate-to 0, :persist-to 0, :durability-level 2, :json true} {:f :add, :value 7, :replicate-to 0, :persist-to 0, :durability-level 2, :json true} ...)})]})})}) (jepsen.generator.Synchronize {:gen {:f :read, :value nil}})), :max_add 0}) :collect-data-files false :doc-threads 3 :kv-timeout #object[java.time.Duration "0x478cebdb" "PT2S"] :disable-auto-compaction false :os #object[jepsen.os$reify__2507 "0x400df2b3" "jepsen.os$reify__2507@400df2b3"] :time-limit 60 :use-json-docs true :eviction-policy "fullEviction" :scenario :kill-memcached :workload "set-kill" :recovery-type :delta :enable-tcp-capture true} INFO [2022-01-10 08:38:06,897] jepsen test runner - jepsen.db Tearing down DB INFO [2022-01-10 08:38:06,905] jepsen node 172.23.107.228 - couchbase.util Tearing down couchbase node INFO [2022-01-10 08:38:06,912] jepsen node 172.23.107.227 - couchbase.util Tearing down couchbase node INFO [2022-01-10 08:38:06,913] jepsen node 172.23.106.61 - couchbase.util Tearing down couchbase node INFO [2022-01-10 08:38:06,907] jepsen node 172.23.105.187 - couchbase.util Tearing down couchbase node INFO [2022-01-10 08:38:08,680] jepsen node 172.23.105.187 - couchbase.util Teardown Complete INFO [2022-01-10 08:38:08,893] jepsen node 172.23.107.227 - couchbase.util Teardown Complete INFO [2022-01-10 08:38:09,104] jepsen node 172.23.107.228 - couchbase.util Teardown Complete INFO [2022-01-10 08:38:09,312] jepsen node 172.23.106.61 - couchbase.util Teardown Complete INFO [2022-01-10 08:38:09,316] jepsen test runner - jepsen.db Setting up DB INFO [2022-01-10 08:38:09,319] jepsen node 172.23.106.61 - couchbase.util Setting up Couchbase Server INFO [2022-01-10 08:38:09,319] jepsen node 172.23.105.187 - couchbase.util Setting up Couchbase Server INFO [2022-01-10 08:38:09,319] jepsen node 172.23.107.228 - couchbase.util Setting up Couchbase Server INFO [2022-01-10 08:38:09,320] jepsen node 172.23.107.228 - couchbase.util Setting up data paths INFO [2022-01-10 08:38:09,320] jepsen node 172.23.105.187 - couchbase.util Setting up data paths INFO [2022-01-10 08:38:09,320] jepsen node 172.23.106.61 - couchbase.util Setting up data paths INFO [2022-01-10 08:38:09,321] jepsen node 172.23.107.227 - couchbase.util Setting up Couchbase Server INFO [2022-01-10 08:38:09,321] jepsen node 172.23.107.227 - couchbase.util Setting up data paths INFO [2022-01-10 08:38:09,423] jepsen node 172.23.107.227 - couchbase.util Changing permissions for /opt/couchbase INFO [2022-01-10 08:38:09,423] jepsen node 172.23.107.228 - couchbase.util Changing permissions for /opt/couchbase INFO [2022-01-10 08:38:09,423] jepsen node 172.23.105.187 - couchbase.util Changing permissions for /opt/couchbase INFO [2022-01-10 08:38:09,423] jepsen node 172.23.106.61 - couchbase.util Changing permissions for /opt/couchbase INFO [2022-01-10 08:38:09,626] jepsen node 172.23.107.228 - couchbase.util Starting daemon INFO [2022-01-10 08:38:09,626] jepsen node 172.23.106.61 - couchbase.util Starting daemon INFO [2022-01-10 08:38:09,626] jepsen node 172.23.107.227 - couchbase.util Starting daemon INFO [2022-01-10 08:38:09,726] jepsen node 172.23.105.187 - couchbase.util Starting daemon INFO [2022-01-10 08:38:26,340] jepsen node 172.23.107.227 - couchbase.util Daemon started INFO [2022-01-10 08:38:26,340] jepsen node 172.23.107.228 - couchbase.util Daemon started INFO [2022-01-10 08:38:26,340] jepsen node 172.23.106.61 - couchbase.util Daemon started INFO [2022-01-10 08:38:26,443] jepsen node 172.23.106.61 - couchbase.util packet dump file name Couchbase-172.23.106.61.pcap INFO [2022-01-10 08:38:26,443] jepsen node 172.23.107.227 - couchbase.util packet dump file name Couchbase-172.23.107.227.pcap INFO [2022-01-10 08:38:26,443] jepsen node 172.23.107.228 - couchbase.util packet dump file name Couchbase-172.23.107.228.pcap INFO [2022-01-10 08:38:28,314] jepsen node 172.23.105.187 - couchbase.util Daemon started INFO [2022-01-10 08:38:28,415] jepsen node 172.23.105.187 - couchbase.util packet dump file name Couchbase-172.23.105.187.pcap INFO [2022-01-10 08:38:28,619] jepsen test runner - jepsen.db Setting up primary 172.23.105.187 INFO [2022-01-10 08:38:28,621] jepsen node 172.23.105.187 - couchbase.util Creating couchbase cluster from 172.23.105.187 INFO [2022-01-10 08:38:28,852] jepsen node 172.23.105.187 - couchbase.util Adding node 172.23.107.228 to cluster INFO [2022-01-10 08:38:42,497] jepsen node 172.23.105.187 - couchbase.util Adding node 172.23.106.61 to cluster INFO [2022-01-10 08:38:49,028] jepsen node 172.23.105.187 - couchbase.util Adding node 172.23.107.227 to cluster INFO [2022-01-10 08:38:56,779] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {} INFO [2022-01-10 08:38:56,779] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 0 INFO [2022-01-10 08:38:56,779] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 08:38:57,801] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 08:38:57,801] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 1 INFO [2022-01-10 08:38:57,801] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 08:38:58,808] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 08:38:58,808] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 2 INFO [2022-01-10 08:38:58,809] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 1 INFO [2022-01-10 08:38:59,816] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 08:38:59,816] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 3 INFO [2022-01-10 08:38:59,816] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 2 INFO [2022-01-10 08:39:00,824] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 08:39:00,824] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 4 INFO [2022-01-10 08:39:00,824] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 3 INFO [2022-01-10 08:39:01,832] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status none} INFO [2022-01-10 08:39:01,833] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 5 INFO [2022-01-10 08:39:01,833] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 3 INFO [2022-01-10 08:39:01,834] jepsen node 172.23.105.187 - couchbase.util Rebalance complete INFO [2022-01-10 08:39:01,921] jepsen node 172.23.105.187 - couchbase.util Waiting for bucket warmup to complete... INFO [2022-01-10 08:39:01,940] jepsen node 172.23.105.187 - couchbase.util Setting memcached log level to debug INFO [2022-01-10 08:39:01,970] jepsen node 172.23.105.187 - couchbase.util Setup complete INFO [2022-01-10 08:39:01,974] jepsen test runner - jepsen.core Relative time begins now INFO [2022-01-10 08:39:01,983] jepsen node 172.23.105.187 - couchbase.cbclients Opening new client INFO [2022-01-10 08:39:04,198] jepsen node 172.23.105.187 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 08:39:04,198] jepsen node 172.23.105.187 - couchbase.cbclients Opening new client INFO [2022-01-10 08:39:04,219] jepsen node 172.23.105.187 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 08:39:04,220] jepsen node 172.23.105.187 - couchbase.cbclients Opening new client INFO [2022-01-10 08:39:04,253] jepsen node 172.23.105.187 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 08:39:04,255] jepsen node 172.23.105.187 - couchbase.cbclients Opening new client INFO [2022-01-10 08:39:04,257] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0x2b99390f00000001","seedNodes":[{"address":"172.23.107.228"}]} INFO [2022-01-10 08:39:04,274] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000001","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 08:39:04,278] jepsen node 172.23.105.187 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 08:39:04,279] jepsen node 172.23.105.187 - couchbase.cbclients Opening new client INFO [2022-01-10 08:39:04,305] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0x2b99390f00000002","seedNodes":[{"address":"172.23.105.187"}]} INFO [2022-01-10 08:39:04,306] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000002","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 08:39:04,323] jepsen node 172.23.105.187 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 08:39:04,323] jepsen node 172.23.105.187 - couchbase.cbclients Opening new client INFO [2022-01-10 08:39:04,336] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0x2b99390f00000003","seedNodes":[{"address":"172.23.107.228"}]} INFO [2022-01-10 08:39:04,336] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000003","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 08:39:04,358] jepsen node 172.23.105.187 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 08:39:04,365] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0x2b99390f00000004","seedNodes":[{"address":"172.23.105.187"}]} INFO [2022-01-10 08:39:04,365] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000004","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 08:39:04,414] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0x2b99390f00000005","seedNodes":[{"address":"172.23.107.228"}]} INFO [2022-01-10 08:39:04,414] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000005","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 08:39:04,443] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0x2b99390f00000006","seedNodes":[{"address":"172.23.105.187"}]} INFO [2022-01-10 08:39:04,444] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000006","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 08:39:04,908] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][660ms] Opened bucket "default" {"coreId":"0x2b99390f00000002"} INFO [2022-01-10 08:39:04,909] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000002","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 08:39:04,910] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000002","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 08:39:04,910] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000002","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 08:39:04,916] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000005","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 08:39:04,916] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000005","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 08:39:04,917] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000005","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 08:39:04,944] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000003","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 08:39:04,945] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][628ms] Opened bucket "default" {"coreId":"0x2b99390f00000003"} INFO [2022-01-10 08:39:04,945] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000006","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 08:39:04,955] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000006","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 08:39:04,955] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000003","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 08:39:04,955] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000003","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 08:39:04,955] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000006","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 08:39:04,967] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][560ms] Opened bucket "default" {"coreId":"0x2b99390f00000006"} INFO [2022-01-10 08:39:04,973] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][601ms] Opened bucket "default" {"coreId":"0x2b99390f00000004"} INFO [2022-01-10 08:39:04,983] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000001","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 08:39:04,986] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000001","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 08:39:04,986] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000004","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 08:39:04,987] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000001","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 08:39:04,987] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000004","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 08:39:04,987] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][804ms] Opened bucket "default" {"coreId":"0x2b99390f00000001"} INFO [2022-01-10 08:39:04,987] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b99390f00000004","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 08:39:05,017] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][599ms] Opened bucket "default" {"coreId":"0x2b99390f00000005"} INFO [2022-01-10 08:39:10,036] jepsen worker 2 - jepsen.util 2 :invoke :add 0 INFO [2022-01-10 08:39:10,049] jepsen worker 873 - jepsen.util 873 :invoke :add 1 INFO [2022-01-10 08:39:10,060] jepsen worker 949 - jepsen.util 949 :invoke :add 2 INFO [2022-01-10 08:39:10,075] jepsen worker 280 - jepsen.util 280 :invoke :add 3 INFO [2022-01-10 08:39:10,087] jepsen worker 594 - jepsen.util 594 :invoke :add 4 INFO [2022-01-10 08:39:10,103] jepsen worker 885 - jepsen.util 885 :invoke :add 5 INFO [2022-01-10 08:39:10,116] jepsen worker 740 - jepsen.util 740 :invoke :add 6 INFO [2022-01-10 08:39:10,142] jepsen worker 183 - jepsen.util 183 :invoke :add 7 INFO [2022-01-10 08:39:10,201] jepsen worker 773 - jepsen.util 773 :invoke :add 8 INFO [2022-01-10 08:39:10,234] jepsen worker 372 - jepsen.util 372 :invoke :add 9 INFO [2022-01-10 08:39:10,283] jepsen worker 991 - jepsen.util 991 :invoke :add 10 INFO [2022-01-10 08:39:10,327] jepsen worker 262 - jepsen.util 262 :invoke :add 11 INFO [2022-01-10 08:39:10,339] jepsen worker 174 - jepsen.util 174 :invoke :add 12 INFO [2022-01-10 08:39:10,356] jepsen worker 423 - jepsen.util 423 :invoke :add 13 INFO [2022-01-10 08:39:10,389] jepsen worker 184 - jepsen.util 184 :invoke :add 14 INFO [2022-01-10 08:39:10,416] jepsen worker 369 - jepsen.util 369 :invoke :add 15 INFO [2022-01-10 08:39:10,473] jepsen worker 897 - jepsen.util 897 :invoke :add 16 INFO [2022-01-10 08:39:10,489] jepsen worker 874 - jepsen.util 874 :invoke :add 17 INFO [2022-01-10 08:39:10,495] jepsen worker 546 - jepsen.util 546 :invoke :add 18 INFO [2022-01-10 08:39:10,501] jepsen worker 579 - jepsen.util 579 :invoke :add 19 INFO [2022-01-10 08:39:10,515] jepsen worker 909 - jepsen.util 909 :invoke :add 20 INFO [2022-01-10 08:39:10,517] jepsen worker 949 - jepsen.util 949 :ok :add 2 INFO [2022-01-10 08:39:10,517] jepsen worker 874 - jepsen.util 874 :ok :add 17 INFO [2022-01-10 08:39:10,517] jepsen worker 885 - jepsen.util 885 :ok :add 5 INFO [2022-01-10 08:39:10,517] jepsen worker 184 - jepsen.util 184 :ok :add 14 INFO [2022-01-10 08:39:10,518] jepsen worker 897 - jepsen.util 897 :ok :add 16 INFO [2022-01-10 08:39:10,518] jepsen worker 280 - jepsen.util 280 :ok :add 3 INFO [2022-01-10 08:39:10,518] jepsen worker 174 - jepsen.util 174 :ok :add 12 INFO [2022-01-10 08:39:10,518] jepsen worker 740 - jepsen.util 740 :ok :add 6 INFO [2022-01-10 08:39:10,518] jepsen worker 773 - jepsen.util 773 :ok :add 8 INFO [2022-01-10 08:39:10,518] jepsen worker 991 - jepsen.util 991 :ok :add 10 INFO [2022-01-10 08:39:10,518] jepsen worker 369 - jepsen.util 369 :ok :add 15 INFO [2022-01-10 08:39:10,524] jepsen worker 423 - jepsen.util 423 :ok :add 13 INFO [2022-01-10 08:39:10,524] jepsen worker 594 - jepsen.util 594 :ok :add 4 INFO [2022-01-10 08:39:10,518] jepsen worker 873 - jepsen.util 873 :ok :add 1 INFO [2022-01-10 08:39:10,524] jepsen worker 262 - jepsen.util 262 :ok :add 11 INFO [2022-01-10 08:39:10,525] jepsen worker 183 - jepsen.util 183 :ok :add 7 INFO [2022-01-10 08:39:10,525] jepsen worker 2 - jepsen.util 2 :ok :add 0 INFO [2022-01-10 08:39:10,526] jepsen worker 579 - jepsen.util 579 :ok :add 19 INFO [2022-01-10 08:39:10,529] jepsen worker 957 - jepsen.util 957 :invoke :add 21 INFO [2022-01-10 08:39:10,536] jepsen worker 372 - jepsen.util 372 :ok :add 9 INFO [2022-01-10 08:39:10,536] jepsen worker 546 - jepsen.util 546 :ok :add 18 INFO [2022-01-10 08:39:10,539] jepsen worker 909 - jepsen.util 909 :ok :add 20 INFO [2022-01-10 08:39:10,545] jepsen worker 957 - jepsen.util 957 :ok :add 21 INFO [2022-01-10 08:39:10,559] jepsen worker 469 - jepsen.util 469 :invoke :add 22 INFO [2022-01-10 08:39:10,566] jepsen worker 512 - jepsen.util 512 :invoke :add 23 INFO [2022-01-10 08:39:10,569] jepsen worker 469 - jepsen.util 469 :ok :add 22 INFO [2022-01-10 08:39:10,572] jepsen worker 496 - jepsen.util 496 :invoke :add 24 INFO [2022-01-10 08:39:10,576] jepsen worker 512 - jepsen.util 512 :ok :add 23 INFO [2022-01-10 08:39:10,579] jepsen worker 644 - jepsen.util 644 :invoke :add 25 INFO [2022-01-10 08:39:10,581] jepsen worker 496 - jepsen.util 496 :ok :add 24 INFO [2022-01-10 08:39:10,587] jepsen worker 242 - jepsen.util 242 :invoke :add 26 INFO [2022-01-10 08:39:10,589] jepsen worker 644 - jepsen.util 644 :ok :add 25 INFO [2022-01-10 08:39:10,593] jepsen worker 345 - jepsen.util 345 :invoke :add 27 INFO [2022-01-10 08:39:10,594] jepsen worker 242 - jepsen.util 242 :ok :add 26 INFO [2022-01-10 08:39:10,601] jepsen worker 345 - jepsen.util 345 :ok :add 27 INFO [2022-01-10 08:39:10,605] jepsen worker 85 - jepsen.util 85 :invoke :add 28 INFO [2022-01-10 08:39:10,612] jepsen worker 526 - jepsen.util 526 :invoke :add 29 INFO [2022-01-10 08:39:10,614] jepsen worker 85 - jepsen.util 85 :ok :add 28 INFO [2022-01-10 08:39:10,618] jepsen worker 677 - jepsen.util 677 :invoke :add 30 INFO [2022-01-10 08:39:10,621] jepsen worker 526 - jepsen.util 526 :ok :add 29 INFO [2022-01-10 08:39:10,625] jepsen worker 591 - jepsen.util 591 :invoke :add 31 INFO [2022-01-10 08:39:10,627] jepsen worker 677 - jepsen.util 677 :ok :add 30 INFO [2022-01-10 08:39:10,633] jepsen worker 357 - jepsen.util 357 :invoke :add 32 INFO [2022-01-10 08:39:10,634] jepsen worker 591 - jepsen.util 591 :ok :add 31 INFO [2022-01-10 08:39:10,642] jepsen worker 105 - jepsen.util 105 :invoke :add 33 INFO [2022-01-10 08:39:10,642] jepsen worker 357 - jepsen.util 357 :ok :add 32 INFO [2022-01-10 08:39:10,650] jepsen worker 105 - jepsen.util 105 :ok :add 33 INFO [2022-01-10 08:39:10,650] jepsen worker 151 - jepsen.util 151 :invoke :add 34 INFO [2022-01-10 08:39:10,656] jepsen worker 444 - jepsen.util 444 :invoke :add 35 INFO [2022-01-10 08:39:10,657] jepsen worker 151 - jepsen.util 151 :ok :add 34 INFO [2022-01-10 08:39:10,665] jepsen worker 571 - jepsen.util 571 :invoke :add 36 INFO [2022-01-10 08:39:10,666] jepsen worker 444 - jepsen.util 444 :ok :add 35 INFO [2022-01-10 08:39:10,675] jepsen worker 571 - jepsen.util 571 :ok :add 36 INFO [2022-01-10 08:39:10,677] jepsen worker 821 - jepsen.util 821 :invoke :add 37 INFO [2022-01-10 08:39:10,684] jepsen worker 611 - jepsen.util 611 :invoke :add 38 INFO [2022-01-10 08:39:10,689] jepsen worker 821 - jepsen.util 821 :ok :add 37 INFO [2022-01-10 08:39:10,694] jepsen worker 611 - jepsen.util 611 :ok :add 38 INFO [2022-01-10 08:39:10,694] jepsen worker 76 - jepsen.util 76 :invoke :add 39 INFO [2022-01-10 08:39:10,703] jepsen worker 76 - jepsen.util 76 :ok :add 39 INFO [2022-01-10 08:39:10,703] jepsen worker 308 - jepsen.util 308 :invoke :add 40 INFO [2022-01-10 08:39:10,736] jepsen worker 308 - jepsen.util 308 :ok :add 40 INFO [2022-01-10 08:39:10,738] jepsen worker 85 - jepsen.util 85 :invoke :add 41 INFO [2022-01-10 08:39:10,745] jepsen worker 85 - jepsen.util 85 :ok :add 41 INFO [2022-01-10 08:39:10,746] jepsen worker 262 - jepsen.util 262 :invoke :add 42 INFO [2022-01-10 08:39:10,755] jepsen worker 601 - jepsen.util 601 :invoke :add 43 INFO [2022-01-10 08:39:10,759] jepsen worker 451 - jepsen.util 451 :invoke :add 44 INFO [2022-01-10 08:39:10,763] jepsen worker 762 - jepsen.util 762 :invoke :add 45 INFO [2022-01-10 08:39:10,766] jepsen worker 262 - jepsen.util 262 :ok :add 42 INFO [2022-01-10 08:39:10,767] jepsen worker 354 - jepsen.util 354 :invoke :add 46 INFO [2022-01-10 08:39:10,769] jepsen worker 451 - jepsen.util 451 :ok :add 44 INFO [2022-01-10 08:39:10,770] jepsen worker 240 - jepsen.util 240 :invoke :add 47 INFO [2022-01-10 08:39:10,771] jepsen worker 601 - jepsen.util 601 :ok :add 43 INFO [2022-01-10 08:39:10,775] jepsen worker 762 - jepsen.util 762 :ok :add 45 INFO [2022-01-10 08:39:10,775] jepsen worker 113 - jepsen.util 113 :invoke :add 48 INFO [2022-01-10 08:39:10,778] jepsen worker 354 - jepsen.util 354 :ok :add 46 INFO [2022-01-10 08:39:10,781] jepsen worker 626 - jepsen.util 626 :invoke :add 49 INFO [2022-01-10 08:39:10,784] jepsen worker 113 - jepsen.util 113 :ok :add 48 INFO [2022-01-10 08:39:10,785] jepsen worker 435 - jepsen.util 435 :invoke :add 50 INFO [2022-01-10 08:39:10,788] jepsen worker 240 - jepsen.util 240 :ok :add 47 INFO [2022-01-10 08:39:10,789] jepsen worker 226 - jepsen.util 226 :invoke :add 51 INFO [2022-01-10 08:39:10,792] jepsen worker 626 - jepsen.util 626 :ok :add 49 INFO [2022-01-10 08:39:10,794] jepsen worker 48 - jepsen.util 48 :invoke :add 52 INFO [2022-01-10 08:39:10,795] jepsen worker 435 - jepsen.util 435 :ok :add 50 INFO [2022-01-10 08:39:10,799] jepsen worker 226 - jepsen.util 226 :ok :add 51 INFO [2022-01-10 08:39:10,799] jepsen worker 456 - jepsen.util 456 :invoke :add 53 INFO [2022-01-10 08:39:10,802] jepsen worker 48 - jepsen.util 48 :ok :add 52 INFO [2022-01-10 08:39:10,804] jepsen worker 193 - jepsen.util 193 :invoke :add 54 INFO [2022-01-10 08:39:10,807] jepsen worker 456 - jepsen.util 456 :ok :add 53 INFO [2022-01-10 08:39:10,809] jepsen worker 568 - jepsen.util 568 :invoke :add 55 INFO [2022-01-10 08:39:10,812] jepsen worker 787 - jepsen.util 787 :invoke :add 56 INFO [2022-01-10 08:39:10,816] jepsen worker 16 - jepsen.util 16 :invoke :add 57 INFO [2022-01-10 08:39:10,817] jepsen worker 193 - jepsen.util 193 :ok :add 54 INFO [2022-01-10 08:39:10,821] jepsen worker 134 - jepsen.util 134 :invoke :add 58 INFO [2022-01-10 08:39:10,822] jepsen worker 568 - jepsen.util 568 :ok :add 55 INFO [2022-01-10 08:39:10,823] jepsen worker 787 - jepsen.util 787 :ok :add 56 INFO [2022-01-10 08:39:10,825] jepsen worker 256 - jepsen.util 256 :invoke :add 59 INFO [2022-01-10 08:39:10,828] jepsen worker 16 - jepsen.util 16 :ok :add 57 INFO [2022-01-10 08:39:10,829] jepsen worker 327 - jepsen.util 327 :invoke :add 60 INFO [2022-01-10 08:39:10,832] jepsen worker 34 - jepsen.util 34 :invoke :add 61 INFO [2022-01-10 08:39:10,833] jepsen worker 134 - jepsen.util 134 :ok :add 58 INFO [2022-01-10 08:39:10,835] jepsen worker 256 - jepsen.util 256 :ok :add 59 INFO [2022-01-10 08:39:10,836] jepsen worker 844 - jepsen.util 844 :invoke :add 62 INFO [2022-01-10 08:39:10,840] jepsen worker 19 - jepsen.util 19 :invoke :add 63 INFO [2022-01-10 08:39:10,843] jepsen worker 453 - jepsen.util 453 :invoke :add 64 INFO [2022-01-10 08:39:10,844] jepsen worker 327 - jepsen.util 327 :ok :add 60 INFO [2022-01-10 08:39:10,848] jepsen worker 600 - jepsen.util 600 :invoke :add 65 INFO [2022-01-10 08:39:10,849] jepsen worker 844 - jepsen.util 844 :ok :add 62 INFO [2022-01-10 08:39:10,850] jepsen worker 34 - jepsen.util 34 :ok :add 61 INFO [2022-01-10 08:39:10,853] jepsen worker 19 - jepsen.util 19 :ok :add 63 INFO [2022-01-10 08:39:10,854] jepsen worker 815 - jepsen.util 815 :invoke :add 66 INFO [2022-01-10 08:39:10,854] jepsen worker 453 - jepsen.util 453 :ok :add 64 INFO [2022-01-10 08:39:10,858] jepsen worker 910 - jepsen.util 910 :invoke :add 67 INFO [2022-01-10 08:39:10,860] jepsen worker 600 - jepsen.util 600 :ok :add 65 INFO [2022-01-10 08:39:10,861] jepsen worker 32 - jepsen.util 32 :invoke :add 68 INFO [2022-01-10 08:39:10,864] jepsen worker 584 - jepsen.util 584 :invoke :add 69 INFO [2022-01-10 08:39:10,866] jepsen worker 226 - jepsen.util 226 :invoke :add 70 INFO [2022-01-10 08:39:10,869] jepsen worker 910 - jepsen.util 910 :ok :add 67 INFO [2022-01-10 08:39:10,870] jepsen worker 585 - jepsen.util 585 :invoke :add 71 INFO [2022-01-10 08:39:10,873] jepsen worker 32 - jepsen.util 32 :ok :add 68 INFO [2022-01-10 08:39:10,875] jepsen worker 268 - jepsen.util 268 :invoke :add 72 INFO [2022-01-10 08:39:10,876] jepsen worker 815 - jepsen.util 815 :ok :add 66 INFO [2022-01-10 08:39:10,877] jepsen worker 584 - jepsen.util 584 :ok :add 69 INFO [2022-01-10 08:39:10,878] jepsen worker 281 - jepsen.util 281 :invoke :add 73 INFO [2022-01-10 08:39:10,878] jepsen worker 226 - jepsen.util 226 :ok :add 70 INFO [2022-01-10 08:39:10,881] jepsen worker 222 - jepsen.util 222 :invoke :add 74 INFO [2022-01-10 08:39:10,886] jepsen worker 206 - jepsen.util 206 :invoke :add 75 INFO [2022-01-10 08:39:10,888] jepsen worker 790 - jepsen.util 790 :invoke :add 76 INFO [2022-01-10 08:39:10,889] jepsen worker 585 - jepsen.util 585 :ok :add 71 INFO [2022-01-10 08:39:10,891] jepsen worker 222 - jepsen.util 222 :ok :add 74 INFO [2022-01-10 08:39:10,891] jepsen worker 268 - jepsen.util 268 :ok :add 72 INFO [2022-01-10 08:39:10,892] jepsen worker 603 - jepsen.util 603 :invoke :add 77 INFO [2022-01-10 08:39:10,896] jepsen worker 281 - jepsen.util 281 :ok :add 73 INFO [2022-01-10 08:39:10,897] jepsen worker 952 - jepsen.util 952 :invoke :add 78 INFO [2022-01-10 08:39:10,902] jepsen worker 644 - jepsen.util 644 :invoke :add 79 INFO [2022-01-10 08:39:10,905] jepsen worker 790 - jepsen.util 790 :ok :add 76 INFO [2022-01-10 08:39:10,905] jepsen worker 511 - jepsen.util 511 :invoke :add 80 INFO [2022-01-10 08:39:10,905] jepsen worker 206 - jepsen.util 206 :ok :add 75 INFO [2022-01-10 08:39:10,907] jepsen worker 952 - jepsen.util 952 :ok :add 78 INFO [2022-01-10 08:39:10,909] jepsen worker 603 - jepsen.util 603 :ok :add 77 INFO [2022-01-10 08:39:10,911] jepsen worker 872 - jepsen.util 872 :invoke :add 81 INFO [2022-01-10 08:39:10,914] jepsen worker 971 - jepsen.util 971 :invoke :add 82 INFO [2022-01-10 08:39:10,915] jepsen worker 644 - jepsen.util 644 :ok :add 79 INFO [2022-01-10 08:39:10,918] jepsen worker 47 - jepsen.util 47 :invoke :add 83 INFO [2022-01-10 08:39:10,921] jepsen worker 511 - jepsen.util 511 :ok :add 80 INFO [2022-01-10 08:39:10,921] jepsen worker 257 - jepsen.util 257 :invoke :add 84 INFO [2022-01-10 08:39:10,924] jepsen worker 872 - jepsen.util 872 :ok :add 81 INFO [2022-01-10 08:39:10,925] jepsen worker 944 - jepsen.util 944 :invoke :add 85 INFO [2022-01-10 08:39:10,928] jepsen worker 60 - jepsen.util 60 :invoke :add 86 INFO [2022-01-10 08:39:10,930] jepsen worker 776 - jepsen.util 776 :invoke :add 87 INFO [2022-01-10 08:39:10,933] jepsen worker 454 - jepsen.util 454 :invoke :add 88 INFO [2022-01-10 08:39:10,936] jepsen worker 822 - jepsen.util 822 :invoke :add 89 INFO [2022-01-10 08:39:10,936] jepsen worker 944 - jepsen.util 944 :ok :add 85 INFO [2022-01-10 08:39:10,936] jepsen worker 971 - jepsen.util 971 :ok :add 82 INFO [2022-01-10 08:39:10,936] jepsen worker 257 - jepsen.util 257 :ok :add 84 INFO [2022-01-10 08:39:10,936] jepsen worker 47 - jepsen.util 47 :ok :add 83 INFO [2022-01-10 08:39:10,938] jepsen worker 515 - jepsen.util 515 :invoke :add 90 INFO [2022-01-10 08:39:10,943] jepsen worker 317 - jepsen.util 317 :invoke :add 91 INFO [2022-01-10 08:39:10,943] jepsen worker 60 - jepsen.util 60 :ok :add 86 INFO [2022-01-10 08:39:10,944] jepsen worker 776 - jepsen.util 776 :ok :add 87 INFO [2022-01-10 08:39:10,948] jepsen worker 454 - jepsen.util 454 :ok :add 88 INFO [2022-01-10 08:39:10,948] jepsen worker 239 - jepsen.util 239 :invoke :add 92 INFO [2022-01-10 08:39:10,955] jepsen worker 515 - jepsen.util 515 :ok :add 90 INFO [2022-01-10 08:39:10,955] jepsen worker 542 - jepsen.util 542 :invoke :add 93 INFO [2022-01-10 08:39:10,958] jepsen worker 822 - jepsen.util 822 :ok :add 89 INFO [2022-01-10 08:39:10,958] jepsen worker 301 - jepsen.util 301 :invoke :add 94 INFO [2022-01-10 08:39:10,959] jepsen worker 317 - jepsen.util 317 :ok :add 91 INFO [2022-01-10 08:39:10,960] jepsen worker 239 - jepsen.util 239 :ok :add 92 INFO [2022-01-10 08:39:10,961] jepsen worker 716 - jepsen.util 716 :invoke :add 95 INFO [2022-01-10 08:39:10,964] jepsen worker 566 - jepsen.util 566 :invoke :add 96 INFO [2022-01-10 08:39:10,969] jepsen worker 546 - jepsen.util 546 :invoke :add 97 INFO [2022-01-10 08:39:10,971] jepsen worker 301 - jepsen.util 301 :ok :add 94 INFO [2022-01-10 08:39:10,971] jepsen worker 835 - jepsen.util 835 :invoke :add 98 INFO [2022-01-10 08:39:10,973] jepsen worker 716 - jepsen.util 716 :ok :add 95 INFO [2022-01-10 08:39:10,974] jepsen worker 589 - jepsen.util 589 :invoke :add 99 INFO [2022-01-10 08:39:10,977] jepsen worker 297 - jepsen.util 297 :invoke :add 100 INFO [2022-01-10 08:39:10,980] jepsen worker 885 - jepsen.util 885 :invoke :add 101 INFO [2022-01-10 08:39:10,981] jepsen worker 835 - jepsen.util 835 :ok :add 98 INFO [2022-01-10 08:39:10,983] jepsen worker 566 - jepsen.util 566 :ok :add 96 INFO [2022-01-10 08:39:10,983] jepsen worker 973 - jepsen.util 973 :invoke :add 102 INFO [2022-01-10 08:39:10,983] jepsen worker 589 - jepsen.util 589 :ok :add 99 INFO [2022-01-10 08:39:10,984] jepsen worker 542 - jepsen.util 542 :ok :add 93 INFO [2022-01-10 08:39:10,988] jepsen worker 297 - jepsen.util 297 :ok :add 100 INFO [2022-01-10 08:39:10,988] jepsen worker 546 - jepsen.util 546 :ok :add 97 INFO [2022-01-10 08:39:10,989] jepsen worker 647 - jepsen.util 647 :invoke :add 103 INFO [2022-01-10 08:39:10,993] jepsen worker 501 - jepsen.util 501 :invoke :add 104 INFO [2022-01-10 08:39:10,995] jepsen worker 973 - jepsen.util 973 :ok :add 102 INFO [2022-01-10 08:39:10,995] jepsen worker 316 - jepsen.util 316 :invoke :add 105 INFO [2022-01-10 08:39:10,998] jepsen worker 130 - jepsen.util 130 :invoke :add 106 INFO [2022-01-10 08:39:11,000] jepsen worker 584 - jepsen.util 584 :invoke :add 107 INFO [2022-01-10 08:39:11,002] jepsen worker 966 - jepsen.util 966 :invoke :add 108 INFO [2022-01-10 08:39:11,005] jepsen worker 422 - jepsen.util 422 :invoke :add 109 INFO [2022-01-10 08:39:11,007] jepsen worker 647 - jepsen.util 647 :ok :add 103 INFO [2022-01-10 08:39:11,007] jepsen worker 501 - jepsen.util 501 :ok :add 104 INFO [2022-01-10 08:39:11,008] jepsen worker 413 - jepsen.util 413 :invoke :add 110 INFO [2022-01-10 08:39:11,010] jepsen worker 316 - jepsen.util 316 :ok :add 105 INFO [2022-01-10 08:39:11,011] jepsen worker 885 - jepsen.util 885 :ok :add 101 INFO [2022-01-10 08:39:11,013] jepsen worker 732 - jepsen.util 732 :invoke :add 111 INFO [2022-01-10 08:39:11,014] jepsen worker 130 - jepsen.util 130 :ok :add 106 INFO [2022-01-10 08:39:11,017] jepsen worker 556 - jepsen.util 556 :invoke :add 112 INFO [2022-01-10 08:39:11,017] jepsen worker 966 - jepsen.util 966 :ok :add 108 INFO [2022-01-10 08:39:11,017] jepsen worker 584 - jepsen.util 584 :ok :add 107 INFO [2022-01-10 08:39:11,019] jepsen worker 422 - jepsen.util 422 :ok :add 109 INFO [2022-01-10 08:39:11,021] jepsen worker 818 - jepsen.util 818 :invoke :add 113 INFO [2022-01-10 08:39:11,024] jepsen worker 587 - jepsen.util 587 :invoke :add 114 INFO [2022-01-10 08:39:11,027] jepsen worker 225 - jepsen.util 225 :invoke :add 115 INFO [2022-01-10 08:39:11,032] jepsen worker 143 - jepsen.util 143 :invoke :add 116 INFO [2022-01-10 08:39:11,033] jepsen worker 587 - jepsen.util 587 :ok :add 114 INFO [2022-01-10 08:39:11,035] jepsen worker 413 - jepsen.util 413 :ok :add 110 INFO [2022-01-10 08:39:11,038] jepsen worker 20 - jepsen.util 20 :invoke :add 117 INFO [2022-01-10 08:39:11,042] jepsen worker 732 - jepsen.util 732 :ok :add 111 INFO [2022-01-10 08:39:11,042] jepsen worker 225 - jepsen.util 225 :ok :add 115 INFO [2022-01-10 08:39:11,042] jepsen worker 556 - jepsen.util 556 :ok :add 112 INFO [2022-01-10 08:39:11,042] jepsen worker 794 - jepsen.util 794 :invoke :add 118 INFO [2022-01-10 08:39:11,048] jepsen worker 382 - jepsen.util 382 :invoke :add 119 INFO [2022-01-10 08:39:11,051] jepsen worker 818 - jepsen.util 818 :ok :add 113 INFO [2022-01-10 08:39:11,052] jepsen worker 377 - jepsen.util 377 :invoke :add 120 INFO [2022-01-10 08:39:11,053] jepsen worker 20 - jepsen.util 20 :ok :add 117 INFO [2022-01-10 08:39:11,055] jepsen worker 143 - jepsen.util 143 :ok :add 116 INFO [2022-01-10 08:39:11,055] jepsen worker 794 - jepsen.util 794 :ok :add 118 INFO [2022-01-10 08:39:11,058] jepsen worker 518 - jepsen.util 518 :invoke :add 121 INFO [2022-01-10 08:39:11,061] jepsen worker 382 - jepsen.util 382 :ok :add 119 INFO [2022-01-10 08:39:11,064] jepsen worker 183 - jepsen.util 183 :invoke :add 122 INFO [2022-01-10 08:39:11,065] jepsen worker 518 - jepsen.util 518 :ok :add 121 INFO [2022-01-10 08:39:11,068] jepsen worker 297 - jepsen.util 297 :invoke :add 123 INFO [2022-01-10 08:39:11,072] jepsen worker 732 - jepsen.util 732 :invoke :add 124 INFO [2022-01-10 08:39:11,072] jepsen worker 377 - jepsen.util 377 :ok :add 120 INFO [2022-01-10 08:39:11,076] jepsen worker 701 - jepsen.util 701 :invoke :add 125 INFO [2022-01-10 08:39:11,077] jepsen worker 297 - jepsen.util 297 :ok :add 123 INFO [2022-01-10 08:39:11,078] jepsen worker 183 - jepsen.util 183 :ok :add 122 INFO [2022-01-10 08:39:11,081] jepsen worker 732 - jepsen.util 732 :ok :add 124 INFO [2022-01-10 08:39:11,083] jepsen worker 41 - jepsen.util 41 :invoke :add 126 INFO [2022-01-10 08:39:11,087] jepsen worker 719 - jepsen.util 719 :invoke :add 127 INFO [2022-01-10 08:39:11,088] jepsen worker 701 - jepsen.util 701 :ok :add 125 INFO [2022-01-10 08:39:11,095] jepsen worker 719 - jepsen.util 719 :ok :add 127 INFO [2022-01-10 08:39:11,096] jepsen worker 674 - jepsen.util 674 :invoke :add 128 INFO [2022-01-10 08:39:11,096] jepsen worker 41 - jepsen.util 41 :ok :add 126 INFO [2022-01-10 08:39:11,101] jepsen worker 753 - jepsen.util 753 :invoke :add 129 INFO [2022-01-10 08:39:11,104] jepsen worker 674 - jepsen.util 674 :ok :add 128 INFO [2022-01-10 08:39:11,106] jepsen worker 335 - jepsen.util 335 :invoke :add 130 INFO [2022-01-10 08:39:11,110] jepsen worker 753 - jepsen.util 753 :ok :add 129 INFO [2022-01-10 08:39:11,112] jepsen worker 535 - jepsen.util 535 :invoke :add 131 INFO [2022-01-10 08:39:11,115] jepsen worker 335 - jepsen.util 335 :ok :add 130 INFO [2022-01-10 08:39:11,117] jepsen worker 989 - jepsen.util 989 :invoke :add 132 INFO [2022-01-10 08:39:11,119] jepsen worker 535 - jepsen.util 535 :ok :add 131 INFO [2022-01-10 08:39:11,121] jepsen worker 402 - jepsen.util 402 :invoke :add 133 INFO [2022-01-10 08:39:11,127] jepsen worker 989 - jepsen.util 989 :ok :add 132 INFO [2022-01-10 08:39:11,127] jepsen worker 392 - jepsen.util 392 :invoke :add 134 INFO [2022-01-10 08:39:11,128] jepsen worker 402 - jepsen.util 402 :ok :add 133 INFO [2022-01-10 08:39:11,131] jepsen worker 752 - jepsen.util 752 :invoke :add 135 INFO [2022-01-10 08:39:11,134] jepsen worker 774 - jepsen.util 774 :invoke :add 136 INFO [2022-01-10 08:39:11,134] jepsen worker 392 - jepsen.util 392 :ok :add 134 INFO [2022-01-10 08:39:11,137] jepsen worker 662 - jepsen.util 662 :invoke :add 137 INFO [2022-01-10 08:39:11,140] jepsen worker 329 - jepsen.util 329 :invoke :add 138 INFO [2022-01-10 08:39:11,141] jepsen worker 752 - jepsen.util 752 :ok :add 135 INFO [2022-01-10 08:39:11,145] jepsen worker 764 - jepsen.util 764 :invoke :add 139 INFO [2022-01-10 08:39:11,145] jepsen worker 774 - jepsen.util 774 :ok :add 136 INFO [2022-01-10 08:39:11,147] jepsen worker 662 - jepsen.util 662 :ok :add 137 INFO [2022-01-10 08:39:11,149] jepsen worker 598 - jepsen.util 598 :invoke :add 140 INFO [2022-01-10 08:39:11,153] jepsen worker 329 - jepsen.util 329 :ok :add 138 INFO [2022-01-10 08:39:11,153] jepsen worker 795 - jepsen.util 795 :invoke :add 141 INFO [2022-01-10 08:39:11,155] jepsen worker 598 - jepsen.util 598 :ok :add 140 INFO [2022-01-10 08:39:11,156] jepsen worker 764 - jepsen.util 764 :ok :add 139 INFO [2022-01-10 08:39:11,159] jepsen worker 928 - jepsen.util 928 :invoke :add 142 INFO [2022-01-10 08:39:11,162] jepsen worker 795 - jepsen.util 795 :ok :add 141 INFO [2022-01-10 08:39:11,166] jepsen worker 280 - jepsen.util 280 :invoke :add 143 INFO [2022-01-10 08:39:11,170] jepsen worker 928 - jepsen.util 928 :ok :add 142 INFO [2022-01-10 08:39:11,172] jepsen worker 437 - jepsen.util 437 :invoke :add 144 INFO [2022-01-10 08:39:11,173] jepsen worker 280 - jepsen.util 280 :ok :add 143 INFO [2022-01-10 08:39:11,176] jepsen worker 471 - jepsen.util 471 :invoke :add 145 INFO [2022-01-10 08:39:11,178] jepsen worker 437 - jepsen.util 437 :ok :add 144 INFO [2022-01-10 08:39:11,180] jepsen worker 55 - jepsen.util 55 :invoke :add 146 INFO [2022-01-10 08:39:11,183] jepsen worker 471 - jepsen.util 471 :ok :add 145 INFO [2022-01-10 08:39:11,184] jepsen worker 711 - jepsen.util 711 :invoke :add 147 INFO [2022-01-10 08:39:11,187] jepsen worker 166 - jepsen.util 166 :invoke :add 148 INFO [2022-01-10 08:39:11,190] jepsen worker 725 - jepsen.util 725 :invoke :add 149 INFO [2022-01-10 08:39:11,191] jepsen worker 55 - jepsen.util 55 :ok :add 146 INFO [2022-01-10 08:39:11,192] jepsen worker 711 - jepsen.util 711 :ok :add 147 INFO [2022-01-10 08:39:11,194] jepsen worker 166 - jepsen.util 166 :ok :add 148 INFO [2022-01-10 08:39:11,194] jepsen worker 37 - jepsen.util 37 :invoke :add 150 INFO [2022-01-10 08:39:11,198] jepsen worker 725 - jepsen.util 725 :ok :add 149 INFO [2022-01-10 08:39:11,201] jepsen worker 180 - jepsen.util 180 :invoke :add 151 INFO [2022-01-10 08:39:11,205] jepsen worker 66 - jepsen.util 66 :invoke :add 152 INFO [2022-01-10 08:39:11,208] jepsen worker 37 - jepsen.util 37 :ok :add 150 INFO [2022-01-10 08:39:11,208] jepsen worker 180 - jepsen.util 180 :ok :add 151 INFO [2022-01-10 08:39:11,215] jepsen worker 693 - jepsen.util 693 :invoke :add 153 INFO [2022-01-10 08:39:11,219] jepsen worker 66 - jepsen.util 66 :ok :add 152 INFO [2022-01-10 08:39:11,220] jepsen worker 266 - jepsen.util 266 :invoke :add 154 INFO [2022-01-10 08:39:11,222] jepsen worker 693 - jepsen.util 693 :ok :add 153 INFO [2022-01-10 08:39:11,223] jepsen worker 796 - jepsen.util 796 :invoke :add 155 INFO [2022-01-10 08:39:11,226] jepsen worker 736 - jepsen.util 736 :invoke :add 156 INFO [2022-01-10 08:39:11,229] jepsen worker 413 - jepsen.util 413 :invoke :add 157 INFO [2022-01-10 08:39:11,231] jepsen worker 49 - jepsen.util 49 :invoke :add 158 INFO [2022-01-10 08:39:11,233] jepsen worker 266 - jepsen.util 266 :ok :add 154 INFO [2022-01-10 08:39:11,236] jepsen worker 563 - jepsen.util 563 :invoke :add 159 INFO [2022-01-10 08:39:11,239] jepsen worker 426 - jepsen.util 426 :invoke :add 160 INFO [2022-01-10 08:39:11,241] jepsen worker 413 - jepsen.util 413 :ok :add 157 INFO [2022-01-10 08:39:11,241] jepsen worker 736 - jepsen.util 736 :ok :add 156 INFO [2022-01-10 08:39:11,241] jepsen worker 796 - jepsen.util 796 :ok :add 155 INFO [2022-01-10 08:39:11,242] jepsen worker 55 - jepsen.util 55 :invoke :add 161 INFO [2022-01-10 08:39:11,243] jepsen worker 49 - jepsen.util 49 :ok :add 158 INFO [2022-01-10 08:39:11,244] jepsen worker 563 - jepsen.util 563 :ok :add 159 INFO [2022-01-10 08:39:11,249] jepsen worker 349 - jepsen.util 349 :invoke :add 162 INFO [2022-01-10 08:39:11,254] jepsen worker 191 - jepsen.util 191 :invoke :add 163 INFO [2022-01-10 08:39:11,255] jepsen worker 426 - jepsen.util 426 :ok :add 160 INFO [2022-01-10 08:39:11,258] jepsen worker 865 - jepsen.util 865 :invoke :add 164 INFO [2022-01-10 08:39:11,261] jepsen worker 55 - jepsen.util 55 :ok :add 161 INFO [2022-01-10 08:39:11,262] jepsen worker 213 - jepsen.util 213 :invoke :add 165 INFO [2022-01-10 08:39:11,265] jepsen worker 865 - jepsen.util 865 :ok :add 164 INFO [2022-01-10 08:39:11,266] jepsen worker 961 - jepsen.util 961 :invoke :add 166 INFO [2022-01-10 08:39:11,268] jepsen worker 349 - jepsen.util 349 :ok :add 162 INFO [2022-01-10 08:39:11,271] jepsen worker 845 - jepsen.util 845 :invoke :add 167 INFO [2022-01-10 08:39:11,272] jepsen worker 213 - jepsen.util 213 :ok :add 165 INFO [2022-01-10 08:39:11,273] jepsen worker 191 - jepsen.util 191 :ok :add 163 INFO [2022-01-10 08:39:11,275] jepsen worker 535 - jepsen.util 535 :invoke :add 168 INFO [2022-01-10 08:39:11,277] jepsen worker 961 - jepsen.util 961 :ok :add 166 INFO [2022-01-10 08:39:11,279] jepsen worker 845 - jepsen.util 845 :ok :add 167 INFO [2022-01-10 08:39:11,280] jepsen worker 58 - jepsen.util 58 :invoke :add 169 INFO [2022-01-10 08:39:11,284] jepsen worker 756 - jepsen.util 756 :invoke :add 170 INFO [2022-01-10 08:39:11,286] jepsen worker 535 - jepsen.util 535 :ok :add 168 INFO [2022-01-10 08:39:11,288] jepsen worker 178 - jepsen.util 178 :invoke :add 171 INFO [2022-01-10 08:39:11,293] jepsen worker 525 - jepsen.util 525 :invoke :add 172 INFO [2022-01-10 08:39:11,297] jepsen worker 178 - jepsen.util 178 :ok :add 171 INFO [2022-01-10 08:39:11,297] jepsen worker 58 - jepsen.util 58 :ok :add 169 INFO [2022-01-10 08:39:11,297] jepsen worker 756 - jepsen.util 756 :ok :add 170 INFO [2022-01-10 08:39:11,299] jepsen worker 404 - jepsen.util 404 :invoke :add 173 INFO [2022-01-10 08:39:11,301] jepsen worker 525 - jepsen.util 525 :ok :add 172 INFO [2022-01-10 08:39:11,304] jepsen worker 198 - jepsen.util 198 :invoke :add 174 INFO [2022-01-10 08:39:11,306] jepsen worker 404 - jepsen.util 404 :ok :add 173 INFO [2022-01-10 08:39:11,307] jepsen worker 654 - jepsen.util 654 :invoke :add 175 INFO [2022-01-10 08:39:11,312] jepsen worker 315 - jepsen.util 315 :invoke :add 176 INFO [2022-01-10 08:39:11,313] jepsen worker 198 - jepsen.util 198 :ok :add 174 INFO [2022-01-10 08:39:11,315] jepsen worker 500 - jepsen.util 500 :invoke :add 177 INFO [2022-01-10 08:39:11,318] jepsen worker 654 - jepsen.util 654 :ok :add 175 INFO [2022-01-10 08:39:11,320] jepsen worker 35 - jepsen.util 35 :invoke :add 178 INFO [2022-01-10 08:39:11,322] jepsen worker 315 - jepsen.util 315 :ok :add 176 INFO [2022-01-10 08:39:11,325] jepsen worker 743 - jepsen.util 743 :invoke :add 179 INFO [2022-01-10 08:39:11,327] jepsen worker 35 - jepsen.util 35 :ok :add 178 INFO [2022-01-10 08:39:11,327] jepsen worker 500 - jepsen.util 500 :ok :add 177 INFO [2022-01-10 08:39:11,330] jepsen worker 868 - jepsen.util 868 :invoke :add 180 INFO [2022-01-10 08:39:11,334] jepsen worker 743 - jepsen.util 743 :ok :add 179 INFO [2022-01-10 08:39:11,334] jepsen worker 998 - jepsen.util 998 :invoke :add 181 INFO [2022-01-10 08:39:11,339] jepsen worker 421 - jepsen.util 421 :invoke :add 182 INFO [2022-01-10 08:39:11,342] jepsen worker 345 - jepsen.util 345 :invoke :add 183 INFO [2022-01-10 08:39:11,345] jepsen worker 495 - jepsen.util 495 :invoke :add 184 INFO [2022-01-10 08:39:11,348] jepsen worker 998 - jepsen.util 998 :ok :add 181 INFO [2022-01-10 08:39:11,350] jepsen worker 868 - jepsen.util 868 :ok :add 180 INFO [2022-01-10 08:39:11,352] jepsen worker 812 - jepsen.util 812 :invoke :add 185 INFO [2022-01-10 08:39:11,355] jepsen worker 345 - jepsen.util 345 :ok :add 183 INFO [2022-01-10 08:39:11,355] jepsen worker 421 - jepsen.util 421 :ok :add 182 INFO [2022-01-10 08:39:11,356] jepsen worker 495 - jepsen.util 495 :ok :add 184 INFO [2022-01-10 08:39:11,356] jepsen worker 15 - jepsen.util 15 :invoke :add 186 INFO [2022-01-10 08:39:11,360] jepsen worker 812 - jepsen.util 812 :ok :add 185 INFO [2022-01-10 08:39:11,361] jepsen worker 95 - jepsen.util 95 :invoke :add 187 INFO [2022-01-10 08:39:11,365] jepsen worker 415 - jepsen.util 415 :invoke :add 188 INFO [2022-01-10 08:39:11,369] jepsen worker 95 - jepsen.util 95 :ok :add 187 INFO [2022-01-10 08:39:11,369] jepsen worker 831 - jepsen.util 831 :invoke :add 189 INFO [2022-01-10 08:39:11,371] jepsen worker 15 - jepsen.util 15 :ok :add 186 INFO [2022-01-10 08:39:11,376] jepsen worker 415 - jepsen.util 415 :ok :add 188 INFO [2022-01-10 08:39:11,376] jepsen worker 420 - jepsen.util 420 :invoke :add 190 INFO [2022-01-10 08:39:11,377] jepsen worker 831 - jepsen.util 831 :ok :add 189 INFO [2022-01-10 08:39:11,381] jepsen worker 878 - jepsen.util 878 :invoke :add 191 INFO [2022-01-10 08:39:11,385] jepsen worker 763 - jepsen.util 763 :invoke :add 192 INFO [2022-01-10 08:39:11,389] jepsen worker 963 - jepsen.util 963 :invoke :add 193 INFO [2022-01-10 08:39:11,392] jepsen worker 453 - jepsen.util 453 :invoke :add 194 INFO [2022-01-10 08:39:11,396] jepsen worker 420 - jepsen.util 420 :ok :add 190 INFO [2022-01-10 08:39:11,397] jepsen worker 792 - jepsen.util 792 :invoke :add 195 INFO [2022-01-10 08:39:11,399] jepsen worker 453 - jepsen.util 453 :ok :add 194 INFO [2022-01-10 08:39:11,399] jepsen worker 878 - jepsen.util 878 :ok :add 191 INFO [2022-01-10 08:39:11,399] jepsen worker 763 - jepsen.util 763 :ok :add 192 INFO [2022-01-10 08:39:11,401] jepsen worker 21 - jepsen.util 21 :invoke :add 196 INFO [2022-01-10 08:39:11,404] jepsen worker 963 - jepsen.util 963 :ok :add 193 INFO [2022-01-10 08:39:11,406] jepsen worker 792 - jepsen.util 792 :ok :add 195 INFO [2022-01-10 08:39:11,407] jepsen worker 88 - jepsen.util 88 :invoke :add 197 INFO [2022-01-10 08:39:11,411] jepsen worker 97 - jepsen.util 97 :invoke :add 198 INFO [2022-01-10 08:39:11,414] jepsen worker 21 - jepsen.util 21 :ok :add 196 INFO [2022-01-10 08:39:11,414] jepsen worker 88 - jepsen.util 88 :ok :add 197 INFO [2022-01-10 08:39:11,415] jepsen worker 990 - jepsen.util 990 :invoke :add 199 INFO [2022-01-10 08:39:11,419] jepsen worker 97 - jepsen.util 97 :ok :add 198 INFO [2022-01-10 08:39:11,420] jepsen worker 999 - jepsen.util 999 :invoke :add 200 INFO [2022-01-10 08:39:11,424] jepsen worker 990 - jepsen.util 990 :ok :add 199 INFO [2022-01-10 08:39:11,425] jepsen worker 508 - jepsen.util 508 :invoke :add 201 INFO [2022-01-10 08:39:11,430] jepsen worker 999 - jepsen.util 999 :ok :add 200 INFO [2022-01-10 08:39:11,431] jepsen worker 445 - jepsen.util 445 :invoke :add 202 INFO [2022-01-10 08:39:11,436] jepsen worker 274 - jepsen.util 274 :invoke :add 203 INFO [2022-01-10 08:39:11,438] jepsen worker 445 - jepsen.util 445 :ok :add 202 INFO [2022-01-10 08:39:11,438] jepsen worker 508 - jepsen.util 508 :ok :add 201 INFO [2022-01-10 08:39:11,441] jepsen worker 440 - jepsen.util 440 :invoke :add 204 INFO [2022-01-10 08:39:11,444] jepsen worker 274 - jepsen.util 274 :ok :add 203 INFO [2022-01-10 08:39:11,447] jepsen worker 367 - jepsen.util 367 :invoke :add 205 INFO [2022-01-10 08:39:11,449] jepsen worker 440 - jepsen.util 440 :ok :add 204 INFO [2022-01-10 08:39:11,454] jepsen worker 495 - jepsen.util 495 :invoke :add 206 INFO [2022-01-10 08:39:11,455] jepsen worker 367 - jepsen.util 367 :ok :add 205 INFO [2022-01-10 08:39:11,459] jepsen worker 557 - jepsen.util 557 :invoke :add 207 INFO [2022-01-10 08:39:11,461] jepsen worker 495 - jepsen.util 495 :ok :add 206 INFO [2022-01-10 08:39:11,465] jepsen worker 364 - jepsen.util 364 :invoke :add 208 INFO [2022-01-10 08:39:11,467] jepsen worker 557 - jepsen.util 557 :ok :add 207 INFO [2022-01-10 08:39:11,470] jepsen worker 690 - jepsen.util 690 :invoke :add 209 INFO [2022-01-10 08:39:11,474] jepsen worker 364 - jepsen.util 364 :ok :add 208 INFO [2022-01-10 08:39:11,476] jepsen worker 36 - jepsen.util 36 :invoke :add 210 INFO [2022-01-10 08:39:11,479] jepsen worker 730 - jepsen.util 730 :invoke :add 211 INFO [2022-01-10 08:39:11,482] jepsen worker 690 - jepsen.util 690 :ok :add 209 INFO [2022-01-10 08:39:11,482] jepsen worker 36 - jepsen.util 36 :ok :add 210 INFO [2022-01-10 08:39:11,483] jepsen worker 117 - jepsen.util 117 :invoke :add 212 INFO [2022-01-10 08:39:11,487] jepsen worker 797 - jepsen.util 797 :invoke :add 213 INFO [2022-01-10 08:39:11,489] jepsen worker 730 - jepsen.util 730 :ok :add 211 INFO [2022-01-10 08:39:11,491] jepsen worker 362 - jepsen.util 362 :invoke :add 214 INFO [2022-01-10 08:39:11,494] jepsen worker 117 - jepsen.util 117 :ok :add 212 INFO [2022-01-10 08:39:11,494] jepsen worker 797 - jepsen.util 797 :ok :add 213 INFO [2022-01-10 08:39:11,496] jepsen worker 62 - jepsen.util 62 :invoke :add 215 INFO [2022-01-10 08:39:11,500] jepsen worker 149 - jepsen.util 149 :invoke :add 216 INFO [2022-01-10 08:39:11,501] jepsen worker 362 - jepsen.util 362 :ok :add 214 INFO [2022-01-10 08:39:11,504] jepsen worker 856 - jepsen.util 856 :invoke :add 217 INFO [2022-01-10 08:39:11,505] jepsen worker 62 - jepsen.util 62 :ok :add 215 INFO [2022-01-10 08:39:11,509] jepsen worker 149 - jepsen.util 149 :ok :add 216 INFO [2022-01-10 08:39:11,509] jepsen worker 3 - jepsen.util 3 :invoke :add 218 INFO [2022-01-10 08:39:11,514] jepsen worker 856 - jepsen.util 856 :ok :add 217 INFO [2022-01-10 08:39:11,514] jepsen worker 828 - jepsen.util 828 :invoke :add 219 INFO [2022-01-10 08:39:11,520] jepsen worker 3 - jepsen.util 3 :ok :add 218 INFO [2022-01-10 08:39:11,520] jepsen worker 746 - jepsen.util 746 :invoke :add 220 INFO [2022-01-10 08:39:11,523] jepsen worker 828 - jepsen.util 828 :ok :add 219 INFO [2022-01-10 08:39:11,527] jepsen worker 746 - jepsen.util 746 :ok :add 220 INFO [2022-01-10 08:39:11,528] jepsen worker 624 - jepsen.util 624 :invoke :add 221 INFO [2022-01-10 08:39:11,538] jepsen worker 624 - jepsen.util 624 :ok :add 221 INFO [2022-01-10 08:39:11,539] jepsen worker 697 - jepsen.util 697 :invoke :add 222 INFO [2022-01-10 08:39:11,543] jepsen worker 250 - jepsen.util 250 :invoke :add 223 INFO [2022-01-10 08:39:11,547] jepsen worker 697 - jepsen.util 697 :ok :add 222 INFO [2022-01-10 08:39:11,548] jepsen worker 965 - jepsen.util 965 :invoke :add 224 INFO [2022-01-10 08:39:11,550] jepsen worker 250 - jepsen.util 250 :ok :add 223 INFO [2022-01-10 08:39:11,552] jepsen worker 27 - jepsen.util 27 :invoke :add 225 INFO [2022-01-10 08:39:11,557] jepsen worker 965 - jepsen.util 965 :ok :add 224 INFO [2022-01-10 08:39:11,557] jepsen worker 247 - jepsen.util 247 :invoke :add 226 INFO [2022-01-10 08:39:11,560] jepsen worker 27 - jepsen.util 27 :ok :add 225 INFO [2022-01-10 08:39:11,562] jepsen worker 118 - jepsen.util 118 :invoke :add 227 INFO [2022-01-10 08:39:11,567] jepsen worker 745 - jepsen.util 745 :invoke :add 228 INFO [2022-01-10 08:39:11,568] jepsen worker 118 - jepsen.util 118 :ok :add 227 INFO [2022-01-10 08:39:11,571] jepsen worker 680 - jepsen.util 680 :invoke :add 229 INFO [2022-01-10 08:39:11,573] jepsen worker 247 - jepsen.util 247 :ok :add 226 INFO [2022-01-10 08:39:11,575] jepsen worker 224 - jepsen.util 224 :invoke :add 230 INFO [2022-01-10 08:39:11,577] jepsen worker 745 - jepsen.util 745 :ok :add 228 INFO [2022-01-10 08:39:11,579] jepsen worker 368 - jepsen.util 368 :invoke :add 231 INFO [2022-01-10 08:39:11,584] jepsen worker 907 - jepsen.util 907 :invoke :add 232 INFO [2022-01-10 08:39:11,584] jepsen worker 680 - jepsen.util 680 :ok :add 229 INFO [2022-01-10 08:39:11,588] jepsen worker 224 - jepsen.util 224 :ok :add 230 INFO [2022-01-10 08:39:11,589] jepsen worker 524 - jepsen.util 524 :invoke :add 233 INFO [2022-01-10 08:39:11,594] jepsen worker 335 - jepsen.util 335 :invoke :add 234 INFO [2022-01-10 08:39:11,597] jepsen worker 301 - jepsen.util 301 :invoke :add 235 INFO [2022-01-10 08:39:11,601] jepsen worker 52 - jepsen.util 52 :invoke :add 236 INFO [2022-01-10 08:39:11,603] jepsen worker 335 - jepsen.util 335 :ok :add 234 INFO [2022-01-10 08:39:11,604] jepsen worker 206 - jepsen.util 206 :invoke :add 237 INFO [2022-01-10 08:39:11,605] jepsen worker 301 - jepsen.util 301 :ok :add 235 INFO [2022-01-10 08:39:11,607] jepsen worker 833 - jepsen.util 833 :invoke :add 238 INFO [2022-01-10 08:39:11,610] jepsen worker 368 - jepsen.util 368 :ok :add 231 INFO [2022-01-10 08:39:11,612] jepsen worker 907 - jepsen.util 907 :ok :add 232 INFO [2022-01-10 08:39:11,613] jepsen worker 429 - jepsen.util 429 :invoke :add 239 INFO [2022-01-10 08:39:11,614] jepsen worker 524 - jepsen.util 524 :ok :add 233 INFO [2022-01-10 08:39:11,616] jepsen worker 52 - jepsen.util 52 :ok :add 236 INFO [2022-01-10 08:39:11,619] jepsen worker 201 - jepsen.util 201 :invoke :add 240 INFO [2022-01-10 08:39:11,619] jepsen worker 206 - jepsen.util 206 :ok :add 237 INFO [2022-01-10 08:39:11,619] jepsen worker 833 - jepsen.util 833 :ok :add 238 INFO [2022-01-10 08:39:11,623] jepsen worker 429 - jepsen.util 429 :ok :add 239 INFO [2022-01-10 08:39:11,626] jepsen worker 837 - jepsen.util 837 :invoke :add 241 INFO [2022-01-10 08:39:11,632] jepsen worker 438 - jepsen.util 438 :invoke :add 242 INFO [2022-01-10 08:39:11,636] jepsen worker 547 - jepsen.util 547 :invoke :add 243 INFO [2022-01-10 08:39:11,638] jepsen worker 201 - jepsen.util 201 :ok :add 240 INFO [2022-01-10 08:39:11,638] jepsen worker 837 - jepsen.util 837 :ok :add 241 INFO [2022-01-10 08:39:11,639] jepsen worker 998 - jepsen.util 998 :invoke :add 244 INFO [2022-01-10 08:39:11,644] jepsen worker 16 - jepsen.util 16 :invoke :add 245 INFO [2022-01-10 08:39:11,648] jepsen worker 438 - jepsen.util 438 :ok :add 242 INFO [2022-01-10 08:39:11,648] jepsen worker 95 - jepsen.util 95 :invoke :add 246 INFO [2022-01-10 08:39:11,649] jepsen worker 998 - jepsen.util 998 :ok :add 244 INFO [2022-01-10 08:39:11,650] jepsen worker 547 - jepsen.util 547 :ok :add 243 INFO [2022-01-10 08:39:11,652] jepsen worker 138 - jepsen.util 138 :invoke :add 247 INFO [2022-01-10 08:39:11,654] jepsen worker 16 - jepsen.util 16 :ok :add 245 INFO [2022-01-10 08:39:11,657] jepsen worker 318 - jepsen.util 318 :invoke :add 248 INFO [2022-01-10 08:39:11,661] jepsen worker 95 - jepsen.util 95 :ok :add 246 INFO [2022-01-10 08:39:11,664] jepsen worker 318 - jepsen.util 318 :ok :add 248 INFO [2022-01-10 08:39:11,665] jepsen worker 656 - jepsen.util 656 :invoke :add 249 INFO [2022-01-10 08:39:11,667] jepsen worker 138 - jepsen.util 138 :ok :add 247 INFO [2022-01-10 08:39:11,671] jepsen worker 691 - jepsen.util 691 :invoke :add 250 INFO [2022-01-10 08:39:11,672] jepsen worker 656 - jepsen.util 656 :ok :add 249 INFO [2022-01-10 08:39:11,676] jepsen worker 362 - jepsen.util 362 :invoke :add 251 INFO [2022-01-10 08:39:11,680] jepsen worker 691 - jepsen.util 691 :ok :add 250 INFO [2022-01-10 08:39:11,682] jepsen worker 331 - jepsen.util 331 :invoke :add 252 INFO [2022-01-10 08:39:11,684] jepsen worker 362 - jepsen.util 362 :ok :add 251 INFO [2022-01-10 08:39:11,686] jepsen worker 233 - jepsen.util 233 :invoke :add 253 INFO [2022-01-10 08:39:11,691] jepsen worker 816 - jepsen.util 816 :invoke :add 254 INFO [2022-01-10 08:39:11,692] jepsen worker 331 - jepsen.util 331 :ok :add 252 INFO [2022-01-10 08:39:11,695] jepsen worker 73 - jepsen.util 73 :invoke :add 255 INFO [2022-01-10 08:39:11,697] jepsen worker 233 - jepsen.util 233 :ok :add 253 INFO [2022-01-10 08:39:11,698] jepsen worker 320 - jepsen.util 320 :invoke :add 256 INFO [2022-01-10 08:39:11,699] jepsen worker 816 - jepsen.util 816 :ok :add 254 INFO [2022-01-10 08:39:11,703] jepsen worker 836 - jepsen.util 836 :invoke :add 257 INFO [2022-01-10 08:39:11,704] jepsen worker 73 - jepsen.util 73 :ok :add 255 INFO [2022-01-10 08:39:11,707] jepsen worker 497 - jepsen.util 497 :invoke :add 258 INFO [2022-01-10 08:39:11,711] jepsen worker 202 - jepsen.util 202 :invoke :add 259 INFO [2022-01-10 08:39:11,714] jepsen worker 681 - jepsen.util 681 :invoke :add 260 INFO [2022-01-10 08:39:11,714] jepsen worker 836 - jepsen.util 836 :ok :add 257 INFO [2022-01-10 08:39:11,715] jepsen worker 497 - jepsen.util 497 :ok :add 258 INFO [2022-01-10 08:39:11,716] jepsen worker 320 - jepsen.util 320 :ok :add 256 INFO [2022-01-10 08:39:11,718] jepsen worker 371 - jepsen.util 371 :invoke :add 261 INFO [2022-01-10 08:39:11,719] jepsen worker 202 - jepsen.util 202 :ok :add 259 INFO [2022-01-10 08:39:11,722] jepsen worker 990 - jepsen.util 990 :invoke :add 262 INFO [2022-01-10 08:39:11,726] jepsen worker 371 - jepsen.util 371 :ok :add 261 INFO [2022-01-10 08:39:11,728] jepsen worker 573 - jepsen.util 573 :invoke :add 263 INFO [2022-01-10 08:39:11,730] jepsen worker 681 - jepsen.util 681 :ok :add 260 INFO [2022-01-10 08:39:11,733] jepsen worker 449 - jepsen.util 449 :invoke :add 264 INFO [2022-01-10 08:39:11,734] jepsen worker 990 - jepsen.util 990 :ok :add 262 INFO [2022-01-10 08:39:11,736] jepsen worker 573 - jepsen.util 573 :ok :add 263 INFO [2022-01-10 08:39:11,737] jepsen worker 942 - jepsen.util 942 :invoke :add 265 INFO [2022-01-10 08:39:11,741] jepsen worker 203 - jepsen.util 203 :invoke :add 266 INFO [2022-01-10 08:39:11,743] jepsen worker 449 - jepsen.util 449 :ok :add 264 INFO [2022-01-10 08:39:11,745] jepsen worker 662 - jepsen.util 662 :invoke :add 267 INFO [2022-01-10 08:39:11,749] jepsen worker 909 - jepsen.util 909 :invoke :add 268 INFO [2022-01-10 08:39:11,750] jepsen worker 942 - jepsen.util 942 :ok :add 265 INFO [2022-01-10 08:39:11,753] jepsen worker 418 - jepsen.util 418 :invoke :add 269 INFO [2022-01-10 08:39:11,755] jepsen worker 662 - jepsen.util 662 :ok :add 267 INFO [2022-01-10 08:39:11,755] jepsen worker 203 - jepsen.util 203 :ok :add 266 INFO [2022-01-10 08:39:11,760] jepsen worker 909 - jepsen.util 909 :ok :add 268 INFO [2022-01-10 08:39:11,760] jepsen worker 678 - jepsen.util 678 :invoke :add 270 INFO [2022-01-10 08:39:11,762] jepsen worker 418 - jepsen.util 418 :ok :add 269 INFO [2022-01-10 08:39:11,768] jepsen worker 103 - jepsen.util 103 :invoke :add 271 INFO [2022-01-10 08:39:11,773] jepsen worker 225 - jepsen.util 225 :invoke :add 272 INFO [2022-01-10 08:39:11,775] jepsen worker 678 - jepsen.util 678 :ok :add 270 INFO [2022-01-10 08:39:11,776] jepsen worker 103 - jepsen.util 103 :ok :add 271 INFO [2022-01-10 08:39:11,778] jepsen worker 969 - jepsen.util 969 :invoke :add 273 INFO [2022-01-10 08:39:11,782] jepsen worker 225 - jepsen.util 225 :ok :add 272 INFO [2022-01-10 08:39:11,785] jepsen worker 486 - jepsen.util 486 :invoke :add 274 INFO [2022-01-10 08:39:11,786] jepsen worker 969 - jepsen.util 969 :ok :add 273 INFO [2022-01-10 08:39:11,790] jepsen worker 804 - jepsen.util 804 :invoke :add 275 INFO [2022-01-10 08:39:11,792] jepsen worker 486 - jepsen.util 486 :ok :add 274 INFO [2022-01-10 08:39:11,796] jepsen worker 841 - jepsen.util 841 :invoke :add 276 INFO [2022-01-10 08:39:11,798] jepsen worker 804 - jepsen.util 804 :ok :add 275 INFO [2022-01-10 08:39:11,803] jepsen worker 936 - jepsen.util 936 :invoke :add 277 INFO [2022-01-10 08:39:11,805] jepsen worker 841 - jepsen.util 841 :ok :add 276 INFO [2022-01-10 08:39:11,808] jepsen worker 665 - jepsen.util 665 :invoke :add 278 INFO [2022-01-10 08:39:11,810] jepsen worker 936 - jepsen.util 936 :ok :add 277 INFO [2022-01-10 08:39:11,812] jepsen worker 121 - jepsen.util 121 :invoke :add 279 INFO [2022-01-10 08:39:11,816] jepsen worker 890 - jepsen.util 890 :invoke :add 280 INFO [2022-01-10 08:39:11,817] jepsen worker 665 - jepsen.util 665 :ok :add 278 INFO [2022-01-10 08:39:11,820] jepsen worker 770 - jepsen.util 770 :invoke :add 281 INFO [2022-01-10 08:39:11,820] jepsen worker 121 - jepsen.util 121 :ok :add 279 INFO [2022-01-10 08:39:11,824] jepsen worker 107 - jepsen.util 107 :invoke :add 282 INFO [2022-01-10 08:39:11,826] jepsen worker 890 - jepsen.util 890 :ok :add 280 INFO [2022-01-10 08:39:11,827] jepsen worker 39 - jepsen.util 39 :invoke :add 283 INFO [2022-01-10 08:39:11,831] jepsen worker 793 - jepsen.util 793 :invoke :add 284 INFO [2022-01-10 08:39:11,833] jepsen worker 107 - jepsen.util 107 :ok :add 282 INFO [2022-01-10 08:39:11,833] jepsen worker 788 - jepsen.util 788 :invoke :add 285 INFO [2022-01-10 08:39:11,835] jepsen worker 770 - jepsen.util 770 :ok :add 281 INFO [2022-01-10 08:39:11,836] jepsen worker 582 - jepsen.util 582 :invoke :add 286 INFO [2022-01-10 08:39:11,838] jepsen worker 95 - jepsen.util 95 :invoke :add 287 INFO [2022-01-10 08:39:11,839] jepsen worker 39 - jepsen.util 39 :ok :add 283 INFO [2022-01-10 08:39:11,841] jepsen worker 494 - jepsen.util 494 :invoke :add 288 INFO [2022-01-10 08:39:11,843] jepsen worker 793 - jepsen.util 793 :ok :add 284 INFO [2022-01-10 08:39:11,846] jepsen worker 743 - jepsen.util 743 :invoke :add 289 INFO [2022-01-10 08:39:11,847] jepsen worker 788 - jepsen.util 788 :ok :add 285 INFO [2022-01-10 08:39:11,849] jepsen worker 494 - jepsen.util 494 :ok :add 288 INFO [2022-01-10 08:39:11,849] jepsen worker 427 - jepsen.util 427 :invoke :add 290 INFO [2022-01-10 08:39:11,853] jepsen worker 582 - jepsen.util 582 :ok :add 286 INFO [2022-01-10 08:39:11,853] jepsen worker 95 - jepsen.util 95 :ok :add 287 INFO [2022-01-10 08:39:11,854] jepsen worker 319 - jepsen.util 319 :invoke :add 291 INFO [2022-01-10 08:39:11,856] jepsen worker 427 - jepsen.util 427 :ok :add 290 INFO [2022-01-10 08:39:11,858] jepsen worker 297 - jepsen.util 297 :invoke :add 292 INFO [2022-01-10 08:39:11,859] jepsen worker 743 - jepsen.util 743 :ok :add 289 INFO [2022-01-10 08:39:11,861] jepsen worker 74 - jepsen.util 74 :invoke :add 293 INFO [2022-01-10 08:39:11,864] jepsen worker 760 - jepsen.util 760 :invoke :add 294 INFO [2022-01-10 08:39:11,866] jepsen worker 319 - jepsen.util 319 :ok :add 291 INFO [2022-01-10 08:39:11,867] jepsen worker 297 - jepsen.util 297 :ok :add 292 INFO [2022-01-10 08:39:11,868] jepsen worker 679 - jepsen.util 679 :invoke :add 295 INFO [2022-01-10 08:39:11,871] jepsen worker 74 - jepsen.util 74 :ok :add 293 INFO [2022-01-10 08:39:11,872] jepsen worker 194 - jepsen.util 194 :invoke :add 296 INFO [2022-01-10 08:39:11,876] jepsen worker 302 - jepsen.util 302 :invoke :add 297 INFO [2022-01-10 08:39:11,878] jepsen worker 679 - jepsen.util 679 :ok :add 295 INFO [2022-01-10 08:39:11,879] jepsen worker 242 - jepsen.util 242 :invoke :add 298 INFO [2022-01-10 08:39:11,883] jepsen worker 760 - jepsen.util 760 :ok :add 294 INFO [2022-01-10 08:39:11,883] jepsen worker 720 - jepsen.util 720 :invoke :add 299 INFO [2022-01-10 08:39:11,886] jepsen worker 194 - jepsen.util 194 :ok :add 296 INFO [2022-01-10 08:39:11,887] jepsen worker 242 - jepsen.util 242 :ok :add 298 INFO [2022-01-10 08:39:11,887] jepsen worker 476 - jepsen.util 476 :invoke :add 300 INFO [2022-01-10 08:39:11,889] jepsen worker 302 - jepsen.util 302 :ok :add 297 INFO [2022-01-10 08:39:11,892] jepsen worker 160 - jepsen.util 160 :invoke :add 301 INFO [2022-01-10 08:39:11,893] jepsen worker 720 - jepsen.util 720 :ok :add 299 INFO [2022-01-10 08:39:11,896] jepsen worker 338 - jepsen.util 338 :invoke :add 302 INFO [2022-01-10 08:39:11,899] jepsen worker 460 - jepsen.util 460 :invoke :add 303 INFO [2022-01-10 08:39:11,899] jepsen worker 160 - jepsen.util 160 :ok :add 301 INFO [2022-01-10 08:39:11,901] jepsen worker 854 - jepsen.util 854 :invoke :add 304 INFO [2022-01-10 08:39:11,903] jepsen worker 476 - jepsen.util 476 :ok :add 300 INFO [2022-01-10 08:39:11,904] jepsen worker 961 - jepsen.util 961 :invoke :add 305 INFO [2022-01-10 08:39:11,907] jepsen worker 460 - jepsen.util 460 :ok :add 303 INFO [2022-01-10 08:39:11,907] jepsen worker 204 - jepsen.util 204 :invoke :add 306 INFO [2022-01-10 08:39:11,912] jepsen worker 854 - jepsen.util 854 :ok :add 304 INFO [2022-01-10 08:39:11,913] jepsen worker 968 - jepsen.util 968 :invoke :add 307 INFO [2022-01-10 08:39:11,915] jepsen worker 338 - jepsen.util 338 :ok :add 302 INFO [2022-01-10 08:39:11,917] jepsen worker 961 - jepsen.util 961 :ok :add 305 INFO [2022-01-10 08:39:11,918] jepsen worker 307 - jepsen.util 307 :invoke :add 308 INFO [2022-01-10 08:39:11,920] jepsen worker 204 - jepsen.util 204 :ok :add 306 INFO [2022-01-10 08:39:11,920] jepsen worker 968 - jepsen.util 968 :ok :add 307 INFO [2022-01-10 08:39:11,922] jepsen worker 814 - jepsen.util 814 :invoke :add 309 INFO [2022-01-10 08:39:11,927] jepsen worker 14 - jepsen.util 14 :invoke :add 310 INFO [2022-01-10 08:39:11,929] jepsen worker 307 - jepsen.util 307 :ok :add 308 INFO [2022-01-10 08:39:11,932] jepsen worker 656 - jepsen.util 656 :invoke :add 311 INFO [2022-01-10 08:39:11,937] jepsen worker 316 - jepsen.util 316 :invoke :add 312 INFO [2022-01-10 08:39:11,941] jepsen worker 192 - jepsen.util 192 :invoke :add 313 INFO [2022-01-10 08:39:11,944] jepsen worker 611 - jepsen.util 611 :invoke :add 314 INFO [2022-01-10 08:39:11,945] jepsen worker 14 - jepsen.util 14 :ok :add 310 INFO [2022-01-10 08:39:11,947] jepsen worker 408 - jepsen.util 408 :invoke :add 315 INFO [2022-01-10 08:39:11,950] jepsen worker 101 - jepsen.util 101 :invoke :add 316 INFO [2022-01-10 08:39:11,950] jepsen worker 814 - jepsen.util 814 :ok :add 309 INFO [2022-01-10 08:39:11,951] jepsen worker 316 - jepsen.util 316 :ok :add 312 INFO [2022-01-10 08:39:11,952] jepsen worker 611 - jepsen.util 611 :ok :add 314 INFO [2022-01-10 08:39:11,954] jepsen worker 53 - jepsen.util 53 :invoke :add 317 INFO [2022-01-10 08:39:11,957] jepsen worker 742 - jepsen.util 742 :invoke :add 318 INFO [2022-01-10 08:39:11,959] jepsen worker 656 - jepsen.util 656 :ok :add 311 INFO [2022-01-10 08:39:11,960] jepsen worker 356 - jepsen.util 356 :invoke :add 319 INFO [2022-01-10 08:39:11,962] jepsen worker 408 - jepsen.util 408 :ok :add 315 INFO [2022-01-10 08:39:11,964] jepsen worker 101 - jepsen.util 101 :ok :add 316 INFO [2022-01-10 08:39:11,964] jepsen worker 424 - jepsen.util 424 :invoke :add 320 INFO [2022-01-10 08:39:11,965] jepsen worker 192 - jepsen.util 192 :ok :add 313 INFO [2022-01-10 08:39:11,969] jepsen worker 356 - jepsen.util 356 :ok :add 319 INFO [2022-01-10 08:39:11,970] jepsen worker 684 - jepsen.util 684 :invoke :add 321 INFO [2022-01-10 08:39:11,972] jepsen worker 424 - jepsen.util 424 :ok :add 320 INFO [2022-01-10 08:39:11,972] jepsen worker 53 - jepsen.util 53 :ok :add 317 INFO [2022-01-10 08:39:11,974] jepsen worker 742 - jepsen.util 742 :ok :add 318 INFO [2022-01-10 08:39:11,976] jepsen worker 262 - jepsen.util 262 :invoke :add 322 INFO [2022-01-10 08:39:11,980] jepsen worker 360 - jepsen.util 360 :invoke :add 323 INFO [2022-01-10 08:39:11,983] jepsen worker 611 - jepsen.util 611 :invoke :add 324 INFO [2022-01-10 08:39:11,983] jepsen worker 262 - jepsen.util 262 :ok :add 322 INFO [2022-01-10 08:39:11,985] jepsen worker 684 - jepsen.util 684 :ok :add 321 INFO [2022-01-10 08:39:11,985] jepsen worker 473 - jepsen.util 473 :invoke :add 325 INFO [2022-01-10 08:39:11,988] jepsen worker 870 - jepsen.util 870 :invoke :add 326 INFO [2022-01-10 08:39:11,989] jepsen worker 360 - jepsen.util 360 :ok :add 323 INFO [2022-01-10 08:39:11,991] jepsen worker 641 - jepsen.util 641 :invoke :add 327 INFO [2022-01-10 08:39:11,994] jepsen worker 482 - jepsen.util 482 :invoke :add 328 INFO [2022-01-10 08:39:11,998] jepsen worker 501 - jepsen.util 501 :invoke :add 329 INFO [2022-01-10 08:39:11,998] jepsen worker 473 - jepsen.util 473 :ok :add 325 INFO [2022-01-10 08:39:11,999] jepsen worker 611 - jepsen.util 611 :ok :add 324 INFO [2022-01-10 08:39:12,003] jepsen worker 348 - jepsen.util 348 :invoke :add 330 INFO [2022-01-10 08:39:12,005] jepsen worker 641 - jepsen.util 641 :ok :add 327 INFO [2022-01-10 08:39:12,008] jepsen worker 292 - jepsen.util 292 :invoke :add 331 INFO [2022-01-10 08:39:12,009] jepsen worker 870 - jepsen.util 870 :ok :add 326 INFO [2022-01-10 08:39:12,012] jepsen worker 574 - jepsen.util 574 :invoke :add 332 INFO [2022-01-10 08:39:12,016] jepsen worker 721 - jepsen.util 721 :invoke :add 333 INFO [2022-01-10 08:39:12,017] jepsen worker 482 - jepsen.util 482 :ok :add 328 INFO [2022-01-10 08:39:12,020] jepsen worker 436 - jepsen.util 436 :invoke :add 334 INFO [2022-01-10 08:39:12,023] jepsen worker 501 - jepsen.util 501 :ok :add 329 INFO [2022-01-10 08:39:12,023] jepsen worker 348 - jepsen.util 348 :ok :add 330 INFO [2022-01-10 08:39:12,023] jepsen worker 292 - jepsen.util 292 :ok :add 331 INFO [2022-01-10 08:39:12,024] jepsen worker 430 - jepsen.util 430 :invoke :add 335 INFO [2022-01-10 08:39:12,027] jepsen worker 574 - jepsen.util 574 :ok :add 332 INFO [2022-01-10 08:39:12,030] jepsen worker 436 - jepsen.util 436 :ok :add 334 INFO [2022-01-10 08:39:12,030] jepsen worker 989 - jepsen.util 989 :invoke :add 336 INFO [2022-01-10 08:39:12,034] jepsen worker 721 - jepsen.util 721 :ok :add 333 INFO [2022-01-10 08:39:12,034] jepsen worker 61 - jepsen.util 61 :invoke :add 337 INFO [2022-01-10 08:39:12,036] jepsen worker 430 - jepsen.util 430 :ok :add 335 INFO [2022-01-10 08:39:12,038] jepsen worker 684 - jepsen.util 684 :invoke :add 338 INFO [2022-01-10 08:39:12,041] jepsen worker 194 - jepsen.util 194 :invoke :add 339 INFO [2022-01-10 08:39:12,042] jepsen worker 989 - jepsen.util 989 :ok :add 336 INFO [2022-01-10 08:39:12,044] jepsen worker 13 - jepsen.util 13 :invoke :add 340 INFO [2022-01-10 08:39:12,047] jepsen worker 61 - jepsen.util 61 :ok :add 337 INFO [2022-01-10 08:39:12,047] jepsen worker 469 - jepsen.util 469 :invoke :add 341 INFO [2022-01-10 08:39:12,048] jepsen worker 684 - jepsen.util 684 :ok :add 338 INFO [2022-01-10 08:39:12,051] jepsen worker 253 - jepsen.util 253 :invoke :add 342 INFO [2022-01-10 08:39:12,052] jepsen worker 194 - jepsen.util 194 :ok :add 339 INFO [2022-01-10 08:39:12,054] jepsen worker 167 - jepsen.util 167 :invoke :add 343 INFO [2022-01-10 08:39:12,057] jepsen worker 2 - jepsen.util 2 :invoke :add 344 INFO [2022-01-10 08:39:12,059] jepsen worker 13 - jepsen.util 13 :ok :add 340 INFO [2022-01-10 08:39:12,059] jepsen worker 542 - jepsen.util 542 :invoke :add 345 INFO [2022-01-10 08:39:12,062] jepsen worker 647 - jepsen.util 647 :invoke :add 346 INFO [2022-01-10 08:39:12,063] jepsen worker 253 - jepsen.util 253 :ok :add 342 INFO [2022-01-10 08:39:12,064] jepsen worker 931 - jepsen.util 931 :invoke :add 347 INFO [2022-01-10 08:39:12,065] jepsen worker 469 - jepsen.util 469 :ok :add 341 INFO [2022-01-10 08:39:12,068] jepsen worker 572 - jepsen.util 572 :invoke :add 348 INFO [2022-01-10 08:39:12,071] jepsen worker 63 - jepsen.util 63 :invoke :add 349 INFO [2022-01-10 08:39:12,071] jepsen worker 167 - jepsen.util 167 :ok :add 343 INFO [2022-01-10 08:39:12,073] jepsen worker 542 - jepsen.util 542 :ok :add 345 INFO [2022-01-10 08:39:12,073] jepsen worker 2 - jepsen.util 2 :ok :add 344 INFO [2022-01-10 08:39:12,073] jepsen worker 372 - jepsen.util 372 :invoke :add 350 INFO [2022-01-10 08:39:12,076] jepsen worker 22 - jepsen.util 22 :invoke :add 351 INFO [2022-01-10 08:39:12,078] jepsen worker 931 - jepsen.util 931 :ok :add 347 INFO [2022-01-10 08:39:12,078] jepsen worker 647 - jepsen.util 647 :ok :add 346 INFO [2022-01-10 08:39:12,079] jepsen worker 953 - jepsen.util 953 :invoke :add 352 INFO [2022-01-10 08:39:12,081] jepsen worker 572 - jepsen.util 572 :ok :add 348 INFO [2022-01-10 08:39:12,083] jepsen worker 299 - jepsen.util 299 :invoke :add 353 INFO [2022-01-10 08:39:12,086] jepsen worker 467 - jepsen.util 467 :invoke :add 354 INFO [2022-01-10 08:39:12,088] jepsen worker 194 - jepsen.util 194 :invoke :add 355 INFO [2022-01-10 08:39:12,089] jepsen worker 63 - jepsen.util 63 :ok :add 349 INFO [2022-01-10 08:39:12,089] jepsen worker 372 - jepsen.util 372 :ok :add 350 INFO [2022-01-10 08:39:12,089] jepsen worker 22 - jepsen.util 22 :ok :add 351 INFO [2022-01-10 08:39:12,091] jepsen worker 806 - jepsen.util 806 :invoke :add 356 INFO [2022-01-10 08:39:12,095] jepsen worker 953 - jepsen.util 953 :ok :add 352 INFO [2022-01-10 08:39:12,096] jepsen worker 299 - jepsen.util 299 :ok :add 353 INFO [2022-01-10 08:39:12,095] jepsen worker 778 - jepsen.util 778 :invoke :add 357 INFO [2022-01-10 08:39:12,096] jepsen worker 467 - jepsen.util 467 :ok :add 354 INFO [2022-01-10 08:39:12,097] jepsen worker 194 - jepsen.util 194 :ok :add 355 INFO [2022-01-10 08:39:12,097] jepsen worker 927 - jepsen.util 927 :invoke :add 358 INFO [2022-01-10 08:39:12,102] jepsen worker 718 - jepsen.util 718 :invoke :add 359 INFO [2022-01-10 08:39:12,104] jepsen worker 806 - jepsen.util 806 :ok :add 356 INFO [2022-01-10 08:39:12,105] jepsen worker 160 - jepsen.util 160 :invoke :add 360 INFO [2022-01-10 08:39:12,107] jepsen worker 927 - jepsen.util 927 :ok :add 358 INFO [2022-01-10 08:39:12,107] jepsen worker 798 - jepsen.util 798 :invoke :add 361 INFO [2022-01-10 08:39:12,110] jepsen worker 35 - jepsen.util 35 :invoke :add 362 INFO [2022-01-10 08:39:12,110] jepsen worker 778 - jepsen.util 778 :ok :add 357 INFO [2022-01-10 08:39:12,111] jepsen worker 718 - jepsen.util 718 :ok :add 359 INFO [2022-01-10 08:39:12,113] jepsen worker 20 - jepsen.util 20 :invoke :add 363 INFO [2022-01-10 08:39:12,116] jepsen worker 437 - jepsen.util 437 :invoke :add 364 INFO [2022-01-10 08:39:12,118] jepsen worker 798 - jepsen.util 798 :ok :add 361 INFO [2022-01-10 08:39:12,120] jepsen worker 366 - jepsen.util 366 :invoke :add 365 INFO [2022-01-10 08:39:12,122] jepsen worker 20 - jepsen.util 20 :ok :add 363 INFO [2022-01-10 08:39:12,123] jepsen worker 630 - jepsen.util 630 :invoke :add 366 INFO [2022-01-10 08:39:12,124] jepsen worker 437 - jepsen.util 437 :ok :add 364 INFO [2022-01-10 08:39:12,126] jepsen worker 160 - jepsen.util 160 :ok :add 360 INFO [2022-01-10 08:39:12,126] jepsen worker 970 - jepsen.util 970 :invoke :add 367 INFO [2022-01-10 08:39:12,129] jepsen worker 35 - jepsen.util 35 :ok :add 362 INFO [2022-01-10 08:39:12,130] jepsen worker 366 - jepsen.util 366 :ok :add 365 INFO [2022-01-10 08:39:12,130] jepsen worker 384 - jepsen.util 384 :invoke :add 368 INFO [2022-01-10 08:39:12,133] jepsen worker 575 - jepsen.util 575 :invoke :add 369 INFO [2022-01-10 08:39:12,136] jepsen worker 359 - jepsen.util 359 :invoke :add 370 INFO [2022-01-10 08:39:12,136] jepsen worker 630 - jepsen.util 630 :ok :add 366 INFO [2022-01-10 08:39:12,138] jepsen worker 774 - jepsen.util 774 :invoke :add 371 INFO [2022-01-10 08:39:12,140] jepsen worker 620 - jepsen.util 620 :invoke :add 372 INFO [2022-01-10 08:39:12,142] jepsen worker 970 - jepsen.util 970 :ok :add 367 INFO [2022-01-10 08:39:12,143] jepsen worker 587 - jepsen.util 587 :invoke :add 373 INFO [2022-01-10 08:39:12,145] jepsen worker 368 - jepsen.util 368 :invoke :add 374 INFO [2022-01-10 08:39:12,148] jepsen worker 505 - jepsen.util 505 :invoke :add 375 INFO [2022-01-10 08:39:12,150] jepsen worker 798 - jepsen.util 798 :invoke :add 376 INFO [2022-01-10 08:39:12,152] jepsen worker 368 - jepsen.util 368 :ok :add 374 INFO [2022-01-10 08:39:12,154] jepsen worker 240 - jepsen.util 240 :invoke :add 377 INFO [2022-01-10 08:39:12,157] jepsen worker 575 - jepsen.util 575 :ok :add 369 INFO [2022-01-10 08:39:12,157] jepsen worker 494 - jepsen.util 494 :invoke :add 378 INFO [2022-01-10 08:39:12,159] jepsen worker 359 - jepsen.util 359 :ok :add 370 INFO [2022-01-10 08:39:12,159] jepsen worker 774 - jepsen.util 774 :ok :add 371 INFO [2022-01-10 08:39:12,160] jepsen worker 384 - jepsen.util 384 :ok :add 368 INFO [2022-01-10 08:39:12,160] jepsen worker 459 - jepsen.util 459 :invoke :add 379 INFO [2022-01-10 08:39:12,162] jepsen worker 505 - jepsen.util 505 :ok :add 375 INFO [2022-01-10 08:39:12,162] jepsen worker 828 - jepsen.util 828 :invoke :add 380 INFO [2022-01-10 08:39:12,163] jepsen worker 240 - jepsen.util 240 :ok :add 377 INFO [2022-01-10 08:39:12,164] jepsen worker 798 - jepsen.util 798 :ok :add 376 INFO [2022-01-10 08:39:12,166] jepsen worker 872 - jepsen.util 872 :invoke :add 381 INFO [2022-01-10 08:39:12,169] jepsen worker 144 - jepsen.util 144 :invoke :add 382 INFO [2022-01-10 08:39:12,170] jepsen worker 494 - jepsen.util 494 :ok :add 378 INFO [2022-01-10 08:39:12,173] jepsen worker 773 - jepsen.util 773 :invoke :add 383 INFO [2022-01-10 08:39:12,173] jepsen worker 459 - jepsen.util 459 :ok :add 379 INFO [2022-01-10 08:39:12,176] jepsen worker 811 - jepsen.util 811 :invoke :add 384 INFO [2022-01-10 08:39:12,180] jepsen worker 628 - jepsen.util 628 :invoke :add 385 INFO [2022-01-10 08:39:12,181] jepsen worker 828 - jepsen.util 828 :ok :add 380 INFO [2022-01-10 08:39:12,182] jepsen worker 620 - jepsen.util 620 :ok :add 372 INFO [2022-01-10 08:39:12,182] jepsen worker 587 - jepsen.util 587 :ok :add 373 INFO [2022-01-10 08:39:12,183] jepsen worker 959 - jepsen.util 959 :invoke :add 386 INFO [2022-01-10 08:39:12,185] jepsen worker 872 - jepsen.util 872 :ok :add 381 INFO [2022-01-10 08:39:12,185] jepsen worker 144 - jepsen.util 144 :ok :add 382 INFO [2022-01-10 08:39:12,187] jepsen worker 933 - jepsen.util 933 :invoke :add 387 INFO [2022-01-10 08:39:12,188] jepsen worker 811 - jepsen.util 811 :ok :add 384 INFO [2022-01-10 08:39:12,191] jepsen worker 773 - jepsen.util 773 :ok :add 383 INFO [2022-01-10 08:39:12,191] jepsen worker 887 - jepsen.util 887 :invoke :add 388 INFO [2022-01-10 08:39:12,194] jepsen worker 233 - jepsen.util 233 :invoke :add 389 INFO [2022-01-10 08:39:12,197] jepsen worker 628 - jepsen.util 628 :ok :add 385 INFO [2022-01-10 08:39:12,199] jepsen worker 968 - jepsen.util 968 :invoke :add 390 INFO [2022-01-10 08:39:12,203] jepsen worker 933 - jepsen.util 933 :ok :add 387 INFO [2022-01-10 08:39:12,206] jepsen worker 623 - jepsen.util 623 :invoke :add 391 INFO [2022-01-10 08:39:12,207] jepsen worker 959 - jepsen.util 959 :ok :add 386 INFO [2022-01-10 08:39:12,210] jepsen worker 233 - jepsen.util 233 :ok :add 389 INFO [2022-01-10 08:39:12,210] jepsen worker 107 - jepsen.util 107 :invoke :add 392 INFO [2022-01-10 08:39:12,216] jepsen worker 887 - jepsen.util 887 :ok :add 388 INFO [2022-01-10 08:39:12,217] jepsen worker 594 - jepsen.util 594 :invoke :add 393 INFO [2022-01-10 08:39:12,219] jepsen worker 623 - jepsen.util 623 :ok :add 391 INFO [2022-01-10 08:39:12,219] jepsen worker 968 - jepsen.util 968 :ok :add 390 INFO [2022-01-10 08:39:12,221] jepsen worker 716 - jepsen.util 716 :invoke :add 394 INFO [2022-01-10 08:39:12,225] jepsen worker 107 - jepsen.util 107 :ok :add 392 INFO [2022-01-10 08:39:12,225] jepsen worker 902 - jepsen.util 902 :invoke :add 395 INFO [2022-01-10 08:39:12,225] jepsen worker 594 - jepsen.util 594 :ok :add 393 INFO [2022-01-10 08:39:12,228] jepsen worker 564 - jepsen.util 564 :invoke :add 396 INFO [2022-01-10 08:39:12,229] jepsen worker 716 - jepsen.util 716 :ok :add 394 INFO [2022-01-10 08:39:12,231] jepsen worker 27 - jepsen.util 27 :invoke :add 397 INFO [2022-01-10 08:39:12,233] jepsen worker 902 - jepsen.util 902 :ok :add 395 INFO [2022-01-10 08:39:12,234] jepsen worker 174 - jepsen.util 174 :invoke :add 398 INFO [2022-01-10 08:39:12,236] jepsen worker 564 - jepsen.util 564 :ok :add 396 INFO [2022-01-10 08:39:12,237] jepsen worker 166 - jepsen.util 166 :invoke :add 399 INFO [2022-01-10 08:39:12,240] jepsen worker 376 - jepsen.util 376 :invoke :add 400 INFO [2022-01-10 08:39:12,243] jepsen worker 170 - jepsen.util 170 :invoke :add 401 INFO [2022-01-10 08:39:12,245] jepsen worker 27 - jepsen.util 27 :ok :add 397 INFO [2022-01-10 08:39:12,246] jepsen worker 400 - jepsen.util 400 :invoke :add 402 INFO [2022-01-10 08:39:12,249] jepsen worker 2 - jepsen.util 2 :invoke :add 403 INFO [2022-01-10 08:39:12,250] jepsen worker 376 - jepsen.util 376 :ok :add 400 INFO [2022-01-10 08:39:12,250] jepsen worker 174 - jepsen.util 174 :ok :add 398 INFO [2022-01-10 08:39:12,251] jepsen worker 941 - jepsen.util 941 :invoke :add 404 INFO [2022-01-10 08:39:12,253] jepsen worker 170 - jepsen.util 170 :ok :add 401 INFO [2022-01-10 08:39:12,253] jepsen worker 166 - jepsen.util 166 :ok :add 399 INFO [2022-01-10 08:39:12,256] jepsen worker 684 - jepsen.util 684 :invoke :add 405 INFO [2022-01-10 08:39:12,258] jepsen worker 400 - jepsen.util 400 :ok :add 402 INFO [2022-01-10 08:39:12,258] jepsen worker 941 - jepsen.util 941 :ok :add 404 INFO [2022-01-10 08:39:12,260] jepsen worker 866 - jepsen.util 866 :invoke :add 406 INFO [2022-01-10 08:39:12,262] jepsen worker 2 - jepsen.util 2 :ok :add 403 INFO [2022-01-10 08:39:12,267] jepsen worker 684 - jepsen.util 684 :ok :add 405 INFO [2022-01-10 08:39:12,267] jepsen worker 866 - jepsen.util 866 :ok :add 406 INFO [2022-01-10 08:39:12,269] jepsen worker 244 - jepsen.util 244 :invoke :add 407 INFO [2022-01-10 08:39:12,274] jepsen worker 461 - jepsen.util 461 :invoke :add 408 INFO [2022-01-10 08:39:12,278] jepsen worker 556 - jepsen.util 556 :invoke :add 409 INFO [2022-01-10 08:39:12,279] jepsen worker 244 - jepsen.util 244 :ok :add 407 INFO [2022-01-10 08:39:12,282] jepsen worker 461 - jepsen.util 461 :ok :add 408 INFO [2022-01-10 08:39:12,282] jepsen worker 896 - jepsen.util 896 :invoke :add 410 INFO [2022-01-10 08:39:12,288] jepsen worker 556 - jepsen.util 556 :ok :add 409 INFO [2022-01-10 08:39:12,289] jepsen worker 633 - jepsen.util 633 :invoke :add 411 INFO [2022-01-10 08:39:12,293] jepsen worker 896 - jepsen.util 896 :ok :add 410 INFO [2022-01-10 08:39:12,293] jepsen worker 589 - jepsen.util 589 :invoke :add 412 INFO [2022-01-10 08:39:12,298] jepsen worker 423 - jepsen.util 423 :invoke :add 413 INFO [2022-01-10 08:39:12,298] jepsen worker 633 - jepsen.util 633 :ok :add 411 INFO [2022-01-10 08:39:12,301] jepsen worker 21 - jepsen.util 21 :invoke :add 414 INFO [2022-01-10 08:39:12,303] jepsen worker 589 - jepsen.util 589 :ok :add 412 INFO [2022-01-10 08:39:12,306] jepsen worker 423 - jepsen.util 423 :ok :add 413 INFO [2022-01-10 08:39:12,308] jepsen worker 364 - jepsen.util 364 :invoke :add 415 INFO [2022-01-10 08:39:12,309] jepsen worker 21 - jepsen.util 21 :ok :add 414 INFO [2022-01-10 08:39:12,312] jepsen worker 823 - jepsen.util 823 :invoke :add 416 INFO [2022-01-10 08:39:12,316] jepsen worker 104 - jepsen.util 104 :invoke :add 417 INFO [2022-01-10 08:39:12,317] jepsen worker 364 - jepsen.util 364 :ok :add 415 INFO [2022-01-10 08:39:12,322] jepsen worker 786 - jepsen.util 786 :invoke :add 418 INFO [2022-01-10 08:39:12,324] jepsen worker 823 - jepsen.util 823 :ok :add 416 INFO [2022-01-10 08:39:12,327] jepsen worker 641 - jepsen.util 641 :invoke :add 419 INFO [2022-01-10 08:39:12,327] jepsen worker 104 - jepsen.util 104 :ok :add 417 INFO [2022-01-10 08:39:12,329] jepsen worker 786 - jepsen.util 786 :ok :add 418 INFO [2022-01-10 08:39:12,332] jepsen worker 725 - jepsen.util 725 :invoke :add 420 INFO [2022-01-10 08:39:12,335] jepsen worker 641 - jepsen.util 641 :ok :add 419 INFO [2022-01-10 08:39:12,337] jepsen worker 628 - jepsen.util 628 :invoke :add 421 INFO [2022-01-10 08:39:12,341] jepsen worker 725 - jepsen.util 725 :ok :add 420 INFO [2022-01-10 08:39:12,342] jepsen worker 769 - jepsen.util 769 :invoke :add 422 INFO [2022-01-10 08:39:12,348] jepsen worker 567 - jepsen.util 567 :invoke :add 423 INFO [2022-01-10 08:39:12,350] jepsen worker 628 - jepsen.util 628 :ok :add 421 INFO [2022-01-10 08:39:12,352] jepsen worker 88 - jepsen.util 88 :invoke :add 424 INFO [2022-01-10 08:39:12,354] jepsen worker 769 - jepsen.util 769 :ok :add 422 INFO [2022-01-10 08:39:12,356] jepsen worker 650 - jepsen.util 650 :invoke :add 425 INFO [2022-01-10 08:39:12,358] jepsen worker 88 - jepsen.util 88 :ok :add 424 INFO [2022-01-10 08:39:12,359] jepsen worker 24 - jepsen.util 24 :invoke :add 426 INFO [2022-01-10 08:39:12,361] jepsen worker 567 - jepsen.util 567 :ok :add 423 INFO [2022-01-10 08:39:12,362] jepsen worker 719 - jepsen.util 719 :invoke :add 427 INFO [2022-01-10 08:39:12,365] jepsen worker 149 - jepsen.util 149 :invoke :add 428 INFO [2022-01-10 08:39:12,367] jepsen worker 650 - jepsen.util 650 :ok :add 425 INFO [2022-01-10 08:39:12,367] jepsen worker 24 - jepsen.util 24 :ok :add 426 INFO [2022-01-10 08:39:12,368] jepsen worker 977 - jepsen.util 977 :invoke :add 429 INFO [2022-01-10 08:39:12,372] jepsen worker 503 - jepsen.util 503 :invoke :add 430 INFO [2022-01-10 08:39:12,374] jepsen worker 234 - jepsen.util 234 :invoke :add 431 INFO [2022-01-10 08:39:12,374] jepsen worker 719 - jepsen.util 719 :ok :add 427 INFO [2022-01-10 08:39:12,377] jepsen worker 523 - jepsen.util 523 :invoke :add 432 INFO [2022-01-10 08:39:12,379] jepsen worker 149 - jepsen.util 149 :ok :add 428 INFO [2022-01-10 08:39:12,380] jepsen worker 904 - jepsen.util 904 :invoke :add 433 INFO [2022-01-10 08:39:12,381] jepsen worker 977 - jepsen.util 977 :ok :add 429 INFO [2022-01-10 08:39:12,382] jepsen worker 888 - jepsen.util 888 :invoke :add 434 INFO [2022-01-10 08:39:12,385] jepsen worker 234 - jepsen.util 234 :ok :add 431 INFO [2022-01-10 08:39:12,385] jepsen worker 503 - jepsen.util 503 :ok :add 430 INFO [2022-01-10 08:39:12,386] jepsen worker 833 - jepsen.util 833 :invoke :add 435 INFO [2022-01-10 08:39:12,389] jepsen worker 523 - jepsen.util 523 :ok :add 432 INFO [2022-01-10 08:39:12,389] jepsen worker 425 - jepsen.util 425 :invoke :add 436 INFO [2022-01-10 08:39:12,392] jepsen worker 376 - jepsen.util 376 :invoke :add 437 INFO [2022-01-10 08:39:12,394] jepsen worker 223 - jepsen.util 223 :invoke :add 438 INFO [2022-01-10 08:39:12,396] jepsen worker 904 - jepsen.util 904 :ok :add 433 INFO [2022-01-10 08:39:12,397] jepsen worker 833 - jepsen.util 833 :ok :add 435 INFO [2022-01-10 08:39:12,400] jepsen worker 937 - jepsen.util 937 :invoke :add 440 INFO [2022-01-10 08:39:12,401] jepsen worker 881 - jepsen.util 881 :invoke :add 439 INFO [2022-01-10 08:39:12,402] jepsen worker 872 - jepsen.util 872 :invoke :add 441 INFO [2022-01-10 08:39:12,403] jepsen worker 888 - jepsen.util 888 :ok :add 434 INFO [2022-01-10 08:39:12,403] jepsen worker 376 - jepsen.util 376 :ok :add 437 INFO [2022-01-10 08:39:12,405] jepsen worker 737 - jepsen.util 737 :invoke :add 442 INFO [2022-01-10 08:39:12,405] jepsen worker 223 - jepsen.util 223 :ok :add 438 INFO [2022-01-10 08:39:12,406] jepsen worker 937 - jepsen.util 937 :ok :add 440 INFO [2022-01-10 08:39:12,407] jepsen worker 425 - jepsen.util 425 :ok :add 436 INFO [2022-01-10 08:39:12,408] jepsen worker 529 - jepsen.util 529 :invoke :add 443 INFO [2022-01-10 08:39:12,413] jepsen worker 881 - jepsen.util 881 :ok :add 439 INFO [2022-01-10 08:39:12,413] jepsen worker 424 - jepsen.util 424 :invoke :add 444 INFO [2022-01-10 08:39:12,415] jepsen worker 872 - jepsen.util 872 :ok :add 441 INFO [2022-01-10 08:39:12,416] jepsen worker 724 - jepsen.util 724 :invoke :add 445 INFO [2022-01-10 08:39:12,420] jepsen worker 479 - jepsen.util 479 :invoke :add 446 INFO [2022-01-10 08:39:12,422] jepsen worker 737 - jepsen.util 737 :ok :add 442 INFO [2022-01-10 08:39:12,422] jepsen worker 529 - jepsen.util 529 :ok :add 443 INFO [2022-01-10 08:39:12,423] jepsen worker 424 - jepsen.util 424 :ok :add 444 INFO [2022-01-10 08:39:12,424] jepsen worker 827 - jepsen.util 827 :invoke :add 447 INFO [2022-01-10 08:39:12,428] jepsen worker 234 - jepsen.util 234 :invoke :add 448 INFO [2022-01-10 08:39:12,432] jepsen worker 223 - jepsen.util 223 :invoke :add 449 INFO [2022-01-10 08:39:12,435] jepsen worker 173 - jepsen.util 173 :invoke :add 450 INFO [2022-01-10 08:39:12,438] jepsen worker 520 - jepsen.util 520 :invoke :add 451 INFO [2022-01-10 08:39:12,439] jepsen worker 234 - jepsen.util 234 :ok :add 448 INFO [2022-01-10 08:39:12,439] jepsen worker 223 - jepsen.util 223 :ok :add 449 INFO [2022-01-10 08:39:12,442] jepsen worker 145 - jepsen.util 145 :invoke :add 452 INFO [2022-01-10 08:39:12,445] jepsen worker 173 - jepsen.util 173 :ok :add 450 INFO [2022-01-10 08:39:12,447] jepsen worker 133 - jepsen.util 133 :invoke :add 453 INFO [2022-01-10 08:39:12,449] jepsen worker 724 - jepsen.util 724 :ok :add 445 INFO [2022-01-10 08:39:12,450] jepsen worker 611 - jepsen.util 611 :invoke :add 454 INFO [2022-01-10 08:39:12,450] jepsen worker 520 - jepsen.util 520 :ok :add 451 INFO [2022-01-10 08:39:12,450] jepsen worker 145 - jepsen.util 145 :ok :add 452 INFO [2022-01-10 08:39:12,453] jepsen worker 827 - jepsen.util 827 :ok :add 447 INFO [2022-01-10 08:39:12,454] jepsen worker 344 - jepsen.util 344 :invoke :add 455 INFO [2022-01-10 08:39:12,456] jepsen worker 641 - jepsen.util 641 :invoke :add 456 INFO [2022-01-10 08:39:12,457] jepsen worker 133 - jepsen.util 133 :ok :add 453 INFO [2022-01-10 08:39:12,458] jepsen worker 479 - jepsen.util 479 :ok :add 446 INFO [2022-01-10 08:39:12,459] jepsen worker 238 - jepsen.util 238 :invoke :add 457 INFO [2022-01-10 08:39:12,463] jepsen worker 643 - jepsen.util 643 :invoke :add 458 INFO [2022-01-10 08:39:12,465] jepsen worker 344 - jepsen.util 344 :ok :add 455 INFO [2022-01-10 08:39:12,466] jepsen worker 424 - jepsen.util 424 :invoke :add 459 INFO [2022-01-10 08:39:12,468] jepsen worker 611 - jepsen.util 611 :ok :add 454 INFO [2022-01-10 08:39:12,468] jepsen worker 238 - jepsen.util 238 :ok :add 457 INFO [2022-01-10 08:39:12,468] jepsen worker 251 - jepsen.util 251 :invoke :add 460 INFO [2022-01-10 08:39:12,471] jepsen worker 249 - jepsen.util 249 :invoke :add 461 INFO [2022-01-10 08:39:12,474] jepsen worker 102 - jepsen.util 102 :invoke :add 462 INFO [2022-01-10 08:39:12,475] jepsen worker 641 - jepsen.util 641 :ok :add 456 INFO [2022-01-10 08:39:12,477] jepsen worker 83 - jepsen.util 83 :invoke :add 463 INFO [2022-01-10 08:39:12,480] jepsen worker 250 - jepsen.util 250 :invoke :add 464 INFO [2022-01-10 08:39:12,483] jepsen worker 627 - jepsen.util 627 :invoke :add 465 INFO [2022-01-10 08:39:12,484] jepsen worker 249 - jepsen.util 249 :ok :add 461 INFO [2022-01-10 08:39:12,486] jepsen worker 251 - jepsen.util 251 :ok :add 460 INFO [2022-01-10 08:39:12,487] jepsen worker 599 - jepsen.util 599 :invoke :add 466 INFO [2022-01-10 08:39:12,489] jepsen worker 250 - jepsen.util 250 :ok :add 464 INFO [2022-01-10 08:39:12,489] jepsen worker 83 - jepsen.util 83 :ok :add 463 INFO [2022-01-10 08:39:12,490] jepsen worker 856 - jepsen.util 856 :invoke :add 467 INFO [2022-01-10 08:39:12,491] jepsen worker 627 - jepsen.util 627 :ok :add 465 INFO [2022-01-10 08:39:12,491] jepsen worker 102 - jepsen.util 102 :ok :add 462 INFO [2022-01-10 08:39:12,496] jepsen worker 94 - jepsen.util 94 :invoke :add 468 INFO [2022-01-10 08:39:12,499] jepsen worker 599 - jepsen.util 599 :ok :add 466 INFO [2022-01-10 08:39:12,503] jepsen worker 372 - jepsen.util 372 :invoke :add 469 INFO [2022-01-10 08:39:12,504] jepsen worker 94 - jepsen.util 94 :ok :add 468 INFO [2022-01-10 08:39:12,505] jepsen worker 856 - jepsen.util 856 :ok :add 467 INFO [2022-01-10 08:39:12,507] jepsen worker 424 - jepsen.util 424 :ok :add 459 INFO [2022-01-10 08:39:12,507] jepsen worker 643 - jepsen.util 643 :ok :add 458 INFO [2022-01-10 08:39:12,507] jepsen worker 96 - jepsen.util 96 :invoke :add 470 INFO [2022-01-10 08:39:12,512] jepsen worker 168 - jepsen.util 168 :invoke :add 471 INFO [2022-01-10 08:39:12,513] jepsen worker 372 - jepsen.util 372 :ok :add 469 INFO [2022-01-10 08:39:12,515] jepsen worker 184 - jepsen.util 184 :invoke :add 472 INFO [2022-01-10 08:39:12,521] jepsen worker 492 - jepsen.util 492 :invoke :add 473 INFO [2022-01-10 08:39:12,522] jepsen worker 168 - jepsen.util 168 :ok :add 471 INFO [2022-01-10 08:39:12,522] jepsen worker 96 - jepsen.util 96 :ok :add 470 INFO [2022-01-10 08:39:12,524] jepsen worker 653 - jepsen.util 653 :invoke :add 474 INFO [2022-01-10 08:39:12,529] jepsen worker 331 - jepsen.util 331 :invoke :add 475 INFO [2022-01-10 08:39:12,529] jepsen worker 184 - jepsen.util 184 :ok :add 472 INFO [2022-01-10 08:39:12,530] jepsen worker 492 - jepsen.util 492 :ok :add 473 INFO [2022-01-10 08:39:12,535] jepsen worker 310 - jepsen.util 310 :invoke :add 476 INFO [2022-01-10 08:39:12,542] jepsen worker 653 - jepsen.util 653 :ok :add 474 INFO [2022-01-10 08:39:12,544] jepsen worker 944 - jepsen.util 944 :invoke :add 477 INFO [2022-01-10 08:39:12,548] jepsen worker 731 - jepsen.util 731 :invoke :add 478 INFO [2022-01-10 08:39:12,550] jepsen worker 331 - jepsen.util 331 :ok :add 475 INFO [2022-01-10 08:39:12,554] jepsen worker 336 - jepsen.util 336 :invoke :add 479 INFO [2022-01-10 08:39:12,555] jepsen worker 944 - jepsen.util 944 :ok :add 477 INFO [2022-01-10 08:39:12,556] jepsen worker 731 - jepsen.util 731 :ok :add 478 INFO [2022-01-10 08:39:12,558] jepsen worker 914 - jepsen.util 914 :invoke :add 480 INFO [2022-01-10 08:39:12,564] jepsen worker 446 - jepsen.util 446 :invoke :add 481 INFO [2022-01-10 08:39:12,565] jepsen worker 336 - jepsen.util 336 :ok :add 479 INFO [2022-01-10 08:39:12,568] jepsen worker 543 - jepsen.util 543 :invoke :add 482 INFO [2022-01-10 08:39:12,569] jepsen worker 914 - jepsen.util 914 :ok :add 480 INFO [2022-01-10 08:39:12,573] jepsen worker 767 - jepsen.util 767 :invoke :add 483 INFO [2022-01-10 08:39:12,575] jepsen worker 446 - jepsen.util 446 :ok :add 481 INFO [2022-01-10 08:39:12,577] jepsen worker 310 - jepsen.util 310 :ok :add 476 INFO [2022-01-10 08:39:12,578] jepsen worker 454 - jepsen.util 454 :invoke :add 484 INFO [2022-01-10 08:39:12,582] jepsen worker 114 - jepsen.util 114 :invoke :add 485 INFO [2022-01-10 08:39:12,585] jepsen worker 454 - jepsen.util 454 :ok :add 484 INFO [2022-01-10 08:39:12,585] jepsen worker 646 - jepsen.util 646 :invoke :add 486 INFO [2022-01-10 08:39:12,585] jepsen worker 767 - jepsen.util 767 :ok :add 483 INFO [2022-01-10 08:39:12,587] jepsen worker 543 - jepsen.util 543 :ok :add 482 INFO [2022-01-10 08:39:12,592] jepsen worker 646 - jepsen.util 646 :ok :add 486 INFO [2022-01-10 08:39:12,593] jepsen worker 822 - jepsen.util 822 :invoke :add 488 INFO [2022-01-10 08:39:12,593] jepsen worker 69 - jepsen.util 69 :invoke :add 487 INFO [2022-01-10 08:39:12,595] jepsen worker 114 - jepsen.util 114 :ok :add 485 INFO [2022-01-10 08:39:12,596] jepsen worker 927 - jepsen.util 927 :invoke :add 489 INFO [2022-01-10 08:39:12,600] jepsen worker 436 - jepsen.util 436 :invoke :add 490 INFO [2022-01-10 08:39:12,603] jepsen worker 497 - jepsen.util 497 :invoke :add 491 INFO [2022-01-10 08:39:12,604] jepsen worker 927 - jepsen.util 927 :ok :add 489 INFO [2022-01-10 08:39:12,605] jepsen worker 69 - jepsen.util 69 :ok :add 487 INFO [2022-01-10 08:39:12,607] jepsen worker 330 - jepsen.util 330 :invoke :add 492 INFO [2022-01-10 08:39:12,610] jepsen worker 822 - jepsen.util 822 :ok :add 488 INFO [2022-01-10 08:39:12,611] jepsen worker 608 - jepsen.util 608 :invoke :add 493 INFO [2022-01-10 08:39:12,615] jepsen worker 275 - jepsen.util 275 :invoke :add 494 INFO [2022-01-10 08:39:12,616] jepsen worker 436 - jepsen.util 436 :ok :add 490 INFO [2022-01-10 08:39:12,617] jepsen worker 920 - jepsen.util 920 :invoke :add 495 INFO [2022-01-10 08:39:12,618] jepsen worker 497 - jepsen.util 497 :ok :add 491 INFO [2022-01-10 08:39:12,622] jepsen worker 796 - jepsen.util 796 :invoke :add 496 INFO [2022-01-10 08:39:12,622] jepsen worker 608 - jepsen.util 608 :ok :add 493 INFO [2022-01-10 08:39:12,623] jepsen worker 330 - jepsen.util 330 :ok :add 492 INFO [2022-01-10 08:39:12,624] jepsen worker 275 - jepsen.util 275 :ok :add 494 INFO [2022-01-10 08:39:12,625] jepsen worker 850 - jepsen.util 850 :invoke :add 497 INFO [2022-01-10 08:39:12,630] jepsen worker 771 - jepsen.util 771 :invoke :add 498 INFO [2022-01-10 08:39:12,632] jepsen worker 920 - jepsen.util 920 :ok :add 495 INFO [2022-01-10 08:39:12,633] jepsen worker 256 - jepsen.util 256 :invoke :add 499 INFO [2022-01-10 08:39:12,634] jepsen worker 796 - jepsen.util 796 :ok :add 496 INFO [2022-01-10 08:39:12,636] jepsen worker 890 - jepsen.util 890 :invoke :add 500 INFO [2022-01-10 08:39:12,638] jepsen worker 771 - jepsen.util 771 :ok :add 498 INFO [2022-01-10 08:39:12,640] jepsen worker 309 - jepsen.util 309 :invoke :add 501 INFO [2022-01-10 08:39:12,642] jepsen worker 256 - jepsen.util 256 :ok :add 499 INFO [2022-01-10 08:39:12,644] jepsen worker 591 - jepsen.util 591 :invoke :add 502 INFO [2022-01-10 08:39:12,646] jepsen worker 890 - jepsen.util 890 :ok :add 500 INFO [2022-01-10 08:39:12,647] jepsen worker 850 - jepsen.util 850 :ok :add 497 INFO [2022-01-10 08:39:12,648] jepsen worker 10 - jepsen.util 10 :invoke :add 503 INFO [2022-01-10 08:39:12,650] jepsen worker 309 - jepsen.util 309 :ok :add 501 INFO [2022-01-10 08:39:12,652] jepsen worker 728 - jepsen.util 728 :invoke :add 504 INFO [2022-01-10 08:39:12,655] jepsen worker 814 - jepsen.util 814 :invoke :add 505 INFO [2022-01-10 08:39:12,656] jepsen worker 591 - jepsen.util 591 :ok :add 502 INFO [2022-01-10 08:39:12,657] jepsen worker 10 - jepsen.util 10 :ok :add 503 INFO [2022-01-10 08:39:12,660] jepsen worker 241 - jepsen.util 241 :invoke :add 506 INFO [2022-01-10 08:39:12,663] jepsen worker 728 - jepsen.util 728 :ok :add 504 INFO [2022-01-10 08:39:12,666] jepsen worker 428 - jepsen.util 428 :invoke :add 507 INFO [2022-01-10 08:39:12,670] jepsen worker 337 - jepsen.util 337 :invoke :add 508 INFO [2022-01-10 08:39:12,674] jepsen worker 181 - jepsen.util 181 :invoke :add 509 INFO [2022-01-10 08:39:12,678] jepsen worker 576 - jepsen.util 576 :invoke :add 510 INFO [2022-01-10 08:39:12,678] jepsen worker 241 - jepsen.util 241 :ok :add 506 INFO [2022-01-10 08:39:12,681] jepsen worker 337 - jepsen.util 337 :ok :add 508 INFO [2022-01-10 08:39:12,681] jepsen worker 717 - jepsen.util 717 :invoke :add 511 INFO [2022-01-10 08:39:12,683] jepsen worker 181 - jepsen.util 181 :ok :add 509 INFO [2022-01-10 08:39:12,684] jepsen worker 814 - jepsen.util 814 :ok :add 505 INFO [2022-01-10 08:39:12,686] jepsen worker 830 - jepsen.util 830 :invoke :add 512 INFO [2022-01-10 08:39:12,688] jepsen worker 576 - jepsen.util 576 :ok :add 510 INFO [2022-01-10 08:39:12,690] jepsen worker 440 - jepsen.util 440 :invoke :add 513 INFO [2022-01-10 08:39:12,693] jepsen worker 717 - jepsen.util 717 :ok :add 511 INFO [2022-01-10 08:39:12,694] jepsen worker 428 - jepsen.util 428 :ok :add 507 INFO [2022-01-10 08:39:12,695] jepsen worker 628 - jepsen.util 628 :invoke :add 514 INFO [2022-01-10 08:39:12,698] jepsen worker 830 - jepsen.util 830 :ok :add 512 INFO [2022-01-10 08:39:12,700] jepsen worker 440 - jepsen.util 440 :ok :add 513 INFO [2022-01-10 08:39:12,700] jepsen worker 623 - jepsen.util 623 :invoke :add 515 INFO [2022-01-10 08:39:12,704] jepsen worker 628 - jepsen.util 628 :ok :add 514 INFO [2022-01-10 08:39:12,705] jepsen worker 632 - jepsen.util 632 :invoke :add 516 INFO [2022-01-10 08:39:12,709] jepsen worker 445 - jepsen.util 445 :invoke :add 517 INFO [2022-01-10 08:39:12,709] jepsen worker 623 - jepsen.util 623 :ok :add 515 INFO [2022-01-10 08:39:12,712] jepsen worker 832 - jepsen.util 832 :invoke :add 518 INFO [2022-01-10 08:39:12,715] jepsen worker 788 - jepsen.util 788 :invoke :add 519 INFO [2022-01-10 08:39:12,718] jepsen worker 389 - jepsen.util 389 :invoke :add 520 INFO [2022-01-10 08:39:12,719] jepsen worker 832 - jepsen.util 832 :ok :add 518 INFO [2022-01-10 08:39:12,722] jepsen worker 241 - jepsen.util 241 :invoke :add 521 INFO [2022-01-10 08:39:12,723] jepsen worker 788 - jepsen.util 788 :ok :add 519 INFO [2022-01-10 08:39:12,725] jepsen worker 233 - jepsen.util 233 :invoke :add 522 INFO [2022-01-10 08:39:12,729] jepsen worker 722 - jepsen.util 722 :invoke :add 523 INFO [2022-01-10 08:39:12,732] jepsen worker 988 - jepsen.util 988 :invoke :add 524 INFO [2022-01-10 08:39:12,736] jepsen worker 168 - jepsen.util 168 :invoke :add 525 INFO [2022-01-10 08:39:12,737] jepsen worker 445 - jepsen.util 445 :ok :add 517 INFO [2022-01-10 08:39:12,738] jepsen worker 233 - jepsen.util 233 :ok :add 522 INFO [2022-01-10 08:39:12,738] jepsen worker 632 - jepsen.util 632 :ok :add 516 INFO [2022-01-10 08:39:12,740] jepsen worker 160 - jepsen.util 160 :invoke :add 526 INFO [2022-01-10 08:39:12,743] jepsen worker 988 - jepsen.util 988 :ok :add 524 INFO [2022-01-10 08:39:12,744] jepsen worker 389 - jepsen.util 389 :ok :add 520 INFO [2022-01-10 08:39:12,744] jepsen worker 335 - jepsen.util 335 :invoke :add 527 INFO [2022-01-10 08:39:12,745] jepsen worker 168 - jepsen.util 168 :ok :add 525 INFO [2022-01-10 08:39:12,749] jepsen worker 342 - jepsen.util 342 :invoke :add 528 INFO [2022-01-10 08:39:12,751] jepsen worker 241 - jepsen.util 241 :ok :add 521 INFO [2022-01-10 08:39:12,752] jepsen worker 160 - jepsen.util 160 :ok :add 526 INFO [2022-01-10 08:39:12,753] jepsen worker 335 - jepsen.util 335 :ok :add 527 INFO [2022-01-10 08:39:12,753] jepsen worker 462 - jepsen.util 462 :invoke :add 529 INFO [2022-01-10 08:39:12,756] jepsen worker 722 - jepsen.util 722 :ok :add 523 INFO [2022-01-10 08:39:12,759] jepsen worker 342 - jepsen.util 342 :ok :add 528 INFO [2022-01-10 08:39:12,760] jepsen worker 647 - jepsen.util 647 :invoke :add 530 INFO [2022-01-10 08:39:12,765] jepsen worker 462 - jepsen.util 462 :ok :add 529 INFO [2022-01-10 08:39:12,766] jepsen worker 271 - jepsen.util 271 :invoke :add 531 INFO [2022-01-10 08:39:12,769] jepsen worker 313 - jepsen.util 313 :invoke :add 532 INFO [2022-01-10 08:39:12,770] jepsen worker 647 - jepsen.util 647 :ok :add 530 INFO [2022-01-10 08:39:12,773] jepsen worker 185 - jepsen.util 185 :invoke :add 533 INFO [2022-01-10 08:39:12,776] jepsen worker 698 - jepsen.util 698 :invoke :add 534 INFO [2022-01-10 08:39:12,777] jepsen worker 271 - jepsen.util 271 :ok :add 531 INFO [2022-01-10 08:39:12,780] jepsen worker 616 - jepsen.util 616 :invoke :add 535 INFO [2022-01-10 08:39:12,782] jepsen worker 185 - jepsen.util 185 :ok :add 533 INFO [2022-01-10 08:39:12,784] jepsen worker 313 - jepsen.util 313 :ok :add 532 INFO [2022-01-10 08:39:12,785] jepsen worker 92 - jepsen.util 92 :invoke :add 536 INFO [2022-01-10 08:39:12,787] jepsen worker 616 - jepsen.util 616 :ok :add 535 INFO [2022-01-10 08:39:12,791] jepsen worker 885 - jepsen.util 885 :invoke :add 537 INFO [2022-01-10 08:39:12,792] jepsen worker 698 - jepsen.util 698 :ok :add 534 INFO [2022-01-10 08:39:12,796] jepsen worker 463 - jepsen.util 463 :invoke :add 538 INFO [2022-01-10 08:39:12,798] jepsen worker 885 - jepsen.util 885 :ok :add 537 INFO [2022-01-10 08:39:12,799] jepsen worker 980 - jepsen.util 980 :invoke :add 539 INFO [2022-01-10 08:39:12,802] jepsen worker 92 - jepsen.util 92 :ok :add 536 INFO [2022-01-10 08:39:12,803] jepsen worker 370 - jepsen.util 370 :invoke :add 540 INFO [2022-01-10 08:39:12,806] jepsen worker 163 - jepsen.util 163 :invoke :add 541 INFO [2022-01-10 08:39:12,807] jepsen worker 463 - jepsen.util 463 :ok :add 538 INFO [2022-01-10 08:39:12,810] jepsen worker 727 - jepsen.util 727 :invoke :add 542 INFO [2022-01-10 08:39:12,812] jepsen worker 980 - jepsen.util 980 :ok :add 539 INFO [2022-01-10 08:39:12,813] jepsen worker 738 - jepsen.util 738 :invoke :add 543 INFO [2022-01-10 08:39:12,818] jepsen worker 163 - jepsen.util 163 :ok :add 541 INFO [2022-01-10 08:39:12,819] jepsen worker 480 - jepsen.util 480 :invoke :add 544 INFO [2022-01-10 08:39:12,820] jepsen worker 370 - jepsen.util 370 :ok :add 540 INFO [2022-01-10 08:39:12,821] jepsen worker 738 - jepsen.util 738 :ok :add 543 INFO [2022-01-10 08:39:12,824] jepsen worker 60 - jepsen.util 60 :invoke :add 545 INFO [2022-01-10 08:39:12,825] jepsen worker 727 - jepsen.util 727 :ok :add 542 INFO [2022-01-10 08:39:12,827] jepsen worker 480 - jepsen.util 480 :ok :add 544 INFO [2022-01-10 08:39:12,829] jepsen worker 983 - jepsen.util 983 :invoke :add 546 INFO [2022-01-10 08:39:12,833] jepsen worker 284 - jepsen.util 284 :invoke :add 547 INFO [2022-01-10 08:39:12,834] jepsen worker 60 - jepsen.util 60 :ok :add 545 INFO [2022-01-10 08:39:12,836] jepsen worker 353 - jepsen.util 353 :invoke :add 548 INFO [2022-01-10 08:39:12,837] jepsen worker 983 - jepsen.util 983 :ok :add 546 INFO [2022-01-10 08:39:12,841] jepsen worker 568 - jepsen.util 568 :invoke :add 549 INFO [2022-01-10 08:39:12,845] jepsen worker 284 - jepsen.util 284 :ok :add 547 INFO [2022-01-10 08:39:12,845] jepsen worker 731 - jepsen.util 731 :invoke :add 550 INFO [2022-01-10 08:39:12,846] jepsen worker 353 - jepsen.util 353 :ok :add 548 INFO [2022-01-10 08:39:12,849] jepsen worker 955 - jepsen.util 955 :invoke :add 551 INFO [2022-01-10 08:39:12,852] jepsen worker 568 - jepsen.util 568 :ok :add 549 INFO [2022-01-10 08:39:12,852] jepsen worker 948 - jepsen.util 948 :invoke :add 552 INFO [2022-01-10 08:39:12,857] jepsen worker 731 - jepsen.util 731 :ok :add 550 INFO [2022-01-10 08:39:12,857] jepsen worker 591 - jepsen.util 591 :invoke :add 553 INFO [2022-01-10 08:39:12,860] jepsen worker 815 - jepsen.util 815 :invoke :add 554 INFO [2022-01-10 08:39:12,864] jepsen worker 865 - jepsen.util 865 :invoke :add 555 INFO [2022-01-10 08:39:12,865] jepsen worker 955 - jepsen.util 955 :ok :add 551 INFO [2022-01-10 08:39:12,867] jepsen worker 948 - jepsen.util 948 :ok :add 552 INFO [2022-01-10 08:39:12,868] jepsen worker 815 - jepsen.util 815 :ok :add 554 INFO [2022-01-10 08:39:12,869] jepsen worker 843 - jepsen.util 843 :invoke :add 556 INFO [2022-01-10 08:39:12,870] jepsen worker 865 - jepsen.util 865 :ok :add 555 INFO [2022-01-10 08:39:12,871] jepsen worker 591 - jepsen.util 591 :ok :add 553 INFO [2022-01-10 08:39:12,874] jepsen worker 8 - jepsen.util 8 :invoke :add 557 INFO [2022-01-10 08:39:12,879] jepsen worker 843 - jepsen.util 843 :ok :add 556 INFO [2022-01-10 08:39:12,879] jepsen worker 658 - jepsen.util 658 :invoke :add 558 INFO [2022-01-10 08:39:12,882] jepsen worker 8 - jepsen.util 8 :ok :add 557 INFO [2022-01-10 08:39:12,884] jepsen worker 459 - jepsen.util 459 :invoke :add 559 INFO [2022-01-10 08:39:12,889] jepsen worker 387 - jepsen.util 387 :invoke :add 560 INFO [2022-01-10 08:39:12,894] jepsen worker 516 - jepsen.util 516 :invoke :add 561 INFO [2022-01-10 08:39:12,895] jepsen worker 658 - jepsen.util 658 :ok :add 558 INFO [2022-01-10 08:39:12,898] jepsen worker 392 - jepsen.util 392 :invoke :add 562 INFO [2022-01-10 08:39:12,900] jepsen worker 387 - jepsen.util 387 :ok :add 560 INFO [2022-01-10 08:39:12,902] jepsen worker 689 - jepsen.util 689 :invoke :add 563 INFO [2022-01-10 08:39:12,903] jepsen worker 459 - jepsen.util 459 :ok :add 559 INFO [2022-01-10 08:39:12,904] jepsen worker 516 - jepsen.util 516 :ok :add 561 INFO [2022-01-10 08:39:12,906] jepsen worker 725 - jepsen.util 725 :invoke :add 564 INFO [2022-01-10 08:39:12,908] jepsen worker 392 - jepsen.util 392 :ok :add 562 INFO [2022-01-10 08:39:12,910] jepsen worker 463 - jepsen.util 463 :invoke :add 565 INFO [2022-01-10 08:39:12,913] jepsen worker 782 - jepsen.util 782 :invoke :add 566 INFO [2022-01-10 08:39:12,914] jepsen worker 689 - jepsen.util 689 :ok :add 563 INFO [2022-01-10 08:39:12,917] jepsen worker 115 - jepsen.util 115 :invoke :add 567 INFO [2022-01-10 08:39:12,918] jepsen worker 725 - jepsen.util 725 :ok :add 564 INFO [2022-01-10 08:39:12,921] jepsen worker 760 - jepsen.util 760 :invoke :add 568 INFO [2022-01-10 08:39:12,921] jepsen worker 463 - jepsen.util 463 :ok :add 565 INFO [2022-01-10 08:39:12,922] jepsen worker 782 - jepsen.util 782 :ok :add 566 INFO [2022-01-10 08:39:12,925] jepsen worker 765 - jepsen.util 765 :invoke :add 569 INFO [2022-01-10 08:39:12,926] jepsen worker 115 - jepsen.util 115 :ok :add 567 INFO [2022-01-10 08:39:12,929] jepsen worker 283 - jepsen.util 283 :invoke :add 570 INFO [2022-01-10 08:39:12,934] jepsen worker 753 - jepsen.util 753 :invoke :add 571 INFO [2022-01-10 08:39:12,935] jepsen worker 765 - jepsen.util 765 :ok :add 569 INFO [2022-01-10 08:39:12,937] jepsen worker 283 - jepsen.util 283 :ok :add 570 INFO [2022-01-10 08:39:12,937] jepsen worker 957 - jepsen.util 957 :invoke :add 572 INFO [2022-01-10 08:39:12,938] jepsen worker 760 - jepsen.util 760 :ok :add 568 INFO [2022-01-10 08:39:12,942] jepsen worker 795 - jepsen.util 795 :invoke :add 573 INFO [2022-01-10 08:39:12,945] jepsen worker 547 - jepsen.util 547 :invoke :add 574 INFO [2022-01-10 08:39:12,946] jepsen worker 957 - jepsen.util 957 :ok :add 572 INFO [2022-01-10 08:39:12,948] jepsen worker 368 - jepsen.util 368 :invoke :add 575 INFO [2022-01-10 08:39:12,952] jepsen worker 38 - jepsen.util 38 :invoke :add 576 INFO [2022-01-10 08:39:12,955] jepsen worker 753 - jepsen.util 753 :ok :add 571 INFO [2022-01-10 08:39:12,956] jepsen worker 795 - jepsen.util 795 :ok :add 573 INFO [2022-01-10 08:39:12,957] jepsen worker 18 - jepsen.util 18 :invoke :add 577 INFO [2022-01-10 08:39:12,957] jepsen worker 547 - jepsen.util 547 :ok :add 574 INFO [2022-01-10 08:39:12,966] jepsen worker 212 - jepsen.util 212 :invoke :add 579 INFO [2022-01-10 08:39:12,966] jepsen worker 368 - jepsen.util 368 :ok :add 575 INFO [2022-01-10 08:39:12,967] jepsen worker 313 - jepsen.util 313 :invoke :add 578 INFO [2022-01-10 08:39:12,970] jepsen worker 38 - jepsen.util 38 :ok :add 576 INFO [2022-01-10 08:39:12,970] jepsen worker 495 - jepsen.util 495 :invoke :add 580 INFO [2022-01-10 08:39:12,974] jepsen worker 790 - jepsen.util 790 :invoke :add 581 INFO [2022-01-10 08:39:12,976] jepsen worker 18 - jepsen.util 18 :ok :add 577 INFO [2022-01-10 08:39:12,977] jepsen worker 955 - jepsen.util 955 :invoke :add 582 INFO [2022-01-10 08:39:12,978] jepsen worker 212 - jepsen.util 212 :ok :add 579 INFO [2022-01-10 08:39:12,978] jepsen worker 313 - jepsen.util 313 :ok :add 578 INFO [2022-01-10 08:39:12,980] jepsen worker 495 - jepsen.util 495 :ok :add 580 INFO [2022-01-10 08:39:12,983] jepsen worker 898 - jepsen.util 898 :invoke :add 583 INFO [2022-01-10 08:39:12,986] jepsen worker 790 - jepsen.util 790 :ok :add 581 INFO [2022-01-10 08:39:12,987] jepsen worker 331 - jepsen.util 331 :invoke :add 584 INFO [2022-01-10 08:39:12,992] jepsen worker 82 - jepsen.util 82 :invoke :add 585 INFO [2022-01-10 08:39:12,994] jepsen worker 955 - jepsen.util 955 :ok :add 582 INFO [2022-01-10 08:39:12,994] jepsen worker 898 - jepsen.util 898 :ok :add 583 INFO [2022-01-10 08:39:12,996] jepsen worker 868 - jepsen.util 868 :invoke :add 586 INFO [2022-01-10 08:39:12,997] jepsen worker 331 - jepsen.util 331 :ok :add 584 INFO [2022-01-10 08:39:13,001] jepsen worker 356 - jepsen.util 356 :invoke :add 587 INFO [2022-01-10 08:39:13,005] jepsen worker 506 - jepsen.util 506 :invoke :add 588 INFO [2022-01-10 08:39:13,009] jepsen worker 613 - jepsen.util 613 :invoke :add 589 INFO [2022-01-10 08:39:13,012] jepsen worker 630 - jepsen.util 630 :invoke :add 590 INFO [2022-01-10 08:39:13,012] jepsen worker 506 - jepsen.util 506 :ok :add 588 INFO [2022-01-10 08:39:13,016] jepsen worker 613 - jepsen.util 613 :ok :add 589 INFO [2022-01-10 08:39:13,017] jepsen worker 772 - jepsen.util 772 :invoke :add 591 INFO [2022-01-10 08:39:13,020] jepsen worker 630 - jepsen.util 630 :ok :add 590 INFO [2022-01-10 08:39:13,022] jepsen worker 82 - jepsen.util 82 :ok :add 585 INFO [2022-01-10 08:39:13,022] jepsen worker 432 - jepsen.util 432 :invoke :add 592 INFO [2022-01-10 08:39:13,024] jepsen worker 868 - jepsen.util 868 :ok :add 586 INFO [2022-01-10 08:39:13,026] jepsen worker 356 - jepsen.util 356 :ok :add 587 INFO [2022-01-10 08:39:13,026] jepsen worker 332 - jepsen.util 332 :invoke :add 593 INFO [2022-01-10 08:39:13,029] jepsen worker 772 - jepsen.util 772 :ok :add 591 INFO [2022-01-10 08:39:13,030] jepsen worker 338 - jepsen.util 338 :invoke :add 594 INFO [2022-01-10 08:39:13,034] jepsen worker 20 - jepsen.util 20 :invoke :add 595 INFO [2022-01-10 08:39:13,035] jepsen worker 432 - jepsen.util 432 :ok :add 592 INFO [2022-01-10 08:39:13,039] jepsen worker 332 - jepsen.util 332 :ok :add 593 INFO [2022-01-10 08:39:13,039] jepsen worker 338 - jepsen.util 338 :ok :add 594 INFO [2022-01-10 08:39:13,040] jepsen worker 847 - jepsen.util 847 :invoke :add 596 INFO [2022-01-10 08:39:13,042] jepsen worker 910 - jepsen.util 910 :invoke :add 597 INFO [2022-01-10 08:39:13,043] jepsen worker 20 - jepsen.util 20 :ok :add 595 INFO [2022-01-10 08:39:13,045] jepsen worker 961 - jepsen.util 961 :invoke :add 598 INFO [2022-01-10 08:39:13,049] jepsen worker 847 - jepsen.util 847 :ok :add 596 INFO [2022-01-10 08:39:13,050] jepsen worker 815 - jepsen.util 815 :invoke :add 599 INFO [2022-01-10 08:39:13,052] jepsen worker 910 - jepsen.util 910 :ok :add 597 INFO [2022-01-10 08:39:13,054] jepsen worker 517 - jepsen.util 517 :invoke :add 600 INFO [2022-01-10 08:39:13,055] jepsen worker 961 - jepsen.util 961 :ok :add 598 INFO [2022-01-10 08:39:13,058] jepsen worker 776 - jepsen.util 776 :invoke :add 601 INFO [2022-01-10 08:39:13,060] jepsen worker 815 - jepsen.util 815 :ok :add 599 INFO [2022-01-10 08:39:13,063] jepsen worker 517 - jepsen.util 517 :ok :add 600 INFO [2022-01-10 08:39:13,064] jepsen worker 2 - jepsen.util 2 :invoke :add 602 INFO [2022-01-10 08:39:13,067] jepsen worker 776 - jepsen.util 776 :ok :add 601 INFO [2022-01-10 08:39:13,069] jepsen worker 213 - jepsen.util 213 :invoke :add 603 INFO [2022-01-10 08:39:13,073] jepsen worker 2 - jepsen.util 2 :ok :add 602 INFO [2022-01-10 08:39:13,074] jepsen worker 501 - jepsen.util 501 :invoke :add 604 INFO [2022-01-10 08:39:13,078] jepsen worker 213 - jepsen.util 213 :ok :add 603 INFO [2022-01-10 08:39:13,079] jepsen worker 395 - jepsen.util 395 :invoke :add 605 INFO [2022-01-10 08:39:13,083] jepsen worker 136 - jepsen.util 136 :invoke :add 606 INFO [2022-01-10 08:39:13,083] jepsen worker 501 - jepsen.util 501 :ok :add 604 INFO [2022-01-10 08:39:13,085] jepsen worker 395 - jepsen.util 395 :ok :add 605 INFO [2022-01-10 08:39:13,087] jepsen worker 926 - jepsen.util 926 :invoke :add 607 INFO [2022-01-10 08:39:13,093] jepsen worker 952 - jepsen.util 952 :invoke :add 608 INFO [2022-01-10 08:39:13,093] jepsen worker 926 - jepsen.util 926 :ok :add 607 INFO [2022-01-10 08:39:13,094] jepsen worker 136 - jepsen.util 136 :ok :add 606 INFO [2022-01-10 08:39:13,097] jepsen worker 951 - jepsen.util 951 :invoke :add 609 INFO [2022-01-10 08:39:13,102] jepsen worker 681 - jepsen.util 681 :invoke :add 610 INFO [2022-01-10 08:39:13,102] jepsen worker 952 - jepsen.util 952 :ok :add 608 INFO [2022-01-10 08:39:13,104] jepsen worker 951 - jepsen.util 951 :ok :add 609 INFO [2022-01-10 08:39:13,106] jepsen worker 868 - jepsen.util 868 :invoke :add 611 INFO [2022-01-10 08:39:13,109] jepsen worker 681 - jepsen.util 681 :ok :add 610 INFO [2022-01-10 08:39:13,110] jepsen worker 835 - jepsen.util 835 :invoke :add 612 INFO [2022-01-10 08:39:13,113] jepsen worker 868 - jepsen.util 868 :ok :add 611 INFO [2022-01-10 08:39:13,115] jepsen worker 190 - jepsen.util 190 :invoke :add 613 INFO [2022-01-10 08:39:13,118] jepsen worker 835 - jepsen.util 835 :ok :add 612 INFO [2022-01-10 08:39:13,120] jepsen worker 958 - jepsen.util 958 :invoke :add 614 INFO [2022-01-10 08:39:13,122] jepsen worker 190 - jepsen.util 190 :ok :add 613 INFO [2022-01-10 08:39:13,124] jepsen worker 374 - jepsen.util 374 :invoke :add 615 INFO [2022-01-10 08:39:13,128] jepsen worker 958 - jepsen.util 958 :ok :add 614 INFO [2022-01-10 08:39:13,128] jepsen worker 508 - jepsen.util 508 :invoke :add 616 INFO [2022-01-10 08:39:13,132] jepsen worker 732 - jepsen.util 732 :invoke :add 617 INFO [2022-01-10 08:39:13,135] jepsen worker 374 - jepsen.util 374 :ok :add 615 INFO [2022-01-10 08:39:13,136] jepsen worker 999 - jepsen.util 999 :invoke :add 618 INFO [2022-01-10 08:39:13,137] jepsen worker 508 - jepsen.util 508 :ok :add 616 INFO [2022-01-10 08:39:13,139] jepsen worker 202 - jepsen.util 202 :invoke :add 619 INFO [2022-01-10 08:39:13,143] jepsen worker 380 - jepsen.util 380 :invoke :add 620 INFO [2022-01-10 08:39:13,143] jepsen worker 732 - jepsen.util 732 :ok :add 617 INFO [2022-01-10 08:39:13,145] jepsen worker 999 - jepsen.util 999 :ok :add 618 INFO [2022-01-10 08:39:13,147] jepsen worker 13 - jepsen.util 13 :invoke :add 621 INFO [2022-01-10 08:39:13,147] jepsen worker 202 - jepsen.util 202 :ok :add 619 INFO [2022-01-10 08:39:13,152] jepsen worker 823 - jepsen.util 823 :invoke :add 622 INFO [2022-01-10 08:39:13,155] jepsen worker 465 - jepsen.util 465 :invoke :add 623 INFO [2022-01-10 08:39:13,156] jepsen worker 380 - jepsen.util 380 :ok :add 620 INFO [2022-01-10 08:39:13,158] jepsen worker 13 - jepsen.util 13 :ok :add 621 INFO [2022-01-10 08:39:13,158] jepsen worker 994 - jepsen.util 994 :invoke :add 624 INFO [2022-01-10 08:39:13,163] jepsen worker 999 - jepsen.util 999 :invoke :add 625 INFO [2022-01-10 08:39:13,167] jepsen worker 109 - jepsen.util 109 :invoke :add 626 INFO [2022-01-10 08:39:13,167] jepsen worker 994 - jepsen.util 994 :ok :add 624 INFO [2022-01-10 08:39:13,168] jepsen worker 823 - jepsen.util 823 :ok :add 622 INFO [2022-01-10 08:39:13,170] jepsen worker 195 - jepsen.util 195 :invoke :add 627 INFO [2022-01-10 08:39:13,171] jepsen worker 465 - jepsen.util 465 :ok :add 623 INFO [2022-01-10 08:39:13,172] jepsen worker 999 - jepsen.util 999 :ok :add 625 INFO [2022-01-10 08:39:13,175] jepsen worker 841 - jepsen.util 841 :invoke :add 628 INFO [2022-01-10 08:39:13,178] jepsen worker 109 - jepsen.util 109 :ok :add 626 INFO [2022-01-10 08:39:13,179] jepsen worker 641 - jepsen.util 641 :invoke :add 629 INFO [2022-01-10 08:39:13,182] jepsen worker 195 - jepsen.util 195 :ok :add 627 INFO [2022-01-10 08:39:13,182] jepsen worker 841 - jepsen.util 841 :ok :add 628 INFO [2022-01-10 08:39:13,183] jepsen worker 565 - jepsen.util 565 :invoke :add 630 INFO [2022-01-10 08:39:13,188] jepsen worker 677 - jepsen.util 677 :invoke :add 631 INFO [2022-01-10 08:39:13,189] jepsen worker 641 - jepsen.util 641 :ok :add 629 INFO [2022-01-10 08:39:13,192] jepsen worker 655 - jepsen.util 655 :invoke :add 632 INFO [2022-01-10 08:39:13,193] jepsen worker 565 - jepsen.util 565 :ok :add 630 INFO [2022-01-10 08:39:13,196] jepsen worker 25 - jepsen.util 25 :invoke :add 633 INFO [2022-01-10 08:39:13,200] jepsen worker 724 - jepsen.util 724 :invoke :add 634 INFO [2022-01-10 08:39:13,204] jepsen worker 180 - jepsen.util 180 :invoke :add 635 INFO [2022-01-10 08:39:13,207] jepsen worker 724 - jepsen.util 724 :ok :add 634 INFO [2022-01-10 08:39:13,208] jepsen worker 40 - jepsen.util 40 :invoke :add 636 INFO [2022-01-10 08:39:13,213] jepsen worker 180 - jepsen.util 180 :ok :add 635 INFO [2022-01-10 08:39:13,214] jepsen worker 881 - jepsen.util 881 :invoke :add 637 INFO [2022-01-10 08:39:13,216] jepsen worker 40 - jepsen.util 40 :ok :add 636 INFO [2022-01-10 08:39:13,220] jepsen worker 873 - jepsen.util 873 :invoke :add 638 INFO [2022-01-10 08:39:13,222] jepsen worker 881 - jepsen.util 881 :ok :add 637 INFO [2022-01-10 08:39:13,222] jepsen worker 677 - jepsen.util 677 :ok :add 631 INFO [2022-01-10 08:39:13,225] jepsen worker 466 - jepsen.util 466 :invoke :add 639 INFO [2022-01-10 08:39:13,225] jepsen worker 25 - jepsen.util 25 :ok :add 633 INFO [2022-01-10 08:39:13,227] jepsen worker 655 - jepsen.util 655 :ok :add 632 INFO [2022-01-10 08:39:13,230] jepsen worker 873 - jepsen.util 873 :ok :add 638 INFO [2022-01-10 08:39:13,231] jepsen worker 750 - jepsen.util 750 :invoke :add 640 INFO [2022-01-10 08:39:13,237] jepsen worker 750 - jepsen.util 750 :ok :add 640 INFO [2022-01-10 08:39:13,237] jepsen worker 466 - jepsen.util 466 :ok :add 639 INFO [2022-01-10 08:39:13,238] jepsen worker 681 - jepsen.util 681 :invoke :add 641 INFO [2022-01-10 08:39:13,243] jepsen worker 778 - jepsen.util 778 :invoke :add 642 INFO [2022-01-10 08:39:13,244] jepsen worker 681 - jepsen.util 681 :ok :add 641 INFO [2022-01-10 08:39:13,250] jepsen worker 464 - jepsen.util 464 :invoke :add 643 INFO [2022-01-10 08:39:13,250] jepsen worker 778 - jepsen.util 778 :ok :add 642 INFO [2022-01-10 08:39:13,253] jepsen worker 273 - jepsen.util 273 :invoke :add 644 INFO [2022-01-10 08:39:13,256] jepsen worker 464 - jepsen.util 464 :ok :add 643 INFO [2022-01-10 08:39:13,259] jepsen worker 216 - jepsen.util 216 :invoke :add 645 INFO [2022-01-10 08:39:13,262] jepsen worker 273 - jepsen.util 273 :ok :add 644 INFO [2022-01-10 08:39:13,264] jepsen worker 164 - jepsen.util 164 :invoke :add 646 INFO [2022-01-10 08:39:13,267] jepsen worker 216 - jepsen.util 216 :ok :add 645 INFO [2022-01-10 08:39:13,270] jepsen worker 981 - jepsen.util 981 :invoke :add 647 INFO [2022-01-10 08:39:13,272] jepsen worker 164 - jepsen.util 164 :ok :add 646 INFO [2022-01-10 08:39:13,275] jepsen worker 897 - jepsen.util 897 :invoke :add 648 INFO [2022-01-10 08:39:13,277] jepsen worker 981 - jepsen.util 981 :ok :add 647 INFO [2022-01-10 08:39:13,279] jepsen worker 284 - jepsen.util 284 :invoke :add 649 INFO [2022-01-10 08:39:13,283] jepsen worker 897 - jepsen.util 897 :ok :add 648 INFO [2022-01-10 08:39:13,284] jepsen worker 199 - jepsen.util 199 :invoke :add 650 INFO [2022-01-10 08:39:13,287] jepsen worker 284 - jepsen.util 284 :ok :add 649 INFO [2022-01-10 08:39:13,288] jepsen worker 506 - jepsen.util 506 :invoke :add 651 INFO [2022-01-10 08:39:13,292] jepsen worker 682 - jepsen.util 682 :invoke :add 652 INFO [2022-01-10 08:39:13,293] jepsen worker 199 - jepsen.util 199 :ok :add 650 INFO [2022-01-10 08:39:13,295] jepsen worker 504 - jepsen.util 504 :invoke :add 653 INFO [2022-01-10 08:39:13,298] jepsen worker 463 - jepsen.util 463 :invoke :add 654 INFO [2022-01-10 08:39:13,302] jepsen worker 506 - jepsen.util 506 :ok :add 651 INFO [2022-01-10 08:39:13,304] jepsen worker 393 - jepsen.util 393 :invoke :add 655 INFO [2022-01-10 08:39:13,306] jepsen worker 463 - jepsen.util 463 :ok :add 654 INFO [2022-01-10 08:39:13,307] jepsen worker 682 - jepsen.util 682 :ok :add 652 INFO [2022-01-10 08:39:13,309] jepsen worker 557 - jepsen.util 557 :invoke :add 656 INFO [2022-01-10 08:39:13,404] jepsen worker 772 - jepsen.util 772 :invoke :add 657 INFO [2022-01-10 08:39:13,406] jepsen worker 516 - jepsen.util 516 :invoke :add 658 INFO [2022-01-10 08:39:13,408] jepsen worker 393 - jepsen.util 393 :ok :add 655 INFO [2022-01-10 08:39:13,409] jepsen worker 504 - jepsen.util 504 :ok :add 653 INFO [2022-01-10 08:39:13,413] jepsen worker 516 - jepsen.util 516 :ok :add 658 INFO [2022-01-10 08:39:13,413] jepsen worker 606 - jepsen.util 606 :invoke :add 659 INFO [2022-01-10 08:39:13,416] jepsen worker 557 - jepsen.util 557 :ok :add 656 INFO [2022-01-10 08:39:13,416] jepsen worker 772 - jepsen.util 772 :ok :add 657 INFO [2022-01-10 08:39:13,417] jepsen worker 433 - jepsen.util 433 :invoke :add 660 INFO [2022-01-10 08:39:13,420] jepsen worker 606 - jepsen.util 606 :ok :add 659 INFO [2022-01-10 08:39:13,421] jepsen worker 476 - jepsen.util 476 :invoke :add 661 INFO [2022-01-10 08:39:13,424] jepsen worker 856 - jepsen.util 856 :invoke :add 662 INFO [2022-01-10 08:39:13,426] jepsen worker 433 - jepsen.util 433 :ok :add 660 INFO [2022-01-10 08:39:13,427] jepsen worker 315 - jepsen.util 315 :invoke :add 663 INFO [2022-01-10 08:39:13,429] jepsen worker 476 - jepsen.util 476 :ok :add 661 INFO [2022-01-10 08:39:13,430] jepsen worker 230 - jepsen.util 230 :invoke :add 664 INFO [2022-01-10 08:39:13,434] jepsen worker 268 - jepsen.util 268 :invoke :add 665 INFO [2022-01-10 08:39:13,435] jepsen worker 856 - jepsen.util 856 :ok :add 662 INFO [2022-01-10 08:39:13,437] jepsen worker 485 - jepsen.util 485 :invoke :add 666 INFO [2022-01-10 08:39:13,440] jepsen worker 315 - jepsen.util 315 :ok :add 663 INFO [2022-01-10 08:39:13,440] jepsen worker 748 - jepsen.util 748 :invoke :add 667 INFO [2022-01-10 08:39:13,441] jepsen worker 230 - jepsen.util 230 :ok :add 664 INFO [2022-01-10 08:39:13,444] jepsen worker 615 - jepsen.util 615 :invoke :add 668 INFO [2022-01-10 08:39:13,445] jepsen worker 268 - jepsen.util 268 :ok :add 665 INFO [2022-01-10 08:39:13,447] jepsen worker 546 - jepsen.util 546 :invoke :add 669 INFO [2022-01-10 08:39:13,450] jepsen worker 485 - jepsen.util 485 :ok :add 666 INFO [2022-01-10 08:39:13,451] jepsen worker 526 - jepsen.util 526 :invoke :add 670 INFO [2022-01-10 08:39:13,452] jepsen worker 615 - jepsen.util 615 :ok :add 668 INFO [2022-01-10 08:39:13,454] jepsen worker 314 - jepsen.util 314 :invoke :add 671 INFO [2022-01-10 08:39:13,456] jepsen worker 748 - jepsen.util 748 :ok :add 667 INFO [2022-01-10 08:39:13,457] jepsen worker 888 - jepsen.util 888 :invoke :add 672 INFO [2022-01-10 08:39:13,461] jepsen worker 695 - jepsen.util 695 :invoke :add 673 INFO [2022-01-10 08:39:13,463] jepsen worker 526 - jepsen.util 526 :ok :add 670 INFO [2022-01-10 08:39:13,463] jepsen worker 546 - jepsen.util 546 :ok :add 669 INFO [2022-01-10 08:39:13,463] jepsen worker 522 - jepsen.util 522 :invoke :add 674 INFO [2022-01-10 08:39:13,466] jepsen worker 888 - jepsen.util 888 :ok :add 672 INFO [2022-01-10 08:39:13,467] jepsen worker 667 - jepsen.util 667 :invoke :add 675 INFO [2022-01-10 08:39:13,470] jepsen worker 930 - jepsen.util 930 :invoke :add 676 INFO [2022-01-10 08:39:13,471] jepsen worker 522 - jepsen.util 522 :ok :add 674 INFO [2022-01-10 08:39:13,472] jepsen worker 314 - jepsen.util 314 :ok :add 671 INFO [2022-01-10 08:39:13,473] jepsen worker 958 - jepsen.util 958 :invoke :add 677 INFO [2022-01-10 08:39:13,475] jepsen worker 695 - jepsen.util 695 :ok :add 673 INFO [2022-01-10 08:39:13,478] jepsen worker 556 - jepsen.util 556 :invoke :add 678 INFO [2022-01-10 08:39:13,478] jepsen worker 667 - jepsen.util 667 :ok :add 675 INFO [2022-01-10 08:39:13,481] jepsen worker 32 - jepsen.util 32 :invoke :add 679 INFO [2022-01-10 08:39:13,481] jepsen worker 958 - jepsen.util 958 :ok :add 677 INFO [2022-01-10 08:39:13,483] jepsen worker 930 - jepsen.util 930 :ok :add 676 INFO [2022-01-10 08:39:13,484] jepsen worker 738 - jepsen.util 738 :invoke :add 680 INFO [2022-01-10 08:39:13,486] jepsen worker 510 - jepsen.util 510 :invoke :add 681 INFO [2022-01-10 08:39:13,487] jepsen worker 556 - jepsen.util 556 :ok :add 678 INFO [2022-01-10 08:39:13,490] jepsen worker 854 - jepsen.util 854 :invoke :add 682 INFO [2022-01-10 08:39:13,493] jepsen worker 280 - jepsen.util 280 :invoke :add 683 INFO [2022-01-10 08:39:13,495] jepsen worker 32 - jepsen.util 32 :ok :add 679 INFO [2022-01-10 08:39:13,496] jepsen worker 306 - jepsen.util 306 :invoke :add 684 INFO [2022-01-10 08:39:13,499] jepsen worker 403 - jepsen.util 403 :invoke :add 685 INFO [2022-01-10 08:39:13,499] jepsen worker 510 - jepsen.util 510 :ok :add 681 INFO [2022-01-10 08:39:13,500] jepsen worker 738 - jepsen.util 738 :ok :add 680 INFO [2022-01-10 08:39:13,502] jepsen worker 780 - jepsen.util 780 :invoke :add 686 INFO [2022-01-10 08:39:13,503] jepsen worker 280 - jepsen.util 280 :ok :add 683 INFO [2022-01-10 08:39:13,505] jepsen worker 877 - jepsen.util 877 :invoke :add 687 INFO [2022-01-10 08:39:13,505] jepsen worker 306 - jepsen.util 306 :ok :add 684 INFO [2022-01-10 08:39:13,508] jepsen worker 63 - jepsen.util 63 :invoke :add 688 INFO [2022-01-10 08:39:13,510] jepsen worker 403 - jepsen.util 403 :ok :add 685 INFO [2022-01-10 08:39:13,510] jepsen worker 854 - jepsen.util 854 :ok :add 682 INFO [2022-01-10 08:39:13,510] jepsen worker 230 - jepsen.util 230 :invoke :add 689 INFO [2022-01-10 08:39:13,514] jepsen worker 877 - jepsen.util 877 :ok :add 687 INFO [2022-01-10 08:39:13,514] jepsen worker 780 - jepsen.util 780 :ok :add 686 INFO [2022-01-10 08:39:13,514] jepsen worker 774 - jepsen.util 774 :invoke :add 690 INFO [2022-01-10 08:39:13,516] jepsen worker 63 - jepsen.util 63 :ok :add 688 INFO [2022-01-10 08:39:13,520] jepsen worker 555 - jepsen.util 555 :invoke :add 691 INFO [2022-01-10 08:39:13,521] jepsen worker 230 - jepsen.util 230 :ok :add 689 INFO [2022-01-10 08:39:13,522] jepsen worker 774 - jepsen.util 774 :ok :add 690 INFO [2022-01-10 08:39:13,523] jepsen worker 250 - jepsen.util 250 :invoke :add 692 INFO [2022-01-10 08:39:13,526] jepsen worker 105 - jepsen.util 105 :invoke :add 693 INFO [2022-01-10 08:39:13,528] jepsen worker 36 - jepsen.util 36 :invoke :add 694 INFO [2022-01-10 08:39:13,529] jepsen worker 555 - jepsen.util 555 :ok :add 691 INFO [2022-01-10 08:39:13,531] jepsen worker 626 - jepsen.util 626 :invoke :add 695 INFO [2022-01-10 08:39:13,532] jepsen worker 250 - jepsen.util 250 :ok :add 692 INFO [2022-01-10 08:39:13,534] jepsen worker 452 - jepsen.util 452 :invoke :add 696 INFO [2022-01-10 08:39:13,535] jepsen worker 105 - jepsen.util 105 :ok :add 693 INFO [2022-01-10 08:39:13,536] jepsen worker 179 - jepsen.util 179 :invoke :add 697 INFO [2022-01-10 08:39:13,539] jepsen worker 36 - jepsen.util 36 :ok :add 694 INFO [2022-01-10 08:39:13,541] jepsen worker 626 - jepsen.util 626 :ok :add 695 INFO [2022-01-10 08:39:13,542] jepsen worker 116 - jepsen.util 116 :invoke :add 698 INFO [2022-01-10 08:39:13,546] jepsen worker 953 - jepsen.util 953 :invoke :add 699 INFO [2022-01-10 08:39:13,548] jepsen worker 179 - jepsen.util 179 :ok :add 697 INFO [2022-01-10 08:39:13,550] jepsen worker 116 - jepsen.util 116 :ok :add 698 INFO [2022-01-10 08:39:13,550] jepsen worker 452 - jepsen.util 452 :ok :add 696 INFO [2022-01-10 08:39:13,550] jepsen worker 118 - jepsen.util 118 :invoke :add 700 INFO [2022-01-10 08:39:13,555] jepsen worker 953 - jepsen.util 953 :ok :add 699 INFO [2022-01-10 08:39:13,555] jepsen worker 371 - jepsen.util 371 :invoke :add 701 INFO [2022-01-10 08:39:13,556] jepsen worker 118 - jepsen.util 118 :ok :add 700 INFO [2022-01-10 08:39:13,558] jepsen worker 550 - jepsen.util 550 :invoke :add 702 INFO [2022-01-10 08:39:13,561] jepsen worker 746 - jepsen.util 746 :invoke :add 703 INFO [2022-01-10 08:39:13,561] jepsen worker 371 - jepsen.util 371 :ok :add 701 INFO [2022-01-10 08:39:13,563] jepsen worker 861 - jepsen.util 861 :invoke :add 704 INFO [2022-01-10 08:39:13,567] jepsen worker 238 - jepsen.util 238 :invoke :add 705 INFO [2022-01-10 08:39:13,568] jepsen worker 746 - jepsen.util 746 :ok :add 703 INFO [2022-01-10 08:39:13,569] jepsen worker 550 - jepsen.util 550 :ok :add 702 INFO [2022-01-10 08:39:13,569] jepsen worker 893 - jepsen.util 893 :invoke :add 706 INFO [2022-01-10 08:39:13,570] jepsen worker 861 - jepsen.util 861 :ok :add 704 INFO [2022-01-10 08:39:13,573] jepsen worker 213 - jepsen.util 213 :invoke :add 707 INFO [2022-01-10 08:39:13,576] jepsen worker 762 - jepsen.util 762 :invoke :add 708 INFO [2022-01-10 08:39:13,577] jepsen worker 238 - jepsen.util 238 :ok :add 705 INFO [2022-01-10 08:39:13,579] jepsen worker 154 - jepsen.util 154 :invoke :add 709 INFO [2022-01-10 08:39:13,581] jepsen worker 141 - jepsen.util 141 :invoke :add 710 INFO [2022-01-10 08:39:13,584] jepsen worker 520 - jepsen.util 520 :invoke :add 711 INFO [2022-01-10 08:39:13,587] jepsen worker 534 - jepsen.util 534 :invoke :add 712 INFO [2022-01-10 08:39:13,587] jepsen worker 762 - jepsen.util 762 :ok :add 708 INFO [2022-01-10 08:39:13,588] jepsen worker 154 - jepsen.util 154 :ok :add 709 INFO [2022-01-10 08:39:13,590] jepsen worker 893 - jepsen.util 893 :ok :add 706 INFO [2022-01-10 08:39:13,590] jepsen worker 213 - jepsen.util 213 :ok :add 707 INFO [2022-01-10 08:39:13,591] jepsen worker 917 - jepsen.util 917 :invoke :add 713 INFO [2022-01-10 08:39:13,594] jepsen worker 847 - jepsen.util 847 :invoke :add 714 INFO [2022-01-10 08:39:13,595] jepsen worker 141 - jepsen.util 141 :ok :add 710 INFO [2022-01-10 08:39:13,597] jepsen worker 736 - jepsen.util 736 :invoke :add 715 INFO [2022-01-10 08:39:13,599] jepsen worker 160 - jepsen.util 160 :invoke :add 716 INFO [2022-01-10 08:39:13,601] jepsen worker 520 - jepsen.util 520 :ok :add 711 INFO [2022-01-10 08:39:13,603] jepsen worker 729 - jepsen.util 729 :invoke :add 717 INFO [2022-01-10 08:39:13,606] jepsen worker 534 - jepsen.util 534 :ok :add 712 INFO [2022-01-10 08:39:13,606] jepsen worker 847 - jepsen.util 847 :ok :add 714 INFO [2022-01-10 08:39:13,607] jepsen worker 736 - jepsen.util 736 :ok :add 715 INFO [2022-01-10 08:39:13,609] jepsen worker 723 - jepsen.util 723 :invoke :add 719 INFO [2022-01-10 08:39:13,609] jepsen worker 590 - jepsen.util 590 :invoke :add 718 INFO [2022-01-10 08:39:13,610] jepsen worker 917 - jepsen.util 917 :ok :add 713 INFO [2022-01-10 08:39:13,612] jepsen worker 729 - jepsen.util 729 :ok :add 717 INFO [2022-01-10 08:39:13,612] jepsen worker 265 - jepsen.util 265 :invoke :add 720 INFO [2022-01-10 08:39:13,615] jepsen worker 671 - jepsen.util 671 :invoke :add 721 INFO [2022-01-10 08:39:13,617] jepsen worker 184 - jepsen.util 184 :invoke :add 722 INFO [2022-01-10 08:39:13,619] jepsen worker 723 - jepsen.util 723 :ok :add 719 INFO [2022-01-10 08:39:13,620] jepsen worker 754 - jepsen.util 754 :invoke :add 723 INFO [2022-01-10 08:39:13,622] jepsen worker 517 - jepsen.util 517 :invoke :add 724 INFO [2022-01-10 08:39:13,622] jepsen worker 590 - jepsen.util 590 :ok :add 718 INFO [2022-01-10 08:39:13,624] jepsen worker 160 - jepsen.util 160 :ok :add 716 INFO [2022-01-10 08:39:13,625] jepsen worker 597 - jepsen.util 597 :invoke :add 725 INFO [2022-01-10 08:39:13,627] jepsen worker 671 - jepsen.util 671 :ok :add 721 INFO [2022-01-10 08:39:13,628] jepsen worker 265 - jepsen.util 265 :ok :add 720 INFO [2022-01-10 08:39:13,629] jepsen worker 465 - jepsen.util 465 :invoke :add 726 INFO [2022-01-10 08:39:13,632] jepsen worker 940 - jepsen.util 940 :invoke :add 727 INFO [2022-01-10 08:39:13,634] jepsen worker 754 - jepsen.util 754 :ok :add 723 INFO [2022-01-10 08:39:13,634] jepsen worker 740 - jepsen.util 740 :invoke :add 728 INFO [2022-01-10 08:39:13,635] jepsen worker 184 - jepsen.util 184 :ok :add 722 INFO [2022-01-10 08:39:13,637] jepsen worker 473 - jepsen.util 473 :invoke :add 729 INFO [2022-01-10 08:39:13,642] jepsen worker 716 - jepsen.util 716 :invoke :add 730 INFO [2022-01-10 08:39:13,645] jepsen worker 740 - jepsen.util 740 :ok :add 728 INFO [2022-01-10 08:39:13,646] jepsen worker 473 - jepsen.util 473 :ok :add 729 INFO [2022-01-10 08:39:13,646] jepsen worker 978 - jepsen.util 978 :invoke :add 731 INFO [2022-01-10 08:39:13,650] jepsen worker 834 - jepsen.util 834 :invoke :add 732 INFO [2022-01-10 08:39:13,650] jepsen worker 597 - jepsen.util 597 :ok :add 725 INFO [2022-01-10 08:39:13,650] jepsen worker 517 - jepsen.util 517 :ok :add 724 INFO [2022-01-10 08:39:13,651] jepsen worker 716 - jepsen.util 716 :ok :add 730 INFO [2022-01-10 08:39:13,653] jepsen worker 168 - jepsen.util 168 :invoke :add 733 INFO [2022-01-10 08:39:13,654] jepsen worker 940 - jepsen.util 940 :ok :add 727 INFO [2022-01-10 08:39:13,654] jepsen worker 978 - jepsen.util 978 :ok :add 731 INFO [2022-01-10 08:39:13,655] jepsen worker 465 - jepsen.util 465 :ok :add 726 INFO [2022-01-10 08:39:13,657] jepsen worker 938 - jepsen.util 938 :invoke :add 734 INFO [2022-01-10 08:39:13,660] jepsen worker 139 - jepsen.util 139 :invoke :add 735 INFO [2022-01-10 08:39:13,662] jepsen worker 168 - jepsen.util 168 :ok :add 733 INFO [2022-01-10 08:39:13,663] jepsen worker 738 - jepsen.util 738 :invoke :add 736 INFO [2022-01-10 08:39:13,666] jepsen worker 345 - jepsen.util 345 :invoke :add 737 INFO [2022-01-10 08:39:13,667] jepsen worker 834 - jepsen.util 834 :ok :add 732 INFO [2022-01-10 08:39:13,669] jepsen worker 116 - jepsen.util 116 :invoke :add 738 INFO [2022-01-10 08:39:13,670] jepsen worker 139 - jepsen.util 139 :ok :add 735 INFO [2022-01-10 08:39:13,671] jepsen worker 938 - jepsen.util 938 :ok :add 734 INFO [2022-01-10 08:39:13,672] jepsen worker 241 - jepsen.util 241 :invoke :add 739 INFO [2022-01-10 08:39:13,675] jepsen worker 738 - jepsen.util 738 :ok :add 736 INFO [2022-01-10 08:39:13,676] jepsen worker 372 - jepsen.util 372 :invoke :add 740 INFO [2022-01-10 08:39:13,677] jepsen worker 116 - jepsen.util 116 :ok :add 738 INFO [2022-01-10 08:39:13,679] jepsen worker 826 - jepsen.util 826 :invoke :add 741 INFO [2022-01-10 08:39:13,681] jepsen worker 241 - jepsen.util 241 :ok :add 739 INFO [2022-01-10 08:39:13,682] jepsen worker 372 - jepsen.util 372 :ok :add 740 INFO [2022-01-10 08:39:13,683] jepsen worker 126 - jepsen.util 126 :invoke :add 742 INFO [2022-01-10 08:39:13,686] jepsen worker 789 - jepsen.util 789 :invoke :add 743 INFO [2022-01-10 08:39:13,686] jepsen worker 826 - jepsen.util 826 :ok :add 741 INFO [2022-01-10 08:39:13,687] jepsen worker 345 - jepsen.util 345 :ok :add 737 INFO [2022-01-10 08:39:13,689] jepsen worker 708 - jepsen.util 708 :invoke :add 744 INFO [2022-01-10 08:39:13,693] jepsen worker 837 - jepsen.util 837 :invoke :add 745 INFO [2022-01-10 08:39:13,694] jepsen worker 126 - jepsen.util 126 :ok :add 742 INFO [2022-01-10 08:39:13,696] jepsen worker 667 - jepsen.util 667 :invoke :add 746 INFO [2022-01-10 08:39:13,696] jepsen worker 789 - jepsen.util 789 :ok :add 743 INFO [2022-01-10 08:39:13,696] jepsen worker 708 - jepsen.util 708 :ok :add 744 INFO [2022-01-10 08:39:13,699] jepsen worker 234 - jepsen.util 234 :invoke :add 747 INFO [2022-01-10 08:39:13,702] jepsen worker 580 - jepsen.util 580 :invoke :add 748 INFO [2022-01-10 08:39:13,704] jepsen worker 667 - jepsen.util 667 :ok :add 746 INFO [2022-01-10 08:39:13,705] jepsen worker 360 - jepsen.util 360 :invoke :add 749 INFO [2022-01-10 08:39:13,708] jepsen worker 854 - jepsen.util 854 :invoke :add 750 INFO [2022-01-10 08:39:13,709] jepsen worker 837 - jepsen.util 837 :ok :add 745 INFO [2022-01-10 08:39:13,710] jepsen worker 580 - jepsen.util 580 :ok :add 748 INFO [2022-01-10 08:39:13,711] jepsen worker 888 - jepsen.util 888 :invoke :add 751 INFO [2022-01-10 08:39:13,714] jepsen worker 808 - jepsen.util 808 :invoke :add 752 INFO [2022-01-10 08:39:13,715] jepsen worker 234 - jepsen.util 234 :ok :add 747 INFO [2022-01-10 08:39:13,716] jepsen worker 360 - jepsen.util 360 :ok :add 749 INFO [2022-01-10 08:39:13,716] jepsen worker 893 - jepsen.util 893 :invoke :add 753 INFO [2022-01-10 08:39:13,719] jepsen worker 882 - jepsen.util 882 :invoke :add 754 INFO [2022-01-10 08:39:13,722] jepsen worker 854 - jepsen.util 854 :ok :add 750 INFO [2022-01-10 08:39:13,722] jepsen worker 951 - jepsen.util 951 :invoke :add 755 INFO [2022-01-10 08:39:13,723] jepsen worker 888 - jepsen.util 888 :ok :add 751 INFO [2022-01-10 08:39:13,725] jepsen worker 521 - jepsen.util 521 :invoke :add 756 INFO [2022-01-10 08:39:13,726] jepsen worker 808 - jepsen.util 808 :ok :add 752 INFO [2022-01-10 08:39:13,727] jepsen worker 882 - jepsen.util 882 :ok :add 754 INFO [2022-01-10 08:39:13,728] jepsen worker 859 - jepsen.util 859 :invoke :add 757 INFO [2022-01-10 08:39:13,729] jepsen worker 893 - jepsen.util 893 :ok :add 753 INFO [2022-01-10 08:39:13,732] jepsen worker 578 - jepsen.util 578 :invoke :add 758 INFO [2022-01-10 08:39:13,734] jepsen worker 309 - jepsen.util 309 :invoke :add 759 INFO [2022-01-10 08:39:13,736] jepsen worker 888 - jepsen.util 888 :invoke :add 760 INFO [2022-01-10 08:39:13,738] jepsen worker 195 - jepsen.util 195 :invoke :add 761 INFO [2022-01-10 08:39:13,739] jepsen worker 951 - jepsen.util 951 :ok :add 755 INFO [2022-01-10 08:39:13,739] jepsen worker 521 - jepsen.util 521 :ok :add 756 INFO [2022-01-10 08:39:13,740] jepsen worker 578 - jepsen.util 578 :ok :add 758 INFO [2022-01-10 08:39:13,741] jepsen worker 479 - jepsen.util 479 :invoke :add 762 INFO [2022-01-10 08:39:13,741] jepsen worker 309 - jepsen.util 309 :ok :add 759 INFO [2022-01-10 08:39:13,743] jepsen worker 165 - jepsen.util 165 :invoke :add 763 INFO [2022-01-10 08:39:13,744] jepsen worker 859 - jepsen.util 859 :ok :add 757 INFO [2022-01-10 08:39:13,746] jepsen worker 354 - jepsen.util 354 :invoke :add 764 INFO [2022-01-10 08:39:13,748] jepsen worker 219 - jepsen.util 219 :invoke :add 765 INFO [2022-01-10 08:39:13,751] jepsen worker 888 - jepsen.util 888 :ok :add 760 INFO [2022-01-10 08:39:13,751] jepsen worker 355 - jepsen.util 355 :invoke :add 766 INFO [2022-01-10 08:39:13,753] jepsen worker 195 - jepsen.util 195 :ok :add 761 INFO [2022-01-10 08:39:13,755] jepsen worker 600 - jepsen.util 600 :invoke :add 767 INFO [2022-01-10 08:39:13,755] jepsen worker 219 - jepsen.util 219 :ok :add 765 INFO [2022-01-10 08:39:13,757] jepsen worker 985 - jepsen.util 985 :invoke :add 768 INFO [2022-01-10 08:39:13,758] jepsen worker 479 - jepsen.util 479 :ok :add 762 INFO [2022-01-10 08:39:13,759] jepsen worker 165 - jepsen.util 165 :ok :add 763 INFO [2022-01-10 08:39:13,761] jepsen worker 566 - jepsen.util 566 :invoke :add 769 INFO [2022-01-10 08:39:13,763] jepsen worker 519 - jepsen.util 519 :invoke :add 770 INFO [2022-01-10 08:39:13,765] jepsen worker 354 - jepsen.util 354 :ok :add 764 INFO [2022-01-10 08:39:13,765] jepsen worker 785 - jepsen.util 785 :invoke :add 771 INFO [2022-01-10 08:39:13,765] jepsen worker 985 - jepsen.util 985 :ok :add 768 INFO [2022-01-10 08:39:13,769] jepsen worker 272 - jepsen.util 272 :invoke :add 772 INFO [2022-01-10 08:39:13,769] jepsen worker 600 - jepsen.util 600 :ok :add 767 INFO [2022-01-10 08:39:13,769] jepsen worker 566 - jepsen.util 566 :ok :add 769 INFO [2022-01-10 08:39:13,771] jepsen worker 355 - jepsen.util 355 :ok :add 766 INFO [2022-01-10 08:39:13,772] jepsen worker 770 - jepsen.util 770 :invoke :add 773 INFO [2022-01-10 08:39:13,775] jepsen worker 519 - jepsen.util 519 :ok :add 770 INFO [2022-01-10 08:39:13,775] jepsen worker 785 - jepsen.util 785 :ok :add 771 INFO [2022-01-10 08:39:13,776] jepsen worker 804 - jepsen.util 804 :invoke :add 774 INFO [2022-01-10 08:39:13,778] jepsen worker 651 - jepsen.util 651 :invoke :add 775 INFO [2022-01-10 08:39:13,780] jepsen worker 770 - jepsen.util 770 :ok :add 773 INFO [2022-01-10 08:39:13,780] jepsen worker 581 - jepsen.util 581 :invoke :add 776 INFO [2022-01-10 08:39:13,782] jepsen worker 272 - jepsen.util 272 :ok :add 772 INFO [2022-01-10 08:39:13,783] jepsen worker 195 - jepsen.util 195 :invoke :add 777 INFO [2022-01-10 08:39:13,786] jepsen worker 45 - jepsen.util 45 :invoke :add 778 INFO [2022-01-10 08:39:13,788] jepsen worker 645 - jepsen.util 645 :invoke :add 779 INFO [2022-01-10 08:39:13,789] jepsen worker 651 - jepsen.util 651 :ok :add 775 INFO [2022-01-10 08:39:13,790] jepsen worker 804 - jepsen.util 804 :ok :add 774 INFO [2022-01-10 08:39:13,791] jepsen worker 495 - jepsen.util 495 :invoke :add 780 INFO [2022-01-10 08:39:13,793] jepsen worker 195 - jepsen.util 195 :ok :add 777 INFO [2022-01-10 08:39:13,795] jepsen worker 638 - jepsen.util 638 :invoke :add 781 INFO [2022-01-10 08:39:13,795] jepsen worker 45 - jepsen.util 45 :ok :add 778 INFO [2022-01-10 08:39:13,797] jepsen worker 581 - jepsen.util 581 :ok :add 776 INFO [2022-01-10 08:39:13,797] jepsen worker 474 - jepsen.util 474 :invoke :add 782 INFO [2022-01-10 08:39:13,800] jepsen worker 202 - jepsen.util 202 :invoke :add 783 INFO [2022-01-10 08:39:13,802] jepsen worker 261 - jepsen.util 261 :invoke :add 784 INFO [2022-01-10 08:39:13,803] jepsen worker 638 - jepsen.util 638 :ok :add 781 INFO [2022-01-10 08:39:13,805] jepsen worker 645 - jepsen.util 645 :ok :add 779 INFO [2022-01-10 08:39:13,805] jepsen worker 853 - jepsen.util 853 :invoke :add 785 INFO [2022-01-10 08:39:13,807] jepsen worker 202 - jepsen.util 202 :ok :add 783 INFO [2022-01-10 08:39:13,809] jepsen worker 198 - jepsen.util 198 :invoke :add 786 INFO [2022-01-10 08:39:13,810] jepsen worker 495 - jepsen.util 495 :ok :add 780 INFO [2022-01-10 08:39:13,811] jepsen worker 261 - jepsen.util 261 :ok :add 784 INFO [2022-01-10 08:39:13,813] jepsen worker 918 - jepsen.util 918 :invoke :add 787 INFO [2022-01-10 08:39:13,815] jepsen worker 853 - jepsen.util 853 :ok :add 785 INFO [2022-01-10 08:39:13,816] jepsen worker 547 - jepsen.util 547 :invoke :add 788 INFO [2022-01-10 08:39:13,818] jepsen worker 715 - jepsen.util 715 :invoke :add 789 INFO [2022-01-10 08:39:13,820] jepsen worker 654 - jepsen.util 654 :invoke :add 790 INFO [2022-01-10 08:39:13,823] jepsen worker 198 - jepsen.util 198 :ok :add 786 INFO [2022-01-10 08:39:13,823] jepsen worker 918 - jepsen.util 918 :ok :add 787 INFO [2022-01-10 08:39:13,823] jepsen worker 629 - jepsen.util 629 :invoke :add 791 INFO [2022-01-10 08:39:13,824] jepsen worker 547 - jepsen.util 547 :ok :add 788 INFO [2022-01-10 08:39:13,827] jepsen worker 697 - jepsen.util 697 :invoke :add 792 INFO [2022-01-10 08:39:13,829] jepsen worker 654 - jepsen.util 654 :ok :add 790 INFO [2022-01-10 08:39:13,829] jepsen worker 744 - jepsen.util 744 :invoke :add 793 INFO [2022-01-10 08:39:13,833] jepsen worker 406 - jepsen.util 406 :invoke :add 794 INFO [2022-01-10 08:39:13,835] jepsen worker 715 - jepsen.util 715 :ok :add 789 INFO [2022-01-10 08:39:13,836] jepsen worker 629 - jepsen.util 629 :ok :add 791 INFO [2022-01-10 08:39:13,837] jepsen worker 914 - jepsen.util 914 :invoke :add 795 INFO [2022-01-10 08:39:13,837] jepsen worker 474 - jepsen.util 474 :ok :add 782 INFO [2022-01-10 08:39:13,839] jepsen worker 697 - jepsen.util 697 :ok :add 792 INFO [2022-01-10 08:39:13,839] jepsen worker 744 - jepsen.util 744 :ok :add 793 INFO [2022-01-10 08:39:13,841] jepsen worker 451 - jepsen.util 451 :invoke :add 796 INFO [2022-01-10 08:39:13,844] jepsen worker 406 - jepsen.util 406 :ok :add 794 INFO [2022-01-10 08:39:13,845] jepsen worker 644 - jepsen.util 644 :invoke :add 797 INFO [2022-01-10 08:39:13,848] jepsen worker 914 - jepsen.util 914 :ok :add 795 INFO [2022-01-10 08:39:13,848] jepsen worker 946 - jepsen.util 946 :invoke :add 798 INFO [2022-01-10 08:39:13,851] jepsen worker 242 - jepsen.util 242 :invoke :add 799 INFO [2022-01-10 08:39:13,853] jepsen worker 754 - jepsen.util 754 :invoke :add 800 INFO [2022-01-10 08:39:13,856] jepsen worker 312 - jepsen.util 312 :invoke :add 801 INFO [2022-01-10 08:39:13,858] jepsen worker 415 - jepsen.util 415 :invoke :add 802 INFO [2022-01-10 08:39:13,859] jepsen worker 451 - jepsen.util 451 :ok :add 796 INFO [2022-01-10 08:39:13,859] jepsen worker 644 - jepsen.util 644 :ok :add 797 INFO [2022-01-10 08:39:13,860] jepsen worker 678 - jepsen.util 678 :invoke :add 803 INFO [2022-01-10 08:39:13,861] jepsen worker 946 - jepsen.util 946 :ok :add 798 INFO [2022-01-10 08:39:13,861] jepsen worker 242 - jepsen.util 242 :ok :add 799 INFO [2022-01-10 08:39:13,863] jepsen worker 470 - jepsen.util 470 :invoke :add 804 INFO [2022-01-10 08:39:13,866] jepsen worker 50 - jepsen.util 50 :invoke :add 805 INFO [2022-01-10 08:39:13,868] jepsen worker 312 - jepsen.util 312 :ok :add 801 INFO [2022-01-10 08:39:13,869] jepsen worker 754 - jepsen.util 754 :ok :add 800 INFO [2022-01-10 08:39:13,869] jepsen worker 874 - jepsen.util 874 :invoke :add 806 INFO [2022-01-10 08:39:13,872] jepsen worker 470 - jepsen.util 470 :ok :add 804 INFO [2022-01-10 08:39:13,873] jepsen worker 766 - jepsen.util 766 :invoke :add 807 INFO [2022-01-10 08:39:13,873] jepsen worker 415 - jepsen.util 415 :ok :add 802 INFO [2022-01-10 08:39:13,875] jepsen worker 881 - jepsen.util 881 :invoke :add 808 INFO [2022-01-10 08:39:13,878] jepsen worker 930 - jepsen.util 930 :invoke :add 809 INFO [2022-01-10 08:39:13,881] jepsen worker 874 - jepsen.util 874 :ok :add 806 INFO [2022-01-10 08:39:13,882] jepsen worker 881 - jepsen.util 881 :ok :add 808 INFO [2022-01-10 08:39:13,882] jepsen worker 678 - jepsen.util 678 :ok :add 803 INFO [2022-01-10 08:39:13,882] jepsen worker 257 - jepsen.util 257 :invoke :add 810 INFO [2022-01-10 08:39:13,883] jepsen worker 50 - jepsen.util 50 :ok :add 805 INFO [2022-01-10 08:39:13,886] jepsen worker 930 - jepsen.util 930 :ok :add 809 INFO [2022-01-10 08:39:13,887] jepsen worker 232 - jepsen.util 232 :invoke :add 811 INFO [2022-01-10 08:39:13,888] jepsen worker 766 - jepsen.util 766 :ok :add 807 INFO [2022-01-10 08:39:13,889] jepsen worker 134 - jepsen.util 134 :invoke :add 812 INFO [2022-01-10 08:39:13,892] jepsen worker 193 - jepsen.util 193 :invoke :add 813 INFO [2022-01-10 08:39:13,894] jepsen worker 767 - jepsen.util 767 :invoke :add 814 INFO [2022-01-10 08:39:13,896] jepsen worker 257 - jepsen.util 257 :ok :add 810 INFO [2022-01-10 08:39:13,897] jepsen worker 372 - jepsen.util 372 :invoke :add 815 INFO [2022-01-10 08:39:13,899] jepsen worker 821 - jepsen.util 821 :invoke :add 816 INFO [2022-01-10 08:39:13,899] jepsen worker 134 - jepsen.util 134 :ok :add 812 INFO [2022-01-10 08:39:13,901] jepsen worker 296 - jepsen.util 296 :invoke :add 817 INFO [2022-01-10 08:39:13,904] jepsen worker 642 - jepsen.util 642 :invoke :add 818 INFO [2022-01-10 08:39:13,906] jepsen worker 767 - jepsen.util 767 :ok :add 814 INFO [2022-01-10 08:39:13,906] jepsen worker 232 - jepsen.util 232 :ok :add 811 INFO [2022-01-10 08:39:13,906] jepsen worker 630 - jepsen.util 630 :invoke :add 819 INFO [2022-01-10 08:39:13,909] jepsen worker 193 - jepsen.util 193 :ok :add 813 INFO [2022-01-10 08:39:13,909] jepsen worker 372 - jepsen.util 372 :ok :add 815 INFO [2022-01-10 08:39:13,909] jepsen worker 413 - jepsen.util 413 :invoke :add 820 INFO [2022-01-10 08:39:13,912] jepsen worker 612 - jepsen.util 612 :invoke :add 821 INFO [2022-01-10 08:39:13,915] jepsen worker 967 - jepsen.util 967 :invoke :add 822 INFO [2022-01-10 08:39:13,917] jepsen worker 630 - jepsen.util 630 :ok :add 819 INFO [2022-01-10 08:39:13,917] jepsen worker 642 - jepsen.util 642 :ok :add 818 INFO [2022-01-10 08:39:13,918] jepsen worker 261 - jepsen.util 261 :invoke :add 823 INFO [2022-01-10 08:39:13,922] jepsen worker 794 - jepsen.util 794 :invoke :add 824 INFO [2022-01-10 08:39:13,922] jepsen worker 296 - jepsen.util 296 :ok :add 817 INFO [2022-01-10 08:39:13,923] jepsen worker 821 - jepsen.util 821 :ok :add 816 INFO [2022-01-10 08:39:13,924] jepsen worker 658 - jepsen.util 658 :invoke :add 825 INFO [2022-01-10 08:39:13,926] jepsen worker 413 - jepsen.util 413 :ok :add 820 INFO [2022-01-10 08:39:13,926] jepsen worker 612 - jepsen.util 612 :ok :add 821 INFO [2022-01-10 08:39:13,927] jepsen worker 61 - jepsen.util 61 :invoke :add 826 INFO [2022-01-10 08:39:13,930] jepsen worker 788 - jepsen.util 788 :invoke :add 827 INFO [2022-01-10 08:39:13,931] jepsen worker 967 - jepsen.util 967 :ok :add 822 INFO [2022-01-10 08:39:13,932] jepsen worker 278 - jepsen.util 278 :invoke :add 828 INFO [2022-01-10 08:39:13,935] jepsen worker 29 - jepsen.util 29 :invoke :add 829 INFO [2022-01-10 08:39:13,935] jepsen worker 658 - jepsen.util 658 :ok :add 825 INFO [2022-01-10 08:39:13,936] jepsen worker 261 - jepsen.util 261 :ok :add 823 INFO [2022-01-10 08:39:13,937] jepsen worker 794 - jepsen.util 794 :ok :add 824 INFO [2022-01-10 08:39:13,937] jepsen worker 250 - jepsen.util 250 :invoke :add 830 INFO [2022-01-10 08:39:13,938] jepsen worker 788 - jepsen.util 788 :ok :add 827 INFO [2022-01-10 08:39:13,941] jepsen worker 942 - jepsen.util 942 :invoke :add 831 INFO [2022-01-10 08:39:13,942] jepsen worker 278 - jepsen.util 278 :ok :add 828 INFO [2022-01-10 08:39:13,943] jepsen worker 29 - jepsen.util 29 :ok :add 829 INFO [2022-01-10 08:39:13,944] jepsen worker 864 - jepsen.util 864 :invoke :add 832 INFO [2022-01-10 08:39:13,947] jepsen worker 15 - jepsen.util 15 :invoke :add 833 INFO [2022-01-10 08:39:13,948] jepsen worker 250 - jepsen.util 250 :ok :add 830 INFO [2022-01-10 08:39:13,948] jepsen worker 61 - jepsen.util 61 :ok :add 826 INFO [2022-01-10 08:39:13,950] jepsen worker 410 - jepsen.util 410 :invoke :add 834 INFO [2022-01-10 08:39:13,951] jepsen worker 942 - jepsen.util 942 :ok :add 831 INFO [2022-01-10 08:39:13,953] jepsen worker 347 - jepsen.util 347 :invoke :add 835 INFO [2022-01-10 08:39:13,955] jepsen worker 1 - jepsen.util 1 :invoke :add 836 INFO [2022-01-10 08:39:13,956] jepsen worker 864 - jepsen.util 864 :ok :add 832 INFO [2022-01-10 08:39:13,957] jepsen worker 15 - jepsen.util 15 :ok :add 833 INFO [2022-01-10 08:39:13,958] jepsen worker 635 - jepsen.util 635 :invoke :add 837 INFO [2022-01-10 08:39:13,961] jepsen worker 646 - jepsen.util 646 :invoke :add 838 INFO [2022-01-10 08:39:13,964] jepsen worker 424 - jepsen.util 424 :invoke :add 839 INFO [2022-01-10 08:39:13,968] jepsen worker 124 - jepsen.util 124 :invoke :add 840 INFO [2022-01-10 08:39:13,970] jepsen worker 646 - jepsen.util 646 :ok :add 838 INFO [2022-01-10 08:39:13,971] jepsen worker 156 - jepsen.util 156 :invoke :add 841 INFO [2022-01-10 08:39:13,971] jepsen worker 424 - jepsen.util 424 :ok :add 839 INFO [2022-01-10 08:39:13,973] jepsen worker 235 - jepsen.util 235 :invoke :add 842 INFO [2022-01-10 08:39:13,974] jepsen worker 410 - jepsen.util 410 :ok :add 834 INFO [2022-01-10 08:39:13,974] jepsen worker 347 - jepsen.util 347 :ok :add 835 INFO [2022-01-10 08:39:13,977] jepsen worker 102 - jepsen.util 102 :invoke :add 843 INFO [2022-01-10 08:39:13,979] jepsen worker 1 - jepsen.util 1 :ok :add 836 INFO [2022-01-10 08:39:13,979] jepsen worker 635 - jepsen.util 635 :ok :add 837 INFO [2022-01-10 08:39:13,980] jepsen worker 124 - jepsen.util 124 :ok :add 840 INFO [2022-01-10 08:39:13,980] jepsen worker 156 - jepsen.util 156 :ok :add 841 INFO [2022-01-10 08:39:13,980] jepsen worker 160 - jepsen.util 160 :invoke :add 844 INFO [2022-01-10 08:39:13,985] jepsen worker 319 - jepsen.util 319 :invoke :add 845 INFO [2022-01-10 08:39:13,988] jepsen worker 313 - jepsen.util 313 :invoke :add 846 INFO [2022-01-10 08:39:13,989] jepsen worker 102 - jepsen.util 102 :ok :add 843 INFO [2022-01-10 08:39:13,990] jepsen worker 69 - jepsen.util 69 :invoke :add 847 INFO [2022-01-10 08:39:13,990] jepsen worker 235 - jepsen.util 235 :ok :add 842 INFO [2022-01-10 08:39:13,992] jepsen worker 407 - jepsen.util 407 :invoke :add 848 INFO [2022-01-10 08:39:13,995] jepsen worker 650 - jepsen.util 650 :invoke :add 849 INFO [2022-01-10 08:39:13,995] jepsen worker 319 - jepsen.util 319 :ok :add 845 INFO [2022-01-10 08:39:13,996] jepsen worker 160 - jepsen.util 160 :ok :add 844 INFO [2022-01-10 08:39:13,998] jepsen worker 933 - jepsen.util 933 :invoke :add 850 INFO [2022-01-10 08:39:14,000] jepsen worker 69 - jepsen.util 69 :ok :add 847 INFO [2022-01-10 08:39:14,001] jepsen worker 313 - jepsen.util 313 :ok :add 846 INFO [2022-01-10 08:39:14,001] jepsen worker 407 - jepsen.util 407 :ok :add 848 INFO [2022-01-10 08:39:14,002] jepsen worker 8 - jepsen.util 8 :invoke :add 851 INFO [2022-01-10 08:39:14,004] jepsen worker 650 - jepsen.util 650 :ok :add 849 INFO [2022-01-10 08:39:14,006] jepsen worker 28 - jepsen.util 28 :invoke :add 852 INFO [2022-01-10 08:39:14,006] jepsen worker 933 - jepsen.util 933 :ok :add 850 INFO [2022-01-10 08:39:14,009] jepsen worker 460 - jepsen.util 460 :invoke :add 853 INFO [2022-01-10 08:39:14,012] jepsen worker 148 - jepsen.util 148 :invoke :add 854 INFO [2022-01-10 08:39:14,014] jepsen worker 985 - jepsen.util 985 :invoke :add 855 INFO [2022-01-10 08:39:14,015] jepsen worker 8 - jepsen.util 8 :ok :add 851 INFO [2022-01-10 08:39:14,016] jepsen worker 28 - jepsen.util 28 :ok :add 852 INFO [2022-01-10 08:39:14,016] jepsen worker 716 - jepsen.util 716 :invoke :add 856 INFO [2022-01-10 08:39:14,020] jepsen worker 509 - jepsen.util 509 :invoke :add 857 INFO [2022-01-10 08:39:14,023] jepsen worker 460 - jepsen.util 460 :ok :add 853 INFO [2022-01-10 08:39:14,023] jepsen worker 725 - jepsen.util 725 :invoke :add 858 INFO [2022-01-10 08:39:14,025] jepsen worker 985 - jepsen.util 985 :ok :add 855 INFO [2022-01-10 08:39:14,026] jepsen worker 953 - jepsen.util 953 :invoke :add 859 INFO [2022-01-10 08:39:14,030] jepsen worker 148 - jepsen.util 148 :ok :add 854 INFO [2022-01-10 08:39:14,031] jepsen worker 666 - jepsen.util 666 :invoke :add 860 INFO [2022-01-10 08:39:14,034] jepsen worker 868 - jepsen.util 868 :invoke :add 861 INFO [2022-01-10 08:39:14,036] jepsen worker 725 - jepsen.util 725 :ok :add 858 INFO [2022-01-10 08:39:14,036] jepsen worker 953 - jepsen.util 953 :ok :add 859 INFO [2022-01-10 08:39:14,037] jepsen worker 269 - jepsen.util 269 :invoke :add 862 INFO [2022-01-10 08:39:14,039] jepsen worker 587 - jepsen.util 587 :invoke :add 863 INFO [2022-01-10 08:39:14,043] jepsen worker 304 - jepsen.util 304 :invoke :add 864 INFO [2022-01-10 08:39:14,046] jepsen worker 138 - jepsen.util 138 :invoke :add 865 INFO [2022-01-10 08:39:14,048] jepsen worker 935 - jepsen.util 935 :invoke :add 866 INFO [2022-01-10 08:39:14,051] jepsen worker 586 - jepsen.util 586 :invoke :add 867 INFO [2022-01-10 08:39:14,054] jepsen worker 334 - jepsen.util 334 :invoke :add 868 INFO [2022-01-10 08:39:14,057] jepsen worker 493 - jepsen.util 493 :invoke :add 869 INFO [2022-01-10 08:39:14,057] jepsen worker 509 - jepsen.util 509 :ok :add 857 INFO [2022-01-10 08:39:14,057] jepsen worker 269 - jepsen.util 269 :ok :add 862 INFO [2022-01-10 08:39:14,057] jepsen worker 716 - jepsen.util 716 :ok :add 856 INFO [2022-01-10 08:39:14,058] jepsen worker 587 - jepsen.util 587 :ok :add 863 INFO [2022-01-10 08:39:14,060] jepsen worker 721 - jepsen.util 721 :invoke :add 870 INFO [2022-01-10 08:39:14,063] jepsen worker 491 - jepsen.util 491 :invoke :add 871 INFO [2022-01-10 08:39:14,064] jepsen worker 304 - jepsen.util 304 :ok :add 864 INFO [2022-01-10 08:39:14,065] jepsen worker 868 - jepsen.util 868 :ok :add 861 INFO [2022-01-10 08:39:14,066] jepsen worker 334 - jepsen.util 334 :ok :add 868 INFO [2022-01-10 08:39:14,066] jepsen worker 2 - jepsen.util 2 :invoke :add 872 INFO [2022-01-10 08:39:14,069] jepsen worker 908 - jepsen.util 908 :invoke :add 873 INFO [2022-01-10 08:39:14,072] jepsen worker 318 - jepsen.util 318 :invoke :add 874 INFO [2022-01-10 08:39:14,074] jepsen worker 73 - jepsen.util 73 :invoke :add 875 INFO [2022-01-10 08:39:14,077] jepsen worker 493 - jepsen.util 493 :ok :add 869 INFO [2022-01-10 08:39:14,078] jepsen worker 889 - jepsen.util 889 :invoke :add 876 INFO [2022-01-10 08:39:14,080] jepsen worker 935 - jepsen.util 935 :ok :add 866 INFO [2022-01-10 08:39:14,080] jepsen worker 311 - jepsen.util 311 :invoke :add 877 INFO [2022-01-10 08:39:14,081] jepsen worker 318 - jepsen.util 318 :ok :add 874 INFO [2022-01-10 08:39:14,082] jepsen worker 586 - jepsen.util 586 :ok :add 867 INFO [2022-01-10 08:39:14,083] jepsen worker 666 - jepsen.util 666 :ok :add 860 INFO [2022-01-10 08:39:14,085] jepsen worker 893 - jepsen.util 893 :invoke :add 878 INFO [2022-01-10 08:39:14,088] jepsen worker 721 - jepsen.util 721 :ok :add 870 INFO [2022-01-10 08:39:14,088] jepsen worker 138 - jepsen.util 138 :ok :add 865 INFO [2022-01-10 08:39:14,088] jepsen worker 321 - jepsen.util 321 :invoke :add 879 INFO [2022-01-10 08:39:14,089] jepsen worker 73 - jepsen.util 73 :ok :add 875 INFO [2022-01-10 08:39:14,091] jepsen worker 889 - jepsen.util 889 :ok :add 876 INFO [2022-01-10 08:39:14,093] jepsen worker 828 - jepsen.util 828 :invoke :add 880 INFO [2022-01-10 08:39:14,094] jepsen worker 311 - jepsen.util 311 :ok :add 877 INFO [2022-01-10 08:39:14,095] jepsen worker 94 - jepsen.util 94 :invoke :add 881 INFO [2022-01-10 08:39:14,098] jepsen worker 737 - jepsen.util 737 :invoke :add 882 INFO [2022-01-10 08:39:14,099] jepsen worker 321 - jepsen.util 321 :ok :add 879 INFO [2022-01-10 08:39:14,101] jepsen worker 907 - jepsen.util 907 :invoke :add 883 INFO [2022-01-10 08:39:14,101] jepsen worker 491 - jepsen.util 491 :ok :add 871 INFO [2022-01-10 08:39:14,102] jepsen worker 828 - jepsen.util 828 :ok :add 880 INFO [2022-01-10 08:39:14,103] jepsen worker 95 - jepsen.util 95 :invoke :add 884 INFO [2022-01-10 08:39:14,106] jepsen worker 894 - jepsen.util 894 :invoke :add 885 INFO [2022-01-10 08:39:14,108] jepsen worker 908 - jepsen.util 908 :ok :add 873 INFO [2022-01-10 08:39:14,108] jepsen worker 318 - jepsen.util 318 :invoke :add 886 INFO [2022-01-10 08:39:14,110] jepsen worker 893 - jepsen.util 893 :ok :add 878 INFO [2022-01-10 08:39:14,110] jepsen worker 737 - jepsen.util 737 :ok :add 882 INFO [2022-01-10 08:39:14,111] jepsen worker 248 - jepsen.util 248 :invoke :add 887 INFO [2022-01-10 08:39:14,115] jepsen worker 894 - jepsen.util 894 :ok :add 885 INFO [2022-01-10 08:39:14,115] jepsen worker 2 - jepsen.util 2 :ok :add 872 INFO [2022-01-10 08:39:14,116] jepsen worker 932 - jepsen.util 932 :invoke :add 888 INFO [2022-01-10 08:39:14,117] jepsen worker 94 - jepsen.util 94 :ok :add 881 INFO [2022-01-10 08:39:14,118] jepsen worker 95 - jepsen.util 95 :ok :add 884 INFO [2022-01-10 08:39:14,119] jepsen worker 76 - jepsen.util 76 :invoke :add 889 INFO [2022-01-10 08:39:14,121] jepsen worker 907 - jepsen.util 907 :ok :add 883 INFO [2022-01-10 08:39:14,122] jepsen worker 248 - jepsen.util 248 :ok :add 887 INFO [2022-01-10 08:39:14,123] jepsen worker 180 - jepsen.util 180 :invoke :add 890 INFO [2022-01-10 08:39:14,126] jepsen worker 749 - jepsen.util 749 :invoke :add 891 INFO [2022-01-10 08:39:14,126] jepsen worker 318 - jepsen.util 318 :ok :add 886 INFO [2022-01-10 08:39:14,128] jepsen worker 939 - jepsen.util 939 :invoke :add 892 INFO [2022-01-10 08:39:14,132] jepsen worker 589 - jepsen.util 589 :invoke :add 893 INFO [2022-01-10 08:39:14,134] jepsen worker 852 - jepsen.util 852 :invoke :add 894 INFO [2022-01-10 08:39:14,137] jepsen worker 331 - jepsen.util 331 :invoke :add 895 INFO [2022-01-10 08:39:14,141] jepsen worker 336 - jepsen.util 336 :invoke :add 896 INFO [2022-01-10 08:39:14,145] jepsen worker 932 - jepsen.util 932 :ok :add 888 INFO [2022-01-10 08:39:14,145] jepsen worker 180 - jepsen.util 180 :ok :add 890 INFO [2022-01-10 08:39:14,145] jepsen worker 76 - jepsen.util 76 :ok :add 889 INFO [2022-01-10 08:39:14,145] jepsen worker 749 - jepsen.util 749 :ok :add 891 INFO [2022-01-10 08:39:14,146] jepsen worker 961 - jepsen.util 961 :invoke :add 897 INFO [2022-01-10 08:39:14,149] jepsen worker 589 - jepsen.util 589 :ok :add 893 INFO [2022-01-10 08:39:14,150] jepsen worker 53 - jepsen.util 53 :invoke :add 898 INFO [2022-01-10 08:39:14,151] jepsen worker 939 - jepsen.util 939 :ok :add 892 INFO [2022-01-10 08:39:14,152] jepsen worker 852 - jepsen.util 852 :ok :add 894 INFO [2022-01-10 08:39:14,154] jepsen worker 614 - jepsen.util 614 :invoke :add 899 INFO [2022-01-10 08:39:14,155] jepsen worker 331 - jepsen.util 331 :ok :add 895 INFO [2022-01-10 08:39:14,158] jepsen worker 187 - jepsen.util 187 :invoke :add 900 INFO [2022-01-10 08:39:14,160] jepsen worker 336 - jepsen.util 336 :ok :add 896 INFO [2022-01-10 08:39:14,160] jepsen worker 961 - jepsen.util 961 :ok :add 897 INFO [2022-01-10 08:39:14,161] jepsen worker 481 - jepsen.util 481 :invoke :add 901 INFO [2022-01-10 08:39:14,165] jepsen worker 53 - jepsen.util 53 :ok :add 898 INFO [2022-01-10 08:39:14,165] jepsen worker 614 - jepsen.util 614 :ok :add 899 INFO [2022-01-10 08:39:14,168] jepsen worker 992 - jepsen.util 992 :invoke :add 902 INFO [2022-01-10 08:39:14,170] jepsen worker 187 - jepsen.util 187 :ok :add 900 INFO [2022-01-10 08:39:14,172] jepsen worker 636 - jepsen.util 636 :invoke :add 903 INFO [2022-01-10 08:39:14,173] jepsen worker 481 - jepsen.util 481 :ok :add 901 INFO [2022-01-10 08:39:14,175] jepsen worker 773 - jepsen.util 773 :invoke :add 904 INFO [2022-01-10 08:39:14,178] jepsen worker 92 - jepsen.util 92 :invoke :add 905 INFO [2022-01-10 08:39:14,180] jepsen worker 133 - jepsen.util 133 :invoke :add 906 INFO [2022-01-10 08:39:14,180] jepsen worker 992 - jepsen.util 992 :ok :add 902 INFO [2022-01-10 08:39:14,182] jepsen worker 636 - jepsen.util 636 :ok :add 903 INFO [2022-01-10 08:39:14,182] jepsen worker 773 - jepsen.util 773 :ok :add 904 INFO [2022-01-10 08:39:14,185] jepsen worker 318 - jepsen.util 318 :invoke :add 907 INFO [2022-01-10 08:39:14,187] jepsen worker 92 - jepsen.util 92 :ok :add 905 INFO [2022-01-10 08:39:14,189] jepsen worker 244 - jepsen.util 244 :invoke :add 908 INFO [2022-01-10 08:39:14,191] jepsen worker 508 - jepsen.util 508 :invoke :add 909 INFO [2022-01-10 08:39:14,192] jepsen worker 133 - jepsen.util 133 :ok :add 906 INFO [2022-01-10 08:39:14,193] jepsen worker 703 - jepsen.util 703 :invoke :add 910 INFO [2022-01-10 08:39:14,196] jepsen worker 798 - jepsen.util 798 :invoke :add 911 INFO [2022-01-10 08:39:14,197] jepsen worker 318 - jepsen.util 318 :ok :add 907 INFO [2022-01-10 08:39:14,198] jepsen worker 573 - jepsen.util 573 :invoke :add 912 INFO [2022-01-10 08:39:14,201] jepsen worker 244 - jepsen.util 244 :ok :add 908 INFO [2022-01-10 08:39:14,201] jepsen worker 307 - jepsen.util 307 :invoke :add 913 INFO [2022-01-10 08:39:14,203] jepsen worker 508 - jepsen.util 508 :ok :add 909 INFO [2022-01-10 08:39:14,205] jepsen worker 549 - jepsen.util 549 :invoke :add 914 INFO [2022-01-10 08:39:14,207] jepsen worker 367 - jepsen.util 367 :invoke :add 915 INFO [2022-01-10 08:39:14,210] jepsen worker 798 - jepsen.util 798 :ok :add 911 INFO [2022-01-10 08:39:14,210] jepsen worker 541 - jepsen.util 541 :invoke :add 916 INFO [2022-01-10 08:39:14,212] jepsen worker 430 - jepsen.util 430 :invoke :add 917 INFO [2022-01-10 08:39:14,212] jepsen worker 703 - jepsen.util 703 :ok :add 910 INFO [2022-01-10 08:39:14,215] jepsen worker 790 - jepsen.util 790 :invoke :add 918 INFO [2022-01-10 08:39:14,217] jepsen worker 307 - jepsen.util 307 :ok :add 913 INFO [2022-01-10 08:39:14,218] jepsen worker 912 - jepsen.util 912 :invoke :add 919 INFO [2022-01-10 08:39:14,219] jepsen worker 573 - jepsen.util 573 :ok :add 912 INFO [2022-01-10 08:39:14,221] jepsen worker 375 - jepsen.util 375 :invoke :add 920 INFO [2022-01-10 08:39:14,222] jepsen worker 790 - jepsen.util 790 :ok :add 918 INFO [2022-01-10 08:39:14,223] jepsen worker 664 - jepsen.util 664 :invoke :add 921 INFO [2022-01-10 08:39:14,226] jepsen worker 418 - jepsen.util 418 :invoke :add 922 INFO [2022-01-10 08:39:14,228] jepsen worker 912 - jepsen.util 912 :ok :add 919 INFO [2022-01-10 08:39:14,229] jepsen worker 524 - jepsen.util 524 :invoke :add 923 INFO [2022-01-10 08:39:14,233] jepsen worker 229 - jepsen.util 229 :invoke :add 924 INFO [2022-01-10 08:39:14,233] jepsen worker 549 - jepsen.util 549 :ok :add 914 INFO [2022-01-10 08:39:14,235] jepsen worker 68 - jepsen.util 68 :invoke :add 925 INFO [2022-01-10 08:39:14,236] jepsen worker 367 - jepsen.util 367 :ok :add 915 INFO [2022-01-10 08:39:14,238] jepsen worker 418 - jepsen.util 418 :ok :add 922 INFO [2022-01-10 08:39:14,238] jepsen worker 237 - jepsen.util 237 :invoke :add 926 INFO [2022-01-10 08:39:14,238] jepsen worker 541 - jepsen.util 541 :ok :add 916 INFO [2022-01-10 08:39:14,241] jepsen worker 810 - jepsen.util 810 :invoke :add 927 INFO [2022-01-10 08:39:14,242] jepsen worker 229 - jepsen.util 229 :ok :add 924 INFO [2022-01-10 08:39:14,244] jepsen worker 368 - jepsen.util 368 :invoke :add 928 INFO [2022-01-10 08:39:14,246] jepsen worker 430 - jepsen.util 430 :ok :add 917 INFO [2022-01-10 08:39:14,246] jepsen worker 524 - jepsen.util 524 :ok :add 923 INFO [2022-01-10 08:39:14,246] jepsen worker 375 - jepsen.util 375 :ok :add 920 INFO [2022-01-10 08:39:14,247] jepsen worker 68 - jepsen.util 68 :ok :add 925 INFO [2022-01-10 08:39:14,248] jepsen worker 346 - jepsen.util 346 :invoke :add 929 INFO [2022-01-10 08:39:14,252] jepsen worker 676 - jepsen.util 676 :invoke :add 930 INFO [2022-01-10 08:39:14,253] jepsen worker 368 - jepsen.util 368 :ok :add 928 INFO [2022-01-10 08:39:14,253] jepsen worker 810 - jepsen.util 810 :ok :add 927 INFO [2022-01-10 08:39:14,253] jepsen worker 237 - jepsen.util 237 :ok :add 926 INFO [2022-01-10 08:39:14,255] jepsen worker 133 - jepsen.util 133 :invoke :add 931 INFO [2022-01-10 08:39:14,256] jepsen worker 664 - jepsen.util 664 :ok :add 921 INFO [2022-01-10 08:39:14,258] jepsen worker 38 - jepsen.util 38 :invoke :add 932 INFO [2022-01-10 08:39:14,261] jepsen worker 130 - jepsen.util 130 :invoke :add 933 INFO [2022-01-10 08:39:14,264] jepsen worker 331 - jepsen.util 331 :invoke :add 934 INFO [2022-01-10 08:39:14,264] jepsen worker 676 - jepsen.util 676 :ok :add 930 INFO [2022-01-10 08:39:14,266] jepsen worker 765 - jepsen.util 765 :invoke :add 935 INFO [2022-01-10 08:39:14,266] jepsen worker 133 - jepsen.util 133 :ok :add 931 INFO [2022-01-10 08:39:14,266] jepsen worker 346 - jepsen.util 346 :ok :add 929 INFO [2022-01-10 08:39:14,272] jepsen worker 752 - jepsen.util 752 :invoke :add 936 INFO [2022-01-10 08:39:14,272] jepsen worker 38 - jepsen.util 38 :ok :add 932 INFO [2022-01-10 08:39:14,272] jepsen worker 130 - jepsen.util 130 :ok :add 933 INFO [2022-01-10 08:39:14,276] jepsen worker 331 - jepsen.util 331 :ok :add 934 INFO [2022-01-10 08:39:14,277] jepsen worker 695 - jepsen.util 695 :invoke :add 937 INFO [2022-01-10 08:39:14,279] jepsen worker 765 - jepsen.util 765 :ok :add 935 INFO [2022-01-10 08:39:14,281] jepsen worker 176 - jepsen.util 176 :invoke :add 938 INFO [2022-01-10 08:39:14,285] jepsen worker 808 - jepsen.util 808 :invoke :add 939 INFO [2022-01-10 08:39:14,289] jepsen worker 28 - jepsen.util 28 :invoke :add 940 INFO [2022-01-10 08:39:14,291] jepsen worker 695 - jepsen.util 695 :ok :add 937 INFO [2022-01-10 08:39:14,292] jepsen worker 808 - jepsen.util 808 :ok :add 939 INFO [2022-01-10 08:39:14,294] jepsen worker 329 - jepsen.util 329 :invoke :add 941 INFO [2022-01-10 08:39:14,295] jepsen worker 176 - jepsen.util 176 :ok :add 938 INFO [2022-01-10 08:39:14,295] jepsen worker 752 - jepsen.util 752 :ok :add 936 INFO [2022-01-10 08:39:14,298] jepsen worker 964 - jepsen.util 964 :invoke :add 942 INFO [2022-01-10 08:39:14,299] jepsen worker 28 - jepsen.util 28 :ok :add 940 INFO [2022-01-10 08:39:14,301] jepsen worker 441 - jepsen.util 441 :invoke :add 943 INFO [2022-01-10 08:39:14,304] jepsen worker 530 - jepsen.util 530 :invoke :add 944 INFO [2022-01-10 08:39:14,305] jepsen worker 329 - jepsen.util 329 :ok :add 941 INFO [2022-01-10 08:39:14,306] jepsen worker 173 - jepsen.util 173 :invoke :add 945 INFO [2022-01-10 08:39:14,310] jepsen worker 837 - jepsen.util 837 :invoke :add 946 INFO [2022-01-10 08:39:14,311] jepsen worker 441 - jepsen.util 441 :ok :add 943 INFO [2022-01-10 08:39:14,312] jepsen worker 964 - jepsen.util 964 :ok :add 942 INFO [2022-01-10 08:39:14,314] jepsen worker 173 - jepsen.util 173 :ok :add 945 INFO [2022-01-10 08:39:14,315] jepsen worker 535 - jepsen.util 535 :invoke :add 947 INFO [2022-01-10 08:39:14,318] jepsen worker 530 - jepsen.util 530 :ok :add 944 INFO [2022-01-10 08:39:14,319] jepsen worker 188 - jepsen.util 188 :invoke :add 948 INFO [2022-01-10 08:39:14,321] jepsen worker 142 - jepsen.util 142 :invoke :add 949 INFO [2022-01-10 08:39:14,324] jepsen worker 535 - jepsen.util 535 :ok :add 947 INFO [2022-01-10 08:39:14,324] jepsen worker 783 - jepsen.util 783 :invoke :add 950 INFO [2022-01-10 08:39:14,327] jepsen worker 445 - jepsen.util 445 :invoke :add 951 INFO [2022-01-10 08:39:14,328] jepsen worker 837 - jepsen.util 837 :ok :add 946 INFO [2022-01-10 08:39:14,330] jepsen worker 185 - jepsen.util 185 :invoke :add 952 INFO [2022-01-10 08:39:14,333] jepsen worker 639 - jepsen.util 639 :invoke :add 953 INFO [2022-01-10 08:39:14,336] jepsen worker 159 - jepsen.util 159 :invoke :add 954 INFO [2022-01-10 08:39:14,338] jepsen worker 402 - jepsen.util 402 :invoke :add 955 INFO [2022-01-10 08:39:14,340] jepsen worker 947 - jepsen.util 947 :invoke :add 956 INFO [2022-01-10 08:39:14,343] jepsen worker 188 - jepsen.util 188 :ok :add 948 INFO [2022-01-10 08:39:14,344] jepsen worker 86 - jepsen.util 86 :invoke :add 957 INFO [2022-01-10 08:39:14,346] jepsen worker 142 - jepsen.util 142 :ok :add 949 INFO [2022-01-10 08:39:14,347] jepsen worker 402 - jepsen.util 402 :ok :add 955 INFO [2022-01-10 08:39:14,347] jepsen worker 330 - jepsen.util 330 :invoke :add 958 INFO [2022-01-10 08:39:14,347] jepsen worker 783 - jepsen.util 783 :ok :add 950 INFO [2022-01-10 08:39:14,349] jepsen worker 159 - jepsen.util 159 :ok :add 954 INFO [2022-01-10 08:39:14,351] jepsen worker 445 - jepsen.util 445 :ok :add 951 INFO [2022-01-10 08:39:14,351] jepsen worker 404 - jepsen.util 404 :invoke :add 959 INFO [2022-01-10 08:39:14,352] jepsen worker 185 - jepsen.util 185 :ok :add 952 INFO [2022-01-10 08:39:14,354] jepsen worker 205 - jepsen.util 205 :invoke :add 960 INFO [2022-01-10 08:39:14,354] jepsen worker 947 - jepsen.util 947 :ok :add 956 INFO [2022-01-10 08:39:14,356] jepsen worker 639 - jepsen.util 639 :ok :add 953 INFO [2022-01-10 08:39:14,357] jepsen worker 329 - jepsen.util 329 :invoke :add 961 INFO [2022-01-10 08:39:14,359] jepsen worker 330 - jepsen.util 330 :ok :add 958 INFO [2022-01-10 08:39:14,361] jepsen worker 586 - jepsen.util 586 :invoke :add 962 INFO [2022-01-10 08:39:14,363] jepsen worker 404 - jepsen.util 404 :ok :add 959 INFO [2022-01-10 08:39:14,364] jepsen worker 86 - jepsen.util 86 :ok :add 957 INFO [2022-01-10 08:39:14,365] jepsen worker 911 - jepsen.util 911 :invoke :add 963 INFO [2022-01-10 08:39:14,369] jepsen worker 329 - jepsen.util 329 :ok :add 961 INFO [2022-01-10 08:39:14,369] jepsen worker 486 - jepsen.util 486 :invoke :add 964 INFO [2022-01-10 08:39:14,372] jepsen worker 586 - jepsen.util 586 :ok :add 962 INFO [2022-01-10 08:39:14,372] jepsen worker 911 - jepsen.util 911 :ok :add 963 INFO [2022-01-10 08:39:14,373] jepsen worker 306 - jepsen.util 306 :invoke :add 965 INFO [2022-01-10 08:39:14,376] jepsen worker 205 - jepsen.util 205 :ok :add 960 INFO [2022-01-10 08:39:14,377] jepsen worker 486 - jepsen.util 486 :ok :add 964 INFO [2022-01-10 08:39:14,377] jepsen worker 862 - jepsen.util 862 :invoke :add 966 INFO [2022-01-10 08:39:14,381] jepsen worker 922 - jepsen.util 922 :invoke :add 967 INFO [2022-01-10 08:39:14,383] jepsen worker 306 - jepsen.util 306 :ok :add 965 INFO [2022-01-10 08:39:14,384] jepsen worker 797 - jepsen.util 797 :invoke :add 968 INFO [2022-01-10 08:39:14,386] jepsen worker 862 - jepsen.util 862 :ok :add 966 INFO [2022-01-10 08:39:14,387] jepsen worker 801 - jepsen.util 801 :invoke :add 969 INFO [2022-01-10 08:39:14,390] jepsen worker 846 - jepsen.util 846 :invoke :add 970 INFO [2022-01-10 08:39:14,391] jepsen worker 797 - jepsen.util 797 :ok :add 968 INFO [2022-01-10 08:39:14,392] jepsen worker 922 - jepsen.util 922 :ok :add 967 INFO [2022-01-10 08:39:14,394] jepsen worker 107 - jepsen.util 107 :invoke :add 971 INFO [2022-01-10 08:39:14,397] jepsen worker 801 - jepsen.util 801 :ok :add 969 INFO [2022-01-10 08:39:14,399] jepsen worker 872 - jepsen.util 872 :invoke :add 972 INFO [2022-01-10 08:39:14,399] jepsen worker 846 - jepsen.util 846 :ok :add 970 INFO [2022-01-10 08:39:14,406] jepsen worker 107 - jepsen.util 107 :ok :add 971 INFO [2022-01-10 08:39:14,407] jepsen worker 872 - jepsen.util 872 :ok :add 972 INFO [2022-01-10 08:39:14,408] jepsen worker 705 - jepsen.util 705 :invoke :add 973 INFO [2022-01-10 08:39:14,412] jepsen worker 932 - jepsen.util 932 :invoke :add 974 INFO [2022-01-10 08:39:14,415] jepsen worker 705 - jepsen.util 705 :ok :add 973 INFO [2022-01-10 08:39:14,415] jepsen worker 974 - jepsen.util 974 :invoke :add 975 INFO [2022-01-10 08:39:14,418] jepsen worker 919 - jepsen.util 919 :invoke :add 976 INFO [2022-01-10 08:39:14,419] jepsen worker 932 - jepsen.util 932 :ok :add 974 INFO [2022-01-10 08:39:14,421] jepsen worker 216 - jepsen.util 216 :invoke :add 977 INFO [2022-01-10 08:39:14,423] jepsen worker 974 - jepsen.util 974 :ok :add 975 INFO [2022-01-10 08:39:14,425] jepsen worker 860 - jepsen.util 860 :invoke :add 978 INFO [2022-01-10 08:39:14,428] jepsen worker 696 - jepsen.util 696 :invoke :add 979 INFO [2022-01-10 08:39:14,430] jepsen worker 153 - jepsen.util 153 :invoke :add 980 INFO [2022-01-10 08:39:14,432] jepsen worker 109 - jepsen.util 109 :invoke :add 981 INFO [2022-01-10 08:39:14,435] jepsen worker 908 - jepsen.util 908 :invoke :add 982 INFO [2022-01-10 08:39:14,437] jepsen worker 426 - jepsen.util 426 :invoke :add 983 INFO [2022-01-10 08:39:14,439] jepsen worker 696 - jepsen.util 696 :ok :add 979 INFO [2022-01-10 08:39:14,440] jepsen worker 25 - jepsen.util 25 :invoke :add 984 INFO [2022-01-10 08:39:14,440] jepsen worker 860 - jepsen.util 860 :ok :add 978 INFO [2022-01-10 08:39:14,442] jepsen worker 153 - jepsen.util 153 :ok :add 980 INFO [2022-01-10 08:39:14,443] jepsen worker 936 - jepsen.util 936 :invoke :add 985 INFO [2022-01-10 08:39:14,446] jepsen worker 294 - jepsen.util 294 :invoke :add 986 INFO [2022-01-10 08:39:14,447] jepsen worker 109 - jepsen.util 109 :ok :add 981 INFO [2022-01-10 08:39:14,448] jepsen worker 25 - jepsen.util 25 :ok :add 984 INFO [2022-01-10 08:39:14,448] jepsen worker 919 - jepsen.util 919 :ok :add 976 INFO [2022-01-10 08:39:14,449] jepsen worker 527 - jepsen.util 527 :invoke :add 987 INFO [2022-01-10 08:39:14,452] jepsen worker 658 - jepsen.util 658 :invoke :add 988 INFO [2022-01-10 08:39:14,453] jepsen worker 216 - jepsen.util 216 :ok :add 977 INFO [2022-01-10 08:39:14,456] jepsen worker 749 - jepsen.util 749 :invoke :add 989 INFO [2022-01-10 08:39:14,456] jepsen worker 936 - jepsen.util 936 :ok :add 985 INFO [2022-01-10 08:39:14,457] jepsen worker 908 - jepsen.util 908 :ok :add 982 INFO [2022-01-10 08:39:14,458] jepsen worker 426 - jepsen.util 426 :ok :add 983 INFO [2022-01-10 08:39:14,459] jepsen worker 603 - jepsen.util 603 :invoke :add 990 INFO [2022-01-10 08:39:14,462] jepsen worker 527 - jepsen.util 527 :ok :add 987 INFO [2022-01-10 08:39:14,464] jepsen worker 294 - jepsen.util 294 :ok :add 986 INFO [2022-01-10 08:39:14,465] jepsen worker 373 - jepsen.util 373 :invoke :add 991 INFO [2022-01-10 08:39:14,466] jepsen worker 658 - jepsen.util 658 :ok :add 988 INFO [2022-01-10 08:39:14,468] jepsen worker 749 - jepsen.util 749 :ok :add 989 INFO [2022-01-10 08:39:14,470] jepsen worker 639 - jepsen.util 639 :invoke :add 992 INFO [2022-01-10 08:39:14,475] jepsen worker 373 - jepsen.util 373 :ok :add 991 INFO [2022-01-10 08:39:14,475] jepsen worker 358 - jepsen.util 358 :invoke :add 993 INFO [2022-01-10 08:39:14,475] jepsen worker 603 - jepsen.util 603 :ok :add 990 INFO [2022-01-10 08:39:14,477] jepsen worker 190 - jepsen.util 190 :invoke :add 994 INFO [2022-01-10 08:39:14,480] jepsen worker 639 - jepsen.util 639 :ok :add 992 INFO [2022-01-10 08:39:14,481] jepsen worker 379 - jepsen.util 379 :invoke :add 995 INFO [2022-01-10 08:39:14,484] jepsen worker 627 - jepsen.util 627 :invoke :add 996 INFO [2022-01-10 08:39:14,485] jepsen worker 190 - jepsen.util 190 :ok :add 994 INFO [2022-01-10 08:39:14,486] jepsen worker 221 - jepsen.util 221 :invoke :add 997 INFO [2022-01-10 08:39:14,487] jepsen worker 358 - jepsen.util 358 :ok :add 993 INFO [2022-01-10 08:39:14,491] jepsen worker 41 - jepsen.util 41 :invoke :add 998 INFO [2022-01-10 08:39:14,494] jepsen worker 627 - jepsen.util 627 :ok :add 996 INFO [2022-01-10 08:39:14,495] jepsen worker 379 - jepsen.util 379 :ok :add 995 INFO [2022-01-10 08:39:14,496] jepsen worker 628 - jepsen.util 628 :invoke :add 999 INFO [2022-01-10 08:39:14,497] jepsen worker 41 - jepsen.util 41 :ok :add 998 INFO [2022-01-10 08:39:14,500] jepsen worker 221 - jepsen.util 221 :ok :add 997 INFO [2022-01-10 08:39:14,501] jepsen worker 878 - jepsen.util 878 :invoke :add 1000 INFO [2022-01-10 08:39:14,503] jepsen worker 38 - jepsen.util 38 :invoke :add 1001 INFO [2022-01-10 08:39:14,505] jepsen worker 628 - jepsen.util 628 :ok :add 999 INFO [2022-01-10 08:39:14,506] jepsen worker 661 - jepsen.util 661 :invoke :add 1002 INFO [2022-01-10 08:39:14,510] jepsen worker 506 - jepsen.util 506 :invoke :add 1003 INFO [2022-01-10 08:39:14,513] jepsen worker 94 - jepsen.util 94 :invoke :add 1004 INFO [2022-01-10 08:39:14,516] jepsen worker 155 - jepsen.util 155 :invoke :add 1005 INFO [2022-01-10 08:39:14,517] jepsen worker 38 - jepsen.util 38 :ok :add 1001 INFO [2022-01-10 08:39:14,519] jepsen worker 661 - jepsen.util 661 :ok :add 1002 INFO [2022-01-10 08:39:14,519] jepsen worker 596 - jepsen.util 596 :invoke :add 1006 INFO [2022-01-10 08:39:14,520] jepsen worker 878 - jepsen.util 878 :ok :add 1000 INFO [2022-01-10 08:39:14,520] jepsen worker 94 - jepsen.util 94 :ok :add 1004 INFO [2022-01-10 08:39:14,524] jepsen worker 155 - jepsen.util 155 :ok :add 1005 INFO [2022-01-10 08:39:14,525] jepsen worker 124 - jepsen.util 124 :invoke :add 1007 INFO [2022-01-10 08:39:14,530] jepsen worker 799 - jepsen.util 799 :invoke :add 1008 INFO [2022-01-10 08:39:14,530] jepsen worker 506 - jepsen.util 506 :ok :add 1003 INFO [2022-01-10 08:39:14,533] jepsen worker 124 - jepsen.util 124 :ok :add 1007 INFO [2022-01-10 08:39:14,534] jepsen worker 596 - jepsen.util 596 :ok :add 1006 INFO [2022-01-10 08:39:14,534] jepsen worker 796 - jepsen.util 796 :invoke :add 1009 INFO [2022-01-10 08:39:14,539] jepsen worker 787 - jepsen.util 787 :invoke :add 1010 INFO [2022-01-10 08:39:14,542] jepsen worker 796 - jepsen.util 796 :ok :add 1009 INFO [2022-01-10 08:39:14,542] jepsen worker 799 - jepsen.util 799 :ok :add 1008 INFO [2022-01-10 08:39:14,544] jepsen worker 353 - jepsen.util 353 :invoke :add 1011 INFO [2022-01-10 08:39:14,547] jepsen worker 140 - jepsen.util 140 :invoke :add 1012 INFO [2022-01-10 08:39:14,550] jepsen worker 647 - jepsen.util 647 :invoke :add 1013 INFO [2022-01-10 08:39:14,550] jepsen worker 353 - jepsen.util 353 :ok :add 1011 INFO [2022-01-10 08:39:14,552] jepsen worker 787 - jepsen.util 787 :ok :add 1010 INFO [2022-01-10 08:39:14,553] jepsen worker 820 - jepsen.util 820 :invoke :add 1014 INFO [2022-01-10 08:39:14,555] jepsen worker 577 - jepsen.util 577 :invoke :add 1015 INFO [2022-01-10 08:39:14,557] jepsen worker 140 - jepsen.util 140 :ok :add 1012 INFO [2022-01-10 08:39:14,557] jepsen worker 647 - jepsen.util 647 :ok :add 1013 INFO [2022-01-10 08:39:14,558] jepsen worker 592 - jepsen.util 592 :invoke :add 1016 INFO [2022-01-10 08:39:14,562] jepsen worker 409 - jepsen.util 409 :invoke :add 1017 INFO [2022-01-10 08:39:14,562] jepsen worker 577 - jepsen.util 577 :ok :add 1015 INFO [2022-01-10 08:39:14,563] jepsen worker 820 - jepsen.util 820 :ok :add 1014 INFO [2022-01-10 08:39:14,566] jepsen worker 570 - jepsen.util 570 :invoke :add 1018 INFO [2022-01-10 08:39:14,569] jepsen worker 592 - jepsen.util 592 :ok :add 1016 INFO [2022-01-10 08:39:14,573] jepsen worker 237 - jepsen.util 237 :invoke :add 1020 INFO [2022-01-10 08:39:14,573] jepsen worker 554 - jepsen.util 554 :invoke :add 1019 INFO [2022-01-10 08:39:14,574] jepsen worker 570 - jepsen.util 570 :ok :add 1018 INFO [2022-01-10 08:39:14,573] jepsen worker 409 - jepsen.util 409 :ok :add 1017 INFO [2022-01-10 08:39:14,576] jepsen worker 958 - jepsen.util 958 :invoke :add 1021 INFO [2022-01-10 08:39:14,579] jepsen worker 191 - jepsen.util 191 :invoke :add 1022 INFO [2022-01-10 08:39:14,582] jepsen worker 890 - jepsen.util 890 :invoke :add 1023 INFO [2022-01-10 08:39:14,583] jepsen worker 958 - jepsen.util 958 :ok :add 1021 INFO [2022-01-10 08:39:14,585] jepsen worker 94 - jepsen.util 94 :invoke :add 1024 INFO [2022-01-10 08:39:14,586] jepsen worker 554 - jepsen.util 554 :ok :add 1019 INFO [2022-01-10 08:39:14,588] jepsen worker 642 - jepsen.util 642 :invoke :add 1025 INFO [2022-01-10 08:39:14,589] jepsen worker 237 - jepsen.util 237 :ok :add 1020 INFO [2022-01-10 08:39:14,591] jepsen worker 575 - jepsen.util 575 :invoke :add 1026 INFO [2022-01-10 08:39:14,592] jepsen worker 890 - jepsen.util 890 :ok :add 1023 INFO [2022-01-10 08:39:14,594] jepsen worker 191 - jepsen.util 191 :ok :add 1022 INFO [2022-01-10 08:39:14,595] jepsen worker 94 - jepsen.util 94 :ok :add 1024 INFO [2022-01-10 08:39:14,597] jepsen worker 387 - jepsen.util 387 :invoke :add 1027 INFO [2022-01-10 08:39:14,597] jepsen worker 642 - jepsen.util 642 :ok :add 1025 INFO [2022-01-10 08:39:14,602] jepsen worker 575 - jepsen.util 575 :ok :add 1026 INFO [2022-01-10 08:39:14,602] jepsen worker 739 - jepsen.util 739 :invoke :add 1028 INFO [2022-01-10 08:39:14,605] jepsen worker 832 - jepsen.util 832 :invoke :add 1029 INFO [2022-01-10 08:39:14,608] jepsen worker 515 - jepsen.util 515 :invoke :add 1030 INFO [2022-01-10 08:39:14,610] jepsen worker 739 - jepsen.util 739 :ok :add 1028 INFO [2022-01-10 08:39:14,611] jepsen worker 387 - jepsen.util 387 :ok :add 1027 INFO [2022-01-10 08:39:14,612] jepsen worker 868 - jepsen.util 868 :invoke :add 1031 INFO [2022-01-10 08:39:14,615] jepsen worker 832 - jepsen.util 832 :ok :add 1029 INFO [2022-01-10 08:39:14,615] jepsen worker 562 - jepsen.util 562 :invoke :add 1032 INFO [2022-01-10 08:39:14,617] jepsen worker 515 - jepsen.util 515 :ok :add 1030 INFO [2022-01-10 08:39:14,619] jepsen worker 568 - jepsen.util 568 :invoke :add 1033 INFO [2022-01-10 08:39:14,621] jepsen worker 81 - jepsen.util 81 :invoke :add 1034 INFO [2022-01-10 08:39:14,624] jepsen worker 891 - jepsen.util 891 :invoke :add 1035 INFO [2022-01-10 08:39:14,624] jepsen worker 562 - jepsen.util 562 :ok :add 1032 INFO [2022-01-10 08:39:14,626] jepsen worker 868 - jepsen.util 868 :ok :add 1031 INFO [2022-01-10 08:39:14,628] jepsen worker 907 - jepsen.util 907 :invoke :add 1036 INFO [2022-01-10 08:39:14,631] jepsen worker 252 - jepsen.util 252 :invoke :add 1037 INFO [2022-01-10 08:39:14,632] jepsen worker 81 - jepsen.util 81 :ok :add 1034 INFO [2022-01-10 08:39:14,632] jepsen worker 568 - jepsen.util 568 :ok :add 1033 INFO [2022-01-10 08:39:14,633] jepsen worker 891 - jepsen.util 891 :ok :add 1035 INFO [2022-01-10 08:39:14,634] jepsen worker 89 - jepsen.util 89 :invoke :add 1038 INFO [2022-01-10 08:39:14,638] jepsen worker 135 - jepsen.util 135 :invoke :add 1039 INFO [2022-01-10 08:39:14,641] jepsen worker 53 - jepsen.util 53 :invoke :add 1040 INFO [2022-01-10 08:39:14,643] jepsen worker 907 - jepsen.util 907 :ok :add 1036 INFO [2022-01-10 08:39:14,644] jepsen worker 736 - jepsen.util 736 :invoke :add 1041 INFO [2022-01-10 08:39:14,644] jepsen worker 252 - jepsen.util 252 :ok :add 1037 INFO [2022-01-10 08:39:14,646] jepsen worker 135 - jepsen.util 135 :ok :add 1039 INFO [2022-01-10 08:39:14,648] jepsen worker 657 - jepsen.util 657 :invoke :add 1042 INFO [2022-01-10 08:39:14,649] jepsen worker 53 - jepsen.util 53 :ok :add 1040 INFO [2022-01-10 08:39:14,651] jepsen worker 89 - jepsen.util 89 :ok :add 1038 INFO [2022-01-10 08:39:14,651] jepsen worker 486 - jepsen.util 486 :invoke :add 1043 INFO [2022-01-10 08:39:14,654] jepsen worker 618 - jepsen.util 618 :invoke :add 1044 INFO [2022-01-10 08:39:14,655] jepsen worker 736 - jepsen.util 736 :ok :add 1041 INFO [2022-01-10 08:39:14,660] jepsen worker 486 - jepsen.util 486 :ok :add 1043 INFO [2022-01-10 08:39:14,661] jepsen worker 158 - jepsen.util 158 :invoke :add 1045 INFO [2022-01-10 08:39:14,663] jepsen worker 657 - jepsen.util 657 :ok :add 1042 INFO [2022-01-10 08:39:14,666] jepsen worker 506 - jepsen.util 506 :invoke :add 1046 INFO [2022-01-10 08:39:14,669] jepsen worker 158 - jepsen.util 158 :ok :add 1045 INFO [2022-01-10 08:39:14,670] jepsen worker 618 - jepsen.util 618 :ok :add 1044 INFO [2022-01-10 08:39:14,671] jepsen worker 83 - jepsen.util 83 :invoke :add 1047 INFO [2022-01-10 08:39:14,674] jepsen worker 291 - jepsen.util 291 :invoke :add 1048 INFO [2022-01-10 08:39:14,676] jepsen worker 506 - jepsen.util 506 :ok :add 1046 INFO [2022-01-10 08:39:14,677] jepsen worker 849 - jepsen.util 849 :invoke :add 1049 INFO [2022-01-10 08:39:14,681] jepsen worker 912 - jepsen.util 912 :invoke :add 1050 INFO [2022-01-10 08:39:14,682] jepsen worker 83 - jepsen.util 83 :ok :add 1047 INFO [2022-01-10 08:39:14,684] jepsen worker 291 - jepsen.util 291 :ok :add 1048 INFO [2022-01-10 08:39:14,684] jepsen worker 818 - jepsen.util 818 :invoke :add 1051 INFO [2022-01-10 08:39:14,687] jepsen worker 849 - jepsen.util 849 :ok :add 1049 INFO [2022-01-10 08:39:14,688] jepsen worker 266 - jepsen.util 266 :invoke :add 1052 INFO [2022-01-10 08:39:14,691] jepsen worker 789 - jepsen.util 789 :invoke :add 1053 INFO [2022-01-10 08:39:14,691] jepsen worker 912 - jepsen.util 912 :ok :add 1050 INFO [2022-01-10 08:39:14,695] jepsen worker 698 - jepsen.util 698 :invoke :add 1054 INFO [2022-01-10 08:39:14,695] jepsen worker 818 - jepsen.util 818 :ok :add 1051 INFO [2022-01-10 08:39:14,696] jepsen worker 266 - jepsen.util 266 :ok :add 1052 INFO [2022-01-10 08:39:14,698] jepsen worker 396 - jepsen.util 396 :invoke :add 1055 INFO [2022-01-10 08:39:14,701] jepsen worker 789 - jepsen.util 789 :ok :add 1053 INFO [2022-01-10 08:39:14,702] jepsen worker 698 - jepsen.util 698 :ok :add 1054 INFO [2022-01-10 08:39:14,703] jepsen worker 936 - jepsen.util 936 :invoke :add 1056 INFO [2022-01-10 08:39:14,708] jepsen worker 396 - jepsen.util 396 :ok :add 1055 INFO [2022-01-10 08:39:14,709] jepsen worker 397 - jepsen.util 397 :invoke :add 1057 INFO [2022-01-10 08:39:14,710] jepsen worker 936 - jepsen.util 936 :ok :add 1056 INFO [2022-01-10 08:39:14,712] jepsen worker 190 - jepsen.util 190 :invoke :add 1058 INFO [2022-01-10 08:39:14,715] jepsen worker 818 - jepsen.util 818 :invoke :add 1059 INFO [2022-01-10 08:39:14,716] jepsen worker 397 - jepsen.util 397 :ok :add 1057 INFO [2022-01-10 08:39:14,718] jepsen worker 565 - jepsen.util 565 :invoke :add 1060 INFO [2022-01-10 08:39:14,722] jepsen worker 648 - jepsen.util 648 :invoke :add 1061 INFO [2022-01-10 08:39:14,724] jepsen worker 716 - jepsen.util 716 :invoke :add 1062 INFO [2022-01-10 08:39:14,725] jepsen worker 818 - jepsen.util 818 :ok :add 1059 INFO [2022-01-10 08:39:14,727] jepsen worker 190 - jepsen.util 190 :ok :add 1058 INFO [2022-01-10 08:39:14,728] jepsen worker 565 - jepsen.util 565 :ok :add 1060 INFO [2022-01-10 08:39:14,729] jepsen worker 513 - jepsen.util 513 :invoke :add 1063 INFO [2022-01-10 08:39:14,732] jepsen worker 521 - jepsen.util 521 :invoke :add 1064 INFO [2022-01-10 08:39:14,733] jepsen worker 648 - jepsen.util 648 :ok :add 1061 INFO [2022-01-10 08:39:14,734] jepsen worker 716 - jepsen.util 716 :ok :add 1062 INFO [2022-01-10 08:39:14,735] jepsen worker 773 - jepsen.util 773 :invoke :add 1065 INFO [2022-01-10 08:39:14,737] jepsen worker 382 - jepsen.util 382 :invoke :add 1066 INFO [2022-01-10 08:39:14,742] jepsen worker 649 - jepsen.util 649 :invoke :add 1067 INFO [2022-01-10 08:39:14,743] jepsen worker 773 - jepsen.util 773 :ok :add 1065 INFO [2022-01-10 08:39:14,746] jepsen worker 490 - jepsen.util 490 :invoke :add 1068 INFO [2022-01-10 08:39:14,748] jepsen worker 521 - jepsen.util 521 :ok :add 1064 INFO [2022-01-10 08:39:14,749] jepsen worker 513 - jepsen.util 513 :ok :add 1063 INFO [2022-01-10 08:39:14,750] jepsen worker 309 - jepsen.util 309 :invoke :add 1069 INFO [2022-01-10 08:39:14,752] jepsen worker 649 - jepsen.util 649 :ok :add 1067 INFO [2022-01-10 08:39:14,753] jepsen worker 234 - jepsen.util 234 :invoke :add 1070 INFO [2022-01-10 08:39:14,754] jepsen worker 382 - jepsen.util 382 :ok :add 1066 INFO [2022-01-10 08:39:14,754] jepsen worker 490 - jepsen.util 490 :ok :add 1068 INFO [2022-01-10 08:39:14,756] jepsen worker 578 - jepsen.util 578 :invoke :add 1071 INFO [2022-01-10 08:39:14,759] jepsen worker 177 - jepsen.util 177 :invoke :add 1072 INFO [2022-01-10 08:39:14,761] jepsen worker 878 - jepsen.util 878 :invoke :add 1073 INFO [2022-01-10 08:39:14,764] jepsen worker 234 - jepsen.util 234 :ok :add 1070 INFO [2022-01-10 08:39:14,765] jepsen worker 578 - jepsen.util 578 :ok :add 1071 INFO [2022-01-10 08:39:14,765] jepsen worker 309 - jepsen.util 309 :ok :add 1069 INFO [2022-01-10 08:39:14,765] jepsen worker 890 - jepsen.util 890 :invoke :add 1074 INFO [2022-01-10 08:39:14,769] jepsen worker 850 - jepsen.util 850 :invoke :add 1075 INFO [2022-01-10 08:39:14,770] jepsen worker 878 - jepsen.util 878 :ok :add 1073 INFO [2022-01-10 08:39:14,771] jepsen worker 177 - jepsen.util 177 :ok :add 1072 INFO [2022-01-10 08:39:14,772] jepsen worker 282 - jepsen.util 282 :invoke :add 1076 INFO [2022-01-10 08:39:14,775] jepsen worker 890 - jepsen.util 890 :ok :add 1074 INFO [2022-01-10 08:39:14,777] jepsen worker 359 - jepsen.util 359 :invoke :add 1077 INFO [2022-01-10 08:39:14,778] jepsen worker 850 - jepsen.util 850 :ok :add 1075 INFO [2022-01-10 08:39:14,780] jepsen worker 223 - jepsen.util 223 :invoke :add 1078 INFO [2022-01-10 08:39:14,783] jepsen worker 303 - jepsen.util 303 :invoke :add 1079 INFO [2022-01-10 08:39:14,785] jepsen worker 269 - jepsen.util 269 :invoke :add 1080 INFO [2022-01-10 08:39:14,785] jepsen worker 282 - jepsen.util 282 :ok :add 1076 INFO [2022-01-10 08:39:14,788] jepsen worker 297 - jepsen.util 297 :invoke :add 1081 INFO [2022-01-10 08:39:14,788] jepsen worker 359 - jepsen.util 359 :ok :add 1077 INFO [2022-01-10 08:39:14,789] jepsen worker 223 - jepsen.util 223 :ok :add 1078 INFO [2022-01-10 08:39:14,789] jepsen worker 303 - jepsen.util 303 :ok :add 1079 INFO [2022-01-10 08:39:14,790] jepsen worker 36 - jepsen.util 36 :invoke :add 1082 INFO [2022-01-10 08:39:14,793] jepsen worker 43 - jepsen.util 43 :invoke :add 1083 INFO [2022-01-10 08:39:14,796] jepsen worker 496 - jepsen.util 496 :invoke :add 1084 INFO [2022-01-10 08:39:14,799] jepsen worker 958 - jepsen.util 958 :invoke :add 1085 INFO [2022-01-10 08:39:14,800] jepsen worker 269 - jepsen.util 269 :ok :add 1080 INFO [2022-01-10 08:39:14,802] jepsen worker 297 - jepsen.util 297 :ok :add 1081 INFO [2022-01-10 08:39:14,805] jepsen worker 496 - jepsen.util 496 :ok :add 1084 INFO [2022-01-10 08:39:14,805] jepsen worker 43 - jepsen.util 43 :ok :add 1083 INFO [2022-01-10 08:39:14,806] jepsen worker 499 - jepsen.util 499 :invoke :add 1086 INFO [2022-01-10 08:39:14,807] jepsen worker 36 - jepsen.util 36 :ok :add 1082 INFO [2022-01-10 08:39:14,809] jepsen worker 958 - jepsen.util 958 :ok :add 1085 INFO [2022-01-10 08:39:14,810] jepsen worker 446 - jepsen.util 446 :invoke :add 1087 INFO [2022-01-10 08:39:14,814] jepsen worker 638 - jepsen.util 638 :invoke :add 1088 INFO [2022-01-10 08:39:14,816] jepsen worker 76 - jepsen.util 76 :invoke :add 1089 INFO [2022-01-10 08:39:14,819] jepsen worker 446 - jepsen.util 446 :ok :add 1087 INFO [2022-01-10 08:39:14,819] jepsen worker 107 - jepsen.util 107 :invoke :add 1090 INFO [2022-01-10 08:39:14,822] jepsen worker 638 - jepsen.util 638 :ok :add 1088 INFO [2022-01-10 08:39:14,823] jepsen worker 402 - jepsen.util 402 :invoke :add 1091 INFO [2022-01-10 08:39:14,825] jepsen worker 76 - jepsen.util 76 :ok :add 1089 INFO [2022-01-10 08:39:14,826] jepsen worker 139 - jepsen.util 139 :invoke :add 1092 INFO [2022-01-10 08:39:14,828] jepsen worker 107 - jepsen.util 107 :ok :add 1090 INFO [2022-01-10 08:39:14,828] jepsen worker 499 - jepsen.util 499 :ok :add 1086 INFO [2022-01-10 08:39:14,830] jepsen worker 807 - jepsen.util 807 :invoke :add 1093 INFO [2022-01-10 08:39:14,832] jepsen worker 402 - jepsen.util 402 :ok :add 1091 INFO [2022-01-10 08:39:14,835] jepsen worker 54 - jepsen.util 54 :invoke :add 1094 INFO [2022-01-10 08:39:14,839] jepsen worker 685 - jepsen.util 685 :invoke :add 1095 INFO [2022-01-10 08:39:14,840] jepsen worker 139 - jepsen.util 139 :ok :add 1092 INFO [2022-01-10 08:39:14,841] jepsen worker 788 - jepsen.util 788 :invoke :add 1096 INFO [2022-01-10 08:39:14,842] jepsen worker 807 - jepsen.util 807 :ok :add 1093 INFO [2022-01-10 08:39:14,845] jepsen worker 738 - jepsen.util 738 :invoke :add 1097 INFO [2022-01-10 08:39:14,849] jepsen worker 295 - jepsen.util 295 :invoke :add 1098 INFO [2022-01-10 08:39:14,853] jepsen worker 262 - jepsen.util 262 :invoke :add 1099 INFO [2022-01-10 08:39:14,857] jepsen worker 295 - jepsen.util 295 :ok :add 1098 INFO [2022-01-10 08:39:14,857] jepsen worker 153 - jepsen.util 153 :invoke :add 1100 INFO [2022-01-10 08:39:14,858] jepsen worker 685 - jepsen.util 685 :ok :add 1095 INFO [2022-01-10 08:39:14,858] jepsen worker 54 - jepsen.util 54 :ok :add 1094 INFO [2022-01-10 08:39:14,859] jepsen worker 262 - jepsen.util 262 :ok :add 1099 INFO [2022-01-10 08:39:14,862] jepsen worker 269 - jepsen.util 269 :invoke :add 1101 INFO [2022-01-10 08:39:14,866] jepsen worker 114 - jepsen.util 114 :invoke :add 1102 INFO [2022-01-10 08:39:14,867] jepsen worker 153 - jepsen.util 153 :ok :add 1100 INFO [2022-01-10 08:39:14,869] jepsen worker 499 - jepsen.util 499 :invoke :add 1103 INFO [2022-01-10 08:39:14,871] jepsen worker 738 - jepsen.util 738 :ok :add 1097 INFO [2022-01-10 08:39:14,871] jepsen worker 788 - jepsen.util 788 :ok :add 1096 INFO [2022-01-10 08:39:14,872] jepsen worker 37 - jepsen.util 37 :invoke :add 1104 INFO [2022-01-10 08:39:14,873] jepsen worker 269 - jepsen.util 269 :ok :add 1101 INFO [2022-01-10 08:39:14,875] jepsen worker 948 - jepsen.util 948 :invoke :add 1105 INFO [2022-01-10 08:39:14,879] jepsen worker 524 - jepsen.util 524 :invoke :add 1106 INFO [2022-01-10 08:39:14,880] jepsen worker 114 - jepsen.util 114 :ok :add 1102 INFO [2022-01-10 08:39:14,882] jepsen worker 499 - jepsen.util 499 :ok :add 1103 INFO [2022-01-10 08:39:14,882] jepsen worker 264 - jepsen.util 264 :invoke :add 1107 INFO [2022-01-10 08:39:14,886] jepsen worker 619 - jepsen.util 619 :invoke :add 1108 INFO [2022-01-10 08:39:14,889] jepsen worker 37 - jepsen.util 37 :ok :add 1104 INFO [2022-01-10 08:39:14,890] jepsen worker 916 - jepsen.util 916 :invoke :add 1109 INFO [2022-01-10 08:39:14,892] jepsen worker 948 - jepsen.util 948 :ok :add 1105 INFO [2022-01-10 08:39:14,892] jepsen worker 619 - jepsen.util 619 :ok :add 1108 INFO [2022-01-10 08:39:14,893] jepsen worker 524 - jepsen.util 524 :ok :add 1106 INFO [2022-01-10 08:39:14,895] jepsen worker 38 - jepsen.util 38 :invoke :add 1110 INFO [2022-01-10 08:39:14,896] jepsen worker 264 - jepsen.util 264 :ok :add 1107 INFO [2022-01-10 08:39:14,899] jepsen worker 340 - jepsen.util 340 :invoke :add 1111 INFO [2022-01-10 08:39:14,902] jepsen worker 369 - jepsen.util 369 :invoke :add 1112 INFO [2022-01-10 08:39:14,906] jepsen worker 269 - jepsen.util 269 :invoke :add 1113 INFO [2022-01-10 08:39:14,909] jepsen worker 974 - jepsen.util 974 :invoke :add 1114 INFO [2022-01-10 08:39:14,912] jepsen worker 296 - jepsen.util 296 :invoke :add 1115 INFO [2022-01-10 08:39:14,914] jepsen worker 430 - jepsen.util 430 :invoke :add 1116 INFO [2022-01-10 08:39:14,917] jepsen worker 674 - jepsen.util 674 :invoke :add 1117 INFO [2022-01-10 08:39:14,918] jepsen worker 38 - jepsen.util 38 :ok :add 1110 INFO [2022-01-10 08:39:14,918] jepsen worker 974 - jepsen.util 974 :ok :add 1114 INFO [2022-01-10 08:39:14,919] jepsen worker 506 - jepsen.util 506 :invoke :add 1118 INFO [2022-01-10 08:39:14,921] jepsen worker 916 - jepsen.util 916 :ok :add 1109 INFO [2022-01-10 08:39:14,921] jepsen worker 296 - jepsen.util 296 :ok :add 1115 INFO [2022-01-10 08:39:14,922] jepsen worker 340 - jepsen.util 340 :ok :add 1111 INFO [2022-01-10 08:39:14,922] jepsen worker 337 - jepsen.util 337 :invoke :add 1119 INFO [2022-01-10 08:39:14,923] jepsen worker 369 - jepsen.util 369 :ok :add 1112 INFO [2022-01-10 08:39:14,926] jepsen worker 674 - jepsen.util 674 :ok :add 1117 INFO [2022-01-10 08:39:14,926] jepsen worker 491 - jepsen.util 491 :invoke :add 1120 INFO [2022-01-10 08:39:14,927] jepsen worker 269 - jepsen.util 269 :ok :add 1113 INFO [2022-01-10 08:39:14,927] jepsen worker 430 - jepsen.util 430 :ok :add 1116 INFO [2022-01-10 08:39:14,929] jepsen worker 506 - jepsen.util 506 :ok :add 1118 INFO [2022-01-10 08:39:14,930] jepsen worker 248 - jepsen.util 248 :invoke :add 1121 INFO [2022-01-10 08:39:14,931] jepsen worker 337 - jepsen.util 337 :ok :add 1119 INFO [2022-01-10 08:39:14,934] jepsen worker 284 - jepsen.util 284 :invoke :add 1122 INFO [2022-01-10 08:39:14,936] jepsen worker 952 - jepsen.util 952 :invoke :add 1123 INFO [2022-01-10 08:39:14,938] jepsen worker 248 - jepsen.util 248 :ok :add 1121 INFO [2022-01-10 08:39:14,939] jepsen worker 553 - jepsen.util 553 :invoke :add 1124 INFO [2022-01-10 08:39:14,942] jepsen worker 269 - jepsen.util 269 :invoke :add 1125 INFO [2022-01-10 08:39:14,944] jepsen worker 44 - jepsen.util 44 :invoke :add 1126 INFO [2022-01-10 08:39:14,945] jepsen worker 284 - jepsen.util 284 :ok :add 1122 INFO [2022-01-10 08:39:14,947] jepsen worker 952 - jepsen.util 952 :ok :add 1123 INFO [2022-01-10 08:39:14,948] jepsen worker 652 - jepsen.util 652 :invoke :add 1127 INFO [2022-01-10 08:39:14,951] jepsen worker 981 - jepsen.util 981 :invoke :add 1128 INFO [2022-01-10 08:39:14,953] jepsen worker 553 - jepsen.util 553 :ok :add 1124 INFO [2022-01-10 08:39:14,955] jepsen worker 936 - jepsen.util 936 :invoke :add 1129 INFO [2022-01-10 08:39:14,957] jepsen worker 269 - jepsen.util 269 :ok :add 1125 INFO [2022-01-10 08:39:14,957] jepsen worker 491 - jepsen.util 491 :ok :add 1120 INFO [2022-01-10 08:39:14,958] jepsen worker 512 - jepsen.util 512 :invoke :add 1130 INFO [2022-01-10 08:39:14,959] jepsen worker 981 - jepsen.util 981 :ok :add 1128 INFO [2022-01-10 08:39:14,959] jepsen worker 44 - jepsen.util 44 :ok :add 1126 INFO [2022-01-10 08:39:14,962] jepsen worker 63 - jepsen.util 63 :invoke :add 1131 INFO [2022-01-10 08:39:14,965] jepsen worker 652 - jepsen.util 652 :ok :add 1127 INFO [2022-01-10 08:39:14,965] jepsen worker 891 - jepsen.util 891 :invoke :add 1132 INFO [2022-01-10 08:39:14,967] jepsen worker 936 - jepsen.util 936 :ok :add 1129 INFO [2022-01-10 08:39:14,969] jepsen worker 243 - jepsen.util 243 :invoke :add 1133 INFO [2022-01-10 08:39:14,972] jepsen worker 601 - jepsen.util 601 :invoke :add 1134 INFO [2022-01-10 08:39:14,975] jepsen worker 418 - jepsen.util 418 :invoke :add 1135 INFO [2022-01-10 08:39:14,976] jepsen worker 512 - jepsen.util 512 :ok :add 1130 INFO [2022-01-10 08:39:14,977] jepsen worker 63 - jepsen.util 63 :ok :add 1131 INFO [2022-01-10 08:39:14,979] jepsen worker 383 - jepsen.util 383 :invoke :add 1136 INFO [2022-01-10 08:39:14,982] jepsen worker 891 - jepsen.util 891 :ok :add 1132 INFO [2022-01-10 08:39:14,983] jepsen worker 243 - jepsen.util 243 :ok :add 1133 INFO [2022-01-10 08:39:14,984] jepsen worker 867 - jepsen.util 867 :invoke :add 1137 INFO [2022-01-10 08:39:14,986] jepsen worker 601 - jepsen.util 601 :ok :add 1134 INFO [2022-01-10 08:39:14,988] jepsen worker 418 - jepsen.util 418 :ok :add 1135 INFO [2022-01-10 08:39:14,989] jepsen worker 49 - jepsen.util 49 :invoke :add 1138 INFO [2022-01-10 08:39:14,992] jepsen worker 383 - jepsen.util 383 :ok :add 1136 INFO [2022-01-10 08:39:14,992] jepsen worker 753 - jepsen.util 753 :invoke :add 1139 INFO [2022-01-10 08:39:14,996] jepsen worker 867 - jepsen.util 867 :ok :add 1137 INFO [2022-01-10 08:39:14,997] jepsen worker 271 - jepsen.util 271 :invoke :add 1140 INFO [2022-01-10 08:39:14,998] jepsen worker 49 - jepsen.util 49 :ok :add 1138 INFO [2022-01-10 08:39:15,000] jepsen worker 263 - jepsen.util 263 :invoke :add 1141 INFO [2022-01-10 08:39:15,001] jepsen worker 753 - jepsen.util 753 :ok :add 1139 INFO [2022-01-10 08:39:15,003] jepsen worker 389 - jepsen.util 389 :invoke :add 1142 INFO [2022-01-10 08:39:15,007] jepsen worker 198 - jepsen.util 198 :invoke :add 1143 INFO [2022-01-10 08:39:15,010] jepsen worker 932 - jepsen.util 932 :invoke :add 1144 INFO [2022-01-10 08:39:15,012] jepsen worker 271 - jepsen.util 271 :ok :add 1140 INFO [2022-01-10 08:39:15,012] jepsen worker 263 - jepsen.util 263 :ok :add 1141 INFO [2022-01-10 08:39:15,015] jepsen worker 740 - jepsen.util 740 :invoke :add 1145 INFO [2022-01-10 08:39:15,018] jepsen worker 389 - jepsen.util 389 :ok :add 1142 INFO [2022-01-10 08:39:15,019] jepsen worker 198 - jepsen.util 198 :ok :add 1143 INFO [2022-01-10 08:39:15,019] jepsen worker 855 - jepsen.util 855 :invoke :add 1146 INFO [2022-01-10 08:39:15,020] jepsen worker 932 - jepsen.util 932 :ok :add 1144 INFO [2022-01-10 08:39:15,022] jepsen worker 66 - jepsen.util 66 :invoke :add 1147 INFO [2022-01-10 08:39:15,025] jepsen worker 695 - jepsen.util 695 :invoke :add 1148 INFO [2022-01-10 08:39:15,027] jepsen worker 740 - jepsen.util 740 :ok :add 1145 INFO [2022-01-10 08:39:15,028] jepsen worker 42 - jepsen.util 42 :invoke :add 1149 INFO [2022-01-10 08:39:15,030] jepsen worker 237 - jepsen.util 237 :invoke :add 1150 INFO [2022-01-10 08:39:15,031] jepsen worker 855 - jepsen.util 855 :ok :add 1146 INFO [2022-01-10 08:39:15,033] jepsen worker 265 - jepsen.util 265 :invoke :add 1151 INFO [2022-01-10 08:39:15,036] jepsen worker 66 - jepsen.util 66 :ok :add 1147 INFO [2022-01-10 08:39:15,036] jepsen worker 695 - jepsen.util 695 :ok :add 1148 INFO [2022-01-10 08:39:15,037] jepsen worker 42 - jepsen.util 42 :ok :add 1149 INFO [2022-01-10 08:39:15,041] jepsen worker 237 - jepsen.util 237 :ok :add 1150 INFO [2022-01-10 08:39:15,041] jepsen worker 800 - jepsen.util 800 :invoke :add 1152 INFO [2022-01-10 08:39:15,041] jepsen worker 265 - jepsen.util 265 :ok :add 1151 INFO [2022-01-10 08:39:15,047] jepsen worker 800 - jepsen.util 800 :ok :add 1152 INFO [2022-01-10 08:39:15,047] jepsen worker 438 - jepsen.util 438 :invoke :add 1153 INFO [2022-01-10 08:39:15,050] jepsen worker 949 - jepsen.util 949 :invoke :add 1154 INFO [2022-01-10 08:39:15,052] jepsen worker 868 - jepsen.util 868 :invoke :add 1155 INFO [2022-01-10 08:39:15,055] jepsen worker 438 - jepsen.util 438 :ok :add 1153 INFO [2022-01-10 08:39:15,055] jepsen worker 835 - jepsen.util 835 :invoke :add 1156 INFO [2022-01-10 08:39:15,058] jepsen worker 910 - jepsen.util 910 :invoke :add 1157 INFO [2022-01-10 08:39:15,060] jepsen worker 949 - jepsen.util 949 :ok :add 1154 INFO [2022-01-10 08:39:15,061] jepsen worker 221 - jepsen.util 221 :invoke :add 1158 INFO [2022-01-10 08:39:15,063] jepsen worker 510 - jepsen.util 510 :invoke :add 1159 INFO [2022-01-10 08:39:15,063] jepsen worker 868 - jepsen.util 868 :ok :add 1155 INFO [2022-01-10 08:39:15,064] jepsen worker 713 - jepsen.util 713 :invoke :add 1160 INFO [2022-01-10 08:39:15,067] jepsen worker 985 - jepsen.util 985 :invoke :add 1161 INFO [2022-01-10 08:39:15,068] jepsen worker 221 - jepsen.util 221 :ok :add 1158 INFO [2022-01-10 08:39:15,069] jepsen worker 426 - jepsen.util 426 :invoke :add 1162 INFO [2022-01-10 08:39:15,071] jepsen worker 510 - jepsen.util 510 :ok :add 1159 INFO [2022-01-10 08:39:15,071] jepsen worker 835 - jepsen.util 835 :ok :add 1156 INFO [2022-01-10 08:39:15,073] jepsen worker 910 - jepsen.util 910 :ok :add 1157 INFO [2022-01-10 08:39:15,073] jepsen worker 590 - jepsen.util 590 :invoke :add 1163 INFO [2022-01-10 08:39:15,076] jepsen worker 713 - jepsen.util 713 :ok :add 1160 INFO [2022-01-10 08:39:15,077] jepsen worker 649 - jepsen.util 649 :invoke :add 1164 INFO [2022-01-10 08:39:15,080] jepsen worker 704 - jepsen.util 704 :invoke :add 1165 INFO [2022-01-10 08:39:15,083] jepsen worker 426 - jepsen.util 426 :ok :add 1162 INFO [2022-01-10 08:39:15,083] jepsen worker 924 - jepsen.util 924 :invoke :add 1166 INFO [2022-01-10 08:39:15,084] jepsen worker 985 - jepsen.util 985 :ok :add 1161 INFO [2022-01-10 08:39:15,085] jepsen worker 649 - jepsen.util 649 :ok :add 1164 INFO [2022-01-10 08:39:15,085] jepsen worker 916 - jepsen.util 916 :invoke :add 1167 INFO [2022-01-10 08:39:15,088] jepsen worker 590 - jepsen.util 590 :ok :add 1163 INFO [2022-01-10 08:39:15,089] jepsen worker 868 - jepsen.util 868 :invoke :add 1168 INFO [2022-01-10 08:39:15,092] jepsen worker 537 - jepsen.util 537 :invoke :add 1169 INFO [2022-01-10 08:39:15,096] jepsen worker 852 - jepsen.util 852 :invoke :add 1170 INFO [2022-01-10 08:39:15,097] jepsen worker 704 - jepsen.util 704 :ok :add 1165 INFO [2022-01-10 08:39:15,098] jepsen worker 804 - jepsen.util 804 :invoke :add 1171 INFO [2022-01-10 08:39:15,099] jepsen worker 868 - jepsen.util 868 :ok :add 1168 INFO [2022-01-10 08:39:15,101] jepsen worker 853 - jepsen.util 853 :invoke :add 1172 INFO [2022-01-10 08:39:15,104] jepsen worker 537 - jepsen.util 537 :ok :add 1169 INFO [2022-01-10 08:39:15,104] jepsen worker 310 - jepsen.util 310 :invoke :add 1173 INFO [2022-01-10 08:39:15,106] jepsen worker 852 - jepsen.util 852 :ok :add 1170 INFO [2022-01-10 08:39:15,108] jepsen worker 924 - jepsen.util 924 :ok :add 1166 INFO [2022-01-10 08:39:15,108] jepsen worker 466 - jepsen.util 466 :invoke :add 1174 INFO [2022-01-10 08:39:15,111] jepsen worker 916 - jepsen.util 916 :ok :add 1167 INFO [2022-01-10 08:39:15,112] jepsen worker 853 - jepsen.util 853 :ok :add 1172 INFO [2022-01-10 08:39:15,112] jepsen worker 804 - jepsen.util 804 :ok :add 1171 INFO [2022-01-10 08:39:15,114] jepsen worker 567 - jepsen.util 567 :invoke :add 1175 INFO [2022-01-10 08:39:15,117] jepsen worker 310 - jepsen.util 310 :ok :add 1173 INFO [2022-01-10 08:39:15,118] jepsen worker 691 - jepsen.util 691 :invoke :add 1176 INFO [2022-01-10 08:39:15,119] jepsen worker 466 - jepsen.util 466 :ok :add 1174 INFO [2022-01-10 08:39:15,121] jepsen worker 257 - jepsen.util 257 :invoke :add 1177 INFO [2022-01-10 08:39:15,124] jepsen worker 11 - jepsen.util 11 :invoke :add 1178 INFO [2022-01-10 08:39:15,127] jepsen worker 992 - jepsen.util 992 :invoke :add 1179 INFO [2022-01-10 08:39:15,129] jepsen worker 567 - jepsen.util 567 :ok :add 1175 INFO [2022-01-10 08:39:15,131] jepsen worker 721 - jepsen.util 721 :invoke :add 1180 INFO [2022-01-10 08:39:15,133] jepsen worker 691 - jepsen.util 691 :ok :add 1176 INFO [2022-01-10 08:39:15,133] jepsen worker 257 - jepsen.util 257 :ok :add 1177 INFO [2022-01-10 08:39:15,134] jepsen worker 11 - jepsen.util 11 :ok :add 1178 INFO [2022-01-10 08:39:15,134] jepsen worker 992 - jepsen.util 992 :ok :add 1179 INFO [2022-01-10 08:39:15,135] jepsen worker 388 - jepsen.util 388 :invoke :add 1181 INFO [2022-01-10 08:39:15,138] jepsen worker 85 - jepsen.util 85 :invoke :add 1182 INFO [2022-01-10 08:39:15,140] jepsen worker 641 - jepsen.util 641 :invoke :add 1183 INFO [2022-01-10 08:39:15,143] jepsen worker 453 - jepsen.util 453 :invoke :add 1184 INFO [2022-01-10 08:39:15,146] jepsen worker 76 - jepsen.util 76 :invoke :add 1185 INFO [2022-01-10 08:39:15,149] jepsen worker 588 - jepsen.util 588 :invoke :add 1186 INFO [2022-01-10 08:39:15,151] jepsen worker 453 - jepsen.util 453 :ok :add 1184 INFO [2022-01-10 08:39:15,153] jepsen worker 402 - jepsen.util 402 :invoke :add 1187 INFO [2022-01-10 08:39:15,157] jepsen worker 779 - jepsen.util 779 :invoke :add 1188 INFO [2022-01-10 08:39:15,157] jepsen worker 76 - jepsen.util 76 :ok :add 1185 INFO [2022-01-10 08:39:15,161] jepsen worker 413 - jepsen.util 413 :invoke :add 1189 INFO [2022-01-10 08:39:15,162] jepsen worker 721 - jepsen.util 721 :ok :add 1180 INFO [2022-01-10 08:39:15,163] jepsen worker 388 - jepsen.util 388 :ok :add 1181 INFO [2022-01-10 08:39:15,164] jepsen worker 402 - jepsen.util 402 :ok :add 1187 INFO [2022-01-10 08:39:15,165] jepsen worker 2 - jepsen.util 2 :invoke :add 1190 INFO [2022-01-10 08:39:15,166] jepsen worker 779 - jepsen.util 779 :ok :add 1188 INFO [2022-01-10 08:39:15,167] jepsen worker 641 - jepsen.util 641 :ok :add 1183 INFO [2022-01-10 08:39:15,167] jepsen worker 85 - jepsen.util 85 :ok :add 1182 INFO [2022-01-10 08:39:15,168] jepsen worker 588 - jepsen.util 588 :ok :add 1186 INFO [2022-01-10 08:39:15,171] jepsen worker 841 - jepsen.util 841 :invoke :add 1191 INFO [2022-01-10 08:39:15,175] jepsen worker 2 - jepsen.util 2 :ok :add 1190 INFO [2022-01-10 08:39:15,175] jepsen worker 366 - jepsen.util 366 :invoke :add 1192 INFO [2022-01-10 08:39:15,176] jepsen worker 413 - jepsen.util 413 :ok :add 1189 INFO [2022-01-10 08:39:15,178] jepsen worker 990 - jepsen.util 990 :invoke :add 1193 INFO [2022-01-10 08:39:15,181] jepsen worker 290 - jepsen.util 290 :invoke :add 1194 INFO [2022-01-10 08:39:15,183] jepsen worker 26 - jepsen.util 26 :invoke :add 1195 INFO [2022-01-10 08:39:15,186] jepsen worker 116 - jepsen.util 116 :invoke :add 1196 INFO [2022-01-10 08:39:15,189] jepsen worker 861 - jepsen.util 861 :invoke :add 1197 INFO [2022-01-10 08:39:15,191] jepsen worker 737 - jepsen.util 737 :invoke :add 1198 INFO [2022-01-10 08:39:15,194] jepsen worker 377 - jepsen.util 377 :invoke :add 1199 INFO [2022-01-10 08:39:15,196] jepsen worker 424 - jepsen.util 424 :invoke :add 1200 INFO [2022-01-10 08:39:15,198] jepsen worker 0 - jepsen.util 0 :invoke :add 1201 INFO [2022-01-10 08:39:15,200] jepsen worker 584 - jepsen.util 584 :invoke :add 1202 INFO [2022-01-10 08:39:15,202] jepsen worker 549 - jepsen.util 549 :invoke :add 1203 INFO [2022-01-10 08:39:15,206] jepsen worker 597 - jepsen.util 597 :invoke :add 1204 INFO [2022-01-10 08:39:15,207] jepsen worker 26 - jepsen.util 26 :ok :add 1195 INFO [2022-01-10 08:39:15,210] jepsen worker 737 - jepsen.util 737 :ok :add 1198 INFO [2022-01-10 08:39:15,211] jepsen worker 377 - jepsen.util 377 :ok :add 1199 INFO [2022-01-10 08:39:15,211] jepsen worker 811 - jepsen.util 811 :invoke :add 1205 INFO [2022-01-10 08:39:15,214] jepsen worker 290 - jepsen.util 290 :ok :add 1194 INFO [2022-01-10 08:39:15,215] jepsen worker 543 - jepsen.util 543 :invoke :add 1206 INFO [2022-01-10 08:39:15,217] jepsen worker 199 - jepsen.util 199 :invoke :add 1207 INFO [2022-01-10 08:39:15,220] jepsen worker 804 - jepsen.util 804 :invoke :add 1208 INFO [2022-01-10 08:39:15,220] jepsen worker 366 - jepsen.util 366 :ok :add 1192 INFO [2022-01-10 08:39:15,221] jepsen worker 116 - jepsen.util 116 :ok :add 1196 INFO [2022-01-10 08:39:15,222] jepsen worker 824 - jepsen.util 824 :invoke :add 1209 INFO [2022-01-10 08:39:15,223] jepsen worker 861 - jepsen.util 861 :ok :add 1197 INFO [2022-01-10 08:39:15,223] jepsen worker 841 - jepsen.util 841 :ok :add 1191 INFO [2022-01-10 08:39:15,225] jepsen worker 862 - jepsen.util 862 :invoke :add 1210 INFO [2022-01-10 08:39:15,227] jepsen worker 424 - jepsen.util 424 :ok :add 1200 INFO [2022-01-10 08:39:15,229] jepsen worker 779 - jepsen.util 779 :invoke :add 1211 INFO [2022-01-10 08:39:15,229] jepsen worker 597 - jepsen.util 597 :ok :add 1204 INFO [2022-01-10 08:39:15,232] jepsen worker 204 - jepsen.util 204 :invoke :add 1212 INFO [2022-01-10 08:39:15,232] jepsen worker 990 - jepsen.util 990 :ok :add 1193 INFO [2022-01-10 08:39:15,234] jepsen worker 188 - jepsen.util 188 :invoke :add 1213 INFO [2022-01-10 08:39:15,238] jepsen worker 362 - jepsen.util 362 :invoke :add 1214 INFO [2022-01-10 08:39:15,241] jepsen worker 494 - jepsen.util 494 :invoke :add 1215 INFO [2022-01-10 08:39:15,242] jepsen worker 811 - jepsen.util 811 :ok :add 1205 INFO [2022-01-10 08:39:15,243] jepsen worker 779 - jepsen.util 779 :ok :add 1211 INFO [2022-01-10 08:39:15,243] jepsen worker 824 - jepsen.util 824 :ok :add 1209 INFO [2022-01-10 08:39:15,243] jepsen worker 584 - jepsen.util 584 :ok :add 1202 INFO [2022-01-10 08:39:15,245] jepsen worker 867 - jepsen.util 867 :invoke :add 1216 INFO [2022-01-10 08:39:15,245] jepsen worker 0 - jepsen.util 0 :ok :add 1201 INFO [2022-01-10 08:39:15,246] jepsen worker 362 - jepsen.util 362 :ok :add 1214 INFO [2022-01-10 08:39:15,247] jepsen worker 804 - jepsen.util 804 :ok :add 1208 INFO [2022-01-10 08:39:15,247] jepsen worker 862 - jepsen.util 862 :ok :add 1210 INFO [2022-01-10 08:39:15,249] jepsen worker 188 - jepsen.util 188 :ok :add 1213 INFO [2022-01-10 08:39:15,249] jepsen worker 199 - jepsen.util 199 :ok :add 1207 INFO [2022-01-10 08:39:15,251] jepsen worker 549 - jepsen.util 549 :ok :add 1203 INFO [2022-01-10 08:39:15,252] jepsen worker 394 - jepsen.util 394 :invoke :add 1217 INFO [2022-01-10 08:39:15,255] jepsen worker 4 - jepsen.util 4 :invoke :add 1218 INFO [2022-01-10 08:39:15,255] jepsen worker 543 - jepsen.util 543 :ok :add 1206 INFO [2022-01-10 08:39:15,258] jepsen worker 219 - jepsen.util 219 :invoke :add 1219 INFO [2022-01-10 08:39:15,258] jepsen worker 494 - jepsen.util 494 :ok :add 1215 INFO [2022-01-10 08:39:15,261] jepsen worker 100 - jepsen.util 100 :invoke :add 1220 INFO [2022-01-10 08:39:15,262] jepsen worker 204 - jepsen.util 204 :ok :add 1212 INFO [2022-01-10 08:39:15,263] jepsen worker 690 - jepsen.util 690 :invoke :add 1221 INFO [2022-01-10 08:39:15,266] jepsen worker 4 - jepsen.util 4 :ok :add 1218 INFO [2022-01-10 08:39:15,268] jepsen worker 475 - jepsen.util 475 :invoke :add 1222 INFO [2022-01-10 08:39:15,268] jepsen worker 867 - jepsen.util 867 :ok :add 1216 INFO [2022-01-10 08:39:15,271] jepsen worker 283 - jepsen.util 283 :invoke :add 1223 INFO [2022-01-10 08:39:15,273] jepsen worker 219 - jepsen.util 219 :ok :add 1219 INFO [2022-01-10 08:39:15,274] jepsen worker 940 - jepsen.util 940 :invoke :add 1224 INFO [2022-01-10 08:39:15,277] jepsen worker 602 - jepsen.util 602 :invoke :add 1225 INFO [2022-01-10 08:39:15,280] jepsen worker 149 - jepsen.util 149 :invoke :add 1226 INFO [2022-01-10 08:39:15,283] jepsen worker 100 - jepsen.util 100 :ok :add 1220 INFO [2022-01-10 08:39:15,284] jepsen worker 843 - jepsen.util 843 :invoke :add 1227 INFO [2022-01-10 08:39:15,287] jepsen worker 632 - jepsen.util 632 :invoke :add 1228 INFO [2022-01-10 08:39:15,287] jepsen worker 394 - jepsen.util 394 :ok :add 1217 INFO [2022-01-10 08:39:15,287] jepsen worker 283 - jepsen.util 283 :ok :add 1223 INFO [2022-01-10 08:39:15,289] jepsen worker 475 - jepsen.util 475 :ok :add 1222 INFO [2022-01-10 08:39:15,289] jepsen worker 207 - jepsen.util 207 :invoke :add 1229 INFO [2022-01-10 08:39:15,291] jepsen worker 149 - jepsen.util 149 :ok :add 1226 INFO [2022-01-10 08:39:15,293] jepsen worker 66 - jepsen.util 66 :invoke :add 1230 INFO [2022-01-10 08:39:15,294] jepsen worker 690 - jepsen.util 690 :ok :add 1221 INFO [2022-01-10 08:39:15,295] jepsen worker 940 - jepsen.util 940 :ok :add 1224 INFO [2022-01-10 08:39:15,297] jepsen worker 581 - jepsen.util 581 :invoke :add 1231 INFO [2022-01-10 08:39:15,300] jepsen worker 885 - jepsen.util 885 :invoke :add 1232 INFO [2022-01-10 08:39:15,302] jepsen worker 679 - jepsen.util 679 :invoke :add 1233 INFO [2022-01-10 08:39:15,304] jepsen worker 620 - jepsen.util 620 :invoke :add 1234 INFO [2022-01-10 08:39:15,307] jepsen worker 469 - jepsen.util 469 :invoke :add 1235 INFO [2022-01-10 08:39:15,312] jepsen worker 537 - jepsen.util 537 :invoke :add 1236 INFO [2022-01-10 08:39:15,313] jepsen worker 679 - jepsen.util 679 :ok :add 1233 INFO [2022-01-10 08:39:15,313] jepsen worker 602 - jepsen.util 602 :ok :add 1225 INFO [2022-01-10 08:39:15,316] jepsen worker 469 - jepsen.util 469 :ok :add 1235 INFO [2022-01-10 08:39:15,316] jepsen worker 352 - jepsen.util 352 :invoke :add 1237 INFO [2022-01-10 08:39:15,319] jepsen worker 632 - jepsen.util 632 :ok :add 1228 INFO [2022-01-10 08:39:15,319] jepsen worker 66 - jepsen.util 66 :ok :add 1230 INFO [2022-01-10 08:39:15,319] jepsen worker 843 - jepsen.util 843 :ok :add 1227 INFO [2022-01-10 08:39:15,322] jepsen worker 838 - jepsen.util 838 :invoke :add 1238 INFO [2022-01-10 08:39:15,325] jepsen worker 620 - jepsen.util 620 :ok :add 1234 INFO [2022-01-10 08:39:15,327] jepsen worker 214 - jepsen.util 214 :invoke :add 1239 INFO [2022-01-10 08:39:15,328] jepsen worker 352 - jepsen.util 352 :ok :add 1237 INFO [2022-01-10 08:39:15,331] jepsen worker 838 - jepsen.util 838 :ok :add 1238 INFO [2022-01-10 08:39:15,331] jepsen worker 292 - jepsen.util 292 :invoke :add 1240 INFO [2022-01-10 08:39:15,332] jepsen worker 885 - jepsen.util 885 :ok :add 1232 INFO [2022-01-10 08:39:15,333] jepsen worker 537 - jepsen.util 537 :ok :add 1236 INFO [2022-01-10 08:39:15,335] jepsen worker 12 - jepsen.util 12 :invoke :add 1241 INFO [2022-01-10 08:39:15,336] jepsen worker 214 - jepsen.util 214 :ok :add 1239 INFO [2022-01-10 08:39:15,337] jepsen worker 581 - jepsen.util 581 :ok :add 1231 INFO [2022-01-10 08:39:15,338] jepsen worker 207 - jepsen.util 207 :ok :add 1229 INFO [2022-01-10 08:39:15,339] jepsen worker 31 - jepsen.util 31 :invoke :add 1242 INFO [2022-01-10 08:39:15,342] jepsen worker 287 - jepsen.util 287 :invoke :add 1243 INFO [2022-01-10 08:39:15,345] jepsen worker 416 - jepsen.util 416 :invoke :add 1244 INFO [2022-01-10 08:39:15,349] jepsen worker 981 - jepsen.util 981 :invoke :add 1245 INFO [2022-01-10 08:39:15,352] jepsen worker 12 - jepsen.util 12 :ok :add 1241 INFO [2022-01-10 08:39:15,353] jepsen worker 416 - jepsen.util 416 :ok :add 1244 INFO [2022-01-10 08:39:15,354] jepsen worker 814 - jepsen.util 814 :invoke :add 1246 INFO [2022-01-10 08:39:15,356] jepsen worker 287 - jepsen.util 287 :ok :add 1243 INFO [2022-01-10 08:39:15,356] jepsen worker 292 - jepsen.util 292 :ok :add 1240 INFO [2022-01-10 08:39:15,358] jepsen worker 981 - jepsen.util 981 :ok :add 1245 INFO [2022-01-10 08:39:15,359] jepsen worker 513 - jepsen.util 513 :invoke :add 1247 INFO [2022-01-10 08:39:15,364] jepsen worker 529 - jepsen.util 529 :invoke :add 1248 INFO [2022-01-10 08:39:15,364] jepsen worker 31 - jepsen.util 31 :ok :add 1242 INFO [2022-01-10 08:39:15,366] jepsen worker 513 - jepsen.util 513 :ok :add 1247 INFO [2022-01-10 08:39:15,367] jepsen worker 632 - jepsen.util 632 :invoke :add 1249 INFO [2022-01-10 08:39:15,368] jepsen worker 814 - jepsen.util 814 :ok :add 1246 INFO [2022-01-10 08:39:15,370] jepsen worker 249 - jepsen.util 249 :invoke :add 1250 INFO [2022-01-10 08:39:15,373] jepsen worker 79 - jepsen.util 79 :invoke :add 1251 INFO [2022-01-10 08:39:15,375] jepsen worker 632 - jepsen.util 632 :ok :add 1249 INFO [2022-01-10 08:39:15,375] jepsen worker 529 - jepsen.util 529 :ok :add 1248 INFO [2022-01-10 08:39:15,376] jepsen worker 369 - jepsen.util 369 :invoke :add 1252 INFO [2022-01-10 08:39:15,380] jepsen worker 961 - jepsen.util 961 :invoke :add 1253 INFO [2022-01-10 08:39:15,381] jepsen worker 249 - jepsen.util 249 :ok :add 1250 INFO [2022-01-10 08:39:15,384] jepsen worker 779 - jepsen.util 779 :invoke :add 1254 INFO [2022-01-10 08:39:15,384] jepsen worker 79 - jepsen.util 79 :ok :add 1251 INFO [2022-01-10 08:39:15,387] jepsen worker 773 - jepsen.util 773 :invoke :add 1255 INFO [2022-01-10 08:39:15,388] jepsen worker 369 - jepsen.util 369 :ok :add 1252 INFO [2022-01-10 08:39:15,391] jepsen worker 469 - jepsen.util 469 :invoke :add 1256 INFO [2022-01-10 08:39:15,392] jepsen worker 779 - jepsen.util 779 :ok :add 1254 INFO [2022-01-10 08:39:15,392] jepsen worker 961 - jepsen.util 961 :ok :add 1253 INFO [2022-01-10 08:39:15,393] jepsen worker 77 - jepsen.util 77 :invoke :add 1257 INFO [2022-01-10 08:39:15,396] jepsen worker 24 - jepsen.util 24 :invoke :add 1258 INFO [2022-01-10 08:39:15,397] jepsen worker 773 - jepsen.util 773 :ok :add 1255 INFO [2022-01-10 08:39:15,399] jepsen worker 633 - jepsen.util 633 :invoke :add 1259 INFO [2022-01-10 08:39:15,403] jepsen worker 358 - jepsen.util 358 :invoke :add 1260 INFO [2022-01-10 08:39:15,403] jepsen worker 24 - jepsen.util 24 :ok :add 1258 INFO [2022-01-10 08:39:15,410] jepsen worker 77 - jepsen.util 77 :ok :add 1257 INFO [2022-01-10 08:39:15,411] jepsen worker 633 - jepsen.util 633 :ok :add 1259 INFO [2022-01-10 08:39:15,411] jepsen worker 469 - jepsen.util 469 :ok :add 1256 INFO [2022-01-10 08:39:15,412] jepsen worker 865 - jepsen.util 865 :invoke :add 1261 INFO [2022-01-10 08:39:15,416] jepsen worker 584 - jepsen.util 584 :invoke :add 1262 INFO [2022-01-10 08:39:15,418] jepsen worker 713 - jepsen.util 713 :invoke :add 1263 INFO [2022-01-10 08:39:15,420] jepsen worker 865 - jepsen.util 865 :ok :add 1261 INFO [2022-01-10 08:39:15,420] jepsen worker 471 - jepsen.util 471 :invoke :add 1264 INFO [2022-01-10 08:39:15,422] jepsen worker 358 - jepsen.util 358 :ok :add 1260 INFO [2022-01-10 08:39:15,423] jepsen worker 385 - jepsen.util 385 :invoke :add 1265 INFO [2022-01-10 08:39:15,426] jepsen worker 496 - jepsen.util 496 :invoke :add 1266 INFO [2022-01-10 08:39:15,427] jepsen worker 584 - jepsen.util 584 :ok :add 1262 INFO [2022-01-10 08:39:15,429] jepsen worker 471 - jepsen.util 471 :ok :add 1264 INFO [2022-01-10 08:39:15,430] jepsen worker 713 - jepsen.util 713 :ok :add 1263 INFO [2022-01-10 08:39:15,430] jepsen worker 247 - jepsen.util 247 :invoke :add 1267 INFO [2022-01-10 08:39:15,433] jepsen worker 496 - jepsen.util 496 :ok :add 1266 INFO [2022-01-10 08:39:15,434] jepsen worker 385 - jepsen.util 385 :ok :add 1265 INFO [2022-01-10 08:39:15,435] jepsen worker 94 - jepsen.util 94 :invoke :add 1268 INFO [2022-01-10 08:39:15,438] jepsen worker 657 - jepsen.util 657 :invoke :add 1269 INFO [2022-01-10 08:39:15,440] jepsen worker 511 - jepsen.util 511 :invoke :add 1270 INFO [2022-01-10 08:39:15,442] jepsen worker 247 - jepsen.util 247 :ok :add 1267 INFO [2022-01-10 08:39:15,442] jepsen worker 416 - jepsen.util 416 :invoke :add 1271 INFO [2022-01-10 08:39:15,443] jepsen worker 94 - jepsen.util 94 :ok :add 1268 INFO [2022-01-10 08:39:15,445] jepsen worker 506 - jepsen.util 506 :invoke :add 1272 INFO [2022-01-10 08:39:15,448] jepsen worker 499 - jepsen.util 499 :invoke :add 1273 INFO [2022-01-10 08:39:15,448] jepsen worker 657 - jepsen.util 657 :ok :add 1269 INFO [2022-01-10 08:39:15,450] jepsen worker 352 - jepsen.util 352 :invoke :add 1274 INFO [2022-01-10 08:39:15,452] jepsen worker 511 - jepsen.util 511 :ok :add 1270 INFO [2022-01-10 08:39:15,453] jepsen worker 758 - jepsen.util 758 :invoke :add 1275 INFO [2022-01-10 08:39:15,456] jepsen worker 989 - jepsen.util 989 :invoke :add 1276 INFO [2022-01-10 08:39:15,457] jepsen worker 416 - jepsen.util 416 :ok :add 1271 INFO [2022-01-10 08:39:15,457] jepsen worker 506 - jepsen.util 506 :ok :add 1272 INFO [2022-01-10 08:39:15,458] jepsen worker 137 - jepsen.util 137 :invoke :add 1277 INFO [2022-01-10 08:39:15,462] jepsen worker 352 - jepsen.util 352 :ok :add 1274 INFO [2022-01-10 08:39:15,463] jepsen worker 499 - jepsen.util 499 :ok :add 1273 INFO [2022-01-10 08:39:15,463] jepsen worker 983 - jepsen.util 983 :invoke :add 1278 INFO [2022-01-10 08:39:15,465] jepsen worker 758 - jepsen.util 758 :ok :add 1275 INFO [2022-01-10 08:39:15,466] jepsen worker 339 - jepsen.util 339 :invoke :add 1279 INFO [2022-01-10 08:39:15,469] jepsen worker 868 - jepsen.util 868 :invoke :add 1280 INFO [2022-01-10 08:39:15,470] jepsen worker 989 - jepsen.util 989 :ok :add 1276 INFO [2022-01-10 08:39:15,470] jepsen worker 137 - jepsen.util 137 :ok :add 1277 INFO [2022-01-10 08:39:15,473] jepsen worker 859 - jepsen.util 859 :invoke :add 1281 INFO [2022-01-10 08:39:15,473] jepsen worker 339 - jepsen.util 339 :ok :add 1279 INFO [2022-01-10 08:39:15,476] jepsen worker 983 - jepsen.util 983 :ok :add 1278 INFO [2022-01-10 08:39:15,479] jepsen worker 38 - jepsen.util 38 :invoke :add 1282 INFO [2022-01-10 08:39:15,480] jepsen worker 859 - jepsen.util 859 :ok :add 1281 INFO [2022-01-10 08:39:15,483] jepsen worker 405 - jepsen.util 405 :invoke :add 1283 INFO [2022-01-10 08:39:15,484] jepsen worker 868 - jepsen.util 868 :ok :add 1280 INFO [2022-01-10 08:39:15,487] jepsen worker 389 - jepsen.util 389 :invoke :add 1284 INFO [2022-01-10 08:39:15,492] jepsen worker 38 - jepsen.util 38 :ok :add 1282 INFO [2022-01-10 08:39:15,493] jepsen worker 203 - jepsen.util 203 :invoke :add 1285 INFO [2022-01-10 08:39:15,495] jepsen worker 389 - jepsen.util 389 :ok :add 1284 INFO [2022-01-10 08:39:15,495] jepsen worker 405 - jepsen.util 405 :ok :add 1283 INFO [2022-01-10 08:39:15,498] jepsen worker 500 - jepsen.util 500 :invoke :add 1286 INFO [2022-01-10 08:39:15,501] jepsen worker 989 - jepsen.util 989 :invoke :add 1287 INFO [2022-01-10 08:39:15,502] jepsen worker 203 - jepsen.util 203 :ok :add 1285 INFO [2022-01-10 08:39:15,504] jepsen worker 112 - jepsen.util 112 :invoke :add 1288 INFO [2022-01-10 08:39:15,508] jepsen worker 901 - jepsen.util 901 :invoke :add 1289 INFO [2022-01-10 08:39:15,509] jepsen worker 500 - jepsen.util 500 :ok :add 1286 INFO [2022-01-10 08:39:15,512] jepsen worker 153 - jepsen.util 153 :invoke :add 1290 INFO [2022-01-10 08:39:15,513] jepsen worker 989 - jepsen.util 989 :ok :add 1287 INFO [2022-01-10 08:39:15,515] jepsen worker 947 - jepsen.util 947 :invoke :add 1291 INFO [2022-01-10 08:39:15,517] jepsen worker 112 - jepsen.util 112 :ok :add 1288 INFO [2022-01-10 08:39:15,518] jepsen worker 491 - jepsen.util 491 :invoke :add 1292 INFO [2022-01-10 08:39:15,518] jepsen worker 901 - jepsen.util 901 :ok :add 1289 INFO [2022-01-10 08:39:15,522] jepsen worker 941 - jepsen.util 941 :invoke :add 1293 INFO [2022-01-10 08:39:15,525] jepsen worker 896 - jepsen.util 896 :invoke :add 1294 INFO [2022-01-10 08:39:15,526] jepsen worker 947 - jepsen.util 947 :ok :add 1291 INFO [2022-01-10 08:39:15,528] jepsen worker 247 - jepsen.util 247 :invoke :add 1295 INFO [2022-01-10 08:39:15,528] jepsen worker 153 - jepsen.util 153 :ok :add 1290 INFO [2022-01-10 08:39:15,530] jepsen worker 287 - jepsen.util 287 :invoke :add 1296 INFO [2022-01-10 08:39:15,533] jepsen worker 343 - jepsen.util 343 :invoke :add 1297 INFO [2022-01-10 08:39:15,533] jepsen worker 491 - jepsen.util 491 :ok :add 1292 INFO [2022-01-10 08:39:15,534] jepsen worker 896 - jepsen.util 896 :ok :add 1294 INFO [2022-01-10 08:39:15,536] jepsen worker 41 - jepsen.util 41 :invoke :add 1298 INFO [2022-01-10 08:39:15,538] jepsen worker 941 - jepsen.util 941 :ok :add 1293 INFO [2022-01-10 08:39:15,538] jepsen worker 287 - jepsen.util 287 :ok :add 1296 INFO [2022-01-10 08:39:15,539] jepsen worker 683 - jepsen.util 683 :invoke :add 1299 INFO [2022-01-10 08:39:15,542] jepsen worker 827 - jepsen.util 827 :invoke :add 1300 INFO [2022-01-10 08:39:15,543] jepsen worker 247 - jepsen.util 247 :ok :add 1295 INFO [2022-01-10 08:39:15,544] jepsen worker 953 - jepsen.util 953 :invoke :add 1301 INFO [2022-01-10 08:39:15,548] jepsen worker 41 - jepsen.util 41 :ok :add 1298 INFO [2022-01-10 08:39:15,548] jepsen worker 524 - jepsen.util 524 :invoke :add 1302 INFO [2022-01-10 08:39:15,548] jepsen worker 343 - jepsen.util 343 :ok :add 1297 INFO [2022-01-10 08:39:15,550] jepsen worker 615 - jepsen.util 615 :invoke :add 1303 INFO [2022-01-10 08:39:15,551] jepsen worker 683 - jepsen.util 683 :ok :add 1299 INFO [2022-01-10 08:39:15,553] jepsen worker 845 - jepsen.util 845 :invoke :add 1304 INFO [2022-01-10 08:39:15,556] jepsen worker 270 - jepsen.util 270 :invoke :add 1305 INFO [2022-01-10 08:39:15,559] jepsen worker 285 - jepsen.util 285 :invoke :add 1306 INFO [2022-01-10 08:39:15,560] jepsen worker 953 - jepsen.util 953 :ok :add 1301 INFO [2022-01-10 08:39:15,561] jepsen worker 845 - jepsen.util 845 :ok :add 1304 INFO [2022-01-10 08:39:15,561] jepsen worker 864 - jepsen.util 864 :invoke :add 1307 INFO [2022-01-10 08:39:15,564] jepsen worker 982 - jepsen.util 982 :invoke :add 1308 INFO [2022-01-10 08:39:15,565] jepsen worker 827 - jepsen.util 827 :ok :add 1300 INFO [2022-01-10 08:39:15,566] jepsen worker 53 - jepsen.util 53 :invoke :add 1309 INFO [2022-01-10 08:39:15,568] jepsen worker 270 - jepsen.util 270 :ok :add 1305 INFO [2022-01-10 08:39:15,569] jepsen worker 661 - jepsen.util 661 :invoke :add 1310 INFO [2022-01-10 08:39:15,570] jepsen worker 285 - jepsen.util 285 :ok :add 1306 INFO [2022-01-10 08:39:15,571] jepsen worker 615 - jepsen.util 615 :ok :add 1303 INFO [2022-01-10 08:39:15,571] jepsen worker 982 - jepsen.util 982 :ok :add 1308 INFO [2022-01-10 08:39:15,573] jepsen worker 864 - jepsen.util 864 :ok :add 1307 INFO [2022-01-10 08:39:15,573] jepsen worker 432 - jepsen.util 432 :invoke :add 1311 INFO [2022-01-10 08:39:15,576] jepsen worker 524 - jepsen.util 524 :ok :add 1302 INFO [2022-01-10 08:39:15,577] jepsen worker 74 - jepsen.util 74 :invoke :add 1312 INFO [2022-01-10 08:39:15,581] jepsen worker 963 - jepsen.util 963 :invoke :add 1313 INFO [2022-01-10 08:39:15,584] jepsen worker 331 - jepsen.util 331 :invoke :add 1314 INFO [2022-01-10 08:39:15,587] jepsen worker 994 - jepsen.util 994 :invoke :add 1315 INFO [2022-01-10 08:39:15,591] jepsen worker 935 - jepsen.util 935 :invoke :add 1316 INFO [2022-01-10 08:39:15,592] jepsen worker 331 - jepsen.util 331 :ok :add 1314 INFO [2022-01-10 08:39:15,592] jepsen worker 53 - jepsen.util 53 :ok :add 1309 INFO [2022-01-10 08:39:15,594] jepsen worker 994 - jepsen.util 994 :ok :add 1315 INFO [2022-01-10 08:39:15,596] jepsen worker 501 - jepsen.util 501 :invoke :add 1317 INFO [2022-01-10 08:39:15,597] jepsen worker 432 - jepsen.util 432 :ok :add 1311 INFO [2022-01-10 08:39:15,599] jepsen worker 780 - jepsen.util 780 :invoke :add 1318 INFO [2022-01-10 08:39:15,600] jepsen worker 661 - jepsen.util 661 :ok :add 1310 INFO [2022-01-10 08:39:15,600] jepsen worker 935 - jepsen.util 935 :ok :add 1316 INFO [2022-01-10 08:39:15,604] jepsen worker 74 - jepsen.util 74 :ok :add 1312 INFO [2022-01-10 08:39:15,604] jepsen worker 963 - jepsen.util 963 :ok :add 1313 INFO [2022-01-10 08:39:15,604] jepsen worker 501 - jepsen.util 501 :ok :add 1317 INFO [2022-01-10 08:39:15,604] jepsen worker 873 - jepsen.util 873 :invoke :add 1319 INFO [2022-01-10 08:39:15,607] jepsen worker 825 - jepsen.util 825 :invoke :add 1320 INFO [2022-01-10 08:39:15,609] jepsen worker 780 - jepsen.util 780 :ok :add 1318 INFO [2022-01-10 08:39:15,612] jepsen worker 973 - jepsen.util 973 :invoke :add 1321 INFO [2022-01-10 08:39:15,613] jepsen worker 873 - jepsen.util 873 :ok :add 1319 INFO [2022-01-10 08:39:15,615] jepsen worker 732 - jepsen.util 732 :invoke :add 1322 INFO [2022-01-10 08:39:15,616] jepsen worker 825 - jepsen.util 825 :ok :add 1320 INFO [2022-01-10 08:39:15,618] jepsen worker 604 - jepsen.util 604 :invoke :add 1323 INFO [2022-01-10 08:39:15,621] jepsen worker 465 - jepsen.util 465 :invoke :add 1324 INFO [2022-01-10 08:39:15,623] jepsen worker 338 - jepsen.util 338 :invoke :add 1325 INFO [2022-01-10 08:39:15,626] jepsen worker 864 - jepsen.util 864 :invoke :add 1326 INFO [2022-01-10 08:39:15,627] jepsen worker 732 - jepsen.util 732 :ok :add 1322 INFO [2022-01-10 08:39:15,628] jepsen worker 973 - jepsen.util 973 :ok :add 1321 INFO [2022-01-10 08:39:15,629] jepsen worker 197 - jepsen.util 197 :invoke :add 1327 INFO [2022-01-10 08:39:15,632] jepsen worker 616 - jepsen.util 616 :invoke :add 1328 INFO [2022-01-10 08:39:15,632] jepsen worker 338 - jepsen.util 338 :ok :add 1325 INFO [2022-01-10 08:39:15,633] jepsen worker 604 - jepsen.util 604 :ok :add 1323 INFO [2022-01-10 08:39:15,635] jepsen worker 332 - jepsen.util 332 :invoke :add 1329 INFO [2022-01-10 08:39:15,635] jepsen worker 465 - jepsen.util 465 :ok :add 1324 INFO [2022-01-10 08:39:15,637] jepsen worker 716 - jepsen.util 716 :invoke :add 1330 INFO [2022-01-10 08:39:15,641] jepsen worker 932 - jepsen.util 932 :invoke :add 1331 INFO [2022-01-10 08:39:15,643] jepsen worker 733 - jepsen.util 733 :invoke :add 1332 INFO [2022-01-10 08:39:15,644] jepsen worker 864 - jepsen.util 864 :ok :add 1326 INFO [2022-01-10 08:39:15,645] jepsen worker 616 - jepsen.util 616 :ok :add 1328 INFO [2022-01-10 08:39:15,646] jepsen worker 489 - jepsen.util 489 :invoke :add 1333 INFO [2022-01-10 08:39:15,647] jepsen worker 197 - jepsen.util 197 :ok :add 1327 INFO [2022-01-10 08:39:15,649] jepsen worker 716 - jepsen.util 716 :ok :add 1330 INFO [2022-01-10 08:39:15,650] jepsen worker 753 - jepsen.util 753 :invoke :add 1334 INFO [2022-01-10 08:39:15,651] jepsen worker 332 - jepsen.util 332 :ok :add 1329 INFO [2022-01-10 08:39:15,655] jepsen worker 915 - jepsen.util 915 :invoke :add 1335 INFO [2022-01-10 08:39:15,656] jepsen worker 733 - jepsen.util 733 :ok :add 1332 INFO [2022-01-10 08:39:15,656] jepsen worker 932 - jepsen.util 932 :ok :add 1331 INFO [2022-01-10 08:39:15,660] jepsen worker 220 - jepsen.util 220 :invoke :add 1336 INFO [2022-01-10 08:39:15,661] jepsen worker 753 - jepsen.util 753 :ok :add 1334 INFO [2022-01-10 08:39:15,663] jepsen worker 915 - jepsen.util 915 :ok :add 1335 INFO [2022-01-10 08:39:15,665] jepsen worker 816 - jepsen.util 816 :invoke :add 1337 INFO [2022-01-10 08:39:15,666] jepsen worker 489 - jepsen.util 489 :ok :add 1333 INFO [2022-01-10 08:39:15,668] jepsen worker 435 - jepsen.util 435 :invoke :add 1338 INFO [2022-01-10 08:39:15,671] jepsen worker 136 - jepsen.util 136 :invoke :add 1339 INFO [2022-01-10 08:39:15,674] jepsen worker 878 - jepsen.util 878 :invoke :add 1340 INFO [2022-01-10 08:39:15,674] jepsen worker 816 - jepsen.util 816 :ok :add 1337 INFO [2022-01-10 08:39:15,678] jepsen worker 948 - jepsen.util 948 :invoke :add 1341 INFO [2022-01-10 08:39:15,679] jepsen worker 136 - jepsen.util 136 :ok :add 1339 INFO [2022-01-10 08:39:15,679] jepsen worker 435 - jepsen.util 435 :ok :add 1338 INFO [2022-01-10 08:39:15,680] jepsen worker 220 - jepsen.util 220 :ok :add 1336 INFO [2022-01-10 08:39:15,683] jepsen worker 907 - jepsen.util 907 :invoke :add 1342 INFO [2022-01-10 08:39:15,686] jepsen worker 523 - jepsen.util 523 :invoke :add 1343 INFO [2022-01-10 08:39:15,687] jepsen worker 878 - jepsen.util 878 :ok :add 1340 INFO [2022-01-10 08:39:15,689] jepsen worker 417 - jepsen.util 417 :invoke :add 1344 INFO [2022-01-10 08:39:15,692] jepsen worker 907 - jepsen.util 907 :ok :add 1342 INFO [2022-01-10 08:39:15,693] jepsen worker 53 - jepsen.util 53 :invoke :add 1345 INFO [2022-01-10 08:39:15,695] jepsen worker 948 - jepsen.util 948 :ok :add 1341 INFO [2022-01-10 08:39:15,697] jepsen worker 371 - jepsen.util 371 :invoke :add 1346 INFO [2022-01-10 08:39:15,699] jepsen worker 523 - jepsen.util 523 :ok :add 1343 INFO [2022-01-10 08:39:15,700] jepsen worker 477 - jepsen.util 477 :invoke :add 1347 INFO [2022-01-10 08:39:15,701] jepsen worker 417 - jepsen.util 417 :ok :add 1344 INFO [2022-01-10 08:39:15,704] jepsen worker 595 - jepsen.util 595 :invoke :add 1348 INFO [2022-01-10 08:39:15,707] jepsen worker 64 - jepsen.util 64 :invoke :add 1349 INFO [2022-01-10 08:39:15,708] jepsen worker 53 - jepsen.util 53 :ok :add 1345 INFO [2022-01-10 08:39:15,709] jepsen worker 607 - jepsen.util 607 :invoke :add 1350 INFO [2022-01-10 08:39:15,710] jepsen worker 371 - jepsen.util 371 :ok :add 1346 INFO [2022-01-10 08:39:15,713] jepsen worker 53 - jepsen.util 53 :invoke :add 1351 INFO [2022-01-10 08:39:15,716] jepsen worker 11 - jepsen.util 11 :invoke :add 1352 INFO [2022-01-10 08:39:15,717] jepsen worker 607 - jepsen.util 607 :ok :add 1350 INFO [2022-01-10 08:39:15,717] jepsen worker 595 - jepsen.util 595 :ok :add 1348 INFO [2022-01-10 08:39:15,719] jepsen worker 230 - jepsen.util 230 :invoke :add 1353 INFO [2022-01-10 08:39:15,720] jepsen worker 477 - jepsen.util 477 :ok :add 1347 INFO [2022-01-10 08:39:15,723] jepsen worker 291 - jepsen.util 291 :invoke :add 1354 INFO [2022-01-10 08:39:15,725] jepsen worker 11 - jepsen.util 11 :ok :add 1352 INFO [2022-01-10 08:39:15,725] jepsen worker 64 - jepsen.util 64 :ok :add 1349 INFO [2022-01-10 08:39:15,725] jepsen worker 787 - jepsen.util 787 :invoke :add 1355 INFO [2022-01-10 08:39:15,730] jepsen worker 383 - jepsen.util 383 :invoke :add 1356 INFO [2022-01-10 08:39:15,732] jepsen worker 787 - jepsen.util 787 :ok :add 1355 INFO [2022-01-10 08:39:15,732] jepsen worker 53 - jepsen.util 53 :ok :add 1351 INFO [2022-01-10 08:39:15,734] jepsen worker 800 - jepsen.util 800 :invoke :add 1357 INFO [2022-01-10 08:39:15,737] jepsen worker 291 - jepsen.util 291 :ok :add 1354 INFO [2022-01-10 08:39:15,739] jepsen worker 383 - jepsen.util 383 :ok :add 1356 INFO [2022-01-10 08:39:15,740] jepsen worker 55 - jepsen.util 55 :invoke :add 1358 INFO [2022-01-10 08:39:15,740] jepsen worker 230 - jepsen.util 230 :ok :add 1353 INFO [2022-01-10 08:39:15,742] jepsen worker 800 - jepsen.util 800 :ok :add 1357 INFO [2022-01-10 08:39:15,744] jepsen worker 909 - jepsen.util 909 :invoke :add 1359 INFO [2022-01-10 08:39:15,747] jepsen worker 285 - jepsen.util 285 :invoke :add 1360 INFO [2022-01-10 08:39:15,748] jepsen worker 55 - jepsen.util 55 :ok :add 1358 INFO [2022-01-10 08:39:15,750] jepsen worker 231 - jepsen.util 231 :invoke :add 1361 INFO [2022-01-10 08:39:15,753] jepsen worker 748 - jepsen.util 748 :invoke :add 1362 INFO [2022-01-10 08:39:15,754] jepsen worker 909 - jepsen.util 909 :ok :add 1359 INFO [2022-01-10 08:39:15,755] jepsen worker 99 - jepsen.util 99 :invoke :add 1363 INFO [2022-01-10 08:39:15,756] jepsen worker 231 - jepsen.util 231 :ok :add 1361 INFO [2022-01-10 08:39:15,758] jepsen worker 677 - jepsen.util 677 :invoke :add 1364 INFO [2022-01-10 08:39:15,761] jepsen worker 572 - jepsen.util 572 :invoke :add 1365 INFO [2022-01-10 08:39:15,764] jepsen worker 351 - jepsen.util 351 :invoke :add 1366 INFO [2022-01-10 08:39:15,766] jepsen worker 68 - jepsen.util 68 :invoke :add 1367 INFO [2022-01-10 08:39:15,767] jepsen worker 285 - jepsen.util 285 :ok :add 1360 INFO [2022-01-10 08:39:15,767] jepsen worker 677 - jepsen.util 677 :ok :add 1364 INFO [2022-01-10 08:39:15,768] jepsen worker 99 - jepsen.util 99 :ok :add 1363 INFO [2022-01-10 08:39:15,770] jepsen worker 572 - jepsen.util 572 :ok :add 1365 INFO [2022-01-10 08:39:15,772] jepsen worker 147 - jepsen.util 147 :invoke :add 1368 INFO [2022-01-10 08:39:15,773] jepsen worker 748 - jepsen.util 748 :ok :add 1362 INFO [2022-01-10 08:39:15,777] jepsen worker 68 - jepsen.util 68 :ok :add 1367 INFO [2022-01-10 08:39:15,779] jepsen worker 351 - jepsen.util 351 :ok :add 1366 INFO [2022-01-10 08:39:15,780] jepsen worker 934 - jepsen.util 934 :invoke :add 1369 INFO [2022-01-10 08:39:15,784] jepsen worker 468 - jepsen.util 468 :invoke :add 1370 INFO [2022-01-10 08:39:15,787] jepsen worker 976 - jepsen.util 976 :invoke :add 1371 INFO [2022-01-10 08:39:15,789] jepsen worker 989 - jepsen.util 989 :invoke :add 1372 INFO [2022-01-10 08:39:15,792] jepsen worker 97 - jepsen.util 97 :invoke :add 1373 INFO [2022-01-10 08:39:15,795] jepsen worker 403 - jepsen.util 403 :invoke :add 1374 INFO [2022-01-10 08:39:15,796] jepsen worker 934 - jepsen.util 934 :ok :add 1369 INFO [2022-01-10 08:39:15,796] jepsen worker 147 - jepsen.util 147 :ok :add 1368 INFO [2022-01-10 08:39:15,797] jepsen worker 859 - jepsen.util 859 :invoke :add 1375 INFO [2022-01-10 08:39:15,801] jepsen worker 954 - jepsen.util 954 :invoke :add 1376 INFO [2022-01-10 08:39:15,801] jepsen worker 403 - jepsen.util 403 :ok :add 1374 INFO [2022-01-10 08:39:15,803] jepsen worker 712 - jepsen.util 712 :invoke :add 1377 INFO [2022-01-10 08:39:15,803] jepsen worker 976 - jepsen.util 976 :ok :add 1371 INFO [2022-01-10 08:39:15,806] jepsen worker 394 - jepsen.util 394 :invoke :add 1378 INFO [2022-01-10 08:39:15,806] jepsen worker 468 - jepsen.util 468 :ok :add 1370 INFO [2022-01-10 08:39:15,809] jepsen worker 859 - jepsen.util 859 :ok :add 1375 INFO [2022-01-10 08:39:15,809] jepsen worker 0 - jepsen.util 0 :invoke :add 1379 INFO [2022-01-10 08:39:15,812] jepsen worker 954 - jepsen.util 954 :ok :add 1376 INFO [2022-01-10 08:39:15,813] jepsen worker 471 - jepsen.util 471 :invoke :add 1380 INFO [2022-01-10 08:39:15,814] jepsen worker 394 - jepsen.util 394 :ok :add 1378 INFO [2022-01-10 08:39:15,814] jepsen worker 989 - jepsen.util 989 :ok :add 1372 INFO [2022-01-10 08:39:15,815] jepsen worker 712 - jepsen.util 712 :ok :add 1377 INFO [2022-01-10 08:39:15,815] jepsen worker 97 - jepsen.util 97 :ok :add 1373 INFO [2022-01-10 08:39:15,817] jepsen worker 491 - jepsen.util 491 :invoke :add 1381 INFO [2022-01-10 08:39:15,822] jepsen worker 186 - jepsen.util 186 :invoke :add 1382 INFO [2022-01-10 08:39:15,823] jepsen worker 0 - jepsen.util 0 :ok :add 1379 INFO [2022-01-10 08:39:15,824] jepsen worker 491 - jepsen.util 491 :ok :add 1381 INFO [2022-01-10 08:39:15,826] jepsen worker 285 - jepsen.util 285 :invoke :add 1383 INFO [2022-01-10 08:39:15,830] jepsen worker 461 - jepsen.util 461 :invoke :add 1384 INFO [2022-01-10 08:39:15,831] jepsen worker 471 - jepsen.util 471 :ok :add 1380 INFO [2022-01-10 08:39:15,834] jepsen worker 627 - jepsen.util 627 :invoke :add 1385 INFO [2022-01-10 08:39:15,837] jepsen worker 285 - jepsen.util 285 :ok :add 1383 INFO [2022-01-10 08:39:15,838] jepsen worker 695 - jepsen.util 695 :invoke :add 1386 INFO [2022-01-10 08:39:15,839] jepsen worker 461 - jepsen.util 461 :ok :add 1384 INFO [2022-01-10 08:39:15,842] jepsen worker 566 - jepsen.util 566 :invoke :add 1387 INFO [2022-01-10 08:39:15,842] jepsen worker 627 - jepsen.util 627 :ok :add 1385 INFO [2022-01-10 08:39:15,845] jepsen worker 186 - jepsen.util 186 :ok :add 1382 INFO [2022-01-10 08:39:15,845] jepsen worker 218 - jepsen.util 218 :invoke :add 1388 INFO [2022-01-10 08:39:15,849] jepsen worker 425 - jepsen.util 425 :invoke :add 1389 INFO [2022-01-10 08:39:15,851] jepsen worker 695 - jepsen.util 695 :ok :add 1386 INFO [2022-01-10 08:39:15,852] jepsen worker 566 - jepsen.util 566 :ok :add 1387 INFO [2022-01-10 08:39:15,852] jepsen worker 359 - jepsen.util 359 :invoke :add 1390 INFO [2022-01-10 08:39:15,853] jepsen worker 218 - jepsen.util 218 :ok :add 1388 INFO [2022-01-10 08:39:15,856] jepsen worker 354 - jepsen.util 354 :invoke :add 1391 INFO [2022-01-10 08:39:15,858] jepsen worker 382 - jepsen.util 382 :invoke :add 1392 INFO [2022-01-10 08:39:15,859] jepsen worker 359 - jepsen.util 359 :ok :add 1390 INFO [2022-01-10 08:39:15,860] jepsen worker 24 - jepsen.util 24 :invoke :add 1393 INFO [2022-01-10 08:39:15,863] jepsen worker 425 - jepsen.util 425 :ok :add 1389 INFO [2022-01-10 08:39:15,863] jepsen worker 829 - jepsen.util 829 :invoke :add 1394 INFO [2022-01-10 08:39:15,869] jepsen worker 238 - jepsen.util 238 :invoke :add 1395 INFO [2022-01-10 08:39:15,869] jepsen worker 354 - jepsen.util 354 :ok :add 1391 INFO [2022-01-10 08:39:15,870] jepsen worker 829 - jepsen.util 829 :ok :add 1394 INFO [2022-01-10 08:39:15,872] jepsen worker 382 - jepsen.util 382 :ok :add 1392 INFO [2022-01-10 08:39:15,872] jepsen worker 746 - jepsen.util 746 :invoke :add 1396 INFO [2022-01-10 08:39:15,876] jepsen worker 952 - jepsen.util 952 :invoke :add 1397 INFO [2022-01-10 08:39:15,876] jepsen worker 24 - jepsen.util 24 :ok :add 1393 INFO [2022-01-10 08:39:15,877] jepsen worker 238 - jepsen.util 238 :ok :add 1395 INFO [2022-01-10 08:39:15,878] jepsen worker 383 - jepsen.util 383 :invoke :add 1398 INFO [2022-01-10 08:39:15,881] jepsen worker 458 - jepsen.util 458 :invoke :add 1399 INFO [2022-01-10 08:39:15,884] jepsen worker 746 - jepsen.util 746 :ok :add 1396 INFO [2022-01-10 08:39:15,885] jepsen worker 767 - jepsen.util 767 :invoke :add 1400 INFO [2022-01-10 08:39:15,886] jepsen worker 952 - jepsen.util 952 :ok :add 1397 INFO [2022-01-10 08:39:15,888] jepsen worker 383 - jepsen.util 383 :ok :add 1398 INFO [2022-01-10 08:39:15,890] jepsen worker 458 - jepsen.util 458 :ok :add 1399 INFO [2022-01-10 08:39:15,890] jepsen worker 490 - jepsen.util 490 :invoke :add 1401 INFO [2022-01-10 08:39:15,893] jepsen worker 767 - jepsen.util 767 :ok :add 1400 INFO [2022-01-10 08:39:15,894] jepsen worker 923 - jepsen.util 923 :invoke :add 1402 INFO [2022-01-10 08:39:15,897] jepsen worker 123 - jepsen.util 123 :invoke :add 1403 INFO [2022-01-10 08:39:15,900] jepsen worker 549 - jepsen.util 549 :invoke :add 1404 INFO [2022-01-10 08:39:15,901] jepsen worker 923 - jepsen.util 923 :ok :add 1402 INFO [2022-01-10 08:39:15,903] jepsen worker 441 - jepsen.util 441 :invoke :add 1405 INFO [2022-01-10 08:39:15,906] jepsen worker 668 - jepsen.util 668 :invoke :add 1406 INFO [2022-01-10 08:39:15,908] jepsen worker 549 - jepsen.util 549 :ok :add 1404 INFO [2022-01-10 08:39:15,908] jepsen worker 518 - jepsen.util 518 :invoke :add 1407 INFO [2022-01-10 08:39:15,911] jepsen worker 490 - jepsen.util 490 :ok :add 1401 INFO [2022-01-10 08:39:15,911] jepsen worker 786 - jepsen.util 786 :invoke :add 1408 INFO [2022-01-10 08:39:15,914] jepsen worker 849 - jepsen.util 849 :invoke :add 1409 INFO [2022-01-10 08:39:15,915] jepsen worker 668 - jepsen.util 668 :ok :add 1406 INFO [2022-01-10 08:39:15,916] jepsen worker 123 - jepsen.util 123 :ok :add 1403 INFO [2022-01-10 08:39:15,917] jepsen worker 26 - jepsen.util 26 :invoke :add 1410 INFO [2022-01-10 08:39:15,920] jepsen worker 441 - jepsen.util 441 :ok :add 1405 INFO [2022-01-10 08:39:15,921] jepsen worker 930 - jepsen.util 930 :invoke :add 1411 INFO [2022-01-10 08:39:15,925] jepsen worker 518 - jepsen.util 518 :ok :add 1407 INFO [2022-01-10 08:39:15,928] jepsen worker 160 - jepsen.util 160 :invoke :add 1412 INFO [2022-01-10 08:39:15,932] jepsen worker 849 - jepsen.util 849 :ok :add 1409 INFO [2022-01-10 08:39:15,932] jepsen worker 786 - jepsen.util 786 :ok :add 1408 INFO [2022-01-10 08:39:15,933] jepsen worker 980 - jepsen.util 980 :invoke :add 1413 INFO [2022-01-10 08:39:15,937] jepsen worker 930 - jepsen.util 930 :ok :add 1411 INFO [2022-01-10 08:39:15,939] jepsen worker 962 - jepsen.util 962 :invoke :add 1414 INFO [2022-01-10 08:39:15,941] jepsen worker 980 - jepsen.util 980 :ok :add 1413 INFO [2022-01-10 08:39:15,942] jepsen worker 472 - jepsen.util 472 :invoke :add 1415 INFO [2022-01-10 08:39:15,942] jepsen worker 26 - jepsen.util 26 :ok :add 1410 INFO [2022-01-10 08:39:15,946] jepsen worker 236 - jepsen.util 236 :invoke :add 1416 INFO [2022-01-10 08:39:15,948] jepsen worker 160 - jepsen.util 160 :ok :add 1412 INFO [2022-01-10 08:39:15,948] jepsen worker 962 - jepsen.util 962 :ok :add 1414 INFO [2022-01-10 08:39:15,950] jepsen worker 401 - jepsen.util 401 :invoke :add 1417 INFO [2022-01-10 08:39:15,950] jepsen worker 472 - jepsen.util 472 :ok :add 1415 INFO [2022-01-10 08:39:15,953] jepsen worker 194 - jepsen.util 194 :invoke :add 1418 INFO [2022-01-10 08:39:15,956] jepsen worker 954 - jepsen.util 954 :invoke :add 1419 INFO [2022-01-10 08:39:15,958] jepsen worker 236 - jepsen.util 236 :ok :add 1416 INFO [2022-01-10 08:39:15,959] jepsen worker 170 - jepsen.util 170 :invoke :add 1420 INFO [2022-01-10 08:39:15,959] jepsen worker 401 - jepsen.util 401 :ok :add 1417 INFO [2022-01-10 08:39:15,962] jepsen worker 882 - jepsen.util 882 :invoke :add 1421 INFO [2022-01-10 08:39:15,964] jepsen worker 7 - jepsen.util 7 :invoke :add 1422 INFO [2022-01-10 08:39:15,964] jepsen worker 954 - jepsen.util 954 :ok :add 1419 INFO [2022-01-10 08:39:15,967] jepsen worker 732 - jepsen.util 732 :invoke :add 1423 INFO [2022-01-10 08:39:15,968] jepsen worker 194 - jepsen.util 194 :ok :add 1418 INFO [2022-01-10 08:39:15,969] jepsen worker 876 - jepsen.util 876 :invoke :add 1424 INFO [2022-01-10 08:39:15,970] jepsen worker 170 - jepsen.util 170 :ok :add 1420 INFO [2022-01-10 08:39:15,974] jepsen worker 771 - jepsen.util 771 :invoke :add 1425 INFO [2022-01-10 08:39:15,975] jepsen worker 882 - jepsen.util 882 :ok :add 1421 INFO [2022-01-10 08:39:15,975] jepsen worker 7 - jepsen.util 7 :ok :add 1422 INFO [2022-01-10 08:39:15,975] jepsen worker 876 - jepsen.util 876 :ok :add 1424 INFO [2022-01-10 08:39:15,977] jepsen worker 526 - jepsen.util 526 :invoke :add 1426 INFO [2022-01-10 08:39:15,979] jepsen worker 732 - jepsen.util 732 :ok :add 1423 INFO [2022-01-10 08:39:15,981] jepsen worker 609 - jepsen.util 609 :invoke :add 1427 INFO [2022-01-10 08:39:15,985] jepsen worker 953 - jepsen.util 953 :invoke :add 1428 INFO [2022-01-10 08:39:15,985] jepsen worker 771 - jepsen.util 771 :ok :add 1425 INFO [2022-01-10 08:39:15,986] jepsen worker 526 - jepsen.util 526 :ok :add 1426 INFO [2022-01-10 08:39:15,989] jepsen worker 269 - jepsen.util 269 :invoke :add 1429 INFO [2022-01-10 08:39:15,992] jepsen worker 609 - jepsen.util 609 :ok :add 1427 INFO [2022-01-10 08:39:15,993] jepsen worker 953 - jepsen.util 953 :ok :add 1428 INFO [2022-01-10 08:39:15,995] jepsen worker 737 - jepsen.util 737 :invoke :add 1430 INFO [2022-01-10 08:39:15,999] jepsen worker 269 - jepsen.util 269 :ok :add 1429 INFO [2022-01-10 08:39:16,000] jepsen worker 151 - jepsen.util 151 :invoke :add 1431 INFO [2022-01-10 08:39:16,002] jepsen worker 737 - jepsen.util 737 :ok :add 1430 INFO [2022-01-10 08:39:16,003] jepsen worker 745 - jepsen.util 745 :invoke :add 1432 INFO [2022-01-10 08:39:16,006] jepsen worker 151 - jepsen.util 151 :ok :add 1431 INFO [2022-01-10 08:39:16,006] jepsen worker 680 - jepsen.util 680 :invoke :add 1433 INFO [2022-01-10 08:39:16,009] jepsen worker 626 - jepsen.util 626 :invoke :add 1434 INFO [2022-01-10 08:39:16,010] jepsen worker 745 - jepsen.util 745 :ok :add 1432 INFO [2022-01-10 08:39:16,012] jepsen worker 687 - jepsen.util 687 :invoke :add 1435 INFO [2022-01-10 08:39:16,014] jepsen worker 680 - jepsen.util 680 :ok :add 1433 INFO [2022-01-10 08:39:16,016] jepsen worker 633 - jepsen.util 633 :invoke :add 1436 INFO [2022-01-10 08:39:16,020] jepsen worker 626 - jepsen.util 626 :ok :add 1434 INFO [2022-01-10 08:39:16,020] jepsen worker 842 - jepsen.util 842 :invoke :add 1437 INFO [2022-01-10 08:39:16,021] jepsen worker 687 - jepsen.util 687 :ok :add 1435 INFO [2022-01-10 08:39:16,026] jepsen worker 567 - jepsen.util 567 :invoke :add 1438 INFO [2022-01-10 08:39:16,030] jepsen worker 629 - jepsen.util 629 :invoke :add 1439 INFO [2022-01-10 08:39:16,035] jepsen worker 83 - jepsen.util 83 :invoke :add 1440 INFO [2022-01-10 08:39:16,037] jepsen worker 567 - jepsen.util 567 :ok :add 1438 INFO [2022-01-10 08:39:16,037] jepsen worker 629 - jepsen.util 629 :ok :add 1439 INFO [2022-01-10 08:39:16,039] jepsen worker 187 - jepsen.util 187 :invoke :add 1441 INFO [2022-01-10 08:39:16,042] jepsen worker 829 - jepsen.util 829 :invoke :add 1442 INFO [2022-01-10 08:39:16,043] jepsen worker 83 - jepsen.util 83 :ok :add 1440 INFO [2022-01-10 08:39:16,044] jepsen worker 633 - jepsen.util 633 :ok :add 1436 INFO [2022-01-10 08:39:16,046] jepsen worker 187 - jepsen.util 187 :ok :add 1441 INFO [2022-01-10 08:39:16,047] jepsen worker 943 - jepsen.util 943 :invoke :add 1443 INFO [2022-01-10 08:39:16,051] jepsen worker 842 - jepsen.util 842 :ok :add 1437 INFO [2022-01-10 08:39:16,053] jepsen worker 943 - jepsen.util 943 :ok :add 1443 INFO [2022-01-10 08:39:16,055] jepsen worker 705 - jepsen.util 705 :invoke :add 1444 INFO [2022-01-10 08:39:16,057] jepsen worker 829 - jepsen.util 829 :ok :add 1442 INFO [2022-01-10 08:39:16,059] jepsen worker 765 - jepsen.util 765 :invoke :add 1445 INFO [2022-01-10 08:39:16,063] jepsen worker 772 - jepsen.util 772 :invoke :add 1446 INFO [2022-01-10 08:39:16,065] jepsen worker 517 - jepsen.util 517 :invoke :add 1447 INFO [2022-01-10 08:39:16,068] jepsen worker 961 - jepsen.util 961 :invoke :add 1448 INFO [2022-01-10 08:39:16,070] jepsen worker 65 - jepsen.util 65 :invoke :add 1449 INFO [2022-01-10 08:39:16,073] jepsen worker 18 - jepsen.util 18 :invoke :add 1450 INFO [2022-01-10 08:39:16,076] jepsen worker 407 - jepsen.util 407 :invoke :add 1451 INFO [2022-01-10 08:39:16,079] jepsen worker 531 - jepsen.util 531 :invoke :add 1452 INFO [2022-01-10 08:39:16,081] jepsen worker 705 - jepsen.util 705 :ok :add 1444 INFO [2022-01-10 08:39:16,081] jepsen worker 65 - jepsen.util 65 :ok :add 1449 INFO [2022-01-10 08:39:16,082] jepsen worker 961 - jepsen.util 961 :ok :add 1448 INFO [2022-01-10 08:39:16,083] jepsen worker 610 - jepsen.util 610 :invoke :add 1453 INFO [2022-01-10 08:39:16,084] jepsen worker 517 - jepsen.util 517 :ok :add 1447 INFO [2022-01-10 08:39:16,086] jepsen worker 772 - jepsen.util 772 :ok :add 1446 INFO [2022-01-10 08:39:16,087] jepsen worker 49 - jepsen.util 49 :invoke :add 1454 INFO [2022-01-10 08:39:16,090] jepsen worker 518 - jepsen.util 518 :invoke :add 1455 INFO [2022-01-10 08:39:16,091] jepsen worker 765 - jepsen.util 765 :ok :add 1445 INFO [2022-01-10 08:39:16,094] jepsen worker 238 - jepsen.util 238 :invoke :add 1456 INFO [2022-01-10 08:39:16,096] jepsen worker 49 - jepsen.util 49 :ok :add 1454 INFO [2022-01-10 08:39:16,098] jepsen worker 395 - jepsen.util 395 :invoke :add 1457 INFO [2022-01-10 08:39:16,100] jepsen worker 518 - jepsen.util 518 :ok :add 1455 INFO [2022-01-10 08:39:16,102] jepsen worker 353 - jepsen.util 353 :invoke :add 1458 INFO [2022-01-10 08:39:16,104] jepsen worker 471 - jepsen.util 471 :invoke :add 1459 INFO [2022-01-10 08:39:16,107] jepsen worker 562 - jepsen.util 562 :invoke :add 1460 INFO [2022-01-10 08:39:16,108] jepsen worker 238 - jepsen.util 238 :ok :add 1456 INFO [2022-01-10 08:39:16,109] jepsen worker 41 - jepsen.util 41 :invoke :add 1461 INFO [2022-01-10 08:39:16,111] jepsen worker 395 - jepsen.util 395 :ok :add 1457 INFO [2022-01-10 08:39:16,111] jepsen worker 471 - jepsen.util 471 :ok :add 1459 INFO [2022-01-10 08:39:16,113] jepsen worker 929 - jepsen.util 929 :invoke :add 1462 INFO [2022-01-10 08:39:16,114] jepsen worker 353 - jepsen.util 353 :ok :add 1458 INFO [2022-01-10 08:39:16,118] jepsen worker 114 - jepsen.util 114 :invoke :add 1463 INFO [2022-01-10 08:39:16,119] jepsen worker 562 - jepsen.util 562 :ok :add 1460 INFO [2022-01-10 08:39:16,120] jepsen worker 50 - jepsen.util 50 :invoke :add 1464 INFO [2022-01-10 08:39:16,122] jepsen worker 41 - jepsen.util 41 :ok :add 1461 INFO [2022-01-10 08:39:16,123] jepsen worker 388 - jepsen.util 388 :invoke :add 1465 INFO [2022-01-10 08:39:16,124] jepsen worker 929 - jepsen.util 929 :ok :add 1462 INFO [2022-01-10 08:39:16,126] jepsen worker 629 - jepsen.util 629 :invoke :add 1466 INFO [2022-01-10 08:39:16,126] jepsen worker 114 - jepsen.util 114 :ok :add 1463 INFO [2022-01-10 08:39:16,129] jepsen worker 220 - jepsen.util 220 :invoke :add 1467 INFO [2022-01-10 08:39:16,131] jepsen worker 478 - jepsen.util 478 :invoke :add 1468 INFO [2022-01-10 08:39:16,135] jepsen worker 647 - jepsen.util 647 :invoke :add 1469 INFO [2022-01-10 08:39:16,139] jepsen worker 720 - jepsen.util 720 :invoke :add 1470 INFO [2022-01-10 08:39:16,141] jepsen worker 456 - jepsen.util 456 :invoke :add 1471 INFO [2022-01-10 08:39:16,144] jepsen worker 198 - jepsen.util 198 :invoke :add 1472 INFO [2022-01-10 08:39:16,146] jepsen worker 71 - jepsen.util 71 :invoke :add 1473 INFO [2022-01-10 08:39:16,148] jepsen worker 415 - jepsen.util 415 :invoke :add 1474 INFO [2022-01-10 08:39:16,150] jepsen worker 112 - jepsen.util 112 :invoke :add 1475 INFO [2022-01-10 08:39:16,152] jepsen worker 353 - jepsen.util 353 :invoke :add 1476 INFO [2022-01-10 08:39:16,154] jepsen worker 375 - jepsen.util 375 :invoke :add 1477 INFO [2022-01-10 08:39:16,158] jepsen worker 415 - jepsen.util 415 :ok :add 1474 INFO [2022-01-10 08:39:16,158] jepsen worker 214 - jepsen.util 214 :invoke :add 1478 INFO [2022-01-10 08:39:16,160] jepsen worker 112 - jepsen.util 112 :ok :add 1475 INFO [2022-01-10 08:39:16,163] jepsen worker 353 - jepsen.util 353 :ok :add 1476 INFO [2022-01-10 08:39:16,163] jepsen worker 379 - jepsen.util 379 :invoke :add 1479 INFO [2022-01-10 08:39:16,164] jepsen worker 375 - jepsen.util 375 :ok :add 1477 INFO [2022-01-10 08:39:16,165] jepsen worker 647 - jepsen.util 647 :ok :add 1469 INFO [2022-01-10 08:39:16,168] jepsen worker 981 - jepsen.util 981 :invoke :add 1480 INFO [2022-01-10 08:39:16,168] jepsen worker 478 - jepsen.util 478 :ok :add 1468 INFO [2022-01-10 08:39:16,168] jepsen worker 720 - jepsen.util 720 :ok :add 1470 INFO [2022-01-10 08:39:16,168] jepsen worker 18 - jepsen.util 18 :ok :add 1450 INFO [2022-01-10 08:39:16,170] jepsen worker 214 - jepsen.util 214 :ok :add 1478 INFO [2022-01-10 08:39:16,172] jepsen worker 456 - jepsen.util 456 :ok :add 1471 INFO [2022-01-10 08:39:16,173] jepsen worker 379 - jepsen.util 379 :ok :add 1479 INFO [2022-01-10 08:39:16,173] jepsen worker 958 - jepsen.util 958 :invoke :add 1481 INFO [2022-01-10 08:39:16,174] jepsen worker 198 - jepsen.util 198 :ok :add 1472 INFO [2022-01-10 08:39:16,178] jepsen worker 455 - jepsen.util 455 :invoke :add 1482 INFO [2022-01-10 08:39:16,178] jepsen worker 71 - jepsen.util 71 :ok :add 1473 INFO [2022-01-10 08:39:16,181] jepsen worker 483 - jepsen.util 483 :invoke :add 1483 INFO [2022-01-10 08:39:16,185] jepsen worker 783 - jepsen.util 783 :invoke :add 1484 INFO [2022-01-10 08:39:16,188] jepsen worker 856 - jepsen.util 856 :invoke :add 1485 INFO [2022-01-10 08:39:16,191] jepsen worker 958 - jepsen.util 958 :ok :add 1481 INFO [2022-01-10 08:39:16,191] jepsen worker 689 - jepsen.util 689 :invoke :add 1486 INFO [2022-01-10 08:39:16,194] jepsen worker 115 - jepsen.util 115 :invoke :add 1487 INFO [2022-01-10 08:39:16,195] jepsen worker 981 - jepsen.util 981 :ok :add 1480 INFO [2022-01-10 08:39:16,197] jepsen worker 483 - jepsen.util 483 :ok :add 1483 INFO [2022-01-10 08:39:16,198] jepsen worker 601 - jepsen.util 601 :invoke :add 1488 INFO [2022-01-10 08:39:16,199] jepsen worker 455 - jepsen.util 455 :ok :add 1482 INFO [2022-01-10 08:39:16,202] jepsen worker 405 - jepsen.util 405 :invoke :add 1489 INFO [2022-01-10 08:39:16,203] jepsen worker 407 - jepsen.util 407 :ok :add 1451 INFO [2022-01-10 08:39:16,203] jepsen worker 388 - jepsen.util 388 :ok :add 1465 INFO [2022-01-10 08:39:16,206] jepsen worker 975 - jepsen.util 975 :invoke :add 1490 INFO [2022-01-10 08:39:16,208] jepsen worker 601 - jepsen.util 601 :ok :add 1488 INFO [2022-01-10 08:39:16,209] jepsen worker 952 - jepsen.util 952 :invoke :add 1491 INFO [2022-01-10 08:39:16,209] jepsen worker 405 - jepsen.util 405 :ok :add 1489 INFO [2022-01-10 08:39:16,210] jepsen worker 531 - jepsen.util 531 :ok :add 1452 INFO [2022-01-10 08:39:16,211] jepsen worker 629 - jepsen.util 629 :ok :add 1466 INFO [2022-01-10 08:39:16,215] jepsen worker 999 - jepsen.util 999 :invoke :add 1492 INFO [2022-01-10 08:39:16,218] jepsen worker 975 - jepsen.util 975 :ok :add 1490 INFO [2022-01-10 08:39:16,218] jepsen worker 37 - jepsen.util 37 :invoke :add 1493 INFO [2022-01-10 08:39:16,220] jepsen worker 952 - jepsen.util 952 :ok :add 1491 INFO [2022-01-10 08:39:16,222] jepsen worker 753 - jepsen.util 753 :invoke :add 1494 INFO [2022-01-10 08:39:16,225] jepsen worker 410 - jepsen.util 410 :invoke :add 1495 INFO [2022-01-10 08:39:16,226] jepsen worker 999 - jepsen.util 999 :ok :add 1492 INFO [2022-01-10 08:39:16,229] jepsen worker 870 - jepsen.util 870 :invoke :add 1496 INFO [2022-01-10 08:39:16,233] jepsen worker 746 - jepsen.util 746 :invoke :add 1497 INFO [2022-01-10 08:39:16,235] jepsen worker 37 - jepsen.util 37 :ok :add 1493 INFO [2022-01-10 08:39:16,237] jepsen worker 532 - jepsen.util 532 :invoke :add 1498 INFO [2022-01-10 08:39:16,237] jepsen worker 753 - jepsen.util 753 :ok :add 1494 INFO [2022-01-10 08:39:16,241] jepsen worker 410 - jepsen.util 410 :ok :add 1495 INFO [2022-01-10 08:39:16,243] jepsen worker 57 - jepsen.util 57 :invoke :add 1499 INFO [2022-01-10 08:39:16,245] jepsen worker 870 - jepsen.util 870 :ok :add 1496 INFO [2022-01-10 08:39:16,246] jepsen worker 532 - jepsen.util 532 :ok :add 1498 INFO [2022-01-10 08:39:16,246] jepsen worker 221 - jepsen.util 221 :invoke :add 1500 INFO [2022-01-10 08:39:16,248] jepsen worker 746 - jepsen.util 746 :ok :add 1497 INFO [2022-01-10 08:39:16,252] jepsen worker 897 - jepsen.util 897 :invoke :add 1501 INFO [2022-01-10 08:39:16,255] jepsen worker 220 - jepsen.util 220 :ok :add 1467 INFO [2022-01-10 08:39:16,255] jepsen worker 610 - jepsen.util 610 :ok :add 1453 INFO [2022-01-10 08:39:16,255] jepsen worker 221 - jepsen.util 221 :ok :add 1500 INFO [2022-01-10 08:39:16,256] jepsen worker 57 - jepsen.util 57 :ok :add 1499 INFO [2022-01-10 08:39:16,257] jepsen worker 999 - jepsen.util 999 :invoke :add 1502 INFO [2022-01-10 08:39:16,260] jepsen worker 897 - jepsen.util 897 :ok :add 1501 INFO [2022-01-10 08:39:16,262] jepsen worker 271 - jepsen.util 271 :invoke :add 1503 INFO [2022-01-10 08:39:16,265] jepsen worker 432 - jepsen.util 432 :invoke :add 1504 INFO [2022-01-10 08:39:16,268] jepsen worker 50 - jepsen.util 50 :ok :add 1464 INFO [2022-01-10 08:39:16,268] jepsen worker 840 - jepsen.util 840 :invoke :add 1505 INFO [2022-01-10 08:39:16,269] jepsen worker 271 - jepsen.util 271 :ok :add 1503 INFO [2022-01-10 08:39:16,273] jepsen worker 420 - jepsen.util 420 :invoke :add 1506 INFO [2022-01-10 08:39:16,275] jepsen worker 999 - jepsen.util 999 :ok :add 1502 INFO [2022-01-10 08:39:16,278] jepsen worker 581 - jepsen.util 581 :invoke :add 1507 INFO [2022-01-10 08:39:16,278] jepsen worker 432 - jepsen.util 432 :ok :add 1504 INFO [2022-01-10 08:39:16,282] jepsen worker 856 - jepsen.util 856 :ok :add 1485 INFO [2022-01-10 08:39:16,283] jepsen worker 578 - jepsen.util 578 :invoke :add 1508 INFO [2022-01-10 08:39:16,286] jepsen worker 840 - jepsen.util 840 :ok :add 1505 INFO [2022-01-10 08:39:16,287] jepsen worker 783 - jepsen.util 783 :ok :add 1484 INFO [2022-01-10 08:39:16,288] jepsen worker 115 - jepsen.util 115 :ok :add 1487 INFO [2022-01-10 08:39:16,288] jepsen worker 599 - jepsen.util 599 :invoke :add 1509 INFO [2022-01-10 08:39:16,293] jepsen worker 689 - jepsen.util 689 :ok :add 1486 INFO [2022-01-10 08:39:16,294] jepsen worker 263 - jepsen.util 263 :invoke :add 1510 INFO [2022-01-10 08:39:16,298] jepsen worker 698 - jepsen.util 698 :invoke :add 1511 INFO [2022-01-10 08:39:16,299] jepsen worker 599 - jepsen.util 599 :ok :add 1509 INFO [2022-01-10 08:39:16,301] jepsen worker 894 - jepsen.util 894 :invoke :add 1512 INFO [2022-01-10 08:39:16,386] jepsen worker 581 - jepsen.util 581 :ok :add 1507 INFO [2022-01-10 08:39:16,387] jepsen worker 420 - jepsen.util 420 :ok :add 1506 INFO [2022-01-10 08:39:16,388] jepsen worker 698 - jepsen.util 698 :ok :add 1511 INFO [2022-01-10 08:39:16,389] jepsen worker 263 - jepsen.util 263 :ok :add 1510 INFO [2022-01-10 08:39:16,390] jepsen worker 578 - jepsen.util 578 :ok :add 1508 INFO [2022-01-10 08:39:16,390] jepsen worker 84 - jepsen.util 84 :invoke :add 1513 INFO [2022-01-10 08:39:16,391] jepsen worker 894 - jepsen.util 894 :ok :add 1512 INFO [2022-01-10 08:39:16,393] jepsen worker 4 - jepsen.util 4 :invoke :add 1514 INFO [2022-01-10 08:39:16,397] jepsen worker 943 - jepsen.util 943 :invoke :add 1515 INFO [2022-01-10 08:39:16,399] jepsen worker 84 - jepsen.util 84 :ok :add 1513 INFO [2022-01-10 08:39:16,400] jepsen worker 4 - jepsen.util 4 :ok :add 1514 INFO [2022-01-10 08:39:16,400] jepsen worker 338 - jepsen.util 338 :invoke :add 1516 INFO [2022-01-10 08:39:16,404] jepsen worker 198 - jepsen.util 198 :invoke :add 1517 INFO [2022-01-10 08:39:16,407] jepsen worker 305 - jepsen.util 305 :invoke :add 1518 INFO [2022-01-10 08:39:16,408] jepsen worker 943 - jepsen.util 943 :ok :add 1515 INFO [2022-01-10 08:39:16,410] jepsen worker 338 - jepsen.util 338 :ok :add 1516 INFO [2022-01-10 08:39:16,412] jepsen worker 198 - jepsen.util 198 :ok :add 1517 INFO [2022-01-10 08:39:16,412] jepsen worker 230 - jepsen.util 230 :invoke :add 1519 INFO [2022-01-10 08:39:16,415] jepsen worker 305 - jepsen.util 305 :ok :add 1518 INFO [2022-01-10 08:39:16,416] jepsen worker 102 - jepsen.util 102 :invoke :add 1520 INFO [2022-01-10 08:39:16,420] jepsen worker 841 - jepsen.util 841 :invoke :add 1521 INFO [2022-01-10 08:39:16,422] jepsen worker 230 - jepsen.util 230 :ok :add 1519 INFO [2022-01-10 08:39:16,423] jepsen worker 561 - jepsen.util 561 :invoke :add 1522 INFO [2022-01-10 08:39:16,426] jepsen worker 281 - jepsen.util 281 :invoke :add 1523 INFO [2022-01-10 08:39:16,428] jepsen worker 102 - jepsen.util 102 :ok :add 1520 INFO [2022-01-10 08:39:16,429] jepsen worker 181 - jepsen.util 181 :invoke :add 1524 INFO [2022-01-10 08:39:16,431] jepsen worker 966 - jepsen.util 966 :invoke :add 1525 INFO [2022-01-10 08:39:16,432] jepsen worker 841 - jepsen.util 841 :ok :add 1521 INFO [2022-01-10 08:39:16,433] jepsen worker 173 - jepsen.util 173 :invoke :add 1526 INFO [2022-01-10 08:39:16,435] jepsen worker 561 - jepsen.util 561 :ok :add 1522 INFO [2022-01-10 08:39:16,436] jepsen worker 831 - jepsen.util 831 :invoke :add 1527 INFO [2022-01-10 08:39:16,437] jepsen worker 181 - jepsen.util 181 :ok :add 1524 INFO [2022-01-10 08:39:16,437] jepsen worker 281 - jepsen.util 281 :ok :add 1523 INFO [2022-01-10 08:39:16,439] jepsen worker 642 - jepsen.util 642 :invoke :add 1528 INFO [2022-01-10 08:39:16,442] jepsen worker 895 - jepsen.util 895 :invoke :add 1529 INFO [2022-01-10 08:39:16,445] jepsen worker 966 - jepsen.util 966 :ok :add 1525 INFO [2022-01-10 08:39:16,445] jepsen worker 15 - jepsen.util 15 :invoke :add 1530 INFO [2022-01-10 08:39:16,448] jepsen worker 670 - jepsen.util 670 :invoke :add 1531 INFO [2022-01-10 08:39:16,448] jepsen worker 173 - jepsen.util 173 :ok :add 1526 INFO [2022-01-10 08:39:16,450] jepsen worker 642 - jepsen.util 642 :ok :add 1528 INFO [2022-01-10 08:39:16,450] jepsen worker 831 - jepsen.util 831 :ok :add 1527 INFO [2022-01-10 08:39:16,451] jepsen worker 709 - jepsen.util 709 :invoke :add 1532 INFO [2022-01-10 08:39:16,455] jepsen worker 895 - jepsen.util 895 :ok :add 1529 INFO [2022-01-10 08:39:16,455] jepsen worker 399 - jepsen.util 399 :invoke :add 1533 INFO [2022-01-10 08:39:16,458] jepsen worker 15 - jepsen.util 15 :ok :add 1530 INFO [2022-01-10 08:39:16,460] jepsen worker 638 - jepsen.util 638 :invoke :add 1534 INFO [2022-01-10 08:39:16,463] jepsen worker 773 - jepsen.util 773 :invoke :add 1535 INFO [2022-01-10 08:39:16,465] jepsen worker 209 - jepsen.util 209 :invoke :add 1536 INFO [2022-01-10 08:39:16,468] jepsen worker 116 - jepsen.util 116 :invoke :add 1537 INFO [2022-01-10 08:39:16,471] jepsen worker 967 - jepsen.util 967 :invoke :add 1538 INFO [2022-01-10 08:39:16,471] jepsen worker 638 - jepsen.util 638 :ok :add 1534 INFO [2022-01-10 08:39:16,473] jepsen worker 986 - jepsen.util 986 :invoke :add 1539 INFO [2022-01-10 08:39:16,473] jepsen worker 773 - jepsen.util 773 :ok :add 1535 INFO [2022-01-10 08:39:16,475] jepsen worker 209 - jepsen.util 209 :ok :add 1536 INFO [2022-01-10 08:39:16,477] jepsen worker 999 - jepsen.util 999 :invoke :add 1540 INFO [2022-01-10 08:39:16,478] jepsen worker 116 - jepsen.util 116 :ok :add 1537 INFO [2022-01-10 08:39:16,481] jepsen worker 967 - jepsen.util 967 :ok :add 1538 INFO [2022-01-10 08:39:16,481] jepsen worker 986 - jepsen.util 986 :ok :add 1539 INFO [2022-01-10 08:39:16,481] jepsen worker 480 - jepsen.util 480 :invoke :add 1541 INFO [2022-01-10 08:39:16,484] jepsen worker 347 - jepsen.util 347 :invoke :add 1542 INFO [2022-01-10 08:39:16,486] jepsen worker 709 - jepsen.util 709 :ok :add 1532 INFO [2022-01-10 08:39:16,486] jepsen worker 100 - jepsen.util 100 :invoke :add 1543 INFO [2022-01-10 08:39:16,489] jepsen worker 670 - jepsen.util 670 :ok :add 1531 INFO [2022-01-10 08:39:16,489] jepsen worker 663 - jepsen.util 663 :invoke :add 1544 INFO [2022-01-10 08:39:16,492] jepsen worker 105 - jepsen.util 105 :invoke :add 1545 INFO [2022-01-10 08:39:16,494] jepsen worker 619 - jepsen.util 619 :invoke :add 1546 INFO [2022-01-10 08:39:16,496] jepsen worker 225 - jepsen.util 225 :invoke :add 1547 INFO [2022-01-10 08:39:16,498] jepsen worker 853 - jepsen.util 853 :invoke :add 1548 INFO [2022-01-10 08:39:16,500] jepsen worker 105 - jepsen.util 105 :ok :add 1545 INFO [2022-01-10 08:39:16,501] jepsen worker 663 - jepsen.util 663 :ok :add 1544 INFO [2022-01-10 08:39:16,502] jepsen worker 566 - jepsen.util 566 :invoke :add 1549 INFO [2022-01-10 08:39:16,506] jepsen worker 446 - jepsen.util 446 :invoke :add 1550 INFO [2022-01-10 08:39:16,506] jepsen worker 619 - jepsen.util 619 :ok :add 1546 INFO [2022-01-10 08:39:16,506] jepsen worker 225 - jepsen.util 225 :ok :add 1547 INFO [2022-01-10 08:39:16,508] jepsen worker 384 - jepsen.util 384 :invoke :add 1551 INFO [2022-01-10 08:39:16,511] jepsen worker 804 - jepsen.util 804 :invoke :add 1552 INFO [2022-01-10 08:39:16,515] jepsen worker 399 - jepsen.util 399 :ok :add 1533 INFO [2022-01-10 08:39:16,515] jepsen worker 316 - jepsen.util 316 :invoke :add 1553 INFO [2022-01-10 08:39:16,517] jepsen worker 141 - jepsen.util 141 :invoke :add 1554 INFO [2022-01-10 08:39:16,520] jepsen worker 270 - jepsen.util 270 :invoke :add 1555 INFO [2022-01-10 08:39:16,521] jepsen worker 446 - jepsen.util 446 :ok :add 1550 INFO [2022-01-10 08:39:16,521] jepsen worker 853 - jepsen.util 853 :ok :add 1548 INFO [2022-01-10 08:39:16,523] jepsen worker 899 - jepsen.util 899 :invoke :add 1556 INFO [2022-01-10 08:39:16,524] jepsen worker 480 - jepsen.util 480 :ok :add 1541 INFO [2022-01-10 08:39:16,526] jepsen worker 460 - jepsen.util 460 :invoke :add 1557 INFO [2022-01-10 08:39:16,526] jepsen worker 804 - jepsen.util 804 :ok :add 1552 INFO [2022-01-10 08:39:16,527] jepsen worker 566 - jepsen.util 566 :ok :add 1549 INFO [2022-01-10 08:39:16,528] jepsen worker 384 - jepsen.util 384 :ok :add 1551 INFO [2022-01-10 08:39:16,529] jepsen worker 88 - jepsen.util 88 :invoke :add 1558 INFO [2022-01-10 08:39:16,531] jepsen worker 270 - jepsen.util 270 :ok :add 1555 INFO [2022-01-10 08:39:16,532] jepsen worker 316 - jepsen.util 316 :ok :add 1553 INFO [2022-01-10 08:39:16,532] jepsen worker 585 - jepsen.util 585 :invoke :add 1559 INFO [2022-01-10 08:39:16,535] jepsen worker 460 - jepsen.util 460 :ok :add 1557 INFO [2022-01-10 08:39:16,535] jepsen worker 454 - jepsen.util 454 :invoke :add 1560 INFO [2022-01-10 08:39:16,538] jepsen worker 357 - jepsen.util 357 :invoke :add 1561 INFO [2022-01-10 08:39:16,539] jepsen worker 899 - jepsen.util 899 :ok :add 1556 INFO [2022-01-10 08:39:16,539] jepsen worker 141 - jepsen.util 141 :ok :add 1554 INFO [2022-01-10 08:39:16,541] jepsen worker 351 - jepsen.util 351 :invoke :add 1562 INFO [2022-01-10 08:39:16,543] jepsen worker 687 - jepsen.util 687 :invoke :add 1563 INFO [2022-01-10 08:39:16,545] jepsen worker 428 - jepsen.util 428 :invoke :add 1564 INFO [2022-01-10 08:39:16,548] jepsen worker 166 - jepsen.util 166 :invoke :add 1565 INFO [2022-01-10 08:39:16,550] jepsen worker 741 - jepsen.util 741 :invoke :add 1566 INFO [2022-01-10 08:39:16,553] jepsen worker 338 - jepsen.util 338 :invoke :add 1567 INFO [2022-01-10 08:39:16,556] jepsen worker 307 - jepsen.util 307 :invoke :add 1568 INFO [2022-01-10 08:39:16,558] jepsen worker 428 - jepsen.util 428 :ok :add 1564 INFO [2022-01-10 08:39:16,560] jepsen worker 106 - jepsen.util 106 :invoke :add 1569 INFO [2022-01-10 08:39:16,563] jepsen worker 307 - jepsen.util 307 :ok :add 1568 INFO [2022-01-10 08:39:16,564] jepsen worker 436 - jepsen.util 436 :invoke :add 1570 INFO [2022-01-10 08:39:16,567] jepsen worker 171 - jepsen.util 171 :invoke :add 1571 INFO [2022-01-10 08:39:16,568] jepsen worker 741 - jepsen.util 741 :ok :add 1566 INFO [2022-01-10 08:39:16,568] jepsen worker 106 - jepsen.util 106 :ok :add 1569 INFO [2022-01-10 08:39:16,571] jepsen worker 492 - jepsen.util 492 :invoke :add 1572 INFO [2022-01-10 08:39:16,572] jepsen worker 436 - jepsen.util 436 :ok :add 1570 INFO [2022-01-10 08:39:16,575] jepsen worker 913 - jepsen.util 913 :invoke :add 1573 INFO [2022-01-10 08:39:16,576] jepsen worker 999 - jepsen.util 999 :ok :add 1540 INFO [2022-01-10 08:39:16,579] jepsen worker 583 - jepsen.util 583 :invoke :add 1574 INFO [2022-01-10 08:39:16,583] jepsen worker 347 - jepsen.util 347 :ok :add 1542 INFO [2022-01-10 08:39:16,583] jepsen worker 144 - jepsen.util 144 :invoke :add 1575 INFO [2022-01-10 08:39:16,583] jepsen worker 100 - jepsen.util 100 :ok :add 1543 INFO [2022-01-10 08:39:16,588] jepsen worker 140 - jepsen.util 140 :invoke :add 1576 INFO [2022-01-10 08:39:16,590] jepsen worker 847 - jepsen.util 847 :invoke :add 1577 INFO [2022-01-10 08:39:16,593] jepsen worker 886 - jepsen.util 886 :invoke :add 1578 INFO [2022-01-10 08:39:16,593] jepsen worker 357 - jepsen.util 357 :ok :add 1561 INFO [2022-01-10 08:39:16,595] jepsen worker 454 - jepsen.util 454 :ok :add 1560 INFO [2022-01-10 08:39:16,596] jepsen worker 497 - jepsen.util 497 :invoke :add 1579 INFO [2022-01-10 08:39:16,596] jepsen worker 687 - jepsen.util 687 :ok :add 1563 INFO [2022-01-10 08:39:16,598] jepsen worker 330 - jepsen.util 330 :invoke :add 1580 INFO [2022-01-10 08:39:16,601] jepsen worker 167 - jepsen.util 167 :invoke :add 1581 INFO [2022-01-10 08:39:16,603] jepsen worker 166 - jepsen.util 166 :ok :add 1565 INFO [2022-01-10 08:39:16,603] jepsen worker 477 - jepsen.util 477 :invoke :add 1582 INFO [2022-01-10 08:39:16,606] jepsen worker 497 - jepsen.util 497 :ok :add 1579 INFO [2022-01-10 08:39:16,606] jepsen worker 315 - jepsen.util 315 :invoke :add 1583 INFO [2022-01-10 08:39:16,607] jepsen worker 338 - jepsen.util 338 :ok :add 1567 INFO [2022-01-10 08:39:16,610] jepsen worker 814 - jepsen.util 814 :invoke :add 1584 INFO [2022-01-10 08:39:16,610] jepsen worker 351 - jepsen.util 351 :ok :add 1562 INFO [2022-01-10 08:39:16,612] jepsen worker 330 - jepsen.util 330 :ok :add 1580 INFO [2022-01-10 08:39:16,612] jepsen worker 492 - jepsen.util 492 :ok :add 1572 INFO [2022-01-10 08:39:16,613] jepsen worker 171 - jepsen.util 171 :ok :add 1571 INFO [2022-01-10 08:39:16,615] jepsen worker 769 - jepsen.util 769 :invoke :add 1585 INFO [2022-01-10 08:39:16,617] jepsen worker 477 - jepsen.util 477 :ok :add 1582 INFO [2022-01-10 08:39:16,620] jepsen worker 340 - jepsen.util 340 :invoke :add 1586 INFO [2022-01-10 08:39:16,621] jepsen worker 913 - jepsen.util 913 :ok :add 1573 INFO [2022-01-10 08:39:16,624] jepsen worker 902 - jepsen.util 902 :invoke :add 1587 INFO [2022-01-10 08:39:16,625] jepsen worker 769 - jepsen.util 769 :ok :add 1585 INFO [2022-01-10 08:39:16,626] jepsen worker 886 - jepsen.util 886 :ok :add 1578 INFO [2022-01-10 08:39:16,627] jepsen worker 814 - jepsen.util 814 :ok :add 1584 INFO [2022-01-10 08:39:16,628] jepsen worker 612 - jepsen.util 612 :invoke :add 1588 INFO [2022-01-10 08:39:16,630] jepsen worker 167 - jepsen.util 167 :ok :add 1581 INFO [2022-01-10 08:39:16,632] jepsen worker 288 - jepsen.util 288 :invoke :add 1589 INFO [2022-01-10 08:39:16,634] jepsen worker 902 - jepsen.util 902 :ok :add 1587 INFO [2022-01-10 08:39:16,635] jepsen worker 315 - jepsen.util 315 :ok :add 1583 INFO [2022-01-10 08:39:16,636] jepsen worker 965 - jepsen.util 965 :invoke :add 1590 INFO [2022-01-10 08:39:16,636] jepsen worker 144 - jepsen.util 144 :ok :add 1575 INFO [2022-01-10 08:39:16,637] jepsen worker 585 - jepsen.util 585 :ok :add 1559 INFO [2022-01-10 08:39:16,637] jepsen worker 340 - jepsen.util 340 :ok :add 1586 INFO [2022-01-10 08:39:16,638] jepsen worker 288 - jepsen.util 288 :ok :add 1589 INFO [2022-01-10 08:39:16,639] jepsen worker 436 - jepsen.util 436 :invoke :add 1591 INFO [2022-01-10 08:39:16,641] jepsen worker 88 - jepsen.util 88 :ok :add 1558 INFO [2022-01-10 08:39:16,641] jepsen worker 583 - jepsen.util 583 :ok :add 1574 INFO [2022-01-10 08:39:16,643] jepsen worker 153 - jepsen.util 153 :invoke :add 1592 INFO [2022-01-10 08:39:16,645] jepsen worker 228 - jepsen.util 228 :invoke :add 1593 INFO [2022-01-10 08:39:16,647] jepsen worker 69 - jepsen.util 69 :invoke :add 1594 INFO [2022-01-10 08:39:16,648] jepsen worker 140 - jepsen.util 140 :ok :add 1576 INFO [2022-01-10 08:39:16,650] jepsen worker 349 - jepsen.util 349 :invoke :add 1595 INFO [2022-01-10 08:39:16,651] jepsen worker 612 - jepsen.util 612 :ok :add 1588 INFO [2022-01-10 08:39:16,651] jepsen worker 965 - jepsen.util 965 :ok :add 1590 INFO [2022-01-10 08:39:16,653] jepsen worker 847 - jepsen.util 847 :ok :add 1577 INFO [2022-01-10 08:39:16,653] jepsen worker 917 - jepsen.util 917 :invoke :add 1596 INFO [2022-01-10 08:39:16,656] jepsen worker 436 - jepsen.util 436 :ok :add 1591 INFO [2022-01-10 08:39:16,657] jepsen worker 149 - jepsen.util 149 :invoke :add 1597 INFO [2022-01-10 08:39:16,659] jepsen worker 131 - jepsen.util 131 :invoke :add 1598 INFO [2022-01-10 08:39:16,662] jepsen worker 858 - jepsen.util 858 :invoke :add 1599 INFO [2022-01-10 08:39:16,662] jepsen worker 228 - jepsen.util 228 :ok :add 1593 INFO [2022-01-10 08:39:16,662] jepsen worker 153 - jepsen.util 153 :ok :add 1592 INFO [2022-01-10 08:39:16,664] jepsen worker 219 - jepsen.util 219 :invoke :add 1600 INFO [2022-01-10 08:39:16,668] jepsen worker 754 - jepsen.util 754 :invoke :add 1601 INFO [2022-01-10 08:39:16,670] jepsen worker 944 - jepsen.util 944 :invoke :add 1602 INFO [2022-01-10 08:39:16,673] jepsen worker 997 - jepsen.util 997 :invoke :add 1603 INFO [2022-01-10 08:39:16,674] jepsen worker 131 - jepsen.util 131 :ok :add 1598 INFO [2022-01-10 08:39:16,675] jepsen worker 802 - jepsen.util 802 :invoke :add 1604 INFO [2022-01-10 08:39:16,676] jepsen worker 858 - jepsen.util 858 :ok :add 1599 INFO [2022-01-10 08:39:16,677] jepsen worker 219 - jepsen.util 219 :ok :add 1600 INFO [2022-01-10 08:39:16,678] jepsen worker 307 - jepsen.util 307 :invoke :add 1605 INFO [2022-01-10 08:39:16,682] jepsen worker 890 - jepsen.util 890 :invoke :add 1606 INFO [2022-01-10 08:39:16,683] jepsen worker 944 - jepsen.util 944 :ok :add 1602 INFO [2022-01-10 08:39:16,684] jepsen worker 754 - jepsen.util 754 :ok :add 1601 INFO [2022-01-10 08:39:16,686] jepsen worker 973 - jepsen.util 973 :invoke :add 1607 INFO [2022-01-10 08:39:16,689] jepsen worker 856 - jepsen.util 856 :invoke :add 1608 INFO [2022-01-10 08:39:16,690] jepsen worker 69 - jepsen.util 69 :ok :add 1594 INFO [2022-01-10 08:39:16,692] jepsen worker 706 - jepsen.util 706 :invoke :add 1609 INFO [2022-01-10 08:39:16,693] jepsen worker 349 - jepsen.util 349 :ok :add 1595 INFO [2022-01-10 08:39:16,694] jepsen worker 786 - jepsen.util 786 :invoke :add 1610 INFO [2022-01-10 08:39:16,694] jepsen worker 997 - jepsen.util 997 :ok :add 1603 INFO [2022-01-10 08:39:16,697] jepsen worker 806 - jepsen.util 806 :invoke :add 1611 INFO [2022-01-10 08:39:16,700] jepsen worker 774 - jepsen.util 774 :invoke :add 1612 INFO [2022-01-10 08:39:16,701] jepsen worker 149 - jepsen.util 149 :ok :add 1597 INFO [2022-01-10 08:39:16,702] jepsen worker 186 - jepsen.util 186 :invoke :add 1613 INFO [2022-01-10 08:39:16,703] jepsen worker 890 - jepsen.util 890 :ok :add 1606 INFO [2022-01-10 08:39:16,706] jepsen worker 174 - jepsen.util 174 :invoke :add 1614 INFO [2022-01-10 08:39:16,708] jepsen worker 917 - jepsen.util 917 :ok :add 1596 INFO [2022-01-10 08:39:16,709] jepsen worker 618 - jepsen.util 618 :invoke :add 1615 INFO [2022-01-10 08:39:16,710] jepsen worker 307 - jepsen.util 307 :ok :add 1605 INFO [2022-01-10 08:39:16,712] jepsen worker 409 - jepsen.util 409 :invoke :add 1616 INFO [2022-01-10 08:39:16,713] jepsen worker 174 - jepsen.util 174 :ok :add 1614 INFO [2022-01-10 08:39:16,716] jepsen worker 387 - jepsen.util 387 :invoke :add 1617 INFO [2022-01-10 08:39:16,717] jepsen worker 618 - jepsen.util 618 :ok :add 1615 INFO [2022-01-10 08:39:16,717] jepsen worker 973 - jepsen.util 973 :ok :add 1607 INFO [2022-01-10 08:39:16,719] jepsen worker 255 - jepsen.util 255 :invoke :add 1618 INFO [2022-01-10 08:39:16,722] jepsen worker 737 - jepsen.util 737 :invoke :add 1619 INFO [2022-01-10 08:39:16,725] jepsen worker 405 - jepsen.util 405 :invoke :add 1620 INFO [2022-01-10 08:39:16,725] jepsen worker 387 - jepsen.util 387 :ok :add 1617 INFO [2022-01-10 08:39:16,725] jepsen worker 255 - jepsen.util 255 :ok :add 1618 INFO [2022-01-10 08:39:16,726] jepsen worker 802 - jepsen.util 802 :ok :add 1604 INFO [2022-01-10 08:39:16,728] jepsen worker 737 - jepsen.util 737 :ok :add 1619 INFO [2022-01-10 08:39:16,730] jepsen worker 409 - jepsen.util 409 :ok :add 1616 INFO [2022-01-10 08:39:16,731] jepsen worker 856 - jepsen.util 856 :ok :add 1608 INFO [2022-01-10 08:39:16,731] jepsen worker 326 - jepsen.util 326 :invoke :add 1621 INFO [2022-01-10 08:39:16,734] jepsen worker 405 - jepsen.util 405 :ok :add 1620 INFO [2022-01-10 08:39:16,735] jepsen worker 233 - jepsen.util 233 :invoke :add 1622 INFO [2022-01-10 08:39:16,739] jepsen worker 645 - jepsen.util 645 :invoke :add 1623 INFO [2022-01-10 08:39:16,740] jepsen worker 326 - jepsen.util 326 :ok :add 1621 INFO [2022-01-10 08:39:16,742] jepsen worker 792 - jepsen.util 792 :invoke :add 1624 INFO [2022-01-10 08:39:16,746] jepsen worker 745 - jepsen.util 745 :invoke :add 1625 INFO [2022-01-10 08:39:16,749] jepsen worker 654 - jepsen.util 654 :invoke :add 1626 INFO [2022-01-10 08:39:16,751] jepsen worker 233 - jepsen.util 233 :ok :add 1622 INFO [2022-01-10 08:39:16,751] jepsen worker 645 - jepsen.util 645 :ok :add 1623 INFO [2022-01-10 08:39:16,752] jepsen worker 204 - jepsen.util 204 :invoke :add 1627 INFO [2022-01-10 08:39:16,754] jepsen worker 132 - jepsen.util 132 :invoke :add 1628 INFO [2022-01-10 08:39:16,757] jepsen worker 936 - jepsen.util 936 :invoke :add 1629 INFO [2022-01-10 08:39:16,760] jepsen worker 655 - jepsen.util 655 :invoke :add 1630 INFO [2022-01-10 08:39:16,763] jepsen worker 969 - jepsen.util 969 :invoke :add 1631 INFO [2022-01-10 08:39:16,764] jepsen worker 936 - jepsen.util 936 :ok :add 1629 INFO [2022-01-10 08:39:16,766] jepsen worker 150 - jepsen.util 150 :invoke :add 1632 INFO [2022-01-10 08:39:16,769] jepsen worker 41 - jepsen.util 41 :invoke :add 1633 INFO [2022-01-10 08:39:16,771] jepsen worker 431 - jepsen.util 431 :invoke :add 1634 INFO [2022-01-10 08:39:16,773] jepsen worker 969 - jepsen.util 969 :ok :add 1631 INFO [2022-01-10 08:39:16,774] jepsen worker 171 - jepsen.util 171 :invoke :add 1635 INFO [2022-01-10 08:39:16,776] jepsen worker 655 - jepsen.util 655 :ok :add 1630 INFO [2022-01-10 08:39:16,776] jepsen worker 132 - jepsen.util 132 :ok :add 1628 INFO [2022-01-10 08:39:16,776] jepsen worker 41 - jepsen.util 41 :ok :add 1633 INFO [2022-01-10 08:39:16,776] jepsen worker 786 - jepsen.util 786 :ok :add 1610 INFO [2022-01-10 08:39:16,777] jepsen worker 388 - jepsen.util 388 :invoke :add 1636 INFO [2022-01-10 08:39:16,780] jepsen worker 185 - jepsen.util 185 :invoke :add 1637 INFO [2022-01-10 08:39:16,782] jepsen worker 150 - jepsen.util 150 :ok :add 1632 INFO [2022-01-10 08:39:16,782] jepsen worker 431 - jepsen.util 431 :ok :add 1634 INFO [2022-01-10 08:39:16,782] jepsen worker 171 - jepsen.util 171 :ok :add 1635 INFO [2022-01-10 08:39:16,782] jepsen worker 774 - jepsen.util 774 :ok :add 1612 INFO [2022-01-10 08:39:16,783] jepsen worker 654 - jepsen.util 654 :ok :add 1626 INFO [2022-01-10 08:39:16,783] jepsen worker 972 - jepsen.util 972 :invoke :add 1638 INFO [2022-01-10 08:39:16,786] jepsen worker 743 - jepsen.util 743 :invoke :add 1639 INFO [2022-01-10 08:39:16,789] jepsen worker 706 - jepsen.util 706 :ok :add 1609 INFO [2022-01-10 08:39:16,789] jepsen worker 388 - jepsen.util 388 :ok :add 1636 INFO [2022-01-10 08:39:16,789] jepsen worker 806 - jepsen.util 806 :ok :add 1611 INFO [2022-01-10 08:39:16,789] jepsen worker 745 - jepsen.util 745 :ok :add 1625 INFO [2022-01-10 08:39:16,789] jepsen worker 783 - jepsen.util 783 :invoke :add 1640 INFO [2022-01-10 08:39:16,793] jepsen worker 972 - jepsen.util 972 :ok :add 1638 INFO [2022-01-10 08:39:16,793] jepsen worker 139 - jepsen.util 139 :invoke :add 1641 INFO [2022-01-10 08:39:16,794] jepsen worker 743 - jepsen.util 743 :ok :add 1639 INFO [2022-01-10 08:39:16,797] jepsen worker 762 - jepsen.util 762 :invoke :add 1642 INFO [2022-01-10 08:39:16,797] jepsen worker 792 - jepsen.util 792 :ok :add 1624 INFO [2022-01-10 08:39:16,798] jepsen worker 186 - jepsen.util 186 :ok :add 1613 INFO [2022-01-10 08:39:16,799] jepsen worker 204 - jepsen.util 204 :ok :add 1627 INFO [2022-01-10 08:39:16,799] jepsen worker 185 - jepsen.util 185 :ok :add 1637 INFO [2022-01-10 08:39:16,801] jepsen worker 86 - jepsen.util 86 :invoke :add 1643 INFO [2022-01-10 08:39:16,805] jepsen worker 661 - jepsen.util 661 :invoke :add 1644 INFO [2022-01-10 08:39:16,808] jepsen worker 749 - jepsen.util 749 :invoke :add 1645 INFO [2022-01-10 08:39:16,811] jepsen worker 324 - jepsen.util 324 :invoke :add 1646 INFO [2022-01-10 08:39:16,814] jepsen worker 749 - jepsen.util 749 :ok :add 1645 INFO [2022-01-10 08:39:16,815] jepsen worker 704 - jepsen.util 704 :invoke :add 1647 INFO [2022-01-10 08:39:16,815] jepsen worker 661 - jepsen.util 661 :ok :add 1644 INFO [2022-01-10 08:39:16,817] jepsen worker 207 - jepsen.util 207 :invoke :add 1648 INFO [2022-01-10 08:39:16,820] jepsen worker 577 - jepsen.util 577 :invoke :add 1649 INFO [2022-01-10 08:39:16,821] jepsen worker 704 - jepsen.util 704 :ok :add 1647 INFO [2022-01-10 08:39:16,821] jepsen worker 324 - jepsen.util 324 :ok :add 1646 INFO [2022-01-10 08:39:16,823] jepsen worker 999 - jepsen.util 999 :invoke :add 1650 INFO [2022-01-10 08:39:16,827] jepsen worker 622 - jepsen.util 622 :invoke :add 1651 INFO [2022-01-10 08:39:16,829] jepsen worker 646 - jepsen.util 646 :invoke :add 1652 INFO [2022-01-10 08:39:16,831] jepsen worker 980 - jepsen.util 980 :invoke :add 1653 INFO [2022-01-10 08:39:16,834] jepsen worker 445 - jepsen.util 445 :invoke :add 1654 INFO [2022-01-10 08:39:16,836] jepsen worker 852 - jepsen.util 852 :invoke :add 1655 INFO [2022-01-10 08:39:16,838] jepsen worker 818 - jepsen.util 818 :invoke :add 1656 INFO [2022-01-10 08:39:16,840] jepsen worker 792 - jepsen.util 792 :invoke :add 1657 INFO [2022-01-10 08:39:16,842] jepsen worker 445 - jepsen.util 445 :ok :add 1654 INFO [2022-01-10 08:39:16,843] jepsen worker 449 - jepsen.util 449 :invoke :add 1658 INFO [2022-01-10 08:39:16,845] jepsen worker 852 - jepsen.util 852 :ok :add 1655 INFO [2022-01-10 08:39:16,845] jepsen worker 911 - jepsen.util 911 :invoke :add 1659 INFO [2022-01-10 08:39:16,847] jepsen worker 818 - jepsen.util 818 :ok :add 1656 INFO [2022-01-10 08:39:16,849] jepsen worker 404 - jepsen.util 404 :invoke :add 1660 INFO [2022-01-10 08:39:16,850] jepsen worker 792 - jepsen.util 792 :ok :add 1657 INFO [2022-01-10 08:39:16,851] jepsen worker 207 - jepsen.util 207 :ok :add 1648 INFO [2022-01-10 08:39:16,851] jepsen worker 999 - jepsen.util 999 :ok :add 1650 INFO [2022-01-10 08:39:16,851] jepsen worker 783 - jepsen.util 783 :ok :add 1640 INFO [2022-01-10 08:39:16,852] jepsen worker 564 - jepsen.util 564 :invoke :add 1661 INFO [2022-01-10 08:39:16,853] jepsen worker 139 - jepsen.util 139 :ok :add 1641 INFO [2022-01-10 08:39:16,855] jepsen worker 548 - jepsen.util 548 :invoke :add 1662 INFO [2022-01-10 08:39:16,856] jepsen worker 646 - jepsen.util 646 :ok :add 1652 INFO [2022-01-10 08:39:16,857] jepsen worker 762 - jepsen.util 762 :ok :add 1642 INFO [2022-01-10 08:39:16,858] jepsen worker 296 - jepsen.util 296 :invoke :add 1663 INFO [2022-01-10 08:39:16,861] jepsen worker 814 - jepsen.util 814 :invoke :add 1664 INFO [2022-01-10 08:39:16,862] jepsen worker 577 - jepsen.util 577 :ok :add 1649 INFO [2022-01-10 08:39:16,863] jepsen worker 622 - jepsen.util 622 :ok :add 1651 INFO [2022-01-10 08:39:16,864] jepsen worker 341 - jepsen.util 341 :invoke :add 1665 INFO [2022-01-10 08:39:16,865] jepsen worker 86 - jepsen.util 86 :ok :add 1643 INFO [2022-01-10 08:39:16,868] jepsen worker 980 - jepsen.util 980 :ok :add 1653 INFO [2022-01-10 08:39:16,868] jepsen worker 460 - jepsen.util 460 :invoke :add 1666 INFO [2022-01-10 08:39:16,873] jepsen worker 553 - jepsen.util 553 :invoke :add 1667 INFO [2022-01-10 08:39:16,873] jepsen worker 341 - jepsen.util 341 :ok :add 1665 INFO [2022-01-10 08:39:16,876] jepsen worker 587 - jepsen.util 587 :invoke :add 1668 INFO [2022-01-10 08:39:16,878] jepsen worker 449 - jepsen.util 449 :ok :add 1658 INFO [2022-01-10 08:39:16,879] jepsen worker 248 - jepsen.util 248 :invoke :add 1669 INFO [2022-01-10 08:39:16,880] jepsen worker 911 - jepsen.util 911 :ok :add 1659 INFO [2022-01-10 08:39:16,882] jepsen worker 69 - jepsen.util 69 :invoke :add 1670 INFO [2022-01-10 08:39:16,884] jepsen worker 553 - jepsen.util 553 :ok :add 1667 INFO [2022-01-10 08:39:16,885] jepsen worker 404 - jepsen.util 404 :ok :add 1660 INFO [2022-01-10 08:39:16,885] jepsen worker 411 - jepsen.util 411 :invoke :add 1671 INFO [2022-01-10 08:39:16,885] jepsen worker 296 - jepsen.util 296 :ok :add 1663 INFO [2022-01-10 08:39:16,886] jepsen worker 814 - jepsen.util 814 :ok :add 1664 INFO [2022-01-10 08:39:16,886] jepsen worker 587 - jepsen.util 587 :ok :add 1668 INFO [2022-01-10 08:39:16,890] jepsen worker 440 - jepsen.util 440 :invoke :add 1672 INFO [2022-01-10 08:39:16,890] jepsen worker 248 - jepsen.util 248 :ok :add 1669 INFO [2022-01-10 08:39:16,890] jepsen worker 548 - jepsen.util 548 :ok :add 1662 INFO [2022-01-10 08:39:16,891] jepsen worker 69 - jepsen.util 69 :ok :add 1670 INFO [2022-01-10 08:39:16,892] jepsen worker 460 - jepsen.util 460 :ok :add 1666 INFO [2022-01-10 08:39:16,893] jepsen worker 324 - jepsen.util 324 :invoke :add 1673 INFO [2022-01-10 08:39:16,895] jepsen worker 411 - jepsen.util 411 :ok :add 1671 INFO [2022-01-10 08:39:16,897] jepsen worker 456 - jepsen.util 456 :invoke :add 1674 INFO [2022-01-10 08:39:16,898] jepsen worker 564 - jepsen.util 564 :ok :add 1661 INFO [2022-01-10 08:39:16,899] jepsen worker 823 - jepsen.util 823 :invoke :add 1675 INFO [2022-01-10 08:39:16,902] jepsen worker 807 - jepsen.util 807 :invoke :add 1676 INFO [2022-01-10 08:39:16,903] jepsen worker 440 - jepsen.util 440 :ok :add 1672 INFO [2022-01-10 08:39:16,906] jepsen worker 935 - jepsen.util 935 :invoke :add 1677 INFO [2022-01-10 08:39:16,909] jepsen worker 545 - jepsen.util 545 :invoke :add 1678 INFO [2022-01-10 08:39:16,912] jepsen worker 104 - jepsen.util 104 :invoke :add 1679 INFO [2022-01-10 08:39:16,914] jepsen worker 823 - jepsen.util 823 :ok :add 1675 INFO [2022-01-10 08:39:16,914] jepsen worker 456 - jepsen.util 456 :ok :add 1674 INFO [2022-01-10 08:39:16,915] jepsen worker 271 - jepsen.util 271 :invoke :add 1680 INFO [2022-01-10 08:39:16,918] jepsen worker 349 - jepsen.util 349 :invoke :add 1681 INFO [2022-01-10 08:39:16,919] jepsen worker 807 - jepsen.util 807 :ok :add 1676 INFO [2022-01-10 08:39:16,921] jepsen worker 356 - jepsen.util 356 :invoke :add 1682 INFO [2022-01-10 08:39:16,923] jepsen worker 935 - jepsen.util 935 :ok :add 1677 INFO [2022-01-10 08:39:16,924] jepsen worker 727 - jepsen.util 727 :invoke :add 1683 INFO [2022-01-10 08:39:16,927] jepsen worker 643 - jepsen.util 643 :invoke :add 1684 INFO [2022-01-10 08:39:16,930] jepsen worker 597 - jepsen.util 597 :invoke :add 1685 INFO [2022-01-10 08:39:16,932] jepsen worker 279 - jepsen.util 279 :invoke :add 1686 INFO [2022-01-10 08:39:16,935] jepsen worker 807 - jepsen.util 807 :invoke :add 1687 INFO [2022-01-10 08:39:16,938] jepsen worker 479 - jepsen.util 479 :invoke :add 1688 INFO [2022-01-10 08:39:16,939] jepsen worker 597 - jepsen.util 597 :ok :add 1685 INFO [2022-01-10 08:39:16,940] jepsen worker 748 - jepsen.util 748 :invoke :add 1689 INFO [2022-01-10 08:39:16,941] jepsen worker 643 - jepsen.util 643 :ok :add 1684 INFO [2022-01-10 08:39:16,943] jepsen worker 713 - jepsen.util 713 :invoke :add 1690 INFO [2022-01-10 08:39:16,945] jepsen worker 131 - jepsen.util 131 :invoke :add 1691 INFO [2022-01-10 08:39:16,948] jepsen worker 94 - jepsen.util 94 :invoke :add 1692 INFO [2022-01-10 08:39:16,950] jepsen worker 807 - jepsen.util 807 :ok :add 1687 INFO [2022-01-10 08:39:16,950] jepsen worker 279 - jepsen.util 279 :ok :add 1686 INFO [2022-01-10 08:39:16,951] jepsen worker 526 - jepsen.util 526 :invoke :add 1693 INFO [2022-01-10 08:39:16,954] jepsen worker 381 - jepsen.util 381 :invoke :add 1694 INFO [2022-01-10 08:39:16,956] jepsen worker 324 - jepsen.util 324 :ok :add 1673 INFO [2022-01-10 08:39:16,957] jepsen worker 884 - jepsen.util 884 :invoke :add 1695 INFO [2022-01-10 08:39:16,959] jepsen worker 407 - jepsen.util 407 :invoke :add 1696 INFO [2022-01-10 08:39:16,962] jepsen worker 355 - jepsen.util 355 :invoke :add 1697 INFO [2022-01-10 08:39:16,964] jepsen worker 131 - jepsen.util 131 :ok :add 1691 INFO [2022-01-10 08:39:16,964] jepsen worker 748 - jepsen.util 748 :ok :add 1689 INFO [2022-01-10 08:39:16,965] jepsen worker 381 - jepsen.util 381 :ok :add 1694 INFO [2022-01-10 08:39:16,966] jepsen worker 448 - jepsen.util 448 :invoke :add 1698 INFO [2022-01-10 08:39:16,970] jepsen worker 695 - jepsen.util 695 :invoke :add 1699 INFO [2022-01-10 08:39:16,971] jepsen worker 479 - jepsen.util 479 :ok :add 1688 INFO [2022-01-10 08:39:16,971] jepsen worker 407 - jepsen.util 407 :ok :add 1696 INFO [2022-01-10 08:39:16,971] jepsen worker 713 - jepsen.util 713 :ok :add 1690 INFO [2022-01-10 08:39:16,971] jepsen worker 526 - jepsen.util 526 :ok :add 1693 INFO [2022-01-10 08:39:16,972] jepsen worker 884 - jepsen.util 884 :ok :add 1695 INFO [2022-01-10 08:39:16,973] jepsen worker 584 - jepsen.util 584 :invoke :add 1700 INFO [2022-01-10 08:39:16,975] jepsen worker 958 - jepsen.util 958 :invoke :add 1701 INFO [2022-01-10 08:39:16,976] jepsen worker 94 - jepsen.util 94 :ok :add 1692 INFO [2022-01-10 08:39:16,977] jepsen worker 355 - jepsen.util 355 :ok :add 1697 INFO [2022-01-10 08:39:16,978] jepsen worker 357 - jepsen.util 357 :invoke :add 1702 INFO [2022-01-10 08:39:16,981] jepsen worker 848 - jepsen.util 848 :invoke :add 1703 INFO [2022-01-10 08:39:16,984] jepsen worker 802 - jepsen.util 802 :invoke :add 1704 INFO [2022-01-10 08:39:16,985] jepsen worker 104 - jepsen.util 104 :ok :add 1679 INFO [2022-01-10 08:39:16,986] jepsen worker 880 - jepsen.util 880 :invoke :add 1705 INFO [2022-01-10 08:39:16,988] jepsen worker 545 - jepsen.util 545 :ok :add 1678 INFO [2022-01-10 08:39:16,990] jepsen worker 645 - jepsen.util 645 :invoke :add 1706 INFO [2022-01-10 08:39:16,993] jepsen worker 409 - jepsen.util 409 :invoke :add 1707 INFO [2022-01-10 08:39:16,994] jepsen worker 584 - jepsen.util 584 :ok :add 1700 INFO [2022-01-10 08:39:16,996] jepsen worker 998 - jepsen.util 998 :invoke :add 1708 INFO [2022-01-10 08:39:16,997] jepsen worker 802 - jepsen.util 802 :ok :add 1704 INFO [2022-01-10 08:39:16,998] jepsen worker 513 - jepsen.util 513 :invoke :add 1709 INFO [2022-01-10 08:39:17,002] jepsen worker 114 - jepsen.util 114 :invoke :add 1710 INFO [2022-01-10 08:39:17,003] jepsen worker 880 - jepsen.util 880 :ok :add 1705 INFO [2022-01-10 08:39:17,003] jepsen worker 645 - jepsen.util 645 :ok :add 1706 INFO [2022-01-10 08:39:17,004] jepsen worker 357 - jepsen.util 357 :ok :add 1702 INFO [2022-01-10 08:39:17,005] jepsen worker 326 - jepsen.util 326 :invoke :add 1711 INFO [2022-01-10 08:39:17,008] jepsen worker 681 - jepsen.util 681 :invoke :add 1712 INFO [2022-01-10 08:39:17,011] jepsen worker 848 - jepsen.util 848 :ok :add 1703 INFO [2022-01-10 08:39:17,011] jepsen worker 455 - jepsen.util 455 :invoke :add 1713 INFO [2022-01-10 08:39:17,012] jepsen worker 409 - jepsen.util 409 :ok :add 1707 INFO [2022-01-10 08:39:17,013] jepsen worker 430 - jepsen.util 430 :invoke :add 1714 INFO [2022-01-10 08:39:17,014] jepsen worker 958 - jepsen.util 958 :ok :add 1701 INFO [2022-01-10 08:39:17,016] jepsen worker 868 - jepsen.util 868 :invoke :add 1715 INFO [2022-01-10 08:39:17,019] jepsen worker 954 - jepsen.util 954 :invoke :add 1716 INFO [2022-01-10 08:39:17,021] jepsen worker 823 - jepsen.util 823 :invoke :add 1717 INFO [2022-01-10 08:39:17,024] jepsen worker 301 - jepsen.util 301 :invoke :add 1718 INFO [2022-01-10 08:39:17,024] jepsen worker 868 - jepsen.util 868 :ok :add 1715 INFO [2022-01-10 08:39:17,026] jepsen worker 306 - jepsen.util 306 :invoke :add 1719 INFO [2022-01-10 08:39:17,026] jepsen worker 823 - jepsen.util 823 :ok :add 1717 INFO [2022-01-10 08:39:17,028] jepsen worker 430 - jepsen.util 430 :ok :add 1714 INFO [2022-01-10 08:39:17,029] jepsen worker 812 - jepsen.util 812 :invoke :add 1720 INFO [2022-01-10 08:39:17,031] jepsen worker 301 - jepsen.util 301 :ok :add 1718 INFO [2022-01-10 08:39:17,032] jepsen worker 227 - jepsen.util 227 :invoke :add 1721 INFO [2022-01-10 08:39:17,034] jepsen worker 954 - jepsen.util 954 :ok :add 1716 INFO [2022-01-10 08:39:17,035] jepsen worker 93 - jepsen.util 93 :invoke :add 1722 INFO [2022-01-10 08:39:17,039] jepsen worker 566 - jepsen.util 566 :invoke :add 1723 INFO [2022-01-10 08:39:17,040] jepsen worker 306 - jepsen.util 306 :ok :add 1719 INFO [2022-01-10 08:39:17,041] jepsen worker 227 - jepsen.util 227 :ok :add 1721 INFO [2022-01-10 08:39:17,041] jepsen worker 153 - jepsen.util 153 :invoke :add 1724 INFO [2022-01-10 08:39:17,043] jepsen worker 812 - jepsen.util 812 :ok :add 1720 INFO [2022-01-10 08:39:17,045] jepsen worker 292 - jepsen.util 292 :invoke :add 1725 INFO [2022-01-10 08:39:17,047] jepsen worker 93 - jepsen.util 93 :ok :add 1722 INFO [2022-01-10 08:39:17,047] jepsen worker 474 - jepsen.util 474 :invoke :add 1726 INFO [2022-01-10 08:39:17,050] jepsen worker 705 - jepsen.util 705 :invoke :add 1727 INFO [2022-01-10 08:39:17,052] jepsen worker 566 - jepsen.util 566 :ok :add 1723 INFO [2022-01-10 08:39:17,053] jepsen worker 728 - jepsen.util 728 :invoke :add 1728 INFO [2022-01-10 08:39:17,057] jepsen worker 558 - jepsen.util 558 :invoke :add 1729 INFO [2022-01-10 08:39:17,059] jepsen worker 849 - jepsen.util 849 :invoke :add 1730 INFO [2022-01-10 08:39:17,062] jepsen worker 684 - jepsen.util 684 :invoke :add 1731 INFO [2022-01-10 08:39:17,064] jepsen worker 674 - jepsen.util 674 :invoke :add 1732 INFO [2022-01-10 08:39:17,064] jepsen worker 558 - jepsen.util 558 :ok :add 1729 INFO [2022-01-10 08:39:17,065] jepsen worker 728 - jepsen.util 728 :ok :add 1728 INFO [2022-01-10 08:39:17,067] jepsen worker 170 - jepsen.util 170 :invoke :add 1733 INFO [2022-01-10 08:39:17,069] jepsen worker 548 - jepsen.util 548 :invoke :add 1734 INFO [2022-01-10 08:39:17,071] jepsen worker 849 - jepsen.util 849 :ok :add 1730 INFO [2022-01-10 08:39:17,072] jepsen worker 249 - jepsen.util 249 :invoke :add 1735 INFO [2022-01-10 08:39:17,075] jepsen worker 429 - jepsen.util 429 :invoke :add 1736 INFO [2022-01-10 08:39:17,076] jepsen worker 684 - jepsen.util 684 :ok :add 1731 INFO [2022-01-10 08:39:17,076] jepsen worker 674 - jepsen.util 674 :ok :add 1732 INFO [2022-01-10 08:39:17,079] jepsen worker 548 - jepsen.util 548 :ok :add 1734 INFO [2022-01-10 08:39:17,080] jepsen worker 349 - jepsen.util 349 :ok :add 1681 INFO [2022-01-10 08:39:17,080] jepsen worker 271 - jepsen.util 271 :ok :add 1680 INFO [2022-01-10 08:39:17,081] jepsen worker 58 - jepsen.util 58 :invoke :add 1737 INFO [2022-01-10 08:39:17,082] jepsen worker 695 - jepsen.util 695 :ok :add 1699 INFO [2022-01-10 08:39:17,082] jepsen worker 448 - jepsen.util 448 :ok :add 1698 INFO [2022-01-10 08:39:17,082] jepsen worker 249 - jepsen.util 249 :ok :add 1735 INFO [2022-01-10 08:39:17,082] jepsen worker 429 - jepsen.util 429 :ok :add 1736 INFO [2022-01-10 08:39:17,086] jepsen worker 860 - jepsen.util 860 :invoke :add 1738 INFO [2022-01-10 08:39:17,088] jepsen worker 170 - jepsen.util 170 :ok :add 1733 INFO [2022-01-10 08:39:17,089] jepsen worker 474 - jepsen.util 474 :ok :add 1726 INFO [2022-01-10 08:39:17,089] jepsen worker 356 - jepsen.util 356 :ok :add 1682 INFO [2022-01-10 08:39:17,089] jepsen worker 705 - jepsen.util 705 :ok :add 1727 INFO [2022-01-10 08:39:17,089] jepsen worker 455 - jepsen.util 455 :ok :add 1713 INFO [2022-01-10 08:39:17,089] jepsen worker 727 - jepsen.util 727 :ok :add 1683 INFO [2022-01-10 08:39:17,089] jepsen worker 681 - jepsen.util 681 :ok :add 1712 INFO [2022-01-10 08:39:17,090] jepsen worker 621 - jepsen.util 621 :invoke :add 1739 INFO [2022-01-10 08:39:17,095] jepsen worker 881 - jepsen.util 881 :invoke :add 1740 INFO [2022-01-10 08:39:17,095] jepsen worker 58 - jepsen.util 58 :ok :add 1737 INFO [2022-01-10 08:39:17,098] jepsen worker 693 - jepsen.util 693 :invoke :add 1741 INFO [2022-01-10 08:39:17,101] jepsen worker 780 - jepsen.util 780 :invoke :add 1742 INFO [2022-01-10 08:39:17,103] jepsen worker 621 - jepsen.util 621 :ok :add 1739 INFO [2022-01-10 08:39:17,104] jepsen worker 358 - jepsen.util 358 :invoke :add 1743 INFO [2022-01-10 08:39:17,107] jepsen worker 860 - jepsen.util 860 :ok :add 1738 INFO [2022-01-10 08:39:17,107] jepsen worker 881 - jepsen.util 881 :ok :add 1740 INFO [2022-01-10 08:39:17,108] jepsen worker 454 - jepsen.util 454 :invoke :add 1744 INFO [2022-01-10 08:39:17,108] jepsen worker 513 - jepsen.util 513 :ok :add 1709 INFO [2022-01-10 08:39:17,108] jepsen worker 292 - jepsen.util 292 :ok :add 1725 INFO [2022-01-10 08:39:17,108] jepsen worker 326 - jepsen.util 326 :ok :add 1711 INFO [2022-01-10 08:39:17,110] jepsen worker 693 - jepsen.util 693 :ok :add 1741 INFO [2022-01-10 08:39:17,112] jepsen worker 15 - jepsen.util 15 :invoke :add 1745 INFO [2022-01-10 08:39:17,113] jepsen worker 358 - jepsen.util 358 :ok :add 1743 INFO [2022-01-10 08:39:17,114] jepsen worker 780 - jepsen.util 780 :ok :add 1742 INFO [2022-01-10 08:39:17,114] jepsen worker 454 - jepsen.util 454 :ok :add 1744 INFO [2022-01-10 08:39:17,114] jepsen worker 998 - jepsen.util 998 :ok :add 1708 INFO [2022-01-10 08:39:17,116] jepsen worker 153 - jepsen.util 153 :ok :add 1724 INFO [2022-01-10 08:39:17,116] jepsen worker 114 - jepsen.util 114 :ok :add 1710 INFO [2022-01-10 08:39:17,117] jepsen worker 19 - jepsen.util 19 :invoke :add 1746 INFO [2022-01-10 08:39:17,121] jepsen worker 15 - jepsen.util 15 :ok :add 1745 INFO [2022-01-10 08:39:17,121] jepsen worker 126 - jepsen.util 126 :invoke :add 1747 INFO [2022-01-10 08:39:17,124] jepsen worker 841 - jepsen.util 841 :invoke :add 1748 INFO [2022-01-10 08:39:17,124] jepsen worker 19 - jepsen.util 19 :ok :add 1746 INFO [2022-01-10 08:39:17,126] jepsen worker 476 - jepsen.util 476 :invoke :add 1749 INFO [2022-01-10 08:39:17,128] jepsen worker 126 - jepsen.util 126 :ok :add 1747 INFO [2022-01-10 08:39:17,129] jepsen worker 95 - jepsen.util 95 :invoke :add 1750 INFO [2022-01-10 08:39:17,133] jepsen worker 799 - jepsen.util 799 :invoke :add 1751 INFO [2022-01-10 08:39:17,135] jepsen worker 145 - jepsen.util 145 :invoke :add 1752 INFO [2022-01-10 08:39:17,138] jepsen worker 870 - jepsen.util 870 :invoke :add 1753 INFO [2022-01-10 08:39:17,140] jepsen worker 520 - jepsen.util 520 :invoke :add 1754 INFO [2022-01-10 08:39:17,143] jepsen worker 676 - jepsen.util 676 :invoke :add 1755 INFO [2022-01-10 08:39:17,146] jepsen worker 539 - jepsen.util 539 :invoke :add 1756 INFO [2022-01-10 08:39:17,147] jepsen worker 520 - jepsen.util 520 :ok :add 1754 INFO [2022-01-10 08:39:17,149] jepsen worker 208 - jepsen.util 208 :invoke :add 1757 INFO [2022-01-10 08:39:17,152] jepsen worker 158 - jepsen.util 158 :invoke :add 1758 INFO [2022-01-10 08:39:17,152] jepsen worker 676 - jepsen.util 676 :ok :add 1755 INFO [2022-01-10 08:39:17,155] jepsen worker 299 - jepsen.util 299 :invoke :add 1759 INFO [2022-01-10 08:39:17,156] jepsen worker 208 - jepsen.util 208 :ok :add 1757 INFO [2022-01-10 08:39:17,158] jepsen worker 628 - jepsen.util 628 :invoke :add 1760 INFO [2022-01-10 08:39:17,158] jepsen worker 539 - jepsen.util 539 :ok :add 1756 INFO [2022-01-10 08:39:17,159] jepsen worker 476 - jepsen.util 476 :ok :add 1749 INFO [2022-01-10 08:39:17,161] jepsen worker 162 - jepsen.util 162 :invoke :add 1761 INFO [2022-01-10 08:39:17,163] jepsen worker 799 - jepsen.util 799 :ok :add 1751 INFO [2022-01-10 08:39:17,164] jepsen worker 356 - jepsen.util 356 :invoke :add 1762 INFO [2022-01-10 08:39:17,166] jepsen worker 841 - jepsen.util 841 :ok :add 1748 INFO [2022-01-10 08:39:17,167] jepsen worker 95 - jepsen.util 95 :ok :add 1750 INFO [2022-01-10 08:39:17,167] jepsen worker 870 - jepsen.util 870 :ok :add 1753 INFO [2022-01-10 08:39:17,167] jepsen worker 457 - jepsen.util 457 :invoke :add 1763 INFO [2022-01-10 08:39:17,168] jepsen worker 162 - jepsen.util 162 :ok :add 1761 INFO [2022-01-10 08:39:17,170] jepsen worker 628 - jepsen.util 628 :ok :add 1760 INFO [2022-01-10 08:39:17,171] jepsen worker 145 - jepsen.util 145 :ok :add 1752 INFO [2022-01-10 08:39:17,172] jepsen worker 92 - jepsen.util 92 :invoke :add 1764 INFO [2022-01-10 08:39:17,175] jepsen worker 767 - jepsen.util 767 :invoke :add 1765 INFO [2022-01-10 08:39:17,178] jepsen worker 243 - jepsen.util 243 :invoke :add 1766 INFO [2022-01-10 08:39:17,178] jepsen worker 92 - jepsen.util 92 :ok :add 1764 INFO [2022-01-10 08:39:17,180] jepsen worker 931 - jepsen.util 931 :invoke :add 1767 INFO [2022-01-10 08:39:17,183] jepsen worker 186 - jepsen.util 186 :invoke :add 1768 INFO [2022-01-10 08:39:17,186] jepsen worker 915 - jepsen.util 915 :invoke :add 1769 INFO [2022-01-10 08:39:17,188] jepsen worker 763 - jepsen.util 763 :invoke :add 1770 INFO [2022-01-10 08:39:17,189] jepsen worker 186 - jepsen.util 186 :ok :add 1768 INFO [2022-01-10 08:39:17,191] jepsen worker 829 - jepsen.util 829 :invoke :add 1771 INFO [2022-01-10 08:39:17,194] jepsen worker 159 - jepsen.util 159 :invoke :add 1772 INFO [2022-01-10 08:39:17,195] jepsen worker 299 - jepsen.util 299 :ok :add 1759 INFO [2022-01-10 08:39:17,196] jepsen worker 158 - jepsen.util 158 :ok :add 1758 INFO [2022-01-10 08:39:17,197] jepsen worker 915 - jepsen.util 915 :ok :add 1769 INFO [2022-01-10 08:39:17,197] jepsen worker 176 - jepsen.util 176 :invoke :add 1773 INFO [2022-01-10 08:39:17,198] jepsen worker 763 - jepsen.util 763 :ok :add 1770 INFO [2022-01-10 08:39:17,200] jepsen worker 786 - jepsen.util 786 :invoke :add 1774 INFO [2022-01-10 08:39:17,203] jepsen worker 349 - jepsen.util 349 :invoke :add 1775 INFO [2022-01-10 08:39:17,203] jepsen worker 829 - jepsen.util 829 :ok :add 1771 INFO [2022-01-10 08:39:17,203] jepsen worker 159 - jepsen.util 159 :ok :add 1772 INFO [2022-01-10 08:39:17,205] jepsen worker 629 - jepsen.util 629 :invoke :add 1776 INFO [2022-01-10 08:39:17,207] jepsen worker 176 - jepsen.util 176 :ok :add 1773 INFO [2022-01-10 08:39:17,208] jepsen worker 723 - jepsen.util 723 :invoke :add 1777 INFO [2022-01-10 08:39:17,210] jepsen worker 356 - jepsen.util 356 :ok :add 1762 INFO [2022-01-10 08:39:17,212] jepsen worker 681 - jepsen.util 681 :invoke :add 1778 INFO [2022-01-10 08:39:17,212] jepsen worker 349 - jepsen.util 349 :ok :add 1775 INFO [2022-01-10 08:39:17,217] jepsen worker 786 - jepsen.util 786 :ok :add 1774 INFO [2022-01-10 08:39:17,218] jepsen worker 858 - jepsen.util 858 :invoke :add 1780 INFO [2022-01-10 08:39:17,219] jepsen worker 272 - jepsen.util 272 :invoke :add 1779 INFO [2022-01-10 08:39:17,220] jepsen worker 723 - jepsen.util 723 :ok :add 1777 INFO [2022-01-10 08:39:17,221] jepsen worker 629 - jepsen.util 629 :ok :add 1776 INFO [2022-01-10 08:39:17,221] jepsen worker 826 - jepsen.util 826 :invoke :add 1781 INFO [2022-01-10 08:39:17,224] jepsen worker 425 - jepsen.util 425 :invoke :add 1782 INFO [2022-01-10 08:39:17,227] jepsen worker 271 - jepsen.util 271 :invoke :add 1783 INFO [2022-01-10 08:39:17,229] jepsen worker 602 - jepsen.util 602 :invoke :add 1784 INFO [2022-01-10 08:39:17,232] jepsen worker 173 - jepsen.util 173 :invoke :add 1785 INFO [2022-01-10 08:39:17,234] jepsen worker 569 - jepsen.util 569 :invoke :add 1786 INFO [2022-01-10 08:39:17,236] jepsen worker 108 - jepsen.util 108 :invoke :add 1787 INFO [2022-01-10 08:39:17,239] jepsen worker 847 - jepsen.util 847 :invoke :add 1788 INFO [2022-01-10 08:39:17,241] jepsen worker 317 - jepsen.util 317 :invoke :add 1789 INFO [2022-01-10 08:39:17,243] jepsen worker 650 - jepsen.util 650 :invoke :add 1790 INFO [2022-01-10 08:39:17,246] jepsen worker 595 - jepsen.util 595 :invoke :add 1791 INFO [2022-01-10 08:39:17,248] jepsen worker 470 - jepsen.util 470 :invoke :add 1792 INFO [2022-01-10 08:39:17,251] jepsen worker 457 - jepsen.util 457 :ok :add 1763 INFO [2022-01-10 08:39:17,251] jepsen worker 377 - jepsen.util 377 :invoke :add 1793 INFO [2022-01-10 08:39:17,254] jepsen worker 479 - jepsen.util 479 :invoke :add 1794 INFO [2022-01-10 08:39:17,256] jepsen worker 797 - jepsen.util 797 :invoke :add 1795 INFO [2022-01-10 08:39:17,258] jepsen worker 286 - jepsen.util 286 :invoke :add 1796 INFO [2022-01-10 08:39:17,260] jepsen worker 858 - jepsen.util 858 :ok :add 1780 INFO [2022-01-10 08:39:17,262] jepsen worker 405 - jepsen.util 405 :invoke :add 1797 INFO [2022-01-10 08:39:17,262] jepsen worker 681 - jepsen.util 681 :ok :add 1778 INFO [2022-01-10 08:39:17,263] jepsen worker 797 - jepsen.util 797 :ok :add 1795 INFO [2022-01-10 08:39:17,265] jepsen worker 262 - jepsen.util 262 :invoke :add 1798 INFO [2022-01-10 08:39:17,266] jepsen worker 272 - jepsen.util 272 :ok :add 1779 INFO [2022-01-10 08:39:17,266] jepsen worker 826 - jepsen.util 826 :ok :add 1781 INFO [2022-01-10 08:39:17,267] jepsen worker 346 - jepsen.util 346 :invoke :add 1799 INFO [2022-01-10 08:39:17,267] jepsen worker 479 - jepsen.util 479 :ok :add 1794 INFO [2022-01-10 08:39:17,271] jepsen worker 690 - jepsen.util 690 :invoke :add 1800 INFO [2022-01-10 08:39:17,272] jepsen worker 286 - jepsen.util 286 :ok :add 1796 INFO [2022-01-10 08:39:17,273] jepsen worker 425 - jepsen.util 425 :ok :add 1782 INFO [2022-01-10 08:39:17,273] jepsen worker 271 - jepsen.util 271 :ok :add 1783 INFO [2022-01-10 08:39:17,274] jepsen worker 564 - jepsen.util 564 :invoke :add 1801 INFO [2022-01-10 08:39:17,274] jepsen worker 767 - jepsen.util 767 :ok :add 1765 INFO [2022-01-10 08:39:17,276] jepsen worker 243 - jepsen.util 243 :ok :add 1766 INFO [2022-01-10 08:39:17,277] jepsen worker 931 - jepsen.util 931 :ok :add 1767 INFO [2022-01-10 08:39:17,278] jepsen worker 230 - jepsen.util 230 :invoke :add 1802 INFO [2022-01-10 08:39:17,280] jepsen worker 405 - jepsen.util 405 :ok :add 1797 INFO [2022-01-10 08:39:17,282] jepsen worker 175 - jepsen.util 175 :invoke :add 1803 INFO [2022-01-10 08:39:17,285] jepsen worker 847 - jepsen.util 847 :ok :add 1788 INFO [2022-01-10 08:39:17,286] jepsen worker 268 - jepsen.util 268 :invoke :add 1804 INFO [2022-01-10 08:39:17,287] jepsen worker 317 - jepsen.util 317 :ok :add 1789 INFO [2022-01-10 08:39:17,290] jepsen worker 773 - jepsen.util 773 :invoke :add 1805 INFO [2022-01-10 08:39:17,291] jepsen worker 262 - jepsen.util 262 :ok :add 1798 INFO [2022-01-10 08:39:17,294] jepsen worker 794 - jepsen.util 794 :invoke :add 1806 INFO [2022-01-10 08:39:17,294] jepsen worker 650 - jepsen.util 650 :ok :add 1790 INFO [2022-01-10 08:39:17,294] jepsen worker 346 - jepsen.util 346 :ok :add 1799 INFO [2022-01-10 08:39:17,294] jepsen worker 595 - jepsen.util 595 :ok :add 1791 INFO [2022-01-10 08:39:17,297] jepsen worker 602 - jepsen.util 602 :ok :add 1784 INFO [2022-01-10 08:39:17,297] jepsen worker 981 - jepsen.util 981 :invoke :add 1807 INFO [2022-01-10 08:39:17,300] jepsen worker 470 - jepsen.util 470 :ok :add 1792 INFO [2022-01-10 08:39:17,301] jepsen worker 569 - jepsen.util 569 :ok :add 1786 INFO [2022-01-10 08:39:17,301] jepsen worker 173 - jepsen.util 173 :ok :add 1785 INFO [2022-01-10 08:39:17,301] jepsen worker 712 - jepsen.util 712 :invoke :add 1808 INFO [2022-01-10 08:39:17,302] jepsen worker 377 - jepsen.util 377 :ok :add 1793 INFO [2022-01-10 08:39:17,302] jepsen worker 564 - jepsen.util 564 :ok :add 1801 INFO [2022-01-10 08:39:17,305] jepsen worker 715 - jepsen.util 715 :invoke :add 1809 INFO [2022-01-10 08:39:17,307] jepsen worker 108 - jepsen.util 108 :ok :add 1787 INFO [2022-01-10 08:39:17,307] jepsen worker 690 - jepsen.util 690 :ok :add 1800 INFO [2022-01-10 08:39:17,308] jepsen worker 175 - jepsen.util 175 :ok :add 1803 INFO [2022-01-10 08:39:17,308] jepsen worker 790 - jepsen.util 790 :invoke :add 1810 INFO [2022-01-10 08:39:17,311] jepsen worker 987 - jepsen.util 987 :invoke :add 1811 INFO [2022-01-10 08:39:17,315] jepsen worker 268 - jepsen.util 268 :ok :add 1804 INFO [2022-01-10 08:39:17,315] jepsen worker 230 - jepsen.util 230 :ok :add 1802 INFO [2022-01-10 08:39:17,315] jepsen worker 410 - jepsen.util 410 :invoke :add 1812 INFO [2022-01-10 08:39:17,319] jepsen worker 794 - jepsen.util 794 :ok :add 1806 INFO [2022-01-10 08:39:17,320] jepsen worker 680 - jepsen.util 680 :invoke :add 1813 INFO [2022-01-10 08:39:17,325] jepsen worker 689 - jepsen.util 689 :invoke :add 1814 INFO [2022-01-10 08:39:17,327] jepsen worker 773 - jepsen.util 773 :ok :add 1805 INFO [2022-01-10 08:39:17,330] jepsen worker 673 - jepsen.util 673 :invoke :add 1815 INFO [2022-01-10 08:39:17,333] jepsen worker 511 - jepsen.util 511 :invoke :add 1816 INFO [2022-01-10 08:39:17,334] jepsen worker 987 - jepsen.util 987 :ok :add 1811 INFO [2022-01-10 08:39:17,334] jepsen worker 715 - jepsen.util 715 :ok :add 1809 INFO [2022-01-10 08:39:17,335] jepsen worker 689 - jepsen.util 689 :ok :add 1814 INFO [2022-01-10 08:39:17,336] jepsen worker 917 - jepsen.util 917 :invoke :add 1817 INFO [2022-01-10 08:39:17,339] jepsen worker 981 - jepsen.util 981 :ok :add 1807 INFO [2022-01-10 08:39:17,340] jepsen worker 443 - jepsen.util 443 :invoke :add 1818 INFO [2022-01-10 08:39:17,340] jepsen worker 680 - jepsen.util 680 :ok :add 1813 INFO [2022-01-10 08:39:17,343] jepsen worker 673 - jepsen.util 673 :ok :add 1815 INFO [2022-01-10 08:39:17,343] jepsen worker 511 - jepsen.util 511 :ok :add 1816 INFO [2022-01-10 08:39:17,345] jepsen worker 842 - jepsen.util 842 :invoke :add 1819 INFO [2022-01-10 08:39:17,350] jepsen worker 917 - jepsen.util 917 :ok :add 1817 INFO [2022-01-10 08:39:17,351] jepsen worker 807 - jepsen.util 807 :invoke :add 1820 INFO [2022-01-10 08:39:17,351] jepsen worker 443 - jepsen.util 443 :ok :add 1818 INFO [2022-01-10 08:39:17,352] jepsen worker 842 - jepsen.util 842 :ok :add 1819 INFO [2022-01-10 08:39:17,353] jepsen worker 790 - jepsen.util 790 :ok :add 1810 INFO [2022-01-10 08:39:17,353] jepsen worker 712 - jepsen.util 712 :ok :add 1808 INFO [2022-01-10 08:39:17,355] jepsen worker 901 - jepsen.util 901 :invoke :add 1821 INFO [2022-01-10 08:39:17,357] jepsen worker 410 - jepsen.util 410 :ok :add 1812 INFO [2022-01-10 08:39:17,359] jepsen worker 496 - jepsen.util 496 :invoke :add 1822 INFO [2022-01-10 08:39:17,362] jepsen worker 617 - jepsen.util 617 :invoke :add 1823 INFO [2022-01-10 08:39:17,365] jepsen worker 815 - jepsen.util 815 :invoke :add 1824 INFO [2022-01-10 08:39:17,366] jepsen worker 807 - jepsen.util 807 :ok :add 1820 INFO [2022-01-10 08:39:17,367] jepsen worker 901 - jepsen.util 901 :ok :add 1821 INFO [2022-01-10 08:39:17,368] jepsen worker 806 - jepsen.util 806 :invoke :add 1825 INFO [2022-01-10 08:39:17,371] jepsen worker 698 - jepsen.util 698 :invoke :add 1826 INFO [2022-01-10 08:39:17,371] jepsen worker 496 - jepsen.util 496 :ok :add 1822 INFO [2022-01-10 08:39:17,371] jepsen worker 617 - jepsen.util 617 :ok :add 1823 INFO [2022-01-10 08:39:17,374] jepsen worker 112 - jepsen.util 112 :invoke :add 1827 INFO [2022-01-10 08:39:17,377] jepsen worker 806 - jepsen.util 806 :ok :add 1825 INFO [2022-01-10 08:39:17,378] jepsen worker 437 - jepsen.util 437 :invoke :add 1828 INFO [2022-01-10 08:39:17,380] jepsen worker 292 - jepsen.util 292 :invoke :add 1829 INFO [2022-01-10 08:39:17,382] jepsen worker 815 - jepsen.util 815 :ok :add 1824 INFO [2022-01-10 08:39:17,382] jepsen worker 273 - jepsen.util 273 :invoke :add 1830 INFO [2022-01-10 08:39:17,385] jepsen worker 134 - jepsen.util 134 :invoke :add 1831 INFO [2022-01-10 08:39:17,390] jepsen worker 437 - jepsen.util 437 :ok :add 1828 INFO [2022-01-10 08:39:17,392] jepsen worker 273 - jepsen.util 273 :ok :add 1830 INFO [2022-01-10 08:39:17,392] jepsen worker 952 - jepsen.util 952 :invoke :add 1832 INFO [2022-01-10 08:39:17,395] jepsen worker 292 - jepsen.util 292 :ok :add 1829 INFO [2022-01-10 08:39:17,395] jepsen worker 512 - jepsen.util 512 :invoke :add 1833 INFO [2022-01-10 08:39:17,398] jepsen worker 308 - jepsen.util 308 :invoke :add 1834 INFO [2022-01-10 08:39:17,400] jepsen worker 134 - jepsen.util 134 :ok :add 1831 INFO [2022-01-10 08:39:17,401] jepsen worker 735 - jepsen.util 735 :invoke :add 1835 INFO [2022-01-10 08:39:17,404] jepsen worker 857 - jepsen.util 857 :invoke :add 1836 INFO [2022-01-10 08:39:17,405] jepsen worker 952 - jepsen.util 952 :ok :add 1832 INFO [2022-01-10 08:39:17,407] jepsen worker 301 - jepsen.util 301 :invoke :add 1837 INFO [2022-01-10 08:39:17,408] jepsen worker 308 - jepsen.util 308 :ok :add 1834 INFO [2022-01-10 08:39:17,408] jepsen worker 512 - jepsen.util 512 :ok :add 1833 INFO [2022-01-10 08:39:17,408] jepsen worker 735 - jepsen.util 735 :ok :add 1835 INFO [2022-01-10 08:39:17,410] jepsen worker 668 - jepsen.util 668 :invoke :add 1838 INFO [2022-01-10 08:39:17,413] jepsen worker 584 - jepsen.util 584 :invoke :add 1839 INFO [2022-01-10 08:39:17,416] jepsen worker 120 - jepsen.util 120 :invoke :add 1840 INFO [2022-01-10 08:39:17,417] jepsen worker 301 - jepsen.util 301 :ok :add 1837 INFO [2022-01-10 08:39:17,418] jepsen worker 857 - jepsen.util 857 :ok :add 1836 INFO [2022-01-10 08:39:17,419] jepsen worker 709 - jepsen.util 709 :invoke :add 1841 INFO [2022-01-10 08:39:17,419] jepsen worker 668 - jepsen.util 668 :ok :add 1838 INFO [2022-01-10 08:39:17,420] jepsen worker 584 - jepsen.util 584 :ok :add 1839 INFO [2022-01-10 08:39:17,422] jepsen worker 625 - jepsen.util 625 :invoke :add 1842 INFO [2022-01-10 08:39:17,425] jepsen worker 719 - jepsen.util 719 :invoke :add 1843 INFO [2022-01-10 08:39:17,428] jepsen worker 328 - jepsen.util 328 :invoke :add 1844 INFO [2022-01-10 08:39:17,429] jepsen worker 709 - jepsen.util 709 :ok :add 1841 INFO [2022-01-10 08:39:17,431] jepsen worker 597 - jepsen.util 597 :invoke :add 1845 INFO [2022-01-10 08:39:17,433] jepsen worker 120 - jepsen.util 120 :ok :add 1840 INFO [2022-01-10 08:39:17,434] jepsen worker 674 - jepsen.util 674 :invoke :add 1846 INFO [2022-01-10 08:39:17,436] jepsen worker 625 - jepsen.util 625 :ok :add 1842 INFO [2022-01-10 08:39:17,437] jepsen worker 917 - jepsen.util 917 :invoke :add 1847 INFO [2022-01-10 08:39:17,437] jepsen worker 719 - jepsen.util 719 :ok :add 1843 INFO [2022-01-10 08:39:17,441] jepsen worker 840 - jepsen.util 840 :invoke :add 1848 INFO [2022-01-10 08:39:17,441] jepsen worker 597 - jepsen.util 597 :ok :add 1845 INFO [2022-01-10 08:39:17,441] jepsen worker 328 - jepsen.util 328 :ok :add 1844 INFO [2022-01-10 08:39:17,444] jepsen worker 212 - jepsen.util 212 :invoke :add 1849 INFO [2022-01-10 08:39:17,446] jepsen worker 674 - jepsen.util 674 :ok :add 1846 INFO [2022-01-10 08:39:17,446] jepsen worker 917 - jepsen.util 917 :ok :add 1847 INFO [2022-01-10 08:39:17,447] jepsen worker 307 - jepsen.util 307 :invoke :add 1850 INFO [2022-01-10 08:39:17,450] jepsen worker 383 - jepsen.util 383 :invoke :add 1851 INFO [2022-01-10 08:39:17,451] jepsen worker 112 - jepsen.util 112 :ok :add 1827 INFO [2022-01-10 08:39:17,453] jepsen worker 698 - jepsen.util 698 :ok :add 1826 INFO [2022-01-10 08:39:17,453] jepsen worker 976 - jepsen.util 976 :invoke :add 1852 INFO [2022-01-10 08:39:17,456] jepsen worker 22 - jepsen.util 22 :invoke :add 1853 INFO [2022-01-10 08:39:17,456] jepsen worker 212 - jepsen.util 212 :ok :add 1849 INFO [2022-01-10 08:39:17,460] jepsen worker 840 - jepsen.util 840 :ok :add 1848 INFO [2022-01-10 08:39:17,460] jepsen worker 307 - jepsen.util 307 :ok :add 1850 INFO [2022-01-10 08:39:17,460] jepsen worker 209 - jepsen.util 209 :invoke :add 1854 INFO [2022-01-10 08:39:17,464] jepsen worker 266 - jepsen.util 266 :invoke :add 1855 INFO [2022-01-10 08:39:17,466] jepsen worker 904 - jepsen.util 904 :invoke :add 1856 INFO [2022-01-10 08:39:17,470] jepsen worker 972 - jepsen.util 972 :invoke :add 1857 INFO [2022-01-10 08:39:17,473] jepsen worker 584 - jepsen.util 584 :invoke :add 1858 INFO [2022-01-10 08:39:17,476] jepsen worker 900 - jepsen.util 900 :invoke :add 1859 INFO [2022-01-10 08:39:17,477] jepsen worker 209 - jepsen.util 209 :ok :add 1854 INFO [2022-01-10 08:39:17,480] jepsen worker 384 - jepsen.util 384 :invoke :add 1860 INFO [2022-01-10 08:39:17,483] jepsen worker 266 - jepsen.util 266 :ok :add 1855 INFO [2022-01-10 08:39:17,483] jepsen worker 912 - jepsen.util 912 :invoke :add 1861 INFO [2022-01-10 08:39:17,483] jepsen worker 904 - jepsen.util 904 :ok :add 1856 INFO [2022-01-10 08:39:17,483] jepsen worker 584 - jepsen.util 584 :ok :add 1858 INFO [2022-01-10 08:39:17,484] jepsen worker 900 - jepsen.util 900 :ok :add 1859 INFO [2022-01-10 08:39:17,486] jepsen worker 413 - jepsen.util 413 :invoke :add 1862 INFO [2022-01-10 08:39:17,487] jepsen worker 972 - jepsen.util 972 :ok :add 1857 INFO [2022-01-10 08:39:17,489] jepsen worker 384 - jepsen.util 384 :ok :add 1860 INFO [2022-01-10 08:39:17,490] jepsen worker 574 - jepsen.util 574 :invoke :add 1863 INFO [2022-01-10 08:39:17,493] jepsen worker 619 - jepsen.util 619 :invoke :add 1864 INFO [2022-01-10 08:39:17,493] jepsen worker 912 - jepsen.util 912 :ok :add 1861 INFO [2022-01-10 08:39:17,496] jepsen worker 595 - jepsen.util 595 :invoke :add 1865 INFO [2022-01-10 08:39:17,497] jepsen worker 574 - jepsen.util 574 :ok :add 1863 INFO [2022-01-10 08:39:17,500] jepsen worker 717 - jepsen.util 717 :invoke :add 1866 INFO [2022-01-10 08:39:17,502] jepsen worker 413 - jepsen.util 413 :ok :add 1862 INFO [2022-01-10 08:39:17,503] jepsen worker 383 - jepsen.util 383 :ok :add 1851 INFO [2022-01-10 08:39:17,505] jepsen worker 96 - jepsen.util 96 :invoke :add 1867 INFO [2022-01-10 08:39:17,506] jepsen worker 976 - jepsen.util 976 :ok :add 1852 INFO [2022-01-10 08:39:17,510] jepsen worker 233 - jepsen.util 233 :invoke :add 1868 INFO [2022-01-10 08:39:17,510] jepsen worker 22 - jepsen.util 22 :ok :add 1853 INFO [2022-01-10 08:39:17,516] jepsen worker 486 - jepsen.util 486 :invoke :add 1869 INFO [2022-01-10 08:39:17,518] jepsen worker 233 - jepsen.util 233 :ok :add 1868 INFO [2022-01-10 08:39:17,519] jepsen worker 619 - jepsen.util 619 :ok :add 1864 INFO [2022-01-10 08:39:17,519] jepsen worker 187 - jepsen.util 187 :invoke :add 1870 INFO [2022-01-10 08:39:17,521] jepsen worker 733 - jepsen.util 733 :invoke :add 1871 INFO [2022-01-10 08:39:17,523] jepsen worker 164 - jepsen.util 164 :invoke :add 1872 INFO [2022-01-10 08:39:17,526] jepsen worker 595 - jepsen.util 595 :ok :add 1865 INFO [2022-01-10 08:39:17,526] jepsen worker 717 - jepsen.util 717 :ok :add 1866 INFO [2022-01-10 08:39:17,526] jepsen worker 486 - jepsen.util 486 :ok :add 1869 INFO [2022-01-10 08:39:17,526] jepsen worker 187 - jepsen.util 187 :ok :add 1870 INFO [2022-01-10 08:39:17,526] jepsen worker 253 - jepsen.util 253 :invoke :add 1873 INFO [2022-01-10 08:39:17,530] jepsen worker 545 - jepsen.util 545 :invoke :add 1874 INFO [2022-01-10 08:39:17,530] jepsen worker 96 - jepsen.util 96 :ok :add 1867 INFO [2022-01-10 08:39:17,531] jepsen worker 733 - jepsen.util 733 :ok :add 1871 INFO [2022-01-10 08:39:17,532] jepsen worker 164 - jepsen.util 164 :ok :add 1872 INFO [2022-01-10 08:39:17,533] jepsen worker 732 - jepsen.util 732 :invoke :add 1875 INFO [2022-01-10 08:39:17,536] jepsen worker 253 - jepsen.util 253 :ok :add 1873 INFO [2022-01-10 08:39:17,537] jepsen worker 15 - jepsen.util 15 :invoke :add 1876 INFO [2022-01-10 08:39:17,540] jepsen worker 208 - jepsen.util 208 :invoke :add 1877 INFO [2022-01-10 08:39:17,542] jepsen worker 340 - jepsen.util 340 :invoke :add 1878 INFO [2022-01-10 08:39:17,545] jepsen worker 494 - jepsen.util 494 :invoke :add 1879 INFO [2022-01-10 08:39:17,548] jepsen worker 254 - jepsen.util 254 :invoke :add 1880 INFO [2022-01-10 08:39:17,550] jepsen worker 852 - jepsen.util 852 :invoke :add 1881 INFO [2022-01-10 08:39:17,552] jepsen worker 340 - jepsen.util 340 :ok :add 1878 INFO [2022-01-10 08:39:17,553] jepsen worker 729 - jepsen.util 729 :invoke :add 1882 INFO [2022-01-10 08:39:17,556] jepsen worker 66 - jepsen.util 66 :invoke :add 1883 INFO [2022-01-10 08:39:17,559] jepsen worker 494 - jepsen.util 494 :ok :add 1879 INFO [2022-01-10 08:39:17,559] jepsen worker 18 - jepsen.util 18 :invoke :add 1884 INFO [2022-01-10 08:39:17,561] jepsen worker 254 - jepsen.util 254 :ok :add 1880 INFO [2022-01-10 08:39:17,563] jepsen worker 801 - jepsen.util 801 :invoke :add 1885 INFO [2022-01-10 08:39:17,563] jepsen worker 545 - jepsen.util 545 :ok :add 1874 INFO [2022-01-10 08:39:17,564] jepsen worker 852 - jepsen.util 852 :ok :add 1881 INFO [2022-01-10 08:39:17,566] jepsen worker 428 - jepsen.util 428 :invoke :add 1886 INFO [2022-01-10 08:39:17,566] jepsen worker 729 - jepsen.util 729 :ok :add 1882 INFO [2022-01-10 08:39:17,570] jepsen worker 587 - jepsen.util 587 :invoke :add 1887 INFO [2022-01-10 08:39:17,570] jepsen worker 66 - jepsen.util 66 :ok :add 1883 INFO [2022-01-10 08:39:17,570] jepsen worker 18 - jepsen.util 18 :ok :add 1884 INFO [2022-01-10 08:39:17,573] jepsen worker 15 - jepsen.util 15 :ok :add 1876 INFO [2022-01-10 08:39:17,573] jepsen worker 946 - jepsen.util 946 :invoke :add 1888 INFO [2022-01-10 08:39:17,574] jepsen worker 732 - jepsen.util 732 :ok :add 1875 INFO [2022-01-10 08:39:17,574] jepsen worker 801 - jepsen.util 801 :ok :add 1885 INFO [2022-01-10 08:39:17,578] jepsen worker 428 - jepsen.util 428 :ok :add 1886 INFO [2022-01-10 08:39:17,580] jepsen worker 756 - jepsen.util 756 :invoke :add 1889 INFO [2022-01-10 08:39:17,582] jepsen worker 208 - jepsen.util 208 :ok :add 1877 INFO [2022-01-10 08:39:17,583] jepsen worker 587 - jepsen.util 587 :ok :add 1887 INFO [2022-01-10 08:39:17,584] jepsen worker 418 - jepsen.util 418 :invoke :add 1890 INFO [2022-01-10 08:39:17,588] jepsen worker 669 - jepsen.util 669 :invoke :add 1891 INFO [2022-01-10 08:39:17,592] jepsen worker 460 - jepsen.util 460 :invoke :add 1892 INFO [2022-01-10 08:39:17,594] jepsen worker 961 - jepsen.util 961 :invoke :add 1893 INFO [2022-01-10 08:39:17,597] jepsen worker 65 - jepsen.util 65 :invoke :add 1894 INFO [2022-01-10 08:39:17,599] jepsen worker 188 - jepsen.util 188 :invoke :add 1895 INFO [2022-01-10 08:39:17,603] jepsen worker 150 - jepsen.util 150 :invoke :add 1896 INFO [2022-01-10 08:39:17,604] jepsen worker 65 - jepsen.util 65 :ok :add 1894 INFO [2022-01-10 08:39:17,605] jepsen worker 16 - jepsen.util 16 :invoke :add 1897 INFO [2022-01-10 08:39:17,608] jepsen worker 563 - jepsen.util 563 :invoke :add 1898 INFO [2022-01-10 08:39:17,610] jepsen worker 188 - jepsen.util 188 :ok :add 1895 INFO [2022-01-10 08:39:17,611] jepsen worker 862 - jepsen.util 862 :invoke :add 1899 INFO [2022-01-10 08:39:17,614] jepsen worker 150 - jepsen.util 150 :ok :add 1896 INFO [2022-01-10 08:39:17,615] jepsen worker 133 - jepsen.util 133 :invoke :add 1900 INFO [2022-01-10 08:39:17,615] jepsen worker 16 - jepsen.util 16 :ok :add 1897 INFO [2022-01-10 08:39:17,617] jepsen worker 980 - jepsen.util 980 :invoke :add 1901 INFO [2022-01-10 08:39:17,620] jepsen worker 760 - jepsen.util 760 :invoke :add 1902 INFO [2022-01-10 08:39:17,623] jepsen worker 964 - jepsen.util 964 :invoke :add 1903 INFO [2022-01-10 08:39:17,626] jepsen worker 377 - jepsen.util 377 :invoke :add 1904 INFO [2022-01-10 08:39:17,628] jepsen worker 555 - jepsen.util 555 :invoke :add 1905 INFO [2022-01-10 08:39:17,631] jepsen worker 193 - jepsen.util 193 :invoke :add 1906 INFO [2022-01-10 08:39:17,634] jepsen worker 12 - jepsen.util 12 :invoke :add 1907 INFO [2022-01-10 08:39:17,636] jepsen worker 7 - jepsen.util 7 :invoke :add 1908 INFO [2022-01-10 08:39:17,637] jepsen worker 555 - jepsen.util 555 :ok :add 1905 INFO [2022-01-10 08:39:17,637] jepsen worker 377 - jepsen.util 377 :ok :add 1904 INFO [2022-01-10 08:39:17,638] jepsen worker 980 - jepsen.util 980 :ok :add 1901 INFO [2022-01-10 08:39:17,638] jepsen worker 756 - jepsen.util 756 :ok :add 1889 INFO [2022-01-10 08:39:17,638] jepsen worker 669 - jepsen.util 669 :ok :add 1891 INFO [2022-01-10 08:39:17,640] jepsen worker 228 - jepsen.util 228 :invoke :add 1909 INFO [2022-01-10 08:39:17,641] jepsen worker 193 - jepsen.util 193 :ok :add 1906 INFO [2022-01-10 08:39:17,642] jepsen worker 961 - jepsen.util 961 :ok :add 1893 INFO [2022-01-10 08:39:17,642] jepsen worker 964 - jepsen.util 964 :ok :add 1903 INFO [2022-01-10 08:39:17,645] jepsen worker 108 - jepsen.util 108 :invoke :add 1910 INFO [2022-01-10 08:39:17,646] jepsen worker 12 - jepsen.util 12 :ok :add 1907 INFO [2022-01-10 08:39:17,648] jepsen worker 150 - jepsen.util 150 :invoke :add 1911 INFO [2022-01-10 08:39:17,651] jepsen worker 325 - jepsen.util 325 :invoke :add 1912 INFO [2022-01-10 08:39:17,653] jepsen worker 563 - jepsen.util 563 :ok :add 1898 INFO [2022-01-10 08:39:17,654] jepsen worker 424 - jepsen.util 424 :invoke :add 1913 INFO [2022-01-10 08:39:17,656] jepsen worker 862 - jepsen.util 862 :ok :add 1899 INFO [2022-01-10 08:39:17,656] jepsen worker 418 - jepsen.util 418 :ok :add 1890 INFO [2022-01-10 08:39:17,656] jepsen worker 946 - jepsen.util 946 :ok :add 1888 INFO [2022-01-10 08:39:17,656] jepsen worker 133 - jepsen.util 133 :ok :add 1900 INFO [2022-01-10 08:39:17,657] jepsen worker 671 - jepsen.util 671 :invoke :add 1914 INFO [2022-01-10 08:39:17,661] jepsen worker 399 - jepsen.util 399 :invoke :add 1915 INFO [2022-01-10 08:39:17,663] jepsen worker 228 - jepsen.util 228 :ok :add 1909 INFO [2022-01-10 08:39:17,663] jepsen worker 150 - jepsen.util 150 :ok :add 1911 INFO [2022-01-10 08:39:17,664] jepsen worker 367 - jepsen.util 367 :invoke :add 1916 INFO [2022-01-10 08:39:17,666] jepsen worker 671 - jepsen.util 671 :ok :add 1914 INFO [2022-01-10 08:39:17,666] jepsen worker 424 - jepsen.util 424 :ok :add 1913 INFO [2022-01-10 08:39:17,667] jepsen worker 760 - jepsen.util 760 :ok :add 1902 INFO [2022-01-10 08:39:17,667] jepsen worker 460 - jepsen.util 460 :ok :add 1892 INFO [2022-01-10 08:39:17,670] jepsen worker 931 - jepsen.util 931 :invoke :add 1917 INFO [2022-01-10 08:39:17,671] jepsen worker 399 - jepsen.util 399 :ok :add 1915 INFO [2022-01-10 08:39:17,675] jepsen worker 894 - jepsen.util 894 :invoke :add 1918 INFO [2022-01-10 08:39:17,675] jepsen worker 7 - jepsen.util 7 :ok :add 1908 INFO [2022-01-10 08:39:17,676] jepsen worker 108 - jepsen.util 108 :ok :add 1910 INFO [2022-01-10 08:39:17,677] jepsen worker 367 - jepsen.util 367 :ok :add 1916 INFO [2022-01-10 08:39:17,677] jepsen worker 931 - jepsen.util 931 :ok :add 1917 INFO [2022-01-10 08:39:17,680] jepsen worker 325 - jepsen.util 325 :ok :add 1912 INFO [2022-01-10 08:39:17,681] jepsen worker 381 - jepsen.util 381 :invoke :add 1919 INFO [2022-01-10 08:39:17,686] jepsen worker 418 - jepsen.util 418 :invoke :add 1920 INFO [2022-01-10 08:39:17,686] jepsen worker 894 - jepsen.util 894 :ok :add 1918 INFO [2022-01-10 08:39:17,689] jepsen worker 810 - jepsen.util 810 :invoke :add 1921 INFO [2022-01-10 08:39:17,690] jepsen worker 381 - jepsen.util 381 :ok :add 1919 INFO [2022-01-10 08:39:17,692] jepsen worker 547 - jepsen.util 547 :invoke :add 1922 INFO [2022-01-10 08:39:17,696] jepsen worker 418 - jepsen.util 418 :ok :add 1920 INFO [2022-01-10 08:39:17,697] jepsen worker 766 - jepsen.util 766 :invoke :add 1923 INFO [2022-01-10 08:39:17,700] jepsen worker 547 - jepsen.util 547 :ok :add 1922 INFO [2022-01-10 08:39:17,700] jepsen worker 78 - jepsen.util 78 :invoke :add 1924 INFO [2022-01-10 08:39:17,703] jepsen worker 154 - jepsen.util 154 :invoke :add 1925 INFO [2022-01-10 08:39:17,706] jepsen worker 810 - jepsen.util 810 :ok :add 1921 INFO [2022-01-10 08:39:17,707] jepsen worker 781 - jepsen.util 781 :invoke :add 1926 INFO [2022-01-10 08:39:17,709] jepsen worker 766 - jepsen.util 766 :ok :add 1923 INFO [2022-01-10 08:39:17,710] jepsen worker 221 - jepsen.util 221 :invoke :add 1927 INFO [2022-01-10 08:39:17,712] jepsen worker 78 - jepsen.util 78 :ok :add 1924 INFO [2022-01-10 08:39:17,714] jepsen worker 648 - jepsen.util 648 :invoke :add 1928 INFO [2022-01-10 08:39:17,717] jepsen worker 154 - jepsen.util 154 :ok :add 1925 INFO [2022-01-10 08:39:17,717] jepsen worker 848 - jepsen.util 848 :invoke :add 1929 INFO [2022-01-10 08:39:17,722] jepsen worker 435 - jepsen.util 435 :invoke :add 1930 INFO [2022-01-10 08:39:17,723] jepsen worker 648 - jepsen.util 648 :ok :add 1928 INFO [2022-01-10 08:39:17,723] jepsen worker 848 - jepsen.util 848 :ok :add 1929 INFO [2022-01-10 08:39:17,724] jepsen worker 221 - jepsen.util 221 :ok :add 1927 INFO [2022-01-10 08:39:17,725] jepsen worker 770 - jepsen.util 770 :invoke :add 1931 INFO [2022-01-10 08:39:17,728] jepsen worker 781 - jepsen.util 781 :ok :add 1926 INFO [2022-01-10 08:39:17,729] jepsen worker 955 - jepsen.util 955 :invoke :add 1932 INFO [2022-01-10 08:39:17,732] jepsen worker 237 - jepsen.util 237 :invoke :add 1933 INFO [2022-01-10 08:39:17,736] jepsen worker 219 - jepsen.util 219 :invoke :add 1934 INFO [2022-01-10 08:39:17,737] jepsen worker 435 - jepsen.util 435 :ok :add 1930 INFO [2022-01-10 08:39:17,739] jepsen worker 770 - jepsen.util 770 :ok :add 1931 INFO [2022-01-10 08:39:17,739] jepsen worker 446 - jepsen.util 446 :invoke :add 1935 INFO [2022-01-10 08:39:17,743] jepsen worker 237 - jepsen.util 237 :ok :add 1933 INFO [2022-01-10 08:39:17,743] jepsen worker 955 - jepsen.util 955 :ok :add 1932 INFO [2022-01-10 08:39:17,743] jepsen worker 679 - jepsen.util 679 :invoke :add 1936 INFO [2022-01-10 08:39:17,745] jepsen worker 219 - jepsen.util 219 :ok :add 1934 INFO [2022-01-10 08:39:17,748] jepsen worker 878 - jepsen.util 878 :invoke :add 1937 INFO [2022-01-10 08:39:17,749] jepsen worker 446 - jepsen.util 446 :ok :add 1935 INFO [2022-01-10 08:39:17,753] jepsen worker 66 - jepsen.util 66 :invoke :add 1938 INFO [2022-01-10 08:39:17,756] jepsen worker 92 - jepsen.util 92 :invoke :add 1939 INFO [2022-01-10 08:39:17,757] jepsen worker 679 - jepsen.util 679 :ok :add 1936 INFO [2022-01-10 08:39:17,758] jepsen worker 207 - jepsen.util 207 :invoke :add 1940 INFO [2022-01-10 08:39:17,759] jepsen worker 878 - jepsen.util 878 :ok :add 1937 INFO [2022-01-10 08:39:17,760] jepsen worker 66 - jepsen.util 66 :ok :add 1938 INFO [2022-01-10 08:39:17,761] jepsen worker 959 - jepsen.util 959 :invoke :add 1941 INFO [2022-01-10 08:39:17,765] jepsen worker 889 - jepsen.util 889 :invoke :add 1942 INFO [2022-01-10 08:39:17,768] jepsen worker 165 - jepsen.util 165 :invoke :add 1943 INFO [2022-01-10 08:39:17,769] jepsen worker 207 - jepsen.util 207 :ok :add 1940 INFO [2022-01-10 08:39:17,770] jepsen worker 959 - jepsen.util 959 :ok :add 1941 INFO [2022-01-10 08:39:17,770] jepsen worker 865 - jepsen.util 865 :invoke :add 1944 INFO [2022-01-10 08:39:17,773] jepsen worker 153 - jepsen.util 153 :invoke :add 1945 INFO [2022-01-10 08:39:17,774] jepsen worker 889 - jepsen.util 889 :ok :add 1942 INFO [2022-01-10 08:39:17,776] jepsen worker 253 - jepsen.util 253 :invoke :add 1946 INFO [2022-01-10 08:39:17,778] jepsen worker 165 - jepsen.util 165 :ok :add 1943 INFO [2022-01-10 08:39:17,779] jepsen worker 950 - jepsen.util 950 :invoke :add 1947 INFO [2022-01-10 08:39:17,782] jepsen worker 729 - jepsen.util 729 :invoke :add 1948 INFO [2022-01-10 08:39:17,783] jepsen worker 865 - jepsen.util 865 :ok :add 1944 INFO [2022-01-10 08:39:17,786] jepsen worker 935 - jepsen.util 935 :invoke :add 1949 INFO [2022-01-10 08:39:17,788] jepsen worker 153 - jepsen.util 153 :ok :add 1945 INFO [2022-01-10 08:39:17,788] jepsen worker 279 - jepsen.util 279 :invoke :add 1950 INFO [2022-01-10 08:39:17,791] jepsen worker 466 - jepsen.util 466 :invoke :add 1951 INFO [2022-01-10 08:39:17,796] jepsen worker 122 - jepsen.util 122 :invoke :add 1952 INFO [2022-01-10 08:39:17,797] jepsen worker 253 - jepsen.util 253 :ok :add 1946 INFO [2022-01-10 08:39:17,799] jepsen worker 259 - jepsen.util 259 :invoke :add 1953 INFO [2022-01-10 08:39:17,804] jepsen worker 178 - jepsen.util 178 :invoke :add 1954 INFO [2022-01-10 08:39:17,806] jepsen worker 950 - jepsen.util 950 :ok :add 1947 INFO [2022-01-10 08:39:17,809] jepsen worker 969 - jepsen.util 969 :invoke :add 1955 INFO [2022-01-10 08:39:17,813] jepsen worker 918 - jepsen.util 918 :invoke :add 1956 INFO [2022-01-10 08:39:17,815] jepsen worker 178 - jepsen.util 178 :ok :add 1954 INFO [2022-01-10 08:39:17,817] jepsen worker 969 - jepsen.util 969 :ok :add 1955 INFO [2022-01-10 08:39:17,818] jepsen worker 386 - jepsen.util 386 :invoke :add 1957 INFO [2022-01-10 08:39:17,822] jepsen worker 680 - jepsen.util 680 :invoke :add 1958 INFO [2022-01-10 08:39:17,824] jepsen worker 386 - jepsen.util 386 :ok :add 1957 INFO [2022-01-10 08:39:17,826] jepsen worker 513 - jepsen.util 513 :invoke :add 1959 INFO [2022-01-10 08:39:17,828] jepsen worker 918 - jepsen.util 918 :ok :add 1956 INFO [2022-01-10 08:39:17,829] jepsen worker 680 - jepsen.util 680 :ok :add 1958 INFO [2022-01-10 08:39:17,830] jepsen worker 51 - jepsen.util 51 :invoke :add 1960 INFO [2022-01-10 08:39:17,832] jepsen worker 513 - jepsen.util 513 :ok :add 1959 INFO [2022-01-10 08:39:17,835] jepsen worker 736 - jepsen.util 736 :invoke :add 1961 INFO [2022-01-10 08:39:17,839] jepsen worker 51 - jepsen.util 51 :ok :add 1960 INFO [2022-01-10 08:39:17,839] jepsen worker 8 - jepsen.util 8 :invoke :add 1962 INFO [2022-01-10 08:39:17,841] jepsen worker 736 - jepsen.util 736 :ok :add 1961 INFO [2022-01-10 08:39:17,844] jepsen worker 766 - jepsen.util 766 :invoke :add 1963 INFO [2022-01-10 08:39:17,845] jepsen worker 8 - jepsen.util 8 :ok :add 1962 INFO [2022-01-10 08:39:17,848] jepsen worker 668 - jepsen.util 668 :invoke :add 1964 INFO [2022-01-10 08:39:17,853] jepsen worker 320 - jepsen.util 320 :invoke :add 1965 INFO [2022-01-10 08:39:17,855] jepsen worker 766 - jepsen.util 766 :ok :add 1963 INFO [2022-01-10 08:39:17,857] jepsen worker 124 - jepsen.util 124 :invoke :add 1966 INFO [2022-01-10 08:39:17,861] jepsen worker 559 - jepsen.util 559 :invoke :add 1967 INFO [2022-01-10 08:39:17,865] jepsen worker 948 - jepsen.util 948 :invoke :add 1968 INFO [2022-01-10 08:39:17,867] jepsen worker 577 - jepsen.util 577 :invoke :add 1969 INFO [2022-01-10 08:39:17,871] jepsen worker 971 - jepsen.util 971 :invoke :add 1970 INFO [2022-01-10 08:39:17,874] jepsen worker 577 - jepsen.util 577 :ok :add 1969 INFO [2022-01-10 08:39:17,874] jepsen worker 569 - jepsen.util 569 :invoke :add 1971 INFO [2022-01-10 08:39:17,877] jepsen worker 363 - jepsen.util 363 :invoke :add 1972 INFO [2022-01-10 08:39:17,878] jepsen worker 948 - jepsen.util 948 :ok :add 1968 INFO [2022-01-10 08:39:17,879] jepsen worker 282 - jepsen.util 282 :invoke :add 1973 INFO [2022-01-10 08:39:17,880] jepsen worker 971 - jepsen.util 971 :ok :add 1970 INFO [2022-01-10 08:39:17,882] jepsen worker 275 - jepsen.util 275 :invoke :add 1974 INFO [2022-01-10 08:39:17,885] jepsen worker 569 - jepsen.util 569 :ok :add 1971 INFO [2022-01-10 08:39:17,885] jepsen worker 633 - jepsen.util 633 :invoke :add 1975 INFO [2022-01-10 08:39:17,887] jepsen worker 363 - jepsen.util 363 :ok :add 1972 INFO [2022-01-10 08:39:17,888] jepsen worker 698 - jepsen.util 698 :invoke :add 1976 INFO [2022-01-10 08:39:17,891] jepsen worker 939 - jepsen.util 939 :invoke :add 1977 INFO [2022-01-10 08:39:17,893] jepsen worker 577 - jepsen.util 577 :invoke :add 1978 INFO [2022-01-10 08:39:17,894] jepsen worker 282 - jepsen.util 282 :ok :add 1973 INFO [2022-01-10 08:39:17,895] jepsen worker 275 - jepsen.util 275 :ok :add 1974 INFO [2022-01-10 08:39:17,896] jepsen worker 934 - jepsen.util 934 :invoke :add 1979 INFO [2022-01-10 08:39:17,900] jepsen worker 569 - jepsen.util 569 :invoke :add 1980 INFO [2022-01-10 08:39:17,903] jepsen worker 698 - jepsen.util 698 :ok :add 1976 INFO [2022-01-10 08:39:17,904] jepsen worker 577 - jepsen.util 577 :ok :add 1978 INFO [2022-01-10 08:39:17,904] jepsen worker 92 - jepsen.util 92 :ok :add 1939 INFO [2022-01-10 08:39:17,904] jepsen worker 150 - jepsen.util 150 :invoke :add 1981 INFO [2022-01-10 08:39:17,904] jepsen worker 934 - jepsen.util 934 :ok :add 1979 INFO [2022-01-10 08:39:17,908] jepsen worker 925 - jepsen.util 925 :invoke :add 1982 INFO [2022-01-10 08:39:17,910] jepsen worker 569 - jepsen.util 569 :ok :add 1980 INFO [2022-01-10 08:39:17,911] jepsen worker 423 - jepsen.util 423 :invoke :add 1983 INFO [2022-01-10 08:39:17,913] jepsen worker 633 - jepsen.util 633 :ok :add 1975 INFO [2022-01-10 08:39:17,914] jepsen worker 354 - jepsen.util 354 :invoke :add 1984 INFO [2022-01-10 08:39:17,915] jepsen worker 150 - jepsen.util 150 :ok :add 1981 INFO [2022-01-10 08:39:17,917] jepsen worker 657 - jepsen.util 657 :invoke :add 1985 INFO [2022-01-10 08:39:17,919] jepsen worker 939 - jepsen.util 939 :ok :add 1977 INFO [2022-01-10 08:39:17,920] jepsen worker 423 - jepsen.util 423 :ok :add 1983 INFO [2022-01-10 08:39:17,920] jepsen worker 213 - jepsen.util 213 :invoke :add 1986 INFO [2022-01-10 08:39:17,925] jepsen worker 754 - jepsen.util 754 :invoke :add 1987 INFO [2022-01-10 08:39:17,926] jepsen worker 925 - jepsen.util 925 :ok :add 1982 INFO [2022-01-10 08:39:17,927] jepsen worker 646 - jepsen.util 646 :invoke :add 1988 INFO [2022-01-10 08:39:17,931] jepsen worker 640 - jepsen.util 640 :invoke :add 1989 INFO [2022-01-10 08:39:17,934] jepsen worker 61 - jepsen.util 61 :invoke :add 1990 INFO [2022-01-10 08:39:17,937] jepsen worker 315 - jepsen.util 315 :invoke :add 1991 INFO [2022-01-10 08:39:17,940] jepsen worker 397 - jepsen.util 397 :invoke :add 1992 INFO [2022-01-10 08:39:17,942] jepsen worker 806 - jepsen.util 806 :invoke :add 1993 INFO [2022-01-10 08:39:17,946] jepsen worker 497 - jepsen.util 497 :invoke :add 1994 INFO [2022-01-10 08:39:17,948] jepsen worker 433 - jepsen.util 433 :invoke :add 1995 INFO [2022-01-10 08:39:17,950] jepsen worker 37 - jepsen.util 37 :invoke :add 1996 INFO [2022-01-10 08:39:17,952] jepsen worker 45 - jepsen.util 45 :invoke :add 1997 INFO [2022-01-10 08:39:17,953] jepsen worker 668 - jepsen.util 668 :ok :add 1964 INFO [2022-01-10 08:39:17,953] jepsen worker 279 - jepsen.util 279 :ok :add 1950 INFO [2022-01-10 08:39:17,953] jepsen worker 729 - jepsen.util 729 :ok :add 1948 INFO [2022-01-10 08:39:17,955] jepsen worker 497 - jepsen.util 497 :ok :add 1994 INFO [2022-01-10 08:39:17,956] jepsen worker 935 - jepsen.util 935 :ok :add 1949 INFO [2022-01-10 08:39:17,956] jepsen worker 433 - jepsen.util 433 :ok :add 1995 INFO [2022-01-10 08:39:17,957] jepsen worker 564 - jepsen.util 564 :invoke :add 1998 INFO [2022-01-10 08:39:17,957] jepsen worker 320 - jepsen.util 320 :ok :add 1965 INFO [2022-01-10 08:39:17,958] jepsen worker 466 - jepsen.util 466 :ok :add 1951 INFO [2022-01-10 08:39:17,959] jepsen worker 646 - jepsen.util 646 :ok :add 1988 INFO [2022-01-10 08:39:17,959] jepsen worker 122 - jepsen.util 122 :ok :add 1952 INFO [2022-01-10 08:39:17,959] jepsen worker 640 - jepsen.util 640 :ok :add 1989 INFO [2022-01-10 08:39:17,960] jepsen worker 124 - jepsen.util 124 :ok :add 1966 INFO [2022-01-10 08:39:17,961] jepsen worker 574 - jepsen.util 574 :invoke :add 1999 INFO [2022-01-10 08:39:17,964] jepsen worker 315 - jepsen.util 315 :ok :add 1991 INFO [2022-01-10 08:39:17,964] jepsen worker 925 - jepsen.util 925 :invoke :add 2000 INFO [2022-01-10 08:39:17,964] jepsen worker 37 - jepsen.util 37 :ok :add 1996 INFO [2022-01-10 08:39:17,965] jepsen worker 45 - jepsen.util 45 :ok :add 1997 INFO [2022-01-10 08:39:17,966] jepsen worker 61 - jepsen.util 61 :ok :add 1990 INFO [2022-01-10 08:39:17,968] jepsen worker 906 - jepsen.util 906 :invoke :add 2001 INFO [2022-01-10 08:39:17,970] jepsen worker 574 - jepsen.util 574 :ok :add 1999 INFO [2022-01-10 08:39:17,970] jepsen worker 397 - jepsen.util 397 :ok :add 1992 INFO [2022-01-10 08:39:17,970] jepsen worker 806 - jepsen.util 806 :ok :add 1993 INFO [2022-01-10 08:39:17,971] jepsen worker 451 - jepsen.util 451 :invoke :add 2002 INFO [2022-01-10 08:39:17,974] jepsen worker 564 - jepsen.util 564 :ok :add 1998 INFO [2022-01-10 08:39:17,974] jepsen worker 140 - jepsen.util 140 :invoke :add 2003 INFO [2022-01-10 08:39:17,977] jepsen worker 31 - jepsen.util 31 :invoke :add 2004 INFO [2022-01-10 08:39:17,978] jepsen worker 906 - jepsen.util 906 :ok :add 2001 INFO [2022-01-10 08:39:17,979] jepsen worker 742 - jepsen.util 742 :invoke :add 2005 INFO [2022-01-10 08:39:17,981] jepsen worker 140 - jepsen.util 140 :ok :add 2003 INFO [2022-01-10 08:39:17,982] jepsen worker 230 - jepsen.util 230 :invoke :add 2006 INFO [2022-01-10 08:39:17,986] jepsen worker 450 - jepsen.util 450 :invoke :add 2007 INFO [2022-01-10 08:39:17,987] jepsen worker 925 - jepsen.util 925 :ok :add 2000 INFO [2022-01-10 08:39:17,989] jepsen worker 967 - jepsen.util 967 :invoke :add 2008 INFO [2022-01-10 08:39:17,992] jepsen worker 730 - jepsen.util 730 :invoke :add 2009 INFO [2022-01-10 08:39:17,995] jepsen worker 325 - jepsen.util 325 :invoke :add 2010 INFO [2022-01-10 08:39:17,996] jepsen worker 451 - jepsen.util 451 :ok :add 2002 INFO [2022-01-10 08:39:17,998] jepsen worker 259 - jepsen.util 259 :ok :add 1953 INFO [2022-01-10 08:39:17,998] jepsen worker 559 - jepsen.util 559 :ok :add 1967 INFO [2022-01-10 08:39:17,998] jepsen worker 305 - jepsen.util 305 :invoke :add 2011 INFO [2022-01-10 08:39:18,003] jepsen worker 850 - jepsen.util 850 :invoke :add 2012 INFO [2022-01-10 08:39:18,006] jepsen worker 837 - jepsen.util 837 :invoke :add 2013 INFO [2022-01-10 08:39:18,007] jepsen worker 354 - jepsen.util 354 :ok :add 1984 INFO [2022-01-10 08:39:18,009] jepsen worker 768 - jepsen.util 768 :invoke :add 2014 INFO [2022-01-10 08:39:18,011] jepsen worker 213 - jepsen.util 213 :ok :add 1986 INFO [2022-01-10 08:39:18,012] jepsen worker 56 - jepsen.util 56 :invoke :add 2015 INFO [2022-01-10 08:39:18,015] jepsen worker 657 - jepsen.util 657 :ok :add 1985 INFO [2022-01-10 08:39:18,015] jepsen worker 768 - jepsen.util 768 :ok :add 2014 INFO [2022-01-10 08:39:18,016] jepsen worker 705 - jepsen.util 705 :invoke :add 2016 INFO [2022-01-10 08:39:18,019] jepsen worker 473 - jepsen.util 473 :invoke :add 2017 INFO [2022-01-10 08:39:18,022] jepsen worker 204 - jepsen.util 204 :invoke :add 2018 INFO [2022-01-10 08:39:18,026] jepsen worker 536 - jepsen.util 536 :invoke :add 2019 INFO [2022-01-10 08:39:18,029] jepsen worker 997 - jepsen.util 997 :invoke :add 2020 INFO [2022-01-10 08:39:18,032] jepsen worker 775 - jepsen.util 775 :invoke :add 2021 INFO [2022-01-10 08:39:18,035] jepsen worker 798 - jepsen.util 798 :invoke :add 2022 INFO [2022-01-10 08:39:18,037] jepsen worker 568 - jepsen.util 568 :invoke :add 2023 INFO [2022-01-10 08:39:18,043] jepsen worker 742 - jepsen.util 742 :ok :add 2005 INFO [2022-01-10 08:39:18,043] jepsen worker 903 - jepsen.util 903 :invoke :add 2024 INFO [2022-01-10 08:39:18,044] jepsen worker 31 - jepsen.util 31 :ok :add 2004 INFO [2022-01-10 08:39:18,047] jepsen worker 244 - jepsen.util 244 :invoke :add 2025 INFO [2022-01-10 08:39:18,047] jepsen worker 450 - jepsen.util 450 :ok :add 2007 INFO [2022-01-10 08:39:18,049] jepsen worker 230 - jepsen.util 230 :ok :add 2006 INFO [2022-01-10 08:39:18,049] jepsen worker 440 - jepsen.util 440 :invoke :add 2026 INFO [2022-01-10 08:39:18,052] jepsen worker 210 - jepsen.util 210 :invoke :add 2027 INFO [2022-01-10 08:39:18,055] jepsen worker 290 - jepsen.util 290 :invoke :add 2028 INFO [2022-01-10 08:39:18,057] jepsen worker 411 - jepsen.util 411 :invoke :add 2029 INFO [2022-01-10 08:39:18,058] jepsen worker 56 - jepsen.util 56 :ok :add 2015 INFO [2022-01-10 08:39:18,059] jepsen worker 502 - jepsen.util 502 :invoke :add 2030 INFO [2022-01-10 08:39:18,060] jepsen worker 204 - jepsen.util 204 :ok :add 2018 INFO [2022-01-10 08:39:18,060] jepsen worker 536 - jepsen.util 536 :ok :add 2019 INFO [2022-01-10 08:39:18,061] jepsen worker 754 - jepsen.util 754 :ok :add 1987 INFO [2022-01-10 08:39:18,062] jepsen worker 290 - jepsen.util 290 :ok :add 2028 INFO [2022-01-10 08:39:18,062] jepsen worker 442 - jepsen.util 442 :invoke :add 2031 INFO [2022-01-10 08:39:18,063] jepsen worker 798 - jepsen.util 798 :ok :add 2022 INFO [2022-01-10 08:39:18,063] jepsen worker 705 - jepsen.util 705 :ok :add 2016 INFO [2022-01-10 08:39:18,065] jepsen worker 473 - jepsen.util 473 :ok :add 2017 INFO [2022-01-10 08:39:18,065] jepsen worker 568 - jepsen.util 568 :ok :add 2023 INFO [2022-01-10 08:39:18,067] jepsen worker 996 - jepsen.util 996 :invoke :add 2032 INFO [2022-01-10 08:39:18,069] jepsen worker 997 - jepsen.util 997 :ok :add 2020 INFO [2022-01-10 08:39:18,071] jepsen worker 244 - jepsen.util 244 :ok :add 2025 INFO [2022-01-10 08:39:18,072] jepsen worker 305 - jepsen.util 305 :ok :add 2011 INFO [2022-01-10 08:39:18,072] jepsen worker 555 - jepsen.util 555 :invoke :add 2033 INFO [2022-01-10 08:39:18,073] jepsen worker 775 - jepsen.util 775 :ok :add 2021 INFO [2022-01-10 08:39:18,073] jepsen worker 730 - jepsen.util 730 :ok :add 2009 INFO [2022-01-10 08:39:18,074] jepsen worker 77 - jepsen.util 77 :invoke :add 2034 INFO [2022-01-10 08:39:18,079] jepsen worker 172 - jepsen.util 172 :invoke :add 2035 INFO [2022-01-10 08:39:18,083] jepsen worker 837 - jepsen.util 837 :ok :add 2013 INFO [2022-01-10 08:39:18,083] jepsen worker 210 - jepsen.util 210 :ok :add 2027 INFO [2022-01-10 08:39:18,083] jepsen worker 215 - jepsen.util 215 :invoke :add 2036 INFO [2022-01-10 08:39:18,087] jepsen worker 897 - jepsen.util 897 :invoke :add 2037 INFO [2022-01-10 08:39:18,087] jepsen worker 172 - jepsen.util 172 :ok :add 2035 INFO [2022-01-10 08:39:18,088] jepsen worker 411 - jepsen.util 411 :ok :add 2029 INFO [2022-01-10 08:39:18,089] jepsen worker 502 - jepsen.util 502 :ok :add 2030 INFO [2022-01-10 08:39:18,089] jepsen worker 493 - jepsen.util 493 :invoke :add 2038 INFO [2022-01-10 08:39:18,093] jepsen worker 137 - jepsen.util 137 :invoke :add 2039 INFO [2022-01-10 08:39:18,094] jepsen worker 555 - jepsen.util 555 :ok :add 2033 INFO [2022-01-10 08:39:18,094] jepsen worker 967 - jepsen.util 967 :ok :add 2008 INFO [2022-01-10 08:39:18,095] jepsen worker 325 - jepsen.util 325 :ok :add 2010 INFO [2022-01-10 08:39:18,095] jepsen worker 903 - jepsen.util 903 :ok :add 2024 INFO [2022-01-10 08:39:18,096] jepsen worker 238 - jepsen.util 238 :invoke :add 2040 INFO [2022-01-10 08:39:18,097] jepsen worker 897 - jepsen.util 897 :ok :add 2037 INFO [2022-01-10 08:39:18,097] jepsen worker 77 - jepsen.util 77 :ok :add 2034 INFO [2022-01-10 08:39:18,099] jepsen worker 476 - jepsen.util 476 :invoke :add 2041 INFO [2022-01-10 08:39:18,099] jepsen worker 493 - jepsen.util 493 :ok :add 2038 INFO [2022-01-10 08:39:18,103] jepsen worker 137 - jepsen.util 137 :ok :add 2039 INFO [2022-01-10 08:39:18,103] jepsen worker 440 - jepsen.util 440 :ok :add 2026 INFO [2022-01-10 08:39:18,103] jepsen worker 850 - jepsen.util 850 :ok :add 2012 INFO [2022-01-10 08:39:18,103] jepsen worker 270 - jepsen.util 270 :invoke :add 2042 INFO [2022-01-10 08:39:18,106] jepsen worker 447 - jepsen.util 447 :invoke :add 2043 INFO [2022-01-10 08:39:18,109] jepsen worker 16 - jepsen.util 16 :invoke :add 2044 INFO [2022-01-10 08:39:18,111] jepsen worker 691 - jepsen.util 691 :invoke :add 2045 INFO [2022-01-10 08:39:18,112] jepsen worker 215 - jepsen.util 215 :ok :add 2036 INFO [2022-01-10 08:39:18,115] jepsen worker 127 - jepsen.util 127 :invoke :add 2046 INFO [2022-01-10 08:39:18,116] jepsen worker 16 - jepsen.util 16 :ok :add 2044 INFO [2022-01-10 08:39:18,117] jepsen worker 479 - jepsen.util 479 :invoke :add 2047 INFO [2022-01-10 08:39:18,120] jepsen worker 700 - jepsen.util 700 :invoke :add 2048 INFO [2022-01-10 08:39:18,123] jepsen worker 419 - jepsen.util 419 :invoke :add 2049 INFO [2022-01-10 08:39:18,125] jepsen worker 996 - jepsen.util 996 :ok :add 2032 INFO [2022-01-10 08:39:18,126] jepsen worker 59 - jepsen.util 59 :invoke :add 2050 INFO [2022-01-10 08:39:18,126] jepsen worker 127 - jepsen.util 127 :ok :add 2046 INFO [2022-01-10 08:39:18,128] jepsen worker 763 - jepsen.util 763 :invoke :add 2051 INFO [2022-01-10 08:39:18,129] jepsen worker 700 - jepsen.util 700 :ok :add 2048 INFO [2022-01-10 08:39:18,131] jepsen worker 624 - jepsen.util 624 :invoke :add 2052 INFO [2022-01-10 08:39:18,133] jepsen worker 419 - jepsen.util 419 :ok :add 2049 INFO [2022-01-10 08:39:18,133] jepsen worker 691 - jepsen.util 691 :ok :add 2045 INFO [2022-01-10 08:39:18,135] jepsen worker 59 - jepsen.util 59 :ok :add 2050 INFO [2022-01-10 08:39:18,136] jepsen worker 125 - jepsen.util 125 :invoke :add 2053 INFO [2022-01-10 08:39:18,138] jepsen worker 763 - jepsen.util 763 :ok :add 2051 INFO [2022-01-10 08:39:18,139] jepsen worker 479 - jepsen.util 479 :ok :add 2047 INFO [2022-01-10 08:39:18,139] jepsen worker 948 - jepsen.util 948 :invoke :add 2054 INFO [2022-01-10 08:39:18,140] jepsen worker 624 - jepsen.util 624 :ok :add 2052 INFO [2022-01-10 08:39:18,144] jepsen worker 177 - jepsen.util 177 :invoke :add 2055 INFO [2022-01-10 08:39:18,146] jepsen worker 125 - jepsen.util 125 :ok :add 2053 INFO [2022-01-10 08:39:18,147] jepsen worker 486 - jepsen.util 486 :invoke :add 2056 INFO [2022-01-10 08:39:18,148] jepsen worker 948 - jepsen.util 948 :ok :add 2054 INFO [2022-01-10 08:39:18,150] jepsen worker 276 - jepsen.util 276 :invoke :add 2057 INFO [2022-01-10 08:39:18,153] jepsen worker 744 - jepsen.util 744 :invoke :add 2058 INFO [2022-01-10 08:39:18,154] jepsen worker 442 - jepsen.util 442 :ok :add 2031 INFO [2022-01-10 08:39:18,155] jepsen worker 486 - jepsen.util 486 :ok :add 2056 INFO [2022-01-10 08:39:18,155] jepsen worker 177 - jepsen.util 177 :ok :add 2055 INFO [2022-01-10 08:39:18,156] jepsen worker 754 - jepsen.util 754 :invoke :add 2059 INFO [2022-01-10 08:39:18,158] jepsen worker 476 - jepsen.util 476 :ok :add 2041 INFO [2022-01-10 08:39:18,161] jepsen worker 276 - jepsen.util 276 :ok :add 2057 INFO [2022-01-10 08:39:18,161] jepsen worker 451 - jepsen.util 451 :invoke :add 2060 INFO [2022-01-10 08:39:18,163] jepsen worker 447 - jepsen.util 447 :ok :add 2043 INFO [2022-01-10 08:39:18,165] jepsen worker 363 - jepsen.util 363 :invoke :add 2061 INFO [2022-01-10 08:39:18,168] jepsen worker 451 - jepsen.util 451 :ok :add 2060 INFO [2022-01-10 08:39:18,169] jepsen worker 960 - jepsen.util 960 :invoke :add 2062 INFO [2022-01-10 08:39:18,173] jepsen worker 932 - jepsen.util 932 :invoke :add 2063 INFO [2022-01-10 08:39:18,175] jepsen worker 363 - jepsen.util 363 :ok :add 2061 INFO [2022-01-10 08:39:18,176] jepsen worker 66 - jepsen.util 66 :invoke :add 2064 INFO [2022-01-10 08:39:18,178] jepsen worker 960 - jepsen.util 960 :ok :add 2062 INFO [2022-01-10 08:39:18,178] jepsen worker 454 - jepsen.util 454 :invoke :add 2065 INFO [2022-01-10 08:39:18,181] jepsen worker 922 - jepsen.util 922 :invoke :add 2066 INFO [2022-01-10 08:39:18,184] jepsen worker 66 - jepsen.util 66 :ok :add 2064 INFO [2022-01-10 08:39:18,184] jepsen worker 638 - jepsen.util 638 :invoke :add 2067 INFO [2022-01-10 08:39:18,186] jepsen worker 932 - jepsen.util 932 :ok :add 2063 INFO [2022-01-10 08:39:18,187] jepsen worker 238 - jepsen.util 238 :ok :add 2040 INFO [2022-01-10 08:39:18,187] jepsen worker 744 - jepsen.util 744 :ok :add 2058 INFO [2022-01-10 08:39:18,187] jepsen worker 454 - jepsen.util 454 :ok :add 2065 INFO [2022-01-10 08:39:18,189] jepsen worker 498 - jepsen.util 498 :invoke :add 2068 INFO [2022-01-10 08:39:18,192] jepsen worker 534 - jepsen.util 534 :invoke :add 2069 INFO [2022-01-10 08:39:18,192] jepsen worker 270 - jepsen.util 270 :ok :add 2042 INFO [2022-01-10 08:39:18,193] jepsen worker 922 - jepsen.util 922 :ok :add 2066 INFO [2022-01-10 08:39:18,196] jepsen worker 638 - jepsen.util 638 :ok :add 2067 INFO [2022-01-10 08:39:18,197] jepsen worker 942 - jepsen.util 942 :invoke :add 2070 INFO [2022-01-10 08:39:18,200] jepsen worker 618 - jepsen.util 618 :invoke :add 2071 INFO [2022-01-10 08:39:18,203] jepsen worker 738 - jepsen.util 738 :invoke :add 2072 INFO [2022-01-10 08:39:18,205] jepsen worker 450 - jepsen.util 450 :invoke :add 2073 INFO [2022-01-10 08:39:18,205] jepsen worker 754 - jepsen.util 754 :ok :add 2059 INFO [2022-01-10 08:39:18,208] jepsen worker 497 - jepsen.util 497 :invoke :add 2074 INFO [2022-01-10 08:39:18,212] jepsen worker 686 - jepsen.util 686 :invoke :add 2075 INFO [2022-01-10 08:39:18,215] jepsen worker 358 - jepsen.util 358 :invoke :add 2076 INFO [2022-01-10 08:39:18,217] jepsen worker 498 - jepsen.util 498 :ok :add 2068 INFO [2022-01-10 08:39:18,218] jepsen worker 534 - jepsen.util 534 :ok :add 2069 INFO [2022-01-10 08:39:18,218] jepsen worker 484 - jepsen.util 484 :invoke :add 2077 INFO [2022-01-10 08:39:18,222] jepsen worker 867 - jepsen.util 867 :invoke :add 2078 INFO [2022-01-10 08:39:18,223] jepsen worker 618 - jepsen.util 618 :ok :add 2071 INFO [2022-01-10 08:39:18,224] jepsen worker 942 - jepsen.util 942 :ok :add 2070 INFO [2022-01-10 08:39:18,225] jepsen worker 713 - jepsen.util 713 :invoke :add 2079 INFO [2022-01-10 08:39:18,228] jepsen worker 450 - jepsen.util 450 :ok :add 2073 INFO [2022-01-10 08:39:18,230] jepsen worker 738 - jepsen.util 738 :ok :add 2072 INFO [2022-01-10 08:39:18,230] jepsen worker 280 - jepsen.util 280 :invoke :add 2080 INFO [2022-01-10 08:39:18,231] jepsen worker 867 - jepsen.util 867 :ok :add 2078 INFO [2022-01-10 08:39:18,234] jepsen worker 972 - jepsen.util 972 :invoke :add 2081 INFO [2022-01-10 08:39:18,236] jepsen worker 713 - jepsen.util 713 :ok :add 2079 INFO [2022-01-10 08:39:18,237] jepsen worker 394 - jepsen.util 394 :invoke :add 2082 INFO [2022-01-10 08:39:18,241] jepsen worker 400 - jepsen.util 400 :invoke :add 2083 INFO [2022-01-10 08:39:18,244] jepsen worker 15 - jepsen.util 15 :invoke :add 2084 INFO [2022-01-10 08:39:18,245] jepsen worker 280 - jepsen.util 280 :ok :add 2080 INFO [2022-01-10 08:39:18,247] jepsen worker 419 - jepsen.util 419 :invoke :add 2085 INFO [2022-01-10 08:39:18,249] jepsen worker 686 - jepsen.util 686 :ok :add 2075 INFO [2022-01-10 08:39:18,249] jepsen worker 358 - jepsen.util 358 :ok :add 2076 INFO [2022-01-10 08:39:18,249] jepsen worker 972 - jepsen.util 972 :ok :add 2081 INFO [2022-01-10 08:39:18,250] jepsen worker 799 - jepsen.util 799 :invoke :add 2086 INFO [2022-01-10 08:39:18,254] jepsen worker 15 - jepsen.util 15 :ok :add 2084 INFO [2022-01-10 08:39:18,254] jepsen worker 158 - jepsen.util 158 :invoke :add 2087 INFO [2022-01-10 08:39:18,256] jepsen worker 400 - jepsen.util 400 :ok :add 2083 INFO [2022-01-10 08:39:18,257] jepsen worker 394 - jepsen.util 394 :ok :add 2082 INFO [2022-01-10 08:39:18,257] jepsen worker 237 - jepsen.util 237 :invoke :add 2088 INFO [2022-01-10 08:39:18,258] jepsen worker 484 - jepsen.util 484 :ok :add 2077 INFO [2022-01-10 08:39:18,259] jepsen worker 497 - jepsen.util 497 :ok :add 2074 INFO [2022-01-10 08:39:18,260] jepsen worker 799 - jepsen.util 799 :ok :add 2086 INFO [2022-01-10 08:39:18,261] jepsen worker 419 - jepsen.util 419 :ok :add 2085 INFO [2022-01-10 08:39:18,262] jepsen worker 553 - jepsen.util 553 :invoke :add 2089 INFO [2022-01-10 08:39:18,265] jepsen worker 158 - jepsen.util 158 :ok :add 2087 INFO [2022-01-10 08:39:18,266] jepsen worker 839 - jepsen.util 839 :invoke :add 2090 INFO [2022-01-10 08:39:18,271] jepsen worker 650 - jepsen.util 650 :invoke :add 2091 INFO [2022-01-10 08:39:18,274] jepsen worker 17 - jepsen.util 17 :invoke :add 2092 INFO [2022-01-10 08:39:18,276] jepsen worker 553 - jepsen.util 553 :ok :add 2089 INFO [2022-01-10 08:39:18,277] jepsen worker 750 - jepsen.util 750 :invoke :add 2093 INFO [2022-01-10 08:39:18,280] jepsen worker 460 - jepsen.util 460 :invoke :add 2094 INFO [2022-01-10 08:39:18,283] jepsen worker 890 - jepsen.util 890 :invoke :add 2095 INFO [2022-01-10 08:39:18,285] jepsen worker 650 - jepsen.util 650 :ok :add 2091 INFO [2022-01-10 08:39:18,286] jepsen worker 494 - jepsen.util 494 :invoke :add 2096 INFO [2022-01-10 08:39:18,287] jepsen worker 839 - jepsen.util 839 :ok :add 2090 INFO [2022-01-10 08:39:18,287] jepsen worker 237 - jepsen.util 237 :ok :add 2088 INFO [2022-01-10 08:39:18,289] jepsen worker 750 - jepsen.util 750 :ok :add 2093 INFO [2022-01-10 08:39:18,289] jepsen worker 460 - jepsen.util 460 :ok :add 2094 INFO [2022-01-10 08:39:18,290] jepsen worker 836 - jepsen.util 836 :invoke :add 2097 INFO [2022-01-10 08:39:18,292] jepsen worker 890 - jepsen.util 890 :ok :add 2095 INFO [2022-01-10 08:39:18,293] jepsen worker 17 - jepsen.util 17 :ok :add 2092 INFO [2022-01-10 08:39:18,294] jepsen worker 918 - jepsen.util 918 :invoke :add 2098 INFO [2022-01-10 08:39:18,297] jepsen worker 437 - jepsen.util 437 :invoke :add 2099 INFO [2022-01-10 08:39:18,300] jepsen worker 494 - jepsen.util 494 :ok :add 2096 INFO [2022-01-10 08:39:18,301] jepsen worker 7 - jepsen.util 7 :invoke :add 2100 INFO [2022-01-10 08:39:18,303] jepsen worker 836 - jepsen.util 836 :ok :add 2097 INFO [2022-01-10 08:39:18,303] jepsen worker 918 - jepsen.util 918 :ok :add 2098 INFO [2022-01-10 08:39:18,304] jepsen worker 871 - jepsen.util 871 :invoke :add 2101 INFO [2022-01-10 08:39:18,306] jepsen worker 437 - jepsen.util 437 :ok :add 2099 INFO [2022-01-10 08:39:18,308] jepsen worker 68 - jepsen.util 68 :invoke :add 2102 INFO [2022-01-10 08:39:18,311] jepsen worker 96 - jepsen.util 96 :invoke :add 2103 INFO [2022-01-10 08:39:18,313] jepsen worker 7 - jepsen.util 7 :ok :add 2100 INFO [2022-01-10 08:39:18,313] jepsen worker 871 - jepsen.util 871 :ok :add 2101 INFO [2022-01-10 08:39:18,315] jepsen worker 165 - jepsen.util 165 :invoke :add 2104 INFO [2022-01-10 08:39:18,317] jepsen worker 33 - jepsen.util 33 :invoke :add 2105 INFO [2022-01-10 08:39:18,321] jepsen worker 68 - jepsen.util 68 :ok :add 2102 INFO [2022-01-10 08:39:18,322] jepsen worker 96 - jepsen.util 96 :ok :add 2103 INFO [2022-01-10 08:39:18,323] jepsen worker 290 - jepsen.util 290 :invoke :add 2106 INFO [2022-01-10 08:39:18,324] jepsen worker 165 - jepsen.util 165 :ok :add 2104 INFO [2022-01-10 08:39:18,327] jepsen worker 33 - jepsen.util 33 :ok :add 2105 INFO [2022-01-10 08:39:18,328] jepsen worker 439 - jepsen.util 439 :invoke :add 2107 INFO [2022-01-10 08:39:18,332] jepsen worker 542 - jepsen.util 542 :invoke :add 2108 INFO [2022-01-10 08:39:18,334] jepsen worker 290 - jepsen.util 290 :ok :add 2106 INFO [2022-01-10 08:39:18,335] jepsen worker 733 - jepsen.util 733 :invoke :add 2109 INFO [2022-01-10 08:39:18,338] jepsen worker 549 - jepsen.util 549 :invoke :add 2110 INFO [2022-01-10 08:39:18,340] jepsen worker 542 - jepsen.util 542 :ok :add 2108 INFO [2022-01-10 08:39:18,341] jepsen worker 439 - jepsen.util 439 :ok :add 2107 INFO [2022-01-10 08:39:18,342] jepsen worker 885 - jepsen.util 885 :invoke :add 2111 INFO [2022-01-10 08:39:18,345] jepsen worker 733 - jepsen.util 733 :ok :add 2109 INFO [2022-01-10 08:39:18,345] jepsen worker 687 - jepsen.util 687 :invoke :add 2112 INFO [2022-01-10 08:39:18,349] jepsen worker 882 - jepsen.util 882 :invoke :add 2113 INFO [2022-01-10 08:39:18,350] jepsen worker 885 - jepsen.util 885 :ok :add 2111 INFO [2022-01-10 08:39:18,351] jepsen worker 400 - jepsen.util 400 :invoke :add 2114 INFO [2022-01-10 08:39:18,354] jepsen worker 710 - jepsen.util 710 :invoke :add 2115 INFO [2022-01-10 08:39:18,355] jepsen worker 549 - jepsen.util 549 :ok :add 2110 INFO [2022-01-10 08:39:18,358] jepsen worker 263 - jepsen.util 263 :invoke :add 2116 INFO [2022-01-10 08:39:18,360] jepsen worker 882 - jepsen.util 882 :ok :add 2113 INFO [2022-01-10 08:39:18,361] jepsen worker 400 - jepsen.util 400 :ok :add 2114 INFO [2022-01-10 08:39:18,361] jepsen worker 687 - jepsen.util 687 :ok :add 2112 INFO [2022-01-10 08:39:18,362] jepsen worker 535 - jepsen.util 535 :invoke :add 2117 INFO [2022-01-10 08:39:18,364] jepsen worker 414 - jepsen.util 414 :invoke :add 2118 INFO [2022-01-10 08:39:18,368] jepsen worker 574 - jepsen.util 574 :invoke :add 2119 INFO [2022-01-10 08:39:18,368] jepsen worker 263 - jepsen.util 263 :ok :add 2116 INFO [2022-01-10 08:39:18,371] jepsen worker 710 - jepsen.util 710 :ok :add 2115 INFO [2022-01-10 08:39:18,372] jepsen worker 672 - jepsen.util 672 :invoke :add 2120 INFO [2022-01-10 08:39:18,375] jepsen worker 574 - jepsen.util 574 :ok :add 2119 INFO [2022-01-10 08:39:18,375] jepsen worker 681 - jepsen.util 681 :invoke :add 2121 INFO [2022-01-10 08:39:18,378] jepsen worker 674 - jepsen.util 674 :invoke :add 2122 INFO [2022-01-10 08:39:18,378] jepsen worker 414 - jepsen.util 414 :ok :add 2118 INFO [2022-01-10 08:39:18,380] jepsen worker 860 - jepsen.util 860 :invoke :add 2123 INFO [2022-01-10 08:39:18,383] jepsen worker 838 - jepsen.util 838 :invoke :add 2124 INFO [2022-01-10 08:39:18,384] jepsen worker 535 - jepsen.util 535 :ok :add 2117 INFO [2022-01-10 08:39:18,385] jepsen worker 402 - jepsen.util 402 :invoke :add 2125 INFO [2022-01-10 08:39:18,388] jepsen worker 672 - jepsen.util 672 :ok :add 2120 INFO [2022-01-10 08:39:18,389] jepsen worker 718 - jepsen.util 718 :invoke :add 2126 INFO [2022-01-10 08:39:18,394] jepsen worker 674 - jepsen.util 674 :ok :add 2122 INFO [2022-01-10 08:39:18,395] jepsen worker 402 - jepsen.util 402 :ok :add 2125 INFO [2022-01-10 08:39:18,395] jepsen worker 681 - jepsen.util 681 :ok :add 2121 INFO [2022-01-10 08:39:18,395] jepsen worker 838 - jepsen.util 838 :ok :add 2124 INFO [2022-01-10 08:39:18,397] jepsen worker 142 - jepsen.util 142 :invoke :add 2127 INFO [2022-01-10 08:39:18,400] jepsen worker 790 - jepsen.util 790 :invoke :add 2128 INFO [2022-01-10 08:39:18,400] jepsen worker 860 - jepsen.util 860 :ok :add 2123 INFO [2022-01-10 08:39:18,403] jepsen worker 632 - jepsen.util 632 :invoke :add 2129 INFO [2022-01-10 08:39:18,404] jepsen worker 142 - jepsen.util 142 :ok :add 2127 INFO [2022-01-10 08:39:18,404] jepsen worker 718 - jepsen.util 718 :ok :add 2126 INFO [2022-01-10 08:39:18,406] jepsen worker 936 - jepsen.util 936 :invoke :add 2130 INFO [2022-01-10 08:39:18,409] jepsen worker 698 - jepsen.util 698 :invoke :add 2131 INFO [2022-01-10 08:39:18,411] jepsen worker 618 - jepsen.util 618 :invoke :add 2132 INFO [2022-01-10 08:39:18,414] jepsen worker 453 - jepsen.util 453 :invoke :add 2133 INFO [2022-01-10 08:39:18,415] jepsen worker 790 - jepsen.util 790 :ok :add 2128 INFO [2022-01-10 08:39:18,416] jepsen worker 166 - jepsen.util 166 :invoke :add 2134 INFO [2022-01-10 08:39:18,416] jepsen worker 632 - jepsen.util 632 :ok :add 2129 INFO [2022-01-10 08:39:18,419] jepsen worker 607 - jepsen.util 607 :invoke :add 2135 INFO [2022-01-10 08:39:18,422] jepsen worker 154 - jepsen.util 154 :invoke :add 2136 INFO [2022-01-10 08:39:18,423] jepsen worker 936 - jepsen.util 936 :ok :add 2130 INFO [2022-01-10 08:39:18,423] jepsen worker 698 - jepsen.util 698 :ok :add 2131 INFO [2022-01-10 08:39:18,426] jepsen worker 681 - jepsen.util 681 :invoke :add 2137 INFO [2022-01-10 08:39:18,428] jepsen worker 166 - jepsen.util 166 :ok :add 2134 INFO [2022-01-10 08:39:18,428] jepsen worker 377 - jepsen.util 377 :invoke :add 2138 INFO [2022-01-10 08:39:18,429] jepsen worker 607 - jepsen.util 607 :ok :add 2135 INFO [2022-01-10 08:39:18,429] jepsen worker 618 - jepsen.util 618 :ok :add 2132 INFO [2022-01-10 08:39:18,429] jepsen worker 453 - jepsen.util 453 :ok :add 2133 INFO [2022-01-10 08:39:18,432] jepsen worker 554 - jepsen.util 554 :invoke :add 2139 INFO [2022-01-10 08:39:18,436] jepsen worker 14 - jepsen.util 14 :invoke :add 2140 INFO [2022-01-10 08:39:18,438] jepsen worker 154 - jepsen.util 154 :ok :add 2136 INFO [2022-01-10 08:39:18,438] jepsen worker 681 - jepsen.util 681 :ok :add 2137 INFO [2022-01-10 08:39:18,439] jepsen worker 130 - jepsen.util 130 :invoke :add 2141 INFO [2022-01-10 08:39:18,442] jepsen worker 706 - jepsen.util 706 :invoke :add 2142 INFO [2022-01-10 08:39:18,444] jepsen worker 101 - jepsen.util 101 :invoke :add 2143 INFO [2022-01-10 08:39:18,446] jepsen worker 489 - jepsen.util 489 :invoke :add 2144 INFO [2022-01-10 08:39:18,450] jepsen worker 19 - jepsen.util 19 :invoke :add 2145 INFO [2022-01-10 08:39:18,451] jepsen worker 377 - jepsen.util 377 :ok :add 2138 INFO [2022-01-10 08:39:18,453] jepsen worker 233 - jepsen.util 233 :invoke :add 2146 INFO [2022-01-10 08:39:18,456] jepsen worker 558 - jepsen.util 558 :invoke :add 2147 INFO [2022-01-10 08:39:18,457] jepsen worker 19 - jepsen.util 19 :ok :add 2145 INFO [2022-01-10 08:39:18,458] jepsen worker 489 - jepsen.util 489 :ok :add 2144 INFO [2022-01-10 08:39:18,459] jepsen worker 15 - jepsen.util 15 :invoke :add 2148 INFO [2022-01-10 08:39:18,460] jepsen worker 130 - jepsen.util 130 :ok :add 2141 INFO [2022-01-10 08:39:18,464] jepsen worker 233 - jepsen.util 233 :ok :add 2146 INFO [2022-01-10 08:39:18,464] jepsen worker 723 - jepsen.util 723 :invoke :add 2149 INFO [2022-01-10 08:39:18,466] jepsen worker 14 - jepsen.util 14 :ok :add 2140 INFO [2022-01-10 08:39:18,466] jepsen worker 101 - jepsen.util 101 :ok :add 2143 INFO [2022-01-10 08:39:18,468] jepsen worker 558 - jepsen.util 558 :ok :add 2147 INFO [2022-01-10 08:39:18,469] jepsen worker 896 - jepsen.util 896 :invoke :add 2150 INFO [2022-01-10 08:39:18,472] jepsen worker 554 - jepsen.util 554 :ok :add 2139 INFO [2022-01-10 08:39:18,474] jepsen worker 160 - jepsen.util 160 :invoke :add 2151 INFO [2022-01-10 08:39:18,474] jepsen worker 723 - jepsen.util 723 :ok :add 2149 INFO [2022-01-10 08:39:18,474] jepsen worker 706 - jepsen.util 706 :ok :add 2142 INFO [2022-01-10 08:39:18,478] jepsen worker 68 - jepsen.util 68 :invoke :add 2152 INFO [2022-01-10 08:39:18,481] jepsen worker 314 - jepsen.util 314 :invoke :add 2153 INFO [2022-01-10 08:39:18,484] jepsen worker 871 - jepsen.util 871 :invoke :add 2154 INFO [2022-01-10 08:39:18,487] jepsen worker 178 - jepsen.util 178 :invoke :add 2155 INFO [2022-01-10 08:39:18,490] jepsen worker 914 - jepsen.util 914 :invoke :add 2156 INFO [2022-01-10 08:39:18,490] jepsen worker 871 - jepsen.util 871 :ok :add 2154 INFO [2022-01-10 08:39:18,493] jepsen worker 15 - jepsen.util 15 :ok :add 2148 INFO [2022-01-10 08:39:18,493] jepsen worker 896 - jepsen.util 896 :ok :add 2150 INFO [2022-01-10 08:39:18,493] jepsen worker 178 - jepsen.util 178 :ok :add 2155 INFO [2022-01-10 08:39:18,493] jepsen worker 845 - jepsen.util 845 :invoke :add 2157 INFO [2022-01-10 08:39:18,494] jepsen worker 160 - jepsen.util 160 :ok :add 2151 INFO [2022-01-10 08:39:18,498] jepsen worker 314 - jepsen.util 314 :ok :add 2153 INFO [2022-01-10 08:39:18,498] jepsen worker 783 - jepsen.util 783 :invoke :add 2158 INFO [2022-01-10 08:39:18,500] jepsen worker 845 - jepsen.util 845 :ok :add 2157 INFO [2022-01-10 08:39:18,502] jepsen worker 314 - jepsen.util 314 :invoke :add 2159 INFO [2022-01-10 08:39:18,503] jepsen worker 68 - jepsen.util 68 :ok :add 2152 INFO [2022-01-10 08:39:18,504] jepsen worker 914 - jepsen.util 914 :ok :add 2156 INFO [2022-01-10 08:39:18,504] jepsen worker 642 - jepsen.util 642 :invoke :add 2160 INFO [2022-01-10 08:39:18,507] jepsen worker 38 - jepsen.util 38 :invoke :add 2161 INFO [2022-01-10 08:39:18,509] jepsen worker 728 - jepsen.util 728 :invoke :add 2162 INFO [2022-01-10 08:39:18,510] jepsen worker 783 - jepsen.util 783 :ok :add 2158 INFO [2022-01-10 08:39:18,512] jepsen worker 314 - jepsen.util 314 :ok :add 2159 INFO [2022-01-10 08:39:18,512] jepsen worker 22 - jepsen.util 22 :invoke :add 2163 INFO [2022-01-10 08:39:18,515] jepsen worker 38 - jepsen.util 38 :ok :add 2161 INFO [2022-01-10 08:39:18,515] jepsen worker 313 - jepsen.util 313 :invoke :add 2164 INFO [2022-01-10 08:39:18,518] jepsen worker 366 - jepsen.util 366 :invoke :add 2165 INFO [2022-01-10 08:39:18,518] jepsen worker 642 - jepsen.util 642 :ok :add 2160 INFO [2022-01-10 08:39:18,521] jepsen worker 22 - jepsen.util 22 :ok :add 2163 INFO [2022-01-10 08:39:18,521] jepsen worker 177 - jepsen.util 177 :invoke :add 2166 INFO [2022-01-10 08:39:18,523] jepsen worker 728 - jepsen.util 728 :ok :add 2162 INFO [2022-01-10 08:39:18,524] jepsen worker 693 - jepsen.util 693 :invoke :add 2167 INFO [2022-01-10 08:39:18,525] jepsen worker 313 - jepsen.util 313 :ok :add 2164 INFO [2022-01-10 08:39:18,527] jepsen worker 655 - jepsen.util 655 :invoke :add 2168 INFO [2022-01-10 08:39:18,528] jepsen worker 366 - jepsen.util 366 :ok :add 2165 INFO [2022-01-10 08:39:18,531] jepsen worker 149 - jepsen.util 149 :invoke :add 2169 INFO [2022-01-10 08:39:18,532] jepsen worker 177 - jepsen.util 177 :ok :add 2166 INFO [2022-01-10 08:39:18,533] jepsen worker 693 - jepsen.util 693 :ok :add 2167 INFO [2022-01-10 08:39:18,535] jepsen worker 184 - jepsen.util 184 :invoke :add 2170 INFO [2022-01-10 08:39:18,539] jepsen worker 328 - jepsen.util 328 :invoke :add 2171 INFO [2022-01-10 08:39:18,540] jepsen worker 149 - jepsen.util 149 :ok :add 2169 INFO [2022-01-10 08:39:18,542] jepsen worker 773 - jepsen.util 773 :invoke :add 2172 INFO [2022-01-10 08:39:18,544] jepsen worker 467 - jepsen.util 467 :invoke :add 2173 INFO [2022-01-10 08:39:18,547] jepsen worker 246 - jepsen.util 246 :invoke :add 2174 INFO [2022-01-10 08:39:18,549] jepsen worker 76 - jepsen.util 76 :invoke :add 2175 INFO [2022-01-10 08:39:18,552] jepsen worker 655 - jepsen.util 655 :ok :add 2168 INFO [2022-01-10 08:39:18,552] jepsen worker 328 - jepsen.util 328 :ok :add 2171 INFO [2022-01-10 08:39:18,552] jepsen worker 184 - jepsen.util 184 :ok :add 2170 INFO [2022-01-10 08:39:18,553] jepsen worker 25 - jepsen.util 25 :invoke :add 2176 INFO [2022-01-10 08:39:18,556] jepsen worker 773 - jepsen.util 773 :ok :add 2172 INFO [2022-01-10 08:39:18,556] jepsen worker 154 - jepsen.util 154 :invoke :add 2177 INFO [2022-01-10 08:39:18,557] jepsen worker 246 - jepsen.util 246 :ok :add 2174 INFO [2022-01-10 08:39:18,557] jepsen worker 467 - jepsen.util 467 :ok :add 2173 INFO [2022-01-10 08:39:18,557] jepsen worker 76 - jepsen.util 76 :ok :add 2175 INFO [2022-01-10 08:39:18,559] jepsen worker 856 - jepsen.util 856 :invoke :add 2178 INFO [2022-01-10 08:39:18,562] jepsen worker 584 - jepsen.util 584 :invoke :add 2179 INFO [2022-01-10 08:39:18,564] jepsen worker 990 - jepsen.util 990 :invoke :add 2180 INFO [2022-01-10 08:39:18,567] jepsen worker 458 - jepsen.util 458 :invoke :add 2181 INFO [2022-01-10 08:39:18,569] jepsen worker 749 - jepsen.util 749 :invoke :add 2182 INFO [2022-01-10 08:39:18,569] jepsen worker 856 - jepsen.util 856 :ok :add 2178 INFO [2022-01-10 08:39:18,569] jepsen worker 25 - jepsen.util 25 :ok :add 2176 INFO [2022-01-10 08:39:18,571] jepsen worker 693 - jepsen.util 693 :invoke :add 2183 INFO [2022-01-10 08:39:18,574] jepsen worker 231 - jepsen.util 231 :invoke :add 2184 INFO [2022-01-10 08:39:18,576] jepsen worker 154 - jepsen.util 154 :ok :add 2177 INFO [2022-01-10 08:39:18,576] jepsen worker 701 - jepsen.util 701 :invoke :add 2185 INFO [2022-01-10 08:39:18,578] jepsen worker 584 - jepsen.util 584 :ok :add 2179 INFO [2022-01-10 08:39:18,578] jepsen worker 48 - jepsen.util 48 :invoke :add 2186 INFO [2022-01-10 08:39:18,581] jepsen worker 261 - jepsen.util 261 :invoke :add 2187 INFO [2022-01-10 08:39:18,582] jepsen worker 231 - jepsen.util 231 :ok :add 2184 INFO [2022-01-10 08:39:18,583] jepsen worker 458 - jepsen.util 458 :ok :add 2181 INFO [2022-01-10 08:39:18,584] jepsen worker 940 - jepsen.util 940 :invoke :add 2188 INFO [2022-01-10 08:39:18,585] jepsen worker 990 - jepsen.util 990 :ok :add 2180 INFO [2022-01-10 08:39:18,587] jepsen worker 320 - jepsen.util 320 :invoke :add 2189 INFO [2022-01-10 08:39:18,589] jepsen worker 693 - jepsen.util 693 :ok :add 2183 INFO [2022-01-10 08:39:18,589] jepsen worker 591 - jepsen.util 591 :invoke :add 2190 INFO [2022-01-10 08:39:18,591] jepsen worker 749 - jepsen.util 749 :ok :add 2182 INFO [2022-01-10 08:39:18,592] jepsen worker 940 - jepsen.util 940 :ok :add 2188 INFO [2022-01-10 08:39:18,592] jepsen worker 907 - jepsen.util 907 :invoke :add 2191 INFO [2022-01-10 08:39:18,595] jepsen worker 454 - jepsen.util 454 :invoke :add 2192 INFO [2022-01-10 08:39:18,596] jepsen worker 320 - jepsen.util 320 :ok :add 2189 INFO [2022-01-10 08:39:18,597] jepsen worker 689 - jepsen.util 689 :invoke :add 2193 INFO [2022-01-10 08:39:18,600] jepsen worker 687 - jepsen.util 687 :invoke :add 2194 INFO [2022-01-10 08:39:18,601] jepsen worker 907 - jepsen.util 907 :ok :add 2191 INFO [2022-01-10 08:39:18,602] jepsen worker 857 - jepsen.util 857 :invoke :add 2195 INFO [2022-01-10 08:39:18,603] jepsen worker 701 - jepsen.util 701 :ok :add 2185 INFO [2022-01-10 08:39:18,603] jepsen worker 591 - jepsen.util 591 :ok :add 2190 INFO [2022-01-10 08:39:18,605] jepsen worker 949 - jepsen.util 949 :invoke :add 2196 INFO [2022-01-10 08:39:18,606] jepsen worker 48 - jepsen.util 48 :ok :add 2186 INFO [2022-01-10 08:39:18,608] jepsen worker 261 - jepsen.util 261 :ok :add 2187 INFO [2022-01-10 08:39:18,609] jepsen worker 760 - jepsen.util 760 :invoke :add 2197 INFO [2022-01-10 08:39:18,609] jepsen worker 689 - jepsen.util 689 :ok :add 2193 INFO [2022-01-10 08:39:18,609] jepsen worker 454 - jepsen.util 454 :ok :add 2192 INFO [2022-01-10 08:39:18,611] jepsen worker 523 - jepsen.util 523 :invoke :add 2198 INFO [2022-01-10 08:39:18,614] jepsen worker 52 - jepsen.util 52 :invoke :add 2199 INFO [2022-01-10 08:39:18,615] jepsen worker 687 - jepsen.util 687 :ok :add 2194 INFO [2022-01-10 08:39:18,617] jepsen worker 257 - jepsen.util 257 :invoke :add 2200 INFO [2022-01-10 08:39:18,620] jepsen worker 350 - jepsen.util 350 :invoke :add 2201 INFO [2022-01-10 08:39:18,620] jepsen worker 949 - jepsen.util 949 :ok :add 2196 INFO [2022-01-10 08:39:18,622] jepsen worker 52 - jepsen.util 52 :ok :add 2199 INFO [2022-01-10 08:39:18,622] jepsen worker 502 - jepsen.util 502 :invoke :add 2202 INFO [2022-01-10 08:39:18,625] jepsen worker 275 - jepsen.util 275 :invoke :add 2203 INFO [2022-01-10 08:39:18,628] jepsen worker 350 - jepsen.util 350 :ok :add 2201 INFO [2022-01-10 08:39:18,630] jepsen worker 899 - jepsen.util 899 :invoke :add 2204 INFO [2022-01-10 08:39:18,632] jepsen worker 257 - jepsen.util 257 :ok :add 2200 INFO [2022-01-10 08:39:18,634] jepsen worker 246 - jepsen.util 246 :invoke :add 2205 INFO [2022-01-10 08:39:18,634] jepsen worker 275 - jepsen.util 275 :ok :add 2203 INFO [2022-01-10 08:39:18,637] jepsen worker 502 - jepsen.util 502 :ok :add 2202 INFO [2022-01-10 08:39:18,637] jepsen worker 213 - jepsen.util 213 :invoke :add 2206 INFO [2022-01-10 08:39:18,640] jepsen worker 523 - jepsen.util 523 :ok :add 2198 INFO [2022-01-10 08:39:18,641] jepsen worker 684 - jepsen.util 684 :invoke :add 2207 INFO [2022-01-10 08:39:18,644] jepsen worker 857 - jepsen.util 857 :ok :add 2195 INFO [2022-01-10 08:39:18,645] jepsen worker 526 - jepsen.util 526 :invoke :add 2208 INFO [2022-01-10 08:39:18,648] jepsen worker 180 - jepsen.util 180 :invoke :add 2209 INFO [2022-01-10 08:39:18,651] jepsen worker 760 - jepsen.util 760 :ok :add 2197 INFO [2022-01-10 08:39:18,651] jepsen worker 820 - jepsen.util 820 :invoke :add 2210 INFO [2022-01-10 08:39:18,654] jepsen worker 28 - jepsen.util 28 :invoke :add 2211 INFO [2022-01-10 08:39:18,655] jepsen worker 180 - jepsen.util 180 :ok :add 2209 INFO [2022-01-10 08:39:18,657] jepsen worker 570 - jepsen.util 570 :invoke :add 2212 INFO [2022-01-10 08:39:18,659] jepsen worker 49 - jepsen.util 49 :invoke :add 2213 INFO [2022-01-10 08:39:18,660] jepsen worker 246 - jepsen.util 246 :ok :add 2205 INFO [2022-01-10 08:39:18,661] jepsen worker 526 - jepsen.util 526 :ok :add 2208 INFO [2022-01-10 08:39:18,662] jepsen worker 820 - jepsen.util 820 :ok :add 2210 INFO [2022-01-10 08:39:18,663] jepsen worker 899 - jepsen.util 899 :ok :add 2204 INFO [2022-01-10 08:39:18,663] jepsen worker 852 - jepsen.util 852 :invoke :add 2214 INFO [2022-01-10 08:39:18,664] jepsen worker 684 - jepsen.util 684 :ok :add 2207 INFO [2022-01-10 08:39:18,667] jepsen worker 28 - jepsen.util 28 :ok :add 2211 INFO [2022-01-10 08:39:18,668] jepsen worker 213 - jepsen.util 213 :ok :add 2206 INFO [2022-01-10 08:39:18,669] jepsen worker 570 - jepsen.util 570 :ok :add 2212 INFO [2022-01-10 08:39:18,670] jepsen worker 566 - jepsen.util 566 :invoke :add 2215 INFO [2022-01-10 08:39:18,671] jepsen worker 852 - jepsen.util 852 :ok :add 2214 INFO [2022-01-10 08:39:18,673] jepsen worker 50 - jepsen.util 50 :invoke :add 2216 INFO [2022-01-10 08:39:18,673] jepsen worker 49 - jepsen.util 49 :ok :add 2213 INFO [2022-01-10 08:39:18,677] jepsen worker 722 - jepsen.util 722 :invoke :add 2217 INFO [2022-01-10 08:39:18,680] jepsen worker 742 - jepsen.util 742 :invoke :add 2218 INFO [2022-01-10 08:39:18,680] jepsen worker 566 - jepsen.util 566 :ok :add 2215 INFO [2022-01-10 08:39:18,682] jepsen worker 50 - jepsen.util 50 :ok :add 2216 INFO [2022-01-10 08:39:18,683] jepsen worker 410 - jepsen.util 410 :invoke :add 2219 INFO [2022-01-10 08:39:18,686] jepsen worker 462 - jepsen.util 462 :invoke :add 2220 INFO [2022-01-10 08:39:18,687] jepsen worker 742 - jepsen.util 742 :ok :add 2218 INFO [2022-01-10 08:39:18,689] jepsen worker 731 - jepsen.util 731 :invoke :add 2221 INFO [2022-01-10 08:39:18,691] jepsen worker 291 - jepsen.util 291 :invoke :add 2222 INFO [2022-01-10 08:39:18,692] jepsen worker 722 - jepsen.util 722 :ok :add 2217 INFO [2022-01-10 08:39:18,694] jepsen worker 526 - jepsen.util 526 :invoke :add 2223 INFO [2022-01-10 08:39:18,695] jepsen worker 410 - jepsen.util 410 :ok :add 2219 INFO [2022-01-10 08:39:18,698] jepsen worker 731 - jepsen.util 731 :ok :add 2221 INFO [2022-01-10 08:39:18,698] jepsen worker 93 - jepsen.util 93 :invoke :add 2224 INFO [2022-01-10 08:39:18,701] jepsen worker 657 - jepsen.util 657 :invoke :add 2225 INFO [2022-01-10 08:39:18,702] jepsen worker 526 - jepsen.util 526 :ok :add 2223 INFO [2022-01-10 08:39:18,704] jepsen worker 462 - jepsen.util 462 :ok :add 2220 INFO [2022-01-10 08:39:18,705] jepsen worker 736 - jepsen.util 736 :invoke :add 2226 INFO [2022-01-10 08:39:18,708] jepsen worker 202 - jepsen.util 202 :invoke :add 2227 INFO [2022-01-10 08:39:18,709] jepsen worker 657 - jepsen.util 657 :ok :add 2225 INFO [2022-01-10 08:39:18,711] jepsen worker 876 - jepsen.util 876 :invoke :add 2228 INFO [2022-01-10 08:39:18,712] jepsen worker 291 - jepsen.util 291 :ok :add 2222 INFO [2022-01-10 08:39:18,712] jepsen worker 93 - jepsen.util 93 :ok :add 2224 INFO [2022-01-10 08:39:18,715] jepsen worker 618 - jepsen.util 618 :invoke :add 2229 INFO [2022-01-10 08:39:18,716] jepsen worker 202 - jepsen.util 202 :ok :add 2227 INFO [2022-01-10 08:39:18,717] jepsen worker 461 - jepsen.util 461 :invoke :add 2230 INFO [2022-01-10 08:39:18,718] jepsen worker 736 - jepsen.util 736 :ok :add 2226 INFO [2022-01-10 08:39:18,721] jepsen worker 989 - jepsen.util 989 :invoke :add 2231 INFO [2022-01-10 08:39:18,723] jepsen worker 600 - jepsen.util 600 :invoke :add 2232 INFO [2022-01-10 08:39:18,723] jepsen worker 876 - jepsen.util 876 :ok :add 2228 INFO [2022-01-10 08:39:18,725] jepsen worker 461 - jepsen.util 461 :ok :add 2230 INFO [2022-01-10 08:39:18,728] jepsen worker 919 - jepsen.util 919 :invoke :add 2233 INFO [2022-01-10 08:39:18,731] jepsen worker 502 - jepsen.util 502 :invoke :add 2234 INFO [2022-01-10 08:39:18,733] jepsen worker 153 - jepsen.util 153 :invoke :add 2235 INFO [2022-01-10 08:39:18,736] jepsen worker 89 - jepsen.util 89 :invoke :add 2236 INFO [2022-01-10 08:39:18,740] jepsen worker 602 - jepsen.util 602 :invoke :add 2237 INFO [2022-01-10 08:39:18,743] jepsen worker 153 - jepsen.util 153 :ok :add 2235 INFO [2022-01-10 08:39:18,743] jepsen worker 502 - jepsen.util 502 :ok :add 2234 INFO [2022-01-10 08:39:18,744] jepsen worker 653 - jepsen.util 653 :invoke :add 2238 INFO [2022-01-10 08:39:18,746] jepsen worker 989 - jepsen.util 989 :ok :add 2231 INFO [2022-01-10 08:39:18,746] jepsen worker 618 - jepsen.util 618 :ok :add 2229 INFO [2022-01-10 08:39:18,746] jepsen worker 600 - jepsen.util 600 :ok :add 2232 INFO [2022-01-10 08:39:18,747] jepsen worker 973 - jepsen.util 973 :invoke :add 2239 INFO [2022-01-10 08:39:18,748] jepsen worker 89 - jepsen.util 89 :ok :add 2236 INFO [2022-01-10 08:39:18,750] jepsen worker 919 - jepsen.util 919 :ok :add 2233 INFO [2022-01-10 08:39:18,752] jepsen worker 83 - jepsen.util 83 :invoke :add 2240 INFO [2022-01-10 08:39:18,754] jepsen worker 602 - jepsen.util 602 :ok :add 2237 INFO [2022-01-10 08:39:18,756] jepsen worker 446 - jepsen.util 446 :invoke :add 2241 INFO [2022-01-10 08:39:18,756] jepsen worker 973 - jepsen.util 973 :ok :add 2239 INFO [2022-01-10 08:39:18,756] jepsen worker 653 - jepsen.util 653 :ok :add 2238 INFO [2022-01-10 08:39:18,760] jepsen worker 347 - jepsen.util 347 :invoke :add 2242 INFO [2022-01-10 08:39:18,762] jepsen worker 83 - jepsen.util 83 :ok :add 2240 INFO [2022-01-10 08:39:18,763] jepsen worker 126 - jepsen.util 126 :invoke :add 2243 INFO [2022-01-10 08:39:18,766] jepsen worker 417 - jepsen.util 417 :invoke :add 2244 INFO [2022-01-10 08:39:18,767] jepsen worker 446 - jepsen.util 446 :ok :add 2241 INFO [2022-01-10 08:39:18,769] jepsen worker 369 - jepsen.util 369 :invoke :add 2245 INFO [2022-01-10 08:39:18,771] jepsen worker 347 - jepsen.util 347 :ok :add 2242 INFO [2022-01-10 08:39:18,771] jepsen worker 126 - jepsen.util 126 :ok :add 2243 INFO [2022-01-10 08:39:18,772] jepsen worker 417 - jepsen.util 417 :ok :add 2244 INFO [2022-01-10 08:39:18,774] jepsen worker 392 - jepsen.util 392 :invoke :add 2246 INFO [2022-01-10 08:39:18,775] jepsen worker 369 - jepsen.util 369 :ok :add 2245 INFO [2022-01-10 08:39:18,779] jepsen worker 246 - jepsen.util 246 :invoke :add 2247 INFO [2022-01-10 08:39:18,781] jepsen worker 263 - jepsen.util 263 :invoke :add 2248 INFO [2022-01-10 08:39:18,783] jepsen worker 392 - jepsen.util 392 :ok :add 2246 INFO [2022-01-10 08:39:18,783] jepsen worker 521 - jepsen.util 521 :invoke :add 2249 INFO [2022-01-10 08:39:18,786] jepsen worker 883 - jepsen.util 883 :invoke :add 2250 INFO [2022-01-10 08:39:18,787] jepsen worker 246 - jepsen.util 246 :ok :add 2247 INFO [2022-01-10 08:39:18,789] jepsen worker 50 - jepsen.util 50 :invoke :add 2251 INFO [2022-01-10 08:39:18,792] jepsen worker 383 - jepsen.util 383 :invoke :add 2252 INFO [2022-01-10 08:39:18,794] jepsen worker 106 - jepsen.util 106 :invoke :add 2253 INFO [2022-01-10 08:39:18,795] jepsen worker 521 - jepsen.util 521 :ok :add 2249 INFO [2022-01-10 08:39:18,796] jepsen worker 263 - jepsen.util 263 :ok :add 2248 INFO [2022-01-10 08:39:18,798] jepsen worker 285 - jepsen.util 285 :invoke :add 2254 INFO [2022-01-10 08:39:18,801] jepsen worker 677 - jepsen.util 677 :invoke :add 2255 INFO [2022-01-10 08:39:18,802] jepsen worker 50 - jepsen.util 50 :ok :add 2251 INFO [2022-01-10 08:39:18,805] jepsen worker 967 - jepsen.util 967 :invoke :add 2256 INFO [2022-01-10 08:39:18,808] jepsen worker 106 - jepsen.util 106 :ok :add 2253 INFO [2022-01-10 08:39:18,808] jepsen worker 285 - jepsen.util 285 :ok :add 2254 INFO [2022-01-10 08:39:18,809] jepsen worker 883 - jepsen.util 883 :ok :add 2250 INFO [2022-01-10 08:39:18,809] jepsen worker 573 - jepsen.util 573 :invoke :add 2257 INFO [2022-01-10 08:39:18,813] jepsen worker 677 - jepsen.util 677 :ok :add 2255 INFO [2022-01-10 08:39:18,814] jepsen worker 383 - jepsen.util 383 :ok :add 2252 INFO [2022-01-10 08:39:18,815] jepsen worker 257 - jepsen.util 257 :invoke :add 2258 INFO [2022-01-10 08:39:18,817] jepsen worker 967 - jepsen.util 967 :ok :add 2256 INFO [2022-01-10 08:39:18,819] jepsen worker 344 - jepsen.util 344 :invoke :add 2259 INFO [2022-01-10 08:39:18,821] jepsen worker 257 - jepsen.util 257 :ok :add 2258 INFO [2022-01-10 08:39:18,822] jepsen worker 933 - jepsen.util 933 :invoke :add 2260 INFO [2022-01-10 08:39:18,825] jepsen worker 769 - jepsen.util 769 :invoke :add 2261 INFO [2022-01-10 08:39:18,828] jepsen worker 857 - jepsen.util 857 :invoke :add 2262 INFO [2022-01-10 08:39:18,829] jepsen worker 344 - jepsen.util 344 :ok :add 2259 INFO [2022-01-10 08:39:18,832] jepsen worker 941 - jepsen.util 941 :invoke :add 2263 INFO [2022-01-10 08:39:18,835] jepsen worker 377 - jepsen.util 377 :invoke :add 2264 INFO [2022-01-10 08:39:18,838] jepsen worker 338 - jepsen.util 338 :invoke :add 2265 INFO [2022-01-10 08:39:18,841] jepsen worker 49 - jepsen.util 49 :invoke :add 2266 INFO [2022-01-10 08:39:18,842] jepsen worker 377 - jepsen.util 377 :ok :add 2264 INFO [2022-01-10 08:39:18,844] jepsen worker 338 - jepsen.util 338 :ok :add 2265 INFO [2022-01-10 08:39:18,845] jepsen worker 60 - jepsen.util 60 :invoke :add 2267 INFO [2022-01-10 08:39:18,848] jepsen worker 933 - jepsen.util 933 :ok :add 2260 INFO [2022-01-10 08:39:18,848] jepsen worker 579 - jepsen.util 579 :invoke :add 2268 INFO [2022-01-10 08:39:18,849] jepsen worker 573 - jepsen.util 573 :ok :add 2257 INFO [2022-01-10 08:39:18,849] jepsen worker 941 - jepsen.util 941 :ok :add 2263 INFO [2022-01-10 08:39:18,851] jepsen worker 281 - jepsen.util 281 :invoke :add 2269 INFO [2022-01-10 08:39:18,852] jepsen worker 769 - jepsen.util 769 :ok :add 2261 INFO [2022-01-10 08:39:18,852] jepsen worker 49 - jepsen.util 49 :ok :add 2266 INFO [2022-01-10 08:39:18,854] jepsen worker 857 - jepsen.util 857 :ok :add 2262 INFO [2022-01-10 08:39:18,857] jepsen worker 659 - jepsen.util 659 :invoke :add 2270 INFO [2022-01-10 08:39:18,858] jepsen worker 579 - jepsen.util 579 :ok :add 2268 INFO [2022-01-10 08:39:18,858] jepsen worker 60 - jepsen.util 60 :ok :add 2267 INFO [2022-01-10 08:39:18,860] jepsen worker 281 - jepsen.util 281 :ok :add 2269 INFO [2022-01-10 08:39:18,862] jepsen worker 171 - jepsen.util 171 :invoke :add 2271 INFO [2022-01-10 08:39:18,866] jepsen worker 659 - jepsen.util 659 :ok :add 2270 INFO [2022-01-10 08:39:18,866] jepsen worker 198 - jepsen.util 198 :invoke :add 2272 INFO [2022-01-10 08:39:18,869] jepsen worker 688 - jepsen.util 688 :invoke :add 2273 INFO [2022-01-10 08:39:18,871] jepsen worker 171 - jepsen.util 171 :ok :add 2271 INFO [2022-01-10 08:39:18,871] jepsen worker 433 - jepsen.util 433 :invoke :add 2274 INFO [2022-01-10 08:39:18,874] jepsen worker 236 - jepsen.util 236 :invoke :add 2275 INFO [2022-01-10 08:39:18,876] jepsen worker 466 - jepsen.util 466 :invoke :add 2276 INFO [2022-01-10 08:39:18,877] jepsen worker 198 - jepsen.util 198 :ok :add 2272 INFO [2022-01-10 08:39:18,879] jepsen worker 688 - jepsen.util 688 :ok :add 2273 INFO [2022-01-10 08:39:18,880] jepsen worker 622 - jepsen.util 622 :invoke :add 2277 INFO [2022-01-10 08:39:18,883] jepsen worker 433 - jepsen.util 433 :ok :add 2274 INFO [2022-01-10 08:39:18,884] jepsen worker 82 - jepsen.util 82 :invoke :add 2278 INFO [2022-01-10 08:39:18,884] jepsen worker 236 - jepsen.util 236 :ok :add 2275 INFO [2022-01-10 08:39:18,887] jepsen worker 435 - jepsen.util 435 :invoke :add 2279 INFO [2022-01-10 08:39:18,889] jepsen worker 622 - jepsen.util 622 :ok :add 2277 INFO [2022-01-10 08:39:18,889] jepsen worker 180 - jepsen.util 180 :invoke :add 2280 INFO [2022-01-10 08:39:18,891] jepsen worker 82 - jepsen.util 82 :ok :add 2278 INFO [2022-01-10 08:39:18,892] jepsen worker 127 - jepsen.util 127 :invoke :add 2281 INFO [2022-01-10 08:39:18,893] jepsen worker 466 - jepsen.util 466 :ok :add 2276 INFO [2022-01-10 08:39:18,895] jepsen worker 905 - jepsen.util 905 :invoke :add 2282 INFO [2022-01-10 08:39:18,896] jepsen worker 435 - jepsen.util 435 :ok :add 2279 INFO [2022-01-10 08:39:18,897] jepsen worker 360 - jepsen.util 360 :invoke :add 2283 INFO [2022-01-10 08:39:18,900] jepsen worker 78 - jepsen.util 78 :invoke :add 2284 INFO [2022-01-10 08:39:18,902] jepsen worker 706 - jepsen.util 706 :invoke :add 2285 INFO [2022-01-10 08:39:18,905] jepsen worker 232 - jepsen.util 232 :invoke :add 2286 INFO [2022-01-10 08:39:18,906] jepsen worker 127 - jepsen.util 127 :ok :add 2281 INFO [2022-01-10 08:39:18,907] jepsen worker 180 - jepsen.util 180 :ok :add 2280 INFO [2022-01-10 08:39:18,908] jepsen worker 456 - jepsen.util 456 :invoke :add 2287 INFO [2022-01-10 08:39:18,911] jepsen worker 220 - jepsen.util 220 :invoke :add 2288 INFO [2022-01-10 08:39:18,911] jepsen worker 360 - jepsen.util 360 :ok :add 2283 INFO [2022-01-10 08:39:18,912] jepsen worker 706 - jepsen.util 706 :ok :add 2285 INFO [2022-01-10 08:39:18,912] jepsen worker 78 - jepsen.util 78 :ok :add 2284 INFO [2022-01-10 08:39:18,913] jepsen worker 622 - jepsen.util 622 :invoke :add 2289 INFO [2022-01-10 08:39:18,916] jepsen worker 462 - jepsen.util 462 :invoke :add 2290 INFO [2022-01-10 08:39:18,917] jepsen worker 905 - jepsen.util 905 :ok :add 2282 INFO [2022-01-10 08:39:18,920] jepsen worker 373 - jepsen.util 373 :invoke :add 2291 INFO [2022-01-10 08:39:18,922] jepsen worker 333 - jepsen.util 333 :invoke :add 2292 INFO [2022-01-10 08:39:18,925] jepsen worker 538 - jepsen.util 538 :invoke :add 2293 INFO [2022-01-10 08:39:18,928] jepsen worker 778 - jepsen.util 778 :invoke :add 2294 INFO [2022-01-10 08:39:18,930] jepsen worker 916 - jepsen.util 916 :invoke :add 2295 INFO [2022-01-10 08:39:18,933] jepsen worker 269 - jepsen.util 269 :invoke :add 2296 INFO [2022-01-10 08:39:18,937] jepsen worker 456 - jepsen.util 456 :ok :add 2287 INFO [2022-01-10 08:39:18,937] jepsen worker 25 - jepsen.util 25 :invoke :add 2297 INFO [2022-01-10 08:39:18,941] jepsen worker 43 - jepsen.util 43 :invoke :add 2298 INFO [2022-01-10 08:39:18,941] jepsen worker 232 - jepsen.util 232 :ok :add 2286 INFO [2022-01-10 08:39:18,944] jepsen worker 397 - jepsen.util 397 :invoke :add 2299 INFO [2022-01-10 08:39:18,947] jepsen worker 138 - jepsen.util 138 :invoke :add 2300 INFO [2022-01-10 08:39:18,949] jepsen worker 43 - jepsen.util 43 :ok :add 2298 INFO [2022-01-10 08:39:18,950] jepsen worker 682 - jepsen.util 682 :invoke :add 2301 INFO [2022-01-10 08:39:18,951] jepsen worker 220 - jepsen.util 220 :ok :add 2288 INFO [2022-01-10 08:39:18,953] jepsen worker 207 - jepsen.util 207 :invoke :add 2302 INFO [2022-01-10 08:39:18,956] jepsen worker 111 - jepsen.util 111 :invoke :add 2303 INFO [2022-01-10 08:39:18,958] jepsen worker 448 - jepsen.util 448 :invoke :add 2304 INFO [2022-01-10 08:39:18,959] jepsen worker 778 - jepsen.util 778 :ok :add 2294 INFO [2022-01-10 08:39:18,959] jepsen worker 622 - jepsen.util 622 :ok :add 2289 INFO [2022-01-10 08:39:18,961] jepsen worker 397 - jepsen.util 397 :ok :add 2299 INFO [2022-01-10 08:39:18,961] jepsen worker 382 - jepsen.util 382 :invoke :add 2305 INFO [2022-01-10 08:39:18,965] jepsen worker 793 - jepsen.util 793 :invoke :add 2306 INFO [2022-01-10 08:39:18,967] jepsen worker 3 - jepsen.util 3 :invoke :add 2307 INFO [2022-01-10 08:39:18,971] jepsen worker 717 - jepsen.util 717 :invoke :add 2308 INFO [2022-01-10 08:39:18,973] jepsen worker 474 - jepsen.util 474 :invoke :add 2309 INFO [2022-01-10 08:39:18,976] jepsen worker 418 - jepsen.util 418 :invoke :add 2310 INFO [2022-01-10 08:39:18,979] jepsen worker 581 - jepsen.util 581 :invoke :add 2311 INFO [2022-01-10 08:39:18,981] jepsen worker 1 - jepsen.util 1 :invoke :add 2312 INFO [2022-01-10 08:39:18,984] jepsen worker 669 - jepsen.util 669 :invoke :add 2313 INFO [2022-01-10 08:39:18,986] jepsen worker 916 - jepsen.util 916 :ok :add 2295 INFO [2022-01-10 08:39:18,987] jepsen worker 269 - jepsen.util 269 :ok :add 2296 INFO [2022-01-10 08:39:18,988] jepsen worker 563 - jepsen.util 563 :invoke :add 2314 INFO [2022-01-10 08:39:18,992] jepsen worker 843 - jepsen.util 843 :invoke :add 2315 INFO [2022-01-10 08:39:18,995] jepsen worker 312 - jepsen.util 312 :invoke :add 2316 INFO [2022-01-10 08:39:18,997] jepsen worker 504 - jepsen.util 504 :invoke :add 2317 INFO [2022-01-10 08:39:18,998] jepsen worker 563 - jepsen.util 563 :ok :add 2314 INFO [2022-01-10 08:39:19,000] jepsen worker 843 - jepsen.util 843 :ok :add 2315 INFO [2022-01-10 08:39:19,001] jepsen worker 388 - jepsen.util 388 :invoke :add 2318 INFO [2022-01-10 08:39:19,003] jepsen worker 312 - jepsen.util 312 :ok :add 2316 INFO [2022-01-10 08:39:19,004] jepsen worker 546 - jepsen.util 546 :invoke :add 2319 INFO [2022-01-10 08:39:19,005] jepsen worker 504 - jepsen.util 504 :ok :add 2317 INFO [2022-01-10 08:39:19,007] jepsen worker 133 - jepsen.util 133 :invoke :add 2320 INFO [2022-01-10 08:39:19,008] jepsen worker 717 - jepsen.util 717 :ok :add 2308 INFO [2022-01-10 08:39:19,010] jepsen worker 763 - jepsen.util 763 :invoke :add 2321 INFO [2022-01-10 08:39:19,013] jepsen worker 397 - jepsen.util 397 :invoke :add 2322 INFO [2022-01-10 08:39:19,015] jepsen worker 658 - jepsen.util 658 :invoke :add 2323 INFO [2022-01-10 08:39:19,018] jepsen worker 133 - jepsen.util 133 :ok :add 2320 INFO [2022-01-10 08:39:19,018] jepsen worker 98 - jepsen.util 98 :invoke :add 2324 INFO [2022-01-10 08:39:19,020] jepsen worker 25 - jepsen.util 25 :ok :add 2297 INFO [2022-01-10 08:39:19,020] jepsen worker 763 - jepsen.util 763 :ok :add 2321 INFO [2022-01-10 08:39:19,020] jepsen worker 3 - jepsen.util 3 :ok :add 2307 INFO [2022-01-10 08:39:19,021] jepsen worker 397 - jepsen.util 397 :ok :add 2322 INFO [2022-01-10 08:39:19,023] jepsen worker 356 - jepsen.util 356 :invoke :add 2325 INFO [2022-01-10 08:39:19,025] jepsen worker 658 - jepsen.util 658 :ok :add 2323 INFO [2022-01-10 08:39:19,026] jepsen worker 65 - jepsen.util 65 :invoke :add 2326 INFO [2022-01-10 08:39:19,030] jepsen worker 429 - jepsen.util 429 :invoke :add 2327 INFO [2022-01-10 08:39:19,032] jepsen worker 68 - jepsen.util 68 :invoke :add 2328 INFO [2022-01-10 08:39:19,035] jepsen worker 879 - jepsen.util 879 :invoke :add 2329 INFO [2022-01-10 08:39:19,038] jepsen worker 605 - jepsen.util 605 :invoke :add 2330 INFO [2022-01-10 08:39:19,040] jepsen worker 474 - jepsen.util 474 :ok :add 2309 INFO [2022-01-10 08:39:19,041] jepsen worker 581 - jepsen.util 581 :ok :add 2311 INFO [2022-01-10 08:39:19,041] jepsen worker 893 - jepsen.util 893 :invoke :add 2331 INFO [2022-01-10 08:39:19,043] jepsen worker 418 - jepsen.util 418 :ok :add 2310 INFO [2022-01-10 08:39:19,044] jepsen worker 684 - jepsen.util 684 :invoke :add 2332 INFO [2022-01-10 08:39:19,046] jepsen worker 669 - jepsen.util 669 :ok :add 2313 INFO [2022-01-10 08:39:19,046] jepsen worker 429 - jepsen.util 429 :ok :add 2327 INFO [2022-01-10 08:39:19,047] jepsen worker 130 - jepsen.util 130 :invoke :add 2333 INFO [2022-01-10 08:39:19,048] jepsen worker 65 - jepsen.util 65 :ok :add 2326 INFO [2022-01-10 08:39:19,048] jepsen worker 1 - jepsen.util 1 :ok :add 2312 INFO [2022-01-10 08:39:19,051] jepsen worker 376 - jepsen.util 376 :invoke :add 2334 INFO [2022-01-10 08:39:19,054] jepsen worker 772 - jepsen.util 772 :invoke :add 2335 INFO [2022-01-10 08:39:19,056] jepsen worker 10 - jepsen.util 10 :invoke :add 2336 INFO [2022-01-10 08:39:19,059] jepsen worker 498 - jepsen.util 498 :invoke :add 2337 INFO [2022-01-10 08:39:19,061] jepsen worker 6 - jepsen.util 6 :invoke :add 2338 INFO [2022-01-10 08:39:19,062] jepsen worker 98 - jepsen.util 98 :ok :add 2324 INFO [2022-01-10 08:39:19,064] jepsen worker 914 - jepsen.util 914 :invoke :add 2339 INFO [2022-01-10 08:39:19,065] jepsen worker 356 - jepsen.util 356 :ok :add 2325 INFO [2022-01-10 08:39:19,067] jepsen worker 68 - jepsen.util 68 :ok :add 2328 INFO [2022-01-10 08:39:19,067] jepsen worker 448 - jepsen.util 448 :ok :add 2304 INFO [2022-01-10 08:39:19,067] jepsen worker 605 - jepsen.util 605 :ok :add 2330 INFO [2022-01-10 08:39:19,068] jepsen worker 6 - jepsen.util 6 :ok :add 2338 INFO [2022-01-10 08:39:19,069] jepsen worker 390 - jepsen.util 390 :invoke :add 2340 INFO [2022-01-10 08:39:19,073] jepsen worker 742 - jepsen.util 742 :invoke :add 2341 INFO [2022-01-10 08:39:19,073] jepsen worker 914 - jepsen.util 914 :ok :add 2339 INFO [2022-01-10 08:39:19,073] jepsen worker 382 - jepsen.util 382 :ok :add 2305 INFO [2022-01-10 08:39:19,073] jepsen worker 879 - jepsen.util 879 :ok :add 2329 INFO [2022-01-10 08:39:19,073] jepsen worker 893 - jepsen.util 893 :ok :add 2331 INFO [2022-01-10 08:39:19,076] jepsen worker 136 - jepsen.util 136 :invoke :add 2342 INFO [2022-01-10 08:39:19,079] jepsen worker 736 - jepsen.util 736 :invoke :add 2343 INFO [2022-01-10 08:39:19,081] jepsen worker 462 - jepsen.util 462 :ok :add 2290 INFO [2022-01-10 08:39:19,083] jepsen worker 602 - jepsen.util 602 :invoke :add 2344 INFO [2022-01-10 08:39:19,086] jepsen worker 677 - jepsen.util 677 :invoke :add 2345 INFO [2022-01-10 08:39:19,088] jepsen worker 888 - jepsen.util 888 :invoke :add 2346 INFO [2022-01-10 08:39:19,091] jepsen worker 357 - jepsen.util 357 :invoke :add 2347 INFO [2022-01-10 08:39:19,094] jepsen worker 470 - jepsen.util 470 :invoke :add 2348 INFO [2022-01-10 08:39:19,096] jepsen worker 366 - jepsen.util 366 :invoke :add 2349 INFO [2022-01-10 08:39:19,099] jepsen worker 105 - jepsen.util 105 :invoke :add 2350 INFO [2022-01-10 08:39:19,101] jepsen worker 654 - jepsen.util 654 :invoke :add 2351 INFO [2022-01-10 08:39:19,103] jepsen worker 935 - jepsen.util 935 :invoke :add 2352 INFO [2022-01-10 08:39:19,104] jepsen worker 366 - jepsen.util 366 :ok :add 2349 INFO [2022-01-10 08:39:19,106] jepsen worker 94 - jepsen.util 94 :invoke :add 2353 INFO [2022-01-10 08:39:19,108] jepsen worker 105 - jepsen.util 105 :ok :add 2350 INFO [2022-01-10 08:39:19,108] jepsen worker 470 - jepsen.util 470 :ok :add 2348 INFO [2022-01-10 08:39:19,109] jepsen worker 683 - jepsen.util 683 :invoke :add 2354 INFO [2022-01-10 08:39:19,111] jepsen worker 654 - jepsen.util 654 :ok :add 2351 INFO [2022-01-10 08:39:19,111] jepsen worker 408 - jepsen.util 408 :invoke :add 2355 INFO [2022-01-10 08:39:19,112] jepsen worker 935 - jepsen.util 935 :ok :add 2352 INFO [2022-01-10 08:39:19,115] jepsen worker 18 - jepsen.util 18 :invoke :add 2356 INFO [2022-01-10 08:39:19,116] jepsen worker 130 - jepsen.util 130 :ok :add 2333 INFO [2022-01-10 08:39:19,116] jepsen worker 94 - jepsen.util 94 :ok :add 2353 INFO [2022-01-10 08:39:19,118] jepsen worker 765 - jepsen.util 765 :invoke :add 2357 INFO [2022-01-10 08:39:19,119] jepsen worker 793 - jepsen.util 793 :ok :add 2306 INFO [2022-01-10 08:39:19,119] jepsen worker 684 - jepsen.util 684 :ok :add 2332 INFO [2022-01-10 08:39:19,121] jepsen worker 22 - jepsen.util 22 :invoke :add 2358 INFO [2022-01-10 08:39:19,125] jepsen worker 390 - jepsen.util 390 :ok :add 2340 INFO [2022-01-10 08:39:19,125] jepsen worker 208 - jepsen.util 208 :invoke :add 2359 INFO [2022-01-10 08:39:19,129] jepsen worker 908 - jepsen.util 908 :invoke :add 2360 INFO [2022-01-10 08:39:19,130] jepsen worker 683 - jepsen.util 683 :ok :add 2354 INFO [2022-01-10 08:39:19,132] jepsen worker 136 - jepsen.util 136 :ok :add 2342 INFO [2022-01-10 08:39:19,133] jepsen worker 11 - jepsen.util 11 :invoke :add 2361 INFO [2022-01-10 08:39:19,136] jepsen worker 753 - jepsen.util 753 :invoke :add 2362 INFO [2022-01-10 08:39:19,138] jepsen worker 439 - jepsen.util 439 :invoke :add 2363 INFO [2022-01-10 08:39:19,142] jepsen worker 787 - jepsen.util 787 :invoke :add 2364 INFO [2022-01-10 08:39:19,144] jepsen worker 550 - jepsen.util 550 :invoke :add 2365 INFO [2022-01-10 08:39:19,146] jepsen worker 408 - jepsen.util 408 :ok :add 2355 INFO [2022-01-10 08:39:19,147] jepsen worker 56 - jepsen.util 56 :invoke :add 2366 INFO [2022-01-10 08:39:19,147] jepsen worker 18 - jepsen.util 18 :ok :add 2356 INFO [2022-01-10 08:39:19,149] jepsen worker 685 - jepsen.util 685 :invoke :add 2367 INFO [2022-01-10 08:39:19,151] jepsen worker 787 - jepsen.util 787 :ok :add 2364 INFO [2022-01-10 08:39:19,152] jepsen worker 689 - jepsen.util 689 :invoke :add 2368 INFO [2022-01-10 08:39:19,154] jepsen worker 550 - jepsen.util 550 :ok :add 2365 INFO [2022-01-10 08:39:19,155] jepsen worker 429 - jepsen.util 429 :invoke :add 2369 INFO [2022-01-10 08:39:19,158] jepsen worker 685 - jepsen.util 685 :ok :add 2367 INFO [2022-01-10 08:39:19,158] jepsen worker 919 - jepsen.util 919 :invoke :add 2370 INFO [2022-01-10 08:39:19,158] jepsen worker 56 - jepsen.util 56 :ok :add 2366 INFO [2022-01-10 08:39:19,161] jepsen worker 499 - jepsen.util 499 :invoke :add 2371 INFO [2022-01-10 08:39:19,162] jepsen worker 439 - jepsen.util 439 :ok :add 2363 INFO [2022-01-10 08:39:19,163] jepsen worker 765 - jepsen.util 765 :ok :add 2357 INFO [2022-01-10 08:39:19,164] jepsen worker 576 - jepsen.util 576 :invoke :add 2372 INFO [2022-01-10 08:39:19,166] jepsen worker 908 - jepsen.util 908 :ok :add 2360 INFO [2022-01-10 08:39:19,167] jepsen worker 390 - jepsen.util 390 :invoke :add 2373 INFO [2022-01-10 08:39:19,167] jepsen worker 11 - jepsen.util 11 :ok :add 2361 INFO [2022-01-10 08:39:19,171] jepsen worker 725 - jepsen.util 725 :invoke :add 2374 INFO [2022-01-10 08:39:19,171] jepsen worker 373 - jepsen.util 373 :ok :add 2291 INFO [2022-01-10 08:39:19,171] jepsen worker 682 - jepsen.util 682 :ok :add 2301 INFO [2022-01-10 08:39:19,171] jepsen worker 333 - jepsen.util 333 :ok :add 2292 INFO [2022-01-10 08:39:19,171] jepsen worker 546 - jepsen.util 546 :ok :add 2319 INFO [2022-01-10 08:39:19,172] jepsen worker 753 - jepsen.util 753 :ok :add 2362 INFO [2022-01-10 08:39:19,172] jepsen worker 772 - jepsen.util 772 :ok :add 2335 INFO [2022-01-10 08:39:19,172] jepsen worker 207 - jepsen.util 207 :ok :add 2302 INFO [2022-01-10 08:39:19,174] jepsen worker 692 - jepsen.util 692 :invoke :add 2375 INFO [2022-01-10 08:39:19,178] jepsen worker 221 - jepsen.util 221 :invoke :add 2376 INFO [2022-01-10 08:39:19,182] jepsen worker 365 - jepsen.util 365 :invoke :add 2377 INFO [2022-01-10 08:39:19,184] jepsen worker 816 - jepsen.util 816 :invoke :add 2378 INFO [2022-01-10 08:39:19,185] jepsen worker 538 - jepsen.util 538 :ok :add 2293 INFO [2022-01-10 08:39:19,185] jepsen worker 498 - jepsen.util 498 :ok :add 2337 INFO [2022-01-10 08:39:19,185] jepsen worker 111 - jepsen.util 111 :ok :add 2303 INFO [2022-01-10 08:39:19,185] jepsen worker 376 - jepsen.util 376 :ok :add 2334 INFO [2022-01-10 08:39:19,185] jepsen worker 138 - jepsen.util 138 :ok :add 2300 INFO [2022-01-10 08:39:19,185] jepsen worker 388 - jepsen.util 388 :ok :add 2318 INFO [2022-01-10 08:39:19,186] jepsen worker 921 - jepsen.util 921 :invoke :add 2379 INFO [2022-01-10 08:39:19,190] jepsen worker 104 - jepsen.util 104 :invoke :add 2380 INFO [2022-01-10 08:39:19,192] jepsen worker 816 - jepsen.util 816 :ok :add 2378 INFO [2022-01-10 08:39:19,193] jepsen worker 971 - jepsen.util 971 :invoke :add 2381 INFO [2022-01-10 08:39:19,194] jepsen worker 10 - jepsen.util 10 :ok :add 2336 INFO [2022-01-10 08:39:19,196] jepsen worker 441 - jepsen.util 441 :invoke :add 2382 INFO [2022-01-10 08:39:19,198] jepsen worker 358 - jepsen.util 358 :invoke :add 2383 INFO [2022-01-10 08:39:19,202] jepsen worker 250 - jepsen.util 250 :invoke :add 2384 INFO [2022-01-10 08:39:19,202] jepsen worker 921 - jepsen.util 921 :ok :add 2379 INFO [2022-01-10 08:39:19,204] jepsen worker 781 - jepsen.util 781 :invoke :add 2385 INFO [2022-01-10 08:39:19,206] jepsen worker 104 - jepsen.util 104 :ok :add 2380 INFO [2022-01-10 08:39:19,206] jepsen worker 971 - jepsen.util 971 :ok :add 2381 INFO [2022-01-10 08:39:19,207] jepsen worker 643 - jepsen.util 643 :invoke :add 2386 INFO [2022-01-10 08:39:19,210] jepsen worker 358 - jepsen.util 358 :ok :add 2383 INFO [2022-01-10 08:39:19,210] jepsen worker 279 - jepsen.util 279 :invoke :add 2387 INFO [2022-01-10 08:39:19,214] jepsen worker 272 - jepsen.util 272 :invoke :add 2388 INFO [2022-01-10 08:39:19,214] jepsen worker 441 - jepsen.util 441 :ok :add 2382 INFO [2022-01-10 08:39:19,216] jepsen worker 915 - jepsen.util 915 :invoke :add 2389 INFO [2022-01-10 08:39:19,217] jepsen worker 250 - jepsen.util 250 :ok :add 2384 INFO [2022-01-10 08:39:19,218] jepsen worker 602 - jepsen.util 602 :ok :add 2344 INFO [2022-01-10 08:39:19,218] jepsen worker 919 - jepsen.util 919 :ok :add 2370 INFO [2022-01-10 08:39:19,218] jepsen worker 689 - jepsen.util 689 :ok :add 2368 INFO [2022-01-10 08:39:19,219] jepsen worker 781 - jepsen.util 781 :ok :add 2385 INFO [2022-01-10 08:39:19,219] jepsen worker 24 - jepsen.util 24 :invoke :add 2390 INFO [2022-01-10 08:39:19,222] jepsen worker 643 - jepsen.util 643 :ok :add 2386 INFO [2022-01-10 08:39:19,223] jepsen worker 429 - jepsen.util 429 :ok :add 2369 INFO [2022-01-10 08:39:19,223] jepsen worker 774 - jepsen.util 774 :invoke :add 2391 INFO [2022-01-10 08:39:19,223] jepsen worker 499 - jepsen.util 499 :ok :add 2371 INFO [2022-01-10 08:39:19,224] jepsen worker 677 - jepsen.util 677 :ok :add 2345 INFO [2022-01-10 08:39:19,226] jepsen worker 279 - jepsen.util 279 :ok :add 2387 INFO [2022-01-10 08:39:19,226] jepsen worker 888 - jepsen.util 888 :ok :add 2346 INFO [2022-01-10 08:39:19,227] jepsen worker 576 - jepsen.util 576 :ok :add 2372 INFO [2022-01-10 08:39:19,227] jepsen worker 392 - jepsen.util 392 :invoke :add 2392 INFO [2022-01-10 08:39:19,230] jepsen worker 924 - jepsen.util 924 :invoke :add 2393 INFO [2022-01-10 08:39:19,231] jepsen worker 357 - jepsen.util 357 :ok :add 2347 INFO [2022-01-10 08:39:19,231] jepsen worker 390 - jepsen.util 390 :ok :add 2373 INFO [2022-01-10 08:39:19,232] jepsen worker 742 - jepsen.util 742 :ok :add 2341 INFO [2022-01-10 08:39:19,232] jepsen worker 208 - jepsen.util 208 :ok :add 2359 INFO [2022-01-10 08:39:19,234] jepsen worker 21 - jepsen.util 21 :invoke :add 2394 INFO [2022-01-10 08:39:19,237] jepsen worker 318 - jepsen.util 318 :invoke :add 2395 INFO [2022-01-10 08:39:19,239] jepsen worker 603 - jepsen.util 603 :invoke :add 2396 INFO [2022-01-10 08:39:19,241] jepsen worker 548 - jepsen.util 548 :invoke :add 2397 INFO [2022-01-10 08:39:19,244] jepsen worker 842 - jepsen.util 842 :invoke :add 2398 INFO [2022-01-10 08:39:19,248] jepsen worker 911 - jepsen.util 911 :invoke :add 2399 INFO [2022-01-10 08:39:19,251] jepsen worker 462 - jepsen.util 462 :invoke :add 2400 INFO [2022-01-10 08:39:19,253] jepsen worker 842 - jepsen.util 842 :ok :add 2398 INFO [2022-01-10 08:39:19,254] jepsen worker 274 - jepsen.util 274 :invoke :add 2401 INFO [2022-01-10 08:39:19,256] jepsen worker 98 - jepsen.util 98 :invoke :add 2402 INFO [2022-01-10 08:39:19,257] jepsen worker 911 - jepsen.util 911 :ok :add 2399 INFO [2022-01-10 08:39:19,259] jepsen worker 69 - jepsen.util 69 :invoke :add 2403 INFO [2022-01-10 08:39:19,260] jepsen worker 736 - jepsen.util 736 :ok :add 2343 INFO [2022-01-10 08:39:19,260] jepsen worker 365 - jepsen.util 365 :ok :add 2377 INFO [2022-01-10 08:39:19,263] jepsen worker 519 - jepsen.util 519 :invoke :add 2404 INFO [2022-01-10 08:39:19,266] jepsen worker 831 - jepsen.util 831 :invoke :add 2405 INFO [2022-01-10 08:39:19,269] jepsen worker 575 - jepsen.util 575 :invoke :add 2406 INFO [2022-01-10 08:39:19,272] jepsen worker 114 - jepsen.util 114 :invoke :add 2407 INFO [2022-01-10 08:39:19,275] jepsen worker 886 - jepsen.util 886 :invoke :add 2408 INFO [2022-01-10 08:39:19,278] jepsen worker 37 - jepsen.util 37 :invoke :add 2409 INFO [2022-01-10 08:39:19,280] jepsen worker 54 - jepsen.util 54 :invoke :add 2410 INFO [2022-01-10 08:39:19,282] jepsen worker 590 - jepsen.util 590 :invoke :add 2411 INFO [2022-01-10 08:39:19,283] jepsen worker 886 - jepsen.util 886 :ok :add 2408 INFO [2022-01-10 08:39:19,285] jepsen worker 476 - jepsen.util 476 :invoke :add 2412 INFO [2022-01-10 08:39:19,287] jepsen worker 98 - jepsen.util 98 :ok :add 2402 INFO [2022-01-10 08:39:19,288] jepsen worker 272 - jepsen.util 272 :ok :add 2388 INFO [2022-01-10 08:39:19,288] jepsen worker 24 - jepsen.util 24 :ok :add 2390 INFO [2022-01-10 08:39:19,289] jepsen worker 694 - jepsen.util 694 :invoke :add 2413 INFO [2022-01-10 08:39:19,293] jepsen worker 766 - jepsen.util 766 :invoke :add 2414 INFO [2022-01-10 08:39:19,293] jepsen worker 37 - jepsen.util 37 :ok :add 2409 INFO [2022-01-10 08:39:19,294] jepsen worker 54 - jepsen.util 54 :ok :add 2410 INFO [2022-01-10 08:39:19,296] jepsen worker 581 - jepsen.util 581 :invoke :add 2415 INFO [2022-01-10 08:39:19,300] jepsen worker 476 - jepsen.util 476 :ok :add 2412 INFO [2022-01-10 08:39:19,300] jepsen worker 590 - jepsen.util 590 :ok :add 2411 INFO [2022-01-10 08:39:19,301] jepsen worker 335 - jepsen.util 335 :invoke :add 2416 INFO [2022-01-10 08:39:19,301] jepsen worker 774 - jepsen.util 774 :ok :add 2391 INFO [2022-01-10 08:39:19,301] jepsen worker 915 - jepsen.util 915 :ok :add 2389 INFO [2022-01-10 08:39:19,302] jepsen worker 69 - jepsen.util 69 :ok :add 2403 INFO [2022-01-10 08:39:19,306] jepsen worker 631 - jepsen.util 631 :invoke :add 2417 INFO [2022-01-10 08:39:19,307] jepsen worker 462 - jepsen.util 462 :ok :add 2400 INFO [2022-01-10 08:39:19,309] jepsen worker 694 - jepsen.util 694 :ok :add 2413 INFO [2022-01-10 08:39:19,309] jepsen worker 392 - jepsen.util 392 :ok :add 2392 INFO [2022-01-10 08:39:19,309] jepsen worker 833 - jepsen.util 833 :invoke :add 2418 INFO [2022-01-10 08:39:19,311] jepsen worker 581 - jepsen.util 581 :ok :add 2415 INFO [2022-01-10 08:39:19,312] jepsen worker 401 - jepsen.util 401 :invoke :add 2419 INFO [2022-01-10 08:39:19,315] jepsen worker 280 - jepsen.util 280 :invoke :add 2420 INFO [2022-01-10 08:39:19,317] jepsen worker 744 - jepsen.util 744 :invoke :add 2421 INFO [2022-01-10 08:39:19,318] jepsen worker 631 - jepsen.util 631 :ok :add 2417 INFO [2022-01-10 08:39:19,319] jepsen worker 924 - jepsen.util 924 :ok :add 2393 INFO [2022-01-10 08:39:19,321] jepsen worker 518 - jepsen.util 518 :invoke :add 2422 INFO [2022-01-10 08:39:19,323] jepsen worker 274 - jepsen.util 274 :ok :add 2401 INFO [2022-01-10 08:39:19,324] jepsen worker 766 - jepsen.util 766 :ok :add 2414 INFO [2022-01-10 08:39:19,324] jepsen worker 527 - jepsen.util 527 :invoke :add 2423 INFO [2022-01-10 08:39:19,327] jepsen worker 43 - jepsen.util 43 :invoke :add 2424 INFO [2022-01-10 08:39:19,330] jepsen worker 723 - jepsen.util 723 :invoke :add 2425 INFO [2022-01-10 08:39:19,332] jepsen worker 744 - jepsen.util 744 :ok :add 2421 INFO [2022-01-10 08:39:19,332] jepsen worker 499 - jepsen.util 499 :invoke :add 2426 INFO [2022-01-10 08:39:19,333] jepsen worker 335 - jepsen.util 335 :ok :add 2416 INFO [2022-01-10 08:39:19,333] jepsen worker 518 - jepsen.util 518 :ok :add 2422 INFO [2022-01-10 08:39:19,337] jepsen worker 134 - jepsen.util 134 :invoke :add 2427 INFO [2022-01-10 08:39:19,339] jepsen worker 527 - jepsen.util 527 :ok :add 2423 INFO [2022-01-10 08:39:19,339] jepsen worker 77 - jepsen.util 77 :invoke :add 2428 INFO [2022-01-10 08:39:19,341] jepsen worker 326 - jepsen.util 326 :invoke :add 2429 INFO [2022-01-10 08:39:19,343] jepsen worker 101 - jepsen.util 101 :invoke :add 2430 INFO [2022-01-10 08:39:19,345] jepsen worker 833 - jepsen.util 833 :ok :add 2418 INFO [2022-01-10 08:39:19,346] jepsen worker 942 - jepsen.util 942 :invoke :add 2431 INFO [2022-01-10 08:39:19,348] jepsen worker 280 - jepsen.util 280 :ok :add 2420 INFO [2022-01-10 08:39:19,348] jepsen worker 163 - jepsen.util 163 :invoke :add 2432 INFO [2022-01-10 08:39:19,351] jepsen worker 180 - jepsen.util 180 :invoke :add 2433 INFO [2022-01-10 08:39:19,353] jepsen worker 558 - jepsen.util 558 :invoke :add 2434 INFO [2022-01-10 08:39:19,355] jepsen worker 22 - jepsen.util 22 :ok :add 2358 INFO [2022-01-10 08:39:19,356] jepsen worker 709 - jepsen.util 709 :invoke :add 2435 INFO [2022-01-10 08:39:19,356] jepsen worker 725 - jepsen.util 725 :ok :add 2374 INFO [2022-01-10 08:39:19,356] jepsen worker 575 - jepsen.util 575 :ok :add 2406 INFO [2022-01-10 08:39:19,359] jepsen worker 138 - jepsen.util 138 :invoke :add 2436 INFO [2022-01-10 08:39:19,359] jepsen worker 723 - jepsen.util 723 :ok :add 2425 INFO [2022-01-10 08:39:19,359] jepsen worker 43 - jepsen.util 43 :ok :add 2424 INFO [2022-01-10 08:39:19,361] jepsen worker 573 - jepsen.util 573 :invoke :add 2437 INFO [2022-01-10 08:39:19,363] jepsen worker 499 - jepsen.util 499 :ok :add 2426 INFO [2022-01-10 08:39:19,364] jepsen worker 692 - jepsen.util 692 :ok :add 2375 INFO [2022-01-10 08:39:19,364] jepsen worker 519 - jepsen.util 519 :ok :add 2404 INFO [2022-01-10 08:39:19,364] jepsen worker 221 - jepsen.util 221 :ok :add 2376 INFO [2022-01-10 08:39:19,364] jepsen worker 180 - jepsen.util 180 :ok :add 2433 INFO [2022-01-10 08:39:19,364] jepsen worker 101 - jepsen.util 101 :ok :add 2430 INFO [2022-01-10 08:39:19,364] jepsen worker 77 - jepsen.util 77 :ok :add 2428 INFO [2022-01-10 08:39:19,364] jepsen worker 37 - jepsen.util 37 :invoke :add 2438 INFO [2022-01-10 08:39:19,364] jepsen worker 114 - jepsen.util 114 :ok :add 2407 INFO [2022-01-10 08:39:19,367] jepsen worker 134 - jepsen.util 134 :ok :add 2427 INFO [2022-01-10 08:39:19,368] jepsen worker 590 - jepsen.util 590 :invoke :add 2439 INFO [2022-01-10 08:39:19,372] jepsen worker 278 - jepsen.util 278 :invoke :add 2440 INFO [2022-01-10 08:39:19,374] jepsen worker 360 - jepsen.util 360 :invoke :add 2441 INFO [2022-01-10 08:39:19,376] jepsen worker 590 - jepsen.util 590 :ok :add 2439 INFO [2022-01-10 08:39:19,377] jepsen worker 657 - jepsen.util 657 :invoke :add 2442 INFO [2022-01-10 08:39:19,380] jepsen worker 675 - jepsen.util 675 :invoke :add 2443 INFO [2022-01-10 08:39:19,382] jepsen worker 37 - jepsen.util 37 :ok :add 2438 INFO [2022-01-10 08:39:19,383] jepsen worker 521 - jepsen.util 521 :invoke :add 2444 INFO [2022-01-10 08:39:19,385] jepsen worker 558 - jepsen.util 558 :ok :add 2434 INFO [2022-01-10 08:39:19,386] jepsen worker 385 - jepsen.util 385 :invoke :add 2445 INFO [2022-01-10 08:39:19,389] jepsen worker 701 - jepsen.util 701 :invoke :add 2446 INFO [2022-01-10 08:39:19,392] jepsen worker 417 - jepsen.util 417 :invoke :add 2447 INFO [2022-01-10 08:39:19,399] jepsen worker 573 - jepsen.util 573 :ok :add 2437 INFO [2022-01-10 08:39:19,400] jepsen worker 87 - jepsen.util 87 :invoke :add 2448 INFO [2022-01-10 08:39:19,402] jepsen worker 136 - jepsen.util 136 :invoke :add 2449 INFO [2022-01-10 08:39:19,404] jepsen worker 138 - jepsen.util 138 :ok :add 2436 INFO [2022-01-10 08:39:19,405] jepsen worker 28 - jepsen.util 28 :invoke :add 2450 INFO [2022-01-10 08:39:19,408] jepsen worker 386 - jepsen.util 386 :invoke :add 2451 INFO [2022-01-10 08:39:19,410] jepsen worker 836 - jepsen.util 836 :invoke :add 2452 INFO [2022-01-10 08:39:19,411] jepsen worker 87 - jepsen.util 87 :ok :add 2448 INFO [2022-01-10 08:39:19,411] jepsen worker 136 - jepsen.util 136 :ok :add 2449 INFO [2022-01-10 08:39:19,414] jepsen worker 742 - jepsen.util 742 :invoke :add 2453 INFO [2022-01-10 08:39:19,416] jepsen worker 28 - jepsen.util 28 :ok :add 2450 INFO [2022-01-10 08:39:19,418] jepsen worker 25 - jepsen.util 25 :invoke :add 2454 INFO [2022-01-10 08:39:19,418] jepsen worker 386 - jepsen.util 386 :ok :add 2451 INFO [2022-01-10 08:39:19,422] jepsen worker 449 - jepsen.util 449 :invoke :add 2455 INFO [2022-01-10 08:39:19,425] jepsen worker 742 - jepsen.util 742 :ok :add 2453 INFO [2022-01-10 08:39:19,426] jepsen worker 467 - jepsen.util 467 :invoke :add 2456 INFO [2022-01-10 08:39:19,427] jepsen worker 836 - jepsen.util 836 :ok :add 2452 INFO [2022-01-10 08:39:19,427] jepsen worker 25 - jepsen.util 25 :ok :add 2454 INFO [2022-01-10 08:39:19,430] jepsen worker 328 - jepsen.util 328 :invoke :add 2457 INFO [2022-01-10 08:39:19,431] jepsen worker 449 - jepsen.util 449 :ok :add 2455 INFO [2022-01-10 08:39:19,434] jepsen worker 50 - jepsen.util 50 :invoke :add 2458 INFO [2022-01-10 08:39:19,437] jepsen worker 224 - jepsen.util 224 :invoke :add 2459 INFO [2022-01-10 08:39:19,441] jepsen worker 958 - jepsen.util 958 :invoke :add 2460 INFO [2022-01-10 08:39:19,441] jepsen worker 467 - jepsen.util 467 :ok :add 2456 INFO [2022-01-10 08:39:19,532] jepsen worker 21 - jepsen.util 21 :ok :add 2394 INFO [2022-01-10 08:39:19,533] jepsen worker 521 - jepsen.util 521 :ok :add 2444 INFO [2022-01-10 08:39:19,533] jepsen worker 385 - jepsen.util 385 :ok :add 2445 INFO [2022-01-10 08:39:19,533] jepsen worker 401 - jepsen.util 401 :ok :add 2419 INFO [2022-01-10 08:39:19,533] jepsen worker 701 - jepsen.util 701 :ok :add 2446 INFO [2022-01-10 08:39:19,533] jepsen worker 417 - jepsen.util 417 :ok :add 2447 INFO [2022-01-10 08:39:19,533] jepsen worker 709 - jepsen.util 709 :ok :add 2435 INFO [2022-01-10 08:39:19,533] jepsen worker 328 - jepsen.util 328 :ok :add 2457 INFO [2022-01-10 08:39:19,535] jepsen worker 782 - jepsen.util 782 :invoke :add 2461 INFO [2022-01-10 08:39:19,539] jepsen worker 409 - jepsen.util 409 :invoke :add 2462 INFO [2022-01-10 08:39:19,542] jepsen worker 497 - jepsen.util 497 :invoke :add 2463 INFO [2022-01-10 08:39:19,544] jepsen worker 655 - jepsen.util 655 :invoke :add 2464 INFO [2022-01-10 08:39:19,546] jepsen worker 990 - jepsen.util 990 :invoke :add 2465 INFO [2022-01-10 08:39:19,548] jepsen worker 352 - jepsen.util 352 :invoke :add 2466 INFO [2022-01-10 08:39:19,551] jepsen worker 210 - jepsen.util 210 :invoke :add 2467 INFO [2022-01-10 08:39:19,553] jepsen worker 790 - jepsen.util 790 :invoke :add 2468 INFO [2022-01-10 08:39:19,554] jepsen worker 655 - jepsen.util 655 :ok :add 2464 INFO [2022-01-10 08:39:19,555] jepsen worker 825 - jepsen.util 825 :invoke :add 2469 INFO [2022-01-10 08:39:19,556] jepsen worker 990 - jepsen.util 990 :ok :add 2465 INFO [2022-01-10 08:39:19,558] jepsen worker 890 - jepsen.util 890 :invoke :add 2470 INFO [2022-01-10 08:39:19,558] jepsen worker 352 - jepsen.util 352 :ok :add 2466 INFO [2022-01-10 08:39:19,561] jepsen worker 543 - jepsen.util 543 :invoke :add 2471 INFO [2022-01-10 08:39:19,562] jepsen worker 603 - jepsen.util 603 :ok :add 2396 INFO [2022-01-10 08:39:19,562] jepsen worker 210 - jepsen.util 210 :ok :add 2467 INFO [2022-01-10 08:39:19,563] jepsen worker 106 - jepsen.util 106 :invoke :add 2472 INFO [2022-01-10 08:39:19,565] jepsen worker 75 - jepsen.util 75 :invoke :add 2473 INFO [2022-01-10 08:39:19,568] jepsen worker 268 - jepsen.util 268 :invoke :add 2474 INFO [2022-01-10 08:39:19,569] jepsen worker 790 - jepsen.util 790 :ok :add 2468 INFO [2022-01-10 08:39:19,570] jepsen worker 825 - jepsen.util 825 :ok :add 2469 INFO [2022-01-10 08:39:19,571] jepsen worker 552 - jepsen.util 552 :invoke :add 2475 INFO [2022-01-10 08:39:19,572] jepsen worker 890 - jepsen.util 890 :ok :add 2470 INFO [2022-01-10 08:39:19,574] jepsen worker 155 - jepsen.util 155 :invoke :add 2476 INFO [2022-01-10 08:39:19,577] jepsen worker 958 - jepsen.util 958 :ok :add 2460 INFO [2022-01-10 08:39:19,577] jepsen worker 782 - jepsen.util 782 :ok :add 2461 INFO [2022-01-10 08:39:19,577] jepsen worker 544 - jepsen.util 544 :invoke :add 2477 INFO [2022-01-10 08:39:19,579] jepsen worker 106 - jepsen.util 106 :ok :add 2472 INFO [2022-01-10 08:39:19,580] jepsen worker 543 - jepsen.util 543 :ok :add 2471 INFO [2022-01-10 08:39:19,580] jepsen worker 54 - jepsen.util 54 :invoke :add 2478 INFO [2022-01-10 08:39:19,582] jepsen worker 497 - jepsen.util 497 :ok :add 2463 INFO [2022-01-10 08:39:19,582] jepsen worker 409 - jepsen.util 409 :ok :add 2462 INFO [2022-01-10 08:39:19,583] jepsen worker 628 - jepsen.util 628 :invoke :add 2479 INFO [2022-01-10 08:39:19,586] jepsen worker 891 - jepsen.util 891 :invoke :add 2480 INFO [2022-01-10 08:39:19,589] jepsen worker 682 - jepsen.util 682 :invoke :add 2481 INFO [2022-01-10 08:39:19,591] jepsen worker 755 - jepsen.util 755 :invoke :add 2482 INFO [2022-01-10 08:39:19,594] jepsen worker 938 - jepsen.util 938 :invoke :add 2483 INFO [2022-01-10 08:39:19,596] jepsen worker 308 - jepsen.util 308 :invoke :add 2484 INFO [2022-01-10 08:39:19,596] jepsen worker 318 - jepsen.util 318 :ok :add 2395 INFO [2022-01-10 08:39:19,597] jepsen worker 75 - jepsen.util 75 :ok :add 2473 INFO [2022-01-10 08:39:19,598] jepsen worker 670 - jepsen.util 670 :invoke :add 2485 INFO [2022-01-10 08:39:19,599] jepsen worker 942 - jepsen.util 942 :ok :add 2431 INFO [2022-01-10 08:39:19,599] jepsen worker 831 - jepsen.util 831 :ok :add 2405 INFO [2022-01-10 08:39:19,600] jepsen worker 326 - jepsen.util 326 :ok :add 2429 INFO [2022-01-10 08:39:19,600] jepsen worker 628 - jepsen.util 628 :ok :add 2479 INFO [2022-01-10 08:39:19,602] jepsen worker 886 - jepsen.util 886 :invoke :add 2486 INFO [2022-01-10 08:39:19,605] jepsen worker 926 - jepsen.util 926 :invoke :add 2487 INFO [2022-01-10 08:39:19,607] jepsen worker 98 - jepsen.util 98 :invoke :add 2488 INFO [2022-01-10 08:39:19,608] jepsen worker 54 - jepsen.util 54 :ok :add 2478 INFO [2022-01-10 08:39:19,610] jepsen worker 334 - jepsen.util 334 :invoke :add 2489 INFO [2022-01-10 08:39:19,611] jepsen worker 670 - jepsen.util 670 :ok :add 2485 INFO [2022-01-10 08:39:19,611] jepsen worker 308 - jepsen.util 308 :ok :add 2484 INFO [2022-01-10 08:39:19,612] jepsen worker 891 - jepsen.util 891 :ok :add 2480 INFO [2022-01-10 08:39:19,613] jepsen worker 566 - jepsen.util 566 :invoke :add 2490 INFO [2022-01-10 08:39:19,615] jepsen worker 245 - jepsen.util 245 :invoke :add 2491 INFO [2022-01-10 08:39:19,617] jepsen worker 755 - jepsen.util 755 :ok :add 2482 INFO [2022-01-10 08:39:19,617] jepsen worker 682 - jepsen.util 682 :ok :add 2481 INFO [2022-01-10 08:39:19,617] jepsen worker 886 - jepsen.util 886 :ok :add 2486 INFO [2022-01-10 08:39:19,618] jepsen worker 39 - jepsen.util 39 :invoke :add 2492 INFO [2022-01-10 08:39:19,618] jepsen worker 926 - jepsen.util 926 :ok :add 2487 INFO [2022-01-10 08:39:19,621] jepsen worker 938 - jepsen.util 938 :ok :add 2483 INFO [2022-01-10 08:39:19,621] jepsen worker 955 - jepsen.util 955 :invoke :add 2493 INFO [2022-01-10 08:39:19,624] jepsen worker 436 - jepsen.util 436 :invoke :add 2494 INFO [2022-01-10 08:39:19,627] jepsen worker 933 - jepsen.util 933 :invoke :add 2495 INFO [2022-01-10 08:39:19,630] jepsen worker 940 - jepsen.util 940 :invoke :add 2496 INFO [2022-01-10 08:39:19,632] jepsen worker 711 - jepsen.util 711 :invoke :add 2497 INFO [2022-01-10 08:39:19,635] jepsen worker 245 - jepsen.util 245 :ok :add 2491 INFO [2022-01-10 08:39:19,635] jepsen worker 739 - jepsen.util 739 :invoke :add 2498 INFO [2022-01-10 08:39:19,635] jepsen worker 334 - jepsen.util 334 :ok :add 2489 INFO [2022-01-10 08:39:19,639] jepsen worker 119 - jepsen.util 119 :invoke :add 2499 INFO [2022-01-10 08:39:19,639] jepsen worker 548 - jepsen.util 548 :ok :add 2397 INFO [2022-01-10 08:39:19,642] jepsen worker 959 - jepsen.util 959 :invoke :add 2500 INFO [2022-01-10 08:39:19,643] jepsen worker 739 - jepsen.util 739 :ok :add 2498 INFO [2022-01-10 08:39:19,646] jepsen worker 858 - jepsen.util 858 :invoke :add 2501 INFO [2022-01-10 08:39:19,649] jepsen worker 703 - jepsen.util 703 :invoke :add 2502 INFO [2022-01-10 08:39:19,651] jepsen worker 50 - jepsen.util 50 :ok :add 2458 INFO [2022-01-10 08:39:19,651] jepsen worker 278 - jepsen.util 278 :ok :add 2440 INFO [2022-01-10 08:39:19,652] jepsen worker 268 - jepsen.util 268 :ok :add 2474 INFO [2022-01-10 08:39:19,652] jepsen worker 212 - jepsen.util 212 :invoke :add 2503 INFO [2022-01-10 08:39:19,652] jepsen worker 119 - jepsen.util 119 :ok :add 2499 INFO [2022-01-10 08:39:19,654] jepsen worker 955 - jepsen.util 955 :ok :add 2493 INFO [2022-01-10 08:39:19,656] jepsen worker 364 - jepsen.util 364 :invoke :add 2504 INFO [2022-01-10 08:39:19,656] jepsen worker 657 - jepsen.util 657 :ok :add 2442 INFO [2022-01-10 08:39:19,656] jepsen worker 155 - jepsen.util 155 :ok :add 2476 INFO [2022-01-10 08:39:19,656] jepsen worker 98 - jepsen.util 98 :ok :add 2488 INFO [2022-01-10 08:39:19,657] jepsen worker 566 - jepsen.util 566 :ok :add 2490 INFO [2022-01-10 08:39:19,659] jepsen worker 163 - jepsen.util 163 :ok :add 2432 INFO [2022-01-10 08:39:19,660] jepsen worker 522 - jepsen.util 522 :invoke :add 2505 INFO [2022-01-10 08:39:19,663] jepsen worker 433 - jepsen.util 433 :invoke :add 2506 INFO [2022-01-10 08:39:19,665] jepsen worker 461 - jepsen.util 461 :invoke :add 2507 INFO [2022-01-10 08:39:19,668] jepsen worker 493 - jepsen.util 493 :invoke :add 2508 INFO [2022-01-10 08:39:19,670] jepsen worker 440 - jepsen.util 440 :invoke :add 2509 INFO [2022-01-10 08:39:19,672] jepsen worker 360 - jepsen.util 360 :ok :add 2441 INFO [2022-01-10 08:39:19,672] jepsen worker 552 - jepsen.util 552 :ok :add 2475 INFO [2022-01-10 08:39:19,672] jepsen worker 224 - jepsen.util 224 :ok :add 2459 INFO [2022-01-10 08:39:19,673] jepsen worker 159 - jepsen.util 159 :invoke :add 2510 INFO [2022-01-10 08:39:19,676] jepsen worker 493 - jepsen.util 493 :ok :add 2508 INFO [2022-01-10 08:39:19,676] jepsen worker 45 - jepsen.util 45 :invoke :add 2511 INFO [2022-01-10 08:39:19,677] jepsen worker 675 - jepsen.util 675 :ok :add 2443 INFO [2022-01-10 08:39:19,677] jepsen worker 544 - jepsen.util 544 :ok :add 2477 INFO [2022-01-10 08:39:19,679] jepsen worker 864 - jepsen.util 864 :invoke :add 2512 INFO [2022-01-10 08:39:19,680] jepsen worker 440 - jepsen.util 440 :ok :add 2509 INFO [2022-01-10 08:39:19,682] jepsen worker 215 - jepsen.util 215 :invoke :add 2513 INFO [2022-01-10 08:39:19,685] jepsen worker 122 - jepsen.util 122 :invoke :add 2514 INFO [2022-01-10 08:39:19,687] jepsen worker 65 - jepsen.util 65 :invoke :add 2515 INFO [2022-01-10 08:39:19,689] jepsen worker 45 - jepsen.util 45 :ok :add 2511 INFO [2022-01-10 08:39:19,690] jepsen worker 276 - jepsen.util 276 :invoke :add 2516 INFO [2022-01-10 08:39:19,690] jepsen worker 159 - jepsen.util 159 :ok :add 2510 INFO [2022-01-10 08:39:19,692] jepsen worker 765 - jepsen.util 765 :invoke :add 2517 INFO [2022-01-10 08:39:19,696] jepsen worker 864 - jepsen.util 864 :ok :add 2512 INFO [2022-01-10 08:39:19,696] jepsen worker 381 - jepsen.util 381 :invoke :add 2518 INFO [2022-01-10 08:39:19,699] jepsen worker 390 - jepsen.util 390 :invoke :add 2519 INFO [2022-01-10 08:39:19,699] jepsen worker 215 - jepsen.util 215 :ok :add 2513 INFO [2022-01-10 08:39:19,700] jepsen worker 39 - jepsen.util 39 :ok :add 2492 INFO [2022-01-10 08:39:19,700] jepsen worker 703 - jepsen.util 703 :ok :add 2502 INFO [2022-01-10 08:39:19,702] jepsen worker 58 - jepsen.util 58 :invoke :add 2520 INFO [2022-01-10 08:39:19,705] jepsen worker 65 - jepsen.util 65 :ok :add 2515 INFO [2022-01-10 08:39:19,706] jepsen worker 277 - jepsen.util 277 :invoke :add 2521 INFO [2022-01-10 08:39:19,707] jepsen worker 122 - jepsen.util 122 :ok :add 2514 INFO [2022-01-10 08:39:19,708] jepsen worker 765 - jepsen.util 765 :ok :add 2517 INFO [2022-01-10 08:39:19,709] jepsen worker 537 - jepsen.util 537 :invoke :add 2522 INFO [2022-01-10 08:39:19,711] jepsen worker 858 - jepsen.util 858 :ok :add 2501 INFO [2022-01-10 08:39:19,712] jepsen worker 939 - jepsen.util 939 :invoke :add 2523 INFO [2022-01-10 08:39:19,713] jepsen worker 276 - jepsen.util 276 :ok :add 2516 INFO [2022-01-10 08:39:19,713] jepsen worker 277 - jepsen.util 277 :ok :add 2521 INFO [2022-01-10 08:39:19,715] jepsen worker 959 - jepsen.util 959 :ok :add 2500 INFO [2022-01-10 08:39:19,715] jepsen worker 292 - jepsen.util 292 :invoke :add 2524 INFO [2022-01-10 08:39:19,715] jepsen worker 381 - jepsen.util 381 :ok :add 2518 INFO [2022-01-10 08:39:19,719] jepsen worker 939 - jepsen.util 939 :ok :add 2523 INFO [2022-01-10 08:39:19,719] jepsen worker 833 - jepsen.util 833 :invoke :add 2525 INFO [2022-01-10 08:39:19,722] jepsen worker 697 - jepsen.util 697 :invoke :add 2526 INFO [2022-01-10 08:39:19,724] jepsen worker 292 - jepsen.util 292 :ok :add 2524 INFO [2022-01-10 08:39:19,725] jepsen worker 561 - jepsen.util 561 :invoke :add 2527 INFO [2022-01-10 08:39:19,727] jepsen worker 58 - jepsen.util 58 :ok :add 2520 INFO [2022-01-10 08:39:19,728] jepsen worker 767 - jepsen.util 767 :invoke :add 2528 INFO [2022-01-10 08:39:19,731] jepsen worker 212 - jepsen.util 212 :ok :add 2503 INFO [2022-01-10 08:39:19,731] jepsen worker 689 - jepsen.util 689 :invoke :add 2529 INFO [2022-01-10 08:39:19,732] jepsen worker 833 - jepsen.util 833 :ok :add 2525 INFO [2022-01-10 08:39:19,732] jepsen worker 537 - jepsen.util 537 :ok :add 2522 INFO [2022-01-10 08:39:19,736] jepsen worker 108 - jepsen.util 108 :invoke :add 2530 INFO [2022-01-10 08:39:19,738] jepsen worker 561 - jepsen.util 561 :ok :add 2527 INFO [2022-01-10 08:39:19,738] jepsen worker 697 - jepsen.util 697 :ok :add 2526 INFO [2022-01-10 08:39:19,739] jepsen worker 25 - jepsen.util 25 :invoke :add 2531 INFO [2022-01-10 08:39:19,742] jepsen worker 641 - jepsen.util 641 :invoke :add 2532 INFO [2022-01-10 08:39:19,745] jepsen worker 378 - jepsen.util 378 :invoke :add 2533 INFO [2022-01-10 08:39:19,747] jepsen worker 432 - jepsen.util 432 :invoke :add 2534 INFO [2022-01-10 08:39:19,750] jepsen worker 241 - jepsen.util 241 :invoke :add 2535 INFO [2022-01-10 08:39:19,753] jepsen worker 300 - jepsen.util 300 :invoke :add 2536 INFO [2022-01-10 08:39:19,755] jepsen worker 933 - jepsen.util 933 :ok :add 2495 INFO [2022-01-10 08:39:19,755] jepsen worker 522 - jepsen.util 522 :ok :add 2505 INFO [2022-01-10 08:39:19,756] jepsen worker 289 - jepsen.util 289 :invoke :add 2537 INFO [2022-01-10 08:39:19,759] jepsen worker 374 - jepsen.util 374 :invoke :add 2538 INFO [2022-01-10 08:39:19,762] jepsen worker 770 - jepsen.util 770 :invoke :add 2539 INFO [2022-01-10 08:39:19,764] jepsen worker 335 - jepsen.util 335 :invoke :add 2540 INFO [2022-01-10 08:39:19,767] jepsen worker 12 - jepsen.util 12 :invoke :add 2541 INFO [2022-01-10 08:39:19,767] jepsen worker 374 - jepsen.util 374 :ok :add 2538 INFO [2022-01-10 08:39:19,769] jepsen worker 770 - jepsen.util 770 :ok :add 2539 INFO [2022-01-10 08:39:19,770] jepsen worker 536 - jepsen.util 536 :invoke :add 2542 INFO [2022-01-10 08:39:19,773] jepsen worker 490 - jepsen.util 490 :invoke :add 2543 INFO [2022-01-10 08:39:19,775] jepsen worker 438 - jepsen.util 438 :invoke :add 2544 INFO [2022-01-10 08:39:19,778] jepsen worker 270 - jepsen.util 270 :invoke :add 2545 INFO [2022-01-10 08:39:19,780] jepsen worker 418 - jepsen.util 418 :invoke :add 2546 INFO [2022-01-10 08:39:19,782] jepsen worker 820 - jepsen.util 820 :invoke :add 2547 INFO [2022-01-10 08:39:19,785] jepsen worker 244 - jepsen.util 244 :invoke :add 2548 INFO [2022-01-10 08:39:19,787] jepsen worker 432 - jepsen.util 432 :ok :add 2534 INFO [2022-01-10 08:39:19,789] jepsen worker 742 - jepsen.util 742 :invoke :add 2549 INFO [2022-01-10 08:39:19,790] jepsen worker 711 - jepsen.util 711 :ok :add 2497 INFO [2022-01-10 08:39:19,790] jepsen worker 378 - jepsen.util 378 :ok :add 2533 INFO [2022-01-10 08:39:19,790] jepsen worker 461 - jepsen.util 461 :ok :add 2507 INFO [2022-01-10 08:39:19,792] jepsen worker 21 - jepsen.util 21 :invoke :add 2550 INFO [2022-01-10 08:39:19,796] jepsen worker 929 - jepsen.util 929 :invoke :add 2551 INFO [2022-01-10 08:39:19,799] jepsen worker 537 - jepsen.util 537 :invoke :add 2552 INFO [2022-01-10 08:39:19,800] jepsen worker 300 - jepsen.util 300 :ok :add 2536 INFO [2022-01-10 08:39:19,802] jepsen worker 511 - jepsen.util 511 :invoke :add 2553 INFO [2022-01-10 08:39:19,804] jepsen worker 270 - jepsen.util 270 :ok :add 2545 INFO [2022-01-10 08:39:19,805] jepsen worker 44 - jepsen.util 44 :invoke :add 2554 INFO [2022-01-10 08:39:19,808] jepsen worker 390 - jepsen.util 390 :ok :add 2519 INFO [2022-01-10 08:39:19,808] jepsen worker 241 - jepsen.util 241 :ok :add 2535 INFO [2022-01-10 08:39:19,808] jepsen worker 891 - jepsen.util 891 :invoke :add 2555 INFO [2022-01-10 08:39:19,809] jepsen worker 820 - jepsen.util 820 :ok :add 2547 INFO [2022-01-10 08:39:19,811] jepsen worker 146 - jepsen.util 146 :invoke :add 2556 INFO [2022-01-10 08:39:19,814] jepsen worker 471 - jepsen.util 471 :invoke :add 2557 INFO [2022-01-10 08:39:19,816] jepsen worker 891 - jepsen.util 891 :ok :add 2555 INFO [2022-01-10 08:39:19,816] jepsen worker 987 - jepsen.util 987 :invoke :add 2558 INFO [2022-01-10 08:39:19,818] jepsen worker 44 - jepsen.util 44 :ok :add 2554 INFO [2022-01-10 08:39:19,820] jepsen worker 57 - jepsen.util 57 :invoke :add 2559 INFO [2022-01-10 08:39:19,824] jepsen worker 367 - jepsen.util 367 :invoke :add 2560 INFO [2022-01-10 08:39:19,827] jepsen worker 907 - jepsen.util 907 :invoke :add 2561 INFO [2022-01-10 08:39:19,830] jepsen worker 63 - jepsen.util 63 :invoke :add 2562 INFO [2022-01-10 08:39:19,830] jepsen worker 471 - jepsen.util 471 :ok :add 2557 INFO [2022-01-10 08:39:19,832] jepsen worker 81 - jepsen.util 81 :invoke :add 2563 INFO [2022-01-10 08:39:19,833] jepsen worker 146 - jepsen.util 146 :ok :add 2556 INFO [2022-01-10 08:39:19,835] jepsen worker 486 - jepsen.util 486 :invoke :add 2564 INFO [2022-01-10 08:39:19,836] jepsen worker 907 - jepsen.util 907 :ok :add 2561 INFO [2022-01-10 08:39:19,838] jepsen worker 967 - jepsen.util 967 :invoke :add 2565 INFO [2022-01-10 08:39:19,841] jepsen worker 580 - jepsen.util 580 :invoke :add 2566 INFO [2022-01-10 08:39:19,842] jepsen worker 81 - jepsen.util 81 :ok :add 2563 INFO [2022-01-10 08:39:19,842] jepsen worker 367 - jepsen.util 367 :ok :add 2560 INFO [2022-01-10 08:39:19,844] jepsen worker 379 - jepsen.util 379 :invoke :add 2567 INFO [2022-01-10 08:39:19,847] jepsen worker 844 - jepsen.util 844 :invoke :add 2568 INFO [2022-01-10 08:39:19,849] jepsen worker 411 - jepsen.util 411 :invoke :add 2569 INFO [2022-01-10 08:39:19,851] jepsen worker 931 - jepsen.util 931 :invoke :add 2570 INFO [2022-01-10 08:39:19,854] jepsen worker 121 - jepsen.util 121 :invoke :add 2571 INFO [2022-01-10 08:39:19,856] jepsen worker 399 - jepsen.util 399 :invoke :add 2572 INFO [2022-01-10 08:39:19,857] jepsen worker 63 - jepsen.util 63 :ok :add 2562 INFO [2022-01-10 08:39:19,857] jepsen worker 108 - jepsen.util 108 :ok :add 2530 INFO [2022-01-10 08:39:19,857] jepsen worker 364 - jepsen.util 364 :ok :add 2504 INFO [2022-01-10 08:39:19,857] jepsen worker 767 - jepsen.util 767 :ok :add 2528 INFO [2022-01-10 08:39:19,857] jepsen worker 436 - jepsen.util 436 :ok :add 2494 INFO [2022-01-10 08:39:19,859] jepsen worker 487 - jepsen.util 487 :invoke :add 2573 INFO [2022-01-10 08:39:19,865] jepsen worker 319 - jepsen.util 319 :invoke :add 2574 INFO [2022-01-10 08:39:19,867] jepsen worker 608 - jepsen.util 608 :invoke :add 2575 INFO [2022-01-10 08:39:19,869] jepsen worker 919 - jepsen.util 919 :invoke :add 2576 INFO [2022-01-10 08:39:19,872] jepsen worker 564 - jepsen.util 564 :invoke :add 2577 INFO [2022-01-10 08:39:19,875] jepsen worker 612 - jepsen.util 612 :invoke :add 2578 INFO [2022-01-10 08:39:19,879] jepsen worker 271 - jepsen.util 271 :invoke :add 2579 INFO [2022-01-10 08:39:19,881] jepsen worker 289 - jepsen.util 289 :ok :add 2537 INFO [2022-01-10 08:39:19,882] jepsen worker 371 - jepsen.util 371 :invoke :add 2580 INFO [2022-01-10 08:39:19,885] jepsen worker 612 - jepsen.util 612 :ok :add 2578 INFO [2022-01-10 08:39:19,885] jepsen worker 271 - jepsen.util 271 :ok :add 2579 INFO [2022-01-10 08:39:19,886] jepsen worker 776 - jepsen.util 776 :invoke :add 2581 INFO [2022-01-10 08:39:19,890] jepsen worker 371 - jepsen.util 371 :ok :add 2580 INFO [2022-01-10 08:39:19,891] jepsen worker 544 - jepsen.util 544 :invoke :add 2582 INFO [2022-01-10 08:39:19,894] jepsen worker 767 - jepsen.util 767 :invoke :add 2583 INFO [2022-01-10 08:39:19,894] jepsen worker 411 - jepsen.util 411 :ok :add 2569 INFO [2022-01-10 08:39:19,895] jepsen worker 121 - jepsen.util 121 :ok :add 2571 INFO [2022-01-10 08:39:19,896] jepsen worker 341 - jepsen.util 341 :invoke :add 2584 INFO [2022-01-10 08:39:19,898] jepsen worker 438 - jepsen.util 438 :ok :add 2544 INFO [2022-01-10 08:39:19,899] jepsen worker 844 - jepsen.util 844 :ok :add 2568 INFO [2022-01-10 08:39:19,899] jepsen worker 931 - jepsen.util 931 :ok :add 2570 INFO [2022-01-10 08:39:19,900] jepsen worker 766 - jepsen.util 766 :invoke :add 2585 INFO [2022-01-10 08:39:19,901] jepsen worker 776 - jepsen.util 776 :ok :add 2581 INFO [2022-01-10 08:39:19,904] jepsen worker 912 - jepsen.util 912 :invoke :add 2586 INFO [2022-01-10 08:39:19,906] jepsen worker 940 - jepsen.util 940 :ok :add 2496 INFO [2022-01-10 08:39:19,906] jepsen worker 433 - jepsen.util 433 :ok :add 2506 INFO [2022-01-10 08:39:19,906] jepsen worker 641 - jepsen.util 641 :ok :add 2532 INFO [2022-01-10 08:39:19,908] jepsen worker 700 - jepsen.util 700 :invoke :add 2587 INFO [2022-01-10 08:39:19,908] jepsen worker 544 - jepsen.util 544 :ok :add 2582 INFO [2022-01-10 08:39:19,909] jepsen worker 487 - jepsen.util 487 :ok :add 2573 INFO [2022-01-10 08:39:19,910] jepsen worker 25 - jepsen.util 25 :ok :add 2531 INFO [2022-01-10 08:39:19,910] jepsen worker 689 - jepsen.util 689 :ok :add 2529 INFO [2022-01-10 08:39:19,911] jepsen worker 2 - jepsen.util 2 :invoke :add 2588 INFO [2022-01-10 08:39:19,912] jepsen worker 418 - jepsen.util 418 :ok :add 2546 INFO [2022-01-10 08:39:19,912] jepsen worker 767 - jepsen.util 767 :ok :add 2583 INFO [2022-01-10 08:39:19,912] jepsen worker 12 - jepsen.util 12 :ok :add 2541 INFO [2022-01-10 08:39:19,912] jepsen worker 608 - jepsen.util 608 :ok :add 2575 INFO [2022-01-10 08:39:19,912] jepsen worker 57 - jepsen.util 57 :ok :add 2559 INFO [2022-01-10 08:39:19,914] jepsen worker 399 - jepsen.util 399 :ok :add 2572 INFO [2022-01-10 08:39:19,917] jepsen worker 125 - jepsen.util 125 :invoke :add 2589 INFO [2022-01-10 08:39:19,919] jepsen worker 559 - jepsen.util 559 :invoke :add 2590 INFO [2022-01-10 08:39:19,920] jepsen worker 335 - jepsen.util 335 :ok :add 2540 INFO [2022-01-10 08:39:19,921] jepsen worker 319 - jepsen.util 319 :ok :add 2574 INFO [2022-01-10 08:39:19,921] jepsen worker 987 - jepsen.util 987 :ok :add 2558 INFO [2022-01-10 08:39:19,922] jepsen worker 666 - jepsen.util 666 :invoke :add 2591 INFO [2022-01-10 08:39:19,926] jepsen worker 933 - jepsen.util 933 :invoke :add 2592 INFO [2022-01-10 08:39:19,927] jepsen worker 536 - jepsen.util 536 :ok :add 2542 INFO [2022-01-10 08:39:19,927] jepsen worker 919 - jepsen.util 919 :ok :add 2576 INFO [2022-01-10 08:39:19,931] jepsen worker 118 - jepsen.util 118 :invoke :add 2593 INFO [2022-01-10 08:39:19,934] jepsen worker 779 - jepsen.util 779 :invoke :add 2594 INFO [2022-01-10 08:39:19,935] jepsen worker 564 - jepsen.util 564 :ok :add 2577 INFO [2022-01-10 08:39:19,935] jepsen worker 490 - jepsen.util 490 :ok :add 2543 INFO [2022-01-10 08:39:19,935] jepsen worker 742 - jepsen.util 742 :ok :add 2549 INFO [2022-01-10 08:39:19,935] jepsen worker 967 - jepsen.util 967 :ok :add 2565 INFO [2022-01-10 08:39:19,936] jepsen worker 929 - jepsen.util 929 :ok :add 2551 INFO [2022-01-10 08:39:19,937] jepsen worker 364 - jepsen.util 364 :invoke :add 2595 INFO [2022-01-10 08:39:19,941] jepsen worker 618 - jepsen.util 618 :invoke :add 2596 INFO [2022-01-10 08:39:19,945] jepsen worker 509 - jepsen.util 509 :invoke :add 2597 INFO [2022-01-10 08:39:19,948] jepsen worker 365 - jepsen.util 365 :invoke :add 2598 INFO [2022-01-10 08:39:19,951] jepsen worker 802 - jepsen.util 802 :invoke :add 2599 INFO [2022-01-10 08:39:19,953] jepsen worker 112 - jepsen.util 112 :invoke :add 2600 INFO [2022-01-10 08:39:19,955] jepsen worker 78 - jepsen.util 78 :invoke :add 2601 INFO [2022-01-10 08:39:19,957] jepsen worker 365 - jepsen.util 365 :ok :add 2598 INFO [2022-01-10 08:39:19,958] jepsen worker 27 - jepsen.util 27 :invoke :add 2602 INFO [2022-01-10 08:39:19,961] jepsen worker 553 - jepsen.util 553 :invoke :add 2603 INFO [2022-01-10 08:39:19,963] jepsen worker 386 - jepsen.util 386 :invoke :add 2604 INFO [2022-01-10 08:39:19,966] jepsen worker 769 - jepsen.util 769 :invoke :add 2605 INFO [2022-01-10 08:39:19,966] jepsen worker 802 - jepsen.util 802 :ok :add 2599 INFO [2022-01-10 08:39:19,969] jepsen worker 285 - jepsen.util 285 :invoke :add 2606 INFO [2022-01-10 08:39:19,972] jepsen worker 78 - jepsen.util 78 :ok :add 2601 INFO [2022-01-10 08:39:19,973] jepsen worker 381 - jepsen.util 381 :invoke :add 2607 INFO [2022-01-10 08:39:19,976] jepsen worker 69 - jepsen.util 69 :invoke :add 2608 INFO [2022-01-10 08:39:19,977] jepsen worker 2 - jepsen.util 2 :ok :add 2588 INFO [2022-01-10 08:39:19,977] jepsen worker 559 - jepsen.util 559 :ok :add 2590 INFO [2022-01-10 08:39:19,978] jepsen worker 172 - jepsen.util 172 :invoke :add 2609 INFO [2022-01-10 08:39:19,981] jepsen worker 130 - jepsen.util 130 :invoke :add 2610 INFO [2022-01-10 08:39:19,984] jepsen worker 548 - jepsen.util 548 :invoke :add 2611 INFO [2022-01-10 08:39:19,986] jepsen worker 875 - jepsen.util 875 :invoke :add 2612 INFO [2022-01-10 08:39:19,989] jepsen worker 332 - jepsen.util 332 :invoke :add 2613 INFO [2022-01-10 08:39:19,991] jepsen worker 553 - jepsen.util 553 :ok :add 2603 INFO [2022-01-10 08:39:19,992] jepsen worker 933 - jepsen.util 933 :ok :add 2592 INFO [2022-01-10 08:39:19,992] jepsen worker 366 - jepsen.util 366 :invoke :add 2614 INFO [2022-01-10 08:39:19,995] jepsen worker 560 - jepsen.util 560 :invoke :add 2615 INFO [2022-01-10 08:39:19,998] jepsen worker 656 - jepsen.util 656 :invoke :add 2616 INFO [2022-01-10 08:39:20,000] jepsen worker 188 - jepsen.util 188 :invoke :add 2617 INFO [2022-01-10 08:39:20,003] jepsen worker 805 - jepsen.util 805 :invoke :add 2618 INFO [2022-01-10 08:39:20,004] jepsen worker 244 - jepsen.util 244 :ok :add 2548 INFO [2022-01-10 08:39:20,004] jepsen worker 486 - jepsen.util 486 :ok :add 2564 INFO [2022-01-10 08:39:20,004] jepsen worker 21 - jepsen.util 21 :ok :add 2550 INFO [2022-01-10 08:39:20,006] jepsen worker 129 - jepsen.util 129 :invoke :add 2619 INFO [2022-01-10 08:39:20,009] jepsen worker 397 - jepsen.util 397 :invoke :add 2620 INFO [2022-01-10 08:39:20,012] jepsen worker 7 - jepsen.util 7 :invoke :add 2621 INFO [2022-01-10 08:39:20,015] jepsen worker 350 - jepsen.util 350 :invoke :add 2622 INFO [2022-01-10 08:39:20,017] jepsen worker 69 - jepsen.util 69 :ok :add 2608 INFO [2022-01-10 08:39:20,017] jepsen worker 130 - jepsen.util 130 :ok :add 2610 INFO [2022-01-10 08:39:20,018] jepsen worker 227 - jepsen.util 227 :invoke :add 2623 INFO [2022-01-10 08:39:20,020] jepsen worker 831 - jepsen.util 831 :invoke :add 2624 INFO [2022-01-10 08:39:20,022] jepsen worker 282 - jepsen.util 282 :invoke :add 2625 INFO [2022-01-10 08:39:20,025] jepsen worker 42 - jepsen.util 42 :invoke :add 2626 INFO [2022-01-10 08:39:20,028] jepsen worker 633 - jepsen.util 633 :invoke :add 2627 INFO [2022-01-10 08:39:20,030] jepsen worker 608 - jepsen.util 608 :invoke :add 2628 INFO [2022-01-10 08:39:20,035] jepsen worker nemesis - jepsen.util :nemesis :info :kill-process nil INFO [2022-01-10 08:39:20,038] jepsen worker 312 - jepsen.util 312 :invoke :add 2629 INFO [2022-01-10 08:39:20,039] jepsen worker 875 - jepsen.util 875 :ok :add 2612 INFO [2022-01-10 08:39:20,040] jepsen worker 548 - jepsen.util 548 :ok :add 2611 INFO [2022-01-10 08:39:20,041] jepsen worker 172 - jepsen.util 172 :ok :add 2609 INFO [2022-01-10 08:39:20,041] jepsen worker 674 - jepsen.util 674 :invoke :add 2630 INFO [2022-01-10 08:39:20,044] jepsen worker 87 - jepsen.util 87 :invoke :add 2631 INFO [2022-01-10 08:39:20,046] jepsen worker 26 - jepsen.util 26 :invoke :add 2632 INFO [2022-01-10 08:39:20,049] jepsen worker 209 - jepsen.util 209 :invoke :add 2633 INFO [2022-01-10 08:39:20,052] jepsen worker 812 - jepsen.util 812 :invoke :add 2634 INFO [2022-01-10 08:39:20,055] jepsen worker 709 - jepsen.util 709 :invoke :add 2635 INFO [2022-01-10 08:39:20,057] jepsen worker 107 - jepsen.util 107 :invoke :add 2636 INFO [2022-01-10 08:39:20,060] jepsen worker 556 - jepsen.util 556 :invoke :add 2637 INFO [2022-01-10 08:39:20,062] jepsen worker 633 - jepsen.util 633 :ok :add 2627 INFO [2022-01-10 08:39:20,062] jepsen worker 332 - jepsen.util 332 :ok :add 2613 INFO [2022-01-10 08:39:20,063] jepsen worker 946 - jepsen.util 946 :invoke :add 2638 INFO [2022-01-10 08:39:20,066] jepsen worker 778 - jepsen.util 778 :invoke :add 2639 INFO [2022-01-10 08:39:20,069] jepsen worker 196 - jepsen.util 196 :invoke :add 2640 INFO [2022-01-10 08:39:20,072] jepsen worker 670 - jepsen.util 670 :invoke :add 2641 INFO [2022-01-10 08:39:20,073] jepsen worker 831 - jepsen.util 831 :ok :add 2624 INFO [2022-01-10 08:39:20,075] jepsen worker 271 - jepsen.util 271 :invoke :add 2642 INFO [2022-01-10 08:39:20,076] jepsen worker 805 - jepsen.util 805 :ok :add 2618 INFO [2022-01-10 08:39:20,076] jepsen worker 112 - jepsen.util 112 :ok :add 2600 INFO [2022-01-10 08:39:20,077] jepsen worker 666 - jepsen.util 666 :ok :add 2591 INFO [2022-01-10 08:39:20,077] jepsen worker 125 - jepsen.util 125 :ok :add 2589 INFO [2022-01-10 08:39:20,078] jepsen worker 42 - jepsen.util 42 :ok :add 2626 INFO [2022-01-10 08:39:20,078] jepsen worker 732 - jepsen.util 732 :invoke :add 2643 INFO [2022-01-10 08:39:20,082] jepsen worker 293 - jepsen.util 293 :invoke :add 2644 INFO [2022-01-10 08:39:20,084] jepsen worker 640 - jepsen.util 640 :invoke :add 2645 INFO [2022-01-10 08:39:20,085] jepsen worker 87 - jepsen.util 87 :ok :add 2631 INFO [2022-01-10 08:39:20,085] jepsen worker 312 - jepsen.util 312 :ok :add 2629 INFO [2022-01-10 08:39:20,086] jepsen worker 769 - jepsen.util 769 :ok :add 2605 INFO [2022-01-10 08:39:20,087] jepsen worker 762 - jepsen.util 762 :invoke :add 2646 INFO [2022-01-10 08:39:20,090] jepsen worker 873 - jepsen.util 873 :invoke :add 2647 INFO [2022-01-10 08:39:20,093] jepsen worker 775 - jepsen.util 775 :invoke :add 2648 INFO [2022-01-10 08:39:20,096] jepsen worker 282 - jepsen.util 282 :ok :add 2625 INFO [2022-01-10 08:39:20,096] jepsen worker 59 - jepsen.util 59 :invoke :add 2649 INFO [2022-01-10 08:39:20,099] jepsen worker 114 - jepsen.util 114 :invoke :add 2650 INFO [2022-01-10 08:39:20,101] jepsen worker 993 - jepsen.util 993 :invoke :add 2651 INFO [2022-01-10 08:39:20,104] jepsen worker 916 - jepsen.util 916 :invoke :add 2652 INFO [2022-01-10 08:39:20,106] jepsen worker 897 - jepsen.util 897 :invoke :add 2653 INFO [2022-01-10 08:39:20,108] jepsen worker 135 - jepsen.util 135 :invoke :add 2654 INFO [2022-01-10 08:39:20,111] jepsen worker 895 - jepsen.util 895 :invoke :add 2655 INFO [2022-01-10 08:39:20,113] jepsen worker 710 - jepsen.util 710 :invoke :add 2656 INFO [2022-01-10 08:39:20,116] jepsen worker 763 - jepsen.util 763 :invoke :add 2657 INFO [2022-01-10 08:39:20,118] jepsen worker 898 - jepsen.util 898 :invoke :add 2658 INFO [2022-01-10 08:39:20,121] jepsen worker 758 - jepsen.util 758 :invoke :add 2659 INFO [2022-01-10 08:39:20,122] jepsen worker 381 - jepsen.util 381 :ok :add 2607 INFO [2022-01-10 08:39:20,122] jepsen worker 209 - jepsen.util 209 :ok :add 2633 INFO [2022-01-10 08:39:20,123] jepsen worker 48 - jepsen.util 48 :invoke :add 2660 INFO [2022-01-10 08:39:20,126] jepsen worker 927 - jepsen.util 927 :invoke :add 2661 INFO [2022-01-10 08:39:20,129] jepsen worker 376 - jepsen.util 376 :invoke :add 2662 INFO [2022-01-10 08:39:20,131] jepsen worker 40 - jepsen.util 40 :invoke :add 2663 INFO [2022-01-10 08:39:20,134] jepsen worker 920 - jepsen.util 920 :invoke :add 2664 INFO [2022-01-10 08:39:20,137] jepsen worker 856 - jepsen.util 856 :invoke :add 2665 INFO [2022-01-10 08:39:20,139] jepsen worker 389 - jepsen.util 389 :invoke :add 2666 INFO [2022-01-10 08:39:20,141] jepsen worker 732 - jepsen.util 732 :ok :add 2643 INFO [2022-01-10 08:39:20,141] jepsen worker 138 - jepsen.util 138 :invoke :add 2667 INFO [2022-01-10 08:39:20,145] jepsen worker 364 - jepsen.util 364 :ok :add 2595 INFO [2022-01-10 08:39:20,145] jepsen worker 848 - jepsen.util 848 :invoke :add 2668 INFO [2022-01-10 08:39:20,147] jepsen worker 201 - jepsen.util 201 :invoke :add 2669 INFO [2022-01-10 08:39:20,150] jepsen worker 547 - jepsen.util 547 :invoke :add 2670 INFO [2022-01-10 08:39:20,153] jepsen worker 896 - jepsen.util 896 :invoke :add 2671 INFO [2022-01-10 08:39:20,156] jepsen worker 629 - jepsen.util 629 :invoke :add 2672 INFO [2022-01-10 08:39:20,158] jepsen worker 256 - jepsen.util 256 :invoke :add 2673 INFO [2022-01-10 08:39:20,160] jepsen worker 705 - jepsen.util 705 :invoke :add 2674 INFO [2022-01-10 08:39:20,163] jepsen worker 263 - jepsen.util 263 :invoke :add 2675 INFO [2022-01-10 08:39:20,166] jepsen worker 726 - jepsen.util 726 :invoke :add 2676 INFO [2022-01-10 08:39:20,168] jepsen worker 716 - jepsen.util 716 :invoke :add 2677 INFO [2022-01-10 08:39:20,170] jepsen worker 509 - jepsen.util 509 :ok :add 2597 INFO [2022-01-10 08:39:20,171] jepsen worker 608 - jepsen.util 608 :ok :add 2628 INFO [2022-01-10 08:39:20,171] jepsen worker 326 - jepsen.util 326 :invoke :add 2678 INFO [2022-01-10 08:39:20,172] jepsen worker 386 - jepsen.util 386 :ok :add 2604 INFO [2022-01-10 08:39:20,172] jepsen worker 674 - jepsen.util 674 :ok :add 2630 INFO [2022-01-10 08:39:20,174] jepsen worker 944 - jepsen.util 944 :invoke :add 2679 INFO [2022-01-10 08:39:20,177] jepsen worker 336 - jepsen.util 336 :invoke :add 2680 INFO [2022-01-10 08:39:20,180] jepsen worker 211 - jepsen.util 211 :invoke :add 2681 INFO [2022-01-10 08:39:20,182] jepsen worker 84 - jepsen.util 84 :invoke :add 2682 INFO [2022-01-10 08:39:20,185] jepsen worker 800 - jepsen.util 800 :invoke :add 2683 INFO [2022-01-10 08:39:20,187] jepsen worker 895 - jepsen.util 895 :ok :add 2655 INFO [2022-01-10 08:39:20,187] jepsen worker 927 - jepsen.util 927 :ok :add 2661 INFO [2022-01-10 08:39:20,187] jepsen worker 132 - jepsen.util 132 :invoke :add 2684 INFO [2022-01-10 08:39:20,190] jepsen worker 786 - jepsen.util 786 :invoke :add 2685 INFO [2022-01-10 08:39:20,193] jepsen worker 93 - jepsen.util 93 :invoke :add 2686 INFO [2022-01-10 08:39:20,197] jepsen worker 981 - jepsen.util 981 :invoke :add 2687 INFO [2022-01-10 08:39:20,197] jepsen worker 763 - jepsen.util 763 :ok :add 2657 INFO [2022-01-10 08:39:20,198] jepsen worker 40 - jepsen.util 40 :ok :add 2663 INFO [2022-01-10 08:39:20,200] jepsen worker 702 - jepsen.util 702 :invoke :add 2688 INFO [2022-01-10 08:39:20,204] jepsen worker 694 - jepsen.util 694 :invoke :add 2689 INFO [2022-01-10 08:39:20,207] jepsen worker 717 - jepsen.util 717 :invoke :add 2690 INFO [2022-01-10 08:39:20,211] jepsen worker 895 - jepsen.util 895 :invoke :add 2691 INFO [2022-01-10 08:39:20,213] jepsen worker 416 - jepsen.util 416 :invoke :add 2692 INFO [2022-01-10 08:39:20,214] jepsen worker 135 - jepsen.util 135 :ok :add 2654 INFO [2022-01-10 08:39:20,214] jepsen worker 48 - jepsen.util 48 :ok :add 2660 INFO [2022-01-10 08:39:20,214] jepsen worker 326 - jepsen.util 326 :ok :add 2678 INFO [2022-01-10 08:39:20,216] jepsen worker 171 - jepsen.util 171 :invoke :add 2693 INFO [2022-01-10 08:39:20,219] jepsen worker 425 - jepsen.util 425 :invoke :add 2694 INFO [2022-01-10 08:39:20,224] jepsen worker 27 - jepsen.util 27 :ok :add 2602 INFO [2022-01-10 08:39:20,224] jepsen worker 107 - jepsen.util 107 :ok :add 2636 INFO [2022-01-10 08:39:20,225] jepsen worker 562 - jepsen.util 562 :invoke :add 2696 INFO [2022-01-10 08:39:20,226] jepsen worker 118 - jepsen.util 118 :ok :add 2593 INFO [2022-01-10 08:39:20,226] jepsen worker 819 - jepsen.util 819 :invoke :add 2695 INFO [2022-01-10 08:39:20,228] jepsen worker 32 - jepsen.util 32 :invoke :add 2697 INFO [2022-01-10 08:39:20,231] jepsen worker 332 - jepsen.util 332 :invoke :add 2698 INFO [2022-01-10 08:39:20,233] jepsen worker 935 - jepsen.util 935 :invoke :add 2699 INFO [2022-01-10 08:39:20,235] jepsen worker 707 - jepsen.util 707 :invoke :add 2700 INFO [2022-01-10 08:39:20,237] jepsen worker 130 - jepsen.util 130 :invoke :add 2701 INFO [2022-01-10 08:39:20,240] jepsen worker 917 - jepsen.util 917 :invoke :add 2702 INFO [2022-01-10 08:39:20,242] jepsen worker 549 - jepsen.util 549 :invoke :add 2703 INFO [2022-01-10 08:39:20,244] jepsen worker 531 - jepsen.util 531 :invoke :add 2704 INFO [2022-01-10 08:39:20,247] jepsen worker 481 - jepsen.util 481 :invoke :add 2705 INFO [2022-01-10 08:39:20,250] jepsen worker 607 - jepsen.util 607 :invoke :add 2706 INFO [2022-01-10 08:39:20,253] jepsen worker 53 - jepsen.util 53 :invoke :add 2707 INFO [2022-01-10 08:39:20,255] jepsen worker 966 - jepsen.util 966 :invoke :add 2708 INFO [2022-01-10 08:39:20,256] jepsen worker 26 - jepsen.util 26 :ok :add 2632 INFO [2022-01-10 08:39:20,256] jepsen worker 285 - jepsen.util 285 :ok :add 2606 INFO [2022-01-10 08:39:20,257] jepsen worker 312 - jepsen.util 312 :invoke :add 2709 INFO [2022-01-10 08:39:20,261] jepsen worker 663 - jepsen.util 663 :invoke :add 2710 INFO [2022-01-10 08:39:20,263] jepsen worker 855 - jepsen.util 855 :invoke :add 2711 INFO [2022-01-10 08:39:20,265] jepsen worker 372 - jepsen.util 372 :invoke :add 2712 INFO [2022-01-10 08:39:20,267] jepsen worker 333 - jepsen.util 333 :invoke :add 2713 INFO [2022-01-10 08:39:20,270] jepsen worker 63 - jepsen.util 63 :invoke :add 2714 INFO [2022-01-10 08:39:20,272] jepsen worker 944 - jepsen.util 944 :ok :add 2679 INFO [2022-01-10 08:39:20,273] jepsen worker 45 - jepsen.util 45 :invoke :add 2715 INFO [2022-01-10 08:39:20,274] jepsen worker 779 - jepsen.util 779 :ok :add 2594 INFO [2022-01-10 08:39:20,276] jepsen worker 610 - jepsen.util 610 :invoke :add 2716 INFO [2022-01-10 08:39:20,279] jepsen worker 245 - jepsen.util 245 :invoke :add 2717 INFO [2022-01-10 08:39:20,281] jepsen worker 280 - jepsen.util 280 :invoke :add 2718 INFO [2022-01-10 08:39:20,284] jepsen worker 750 - jepsen.util 750 :invoke :add 2719 INFO [2022-01-10 08:39:20,285] jepsen worker 710 - jepsen.util 710 :ok :add 2656 INFO [2022-01-10 08:39:20,285] jepsen worker 376 - jepsen.util 376 :ok :add 2662 INFO [2022-01-10 08:39:20,285] jepsen worker 705 - jepsen.util 705 :ok :add 2674 INFO [2022-01-10 08:39:20,285] jepsen worker 196 - jepsen.util 196 :ok :add 2640 INFO [2022-01-10 08:39:20,285] jepsen worker 898 - jepsen.util 898 :ok :add 2658 INFO [2022-01-10 08:39:20,287] jepsen worker 774 - jepsen.util 774 :invoke :add 2720 INFO [2022-01-10 08:39:20,291] jepsen worker 276 - jepsen.util 276 :invoke :add 2721 INFO [2022-01-10 08:39:20,291] jepsen worker 332 - jepsen.util 332 :ok :add 2698 INFO [2022-01-10 08:39:20,292] jepsen worker 336 - jepsen.util 336 :ok :add 2680 INFO [2022-01-10 08:39:20,292] jepsen worker 618 - jepsen.util 618 :ok :add 2596 INFO [2022-01-10 08:39:20,292] jepsen worker 663 - jepsen.util 663 :ok :add 2710 INFO [2022-01-10 08:39:20,293] jepsen worker 966 - jepsen.util 966 :ok :add 2708 INFO [2022-01-10 08:39:20,295] jepsen worker 881 - jepsen.util 881 :invoke :add 2722 INFO [2022-01-10 08:39:20,295] jepsen worker 271 - jepsen.util 271 :ok :add 2642 INFO [2022-01-10 08:39:20,296] jepsen worker 726 - jepsen.util 726 :ok :add 2676 INFO [2022-01-10 08:39:20,299] jepsen worker 107 - jepsen.util 107 :invoke :add 2723 INFO [2022-01-10 08:39:20,301] jepsen worker 886 - jepsen.util 886 :invoke :add 2724 INFO [2022-01-10 08:39:20,303] jepsen worker 823 - jepsen.util 823 :invoke :add 2725 INFO [2022-01-10 08:39:20,305] jepsen worker 918 - jepsen.util 918 :invoke :add 2726 INFO [2022-01-10 08:39:20,308] jepsen worker 49 - jepsen.util 49 :invoke :add 2727 INFO [2022-01-10 08:39:20,310] jepsen worker 858 - jepsen.util 858 :invoke :add 2728 INFO [2022-01-10 08:39:20,313] jepsen worker 417 - jepsen.util 417 :invoke :add 2729 INFO [2022-01-10 08:39:20,315] jepsen worker 874 - jepsen.util 874 :invoke :add 2730 INFO [2022-01-10 08:39:20,317] jepsen worker 816 - jepsen.util 816 :invoke :add 2731 INFO [2022-01-10 08:39:20,320] jepsen worker 377 - jepsen.util 377 :invoke :add 2732 INFO [2022-01-10 08:39:20,323] jepsen worker 644 - jepsen.util 644 :invoke :add 2733 INFO [2022-01-10 08:39:20,326] jepsen worker 402 - jepsen.util 402 :invoke :add 2734 INFO [2022-01-10 08:39:20,328] jepsen worker 258 - jepsen.util 258 :invoke :add 2735 INFO [2022-01-10 08:39:20,331] jepsen worker 795 - jepsen.util 795 :invoke :add 2736 INFO [2022-01-10 08:39:20,333] jepsen worker 755 - jepsen.util 755 :invoke :add 2737 INFO [2022-01-10 08:39:20,335] jepsen worker 113 - jepsen.util 113 :invoke :add 2738 INFO [2022-01-10 08:39:20,337] jepsen worker 326 - jepsen.util 326 :invoke :add 2739 INFO [2022-01-10 08:39:20,340] jepsen worker 303 - jepsen.util 303 :invoke :add 2740 INFO [2022-01-10 08:39:20,342] jepsen worker 396 - jepsen.util 396 :invoke :add 2741 INFO [2022-01-10 08:39:20,344] jepsen worker 188 - jepsen.util 188 :ok :add 2617 INFO [2022-01-10 08:39:20,344] jepsen worker 537 - jepsen.util 537 :ok :add 2552 INFO [2022-01-10 08:39:20,345] jepsen worker 290 - jepsen.util 290 :invoke :add 2742 INFO [2022-01-10 08:39:20,345] jepsen worker 580 - jepsen.util 580 :ok :add 2566 INFO [2022-01-10 08:39:20,348] jepsen worker 909 - jepsen.util 909 :invoke :add 2743 INFO [2022-01-10 08:39:20,350] jepsen worker 517 - jepsen.util 517 :invoke :add 2744 INFO [2022-01-10 08:39:20,353] jepsen worker 8 - jepsen.util 8 :invoke :add 2745 INFO [2022-01-10 08:39:20,355] jepsen worker 567 - jepsen.util 567 :invoke :add 2746 INFO [2022-01-10 08:39:20,357] jepsen worker 615 - jepsen.util 615 :invoke :add 2747 INFO [2022-01-10 08:39:20,360] jepsen worker 544 - jepsen.util 544 :invoke :add 2748 INFO [2022-01-10 08:39:20,362] jepsen worker 315 - jepsen.util 315 :invoke :add 2749 INFO [2022-01-10 08:39:20,364] jepsen worker 794 - jepsen.util 794 :invoke :add 2750 INFO [2022-01-10 08:39:20,367] jepsen worker 808 - jepsen.util 808 :invoke :add 2751 INFO [2022-01-10 08:39:20,370] jepsen worker 735 - jepsen.util 735 :invoke :add 2752 INFO [2022-01-10 08:39:20,371] jepsen worker 758 - jepsen.util 758 :ok :add 2659 INFO [2022-01-10 08:39:20,371] jepsen worker 670 - jepsen.util 670 :ok :add 2641 INFO [2022-01-10 08:39:20,371] jepsen worker 263 - jepsen.util 263 :ok :add 2675 INFO [2022-01-10 08:39:20,374] jepsen worker 813 - jepsen.util 813 :invoke :add 2753 INFO [2022-01-10 08:39:20,378] jepsen worker 740 - jepsen.util 740 :invoke :add 2754 INFO [2022-01-10 08:39:20,382] jepsen worker 239 - jepsen.util 239 :invoke :add 2755 INFO [2022-01-10 08:39:20,386] jepsen worker 288 - jepsen.util 288 :invoke :add 2756 INFO [2022-01-10 08:39:20,387] jepsen worker 303 - jepsen.util 303 :ok :add 2740 INFO [2022-01-10 08:39:20,389] jepsen worker 608 - jepsen.util 608 :invoke :add 2757 INFO [2022-01-10 08:39:20,393] jepsen worker 972 - jepsen.util 972 :invoke :add 2758 INFO [2022-01-10 08:39:20,396] jepsen worker 825 - jepsen.util 825 :invoke :add 2759 INFO [2022-01-10 08:39:20,398] jepsen worker 986 - jepsen.util 986 :invoke :add 2760 INFO [2022-01-10 08:39:20,401] jepsen worker 898 - jepsen.util 898 :invoke :add 2761 INFO [2022-01-10 08:39:20,405] jepsen worker 831 - jepsen.util 831 :invoke :add 2762 INFO [2022-01-10 08:39:20,407] jepsen worker 785 - jepsen.util 785 :invoke :add 2763 INFO [2022-01-10 08:39:20,411] jepsen worker 290 - jepsen.util 290 :ok :add 2742 INFO [2022-01-10 08:39:20,412] jepsen worker 708 - jepsen.util 708 :invoke :add 2764 INFO [2022-01-10 08:39:20,415] jepsen worker 862 - jepsen.util 862 :invoke :add 2765 INFO [2022-01-10 08:39:20,416] jepsen worker 84 - jepsen.util 84 :ok :add 2682 INFO [2022-01-10 08:39:20,416] jepsen worker 372 - jepsen.util 372 :ok :add 2712 INFO [2022-01-10 08:39:20,416] jepsen worker 935 - jepsen.util 935 :ok :add 2699 INFO [2022-01-10 08:39:20,416] jepsen worker 312 - jepsen.util 312 :ok :add 2709 INFO [2022-01-10 08:39:20,416] jepsen worker 855 - jepsen.util 855 :ok :add 2711 INFO [2022-01-10 08:39:20,417] jepsen worker 211 - jepsen.util 211 :ok :add 2681 INFO [2022-01-10 08:39:20,418] jepsen worker 352 - jepsen.util 352 :invoke :add 2766 INFO [2022-01-10 08:39:20,421] jepsen worker 350 - jepsen.util 350 :ok :add 2622 INFO [2022-01-10 08:39:20,421] jepsen worker 656 - jepsen.util 656 :ok :add 2616 INFO [2022-01-10 08:39:20,421] jepsen worker 379 - jepsen.util 379 :ok :add 2567 INFO [2022-01-10 08:39:20,421] jepsen worker 511 - jepsen.util 511 :ok :add 2553 INFO [2022-01-10 08:39:20,423] jepsen worker 756 - jepsen.util 756 :invoke :add 2767 INFO [2022-01-10 08:39:20,424] jepsen worker 49 - jepsen.util 49 :ok :add 2727 INFO [2022-01-10 08:39:20,424] jepsen worker 800 - jepsen.util 800 :ok :add 2683 INFO [2022-01-10 08:39:20,425] jepsen worker 333 - jepsen.util 333 :ok :add 2713 INFO [2022-01-10 08:39:20,430] jepsen worker 987 - jepsen.util 987 :invoke :add 2768 INFO [2022-01-10 08:39:20,432] jepsen worker 766 - jepsen.util 766 :ok :add 2585 INFO [2022-01-10 08:39:20,434] jepsen worker 525 - jepsen.util 525 :invoke :add 2769 INFO [2022-01-10 08:39:20,438] jepsen worker 575 - jepsen.util 575 :invoke :add 2770 INFO [2022-01-10 08:39:20,441] jepsen worker 561 - jepsen.util 561 :invoke :add 2771 INFO [2022-01-10 08:39:20,442] jepsen worker 700 - jepsen.util 700 :ok :add 2587 INFO [2022-01-10 08:39:20,442] jepsen worker 735 - jepsen.util 735 :ok :add 2752 INFO [2022-01-10 08:39:20,443] jepsen worker 716 - jepsen.util 716 :ok :add 2677 INFO [2022-01-10 08:39:20,443] jepsen worker 341 - jepsen.util 341 :ok :add 2584 INFO [2022-01-10 08:39:20,444] jepsen worker 988 - jepsen.util 988 :invoke :add 2772 INFO [2022-01-10 08:39:20,444] jepsen worker 808 - jepsen.util 808 :ok :add 2751 INFO [2022-01-10 08:39:20,449] jepsen worker 924 - jepsen.util 924 :invoke :add 2773 INFO [2022-01-10 08:39:20,449] jepsen worker 912 - jepsen.util 912 :ok :add 2586 INFO [2022-01-10 08:39:20,450] jepsen worker 560 - jepsen.util 560 :ok :add 2615 INFO [2022-01-10 08:39:20,450] jepsen worker 7 - jepsen.util 7 :ok :add 2621 INFO [2022-01-10 08:39:20,450] jepsen worker 778 - jepsen.util 778 :ok :add 2639 INFO [2022-01-10 08:39:20,452] jepsen worker 109 - jepsen.util 109 :invoke :add 2774 INFO [2022-01-10 08:39:20,456] jepsen worker 227 - jepsen.util 227 :ok :add 2623 INFO [2022-01-10 08:39:20,456] jepsen worker 117 - jepsen.util 117 :invoke :add 2775 INFO [2022-01-10 08:39:20,460] jepsen worker 758 - jepsen.util 758 :invoke :add 2776 INFO [2022-01-10 08:39:20,462] jepsen worker 920 - jepsen.util 920 :ok :add 2664 INFO [2022-01-10 08:39:20,462] jepsen worker 114 - jepsen.util 114 :ok :add 2650 INFO [2022-01-10 08:39:20,462] jepsen worker 775 - jepsen.util 775 :ok :add 2648 INFO [2022-01-10 08:39:20,463] jepsen worker 840 - jepsen.util 840 :invoke :add 2777 INFO [2022-01-10 08:39:20,463] jepsen worker 366 - jepsen.util 366 :ok :add 2614 INFO [2022-01-10 08:39:20,463] jepsen worker 397 - jepsen.util 397 :ok :add 2620 INFO [2022-01-10 08:39:20,464] jepsen worker 946 - jepsen.util 946 :ok :add 2638 INFO [2022-01-10 08:39:20,466] jepsen worker 389 - jepsen.util 389 :ok :add 2666 INFO [2022-01-10 08:39:20,466] jepsen worker 916 - jepsen.util 916 :ok :add 2652 INFO [2022-01-10 08:39:20,467] jepsen worker 224 - jepsen.util 224 :invoke :add 2778 INFO [2022-01-10 08:39:20,470] jepsen worker 992 - jepsen.util 992 :invoke :add 2779 INFO [2022-01-10 08:39:20,473] jepsen worker 936 - jepsen.util 936 :invoke :add 2780 INFO [2022-01-10 08:39:20,475] jepsen worker 787 - jepsen.util 787 :invoke :add 2781 INFO [2022-01-10 08:39:20,477] jepsen worker 880 - jepsen.util 880 :invoke :add 2782 INFO [2022-01-10 08:39:20,479] jepsen worker 235 - jepsen.util 235 :invoke :add 2783 INFO [2022-01-10 08:39:20,481] jepsen worker 863 - jepsen.util 863 :invoke :add 2784 INFO [2022-01-10 08:39:20,485] jepsen worker 953 - jepsen.util 953 :invoke :add 2785 INFO [2022-01-10 08:39:20,488] jepsen worker 382 - jepsen.util 382 :invoke :add 2786 INFO [2022-01-10 08:39:20,490] jepsen worker 786 - jepsen.util 786 :ok :add 2685 INFO [2022-01-10 08:39:20,491] jepsen worker 150 - jepsen.util 150 :invoke :add 2787 INFO [2022-01-10 08:39:20,494] jepsen worker 973 - jepsen.util 973 :invoke :add 2788 INFO [2022-01-10 08:39:20,496] jepsen worker 186 - jepsen.util 186 :invoke :add 2789 INFO [2022-01-10 08:39:20,497] jepsen worker 59 - jepsen.util 59 :ok :add 2649 INFO [2022-01-10 08:39:20,497] jepsen worker 993 - jepsen.util 993 :ok :add 2651 INFO [2022-01-10 08:39:20,497] jepsen worker 107 - jepsen.util 107 :ok :add 2723 INFO [2022-01-10 08:39:20,497] jepsen worker 856 - jepsen.util 856 :ok :add 2665 INFO [2022-01-10 08:39:20,498] jepsen worker 981 - jepsen.util 981 :ok :add 2687 INFO [2022-01-10 08:39:20,498] jepsen worker 245 - jepsen.util 245 :ok :add 2717 INFO [2022-01-10 08:39:20,503] jepsen worker 550 - jepsen.util 550 :invoke :add 2790 INFO [2022-01-10 08:39:20,506] jepsen worker 897 - jepsen.util 897 :ok :add 2653 INFO [2022-01-10 08:39:20,506] jepsen worker 248 - jepsen.util 248 :invoke :add 2791 INFO [2022-01-10 08:39:20,507] jepsen worker 138 - jepsen.util 138 :ok :add 2667 INFO [2022-01-10 08:39:20,511] jepsen worker 794 - jepsen.util 794 :ok :add 2750 INFO [2022-01-10 08:39:20,511] jepsen worker 922 - jepsen.util 922 :invoke :add 2792 INFO [2022-01-10 08:39:20,515] jepsen worker 151 - jepsen.util 151 :invoke :add 2793 INFO [2022-01-10 08:39:20,517] jepsen worker 954 - jepsen.util 954 :invoke :add 2794 INFO [2022-01-10 08:39:20,519] jepsen worker 599 - jepsen.util 599 :invoke :add 2795 INFO [2022-01-10 08:39:20,521] jepsen worker 113 - jepsen.util 113 :ok :add 2738 INFO [2022-01-10 08:39:20,521] jepsen worker 63 - jepsen.util 63 :ok :add 2714 INFO [2022-01-10 08:39:20,521] jepsen worker 132 - jepsen.util 132 :ok :add 2684 INFO [2022-01-10 08:39:20,521] jepsen worker 774 - jepsen.util 774 :ok :add 2720 INFO [2022-01-10 08:39:20,522] jepsen worker 985 - jepsen.util 985 :invoke :add 2796 INFO [2022-01-10 08:39:20,526] jepsen worker 438 - jepsen.util 438 :invoke :add 2797 INFO [2022-01-10 08:39:20,528] jepsen worker 610 - jepsen.util 610 :ok :add 2716 INFO [2022-01-10 08:39:20,528] jepsen worker 881 - jepsen.util 881 :ok :add 2722 INFO [2022-01-10 08:39:20,528] jepsen worker 93 - jepsen.util 93 :ok :add 2686 INFO [2022-01-10 08:39:20,531] jepsen worker 532 - jepsen.util 532 :invoke :add 2798 INFO [2022-01-10 08:39:20,539] jepsen worker 382 - jepsen.util 382 :ok :add 2786 INFO [2022-01-10 08:39:20,540] jepsen worker 150 - jepsen.util 150 :ok :add 2787 INFO [2022-01-10 08:39:20,542] jepsen worker 897 - jepsen.util 897 :invoke :add 2799 INFO [2022-01-10 08:39:20,544] jepsen worker 573 - jepsen.util 573 :invoke :add 2800 INFO [2022-01-10 08:39:20,547] jepsen worker 797 - jepsen.util 797 :invoke :add 2801 INFO [2022-01-10 08:39:20,548] jepsen worker 862 - jepsen.util 862 :ok :add 2765 INFO [2022-01-10 08:39:20,548] jepsen worker 45 - jepsen.util 45 :ok :add 2715 INFO [2022-01-10 08:39:20,549] jepsen worker 771 - jepsen.util 771 :invoke :add 2802 INFO [2022-01-10 08:39:20,549] jepsen worker 276 - jepsen.util 276 :ok :add 2721 INFO [2022-01-10 08:39:20,549] jepsen worker 315 - jepsen.util 315 :ok :add 2749 INFO [2022-01-10 08:39:20,550] jepsen worker 326 - jepsen.util 326 :ok :add 2739 INFO [2022-01-10 08:39:20,554] jepsen worker 176 - jepsen.util 176 :invoke :add 2803 INFO [2022-01-10 08:39:20,556] jepsen worker 813 - jepsen.util 813 :ok :add 2753 INFO [2022-01-10 08:39:20,556] jepsen worker 756 - jepsen.util 756 :ok :add 2767 INFO [2022-01-10 08:39:20,557] jepsen worker 681 - jepsen.util 681 :invoke :add 2804 INFO [2022-01-10 08:39:20,557] jepsen worker 886 - jepsen.util 886 :ok :add 2724 INFO [2022-01-10 08:39:20,560] jepsen worker 94 - jepsen.util 94 :invoke :add 2805 INFO [2022-01-10 08:39:20,560] jepsen worker 863 - jepsen.util 863 :ok :add 2784 INFO [2022-01-10 08:39:20,563] jepsen worker 706 - jepsen.util 706 :invoke :add 2806 INFO [2022-01-10 08:39:20,564] jepsen worker 823 - jepsen.util 823 :ok :add 2725 INFO [2022-01-10 08:39:20,564] jepsen worker 573 - jepsen.util 573 :ok :add 2800 INFO [2022-01-10 08:39:20,565] jepsen worker 918 - jepsen.util 918 :ok :add 2726 INFO [2022-01-10 08:39:20,565] jepsen worker 176 - jepsen.util 176 :ok :add 2803 INFO [2022-01-10 08:39:20,566] jepsen worker 708 - jepsen.util 708 :ok :add 2764 INFO [2022-01-10 08:39:20,566] jepsen worker 544 - jepsen.util 544 :ok :add 2748 INFO [2022-01-10 08:39:20,566] jepsen worker 132 - jepsen.util 132 :invoke :add 2807 INFO [2022-01-10 08:39:20,570] jepsen worker 946 - jepsen.util 946 :invoke :add 2808 INFO [2022-01-10 08:39:20,573] jepsen worker 129 - jepsen.util 129 :ok :add 2619 INFO [2022-01-10 08:39:20,573] jepsen worker 709 - jepsen.util 709 :ok :add 2635 INFO [2022-01-10 08:39:20,573] jepsen worker 97 - jepsen.util 97 :invoke :add 2809 INFO [2022-01-10 08:39:20,576] jepsen worker 157 - jepsen.util 157 :invoke :add 2810 INFO [2022-01-10 08:39:20,577] jepsen worker 531 - jepsen.util 531 :ok :add 2704 INFO [2022-01-10 08:39:20,577] jepsen worker 425 - jepsen.util 425 :ok :add 2694 INFO [2022-01-10 08:39:20,577] jepsen worker 481 - jepsen.util 481 :ok :add 2705 INFO [2022-01-10 08:39:20,577] jepsen worker 417 - jepsen.util 417 :ok :add 2729 INFO [2022-01-10 08:39:20,578] jepsen worker 816 - jepsen.util 816 :ok :add 2731 INFO [2022-01-10 08:39:20,578] jepsen worker 858 - jepsen.util 858 :ok :add 2728 INFO [2022-01-10 08:39:20,578] jepsen worker 819 - jepsen.util 819 :ok :add 2695 INFO [2022-01-10 08:39:20,579] jepsen worker 942 - jepsen.util 942 :invoke :add 2811 INFO [2022-01-10 08:39:20,581] jepsen worker 224 - jepsen.util 224 :ok :add 2778 INFO [2022-01-10 08:39:20,581] jepsen worker 740 - jepsen.util 740 :ok :add 2754 INFO [2022-01-10 08:39:20,581] jepsen worker 986 - jepsen.util 986 :ok :add 2760 INFO [2022-01-10 08:39:20,583] jepsen worker 898 - jepsen.util 898 :ok :add 2761 INFO [2022-01-10 08:39:20,583] jepsen worker 239 - jepsen.util 239 :ok :add 2755 INFO [2022-01-10 08:39:20,583] jepsen worker 992 - jepsen.util 992 :ok :add 2779 INFO [2022-01-10 08:39:20,585] jepsen worker 439 - jepsen.util 439 :invoke :add 2812 INFO [2022-01-10 08:39:20,588] jepsen worker 432 - jepsen.util 432 :invoke :add 2813 INFO [2022-01-10 08:39:20,590] jepsen worker 288 - jepsen.util 288 :ok :add 2756 INFO [2022-01-10 08:39:20,590] jepsen worker 831 - jepsen.util 831 :ok :add 2762 INFO [2022-01-10 08:39:20,590] jepsen worker 608 - jepsen.util 608 :ok :add 2757 INFO [2022-01-10 08:39:20,590] jepsen worker 785 - jepsen.util 785 :ok :add 2763 INFO [2022-01-10 08:39:20,590] jepsen worker 996 - jepsen.util 996 :invoke :add 2814 INFO [2022-01-10 08:39:20,594] jepsen worker 398 - jepsen.util 398 :invoke :add 2815 INFO [2022-01-10 08:39:20,597] jepsen worker 863 - jepsen.util 863 :invoke :add 2816 INFO [2022-01-10 08:39:20,598] jepsen worker 936 - jepsen.util 936 :ok :add 2780 INFO [2022-01-10 08:39:20,599] jepsen worker 532 - jepsen.util 532 :ok :add 2798 INFO [2022-01-10 08:39:20,599] jepsen worker 510 - jepsen.util 510 :invoke :add 2817 INFO [2022-01-10 08:39:20,602] jepsen worker 120 - jepsen.util 120 :invoke :add 2818 INFO [2022-01-10 08:39:20,603] jepsen worker 880 - jepsen.util 880 :ok :add 2782 INFO [2022-01-10 08:39:20,603] jepsen worker 771 - jepsen.util 771 :ok :add 2802 INFO [2022-01-10 08:39:20,605] jepsen worker 376 - jepsen.util 376 :invoke :add 2819 INFO [2022-01-10 08:39:20,605] jepsen worker 352 - jepsen.util 352 :ok :add 2766 INFO [2022-01-10 08:39:20,608] jepsen worker 613 - jepsen.util 613 :invoke :add 2820 INFO [2022-01-10 08:39:20,609] jepsen worker 953 - jepsen.util 953 :ok :add 2785 INFO [2022-01-10 08:39:20,610] jepsen worker 598 - jepsen.util 598 :invoke :add 2821 INFO [2022-01-10 08:39:20,612] jepsen worker 681 - jepsen.util 681 :ok :add 2804 INFO [2022-01-10 08:39:20,614] jepsen worker 269 - jepsen.util 269 :invoke :add 2822 INFO [2022-01-10 08:39:20,615] jepsen worker 94 - jepsen.util 94 :ok :add 2805 INFO [2022-01-10 08:39:20,617] jepsen worker 379 - jepsen.util 379 :invoke :add 2823 INFO [2022-01-10 08:39:20,619] jepsen worker 132 - jepsen.util 132 :ok :add 2807 INFO [2022-01-10 08:39:20,619] jepsen worker 577 - jepsen.util 577 :invoke :add 2824 INFO [2022-01-10 08:39:20,622] jepsen worker 897 - jepsen.util 897 :ok :add 2799 INFO [2022-01-10 08:39:20,622] jepsen worker 787 - jepsen.util 787 :ok :add 2781 INFO [2022-01-10 08:39:20,622] jepsen worker 501 - jepsen.util 501 :invoke :add 2825 INFO [2022-01-10 08:39:20,625] jepsen worker 393 - jepsen.util 393 :invoke :add 2826 INFO [2022-01-10 08:39:20,627] jepsen worker 235 - jepsen.util 235 :ok :add 2783 INFO [2022-01-10 08:39:20,628] jepsen worker 920 - jepsen.util 920 :invoke :add 2827 INFO [2022-01-10 08:39:20,629] jepsen worker 562 - jepsen.util 562 :ok :add 2696 INFO [2022-01-10 08:39:20,629] jepsen worker 607 - jepsen.util 607 :ok :add 2706 INFO [2022-01-10 08:39:20,629] jepsen worker 797 - jepsen.util 797 :ok :add 2801 INFO [2022-01-10 08:39:20,630] jepsen worker 376 - jepsen.util 376 :ok :add 2819 INFO [2022-01-10 08:39:20,630] jepsen worker 706 - jepsen.util 706 :ok :add 2806 INFO [2022-01-10 08:39:20,630] jepsen worker 53 - jepsen.util 53 :ok :add 2707 INFO [2022-01-10 08:39:20,630] jepsen worker 644 - jepsen.util 644 :ok :add 2733 INFO [2022-01-10 08:39:20,631] jepsen worker 979 - jepsen.util 979 :invoke :add 2828 INFO [2022-01-10 08:39:20,632] jepsen worker 32 - jepsen.util 32 :ok :add 2697 INFO [2022-01-10 08:39:20,635] jepsen worker 981 - jepsen.util 981 :invoke :add 2829 INFO [2022-01-10 08:39:20,638] jepsen worker 180 - jepsen.util 180 :invoke :add 2830 INFO [2022-01-10 08:39:20,641] jepsen worker 875 - jepsen.util 875 :invoke :add 2831 INFO [2022-01-10 08:39:20,642] jepsen worker 985 - jepsen.util 985 :ok :add 2796 INFO [2022-01-10 08:39:20,643] jepsen worker 197 - jepsen.util 197 :invoke :add 2832 INFO [2022-01-10 08:39:20,645] jepsen worker 556 - jepsen.util 556 :ok :add 2637 INFO [2022-01-10 08:39:20,646] jepsen worker 967 - jepsen.util 967 :invoke :add 2833 INFO [2022-01-10 08:39:20,648] jepsen worker 120 - jepsen.util 120 :ok :add 2818 INFO [2022-01-10 08:39:20,650] jepsen worker 543 - jepsen.util 543 :invoke :add 2834 INFO [2022-01-10 08:39:20,651] jepsen worker 825 - jepsen.util 825 :ok :add 2759 INFO [2022-01-10 08:39:20,651] jepsen worker 396 - jepsen.util 396 :ok :add 2741 INFO [2022-01-10 08:39:20,651] jepsen worker 117 - jepsen.util 117 :ok :add 2775 INFO [2022-01-10 08:39:20,653] jepsen worker 288 - jepsen.util 288 :invoke :add 2835 INFO [2022-01-10 08:39:20,654] jepsen worker 979 - jepsen.util 979 :ok :add 2828 INFO [2022-01-10 08:39:20,654] jepsen worker 180 - jepsen.util 180 :ok :add 2830 INFO [2022-01-10 08:39:20,657] jepsen worker 154 - jepsen.util 154 :invoke :add 2836 INFO [2022-01-10 08:39:20,658] jepsen worker 197 - jepsen.util 197 :ok :add 2832 INFO [2022-01-10 08:39:20,659] jepsen worker 527 - jepsen.util 527 :invoke :add 2837 INFO [2022-01-10 08:39:20,661] jepsen worker 875 - jepsen.util 875 :ok :add 2831 INFO [2022-01-10 08:39:20,661] jepsen worker 981 - jepsen.util 981 :ok :add 2829 INFO [2022-01-10 08:39:20,662] jepsen worker 418 - jepsen.util 418 :invoke :add 2838 INFO [2022-01-10 08:39:20,665] jepsen worker 374 - jepsen.util 374 :invoke :add 2839 INFO [2022-01-10 08:39:20,667] jepsen worker 967 - jepsen.util 967 :ok :add 2833 INFO [2022-01-10 08:39:20,668] jepsen worker 492 - jepsen.util 492 :invoke :add 2840 INFO [2022-01-10 08:39:20,671] jepsen worker 990 - jepsen.util 990 :invoke :add 2841 INFO [2022-01-10 08:39:20,673] jepsen worker 289 - jepsen.util 289 :invoke :add 2842 INFO [2022-01-10 08:39:20,675] jepsen worker 288 - jepsen.util 288 :ok :add 2835 INFO [2022-01-10 08:39:20,676] jepsen worker 108 - jepsen.util 108 :invoke :add 2843 INFO [2022-01-10 08:39:20,680] jepsen worker 55 - jepsen.util 55 :invoke :add 2844 INFO [2022-01-10 08:39:20,680] jepsen worker 492 - jepsen.util 492 :ok :add 2840 INFO [2022-01-10 08:39:20,682] jepsen worker 100 - jepsen.util 100 :invoke :add 2845 INFO [2022-01-10 08:39:20,685] jepsen worker 289 - jepsen.util 289 :ok :add 2842 INFO [2022-01-10 08:39:20,685] jepsen worker 181 - jepsen.util 181 :invoke :add 2846 INFO [2022-01-10 08:39:20,687] jepsen worker 543 - jepsen.util 543 :ok :add 2834 INFO [2022-01-10 08:39:20,688] jepsen worker 113 - jepsen.util 113 :invoke :add 2847 INFO [2022-01-10 08:39:20,691] jepsen worker 637 - jepsen.util 637 :invoke :add 2848 INFO [2022-01-10 08:39:20,691] jepsen worker 527 - jepsen.util 527 :ok :add 2837 INFO [2022-01-10 08:39:20,696] jepsen worker 617 - jepsen.util 617 :invoke :add 2849 INFO [2022-01-10 08:39:20,696] jepsen worker 909 - jepsen.util 909 :ok :add 2743 INFO [2022-01-10 08:39:20,697] jepsen worker 517 - jepsen.util 517 :ok :add 2744 INFO [2022-01-10 08:39:20,698] jepsen worker 154 - jepsen.util 154 :ok :add 2836 INFO [2022-01-10 08:39:20,698] jepsen worker 840 - jepsen.util 840 :ok :add 2777 INFO [2022-01-10 08:39:20,700] jepsen worker 55 - jepsen.util 55 :ok :add 2844 INFO [2022-01-10 08:39:20,700] jepsen worker 691 - jepsen.util 691 :invoke :add 2850 INFO [2022-01-10 08:39:20,700] jepsen worker 812 - jepsen.util 812 :ok :add 2634 INFO [2022-01-10 08:39:20,700] jepsen worker 550 - jepsen.util 550 :ok :add 2790 INFO [2022-01-10 08:39:20,701] jepsen worker 990 - jepsen.util 990 :ok :add 2841 INFO [2022-01-10 08:39:20,704] jepsen worker 972 - jepsen.util 972 :ok :add 2758 INFO [2022-01-10 08:39:20,704] jepsen worker 109 - jepsen.util 109 :ok :add 2774 INFO [2022-01-10 08:39:20,705] jepsen worker 616 - jepsen.util 616 :invoke :add 2851 INFO [2022-01-10 08:39:20,706] jepsen worker 100 - jepsen.util 100 :ok :add 2845 INFO [2022-01-10 08:39:20,706] jepsen worker 181 - jepsen.util 181 :ok :add 2846 INFO [2022-01-10 08:39:20,706] jepsen worker 108 - jepsen.util 108 :ok :add 2843 INFO [2022-01-10 08:39:20,709] jepsen worker 557 - jepsen.util 557 :invoke :add 2852 INFO [2022-01-10 08:39:20,711] jepsen worker 758 - jepsen.util 758 :ok :add 2776 INFO [2022-01-10 08:39:20,713] jepsen worker 583 - jepsen.util 583 :invoke :add 2853 INFO [2022-01-10 08:39:20,714] jepsen worker 113 - jepsen.util 113 :ok :add 2847 INFO [2022-01-10 08:39:20,716] jepsen worker 656 - jepsen.util 656 :invoke :add 2854 INFO [2022-01-10 08:39:20,720] jepsen worker 899 - jepsen.util 899 :invoke :add 2855 INFO [2022-01-10 08:39:20,723] jepsen worker 245 - jepsen.util 245 :invoke :add 2856 INFO [2022-01-10 08:39:20,726] jepsen worker 593 - jepsen.util 593 :invoke :add 2857 INFO [2022-01-10 08:39:20,729] jepsen worker 890 - jepsen.util 890 :invoke :add 2858 INFO [2022-01-10 08:39:20,731] jepsen worker 435 - jepsen.util 435 :invoke :add 2859 INFO [2022-01-10 08:39:20,734] jepsen worker 731 - jepsen.util 731 :invoke :add 2860 INFO [2022-01-10 08:39:20,736] jepsen worker 890 - jepsen.util 890 :ok :add 2858 INFO [2022-01-10 08:39:20,736] jepsen worker 526 - jepsen.util 526 :invoke :add 2861 INFO [2022-01-10 08:39:20,739] jepsen worker 435 - jepsen.util 435 :ok :add 2859 INFO [2022-01-10 08:39:20,739] jepsen worker 107 - jepsen.util 107 :invoke :add 2862 INFO [2022-01-10 08:39:20,742] jepsen worker 845 - jepsen.util 845 :invoke :add 2863 INFO [2022-01-10 08:39:20,745] jepsen worker 609 - jepsen.util 609 :invoke :add 2864 INFO [2022-01-10 08:39:20,747] jepsen worker 679 - jepsen.util 679 :invoke :add 2865 INFO [2022-01-10 08:39:20,749] jepsen worker 519 - jepsen.util 519 :invoke :add 2866 INFO [2022-01-10 08:39:20,752] jepsen worker 456 - jepsen.util 456 :invoke :add 2867 INFO [2022-01-10 08:39:20,754] jepsen worker 352 - jepsen.util 352 :invoke :add 2868 INFO [2022-01-10 08:39:20,757] jepsen worker 412 - jepsen.util 412 :invoke :add 2869 INFO [2022-01-10 08:39:20,760] jepsen worker 661 - jepsen.util 661 :invoke :add 2870 INFO [2022-01-10 08:39:20,761] jepsen worker 352 - jepsen.util 352 :ok :add 2868 INFO [2022-01-10 08:39:20,763] jepsen worker 643 - jepsen.util 643 :invoke :add 2871 INFO [2022-01-10 08:39:20,765] jepsen worker 894 - jepsen.util 894 :invoke :add 2872 INFO [2022-01-10 08:39:20,767] jepsen worker 528 - jepsen.util 528 :invoke :add 2873 INFO [2022-01-10 08:39:20,770] jepsen worker 972 - jepsen.util 972 :invoke :add 2874 INFO [2022-01-10 08:39:20,770] jepsen worker 412 - jepsen.util 412 :ok :add 2869 INFO [2022-01-10 08:39:20,772] jepsen worker 614 - jepsen.util 614 :invoke :add 2875 INFO [2022-01-10 08:39:20,774] jepsen worker 812 - jepsen.util 812 :invoke :add 2876 INFO [2022-01-10 08:39:20,776] jepsen worker 643 - jepsen.util 643 :ok :add 2871 INFO [2022-01-10 08:39:20,777] jepsen worker 801 - jepsen.util 801 :invoke :add 2877 INFO [2022-01-10 08:39:20,778] jepsen worker 894 - jepsen.util 894 :ok :add 2872 INFO [2022-01-10 08:39:20,779] jepsen worker 68 - jepsen.util 68 :invoke :add 2878 INFO [2022-01-10 08:39:20,784] jepsen worker 120 - jepsen.util 120 :invoke :add 2879 INFO [2022-01-10 08:39:20,784] jepsen worker 528 - jepsen.util 528 :ok :add 2873 INFO [2022-01-10 08:39:20,784] jepsen worker 661 - jepsen.util 661 :ok :add 2870 INFO [2022-01-10 08:39:20,788] jepsen worker 351 - jepsen.util 351 :invoke :add 2880 INFO [2022-01-10 08:39:20,792] jepsen worker 953 - jepsen.util 953 :invoke :add 2881 INFO [2022-01-10 08:39:20,794] jepsen worker 614 - jepsen.util 614 :ok :add 2875 INFO [2022-01-10 08:39:20,796] jepsen worker 262 - jepsen.util 262 :invoke :add 2882 INFO [2022-01-10 08:39:20,799] jepsen worker 980 - jepsen.util 980 :invoke :add 2883 INFO [2022-01-10 08:39:20,800] jepsen worker 801 - jepsen.util 801 :ok :add 2877 INFO [2022-01-10 08:39:20,801] jepsen worker 997 - jepsen.util 997 :invoke :add 2884 INFO [2022-01-10 08:39:20,804] jepsen worker 936 - jepsen.util 936 :invoke :add 2885 INFO [2022-01-10 08:39:20,804] jepsen worker 972 - jepsen.util 972 :ok :add 2874 INFO [2022-01-10 08:39:20,807] jepsen worker 168 - jepsen.util 168 :invoke :add 2886 INFO [2022-01-10 08:39:20,807] jepsen worker 812 - jepsen.util 812 :ok :add 2876 INFO [2022-01-10 08:39:20,810] jepsen worker 596 - jepsen.util 596 :invoke :add 2887 INFO [2022-01-10 08:39:20,811] jepsen worker 547 - jepsen.util 547 :ok :add 2670 INFO [2022-01-10 08:39:20,812] jepsen worker 201 - jepsen.util 201 :ok :add 2669 INFO [2022-01-10 08:39:20,812] jepsen worker 293 - jepsen.util 293 :ok :add 2644 INFO [2022-01-10 08:39:20,812] jepsen worker 640 - jepsen.util 640 :ok :add 2645 INFO [2022-01-10 08:39:20,812] jepsen worker 896 - jepsen.util 896 :ok :add 2671 INFO [2022-01-10 08:39:20,812] jepsen worker 848 - jepsen.util 848 :ok :add 2668 INFO [2022-01-10 08:39:20,813] jepsen worker 986 - jepsen.util 986 :invoke :add 2888 INFO [2022-01-10 08:39:20,817] jepsen worker 832 - jepsen.util 832 :invoke :add 2889 INFO [2022-01-10 08:39:20,819] jepsen worker 958 - jepsen.util 958 :invoke :add 2890 INFO [2022-01-10 08:39:20,820] jepsen worker 986 - jepsen.util 986 :ok :add 2888 INFO [2022-01-10 08:39:20,822] jepsen worker 603 - jepsen.util 603 :invoke :add 2891 INFO [2022-01-10 08:39:20,825] jepsen worker 141 - jepsen.util 141 :invoke :add 2892 INFO [2022-01-10 08:39:20,825] jepsen worker 832 - jepsen.util 832 :ok :add 2889 INFO [2022-01-10 08:39:20,828] jepsen worker 74 - jepsen.util 74 :invoke :add 2893 INFO [2022-01-10 08:39:20,831] jepsen worker 906 - jepsen.util 906 :invoke :add 2894 INFO [2022-01-10 08:39:20,834] jepsen worker 994 - jepsen.util 994 :invoke :add 2895 INFO [2022-01-10 08:39:20,836] jepsen worker 180 - jepsen.util 180 :invoke :add 2896 INFO [2022-01-10 08:39:20,839] jepsen worker 906 - jepsen.util 906 :ok :add 2894 INFO [2022-01-10 08:39:20,840] jepsen worker 820 - jepsen.util 820 :invoke :add 2897 INFO [2022-01-10 08:39:20,841] jepsen worker 994 - jepsen.util 994 :ok :add 2895 INFO [2022-01-10 08:39:20,842] jepsen worker 529 - jepsen.util 529 :invoke :add 2898 INFO [2022-01-10 08:39:20,844] jepsen worker 180 - jepsen.util 180 :ok :add 2896 INFO [2022-01-10 08:39:20,845] jepsen worker 147 - jepsen.util 147 :invoke :add 2899 INFO [2022-01-10 08:39:20,849] jepsen worker 444 - jepsen.util 444 :invoke :add 2900 INFO [2022-01-10 08:39:20,850] jepsen worker 820 - jepsen.util 820 :ok :add 2897 INFO [2022-01-10 08:39:20,852] jepsen worker 6 - jepsen.util 6 :invoke :add 2901 INFO [2022-01-10 08:39:20,856] jepsen worker 87 - jepsen.util 87 :invoke :add 2902 INFO [2022-01-10 08:39:20,859] jepsen worker 99 - jepsen.util 99 :invoke :add 2903 INFO [2022-01-10 08:39:20,861] jepsen worker 815 - jepsen.util 815 :invoke :add 2904 INFO [2022-01-10 08:39:20,864] jepsen worker 311 - jepsen.util 311 :invoke :add 2905 INFO [2022-01-10 08:39:20,867] jepsen worker 903 - jepsen.util 903 :invoke :add 2906 INFO [2022-01-10 08:39:20,869] jepsen worker 578 - jepsen.util 578 :invoke :add 2907 INFO [2022-01-10 08:39:20,870] jepsen worker 815 - jepsen.util 815 :ok :add 2904 INFO [2022-01-10 08:39:20,871] jepsen worker 599 - jepsen.util 599 :ok :add 2795 INFO [2022-01-10 08:39:20,871] jepsen worker 997 - jepsen.util 997 :ok :add 2884 INFO [2022-01-10 08:39:20,873] jepsen worker 89 - jepsen.util 89 :invoke :add 2908 INFO [2022-01-10 08:39:20,874] jepsen worker 311 - jepsen.util 311 :ok :add 2905 INFO [2022-01-10 08:39:20,876] jepsen worker 879 - jepsen.util 879 :invoke :add 2909 INFO [2022-01-10 08:39:20,877] jepsen worker 578 - jepsen.util 578 :ok :add 2907 INFO [2022-01-10 08:39:20,879] jepsen worker 47 - jepsen.util 47 :invoke :add 2910 INFO [2022-01-10 08:39:20,880] jepsen worker 903 - jepsen.util 903 :ok :add 2906 INFO [2022-01-10 08:39:20,881] jepsen worker 611 - jepsen.util 611 :invoke :add 2911 INFO [2022-01-10 08:39:20,885] jepsen worker 654 - jepsen.util 654 :invoke :add 2912 INFO [2022-01-10 08:39:20,886] jepsen worker 958 - jepsen.util 958 :ok :add 2890 INFO [2022-01-10 08:39:20,887] jepsen worker 603 - jepsen.util 603 :ok :add 2891 INFO [2022-01-10 08:39:20,888] jepsen worker 514 - jepsen.util 514 :invoke :add 2913 INFO [2022-01-10 08:39:20,890] jepsen worker 239 - jepsen.util 239 :invoke :add 2914 INFO [2022-01-10 08:39:20,891] jepsen worker 87 - jepsen.util 87 :ok :add 2902 INFO [2022-01-10 08:39:20,892] jepsen worker 6 - jepsen.util 6 :ok :add 2901 INFO [2022-01-10 08:39:20,892] jepsen worker 141 - jepsen.util 141 :ok :add 2892 INFO [2022-01-10 08:39:20,894] jepsen worker 670 - jepsen.util 670 :invoke :add 2915 INFO [2022-01-10 08:39:20,896] jepsen worker 444 - jepsen.util 444 :ok :add 2900 INFO [2022-01-10 08:39:20,896] jepsen worker 74 - jepsen.util 74 :ok :add 2893 INFO [2022-01-10 08:39:20,897] jepsen worker 99 - jepsen.util 99 :ok :add 2903 INFO [2022-01-10 08:39:20,898] jepsen worker 243 - jepsen.util 243 :invoke :add 2916 INFO [2022-01-10 08:39:20,901] jepsen worker 762 - jepsen.util 762 :ok :add 2646 INFO [2022-01-10 08:39:20,901] jepsen worker 629 - jepsen.util 629 :ok :add 2672 INFO [2022-01-10 08:39:20,901] jepsen worker 873 - jepsen.util 873 :ok :add 2647 INFO [2022-01-10 08:39:20,901] jepsen worker 256 - jepsen.util 256 :ok :add 2673 INFO [2022-01-10 08:39:20,902] jepsen worker 548 - jepsen.util 548 :invoke :add 2917 INFO [2022-01-10 08:39:20,906] jepsen worker 528 - jepsen.util 528 :invoke :add 2918 INFO [2022-01-10 08:39:20,909] jepsen worker 242 - jepsen.util 242 :invoke :add 2919 INFO [2022-01-10 08:39:20,912] jepsen worker 885 - jepsen.util 885 :invoke :add 2920 INFO [2022-01-10 08:39:20,916] jepsen worker 329 - jepsen.util 329 :invoke :add 2921 INFO [2022-01-10 08:39:20,918] jepsen worker 528 - jepsen.util 528 :ok :add 2918 INFO [2022-01-10 08:39:20,919] jepsen worker 589 - jepsen.util 589 :invoke :add 2922 INFO [2022-01-10 08:39:20,922] jepsen worker 20 - jepsen.util 20 :invoke :add 2923 INFO [2022-01-10 08:39:20,924] jepsen worker 789 - jepsen.util 789 :invoke :add 2924 INFO [2022-01-10 08:39:20,924] jepsen worker 242 - jepsen.util 242 :ok :add 2919 INFO [2022-01-10 08:39:20,926] jepsen worker 311 - jepsen.util 311 :invoke :add 2925 INFO [2022-01-10 08:39:20,928] jepsen worker 915 - jepsen.util 915 :invoke :add 2926 INFO [2022-01-10 08:39:20,931] jepsen worker 10 - jepsen.util 10 :invoke :add 2927 INFO [2022-01-10 08:39:20,932] jepsen worker 613 - jepsen.util 613 :ok :add 2820 INFO [2022-01-10 08:39:20,932] jepsen worker 996 - jepsen.util 996 :ok :add 2814 INFO [2022-01-10 08:39:20,933] jepsen worker 735 - jepsen.util 735 :invoke :add 2928 INFO [2022-01-10 08:39:20,936] jepsen worker 224 - jepsen.util 224 :invoke :add 2929 INFO [2022-01-10 08:39:20,939] jepsen worker 677 - jepsen.util 677 :invoke :add 2930 INFO [2022-01-10 08:39:20,941] jepsen worker 453 - jepsen.util 453 :invoke :add 2931 INFO [2022-01-10 08:39:20,942] jepsen worker 936 - jepsen.util 936 :ok :add 2885 INFO [2022-01-10 08:39:20,942] jepsen worker 954 - jepsen.util 954 :ok :add 2794 INFO [2022-01-10 08:39:20,943] jepsen worker 333 - jepsen.util 333 :invoke :add 2932 INFO [2022-01-10 08:39:20,946] jepsen worker 760 - jepsen.util 760 :invoke :add 2933 INFO [2022-01-10 08:39:20,948] jepsen worker 835 - jepsen.util 835 :invoke :add 2934 INFO [2022-01-10 08:39:20,950] jepsen worker 269 - jepsen.util 269 :ok :add 2822 INFO [2022-01-10 08:39:20,950] jepsen worker 863 - jepsen.util 863 :ok :add 2816 INFO [2022-01-10 08:39:20,950] jepsen worker 598 - jepsen.util 598 :ok :add 2821 INFO [2022-01-10 08:39:20,950] jepsen worker 850 - jepsen.util 850 :invoke :add 2935 INFO [2022-01-10 08:39:20,950] jepsen worker 874 - jepsen.util 874 :ok :add 2730 INFO [2022-01-10 08:39:20,951] jepsen worker 374 - jepsen.util 374 :ok :add 2839 INFO [2022-01-10 08:39:20,951] jepsen worker 398 - jepsen.util 398 :ok :add 2815 INFO [2022-01-10 08:39:20,954] jepsen worker 206 - jepsen.util 206 :invoke :add 2936 INFO [2022-01-10 08:39:20,955] jepsen worker 735 - jepsen.util 735 :ok :add 2928 INFO [2022-01-10 08:39:20,955] jepsen worker 677 - jepsen.util 677 :ok :add 2930 INFO [2022-01-10 08:39:20,957] jepsen worker 58 - jepsen.util 58 :invoke :add 2937 INFO [2022-01-10 08:39:20,958] jepsen worker 835 - jepsen.util 835 :ok :add 2934 INFO [2022-01-10 08:39:20,960] jepsen worker 292 - jepsen.util 292 :invoke :add 2938 INFO [2022-01-10 08:39:20,962] jepsen worker 568 - jepsen.util 568 :invoke :add 2939 INFO [2022-01-10 08:39:20,962] jepsen worker 850 - jepsen.util 850 :ok :add 2935 INFO [2022-01-10 08:39:20,962] jepsen worker 206 - jepsen.util 206 :ok :add 2936 INFO [2022-01-10 08:39:20,965] jepsen worker 265 - jepsen.util 265 :invoke :add 2940 INFO [2022-01-10 08:39:20,968] jepsen worker 809 - jepsen.util 809 :invoke :add 2941 INFO [2022-01-10 08:39:20,971] jepsen worker 266 - jepsen.util 266 :invoke :add 2942 INFO [2022-01-10 08:39:20,972] jepsen worker 58 - jepsen.util 58 :ok :add 2937 INFO [2022-01-10 08:39:20,973] jepsen worker 976 - jepsen.util 976 :invoke :add 2943 INFO [2022-01-10 08:39:20,975] jepsen worker 115 - jepsen.util 115 :invoke :add 2944 INFO [2022-01-10 08:39:20,978] jepsen worker 974 - jepsen.util 974 :invoke :add 2945 INFO [2022-01-10 08:39:20,980] jepsen worker 368 - jepsen.util 368 :invoke :add 2946 INFO [2022-01-10 08:39:20,982] jepsen worker 564 - jepsen.util 564 :invoke :add 2947 INFO [2022-01-10 08:39:20,985] jepsen worker 380 - jepsen.util 380 :invoke :add 2948 INFO [2022-01-10 08:39:20,986] jepsen worker 974 - jepsen.util 974 :ok :add 2945 INFO [2022-01-10 08:39:20,987] jepsen worker 388 - jepsen.util 388 :invoke :add 2949 INFO [2022-01-10 08:39:20,989] jepsen worker 564 - jepsen.util 564 :ok :add 2947 INFO [2022-01-10 08:39:20,990] jepsen worker 410 - jepsen.util 410 :invoke :add 2950 INFO [2022-01-10 08:39:20,990] jepsen worker 333 - jepsen.util 333 :ok :add 2932 INFO [2022-01-10 08:39:20,991] jepsen worker 168 - jepsen.util 168 :ok :add 2886 INFO [2022-01-10 08:39:20,991] jepsen worker 438 - jepsen.util 438 :ok :add 2797 INFO [2022-01-10 08:39:20,991] jepsen worker 377 - jepsen.util 377 :ok :add 2732 INFO [2022-01-10 08:39:20,991] jepsen worker 510 - jepsen.util 510 :ok :add 2817 INFO [2022-01-10 08:39:20,991] jepsen worker 379 - jepsen.util 379 :ok :add 2823 INFO [2022-01-10 08:39:20,991] jepsen worker 243 - jepsen.util 243 :ok :add 2916 INFO [2022-01-10 08:39:20,991] jepsen worker 589 - jepsen.util 589 :ok :add 2922 INFO [2022-01-10 08:39:20,991] jepsen worker 115 - jepsen.util 115 :ok :add 2944 INFO [2022-01-10 08:39:20,992] jepsen worker 676 - jepsen.util 676 :invoke :add 2951 INFO [2022-01-10 08:39:20,994] jepsen worker 453 - jepsen.util 453 :ok :add 2931 INFO [2022-01-10 08:39:20,994] jepsen worker 224 - jepsen.util 224 :ok :add 2929 INFO [2022-01-10 08:39:20,996] jepsen worker 760 - jepsen.util 760 :ok :add 2933 INFO [2022-01-10 08:39:20,997] jepsen worker 368 - jepsen.util 368 :ok :add 2946 INFO [2022-01-10 08:39:20,997] jepsen worker 319 - jepsen.util 319 :invoke :add 2952 INFO [2022-01-10 08:39:21,000] jepsen worker 596 - jepsen.util 596 :ok :add 2887 INFO [2022-01-10 08:39:21,000] jepsen worker 20 - jepsen.util 20 :ok :add 2923 INFO [2022-01-10 08:39:21,000] jepsen worker 548 - jepsen.util 548 :ok :add 2917 INFO [2022-01-10 08:39:21,000] jepsen worker 840 - jepsen.util 840 :invoke :add 2953 INFO [2022-01-10 08:39:21,003] jepsen worker 453 - jepsen.util 453 :invoke :add 2954 INFO [2022-01-10 08:39:21,005] jepsen worker 678 - jepsen.util 678 :invoke :add 2955 INFO [2022-01-10 08:39:21,006] jepsen worker 616 - jepsen.util 616 :ok :add 2851 INFO [2022-01-10 08:39:21,007] jepsen worker 679 - jepsen.util 679 :ok :add 2865 INFO [2022-01-10 08:39:21,007] jepsen worker 617 - jepsen.util 617 :ok :add 2849 INFO [2022-01-10 08:39:21,007] jepsen worker 691 - jepsen.util 691 :ok :add 2850 INFO [2022-01-10 08:39:21,007] jepsen worker 637 - jepsen.util 637 :ok :add 2848 INFO [2022-01-10 08:39:21,008] jepsen worker 609 - jepsen.util 609 :ok :add 2864 INFO [2022-01-10 08:39:21,008] jepsen worker 220 - jepsen.util 220 :invoke :add 2956 INFO [2022-01-10 08:39:21,012] jepsen worker 880 - jepsen.util 880 :invoke :add 2957 INFO [2022-01-10 08:39:21,014] jepsen worker 752 - jepsen.util 752 :invoke :add 2958 INFO [2022-01-10 08:39:21,015] jepsen worker 456 - jepsen.util 456 :ok :add 2867 INFO [2022-01-10 08:39:21,015] jepsen worker 519 - jepsen.util 519 :ok :add 2866 INFO [2022-01-10 08:39:21,015] jepsen worker 583 - jepsen.util 583 :ok :add 2853 INFO [2022-01-10 08:39:21,015] jepsen worker 809 - jepsen.util 809 :ok :add 2941 INFO [2022-01-10 08:39:21,015] jepsen worker 265 - jepsen.util 265 :ok :add 2940 INFO [2022-01-10 08:39:21,015] jepsen worker 557 - jepsen.util 557 :ok :add 2852 INFO [2022-01-10 08:39:21,019] jepsen worker 426 - jepsen.util 426 :invoke :add 2959 INFO [2022-01-10 08:39:21,019] jepsen worker 976 - jepsen.util 976 :ok :add 2943 INFO [2022-01-10 08:39:21,020] jepsen worker 266 - jepsen.util 266 :ok :add 2942 INFO [2022-01-10 08:39:21,021] jepsen worker 223 - jepsen.util 223 :invoke :add 2960 INFO [2022-01-10 08:39:21,023] jepsen worker 951 - jepsen.util 951 :invoke :add 2961 INFO [2022-01-10 08:39:21,026] jepsen worker 908 - jepsen.util 908 :invoke :add 2962 INFO [2022-01-10 08:39:21,026] jepsen worker 885 - jepsen.util 885 :ok :add 2920 INFO [2022-01-10 08:39:21,027] jepsen worker 239 - jepsen.util 239 :ok :add 2914 INFO [2022-01-10 08:39:21,027] jepsen worker 292 - jepsen.util 292 :ok :add 2938 INFO [2022-01-10 08:39:21,028] jepsen worker 830 - jepsen.util 830 :invoke :add 2963 INFO [2022-01-10 08:39:21,031] jepsen worker 172 - jepsen.util 172 :invoke :add 2964 INFO [2022-01-10 08:39:21,033] jepsen worker 447 - jepsen.util 447 :invoke :add 2965 INFO [2022-01-10 08:39:21,035] jepsen worker 481 - jepsen.util 481 :invoke :add 2966 INFO [2022-01-10 08:39:21,038] jepsen worker 231 - jepsen.util 231 :invoke :add 2967 INFO [2022-01-10 08:39:21,040] jepsen worker 787 - jepsen.util 787 :invoke :add 2968 INFO [2022-01-10 08:39:21,042] jepsen worker 682 - jepsen.util 682 :invoke :add 2969 INFO [2022-01-10 08:39:21,045] jepsen worker 643 - jepsen.util 643 :invoke :add 2970 INFO [2022-01-10 08:39:21,048] jepsen worker 560 - jepsen.util 560 :invoke :add 2971 INFO [2022-01-10 08:39:21,048] jepsen worker 787 - jepsen.util 787 :ok :add 2968 INFO [2022-01-10 08:39:21,050] jepsen worker 681 - jepsen.util 681 :invoke :add 2972 INFO [2022-01-10 08:39:21,051] jepsen worker 682 - jepsen.util 682 :ok :add 2969 INFO [2022-01-10 08:39:21,051] jepsen worker 426 - jepsen.util 426 :ok :add 2959 INFO [2022-01-10 08:39:21,052] jepsen worker 774 - jepsen.util 774 :invoke :add 2973 INFO [2022-01-10 08:39:21,055] jepsen worker 118 - jepsen.util 118 :invoke :add 2974 INFO [2022-01-10 08:39:21,056] jepsen worker 643 - jepsen.util 643 :ok :add 2970 INFO [2022-01-10 08:39:21,058] jepsen worker 585 - jepsen.util 585 :invoke :add 2975 INFO [2022-01-10 08:39:21,059] jepsen worker 560 - jepsen.util 560 :ok :add 2971 INFO [2022-01-10 08:39:21,060] jepsen worker 681 - jepsen.util 681 :ok :add 2972 INFO [2022-01-10 08:39:21,061] jepsen worker 95 - jepsen.util 95 :invoke :add 2976 INFO [2022-01-10 08:39:21,062] jepsen worker 418 - jepsen.util 418 :ok :add 2838 INFO [2022-01-10 08:39:21,064] jepsen worker 752 - jepsen.util 752 :ok :add 2958 INFO [2022-01-10 08:39:21,064] jepsen worker 774 - jepsen.util 774 :ok :add 2973 INFO [2022-01-10 08:39:21,065] jepsen worker 428 - jepsen.util 428 :invoke :add 2977 INFO [2022-01-10 08:39:21,066] jepsen worker 585 - jepsen.util 585 :ok :add 2975 INFO [2022-01-10 08:39:21,067] jepsen worker 926 - jepsen.util 926 :invoke :add 2978 INFO [2022-01-10 08:39:21,071] jepsen worker 244 - jepsen.util 244 :invoke :add 2979 INFO [2022-01-10 08:39:21,072] jepsen worker 95 - jepsen.util 95 :ok :add 2976 INFO [2022-01-10 08:39:21,074] jepsen worker 45 - jepsen.util 45 :invoke :add 2980 INFO [2022-01-10 08:39:21,075] jepsen worker 428 - jepsen.util 428 :ok :add 2977 INFO [2022-01-10 08:39:21,077] jepsen worker 701 - jepsen.util 701 :invoke :add 2981 INFO [2022-01-10 08:39:21,077] jepsen worker 118 - jepsen.util 118 :ok :add 2974 INFO [2022-01-10 08:39:21,079] jepsen worker 833 - jepsen.util 833 :invoke :add 2982 INFO [2022-01-10 08:39:21,082] jepsen worker 387 - jepsen.util 387 :invoke :add 2983 INFO [2022-01-10 08:39:21,084] jepsen worker 894 - jepsen.util 894 :invoke :add 2984 INFO [2022-01-10 08:39:21,085] jepsen worker 388 - jepsen.util 388 :ok :add 2949 INFO [2022-01-10 08:39:21,086] jepsen worker 676 - jepsen.util 676 :ok :add 2951 INFO [2022-01-10 08:39:21,086] jepsen worker 511 - jepsen.util 511 :invoke :add 2985 INFO [2022-01-10 08:39:21,087] jepsen worker 568 - jepsen.util 568 :ok :add 2939 INFO [2022-01-10 08:39:21,087] jepsen worker 447 - jepsen.util 447 :ok :add 2965 INFO [2022-01-10 08:39:21,087] jepsen worker 670 - jepsen.util 670 :ok :add 2915 INFO [2022-01-10 08:39:21,087] jepsen worker 329 - jepsen.util 329 :ok :add 2921 INFO [2022-01-10 08:39:21,088] jepsen worker 40 - jepsen.util 40 :invoke :add 2986 INFO [2022-01-10 08:39:21,091] jepsen worker 384 - jepsen.util 384 :invoke :add 2987 INFO [2022-01-10 08:39:21,093] jepsen worker 231 - jepsen.util 231 :ok :add 2967 INFO [2022-01-10 08:39:21,093] jepsen worker 484 - jepsen.util 484 :invoke :add 2988 INFO [2022-01-10 08:39:21,094] jepsen worker 840 - jepsen.util 840 :ok :add 2953 INFO [2022-01-10 08:39:21,096] jepsen worker 294 - jepsen.util 294 :invoke :add 2989 INFO [2022-01-10 08:39:21,098] jepsen worker 433 - jepsen.util 433 :invoke :add 2990 INFO [2022-01-10 08:39:21,100] jepsen worker 188 - jepsen.util 188 :invoke :add 2991 INFO [2022-01-10 08:39:21,102] jepsen worker 484 - jepsen.util 484 :ok :add 2988 INFO [2022-01-10 08:39:21,103] jepsen worker 166 - jepsen.util 166 :invoke :add 2992 INFO [2022-01-10 08:39:21,106] jepsen worker 294 - jepsen.util 294 :ok :add 2989 INFO [2022-01-10 08:39:21,106] jepsen worker 864 - jepsen.util 864 :invoke :add 2993 INFO [2022-01-10 08:39:21,107] jepsen worker 433 - jepsen.util 433 :ok :add 2990 INFO [2022-01-10 08:39:21,108] jepsen worker 578 - jepsen.util 578 :invoke :add 2994 INFO [2022-01-10 08:39:21,111] jepsen worker 139 - jepsen.util 139 :invoke :add 2995 INFO [2022-01-10 08:39:21,111] jepsen worker 188 - jepsen.util 188 :ok :add 2991 INFO [2022-01-10 08:39:21,113] jepsen worker 100 - jepsen.util 100 :invoke :add 2996 INFO [2022-01-10 08:39:21,115] jepsen worker 842 - jepsen.util 842 :invoke :add 2997 INFO [2022-01-10 08:39:21,116] jepsen worker 166 - jepsen.util 166 :ok :add 2992 INFO [2022-01-10 08:39:21,118] jepsen worker 652 - jepsen.util 652 :invoke :add 2998 INFO [2022-01-10 08:39:21,118] jepsen worker 578 - jepsen.util 578 :ok :add 2994 INFO [2022-01-10 08:39:21,118] jepsen worker 864 - jepsen.util 864 :ok :add 2993 INFO [2022-01-10 08:39:21,118] jepsen worker 139 - jepsen.util 139 :ok :add 2995 INFO [2022-01-10 08:39:21,120] jepsen worker 558 - jepsen.util 558 :invoke :add 2999 INFO [2022-01-10 08:39:21,123] jepsen worker 368 - jepsen.util 368 :invoke :add 3000 INFO [2022-01-10 08:39:21,124] jepsen worker 842 - jepsen.util 842 :ok :add 2997 INFO [2022-01-10 08:39:21,125] jepsen worker 578 - jepsen.util 578 :invoke :add 3001 INFO [2022-01-10 08:39:21,127] jepsen worker 613 - jepsen.util 613 :invoke :add 3002 INFO [2022-01-10 08:39:21,131] jepsen worker 520 - jepsen.util 520 :invoke :add 3003 INFO [2022-01-10 08:39:21,133] jepsen worker 765 - jepsen.util 765 :invoke :add 3004 INFO [2022-01-10 08:39:21,134] jepsen worker 100 - jepsen.util 100 :ok :add 2996 INFO [2022-01-10 08:39:21,136] jepsen worker 273 - jepsen.util 273 :invoke :add 3005 INFO [2022-01-10 08:39:21,137] jepsen worker 578 - jepsen.util 578 :ok :add 3001 INFO [2022-01-10 08:39:21,139] jepsen worker 431 - jepsen.util 431 :invoke :add 3006 INFO [2022-01-10 08:39:21,140] jepsen worker 520 - jepsen.util 520 :ok :add 3003 INFO [2022-01-10 08:39:21,141] jepsen worker 594 - jepsen.util 594 :invoke :add 3007 INFO [2022-01-10 08:39:21,144] jepsen worker 616 - jepsen.util 616 :invoke :add 3008 INFO [2022-01-10 08:39:21,146] jepsen worker 201 - jepsen.util 201 :invoke :add 3009 INFO [2022-01-10 08:39:21,147] jepsen worker 368 - jepsen.util 368 :ok :add 3000 INFO [2022-01-10 08:39:21,148] jepsen worker 137 - jepsen.util 137 :invoke :add 3010 INFO [2022-01-10 08:39:21,151] jepsen worker 613 - jepsen.util 613 :ok :add 3002 INFO [2022-01-10 08:39:21,151] jepsen worker 240 - jepsen.util 240 :invoke :add 3011 INFO [2022-01-10 08:39:21,153] jepsen worker 956 - jepsen.util 956 :invoke :add 3012 INFO [2022-01-10 08:39:21,155] jepsen worker 811 - jepsen.util 811 :invoke :add 3013 INFO [2022-01-10 08:39:21,157] jepsen worker 55 - jepsen.util 55 :invoke :add 3014 INFO [2022-01-10 08:39:21,160] jepsen worker 93 - jepsen.util 93 :invoke :add 3015 INFO [2022-01-10 08:39:21,164] jepsen worker 28 - jepsen.util 28 :invoke :add 3016 INFO [2022-01-10 08:39:21,166] jepsen worker 598 - jepsen.util 598 :invoke :add 3017 INFO [2022-01-10 08:39:21,169] jepsen worker 568 - jepsen.util 568 :invoke :add 3018 INFO [2022-01-10 08:39:21,171] jepsen worker 140 - jepsen.util 140 :invoke :add 3019 INFO [2022-01-10 08:39:21,174] jepsen worker 425 - jepsen.util 425 :invoke :add 3020 INFO [2022-01-10 08:39:21,176] jepsen worker 714 - jepsen.util 714 :invoke :add 3021 INFO [2022-01-10 08:39:21,176] jepsen worker 568 - jepsen.util 568 :ok :add 3018 INFO [2022-01-10 08:39:21,177] jepsen worker 380 - jepsen.util 380 :ok :add 2948 INFO [2022-01-10 08:39:21,178] jepsen worker 410 - jepsen.util 410 :ok :add 2950 INFO [2022-01-10 08:39:21,179] jepsen worker 172 - jepsen.util 172 :ok :add 2964 INFO [2022-01-10 08:39:21,179] jepsen worker 140 - jepsen.util 140 :ok :add 3019 INFO [2022-01-10 08:39:21,180] jepsen worker 304 - jepsen.util 304 :invoke :add 3022 INFO [2022-01-10 08:39:21,180] jepsen worker 616 - jepsen.util 616 :ok :add 3008 INFO [2022-01-10 08:39:21,180] jepsen worker 137 - jepsen.util 137 :ok :add 3010 INFO [2022-01-10 08:39:21,183] jepsen worker 705 - jepsen.util 705 :invoke :add 3023 INFO [2022-01-10 08:39:21,185] jepsen worker 384 - jepsen.util 384 :ok :add 2987 INFO [2022-01-10 08:39:21,185] jepsen worker 894 - jepsen.util 894 :ok :add 2984 INFO [2022-01-10 08:39:21,186] jepsen worker 134 - jepsen.util 134 :invoke :add 3024 INFO [2022-01-10 08:39:21,189] jepsen worker 699 - jepsen.util 699 :invoke :add 3025 INFO [2022-01-10 08:39:21,191] jepsen worker 175 - jepsen.util 175 :invoke :add 3026 INFO [2022-01-10 08:39:21,193] jepsen worker 359 - jepsen.util 359 :invoke :add 3027 INFO [2022-01-10 08:39:21,194] jepsen worker 319 - jepsen.util 319 :ok :add 2952 INFO [2022-01-10 08:39:21,195] jepsen worker 481 - jepsen.util 481 :ok :add 2966 INFO [2022-01-10 08:39:21,196] jepsen worker 371 - jepsen.util 371 :invoke :add 3028 INFO [2022-01-10 08:39:21,199] jepsen worker 22 - jepsen.util 22 :invoke :add 3029 INFO [2022-01-10 08:39:21,202] jepsen worker 811 - jepsen.util 811 :ok :add 3013 INFO [2022-01-10 08:39:21,202] jepsen worker 359 - jepsen.util 359 :ok :add 3027 INFO [2022-01-10 08:39:21,202] jepsen worker 492 - jepsen.util 492 :invoke :add 3030 INFO [2022-01-10 08:39:21,204] jepsen worker 40 - jepsen.util 40 :ok :add 2986 INFO [2022-01-10 08:39:21,205] jepsen worker 338 - jepsen.util 338 :invoke :add 3031 INFO [2022-01-10 08:39:21,207] jepsen worker 959 - jepsen.util 959 :invoke :add 3032 INFO [2022-01-10 08:39:21,209] jepsen worker 853 - jepsen.util 853 :invoke :add 3033 INFO [2022-01-10 08:39:21,211] jepsen worker 497 - jepsen.util 497 :invoke :add 3034 INFO [2022-01-10 08:39:21,214] jepsen worker 12 - jepsen.util 12 :invoke :add 3035 INFO [2022-01-10 08:39:21,217] jepsen worker 788 - jepsen.util 788 :invoke :add 3036 INFO [2022-01-10 08:39:21,220] jepsen worker 436 - jepsen.util 436 :invoke :add 3037 INFO [2022-01-10 08:39:21,220] jepsen worker 497 - jepsen.util 497 :ok :add 3034 INFO [2022-01-10 08:39:21,221] jepsen worker 12 - jepsen.util 12 :ok :add 3035 INFO [2022-01-10 08:39:21,222] jepsen worker 640 - jepsen.util 640 :invoke :add 3038 INFO [2022-01-10 08:39:21,226] jepsen worker 979 - jepsen.util 979 :invoke :add 3039 INFO [2022-01-10 08:39:21,227] jepsen worker 765 - jepsen.util 765 :ok :add 3004 INFO [2022-01-10 08:39:21,228] jepsen worker 69 - jepsen.util 69 :invoke :add 3040 INFO [2022-01-10 08:39:21,229] jepsen worker 788 - jepsen.util 788 :ok :add 3036 INFO [2022-01-10 08:39:21,231] jepsen worker 547 - jepsen.util 547 :invoke :add 3041 INFO [2022-01-10 08:39:21,231] jepsen worker 436 - jepsen.util 436 :ok :add 3037 INFO [2022-01-10 08:39:21,234] jepsen worker 937 - jepsen.util 937 :invoke :add 3042 INFO [2022-01-10 08:39:21,236] jepsen worker 856 - jepsen.util 856 :invoke :add 3043 INFO [2022-01-10 08:39:21,239] jepsen worker 345 - jepsen.util 345 :invoke :add 3044 INFO [2022-01-10 08:39:21,242] jepsen worker 472 - jepsen.util 472 :invoke :add 3045 INFO [2022-01-10 08:39:21,243] jepsen worker 959 - jepsen.util 959 :ok :add 3032 INFO [2022-01-10 08:39:21,243] jepsen worker 431 - jepsen.util 431 :ok :add 3006 INFO [2022-01-10 08:39:21,243] jepsen worker 273 - jepsen.util 273 :ok :add 3005 INFO [2022-01-10 08:39:21,243] jepsen worker 22 - jepsen.util 22 :ok :add 3029 INFO [2022-01-10 08:39:21,243] jepsen worker 338 - jepsen.util 338 :ok :add 3031 INFO [2022-01-10 08:39:21,245] jepsen worker 50 - jepsen.util 50 :invoke :add 3046 INFO [2022-01-10 08:39:21,248] jepsen worker 141 - jepsen.util 141 :invoke :add 3047 INFO [2022-01-10 08:39:21,250] jepsen worker 472 - jepsen.util 472 :ok :add 3045 INFO [2022-01-10 08:39:21,250] jepsen worker 345 - jepsen.util 345 :ok :add 3044 INFO [2022-01-10 08:39:21,250] jepsen worker 699 - jepsen.util 699 :ok :add 3025 INFO [2022-01-10 08:39:21,250] jepsen worker 483 - jepsen.util 483 :invoke :add 3048 INFO [2022-01-10 08:39:21,250] jepsen worker 201 - jepsen.util 201 :ok :add 3009 INFO [2022-01-10 08:39:21,250] jepsen worker 240 - jepsen.util 240 :ok :add 3011 INFO [2022-01-10 08:39:21,254] jepsen worker 601 - jepsen.util 601 :invoke :add 3049 INFO [2022-01-10 08:39:21,255] jepsen worker 511 - jepsen.util 511 :ok :add 2985 INFO [2022-01-10 08:39:21,256] jepsen worker 27 - jepsen.util 27 :invoke :add 3050 INFO [2022-01-10 08:39:21,256] jepsen worker 956 - jepsen.util 956 :ok :add 3012 INFO [2022-01-10 08:39:21,257] jepsen worker 50 - jepsen.util 50 :ok :add 3046 INFO [2022-01-10 08:39:21,257] jepsen worker 175 - jepsen.util 175 :ok :add 3026 INFO [2022-01-10 08:39:21,260] jepsen worker 141 - jepsen.util 141 :ok :add 3047 INFO [2022-01-10 08:39:21,260] jepsen worker 777 - jepsen.util 777 :invoke :add 3051 INFO [2022-01-10 08:39:21,261] jepsen worker 134 - jepsen.util 134 :ok :add 3024 INFO [2022-01-10 08:39:21,263] jepsen worker 450 - jepsen.util 450 :invoke :add 3052 INFO [2022-01-10 08:39:21,265] jepsen worker 808 - jepsen.util 808 :invoke :add 3053 INFO [2022-01-10 08:39:21,268] jepsen worker 722 - jepsen.util 722 :invoke :add 3054 INFO [2022-01-10 08:39:21,271] jepsen worker 478 - jepsen.util 478 :invoke :add 3055 INFO [2022-01-10 08:39:21,273] jepsen worker 99 - jepsen.util 99 :invoke :add 3056 INFO [2022-01-10 08:39:21,275] jepsen worker 541 - jepsen.util 541 :invoke :add 3057 INFO [2022-01-10 08:39:21,278] jepsen worker 534 - jepsen.util 534 :invoke :add 3058 INFO [2022-01-10 08:39:21,280] jepsen worker 752 - jepsen.util 752 :invoke :add 3059 INFO [2022-01-10 08:39:21,282] jepsen worker 823 - jepsen.util 823 :invoke :add 3060 INFO [2022-01-10 08:39:21,284] jepsen worker 907 - jepsen.util 907 :invoke :add 3061 INFO [2022-01-10 08:39:21,287] jepsen worker 395 - jepsen.util 395 :invoke :add 3062 INFO [2022-01-10 08:39:21,289] jepsen worker 600 - jepsen.util 600 :invoke :add 3063 INFO [2022-01-10 08:39:21,290] jepsen worker 853 - jepsen.util 853 :ok :add 3033 INFO [2022-01-10 08:39:21,290] jepsen worker 594 - jepsen.util 594 :ok :add 3007 INFO [2022-01-10 08:39:21,292] jepsen worker 672 - jepsen.util 672 :invoke :add 3064 INFO [2022-01-10 08:39:21,294] jepsen worker 152 - jepsen.util 152 :invoke :add 3065 INFO [2022-01-10 08:39:21,296] jepsen worker 371 - jepsen.util 371 :ok :add 3028 INFO [2022-01-10 08:39:21,296] jepsen worker 492 - jepsen.util 492 :ok :add 3030 INFO [2022-01-10 08:39:21,297] jepsen worker 495 - jepsen.util 495 :invoke :add 3066 INFO [2022-01-10 08:39:21,299] jepsen worker 547 - jepsen.util 547 :ok :add 3041 INFO [2022-01-10 08:39:21,300] jepsen worker 752 - jepsen.util 752 :ok :add 3059 INFO [2022-01-10 08:39:21,300] jepsen worker 291 - jepsen.util 291 :invoke :add 3067 INFO [2022-01-10 08:39:21,303] jepsen worker 20 - jepsen.util 20 :invoke :add 3068 INFO [2022-01-10 08:39:21,304] jepsen worker 856 - jepsen.util 856 :ok :add 3043 INFO [2022-01-10 08:39:21,304] jepsen worker 69 - jepsen.util 69 :ok :add 3040 INFO [2022-01-10 08:39:21,304] jepsen worker 534 - jepsen.util 534 :ok :add 3058 INFO [2022-01-10 08:39:21,305] jepsen worker 419 - jepsen.util 419 :invoke :add 3069 INFO [2022-01-10 08:39:21,308] jepsen worker 764 - jepsen.util 764 :invoke :add 3070 INFO [2022-01-10 08:39:21,310] jepsen worker 937 - jepsen.util 937 :ok :add 3042 INFO [2022-01-10 08:39:21,310] jepsen worker 20 - jepsen.util 20 :ok :add 3068 INFO [2022-01-10 08:39:21,311] jepsen worker 637 - jepsen.util 637 :invoke :add 3071 INFO [2022-01-10 08:39:21,313] jepsen worker 59 - jepsen.util 59 :invoke :add 3072 INFO [2022-01-10 08:39:21,315] jepsen worker 823 - jepsen.util 823 :ok :add 3060 INFO [2022-01-10 08:39:21,315] jepsen worker 722 - jepsen.util 722 :ok :add 3054 INFO [2022-01-10 08:39:21,315] jepsen worker 419 - jepsen.util 419 :ok :add 3069 INFO [2022-01-10 08:39:21,316] jepsen worker 168 - jepsen.util 168 :invoke :add 3073 INFO [2022-01-10 08:39:21,320] jepsen worker 313 - jepsen.util 313 :invoke :add 3074 INFO [2022-01-10 08:39:21,322] jepsen worker 764 - jepsen.util 764 :ok :add 3070 INFO [2022-01-10 08:39:21,322] jepsen worker 637 - jepsen.util 637 :ok :add 3071 INFO [2022-01-10 08:39:21,322] jepsen worker 336 - jepsen.util 336 :invoke :add 3075 INFO [2022-01-10 08:39:21,324] jepsen worker 773 - jepsen.util 773 :invoke :add 3076 INFO [2022-01-10 08:39:21,326] jepsen worker 168 - jepsen.util 168 :ok :add 3073 INFO [2022-01-10 08:39:21,326] jepsen worker 59 - jepsen.util 59 :ok :add 3072 INFO [2022-01-10 08:39:21,327] jepsen worker 406 - jepsen.util 406 :invoke :add 3077 INFO [2022-01-10 08:39:21,330] jepsen worker 487 - jepsen.util 487 :invoke :add 3078 INFO [2022-01-10 08:39:21,331] jepsen worker 313 - jepsen.util 313 :ok :add 3074 INFO [2022-01-10 08:39:21,331] jepsen worker 395 - jepsen.util 395 :ok :add 3062 INFO [2022-01-10 08:39:21,331] jepsen worker 99 - jepsen.util 99 :ok :add 3056 INFO [2022-01-10 08:39:21,332] jepsen worker 170 - jepsen.util 170 :invoke :add 3079 INFO [2022-01-10 08:39:21,334] jepsen worker 478 - jepsen.util 478 :ok :add 3055 INFO [2022-01-10 08:39:21,334] jepsen worker 907 - jepsen.util 907 :ok :add 3061 INFO [2022-01-10 08:39:21,336] jepsen worker 773 - jepsen.util 773 :ok :add 3076 INFO [2022-01-10 08:39:21,336] jepsen worker 447 - jepsen.util 447 :invoke :add 3080 INFO [2022-01-10 08:39:21,340] jepsen worker 626 - jepsen.util 626 :invoke :add 3081 INFO [2022-01-10 08:39:21,342] jepsen worker 336 - jepsen.util 336 :ok :add 3075 INFO [2022-01-10 08:39:21,343] jepsen worker 339 - jepsen.util 339 :invoke :add 3082 INFO [2022-01-10 08:39:21,345] jepsen worker 541 - jepsen.util 541 :ok :add 3057 INFO [2022-01-10 08:39:21,345] jepsen worker 600 - jepsen.util 600 :ok :add 3063 INFO [2022-01-10 08:39:21,346] jepsen worker 406 - jepsen.util 406 :ok :add 3077 INFO [2022-01-10 08:39:21,347] jepsen worker 142 - jepsen.util 142 :invoke :add 3083 INFO [2022-01-10 08:39:21,351] jepsen worker 396 - jepsen.util 396 :invoke :add 3084 INFO [2022-01-10 08:39:21,353] jepsen worker 43 - jepsen.util 43 :invoke :add 3085 INFO [2022-01-10 08:39:21,354] jepsen worker 626 - jepsen.util 626 :ok :add 3081 INFO [2022-01-10 08:39:21,356] jepsen worker 625 - jepsen.util 625 :invoke :add 3086 INFO [2022-01-10 08:39:21,357] jepsen worker 170 - jepsen.util 170 :ok :add 3079 INFO [2022-01-10 08:39:21,358] jepsen worker 113 - jepsen.util 113 :invoke :add 3087 INFO [2022-01-10 08:39:21,359] jepsen worker 487 - jepsen.util 487 :ok :add 3078 INFO [2022-01-10 08:39:21,361] jepsen worker 761 - jepsen.util 761 :invoke :add 3088 INFO [2022-01-10 08:39:21,364] jepsen worker 447 - jepsen.util 447 :ok :add 3080 INFO [2022-01-10 08:39:21,364] jepsen worker 746 - jepsen.util 746 :invoke :add 3089 INFO [2022-01-10 08:39:21,365] jepsen worker 142 - jepsen.util 142 :ok :add 3083 INFO [2022-01-10 08:39:21,367] jepsen worker 119 - jepsen.util 119 :invoke :add 3090 INFO [2022-01-10 08:39:21,368] jepsen worker 761 - jepsen.util 761 :ok :add 3088 INFO [2022-01-10 08:39:21,370] jepsen worker 998 - jepsen.util 998 :invoke :add 3091 INFO [2022-01-10 08:39:21,373] jepsen worker 746 - jepsen.util 746 :ok :add 3089 INFO [2022-01-10 08:39:21,373] jepsen worker 532 - jepsen.util 532 :invoke :add 3092 INFO [2022-01-10 08:39:21,376] jepsen worker 998 - jepsen.util 998 :ok :add 3091 INFO [2022-01-10 08:39:21,376] jepsen worker 554 - jepsen.util 554 :invoke :add 3093 INFO [2022-01-10 08:39:21,377] jepsen worker 119 - jepsen.util 119 :ok :add 3090 INFO [2022-01-10 08:39:21,378] jepsen worker 339 - jepsen.util 339 :ok :add 3082 INFO [2022-01-10 08:39:21,379] jepsen worker 620 - jepsen.util 620 :invoke :add 3094 INFO [2022-01-10 08:39:21,380] jepsen worker 532 - jepsen.util 532 :ok :add 3092 INFO [2022-01-10 08:39:21,382] jepsen worker 781 - jepsen.util 781 :invoke :add 3095 INFO [2022-01-10 08:39:21,385] jepsen worker 554 - jepsen.util 554 :ok :add 3093 INFO [2022-01-10 08:39:21,385] jepsen worker 481 - jepsen.util 481 :invoke :add 3096 INFO [2022-01-10 08:39:21,388] jepsen worker 231 - jepsen.util 231 :invoke :add 3097 INFO [2022-01-10 08:39:21,390] jepsen worker 273 - jepsen.util 273 :invoke :add 3098 INFO [2022-01-10 08:39:21,391] jepsen worker 620 - jepsen.util 620 :ok :add 3094 INFO [2022-01-10 08:39:21,391] jepsen worker 781 - jepsen.util 781 :ok :add 3095 INFO [2022-01-10 08:39:21,393] jepsen worker 428 - jepsen.util 428 :invoke :add 3099 INFO [2022-01-10 08:39:21,395] jepsen worker 231 - jepsen.util 231 :ok :add 3097 INFO [2022-01-10 08:39:21,396] jepsen worker 692 - jepsen.util 692 :invoke :add 3100 INFO [2022-01-10 08:39:21,398] jepsen worker 357 - jepsen.util 357 :invoke :add 3101 INFO [2022-01-10 08:39:21,399] jepsen worker 481 - jepsen.util 481 :ok :add 3096 INFO [2022-01-10 08:39:21,400] jepsen worker 342 - jepsen.util 342 :invoke :add 3102 INFO [2022-01-10 08:39:21,403] jepsen worker 57 - jepsen.util 57 :invoke :add 3103 INFO [2022-01-10 08:39:21,404] jepsen worker 692 - jepsen.util 692 :ok :add 3100 INFO [2022-01-10 08:39:21,406] jepsen worker 419 - jepsen.util 419 :invoke :add 3104 INFO [2022-01-10 08:39:21,408] jepsen worker 357 - jepsen.util 357 :ok :add 3101 INFO [2022-01-10 08:39:21,408] jepsen worker 342 - jepsen.util 342 :ok :add 3102 INFO [2022-01-10 08:39:21,408] jepsen worker 353 - jepsen.util 353 :invoke :add 3105 INFO [2022-01-10 08:39:21,411] jepsen worker 528 - jepsen.util 528 :invoke :add 3106 INFO [2022-01-10 08:39:21,412] jepsen worker 57 - jepsen.util 57 :ok :add 3103 INFO [2022-01-10 08:39:21,414] jepsen worker 263 - jepsen.util 263 :invoke :add 3107 INFO [2022-01-10 08:39:21,415] jepsen worker 419 - jepsen.util 419 :ok :add 3104 INFO [2022-01-10 08:39:21,417] jepsen worker 597 - jepsen.util 597 :invoke :add 3108 INFO [2022-01-10 08:39:21,419] jepsen worker 342 - jepsen.util 342 :invoke :add 3109 INFO [2022-01-10 08:39:21,421] jepsen worker 528 - jepsen.util 528 :ok :add 3106 INFO [2022-01-10 08:39:21,422] jepsen worker 83 - jepsen.util 83 :invoke :add 3110 INFO [2022-01-10 08:39:21,423] jepsen worker 428 - jepsen.util 428 :ok :add 3099 INFO [2022-01-10 08:39:21,424] jepsen worker 353 - jepsen.util 353 :ok :add 3105 INFO [2022-01-10 08:39:21,424] jepsen worker 100 - jepsen.util 100 :invoke :add 3111 INFO [2022-01-10 08:39:21,427] jepsen worker 108 - jepsen.util 108 :invoke :add 3112 INFO [2022-01-10 08:39:21,428] jepsen worker 597 - jepsen.util 597 :ok :add 3108 INFO [2022-01-10 08:39:21,429] jepsen worker 273 - jepsen.util 273 :ok :add 3098 INFO [2022-01-10 08:39:21,429] jepsen worker 263 - jepsen.util 263 :ok :add 3107 INFO [2022-01-10 08:39:21,429] jepsen worker 451 - jepsen.util 451 :invoke :add 3113 INFO [2022-01-10 08:39:21,431] jepsen worker 342 - jepsen.util 342 :ok :add 3109 INFO [2022-01-10 08:39:21,432] jepsen worker 913 - jepsen.util 913 :invoke :add 3114 INFO [2022-01-10 08:39:21,435] jepsen worker 536 - jepsen.util 536 :invoke :add 3115 INFO [2022-01-10 08:39:21,438] jepsen worker 630 - jepsen.util 630 :invoke :add 3116 INFO [2022-01-10 08:39:21,439] jepsen worker 108 - jepsen.util 108 :ok :add 3112 INFO [2022-01-10 08:39:21,439] jepsen worker 100 - jepsen.util 100 :ok :add 3111 INFO [2022-01-10 08:39:21,441] jepsen worker 79 - jepsen.util 79 :invoke :add 3117 INFO [2022-01-10 08:39:21,442] jepsen worker 451 - jepsen.util 451 :ok :add 3113 INFO [2022-01-10 08:39:21,443] jepsen worker 102 - jepsen.util 102 :invoke :add 3118 INFO [2022-01-10 08:39:21,446] jepsen worker 83 - jepsen.util 83 :ok :add 3110 INFO [2022-01-10 08:39:21,446] jepsen worker 978 - jepsen.util 978 :invoke :add 3119 INFO [2022-01-10 08:39:21,448] jepsen worker 556 - jepsen.util 556 :invoke :add 3120 INFO [2022-01-10 08:39:21,449] jepsen worker 102 - jepsen.util 102 :ok :add 3118 INFO [2022-01-10 08:39:21,451] jepsen worker 471 - jepsen.util 471 :invoke :add 3121 INFO [2022-01-10 08:39:21,453] jepsen worker 290 - jepsen.util 290 :invoke :add 3122 INFO [2022-01-10 08:39:21,454] jepsen worker 978 - jepsen.util 978 :ok :add 3119 INFO [2022-01-10 08:39:21,456] jepsen worker 270 - jepsen.util 270 :invoke :add 3123 INFO [2022-01-10 08:39:21,458] jepsen worker 616 - jepsen.util 616 :invoke :add 3124 INFO [2022-01-10 08:39:21,461] jepsen worker 279 - jepsen.util 279 :invoke :add 3125 INFO [2022-01-10 08:39:21,464] jepsen worker 781 - jepsen.util 781 :invoke :add 3126 INFO [2022-01-10 08:39:21,466] jepsen worker 141 - jepsen.util 141 :invoke :add 3127 INFO [2022-01-10 08:39:21,467] jepsen worker 616 - jepsen.util 616 :ok :add 3124 INFO [2022-01-10 08:39:21,468] jepsen worker 720 - jepsen.util 720 :invoke :add 3128 INFO [2022-01-10 08:39:21,470] jepsen worker 281 - jepsen.util 281 :invoke :add 3129 INFO [2022-01-10 08:39:21,471] jepsen worker 279 - jepsen.util 279 :ok :add 3125 INFO [2022-01-10 08:39:21,473] jepsen worker 249 - jepsen.util 249 :invoke :add 3130 INFO [2022-01-10 08:39:21,475] jepsen worker 141 - jepsen.util 141 :ok :add 3127 INFO [2022-01-10 08:39:21,476] jepsen worker 150 - jepsen.util 150 :invoke :add 3131 INFO [2022-01-10 08:39:21,476] jepsen worker 720 - jepsen.util 720 :ok :add 3128 INFO [2022-01-10 08:39:21,477] jepsen worker 781 - jepsen.util 781 :ok :add 3126 INFO [2022-01-10 08:39:21,478] jepsen worker 957 - jepsen.util 957 :invoke :add 3132 INFO [2022-01-10 08:39:21,481] jepsen worker 86 - jepsen.util 86 :invoke :add 3133 INFO [2022-01-10 08:39:21,483] jepsen worker 548 - jepsen.util 548 :invoke :add 3134 INFO [2022-01-10 08:39:21,485] jepsen worker 779 - jepsen.util 779 :invoke :add 3135 INFO [2022-01-10 08:39:21,488] jepsen worker 662 - jepsen.util 662 :invoke :add 3136 INFO [2022-01-10 08:39:21,489] jepsen worker 281 - jepsen.util 281 :ok :add 3129 INFO [2022-01-10 08:39:21,490] jepsen worker 546 - jepsen.util 546 :invoke :add 3137 INFO [2022-01-10 08:39:21,493] jepsen worker 548 - jepsen.util 548 :ok :add 3134 INFO [2022-01-10 08:39:21,493] jepsen worker 127 - jepsen.util 127 :invoke :add 3138 INFO [2022-01-10 08:39:21,494] jepsen worker 779 - jepsen.util 779 :ok :add 3135 INFO [2022-01-10 08:39:21,494] jepsen worker 249 - jepsen.util 249 :ok :add 3130 INFO [2022-01-10 08:39:21,496] jepsen worker 150 - jepsen.util 150 :ok :add 3131 INFO [2022-01-10 08:39:21,496] jepsen worker 753 - jepsen.util 753 :invoke :add 3139 INFO [2022-01-10 08:39:21,499] jepsen worker 554 - jepsen.util 554 :invoke :add 3140 INFO [2022-01-10 08:39:21,500] jepsen worker 957 - jepsen.util 957 :ok :add 3132 INFO [2022-01-10 08:39:21,501] jepsen worker 86 - jepsen.util 86 :ok :add 3133 INFO [2022-01-10 08:39:21,501] jepsen worker 546 - jepsen.util 546 :ok :add 3137 INFO [2022-01-10 08:39:21,502] jepsen worker 822 - jepsen.util 822 :invoke :add 3141 INFO [2022-01-10 08:39:21,506] jepsen worker 252 - jepsen.util 252 :invoke :add 3142 INFO [2022-01-10 08:39:21,507] jepsen worker 662 - jepsen.util 662 :ok :add 3136 INFO [2022-01-10 08:39:21,509] jepsen worker 497 - jepsen.util 497 :invoke :add 3143 INFO [2022-01-10 08:39:21,512] jepsen worker 893 - jepsen.util 893 :invoke :add 3144 INFO [2022-01-10 08:39:21,514] jepsen worker 854 - jepsen.util 854 :invoke :add 3145 INFO [2022-01-10 08:39:21,516] jepsen worker 683 - jepsen.util 683 :invoke :add 3146 INFO [2022-01-10 08:39:21,518] jepsen worker 554 - jepsen.util 554 :ok :add 3140 INFO [2022-01-10 08:39:21,519] jepsen worker 822 - jepsen.util 822 :ok :add 3141 INFO [2022-01-10 08:39:21,519] jepsen worker 760 - jepsen.util 760 :invoke :add 3147 INFO [2022-01-10 08:39:21,522] jepsen worker 819 - jepsen.util 819 :invoke :add 3148 INFO [2022-01-10 08:39:21,524] jepsen worker 497 - jepsen.util 497 :ok :add 3143 INFO [2022-01-10 08:39:21,524] jepsen worker 338 - jepsen.util 338 :invoke :add 3149 INFO [2022-01-10 08:39:21,527] jepsen worker 916 - jepsen.util 916 :invoke :add 3150 INFO [2022-01-10 08:39:21,528] jepsen worker 252 - jepsen.util 252 :ok :add 3142 INFO [2022-01-10 08:39:21,529] jepsen worker 515 - jepsen.util 515 :invoke :add 3151 INFO [2022-01-10 08:39:21,532] jepsen worker 123 - jepsen.util 123 :invoke :add 3152 INFO [2022-01-10 08:39:21,535] jepsen worker 900 - jepsen.util 900 :invoke :add 3153 INFO [2022-01-10 08:39:21,537] jepsen worker 964 - jepsen.util 964 :invoke :add 3154 INFO [2022-01-10 08:39:21,544] jepsen worker 420 - jepsen.util 420 :invoke :add 3155 INFO [2022-01-10 08:39:21,544] jepsen worker 854 - jepsen.util 854 :ok :add 3145 INFO [2022-01-10 08:39:21,544] jepsen worker 893 - jepsen.util 893 :ok :add 3144 INFO [2022-01-10 08:39:21,547] jepsen worker 384 - jepsen.util 384 :invoke :add 3156 INFO [2022-01-10 08:39:21,548] jepsen worker 760 - jepsen.util 760 :ok :add 3147 INFO [2022-01-10 08:39:21,548] jepsen worker 683 - jepsen.util 683 :ok :add 3146 INFO [2022-01-10 08:39:21,550] jepsen worker 671 - jepsen.util 671 :invoke :add 3157 INFO [2022-01-10 08:39:21,553] jepsen worker 572 - jepsen.util 572 :invoke :add 3158 INFO [2022-01-10 08:39:21,556] jepsen worker 775 - jepsen.util 775 :invoke :add 3159 INFO [2022-01-10 08:39:21,558] jepsen worker 205 - jepsen.util 205 :invoke :add 3160 INFO [2022-01-10 08:39:21,560] jepsen worker 964 - jepsen.util 964 :ok :add 3154 INFO [2022-01-10 08:39:21,560] jepsen worker 572 - jepsen.util 572 :ok :add 3158 INFO [2022-01-10 08:39:21,560] jepsen worker 420 - jepsen.util 420 :ok :add 3155 INFO [2022-01-10 08:39:21,562] jepsen worker 682 - jepsen.util 682 :invoke :add 3161 INFO [2022-01-10 08:39:21,563] jepsen worker 775 - jepsen.util 775 :ok :add 3159 INFO [2022-01-10 08:39:21,564] jepsen worker 671 - jepsen.util 671 :ok :add 3157 INFO [2022-01-10 08:39:21,565] jepsen worker 521 - jepsen.util 521 :invoke :add 3162 INFO [2022-01-10 08:39:21,568] jepsen worker 932 - jepsen.util 932 :invoke :add 3163 INFO [2022-01-10 08:39:21,568] jepsen worker 682 - jepsen.util 682 :ok :add 3161 INFO [2022-01-10 08:39:21,570] jepsen worker 403 - jepsen.util 403 :invoke :add 3164 INFO [2022-01-10 08:39:21,573] jepsen worker 822 - jepsen.util 822 :invoke :add 3165 INFO [2022-01-10 08:39:21,573] jepsen worker 384 - jepsen.util 384 :ok :add 3156 INFO [2022-01-10 08:39:21,576] jepsen worker 163 - jepsen.util 163 :invoke :add 3166 INFO [2022-01-10 08:39:21,576] jepsen worker 205 - jepsen.util 205 :ok :add 3160 INFO [2022-01-10 08:39:21,578] jepsen worker 622 - jepsen.util 622 :invoke :add 3167 INFO [2022-01-10 08:39:21,579] jepsen worker 932 - jepsen.util 932 :ok :add 3163 INFO [2022-01-10 08:39:21,580] jepsen worker 521 - jepsen.util 521 :ok :add 3162 INFO [2022-01-10 08:39:21,580] jepsen worker 473 - jepsen.util 473 :invoke :add 3168 INFO [2022-01-10 08:39:21,583] jepsen worker 461 - jepsen.util 461 :invoke :add 3169 INFO [2022-01-10 08:39:21,585] jepsen worker 952 - jepsen.util 952 :invoke :add 3170 INFO [2022-01-10 08:39:21,587] jepsen worker 315 - jepsen.util 315 :invoke :add 3171 INFO [2022-01-10 08:39:21,589] jepsen worker 582 - jepsen.util 582 :invoke :add 3172 INFO [2022-01-10 08:39:21,591] jepsen worker 461 - jepsen.util 461 :ok :add 3169 INFO [2022-01-10 08:39:21,591] jepsen worker 473 - jepsen.util 473 :ok :add 3168 INFO [2022-01-10 08:39:21,592] jepsen worker 990 - jepsen.util 990 :invoke :add 3173 INFO [2022-01-10 08:39:21,595] jepsen worker 570 - jepsen.util 570 :invoke :add 3174 INFO [2022-01-10 08:39:21,598] jepsen worker 3 - jepsen.util 3 :invoke :add 3175 INFO [2022-01-10 08:39:21,600] jepsen worker 952 - jepsen.util 952 :ok :add 3170 INFO [2022-01-10 08:39:21,600] jepsen worker 315 - jepsen.util 315 :ok :add 3171 INFO [2022-01-10 08:39:21,601] jepsen worker 811 - jepsen.util 811 :invoke :add 3176 INFO [2022-01-10 08:39:21,605] jepsen worker 564 - jepsen.util 564 :invoke :add 3177 INFO [2022-01-10 08:39:21,606] jepsen worker 582 - jepsen.util 582 :ok :add 3172 INFO [2022-01-10 08:39:21,607] jepsen worker 990 - jepsen.util 990 :ok :add 3173 INFO [2022-01-10 08:39:21,608] jepsen worker 783 - jepsen.util 783 :invoke :add 3178 INFO [2022-01-10 08:39:21,611] jepsen worker 954 - jepsen.util 954 :invoke :add 3179 INFO [2022-01-10 08:39:21,613] jepsen worker 544 - jepsen.util 544 :invoke :add 3180 INFO [2022-01-10 08:39:21,616] jepsen worker 538 - jepsen.util 538 :invoke :add 3181 INFO [2022-01-10 08:39:21,619] jepsen worker 638 - jepsen.util 638 :invoke :add 3182 INFO [2022-01-10 08:39:21,621] jepsen worker 703 - jepsen.util 703 :invoke :add 3183 INFO [2022-01-10 08:39:21,624] jepsen worker 911 - jepsen.util 911 :invoke :add 3184 INFO [2022-01-10 08:39:21,626] jepsen worker 688 - jepsen.util 688 :invoke :add 3185 INFO [2022-01-10 08:39:21,628] jepsen worker 685 - jepsen.util 685 :invoke :add 3186 INFO [2022-01-10 08:39:21,631] jepsen worker 944 - jepsen.util 944 :invoke :add 3187 INFO [2022-01-10 08:39:21,633] jepsen worker 95 - jepsen.util 95 :invoke :add 3188 INFO [2022-01-10 08:39:21,635] jepsen worker 413 - jepsen.util 413 :invoke :add 3189 INFO [2022-01-10 08:39:21,637] jepsen worker 570 - jepsen.util 570 :ok :add 3174 INFO [2022-01-10 08:39:21,637] jepsen worker 3 - jepsen.util 3 :ok :add 3175 INFO [2022-01-10 08:39:21,640] jepsen worker 384 - jepsen.util 384 :invoke :add 3190 INFO [2022-01-10 08:39:21,640] jepsen worker 811 - jepsen.util 811 :ok :add 3176 INFO [2022-01-10 08:39:21,642] jepsen worker 954 - jepsen.util 954 :ok :add 3179 INFO [2022-01-10 08:39:21,643] jepsen worker 564 - jepsen.util 564 :ok :add 3177 INFO [2022-01-10 08:39:21,643] jepsen worker 498 - jepsen.util 498 :invoke :add 3191 INFO [2022-01-10 08:39:21,646] jepsen worker 783 - jepsen.util 783 :ok :add 3178 INFO [2022-01-10 08:39:21,646] jepsen worker 544 - jepsen.util 544 :ok :add 3180 INFO [2022-01-10 08:39:21,647] jepsen worker 664 - jepsen.util 664 :invoke :add 3192 INFO [2022-01-10 08:39:21,650] jepsen worker 638 - jepsen.util 638 :ok :add 3182 INFO [2022-01-10 08:39:21,651] jepsen worker 36 - jepsen.util 36 :invoke :add 3193 INFO [2022-01-10 08:39:21,652] jepsen worker 538 - jepsen.util 538 :ok :add 3181 INFO [2022-01-10 08:39:21,654] jepsen worker 780 - jepsen.util 780 :invoke :add 3194 INFO [2022-01-10 08:39:21,656] jepsen worker 703 - jepsen.util 703 :ok :add 3183 INFO [2022-01-10 08:39:21,658] jepsen worker 871 - jepsen.util 871 :invoke :add 3195 INFO [2022-01-10 08:39:21,661] jepsen worker 839 - jepsen.util 839 :invoke :add 3196 INFO [2022-01-10 08:39:21,664] jepsen worker 642 - jepsen.util 642 :invoke :add 3197 INFO [2022-01-10 08:39:21,667] jepsen worker 413 - jepsen.util 413 :ok :add 3189 INFO [2022-01-10 08:39:21,667] jepsen worker 710 - jepsen.util 710 :invoke :add 3198 INFO [2022-01-10 08:39:21,669] jepsen worker 871 - jepsen.util 871 :ok :add 3195 INFO [2022-01-10 08:39:21,670] jepsen worker 780 - jepsen.util 780 :ok :add 3194 INFO [2022-01-10 08:39:21,671] jepsen worker 641 - jepsen.util 641 :invoke :add 3199 INFO [2022-01-10 08:39:21,672] jepsen worker 95 - jepsen.util 95 :ok :add 3188 INFO [2022-01-10 08:39:21,672] jepsen worker 384 - jepsen.util 384 :ok :add 3190 INFO [2022-01-10 08:39:21,672] jepsen worker 642 - jepsen.util 642 :ok :add 3197 INFO [2022-01-10 08:39:21,674] jepsen worker 35 - jepsen.util 35 :invoke :add 3200 INFO [2022-01-10 08:39:21,676] jepsen worker 664 - jepsen.util 664 :ok :add 3192 INFO [2022-01-10 08:39:21,676] jepsen worker 839 - jepsen.util 839 :ok :add 3196 INFO [2022-01-10 08:39:21,678] jepsen worker 710 - jepsen.util 710 :ok :add 3198 INFO [2022-01-10 08:39:21,678] jepsen worker 543 - jepsen.util 543 :invoke :add 3201 INFO [2022-01-10 08:39:21,681] jepsen worker 269 - jepsen.util 269 :invoke :add 3202 INFO [2022-01-10 08:39:21,681] jepsen worker 498 - jepsen.util 498 :ok :add 3191 INFO [2022-01-10 08:39:21,684] jepsen worker 183 - jepsen.util 183 :invoke :add 3203 INFO [2022-01-10 08:39:21,686] jepsen worker 154 - jepsen.util 154 :invoke :add 3204 INFO [2022-01-10 08:39:21,688] jepsen worker 36 - jepsen.util 36 :ok :add 3193 INFO [2022-01-10 08:39:21,688] jepsen worker 641 - jepsen.util 641 :ok :add 3199 INFO [2022-01-10 08:39:21,689] jepsen worker 826 - jepsen.util 826 :invoke :add 3205 INFO [2022-01-10 08:39:21,692] jepsen worker 543 - jepsen.util 543 :ok :add 3201 INFO [2022-01-10 08:39:21,693] jepsen worker 721 - jepsen.util 721 :invoke :add 3206 INFO [2022-01-10 08:39:21,695] jepsen worker 154 - jepsen.util 154 :ok :add 3204 INFO [2022-01-10 08:39:21,697] jepsen worker 747 - jepsen.util 747 :invoke :add 3207 INFO [2022-01-10 08:39:21,697] jepsen worker 826 - jepsen.util 826 :ok :add 3205 INFO [2022-01-10 08:39:21,698] jepsen worker 183 - jepsen.util 183 :ok :add 3203 INFO [2022-01-10 08:39:21,700] jepsen worker 775 - jepsen.util 775 :invoke :add 3208 INFO [2022-01-10 08:39:21,700] jepsen worker 35 - jepsen.util 35 :ok :add 3200 INFO [2022-01-10 08:39:21,703] jepsen worker 721 - jepsen.util 721 :ok :add 3206 INFO [2022-01-10 08:39:21,703] jepsen worker 337 - jepsen.util 337 :invoke :add 3209 INFO [2022-01-10 08:39:21,704] jepsen worker 269 - jepsen.util 269 :ok :add 3202 INFO [2022-01-10 08:39:21,706] jepsen worker 447 - jepsen.util 447 :invoke :add 3210 INFO [2022-01-10 08:39:21,706] jepsen worker 747 - jepsen.util 747 :ok :add 3207 INFO [2022-01-10 08:39:21,708] jepsen worker 116 - jepsen.util 116 :invoke :add 3211 INFO [2022-01-10 08:39:21,709] jepsen worker 717 - jepsen.util 717 :ok :add 2690 INFO [2022-01-10 08:39:21,709] jepsen worker 707 - jepsen.util 707 :ok :add 2700 INFO [2022-01-10 08:39:21,709] jepsen worker 97 - jepsen.util 97 :ok :add 2809 INFO [2022-01-10 08:39:21,709] jepsen worker 501 - jepsen.util 501 :ok :add 2825 INFO [2022-01-10 08:39:21,710] jepsen worker 280 - jepsen.util 280 :ok :add 2718 INFO [2022-01-10 08:39:21,710] jepsen worker 402 - jepsen.util 402 :ok :add 2734 INFO [2022-01-10 08:39:21,710] jepsen worker 43 - jepsen.util 43 :ok :add 3085 INFO [2022-01-10 08:39:21,711] jepsen worker 702 - jepsen.util 702 :ok :add 2688 INFO [2022-01-10 08:39:21,711] jepsen worker 941 - jepsen.util 941 :invoke :add 3212 INFO [2022-01-10 08:39:21,714] jepsen worker 775 - jepsen.util 775 :ok :add 3208 INFO [2022-01-10 08:39:21,715] jepsen worker 857 - jepsen.util 857 :invoke :add 3213 INFO [2022-01-10 08:39:21,718] jepsen worker 571 - jepsen.util 571 :invoke :add 3214 INFO [2022-01-10 08:39:21,720] jepsen worker 691 - jepsen.util 691 :invoke :add 3215 INFO [2022-01-10 08:39:21,723] jepsen worker 447 - jepsen.util 447 :ok :add 3210 INFO [2022-01-10 08:39:21,723] jepsen worker 876 - jepsen.util 876 :invoke :add 3216 INFO [2022-01-10 08:39:21,727] jepsen worker 629 - jepsen.util 629 :invoke :add 3217 INFO [2022-01-10 08:39:21,728] jepsen worker 337 - jepsen.util 337 :ok :add 3209 INFO [2022-01-10 08:39:21,729] jepsen worker 116 - jepsen.util 116 :ok :add 3211 INFO [2022-01-10 08:39:21,730] jepsen worker 367 - jepsen.util 367 :invoke :add 3218 INFO [2022-01-10 08:39:21,733] jepsen worker 392 - jepsen.util 392 :invoke :add 3219 INFO [2022-01-10 08:39:21,733] jepsen worker 857 - jepsen.util 857 :ok :add 3213 INFO [2022-01-10 08:39:21,736] jepsen worker 722 - jepsen.util 722 :invoke :add 3220 INFO [2022-01-10 08:39:21,737] jepsen worker 367 - jepsen.util 367 :ok :add 3218 INFO [2022-01-10 08:39:21,739] jepsen worker 402 - jepsen.util 402 :invoke :add 3221 INFO [2022-01-10 08:39:21,741] jepsen worker 941 - jepsen.util 941 :ok :add 3212 INFO [2022-01-10 08:39:21,741] jepsen worker 392 - jepsen.util 392 :ok :add 3219 INFO [2022-01-10 08:39:21,743] jepsen worker 202 - jepsen.util 202 :invoke :add 3222 INFO [2022-01-10 08:39:21,746] jepsen worker 937 - jepsen.util 937 :invoke :add 3223 INFO [2022-01-10 08:39:21,748] jepsen worker 195 - jepsen.util 195 :invoke :add 3224 INFO [2022-01-10 08:39:21,750] jepsen worker 917 - jepsen.util 917 :ok :add 2702 INFO [2022-01-10 08:39:21,750] jepsen worker 920 - jepsen.util 920 :ok :add 2827 INFO [2022-01-10 08:39:21,751] jepsen worker 260 - jepsen.util 260 :invoke :add 3225 INFO [2022-01-10 08:39:21,752] jepsen worker 416 - jepsen.util 416 :ok :add 2692 INFO [2022-01-10 08:39:21,754] jepsen worker 750 - jepsen.util 750 :ok :add 2719 INFO [2022-01-10 08:39:21,754] jepsen worker 791 - jepsen.util 791 :invoke :add 3226 INFO [2022-01-10 08:39:21,754] jepsen worker 694 - jepsen.util 694 :ok :add 2689 INFO [2022-01-10 08:39:21,754] jepsen worker 130 - jepsen.util 130 :ok :add 2701 INFO [2022-01-10 08:39:21,754] jepsen worker 258 - jepsen.util 258 :ok :add 2735 INFO [2022-01-10 08:39:21,754] jepsen worker 946 - jepsen.util 946 :ok :add 2808 INFO [2022-01-10 08:39:21,755] jepsen worker 652 - jepsen.util 652 :ok :add 2998 INFO [2022-01-10 08:39:21,755] jepsen worker 556 - jepsen.util 556 :ok :add 3120 INFO [2022-01-10 08:39:21,758] jepsen worker 533 - jepsen.util 533 :invoke :add 3227 INFO [2022-01-10 08:39:21,760] jepsen worker 895 - jepsen.util 895 :ok :add 2691 INFO [2022-01-10 08:39:21,761] jepsen worker 681 - jepsen.util 681 :invoke :add 3228 INFO [2022-01-10 08:39:21,764] jepsen worker 104 - jepsen.util 104 :invoke :add 3229 INFO [2022-01-10 08:39:21,767] jepsen worker 484 - jepsen.util 484 :invoke :add 3230 INFO [2022-01-10 08:39:21,770] jepsen worker 741 - jepsen.util 741 :invoke :add 3231 INFO [2022-01-10 08:39:21,772] jepsen worker 13 - jepsen.util 13 :invoke :add 3232 INFO [2022-01-10 08:39:21,775] jepsen worker 573 - jepsen.util 573 :invoke :add 3233 INFO [2022-01-10 08:39:21,775] jepsen worker 104 - jepsen.util 104 :ok :add 3229 INFO [2022-01-10 08:39:21,778] jepsen worker 348 - jepsen.util 348 :invoke :add 3234 INFO [2022-01-10 08:39:21,780] jepsen worker 101 - jepsen.util 101 :invoke :add 3235 INFO [2022-01-10 08:39:21,782] jepsen worker 428 - jepsen.util 428 :invoke :add 3236 INFO [2022-01-10 08:39:21,785] jepsen worker 280 - jepsen.util 280 :invoke :add 3237 INFO [2022-01-10 08:39:21,788] jepsen worker 775 - jepsen.util 775 :invoke :add 3238 INFO [2022-01-10 08:39:21,790] jepsen worker 101 - jepsen.util 101 :ok :add 3235 INFO [2022-01-10 08:39:21,791] jepsen worker 67 - jepsen.util 67 :invoke :add 3239 INFO [2022-01-10 08:39:21,793] jepsen worker 409 - jepsen.util 409 :invoke :add 3240 INFO [2022-01-10 08:39:21,795] jepsen worker 959 - jepsen.util 959 :invoke :add 3241 INFO [2022-01-10 08:39:21,796] jepsen worker 280 - jepsen.util 280 :ok :add 3237 INFO [2022-01-10 08:39:21,797] jepsen worker 741 - jepsen.util 741 :ok :add 3231 INFO [2022-01-10 08:39:21,799] jepsen worker 195 - jepsen.util 195 :ok :add 3224 INFO [2022-01-10 08:39:21,799] jepsen worker 260 - jepsen.util 260 :ok :add 3225 INFO [2022-01-10 08:39:21,799] jepsen worker 844 - jepsen.util 844 :invoke :add 3242 INFO [2022-01-10 08:39:21,802] jepsen worker 573 - jepsen.util 573 :ok :add 3233 INFO [2022-01-10 08:39:21,802] jepsen worker 533 - jepsen.util 533 :ok :add 3227 INFO [2022-01-10 08:39:21,802] jepsen worker 859 - jepsen.util 859 :invoke :add 3243 INFO [2022-01-10 08:39:21,804] jepsen worker 348 - jepsen.util 348 :ok :add 3234 INFO [2022-01-10 08:39:21,806] jepsen worker 431 - jepsen.util 431 :invoke :add 3244 INFO [2022-01-10 08:39:21,807] jepsen worker 681 - jepsen.util 681 :ok :add 3228 INFO [2022-01-10 08:39:21,807] jepsen worker 484 - jepsen.util 484 :ok :add 3230 INFO [2022-01-10 08:39:21,809] jepsen worker 428 - jepsen.util 428 :ok :add 3236 INFO [2022-01-10 08:39:21,810] jepsen worker 142 - jepsen.util 142 :invoke :add 3245 INFO [2022-01-10 08:39:21,813] jepsen worker 298 - jepsen.util 298 :invoke :add 3246 INFO [2022-01-10 08:39:21,815] jepsen worker 999 - jepsen.util 999 :invoke :add 3247 INFO [2022-01-10 08:39:21,816] jepsen worker 791 - jepsen.util 791 :ok :add 3226 INFO [2022-01-10 08:39:21,816] jepsen worker 13 - jepsen.util 13 :ok :add 3232 INFO [2022-01-10 08:39:21,818] jepsen worker 114 - jepsen.util 114 :invoke :add 3248 INFO [2022-01-10 08:39:21,821] jepsen worker 797 - jepsen.util 797 :invoke :add 3249 INFO [2022-01-10 08:39:21,822] jepsen worker 431 - jepsen.util 431 :ok :add 3244 INFO [2022-01-10 08:39:21,823] jepsen worker 409 - jepsen.util 409 :ok :add 3240 INFO [2022-01-10 08:39:21,824] jepsen worker 735 - jepsen.util 735 :invoke :add 3250 INFO [2022-01-10 08:39:21,827] jepsen worker 810 - jepsen.util 810 :invoke :add 3251 INFO [2022-01-10 08:39:21,829] jepsen worker 65 - jepsen.util 65 :invoke :add 3252 INFO [2022-01-10 08:39:21,830] jepsen worker 298 - jepsen.util 298 :ok :add 3246 INFO [2022-01-10 08:39:21,830] jepsen worker 142 - jepsen.util 142 :ok :add 3245 INFO [2022-01-10 08:39:21,830] jepsen worker 844 - jepsen.util 844 :ok :add 3242 INFO [2022-01-10 08:39:21,832] jepsen worker 363 - jepsen.util 363 :invoke :add 3253 INFO [2022-01-10 08:39:21,833] jepsen worker 999 - jepsen.util 999 :ok :add 3247 INFO [2022-01-10 08:39:21,834] jepsen worker 959 - jepsen.util 959 :ok :add 3241 INFO [2022-01-10 08:39:21,835] jepsen worker 658 - jepsen.util 658 :invoke :add 3254 INFO [2022-01-10 08:39:21,837] jepsen worker 800 - jepsen.util 800 :invoke :add 3255 INFO [2022-01-10 08:39:21,840] jepsen worker 975 - jepsen.util 975 :invoke :add 3256 INFO [2022-01-10 08:39:21,842] jepsen worker 885 - jepsen.util 885 :invoke :add 3257 INFO [2022-01-10 08:39:21,844] jepsen worker 474 - jepsen.util 474 :invoke :add 3258 INFO [2022-01-10 08:39:21,845] jepsen worker 859 - jepsen.util 859 :ok :add 3243 INFO [2022-01-10 08:39:21,846] jepsen worker 160 - jepsen.util 160 :invoke :add 3259 INFO [2022-01-10 08:39:21,848] jepsen worker 497 - jepsen.util 497 :invoke :add 3260 INFO [2022-01-10 08:39:21,851] jepsen worker 43 - jepsen.util 43 :invoke :add 3261 INFO [2022-01-10 08:39:21,853] jepsen worker 388 - jepsen.util 388 :invoke :add 3262 INFO [2022-01-10 08:39:21,855] jepsen worker 474 - jepsen.util 474 :ok :add 3258 INFO [2022-01-10 08:39:21,856] jepsen worker 970 - jepsen.util 970 :invoke :add 3263 INFO [2022-01-10 08:39:21,858] jepsen worker 347 - jepsen.util 347 :invoke :add 3264 INFO [2022-01-10 08:39:21,861] jepsen worker 158 - jepsen.util 158 :invoke :add 3265 INFO [2022-01-10 08:39:21,864] jepsen worker 160 - jepsen.util 160 :ok :add 3259 INFO [2022-01-10 08:39:21,864] jepsen worker 368 - jepsen.util 368 :invoke :add 3266 INFO [2022-01-10 08:39:21,867] jepsen worker 341 - jepsen.util 341 :invoke :add 3267 INFO [2022-01-10 08:39:21,869] jepsen worker 944 - jepsen.util 944 :ok :add 3187 INFO [2022-01-10 08:39:21,870] jepsen worker 923 - jepsen.util 923 :invoke :add 3268 INFO [2022-01-10 08:39:21,870] jepsen worker 980 - jepsen.util 980 :ok :add 2883 INFO [2022-01-10 08:39:21,873] jepsen worker 578 - jepsen.util 578 :invoke :add 3269 INFO [2022-01-10 08:39:21,876] jepsen worker 75 - jepsen.util 75 :invoke :add 3270 INFO [2022-01-10 08:39:21,877] jepsen worker 923 - jepsen.util 923 :ok :add 3268 INFO [2022-01-10 08:39:21,880] jepsen worker 578 - jepsen.util 578 :ok :add 3269 INFO [2022-01-10 08:39:21,881] jepsen worker 726 - jepsen.util 726 :invoke :add 3271 INFO [2022-01-10 08:39:21,883] jepsen worker 800 - jepsen.util 800 :ok :add 3255 INFO [2022-01-10 08:39:21,883] jepsen worker 43 - jepsen.util 43 :ok :add 3261 INFO [2022-01-10 08:39:21,884] jepsen worker 658 - jepsen.util 658 :ok :add 3254 INFO [2022-01-10 08:39:21,884] jepsen worker 497 - jepsen.util 497 :ok :add 3260 INFO [2022-01-10 08:39:21,885] jepsen worker 75 - jepsen.util 75 :ok :add 3270 INFO [2022-01-10 08:39:21,885] jepsen worker 689 - jepsen.util 689 :invoke :add 3272 INFO [2022-01-10 08:39:21,886] jepsen worker 726 - jepsen.util 726 :ok :add 3271 INFO [2022-01-10 08:39:21,888] jepsen worker 490 - jepsen.util 490 :invoke :add 3273 INFO [2022-01-10 08:39:21,889] jepsen worker 975 - jepsen.util 975 :ok :add 3256 INFO [2022-01-10 08:39:21,889] jepsen worker 388 - jepsen.util 388 :ok :add 3262 INFO [2022-01-10 08:39:21,891] jepsen worker 271 - jepsen.util 271 :invoke :add 3274 INFO [2022-01-10 08:39:21,891] jepsen worker 885 - jepsen.util 885 :ok :add 3257 INFO [2022-01-10 08:39:21,891] jepsen worker 970 - jepsen.util 970 :ok :add 3263 INFO [2022-01-10 08:39:21,891] jepsen worker 689 - jepsen.util 689 :ok :add 3272 INFO [2022-01-10 08:39:21,894] jepsen worker 778 - jepsen.util 778 :invoke :add 3275 INFO [2022-01-10 08:39:21,897] jepsen worker 664 - jepsen.util 664 :invoke :add 3276 INFO [2022-01-10 08:39:21,899] jepsen worker 490 - jepsen.util 490 :ok :add 3273 INFO [2022-01-10 08:39:21,901] jepsen worker 618 - jepsen.util 618 :invoke :add 3277 INFO [2022-01-10 08:39:21,903] jepsen worker 982 - jepsen.util 982 :invoke :add 3278 INFO [2022-01-10 08:39:21,906] jepsen worker 229 - jepsen.util 229 :invoke :add 3279 INFO [2022-01-10 08:39:21,908] jepsen worker 684 - jepsen.util 684 :invoke :add 3280 INFO [2022-01-10 08:39:21,911] jepsen worker 133 - jepsen.util 133 :invoke :add 3281 INFO [2022-01-10 08:39:21,914] jepsen worker 982 - jepsen.util 982 :ok :add 3278 INFO [2022-01-10 08:39:21,915] jepsen worker 447 - jepsen.util 447 :invoke :add 3282 INFO [2022-01-10 08:39:21,917] jepsen worker 785 - jepsen.util 785 :invoke :add 3283 INFO [2022-01-10 08:39:21,918] jepsen worker 229 - jepsen.util 229 :ok :add 3279 INFO [2022-01-10 08:39:21,919] jepsen worker 312 - jepsen.util 312 :invoke :add 3284 INFO [2022-01-10 08:39:21,920] jepsen worker 133 - jepsen.util 133 :ok :add 3281 INFO [2022-01-10 08:39:21,922] jepsen worker 873 - jepsen.util 873 :invoke :add 3285 INFO [2022-01-10 08:39:21,923] jepsen worker 684 - jepsen.util 684 :ok :add 3280 INFO [2022-01-10 08:39:21,924] jepsen worker 858 - jepsen.util 858 :invoke :add 3286 INFO [2022-01-10 08:39:21,925] jepsen worker 785 - jepsen.util 785 :ok :add 3283 INFO [2022-01-10 08:39:21,927] jepsen worker 447 - jepsen.util 447 :ok :add 3282 INFO [2022-01-10 08:39:21,927] jepsen worker 933 - jepsen.util 933 :invoke :add 3287 INFO [2022-01-10 08:39:21,931] jepsen worker 565 - jepsen.util 565 :invoke :add 3288 INFO [2022-01-10 08:39:21,933] jepsen worker 193 - jepsen.util 193 :invoke :add 3289 INFO [2022-01-10 08:39:21,936] jepsen worker 136 - jepsen.util 136 :invoke :add 3290 INFO [2022-01-10 08:39:21,938] jepsen worker 170 - jepsen.util 170 :invoke :add 3291 INFO [2022-01-10 08:39:21,942] jepsen worker 896 - jepsen.util 896 :invoke :add 3292 INFO [2022-01-10 08:39:21,944] jepsen worker 102 - jepsen.util 102 :invoke :add 3293 INFO [2022-01-10 08:39:21,946] jepsen worker 271 - jepsen.util 271 :ok :add 3274 INFO [2022-01-10 08:39:21,947] jepsen worker 43 - jepsen.util 43 :invoke :add 3294 INFO [2022-01-10 08:39:21,948] jepsen worker 778 - jepsen.util 778 :ok :add 3275 INFO [2022-01-10 08:39:21,949] jepsen worker 196 - jepsen.util 196 :invoke :add 3295 INFO [2022-01-10 08:39:21,951] jepsen worker 439 - jepsen.util 439 :ok :add 2812 INFO [2022-01-10 08:39:21,952] jepsen worker 171 - jepsen.util 171 :ok :add 2693 INFO [2022-01-10 08:39:21,952] jepsen worker 17 - jepsen.util 17 :invoke :add 3296 INFO [2022-01-10 08:39:21,955] jepsen worker 664 - jepsen.util 664 :ok :add 3276 INFO [2022-01-10 08:39:21,955] jepsen worker 618 - jepsen.util 618 :ok :add 3277 INFO [2022-01-10 08:39:21,956] jepsen worker 486 - jepsen.util 486 :invoke :add 3297 INFO [2022-01-10 08:39:21,958] jepsen worker 431 - jepsen.util 431 :invoke :add 3298 INFO [2022-01-10 08:39:21,958] jepsen worker 549 - jepsen.util 549 :ok :add 2703 INFO [2022-01-10 08:39:21,959] jepsen worker 755 - jepsen.util 755 :ok :add 2737 INFO [2022-01-10 08:39:21,962] jepsen worker 301 - jepsen.util 301 :invoke :add 3299 INFO [2022-01-10 08:39:21,963] jepsen worker 136 - jepsen.util 136 :ok :add 3290 INFO [2022-01-10 08:39:21,963] jepsen worker 565 - jepsen.util 565 :ok :add 3288 INFO [2022-01-10 08:39:21,965] jepsen worker 663 - jepsen.util 663 :invoke :add 3300 INFO [2022-01-10 08:39:21,968] jepsen worker 170 - jepsen.util 170 :ok :add 3291 INFO [2022-01-10 08:39:21,968] jepsen worker 955 - jepsen.util 955 :invoke :add 3301 INFO [2022-01-10 08:39:21,968] jepsen worker 193 - jepsen.util 193 :ok :add 3289 INFO [2022-01-10 08:39:21,970] jepsen worker 43 - jepsen.util 43 :ok :add 3294 INFO [2022-01-10 08:39:21,971] jepsen worker 896 - jepsen.util 896 :ok :add 3292 INFO [2022-01-10 08:39:21,971] jepsen worker 794 - jepsen.util 794 :invoke :add 3302 INFO [2022-01-10 08:39:21,975] jepsen worker 196 - jepsen.util 196 :ok :add 3295 INFO [2022-01-10 08:39:21,975] jepsen worker 209 - jepsen.util 209 :invoke :add 3303 INFO [2022-01-10 08:39:21,977] jepsen worker 497 - jepsen.util 497 :invoke :add 3304 INFO [2022-01-10 08:39:21,979] jepsen worker 17 - jepsen.util 17 :ok :add 3296 INFO [2022-01-10 08:39:21,980] jepsen worker 102 - jepsen.util 102 :ok :add 3293 INFO [2022-01-10 08:39:21,981] jepsen worker 145 - jepsen.util 145 :invoke :add 3305 INFO [2022-01-10 08:39:21,982] jepsen worker 486 - jepsen.util 486 :ok :add 3297 INFO [2022-01-10 08:39:21,984] jepsen worker 686 - jepsen.util 686 :invoke :add 3306 INFO [2022-01-10 08:39:21,986] jepsen worker 886 - jepsen.util 886 :invoke :add 3307 INFO [2022-01-10 08:39:21,988] jepsen worker 500 - jepsen.util 500 :invoke :add 3308 INFO [2022-01-10 08:39:21,992] jepsen worker 875 - jepsen.util 875 :invoke :add 3309 INFO [2022-01-10 08:39:21,994] jepsen worker 518 - jepsen.util 518 :invoke :add 3310 INFO [2022-01-10 08:39:21,996] jepsen worker 145 - jepsen.util 145 :ok :add 3305 INFO [2022-01-10 08:39:21,997] jepsen worker 574 - jepsen.util 574 :invoke :add 3311 INFO [2022-01-10 08:39:21,997] jepsen worker 955 - jepsen.util 955 :ok :add 3301 INFO [2022-01-10 08:39:22,000] jepsen worker 31 - jepsen.util 31 :invoke :add 3312 INFO [2022-01-10 08:39:22,001] jepsen worker 886 - jepsen.util 886 :ok :add 3307 INFO [2022-01-10 08:39:22,002] jepsen worker 209 - jepsen.util 209 :ok :add 3303 INFO [2022-01-10 08:39:22,003] jepsen worker 246 - jepsen.util 246 :invoke :add 3313 INFO [2022-01-10 08:39:22,006] jepsen worker 394 - jepsen.util 394 :invoke :add 3314 INFO [2022-01-10 08:39:22,008] jepsen worker 729 - jepsen.util 729 :invoke :add 3315 INFO [2022-01-10 08:39:22,010] jepsen worker 663 - jepsen.util 663 :ok :add 3300 INFO [2022-01-10 08:39:22,010] jepsen worker 594 - jepsen.util 594 :invoke :add 3316 INFO [2022-01-10 08:39:22,011] jepsen worker 497 - jepsen.util 497 :ok :add 3304 INFO [2022-01-10 08:39:22,013] jepsen worker 499 - jepsen.util 499 :invoke :add 3317 INFO [2022-01-10 08:39:22,016] jepsen worker 674 - jepsen.util 674 :invoke :add 3318 INFO [2022-01-10 08:39:22,017] jepsen worker 686 - jepsen.util 686 :ok :add 3306 INFO [2022-01-10 08:39:22,018] jepsen worker 707 - jepsen.util 707 :invoke :add 3319 INFO [2022-01-10 08:39:22,021] jepsen worker 9 - jepsen.util 9 :invoke :add 3320 INFO [2022-01-10 08:39:22,023] jepsen worker 794 - jepsen.util 794 :ok :add 3302 INFO [2022-01-10 08:39:22,024] jepsen worker 426 - jepsen.util 426 :invoke :add 3321 INFO [2022-01-10 08:39:22,025] jepsen worker 574 - jepsen.util 574 :ok :add 3311 INFO [2022-01-10 08:39:22,025] jepsen worker 301 - jepsen.util 301 :ok :add 3299 INFO [2022-01-10 08:39:22,026] jepsen worker 875 - jepsen.util 875 :ok :add 3309 INFO [2022-01-10 08:39:22,026] jepsen worker 449 - jepsen.util 449 :invoke :add 3322 INFO [2022-01-10 08:39:22,030] jepsen worker 580 - jepsen.util 580 :invoke :add 3323 INFO [2022-01-10 08:39:22,033] jepsen worker 590 - jepsen.util 590 :invoke :add 3324 INFO [2022-01-10 08:39:22,034] jepsen worker 518 - jepsen.util 518 :ok :add 3310 INFO [2022-01-10 08:39:22,034] jepsen worker 431 - jepsen.util 431 :ok :add 3298 INFO [2022-01-10 08:39:22,037] jepsen worker 404 - jepsen.util 404 :invoke :add 3325 INFO [2022-01-10 08:39:22,038] jepsen worker 707 - jepsen.util 707 :ok :add 3319 INFO [2022-01-10 08:39:22,040] jepsen worker 500 - jepsen.util 500 :ok :add 3308 INFO [2022-01-10 08:39:22,041] jepsen worker 343 - jepsen.util 343 :invoke :add 3326 INFO [2022-01-10 08:39:22,042] jepsen worker 674 - jepsen.util 674 :ok :add 3318 INFO [2022-01-10 08:39:22,045] jepsen worker 606 - jepsen.util 606 :invoke :add 3327 INFO [2022-01-10 08:39:22,048] jepsen worker 104 - jepsen.util 104 :invoke :add 3328 INFO [2022-01-10 08:39:22,050] jepsen worker 282 - jepsen.util 282 :invoke :add 3329 INFO [2022-01-10 08:39:22,053] jepsen worker 650 - jepsen.util 650 :invoke :add 3330 INFO [2022-01-10 08:39:22,054] jepsen worker 246 - jepsen.util 246 :ok :add 3313 INFO [2022-01-10 08:39:22,056] jepsen worker 104 - jepsen.util 104 :ok :add 3328 INFO [2022-01-10 08:39:22,056] jepsen worker 456 - jepsen.util 456 :invoke :add 3331 INFO [2022-01-10 08:39:22,059] jepsen worker 593 - jepsen.util 593 :ok :add 2857 INFO [2022-01-10 08:39:22,060] jepsen worker 845 - jepsen.util 845 :ok :add 2863 INFO [2022-01-10 08:39:22,060] jepsen worker 900 - jepsen.util 900 :ok :add 3153 INFO [2022-01-10 08:39:22,060] jepsen worker 202 - jepsen.util 202 :ok :add 3222 INFO [2022-01-10 08:39:22,060] jepsen worker 988 - jepsen.util 988 :ok :add 2772 INFO [2022-01-10 08:39:22,060] jepsen worker 567 - jepsen.util 567 :ok :add 2746 INFO [2022-01-10 08:39:22,061] jepsen worker 991 - jepsen.util 991 :invoke :add 3332 INFO [2022-01-10 08:39:22,062] jepsen worker 282 - jepsen.util 282 :ok :add 3329 INFO [2022-01-10 08:39:22,064] jepsen worker 31 - jepsen.util 31 :ok :add 3312 INFO [2022-01-10 08:39:22,064] jepsen worker 343 - jepsen.util 343 :ok :add 3326 INFO [2022-01-10 08:39:22,064] jepsen worker 404 - jepsen.util 404 :ok :add 3325 INFO [2022-01-10 08:39:22,064] jepsen worker 650 - jepsen.util 650 :ok :add 3330 INFO [2022-01-10 08:39:22,066] jepsen worker 125 - jepsen.util 125 :invoke :add 3333 INFO [2022-01-10 08:39:22,069] jepsen worker 763 - jepsen.util 763 :invoke :add 3334 INFO [2022-01-10 08:39:22,071] jepsen worker 456 - jepsen.util 456 :ok :add 3331 INFO [2022-01-10 08:39:22,072] jepsen worker 606 - jepsen.util 606 :ok :add 3327 INFO [2022-01-10 08:39:22,073] jepsen worker 590 - jepsen.util 590 :ok :add 3324 INFO [2022-01-10 08:39:22,073] jepsen worker 704 - jepsen.util 704 :invoke :add 3335 INFO [2022-01-10 08:39:22,075] jepsen worker 991 - jepsen.util 991 :ok :add 3332 INFO [2022-01-10 08:39:22,076] jepsen worker 506 - jepsen.util 506 :invoke :add 3336 INFO [2022-01-10 08:39:22,079] jepsen worker 970 - jepsen.util 970 :invoke :add 3337 INFO [2022-01-10 08:39:22,081] jepsen worker 763 - jepsen.util 763 :ok :add 3334 INFO [2022-01-10 08:39:22,081] jepsen worker 746 - jepsen.util 746 :invoke :add 3338 INFO [2022-01-10 08:39:22,083] jepsen worker 125 - jepsen.util 125 :ok :add 3333 INFO [2022-01-10 08:39:22,084] jepsen worker 165 - jepsen.util 165 :invoke :add 3339 INFO [2022-01-10 08:39:22,086] jepsen worker 704 - jepsen.util 704 :ok :add 3335 INFO [2022-01-10 08:39:22,086] jepsen worker 340 - jepsen.util 340 :invoke :add 3340 INFO [2022-01-10 08:39:22,089] jepsen worker 133 - jepsen.util 133 :invoke :add 3341 INFO [2022-01-10 08:39:22,092] jepsen worker 22 - jepsen.util 22 :invoke :add 3342 INFO [2022-01-10 08:39:22,092] jepsen worker 506 - jepsen.util 506 :ok :add 3336 INFO [2022-01-10 08:39:22,094] jepsen worker 970 - jepsen.util 970 :ok :add 3337 INFO [2022-01-10 08:39:22,094] jepsen worker 832 - jepsen.util 832 :invoke :add 3343 INFO [2022-01-10 08:39:22,097] jepsen worker 485 - jepsen.util 485 :invoke :add 3344 INFO [2022-01-10 08:39:22,097] jepsen worker 133 - jepsen.util 133 :ok :add 3341 INFO [2022-01-10 08:39:22,099] jepsen worker 538 - jepsen.util 538 :invoke :add 3345 INFO [2022-01-10 08:39:22,102] jepsen worker 197 - jepsen.util 197 :invoke :add 3346 INFO [2022-01-10 08:39:22,105] jepsen worker 95 - jepsen.util 95 :invoke :add 3347 INFO [2022-01-10 08:39:22,107] jepsen worker 340 - jepsen.util 340 :ok :add 3340 INFO [2022-01-10 08:39:22,108] jepsen worker 64 - jepsen.util 64 :invoke :add 3348 INFO [2022-01-10 08:39:22,109] jepsen worker 832 - jepsen.util 832 :ok :add 3343 INFO [2022-01-10 08:39:22,111] jepsen worker 718 - jepsen.util 718 :invoke :add 3349 INFO [2022-01-10 08:39:22,112] jepsen worker 22 - jepsen.util 22 :ok :add 3342 INFO [2022-01-10 08:39:22,113] jepsen worker 883 - jepsen.util 883 :invoke :add 3350 INFO [2022-01-10 08:39:22,115] jepsen worker 538 - jepsen.util 538 :ok :add 3345 INFO [2022-01-10 08:39:22,116] jepsen worker 92 - jepsen.util 92 :invoke :add 3351 INFO [2022-01-10 08:39:22,119] jepsen worker 142 - jepsen.util 142 :invoke :add 3352 INFO [2022-01-10 08:39:22,120] jepsen worker 485 - jepsen.util 485 :ok :add 3344 INFO [2022-01-10 08:39:22,122] jepsen worker 905 - jepsen.util 905 :invoke :add 3353 INFO [2022-01-10 08:39:22,123] jepsen worker 95 - jepsen.util 95 :ok :add 3347 INFO [2022-01-10 08:39:22,125] jepsen worker 234 - jepsen.util 234 :invoke :add 3354 INFO [2022-01-10 08:39:22,127] jepsen worker 649 - jepsen.util 649 :invoke :add 3355 INFO [2022-01-10 08:39:22,128] jepsen worker 197 - jepsen.util 197 :ok :add 3346 INFO [2022-01-10 08:39:22,129] jepsen worker 902 - jepsen.util 902 :invoke :add 3356 INFO [2022-01-10 08:39:22,132] jepsen worker 621 - jepsen.util 621 :invoke :add 3357 INFO [2022-01-10 08:39:22,133] jepsen worker 649 - jepsen.util 649 :ok :add 3355 INFO [2022-01-10 08:39:22,134] jepsen worker 234 - jepsen.util 234 :ok :add 3354 INFO [2022-01-10 08:39:22,134] jepsen worker 413 - jepsen.util 413 :invoke :add 3358 INFO [2022-01-10 08:39:22,137] jepsen worker 726 - jepsen.util 726 :invoke :add 3359 INFO [2022-01-10 08:39:22,139] jepsen worker 189 - jepsen.util 189 :invoke :add 3360 INFO [2022-01-10 08:39:22,142] jepsen worker 943 - jepsen.util 943 :invoke :add 3361 INFO [2022-01-10 08:39:22,146] jepsen worker 603 - jepsen.util 603 :invoke :add 3362 INFO [2022-01-10 08:39:22,149] jepsen worker 780 - jepsen.util 780 :invoke :add 3363 INFO [2022-01-10 08:39:22,152] jepsen worker 69 - jepsen.util 69 :invoke :add 3364 INFO [2022-01-10 08:39:22,154] jepsen worker 230 - jepsen.util 230 :invoke :add 3365 INFO [2022-01-10 08:39:22,157] jepsen worker 690 - jepsen.util 690 :invoke :add 3366 INFO [2022-01-10 08:39:22,159] jepsen worker 823 - jepsen.util 823 :invoke :add 3367 INFO [2022-01-10 08:39:22,161] jepsen worker 106 - jepsen.util 106 :invoke :add 3368 INFO [2022-01-10 08:39:22,163] jepsen worker 652 - jepsen.util 652 :invoke :add 3369 INFO [2022-01-10 08:39:22,165] jepsen worker 844 - jepsen.util 844 :invoke :add 3370 INFO [2022-01-10 08:39:22,168] jepsen worker 548 - jepsen.util 548 :invoke :add 3371 INFO [2022-01-10 08:39:22,170] jepsen worker 956 - jepsen.util 956 :invoke :add 3372 INFO [2022-01-10 08:39:22,170] jepsen worker 106 - jepsen.util 106 :ok :add 3368 INFO [2022-01-10 08:39:22,172] jepsen worker 472 - jepsen.util 472 :invoke :add 3373 INFO [2022-01-10 08:39:22,174] jepsen worker 202 - jepsen.util 202 :invoke :add 3374 INFO [2022-01-10 08:39:22,176] jepsen worker 652 - jepsen.util 652 :ok :add 3369 INFO [2022-01-10 08:39:22,177] jepsen worker 193 - jepsen.util 193 :invoke :add 3375 INFO [2022-01-10 08:39:22,179] jepsen worker 844 - jepsen.util 844 :ok :add 3370 INFO [2022-01-10 08:39:22,179] jepsen worker 663 - jepsen.util 663 :invoke :add 3376 INFO [2022-01-10 08:39:22,181] jepsen worker 548 - jepsen.util 548 :ok :add 3371 INFO [2022-01-10 08:39:22,182] jepsen worker 78 - jepsen.util 78 :invoke :add 3377 INFO [2022-01-10 08:39:22,184] jepsen worker 956 - jepsen.util 956 :ok :add 3372 INFO [2022-01-10 08:39:22,185] jepsen worker 482 - jepsen.util 482 :invoke :add 3378 INFO [2022-01-10 08:39:22,187] jepsen worker 472 - jepsen.util 472 :ok :add 3373 INFO [2022-01-10 08:39:22,188] jepsen worker 184 - jepsen.util 184 :invoke :add 3379 INFO [2022-01-10 08:39:22,189] jepsen worker 550 - jepsen.util 550 :invoke :add 3380 INFO [2022-01-10 08:39:22,192] jepsen worker 725 - jepsen.util 725 :invoke :add 3381 INFO [2022-01-10 08:39:22,195] jepsen worker 829 - jepsen.util 829 :invoke :add 3382 INFO [2022-01-10 08:39:22,197] jepsen worker 961 - jepsen.util 961 :invoke :add 3383 INFO [2022-01-10 08:39:22,199] jepsen worker 420 - jepsen.util 420 :invoke :add 3384 INFO [2022-01-10 08:39:22,204] jepsen worker 647 - jepsen.util 647 :invoke :add 3385 INFO [2022-01-10 08:39:22,207] jepsen worker 766 - jepsen.util 766 :invoke :add 3386 INFO [2022-01-10 08:39:22,209] jepsen worker 940 - jepsen.util 940 :invoke :add 3387 INFO [2022-01-10 08:39:22,212] jepsen worker 750 - jepsen.util 750 :invoke :add 3388 INFO [2022-01-10 08:39:22,214] jepsen worker 474 - jepsen.util 474 :invoke :add 3389 INFO [2022-01-10 08:39:22,215] jepsen worker 902 - jepsen.util 902 :ok :add 3356 INFO [2022-01-10 08:39:22,217] jepsen worker 132 - jepsen.util 132 :invoke :add 3390 INFO [2022-01-10 08:39:22,217] jepsen worker 780 - jepsen.util 780 :ok :add 3363 INFO [2022-01-10 08:39:22,217] jepsen worker 621 - jepsen.util 621 :ok :add 3357 INFO [2022-01-10 08:39:22,220] jepsen worker 87 - jepsen.util 87 :invoke :add 3391 INFO [2022-01-10 08:39:22,221] jepsen worker 750 - jepsen.util 750 :ok :add 3388 INFO [2022-01-10 08:39:22,221] jepsen worker 726 - jepsen.util 726 :ok :add 3359 INFO [2022-01-10 08:39:22,222] jepsen worker 413 - jepsen.util 413 :ok :add 3358 INFO [2022-01-10 08:39:22,222] jepsen worker 474 - jepsen.util 474 :ok :add 3389 INFO [2022-01-10 08:39:22,222] jepsen worker 131 - jepsen.util 131 :invoke :add 3392 INFO [2022-01-10 08:39:22,225] jepsen worker 482 - jepsen.util 482 :ok :add 3378 INFO [2022-01-10 08:39:22,225] jepsen worker 184 - jepsen.util 184 :ok :add 3379 INFO [2022-01-10 08:39:22,226] jepsen worker 189 - jepsen.util 189 :ok :add 3360 INFO [2022-01-10 08:39:22,226] jepsen worker 943 - jepsen.util 943 :ok :add 3361 INFO [2022-01-10 08:39:22,226] jepsen worker 716 - jepsen.util 716 :invoke :add 3393 INFO [2022-01-10 08:39:22,230] jepsen worker 146 - jepsen.util 146 :invoke :add 3394 INFO [2022-01-10 08:39:22,231] jepsen worker 193 - jepsen.util 193 :ok :add 3375 INFO [2022-01-10 08:39:22,231] jepsen worker 202 - jepsen.util 202 :ok :add 3374 INFO [2022-01-10 08:39:22,232] jepsen worker 603 - jepsen.util 603 :ok :add 3362 INFO [2022-01-10 08:39:22,233] jepsen worker 537 - jepsen.util 537 :invoke :add 3395 INFO [2022-01-10 08:39:22,234] jepsen worker 132 - jepsen.util 132 :ok :add 3390 INFO [2022-01-10 08:39:22,234] jepsen worker 550 - jepsen.util 550 :ok :add 3380 INFO [2022-01-10 08:39:22,236] jepsen worker 78 - jepsen.util 78 :ok :add 3377 INFO [2022-01-10 08:39:22,236] jepsen worker 198 - jepsen.util 198 :invoke :add 3396 INFO [2022-01-10 08:39:22,238] jepsen worker 663 - jepsen.util 663 :ok :add 3376 INFO [2022-01-10 08:39:22,239] jepsen worker 87 - jepsen.util 87 :ok :add 3391 INFO [2022-01-10 08:39:22,240] jepsen worker 451 - jepsen.util 451 :invoke :add 3397 INFO [2022-01-10 08:39:22,241] jepsen worker 829 - jepsen.util 829 :ok :add 3382 INFO [2022-01-10 08:39:22,243] jepsen worker 170 - jepsen.util 170 :invoke :add 3398 INFO [2022-01-10 08:39:22,246] jepsen worker 849 - jepsen.util 849 :invoke :add 3399 INFO [2022-01-10 08:39:22,249] jepsen worker 26 - jepsen.util 26 :invoke :add 3400 INFO [2022-01-10 08:39:22,251] jepsen worker 3 - jepsen.util 3 :invoke :add 3401 INFO [2022-01-10 08:39:22,254] jepsen worker 692 - jepsen.util 692 :invoke :add 3402 INFO [2022-01-10 08:39:22,257] jepsen worker 537 - jepsen.util 537 :ok :add 3395 INFO [2022-01-10 08:39:22,257] jepsen worker 346 - jepsen.util 346 :invoke :add 3403 INFO [2022-01-10 08:39:22,260] jepsen worker 960 - jepsen.util 960 :invoke :add 3404 INFO [2022-01-10 08:39:22,261] jepsen worker 516 - jepsen.util 516 :invoke :add 3405 INFO [2022-01-10 08:39:22,264] jepsen worker 829 - jepsen.util 829 :invoke :add 3406 INFO [2022-01-10 08:39:22,264] jepsen worker 146 - jepsen.util 146 :ok :add 3394 INFO [2022-01-10 08:39:22,265] jepsen worker 451 - jepsen.util 451 :ok :add 3397 INFO [2022-01-10 08:39:22,267] jepsen worker 85 - jepsen.util 85 :invoke :add 3407 INFO [2022-01-10 08:39:22,271] jepsen worker 56 - jepsen.util 56 :invoke :add 3408 INFO [2022-01-10 08:39:22,273] jepsen worker 299 - jepsen.util 299 :invoke :add 3409 INFO [2022-01-10 08:39:22,275] jepsen worker 205 - jepsen.util 205 :invoke :add 3410 INFO [2022-01-10 08:39:22,277] jepsen worker 706 - jepsen.util 706 :invoke :add 3411 INFO [2022-01-10 08:39:22,279] jepsen worker 193 - jepsen.util 193 :invoke :add 3412 INFO [2022-01-10 08:39:22,282] jepsen worker 140 - jepsen.util 140 :invoke :add 3413 INFO [2022-01-10 08:39:22,284] jepsen worker 418 - jepsen.util 418 :invoke :add 3414 INFO [2022-01-10 08:39:22,286] jepsen worker 219 - jepsen.util 219 :invoke :add 3415 INFO [2022-01-10 08:39:22,287] jepsen worker 131 - jepsen.util 131 :ok :add 3392 INFO [2022-01-10 08:39:22,289] jepsen worker 716 - jepsen.util 716 :ok :add 3393 INFO [2022-01-10 08:39:22,289] jepsen worker 116 - jepsen.util 116 :invoke :add 3416 INFO [2022-01-10 08:39:22,292] jepsen worker 821 - jepsen.util 821 :invoke :add 3417 INFO [2022-01-10 08:39:22,294] jepsen worker 725 - jepsen.util 725 :ok :add 3381 INFO [2022-01-10 08:39:22,294] jepsen worker 734 - jepsen.util 734 :invoke :add 3418 INFO [2022-01-10 08:39:22,297] jepsen worker 817 - jepsen.util 817 :invoke :add 3419 INFO [2022-01-10 08:39:22,299] jepsen worker 728 - jepsen.util 728 :invoke :add 3420 INFO [2022-01-10 08:39:22,301] jepsen worker 3 - jepsen.util 3 :ok :add 3401 INFO [2022-01-10 08:39:22,302] jepsen worker 516 - jepsen.util 516 :ok :add 3405 INFO [2022-01-10 08:39:22,303] jepsen worker 660 - jepsen.util 660 :invoke :add 3421 INFO [2022-01-10 08:39:22,305] jepsen worker 346 - jepsen.util 346 :ok :add 3403 INFO [2022-01-10 08:39:22,306] jepsen worker 198 - jepsen.util 198 :ok :add 3396 INFO [2022-01-10 08:39:22,306] jepsen worker 509 - jepsen.util 509 :invoke :add 3422 INFO [2022-01-10 08:39:22,309] jepsen worker 623 - jepsen.util 623 :invoke :add 3423 INFO [2022-01-10 08:39:22,309] jepsen worker 960 - jepsen.util 960 :ok :add 3404 INFO [2022-01-10 08:39:22,311] jepsen worker 383 - jepsen.util 383 :invoke :add 3424 INFO [2022-01-10 08:39:22,313] jepsen worker 438 - jepsen.util 438 :invoke :add 3425 INFO [2022-01-10 08:39:22,316] jepsen worker 866 - jepsen.util 866 :invoke :add 3426 INFO [2022-01-10 08:39:22,317] jepsen worker 829 - jepsen.util 829 :ok :add 3406 INFO [2022-01-10 08:39:22,319] jepsen worker 548 - jepsen.util 548 :invoke :add 3427 INFO [2022-01-10 08:39:22,319] jepsen worker 85 - jepsen.util 85 :ok :add 3407 INFO [2022-01-10 08:39:22,323] jepsen worker 212 - jepsen.util 212 :invoke :add 3428 INFO [2022-01-10 08:39:22,325] jepsen worker 961 - jepsen.util 961 :ok :add 3383 INFO [2022-01-10 08:39:22,326] jepsen worker 140 - jepsen.util 140 :ok :add 3413 INFO [2022-01-10 08:39:22,326] jepsen worker 525 - jepsen.util 525 :ok :add 2769 INFO [2022-01-10 08:39:22,327] jepsen worker 8 - jepsen.util 8 :ok :add 2745 INFO [2022-01-10 08:39:22,327] jepsen worker 819 - jepsen.util 819 :ok :add 3148 INFO [2022-01-10 08:39:22,327] jepsen worker 561 - jepsen.util 561 :ok :add 2771 INFO [2022-01-10 08:39:22,327] jepsen worker 403 - jepsen.util 403 :ok :add 3164 INFO [2022-01-10 08:39:22,327] jepsen worker 67 - jepsen.util 67 :ok :add 3239 INFO [2022-01-10 08:39:22,328] jepsen worker 916 - jepsen.util 916 :ok :add 3150 INFO [2022-01-10 08:39:22,328] jepsen worker 691 - jepsen.util 691 :ok :add 3215 INFO [2022-01-10 08:39:22,330] jepsen worker 764 - jepsen.util 764 :invoke :add 3429 INFO [2022-01-10 08:39:22,330] jepsen worker 656 - jepsen.util 656 :ok :add 2854 INFO [2022-01-10 08:39:22,331] jepsen worker 731 - jepsen.util 731 :ok :add 2860 INFO [2022-01-10 08:39:22,331] jepsen worker 68 - jepsen.util 68 :ok :add 2878 INFO [2022-01-10 08:39:22,331] jepsen worker 402 - jepsen.util 402 :ok :add 3221 INFO [2022-01-10 08:39:22,335] jepsen worker 170 - jepsen.util 170 :ok :add 3398 INFO [2022-01-10 08:39:22,336] jepsen worker 212 - jepsen.util 212 :ok :add 3428 INFO [2022-01-10 08:39:22,336] jepsen worker 26 - jepsen.util 26 :ok :add 3400 INFO [2022-01-10 08:39:22,336] jepsen worker 734 - jepsen.util 734 :ok :add 3418 INFO [2022-01-10 08:39:22,336] jepsen worker 95 - jepsen.util 95 :invoke :add 3430 INFO [2022-01-10 08:39:22,338] jepsen worker 990 - jepsen.util 990 :invoke :add 3431 INFO [2022-01-10 08:39:22,342] jepsen worker 442 - jepsen.util 442 :invoke :add 3432 INFO [2022-01-10 08:39:22,343] jepsen worker 692 - jepsen.util 692 :ok :add 3402 INFO [2022-01-10 08:39:22,345] jepsen worker 493 - jepsen.util 493 :invoke :add 3433 INFO [2022-01-10 08:39:22,345] jepsen worker 817 - jepsen.util 817 :ok :add 3419 INFO [2022-01-10 08:39:22,350] jepsen worker 132 - jepsen.util 132 :invoke :add 3434 INFO [2022-01-10 08:39:22,353] jepsen worker 344 - jepsen.util 344 :invoke :add 3435 INFO [2022-01-10 08:39:22,353] jepsen worker 764 - jepsen.util 764 :ok :add 3429 INFO [2022-01-10 08:39:22,353] jepsen worker 990 - jepsen.util 990 :ok :add 3431 INFO [2022-01-10 08:39:22,356] jepsen worker 236 - jepsen.util 236 :invoke :add 3436 INFO [2022-01-10 08:39:22,358] jepsen worker 95 - jepsen.util 95 :ok :add 3430 INFO [2022-01-10 08:39:22,359] jepsen worker 88 - jepsen.util 88 :invoke :add 3437 INFO [2022-01-10 08:39:22,362] jepsen worker 846 - jepsen.util 846 :invoke :add 3438 INFO [2022-01-10 08:39:22,365] jepsen worker 369 - jepsen.util 369 :invoke :add 3439 INFO [2022-01-10 08:39:22,366] jepsen worker 442 - jepsen.util 442 :ok :add 3432 INFO [2022-01-10 08:39:22,368] jepsen worker 545 - jepsen.util 545 :invoke :add 3440 INFO [2022-01-10 08:39:22,368] jepsen worker 493 - jepsen.util 493 :ok :add 3433 INFO [2022-01-10 08:39:22,369] jepsen worker 344 - jepsen.util 344 :ok :add 3435 INFO [2022-01-10 08:39:22,370] jepsen worker 783 - jepsen.util 783 :invoke :add 3441 INFO [2022-01-10 08:39:22,373] jepsen worker 30 - jepsen.util 30 :invoke :add 3442 INFO [2022-01-10 08:39:22,374] jepsen worker 88 - jepsen.util 88 :ok :add 3437 INFO [2022-01-10 08:39:22,374] jepsen worker 193 - jepsen.util 193 :ok :add 3412 INFO [2022-01-10 08:39:22,374] jepsen worker 866 - jepsen.util 866 :ok :add 3426 INFO [2022-01-10 08:39:22,376] jepsen worker 386 - jepsen.util 386 :invoke :add 3443 INFO [2022-01-10 08:39:22,379] jepsen worker 57 - jepsen.util 57 :invoke :add 3444 INFO [2022-01-10 08:39:22,381] jepsen worker 132 - jepsen.util 132 :ok :add 3434 INFO [2022-01-10 08:39:22,382] jepsen worker 706 - jepsen.util 706 :ok :add 3411 INFO [2022-01-10 08:39:22,382] jepsen worker 205 - jepsen.util 205 :ok :add 3410 INFO [2022-01-10 08:39:22,382] jepsen worker 567 - jepsen.util 567 :invoke :add 3445 INFO [2022-01-10 08:39:22,382] jepsen worker 56 - jepsen.util 56 :ok :add 3408 INFO [2022-01-10 08:39:22,382] jepsen worker 383 - jepsen.util 383 :ok :add 3424 INFO [2022-01-10 08:39:22,382] jepsen worker 299 - jepsen.util 299 :ok :add 3409 INFO [2022-01-10 08:39:22,382] jepsen worker 438 - jepsen.util 438 :ok :add 3425 INFO [2022-01-10 08:39:22,385] jepsen worker 57 - jepsen.util 57 :ok :add 3444 INFO [2022-01-10 08:39:22,387] jepsen worker 500 - jepsen.util 500 :invoke :add 3446 INFO [2022-01-10 08:39:22,389] jepsen worker 887 - jepsen.util 887 :invoke :add 3447 INFO [2022-01-10 08:39:22,389] jepsen worker 236 - jepsen.util 236 :ok :add 3436 INFO [2022-01-10 08:39:22,390] jepsen worker 545 - jepsen.util 545 :ok :add 3440 INFO [2022-01-10 08:39:22,392] jepsen worker 693 - jepsen.util 693 :invoke :add 3448 INFO [2022-01-10 08:39:22,393] jepsen worker 500 - jepsen.util 500 :ok :add 3446 INFO [2022-01-10 08:39:22,395] jepsen worker 590 - jepsen.util 590 :invoke :add 3449 INFO [2022-01-10 08:39:22,395] jepsen worker 30 - jepsen.util 30 :ok :add 3442 INFO [2022-01-10 08:39:22,475] jepsen worker 548 - jepsen.util 548 :ok :add 3427 INFO [2022-01-10 08:39:22,475] jepsen worker 887 - jepsen.util 887 :ok :add 3447 INFO [2022-01-10 08:39:22,475] jepsen worker 386 - jepsen.util 386 :ok :add 3443 INFO [2022-01-10 08:39:22,475] jepsen worker 567 - jepsen.util 567 :ok :add 3445 INFO [2022-01-10 08:39:22,475] jepsen worker 783 - jepsen.util 783 :ok :add 3441 INFO [2022-01-10 08:39:22,475] jepsen worker 849 - jepsen.util 849 :ok :add 3399 INFO [2022-01-10 08:39:22,477] jepsen worker 665 - jepsen.util 665 :invoke :add 3450 INFO [2022-01-10 08:39:22,481] jepsen worker 791 - jepsen.util 791 :invoke :add 3451 INFO [2022-01-10 08:39:22,483] jepsen worker 853 - jepsen.util 853 :invoke :add 3452 INFO [2022-01-10 08:39:22,485] jepsen worker 143 - jepsen.util 143 :invoke :add 3453 INFO [2022-01-10 08:39:22,488] jepsen worker 377 - jepsen.util 377 :invoke :add 3454 INFO [2022-01-10 08:39:22,490] jepsen worker 941 - jepsen.util 941 :invoke :add 3455 INFO [2022-01-10 08:39:22,492] jepsen worker 429 - jepsen.util 429 :invoke :add 3456 INFO [2022-01-10 08:39:22,495] jepsen worker 967 - jepsen.util 967 :invoke :add 3457 INFO [2022-01-10 08:39:22,495] jepsen worker 377 - jepsen.util 377 :ok :add 3454 INFO [2022-01-10 08:39:22,497] jepsen worker 412 - jepsen.util 412 :invoke :add 3458 INFO [2022-01-10 08:39:22,499] jepsen worker 447 - jepsen.util 447 :invoke :add 3459 INFO [2022-01-10 08:39:22,501] jepsen worker 941 - jepsen.util 941 :ok :add 3455 INFO [2022-01-10 08:39:22,502] jepsen worker 829 - jepsen.util 829 :invoke :add 3460 INFO [2022-01-10 08:39:22,504] jepsen worker 429 - jepsen.util 429 :ok :add 3456 INFO [2022-01-10 08:39:22,504] jepsen worker 780 - jepsen.util 780 :invoke :add 3461 INFO [2022-01-10 08:39:22,504] jepsen worker 412 - jepsen.util 412 :ok :add 3458 INFO [2022-01-10 08:39:22,508] jepsen worker 271 - jepsen.util 271 :invoke :add 3462 INFO [2022-01-10 08:39:22,508] jepsen worker 447 - jepsen.util 447 :ok :add 3459 INFO [2022-01-10 08:39:22,508] jepsen worker 967 - jepsen.util 967 :ok :add 3457 INFO [2022-01-10 08:39:22,512] jepsen worker 281 - jepsen.util 281 :invoke :add 3463 INFO [2022-01-10 08:39:22,512] jepsen worker 973 - jepsen.util 973 :ok :add 2788 INFO [2022-01-10 08:39:22,513] jepsen worker 611 - jepsen.util 611 :ok :add 2911 INFO [2022-01-10 08:39:22,513] jepsen worker 979 - jepsen.util 979 :ok :add 3039 INFO [2022-01-10 08:39:22,513] jepsen worker 883 - jepsen.util 883 :ok :add 3350 INFO [2022-01-10 08:39:22,513] jepsen worker 509 - jepsen.util 509 :ok :add 3422 INFO [2022-01-10 08:39:22,514] jepsen worker 953 - jepsen.util 953 :ok :add 2881 INFO [2022-01-10 08:39:22,514] jepsen worker 879 - jepsen.util 879 :ok :add 2909 INFO [2022-01-10 08:39:22,514] jepsen worker 714 - jepsen.util 714 :ok :add 3021 INFO [2022-01-10 08:39:22,514] jepsen worker 93 - jepsen.util 93 :ok :add 3015 INFO [2022-01-10 08:39:22,514] jepsen worker 688 - jepsen.util 688 :ok :add 3185 INFO [2022-01-10 08:39:22,514] jepsen worker 147 - jepsen.util 147 :ok :add 2899 INFO [2022-01-10 08:39:22,515] jepsen worker 116 - jepsen.util 116 :ok :add 3416 INFO [2022-01-10 08:39:22,515] jepsen worker 64 - jepsen.util 64 :ok :add 3348 INFO [2022-01-10 08:39:22,515] jepsen worker 311 - jepsen.util 311 :ok :add 2925 INFO [2022-01-10 08:39:22,516] jepsen worker 780 - jepsen.util 780 :ok :add 3461 INFO [2022-01-10 08:39:22,516] jepsen worker 829 - jepsen.util 829 :ok :add 3460 INFO [2022-01-10 08:39:22,516] jepsen worker 295 - jepsen.util 295 :invoke :add 3464 INFO [2022-01-10 08:39:22,517] jepsen worker 55 - jepsen.util 55 :ok :add 3014 INFO [2022-01-10 08:39:22,519] jepsen worker 640 - jepsen.util 640 :ok :add 3038 INFO [2022-01-10 08:39:22,520] jepsen worker 47 - jepsen.util 47 :ok :add 2910 INFO [2022-01-10 08:39:22,520] jepsen worker 821 - jepsen.util 821 :ok :add 3417 INFO [2022-01-10 08:39:22,520] jepsen worker 186 - jepsen.util 186 :ok :add 2789 INFO [2022-01-10 08:39:22,520] jepsen worker 789 - jepsen.util 789 :ok :add 2924 INFO [2022-01-10 08:39:22,520] jepsen worker 425 - jepsen.util 425 :ok :add 3020 INFO [2022-01-10 08:39:22,520] jepsen worker 89 - jepsen.util 89 :ok :add 2908 INFO [2022-01-10 08:39:22,520] jepsen worker 271 - jepsen.util 271 :ok :add 3462 INFO [2022-01-10 08:39:22,521] jepsen worker 351 - jepsen.util 351 :ok :add 2880 INFO [2022-01-10 08:39:22,521] jepsen worker 529 - jepsen.util 529 :ok :add 2898 INFO [2022-01-10 08:39:22,521] jepsen worker 92 - jepsen.util 92 :ok :add 3351 INFO [2022-01-10 08:39:22,521] jepsen worker 623 - jepsen.util 623 :ok :add 3423 INFO [2022-01-10 08:39:22,521] jepsen worker 248 - jepsen.util 248 :ok :add 2791 INFO [2022-01-10 08:39:22,522] jepsen worker 230 - jepsen.util 230 :ok :add 3365 INFO [2022-01-10 08:39:22,522] jepsen worker 911 - jepsen.util 911 :ok :add 3184 INFO [2022-01-10 08:39:22,528] jepsen worker 707 - jepsen.util 707 :invoke :add 3465 INFO [2022-01-10 08:39:22,531] jepsen worker 246 - jepsen.util 246 :invoke :add 3466 INFO [2022-01-10 08:39:22,533] jepsen worker 196 - jepsen.util 196 :invoke :add 3467 INFO [2022-01-10 08:39:22,535] jepsen worker 521 - jepsen.util 521 :invoke :add 3468 INFO [2022-01-10 08:39:22,537] jepsen worker 405 - jepsen.util 405 :invoke :add 3469 INFO [2022-01-10 08:39:22,539] jepsen worker 716 - jepsen.util 716 :invoke :add 3470 INFO [2022-01-10 08:39:22,543] jepsen worker 944 - jepsen.util 944 :invoke :add 3471 INFO [2022-01-10 08:39:22,545] jepsen worker 66 - jepsen.util 66 :invoke :add 3472 INFO [2022-01-10 08:39:22,548] jepsen worker 566 - jepsen.util 566 :invoke :add 3473 INFO [2022-01-10 08:39:22,551] jepsen worker 860 - jepsen.util 860 :invoke :add 3474 INFO [2022-01-10 08:39:22,553] jepsen worker 249 - jepsen.util 249 :invoke :add 3475 INFO [2022-01-10 08:39:22,554] jepsen worker 405 - jepsen.util 405 :ok :add 3469 INFO [2022-01-10 08:39:22,554] jepsen worker 944 - jepsen.util 944 :ok :add 3471 INFO [2022-01-10 08:39:22,556] jepsen worker 521 - jepsen.util 521 :ok :add 3468 INFO [2022-01-10 08:39:22,556] jepsen worker 716 - jepsen.util 716 :ok :add 3470 INFO [2022-01-10 08:39:22,556] jepsen worker 683 - jepsen.util 683 :invoke :add 3476 INFO [2022-01-10 08:39:22,557] jepsen worker 860 - jepsen.util 860 :ok :add 3474 INFO [2022-01-10 08:39:22,559] jepsen worker 528 - jepsen.util 528 :invoke :add 3477 INFO [2022-01-10 08:39:22,561] jepsen worker 566 - jepsen.util 566 :ok :add 3473 INFO [2022-01-10 08:39:22,561] jepsen worker 66 - jepsen.util 66 :ok :add 3472 INFO [2022-01-10 08:39:22,563] jepsen worker 38 - jepsen.util 38 :invoke :add 3478 INFO [2022-01-10 08:39:22,564] jepsen worker 249 - jepsen.util 249 :ok :add 3475 INFO [2022-01-10 08:39:22,566] jepsen worker 148 - jepsen.util 148 :invoke :add 3479 INFO [2022-01-10 08:39:22,566] jepsen worker 281 - jepsen.util 281 :ok :add 3463 INFO [2022-01-10 08:39:22,568] jepsen worker 528 - jepsen.util 528 :ok :add 3477 INFO [2022-01-10 08:39:22,569] jepsen worker 41 - jepsen.util 41 :invoke :add 3480 INFO [2022-01-10 08:39:22,572] jepsen worker 683 - jepsen.util 683 :ok :add 3476 INFO [2022-01-10 08:39:22,573] jepsen worker 668 - jepsen.util 668 :invoke :add 3481 INFO [2022-01-10 08:39:22,578] jepsen worker 834 - jepsen.util 834 :invoke :add 3482 INFO [2022-01-10 08:39:22,581] jepsen worker 781 - jepsen.util 781 :invoke :add 3483 INFO [2022-01-10 08:39:22,583] jepsen worker 38 - jepsen.util 38 :ok :add 3478 INFO [2022-01-10 08:39:22,584] jepsen worker 334 - jepsen.util 334 :invoke :add 3484 INFO [2022-01-10 08:39:22,587] jepsen worker 391 - jepsen.util 391 :invoke :add 3485 INFO [2022-01-10 08:39:22,589] jepsen worker 478 - jepsen.util 478 :invoke :add 3486 INFO [2022-01-10 08:39:22,592] jepsen worker 345 - jepsen.util 345 :invoke :add 3487 INFO [2022-01-10 08:39:22,594] jepsen worker 806 - jepsen.util 806 :invoke :add 3488 INFO [2022-01-10 08:39:22,596] jepsen worker 891 - jepsen.util 891 :invoke :add 3489 INFO [2022-01-10 08:39:22,598] jepsen worker 610 - jepsen.util 610 :invoke :add 3490 INFO [2022-01-10 08:39:22,600] jepsen worker 508 - jepsen.util 508 :invoke :add 3491 INFO [2022-01-10 08:39:22,601] jepsen worker 806 - jepsen.util 806 :ok :add 3488 INFO [2022-01-10 08:39:22,602] jepsen worker 150 - jepsen.util 150 :invoke :add 3492 INFO [2022-01-10 08:39:22,605] jepsen worker 669 - jepsen.util 669 :invoke :add 3493 INFO [2022-01-10 08:39:22,607] jepsen worker 805 - jepsen.util 805 :invoke :add 3494 INFO [2022-01-10 08:39:22,608] jepsen worker 891 - jepsen.util 891 :ok :add 3489 INFO [2022-01-10 08:39:22,609] jepsen worker 610 - jepsen.util 610 :ok :add 3490 INFO [2022-01-10 08:39:22,610] jepsen worker 931 - jepsen.util 931 :invoke :add 3495 INFO [2022-01-10 08:39:22,613] jepsen worker 585 - jepsen.util 585 :invoke :add 3496 INFO [2022-01-10 08:39:22,615] jepsen worker 508 - jepsen.util 508 :ok :add 3491 INFO [2022-01-10 08:39:22,616] jepsen worker 994 - jepsen.util 994 :invoke :add 3497 INFO [2022-01-10 08:39:22,618] jepsen worker 898 - jepsen.util 898 :invoke :add 3498 INFO [2022-01-10 08:39:22,619] jepsen worker 931 - jepsen.util 931 :ok :add 3495 INFO [2022-01-10 08:39:22,619] jepsen worker 150 - jepsen.util 150 :ok :add 3492 INFO [2022-01-10 08:39:22,619] jepsen worker 669 - jepsen.util 669 :ok :add 3493 INFO [2022-01-10 08:39:22,620] jepsen worker 148 - jepsen.util 148 :ok :add 3479 INFO [2022-01-10 08:39:22,621] jepsen worker 668 - jepsen.util 668 :ok :add 3481 INFO [2022-01-10 08:39:22,622] jepsen worker 362 - jepsen.util 362 :invoke :add 3499 INFO [2022-01-10 08:39:22,623] jepsen worker 994 - jepsen.util 994 :ok :add 3497 INFO [2022-01-10 08:39:22,625] jepsen worker 911 - jepsen.util 911 :invoke :add 3500 INFO [2022-01-10 08:39:22,627] jepsen worker 805 - jepsen.util 805 :ok :add 3494 INFO [2022-01-10 08:39:22,628] jepsen worker 477 - jepsen.util 477 :invoke :add 3501 INFO [2022-01-10 08:39:22,630] jepsen worker 724 - jepsen.util 724 :invoke :add 3502 INFO [2022-01-10 08:39:22,632] jepsen worker 585 - jepsen.util 585 :ok :add 3496 INFO [2022-01-10 08:39:22,633] jepsen worker 700 - jepsen.util 700 :invoke :add 3503 INFO [2022-01-10 08:39:22,635] jepsen worker 235 - jepsen.util 235 :invoke :add 3504 INFO [2022-01-10 08:39:22,635] jepsen worker 911 - jepsen.util 911 :ok :add 3500 INFO [2022-01-10 08:39:22,638] jepsen worker 573 - jepsen.util 573 :invoke :add 3505 INFO [2022-01-10 08:39:22,641] jepsen worker 724 - jepsen.util 724 :ok :add 3502 INFO [2022-01-10 08:39:22,641] jepsen worker 29 - jepsen.util 29 :invoke :add 3506 INFO [2022-01-10 08:39:22,643] jepsen worker 700 - jepsen.util 700 :ok :add 3503 INFO [2022-01-10 08:39:22,646] jepsen worker 47 - jepsen.util 47 :invoke :add 3507 INFO [2022-01-10 08:39:22,648] jepsen worker 551 - jepsen.util 551 :invoke :add 3508 INFO [2022-01-10 08:39:22,649] jepsen worker 223 - jepsen.util 223 :ok :add 2960 INFO [2022-01-10 08:39:22,649] jepsen worker 873 - jepsen.util 873 :ok :add 3285 INFO [2022-01-10 08:39:22,649] jepsen worker 165 - jepsen.util 165 :ok :add 3339 INFO [2022-01-10 08:39:22,649] jepsen worker 143 - jepsen.util 143 :ok :add 3453 INFO [2022-01-10 08:39:22,649] jepsen worker 729 - jepsen.util 729 :ok :add 3315 INFO [2022-01-10 08:39:22,649] jepsen worker 453 - jepsen.util 453 :ok :add 2954 INFO [2022-01-10 08:39:22,650] jepsen worker 27 - jepsen.util 27 :ok :add 3050 INFO [2022-01-10 08:39:22,650] jepsen worker 426 - jepsen.util 426 :ok :add 3321 INFO [2022-01-10 08:39:22,650] jepsen worker 926 - jepsen.util 926 :ok :add 2978 INFO [2022-01-10 08:39:22,650] jepsen worker 483 - jepsen.util 483 :ok :add 3048 INFO [2022-01-10 08:39:22,650] jepsen worker 672 - jepsen.util 672 :ok :add 3064 INFO [2022-01-10 08:39:22,650] jepsen worker 357 - jepsen.util 357 :invoke :add 3509 INFO [2022-01-10 08:39:22,654] jepsen worker 121 - jepsen.util 121 :invoke :add 3510 INFO [2022-01-10 08:39:22,656] jepsen worker 472 - jepsen.util 472 :invoke :add 3511 INFO [2022-01-10 08:39:22,658] jepsen worker 130 - jepsen.util 130 :invoke :add 3512 INFO [2022-01-10 08:39:22,661] jepsen worker 567 - jepsen.util 567 :invoke :add 3513 INFO [2022-01-10 08:39:22,663] jepsen worker 477 - jepsen.util 477 :ok :add 3501 INFO [2022-01-10 08:39:22,663] jepsen worker 41 - jepsen.util 41 :ok :add 3480 INFO [2022-01-10 08:39:22,664] jepsen worker 252 - jepsen.util 252 :invoke :add 3514 INFO [2022-01-10 08:39:22,666] jepsen worker 508 - jepsen.util 508 :invoke :add 3515 INFO [2022-01-10 08:39:22,668] jepsen worker 226 - jepsen.util 226 :invoke :add 3516 INFO [2022-01-10 08:39:22,670] jepsen worker 744 - jepsen.util 744 :invoke :add 3517 INFO [2022-01-10 08:39:22,673] jepsen worker 865 - jepsen.util 865 :invoke :add 3518 INFO [2022-01-10 08:39:22,675] jepsen worker 552 - jepsen.util 552 :invoke :add 3519 INFO [2022-01-10 08:39:22,677] jepsen worker 573 - jepsen.util 573 :ok :add 3505 INFO [2022-01-10 08:39:22,678] jepsen worker 29 - jepsen.util 29 :ok :add 3506 INFO [2022-01-10 08:39:22,679] jepsen worker 564 - jepsen.util 564 :invoke :add 3520 INFO [2022-01-10 08:39:22,680] jepsen worker 865 - jepsen.util 865 :ok :add 3518 INFO [2022-01-10 08:39:22,682] jepsen worker 811 - jepsen.util 811 :invoke :add 3521 INFO [2022-01-10 08:39:22,682] jepsen worker 47 - jepsen.util 47 :ok :add 3507 INFO [2022-01-10 08:39:22,683] jepsen worker 552 - jepsen.util 552 :ok :add 3519 INFO [2022-01-10 08:39:22,684] jepsen worker 235 - jepsen.util 235 :ok :add 3504 INFO [2022-01-10 08:39:22,685] jepsen worker 192 - jepsen.util 192 :invoke :add 3522 INFO [2022-01-10 08:39:22,687] jepsen worker 442 - jepsen.util 442 :invoke :add 3523 INFO [2022-01-10 08:39:22,690] jepsen worker 895 - jepsen.util 895 :invoke :add 3524 INFO [2022-01-10 08:39:22,692] jepsen worker 378 - jepsen.util 378 :invoke :add 3525 INFO [2022-01-10 08:39:22,694] jepsen worker 716 - jepsen.util 716 :invoke :add 3526 INFO [2022-01-10 08:39:22,697] jepsen worker 844 - jepsen.util 844 :invoke :add 3527 INFO [2022-01-10 08:39:22,699] jepsen worker 603 - jepsen.util 603 :invoke :add 3528 INFO [2022-01-10 08:39:22,699] jepsen worker 781 - jepsen.util 781 :ok :add 3483 INFO [2022-01-10 08:39:22,701] jepsen worker 834 - jepsen.util 834 :ok :add 3482 INFO [2022-01-10 08:39:22,701] jepsen worker 130 - jepsen.util 130 :ok :add 3512 INFO [2022-01-10 08:39:22,701] jepsen worker 91 - jepsen.util 91 :invoke :add 3529 INFO [2022-01-10 08:39:22,704] jepsen worker 732 - jepsen.util 732 :invoke :add 3530 INFO [2022-01-10 08:39:22,705] jepsen worker 603 - jepsen.util 603 :ok :add 3528 INFO [2022-01-10 08:39:22,707] jepsen worker 692 - jepsen.util 692 :invoke :add 3531 INFO [2022-01-10 08:39:22,709] jepsen worker 91 - jepsen.util 91 :ok :add 3529 INFO [2022-01-10 08:39:22,709] jepsen worker 218 - jepsen.util 218 :invoke :add 3532 INFO [2022-01-10 08:39:22,712] jepsen worker 845 - jepsen.util 845 :invoke :add 3533 INFO [2022-01-10 08:39:22,713] jepsen worker 732 - jepsen.util 732 :ok :add 3530 INFO [2022-01-10 08:39:22,714] jepsen worker 648 - jepsen.util 648 :invoke :add 3534 INFO [2022-01-10 08:39:22,715] jepsen worker 692 - jepsen.util 692 :ok :add 3531 INFO [2022-01-10 08:39:22,715] jepsen worker 218 - jepsen.util 218 :ok :add 3532 INFO [2022-01-10 08:39:22,717] jepsen worker 545 - jepsen.util 545 :invoke :add 3535 INFO [2022-01-10 08:39:22,719] jepsen worker 845 - jepsen.util 845 :ok :add 3533 INFO [2022-01-10 08:39:22,720] jepsen worker 132 - jepsen.util 132 :invoke :add 3536 INFO [2022-01-10 08:39:22,722] jepsen worker 648 - jepsen.util 648 :ok :add 3534 INFO [2022-01-10 08:39:22,724] jepsen worker 997 - jepsen.util 997 :invoke :add 3537 INFO [2022-01-10 08:39:22,727] jepsen worker 599 - jepsen.util 599 :invoke :add 3538 INFO [2022-01-10 08:39:22,729] jepsen worker 458 - jepsen.util 458 :invoke :add 3539 INFO [2022-01-10 08:39:22,732] jepsen worker 974 - jepsen.util 974 :invoke :add 3540 INFO [2022-01-10 08:39:22,734] jepsen worker 132 - jepsen.util 132 :ok :add 3536 INFO [2022-01-10 08:39:22,734] jepsen worker 284 - jepsen.util 284 :invoke :add 3541 INFO [2022-01-10 08:39:22,737] jepsen worker 491 - jepsen.util 491 :invoke :add 3542 INFO [2022-01-10 08:39:22,740] jepsen worker 904 - jepsen.util 904 :invoke :add 3543 INFO [2022-01-10 08:39:22,742] jepsen worker 974 - jepsen.util 974 :ok :add 3540 INFO [2022-01-10 08:39:22,742] jepsen worker 545 - jepsen.util 545 :ok :add 3535 INFO [2022-01-10 08:39:22,742] jepsen worker 463 - jepsen.util 463 :invoke :add 3544 INFO [2022-01-10 08:39:22,745] jepsen worker 230 - jepsen.util 230 :invoke :add 3545 INFO [2022-01-10 08:39:22,745] jepsen worker 284 - jepsen.util 284 :ok :add 3541 INFO [2022-01-10 08:39:22,747] jepsen worker 548 - jepsen.util 548 :invoke :add 3546 INFO [2022-01-10 08:39:22,748] jepsen worker 491 - jepsen.util 491 :ok :add 3542 INFO [2022-01-10 08:39:22,750] jepsen worker 398 - jepsen.util 398 :invoke :add 3547 INFO [2022-01-10 08:39:22,750] jepsen worker 904 - jepsen.util 904 :ok :add 3543 INFO [2022-01-10 08:39:22,750] jepsen worker 997 - jepsen.util 997 :ok :add 3537 INFO [2022-01-10 08:39:22,753] jepsen worker 486 - jepsen.util 486 :invoke :add 3548 INFO [2022-01-10 08:39:22,755] jepsen worker 3 - jepsen.util 3 :invoke :add 3549 INFO [2022-01-10 08:39:22,757] jepsen worker 463 - jepsen.util 463 :ok :add 3544 INFO [2022-01-10 08:39:22,758] jepsen worker 230 - jepsen.util 230 :ok :add 3545 INFO [2022-01-10 08:39:22,758] jepsen worker 199 - jepsen.util 199 :invoke :add 3550 INFO [2022-01-10 08:39:22,761] jepsen worker 528 - jepsen.util 528 :invoke :add 3551 INFO [2022-01-10 08:39:22,762] jepsen worker 548 - jepsen.util 548 :ok :add 3546 INFO [2022-01-10 08:39:22,763] jepsen worker 960 - jepsen.util 960 :invoke :add 3552 INFO [2022-01-10 08:39:22,766] jepsen worker 974 - jepsen.util 974 :invoke :add 3553 INFO [2022-01-10 08:39:22,768] jepsen worker 676 - jepsen.util 676 :invoke :add 3554 INFO [2022-01-10 08:39:22,769] jepsen worker 398 - jepsen.util 398 :ok :add 3547 INFO [2022-01-10 08:39:22,770] jepsen worker 786 - jepsen.util 786 :invoke :add 3555 INFO [2022-01-10 08:39:22,773] jepsen worker 605 - jepsen.util 605 :invoke :add 3556 INFO [2022-01-10 08:39:22,776] jepsen worker 696 - jepsen.util 696 :invoke :add 3557 INFO [2022-01-10 08:39:22,778] jepsen worker 821 - jepsen.util 821 :invoke :add 3558 INFO [2022-01-10 08:39:22,781] jepsen worker 26 - jepsen.util 26 :invoke :add 3559 INFO [2022-01-10 08:39:22,783] jepsen worker 17 - jepsen.util 17 :invoke :add 3560 INFO [2022-01-10 08:39:22,784] jepsen worker 821 - jepsen.util 821 :ok :add 3558 INFO [2022-01-10 08:39:22,786] jepsen worker 878 - jepsen.util 878 :invoke :add 3561 INFO [2022-01-10 08:39:22,787] jepsen worker 915 - jepsen.util 915 :ok :add 2926 INFO [2022-01-10 08:39:22,787] jepsen worker 28 - jepsen.util 28 :ok :add 3016 INFO [2022-01-10 08:39:22,787] jepsen worker 685 - jepsen.util 685 :ok :add 3186 INFO [2022-01-10 08:39:22,787] jepsen worker 660 - jepsen.util 660 :ok :add 3421 INFO [2022-01-10 08:39:22,787] jepsen worker 219 - jepsen.util 219 :ok :add 3415 INFO [2022-01-10 08:39:22,788] jepsen worker 262 - jepsen.util 262 :ok :add 2882 INFO [2022-01-10 08:39:22,788] jepsen worker 823 - jepsen.util 823 :ok :add 3367 INFO [2022-01-10 08:39:22,788] jepsen worker 369 - jepsen.util 369 :ok :add 3439 INFO [2022-01-10 08:39:22,788] jepsen worker 151 - jepsen.util 151 :ok :add 2793 INFO [2022-01-10 08:39:22,788] jepsen worker 654 - jepsen.util 654 :ok :add 2912 INFO [2022-01-10 08:39:22,788] jepsen worker 304 - jepsen.util 304 :ok :add 3022 INFO [2022-01-10 08:39:22,788] jepsen worker 905 - jepsen.util 905 :ok :add 3353 INFO [2022-01-10 08:39:22,789] jepsen worker 62 - jepsen.util 62 :invoke :add 3562 INFO [2022-01-10 08:39:22,791] jepsen worker 26 - jepsen.util 26 :ok :add 3559 INFO [2022-01-10 08:39:22,795] jepsen worker 178 - jepsen.util 178 :invoke :add 3563 INFO [2022-01-10 08:39:22,797] jepsen worker 13 - jepsen.util 13 :invoke :add 3564 INFO [2022-01-10 08:39:22,799] jepsen worker 193 - jepsen.util 193 :invoke :add 3565 INFO [2022-01-10 08:39:22,801] jepsen worker 717 - jepsen.util 717 :invoke :add 3566 INFO [2022-01-10 08:39:22,804] jepsen worker 228 - jepsen.util 228 :invoke :add 3567 INFO [2022-01-10 08:39:22,806] jepsen worker 240 - jepsen.util 240 :invoke :add 3568 INFO [2022-01-10 08:39:22,809] jepsen worker 457 - jepsen.util 457 :invoke :add 3569 INFO [2022-01-10 08:39:22,812] jepsen worker 16 - jepsen.util 16 :invoke :add 3570 INFO [2022-01-10 08:39:22,814] jepsen worker 821 - jepsen.util 821 :invoke :add 3571 INFO [2022-01-10 08:39:22,814] jepsen worker 240 - jepsen.util 240 :ok :add 3568 INFO [2022-01-10 08:39:22,817] jepsen worker 554 - jepsen.util 554 :invoke :add 3572 INFO [2022-01-10 08:39:22,820] jepsen worker 587 - jepsen.util 587 :invoke :add 3573 INFO [2022-01-10 08:39:22,822] jepsen worker 18 - jepsen.util 18 :invoke :add 3574 INFO [2022-01-10 08:39:22,823] jepsen worker 457 - jepsen.util 457 :ok :add 3569 INFO [2022-01-10 08:39:22,824] jepsen worker 128 - jepsen.util 128 :invoke :add 3575 INFO [2022-01-10 08:39:22,826] jepsen worker 186 - jepsen.util 186 :invoke :add 3576 INFO [2022-01-10 08:39:22,828] jepsen worker 893 - jepsen.util 893 :invoke :add 3577 INFO [2022-01-10 08:39:22,831] jepsen worker 302 - jepsen.util 302 :invoke :add 3578 INFO [2022-01-10 08:39:22,833] jepsen worker 747 - jepsen.util 747 :invoke :add 3579 INFO [2022-01-10 08:39:22,833] jepsen worker 128 - jepsen.util 128 :ok :add 3575 INFO [2022-01-10 08:39:22,836] jepsen worker 572 - jepsen.util 572 :invoke :add 3580 INFO [2022-01-10 08:39:22,838] jepsen worker 893 - jepsen.util 893 :ok :add 3577 INFO [2022-01-10 08:39:22,838] jepsen worker 925 - jepsen.util 925 :invoke :add 3581 INFO [2022-01-10 08:39:22,838] jepsen worker 18 - jepsen.util 18 :ok :add 3574 INFO [2022-01-10 08:39:22,839] jepsen worker 186 - jepsen.util 186 :ok :add 3576 INFO [2022-01-10 08:39:22,840] jepsen worker 16 - jepsen.util 16 :ok :add 3570 INFO [2022-01-10 08:39:22,840] jepsen worker 604 - jepsen.util 604 :invoke :add 3582 INFO [2022-01-10 08:39:22,844] jepsen worker 76 - jepsen.util 76 :invoke :add 3583 INFO [2022-01-10 08:39:22,844] jepsen worker 821 - jepsen.util 821 :ok :add 3571 INFO [2022-01-10 08:39:22,846] jepsen worker 554 - jepsen.util 554 :ok :add 3572 INFO [2022-01-10 08:39:22,847] jepsen worker 303 - jepsen.util 303 :invoke :add 3584 INFO [2022-01-10 08:39:22,848] jepsen worker 587 - jepsen.util 587 :ok :add 3573 INFO [2022-01-10 08:39:22,850] jepsen worker 845 - jepsen.util 845 :invoke :add 3585 INFO [2022-01-10 08:39:22,852] jepsen worker 357 - jepsen.util 357 :ok :add 3509 INFO [2022-01-10 08:39:22,852] jepsen worker 378 - jepsen.util 378 :ok :add 3525 INFO [2022-01-10 08:39:22,852] jepsen worker 362 - jepsen.util 362 :ok :add 3499 INFO [2022-01-10 08:39:22,852] jepsen worker 472 - jepsen.util 472 :ok :add 3511 INFO [2022-01-10 08:39:22,853] jepsen worker 308 - jepsen.util 308 :invoke :add 3586 INFO [2022-01-10 08:39:22,855] jepsen worker 567 - jepsen.util 567 :ok :add 3513 INFO [2022-01-10 08:39:22,855] jepsen worker 844 - jepsen.util 844 :ok :add 3527 INFO [2022-01-10 08:39:22,856] jepsen worker 825 - jepsen.util 825 :invoke :add 3587 INFO [2022-01-10 08:39:22,859] jepsen worker 470 - jepsen.util 470 :invoke :add 3588 INFO [2022-01-10 08:39:22,861] jepsen worker 60 - jepsen.util 60 :invoke :add 3589 INFO [2022-01-10 08:39:22,864] jepsen worker 367 - jepsen.util 367 :invoke :add 3590 INFO [2022-01-10 08:39:22,866] jepsen worker 688 - jepsen.util 688 :invoke :add 3591 INFO [2022-01-10 08:39:22,868] jepsen worker 434 - jepsen.util 434 :invoke :add 3592 INFO [2022-01-10 08:39:22,870] jepsen worker 415 - jepsen.util 415 :invoke :add 3593 INFO [2022-01-10 08:39:22,872] jepsen worker 323 - jepsen.util 323 :invoke :add 3594 INFO [2022-01-10 08:39:22,874] jepsen worker 745 - jepsen.util 745 :invoke :add 3595 INFO [2022-01-10 08:39:22,876] jepsen worker 367 - jepsen.util 367 :ok :add 3590 INFO [2022-01-10 08:39:22,876] jepsen worker 374 - jepsen.util 374 :invoke :add 3596 INFO [2022-01-10 08:39:22,877] jepsen worker 434 - jepsen.util 434 :ok :add 3592 INFO [2022-01-10 08:39:22,878] jepsen worker 470 - jepsen.util 470 :ok :add 3588 INFO [2022-01-10 08:39:22,879] jepsen worker 60 - jepsen.util 60 :ok :add 3589 INFO [2022-01-10 08:39:22,880] jepsen worker 666 - jepsen.util 666 :invoke :add 3597 INFO [2022-01-10 08:39:22,881] jepsen worker 323 - jepsen.util 323 :ok :add 3594 INFO [2022-01-10 08:39:22,882] jepsen worker 156 - jepsen.util 156 :invoke :add 3598 INFO [2022-01-10 08:39:22,884] jepsen worker 745 - jepsen.util 745 :ok :add 3595 INFO [2022-01-10 08:39:22,884] jepsen worker 688 - jepsen.util 688 :ok :add 3591 INFO [2022-01-10 08:39:22,885] jepsen worker 16 - jepsen.util 16 :invoke :add 3599 INFO [2022-01-10 08:39:22,887] jepsen worker 374 - jepsen.util 374 :ok :add 3596 INFO [2022-01-10 08:39:22,888] jepsen worker 415 - jepsen.util 415 :ok :add 3593 INFO [2022-01-10 08:39:22,889] jepsen worker 778 - jepsen.util 778 :invoke :add 3600 INFO [2022-01-10 08:39:22,890] jepsen worker 666 - jepsen.util 666 :ok :add 3597 INFO [2022-01-10 08:39:22,892] jepsen worker 422 - jepsen.util 422 :invoke :add 3601 INFO [2022-01-10 08:39:22,896] jepsen worker 462 - jepsen.util 462 :invoke :add 3602 INFO [2022-01-10 08:39:22,897] jepsen worker 895 - jepsen.util 895 :ok :add 3524 INFO [2022-01-10 08:39:22,897] jepsen worker 121 - jepsen.util 121 :ok :add 3510 INFO [2022-01-10 08:39:22,897] jepsen worker 898 - jepsen.util 898 :ok :add 3498 INFO [2022-01-10 08:39:22,897] jepsen worker 551 - jepsen.util 551 :ok :add 3508 INFO [2022-01-10 08:39:22,899] jepsen worker 338 - jepsen.util 338 :ok :add 3149 INFO [2022-01-10 08:39:22,899] jepsen worker 571 - jepsen.util 571 :ok :add 3214 INFO [2022-01-10 08:39:22,899] jepsen worker 815 - jepsen.util 815 :invoke :add 3603 INFO [2022-01-10 08:39:22,899] jepsen worker 960 - jepsen.util 960 :ok :add 3552 INFO [2022-01-10 08:39:22,899] jepsen worker 722 - jepsen.util 722 :ok :add 3220 INFO [2022-01-10 08:39:22,899] jepsen worker 822 - jepsen.util 822 :ok :add 3165 INFO [2022-01-10 08:39:22,899] jepsen worker 420 - jepsen.util 420 :ok :add 3384 INFO [2022-01-10 08:39:22,900] jepsen worker 775 - jepsen.util 775 :ok :add 3238 INFO [2022-01-10 08:39:22,900] jepsen worker 899 - jepsen.util 899 :ok :add 2855 INFO [2022-01-10 08:39:22,900] jepsen worker 515 - jepsen.util 515 :ok :add 3151 INFO [2022-01-10 08:39:22,901] jepsen worker 575 - jepsen.util 575 :ok :add 2770 INFO [2022-01-10 08:39:22,901] jepsen worker 987 - jepsen.util 987 :ok :add 2768 INFO [2022-01-10 08:39:22,901] jepsen worker 120 - jepsen.util 120 :ok :add 2879 INFO [2022-01-10 08:39:22,901] jepsen worker 526 - jepsen.util 526 :ok :add 2861 INFO [2022-01-10 08:39:22,903] jepsen worker 778 - jepsen.util 778 :ok :add 3600 INFO [2022-01-10 08:39:22,905] jepsen worker 315 - jepsen.util 315 :invoke :add 3604 INFO [2022-01-10 08:39:22,906] jepsen worker 422 - jepsen.util 422 :ok :add 3601 INFO [2022-01-10 08:39:22,906] jepsen worker 462 - jepsen.util 462 :ok :add 3602 INFO [2022-01-10 08:39:22,908] jepsen worker 633 - jepsen.util 633 :invoke :add 3605 INFO [2022-01-10 08:39:22,911] jepsen worker 190 - jepsen.util 190 :invoke :add 3606 INFO [2022-01-10 08:39:22,914] jepsen worker 821 - jepsen.util 821 :invoke :add 3607 INFO [2022-01-10 08:39:22,914] jepsen worker 815 - jepsen.util 815 :ok :add 3603 INFO [2022-01-10 08:39:22,915] jepsen worker 315 - jepsen.util 315 :ok :add 3604 INFO [2022-01-10 08:39:22,916] jepsen worker 633 - jepsen.util 633 :ok :add 3605 INFO [2022-01-10 08:39:22,916] jepsen worker 223 - jepsen.util 223 :invoke :add 3608 INFO [2022-01-10 08:39:22,921] jepsen worker 679 - jepsen.util 679 :invoke :add 3609 INFO [2022-01-10 08:39:22,921] jepsen worker 190 - jepsen.util 190 :ok :add 3606 INFO [2022-01-10 08:39:22,923] jepsen worker 706 - jepsen.util 706 :invoke :add 3610 INFO [2022-01-10 08:39:22,925] jepsen worker 821 - jepsen.util 821 :ok :add 3607 INFO [2022-01-10 08:39:22,926] jepsen worker 74 - jepsen.util 74 :invoke :add 3611 INFO [2022-01-10 08:39:22,928] jepsen worker 716 - jepsen.util 716 :ok :add 3526 INFO [2022-01-10 08:39:22,928] jepsen worker 676 - jepsen.util 676 :ok :add 3554 INFO [2022-01-10 08:39:22,929] jepsen worker 302 - jepsen.util 302 :ok :add 3578 INFO [2022-01-10 08:39:22,929] jepsen worker 17 - jepsen.util 17 :ok :add 3560 INFO [2022-01-10 08:39:22,930] jepsen worker 271 - jepsen.util 271 :invoke :add 3612 INFO [2022-01-10 08:39:22,933] jepsen worker 755 - jepsen.util 755 :invoke :add 3613 INFO [2022-01-10 08:39:22,935] jepsen worker 343 - jepsen.util 343 :invoke :add 3614 INFO [2022-01-10 08:39:22,937] jepsen worker 605 - jepsen.util 605 :ok :add 3556 INFO [2022-01-10 08:39:22,937] jepsen worker 62 - jepsen.util 62 :ok :add 3562 INFO [2022-01-10 08:39:22,937] jepsen worker 259 - jepsen.util 259 :invoke :add 3615 INFO [2022-01-10 08:39:22,940] jepsen worker 970 - jepsen.util 970 :invoke :add 3616 INFO [2022-01-10 08:39:22,943] jepsen worker 709 - jepsen.util 709 :invoke :add 3617 INFO [2022-01-10 08:39:22,945] jepsen worker 70 - jepsen.util 70 :invoke :add 3618 INFO [2022-01-10 08:39:22,947] jepsen worker 818 - jepsen.util 818 :invoke :add 3619 INFO [2022-01-10 08:39:22,951] jepsen worker 351 - jepsen.util 351 :invoke :add 3620 INFO [2022-01-10 08:39:22,953] jepsen worker 70 - jepsen.util 70 :ok :add 3618 INFO [2022-01-10 08:39:22,954] jepsen worker 419 - jepsen.util 419 :invoke :add 3621 INFO [2022-01-10 08:39:22,956] jepsen worker 671 - jepsen.util 671 :invoke :add 3622 INFO [2022-01-10 08:39:22,959] jepsen worker 925 - jepsen.util 925 :ok :add 3581 INFO [2022-01-10 08:39:22,960] jepsen worker 16 - jepsen.util 16 :ok :add 3599 INFO [2022-01-10 08:39:22,960] jepsen worker 257 - jepsen.util 257 :invoke :add 3623 INFO [2022-01-10 08:39:22,963] jepsen worker 818 - jepsen.util 818 :ok :add 3619 INFO [2022-01-10 08:39:22,963] jepsen worker 600 - jepsen.util 600 :invoke :add 3624 INFO [2022-01-10 08:39:22,964] jepsen worker 76 - jepsen.util 76 :ok :add 3583 INFO [2022-01-10 08:39:22,967] jepsen worker 560 - jepsen.util 560 :invoke :add 3625 INFO [2022-01-10 08:39:22,970] jepsen worker 533 - jepsen.util 533 :invoke :add 3626 INFO [2022-01-10 08:39:22,973] jepsen worker 495 - jepsen.util 495 :ok :add 3066 INFO [2022-01-10 08:39:22,973] jepsen worker 777 - jepsen.util 777 :ok :add 3051 INFO [2022-01-10 08:39:22,973] jepsen worker 951 - jepsen.util 951 :ok :add 2961 INFO [2022-01-10 08:39:22,973] jepsen worker 246 - jepsen.util 246 :ok :add 3466 INFO [2022-01-10 08:39:22,973] jepsen worker 580 - jepsen.util 580 :ok :add 3323 INFO [2022-01-10 08:39:22,973] jepsen worker 389 - jepsen.util 389 :invoke :add 3627 INFO [2022-01-10 08:39:22,973] jepsen worker 933 - jepsen.util 933 :ok :add 3287 INFO [2022-01-10 08:39:22,974] jepsen worker 499 - jepsen.util 499 :ok :add 3317 INFO [2022-01-10 08:39:22,974] jepsen worker 312 - jepsen.util 312 :ok :add 3284 INFO [2022-01-10 08:39:22,974] jepsen worker 394 - jepsen.util 394 :ok :add 3314 INFO [2022-01-10 08:39:22,974] jepsen worker 601 - jepsen.util 601 :ok :add 3049 INFO [2022-01-10 08:39:22,974] jepsen worker 908 - jepsen.util 908 :ok :add 2962 INFO [2022-01-10 08:39:22,974] jepsen worker 450 - jepsen.util 450 :ok :add 3052 INFO [2022-01-10 08:39:22,974] jepsen worker 590 - jepsen.util 590 :ok :add 3449 INFO [2022-01-10 08:39:22,974] jepsen worker 791 - jepsen.util 791 :ok :add 3451 INFO [2022-01-10 08:39:22,975] jepsen worker 244 - jepsen.util 244 :ok :add 2979 INFO [2022-01-10 08:39:22,975] jepsen worker 220 - jepsen.util 220 :ok :add 2956 INFO [2022-01-10 08:39:22,975] jepsen worker 746 - jepsen.util 746 :ok :add 3338 INFO [2022-01-10 08:39:22,975] jepsen worker 152 - jepsen.util 152 :ok :add 3065 INFO [2022-01-10 08:39:22,975] jepsen worker 707 - jepsen.util 707 :ok :add 3465 INFO [2022-01-10 08:39:22,976] jepsen worker 678 - jepsen.util 678 :ok :add 2955 INFO [2022-01-10 08:39:22,976] jepsen worker 9 - jepsen.util 9 :ok :add 3320 INFO [2022-01-10 08:39:22,976] jepsen worker 853 - jepsen.util 853 :ok :add 3452 INFO [2022-01-10 08:39:22,980] jepsen worker 694 - jepsen.util 694 :invoke :add 3628 INFO [2022-01-10 08:39:22,981] jepsen worker 223 - jepsen.util 223 :ok :add 3608 INFO [2022-01-10 08:39:22,981] jepsen worker 786 - jepsen.util 786 :ok :add 3555 INFO [2022-01-10 08:39:22,981] jepsen worker 747 - jepsen.util 747 :ok :add 3579 INFO [2022-01-10 08:39:22,981] jepsen worker 706 - jepsen.util 706 :ok :add 3610 INFO [2022-01-10 08:39:22,982] jepsen worker 878 - jepsen.util 878 :ok :add 3561 INFO [2022-01-10 08:39:22,982] jepsen worker 600 - jepsen.util 600 :ok :add 3624 INFO [2022-01-10 08:39:22,983] jepsen worker 430 - jepsen.util 430 :invoke :add 3629 INFO [2022-01-10 08:39:22,987] jepsen worker 697 - jepsen.util 697 :invoke :add 3630 INFO [2022-01-10 08:39:22,988] jepsen worker 694 - jepsen.util 694 :ok :add 3628 INFO [2022-01-10 08:39:22,989] jepsen worker 175 - jepsen.util 175 :invoke :add 3631 INFO [2022-01-10 08:39:22,990] jepsen worker 430 - jepsen.util 430 :ok :add 3629 INFO [2022-01-10 08:39:22,991] jepsen worker 707 - jepsen.util 707 :invoke :add 3632 INFO [2022-01-10 08:39:22,993] jepsen worker 499 - jepsen.util 499 :invoke :add 3633 INFO [2022-01-10 08:39:22,995] jepsen worker 681 - jepsen.util 681 :invoke :add 3634 INFO [2022-01-10 08:39:22,997] jepsen worker 429 - jepsen.util 429 :invoke :add 3635 INFO [2022-01-10 08:39:23,000] jepsen worker 409 - jepsen.util 409 :invoke :add 3636 INFO [2022-01-10 08:39:23,001] jepsen worker 697 - jepsen.util 697 :ok :add 3630 INFO [2022-01-10 08:39:23,001] jepsen worker 175 - jepsen.util 175 :ok :add 3631 INFO [2022-01-10 08:39:23,002] jepsen worker 917 - jepsen.util 917 :invoke :add 3637 INFO [2022-01-10 08:39:23,005] jepsen worker 350 - jepsen.util 350 :invoke :add 3638 INFO [2022-01-10 08:39:23,007] jepsen worker 499 - jepsen.util 499 :ok :add 3633 INFO [2022-01-10 08:39:23,007] jepsen worker 428 - jepsen.util 428 :invoke :add 3639 INFO [2022-01-10 08:39:23,008] jepsen worker 707 - jepsen.util 707 :ok :add 3632 INFO [2022-01-10 08:39:23,009] jepsen worker 696 - jepsen.util 696 :ok :add 3557 INFO [2022-01-10 08:39:23,009] jepsen worker 271 - jepsen.util 271 :ok :add 3612 INFO [2022-01-10 08:39:23,009] jepsen worker 178 - jepsen.util 178 :ok :add 3563 INFO [2022-01-10 08:39:23,009] jepsen worker 533 - jepsen.util 533 :ok :add 3626 INFO [2022-01-10 08:39:23,009] jepsen worker 429 - jepsen.util 429 :ok :add 3635 INFO [2022-01-10 08:39:23,011] jepsen worker 376 - jepsen.util 376 :invoke :add 3640 INFO [2022-01-10 08:39:23,013] jepsen worker 917 - jepsen.util 917 :ok :add 3637 INFO [2022-01-10 08:39:23,015] jepsen worker 587 - jepsen.util 587 :invoke :add 3641 INFO [2022-01-10 08:39:23,017] jepsen worker 194 - jepsen.util 194 :invoke :add 3642 INFO [2022-01-10 08:39:23,019] jepsen worker 495 - jepsen.util 495 :invoke :add 3643 INFO [2022-01-10 08:39:23,020] jepsen worker 681 - jepsen.util 681 :ok :add 3634 INFO [2022-01-10 08:39:23,022] jepsen worker 571 - jepsen.util 571 :invoke :add 3644 INFO [2022-01-10 08:39:23,024] jepsen worker 374 - jepsen.util 374 :invoke :add 3645 INFO [2022-01-10 08:39:23,025] jepsen worker 409 - jepsen.util 409 :ok :add 3636 INFO [2022-01-10 08:39:23,027] jepsen worker 632 - jepsen.util 632 :invoke :add 3646 INFO [2022-01-10 08:39:23,029] jepsen worker 66 - jepsen.util 66 :invoke :add 3647 INFO [2022-01-10 08:39:23,030] jepsen worker 587 - jepsen.util 587 :ok :add 3641 INFO [2022-01-10 08:39:23,032] jepsen worker 373 - jepsen.util 373 :invoke :add 3648 INFO [2022-01-10 08:39:23,033] jepsen worker 376 - jepsen.util 376 :ok :add 3640 INFO [2022-01-10 08:39:23,034] jepsen worker 195 - jepsen.util 195 :invoke :add 3649 INFO [2022-01-10 08:39:23,034] jepsen worker 571 - jepsen.util 571 :ok :add 3644 INFO [2022-01-10 08:39:23,034] jepsen worker 572 - jepsen.util 572 :ok :add 3580 INFO [2022-01-10 08:39:23,035] jepsen worker 156 - jepsen.util 156 :ok :add 3598 INFO [2022-01-10 08:39:23,037] jepsen worker 656 - jepsen.util 656 :invoke :add 3650 INFO [2022-01-10 08:39:23,038] jepsen worker 495 - jepsen.util 495 :ok :add 3643 INFO [2022-01-10 08:39:23,039] jepsen worker 632 - jepsen.util 632 :ok :add 3646 INFO [2022-01-10 08:39:23,040] jepsen worker 679 - jepsen.util 679 :ok :add 3609 INFO [2022-01-10 08:39:23,040] jepsen worker 74 - jepsen.util 74 :ok :add 3611 INFO [2022-01-10 08:39:23,040] jepsen worker 560 - jepsen.util 560 :ok :add 3625 INFO [2022-01-10 08:39:23,040] jepsen worker 604 - jepsen.util 604 :ok :add 3582 INFO [2022-01-10 08:39:23,041] jepsen worker 935 - jepsen.util 935 :invoke :add 3651 INFO [2022-01-10 08:39:23,041] jepsen worker 374 - jepsen.util 374 :ok :add 3645 INFO [2022-01-10 08:39:23,043] jepsen worker 194 - jepsen.util 194 :ok :add 3642 INFO [2022-01-10 08:39:23,045] jepsen worker 63 - jepsen.util 63 :invoke :add 3652 INFO [2022-01-10 08:39:23,047] jepsen worker 66 - jepsen.util 66 :ok :add 3647 INFO [2022-01-10 08:39:23,048] jepsen worker 455 - jepsen.util 455 :invoke :add 3653 INFO [2022-01-10 08:39:23,051] jepsen worker 68 - jepsen.util 68 :invoke :add 3654 INFO [2022-01-10 08:39:23,053] jepsen worker 612 - jepsen.util 612 :invoke :add 3655 INFO [2022-01-10 08:39:23,056] jepsen worker 834 - jepsen.util 834 :invoke :add 3656 INFO [2022-01-10 08:39:23,058] jepsen worker 301 - jepsen.util 301 :invoke :add 3657 INFO [2022-01-10 08:39:23,060] jepsen worker 644 - jepsen.util 644 :invoke :add 3658 INFO [2022-01-10 08:39:23,063] jepsen worker 348 - jepsen.util 348 :invoke :add 3659 INFO [2022-01-10 08:39:23,065] jepsen worker 358 - jepsen.util 358 :invoke :add 3660 INFO [2022-01-10 08:39:23,067] jepsen worker 22 - jepsen.util 22 :invoke :add 3661 INFO [2022-01-10 08:39:23,069] jepsen worker 644 - jepsen.util 644 :ok :add 3658 INFO [2022-01-10 08:39:23,069] jepsen worker 669 - jepsen.util 669 :invoke :add 3662 INFO [2022-01-10 08:39:23,072] jepsen worker 555 - jepsen.util 555 :invoke :add 3663 INFO [2022-01-10 08:39:23,072] jepsen worker 348 - jepsen.util 348 :ok :add 3659 INFO [2022-01-10 08:39:23,074] jepsen worker 781 - jepsen.util 781 :invoke :add 3664 INFO [2022-01-10 08:39:23,077] jepsen worker 236 - jepsen.util 236 :invoke :add 3665 WARN [2022-01-10 08:39:23,078] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":1891216,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":1891216,"total_server_duration_us":1889496,"encode_duration_us":454,"operation_id":"0xd69","timeout_ms":10000,"last_server_duration_us":1889496,"total_duration_us":1891529},{"operation_name":"insert","last_dispatch_duration_us":1703944,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":1703944,"total_server_duration_us":1701130,"encode_duration_us":289,"operation_id":"0xd52","timeout_ms":10000,"last_server_duration_us":1701130,"total_duration_us":1704161},{"operation_name":"insert","last_dispatch_duration_us":1645342,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":1645342,"total_server_duration_us":319455,"encode_duration_us":324,"operation_id":"0xe28","timeout_ms":10000,"last_server_duration_us":319455,"total_duration_us":1645608},{"operation_name":"insert","last_dispatch_duration_us":1630872,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":1630872,"total_server_duration_us":184327,"encode_duration_us":277,"operation_id":"0xdfd","timeout_ms":10000,"last_server_duration_us":184327,"total_duration_us":1631125},{"operation_name":"insert","last_dispatch_duration_us":1615510,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":1615510,"total_server_duration_us":1613941,"encode_duration_us":292,"operation_id":"0xd6c","timeout_ms":10000,"last_server_duration_us":1613941,"total_duration_us":1615773},{"operation_name":"insert","last_dispatch_duration_us":1509561,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":1509561,"total_server_duration_us":1508331,"encode_duration_us":311,"operation_id":"0xd0e","timeout_ms":10000,"last_server_duration_us":1508331,"total_duration_us":1509874},{"operation_name":"insert","last_dispatch_duration_us":1500888,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":1500888,"total_server_duration_us":1499421,"encode_duration_us":328,"operation_id":"0xcfc","timeout_ms":10000,"last_server_duration_us":1499421,"total_duration_us":1501122},{"operation_name":"insert","last_dispatch_duration_us":1472715,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":1472715,"total_server_duration_us":1471352,"encode_duration_us":305,"operation_id":"0xd0c","timeout_ms":10000,"last_server_duration_us":1471352,"total_duration_us":1473019},{"operation_name":"insert","last_dispatch_duration_us":1468114,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":1468114,"total_server_duration_us":1466941,"encode_duration_us":342,"operation_id":"0xd2b","timeout_ms":10000,"last_server_duration_us":1466941,"total_duration_us":1468931},{"operation_name":"insert","last_dispatch_duration_us":1365744,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":1365744,"total_server_duration_us":1364667,"encode_duration_us":335,"operation_id":"0xd94","timeout_ms":10000,"last_server_duration_us":1364667,"total_duration_us":1366032}],"total_count":20}} INFO [2022-01-10 08:39:23,079] jepsen worker 755 - jepsen.util 755 :ok :add 3613 INFO [2022-01-10 08:39:23,079] jepsen worker 389 - jepsen.util 389 :ok :add 3627 INFO [2022-01-10 08:39:23,080] jepsen worker 331 - jepsen.util 331 :invoke :add 3666 INFO [2022-01-10 08:39:23,081] jepsen worker 830 - jepsen.util 830 :ok :add 2963 INFO [2022-01-10 08:39:23,081] jepsen worker 880 - jepsen.util 880 :ok :add 2957 INFO [2022-01-10 08:39:23,082] jepsen worker 295 - jepsen.util 295 :ok :add 3464 INFO [2022-01-10 08:39:23,082] jepsen worker 808 - jepsen.util 808 :ok :add 3053 INFO [2022-01-10 08:39:23,083] jepsen worker 449 - jepsen.util 449 :ok :add 3322 INFO [2022-01-10 08:39:23,083] jepsen worker 858 - jepsen.util 858 :ok :add 3286 INFO [2022-01-10 08:39:23,083] jepsen worker 594 - jepsen.util 594 :ok :add 3316 INFO [2022-01-10 08:39:23,083] jepsen worker 291 - jepsen.util 291 :ok :add 3067 INFO [2022-01-10 08:39:23,083] jepsen worker 665 - jepsen.util 665 :ok :add 3450 INFO [2022-01-10 08:39:23,083] jepsen worker 693 - jepsen.util 693 :ok :add 3448 INFO [2022-01-10 08:39:23,085] jepsen worker 570 - jepsen.util 570 :invoke :add 3667 INFO [2022-01-10 08:39:23,089] jepsen worker 186 - jepsen.util 186 :invoke :add 3668 INFO [2022-01-10 08:39:23,092] jepsen worker 153 - jepsen.util 153 :invoke :add 3669 INFO [2022-01-10 08:39:23,094] jepsen worker 276 - jepsen.util 276 :invoke :add 3670 INFO [2022-01-10 08:39:23,097] jepsen worker 664 - jepsen.util 664 :invoke :add 3671 INFO [2022-01-10 08:39:23,099] jepsen worker 981 - jepsen.util 981 :invoke :add 3672 INFO [2022-01-10 08:39:23,099] jepsen worker 186 - jepsen.util 186 :ok :add 3668 INFO [2022-01-10 08:39:23,101] jepsen worker 619 - jepsen.util 619 :invoke :add 3673 INFO [2022-01-10 08:39:23,104] jepsen worker 777 - jepsen.util 777 :invoke :add 3674 INFO [2022-01-10 08:39:23,104] jepsen worker 153 - jepsen.util 153 :ok :add 3669 INFO [2022-01-10 08:39:23,105] jepsen worker 276 - jepsen.util 276 :ok :add 3670 INFO [2022-01-10 08:39:23,107] jepsen worker 943 - jepsen.util 943 :invoke :add 3675 INFO [2022-01-10 08:39:23,110] jepsen worker 664 - jepsen.util 664 :ok :add 3671 INFO [2022-01-10 08:39:23,110] jepsen worker 981 - jepsen.util 981 :ok :add 3672 INFO [2022-01-10 08:39:23,111] jepsen worker 249 - jepsen.util 249 :invoke :add 3676 INFO [2022-01-10 08:39:23,114] jepsen worker 777 - jepsen.util 777 :ok :add 3674 INFO [2022-01-10 08:39:23,114] jepsen worker 867 - jepsen.util 867 :invoke :add 3677 INFO [2022-01-10 08:39:23,115] jepsen worker 619 - jepsen.util 619 :ok :add 3673 INFO [2022-01-10 08:39:23,116] jepsen worker 22 - jepsen.util 22 :ok :add 3661 INFO [2022-01-10 08:39:23,117] jepsen worker 612 - jepsen.util 612 :ok :add 3655 INFO [2022-01-10 08:39:23,118] jepsen worker 820 - jepsen.util 820 :invoke :add 3678 INFO [2022-01-10 08:39:23,121] jepsen worker 676 - jepsen.util 676 :invoke :add 3679 INFO [2022-01-10 08:39:23,121] jepsen worker 555 - jepsen.util 555 :ok :add 3663 INFO [2022-01-10 08:39:23,122] jepsen worker 301 - jepsen.util 301 :ok :add 3657 INFO [2022-01-10 08:39:23,125] jepsen worker 943 - jepsen.util 943 :ok :add 3675 INFO [2022-01-10 08:39:23,126] jepsen worker 476 - jepsen.util 476 :invoke :add 3680 INFO [2022-01-10 08:39:23,129] jepsen worker 481 - jepsen.util 481 :invoke :add 3681 INFO [2022-01-10 08:39:23,129] jepsen worker 249 - jepsen.util 249 :ok :add 3676 INFO [2022-01-10 08:39:23,132] jepsen worker 660 - jepsen.util 660 :invoke :add 3682 INFO [2022-01-10 08:39:23,132] jepsen worker 867 - jepsen.util 867 :ok :add 3677 INFO [2022-01-10 08:39:23,135] jepsen worker 746 - jepsen.util 746 :invoke :add 3683 INFO [2022-01-10 08:39:23,138] jepsen worker 611 - jepsen.util 611 :invoke :add 3684 INFO [2022-01-10 08:39:23,141] jepsen worker 406 - jepsen.util 406 :invoke :add 3685 INFO [2022-01-10 08:39:23,143] jepsen worker 507 - jepsen.util 507 :invoke :add 3686 INFO [2022-01-10 08:39:23,144] jepsen worker 358 - jepsen.util 358 :ok :add 3660 INFO [2022-01-10 08:39:23,144] jepsen worker 68 - jepsen.util 68 :ok :add 3654 INFO [2022-01-10 08:39:23,144] jepsen worker 820 - jepsen.util 820 :ok :add 3678 INFO [2022-01-10 08:39:23,146] jepsen worker 621 - jepsen.util 621 :invoke :add 3687 INFO [2022-01-10 08:39:23,149] jepsen worker 834 - jepsen.util 834 :ok :add 3656 INFO [2022-01-10 08:39:23,149] jepsen worker 669 - jepsen.util 669 :ok :add 3662 INFO [2022-01-10 08:39:23,150] jepsen worker 658 - jepsen.util 658 :invoke :add 3688 INFO [2022-01-10 08:39:23,152] jepsen worker 818 - jepsen.util 818 :invoke :add 3689 INFO [2022-01-10 08:39:23,156] jepsen worker 661 - jepsen.util 661 :invoke :add 3690 INFO [2022-01-10 08:39:23,158] jepsen worker 587 - jepsen.util 587 :invoke :add 3691 INFO [2022-01-10 08:39:23,158] jepsen worker 658 - jepsen.util 658 :ok :add 3688 INFO [2022-01-10 08:39:23,160] jepsen worker 773 - jepsen.util 773 :invoke :add 3692 INFO [2022-01-10 08:39:23,161] jepsen worker 481 - jepsen.util 481 :ok :add 3681 INFO [2022-01-10 08:39:23,163] jepsen worker 461 - jepsen.util 461 :invoke :add 3693 INFO [2022-01-10 08:39:23,166] jepsen worker 961 - jepsen.util 961 :invoke :add 3694 INFO [2022-01-10 08:39:23,166] jepsen worker 818 - jepsen.util 818 :ok :add 3689 INFO [2022-01-10 08:39:23,167] jepsen worker 661 - jepsen.util 661 :ok :add 3690 INFO [2022-01-10 08:39:23,168] jepsen worker 430 - jepsen.util 430 :invoke :add 3695 INFO [2022-01-10 08:39:23,168] jepsen worker 746 - jepsen.util 746 :ok :add 3683 INFO [2022-01-10 08:39:23,171] jepsen worker 380 - jepsen.util 380 :invoke :add 3696 INFO [2022-01-10 08:39:23,173] jepsen worker 461 - jepsen.util 461 :ok :add 3693 INFO [2022-01-10 08:39:23,173] jepsen worker 773 - jepsen.util 773 :ok :add 3692 INFO [2022-01-10 08:39:23,174] jepsen worker 34 - jepsen.util 34 :invoke :add 3697 INFO [2022-01-10 08:39:23,175] jepsen worker 961 - jepsen.util 961 :ok :add 3694 INFO [2022-01-10 08:39:23,178] jepsen worker 587 - jepsen.util 587 :ok :add 3691 INFO [2022-01-10 08:39:23,178] jepsen worker 352 - jepsen.util 352 :invoke :add 3698 INFO [2022-01-10 08:39:23,179] jepsen worker 676 - jepsen.util 676 :ok :add 3679 INFO [2022-01-10 08:39:23,181] jepsen worker 838 - jepsen.util 838 :invoke :add 3699 INFO [2022-01-10 08:39:23,182] jepsen worker 380 - jepsen.util 380 :ok :add 3696 INFO [2022-01-10 08:39:23,185] jepsen worker 479 - jepsen.util 479 :invoke :add 3701 INFO [2022-01-10 08:39:23,186] jepsen worker 430 - jepsen.util 430 :ok :add 3695 INFO [2022-01-10 08:39:23,188] jepsen worker 237 - jepsen.util 237 :invoke :add 3700 INFO [2022-01-10 08:39:23,188] jepsen worker 723 - jepsen.util 723 :invoke :add 3702 INFO [2022-01-10 08:39:23,190] jepsen worker 583 - jepsen.util 583 :invoke :add 3703 INFO [2022-01-10 08:39:23,192] jepsen worker 34 - jepsen.util 34 :ok :add 3697 INFO [2022-01-10 08:39:23,193] jepsen worker 746 - jepsen.util 746 :invoke :add 3704 INFO [2022-01-10 08:39:23,195] jepsen worker 517 - jepsen.util 517 :invoke :add 3705 INFO [2022-01-10 08:39:23,197] jepsen worker 479 - jepsen.util 479 :ok :add 3701 INFO [2022-01-10 08:39:23,198] jepsen worker 390 - jepsen.util 390 :invoke :add 3706 INFO [2022-01-10 08:39:23,199] jepsen worker 237 - jepsen.util 237 :ok :add 3700 INFO [2022-01-10 08:39:23,201] jepsen worker 515 - jepsen.util 515 :invoke :add 3707 INFO [2022-01-10 08:39:23,202] jepsen worker 583 - jepsen.util 583 :ok :add 3703 INFO [2022-01-10 08:39:23,203] jepsen worker 136 - jepsen.util 136 :invoke :add 3708 INFO [2022-01-10 08:39:23,204] jepsen worker 746 - jepsen.util 746 :ok :add 3704 INFO [2022-01-10 08:39:23,206] jepsen worker 399 - jepsen.util 399 :invoke :add 3709 INFO [2022-01-10 08:39:23,208] jepsen worker 390 - jepsen.util 390 :ok :add 3706 INFO [2022-01-10 08:39:23,209] jepsen worker 504 - jepsen.util 504 :invoke :add 3710 INFO [2022-01-10 08:39:23,209] jepsen worker 723 - jepsen.util 723 :ok :add 3702 INFO [2022-01-10 08:39:23,211] jepsen worker 517 - jepsen.util 517 :ok :add 3705 INFO [2022-01-10 08:39:23,212] jepsen worker 476 - jepsen.util 476 :ok :add 3680 INFO [2022-01-10 08:39:23,212] jepsen worker 352 - jepsen.util 352 :ok :add 3698 INFO [2022-01-10 08:39:23,212] jepsen worker 663 - jepsen.util 663 :invoke :add 3711 INFO [2022-01-10 08:39:23,217] jepsen worker 660 - jepsen.util 660 :ok :add 3682 INFO [2022-01-10 08:39:23,217] jepsen worker 221 - jepsen.util 221 :invoke :add 3712 INFO [2022-01-10 08:39:23,217] jepsen worker 515 - jepsen.util 515 :ok :add 3707 INFO [2022-01-10 08:39:23,218] jepsen worker 838 - jepsen.util 838 :ok :add 3699 INFO [2022-01-10 08:39:23,219] jepsen worker 399 - jepsen.util 399 :ok :add 3709 INFO [2022-01-10 08:39:23,221] jepsen worker 934 - jepsen.util 934 :invoke :add 3713 INFO [2022-01-10 08:39:23,224] jepsen worker 491 - jepsen.util 491 :invoke :add 3714 INFO [2022-01-10 08:39:23,227] jepsen worker 839 - jepsen.util 839 :invoke :add 3715 INFO [2022-01-10 08:39:23,229] jepsen worker 109 - jepsen.util 109 :invoke :add 3716 INFO [2022-01-10 08:39:23,231] jepsen worker 48 - jepsen.util 48 :invoke :add 3717 INFO [2022-01-10 08:39:23,233] jepsen worker 459 - jepsen.util 459 :invoke :add 3718 INFO [2022-01-10 08:39:23,236] jepsen worker 427 - jepsen.util 427 :invoke :add 3719 INFO [2022-01-10 08:39:23,238] jepsen worker 180 - jepsen.util 180 :invoke :add 3720 INFO [2022-01-10 08:39:23,241] jepsen worker 767 - jepsen.util 767 :invoke :add 3721 INFO [2022-01-10 08:39:23,243] jepsen worker 427 - jepsen.util 427 :ok :add 3719 INFO [2022-01-10 08:39:23,244] jepsen worker 435 - jepsen.util 435 :invoke :add 3722 INFO [2022-01-10 08:39:23,246] jepsen worker 459 - jepsen.util 459 :ok :add 3718 INFO [2022-01-10 08:39:23,248] jepsen worker 447 - jepsen.util 447 :invoke :add 3723 INFO [2022-01-10 08:39:23,252] jepsen worker 740 - jepsen.util 740 :invoke :add 3724 INFO [2022-01-10 08:39:23,253] jepsen worker 663 - jepsen.util 663 :ok :add 3711 INFO [2022-01-10 08:39:23,255] jepsen worker 304 - jepsen.util 304 :invoke :add 3725 INFO [2022-01-10 08:39:23,258] jepsen worker 392 - jepsen.util 392 :invoke :add 3726 INFO [2022-01-10 08:39:23,258] jepsen worker 934 - jepsen.util 934 :ok :add 3713 INFO [2022-01-10 08:39:23,260] jepsen worker 534 - jepsen.util 534 :invoke :add 3727 INFO [2022-01-10 08:39:23,262] jepsen worker 501 - jepsen.util 501 :invoke :add 3728 INFO [2022-01-10 08:39:23,263] jepsen worker 136 - jepsen.util 136 :ok :add 3708 INFO [2022-01-10 08:39:23,263] jepsen worker 504 - jepsen.util 504 :ok :add 3710 INFO [2022-01-10 08:39:23,263] jepsen worker 740 - jepsen.util 740 :ok :add 3724 INFO [2022-01-10 08:39:23,264] jepsen worker 304 - jepsen.util 304 :ok :add 3725 INFO [2022-01-10 08:39:23,264] jepsen worker 487 - jepsen.util 487 :invoke :add 3729 INFO [2022-01-10 08:39:23,267] jepsen worker 460 - jepsen.util 460 :invoke :add 3730 INFO [2022-01-10 08:39:23,269] jepsen worker 221 - jepsen.util 221 :ok :add 3712 INFO [2022-01-10 08:39:23,269] jepsen worker 534 - jepsen.util 534 :ok :add 3727 INFO [2022-01-10 08:39:23,272] jepsen worker 501 - jepsen.util 501 :ok :add 3728 INFO [2022-01-10 08:39:23,273] jepsen worker 416 - jepsen.util 416 :invoke :add 3732 INFO [2022-01-10 08:39:23,274] jepsen worker 487 - jepsen.util 487 :ok :add 3729 INFO [2022-01-10 08:39:23,274] jepsen worker 584 - jepsen.util 584 :invoke :add 3731 INFO [2022-01-10 08:39:23,276] jepsen worker 60 - jepsen.util 60 :invoke :add 3733 INFO [2022-01-10 08:39:23,279] jepsen worker 655 - jepsen.util 655 :invoke :add 3734 INFO [2022-01-10 08:39:23,280] jepsen worker 460 - jepsen.util 460 :ok :add 3730 INFO [2022-01-10 08:39:23,282] jepsen worker 93 - jepsen.util 93 :invoke :add 3735 INFO [2022-01-10 08:39:23,282] jepsen worker 392 - jepsen.util 392 :ok :add 3726 INFO [2022-01-10 08:39:23,284] jepsen worker 416 - jepsen.util 416 :ok :add 3732 INFO [2022-01-10 08:39:23,285] jepsen worker 762 - jepsen.util 762 :invoke :add 3736 INFO [2022-01-10 08:39:23,287] jepsen worker 32 - jepsen.util 32 :invoke :add 3737 INFO [2022-01-10 08:39:23,289] jepsen worker 584 - jepsen.util 584 :ok :add 3731 INFO [2022-01-10 08:39:23,290] jepsen worker 64 - jepsen.util 64 :invoke :add 3738 INFO [2022-01-10 08:39:23,291] jepsen worker 60 - jepsen.util 60 :ok :add 3733 INFO [2022-01-10 08:39:23,292] jepsen worker 237 - jepsen.util 237 :invoke :add 3739 INFO [2022-01-10 08:39:23,295] jepsen worker 385 - jepsen.util 385 :invoke :add 3740 INFO [2022-01-10 08:39:23,297] jepsen worker 976 - jepsen.util 976 :invoke :add 3741 INFO [2022-01-10 08:39:23,299] jepsen worker 77 - jepsen.util 77 :invoke :add 3742 INFO [2022-01-10 08:39:23,302] jepsen worker 282 - jepsen.util 282 :invoke :add 3743 INFO [2022-01-10 08:39:23,304] jepsen worker 421 - jepsen.util 421 :invoke :add 3744 INFO [2022-01-10 08:39:23,306] jepsen worker 790 - jepsen.util 790 :invoke :add 3745 INFO [2022-01-10 08:39:23,306] jepsen worker 385 - jepsen.util 385 :ok :add 3740 INFO [2022-01-10 08:39:23,307] jepsen worker 976 - jepsen.util 976 :ok :add 3741 INFO [2022-01-10 08:39:23,308] jepsen worker 585 - jepsen.util 585 :invoke :add 3746 INFO [2022-01-10 08:39:23,310] jepsen worker 77 - jepsen.util 77 :ok :add 3742 INFO [2022-01-10 08:39:23,310] jepsen worker 282 - jepsen.util 282 :ok :add 3743 INFO [2022-01-10 08:39:23,311] jepsen worker 760 - jepsen.util 760 :invoke :add 3747 INFO [2022-01-10 08:39:23,314] jepsen worker 460 - jepsen.util 460 :invoke :add 3748 INFO [2022-01-10 08:39:23,317] jepsen worker 431 - jepsen.util 431 :invoke :add 3749 INFO [2022-01-10 08:39:23,322] jepsen worker 93 - jepsen.util 93 :ok :add 3735 INFO [2022-01-10 08:39:23,322] jepsen worker 655 - jepsen.util 655 :ok :add 3734 INFO [2022-01-10 08:39:23,323] jepsen worker 281 - jepsen.util 281 :invoke :add 3750 INFO [2022-01-10 08:39:23,326] jepsen worker 920 - jepsen.util 920 :invoke :add 3751 INFO [2022-01-10 08:39:23,326] jepsen worker 32 - jepsen.util 32 :ok :add 3737 INFO [2022-01-10 08:39:23,326] jepsen worker 762 - jepsen.util 762 :ok :add 3736 INFO [2022-01-10 08:39:23,328] jepsen worker 498 - jepsen.util 498 :invoke :add 3752 INFO [2022-01-10 08:39:23,332] jepsen worker 52 - jepsen.util 52 :invoke :add 3753 INFO [2022-01-10 08:39:23,333] jepsen worker 421 - jepsen.util 421 :ok :add 3744 INFO [2022-01-10 08:39:23,335] jepsen worker 595 - jepsen.util 595 :invoke :add 3754 INFO [2022-01-10 08:39:23,337] jepsen worker 325 - jepsen.util 325 :invoke :add 3755 INFO [2022-01-10 08:39:23,338] jepsen worker 790 - jepsen.util 790 :ok :add 3745 INFO [2022-01-10 08:39:23,341] jepsen worker 448 - jepsen.util 448 :invoke :add 3756 INFO [2022-01-10 08:39:23,343] jepsen worker 585 - jepsen.util 585 :ok :add 3746 INFO [2022-01-10 08:39:23,345] jepsen worker 695 - jepsen.util 695 :invoke :add 3757 INFO [2022-01-10 08:39:23,345] jepsen worker 760 - jepsen.util 760 :ok :add 3747 INFO [2022-01-10 08:39:23,352] jepsen worker 366 - jepsen.util 366 :invoke :add 3758 INFO [2022-01-10 08:39:23,352] jepsen worker 471 - jepsen.util 471 :ok :add 3121 INFO [2022-01-10 08:39:23,352] jepsen worker 735 - jepsen.util 735 :ok :add 3250 INFO [2022-01-10 08:39:23,352] jepsen worker 478 - jepsen.util 478 :ok :add 3486 INFO [2022-01-10 08:39:23,352] jepsen worker 536 - jepsen.util 536 :ok :add 3115 INFO [2022-01-10 08:39:23,355] jepsen worker 325 - jepsen.util 325 :ok :add 3755 INFO [2022-01-10 08:39:23,356] jepsen worker 114 - jepsen.util 114 :ok :add 3248 INFO [2022-01-10 08:39:23,356] jepsen worker 305 - jepsen.util 305 :invoke :add 3759 INFO [2022-01-10 08:39:23,357] jepsen worker 192 - jepsen.util 192 :ok :add 3522 INFO [2022-01-10 08:39:23,357] jepsen worker 753 - jepsen.util 753 :ok :add 3139 INFO [2022-01-10 08:39:23,357] jepsen worker 570 - jepsen.util 570 :ok :add 3667 INFO [2022-01-10 08:39:23,357] jepsen worker 347 - jepsen.util 347 :ok :add 3264 INFO [2022-01-10 08:39:23,357] jepsen worker 455 - jepsen.util 455 :ok :add 3653 INFO [2022-01-10 08:39:23,358] jepsen worker 701 - jepsen.util 701 :ok :add 2981 INFO [2022-01-10 08:39:23,358] jepsen worker 226 - jepsen.util 226 :ok :add 3516 INFO [2022-01-10 08:39:23,358] jepsen worker 942 - jepsen.util 942 :ok :add 2811 INFO [2022-01-10 08:39:23,358] jepsen worker 558 - jepsen.util 558 :ok :add 2999 INFO [2022-01-10 08:39:23,360] jepsen worker 595 - jepsen.util 595 :ok :add 3754 INFO [2022-01-10 08:39:23,360] jepsen worker 366 - jepsen.util 366 :ok :add 3758 INFO [2022-01-10 08:39:23,362] jepsen worker 695 - jepsen.util 695 :ok :add 3757 INFO [2022-01-10 08:39:23,362] jepsen worker 169 - jepsen.util 169 :invoke :add 3760 INFO [2022-01-10 08:39:23,365] jepsen worker 27 - jepsen.util 27 :invoke :add 3761 INFO [2022-01-10 08:39:23,366] jepsen worker 305 - jepsen.util 305 :ok :add 3759 INFO [2022-01-10 08:39:23,368] jepsen worker 120 - jepsen.util 120 :invoke :add 3762 INFO [2022-01-10 08:39:23,371] jepsen worker 841 - jepsen.util 841 :invoke :add 3763 INFO [2022-01-10 08:39:23,375] jepsen worker 485 - jepsen.util 485 :invoke :add 3764 INFO [2022-01-10 08:39:23,377] jepsen worker 724 - jepsen.util 724 :invoke :add 3765 INFO [2022-01-10 08:39:23,380] jepsen worker 451 - jepsen.util 451 :invoke :add 3766 INFO [2022-01-10 08:39:23,382] jepsen worker 81 - jepsen.util 81 :invoke :add 3767 INFO [2022-01-10 08:39:23,384] jepsen worker 131 - jepsen.util 131 :invoke :add 3768 INFO [2022-01-10 08:39:23,387] jepsen worker 643 - jepsen.util 643 :invoke :add 3769 INFO [2022-01-10 08:39:23,389] jepsen worker 138 - jepsen.util 138 :invoke :add 3770 INFO [2022-01-10 08:39:23,391] jepsen worker 565 - jepsen.util 565 :invoke :add 3771 INFO [2022-01-10 08:39:23,393] jepsen worker 448 - jepsen.util 448 :ok :add 3756 INFO [2022-01-10 08:39:23,393] jepsen worker 715 - jepsen.util 715 :invoke :add 3772 INFO [2022-01-10 08:39:23,396] jepsen worker 30 - jepsen.util 30 :invoke :add 3773 INFO [2022-01-10 08:39:23,397] jepsen worker 841 - jepsen.util 841 :ok :add 3763 INFO [2022-01-10 08:39:23,398] jepsen worker 691 - jepsen.util 691 :invoke :add 3774 INFO [2022-01-10 08:39:23,400] jepsen worker 643 - jepsen.util 643 :ok :add 3769 INFO [2022-01-10 08:39:23,401] jepsen worker 544 - jepsen.util 544 :invoke :add 3775 INFO [2022-01-10 08:39:23,402] jepsen worker 27 - jepsen.util 27 :ok :add 3761 INFO [2022-01-10 08:39:23,404] jepsen worker 847 - jepsen.util 847 :invoke :add 3776 INFO [2022-01-10 08:39:23,405] jepsen worker 113 - jepsen.util 113 :ok :add 3087 INFO [2022-01-10 08:39:23,405] jepsen worker 935 - jepsen.util 935 :ok :add 3651 INFO [2022-01-10 08:39:23,405] jepsen worker 79 - jepsen.util 79 :ok :add 3117 INFO [2022-01-10 08:39:23,405] jepsen worker 368 - jepsen.util 368 :ok :add 3266 INFO [2022-01-10 08:39:23,405] jepsen worker 65 - jepsen.util 65 :ok :add 3252 INFO [2022-01-10 08:39:23,405] jepsen worker 252 - jepsen.util 252 :ok :add 3514 INFO [2022-01-10 08:39:23,405] jepsen worker 334 - jepsen.util 334 :ok :add 3484 INFO [2022-01-10 08:39:23,405] jepsen worker 387 - jepsen.util 387 :ok :add 2983 INFO [2022-01-10 08:39:23,405] jepsen worker 599 - jepsen.util 599 :ok :add 3538 INFO [2022-01-10 08:39:23,405] jepsen worker 564 - jepsen.util 564 :ok :add 3520 INFO [2022-01-10 08:39:23,406] jepsen worker 236 - jepsen.util 236 :ok :add 3665 INFO [2022-01-10 08:39:23,406] jepsen worker 195 - jepsen.util 195 :ok :add 3649 INFO [2022-01-10 08:39:23,406] jepsen worker 795 - jepsen.util 795 :ok :add 2736 INFO [2022-01-10 08:39:23,406] jepsen worker 270 - jepsen.util 270 :ok :add 3123 INFO [2022-01-10 08:39:23,406] jepsen worker 565 - jepsen.util 565 :ok :add 3771 INFO [2022-01-10 08:39:23,406] jepsen worker 432 - jepsen.util 432 :ok :add 2813 INFO [2022-01-10 08:39:23,407] jepsen worker 843 - jepsen.util 843 :invoke :add 3777 INFO [2022-01-10 08:39:23,407] jepsen worker 138 - jepsen.util 138 :ok :add 3770 INFO [2022-01-10 08:39:23,407] jepsen worker 131 - jepsen.util 131 :ok :add 3768 INFO [2022-01-10 08:39:23,407] jepsen worker 691 - jepsen.util 691 :ok :add 3774 INFO [2022-01-10 08:39:23,411] jepsen worker 169 - jepsen.util 169 :ok :add 3760 INFO [2022-01-10 08:39:23,413] jepsen worker 30 - jepsen.util 30 :ok :add 3773 INFO [2022-01-10 08:39:23,413] jepsen worker 715 - jepsen.util 715 :ok :add 3772 INFO [2022-01-10 08:39:23,413] jepsen worker 847 - jepsen.util 847 :ok :add 3776 INFO [2022-01-10 08:39:23,414] jepsen worker 944 - jepsen.util 944 :invoke :add 3778 INFO [2022-01-10 08:39:23,414] jepsen worker 544 - jepsen.util 544 :ok :add 3775 INFO [2022-01-10 08:39:23,417] jepsen worker 120 - jepsen.util 120 :ok :add 3762 INFO [2022-01-10 08:39:23,417] jepsen worker 333 - jepsen.util 333 :invoke :add 3779 INFO [2022-01-10 08:39:23,420] jepsen worker 969 - jepsen.util 969 :invoke :add 3780 INFO [2022-01-10 08:39:23,422] jepsen worker 910 - jepsen.util 910 :invoke :add 3781 INFO [2022-01-10 08:39:23,424] jepsen worker 843 - jepsen.util 843 :ok :add 3777 INFO [2022-01-10 08:39:23,425] jepsen worker 603 - jepsen.util 603 :invoke :add 3782 INFO [2022-01-10 08:39:23,427] jepsen worker 899 - jepsen.util 899 :invoke :add 3783 INFO [2022-01-10 08:39:23,429] jepsen worker 719 - jepsen.util 719 :invoke :add 3784 INFO [2022-01-10 08:39:23,431] jepsen worker 333 - jepsen.util 333 :ok :add 3779 INFO [2022-01-10 08:39:23,432] jepsen worker 772 - jepsen.util 772 :invoke :add 3785 INFO [2022-01-10 08:39:23,435] jepsen worker 944 - jepsen.util 944 :ok :add 3778 INFO [2022-01-10 08:39:23,435] jepsen worker 726 - jepsen.util 726 :invoke :add 3786 INFO [2022-01-10 08:39:23,436] jepsen worker 969 - jepsen.util 969 :ok :add 3780 INFO [2022-01-10 08:39:23,438] jepsen worker 325 - jepsen.util 325 :invoke :add 3787 INFO [2022-01-10 08:39:23,441] jepsen worker 603 - jepsen.util 603 :ok :add 3782 INFO [2022-01-10 08:39:23,441] jepsen worker 321 - jepsen.util 321 :invoke :add 3788 INFO [2022-01-10 08:39:23,443] jepsen worker 910 - jepsen.util 910 :ok :add 3781 INFO [2022-01-10 08:39:23,444] jepsen worker 800 - jepsen.util 800 :invoke :add 3789 INFO [2022-01-10 08:39:23,447] jepsen worker 899 - jepsen.util 899 :ok :add 3783 INFO [2022-01-10 08:39:23,448] jepsen worker 957 - jepsen.util 957 :invoke :add 3790 INFO [2022-01-10 08:39:23,451] jepsen worker 132 - jepsen.util 132 :invoke :add 3791 INFO [2022-01-10 08:39:23,453] jepsen worker 800 - jepsen.util 800 :ok :add 3789 INFO [2022-01-10 08:39:23,456] jepsen worker 241 - jepsen.util 241 :invoke :add 3792 INFO [2022-01-10 08:39:23,458] jepsen worker 776 - jepsen.util 776 :invoke :add 3793 INFO [2022-01-10 08:39:23,460] jepsen worker 497 - jepsen.util 497 :invoke :add 3794 INFO [2022-01-10 08:39:23,460] jepsen worker 321 - jepsen.util 321 :ok :add 3788 INFO [2022-01-10 08:39:23,461] jepsen worker 957 - jepsen.util 957 :ok :add 3790 INFO [2022-01-10 08:39:23,462] jepsen worker 132 - jepsen.util 132 :ok :add 3791 INFO [2022-01-10 08:39:23,462] jepsen worker 468 - jepsen.util 468 :invoke :add 3795 INFO [2022-01-10 08:39:23,466] jepsen worker 241 - jepsen.util 241 :ok :add 3792 INFO [2022-01-10 08:39:23,466] jepsen worker 298 - jepsen.util 298 :invoke :add 3796 INFO [2022-01-10 08:39:23,468] jepsen worker 776 - jepsen.util 776 :ok :add 3793 INFO [2022-01-10 08:39:23,468] jepsen worker 820 - jepsen.util 820 :invoke :add 3797 INFO [2022-01-10 08:39:23,470] jepsen worker 136 - jepsen.util 136 :invoke :add 3798 INFO [2022-01-10 08:39:23,471] jepsen worker 497 - jepsen.util 497 :ok :add 3794 INFO [2022-01-10 08:39:23,473] jepsen worker 468 - jepsen.util 468 :ok :add 3795 INFO [2022-01-10 08:39:23,473] jepsen worker 23 - jepsen.util 23 :invoke :add 3799 INFO [2022-01-10 08:39:23,478] jepsen worker 820 - jepsen.util 820 :ok :add 3797 INFO [2022-01-10 08:39:23,478] jepsen worker 298 - jepsen.util 298 :ok :add 3796 INFO [2022-01-10 08:39:23,481] jepsen worker 407 - jepsen.util 407 :invoke :add 3800 INFO [2022-01-10 08:39:23,484] jepsen worker 339 - jepsen.util 339 :invoke :add 3801 INFO [2022-01-10 08:39:23,486] jepsen worker 356 - jepsen.util 356 :invoke :add 3802 INFO [2022-01-10 08:39:23,489] jepsen worker 348 - jepsen.util 348 :invoke :add 3803 INFO [2022-01-10 08:39:23,492] jepsen worker 947 - jepsen.util 947 :invoke :add 3804 INFO [2022-01-10 08:39:23,494] jepsen worker 302 - jepsen.util 302 :invoke :add 3805 INFO [2022-01-10 08:39:23,496] jepsen worker 824 - jepsen.util 824 :invoke :add 3806 INFO [2022-01-10 08:39:23,499] jepsen worker 312 - jepsen.util 312 :invoke :add 3807 INFO [2022-01-10 08:39:23,502] jepsen worker 924 - jepsen.util 924 :ok :add 2773 INFO [2022-01-10 08:39:23,502] jepsen worker 528 - jepsen.util 528 :ok :add 3551 INFO [2022-01-10 08:39:23,502] jepsen worker 629 - jepsen.util 629 :ok :add 3217 INFO [2022-01-10 08:39:23,502] jepsen worker 350 - jepsen.util 350 :ok :add 3638 INFO [2022-01-10 08:39:23,502] jepsen worker 245 - jepsen.util 245 :ok :add 2856 INFO [2022-01-10 08:39:23,502] jepsen worker 3 - jepsen.util 3 :ok :add 3549 INFO [2022-01-10 08:39:23,502] jepsen worker 254 - jepsen.util 254 :invoke :add 3808 INFO [2022-01-10 08:39:23,502] jepsen worker 937 - jepsen.util 937 :ok :add 3223 INFO [2022-01-10 08:39:23,502] jepsen worker 107 - jepsen.util 107 :ok :add 2862 INFO [2022-01-10 08:39:23,502] jepsen worker 343 - jepsen.util 343 :ok :add 3614 INFO [2022-01-10 08:39:23,502] jepsen worker 163 - jepsen.util 163 :ok :add 3166 INFO [2022-01-10 08:39:23,503] jepsen worker 123 - jepsen.util 123 :ok :add 3152 INFO [2022-01-10 08:39:23,503] jepsen worker 193 - jepsen.util 193 :ok :add 3565 INFO [2022-01-10 08:39:23,503] jepsen worker 351 - jepsen.util 351 :ok :add 3620 INFO [2022-01-10 08:39:23,503] jepsen worker 940 - jepsen.util 940 :ok :add 3387 INFO [2022-01-10 08:39:23,503] jepsen worker 615 - jepsen.util 615 :ok :add 2747 INFO [2022-01-10 08:39:23,508] jepsen worker 858 - jepsen.util 858 :invoke :add 3809 INFO [2022-01-10 08:39:23,509] jepsen worker 254 - jepsen.util 254 :ok :add 3808 INFO [2022-01-10 08:39:23,511] jepsen worker 295 - jepsen.util 295 :invoke :add 3810 INFO [2022-01-10 08:39:23,514] jepsen worker 241 - jepsen.util 241 :invoke :add 3811 INFO [2022-01-10 08:39:23,515] jepsen worker 858 - jepsen.util 858 :ok :add 3809 INFO [2022-01-10 08:39:23,517] jepsen worker 54 - jepsen.util 54 :invoke :add 3812 INFO [2022-01-10 08:39:23,518] jepsen worker 295 - jepsen.util 295 :ok :add 3810 INFO [2022-01-10 08:39:23,520] jepsen worker 874 - jepsen.util 874 :invoke :add 3813 INFO [2022-01-10 08:39:23,520] jepsen worker 241 - jepsen.util 241 :ok :add 3811 INFO [2022-01-10 08:39:23,521] jepsen worker 23 - jepsen.util 23 :ok :add 3799 INFO [2022-01-10 08:39:23,523] jepsen worker 902 - jepsen.util 902 :invoke :add 3814 INFO [2022-01-10 08:39:23,524] jepsen worker 54 - jepsen.util 54 :ok :add 3812 INFO [2022-01-10 08:39:23,526] jepsen worker 998 - jepsen.util 998 :invoke :add 3815 INFO [2022-01-10 08:39:23,526] jepsen worker 136 - jepsen.util 136 :ok :add 3798 INFO [2022-01-10 08:39:23,528] jepsen worker 407 - jepsen.util 407 :ok :add 3800 INFO [2022-01-10 08:39:23,528] jepsen worker 347 - jepsen.util 347 :invoke :add 3816 INFO [2022-01-10 08:39:23,529] jepsen worker 902 - jepsen.util 902 :ok :add 3814 INFO [2022-01-10 08:39:23,532] jepsen worker 874 - jepsen.util 874 :ok :add 3813 INFO [2022-01-10 08:39:23,532] jepsen worker 915 - jepsen.util 915 :invoke :add 3817 INFO [2022-01-10 08:39:23,534] jepsen worker 157 - jepsen.util 157 :ok :add 2810 INFO [2022-01-10 08:39:23,534] jepsen worker 577 - jepsen.util 577 :ok :add 2824 INFO [2022-01-10 08:39:23,534] jepsen worker 345 - jepsen.util 345 :ok :add 3487 INFO [2022-01-10 08:39:23,534] jepsen worker 158 - jepsen.util 158 :ok :add 3265 INFO [2022-01-10 08:39:23,534] jepsen worker 127 - jepsen.util 127 :ok :add 3138 INFO [2022-01-10 08:39:23,534] jepsen worker 442 - jepsen.util 442 :ok :add 3523 INFO [2022-01-10 08:39:23,534] jepsen worker 913 - jepsen.util 913 :ok :add 3114 INFO [2022-01-10 08:39:23,534] jepsen worker 63 - jepsen.util 63 :ok :add 3652 INFO [2022-01-10 08:39:23,534] jepsen worker 396 - jepsen.util 396 :ok :add 3084 INFO [2022-01-10 08:39:23,534] jepsen worker 744 - jepsen.util 744 :ok :add 3517 INFO [2022-01-10 08:39:23,534] jepsen worker 331 - jepsen.util 331 :ok :add 3666 INFO [2022-01-10 08:39:23,535] jepsen worker 339 - jepsen.util 339 :ok :add 3801 INFO [2022-01-10 08:39:23,535] jepsen worker 356 - jepsen.util 356 :ok :add 3802 INFO [2022-01-10 08:39:23,535] jepsen worker 45 - jepsen.util 45 :ok :add 2980 INFO [2022-01-10 08:39:23,535] jepsen worker 797 - jepsen.util 797 :ok :add 3249 INFO [2022-01-10 08:39:23,535] jepsen worker 810 - jepsen.util 810 :ok :add 3251 INFO [2022-01-10 08:39:23,536] jepsen worker 255 - jepsen.util 255 :invoke :add 3818 INFO [2022-01-10 08:39:23,541] jepsen worker 348 - jepsen.util 348 :ok :add 3803 INFO [2022-01-10 08:39:23,541] jepsen worker 10 - jepsen.util 10 :ok :add 2927 INFO [2022-01-10 08:39:23,541] jepsen worker 705 - jepsen.util 705 :ok :add 3023 INFO [2022-01-10 08:39:23,541] jepsen worker 142 - jepsen.util 142 :ok :add 3352 INFO [2022-01-10 08:39:23,541] jepsen worker 514 - jepsen.util 514 :ok :add 2913 INFO [2022-01-10 08:39:23,542] jepsen worker 418 - jepsen.util 418 :ok :add 3414 INFO [2022-01-10 08:39:23,542] jepsen worker 303 - jepsen.util 303 :ok :add 3584 INFO [2022-01-10 08:39:23,542] jepsen worker 431 - jepsen.util 431 :ok :add 3749 INFO [2022-01-10 08:39:23,542] jepsen worker 922 - jepsen.util 922 :ok :add 2792 INFO [2022-01-10 08:39:23,543] jepsen worker 820 - jepsen.util 820 :invoke :add 3819 INFO [2022-01-10 08:39:23,546] jepsen worker 728 - jepsen.util 728 :ok :add 3420 INFO [2022-01-10 08:39:23,547] jepsen worker 598 - jepsen.util 598 :ok :add 3017 INFO [2022-01-10 08:39:23,547] jepsen worker 724 - jepsen.util 724 :ok :add 3765 INFO [2022-01-10 08:39:23,547] jepsen worker 920 - jepsen.util 920 :ok :add 3751 INFO [2022-01-10 08:39:23,547] jepsen worker 690 - jepsen.util 690 :ok :add 3366 INFO [2022-01-10 08:39:23,547] jepsen worker 846 - jepsen.util 846 :ok :add 3438 INFO [2022-01-10 08:39:23,548] jepsen worker 840 - jepsen.util 840 :invoke :add 3820 INFO [2022-01-10 08:39:23,549] jepsen worker 255 - jepsen.util 255 :ok :add 3818 INFO [2022-01-10 08:39:23,552] jepsen worker 354 - jepsen.util 354 :invoke :add 3821 INFO [2022-01-10 08:39:23,555] jepsen worker 855 - jepsen.util 855 :invoke :add 3822 INFO [2022-01-10 08:39:23,555] jepsen worker 820 - jepsen.util 820 :ok :add 3819 INFO [2022-01-10 08:39:23,558] jepsen worker 844 - jepsen.util 844 :invoke :add 3823 INFO [2022-01-10 08:39:23,561] jepsen worker 920 - jepsen.util 920 :invoke :add 3824 INFO [2022-01-10 08:39:23,565] jepsen worker 749 - jepsen.util 749 :invoke :add 3825 INFO [2022-01-10 08:39:23,565] jepsen worker 998 - jepsen.util 998 :ok :add 3815 INFO [2022-01-10 08:39:23,568] jepsen worker 384 - jepsen.util 384 :invoke :add 3826 INFO [2022-01-10 08:39:23,569] jepsen worker 920 - jepsen.util 920 :ok :add 3824 INFO [2022-01-10 08:39:23,569] jepsen worker 855 - jepsen.util 855 :ok :add 3822 INFO [2022-01-10 08:39:23,570] jepsen worker 347 - jepsen.util 347 :ok :add 3816 INFO [2022-01-10 08:39:23,572] jepsen worker 152 - jepsen.util 152 :invoke :add 3827 INFO [2022-01-10 08:39:23,573] jepsen worker 844 - jepsen.util 844 :ok :add 3823 INFO [2022-01-10 08:39:23,573] jepsen worker 915 - jepsen.util 915 :ok :add 3817 INFO [2022-01-10 08:39:23,576] jepsen worker 989 - jepsen.util 989 :invoke :add 3828 INFO [2022-01-10 08:39:23,577] jepsen worker 354 - jepsen.util 354 :ok :add 3821 INFO [2022-01-10 08:39:23,577] jepsen worker 749 - jepsen.util 749 :ok :add 3825 INFO [2022-01-10 08:39:23,577] jepsen worker 384 - jepsen.util 384 :ok :add 3826 INFO [2022-01-10 08:39:23,581] jepsen worker 288 - jepsen.util 288 :invoke :add 3829 INFO [2022-01-10 08:39:23,585] jepsen worker 189 - jepsen.util 189 :invoke :add 3830 INFO [2022-01-10 08:39:23,587] jepsen worker 840 - jepsen.util 840 :ok :add 3820 INFO [2022-01-10 08:39:23,587] jepsen worker 475 - jepsen.util 475 :invoke :add 3831 INFO [2022-01-10 08:39:23,590] jepsen worker 274 - jepsen.util 274 :invoke :add 3832 INFO [2022-01-10 08:39:23,592] jepsen worker 60 - jepsen.util 60 :invoke :add 3833 INFO [2022-01-10 08:39:23,595] jepsen worker 877 - jepsen.util 877 :invoke :add 3834 INFO [2022-01-10 08:39:23,598] jepsen worker 7 - jepsen.util 7 :invoke :add 3835 INFO [2022-01-10 08:39:23,601] jepsen worker 820 - jepsen.util 820 :invoke :add 3836 INFO [2022-01-10 08:39:23,602] jepsen worker 877 - jepsen.util 877 :ok :add 3834 INFO [2022-01-10 08:39:23,605] jepsen worker 746 - jepsen.util 746 :invoke :add 3837 INFO [2022-01-10 08:39:23,606] jepsen worker 7 - jepsen.util 7 :ok :add 3835 INFO [2022-01-10 08:39:23,608] jepsen worker 820 - jepsen.util 820 :ok :add 3836 INFO [2022-01-10 08:39:23,609] jepsen worker 455 - jepsen.util 455 :invoke :add 3838 INFO [2022-01-10 08:39:23,613] jepsen worker 913 - jepsen.util 913 :invoke :add 3839 INFO [2022-01-10 08:39:23,615] jepsen worker 152 - jepsen.util 152 :ok :add 3827 INFO [2022-01-10 08:39:23,617] jepsen worker 101 - jepsen.util 101 :invoke :add 3840 INFO [2022-01-10 08:39:23,617] jepsen worker 746 - jepsen.util 746 :ok :add 3837 INFO [2022-01-10 08:39:23,618] jepsen worker 455 - jepsen.util 455 :ok :add 3838 INFO [2022-01-10 08:39:23,620] jepsen worker 913 - jepsen.util 913 :ok :add 3839 INFO [2022-01-10 08:39:23,621] jepsen worker 179 - jepsen.util 179 :invoke :add 3841 INFO [2022-01-10 08:39:23,623] jepsen worker 702 - jepsen.util 702 :invoke :add 3842 INFO [2022-01-10 08:39:23,626] jepsen worker 363 - jepsen.util 363 :ok :add 3253 INFO [2022-01-10 08:39:23,626] jepsen worker 341 - jepsen.util 341 :ok :add 3267 INFO [2022-01-10 08:39:23,626] jepsen worker 776 - jepsen.util 776 :invoke :add 3843 INFO [2022-01-10 08:39:23,626] jepsen worker 290 - jepsen.util 290 :ok :add 3122 INFO [2022-01-10 08:39:23,626] jepsen worker 781 - jepsen.util 781 :ok :add 3664 INFO [2022-01-10 08:39:23,627] jepsen worker 458 - jepsen.util 458 :ok :add 3539 INFO [2022-01-10 08:39:23,627] jepsen worker 630 - jepsen.util 630 :ok :add 3116 INFO [2022-01-10 08:39:23,627] jepsen worker 833 - jepsen.util 833 :ok :add 2982 INFO [2022-01-10 08:39:23,627] jepsen worker 391 - jepsen.util 391 :ok :add 3485 INFO [2022-01-10 08:39:23,627] jepsen worker 373 - jepsen.util 373 :ok :add 3648 INFO [2022-01-10 08:39:23,628] jepsen worker 656 - jepsen.util 656 :ok :add 3650 INFO [2022-01-10 08:39:23,628] jepsen worker 393 - jepsen.util 393 :ok :add 2826 INFO [2022-01-10 08:39:23,628] jepsen worker 508 - jepsen.util 508 :ok :add 3515 INFO [2022-01-10 08:39:23,628] jepsen worker 625 - jepsen.util 625 :ok :add 3086 INFO [2022-01-10 08:39:23,628] jepsen worker 811 - jepsen.util 811 :ok :add 3521 INFO [2022-01-10 08:39:23,630] jepsen worker 317 - jepsen.util 317 :invoke :add 3844 INFO [2022-01-10 08:39:23,634] jepsen worker 204 - jepsen.util 204 :invoke :add 3845 INFO [2022-01-10 08:39:23,637] jepsen worker 80 - jepsen.util 80 :invoke :add 3846 INFO [2022-01-10 08:39:23,640] jepsen worker 90 - jepsen.util 90 :invoke :add 3847 INFO [2022-01-10 08:39:23,643] jepsen worker 36 - jepsen.util 36 :invoke :add 3848 INFO [2022-01-10 08:39:23,645] jepsen worker 346 - jepsen.util 346 :invoke :add 3849 INFO [2022-01-10 08:39:23,648] jepsen worker 358 - jepsen.util 358 :invoke :add 3850 INFO [2022-01-10 08:39:23,651] jepsen worker 594 - jepsen.util 594 :invoke :add 3851 INFO [2022-01-10 08:39:23,652] jepsen worker 36 - jepsen.util 36 :ok :add 3848 INFO [2022-01-10 08:39:23,653] jepsen worker 156 - jepsen.util 156 :invoke :add 3852 INFO [2022-01-10 08:39:23,655] jepsen worker 228 - jepsen.util 228 :ok :add 3567 INFO [2022-01-10 08:39:23,655] jepsen worker 726 - jepsen.util 726 :ok :add 3786 INFO [2022-01-10 08:39:23,655] jepsen worker 970 - jepsen.util 970 :ok :add 3616 INFO [2022-01-10 08:39:23,655] jepsen worker 974 - jepsen.util 974 :ok :add 3553 INFO [2022-01-10 08:39:23,656] jepsen worker 671 - jepsen.util 671 :ok :add 3622 INFO [2022-01-10 08:39:23,656] jepsen worker 647 - jepsen.util 647 :ok :add 3385 INFO [2022-01-10 08:39:23,656] jepsen worker 44 - jepsen.util 44 :invoke :add 3853 INFO [2022-01-10 08:39:23,658] jepsen worker 346 - jepsen.util 346 :ok :add 3849 INFO [2022-01-10 08:39:23,659] jepsen worker 358 - jepsen.util 358 :ok :add 3850 INFO [2022-01-10 08:39:23,660] jepsen worker 245 - jepsen.util 245 :invoke :add 3854 INFO [2022-01-10 08:39:23,663] jepsen worker 204 - jepsen.util 204 :ok :add 3845 INFO [2022-01-10 08:39:23,663] jepsen worker 317 - jepsen.util 317 :ok :add 3844 INFO [2022-01-10 08:39:23,664] jepsen worker 939 - jepsen.util 939 :invoke :add 3855 INFO [2022-01-10 08:39:23,664] jepsen worker 594 - jepsen.util 594 :ok :add 3851 INFO [2022-01-10 08:39:23,666] jepsen worker 156 - jepsen.util 156 :ok :add 3852 INFO [2022-01-10 08:39:23,666] jepsen worker 44 - jepsen.util 44 :ok :add 3853 INFO [2022-01-10 08:39:23,667] jepsen worker 672 - jepsen.util 672 :invoke :add 3856 INFO [2022-01-10 08:39:23,668] jepsen worker 245 - jepsen.util 245 :ok :add 3854 INFO [2022-01-10 08:39:23,669] jepsen worker 90 - jepsen.util 90 :ok :add 3847 INFO [2022-01-10 08:39:23,672] jepsen worker 80 - jepsen.util 80 :ok :add 3846 INFO [2022-01-10 08:39:23,672] jepsen worker 427 - jepsen.util 427 :invoke :add 3857 INFO [2022-01-10 08:39:23,675] jepsen worker 2 - jepsen.util 2 :invoke :add 3858 INFO [2022-01-10 08:39:23,677] jepsen worker 796 - jepsen.util 796 :invoke :add 3859 INFO [2022-01-10 08:39:23,677] jepsen worker 939 - jepsen.util 939 :ok :add 3855 INFO [2022-01-10 08:39:23,678] jepsen worker 672 - jepsen.util 672 :ok :add 3856 INFO [2022-01-10 08:39:23,680] jepsen worker 106 - jepsen.util 106 :invoke :add 3860 INFO [2022-01-10 08:39:23,682] jepsen worker 77 - jepsen.util 77 :invoke :add 3861 INFO [2022-01-10 08:39:23,684] jepsen worker 427 - jepsen.util 427 :ok :add 3857 INFO [2022-01-10 08:39:23,684] jepsen worker 6 - jepsen.util 6 :invoke :add 3862 INFO [2022-01-10 08:39:23,687] jepsen worker 576 - jepsen.util 576 :invoke :add 3863 INFO [2022-01-10 08:39:23,689] jepsen worker 199 - jepsen.util 199 :ok :add 3550 INFO [2022-01-10 08:39:23,689] jepsen worker 419 - jepsen.util 419 :ok :add 3621 INFO [2022-01-10 08:39:23,689] jepsen worker 77 - jepsen.util 77 :ok :add 3861 INFO [2022-01-10 08:39:23,689] jepsen worker 468 - jepsen.util 468 :invoke :add 3864 INFO [2022-01-10 08:39:23,691] jepsen worker 428 - jepsen.util 428 :ok :add 3639 INFO [2022-01-10 08:39:23,691] jepsen worker 622 - jepsen.util 622 :ok :add 3167 INFO [2022-01-10 08:39:23,692] jepsen worker 259 - jepsen.util 259 :ok :add 3615 INFO [2022-01-10 08:39:23,692] jepsen worker 876 - jepsen.util 876 :ok :add 3216 INFO [2022-01-10 08:39:23,691] jepsen worker 13 - jepsen.util 13 :ok :add 3564 INFO [2022-01-10 08:39:23,691] jepsen worker 772 - jepsen.util 772 :ok :add 3785 INFO [2022-01-10 08:39:23,692] jepsen worker 106 - jepsen.util 106 :ok :add 3860 INFO [2022-01-10 08:39:23,693] jepsen worker 766 - jepsen.util 766 :ok :add 3386 INFO [2022-01-10 08:39:23,693] jepsen worker 486 - jepsen.util 486 :ok :add 3548 INFO [2022-01-10 08:39:23,695] jepsen worker 449 - jepsen.util 449 :invoke :add 3865 INFO [2022-01-10 08:39:23,698] jepsen worker 6 - jepsen.util 6 :ok :add 3862 INFO [2022-01-10 08:39:23,698] jepsen worker 576 - jepsen.util 576 :ok :add 3863 INFO [2022-01-10 08:39:23,701] jepsen worker 134 - jepsen.util 134 :invoke :add 3866 INFO [2022-01-10 08:39:23,702] jepsen worker 468 - jepsen.util 468 :ok :add 3864 INFO [2022-01-10 08:39:23,702] jepsen worker 449 - jepsen.util 449 :ok :add 3865 INFO [2022-01-10 08:39:23,704] jepsen worker 165 - jepsen.util 165 :invoke :add 3867 INFO [2022-01-10 08:39:23,708] jepsen worker 144 - jepsen.util 144 :invoke :add 3868 INFO [2022-01-10 08:39:23,711] jepsen worker 18 - jepsen.util 18 :invoke :add 3869 INFO [2022-01-10 08:39:23,711] jepsen worker 165 - jepsen.util 165 :ok :add 3867 INFO [2022-01-10 08:39:23,713] jepsen worker 306 - jepsen.util 306 :invoke :add 3870 INFO [2022-01-10 08:39:23,716] jepsen worker 81 - jepsen.util 81 :ok :add 3767 INFO [2022-01-10 08:39:23,716] jepsen worker 308 - jepsen.util 308 :ok :add 3586 INFO [2022-01-10 08:39:23,717] jepsen worker 472 - jepsen.util 472 :invoke :add 3871 INFO [2022-01-10 08:39:23,717] jepsen worker 702 - jepsen.util 702 :ok :add 3842 INFO [2022-01-10 08:39:23,717] jepsen worker 52 - jepsen.util 52 :ok :add 3753 INFO [2022-01-10 08:39:23,717] jepsen worker 69 - jepsen.util 69 :ok :add 3364 INFO [2022-01-10 08:39:23,717] jepsen worker 134 - jepsen.util 134 :ok :add 3866 INFO [2022-01-10 08:39:23,720] jepsen worker 144 - jepsen.util 144 :ok :add 3868 INFO [2022-01-10 08:39:23,721] jepsen worker 258 - jepsen.util 258 :invoke :add 3872 INFO [2022-01-10 08:39:23,724] jepsen worker 87 - jepsen.util 87 :invoke :add 3873 INFO [2022-01-10 08:39:23,727] jepsen worker 584 - jepsen.util 584 :invoke :add 3874 INFO [2022-01-10 08:39:23,730] jepsen worker 255 - jepsen.util 255 :invoke :add 3875 INFO [2022-01-10 08:39:23,733] jepsen worker 645 - jepsen.util 645 :invoke :add 3876 INFO [2022-01-10 08:39:23,736] jepsen worker 904 - jepsen.util 904 :invoke :add 3877 INFO [2022-01-10 08:39:23,736] jepsen worker 18 - jepsen.util 18 :ok :add 3869 INFO [2022-01-10 08:39:23,737] jepsen worker 306 - jepsen.util 306 :ok :add 3870 INFO [2022-01-10 08:39:23,738] jepsen worker 952 - jepsen.util 952 :invoke :add 3878 INFO [2022-01-10 08:39:23,741] jepsen worker 499 - jepsen.util 499 :invoke :add 3879 INFO [2022-01-10 08:39:23,741] jepsen worker 258 - jepsen.util 258 :ok :add 3872 INFO [2022-01-10 08:39:23,742] jepsen worker 472 - jepsen.util 472 :ok :add 3871 INFO [2022-01-10 08:39:23,744] jepsen worker 535 - jepsen.util 535 :invoke :add 3880 INFO [2022-01-10 08:39:23,746] jepsen worker 934 - jepsen.util 934 :invoke :add 3881 INFO [2022-01-10 08:39:23,749] jepsen worker 338 - jepsen.util 338 :invoke :add 3882 INFO [2022-01-10 08:39:23,749] jepsen worker 952 - jepsen.util 952 :ok :add 3878 INFO [2022-01-10 08:39:23,753] jepsen worker 421 - jepsen.util 421 :invoke :add 3883 INFO [2022-01-10 08:39:23,753] jepsen worker 87 - jepsen.util 87 :ok :add 3873 INFO [2022-01-10 08:39:23,755] jepsen worker 65 - jepsen.util 65 :invoke :add 3884 INFO [2022-01-10 08:39:23,758] jepsen worker 930 - jepsen.util 930 :invoke :add 3885 INFO [2022-01-10 08:39:23,758] jepsen worker 499 - jepsen.util 499 :ok :add 3879 INFO [2022-01-10 08:39:23,760] jepsen worker 793 - jepsen.util 793 :invoke :add 3886 INFO [2022-01-10 08:39:23,763] jepsen worker 251 - jepsen.util 251 :invoke :add 3887 INFO [2022-01-10 08:39:23,764] jepsen worker 934 - jepsen.util 934 :ok :add 3881 INFO [2022-01-10 08:39:23,765] jepsen worker 594 - jepsen.util 594 :invoke :add 3888 INFO [2022-01-10 08:39:23,766] jepsen worker 535 - jepsen.util 535 :ok :add 3880 INFO [2022-01-10 08:39:23,767] jepsen worker 978 - jepsen.util 978 :invoke :add 3889 INFO [2022-01-10 08:39:23,768] jepsen worker 65 - jepsen.util 65 :ok :add 3884 INFO [2022-01-10 08:39:23,769] jepsen worker 930 - jepsen.util 930 :ok :add 3885 INFO [2022-01-10 08:39:23,770] jepsen worker 557 - jepsen.util 557 :invoke :add 3890 INFO [2022-01-10 08:39:23,770] jepsen worker 421 - jepsen.util 421 :ok :add 3883 INFO [2022-01-10 08:39:23,772] jepsen worker 308 - jepsen.util 308 :invoke :add 3891 INFO [2022-01-10 08:39:23,772] jepsen worker 793 - jepsen.util 793 :ok :add 3886 INFO [2022-01-10 08:39:23,772] jepsen worker 338 - jepsen.util 338 :ok :add 3882 INFO [2022-01-10 08:39:23,775] jepsen worker 251 - jepsen.util 251 :ok :add 3887 INFO [2022-01-10 08:39:23,777] jepsen worker 594 - jepsen.util 594 :ok :add 3888 INFO [2022-01-10 08:39:23,777] jepsen worker 129 - jepsen.util 129 :invoke :add 3892 INFO [2022-01-10 08:39:23,780] jepsen worker 492 - jepsen.util 492 :invoke :add 3893 INFO [2022-01-10 08:39:23,782] jepsen worker 978 - jepsen.util 978 :ok :add 3889 INFO [2022-01-10 08:39:23,784] jepsen worker 946 - jepsen.util 946 :invoke :add 3894 INFO [2022-01-10 08:39:23,788] jepsen worker 725 - jepsen.util 725 :invoke :add 3895 INFO [2022-01-10 08:39:23,788] jepsen worker 557 - jepsen.util 557 :ok :add 3890 INFO [2022-01-10 08:39:23,789] jepsen worker 312 - jepsen.util 312 :ok :add 3807 INFO [2022-01-10 08:39:23,789] jepsen worker 507 - jepsen.util 507 :ok :add 3686 INFO [2022-01-10 08:39:23,789] jepsen worker 109 - jepsen.util 109 :ok :add 3716 INFO [2022-01-10 08:39:23,789] jepsen worker 196 - jepsen.util 196 :ok :add 3467 INFO [2022-01-10 08:39:23,789] jepsen worker 60 - jepsen.util 60 :ok :add 3833 INFO [2022-01-10 08:39:23,789] jepsen worker 435 - jepsen.util 435 :ok :add 3722 INFO [2022-01-10 08:39:23,791] jepsen worker 483 - jepsen.util 483 :invoke :add 3896 INFO [2022-01-10 08:39:23,792] jepsen worker 129 - jepsen.util 129 :ok :add 3892 INFO [2022-01-10 08:39:23,794] jepsen worker 308 - jepsen.util 308 :ok :add 3891 INFO [2022-01-10 08:39:23,796] jepsen worker 916 - jepsen.util 916 :invoke :add 3897 INFO [2022-01-10 08:39:23,798] jepsen worker 914 - jepsen.util 914 :invoke :add 3898 INFO [2022-01-10 08:39:23,798] jepsen worker 492 - jepsen.util 492 :ok :add 3893 INFO [2022-01-10 08:39:23,800] jepsen worker 451 - jepsen.util 451 :ok :add 3766 INFO [2022-01-10 08:39:23,800] jepsen worker 498 - jepsen.util 498 :ok :add 3752 INFO [2022-01-10 08:39:23,800] jepsen worker 904 - jepsen.util 904 :ok :add 3877 INFO [2022-01-10 08:39:23,801] jepsen worker 825 - jepsen.util 825 :ok :add 3587 INFO [2022-01-10 08:39:23,801] jepsen worker 718 - jepsen.util 718 :ok :add 3349 INFO [2022-01-10 08:39:23,801] jepsen worker 776 - jepsen.util 776 :ok :add 3843 INFO [2022-01-10 08:39:23,802] jepsen worker 843 - jepsen.util 843 :invoke :add 3899 INFO [2022-01-10 08:39:23,806] jepsen worker 914 - jepsen.util 914 :ok :add 3898 INFO [2022-01-10 08:39:23,806] jepsen worker 991 - jepsen.util 991 :invoke :add 3900 INFO [2022-01-10 08:39:23,811] jepsen worker 593 - jepsen.util 593 :invoke :add 3901 INFO [2022-01-10 08:39:23,813] jepsen worker 843 - jepsen.util 843 :ok :add 3899 INFO [2022-01-10 08:39:23,814] jepsen worker 393 - jepsen.util 393 :invoke :add 3902 INFO [2022-01-10 08:39:23,817] jepsen worker 646 - jepsen.util 646 :invoke :add 3903 INFO [2022-01-10 08:39:23,818] jepsen worker 991 - jepsen.util 991 :ok :add 3900 INFO [2022-01-10 08:39:23,818] jepsen worker 593 - jepsen.util 593 :ok :add 3901 INFO [2022-01-10 08:39:23,822] jepsen worker 913 - jepsen.util 913 :invoke :add 3904 INFO [2022-01-10 08:39:23,824] jepsen worker 393 - jepsen.util 393 :ok :add 3902 INFO [2022-01-10 08:39:23,825] jepsen worker 78 - jepsen.util 78 :invoke :add 3905 INFO [2022-01-10 08:39:23,827] jepsen worker 646 - jepsen.util 646 :ok :add 3903 INFO [2022-01-10 08:39:23,828] jepsen worker 948 - jepsen.util 948 :invoke :add 3906 INFO [2022-01-10 08:39:23,831] jepsen worker 955 - jepsen.util 955 :invoke :add 3907 INFO [2022-01-10 08:39:23,833] jepsen worker 228 - jepsen.util 228 :invoke :add 3908 INFO [2022-01-10 08:39:23,836] jepsen worker 112 - jepsen.util 112 :invoke :add 3909 INFO [2022-01-10 08:39:23,838] jepsen worker 884 - jepsen.util 884 :invoke :add 3910 INFO [2022-01-10 08:39:23,840] jepsen worker 154 - jepsen.util 154 :invoke :add 3911 INFO [2022-01-10 08:39:23,840] jepsen worker 228 - jepsen.util 228 :ok :add 3908 INFO [2022-01-10 08:39:23,843] jepsen worker 875 - jepsen.util 875 :invoke :add 3912 INFO [2022-01-10 08:39:23,844] jepsen worker 112 - jepsen.util 112 :ok :add 3909 INFO [2022-01-10 08:39:23,846] jepsen worker 702 - jepsen.util 702 :invoke :add 3913 INFO [2022-01-10 08:39:23,847] jepsen worker 884 - jepsen.util 884 :ok :add 3910 INFO [2022-01-10 08:39:23,849] jepsen worker 999 - jepsen.util 999 :invoke :add 3914 INFO [2022-01-10 08:39:23,850] jepsen worker 154 - jepsen.util 154 :ok :add 3911 INFO [2022-01-10 08:39:23,851] jepsen worker 875 - jepsen.util 875 :ok :add 3912 INFO [2022-01-10 08:39:23,853] jepsen worker 733 - jepsen.util 733 :invoke :add 3915 INFO [2022-01-10 08:39:23,855] jepsen worker 576 - jepsen.util 576 :invoke :add 3916 INFO [2022-01-10 08:39:23,857] jepsen worker 702 - jepsen.util 702 :ok :add 3913 INFO [2022-01-10 08:39:23,859] jepsen worker 370 - jepsen.util 370 :invoke :add 3917 INFO [2022-01-10 08:39:23,862] jepsen worker 153 - jepsen.util 153 :invoke :add 3918 INFO [2022-01-10 08:39:23,864] jepsen worker 998 - jepsen.util 998 :invoke :add 3919 INFO [2022-01-10 08:39:23,867] jepsen worker 446 - jepsen.util 446 :invoke :add 3920 INFO [2022-01-10 08:39:23,868] jepsen worker 153 - jepsen.util 153 :ok :add 3918 INFO [2022-01-10 08:39:23,869] jepsen worker 726 - jepsen.util 726 :invoke :add 3921 INFO [2022-01-10 08:39:23,872] jepsen worker 998 - jepsen.util 998 :ok :add 3919 INFO [2022-01-10 08:39:23,872] jepsen worker 37 - jepsen.util 37 :invoke :add 3922 INFO [2022-01-10 08:39:23,875] jepsen worker 866 - jepsen.util 866 :invoke :add 3923 INFO [2022-01-10 08:39:23,878] jepsen worker 400 - jepsen.util 400 :invoke :add 3924 INFO [2022-01-10 08:39:23,880] jepsen worker 999 - jepsen.util 999 :ok :add 3914 INFO [2022-01-10 08:39:23,881] jepsen worker 733 - jepsen.util 733 :ok :add 3915 INFO [2022-01-10 08:39:23,882] jepsen worker 939 - jepsen.util 939 :invoke :add 3925 INFO [2022-01-10 08:39:23,885] jepsen worker 183 - jepsen.util 183 :invoke :add 3926 INFO [2022-01-10 08:39:23,887] jepsen worker 576 - jepsen.util 576 :ok :add 3916 INFO [2022-01-10 08:39:23,887] jepsen worker 370 - jepsen.util 370 :ok :add 3917 INFO [2022-01-10 08:39:23,887] jepsen worker 866 - jepsen.util 866 :ok :add 3923 INFO [2022-01-10 08:39:23,888] jepsen worker 37 - jepsen.util 37 :ok :add 3922 INFO [2022-01-10 08:39:23,888] jepsen worker 49 - jepsen.util 49 :invoke :add 3927 INFO [2022-01-10 08:39:23,888] jepsen worker 400 - jepsen.util 400 :ok :add 3924 INFO [2022-01-10 08:39:23,891] jepsen worker 939 - jepsen.util 939 :ok :add 3925 INFO [2022-01-10 08:39:23,893] jepsen worker 603 - jepsen.util 603 :invoke :add 3928 INFO [2022-01-10 08:39:23,893] jepsen worker 446 - jepsen.util 446 :ok :add 3920 INFO [2022-01-10 08:39:23,893] jepsen worker 726 - jepsen.util 726 :ok :add 3921 INFO [2022-01-10 08:39:23,897] jepsen worker 543 - jepsen.util 543 :invoke :add 3929 INFO [2022-01-10 08:39:23,900] jepsen worker 330 - jepsen.util 330 :invoke :add 3930 INFO [2022-01-10 08:39:23,902] jepsen worker 333 - jepsen.util 333 :invoke :add 3931 INFO [2022-01-10 08:39:23,905] jepsen worker 678 - jepsen.util 678 :invoke :add 3932 INFO [2022-01-10 08:39:23,907] jepsen worker 942 - jepsen.util 942 :invoke :add 3933 INFO [2022-01-10 08:39:23,911] jepsen worker 27 - jepsen.util 27 :invoke :add 3934 INFO [2022-01-10 08:39:23,913] jepsen worker 53 - jepsen.util 53 :invoke :add 3935 INFO [2022-01-10 08:39:23,916] jepsen worker 27 - jepsen.util 27 :ok :add 3934 INFO [2022-01-10 08:39:23,916] jepsen worker 134 - jepsen.util 134 :invoke :add 3936 INFO [2022-01-10 08:39:23,918] jepsen worker 183 - jepsen.util 183 :ok :add 3926 INFO [2022-01-10 08:39:23,919] jepsen worker 884 - jepsen.util 884 :invoke :add 3937 INFO [2022-01-10 08:39:23,922] jepsen worker 49 - jepsen.util 49 :ok :add 3927 INFO [2022-01-10 08:39:23,922] jepsen worker 317 - jepsen.util 317 :invoke :add 3938 INFO [2022-01-10 08:39:23,924] jepsen worker 53 - jepsen.util 53 :ok :add 3935 INFO [2022-01-10 08:39:23,925] jepsen worker 134 - jepsen.util 134 :ok :add 3936 INFO [2022-01-10 08:39:23,926] jepsen worker 294 - jepsen.util 294 :invoke :add 3939 INFO [2022-01-10 08:39:23,929] jepsen worker 491 - jepsen.util 491 :ok :add 3714 INFO [2022-01-10 08:39:23,929] jepsen worker 180 - jepsen.util 180 :ok :add 3720 INFO [2022-01-10 08:39:23,929] jepsen worker 302 - jepsen.util 302 :ok :add 3805 INFO [2022-01-10 08:39:23,929] jepsen worker 475 - jepsen.util 475 :ok :add 3831 INFO [2022-01-10 08:39:23,929] jepsen worker 611 - jepsen.util 611 :ok :add 3684 INFO [2022-01-10 08:39:23,930] jepsen worker 288 - jepsen.util 288 :ok :add 3829 INFO [2022-01-10 08:39:23,930] jepsen worker 900 - jepsen.util 900 :invoke :add 3940 INFO [2022-01-10 08:39:23,930] jepsen worker 485 - jepsen.util 485 :ok :add 3764 INFO [2022-01-10 08:39:23,931] jepsen worker 317 - jepsen.util 317 :ok :add 3938 INFO [2022-01-10 08:39:23,931] jepsen worker 460 - jepsen.util 460 :ok :add 3748 INFO [2022-01-10 08:39:23,931] jepsen worker 179 - jepsen.util 179 :ok :add 3841 INFO [2022-01-10 08:39:23,931] jepsen worker 64 - jepsen.util 64 :ok :add 3738 INFO [2022-01-10 08:39:23,931] jepsen worker 796 - jepsen.util 796 :ok :add 3859 INFO [2022-01-10 08:39:23,931] jepsen worker 294 - jepsen.util 294 :ok :add 3939 INFO [2022-01-10 08:39:23,932] jepsen worker 884 - jepsen.util 884 :ok :add 3937 INFO [2022-01-10 08:39:23,933] jepsen worker 845 - jepsen.util 845 :ok :add 3585 INFO [2022-01-10 08:39:23,933] jepsen worker 281 - jepsen.util 281 :ok :add 3750 INFO [2022-01-10 08:39:23,933] jepsen worker 255 - jepsen.util 255 :ok :add 3875 INFO [2022-01-10 08:39:23,937] jepsen worker 443 - jepsen.util 443 :invoke :add 3941 INFO [2022-01-10 08:39:23,939] jepsen worker 383 - jepsen.util 383 :invoke :add 3942 INFO [2022-01-10 08:39:23,941] jepsen worker 145 - jepsen.util 145 :invoke :add 3943 INFO [2022-01-10 08:39:23,944] jepsen worker 741 - jepsen.util 741 :invoke :add 3944 INFO [2022-01-10 08:39:23,946] jepsen worker 252 - jepsen.util 252 :invoke :add 3945 INFO [2022-01-10 08:39:23,948] jepsen worker 267 - jepsen.util 267 :invoke :add 3946 INFO [2022-01-10 08:39:23,950] jepsen worker 69 - jepsen.util 69 :invoke :add 3947 INFO [2022-01-10 08:39:23,953] jepsen worker 656 - jepsen.util 656 :invoke :add 3948 INFO [2022-01-10 08:39:23,956] jepsen worker 690 - jepsen.util 690 :invoke :add 3949 INFO [2022-01-10 08:39:23,958] jepsen worker 710 - jepsen.util 710 :invoke :add 3950 INFO [2022-01-10 08:39:23,959] jepsen worker 252 - jepsen.util 252 :ok :add 3945 INFO [2022-01-10 08:39:23,961] jepsen worker 656 - jepsen.util 656 :ok :add 3948 INFO [2022-01-10 08:39:23,961] jepsen worker 680 - jepsen.util 680 :invoke :add 3951 INFO [2022-01-10 08:39:23,963] jepsen worker 690 - jepsen.util 690 :ok :add 3949 INFO [2022-01-10 08:39:23,964] jepsen worker 706 - jepsen.util 706 :invoke :add 3952 INFO [2022-01-10 08:39:23,968] jepsen worker 783 - jepsen.util 783 :invoke :add 3953 INFO [2022-01-10 08:39:23,970] jepsen worker 811 - jepsen.util 811 :invoke :add 3954 INFO [2022-01-10 08:39:23,972] jepsen worker 680 - jepsen.util 680 :ok :add 3951 INFO [2022-01-10 08:39:23,972] jepsen worker 208 - jepsen.util 208 :invoke :add 3955 INFO [2022-01-10 08:39:23,974] jepsen worker 710 - jepsen.util 710 :ok :add 3950 INFO [2022-01-10 08:39:23,975] jepsen worker 686 - jepsen.util 686 :invoke :add 3956 INFO [2022-01-10 08:39:23,976] jepsen worker 783 - jepsen.util 783 :ok :add 3953 INFO [2022-01-10 08:39:23,978] jepsen worker 771 - jepsen.util 771 :invoke :add 3957 INFO [2022-01-10 08:39:23,978] jepsen worker 811 - jepsen.util 811 :ok :add 3954 INFO [2022-01-10 08:39:23,980] jepsen worker 706 - jepsen.util 706 :ok :add 3952 INFO [2022-01-10 08:39:23,981] jepsen worker 757 - jepsen.util 757 :invoke :add 3958 INFO [2022-01-10 08:39:23,984] jepsen worker 458 - jepsen.util 458 :invoke :add 3959 INFO [2022-01-10 08:39:23,985] jepsen worker 208 - jepsen.util 208 :ok :add 3955 INFO [2022-01-10 08:39:23,985] jepsen worker 686 - jepsen.util 686 :ok :add 3956 INFO [2022-01-10 08:39:23,987] jepsen worker 112 - jepsen.util 112 :invoke :add 3960 INFO [2022-01-10 08:39:23,989] jepsen worker 69 - jepsen.util 69 :ok :add 3947 INFO [2022-01-10 08:39:23,989] jepsen worker 218 - jepsen.util 218 :invoke :add 3961 INFO [2022-01-10 08:39:23,991] jepsen worker 771 - jepsen.util 771 :ok :add 3957 INFO [2022-01-10 08:39:23,992] jepsen worker 970 - jepsen.util 970 :invoke :add 3962 INFO [2022-01-10 08:39:23,995] jepsen worker 636 - jepsen.util 636 :invoke :add 3963 INFO [2022-01-10 08:39:23,997] jepsen worker 607 - jepsen.util 607 :invoke :add 3964 INFO [2022-01-10 08:39:23,999] jepsen worker 112 - jepsen.util 112 :ok :add 3960 INFO [2022-01-10 08:39:23,999] jepsen worker 218 - jepsen.util 218 :ok :add 3961 INFO [2022-01-10 08:39:23,999] jepsen worker 263 - jepsen.util 263 :invoke :add 3965 INFO [2022-01-10 08:39:24,001] jepsen worker 741 - jepsen.util 741 :ok :add 3944 INFO [2022-01-10 08:39:24,002] jepsen worker 218 - jepsen.util 218 :invoke :add 3966 INFO [2022-01-10 08:39:24,004] jepsen worker 636 - jepsen.util 636 :ok :add 3963 INFO [2022-01-10 08:39:24,004] jepsen worker 970 - jepsen.util 970 :ok :add 3962 INFO [2022-01-10 08:39:24,004] jepsen worker 417 - jepsen.util 417 :invoke :add 3967 INFO [2022-01-10 08:39:24,007] jepsen worker 882 - jepsen.util 882 :invoke :add 3968 INFO [2022-01-10 08:39:24,009] jepsen worker 948 - jepsen.util 948 :ok :add 3906 INFO [2022-01-10 08:39:24,009] jepsen worker 325 - jepsen.util 325 :ok :add 3787 INFO [2022-01-10 08:39:24,009] jepsen worker 678 - jepsen.util 678 :ok :add 3932 INFO [2022-01-10 08:39:24,009] jepsen worker 709 - jepsen.util 709 :ok :add 3617 INFO [2022-01-10 08:39:24,009] jepsen worker 483 - jepsen.util 483 :ok :add 3896 INFO [2022-01-10 08:39:24,010] jepsen worker 717 - jepsen.util 717 :ok :add 3566 INFO [2022-01-10 08:39:24,010] jepsen worker 607 - jepsen.util 607 :ok :add 3964 INFO [2022-01-10 08:39:24,010] jepsen worker 257 - jepsen.util 257 :ok :add 3623 INFO [2022-01-10 08:39:24,010] jepsen worker 759 - jepsen.util 759 :invoke :add 3969 INFO [2022-01-10 08:39:24,012] jepsen worker 263 - jepsen.util 263 :ok :add 3965 INFO [2022-01-10 08:39:24,014] jepsen worker 730 - jepsen.util 730 :invoke :add 3970 INFO [2022-01-10 08:39:24,015] jepsen worker 417 - jepsen.util 417 :ok :add 3967 INFO [2022-01-10 08:39:24,017] jepsen worker 359 - jepsen.util 359 :invoke :add 3971 INFO [2022-01-10 08:39:24,019] jepsen worker 218 - jepsen.util 218 :ok :add 3966 INFO [2022-01-10 08:39:24,020] jepsen worker 688 - jepsen.util 688 :invoke :add 3972 INFO [2022-01-10 08:39:24,022] jepsen worker 670 - jepsen.util 670 :invoke :add 3973 INFO [2022-01-10 08:39:24,024] jepsen worker 804 - jepsen.util 804 :invoke :add 3974 INFO [2022-01-10 08:39:24,026] jepsen worker 588 - jepsen.util 588 :invoke :add 3975 INFO [2022-01-10 08:39:24,029] jepsen worker 226 - jepsen.util 226 :invoke :add 3976 INFO [2022-01-10 08:39:24,030] jepsen worker 267 - jepsen.util 267 :ok :add 3946 INFO [2022-01-10 08:39:24,032] jepsen worker 758 - jepsen.util 758 :invoke :add 3977 INFO [2022-01-10 08:39:24,034] jepsen worker 351 - jepsen.util 351 :invoke :add 3978 INFO [2022-01-10 08:39:24,037] jepsen worker 356 - jepsen.util 356 :invoke :add 3979 INFO [2022-01-10 08:39:24,038] jepsen worker 670 - jepsen.util 670 :ok :add 3973 INFO [2022-01-10 08:39:24,040] jepsen worker 613 - jepsen.util 613 :invoke :add 3980 INFO [2022-01-10 08:39:24,042] jepsen worker 600 - jepsen.util 600 :invoke :add 3981 INFO [2022-01-10 08:39:24,043] jepsen worker 882 - jepsen.util 882 :ok :add 3968 INFO [2022-01-10 08:39:24,043] jepsen worker 730 - jepsen.util 730 :ok :add 3970 INFO [2022-01-10 08:39:24,045] jepsen worker 738 - jepsen.util 738 :invoke :add 3982 INFO [2022-01-10 08:39:24,048] jepsen worker 617 - jepsen.util 617 :invoke :add 3983 INFO [2022-01-10 08:39:24,050] jepsen worker 196 - jepsen.util 196 :invoke :add 3984 INFO [2022-01-10 08:39:24,051] jepsen worker 688 - jepsen.util 688 :ok :add 3972 INFO [2022-01-10 08:39:24,053] jepsen worker 869 - jepsen.util 869 :invoke :add 3985 INFO [2022-01-10 08:39:24,054] jepsen worker 359 - jepsen.util 359 :ok :add 3971 INFO [2022-01-10 08:39:24,054] jepsen worker 759 - jepsen.util 759 :ok :add 3969 INFO [2022-01-10 08:39:24,055] jepsen worker 351 - jepsen.util 351 :ok :add 3978 INFO [2022-01-10 08:39:24,055] jepsen worker 356 - jepsen.util 356 :ok :add 3979 INFO [2022-01-10 08:39:24,057] jepsen worker 493 - jepsen.util 493 :invoke :add 3986 INFO [2022-01-10 08:39:24,057] jepsen worker 196 - jepsen.util 196 :ok :add 3984 INFO [2022-01-10 08:39:24,059] jepsen worker 987 - jepsen.util 987 :invoke :add 3987 INFO [2022-01-10 08:39:24,060] jepsen worker 600 - jepsen.util 600 :ok :add 3981 INFO [2022-01-10 08:39:24,062] jepsen worker 250 - jepsen.util 250 :invoke :add 3988 INFO [2022-01-10 08:39:24,063] jepsen worker 617 - jepsen.util 617 :ok :add 3983 INFO [2022-01-10 08:39:24,063] jepsen worker 613 - jepsen.util 613 :ok :add 3980 INFO [2022-01-10 08:39:24,065] jepsen worker 759 - jepsen.util 759 :invoke :add 3989 INFO [2022-01-10 08:39:24,068] jepsen worker 901 - jepsen.util 901 :invoke :add 3990 INFO [2022-01-10 08:39:24,070] jepsen worker 230 - jepsen.util 230 :invoke :add 3991 INFO [2022-01-10 08:39:24,072] jepsen worker 383 - jepsen.util 383 :ok :add 3942 INFO [2022-01-10 08:39:24,074] jepsen worker 420 - jepsen.util 420 :invoke :add 3992 INFO [2022-01-10 08:39:24,074] jepsen worker 493 - jepsen.util 493 :ok :add 3986 INFO [2022-01-10 08:39:24,075] jepsen worker 869 - jepsen.util 869 :ok :add 3985 INFO [2022-01-10 08:39:24,077] jepsen worker 210 - jepsen.util 210 :invoke :add 3993 INFO [2022-01-10 08:39:24,079] jepsen worker 738 - jepsen.util 738 :ok :add 3982 INFO [2022-01-10 08:39:24,080] jepsen worker 507 - jepsen.util 507 :invoke :add 3994 INFO [2022-01-10 08:39:24,081] jepsen worker 987 - jepsen.util 987 :ok :add 3987 INFO [2022-01-10 08:39:24,083] jepsen worker 20 - jepsen.util 20 :invoke :add 3995 INFO [2022-01-10 08:39:24,086] jepsen worker 292 - jepsen.util 292 :invoke :add 3996 INFO [2022-01-10 08:39:24,089] jepsen worker 835 - jepsen.util 835 :invoke :add 3997 INFO [2022-01-10 08:39:24,092] jepsen worker 121 - jepsen.util 121 :invoke :add 3998 INFO [2022-01-10 08:39:24,094] jepsen worker 680 - jepsen.util 680 :invoke :add 3999 INFO [2022-01-10 08:39:24,096] jepsen worker 987 - jepsen.util 987 :invoke :add 4000 INFO [2022-01-10 08:39:24,098] jepsen worker 121 - jepsen.util 121 :ok :add 3998 INFO [2022-01-10 08:39:24,098] jepsen worker 789 - jepsen.util 789 :invoke :add 4001 INFO [2022-01-10 08:39:24,099] jepsen worker 759 - jepsen.util 759 :ok :add 3989 INFO [2022-01-10 08:39:24,101] jepsen worker 680 - jepsen.util 680 :ok :add 3999 INFO [2022-01-10 08:39:24,101] jepsen worker 836 - jepsen.util 836 :invoke :add 4002 INFO [2022-01-10 08:39:24,104] jepsen worker 577 - jepsen.util 577 :invoke :add 4003 INFO [2022-01-10 08:39:24,107] jepsen worker 230 - jepsen.util 230 :ok :add 3991 INFO [2022-01-10 08:39:24,107] jepsen worker 852 - jepsen.util 852 :invoke :add 4004 INFO [2022-01-10 08:39:24,111] jepsen worker 250 - jepsen.util 250 :ok :add 3988 INFO [2022-01-10 08:39:24,111] jepsen worker 344 - jepsen.util 344 :invoke :add 4005 INFO [2022-01-10 08:39:24,111] jepsen worker 621 - jepsen.util 621 :ok :add 3687 INFO [2022-01-10 08:39:24,111] jepsen worker 274 - jepsen.util 274 :ok :add 3832 INFO [2022-01-10 08:39:24,112] jepsen worker 901 - jepsen.util 901 :ok :add 3990 INFO [2022-01-10 08:39:24,112] jepsen worker 824 - jepsen.util 824 :ok :add 3806 INFO [2022-01-10 08:39:24,112] jepsen worker 447 - jepsen.util 447 :ok :add 3723 INFO [2022-01-10 08:39:24,113] jepsen worker 48 - jepsen.util 48 :ok :add 3717 INFO [2022-01-10 08:39:24,115] jepsen worker 213 - jepsen.util 213 :invoke :add 4006 INFO [2022-01-10 08:39:24,117] jepsen worker 757 - jepsen.util 757 :ok :add 3958 INFO [2022-01-10 08:39:24,117] jepsen worker 210 - jepsen.util 210 :ok :add 3993 INFO [2022-01-10 08:39:24,118] jepsen worker 900 - jepsen.util 900 :ok :add 3940 INFO [2022-01-10 08:39:24,118] jepsen worker 804 - jepsen.util 804 :ok :add 3974 INFO [2022-01-10 08:39:24,119] jepsen worker 792 - jepsen.util 792 :invoke :add 4007 INFO [2022-01-10 08:39:24,122] jepsen worker 445 - jepsen.util 445 :invoke :add 4008 INFO [2022-01-10 08:39:24,125] jepsen worker 582 - jepsen.util 582 :invoke :add 4009 INFO [2022-01-10 08:39:24,127] jepsen worker 259 - jepsen.util 259 :invoke :add 4010 INFO [2022-01-10 08:39:24,129] jepsen worker 445 - jepsen.util 445 :ok :add 4008 INFO [2022-01-10 08:39:24,130] jepsen worker 232 - jepsen.util 232 :invoke :add 4011 INFO [2022-01-10 08:39:24,133] jepsen worker 965 - jepsen.util 965 :invoke :add 4012 INFO [2022-01-10 08:39:24,136] jepsen worker 303 - jepsen.util 303 :invoke :add 4013 INFO [2022-01-10 08:39:24,136] jepsen worker 582 - jepsen.util 582 :ok :add 4009 INFO [2022-01-10 08:39:24,138] jepsen worker 444 - jepsen.util 444 :invoke :add 4014 INFO [2022-01-10 08:39:24,139] jepsen worker 259 - jepsen.util 259 :ok :add 4010 INFO [2022-01-10 08:39:24,141] jepsen worker 897 - jepsen.util 897 :invoke :add 4015 INFO [2022-01-10 08:39:24,143] jepsen worker 232 - jepsen.util 232 :ok :add 4011 INFO [2022-01-10 08:39:24,144] jepsen worker 420 - jepsen.util 420 :ok :add 3992 INFO [2022-01-10 08:39:24,144] jepsen worker 966 - jepsen.util 966 :invoke :add 4016 INFO [2022-01-10 08:39:24,146] jepsen worker 406 - jepsen.util 406 :ok :add 3685 INFO [2022-01-10 08:39:24,146] jepsen worker 989 - jepsen.util 989 :ok :add 3828 INFO [2022-01-10 08:39:24,146] jepsen worker 767 - jepsen.util 767 :ok :add 3721 INFO [2022-01-10 08:39:24,147] jepsen worker 965 - jepsen.util 965 :ok :add 4012 INFO [2022-01-10 08:39:24,148] jepsen worker 737 - jepsen.util 737 :invoke :add 4017 INFO [2022-01-10 08:39:24,148] jepsen worker 987 - jepsen.util 987 :ok :add 4000 INFO [2022-01-10 08:39:24,148] jepsen worker 789 - jepsen.util 789 :ok :add 4001 INFO [2022-01-10 08:39:24,148] jepsen worker 507 - jepsen.util 507 :ok :add 3994 INFO [2022-01-10 08:39:24,148] jepsen worker 189 - jepsen.util 189 :ok :add 3830 INFO [2022-01-10 08:39:24,148] jepsen worker 839 - jepsen.util 839 :ok :add 3715 INFO [2022-01-10 08:39:24,148] jepsen worker 947 - jepsen.util 947 :ok :add 3804 INFO [2022-01-10 08:39:24,148] jepsen worker 237 - jepsen.util 237 :ok :add 3739 INFO [2022-01-10 08:39:24,150] jepsen worker 303 - jepsen.util 303 :ok :add 4013 INFO [2022-01-10 08:39:24,151] jepsen worker 444 - jepsen.util 444 :ok :add 4014 INFO [2022-01-10 08:39:24,154] jepsen worker 59 - jepsen.util 59 :invoke :add 4018 INFO [2022-01-10 08:39:24,154] jepsen worker 897 - jepsen.util 897 :ok :add 4015 INFO [2022-01-10 08:39:24,156] jepsen worker 611 - jepsen.util 611 :invoke :add 4019 INFO [2022-01-10 08:39:24,159] jepsen worker 87 - jepsen.util 87 :invoke :add 4020 INFO [2022-01-10 08:39:24,161] jepsen worker 260 - jepsen.util 260 :invoke :add 4021 INFO [2022-01-10 08:39:24,162] jepsen worker 737 - jepsen.util 737 :ok :add 4017 INFO [2022-01-10 08:39:24,164] jepsen worker 637 - jepsen.util 637 :invoke :add 4022 INFO [2022-01-10 08:39:24,164] jepsen worker 966 - jepsen.util 966 :ok :add 4016 INFO [2022-01-10 08:39:24,167] jepsen worker 900 - jepsen.util 900 :invoke :add 4023 INFO [2022-01-10 08:39:24,170] jepsen worker 183 - jepsen.util 183 :invoke :add 4024 INFO [2022-01-10 08:39:24,170] jepsen worker 87 - jepsen.util 87 :ok :add 4020 INFO [2022-01-10 08:39:24,173] jepsen worker 326 - jepsen.util 326 :invoke :add 4025 INFO [2022-01-10 08:39:24,175] jepsen worker 637 - jepsen.util 637 :ok :add 4022 INFO [2022-01-10 08:39:24,176] jepsen worker 260 - jepsen.util 260 :ok :add 4021 INFO [2022-01-10 08:39:24,177] jepsen worker 327 - jepsen.util 327 :invoke :add 4026 INFO [2022-01-10 08:39:24,178] jepsen worker 836 - jepsen.util 836 :ok :add 4002 INFO [2022-01-10 08:39:24,179] jepsen worker 284 - jepsen.util 284 :invoke :add 4027 INFO [2022-01-10 08:39:24,181] jepsen worker 900 - jepsen.util 900 :ok :add 4023 INFO [2022-01-10 08:39:24,182] jepsen worker 333 - jepsen.util 333 :ok :add 3931 INFO [2022-01-10 08:39:24,182] jepsen worker 725 - jepsen.util 725 :ok :add 3895 INFO [2022-01-10 08:39:24,182] jepsen worker 577 - jepsen.util 577 :ok :add 4003 INFO [2022-01-10 08:39:24,182] jepsen worker 307 - jepsen.util 307 :invoke :add 4028 INFO [2022-01-10 08:39:24,182] jepsen worker 543 - jepsen.util 543 :ok :add 3929 INFO [2022-01-10 08:39:24,183] jepsen worker 326 - jepsen.util 326 :ok :add 4025 INFO [2022-01-10 08:39:24,184] jepsen worker 78 - jepsen.util 78 :ok :add 3905 INFO [2022-01-10 08:39:24,184] jepsen worker 719 - jepsen.util 719 :ok :add 3784 INFO [2022-01-10 08:39:24,188] jepsen worker 51 - jepsen.util 51 :invoke :add 4029 INFO [2022-01-10 08:39:24,190] jepsen worker 183 - jepsen.util 183 :ok :add 4024 INFO [2022-01-10 08:39:24,191] jepsen worker 419 - jepsen.util 419 :invoke :add 4030 INFO [2022-01-10 08:39:24,194] jepsen worker 757 - jepsen.util 757 :invoke :add 4031 INFO [2022-01-10 08:39:24,194] jepsen worker 327 - jepsen.util 327 :ok :add 4026 INFO [2022-01-10 08:39:24,195] jepsen worker 284 - jepsen.util 284 :ok :add 4027 INFO [2022-01-10 08:39:24,197] jepsen worker 941 - jepsen.util 941 :invoke :add 4032 INFO [2022-01-10 08:39:24,199] jepsen worker 580 - jepsen.util 580 :invoke :add 4033 INFO [2022-01-10 08:39:24,201] jepsen worker 70 - jepsen.util 70 :invoke :add 4034 INFO [2022-01-10 08:39:24,203] jepsen worker 832 - jepsen.util 832 :invoke :add 4035 INFO [2022-01-10 08:39:24,205] jepsen worker 970 - jepsen.util 970 :invoke :add 4036 INFO [2022-01-10 08:39:24,208] jepsen worker 726 - jepsen.util 726 :invoke :add 4037 INFO [2022-01-10 08:39:24,211] jepsen worker 126 - jepsen.util 126 :invoke :add 4038 INFO [2022-01-10 08:39:24,213] jepsen worker 475 - jepsen.util 475 :invoke :add 4039 INFO [2022-01-10 08:39:24,215] jepsen worker 611 - jepsen.util 611 :ok :add 4019 INFO [2022-01-10 08:39:24,216] jepsen worker 415 - jepsen.util 415 :invoke :add 4040 INFO [2022-01-10 08:39:24,218] jepsen worker 581 - jepsen.util 581 :invoke :add 4041 INFO [2022-01-10 08:39:24,220] jepsen worker 126 - jepsen.util 126 :ok :add 4038 INFO [2022-01-10 08:39:24,220] jepsen worker 360 - jepsen.util 360 :invoke :add 4042 INFO [2022-01-10 08:39:24,222] jepsen worker 475 - jepsen.util 475 :ok :add 4039 INFO [2022-01-10 08:39:24,223] jepsen worker 138 - jepsen.util 138 :invoke :add 4043 INFO [2022-01-10 08:39:24,226] jepsen worker 237 - jepsen.util 237 :invoke :add 4044 INFO [2022-01-10 08:39:24,229] jepsen worker 935 - jepsen.util 935 :invoke :add 4045 INFO [2022-01-10 08:39:24,230] jepsen worker 59 - jepsen.util 59 :ok :add 4018 INFO [2022-01-10 08:39:24,230] jepsen worker 726 - jepsen.util 726 :ok :add 4037 INFO [2022-01-10 08:39:24,230] jepsen worker 70 - jepsen.util 70 :ok :add 4034 INFO [2022-01-10 08:39:24,231] jepsen worker 239 - jepsen.util 239 :invoke :add 4046 INFO [2022-01-10 08:39:24,232] jepsen worker 2 - jepsen.util 2 :ok :add 3858 INFO [2022-01-10 08:39:24,232] jepsen worker 584 - jepsen.util 584 :ok :add 3874 INFO [2022-01-10 08:39:24,234] jepsen worker 643 - jepsen.util 643 :invoke :add 4047 INFO [2022-01-10 08:39:24,235] jepsen worker 101 - jepsen.util 101 :ok :add 3840 INFO [2022-01-10 08:39:24,237] jepsen worker 734 - jepsen.util 734 :invoke :add 4048 INFO [2022-01-10 08:39:24,240] jepsen worker 804 - jepsen.util 804 :invoke :add 4049 INFO [2022-01-10 08:39:24,244] jepsen worker 326 - jepsen.util 326 :invoke :add 4050 INFO [2022-01-10 08:39:24,246] jepsen worker 970 - jepsen.util 970 :ok :add 4036 INFO [2022-01-10 08:39:24,246] jepsen worker 360 - jepsen.util 360 :ok :add 4042 INFO [2022-01-10 08:39:24,246] jepsen worker 955 - jepsen.util 955 :ok :add 3907 INFO [2022-01-10 08:39:24,246] jepsen worker 942 - jepsen.util 942 :ok :add 3933 INFO [2022-01-10 08:39:24,246] jepsen worker 916 - jepsen.util 916 :ok :add 3897 INFO [2022-01-10 08:39:24,248] jepsen worker 57 - jepsen.util 57 :invoke :add 4051 INFO [2022-01-10 08:39:24,249] jepsen worker 734 - jepsen.util 734 :ok :add 4048 INFO [2022-01-10 08:39:24,251] jepsen worker 804 - jepsen.util 804 :ok :add 4049 INFO [2022-01-10 08:39:24,251] jepsen worker 227 - jepsen.util 227 :invoke :add 4052 INFO [2022-01-10 08:39:24,254] jepsen worker 744 - jepsen.util 744 :invoke :add 4053 INFO [2022-01-10 08:39:24,255] jepsen worker 832 - jepsen.util 832 :ok :add 4035 INFO [2022-01-10 08:39:24,256] jepsen worker 57 - jepsen.util 57 :ok :add 4051 INFO [2022-01-10 08:39:24,256] jepsen worker 326 - jepsen.util 326 :ok :add 4050 INFO [2022-01-10 08:39:24,258] jepsen worker 693 - jepsen.util 693 :invoke :add 4054 INFO [2022-01-10 08:39:24,261] jepsen worker 227 - jepsen.util 227 :ok :add 4052 INFO [2022-01-10 08:39:24,263] jepsen worker 237 - jepsen.util 237 :ok :add 4044 INFO [2022-01-10 08:39:24,263] jepsen worker 14 - jepsen.util 14 :invoke :add 4055 INFO [2022-01-10 08:39:24,264] jepsen worker 744 - jepsen.util 744 :ok :add 4053 INFO [2022-01-10 08:39:24,266] jepsen worker 275 - jepsen.util 275 :invoke :add 4056 INFO [2022-01-10 08:39:24,269] jepsen worker 435 - jepsen.util 435 :invoke :add 4057 INFO [2022-01-10 08:39:24,271] jepsen worker 935 - jepsen.util 935 :ok :add 4045 INFO [2022-01-10 08:39:24,271] jepsen worker 213 - jepsen.util 213 :ok :add 4006 INFO [2022-01-10 08:39:24,271] jepsen worker 758 - jepsen.util 758 :ok :add 3977 INFO [2022-01-10 08:39:24,271] jepsen worker 145 - jepsen.util 145 :ok :add 3943 INFO [2022-01-10 08:39:24,271] jepsen worker 941 - jepsen.util 941 :ok :add 4032 INFO [2022-01-10 08:39:24,273] jepsen worker 393 - jepsen.util 393 :invoke :add 4058 INFO [2022-01-10 08:39:24,276] jepsen worker 14 - jepsen.util 14 :ok :add 4055 INFO [2022-01-10 08:39:24,276] jepsen worker 643 - jepsen.util 643 :ok :add 4047 INFO [2022-01-10 08:39:24,277] jepsen worker 945 - jepsen.util 945 :invoke :add 4059 INFO [2022-01-10 08:39:24,278] jepsen worker 239 - jepsen.util 239 :ok :add 4046 INFO [2022-01-10 08:39:24,280] jepsen worker 91 - jepsen.util 91 :invoke :add 4060 INFO [2022-01-10 08:39:24,282] jepsen worker 64 - jepsen.util 64 :invoke :add 4061 INFO [2022-01-10 08:39:24,283] jepsen worker 693 - jepsen.util 693 :ok :add 4054 INFO [2022-01-10 08:39:24,283] jepsen worker 435 - jepsen.util 435 :ok :add 4057 INFO [2022-01-10 08:39:24,284] jepsen worker 842 - jepsen.util 842 :invoke :add 4062 INFO [2022-01-10 08:39:24,287] jepsen worker 370 - jepsen.util 370 :invoke :add 4063 INFO [2022-01-10 08:39:24,289] jepsen worker 120 - jepsen.util 120 :invoke :add 4064 INFO [2022-01-10 08:39:24,291] jepsen worker 562 - jepsen.util 562 :invoke :add 4065 INFO [2022-01-10 08:39:24,293] jepsen worker 275 - jepsen.util 275 :ok :add 4056 INFO [2022-01-10 08:39:24,293] jepsen worker 984 - jepsen.util 984 :invoke :add 4066 INFO [2022-01-10 08:39:24,295] jepsen worker 120 - jepsen.util 120 :ok :add 4064 INFO [2022-01-10 08:39:24,296] jepsen worker 46 - jepsen.util 46 :invoke :add 4067 INFO [2022-01-10 08:39:24,299] jepsen worker 462 - jepsen.util 462 :invoke :add 4068 INFO [2022-01-10 08:39:24,301] jepsen worker 66 - jepsen.util 66 :invoke :add 4069 INFO [2022-01-10 08:39:24,302] jepsen worker 370 - jepsen.util 370 :ok :add 4063 INFO [2022-01-10 08:39:24,302] jepsen worker 562 - jepsen.util 562 :ok :add 4065 INFO [2022-01-10 08:39:24,303] jepsen worker 984 - jepsen.util 984 :ok :add 4066 INFO [2022-01-10 08:39:24,304] jepsen worker 645 - jepsen.util 645 :ok :add 3876 INFO [2022-01-10 08:39:24,304] jepsen worker 394 - jepsen.util 394 :invoke :add 4070 INFO [2022-01-10 08:39:24,306] jepsen worker 46 - jepsen.util 46 :ok :add 4067 INFO [2022-01-10 08:39:24,307] jepsen worker 733 - jepsen.util 733 :invoke :add 4071 INFO [2022-01-10 08:39:24,310] jepsen worker 627 - jepsen.util 627 :invoke :add 4072 INFO [2022-01-10 08:39:24,312] jepsen worker 564 - jepsen.util 564 :invoke :add 4073 INFO [2022-01-10 08:39:24,314] jepsen worker 377 - jepsen.util 377 :invoke :add 4074 INFO [2022-01-10 08:39:24,315] jepsen worker 64 - jepsen.util 64 :ok :add 4061 INFO [2022-01-10 08:39:24,317] jepsen worker 462 - jepsen.util 462 :ok :add 4068 INFO [2022-01-10 08:39:24,317] jepsen worker 637 - jepsen.util 637 :invoke :add 4075 INFO [2022-01-10 08:39:24,317] jepsen worker 394 - jepsen.util 394 :ok :add 4070 INFO [2022-01-10 08:39:24,318] jepsen worker 91 - jepsen.util 91 :ok :add 4060 INFO [2022-01-10 08:39:24,319] jepsen worker 501 - jepsen.util 501 :invoke :add 4076 INFO [2022-01-10 08:39:24,322] jepsen worker 619 - jepsen.util 619 :invoke :add 4077 INFO [2022-01-10 08:39:24,324] jepsen worker 135 - jepsen.util 135 :invoke :add 4078 INFO [2022-01-10 08:39:24,326] jepsen worker 505 - jepsen.util 505 :invoke :add 4079 INFO [2022-01-10 08:39:24,329] jepsen worker 790 - jepsen.util 790 :invoke :add 4080 INFO [2022-01-10 08:39:24,331] jepsen worker 547 - jepsen.util 547 :invoke :add 4081 INFO [2022-01-10 08:39:24,333] jepsen worker 304 - jepsen.util 304 :invoke :add 4082 INFO [2022-01-10 08:39:24,335] jepsen worker 804 - jepsen.util 804 :invoke :add 4083 INFO [2022-01-10 08:39:24,338] jepsen worker 282 - jepsen.util 282 :invoke :add 4084 INFO [2022-01-10 08:39:24,340] jepsen worker 107 - jepsen.util 107 :invoke :add 4085 INFO [2022-01-10 08:39:24,343] jepsen worker 178 - jepsen.util 178 :invoke :add 4086 INFO [2022-01-10 08:39:24,345] jepsen worker 59 - jepsen.util 59 :invoke :add 4087 INFO [2022-01-10 08:39:24,346] jepsen worker 282 - jepsen.util 282 :ok :add 4084 INFO [2022-01-10 08:39:24,349] jepsen worker 107 - jepsen.util 107 :ok :add 4085 INFO [2022-01-10 08:39:24,350] jepsen worker 791 - jepsen.util 791 :invoke :add 4088 INFO [2022-01-10 08:39:24,352] jepsen worker 801 - jepsen.util 801 :invoke :add 4089 INFO [2022-01-10 08:39:24,355] jepsen worker 940 - jepsen.util 940 :invoke :add 4090 INFO [2022-01-10 08:39:24,356] jepsen worker 178 - jepsen.util 178 :ok :add 4086 INFO [2022-01-10 08:39:24,357] jepsen worker 59 - jepsen.util 59 :ok :add 4087 INFO [2022-01-10 08:39:24,358] jepsen worker 879 - jepsen.util 879 :invoke :add 4091 INFO [2022-01-10 08:39:24,360] jepsen worker 224 - jepsen.util 224 :invoke :add 4092 INFO [2022-01-10 08:39:24,363] jepsen worker 764 - jepsen.util 764 :invoke :add 4093 INFO [2022-01-10 08:39:24,365] jepsen worker 70 - jepsen.util 70 :invoke :add 4094 INFO [2022-01-10 08:39:24,367] jepsen worker 802 - jepsen.util 802 :invoke :add 4095 INFO [2022-01-10 08:39:24,370] jepsen worker 600 - jepsen.util 600 :invoke :add 4096 INFO [2022-01-10 08:39:24,372] jepsen worker 318 - jepsen.util 318 :invoke :add 4097 INFO [2022-01-10 08:39:24,374] jepsen worker 103 - jepsen.util 103 :invoke :add 4098 INFO [2022-01-10 08:39:24,377] jepsen worker 709 - jepsen.util 709 :invoke :add 4099 INFO [2022-01-10 08:39:24,378] jepsen worker 950 - jepsen.util 950 :invoke :add 4100 INFO [2022-01-10 08:39:24,381] jepsen worker 895 - jepsen.util 895 :invoke :add 4101 WARN [2022-01-10 08:39:24,383] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":3030667,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":3030667,"total_server_duration_us":1783336,"encode_duration_us":383,"operation_id":"0xd80","timeout_ms":10000,"last_server_duration_us":1783336,"total_duration_us":3030994},{"operation_name":"insert","last_dispatch_duration_us":2470434,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":2470434,"total_server_duration_us":1144837,"encode_duration_us":326,"operation_id":"0xd68","timeout_ms":10000,"last_server_duration_us":1144837,"total_duration_us":2470719},{"operation_name":"insert","last_dispatch_duration_us":2460050,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":2460050,"total_server_duration_us":1762101,"encode_duration_us":275,"operation_id":"0xe42","timeout_ms":10000,"last_server_duration_us":1762101,"total_duration_us":2460415},{"operation_name":"insert","last_dispatch_duration_us":2375678,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":2375678,"total_server_duration_us":1768460,"encode_duration_us":359,"operation_id":"0xe67","timeout_ms":10000,"last_server_duration_us":1768460,"total_duration_us":2375944},{"operation_name":"insert","last_dispatch_duration_us":2272313,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":2272313,"total_server_duration_us":1027200,"encode_duration_us":318,"operation_id":"0xd81","timeout_ms":10000,"last_server_duration_us":1027200,"total_duration_us":2272576},{"operation_name":"insert","last_dispatch_duration_us":2116344,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":2116344,"total_server_duration_us":1134278,"encode_duration_us":451,"operation_id":"0xddd","timeout_ms":10000,"last_server_duration_us":1134278,"total_duration_us":2116618},{"operation_name":"insert","last_dispatch_duration_us":2045452,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":2045452,"total_server_duration_us":1624158,"encode_duration_us":390,"operation_id":"0xebc","timeout_ms":10000,"last_server_duration_us":1624158,"total_duration_us":2045752},{"operation_name":"insert","last_dispatch_duration_us":2013403,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":2013403,"total_server_duration_us":763355,"encode_duration_us":417,"operation_id":"0xd7f","timeout_ms":10000,"last_server_duration_us":763355,"total_duration_us":2013698},{"operation_name":"insert","last_dispatch_duration_us":1970282,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":1970282,"total_server_duration_us":1721527,"encode_duration_us":378,"operation_id":"0xefd","timeout_ms":10000,"last_server_duration_us":1721527,"total_duration_us":1970498},{"operation_name":"insert","last_dispatch_duration_us":1781968,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":1781968,"total_server_duration_us":1302581,"encode_duration_us":298,"operation_id":"0xea2","timeout_ms":10000,"last_server_duration_us":1302581,"total_duration_us":1782243}],"total_count":51}} INFO [2022-01-10 08:39:24,384] jepsen worker 608 - jepsen.util 608 :invoke :add 4102 INFO [2022-01-10 08:39:24,385] jepsen worker 970 - jepsen.util 970 :invoke :add 4103 INFO [2022-01-10 08:39:24,387] jepsen worker 800 - jepsen.util 800 :invoke :add 4104 INFO [2022-01-10 08:39:24,389] jepsen worker 257 - jepsen.util 257 :invoke :add 4105 INFO [2022-01-10 08:39:24,391] jepsen worker 733 - jepsen.util 733 :ok :add 4071 INFO [2022-01-10 08:39:24,391] jepsen worker 66 - jepsen.util 66 :ok :add 4069 INFO [2022-01-10 08:39:24,392] jepsen worker 223 - jepsen.util 223 :invoke :add 4106 INFO [2022-01-10 08:39:24,395] jepsen worker 52 - jepsen.util 52 :invoke :add 4107 INFO [2022-01-10 08:39:24,397] jepsen worker 835 - jepsen.util 835 :ok :add 3997 INFO [2022-01-10 08:39:24,397] jepsen worker 721 - jepsen.util 721 :invoke :add 4108 INFO [2022-01-10 08:39:24,399] jepsen worker 627 - jepsen.util 627 :ok :add 4072 INFO [2022-01-10 08:39:24,399] jepsen worker 329 - jepsen.util 329 :invoke :add 4109 INFO [2022-01-10 08:39:24,400] jepsen worker 842 - jepsen.util 842 :ok :add 4062 INFO [2022-01-10 08:39:24,401] jepsen worker 985 - jepsen.util 985 :invoke :add 4110 INFO [2022-01-10 08:39:24,404] jepsen worker 588 - jepsen.util 588 :ok :add 3975 INFO [2022-01-10 08:39:24,404] jepsen worker 564 - jepsen.util 564 :ok :add 4073 INFO [2022-01-10 08:39:24,404] jepsen worker 775 - jepsen.util 775 :invoke :add 4111 INFO [2022-01-10 08:39:24,405] jepsen worker 443 - jepsen.util 443 :ok :add 3941 INFO [2022-01-10 08:39:24,406] jepsen worker 307 - jepsen.util 307 :ok :add 4028 INFO [2022-01-10 08:39:24,406] jepsen worker 458 - jepsen.util 458 :ok :add 3959 INFO [2022-01-10 08:39:24,407] jepsen worker 852 - jepsen.util 852 :ok :add 4004 INFO [2022-01-10 08:39:24,407] jepsen worker 419 - jepsen.util 419 :ok :add 4030 INFO [2022-01-10 08:39:24,407] jepsen worker 853 - jepsen.util 853 :invoke :add 4112 INFO [2022-01-10 08:39:24,411] jepsen worker 424 - jepsen.util 424 :invoke :add 4113 INFO [2022-01-10 08:39:24,413] jepsen worker 25 - jepsen.util 25 :invoke :add 4114 INFO [2022-01-10 08:39:24,415] jepsen worker 355 - jepsen.util 355 :invoke :add 4115 INFO [2022-01-10 08:39:24,418] jepsen worker 267 - jepsen.util 267 :invoke :add 4116 INFO [2022-01-10 08:39:24,420] jepsen worker 998 - jepsen.util 998 :invoke :add 4117 INFO [2022-01-10 08:39:24,422] jepsen worker 279 - jepsen.util 279 :invoke :add 4118 INFO [2022-01-10 08:39:24,425] jepsen worker 892 - jepsen.util 892 :invoke :add 4119 INFO [2022-01-10 08:39:24,427] jepsen worker 809 - jepsen.util 809 :invoke :add 4120 INFO [2022-01-10 08:39:24,430] jepsen worker 896 - jepsen.util 896 :invoke :add 4121 INFO [2022-01-10 08:39:24,432] jepsen worker 220 - jepsen.util 220 :invoke :add 4122 INFO [2022-01-10 08:39:24,434] jepsen worker 559 - jepsen.util 559 :invoke :add 4123 INFO [2022-01-10 08:39:24,436] jepsen worker 817 - jepsen.util 817 :invoke :add 4124 INFO [2022-01-10 08:39:24,439] jepsen worker 276 - jepsen.util 276 :invoke :add 4125 INFO [2022-01-10 08:39:24,441] jepsen worker 25 - jepsen.util 25 :ok :add 4114 INFO [2022-01-10 08:39:24,441] jepsen worker 105 - jepsen.util 105 :invoke :add 4126 INFO [2022-01-10 08:39:24,443] jepsen worker 355 - jepsen.util 355 :ok :add 4115 INFO [2022-01-10 08:39:24,443] jepsen worker 896 - jepsen.util 896 :ok :add 4121 INFO [2022-01-10 08:39:24,444] jepsen worker 722 - jepsen.util 722 :invoke :add 4127 INFO [2022-01-10 08:39:24,447] jepsen worker 779 - jepsen.util 779 :invoke :add 4128 INFO [2022-01-10 08:39:24,450] jepsen worker 873 - jepsen.util 873 :invoke :add 4129 INFO [2022-01-10 08:39:24,453] jepsen worker 806 - jepsen.util 806 :invoke :add 4130 INFO [2022-01-10 08:39:24,455] jepsen worker 324 - jepsen.util 324 :invoke :add 4131 INFO [2022-01-10 08:39:24,457] jepsen worker 989 - jepsen.util 989 :invoke :add 4132 INFO [2022-01-10 08:39:24,460] jepsen worker 227 - jepsen.util 227 :invoke :add 4133 INFO [2022-01-10 08:39:24,462] jepsen worker 505 - jepsen.util 505 :ok :add 4079 INFO [2022-01-10 08:39:24,462] jepsen worker 715 - jepsen.util 715 :invoke :add 4134 INFO [2022-01-10 08:39:24,465] jepsen worker 901 - jepsen.util 901 :invoke :add 4135 INFO [2022-01-10 08:39:24,467] jepsen worker 569 - jepsen.util 569 :invoke :add 4136 INFO [2022-01-10 08:39:24,469] jepsen worker 113 - jepsen.util 113 :invoke :add 4137 INFO [2022-01-10 08:39:24,471] jepsen worker 940 - jepsen.util 940 :ok :add 4090 INFO [2022-01-10 08:39:24,471] jepsen worker 791 - jepsen.util 791 :ok :add 4088 INFO [2022-01-10 08:39:24,471] jepsen worker 279 - jepsen.util 279 :ok :add 4118 INFO [2022-01-10 08:39:24,471] jepsen worker 950 - jepsen.util 950 :ok :add 4100 INFO [2022-01-10 08:39:24,472] jepsen worker 512 - jepsen.util 512 :invoke :add 4138 WARN [2022-01-10 08:39:24,473] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":2068202,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":2068202,"total_server_duration_us":2067470,"encode_duration_us":407,"operation_id":"0xe2b","timeout_ms":10000,"last_server_duration_us":2067470,"total_duration_us":2068450},{"operation_name":"insert","last_dispatch_duration_us":2053008,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":2053008,"total_server_duration_us":2052191,"encode_duration_us":291,"operation_id":"0xe31","timeout_ms":10000,"last_server_duration_us":2052191,"total_duration_us":2053246},{"operation_name":"insert","last_dispatch_duration_us":1969495,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":1969495,"total_server_duration_us":1964303,"encode_duration_us":274,"operation_id":"0xe29","timeout_ms":10000,"last_server_duration_us":1964303,"total_duration_us":1969774},{"operation_name":"insert","last_dispatch_duration_us":1715539,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":1715539,"total_server_duration_us":1055994,"encode_duration_us":340,"operation_id":"0xd0f","timeout_ms":10000,"last_server_duration_us":1055994,"total_duration_us":1715760},{"operation_name":"insert","last_dispatch_duration_us":1671669,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":1671669,"total_server_duration_us":1670470,"encode_duration_us":413,"operation_id":"0xf71","timeout_ms":10000,"last_server_duration_us":1670470,"total_duration_us":1671956},{"operation_name":"insert","last_dispatch_duration_us":1596271,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":1596271,"total_server_duration_us":1424590,"encode_duration_us":353,"operation_id":"0xdca","timeout_ms":10000,"last_server_duration_us":1424590,"total_duration_us":1596530},{"operation_name":"insert","last_dispatch_duration_us":1550450,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":1550450,"total_server_duration_us":1422172,"encode_duration_us":327,"operation_id":"0xddc","timeout_ms":10000,"last_server_duration_us":1422172,"total_duration_us":1550776},{"operation_name":"insert","last_dispatch_duration_us":1537444,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":1537444,"total_server_duration_us":1535198,"encode_duration_us":286,"operation_id":"0xf60","timeout_ms":10000,"last_server_duration_us":1535198,"total_duration_us":1537722},{"operation_name":"insert","last_dispatch_duration_us":1427776,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":1427776,"total_server_duration_us":807287,"encode_duration_us":278,"operation_id":"0xd2a","timeout_ms":10000,"last_server_duration_us":807287,"total_duration_us":1428020},{"operation_name":"insert","last_dispatch_duration_us":1425240,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":1425240,"total_server_duration_us":850706,"encode_duration_us":293,"operation_id":"0xd41","timeout_ms":10000,"last_server_duration_us":850706,"total_duration_us":1425528}],"total_count":34}} INFO [2022-01-10 08:39:24,475] jepsen worker 186 - jepsen.util 186 :invoke :add 4139 INFO [2022-01-10 08:39:24,477] jepsen worker 963 - jepsen.util 963 :invoke :add 4140 INFO [2022-01-10 08:39:24,482] jepsen worker 20 - jepsen.util 20 :ok :add 3995 INFO [2022-01-10 08:39:24,482] jepsen worker 998 - jepsen.util 998 :ok :add 4117 WARN [2022-01-10 08:39:24,483] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":3145248,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":3145248,"total_server_duration_us":1549208,"encode_duration_us":320,"operation_id":"0xd53","timeout_ms":10000,"last_server_duration_us":1549208,"total_duration_us":3145481},{"operation_name":"insert","last_dispatch_duration_us":3074323,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":3074323,"total_server_duration_us":1452278,"encode_duration_us":279,"operation_id":"0xd42","timeout_ms":10000,"last_server_duration_us":1452278,"total_duration_us":3074606},{"operation_name":"insert","last_dispatch_duration_us":3002500,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":3002500,"total_server_duration_us":1666331,"encode_duration_us":334,"operation_id":"0xda2","timeout_ms":10000,"last_server_duration_us":1666331,"total_duration_us":3002799},{"operation_name":"insert","last_dispatch_duration_us":2817357,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":2817357,"total_server_duration_us":1445456,"encode_duration_us":327,"operation_id":"0xd95","timeout_ms":10000,"last_server_duration_us":1445456,"total_duration_us":2817594},{"operation_name":"insert","last_dispatch_duration_us":2461503,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":2461503,"total_server_duration_us":946110,"encode_duration_us":458,"operation_id":"0xd6a","timeout_ms":10000,"last_server_duration_us":946110,"total_duration_us":2461759},{"operation_name":"insert","last_dispatch_duration_us":2271566,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":2271566,"total_server_duration_us":1639533,"encode_duration_us":375,"operation_id":"0xebb","timeout_ms":10000,"last_server_duration_us":1639533,"total_duration_us":2271890},{"operation_name":"insert","last_dispatch_duration_us":2163810,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":2163810,"total_server_duration_us":936376,"encode_duration_us":323,"operation_id":"0xdcb","timeout_ms":10000,"last_server_duration_us":936376,"total_duration_us":2164011},{"operation_name":"insert","last_dispatch_duration_us":2112413,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":2112413,"total_server_duration_us":1698001,"encode_duration_us":368,"operation_id":"0xf0c","timeout_ms":10000,"last_server_duration_us":1698001,"total_duration_us":2112709},{"operation_name":"insert","last_dispatch_duration_us":2022468,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":2022468,"total_server_duration_us":562885,"encode_duration_us":515,"operation_id":"0xd7d","timeout_ms":10000,"last_server_duration_us":562885,"total_duration_us":2022830},{"operation_name":"insert","last_dispatch_duration_us":1990792,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":1990792,"total_server_duration_us":820632,"encode_duration_us":367,"operation_id":"0xde0","timeout_ms":10000,"last_server_duration_us":820632,"total_duration_us":1991049}],"total_count":49}} INFO [2022-01-10 08:39:24,483] jepsen worker 559 - jepsen.util 559 :ok :add 4123 INFO [2022-01-10 08:39:24,483] jepsen worker 220 - jepsen.util 220 :ok :add 4122 INFO [2022-01-10 08:39:24,483] jepsen worker 267 - jepsen.util 267 :ok :add 4116 WARN [2022-01-10 08:39:24,483] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":3051705,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":3051705,"total_server_duration_us":1739920,"encode_duration_us":551,"operation_id":"0xd6d","timeout_ms":10000,"last_server_duration_us":1739920,"total_duration_us":3051976},{"operation_name":"insert","last_dispatch_duration_us":2913632,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":2913632,"total_server_duration_us":1771113,"encode_duration_us":294,"operation_id":"0xda0","timeout_ms":10000,"last_server_duration_us":1771113,"total_duration_us":2913860},{"operation_name":"insert","last_dispatch_duration_us":2777815,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":2777815,"total_server_duration_us":1735182,"encode_duration_us":424,"operation_id":"0xdc6","timeout_ms":10000,"last_server_duration_us":1735182,"total_duration_us":2778036},{"operation_name":"insert","last_dispatch_duration_us":2652341,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":2652341,"total_server_duration_us":1771643,"encode_duration_us":289,"operation_id":"0xdff","timeout_ms":10000,"last_server_duration_us":1771643,"total_duration_us":2652678},{"operation_name":"insert","last_dispatch_duration_us":2609845,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":2609845,"total_server_duration_us":1765809,"encode_duration_us":287,"operation_id":"0xe0d","timeout_ms":10000,"last_server_duration_us":1765809,"total_duration_us":2610091},{"operation_name":"insert","last_dispatch_duration_us":2357243,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":2357243,"total_server_duration_us":1762101,"encode_duration_us":298,"operation_id":"0xe6d","timeout_ms":10000,"last_server_duration_us":1762101,"total_duration_us":2357464},{"operation_name":"insert","last_dispatch_duration_us":2279982,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":2279982,"total_server_duration_us":1579917,"encode_duration_us":295,"operation_id":"0xe43","timeout_ms":10000,"last_server_duration_us":1579917,"total_duration_us":2280235},{"operation_name":"insert","last_dispatch_duration_us":2171638,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":2171638,"total_server_duration_us":1840673,"encode_duration_us":270,"operation_id":"0xedf","timeout_ms":10000,"last_server_duration_us":1840673,"total_duration_us":2171866},{"operation_name":"insert","last_dispatch_duration_us":1857904,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":1857904,"total_server_duration_us":1014605,"encode_duration_us":366,"operation_id":"0xe0c","timeout_ms":10000,"last_server_duration_us":1014605,"total_duration_us":1858182},{"operation_name":"insert","last_dispatch_duration_us":1793047,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":1793047,"total_server_duration_us":1791328,"encode_duration_us":279,"operation_id":"0xf65","timeout_ms":10000,"last_server_duration_us":1791328,"total_duration_us":1793806}],"total_count":48}} WARN [2022-01-10 08:39:24,484] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":2956345,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":2956345,"total_server_duration_us":1815929,"encode_duration_us":541,"operation_id":"0xd92","timeout_ms":10000,"last_server_duration_us":1815929,"total_duration_us":2956601},{"operation_name":"insert","last_dispatch_duration_us":2777654,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":2777654,"total_server_duration_us":1638506,"encode_duration_us":307,"operation_id":"0xd93","timeout_ms":10000,"last_server_duration_us":1638506,"total_duration_us":2777931},{"operation_name":"insert","last_dispatch_duration_us":2236957,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":2236957,"total_server_duration_us":1631325,"encode_duration_us":310,"operation_id":"0xe55","timeout_ms":10000,"last_server_duration_us":1631325,"total_duration_us":2237194},{"operation_name":"insert","last_dispatch_duration_us":2182590,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":2182590,"total_server_duration_us":1802541,"encode_duration_us":293,"operation_id":"0xeb9","timeout_ms":10000,"last_server_duration_us":1802541,"total_duration_us":2182846},{"operation_name":"insert","last_dispatch_duration_us":2100907,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":2100907,"total_server_duration_us":1801472,"encode_duration_us":299,"operation_id":"0xed7","timeout_ms":10000,"last_server_duration_us":1801472,"total_duration_us":2101156},{"operation_name":"insert","last_dispatch_duration_us":2040299,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":2040299,"total_server_duration_us":1798800,"encode_duration_us":365,"operation_id":"0xeef","timeout_ms":10000,"last_server_duration_us":1798800,"total_duration_us":2040532},{"operation_name":"insert","last_dispatch_duration_us":2017010,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":2017010,"total_server_duration_us":795923,"encode_duration_us":564,"operation_id":"0xd7c","timeout_ms":10000,"last_server_duration_us":795923,"total_duration_us":2017302},{"operation_name":"insert","last_dispatch_duration_us":1973207,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":1973207,"total_server_duration_us":615040,"encode_duration_us":311,"operation_id":"0xd51","timeout_ms":10000,"last_server_duration_us":615040,"total_duration_us":1973460},{"operation_name":"insert","last_dispatch_duration_us":1963449,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":1963449,"total_server_duration_us":1671505,"encode_duration_us":290,"operation_id":"0xeda","timeout_ms":10000,"last_server_duration_us":1671505,"total_duration_us":1963706},{"operation_name":"insert","last_dispatch_duration_us":1948104,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":1948104,"total_server_duration_us":1248581,"encode_duration_us":405,"operation_id":"0xe2f","timeout_ms":10000,"last_server_duration_us":1248581,"total_duration_us":1948392}],"total_count":33}} INFO [2022-01-10 08:39:24,485] jepsen worker 556 - jepsen.util 556 :invoke :add 4141 INFO [2022-01-10 08:39:24,489] jepsen worker 256 - jepsen.util 256 :invoke :add 4142 INFO [2022-01-10 08:39:24,491] jepsen worker 162 - jepsen.util 162 :invoke :add 4143 INFO [2022-01-10 08:39:24,492] jepsen worker 809 - jepsen.util 809 :ok :add 4120 INFO [2022-01-10 08:39:24,492] jepsen worker 512 - jepsen.util 512 :ok :add 4138 INFO [2022-01-10 08:39:24,494] jepsen worker 485 - jepsen.util 485 :invoke :add 4144 INFO [2022-01-10 08:39:24,496] jepsen worker 186 - jepsen.util 186 :ok :add 4139 INFO [2022-01-10 08:39:24,497] jepsen worker 503 - jepsen.util 503 :invoke :add 4145 INFO [2022-01-10 08:39:24,500] jepsen worker 364 - jepsen.util 364 :invoke :add 4146 INFO [2022-01-10 08:39:24,503] jepsen worker 721 - jepsen.util 721 :ok :add 4108 INFO [2022-01-10 08:39:24,503] jepsen worker 817 - jepsen.util 817 :ok :add 4124 INFO [2022-01-10 08:39:24,503] jepsen worker 103 - jepsen.util 103 :ok :add 4098 INFO [2022-01-10 08:39:24,503] jepsen worker 985 - jepsen.util 985 :ok :add 4110 INFO [2022-01-10 08:39:24,503] jepsen worker 790 - jepsen.util 790 :ok :add 4080 INFO [2022-01-10 08:39:24,503] jepsen worker 981 - jepsen.util 981 :invoke :add 4147 INFO [2022-01-10 08:39:24,507] jepsen worker 79 - jepsen.util 79 :invoke :add 4148 INFO [2022-01-10 08:39:24,509] jepsen worker 734 - jepsen.util 734 :invoke :add 4149 INFO [2022-01-10 08:39:24,512] jepsen worker 50 - jepsen.util 50 :invoke :add 4150 INFO [2022-01-10 08:39:24,515] jepsen worker 224 - jepsen.util 224 :ok :add 4092 INFO [2022-01-10 08:39:24,515] jepsen worker 608 - jepsen.util 608 :ok :add 4102 INFO [2022-01-10 08:39:24,515] jepsen worker 569 - jepsen.util 569 :ok :add 4136 INFO [2022-01-10 08:39:24,515] jepsen worker 337 - jepsen.util 337 :invoke :add 4151 INFO [2022-01-10 08:39:24,518] jepsen worker 591 - jepsen.util 591 :invoke :add 4152 INFO [2022-01-10 08:39:24,520] jepsen worker 107 - jepsen.util 107 :invoke :add 4153 INFO [2022-01-10 08:39:24,522] jepsen worker 208 - jepsen.util 208 :invoke :add 4154 INFO [2022-01-10 08:39:24,524] jepsen worker 734 - jepsen.util 734 :ok :add 4149 INFO [2022-01-10 08:39:24,524] jepsen worker 337 - jepsen.util 337 :ok :add 4151 INFO [2022-01-10 08:39:24,525] jepsen worker 581 - jepsen.util 581 :ok :add 4041 INFO [2022-01-10 08:39:24,525] jepsen worker 637 - jepsen.util 637 :ok :add 4075 INFO [2022-01-10 08:39:24,525] jepsen worker 946 - jepsen.util 946 :ok :add 3894 INFO [2022-01-10 08:39:24,525] jepsen worker 603 - jepsen.util 603 :ok :add 3928 INFO [2022-01-10 08:39:24,525] jepsen worker 945 - jepsen.util 945 :ok :add 4059 INFO [2022-01-10 08:39:24,525] jepsen worker 913 - jepsen.util 913 :ok :add 3904 INFO [2022-01-10 08:39:24,525] jepsen worker 330 - jepsen.util 330 :ok :add 3930 INFO [2022-01-10 08:39:24,525] jepsen worker 196 - jepsen.util 196 :invoke :add 4155 INFO [2022-01-10 08:39:24,529] jepsen worker 400 - jepsen.util 400 :invoke :add 4156 INFO [2022-01-10 08:39:24,531] jepsen worker 79 - jepsen.util 79 :ok :add 4148 INFO [2022-01-10 08:39:24,531] jepsen worker 50 - jepsen.util 50 :ok :add 4150 INFO [2022-01-10 08:39:24,532] jepsen worker 597 - jepsen.util 597 :invoke :add 4157 INFO [2022-01-10 08:39:24,534] jepsen worker 88 - jepsen.util 88 :invoke :add 4158 INFO [2022-01-10 08:39:24,535] jepsen worker 304 - jepsen.util 304 :ok :add 4082 INFO [2022-01-10 08:39:24,536] jepsen worker 107 - jepsen.util 107 :ok :add 4153 INFO [2022-01-10 08:39:24,536] jepsen worker 591 - jepsen.util 591 :ok :add 4152 INFO [2022-01-10 08:39:24,537] jepsen worker 853 - jepsen.util 853 :ok :add 4112 INFO [2022-01-10 08:39:24,537] jepsen worker 105 - jepsen.util 105 :ok :add 4126 INFO [2022-01-10 08:39:24,537] jepsen worker 29 - jepsen.util 29 :invoke :add 4159 INFO [2022-01-10 08:39:24,540] jepsen worker 865 - jepsen.util 865 :invoke :add 4160 INFO [2022-01-10 08:39:24,543] jepsen worker 742 - jepsen.util 742 :invoke :add 4161 INFO [2022-01-10 08:39:24,546] jepsen worker 493 - jepsen.util 493 :invoke :add 4162 INFO [2022-01-10 08:39:24,549] jepsen worker 846 - jepsen.util 846 :invoke :add 4163 INFO [2022-01-10 08:39:24,551] jepsen worker 228 - jepsen.util 228 :invoke :add 4164 INFO [2022-01-10 08:39:24,554] jepsen worker 478 - jepsen.util 478 :invoke :add 4165 INFO [2022-01-10 08:39:24,557] jepsen worker 763 - jepsen.util 763 :invoke :add 4166 INFO [2022-01-10 08:39:24,559] jepsen worker 99 - jepsen.util 99 :invoke :add 4167 INFO [2022-01-10 08:39:24,559] jepsen worker 228 - jepsen.util 228 :ok :add 4164 INFO [2022-01-10 08:39:24,561] jepsen worker 220 - jepsen.util 220 :invoke :add 4168 INFO [2022-01-10 08:39:24,563] jepsen worker 410 - jepsen.util 410 :invoke :add 4169 INFO [2022-01-10 08:39:24,566] jepsen worker 861 - jepsen.util 861 :invoke :add 4170 INFO [2022-01-10 08:39:24,568] jepsen worker 478 - jepsen.util 478 :ok :add 4165 INFO [2022-01-10 08:39:24,568] jepsen worker 182 - jepsen.util 182 :invoke :add 4171 INFO [2022-01-10 08:39:24,569] jepsen worker 763 - jepsen.util 763 :ok :add 4166 INFO [2022-01-10 08:39:24,571] jepsen worker 278 - jepsen.util 278 :invoke :add 4172 INFO [2022-01-10 08:39:24,573] jepsen worker 721 - jepsen.util 721 :invoke :add 4173 INFO [2022-01-10 08:39:24,575] jepsen worker 99 - jepsen.util 99 :ok :add 4167 INFO [2022-01-10 08:39:24,576] jepsen worker 334 - jepsen.util 334 :invoke :add 4174 INFO [2022-01-10 08:39:24,579] jepsen worker 591 - jepsen.util 591 :invoke :add 4175 INFO [2022-01-10 08:39:24,581] jepsen worker 130 - jepsen.util 130 :invoke :add 4176 INFO [2022-01-10 08:39:24,582] jepsen worker 135 - jepsen.util 135 :ok :add 4078 INFO [2022-01-10 08:39:24,583] jepsen worker 653 - jepsen.util 653 :invoke :add 4177 INFO [2022-01-10 08:39:24,586] jepsen worker 974 - jepsen.util 974 :invoke :add 4178 INFO [2022-01-10 08:39:24,588] jepsen worker 291 - jepsen.util 291 :invoke :add 4179 INFO [2022-01-10 08:39:24,592] jepsen worker 519 - jepsen.util 519 :invoke :add 4180 INFO [2022-01-10 08:39:24,595] jepsen worker 781 - jepsen.util 781 :invoke :add 4181 INFO [2022-01-10 08:39:24,597] jepsen worker 442 - jepsen.util 442 :invoke :add 4182 INFO [2022-01-10 08:39:24,597] jepsen worker 547 - jepsen.util 547 :ok :add 4081 INFO [2022-01-10 08:39:24,598] jepsen worker 775 - jepsen.util 775 :ok :add 4111 INFO [2022-01-10 08:39:24,598] jepsen worker 709 - jepsen.util 709 :ok :add 4099 INFO [2022-01-10 08:39:24,598] jepsen worker 329 - jepsen.util 329 :ok :add 4109 INFO [2022-01-10 08:39:24,598] jepsen worker 276 - jepsen.util 276 :ok :add 4125 INFO [2022-01-10 08:39:24,599] jepsen worker 801 - jepsen.util 801 :ok :add 4089 INFO [2022-01-10 08:39:24,599] jepsen worker 895 - jepsen.util 895 :ok :add 4101 INFO [2022-01-10 08:39:24,599] jepsen worker 879 - jepsen.util 879 :ok :add 4091 INFO [2022-01-10 08:39:24,600] jepsen worker 901 - jepsen.util 901 :ok :add 4135 INFO [2022-01-10 08:39:24,600] jepsen worker 892 - jepsen.util 892 :ok :add 4119 INFO [2022-01-10 08:39:24,601] jepsen worker 31 - jepsen.util 31 :invoke :add 4183 INFO [2022-01-10 08:39:24,606] jepsen worker 61 - jepsen.util 61 :invoke :add 4184 INFO [2022-01-10 08:39:24,607] jepsen worker 804 - jepsen.util 804 :ok :add 4083 INFO [2022-01-10 08:39:24,607] jepsen worker 722 - jepsen.util 722 :ok :add 4127 INFO [2022-01-10 08:39:24,607] jepsen worker 424 - jepsen.util 424 :ok :add 4113 INFO [2022-01-10 08:39:24,609] jepsen worker 562 - jepsen.util 562 :invoke :add 4185 INFO [2022-01-10 08:39:24,614] jepsen worker 202 - jepsen.util 202 :invoke :add 4186 INFO [2022-01-10 08:39:24,614] jepsen worker 61 - jepsen.util 61 :ok :add 4184 INFO [2022-01-10 08:39:24,616] jepsen worker 291 - jepsen.util 291 :ok :add 4179 INFO [2022-01-10 08:39:24,616] jepsen worker 196 - jepsen.util 196 :ok :add 4155 INFO [2022-01-10 08:39:24,616] jepsen worker 742 - jepsen.util 742 :ok :add 4161 INFO [2022-01-10 08:39:24,617] jepsen worker 876 - jepsen.util 876 :invoke :add 4187 INFO [2022-01-10 08:39:24,617] jepsen worker 562 - jepsen.util 562 :ok :add 4185 INFO [2022-01-10 08:39:24,621] jepsen worker 657 - jepsen.util 657 :invoke :add 4188 INFO [2022-01-10 08:39:24,622] jepsen worker 597 - jepsen.util 597 :ok :add 4157 INFO [2022-01-10 08:39:24,623] jepsen worker 846 - jepsen.util 846 :ok :add 4163 INFO [2022-01-10 08:39:24,623] jepsen worker 966 - jepsen.util 966 :invoke :add 4189 INFO [2022-01-10 08:39:24,626] jepsen worker 958 - jepsen.util 958 :invoke :add 4190 INFO [2022-01-10 08:39:24,628] jepsen worker 273 - jepsen.util 273 :invoke :add 4191 INFO [2022-01-10 08:39:24,631] jepsen worker 159 - jepsen.util 159 :invoke :add 4192 INFO [2022-01-10 08:39:24,634] jepsen worker 519 - jepsen.util 519 :ok :add 4180 INFO [2022-01-10 08:39:24,634] jepsen worker 300 - jepsen.util 300 :invoke :add 4193 INFO [2022-01-10 08:39:24,637] jepsen worker 863 - jepsen.util 863 :invoke :add 4194 INFO [2022-01-10 08:39:24,637] jepsen worker 208 - jepsen.util 208 :ok :add 4154 INFO [2022-01-10 08:39:24,638] jepsen worker 865 - jepsen.util 865 :ok :add 4160 INFO [2022-01-10 08:39:24,638] jepsen worker 974 - jepsen.util 974 :ok :add 4178 INFO [2022-01-10 08:39:24,638] jepsen worker 113 - jepsen.util 113 :ok :add 4137 INFO [2022-01-10 08:39:24,638] jepsen worker 970 - jepsen.util 970 :ok :add 4103 INFO [2022-01-10 08:39:24,638] jepsen worker 764 - jepsen.util 764 :ok :add 4093 INFO [2022-01-10 08:39:24,640] jepsen worker 442 - jepsen.util 442 :ok :add 4182 INFO [2022-01-10 08:39:24,640] jepsen worker 771 - jepsen.util 771 :invoke :add 4195 INFO [2022-01-10 08:39:24,642] jepsen worker 493 - jepsen.util 493 :ok :add 4162 INFO [2022-01-10 08:39:24,642] jepsen worker 400 - jepsen.util 400 :ok :add 4156 INFO [2022-01-10 08:39:24,644] jepsen worker 97 - jepsen.util 97 :invoke :add 4196 INFO [2022-01-10 08:39:24,646] jepsen worker 851 - jepsen.util 851 :invoke :add 4197 INFO [2022-01-10 08:39:24,648] jepsen worker 165 - jepsen.util 165 :invoke :add 4198 INFO [2022-01-10 08:39:24,650] jepsen worker 781 - jepsen.util 781 :ok :add 4181 INFO [2022-01-10 08:39:24,651] jepsen worker 60 - jepsen.util 60 :invoke :add 4199 INFO [2022-01-10 08:39:24,654] jepsen worker 262 - jepsen.util 262 :invoke :add 4200 INFO [2022-01-10 08:39:24,656] jepsen worker 740 - jepsen.util 740 :invoke :add 4201 INFO [2022-01-10 08:39:24,658] jepsen worker 346 - jepsen.util 346 :invoke :add 4202 INFO [2022-01-10 08:39:24,660] jepsen worker 31 - jepsen.util 31 :ok :add 4183 INFO [2022-01-10 08:39:24,661] jepsen worker 570 - jepsen.util 570 :invoke :add 4203 INFO [2022-01-10 08:39:24,661] jepsen worker 202 - jepsen.util 202 :ok :add 4186 INFO [2022-01-10 08:39:24,664] jepsen worker 889 - jepsen.util 889 :invoke :add 4204 INFO [2022-01-10 08:39:24,666] jepsen worker 876 - jepsen.util 876 :ok :add 4187 INFO [2022-01-10 08:39:24,667] jepsen worker 482 - jepsen.util 482 :invoke :add 4205 INFO [2022-01-10 08:39:24,670] jepsen worker 165 - jepsen.util 165 :ok :add 4198 INFO [2022-01-10 08:39:24,671] jepsen worker 298 - jepsen.util 298 :invoke :add 4206 INFO [2022-01-10 08:39:24,674] jepsen worker 874 - jepsen.util 874 :invoke :add 4207 INFO [2022-01-10 08:39:24,676] jepsen worker 60 - jepsen.util 60 :ok :add 4199 INFO [2022-01-10 08:39:24,676] jepsen worker 314 - jepsen.util 314 :invoke :add 4208 INFO [2022-01-10 08:39:24,679] jepsen worker 354 - jepsen.util 354 :invoke :add 4209 INFO [2022-01-10 08:39:24,681] jepsen worker 506 - jepsen.util 506 :invoke :add 4210 INFO [2022-01-10 08:39:24,683] jepsen worker 942 - jepsen.util 942 :invoke :add 4211 INFO [2022-01-10 08:39:24,686] jepsen worker 164 - jepsen.util 164 :invoke :add 4212 INFO [2022-01-10 08:39:24,688] jepsen worker 774 - jepsen.util 774 :invoke :add 4213 INFO [2022-01-10 08:39:24,690] jepsen worker 276 - jepsen.util 276 :invoke :add 4214 INFO [2022-01-10 08:39:24,691] jepsen worker 314 - jepsen.util 314 :ok :add 4208 INFO [2022-01-10 08:39:24,692] jepsen worker 506 - jepsen.util 506 :ok :add 4210 INFO [2022-01-10 08:39:24,693] jepsen worker 330 - jepsen.util 330 :invoke :add 4215 INFO [2022-01-10 08:39:24,696] jepsen worker 115 - jepsen.util 115 :invoke :add 4216 INFO [2022-01-10 08:39:24,697] jepsen worker 164 - jepsen.util 164 :ok :add 4212 INFO [2022-01-10 08:39:24,699] jepsen worker 11 - jepsen.util 11 :invoke :add 4217 INFO [2022-01-10 08:39:24,700] jepsen worker 715 - jepsen.util 715 :ok :add 4134 INFO [2022-01-10 08:39:24,700] jepsen worker 942 - jepsen.util 942 :ok :add 4211 INFO [2022-01-10 08:39:24,700] jepsen worker 354 - jepsen.util 354 :ok :add 4209 INFO [2022-01-10 08:39:24,702] jepsen worker 220 - jepsen.util 220 :ok :add 4168 INFO [2022-01-10 08:39:24,702] jepsen worker 485 - jepsen.util 485 :ok :add 4144 INFO [2022-01-10 08:39:24,703] jepsen worker 774 - jepsen.util 774 :ok :add 4213 INFO [2022-01-10 08:39:24,703] jepsen worker 861 - jepsen.util 861 :ok :add 4170 INFO [2022-01-10 08:39:24,703] jepsen worker 994 - jepsen.util 994 :invoke :add 4218 INFO [2022-01-10 08:39:24,706] jepsen worker 922 - jepsen.util 922 :invoke :add 4219 INFO [2022-01-10 08:39:24,709] jepsen worker 411 - jepsen.util 411 :invoke :add 4220 INFO [2022-01-10 08:39:24,712] jepsen worker 382 - jepsen.util 382 :invoke :add 4221 INFO [2022-01-10 08:39:24,715] jepsen worker 819 - jepsen.util 819 :invoke :add 4222 INFO [2022-01-10 08:39:24,718] jepsen worker 633 - jepsen.util 633 :invoke :add 4223 INFO [2022-01-10 08:39:24,720] jepsen worker 777 - jepsen.util 777 :invoke :add 4224 INFO [2022-01-10 08:39:24,723] jepsen worker 16 - jepsen.util 16 :invoke :add 4225 INFO [2022-01-10 08:39:24,725] jepsen worker 67 - jepsen.util 67 :invoke :add 4226 INFO [2022-01-10 08:39:24,728] jepsen worker 426 - jepsen.util 426 :invoke :add 4227 INFO [2022-01-10 08:39:24,730] jepsen worker 702 - jepsen.util 702 :invoke :add 4228 INFO [2022-01-10 08:39:24,732] jepsen worker 632 - jepsen.util 632 :invoke :add 4229 INFO [2022-01-10 08:39:24,734] jepsen worker 109 - jepsen.util 109 :invoke :add 4230 INFO [2022-01-10 08:39:24,736] jepsen worker 494 - jepsen.util 494 :invoke :add 4231 INFO [2022-01-10 08:39:24,739] jepsen worker 761 - jepsen.util 761 :invoke :add 4232 INFO [2022-01-10 08:39:24,742] jepsen worker 691 - jepsen.util 691 :invoke :add 4233 INFO [2022-01-10 08:39:24,744] jepsen worker 503 - jepsen.util 503 :ok :add 4145 INFO [2022-01-10 08:39:24,744] jepsen worker 410 - jepsen.util 410 :ok :add 4169 INFO [2022-01-10 08:39:24,744] jepsen worker 182 - jepsen.util 182 :ok :add 4171 INFO [2022-01-10 08:39:24,745] jepsen worker 262 - jepsen.util 262 :ok :add 4200 INFO [2022-01-10 08:39:24,745] jepsen worker 536 - jepsen.util 536 :invoke :add 4234 INFO [2022-01-10 08:39:24,748] jepsen worker 674 - jepsen.util 674 :invoke :add 4235 INFO [2022-01-10 08:39:24,749] jepsen worker 426 - jepsen.util 426 :ok :add 4227 INFO [2022-01-10 08:39:24,750] jepsen worker 777 - jepsen.util 777 :ok :add 4224 INFO [2022-01-10 08:39:24,751] jepsen worker 983 - jepsen.util 983 :invoke :add 4236 INFO [2022-01-10 08:39:24,753] jepsen worker 454 - jepsen.util 454 :invoke :add 4237 INFO [2022-01-10 08:39:24,755] jepsen worker 251 - jepsen.util 251 :invoke :add 4238 INFO [2022-01-10 08:39:24,756] jepsen worker 67 - jepsen.util 67 :ok :add 4226 INFO [2022-01-10 08:39:24,758] jepsen worker 112 - jepsen.util 112 :invoke :add 4239 INFO [2022-01-10 08:39:24,760] jepsen worker 223 - jepsen.util 223 :ok :add 4106 INFO [2022-01-10 08:39:24,760] jepsen worker 989 - jepsen.util 989 :ok :add 4132 INFO [2022-01-10 08:39:24,760] jepsen worker 600 - jepsen.util 600 :ok :add 4096 INFO [2022-01-10 08:39:24,761] jepsen worker 224 - jepsen.util 224 :invoke :add 4240 INFO [2022-01-10 08:39:24,762] jepsen worker 16 - jepsen.util 16 :ok :add 4225 INFO [2022-01-10 08:39:24,764] jepsen worker 425 - jepsen.util 425 :invoke :add 4241 INFO [2022-01-10 08:39:24,767] jepsen worker 296 - jepsen.util 296 :invoke :add 4242 INFO [2022-01-10 08:39:24,768] jepsen worker 276 - jepsen.util 276 :ok :add 4214 INFO [2022-01-10 08:39:24,768] jepsen worker 411 - jepsen.util 411 :ok :add 4220 INFO [2022-01-10 08:39:24,769] jepsen worker 512 - jepsen.util 512 :invoke :add 4243 INFO [2022-01-10 08:39:24,773] jepsen worker 213 - jepsen.util 213 :invoke :add 4244 INFO [2022-01-10 08:39:24,775] jepsen worker 115 - jepsen.util 115 :ok :add 4216 INFO [2022-01-10 08:39:24,775] jepsen worker 330 - jepsen.util 330 :ok :add 4215 INFO [2022-01-10 08:39:24,776] jepsen worker 865 - jepsen.util 865 :invoke :add 4245 INFO [2022-01-10 08:39:24,776] jepsen worker 819 - jepsen.util 819 :ok :add 4222 INFO [2022-01-10 08:39:24,776] jepsen worker 112 - jepsen.util 112 :ok :add 4239 INFO [2022-01-10 08:39:24,777] jepsen worker 364 - jepsen.util 364 :ok :add 4146 INFO [2022-01-10 08:39:24,777] jepsen worker 278 - jepsen.util 278 :ok :add 4172 INFO [2022-01-10 08:39:24,777] jepsen worker 382 - jepsen.util 382 :ok :add 4221 INFO [2022-01-10 08:39:24,777] jepsen worker 346 - jepsen.util 346 :ok :add 4202 INFO [2022-01-10 08:39:24,777] jepsen worker 981 - jepsen.util 981 :ok :add 4147 INFO [2022-01-10 08:39:24,777] jepsen worker 721 - jepsen.util 721 :ok :add 4173 INFO [2022-01-10 08:39:24,777] jepsen worker 570 - jepsen.util 570 :ok :add 4203 INFO [2022-01-10 08:39:24,779] jepsen worker 917 - jepsen.util 917 :invoke :add 4246 INFO [2022-01-10 08:39:24,782] jepsen worker 11 - jepsen.util 11 :ok :add 4217 INFO [2022-01-10 08:39:24,782] jepsen worker 633 - jepsen.util 633 :ok :add 4223 INFO [2022-01-10 08:39:24,783] jepsen worker 813 - jepsen.util 813 :invoke :add 4247 INFO [2022-01-10 08:39:24,784] jepsen worker 251 - jepsen.util 251 :ok :add 4238 INFO [2022-01-10 08:39:24,786] jepsen worker 158 - jepsen.util 158 :invoke :add 4248 INFO [2022-01-10 08:39:24,789] jepsen worker 136 - jepsen.util 136 :invoke :add 4249 INFO [2022-01-10 08:39:24,791] jepsen worker 774 - jepsen.util 774 :invoke :add 4250 INFO [2022-01-10 08:39:24,794] jepsen worker 588 - jepsen.util 588 :invoke :add 4251 INFO [2022-01-10 08:39:24,796] jepsen worker 191 - jepsen.util 191 :invoke :add 4252 INFO [2022-01-10 08:39:24,799] jepsen worker 762 - jepsen.util 762 :invoke :add 4253 INFO [2022-01-10 08:39:24,801] jepsen worker 495 - jepsen.util 495 :invoke :add 4254 INFO [2022-01-10 08:39:24,804] jepsen worker 752 - jepsen.util 752 :invoke :add 4255 INFO [2022-01-10 08:39:24,804] jepsen worker 774 - jepsen.util 774 :ok :add 4250 INFO [2022-01-10 08:39:24,804] jepsen worker 158 - jepsen.util 158 :ok :add 4248 INFO [2022-01-10 08:39:24,807] jepsen worker 952 - jepsen.util 952 :invoke :add 4256 INFO [2022-01-10 08:39:24,810] jepsen worker 386 - jepsen.util 386 :invoke :add 4257 INFO [2022-01-10 08:39:24,812] jepsen worker 495 - jepsen.util 495 :ok :add 4254 INFO [2022-01-10 08:39:24,813] jepsen worker 576 - jepsen.util 576 :invoke :add 4258 INFO [2022-01-10 08:39:24,813] jepsen worker 191 - jepsen.util 191 :ok :add 4252 INFO [2022-01-10 08:39:24,813] jepsen worker 752 - jepsen.util 752 :ok :add 4255 INFO [2022-01-10 08:39:24,816] jepsen worker 61 - jepsen.util 61 :invoke :add 4259 INFO [2022-01-10 08:39:24,817] jepsen worker 952 - jepsen.util 952 :ok :add 4256 INFO [2022-01-10 08:39:24,818] jepsen worker 386 - jepsen.util 386 :ok :add 4257 INFO [2022-01-10 08:39:24,819] jepsen worker 839 - jepsen.util 839 :invoke :add 4260 INFO [2022-01-10 08:39:24,822] jepsen worker 360 - jepsen.util 360 :invoke :add 4261 INFO [2022-01-10 08:39:24,823] jepsen worker 588 - jepsen.util 588 :ok :add 4251 INFO [2022-01-10 08:39:24,823] jepsen worker 136 - jepsen.util 136 :ok :add 4249 INFO [2022-01-10 08:39:24,824] jepsen worker 444 - jepsen.util 444 :invoke :add 4262 INFO [2022-01-10 08:39:24,826] jepsen worker 762 - jepsen.util 762 :ok :add 4253 INFO [2022-01-10 08:39:24,827] jepsen worker 834 - jepsen.util 834 :invoke :add 4263 INFO [2022-01-10 08:39:24,830] jepsen worker 797 - jepsen.util 797 :invoke :add 4264 INFO [2022-01-10 08:39:24,832] jepsen worker 839 - jepsen.util 839 :ok :add 4260 INFO [2022-01-10 08:39:24,833] jepsen worker 412 - jepsen.util 412 :invoke :add 4265 INFO [2022-01-10 08:39:24,837] jepsen worker 360 - jepsen.util 360 :ok :add 4261 INFO [2022-01-10 08:39:24,837] jepsen worker 444 - jepsen.util 444 :ok :add 4262 INFO [2022-01-10 08:39:24,838] jepsen worker 191 - jepsen.util 191 :invoke :add 4266 INFO [2022-01-10 08:39:24,841] jepsen worker 551 - jepsen.util 551 :invoke :add 4267 INFO [2022-01-10 08:39:24,842] jepsen worker 834 - jepsen.util 834 :ok :add 4263 INFO [2022-01-10 08:39:24,843] jepsen worker 797 - jepsen.util 797 :ok :add 4264 INFO [2022-01-10 08:39:24,844] jepsen worker 428 - jepsen.util 428 :invoke :add 4268 INFO [2022-01-10 08:39:24,847] jepsen worker 392 - jepsen.util 392 :invoke :add 4269 INFO [2022-01-10 08:39:24,850] jepsen worker 597 - jepsen.util 597 :invoke :add 4270 INFO [2022-01-10 08:39:24,850] jepsen worker 412 - jepsen.util 412 :ok :add 4265 INFO [2022-01-10 08:39:24,852] jepsen worker 467 - jepsen.util 467 :invoke :add 4271 INFO [2022-01-10 08:39:24,855] jepsen worker 711 - jepsen.util 711 :invoke :add 4272 INFO [2022-01-10 08:39:24,857] jepsen worker 836 - jepsen.util 836 :invoke :add 4273 INFO [2022-01-10 08:39:24,860] jepsen worker 505 - jepsen.util 505 :invoke :add 4274 INFO [2022-01-10 08:39:24,863] jepsen worker 450 - jepsen.util 450 :invoke :add 4275 INFO [2022-01-10 08:39:24,866] jepsen worker 777 - jepsen.util 777 :invoke :add 4276 INFO [2022-01-10 08:39:24,870] jepsen worker 663 - jepsen.util 663 :invoke :add 4277 INFO [2022-01-10 08:39:24,873] jepsen worker 118 - jepsen.util 118 :invoke :add 4278 INFO [2022-01-10 08:39:24,876] jepsen worker 64 - jepsen.util 64 :invoke :add 4279 INFO [2022-01-10 08:39:24,878] jepsen worker 218 - jepsen.util 218 :invoke :add 4280 INFO [2022-01-10 08:39:24,880] jepsen worker 498 - jepsen.util 498 :invoke :add 4281 INFO [2022-01-10 08:39:24,881] jepsen worker 118 - jepsen.util 118 :ok :add 4278 INFO [2022-01-10 08:39:24,883] jepsen worker 151 - jepsen.util 151 :invoke :add 4282 INFO [2022-01-10 08:39:24,884] jepsen worker 64 - jepsen.util 64 :ok :add 4279 INFO [2022-01-10 08:39:24,886] jepsen worker 347 - jepsen.util 347 :invoke :add 4283 INFO [2022-01-10 08:39:24,889] jepsen worker 818 - jepsen.util 818 :invoke :add 4284 INFO [2022-01-10 08:39:24,892] jepsen worker 18 - jepsen.util 18 :invoke :add 4285 INFO [2022-01-10 08:39:24,895] jepsen worker 852 - jepsen.util 852 :invoke :add 4286 INFO [2022-01-10 08:39:24,895] jepsen worker 792 - jepsen.util 792 :ok :add 4007 INFO [2022-01-10 08:39:24,895] jepsen worker 851 - jepsen.util 851 :ok :add 4197 INFO [2022-01-10 08:39:24,896] jepsen worker 226 - jepsen.util 226 :ok :add 3976 INFO [2022-01-10 08:39:24,896] jepsen worker 218 - jepsen.util 218 :ok :add 4280 INFO [2022-01-10 08:39:24,896] jepsen worker 580 - jepsen.util 580 :ok :add 4033 INFO [2022-01-10 08:39:24,896] jepsen worker 498 - jepsen.util 498 :ok :add 4281 INFO [2022-01-10 08:39:24,897] jepsen worker 370 - jepsen.util 370 :invoke :add 4287 INFO [2022-01-10 08:39:24,900] jepsen worker 366 - jepsen.util 366 :invoke :add 4288 INFO [2022-01-10 08:39:24,903] jepsen worker 402 - jepsen.util 402 :invoke :add 4289 INFO [2022-01-10 08:39:24,906] jepsen worker 495 - jepsen.util 495 :invoke :add 4290 INFO [2022-01-10 08:39:24,907] jepsen worker 347 - jepsen.util 347 :ok :add 4283 INFO [2022-01-10 08:39:24,908] jepsen worker 191 - jepsen.util 191 :ok :add 4266 INFO [2022-01-10 08:39:24,908] jepsen worker 630 - jepsen.util 630 :invoke :add 4291 INFO [2022-01-10 08:39:24,909] jepsen worker 151 - jepsen.util 151 :ok :add 4282 INFO [2022-01-10 08:39:24,911] jepsen worker 551 - jepsen.util 551 :ok :add 4267 INFO [2022-01-10 08:39:24,911] jepsen worker 625 - jepsen.util 625 :invoke :add 4292 INFO [2022-01-10 08:39:24,914] jepsen worker 289 - jepsen.util 289 :invoke :add 4293 INFO [2022-01-10 08:39:24,916] jepsen worker 156 - jepsen.util 156 :invoke :add 4294 INFO [2022-01-10 08:39:24,918] jepsen worker 555 - jepsen.util 555 :invoke :add 4295 INFO [2022-01-10 08:39:24,922] jepsen worker 18 - jepsen.util 18 :ok :add 4285 INFO [2022-01-10 08:39:24,922] jepsen worker 151 - jepsen.util 151 :invoke :add 4296 INFO [2022-01-10 08:39:24,925] jepsen worker 623 - jepsen.util 623 :invoke :add 4297 INFO [2022-01-10 08:39:24,926] jepsen worker 370 - jepsen.util 370 :ok :add 4287 INFO [2022-01-10 08:39:24,927] jepsen worker 113 - jepsen.util 113 :invoke :add 4298 INFO [2022-01-10 08:39:24,929] jepsen worker 379 - jepsen.util 379 :invoke :add 4299 INFO [2022-01-10 08:39:24,932] jepsen worker 10 - jepsen.util 10 :invoke :add 4300 INFO [2022-01-10 08:39:24,934] jepsen worker 641 - jepsen.util 641 :invoke :add 4301 INFO [2022-01-10 08:39:24,935] jepsen worker 818 - jepsen.util 818 :ok :add 4284 INFO [2022-01-10 08:39:24,935] jepsen worker 113 - jepsen.util 113 :ok :add 4298 INFO [2022-01-10 08:39:24,936] jepsen worker 785 - jepsen.util 785 :invoke :add 4302 INFO [2022-01-10 08:39:24,938] jepsen worker 379 - jepsen.util 379 :ok :add 4299 INFO [2022-01-10 08:39:24,939] jepsen worker 231 - jepsen.util 231 :invoke :add 4303 INFO [2022-01-10 08:39:24,940] jepsen worker 852 - jepsen.util 852 :ok :add 4286 INFO [2022-01-10 08:39:24,942] jepsen worker 295 - jepsen.util 295 :invoke :add 4304 INFO [2022-01-10 08:39:24,944] jepsen worker 540 - jepsen.util 540 :invoke :add 4305 INFO [2022-01-10 08:39:24,946] jepsen worker 737 - jepsen.util 737 :invoke :add 4306 INFO [2022-01-10 08:39:24,949] jepsen worker 363 - jepsen.util 363 :invoke :add 4307 INFO [2022-01-10 08:39:24,951] jepsen worker 433 - jepsen.util 433 :invoke :add 4308 INFO [2022-01-10 08:39:24,953] jepsen worker 465 - jepsen.util 465 :invoke :add 4309 INFO [2022-01-10 08:39:24,955] jepsen worker 241 - jepsen.util 241 :invoke :add 4310 INFO [2022-01-10 08:39:24,957] jepsen worker 654 - jepsen.util 654 :invoke :add 4311 INFO [2022-01-10 08:39:24,959] jepsen worker 739 - jepsen.util 739 :invoke :add 4312 INFO [2022-01-10 08:39:24,962] jepsen worker 868 - jepsen.util 868 :invoke :add 4313 INFO [2022-01-10 08:39:24,964] jepsen worker 456 - jepsen.util 456 :invoke :add 4314 INFO [2022-01-10 08:39:24,967] jepsen worker 947 - jepsen.util 947 :invoke :add 4315 INFO [2022-01-10 08:39:24,967] jepsen worker 433 - jepsen.util 433 :ok :add 4308 INFO [2022-01-10 08:39:24,968] jepsen worker 213 - jepsen.util 213 :ok :add 4244 INFO [2022-01-10 08:39:24,968] jepsen worker 428 - jepsen.util 428 :ok :add 4268 INFO [2022-01-10 08:39:24,968] jepsen worker 597 - jepsen.util 597 :ok :add 4270 INFO [2022-01-10 08:39:24,969] jepsen worker 861 - jepsen.util 861 :invoke :add 4316 INFO [2022-01-10 08:39:24,971] jepsen worker 456 - jepsen.util 456 :ok :add 4314 INFO [2022-01-10 08:39:24,972] jepsen worker 862 - jepsen.util 862 :invoke :add 4317 INFO [2022-01-10 08:39:24,974] jepsen worker 465 - jepsen.util 465 :ok :add 4309 INFO [2022-01-10 08:39:24,975] jepsen worker 241 - jepsen.util 241 :ok :add 4310 INFO [2022-01-10 08:39:24,975] jepsen worker 654 - jepsen.util 654 :ok :add 4311 INFO [2022-01-10 08:39:24,975] jepsen worker 947 - jepsen.util 947 :ok :add 4315 INFO [2022-01-10 08:39:24,976] jepsen worker 93 - jepsen.util 93 :invoke :add 4318 INFO [2022-01-10 08:39:24,979] jepsen worker 862 - jepsen.util 862 :ok :add 4317 INFO [2022-01-10 08:39:24,979] jepsen worker 95 - jepsen.util 95 :invoke :add 4319 INFO [2022-01-10 08:39:24,982] jepsen worker 549 - jepsen.util 549 :invoke :add 4320 INFO [2022-01-10 08:39:24,982] jepsen worker 739 - jepsen.util 739 :ok :add 4312 INFO [2022-01-10 08:39:24,984] jepsen worker 868 - jepsen.util 868 :ok :add 4313 INFO [2022-01-10 08:39:24,984] jepsen worker 28 - jepsen.util 28 :invoke :add 4321 INFO [2022-01-10 08:39:24,987] jepsen worker 100 - jepsen.util 100 :invoke :add 4322 INFO [2022-01-10 08:39:24,988] jepsen worker 861 - jepsen.util 861 :ok :add 4316 INFO [2022-01-10 08:39:24,989] jepsen worker 260 - jepsen.util 260 :invoke :add 4323 INFO [2022-01-10 08:39:24,992] jepsen worker 750 - jepsen.util 750 :invoke :add 4324 INFO [2022-01-10 08:39:24,993] jepsen worker 28 - jepsen.util 28 :ok :add 4321 INFO [2022-01-10 08:39:24,994] jepsen worker 834 - jepsen.util 834 :invoke :add 4325 INFO [2022-01-10 08:39:24,996] jepsen worker 719 - jepsen.util 719 :invoke :add 4326 INFO [2022-01-10 08:39:24,997] jepsen worker 260 - jepsen.util 260 :ok :add 4323 INFO [2022-01-10 08:39:24,999] jepsen worker 316 - jepsen.util 316 :invoke :add 4327 INFO [2022-01-10 08:39:24,999] jepsen worker 549 - jepsen.util 549 :ok :add 4320 INFO [2022-01-10 08:39:25,001] jepsen worker 117 - jepsen.util 117 :invoke :add 4328 INFO [2022-01-10 08:39:25,004] jepsen worker 100 - jepsen.util 100 :ok :add 4322 INFO [2022-01-10 08:39:25,004] jepsen worker 409 - jepsen.util 409 :invoke :add 4329 INFO [2022-01-10 08:39:25,005] jepsen worker 657 - jepsen.util 657 :ok :add 4188 INFO [2022-01-10 08:39:25,006] jepsen worker 175 - jepsen.util 175 :invoke :add 4330 INFO [2022-01-10 08:39:25,006] jepsen worker 958 - jepsen.util 958 :ok :add 4190 INFO [2022-01-10 08:39:25,008] jepsen worker 750 - jepsen.util 750 :ok :add 4324 INFO [2022-01-10 08:39:25,008] jepsen worker 792 - jepsen.util 792 :invoke :add 4331 INFO [2022-01-10 08:39:25,011] jepsen worker 754 - jepsen.util 754 :invoke :add 4332 INFO [2022-01-10 08:39:25,014] jepsen worker 60 - jepsen.util 60 :invoke :add 4333 INFO [2022-01-10 08:39:25,014] jepsen worker 719 - jepsen.util 719 :ok :add 4326 INFO [2022-01-10 08:39:25,016] jepsen worker 316 - jepsen.util 316 :ok :add 4327 INFO [2022-01-10 08:39:25,017] jepsen worker 908 - jepsen.util 908 :invoke :add 4334 INFO [2022-01-10 08:39:25,020] jepsen worker 246 - jepsen.util 246 :invoke :add 4335 INFO [2022-01-10 08:39:25,022] jepsen worker 998 - jepsen.util 998 :invoke :add 4336 INFO [2022-01-10 08:39:25,024] jepsen worker 188 - jepsen.util 188 :invoke :add 4337 INFO [2022-01-10 08:39:25,026] jepsen worker 834 - jepsen.util 834 :ok :add 4325 INFO [2022-01-10 08:39:25,028] jepsen worker 242 - jepsen.util 242 :invoke :add 4338 INFO [2022-01-10 08:39:25,030] jepsen worker 814 - jepsen.util 814 :invoke :add 4339 INFO [2022-01-10 08:39:25,032] jepsen worker 901 - jepsen.util 901 :invoke :add 4340 INFO [2022-01-10 08:39:25,035] jepsen worker 267 - jepsen.util 267 :invoke :add 4341 INFO [2022-01-10 08:39:25,037] jepsen worker 242 - jepsen.util 242 :ok :add 4338 INFO [2022-01-10 08:39:25,037] jepsen worker 814 - jepsen.util 814 :ok :add 4339 INFO [2022-01-10 08:39:25,038] jepsen worker 808 - jepsen.util 808 :invoke :add 4342 INFO [2022-01-10 08:39:25,040] jepsen worker 373 - jepsen.util 373 :invoke :add 4343 INFO [2022-01-10 08:39:25,042] jepsen worker 572 - jepsen.util 572 :invoke :add 4344 INFO [2022-01-10 08:39:25,044] jepsen worker 71 - jepsen.util 71 :invoke :add 4345 INFO [2022-01-10 08:39:25,047] jepsen worker 839 - jepsen.util 839 :invoke :add 4346 INFO [2022-01-10 08:39:25,049] jepsen worker 935 - jepsen.util 935 :invoke :add 4347 INFO [2022-01-10 08:39:25,052] jepsen worker 932 - jepsen.util 932 :invoke :add 4348 INFO [2022-01-10 08:39:25,055] jepsen worker 660 - jepsen.util 660 :invoke :add 4349 INFO [2022-01-10 08:39:25,056] jepsen worker 159 - jepsen.util 159 :ok :add 4192 INFO [2022-01-10 08:39:25,057] jepsen worker 273 - jepsen.util 273 :ok :add 4191 INFO [2022-01-10 08:39:25,057] jepsen worker 966 - jepsen.util 966 :ok :add 4189 INFO [2022-01-10 08:39:25,059] jepsen worker 932 - jepsen.util 932 :ok :add 4348 INFO [2022-01-10 08:39:25,059] jepsen worker 114 - jepsen.util 114 :invoke :add 4350 INFO [2022-01-10 08:39:25,060] jepsen worker 300 - jepsen.util 300 :ok :add 4193 INFO [2022-01-10 08:39:25,060] jepsen worker 839 - jepsen.util 839 :ok :add 4346 INFO [2022-01-10 08:39:25,061] jepsen worker 101 - jepsen.util 101 :invoke :add 4351 INFO [2022-01-10 08:39:25,064] jepsen worker 525 - jepsen.util 525 :invoke :add 4352 INFO [2022-01-10 08:39:25,064] jepsen worker 660 - jepsen.util 660 :ok :add 4349 INFO [2022-01-10 08:39:25,067] jepsen worker 787 - jepsen.util 787 :invoke :add 4353 INFO [2022-01-10 08:39:25,070] jepsen worker 431 - jepsen.util 431 :invoke :add 4354 INFO [2022-01-10 08:39:25,071] jepsen worker 101 - jepsen.util 101 :ok :add 4351 INFO [2022-01-10 08:39:25,072] jepsen worker 759 - jepsen.util 759 :invoke :add 4355 INFO [2022-01-10 08:39:25,074] jepsen worker 445 - jepsen.util 445 :invoke :add 4356 INFO [2022-01-10 08:39:25,076] jepsen worker 787 - jepsen.util 787 :ok :add 4353 INFO [2022-01-10 08:39:25,076] jepsen worker 638 - jepsen.util 638 :invoke :add 4357 INFO [2022-01-10 08:39:25,076] jepsen worker 114 - jepsen.util 114 :ok :add 4350 INFO [2022-01-10 08:39:25,078] jepsen worker 729 - jepsen.util 729 :invoke :add 4358 INFO [2022-01-10 08:39:25,081] jepsen worker 525 - jepsen.util 525 :ok :add 4352 INFO [2022-01-10 08:39:25,081] jepsen worker 99 - jepsen.util 99 :invoke :add 4359 INFO [2022-01-10 08:39:25,084] jepsen worker 635 - jepsen.util 635 :invoke :add 4360 INFO [2022-01-10 08:39:25,087] jepsen worker 129 - jepsen.util 129 :invoke :add 4361 INFO [2022-01-10 08:39:25,089] jepsen worker 332 - jepsen.util 332 :invoke :add 4362 INFO [2022-01-10 08:39:25,091] jepsen worker 526 - jepsen.util 526 :invoke :add 4363 INFO [2022-01-10 08:39:25,093] jepsen worker 821 - jepsen.util 821 :invoke :add 4364 INFO [2022-01-10 08:39:25,095] jepsen worker 137 - jepsen.util 137 :invoke :add 4365 INFO [2022-01-10 08:39:25,097] jepsen worker 827 - jepsen.util 827 :invoke :add 4366 INFO [2022-01-10 08:39:25,099] jepsen worker 59 - jepsen.util 59 :invoke :add 4367 INFO [2022-01-10 08:39:25,100] jepsen worker 344 - jepsen.util 344 :ok :add 4005 INFO [2022-01-10 08:39:25,101] jepsen worker 51 - jepsen.util 51 :ok :add 4029 INFO [2022-01-10 08:39:25,101] jepsen worker 757 - jepsen.util 757 :ok :add 4031 INFO [2022-01-10 08:39:25,101] jepsen worker 771 - jepsen.util 771 :ok :add 4195 INFO [2022-01-10 08:39:25,102] jepsen worker 610 - jepsen.util 610 :invoke :add 4368 INFO [2022-01-10 08:39:25,103] jepsen worker 70 - jepsen.util 70 :ok :add 4094 INFO [2022-01-10 08:39:25,103] jepsen worker 800 - jepsen.util 800 :ok :add 4104 INFO [2022-01-10 08:39:25,103] jepsen worker 61 - jepsen.util 61 :ok :add 4259 INFO [2022-01-10 08:39:25,103] jepsen worker 425 - jepsen.util 425 :ok :add 4241 INFO [2022-01-10 08:39:25,103] jepsen worker 779 - jepsen.util 779 :ok :add 4128 INFO [2022-01-10 08:39:25,103] jepsen worker 450 - jepsen.util 450 :ok :add 4275 INFO [2022-01-10 08:39:25,104] jepsen worker 806 - jepsen.util 806 :ok :add 4130 INFO [2022-01-10 08:39:25,105] jepsen worker 431 - jepsen.util 431 :ok :add 4354 INFO [2022-01-10 08:39:25,106] jepsen worker 137 - jepsen.util 137 :ok :add 4365 INFO [2022-01-10 08:39:25,109] jepsen worker 149 - jepsen.util 149 :invoke :add 4369 INFO [2022-01-10 08:39:25,111] jepsen worker 821 - jepsen.util 821 :ok :add 4364 INFO [2022-01-10 08:39:25,112] jepsen worker 59 - jepsen.util 59 :ok :add 4367 INFO [2022-01-10 08:39:25,112] jepsen worker 445 - jepsen.util 445 :ok :add 4356 INFO [2022-01-10 08:39:25,112] jepsen worker 332 - jepsen.util 332 :ok :add 4362 INFO [2022-01-10 08:39:25,112] jepsen worker 485 - jepsen.util 485 :invoke :add 4370 INFO [2022-01-10 08:39:25,116] jepsen worker 827 - jepsen.util 827 :ok :add 4366 INFO [2022-01-10 08:39:25,117] jepsen worker 564 - jepsen.util 564 :invoke :add 4371 INFO [2022-01-10 08:39:25,117] jepsen worker 635 - jepsen.util 635 :ok :add 4360 INFO [2022-01-10 08:39:25,121] jepsen worker 918 - jepsen.util 918 :invoke :add 4372 INFO [2022-01-10 08:39:25,124] jepsen worker 759 - jepsen.util 759 :ok :add 4355 INFO [2022-01-10 08:39:25,124] jepsen worker 129 - jepsen.util 129 :ok :add 4361 INFO [2022-01-10 08:39:25,125] jepsen worker 731 - jepsen.util 731 :invoke :add 4373 INFO [2022-01-10 08:39:25,128] jepsen worker 476 - jepsen.util 476 :invoke :add 4374 INFO [2022-01-10 08:39:25,130] jepsen worker 638 - jepsen.util 638 :ok :add 4357 INFO [2022-01-10 08:39:25,130] jepsen worker 526 - jepsen.util 526 :ok :add 4363 INFO [2022-01-10 08:39:25,131] jepsen worker 786 - jepsen.util 786 :invoke :add 4375 INFO [2022-01-10 08:39:25,133] jepsen worker 917 - jepsen.util 917 :ok :add 4246 INFO [2022-01-10 08:39:25,133] jepsen worker 711 - jepsen.util 711 :ok :add 4272 INFO [2022-01-10 08:39:25,135] jepsen worker 988 - jepsen.util 988 :invoke :add 4376 INFO [2022-01-10 08:39:25,139] jepsen worker 657 - jepsen.util 657 :invoke :add 4377 INFO [2022-01-10 08:39:25,142] jepsen worker 694 - jepsen.util 694 :invoke :add 4378 INFO [2022-01-10 08:39:25,144] jepsen worker 708 - jepsen.util 708 :invoke :add 4379 INFO [2022-01-10 08:39:25,146] jepsen worker 67 - jepsen.util 67 :invoke :add 4380 INFO [2022-01-10 08:39:25,149] jepsen worker 349 - jepsen.util 349 :invoke :add 4381 INFO [2022-01-10 08:39:25,151] jepsen worker 616 - jepsen.util 616 :invoke :add 4382 INFO [2022-01-10 08:39:25,154] jepsen worker 154 - jepsen.util 154 :invoke :add 4383 INFO [2022-01-10 08:39:25,155] jepsen worker 694 - jepsen.util 694 :ok :add 4378 INFO [2022-01-10 08:39:25,157] jepsen worker 981 - jepsen.util 981 :invoke :add 4384 INFO [2022-01-10 08:39:25,158] jepsen worker 708 - jepsen.util 708 :ok :add 4379 INFO [2022-01-10 08:39:25,160] jepsen worker 511 - jepsen.util 511 :invoke :add 4385 INFO [2022-01-10 08:39:25,161] jepsen worker 138 - jepsen.util 138 :ok :add 4043 INFO [2022-01-10 08:39:25,161] jepsen worker 737 - jepsen.util 737 :ok :add 4306 INFO [2022-01-10 08:39:25,161] jepsen worker 754 - jepsen.util 754 :ok :add 4332 INFO [2022-01-10 08:39:25,161] jepsen worker 619 - jepsen.util 619 :ok :add 4077 INFO [2022-01-10 08:39:25,161] jepsen worker 151 - jepsen.util 151 :ok :add 4296 INFO [2022-01-10 08:39:25,164] jepsen worker 466 - jepsen.util 466 :invoke :add 4386 INFO [2022-01-10 08:39:25,166] jepsen worker 349 - jepsen.util 349 :ok :add 4381 INFO [2022-01-10 08:39:25,168] jepsen worker 67 - jepsen.util 67 :ok :add 4380 INFO [2022-01-10 08:39:25,169] jepsen worker 378 - jepsen.util 378 :invoke :add 4387 INFO [2022-01-10 08:39:25,171] jepsen worker 154 - jepsen.util 154 :ok :add 4383 INFO [2022-01-10 08:39:25,172] jepsen worker 616 - jepsen.util 616 :ok :add 4382 INFO [2022-01-10 08:39:25,172] jepsen worker 54 - jepsen.util 54 :invoke :add 4388 INFO [2022-01-10 08:39:25,176] jepsen worker 198 - jepsen.util 198 :invoke :add 4389 INFO [2022-01-10 08:39:25,177] jepsen worker 232 - jepsen.util 232 :invoke :add 4390 INFO [2022-01-10 08:39:25,179] jepsen worker 865 - jepsen.util 865 :ok :add 4245 INFO [2022-01-10 08:39:25,179] jepsen worker 467 - jepsen.util 467 :ok :add 4271 INFO [2022-01-10 08:39:25,179] jepsen worker 392 - jepsen.util 392 :ok :add 4269 INFO [2022-01-10 08:39:25,180] jepsen worker 411 - jepsen.util 411 :invoke :add 4391 INFO [2022-01-10 08:39:25,182] jepsen worker 319 - jepsen.util 319 :invoke :add 4392 INFO [2022-01-10 08:39:25,185] jepsen worker 250 - jepsen.util 250 :invoke :add 4393 INFO [2022-01-10 08:39:25,187] jepsen worker 328 - jepsen.util 328 :invoke :add 4394 INFO [2022-01-10 08:39:25,190] jepsen worker 975 - jepsen.util 975 :invoke :add 4395 INFO [2022-01-10 08:39:25,191] jepsen worker 981 - jepsen.util 981 :ok :add 4384 INFO [2022-01-10 08:39:25,194] jepsen worker 511 - jepsen.util 511 :ok :add 4385 INFO [2022-01-10 08:39:25,194] jepsen worker 73 - jepsen.util 73 :invoke :add 4396 INFO [2022-01-10 08:39:25,196] jepsen worker 415 - jepsen.util 415 :ok :add 4040 INFO [2022-01-10 08:39:25,196] jepsen worker 393 - jepsen.util 393 :ok :add 4058 INFO [2022-01-10 08:39:25,196] jepsen worker 555 - jepsen.util 555 :ok :add 4295 INFO [2022-01-10 08:39:25,196] jepsen worker 377 - jepsen.util 377 :ok :add 4074 INFO [2022-01-10 08:39:25,196] jepsen worker 466 - jepsen.util 466 :ok :add 4386 INFO [2022-01-10 08:39:25,196] jepsen worker 792 - jepsen.util 792 :ok :add 4331 INFO [2022-01-10 08:39:25,196] jepsen worker 409 - jepsen.util 409 :ok :add 4329 INFO [2022-01-10 08:39:25,196] jepsen worker 540 - jepsen.util 540 :ok :add 4305 INFO [2022-01-10 08:39:25,197] jepsen worker 471 - jepsen.util 471 :invoke :add 4397 INFO [2022-01-10 08:39:25,201] jepsen worker 45 - jepsen.util 45 :invoke :add 4398 INFO [2022-01-10 08:39:25,201] jepsen worker 378 - jepsen.util 378 :ok :add 4387 INFO [2022-01-10 08:39:25,203] jepsen worker 81 - jepsen.util 81 :invoke :add 4399 INFO [2022-01-10 08:39:25,204] jepsen worker 922 - jepsen.util 922 :ok :add 4219 INFO [2022-01-10 08:39:25,204] jepsen worker 674 - jepsen.util 674 :ok :add 4235 INFO [2022-01-10 08:39:25,204] jepsen worker 198 - jepsen.util 198 :ok :add 4389 INFO [2022-01-10 08:39:25,204] jepsen worker 740 - jepsen.util 740 :ok :add 4201 INFO [2022-01-10 08:39:25,206] jepsen worker 232 - jepsen.util 232 :ok :add 4390 INFO [2022-01-10 08:39:25,206] jepsen worker 54 - jepsen.util 54 :ok :add 4388 INFO [2022-01-10 08:39:25,206] jepsen worker 994 - jepsen.util 994 :ok :add 4218 INFO [2022-01-10 08:39:25,206] jepsen worker 536 - jepsen.util 536 :ok :add 4234 INFO [2022-01-10 08:39:25,206] jepsen worker 885 - jepsen.util 885 :invoke :add 4400 INFO [2022-01-10 08:39:25,211] jepsen worker 699 - jepsen.util 699 :invoke :add 4401 INFO [2022-01-10 08:39:25,211] jepsen worker 45 - jepsen.util 45 :ok :add 4398 INFO [2022-01-10 08:39:25,213] jepsen worker 275 - jepsen.util 275 :invoke :add 4402 INFO [2022-01-10 08:39:25,216] jepsen worker 81 - jepsen.util 81 :ok :add 4399 INFO [2022-01-10 08:39:25,216] jepsen worker 506 - jepsen.util 506 :invoke :add 4403 INFO [2022-01-10 08:39:25,220] jepsen worker 763 - jepsen.util 763 :invoke :add 4404 INFO [2022-01-10 08:39:25,222] jepsen worker 555 - jepsen.util 555 :invoke :add 4405 INFO [2022-01-10 08:39:25,226] jepsen worker 834 - jepsen.util 834 :invoke :add 4406 INFO [2022-01-10 08:39:25,228] jepsen worker 888 - jepsen.util 888 :invoke :add 4407 INFO [2022-01-10 08:39:25,232] jepsen worker 142 - jepsen.util 142 :invoke :add 4408 INFO [2022-01-10 08:39:25,234] jepsen worker 34 - jepsen.util 34 :invoke :add 4409 INFO [2022-01-10 08:39:25,234] jepsen worker 319 - jepsen.util 319 :ok :add 4392 INFO [2022-01-10 08:39:25,235] jepsen worker 983 - jepsen.util 983 :ok :add 4236 INFO [2022-01-10 08:39:25,236] jepsen worker 86 - jepsen.util 86 :invoke :add 4410 INFO [2022-01-10 08:39:25,239] jepsen worker 736 - jepsen.util 736 :invoke :add 4411 INFO [2022-01-10 08:39:25,241] jepsen worker 573 - jepsen.util 573 :invoke :add 4412 INFO [2022-01-10 08:39:25,244] jepsen worker 964 - jepsen.util 964 :invoke :add 4413 INFO [2022-01-10 08:39:25,244] jepsen worker 34 - jepsen.util 34 :ok :add 4409 INFO [2022-01-10 08:39:25,246] jepsen worker 672 - jepsen.util 672 :invoke :add 4414 INFO [2022-01-10 08:39:25,246] jepsen worker 142 - jepsen.util 142 :ok :add 4408 INFO [2022-01-10 08:39:25,249] jepsen worker 812 - jepsen.util 812 :invoke :add 4415 INFO [2022-01-10 08:39:25,252] jepsen worker 129 - jepsen.util 129 :invoke :add 4416 INFO [2022-01-10 08:39:25,253] jepsen worker 736 - jepsen.util 736 :ok :add 4411 INFO [2022-01-10 08:39:25,254] jepsen worker 86 - jepsen.util 86 :ok :add 4410 INFO [2022-01-10 08:39:25,254] jepsen worker 672 - jepsen.util 672 :ok :add 4414 INFO [2022-01-10 08:39:25,256] jepsen worker 113 - jepsen.util 113 :invoke :add 4417 INFO [2022-01-10 08:39:25,258] jepsen worker 812 - jepsen.util 812 :ok :add 4415 INFO [2022-01-10 08:39:25,259] jepsen worker 964 - jepsen.util 964 :ok :add 4413 INFO [2022-01-10 08:39:25,259] jepsen worker 47 - jepsen.util 47 :invoke :add 4418 INFO [2022-01-10 08:39:25,260] jepsen worker 573 - jepsen.util 573 :ok :add 4412 INFO [2022-01-10 08:39:25,261] jepsen worker 56 - jepsen.util 56 :invoke :add 4419 INFO [2022-01-10 08:39:25,264] jepsen worker 720 - jepsen.util 720 :invoke :add 4420 INFO [2022-01-10 08:39:25,265] jepsen worker 129 - jepsen.util 129 :ok :add 4416 INFO [2022-01-10 08:39:25,267] jepsen worker 200 - jepsen.util 200 :invoke :add 4421 INFO [2022-01-10 08:39:25,268] jepsen worker 298 - jepsen.util 298 :ok :add 4206 INFO [2022-01-10 08:39:25,268] jepsen worker 653 - jepsen.util 653 :ok :add 4177 INFO [2022-01-10 08:39:25,268] jepsen worker 162 - jepsen.util 162 :ok :add 4143 INFO [2022-01-10 08:39:25,268] jepsen worker 292 - jepsen.util 292 :ok :add 3996 INFO [2022-01-10 08:39:25,268] jepsen worker 761 - jepsen.util 761 :ok :add 4232 INFO [2022-01-10 08:39:25,269] jepsen worker 113 - jepsen.util 113 :ok :add 4417 INFO [2022-01-10 08:39:25,270] jepsen worker 891 - jepsen.util 891 :invoke :add 4422 INFO [2022-01-10 08:39:25,274] jepsen worker 738 - jepsen.util 738 :invoke :add 4423 INFO [2022-01-10 08:39:25,276] jepsen worker 503 - jepsen.util 503 :invoke :add 4424 INFO [2022-01-10 08:39:25,278] jepsen worker 24 - jepsen.util 24 :invoke :add 4425 INFO [2022-01-10 08:39:25,281] jepsen worker 200 - jepsen.util 200 :ok :add 4421 INFO [2022-01-10 08:39:25,281] jepsen worker 666 - jepsen.util 666 :invoke :add 4426 INFO [2022-01-10 08:39:25,284] jepsen worker 628 - jepsen.util 628 :invoke :add 4427 INFO [2022-01-10 08:39:25,284] jepsen worker 738 - jepsen.util 738 :ok :add 4423 INFO [2022-01-10 08:39:25,284] jepsen worker 891 - jepsen.util 891 :ok :add 4422 INFO [2022-01-10 08:39:25,286] jepsen worker 276 - jepsen.util 276 :invoke :add 4428 INFO [2022-01-10 08:39:25,288] jepsen worker 846 - jepsen.util 846 :invoke :add 4429 INFO [2022-01-10 08:39:25,291] jepsen worker 207 - jepsen.util 207 :invoke :add 4430 INFO [2022-01-10 08:39:25,293] jepsen worker 743 - jepsen.util 743 :invoke :add 4431 INFO [2022-01-10 08:39:25,295] jepsen worker 823 - jepsen.util 823 :invoke :add 4432 INFO [2022-01-10 08:39:25,296] jepsen worker 4 - jepsen.util 4 :invoke :add 4433 INFO [2022-01-10 08:39:25,298] jepsen worker 720 - jepsen.util 720 :ok :add 4420 INFO [2022-01-10 08:39:25,298] jepsen worker 619 - jepsen.util 619 :invoke :add 4434 INFO [2022-01-10 08:39:25,301] jepsen worker 498 - jepsen.util 498 :invoke :add 4435 INFO [2022-01-10 08:39:25,376] jepsen worker 250 - jepsen.util 250 :ok :add 4393 INFO [2022-01-10 08:39:25,376] jepsen worker 454 - jepsen.util 454 :ok :add 4237 INFO [2022-01-10 08:39:25,376] jepsen worker 666 - jepsen.util 666 :ok :add 4426 INFO [2022-01-10 08:39:25,376] jepsen worker 503 - jepsen.util 503 :ok :add 4424 INFO [2022-01-10 08:39:25,379] jepsen worker 522 - jepsen.util 522 :invoke :add 4436 INFO [2022-01-10 08:39:25,382] jepsen worker 75 - jepsen.util 75 :invoke :add 4437 INFO [2022-01-10 08:39:25,384] jepsen worker 140 - jepsen.util 140 :invoke :add 4438 INFO [2022-01-10 08:39:25,387] jepsen worker 356 - jepsen.util 356 :invoke :add 4439 INFO [2022-01-10 08:39:25,388] jepsen worker 802 - jepsen.util 802 :ok :add 4095 INFO [2022-01-10 08:39:25,388] jepsen worker 257 - jepsen.util 257 :ok :add 4105 INFO [2022-01-10 08:39:25,388] jepsen worker 324 - jepsen.util 324 :ok :add 4131 INFO [2022-01-10 08:39:25,388] jepsen worker 505 - jepsen.util 505 :ok :add 4274 INFO [2022-01-10 08:39:25,388] jepsen worker 224 - jepsen.util 224 :ok :add 4240 INFO [2022-01-10 08:39:25,388] jepsen worker 873 - jepsen.util 873 :ok :add 4129 INFO [2022-01-10 08:39:25,388] jepsen worker 576 - jepsen.util 576 :ok :add 4258 INFO [2022-01-10 08:39:25,389] jepsen worker 915 - jepsen.util 915 :invoke :add 4440 INFO [2022-01-10 08:39:25,393] jepsen worker 580 - jepsen.util 580 :invoke :add 4441 INFO [2022-01-10 08:39:25,393] jepsen worker 140 - jepsen.util 140 :ok :add 4438 INFO [2022-01-10 08:39:25,395] jepsen worker 24 - jepsen.util 24 :ok :add 4425 INFO [2022-01-10 08:39:25,398] jepsen worker 356 - jepsen.util 356 :ok :add 4439 INFO [2022-01-10 08:39:25,398] jepsen worker 483 - jepsen.util 483 :invoke :add 4443 INFO [2022-01-10 08:39:25,399] jepsen worker 369 - jepsen.util 369 :invoke :add 4442 INFO [2022-01-10 08:39:25,401] jepsen worker 152 - jepsen.util 152 :invoke :add 4444 INFO [2022-01-10 08:39:25,403] jepsen worker 205 - jepsen.util 205 :invoke :add 4445 INFO [2022-01-10 08:39:25,405] jepsen worker 898 - jepsen.util 898 :invoke :add 4446 INFO [2022-01-10 08:39:25,408] jepsen worker 981 - jepsen.util 981 :invoke :add 4447 INFO [2022-01-10 08:39:25,410] jepsen worker 829 - jepsen.util 829 :invoke :add 4448 INFO [2022-01-10 08:39:25,414] jepsen worker 903 - jepsen.util 903 :invoke :add 4449 INFO [2022-01-10 08:39:25,415] jepsen worker 829 - jepsen.util 829 :ok :add 4448 INFO [2022-01-10 08:39:25,416] jepsen worker 304 - jepsen.util 304 :invoke :add 4450 INFO [2022-01-10 08:39:25,419] jepsen worker 151 - jepsen.util 151 :invoke :add 4451 INFO [2022-01-10 08:39:25,421] jepsen worker 903 - jepsen.util 903 :ok :add 4449 INFO [2022-01-10 08:39:25,422] jepsen worker 46 - jepsen.util 46 :invoke :add 4452 INFO [2022-01-10 08:39:25,422] jepsen worker 304 - jepsen.util 304 :ok :add 4450 INFO [2022-01-10 08:39:25,425] jepsen worker 865 - jepsen.util 865 :invoke :add 4453 INFO [2022-01-10 08:39:25,427] jepsen worker 718 - jepsen.util 718 :invoke :add 4454 INFO [2022-01-10 08:39:25,428] jepsen worker 151 - jepsen.util 151 :ok :add 4451 INFO [2022-01-10 08:39:25,429] jepsen worker 515 - jepsen.util 515 :invoke :add 4455 INFO [2022-01-10 08:39:25,432] jepsen worker 695 - jepsen.util 695 :invoke :add 4456 INFO [2022-01-10 08:39:25,432] jepsen worker 865 - jepsen.util 865 :ok :add 4453 INFO [2022-01-10 08:39:25,433] jepsen worker 556 - jepsen.util 556 :ok :add 4141 INFO [2022-01-10 08:39:25,434] jepsen worker 46 - jepsen.util 46 :ok :add 4452 INFO [2022-01-10 08:39:25,434] jepsen worker 29 - jepsen.util 29 :ok :add 4159 INFO [2022-01-10 08:39:25,434] jepsen worker 591 - jepsen.util 591 :ok :add 4175 INFO [2022-01-10 08:39:25,434] jepsen worker 109 - jepsen.util 109 :ok :add 4230 INFO [2022-01-10 08:39:25,434] jepsen worker 889 - jepsen.util 889 :ok :add 4204 INFO [2022-01-10 08:39:25,434] jepsen worker 702 - jepsen.util 702 :ok :add 4228 INFO [2022-01-10 08:39:25,434] jepsen worker 328 - jepsen.util 328 :ok :add 4394 INFO [2022-01-10 08:39:25,435] jepsen worker 585 - jepsen.util 585 :invoke :add 4457 INFO [2022-01-10 08:39:25,439] jepsen worker 410 - jepsen.util 410 :invoke :add 4458 INFO [2022-01-10 08:39:25,441] jepsen worker 248 - jepsen.util 248 :invoke :add 4459 INFO [2022-01-10 08:39:25,444] jepsen worker 709 - jepsen.util 709 :invoke :add 4460 INFO [2022-01-10 08:39:25,446] jepsen worker 818 - jepsen.util 818 :invoke :add 4461 INFO [2022-01-10 08:39:25,448] jepsen worker 70 - jepsen.util 70 :invoke :add 4462 INFO [2022-01-10 08:39:25,451] jepsen worker 570 - jepsen.util 570 :invoke :add 4463 INFO [2022-01-10 08:39:25,453] jepsen worker 165 - jepsen.util 165 :invoke :add 4464 INFO [2022-01-10 08:39:25,455] jepsen worker 195 - jepsen.util 195 :invoke :add 4465 INFO [2022-01-10 08:39:25,457] jepsen worker 869 - jepsen.util 869 :invoke :add 4466 INFO [2022-01-10 08:39:25,459] jepsen worker 799 - jepsen.util 799 :invoke :add 4467 INFO [2022-01-10 08:39:25,462] jepsen worker 979 - jepsen.util 979 :invoke :add 4468 INFO [2022-01-10 08:39:25,463] jepsen worker 628 - jepsen.util 628 :ok :add 4427 INFO [2022-01-10 08:39:25,464] jepsen worker 328 - jepsen.util 328 :invoke :add 4469 INFO [2022-01-10 08:39:25,465] jepsen worker 195 - jepsen.util 195 :ok :add 4465 INFO [2022-01-10 08:39:25,467] jepsen worker 363 - jepsen.util 363 :ok :add 4307 INFO [2022-01-10 08:39:25,467] jepsen worker 501 - jepsen.util 501 :ok :add 4076 INFO [2022-01-10 08:39:25,468] jepsen worker 60 - jepsen.util 60 :ok :add 4333 INFO [2022-01-10 08:39:25,468] jepsen worker 407 - jepsen.util 407 :invoke :add 4470 INFO [2022-01-10 08:39:25,468] jepsen worker 623 - jepsen.util 623 :ok :add 4297 INFO [2022-01-10 08:39:25,469] jepsen worker 165 - jepsen.util 165 :ok :add 4464 INFO [2022-01-10 08:39:25,471] jepsen worker 804 - jepsen.util 804 :invoke :add 4471 INFO [2022-01-10 08:39:25,472] jepsen worker 869 - jepsen.util 869 :ok :add 4466 INFO [2022-01-10 08:39:25,474] jepsen worker 540 - jepsen.util 540 :invoke :add 4472 INFO [2022-01-10 08:39:25,474] jepsen worker 799 - jepsen.util 799 :ok :add 4467 INFO [2022-01-10 08:39:25,477] jepsen worker 432 - jepsen.util 432 :invoke :add 4473 INFO [2022-01-10 08:39:25,479] jepsen worker 706 - jepsen.util 706 :invoke :add 4474 INFO [2022-01-10 08:39:25,481] jepsen worker 994 - jepsen.util 994 :invoke :add 4475 INFO [2022-01-10 08:39:25,484] jepsen worker 942 - jepsen.util 942 :invoke :add 4476 INFO [2022-01-10 08:39:25,487] jepsen worker 790 - jepsen.util 790 :invoke :add 4477 INFO [2022-01-10 08:39:25,489] jepsen worker 140 - jepsen.util 140 :invoke :add 4478 INFO [2022-01-10 08:39:25,491] jepsen worker 91 - jepsen.util 91 :invoke :add 4479 INFO [2022-01-10 08:39:25,494] jepsen worker 986 - jepsen.util 986 :invoke :add 4480 INFO [2022-01-10 08:39:25,496] jepsen worker 46 - jepsen.util 46 :invoke :add 4481 INFO [2022-01-10 08:39:25,498] jepsen worker 413 - jepsen.util 413 :invoke :add 4482 INFO [2022-01-10 08:39:25,500] jepsen worker 221 - jepsen.util 221 :invoke :add 4483 INFO [2022-01-10 08:39:25,503] jepsen worker 629 - jepsen.util 629 :invoke :add 4484 INFO [2022-01-10 08:39:25,506] jepsen worker 904 - jepsen.util 904 :invoke :add 4485 INFO [2022-01-10 08:39:25,508] jepsen worker 200 - jepsen.util 200 :invoke :add 4486 INFO [2022-01-10 08:39:25,511] jepsen worker 919 - jepsen.util 919 :invoke :add 4487 INFO [2022-01-10 08:39:25,514] jepsen worker 524 - jepsen.util 524 :invoke :add 4488 INFO [2022-01-10 08:39:25,515] jepsen worker 629 - jepsen.util 629 :ok :add 4484 INFO [2022-01-10 08:39:25,515] jepsen worker 904 - jepsen.util 904 :ok :add 4485 INFO [2022-01-10 08:39:25,517] jepsen worker 362 - jepsen.util 362 :invoke :add 4489 INFO [2022-01-10 08:39:25,520] jepsen worker 2 - jepsen.util 2 :invoke :add 4490 INFO [2022-01-10 08:39:25,523] jepsen worker 730 - jepsen.util 730 :invoke :add 4491 INFO [2022-01-10 08:39:25,524] jepsen worker 200 - jepsen.util 200 :ok :add 4486 INFO [2022-01-10 08:39:25,525] jepsen worker 609 - jepsen.util 609 :invoke :add 4492 INFO [2022-01-10 08:39:25,526] jepsen worker 919 - jepsen.util 919 :ok :add 4487 INFO [2022-01-10 08:39:25,528] jepsen worker 737 - jepsen.util 737 :invoke :add 4493 INFO [2022-01-10 08:39:25,531] jepsen worker 434 - jepsen.util 434 :invoke :add 4494 INFO [2022-01-10 08:39:25,533] jepsen worker 239 - jepsen.util 239 :invoke :add 4495 INFO [2022-01-10 08:39:25,536] jepsen worker 711 - jepsen.util 711 :invoke :add 4496 INFO [2022-01-10 08:39:25,538] jepsen worker 515 - jepsen.util 515 :ok :add 4455 INFO [2022-01-10 08:39:25,538] jepsen worker 91 - jepsen.util 91 :ok :add 4479 INFO [2022-01-10 08:39:25,538] jepsen worker 140 - jepsen.util 140 :ok :add 4478 INFO [2022-01-10 08:39:25,538] jepsen worker 818 - jepsen.util 818 :ok :add 4461 INFO [2022-01-10 08:39:25,538] jepsen worker 709 - jepsen.util 709 :ok :add 4460 INFO [2022-01-10 08:39:25,538] jepsen worker 718 - jepsen.util 718 :ok :add 4454 INFO [2022-01-10 08:39:25,540] jepsen worker 606 - jepsen.util 606 :invoke :add 4497 INFO [2022-01-10 08:39:25,544] jepsen worker 750 - jepsen.util 750 :invoke :add 4498 INFO [2022-01-10 08:39:25,548] jepsen worker 692 - jepsen.util 692 :invoke :add 4499 INFO [2022-01-10 08:39:25,552] jepsen worker 698 - jepsen.util 698 :invoke :add 4500 INFO [2022-01-10 08:39:25,554] jepsen worker 504 - jepsen.util 504 :invoke :add 4501 INFO [2022-01-10 08:39:25,556] jepsen worker 539 - jepsen.util 539 :invoke :add 4502 INFO [2022-01-10 08:39:25,557] jepsen worker 555 - jepsen.util 555 :ok :add 4405 INFO [2022-01-10 08:39:25,557] jepsen worker 97 - jepsen.util 97 :ok :add 4196 INFO [2022-01-10 08:39:25,558] jepsen worker 743 - jepsen.util 743 :ok :add 4431 INFO [2022-01-10 08:39:25,558] jepsen worker 373 - jepsen.util 373 :ok :add 4343 INFO [2022-01-10 08:39:25,558] jepsen worker 846 - jepsen.util 846 :ok :add 4429 INFO [2022-01-10 08:39:25,558] jepsen worker 657 - jepsen.util 657 :ok :add 4377 INFO [2022-01-10 08:39:25,560] jepsen worker 662 - jepsen.util 662 :invoke :add 4503 INFO [2022-01-10 08:39:25,564] jepsen worker 679 - jepsen.util 679 :invoke :add 4504 INFO [2022-01-10 08:39:25,567] jepsen worker 553 - jepsen.util 553 :invoke :add 4505 INFO [2022-01-10 08:39:25,570] jepsen worker 209 - jepsen.util 209 :invoke :add 4506 INFO [2022-01-10 08:39:25,573] jepsen worker 976 - jepsen.util 976 :invoke :add 4507 INFO [2022-01-10 08:39:25,576] jepsen worker 216 - jepsen.util 216 :invoke :add 4508 INFO [2022-01-10 08:39:25,578] jepsen worker 523 - jepsen.util 523 :invoke :add 4509 INFO [2022-01-10 08:39:25,580] jepsen worker 238 - jepsen.util 238 :invoke :add 4510 INFO [2022-01-10 08:39:25,583] jepsen worker 573 - jepsen.util 573 :invoke :add 4511 INFO [2022-01-10 08:39:25,585] jepsen worker 14 - jepsen.util 14 :invoke :add 4512 INFO [2022-01-10 08:39:25,587] jepsen worker 782 - jepsen.util 782 :invoke :add 4513 INFO [2022-01-10 08:39:25,589] jepsen worker 110 - jepsen.util 110 :invoke :add 4514 INFO [2022-01-10 08:39:25,592] jepsen worker 659 - jepsen.util 659 :invoke :add 4515 INFO [2022-01-10 08:39:25,594] jepsen worker 538 - jepsen.util 538 :invoke :add 4516 INFO [2022-01-10 08:39:25,597] jepsen worker 414 - jepsen.util 414 :invoke :add 4517 INFO [2022-01-10 08:39:25,598] jepsen worker 570 - jepsen.util 570 :ok :add 4463 INFO [2022-01-10 08:39:25,598] jepsen worker 585 - jepsen.util 585 :ok :add 4457 INFO [2022-01-10 08:39:25,598] jepsen worker 695 - jepsen.util 695 :ok :add 4456 INFO [2022-01-10 08:39:25,598] jepsen worker 70 - jepsen.util 70 :ok :add 4462 INFO [2022-01-10 08:39:25,600] jepsen worker 26 - jepsen.util 26 :invoke :add 4518 INFO [2022-01-10 08:39:25,601] jepsen worker 659 - jepsen.util 659 :ok :add 4515 INFO [2022-01-10 08:39:25,602] jepsen worker 110 - jepsen.util 110 :ok :add 4514 INFO [2022-01-10 08:39:25,603] jepsen worker 315 - jepsen.util 315 :invoke :add 4519 INFO [2022-01-10 08:39:25,604] jepsen worker 836 - jepsen.util 836 :ok :add 4273 INFO [2022-01-10 08:39:25,605] jepsen worker 813 - jepsen.util 813 :ok :add 4247 INFO [2022-01-10 08:39:25,606] jepsen worker 168 - jepsen.util 168 :invoke :add 4520 INFO [2022-01-10 08:39:25,607] jepsen worker 538 - jepsen.util 538 :ok :add 4516 INFO [2022-01-10 08:39:25,609] jepsen worker 145 - jepsen.util 145 :invoke :add 4521 INFO [2022-01-10 08:39:25,611] jepsen worker 548 - jepsen.util 548 :invoke :add 4522 INFO [2022-01-10 08:39:25,614] jepsen worker 87 - jepsen.util 87 :invoke :add 4523 INFO [2022-01-10 08:39:25,616] jepsen worker 356 - jepsen.util 356 :invoke :add 4524 INFO [2022-01-10 08:39:25,619] jepsen worker 208 - jepsen.util 208 :invoke :add 4525 INFO [2022-01-10 08:39:25,620] jepsen worker 227 - jepsen.util 227 :ok :add 4133 INFO [2022-01-10 08:39:25,620] jepsen worker 777 - jepsen.util 777 :ok :add 4276 INFO [2022-01-10 08:39:25,621] jepsen worker 296 - jepsen.util 296 :ok :add 4242 INFO [2022-01-10 08:39:25,621] jepsen worker 52 - jepsen.util 52 :ok :add 4107 INFO [2022-01-10 08:39:25,621] jepsen worker 318 - jepsen.util 318 :ok :add 4097 INFO [2022-01-10 08:39:25,622] jepsen worker 347 - jepsen.util 347 :invoke :add 4526 INFO [2022-01-10 08:39:25,626] jepsen worker 414 - jepsen.util 414 :ok :add 4517 INFO [2022-01-10 08:39:25,626] jepsen worker 87 - jepsen.util 87 :ok :add 4523 INFO [2022-01-10 08:39:25,627] jepsen worker 807 - jepsen.util 807 :invoke :add 4527 INFO [2022-01-10 08:39:25,631] jepsen worker 597 - jepsen.util 597 :invoke :add 4528 INFO [2022-01-10 08:39:25,633] jepsen worker 425 - jepsen.util 425 :invoke :add 4529 INFO [2022-01-10 08:39:25,634] jepsen worker 145 - jepsen.util 145 :ok :add 4521 INFO [2022-01-10 08:39:25,636] jepsen worker 194 - jepsen.util 194 :invoke :add 4530 INFO [2022-01-10 08:39:25,639] jepsen worker 260 - jepsen.util 260 :invoke :add 4531 INFO [2022-01-10 08:39:25,641] jepsen worker 548 - jepsen.util 548 :ok :add 4522 INFO [2022-01-10 08:39:25,643] jepsen worker 814 - jepsen.util 814 :invoke :add 4532 INFO [2022-01-10 08:39:25,646] jepsen worker 86 - jepsen.util 86 :invoke :add 4533 INFO [2022-01-10 08:39:25,649] jepsen worker 111 - jepsen.util 111 :invoke :add 4534 INFO [2022-01-10 08:39:25,652] jepsen worker 753 - jepsen.util 753 :invoke :add 4535 INFO [2022-01-10 08:39:25,655] jepsen worker 51 - jepsen.util 51 :invoke :add 4536 INFO [2022-01-10 08:39:25,656] jepsen worker 573 - jepsen.util 573 :ok :add 4511 INFO [2022-01-10 08:39:25,656] jepsen worker 46 - jepsen.util 46 :ok :add 4481 INFO [2022-01-10 08:39:25,656] jepsen worker 208 - jepsen.util 208 :ok :add 4525 INFO [2022-01-10 08:39:25,656] jepsen worker 692 - jepsen.util 692 :ok :add 4499 INFO [2022-01-10 08:39:25,656] jepsen worker 523 - jepsen.util 523 :ok :add 4509 INFO [2022-01-10 08:39:25,657] jepsen worker 314 - jepsen.util 314 :invoke :add 4537 INFO [2022-01-10 08:39:25,658] jepsen worker 168 - jepsen.util 168 :ok :add 4520 INFO [2022-01-10 08:39:25,661] jepsen worker 355 - jepsen.util 355 :invoke :add 4538 INFO [2022-01-10 08:39:25,663] jepsen worker 750 - jepsen.util 750 :ok :add 4498 INFO [2022-01-10 08:39:25,663] jepsen worker 986 - jepsen.util 986 :ok :add 4480 INFO [2022-01-10 08:39:25,663] jepsen worker 558 - jepsen.util 558 :invoke :add 4539 INFO [2022-01-10 08:39:25,663] jepsen worker 238 - jepsen.util 238 :ok :add 4510 INFO [2022-01-10 08:39:25,664] jepsen worker 216 - jepsen.util 216 :ok :add 4508 INFO [2022-01-10 08:39:25,664] jepsen worker 356 - jepsen.util 356 :ok :add 4524 INFO [2022-01-10 08:39:25,666] jepsen worker 221 - jepsen.util 221 :ok :add 4483 INFO [2022-01-10 08:39:25,666] jepsen worker 807 - jepsen.util 807 :ok :add 4527 INFO [2022-01-10 08:39:25,666] jepsen worker 782 - jepsen.util 782 :ok :add 4513 INFO [2022-01-10 08:39:25,666] jepsen worker 178 - jepsen.util 178 :invoke :add 4540 INFO [2022-01-10 08:39:25,670] jepsen worker 555 - jepsen.util 555 :invoke :add 4541 INFO [2022-01-10 08:39:25,673] jepsen worker 105 - jepsen.util 105 :invoke :add 4542 INFO [2022-01-10 08:39:25,674] jepsen worker 558 - jepsen.util 558 :ok :add 4539 INFO [2022-01-10 08:39:25,676] jepsen worker 190 - jepsen.util 190 :invoke :add 4543 INFO [2022-01-10 08:39:25,678] jepsen worker 463 - jepsen.util 463 :invoke :add 4544 INFO [2022-01-10 08:39:25,680] jepsen worker 355 - jepsen.util 355 :ok :add 4538 INFO [2022-01-10 08:39:25,680] jepsen worker 629 - jepsen.util 629 :invoke :add 4545 INFO [2022-01-10 08:39:25,683] jepsen worker 917 - jepsen.util 917 :invoke :add 4546 INFO [2022-01-10 08:39:25,686] jepsen worker 386 - jepsen.util 386 :invoke :add 4547 INFO [2022-01-10 08:39:25,688] jepsen worker 140 - jepsen.util 140 :invoke :add 4548 INFO [2022-01-10 08:39:25,691] jepsen worker 438 - jepsen.util 438 :invoke :add 4549 INFO [2022-01-10 08:39:25,693] jepsen worker 202 - jepsen.util 202 :invoke :add 4550 INFO [2022-01-10 08:39:25,696] jepsen worker 909 - jepsen.util 909 :invoke :add 4551 INFO [2022-01-10 08:39:25,698] jepsen worker 338 - jepsen.util 338 :invoke :add 4552 INFO [2022-01-10 08:39:25,702] jepsen worker 557 - jepsen.util 557 :invoke :add 4553 INFO [2022-01-10 08:39:25,704] jepsen worker 202 - jepsen.util 202 :ok :add 4550 INFO [2022-01-10 08:39:25,705] jepsen worker 140 - jepsen.util 140 :ok :add 4548 INFO [2022-01-10 08:39:25,706] jepsen worker 243 - jepsen.util 243 :invoke :add 4554 INFO [2022-01-10 08:39:25,710] jepsen worker 375 - jepsen.util 375 :invoke :add 4555 INFO [2022-01-10 08:39:25,711] jepsen worker 338 - jepsen.util 338 :ok :add 4552 INFO [2022-01-10 08:39:25,714] jepsen worker 858 - jepsen.util 858 :invoke :add 4556 INFO [2022-01-10 08:39:25,717] jepsen worker 134 - jepsen.util 134 :invoke :add 4557 INFO [2022-01-10 08:39:25,720] jepsen worker 65 - jepsen.util 65 :invoke :add 4558 INFO [2022-01-10 08:39:25,722] jepsen worker 60 - jepsen.util 60 :invoke :add 4559 INFO [2022-01-10 08:39:25,724] jepsen worker 909 - jepsen.util 909 :ok :add 4551 INFO [2022-01-10 08:39:25,724] jepsen worker 438 - jepsen.util 438 :ok :add 4549 INFO [2022-01-10 08:39:25,725] jepsen worker 589 - jepsen.util 589 :invoke :add 4560 INFO [2022-01-10 08:39:25,728] jepsen worker 110 - jepsen.util 110 :invoke :add 4561 INFO [2022-01-10 08:39:25,728] jepsen worker 14 - jepsen.util 14 :ok :add 4512 INFO [2022-01-10 08:39:25,728] jepsen worker 413 - jepsen.util 413 :ok :add 4482 INFO [2022-01-10 08:39:25,728] jepsen worker 557 - jepsen.util 557 :ok :add 4553 INFO [2022-01-10 08:39:25,729] jepsen worker 347 - jepsen.util 347 :ok :add 4526 INFO [2022-01-10 08:39:25,732] jepsen worker 154 - jepsen.util 154 :invoke :add 4562 INFO [2022-01-10 08:39:25,735] jepsen worker 844 - jepsen.util 844 :invoke :add 4563 INFO [2022-01-10 08:39:25,737] jepsen worker 872 - jepsen.util 872 :invoke :add 4564 INFO [2022-01-10 08:39:25,742] jepsen worker 76 - jepsen.util 76 :invoke :add 4565 INFO [2022-01-10 08:39:25,744] jepsen worker 225 - jepsen.util 225 :invoke :add 4566 INFO [2022-01-10 08:39:25,747] jepsen worker 883 - jepsen.util 883 :invoke :add 4567 INFO [2022-01-10 08:39:25,749] jepsen worker 439 - jepsen.util 439 :invoke :add 4568 INFO [2022-01-10 08:39:25,751] jepsen worker 225 - jepsen.util 225 :ok :add 4566 INFO [2022-01-10 08:39:25,751] jepsen worker 872 - jepsen.util 872 :ok :add 4564 INFO [2022-01-10 08:39:25,752] jepsen worker 76 - jepsen.util 76 :ok :add 4565 INFO [2022-01-10 08:39:25,753] jepsen worker 810 - jepsen.util 810 :invoke :add 4569 INFO [2022-01-10 08:39:25,756] jepsen worker 430 - jepsen.util 430 :invoke :add 4570 INFO [2022-01-10 08:39:25,758] jepsen worker 883 - jepsen.util 883 :ok :add 4567 INFO [2022-01-10 08:39:25,758] jepsen worker 982 - jepsen.util 982 :invoke :add 4571 INFO [2022-01-10 08:39:25,762] jepsen worker 766 - jepsen.util 766 :invoke :add 4572 INFO [2022-01-10 08:39:25,765] jepsen worker 268 - jepsen.util 268 :invoke :add 4573 INFO [2022-01-10 08:39:25,767] jepsen worker 851 - jepsen.util 851 :invoke :add 4574 INFO [2022-01-10 08:39:25,770] jepsen worker 399 - jepsen.util 399 :invoke :add 4575 INFO [2022-01-10 08:39:25,772] jepsen worker 713 - jepsen.util 713 :invoke :add 4576 INFO [2022-01-10 08:39:25,775] jepsen worker 912 - jepsen.util 912 :invoke :add 4577 INFO [2022-01-10 08:39:25,777] jepsen worker 378 - jepsen.util 378 :invoke :add 4578 INFO [2022-01-10 08:39:25,780] jepsen worker 533 - jepsen.util 533 :invoke :add 4579 INFO [2022-01-10 08:39:25,782] jepsen worker 284 - jepsen.util 284 :invoke :add 4580 INFO [2022-01-10 08:39:25,785] jepsen worker 122 - jepsen.util 122 :invoke :add 4581 INFO [2022-01-10 08:39:25,788] jepsen worker 511 - jepsen.util 511 :invoke :add 4582 INFO [2022-01-10 08:39:25,790] jepsen worker 169 - jepsen.util 169 :invoke :add 4583 INFO [2022-01-10 08:39:25,793] jepsen worker 860 - jepsen.util 860 :invoke :add 4584 INFO [2022-01-10 08:39:25,795] jepsen worker 418 - jepsen.util 418 :invoke :add 4585 INFO [2022-01-10 08:39:25,797] jepsen worker 52 - jepsen.util 52 :invoke :add 4586 INFO [2022-01-10 08:39:25,801] jepsen worker 888 - jepsen.util 888 :ok :add 4407 INFO [2022-01-10 08:39:25,801] jepsen worker 4 - jepsen.util 4 :ok :add 4433 INFO [2022-01-10 08:39:25,801] jepsen worker 863 - jepsen.util 863 :ok :add 4194 INFO [2022-01-10 08:39:25,801] jepsen worker 786 - jepsen.util 786 :ok :add 4375 INFO [2022-01-10 08:39:25,802] jepsen worker 267 - jepsen.util 267 :ok :add 4341 INFO [2022-01-10 08:39:25,802] jepsen worker 99 - jepsen.util 99 :ok :add 4359 INFO [2022-01-10 08:39:25,804] jepsen worker 418 - jepsen.util 418 :ok :add 4585 INFO [2022-01-10 08:39:25,804] jepsen worker 388 - jepsen.util 388 :invoke :add 4587 INFO [2022-01-10 08:39:25,805] jepsen worker 860 - jepsen.util 860 :ok :add 4584 INFO [2022-01-10 08:39:25,807] jepsen worker 17 - jepsen.util 17 :invoke :add 4588 INFO [2022-01-10 08:39:25,811] jepsen worker 249 - jepsen.util 249 :invoke :add 4589 INFO [2022-01-10 08:39:25,814] jepsen worker 910 - jepsen.util 910 :invoke :add 4590 INFO [2022-01-10 08:39:25,814] jepsen worker 52 - jepsen.util 52 :ok :add 4586 INFO [2022-01-10 08:39:25,816] jepsen worker 575 - jepsen.util 575 :invoke :add 4591 INFO [2022-01-10 08:39:25,818] jepsen worker 388 - jepsen.util 388 :ok :add 4587 INFO [2022-01-10 08:39:25,820] jepsen worker 633 - jepsen.util 633 :invoke :add 4592 INFO [2022-01-10 08:39:25,823] jepsen worker 140 - jepsen.util 140 :invoke :add 4593 INFO [2022-01-10 08:39:25,826] jepsen worker 795 - jepsen.util 795 :invoke :add 4594 INFO [2022-01-10 08:39:25,828] jepsen worker 642 - jepsen.util 642 :invoke :add 4595 INFO [2022-01-10 08:39:25,832] jepsen worker 220 - jepsen.util 220 :invoke :add 4596 INFO [2022-01-10 08:39:25,836] jepsen worker 323 - jepsen.util 323 :invoke :add 4597 INFO [2022-01-10 08:39:25,838] jepsen worker 732 - jepsen.util 732 :invoke :add 4598 INFO [2022-01-10 08:39:25,841] jepsen worker 680 - jepsen.util 680 :invoke :add 4599 INFO [2022-01-10 08:39:25,843] jepsen worker 854 - jepsen.util 854 :invoke :add 4600 INFO [2022-01-10 08:39:25,846] jepsen worker 684 - jepsen.util 684 :invoke :add 4601 INFO [2022-01-10 08:39:25,849] jepsen worker 224 - jepsen.util 224 :invoke :add 4602 INFO [2022-01-10 08:39:25,852] jepsen worker 593 - jepsen.util 593 :invoke :add 4603 INFO [2022-01-10 08:39:25,854] jepsen worker 508 - jepsen.util 508 :invoke :add 4604 INFO [2022-01-10 08:39:25,856] jepsen worker 818 - jepsen.util 818 :invoke :add 4605 INFO [2022-01-10 08:39:25,859] jepsen worker 262 - jepsen.util 262 :invoke :add 4606 INFO [2022-01-10 08:39:25,861] jepsen worker 203 - jepsen.util 203 :invoke :add 4607 INFO [2022-01-10 08:39:25,863] jepsen worker 683 - jepsen.util 683 :invoke :add 4608 INFO [2022-01-10 08:39:25,865] jepsen worker 5 - jepsen.util 5 :invoke :add 4609 INFO [2022-01-10 08:39:25,869] jepsen worker 754 - jepsen.util 754 :invoke :add 4610 INFO [2022-01-10 08:39:25,872] jepsen worker 204 - jepsen.util 204 :invoke :add 4611 INFO [2022-01-10 08:39:25,875] jepsen worker 66 - jepsen.util 66 :invoke :add 4612 INFO [2022-01-10 08:39:25,878] jepsen worker 856 - jepsen.util 856 :invoke :add 4613 INFO [2022-01-10 08:39:25,880] jepsen worker 24 - jepsen.util 24 :invoke :add 4614 INFO [2022-01-10 08:39:25,882] jepsen worker 394 - jepsen.util 394 :invoke :add 4615 INFO [2022-01-10 08:39:25,885] jepsen worker 928 - jepsen.util 928 :invoke :add 4616 INFO [2022-01-10 08:39:25,887] jepsen worker 24 - jepsen.util 24 :ok :add 4614 INFO [2022-01-10 08:39:25,888] jepsen worker 515 - jepsen.util 515 :invoke :add 4617 INFO [2022-01-10 08:39:25,890] jepsen worker 807 - jepsen.util 807 :invoke :add 4618 INFO [2022-01-10 08:39:25,891] jepsen worker 394 - jepsen.util 394 :ok :add 4615 INFO [2022-01-10 08:39:25,893] jepsen worker 219 - jepsen.util 219 :invoke :add 4619 INFO [2022-01-10 08:39:25,896] jepsen worker 776 - jepsen.util 776 :invoke :add 4620 INFO [2022-01-10 08:39:25,898] jepsen worker 742 - jepsen.util 742 :invoke :add 4621 INFO [2022-01-10 08:39:25,901] jepsen worker 338 - jepsen.util 338 :invoke :add 4622 INFO [2022-01-10 08:39:25,902] jepsen worker 515 - jepsen.util 515 :ok :add 4617 INFO [2022-01-10 08:39:25,904] jepsen worker 618 - jepsen.util 618 :invoke :add 4623 INFO [2022-01-10 08:39:25,907] jepsen worker 549 - jepsen.util 549 :invoke :add 4624 INFO [2022-01-10 08:39:25,910] jepsen worker 742 - jepsen.util 742 :ok :add 4621 INFO [2022-01-10 08:39:25,910] jepsen worker 928 - jepsen.util 928 :ok :add 4616 INFO [2022-01-10 08:39:25,911] jepsen worker 965 - jepsen.util 965 :invoke :add 4625 INFO [2022-01-10 08:39:25,914] jepsen worker 776 - jepsen.util 776 :ok :add 4620 INFO [2022-01-10 08:39:25,915] jepsen worker 185 - jepsen.util 185 :invoke :add 4626 INFO [2022-01-10 08:39:25,916] jepsen worker 618 - jepsen.util 618 :ok :add 4623 INFO [2022-01-10 08:39:25,918] jepsen worker 338 - jepsen.util 338 :ok :add 4622 INFO [2022-01-10 08:39:25,918] jepsen worker 303 - jepsen.util 303 :invoke :add 4627 INFO [2022-01-10 08:39:25,921] jepsen worker 656 - jepsen.util 656 :invoke :add 4628 INFO [2022-01-10 08:39:25,924] jepsen worker 274 - jepsen.util 274 :invoke :add 4629 INFO [2022-01-10 08:39:25,926] jepsen worker 846 - jepsen.util 846 :invoke :add 4630 INFO [2022-01-10 08:39:25,929] jepsen worker 44 - jepsen.util 44 :invoke :add 4631 INFO [2022-01-10 08:39:25,932] jepsen worker 42 - jepsen.util 42 :invoke :add 4632 INFO [2022-01-10 08:39:25,935] jepsen worker 523 - jepsen.util 523 :invoke :add 4633 INFO [2022-01-10 08:39:25,937] jepsen worker 176 - jepsen.util 176 :invoke :add 4634 INFO [2022-01-10 08:39:25,940] jepsen worker 85 - jepsen.util 85 :invoke :add 4635 INFO [2022-01-10 08:39:25,942] jepsen worker 468 - jepsen.util 468 :invoke :add 4636 INFO [2022-01-10 08:39:25,944] jepsen worker 903 - jepsen.util 903 :invoke :add 4637 INFO [2022-01-10 08:39:25,947] jepsen worker 445 - jepsen.util 445 :invoke :add 4638 INFO [2022-01-10 08:39:25,949] jepsen worker 573 - jepsen.util 573 :invoke :add 4639 INFO [2022-01-10 08:39:25,951] jepsen worker 848 - jepsen.util 848 :invoke :add 4640 INFO [2022-01-10 08:39:25,953] jepsen worker 445 - jepsen.util 445 :ok :add 4638 INFO [2022-01-10 08:39:25,954] jepsen worker 628 - jepsen.util 628 :invoke :add 4641 INFO [2022-01-10 08:39:25,956] jepsen worker 469 - jepsen.util 469 :invoke :add 4642 INFO [2022-01-10 08:39:25,956] jepsen worker 573 - jepsen.util 573 :ok :add 4639 INFO [2022-01-10 08:39:25,959] jepsen worker 906 - jepsen.util 906 :invoke :add 4643 INFO [2022-01-10 08:39:25,961] jepsen worker 233 - jepsen.util 233 :invoke :add 4644 INFO [2022-01-10 08:39:25,964] jepsen worker 288 - jepsen.util 288 :invoke :add 4645 INFO [2022-01-10 08:39:25,966] jepsen worker 428 - jepsen.util 428 :invoke :add 4646 INFO [2022-01-10 08:39:25,969] jepsen worker 667 - jepsen.util 667 :invoke :add 4647 INFO [2022-01-10 08:39:25,972] jepsen worker 962 - jepsen.util 962 :invoke :add 4648 INFO [2022-01-10 08:39:25,975] jepsen worker 346 - jepsen.util 346 :invoke :add 4649 INFO [2022-01-10 08:39:25,978] jepsen worker 962 - jepsen.util 962 :ok :add 4648 INFO [2022-01-10 08:39:25,979] jepsen worker 199 - jepsen.util 199 :invoke :add 4650 INFO [2022-01-10 08:39:25,982] jepsen worker 346 - jepsen.util 346 :ok :add 4649 INFO [2022-01-10 08:39:25,983] jepsen worker 438 - jepsen.util 438 :invoke :add 4651 INFO [2022-01-10 08:39:25,986] jepsen worker 64 - jepsen.util 64 :invoke :add 4652 INFO [2022-01-10 08:39:25,987] jepsen worker 199 - jepsen.util 199 :ok :add 4650 INFO [2022-01-10 08:39:25,989] jepsen worker 769 - jepsen.util 769 :invoke :add 4653 INFO [2022-01-10 08:39:25,992] jepsen worker 63 - jepsen.util 63 :invoke :add 4654 INFO [2022-01-10 08:39:25,993] jepsen worker 438 - jepsen.util 438 :ok :add 4651 INFO [2022-01-10 08:39:25,995] jepsen worker 118 - jepsen.util 118 :invoke :add 4655 INFO [2022-01-10 08:39:25,997] jepsen worker 709 - jepsen.util 709 :invoke :add 4656 INFO [2022-01-10 08:39:25,998] jepsen worker 64 - jepsen.util 64 :ok :add 4652 INFO [2022-01-10 08:39:25,999] jepsen worker 769 - jepsen.util 769 :ok :add 4653 INFO [2022-01-10 08:39:26,000] jepsen worker 944 - jepsen.util 944 :invoke :add 4657 INFO [2022-01-10 08:39:26,004] jepsen worker 151 - jepsen.util 151 :invoke :add 4658 INFO [2022-01-10 08:39:26,006] jepsen worker 546 - jepsen.util 546 :invoke :add 4659 INFO [2022-01-10 08:39:26,009] jepsen worker 128 - jepsen.util 128 :invoke :add 4660 INFO [2022-01-10 08:39:26,012] jepsen worker 567 - jepsen.util 567 :invoke :add 4661 INFO [2022-01-10 08:39:26,014] jepsen worker 212 - jepsen.util 212 :invoke :add 4662 INFO [2022-01-10 08:39:26,017] jepsen worker 526 - jepsen.util 526 :invoke :add 4663 INFO [2022-01-10 08:39:26,020] jepsen worker 250 - jepsen.util 250 :invoke :add 4664 INFO [2022-01-10 08:39:26,022] jepsen worker 842 - jepsen.util 842 :invoke :add 4665 INFO [2022-01-10 08:39:26,024] jepsen worker 774 - jepsen.util 774 :invoke :add 4666 INFO [2022-01-10 08:39:26,027] jepsen worker 48 - jepsen.util 48 :invoke :add 4667 INFO [2022-01-10 08:39:26,029] jepsen worker 467 - jepsen.util 467 :invoke :add 4668 INFO [2022-01-10 08:39:26,030] jepsen worker 842 - jepsen.util 842 :ok :add 4665 INFO [2022-01-10 08:39:26,031] jepsen worker 46 - jepsen.util 46 :invoke :add 4669 INFO [2022-01-10 08:39:26,032] jepsen worker 250 - jepsen.util 250 :ok :add 4664 INFO [2022-01-10 08:39:26,034] jepsen worker 651 - jepsen.util 651 :invoke :add 4670 INFO [2022-01-10 08:39:26,036] jepsen worker 646 - jepsen.util 646 :invoke :add 4671 INFO [2022-01-10 08:39:26,038] jepsen worker 390 - jepsen.util 390 :invoke :add 4672 INFO [2022-01-10 08:39:26,039] jepsen worker 774 - jepsen.util 774 :ok :add 4666 INFO [2022-01-10 08:39:26,040] jepsen worker 310 - jepsen.util 310 :invoke :add 4673 INFO [2022-01-10 08:39:26,041] jepsen worker 48 - jepsen.util 48 :ok :add 4667 INFO [2022-01-10 08:39:26,043] jepsen worker 653 - jepsen.util 653 :invoke :add 4674 INFO [2022-01-10 08:39:26,045] jepsen worker 401 - jepsen.util 401 :invoke :add 4675 INFO [2022-01-10 08:39:26,047] jepsen worker 561 - jepsen.util 561 :invoke :add 4676 INFO [2022-01-10 08:39:26,050] jepsen worker 638 - jepsen.util 638 :invoke :add 4677 INFO [2022-01-10 08:39:26,052] jepsen worker 986 - jepsen.util 986 :invoke :add 4678 INFO [2022-01-10 08:39:26,054] jepsen worker 243 - jepsen.util 243 :ok :add 4554 INFO [2022-01-10 08:39:26,054] jepsen worker 589 - jepsen.util 589 :ok :add 4560 INFO [2022-01-10 08:39:26,054] jepsen worker 378 - jepsen.util 378 :ok :add 4578 INFO [2022-01-10 08:39:26,054] jepsen worker 5 - jepsen.util 5 :ok :add 4609 INFO [2022-01-10 08:39:26,055] jepsen worker 204 - jepsen.util 204 :ok :add 4611 INFO [2022-01-10 08:39:26,055] jepsen worker 965 - jepsen.util 965 :ok :add 4625 INFO [2022-01-10 08:39:26,055] jepsen worker 728 - jepsen.util 728 :invoke :add 4679 INFO [2022-01-10 08:39:26,059] jepsen worker 116 - jepsen.util 116 :invoke :add 4680 INFO [2022-01-10 08:39:26,061] jepsen worker 623 - jepsen.util 623 :invoke :add 4681 INFO [2022-01-10 08:39:26,064] jepsen worker 452 - jepsen.util 452 :invoke :add 4682 INFO [2022-01-10 08:39:26,066] jepsen worker 618 - jepsen.util 618 :invoke :add 4683 INFO [2022-01-10 08:39:26,069] jepsen worker 801 - jepsen.util 801 :invoke :add 4684 INFO [2022-01-10 08:39:26,071] jepsen worker 163 - jepsen.util 163 :invoke :add 4685 INFO [2022-01-10 08:39:26,073] jepsen worker 940 - jepsen.util 940 :invoke :add 4686 INFO [2022-01-10 08:39:26,076] jepsen worker 591 - jepsen.util 591 :invoke :add 4687 INFO [2022-01-10 08:39:26,077] jepsen worker 801 - jepsen.util 801 :ok :add 4684 INFO [2022-01-10 08:39:26,077] jepsen worker 163 - jepsen.util 163 :ok :add 4685 INFO [2022-01-10 08:39:26,078] jepsen worker 132 - jepsen.util 132 :invoke :add 4688 INFO [2022-01-10 08:39:26,081] jepsen worker 890 - jepsen.util 890 :invoke :add 4689 INFO [2022-01-10 08:39:26,083] jepsen worker 586 - jepsen.util 586 :invoke :add 4690 INFO [2022-01-10 08:39:26,086] jepsen worker 997 - jepsen.util 997 :invoke :add 4691 INFO [2022-01-10 08:39:26,086] jepsen worker 591 - jepsen.util 591 :ok :add 4687 INFO [2022-01-10 08:39:26,087] jepsen worker 940 - jepsen.util 940 :ok :add 4686 INFO [2022-01-10 08:39:26,089] jepsen worker 5 - jepsen.util 5 :invoke :add 4692 INFO [2022-01-10 08:39:26,092] jepsen worker 447 - jepsen.util 447 :invoke :add 4693 INFO [2022-01-10 08:39:26,094] jepsen worker 177 - jepsen.util 177 :invoke :add 4694 INFO [2022-01-10 08:39:26,097] jepsen worker 645 - jepsen.util 645 :invoke :add 4695 INFO [2022-01-10 08:39:26,099] jepsen worker 96 - jepsen.util 96 :invoke :add 4696 INFO [2022-01-10 08:39:26,102] jepsen worker 995 - jepsen.util 995 :invoke :add 4697 INFO [2022-01-10 08:39:26,105] jepsen worker 181 - jepsen.util 181 :invoke :add 4698 INFO [2022-01-10 08:39:26,107] jepsen worker 621 - jepsen.util 621 :invoke :add 4699 INFO [2022-01-10 08:39:26,109] jepsen worker 412 - jepsen.util 412 :invoke :add 4700 INFO [2022-01-10 08:39:26,112] jepsen worker 84 - jepsen.util 84 :invoke :add 4701 INFO [2022-01-10 08:39:26,113] jepsen worker 93 - jepsen.util 93 :ok :add 4318 INFO [2022-01-10 08:39:26,113] jepsen worker 908 - jepsen.util 908 :ok :add 4334 INFO [2022-01-10 08:39:26,114] jepsen worker 366 - jepsen.util 366 :ok :add 4288 INFO [2022-01-10 08:39:26,114] jepsen worker 10 - jepsen.util 10 :ok :add 4300 INFO [2022-01-10 08:39:26,114] jepsen worker 540 - jepsen.util 540 :ok :add 4472 INFO [2022-01-10 08:39:26,114] jepsen worker 495 - jepsen.util 495 :ok :add 4290 INFO [2022-01-10 08:39:26,114] jepsen worker 216 - jepsen.util 216 :invoke :add 4702 INFO [2022-01-10 08:39:26,115] jepsen worker 898 - jepsen.util 898 :ok :add 4446 INFO [2022-01-10 08:39:26,118] jepsen worker 856 - jepsen.util 856 :ok :add 4613 INFO [2022-01-10 08:39:26,118] jepsen worker 154 - jepsen.util 154 :ok :add 4562 INFO [2022-01-10 08:39:26,118] jepsen worker 303 - jepsen.util 303 :ok :add 4627 INFO [2022-01-10 08:39:26,118] jepsen worker 368 - jepsen.util 368 :invoke :add 4703 INFO [2022-01-10 08:39:26,118] jepsen worker 858 - jepsen.util 858 :ok :add 4556 INFO [2022-01-10 08:39:26,119] jepsen worker 630 - jepsen.util 630 :ok :add 4291 INFO [2022-01-10 08:39:26,120] jepsen worker 246 - jepsen.util 246 :ok :add 4335 INFO [2022-01-10 08:39:26,120] jepsen worker 110 - jepsen.util 110 :ok :add 4561 INFO [2022-01-10 08:39:26,120] jepsen worker 683 - jepsen.util 683 :ok :add 4608 INFO [2022-01-10 08:39:26,120] jepsen worker 375 - jepsen.util 375 :ok :add 4555 INFO [2022-01-10 08:39:26,121] jepsen worker 549 - jepsen.util 549 :ok :add 4624 INFO [2022-01-10 08:39:26,120] jepsen worker 432 - jepsen.util 432 :ok :add 4473 INFO [2022-01-10 08:39:26,121] jepsen worker 641 - jepsen.util 641 :ok :add 4301 INFO [2022-01-10 08:39:26,120] jepsen worker 754 - jepsen.util 754 :ok :add 4610 INFO [2022-01-10 08:39:26,120] jepsen worker 533 - jepsen.util 533 :ok :add 4579 INFO [2022-01-10 08:39:26,121] jepsen worker 981 - jepsen.util 981 :ok :add 4447 INFO [2022-01-10 08:39:26,121] jepsen worker 95 - jepsen.util 95 :ok :add 4319 INFO [2022-01-10 08:39:26,121] jepsen worker 402 - jepsen.util 402 :ok :add 4289 INFO [2022-01-10 08:39:26,123] jepsen worker 900 - jepsen.util 900 :invoke :add 4704 INFO [2022-01-10 08:39:26,128] jepsen worker 570 - jepsen.util 570 :invoke :add 4705 INFO [2022-01-10 08:39:26,130] jepsen worker 144 - jepsen.util 144 :invoke :add 4706 INFO [2022-01-10 08:39:26,131] jepsen worker 122 - jepsen.util 122 :ok :add 4581 INFO [2022-01-10 08:39:26,131] jepsen worker 680 - jepsen.util 680 :ok :add 4599 INFO [2022-01-10 08:39:26,132] jepsen worker 134 - jepsen.util 134 :ok :add 4557 INFO [2022-01-10 08:39:26,132] jepsen worker 844 - jepsen.util 844 :ok :add 4563 INFO [2022-01-10 08:39:26,132] jepsen worker 66 - jepsen.util 66 :ok :add 4612 INFO [2022-01-10 08:39:26,132] jepsen worker 185 - jepsen.util 185 :ok :add 4626 INFO [2022-01-10 08:39:26,132] jepsen worker 872 - jepsen.util 872 :invoke :add 4707 INFO [2022-01-10 08:39:26,137] jepsen worker 169 - jepsen.util 169 :ok :add 4583 INFO [2022-01-10 08:39:26,138] jepsen worker 245 - jepsen.util 245 :invoke :add 4708 INFO [2022-01-10 08:39:26,138] jepsen worker 526 - jepsen.util 526 :ok :add 4663 INFO [2022-01-10 08:39:26,139] jepsen worker 944 - jepsen.util 944 :ok :add 4657 INFO [2022-01-10 08:39:26,141] jepsen worker 160 - jepsen.util 160 :invoke :add 4709 INFO [2022-01-10 08:39:26,144] jepsen worker 534 - jepsen.util 534 :invoke :add 4710 INFO [2022-01-10 08:39:26,145] jepsen worker 284 - jepsen.util 284 :ok :add 4580 INFO [2022-01-10 08:39:26,145] jepsen worker 732 - jepsen.util 732 :ok :add 4598 INFO [2022-01-10 08:39:26,146] jepsen worker 881 - jepsen.util 881 :invoke :add 4711 INFO [2022-01-10 08:39:26,149] jepsen worker 221 - jepsen.util 221 :invoke :add 4712 INFO [2022-01-10 08:39:26,151] jepsen worker 511 - jepsen.util 511 :ok :add 4582 INFO [2022-01-10 08:39:26,151] jepsen worker 774 - jepsen.util 774 :invoke :add 4713 INFO [2022-01-10 08:39:26,154] jepsen worker 788 - jepsen.util 788 :invoke :add 4714 INFO [2022-01-10 08:39:26,156] jepsen worker 705 - jepsen.util 705 :invoke :add 4715 INFO [2022-01-10 08:39:26,159] jepsen worker 549 - jepsen.util 549 :invoke :add 4716 INFO [2022-01-10 08:39:26,161] jepsen worker 788 - jepsen.util 788 :ok :add 4714 INFO [2022-01-10 08:39:26,162] jepsen worker 559 - jepsen.util 559 :invoke :add 4717 INFO [2022-01-10 08:39:26,165] jepsen worker 321 - jepsen.util 321 :invoke :add 4718 INFO [2022-01-10 08:39:26,165] jepsen worker 705 - jepsen.util 705 :ok :add 4715 INFO [2022-01-10 08:39:26,167] jepsen worker 174 - jepsen.util 174 :invoke :add 4719 INFO [2022-01-10 08:39:26,169] jepsen worker 559 - jepsen.util 559 :ok :add 4717 INFO [2022-01-10 08:39:26,170] jepsen worker 170 - jepsen.util 170 :invoke :add 4720 INFO [2022-01-10 08:39:26,173] jepsen worker 204 - jepsen.util 204 :invoke :add 4721 INFO [2022-01-10 08:39:26,175] jepsen worker 549 - jepsen.util 549 :ok :add 4716 INFO [2022-01-10 08:39:26,176] jepsen worker 685 - jepsen.util 685 :invoke :add 4722 INFO [2022-01-10 08:39:26,178] jepsen worker 947 - jepsen.util 947 :invoke :add 4723 INFO [2022-01-10 08:39:26,180] jepsen worker 720 - jepsen.util 720 :invoke :add 4724 INFO [2022-01-10 08:39:26,183] jepsen worker 324 - jepsen.util 324 :invoke :add 4725 INFO [2022-01-10 08:39:26,183] jepsen worker 170 - jepsen.util 170 :ok :add 4720 INFO [2022-01-10 08:39:26,186] jepsen worker 770 - jepsen.util 770 :invoke :add 4726 INFO [2022-01-10 08:39:26,187] jepsen worker 204 - jepsen.util 204 :ok :add 4721 INFO [2022-01-10 08:39:26,188] jepsen worker 685 - jepsen.util 685 :ok :add 4722 INFO [2022-01-10 08:39:26,189] jepsen worker 134 - jepsen.util 134 :invoke :add 4727 INFO [2022-01-10 08:39:26,192] jepsen worker 792 - jepsen.util 792 :invoke :add 4728 INFO [2022-01-10 08:39:26,193] jepsen worker 947 - jepsen.util 947 :ok :add 4723 INFO [2022-01-10 08:39:26,194] jepsen worker 899 - jepsen.util 899 :invoke :add 4729 INFO [2022-01-10 08:39:26,201] jepsen worker 981 - jepsen.util 981 :invoke :add 4731 INFO [2022-01-10 08:39:26,202] jepsen worker 783 - jepsen.util 783 :invoke :add 4730 INFO [2022-01-10 08:39:26,203] jepsen worker 718 - jepsen.util 718 :invoke :add 4732 INFO [2022-01-10 08:39:26,205] jepsen worker 41 - jepsen.util 41 :invoke :add 4733 INFO [2022-01-10 08:39:26,207] jepsen worker 472 - jepsen.util 472 :invoke :add 4734 INFO [2022-01-10 08:39:26,209] jepsen worker 173 - jepsen.util 173 :invoke :add 4735 INFO [2022-01-10 08:39:26,212] jepsen worker 857 - jepsen.util 857 :invoke :add 4736 INFO [2022-01-10 08:39:26,214] jepsen worker 286 - jepsen.util 286 :invoke :add 4737 INFO [2022-01-10 08:39:26,217] jepsen worker 643 - jepsen.util 643 :invoke :add 4738 INFO [2022-01-10 08:39:26,220] jepsen worker 186 - jepsen.util 186 :invoke :add 4739 INFO [2022-01-10 08:39:26,222] jepsen worker 640 - jepsen.util 640 :invoke :add 4740 INFO [2022-01-10 08:39:26,224] jepsen worker 643 - jepsen.util 643 :ok :add 4738 INFO [2022-01-10 08:39:26,225] jepsen worker 992 - jepsen.util 992 :invoke :add 4741 INFO [2022-01-10 08:39:26,226] jepsen worker 186 - jepsen.util 186 :ok :add 4739 INFO [2022-01-10 08:39:26,227] jepsen worker 583 - jepsen.util 583 :invoke :add 4742 INFO [2022-01-10 08:39:26,230] jepsen worker 722 - jepsen.util 722 :invoke :add 4743 INFO [2022-01-10 08:39:26,232] jepsen worker 349 - jepsen.util 349 :invoke :add 4744 INFO [2022-01-10 08:39:26,234] jepsen worker 167 - jepsen.util 167 :invoke :add 4745 INFO [2022-01-10 08:39:26,236] jepsen worker 20 - jepsen.util 20 :invoke :add 4746 INFO [2022-01-10 08:39:26,239] jepsen worker 484 - jepsen.util 484 :invoke :add 4747 INFO [2022-01-10 08:39:26,241] jepsen worker 357 - jepsen.util 357 :invoke :add 4748 INFO [2022-01-10 08:39:26,244] jepsen worker 180 - jepsen.util 180 :invoke :add 4749 INFO [2022-01-10 08:39:26,246] jepsen worker 338 - jepsen.util 338 :invoke :add 4750 INFO [2022-01-10 08:39:26,252] jepsen worker 574 - jepsen.util 574 :invoke :add 4751 INFO [2022-01-10 08:39:26,253] jepsen worker 1 - jepsen.util 1 :invoke :add 4752 INFO [2022-01-10 08:39:26,256] jepsen worker 171 - jepsen.util 171 :invoke :add 4753 INFO [2022-01-10 08:39:26,258] jepsen worker 874 - jepsen.util 874 :ok :add 4207 INFO [2022-01-10 08:39:26,258] jepsen worker 691 - jepsen.util 691 :ok :add 4233 INFO [2022-01-10 08:39:26,258] jepsen worker 130 - jepsen.util 130 :ok :add 4176 INFO [2022-01-10 08:39:26,258] jepsen worker 60 - jepsen.util 60 :ok :add 4559 INFO [2022-01-10 08:39:26,258] jepsen worker 555 - jepsen.util 555 :ok :add 4541 INFO [2022-01-10 08:39:26,258] jepsen worker 399 - jepsen.util 399 :ok :add 4575 INFO [2022-01-10 08:39:26,258] jepsen worker 256 - jepsen.util 256 :ok :add 4142 INFO [2022-01-10 08:39:26,258] jepsen worker 508 - jepsen.util 508 :ok :add 4604 INFO [2022-01-10 08:39:26,258] jepsen worker 656 - jepsen.util 656 :ok :add 4628 INFO [2022-01-10 08:39:26,258] jepsen worker 846 - jepsen.util 846 :ok :add 4630 INFO [2022-01-10 08:39:26,258] jepsen worker 471 - jepsen.util 471 :ok :add 4397 INFO [2022-01-10 08:39:26,259] jepsen worker 543 - jepsen.util 543 :invoke :add 4754 INFO [2022-01-10 08:39:26,263] jepsen worker 771 - jepsen.util 771 :invoke :add 4755 INFO [2022-01-10 08:39:26,264] jepsen worker 357 - jepsen.util 357 :ok :add 4748 INFO [2022-01-10 08:39:26,264] jepsen worker 338 - jepsen.util 338 :ok :add 4750 INFO [2022-01-10 08:39:26,265] jepsen worker 180 - jepsen.util 180 :ok :add 4749 INFO [2022-01-10 08:39:26,265] jepsen worker 244 - jepsen.util 244 :invoke :add 4756 INFO [2022-01-10 08:39:26,266] jepsen worker 574 - jepsen.util 574 :ok :add 4751 INFO [2022-01-10 08:39:26,270] jepsen worker 171 - jepsen.util 171 :ok :add 4753 INFO [2022-01-10 08:39:26,270] jepsen worker 648 - jepsen.util 648 :invoke :add 4757 INFO [2022-01-10 08:39:26,272] jepsen worker 1 - jepsen.util 1 :ok :add 4752 INFO [2022-01-10 08:39:26,273] jepsen worker 577 - jepsen.util 577 :invoke :add 4758 INFO [2022-01-10 08:39:26,275] jepsen worker 445 - jepsen.util 445 :invoke :add 4759 INFO [2022-01-10 08:39:26,278] jepsen worker 161 - jepsen.util 161 :invoke :add 4760 INFO [2022-01-10 08:39:26,281] jepsen worker 562 - jepsen.util 562 :invoke :add 4761 INFO [2022-01-10 08:39:26,284] jepsen worker 358 - jepsen.util 358 :invoke :add 4762 INFO [2022-01-10 08:39:26,286] jepsen worker 118 - jepsen.util 118 :ok :add 4655 INFO [2022-01-10 08:39:26,287] jepsen worker 728 - jepsen.util 728 :ok :add 4679 INFO [2022-01-10 08:39:26,287] jepsen worker 567 - jepsen.util 567 :ok :add 4661 INFO [2022-01-10 08:39:26,287] jepsen worker 601 - jepsen.util 601 :invoke :add 4763 INFO [2022-01-10 08:39:26,287] jepsen worker 63 - jepsen.util 63 :ok :add 4654 INFO [2022-01-10 08:39:26,288] jepsen worker 128 - jepsen.util 128 :ok :add 4660 INFO [2022-01-10 08:39:26,288] jepsen worker 986 - jepsen.util 986 :ok :add 4678 INFO [2022-01-10 08:39:26,289] jepsen worker 295 - jepsen.util 295 :ok :add 4304 INFO [2022-01-10 08:39:26,289] jepsen worker 995 - jepsen.util 995 :ok :add 4697 INFO [2022-01-10 08:39:26,289] jepsen worker 872 - jepsen.util 872 :ok :add 4707 INFO [2022-01-10 08:39:26,289] jepsen worker 942 - jepsen.util 942 :ok :add 4476 INFO [2022-01-10 08:39:26,290] jepsen worker 702 - jepsen.util 702 :invoke :add 4764 INFO [2022-01-10 08:39:26,290] jepsen worker 156 - jepsen.util 156 :ok :add 4294 INFO [2022-01-10 08:39:26,290] jepsen worker 117 - jepsen.util 117 :ok :add 4328 INFO [2022-01-10 08:39:26,291] jepsen worker 175 - jepsen.util 175 :ok :add 4330 INFO [2022-01-10 08:39:26,291] jepsen worker 369 - jepsen.util 369 :ok :add 4442 INFO [2022-01-10 08:39:26,294] jepsen worker 150 - jepsen.util 150 :invoke :add 4765 INFO [2022-01-10 08:39:26,296] jepsen worker 376 - jepsen.util 376 :invoke :add 4766 INFO [2022-01-10 08:39:26,298] jepsen worker 989 - jepsen.util 989 :invoke :add 4767 INFO [2022-01-10 08:39:26,301] jepsen worker 708 - jepsen.util 708 :invoke :add 4768 INFO [2022-01-10 08:39:26,301] jepsen worker 702 - jepsen.util 702 :ok :add 4764 INFO [2022-01-10 08:39:26,303] jepsen worker 876 - jepsen.util 876 :invoke :add 4769 INFO [2022-01-10 08:39:26,307] jepsen worker 53 - jepsen.util 53 :invoke :add 4770 INFO [2022-01-10 08:39:26,310] jepsen worker 97 - jepsen.util 97 :invoke :add 4771 INFO [2022-01-10 08:39:26,311] jepsen worker 376 - jepsen.util 376 :ok :add 4766 INFO [2022-01-10 08:39:26,314] jepsen worker 186 - jepsen.util 186 :invoke :add 4772 INFO [2022-01-10 08:39:26,315] jepsen worker 150 - jepsen.util 150 :ok :add 4765 INFO [2022-01-10 08:39:26,318] jepsen worker 870 - jepsen.util 870 :invoke :add 4773 INFO [2022-01-10 08:39:26,320] jepsen worker 989 - jepsen.util 989 :ok :add 4767 INFO [2022-01-10 08:39:26,321] jepsen worker 772 - jepsen.util 772 :invoke :add 4774 INFO [2022-01-10 08:39:26,323] jepsen worker 519 - jepsen.util 519 :invoke :add 4775 INFO [2022-01-10 08:39:26,326] jepsen worker 146 - jepsen.util 146 :invoke :add 4776 INFO [2022-01-10 08:39:26,329] jepsen worker 859 - jepsen.util 859 :invoke :add 4777 INFO [2022-01-10 08:39:26,332] jepsen worker 412 - jepsen.util 412 :ok :add 4700 INFO [2022-01-10 08:39:26,333] jepsen worker 188 - jepsen.util 188 :ok :add 4337 INFO [2022-01-10 08:39:26,333] jepsen worker 289 - jepsen.util 289 :ok :add 4293 INFO [2022-01-10 08:39:26,333] jepsen worker 328 - jepsen.util 328 :ok :add 4469 INFO [2022-01-10 08:39:26,333] jepsen worker 132 - jepsen.util 132 :ok :add 4688 INFO [2022-01-10 08:39:26,333] jepsen worker 231 - jepsen.util 231 :ok :add 4303 INFO [2022-01-10 08:39:26,333] jepsen worker 855 - jepsen.util 855 :invoke :add 4778 INFO [2022-01-10 08:39:26,333] jepsen worker 804 - jepsen.util 804 :ok :add 4471 INFO [2022-01-10 08:39:26,333] jepsen worker 205 - jepsen.util 205 :ok :add 4445 INFO [2022-01-10 08:39:26,333] jepsen worker 586 - jepsen.util 586 :ok :add 4690 INFO [2022-01-10 08:39:26,337] jepsen worker 785 - jepsen.util 785 :ok :add 4302 INFO [2022-01-10 08:39:26,337] jepsen worker 998 - jepsen.util 998 :ok :add 4336 INFO [2022-01-10 08:39:26,337] jepsen worker 979 - jepsen.util 979 :ok :add 4468 INFO [2022-01-10 08:39:26,337] jepsen worker 625 - jepsen.util 625 :ok :add 4292 INFO [2022-01-10 08:39:26,337] jepsen worker 890 - jepsen.util 890 :ok :add 4689 INFO [2022-01-10 08:39:26,337] jepsen worker 152 - jepsen.util 152 :ok :add 4444 INFO [2022-01-10 08:39:26,337] jepsen worker 407 - jepsen.util 407 :ok :add 4470 INFO [2022-01-10 08:39:26,337] jepsen worker 997 - jepsen.util 997 :ok :add 4691 INFO [2022-01-10 08:39:26,338] jepsen worker 913 - jepsen.util 913 :invoke :add 4779 INFO [2022-01-10 08:39:26,342] jepsen worker 278 - jepsen.util 278 :invoke :add 4780 INFO [2022-01-10 08:39:26,344] jepsen worker 680 - jepsen.util 680 :invoke :add 4781 INFO [2022-01-10 08:39:26,348] jepsen worker 263 - jepsen.util 263 :invoke :add 4782 INFO [2022-01-10 08:39:26,351] jepsen worker 998 - jepsen.util 998 :invoke :add 4783 INFO [2022-01-10 08:39:26,354] jepsen worker 936 - jepsen.util 936 :invoke :add 4784 INFO [2022-01-10 08:39:26,356] jepsen worker 462 - jepsen.util 462 :invoke :add 4785 INFO [2022-01-10 08:39:26,359] jepsen worker 549 - jepsen.util 549 :invoke :add 4786 INFO [2022-01-10 08:39:26,361] jepsen worker 936 - jepsen.util 936 :ok :add 4784 INFO [2022-01-10 08:39:26,362] jepsen worker 298 - jepsen.util 298 :invoke :add 4787 INFO [2022-01-10 08:39:26,362] jepsen worker 462 - jepsen.util 462 :ok :add 4785 INFO [2022-01-10 08:39:26,366] jepsen worker 374 - jepsen.util 374 :invoke :add 4788 INFO [2022-01-10 08:39:26,368] jepsen worker 549 - jepsen.util 549 :ok :add 4786 INFO [2022-01-10 08:39:26,369] jepsen worker 403 - jepsen.util 403 :invoke :add 4789 INFO [2022-01-10 08:39:26,372] jepsen worker 958 - jepsen.util 958 :invoke :add 4790 INFO [2022-01-10 08:39:26,373] jepsen worker 298 - jepsen.util 298 :ok :add 4787 INFO [2022-01-10 08:39:26,375] jepsen worker 541 - jepsen.util 541 :invoke :add 4791 INFO [2022-01-10 08:39:26,382] jepsen worker 166 - jepsen.util 166 :invoke :add 4792 INFO [2022-01-10 08:39:26,384] jepsen worker 980 - jepsen.util 980 :invoke :add 4793 INFO [2022-01-10 08:39:26,386] jepsen worker 587 - jepsen.util 587 :invoke :add 4794 INFO [2022-01-10 08:39:26,388] jepsen worker 282 - jepsen.util 282 :invoke :add 4795 INFO [2022-01-10 08:39:26,391] jepsen worker 844 - jepsen.util 844 :invoke :add 4796 INFO [2022-01-10 08:39:26,393] jepsen worker 856 - jepsen.util 856 :invoke :add 4797 INFO [2022-01-10 08:39:26,396] jepsen worker 120 - jepsen.util 120 :invoke :add 4798 INFO [2022-01-10 08:39:26,399] jepsen worker 237 - jepsen.util 237 :invoke :add 4799 INFO [2022-01-10 08:39:26,399] jepsen worker 709 - jepsen.util 709 :ok :add 4656 INFO [2022-01-10 08:39:26,399] jepsen worker 212 - jepsen.util 212 :ok :add 4662 INFO [2022-01-10 08:39:26,402] jepsen worker 16 - jepsen.util 16 :invoke :add 4800 INFO [2022-01-10 08:39:26,403] jepsen worker 116 - jepsen.util 116 :ok :add 4680 INFO [2022-01-10 08:39:26,403] jepsen worker 720 - jepsen.util 720 :ok :add 4724 INFO [2022-01-10 08:39:26,403] jepsen worker 245 - jepsen.util 245 :ok :add 4708 INFO [2022-01-10 08:39:26,403] jepsen worker 534 - jepsen.util 534 :ok :add 4710 INFO [2022-01-10 08:39:26,403] jepsen worker 181 - jepsen.util 181 :ok :add 4698 INFO [2022-01-10 08:39:26,405] jepsen worker 494 - jepsen.util 494 :ok :add 4231 INFO [2022-01-10 08:39:26,405] jepsen worker 190 - jepsen.util 190 :ok :add 4543 INFO [2022-01-10 08:39:26,405] jepsen worker 963 - jepsen.util 963 :ok :add 4140 INFO [2022-01-10 08:39:26,405] jepsen worker 632 - jepsen.util 632 :ok :add 4229 INFO [2022-01-10 08:39:26,405] jepsen worker 88 - jepsen.util 88 :ok :add 4158 INFO [2022-01-10 08:39:26,405] jepsen worker 975 - jepsen.util 975 :ok :add 4395 INFO [2022-01-10 08:39:26,405] jepsen worker 334 - jepsen.util 334 :ok :add 4174 INFO [2022-01-10 08:39:26,405] jepsen worker 482 - jepsen.util 482 :ok :add 4205 INFO [2022-01-10 08:39:26,405] jepsen worker 912 - jepsen.util 912 :ok :add 4577 INFO [2022-01-10 08:39:26,405] jepsen worker 262 - jepsen.util 262 :ok :add 4606 INFO [2022-01-10 08:39:26,405] jepsen worker 42 - jepsen.util 42 :ok :add 4632 INFO [2022-01-10 08:39:26,406] jepsen worker 277 - jepsen.util 277 :invoke :add 4801 INFO [2022-01-10 08:39:26,412] jepsen worker 714 - jepsen.util 714 :invoke :add 4802 INFO [2022-01-10 08:39:26,415] jepsen worker 632 - jepsen.util 632 :invoke :add 4803 INFO [2022-01-10 08:39:26,416] jepsen worker 680 - jepsen.util 680 :ok :add 4781 INFO [2022-01-10 08:39:26,417] jepsen worker 550 - jepsen.util 550 :invoke :add 4804 INFO [2022-01-10 08:39:26,417] jepsen worker 221 - jepsen.util 221 :ok :add 4712 INFO [2022-01-10 08:39:26,417] jepsen worker 452 - jepsen.util 452 :ok :add 4682 INFO [2022-01-10 08:39:26,418] jepsen worker 770 - jepsen.util 770 :ok :add 4726 INFO [2022-01-10 08:39:26,420] jepsen worker 712 - jepsen.util 712 :invoke :add 4805 INFO [2022-01-10 08:39:26,424] jepsen worker 320 - jepsen.util 320 :invoke :add 4806 INFO [2022-01-10 08:39:26,426] jepsen worker 550 - jepsen.util 550 :ok :add 4804 INFO [2022-01-10 08:39:26,427] jepsen worker 634 - jepsen.util 634 :invoke :add 4807 INFO [2022-01-10 08:39:26,429] jepsen worker 712 - jepsen.util 712 :ok :add 4805 INFO [2022-01-10 08:39:26,429] jepsen worker 592 - jepsen.util 592 :invoke :add 4808 INFO [2022-01-10 08:39:26,432] jepsen worker 371 - jepsen.util 371 :invoke :add 4809 INFO [2022-01-10 08:39:26,435] jepsen worker 281 - jepsen.util 281 :invoke :add 4810 INFO [2022-01-10 08:39:26,437] jepsen worker 320 - jepsen.util 320 :ok :add 4806 INFO [2022-01-10 08:39:26,439] jepsen worker 460 - jepsen.util 460 :invoke :add 4811 INFO [2022-01-10 08:39:26,440] jepsen worker 634 - jepsen.util 634 :ok :add 4807 INFO [2022-01-10 08:39:26,443] jepsen worker 788 - jepsen.util 788 :invoke :add 4812 INFO [2022-01-10 08:39:26,446] jepsen worker 436 - jepsen.util 436 :invoke :add 4813 INFO [2022-01-10 08:39:26,448] jepsen worker 357 - jepsen.util 357 :invoke :add 4814 INFO [2022-01-10 08:39:26,450] jepsen worker 210 - jepsen.util 210 :invoke :add 4815 INFO [2022-01-10 08:39:26,453] jepsen worker 379 - jepsen.util 379 :invoke :add 4816 INFO [2022-01-10 08:39:26,456] jepsen worker 489 - jepsen.util 489 :invoke :add 4817 INFO [2022-01-10 08:39:26,458] jepsen worker 623 - jepsen.util 623 :ok :add 4681 INFO [2022-01-10 08:39:26,458] jepsen worker 621 - jepsen.util 621 :ok :add 4699 INFO [2022-01-10 08:39:26,458] jepsen worker 16 - jepsen.util 16 :ok :add 4800 INFO [2022-01-10 08:39:26,458] jepsen worker 881 - jepsen.util 881 :ok :add 4711 INFO [2022-01-10 08:39:26,458] jepsen worker 324 - jepsen.util 324 :ok :add 4725 INFO [2022-01-10 08:39:26,458] jepsen worker 160 - jepsen.util 160 :ok :add 4709 INFO [2022-01-10 08:39:26,458] jepsen worker 342 - jepsen.util 342 :invoke :add 4818 INFO [2022-01-10 08:39:26,462] jepsen worker 959 - jepsen.util 959 :invoke :add 4819 INFO [2022-01-10 08:39:26,464] jepsen worker 370 - jepsen.util 370 :invoke :add 4820 INFO [2022-01-10 08:39:26,466] jepsen worker 754 - jepsen.util 754 :invoke :add 4821 INFO [2022-01-10 08:39:26,469] jepsen worker 760 - jepsen.util 760 :invoke :add 4822 INFO [2022-01-10 08:39:26,472] jepsen worker 637 - jepsen.util 637 :invoke :add 4823 INFO [2022-01-10 08:39:26,474] jepsen worker 548 - jepsen.util 548 :invoke :add 4824 INFO [2022-01-10 08:39:26,476] jepsen worker 894 - jepsen.util 894 :invoke :add 4825 INFO [2022-01-10 08:39:26,480] jepsen worker 49 - jepsen.util 49 :invoke :add 4826 INFO [2022-01-10 08:39:26,482] jepsen worker 110 - jepsen.util 110 :invoke :add 4827 INFO [2022-01-10 08:39:26,483] jepsen worker 774 - jepsen.util 774 :ok :add 4713 INFO [2022-01-10 08:39:26,483] jepsen worker 618 - jepsen.util 618 :ok :add 4683 INFO [2022-01-10 08:39:26,484] jepsen worker 134 - jepsen.util 134 :ok :add 4727 INFO [2022-01-10 08:39:26,484] jepsen worker 714 - jepsen.util 714 :ok :add 4802 INFO [2022-01-10 08:39:26,485] jepsen worker 545 - jepsen.util 545 :invoke :add 4828 INFO [2022-01-10 08:39:26,489] jepsen worker 688 - jepsen.util 688 :invoke :add 4829 INFO [2022-01-10 08:39:26,492] jepsen worker 728 - jepsen.util 728 :invoke :add 4830 INFO [2022-01-10 08:39:26,494] jepsen worker 221 - jepsen.util 221 :invoke :add 4831 INFO [2022-01-10 08:39:26,497] jepsen worker 986 - jepsen.util 986 :invoke :add 4832 INFO [2022-01-10 08:39:26,499] jepsen worker 478 - jepsen.util 478 :invoke :add 4833 INFO [2022-01-10 08:39:26,501] jepsen worker 346 - jepsen.util 346 :invoke :add 4834 INFO [2022-01-10 08:39:26,504] jepsen worker 191 - jepsen.util 191 :invoke :add 4835 INFO [2022-01-10 08:39:26,506] jepsen worker 565 - jepsen.util 565 :invoke :add 4836 INFO [2022-01-10 08:39:26,508] jepsen worker 908 - jepsen.util 908 :invoke :add 4837 INFO [2022-01-10 08:39:26,511] jepsen worker 998 - jepsen.util 998 :ok :add 4783 INFO [2022-01-10 08:39:26,511] jepsen worker 697 - jepsen.util 697 :invoke :add 4838 INFO [2022-01-10 08:39:26,513] jepsen worker 801 - jepsen.util 801 :invoke :add 4839 INFO [2022-01-10 08:39:26,516] jepsen worker 710 - jepsen.util 710 :invoke :add 4840 INFO [2022-01-10 08:39:26,518] jepsen worker 659 - jepsen.util 659 :invoke :add 4841 INFO [2022-01-10 08:39:26,521] jepsen worker 40 - jepsen.util 40 :invoke :add 4842 INFO [2022-01-10 08:39:26,523] jepsen worker 726 - jepsen.util 726 :invoke :add 4843 INFO [2022-01-10 08:39:26,526] jepsen worker 448 - jepsen.util 448 :invoke :add 4844 INFO [2022-01-10 08:39:26,528] jepsen worker 264 - jepsen.util 264 :invoke :add 4845 INFO [2022-01-10 08:39:26,531] jepsen worker 890 - jepsen.util 890 :invoke :add 4846 INFO [2022-01-10 08:39:26,533] jepsen worker 134 - jepsen.util 134 :invoke :add 4847 INFO [2022-01-10 08:39:26,535] jepsen worker 671 - jepsen.util 671 :invoke :add 4848 INFO [2022-01-10 08:39:26,537] jepsen worker 66 - jepsen.util 66 :invoke :add 4849 INFO [2022-01-10 08:39:26,540] jepsen worker 301 - jepsen.util 301 :invoke :add 4850 INFO [2022-01-10 08:39:26,541] jepsen worker 545 - jepsen.util 545 :ok :add 4828 INFO [2022-01-10 08:39:26,541] jepsen worker 688 - jepsen.util 688 :ok :add 4829 INFO [2022-01-10 08:39:26,542] jepsen worker 694 - jepsen.util 694 :invoke :add 4851 INFO [2022-01-10 08:39:26,545] jepsen worker 822 - jepsen.util 822 :invoke :add 4852 INFO [2022-01-10 08:39:26,546] jepsen worker 221 - jepsen.util 221 :ok :add 4831 INFO [2022-01-10 08:39:26,546] jepsen worker 728 - jepsen.util 728 :ok :add 4830 INFO [2022-01-10 08:39:26,547] jepsen worker 989 - jepsen.util 989 :invoke :add 4853 INFO [2022-01-10 08:39:26,550] jepsen worker 188 - jepsen.util 188 :invoke :add 4854 INFO [2022-01-10 08:39:26,550] jepsen worker 986 - jepsen.util 986 :ok :add 4832 INFO [2022-01-10 08:39:26,551] jepsen worker 478 - jepsen.util 478 :ok :add 4833 INFO [2022-01-10 08:39:26,552] jepsen worker 931 - jepsen.util 931 :invoke :add 4855 INFO [2022-01-10 08:39:26,554] jepsen worker 710 - jepsen.util 710 :ok :add 4840 INFO [2022-01-10 08:39:26,555] jepsen worker 724 - jepsen.util 724 :invoke :add 4856 INFO [2022-01-10 08:39:26,558] jepsen worker 95 - jepsen.util 95 :invoke :add 4857 INFO [2022-01-10 08:39:26,560] jepsen worker 10 - jepsen.util 10 :invoke :add 4858 INFO [2022-01-10 08:39:26,562] jepsen worker 508 - jepsen.util 508 :invoke :add 4859 INFO [2022-01-10 08:39:26,565] jepsen worker 978 - jepsen.util 978 :invoke :add 4860 INFO [2022-01-10 08:39:26,567] jepsen worker 707 - jepsen.util 707 :invoke :add 4861 INFO [2022-01-10 08:39:26,569] jepsen worker 163 - jepsen.util 163 :invoke :add 4862 INFO [2022-01-10 08:39:26,571] jepsen worker 490 - jepsen.util 490 :invoke :add 4863 INFO [2022-01-10 08:39:26,573] jepsen worker 204 - jepsen.util 204 :invoke :add 4864 INFO [2022-01-10 08:39:26,576] jepsen worker 479 - jepsen.util 479 :invoke :add 4865 INFO [2022-01-10 08:39:26,578] jepsen worker 939 - jepsen.util 939 :invoke :add 4866 INFO [2022-01-10 08:39:26,580] jepsen worker 590 - jepsen.util 590 :invoke :add 4867 INFO [2022-01-10 08:39:26,583] jepsen worker 779 - jepsen.util 779 :invoke :add 4868 INFO [2022-01-10 08:39:26,585] jepsen worker 998 - jepsen.util 998 :invoke :add 4869 INFO [2022-01-10 08:39:26,588] jepsen worker 81 - jepsen.util 81 :invoke :add 4870 INFO [2022-01-10 08:39:26,590] jepsen worker 322 - jepsen.util 322 :invoke :add 4871 INFO [2022-01-10 08:39:26,592] jepsen worker 355 - jepsen.util 355 :invoke :add 4872 INFO [2022-01-10 08:39:26,595] jepsen worker 393 - jepsen.util 393 :invoke :add 4873 INFO [2022-01-10 08:39:26,597] jepsen worker 703 - jepsen.util 703 :invoke :add 4874 INFO [2022-01-10 08:39:26,599] jepsen worker 230 - jepsen.util 230 :invoke :add 4875 INFO [2022-01-10 08:39:26,601] jepsen worker 326 - jepsen.util 326 :invoke :add 4876 INFO [2022-01-10 08:39:26,603] jepsen worker 794 - jepsen.util 794 :invoke :add 4877 INFO [2022-01-10 08:39:26,604] jepsen worker 659 - jepsen.util 659 :ok :add 4841 INFO [2022-01-10 08:39:26,604] jepsen worker 40 - jepsen.util 40 :ok :add 4842 INFO [2022-01-10 08:39:26,605] jepsen worker 295 - jepsen.util 295 :invoke :add 4878 INFO [2022-01-10 08:39:26,608] jepsen worker 43 - jepsen.util 43 :invoke :add 4879 INFO [2022-01-10 08:39:26,608] jepsen worker 726 - jepsen.util 726 :ok :add 4843 INFO [2022-01-10 08:39:26,612] jepsen worker 31 - jepsen.util 31 :invoke :add 4880 INFO [2022-01-10 08:39:26,614] jepsen worker 680 - jepsen.util 680 :invoke :add 4881 INFO [2022-01-10 08:39:26,615] jepsen worker 161 - jepsen.util 161 :ok :add 4760 INFO [2022-01-10 08:39:26,615] jepsen worker 543 - jepsen.util 543 :ok :add 4754 INFO [2022-01-10 08:39:26,616] jepsen worker 855 - jepsen.util 855 :ok :add 4778 INFO [2022-01-10 08:39:26,617] jepsen worker 122 - jepsen.util 122 :invoke :add 4882 INFO [2022-01-10 08:39:26,621] jepsen worker 777 - jepsen.util 777 :invoke :add 4883 INFO [2022-01-10 08:39:26,624] jepsen worker 602 - jepsen.util 602 :invoke :add 4884 INFO [2022-01-10 08:39:26,626] jepsen worker 825 - jepsen.util 825 :invoke :add 4885 INFO [2022-01-10 08:39:26,629] jepsen worker 82 - jepsen.util 82 :invoke :add 4886 INFO [2022-01-10 08:39:26,631] jepsen worker 399 - jepsen.util 399 :invoke :add 4887 INFO [2022-01-10 08:39:26,633] jepsen worker 199 - jepsen.util 199 :invoke :add 4888 INFO [2022-01-10 08:39:26,635] jepsen worker 431 - jepsen.util 431 :invoke :add 4889 INFO [2022-01-10 08:39:26,638] jepsen worker 862 - jepsen.util 862 :invoke :add 4890 INFO [2022-01-10 08:39:26,641] jepsen worker 944 - jepsen.util 944 :invoke :add 4891 INFO [2022-01-10 08:39:26,644] jepsen worker 746 - jepsen.util 746 :invoke :add 4892 INFO [2022-01-10 08:39:26,644] jepsen worker 10 - jepsen.util 10 :ok :add 4858 INFO [2022-01-10 08:39:26,644] jepsen worker 663 - jepsen.util 663 :ok :add 4277 INFO [2022-01-10 08:39:26,644] jepsen worker 653 - jepsen.util 653 :ok :add 4674 INFO [2022-01-10 08:39:26,644] jepsen worker 703 - jepsen.util 703 :ok :add 4874 INFO [2022-01-10 08:39:26,644] jepsen worker 553 - jepsen.util 553 :ok :add 4505 INFO [2022-01-10 08:39:26,645] jepsen worker 848 - jepsen.util 848 :ok :add 4640 INFO [2022-01-10 08:39:26,645] jepsen worker 260 - jepsen.util 260 :ok :add 4531 INFO [2022-01-10 08:39:26,645] jepsen worker 512 - jepsen.util 512 :ok :add 4243 INFO [2022-01-10 08:39:26,645] jepsen worker 239 - jepsen.util 239 :ok :add 4495 INFO [2022-01-10 08:39:26,645] jepsen worker 425 - jepsen.util 425 :ok :add 4529 INFO [2022-01-10 08:39:26,645] jepsen worker 151 - jepsen.util 151 :ok :add 4658 INFO [2022-01-10 08:39:26,646] jepsen worker 974 - jepsen.util 974 :invoke :add 4893 INFO [2022-01-10 08:39:26,652] jepsen worker 320 - jepsen.util 320 :invoke :add 4894 INFO [2022-01-10 08:39:26,654] jepsen worker 613 - jepsen.util 613 :invoke :add 4895 INFO [2022-01-10 08:39:26,656] jepsen worker 966 - jepsen.util 966 :invoke :add 4896 INFO [2022-01-10 08:39:26,658] jepsen worker 98 - jepsen.util 98 :invoke :add 4897 INFO [2022-01-10 08:39:26,661] jepsen worker 72 - jepsen.util 72 :invoke :add 4898 INFO [2022-01-10 08:39:26,662] jepsen worker 508 - jepsen.util 508 :ok :add 4859 INFO [2022-01-10 08:39:26,663] jepsen worker 230 - jepsen.util 230 :ok :add 4875 INFO [2022-01-10 08:39:26,663] jepsen worker 213 - jepsen.util 213 :invoke :add 4899 INFO [2022-01-10 08:39:26,665] jepsen worker 278 - jepsen.util 278 :ok :add 4780 INFO [2022-01-10 08:39:26,665] jepsen worker 120 - jepsen.util 120 :ok :add 4798 INFO [2022-01-10 08:39:26,667] jepsen worker 419 - jepsen.util 419 :invoke :add 4900 INFO [2022-01-10 08:39:26,669] jepsen worker 794 - jepsen.util 794 :ok :add 4877 INFO [2022-01-10 08:39:26,671] jepsen worker 725 - jepsen.util 725 :invoke :add 4901 INFO [2022-01-10 08:39:26,673] jepsen worker 397 - jepsen.util 397 :invoke :add 4902 INFO [2022-01-10 08:39:26,676] jepsen worker 24 - jepsen.util 24 :invoke :add 4903 INFO [2022-01-10 08:39:26,678] jepsen worker 926 - jepsen.util 926 :invoke :add 4904 INFO [2022-01-10 08:39:26,680] jepsen worker 653 - jepsen.util 653 :invoke :add 4905 INFO [2022-01-10 08:39:26,682] jepsen worker 618 - jepsen.util 618 :invoke :add 4906 INFO [2022-01-10 08:39:26,684] jepsen worker 584 - jepsen.util 584 :invoke :add 4907 INFO [2022-01-10 08:39:26,687] jepsen worker 853 - jepsen.util 853 :invoke :add 4908 INFO [2022-01-10 08:39:26,689] jepsen worker 768 - jepsen.util 768 :invoke :add 4909 INFO [2022-01-10 08:39:26,691] jepsen worker 353 - jepsen.util 353 :invoke :add 4910 INFO [2022-01-10 08:39:26,693] jepsen worker 685 - jepsen.util 685 :invoke :add 4911 INFO [2022-01-10 08:39:26,696] jepsen worker 435 - jepsen.util 435 :invoke :add 4912 INFO [2022-01-10 08:39:26,698] jepsen worker 964 - jepsen.util 964 :invoke :add 4913 INFO [2022-01-10 08:39:26,700] jepsen worker 904 - jepsen.util 904 :invoke :add 4914 INFO [2022-01-10 08:39:26,702] jepsen worker 496 - jepsen.util 496 :invoke :add 4915 INFO [2022-01-10 08:39:26,703] jepsen worker 326 - jepsen.util 326 :ok :add 4876 INFO [2022-01-10 08:39:26,704] jepsen worker 716 - jepsen.util 716 :invoke :add 4916 INFO [2022-01-10 08:39:26,704] jepsen worker 320 - jepsen.util 320 :ok :add 4894 INFO [2022-01-10 08:39:26,707] jepsen worker 68 - jepsen.util 68 :invoke :add 4917 INFO [2022-01-10 08:39:26,708] jepsen worker 618 - jepsen.util 618 :ok :add 4906 INFO [2022-01-10 08:39:26,708] jepsen worker 966 - jepsen.util 966 :ok :add 4896 INFO [2022-01-10 08:39:26,709] jepsen worker 531 - jepsen.util 531 :invoke :add 4918 INFO [2022-01-10 08:39:26,712] jepsen worker 185 - jepsen.util 185 :invoke :add 4919 INFO [2022-01-10 08:39:26,716] jepsen worker 406 - jepsen.util 406 :invoke :add 4920 INFO [2022-01-10 08:39:26,718] jepsen worker 115 - jepsen.util 115 :invoke :add 4921 INFO [2022-01-10 08:39:26,721] jepsen worker 149 - jepsen.util 149 :ok :add 4369 INFO [2022-01-10 08:39:26,721] jepsen worker 299 - jepsen.util 299 :invoke :add 4922 INFO [2022-01-10 08:39:26,721] jepsen worker 71 - jepsen.util 71 :ok :add 4345 INFO [2022-01-10 08:39:26,721] jepsen worker 75 - jepsen.util 75 :ok :add 4437 INFO [2022-01-10 08:39:26,721] jepsen worker 140 - jepsen.util 140 :ok :add 4593 INFO [2022-01-10 08:39:26,721] jepsen worker 564 - jepsen.util 564 :ok :add 4371 INFO [2022-01-10 08:39:26,721] jepsen worker 20 - jepsen.util 20 :ok :add 4746 INFO [2022-01-10 08:39:26,722] jepsen worker 506 - jepsen.util 506 :ok :add 4403 INFO [2022-01-10 08:39:26,726] jepsen worker 247 - jepsen.util 247 :invoke :add 4923 INFO [2022-01-10 08:39:26,728] jepsen worker 474 - jepsen.util 474 :invoke :add 4924 INFO [2022-01-10 08:39:26,730] jepsen worker 487 - jepsen.util 487 :invoke :add 4925 INFO [2022-01-10 08:39:26,732] jepsen worker 682 - jepsen.util 682 :invoke :add 4926 INFO [2022-01-10 08:39:26,735] jepsen worker 954 - jepsen.util 954 :invoke :add 4927 INFO [2022-01-10 08:39:26,737] jepsen worker 273 - jepsen.util 273 :invoke :add 4928 INFO [2022-01-10 08:39:26,739] jepsen worker 345 - jepsen.util 345 :invoke :add 4929 INFO [2022-01-10 08:39:26,743] jepsen worker 563 - jepsen.util 563 :invoke :add 4930 INFO [2022-01-10 08:39:26,746] jepsen worker 896 - jepsen.util 896 :invoke :add 4931 INFO [2022-01-10 08:39:26,749] jepsen worker 591 - jepsen.util 591 :invoke :add 4932 INFO [2022-01-10 08:39:26,751] jepsen worker 9 - jepsen.util 9 :invoke :add 4933 INFO [2022-01-10 08:39:26,755] jepsen worker 647 - jepsen.util 647 :invoke :add 4934 INFO [2022-01-10 08:39:26,757] jepsen worker 222 - jepsen.util 222 :invoke :add 4935 INFO [2022-01-10 08:39:26,759] jepsen worker 291 - jepsen.util 291 :invoke :add 4936 INFO [2022-01-10 08:39:26,761] jepsen worker 780 - jepsen.util 780 :invoke :add 4937 INFO [2022-01-10 08:39:26,763] jepsen worker 350 - jepsen.util 350 :invoke :add 4938 INFO [2022-01-10 08:39:26,767] jepsen worker 364 - jepsen.util 364 :invoke :add 4939 INFO [2022-01-10 08:39:26,767] jepsen worker 134 - jepsen.util 134 :ok :add 4847 INFO [2022-01-10 08:39:26,767] jepsen worker 393 - jepsen.util 393 :ok :add 4873 INFO [2022-01-10 08:39:26,767] jepsen worker 358 - jepsen.util 358 :ok :add 4762 INFO [2022-01-10 08:39:26,767] jepsen worker 244 - jepsen.util 244 :ok :add 4756 INFO [2022-01-10 08:39:26,770] jepsen worker 750 - jepsen.util 750 :invoke :add 4940 INFO [2022-01-10 08:39:26,770] jepsen worker 653 - jepsen.util 653 :ok :add 4905 INFO [2022-01-10 08:39:26,770] jepsen worker 613 - jepsen.util 613 :ok :add 4895 INFO [2022-01-10 08:39:26,775] jepsen worker 182 - jepsen.util 182 :invoke :add 4941 INFO [2022-01-10 08:39:26,776] jepsen worker 584 - jepsen.util 584 :ok :add 4907 INFO [2022-01-10 08:39:26,776] jepsen worker 9 - jepsen.util 9 :ok :add 4933 INFO [2022-01-10 08:39:26,776] jepsen worker 98 - jepsen.util 98 :ok :add 4897 INFO [2022-01-10 08:39:26,778] jepsen worker 187 - jepsen.util 187 :invoke :add 4942 INFO [2022-01-10 08:39:26,784] jepsen worker 417 - jepsen.util 417 :invoke :add 4943 INFO [2022-01-10 08:39:26,785] jepsen worker 273 - jepsen.util 273 :ok :add 4928 INFO [2022-01-10 08:39:26,785] jepsen worker 926 - jepsen.util 926 :ok :add 4904 INFO [2022-01-10 08:39:26,787] jepsen worker 832 - jepsen.util 832 :invoke :add 4944 INFO [2022-01-10 08:39:26,788] jepsen worker 563 - jepsen.util 563 :ok :add 4930 INFO [2022-01-10 08:39:26,788] jepsen worker 763 - jepsen.util 763 :ok :add 4404 INFO [2022-01-10 08:39:26,788] jepsen worker 808 - jepsen.util 808 :ok :add 4342 INFO [2022-01-10 08:39:26,788] jepsen worker 988 - jepsen.util 988 :ok :add 4376 INFO [2022-01-10 08:39:26,788] jepsen worker 992 - jepsen.util 992 :ok :add 4741 INFO [2022-01-10 08:39:26,788] jepsen worker 445 - jepsen.util 445 :ok :add 4759 INFO [2022-01-10 08:39:26,789] jepsen worker 207 - jepsen.util 207 :ok :add 4430 INFO [2022-01-10 08:39:26,790] jepsen worker 276 - jepsen.util 276 :ok :add 4428 INFO [2022-01-10 08:39:26,790] jepsen worker 795 - jepsen.util 795 :ok :add 4594 INFO [2022-01-10 08:39:26,791] jepsen worker 71 - jepsen.util 71 :invoke :add 4945 INFO [2022-01-10 08:39:26,792] jepsen worker 591 - jepsen.util 591 :ok :add 4932 INFO [2022-01-10 08:39:26,794] jepsen worker 345 - jepsen.util 345 :ok :add 4929 INFO [2022-01-10 08:39:26,794] jepsen worker 896 - jepsen.util 896 :ok :add 4931 INFO [2022-01-10 08:39:26,794] jepsen worker 263 - jepsen.util 263 :ok :add 4782 INFO [2022-01-10 08:39:26,794] jepsen worker 974 - jepsen.util 974 :ok :add 4893 INFO [2022-01-10 08:39:26,796] jepsen worker 162 - jepsen.util 162 :invoke :add 4946 INFO [2022-01-10 08:39:26,799] jepsen worker 928 - jepsen.util 928 :invoke :add 4947 INFO [2022-01-10 08:39:26,801] jepsen worker 584 - jepsen.util 584 :invoke :add 4948 INFO [2022-01-10 08:39:26,804] jepsen worker 409 - jepsen.util 409 :invoke :add 4949 INFO [2022-01-10 08:39:26,806] jepsen worker 441 - jepsen.util 441 :invoke :add 4950 INFO [2022-01-10 08:39:26,809] jepsen worker 464 - jepsen.util 464 :invoke :add 4951 INFO [2022-01-10 08:39:26,811] jepsen worker 131 - jepsen.util 131 :invoke :add 4952 INFO [2022-01-10 08:39:26,813] jepsen worker 805 - jepsen.util 805 :invoke :add 4953 INFO [2022-01-10 08:39:26,815] jepsen worker 229 - jepsen.util 229 :invoke :add 4954 INFO [2022-01-10 08:39:26,817] jepsen worker 681 - jepsen.util 681 :invoke :add 4955 INFO [2022-01-10 08:39:26,819] jepsen worker 771 - jepsen.util 771 :ok :add 4755 INFO [2022-01-10 08:39:26,819] jepsen worker 779 - jepsen.util 779 :ok :add 4868 INFO [2022-01-10 08:39:26,820] jepsen worker 448 - jepsen.util 448 :ok :add 4844 INFO [2022-01-10 08:39:26,820] jepsen worker 913 - jepsen.util 913 :ok :add 4779 INFO [2022-01-10 08:39:26,820] jepsen worker 81 - jepsen.util 81 :ok :add 4870 INFO [2022-01-10 08:39:26,819] jepsen worker 562 - jepsen.util 562 :ok :add 4761 INFO [2022-01-10 08:39:26,820] jepsen worker 135 - jepsen.util 135 :invoke :add 4956 INFO [2022-01-10 08:39:26,822] jepsen worker 182 - jepsen.util 182 :ok :add 4941 INFO [2022-01-10 08:39:26,824] jepsen worker 376 - jepsen.util 376 :invoke :add 4957 INFO [2022-01-10 08:39:26,827] jepsen worker 869 - jepsen.util 869 :invoke :add 4958 INFO [2022-01-10 08:39:26,828] jepsen worker 750 - jepsen.util 750 :ok :add 4940 INFO [2022-01-10 08:39:26,830] jepsen worker 626 - jepsen.util 626 :invoke :add 4959 INFO [2022-01-10 08:39:26,833] jepsen worker 438 - jepsen.util 438 :invoke :add 4960 INFO [2022-01-10 08:39:26,836] jepsen worker 417 - jepsen.util 417 :ok :add 4943 INFO [2022-01-10 08:39:26,836] jepsen worker 494 - jepsen.util 494 :invoke :add 4961 INFO [2022-01-10 08:39:26,839] jepsen worker 67 - jepsen.util 67 :invoke :add 4962 INFO [2022-01-10 08:39:26,840] jepsen worker 187 - jepsen.util 187 :ok :add 4942 INFO [2022-01-10 08:39:26,841] jepsen worker 762 - jepsen.util 762 :invoke :add 4963 INFO [2022-01-10 08:39:26,844] jepsen worker 641 - jepsen.util 641 :invoke :add 4964 INFO [2022-01-10 08:39:26,846] jepsen worker 526 - jepsen.util 526 :invoke :add 4965 INFO [2022-01-10 08:39:26,849] jepsen worker 293 - jepsen.util 293 :invoke :add 4966 INFO [2022-01-10 08:39:26,849] jepsen worker 626 - jepsen.util 626 :ok :add 4959 INFO [2022-01-10 08:39:26,851] jepsen worker 205 - jepsen.util 205 :invoke :add 4967 INFO [2022-01-10 08:39:26,855] jepsen worker 529 - jepsen.util 529 :invoke :add 4968 INFO [2022-01-10 08:39:26,855] jepsen worker 869 - jepsen.util 869 :ok :add 4958 INFO [2022-01-10 08:39:26,858] jepsen worker 480 - jepsen.util 480 :invoke :add 4969 INFO [2022-01-10 08:39:26,858] jepsen worker 355 - jepsen.util 355 :ok :add 4872 INFO [2022-01-10 08:39:26,858] jepsen worker 601 - jepsen.util 601 :ok :add 4763 INFO [2022-01-10 08:39:26,858] jepsen worker 890 - jepsen.util 890 :ok :add 4846 INFO [2022-01-10 08:39:26,860] jepsen worker 648 - jepsen.util 648 :ok :add 4757 INFO [2022-01-10 08:39:26,861] jepsen worker 324 - jepsen.util 324 :invoke :add 4970 INFO [2022-01-10 08:39:26,865] jepsen worker 486 - jepsen.util 486 :invoke :add 4971 INFO [2022-01-10 08:39:26,868] jepsen worker 703 - jepsen.util 703 :invoke :add 4972 INFO [2022-01-10 08:39:26,870] jepsen worker 372 - jepsen.util 372 :invoke :add 4973 INFO [2022-01-10 08:39:26,873] jepsen worker 941 - jepsen.util 941 :invoke :add 4974 INFO [2022-01-10 08:39:26,875] jepsen worker 318 - jepsen.util 318 :invoke :add 4975 INFO [2022-01-10 08:39:26,878] jepsen worker 992 - jepsen.util 992 :invoke :add 4976 INFO [2022-01-10 08:39:26,882] jepsen worker 598 - jepsen.util 598 :invoke :add 4977 INFO [2022-01-10 08:39:26,882] jepsen worker 941 - jepsen.util 941 :ok :add 4974 INFO [2022-01-10 08:39:26,882] jepsen worker 318 - jepsen.util 318 :ok :add 4975 INFO [2022-01-10 08:39:26,884] jepsen worker 949 - jepsen.util 949 :invoke :add 4978 INFO [2022-01-10 08:39:26,886] jepsen worker 992 - jepsen.util 992 :ok :add 4976 INFO [2022-01-10 08:39:26,887] jepsen worker 513 - jepsen.util 513 :invoke :add 4979 INFO [2022-01-10 08:39:26,889] jepsen worker 598 - jepsen.util 598 :ok :add 4977 INFO [2022-01-10 08:39:26,890] jepsen worker 858 - jepsen.util 858 :invoke :add 4980 INFO [2022-01-10 08:39:26,893] jepsen worker 336 - jepsen.util 336 :invoke :add 4981 INFO [2022-01-10 08:39:26,894] jepsen worker 71 - jepsen.util 71 :ok :add 4945 INFO [2022-01-10 08:39:26,895] jepsen worker 202 - jepsen.util 202 :invoke :add 4982 INFO [2022-01-10 08:39:26,898] jepsen worker 656 - jepsen.util 656 :invoke :add 4983 INFO [2022-01-10 08:39:26,900] jepsen worker 886 - jepsen.util 886 :invoke :add 4984 INFO [2022-01-10 08:39:26,900] jepsen worker 609 - jepsen.util 609 :ok :add 4492 INFO [2022-01-10 08:39:26,900] jepsen worker 539 - jepsen.util 539 :ok :add 4502 INFO [2022-01-10 08:39:26,901] jepsen worker 310 - jepsen.util 310 :ok :add 4673 INFO [2022-01-10 08:39:26,901] jepsen worker 667 - jepsen.util 667 :ok :add 4647 INFO [2022-01-10 08:39:26,901] jepsen worker 51 - jepsen.util 51 :ok :add 4536 INFO [2022-01-10 08:39:26,902] jepsen worker 659 - jepsen.util 659 :invoke :add 4985 INFO [2022-01-10 08:39:26,905] jepsen worker 884 - jepsen.util 884 :invoke :add 4986 INFO [2022-01-10 08:39:26,906] jepsen worker 935 - jepsen.util 935 :ok :add 4347 INFO [2022-01-10 08:39:26,906] jepsen worker 731 - jepsen.util 731 :ok :add 4373 INFO [2022-01-10 08:39:26,906] jepsen worker 249 - jepsen.util 249 :ok :add 4589 INFO [2022-01-10 08:39:26,907] jepsen worker 708 - jepsen.util 708 :ok :add 4768 INFO [2022-01-10 08:39:26,907] jepsen worker 707 - jepsen.util 707 :ok :add 4861 INFO [2022-01-10 08:39:26,907] jepsen worker 699 - jepsen.util 699 :ok :add 4401 INFO [2022-01-10 08:39:26,907] jepsen worker 349 - jepsen.util 349 :ok :add 4744 INFO [2022-01-10 08:39:26,907] jepsen worker 575 - jepsen.util 575 :ok :add 4591 INFO [2022-01-10 08:39:26,907] jepsen worker 498 - jepsen.util 498 :ok :add 4435 INFO [2022-01-10 08:39:26,907] jepsen worker 56 - jepsen.util 56 :ok :add 4419 INFO [2022-01-10 08:39:26,908] jepsen worker 294 - jepsen.util 294 :invoke :add 4987 INFO [2022-01-10 08:39:26,912] jepsen worker 660 - jepsen.util 660 :invoke :add 4988 INFO [2022-01-10 08:39:26,914] jepsen worker 715 - jepsen.util 715 :invoke :add 4989 INFO [2022-01-10 08:39:26,916] jepsen worker 738 - jepsen.util 738 :invoke :add 4990 INFO [2022-01-10 08:39:26,919] jepsen worker 448 - jepsen.util 448 :invoke :add 4991 INFO [2022-01-10 08:39:26,921] jepsen worker 400 - jepsen.util 400 :invoke :add 4992 INFO [2022-01-10 08:39:26,923] jepsen worker 63 - jepsen.util 63 :invoke :add 4993 INFO [2022-01-10 08:39:26,926] jepsen worker 304 - jepsen.util 304 :invoke :add 4994 INFO [2022-01-10 08:39:26,928] jepsen worker 815 - jepsen.util 815 :invoke :add 4995 INFO [2022-01-10 08:39:26,930] jepsen worker 300 - jepsen.util 300 :invoke :add 4996 INFO [2022-01-10 08:39:26,932] jepsen worker 723 - jepsen.util 723 :invoke :add 4997 INFO [2022-01-10 08:39:26,934] jepsen worker 279 - jepsen.util 279 :invoke :add 4998 INFO [2022-01-10 08:39:26,937] jepsen worker 540 - jepsen.util 540 :invoke :add 4999 INFO [2022-01-10 08:39:26,939] jepsen worker 551 - jepsen.util 551 :invoke :add 5000 INFO [2022-01-10 08:39:26,941] jepsen worker 657 - jepsen.util 657 :invoke :add 5001 INFO [2022-01-10 08:39:26,943] jepsen worker 481 - jepsen.util 481 :invoke :add 5002 INFO [2022-01-10 08:39:26,945] jepsen worker 815 - jepsen.util 815 :ok :add 4995 INFO [2022-01-10 08:39:26,946] jepsen worker 339 - jepsen.util 339 :invoke :add 5003 INFO [2022-01-10 08:39:26,948] jepsen worker 920 - jepsen.util 920 :invoke :add 5004 INFO [2022-01-10 08:39:26,950] jepsen worker 983 - jepsen.util 983 :invoke :add 5005 INFO [2022-01-10 08:39:26,953] jepsen worker 12 - jepsen.util 12 :invoke :add 5006 INFO [2022-01-10 08:39:26,955] jepsen worker 758 - jepsen.util 758 :invoke :add 5007 INFO [2022-01-10 08:39:26,956] jepsen worker 304 - jepsen.util 304 :ok :add 4994 INFO [2022-01-10 08:39:26,956] jepsen worker 723 - jepsen.util 723 :ok :add 4997 INFO [2022-01-10 08:39:26,957] jepsen worker 799 - jepsen.util 799 :invoke :add 5008 INFO [2022-01-10 08:39:26,960] jepsen worker 735 - jepsen.util 735 :invoke :add 5009 INFO [2022-01-10 08:39:26,962] jepsen worker 1 - jepsen.util 1 :invoke :add 5010 INFO [2022-01-10 08:39:26,964] jepsen worker 672 - jepsen.util 672 :invoke :add 5011 INFO [2022-01-10 08:39:26,966] jepsen worker 475 - jepsen.util 475 :invoke :add 5012 INFO [2022-01-10 08:39:26,969] jepsen worker 344 - jepsen.util 344 :invoke :add 5013 INFO [2022-01-10 08:39:26,972] jepsen worker 742 - jepsen.util 742 :invoke :add 5014 INFO [2022-01-10 08:39:26,975] jepsen worker 148 - jepsen.util 148 :invoke :add 5015 INFO [2022-01-10 08:39:26,978] jepsen worker 634 - jepsen.util 634 :invoke :add 5016 INFO [2022-01-10 08:39:26,980] jepsen worker 446 - jepsen.util 446 :invoke :add 5017 INFO [2022-01-10 08:39:26,980] jepsen worker 300 - jepsen.util 300 :ok :add 4996 INFO [2022-01-10 08:39:26,981] jepsen worker 657 - jepsen.util 657 :ok :add 5001 INFO [2022-01-10 08:39:26,983] jepsen worker 88 - jepsen.util 88 :invoke :add 5018 INFO [2022-01-10 08:39:26,985] jepsen worker 750 - jepsen.util 750 :invoke :add 5019 INFO [2022-01-10 08:39:26,987] jepsen worker 835 - jepsen.util 835 :invoke :add 5020 INFO [2022-01-10 08:39:26,990] jepsen worker 732 - jepsen.util 732 :invoke :add 5021 INFO [2022-01-10 08:39:26,992] jepsen worker 100 - jepsen.util 100 :invoke :add 5022 INFO [2022-01-10 08:39:26,994] jepsen worker 622 - jepsen.util 622 :invoke :add 5023 INFO [2022-01-10 08:39:26,996] jepsen worker 450 - jepsen.util 450 :invoke :add 5024 INFO [2022-01-10 08:39:26,998] jepsen worker 500 - jepsen.util 500 :invoke :add 5025 INFO [2022-01-10 08:39:27,001] jepsen worker 385 - jepsen.util 385 :invoke :add 5026 INFO [2022-01-10 08:39:27,004] jepsen worker 243 - jepsen.util 243 :invoke :add 5027 INFO [2022-01-10 08:39:27,006] jepsen worker 297 - jepsen.util 297 :invoke :add 5028 INFO [2022-01-10 08:39:27,008] jepsen worker 78 - jepsen.util 78 :invoke :add 5029 INFO [2022-01-10 08:39:27,009] jepsen worker 551 - jepsen.util 551 :ok :add 5000 INFO [2022-01-10 08:39:27,009] jepsen worker 901 - jepsen.util 901 :ok :add 4340 INFO [2022-01-10 08:39:27,009] jepsen worker 220 - jepsen.util 220 :ok :add 4596 INFO [2022-01-10 08:39:27,010] jepsen worker 729 - jepsen.util 729 :ok :add 4358 INFO [2022-01-10 08:39:27,011] jepsen worker 476 - jepsen.util 476 :ok :add 4374 INFO [2022-01-10 08:39:27,011] jepsen worker 823 - jepsen.util 823 :ok :add 4432 INFO [2022-01-10 08:39:27,011] jepsen worker 704 - jepsen.util 704 :invoke :add 5030 INFO [2022-01-10 08:39:27,012] jepsen worker 834 - jepsen.util 834 :ok :add 4406 INFO [2022-01-10 08:39:27,012] jepsen worker 722 - jepsen.util 722 :ok :add 4743 INFO [2022-01-10 08:39:27,015] jepsen worker 372 - jepsen.util 372 :ok :add 4973 INFO [2022-01-10 08:39:27,015] jepsen worker 928 - jepsen.util 928 :ok :add 4947 INFO [2022-01-10 08:39:27,015] jepsen worker 195 - jepsen.util 195 :invoke :add 5031 INFO [2022-01-10 08:39:27,018] jepsen worker 206 - jepsen.util 206 :invoke :add 5032 INFO [2022-01-10 08:39:27,020] jepsen worker 230 - jepsen.util 230 :invoke :add 5033 INFO [2022-01-10 08:39:27,021] jepsen worker 832 - jepsen.util 832 :ok :add 4944 INFO [2022-01-10 08:39:27,022] jepsen worker 324 - jepsen.util 324 :ok :add 4970 INFO [2022-01-10 08:39:27,022] jepsen worker 529 - jepsen.util 529 :ok :add 4968 INFO [2022-01-10 08:39:27,023] jepsen worker 626 - jepsen.util 626 :invoke :add 5034 INFO [2022-01-10 08:39:27,026] jepsen worker 245 - jepsen.util 245 :invoke :add 5035 INFO [2022-01-10 08:39:27,028] jepsen worker 18 - jepsen.util 18 :invoke :add 5036 INFO [2022-01-10 08:39:27,031] jepsen worker 226 - jepsen.util 226 :invoke :add 5037 INFO [2022-01-10 08:39:27,033] jepsen worker 373 - jepsen.util 373 :invoke :add 5038 INFO [2022-01-10 08:39:27,035] jepsen worker 349 - jepsen.util 349 :invoke :add 5039 INFO [2022-01-10 08:39:27,036] jepsen worker 339 - jepsen.util 339 :ok :add 5003 INFO [2022-01-10 08:39:27,037] jepsen worker 481 - jepsen.util 481 :ok :add 5002 INFO [2022-01-10 08:39:27,037] jepsen worker 70 - jepsen.util 70 :invoke :add 5040 INFO [2022-01-10 08:39:27,040] jepsen worker 841 - jepsen.util 841 :invoke :add 5041 INFO [2022-01-10 08:39:27,043] jepsen worker 94 - jepsen.util 94 :invoke :add 5042 INFO [2022-01-10 08:39:27,044] jepsen worker 750 - jepsen.util 750 :ok :add 5019 INFO [2022-01-10 08:39:27,044] jepsen worker 245 - jepsen.util 245 :ok :add 5035 INFO [2022-01-10 08:39:27,045] jepsen worker 440 - jepsen.util 440 :invoke :add 5043 INFO [2022-01-10 08:39:27,049] jepsen worker 236 - jepsen.util 236 :invoke :add 5044 INFO [2022-01-10 08:39:27,051] jepsen worker 251 - jepsen.util 251 :invoke :add 5045 INFO [2022-01-10 08:39:27,054] jepsen worker 744 - jepsen.util 744 :invoke :add 5046 INFO [2022-01-10 08:39:27,056] jepsen worker 179 - jepsen.util 179 :invoke :add 5047 INFO [2022-01-10 08:39:27,057] jepsen worker 88 - jepsen.util 88 :ok :add 5018 INFO [2022-01-10 08:39:27,057] jepsen worker 626 - jepsen.util 626 :ok :add 5034 INFO [2022-01-10 08:39:27,059] jepsen worker 872 - jepsen.util 872 :invoke :add 5048 INFO [2022-01-10 08:39:27,061] jepsen worker 890 - jepsen.util 890 :invoke :add 5049 INFO [2022-01-10 08:39:27,063] jepsen worker 530 - jepsen.util 530 :invoke :add 5050 INFO [2022-01-10 08:39:27,066] jepsen worker 88 - jepsen.util 88 :invoke :add 5051 INFO [2022-01-10 08:39:27,068] jepsen worker 834 - jepsen.util 834 :invoke :add 5052 INFO [2022-01-10 08:39:27,070] jepsen worker 101 - jepsen.util 101 :invoke :add 5053 INFO [2022-01-10 08:39:27,073] jepsen worker 199 - jepsen.util 199 :ok :add 4888 INFO [2022-01-10 08:39:27,073] jepsen worker 419 - jepsen.util 419 :ok :add 4900 INFO [2022-01-10 08:39:27,073] jepsen worker 704 - jepsen.util 704 :ok :add 5030 INFO [2022-01-10 08:39:27,073] jepsen worker 297 - jepsen.util 297 :ok :add 5028 INFO [2022-01-10 08:39:27,073] jepsen worker 531 - jepsen.util 531 :ok :add 4918 INFO [2022-01-10 08:39:27,073] jepsen worker 237 - jepsen.util 237 :ok :add 4799 INFO [2022-01-10 08:39:27,073] jepsen worker 647 - jepsen.util 647 :ok :add 4934 INFO [2022-01-10 08:39:27,073] jepsen worker 16 - jepsen.util 16 :invoke :add 5054 INFO [2022-01-10 08:39:27,074] jepsen worker 862 - jepsen.util 862 :ok :add 4890 INFO [2022-01-10 08:39:27,074] jepsen worker 920 - jepsen.util 920 :ok :add 5004 INFO [2022-01-10 08:39:27,076] jepsen worker 572 - jepsen.util 572 :ok :add 4344 INFO [2022-01-10 08:39:27,076] jepsen worker 610 - jepsen.util 610 :ok :add 4368 INFO [2022-01-10 08:39:27,076] jepsen worker 275 - jepsen.util 275 :ok :add 4402 INFO [2022-01-10 08:39:27,077] jepsen worker 633 - jepsen.util 633 :ok :add 4592 INFO [2022-01-10 08:39:27,077] jepsen worker 522 - jepsen.util 522 :ok :add 4436 INFO [2022-01-10 08:39:27,077] jepsen worker 163 - jepsen.util 163 :ok :add 4862 INFO [2022-01-10 08:39:27,077] jepsen worker 485 - jepsen.util 485 :ok :add 4370 INFO [2022-01-10 08:39:27,078] jepsen worker 963 - jepsen.util 963 :invoke :add 5055 INFO [2022-01-10 08:39:27,081] jepsen worker 791 - jepsen.util 791 :invoke :add 5056 INFO [2022-01-10 08:39:27,082] jepsen worker 191 - jepsen.util 191 :ok :add 4835 INFO [2022-01-10 08:39:27,082] jepsen worker 715 - jepsen.util 715 :ok :add 4989 INFO [2022-01-10 08:39:27,083] jepsen worker 277 - jepsen.util 277 :ok :add 4801 INFO [2022-01-10 08:39:27,083] jepsen worker 959 - jepsen.util 959 :ok :add 4819 INFO [2022-01-10 08:39:27,083] jepsen worker 448 - jepsen.util 448 :ok :add 4991 INFO [2022-01-10 08:39:27,084] jepsen worker 416 - jepsen.util 416 :invoke :add 5057 INFO [2022-01-10 08:39:27,087] jepsen worker 266 - jepsen.util 266 :invoke :add 5058 INFO [2022-01-10 08:39:27,089] jepsen worker 508 - jepsen.util 508 :invoke :add 5059 INFO [2022-01-10 08:39:27,092] jepsen worker 309 - jepsen.util 309 :invoke :add 5060 INFO [2022-01-10 08:39:27,093] jepsen worker 226 - jepsen.util 226 :ok :add 5037 INFO [2022-01-10 08:39:27,094] jepsen worker 482 - jepsen.util 482 :invoke :add 5061 INFO [2022-01-10 08:39:27,096] jepsen worker 160 - jepsen.util 160 :invoke :add 5062 INFO [2022-01-10 08:39:27,099] jepsen worker 793 - jepsen.util 793 :invoke :add 5063 INFO [2022-01-10 08:39:27,101] jepsen worker 33 - jepsen.util 33 :invoke :add 5064 INFO [2022-01-10 08:39:27,103] jepsen worker 345 - jepsen.util 345 :invoke :add 5065 INFO [2022-01-10 08:39:27,105] jepsen worker 688 - jepsen.util 688 :invoke :add 5066 INFO [2022-01-10 08:39:27,105] jepsen worker 18 - jepsen.util 18 :ok :add 5036 INFO [2022-01-10 08:39:27,108] jepsen worker 149 - jepsen.util 149 :invoke :add 5067 INFO [2022-01-10 08:39:27,108] jepsen worker 236 - jepsen.util 236 :ok :add 5044 INFO [2022-01-10 08:39:27,110] jepsen worker 745 - jepsen.util 745 :invoke :add 5068 INFO [2022-01-10 08:39:27,111] jepsen worker 251 - jepsen.util 251 :ok :add 5045 INFO [2022-01-10 08:39:27,113] jepsen worker 972 - jepsen.util 972 :invoke :add 5069 INFO [2022-01-10 08:39:27,115] jepsen worker 197 - jepsen.util 197 :invoke :add 5070 INFO [2022-01-10 08:39:27,117] jepsen worker 366 - jepsen.util 366 :invoke :add 5071 INFO [2022-01-10 08:39:27,119] jepsen worker 191 - jepsen.util 191 :invoke :add 5072 INFO [2022-01-10 08:39:27,121] jepsen worker 960 - jepsen.util 960 :invoke :add 5073 INFO [2022-01-10 08:39:27,124] jepsen worker 420 - jepsen.util 420 :invoke :add 5074 INFO [2022-01-10 08:39:27,126] jepsen worker 408 - jepsen.util 408 :invoke :add 5075 INFO [2022-01-10 08:39:27,128] jepsen worker 179 - jepsen.util 179 :ok :add 5047 INFO [2022-01-10 08:39:27,128] jepsen worker 130 - jepsen.util 130 :invoke :add 5076 INFO [2022-01-10 08:39:27,128] jepsen worker 744 - jepsen.util 744 :ok :add 5046 INFO [2022-01-10 08:39:27,131] jepsen worker 461 - jepsen.util 461 :invoke :add 5077 INFO [2022-01-10 08:39:27,133] jepsen worker 967 - jepsen.util 967 :invoke :add 5078 INFO [2022-01-10 08:39:27,136] jepsen worker 324 - jepsen.util 324 :invoke :add 5079 INFO [2022-01-10 08:39:27,138] jepsen worker 752 - jepsen.util 752 :invoke :add 5080 INFO [2022-01-10 08:39:27,139] jepsen worker 197 - jepsen.util 197 :ok :add 5070 INFO [2022-01-10 08:39:27,139] jepsen worker 745 - jepsen.util 745 :ok :add 5068 INFO [2022-01-10 08:39:27,139] jepsen worker 972 - jepsen.util 972 :ok :add 5069 INFO [2022-01-10 08:39:27,139] jepsen worker 366 - jepsen.util 366 :ok :add 5071 INFO [2022-01-10 08:39:27,140] jepsen worker 251 - jepsen.util 251 :invoke :add 5081 INFO [2022-01-10 08:39:27,143] jepsen worker 437 - jepsen.util 437 :invoke :add 5082 INFO [2022-01-10 08:39:27,144] jepsen worker 960 - jepsen.util 960 :ok :add 5073 INFO [2022-01-10 08:39:27,145] jepsen worker 830 - jepsen.util 830 :invoke :add 5083 INFO [2022-01-10 08:39:27,148] jepsen worker 402 - jepsen.util 402 :invoke :add 5084 INFO [2022-01-10 08:39:27,149] jepsen worker 191 - jepsen.util 191 :ok :add 5072 INFO [2022-01-10 08:39:27,152] jepsen worker 478 - jepsen.util 478 :invoke :add 5085 INFO [2022-01-10 08:39:27,154] jepsen worker 11 - jepsen.util 11 :invoke :add 5086 INFO [2022-01-10 08:39:27,157] jepsen worker 256 - jepsen.util 256 :invoke :add 5087 INFO [2022-01-10 08:39:27,160] jepsen worker 769 - jepsen.util 769 :invoke :add 5088 INFO [2022-01-10 08:39:27,162] jepsen worker 710 - jepsen.util 710 :invoke :add 5089 INFO [2022-01-10 08:39:27,164] jepsen worker 505 - jepsen.util 505 :invoke :add 5090 INFO [2022-01-10 08:39:27,167] jepsen worker 99 - jepsen.util 99 :invoke :add 5091 INFO [2022-01-10 08:39:27,169] jepsen worker 603 - jepsen.util 603 :invoke :add 5092 INFO [2022-01-10 08:39:27,171] jepsen worker 683 - jepsen.util 683 :invoke :add 5093 INFO [2022-01-10 08:39:27,174] jepsen worker 332 - jepsen.util 332 :invoke :add 5094 INFO [2022-01-10 08:39:27,176] jepsen worker 327 - jepsen.util 327 :invoke :add 5095 INFO [2022-01-10 08:39:27,179] jepsen worker 120 - jepsen.util 120 :invoke :add 5096 INFO [2022-01-10 08:39:27,182] jepsen worker 769 - jepsen.util 769 :ok :add 5088 INFO [2022-01-10 08:39:27,182] jepsen worker 997 - jepsen.util 997 :invoke :add 5097 INFO [2022-01-10 08:39:27,185] jepsen worker 951 - jepsen.util 951 :invoke :add 5098 INFO [2022-01-10 08:39:27,187] jepsen worker 346 - jepsen.util 346 :ok :add 4834 INFO [2022-01-10 08:39:27,187] jepsen worker 342 - jepsen.util 342 :ok :add 4818 INFO [2022-01-10 08:39:27,187] jepsen worker 738 - jepsen.util 738 :ok :add 4990 INFO [2022-01-10 08:39:27,187] jepsen worker 660 - jepsen.util 660 :ok :add 4988 INFO [2022-01-10 08:39:27,188] jepsen worker 785 - jepsen.util 785 :invoke :add 5099 INFO [2022-01-10 08:39:27,192] jepsen worker 539 - jepsen.util 539 :invoke :add 5100 INFO [2022-01-10 08:39:27,194] jepsen worker 212 - jepsen.util 212 :invoke :add 5101 INFO [2022-01-10 08:39:27,197] jepsen worker 132 - jepsen.util 132 :invoke :add 5102 INFO [2022-01-10 08:39:27,199] jepsen worker 708 - jepsen.util 708 :invoke :add 5103 INFO [2022-01-10 08:39:27,200] jepsen worker 710 - jepsen.util 710 :ok :add 5089 INFO [2022-01-10 08:39:27,200] jepsen worker 99 - jepsen.util 99 :ok :add 5091 INFO [2022-01-10 08:39:27,201] jepsen worker 564 - jepsen.util 564 :invoke :add 5104 INFO [2022-01-10 08:39:27,205] jepsen worker 505 - jepsen.util 505 :ok :add 5090 INFO [2022-01-10 08:39:27,205] jepsen worker 562 - jepsen.util 562 :invoke :add 5105 INFO [2022-01-10 08:39:27,207] jepsen worker 683 - jepsen.util 683 :ok :add 5093 INFO [2022-01-10 08:39:27,209] jepsen worker 809 - jepsen.util 809 :invoke :add 5106 INFO [2022-01-10 08:39:27,212] jepsen worker 614 - jepsen.util 614 :invoke :add 5107 INFO [2022-01-10 08:39:27,213] jepsen worker 212 - jepsen.util 212 :ok :add 5101 INFO [2022-01-10 08:39:27,215] jepsen worker 267 - jepsen.util 267 :invoke :add 5108 INFO [2022-01-10 08:39:27,217] jepsen worker 926 - jepsen.util 926 :invoke :add 5109 INFO [2022-01-10 08:39:27,218] jepsen worker 708 - jepsen.util 708 :ok :add 5103 INFO [2022-01-10 08:39:27,218] jepsen worker 132 - jepsen.util 132 :ok :add 5102 INFO [2022-01-10 08:39:27,219] jepsen worker 603 - jepsen.util 603 :ok :add 5092 INFO [2022-01-10 08:39:27,220] jepsen worker 241 - jepsen.util 241 :invoke :add 5110 INFO [2022-01-10 08:39:27,223] jepsen worker 271 - jepsen.util 271 :invoke :add 5111 INFO [2022-01-10 08:39:27,226] jepsen worker 959 - jepsen.util 959 :invoke :add 5112 INFO [2022-01-10 08:39:27,228] jepsen worker 669 - jepsen.util 669 :invoke :add 5113 INFO [2022-01-10 08:39:27,231] jepsen worker 603 - jepsen.util 603 :invoke :add 5114 INFO [2022-01-10 08:39:27,233] jepsen worker 392 - jepsen.util 392 :invoke :add 5115 INFO [2022-01-10 08:39:27,235] jepsen worker 304 - jepsen.util 304 :invoke :add 5116 INFO [2022-01-10 08:39:27,237] jepsen worker 220 - jepsen.util 220 :invoke :add 5117 INFO [2022-01-10 08:39:27,240] jepsen worker 272 - jepsen.util 272 :invoke :add 5118 INFO [2022-01-10 08:39:27,242] jepsen worker 528 - jepsen.util 528 :invoke :add 5119 INFO [2022-01-10 08:39:27,245] jepsen worker 757 - jepsen.util 757 :invoke :add 5120 INFO [2022-01-10 08:39:27,247] jepsen worker 157 - jepsen.util 157 :invoke :add 5121 INFO [2022-01-10 08:39:27,249] jepsen worker 231 - jepsen.util 231 :invoke :add 5122 INFO [2022-01-10 08:39:27,251] jepsen worker 339 - jepsen.util 339 :invoke :add 5123 INFO [2022-01-10 08:39:27,253] jepsen worker 505 - jepsen.util 505 :invoke :add 5124 INFO [2022-01-10 08:39:27,255] jepsen worker 545 - jepsen.util 545 :invoke :add 5125 INFO [2022-01-10 08:39:27,257] jepsen worker 351 - jepsen.util 351 :invoke :add 5126 INFO [2022-01-10 08:39:27,259] jepsen worker 54 - jepsen.util 54 :invoke :add 5127 INFO [2022-01-10 08:39:27,263] jepsen worker 269 - jepsen.util 269 :invoke :add 5128 INFO [2022-01-10 08:39:27,265] jepsen worker 539 - jepsen.util 539 :ok :add 5100 INFO [2022-01-10 08:39:27,266] jepsen worker 775 - jepsen.util 775 :invoke :add 5129 INFO [2022-01-10 08:39:27,269] jepsen worker 45 - jepsen.util 45 :invoke :add 5130 INFO [2022-01-10 08:39:27,272] jepsen worker 528 - jepsen.util 528 :ok :add 5119 INFO [2022-01-10 08:39:27,272] jepsen worker 334 - jepsen.util 334 :invoke :add 5131 INFO [2022-01-10 08:39:27,275] jepsen worker 92 - jepsen.util 92 :invoke :add 5132 INFO [2022-01-10 08:39:27,278] jepsen worker 497 - jepsen.util 497 :invoke :add 5133 INFO [2022-01-10 08:39:27,281] jepsen worker 481 - jepsen.util 481 :invoke :add 5134 INFO [2022-01-10 08:39:27,283] jepsen worker 232 - jepsen.util 232 :invoke :add 5135 INFO [2022-01-10 08:39:27,285] jepsen worker 648 - jepsen.util 648 :invoke :add 5136 INFO [2022-01-10 08:39:27,287] jepsen worker 804 - jepsen.util 804 :invoke :add 5137 INFO [2022-01-10 08:39:27,289] jepsen worker 975 - jepsen.util 975 :invoke :add 5138 INFO [2022-01-10 08:39:27,292] jepsen worker 789 - jepsen.util 789 :invoke :add 5139 INFO [2022-01-10 08:39:27,294] jepsen worker 398 - jepsen.util 398 :invoke :add 5140 INFO [2022-01-10 08:39:27,296] jepsen worker 984 - jepsen.util 984 :invoke :add 5141 INFO [2022-01-10 08:39:27,299] jepsen worker 208 - jepsen.util 208 :invoke :add 5142 INFO [2022-01-10 08:39:27,301] jepsen worker 623 - jepsen.util 623 :invoke :add 5143 INFO [2022-01-10 08:39:27,303] jepsen worker 400 - jepsen.util 400 :ok :add 4992 INFO [2022-01-10 08:39:27,303] jepsen worker 565 - jepsen.util 565 :ok :add 4836 INFO [2022-01-10 08:39:27,303] jepsen worker 120 - jepsen.util 120 :ok :add 5096 INFO [2022-01-10 08:39:27,304] jepsen worker 181 - jepsen.util 181 :invoke :add 5144 INFO [2022-01-10 08:39:27,307] jepsen worker 661 - jepsen.util 661 :invoke :add 5145 INFO [2022-01-10 08:39:27,309] jepsen worker 458 - jepsen.util 458 :invoke :add 5146 INFO [2022-01-10 08:39:27,311] jepsen worker 704 - jepsen.util 704 :invoke :add 5147 INFO [2022-01-10 08:39:27,313] jepsen worker 76 - jepsen.util 76 :invoke :add 5148 INFO [2022-01-10 08:39:27,315] jepsen worker 521 - jepsen.util 521 :invoke :add 5149 INFO [2022-01-10 08:39:27,315] jepsen worker 63 - jepsen.util 63 :ok :add 4993 INFO [2022-01-10 08:39:27,315] jepsen worker 614 - jepsen.util 614 :ok :add 5107 INFO [2022-01-10 08:39:27,316] jepsen worker 997 - jepsen.util 997 :ok :add 5097 INFO [2022-01-10 08:39:27,316] jepsen worker 908 - jepsen.util 908 :ok :add 4837 INFO [2022-01-10 08:39:27,316] jepsen worker 497 - jepsen.util 497 :ok :add 5133 INFO [2022-01-10 08:39:27,316] jepsen worker 632 - jepsen.util 632 :ok :add 4803 INFO [2022-01-10 08:39:27,318] jepsen worker 693 - jepsen.util 693 :invoke :add 5150 INFO [2022-01-10 08:39:27,322] jepsen worker 905 - jepsen.util 905 :invoke :add 5151 INFO [2022-01-10 08:39:27,324] jepsen worker 109 - jepsen.util 109 :invoke :add 5152 INFO [2022-01-10 08:39:27,327] jepsen worker 850 - jepsen.util 850 :invoke :add 5153 INFO [2022-01-10 08:39:27,329] jepsen worker 943 - jepsen.util 943 :invoke :add 5154 INFO [2022-01-10 08:39:27,331] jepsen worker 528 - jepsen.util 528 :invoke :add 5155 INFO [2022-01-10 08:39:27,333] jepsen worker 355 - jepsen.util 355 :invoke :add 5156 INFO [2022-01-10 08:39:27,335] jepsen worker 885 - jepsen.util 885 :ok :add 4400 INFO [2022-01-10 08:39:27,335] jepsen worker 17 - jepsen.util 17 :ok :add 4588 INFO [2022-01-10 08:39:27,335] jepsen worker 910 - jepsen.util 910 :ok :add 4590 INFO [2022-01-10 08:39:27,335] jepsen worker 918 - jepsen.util 918 :ok :add 4372 INFO [2022-01-10 08:39:27,335] jepsen worker 619 - jepsen.util 619 :ok :add 4434 INFO [2022-01-10 08:39:27,335] jepsen worker 47 - jepsen.util 47 :ok :add 4418 INFO [2022-01-10 08:39:27,335] jepsen worker 167 - jepsen.util 167 :ok :add 4745 INFO [2022-01-10 08:39:27,335] jepsen worker 97 - jepsen.util 97 :ok :add 4771 INFO [2022-01-10 08:39:27,335] jepsen worker 295 - jepsen.util 295 :ok :add 4878 INFO [2022-01-10 08:39:27,336] jepsen worker 878 - jepsen.util 878 :invoke :add 5157 INFO [2022-01-10 08:39:27,341] jepsen worker 778 - jepsen.util 778 :invoke :add 5158 INFO [2022-01-10 08:39:27,343] jepsen worker 948 - jepsen.util 948 :invoke :add 5159 INFO [2022-01-10 08:39:27,346] jepsen worker 347 - jepsen.util 347 :invoke :add 5160 INFO [2022-01-10 08:39:27,349] jepsen worker 631 - jepsen.util 631 :invoke :add 5161 INFO [2022-01-10 08:39:27,351] jepsen worker 136 - jepsen.util 136 :invoke :add 5162 INFO [2022-01-10 08:39:27,353] jepsen worker 802 - jepsen.util 802 :invoke :add 5163 INFO [2022-01-10 08:39:27,356] jepsen worker 291 - jepsen.util 291 :ok :add 4936 INFO [2022-01-10 08:39:27,356] jepsen worker 746 - jepsen.util 746 :ok :add 4892 INFO [2022-01-10 08:39:27,356] jepsen worker 206 - jepsen.util 206 :ok :add 5032 INFO [2022-01-10 08:39:27,356] jepsen worker 12 - jepsen.util 12 :ok :add 5006 INFO [2022-01-10 08:39:27,356] jepsen worker 397 - jepsen.util 397 :ok :add 4902 INFO [2022-01-10 08:39:27,356] jepsen worker 748 - jepsen.util 748 :invoke :add 5164 INFO [2022-01-10 08:39:27,359] jepsen worker 394 - jepsen.util 394 :invoke :add 5165 INFO [2022-01-10 08:39:27,361] jepsen worker 986 - jepsen.util 986 :invoke :add 5166 INFO [2022-01-10 08:39:27,364] jepsen worker 838 - jepsen.util 838 :invoke :add 5167 INFO [2022-01-10 08:39:27,366] jepsen worker 579 - jepsen.util 579 :invoke :add 5168 INFO [2022-01-10 08:39:27,368] jepsen worker 38 - jepsen.util 38 :invoke :add 5169 INFO [2022-01-10 08:39:27,371] jepsen worker 868 - jepsen.util 868 :invoke :add 5170 INFO [2022-01-10 08:39:27,372] jepsen worker 196 - jepsen.util 196 :invoke :add 5171 INFO [2022-01-10 08:39:27,374] jepsen worker 952 - jepsen.util 952 :invoke :add 5172 INFO [2022-01-10 08:39:27,377] jepsen worker 412 - jepsen.util 412 :invoke :add 5173 INFO [2022-01-10 08:39:27,379] jepsen worker 683 - jepsen.util 683 :invoke :add 5174 INFO [2022-01-10 08:39:27,386] jepsen worker 771 - jepsen.util 771 :invoke :add 5175 INFO [2022-01-10 08:39:27,388] jepsen worker 465 - jepsen.util 465 :invoke :add 5176 INFO [2022-01-10 08:39:27,391] jepsen worker 938 - jepsen.util 938 :invoke :add 5177 INFO [2022-01-10 08:39:27,394] jepsen worker 34 - jepsen.util 34 :invoke :add 5178 INFO [2022-01-10 08:39:27,395] jepsen worker 272 - jepsen.util 272 :ok :add 5118 INFO [2022-01-10 08:39:27,396] jepsen worker 422 - jepsen.util 422 :invoke :add 5179 INFO [2022-01-10 08:39:27,399] jepsen worker 515 - jepsen.util 515 :invoke :add 5180 INFO [2022-01-10 08:39:27,402] jepsen worker 535 - jepsen.util 535 :invoke :add 5181 INFO [2022-01-10 08:39:27,404] jepsen worker 600 - jepsen.util 600 :invoke :add 5182 INFO [2022-01-10 08:39:27,406] jepsen worker 549 - jepsen.util 549 :invoke :add 5183 INFO [2022-01-10 08:39:27,408] jepsen worker 763 - jepsen.util 763 :invoke :add 5184 INFO [2022-01-10 08:39:27,411] jepsen worker 897 - jepsen.util 897 :invoke :add 5185 INFO [2022-01-10 08:39:27,414] jepsen worker 419 - jepsen.util 419 :invoke :add 5186 INFO [2022-01-10 08:39:27,416] jepsen worker 940 - jepsen.util 940 :invoke :add 5187 INFO [2022-01-10 08:39:27,419] jepsen worker 127 - jepsen.util 127 :invoke :add 5188 INFO [2022-01-10 08:39:27,421] jepsen worker 356 - jepsen.util 356 :invoke :add 5189 INFO [2022-01-10 08:39:27,423] jepsen worker 852 - jepsen.util 852 :invoke :add 5190 INFO [2022-01-10 08:39:27,425] jepsen worker 947 - jepsen.util 947 :invoke :add 5191 INFO [2022-01-10 08:39:27,427] jepsen worker 686 - jepsen.util 686 :invoke :add 5192 INFO [2022-01-10 08:39:27,429] jepsen worker 608 - jepsen.util 608 :invoke :add 5193 INFO [2022-01-10 08:39:27,431] jepsen worker 113 - jepsen.util 113 :invoke :add 5194 INFO [2022-01-10 08:39:27,435] jepsen worker 23 - jepsen.util 23 :invoke :add 5195 INFO [2022-01-10 08:39:27,437] jepsen worker 246 - jepsen.util 246 :invoke :add 5196 INFO [2022-01-10 08:39:27,440] jepsen worker 328 - jepsen.util 328 :invoke :add 5197 INFO [2022-01-10 08:39:27,442] jepsen worker 806 - jepsen.util 806 :invoke :add 5198 INFO [2022-01-10 08:39:27,445] jepsen worker 827 - jepsen.util 827 :invoke :add 5199 INFO [2022-01-10 08:39:27,447] jepsen worker 925 - jepsen.util 925 :invoke :add 5200 INFO [2022-01-10 08:39:27,447] jepsen worker 264 - jepsen.util 264 :ok :add 4845 INFO [2022-01-10 08:39:27,447] jepsen worker 998 - jepsen.util 998 :ok :add 4869 INFO [2022-01-10 08:39:27,447] jepsen worker 322 - jepsen.util 322 :ok :add 4871 INFO [2022-01-10 08:39:27,447] jepsen worker 948 - jepsen.util 948 :ok :add 5159 INFO [2022-01-10 08:39:27,447] jepsen worker 984 - jepsen.util 984 :ok :add 5141 INFO [2022-01-10 08:39:27,449] jepsen worker 28 - jepsen.util 28 :invoke :add 5201 INFO [2022-01-10 08:39:27,453] jepsen worker 657 - jepsen.util 657 :invoke :add 5202 INFO [2022-01-10 08:39:27,455] jepsen worker 297 - jepsen.util 297 :invoke :add 5203 INFO [2022-01-10 08:39:27,457] jepsen worker 848 - jepsen.util 848 :invoke :add 5204 INFO [2022-01-10 08:39:27,459] jepsen worker 979 - jepsen.util 979 :invoke :add 5205 INFO [2022-01-10 08:39:27,462] jepsen worker 776 - jepsen.util 776 :invoke :add 5206 INFO [2022-01-10 08:39:27,465] jepsen worker 911 - jepsen.util 911 :invoke :add 5207 INFO [2022-01-10 08:39:27,466] jepsen worker 615 - jepsen.util 615 :invoke :add 5208 INFO [2022-01-10 08:39:27,469] jepsen worker 12 - jepsen.util 12 :invoke :add 5209 INFO [2022-01-10 08:39:27,471] jepsen worker 32 - jepsen.util 32 :invoke :add 5210 INFO [2022-01-10 08:39:27,472] jepsen worker 340 - jepsen.util 340 :invoke :add 5211 INFO [2022-01-10 08:39:27,475] jepsen worker 723 - jepsen.util 723 :invoke :add 5212 INFO [2022-01-10 08:39:27,477] jepsen worker 333 - jepsen.util 333 :invoke :add 5213 INFO [2022-01-10 08:39:27,479] jepsen worker 391 - jepsen.util 391 :invoke :add 5214 INFO [2022-01-10 08:39:27,481] jepsen worker 790 - jepsen.util 790 :ok :add 4477 INFO [2022-01-10 08:39:27,481] jepsen worker 144 - jepsen.util 144 :ok :add 4706 INFO [2022-01-10 08:39:27,481] jepsen worker 718 - jepsen.util 718 :ok :add 4732 INFO [2022-01-10 08:39:27,481] jepsen worker 483 - jepsen.util 483 :ok :add 4443 INFO [2022-01-10 08:39:27,481] jepsen worker 96 - jepsen.util 96 :ok :add 4696 INFO [2022-01-10 08:39:27,482] jepsen worker 556 - jepsen.util 556 :invoke :add 5215 INFO [2022-01-10 08:39:27,486] jepsen worker 37 - jepsen.util 37 :invoke :add 5216 INFO [2022-01-10 08:39:27,489] jepsen worker 227 - jepsen.util 227 :invoke :add 5217 INFO [2022-01-10 08:39:27,491] jepsen worker 522 - jepsen.util 522 :invoke :add 5218 INFO [2022-01-10 08:39:27,494] jepsen worker 571 - jepsen.util 571 :invoke :add 5219 INFO [2022-01-10 08:39:27,496] jepsen worker 881 - jepsen.util 881 :invoke :add 5220 INFO [2022-01-10 08:39:27,499] jepsen worker 529 - jepsen.util 529 :invoke :add 5221 INFO [2022-01-10 08:39:27,501] jepsen worker 454 - jepsen.util 454 :invoke :add 5222 INFO [2022-01-10 08:39:27,504] jepsen worker 4 - jepsen.util 4 :invoke :add 5223 INFO [2022-01-10 08:39:27,506] jepsen worker 338 - jepsen.util 338 :invoke :add 5224 INFO [2022-01-10 08:39:27,509] jepsen worker 668 - jepsen.util 668 :invoke :add 5225 INFO [2022-01-10 08:39:27,511] jepsen worker 893 - jepsen.util 893 :invoke :add 5226 INFO [2022-01-10 08:39:27,514] jepsen worker 976 - jepsen.util 976 :ok :add 4507 INFO [2022-01-10 08:39:27,514] jepsen worker 86 - jepsen.util 86 :ok :add 4533 INFO [2022-01-10 08:39:27,514] jepsen worker 606 - jepsen.util 606 :ok :add 4497 INFO [2022-01-10 08:39:27,514] jepsen worker 561 - jepsen.util 561 :ok :add 4676 INFO [2022-01-10 08:39:27,514] jepsen worker 298 - jepsen.util 298 :invoke :add 5227 INFO [2022-01-10 08:39:27,514] jepsen worker 469 - jepsen.util 469 :ok :add 4642 INFO [2022-01-10 08:39:27,517] jepsen worker 652 - jepsen.util 652 :invoke :add 5228 INFO [2022-01-10 08:39:27,520] jepsen worker 867 - jepsen.util 867 :invoke :add 5229 INFO [2022-01-10 08:39:27,522] jepsen worker 225 - jepsen.util 225 :invoke :add 5230 INFO [2022-01-10 08:39:27,524] jepsen worker 998 - jepsen.util 998 :invoke :add 5231 INFO [2022-01-10 08:39:27,526] jepsen worker 191 - jepsen.util 191 :invoke :add 5232 INFO [2022-01-10 08:39:27,529] jepsen worker 171 - jepsen.util 171 :invoke :add 5233 INFO [2022-01-10 08:39:27,530] jepsen worker 185 - jepsen.util 185 :ok :add 4919 INFO [2022-01-10 08:39:27,530] jepsen worker 431 - jepsen.util 431 :ok :add 4889 INFO [2022-01-10 08:39:27,531] jepsen worker 195 - jepsen.util 195 :ok :add 5031 INFO [2022-01-10 08:39:27,531] jepsen worker 944 - jepsen.util 944 :ok :add 4891 INFO [2022-01-10 08:39:27,531] jepsen worker 970 - jepsen.util 970 :invoke :add 5234 INFO [2022-01-10 08:39:27,532] jepsen worker 222 - jepsen.util 222 :ok :add 4935 INFO [2022-01-10 08:39:27,532] jepsen worker 78 - jepsen.util 78 :ok :add 5029 INFO [2022-01-10 08:39:27,532] jepsen worker 983 - jepsen.util 983 :ok :add 5005 INFO [2022-01-10 08:39:27,532] jepsen worker 725 - jepsen.util 725 :ok :add 4901 INFO [2022-01-10 08:39:27,532] jepsen worker 23 - jepsen.util 23 :ok :add 5195 INFO [2022-01-10 08:39:27,536] jepsen worker 17 - jepsen.util 17 :invoke :add 5235 INFO [2022-01-10 08:39:27,537] jepsen worker 232 - jepsen.util 232 :ok :add 5135 INFO [2022-01-10 08:39:27,540] jepsen worker 869 - jepsen.util 869 :invoke :add 5236 INFO [2022-01-10 08:39:27,543] jepsen worker 341 - jepsen.util 341 :invoke :add 5237 INFO [2022-01-10 08:39:27,547] jepsen worker 126 - jepsen.util 126 :invoke :add 5238 INFO [2022-01-10 08:39:27,550] jepsen worker 444 - jepsen.util 444 :invoke :add 5239 INFO [2022-01-10 08:39:27,552] jepsen worker 244 - jepsen.util 244 :invoke :add 5240 INFO [2022-01-10 08:39:27,554] jepsen worker 702 - jepsen.util 702 :invoke :add 5241 INFO [2022-01-10 08:39:27,556] jepsen worker 992 - jepsen.util 992 :invoke :add 5242 INFO [2022-01-10 08:39:27,559] jepsen worker 619 - jepsen.util 619 :invoke :add 5243 INFO [2022-01-10 08:39:27,561] jepsen worker 514 - jepsen.util 514 :invoke :add 5244 INFO [2022-01-10 08:39:27,563] jepsen worker 469 - jepsen.util 469 :invoke :add 5245 INFO [2022-01-10 08:39:27,566] jepsen worker 55 - jepsen.util 55 :invoke :add 5246 INFO [2022-01-10 08:39:27,568] jepsen worker 929 - jepsen.util 929 :invoke :add 5247 INFO [2022-01-10 08:39:27,571] jepsen worker 358 - jepsen.util 358 :invoke :add 5248 INFO [2022-01-10 08:39:27,572] jepsen worker 99 - jepsen.util 99 :invoke :add 5249 INFO [2022-01-10 08:39:27,575] jepsen worker 797 - jepsen.util 797 :invoke :add 5250 INFO [2022-01-10 08:39:27,577] jepsen worker 337 - jepsen.util 337 :invoke :add 5251 INFO [2022-01-10 08:39:27,579] jepsen worker 741 - jepsen.util 741 :invoke :add 5252 INFO [2022-01-10 08:39:27,581] jepsen worker 999 - jepsen.util 999 :invoke :add 5253 INFO [2022-01-10 08:39:27,583] jepsen worker 429 - jepsen.util 429 :invoke :add 5254 INFO [2022-01-10 08:39:27,586] jepsen worker 896 - jepsen.util 896 :invoke :add 5255 INFO [2022-01-10 08:39:27,588] jepsen worker 960 - jepsen.util 960 :invoke :add 5256 INFO [2022-01-10 08:39:27,590] jepsen worker 710 - jepsen.util 710 :invoke :add 5257 INFO [2022-01-10 08:39:27,592] jepsen worker 8 - jepsen.util 8 :invoke :add 5258 INFO [2022-01-10 08:39:27,595] jepsen worker 965 - jepsen.util 965 :invoke :add 5259 INFO [2022-01-10 08:39:27,597] jepsen worker 302 - jepsen.util 302 :invoke :add 5260 INFO [2022-01-10 08:39:27,600] jepsen worker 125 - jepsen.util 125 :invoke :add 5261 INFO [2022-01-10 08:39:27,601] jepsen worker 77 - jepsen.util 77 :invoke :add 5262 INFO [2022-01-10 08:39:27,603] jepsen worker 193 - jepsen.util 193 :invoke :add 5263 INFO [2022-01-10 08:39:27,605] jepsen worker 971 - jepsen.util 971 :invoke :add 5264 INFO [2022-01-10 08:39:27,608] jepsen worker 184 - jepsen.util 184 :invoke :add 5265 INFO [2022-01-10 08:39:27,610] jepsen worker 736 - jepsen.util 736 :invoke :add 5266 INFO [2022-01-10 08:39:27,612] jepsen worker 585 - jepsen.util 585 :invoke :add 5267 INFO [2022-01-10 08:39:27,612] jepsen worker 906 - jepsen.util 906 :ok :add 4643 INFO [2022-01-10 08:39:27,612] jepsen worker 814 - jepsen.util 814 :ok :add 4532 INFO [2022-01-10 08:39:27,612] jepsen worker 711 - jepsen.util 711 :ok :add 4496 INFO [2022-01-10 08:39:27,612] jepsen worker 638 - jepsen.util 638 :ok :add 4677 INFO [2022-01-10 08:39:27,612] jepsen worker 209 - jepsen.util 209 :ok :add 4506 INFO [2022-01-10 08:39:27,615] jepsen worker 24 - jepsen.util 24 :ok :add 4903 INFO [2022-01-10 08:39:27,615] jepsen worker 758 - jepsen.util 758 :ok :add 5007 INFO [2022-01-10 08:39:27,615] jepsen worker 328 - jepsen.util 328 :ok :add 5197 INFO [2022-01-10 08:39:27,615] jepsen worker 230 - jepsen.util 230 :ok :add 5033 INFO [2022-01-10 08:39:27,615] jepsen worker 780 - jepsen.util 780 :ok :add 4937 INFO [2022-01-10 08:39:27,618] jepsen worker 542 - jepsen.util 542 :invoke :add 5268 INFO [2022-01-10 08:39:27,623] jepsen worker 727 - jepsen.util 727 :invoke :add 5269 INFO [2022-01-10 08:39:27,623] jepsen worker 94 - jepsen.util 94 :ok :add 5042 INFO [2022-01-10 08:39:27,624] jepsen worker 703 - jepsen.util 703 :ok :add 4972 INFO [2022-01-10 08:39:27,624] jepsen worker 162 - jepsen.util 162 :ok :add 4946 INFO [2022-01-10 08:39:27,624] jepsen worker 130 - jepsen.util 130 :ok :add 5076 INFO [2022-01-10 08:39:27,626] jepsen worker 170 - jepsen.util 170 :invoke :add 5270 INFO [2022-01-10 08:39:27,629] jepsen worker 776 - jepsen.util 776 :ok :add 5206 INFO [2022-01-10 08:39:27,629] jepsen worker 938 - jepsen.util 938 :ok :add 5177 INFO [2022-01-10 08:39:27,629] jepsen worker 623 - jepsen.util 623 :ok :add 5143 INFO [2022-01-10 08:39:27,629] jepsen worker 191 - jepsen.util 191 :ok :add 5232 INFO [2022-01-10 08:39:27,631] jepsen worker 988 - jepsen.util 988 :invoke :add 5271 INFO [2022-01-10 08:39:27,636] jepsen worker 874 - jepsen.util 874 :invoke :add 5272 INFO [2022-01-10 08:39:27,638] jepsen worker 879 - jepsen.util 879 :invoke :add 5273 INFO [2022-01-10 08:39:27,640] jepsen worker 291 - jepsen.util 291 :invoke :add 5274 INFO [2022-01-10 08:39:27,642] jepsen worker 623 - jepsen.util 623 :invoke :add 5275 INFO [2022-01-10 08:39:27,644] jepsen worker 6 - jepsen.util 6 :invoke :add 5276 INFO [2022-01-10 08:39:27,647] jepsen worker 495 - jepsen.util 495 :invoke :add 5277 INFO [2022-01-10 08:39:27,649] jepsen worker 832 - jepsen.util 832 :invoke :add 5278 INFO [2022-01-10 08:39:27,651] jepsen worker 456 - jepsen.util 456 :invoke :add 5279 INFO [2022-01-10 08:39:27,654] jepsen worker 209 - jepsen.util 209 :invoke :add 5280 INFO [2022-01-10 08:39:27,656] jepsen worker 42 - jepsen.util 42 :invoke :add 5281 INFO [2022-01-10 08:39:27,658] jepsen worker 195 - jepsen.util 195 :invoke :add 5282 INFO [2022-01-10 08:39:27,660] jepsen worker 566 - jepsen.util 566 :invoke :add 5283 INFO [2022-01-10 08:39:27,662] jepsen worker 839 - jepsen.util 839 :invoke :add 5284 INFO [2022-01-10 08:39:27,663] jepsen worker 486 - jepsen.util 486 :ok :add 4971 INFO [2022-01-10 08:39:27,663] jepsen worker 480 - jepsen.util 480 :ok :add 4969 INFO [2022-01-10 08:39:27,663] jepsen worker 841 - jepsen.util 841 :ok :add 5041 INFO [2022-01-10 08:39:27,663] jepsen worker 508 - jepsen.util 508 :ok :add 5059 INFO [2022-01-10 08:39:27,663] jepsen worker 408 - jepsen.util 408 :ok :add 5075 INFO [2022-01-10 08:39:27,665] jepsen worker 847 - jepsen.util 847 :invoke :add 5285 INFO [2022-01-10 08:39:27,668] jepsen worker 506 - jepsen.util 506 :invoke :add 5286 INFO [2022-01-10 08:39:27,668] jepsen worker 398 - jepsen.util 398 :ok :add 5140 INFO [2022-01-10 08:39:27,669] jepsen worker 683 - jepsen.util 683 :ok :add 5174 INFO [2022-01-10 08:39:27,669] jepsen worker 979 - jepsen.util 979 :ok :add 5205 INFO [2022-01-10 08:39:27,670] jepsen worker 867 - jepsen.util 867 :ok :add 5229 INFO [2022-01-10 08:39:27,670] jepsen worker 998 - jepsen.util 998 :ok :add 5231 INFO [2022-01-10 08:39:27,670] jepsen worker 778 - jepsen.util 778 :ok :add 5158 INFO [2022-01-10 08:39:27,671] jepsen worker 329 - jepsen.util 329 :invoke :add 5287 INFO [2022-01-10 08:39:27,674] jepsen worker 29 - jepsen.util 29 :invoke :add 5288 INFO [2022-01-10 08:39:27,676] jepsen worker 352 - jepsen.util 352 :invoke :add 5289 INFO [2022-01-10 08:39:27,679] jepsen worker 207 - jepsen.util 207 :invoke :add 5290 INFO [2022-01-10 08:39:27,682] jepsen worker 901 - jepsen.util 901 :invoke :add 5291 INFO [2022-01-10 08:39:27,685] jepsen worker 604 - jepsen.util 604 :invoke :add 5292 INFO [2022-01-10 08:39:27,687] jepsen worker 525 - jepsen.util 525 :invoke :add 5293 INFO [2022-01-10 08:39:27,689] jepsen worker 197 - jepsen.util 197 :invoke :add 5294 INFO [2022-01-10 08:39:27,692] jepsen worker 289 - jepsen.util 289 :invoke :add 5295 INFO [2022-01-10 08:39:27,693] jepsen worker 706 - jepsen.util 706 :ok :add 4474 INFO [2022-01-10 08:39:27,694] jepsen worker 532 - jepsen.util 532 :invoke :add 5296 INFO [2022-01-10 08:39:27,694] jepsen worker 981 - jepsen.util 981 :ok :add 4731 INFO [2022-01-10 08:39:27,694] jepsen worker 915 - jepsen.util 915 :ok :add 4440 INFO [2022-01-10 08:39:27,694] jepsen worker 899 - jepsen.util 899 :ok :add 4729 INFO [2022-01-10 08:39:27,694] jepsen worker 645 - jepsen.util 645 :ok :add 4695 INFO [2022-01-10 08:39:27,694] jepsen worker 570 - jepsen.util 570 :ok :add 4705 INFO [2022-01-10 08:39:27,694] jepsen worker 410 - jepsen.util 410 :ok :add 4458 INFO [2022-01-10 08:39:27,694] jepsen worker 357 - jepsen.util 357 :ok :add 4814 INFO [2022-01-10 08:39:27,698] jepsen worker 335 - jepsen.util 335 :invoke :add 5297 INFO [2022-01-10 08:39:27,700] jepsen worker 426 - jepsen.util 426 :invoke :add 5298 INFO [2022-01-10 08:39:27,702] jepsen worker 283 - jepsen.util 283 :invoke :add 5299 INFO [2022-01-10 08:39:27,704] jepsen worker 382 - jepsen.util 382 :invoke :add 5300 INFO [2022-01-10 08:39:27,707] jepsen worker 185 - jepsen.util 185 :invoke :add 5301 INFO [2022-01-10 08:39:27,709] jepsen worker 485 - jepsen.util 485 :invoke :add 5302 INFO [2022-01-10 08:39:27,713] jepsen worker 306 - jepsen.util 306 :invoke :add 5303 INFO [2022-01-10 08:39:27,715] jepsen worker 985 - jepsen.util 985 :invoke :add 5304 INFO [2022-01-10 08:39:27,718] jepsen worker 843 - jepsen.util 843 :invoke :add 5305 INFO [2022-01-10 08:39:27,720] jepsen worker 63 - jepsen.util 63 :invoke :add 5306 INFO [2022-01-10 08:39:27,724] jepsen worker 365 - jepsen.util 365 :invoke :add 5307 INFO [2022-01-10 08:39:27,727] jepsen worker 889 - jepsen.util 889 :invoke :add 5308 INFO [2022-01-10 08:39:27,729] jepsen worker 728 - jepsen.util 728 :invoke :add 5309 INFO [2022-01-10 08:39:27,731] jepsen worker 476 - jepsen.util 476 :invoke :add 5310 INFO [2022-01-10 08:39:27,733] jepsen worker 141 - jepsen.util 141 :invoke :add 5311 INFO [2022-01-10 08:39:27,735] jepsen worker 405 - jepsen.util 405 :invoke :add 5312 INFO [2022-01-10 08:39:27,739] jepsen worker 22 - jepsen.util 22 :invoke :add 5313 INFO [2022-01-10 08:39:27,741] jepsen worker 983 - jepsen.util 983 :invoke :add 5314 INFO [2022-01-10 08:39:27,744] jepsen worker 734 - jepsen.util 734 :invoke :add 5315 INFO [2022-01-10 08:39:27,746] jepsen worker 635 - jepsen.util 635 :invoke :add 5316 INFO [2022-01-10 08:39:27,748] jepsen worker 511 - jepsen.util 511 :invoke :add 5317 INFO [2022-01-10 08:39:27,749] jepsen worker 461 - jepsen.util 461 :ok :add 5077 INFO [2022-01-10 08:39:27,749] jepsen worker 440 - jepsen.util 440 :ok :add 5043 INFO [2022-01-10 08:39:27,750] jepsen worker 208 - jepsen.util 208 :ok :add 5142 INFO [2022-01-10 08:39:27,750] jepsen worker 911 - jepsen.util 911 :ok :add 5207 INFO [2022-01-10 08:39:27,750] jepsen worker 171 - jepsen.util 171 :ok :add 5233 INFO [2022-01-10 08:39:27,750] jepsen worker 465 - jepsen.util 465 :ok :add 5176 INFO [2022-01-10 08:39:27,751] jepsen worker 366 - jepsen.util 366 :invoke :add 5318 INFO [2022-01-10 08:39:27,754] jepsen worker 804 - jepsen.util 804 :ok :add 5137 INFO [2022-01-10 08:39:27,754] jepsen worker 544 - jepsen.util 544 :invoke :add 5319 INFO [2022-01-10 08:39:27,756] jepsen worker 855 - jepsen.util 855 :invoke :add 5320 INFO [2022-01-10 08:39:27,759] jepsen worker 264 - jepsen.util 264 :invoke :add 5321 INFO [2022-01-10 08:39:27,760] jepsen worker 580 - jepsen.util 580 :ok :add 4441 INFO [2022-01-10 08:39:27,760] jepsen worker 900 - jepsen.util 900 :ok :add 4704 INFO [2022-01-10 08:39:27,760] jepsen worker 994 - jepsen.util 994 :ok :add 4475 INFO [2022-01-10 08:39:27,760] jepsen worker 177 - jepsen.util 177 :ok :add 4694 INFO [2022-01-10 08:39:27,761] jepsen worker 783 - jepsen.util 783 :ok :add 4730 INFO [2022-01-10 08:39:27,761] jepsen worker 754 - jepsen.util 754 :ok :add 4821 INFO [2022-01-10 08:39:27,761] jepsen worker 248 - jepsen.util 248 :ok :add 4459 INFO [2022-01-10 08:39:27,761] jepsen worker 792 - jepsen.util 792 :ok :add 4728 INFO [2022-01-10 08:39:27,763] jepsen worker 273 - jepsen.util 273 :invoke :add 5322 INFO [2022-01-10 08:39:27,768] jepsen worker 717 - jepsen.util 717 :invoke :add 5323 INFO [2022-01-10 08:39:27,770] jepsen worker 932 - jepsen.util 932 :invoke :add 5324 INFO [2022-01-10 08:39:27,773] jepsen worker 555 - jepsen.util 555 :invoke :add 5325 INFO [2022-01-10 08:39:27,775] jepsen worker 48 - jepsen.util 48 :invoke :add 5326 INFO [2022-01-10 08:39:27,777] jepsen worker 499 - jepsen.util 499 :invoke :add 5327 INFO [2022-01-10 08:39:27,779] jepsen worker 719 - jepsen.util 719 :invoke :add 5328 INFO [2022-01-10 08:39:27,782] jepsen worker 96 - jepsen.util 96 :invoke :add 5329 INFO [2022-01-10 08:39:27,784] jepsen worker 154 - jepsen.util 154 :invoke :add 5330 INFO [2022-01-10 08:39:27,786] jepsen worker 666 - jepsen.util 666 :invoke :add 5331 INFO [2022-01-10 08:39:27,789] jepsen worker 790 - jepsen.util 790 :invoke :add 5332 INFO [2022-01-10 08:39:27,791] jepsen worker 911 - jepsen.util 911 :invoke :add 5333 INFO [2022-01-10 08:39:27,793] jepsen worker 618 - jepsen.util 618 :invoke :add 5334 INFO [2022-01-10 08:39:27,796] jepsen worker 516 - jepsen.util 516 :invoke :add 5335 INFO [2022-01-10 08:39:27,799] jepsen worker 836 - jepsen.util 836 :invoke :add 5336 INFO [2022-01-10 08:39:27,801] jepsen worker 991 - jepsen.util 991 :invoke :add 5337 INFO [2022-01-10 08:39:27,804] jepsen worker 27 - jepsen.util 27 :invoke :add 5338 INFO [2022-01-10 08:39:27,807] jepsen worker 676 - jepsen.util 676 :invoke :add 5339 INFO [2022-01-10 08:39:27,807] jepsen worker 113 - jepsen.util 113 :ok :add 5194 INFO [2022-01-10 08:39:27,809] jepsen worker 712 - jepsen.util 712 :invoke :add 5340 INFO [2022-01-10 08:39:27,810] jepsen worker 70 - jepsen.util 70 :ok :add 5040 INFO [2022-01-10 08:39:27,810] jepsen worker 266 - jepsen.util 266 :ok :add 5058 INFO [2022-01-10 08:39:27,810] jepsen worker 843 - jepsen.util 843 :ok :add 5305 INFO [2022-01-10 08:39:27,810] jepsen worker 420 - jepsen.util 420 :ok :add 5074 INFO [2022-01-10 08:39:27,810] jepsen worker 289 - jepsen.util 289 :ok :add 5295 INFO [2022-01-10 08:39:27,813] jepsen worker 240 - jepsen.util 240 :invoke :add 5341 INFO [2022-01-10 08:39:27,818] jepsen worker 61 - jepsen.util 61 :invoke :add 5342 INFO [2022-01-10 08:39:27,820] jepsen worker 237 - jepsen.util 237 :invoke :add 5343 INFO [2022-01-10 08:39:27,823] jepsen worker 756 - jepsen.util 756 :invoke :add 5344 INFO [2022-01-10 08:39:27,826] jepsen worker 102 - jepsen.util 102 :invoke :add 5345 INFO [2022-01-10 08:39:27,829] jepsen worker 332 - jepsen.util 332 :ok :add 5094 INFO [2022-01-10 08:39:27,829] jepsen worker 269 - jepsen.util 269 :ok :add 5128 INFO [2022-01-10 08:39:27,829] jepsen worker 564 - jepsen.util 564 :ok :add 5104 INFO [2022-01-10 08:39:27,829] jepsen worker 965 - jepsen.util 965 :ok :add 5259 INFO [2022-01-10 08:39:27,829] jepsen worker 623 - jepsen.util 623 :ok :add 5275 INFO [2022-01-10 08:39:27,829] jepsen worker 702 - jepsen.util 702 :ok :add 5241 INFO [2022-01-10 08:39:27,829] jepsen worker 45 - jepsen.util 45 :ok :add 5130 INFO [2022-01-10 08:39:27,829] jepsen worker 226 - jepsen.util 226 :invoke :add 5346 INFO [2022-01-10 08:39:27,830] jepsen worker 727 - jepsen.util 727 :ok :add 5269 INFO [2022-01-10 08:39:27,830] jepsen worker 469 - jepsen.util 469 :ok :add 5245 INFO [2022-01-10 08:39:27,831] jepsen worker 988 - jepsen.util 988 :ok :add 5271 INFO [2022-01-10 08:39:27,831] jepsen worker 481 - jepsen.util 481 :ok :add 5134 INFO [2022-01-10 08:39:27,834] jepsen worker 824 - jepsen.util 824 :invoke :add 5347 INFO [2022-01-10 08:39:27,837] jepsen worker 997 - jepsen.util 997 :invoke :add 5348 INFO [2022-01-10 08:39:27,839] jepsen worker 557 - jepsen.util 557 :invoke :add 5349 INFO [2022-01-10 08:39:27,842] jepsen worker 346 - jepsen.util 346 :invoke :add 5350 INFO [2022-01-10 08:39:27,843] jepsen worker 7 - jepsen.util 7 :invoke :add 5351 INFO [2022-01-10 08:39:27,846] jepsen worker 398 - jepsen.util 398 :invoke :add 5352 INFO [2022-01-10 08:39:27,848] jepsen worker 829 - jepsen.util 829 :invoke :add 5353 INFO [2022-01-10 08:39:27,850] jepsen worker 290 - jepsen.util 290 :invoke :add 5354 INFO [2022-01-10 08:39:27,852] jepsen worker 524 - jepsen.util 524 :ok :add 4488 INFO [2022-01-10 08:39:27,852] jepsen worker 46 - jepsen.util 46 :ok :add 4669 INFO [2022-01-10 08:39:27,852] jepsen worker 646 - jepsen.util 646 :ok :add 4671 INFO [2022-01-10 08:39:27,852] jepsen worker 2 - jepsen.util 2 :ok :add 4490 INFO [2022-01-10 08:39:27,853] jepsen worker 698 - jepsen.util 698 :ok :add 4500 INFO [2022-01-10 08:39:27,853] jepsen worker 26 - jepsen.util 26 :ok :add 4518 INFO [2022-01-10 08:39:27,853] jepsen worker 111 - jepsen.util 111 :ok :add 4534 INFO [2022-01-10 08:39:27,853] jepsen worker 288 - jepsen.util 288 :ok :add 4645 INFO [2022-01-10 08:39:27,853] jepsen worker 56 - jepsen.util 56 :invoke :add 5355 INFO [2022-01-10 08:39:27,855] jepsen worker 674 - jepsen.util 674 :invoke :add 5356 INFO [2022-01-10 08:39:27,858] jepsen worker 108 - jepsen.util 108 :invoke :add 5357 INFO [2022-01-10 08:39:27,861] jepsen worker 162 - jepsen.util 162 :invoke :add 5358 INFO [2022-01-10 08:39:27,864] jepsen worker 107 - jepsen.util 107 :invoke :add 5359 INFO [2022-01-10 08:39:27,866] jepsen worker 377 - jepsen.util 377 :invoke :add 5360 INFO [2022-01-10 08:39:27,869] jepsen worker 112 - jepsen.util 112 :invoke :add 5361 INFO [2022-01-10 08:39:27,869] jepsen worker 809 - jepsen.util 809 :ok :add 5106 INFO [2022-01-10 08:39:27,869] jepsen worker 495 - jepsen.util 495 :ok :add 5277 INFO [2022-01-10 08:39:27,869] jepsen worker 619 - jepsen.util 619 :ok :add 5243 INFO [2022-01-10 08:39:27,869] jepsen worker 92 - jepsen.util 92 :ok :add 5132 INFO [2022-01-10 08:39:27,871] jepsen worker 650 - jepsen.util 650 :invoke :add 5362 INFO [2022-01-10 08:39:27,871] jepsen worker 775 - jepsen.util 775 :ok :add 5129 INFO [2022-01-10 08:39:27,872] jepsen worker 327 - jepsen.util 327 :ok :add 5095 INFO [2022-01-10 08:39:27,872] jepsen worker 291 - jepsen.util 291 :ok :add 5274 INFO [2022-01-10 08:39:27,871] jepsen worker 562 - jepsen.util 562 :ok :add 5105 INFO [2022-01-10 08:39:27,872] jepsen worker 244 - jepsen.util 244 :ok :add 5240 INFO [2022-01-10 08:39:27,872] jepsen worker 8 - jepsen.util 8 :ok :add 5258 INFO [2022-01-10 08:39:27,872] jepsen worker 334 - jepsen.util 334 :ok :add 5131 INFO [2022-01-10 08:39:27,876] jepsen worker 860 - jepsen.util 860 :invoke :add 5363 INFO [2022-01-10 08:39:27,878] jepsen worker 215 - jepsen.util 215 :invoke :add 5364 INFO [2022-01-10 08:39:27,880] jepsen worker 812 - jepsen.util 812 :invoke :add 5365 INFO [2022-01-10 08:39:27,882] jepsen worker 144 - jepsen.util 144 :invoke :add 5366 INFO [2022-01-10 08:39:27,885] jepsen worker 744 - jepsen.util 744 :invoke :add 5367 INFO [2022-01-10 08:39:27,887] jepsen worker 888 - jepsen.util 888 :invoke :add 5368 INFO [2022-01-10 08:39:27,888] jepsen worker 73 - jepsen.util 73 :ok :add 4396 INFO [2022-01-10 08:39:27,889] jepsen worker 178 - jepsen.util 178 :ok :add 4540 INFO [2022-01-10 08:39:27,889] jepsen worker 274 - jepsen.util 274 :ok :add 4629 INFO [2022-01-10 08:39:27,889] jepsen worker 851 - jepsen.util 851 :ok :add 4574 INFO [2022-01-10 08:39:27,889] jepsen worker 44 - jepsen.util 44 :ok :add 4631 INFO [2022-01-10 08:39:27,889] jepsen worker 331 - jepsen.util 331 :invoke :add 5369 INFO [2022-01-10 08:39:27,890] jepsen worker 818 - jepsen.util 818 :ok :add 4605 INFO [2022-01-10 08:39:27,890] jepsen worker 65 - jepsen.util 65 :ok :add 4558 INFO [2022-01-10 08:39:27,893] jepsen worker 480 - jepsen.util 480 :invoke :add 5370 INFO [2022-01-10 08:39:27,896] jepsen worker 754 - jepsen.util 754 :invoke :add 5371 INFO [2022-01-10 08:39:27,897] jepsen worker 0 - jepsen.util 0 :invoke :add 5372 INFO [2022-01-10 08:39:27,900] jepsen worker 946 - jepsen.util 946 :invoke :add 5373 INFO [2022-01-10 08:39:27,902] jepsen worker 275 - jepsen.util 275 :invoke :add 5374 INFO [2022-01-10 08:39:27,904] jepsen worker 471 - jepsen.util 471 :invoke :add 5375 INFO [2022-01-10 08:39:27,906] jepsen worker 470 - jepsen.util 470 :invoke :add 5376 INFO [2022-01-10 08:39:27,908] jepsen worker 70 - jepsen.util 70 :invoke :add 5377 INFO [2022-01-10 08:39:27,911] jepsen worker 895 - jepsen.util 895 :invoke :add 5378 INFO [2022-01-10 08:39:27,914] jepsen worker 133 - jepsen.util 133 :invoke :add 5379 INFO [2022-01-10 08:39:27,916] jepsen worker 845 - jepsen.util 845 :invoke :add 5380 INFO [2022-01-10 08:39:27,918] jepsen worker 813 - jepsen.util 813 :invoke :add 5381 INFO [2022-01-10 08:39:27,921] jepsen worker 98 - jepsen.util 98 :invoke :add 5382 INFO [2022-01-10 08:39:27,923] jepsen worker 408 - jepsen.util 408 :invoke :add 5383 INFO [2022-01-10 08:39:27,925] jepsen worker 332 - jepsen.util 332 :invoke :add 5384 INFO [2022-01-10 08:39:27,928] jepsen worker 25 - jepsen.util 25 :invoke :add 5385 INFO [2022-01-10 08:39:27,930] jepsen worker 164 - jepsen.util 164 :invoke :add 5386 INFO [2022-01-10 08:39:27,932] jepsen worker 375 - jepsen.util 375 :invoke :add 5387 INFO [2022-01-10 08:39:27,934] jepsen worker 142 - jepsen.util 142 :invoke :add 5388 INFO [2022-01-10 08:39:27,937] jepsen worker 103 - jepsen.util 103 :invoke :add 5389 INFO [2022-01-10 08:39:27,939] jepsen worker 189 - jepsen.util 189 :invoke :add 5390 INFO [2022-01-10 08:39:27,941] jepsen worker 995 - jepsen.util 995 :invoke :add 5391 INFO [2022-01-10 08:39:27,943] jepsen worker 840 - jepsen.util 840 :invoke :add 5392 INFO [2022-01-10 08:39:27,946] jepsen worker 573 - jepsen.util 573 :invoke :add 5393 INFO [2022-01-10 08:39:27,947] jepsen worker 343 - jepsen.util 343 :invoke :add 5394 INFO [2022-01-10 08:39:27,950] jepsen worker 776 - jepsen.util 776 :invoke :add 5395 INFO [2022-01-10 08:39:27,952] jepsen worker 957 - jepsen.util 957 :invoke :add 5396 INFO [2022-01-10 08:39:27,955] jepsen worker 179 - jepsen.util 179 :invoke :add 5397 INFO [2022-01-10 08:39:27,956] jepsen worker 610 - jepsen.util 610 :invoke :add 5398 INFO [2022-01-10 08:39:27,959] jepsen worker 696 - jepsen.util 696 :invoke :add 5399 INFO [2022-01-10 08:39:27,961] jepsen worker 305 - jepsen.util 305 :invoke :add 5400 INFO [2022-01-10 08:39:27,963] jepsen worker 831 - jepsen.util 831 :invoke :add 5401 INFO [2022-01-10 08:39:27,965] jepsen worker 902 - jepsen.util 902 :invoke :add 5402 INFO [2022-01-10 08:39:27,968] jepsen worker 942 - jepsen.util 942 :invoke :add 5403 INFO [2022-01-10 08:39:27,970] jepsen worker 35 - jepsen.util 35 :invoke :add 5404 INFO [2022-01-10 08:39:27,972] jepsen worker 624 - jepsen.util 624 :invoke :add 5405 INFO [2022-01-10 08:39:27,974] jepsen worker 361 - jepsen.util 361 :invoke :add 5406 INFO [2022-01-10 08:39:27,976] jepsen worker 47 - jepsen.util 47 :invoke :add 5407 INFO [2022-01-10 08:39:27,979] jepsen worker 930 - jepsen.util 930 :invoke :add 5408 INFO [2022-01-10 08:39:27,981] jepsen worker 627 - jepsen.util 627 :invoke :add 5409 INFO [2022-01-10 08:39:27,983] jepsen worker 743 - jepsen.util 743 :invoke :add 5410 INFO [2022-01-10 08:39:27,985] jepsen worker 152 - jepsen.util 152 :invoke :add 5411 INFO [2022-01-10 08:39:27,988] jepsen worker 554 - jepsen.util 554 :invoke :add 5412 INFO [2022-01-10 08:39:27,990] jepsen worker 507 - jepsen.util 507 :invoke :add 5413 INFO [2022-01-10 08:39:27,992] jepsen worker 609 - jepsen.util 609 :invoke :add 5414 INFO [2022-01-10 08:39:27,994] jepsen worker 770 - jepsen.util 770 :invoke :add 5415 INFO [2022-01-10 08:39:27,996] jepsen worker 706 - jepsen.util 706 :invoke :add 5416 INFO [2022-01-10 08:39:27,999] jepsen worker 972 - jepsen.util 972 :invoke :add 5417 INFO [2022-01-10 08:39:28,001] jepsen worker 201 - jepsen.util 201 :invoke :add 5418 INFO [2022-01-10 08:39:28,002] jepsen worker 17 - jepsen.util 17 :ok :add 5235 INFO [2022-01-10 08:39:28,002] jepsen worker 181 - jepsen.util 181 :ok :add 5144 INFO [2022-01-10 08:39:28,003] jepsen worker 571 - jepsen.util 571 :ok :add 5219 INFO [2022-01-10 08:39:28,003] jepsen worker 28 - jepsen.util 28 :ok :add 5201 INFO [2022-01-10 08:39:28,003] jepsen worker 563 - jepsen.util 563 :invoke :add 5419 INFO [2022-01-10 08:39:28,003] jepsen worker 868 - jepsen.util 868 :ok :add 5170 INFO [2022-01-10 08:39:28,003] jepsen worker 579 - jepsen.util 579 :ok :add 5168 INFO [2022-01-10 08:39:28,006] jepsen worker 171 - jepsen.util 171 :invoke :add 5420 INFO [2022-01-10 08:39:28,008] jepsen worker 244 - jepsen.util 244 :invoke :add 5421 INFO [2022-01-10 08:39:28,011] jepsen worker 915 - jepsen.util 915 :invoke :add 5422 INFO [2022-01-10 08:39:28,013] jepsen worker 147 - jepsen.util 147 :invoke :add 5423 INFO [2022-01-10 08:39:28,015] jepsen worker 509 - jepsen.util 509 :invoke :add 5424 INFO [2022-01-10 08:39:28,017] jepsen worker 266 - jepsen.util 266 :invoke :add 5425 INFO [2022-01-10 08:39:28,019] jepsen worker 916 - jepsen.util 916 :invoke :add 5426 INFO [2022-01-10 08:39:28,021] jepsen worker 552 - jepsen.util 552 :invoke :add 5427 INFO [2022-01-10 08:39:28,024] jepsen worker 914 - jepsen.util 914 :invoke :add 5428 INFO [2022-01-10 08:39:28,026] jepsen worker 974 - jepsen.util 974 :invoke :add 5429 INFO [2022-01-10 08:39:28,028] jepsen worker 751 - jepsen.util 751 :invoke :add 5430 INFO [2022-01-10 08:39:28,030] jepsen worker 794 - jepsen.util 794 :invoke :add 5431 INFO [2022-01-10 08:39:28,032] jepsen worker 745 - jepsen.util 745 :invoke :add 5432 INFO [2022-01-10 08:39:28,035] jepsen worker 703 - jepsen.util 703 :invoke :add 5433 INFO [2022-01-10 08:39:28,037] jepsen worker 560 - jepsen.util 560 :invoke :add 5434 INFO [2022-01-10 08:39:28,039] jepsen worker 117 - jepsen.util 117 :invoke :add 5435 INFO [2022-01-10 08:39:28,041] jepsen worker 413 - jepsen.util 413 :invoke :add 5436 INFO [2022-01-10 08:39:28,043] jepsen worker 36 - jepsen.util 36 :invoke :add 5437 INFO [2022-01-10 08:39:28,045] jepsen worker 562 - jepsen.util 562 :invoke :add 5438 INFO [2022-01-10 08:39:28,048] jepsen worker 369 - jepsen.util 369 :invoke :add 5439 INFO [2022-01-10 08:39:28,050] jepsen worker 589 - jepsen.util 589 :invoke :add 5440 INFO [2022-01-10 08:39:28,052] jepsen worker 457 - jepsen.util 457 :invoke :add 5441 INFO [2022-01-10 08:39:28,054] jepsen worker 795 - jepsen.util 795 :invoke :add 5442 INFO [2022-01-10 08:39:28,057] jepsen worker 820 - jepsen.util 820 :invoke :add 5443 INFO [2022-01-10 08:39:28,059] jepsen worker 167 - jepsen.util 167 :invoke :add 5444 INFO [2022-01-10 08:39:28,061] jepsen worker 924 - jepsen.util 924 :invoke :add 5445 INFO [2022-01-10 08:39:28,064] jepsen worker 388 - jepsen.util 388 :invoke :add 5446 INFO [2022-01-10 08:39:28,066] jepsen worker 86 - jepsen.util 86 :invoke :add 5447 INFO [2022-01-10 08:39:28,068] jepsen worker 994 - jepsen.util 994 :invoke :add 5448 INFO [2022-01-10 08:39:28,070] jepsen worker 359 - jepsen.util 359 :invoke :add 5449 INFO [2022-01-10 08:39:28,072] jepsen worker 996 - jepsen.util 996 :invoke :add 5450 INFO [2022-01-10 08:39:28,074] jepsen worker 663 - jepsen.util 663 :invoke :add 5451 INFO [2022-01-10 08:39:28,076] jepsen worker 969 - jepsen.util 969 :invoke :add 5452 INFO [2022-01-10 08:39:28,079] jepsen worker 211 - jepsen.util 211 :invoke :add 5453 INFO [2022-01-10 08:39:28,081] jepsen worker 87 - jepsen.util 87 :invoke :add 5454 INFO [2022-01-10 08:39:28,083] jepsen worker 236 - jepsen.util 236 :invoke :add 5455 INFO [2022-01-10 08:39:28,085] jepsen worker 873 - jepsen.util 873 :invoke :add 5456 INFO [2022-01-10 08:39:28,087] jepsen worker 992 - jepsen.util 992 :ok :add 5242 INFO [2022-01-10 08:39:28,087] jepsen worker 6 - jepsen.util 6 :ok :add 5276 INFO [2022-01-10 08:39:28,087] jepsen worker 937 - jepsen.util 937 :invoke :add 5457 INFO [2022-01-10 08:39:28,090] jepsen worker 129 - jepsen.util 129 :invoke :add 5458 INFO [2022-01-10 08:39:28,092] jepsen worker 483 - jepsen.util 483 :invoke :add 5459 INFO [2022-01-10 08:39:28,094] jepsen worker 619 - jepsen.util 619 :invoke :add 5460 INFO [2022-01-10 08:39:28,096] jepsen worker 979 - jepsen.util 979 :invoke :add 5461 INFO [2022-01-10 08:39:28,098] jepsen worker 440 - jepsen.util 440 :invoke :add 5462 INFO [2022-01-10 08:39:28,100] jepsen worker 921 - jepsen.util 921 :invoke :add 5463 INFO [2022-01-10 08:39:28,103] jepsen worker 597 - jepsen.util 597 :ok :add 4528 INFO [2022-01-10 08:39:28,103] jepsen worker 892 - jepsen.util 892 :invoke :add 5464 INFO [2022-01-10 08:39:28,103] jepsen worker 401 - jepsen.util 401 :ok :add 4675 INFO [2022-01-10 08:39:28,103] jepsen worker 679 - jepsen.util 679 :ok :add 4504 INFO [2022-01-10 08:39:28,103] jepsen worker 194 - jepsen.util 194 :ok :add 4530 INFO [2022-01-10 08:39:28,103] jepsen worker 628 - jepsen.util 628 :ok :add 4641 INFO [2022-01-10 08:39:28,103] jepsen worker 434 - jepsen.util 434 :ok :add 4494 INFO [2022-01-10 08:39:28,103] jepsen worker 546 - jepsen.util 546 :ok :add 4659 INFO [2022-01-10 08:39:28,104] jepsen worker 237 - jepsen.util 237 :ok :add 5343 INFO [2022-01-10 08:39:28,104] jepsen worker 246 - jepsen.util 246 :ok :add 5196 INFO [2022-01-10 08:39:28,104] jepsen worker 624 - jepsen.util 624 :ok :add 5405 INFO [2022-01-10 08:39:28,104] jepsen worker 974 - jepsen.util 974 :ok :add 5429 INFO [2022-01-10 08:39:28,104] jepsen worker 70 - jepsen.util 70 :ok :add 5377 INFO [2022-01-10 08:39:28,104] jepsen worker 794 - jepsen.util 794 :ok :add 5431 INFO [2022-01-10 08:39:28,109] jepsen worker 833 - jepsen.util 833 :invoke :add 5465 INFO [2022-01-10 08:39:28,112] jepsen worker 199 - jepsen.util 199 :invoke :add 5466 INFO [2022-01-10 08:39:28,115] jepsen worker 104 - jepsen.util 104 :invoke :add 5467 INFO [2022-01-10 08:39:28,116] jepsen worker 74 - jepsen.util 74 :invoke :add 5468 INFO [2022-01-10 08:39:28,118] jepsen worker 919 - jepsen.util 919 :invoke :add 5469 INFO [2022-01-10 08:39:28,121] jepsen worker 383 - jepsen.util 383 :invoke :add 5470 INFO [2022-01-10 08:39:28,123] jepsen worker 567 - jepsen.util 567 :invoke :add 5471 INFO [2022-01-10 08:39:28,126] jepsen worker 320 - jepsen.util 320 :invoke :add 5472 INFO [2022-01-10 08:39:28,128] jepsen worker 851 - jepsen.util 851 :invoke :add 5473 INFO [2022-01-10 08:39:28,131] jepsen worker 156 - jepsen.util 156 :invoke :add 5474 INFO [2022-01-10 08:39:28,133] jepsen worker 300 - jepsen.util 300 :invoke :add 5475 INFO [2022-01-10 08:39:28,136] jepsen worker 648 - jepsen.util 648 :ok :add 5136 INFO [2022-01-10 08:39:28,136] jepsen worker 879 - jepsen.util 879 :ok :add 5273 INFO [2022-01-10 08:39:28,136] jepsen worker 821 - jepsen.util 821 :invoke :add 5476 INFO [2022-01-10 08:39:28,136] jepsen worker 929 - jepsen.util 929 :ok :add 5247 INFO [2022-01-10 08:39:28,139] jepsen worker 598 - jepsen.util 598 :invoke :add 5477 INFO [2022-01-10 08:39:28,141] jepsen worker 907 - jepsen.util 907 :invoke :add 5478 INFO [2022-01-10 08:39:28,143] jepsen worker 291 - jepsen.util 291 :invoke :add 5479 INFO [2022-01-10 08:39:28,145] jepsen worker 326 - jepsen.util 326 :invoke :add 5480 INFO [2022-01-10 08:39:28,147] jepsen worker 701 - jepsen.util 701 :invoke :add 5481 INFO [2022-01-10 08:39:28,149] jepsen worker 691 - jepsen.util 691 :invoke :add 5482 INFO [2022-01-10 08:39:28,152] jepsen worker 421 - jepsen.util 421 :invoke :add 5483 INFO [2022-01-10 08:39:28,154] jepsen worker 705 - jepsen.util 705 :invoke :add 5484 INFO [2022-01-10 08:39:28,156] jepsen worker 945 - jepsen.util 945 :invoke :add 5485 INFO [2022-01-10 08:39:28,157] jepsen worker 765 - jepsen.util 765 :invoke :add 5486 INFO [2022-01-10 08:39:28,160] jepsen worker 750 - jepsen.util 750 :invoke :add 5487 INFO [2022-01-10 08:39:28,162] jepsen worker 695 - jepsen.util 695 :invoke :add 5488 INFO [2022-01-10 08:39:28,165] jepsen worker 488 - jepsen.util 488 :invoke :add 5489 INFO [2022-01-10 08:39:28,167] jepsen worker 277 - jepsen.util 277 :invoke :add 5490 INFO [2022-01-10 08:39:28,169] jepsen worker 814 - jepsen.util 814 :invoke :add 5491 INFO [2022-01-10 08:39:28,172] jepsen worker 961 - jepsen.util 961 :invoke :add 5492 INFO [2022-01-10 08:39:28,174] jepsen worker 378 - jepsen.util 378 :invoke :add 5493 INFO [2022-01-10 08:39:28,177] jepsen worker 512 - jepsen.util 512 :invoke :add 5494 INFO [2022-01-10 08:39:28,179] jepsen worker 360 - jepsen.util 360 :invoke :add 5495 INFO [2022-01-10 08:39:28,181] jepsen worker 613 - jepsen.util 613 :invoke :add 5496 INFO [2022-01-10 08:39:28,183] jepsen worker 431 - jepsen.util 431 :invoke :add 5497 INFO [2022-01-10 08:39:28,185] jepsen worker 990 - jepsen.util 990 :invoke :add 5498 INFO [2022-01-10 08:39:28,187] jepsen worker 740 - jepsen.util 740 :invoke :add 5499 INFO [2022-01-10 08:39:28,189] jepsen worker 759 - jepsen.util 759 :invoke :add 5500 INFO [2022-01-10 08:39:28,192] jepsen worker 425 - jepsen.util 425 :invoke :add 5501 INFO [2022-01-10 08:39:28,194] jepsen worker 597 - jepsen.util 597 :invoke :add 5502 INFO [2022-01-10 08:39:28,196] jepsen worker 124 - jepsen.util 124 :invoke :add 5503 INFO [2022-01-10 08:39:28,199] jepsen worker 569 - jepsen.util 569 :invoke :add 5504 INFO [2022-01-10 08:39:28,201] jepsen worker 576 - jepsen.util 576 :invoke :add 5505 INFO [2022-01-10 08:39:28,203] jepsen worker 328 - jepsen.util 328 :invoke :add 5506 INFO [2022-01-10 08:39:28,205] jepsen worker 316 - jepsen.util 316 :invoke :add 5507 INFO [2022-01-10 08:39:28,207] jepsen worker 452 - jepsen.util 452 :invoke :add 5508 INFO [2022-01-10 08:39:28,209] jepsen worker 134 - jepsen.util 134 :invoke :add 5509 INFO [2022-01-10 08:39:28,212] jepsen worker 574 - jepsen.util 574 :invoke :add 5510 INFO [2022-01-10 08:39:28,214] jepsen worker 909 - jepsen.util 909 :invoke :add 5511 INFO [2022-01-10 08:39:28,216] jepsen worker 865 - jepsen.util 865 :invoke :add 5512 INFO [2022-01-10 08:39:28,218] jepsen worker 918 - jepsen.util 918 :invoke :add 5513 INFO [2022-01-10 08:39:28,220] jepsen worker 465 - jepsen.util 465 :invoke :add 5514 INFO [2022-01-10 08:39:28,220] jepsen worker 458 - jepsen.util 458 :ok :add 5146 INFO [2022-01-10 08:39:28,221] jepsen worker 297 - jepsen.util 297 :ok :add 5203 INFO [2022-01-10 08:39:28,221] jepsen worker 952 - jepsen.util 952 :ok :add 5172 INFO [2022-01-10 08:39:28,221] jepsen worker 341 - jepsen.util 341 :ok :add 5237 INFO [2022-01-10 08:39:28,221] jepsen worker 573 - jepsen.util 573 :ok :add 5393 INFO [2022-01-10 08:39:28,224] jepsen worker 418 - jepsen.util 418 :invoke :add 5515 INFO [2022-01-10 08:39:28,227] jepsen worker 178 - jepsen.util 178 :invoke :add 5516 INFO [2022-01-10 08:39:28,228] jepsen worker 319 - jepsen.util 319 :invoke :add 5517 INFO [2022-01-10 08:39:28,230] jepsen worker 866 - jepsen.util 866 :invoke :add 5518 INFO [2022-01-10 08:39:28,233] jepsen worker 565 - jepsen.util 565 :invoke :add 5519 INFO [2022-01-10 08:39:28,235] jepsen worker 228 - jepsen.util 228 :invoke :add 5520 INFO [2022-01-10 08:39:28,237] jepsen worker 885 - jepsen.util 885 :invoke :add 5521 INFO [2022-01-10 08:39:28,239] jepsen worker 868 - jepsen.util 868 :invoke :add 5522 INFO [2022-01-10 08:39:28,242] jepsen worker 38 - jepsen.util 38 :ok :add 5169 INFO [2022-01-10 08:39:28,242] jepsen worker 970 - jepsen.util 970 :ok :add 5234 INFO [2022-01-10 08:39:28,242] jepsen worker 194 - jepsen.util 194 :invoke :add 5523 INFO [2022-01-10 08:39:28,242] jepsen worker 925 - jepsen.util 925 :ok :add 5200 INFO [2022-01-10 08:39:28,242] jepsen worker 522 - jepsen.util 522 :ok :add 5218 INFO [2022-01-10 08:39:28,243] jepsen worker 661 - jepsen.util 661 :ok :add 5145 INFO [2022-01-10 08:39:28,243] jepsen worker 196 - jepsen.util 196 :ok :add 5171 INFO [2022-01-10 08:39:28,243] jepsen worker 189 - jepsen.util 189 :ok :add 5390 INFO [2022-01-10 08:39:28,243] jepsen worker 142 - jepsen.util 142 :ok :add 5388 INFO [2022-01-10 08:39:28,246] jepsen worker 718 - jepsen.util 718 :invoke :add 5524 INFO [2022-01-10 08:39:28,248] jepsen worker 987 - jepsen.util 987 :invoke :add 5525 INFO [2022-01-10 08:39:28,251] jepsen worker 278 - jepsen.util 278 :invoke :add 5526 INFO [2022-01-10 08:39:28,253] jepsen worker 660 - jepsen.util 660 :invoke :add 5527 INFO [2022-01-10 08:39:28,254] jepsen worker 105 - jepsen.util 105 :ok :add 4542 INFO [2022-01-10 08:39:28,254] jepsen worker 203 - jepsen.util 203 :ok :add 4607 INFO [2022-01-10 08:39:28,255] jepsen worker 713 - jepsen.util 713 :ok :add 4576 INFO [2022-01-10 08:39:28,256] jepsen worker 481 - jepsen.util 481 :invoke :add 5528 INFO [2022-01-10 08:39:28,258] jepsen worker 445 - jepsen.util 445 :invoke :add 5529 INFO [2022-01-10 08:39:28,260] jepsen worker 856 - jepsen.util 856 :ok :add 4797 INFO [2022-01-10 08:39:28,260] jepsen worker 523 - jepsen.util 523 :ok :add 4633 INFO [2022-01-10 08:39:28,260] jepsen worker 625 - jepsen.util 625 :invoke :add 5530 INFO [2022-01-10 08:39:28,264] jepsen worker 591 - jepsen.util 591 :invoke :add 5531 INFO [2022-01-10 08:39:28,266] jepsen worker 816 - jepsen.util 816 :invoke :add 5532 INFO [2022-01-10 08:39:28,268] jepsen worker 120 - jepsen.util 120 :invoke :add 5533 INFO [2022-01-10 08:39:28,270] jepsen worker 297 - jepsen.util 297 :invoke :add 5534 INFO [2022-01-10 08:39:28,272] jepsen worker 237 - jepsen.util 237 :invoke :add 5535 INFO [2022-01-10 08:39:28,273] jepsen worker 652 - jepsen.util 652 :ok :add 5228 INFO [2022-01-10 08:39:28,273] jepsen worker 343 - jepsen.util 343 :ok :add 5394 INFO [2022-01-10 08:39:28,274] jepsen worker 771 - jepsen.util 771 :ok :add 5175 INFO [2022-01-10 08:39:28,275] jepsen worker 764 - jepsen.util 764 :invoke :add 5536 INFO [2022-01-10 08:39:28,275] jepsen worker 225 - jepsen.util 225 :ok :add 5230 INFO [2022-01-10 08:39:28,276] jepsen worker 848 - jepsen.util 848 :ok :add 5204 INFO [2022-01-10 08:39:28,278] jepsen worker 711 - jepsen.util 711 :invoke :add 5537 INFO [2022-01-10 08:39:28,280] jepsen worker 393 - jepsen.util 393 :invoke :add 5538 INFO [2022-01-10 08:39:28,283] jepsen worker 330 - jepsen.util 330 :invoke :add 5539 INFO [2022-01-10 08:39:28,285] jepsen worker 784 - jepsen.util 784 :invoke :add 5540 INFO [2022-01-10 08:39:28,287] jepsen worker 92 - jepsen.util 92 :invoke :add 5541 INFO [2022-01-10 08:39:28,289] jepsen worker 803 - jepsen.util 803 :invoke :add 5542 INFO [2022-01-10 08:39:28,291] jepsen worker 721 - jepsen.util 721 :invoke :add 5543 INFO [2022-01-10 08:39:28,293] jepsen worker 232 - jepsen.util 232 :invoke :add 5544 INFO [2022-01-10 08:39:28,295] jepsen worker 407 - jepsen.util 407 :invoke :add 5545 INFO [2022-01-10 08:39:28,298] jepsen worker 605 - jepsen.util 605 :invoke :add 5546 INFO [2022-01-10 08:39:28,300] jepsen worker 658 - jepsen.util 658 :invoke :add 5547 INFO [2022-01-10 08:39:28,302] jepsen worker 21 - jepsen.util 21 :invoke :add 5548 INFO [2022-01-10 08:39:28,305] jepsen worker 848 - jepsen.util 848 :invoke :add 5549 INFO [2022-01-10 08:39:28,307] jepsen worker 794 - jepsen.util 794 :invoke :add 5550 INFO [2022-01-10 08:39:28,309] jepsen worker 123 - jepsen.util 123 :invoke :add 5551 INFO [2022-01-10 08:39:28,311] jepsen worker 2 - jepsen.util 2 :invoke :add 5552 INFO [2022-01-10 08:39:28,313] jepsen worker 571 - jepsen.util 571 :invoke :add 5553 INFO [2022-01-10 08:39:28,315] jepsen worker 322 - jepsen.util 322 :invoke :add 5554 INFO [2022-01-10 08:39:28,318] jepsen worker 667 - jepsen.util 667 :invoke :add 5555 INFO [2022-01-10 08:39:28,321] jepsen worker 81 - jepsen.util 81 :invoke :add 5556 INFO [2022-01-10 08:39:28,323] jepsen worker 196 - jepsen.util 196 :invoke :add 5557 INFO [2022-01-10 08:39:28,324] jepsen worker 596 - jepsen.util 596 :invoke :add 5558 INFO [2022-01-10 08:39:28,326] jepsen worker 704 - jepsen.util 704 :ok :add 5147 INFO [2022-01-10 08:39:28,326] jepsen worker 756 - jepsen.util 756 :ok :add 5344 INFO [2022-01-10 08:39:28,326] jepsen worker 869 - jepsen.util 869 :ok :add 5236 INFO [2022-01-10 08:39:28,326] jepsen worker 172 - jepsen.util 172 :invoke :add 5559 INFO [2022-01-10 08:39:28,327] jepsen worker 480 - jepsen.util 480 :ok :add 5370 INFO [2022-01-10 08:39:28,327] jepsen worker 902 - jepsen.util 902 :ok :add 5402 INFO [2022-01-10 08:39:28,327] jepsen worker 657 - jepsen.util 657 :ok :add 5202 INFO [2022-01-10 08:39:28,327] jepsen worker 412 - jepsen.util 412 :ok :add 5173 INFO [2022-01-10 08:39:28,327] jepsen worker 413 - jepsen.util 413 :ok :add 5436 INFO [2022-01-10 08:39:28,327] jepsen worker 356 - jepsen.util 356 :ok :add 5189 INFO [2022-01-10 08:39:28,327] jepsen worker 947 - jepsen.util 947 :ok :add 5191 INFO [2022-01-10 08:39:28,327] jepsen worker 888 - jepsen.util 888 :ok :add 5368 INFO [2022-01-10 08:39:28,327] jepsen worker 840 - jepsen.util 840 :ok :add 5392 INFO [2022-01-10 08:39:28,331] jepsen worker 198 - jepsen.util 198 :invoke :add 5560 INFO [2022-01-10 08:39:28,333] jepsen worker 265 - jepsen.util 265 :invoke :add 5561 INFO [2022-01-10 08:39:28,335] jepsen worker 235 - jepsen.util 235 :invoke :add 5562 INFO [2022-01-10 08:39:28,337] jepsen worker 846 - jepsen.util 846 :invoke :add 5563 INFO [2022-01-10 08:39:28,339] jepsen worker 588 - jepsen.util 588 :invoke :add 5564 INFO [2022-01-10 08:39:28,342] jepsen worker 575 - jepsen.util 575 :invoke :add 5565 INFO [2022-01-10 08:39:28,345] jepsen worker 389 - jepsen.util 389 :invoke :add 5566 INFO [2022-01-10 08:39:28,347] jepsen worker 572 - jepsen.util 572 :invoke :add 5567 INFO [2022-01-10 08:39:28,349] jepsen worker 357 - jepsen.util 357 :invoke :add 5568 INFO [2022-01-10 08:39:28,351] jepsen worker 804 - jepsen.util 804 :invoke :add 5569 INFO [2022-01-10 08:39:28,353] jepsen worker 564 - jepsen.util 564 :invoke :add 5570 INFO [2022-01-10 08:39:28,355] jepsen worker 223 - jepsen.util 223 :invoke :add 5571 INFO [2022-01-10 08:39:28,357] jepsen worker 617 - jepsen.util 617 :invoke :add 5572 INFO [2022-01-10 08:39:28,359] jepsen worker 253 - jepsen.util 253 :invoke :add 5573 INFO [2022-01-10 08:39:28,361] jepsen worker 91 - jepsen.util 91 :invoke :add 5574 INFO [2022-01-10 08:39:28,364] jepsen worker 269 - jepsen.util 269 :invoke :add 5575 INFO [2022-01-10 08:39:28,367] jepsen worker 8 - jepsen.util 8 :invoke :add 5576 INFO [2022-01-10 08:39:28,369] jepsen worker 400 - jepsen.util 400 :invoke :add 5577 INFO [2022-01-10 08:39:28,370] jepsen worker 276 - jepsen.util 276 :invoke :add 5578 INFO [2022-01-10 08:39:28,372] jepsen worker 295 - jepsen.util 295 :invoke :add 5579 INFO [2022-01-10 08:39:28,374] jepsen worker 518 - jepsen.util 518 :invoke :add 5580 INFO [2022-01-10 08:39:28,377] jepsen worker 17 - jepsen.util 17 :invoke :add 5581 INFO [2022-01-10 08:39:28,379] jepsen worker 898 - jepsen.util 898 :invoke :add 5582 INFO [2022-01-10 08:39:28,382] jepsen worker 238 - jepsen.util 238 :invoke :add 5583 INFO [2022-01-10 08:39:28,383] jepsen worker 947 - jepsen.util 947 :invoke :add 5584 INFO [2022-01-10 08:39:28,391] jepsen worker 715 - jepsen.util 715 :invoke :add 5585 INFO [2022-01-10 08:39:28,392] jepsen worker 257 - jepsen.util 257 :invoke :add 5586 INFO [2022-01-10 08:39:28,395] jepsen worker 272 - jepsen.util 272 :invoke :add 5587 INFO [2022-01-10 08:39:28,398] jepsen worker 670 - jepsen.util 670 :invoke :add 5588 INFO [2022-01-10 08:39:28,400] jepsen worker 218 - jepsen.util 218 :invoke :add 5589 INFO [2022-01-10 08:39:28,403] jepsen worker 974 - jepsen.util 974 :invoke :add 5590 INFO [2022-01-10 08:39:28,406] jepsen worker 599 - jepsen.util 599 :invoke :add 5591 INFO [2022-01-10 08:39:28,406] jepsen worker 642 - jepsen.util 642 :ok :add 4595 INFO [2022-01-10 08:39:28,407] jepsen worker 528 - jepsen.util 528 :ok :add 5155 INFO [2022-01-10 08:39:28,407] jepsen worker 577 - jepsen.util 577 :ok :add 4758 INFO [2022-01-10 08:39:28,407] jepsen worker 32 - jepsen.util 32 :ok :add 5210 INFO [2022-01-10 08:39:28,407] jepsen worker 631 - jepsen.util 631 :ok :add 5161 INFO [2022-01-10 08:39:28,407] jepsen worker 422 - jepsen.util 422 :ok :add 5179 INFO [2022-01-10 08:39:28,407] jepsen worker 610 - jepsen.util 610 :ok :add 5398 INFO [2022-01-10 08:39:28,407] jepsen worker 479 - jepsen.util 479 :ok :add 4865 INFO [2022-01-10 08:39:28,407] jepsen worker 338 - jepsen.util 338 :ok :add 5224 INFO [2022-01-10 08:39:28,407] jepsen worker 640 - jepsen.util 640 :ok :add 4740 INFO [2022-01-10 08:39:28,408] jepsen worker 66 - jepsen.util 66 :ok :add 4849 INFO [2022-01-10 08:39:28,408] jepsen worker 305 - jepsen.util 305 :ok :add 5400 INFO [2022-01-10 08:39:28,408] jepsen worker 102 - jepsen.util 102 :ok :add 5345 INFO [2022-01-10 08:39:28,409] jepsen worker 608 - jepsen.util 608 :ok :add 5193 INFO [2022-01-10 08:39:28,409] jepsen worker 36 - jepsen.util 36 :ok :add 5437 INFO [2022-01-10 08:39:28,409] jepsen worker 201 - jepsen.util 201 :ok :add 5418 INFO [2022-01-10 08:39:28,409] jepsen worker 226 - jepsen.util 226 :ok :add 5346 INFO [2022-01-10 08:39:28,409] jepsen worker 560 - jepsen.util 560 :ok :add 5434 INFO [2022-01-10 08:39:28,409] jepsen worker 127 - jepsen.util 127 :ok :add 5188 INFO [2022-01-10 08:39:28,409] jepsen worker 0 - jepsen.util 0 :ok :add 5372 INFO [2022-01-10 08:39:28,409] jepsen worker 942 - jepsen.util 942 :ok :add 5403 INFO [2022-01-10 08:39:28,409] jepsen worker 852 - jepsen.util 852 :ok :add 5190 INFO [2022-01-10 08:39:28,409] jepsen worker 331 - jepsen.util 331 :ok :add 5369 INFO [2022-01-10 08:39:28,409] jepsen worker 754 - jepsen.util 754 :ok :add 5371 INFO [2022-01-10 08:39:28,410] jepsen worker 423 - jepsen.util 423 :invoke :add 5592 INFO [2022-01-10 08:39:28,420] jepsen worker 239 - jepsen.util 239 :invoke :add 5593 INFO [2022-01-10 08:39:28,420] jepsen worker 275 - jepsen.util 275 :ok :add 5374 INFO [2022-01-10 08:39:28,420] jepsen worker 745 - jepsen.util 745 :ok :add 5432 INFO [2022-01-10 08:39:28,421] jepsen worker 162 - jepsen.util 162 :ok :add 5358 INFO [2022-01-10 08:39:28,421] jepsen worker 361 - jepsen.util 361 :ok :add 5406 INFO [2022-01-10 08:39:28,421] jepsen worker 712 - jepsen.util 712 :ok :add 5340 INFO [2022-01-10 08:39:28,423] jepsen worker 498 - jepsen.util 498 :invoke :add 5594 INFO [2022-01-10 08:39:28,425] jepsen worker 479 - jepsen.util 479 :invoke :add 5595 INFO [2022-01-10 08:39:28,428] jepsen worker 579 - jepsen.util 579 :invoke :add 5596 INFO [2022-01-10 08:39:28,432] jepsen worker 843 - jepsen.util 843 :invoke :add 5597 INFO [2022-01-10 08:39:28,435] jepsen worker 965 - jepsen.util 965 :invoke :add 5598 INFO [2022-01-10 08:39:28,438] jepsen worker 458 - jepsen.util 458 :invoke :add 5599 INFO [2022-01-10 08:39:28,440] jepsen worker 453 - jepsen.util 453 :invoke :add 5600 INFO [2022-01-10 08:39:28,442] jepsen worker 956 - jepsen.util 956 :invoke :add 5601 INFO [2022-01-10 08:39:28,444] jepsen worker 900 - jepsen.util 900 :invoke :add 5602 INFO [2022-01-10 08:39:28,446] jepsen worker 570 - jepsen.util 570 :invoke :add 5603 INFO [2022-01-10 08:39:28,449] jepsen worker 906 - jepsen.util 906 :invoke :add 5604 INFO [2022-01-10 08:39:28,450] jepsen worker 581 - jepsen.util 581 :invoke :add 5605 INFO [2022-01-10 08:39:28,452] jepsen worker 561 - jepsen.util 561 :invoke :add 5606 INFO [2022-01-10 08:39:28,453] jepsen worker 514 - jepsen.util 514 :ok :add 5244 INFO [2022-01-10 08:39:28,453] jepsen worker 170 - jepsen.util 170 :ok :add 5270 INFO [2022-01-10 08:39:28,453] jepsen worker 824 - jepsen.util 824 :ok :add 5347 INFO [2022-01-10 08:39:28,453] jepsen worker 686 - jepsen.util 686 :ok :add 5192 INFO [2022-01-10 08:39:28,453] jepsen worker 542 - jepsen.util 542 :ok :add 5268 INFO [2022-01-10 08:39:28,454] jepsen worker 356 - jepsen.util 356 :invoke :add 5607 INFO [2022-01-10 08:39:28,455] jepsen worker 961 - jepsen.util 961 :ok :add 5492 INFO [2022-01-10 08:39:28,455] jepsen worker 597 - jepsen.util 597 :ok :add 5502 INFO [2022-01-10 08:39:28,455] jepsen worker 764 - jepsen.util 764 :ok :add 5536 INFO [2022-01-10 08:39:28,456] jepsen worker 831 - jepsen.util 831 :ok :add 5401 INFO [2022-01-10 08:39:28,456] jepsen worker 218 - jepsen.util 218 :ok :add 5589 INFO [2022-01-10 08:39:28,456] jepsen worker 946 - jepsen.util 946 :ok :add 5373 INFO [2022-01-10 08:39:28,456] jepsen worker 563 - jepsen.util 563 :ok :add 5419 INFO [2022-01-10 08:39:28,456] jepsen worker 117 - jepsen.util 117 :ok :add 5435 INFO [2022-01-10 08:39:28,462] jepsen worker 404 - jepsen.util 404 :invoke :add 5608 INFO [2022-01-10 08:39:28,465] jepsen worker 725 - jepsen.util 725 :invoke :add 5609 INFO [2022-01-10 08:39:28,469] jepsen worker 145 - jepsen.util 145 :invoke :add 5610 INFO [2022-01-10 08:39:28,472] jepsen worker 695 - jepsen.util 695 :ok :add 5488 INFO [2022-01-10 08:39:28,473] jepsen worker 874 - jepsen.util 874 :ok :add 5272 INFO [2022-01-10 08:39:28,473] jepsen worker 759 - jepsen.util 759 :ok :add 5500 INFO [2022-01-10 08:39:28,473] jepsen worker 297 - jepsen.util 297 :ok :add 5534 INFO [2022-01-10 08:39:28,473] jepsen worker 866 - jepsen.util 866 :ok :add 5518 INFO [2022-01-10 08:39:28,473] jepsen worker 55 - jepsen.util 55 :ok :add 5246 INFO [2022-01-10 08:39:28,473] jepsen worker 277 - jepsen.util 277 :ok :add 5490 INFO [2022-01-10 08:39:28,473] jepsen worker 577 - jepsen.util 577 :invoke :add 5611 INFO [2022-01-10 08:39:28,479] jepsen worker 19 - jepsen.util 19 :invoke :add 5612 INFO [2022-01-10 08:39:28,480] jepsen worker 784 - jepsen.util 784 :ok :add 5540 INFO [2022-01-10 08:39:28,480] jepsen worker 957 - jepsen.util 957 :ok :add 5396 INFO [2022-01-10 08:39:28,480] jepsen worker 596 - jepsen.util 596 :ok :add 5558 INFO [2022-01-10 08:39:28,480] jepsen worker 91 - jepsen.util 91 :ok :add 5574 INFO [2022-01-10 08:39:28,482] jepsen worker 15 - jepsen.util 15 :invoke :add 5613 INFO [2022-01-10 08:39:28,486] jepsen worker 722 - jepsen.util 722 :invoke :add 5614 INFO [2022-01-10 08:39:28,489] jepsen worker 192 - jepsen.util 192 :invoke :add 5615 INFO [2022-01-10 08:39:28,491] jepsen worker 869 - jepsen.util 869 :invoke :add 5616 INFO [2022-01-10 08:39:28,494] jepsen worker 733 - jepsen.util 733 :invoke :add 5617 INFO [2022-01-10 08:39:28,498] jepsen worker 875 - jepsen.util 875 :invoke :add 5618 INFO [2022-01-10 08:39:28,501] jepsen worker 260 - jepsen.util 260 :invoke :add 5619 INFO [2022-01-10 08:39:28,503] jepsen worker 185 - jepsen.util 185 :ok :add 5301 INFO [2022-01-10 08:39:28,503] jepsen worker 86 - jepsen.util 86 :ok :add 5447 INFO [2022-01-10 08:39:28,503] jepsen worker 851 - jepsen.util 851 :ok :add 5473 INFO [2022-01-10 08:39:28,503] jepsen worker 516 - jepsen.util 516 :ok :add 5335 INFO [2022-01-10 08:39:28,503] jepsen worker 815 - jepsen.util 815 :invoke :add 5620 INFO [2022-01-10 08:39:28,504] jepsen worker 901 - jepsen.util 901 :ok :add 5291 INFO [2022-01-10 08:39:28,504] jepsen worker 352 - jepsen.util 352 :ok :add 5289 INFO [2022-01-10 08:39:28,504] jepsen worker 544 - jepsen.util 544 :ok :add 5319 INFO [2022-01-10 08:39:28,508] jepsen worker 13 - jepsen.util 13 :invoke :add 5621 INFO [2022-01-10 08:39:28,511] jepsen worker 708 - jepsen.util 708 :invoke :add 5622 INFO [2022-01-10 08:39:28,512] jepsen worker 61 - jepsen.util 61 :ok :add 5342 INFO [2022-01-10 08:39:28,512] jepsen worker 35 - jepsen.util 35 :ok :add 5404 INFO [2022-01-10 08:39:28,513] jepsen worker 470 - jepsen.util 470 :ok :add 5376 INFO [2022-01-10 08:39:28,513] jepsen worker 914 - jepsen.util 914 :ok :add 5428 INFO [2022-01-10 08:39:28,513] jepsen worker 751 - jepsen.util 751 :ok :add 5430 INFO [2022-01-10 08:39:28,514] jepsen worker 94 - jepsen.util 94 :invoke :add 5623 INFO [2022-01-10 08:39:28,519] jepsen worker 879 - jepsen.util 879 :invoke :add 5624 INFO [2022-01-10 08:39:28,522] jepsen worker 127 - jepsen.util 127 :invoke :add 5625 INFO [2022-01-10 08:39:28,526] jepsen worker 23 - jepsen.util 23 :invoke :add 5626 INFO [2022-01-10 08:39:28,529] jepsen worker 310 - jepsen.util 310 :invoke :add 5627 INFO [2022-01-10 08:39:28,532] jepsen worker 655 - jepsen.util 655 :invoke :add 5628 INFO [2022-01-10 08:39:28,535] jepsen worker 882 - jepsen.util 882 :invoke :add 5629 INFO [2022-01-10 08:39:28,537] jepsen worker 343 - jepsen.util 343 :invoke :add 5630 INFO [2022-01-10 08:39:28,540] jepsen worker 610 - jepsen.util 610 :invoke :add 5631 INFO [2022-01-10 08:39:28,544] jepsen worker 9 - jepsen.util 9 :invoke :add 5632 INFO [2022-01-10 08:39:28,544] jepsen worker 567 - jepsen.util 567 :ok :add 5471 INFO [2022-01-10 08:39:28,544] jepsen worker 525 - jepsen.util 525 :ok :add 5293 INFO [2022-01-10 08:39:28,545] jepsen worker 306 - jepsen.util 306 :ok :add 5303 INFO [2022-01-10 08:39:28,545] jepsen worker 919 - jepsen.util 919 :ok :add 5469 INFO [2022-01-10 08:39:28,546] jepsen worker 991 - jepsen.util 991 :ok :add 5337 INFO [2022-01-10 08:39:28,546] jepsen worker 924 - jepsen.util 924 :ok :add 5445 INFO [2022-01-10 08:39:28,548] jepsen worker 185 - jepsen.util 185 :invoke :add 5633 INFO [2022-01-10 08:39:28,554] jepsen worker 14 - jepsen.util 14 :invoke :add 5634 INFO [2022-01-10 08:39:28,557] jepsen worker 153 - jepsen.util 153 :invoke :add 5635 INFO [2022-01-10 08:39:28,559] jepsen worker 217 - jepsen.util 217 :invoke :add 5636 INFO [2022-01-10 08:39:28,562] jepsen worker 252 - jepsen.util 252 :invoke :add 5637 INFO [2022-01-10 08:39:28,565] jepsen worker 89 - jepsen.util 89 :invoke :add 5638 INFO [2022-01-10 08:39:28,568] jepsen worker 761 - jepsen.util 761 :invoke :add 5639 INFO [2022-01-10 08:39:28,572] jepsen worker 941 - jepsen.util 941 :invoke :add 5640 INFO [2022-01-10 08:39:28,575] jepsen worker 61 - jepsen.util 61 :invoke :add 5641 INFO [2022-01-10 08:39:28,578] jepsen worker 26 - jepsen.util 26 :invoke :add 5642 INFO [2022-01-10 08:39:28,581] jepsen worker 824 - jepsen.util 824 :invoke :add 5643 INFO [2022-01-10 08:39:28,585] jepsen worker 520 - jepsen.util 520 :invoke :add 5644 INFO [2022-01-10 08:39:28,588] jepsen worker 612 - jepsen.util 612 :invoke :add 5645 INFO [2022-01-10 08:39:28,591] jepsen worker 944 - jepsen.util 944 :invoke :add 5646 INFO [2022-01-10 08:39:28,594] jepsen worker 568 - jepsen.util 568 :invoke :add 5647 INFO [2022-01-10 08:39:28,596] jepsen worker 508 - jepsen.util 508 :invoke :add 5648 INFO [2022-01-10 08:39:28,599] jepsen worker 106 - jepsen.util 106 :invoke :add 5649 INFO [2022-01-10 08:39:28,603] jepsen worker 226 - jepsen.util 226 :invoke :add 5650 INFO [2022-01-10 08:39:28,603] jepsen worker 29 - jepsen.util 29 :ok :add 5288 INFO [2022-01-10 08:39:28,603] jepsen worker 618 - jepsen.util 618 :ok :add 5334 INFO [2022-01-10 08:39:28,604] jepsen worker 366 - jepsen.util 366 :ok :add 5318 INFO [2022-01-10 08:39:28,604] jepsen worker 207 - jepsen.util 207 :ok :add 5290 INFO [2022-01-10 08:39:28,604] jepsen worker 382 - jepsen.util 382 :ok :add 5300 INFO [2022-01-10 08:39:28,604] jepsen worker 320 - jepsen.util 320 :ok :add 5472 INFO [2022-01-10 08:39:28,604] jepsen worker 388 - jepsen.util 388 :ok :add 5446 INFO [2022-01-10 08:39:28,606] jepsen worker 567 - jepsen.util 567 :invoke :add 5651 INFO [2022-01-10 08:39:28,612] jepsen worker 551 - jepsen.util 551 :invoke :add 5652 INFO [2022-01-10 08:39:28,614] jepsen worker 798 - jepsen.util 798 :invoke :add 5653 INFO [2022-01-10 08:39:28,617] jepsen worker 998 - jepsen.util 998 :invoke :add 5654 INFO [2022-01-10 08:39:28,620] jepsen worker 771 - jepsen.util 771 :invoke :add 5655 INFO [2022-01-10 08:39:28,623] jepsen worker 105 - jepsen.util 105 :invoke :add 5656 INFO [2022-01-10 08:39:28,626] jepsen worker 388 - jepsen.util 388 :invoke :add 5657 INFO [2022-01-10 08:39:28,629] jepsen worker 950 - jepsen.util 950 :invoke :add 5658 INFO [2022-01-10 08:39:28,631] jepsen worker 60 - jepsen.util 60 :invoke :add 5659 INFO [2022-01-10 08:39:28,634] jepsen worker 249 - jepsen.util 249 :invoke :add 5660 INFO [2022-01-10 08:39:28,637] jepsen worker 608 - jepsen.util 608 :invoke :add 5661 INFO [2022-01-10 08:39:28,640] jepsen worker 645 - jepsen.util 645 :invoke :add 5662 INFO [2022-01-10 08:39:28,644] jepsen worker 262 - jepsen.util 262 :invoke :add 5663 INFO [2022-01-10 08:39:28,647] jepsen worker 755 - jepsen.util 755 :invoke :add 5664 INFO [2022-01-10 08:39:28,650] jepsen worker 335 - jepsen.util 335 :ok :add 5297 INFO [2022-01-10 08:39:28,650] jepsen worker 457 - jepsen.util 457 :ok :add 5441 INFO [2022-01-10 08:39:28,650] jepsen worker 300 - jepsen.util 300 :ok :add 5475 INFO [2022-01-10 08:39:28,650] jepsen worker 483 - jepsen.util 483 :ok :add 5459 INFO [2022-01-10 08:39:28,650] jepsen worker 365 - jepsen.util 365 :ok :add 5307 INFO [2022-01-10 08:39:28,650] jepsen worker 911 - jepsen.util 911 :ok :add 5333 INFO [2022-01-10 08:39:28,650] jepsen worker 800 - jepsen.util 800 :invoke :add 5665 INFO [2022-01-10 08:39:28,652] jepsen worker 197 - jepsen.util 197 :ok :add 5294 INFO [2022-01-10 08:39:28,652] jepsen worker 719 - jepsen.util 719 :ok :add 5328 INFO [2022-01-10 08:39:28,652] jepsen worker 985 - jepsen.util 985 :ok :add 5304 INFO [2022-01-10 08:39:28,652] jepsen worker 821 - jepsen.util 821 :ok :add 5476 INFO [2022-01-10 08:39:28,652] jepsen worker 795 - jepsen.util 795 :ok :add 5442 INFO [2022-01-10 08:39:28,652] jepsen worker 154 - jepsen.util 154 :ok :add 5330 INFO [2022-01-10 08:39:28,658] jepsen worker 79 - jepsen.util 79 :invoke :add 5666 INFO [2022-01-10 08:39:28,658] jepsen worker 74 - jepsen.util 74 :ok :add 5468 INFO [2022-01-10 08:39:28,658] jepsen worker 167 - jepsen.util 167 :ok :add 5444 INFO [2022-01-10 08:39:28,658] jepsen worker 604 - jepsen.util 604 :ok :add 5292 INFO [2022-01-10 08:39:28,658] jepsen worker 485 - jepsen.util 485 :ok :add 5302 INFO [2022-01-10 08:39:28,659] jepsen worker 836 - jepsen.util 836 :ok :add 5336 INFO [2022-01-10 08:39:28,659] jepsen worker 383 - jepsen.util 383 :ok :add 5470 INFO [2022-01-10 08:39:28,661] jepsen worker 495 - jepsen.util 495 :invoke :add 5667 INFO [2022-01-10 08:39:28,666] jepsen worker 73 - jepsen.util 73 :invoke :add 5668 INFO [2022-01-10 08:39:28,669] jepsen worker 864 - jepsen.util 864 :invoke :add 5669 INFO [2022-01-10 08:39:28,672] jepsen worker 891 - jepsen.util 891 :invoke :add 5670 INFO [2022-01-10 08:39:28,675] jepsen worker 165 - jepsen.util 165 :invoke :add 5671 INFO [2022-01-10 08:39:28,678] jepsen worker 665 - jepsen.util 665 :invoke :add 5672 INFO [2022-01-10 08:39:28,678] jepsen worker 995 - jepsen.util 995 :ok :add 5391 INFO [2022-01-10 08:39:28,678] jepsen worker 253 - jepsen.util 253 :ok :add 5573 INFO [2022-01-10 08:39:28,678] jepsen worker 103 - jepsen.util 103 :ok :add 5389 INFO [2022-01-10 08:39:28,679] jepsen worker 900 - jepsen.util 900 :ok :add 5602 INFO [2022-01-10 08:39:28,679] jepsen worker 658 - jepsen.util 658 :ok :add 5547 INFO [2022-01-10 08:39:28,679] jepsen worker 217 - jepsen.util 217 :ok :add 5636 INFO [2022-01-10 08:39:28,680] jepsen worker 378 - jepsen.util 378 :ok :add 5493 INFO [2022-01-10 08:39:28,680] jepsen worker 711 - jepsen.util 711 :ok :add 5537 INFO [2022-01-10 08:39:28,680] jepsen worker 124 - jepsen.util 124 :ok :add 5503 INFO [2022-01-10 08:39:28,681] jepsen worker 410 - jepsen.util 410 :invoke :add 5673 INFO [2022-01-10 08:39:28,681] jepsen worker 944 - jepsen.util 944 :ok :add 5646 INFO [2022-01-10 08:39:28,682] jepsen worker 400 - jepsen.util 400 :ok :add 5577 INFO [2022-01-10 08:39:28,682] jepsen worker 9 - jepsen.util 9 :ok :add 5632 INFO [2022-01-10 08:39:28,682] jepsen worker 776 - jepsen.util 776 :ok :add 5395 INFO [2022-01-10 08:39:28,682] jepsen worker 561 - jepsen.util 561 :ok :add 5606 INFO [2022-01-10 08:39:28,682] jepsen worker 721 - jepsen.util 721 :ok :add 5543 INFO [2022-01-10 08:39:28,685] jepsen worker 63 - jepsen.util 63 :ok :add 5306 INFO [2022-01-10 08:39:28,685] jepsen worker 156 - jepsen.util 156 :ok :add 5474 INFO [2022-01-10 08:39:28,685] jepsen worker 532 - jepsen.util 532 :ok :add 5296 INFO [2022-01-10 08:39:28,686] jepsen worker 129 - jepsen.util 129 :ok :add 5458 INFO [2022-01-10 08:39:28,686] jepsen worker 589 - jepsen.util 589 :ok :add 5440 INFO [2022-01-10 08:39:28,685] jepsen worker 790 - jepsen.util 790 :ok :add 5332 INFO [2022-01-10 08:39:28,692] jepsen worker 818 - jepsen.util 818 :invoke :add 5674 INFO [2022-01-10 08:39:28,695] jepsen worker 925 - jepsen.util 925 :invoke :add 5675 INFO [2022-01-10 08:39:28,698] jepsen worker 626 - jepsen.util 626 :invoke :add 5676 INFO [2022-01-10 08:39:28,701] jepsen worker 501 - jepsen.util 501 :invoke :add 5677 INFO [2022-01-10 08:39:28,704] jepsen worker 132 - jepsen.util 132 :invoke :add 5678 INFO [2022-01-10 08:39:28,707] jepsen worker 285 - jepsen.util 285 :invoke :add 5679 INFO [2022-01-10 08:39:28,709] jepsen worker 769 - jepsen.util 769 :invoke :add 5680 INFO [2022-01-10 08:39:28,712] jepsen worker 550 - jepsen.util 550 :invoke :add 5681 INFO [2022-01-10 08:39:28,715] jepsen worker 595 - jepsen.util 595 :invoke :add 5682 INFO [2022-01-10 08:39:28,720] jepsen worker 991 - jepsen.util 991 :invoke :add 5684 INFO [2022-01-10 08:39:28,721] jepsen worker 55 - jepsen.util 55 :invoke :add 5683 INFO [2022-01-10 08:39:28,724] jepsen worker 234 - jepsen.util 234 :invoke :add 5685 INFO [2022-01-10 08:39:28,726] jepsen worker 638 - jepsen.util 638 :invoke :add 5686 INFO [2022-01-10 08:39:28,729] jepsen worker 692 - jepsen.util 692 :invoke :add 5687 INFO [2022-01-10 08:39:28,732] jepsen worker 630 - jepsen.util 630 :invoke :add 5688 INFO [2022-01-10 08:39:28,735] jepsen worker 352 - jepsen.util 352 :invoke :add 5689 INFO [2022-01-10 08:39:28,740] jepsen worker 604 - jepsen.util 604 :invoke :add 5690 INFO [2022-01-10 08:39:28,742] jepsen worker 363 - jepsen.util 363 :invoke :add 5691 INFO [2022-01-10 08:39:28,745] jepsen worker 214 - jepsen.util 214 :invoke :add 5692 INFO [2022-01-10 08:39:28,749] jepsen worker 167 - jepsen.util 167 :invoke :add 5693 INFO [2022-01-10 08:39:28,752] jepsen worker 459 - jepsen.util 459 :invoke :add 5694 INFO [2022-01-10 08:39:28,755] jepsen worker 702 - jepsen.util 702 :invoke :add 5695 INFO [2022-01-10 08:39:28,759] jepsen worker 778 - jepsen.util 778 :invoke :add 5696 INFO [2022-01-10 08:39:28,763] jepsen worker 246 - jepsen.util 246 :invoke :add 5697 INFO [2022-01-10 08:39:28,765] jepsen worker 380 - jepsen.util 380 :invoke :add 5698 INFO [2022-01-10 08:39:28,768] jepsen worker 335 - jepsen.util 335 :invoke :add 5699 INFO [2022-01-10 08:39:28,771] jepsen worker 976 - jepsen.util 976 :invoke :add 5700 INFO [2022-01-10 08:39:28,774] jepsen worker 24 - jepsen.util 24 :invoke :add 5701 INFO [2022-01-10 08:39:28,777] jepsen worker 653 - jepsen.util 653 :invoke :add 5702 INFO [2022-01-10 08:39:28,780] jepsen worker 162 - jepsen.util 162 :invoke :add 5703 INFO [2022-01-10 08:39:28,783] jepsen worker 614 - jepsen.util 614 :invoke :add 5704 INFO [2022-01-10 08:39:28,786] jepsen worker 361 - jepsen.util 361 :invoke :add 5705 INFO [2022-01-10 08:39:28,789] jepsen worker 39 - jepsen.util 39 :invoke :add 5706 INFO [2022-01-10 08:39:28,792] jepsen worker 749 - jepsen.util 749 :invoke :add 5707 INFO [2022-01-10 08:39:28,795] jepsen worker 169 - jepsen.util 169 :invoke :add 5708 INFO [2022-01-10 08:39:28,798] jepsen worker 320 - jepsen.util 320 :invoke :add 5709 INFO [2022-01-10 08:39:28,801] jepsen worker 946 - jepsen.util 946 :invoke :add 5710 INFO [2022-01-10 08:39:28,803] jepsen worker 497 - jepsen.util 497 :invoke :add 5711 INFO [2022-01-10 08:39:28,807] jepsen worker 427 - jepsen.util 427 :invoke :add 5712 INFO [2022-01-10 08:39:28,809] jepsen worker 586 - jepsen.util 586 :invoke :add 5713 INFO [2022-01-10 08:39:28,812] jepsen worker 861 - jepsen.util 861 :invoke :add 5714 INFO [2022-01-10 08:39:28,815] jepsen worker 140 - jepsen.util 140 :invoke :add 5715 INFO [2022-01-10 08:39:28,818] jepsen worker 395 - jepsen.util 395 :invoke :add 5716 INFO [2022-01-10 08:39:28,821] jepsen worker 59 - jepsen.util 59 :invoke :add 5717 INFO [2022-01-10 08:39:28,823] jepsen worker 242 - jepsen.util 242 :invoke :add 5718 INFO [2022-01-10 08:39:28,826] jepsen worker 50 - jepsen.util 50 :invoke :add 5719 INFO [2022-01-10 08:39:28,829] jepsen worker 661 - jepsen.util 661 :invoke :add 5720 INFO [2022-01-10 08:39:28,833] jepsen worker 633 - jepsen.util 633 :invoke :add 5721 INFO [2022-01-10 08:39:28,836] jepsen worker 400 - jepsen.util 400 :invoke :add 5722 INFO [2022-01-10 08:39:28,839] jepsen worker 289 - jepsen.util 289 :invoke :add 5723 INFO [2022-01-10 08:39:28,841] jepsen worker 547 - jepsen.util 547 :invoke :add 5724 INFO [2022-01-10 08:39:28,844] jepsen worker 280 - jepsen.util 280 :invoke :add 5725 INFO [2022-01-10 08:39:28,848] jepsen worker 201 - jepsen.util 201 :invoke :add 5726 INFO [2022-01-10 08:39:28,850] jepsen worker 953 - jepsen.util 953 :invoke :add 5727 INFO [2022-01-10 08:39:28,854] jepsen worker 254 - jepsen.util 254 :invoke :add 5728 INFO [2022-01-10 08:39:28,856] jepsen worker 867 - jepsen.util 867 :invoke :add 5729 INFO [2022-01-10 08:39:28,860] jepsen worker 414 - jepsen.util 414 :invoke :add 5730 INFO [2022-01-10 08:39:28,863] jepsen worker 488 - jepsen.util 488 :ok :add 5489 INFO [2022-01-10 08:39:28,863] jepsen worker 814 - jepsen.util 814 :ok :add 5491 INFO [2022-01-10 08:39:28,863] jepsen worker 425 - jepsen.util 425 :ok :add 5501 INFO [2022-01-10 08:39:28,863] jepsen worker 237 - jepsen.util 237 :ok :add 5535 INFO [2022-01-10 08:39:28,863] jepsen worker 520 - jepsen.util 520 :ok :add 5644 INFO [2022-01-10 08:39:28,863] jepsen worker 565 - jepsen.util 565 :ok :add 5519 INFO [2022-01-10 08:39:28,863] jepsen worker 563 - jepsen.util 563 :invoke :add 5731 INFO [2022-01-10 08:39:28,863] jepsen worker 73 - jepsen.util 73 :ok :add 5668 INFO [2022-01-10 08:39:28,863] jepsen worker 891 - jepsen.util 891 :ok :add 5670 INFO [2022-01-10 08:39:28,869] jepsen worker 639 - jepsen.util 639 :invoke :add 5732 INFO [2022-01-10 08:39:28,872] jepsen worker 952 - jepsen.util 952 :invoke :add 5733 INFO [2022-01-10 08:39:28,875] jepsen worker 517 - jepsen.util 517 :invoke :add 5734 INFO [2022-01-10 08:39:28,877] jepsen worker 407 - jepsen.util 407 :ok :add 5545 INFO [2022-01-10 08:39:28,878] jepsen worker 14 - jepsen.util 14 :ok :add 5634 INFO [2022-01-10 08:39:28,878] jepsen worker 804 - jepsen.util 804 :ok :add 5569 INFO [2022-01-10 08:39:28,878] jepsen worker 223 - jepsen.util 223 :ok :add 5571 INFO [2022-01-10 08:39:28,878] jepsen worker 453 - jepsen.util 453 :ok :add 5600 INFO [2022-01-10 08:39:28,878] jepsen worker 875 - jepsen.util 875 :ok :add 5618 INFO [2022-01-10 08:39:28,878] jepsen worker 782 - jepsen.util 782 :invoke :add 5735 INFO [2022-01-10 08:39:28,882] jepsen worker 207 - jepsen.util 207 :invoke :add 5736 INFO [2022-01-10 08:39:28,882] jepsen worker 605 - jepsen.util 605 :ok :add 5546 INFO [2022-01-10 08:39:28,883] jepsen worker 570 - jepsen.util 570 :ok :add 5603 INFO [2022-01-10 08:39:28,883] jepsen worker 252 - jepsen.util 252 :ok :add 5637 INFO [2022-01-10 08:39:28,883] jepsen worker 617 - jepsen.util 617 :ok :add 5572 INFO [2022-01-10 08:39:28,884] jepsen worker 662 - jepsen.util 662 :ok :add 4503 INFO [2022-01-10 08:39:28,884] jepsen worker 390 - jepsen.util 390 :ok :add 4672 INFO [2022-01-10 08:39:28,885] jepsen worker 314 - jepsen.util 314 :ok :add 4537 INFO [2022-01-10 08:39:28,885] jepsen worker 737 - jepsen.util 737 :ok :add 4493 INFO [2022-01-10 08:39:28,885] jepsen worker 428 - jepsen.util 428 :ok :add 4646 INFO [2022-01-10 08:39:28,889] jepsen worker 263 - jepsen.util 263 :invoke :add 5737 INFO [2022-01-10 08:39:28,892] jepsen worker 491 - jepsen.util 491 :invoke :add 5738 INFO [2022-01-10 08:39:28,895] jepsen worker 809 - jepsen.util 809 :invoke :add 5739 INFO [2022-01-10 08:39:28,897] jepsen worker 102 - jepsen.util 102 :invoke :add 5740 INFO [2022-01-10 08:39:28,900] jepsen worker 516 - jepsen.util 516 :invoke :add 5741 INFO [2022-01-10 08:39:28,903] jepsen worker 177 - jepsen.util 177 :invoke :add 5742 INFO [2022-01-10 08:39:28,906] jepsen worker 664 - jepsen.util 664 :invoke :add 5743 INFO [2022-01-10 08:39:28,909] jepsen worker 117 - jepsen.util 117 :invoke :add 5744 INFO [2022-01-10 08:39:28,912] jepsen worker 908 - jepsen.util 908 :invoke :add 5745 INFO [2022-01-10 08:39:28,914] jepsen worker 758 - jepsen.util 758 :invoke :add 5746 INFO [2022-01-10 08:39:28,918] jepsen worker 783 - jepsen.util 783 :invoke :add 5747 INFO [2022-01-10 08:39:28,921] jepsen worker 91 - jepsen.util 91 :invoke :add 5748 INFO [2022-01-10 08:39:28,921] jepsen worker 179 - jepsen.util 179 :ok :add 5397 INFO [2022-01-10 08:39:28,922] jepsen worker 172 - jepsen.util 172 :ok :add 5559 INFO [2022-01-10 08:39:28,922] jepsen worker 92 - jepsen.util 92 :ok :add 5541 INFO [2022-01-10 08:39:28,922] jepsen worker 906 - jepsen.util 906 :ok :add 5604 INFO [2022-01-10 08:39:28,922] jepsen worker 269 - jepsen.util 269 :ok :add 5575 INFO [2022-01-10 08:39:28,922] jepsen worker 655 - jepsen.util 655 :ok :add 5628 INFO [2022-01-10 08:39:28,922] jepsen worker 343 - jepsen.util 343 :ok :add 5630 INFO [2022-01-10 08:39:28,924] jepsen worker 449 - jepsen.util 449 :invoke :add 5749 INFO [2022-01-10 08:39:28,929] jepsen worker 616 - jepsen.util 616 :invoke :add 5750 INFO [2022-01-10 08:39:28,932] jepsen worker 756 - jepsen.util 756 :invoke :add 5751 INFO [2022-01-10 08:39:28,935] jepsen worker 261 - jepsen.util 261 :invoke :add 5752 INFO [2022-01-10 08:39:28,937] jepsen worker 448 - jepsen.util 448 :invoke :add 5753 INFO [2022-01-10 08:39:28,941] jepsen worker 30 - jepsen.util 30 :invoke :add 5754 INFO [2022-01-10 08:39:28,943] jepsen worker 774 - jepsen.util 774 :invoke :add 5755 INFO [2022-01-10 08:39:28,946] jepsen worker 44 - jepsen.util 44 :invoke :add 5756 INFO [2022-01-10 08:39:28,949] jepsen worker 686 - jepsen.util 686 :invoke :add 5757 INFO [2022-01-10 08:39:28,952] jepsen worker 726 - jepsen.util 726 :invoke :add 5758 INFO [2022-01-10 08:39:28,955] jepsen worker 124 - jepsen.util 124 :invoke :add 5759 INFO [2022-01-10 08:39:28,958] jepsen worker 866 - jepsen.util 866 :invoke :add 5760 INFO [2022-01-10 08:39:28,962] jepsen worker 564 - jepsen.util 564 :ok :add 5570 INFO [2022-01-10 08:39:28,962] jepsen worker 232 - jepsen.util 232 :ok :add 5544 INFO [2022-01-10 08:39:28,963] jepsen worker 357 - jepsen.util 357 :ok :add 5568 INFO [2022-01-10 08:39:28,962] jepsen worker 153 - jepsen.util 153 :ok :add 5635 INFO [2022-01-10 08:39:28,963] jepsen worker 260 - jepsen.util 260 :ok :add 5619 INFO [2022-01-10 08:39:28,963] jepsen worker 956 - jepsen.util 956 :ok :add 5601 INFO [2022-01-10 08:39:28,964] jepsen worker 314 - jepsen.util 314 :invoke :add 5761 INFO [2022-01-10 08:39:28,970] jepsen worker 560 - jepsen.util 560 :invoke :add 5762 INFO [2022-01-10 08:39:28,972] jepsen worker 783 - jepsen.util 783 :ok :add 5747 INFO [2022-01-10 08:39:28,972] jepsen worker 423 - jepsen.util 423 :ok :add 5592 INFO [2022-01-10 08:39:28,975] jepsen worker 258 - jepsen.util 258 :invoke :add 5763 INFO [2022-01-10 08:39:28,980] jepsen worker 779 - jepsen.util 779 :invoke :add 5764 INFO [2022-01-10 08:39:28,980] jepsen worker 239 - jepsen.util 239 :ok :add 5593 INFO [2022-01-10 08:39:28,981] jepsen worker 758 - jepsen.util 758 :ok :add 5746 INFO [2022-01-10 08:39:28,981] jepsen worker 670 - jepsen.util 670 :ok :add 5588 INFO [2022-01-10 08:39:28,981] jepsen worker 974 - jepsen.util 974 :ok :add 5590 INFO [2022-01-10 08:39:28,981] jepsen worker 908 - jepsen.util 908 :ok :add 5745 INFO [2022-01-10 08:39:28,987] jepsen worker 245 - jepsen.util 245 :invoke :add 5765 INFO [2022-01-10 08:39:28,989] jepsen worker 240 - jepsen.util 240 :ok :add 5341 INFO [2022-01-10 08:39:28,989] jepsen worker 471 - jepsen.util 471 :ok :add 5375 INFO [2022-01-10 08:39:28,989] jepsen worker 703 - jepsen.util 703 :ok :add 5433 INFO [2022-01-10 08:39:28,989] jepsen worker 107 - jepsen.util 107 :ok :add 5359 INFO [2022-01-10 08:39:28,989] jepsen worker 843 - jepsen.util 843 :ok :add 5597 INFO [2022-01-10 08:39:28,989] jepsen worker 47 - jepsen.util 47 :ok :add 5407 INFO [2022-01-10 08:39:28,993] jepsen worker 384 - jepsen.util 384 :invoke :add 5766 INFO [2022-01-10 08:39:29,001] jepsen worker 662 - jepsen.util 662 :invoke :add 5767 INFO [2022-01-10 08:39:29,005] jepsen worker 689 - jepsen.util 689 :invoke :add 5768 INFO [2022-01-10 08:39:29,009] jepsen worker 20 - jepsen.util 20 :invoke :add 5769 INFO [2022-01-10 08:39:29,014] jepsen worker 589 - jepsen.util 589 :invoke :add 5770 INFO [2022-01-10 08:39:29,018] jepsen worker 222 - jepsen.util 222 :invoke :add 5771 INFO [2022-01-10 08:39:29,023] jepsen worker 542 - jepsen.util 542 :invoke :add 5772 INFO [2022-01-10 08:39:29,027] jepsen worker 334 - jepsen.util 334 :invoke :add 5773 INFO [2022-01-10 08:39:29,032] jepsen worker 655 - jepsen.util 655 :invoke :add 5774 INFO [2022-01-10 08:39:29,036] jepsen worker 606 - jepsen.util 606 :invoke :add 5775 INFO [2022-01-10 08:39:29,041] jepsen worker 118 - jepsen.util 118 :invoke :add 5776 INFO [2022-01-10 08:39:29,045] jepsen worker 796 - jepsen.util 796 :invoke :add 5777 INFO [2022-01-10 08:39:29,050] jepsen worker 270 - jepsen.util 270 :invoke :add 5778 INFO [2022-01-10 08:39:29,054] jepsen worker 522 - jepsen.util 522 :invoke :add 5779 INFO [2022-01-10 08:39:29,057] jepsen worker 899 - jepsen.util 899 :invoke :add 5780 INFO [2022-01-10 08:39:29,060] jepsen worker 714 - jepsen.util 714 :invoke :add 5781 INFO [2022-01-10 08:39:29,063] jepsen worker 432 - jepsen.util 432 :invoke :add 5782 INFO [2022-01-10 08:39:29,066] jepsen worker 381 - jepsen.util 381 :invoke :add 5783 INFO [2022-01-10 08:39:29,068] jepsen worker 670 - jepsen.util 670 :invoke :add 5784 INFO [2022-01-10 08:39:29,072] jepsen worker 578 - jepsen.util 578 :invoke :add 5785 INFO [2022-01-10 08:39:29,074] jepsen worker 240 - jepsen.util 240 :invoke :add 5786 INFO [2022-01-10 08:39:29,077] jepsen worker 564 - jepsen.util 564 :invoke :add 5787 INFO [2022-01-10 08:39:29,080] jepsen worker 75 - jepsen.util 75 :invoke :add 5788 INFO [2022-01-10 08:39:29,084] jepsen worker 78 - jepsen.util 78 :invoke :add 5789 INFO [2022-01-10 08:39:29,087] jepsen worker 933 - jepsen.util 933 :invoke :add 5790 INFO [2022-01-10 08:39:29,090] jepsen worker 46 - jepsen.util 46 :invoke :add 5791 INFO [2022-01-10 08:39:29,093] jepsen worker 721 - jepsen.util 721 :invoke :add 5792 INFO [2022-01-10 08:39:29,096] jepsen worker 158 - jepsen.util 158 :invoke :add 5793 INFO [2022-01-10 08:39:29,099] jepsen worker 47 - jepsen.util 47 :invoke :add 5794 INFO [2022-01-10 08:39:29,103] jepsen worker 420 - jepsen.util 420 :invoke :add 5795 INFO [2022-01-10 08:39:29,108] jepsen worker 695 - jepsen.util 695 :invoke :add 5796 INFO [2022-01-10 08:39:29,191] jepsen worker 644 - jepsen.util 644 :invoke :add 5797 INFO [2022-01-10 08:39:29,192] jepsen worker 568 - jepsen.util 568 :ok :add 5647 INFO [2022-01-10 08:39:29,192] jepsen worker 117 - jepsen.util 117 :ok :add 5744 INFO [2022-01-10 08:39:29,192] jepsen worker 599 - jepsen.util 599 :ok :add 5591 INFO [2022-01-10 08:39:29,192] jepsen worker 410 - jepsen.util 410 :ok :add 5673 INFO [2022-01-10 08:39:29,195] jepsen worker 729 - jepsen.util 729 :invoke :add 5798 INFO [2022-01-10 08:39:29,197] jepsen worker 139 - jepsen.util 139 :invoke :add 5799 INFO [2022-01-10 08:39:29,198] jepsen worker 221 - jepsen.util 221 :invoke :add 5800 INFO [2022-01-10 08:39:29,200] jepsen worker 536 - jepsen.util 536 :invoke :add 5801 INFO [2022-01-10 08:39:29,202] jepsen worker 919 - jepsen.util 919 :invoke :add 5802 INFO [2022-01-10 08:39:29,204] jepsen worker 928 - jepsen.util 928 :invoke :add 5803 INFO [2022-01-10 08:39:29,206] jepsen worker 704 - jepsen.util 704 :invoke :add 5804 INFO [2022-01-10 08:39:29,208] jepsen worker 10 - jepsen.util 10 :invoke :add 5805 INFO [2022-01-10 08:39:29,210] jepsen worker 993 - jepsen.util 993 :invoke :add 5806 INFO [2022-01-10 08:39:29,213] jepsen worker 297 - jepsen.util 297 :invoke :add 5807 INFO [2022-01-10 08:39:29,216] jepsen worker 781 - jepsen.util 781 :invoke :add 5808 INFO [2022-01-10 08:39:29,218] jepsen worker 966 - jepsen.util 966 :invoke :add 5809 INFO [2022-01-10 08:39:29,220] jepsen worker 401 - jepsen.util 401 :invoke :add 5810 INFO [2022-01-10 08:39:29,222] jepsen worker 183 - jepsen.util 183 :invoke :add 5811 INFO [2022-01-10 08:39:29,224] jepsen worker 759 - jepsen.util 759 :invoke :add 5812 INFO [2022-01-10 08:39:29,226] jepsen worker 206 - jepsen.util 206 :invoke :add 5813 INFO [2022-01-10 08:39:29,228] jepsen worker 248 - jepsen.util 248 :invoke :add 5814 INFO [2022-01-10 08:39:29,230] jepsen worker 92 - jepsen.util 92 :invoke :add 5815 INFO [2022-01-10 08:39:29,232] jepsen worker 643 - jepsen.util 643 :invoke :add 5816 INFO [2022-01-10 08:39:29,234] jepsen worker 52 - jepsen.util 52 :invoke :add 5817 INFO [2022-01-10 08:39:29,236] jepsen worker 957 - jepsen.util 957 :invoke :add 5818 INFO [2022-01-10 08:39:29,237] jepsen worker 348 - jepsen.util 348 :invoke :add 5819 INFO [2022-01-10 08:39:29,239] jepsen worker 203 - jepsen.util 203 :invoke :add 5820 INFO [2022-01-10 08:39:29,241] jepsen worker 514 - jepsen.util 514 :invoke :add 5821 INFO [2022-01-10 08:39:29,242] jepsen worker 154 - jepsen.util 154 :invoke :add 5822 INFO [2022-01-10 08:39:29,244] jepsen worker 453 - jepsen.util 453 :invoke :add 5823 INFO [2022-01-10 08:39:29,246] jepsen worker 477 - jepsen.util 477 :invoke :add 5824 INFO [2022-01-10 08:39:29,248] jepsen worker 390 - jepsen.util 390 :invoke :add 5825 INFO [2022-01-10 08:39:29,251] jepsen worker 808 - jepsen.util 808 :invoke :add 5826 INFO [2022-01-10 08:39:29,252] jepsen worker 462 - jepsen.util 462 :invoke :add 5827 INFO [2022-01-10 08:39:29,254] jepsen worker 80 - jepsen.util 80 :invoke :add 5828 INFO [2022-01-10 08:39:29,256] jepsen worker 871 - jepsen.util 871 :invoke :add 5829 INFO [2022-01-10 08:39:29,258] jepsen worker 843 - jepsen.util 843 :invoke :add 5830 INFO [2022-01-10 08:39:29,260] jepsen worker 451 - jepsen.util 451 :invoke :add 5831 INFO [2022-01-10 08:39:29,262] jepsen worker 984 - jepsen.util 984 :invoke :add 5832 INFO [2022-01-10 08:39:29,264] jepsen worker 648 - jepsen.util 648 :invoke :add 5833 INFO [2022-01-10 08:39:29,266] jepsen worker 922 - jepsen.util 922 :invoke :add 5834 INFO [2022-01-10 08:39:29,268] jepsen worker 413 - jepsen.util 413 :invoke :add 5835 INFO [2022-01-10 08:39:29,270] jepsen worker 305 - jepsen.util 305 :invoke :add 5836 INFO [2022-01-10 08:39:29,271] jepsen worker 312 - jepsen.util 312 :invoke :add 5837 INFO [2022-01-10 08:39:29,273] jepsen worker 687 - jepsen.util 687 :invoke :add 5838 INFO [2022-01-10 08:39:29,275] jepsen worker 63 - jepsen.util 63 :invoke :add 5839 INFO [2022-01-10 08:39:29,276] jepsen worker 341 - jepsen.util 341 :invoke :add 5840 INFO [2022-01-10 08:39:29,279] jepsen worker 97 - jepsen.util 97 :invoke :add 5841 INFO [2022-01-10 08:39:29,280] jepsen worker 208 - jepsen.util 208 :invoke :add 5842 INFO [2022-01-10 08:39:29,282] jepsen worker 292 - jepsen.util 292 :invoke :add 5843 INFO [2022-01-10 08:39:29,284] jepsen worker 128 - jepsen.util 128 :invoke :add 5844 INFO [2022-01-10 08:39:29,285] jepsen worker 287 - jepsen.util 287 :invoke :add 5845 INFO [2022-01-10 08:39:29,287] jepsen worker 620 - jepsen.util 620 :invoke :add 5846 INFO [2022-01-10 08:39:29,289] jepsen worker 786 - jepsen.util 786 :invoke :add 5847 INFO [2022-01-10 08:39:29,290] jepsen worker 851 - jepsen.util 851 :invoke :add 5848 INFO [2022-01-10 08:39:29,292] jepsen worker 525 - jepsen.util 525 :invoke :add 5849 INFO [2022-01-10 08:39:29,294] jepsen worker 828 - jepsen.util 828 :invoke :add 5850 INFO [2022-01-10 08:39:29,296] jepsen worker 532 - jepsen.util 532 :invoke :add 5851 INFO [2022-01-10 08:39:29,298] jepsen worker 852 - jepsen.util 852 :invoke :add 5852 INFO [2022-01-10 08:39:29,300] jepsen worker 841 - jepsen.util 841 :invoke :add 5853 INFO [2022-01-10 08:39:29,302] jepsen worker 9 - jepsen.util 9 :invoke :add 5854 INFO [2022-01-10 08:39:29,304] jepsen worker 654 - jepsen.util 654 :invoke :add 5855 INFO [2022-01-10 08:39:29,306] jepsen worker 325 - jepsen.util 325 :invoke :add 5856 INFO [2022-01-10 08:39:29,307] jepsen worker 988 - jepsen.util 988 :invoke :add 5857 INFO [2022-01-10 08:39:29,309] jepsen worker 528 - jepsen.util 528 :invoke :add 5858 INFO [2022-01-10 08:39:29,311] jepsen worker 911 - jepsen.util 911 :invoke :add 5859 INFO [2022-01-10 08:39:29,313] jepsen worker 471 - jepsen.util 471 :invoke :add 5860 INFO [2022-01-10 08:39:29,315] jepsen worker 811 - jepsen.util 811 :invoke :add 5861 INFO [2022-01-10 08:39:29,318] jepsen worker 313 - jepsen.util 313 :invoke :add 5862 INFO [2022-01-10 08:39:29,319] jepsen worker 288 - jepsen.util 288 :invoke :add 5863 INFO [2022-01-10 08:39:29,321] jepsen worker 117 - jepsen.util 117 :invoke :add 5864 INFO [2022-01-10 08:39:29,323] jepsen worker 880 - jepsen.util 880 :invoke :add 5865 INFO [2022-01-10 08:39:29,326] jepsen worker 948 - jepsen.util 948 :invoke :add 5866 INFO [2022-01-10 08:39:29,328] jepsen worker 318 - jepsen.util 318 :invoke :add 5867 INFO [2022-01-10 08:39:29,329] jepsen worker 431 - jepsen.util 431 :ok :add 5497 INFO [2022-01-10 08:39:29,329] jepsen worker 316 - jepsen.util 316 :ok :add 5507 INFO [2022-01-10 08:39:29,330] jepsen worker 626 - jepsen.util 626 :ok :add 5676 INFO [2022-01-10 08:39:29,330] jepsen worker 26 - jepsen.util 26 :ok :add 5642 INFO [2022-01-10 08:39:29,331] jepsen worker 150 - jepsen.util 150 :invoke :add 5868 INFO [2022-01-10 08:39:29,331] jepsen worker 120 - jepsen.util 120 :ok :add 5533 INFO [2022-01-10 08:39:29,334] jepsen worker 425 - jepsen.util 425 :invoke :add 5869 INFO [2022-01-10 08:39:29,337] jepsen worker 200 - jepsen.util 200 :invoke :add 5870 INFO [2022-01-10 08:39:29,339] jepsen worker 179 - jepsen.util 179 :invoke :add 5871 INFO [2022-01-10 08:39:29,341] jepsen worker 825 - jepsen.util 825 :ok :add 4885 INFO [2022-01-10 08:39:29,342] jepsen worker 898 - jepsen.util 898 :ok :add 5582 INFO [2022-01-10 08:39:29,342] jepsen worker 496 - jepsen.util 496 :ok :add 4915 INFO [2022-01-10 08:39:29,342] jepsen worker 115 - jepsen.util 115 :ok :add 4921 INFO [2022-01-10 08:39:29,342] jepsen worker 587 - jepsen.util 587 :ok :add 4794 INFO [2022-01-10 08:39:29,342] jepsen worker 364 - jepsen.util 364 :ok :add 4939 INFO [2022-01-10 08:39:29,342] jepsen worker 672 - jepsen.util 672 :ok :add 5011 INFO [2022-01-10 08:39:29,342] jepsen worker 377 - jepsen.util 377 :ok :add 5360 INFO [2022-01-10 08:39:29,343] jepsen worker 253 - jepsen.util 253 :invoke :add 5872 INFO [2022-01-10 08:39:29,345] jepsen worker 412 - jepsen.util 412 :invoke :add 5873 INFO [2022-01-10 08:39:29,346] jepsen worker 323 - jepsen.util 323 :ok :add 4597 INFO [2022-01-10 08:39:29,346] jepsen worker 989 - jepsen.util 989 :ok :add 4853 INFO [2022-01-10 08:39:29,347] jepsen worker 583 - jepsen.util 583 :ok :add 4742 INFO [2022-01-10 08:39:29,347] jepsen worker 146 - jepsen.util 146 :ok :add 4776 INFO [2022-01-10 08:39:29,347] jepsen worker 725 - jepsen.util 725 :ok :add 5609 INFO [2022-01-10 08:39:29,347] jepsen worker 127 - jepsen.util 127 :ok :add 5625 INFO [2022-01-10 08:39:29,348] jepsen worker 883 - jepsen.util 883 :invoke :add 5874 INFO [2022-01-10 08:39:29,348] jepsen worker 590 - jepsen.util 590 :ok :add 4867 INFO [2022-01-10 08:39:29,349] jepsen worker 723 - jepsen.util 723 :ok :add 5212 INFO [2022-01-10 08:39:29,349] jepsen worker 893 - jepsen.util 893 :ok :add 5226 INFO [2022-01-10 08:39:29,349] jepsen worker 198 - jepsen.util 198 :ok :add 5560 INFO [2022-01-10 08:39:29,352] jepsen worker 612 - jepsen.util 612 :ok :add 5645 INFO [2022-01-10 08:39:29,351] jepsen worker 175 - jepsen.util 175 :invoke :add 5875 INFO [2022-01-10 08:39:29,352] jepsen worker 165 - jepsen.util 165 :ok :add 5671 INFO [2022-01-10 08:39:29,353] jepsen worker 98 - jepsen.util 98 :ok :add 5382 INFO [2022-01-10 08:39:29,353] jepsen worker 864 - jepsen.util 864 :ok :add 5669 INFO [2022-01-10 08:39:29,355] jepsen worker 431 - jepsen.util 431 :invoke :add 5876 INFO [2022-01-10 08:39:29,359] jepsen worker 583 - jepsen.util 583 :invoke :add 5877 INFO [2022-01-10 08:39:29,362] jepsen worker 364 - jepsen.util 364 :invoke :add 5878 INFO [2022-01-10 08:39:29,365] jepsen worker 116 - jepsen.util 116 :invoke :add 5879 INFO [2022-01-10 08:39:29,368] jepsen worker 787 - jepsen.util 787 :invoke :add 5880 INFO [2022-01-10 08:39:29,372] jepsen worker 66 - jepsen.util 66 :invoke :add 5881 INFO [2022-01-10 08:39:29,374] jepsen worker 970 - jepsen.util 970 :invoke :add 5882 INFO [2022-01-10 08:39:29,376] jepsen worker 32 - jepsen.util 32 :invoke :add 5883 INFO [2022-01-10 08:39:29,379] jepsen worker 255 - jepsen.util 255 :invoke :add 5884 INFO [2022-01-10 08:39:29,381] jepsen worker 902 - jepsen.util 902 :invoke :add 5885 INFO [2022-01-10 08:39:29,382] jepsen worker 862 - jepsen.util 862 :invoke :add 5886 INFO [2022-01-10 08:39:29,385] jepsen worker 488 - jepsen.util 488 :invoke :add 5887 INFO [2022-01-10 08:39:29,387] jepsen worker 875 - jepsen.util 875 :invoke :add 5888 INFO [2022-01-10 08:39:29,389] jepsen worker 874 - jepsen.util 874 :invoke :add 5889 INFO [2022-01-10 08:39:29,391] jepsen worker 720 - jepsen.util 720 :invoke :add 5890 INFO [2022-01-10 08:39:29,394] jepsen worker 14 - jepsen.util 14 :invoke :add 5891 INFO [2022-01-10 08:39:29,396] jepsen worker 837 - jepsen.util 837 :invoke :add 5892 INFO [2022-01-10 08:39:29,398] jepsen worker 962 - jepsen.util 962 :invoke :add 5893 INFO [2022-01-10 08:39:29,400] jepsen worker 520 - jepsen.util 520 :invoke :add 5894 INFO [2022-01-10 08:39:29,402] jepsen worker 473 - jepsen.util 473 :invoke :add 5895 INFO [2022-01-10 08:39:29,404] jepsen worker 485 - jepsen.util 485 :invoke :add 5896 INFO [2022-01-10 08:39:29,406] jepsen worker 612 - jepsen.util 612 :invoke :add 5897 INFO [2022-01-10 08:39:29,408] jepsen worker 989 - jepsen.util 989 :invoke :add 5898 INFO [2022-01-10 08:39:29,410] jepsen worker 831 - jepsen.util 831 :invoke :add 5899 INFO [2022-01-10 08:39:29,414] jepsen worker 677 - jepsen.util 677 :invoke :add 5900 INFO [2022-01-10 08:39:29,416] jepsen worker 65 - jepsen.util 65 :invoke :add 5901 INFO [2022-01-10 08:39:29,418] jepsen worker 252 - jepsen.util 252 :invoke :add 5902 INFO [2022-01-10 08:39:29,420] jepsen worker 480 - jepsen.util 480 :invoke :add 5903 INFO [2022-01-10 08:39:29,422] jepsen worker 784 - jepsen.util 784 :invoke :add 5904 INFO [2022-01-10 08:39:29,423] jepsen worker 367 - jepsen.util 367 :invoke :add 5905 INFO [2022-01-10 08:39:29,425] jepsen worker 773 - jepsen.util 773 :invoke :add 5906 INFO [2022-01-10 08:39:29,427] jepsen worker 90 - jepsen.util 90 :invoke :add 5907 INFO [2022-01-10 08:39:29,429] jepsen worker 300 - jepsen.util 300 :invoke :add 5908 INFO [2022-01-10 08:39:29,430] jepsen worker 62 - jepsen.util 62 :invoke :add 5909 INFO [2022-01-10 08:39:29,433] jepsen worker 180 - jepsen.util 180 :invoke :add 5910 INFO [2022-01-10 08:39:29,435] jepsen worker 628 - jepsen.util 628 :invoke :add 5911 INFO [2022-01-10 08:39:29,436] jepsen worker 323 - jepsen.util 323 :invoke :add 5912 INFO [2022-01-10 08:39:29,438] jepsen worker 366 - jepsen.util 366 :invoke :add 5913 INFO [2022-01-10 08:39:29,440] jepsen worker 36 - jepsen.util 36 :invoke :add 5914 INFO [2022-01-10 08:39:29,442] jepsen worker 678 - jepsen.util 678 :invoke :add 5915 INFO [2022-01-10 08:39:29,443] jepsen worker 483 - jepsen.util 483 :invoke :add 5916 INFO [2022-01-10 08:39:29,445] jepsen worker 956 - jepsen.util 956 :invoke :add 5917 INFO [2022-01-10 08:39:29,447] jepsen worker 836 - jepsen.util 836 :invoke :add 5918 INFO [2022-01-10 08:39:29,449] jepsen worker 207 - jepsen.util 207 :ok :add 5736 INFO [2022-01-10 08:39:29,449] jepsen worker 214 - jepsen.util 214 :ok :add 5692 INFO [2022-01-10 08:39:29,449] jepsen worker 653 - jepsen.util 653 :ok :add 5702 INFO [2022-01-10 08:39:29,449] jepsen worker 306 - jepsen.util 306 :invoke :add 5919 INFO [2022-01-10 08:39:29,449] jepsen worker 206 - jepsen.util 206 :ok :add 5813 INFO [2022-01-10 08:39:29,451] jepsen worker 167 - jepsen.util 167 :ok :add 5693 INFO [2022-01-10 08:39:29,451] jepsen worker 162 - jepsen.util 162 :ok :add 5703 INFO [2022-01-10 08:39:29,451] jepsen worker 263 - jepsen.util 263 :ok :add 5737 INFO [2022-01-10 08:39:29,451] jepsen worker 759 - jepsen.util 759 :ok :add 5812 INFO [2022-01-10 08:39:29,451] jepsen worker 605 - jepsen.util 605 :invoke :add 5920 INFO [2022-01-10 08:39:29,451] jepsen worker 808 - jepsen.util 808 :ok :add 5826 INFO [2022-01-10 08:39:29,455] jepsen worker 103 - jepsen.util 103 :invoke :add 5921 INFO [2022-01-10 08:39:29,456] jepsen worker 913 - jepsen.util 913 :invoke :add 5922 INFO [2022-01-10 08:39:29,458] jepsen worker 156 - jepsen.util 156 :invoke :add 5923 INFO [2022-01-10 08:39:29,461] jepsen worker 537 - jepsen.util 537 :invoke :add 5924 INFO [2022-01-10 08:39:29,463] jepsen worker 727 - jepsen.util 727 :invoke :add 5925 INFO [2022-01-10 08:39:29,464] jepsen worker 747 - jepsen.util 747 :invoke :add 5926 INFO [2022-01-10 08:39:29,466] jepsen worker 86 - jepsen.util 86 :invoke :add 5927 INFO [2022-01-10 08:39:29,468] jepsen worker 856 - jepsen.util 856 :invoke :add 5928 INFO [2022-01-10 08:39:29,470] jepsen worker 107 - jepsen.util 107 :invoke :add 5929 INFO [2022-01-10 08:39:29,472] jepsen worker 623 - jepsen.util 623 :invoke :add 5930 INFO [2022-01-10 08:39:29,474] jepsen worker 565 - jepsen.util 565 :invoke :add 5931 INFO [2022-01-10 08:39:29,476] jepsen worker 338 - jepsen.util 338 :invoke :add 5932 INFO [2022-01-10 08:39:29,478] jepsen worker 98 - jepsen.util 98 :invoke :add 5933 INFO [2022-01-10 08:39:29,480] jepsen worker 69 - jepsen.util 69 :invoke :add 5934 INFO [2022-01-10 08:39:29,482] jepsen worker 731 - jepsen.util 731 :invoke :add 5935 INFO [2022-01-10 08:39:29,484] jepsen worker 410 - jepsen.util 410 :invoke :add 5936 INFO [2022-01-10 08:39:29,485] jepsen worker 377 - jepsen.util 377 :invoke :add 5937 INFO [2022-01-10 08:39:29,487] jepsen worker 119 - jepsen.util 119 :invoke :add 5938 INFO [2022-01-10 08:39:29,489] jepsen worker 151 - jepsen.util 151 :invoke :add 5939 INFO [2022-01-10 08:39:29,492] jepsen worker 647 - jepsen.util 647 :invoke :add 5940 INFO [2022-01-10 08:39:29,494] jepsen worker 783 - jepsen.util 783 :invoke :add 5941 INFO [2022-01-10 08:39:29,495] jepsen worker 821 - jepsen.util 821 :invoke :add 5942 INFO [2022-01-10 08:39:29,497] jepsen worker 111 - jepsen.util 111 :invoke :add 5943 INFO [2022-01-10 08:39:29,499] jepsen worker 759 - jepsen.util 759 :invoke :add 5944 INFO [2022-01-10 08:39:29,501] jepsen worker 269 - jepsen.util 269 :invoke :add 5945 INFO [2022-01-10 08:39:29,503] jepsen worker 992 - jepsen.util 992 :invoke :add 5946 INFO [2022-01-10 08:39:29,505] jepsen worker 985 - jepsen.util 985 :invoke :add 5947 INFO [2022-01-10 08:39:29,507] jepsen worker 342 - jepsen.util 342 :invoke :add 5948 INFO [2022-01-10 08:39:29,509] jepsen worker 433 - jepsen.util 433 :invoke :add 5949 INFO [2022-01-10 08:39:29,511] jepsen worker 486 - jepsen.util 486 :invoke :add 5950 INFO [2022-01-10 08:39:29,513] jepsen worker 723 - jepsen.util 723 :invoke :add 5951 INFO [2022-01-10 08:39:29,515] jepsen worker 190 - jepsen.util 190 :invoke :add 5952 INFO [2022-01-10 08:39:29,517] jepsen worker 493 - jepsen.util 493 :invoke :add 5953 INFO [2022-01-10 08:39:29,519] jepsen worker 303 - jepsen.util 303 :invoke :add 5954 INFO [2022-01-10 08:39:29,521] jepsen worker 738 - jepsen.util 738 :invoke :add 5955 INFO [2022-01-10 08:39:29,524] jepsen worker 544 - jepsen.util 544 :invoke :add 5956 INFO [2022-01-10 08:39:29,526] jepsen worker 461 - jepsen.util 461 :invoke :add 5957 INFO [2022-01-10 08:39:29,528] jepsen worker 237 - jepsen.util 237 :invoke :add 5958 INFO [2022-01-10 08:39:29,529] jepsen worker 826 - jepsen.util 826 :invoke :add 5959 INFO [2022-01-10 08:39:29,532] jepsen worker 646 - jepsen.util 646 :invoke :add 5960 INFO [2022-01-10 08:39:29,534] jepsen worker 607 - jepsen.util 607 :invoke :add 5961 INFO [2022-01-10 08:39:29,536] jepsen worker 214 - jepsen.util 214 :invoke :add 5962 INFO [2022-01-10 08:39:29,537] jepsen worker 206 - jepsen.util 206 :invoke :add 5963 INFO [2022-01-10 08:39:29,539] jepsen worker 275 - jepsen.util 275 :invoke :add 5964 INFO [2022-01-10 08:39:29,541] jepsen worker 652 - jepsen.util 652 :invoke :add 5965 INFO [2022-01-10 08:39:29,543] jepsen worker 83 - jepsen.util 83 :invoke :add 5966 INFO [2022-01-10 08:39:29,545] jepsen worker 590 - jepsen.util 590 :invoke :add 5967 INFO [2022-01-10 08:39:29,547] jepsen worker 973 - jepsen.util 973 :invoke :add 5968 INFO [2022-01-10 08:39:29,549] jepsen worker 407 - jepsen.util 407 :invoke :add 5969 INFO [2022-01-10 08:39:29,551] jepsen worker 138 - jepsen.util 138 :invoke :add 5970 INFO [2022-01-10 08:39:29,553] jepsen worker 673 - jepsen.util 673 :invoke :add 5971 INFO [2022-01-10 08:39:29,555] jepsen worker 808 - jepsen.util 808 :invoke :add 5972 INFO [2022-01-10 08:39:29,557] jepsen worker 942 - jepsen.util 942 :invoke :add 5973 INFO [2022-01-10 08:39:29,559] jepsen worker 284 - jepsen.util 284 :invoke :add 5974 INFO [2022-01-10 08:39:29,561] jepsen worker 428 - jepsen.util 428 :invoke :add 5975 INFO [2022-01-10 08:39:29,563] jepsen worker 621 - jepsen.util 621 :invoke :add 5976 INFO [2022-01-10 08:39:29,565] jepsen worker 146 - jepsen.util 146 :invoke :add 5977 INFO [2022-01-10 08:39:29,567] jepsen worker 961 - jepsen.util 961 :invoke :add 5978 INFO [2022-01-10 08:39:29,569] jepsen worker 163 - jepsen.util 163 :invoke :add 5979 INFO [2022-01-10 08:39:29,572] jepsen worker 596 - jepsen.util 596 :invoke :add 5980 INFO [2022-01-10 08:39:29,574] jepsen worker 70 - jepsen.util 70 :invoke :add 5981 INFO [2022-01-10 08:39:29,576] jepsen worker 343 - jepsen.util 343 :invoke :add 5982 INFO [2022-01-10 08:39:29,578] jepsen worker 658 - jepsen.util 658 :invoke :add 5983 INFO [2022-01-10 08:39:29,580] jepsen worker 263 - jepsen.util 263 :invoke :add 5984 INFO [2022-01-10 08:39:29,582] jepsen worker 675 - jepsen.util 675 :invoke :add 5985 INFO [2022-01-10 08:39:29,585] jepsen worker 842 - jepsen.util 842 :invoke :add 5986 INFO [2022-01-10 08:39:29,586] jepsen worker 223 - jepsen.util 223 :invoke :add 5987 INFO [2022-01-10 08:39:29,588] jepsen worker 503 - jepsen.util 503 :invoke :add 5988 INFO [2022-01-10 08:39:29,591] jepsen worker 423 - jepsen.util 423 :invoke :add 5989 INFO [2022-01-10 08:39:29,593] jepsen worker 277 - jepsen.util 277 :invoke :add 5990 INFO [2022-01-10 08:39:29,595] jepsen worker 317 - jepsen.util 317 :invoke :add 5991 INFO [2022-01-10 08:39:29,597] jepsen worker 365 - jepsen.util 365 :invoke :add 5992 INFO [2022-01-10 08:39:29,599] jepsen worker 711 - jepsen.util 711 :invoke :add 5993 INFO [2022-01-10 08:39:29,601] jepsen worker 981 - jepsen.util 981 :invoke :add 5994 INFO [2022-01-10 08:39:29,604] jepsen worker 64 - jepsen.util 64 :invoke :add 5995 INFO [2022-01-10 08:39:29,605] jepsen worker 893 - jepsen.util 893 :invoke :add 5996 INFO [2022-01-10 08:39:29,607] jepsen worker 130 - jepsen.util 130 :invoke :add 5997 INFO [2022-01-10 08:39:29,610] jepsen worker 533 - jepsen.util 533 :invoke :add 5998 INFO [2022-01-10 08:39:29,612] jepsen worker 559 - jepsen.util 559 :invoke :add 5999 INFO [2022-01-10 08:39:29,614] jepsen worker 936 - jepsen.util 936 :invoke :add 6000 INFO [2022-01-10 08:39:29,616] jepsen worker 817 - jepsen.util 817 :invoke :add 6001 INFO [2022-01-10 08:39:29,618] jepsen worker 424 - jepsen.util 424 :invoke :add 6002 INFO [2022-01-10 08:39:29,619] jepsen worker 40 - jepsen.util 40 :invoke :add 6003 INFO [2022-01-10 08:39:29,622] jepsen worker 51 - jepsen.util 51 :invoke :add 6004 INFO [2022-01-10 08:39:29,623] jepsen worker 316 - jepsen.util 316 :invoke :add 6005 INFO [2022-01-10 08:39:29,626] jepsen worker 296 - jepsen.util 296 :invoke :add 6006 INFO [2022-01-10 08:39:29,628] jepsen worker 327 - jepsen.util 327 :invoke :add 6007 INFO [2022-01-10 08:39:29,630] jepsen worker 780 - jepsen.util 780 :invoke :add 6008 INFO [2022-01-10 08:39:29,633] jepsen worker 71 - jepsen.util 71 :invoke :add 6009 INFO [2022-01-10 08:39:29,635] jepsen worker 618 - jepsen.util 618 :invoke :add 6010 INFO [2022-01-10 08:39:29,637] jepsen worker 546 - jepsen.util 546 :invoke :add 6011 INFO [2022-01-10 08:39:29,638] jepsen worker 631 - jepsen.util 631 :invoke :add 6012 INFO [2022-01-10 08:39:29,640] jepsen worker 331 - jepsen.util 331 :invoke :add 6013 INFO [2022-01-10 08:39:29,642] jepsen worker 624 - jepsen.util 624 :invoke :add 6014 INFO [2022-01-10 08:39:29,644] jepsen worker 502 - jepsen.util 502 :invoke :add 6015 INFO [2022-01-10 08:39:29,645] jepsen worker 995 - jepsen.util 995 :invoke :add 6016 INFO [2022-01-10 08:39:29,647] jepsen worker 863 - jepsen.util 863 :invoke :add 6017 INFO [2022-01-10 08:39:29,649] jepsen worker 754 - jepsen.util 754 :invoke :add 6018 INFO [2022-01-10 08:39:29,651] jepsen worker 877 - jepsen.util 877 :invoke :add 6019 INFO [2022-01-10 08:39:29,653] jepsen worker 725 - jepsen.util 725 :invoke :add 6020 INFO [2022-01-10 08:39:29,655] jepsen worker 700 - jepsen.util 700 :invoke :add 6021 INFO [2022-01-10 08:39:29,657] jepsen worker 527 - jepsen.util 527 :invoke :add 6022 INFO [2022-01-10 08:39:29,659] jepsen worker 891 - jepsen.util 891 :invoke :add 6023 INFO [2022-01-10 08:39:29,661] jepsen worker 912 - jepsen.util 912 :invoke :add 6024 INFO [2022-01-10 08:39:29,663] jepsen worker 707 - jepsen.util 707 :invoke :add 6025 INFO [2022-01-10 08:39:29,666] jepsen worker 640 - jepsen.util 640 :invoke :add 6026 INFO [2022-01-10 08:39:29,669] jepsen worker 232 - jepsen.util 232 :invoke :add 6027 INFO [2022-01-10 08:39:29,672] jepsen worker 573 - jepsen.util 573 :invoke :add 6028 INFO [2022-01-10 08:39:29,675] jepsen worker 250 - jepsen.util 250 :invoke :add 6029 INFO [2022-01-10 08:39:29,677] jepsen worker 653 - jepsen.util 653 :invoke :add 6030 INFO [2022-01-10 08:39:29,679] jepsen worker 804 - jepsen.util 804 :invoke :add 6031 INFO [2022-01-10 08:39:29,681] jepsen worker 751 - jepsen.util 751 :invoke :add 6032 INFO [2022-01-10 08:39:29,683] jepsen worker 165 - jepsen.util 165 :invoke :add 6033 INFO [2022-01-10 08:39:29,685] jepsen worker 914 - jepsen.util 914 :invoke :add 6034 INFO [2022-01-10 08:39:29,687] jepsen worker 443 - jepsen.util 443 :invoke :add 6035 INFO [2022-01-10 08:39:29,689] jepsen worker 938 - jepsen.util 938 :invoke :add 6036 INFO [2022-01-10 08:39:29,690] jepsen worker 713 - jepsen.util 713 :invoke :add 6037 INFO [2022-01-10 08:39:29,692] jepsen worker 745 - jepsen.util 745 :invoke :add 6038 INFO [2022-01-10 08:39:29,694] jepsen worker 814 - jepsen.util 814 :invoke :add 6039 INFO [2022-01-10 08:39:29,696] jepsen worker 929 - jepsen.util 929 :invoke :add 6040 INFO [2022-01-10 08:39:29,698] jepsen worker 167 - jepsen.util 167 :invoke :add 6041 INFO [2022-01-10 08:39:29,701] jepsen worker 6 - jepsen.util 6 :invoke :add 6042 INFO [2022-01-10 08:39:29,703] jepsen worker 587 - jepsen.util 587 :invoke :add 6043 INFO [2022-01-10 08:39:29,705] jepsen worker 739 - jepsen.util 739 :invoke :add 6044 INFO [2022-01-10 08:39:29,707] jepsen worker 944 - jepsen.util 944 :invoke :add 6045 INFO [2022-01-10 08:39:29,709] jepsen worker 570 - jepsen.util 570 :invoke :add 6046 INFO [2022-01-10 08:39:29,710] jepsen worker 239 - jepsen.util 239 :invoke :add 6047 INFO [2022-01-10 08:39:29,712] jepsen worker 636 - jepsen.util 636 :invoke :add 6048 INFO [2022-01-10 08:39:29,715] jepsen worker 924 - jepsen.util 924 :invoke :add 6049 INFO [2022-01-10 08:39:29,717] jepsen worker 115 - jepsen.util 115 :invoke :add 6050 INFO [2022-01-10 08:39:29,720] jepsen worker 539 - jepsen.util 539 :invoke :add 6051 INFO [2022-01-10 08:39:29,721] jepsen worker 121 - jepsen.util 121 :invoke :add 6052 INFO [2022-01-10 08:39:29,723] jepsen worker 470 - jepsen.util 470 :invoke :add 6053 INFO [2022-01-10 08:39:29,726] jepsen worker 767 - jepsen.util 767 :invoke :add 6054 INFO [2022-01-10 08:39:29,728] jepsen worker 737 - jepsen.util 737 :invoke :add 6055 INFO [2022-01-10 08:39:29,730] jepsen worker 795 - jepsen.util 795 :invoke :add 6056 INFO [2022-01-10 08:39:29,731] jepsen worker 127 - jepsen.util 127 :invoke :add 6057 INFO [2022-01-10 08:39:29,734] jepsen worker 611 - jepsen.util 611 :invoke :add 6058 INFO [2022-01-10 08:39:29,735] jepsen worker 159 - jepsen.util 159 :invoke :add 6059 INFO [2022-01-10 08:39:29,737] jepsen worker 776 - jepsen.util 776 :invoke :add 6060 INFO [2022-01-10 08:39:29,739] jepsen worker 698 - jepsen.util 698 :invoke :add 6061 INFO [2022-01-10 08:39:29,741] jepsen worker 28 - jepsen.util 28 :invoke :add 6062 INFO [2022-01-10 08:39:29,744] jepsen worker 35 - jepsen.util 35 :invoke :add 6063 INFO [2022-01-10 08:39:29,746] jepsen worker 466 - jepsen.util 466 :invoke :add 6064 INFO [2022-01-10 08:39:29,748] jepsen worker 170 - jepsen.util 170 :invoke :add 6065 INFO [2022-01-10 08:39:29,751] jepsen worker 534 - jepsen.util 534 :invoke :add 6066 INFO [2022-01-10 08:39:29,753] jepsen worker 153 - jepsen.util 153 :invoke :add 6067 INFO [2022-01-10 08:39:29,755] jepsen worker 910 - jepsen.util 910 :invoke :add 6068 INFO [2022-01-10 08:39:29,756] jepsen worker 818 - jepsen.util 818 :ok :add 5674 INFO [2022-01-10 08:39:29,757] jepsen worker 360 - jepsen.util 360 :ok :add 5495 INFO [2022-01-10 08:39:29,757] jepsen worker 591 - jepsen.util 591 :ok :add 5531 INFO [2022-01-10 08:39:29,757] jepsen worker 950 - jepsen.util 950 :ok :add 5658 INFO [2022-01-10 08:39:29,756] jepsen worker 941 - jepsen.util 941 :ok :add 5640 INFO [2022-01-10 08:39:29,757] jepsen worker 445 - jepsen.util 445 :ok :add 5529 INFO [2022-01-10 08:39:29,757] jepsen worker 576 - jepsen.util 576 :ok :add 5505 INFO [2022-01-10 08:39:29,757] jepsen worker 543 - jepsen.util 543 :invoke :add 6069 INFO [2022-01-10 08:39:29,760] jepsen worker 96 - jepsen.util 96 :ok :add 5329 INFO [2022-01-10 08:39:29,760] jepsen worker 666 - jepsen.util 666 :ok :add 5331 INFO [2022-01-10 08:39:29,761] jepsen worker 820 - jepsen.util 820 :ok :add 5443 INFO [2022-01-10 08:39:29,761] jepsen worker 453 - jepsen.util 453 :ok :add 5823 INFO [2022-01-10 08:39:29,761] jepsen worker 778 - jepsen.util 778 :ok :add 5696 INFO [2022-01-10 08:39:29,761] jepsen worker 39 - jepsen.util 39 :ok :add 5706 INFO [2022-01-10 08:39:29,761] jepsen worker 639 - jepsen.util 639 :ok :add 5732 INFO [2022-01-10 08:39:29,761] jepsen worker 598 - jepsen.util 598 :ok :add 5477 INFO [2022-01-10 08:39:29,761] jepsen worker 52 - jepsen.util 52 :ok :add 5817 INFO [2022-01-10 08:39:29,762] jepsen worker 712 - jepsen.util 712 :invoke :add 6070 INFO [2022-01-10 08:39:29,766] jepsen worker 387 - jepsen.util 387 :invoke :add 6071 INFO [2022-01-10 08:39:29,768] jepsen worker 26 - jepsen.util 26 :invoke :add 6072 INFO [2022-01-10 08:39:29,768] jepsen worker 976 - jepsen.util 976 :ok :add 5700 INFO [2022-01-10 08:39:29,769] jepsen worker 242 - jepsen.util 242 :ok :add 5718 INFO [2022-01-10 08:39:29,769] jepsen worker 517 - jepsen.util 517 :ok :add 5734 INFO [2022-01-10 08:39:29,769] jepsen worker 630 - jepsen.util 630 :ok :add 5688 INFO [2022-01-10 08:39:29,769] jepsen worker 363 - jepsen.util 363 :ok :add 5691 INFO [2022-01-10 08:39:29,769] jepsen worker 477 - jepsen.util 477 :ok :add 5824 INFO [2022-01-10 08:39:29,769] jepsen worker 390 - jepsen.util 390 :ok :add 5825 INFO [2022-01-10 08:39:29,769] jepsen worker 183 - jepsen.util 183 :ok :add 5811 INFO [2022-01-10 08:39:29,770] jepsen worker 782 - jepsen.util 782 :ok :add 5735 INFO [2022-01-10 08:39:29,770] jepsen worker 966 - jepsen.util 966 :ok :add 5809 INFO [2022-01-10 08:39:29,770] jepsen worker 352 - jepsen.util 352 :ok :add 5689 INFO [2022-01-10 08:39:29,770] jepsen worker 604 - jepsen.util 604 :ok :add 5690 INFO [2022-01-10 08:39:29,771] jepsen worker 790 - jepsen.util 790 :invoke :add 6073 INFO [2022-01-10 08:39:29,771] jepsen worker 24 - jepsen.util 24 :ok :add 5701 INFO [2022-01-10 08:39:29,772] jepsen worker 50 - jepsen.util 50 :ok :add 5719 INFO [2022-01-10 08:39:29,772] jepsen worker 781 - jepsen.util 781 :ok :add 5808 INFO [2022-01-10 08:39:29,776] jepsen worker 775 - jepsen.util 775 :invoke :add 6074 INFO [2022-01-10 08:39:29,779] jepsen worker 189 - jepsen.util 189 :invoke :add 6075 INFO [2022-01-10 08:39:29,781] jepsen worker 825 - jepsen.util 825 :invoke :add 6076 INFO [2022-01-10 08:39:29,784] jepsen worker 597 - jepsen.util 597 :invoke :add 6077 INFO [2022-01-10 08:39:29,786] jepsen worker 455 - jepsen.util 455 :invoke :add 6078 INFO [2022-01-10 08:39:29,788] jepsen worker 142 - jepsen.util 142 :invoke :add 6079 INFO [2022-01-10 08:39:29,790] jepsen worker 672 - jepsen.util 672 :invoke :add 6080 INFO [2022-01-10 08:39:29,793] jepsen worker 568 - jepsen.util 568 :invoke :add 6081 INFO [2022-01-10 08:39:29,795] jepsen worker 679 - jepsen.util 679 :invoke :add 6082 INFO [2022-01-10 08:39:29,797] jepsen worker 840 - jepsen.util 840 :invoke :add 6083 INFO [2022-01-10 08:39:29,799] jepsen worker 354 - jepsen.util 354 :invoke :add 6084 INFO [2022-01-10 08:39:29,800] jepsen worker 172 - jepsen.util 172 :invoke :add 6085 INFO [2022-01-10 08:39:29,802] jepsen worker 453 - jepsen.util 453 :invoke :add 6086 INFO [2022-01-10 08:39:29,804] jepsen worker 382 - jepsen.util 382 :invoke :add 6087 INFO [2022-01-10 08:39:29,806] jepsen worker 934 - jepsen.util 934 :invoke :add 6088 INFO [2022-01-10 08:39:29,807] jepsen worker 898 - jepsen.util 898 :invoke :add 6089 INFO [2022-01-10 08:39:29,809] jepsen worker 966 - jepsen.util 966 :invoke :add 6090 INFO [2022-01-10 08:39:29,811] jepsen worker 906 - jepsen.util 906 :invoke :add 6091 INFO [2022-01-10 08:39:29,813] jepsen worker 604 - jepsen.util 604 :invoke :add 6092 INFO [2022-01-10 08:39:29,815] jepsen worker 941 - jepsen.util 941 :invoke :add 6093 INFO [2022-01-10 08:39:29,817] jepsen worker 782 - jepsen.util 782 :invoke :add 6094 INFO [2022-01-10 08:39:29,819] jepsen worker 699 - jepsen.util 699 :invoke :add 6095 INFO [2022-01-10 08:39:29,821] jepsen worker 657 - jepsen.util 657 :invoke :add 6096 INFO [2022-01-10 08:39:29,822] jepsen worker 191 - jepsen.util 191 :invoke :add 6097 INFO [2022-01-10 08:39:29,824] jepsen worker 18 - jepsen.util 18 :invoke :add 6098 INFO [2022-01-10 08:39:29,826] jepsen worker 422 - jepsen.util 422 :invoke :add 6099 INFO [2022-01-10 08:39:29,828] jepsen worker 183 - jepsen.util 183 :invoke :add 6100 INFO [2022-01-10 08:39:29,830] jepsen worker 923 - jepsen.util 923 :invoke :add 6101 INFO [2022-01-10 08:39:29,832] jepsen worker 477 - jepsen.util 477 :invoke :add 6102 INFO [2022-01-10 08:39:29,833] jepsen worker 162 - jepsen.util 162 :invoke :add 6103 INFO [2022-01-10 08:39:29,835] jepsen worker 397 - jepsen.util 397 :invoke :add 6104 INFO [2022-01-10 08:39:29,838] jepsen worker 974 - jepsen.util 974 :invoke :add 6106 INFO [2022-01-10 08:39:29,838] jepsen worker 50 - jepsen.util 50 :invoke :add 6105 INFO [2022-01-10 08:39:29,840] jepsen worker 308 - jepsen.util 308 :invoke :add 6107 INFO [2022-01-10 08:39:29,842] jepsen worker 225 - jepsen.util 225 :invoke :add 6108 INFO [2022-01-10 08:39:29,844] jepsen worker 666 - jepsen.util 666 :invoke :add 6109 INFO [2022-01-10 08:39:29,846] jepsen worker 207 - jepsen.util 207 :invoke :add 6110 INFO [2022-01-10 08:39:29,848] jepsen worker 626 - jepsen.util 626 :invoke :add 6111 INFO [2022-01-10 08:39:29,849] jepsen worker 778 - jepsen.util 778 :invoke :add 6112 INFO [2022-01-10 08:39:29,852] jepsen worker 523 - jepsen.util 523 :invoke :add 6113 INFO [2022-01-10 08:39:29,853] jepsen worker 849 - jepsen.util 849 :invoke :add 6114 INFO [2022-01-10 08:39:29,855] jepsen worker 887 - jepsen.util 887 :invoke :add 6115 INFO [2022-01-10 08:39:29,856] jepsen worker 3 - jepsen.util 3 :invoke :add 6116 INFO [2022-01-10 08:39:29,858] jepsen worker 719 - jepsen.util 719 :invoke :add 6117 INFO [2022-01-10 08:39:29,860] jepsen worker 901 - jepsen.util 901 :invoke :add 6118 INFO [2022-01-10 08:39:29,862] jepsen worker 378 - jepsen.util 378 :invoke :add 6119 INFO [2022-01-10 08:39:29,863] jepsen worker 198 - jepsen.util 198 :invoke :add 6120 INFO [2022-01-10 08:39:29,866] jepsen worker 372 - jepsen.util 372 :invoke :add 6121 INFO [2022-01-10 08:39:29,867] jepsen worker 709 - jepsen.util 709 :invoke :add 6122 INFO [2022-01-10 08:39:29,869] jepsen worker 781 - jepsen.util 781 :invoke :add 6123 INFO [2022-01-10 08:39:29,870] jepsen worker 665 - jepsen.util 665 :ok :add 5672 INFO [2022-01-10 08:39:29,871] jepsen worker 445 - jepsen.util 445 :invoke :add 6124 INFO [2022-01-10 08:39:29,873] jepsen worker 758 - jepsen.util 758 :invoke :add 6125 INFO [2022-01-10 08:39:29,875] jepsen worker 457 - jepsen.util 457 :invoke :add 6126 INFO [2022-01-10 08:39:29,876] jepsen worker 390 - jepsen.util 390 :invoke :add 6127 INFO [2022-01-10 08:39:29,878] jepsen worker 161 - jepsen.util 161 :invoke :add 6128 INFO [2022-01-10 08:39:29,880] jepsen worker 823 - jepsen.util 823 :invoke :add 6129 INFO [2022-01-10 08:39:29,882] jepsen worker 591 - jepsen.util 591 :invoke :add 6130 INFO [2022-01-10 08:39:29,883] jepsen worker 818 - jepsen.util 818 :invoke :add 6131 INFO [2022-01-10 08:39:29,885] jepsen worker 617 - jepsen.util 617 :invoke :add 6132 INFO [2022-01-10 08:39:29,887] jepsen worker 531 - jepsen.util 531 :invoke :add 6133 INFO [2022-01-10 08:39:29,889] jepsen worker 260 - jepsen.util 260 :invoke :add 6134 INFO [2022-01-10 08:39:29,890] jepsen worker 517 - jepsen.util 517 :invoke :add 6135 INFO [2022-01-10 08:39:29,892] jepsen worker 639 - jepsen.util 639 :invoke :add 6136 INFO [2022-01-10 08:39:29,894] jepsen worker 357 - jepsen.util 357 :invoke :add 6137 INFO [2022-01-10 08:39:29,895] jepsen worker 307 - jepsen.util 307 :invoke :add 6138 INFO [2022-01-10 08:39:29,898] jepsen worker 45 - jepsen.util 45 :invoke :add 6139 INFO [2022-01-10 08:39:29,900] jepsen worker 52 - jepsen.util 52 :invoke :add 6140 INFO [2022-01-10 08:39:29,902] jepsen worker 29 - jepsen.util 29 :invoke :add 6141 INFO [2022-01-10 08:39:29,904] jepsen worker 58 - jepsen.util 58 :invoke :add 6142 INFO [2022-01-10 08:39:29,906] jepsen worker 927 - jepsen.util 927 :invoke :add 6143 INFO [2022-01-10 08:39:29,908] jepsen worker 538 - jepsen.util 538 :invoke :add 6144 INFO [2022-01-10 08:39:29,910] jepsen worker 703 - jepsen.util 703 :invoke :add 6145 INFO [2022-01-10 08:39:29,912] jepsen worker 510 - jepsen.util 510 :invoke :add 6146 INFO [2022-01-10 08:39:29,914] jepsen worker 496 - jepsen.util 496 :invoke :add 6147 INFO [2022-01-10 08:39:29,915] jepsen worker 168 - jepsen.util 168 :invoke :add 6148 INFO [2022-01-10 08:39:29,917] jepsen worker 721 - jepsen.util 721 :ok :add 5792 INFO [2022-01-10 08:39:29,917] jepsen worker 242 - jepsen.util 242 :invoke :add 6149 INFO [2022-01-10 08:39:29,919] jepsen worker 935 - jepsen.util 935 :invoke :add 6150 INFO [2022-01-10 08:39:29,920] jepsen worker 366 - jepsen.util 366 :ok :add 5913 INFO [2022-01-10 08:39:29,921] jepsen worker 746 - jepsen.util 746 :invoke :add 6151 INFO [2022-01-10 08:39:29,923] jepsen worker 230 - jepsen.util 230 :invoke :add 6152 INFO [2022-01-10 08:39:29,925] jepsen worker 721 - jepsen.util 721 :invoke :add 6153 INFO [2022-01-10 08:39:29,927] jepsen worker 977 - jepsen.util 977 :invoke :add 6154 INFO [2022-01-10 08:39:29,928] jepsen worker 383 - jepsen.util 383 :invoke :add 6155 INFO [2022-01-10 08:39:29,930] jepsen worker 73 - jepsen.util 73 :invoke :add 6156 INFO [2022-01-10 08:39:29,932] jepsen worker 39 - jepsen.util 39 :invoke :add 6157 INFO [2022-01-10 08:39:29,934] jepsen worker 900 - jepsen.util 900 :invoke :add 6158 INFO [2022-01-10 08:39:29,935] jepsen worker 888 - jepsen.util 888 :invoke :add 6159 INFO [2022-01-10 08:39:29,937] jepsen worker 576 - jepsen.util 576 :invoke :add 6160 INFO [2022-01-10 08:39:29,939] jepsen worker 182 - jepsen.util 182 :invoke :add 6161 INFO [2022-01-10 08:39:29,940] jepsen worker 553 - jepsen.util 553 :invoke :add 6162 INFO [2022-01-10 08:39:29,943] jepsen worker 113 - jepsen.util 113 :invoke :add 6163 INFO [2022-01-10 08:39:29,943] jepsen worker 163 - jepsen.util 163 :ok :add 5979 INFO [2022-01-10 08:39:29,944] jepsen worker 534 - jepsen.util 534 :ok :add 6066 INFO [2022-01-10 08:39:29,944] jepsen worker 607 - jepsen.util 607 :ok :add 5961 INFO [2022-01-10 08:39:29,944] jepsen worker 469 - jepsen.util 469 :invoke :add 6164 INFO [2022-01-10 08:39:29,947] jepsen worker 57 - jepsen.util 57 :invoke :add 6165 INFO [2022-01-10 08:39:29,949] jepsen worker 274 - jepsen.util 274 :invoke :add 6166 INFO [2022-01-10 08:39:29,951] jepsen worker 155 - jepsen.util 155 :invoke :add 6167 INFO [2022-01-10 08:39:29,952] jepsen worker 814 - jepsen.util 814 :ok :add 6039 INFO [2022-01-10 08:39:29,952] jepsen worker 700 - jepsen.util 700 :ok :add 6021 INFO [2022-01-10 08:39:29,952] jepsen worker 502 - jepsen.util 502 :ok :add 6015 INFO [2022-01-10 08:39:29,952] jepsen worker 158 - jepsen.util 158 :ok :add 5793 INFO [2022-01-10 08:39:29,952] jepsen worker 323 - jepsen.util 323 :ok :add 5912 INFO [2022-01-10 08:39:29,953] jepsen worker 632 - jepsen.util 632 :invoke :add 6168 INFO [2022-01-10 08:39:29,956] jepsen worker 700 - jepsen.util 700 :invoke :add 6169 INFO [2022-01-10 08:39:29,958] jepsen worker 819 - jepsen.util 819 :invoke :add 6170 INFO [2022-01-10 08:39:29,959] jepsen worker 561 - jepsen.util 561 :invoke :add 6171 INFO [2022-01-10 08:39:29,961] jepsen worker 259 - jepsen.util 259 :invoke :add 6172 INFO [2022-01-10 08:39:29,963] jepsen worker 396 - jepsen.util 396 :invoke :add 6173 INFO [2022-01-10 08:39:29,965] jepsen worker 580 - jepsen.util 580 :invoke :add 6174 INFO [2022-01-10 08:39:29,967] jepsen worker 599 - jepsen.util 599 :invoke :add 6175 INFO [2022-01-10 08:39:29,968] jepsen worker 323 - jepsen.util 323 :invoke :add 6176 INFO [2022-01-10 08:39:29,970] jepsen worker 163 - jepsen.util 163 :invoke :add 6177 INFO [2022-01-10 08:39:29,972] jepsen worker 690 - jepsen.util 690 :invoke :add 6178 INFO [2022-01-10 08:39:29,974] jepsen worker 950 - jepsen.util 950 :invoke :add 6179 INFO [2022-01-10 08:39:29,976] jepsen worker 814 - jepsen.util 814 :invoke :add 6180 INFO [2022-01-10 08:39:29,978] jepsen worker 181 - jepsen.util 181 :invoke :add 6181 INFO [2022-01-10 08:39:29,980] jepsen worker 492 - jepsen.util 492 :invoke :add 6182 INFO [2022-01-10 08:39:29,982] jepsen worker 360 - jepsen.util 360 :invoke :add 6183 INFO [2022-01-10 08:39:29,984] jepsen worker 968 - jepsen.util 968 :invoke :add 6184 INFO [2022-01-10 08:39:29,986] jepsen worker 366 - jepsen.util 366 :invoke :add 6185 INFO [2022-01-10 08:39:29,989] jepsen worker 601 - jepsen.util 601 :invoke :add 6186 INFO [2022-01-10 08:39:29,991] jepsen worker 976 - jepsen.util 976 :invoke :add 6187 INFO [2022-01-10 08:39:29,994] jepsen worker 920 - jepsen.util 920 :invoke :add 6188 INFO [2022-01-10 08:39:29,996] jepsen worker 534 - jepsen.util 534 :invoke :add 6189 INFO [2022-01-10 08:39:29,998] jepsen worker 75 - jepsen.util 75 :ok :add 5788 INFO [2022-01-10 08:39:29,998] jepsen worker 46 - jepsen.util 46 :ok :add 5791 INFO [2022-01-10 08:39:29,998] jepsen worker 831 - jepsen.util 831 :ok :add 5899 INFO [2022-01-10 08:39:29,998] jepsen worker 628 - jepsen.util 628 :ok :add 5911 INFO [2022-01-10 08:39:29,998] jepsen worker 62 - jepsen.util 62 :ok :add 5909 INFO [2022-01-10 08:39:30,000] jepsen worker 787 - jepsen.util 787 :ok :add 5880 INFO [2022-01-10 08:39:30,001] jepsen worker 78 - jepsen.util 78 :ok :add 5789 INFO [2022-01-10 08:39:30,001] jepsen worker 143 - jepsen.util 143 :invoke :add 6190 INFO [2022-01-10 08:39:30,001] jepsen worker 114 - jepsen.util 114 :invoke :add 6191 INFO [2022-01-10 08:39:30,001] jepsen worker 989 - jepsen.util 989 :ok :add 5898 INFO [2022-01-10 08:39:30,001] jepsen worker 180 - jepsen.util 180 :ok :add 5910 INFO [2022-01-10 08:39:30,001] jepsen worker 537 - jepsen.util 537 :ok :add 5924 INFO [2022-01-10 08:39:30,001] jepsen worker 933 - jepsen.util 933 :ok :add 5790 INFO [2022-01-10 08:39:30,001] jepsen worker 527 - jepsen.util 527 :ok :add 6022 INFO [2022-01-10 08:39:30,004] jepsen worker 415 - jepsen.util 415 :invoke :add 6192 INFO [2022-01-10 08:39:30,006] jepsen worker 502 - jepsen.util 502 :invoke :add 6193 INFO [2022-01-10 08:39:30,008] jepsen worker 524 - jepsen.util 524 :invoke :add 6194 INFO [2022-01-10 08:39:30,011] jepsen worker 598 - jepsen.util 598 :invoke :add 6195 INFO [2022-01-10 08:39:30,013] jepsen worker 792 - jepsen.util 792 :invoke :add 6196 INFO [2022-01-10 08:39:30,015] jepsen worker 93 - jepsen.util 93 :invoke :add 6197 INFO [2022-01-10 08:39:30,017] jepsen worker 62 - jepsen.util 62 :invoke :add 6198 INFO [2022-01-10 08:39:30,019] jepsen worker 787 - jepsen.util 787 :invoke :add 6199 INFO [2022-01-10 08:39:30,020] jepsen worker 313 - jepsen.util 313 :ok :add 5862 INFO [2022-01-10 08:39:30,020] jepsen worker 334 - jepsen.util 334 :ok :add 5773 INFO [2022-01-10 08:39:30,021] jepsen worker 212 - jepsen.util 212 :invoke :add 6200 INFO [2022-01-10 08:39:30,023] jepsen worker 630 - jepsen.util 630 :invoke :add 6201 INFO [2022-01-10 08:39:30,024] jepsen worker 988 - jepsen.util 988 :ok :add 5857 INFO [2022-01-10 08:39:30,024] jepsen worker 935 - jepsen.util 935 :ok :add 6150 INFO [2022-01-10 08:39:30,025] jepsen worker 542 - jepsen.util 542 :ok :add 5772 INFO [2022-01-10 08:39:30,025] jepsen worker 288 - jepsen.util 288 :ok :add 5863 INFO [2022-01-10 08:39:30,025] jepsen worker 417 - jepsen.util 417 :invoke :add 6202 INFO [2022-01-10 08:39:30,028] jepsen worker 74 - jepsen.util 74 :invoke :add 6203 INFO [2022-01-10 08:39:30,029] jepsen worker 935 - jepsen.util 935 :invoke :add 6204 INFO [2022-01-10 08:39:30,031] jepsen worker 187 - jepsen.util 187 :invoke :add 6205 INFO [2022-01-10 08:39:30,032] jepsen worker 613 - jepsen.util 613 :ok :add 5496 INFO [2022-01-10 08:39:30,032] jepsen worker 816 - jepsen.util 816 :ok :add 5532 INFO [2022-01-10 08:39:30,032] jepsen worker 328 - jepsen.util 328 :ok :add 5506 INFO [2022-01-10 08:39:30,032] jepsen worker 501 - jepsen.util 501 :ok :add 5677 INFO [2022-01-10 08:39:30,032] jepsen worker 824 - jepsen.util 824 :ok :add 5643 INFO [2022-01-10 08:39:30,033] jepsen worker 479 - jepsen.util 479 :ok :add 5595 INFO [2022-01-10 08:39:30,033] jepsen worker 908 - jepsen.util 908 :invoke :add 6206 INFO [2022-01-10 08:39:30,033] jepsen worker 102 - jepsen.util 102 :ok :add 5740 INFO [2022-01-10 08:39:30,033] jepsen worker 726 - jepsen.util 726 :ok :add 5758 INFO [2022-01-10 08:39:30,033] jepsen worker 655 - jepsen.util 655 :ok :add 5774 INFO [2022-01-10 08:39:30,033] jepsen worker 532 - jepsen.util 532 :ok :add 5851 INFO [2022-01-10 08:39:30,034] jepsen worker 525 - jepsen.util 525 :ok :add 5849 INFO [2022-01-10 08:39:30,036] jepsen worker 363 - jepsen.util 363 :invoke :add 6207 INFO [2022-01-10 08:39:30,038] jepsen worker 96 - jepsen.util 96 :invoke :add 6208 INFO [2022-01-10 08:39:30,040] jepsen worker 434 - jepsen.util 434 :invoke :add 6209 INFO [2022-01-10 08:39:30,042] jepsen worker 933 - jepsen.util 933 :invoke :add 6210 INFO [2022-01-10 08:39:30,044] jepsen worker 655 - jepsen.util 655 :invoke :add 6211 INFO [2022-01-10 08:39:30,044] jepsen worker 116 - jepsen.util 116 :ok :add 5879 INFO [2022-01-10 08:39:30,044] jepsen worker 589 - jepsen.util 589 :ok :add 5770 INFO [2022-01-10 08:39:30,044] jepsen worker 274 - jepsen.util 274 :ok :add 6166 INFO [2022-01-10 08:39:30,046] jepsen worker 20 - jepsen.util 20 :ok :add 5769 INFO [2022-01-10 08:39:30,046] jepsen worker 471 - jepsen.util 471 :ok :add 5860 INFO [2022-01-10 08:39:30,046] jepsen worker 721 - jepsen.util 721 :ok :add 6153 INFO [2022-01-10 08:39:30,046] jepsen worker 9 - jepsen.util 9 :ok :add 5854 INFO [2022-01-10 08:39:30,046] jepsen worker 689 - jepsen.util 689 :ok :add 5768 INFO [2022-01-10 08:39:30,046] jepsen worker 988 - jepsen.util 988 :invoke :add 6212 INFO [2022-01-10 08:39:30,046] jepsen worker 222 - jepsen.util 222 :ok :add 5771 INFO [2022-01-10 08:39:30,049] jepsen worker 217 - jepsen.util 217 :invoke :add 6213 INFO [2022-01-10 08:39:30,051] jepsen worker 542 - jepsen.util 542 :invoke :add 6214 INFO [2022-01-10 08:39:30,054] jepsen worker 102 - jepsen.util 102 :invoke :add 6215 INFO [2022-01-10 08:39:30,055] jepsen worker 537 - jepsen.util 537 :invoke :add 6216 INFO [2022-01-10 08:39:30,057] jepsen worker 764 - jepsen.util 764 :invoke :add 6217 INFO [2022-01-10 08:39:30,059] jepsen worker 525 - jepsen.util 525 :invoke :add 6218 INFO [2022-01-10 08:39:30,060] jepsen worker 46 - jepsen.util 46 :invoke :add 6219 INFO [2022-01-10 08:39:30,063] jepsen worker 582 - jepsen.util 582 :invoke :add 6220 INFO [2022-01-10 08:39:30,064] jepsen worker 683 - jepsen.util 683 :invoke :add 6221 INFO [2022-01-10 08:39:30,066] jepsen worker 218 - jepsen.util 218 :invoke :add 6222 INFO [2022-01-10 08:39:30,067] jepsen worker 197 - jepsen.util 197 :invoke :add 6223 INFO [2022-01-10 08:39:30,069] jepsen worker 24 - jepsen.util 24 :invoke :add 6224 INFO [2022-01-10 08:39:30,072] jepsen worker 129 - jepsen.util 129 :invoke :add 6225 INFO [2022-01-10 08:39:30,075] jepsen worker 689 - jepsen.util 689 :invoke :add 6226 INFO [2022-01-10 08:39:30,077] jepsen worker 311 - jepsen.util 311 :invoke :add 6227 INFO [2022-01-10 08:39:30,079] jepsen worker 864 - jepsen.util 864 :invoke :add 6228 INFO [2022-01-10 08:39:30,080] jepsen worker 352 - jepsen.util 352 :invoke :add 6229 INFO [2022-01-10 08:39:30,082] jepsen worker 442 - jepsen.util 442 :invoke :add 6230 INFO [2022-01-10 08:39:30,084] jepsen worker 844 - jepsen.util 844 :ok :add 4796 INFO [2022-01-10 08:39:30,084] jepsen worker 344 - jepsen.util 344 :ok :add 5013 INFO [2022-01-10 08:39:30,085] jepsen worker 774 - jepsen.util 774 :ok :add 5755 INFO [2022-01-10 08:39:30,085] jepsen worker 518 - jepsen.util 518 :ok :add 5580 INFO [2022-01-10 08:39:30,085] jepsen worker 158 - jepsen.util 158 :invoke :add 6231 INFO [2022-01-10 08:39:30,085] jepsen worker 314 - jepsen.util 314 :ok :add 5761 INFO [2022-01-10 08:39:30,087] jepsen worker 68 - jepsen.util 68 :ok :add 4917 INFO [2022-01-10 08:39:30,087] jepsen worker 743 - jepsen.util 743 :ok :add 5410 INFO [2022-01-10 08:39:30,087] jepsen worker 650 - jepsen.util 650 :ok :add 5362 INFO [2022-01-10 08:39:30,087] jepsen worker 965 - jepsen.util 965 :ok :add 5598 INFO [2022-01-10 08:39:30,087] jepsen worker 549 - jepsen.util 549 :ok :add 5183 INFO [2022-01-10 08:39:30,087] jepsen worker 247 - jepsen.util 247 :ok :add 4923 INFO [2022-01-10 08:39:30,087] jepsen worker 399 - jepsen.util 399 :ok :add 4887 INFO [2022-01-10 08:39:30,087] jepsen worker 674 - jepsen.util 674 :ok :add 5356 INFO [2022-01-10 08:39:30,087] jepsen worker 930 - jepsen.util 930 :ok :add 5408 INFO [2022-01-10 08:39:30,087] jepsen worker 243 - jepsen.util 243 :ok :add 5027 INFO [2022-01-10 08:39:30,088] jepsen worker 180 - jepsen.util 180 :invoke :add 6232 INFO [2022-01-10 08:39:30,092] jepsen worker 288 - jepsen.util 288 :invoke :add 6233 INFO [2022-01-10 08:39:30,094] jepsen worker 75 - jepsen.util 75 :invoke :add 6234 INFO [2022-01-10 08:39:30,096] jepsen worker 471 - jepsen.util 471 :invoke :add 6235 INFO [2022-01-10 08:39:30,097] jepsen worker 558 - jepsen.util 558 :invoke :add 6236 INFO [2022-01-10 08:39:30,099] jepsen worker 831 - jepsen.util 831 :invoke :add 6237 INFO [2022-01-10 08:39:30,101] jepsen worker 824 - jepsen.util 824 :invoke :add 6238 INFO [2022-01-10 08:39:30,102] jepsen worker 613 - jepsen.util 613 :invoke :add 6239 INFO [2022-01-10 08:39:30,105] jepsen worker 820 - jepsen.util 820 :invoke :add 6240 INFO [2022-01-10 08:39:30,107] jepsen worker 930 - jepsen.util 930 :invoke :add 6241 INFO [2022-01-10 08:39:30,108] jepsen worker 989 - jepsen.util 989 :invoke :add 6242 INFO [2022-01-10 08:39:30,110] jepsen worker 955 - jepsen.util 955 :invoke :add 6243 INFO [2022-01-10 08:39:30,112] jepsen worker 649 - jepsen.util 649 :invoke :add 6244 INFO [2022-01-10 08:39:30,113] jepsen worker 774 - jepsen.util 774 :invoke :add 6245 INFO [2022-01-10 08:39:30,115] jepsen worker 274 - jepsen.util 274 :invoke :add 6246 INFO [2022-01-10 08:39:30,117] jepsen worker 120 - jepsen.util 120 :invoke :add 6247 INFO [2022-01-10 08:39:30,119] jepsen worker 20 - jepsen.util 20 :invoke :add 6248 INFO [2022-01-10 08:39:30,121] jepsen worker 816 - jepsen.util 816 :invoke :add 6249 INFO [2022-01-10 08:39:30,123] jepsen worker 137 - jepsen.util 137 :invoke :add 6250 INFO [2022-01-10 08:39:30,125] jepsen worker 965 - jepsen.util 965 :invoke :add 6251 INFO [2022-01-10 08:39:30,127] jepsen worker 328 - jepsen.util 328 :invoke :add 6252 INFO [2022-01-10 08:39:30,128] jepsen worker 501 - jepsen.util 501 :invoke :add 6253 INFO [2022-01-10 08:39:30,130] jepsen worker 116 - jepsen.util 116 :invoke :add 6254 INFO [2022-01-10 08:39:30,132] jepsen worker 594 - jepsen.util 594 :invoke :add 6255 INFO [2022-01-10 08:39:30,134] jepsen worker 549 - jepsen.util 549 :invoke :add 6256 INFO [2022-01-10 08:39:30,136] jepsen worker 247 - jepsen.util 247 :invoke :add 6257 INFO [2022-01-10 08:39:30,137] jepsen worker 399 - jepsen.util 399 :invoke :add 6258 INFO [2022-01-10 08:39:30,139] jepsen worker 0 - jepsen.util 0 :invoke :add 6259 INFO [2022-01-10 08:39:30,141] jepsen worker 78 - jepsen.util 78 :invoke :add 6260 INFO [2022-01-10 08:39:30,142] jepsen worker 518 - jepsen.util 518 :invoke :add 6261 INFO [2022-01-10 08:39:30,144] jepsen worker 68 - jepsen.util 68 :invoke :add 6262 INFO [2022-01-10 08:39:30,146] jepsen worker 38 - jepsen.util 38 :invoke :add 6263 INFO [2022-01-10 08:39:30,147] jepsen worker 665 - jepsen.util 665 :invoke :add 6264 INFO [2022-01-10 08:39:30,149] jepsen worker 313 - jepsen.util 313 :invoke :add 6265 INFO [2022-01-10 08:39:30,151] jepsen worker 334 - jepsen.util 334 :invoke :add 6266 INFO [2022-01-10 08:39:30,152] jepsen worker 532 - jepsen.util 532 :invoke :add 6267 INFO [2022-01-10 08:39:30,154] jepsen worker 527 - jepsen.util 527 :invoke :add 6268 INFO [2022-01-10 08:39:30,156] jepsen worker 9 - jepsen.util 9 :invoke :add 6269 INFO [2022-01-10 08:39:30,158] jepsen worker 642 - jepsen.util 642 :invoke :add 6270 INFO [2022-01-10 08:39:30,160] jepsen worker 844 - jepsen.util 844 :invoke :add 6271 INFO [2022-01-10 08:39:30,161] jepsen worker 344 - jepsen.util 344 :invoke :add 6272 INFO [2022-01-10 08:39:30,163] jepsen worker 589 - jepsen.util 589 :invoke :add 6273 INFO [2022-01-10 08:39:30,165] jepsen worker 628 - jepsen.util 628 :invoke :add 6274 INFO [2022-01-10 08:39:30,167] jepsen worker 314 - jepsen.util 314 :invoke :add 6275 INFO [2022-01-10 08:39:30,169] jepsen worker 743 - jepsen.util 743 :invoke :add 6276 INFO [2022-01-10 08:39:30,170] jepsen worker 726 - jepsen.util 726 :invoke :add 6277 INFO [2022-01-10 08:39:30,173] jepsen worker 650 - jepsen.util 650 :invoke :add 6278 INFO [2022-01-10 08:39:30,174] jepsen worker 479 - jepsen.util 479 :invoke :add 6279 INFO [2022-01-10 08:39:30,176] jepsen worker 721 - jepsen.util 721 :invoke :add 6280 INFO [2022-01-10 08:39:30,178] jepsen worker 222 - jepsen.util 222 :invoke :add 6281 INFO [2022-01-10 08:39:30,179] jepsen worker 607 - jepsen.util 607 :invoke :add 6282 INFO [2022-01-10 08:39:30,181] jepsen worker 674 - jepsen.util 674 :invoke :add 6283 INFO [2022-01-10 08:39:30,185] jepsen worker 243 - jepsen.util 243 :invoke :add 6284 INFO [2022-01-10 08:39:30,318] jepsen worker 544 - jepsen.util 544 :ok :add 5956 INFO [2022-01-10 08:39:30,319] jepsen worker 214 - jepsen.util 214 :ok :add 5962 INFO [2022-01-10 08:39:30,319] jepsen worker 924 - jepsen.util 924 :ok :add 6049 INFO [2022-01-10 08:39:30,323] jepsen worker 214 - jepsen.util 214 :invoke :add 6285 INFO [2022-01-10 08:39:30,325] jepsen worker 544 - jepsen.util 544 :invoke :add 6286 INFO [2022-01-10 08:39:30,327] jepsen worker 924 - jepsen.util 924 :invoke :add 6287 INFO [2022-01-10 08:39:30,339] jepsen worker 917 - jepsen.util 917 :ok :add 4546 INFO [2022-01-10 08:39:30,339] jepsen worker 122 - jepsen.util 122 :ok :add 4882 INFO [2022-01-10 08:39:30,339] jepsen worker 903 - jepsen.util 903 :ok :add 4637 INFO [2022-01-10 08:39:30,339] jepsen worker 593 - jepsen.util 593 :ok :add 4603 INFO [2022-01-10 08:39:30,340] jepsen worker 766 - jepsen.util 766 :ok :add 4572 INFO [2022-01-10 08:39:30,344] jepsen worker 917 - jepsen.util 917 :invoke :add 6288 INFO [2022-01-10 08:39:30,346] jepsen worker 122 - jepsen.util 122 :invoke :add 6289 INFO [2022-01-10 08:39:30,348] jepsen worker 903 - jepsen.util 903 :invoke :add 6290 INFO [2022-01-10 08:39:30,350] jepsen worker 766 - jepsen.util 766 :invoke :add 6291 INFO [2022-01-10 08:39:30,352] jepsen worker 593 - jepsen.util 593 :invoke :add 6292 INFO [2022-01-10 08:39:30,480] jepsen worker 411 - jepsen.util 411 :ok :add 4391 INFO [2022-01-10 08:39:30,480] jepsen worker 439 - jepsen.util 439 :ok :add 4568 INFO [2022-01-10 08:39:30,480] jepsen worker 430 - jepsen.util 430 :ok :add 4570 INFO [2022-01-10 08:39:30,480] jepsen worker 468 - jepsen.util 468 :ok :add 4636 INFO [2022-01-10 08:39:30,480] jepsen worker 219 - jepsen.util 219 :ok :add 4619 INFO [2022-01-10 08:39:30,480] jepsen worker 463 - jepsen.util 463 :ok :add 4544 INFO [2022-01-10 08:39:30,481] jepsen worker 386 - jepsen.util 386 :ok :add 4547 INFO [2022-01-10 08:39:30,481] jepsen worker 268 - jepsen.util 268 :ok :add 4573 INFO [2022-01-10 08:39:30,481] jepsen worker 684 - jepsen.util 684 :ok :add 4601 INFO [2022-01-10 08:39:30,481] jepsen worker 85 - jepsen.util 85 :ok :add 4635 INFO [2022-01-10 08:39:30,481] jepsen worker 224 - jepsen.util 224 :ok :add 4602 INFO [2022-01-10 08:39:30,485] jepsen worker 439 - jepsen.util 439 :invoke :add 6293 INFO [2022-01-10 08:39:30,488] jepsen worker 463 - jepsen.util 463 :invoke :add 6294 INFO [2022-01-10 08:39:30,490] jepsen worker 268 - jepsen.util 268 :invoke :add 6295 INFO [2022-01-10 08:39:30,492] jepsen worker 411 - jepsen.util 411 :invoke :add 6296 INFO [2022-01-10 08:39:30,493] jepsen worker 684 - jepsen.util 684 :invoke :add 6297 INFO [2022-01-10 08:39:30,495] jepsen worker 468 - jepsen.util 468 :invoke :add 6298 INFO [2022-01-10 08:39:30,497] jepsen worker 219 - jepsen.util 219 :invoke :add 6299 INFO [2022-01-10 08:39:30,498] jepsen worker 224 - jepsen.util 224 :invoke :add 6300 INFO [2022-01-10 08:39:30,500] jepsen worker 85 - jepsen.util 85 :invoke :add 6301 INFO [2022-01-10 08:39:30,502] jepsen worker 386 - jepsen.util 386 :invoke :add 6302 INFO [2022-01-10 08:39:30,504] jepsen worker 430 - jepsen.util 430 :invoke :add 6303 INFO [2022-01-10 08:39:30,611] jepsen worker 437 - jepsen.util 437 :ok :add 5082 INFO [2022-01-10 08:39:30,612] jepsen worker 959 - jepsen.util 959 :ok :add 5112 INFO [2022-01-10 08:39:30,612] jepsen worker 41 - jepsen.util 41 :ok :add 4733 INFO [2022-01-10 08:39:30,612] jepsen worker 884 - jepsen.util 884 :ok :add 4986 INFO [2022-01-10 08:39:30,612] jepsen worker 132 - jepsen.util 132 :ok :add 5678 INFO [2022-01-10 08:39:30,612] jepsen worker 740 - jepsen.util 740 :ok :add 5499 INFO [2022-01-10 08:39:30,612] jepsen worker 351 - jepsen.util 351 :ok :add 5126 INFO [2022-01-10 08:39:30,612] jepsen worker 987 - jepsen.util 987 :ok :add 5525 INFO [2022-01-10 08:39:30,612] jepsen worker 379 - jepsen.util 379 :ok :add 4816 INFO [2022-01-10 08:39:30,612] jepsen worker 760 - jepsen.util 760 :ok :add 4822 INFO [2022-01-10 08:39:30,612] jepsen worker 193 - jepsen.util 193 :ok :add 5263 INFO [2022-01-10 08:39:30,615] jepsen worker 710 - jepsen.util 710 :ok :add 5257 INFO [2022-01-10 08:39:30,616] jepsen worker 959 - jepsen.util 959 :invoke :add 6304 INFO [2022-01-10 08:39:30,618] jepsen worker 884 - jepsen.util 884 :invoke :add 6305 INFO [2022-01-10 08:39:30,619] jepsen worker 740 - jepsen.util 740 :invoke :add 6306 INFO [2022-01-10 08:39:30,621] jepsen worker 379 - jepsen.util 379 :invoke :add 6307 INFO [2022-01-10 08:39:30,623] jepsen worker 710 - jepsen.util 710 :invoke :add 6308 INFO [2022-01-10 08:39:30,625] jepsen worker 760 - jepsen.util 760 :invoke :add 6309 INFO [2022-01-10 08:39:30,627] jepsen worker 351 - jepsen.util 351 :invoke :add 6310 INFO [2022-01-10 08:39:30,628] jepsen worker 193 - jepsen.util 193 :invoke :add 6311 INFO [2022-01-10 08:39:30,630] jepsen worker 437 - jepsen.util 437 :invoke :add 6312 INFO [2022-01-10 08:39:30,631] jepsen worker 512 - jepsen.util 512 :ok :add 5494 INFO [2022-01-10 08:39:30,631] jepsen worker 569 - jepsen.util 569 :ok :add 5504 INFO [2022-01-10 08:39:30,631] jepsen worker 481 - jepsen.util 481 :ok :add 5528 INFO [2022-01-10 08:39:30,632] jepsen worker 60 - jepsen.util 60 :ok :add 5659 INFO [2022-01-10 08:39:30,632] jepsen worker 625 - jepsen.util 625 :ok :add 5530 INFO [2022-01-10 08:39:30,632] jepsen worker 61 - jepsen.util 61 :ok :add 5641 INFO [2022-01-10 08:39:30,633] jepsen worker 132 - jepsen.util 132 :invoke :add 6313 INFO [2022-01-10 08:39:30,633] jepsen worker 925 - jepsen.util 925 :ok :add 5675 INFO [2022-01-10 08:39:30,633] jepsen worker 275 - jepsen.util 275 :ok :add 5964 INFO [2022-01-10 08:39:30,636] jepsen worker 569 - jepsen.util 569 :invoke :add 6314 INFO [2022-01-10 08:39:30,637] jepsen worker 481 - jepsen.util 481 :invoke :add 6315 INFO [2022-01-10 08:39:30,639] jepsen worker 61 - jepsen.util 61 :invoke :add 6316 INFO [2022-01-10 08:39:30,641] jepsen worker 60 - jepsen.util 60 :invoke :add 6317 INFO [2022-01-10 08:39:30,643] jepsen worker 512 - jepsen.util 512 :invoke :add 6318 INFO [2022-01-10 08:39:30,645] jepsen worker 41 - jepsen.util 41 :invoke :add 6319 INFO [2022-01-10 08:39:30,646] jepsen worker 925 - jepsen.util 925 :invoke :add 6320 INFO [2022-01-10 08:39:30,648] jepsen worker 987 - jepsen.util 987 :invoke :add 6321 INFO [2022-01-10 08:39:30,650] jepsen worker 275 - jepsen.util 275 :invoke :add 6322 INFO [2022-01-10 08:39:30,652] jepsen worker 625 - jepsen.util 625 :invoke :add 6323 INFO [2022-01-10 08:39:30,724] jepsen worker 362 - jepsen.util 362 :ok :add 4489 INFO [2022-01-10 08:39:30,724] jepsen worker 753 - jepsen.util 753 :ok :add 4535 INFO [2022-01-10 08:39:30,724] jepsen worker 315 - jepsen.util 315 :ok :add 4519 INFO [2022-01-10 08:39:30,725] jepsen worker 730 - jepsen.util 730 :ok :add 4491 INFO [2022-01-10 08:39:30,725] jepsen worker 651 - jepsen.util 651 :ok :add 4670 INFO [2022-01-10 08:39:30,725] jepsen worker 504 - jepsen.util 504 :ok :add 4501 INFO [2022-01-10 08:39:30,725] jepsen worker 233 - jepsen.util 233 :ok :add 4644 INFO [2022-01-10 08:39:30,725] jepsen worker 467 - jepsen.util 467 :ok :add 4668 INFO [2022-01-10 08:39:30,729] jepsen worker 315 - jepsen.util 315 :invoke :add 6324 INFO [2022-01-10 08:39:30,731] jepsen worker 730 - jepsen.util 730 :invoke :add 6325 INFO [2022-01-10 08:39:30,733] jepsen worker 504 - jepsen.util 504 :invoke :add 6326 INFO [2022-01-10 08:39:30,735] jepsen worker 362 - jepsen.util 362 :invoke :add 6327 INFO [2022-01-10 08:39:30,737] jepsen worker 467 - jepsen.util 467 :invoke :add 6328 INFO [2022-01-10 08:39:30,739] jepsen worker 233 - jepsen.util 233 :invoke :add 6329 INFO [2022-01-10 08:39:30,741] jepsen worker 753 - jepsen.util 753 :invoke :add 6330 INFO [2022-01-10 08:39:30,743] jepsen worker 651 - jepsen.util 651 :invoke :add 6331 INFO [2022-01-10 08:39:30,743] jepsen worker 803 - jepsen.util 803 :ok :add 5542 INFO [2022-01-10 08:39:30,744] jepsen worker 8 - jepsen.util 8 :ok :add 5576 INFO [2022-01-10 08:39:30,744] jepsen worker 356 - jepsen.util 356 :ok :add 5607 INFO [2022-01-10 08:39:30,744] jepsen worker 185 - jepsen.util 185 :ok :add 5633 INFO [2022-01-10 08:39:30,744] jepsen worker 644 - jepsen.util 644 :ok :add 5797 INFO [2022-01-10 08:39:30,748] jepsen worker 644 - jepsen.util 644 :invoke :add 6332 INFO [2022-01-10 08:39:30,750] jepsen worker 185 - jepsen.util 185 :invoke :add 6333 INFO [2022-01-10 08:39:30,752] jepsen worker 356 - jepsen.util 356 :invoke :add 6334 INFO [2022-01-10 08:39:30,754] jepsen worker 803 - jepsen.util 803 :invoke :add 6335 INFO [2022-01-10 08:39:30,756] jepsen worker 8 - jepsen.util 8 :invoke :add 6336 INFO [2022-01-10 08:39:30,812] jepsen worker 470 - jepsen.util 470 :ok :add 6053 INFO [2022-01-10 08:39:30,812] jepsen worker 303 - jepsen.util 303 :ok :add 5954 INFO [2022-01-10 08:39:30,815] jepsen worker 646 - jepsen.util 646 :ok :add 5960 INFO [2022-01-10 08:39:30,815] jepsen worker 961 - jepsen.util 961 :ok :add 5978 INFO [2022-01-10 08:39:30,815] jepsen worker 153 - jepsen.util 153 :ok :add 6067 INFO [2022-01-10 08:39:30,816] jepsen worker 470 - jepsen.util 470 :invoke :add 6337 INFO [2022-01-10 08:39:30,818] jepsen worker 303 - jepsen.util 303 :invoke :add 6338 INFO [2022-01-10 08:39:30,820] jepsen worker 961 - jepsen.util 961 :invoke :add 6339 INFO [2022-01-10 08:39:30,822] jepsen worker 153 - jepsen.util 153 :invoke :add 6340 INFO [2022-01-10 08:39:30,824] jepsen worker 646 - jepsen.util 646 :invoke :add 6341 INFO [2022-01-10 08:39:31,095] jepsen worker 629 - jepsen.util 629 :ok :add 4545 INFO [2022-01-10 08:39:31,096] jepsen worker 958 - jepsen.util 958 :ok :add 4790 INFO [2022-01-10 08:39:31,096] jepsen worker 176 - jepsen.util 176 :ok :add 4634 INFO [2022-01-10 08:39:31,096] jepsen worker 854 - jepsen.util 854 :ok :add 4600 INFO [2022-01-10 08:39:31,096] jepsen worker 680 - jepsen.util 680 :ok :add 4881 INFO [2022-01-10 08:39:31,096] jepsen worker 374 - jepsen.util 374 :ok :add 4788 INFO [2022-01-10 08:39:31,096] jepsen worker 810 - jepsen.util 810 :ok :add 4569 INFO [2022-01-10 08:39:31,096] jepsen worker 807 - jepsen.util 807 :ok :add 4618 INFO [2022-01-10 08:39:31,096] jepsen worker 982 - jepsen.util 982 :ok :add 4571 INFO [2022-01-10 08:39:31,100] jepsen worker 854 - jepsen.util 854 :invoke :add 6342 INFO [2022-01-10 08:39:31,101] jepsen worker 958 - jepsen.util 958 :invoke :add 6343 INFO [2022-01-10 08:39:31,103] jepsen worker 629 - jepsen.util 629 :invoke :add 6344 INFO [2022-01-10 08:39:31,105] jepsen worker 680 - jepsen.util 680 :invoke :add 6345 INFO [2022-01-10 08:39:31,106] jepsen worker 810 - jepsen.util 810 :invoke :add 6346 INFO [2022-01-10 08:39:31,109] jepsen worker 374 - jepsen.util 374 :invoke :add 6347 INFO [2022-01-10 08:39:31,111] jepsen worker 807 - jepsen.util 807 :invoke :add 6348 INFO [2022-01-10 08:39:31,113] jepsen worker 176 - jepsen.util 176 :invoke :add 6349 INFO [2022-01-10 08:39:31,115] jepsen worker 982 - jepsen.util 982 :invoke :add 6350 INFO [2022-01-10 08:39:31,207] jepsen worker 119 - jepsen.util 119 :ok :add 5938 INFO [2022-01-10 08:39:31,207] jepsen worker 882 - jepsen.util 882 :ok :add 5629 INFO [2022-01-10 08:39:31,207] jepsen worker 581 - jepsen.util 581 :ok :add 5605 INFO [2022-01-10 08:39:31,207] jepsen worker 420 - jepsen.util 420 :ok :add 5795 INFO [2022-01-10 08:39:31,209] jepsen worker 610 - jepsen.util 610 :ok :add 5631 INFO [2022-01-10 08:39:31,209] jepsen worker 882 - jepsen.util 882 :invoke :add 6351 INFO [2022-01-10 08:39:31,211] jepsen worker 610 - jepsen.util 610 :invoke :add 6352 INFO [2022-01-10 08:39:31,213] jepsen worker 119 - jepsen.util 119 :invoke :add 6353 INFO [2022-01-10 08:39:31,215] jepsen worker 581 - jepsen.util 581 :invoke :add 6354 INFO [2022-01-10 08:39:31,217] jepsen worker 420 - jepsen.util 420 :invoke :add 6355 INFO [2022-01-10 08:39:31,440] jepsen worker 343 - jepsen.util 343 :ok :add 5982 INFO [2022-01-10 08:39:31,440] jepsen worker 887 - jepsen.util 887 :ok :add 6115 INFO [2022-01-10 08:39:31,440] jepsen worker 172 - jepsen.util 172 :ok :add 6085 INFO [2022-01-10 08:39:31,440] jepsen worker 372 - jepsen.util 372 :ok :add 6121 INFO [2022-01-10 08:39:31,441] jepsen worker 45 - jepsen.util 45 :ok :add 6139 INFO [2022-01-10 08:39:31,441] jepsen worker 137 - jepsen.util 137 :ok :add 6250 INFO [2022-01-10 08:39:31,441] jepsen worker 665 - jepsen.util 665 :ok :add 6264 INFO [2022-01-10 08:39:31,441] jepsen worker 20 - jepsen.util 20 :ok :add 6248 INFO [2022-01-10 08:39:31,443] jepsen worker 137 - jepsen.util 137 :invoke :add 6356 INFO [2022-01-10 08:39:31,445] jepsen worker 343 - jepsen.util 343 :invoke :add 6357 INFO [2022-01-10 08:39:31,447] jepsen worker 45 - jepsen.util 45 :invoke :add 6358 INFO [2022-01-10 08:39:31,449] jepsen worker 887 - jepsen.util 887 :invoke :add 6359 INFO [2022-01-10 08:39:31,451] jepsen worker 665 - jepsen.util 665 :invoke :add 6360 INFO [2022-01-10 08:39:31,452] jepsen worker 172 - jepsen.util 172 :invoke :add 6361 INFO [2022-01-10 08:39:31,454] jepsen worker 20 - jepsen.util 20 :invoke :add 6362 INFO [2022-01-10 08:39:31,456] jepsen worker 372 - jepsen.util 372 :invoke :add 6363 INFO [2022-01-10 08:39:32,087] jepsen worker 952 - jepsen.util 952 :ok :add 5733 INFO [2022-01-10 08:39:32,088] jepsen worker 154 - jepsen.util 154 :ok :add 5822 INFO [2022-01-10 08:39:32,087] jepsen worker 246 - jepsen.util 246 :ok :add 5697 INFO [2022-01-10 08:39:32,087] jepsen worker 749 - jepsen.util 749 :ok :add 5707 INFO [2022-01-10 08:39:32,089] jepsen worker 643 - jepsen.util 643 :ok :add 5816 INFO [2022-01-10 08:39:32,089] jepsen worker 842 - jepsen.util 842 :ok :add 5986 INFO [2022-01-10 08:39:32,090] jepsen worker 154 - jepsen.util 154 :invoke :add 6364 INFO [2022-01-10 08:39:32,094] jepsen worker 749 - jepsen.util 749 :invoke :add 6365 INFO [2022-01-10 08:39:32,096] jepsen worker 842 - jepsen.util 842 :invoke :add 6366 INFO [2022-01-10 08:39:32,098] jepsen worker 643 - jepsen.util 643 :invoke :add 6367 INFO [2022-01-10 08:39:32,101] jepsen worker 246 - jepsen.util 246 :invoke :add 6368 INFO [2022-01-10 08:39:32,103] jepsen worker 952 - jepsen.util 952 :invoke :add 6369 INFO [2022-01-10 08:39:32,327] jepsen worker 462 - jepsen.util 462 :ok :add 5827 INFO [2022-01-10 08:39:32,327] jepsen worker 198 - jepsen.util 198 :ok :add 6120 INFO [2022-01-10 08:39:32,327] jepsen worker 849 - jepsen.util 849 :ok :add 6114 INFO [2022-01-10 08:39:32,327] jepsen worker 307 - jepsen.util 307 :ok :add 6138 INFO [2022-01-10 08:39:32,327] jepsen worker 658 - jepsen.util 658 :ok :add 5983 INFO [2022-01-10 08:39:32,327] jepsen worker 313 - jepsen.util 313 :ok :add 6265 INFO [2022-01-10 08:39:32,327] jepsen worker 354 - jepsen.util 354 :ok :add 6084 INFO [2022-01-10 08:39:32,327] jepsen worker 816 - jepsen.util 816 :ok :add 6249 INFO [2022-01-10 08:39:32,327] jepsen worker 965 - jepsen.util 965 :ok :add 6251 INFO [2022-01-10 08:39:32,329] jepsen worker 382 - jepsen.util 382 :ok :add 6087 INFO [2022-01-10 08:39:32,329] jepsen worker 334 - jepsen.util 334 :ok :add 6266 INFO [2022-01-10 08:39:32,330] jepsen worker 533 - jepsen.util 533 :ok :add 5998 INFO [2022-01-10 08:39:32,330] jepsen worker 328 - jepsen.util 328 :ok :add 6252 INFO [2022-01-10 08:39:32,330] jepsen worker 781 - jepsen.util 781 :ok :add 6123 INFO [2022-01-10 08:39:32,330] jepsen worker 719 - jepsen.util 719 :ok :add 6117 INFO [2022-01-10 08:39:32,330] jepsen worker 401 - jepsen.util 401 :ok :add 5810 INFO [2022-01-10 08:39:32,331] jepsen worker 596 - jepsen.util 596 :ok :add 5980 INFO [2022-01-10 08:39:32,331] jepsen worker 462 - jepsen.util 462 :invoke :add 6370 INFO [2022-01-10 08:39:32,334] jepsen worker 849 - jepsen.util 849 :invoke :add 6371 INFO [2022-01-10 08:39:32,336] jepsen worker 658 - jepsen.util 658 :invoke :add 6372 INFO [2022-01-10 08:39:32,338] jepsen worker 328 - jepsen.util 328 :invoke :add 6373 INFO [2022-01-10 08:39:32,339] jepsen worker 596 - jepsen.util 596 :invoke :add 6374 INFO [2022-01-10 08:39:32,341] jepsen worker 307 - jepsen.util 307 :invoke :add 6375 INFO [2022-01-10 08:39:32,343] jepsen worker 198 - jepsen.util 198 :invoke :add 6376 INFO [2022-01-10 08:39:32,345] jepsen worker 781 - jepsen.util 781 :invoke :add 6377 INFO [2022-01-10 08:39:32,347] jepsen worker 816 - jepsen.util 816 :invoke :add 6378 INFO [2022-01-10 08:39:32,349] jepsen worker 533 - jepsen.util 533 :invoke :add 6379 INFO [2022-01-10 08:39:32,351] jepsen worker 334 - jepsen.util 334 :invoke :add 6380 INFO [2022-01-10 08:39:32,353] jepsen worker 965 - jepsen.util 965 :invoke :add 6381 INFO [2022-01-10 08:39:32,355] jepsen worker 719 - jepsen.util 719 :invoke :add 6382 INFO [2022-01-10 08:39:32,356] jepsen worker 313 - jepsen.util 313 :invoke :add 6383 INFO [2022-01-10 08:39:32,358] jepsen worker 382 - jepsen.util 382 :invoke :add 6384 INFO [2022-01-10 08:39:32,359] jepsen worker 354 - jepsen.util 354 :invoke :add 6385 INFO [2022-01-10 08:39:32,361] jepsen worker 401 - jepsen.util 401 :invoke :add 6386 INFO [2022-01-10 08:39:32,443] jepsen worker 532 - jepsen.util 532 :ok :add 6267 INFO [2022-01-10 08:39:32,443] jepsen worker 501 - jepsen.util 501 :ok :add 6253 INFO [2022-01-10 08:39:32,444] jepsen worker 70 - jepsen.util 70 :ok :add 5981 INFO [2022-01-10 08:39:32,445] jepsen worker 453 - jepsen.util 453 :ok :add 6086 INFO [2022-01-10 08:39:32,445] jepsen worker 559 - jepsen.util 559 :ok :add 5999 INFO [2022-01-10 08:39:32,445] jepsen worker 3 - jepsen.util 3 :ok :add 6116 INFO [2022-01-10 08:39:32,445] jepsen worker 709 - jepsen.util 709 :ok :add 6122 INFO [2022-01-10 08:39:32,446] jepsen worker 501 - jepsen.util 501 :invoke :add 6387 INFO [2022-01-10 08:39:32,448] jepsen worker 559 - jepsen.util 559 :invoke :add 6388 INFO [2022-01-10 08:39:32,449] jepsen worker 481 - jepsen.util 481 :ok :add 6315 INFO [2022-01-10 08:39:32,449] jepsen worker 579 - jepsen.util 579 :ok :add 5596 INFO [2022-01-10 08:39:32,449] jepsen worker 115 - jepsen.util 115 :ok :add 6050 INFO [2022-01-10 08:39:32,449] jepsen worker 664 - jepsen.util 664 :ok :add 5743 INFO [2022-01-10 08:39:32,449] jepsen worker 961 - jepsen.util 961 :ok :add 6339 INFO [2022-01-10 08:39:32,450] jepsen worker 852 - jepsen.util 852 :ok :add 5852 INFO [2022-01-10 08:39:32,450] jepsen worker 206 - jepsen.util 206 :ok :add 5963 INFO [2022-01-10 08:39:32,450] jepsen worker 796 - jepsen.util 796 :ok :add 5777 INFO [2022-01-10 08:39:32,450] jepsen worker 709 - jepsen.util 709 :invoke :add 6389 INFO [2022-01-10 08:39:32,450] jepsen worker 461 - jepsen.util 461 :ok :add 5957 INFO [2022-01-10 08:39:32,450] jepsen worker 636 - jepsen.util 636 :ok :add 6048 INFO [2022-01-10 08:39:32,450] jepsen worker 466 - jepsen.util 466 :ok :add 6064 INFO [2022-01-10 08:39:32,450] jepsen worker 987 - jepsen.util 987 :ok :add 6321 INFO [2022-01-10 08:39:32,450] jepsen worker 214 - jepsen.util 214 :ok :add 6285 INFO [2022-01-10 08:39:32,455] jepsen worker 481 - jepsen.util 481 :invoke :add 6390 INFO [2022-01-10 08:39:32,457] jepsen worker 636 - jepsen.util 636 :invoke :add 6391 INFO [2022-01-10 08:39:32,460] jepsen worker 115 - jepsen.util 115 :invoke :add 6392 INFO [2022-01-10 08:39:32,462] jepsen worker 461 - jepsen.util 461 :invoke :add 6393 INFO [2022-01-10 08:39:32,464] jepsen worker 532 - jepsen.util 532 :invoke :add 6394 INFO [2022-01-10 08:39:32,465] jepsen worker 961 - jepsen.util 961 :invoke :add 6395 INFO [2022-01-10 08:39:32,467] jepsen worker 466 - jepsen.util 466 :invoke :add 6396 INFO [2022-01-10 08:39:32,468] jepsen worker 70 - jepsen.util 70 :invoke :add 6397 INFO [2022-01-10 08:39:32,470] jepsen worker 987 - jepsen.util 987 :invoke :add 6398 INFO [2022-01-10 08:39:32,472] jepsen worker 447 - jepsen.util 447 :ok :add 4693 INFO [2022-01-10 08:39:32,473] jepsen worker 368 - jepsen.util 368 :ok :add 4703 INFO [2022-01-10 08:39:32,473] jepsen worker 11 - jepsen.util 11 :ok :add 5086 INFO [2022-01-10 08:39:32,473] jepsen worker 214 - jepsen.util 214 :invoke :add 6399 INFO [2022-01-10 08:39:32,474] jepsen worker 286 - jepsen.util 286 :ok :add 4737 INFO [2022-01-10 08:39:32,474] jepsen worker 49 - jepsen.util 49 :ok :add 4826 INFO [2022-01-10 08:39:32,476] jepsen worker 11 - jepsen.util 11 :invoke :add 6400 INFO [2022-01-10 08:39:32,479] jepsen worker 852 - jepsen.util 852 :invoke :add 6401 INFO [2022-01-10 08:39:32,481] jepsen worker 453 - jepsen.util 453 :invoke :add 6402 INFO [2022-01-10 08:39:32,482] jepsen worker 664 - jepsen.util 664 :invoke :add 6403 INFO [2022-01-10 08:39:32,484] jepsen worker 49 - jepsen.util 49 :invoke :add 6404 INFO [2022-01-10 08:39:32,486] jepsen worker 3 - jepsen.util 3 :invoke :add 6405 INFO [2022-01-10 08:39:32,488] jepsen worker 206 - jepsen.util 206 :invoke :add 6406 INFO [2022-01-10 08:39:32,489] jepsen worker 286 - jepsen.util 286 :invoke :add 6407 INFO [2022-01-10 08:39:32,491] jepsen worker 368 - jepsen.util 368 :invoke :add 6408 INFO [2022-01-10 08:39:32,493] jepsen worker 447 - jepsen.util 447 :invoke :add 6409 INFO [2022-01-10 08:39:32,494] jepsen worker 796 - jepsen.util 796 :invoke :add 6410 INFO [2022-01-10 08:39:32,496] jepsen worker 579 - jepsen.util 579 :invoke :add 6411 INFO [2022-01-10 08:39:32,735] jepsen worker 738 - jepsen.util 738 :ok :add 5955 INFO [2022-01-10 08:39:32,735] jepsen worker 121 - jepsen.util 121 :ok :add 6052 INFO [2022-01-10 08:39:32,735] jepsen worker 924 - jepsen.util 924 :ok :add 6287 INFO [2022-01-10 08:39:32,735] jepsen worker 625 - jepsen.util 625 :ok :add 6323 INFO [2022-01-10 08:39:32,735] jepsen worker 60 - jepsen.util 60 :ok :add 6317 INFO [2022-01-10 08:39:32,738] jepsen worker 176 - jepsen.util 176 :ok :add 6349 INFO [2022-01-10 08:39:32,738] jepsen worker 738 - jepsen.util 738 :invoke :add 6412 INFO [2022-01-10 08:39:32,738] jepsen worker 968 - jepsen.util 968 :ok :add 6184 INFO [2022-01-10 08:39:32,739] jepsen worker 745 - jepsen.util 745 :ok :add 6038 INFO [2022-01-10 08:39:32,739] jepsen worker 882 - jepsen.util 882 :ok :add 6351 INFO [2022-01-10 08:39:32,739] jepsen worker 32 - jepsen.util 32 :ok :add 5883 INFO [2022-01-10 08:39:32,739] jepsen worker 624 - jepsen.util 624 :ok :add 6014 INFO [2022-01-10 08:39:32,739] jepsen worker 749 - jepsen.util 749 :ok :add 6365 INFO [2022-01-10 08:39:32,739] jepsen worker 725 - jepsen.util 725 :ok :add 6020 INFO [2022-01-10 08:39:32,739] jepsen worker 86 - jepsen.util 86 :ok :add 5927 INFO [2022-01-10 08:39:32,740] jepsen worker 924 - jepsen.util 924 :invoke :add 6413 INFO [2022-01-10 08:39:32,744] jepsen worker 121 - jepsen.util 121 :invoke :add 6414 INFO [2022-01-10 08:39:32,745] jepsen worker 749 - jepsen.util 749 :invoke :add 6415 INFO [2022-01-10 08:39:32,747] jepsen worker 86 - jepsen.util 86 :invoke :add 6416 INFO [2022-01-10 08:39:32,749] jepsen worker 32 - jepsen.util 32 :invoke :add 6417 INFO [2022-01-10 08:39:32,751] jepsen worker 625 - jepsen.util 625 :invoke :add 6418 INFO [2022-01-10 08:39:32,752] jepsen worker 361 - jepsen.util 361 :ok :add 5705 INFO [2022-01-10 08:39:32,752] jepsen worker 687 - jepsen.util 687 :ok :add 5838 INFO [2022-01-10 08:39:32,752] jepsen worker 248 - jepsen.util 248 :ok :add 5814 INFO [2022-01-10 08:39:32,752] jepsen worker 203 - jepsen.util 203 :ok :add 5820 INFO [2022-01-10 08:39:32,752] jepsen worker 563 - jepsen.util 563 :ok :add 5731 INFO [2022-01-10 08:39:32,752] jepsen worker 725 - jepsen.util 725 :invoke :add 6419 INFO [2022-01-10 08:39:32,753] jepsen worker 702 - jepsen.util 702 :ok :add 5695 INFO [2022-01-10 08:39:32,753] jepsen worker 867 - jepsen.util 867 :ok :add 5729 INFO [2022-01-10 08:39:32,755] jepsen worker 702 - jepsen.util 702 :invoke :add 6420 INFO [2022-01-10 08:39:32,757] jepsen worker 968 - jepsen.util 968 :invoke :add 6421 INFO [2022-01-10 08:39:32,759] jepsen worker 563 - jepsen.util 563 :invoke :add 6422 INFO [2022-01-10 08:39:32,761] jepsen worker 745 - jepsen.util 745 :invoke :add 6423 INFO [2022-01-10 08:39:32,763] jepsen worker 60 - jepsen.util 60 :invoke :add 6424 INFO [2022-01-10 08:39:32,764] jepsen worker 84 - jepsen.util 84 :ok :add 4701 INFO [2022-01-10 08:39:32,765] jepsen worker 402 - jepsen.util 402 :ok :add 5084 INFO [2022-01-10 08:39:32,765] jepsen worker 203 - jepsen.util 203 :invoke :add 6425 INFO [2022-01-10 08:39:32,766] jepsen worker 174 - jepsen.util 174 :ok :add 4719 INFO [2022-01-10 08:39:32,766] jepsen worker 173 - jepsen.util 173 :ok :add 4735 INFO [2022-01-10 08:39:32,766] jepsen worker 592 - jepsen.util 592 :ok :add 4808 INFO [2022-01-10 08:39:32,768] jepsen worker 687 - jepsen.util 687 :invoke :add 6426 INFO [2022-01-10 08:39:32,770] jepsen worker 592 - jepsen.util 592 :invoke :add 6427 INFO [2022-01-10 08:39:32,773] jepsen worker 176 - jepsen.util 176 :invoke :add 6428 INFO [2022-01-10 08:39:32,775] jepsen worker 867 - jepsen.util 867 :invoke :add 6429 INFO [2022-01-10 08:39:32,777] jepsen worker 361 - jepsen.util 361 :invoke :add 6430 INFO [2022-01-10 08:39:32,779] jepsen worker 84 - jepsen.util 84 :invoke :add 6431 INFO [2022-01-10 08:39:32,781] jepsen worker 248 - jepsen.util 248 :invoke :add 6432 INFO [2022-01-10 08:39:32,783] jepsen worker 402 - jepsen.util 402 :invoke :add 6433 INFO [2022-01-10 08:39:32,785] jepsen worker 882 - jepsen.util 882 :invoke :add 6434 INFO [2022-01-10 08:39:32,787] jepsen worker 173 - jepsen.util 173 :invoke :add 6435 INFO [2022-01-10 08:39:32,789] jepsen worker 174 - jepsen.util 174 :invoke :add 6436 INFO [2022-01-10 08:39:32,791] jepsen worker 624 - jepsen.util 624 :invoke :add 6437 INFO [2022-01-10 08:39:32,926] jepsen worker 66 - jepsen.util 66 :ok :add 5881 INFO [2022-01-10 08:39:32,926] jepsen worker 727 - jepsen.util 727 :ok :add 5925 INFO [2022-01-10 08:39:32,926] jepsen worker 119 - jepsen.util 119 :ok :add 6353 INFO [2022-01-10 08:39:32,926] jepsen worker 995 - jepsen.util 995 :ok :add 6016 INFO [2022-01-10 08:39:32,926] jepsen worker 601 - jepsen.util 601 :ok :add 6186 INFO [2022-01-10 08:39:32,926] jepsen worker 643 - jepsen.util 643 :ok :add 6367 INFO [2022-01-10 08:39:32,929] jepsen worker 66 - jepsen.util 66 :invoke :add 6438 INFO [2022-01-10 08:39:32,930] jepsen worker 366 - jepsen.util 366 :ok :add 6185 INFO [2022-01-10 08:39:32,930] jepsen worker 982 - jepsen.util 982 :ok :add 6350 INFO [2022-01-10 08:39:32,930] jepsen worker 970 - jepsen.util 970 :ok :add 5882 INFO [2022-01-10 08:39:32,930] jepsen worker 747 - jepsen.util 747 :ok :add 5926 INFO [2022-01-10 08:39:32,931] jepsen worker 807 - jepsen.util 807 :ok :add 6348 INFO [2022-01-10 08:39:32,931] jepsen worker 154 - jepsen.util 154 :ok :add 6364 INFO [2022-01-10 08:39:32,932] jepsen worker 995 - jepsen.util 995 :invoke :add 6439 INFO [2022-01-10 08:39:32,934] jepsen worker 982 - jepsen.util 982 :invoke :add 6440 INFO [2022-01-10 08:39:32,936] jepsen worker 366 - jepsen.util 366 :invoke :add 6441 INFO [2022-01-10 08:39:32,938] jepsen worker 643 - jepsen.util 643 :invoke :add 6442 INFO [2022-01-10 08:39:32,939] jepsen worker 747 - jepsen.util 747 :invoke :add 6443 INFO [2022-01-10 08:39:32,941] jepsen worker 970 - jepsen.util 970 :invoke :add 6444 INFO [2022-01-10 08:39:32,943] jepsen worker 154 - jepsen.util 154 :invoke :add 6445 INFO [2022-01-10 08:39:32,945] jepsen worker 727 - jepsen.util 727 :invoke :add 6446 INFO [2022-01-10 08:39:32,947] jepsen worker 119 - jepsen.util 119 :invoke :add 6447 INFO [2022-01-10 08:39:32,949] jepsen worker 520 - jepsen.util 520 :ok :add 5894 INFO [2022-01-10 08:39:32,949] jepsen worker 807 - jepsen.util 807 :invoke :add 6448 INFO [2022-01-10 08:39:32,952] jepsen worker 5 - jepsen.util 5 :ok :add 4692 INFO [2022-01-10 08:39:32,952] jepsen worker 110 - jepsen.util 110 :ok :add 4827 INFO [2022-01-10 08:39:32,952] jepsen worker 520 - jepsen.util 520 :invoke :add 6449 INFO [2022-01-10 08:39:32,952] jepsen worker 256 - jepsen.util 256 :ok :add 5087 INFO [2022-01-10 08:39:32,952] jepsen worker 857 - jepsen.util 857 :ok :add 4736 INFO [2022-01-10 08:39:32,956] jepsen worker 601 - jepsen.util 601 :invoke :add 6450 INFO [2022-01-10 08:39:32,957] jepsen worker 216 - jepsen.util 216 :ok :add 4702 INFO [2022-01-10 08:39:32,960] jepsen worker 256 - jepsen.util 256 :invoke :add 6451 INFO [2022-01-10 08:39:32,963] jepsen worker 110 - jepsen.util 110 :invoke :add 6452 INFO [2022-01-10 08:39:32,967] jepsen worker 216 - jepsen.util 216 :invoke :add 6453 INFO [2022-01-10 08:39:32,969] jepsen worker 857 - jepsen.util 857 :invoke :add 6454 INFO [2022-01-10 08:39:32,971] jepsen worker 5 - jepsen.util 5 :invoke :add 6455 INFO [2022-01-10 08:39:32,972] jepsen worker 863 - jepsen.util 863 :ok :add 6017 INFO [2022-01-10 08:39:32,972] jepsen worker 121 - jepsen.util 121 :ok :add 6414 INFO [2022-01-10 08:39:32,972] jepsen worker 300 - jepsen.util 300 :ok :add 5908 INFO [2022-01-10 08:39:32,972] jepsen worker 976 - jepsen.util 976 :ok :add 6187 INFO [2022-01-10 08:39:32,972] jepsen worker 842 - jepsen.util 842 :ok :add 6366 INFO [2022-01-10 08:39:32,973] jepsen worker 325 - jepsen.util 325 :ok :add 5856 INFO [2022-01-10 08:39:32,973] jepsen worker 516 - jepsen.util 516 :ok :add 5741 INFO [2022-01-10 08:39:32,974] jepsen worker 498 - jepsen.util 498 :ok :add 5594 INFO [2022-01-10 08:39:32,974] jepsen worker 124 - jepsen.util 124 :ok :add 5759 INFO [2022-01-10 08:39:32,974] jepsen worker 606 - jepsen.util 606 :ok :add 5775 INFO [2022-01-10 08:39:32,974] jepsen worker 851 - jepsen.util 851 :ok :add 5848 INFO [2022-01-10 08:39:32,976] jepsen worker 177 - jepsen.util 177 :ok :add 5742 INFO [2022-01-10 08:39:32,976] jepsen worker 118 - jepsen.util 118 :ok :add 5776 INFO [2022-01-10 08:39:32,977] jepsen worker 702 - jepsen.util 702 :ok :add 6420 INFO [2022-01-10 08:39:32,977] jepsen worker 891 - jepsen.util 891 :ok :add 6023 INFO [2022-01-10 08:39:32,976] jepsen worker 610 - jepsen.util 610 :ok :add 6352 INFO [2022-01-10 08:39:32,977] jepsen worker 977 - jepsen.util 977 :ok :add 6154 INFO [2022-01-10 08:39:32,977] jepsen worker 746 - jepsen.util 746 :ok :add 6151 INFO [2022-01-10 08:39:32,978] jepsen worker 542 - jepsen.util 542 :ok :add 6214 INFO [2022-01-10 08:39:32,978] jepsen worker 242 - jepsen.util 242 :ok :add 6149 INFO [2022-01-10 08:39:32,978] jepsen worker 57 - jepsen.util 57 :ok :add 6165 INFO [2022-01-10 08:39:32,979] jepsen worker 841 - jepsen.util 841 :ok :add 5853 INFO [2022-01-10 08:39:32,979] jepsen worker 842 - jepsen.util 842 :invoke :add 6456 INFO [2022-01-10 08:39:32,980] jepsen worker 582 - jepsen.util 582 :ok :add 6220 INFO [2022-01-10 08:39:32,980] jepsen worker 824 - jepsen.util 824 :ok :add 6238 INFO [2022-01-10 08:39:32,980] jepsen worker 382 - jepsen.util 382 :ok :add 6384 INFO [2022-01-10 08:39:32,985] jepsen worker 325 - jepsen.util 325 :invoke :add 6457 INFO [2022-01-10 08:39:32,988] jepsen worker 124 - jepsen.util 124 :invoke :add 6458 INFO [2022-01-10 08:39:32,990] jepsen worker 863 - jepsen.util 863 :invoke :add 6459 INFO [2022-01-10 08:39:32,992] jepsen worker 977 - jepsen.util 977 :invoke :add 6460 INFO [2022-01-10 08:39:32,995] jepsen worker 57 - jepsen.util 57 :invoke :add 6461 INFO [2022-01-10 08:39:32,998] jepsen worker 841 - jepsen.util 841 :invoke :add 6462 INFO [2022-01-10 08:39:33,000] jepsen worker 242 - jepsen.util 242 :invoke :add 6463 INFO [2022-01-10 08:39:33,001] jepsen worker 702 - jepsen.util 702 :invoke :add 6464 INFO [2022-01-10 08:39:33,004] jepsen worker 582 - jepsen.util 582 :invoke :add 6465 INFO [2022-01-10 08:39:33,006] jepsen worker 177 - jepsen.util 177 :invoke :add 6466 INFO [2022-01-10 08:39:33,008] jepsen worker 382 - jepsen.util 382 :invoke :add 6467 INFO [2022-01-10 08:39:33,010] jepsen worker 976 - jepsen.util 976 :invoke :add 6468 INFO [2022-01-10 08:39:33,013] jepsen worker 610 - jepsen.util 610 :invoke :add 6469 INFO [2022-01-10 08:39:33,016] jepsen worker 542 - jepsen.util 542 :invoke :add 6470 INFO [2022-01-10 08:39:33,018] jepsen worker 121 - jepsen.util 121 :invoke :add 6471 INFO [2022-01-10 08:39:33,020] jepsen worker 118 - jepsen.util 118 :invoke :add 6472 INFO [2022-01-10 08:39:33,021] jepsen worker 301 - jepsen.util 301 :ok :add 4850 INFO [2022-01-10 08:39:33,021] jepsen worker 340 - jepsen.util 340 :ok :add 5211 INFO [2022-01-10 08:39:33,022] jepsen worker 851 - jepsen.util 851 :invoke :add 6473 INFO [2022-01-10 08:39:33,022] jepsen worker 519 - jepsen.util 519 :ok :add 4775 INFO [2022-01-10 08:39:33,022] jepsen worker 813 - jepsen.util 813 :ok :add 5381 INFO [2022-01-10 08:39:33,022] jepsen worker 696 - jepsen.util 696 :ok :add 5399 INFO [2022-01-10 08:39:33,022] jepsen worker 34 - jepsen.util 34 :ok :add 5178 INFO [2022-01-10 08:39:33,022] jepsen worker 668 - jepsen.util 668 :ok :add 5225 INFO [2022-01-10 08:39:33,022] jepsen worker 347 - jepsen.util 347 :ok :add 5160 INFO [2022-01-10 08:39:33,023] jepsen worker 846 - jepsen.util 846 :ok :add 5563 INFO [2022-01-10 08:39:33,023] jepsen worker 23 - jepsen.util 23 :ok :add 5626 INFO [2022-01-10 08:39:33,023] jepsen worker 671 - jepsen.util 671 :ok :add 4848 INFO [2022-01-10 08:39:33,023] jepsen worker 196 - jepsen.util 196 :ok :add 5557 INFO [2022-01-10 08:39:33,024] jepsen worker 204 - jepsen.util 204 :ok :add 4864 INFO [2022-01-10 08:39:33,024] jepsen worker 943 - jepsen.util 943 :ok :add 5154 INFO [2022-01-10 08:39:33,024] jepsen worker 12 - jepsen.util 12 :ok :add 5209 INFO [2022-01-10 08:39:33,029] jepsen worker 943 - jepsen.util 943 :invoke :add 6474 INFO [2022-01-10 08:39:33,031] jepsen worker 347 - jepsen.util 347 :invoke :add 6475 INFO [2022-01-10 08:39:33,033] jepsen worker 498 - jepsen.util 498 :invoke :add 6476 INFO [2022-01-10 08:39:33,035] jepsen worker 846 - jepsen.util 846 :invoke :add 6477 INFO [2022-01-10 08:39:33,037] jepsen worker 204 - jepsen.util 204 :invoke :add 6478 INFO [2022-01-10 08:39:33,039] jepsen worker 23 - jepsen.util 23 :invoke :add 6479 INFO [2022-01-10 08:39:33,041] jepsen worker 300 - jepsen.util 300 :invoke :add 6480 INFO [2022-01-10 08:39:33,044] jepsen worker 824 - jepsen.util 824 :invoke :add 6481 INFO [2022-01-10 08:39:33,046] jepsen worker 746 - jepsen.util 746 :invoke :add 6482 INFO [2022-01-10 08:39:33,049] jepsen worker 696 - jepsen.util 696 :invoke :add 6483 INFO [2022-01-10 08:39:33,050] jepsen worker 34 - jepsen.util 34 :invoke :add 6484 INFO [2022-01-10 08:39:33,052] jepsen worker 590 - jepsen.util 590 :ok :add 5967 INFO [2022-01-10 08:39:33,052] jepsen worker 776 - jepsen.util 776 :ok :add 6060 INFO [2022-01-10 08:39:33,052] jepsen worker 760 - jepsen.util 760 :ok :add 6309 INFO [2022-01-10 08:39:33,052] jepsen worker 222 - jepsen.util 222 :ok :add 6281 INFO [2022-01-10 08:39:33,052] jepsen worker 254 - jepsen.util 254 :ok :add 5728 INFO [2022-01-10 08:39:33,052] jepsen worker 459 - jepsen.util 459 :ok :add 5694 INFO [2022-01-10 08:39:33,052] jepsen worker 614 - jepsen.util 614 :ok :add 5704 INFO [2022-01-10 08:39:33,053] jepsen worker 247 - jepsen.util 247 :ok :add 6257 INFO [2022-01-10 08:39:33,053] jepsen worker 457 - jepsen.util 457 :ok :add 6126 INFO [2022-01-10 08:39:33,053] jepsen worker 606 - jepsen.util 606 :invoke :add 6485 INFO [2022-01-10 08:39:33,054] jepsen worker 514 - jepsen.util 514 :ok :add 5821 INFO [2022-01-10 08:39:33,054] jepsen worker 414 - jepsen.util 414 :ok :add 5730 INFO [2022-01-10 08:39:33,054] jepsen worker 92 - jepsen.util 92 :ok :add 5815 INFO [2022-01-10 08:39:33,058] jepsen worker 247 - jepsen.util 247 :invoke :add 6486 INFO [2022-01-10 08:39:33,060] jepsen worker 414 - jepsen.util 414 :invoke :add 6487 INFO [2022-01-10 08:39:33,061] jepsen worker 776 - jepsen.util 776 :invoke :add 6488 INFO [2022-01-10 08:39:33,064] jepsen worker 222 - jepsen.util 222 :invoke :add 6489 INFO [2022-01-10 08:39:33,065] jepsen worker 92 - jepsen.util 92 :invoke :add 6490 INFO [2022-01-10 08:39:33,067] jepsen worker 459 - jepsen.util 459 :invoke :add 6491 INFO [2022-01-10 08:39:33,069] jepsen worker 457 - jepsen.util 457 :invoke :add 6492 INFO [2022-01-10 08:39:33,072] jepsen worker 891 - jepsen.util 891 :invoke :add 6493 INFO [2022-01-10 08:39:33,075] jepsen worker 614 - jepsen.util 614 :invoke :add 6494 INFO [2022-01-10 08:39:33,078] jepsen worker 301 - jepsen.util 301 :invoke :add 6495 INFO [2022-01-10 08:39:33,081] jepsen worker 514 - jepsen.util 514 :invoke :add 6496 INFO [2022-01-10 08:39:33,085] jepsen worker 196 - jepsen.util 196 :invoke :add 6497 INFO [2022-01-10 08:39:33,088] jepsen worker 813 - jepsen.util 813 :invoke :add 6498 INFO [2022-01-10 08:39:33,089] jepsen worker 590 - jepsen.util 590 :invoke :add 6499 INFO [2022-01-10 08:39:33,092] jepsen worker 340 - jepsen.util 340 :invoke :add 6500 INFO [2022-01-10 08:39:33,094] jepsen worker 12 - jepsen.util 12 :invoke :add 6501 INFO [2022-01-10 08:39:33,096] jepsen worker 564 - jepsen.util 564 :ok :add 5787 INFO [2022-01-10 08:39:33,096] jepsen worker 174 - jepsen.util 174 :ok :add 6436 INFO [2022-01-10 08:39:33,097] jepsen worker 43 - jepsen.util 43 :ok :add 4879 INFO [2022-01-10 08:39:33,097] jepsen worker 391 - jepsen.util 391 :ok :add 5214 INFO [2022-01-10 08:39:33,097] jepsen worker 114 - jepsen.util 114 :ok :add 6191 INFO [2022-01-10 08:39:33,097] jepsen worker 485 - jepsen.util 485 :ok :add 5896 INFO [2022-01-10 08:39:33,097] jepsen worker 760 - jepsen.util 760 :invoke :add 6502 INFO [2022-01-10 08:39:33,100] jepsen worker 671 - jepsen.util 671 :invoke :add 6503 INFO [2022-01-10 08:39:33,100] jepsen worker 321 - jepsen.util 321 :ok :add 4718 INFO [2022-01-10 08:39:33,100] jepsen worker 157 - jepsen.util 157 :ok :add 5121 INFO [2022-01-10 08:39:33,101] jepsen worker 971 - jepsen.util 971 :ok :add 5264 INFO [2022-01-10 08:39:33,101] jepsen worker 371 - jepsen.util 371 :ok :add 4809 INFO [2022-01-10 08:39:33,101] jepsen worker 894 - jepsen.util 894 :ok :add 4825 INFO [2022-01-10 08:39:33,101] jepsen worker 224 - jepsen.util 224 :ok :add 6300 INFO [2022-01-10 08:39:33,101] jepsen worker 336 - jepsen.util 336 :ok :add 4981 INFO [2022-01-10 08:39:33,101] jepsen worker 174 - jepsen.util 174 :invoke :add 6504 INFO [2022-01-10 08:39:33,101] jepsen worker 795 - jepsen.util 795 :ok :add 6056 INFO [2022-01-10 08:39:33,102] jepsen worker 362 - jepsen.util 362 :ok :add 6327 INFO [2022-01-10 08:39:33,102] jepsen worker 472 - jepsen.util 472 :ok :add 4734 INFO [2022-01-10 08:39:33,102] jepsen worker 28 - jepsen.util 28 :ok :add 6062 INFO [2022-01-10 08:39:33,102] jepsen worker 132 - jepsen.util 132 :ok :add 6313 INFO [2022-01-10 08:39:33,102] jepsen worker 23 - jepsen.util 23 :ok :add 6479 INFO [2022-01-10 08:39:33,104] jepsen worker 695 - jepsen.util 695 :ok :add 5796 INFO [2022-01-10 08:39:33,104] jepsen worker 780 - jepsen.util 780 :ok :add 6008 INFO [2022-01-10 08:39:33,105] jepsen worker 618 - jepsen.util 618 :ok :add 6010 INFO [2022-01-10 08:39:33,107] jepsen worker 912 - jepsen.util 912 :ok :add 6024 INFO [2022-01-10 08:39:33,107] jepsen worker 488 - jepsen.util 488 :ok :add 5887 INFO [2022-01-10 08:39:33,107] jepsen worker 814 - jepsen.util 814 :ok :add 6180 INFO [2022-01-10 08:39:33,107] jepsen worker 62 - jepsen.util 62 :ok :add 6198 INFO [2022-01-10 08:39:33,107] jepsen worker 533 - jepsen.util 533 :ok :add 6379 INFO [2022-01-10 08:39:33,108] jepsen worker 254 - jepsen.util 254 :invoke :add 6505 INFO [2022-01-10 08:39:33,111] jepsen worker 814 - jepsen.util 814 :invoke :add 6506 WARN [2022-01-10 08:39:33,111] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":6555451,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":6555451,"total_server_duration_us":6554053,"encode_duration_us":336,"operation_id":"0x158d","timeout_ms":10000,"last_server_duration_us":6554053,"total_duration_us":6555801},{"operation_name":"insert","last_dispatch_duration_us":6335931,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":6335931,"total_server_duration_us":6334377,"encode_duration_us":284,"operation_id":"0x15f0","timeout_ms":10000,"last_server_duration_us":6334377,"total_duration_us":6336193},{"operation_name":"insert","last_dispatch_duration_us":6258855,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":6258855,"total_server_duration_us":6257926,"encode_duration_us":298,"operation_id":"0x1591","timeout_ms":10000,"last_server_duration_us":6257926,"total_duration_us":6259104},{"operation_name":"insert","last_dispatch_duration_us":5993217,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":5993217,"total_server_duration_us":5991683,"encode_duration_us":287,"operation_id":"0x1602","timeout_ms":10000,"last_server_duration_us":5991683,"total_duration_us":5993466},{"operation_name":"insert","last_dispatch_duration_us":5298941,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":5298941,"total_server_duration_us":5297758,"encode_duration_us":456,"operation_id":"0x141f","timeout_ms":10000,"last_server_duration_us":5297758,"total_duration_us":5299245},{"operation_name":"insert","last_dispatch_duration_us":5251455,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":5251455,"total_server_duration_us":5249647,"encode_duration_us":267,"operation_id":"0x1502","timeout_ms":10000,"last_server_duration_us":5249647,"total_duration_us":5251730},{"operation_name":"insert","last_dispatch_duration_us":5200978,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":5200978,"total_server_duration_us":5200044,"encode_duration_us":288,"operation_id":"0x1492","timeout_ms":10000,"last_server_duration_us":5200044,"total_duration_us":5201290},{"operation_name":"insert","last_dispatch_duration_us":5157072,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":5157072,"total_server_duration_us":5155656,"encode_duration_us":392,"operation_id":"0x1524","timeout_ms":10000,"last_server_duration_us":5155656,"total_duration_us":5157308},{"operation_name":"insert","last_dispatch_duration_us":4729826,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":4729826,"total_server_duration_us":4728777,"encode_duration_us":537,"operation_id":"0x14e2","timeout_ms":10000,"last_server_duration_us":4728777,"total_duration_us":4730133},{"operation_name":"insert","last_dispatch_duration_us":4722999,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":4722999,"total_server_duration_us":4722332,"encode_duration_us":271,"operation_id":"0x14e4","timeout_ms":10000,"last_server_duration_us":4722332,"total_duration_us":4723267}],"total_count":166}} INFO [2022-01-10 08:39:33,113] jepsen worker 336 - jepsen.util 336 :invoke :add 6507 INFO [2022-01-10 08:39:33,116] jepsen worker 516 - jepsen.util 516 :invoke :add 6508 INFO [2022-01-10 08:39:33,117] jepsen worker 618 - jepsen.util 618 :invoke :add 6509 INFO [2022-01-10 08:39:33,119] jepsen worker 157 - jepsen.util 157 :invoke :add 6510 INFO [2022-01-10 08:39:33,121] jepsen worker 912 - jepsen.util 912 :invoke :add 6511 INFO [2022-01-10 08:39:33,123] jepsen worker 488 - jepsen.util 488 :invoke :add 6512 INFO [2022-01-10 08:39:33,125] jepsen worker 321 - jepsen.util 321 :invoke :add 6513 INFO [2022-01-10 08:39:33,127] jepsen worker 894 - jepsen.util 894 :invoke :add 6514 INFO [2022-01-10 08:39:33,129] jepsen worker 62 - jepsen.util 62 :invoke :add 6515 INFO [2022-01-10 08:39:33,131] jepsen worker 132 - jepsen.util 132 :invoke :add 6516 INFO [2022-01-10 08:39:33,133] jepsen worker 224 - jepsen.util 224 :invoke :add 6517 INFO [2022-01-10 08:39:33,135] jepsen worker 668 - jepsen.util 668 :invoke :add 6518 INFO [2022-01-10 08:39:33,136] jepsen worker 43 - jepsen.util 43 :invoke :add 6519 INFO [2022-01-10 08:39:33,138] jepsen worker 114 - jepsen.util 114 :invoke :add 6520 INFO [2022-01-10 08:39:33,140] jepsen worker 564 - jepsen.util 564 :invoke :add 6521 INFO [2022-01-10 08:39:33,141] jepsen worker 371 - jepsen.util 371 :invoke :add 6522 INFO [2022-01-10 08:39:33,144] jepsen worker 533 - jepsen.util 533 :invoke :add 6523 INFO [2022-01-10 08:39:33,145] jepsen worker 485 - jepsen.util 485 :invoke :add 6524 INFO [2022-01-10 08:39:33,147] jepsen worker 472 - jepsen.util 472 :invoke :add 6525 INFO [2022-01-10 08:39:33,148] jepsen worker 28 - jepsen.util 28 :invoke :add 6526 INFO [2022-01-10 08:39:33,150] jepsen worker 23 - jepsen.util 23 :invoke :add 6527 INFO [2022-01-10 08:39:33,153] jepsen worker 519 - jepsen.util 519 :invoke :add 6528 INFO [2022-01-10 08:39:33,154] jepsen worker 391 - jepsen.util 391 :invoke :add 6529 INFO [2022-01-10 08:39:33,156] jepsen worker 695 - jepsen.util 695 :invoke :add 6530 INFO [2022-01-10 08:39:33,158] jepsen worker 971 - jepsen.util 971 :invoke :add 6531 INFO [2022-01-10 08:39:33,159] jepsen worker 780 - jepsen.util 780 :invoke :add 6532 INFO [2022-01-10 08:39:33,161] jepsen worker 795 - jepsen.util 795 :invoke :add 6533 INFO [2022-01-10 08:39:33,163] jepsen worker 362 - jepsen.util 362 :invoke :add 6534 INFO [2022-01-10 08:39:33,330] jepsen worker 364 - jepsen.util 364 :ok :add 5878 INFO [2022-01-10 08:39:33,330] jepsen worker 168 - jepsen.util 168 :ok :add 6148 INFO [2022-01-10 08:39:33,330] jepsen worker 102 - jepsen.util 102 :ok :add 6215 INFO [2022-01-10 08:39:33,331] jepsen worker 218 - jepsen.util 218 :ok :add 6222 INFO [2022-01-10 08:39:33,331] jepsen worker 401 - jepsen.util 401 :ok :add 6386 INFO [2022-01-10 08:39:33,331] jepsen worker 613 - jepsen.util 613 :ok :add 6239 INFO [2022-01-10 08:39:33,331] jepsen worker 155 - jepsen.util 155 :ok :add 6167 INFO [2022-01-10 08:39:33,331] jepsen worker 354 - jepsen.util 354 :ok :add 6385 INFO [2022-01-10 08:39:33,331] jepsen worker 537 - jepsen.util 537 :ok :add 6216 INFO [2022-01-10 08:39:33,331] jepsen worker 469 - jepsen.util 469 :ok :add 6164 INFO [2022-01-10 08:39:33,331] jepsen worker 683 - jepsen.util 683 :ok :add 6221 INFO [2022-01-10 08:39:33,335] jepsen worker 168 - jepsen.util 168 :invoke :add 6535 INFO [2022-01-10 08:39:33,336] jepsen worker 401 - jepsen.util 401 :invoke :add 6536 INFO [2022-01-10 08:39:33,338] jepsen worker 537 - jepsen.util 537 :invoke :add 6537 INFO [2022-01-10 08:39:33,340] jepsen worker 364 - jepsen.util 364 :invoke :add 6538 INFO [2022-01-10 08:39:33,341] jepsen worker 102 - jepsen.util 102 :invoke :add 6539 INFO [2022-01-10 08:39:33,343] jepsen worker 218 - jepsen.util 218 :invoke :add 6540 INFO [2022-01-10 08:39:33,345] jepsen worker 469 - jepsen.util 469 :invoke :add 6541 INFO [2022-01-10 08:39:33,347] jepsen worker 354 - jepsen.util 354 :invoke :add 6542 INFO [2022-01-10 08:39:33,349] jepsen worker 613 - jepsen.util 613 :invoke :add 6543 INFO [2022-01-10 08:39:33,350] jepsen worker 683 - jepsen.util 683 :invoke :add 6544 INFO [2022-01-10 08:39:33,352] jepsen worker 155 - jepsen.util 155 :invoke :add 6545 INFO [2022-01-10 08:39:33,438] jepsen worker 811 - jepsen.util 811 :ok :add 5861 INFO [2022-01-10 08:39:33,438] jepsen worker 654 - jepsen.util 654 :ok :add 5855 INFO [2022-01-10 08:39:33,438] jepsen worker 764 - jepsen.util 764 :ok :add 6217 INFO [2022-01-10 08:39:33,438] jepsen worker 197 - jepsen.util 197 :ok :add 6223 INFO [2022-01-10 08:39:33,439] jepsen worker 230 - jepsen.util 230 :ok :add 6152 INFO [2022-01-10 08:39:33,439] jepsen worker 501 - jepsen.util 501 :ok :add 6387 INFO [2022-01-10 08:39:33,442] jepsen worker 764 - jepsen.util 764 :invoke :add 6546 INFO [2022-01-10 08:39:33,444] jepsen worker 654 - jepsen.util 654 :invoke :add 6547 INFO [2022-01-10 08:39:33,445] jepsen worker 501 - jepsen.util 501 :invoke :add 6548 INFO [2022-01-10 08:39:33,447] jepsen worker 230 - jepsen.util 230 :invoke :add 6549 INFO [2022-01-10 08:39:33,449] jepsen worker 197 - jepsen.util 197 :invoke :add 6550 INFO [2022-01-10 08:39:33,451] jepsen worker 811 - jepsen.util 811 :invoke :add 6551 INFO [2022-01-10 08:39:33,565] jepsen worker 265 - jepsen.util 265 :ok :add 5561 INFO [2022-01-10 08:39:33,565] jepsen worker 333 - jepsen.util 333 :ok :add 5213 INFO [2022-01-10 08:39:33,566] jepsen worker 822 - jepsen.util 822 :ok :add 4852 INFO [2022-01-10 08:39:33,566] jepsen worker 939 - jepsen.util 939 :ok :add 4866 INFO [2022-01-10 08:39:33,566] jepsen worker 14 - jepsen.util 14 :ok :add 5891 INFO [2022-01-10 08:39:33,566] jepsen worker 298 - jepsen.util 298 :ok :add 5227 INFO [2022-01-10 08:39:33,568] jepsen worker 859 - jepsen.util 859 :ok :add 4777 INFO [2022-01-10 08:39:33,568] jepsen worker 136 - jepsen.util 136 :ok :add 5162 INFO [2022-01-10 08:39:33,568] jepsen worker 145 - jepsen.util 145 :ok :add 5610 INFO [2022-01-10 08:39:33,568] jepsen worker 879 - jepsen.util 879 :ok :add 5624 INFO [2022-01-10 08:39:33,569] jepsen worker 408 - jepsen.util 408 :ok :add 5383 INFO [2022-01-10 08:39:33,569] jepsen worker 667 - jepsen.util 667 :ok :add 5555 INFO [2022-01-10 08:39:33,569] jepsen worker 355 - jepsen.util 355 :ok :add 5156 INFO [2022-01-10 08:39:33,570] jepsen worker 404 - jepsen.util 404 :ok :add 5608 INFO [2022-01-10 08:39:33,569] jepsen worker 939 - jepsen.util 939 :invoke :add 6552 INFO [2022-01-10 08:39:33,570] jepsen worker 729 - jepsen.util 729 :ok :add 5798 INFO [2022-01-10 08:39:33,572] jepsen worker 136 - jepsen.util 136 :invoke :add 6553 INFO [2022-01-10 08:39:33,575] jepsen worker 859 - jepsen.util 859 :invoke :add 6554 INFO [2022-01-10 08:39:33,577] jepsen worker 145 - jepsen.util 145 :invoke :add 6555 INFO [2022-01-10 08:39:33,579] jepsen worker 667 - jepsen.util 667 :invoke :add 6556 INFO [2022-01-10 08:39:33,582] jepsen worker 408 - jepsen.util 408 :invoke :add 6557 INFO [2022-01-10 08:39:33,583] jepsen worker 355 - jepsen.util 355 :invoke :add 6558 INFO [2022-01-10 08:39:33,585] jepsen worker 879 - jepsen.util 879 :invoke :add 6559 INFO [2022-01-10 08:39:33,587] jepsen worker 822 - jepsen.util 822 :invoke :add 6560 INFO [2022-01-10 08:39:33,588] jepsen worker 298 - jepsen.util 298 :invoke :add 6561 INFO [2022-01-10 08:39:33,590] jepsen worker 729 - jepsen.util 729 :invoke :add 6562 INFO [2022-01-10 08:39:33,592] jepsen worker 14 - jepsen.util 14 :invoke :add 6563 INFO [2022-01-10 08:39:33,593] jepsen worker 265 - jepsen.util 265 :invoke :add 6564 INFO [2022-01-10 08:39:33,595] jepsen worker 333 - jepsen.util 333 :invoke :add 6565 INFO [2022-01-10 08:39:33,598] jepsen worker 404 - jepsen.util 404 :invoke :add 6566 INFO [2022-01-10 08:39:33,696] jepsen worker 67 - jepsen.util 67 :ok :add 4962 INFO [2022-01-10 08:39:33,697] jepsen worker 688 - jepsen.util 688 :ok :add 5066 INFO [2022-01-10 08:39:33,697] jepsen worker 511 - jepsen.util 511 :ok :add 5317 INFO [2022-01-10 08:39:33,697] jepsen worker 491 - jepsen.util 491 :ok :add 5738 INFO [2022-01-10 08:39:33,698] jepsen worker 135 - jepsen.util 135 :ok :add 4956 INFO [2022-01-10 08:39:33,698] jepsen worker 834 - jepsen.util 834 :ok :add 5052 INFO [2022-01-10 08:39:33,698] jepsen worker 329 - jepsen.util 329 :ok :add 5287 INFO [2022-01-10 08:39:33,698] jepsen worker 12 - jepsen.util 12 :ok :add 6501 INFO [2022-01-10 08:39:33,698] jepsen worker 717 - jepsen.util 717 :ok :add 5323 INFO [2022-01-10 08:39:33,698] jepsen worker 833 - jepsen.util 833 :ok :add 5465 INFO [2022-01-10 08:39:33,699] jepsen worker 688 - jepsen.util 688 :invoke :add 6567 INFO [2022-01-10 08:39:33,701] jepsen worker 491 - jepsen.util 491 :invoke :add 6568 INFO [2022-01-10 08:39:33,703] jepsen worker 717 - jepsen.util 717 :invoke :add 6569 INFO [2022-01-10 08:39:33,705] jepsen worker 511 - jepsen.util 511 :invoke :add 6570 INFO [2022-01-10 08:39:33,707] jepsen worker 12 - jepsen.util 12 :invoke :add 6571 INFO [2022-01-10 08:39:33,709] jepsen worker 135 - jepsen.util 135 :invoke :add 6572 INFO [2022-01-10 08:39:33,711] jepsen worker 833 - jepsen.util 833 :invoke :add 6573 INFO [2022-01-10 08:39:33,713] jepsen worker 329 - jepsen.util 329 :invoke :add 6574 INFO [2022-01-10 08:39:33,714] jepsen worker 834 - jepsen.util 834 :invoke :add 6575 INFO [2022-01-10 08:39:33,716] jepsen worker 67 - jepsen.util 67 :invoke :add 6576 INFO [2022-01-10 08:39:33,946] jepsen worker 724 - jepsen.util 724 :ok :add 4856 INFO [2022-01-10 08:39:33,946] jepsen worker 575 - jepsen.util 575 :ok :add 5565 INFO [2022-01-10 08:39:33,947] jepsen worker 484 - jepsen.util 484 :ok :add 4747 INFO [2022-01-10 08:39:33,947] jepsen worker 367 - jepsen.util 367 :ok :add 5905 INFO [2022-01-10 08:39:33,947] jepsen worker 565 - jepsen.util 565 :ok :add 5931 INFO [2022-01-10 08:39:33,947] jepsen worker 227 - jepsen.util 227 :ok :add 5217 INFO [2022-01-10 08:39:33,948] jepsen worker 109 - jepsen.util 109 :ok :add 5152 INFO [2022-01-10 08:39:33,948] jepsen worker 722 - jepsen.util 722 :ok :add 5614 INFO [2022-01-10 08:39:33,950] jepsen worker 848 - jepsen.util 848 :ok :add 5549 INFO [2022-01-10 08:39:33,950] jepsen worker 870 - jepsen.util 870 :ok :add 4773 INFO [2022-01-10 08:39:33,951] jepsen worker 986 - jepsen.util 986 :ok :add 5166 INFO [2022-01-10 08:39:33,951] jepsen worker 4 - jepsen.util 4 :ok :add 5223 INFO [2022-01-10 08:39:33,951] jepsen worker 725 - jepsen.util 725 :ok :add 6419 INFO [2022-01-10 08:39:33,952] jepsen worker 724 - jepsen.util 724 :invoke :add 6577 INFO [2022-01-10 08:39:33,955] jepsen worker 575 - jepsen.util 575 :invoke :add 6578 INFO [2022-01-10 08:39:33,957] jepsen worker 848 - jepsen.util 848 :invoke :add 6579 INFO [2022-01-10 08:39:33,959] jepsen worker 109 - jepsen.util 109 :invoke :add 6580 INFO [2022-01-10 08:39:33,961] jepsen worker 367 - jepsen.util 367 :invoke :add 6581 INFO [2022-01-10 08:39:33,962] jepsen worker 870 - jepsen.util 870 :invoke :add 6582 INFO [2022-01-10 08:39:33,964] jepsen worker 986 - jepsen.util 986 :invoke :add 6583 INFO [2022-01-10 08:39:33,966] jepsen worker 4 - jepsen.util 4 :invoke :add 6584 INFO [2022-01-10 08:39:33,967] jepsen worker 484 - jepsen.util 484 :invoke :add 6585 INFO [2022-01-10 08:39:33,969] jepsen worker 722 - jepsen.util 722 :invoke :add 6586 INFO [2022-01-10 08:39:33,972] jepsen worker 725 - jepsen.util 725 :invoke :add 6587 INFO [2022-01-10 08:39:33,973] jepsen worker 565 - jepsen.util 565 :invoke :add 6588 INFO [2022-01-10 08:39:33,975] jepsen worker 227 - jepsen.util 227 :invoke :add 6589 INFO [2022-01-10 08:39:34,037] jepsen worker 313 - jepsen.util 313 :ok :add 6383 INFO [2022-01-10 08:39:34,037] jepsen worker 73 - jepsen.util 73 :ok :add 6156 INFO [2022-01-10 08:39:34,037] jepsen worker 880 - jepsen.util 880 :ok :add 5865 INFO [2022-01-10 08:39:34,037] jepsen worker 217 - jepsen.util 217 :ok :add 6213 INFO [2022-01-10 08:39:34,038] jepsen worker 313 - jepsen.util 313 :invoke :add 6590 INFO [2022-01-10 08:39:34,040] jepsen worker 243 - jepsen.util 243 :ok :add 6284 INFO [2022-01-10 08:39:34,040] jepsen worker 569 - jepsen.util 569 :ok :add 6314 INFO [2022-01-10 08:39:34,041] jepsen worker 73 - jepsen.util 73 :invoke :add 6591 INFO [2022-01-10 08:39:34,041] jepsen worker 539 - jepsen.util 539 :ok :add 6051 INFO [2022-01-10 08:39:34,041] jepsen worker 170 - jepsen.util 170 :ok :add 6065 INFO [2022-01-10 08:39:34,041] jepsen worker 925 - jepsen.util 925 :ok :add 6320 INFO [2022-01-10 08:39:34,041] jepsen worker 303 - jepsen.util 303 :ok :add 6338 INFO [2022-01-10 08:39:34,041] jepsen worker 110 - jepsen.util 110 :ok :add 6452 INFO [2022-01-10 08:39:34,042] jepsen worker 177 - jepsen.util 177 :ok :add 6466 INFO [2022-01-10 08:39:34,044] jepsen worker 177 - jepsen.util 177 :invoke :add 6592 INFO [2022-01-10 08:39:34,046] jepsen worker 47 - jepsen.util 47 :ok :add 5794 INFO [2022-01-10 08:39:34,046] jepsen worker 243 - jepsen.util 243 :invoke :add 6593 INFO [2022-01-10 08:39:34,048] jepsen worker 217 - jepsen.util 217 :invoke :add 6594 INFO [2022-01-10 08:39:34,049] jepsen worker 631 - jepsen.util 631 :ok :add 6012 INFO [2022-01-10 08:39:34,049] jepsen worker 447 - jepsen.util 447 :ok :add 6409 INFO [2022-01-10 08:39:34,049] jepsen worker 492 - jepsen.util 492 :ok :add 6182 INFO [2022-01-10 08:39:34,049] jepsen worker 902 - jepsen.util 902 :ok :add 5885 INFO [2022-01-10 08:39:34,050] jepsen worker 925 - jepsen.util 925 :invoke :add 6595 INFO [2022-01-10 08:39:34,050] jepsen worker 678 - jepsen.util 678 :ok :add 5915 INFO [2022-01-10 08:39:34,050] jepsen worker 640 - jepsen.util 640 :ok :add 6026 INFO [2022-01-10 08:39:34,050] jepsen worker 372 - jepsen.util 372 :ok :add 6363 INFO [2022-01-10 08:39:34,050] jepsen worker 343 - jepsen.util 343 :ok :add 6357 INFO [2022-01-10 08:39:34,053] jepsen worker 103 - jepsen.util 103 :ok :add 5921 INFO [2022-01-10 08:39:34,053] jepsen worker 151 - jepsen.util 151 :ok :add 5939 INFO [2022-01-10 08:39:34,053] jepsen worker 539 - jepsen.util 539 :invoke :add 6596 INFO [2022-01-10 08:39:34,055] jepsen worker 447 - jepsen.util 447 :invoke :add 6597 INFO [2022-01-10 08:39:34,058] jepsen worker 640 - jepsen.util 640 :invoke :add 6598 INFO [2022-01-10 08:39:34,060] jepsen worker 303 - jepsen.util 303 :invoke :add 6599 INFO [2022-01-10 08:39:34,062] jepsen worker 678 - jepsen.util 678 :invoke :add 6600 INFO [2022-01-10 08:39:34,064] jepsen worker 492 - jepsen.util 492 :invoke :add 6601 INFO [2022-01-10 08:39:34,065] jepsen worker 902 - jepsen.util 902 :invoke :add 6602 INFO [2022-01-10 08:39:34,067] jepsen worker 372 - jepsen.util 372 :invoke :add 6603 INFO [2022-01-10 08:39:34,068] jepsen worker 631 - jepsen.util 631 :invoke :add 6604 INFO [2022-01-10 08:39:34,070] jepsen worker 103 - jepsen.util 103 :invoke :add 6605 INFO [2022-01-10 08:39:34,072] jepsen worker 569 - jepsen.util 569 :invoke :add 6606 INFO [2022-01-10 08:39:34,074] jepsen worker 170 - jepsen.util 170 :invoke :add 6607 INFO [2022-01-10 08:39:34,075] jepsen worker 343 - jepsen.util 343 :invoke :add 6608 INFO [2022-01-10 08:39:34,076] jepsen worker 110 - jepsen.util 110 :invoke :add 6609 INFO [2022-01-10 08:39:34,078] jepsen worker 880 - jepsen.util 880 :invoke :add 6610 INFO [2022-01-10 08:39:34,080] jepsen worker 151 - jepsen.util 151 :invoke :add 6611 INFO [2022-01-10 08:39:34,081] jepsen worker 47 - jepsen.util 47 :invoke :add 6612 INFO [2022-01-10 08:39:34,265] jepsen worker 409 - jepsen.util 409 :ok :add 4949 INFO [2022-01-10 08:39:34,265] jepsen worker 464 - jepsen.util 464 :ok :add 4951 INFO [2022-01-10 08:39:34,265] jepsen worker 55 - jepsen.util 55 :ok :add 5683 INFO [2022-01-10 08:39:34,265] jepsen worker 919 - jepsen.util 919 :ok :add 5802 INFO [2022-01-10 08:39:34,266] jepsen worker 889 - jepsen.util 889 :ok :add 5308 INFO [2022-01-10 08:39:34,266] jepsen worker 586 - jepsen.util 586 :ok :add 5713 INFO [2022-01-10 08:39:34,266] jepsen worker 873 - jepsen.util 873 :ok :add 5456 INFO [2022-01-10 08:39:34,266] jepsen worker 526 - jepsen.util 526 :ok :add 4965 INFO [2022-01-10 08:39:34,266] jepsen worker 963 - jepsen.util 963 :ok :add 5055 INFO [2022-01-10 08:39:34,266] jepsen worker 324 - jepsen.util 324 :ok :add 5079 INFO [2022-01-10 08:39:34,266] jepsen worker 932 - jepsen.util 932 :ok :add 5324 INFO [2022-01-10 08:39:34,266] jepsen worker 476 - jepsen.util 476 :ok :add 5310 INFO [2022-01-10 08:39:34,266] jepsen worker 209 - jepsen.util 209 :ok :add 5280 INFO [2022-01-10 08:39:34,266] jepsen worker 426 - jepsen.util 426 :ok :add 5298 INFO [2022-01-10 08:39:34,267] jepsen worker 482 - jepsen.util 482 :ok :add 5061 INFO [2022-01-10 08:39:34,267] jepsen worker 440 - jepsen.util 440 :ok :add 5462 INFO [2022-01-10 08:39:34,275] jepsen worker 409 - jepsen.util 409 :invoke :add 6613 INFO [2022-01-10 08:39:34,277] jepsen worker 873 - jepsen.util 873 :invoke :add 6614 INFO [2022-01-10 08:39:34,280] jepsen worker 324 - jepsen.util 324 :invoke :add 6615 INFO [2022-01-10 08:39:34,282] jepsen worker 440 - jepsen.util 440 :invoke :add 6616 INFO [2022-01-10 08:39:34,285] jepsen worker 526 - jepsen.util 526 :invoke :add 6617 INFO [2022-01-10 08:39:34,287] jepsen worker 482 - jepsen.util 482 :invoke :add 6618 INFO [2022-01-10 08:39:34,289] jepsen worker 55 - jepsen.util 55 :invoke :add 6619 INFO [2022-01-10 08:39:34,292] jepsen worker 963 - jepsen.util 963 :invoke :add 6620 INFO [2022-01-10 08:39:34,295] jepsen worker 919 - jepsen.util 919 :invoke :add 6621 INFO [2022-01-10 08:39:34,297] jepsen worker 476 - jepsen.util 476 :invoke :add 6622 INFO [2022-01-10 08:39:34,301] jepsen worker 889 - jepsen.util 889 :invoke :add 6623 INFO [2022-01-10 08:39:34,304] jepsen worker 209 - jepsen.util 209 :invoke :add 6624 INFO [2022-01-10 08:39:34,306] jepsen worker 464 - jepsen.util 464 :invoke :add 6625 INFO [2022-01-10 08:39:34,308] jepsen worker 932 - jepsen.util 932 :invoke :add 6626 INFO [2022-01-10 08:39:34,311] jepsen worker 426 - jepsen.util 426 :invoke :add 6627 INFO [2022-01-10 08:39:34,313] jepsen worker 586 - jepsen.util 586 :invoke :add 6628 INFO [2022-01-10 08:39:34,360] jepsen worker 188 - jepsen.util 188 :ok :add 4854 INFO [2022-01-10 08:39:34,360] jepsen worker 748 - jepsen.util 748 :ok :add 5164 INFO [2022-01-10 08:39:34,360] jepsen worker 76 - jepsen.util 76 :ok :add 5148 INFO [2022-01-10 08:39:34,360] jepsen worker 624 - jepsen.util 624 :ok :add 6437 INFO [2022-01-10 08:39:34,360] jepsen worker 572 - jepsen.util 572 :ok :add 5567 INFO [2022-01-10 08:39:34,360] jepsen worker 98 - jepsen.util 98 :ok :add 5933 INFO [2022-01-10 08:39:34,360] jepsen worker 240 - jepsen.util 240 :ok :add 5786 INFO [2022-01-10 08:39:34,360] jepsen worker 936 - jepsen.util 936 :ok :add 6000 INFO [2022-01-10 08:39:34,360] jepsen worker 529 - jepsen.util 529 :ok :add 5221 INFO [2022-01-10 08:39:34,360] jepsen worker 920 - jepsen.util 920 :ok :add 6188 INFO [2022-01-10 08:39:34,360] jepsen worker 693 - jepsen.util 693 :ok :add 5150 INFO [2022-01-10 08:39:34,360] jepsen worker 556 - jepsen.util 556 :ok :add 5215 INFO [2022-01-10 08:39:34,363] jepsen worker 876 - jepsen.util 876 :ok :add 4769 INFO [2022-01-10 08:39:34,363] jepsen worker 978 - jepsen.util 978 :ok :add 4860 INFO [2022-01-10 08:39:34,363] jepsen worker 25 - jepsen.util 25 :ok :add 5385 INFO [2022-01-10 08:39:34,363] jepsen worker 444 - jepsen.util 444 :ok :add 5239 INFO [2022-01-10 08:39:34,364] jepsen worker 143 - jepsen.util 143 :ok :add 6190 INFO [2022-01-10 08:39:34,371] jepsen worker 188 - jepsen.util 188 :invoke :add 6629 INFO [2022-01-10 08:39:34,374] jepsen worker 25 - jepsen.util 25 :invoke :add 6630 INFO [2022-01-10 08:39:34,378] jepsen worker 748 - jepsen.util 748 :invoke :add 6631 INFO [2022-01-10 08:39:34,380] jepsen worker 444 - jepsen.util 444 :invoke :add 6632 INFO [2022-01-10 08:39:34,382] jepsen worker 920 - jepsen.util 920 :invoke :add 6633 INFO [2022-01-10 08:39:34,386] jepsen worker 572 - jepsen.util 572 :invoke :add 6634 INFO [2022-01-10 08:39:34,388] jepsen worker 98 - jepsen.util 98 :invoke :add 6635 INFO [2022-01-10 08:39:34,391] jepsen worker 978 - jepsen.util 978 :invoke :add 6636 INFO [2022-01-10 08:39:34,394] jepsen worker 556 - jepsen.util 556 :invoke :add 6637 INFO [2022-01-10 08:39:34,396] jepsen worker 240 - jepsen.util 240 :invoke :add 6638 INFO [2022-01-10 08:39:34,399] jepsen worker 936 - jepsen.util 936 :invoke :add 6639 INFO [2022-01-10 08:39:34,401] jepsen worker 143 - jepsen.util 143 :invoke :add 6640 INFO [2022-01-10 08:39:34,404] jepsen worker 624 - jepsen.util 624 :invoke :add 6641 INFO [2022-01-10 08:39:34,407] jepsen worker 76 - jepsen.util 76 :invoke :add 6642 INFO [2022-01-10 08:39:34,409] jepsen worker 529 - jepsen.util 529 :invoke :add 6643 INFO [2022-01-10 08:39:34,411] jepsen worker 693 - jepsen.util 693 :invoke :add 6644 INFO [2022-01-10 08:39:34,414] jepsen worker 876 - jepsen.util 876 :invoke :add 6645 WARN [2022-01-10 08:39:34,417] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":7187622,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":7187622,"total_server_duration_us":5855115,"encode_duration_us":292,"operation_id":"0x16ed","timeout_ms":10000,"last_server_duration_us":5855115,"total_duration_us":7187858},{"operation_name":"insert","last_dispatch_duration_us":7129683,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":7129683,"total_server_duration_us":5851586,"encode_duration_us":284,"operation_id":"0x1705","timeout_ms":10000,"last_server_duration_us":5851586,"total_duration_us":7129859},{"operation_name":"insert","last_dispatch_duration_us":6861786,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","total_dispatch_duration_us":6861786,"total_server_duration_us":6860702,"encode_duration_us":278,"operation_id":"0x155e","timeout_ms":10000,"last_server_duration_us":6860702,"total_duration_us":6862044},{"operation_name":"insert","last_dispatch_duration_us":6651712,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","total_dispatch_duration_us":6651712,"total_server_duration_us":6649706,"encode_duration_us":272,"operation_id":"0x1567","timeout_ms":10000,"last_server_duration_us":6649706,"total_duration_us":6652007},{"operation_name":"insert","last_dispatch_duration_us":6590997,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","total_dispatch_duration_us":6590997,"total_server_duration_us":727364,"encode_duration_us":287,"operation_id":"0x16f8","timeout_ms":10000,"last_server_duration_us":727364,"total_duration_us":6591162},{"operation_name":"insert","last_dispatch_duration_us":6565373,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":6565373,"total_server_duration_us":5844530,"encode_duration_us":273,"operation_id":"0x17f2","timeout_ms":10000,"last_server_duration_us":5844530,"total_duration_us":6565565},{"operation_name":"insert","last_dispatch_duration_us":6534571,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":6534571,"total_server_duration_us":5844530,"encode_duration_us":280,"operation_id":"0x17fe","timeout_ms":10000,"last_server_duration_us":5844530,"total_duration_us":6534798},{"operation_name":"insert","last_dispatch_duration_us":6457164,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","total_dispatch_duration_us":6457164,"total_server_duration_us":946517,"encode_duration_us":271,"operation_id":"0x179e","timeout_ms":10000,"last_server_duration_us":946517,"total_duration_us":6457347},{"operation_name":"insert","last_dispatch_duration_us":6353226,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","total_dispatch_duration_us":6353226,"total_server_duration_us":6350811,"encode_duration_us":275,"operation_id":"0x1569","timeout_ms":10000,"last_server_duration_us":6350811,"total_duration_us":6353456},{"operation_name":"insert","last_dispatch_duration_us":6216437,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":6216437,"total_server_duration_us":5149806,"encode_duration_us":277,"operation_id":"0x1767","timeout_ms":10000,"last_server_duration_us":5149806,"total_duration_us":6216627}],"total_count":138}} WARN [2022-01-10 08:39:34,517] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":7419359,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":7419359,"total_server_duration_us":5977432,"encode_duration_us":381,"operation_id":"0x1693","timeout_ms":10000,"last_server_duration_us":5977432,"total_duration_us":7419561},{"operation_name":"insert","last_dispatch_duration_us":7390187,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":7390187,"total_server_duration_us":984647,"encode_duration_us":291,"operation_id":"0x1620","timeout_ms":10000,"last_server_duration_us":984647,"total_duration_us":7390415},{"operation_name":"insert","last_dispatch_duration_us":7237992,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":7237992,"total_server_duration_us":5311296,"encode_duration_us":284,"operation_id":"0x15cb","timeout_ms":10000,"last_server_duration_us":5311296,"total_duration_us":7238203},{"operation_name":"insert","last_dispatch_duration_us":6841778,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":6841778,"total_server_duration_us":6840890,"encode_duration_us":313,"operation_id":"0x1568","timeout_ms":10000,"last_server_duration_us":6840890,"total_duration_us":6841986},{"operation_name":"insert","last_dispatch_duration_us":6698103,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":6698103,"total_server_duration_us":4765905,"encode_duration_us":364,"operation_id":"0x15c9","timeout_ms":10000,"last_server_duration_us":4765905,"total_duration_us":6698354},{"operation_name":"insert","last_dispatch_duration_us":6611262,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":6611262,"total_server_duration_us":5956082,"encode_duration_us":534,"operation_id":"0x17dd","timeout_ms":10000,"last_server_duration_us":5956082,"total_duration_us":6611449},{"operation_name":"insert","last_dispatch_duration_us":6495142,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":6495142,"total_server_duration_us":5954304,"encode_duration_us":284,"operation_id":"0x180c","timeout_ms":10000,"last_server_duration_us":5954304,"total_duration_us":6495290},{"operation_name":"insert","last_dispatch_duration_us":6379685,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":6379685,"total_server_duration_us":6378240,"encode_duration_us":269,"operation_id":"0x155f","timeout_ms":10000,"last_server_duration_us":6378240,"total_duration_us":6379918},{"operation_name":"insert","last_dispatch_duration_us":5675754,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":5675754,"total_server_duration_us":4716696,"encode_duration_us":312,"operation_id":"0x1762","timeout_ms":10000,"last_server_duration_us":4716696,"total_duration_us":5675963},{"operation_name":"insert","last_dispatch_duration_us":5627476,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":5627476,"total_server_duration_us":4715891,"encode_duration_us":393,"operation_id":"0x1777","timeout_ms":10000,"last_server_duration_us":4715891,"total_duration_us":5627735}],"total_count":142}} WARN [2022-01-10 08:39:34,517] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":7809267,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":7809267,"total_server_duration_us":1590041,"encode_duration_us":303,"operation_id":"0x161e","timeout_ms":10000,"last_server_duration_us":1590041,"total_duration_us":7809549},{"operation_name":"insert","last_dispatch_duration_us":7631238,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":7631238,"total_server_duration_us":1183907,"encode_duration_us":285,"operation_id":"0x15c7","timeout_ms":10000,"last_server_duration_us":1183907,"total_duration_us":7631511},{"operation_name":"insert","last_dispatch_duration_us":6876549,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":6876549,"total_server_duration_us":923062,"encode_duration_us":409,"operation_id":"0x168a","timeout_ms":10000,"last_server_duration_us":923062,"total_duration_us":6876820},{"operation_name":"insert","last_dispatch_duration_us":6628375,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":6628375,"total_server_duration_us":917438,"encode_duration_us":341,"operation_id":"0x16ea","timeout_ms":10000,"last_server_duration_us":917438,"total_duration_us":6628584},{"operation_name":"insert","last_dispatch_duration_us":6597382,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":6597382,"total_server_duration_us":6595771,"encode_duration_us":293,"operation_id":"0x1579","timeout_ms":10000,"last_server_duration_us":6595771,"total_duration_us":6597622},{"operation_name":"insert","last_dispatch_duration_us":6588131,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":6588131,"total_server_duration_us":1169635,"encode_duration_us":482,"operation_id":"0x176b","timeout_ms":10000,"last_server_duration_us":1169635,"total_duration_us":6588326},{"operation_name":"insert","last_dispatch_duration_us":5968264,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":5968264,"total_server_duration_us":5965863,"encode_duration_us":282,"operation_id":"0x17fc","timeout_ms":10000,"last_server_duration_us":5965863,"total_duration_us":6538185},{"operation_name":"insert","last_dispatch_duration_us":6479588,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":6479588,"total_server_duration_us":4911558,"encode_duration_us":270,"operation_id":"0x161a","timeout_ms":10000,"last_server_duration_us":4911558,"total_duration_us":6479798},{"operation_name":"insert","last_dispatch_duration_us":6445549,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":6445549,"total_server_duration_us":1166523,"encode_duration_us":354,"operation_id":"0x17a4","timeout_ms":10000,"last_server_duration_us":1166523,"total_duration_us":6445799},{"operation_name":"insert","last_dispatch_duration_us":6087234,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":6087234,"total_server_duration_us":5425258,"encode_duration_us":305,"operation_id":"0x179a","timeout_ms":10000,"last_server_duration_us":5425258,"total_duration_us":6087432}],"total_count":163}} WARN [2022-01-10 08:39:34,517] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":7706801,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","total_dispatch_duration_us":7706801,"total_server_duration_us":3464322,"encode_duration_us":295,"operation_id":"0x159b","timeout_ms":10000,"last_server_duration_us":3464322,"total_duration_us":7707031},{"operation_name":"insert","last_dispatch_duration_us":7460212,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":7460212,"total_server_duration_us":5998814,"encode_duration_us":489,"operation_id":"0x1683","timeout_ms":10000,"last_server_duration_us":5998814,"total_duration_us":7460424},{"operation_name":"insert","last_dispatch_duration_us":7455391,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":7455391,"total_server_duration_us":5996139,"encode_duration_us":305,"operation_id":"0x1685","timeout_ms":10000,"last_server_duration_us":5996139,"total_duration_us":7455591},{"operation_name":"insert","last_dispatch_duration_us":6739088,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","total_dispatch_duration_us":6739088,"total_server_duration_us":2471082,"encode_duration_us":339,"operation_id":"0x1590","timeout_ms":10000,"last_server_duration_us":2471082,"total_duration_us":6739341},{"operation_name":"insert","last_dispatch_duration_us":6667879,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","total_dispatch_duration_us":6667879,"total_server_duration_us":2613412,"encode_duration_us":319,"operation_id":"0x15f1","timeout_ms":10000,"last_server_duration_us":2613412,"total_duration_us":6668136},{"operation_name":"insert","last_dispatch_duration_us":6623518,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","total_dispatch_duration_us":6623518,"total_server_duration_us":2612160,"encode_duration_us":325,"operation_id":"0x1601","timeout_ms":10000,"last_server_duration_us":2612160,"total_duration_us":6623932},{"operation_name":"insert","last_dispatch_duration_us":6487601,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","total_dispatch_duration_us":6487601,"total_server_duration_us":2605277,"encode_duration_us":344,"operation_id":"0x1637","timeout_ms":10000,"last_server_duration_us":2605277,"total_duration_us":6487854},{"operation_name":"insert","last_dispatch_duration_us":6468398,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","total_dispatch_duration_us":6468398,"total_server_duration_us":2461918,"encode_duration_us":528,"operation_id":"0x1603","timeout_ms":10000,"last_server_duration_us":2461918,"total_duration_us":6468804},{"operation_name":"insert","last_dispatch_duration_us":6450061,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":6450061,"total_server_duration_us":4765905,"encode_duration_us":274,"operation_id":"0x1628","timeout_ms":10000,"last_server_duration_us":4765905,"total_duration_us":6450356},{"operation_name":"insert","last_dispatch_duration_us":6235067,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":6235067,"total_server_duration_us":5287614,"encode_duration_us":288,"operation_id":"0x175e","timeout_ms":10000,"last_server_duration_us":5287614,"total_duration_us":6235246}],"total_count":152}} INFO [2022-01-10 08:39:34,552] jepsen worker 18 - jepsen.util 18 :ok :add 6098 INFO [2022-01-10 08:39:34,552] jepsen worker 207 - jepsen.util 207 :ok :add 6110 INFO [2022-01-10 08:39:34,552] jepsen worker 594 - jepsen.util 594 :ok :add 6255 INFO [2022-01-10 08:39:34,552] jepsen worker 672 - jepsen.util 672 :ok :add 6080 INFO [2022-01-10 08:39:34,552] jepsen worker 696 - jepsen.util 696 :ok :add 6483 INFO [2022-01-10 08:39:34,552] jepsen worker 223 - jepsen.util 223 :ok :add 5987 INFO [2022-01-10 08:39:34,552] jepsen worker 445 - jepsen.util 445 :ok :add 6124 INFO [2022-01-10 08:39:34,552] jepsen worker 225 - jepsen.util 225 :ok :add 6108 INFO [2022-01-10 08:39:34,553] jepsen worker 518 - jepsen.util 518 :ok :add 6261 INFO [2022-01-10 08:39:34,553] jepsen worker 479 - jepsen.util 479 :ok :add 6279 INFO [2022-01-10 08:39:34,553] jepsen worker 23 - jepsen.util 23 :ok :add 6527 INFO [2022-01-10 08:39:34,560] jepsen worker 23 - jepsen.util 23 :invoke :add 6646 INFO [2022-01-10 08:39:34,563] jepsen worker 223 - jepsen.util 223 :invoke :add 6647 INFO [2022-01-10 08:39:34,565] jepsen worker 479 - jepsen.util 479 :invoke :add 6648 INFO [2022-01-10 08:39:34,568] jepsen worker 18 - jepsen.util 18 :invoke :add 6649 INFO [2022-01-10 08:39:34,571] jepsen worker 672 - jepsen.util 672 :invoke :add 6650 INFO [2022-01-10 08:39:34,574] jepsen worker 225 - jepsen.util 225 :invoke :add 6651 INFO [2022-01-10 08:39:34,576] jepsen worker 696 - jepsen.util 696 :invoke :add 6652 INFO [2022-01-10 08:39:34,579] jepsen worker 594 - jepsen.util 594 :invoke :add 6653 INFO [2022-01-10 08:39:34,581] jepsen worker 445 - jepsen.util 445 :invoke :add 6654 INFO [2022-01-10 08:39:34,584] jepsen worker 207 - jepsen.util 207 :invoke :add 6655 INFO [2022-01-10 08:39:34,587] jepsen worker 518 - jepsen.util 518 :invoke :add 6656 WARN [2022-01-10 08:39:34,617] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":7020299,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":7020299,"total_server_duration_us":4213573,"encode_duration_us":268,"operation_id":"0x161c","timeout_ms":10000,"last_server_duration_us":4213573,"total_duration_us":7020481},{"operation_name":"insert","last_dispatch_duration_us":6986794,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":6986794,"total_server_duration_us":4211271,"encode_duration_us":315,"operation_id":"0x162a","timeout_ms":10000,"last_server_duration_us":4211271,"total_duration_us":6987002},{"operation_name":"insert","last_dispatch_duration_us":6894021,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","total_dispatch_duration_us":6894021,"total_server_duration_us":6892829,"encode_duration_us":266,"operation_id":"0x1588","timeout_ms":10000,"last_server_duration_us":6892829,"total_duration_us":6894279},{"operation_name":"insert","last_dispatch_duration_us":6856615,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","total_dispatch_duration_us":6856615,"total_server_duration_us":580932,"encode_duration_us":473,"operation_id":"0x1690","timeout_ms":10000,"last_server_duration_us":580932,"total_duration_us":6856988},{"operation_name":"insert","last_dispatch_duration_us":6622754,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","total_dispatch_duration_us":6622754,"total_server_duration_us":806527,"encode_duration_us":290,"operation_id":"0x175a","timeout_ms":10000,"last_server_duration_us":806527,"total_duration_us":6622995},{"operation_name":"insert","last_dispatch_duration_us":6487098,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":6487098,"total_server_duration_us":3670882,"encode_duration_us":269,"operation_id":"0x1618","timeout_ms":10000,"last_server_duration_us":3670882,"total_duration_us":6487316},{"operation_name":"insert","last_dispatch_duration_us":6179729,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":6179729,"total_server_duration_us":4885383,"encode_duration_us":288,"operation_id":"0x1890","timeout_ms":10000,"last_server_duration_us":4885383,"total_duration_us":6179973},{"operation_name":"insert","last_dispatch_duration_us":808182,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","total_dispatch_duration_us":808182,"total_server_duration_us":783501,"encode_duration_us":567,"operation_id":"0x193d","timeout_ms":10000,"last_server_duration_us":783501,"total_duration_us":5461138},{"operation_name":"insert","last_dispatch_duration_us":4815660,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":4815660,"total_server_duration_us":3649204,"encode_duration_us":264,"operation_id":"0x19a3","timeout_ms":10000,"last_server_duration_us":3649204,"total_duration_us":5456183},{"operation_name":"insert","last_dispatch_duration_us":5336241,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","total_dispatch_duration_us":5336241,"total_server_duration_us":5335024,"encode_duration_us":372,"operation_id":"0x14e5","timeout_ms":10000,"last_server_duration_us":5335024,"total_duration_us":5336615}],"total_count":126}} INFO [2022-01-10 08:39:34,786] jepsen worker 344 - jepsen.util 344 :ok :add 6272 INFO [2022-01-10 08:39:34,787] jepsen worker 101 - jepsen.util 101 :ok :add 5053 INFO [2022-01-10 08:39:34,787] jepsen worker 506 - jepsen.util 506 :ok :add 5286 INFO [2022-01-10 08:39:34,787] jepsen worker 996 - jepsen.util 996 :ok :add 5450 INFO [2022-01-10 08:39:34,786] jepsen worker 704 - jepsen.util 704 :ok :add 5804 INFO [2022-01-10 08:39:34,787] jepsen worker 92 - jepsen.util 92 :ok :add 6490 INFO [2022-01-10 08:39:34,787] jepsen worker 273 - jepsen.util 273 :ok :add 5322 INFO [2022-01-10 08:39:34,789] jepsen worker 376 - jepsen.util 376 :ok :add 4957 INFO [2022-01-10 08:39:34,789] jepsen worker 762 - jepsen.util 762 :ok :add 4963 INFO [2022-01-10 08:39:34,789] jepsen worker 149 - jepsen.util 149 :ok :add 5067 INFO [2022-01-10 08:39:34,789] jepsen worker 635 - jepsen.util 635 :ok :add 5316 INFO [2022-01-10 08:39:34,790] jepsen worker 140 - jepsen.util 140 :ok :add 5715 INFO [2022-01-10 08:39:34,794] jepsen worker 762 - jepsen.util 762 :invoke :add 6657 INFO [2022-01-10 08:39:34,797] jepsen worker 376 - jepsen.util 376 :invoke :add 6658 INFO [2022-01-10 08:39:34,800] jepsen worker 704 - jepsen.util 704 :invoke :add 6659 INFO [2022-01-10 08:39:34,803] jepsen worker 635 - jepsen.util 635 :invoke :add 6660 INFO [2022-01-10 08:39:34,805] jepsen worker 149 - jepsen.util 149 :invoke :add 6661 INFO [2022-01-10 08:39:34,808] jepsen worker 344 - jepsen.util 344 :invoke :add 6662 INFO [2022-01-10 08:39:34,810] jepsen worker 273 - jepsen.util 273 :invoke :add 6663 INFO [2022-01-10 08:39:34,813] jepsen worker 140 - jepsen.util 140 :invoke :add 6664 INFO [2022-01-10 08:39:34,816] jepsen worker 92 - jepsen.util 92 :invoke :add 6665 INFO [2022-01-10 08:39:34,819] jepsen worker 101 - jepsen.util 101 :invoke :add 6666 INFO [2022-01-10 08:39:34,822] jepsen worker 506 - jepsen.util 506 :invoke :add 6667 INFO [2022-01-10 08:39:34,824] jepsen worker 996 - jepsen.util 996 :invoke :add 6668 INFO [2022-01-10 08:39:35,030] jepsen worker 634 - jepsen.util 634 :ok :add 5016 INFO [2022-01-10 08:39:35,030] jepsen worker 100 - jepsen.util 100 :ok :add 5022 INFO [2022-01-10 08:39:35,032] jepsen worker 510 - jepsen.util 510 :ok :add 6146 INFO [2022-01-10 08:39:35,032] jepsen worker 872 - jepsen.util 872 :ok :add 5048 INFO [2022-01-10 08:39:35,032] jepsen worker 715 - jepsen.util 715 :ok :add 5585 INFO [2022-01-10 08:39:35,032] jepsen worker 734 - jepsen.util 734 :ok :add 5315 INFO [2022-01-10 08:39:35,032] jepsen worker 530 - jepsen.util 530 :ok :add 5050 INFO [2022-01-10 08:39:35,032] jepsen worker 211 - jepsen.util 211 :ok :add 5453 INFO [2022-01-10 08:39:35,032] jepsen worker 435 - jepsen.util 435 :ok :add 4912 INFO [2022-01-10 08:39:35,032] jepsen worker 438 - jepsen.util 438 :ok :add 4960 INFO [2022-01-10 08:39:35,032] jepsen worker 934 - jepsen.util 934 :ok :add 6088 INFO [2022-01-10 08:39:35,032] jepsen worker 980 - jepsen.util 980 :ok :add 4793 INFO [2022-01-10 08:39:35,032] jepsen worker 844 - jepsen.util 844 :ok :add 6271 INFO [2022-01-10 08:39:35,032] jepsen worker 419 - jepsen.util 419 :ok :add 5186 INFO [2022-01-10 08:39:35,032] jepsen worker 949 - jepsen.util 949 :ok :add 4978 INFO [2022-01-10 08:39:35,032] jepsen worker 770 - jepsen.util 770 :ok :add 5415 INFO [2022-01-10 08:39:35,033] jepsen worker 33 - jepsen.util 33 :ok :add 5064 INFO [2022-01-10 08:39:35,032] jepsen worker 638 - jepsen.util 638 :ok :add 5686 INFO [2022-01-10 08:39:35,033] jepsen worker 671 - jepsen.util 671 :ok :add 6503 INFO [2022-01-10 08:39:35,032] jepsen worker 229 - jepsen.util 229 :ok :add 4954 INFO [2022-01-10 08:39:35,033] jepsen worker 676 - jepsen.util 676 :ok :add 5339 INFO [2022-01-10 08:39:35,033] jepsen worker 682 - jepsen.util 682 :ok :add 4926 INFO [2022-01-10 08:39:35,033] jepsen worker 259 - jepsen.util 259 :ok :add 6172 INFO [2022-01-10 08:39:35,033] jepsen worker 260 - jepsen.util 260 :ok :add 6134 INFO [2022-01-10 08:39:35,033] jepsen worker 369 - jepsen.util 369 :ok :add 5439 INFO [2022-01-10 08:39:35,033] jepsen worker 9 - jepsen.util 9 :ok :add 6269 INFO [2022-01-10 08:39:35,033] jepsen worker 264 - jepsen.util 264 :ok :add 5321 INFO [2022-01-10 08:39:35,033] jepsen worker 74 - jepsen.util 74 :ok :add 6203 INFO [2022-01-10 08:39:35,033] jepsen worker 847 - jepsen.util 847 :ok :add 5285 INFO [2022-01-10 08:39:35,033] jepsen worker 616 - jepsen.util 616 :ok :add 5750 INFO [2022-01-10 08:39:35,038] jepsen worker 211 - jepsen.util 211 :invoke :add 6669 INFO [2022-01-10 08:39:35,040] jepsen worker 510 - jepsen.util 510 :invoke :add 6670 INFO [2022-01-10 08:39:35,042] jepsen worker 33 - jepsen.util 33 :invoke :add 6671 INFO [2022-01-10 08:39:35,044] jepsen worker 634 - jepsen.util 634 :invoke :add 6672 INFO [2022-01-10 08:39:35,045] jepsen worker 934 - jepsen.util 934 :invoke :add 6673 INFO [2022-01-10 08:39:35,047] jepsen worker 100 - jepsen.util 100 :invoke :add 6674 INFO [2022-01-10 08:39:35,049] jepsen worker 682 - jepsen.util 682 :invoke :add 6675 INFO [2022-01-10 08:39:35,051] jepsen worker 438 - jepsen.util 438 :invoke :add 6676 INFO [2022-01-10 08:39:35,052] jepsen worker 638 - jepsen.util 638 :invoke :add 6677 INFO [2022-01-10 08:39:35,054] jepsen worker 9 - jepsen.util 9 :invoke :add 6678 INFO [2022-01-10 08:39:35,056] jepsen worker 616 - jepsen.util 616 :invoke :add 6679 INFO [2022-01-10 08:39:35,057] jepsen worker 530 - jepsen.util 530 :invoke :add 6680 INFO [2022-01-10 08:39:35,059] jepsen worker 229 - jepsen.util 229 :invoke :add 6681 INFO [2022-01-10 08:39:35,061] jepsen worker 260 - jepsen.util 260 :invoke :add 6682 INFO [2022-01-10 08:39:35,062] jepsen worker 264 - jepsen.util 264 :invoke :add 6683 INFO [2022-01-10 08:39:35,064] jepsen worker 872 - jepsen.util 872 :invoke :add 6684 INFO [2022-01-10 08:39:35,066] jepsen worker 734 - jepsen.util 734 :invoke :add 6685 INFO [2022-01-10 08:39:35,067] jepsen worker 770 - jepsen.util 770 :invoke :add 6686 INFO [2022-01-10 08:39:35,069] jepsen worker 259 - jepsen.util 259 :invoke :add 6687 INFO [2022-01-10 08:39:35,070] jepsen worker 949 - jepsen.util 949 :invoke :add 6688 INFO [2022-01-10 08:39:35,072] jepsen worker 369 - jepsen.util 369 :invoke :add 6689 INFO [2022-01-10 08:39:35,074] jepsen worker 847 - jepsen.util 847 :invoke :add 6690 INFO [2022-01-10 08:39:35,075] jepsen worker 435 - jepsen.util 435 :invoke :add 6691 INFO [2022-01-10 08:39:35,077] jepsen worker 980 - jepsen.util 980 :invoke :add 6692 INFO [2022-01-10 08:39:35,079] jepsen worker 715 - jepsen.util 715 :invoke :add 6693 INFO [2022-01-10 08:39:35,080] jepsen worker 74 - jepsen.util 74 :invoke :add 6694 INFO [2022-01-10 08:39:35,082] jepsen worker 419 - jepsen.util 419 :invoke :add 6695 INFO [2022-01-10 08:39:35,084] jepsen worker 671 - jepsen.util 671 :invoke :add 6696 INFO [2022-01-10 08:39:35,085] jepsen worker 844 - jepsen.util 844 :invoke :add 6697 INFO [2022-01-10 08:39:35,087] jepsen worker 676 - jepsen.util 676 :invoke :add 6698 INFO [2022-01-10 08:39:35,271] jepsen worker 147 - jepsen.util 147 :ok :add 5423 INFO [2022-01-10 08:39:35,271] jepsen worker 853 - jepsen.util 853 :ok :add 4908 INFO [2022-01-10 08:39:35,271] jepsen worker 72 - jepsen.util 72 :ok :add 4898 INFO [2022-01-10 08:39:35,271] jepsen worker 373 - jepsen.util 373 :ok :add 5038 INFO [2022-01-10 08:39:35,271] jepsen worker 541 - jepsen.util 541 :ok :add 4791 INFO [2022-01-10 08:39:35,271] jepsen worker 7 - jepsen.util 7 :ok :add 5351 INFO [2022-01-10 08:39:35,271] jepsen worker 31 - jepsen.util 31 :ok :add 4880 INFO [2022-01-10 08:39:35,271] jepsen worker 763 - jepsen.util 763 :ok :add 5184 INFO [2022-01-10 08:39:35,272] jepsen worker 972 - jepsen.util 972 :ok :add 5417 INFO [2022-01-10 08:39:35,271] jepsen worker 835 - jepsen.util 835 :ok :add 5020 INFO [2022-01-10 08:39:35,272] jepsen worker 46 - jepsen.util 46 :ok :add 6219 INFO [2022-01-10 08:39:35,271] jepsen worker 384 - jepsen.util 384 :ok :add 5766 INFO [2022-01-10 08:39:35,272] jepsen worker 632 - jepsen.util 632 :ok :add 6168 INFO [2022-01-10 08:39:35,272] jepsen worker 557 - jepsen.util 557 :ok :add 5349 INFO [2022-01-10 08:39:35,272] jepsen worker 474 - jepsen.util 474 :ok :add 4924 INFO [2022-01-10 08:39:35,272] jepsen worker 403 - jepsen.util 403 :ok :add 4789 INFO [2022-01-10 08:39:35,274] jepsen worker 812 - jepsen.util 812 :ok :add 5365 INFO [2022-01-10 08:39:35,274] jepsen worker 742 - jepsen.util 742 :ok :add 5014 INFO [2022-01-10 08:39:35,274] jepsen worker 353 - jepsen.util 353 :ok :add 4910 INFO [2022-01-10 08:39:35,274] jepsen worker 630 - jepsen.util 630 :ok :add 6201 INFO [2022-01-10 08:39:35,274] jepsen worker 471 - jepsen.util 471 :ok :add 6235 INFO [2022-01-10 08:39:35,274] jepsen worker 261 - jepsen.util 261 :ok :add 5752 INFO [2022-01-10 08:39:35,277] jepsen worker 812 - jepsen.util 812 :invoke :add 6699 INFO [2022-01-10 08:39:35,280] jepsen worker 403 - jepsen.util 403 :invoke :add 6700 INFO [2022-01-10 08:39:35,281] jepsen worker 7 - jepsen.util 7 :invoke :add 6701 INFO [2022-01-10 08:39:35,283] jepsen worker 31 - jepsen.util 31 :invoke :add 6702 INFO [2022-01-10 08:39:35,285] jepsen worker 471 - jepsen.util 471 :invoke :add 6703 INFO [2022-01-10 08:39:35,287] jepsen worker 384 - jepsen.util 384 :invoke :add 6704 INFO [2022-01-10 08:39:35,289] jepsen worker 742 - jepsen.util 742 :invoke :add 6705 INFO [2022-01-10 08:39:35,291] jepsen worker 541 - jepsen.util 541 :invoke :add 6706 INFO [2022-01-10 08:39:35,293] jepsen worker 353 - jepsen.util 353 :invoke :add 6707 INFO [2022-01-10 08:39:35,294] jepsen worker 261 - jepsen.util 261 :invoke :add 6708 INFO [2022-01-10 08:39:35,296] jepsen worker 72 - jepsen.util 72 :invoke :add 6709 INFO [2022-01-10 08:39:35,297] jepsen worker 557 - jepsen.util 557 :invoke :add 6710 INFO [2022-01-10 08:39:35,299] jepsen worker 632 - jepsen.util 632 :invoke :add 6711 INFO [2022-01-10 08:39:35,301] jepsen worker 972 - jepsen.util 972 :invoke :add 6712 INFO [2022-01-10 08:39:35,302] jepsen worker 46 - jepsen.util 46 :invoke :add 6713 INFO [2022-01-10 08:39:35,304] jepsen worker 630 - jepsen.util 630 :invoke :add 6714 INFO [2022-01-10 08:39:35,305] jepsen worker 763 - jepsen.util 763 :invoke :add 6715 INFO [2022-01-10 08:39:35,308] jepsen worker 853 - jepsen.util 853 :invoke :add 6716 INFO [2022-01-10 08:39:35,309] jepsen worker 147 - jepsen.util 147 :invoke :add 6717 INFO [2022-01-10 08:39:35,311] jepsen worker 373 - jepsen.util 373 :invoke :add 6718 INFO [2022-01-10 08:39:35,313] jepsen worker 835 - jepsen.util 835 :invoke :add 6719 INFO [2022-01-10 08:39:35,315] jepsen worker 474 - jepsen.util 474 :invoke :add 6720 INFO [2022-01-10 08:39:35,492] jepsen worker 807 - jepsen.util 807 :ok :add 6448 INFO [2022-01-10 08:39:35,492] jepsen worker 601 - jepsen.util 601 :ok :add 6450 INFO [2022-01-10 08:39:35,492] jepsen worker 275 - jepsen.util 275 :ok :add 6322 INFO [2022-01-10 08:39:35,492] jepsen worker 61 - jepsen.util 61 :ok :add 6316 INFO [2022-01-10 08:39:35,492] jepsen worker 544 - jepsen.util 544 :ok :add 6286 INFO [2022-01-10 08:39:35,492] jepsen worker 702 - jepsen.util 702 :ok :add 6464 INFO [2022-01-10 08:39:35,496] jepsen worker 61 - jepsen.util 61 :invoke :add 6721 INFO [2022-01-10 08:39:35,498] jepsen worker 275 - jepsen.util 275 :invoke :add 6722 INFO [2022-01-10 08:39:35,499] jepsen worker 718 - jepsen.util 718 :ok :add 5524 INFO [2022-01-10 08:39:35,499] jepsen worker 452 - jepsen.util 452 :ok :add 5508 INFO [2022-01-10 08:39:35,499] jepsen worker 574 - jepsen.util 574 :ok :add 5510 INFO [2022-01-10 08:39:35,499] jepsen worker 771 - jepsen.util 771 :ok :add 5655 INFO [2022-01-10 08:39:35,499] jepsen worker 326 - jepsen.util 326 :ok :add 5480 INFO [2022-01-10 08:39:35,500] jepsen worker 807 - jepsen.util 807 :invoke :add 6723 INFO [2022-01-10 08:39:35,500] jepsen worker 637 - jepsen.util 637 :ok :add 4823 INFO [2022-01-10 08:39:35,500] jepsen worker 830 - jepsen.util 830 :ok :add 5083 INFO [2022-01-10 08:39:35,500] jepsen worker 669 - jepsen.util 669 :ok :add 5113 INFO [2022-01-10 08:39:35,500] jepsen worker 138 - jepsen.util 138 :ok :add 5970 INFO [2022-01-10 08:39:35,500] jepsen worker 990 - jepsen.util 990 :ok :add 5498 INFO [2022-01-10 08:39:35,504] jepsen worker 489 - jepsen.util 489 :ok :add 4817 INFO [2022-01-10 08:39:35,504] jepsen worker 54 - jepsen.util 54 :ok :add 5127 INFO [2022-01-10 08:39:35,504] jepsen worker 960 - jepsen.util 960 :ok :add 5256 INFO [2022-01-10 08:39:35,504] jepsen worker 285 - jepsen.util 285 :ok :add 5679 INFO [2022-01-10 08:39:35,504] jepsen worker 637 - jepsen.util 637 :invoke :add 6724 INFO [2022-01-10 08:39:35,505] jepsen worker 294 - jepsen.util 294 :ok :add 4987 INFO [2022-01-10 08:39:35,505] jepsen worker 77 - jepsen.util 77 :ok :add 5262 INFO [2022-01-10 08:39:35,508] jepsen worker 294 - jepsen.util 294 :invoke :add 6725 INFO [2022-01-10 08:39:35,510] jepsen worker 489 - jepsen.util 489 :invoke :add 6726 INFO [2022-01-10 08:39:35,512] jepsen worker 285 - jepsen.util 285 :invoke :add 6727 INFO [2022-01-10 08:39:35,513] jepsen worker 830 - jepsen.util 830 :invoke :add 6728 INFO [2022-01-10 08:39:35,515] jepsen worker 771 - jepsen.util 771 :invoke :add 6729 INFO [2022-01-10 08:39:35,517] jepsen worker 138 - jepsen.util 138 :invoke :add 6730 INFO [2022-01-10 08:39:35,519] jepsen worker 77 - jepsen.util 77 :invoke :add 6731 INFO [2022-01-10 08:39:35,520] jepsen worker 718 - jepsen.util 718 :invoke :add 6732 INFO [2022-01-10 08:39:35,522] jepsen worker 326 - jepsen.util 326 :invoke :add 6733 INFO [2022-01-10 08:39:35,524] jepsen worker 702 - jepsen.util 702 :invoke :add 6734 INFO [2022-01-10 08:39:35,525] jepsen worker 601 - jepsen.util 601 :invoke :add 6735 INFO [2022-01-10 08:39:35,527] jepsen worker 960 - jepsen.util 960 :invoke :add 6736 INFO [2022-01-10 08:39:35,529] jepsen worker 990 - jepsen.util 990 :invoke :add 6737 INFO [2022-01-10 08:39:35,530] jepsen worker 54 - jepsen.util 54 :invoke :add 6738 INFO [2022-01-10 08:39:35,532] jepsen worker 669 - jepsen.util 669 :invoke :add 6739 INFO [2022-01-10 08:39:35,534] jepsen worker 574 - jepsen.util 574 :invoke :add 6740 INFO [2022-01-10 08:39:35,536] jepsen worker 544 - jepsen.util 544 :invoke :add 6741 INFO [2022-01-10 08:39:35,537] jepsen worker 452 - jepsen.util 452 :invoke :add 6742 INFO [2022-01-10 08:39:35,542] jepsen worker 969 - jepsen.util 969 :ok :add 5452 INFO [2022-01-10 08:39:35,542] jepsen worker 457 - jepsen.util 457 :ok :add 6492 INFO [2022-01-10 08:39:35,542] jepsen worker 760 - jepsen.util 760 :ok :add 6502 INFO [2022-01-10 08:39:35,544] jepsen worker 83 - jepsen.util 83 :ok :add 5966 INFO [2022-01-10 08:39:35,544] jepsen worker 721 - jepsen.util 721 :ok :add 6280 INFO [2022-01-10 08:39:35,544] jepsen worker 315 - jepsen.util 315 :ok :add 6324 INFO [2022-01-10 08:39:35,544] jepsen worker 513 - jepsen.util 513 :ok :add 4979 INFO [2022-01-10 08:39:35,544] jepsen worker 378 - jepsen.util 378 :ok :add 6119 INFO [2022-01-10 08:39:35,544] jepsen worker 923 - jepsen.util 923 :ok :add 6101 INFO [2022-01-10 08:39:35,544] jepsen worker 893 - jepsen.util 893 :ok :add 5996 INFO [2022-01-10 08:39:35,544] jepsen worker 839 - jepsen.util 839 :ok :add 5284 INFO [2022-01-10 08:39:35,544] jepsen worker 993 - jepsen.util 993 :ok :add 5806 INFO [2022-01-10 08:39:35,544] jepsen worker 59 - jepsen.util 59 :ok :add 5717 INFO [2022-01-10 08:39:35,544] jepsen worker 345 - jepsen.util 345 :ok :add 5065 INFO [2022-01-10 08:39:35,544] jepsen worker 27 - jepsen.util 27 :ok :add 5338 INFO [2022-01-10 08:39:35,544] jepsen worker 649 - jepsen.util 649 :ok :add 6244 INFO [2022-01-10 08:39:35,544] jepsen worker 351 - jepsen.util 351 :ok :add 6310 INFO [2022-01-10 08:39:35,545] jepsen worker 142 - jepsen.util 142 :ok :add 6079 INFO [2022-01-10 08:39:35,545] jepsen worker 190 - jepsen.util 190 :ok :add 5952 INFO [2022-01-10 08:39:35,545] jepsen worker 468 - jepsen.util 468 :ok :add 6298 INFO [2022-01-10 08:39:35,545] jepsen worker 698 - jepsen.util 698 :ok :add 6061 INFO [2022-01-10 08:39:35,545] jepsen worker 969 - jepsen.util 969 :invoke :add 6743 INFO [2022-01-10 08:39:35,545] jepsen worker 737 - jepsen.util 737 :ok :add 6055 INFO [2022-01-10 08:39:35,546] jepsen worker 710 - jepsen.util 710 :ok :add 6308 INFO [2022-01-10 08:39:35,547] jepsen worker 494 - jepsen.util 494 :ok :add 4961 INFO [2022-01-10 08:39:35,547] jepsen worker 88 - jepsen.util 88 :ok :add 5051 INFO [2022-01-10 08:39:35,547] jepsen worker 890 - jepsen.util 890 :ok :add 5049 INFO [2022-01-10 08:39:35,547] jepsen worker 199 - jepsen.util 199 :ok :add 5466 INFO [2022-01-10 08:39:35,547] jepsen worker 681 - jepsen.util 681 :ok :add 4955 INFO [2022-01-10 08:39:35,547] jepsen worker 983 - jepsen.util 983 :ok :add 5314 INFO [2022-01-10 08:39:35,547] jepsen worker 527 - jepsen.util 527 :ok :add 6268 INFO [2022-01-10 08:39:35,547] jepsen worker 855 - jepsen.util 855 :ok :add 5320 INFO [2022-01-10 08:39:35,547] jepsen worker 898 - jepsen.util 898 :ok :add 6089 INFO [2022-01-10 08:39:35,553] jepsen worker 855 - jepsen.util 855 :invoke :add 6744 INFO [2022-01-10 08:39:35,555] jepsen worker 378 - jepsen.util 378 :invoke :add 6745 INFO [2022-01-10 08:39:35,557] jepsen worker 993 - jepsen.util 993 :invoke :add 6746 INFO [2022-01-10 08:39:35,559] jepsen worker 698 - jepsen.util 698 :invoke :add 6747 INFO [2022-01-10 08:39:35,561] jepsen worker 142 - jepsen.util 142 :invoke :add 6748 INFO [2022-01-10 08:39:35,562] jepsen worker 457 - jepsen.util 457 :invoke :add 6749 INFO [2022-01-10 08:39:35,564] jepsen worker 199 - jepsen.util 199 :invoke :add 6750 INFO [2022-01-10 08:39:35,566] jepsen worker 737 - jepsen.util 737 :invoke :add 6751 INFO [2022-01-10 08:39:35,568] jepsen worker 27 - jepsen.util 27 :invoke :add 6752 INFO [2022-01-10 08:39:35,569] jepsen worker 890 - jepsen.util 890 :invoke :add 6753 INFO [2022-01-10 08:39:35,571] jepsen worker 983 - jepsen.util 983 :invoke :add 6754 INFO [2022-01-10 08:39:35,572] jepsen worker 893 - jepsen.util 893 :invoke :add 6755 INFO [2022-01-10 08:39:35,574] jepsen worker 649 - jepsen.util 649 :invoke :add 6756 INFO [2022-01-10 08:39:35,575] jepsen worker 681 - jepsen.util 681 :invoke :add 6757 INFO [2022-01-10 08:39:35,577] jepsen worker 527 - jepsen.util 527 :invoke :add 6758 INFO [2022-01-10 08:39:35,578] jepsen worker 59 - jepsen.util 59 :invoke :add 6759 INFO [2022-01-10 08:39:35,580] jepsen worker 315 - jepsen.util 315 :invoke :add 6760 INFO [2022-01-10 08:39:35,581] jepsen worker 468 - jepsen.util 468 :invoke :add 6761 INFO [2022-01-10 08:39:35,583] jepsen worker 760 - jepsen.util 760 :invoke :add 6762 INFO [2022-01-10 08:39:35,585] jepsen worker 190 - jepsen.util 190 :invoke :add 6763 INFO [2022-01-10 08:39:35,588] jepsen worker 513 - jepsen.util 513 :invoke :add 6764 INFO [2022-01-10 08:39:35,589] jepsen worker 494 - jepsen.util 494 :invoke :add 6765 INFO [2022-01-10 08:39:35,591] jepsen worker 351 - jepsen.util 351 :invoke :add 6766 INFO [2022-01-10 08:39:35,592] jepsen worker 839 - jepsen.util 839 :invoke :add 6767 INFO [2022-01-10 08:39:35,594] jepsen worker 83 - jepsen.util 83 :invoke :add 6768 INFO [2022-01-10 08:39:35,596] jepsen worker 88 - jepsen.util 88 :invoke :add 6769 INFO [2022-01-10 08:39:35,598] jepsen worker 345 - jepsen.util 345 :invoke :add 6770 INFO [2022-01-10 08:39:35,599] jepsen worker 923 - jepsen.util 923 :invoke :add 6771 INFO [2022-01-10 08:39:35,601] jepsen worker 721 - jepsen.util 721 :invoke :add 6772 INFO [2022-01-10 08:39:35,602] jepsen worker 710 - jepsen.util 710 :invoke :add 6773 INFO [2022-01-10 08:39:35,604] jepsen worker 898 - jepsen.util 898 :invoke :add 6774 INFO [2022-01-10 08:39:35,808] jepsen worker 913 - jepsen.util 913 :ok :add 5922 INFO [2022-01-10 08:39:35,808] jepsen worker 181 - jepsen.util 181 :ok :add 6181 INFO [2022-01-10 08:39:35,808] jepsen worker 546 - jepsen.util 546 :ok :add 6011 INFO [2022-01-10 08:39:35,808] jepsen worker 483 - jepsen.util 483 :ok :add 5916 INFO [2022-01-10 08:39:35,808] jepsen worker 787 - jepsen.util 787 :ok :add 6199 INFO [2022-01-10 08:39:35,808] jepsen worker 862 - jepsen.util 862 :ok :add 5886 INFO [2022-01-10 08:39:35,808] jepsen worker 581 - jepsen.util 581 :ok :add 6354 INFO [2022-01-10 08:39:35,808] jepsen worker 71 - jepsen.util 71 :ok :add 6009 INFO [2022-01-10 08:39:35,809] jepsen worker 665 - jepsen.util 665 :ok :add 6360 INFO [2022-01-10 08:39:35,809] jepsen worker 738 - jepsen.util 738 :ok :add 6412 INFO [2022-01-10 08:39:35,809] jepsen worker 707 - jepsen.util 707 :ok :add 6025 INFO [2022-01-10 08:39:35,810] jepsen worker 166 - jepsen.util 166 :ok :add 4792 INFO [2022-01-10 08:39:35,810] jepsen worker 622 - jepsen.util 622 :ok :add 5023 INFO [2022-01-10 08:39:35,810] jepsen worker 144 - jepsen.util 144 :ok :add 5366 INFO [2022-01-10 08:39:35,810] jepsen worker 449 - jepsen.util 449 :ok :add 5749 INFO [2022-01-10 08:39:35,810] jepsen worker 954 - jepsen.util 954 :ok :add 4927 INFO [2022-01-10 08:39:35,810] jepsen worker 940 - jepsen.util 940 :ok :add 5187 INFO [2022-01-10 08:39:35,810] jepsen worker 417 - jepsen.util 417 :ok :add 6202 INFO [2022-01-10 08:39:35,810] jepsen worker 964 - jepsen.util 964 :ok :add 4913 INFO [2022-01-10 08:39:35,810] jepsen worker 171 - jepsen.util 171 :ok :add 5420 INFO [2022-01-10 08:39:35,810] jepsen worker 947 - jepsen.util 947 :ok :add 5584 INFO [2022-01-10 08:39:35,813] jepsen worker 245 - jepsen.util 245 :ok :add 5765 INFO [2022-01-10 08:39:35,813] jepsen worker 446 - jepsen.util 446 :ok :add 5017 INFO [2022-01-10 08:39:35,813] jepsen worker 777 - jepsen.util 777 :ok :add 4883 INFO [2022-01-10 08:39:35,813] jepsen worker 528 - jepsen.util 528 :ok :add 5858 INFO [2022-01-10 08:39:35,813] jepsen worker 562 - jepsen.util 562 :ok :add 5438 INFO [2022-01-10 08:39:35,819] jepsen worker 449 - jepsen.util 449 :invoke :add 6775 INFO [2022-01-10 08:39:35,820] jepsen worker 417 - jepsen.util 417 :invoke :add 6776 INFO [2022-01-10 08:39:35,822] jepsen worker 562 - jepsen.util 562 :invoke :add 6777 INFO [2022-01-10 08:39:35,824] jepsen worker 707 - jepsen.util 707 :invoke :add 6778 INFO [2022-01-10 08:39:35,826] jepsen worker 483 - jepsen.util 483 :invoke :add 6779 INFO [2022-01-10 08:39:35,828] jepsen worker 171 - jepsen.util 171 :invoke :add 6780 INFO [2022-01-10 08:39:35,830] jepsen worker 787 - jepsen.util 787 :invoke :add 6781 INFO [2022-01-10 08:39:35,832] jepsen worker 446 - jepsen.util 446 :invoke :add 6782 INFO [2022-01-10 08:39:35,834] jepsen worker 581 - jepsen.util 581 :invoke :add 6783 INFO [2022-01-10 08:39:35,835] jepsen worker 777 - jepsen.util 777 :invoke :add 6784 INFO [2022-01-10 08:39:35,837] jepsen worker 947 - jepsen.util 947 :invoke :add 6785 INFO [2022-01-10 08:39:35,839] jepsen worker 528 - jepsen.util 528 :invoke :add 6786 INFO [2022-01-10 08:39:35,842] jepsen worker 622 - jepsen.util 622 :invoke :add 6787 INFO [2022-01-10 08:39:35,845] jepsen worker 166 - jepsen.util 166 :invoke :add 6788 INFO [2022-01-10 08:39:35,847] jepsen worker 181 - jepsen.util 181 :invoke :add 6789 INFO [2022-01-10 08:39:35,848] jepsen worker 964 - jepsen.util 964 :invoke :add 6790 INFO [2022-01-10 08:39:35,850] jepsen worker 738 - jepsen.util 738 :invoke :add 6791 INFO [2022-01-10 08:39:35,852] jepsen worker 71 - jepsen.util 71 :invoke :add 6792 INFO [2022-01-10 08:39:35,854] jepsen worker 245 - jepsen.util 245 :invoke :add 6793 INFO [2022-01-10 08:39:35,856] jepsen worker 913 - jepsen.util 913 :invoke :add 6794 INFO [2022-01-10 08:39:35,858] jepsen worker 665 - jepsen.util 665 :invoke :add 6795 INFO [2022-01-10 08:39:35,859] jepsen worker 144 - jepsen.util 144 :invoke :add 6796 INFO [2022-01-10 08:39:35,861] jepsen worker 546 - jepsen.util 546 :invoke :add 6797 INFO [2022-01-10 08:39:35,863] jepsen worker 862 - jepsen.util 862 :invoke :add 6798 INFO [2022-01-10 08:39:35,865] jepsen worker 940 - jepsen.util 940 :invoke :add 6799 INFO [2022-01-10 08:39:35,867] jepsen worker 954 - jepsen.util 954 :invoke :add 6800 INFO [2022-01-10 08:39:36,020] jepsen worker 226 - jepsen.util 226 :ok :add 5650 INFO [2022-01-10 08:39:36,020] jepsen worker 945 - jepsen.util 945 :ok :add 5485 INFO [2022-01-10 08:39:36,020] jepsen worker 202 - jepsen.util 202 :ok :add 4982 INFO [2022-01-10 08:39:36,021] jepsen worker 755 - jepsen.util 755 :ok :add 5664 INFO [2022-01-10 08:39:36,021] jepsen worker 231 - jepsen.util 231 :ok :add 5122 INFO [2022-01-10 08:39:36,021] jepsen worker 788 - jepsen.util 788 :ok :add 4812 INFO [2022-01-10 08:39:36,021] jepsen worker 585 - jepsen.util 585 :ok :add 5267 INFO [2022-01-10 08:39:36,021] jepsen worker 610 - jepsen.util 610 :ok :add 6469 INFO [2022-01-10 08:39:36,021] jepsen worker 330 - jepsen.util 330 :ok :add 5539 INFO [2022-01-10 08:39:36,021] jepsen worker 154 - jepsen.util 154 :ok :add 6445 INFO [2022-01-10 08:39:36,021] jepsen worker 430 - jepsen.util 430 :ok :add 6303 INFO [2022-01-10 08:39:36,021] jepsen worker 304 - jepsen.util 304 :ok :add 5116 INFO [2022-01-10 08:39:36,021] jepsen worker 121 - jepsen.util 121 :ok :add 6471 INFO [2022-01-10 08:39:36,021] jepsen worker 783 - jepsen.util 783 :ok :add 5941 INFO [2022-01-10 08:39:36,021] jepsen worker 418 - jepsen.util 418 :ok :add 5515 INFO [2022-01-10 08:39:36,023] jepsen worker 826 - jepsen.util 826 :ok :add 5959 INFO [2022-01-10 08:39:36,023] jepsen worker 999 - jepsen.util 999 :ok :add 5253 INFO [2022-01-10 08:39:36,024] jepsen worker 231 - jepsen.util 231 :invoke :add 6801 INFO [2022-01-10 08:39:36,026] jepsen worker 610 - jepsen.util 610 :invoke :add 6802 INFO [2022-01-10 08:39:36,028] jepsen worker 585 - jepsen.util 585 :invoke :add 6803 INFO [2022-01-10 08:39:36,030] jepsen worker 999 - jepsen.util 999 :invoke :add 6804 INFO [2022-01-10 08:39:36,038] jepsen worker 304 - jepsen.util 304 :invoke :add 6805 INFO [2022-01-10 08:39:36,040] jepsen worker 154 - jepsen.util 154 :invoke :add 6806 INFO [2022-01-10 08:39:36,042] jepsen worker 755 - jepsen.util 755 :invoke :add 6807 INFO [2022-01-10 08:39:36,044] jepsen worker 783 - jepsen.util 783 :invoke :add 6808 INFO [2022-01-10 08:39:36,046] jepsen worker 226 - jepsen.util 226 :invoke :add 6809 INFO [2022-01-10 08:39:36,047] jepsen worker 788 - jepsen.util 788 :invoke :add 6810 INFO [2022-01-10 08:39:36,049] jepsen worker 418 - jepsen.util 418 :invoke :add 6811 INFO [2022-01-10 08:39:36,050] jepsen worker 202 - jepsen.util 202 :invoke :add 6812 INFO [2022-01-10 08:39:36,052] jepsen worker 826 - jepsen.util 826 :invoke :add 6813 INFO [2022-01-10 08:39:36,053] jepsen worker 430 - jepsen.util 430 :invoke :add 6814 INFO [2022-01-10 08:39:36,055] jepsen worker 945 - jepsen.util 945 :invoke :add 6815 INFO [2022-01-10 08:39:36,056] jepsen worker 121 - jepsen.util 121 :invoke :add 6816 INFO [2022-01-10 08:39:36,058] jepsen worker 330 - jepsen.util 330 :invoke :add 6817 INFO [2022-01-10 08:39:36,216] jepsen worker 413 - jepsen.util 413 :ok :add 5835 INFO [2022-01-10 08:39:36,216] jepsen worker 547 - jepsen.util 547 :ok :add 5724 INFO [2022-01-10 08:39:36,216] jepsen worker 423 - jepsen.util 423 :ok :add 5989 INFO [2022-01-10 08:39:36,216] jepsen worker 131 - jepsen.util 131 :ok :add 4952 INFO [2022-01-10 08:39:36,216] jepsen worker 791 - jepsen.util 791 :ok :add 5056 INFO [2022-01-10 08:39:36,217] jepsen worker 22 - jepsen.util 22 :ok :add 5313 INFO [2022-01-10 08:39:36,217] jepsen worker 160 - jepsen.util 160 :ok :add 5062 INFO [2022-01-10 08:39:36,217] jepsen worker 499 - jepsen.util 499 :ok :add 5327 INFO [2022-01-10 08:39:36,217] jepsen worker 291 - jepsen.util 291 :ok :add 5479 INFO [2022-01-10 08:39:36,217] jepsen worker 169 - jepsen.util 169 :ok :add 5708 INFO [2022-01-10 08:39:36,217] jepsen worker 380 - jepsen.util 380 :ok :add 5698 INFO [2022-01-10 08:39:36,218] jepsen worker 946 - jepsen.util 946 :ok :add 5710 INFO [2022-01-10 08:39:36,218] jepsen worker 293 - jepsen.util 293 :ok :add 4966 INFO [2022-01-10 08:39:36,219] jepsen worker 566 - jepsen.util 566 :ok :add 5283 INFO [2022-01-10 08:39:36,219] jepsen worker 236 - jepsen.util 236 :ok :add 5455 INFO [2022-01-10 08:39:36,219] jepsen worker 979 - jepsen.util 979 :ok :add 5461 INFO [2022-01-10 08:39:36,222] jepsen worker 291 - jepsen.util 291 :invoke :add 6818 INFO [2022-01-10 08:39:36,225] jepsen worker 547 - jepsen.util 547 :invoke :add 6819 INFO [2022-01-10 08:39:36,227] jepsen worker 160 - jepsen.util 160 :invoke :add 6820 INFO [2022-01-10 08:39:36,228] jepsen worker 423 - jepsen.util 423 :invoke :add 6821 INFO [2022-01-10 08:39:36,230] jepsen worker 22 - jepsen.util 22 :invoke :add 6822 INFO [2022-01-10 08:39:36,233] jepsen worker 979 - jepsen.util 979 :invoke :add 6823 INFO [2022-01-10 08:39:36,236] jepsen worker 499 - jepsen.util 499 :invoke :add 6824 INFO [2022-01-10 08:39:36,237] jepsen worker 605 - jepsen.util 605 :ok :add 5920 INFO [2022-01-10 08:39:36,237] jepsen worker 796 - jepsen.util 796 :ok :add 6410 INFO [2022-01-10 08:39:36,237] jepsen worker 360 - jepsen.util 360 :ok :add 6183 INFO [2022-01-10 08:39:36,237] jepsen worker 331 - jepsen.util 331 :ok :add 6013 INFO [2022-01-10 08:39:36,237] jepsen worker 137 - jepsen.util 137 :ok :add 6356 INFO [2022-01-10 08:39:36,237] jepsen worker 60 - jepsen.util 60 :ok :add 6424 INFO [2022-01-10 08:39:36,237] jepsen worker 232 - jepsen.util 232 :ok :add 6027 INFO [2022-01-10 08:39:36,237] jepsen worker 368 - jepsen.util 368 :ok :add 6408 INFO [2022-01-10 08:39:36,237] jepsen worker 255 - jepsen.util 255 :ok :add 5884 INFO [2022-01-10 08:39:36,237] jepsen worker 36 - jepsen.util 36 :ok :add 5914 INFO [2022-01-10 08:39:36,240] jepsen worker 791 - jepsen.util 791 :invoke :add 6825 INFO [2022-01-10 08:39:36,243] jepsen worker 360 - jepsen.util 360 :invoke :add 6826 INFO [2022-01-10 08:39:36,245] jepsen worker 137 - jepsen.util 137 :invoke :add 6827 INFO [2022-01-10 08:39:36,248] jepsen worker 255 - jepsen.util 255 :invoke :add 6828 INFO [2022-01-10 08:39:36,266] jepsen worker 60 - jepsen.util 60 :invoke :add 6829 INFO [2022-01-10 08:39:36,278] jepsen worker 36 - jepsen.util 36 :invoke :add 6830 INFO [2022-01-10 08:39:36,279] jepsen worker 281 - jepsen.util 281 :ok :add 4810 INFO [2022-01-10 08:39:36,279] jepsen worker 757 - jepsen.util 757 :ok :add 5120 INFO [2022-01-10 08:39:36,279] jepsen worker 279 - jepsen.util 279 :ok :add 4998 INFO [2022-01-10 08:39:36,280] jepsen worker 85 - jepsen.util 85 :ok :add 6301 INFO [2022-01-10 08:39:36,280] jepsen worker 739 - jepsen.util 739 :ok :add 6044 INFO [2022-01-10 08:39:36,280] jepsen worker 975 - jepsen.util 975 :ok :add 5138 INFO [2022-01-10 08:39:36,280] jepsen worker 184 - jepsen.util 184 :ok :add 5265 INFO [2022-01-10 08:39:36,280] jepsen worker 750 - jepsen.util 750 :ok :add 5487 INFO [2022-01-10 08:39:36,280] jepsen worker 119 - jepsen.util 119 :ok :add 6447 INFO [2022-01-10 08:39:36,280] jepsen worker 41 - jepsen.util 41 :ok :add 6319 INFO [2022-01-10 08:39:36,280] jepsen worker 99 - jepsen.util 99 :ok :add 5249 INFO [2022-01-10 08:39:36,280] jepsen worker 548 - jepsen.util 548 :ok :add 4824 INFO [2022-01-10 08:39:36,280] jepsen worker 319 - jepsen.util 319 :ok :add 5517 INFO [2022-01-10 08:39:36,281] jepsen worker 858 - jepsen.util 858 :ok :add 4980 INFO [2022-01-10 08:39:36,281] jepsen worker 851 - jepsen.util 851 :ok :add 6473 INFO [2022-01-10 08:39:36,341] jepsen worker 603 - jepsen.util 603 :ok :add 5114 INFO [2022-01-10 08:39:36,342] jepsen worker 337 - jepsen.util 337 :ok :add 5251 INFO [2022-01-10 08:39:36,342] jepsen worker 349 - jepsen.util 349 :ok :add 5039 INFO [2022-01-10 08:39:36,342] jepsen worker 481 - jepsen.util 481 :ok :add 6390 INFO [2022-01-10 08:39:36,342] jepsen worker 212 - jepsen.util 212 :ok :add 6200 INFO [2022-01-10 08:39:36,342] jepsen worker 605 - jepsen.util 605 :invoke :add 6831 INFO [2022-01-10 08:39:36,343] jepsen worker 79 - jepsen.util 79 :ok :add 5666 INFO [2022-01-10 08:39:36,343] jepsen worker 111 - jepsen.util 111 :ok :add 5943 INFO [2022-01-10 08:39:36,343] jepsen worker 803 - jepsen.util 803 :ok :add 6335 INFO [2022-01-10 08:39:36,344] jepsen worker 768 - jepsen.util 768 :ok :add 4909 INFO [2022-01-10 08:39:36,344] jepsen worker 346 - jepsen.util 346 :ok :add 5350 INFO [2022-01-10 08:39:36,345] jepsen worker 561 - jepsen.util 561 :ok :add 6171 INFO [2022-01-10 08:39:36,345] jepsen worker 559 - jepsen.util 559 :ok :add 6388 INFO [2022-01-10 08:39:36,345] jepsen worker 213 - jepsen.util 213 :ok :add 4899 INFO [2022-01-10 08:39:36,345] jepsen worker 487 - jepsen.util 487 :ok :add 4925 INFO [2022-01-10 08:39:36,345] jepsen worker 732 - jepsen.util 732 :ok :add 5021 INFO [2022-01-10 08:39:36,345] jepsen worker 525 - jepsen.util 525 :ok :add 6218 INFO [2022-01-10 08:39:36,345] jepsen worker 897 - jepsen.util 897 :ok :add 5185 INFO [2022-01-10 08:39:36,345] jepsen worker 257 - jepsen.util 257 :ok :add 5586 INFO [2022-01-10 08:39:36,345] jepsen worker 208 - jepsen.util 208 :ok :add 5842 INFO [2022-01-10 08:39:36,345] jepsen worker 75 - jepsen.util 75 :ok :add 6234 INFO [2022-01-10 08:39:36,346] jepsen worker 372 - jepsen.util 372 :ok :add 6603 INFO [2022-01-10 08:39:36,345] jepsen worker 764 - jepsen.util 764 :ok :add 6546 INFO [2022-01-10 08:39:36,346] jepsen worker 148 - jepsen.util 148 :ok :add 5015 INFO [2022-01-10 08:39:36,346] jepsen worker 685 - jepsen.util 685 :ok :add 4911 INFO [2022-01-10 08:39:36,346] jepsen worker 662 - jepsen.util 662 :ok :add 5767 INFO [2022-01-10 08:39:36,350] jepsen worker 215 - jepsen.util 215 :ok :add 5364 INFO [2022-01-10 08:39:36,350] jepsen worker 706 - jepsen.util 706 :ok :add 5416 INFO [2022-01-10 08:39:36,350] jepsen worker 915 - jepsen.util 915 :ok :add 5422 INFO [2022-01-10 08:39:36,290] jepsen worker 146 - jepsen.util 146 :ok :add 5977 INFO [2022-01-10 08:39:36,373] jepsen worker 53 - jepsen.util 53 :info :add 4770 :RequestTimeoutException INFO [2022-01-10 08:39:36,373] jepsen worker 186 - jepsen.util 186 :info :add 4772 :RequestTimeoutException INFO [2022-01-10 08:39:36,376] jepsen worker 772 - jepsen.util 772 :info :add 4774 :RequestTimeoutException INFO [2022-01-10 08:39:36,389] jepsen worker 757 - jepsen.util 757 :invoke :add 6832 INFO [2022-01-10 08:39:36,390] jepsen worker 208 - jepsen.util 208 :invoke :add 6833 INFO [2022-01-10 08:39:36,402] jepsen worker 282 - jepsen.util 282 :info :add 4795 :RequestTimeoutException INFO [2022-01-10 08:39:36,403] jepsen worker 236 - jepsen.util 236 :invoke :add 6834 INFO [2022-01-10 08:39:36,407] jepsen worker 184 - jepsen.util 184 :invoke :add 6835 INFO [2022-01-10 08:39:36,410] jepsen worker 772 - jepsen.util 1772 :invoke :add 6836 INFO [2022-01-10 08:39:36,413] jepsen worker 79 - jepsen.util 79 :invoke :add 6837 INFO [2022-01-10 08:39:36,416] jepsen worker 331 - jepsen.util 331 :invoke :add 6838 INFO [2022-01-10 08:39:36,418] jepsen worker 41 - jepsen.util 41 :invoke :add 6839 INFO [2022-01-10 08:39:36,421] jepsen worker 346 - jepsen.util 346 :invoke :add 6840 INFO [2022-01-10 08:39:36,424] jepsen worker 119 - jepsen.util 119 :invoke :add 6841 INFO [2022-01-10 08:39:36,425] jepsen worker 851 - jepsen.util 851 :invoke :add 6842 INFO [2022-01-10 08:39:36,427] jepsen worker 662 - jepsen.util 662 :invoke :add 6843 INFO [2022-01-10 08:39:36,429] jepsen worker 215 - jepsen.util 215 :invoke :add 6844 INFO [2022-01-10 08:39:36,431] jepsen worker 213 - jepsen.util 213 :invoke :add 6845 INFO [2022-01-10 08:39:36,433] jepsen worker 975 - jepsen.util 975 :invoke :add 6846 INFO [2022-01-10 08:39:36,434] jepsen worker 186 - jepsen.util 1186 :invoke :add 6847 INFO [2022-01-10 08:39:36,436] jepsen worker 750 - jepsen.util 750 :invoke :add 6848 INFO [2022-01-10 08:39:36,438] jepsen worker 603 - jepsen.util 603 :invoke :add 6849 INFO [2022-01-10 08:39:36,439] jepsen worker 337 - jepsen.util 337 :invoke :add 6850 INFO [2022-01-10 08:39:36,441] jepsen worker 525 - jepsen.util 525 :invoke :add 6851 INFO [2022-01-10 08:39:36,443] jepsen worker 368 - jepsen.util 368 :invoke :add 6852 INFO [2022-01-10 08:39:36,445] jepsen worker 768 - jepsen.util 768 :invoke :add 6853 INFO [2022-01-10 08:39:36,447] jepsen worker 796 - jepsen.util 796 :invoke :add 6854 INFO [2022-01-10 08:39:36,449] jepsen worker 99 - jepsen.util 99 :invoke :add 6855 INFO [2022-01-10 08:39:36,469] jepsen worker 460 - jepsen.util 460 :info :add 4811 :RequestTimeoutException INFO [2022-01-10 08:39:36,470] jepsen worker 915 - jepsen.util 915 :invoke :add 6856 INFO [2022-01-10 08:39:36,474] jepsen worker 210 - jepsen.util 210 :info :add 4815 :RequestTimeoutException INFO [2022-01-10 08:39:36,474] jepsen worker 436 - jepsen.util 436 :info :add 4813 :RequestTimeoutException INFO [2022-01-10 08:39:36,484] jepsen worker 370 - jepsen.util 370 :info :add 4820 :RequestTimeoutException INFO [2022-01-10 08:39:36,489] jepsen worker 739 - jepsen.util 739 :invoke :add 6857 INFO [2022-01-10 08:39:36,491] jepsen worker 169 - jepsen.util 169 :invoke :add 6858 INFO [2022-01-10 08:39:36,493] jepsen worker 946 - jepsen.util 946 :invoke :add 6859 INFO [2022-01-10 08:39:36,495] jepsen worker 561 - jepsen.util 561 :invoke :add 6860 INFO [2022-01-10 08:39:36,497] jepsen worker 85 - jepsen.util 85 :invoke :add 6861 INFO [2022-01-10 08:39:36,498] jepsen worker 858 - jepsen.util 858 :invoke :add 6862 INFO [2022-01-10 08:39:36,501] jepsen worker 559 - jepsen.util 559 :invoke :add 6863 INFO [2022-01-10 08:39:36,503] jepsen worker 380 - jepsen.util 380 :invoke :add 6864 INFO [2022-01-10 08:39:36,505] jepsen worker 53 - jepsen.util 1053 :invoke :add 6865 INFO [2022-01-10 08:39:36,506] jepsen worker 732 - jepsen.util 732 :invoke :add 6866 INFO [2022-01-10 08:39:36,508] jepsen worker 413 - jepsen.util 413 :invoke :add 6867 INFO [2022-01-10 08:39:36,510] jepsen worker 897 - jepsen.util 897 :invoke :add 6868 INFO [2022-01-10 08:39:36,512] jepsen worker 75 - jepsen.util 75 :invoke :add 6869 INFO [2022-01-10 08:39:36,514] jepsen worker 293 - jepsen.util 293 :invoke :add 6870 INFO [2022-01-10 08:39:36,516] jepsen worker 764 - jepsen.util 764 :invoke :add 6871 INFO [2022-01-10 08:39:36,517] jepsen worker 548 - jepsen.util 548 :invoke :add 6872 INFO [2022-01-10 08:39:36,520] jepsen worker 279 - jepsen.util 279 :invoke :add 6873 INFO [2022-01-10 08:39:36,531] jepsen worker 697 - jepsen.util 697 :info :add 4838 :RequestTimeoutException INFO [2022-01-10 08:39:36,531] jepsen worker 281 - jepsen.util 281 :invoke :add 6874 INFO [2022-01-10 08:39:36,541] jepsen worker 801 - jepsen.util 801 :info :add 4839 :RequestTimeoutException INFO [2022-01-10 08:39:36,545] jepsen worker 210 - jepsen.util 1210 :invoke :add 6875 INFO [2022-01-10 08:39:36,548] jepsen worker 481 - jepsen.util 481 :invoke :add 6876 INFO [2022-01-10 08:39:36,550] jepsen worker 148 - jepsen.util 148 :invoke :add 6877 INFO [2022-01-10 08:39:36,552] jepsen worker 697 - jepsen.util 1697 :invoke :add 6878 INFO [2022-01-10 08:39:36,563] jepsen worker 694 - jepsen.util 694 :info :add 4851 :RequestTimeoutException INFO [2022-01-10 08:39:36,568] jepsen worker 232 - jepsen.util 232 :invoke :add 6879 INFO [2022-01-10 08:39:36,585] jepsen worker 95 - jepsen.util 95 :info :add 4857 :RequestTimeoutException INFO [2022-01-10 08:39:36,589] jepsen worker 931 - jepsen.util 931 :info :add 4855 :RequestTimeoutException INFO [2022-01-10 08:39:36,596] jepsen worker 625 - jepsen.util 625 :ok :add 6418 INFO [2022-01-10 08:39:36,596] jepsen worker 810 - jepsen.util 810 :ok :add 6346 INFO [2022-01-10 08:39:36,594] jepsen worker 490 - jepsen.util 490 :info :add 4863 :RequestTimeoutException INFO [2022-01-10 08:39:36,595] jepsen worker 850 - jepsen.util 850 :ok :add 5153 INFO [2022-01-10 08:39:36,595] jepsen worker 794 - jepsen.util 794 :ok :add 5550 INFO [2022-01-10 08:39:36,595] jepsen worker 11 - jepsen.util 11 :ok :add 6400 INFO [2022-01-10 08:39:36,595] jepsen worker 623 - jepsen.util 623 :ok :add 5930 INFO [2022-01-10 08:39:36,595] jepsen worker 882 - jepsen.util 882 :ok :add 6434 INFO [2022-01-10 08:39:36,595] jepsen worker 565 - jepsen.util 565 :ok :add 6588 INFO [2022-01-10 08:39:36,595] jepsen worker 13 - jepsen.util 13 :ok :add 5621 INFO [2022-01-10 08:39:36,595] jepsen worker 192 - jepsen.util 192 :ok :add 5615 INFO [2022-01-10 08:39:36,595] jepsen worker 761 - jepsen.util 761 :ok :add 5639 INFO [2022-01-10 08:39:36,595] jepsen worker 578 - jepsen.util 578 :ok :add 5785 INFO [2022-01-10 08:39:36,597] jepsen worker 37 - jepsen.util 37 :ok :add 5216 INFO [2022-01-10 08:39:36,595] jepsen worker 164 - jepsen.util 164 :ok :add 5386 INFO [2022-01-10 08:39:36,595] jepsen worker 838 - jepsen.util 838 :ok :add 5167 INFO [2022-01-10 08:39:36,595] jepsen worker 454 - jepsen.util 454 :ok :add 5222 INFO [2022-01-10 08:39:36,595] jepsen worker 21 - jepsen.util 21 :ok :add 5548 INFO [2022-01-10 08:39:36,596] jepsen worker 694 - jepsen.util 1694 :invoke :add 6880 INFO [2022-01-10 08:39:36,596] jepsen worker 588 - jepsen.util 588 :ok :add 5564 INFO [2022-01-10 08:39:36,603] jepsen worker 487 - jepsen.util 487 :invoke :add 6881 INFO [2022-01-10 08:39:36,605] jepsen worker 21 - jepsen.util 21 :invoke :add 6882 INFO [2022-01-10 08:39:36,607] jepsen worker 454 - jepsen.util 454 :invoke :add 6883 INFO [2022-01-10 08:39:36,609] jepsen worker 761 - jepsen.util 761 :invoke :add 6884 INFO [2022-01-10 08:39:36,611] jepsen worker 372 - jepsen.util 372 :invoke :add 6885 INFO [2022-01-10 08:39:36,613] jepsen worker 931 - jepsen.util 1931 :invoke :add 6886 INFO [2022-01-10 08:39:36,614] jepsen worker 370 - jepsen.util 1370 :invoke :add 6887 INFO [2022-01-10 08:39:36,616] jepsen worker 625 - jepsen.util 625 :invoke :add 6888 INFO [2022-01-10 08:39:36,618] jepsen worker 566 - jepsen.util 566 :invoke :add 6889 INFO [2022-01-10 08:39:36,620] jepsen worker 801 - jepsen.util 1801 :invoke :add 6890 INFO [2022-01-10 08:39:36,622] jepsen worker 13 - jepsen.util 13 :invoke :add 6891 INFO [2022-01-10 08:39:36,623] jepsen worker 319 - jepsen.util 319 :invoke :add 6892 INFO [2022-01-10 08:39:36,626] jepsen worker 349 - jepsen.util 349 :invoke :add 6893 INFO [2022-01-10 08:39:36,627] jepsen worker 588 - jepsen.util 588 :invoke :add 6894 INFO [2022-01-10 08:39:36,629] jepsen worker 838 - jepsen.util 838 :invoke :add 6895 INFO [2022-01-10 08:39:36,631] jepsen worker 490 - jepsen.util 1490 :invoke :add 6896 INFO [2022-01-10 08:39:36,647] jepsen worker 602 - jepsen.util 602 :info :add 4884 :RequestTimeoutException INFO [2022-01-10 08:39:36,652] jepsen worker 82 - jepsen.util 82 :info :add 4886 :RequestTimeoutException INFO [2022-01-10 08:39:36,652] jepsen worker 164 - jepsen.util 164 :invoke :add 6897 INFO [2022-01-10 08:39:36,656] jepsen worker 794 - jepsen.util 794 :invoke :add 6898 INFO [2022-01-10 08:39:36,658] jepsen worker 565 - jepsen.util 565 :invoke :add 6899 INFO [2022-01-10 08:39:36,662] jepsen worker 803 - jepsen.util 803 :invoke :add 6900 INFO [2022-01-10 08:39:36,664] jepsen worker 706 - jepsen.util 706 :invoke :add 6901 INFO [2022-01-10 08:39:36,666] jepsen worker 810 - jepsen.util 810 :invoke :add 6902 INFO [2022-01-10 08:39:36,668] jepsen worker 95 - jepsen.util 1095 :invoke :add 6903 INFO [2022-01-10 08:39:36,669] jepsen worker 460 - jepsen.util 1460 :invoke :add 6904 INFO [2022-01-10 08:39:36,672] jepsen worker 146 - jepsen.util 146 :invoke :add 6905 INFO [2022-01-10 08:39:36,673] jepsen worker 82 - jepsen.util 1082 :invoke :add 6906 INFO [2022-01-10 08:39:36,675] jepsen worker 212 - jepsen.util 212 :invoke :add 6907 INFO [2022-01-10 08:39:36,677] jepsen worker 131 - jepsen.util 131 :invoke :add 6908 INFO [2022-01-10 08:39:36,679] jepsen worker 37 - jepsen.util 37 :invoke :add 6909 INFO [2022-01-10 08:39:36,681] jepsen worker 257 - jepsen.util 257 :invoke :add 6910 INFO [2022-01-10 08:39:36,682] jepsen worker 111 - jepsen.util 111 :invoke :add 6911 INFO [2022-01-10 08:39:36,684] jepsen worker 282 - jepsen.util 1282 :invoke :add 6912 INFO [2022-01-10 08:39:36,686] jepsen worker 850 - jepsen.util 850 :invoke :add 6913 INFO [2022-01-10 08:39:36,687] jepsen worker 882 - jepsen.util 882 :invoke :add 6914 INFO [2022-01-10 08:39:36,689] jepsen worker 578 - jepsen.util 578 :invoke :add 6915 INFO [2022-01-10 08:39:36,691] jepsen worker 192 - jepsen.util 192 :invoke :add 6916 INFO [2022-01-10 08:39:36,693] jepsen worker 623 - jepsen.util 623 :invoke :add 6917 INFO [2022-01-10 08:39:36,696] jepsen worker 436 - jepsen.util 1436 :invoke :add 6918 INFO [2022-01-10 08:39:36,698] jepsen worker 11 - jepsen.util 11 :invoke :add 6919 INFO [2022-01-10 08:39:36,700] jepsen worker 602 - jepsen.util 1602 :invoke :add 6920 INFO [2022-01-10 08:39:36,702] jepsen worker 685 - jepsen.util 685 :invoke :add 6921 INFO [2022-01-10 08:39:36,721] jepsen worker 716 - jepsen.util 716 :info :add 4916 :RequestTimeoutException INFO [2022-01-10 08:39:36,725] jepsen worker 904 - jepsen.util 904 :info :add 4914 :RequestTimeoutException INFO [2022-01-10 08:39:36,787] jepsen worker 299 - jepsen.util 299 :info :add 4922 :RequestTimeoutException INFO [2022-01-10 08:39:36,788] jepsen worker 406 - jepsen.util 406 :info :add 4920 :RequestTimeoutException INFO [2022-01-10 08:39:36,793] jepsen worker 716 - jepsen.util 1716 :invoke :add 6922 INFO [2022-01-10 08:39:36,798] jepsen worker 350 - jepsen.util 350 :info :add 4938 :RequestTimeoutException INFO [2022-01-10 08:39:36,801] jepsen worker 299 - jepsen.util 1299 :invoke :add 6923 INFO [2022-01-10 08:39:36,803] jepsen worker 406 - jepsen.util 1406 :invoke :add 6924 INFO [2022-01-10 08:39:36,805] jepsen worker 350 - jepsen.util 1350 :invoke :add 6925 INFO [2022-01-10 08:39:36,807] jepsen worker 904 - jepsen.util 1904 :invoke :add 6926 INFO [2022-01-10 08:39:36,820] jepsen worker 584 - jepsen.util 584 :info :add 4948 :RequestTimeoutException INFO [2022-01-10 08:39:36,825] jepsen worker 441 - jepsen.util 441 :info :add 4950 :RequestTimeoutException INFO [2022-01-10 08:39:36,833] jepsen worker 805 - jepsen.util 805 :info :add 4953 :RequestTimeoutException INFO [2022-01-10 08:39:36,840] jepsen worker 805 - jepsen.util 1805 :invoke :add 6927 INFO [2022-01-10 08:39:36,840] jepsen worker 905 - jepsen.util 905 :ok :add 5151 INFO [2022-01-10 08:39:36,840] jepsen worker 733 - jepsen.util 733 :ok :add 5617 INFO [2022-01-10 08:39:36,840] jepsen worker 394 - jepsen.util 394 :ok :add 5165 INFO [2022-01-10 08:39:36,840] jepsen worker 534 - jepsen.util 534 :ok :add 6189 INFO [2022-01-10 08:39:36,840] jepsen worker 94 - jepsen.util 94 :ok :add 5623 INFO [2022-01-10 08:39:36,840] jepsen worker 877 - jepsen.util 877 :ok :add 6019 INFO [2022-01-10 08:39:36,840] jepsen worker 521 - jepsen.util 521 :ok :add 5149 INFO [2022-01-10 08:39:36,840] jepsen worker 126 - jepsen.util 126 :ok :add 5238 INFO [2022-01-10 08:39:36,841] jepsen worker 2 - jepsen.util 2 :ok :add 5552 INFO [2022-01-10 08:39:36,841] jepsen worker 389 - jepsen.util 389 :ok :add 5566 INFO [2022-01-10 08:39:36,841] jepsen worker 881 - jepsen.util 881 :ok :add 5220 INFO [2022-01-10 08:39:36,841] jepsen worker 246 - jepsen.util 246 :ok :add 6368 INFO [2022-01-10 08:39:36,841] jepsen worker 443 - jepsen.util 443 :ok :add 6035 INFO [2022-01-10 08:39:36,841] jepsen worker 338 - jepsen.util 338 :ok :add 5932 INFO [2022-01-10 08:39:36,841] jepsen worker 629 - jepsen.util 629 :ok :add 6344 INFO [2022-01-10 08:39:36,842] jepsen worker 332 - jepsen.util 332 :ok :add 5384 INFO [2022-01-10 08:39:36,842] jepsen worker 441 - jepsen.util 1441 :invoke :add 6928 INFO [2022-01-10 08:39:36,848] jepsen worker 394 - jepsen.util 394 :invoke :add 6929 INFO [2022-01-10 08:39:36,856] jepsen worker 733 - jepsen.util 733 :invoke :add 6930 INFO [2022-01-10 08:39:36,860] jepsen worker 641 - jepsen.util 641 :info :add 4964 :RequestTimeoutException INFO [2022-01-10 08:39:36,870] jepsen worker 205 - jepsen.util 205 :info :add 4967 :RequestTimeoutException INFO [2022-01-10 08:39:36,870] jepsen worker 389 - jepsen.util 389 :invoke :add 6931 INFO [2022-01-10 08:39:36,873] jepsen worker 521 - jepsen.util 521 :invoke :add 6932 INFO [2022-01-10 08:39:36,875] jepsen worker 2 - jepsen.util 2 :invoke :add 6933 INFO [2022-01-10 08:39:36,878] jepsen worker 246 - jepsen.util 246 :invoke :add 6934 INFO [2022-01-10 08:39:36,879] jepsen worker 641 - jepsen.util 1641 :invoke :add 6935 INFO [2022-01-10 08:39:36,881] jepsen worker 94 - jepsen.util 94 :invoke :add 6936 INFO [2022-01-10 08:39:36,883] jepsen worker 881 - jepsen.util 881 :invoke :add 6937 INFO [2022-01-10 08:39:36,885] jepsen worker 126 - jepsen.util 126 :invoke :add 6938 INFO [2022-01-10 08:39:36,887] jepsen worker 443 - jepsen.util 443 :invoke :add 6939 INFO [2022-01-10 08:39:36,889] jepsen worker 629 - jepsen.util 629 :invoke :add 6940 INFO [2022-01-10 08:39:36,891] jepsen worker 584 - jepsen.util 1584 :invoke :add 6941 INFO [2022-01-10 08:39:36,892] jepsen worker 338 - jepsen.util 338 :invoke :add 6942 INFO [2022-01-10 08:39:36,894] jepsen worker 534 - jepsen.util 534 :invoke :add 6943 INFO [2022-01-10 08:39:36,896] jepsen worker 205 - jepsen.util 1205 :invoke :add 6944 INFO [2022-01-10 08:39:36,898] jepsen worker 905 - jepsen.util 905 :invoke :add 6945 INFO [2022-01-10 08:39:36,900] jepsen worker 877 - jepsen.util 877 :invoke :add 6946 INFO [2022-01-10 08:39:36,902] jepsen worker 332 - jepsen.util 332 :invoke :add 6947 INFO [2022-01-10 08:39:36,913] jepsen worker 886 - jepsen.util 886 :info :add 4984 :RequestTimeoutException INFO [2022-01-10 08:39:36,922] jepsen worker 656 - jepsen.util 656 :info :add 4983 :RequestTimeoutException INFO [2022-01-10 08:39:36,931] jepsen worker 659 - jepsen.util 659 :info :add 4985 :RequestTimeoutException INFO [2022-01-10 08:39:36,939] jepsen worker 656 - jepsen.util 1656 :invoke :add 6948 INFO [2022-01-10 08:39:36,946] jepsen worker 886 - jepsen.util 1886 :invoke :add 6949 INFO [2022-01-10 08:39:36,951] jepsen worker 540 - jepsen.util 540 :info :add 4999 :RequestTimeoutException INFO [2022-01-10 08:39:36,953] jepsen worker 659 - jepsen.util 1659 :invoke :add 6950 INFO [2022-01-10 08:39:36,955] jepsen worker 540 - jepsen.util 1540 :invoke :add 6951 INFO [2022-01-10 08:39:36,973] jepsen worker 735 - jepsen.util 735 :info :add 5009 :RequestTimeoutException INFO [2022-01-10 08:39:36,987] jepsen worker 1 - jepsen.util 1 :info :add 5010 :RequestTimeoutException INFO [2022-01-10 08:39:36,987] jepsen worker 475 - jepsen.util 475 :info :add 5012 :RequestTimeoutException INFO [2022-01-10 08:39:36,991] jepsen worker 799 - jepsen.util 799 :info :add 5008 :RequestTimeoutException INFO [2022-01-10 08:39:36,995] jepsen worker 735 - jepsen.util 1735 :invoke :add 6952 INFO [2022-01-10 08:39:36,996] jepsen worker 475 - jepsen.util 1475 :invoke :add 6953 INFO [2022-01-10 08:39:36,998] jepsen worker 1 - jepsen.util 1001 :invoke :add 6954 INFO [2022-01-10 08:39:37,000] jepsen worker 799 - jepsen.util 1799 :invoke :add 6955 INFO [2022-01-10 08:39:37,014] jepsen worker 385 - jepsen.util 385 :info :add 5026 :RequestTimeoutException INFO [2022-01-10 08:39:37,018] jepsen worker 450 - jepsen.util 450 :info :add 5024 :RequestTimeoutException INFO [2022-01-10 08:39:37,019] jepsen worker 500 - jepsen.util 500 :info :add 5025 :RequestTimeoutException INFO [2022-01-10 08:39:37,024] jepsen worker 500 - jepsen.util 1500 :invoke :add 6956 INFO [2022-01-10 08:39:37,028] jepsen worker 385 - jepsen.util 1385 :invoke :add 6957 INFO [2022-01-10 08:39:37,029] jepsen worker 450 - jepsen.util 1450 :invoke :add 6958 INFO [2022-01-10 08:39:37,073] jepsen worker 593 - jepsen.util 593 :ok :add 6292 INFO [2022-01-10 08:39:37,074] jepsen worker 741 - jepsen.util 741 :ok :add 5252 INFO [2022-01-10 08:39:37,074] jepsen worker 386 - jepsen.util 386 :ok :add 6302 INFO [2022-01-10 08:39:37,074] jepsen worker 237 - jepsen.util 237 :ok :add 5958 INFO [2022-01-10 08:39:37,074] jepsen worker 970 - jepsen.util 970 :ok :add 6444 INFO [2022-01-10 08:39:37,075] jepsen worker 369 - jepsen.util 369 :ok :add 6689 INFO [2022-01-10 08:39:37,075] jepsen worker 542 - jepsen.util 542 :ok :add 6470 INFO [2022-01-10 08:39:37,075] jepsen worker 339 - jepsen.util 339 :ok :add 5123 INFO [2022-01-10 08:39:37,075] jepsen worker 8 - jepsen.util 8 :ok :add 6336 INFO [2022-01-10 08:39:37,075] jepsen worker 567 - jepsen.util 567 :ok :add 5651 INFO [2022-01-10 08:39:37,075] jepsen worker 220 - jepsen.util 220 :ok :add 5117 INFO [2022-01-10 08:39:37,075] jepsen worker 976 - jepsen.util 976 :ok :add 6468 INFO [2022-01-10 08:39:37,076] jepsen worker 736 - jepsen.util 736 :ok :add 5266 INFO [2022-01-10 08:39:37,076] jepsen worker 647 - jepsen.util 647 :ok :add 5940 INFO [2022-01-10 08:39:37,076] jepsen worker 705 - jepsen.util 705 :ok :add 5484 INFO [2022-01-10 08:39:37,076] jepsen worker 284 - jepsen.util 284 :ok :add 5974 INFO [2022-01-10 08:39:37,092] jepsen worker 16 - jepsen.util 16 :info :add 5054 :RequestTimeoutException INFO [2022-01-10 08:39:37,101] jepsen worker 416 - jepsen.util 416 :info :add 5057 :RequestTimeoutException INFO [2022-01-10 08:39:37,147] jepsen worker 793 - jepsen.util 793 :info :add 5063 :RequestTimeoutException INFO [2022-01-10 08:39:37,148] jepsen worker 237 - jepsen.util 237 :invoke :add 6959 INFO [2022-01-10 08:39:37,149] jepsen worker 752 - jepsen.util 752 :info :add 5080 :RequestTimeoutException INFO [2022-01-10 08:39:37,149] jepsen worker 309 - jepsen.util 309 :info :add 5060 :RequestTimeoutException INFO [2022-01-10 08:39:37,152] jepsen worker 251 - jepsen.util 251 :info :add 5081 :RequestTimeoutException INFO [2022-01-10 08:39:37,155] jepsen worker 309 - jepsen.util 1309 :invoke :add 6960 INFO [2022-01-10 08:39:37,163] jepsen worker 967 - jepsen.util 967 :info :add 5078 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,164] jepsen worker 705 - jepsen.util 705 :invoke :add 6961 INFO [2022-01-10 08:39:37,169] jepsen worker 951 - jepsen.util 951 :info :add 5098 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,169] jepsen worker 478 - jepsen.util 478 :info :add 5085 :RequestTimeoutException INFO [2022-01-10 08:39:37,174] jepsen worker 785 - jepsen.util 785 :info :add 5099 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,181] jepsen worker 421 - jepsen.util 421 :info :add 5483 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,186] jepsen worker 967 - jepsen.util 1967 :invoke :add 6962 INFO [2022-01-10 08:39:37,186] jepsen worker 976 - jepsen.util 976 :invoke :add 6963 INFO [2022-01-10 08:39:37,188] jepsen worker 951 - jepsen.util 1951 :invoke :add 6964 INFO [2022-01-10 08:39:37,190] jepsen worker 647 - jepsen.util 647 :invoke :add 6965 INFO [2022-01-10 08:39:37,192] jepsen worker 416 - jepsen.util 1416 :invoke :add 6966 INFO [2022-01-10 08:39:37,194] jepsen worker 567 - jepsen.util 567 :invoke :add 6967 INFO [2022-01-10 08:39:37,195] jepsen worker 793 - jepsen.util 1793 :invoke :add 6968 INFO [2022-01-10 08:39:37,198] jepsen worker 593 - jepsen.util 593 :invoke :add 6969 INFO [2022-01-10 08:39:37,199] jepsen worker 785 - jepsen.util 1785 :invoke :add 6970 INFO [2022-01-10 08:39:37,201] jepsen worker 970 - jepsen.util 970 :invoke :add 6971 INFO [2022-01-10 08:39:37,203] jepsen worker 741 - jepsen.util 741 :invoke :add 6972 INFO [2022-01-10 08:39:37,205] jepsen worker 752 - jepsen.util 1752 :invoke :add 6973 INFO [2022-01-10 08:39:37,207] jepsen worker 386 - jepsen.util 386 :invoke :add 6974 INFO [2022-01-10 08:39:37,209] jepsen worker 542 - jepsen.util 542 :invoke :add 6975 INFO [2022-01-10 08:39:37,210] jepsen worker 339 - jepsen.util 339 :invoke :add 6976 INFO [2022-01-10 08:39:37,212] jepsen worker 284 - jepsen.util 284 :invoke :add 6977 INFO [2022-01-10 08:39:37,214] jepsen worker 220 - jepsen.util 220 :invoke :add 6978 INFO [2022-01-10 08:39:37,216] jepsen worker 251 - jepsen.util 1251 :invoke :add 6979 INFO [2022-01-10 08:39:37,218] jepsen worker 8 - jepsen.util 8 :invoke :add 6980 INFO [2022-01-10 08:39:37,220] jepsen worker 421 - jepsen.util 1421 :invoke :add 6981 INFO [2022-01-10 08:39:37,222] jepsen worker 16 - jepsen.util 1016 :invoke :add 6982 INFO [2022-01-10 08:39:37,222] jepsen worker 267 - jepsen.util 267 :info :add 5108 :RequestTimeoutException INFO [2022-01-10 08:39:37,223] jepsen worker 926 - jepsen.util 926 :info :add 5109 :RequestTimeoutException INFO [2022-01-10 08:39:37,223] jepsen worker 478 - jepsen.util 1478 :invoke :add 6983 INFO [2022-01-10 08:39:37,227] jepsen worker 369 - jepsen.util 369 :invoke :add 6984 INFO [2022-01-10 08:39:37,229] jepsen worker 736 - jepsen.util 736 :invoke :add 6985 INFO [2022-01-10 08:39:37,230] jepsen worker 926 - jepsen.util 1926 :invoke :add 6986 INFO [2022-01-10 08:39:37,232] jepsen worker 241 - jepsen.util 241 :info :add 5110 :RequestTimeoutException INFO [2022-01-10 08:39:37,232] jepsen worker 267 - jepsen.util 1267 :invoke :add 6987 INFO [2022-01-10 08:39:37,233] jepsen worker 271 - jepsen.util 271 :info :add 5111 :RequestTimeoutException INFO [2022-01-10 08:39:37,237] jepsen worker 271 - jepsen.util 1271 :invoke :add 6988 INFO [2022-01-10 08:39:37,238] jepsen worker 241 - jepsen.util 1241 :invoke :add 6989 INFO [2022-01-10 08:39:37,244] jepsen worker 392 - jepsen.util 392 :info :add 5115 :RequestTimeoutException INFO [2022-01-10 08:39:37,249] jepsen worker 392 - jepsen.util 1392 :invoke :add 6990 INFO [2022-01-10 08:39:37,262] jepsen worker 545 - jepsen.util 545 :info :add 5125 :RequestTimeoutException INFO [2022-01-10 08:39:37,263] jepsen worker 505 - jepsen.util 505 :info :add 5124 :RequestTimeoutException INFO [2022-01-10 08:39:37,266] jepsen worker 505 - jepsen.util 1505 :invoke :add 6991 INFO [2022-01-10 08:39:37,269] jepsen worker 545 - jepsen.util 1545 :invoke :add 6992 INFO [2022-01-10 08:39:37,297] jepsen worker 666 - jepsen.util 666 :ok :add 6109 INFO [2022-01-10 08:39:37,297] jepsen worker 650 - jepsen.util 650 :ok :add 6278 INFO [2022-01-10 08:39:37,297] jepsen worker 758 - jepsen.util 758 :ok :add 6125 INFO [2022-01-10 08:39:37,297] jepsen worker 422 - jepsen.util 422 :ok :add 6099 INFO [2022-01-10 08:39:37,297] jepsen worker 568 - jepsen.util 568 :ok :add 6081 INFO [2022-01-10 08:39:37,297] jepsen worker 116 - jepsen.util 116 :ok :add 6254 INFO [2022-01-10 08:39:37,297] jepsen worker 28 - jepsen.util 28 :ok :add 6526 INFO [2022-01-10 08:39:37,297] jepsen worker 488 - jepsen.util 488 :ok :add 6512 INFO [2022-01-10 08:39:37,297] jepsen worker 626 - jepsen.util 626 :ok :add 6111 INFO [2022-01-10 08:39:37,297] jepsen worker 78 - jepsen.util 78 :ok :add 6260 INFO [2022-01-10 08:39:37,297] jepsen worker 746 - jepsen.util 746 :ok :add 6482 INFO [2022-01-10 08:39:37,303] jepsen worker 429 - jepsen.util 429 :info :add 5254 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,308] jepsen worker 600 - jepsen.util 600 :info :add 5182 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,313] jepsen worker 456 - jepsen.util 456 :info :add 5279 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,313] jepsen worker 789 - jepsen.util 789 :info :add 5139 :RequestTimeoutException INFO [2022-01-10 08:39:37,314] jepsen worker 42 - jepsen.util 42 :info :add 5281 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,320] jepsen worker 832 - jepsen.util 832 :info :add 5278 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,329] jepsen worker 405 - jepsen.util 405 :info :add 5312 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,335] jepsen worker 133 - jepsen.util 133 :info :add 5379 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,340] jepsen worker 615 - jepsen.util 615 :info :add 5208 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,345] jepsen worker 141 - jepsen.util 141 :info :add 5311 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,346] jepsen worker 878 - jepsen.util 878 :info :add 5157 :RequestTimeoutException INFO [2022-01-10 08:39:37,351] jepsen worker 28 - jepsen.util 28 :invoke :add 6993 INFO [2022-01-10 08:39:37,353] jepsen worker 568 - jepsen.util 568 :invoke :add 6994 INFO [2022-01-10 08:39:37,355] jepsen worker 456 - jepsen.util 1456 :invoke :add 6995 INFO [2022-01-10 08:39:37,356] jepsen worker 141 - jepsen.util 1141 :invoke :add 6996 INFO [2022-01-10 08:39:37,358] jepsen worker 405 - jepsen.util 1405 :invoke :add 6997 INFO [2022-01-10 08:39:37,359] jepsen worker 666 - jepsen.util 666 :invoke :add 6998 INFO [2022-01-10 08:39:37,362] jepsen worker 746 - jepsen.util 746 :invoke :add 6999 INFO [2022-01-10 08:39:37,364] jepsen worker 802 - jepsen.util 802 :info :add 5163 :RequestTimeoutException INFO [2022-01-10 08:39:37,364] jepsen worker 832 - jepsen.util 1832 :invoke :add 7000 INFO [2022-01-10 08:39:37,367] jepsen worker 615 - jepsen.util 1615 :invoke :add 7001 INFO [2022-01-10 08:39:37,369] jepsen worker 78 - jepsen.util 78 :invoke :add 7002 INFO [2022-01-10 08:39:37,371] jepsen worker 789 - jepsen.util 1789 :invoke :add 7003 INFO [2022-01-10 08:39:37,372] jepsen worker 116 - jepsen.util 116 :invoke :add 7004 INFO [2022-01-10 08:39:37,374] jepsen worker 626 - jepsen.util 626 :invoke :add 7005 INFO [2022-01-10 08:39:37,376] jepsen worker 878 - jepsen.util 1878 :invoke :add 7006 INFO [2022-01-10 08:39:37,378] jepsen worker 133 - jepsen.util 1133 :invoke :add 7007 INFO [2022-01-10 08:39:37,379] jepsen worker 758 - jepsen.util 758 :invoke :add 7008 INFO [2022-01-10 08:39:37,381] jepsen worker 802 - jepsen.util 1802 :invoke :add 7009 INFO [2022-01-10 08:39:37,384] jepsen worker 429 - jepsen.util 1429 :invoke :add 7010 INFO [2022-01-10 08:39:37,385] jepsen worker 488 - jepsen.util 488 :invoke :add 7011 INFO [2022-01-10 08:39:37,387] jepsen worker 600 - jepsen.util 1600 :invoke :add 7012 INFO [2022-01-10 08:39:37,389] jepsen worker 650 - jepsen.util 650 :invoke :add 7013 INFO [2022-01-10 08:39:37,391] jepsen worker 42 - jepsen.util 1042 :invoke :add 7014 INFO [2022-01-10 08:39:37,393] jepsen worker 422 - jepsen.util 422 :invoke :add 7015 INFO [2022-01-10 08:39:37,403] jepsen worker 515 - jepsen.util 515 :info :add 5180 :RequestTimeoutException INFO [2022-01-10 08:39:37,405] jepsen worker 515 - jepsen.util 1515 :invoke :add 7016 INFO [2022-01-10 08:39:37,411] jepsen worker 535 - jepsen.util 535 :info :add 5181 :RequestTimeoutException INFO [2022-01-10 08:39:37,413] jepsen worker 535 - jepsen.util 1535 :invoke :add 7017 INFO [2022-01-10 08:39:37,435] jepsen worker 555 - jepsen.util 555 :info :add 5325 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,445] jepsen worker 860 - jepsen.util 860 :info :add 5363 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,445] jepsen worker 125 - jepsen.util 125 :info :add 5261 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,450] jepsen worker 845 - jepsen.util 845 :info :add 5380 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,455] jepsen worker 619 - jepsen.util 619 :info :add 5460 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,455] jepsen worker 507 - jepsen.util 507 :info :add 5413 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,456] jepsen worker 827 - jepsen.util 827 :info :add 5199 :RequestTimeoutException INFO [2022-01-10 08:39:37,456] jepsen worker 806 - jepsen.util 806 :info :add 5198 :RequestTimeoutException INFO [2022-01-10 08:39:37,478] jepsen worker 552 - jepsen.util 552 :info :add 5427 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,479] jepsen worker 921 - jepsen.util 921 :info :add 5463 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,480] jepsen worker 918 - jepsen.util 918 :info :add 5513 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,488] jepsen worker 619 - jepsen.util 1619 :invoke :add 7018 INFO [2022-01-10 08:39:37,490] jepsen worker 918 - jepsen.util 1918 :invoke :add 7019 INFO [2022-01-10 08:39:37,492] jepsen worker 827 - jepsen.util 1827 :invoke :add 7020 INFO [2022-01-10 08:39:37,499] jepsen worker 15 - jepsen.util 15 :info :add 5613 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,500] jepsen worker 125 - jepsen.util 1125 :invoke :add 7021 INFO [2022-01-10 08:39:37,503] jepsen worker 507 - jepsen.util 1507 :invoke :add 7022 INFO [2022-01-10 08:39:37,505] jepsen worker 921 - jepsen.util 1921 :invoke :add 7023 INFO [2022-01-10 08:39:37,506] jepsen worker 15 - jepsen.util 1015 :invoke :add 7024 INFO [2022-01-10 08:39:37,508] jepsen worker 555 - jepsen.util 1555 :invoke :add 7025 INFO [2022-01-10 08:39:37,508] jepsen worker 358 - jepsen.util 358 :ok :add 5248 INFO [2022-01-10 08:39:37,509] jepsen worker 797 - jepsen.util 797 :ok :add 5250 INFO [2022-01-10 08:39:37,509] jepsen worker 944 - jepsen.util 944 :ok :add 6045 INFO [2022-01-10 08:39:37,509] jepsen worker 727 - jepsen.util 727 :ok :add 6446 INFO [2022-01-10 08:39:37,509] jepsen worker 512 - jepsen.util 512 :ok :add 6318 INFO [2022-01-10 08:39:37,509] jepsen worker 387 - jepsen.util 387 :ok :add 6071 INFO [2022-01-10 08:39:37,509] jepsen worker 118 - jepsen.util 118 :ok :add 6472 INFO [2022-01-10 08:39:37,509] jepsen worker 798 - jepsen.util 798 :ok :add 5653 INFO [2022-01-10 08:39:37,509] jepsen worker 471 - jepsen.util 471 :ok :add 6703 INFO [2022-01-10 08:39:37,509] jepsen worker 917 - jepsen.util 917 :ok :add 6288 INFO [2022-01-10 08:39:37,510] jepsen worker 821 - jepsen.util 821 :ok :add 5942 INFO [2022-01-10 08:39:37,510] jepsen worker 621 - jepsen.util 621 :ok :add 5976 INFO [2022-01-10 08:39:37,510] jepsen worker 765 - jepsen.util 765 :ok :add 5486 INFO [2022-01-10 08:39:37,510] jepsen worker 495 - jepsen.util 495 :ok :add 5667 INFO [2022-01-10 08:39:37,510] jepsen worker 552 - jepsen.util 1552 :invoke :add 7026 INFO [2022-01-10 08:39:37,515] jepsen worker 806 - jepsen.util 1806 :invoke :add 7027 INFO [2022-01-10 08:39:37,517] jepsen worker 798 - jepsen.util 798 :invoke :add 7028 INFO [2022-01-10 08:39:37,518] jepsen worker 387 - jepsen.util 387 :invoke :add 7029 INFO [2022-01-10 08:39:37,521] jepsen worker 471 - jepsen.util 471 :invoke :add 7030 INFO [2022-01-10 08:39:37,522] jepsen worker 765 - jepsen.util 765 :invoke :add 7031 INFO [2022-01-10 08:39:37,524] jepsen worker 860 - jepsen.util 1860 :invoke :add 7032 INFO [2022-01-10 08:39:37,526] jepsen worker 118 - jepsen.util 118 :invoke :add 7033 INFO [2022-01-10 08:39:37,528] jepsen worker 358 - jepsen.util 358 :invoke :add 7034 INFO [2022-01-10 08:39:37,531] jepsen worker 944 - jepsen.util 944 :invoke :add 7035 INFO [2022-01-10 08:39:37,533] jepsen worker 845 - jepsen.util 1845 :invoke :add 7036 INFO [2022-01-10 08:39:37,535] jepsen worker 727 - jepsen.util 727 :invoke :add 7037 INFO [2022-01-10 08:39:37,537] jepsen worker 917 - jepsen.util 917 :invoke :add 7038 INFO [2022-01-10 08:39:37,539] jepsen worker 821 - jepsen.util 821 :invoke :add 7039 INFO [2022-01-10 08:39:37,541] jepsen worker 797 - jepsen.util 797 :invoke :add 7040 INFO [2022-01-10 08:39:37,542] jepsen worker 621 - jepsen.util 621 :invoke :add 7041 INFO [2022-01-10 08:39:37,545] jepsen worker 512 - jepsen.util 512 :invoke :add 7042 INFO [2022-01-10 08:39:37,546] jepsen worker 495 - jepsen.util 495 :invoke :add 7043 INFO [2022-01-10 08:39:37,593] jepsen worker 896 - jepsen.util 896 :info :add 5255 :RequestTimeoutException INFO [2022-01-10 08:39:37,597] jepsen worker 896 - jepsen.util 1896 :invoke :add 7044 INFO [2022-01-10 08:39:37,603] jepsen worker 302 - jepsen.util 302 :info :add 5260 :RequestTimeoutException INFO [2022-01-10 08:39:37,606] jepsen worker 302 - jepsen.util 1302 :invoke :add 7045 INFO [2022-01-10 08:39:37,663] jepsen worker 195 - jepsen.util 195 :info :add 5282 :RequestTimeoutException INFO [2022-01-10 08:39:37,666] jepsen worker 195 - jepsen.util 1195 :invoke :add 7046 INFO [2022-01-10 08:39:37,676] jepsen worker 388 - jepsen.util 388 :ok :add 5657 INFO [2022-01-10 08:39:37,676] jepsen worker 262 - jepsen.util 262 :ok :add 5663 INFO [2022-01-10 08:39:37,676] jepsen worker 808 - jepsen.util 808 :ok :add 5972 INFO [2022-01-10 08:39:37,676] jepsen worker 691 - jepsen.util 691 :ok :add 5482 INFO [2022-01-10 08:39:37,679] jepsen worker 691 - jepsen.util 691 :invoke :add 7047 INFO [2022-01-10 08:39:37,680] jepsen worker 388 - jepsen.util 388 :invoke :add 7048 INFO [2022-01-10 08:39:37,682] jepsen worker 808 - jepsen.util 808 :invoke :add 7049 INFO [2022-01-10 08:39:37,684] jepsen worker 262 - jepsen.util 262 :invoke :add 7050 INFO [2022-01-10 08:39:37,713] jepsen worker 283 - jepsen.util 283 :info :add 5299 :RequestTimeoutException INFO [2022-01-10 08:39:37,716] jepsen worker 283 - jepsen.util 1283 :invoke :add 7051 INFO [2022-01-10 08:39:37,733] jepsen worker 728 - jepsen.util 728 :info :add 5309 :RequestTimeoutException INFO [2022-01-10 08:39:37,735] jepsen worker 728 - jepsen.util 1728 :invoke :add 7052 INFO [2022-01-10 08:39:37,782] jepsen worker 335 - jepsen.util 335 :info :add 5699 :SyncWriteAmbiguous INFO [2022-01-10 08:39:37,783] jepsen worker 48 - jepsen.util 48 :info :add 5326 :RequestTimeoutException INFO [2022-01-10 08:39:37,786] jepsen worker 48 - jepsen.util 1048 :invoke :add 7053 INFO [2022-01-10 08:39:37,787] jepsen worker 335 - jepsen.util 1335 :invoke :add 7054 INFO [2022-01-10 08:39:37,817] jepsen worker 977 - jepsen.util 977 :ok :add 6460 INFO [2022-01-10 08:39:37,817] jepsen worker 127 - jepsen.util 127 :ok :add 6057 INFO [2022-01-10 08:39:37,817] jepsen worker 486 - jepsen.util 486 :ok :add 5950 INFO [2022-01-10 08:39:37,817] jepsen worker 35 - jepsen.util 35 :ok :add 6063 INFO [2022-01-10 08:39:37,818] jepsen worker 857 - jepsen.util 857 :ok :add 6454 INFO [2022-01-10 08:39:37,818] jepsen worker 342 - jepsen.util 342 :ok :add 5948 INFO [2022-01-10 08:39:37,818] jepsen worker 204 - jepsen.util 204 :ok :add 6478 INFO [2022-01-10 08:39:37,818] jepsen worker 504 - jepsen.util 504 :ok :add 6326 INFO [2022-01-10 08:39:37,818] jepsen worker 437 - jepsen.util 437 :ok :add 6312 INFO [2022-01-10 08:39:37,818] jepsen worker 607 - jepsen.util 607 :ok :add 6282 INFO [2022-01-10 08:39:37,819] jepsen worker 300 - jepsen.util 300 :ok :add 6480 INFO [2022-01-10 08:39:37,819] jepsen worker 523 - jepsen.util 523 :ok :add 6113 INFO [2022-01-10 08:39:37,819] jepsen worker 157 - jepsen.util 157 :ok :add 6510 INFO [2022-01-10 08:39:37,820] jepsen worker 840 - jepsen.util 840 :ok :add 6083 INFO [2022-01-10 08:39:37,820] jepsen worker 549 - jepsen.util 549 :ok :add 6256 INFO [2022-01-10 08:39:37,820] jepsen worker 68 - jepsen.util 68 :ok :add 6262 INFO [2022-01-10 08:39:37,820] jepsen worker 516 - jepsen.util 516 :ok :add 6508 INFO [2022-01-10 08:39:37,820] jepsen worker 485 - jepsen.util 485 :ok :add 6524 INFO [2022-01-10 08:39:37,822] jepsen worker 320 - jepsen.util 320 :ok :add 5709 INFO [2022-01-10 08:39:37,822] jepsen worker 907 - jepsen.util 907 :ok :add 5478 INFO [2022-01-10 08:39:37,822] jepsen worker 280 - jepsen.util 280 :ok :add 5725 INFO [2022-01-10 08:39:37,822] jepsen worker 497 - jepsen.util 497 :ok :add 5711 INFO [2022-01-10 08:39:37,822] jepsen worker 221 - jepsen.util 221 :ok :add 5800 INFO [2022-01-10 08:39:37,823] jepsen worker 87 - jepsen.util 87 :ok :add 5454 INFO [2022-01-10 08:39:37,824] jepsen worker 263 - jepsen.util 263 :ok :add 5984 INFO [2022-01-10 08:39:37,824] jepsen worker 813 - jepsen.util 813 :ok :add 6498 INFO [2022-01-10 08:39:37,825] jepsen worker 390 - jepsen.util 390 :ok :add 6127 INFO [2022-01-10 08:39:37,828] jepsen worker 485 - jepsen.util 485 :invoke :add 7055 INFO [2022-01-10 08:39:37,830] jepsen worker 204 - jepsen.util 204 :invoke :add 7056 INFO [2022-01-10 08:39:37,832] jepsen worker 437 - jepsen.util 437 :invoke :add 7057 INFO [2022-01-10 08:39:37,834] jepsen worker 504 - jepsen.util 504 :invoke :add 7058 INFO [2022-01-10 08:39:37,836] jepsen worker 35 - jepsen.util 35 :invoke :add 7059 INFO [2022-01-10 08:39:37,838] jepsen worker 87 - jepsen.util 87 :invoke :add 7060 INFO [2022-01-10 08:39:37,839] jepsen worker 977 - jepsen.util 977 :invoke :add 7061 INFO [2022-01-10 08:39:37,841] jepsen worker 997 - jepsen.util 997 :info :add 5348 :RequestTimeoutException INFO [2022-01-10 08:39:37,841] jepsen worker 813 - jepsen.util 813 :invoke :add 7062 INFO [2022-01-10 08:39:37,844] jepsen worker 342 - jepsen.util 342 :invoke :add 7063 INFO [2022-01-10 08:39:37,846] jepsen worker 127 - jepsen.util 127 :invoke :add 7064 INFO [2022-01-10 08:39:37,848] jepsen worker 549 - jepsen.util 549 :invoke :add 7065 INFO [2022-01-10 08:39:37,850] jepsen worker 997 - jepsen.util 1997 :invoke :add 7066 INFO [2022-01-10 08:39:37,850] jepsen worker 398 - jepsen.util 398 :info :add 5352 :RequestTimeoutException INFO [2022-01-10 08:39:37,851] jepsen worker 829 - jepsen.util 829 :info :add 5353 :RequestTimeoutException INFO [2022-01-10 08:39:37,852] jepsen worker 320 - jepsen.util 320 :invoke :add 7067 INFO [2022-01-10 08:39:37,855] jepsen worker 907 - jepsen.util 907 :invoke :add 7068 INFO [2022-01-10 08:39:37,857] jepsen worker 68 - jepsen.util 68 :invoke :add 7069 INFO [2022-01-10 08:39:37,859] jepsen worker 840 - jepsen.util 840 :invoke :add 7070 INFO [2022-01-10 08:39:37,860] jepsen worker 290 - jepsen.util 290 :info :add 5354 :RequestTimeoutException INFO [2022-01-10 08:39:37,861] jepsen worker 108 - jepsen.util 108 :info :add 5357 :RequestTimeoutException INFO [2022-01-10 08:39:37,861] jepsen worker 56 - jepsen.util 56 :info :add 5355 :RequestTimeoutException INFO [2022-01-10 08:39:37,862] jepsen worker 486 - jepsen.util 486 :invoke :add 7071 INFO [2022-01-10 08:39:37,866] jepsen worker 398 - jepsen.util 1398 :invoke :add 7072 INFO [2022-01-10 08:39:37,868] jepsen worker 607 - jepsen.util 607 :invoke :add 7073 INFO [2022-01-10 08:39:37,870] jepsen worker 221 - jepsen.util 221 :invoke :add 7074 INFO [2022-01-10 08:39:37,871] jepsen worker 857 - jepsen.util 857 :invoke :add 7075 INFO [2022-01-10 08:39:37,873] jepsen worker 56 - jepsen.util 1056 :invoke :add 7076 INFO [2022-01-10 08:39:37,875] jepsen worker 300 - jepsen.util 300 :invoke :add 7077 INFO [2022-01-10 08:39:37,877] jepsen worker 497 - jepsen.util 497 :invoke :add 7078 INFO [2022-01-10 08:39:37,879] jepsen worker 829 - jepsen.util 1829 :invoke :add 7079 INFO [2022-01-10 08:39:37,880] jepsen worker 280 - jepsen.util 280 :invoke :add 7080 INFO [2022-01-10 08:39:37,881] jepsen worker 112 - jepsen.util 112 :info :add 5361 :RequestTimeoutException INFO [2022-01-10 08:39:37,882] jepsen worker 108 - jepsen.util 1108 :invoke :add 7081 INFO [2022-01-10 08:39:37,885] jepsen worker 263 - jepsen.util 263 :invoke :add 7082 INFO [2022-01-10 08:39:37,887] jepsen worker 516 - jepsen.util 516 :invoke :add 7083 INFO [2022-01-10 08:39:37,889] jepsen worker 112 - jepsen.util 1112 :invoke :add 7084 INFO [2022-01-10 08:39:37,890] jepsen worker 744 - jepsen.util 744 :info :add 5367 :RequestTimeoutException INFO [2022-01-10 08:39:37,891] jepsen worker 290 - jepsen.util 1290 :invoke :add 7085 INFO [2022-01-10 08:39:37,894] jepsen worker 390 - jepsen.util 390 :invoke :add 7086 INFO [2022-01-10 08:39:37,896] jepsen worker 523 - jepsen.util 523 :invoke :add 7087 INFO [2022-01-10 08:39:37,897] jepsen worker 157 - jepsen.util 157 :invoke :add 7088 INFO [2022-01-10 08:39:37,899] jepsen worker 744 - jepsen.util 1744 :invoke :add 7089 INFO [2022-01-10 08:39:37,921] jepsen worker 895 - jepsen.util 895 :info :add 5378 :RequestTimeoutException INFO [2022-01-10 08:39:37,924] jepsen worker 895 - jepsen.util 1895 :invoke :add 7090 INFO [2022-01-10 08:39:37,937] jepsen worker 375 - jepsen.util 375 :info :add 5387 :RequestTimeoutException INFO [2022-01-10 08:39:37,939] jepsen worker 375 - jepsen.util 1375 :invoke :add 7091 INFO [2022-01-10 08:39:37,991] jepsen worker 627 - jepsen.util 627 :info :add 5409 :RequestTimeoutException INFO [2022-01-10 08:39:37,991] jepsen worker 152 - jepsen.util 152 :info :add 5411 :RequestTimeoutException INFO [2022-01-10 08:39:37,992] jepsen worker 554 - jepsen.util 554 :info :add 5412 :RequestTimeoutException INFO [2022-01-10 08:39:37,996] jepsen worker 554 - jepsen.util 1554 :invoke :add 7092 INFO [2022-01-10 08:39:37,997] jepsen worker 609 - jepsen.util 609 :info :add 5414 :RequestTimeoutException INFO [2022-01-10 08:39:37,998] jepsen worker 152 - jepsen.util 1152 :invoke :add 7093 INFO [2022-01-10 08:39:38,000] jepsen worker 609 - jepsen.util 1609 :invoke :add 7094 INFO [2022-01-10 08:39:38,002] jepsen worker 627 - jepsen.util 1627 :invoke :add 7095 INFO [2022-01-10 08:39:38,017] jepsen worker 244 - jepsen.util 244 :info :add 5421 :RequestTimeoutException INFO [2022-01-10 08:39:38,020] jepsen worker 244 - jepsen.util 1244 :invoke :add 7096 INFO [2022-01-10 08:39:38,022] jepsen worker 266 - jepsen.util 266 :info :add 5425 :RequestTimeoutException INFO [2022-01-10 08:39:38,023] jepsen worker 916 - jepsen.util 916 :info :add 5426 :RequestTimeoutException INFO [2022-01-10 08:39:38,023] jepsen worker 509 - jepsen.util 509 :info :add 5424 :RequestTimeoutException INFO [2022-01-10 08:39:38,027] jepsen worker 266 - jepsen.util 1266 :invoke :add 7097 INFO [2022-01-10 08:39:38,029] jepsen worker 916 - jepsen.util 1916 :invoke :add 7098 INFO [2022-01-10 08:39:38,031] jepsen worker 509 - jepsen.util 1509 :invoke :add 7099 INFO [2022-01-10 08:39:38,078] jepsen worker 994 - jepsen.util 994 :info :add 5448 :RequestTimeoutException INFO [2022-01-10 08:39:38,078] jepsen worker 359 - jepsen.util 359 :info :add 5449 :RequestTimeoutException INFO [2022-01-10 08:39:38,081] jepsen worker 663 - jepsen.util 663 :info :add 5451 :RequestTimeoutException INFO [2022-01-10 08:39:38,081] jepsen worker 359 - jepsen.util 1359 :invoke :add 7100 INFO [2022-01-10 08:39:38,084] jepsen worker 994 - jepsen.util 1994 :invoke :add 7101 INFO [2022-01-10 08:39:38,086] jepsen worker 663 - jepsen.util 1663 :invoke :add 7102 INFO [2022-01-10 08:39:38,091] jepsen worker 937 - jepsen.util 937 :info :add 5457 :RequestTimeoutException INFO [2022-01-10 08:39:38,094] jepsen worker 937 - jepsen.util 1937 :invoke :add 7103 INFO [2022-01-10 08:39:38,095] jepsen worker 222 - jepsen.util 222 :ok :add 6489 INFO [2022-01-10 08:39:38,096] jepsen worker 604 - jepsen.util 604 :ok :add 6092 INFO [2022-01-10 08:39:38,096] jepsen worker 639 - jepsen.util 639 :ok :add 6136 INFO [2022-01-10 08:39:38,096] jepsen worker 861 - jepsen.util 861 :ok :add 5714 INFO [2022-01-10 08:39:38,096] jepsen worker 43 - jepsen.util 43 :ok :add 6519 INFO [2022-01-10 08:39:38,096] jepsen worker 459 - jepsen.util 459 :ok :add 6491 INFO [2022-01-10 08:39:38,096] jepsen worker 120 - jepsen.util 120 :ok :add 6247 INFO [2022-01-10 08:39:38,097] jepsen worker 871 - jepsen.util 871 :ok :add 5829 INFO [2022-01-10 08:39:38,097] jepsen worker 451 - jepsen.util 451 :ok :add 5831 INFO [2022-01-10 08:39:38,098] jepsen worker 222 - jepsen.util 222 :invoke :add 7104 INFO [2022-01-10 08:39:38,102] jepsen worker 120 - jepsen.util 120 :invoke :add 7105 INFO [2022-01-10 08:39:38,104] jepsen worker 43 - jepsen.util 43 :invoke :add 7106 INFO [2022-01-10 08:39:38,106] jepsen worker 604 - jepsen.util 604 :invoke :add 7107 INFO [2022-01-10 08:39:38,108] jepsen worker 861 - jepsen.util 861 :invoke :add 7108 INFO [2022-01-10 08:39:38,110] jepsen worker 459 - jepsen.util 459 :invoke :add 7109 INFO [2022-01-10 08:39:38,112] jepsen worker 639 - jepsen.util 639 :invoke :add 7110 INFO [2022-01-10 08:39:38,114] jepsen worker 892 - jepsen.util 892 :info :add 5464 :RequestTimeoutException INFO [2022-01-10 08:39:38,115] jepsen worker 451 - jepsen.util 451 :invoke :add 7111 INFO [2022-01-10 08:39:38,117] jepsen worker 871 - jepsen.util 871 :invoke :add 7112 INFO [2022-01-10 08:39:38,119] jepsen worker 892 - jepsen.util 1892 :invoke :add 7113 INFO [2022-01-10 08:39:38,123] jepsen worker 104 - jepsen.util 104 :info :add 5467 :RequestTimeoutException INFO [2022-01-10 08:39:38,125] jepsen worker 104 - jepsen.util 1104 :invoke :add 7114 INFO [2022-01-10 08:39:38,151] jepsen worker 701 - jepsen.util 701 :info :add 5481 :RequestTimeoutException INFO [2022-01-10 08:39:38,154] jepsen worker 701 - jepsen.util 1701 :invoke :add 7115 INFO [2022-01-10 08:39:38,213] jepsen worker 134 - jepsen.util 134 :info :add 5509 :RequestTimeoutException INFO [2022-01-10 08:39:38,216] jepsen worker 134 - jepsen.util 1134 :invoke :add 7116 INFO [2022-01-10 08:39:38,220] jepsen worker 865 - jepsen.util 865 :info :add 5512 :RequestTimeoutException INFO [2022-01-10 08:39:38,222] jepsen worker 465 - jepsen.util 465 :info :add 5514 :RequestTimeoutException INFO [2022-01-10 08:39:38,223] jepsen worker 909 - jepsen.util 909 :info :add 5511 :RequestTimeoutException INFO [2022-01-10 08:39:38,224] jepsen worker 865 - jepsen.util 1865 :invoke :add 7117 INFO [2022-01-10 08:39:38,227] jepsen worker 909 - jepsen.util 1909 :invoke :add 7118 INFO [2022-01-10 08:39:38,229] jepsen worker 465 - jepsen.util 1465 :invoke :add 7119 INFO [2022-01-10 08:39:38,231] jepsen worker 178 - jepsen.util 178 :info :add 5516 :RequestTimeoutException INFO [2022-01-10 08:39:38,234] jepsen worker 178 - jepsen.util 1178 :invoke :add 7120 INFO [2022-01-10 08:39:38,242] jepsen worker 885 - jepsen.util 885 :info :add 5521 :RequestTimeoutException INFO [2022-01-10 08:39:38,243] jepsen worker 228 - jepsen.util 228 :info :add 5520 :RequestTimeoutException INFO [2022-01-10 08:39:38,244] jepsen worker 868 - jepsen.util 868 :info :add 5522 :RequestTimeoutException INFO [2022-01-10 08:39:38,248] jepsen worker 868 - jepsen.util 1868 :invoke :add 7121 INFO [2022-01-10 08:39:38,250] jepsen worker 228 - jepsen.util 1228 :invoke :add 7122 INFO [2022-01-10 08:39:38,251] jepsen worker 194 - jepsen.util 194 :info :add 5523 :RequestTimeoutException INFO [2022-01-10 08:39:38,255] jepsen worker 885 - jepsen.util 1885 :invoke :add 7123 INFO [2022-01-10 08:39:38,257] jepsen worker 194 - jepsen.util 1194 :invoke :add 7124 INFO [2022-01-10 08:39:38,261] jepsen worker 278 - jepsen.util 278 :info :add 5526 :RequestTimeoutException INFO [2022-01-10 08:39:38,262] jepsen worker 660 - jepsen.util 660 :info :add 5527 :RequestTimeoutException INFO [2022-01-10 08:39:38,265] jepsen worker 660 - jepsen.util 1660 :invoke :add 7125 INFO [2022-01-10 08:39:38,267] jepsen worker 278 - jepsen.util 1278 :invoke :add 7126 INFO [2022-01-10 08:39:38,282] jepsen worker 393 - jepsen.util 393 :info :add 5538 :RequestTimeoutException INFO [2022-01-10 08:39:38,286] jepsen worker 393 - jepsen.util 1393 :invoke :add 7127 INFO [2022-01-10 08:39:38,315] jepsen worker 123 - jepsen.util 123 :info :add 5551 :RequestTimeoutException INFO [2022-01-10 08:39:38,317] jepsen worker 571 - jepsen.util 571 :info :add 5553 :RequestTimeoutException INFO [2022-01-10 08:39:38,320] jepsen worker 123 - jepsen.util 1123 :invoke :add 7128 INFO [2022-01-10 08:39:38,323] jepsen worker 571 - jepsen.util 1571 :invoke :add 7129 INFO [2022-01-10 08:39:38,323] jepsen worker 322 - jepsen.util 322 :info :add 5554 :RequestTimeoutException INFO [2022-01-10 08:39:38,323] jepsen worker 81 - jepsen.util 81 :info :add 5556 :RequestTimeoutException INFO [2022-01-10 08:39:38,327] jepsen worker 81 - jepsen.util 1081 :invoke :add 7130 INFO [2022-01-10 08:39:38,329] jepsen worker 322 - jepsen.util 1322 :invoke :add 7131 INFO [2022-01-10 08:39:38,343] jepsen worker 235 - jepsen.util 235 :info :add 5562 :RequestTimeoutException INFO [2022-01-10 08:39:38,347] jepsen worker 235 - jepsen.util 1235 :invoke :add 7132 INFO [2022-01-10 08:39:38,366] jepsen worker 928 - jepsen.util 928 :info :add 5803 :SyncWriteAmbiguous INFO [2022-01-10 08:39:38,367] jepsen worker 105 - jepsen.util 105 :info :add 5656 :SyncWriteAmbiguous INFO [2022-01-10 08:39:38,371] jepsen worker 928 - jepsen.util 1928 :invoke :add 7133 INFO [2022-01-10 08:39:38,373] jepsen worker 276 - jepsen.util 276 :info :add 5578 :RequestTimeoutException INFO [2022-01-10 08:39:38,373] jepsen worker 105 - jepsen.util 1105 :invoke :add 7134 INFO [2022-01-10 08:39:38,377] jepsen worker 276 - jepsen.util 1276 :invoke :add 7135 INFO [2022-01-10 08:39:38,382] jepsen worker 17 - jepsen.util 17 :info :add 5581 :RequestTimeoutException INFO [2022-01-10 08:39:38,383] jepsen worker 295 - jepsen.util 295 :info :add 5579 :RequestTimeoutException INFO [2022-01-10 08:39:38,385] jepsen worker 17 - jepsen.util 1017 :invoke :add 7136 INFO [2022-01-10 08:39:38,388] jepsen worker 295 - jepsen.util 1295 :invoke :add 7137 INFO [2022-01-10 08:39:38,390] jepsen worker 238 - jepsen.util 238 :info :add 5583 :RequestTimeoutException INFO [2022-01-10 08:39:38,393] jepsen worker 238 - jepsen.util 1238 :invoke :add 7138 INFO [2022-01-10 08:39:38,400] jepsen worker 150 - jepsen.util 150 :info :add 5868 :SyncWriteAmbiguous INFO [2022-01-10 08:39:38,401] jepsen worker 272 - jepsen.util 272 :info :add 5587 :RequestTimeoutException INFO [2022-01-10 08:39:38,403] jepsen worker 473 - jepsen.util 473 :ok :add 5895 INFO [2022-01-10 08:39:38,403] jepsen worker 415 - jepsen.util 415 :ok :add 6192 INFO [2022-01-10 08:39:38,404] jepsen worker 815 - jepsen.util 815 :ok :add 5620 INFO [2022-01-10 08:39:38,404] jepsen worker 374 - jepsen.util 374 :ok :add 6347 INFO [2022-01-10 08:39:38,404] jepsen worker 73 - jepsen.util 73 :ok :add 6591 INFO [2022-01-10 08:39:38,405] jepsen worker 272 - jepsen.util 1272 :invoke :add 7139 INFO [2022-01-10 08:39:38,405] jepsen worker 938 - jepsen.util 938 :ok :add 6036 INFO [2022-01-10 08:39:38,412] jepsen worker 424 - jepsen.util 424 :ok :add 6002 INFO [2022-01-10 08:39:38,412] jepsen worker 328 - jepsen.util 328 :ok :add 6373 INFO [2022-01-10 08:39:38,412] jepsen worker 620 - jepsen.util 620 :info :add 5846 :SyncWriteAmbiguous INFO [2022-01-10 08:39:38,412] jepsen worker 173 - jepsen.util 173 :ok :add 6435 INFO [2022-01-10 08:39:38,412] jepsen worker 670 - jepsen.util 670 :ok :add 5784 INFO [2022-01-10 08:39:38,412] jepsen worker 855 - jepsen.util 855 :ok :add 6744 INFO [2022-01-10 08:39:38,412] jepsen worker 852 - jepsen.util 852 :ok :add 6401 INFO [2022-01-10 08:39:38,414] jepsen worker 73 - jepsen.util 73 :invoke :add 7140 INFO [2022-01-10 08:39:38,418] jepsen worker 852 - jepsen.util 852 :invoke :add 7141 INFO [2022-01-10 08:39:38,419] jepsen worker 415 - jepsen.util 415 :invoke :add 7142 INFO [2022-01-10 08:39:38,421] jepsen worker 424 - jepsen.util 424 :invoke :add 7143 INFO [2022-01-10 08:39:38,423] jepsen worker 855 - jepsen.util 855 :invoke :add 7144 INFO [2022-01-10 08:39:38,425] jepsen worker 815 - jepsen.util 815 :invoke :add 7145 INFO [2022-01-10 08:39:38,427] jepsen worker 670 - jepsen.util 670 :invoke :add 7146 INFO [2022-01-10 08:39:38,429] jepsen worker 328 - jepsen.util 328 :invoke :add 7147 INFO [2022-01-10 08:39:38,430] jepsen worker 620 - jepsen.util 1620 :invoke :add 7148 INFO [2022-01-10 08:39:38,432] jepsen worker 938 - jepsen.util 938 :invoke :add 7149 INFO [2022-01-10 08:39:38,434] jepsen worker 173 - jepsen.util 173 :invoke :add 7150 INFO [2022-01-10 08:39:38,435] jepsen worker 473 - jepsen.util 473 :invoke :add 7151 INFO [2022-01-10 08:39:38,437] jepsen worker 374 - jepsen.util 374 :invoke :add 7152 INFO [2022-01-10 08:39:38,439] jepsen worker 150 - jepsen.util 1150 :invoke :add 7153 INFO [2022-01-10 08:39:38,443] jepsen worker 458 - jepsen.util 458 :info :add 5599 :RequestTimeoutException INFO [2022-01-10 08:39:38,445] jepsen worker 458 - jepsen.util 1458 :invoke :add 7154 INFO [2022-01-10 08:39:38,481] jepsen worker 577 - jepsen.util 577 :info :add 5611 :RequestTimeoutException INFO [2022-01-10 08:39:38,484] jepsen worker 577 - jepsen.util 1577 :invoke :add 7155 INFO [2022-01-10 08:39:38,490] jepsen worker 19 - jepsen.util 19 :info :add 5612 :RequestTimeoutException INFO [2022-01-10 08:39:38,493] jepsen worker 965 - jepsen.util 965 :ok :add 6381 INFO [2022-01-10 08:39:38,493] jepsen worker 690 - jepsen.util 690 :ok :add 6178 INFO [2022-01-10 08:39:38,494] jepsen worker 129 - jepsen.util 129 :ok :add 6225 INFO [2022-01-10 08:39:38,494] jepsen worker 318 - jepsen.util 318 :ok :add 5867 INFO [2022-01-10 08:39:38,494] jepsen worker 19 - jepsen.util 1019 :invoke :add 7156 INFO [2022-01-10 08:39:38,494] jepsen worker 655 - jepsen.util 655 :ok :add 6211 INFO [2022-01-10 08:39:38,494] jepsen worker 434 - jepsen.util 434 :ok :add 6209 INFO [2022-01-10 08:39:38,494] jepsen worker 576 - jepsen.util 576 :ok :add 6160 INFO [2022-01-10 08:39:38,497] jepsen worker 869 - jepsen.util 869 :info :add 5616 :RequestTimeoutException INFO [2022-01-10 08:39:38,498] jepsen worker 318 - jepsen.util 318 :invoke :add 7157 INFO [2022-01-10 08:39:38,500] jepsen worker 383 - jepsen.util 383 :ok :add 6155 INFO [2022-01-10 08:39:38,500] jepsen worker 182 - jepsen.util 182 :ok :add 6161 INFO [2022-01-10 08:39:38,501] jepsen worker 933 - jepsen.util 933 :ok :add 6210 INFO [2022-01-10 08:39:38,501] jepsen worker 434 - jepsen.util 434 :invoke :add 7158 INFO [2022-01-10 08:39:38,501] jepsen worker 948 - jepsen.util 948 :ok :add 5866 INFO [2022-01-10 08:39:38,501] jepsen worker 96 - jepsen.util 96 :ok :add 6208 INFO [2022-01-10 08:39:38,501] jepsen worker 950 - jepsen.util 950 :ok :add 6179 INFO [2022-01-10 08:39:38,501] jepsen worker 987 - jepsen.util 987 :ok :add 6398 INFO [2022-01-10 08:39:38,502] jepsen worker 24 - jepsen.util 24 :ok :add 6224 INFO [2022-01-10 08:39:38,505] jepsen worker 96 - jepsen.util 96 :invoke :add 7159 INFO [2022-01-10 08:39:38,507] jepsen worker 129 - jepsen.util 129 :invoke :add 7160 INFO [2022-01-10 08:39:38,509] jepsen worker 655 - jepsen.util 655 :invoke :add 7161 INFO [2022-01-10 08:39:38,511] jepsen worker 869 - jepsen.util 1869 :invoke :add 7162 INFO [2022-01-10 08:39:38,512] jepsen worker 950 - jepsen.util 950 :invoke :add 7163 INFO [2022-01-10 08:39:38,514] jepsen worker 690 - jepsen.util 690 :invoke :add 7164 INFO [2022-01-10 08:39:38,516] jepsen worker 933 - jepsen.util 933 :invoke :add 7165 INFO [2022-01-10 08:39:38,517] jepsen worker 708 - jepsen.util 708 :info :add 5622 :RequestTimeoutException INFO [2022-01-10 08:39:38,518] jepsen worker 948 - jepsen.util 948 :invoke :add 7166 INFO [2022-01-10 08:39:38,521] jepsen worker 24 - jepsen.util 24 :invoke :add 7167 INFO [2022-01-10 08:39:38,523] jepsen worker 383 - jepsen.util 383 :invoke :add 7168 INFO [2022-01-10 08:39:38,525] jepsen worker 987 - jepsen.util 987 :invoke :add 7169 INFO [2022-01-10 08:39:38,527] jepsen worker 576 - jepsen.util 576 :invoke :add 7170 INFO [2022-01-10 08:39:38,529] jepsen worker 965 - jepsen.util 965 :invoke :add 7171 INFO [2022-01-10 08:39:38,531] jepsen worker 182 - jepsen.util 182 :invoke :add 7172 INFO [2022-01-10 08:39:38,533] jepsen worker 310 - jepsen.util 310 :info :add 5627 :RequestTimeoutException INFO [2022-01-10 08:39:38,533] jepsen worker 708 - jepsen.util 1708 :invoke :add 7173 INFO [2022-01-10 08:39:38,536] jepsen worker 310 - jepsen.util 1310 :invoke :add 7174 INFO [2022-01-10 08:39:38,572] jepsen worker 89 - jepsen.util 89 :info :add 5638 :RequestTimeoutException INFO [2022-01-10 08:39:38,575] jepsen worker 89 - jepsen.util 1089 :invoke :add 7175 INFO [2022-01-10 08:39:38,602] jepsen worker 508 - jepsen.util 508 :info :add 5648 :RequestTimeoutException INFO [2022-01-10 08:39:38,604] jepsen worker 106 - jepsen.util 106 :info :add 5649 :RequestTimeoutException INFO [2022-01-10 08:39:38,605] jepsen worker 508 - jepsen.util 1508 :invoke :add 7176 INFO [2022-01-10 08:39:38,607] jepsen worker 106 - jepsen.util 1106 :invoke :add 7177 INFO [2022-01-10 08:39:38,614] jepsen worker 551 - jepsen.util 551 :info :add 5652 :RequestTimeoutException INFO [2022-01-10 08:39:38,617] jepsen worker 551 - jepsen.util 1551 :invoke :add 7178 INFO [2022-01-10 08:39:38,622] jepsen worker 998 - jepsen.util 998 :info :add 5654 :RequestTimeoutException INFO [2022-01-10 08:39:38,626] jepsen worker 998 - jepsen.util 1998 :invoke :add 7179 INFO [2022-01-10 08:39:38,641] jepsen worker 608 - jepsen.util 608 :info :add 5661 :RequestTimeoutException INFO [2022-01-10 08:39:38,642] jepsen worker 645 - jepsen.util 645 :info :add 5662 :RequestTimeoutException INFO [2022-01-10 08:39:38,643] jepsen worker 249 - jepsen.util 249 :info :add 5660 :RequestTimeoutException INFO [2022-01-10 08:39:38,645] jepsen worker 608 - jepsen.util 1608 :invoke :add 7180 INFO [2022-01-10 08:39:38,647] jepsen worker 645 - jepsen.util 1645 :invoke :add 7181 INFO [2022-01-10 08:39:38,648] jepsen worker 249 - jepsen.util 1249 :invoke :add 7182 INFO [2022-01-10 08:39:38,654] jepsen worker 800 - jepsen.util 800 :info :add 5665 :RequestTimeoutException INFO [2022-01-10 08:39:38,656] jepsen worker 800 - jepsen.util 1800 :invoke :add 7183 INFO [2022-01-10 08:39:38,668] jepsen worker 776 - jepsen.util 776 :ok :add 6488 INFO [2022-01-10 08:39:38,668] jepsen worker 362 - jepsen.util 362 :ok :add 6534 INFO [2022-01-10 08:39:38,668] jepsen worker 633 - jepsen.util 633 :ok :add 5721 INFO [2022-01-10 08:39:38,668] jepsen worker 668 - jepsen.util 668 :ok :add 6518 INFO [2022-01-10 08:39:38,669] jepsen worker 274 - jepsen.util 274 :ok :add 6246 INFO [2022-01-10 08:39:38,669] jepsen worker 981 - jepsen.util 981 :ok :add 5994 INFO [2022-01-10 08:39:38,672] jepsen worker 633 - jepsen.util 633 :invoke :add 7184 INFO [2022-01-10 08:39:38,674] jepsen worker 362 - jepsen.util 362 :invoke :add 7185 INFO [2022-01-10 08:39:38,675] jepsen worker 340 - jepsen.util 340 :ok :add 6500 INFO [2022-01-10 08:39:38,675] jepsen worker 357 - jepsen.util 357 :ok :add 6137 INFO [2022-01-10 08:39:38,676] jepsen worker 809 - jepsen.util 809 :ok :add 5739 INFO [2022-01-10 08:39:38,676] jepsen worker 668 - jepsen.util 668 :invoke :add 7186 INFO [2022-01-10 08:39:38,678] jepsen worker 274 - jepsen.util 274 :invoke :add 7187 INFO [2022-01-10 08:39:38,679] jepsen worker 776 - jepsen.util 776 :invoke :add 7188 INFO [2022-01-10 08:39:38,681] jepsen worker 357 - jepsen.util 357 :invoke :add 7189 INFO [2022-01-10 08:39:38,683] jepsen worker 981 - jepsen.util 981 :invoke :add 7190 INFO [2022-01-10 08:39:38,685] jepsen worker 809 - jepsen.util 809 :invoke :add 7191 INFO [2022-01-10 08:39:38,687] jepsen worker 340 - jepsen.util 340 :invoke :add 7192 INFO [2022-01-10 08:39:38,721] jepsen worker 769 - jepsen.util 769 :info :add 5680 :RequestTimeoutException INFO [2022-01-10 08:39:38,723] jepsen worker 595 - jepsen.util 595 :info :add 5682 :RequestTimeoutException INFO [2022-01-10 08:39:38,723] jepsen worker 991 - jepsen.util 991 :info :add 5684 :RequestTimeoutException INFO [2022-01-10 08:39:38,723] jepsen worker 550 - jepsen.util 550 :info :add 5681 :RequestTimeoutException INFO [2022-01-10 08:39:38,724] jepsen worker 769 - jepsen.util 1769 :invoke :add 7193 INFO [2022-01-10 08:39:38,728] jepsen worker 550 - jepsen.util 1550 :invoke :add 7194 INFO [2022-01-10 08:39:38,730] jepsen worker 595 - jepsen.util 1595 :invoke :add 7195 INFO [2022-01-10 08:39:38,730] jepsen worker 234 - jepsen.util 234 :info :add 5685 :RequestTimeoutException INFO [2022-01-10 08:39:38,732] jepsen worker 991 - jepsen.util 1991 :invoke :add 7196 INFO [2022-01-10 08:39:38,733] jepsen worker 692 - jepsen.util 692 :info :add 5687 :RequestTimeoutException INFO [2022-01-10 08:39:38,734] jepsen worker 234 - jepsen.util 1234 :invoke :add 7197 INFO [2022-01-10 08:39:38,737] jepsen worker 692 - jepsen.util 1692 :invoke :add 7198 INFO [2022-01-10 08:39:38,813] jepsen worker 427 - jepsen.util 427 :info :add 5712 :RequestTimeoutException INFO [2022-01-10 08:39:38,817] jepsen worker 427 - jepsen.util 1427 :invoke :add 7199 INFO [2022-01-10 08:39:38,822] jepsen worker 395 - jepsen.util 395 :info :add 5716 :RequestTimeoutException INFO [2022-01-10 08:39:38,825] jepsen worker 395 - jepsen.util 1395 :invoke :add 7200 INFO [2022-01-10 08:39:38,840] jepsen worker 661 - jepsen.util 661 :info :add 5720 :RequestTimeoutException INFO [2022-01-10 08:39:38,841] jepsen worker 400 - jepsen.util 400 :info :add 5722 :RequestTimeoutException INFO [2022-01-10 08:39:38,843] jepsen worker 661 - jepsen.util 1661 :invoke :add 7201 INFO [2022-01-10 08:39:38,846] jepsen worker 400 - jepsen.util 1400 :invoke :add 7202 INFO [2022-01-10 08:39:38,847] jepsen worker 289 - jepsen.util 289 :info :add 5723 :RequestTimeoutException INFO [2022-01-10 08:39:38,850] jepsen worker 289 - jepsen.util 1289 :invoke :add 7203 INFO [2022-01-10 08:39:38,853] jepsen worker 953 - jepsen.util 953 :info :add 5727 :RequestTimeoutException INFO [2022-01-10 08:39:38,853] jepsen worker 201 - jepsen.util 201 :info :add 5726 :RequestTimeoutException INFO [2022-01-10 08:39:38,856] jepsen worker 953 - jepsen.util 1953 :invoke :add 7204 INFO [2022-01-10 08:39:38,858] jepsen worker 201 - jepsen.util 1201 :invoke :add 7205 INFO [2022-01-10 08:39:38,872] jepsen worker 493 - jepsen.util 493 :ok :add 5953 INFO [2022-01-10 08:39:38,872] jepsen worker 767 - jepsen.util 767 :ok :add 6054 INFO [2022-01-10 08:39:38,872] jepsen worker 219 - jepsen.util 219 :ok :add 6299 INFO [2022-01-10 08:39:38,872] jepsen worker 325 - jepsen.util 325 :ok :add 6457 INFO [2022-01-10 08:39:38,872] jepsen worker 455 - jepsen.util 455 :ok :add 6078 INFO [2022-01-10 08:39:38,872] jepsen worker 193 - jepsen.util 193 :ok :add 6311 INFO [2022-01-10 08:39:38,876] jepsen worker 767 - jepsen.util 767 :invoke :add 7206 INFO [2022-01-10 08:39:38,878] jepsen worker 325 - jepsen.util 325 :invoke :add 7207 INFO [2022-01-10 08:39:38,879] jepsen worker 193 - jepsen.util 193 :invoke :add 7208 INFO [2022-01-10 08:39:38,881] jepsen worker 455 - jepsen.util 455 :invoke :add 7209 INFO [2022-01-10 08:39:38,883] jepsen worker 219 - jepsen.util 219 :invoke :add 7210 INFO [2022-01-10 08:39:38,885] jepsen worker 493 - jepsen.util 493 :invoke :add 7211 INFO [2022-01-10 08:39:38,932] jepsen worker 91 - jepsen.util 91 :info :add 5748 :RequestTimeoutException INFO [2022-01-10 08:39:38,935] jepsen worker 91 - jepsen.util 1091 :invoke :add 7212 INFO [2022-01-10 08:39:38,937] jepsen worker 756 - jepsen.util 756 :info :add 5751 :RequestTimeoutException INFO [2022-01-10 08:39:38,939] jepsen worker 756 - jepsen.util 1756 :invoke :add 7213 INFO [2022-01-10 08:39:38,941] jepsen worker 448 - jepsen.util 448 :info :add 5753 :RequestTimeoutException INFO [2022-01-10 08:39:38,943] jepsen worker 448 - jepsen.util 1448 :invoke :add 7214 INFO [2022-01-10 08:39:38,944] jepsen worker 30 - jepsen.util 30 :info :add 5754 :RequestTimeoutException INFO [2022-01-10 08:39:38,947] jepsen worker 30 - jepsen.util 1030 :invoke :add 7215 INFO [2022-01-10 08:39:38,953] jepsen worker 44 - jepsen.util 44 :info :add 5756 :RequestTimeoutException INFO [2022-01-10 08:39:38,953] jepsen worker 686 - jepsen.util 686 :info :add 5757 :RequestTimeoutException INFO [2022-01-10 08:39:38,958] jepsen worker 686 - jepsen.util 1686 :invoke :add 7216 INFO [2022-01-10 08:39:38,960] jepsen worker 44 - jepsen.util 1044 :invoke :add 7217 INFO [2022-01-10 08:39:38,962] jepsen worker 866 - jepsen.util 866 :info :add 5760 :RequestTimeoutException INFO [2022-01-10 08:39:38,965] jepsen worker 866 - jepsen.util 1866 :invoke :add 7218 INFO [2022-01-10 08:39:38,973] jepsen worker 560 - jepsen.util 560 :info :add 5762 :RequestTimeoutException INFO [2022-01-10 08:39:38,976] jepsen worker 560 - jepsen.util 1560 :invoke :add 7219 INFO [2022-01-10 08:39:38,981] jepsen worker 258 - jepsen.util 258 :info :add 5763 :RequestTimeoutException INFO [2022-01-10 08:39:38,984] jepsen worker 258 - jepsen.util 1258 :invoke :add 7220 INFO [2022-01-10 08:39:38,987] jepsen worker 779 - jepsen.util 779 :info :add 5764 :RequestTimeoutException INFO [2022-01-10 08:39:38,990] jepsen worker 779 - jepsen.util 1779 :invoke :add 7221 INFO [2022-01-10 08:39:39,063] jepsen worker 270 - jepsen.util 270 :info :add 5778 :RequestTimeoutException INFO [2022-01-10 08:39:39,063] jepsen worker 899 - jepsen.util 899 :info :add 5780 :RequestTimeoutException INFO [2022-01-10 08:39:39,064] jepsen worker 522 - jepsen.util 522 :info :add 5779 :RequestTimeoutException INFO [2022-01-10 08:39:39,064] jepsen worker 714 - jepsen.util 714 :info :add 5781 :RequestTimeoutException INFO [2022-01-10 08:39:39,068] jepsen worker 270 - jepsen.util 1270 :invoke :add 7222 INFO [2022-01-10 08:39:39,070] jepsen worker 714 - jepsen.util 1714 :invoke :add 7223 INFO [2022-01-10 08:39:39,071] jepsen worker 899 - jepsen.util 1899 :invoke :add 7224 INFO [2022-01-10 08:39:39,072] jepsen worker 432 - jepsen.util 432 :info :add 5782 :RequestTimeoutException INFO [2022-01-10 08:39:39,073] jepsen worker 381 - jepsen.util 381 :info :add 5783 :RequestTimeoutException INFO [2022-01-10 08:39:39,073] jepsen worker 522 - jepsen.util 1522 :invoke :add 7225 INFO [2022-01-10 08:39:39,077] jepsen worker 381 - jepsen.util 1381 :invoke :add 7226 INFO [2022-01-10 08:39:39,079] jepsen worker 432 - jepsen.util 1432 :invoke :add 7227 INFO [2022-01-10 08:39:39,102] jepsen worker 786 - jepsen.util 786 :info :add 5847 :SyncWriteAmbiguous INFO [2022-01-10 08:39:39,107] jepsen worker 128 - jepsen.util 128 :info :add 5844 :SyncWriteAmbiguous INFO [2022-01-10 08:39:39,115] jepsen worker 128 - jepsen.util 1128 :invoke :add 7228 INFO [2022-01-10 08:39:39,117] jepsen worker 786 - jepsen.util 1786 :invoke :add 7229 INFO [2022-01-10 08:39:39,204] jepsen worker 139 - jepsen.util 139 :info :add 5799 :RequestTimeoutException INFO [2022-01-10 08:39:39,204] jepsen worker 536 - jepsen.util 536 :info :add 5801 :RequestTimeoutException INFO [2022-01-10 08:39:39,208] jepsen worker 139 - jepsen.util 1139 :invoke :add 7230 INFO [2022-01-10 08:39:39,210] jepsen worker 536 - jepsen.util 1536 :invoke :add 7231 INFO [2022-01-10 08:39:39,210] jepsen worker 10 - jepsen.util 10 :info :add 5805 :RequestTimeoutException INFO [2022-01-10 08:39:39,215] jepsen worker 10 - jepsen.util 1010 :invoke :add 7232 INFO [2022-01-10 08:39:39,222] jepsen worker 297 - jepsen.util 297 :info :add 5807 :RequestTimeoutException INFO [2022-01-10 08:39:39,225] jepsen worker 297 - jepsen.util 1297 :invoke :add 7233 INFO [2022-01-10 08:39:39,241] jepsen worker 957 - jepsen.util 957 :info :add 5818 :RequestTimeoutException INFO [2022-01-10 08:39:39,242] jepsen worker 348 - jepsen.util 348 :info :add 5819 :RequestTimeoutException INFO [2022-01-10 08:39:39,244] jepsen worker 929 - jepsen.util 929 :ok :add 6040 INFO [2022-01-10 08:39:39,246] jepsen worker 348 - jepsen.util 1348 :invoke :add 7234 INFO [2022-01-10 08:39:39,249] jepsen worker 957 - jepsen.util 1957 :invoke :add 7235 INFO [2022-01-10 08:39:39,251] jepsen worker 929 - jepsen.util 929 :invoke :add 7236 INFO [2022-01-10 08:39:39,261] jepsen worker 80 - jepsen.util 80 :info :add 5828 :RequestTimeoutException INFO [2022-01-10 08:39:39,262] jepsen worker 843 - jepsen.util 843 :info :add 5830 :RequestTimeoutException INFO [2022-01-10 08:39:39,264] jepsen worker 80 - jepsen.util 1080 :invoke :add 7237 INFO [2022-01-10 08:39:39,266] jepsen worker 843 - jepsen.util 1843 :invoke :add 7238 INFO [2022-01-10 08:39:39,273] jepsen worker 305 - jepsen.util 305 :info :add 5836 :RequestTimeoutException INFO [2022-01-10 08:39:39,273] jepsen worker 984 - jepsen.util 984 :info :add 5832 :RequestTimeoutException INFO [2022-01-10 08:39:39,273] jepsen worker 648 - jepsen.util 648 :info :add 5833 :RequestTimeoutException INFO [2022-01-10 08:39:39,274] jepsen worker 922 - jepsen.util 922 :info :add 5834 :RequestTimeoutException INFO [2022-01-10 08:39:39,274] jepsen worker 312 - jepsen.util 312 :info :add 5837 :RequestTimeoutException INFO [2022-01-10 08:39:39,280] jepsen worker 984 - jepsen.util 1984 :invoke :add 7239 INFO [2022-01-10 08:39:39,280] jepsen worker 341 - jepsen.util 341 :info :add 5840 :RequestTimeoutException INFO [2022-01-10 08:39:39,282] jepsen worker 63 - jepsen.util 63 :info :add 5839 :RequestTimeoutException INFO [2022-01-10 08:39:39,282] jepsen worker 305 - jepsen.util 1305 :invoke :add 7240 INFO [2022-01-10 08:39:39,283] jepsen worker 97 - jepsen.util 97 :info :add 5841 :RequestTimeoutException INFO [2022-01-10 08:39:39,287] jepsen worker 312 - jepsen.util 1312 :invoke :add 7241 INFO [2022-01-10 08:39:39,289] jepsen worker 63 - jepsen.util 1063 :invoke :add 7242 INFO [2022-01-10 08:39:39,290] jepsen worker 341 - jepsen.util 1341 :invoke :add 7243 INFO [2022-01-10 08:39:39,292] jepsen worker 648 - jepsen.util 1648 :invoke :add 7244 INFO [2022-01-10 08:39:39,294] jepsen worker 287 - jepsen.util 287 :info :add 5845 :RequestTimeoutException INFO [2022-01-10 08:39:39,294] jepsen worker 292 - jepsen.util 292 :info :add 5843 :RequestTimeoutException INFO [2022-01-10 08:39:39,295] jepsen worker 97 - jepsen.util 1097 :invoke :add 7245 INFO [2022-01-10 08:39:39,298] jepsen worker 287 - jepsen.util 1287 :invoke :add 7246 INFO [2022-01-10 08:39:39,300] jepsen worker 292 - jepsen.util 1292 :invoke :add 7247 INFO [2022-01-10 08:39:39,302] jepsen worker 922 - jepsen.util 1922 :invoke :add 7248 INFO [2022-01-10 08:39:39,303] jepsen worker 828 - jepsen.util 828 :info :add 5850 :RequestTimeoutException INFO [2022-01-10 08:39:39,306] jepsen worker 828 - jepsen.util 1828 :invoke :add 7249 INFO [2022-01-10 08:39:39,313] jepsen worker 911 - jepsen.util 911 :info :add 5859 :RequestTimeoutException INFO [2022-01-10 08:39:39,315] jepsen worker 911 - jepsen.util 1911 :invoke :add 7250 INFO [2022-01-10 08:39:39,331] jepsen worker 117 - jepsen.util 117 :info :add 5864 :RequestTimeoutException INFO [2022-01-10 08:39:39,335] jepsen worker 117 - jepsen.util 1117 :invoke :add 7251 INFO [2022-01-10 08:39:39,339] jepsen worker 612 - jepsen.util 612 :ok :add 5897 INFO [2022-01-10 08:39:39,339] jepsen worker 952 - jepsen.util 952 :ok :add 6369 INFO [2022-01-10 08:39:39,339] jepsen worker 754 - jepsen.util 754 :ok :add 6018 INFO [2022-01-10 08:39:39,339] jepsen worker 721 - jepsen.util 721 :ok :add 6772 INFO [2022-01-10 08:39:39,339] jepsen worker 90 - jepsen.util 90 :ok :add 5907 INFO [2022-01-10 08:39:39,339] jepsen worker 680 - jepsen.util 680 :ok :add 6345 INFO [2022-01-10 08:39:39,342] jepsen worker 425 - jepsen.util 425 :info :add 5869 :RequestTimeoutException INFO [2022-01-10 08:39:39,342] jepsen worker 678 - jepsen.util 678 :ok :add 6600 INFO [2022-01-10 08:39:39,342] jepsen worker 175 - jepsen.util 175 :ok :add 5875 INFO [2022-01-10 08:39:39,343] jepsen worker 952 - jepsen.util 952 :invoke :add 7252 INFO [2022-01-10 08:39:39,344] jepsen worker 155 - jepsen.util 155 :ok :add 6545 INFO [2022-01-10 08:39:39,344] jepsen worker 200 - jepsen.util 200 :info :add 5870 :RequestTimeoutException INFO [2022-01-10 08:39:39,344] jepsen worker 717 - jepsen.util 717 :ok :add 6569 INFO [2022-01-10 08:39:39,344] jepsen worker 12 - jepsen.util 12 :ok :add 6571 INFO [2022-01-10 08:39:39,344] jepsen worker 482 - jepsen.util 482 :ok :add 6618 INFO [2022-01-10 08:39:39,344] jepsen worker 831 - jepsen.util 831 :ok :add 6237 INFO [2022-01-10 08:39:39,347] jepsen worker 179 - jepsen.util 179 :info :add 5871 :RequestTimeoutException INFO [2022-01-10 08:39:39,351] jepsen worker 754 - jepsen.util 754 :invoke :add 7253 INFO [2022-01-10 08:39:39,352] jepsen worker 253 - jepsen.util 253 :info :add 5872 :RequestTimeoutException INFO [2022-01-10 08:39:39,352] jepsen worker 412 - jepsen.util 412 :info :add 5873 :RequestTimeoutException INFO [2022-01-10 08:39:39,353] jepsen worker 883 - jepsen.util 883 :info :add 5874 :RequestTimeoutException INFO [2022-01-10 08:39:39,354] jepsen worker 179 - jepsen.util 1179 :invoke :add 7254 INFO [2022-01-10 08:39:39,360] jepsen worker 412 - jepsen.util 1412 :invoke :add 7255 INFO [2022-01-10 08:39:39,363] jepsen worker 431 - jepsen.util 431 :info :add 5876 :RequestTimeoutException INFO [2022-01-10 08:39:39,363] jepsen worker 680 - jepsen.util 680 :invoke :add 7256 INFO [2022-01-10 08:39:39,366] jepsen worker 678 - jepsen.util 678 :invoke :add 7257 INFO [2022-01-10 08:39:39,368] jepsen worker 90 - jepsen.util 90 :invoke :add 7258 INFO [2022-01-10 08:39:39,371] jepsen worker 583 - jepsen.util 583 :info :add 5877 :RequestTimeoutException INFO [2022-01-10 08:39:39,371] jepsen worker 612 - jepsen.util 612 :invoke :add 7259 INFO [2022-01-10 08:39:39,372] jepsen worker 831 - jepsen.util 831 :invoke :add 7260 INFO [2022-01-10 08:39:39,375] jepsen worker 717 - jepsen.util 717 :invoke :add 7261 INFO [2022-01-10 08:39:39,377] jepsen worker 12 - jepsen.util 12 :invoke :add 7262 INFO [2022-01-10 08:39:39,380] jepsen worker 155 - jepsen.util 155 :invoke :add 7263 INFO [2022-01-10 08:39:39,381] jepsen worker 482 - jepsen.util 482 :invoke :add 7264 INFO [2022-01-10 08:39:39,384] jepsen worker 253 - jepsen.util 1253 :invoke :add 7265 INFO [2022-01-10 08:39:39,385] jepsen worker 721 - jepsen.util 721 :invoke :add 7266 INFO [2022-01-10 08:39:39,387] jepsen worker 175 - jepsen.util 175 :invoke :add 7267 INFO [2022-01-10 08:39:39,389] jepsen worker 583 - jepsen.util 1583 :invoke :add 7268 INFO [2022-01-10 08:39:39,391] jepsen worker 431 - jepsen.util 1431 :invoke :add 7269 INFO [2022-01-10 08:39:39,392] jepsen worker 875 - jepsen.util 875 :info :add 5888 :RequestTimeoutException INFO [2022-01-10 08:39:39,392] jepsen worker 874 - jepsen.util 874 :info :add 5889 :RequestTimeoutException INFO [2022-01-10 08:39:39,394] jepsen worker 425 - jepsen.util 1425 :invoke :add 7270 INFO [2022-01-10 08:39:39,397] jepsen worker 874 - jepsen.util 1874 :invoke :add 7271 INFO [2022-01-10 08:39:39,399] jepsen worker 200 - jepsen.util 1200 :invoke :add 7272 INFO [2022-01-10 08:39:39,401] jepsen worker 720 - jepsen.util 720 :info :add 5890 :RequestTimeoutException INFO [2022-01-10 08:39:39,401] jepsen worker 875 - jepsen.util 1875 :invoke :add 7273 INFO [2022-01-10 08:39:39,403] jepsen worker 837 - jepsen.util 837 :info :add 5892 :RequestTimeoutException INFO [2022-01-10 08:39:39,408] jepsen worker 837 - jepsen.util 1837 :invoke :add 7274 INFO [2022-01-10 08:39:39,410] jepsen worker 720 - jepsen.util 1720 :invoke :add 7275 INFO [2022-01-10 08:39:39,411] jepsen worker 962 - jepsen.util 962 :info :add 5893 :RequestTimeoutException INFO [2022-01-10 08:39:39,412] jepsen worker 883 - jepsen.util 1883 :invoke :add 7276 INFO [2022-01-10 08:39:39,414] jepsen worker 962 - jepsen.util 1962 :invoke :add 7277 INFO [2022-01-10 08:39:39,418] jepsen worker 677 - jepsen.util 677 :info :add 5900 :RequestTimeoutException INFO [2022-01-10 08:39:39,421] jepsen worker 677 - jepsen.util 1677 :invoke :add 7278 INFO [2022-01-10 08:39:39,423] jepsen worker 480 - jepsen.util 480 :info :add 5903 :RequestTimeoutException INFO [2022-01-10 08:39:39,424] jepsen worker 65 - jepsen.util 65 :info :add 5901 :RequestTimeoutException INFO [2022-01-10 08:39:39,424] jepsen worker 252 - jepsen.util 252 :info :add 5902 :RequestTimeoutException INFO [2022-01-10 08:39:39,428] jepsen worker 480 - jepsen.util 1480 :invoke :add 7279 INFO [2022-01-10 08:39:39,430] jepsen worker 65 - jepsen.util 1065 :invoke :add 7280 INFO [2022-01-10 08:39:39,431] jepsen worker 784 - jepsen.util 784 :info :add 5904 :RequestTimeoutException INFO [2022-01-10 08:39:39,432] jepsen worker 252 - jepsen.util 1252 :invoke :add 7281 INFO [2022-01-10 08:39:39,433] jepsen worker 773 - jepsen.util 773 :info :add 5906 :RequestTimeoutException INFO [2022-01-10 08:39:39,434] jepsen worker 784 - jepsen.util 1784 :invoke :add 7282 INFO [2022-01-10 08:39:39,437] jepsen worker 773 - jepsen.util 1773 :invoke :add 7283 INFO [2022-01-10 08:39:39,451] jepsen worker 956 - jepsen.util 956 :info :add 5917 :RequestTimeoutException INFO [2022-01-10 08:39:39,452] jepsen worker 836 - jepsen.util 836 :info :add 5918 :RequestTimeoutException INFO [2022-01-10 08:39:39,455] jepsen worker 956 - jepsen.util 1956 :invoke :add 7284 INFO [2022-01-10 08:39:39,457] jepsen worker 306 - jepsen.util 306 :info :add 5919 :RequestTimeoutException INFO [2022-01-10 08:39:39,458] jepsen worker 836 - jepsen.util 1836 :invoke :add 7285 INFO [2022-01-10 08:39:39,460] jepsen worker 306 - jepsen.util 1306 :invoke :add 7286 INFO [2022-01-10 08:39:39,461] jepsen worker 156 - jepsen.util 156 :info :add 5923 :RequestTimeoutException INFO [2022-01-10 08:39:39,465] jepsen worker 156 - jepsen.util 1156 :invoke :add 7287 INFO [2022-01-10 08:39:39,473] jepsen worker 856 - jepsen.util 856 :info :add 5928 :RequestTimeoutException INFO [2022-01-10 08:39:39,474] jepsen worker 107 - jepsen.util 107 :info :add 5929 :RequestTimeoutException INFO [2022-01-10 08:39:39,478] jepsen worker 856 - jepsen.util 1856 :invoke :add 7288 INFO [2022-01-10 08:39:39,479] jepsen worker 107 - jepsen.util 1107 :invoke :add 7289 INFO [2022-01-10 08:39:39,482] jepsen worker 69 - jepsen.util 69 :info :add 5934 :RequestTimeoutException INFO [2022-01-10 08:39:39,486] jepsen worker 69 - jepsen.util 1069 :invoke :add 7290 INFO [2022-01-10 08:39:39,491] jepsen worker 731 - jepsen.util 731 :info :add 5935 :RequestTimeoutException INFO [2022-01-10 08:39:39,492] jepsen worker 377 - jepsen.util 377 :info :add 5937 :RequestTimeoutException INFO [2022-01-10 08:39:39,493] jepsen worker 410 - jepsen.util 410 :info :add 5936 :RequestTimeoutException INFO [2022-01-10 08:39:39,494] jepsen worker 731 - jepsen.util 1731 :invoke :add 7291 INFO [2022-01-10 08:39:39,497] jepsen worker 410 - jepsen.util 1410 :invoke :add 7292 INFO [2022-01-10 08:39:39,499] jepsen worker 377 - jepsen.util 1377 :invoke :add 7293 INFO [2022-01-10 08:39:39,502] jepsen worker 759 - jepsen.util 759 :info :add 5944 :RequestTimeoutException INFO [2022-01-10 08:39:39,504] jepsen worker 269 - jepsen.util 269 :info :add 5945 :RequestTimeoutException INFO [2022-01-10 08:39:39,507] jepsen worker 759 - jepsen.util 1759 :invoke :add 7294 INFO [2022-01-10 08:39:39,507] jepsen worker 992 - jepsen.util 992 :info :add 5946 :RequestTimeoutException INFO [2022-01-10 08:39:39,509] jepsen worker 269 - jepsen.util 1269 :invoke :add 7295 INFO [2022-01-10 08:39:39,512] jepsen worker 992 - jepsen.util 1992 :invoke :add 7296 INFO [2022-01-10 08:39:39,513] jepsen worker 985 - jepsen.util 985 :info :add 5947 :RequestTimeoutException INFO [2022-01-10 08:39:39,516] jepsen worker 985 - jepsen.util 1985 :invoke :add 7297 INFO [2022-01-10 08:39:39,521] jepsen worker 433 - jepsen.util 433 :info :add 5949 :RequestTimeoutException INFO [2022-01-10 08:39:39,523] jepsen worker 433 - jepsen.util 1433 :invoke :add 7298 INFO [2022-01-10 08:39:39,524] jepsen worker 723 - jepsen.util 723 :info :add 5951 :RequestTimeoutException INFO [2022-01-10 08:39:39,527] jepsen worker 723 - jepsen.util 1723 :invoke :add 7299 INFO [2022-01-10 08:39:39,551] jepsen worker 652 - jepsen.util 652 :info :add 5965 :RequestTimeoutException INFO [2022-01-10 08:39:39,552] jepsen worker 407 - jepsen.util 407 :info :add 5969 :RequestTimeoutException INFO [2022-01-10 08:39:39,555] jepsen worker 652 - jepsen.util 1652 :invoke :add 7300 INFO [2022-01-10 08:39:39,557] jepsen worker 407 - jepsen.util 1407 :invoke :add 7301 INFO [2022-01-10 08:39:39,558] jepsen worker 973 - jepsen.util 973 :info :add 5968 :RequestTimeoutException INFO [2022-01-10 08:39:39,561] jepsen worker 973 - jepsen.util 1973 :invoke :add 7302 INFO [2022-01-10 08:39:39,563] jepsen worker 673 - jepsen.util 673 :info :add 5971 :RequestTimeoutException INFO [2022-01-10 08:39:39,564] jepsen worker 428 - jepsen.util 428 :info :add 5975 :RequestTimeoutException INFO [2022-01-10 08:39:39,565] jepsen worker 673 - jepsen.util 1673 :invoke :add 7303 INFO [2022-01-10 08:39:39,567] jepsen worker 942 - jepsen.util 942 :info :add 5973 :RequestTimeoutException INFO [2022-01-10 08:39:39,568] jepsen worker 428 - jepsen.util 1428 :invoke :add 7304 INFO [2022-01-10 08:39:39,570] jepsen worker 942 - jepsen.util 1942 :invoke :add 7305 INFO [2022-01-10 08:39:39,590] jepsen worker 675 - jepsen.util 675 :info :add 5985 :RequestTimeoutException INFO [2022-01-10 08:39:39,593] jepsen worker 675 - jepsen.util 1675 :invoke :add 7306 INFO [2022-01-10 08:39:39,597] jepsen worker 503 - jepsen.util 503 :info :add 5988 :RequestTimeoutException INFO [2022-01-10 08:39:39,600] jepsen worker 503 - jepsen.util 1503 :invoke :add 7307 INFO [2022-01-10 08:39:39,601] jepsen worker 277 - jepsen.util 277 :info :add 5990 :RequestTimeoutException INFO [2022-01-10 08:39:39,601] jepsen worker 317 - jepsen.util 317 :info :add 5991 :RequestTimeoutException INFO [2022-01-10 08:39:39,602] jepsen worker 365 - jepsen.util 365 :info :add 5992 :RequestTimeoutException INFO [2022-01-10 08:39:39,604] jepsen worker 711 - jepsen.util 711 :info :add 5993 :RequestTimeoutException INFO [2022-01-10 08:39:39,607] jepsen worker 64 - jepsen.util 64 :info :add 5995 :RequestTimeoutException INFO [2022-01-10 08:39:39,609] jepsen worker 365 - jepsen.util 1365 :invoke :add 7308 INFO [2022-01-10 08:39:39,612] jepsen worker 277 - jepsen.util 1277 :invoke :add 7309 INFO [2022-01-10 08:39:39,613] jepsen worker 130 - jepsen.util 130 :info :add 5997 :RequestTimeoutException INFO [2022-01-10 08:39:39,614] jepsen worker 64 - jepsen.util 1064 :invoke :add 7310 INFO [2022-01-10 08:39:39,616] jepsen worker 317 - jepsen.util 1317 :invoke :add 7311 INFO [2022-01-10 08:39:39,618] jepsen worker 711 - jepsen.util 1711 :invoke :add 7312 INFO [2022-01-10 08:39:39,620] jepsen worker 130 - jepsen.util 1130 :invoke :add 7313 INFO [2022-01-10 08:39:39,627] jepsen worker 817 - jepsen.util 817 :info :add 6001 :RequestTimeoutException INFO [2022-01-10 08:39:39,627] jepsen worker 40 - jepsen.util 40 :info :add 6003 :RequestTimeoutException INFO [2022-01-10 08:39:39,631] jepsen worker 40 - jepsen.util 1040 :invoke :add 7314 INFO [2022-01-10 08:39:39,632] jepsen worker 817 - jepsen.util 1817 :invoke :add 7315 INFO [2022-01-10 08:39:39,632] jepsen worker 316 - jepsen.util 316 :info :add 6005 :RequestTimeoutException INFO [2022-01-10 08:39:39,632] jepsen worker 296 - jepsen.util 296 :info :add 6006 :RequestTimeoutException INFO [2022-01-10 08:39:39,634] jepsen worker 327 - jepsen.util 327 :info :add 6007 :RequestTimeoutException INFO [2022-01-10 08:39:39,635] jepsen worker 51 - jepsen.util 51 :info :add 6004 :RequestTimeoutException INFO [2022-01-10 08:39:39,639] jepsen worker 51 - jepsen.util 1051 :invoke :add 7316 INFO [2022-01-10 08:39:39,641] jepsen worker 316 - jepsen.util 1316 :invoke :add 7317 INFO [2022-01-10 08:39:39,643] jepsen worker 296 - jepsen.util 1296 :invoke :add 7318 INFO [2022-01-10 08:39:39,644] jepsen worker 327 - jepsen.util 1327 :invoke :add 7319 INFO [2022-01-10 08:39:39,661] jepsen worker 674 - jepsen.util 674 :ok :add 6283 INFO [2022-01-10 08:39:39,661] jepsen worker 5 - jepsen.util 5 :ok :add 6455 INFO [2022-01-10 08:39:39,663] jepsen worker 5 - jepsen.util 5 :invoke :add 7320 INFO [2022-01-10 08:39:39,665] jepsen worker 674 - jepsen.util 674 :invoke :add 7321 INFO [2022-01-10 08:39:39,681] jepsen worker 653 - jepsen.util 653 :info :add 6030 :RequestTimeoutException INFO [2022-01-10 08:39:39,683] jepsen worker 250 - jepsen.util 250 :info :add 6029 :RequestTimeoutException INFO [2022-01-10 08:39:39,683] jepsen worker 573 - jepsen.util 573 :info :add 6028 :RequestTimeoutException INFO [2022-01-10 08:39:39,687] jepsen worker 250 - jepsen.util 1250 :invoke :add 7322 INFO [2022-01-10 08:39:39,689] jepsen worker 653 - jepsen.util 1653 :invoke :add 7323 INFO [2022-01-10 08:39:39,691] jepsen worker 165 - jepsen.util 165 :info :add 6033 :RequestTimeoutException INFO [2022-01-10 08:39:39,691] jepsen worker 804 - jepsen.util 804 :info :add 6031 :RequestTimeoutException INFO [2022-01-10 08:39:39,691] jepsen worker 573 - jepsen.util 1573 :invoke :add 7324 INFO [2022-01-10 08:39:39,692] jepsen worker 751 - jepsen.util 751 :info :add 6032 :RequestTimeoutException INFO [2022-01-10 08:39:39,693] jepsen worker 914 - jepsen.util 914 :info :add 6034 :RequestTimeoutException INFO [2022-01-10 08:39:39,693] jepsen worker 713 - jepsen.util 713 :info :add 6037 :RequestTimeoutException INFO [2022-01-10 08:39:39,698] jepsen worker 751 - jepsen.util 1751 :invoke :add 7325 INFO [2022-01-10 08:39:39,700] jepsen worker 914 - jepsen.util 1914 :invoke :add 7326 INFO [2022-01-10 08:39:39,701] jepsen worker 165 - jepsen.util 1165 :invoke :add 7327 INFO [2022-01-10 08:39:39,704] jepsen worker 713 - jepsen.util 1713 :invoke :add 7328 INFO [2022-01-10 08:39:39,706] jepsen worker 804 - jepsen.util 1804 :invoke :add 7329 INFO [2022-01-10 08:39:39,707] jepsen worker 167 - jepsen.util 167 :info :add 6041 :RequestTimeoutException INFO [2022-01-10 08:39:39,707] jepsen worker 6 - jepsen.util 6 :info :add 6042 :RequestTimeoutException INFO [2022-01-10 08:39:39,711] jepsen worker 167 - jepsen.util 1167 :invoke :add 7330 INFO [2022-01-10 08:39:39,712] jepsen worker 587 - jepsen.util 587 :info :add 6043 :RequestTimeoutException INFO [2022-01-10 08:39:39,713] jepsen worker 6 - jepsen.util 1006 :invoke :add 7331 INFO [2022-01-10 08:39:39,714] jepsen worker 570 - jepsen.util 570 :info :add 6046 :RequestTimeoutException INFO [2022-01-10 08:39:39,714] jepsen worker 239 - jepsen.util 239 :info :add 6047 :RequestTimeoutException INFO [2022-01-10 08:39:39,717] jepsen worker 239 - jepsen.util 1239 :invoke :add 7332 INFO [2022-01-10 08:39:39,719] jepsen worker 587 - jepsen.util 1587 :invoke :add 7333 INFO [2022-01-10 08:39:39,720] jepsen worker 570 - jepsen.util 1570 :invoke :add 7334 INFO [2022-01-10 08:39:39,744] jepsen worker 611 - jepsen.util 611 :info :add 6058 :RequestTimeoutException INFO [2022-01-10 08:39:39,744] jepsen worker 159 - jepsen.util 159 :info :add 6059 :RequestTimeoutException INFO [2022-01-10 08:39:39,747] jepsen worker 611 - jepsen.util 1611 :invoke :add 7335 INFO [2022-01-10 08:39:39,749] jepsen worker 159 - jepsen.util 1159 :invoke :add 7336 INFO [2022-01-10 08:39:39,760] jepsen worker 543 - jepsen.util 543 :info :add 6069 :RequestTimeoutException INFO [2022-01-10 08:39:39,763] jepsen worker 910 - jepsen.util 910 :info :add 6068 :RequestTimeoutException INFO [2022-01-10 08:39:39,764] jepsen worker 712 - jepsen.util 712 :info :add 6070 :RequestTimeoutException INFO [2022-01-10 08:39:39,767] jepsen worker 543 - jepsen.util 1543 :invoke :add 7337 INFO [2022-01-10 08:39:39,769] jepsen worker 712 - jepsen.util 1712 :invoke :add 7338 INFO [2022-01-10 08:39:39,770] jepsen worker 910 - jepsen.util 1910 :invoke :add 7339 INFO [2022-01-10 08:39:39,773] jepsen worker 26 - jepsen.util 26 :info :add 6072 :RequestTimeoutException INFO [2022-01-10 08:39:39,776] jepsen worker 26 - jepsen.util 1026 :invoke :add 7340 INFO [2022-01-10 08:39:39,781] jepsen worker 790 - jepsen.util 790 :info :add 6073 :RequestTimeoutException INFO [2022-01-10 08:39:39,781] jepsen worker 775 - jepsen.util 775 :info :add 6074 :RequestTimeoutException INFO [2022-01-10 08:39:39,783] jepsen worker 189 - jepsen.util 189 :info :add 6075 :RequestTimeoutException INFO [2022-01-10 08:39:39,785] jepsen worker 775 - jepsen.util 1775 :invoke :add 7341 INFO [2022-01-10 08:39:39,786] jepsen worker 709 - jepsen.util 709 :ok :add 6389 INFO [2022-01-10 08:39:39,789] jepsen worker 189 - jepsen.util 1189 :invoke :add 7342 INFO [2022-01-10 08:39:39,789] jepsen worker 891 - jepsen.util 891 :ok :add 6493 INFO [2022-01-10 08:39:39,790] jepsen worker 537 - jepsen.util 537 :ok :add 6537 INFO [2022-01-10 08:39:39,790] jepsen worker 833 - jepsen.util 833 :ok :add 6573 INFO [2022-01-10 08:39:39,790] jepsen worker 774 - jepsen.util 774 :ok :add 6245 INFO [2022-01-10 08:39:39,790] jepsen worker 23 - jepsen.util 23 :ok :add 6646 INFO [2022-01-10 08:39:39,790] jepsen worker 901 - jepsen.util 901 :ok :add 6118 INFO [2022-01-10 08:39:39,791] jepsen worker 597 - jepsen.util 597 :info :add 6077 :RequestTimeoutException INFO [2022-01-10 08:39:39,792] jepsen worker 709 - jepsen.util 709 :invoke :add 7343 INFO [2022-01-10 08:39:39,792] jepsen worker 538 - jepsen.util 538 :ok :add 6144 INFO [2022-01-10 08:39:39,792] jepsen worker 819 - jepsen.util 819 :ok :add 6170 INFO [2022-01-10 08:39:39,792] jepsen worker 274 - jepsen.util 274 :ok :add 7187 INFO [2022-01-10 08:39:39,792] jepsen worker 636 - jepsen.util 636 :ok :add 6391 INFO [2022-01-10 08:39:39,792] jepsen worker 634 - jepsen.util 634 :ok :add 6672 INFO [2022-01-10 08:39:39,792] jepsen worker 183 - jepsen.util 183 :ok :add 6100 INFO [2022-01-10 08:39:39,793] jepsen worker 825 - jepsen.util 825 :info :add 6076 :RequestTimeoutException INFO [2022-01-10 08:39:39,797] jepsen worker 679 - jepsen.util 679 :info :add 6082 :RequestTimeoutException INFO [2022-01-10 08:39:39,798] jepsen worker 819 - jepsen.util 819 :invoke :add 7344 INFO [2022-01-10 08:39:39,801] jepsen worker 790 - jepsen.util 1790 :invoke :add 7345 INFO [2022-01-10 08:39:39,803] jepsen worker 597 - jepsen.util 1597 :invoke :add 7346 INFO [2022-01-10 08:39:39,805] jepsen worker 774 - jepsen.util 774 :invoke :add 7347 INFO [2022-01-10 08:39:39,807] jepsen worker 891 - jepsen.util 891 :invoke :add 7348 INFO [2022-01-10 08:39:39,809] jepsen worker 833 - jepsen.util 833 :invoke :add 7349 INFO [2022-01-10 08:39:39,811] jepsen worker 825 - jepsen.util 1825 :invoke :add 7350 INFO [2022-01-10 08:39:39,812] jepsen worker 966 - jepsen.util 966 :info :add 6090 :RequestTimeoutException INFO [2022-01-10 08:39:39,813] jepsen worker 636 - jepsen.util 636 :invoke :add 7351 INFO [2022-01-10 08:39:39,813] jepsen worker 906 - jepsen.util 906 :info :add 6091 :RequestTimeoutException INFO [2022-01-10 08:39:39,815] jepsen worker 538 - jepsen.util 538 :invoke :add 7352 INFO [2022-01-10 08:39:39,818] jepsen worker 23 - jepsen.util 23 :invoke :add 7353 INFO [2022-01-10 08:39:39,820] jepsen worker 966 - jepsen.util 1966 :invoke :add 7354 INFO [2022-01-10 08:39:39,822] jepsen worker 699 - jepsen.util 699 :info :add 6095 :RequestTimeoutException INFO [2022-01-10 08:39:39,822] jepsen worker 782 - jepsen.util 782 :info :add 6094 :RequestTimeoutException INFO [2022-01-10 08:39:39,822] jepsen worker 183 - jepsen.util 183 :invoke :add 7355 INFO [2022-01-10 08:39:39,824] jepsen worker 941 - jepsen.util 941 :info :add 6093 :RequestTimeoutException INFO [2022-01-10 08:39:39,826] jepsen worker 634 - jepsen.util 634 :invoke :add 7356 INFO [2022-01-10 08:39:39,827] jepsen worker 191 - jepsen.util 191 :info :add 6097 :RequestTimeoutException INFO [2022-01-10 08:39:39,829] jepsen worker 906 - jepsen.util 1906 :invoke :add 7357 INFO [2022-01-10 08:39:39,830] jepsen worker 657 - jepsen.util 657 :info :add 6096 :RequestTimeoutException INFO [2022-01-10 08:39:39,832] jepsen worker 782 - jepsen.util 1782 :invoke :add 7358 INFO [2022-01-10 08:39:39,833] jepsen worker 477 - jepsen.util 477 :info :add 6102 :RequestTimeoutException INFO [2022-01-10 08:39:39,835] jepsen worker 699 - jepsen.util 1699 :invoke :add 7359 INFO [2022-01-10 08:39:39,839] jepsen worker 657 - jepsen.util 1657 :invoke :add 7360 INFO [2022-01-10 08:39:39,841] jepsen worker 397 - jepsen.util 397 :info :add 6104 :RequestTimeoutException INFO [2022-01-10 08:39:39,841] jepsen worker 537 - jepsen.util 537 :invoke :add 7361 INFO [2022-01-10 08:39:39,843] jepsen worker 941 - jepsen.util 1941 :invoke :add 7362 INFO [2022-01-10 08:39:39,843] jepsen worker 974 - jepsen.util 974 :info :add 6106 :RequestTimeoutException INFO [2022-01-10 08:39:39,844] jepsen worker 162 - jepsen.util 162 :info :add 6103 :RequestTimeoutException INFO [2022-01-10 08:39:39,845] jepsen worker 308 - jepsen.util 308 :info :add 6107 :RequestTimeoutException INFO [2022-01-10 08:39:39,846] jepsen worker 162 - jepsen.util 1162 :invoke :add 7363 INFO [2022-01-10 08:39:39,847] jepsen worker 50 - jepsen.util 50 :info :add 6105 :RequestTimeoutException INFO [2022-01-10 08:39:39,849] jepsen worker 274 - jepsen.util 274 :invoke :add 7364 INFO [2022-01-10 08:39:39,852] jepsen worker 50 - jepsen.util 1050 :invoke :add 7365 INFO [2022-01-10 08:39:39,852] jepsen worker 778 - jepsen.util 778 :info :add 6112 :RequestTimeoutException INFO [2022-01-10 08:39:39,853] jepsen worker 679 - jepsen.util 1679 :invoke :add 7366 INFO [2022-01-10 08:39:39,856] jepsen worker 974 - jepsen.util 1974 :invoke :add 7367 INFO [2022-01-10 08:39:39,858] jepsen worker 901 - jepsen.util 901 :invoke :add 7368 INFO [2022-01-10 08:39:39,860] jepsen worker 308 - jepsen.util 1308 :invoke :add 7369 INFO [2022-01-10 08:39:39,861] jepsen worker 778 - jepsen.util 1778 :invoke :add 7370 INFO [2022-01-10 08:39:39,863] jepsen worker 477 - jepsen.util 1477 :invoke :add 7371 INFO [2022-01-10 08:39:39,865] jepsen worker 397 - jepsen.util 1397 :invoke :add 7372 INFO [2022-01-10 08:39:39,867] jepsen worker 191 - jepsen.util 1191 :invoke :add 7373 INFO [2022-01-10 08:39:39,884] jepsen worker 823 - jepsen.util 823 :info :add 6129 :RequestTimeoutException INFO [2022-01-10 08:39:39,887] jepsen worker 161 - jepsen.util 161 :info :add 6128 :RequestTimeoutException INFO [2022-01-10 08:39:39,887] jepsen worker 823 - jepsen.util 1823 :invoke :add 7374 INFO [2022-01-10 08:39:39,890] jepsen worker 161 - jepsen.util 1161 :invoke :add 7375 INFO [2022-01-10 08:39:39,890] jepsen worker 591 - jepsen.util 591 :info :add 6130 :RequestTimeoutException INFO [2022-01-10 08:39:39,890] jepsen worker 617 - jepsen.util 617 :info :add 6132 :RequestTimeoutException INFO [2022-01-10 08:39:39,892] jepsen worker 531 - jepsen.util 531 :info :add 6133 :RequestTimeoutException INFO [2022-01-10 08:39:39,893] jepsen worker 517 - jepsen.util 517 :info :add 6135 :RequestTimeoutException INFO [2022-01-10 08:39:39,894] jepsen worker 818 - jepsen.util 818 :info :add 6131 :RequestTimeoutException INFO [2022-01-10 08:39:39,897] jepsen worker 818 - jepsen.util 1818 :invoke :add 7376 INFO [2022-01-10 08:39:39,899] jepsen worker 591 - jepsen.util 1591 :invoke :add 7377 INFO [2022-01-10 08:39:39,900] jepsen worker 517 - jepsen.util 1517 :invoke :add 7378 INFO [2022-01-10 08:39:39,902] jepsen worker 617 - jepsen.util 1617 :invoke :add 7379 INFO [2022-01-10 08:39:39,904] jepsen worker 531 - jepsen.util 1531 :invoke :add 7380 INFO [2022-01-10 08:39:39,910] jepsen worker 52 - jepsen.util 52 :info :add 6140 :RequestTimeoutException INFO [2022-01-10 08:39:39,912] jepsen worker 58 - jepsen.util 58 :info :add 6142 :RequestTimeoutException INFO [2022-01-10 08:39:39,913] jepsen worker 29 - jepsen.util 29 :info :add 6141 :RequestTimeoutException INFO [2022-01-10 08:39:39,913] jepsen worker 703 - jepsen.util 703 :info :add 6145 :RequestTimeoutException INFO [2022-01-10 08:39:39,913] jepsen worker 927 - jepsen.util 927 :info :add 6143 :RequestTimeoutException INFO [2022-01-10 08:39:39,914] jepsen worker 52 - jepsen.util 1052 :invoke :add 7381 INFO [2022-01-10 08:39:39,918] jepsen worker 496 - jepsen.util 496 :info :add 6147 :RequestTimeoutException INFO [2022-01-10 08:39:39,920] jepsen worker 58 - jepsen.util 1058 :invoke :add 7382 INFO [2022-01-10 08:39:39,922] jepsen worker 29 - jepsen.util 1029 :invoke :add 7383 INFO [2022-01-10 08:39:39,924] jepsen worker 927 - jepsen.util 1927 :invoke :add 7384 INFO [2022-01-10 08:39:39,926] jepsen worker 703 - jepsen.util 1703 :invoke :add 7385 INFO [2022-01-10 08:39:39,928] jepsen worker 496 - jepsen.util 1496 :invoke :add 7386 INFO [2022-01-10 08:39:39,941] jepsen worker 900 - jepsen.util 900 :info :add 6158 :RequestTimeoutException INFO [2022-01-10 08:39:39,941] jepsen worker 39 - jepsen.util 39 :info :add 6157 :RequestTimeoutException INFO [2022-01-10 08:39:39,943] jepsen worker 888 - jepsen.util 888 :info :add 6159 :RequestTimeoutException INFO [2022-01-10 08:39:39,945] jepsen worker 900 - jepsen.util 1900 :invoke :add 7387 INFO [2022-01-10 08:39:39,947] jepsen worker 39 - jepsen.util 1039 :invoke :add 7388 INFO [2022-01-10 08:39:39,949] jepsen worker 888 - jepsen.util 1888 :invoke :add 7389 INFO [2022-01-10 08:39:39,950] jepsen worker 553 - jepsen.util 553 :info :add 6162 :RequestTimeoutException INFO [2022-01-10 08:39:39,953] jepsen worker 553 - jepsen.util 1553 :invoke :add 7390 INFO [2022-01-10 08:39:39,954] jepsen worker 113 - jepsen.util 113 :info :add 6163 :RequestTimeoutException INFO [2022-01-10 08:39:39,957] jepsen worker 113 - jepsen.util 1113 :invoke :add 7391 INFO [2022-01-10 08:39:39,960] jepsen worker 700 - jepsen.util 700 :info :add 6169 :RequestTimeoutException INFO [2022-01-10 08:39:39,964] jepsen worker 700 - jepsen.util 1700 :invoke :add 7392 INFO [2022-01-10 08:39:39,967] jepsen worker 396 - jepsen.util 396 :info :add 6173 :RequestTimeoutException INFO [2022-01-10 08:39:39,969] jepsen worker 396 - jepsen.util 1396 :invoke :add 7393 INFO [2022-01-10 08:39:39,971] jepsen worker 580 - jepsen.util 580 :info :add 6174 :RequestTimeoutException INFO [2022-01-10 08:39:39,973] jepsen worker 580 - jepsen.util 1580 :invoke :add 7394 INFO [2022-01-10 08:39:39,973] jepsen worker 599 - jepsen.util 599 :info :add 6175 :RequestTimeoutException INFO [2022-01-10 08:39:39,973] jepsen worker 323 - jepsen.util 323 :info :add 6176 :RequestTimeoutException INFO [2022-01-10 08:39:39,974] jepsen worker 163 - jepsen.util 163 :info :add 6177 :RequestTimeoutException INFO [2022-01-10 08:39:39,978] jepsen worker 599 - jepsen.util 1599 :invoke :add 7395 INFO [2022-01-10 08:39:39,979] jepsen worker 163 - jepsen.util 1163 :invoke :add 7396 INFO [2022-01-10 08:39:39,981] jepsen worker 323 - jepsen.util 1323 :invoke :add 7397 INFO [2022-01-10 08:39:40,011] jepsen worker 158 - jepsen.util 158 :ok :add 6231 INFO [2022-01-10 08:39:40,011] jepsen worker 961 - jepsen.util 961 :ok :add 6395 INFO [2022-01-10 08:39:40,013] jepsen worker 158 - jepsen.util 158 :invoke :add 7398 INFO [2022-01-10 08:39:40,014] jepsen worker 598 - jepsen.util 598 :info :add 6195 :RequestTimeoutException INFO [2022-01-10 08:39:40,016] jepsen worker 961 - jepsen.util 961 :invoke :add 7399 INFO [2022-01-10 08:39:40,017] jepsen worker 502 - jepsen.util 502 :info :add 6193 :RequestTimeoutException INFO [2022-01-10 08:39:40,017] jepsen worker 524 - jepsen.util 524 :info :add 6194 :RequestTimeoutException INFO [2022-01-10 08:39:40,019] jepsen worker 598 - jepsen.util 1598 :invoke :add 7400 INFO [2022-01-10 08:39:40,022] jepsen worker 792 - jepsen.util 792 :info :add 6196 :RequestTimeoutException INFO [2022-01-10 08:39:40,022] jepsen worker 93 - jepsen.util 93 :info :add 6197 :RequestTimeoutException INFO [2022-01-10 08:39:40,023] jepsen worker 502 - jepsen.util 1502 :invoke :add 7401 INFO [2022-01-10 08:39:40,026] jepsen worker 792 - jepsen.util 1792 :invoke :add 7402 INFO [2022-01-10 08:39:40,028] jepsen worker 524 - jepsen.util 1524 :invoke :add 7403 INFO [2022-01-10 08:39:40,031] jepsen worker 93 - jepsen.util 1093 :invoke :add 7404 INFO [2022-01-10 08:39:40,033] jepsen worker 187 - jepsen.util 187 :info :add 6205 :RequestTimeoutException INFO [2022-01-10 08:39:40,035] jepsen worker 187 - jepsen.util 1187 :invoke :add 7405 INFO [2022-01-10 08:39:40,037] jepsen worker 935 - jepsen.util 935 :info :add 6204 :RequestTimeoutException INFO [2022-01-10 08:39:40,039] jepsen worker 935 - jepsen.util 1935 :invoke :add 7406 INFO [2022-01-10 08:39:40,040] jepsen worker 363 - jepsen.util 363 :info :add 6207 :RequestTimeoutException INFO [2022-01-10 08:39:40,042] jepsen worker 908 - jepsen.util 908 :info :add 6206 :RequestTimeoutException INFO [2022-01-10 08:39:40,080] jepsen worker 363 - jepsen.util 1363 :invoke :add 7407 INFO [2022-01-10 08:39:40,053] jepsen worker 988 - jepsen.util 988 :info :add 6212 :RequestTimeoutException INFO [2022-01-10 08:39:40,080] jepsen worker 988 - jepsen.util 1988 :invoke :add 7409 INFO [2022-01-10 08:39:40,080] jepsen worker 311 - jepsen.util 311 :info :add 6227 :RequestTimeoutException INFO [2022-01-10 08:39:40,080] jepsen worker 908 - jepsen.util 1908 :invoke :add 7408 INFO [2022-01-10 08:39:40,081] jepsen worker 864 - jepsen.util 864 :info :add 6228 :RequestTimeoutException INFO [2022-01-10 08:39:40,082] jepsen worker 352 - jepsen.util 352 :info :add 6229 :RequestTimeoutException INFO [2022-01-10 08:39:40,083] jepsen worker 689 - jepsen.util 689 :info :add 6226 :RequestTimeoutException INFO [2022-01-10 08:39:40,086] jepsen worker 689 - jepsen.util 1689 :invoke :add 7410 INFO [2022-01-10 08:39:40,087] jepsen worker 311 - jepsen.util 1311 :invoke :add 7411 INFO [2022-01-10 08:39:40,089] jepsen worker 352 - jepsen.util 1352 :invoke :add 7412 INFO [2022-01-10 08:39:40,090] jepsen worker 442 - jepsen.util 442 :info :add 6230 :RequestTimeoutException INFO [2022-01-10 08:39:40,091] jepsen worker 864 - jepsen.util 1864 :invoke :add 7413 INFO [2022-01-10 08:39:40,093] jepsen worker 180 - jepsen.util 180 :info :add 6232 :RequestTimeoutException INFO [2022-01-10 08:39:40,094] jepsen worker 442 - jepsen.util 1442 :invoke :add 7414 INFO [2022-01-10 08:39:40,096] jepsen worker 180 - jepsen.util 1180 :invoke :add 7415 INFO [2022-01-10 08:39:40,101] jepsen worker 288 - jepsen.util 288 :info :add 6233 :RequestTimeoutException INFO [2022-01-10 08:39:40,104] jepsen worker 288 - jepsen.util 1288 :invoke :add 7416 INFO [2022-01-10 08:39:40,112] jepsen worker 558 - jepsen.util 558 :info :add 6236 :RequestTimeoutException INFO [2022-01-10 08:39:40,112] jepsen worker 820 - jepsen.util 820 :info :add 6240 :RequestTimeoutException INFO [2022-01-10 08:39:40,113] jepsen worker 989 - jepsen.util 989 :info :add 6242 :RequestTimeoutException INFO [2022-01-10 08:39:40,114] jepsen worker 930 - jepsen.util 930 :info :add 6241 :RequestTimeoutException INFO [2022-01-10 08:39:40,119] jepsen worker 820 - jepsen.util 1820 :invoke :add 7417 INFO [2022-01-10 08:39:40,121] jepsen worker 955 - jepsen.util 955 :info :add 6243 :RequestTimeoutException INFO [2022-01-10 08:39:40,122] jepsen worker 930 - jepsen.util 1930 :invoke :add 7418 INFO [2022-01-10 08:39:40,124] jepsen worker 955 - jepsen.util 1955 :invoke :add 7419 INFO [2022-01-10 08:39:40,126] jepsen worker 989 - jepsen.util 1989 :invoke :add 7420 INFO [2022-01-10 08:39:40,127] jepsen worker 558 - jepsen.util 1558 :invoke :add 7421 INFO [2022-01-10 08:39:40,142] jepsen worker 0 - jepsen.util 0 :info :add 6259 :RequestTimeoutException INFO [2022-01-10 08:39:40,143] jepsen worker 399 - jepsen.util 399 :info :add 6258 :RequestTimeoutException INFO [2022-01-10 08:39:40,146] jepsen worker 0 - jepsen.util 1000 :invoke :add 7422 INFO [2022-01-10 08:39:40,147] jepsen worker 399 - jepsen.util 1399 :invoke :add 7423 INFO [2022-01-10 08:39:40,151] jepsen worker 38 - jepsen.util 38 :info :add 6263 :RequestTimeoutException INFO [2022-01-10 08:39:40,154] jepsen worker 38 - jepsen.util 1038 :invoke :add 7424 INFO [2022-01-10 08:39:40,164] jepsen worker 642 - jepsen.util 642 :info :add 6270 :RequestTimeoutException INFO [2022-01-10 08:39:40,166] jepsen worker 628 - jepsen.util 628 :info :add 6274 :RequestTimeoutException INFO [2022-01-10 08:39:40,167] jepsen worker 642 - jepsen.util 1642 :invoke :add 7425 INFO [2022-01-10 08:39:40,168] jepsen worker 719 - jepsen.util 719 :ok :add 6382 INFO [2022-01-10 08:39:40,170] jepsen worker 628 - jepsen.util 1628 :invoke :add 7426 INFO [2022-01-10 08:39:40,171] jepsen worker 743 - jepsen.util 743 :info :add 6276 :RequestTimeoutException INFO [2022-01-10 08:39:40,173] jepsen worker 719 - jepsen.util 719 :invoke :add 7427 INFO [2022-01-10 08:39:40,173] jepsen worker 589 - jepsen.util 589 :info :add 6273 :RequestTimeoutException INFO [2022-01-10 08:39:40,174] jepsen worker 726 - jepsen.util 726 :info :add 6277 :RequestTimeoutException INFO [2022-01-10 08:39:40,177] jepsen worker 726 - jepsen.util 1726 :invoke :add 7428 INFO [2022-01-10 08:39:40,178] jepsen worker 314 - jepsen.util 314 :info :add 6275 :RequestTimeoutException INFO [2022-01-10 08:39:40,179] jepsen worker 589 - jepsen.util 1589 :invoke :add 7429 INFO [2022-01-10 08:39:40,182] jepsen worker 743 - jepsen.util 1743 :invoke :add 7430 INFO [2022-01-10 08:39:40,184] jepsen worker 314 - jepsen.util 1314 :invoke :add 7431 INFO [2022-01-10 08:39:40,287] jepsen worker 223 - jepsen.util 223 :ok :add 6647 INFO [2022-01-10 08:39:40,288] jepsen worker 401 - jepsen.util 401 :ok :add 6536 INFO [2022-01-10 08:39:40,289] jepsen worker 223 - jepsen.util 223 :invoke :add 7432 INFO [2022-01-10 08:39:40,289] jepsen worker 934 - jepsen.util 934 :ok :add 6673 INFO [2022-01-10 08:39:40,291] jepsen worker 934 - jepsen.util 934 :invoke :add 7433 INFO [2022-01-10 08:39:40,293] jepsen worker 401 - jepsen.util 401 :invoke :add 7434 INFO [2022-01-10 08:39:40,293] jepsen worker 55 - jepsen.util 55 :ok :add 6619 INFO [2022-01-10 08:39:40,293] jepsen worker 98 - jepsen.util 98 :ok :add 6635 INFO [2022-01-10 08:39:40,293] jepsen worker 683 - jepsen.util 683 :ok :add 6544 INFO [2022-01-10 08:39:40,297] jepsen worker 683 - jepsen.util 683 :invoke :add 7435 INFO [2022-01-10 08:39:40,297] jepsen worker 115 - jepsen.util 115 :ok :add 6392 INFO [2022-01-10 08:39:40,297] jepsen worker 492 - jepsen.util 492 :ok :add 6601 INFO [2022-01-10 08:39:40,298] jepsen worker 181 - jepsen.util 181 :ok :add 6789 INFO [2022-01-10 08:39:40,298] jepsen worker 55 - jepsen.util 55 :invoke :add 7436 INFO [2022-01-10 08:39:40,301] jepsen worker 98 - jepsen.util 98 :invoke :add 7437 INFO [2022-01-10 08:39:40,303] jepsen worker 181 - jepsen.util 181 :invoke :add 7438 INFO [2022-01-10 08:39:40,305] jepsen worker 115 - jepsen.util 115 :invoke :add 7439 INFO [2022-01-10 08:39:40,307] jepsen worker 492 - jepsen.util 492 :invoke :add 7440 INFO [2022-01-10 08:39:40,353] jepsen worker 122 - jepsen.util 122 :info :add 6289 :RequestTimeoutException INFO [2022-01-10 08:39:40,353] jepsen worker 903 - jepsen.util 903 :info :add 6290 :RequestTimeoutException INFO [2022-01-10 08:39:40,358] jepsen worker 903 - jepsen.util 1903 :invoke :add 7441 INFO [2022-01-10 08:39:40,361] jepsen worker 766 - jepsen.util 766 :info :add 6291 :RequestTimeoutException INFO [2022-01-10 08:39:40,361] jepsen worker 122 - jepsen.util 1122 :invoke :add 7442 INFO [2022-01-10 08:39:40,363] jepsen worker 766 - jepsen.util 1766 :invoke :add 7443 INFO [2022-01-10 08:39:40,492] jepsen worker 463 - jepsen.util 463 :info :add 6294 :RequestTimeoutException INFO [2022-01-10 08:39:40,492] jepsen worker 268 - jepsen.util 268 :info :add 6295 :RequestTimeoutException INFO [2022-01-10 08:39:40,493] jepsen worker 439 - jepsen.util 439 :info :add 6293 :RequestTimeoutException INFO [2022-01-10 08:39:40,496] jepsen worker 463 - jepsen.util 1463 :invoke :add 7444 INFO [2022-01-10 08:39:40,497] jepsen worker 439 - jepsen.util 1439 :invoke :add 7445 INFO [2022-01-10 08:39:40,499] jepsen worker 268 - jepsen.util 1268 :invoke :add 7446 INFO [2022-01-10 08:39:40,501] jepsen worker 684 - jepsen.util 684 :info :add 6297 :RequestTimeoutException INFO [2022-01-10 08:39:40,503] jepsen worker 684 - jepsen.util 1684 :invoke :add 7447 INFO [2022-01-10 08:39:40,503] jepsen worker 411 - jepsen.util 411 :info :add 6296 :RequestTimeoutException INFO [2022-01-10 08:39:40,507] jepsen worker 411 - jepsen.util 1411 :invoke :add 7448 INFO [2022-01-10 08:39:40,534] jepsen worker 949 - jepsen.util 949 :ok :add 6688 INFO [2022-01-10 08:39:40,534] jepsen worker 702 - jepsen.util 702 :ok :add 6734 INFO [2022-01-10 08:39:40,534] jepsen worker 418 - jepsen.util 418 :ok :add 6811 INFO [2022-01-10 08:39:40,534] jepsen worker 847 - jepsen.util 847 :ok :add 6690 INFO [2022-01-10 08:39:40,534] jepsen worker 403 - jepsen.util 403 :ok :add 6700 INFO [2022-01-10 08:39:40,534] jepsen worker 421 - jepsen.util 1421 :ok :add 6981 INFO [2022-01-10 08:39:40,537] jepsen worker 470 - jepsen.util 470 :ok :add 6337 INFO [2022-01-10 08:39:40,537] jepsen worker 702 - jepsen.util 702 :invoke :add 7449 INFO [2022-01-10 08:39:40,539] jepsen worker 470 - jepsen.util 470 :invoke :add 7450 INFO [2022-01-10 08:39:40,541] jepsen worker 949 - jepsen.util 949 :invoke :add 7451 INFO [2022-01-10 08:39:40,543] jepsen worker 418 - jepsen.util 418 :invoke :add 7452 INFO [2022-01-10 08:39:40,545] jepsen worker 403 - jepsen.util 403 :invoke :add 7453 INFO [2022-01-10 08:39:40,548] jepsen worker 847 - jepsen.util 847 :invoke :add 7454 INFO [2022-01-10 08:39:40,550] jepsen worker 421 - jepsen.util 1421 :invoke :add 7455 INFO [2022-01-10 08:39:40,551] jepsen worker 590 - jepsen.util 590 :ok :add 6499 INFO [2022-01-10 08:39:40,552] jepsen worker 912 - jepsen.util 912 :ok :add 6511 INFO [2022-01-10 08:39:40,554] jepsen worker 590 - jepsen.util 590 :invoke :add 7456 INFO [2022-01-10 08:39:40,556] jepsen worker 912 - jepsen.util 912 :invoke :add 7457 INFO [2022-01-10 08:39:40,557] jepsen worker 824 - jepsen.util 824 :ok :add 6481 INFO [2022-01-10 08:39:40,560] jepsen worker 824 - jepsen.util 824 :invoke :add 7458 INFO [2022-01-10 08:39:40,621] jepsen worker 959 - jepsen.util 959 :info :add 6304 :RequestTimeoutException INFO [2022-01-10 08:39:40,626] jepsen worker 959 - jepsen.util 1959 :invoke :add 7459 INFO [2022-01-10 08:39:40,627] jepsen worker 884 - jepsen.util 884 :info :add 6305 :RequestTimeoutException INFO [2022-01-10 08:39:40,628] jepsen worker 740 - jepsen.util 740 :info :add 6306 :RequestTimeoutException INFO [2022-01-10 08:39:40,631] jepsen worker 740 - jepsen.util 1740 :invoke :add 7460 INFO [2022-01-10 08:39:40,632] jepsen worker 379 - jepsen.util 379 :info :add 6307 :RequestTimeoutException INFO [2022-01-10 08:39:40,633] jepsen worker 884 - jepsen.util 1884 :invoke :add 7461 INFO [2022-01-10 08:39:40,638] jepsen worker 379 - jepsen.util 1379 :invoke :add 7462 INFO [2022-01-10 08:39:40,734] jepsen worker 730 - jepsen.util 730 :info :add 6325 :RequestTimeoutException INFO [2022-01-10 08:39:40,738] jepsen worker 730 - jepsen.util 1730 :invoke :add 7463 INFO [2022-01-10 08:39:40,741] jepsen worker 467 - jepsen.util 467 :info :add 6328 :RequestTimeoutException INFO [2022-01-10 08:39:40,744] jepsen worker 467 - jepsen.util 1467 :invoke :add 7464 INFO [2022-01-10 08:39:40,750] jepsen worker 233 - jepsen.util 233 :info :add 6329 :RequestTimeoutException INFO [2022-01-10 08:39:40,750] jepsen worker 644 - jepsen.util 644 :info :add 6332 :RequestTimeoutException INFO [2022-01-10 08:39:40,752] jepsen worker 753 - jepsen.util 753 :info :add 6330 :RequestTimeoutException INFO [2022-01-10 08:39:40,753] jepsen worker 651 - jepsen.util 651 :info :add 6331 :RequestTimeoutException INFO [2022-01-10 08:39:40,755] jepsen worker 233 - jepsen.util 1233 :invoke :add 7465 INFO [2022-01-10 08:39:40,761] jepsen worker 651 - jepsen.util 1651 :invoke :add 7466 INFO [2022-01-10 08:39:40,761] jepsen worker 185 - jepsen.util 185 :info :add 6333 :RequestTimeoutException INFO [2022-01-10 08:39:40,761] jepsen worker 356 - jepsen.util 356 :info :add 6334 :RequestTimeoutException INFO [2022-01-10 08:39:40,766] jepsen worker 644 - jepsen.util 1644 :invoke :add 7467 INFO [2022-01-10 08:39:40,769] jepsen worker 185 - jepsen.util 1185 :invoke :add 7468 INFO [2022-01-10 08:39:40,772] jepsen worker 753 - jepsen.util 1753 :invoke :add 7469 INFO [2022-01-10 08:39:40,775] jepsen worker 356 - jepsen.util 1356 :invoke :add 7470 INFO [2022-01-10 08:39:40,807] jepsen worker 960 - jepsen.util 960 :ok :add 6736 INFO [2022-01-10 08:39:40,807] jepsen worker 980 - jepsen.util 980 :ok :add 6692 INFO [2022-01-10 08:39:40,808] jepsen worker 865 - jepsen.util 1865 :ok :add 7117 INFO [2022-01-10 08:39:40,808] jepsen worker 885 - jepsen.util 1885 :ok :add 7123 INFO [2022-01-10 08:39:40,813] jepsen worker 960 - jepsen.util 960 :invoke :add 7471 INFO [2022-01-10 08:39:40,816] jepsen worker 885 - jepsen.util 1885 :invoke :add 7472 INFO [2022-01-10 08:39:40,817] jepsen worker 865 - jepsen.util 1865 :invoke :add 7473 INFO [2022-01-10 08:39:40,819] jepsen worker 980 - jepsen.util 980 :invoke :add 7474 INFO [2022-01-10 08:39:40,828] jepsen worker 153 - jepsen.util 153 :info :add 6340 :RequestTimeoutException INFO [2022-01-10 08:39:40,830] jepsen worker 153 - jepsen.util 1153 :invoke :add 7475 INFO [2022-01-10 08:39:40,833] jepsen worker 646 - jepsen.util 646 :info :add 6341 :RequestTimeoutException INFO [2022-01-10 08:39:40,837] jepsen worker 646 - jepsen.util 1646 :invoke :add 7476 INFO [2022-01-10 08:39:41,103] jepsen worker 854 - jepsen.util 854 :info :add 6342 :RequestTimeoutException INFO [2022-01-10 08:39:41,103] jepsen worker 958 - jepsen.util 958 :info :add 6343 :RequestTimeoutException INFO [2022-01-10 08:39:41,106] jepsen worker 958 - jepsen.util 1958 :invoke :add 7477 INFO [2022-01-10 08:39:41,108] jepsen worker 854 - jepsen.util 1854 :invoke :add 7478 INFO [2022-01-10 08:39:41,146] jepsen worker 414 - jepsen.util 414 :ok :add 6487 INFO [2022-01-10 08:39:41,146] jepsen worker 224 - jepsen.util 224 :ok :add 6517 INFO [2022-01-10 08:39:41,146] jepsen worker 533 - jepsen.util 533 :ok :add 6523 INFO [2022-01-10 08:39:41,150] jepsen worker 533 - jepsen.util 533 :invoke :add 7479 INFO [2022-01-10 08:39:41,152] jepsen worker 224 - jepsen.util 224 :invoke :add 7480 INFO [2022-01-10 08:39:41,155] jepsen worker 414 - jepsen.util 414 :invoke :add 7481 INFO [2022-01-10 08:39:41,215] jepsen worker 904 - jepsen.util 1904 :ok :add 6926 INFO [2022-01-10 08:39:41,215] jepsen worker 21 - jepsen.util 21 :ok :add 6882 INFO [2022-01-10 08:39:41,215] jepsen worker 135 - jepsen.util 135 :ok :add 6572 INFO [2022-01-10 08:39:41,215] jepsen worker 556 - jepsen.util 556 :ok :add 6637 INFO [2022-01-10 08:39:41,215] jepsen worker 974 - jepsen.util 1974 :ok :add 7367 INFO [2022-01-10 08:39:41,216] jepsen worker 245 - jepsen.util 245 :ok :add 6793 INFO [2022-01-10 08:39:41,218] jepsen worker 245 - jepsen.util 245 :invoke :add 7482 INFO [2022-01-10 08:39:41,220] jepsen worker 420 - jepsen.util 420 :info :add 6355 :RequestTimeoutException INFO [2022-01-10 08:39:41,221] jepsen worker 135 - jepsen.util 135 :invoke :add 7483 INFO [2022-01-10 08:39:41,223] jepsen worker 904 - jepsen.util 1904 :invoke :add 7484 INFO [2022-01-10 08:39:41,225] jepsen worker 21 - jepsen.util 21 :invoke :add 7485 INFO [2022-01-10 08:39:41,227] jepsen worker 556 - jepsen.util 556 :invoke :add 7486 INFO [2022-01-10 08:39:41,228] jepsen worker 420 - jepsen.util 1420 :invoke :add 7487 INFO [2022-01-10 08:39:41,232] jepsen worker 974 - jepsen.util 1974 :invoke :add 7488 INFO [2022-01-10 08:39:41,234] jepsen worker 796 - jepsen.util 796 :ok :add 6854 INFO [2022-01-10 08:39:41,234] jepsen worker 690 - jepsen.util 690 :ok :add 7164 INFO [2022-01-10 08:39:41,234] jepsen worker 561 - jepsen.util 561 :ok :add 6860 INFO [2022-01-10 08:39:41,234] jepsen worker 30 - jepsen.util 1030 :ok :add 7215 INFO [2022-01-10 08:39:41,234] jepsen worker 313 - jepsen.util 313 :ok :add 6590 INFO [2022-01-10 08:39:41,234] jepsen worker 779 - jepsen.util 1779 :ok :add 7221 INFO [2022-01-10 08:39:41,234] jepsen worker 697 - jepsen.util 1697 :ok :add 6878 INFO [2022-01-10 08:39:41,235] jepsen worker 658 - jepsen.util 658 :ok :add 6372 INFO [2022-01-10 08:39:41,235] jepsen worker 88 - jepsen.util 88 :ok :add 6769 INFO [2022-01-10 08:39:41,240] jepsen worker 658 - jepsen.util 658 :invoke :add 7489 INFO [2022-01-10 08:39:41,243] jepsen worker 561 - jepsen.util 561 :invoke :add 7490 INFO [2022-01-10 08:39:41,245] jepsen worker 690 - jepsen.util 690 :invoke :add 7491 INFO [2022-01-10 08:39:41,247] jepsen worker 697 - jepsen.util 1697 :invoke :add 7492 INFO [2022-01-10 08:39:41,249] jepsen worker 88 - jepsen.util 88 :invoke :add 7493 INFO [2022-01-10 08:39:41,251] jepsen worker 796 - jepsen.util 796 :invoke :add 7494 INFO [2022-01-10 08:39:41,253] jepsen worker 779 - jepsen.util 1779 :invoke :add 7495 INFO [2022-01-10 08:39:41,254] jepsen worker 313 - jepsen.util 313 :invoke :add 7496 INFO [2022-01-10 08:39:41,256] jepsen worker 30 - jepsen.util 1030 :invoke :add 7497 INFO [2022-01-10 08:39:41,288] jepsen worker 172 - jepsen.util 172 :ok :add 6361 INFO [2022-01-10 08:39:41,288] jepsen worker 592 - jepsen.util 592 :ok :add 6427 INFO [2022-01-10 08:39:41,290] jepsen worker 592 - jepsen.util 592 :invoke :add 7498 INFO [2022-01-10 08:39:41,292] jepsen worker 172 - jepsen.util 172 :invoke :add 7499 INFO [2022-01-10 08:39:41,386] jepsen worker 70 - jepsen.util 70 :ok :add 6397 INFO [2022-01-10 08:39:41,389] jepsen worker 70 - jepsen.util 70 :invoke :add 7500 INFO [2022-01-10 08:39:41,451] jepsen worker 887 - jepsen.util 887 :info :add 6359 :RequestTimeoutException INFO [2022-01-10 08:39:41,453] jepsen worker 887 - jepsen.util 1887 :invoke :add 7501 INFO [2022-01-10 08:39:41,453] jepsen worker 45 - jepsen.util 45 :info :add 6358 :RequestTimeoutException INFO [2022-01-10 08:39:41,458] jepsen worker 45 - jepsen.util 1045 :invoke :add 7502 INFO [2022-01-10 08:39:41,463] jepsen worker 20 - jepsen.util 20 :info :add 6362 :RequestTimeoutException INFO [2022-01-10 08:39:41,466] jepsen worker 20 - jepsen.util 1020 :invoke :add 7503 INFO [2022-01-10 08:39:41,619] jepsen worker 915 - jepsen.util 915 :ok :add 6856 INFO [2022-01-10 08:39:41,619] jepsen worker 462 - jepsen.util 462 :ok :add 6370 INFO [2022-01-10 08:39:41,619] jepsen worker 710 - jepsen.util 710 :ok :add 6773 INFO [2022-01-10 08:39:41,620] jepsen worker 177 - jepsen.util 177 :ok :add 6592 INFO [2022-01-10 08:39:41,620] jepsen worker 453 - jepsen.util 453 :ok :add 6402 INFO [2022-01-10 08:39:41,620] jepsen worker 698 - jepsen.util 698 :ok :add 6747 INFO [2022-01-10 08:39:41,620] jepsen worker 858 - jepsen.util 858 :ok :add 6862 INFO [2022-01-10 08:39:41,621] jepsen worker 710 - jepsen.util 710 :invoke :add 7504 INFO [2022-01-10 08:39:41,624] jepsen worker 435 - jepsen.util 435 :ok :add 6691 INFO [2022-01-10 08:39:41,624] jepsen worker 783 - jepsen.util 783 :ok :add 6808 INFO [2022-01-10 08:39:41,625] jepsen worker 238 - jepsen.util 1238 :ok :add 7138 INFO [2022-01-10 08:39:41,625] jepsen worker 112 - jepsen.util 1112 :ok :add 7084 INFO [2022-01-10 08:39:41,625] jepsen worker 104 - jepsen.util 1104 :ok :add 7114 INFO [2022-01-10 08:39:41,625] jepsen worker 178 - jepsen.util 1178 :ok :add 7120 INFO [2022-01-10 08:39:41,625] jepsen worker 858 - jepsen.util 858 :invoke :add 7505 INFO [2022-01-10 08:39:41,626] jepsen worker 7 - jepsen.util 7 :ok :add 6701 INFO [2022-01-10 08:39:41,626] jepsen worker 835 - jepsen.util 835 :ok :add 6719 INFO [2022-01-10 08:39:41,626] jepsen worker 499 - jepsen.util 499 :ok :add 6824 INFO [2022-01-10 08:39:41,628] jepsen worker 8 - jepsen.util 8 :ok :add 6980 INFO [2022-01-10 08:39:41,629] jepsen worker 238 - jepsen.util 1238 :invoke :add 7506 INFO [2022-01-10 08:39:41,631] jepsen worker 698 - jepsen.util 698 :invoke :add 7507 INFO [2022-01-10 08:39:41,633] jepsen worker 783 - jepsen.util 783 :invoke :add 7508 INFO [2022-01-10 08:39:41,635] jepsen worker 435 - jepsen.util 435 :invoke :add 7509 INFO [2022-01-10 08:39:41,636] jepsen worker 499 - jepsen.util 499 :invoke :add 7510 INFO [2022-01-10 08:39:41,638] jepsen worker 453 - jepsen.util 453 :invoke :add 7511 INFO [2022-01-10 08:39:41,641] jepsen worker 8 - jepsen.util 8 :invoke :add 7512 INFO [2022-01-10 08:39:41,643] jepsen worker 104 - jepsen.util 1104 :invoke :add 7513 WARN [2022-01-10 08:39:41,649] jepsen worker nemesis - jepsen.control.retry com.jcraft.jsch.JSchException with conn [:control "172.23.106.61"]; reopening. INFO [2022-01-10 08:39:41,650] jepsen worker 462 - jepsen.util 462 :invoke :add 7514 INFO [2022-01-10 08:39:41,667] jepsen worker 112 - jepsen.util 1112 :invoke :add 7515 INFO [2022-01-10 08:39:41,668] jepsen worker 835 - jepsen.util 835 :invoke :add 7516 INFO [2022-01-10 08:39:41,670] jepsen worker 178 - jepsen.util 1178 :invoke :add 7517 INFO [2022-01-10 08:39:41,672] jepsen worker 915 - jepsen.util 915 :invoke :add 7518 INFO [2022-01-10 08:39:41,674] jepsen worker 7 - jepsen.util 7 :invoke :add 7519 INFO [2022-01-10 08:39:41,675] jepsen worker 177 - jepsen.util 177 :invoke :add 7520 INFO [2022-01-10 08:39:41,758] jepsen worker 86 - jepsen.util 86 :info :add 6416 :SyncWriteAmbiguous INFO [2022-01-10 08:39:41,762] jepsen worker 749 - jepsen.util 749 :info :add 6415 :SyncWriteAmbiguous INFO [2022-01-10 08:39:41,769] jepsen worker 563 - jepsen.util 563 :info :add 6422 :SyncWriteAmbiguous INFO [2022-01-10 08:39:41,769] jepsen worker 32 - jepsen.util 32 :info :add 6417 :SyncWriteAmbiguous INFO [2022-01-10 08:39:41,773] jepsen worker 745 - jepsen.util 745 :info :add 6423 :SyncWriteAmbiguous INFO [2022-01-10 08:39:41,778] jepsen worker 968 - jepsen.util 968 :info :add 6421 :SyncWriteAmbiguous INFO [2022-01-10 08:39:41,779] jepsen worker 86 - jepsen.util 1086 :invoke :add 7521 INFO [2022-01-10 08:39:41,783] jepsen worker 749 - jepsen.util 1749 :invoke :add 7522 INFO [2022-01-10 08:39:41,785] jepsen worker 745 - jepsen.util 1745 :invoke :add 7523 INFO [2022-01-10 08:39:41,787] jepsen worker 563 - jepsen.util 1563 :invoke :add 7524 INFO [2022-01-10 08:39:41,789] jepsen worker 968 - jepsen.util 1968 :invoke :add 7525 INFO [2022-01-10 08:39:41,790] jepsen worker 32 - jepsen.util 1032 :invoke :add 7526 INFO [2022-01-10 08:39:41,947] jepsen worker 995 - jepsen.util 995 :info :add 6439 :SyncWriteAmbiguous INFO [2022-01-10 08:39:41,947] jepsen worker 66 - jepsen.util 66 :info :add 6438 :SyncWriteAmbiguous INFO [2022-01-10 08:39:41,950] jepsen worker 995 - jepsen.util 1995 :invoke :add 7527 INFO [2022-01-10 08:39:41,952] jepsen worker 66 - jepsen.util 1066 :invoke :add 7528 INFO [2022-01-10 08:39:41,966] jepsen worker 520 - jepsen.util 520 :info :add 6449 :SyncWriteAmbiguous INFO [2022-01-10 08:39:41,969] jepsen worker 520 - jepsen.util 1520 :invoke :add 7529 INFO [2022-01-10 08:39:41,977] jepsen worker 256 - jepsen.util 256 :info :add 6451 :SyncWriteAmbiguous INFO [2022-01-10 08:39:41,977] jepsen worker 216 - jepsen.util 216 :info :add 6453 :SyncWriteAmbiguous INFO [2022-01-10 08:39:41,981] jepsen worker 216 - jepsen.util 1216 :invoke :add 7530 INFO [2022-01-10 08:39:41,983] jepsen worker 256 - jepsen.util 1256 :invoke :add 7531 INFO [2022-01-10 08:39:42,036] jepsen worker 582 - jepsen.util 582 :info :add 6465 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,037] jepsen worker 382 - jepsen.util 382 :info :add 6467 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,042] jepsen worker 132 - jepsen.util 132 :ok :add 6516 INFO [2022-01-10 08:39:42,042] jepsen worker 371 - jepsen.util 371 :ok :add 6522 INFO [2022-01-10 08:39:42,042] jepsen worker 247 - jepsen.util 247 :ok :add 6486 INFO [2022-01-10 08:39:42,043] jepsen worker 506 - jepsen.util 506 :ok :add 6667 INFO [2022-01-10 08:39:42,045] jepsen worker 582 - jepsen.util 1582 :invoke :add 7532 INFO [2022-01-10 08:39:42,047] jepsen worker 606 - jepsen.util 606 :ok :add 6485 INFO [2022-01-10 08:39:42,048] jepsen worker 102 - jepsen.util 102 :ok :add 6539 INFO [2022-01-10 08:39:42,048] jepsen worker 62 - jepsen.util 62 :ok :add 6515 INFO [2022-01-10 08:39:42,048] jepsen worker 564 - jepsen.util 564 :ok :add 6521 INFO [2022-01-10 08:39:42,050] jepsen worker 506 - jepsen.util 506 :invoke :add 7533 INFO [2022-01-10 08:39:42,055] jepsen worker 132 - jepsen.util 132 :invoke :add 7534 INFO [2022-01-10 08:39:42,055] jepsen worker 34 - jepsen.util 34 :info :add 6484 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,058] jepsen worker 382 - jepsen.util 1382 :invoke :add 7535 INFO [2022-01-10 08:39:42,062] jepsen worker 247 - jepsen.util 247 :invoke :add 7536 INFO [2022-01-10 08:39:42,066] jepsen worker 606 - jepsen.util 606 :invoke :add 7537 INFO [2022-01-10 08:39:42,069] jepsen worker 34 - jepsen.util 1034 :invoke :add 7538 INFO [2022-01-10 08:39:42,070] jepsen worker 303 - jepsen.util 303 :ok :add 6599 INFO [2022-01-10 08:39:42,070] jepsen worker 213 - jepsen.util 213 :ok :add 6845 INFO [2022-01-10 08:39:42,070] jepsen worker 367 - jepsen.util 367 :ok :add 6581 INFO [2022-01-10 08:39:42,070] jepsen worker 707 - jepsen.util 707 :ok :add 6778 INFO [2022-01-10 08:39:42,070] jepsen worker 579 - jepsen.util 579 :ok :add 6411 INFO [2022-01-10 08:39:42,070] jepsen worker 203 - jepsen.util 203 :ok :add 6425 INFO [2022-01-10 08:39:42,072] jepsen worker 371 - jepsen.util 371 :invoke :add 7539 INFO [2022-01-10 08:39:42,078] jepsen worker 707 - jepsen.util 707 :invoke :add 7540 INFO [2022-01-10 08:39:42,080] jepsen worker 564 - jepsen.util 564 :invoke :add 7541 INFO [2022-01-10 08:39:42,083] jepsen worker 579 - jepsen.util 579 :invoke :add 7542 INFO [2022-01-10 08:39:42,085] jepsen worker 303 - jepsen.util 303 :invoke :add 7543 INFO [2022-01-10 08:39:42,088] jepsen worker 102 - jepsen.util 102 :invoke :add 7544 INFO [2022-01-10 08:39:42,091] jepsen worker 203 - jepsen.util 203 :invoke :add 7545 INFO [2022-01-10 08:39:42,093] jepsen worker 213 - jepsen.util 213 :invoke :add 7546 INFO [2022-01-10 08:39:42,096] jepsen worker 367 - jepsen.util 367 :invoke :add 7547 INFO [2022-01-10 08:39:42,098] jepsen worker 62 - jepsen.util 62 :invoke :add 7548 INFO [2022-01-10 08:39:42,140] jepsen worker 894 - jepsen.util 894 :info :add 6514 :SyncWriteAmbiguous WARN [2022-01-10 08:39:42,141] jepsen worker nemesis - jepsen.generator.interpreter Process :nemesis crashed com.jcraft.jsch.JSchException: channel is not opened. at com.jcraft.jsch.Channel.sendChannelOpen(Channel.java:765) at com.jcraft.jsch.Channel.connect(Channel.java:151) at com.jcraft.jsch.Channel.connect(Channel.java:145) at clj_ssh.ssh$connect_channel.invokeStatic(ssh.clj:512) at clj_ssh.ssh$connect_channel.invoke(ssh.clj:509) at clj_ssh.ssh$ssh_exec_proc.invokeStatic(ssh.clj:667) at clj_ssh.ssh$ssh_exec_proc.invoke(ssh.clj:646) at clj_ssh.ssh$ssh_exec.invokeStatic(ssh.clj:677) at clj_ssh.ssh$ssh_exec.invoke(ssh.clj:670) at clj_ssh.ssh$ssh.invokeStatic(ssh.clj:723) at clj_ssh.ssh$ssh.invoke(ssh.clj:699) at jepsen.control.clj_ssh.Remote.execute_BANG_(clj_ssh.clj:73) at jepsen.control.scp.Remote.execute_BANG_(scp.clj:91) at jepsen.control.retry.Remote$fn__3118.invoke(retry.clj:53) at jepsen.control.retry.Remote.execute_BANG_(retry.clj:53) at jepsen.control$ssh_STAR_.invokeStatic(control.clj:134) at jepsen.control$ssh_STAR_.invoke(control.clj:126) at jepsen.control$exec_STAR_.invokeStatic(control.clj:141) at jepsen.control$exec_STAR_.doInvoke(control.clj:138) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$apply.invoke(core.clj:662) at jepsen.control$exec.invokeStatic(control.clj:157) at jepsen.control$exec.doInvoke(control.clj:151) at clojure.lang.RestFn.invoke(RestFn.java:436) at couchbase.util$perform_kill_command.invokeStatic(util.clj:75) at couchbase.util$perform_kill_command.invoke(util.clj:53) at couchbase.util$kill_process.invokeStatic(util.clj:80) at couchbase.util$kill_process.invoke(util.clj:77) at couchbase.nemesis$kill_process.invokeStatic(nemesis.clj:267) at couchbase.nemesis$kill_process.invoke(nemesis.clj:261) at couchbase.nemesis$couchbase$reify__3984.invoke_BANG_(nemesis.clj:413) at jepsen.generator.interpreter.NemesisWorker.invoke_BANG_(interpreter.clj:74) at jepsen.generator.interpreter$spawn_worker$fn__9837$fn__9838.invoke(interpreter.clj:137) at jepsen.generator.interpreter$spawn_worker$fn__9837.invoke(interpreter.clj:120) at clojure.core$binding_conveyor_fn$fn__5772.invoke(core.clj:2034) at clojure.lang.AFn.call(AFn.java:18) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) INFO [2022-01-10 08:39:42,143] jepsen worker 894 - jepsen.util 1894 :invoke :add 7549 INFO [2022-01-10 08:39:42,151] jepsen worker 114 - jepsen.util 114 :info :add 6520 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,154] jepsen worker 114 - jepsen.util 1114 :invoke :add 7550 INFO [2022-01-10 08:39:42,165] jepsen worker 321 - jepsen.util 321 :ok :add 6513 INFO [2022-01-10 08:39:42,167] jepsen worker 321 - jepsen.util 321 :invoke :add 7551 INFO [2022-01-10 08:39:42,330] jepsen worker 206 - jepsen.util 206 :ok :add 6406 INFO [2022-01-10 08:39:42,330] jepsen worker 432 - jepsen.util 1432 :ok :add 7227 INFO [2022-01-10 08:39:42,330] jepsen worker 562 - jepsen.util 562 :ok :add 6777 INFO [2022-01-10 08:39:42,333] jepsen worker 432 - jepsen.util 1432 :invoke :add 7552 INFO [2022-01-10 08:39:42,335] jepsen worker 206 - jepsen.util 206 :invoke :add 7553 INFO [2022-01-10 08:39:42,337] jepsen worker 562 - jepsen.util 562 :invoke :add 7554 INFO [2022-01-10 08:39:42,339] jepsen worker 596 - jepsen.util 596 :ok :add 6374 INFO [2022-01-10 08:39:42,340] jepsen worker 248 - jepsen.util 248 :ok :add 6432 INFO [2022-01-10 08:39:42,341] jepsen worker 596 - jepsen.util 596 :invoke :add 7555 INFO [2022-01-10 08:39:42,342] jepsen worker 849 - jepsen.util 849 :info :add 6371 :RequestTimeoutException INFO [2022-01-10 08:39:42,344] jepsen worker 248 - jepsen.util 248 :invoke :add 7556 INFO [2022-01-10 08:39:42,346] jepsen worker 849 - jepsen.util 1849 :invoke :add 7557 INFO [2022-01-10 08:39:42,347] jepsen worker 781 - jepsen.util 781 :info :add 6377 :RequestTimeoutException INFO [2022-01-10 08:39:42,350] jepsen worker 781 - jepsen.util 1781 :invoke :add 7558 INFO [2022-01-10 08:39:42,352] jepsen worker 364 - jepsen.util 364 :info :add 6538 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,352] jepsen worker 307 - jepsen.util 307 :info :add 6375 :RequestTimeoutException INFO [2022-01-10 08:39:42,356] jepsen worker 364 - jepsen.util 1364 :invoke :add 7559 INFO [2022-01-10 08:39:42,358] jepsen worker 307 - jepsen.util 1307 :invoke :add 7560 INFO [2022-01-10 08:39:42,361] jepsen worker 198 - jepsen.util 198 :info :add 6376 :RequestTimeoutException INFO [2022-01-10 08:39:42,361] jepsen worker 816 - jepsen.util 816 :info :add 6378 :RequestTimeoutException INFO [2022-01-10 08:39:42,362] jepsen worker 334 - jepsen.util 334 :info :add 6380 :RequestTimeoutException INFO [2022-01-10 08:39:42,366] jepsen worker 198 - jepsen.util 1198 :invoke :add 7561 INFO [2022-01-10 08:39:42,367] jepsen worker 334 - jepsen.util 1334 :invoke :add 7562 INFO [2022-01-10 08:39:42,369] jepsen worker 816 - jepsen.util 1816 :invoke :add 7563 INFO [2022-01-10 08:39:42,456] jepsen worker 197 - jepsen.util 197 :info :add 6550 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,456] jepsen worker 501 - jepsen.util 501 :info :add 6548 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,456] jepsen worker 230 - jepsen.util 230 :info :add 6549 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,457] jepsen worker 811 - jepsen.util 811 :info :add 6551 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,461] jepsen worker 197 - jepsen.util 1197 :invoke :add 7564 INFO [2022-01-10 08:39:42,463] jepsen worker 501 - jepsen.util 1501 :invoke :add 7565 INFO [2022-01-10 08:39:42,466] jepsen worker 230 - jepsen.util 1230 :invoke :add 7566 INFO [2022-01-10 08:39:42,468] jepsen worker 811 - jepsen.util 1811 :invoke :add 7567 INFO [2022-01-10 08:39:42,470] jepsen worker 461 - jepsen.util 461 :info :add 6393 :RequestTimeoutException INFO [2022-01-10 08:39:42,471] jepsen worker 532 - jepsen.util 532 :info :add 6394 :RequestTimeoutException INFO [2022-01-10 08:39:42,471] jepsen worker 466 - jepsen.util 466 :info :add 6396 :RequestTimeoutException INFO [2022-01-10 08:39:42,475] jepsen worker 532 - jepsen.util 1532 :invoke :add 7568 INFO [2022-01-10 08:39:42,477] jepsen worker 461 - jepsen.util 1461 :invoke :add 7569 INFO [2022-01-10 08:39:42,478] jepsen worker 466 - jepsen.util 1466 :invoke :add 7570 INFO [2022-01-10 08:39:42,480] jepsen worker 214 - jepsen.util 214 :info :add 6399 :RequestTimeoutException INFO [2022-01-10 08:39:42,483] jepsen worker 214 - jepsen.util 1214 :invoke :add 7571 INFO [2022-01-10 08:39:42,491] jepsen worker 664 - jepsen.util 664 :info :add 6403 :RequestTimeoutException INFO [2022-01-10 08:39:42,491] jepsen worker 3 - jepsen.util 3 :info :add 6405 :RequestTimeoutException INFO [2022-01-10 08:39:42,493] jepsen worker 49 - jepsen.util 49 :info :add 6404 :RequestTimeoutException INFO [2022-01-10 08:39:42,493] jepsen worker 286 - jepsen.util 286 :info :add 6407 :RequestTimeoutException INFO [2022-01-10 08:39:42,494] jepsen worker 3 - jepsen.util 1003 :invoke :add 7572 INFO [2022-01-10 08:39:42,498] jepsen worker 664 - jepsen.util 1664 :invoke :add 7573 INFO [2022-01-10 08:39:42,499] jepsen worker 842 - jepsen.util 842 :ok :add 6456 INFO [2022-01-10 08:39:42,499] jepsen worker 841 - jepsen.util 841 :ok :add 6462 INFO [2022-01-10 08:39:42,500] jepsen worker 49 - jepsen.util 1049 :invoke :add 7574 INFO [2022-01-10 08:39:42,501] jepsen worker 687 - jepsen.util 687 :ok :add 6426 INFO [2022-01-10 08:39:42,501] jepsen worker 681 - jepsen.util 681 :ok :add 6757 INFO [2022-01-10 08:39:42,501] jepsen worker 870 - jepsen.util 870 :ok :add 6582 INFO [2022-01-10 08:39:42,503] jepsen worker 681 - jepsen.util 681 :invoke :add 7575 INFO [2022-01-10 08:39:42,504] jepsen worker 687 - jepsen.util 687 :invoke :add 7576 INFO [2022-01-10 08:39:42,506] jepsen worker 286 - jepsen.util 1286 :invoke :add 7577 INFO [2022-01-10 08:39:42,508] jepsen worker 842 - jepsen.util 842 :invoke :add 7578 INFO [2022-01-10 08:39:42,510] jepsen worker 841 - jepsen.util 841 :invoke :add 7579 INFO [2022-01-10 08:39:42,513] jepsen worker 870 - jepsen.util 870 :invoke :add 7580 INFO [2022-01-10 08:39:42,583] jepsen worker 136 - jepsen.util 136 :info :add 6553 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,586] jepsen worker 136 - jepsen.util 1136 :invoke :add 7581 INFO [2022-01-10 08:39:42,605] jepsen worker 404 - jepsen.util 404 :info :add 6566 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,605] jepsen worker 265 - jepsen.util 265 :info :add 6564 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,605] jepsen worker 333 - jepsen.util 333 :info :add 6565 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,610] jepsen worker 265 - jepsen.util 1265 :invoke :add 7582 INFO [2022-01-10 08:39:42,612] jepsen worker 404 - jepsen.util 1404 :invoke :add 7583 INFO [2022-01-10 08:39:42,614] jepsen worker 333 - jepsen.util 1333 :invoke :add 7584 INFO [2022-01-10 08:39:42,715] jepsen worker 688 - jepsen.util 688 :info :add 6567 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,718] jepsen worker 688 - jepsen.util 1688 :invoke :add 7585 INFO [2022-01-10 08:39:42,751] jepsen worker 924 - jepsen.util 924 :info :add 6413 :RequestTimeoutException INFO [2022-01-10 08:39:42,754] jepsen worker 924 - jepsen.util 1924 :invoke :add 7586 INFO [2022-01-10 08:39:42,777] jepsen worker 867 - jepsen.util 867 :info :add 6429 :RequestTimeoutException INFO [2022-01-10 08:39:42,780] jepsen worker 867 - jepsen.util 1867 :invoke :add 7587 INFO [2022-01-10 08:39:42,783] jepsen worker 176 - jepsen.util 176 :info :add 6428 :RequestTimeoutException INFO [2022-01-10 08:39:42,783] jepsen worker 361 - jepsen.util 361 :info :add 6430 :RequestTimeoutException INFO [2022-01-10 08:39:42,783] jepsen worker 84 - jepsen.util 84 :info :add 6431 :RequestTimeoutException INFO [2022-01-10 08:39:42,788] jepsen worker 84 - jepsen.util 1084 :invoke :add 7588 INFO [2022-01-10 08:39:42,790] jepsen worker 176 - jepsen.util 1176 :invoke :add 7589 INFO [2022-01-10 08:39:42,791] jepsen worker 402 - jepsen.util 402 :info :add 6433 :RequestTimeoutException INFO [2022-01-10 08:39:42,791] jepsen worker 361 - jepsen.util 1361 :invoke :add 7590 INFO [2022-01-10 08:39:42,793] jepsen worker 402 - jepsen.util 1402 :invoke :add 7591 INFO [2022-01-10 08:39:42,941] jepsen worker 982 - jepsen.util 982 :info :add 6440 :RequestTimeoutException INFO [2022-01-10 08:39:42,941] jepsen worker 366 - jepsen.util 366 :info :add 6441 :RequestTimeoutException INFO [2022-01-10 08:39:42,941] jepsen worker 747 - jepsen.util 747 :info :add 6443 :RequestTimeoutException INFO [2022-01-10 08:39:42,943] jepsen worker 643 - jepsen.util 643 :info :add 6442 :RequestTimeoutException INFO [2022-01-10 08:39:42,945] jepsen worker 982 - jepsen.util 1982 :invoke :add 7592 INFO [2022-01-10 08:39:42,948] jepsen worker 643 - jepsen.util 1643 :invoke :add 7593 INFO [2022-01-10 08:39:42,950] jepsen worker 366 - jepsen.util 1366 :invoke :add 7594 INFO [2022-01-10 08:39:42,952] jepsen worker 747 - jepsen.util 1747 :invoke :add 7595 INFO [2022-01-10 08:39:42,981] jepsen worker 725 - jepsen.util 725 :info :add 6587 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,981] jepsen worker 4 - jepsen.util 4 :info :add 6584 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,981] jepsen worker 484 - jepsen.util 484 :info :add 6585 :SyncWriteAmbiguous INFO [2022-01-10 08:39:42,986] jepsen worker 725 - jepsen.util 1725 :invoke :add 7596 INFO [2022-01-10 08:39:42,988] jepsen worker 4 - jepsen.util 1004 :invoke :add 7597 INFO [2022-01-10 08:39:42,990] jepsen worker 484 - jepsen.util 1484 :invoke :add 7598 INFO [2022-01-10 08:39:42,993] jepsen worker 863 - jepsen.util 863 :info :add 6459 :RequestTimeoutException INFO [2022-01-10 08:39:42,993] jepsen worker 124 - jepsen.util 124 :info :add 6458 :RequestTimeoutException INFO [2022-01-10 08:39:42,997] jepsen worker 863 - jepsen.util 1863 :invoke :add 7599 INFO [2022-01-10 08:39:42,999] jepsen worker 124 - jepsen.util 1124 :invoke :add 7600 INFO [2022-01-10 08:39:43,004] jepsen worker 57 - jepsen.util 57 :info :add 6461 :RequestTimeoutException INFO [2022-01-10 08:39:43,004] jepsen worker 242 - jepsen.util 242 :info :add 6463 :RequestTimeoutException INFO [2022-01-10 08:39:43,007] jepsen worker 242 - jepsen.util 1242 :invoke :add 7601 INFO [2022-01-10 08:39:43,010] jepsen worker 57 - jepsen.util 1057 :invoke :add 7602 INFO [2022-01-10 08:39:43,031] jepsen worker 943 - jepsen.util 943 :info :add 6474 :RequestTimeoutException INFO [2022-01-10 08:39:43,035] jepsen worker 943 - jepsen.util 1943 :invoke :add 7603 INFO [2022-01-10 08:39:43,041] jepsen worker 498 - jepsen.util 498 :info :add 6476 :RequestTimeoutException INFO [2022-01-10 08:39:43,042] jepsen worker 347 - jepsen.util 347 :info :add 6475 :RequestTimeoutException INFO [2022-01-10 08:39:43,045] jepsen worker 347 - jepsen.util 1347 :invoke :add 7604 INFO [2022-01-10 08:39:43,047] jepsen worker 498 - jepsen.util 1498 :invoke :add 7605 INFO [2022-01-10 08:39:43,047] jepsen worker 846 - jepsen.util 846 :info :add 6477 :RequestTimeoutException INFO [2022-01-10 08:39:43,051] jepsen worker 846 - jepsen.util 1846 :invoke :add 7606 INFO [2022-01-10 08:39:43,081] jepsen worker 614 - jepsen.util 614 :info :add 6494 :RequestTimeoutException INFO [2022-01-10 08:39:43,084] jepsen worker 614 - jepsen.util 1614 :invoke :add 7607 INFO [2022-01-10 08:39:43,087] jepsen worker 301 - jepsen.util 301 :info :add 6495 :RequestTimeoutException INFO [2022-01-10 08:39:43,090] jepsen worker 514 - jepsen.util 514 :info :add 6496 :RequestTimeoutException INFO [2022-01-10 08:39:43,090] jepsen worker 196 - jepsen.util 196 :info :add 6497 :RequestTimeoutException WARN [2022-01-10 08:39:43,090] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":10152873,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000001/00000000CB02B145","last_local_socket":"172.23.104.79:41914","total_dispatch_duration_us":10152873,"total_server_duration_us":2329771,"encode_duration_us":268,"operation_id":"0x1c97","timeout_ms":10000,"last_server_duration_us":2329771,"total_duration_us":10013686},{"operation_name":"insert","last_dispatch_duration_us":12199104,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":12199104,"total_server_duration_us":9011909,"encode_duration_us":443,"operation_id":"0x178b","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10012775},{"operation_name":"insert","last_dispatch_duration_us":12658933,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":12658933,"total_server_duration_us":9008729,"encode_duration_us":265,"operation_id":"0x16c2","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10011205},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","encode_duration_us":269,"operation_id":"0x1b32","timeout_ms":10000,"total_duration_us":10011111},{"operation_name":"insert","last_dispatch_duration_us":10356929,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":10356929,"total_server_duration_us":4386919,"encode_duration_us":268,"operation_id":"0x170b","timeout_ms":10000,"last_server_duration_us":4386919,"total_duration_us":10011042},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","encode_duration_us":316,"operation_id":"0x1c1d","timeout_ms":10000,"total_duration_us":10010830},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","encode_duration_us":258,"operation_id":"0x1983","timeout_ms":10000,"total_duration_us":10010429},{"operation_name":"insert","last_dispatch_duration_us":6874317,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":6874317,"total_server_duration_us":4259713,"encode_duration_us":353,"operation_id":"0x1a26","timeout_ms":10000,"last_server_duration_us":4259713,"total_duration_us":10010141},{"operation_name":"insert","last_dispatch_duration_us":11346308,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":11346308,"total_server_duration_us":8408071,"encode_duration_us":336,"operation_id":"0x17f3","timeout_ms":10000,"last_server_duration_us":8408071,"total_duration_us":10010091},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","encode_duration_us":494,"operation_id":"0x191a","timeout_ms":10000,"total_duration_us":10009754}],"total_count":194}} INFO [2022-01-10 08:39:43,091] jepsen worker 301 - jepsen.util 1301 :invoke :add 7608 INFO [2022-01-10 08:39:43,093] jepsen worker 514 - jepsen.util 1514 :invoke :add 7609 INFO [2022-01-10 08:39:43,095] jepsen worker 196 - jepsen.util 1196 :invoke :add 7610 INFO [2022-01-10 08:39:43,107] jepsen worker 174 - jepsen.util 174 :info :add 6504 :RequestTimeoutException INFO [2022-01-10 08:39:43,109] jepsen worker 174 - jepsen.util 1174 :invoke :add 7611 INFO [2022-01-10 08:39:43,113] jepsen worker 254 - jepsen.util 254 :info :add 6505 :RequestTimeoutException INFO [2022-01-10 08:39:43,116] jepsen worker 254 - jepsen.util 1254 :invoke :add 7612 INFO [2022-01-10 08:39:43,117] jepsen worker 814 - jepsen.util 814 :info :add 6506 :RequestTimeoutException INFO [2022-01-10 08:39:43,120] jepsen worker 814 - jepsen.util 1814 :invoke :add 7613 INFO [2022-01-10 08:39:43,123] jepsen worker 618 - jepsen.util 618 :info :add 6509 :RequestTimeoutException INFO [2022-01-10 08:39:43,124] jepsen worker 336 - jepsen.util 336 :info :add 6507 :RequestTimeoutException INFO [2022-01-10 08:39:43,127] jepsen worker 618 - jepsen.util 1618 :invoke :add 7614 INFO [2022-01-10 08:39:43,129] jepsen worker 336 - jepsen.util 1336 :invoke :add 7615 INFO [2022-01-10 08:39:43,150] jepsen worker 472 - jepsen.util 472 :info :add 6525 :RequestTimeoutException INFO [2022-01-10 08:39:43,154] jepsen worker 472 - jepsen.util 1472 :invoke :add 7616 INFO [2022-01-10 08:39:43,160] jepsen worker 695 - jepsen.util 695 :info :add 6530 :RequestTimeoutException INFO [2022-01-10 08:39:43,161] jepsen worker 391 - jepsen.util 391 :info :add 6529 :RequestTimeoutException INFO [2022-01-10 08:39:43,162] jepsen worker 519 - jepsen.util 519 :info :add 6528 :RequestTimeoutException INFO [2022-01-10 08:39:43,163] jepsen worker 780 - jepsen.util 780 :info :add 6532 :RequestTimeoutException INFO [2022-01-10 08:39:43,164] jepsen worker 391 - jepsen.util 1391 :invoke :add 7617 INFO [2022-01-10 08:39:43,167] jepsen worker 971 - jepsen.util 971 :info :add 6531 :RequestTimeoutException INFO [2022-01-10 08:39:43,167] jepsen worker 695 - jepsen.util 1695 :invoke :add 7618 INFO [2022-01-10 08:39:43,168] jepsen worker 519 - jepsen.util 1519 :invoke :add 7619 INFO [2022-01-10 08:39:43,171] jepsen worker 971 - jepsen.util 1971 :invoke :add 7620 INFO [2022-01-10 08:39:43,172] jepsen worker 795 - jepsen.util 795 :info :add 6533 :RequestTimeoutException INFO [2022-01-10 08:39:43,173] jepsen worker 780 - jepsen.util 1780 :invoke :add 7621 INFO [2022-01-10 08:39:43,175] jepsen worker 795 - jepsen.util 1795 :invoke :add 7622 WARN [2022-01-10 08:39:43,191] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":6377838,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":6377838,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":3761797,"encode_duration_us":258,"operation_id":"0x1a27","timeout_ms":10000,"last_server_duration_us":3761797,"total_duration_us":10008939},{"operation_name":"insert","last_dispatch_duration_us":11989990,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11989990,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":8622277,"encode_duration_us":266,"operation_id":"0x1738","timeout_ms":10000,"last_server_duration_us":8622277,"total_duration_us":10009068},{"operation_name":"insert","last_dispatch_duration_us":12909183,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12909183,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":9014029,"encode_duration_us":323,"operation_id":"0x1666","timeout_ms":10000,"last_server_duration_us":9014029,"total_duration_us":10009253},{"operation_name":"insert","last_dispatch_duration_us":10356929,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10356929,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":4386919,"encode_duration_us":268,"operation_id":"0x170b","timeout_ms":10000,"last_server_duration_us":4386919,"total_duration_us":10011042},{"operation_name":"insert","last_dispatch_duration_us":6874317,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":6874317,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":4259713,"encode_duration_us":353,"operation_id":"0x1a26","timeout_ms":10000,"last_server_duration_us":4259713,"total_duration_us":10010141},{"operation_name":"insert","last_dispatch_duration_us":12288438,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12288438,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":9016150,"encode_duration_us":346,"operation_id":"0x1768","timeout_ms":10000,"last_server_duration_us":9016150,"total_duration_us":10009571},{"operation_name":"insert","last_dispatch_duration_us":11346308,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11346308,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":8408071,"encode_duration_us":336,"operation_id":"0x17f3","timeout_ms":10000,"last_server_duration_us":8408071,"total_duration_us":10010091},{"operation_name":"insert","last_dispatch_duration_us":10152873,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10152873,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":2329771,"encode_duration_us":268,"operation_id":"0x1c97","timeout_ms":10000,"last_server_duration_us":2329771,"total_duration_us":10013686},{"operation_name":"insert","last_dispatch_duration_us":12199104,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12199104,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":9011909,"encode_duration_us":443,"operation_id":"0x178b","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10012775},{"operation_name":"insert","last_dispatch_duration_us":12658933,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12658933,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":9008729,"encode_duration_us":265,"operation_id":"0x16c2","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10011205}],"total_count":39}} INFO [2022-01-10 08:39:43,300] jepsen worker 440 - jepsen.util 440 :info :add 6616 :SyncWriteAmbiguous INFO [2022-01-10 08:39:43,304] jepsen worker 440 - jepsen.util 1440 :invoke :add 7623 INFO [2022-01-10 08:39:43,311] jepsen worker 889 - jepsen.util 889 :info :add 6623 :SyncWriteAmbiguous INFO [2022-01-10 08:39:43,314] jepsen worker 889 - jepsen.util 1889 :invoke :add 7624 INFO [2022-01-10 08:39:43,343] jepsen worker 168 - jepsen.util 168 :info :add 6535 :RequestTimeoutException INFO [2022-01-10 08:39:43,346] jepsen worker 168 - jepsen.util 1168 :invoke :add 7625 INFO [2022-01-10 08:39:43,347] jepsen worker 218 - jepsen.util 218 :info :add 6540 :RequestTimeoutException INFO [2022-01-10 08:39:43,350] jepsen worker 218 - jepsen.util 1218 :invoke :add 7626 INFO [2022-01-10 08:39:43,351] jepsen worker 613 - jepsen.util 613 :info :add 6543 :RequestTimeoutException INFO [2022-01-10 08:39:43,353] jepsen worker 354 - jepsen.util 354 :info :add 6542 :RequestTimeoutException INFO [2022-01-10 08:39:43,354] jepsen worker 469 - jepsen.util 469 :info :add 6541 :RequestTimeoutException INFO [2022-01-10 08:39:43,354] jepsen worker 613 - jepsen.util 1613 :invoke :add 7627 INFO [2022-01-10 08:39:43,358] jepsen worker 469 - jepsen.util 1469 :invoke :add 7628 INFO [2022-01-10 08:39:43,359] jepsen worker 354 - jepsen.util 1354 :invoke :add 7629 INFO [2022-01-10 08:39:43,381] jepsen worker 92 - jepsen.util 92 :ok :add 6665 INFO [2022-01-10 08:39:43,381] jepsen worker 18 - jepsen.util 18 :ok :add 6649 INFO [2022-01-10 08:39:43,381] jepsen worker 199 - jepsen.util 199 :ok :add 6750 INFO [2022-01-10 08:39:43,381] jepsen worker 119 - jepsen.util 119 :ok :add 6841 INFO [2022-01-10 08:39:43,384] jepsen worker 119 - jepsen.util 119 :invoke :add 7630 INFO [2022-01-10 08:39:43,385] jepsen worker 92 - jepsen.util 92 :invoke :add 7631 INFO [2022-01-10 08:39:43,387] jepsen worker 199 - jepsen.util 199 :invoke :add 7632 INFO [2022-01-10 08:39:43,389] jepsen worker 18 - jepsen.util 18 :invoke :add 7633 INFO [2022-01-10 08:39:43,453] jepsen worker 654 - jepsen.util 654 :info :add 6547 :RequestTimeoutException INFO [2022-01-10 08:39:43,455] jepsen worker 654 - jepsen.util 1654 :invoke :add 7634 INFO [2022-01-10 08:39:43,458] jepsen worker 275 - jepsen.util 275 :ok :add 6722 INFO [2022-01-10 08:39:43,461] jepsen worker 275 - jepsen.util 275 :invoke :add 7635 INFO [2022-01-10 08:39:43,470] jepsen worker 872 - jepsen.util 872 :ok :add 6684 INFO [2022-01-10 08:39:43,470] jepsen worker 630 - jepsen.util 630 :ok :add 6714 INFO [2022-01-10 08:39:43,472] jepsen worker 630 - jepsen.util 630 :invoke :add 7636 INFO [2022-01-10 08:39:43,473] jepsen worker 872 - jepsen.util 872 :invoke :add 7637 INFO [2022-01-10 08:39:43,480] jepsen worker 67 - jepsen.util 67 :ok :add 6576 INFO [2022-01-10 08:39:43,483] jepsen worker 67 - jepsen.util 67 :invoke :add 7638 INFO [2022-01-10 08:39:43,516] jepsen worker 640 - jepsen.util 640 :ok :add 6598 INFO [2022-01-10 08:39:43,516] jepsen worker 293 - jepsen.util 293 :ok :add 6870 INFO [2022-01-10 08:39:43,519] jepsen worker 293 - jepsen.util 293 :invoke :add 7639 INFO [2022-01-10 08:39:43,521] jepsen worker 640 - jepsen.util 640 :invoke :add 7640 INFO [2022-01-10 08:39:43,565] jepsen worker 722 - jepsen.util 722 :ok :add 6586 INFO [2022-01-10 08:39:43,567] jepsen worker 722 - jepsen.util 722 :invoke :add 7641 INFO [2022-01-10 08:39:43,580] jepsen worker 939 - jepsen.util 939 :info :add 6552 :RequestTimeoutException INFO [2022-01-10 08:39:43,582] jepsen worker 667 - jepsen.util 667 :info :add 6556 :RequestTimeoutException INFO [2022-01-10 08:39:43,582] jepsen worker 859 - jepsen.util 859 :info :add 6554 :RequestTimeoutException INFO [2022-01-10 08:39:43,583] jepsen worker 145 - jepsen.util 145 :info :add 6555 :RequestTimeoutException INFO [2022-01-10 08:39:43,586] jepsen worker 939 - jepsen.util 1939 :invoke :add 7642 INFO [2022-01-10 08:39:43,588] jepsen worker 667 - jepsen.util 1667 :invoke :add 7643 INFO [2022-01-10 08:39:43,590] jepsen worker 145 - jepsen.util 1145 :invoke :add 7644 INFO [2022-01-10 08:39:43,591] jepsen worker 822 - jepsen.util 822 :info :add 6560 :RequestTimeoutException INFO [2022-01-10 08:39:43,591] jepsen worker 408 - jepsen.util 408 :info :add 6557 :RequestTimeoutException INFO [2022-01-10 08:39:43,591] jepsen worker 355 - jepsen.util 355 :info :add 6558 :RequestTimeoutException INFO [2022-01-10 08:39:43,591] jepsen worker 859 - jepsen.util 1859 :invoke :add 7645 INFO [2022-01-10 08:39:43,592] jepsen worker 298 - jepsen.util 298 :info :add 6561 :RequestTimeoutException INFO [2022-01-10 08:39:43,593] jepsen worker 879 - jepsen.util 879 :info :add 6559 :RequestTimeoutException INFO [2022-01-10 08:39:43,597] jepsen worker 298 - jepsen.util 1298 :invoke :add 7646 INFO [2022-01-10 08:39:43,598] jepsen worker 355 - jepsen.util 1355 :invoke :add 7647 INFO [2022-01-10 08:39:43,600] jepsen worker 14 - jepsen.util 14 :info :add 6563 :RequestTimeoutException INFO [2022-01-10 08:39:43,600] jepsen worker 822 - jepsen.util 1822 :invoke :add 7648 INFO [2022-01-10 08:39:43,602] jepsen worker 729 - jepsen.util 729 :info :add 6562 :RequestTimeoutException INFO [2022-01-10 08:39:43,603] jepsen worker 14 - jepsen.util 1014 :invoke :add 7649 INFO [2022-01-10 08:39:43,606] jepsen worker 879 - jepsen.util 1879 :invoke :add 7650 INFO [2022-01-10 08:39:43,609] jepsen worker 408 - jepsen.util 1408 :invoke :add 7651 INFO [2022-01-10 08:39:43,610] jepsen worker 729 - jepsen.util 1729 :invoke :add 7652 INFO [2022-01-10 08:39:43,636] jepsen worker 526 - jepsen.util 526 :ok :add 6617 INFO [2022-01-10 08:39:43,639] jepsen worker 526 - jepsen.util 526 :invoke :add 7653 INFO [2022-01-10 08:39:43,704] jepsen worker 491 - jepsen.util 491 :info :add 6568 :RequestTimeoutException INFO [2022-01-10 08:39:43,707] jepsen worker 491 - jepsen.util 1491 :invoke :add 7654 INFO [2022-01-10 08:39:43,714] jepsen worker 511 - jepsen.util 511 :info :add 6570 :RequestTimeoutException INFO [2022-01-10 08:39:43,716] jepsen worker 511 - jepsen.util 1511 :invoke :add 7655 INFO [2022-01-10 08:39:43,717] jepsen worker 834 - jepsen.util 834 :info :add 6575 :RequestTimeoutException INFO [2022-01-10 08:39:43,721] jepsen worker 834 - jepsen.util 1834 :invoke :add 7656 INFO [2022-01-10 08:39:43,723] jepsen worker 329 - jepsen.util 329 :info :add 6574 :RequestTimeoutException INFO [2022-01-10 08:39:43,726] jepsen worker 329 - jepsen.util 1329 :invoke :add 7657 INFO [2022-01-10 08:39:43,876] jepsen worker 324 - jepsen.util 324 :ok :add 6615 INFO [2022-01-10 08:39:43,876] jepsen worker 936 - jepsen.util 936 :ok :add 6639 INFO [2022-01-10 08:39:43,876] jepsen worker 919 - jepsen.util 919 :ok :add 6621 INFO [2022-01-10 08:39:43,877] jepsen worker 441 - jepsen.util 1441 :ok :add 6928 INFO [2022-01-10 08:39:43,881] jepsen worker 441 - jepsen.util 1441 :invoke :add 7658 INFO [2022-01-10 08:39:43,882] jepsen worker 832 - jepsen.util 1832 :ok :add 7000 INFO [2022-01-10 08:39:43,882] jepsen worker 324 - jepsen.util 324 :invoke :add 7659 INFO [2022-01-10 08:39:43,884] jepsen worker 919 - jepsen.util 919 :invoke :add 7660 INFO [2022-01-10 08:39:43,887] jepsen worker 936 - jepsen.util 936 :invoke :add 7661 INFO [2022-01-10 08:39:43,889] jepsen worker 832 - jepsen.util 1832 :invoke :add 7662 INFO [2022-01-10 08:39:43,897] jepsen worker 259 - jepsen.util 259 :ok :add 6687 INFO [2022-01-10 08:39:43,897] jepsen worker 476 - jepsen.util 476 :ok :add 6622 INFO [2022-01-10 08:39:43,900] jepsen worker 259 - jepsen.util 259 :invoke :add 7663 INFO [2022-01-10 08:39:43,901] jepsen worker 476 - jepsen.util 476 :invoke :add 7664 INFO [2022-01-10 08:39:43,930] jepsen worker 734 - jepsen.util 734 :ok :add 6685 INFO [2022-01-10 08:39:43,934] jepsen worker 734 - jepsen.util 734 :invoke :add 7665 INFO [2022-01-10 08:39:43,934] jepsen worker 963 - jepsen.util 963 :ok :add 6620 INFO [2022-01-10 08:39:43,934] jepsen worker 240 - jepsen.util 240 :ok :add 6638 INFO [2022-01-10 08:39:43,934] jepsen worker 873 - jepsen.util 873 :ok :add 6614 INFO [2022-01-10 08:39:43,935] jepsen worker 918 - jepsen.util 1918 :ok :add 7019 INFO [2022-01-10 08:39:43,939] jepsen worker 873 - jepsen.util 873 :invoke :add 7666 INFO [2022-01-10 08:39:43,942] jepsen worker 918 - jepsen.util 1918 :invoke :add 7667 INFO [2022-01-10 08:39:43,943] jepsen worker 963 - jepsen.util 963 :invoke :add 7668 INFO [2022-01-10 08:39:43,945] jepsen worker 240 - jepsen.util 240 :invoke :add 7669 INFO [2022-01-10 08:39:43,963] jepsen worker 848 - jepsen.util 848 :info :add 6579 :RequestTimeoutException INFO [2022-01-10 08:39:43,963] jepsen worker 724 - jepsen.util 724 :info :add 6577 :RequestTimeoutException INFO [2022-01-10 08:39:43,963] jepsen worker 575 - jepsen.util 575 :info :add 6578 :RequestTimeoutException INFO [2022-01-10 08:39:43,967] jepsen worker 986 - jepsen.util 986 :info :add 6583 :RequestTimeoutException INFO [2022-01-10 08:39:43,967] jepsen worker 848 - jepsen.util 1848 :invoke :add 7670 INFO [2022-01-10 08:39:43,970] jepsen worker 575 - jepsen.util 1575 :invoke :add 7671 INFO [2022-01-10 08:39:43,970] jepsen worker 109 - jepsen.util 109 :info :add 6580 :RequestTimeoutException INFO [2022-01-10 08:39:43,973] jepsen worker 724 - jepsen.util 1724 :invoke :add 7672 INFO [2022-01-10 08:39:43,975] jepsen worker 109 - jepsen.util 1109 :invoke :add 7673 INFO [2022-01-10 08:39:43,977] jepsen worker 986 - jepsen.util 1986 :invoke :add 7674 INFO [2022-01-10 08:39:43,984] jepsen worker 227 - jepsen.util 227 :info :add 6589 :RequestTimeoutException INFO [2022-01-10 08:39:43,987] jepsen worker 227 - jepsen.util 1227 :invoke :add 7675 INFO [2022-01-10 08:39:44,051] jepsen worker 217 - jepsen.util 217 :info :add 6594 :RequestTimeoutException INFO [2022-01-10 08:39:44,053] jepsen worker 925 - jepsen.util 925 :info :add 6595 :RequestTimeoutException INFO [2022-01-10 08:39:44,053] jepsen worker 243 - jepsen.util 243 :info :add 6593 :RequestTimeoutException INFO [2022-01-10 08:39:44,054] jepsen worker 217 - jepsen.util 1217 :invoke :add 7676 INFO [2022-01-10 08:39:44,057] jepsen worker 925 - jepsen.util 1925 :invoke :add 7677 INFO [2022-01-10 08:39:44,059] jepsen worker 243 - jepsen.util 1243 :invoke :add 7678 INFO [2022-01-10 08:39:44,062] jepsen worker 447 - jepsen.util 447 :info :add 6597 :RequestTimeoutException INFO [2022-01-10 08:39:44,063] jepsen worker 539 - jepsen.util 539 :info :add 6596 :RequestTimeoutException INFO [2022-01-10 08:39:44,066] jepsen worker 447 - jepsen.util 1447 :invoke :add 7679 INFO [2022-01-10 08:39:44,068] jepsen worker 539 - jepsen.util 1539 :invoke :add 7680 INFO [2022-01-10 08:39:44,072] jepsen worker 902 - jepsen.util 902 :info :add 6602 :RequestTimeoutException INFO [2022-01-10 08:39:44,074] jepsen worker 103 - jepsen.util 103 :info :add 6605 :RequestTimeoutException INFO [2022-01-10 08:39:44,074] jepsen worker 902 - jepsen.util 1902 :invoke :add 7681 INFO [2022-01-10 08:39:44,077] jepsen worker 631 - jepsen.util 631 :info :add 6604 :RequestTimeoutException INFO [2022-01-10 08:39:44,077] jepsen worker 569 - jepsen.util 569 :info :add 6606 :RequestTimeoutException INFO [2022-01-10 08:39:44,077] jepsen worker 103 - jepsen.util 1103 :invoke :add 7682 INFO [2022-01-10 08:39:44,081] jepsen worker 343 - jepsen.util 343 :info :add 6608 :RequestTimeoutException INFO [2022-01-10 08:39:44,081] jepsen worker 110 - jepsen.util 110 :info :add 6609 :RequestTimeoutException INFO [2022-01-10 08:39:44,081] jepsen worker 880 - jepsen.util 880 :info :add 6610 :RequestTimeoutException INFO [2022-01-10 08:39:44,081] jepsen worker 631 - jepsen.util 1631 :invoke :add 7683 INFO [2022-01-10 08:39:44,083] jepsen worker 170 - jepsen.util 170 :info :add 6607 :RequestTimeoutException INFO [2022-01-10 08:39:44,083] jepsen worker 151 - jepsen.util 151 :info :add 6611 :RequestTimeoutException INFO [2022-01-10 08:39:44,086] jepsen worker 170 - jepsen.util 1170 :invoke :add 7684 INFO [2022-01-10 08:39:44,088] jepsen worker 569 - jepsen.util 1569 :invoke :add 7685 INFO [2022-01-10 08:39:44,090] jepsen worker 47 - jepsen.util 47 :info :add 6612 :RequestTimeoutException INFO [2022-01-10 08:39:44,091] jepsen worker 110 - jepsen.util 1110 :invoke :add 7686 INFO [2022-01-10 08:39:44,094] jepsen worker 151 - jepsen.util 1151 :invoke :add 7687 INFO [2022-01-10 08:39:44,096] jepsen worker 47 - jepsen.util 1047 :invoke :add 7688 INFO [2022-01-10 08:39:44,098] jepsen worker 343 - jepsen.util 1343 :invoke :add 7689 INFO [2022-01-10 08:39:44,100] jepsen worker 880 - jepsen.util 1880 :invoke :add 7690 INFO [2022-01-10 08:39:44,169] jepsen worker 410 - jepsen.util 1410 :ok :add 7292 INFO [2022-01-10 08:39:44,169] jepsen worker 696 - jepsen.util 696 :ok :add 6652 INFO [2022-01-10 08:39:44,169] jepsen worker 159 - jepsen.util 1159 :ok :add 7336 INFO [2022-01-10 08:39:44,169] jepsen worker 101 - jepsen.util 101 :ok :add 6666 INFO [2022-01-10 08:39:44,169] jepsen worker 534 - jepsen.util 534 :ok :add 6943 INFO [2022-01-10 08:39:44,169] jepsen worker 284 - jepsen.util 284 :ok :add 6977 INFO [2022-01-10 08:39:44,169] jepsen worker 607 - jepsen.util 607 :ok :add 7073 INFO [2022-01-10 08:39:44,169] jepsen worker 973 - jepsen.util 1973 :ok :add 7302 INFO [2022-01-10 08:39:44,173] jepsen worker 607 - jepsen.util 607 :invoke :add 7691 INFO [2022-01-10 08:39:44,175] jepsen worker 534 - jepsen.util 534 :invoke :add 7692 INFO [2022-01-10 08:39:44,177] jepsen worker 410 - jepsen.util 1410 :invoke :add 7693 INFO [2022-01-10 08:39:44,179] jepsen worker 696 - jepsen.util 696 :invoke :add 7694 INFO [2022-01-10 08:39:44,181] jepsen worker 973 - jepsen.util 1973 :invoke :add 7695 INFO [2022-01-10 08:39:44,182] jepsen worker 284 - jepsen.util 284 :invoke :add 7696 INFO [2022-01-10 08:39:44,184] jepsen worker 101 - jepsen.util 101 :invoke :add 7697 INFO [2022-01-10 08:39:44,186] jepsen worker 159 - jepsen.util 1159 :invoke :add 7698 INFO [2022-01-10 08:39:44,278] jepsen worker 672 - jepsen.util 672 :ok :add 6650 INFO [2022-01-10 08:39:44,278] jepsen worker 479 - jepsen.util 479 :ok :add 6648 INFO [2022-01-10 08:39:44,278] jepsen worker 140 - jepsen.util 140 :ok :add 6664 INFO [2022-01-10 08:39:44,279] jepsen worker 68 - jepsen.util 68 :ok :add 7069 INFO [2022-01-10 08:39:44,280] jepsen worker 302 - jepsen.util 1302 :ok :add 7045 INFO [2022-01-10 08:39:44,281] jepsen worker 140 - jepsen.util 140 :invoke :add 7699 INFO [2022-01-10 08:39:44,282] jepsen worker 69 - jepsen.util 1069 :ok :add 7290 INFO [2022-01-10 08:39:44,282] jepsen worker 570 - jepsen.util 1570 :ok :add 7334 INFO [2022-01-10 08:39:44,282] jepsen worker 409 - jepsen.util 409 :info :add 6613 :RequestTimeoutException INFO [2022-01-10 08:39:44,285] jepsen worker 302 - jepsen.util 1302 :invoke :add 7700 INFO [2022-01-10 08:39:44,286] jepsen worker 570 - jepsen.util 1570 :invoke :add 7701 INFO [2022-01-10 08:39:44,288] jepsen worker 69 - jepsen.util 1069 :invoke :add 7702 INFO [2022-01-10 08:39:44,290] jepsen worker 409 - jepsen.util 1409 :invoke :add 7703 INFO [2022-01-10 08:39:44,293] jepsen worker 479 - jepsen.util 479 :invoke :add 7704 INFO [2022-01-10 08:39:44,295] jepsen worker 68 - jepsen.util 68 :invoke :add 7705 INFO [2022-01-10 08:39:44,297] jepsen worker 672 - jepsen.util 672 :invoke :add 7706 INFO [2022-01-10 08:39:44,311] jepsen worker 464 - jepsen.util 464 :info :add 6625 :RequestTimeoutException INFO [2022-01-10 08:39:44,313] jepsen worker 209 - jepsen.util 209 :info :add 6624 :RequestTimeoutException INFO [2022-01-10 08:39:44,313] jepsen worker 932 - jepsen.util 932 :info :add 6626 :RequestTimeoutException INFO [2022-01-10 08:39:44,313] jepsen worker 426 - jepsen.util 426 :info :add 6627 :RequestTimeoutException INFO [2022-01-10 08:39:44,320] jepsen worker 464 - jepsen.util 1464 :invoke :add 7707 INFO [2022-01-10 08:39:44,321] jepsen worker 586 - jepsen.util 586 :info :add 6628 :RequestTimeoutException INFO [2022-01-10 08:39:44,322] jepsen worker 426 - jepsen.util 1426 :invoke :add 7708 INFO [2022-01-10 08:39:44,325] jepsen worker 209 - jepsen.util 1209 :invoke :add 7709 INFO [2022-01-10 08:39:44,327] jepsen worker 586 - jepsen.util 1586 :invoke :add 7710 INFO [2022-01-10 08:39:44,328] jepsen worker 932 - jepsen.util 1932 :invoke :add 7711 INFO [2022-01-10 08:39:44,343] jepsen worker 662 - jepsen.util 662 :ok :add 6843 INFO [2022-01-10 08:39:44,343] jepsen worker 929 - jepsen.util 929 :ok :add 7236 INFO [2022-01-10 08:39:44,343] jepsen worker 708 - jepsen.util 1708 :ok :add 7173 INFO [2022-01-10 08:39:44,343] jepsen worker 148 - jepsen.util 148 :ok :add 6877 INFO [2022-01-10 08:39:44,346] jepsen worker 708 - jepsen.util 1708 :invoke :add 7712 INFO [2022-01-10 08:39:44,348] jepsen worker 929 - jepsen.util 929 :invoke :add 7713 INFO [2022-01-10 08:39:44,350] jepsen worker 662 - jepsen.util 662 :invoke :add 7714 INFO [2022-01-10 08:39:44,352] jepsen worker 148 - jepsen.util 148 :invoke :add 7715 INFO [2022-01-10 08:39:44,366] jepsen worker 142 - jepsen.util 142 :ok :add 6748 INFO [2022-01-10 08:39:44,366] jepsen worker 965 - jepsen.util 965 :ok :add 7171 INFO [2022-01-10 08:39:44,366] jepsen worker 395 - jepsen.util 1395 :ok :add 7200 INFO [2022-01-10 08:39:44,366] jepsen worker 210 - jepsen.util 1210 :ok :add 6875 INFO [2022-01-10 08:39:44,366] jepsen worker 866 - jepsen.util 1866 :ok :add 7218 INFO [2022-01-10 08:39:44,367] jepsen worker 348 - jepsen.util 1348 :ok :add 7234 INFO [2022-01-10 08:39:44,367] jepsen worker 513 - jepsen.util 513 :ok :add 6764 INFO [2022-01-10 08:39:44,367] jepsen worker 946 - jepsen.util 946 :ok :add 6859 INFO [2022-01-10 08:39:44,370] jepsen worker 946 - jepsen.util 946 :invoke :add 7716 INFO [2022-01-10 08:39:44,372] jepsen worker 594 - jepsen.util 594 :ok :add 6653 INFO [2022-01-10 08:39:44,372] jepsen worker 142 - jepsen.util 142 :invoke :add 7717 INFO [2022-01-10 08:39:44,373] jepsen worker 339 - jepsen.util 339 :ok :add 6976 INFO [2022-01-10 08:39:44,374] jepsen worker 195 - jepsen.util 1195 :ok :add 7046 INFO [2022-01-10 08:39:44,374] jepsen worker 880 - jepsen.util 1880 :ok :add 7690 INFO [2022-01-10 08:39:44,374] jepsen worker 398 - jepsen.util 1398 :ok :add 7072 INFO [2022-01-10 08:39:44,374] jepsen worker 965 - jepsen.util 965 :invoke :add 7718 INFO [2022-01-10 08:39:44,377] jepsen worker 210 - jepsen.util 1210 :invoke :add 7719 INFO [2022-01-10 08:39:44,378] jepsen worker 188 - jepsen.util 188 :info :add 6629 :RequestTimeoutException INFO [2022-01-10 08:39:44,379] jepsen worker 395 - jepsen.util 1395 :invoke :add 7720 INFO [2022-01-10 08:39:44,381] jepsen worker 195 - jepsen.util 1195 :invoke :add 7721 INFO [2022-01-10 08:39:44,382] jepsen worker 25 - jepsen.util 25 :info :add 6630 :RequestTimeoutException INFO [2022-01-10 08:39:44,383] jepsen worker 444 - jepsen.util 444 :info :add 6632 :RequestTimeoutException INFO [2022-01-10 08:39:44,383] jepsen worker 398 - jepsen.util 1398 :invoke :add 7722 INFO [2022-01-10 08:39:44,383] jepsen worker 748 - jepsen.util 748 :info :add 6631 :RequestTimeoutException INFO [2022-01-10 08:39:44,384] jepsen worker 490 - jepsen.util 1490 :ok :add 6896 INFO [2022-01-10 08:39:44,386] jepsen worker 622 - jepsen.util 622 :ok :add 6787 INFO [2022-01-10 08:39:44,386] jepsen worker 78 - jepsen.util 78 :ok :add 7002 INFO [2022-01-10 08:39:44,387] jepsen worker 191 - jepsen.util 1191 :ok :add 7373 INFO [2022-01-10 08:39:44,388] jepsen worker 188 - jepsen.util 1188 :invoke :add 7723 INFO [2022-01-10 08:39:44,390] jepsen worker 339 - jepsen.util 339 :invoke :add 7724 INFO [2022-01-10 08:39:44,391] jepsen worker 920 - jepsen.util 920 :info :add 6633 :RequestTimeoutException INFO [2022-01-10 08:39:44,392] jepsen worker 748 - jepsen.util 1748 :invoke :add 7725 INFO [2022-01-10 08:39:44,393] jepsen worker 572 - jepsen.util 572 :info :add 6634 :RequestTimeoutException INFO [2022-01-10 08:39:44,397] jepsen worker 191 - jepsen.util 1191 :invoke :add 7726 INFO [2022-01-10 08:39:44,398] jepsen worker 920 - jepsen.util 1920 :invoke :add 7727 INFO [2022-01-10 08:39:44,400] jepsen worker 78 - jepsen.util 78 :invoke :add 7728 INFO [2022-01-10 08:39:44,402] jepsen worker 348 - jepsen.util 1348 :invoke :add 7729 INFO [2022-01-10 08:39:44,403] jepsen worker 978 - jepsen.util 978 :info :add 6636 :RequestTimeoutException INFO [2022-01-10 08:39:44,404] jepsen worker 622 - jepsen.util 622 :invoke :add 7730 INFO [2022-01-10 08:39:44,407] jepsen worker 444 - jepsen.util 1444 :invoke :add 7731 INFO [2022-01-10 08:39:44,408] jepsen worker 624 - jepsen.util 624 :info :add 6641 :RequestTimeoutException INFO [2022-01-10 08:39:44,409] jepsen worker 978 - jepsen.util 1978 :invoke :add 7732 INFO [2022-01-10 08:39:44,411] jepsen worker 143 - jepsen.util 143 :info :add 6640 :RequestTimeoutException INFO [2022-01-10 08:39:44,412] jepsen worker 76 - jepsen.util 76 :info :add 6642 :RequestTimeoutException INFO [2022-01-10 08:39:44,412] jepsen worker 429 - jepsen.util 1429 :ok :add 7010 INFO [2022-01-10 08:39:44,413] jepsen worker 529 - jepsen.util 529 :info :add 6643 :RequestTimeoutException INFO [2022-01-10 08:39:44,414] jepsen worker 693 - jepsen.util 693 :info :add 6644 :RequestTimeoutException INFO [2022-01-10 08:39:44,415] jepsen worker 513 - jepsen.util 513 :invoke :add 7733 INFO [2022-01-10 08:39:44,418] jepsen worker 143 - jepsen.util 1143 :invoke :add 7734 INFO [2022-01-10 08:39:44,420] jepsen worker 76 - jepsen.util 1076 :invoke :add 7735 INFO [2022-01-10 08:39:44,422] jepsen worker 594 - jepsen.util 594 :invoke :add 7736 INFO [2022-01-10 08:39:44,423] jepsen worker 876 - jepsen.util 876 :info :add 6645 :RequestTimeoutException INFO [2022-01-10 08:39:44,424] jepsen worker 429 - jepsen.util 1429 :invoke :add 7737 INFO [2022-01-10 08:39:44,427] jepsen worker 25 - jepsen.util 1025 :invoke :add 7738 INFO [2022-01-10 08:39:44,428] jepsen worker 619 - jepsen.util 1619 :ok :add 7018 INFO [2022-01-10 08:39:44,428] jepsen worker 477 - jepsen.util 1477 :ok :add 7371 INFO [2022-01-10 08:39:44,428] jepsen worker 981 - jepsen.util 981 :ok :add 7190 INFO [2022-01-10 08:39:44,428] jepsen worker 358 - jepsen.util 358 :ok :add 7034 INFO [2022-01-10 08:39:44,428] jepsen worker 524 - jepsen.util 1524 :ok :add 7403 INFO [2022-01-10 08:39:44,429] jepsen worker 308 - jepsen.util 1308 :ok :add 7369 INFO [2022-01-10 08:39:44,429] jepsen worker 776 - jepsen.util 776 :ok :add 7188 INFO [2022-01-10 08:39:44,429] jepsen worker 588 - jepsen.util 588 :ok :add 6894 INFO [2022-01-10 08:39:44,429] jepsen worker 460 - jepsen.util 1460 :ok :add 6904 INFO [2022-01-10 08:39:44,429] jepsen worker 947 - jepsen.util 947 :ok :add 6785 INFO [2022-01-10 08:39:44,429] jepsen worker 733 - jepsen.util 733 :ok :add 6930 INFO [2022-01-10 08:39:44,430] jepsen worker 624 - jepsen.util 1624 :invoke :add 7739 INFO [2022-01-10 08:39:44,434] jepsen worker 733 - jepsen.util 733 :invoke :add 7740 INFO [2022-01-10 08:39:44,436] jepsen worker 907 - jepsen.util 907 :ok :add 7068 INFO [2022-01-10 08:39:44,436] jepsen worker 477 - jepsen.util 1477 :invoke :add 7741 INFO [2022-01-10 08:39:44,436] jepsen worker 225 - jepsen.util 225 :ok :add 6651 INFO [2022-01-10 08:39:44,436] jepsen worker 840 - jepsen.util 840 :ok :add 7070 INFO [2022-01-10 08:39:44,437] jepsen worker 896 - jepsen.util 1896 :ok :add 7044 INFO [2022-01-10 08:39:44,437] jepsen worker 386 - jepsen.util 386 :ok :add 6974 INFO [2022-01-10 08:39:44,437] jepsen worker 450 - jepsen.util 1450 :ok :add 6958 INFO [2022-01-10 08:39:44,439] jepsen worker 776 - jepsen.util 776 :invoke :add 7742 INFO [2022-01-10 08:39:44,441] jepsen worker 460 - jepsen.util 1460 :invoke :add 7743 INFO [2022-01-10 08:39:44,443] jepsen worker 529 - jepsen.util 1529 :invoke :add 7744 INFO [2022-01-10 08:39:44,445] jepsen worker 880 - jepsen.util 1880 :invoke :add 7745 INFO [2022-01-10 08:39:44,447] jepsen worker 308 - jepsen.util 1308 :invoke :add 7746 INFO [2022-01-10 08:39:44,449] jepsen worker 358 - jepsen.util 358 :invoke :add 7747 INFO [2022-01-10 08:39:44,451] jepsen worker 572 - jepsen.util 1572 :invoke :add 7748 INFO [2022-01-10 08:39:44,453] jepsen worker 292 - jepsen.util 1292 :ok :add 7247 INFO [2022-01-10 08:39:44,453] jepsen worker 757 - jepsen.util 757 :ok :add 6832 INFO [2022-01-10 08:39:44,453] jepsen worker 663 - jepsen.util 1663 :ok :add 7102 INFO [2022-01-10 08:39:44,453] jepsen worker 807 - jepsen.util 807 :ok :add 6723 INFO [2022-01-10 08:39:44,453] jepsen worker 17 - jepsen.util 1017 :ok :add 7136 INFO [2022-01-10 08:39:44,453] jepsen worker 154 - jepsen.util 154 :ok :add 6806 INFO [2022-01-10 08:39:44,453] jepsen worker 147 - jepsen.util 147 :ok :add 6717 INFO [2022-01-10 08:39:44,453] jepsen worker 947 - jepsen.util 947 :invoke :add 7749 INFO [2022-01-10 08:39:44,454] jepsen worker 382 - jepsen.util 1382 :ok :add 7535 INFO [2022-01-10 08:39:44,457] jepsen worker 17 - jepsen.util 1017 :invoke :add 7750 INFO [2022-01-10 08:39:44,459] jepsen worker 588 - jepsen.util 588 :invoke :add 7751 INFO [2022-01-10 08:39:44,461] jepsen worker 490 - jepsen.util 1490 :invoke :add 7752 INFO [2022-01-10 08:39:44,463] jepsen worker 896 - jepsen.util 1896 :invoke :add 7753 INFO [2022-01-10 08:39:44,466] jepsen worker 405 - jepsen.util 1405 :ok :add 6997 INFO [2022-01-10 08:39:44,466] jepsen worker 770 - jepsen.util 770 :ok :add 6686 INFO [2022-01-10 08:39:44,466] jepsen worker 200 - jepsen.util 1200 :ok :add 7272 INFO [2022-01-10 08:39:44,466] jepsen worker 907 - jepsen.util 907 :invoke :add 7754 INFO [2022-01-10 08:39:44,466] jepsen worker 853 - jepsen.util 853 :ok :add 6716 INFO [2022-01-10 08:39:44,466] jepsen worker 208 - jepsen.util 208 :ok :add 6833 INFO [2022-01-10 08:39:44,466] jepsen worker 240 - jepsen.util 240 :ok :add 7669 INFO [2022-01-10 08:39:44,466] jepsen worker 152 - jepsen.util 1152 :ok :add 7093 INFO [2022-01-10 08:39:44,466] jepsen worker 70 - jepsen.util 70 :ok :add 7500 INFO [2022-01-10 08:39:44,469] jepsen worker 386 - jepsen.util 386 :invoke :add 7755 INFO [2022-01-10 08:39:44,472] jepsen worker 382 - jepsen.util 1382 :invoke :add 7756 INFO [2022-01-10 08:39:44,474] jepsen worker 208 - jepsen.util 208 :invoke :add 7757 INFO [2022-01-10 08:39:44,475] jepsen worker 225 - jepsen.util 225 :invoke :add 7758 INFO [2022-01-10 08:39:44,477] jepsen worker 619 - jepsen.util 1619 :invoke :add 7759 INFO [2022-01-10 08:39:44,478] jepsen worker 853 - jepsen.util 853 :invoke :add 7760 INFO [2022-01-10 08:39:44,480] jepsen worker 770 - jepsen.util 770 :invoke :add 7761 INFO [2022-01-10 08:39:44,483] jepsen worker 524 - jepsen.util 1524 :invoke :add 7762 INFO [2022-01-10 08:39:44,484] jepsen worker 693 - jepsen.util 1693 :invoke :add 7763 INFO [2022-01-10 08:39:44,486] jepsen worker 200 - jepsen.util 1200 :invoke :add 7764 INFO [2022-01-10 08:39:44,487] jepsen worker 359 - jepsen.util 1359 :ok :add 7100 INFO [2022-01-10 08:39:44,487] jepsen worker 568 - jepsen.util 568 :ok :add 6994 INFO [2022-01-10 08:39:44,487] jepsen worker 909 - jepsen.util 1909 :ok :add 7118 INFO [2022-01-10 08:39:44,487] jepsen worker 36 - jepsen.util 36 :ok :add 6830 INFO [2022-01-10 08:39:44,487] jepsen worker 999 - jepsen.util 999 :ok :add 6804 INFO [2022-01-10 08:39:44,487] jepsen worker 895 - jepsen.util 1895 :ok :add 7090 INFO [2022-01-10 08:39:44,487] jepsen worker 255 - jepsen.util 255 :ok :add 6828 INFO [2022-01-10 08:39:44,487] jepsen worker 431 - jepsen.util 1431 :ok :add 7269 INFO [2022-01-10 08:39:44,488] jepsen worker 763 - jepsen.util 763 :ok :add 6715 INFO [2022-01-10 08:39:44,488] jepsen worker 292 - jepsen.util 1292 :invoke :add 7765 INFO [2022-01-10 08:39:44,488] jepsen worker 61 - jepsen.util 61 :ok :add 6721 INFO [2022-01-10 08:39:44,488] jepsen worker 105 - jepsen.util 1105 :ok :add 7134 INFO [2022-01-10 08:39:44,488] jepsen worker 724 - jepsen.util 1724 :ok :add 7672 INFO [2022-01-10 08:39:44,488] jepsen worker 669 - jepsen.util 669 :ok :add 6739 INFO [2022-01-10 08:39:44,492] jepsen worker 474 - jepsen.util 474 :ok :add 6720 WARN [2022-01-10 08:39:44,492] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":12855281,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12855281,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":2706785,"encode_duration_us":263,"operation_id":"0x1aa0","timeout_ms":10000,"last_server_duration_us":2706785,"total_duration_us":10008644},{"operation_name":"insert","last_dispatch_duration_us":10609652,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10609652,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":9008729,"encode_duration_us":476,"operation_id":"0x1684","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10009000},{"operation_name":"insert","last_dispatch_duration_us":10035811,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10035811,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":3634061,"encode_duration_us":269,"operation_id":"0x1621","timeout_ms":10000,"last_server_duration_us":3634061,"total_duration_us":10009590},{"operation_name":"insert","last_dispatch_duration_us":10225186,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10225186,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":7100337,"encode_duration_us":276,"operation_id":"0x1b89","timeout_ms":10000,"last_server_duration_us":7100337,"total_duration_us":10010022},{"operation_name":"insert","last_dispatch_duration_us":10146606,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10146606,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":6598556,"encode_duration_us":258,"operation_id":"0x1aaf","timeout_ms":10000,"last_server_duration_us":6598556,"total_duration_us":10009369},{"operation_name":"insert","last_dispatch_duration_us":10317391,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10317391,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":9004490,"encode_duration_us":271,"operation_id":"0x16f5","timeout_ms":10000,"last_server_duration_us":9004490,"total_duration_us":10023662},{"operation_name":"insert","last_dispatch_duration_us":10274634,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10274634,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":4536277,"encode_duration_us":281,"operation_id":"0x172c","timeout_ms":10000,"last_server_duration_us":4536277,"total_duration_us":10009686},{"operation_name":"insert","last_dispatch_duration_us":10459029,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10459029,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":9007669,"encode_duration_us":271,"operation_id":"0x16be","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10019613},{"operation_name":"insert","last_dispatch_duration_us":10175038,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10175038,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":4116615,"encode_duration_us":288,"operation_id":"0x16a5","timeout_ms":10000,"last_server_duration_us":4116615,"total_duration_us":10014803},{"operation_name":"insert","last_dispatch_duration_us":11095445,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11095445,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":9010849,"encode_duration_us":314,"operation_id":"0x15c8","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10031144}],"total_count":30}} INFO [2022-01-10 08:39:44,493] jepsen worker 304 - jepsen.util 304 :ok :add 6805 INFO [2022-01-10 08:39:44,493] jepsen worker 425 - jepsen.util 1425 :ok :add 7270 WARN [2022-01-10 08:39:44,493] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":11095445,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":11095445,"total_server_duration_us":9010849,"encode_duration_us":314,"operation_id":"0x15c8","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10031144},{"operation_name":"insert","last_dispatch_duration_us":10317391,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":10317391,"total_server_duration_us":9004490,"encode_duration_us":271,"operation_id":"0x16f5","timeout_ms":10000,"last_server_duration_us":9004490,"total_duration_us":10023662},{"operation_name":"insert","last_dispatch_duration_us":10459029,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","total_dispatch_duration_us":10459029,"total_server_duration_us":9007669,"encode_duration_us":271,"operation_id":"0x16be","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10019613},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","encode_duration_us":314,"operation_id":"0x1c91","timeout_ms":10000,"total_duration_us":10017388},{"operation_name":"insert","last_dispatch_duration_us":10175038,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","total_dispatch_duration_us":10175038,"total_server_duration_us":4116615,"encode_duration_us":288,"operation_id":"0x16a5","timeout_ms":10000,"last_server_duration_us":4116615,"total_duration_us":10014803},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","encode_duration_us":327,"operation_id":"0x1a37","timeout_ms":10000,"total_duration_us":10010706},{"operation_name":"insert","last_dispatch_duration_us":10225186,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","total_dispatch_duration_us":10225186,"total_server_duration_us":7100337,"encode_duration_us":276,"operation_id":"0x1b89","timeout_ms":10000,"last_server_duration_us":7100337,"total_duration_us":10010022},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","encode_duration_us":382,"operation_id":"0x1b02","timeout_ms":10000,"total_duration_us":10009914},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","encode_duration_us":262,"operation_id":"0x1b69","timeout_ms":10000,"total_duration_us":10009890},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","encode_duration_us":302,"operation_id":"0x1898","timeout_ms":10000,"total_duration_us":10009844}],"total_count":176}} INFO [2022-01-10 08:39:44,493] jepsen worker 60 - jepsen.util 60 :ok :add 6829 INFO [2022-01-10 08:39:44,493] jepsen worker 456 - jepsen.util 1456 :ok :add 6995 INFO [2022-01-10 08:39:44,493] jepsen worker 276 - jepsen.util 1276 :ok :add 7135 INFO [2022-01-10 08:39:44,493] jepsen worker 605 - jepsen.util 605 :ok :add 6831 INFO [2022-01-10 08:39:44,493] jepsen worker 648 - jepsen.util 1648 :ok :add 7244 WARN [2022-01-10 08:39:44,493] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":10374490,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":10374490,"total_server_duration_us":9007669,"encode_duration_us":308,"operation_id":"0x16a7","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10019188},{"operation_name":"insert","last_dispatch_duration_us":10136589,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":10136589,"total_server_duration_us":9007669,"encode_duration_us":305,"operation_id":"0x1707","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10010815},{"operation_name":"insert","last_dispatch_duration_us":10057683,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":10057683,"total_server_duration_us":9006609,"encode_duration_us":282,"operation_id":"0x1727","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":10010811},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000002/00000000F4655C1A","last_local_socket":"172.23.104.79:41954","encode_duration_us":308,"operation_id":"0x1d72","timeout_ms":10000,"total_duration_us":10010796},{"operation_name":"insert","last_dispatch_duration_us":10563463,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":10563463,"total_server_duration_us":9004490,"encode_duration_us":285,"operation_id":"0x1660","timeout_ms":10000,"last_server_duration_us":9004490,"total_duration_us":10010361},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","encode_duration_us":267,"operation_id":"0x1bd9","timeout_ms":10000,"total_duration_us":10009934},{"operation_name":"insert","last_dispatch_duration_us":10119089,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000002/00000000F4655C1A","last_local_socket":"172.23.104.79:41954","total_dispatch_duration_us":10119089,"total_server_duration_us":6580925,"encode_duration_us":255,"operation_id":"0x1abd","timeout_ms":10000,"last_server_duration_us":6580925,"total_duration_us":10009536},{"operation_name":"insert","last_dispatch_duration_us":15098574,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","total_dispatch_duration_us":15098574,"total_server_duration_us":5385169,"encode_duration_us":299,"operation_id":"0x1a72","timeout_ms":10000,"last_server_duration_us":5385169,"total_duration_us":10008961},{"operation_name":"insert","last_dispatch_duration_us":10846004,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000002/00000000F4655C1A","last_local_socket":"172.23.104.79:41954","total_dispatch_duration_us":10846004,"total_server_duration_us":7114711,"encode_duration_us":488,"operation_id":"0x1a52","timeout_ms":10000,"last_server_duration_us":7114711,"total_duration_us":10008960},{"operation_name":"insert","last_dispatch_duration_us":10235686,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000002/00000000F4655C1A","last_local_socket":"172.23.104.79:41954","total_dispatch_duration_us":10235686,"total_server_duration_us":7068757,"encode_duration_us":296,"operation_id":"0x1b83","timeout_ms":10000,"last_server_duration_us":7068757,"total_duration_us":10008653}],"total_count":188}} INFO [2022-01-10 08:39:44,496] jepsen worker 54 - jepsen.util 54 :ok :add 6738 INFO [2022-01-10 08:39:44,496] jepsen worker 583 - jepsen.util 1583 :ok :add 7268 INFO [2022-01-10 08:39:44,496] jepsen worker 465 - jepsen.util 1465 :ok :add 7119 INFO [2022-01-10 08:39:44,496] jepsen worker 994 - jepsen.util 1994 :ok :add 7101 INFO [2022-01-10 08:39:44,496] jepsen worker 744 - jepsen.util 1744 :ok :add 7089 INFO [2022-01-10 08:39:44,496] jepsen worker 355 - jepsen.util 1355 :ok :add 7647 INFO [2022-01-10 08:39:44,497] jepsen worker 154 - jepsen.util 154 :invoke :add 7766 WARN [2022-01-10 08:39:44,498] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":10734617,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10734617,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9008729,"encode_duration_us":314,"operation_id":"0x161b","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10009608},{"operation_name":"insert","last_dispatch_duration_us":6270424,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":6270424,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":4380677,"encode_duration_us":379,"operation_id":"0x19c6","timeout_ms":10000,"last_server_duration_us":4380677,"total_duration_us":10010444},{"operation_name":"insert","last_dispatch_duration_us":10475001,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10475001,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9003430,"encode_duration_us":507,"operation_id":"0x1687","timeout_ms":10000,"last_server_duration_us":9003430,"total_duration_us":10010578},{"operation_name":"insert","last_dispatch_duration_us":10293999,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10293999,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9001311,"encode_duration_us":269,"operation_id":"0x16ce","timeout_ms":10000,"last_server_duration_us":9001311,"total_duration_us":10012636},{"operation_name":"insert","last_dispatch_duration_us":10216164,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10216164,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":4533902,"encode_duration_us":275,"operation_id":"0x174a","timeout_ms":10000,"last_server_duration_us":4533902,"total_duration_us":10010541},{"operation_name":"insert","last_dispatch_duration_us":10531429,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10531429,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":3880286,"encode_duration_us":282,"operation_id":"0x15be","timeout_ms":10000,"last_server_duration_us":3880286,"total_duration_us":10044746},{"operation_name":"insert","last_dispatch_duration_us":12243413,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12243413,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":5792604,"encode_duration_us":550,"operation_id":"0x19e4","timeout_ms":10000,"last_server_duration_us":5792604,"total_duration_us":10011346},{"operation_name":"insert","last_dispatch_duration_us":10280797,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10280797,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":3627580,"encode_duration_us":465,"operation_id":"0x15c0","timeout_ms":10000,"last_server_duration_us":3627580,"total_duration_us":10038300},{"operation_name":"insert","last_dispatch_duration_us":10023922,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10023922,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":3623981,"encode_duration_us":374,"operation_id":"0x1627","timeout_ms":10000,"last_server_duration_us":3623981,"total_duration_us":10013070},{"operation_name":"insert","last_dispatch_duration_us":10763261,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10763261,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9011909,"encode_duration_us":316,"operation_id":"0x160f","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10017309}],"total_count":31}} WARN [2022-01-10 08:39:44,499] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":10531429,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":10531429,"total_server_duration_us":3880286,"encode_duration_us":282,"operation_id":"0x15be","timeout_ms":10000,"last_server_duration_us":3880286,"total_duration_us":10044746},{"operation_name":"insert","last_dispatch_duration_us":10280797,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":10280797,"total_server_duration_us":3627580,"encode_duration_us":465,"operation_id":"0x15c0","timeout_ms":10000,"last_server_duration_us":3627580,"total_duration_us":10038300},{"operation_name":"insert","last_dispatch_duration_us":10763261,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":10763261,"total_server_duration_us":9011909,"encode_duration_us":316,"operation_id":"0x160f","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10017309},{"operation_name":"insert","last_dispatch_duration_us":10023922,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":10023922,"total_server_duration_us":3623981,"encode_duration_us":374,"operation_id":"0x1627","timeout_ms":10000,"last_server_duration_us":3623981,"total_duration_us":10013070},{"operation_name":"insert","last_dispatch_duration_us":10293999,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":10293999,"total_server_duration_us":9001311,"encode_duration_us":269,"operation_id":"0x16ce","timeout_ms":10000,"last_server_duration_us":9001311,"total_duration_us":10012636},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","encode_duration_us":265,"operation_id":"0x1ae4","timeout_ms":10000,"total_duration_us":10012570},{"operation_name":"insert","last_dispatch_duration_us":12243413,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":12243413,"total_server_duration_us":5792604,"encode_duration_us":550,"operation_id":"0x19e4","timeout_ms":10000,"last_server_duration_us":5792604,"total_duration_us":10011346},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","encode_duration_us":268,"operation_id":"0x1c45","timeout_ms":10000,"total_duration_us":10010904},{"operation_name":"insert","encode_duration_us":259,"operation_id":"0x1c90","timeout_ms":10000,"total_duration_us":10010848},{"operation_name":"insert","encode_duration_us":294,"operation_id":"0x1cfd","timeout_ms":10000,"total_duration_us":10010612}],"total_count":176}} INFO [2022-01-10 08:39:44,500] jepsen worker 456 - jepsen.util 1456 :invoke :add 7767 INFO [2022-01-10 08:39:44,500] jepsen worker 715 - jepsen.util 715 :ok :add 6693 INFO [2022-01-10 08:39:44,501] jepsen worker 21 - jepsen.util 21 :ok :add 7485 INFO [2022-01-10 08:39:44,501] jepsen worker 185 - jepsen.util 1185 :ok :add 7468 INFO [2022-01-10 08:39:44,501] jepsen worker 691 - jepsen.util 691 :ok :add 7047 INFO [2022-01-10 08:39:44,501] jepsen worker 211 - jepsen.util 211 :ok :add 6669 INFO [2022-01-10 08:39:44,502] jepsen worker 1 - jepsen.util 1001 :ok :add 6954 INFO [2022-01-10 08:39:44,502] jepsen worker 876 - jepsen.util 1876 :invoke :add 7768 INFO [2022-01-10 08:39:44,502] jepsen worker 910 - jepsen.util 1910 :ok :add 7339 INFO [2022-01-10 08:39:44,502] jepsen worker 127 - jepsen.util 127 :ok :add 7064 INFO [2022-01-10 08:39:44,503] jepsen worker 390 - jepsen.util 390 :ok :add 7086 INFO [2022-01-10 08:39:44,503] jepsen worker 403 - jepsen.util 403 :ok :add 7453 INFO [2022-01-10 08:39:44,503] jepsen worker 33 - jepsen.util 33 :ok :add 6671 INFO [2022-01-10 08:39:44,503] jepsen worker 202 - jepsen.util 202 :ok :add 6812 INFO [2022-01-10 08:39:44,505] jepsen worker 360 - jepsen.util 360 :ok :add 6826 INFO [2022-01-10 08:39:44,505] jepsen worker 674 - jepsen.util 674 :ok :add 7321 INFO [2022-01-10 08:39:44,505] jepsen worker 134 - jepsen.util 1134 :ok :add 7116 INFO [2022-01-10 08:39:44,505] jepsen worker 175 - jepsen.util 175 :ok :add 7267 INFO [2022-01-10 08:39:44,505] jepsen worker 990 - jepsen.util 990 :ok :add 6737 INFO [2022-01-10 08:39:44,505] jepsen worker 228 - jepsen.util 1228 :ok :add 7122 INFO [2022-01-10 08:39:44,508] jepsen worker 255 - jepsen.util 255 :invoke :add 7769 INFO [2022-01-10 08:39:44,510] jepsen worker 304 - jepsen.util 304 :invoke :add 7770 INFO [2022-01-10 08:39:44,512] jepsen worker 674 - jepsen.util 674 :invoke :add 7771 INFO [2022-01-10 08:39:44,512] jepsen worker 839 - jepsen.util 839 :ok :add 6767 INFO [2022-01-10 08:39:44,512] jepsen worker 481 - jepsen.util 481 :ok :add 6876 INFO [2022-01-10 08:39:44,513] jepsen worker 670 - jepsen.util 670 :ok :add 7146 INFO [2022-01-10 08:39:44,514] jepsen worker 654 - jepsen.util 1654 :ok :add 7634 INFO [2022-01-10 08:39:44,514] jepsen worker 124 - jepsen.util 1124 :ok :add 7600 INFO [2022-01-10 08:39:44,514] jepsen worker 80 - jepsen.util 1080 :ok :add 7237 INFO [2022-01-10 08:39:44,514] jepsen worker 890 - jepsen.util 890 :ok :add 6753 INFO [2022-01-10 08:39:44,514] jepsen worker 807 - jepsen.util 807 :invoke :add 7772 INFO [2022-01-10 08:39:44,514] jepsen worker 289 - jepsen.util 1289 :ok :add 7203 INFO [2022-01-10 08:39:44,514] jepsen worker 851 - jepsen.util 851 :ok :add 6842 INFO [2022-01-10 08:39:44,514] jepsen worker 214 - jepsen.util 1214 :ok :add 7571 INFO [2022-01-10 08:39:44,514] jepsen worker 695 - jepsen.util 1695 :ok :add 7618 INFO [2022-01-10 08:39:44,519] jepsen worker 360 - jepsen.util 360 :invoke :add 7773 INFO [2022-01-10 08:39:44,520] jepsen worker 134 - jepsen.util 1134 :invoke :add 7774 INFO [2022-01-10 08:39:44,523] jepsen worker 909 - jepsen.util 1909 :invoke :add 7775 INFO [2022-01-10 08:39:44,525] jepsen worker 127 - jepsen.util 127 :invoke :add 7776 INFO [2022-01-10 08:39:44,527] jepsen worker 105 - jepsen.util 1105 :invoke :add 7777 INFO [2022-01-10 08:39:44,528] jepsen worker 542 - jepsen.util 542 :ok :add 6975 INFO [2022-01-10 08:39:44,528] jepsen worker 410 - jepsen.util 1410 :ok :add 7693 INFO [2022-01-10 08:39:44,529] jepsen worker 237 - jepsen.util 237 :ok :add 6959 INFO [2022-01-10 08:39:44,529] jepsen worker 584 - jepsen.util 1584 :ok :add 6941 INFO [2022-01-10 08:39:44,529] jepsen worker 268 - jepsen.util 1268 :ok :add 7446 INFO [2022-01-10 08:39:44,529] jepsen worker 885 - jepsen.util 1885 :ok :add 7472 INFO [2022-01-10 08:39:44,529] jepsen worker 840 - jepsen.util 840 :invoke :add 7778 INFO [2022-01-10 08:39:44,529] jepsen worker 486 - jepsen.util 486 :ok :add 7071 INFO [2022-01-10 08:39:44,529] jepsen worker 296 - jepsen.util 1296 :ok :add 7318 INFO [2022-01-10 08:39:44,534] jepsen worker 744 - jepsen.util 1744 :invoke :add 7779 INFO [2022-01-10 08:39:44,536] jepsen worker 450 - jepsen.util 1450 :invoke :add 7780 INFO [2022-01-10 08:39:44,538] jepsen worker 654 - jepsen.util 1654 :invoke :add 7781 INFO [2022-01-10 08:39:44,539] jepsen worker 648 - jepsen.util 1648 :invoke :add 7782 INFO [2022-01-10 08:39:44,541] jepsen worker 212 - jepsen.util 212 :ok :add 6907 INFO [2022-01-10 08:39:44,541] jepsen worker 597 - jepsen.util 1597 :ok :add 7346 INFO [2022-01-10 08:39:44,541] jepsen worker 789 - jepsen.util 1789 :ok :add 7003 INFO [2022-01-10 08:39:44,541] jepsen worker 361 - jepsen.util 1361 :ok :add 7590 INFO [2022-01-10 08:39:44,541] jepsen worker 598 - jepsen.util 1598 :ok :add 7400 INFO [2022-01-10 08:39:44,541] jepsen worker 27 - jepsen.util 27 :ok :add 6752 INFO [2022-01-10 08:39:44,541] jepsen worker 351 - jepsen.util 351 :ok :add 6766 INFO [2022-01-10 08:39:44,541] jepsen worker 930 - jepsen.util 1930 :ok :add 7418 INFO [2022-01-10 08:39:44,541] jepsen worker 400 - jepsen.util 1400 :ok :add 7202 INFO [2022-01-10 08:39:44,541] jepsen worker 700 - jepsen.util 1700 :ok :add 7392 INFO [2022-01-10 08:39:44,541] jepsen worker 670 - jepsen.util 670 :invoke :add 7783 INFO [2022-01-10 08:39:44,541] jepsen worker 2 - jepsen.util 2 :ok :add 6933 INFO [2022-01-10 08:39:44,541] jepsen worker 528 - jepsen.util 528 :ok :add 6786 INFO [2022-01-10 08:39:44,541] jepsen worker 164 - jepsen.util 164 :ok :add 6897 INFO [2022-01-10 08:39:44,542] jepsen worker 727 - jepsen.util 727 :ok :add 7037 INFO [2022-01-10 08:39:44,542] jepsen worker 84 - jepsen.util 1084 :ok :add 7588 INFO [2022-01-10 08:39:44,542] jepsen worker 401 - jepsen.util 401 :ok :add 7434 INFO [2022-01-10 08:39:44,542] jepsen worker 328 - jepsen.util 328 :ok :add 7147 INFO [2022-01-10 08:39:44,542] jepsen worker 880 - jepsen.util 1880 :ok :add 7745 INFO [2022-01-10 08:39:44,548] jepsen worker 851 - jepsen.util 851 :invoke :add 7784 INFO [2022-01-10 08:39:44,550] jepsen worker 289 - jepsen.util 1289 :invoke :add 7785 INFO [2022-01-10 08:39:44,552] jepsen worker 486 - jepsen.util 486 :invoke :add 7786 INFO [2022-01-10 08:39:44,553] jepsen worker 147 - jepsen.util 147 :invoke :add 7787 INFO [2022-01-10 08:39:44,555] jepsen worker 164 - jepsen.util 164 :invoke :add 7788 INFO [2022-01-10 08:39:44,557] jepsen worker 895 - jepsen.util 1895 :invoke :add 7789 INFO [2022-01-10 08:39:44,559] jepsen worker 21 - jepsen.util 21 :invoke :add 7790 INFO [2022-01-10 08:39:44,561] jepsen worker 598 - jepsen.util 1598 :invoke :add 7791 INFO [2022-01-10 08:39:44,563] jepsen worker 390 - jepsen.util 390 :invoke :add 7792 INFO [2022-01-10 08:39:44,565] jepsen worker 361 - jepsen.util 1361 :invoke :add 7793 INFO [2022-01-10 08:39:44,567] jepsen worker 1 - jepsen.util 1001 :invoke :add 7794 INFO [2022-01-10 08:39:44,569] jepsen worker 695 - jepsen.util 1695 :invoke :add 7795 INFO [2022-01-10 08:39:44,571] jepsen worker 27 - jepsen.util 27 :invoke :add 7796 INFO [2022-01-10 08:39:44,572] jepsen worker 355 - jepsen.util 1355 :invoke :add 7797 INFO [2022-01-10 08:39:44,574] jepsen worker 296 - jepsen.util 1296 :invoke :add 7798 INFO [2022-01-10 08:39:44,576] jepsen worker 2 - jepsen.util 2 :invoke :add 7799 INFO [2022-01-10 08:39:44,578] jepsen worker 240 - jepsen.util 240 :invoke :add 7800 INFO [2022-01-10 08:39:44,579] jepsen worker 757 - jepsen.util 757 :invoke :add 7801 INFO [2022-01-10 08:39:44,581] jepsen worker 124 - jepsen.util 1124 :invoke :add 7802 INFO [2022-01-10 08:39:44,584] jepsen worker 214 - jepsen.util 1214 :invoke :add 7803 INFO [2022-01-10 08:39:44,586] jepsen worker 185 - jepsen.util 1185 :invoke :add 7804 INFO [2022-01-10 08:39:44,587] jepsen worker 445 - jepsen.util 445 :info :add 6654 :RequestTimeoutException INFO [2022-01-10 08:39:44,588] jepsen worker 276 - jepsen.util 1276 :invoke :add 7805 INFO [2022-01-10 08:39:44,590] jepsen worker 465 - jepsen.util 1465 :invoke :add 7806 INFO [2022-01-10 08:39:44,592] jepsen worker 212 - jepsen.util 212 :invoke :add 7807 WARN [2022-01-10 08:39:44,593] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":11606622,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":11606622,"total_server_duration_us":9008729,"encode_duration_us":273,"operation_id":"0x1678","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10023846},{"operation_name":"insert","last_dispatch_duration_us":11741271,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":11741271,"total_server_duration_us":9012969,"encode_duration_us":297,"operation_id":"0x163e","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10012179},{"operation_name":"insert","last_dispatch_duration_us":10514140,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":10514140,"total_server_duration_us":9005550,"encode_duration_us":413,"operation_id":"0x1831","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10010983},{"operation_name":"insert","last_dispatch_duration_us":10151149,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","total_dispatch_duration_us":10151149,"total_server_duration_us":5613718,"encode_duration_us":258,"operation_id":"0x1aad","timeout_ms":10000,"last_server_duration_us":5613718,"total_duration_us":10010843},{"operation_name":"insert","last_dispatch_duration_us":6281522,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","total_dispatch_duration_us":6281522,"total_server_duration_us":1109369,"encode_duration_us":384,"operation_id":"0x1a65","timeout_ms":10000,"last_server_duration_us":1109369,"total_duration_us":10010771},{"operation_name":"insert","last_dispatch_duration_us":12650514,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":12650514,"total_server_duration_us":8063245,"encode_duration_us":272,"operation_id":"0x1b05","timeout_ms":10000,"last_server_duration_us":8063245,"total_duration_us":10010769},{"operation_name":"insert","last_dispatch_duration_us":13009964,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":13009964,"total_server_duration_us":9012969,"encode_duration_us":349,"operation_id":"0x193b","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10010465},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","encode_duration_us":299,"operation_id":"0x1982","timeout_ms":10000,"total_duration_us":10010464},{"operation_name":"insert","last_dispatch_duration_us":10287096,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","total_dispatch_duration_us":10287096,"total_server_duration_us":3735516,"encode_duration_us":297,"operation_id":"0x161f","timeout_ms":10000,"last_server_duration_us":3735516,"total_duration_us":10010444},{"operation_name":"insert","last_dispatch_duration_us":11890883,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":11890883,"total_server_duration_us":9005550,"encode_duration_us":316,"operation_id":"0x15fc","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10010430}],"total_count":204}} WARN [2022-01-10 08:39:44,593] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":11069719,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11069719,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":4406450,"encode_duration_us":293,"operation_id":"0x15f3","timeout_ms":10000,"last_server_duration_us":4406450,"total_duration_us":10010136},{"operation_name":"insert","last_dispatch_duration_us":11890883,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11890883,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9005550,"encode_duration_us":316,"operation_id":"0x15fc","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10010430},{"operation_name":"insert","last_dispatch_duration_us":12650514,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12650514,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":8063245,"encode_duration_us":272,"operation_id":"0x1b05","timeout_ms":10000,"last_server_duration_us":8063245,"total_duration_us":10010769},{"operation_name":"insert","last_dispatch_duration_us":10287096,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10287096,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":3735516,"encode_duration_us":297,"operation_id":"0x161f","timeout_ms":10000,"last_server_duration_us":3735516,"total_duration_us":10010444},{"operation_name":"insert","last_dispatch_duration_us":13009964,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13009964,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9012969,"encode_duration_us":349,"operation_id":"0x193b","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10010465},{"operation_name":"insert","last_dispatch_duration_us":10151149,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10151149,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":5613718,"encode_duration_us":258,"operation_id":"0x1aad","timeout_ms":10000,"last_server_duration_us":5613718,"total_duration_us":10010843},{"operation_name":"insert","last_dispatch_duration_us":11741271,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11741271,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9012969,"encode_duration_us":297,"operation_id":"0x163e","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10012179},{"operation_name":"insert","last_dispatch_duration_us":6281522,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":6281522,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":1109369,"encode_duration_us":384,"operation_id":"0x1a65","timeout_ms":10000,"last_server_duration_us":1109369,"total_duration_us":10010771},{"operation_name":"insert","last_dispatch_duration_us":10514140,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10514140,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9005550,"encode_duration_us":413,"operation_id":"0x1831","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10010983},{"operation_name":"insert","last_dispatch_duration_us":11606622,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11606622,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9008729,"encode_duration_us":273,"operation_id":"0x1678","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10023846}],"total_count":54}} INFO [2022-01-10 08:39:44,594] jepsen worker 207 - jepsen.util 207 :info :add 6655 :RequestTimeoutException WARN [2022-01-10 08:39:44,594] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":10639357,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10639357,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":9005550,"encode_duration_us":271,"operation_id":"0x163c","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10006764},{"operation_name":"insert","last_dispatch_duration_us":12714762,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12714762,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":3212115,"encode_duration_us":264,"operation_id":"0x1ae5","timeout_ms":10000,"last_server_duration_us":3212115,"total_duration_us":10007718},{"operation_name":"insert","last_dispatch_duration_us":10235686,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10235686,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":7068757,"encode_duration_us":296,"operation_id":"0x1b83","timeout_ms":10000,"last_server_duration_us":7068757,"total_duration_us":10008653},{"operation_name":"insert","last_dispatch_duration_us":10846004,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10846004,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":7114711,"encode_duration_us":488,"operation_id":"0x1a52","timeout_ms":10000,"last_server_duration_us":7114711,"total_duration_us":10008960},{"operation_name":"insert","last_dispatch_duration_us":11202192,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11202192,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":2156691,"encode_duration_us":290,"operation_id":"0x1bcf","timeout_ms":10000,"last_server_duration_us":2156691,"total_duration_us":10008319},{"operation_name":"insert","last_dispatch_duration_us":10374490,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10374490,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":9007669,"encode_duration_us":308,"operation_id":"0x16a7","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10019188},{"operation_name":"insert","last_dispatch_duration_us":10057683,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10057683,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":9006609,"encode_duration_us":282,"operation_id":"0x1727","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":10010811},{"operation_name":"insert","last_dispatch_duration_us":10136589,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10136589,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":9007669,"encode_duration_us":305,"operation_id":"0x1707","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10010815},{"operation_name":"insert","last_dispatch_duration_us":10119089,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10119089,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":6580925,"encode_duration_us":255,"operation_id":"0x1abd","timeout_ms":10000,"last_server_duration_us":6580925,"total_duration_us":10009536},{"operation_name":"insert","last_dispatch_duration_us":10563463,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10563463,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":9004490,"encode_duration_us":285,"operation_id":"0x1660","timeout_ms":10000,"last_server_duration_us":9004490,"total_duration_us":10010361}],"total_count":21}} INFO [2022-01-10 08:39:44,594] jepsen worker 211 - jepsen.util 211 :invoke :add 7808 INFO [2022-01-10 08:39:44,597] jepsen worker 152 - jepsen.util 1152 :invoke :add 7809 INFO [2022-01-10 08:39:44,597] jepsen worker 518 - jepsen.util 518 :info :add 6656 :RequestTimeoutException WARN [2022-01-10 08:39:44,598] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":10011702,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":10011702,"total_server_duration_us":9009789,"encode_duration_us":285,"operation_id":"0x1704","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10022821},{"operation_name":"insert","last_dispatch_duration_us":10435138,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":10435138,"total_server_duration_us":9019331,"encode_duration_us":445,"operation_id":"0x1668","timeout_ms":10000,"last_server_duration_us":9019331,"total_duration_us":10011796},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":282,"operation_id":"0x1cff","timeout_ms":10000,"total_duration_us":10010915},{"operation_name":"insert","last_dispatch_duration_us":8096454,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":8096454,"total_server_duration_us":4932869,"encode_duration_us":257,"operation_id":"0x1ba7","timeout_ms":10000,"last_server_duration_us":4932869,"total_duration_us":10010256},{"operation_name":"insert","last_dispatch_duration_us":5269126,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":5269126,"total_server_duration_us":2936406,"encode_duration_us":258,"operation_id":"0x1ae1","timeout_ms":10000,"last_server_duration_us":2936406,"total_duration_us":10010193},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":262,"operation_id":"0x1ac0","timeout_ms":10000,"total_duration_us":10009617},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":267,"operation_id":"0x1bf9","timeout_ms":10000,"total_duration_us":10009430},{"operation_name":"insert","last_dispatch_duration_us":10053598,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":10053598,"total_server_duration_us":9012969,"encode_duration_us":272,"operation_id":"0x16f2","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10009336},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":308,"operation_id":"0x1ac5","timeout_ms":10000,"total_duration_us":10008943},{"operation_name":"insert","last_dispatch_duration_us":7070688,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":7070688,"total_server_duration_us":3707861,"encode_duration_us":272,"operation_id":"0x1bd2","timeout_ms":10000,"last_server_duration_us":3707861,"total_duration_us":10008910}],"total_count":188}} WARN [2022-01-10 08:39:44,598] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":3299731,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":3299731,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":1502389,"encode_duration_us":314,"operation_id":"0x1889","timeout_ms":10000,"last_server_duration_us":1502389,"total_duration_us":10006895},{"operation_name":"insert","last_dispatch_duration_us":4882533,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":4882533,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":1890042,"encode_duration_us":264,"operation_id":"0x1ae3","timeout_ms":10000,"last_server_duration_us":1890042,"total_duration_us":10007013},{"operation_name":"insert","last_dispatch_duration_us":6166623,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":6166623,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":3022475,"encode_duration_us":269,"operation_id":"0x186d","timeout_ms":10000,"last_server_duration_us":3022475,"total_duration_us":10007609},{"operation_name":"insert","last_dispatch_duration_us":7249686,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":7249686,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":4729583,"encode_duration_us":297,"operation_id":"0x19ad","timeout_ms":10000,"last_server_duration_us":4729583,"total_duration_us":10007586},{"operation_name":"insert","last_dispatch_duration_us":7070688,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":7070688,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":3707861,"encode_duration_us":272,"operation_id":"0x1bd2","timeout_ms":10000,"last_server_duration_us":3707861,"total_duration_us":10008910},{"operation_name":"insert","last_dispatch_duration_us":10435138,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10435138,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9019331,"encode_duration_us":445,"operation_id":"0x1668","timeout_ms":10000,"last_server_duration_us":9019331,"total_duration_us":10011796},{"operation_name":"insert","last_dispatch_duration_us":3878332,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":3878332,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":2081660,"encode_duration_us":416,"operation_id":"0x1888","timeout_ms":10000,"last_server_duration_us":2081660,"total_duration_us":10008640},{"operation_name":"insert","last_dispatch_duration_us":10053598,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10053598,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9012969,"encode_duration_us":272,"operation_id":"0x16f2","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10009336},{"operation_name":"insert","last_dispatch_duration_us":8096454,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":8096454,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":4932869,"encode_duration_us":257,"operation_id":"0x1ba7","timeout_ms":10000,"last_server_duration_us":4932869,"total_duration_us":10010256},{"operation_name":"insert","last_dispatch_duration_us":5269126,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":5269126,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":2936406,"encode_duration_us":258,"operation_id":"0x1ae1","timeout_ms":10000,"last_server_duration_us":2936406,"total_duration_us":10010193}],"total_count":25}} INFO [2022-01-10 08:39:44,599] jepsen worker 228 - jepsen.util 1228 :invoke :add 7810 INFO [2022-01-10 08:39:44,602] jepsen worker 481 - jepsen.util 481 :invoke :add 7811 INFO [2022-01-10 08:39:44,603] jepsen worker 669 - jepsen.util 669 :invoke :add 7812 INFO [2022-01-10 08:39:44,605] jepsen worker 700 - jepsen.util 1700 :invoke :add 7813 INFO [2022-01-10 08:39:44,607] jepsen worker 528 - jepsen.util 528 :invoke :add 7814 INFO [2022-01-10 08:39:44,609] jepsen worker 542 - jepsen.util 542 :invoke :add 7815 INFO [2022-01-10 08:39:44,610] jepsen worker 175 - jepsen.util 175 :invoke :add 7816 INFO [2022-01-10 08:39:44,612] jepsen worker 70 - jepsen.util 70 :invoke :add 7817 INFO [2022-01-10 08:39:44,613] jepsen worker 359 - jepsen.util 1359 :invoke :add 7818 INFO [2022-01-10 08:39:44,615] jepsen worker 910 - jepsen.util 1910 :invoke :add 7819 INFO [2022-01-10 08:39:44,617] jepsen worker 328 - jepsen.util 328 :invoke :add 7820 INFO [2022-01-10 08:39:44,619] jepsen worker 207 - jepsen.util 1207 :invoke :add 7821 INFO [2022-01-10 08:39:44,621] jepsen worker 518 - jepsen.util 1518 :invoke :add 7822 INFO [2022-01-10 08:39:44,623] jepsen worker 84 - jepsen.util 1084 :invoke :add 7823 INFO [2022-01-10 08:39:44,625] jepsen worker 999 - jepsen.util 999 :invoke :add 7824 INFO [2022-01-10 08:39:44,626] jepsen worker 890 - jepsen.util 890 :invoke :add 7825 INFO [2022-01-10 08:39:44,628] jepsen worker 425 - jepsen.util 1425 :invoke :add 7826 INFO [2022-01-10 08:39:44,630] jepsen worker 237 - jepsen.util 237 :invoke :add 7827 INFO [2022-01-10 08:39:44,631] jepsen worker 990 - jepsen.util 990 :invoke :add 7828 INFO [2022-01-10 08:39:44,633] jepsen worker 763 - jepsen.util 763 :invoke :add 7829 INFO [2022-01-10 08:39:44,635] jepsen worker 724 - jepsen.util 1724 :invoke :add 7830 INFO [2022-01-10 08:39:44,637] jepsen worker 403 - jepsen.util 403 :invoke :add 7831 INFO [2022-01-10 08:39:44,638] jepsen worker 60 - jepsen.util 60 :invoke :add 7832 INFO [2022-01-10 08:39:44,640] jepsen worker 866 - jepsen.util 1866 :invoke :add 7833 INFO [2022-01-10 08:39:44,642] jepsen worker 401 - jepsen.util 401 :invoke :add 7834 INFO [2022-01-10 08:39:44,644] jepsen worker 663 - jepsen.util 1663 :invoke :add 7835 INFO [2022-01-10 08:39:44,645] jepsen worker 568 - jepsen.util 568 :invoke :add 7836 INFO [2022-01-10 08:39:44,647] jepsen worker 405 - jepsen.util 1405 :invoke :add 7837 INFO [2022-01-10 08:39:44,648] jepsen worker 994 - jepsen.util 1994 :invoke :add 7838 INFO [2022-01-10 08:39:44,650] jepsen worker 981 - jepsen.util 981 :invoke :add 7839 INFO [2022-01-10 08:39:44,652] jepsen worker 691 - jepsen.util 691 :invoke :add 7840 INFO [2022-01-10 08:39:44,653] jepsen worker 36 - jepsen.util 36 :invoke :add 7841 INFO [2022-01-10 08:39:44,655] jepsen worker 410 - jepsen.util 1410 :invoke :add 7842 INFO [2022-01-10 08:39:44,657] jepsen worker 597 - jepsen.util 1597 :invoke :add 7843 INFO [2022-01-10 08:39:44,659] jepsen worker 583 - jepsen.util 1583 :invoke :add 7844 INFO [2022-01-10 08:39:44,661] jepsen worker 885 - jepsen.util 1885 :invoke :add 7845 INFO [2022-01-10 08:39:44,662] jepsen worker 880 - jepsen.util 1880 :invoke :add 7846 INFO [2022-01-10 08:39:44,664] jepsen worker 839 - jepsen.util 839 :invoke :add 7847 INFO [2022-01-10 08:39:44,666] jepsen worker 789 - jepsen.util 1789 :invoke :add 7848 INFO [2022-01-10 08:39:44,668] jepsen worker 400 - jepsen.util 1400 :invoke :add 7849 INFO [2022-01-10 08:39:44,669] jepsen worker 930 - jepsen.util 1930 :invoke :add 7850 INFO [2022-01-10 08:39:44,671] jepsen worker 351 - jepsen.util 351 :invoke :add 7851 INFO [2022-01-10 08:39:44,673] jepsen worker 80 - jepsen.util 1080 :invoke :add 7852 INFO [2022-01-10 08:39:44,675] jepsen worker 268 - jepsen.util 1268 :invoke :add 7853 INFO [2022-01-10 08:39:44,676] jepsen worker 54 - jepsen.util 54 :invoke :add 7854 INFO [2022-01-10 08:39:44,678] jepsen worker 61 - jepsen.util 61 :invoke :add 7855 INFO [2022-01-10 08:39:44,680] jepsen worker 474 - jepsen.util 474 :invoke :add 7856 INFO [2022-01-10 08:39:44,682] jepsen worker 715 - jepsen.util 715 :invoke :add 7857 INFO [2022-01-10 08:39:44,684] jepsen worker 33 - jepsen.util 33 :invoke :add 7858 INFO [2022-01-10 08:39:44,685] jepsen worker 605 - jepsen.util 605 :invoke :add 7859 INFO [2022-01-10 08:39:44,688] jepsen worker 202 - jepsen.util 202 :invoke :add 7860 INFO [2022-01-10 08:39:44,690] jepsen worker 584 - jepsen.util 1584 :invoke :add 7861 INFO [2022-01-10 08:39:44,691] jepsen worker 727 - jepsen.util 727 :invoke :add 7862 INFO [2022-01-10 08:39:44,693] jepsen worker 431 - jepsen.util 1431 :invoke :add 7863 INFO [2022-01-10 08:39:44,695] jepsen worker 445 - jepsen.util 1445 :invoke :add 7864 INFO [2022-01-10 08:39:44,801] jepsen worker 762 - jepsen.util 762 :info :add 6657 :RequestTimeoutException INFO [2022-01-10 08:39:44,801] jepsen worker 376 - jepsen.util 376 :info :add 6658 :RequestTimeoutException INFO [2022-01-10 08:39:44,806] jepsen worker 762 - jepsen.util 1762 :invoke :add 7865 INFO [2022-01-10 08:39:44,807] jepsen worker 704 - jepsen.util 704 :info :add 6659 :RequestTimeoutException INFO [2022-01-10 08:39:44,808] jepsen worker 376 - jepsen.util 1376 :invoke :add 7866 INFO [2022-01-10 08:39:44,810] jepsen worker 704 - jepsen.util 1704 :invoke :add 7867 INFO [2022-01-10 08:39:44,811] jepsen worker 635 - jepsen.util 635 :info :add 6660 :RequestTimeoutException INFO [2022-01-10 08:39:44,811] jepsen worker 149 - jepsen.util 149 :info :add 6661 :RequestTimeoutException INFO [2022-01-10 08:39:44,813] jepsen worker 273 - jepsen.util 273 :info :add 6663 :RequestTimeoutException INFO [2022-01-10 08:39:44,814] jepsen worker 149 - jepsen.util 1149 :invoke :add 7868 INFO [2022-01-10 08:39:44,816] jepsen worker 635 - jepsen.util 1635 :invoke :add 7869 INFO [2022-01-10 08:39:44,817] jepsen worker 344 - jepsen.util 344 :info :add 6662 :RequestTimeoutException INFO [2022-01-10 08:39:44,817] jepsen worker 273 - jepsen.util 1273 :invoke :add 7870 INFO [2022-01-10 08:39:44,820] jepsen worker 344 - jepsen.util 1344 :invoke :add 7871 INFO [2022-01-10 08:39:44,833] jepsen worker 996 - jepsen.util 996 :info :add 6668 :RequestTimeoutException INFO [2022-01-10 08:39:44,837] jepsen worker 996 - jepsen.util 1996 :invoke :add 7872 INFO [2022-01-10 08:39:44,996] jepsen worker 815 - jepsen.util 815 :ok :add 7145 INFO [2022-01-10 08:39:44,996] jepsen worker 987 - jepsen.util 987 :ok :add 7169 INFO [2022-01-10 08:39:44,996] jepsen worker 39 - jepsen.util 1039 :ok :add 7388 INFO [2022-01-10 08:39:44,998] jepsen worker 815 - jepsen.util 815 :invoke :add 7873 INFO [2022-01-10 08:39:44,999] jepsen worker 146 - jepsen.util 146 :ok :add 6905 INFO [2022-01-10 08:39:44,999] jepsen worker 819 - jepsen.util 819 :ok :add 7344 INFO [2022-01-10 08:39:45,000] jepsen worker 615 - jepsen.util 1615 :ok :add 7001 INFO [2022-01-10 08:39:45,000] jepsen worker 944 - jepsen.util 944 :ok :add 7035 INFO [2022-01-10 08:39:45,000] jepsen worker 778 - jepsen.util 1778 :ok :add 7370 INFO [2022-01-10 08:39:45,000] jepsen worker 389 - jepsen.util 389 :ok :add 6931 INFO [2022-01-10 08:39:45,000] jepsen worker 838 - jepsen.util 838 :ok :add 6895 INFO [2022-01-10 08:39:45,000] jepsen worker 55 - jepsen.util 55 :ok :add 7436 INFO [2022-01-10 08:39:45,000] jepsen worker 777 - jepsen.util 777 :ok :add 6784 INFO [2022-01-10 08:39:45,000] jepsen worker 792 - jepsen.util 1792 :ok :add 7402 INFO [2022-01-10 08:39:45,001] jepsen worker 987 - jepsen.util 987 :invoke :add 7874 INFO [2022-01-10 08:39:45,002] jepsen worker 394 - jepsen.util 394 :ok :add 6929 INFO [2022-01-10 08:39:45,004] jepsen worker 838 - jepsen.util 838 :invoke :add 7875 INFO [2022-01-10 08:39:45,007] jepsen worker 778 - jepsen.util 1778 :invoke :add 7876 INFO [2022-01-10 08:39:45,008] jepsen worker 819 - jepsen.util 819 :invoke :add 7877 INFO [2022-01-10 08:39:45,010] jepsen worker 777 - jepsen.util 777 :invoke :add 7878 INFO [2022-01-10 08:39:45,012] jepsen worker 39 - jepsen.util 1039 :invoke :add 7879 INFO [2022-01-10 08:39:45,013] jepsen worker 55 - jepsen.util 55 :invoke :add 7880 INFO [2022-01-10 08:39:45,015] jepsen worker 944 - jepsen.util 944 :invoke :add 7881 INFO [2022-01-10 08:39:45,016] jepsen worker 389 - jepsen.util 389 :invoke :add 7882 INFO [2022-01-10 08:39:45,018] jepsen worker 146 - jepsen.util 146 :invoke :add 7883 INFO [2022-01-10 08:39:45,020] jepsen worker 394 - jepsen.util 394 :invoke :add 7884 INFO [2022-01-10 08:39:45,022] jepsen worker 792 - jepsen.util 1792 :invoke :add 7885 INFO [2022-01-10 08:39:45,024] jepsen worker 615 - jepsen.util 1615 :invoke :add 7886 INFO [2022-01-10 08:39:45,151] jepsen worker 32 - jepsen.util 1032 :ok :add 7526 INFO [2022-01-10 08:39:45,153] jepsen worker 423 - jepsen.util 423 :ok :add 6821 INFO [2022-01-10 08:39:45,154] jepsen worker 32 - jepsen.util 1032 :invoke :add 7887 INFO [2022-01-10 08:39:45,154] jepsen worker 82 - jepsen.util 1082 :ok :add 6906 INFO [2022-01-10 08:39:45,154] jepsen worker 521 - jepsen.util 521 :ok :add 6932 INFO [2022-01-10 08:39:45,154] jepsen worker 683 - jepsen.util 683 :ok :add 7435 INFO [2022-01-10 08:39:45,154] jepsen worker 41 - jepsen.util 41 :ok :add 6839 INFO [2022-01-10 08:39:45,155] jepsen worker 509 - jepsen.util 1509 :ok :add 7099 INFO [2022-01-10 08:39:45,155] jepsen worker 451 - jepsen.util 451 :ok :add 7111 INFO [2022-01-10 08:39:45,155] jepsen worker 845 - jepsen.util 1845 :ok :add 7036 INFO [2022-01-10 08:39:45,155] jepsen worker 774 - jepsen.util 774 :ok :add 7347 INFO [2022-01-10 08:39:45,155] jepsen worker 340 - jepsen.util 340 :ok :add 7192 INFO [2022-01-10 08:39:45,156] jepsen worker 304 - jepsen.util 304 :ok :add 7770 INFO [2022-01-10 08:39:45,157] jepsen worker 423 - jepsen.util 423 :invoke :add 7888 INFO [2022-01-10 08:39:45,159] jepsen worker 855 - jepsen.util 855 :ok :add 7144 INFO [2022-01-10 08:39:45,160] jepsen worker 509 - jepsen.util 1509 :invoke :add 7889 INFO [2022-01-10 08:39:45,163] jepsen worker 494 - jepsen.util 494 :ok :add 6765 INFO [2022-01-10 08:39:45,163] jepsen worker 281 - jepsen.util 281 :ok :add 6874 INFO [2022-01-10 08:39:45,163] jepsen worker 576 - jepsen.util 576 :ok :add 7170 INFO [2022-01-10 08:39:45,163] jepsen worker 661 - jepsen.util 1661 :ok :add 7201 INFO [2022-01-10 08:39:45,163] jepsen worker 888 - jepsen.util 1888 :ok :add 7389 INFO [2022-01-10 08:39:45,163] jepsen worker 41 - jepsen.util 41 :invoke :add 7890 INFO [2022-01-10 08:39:45,163] jepsen worker 560 - jepsen.util 1560 :ok :add 7219 INFO [2022-01-10 08:39:45,166] jepsen worker 560 - jepsen.util 1560 :invoke :add 7891 INFO [2022-01-10 08:39:45,167] jepsen worker 346 - jepsen.util 346 :ok :add 6840 INFO [2022-01-10 08:39:45,167] jepsen worker 113 - jepsen.util 1113 :ok :add 7391 INFO [2022-01-10 08:39:45,167] jepsen worker 367 - jepsen.util 367 :ok :add 7547 INFO [2022-01-10 08:39:45,167] jepsen worker 737 - jepsen.util 737 :ok :add 6751 INFO [2022-01-10 08:39:45,168] jepsen worker 451 - jepsen.util 451 :invoke :add 7892 INFO [2022-01-10 08:39:45,171] jepsen worker 576 - jepsen.util 576 :invoke :add 7893 INFO [2022-01-10 08:39:45,172] jepsen worker 510 - jepsen.util 510 :info :add 6670 :RequestTimeoutException INFO [2022-01-10 08:39:45,173] jepsen worker 281 - jepsen.util 281 :invoke :add 7894 INFO [2022-01-10 08:39:45,175] jepsen worker 737 - jepsen.util 737 :invoke :add 7895 INFO [2022-01-10 08:39:45,177] jepsen worker 457 - jepsen.util 457 :ok :add 6749 INFO [2022-01-10 08:39:45,177] jepsen worker 169 - jepsen.util 169 :ok :add 6858 INFO [2022-01-10 08:39:45,177] jepsen worker 57 - jepsen.util 1057 :ok :add 7602 INFO [2022-01-10 08:39:45,177] jepsen worker 383 - jepsen.util 383 :ok :add 7168 INFO [2022-01-10 08:39:45,177] jepsen worker 664 - jepsen.util 1664 :ok :add 7573 INFO [2022-01-10 08:39:45,177] jepsen worker 304 - jepsen.util 304 :invoke :add 7896 INFO [2022-01-10 08:39:45,178] jepsen worker 438 - jepsen.util 438 :info :add 6676 :RequestTimeoutException INFO [2022-01-10 08:39:45,178] jepsen worker 638 - jepsen.util 638 :info :add 6677 :RequestTimeoutException INFO [2022-01-10 08:39:45,178] jepsen worker 844 - jepsen.util 844 :info :add 6697 :RequestTimeoutException INFO [2022-01-10 08:39:45,181] jepsen worker 100 - jepsen.util 100 :info :add 6674 :RequestTimeoutException INFO [2022-01-10 08:39:45,181] jepsen worker 671 - jepsen.util 671 :info :add 6696 :RequestTimeoutException INFO [2022-01-10 08:39:45,181] jepsen worker 676 - jepsen.util 676 :info :add 6698 :RequestTimeoutException INFO [2022-01-10 08:39:45,182] jepsen worker 682 - jepsen.util 682 :info :add 6675 :RequestTimeoutException INFO [2022-01-10 08:39:45,182] jepsen worker 9 - jepsen.util 9 :info :add 6678 :RequestTimeoutException INFO [2022-01-10 08:39:45,182] jepsen worker 616 - jepsen.util 616 :info :add 6679 :RequestTimeoutException INFO [2022-01-10 08:39:45,182] jepsen worker 264 - jepsen.util 264 :info :add 6683 :RequestTimeoutException INFO [2022-01-10 08:39:45,183] jepsen worker 530 - jepsen.util 530 :info :add 6680 :RequestTimeoutException INFO [2022-01-10 08:39:45,183] jepsen worker 229 - jepsen.util 229 :info :add 6681 :RequestTimeoutException INFO [2022-01-10 08:39:45,183] jepsen worker 419 - jepsen.util 419 :info :add 6695 :RequestTimeoutException INFO [2022-01-10 08:39:45,184] jepsen worker 260 - jepsen.util 260 :info :add 6682 :RequestTimeoutException INFO [2022-01-10 08:39:45,184] jepsen worker 74 - jepsen.util 74 :info :add 6694 :RequestTimeoutException INFO [2022-01-10 08:39:45,186] jepsen worker 98 - jepsen.util 98 :ok :add 7437 INFO [2022-01-10 08:39:45,186] jepsen worker 790 - jepsen.util 1790 :ok :add 7345 INFO [2022-01-10 08:39:45,187] jepsen worker 143 - jepsen.util 1143 :ok :add 7734 INFO [2022-01-10 08:39:45,188] jepsen worker 965 - jepsen.util 965 :ok :add 7718 INFO [2022-01-10 08:39:45,188] jepsen worker 338 - jepsen.util 338 :ok :add 6942 INFO [2022-01-10 08:39:45,188] jepsen worker 960 - jepsen.util 960 :ok :add 7471 INFO [2022-01-10 08:39:45,188] jepsen worker 439 - jepsen.util 1439 :ok :add 7445 INFO [2022-01-10 08:39:45,188] jepsen worker 377 - jepsen.util 1377 :ok :add 7293 INFO [2022-01-10 08:39:45,188] jepsen worker 543 - jepsen.util 1543 :ok :add 7337 INFO [2022-01-10 08:39:45,188] jepsen worker 308 - jepsen.util 1308 :ok :add 7746 INFO [2022-01-10 08:39:45,188] jepsen worker 807 - jepsen.util 807 :ok :add 7772 INFO [2022-01-10 08:39:45,188] jepsen worker 715 - jepsen.util 715 :ok :add 7857 INFO [2022-01-10 08:39:45,188] jepsen worker 753 - jepsen.util 1753 :ok :add 7469 INFO [2022-01-10 08:39:45,195] jepsen worker 676 - jepsen.util 1676 :invoke :add 7897 INFO [2022-01-10 08:39:45,197] jepsen worker 98 - jepsen.util 98 :invoke :add 7898 INFO [2022-01-10 08:39:45,199] jepsen worker 377 - jepsen.util 1377 :invoke :add 7899 INFO [2022-01-10 08:39:45,201] jepsen worker 888 - jepsen.util 1888 :invoke :add 7900 INFO [2022-01-10 08:39:45,203] jepsen worker 960 - jepsen.util 960 :invoke :add 7901 INFO [2022-01-10 08:39:45,204] jepsen worker 682 - jepsen.util 1682 :invoke :add 7902 INFO [2022-01-10 08:39:45,206] jepsen worker 774 - jepsen.util 774 :invoke :add 7903 INFO [2022-01-10 08:39:45,208] jepsen worker 439 - jepsen.util 1439 :invoke :add 7904 INFO [2022-01-10 08:39:45,211] jepsen worker 143 - jepsen.util 1143 :invoke :add 7905 INFO [2022-01-10 08:39:45,212] jepsen worker 74 - jepsen.util 1074 :invoke :add 7906 INFO [2022-01-10 08:39:45,215] jepsen worker 855 - jepsen.util 855 :invoke :add 7907 INFO [2022-01-10 08:39:45,218] jepsen worker 661 - jepsen.util 1661 :invoke :add 7908 INFO [2022-01-10 08:39:45,219] jepsen worker 37 - jepsen.util 37 :ok :add 6909 INFO [2022-01-10 08:39:45,219] jepsen worker 274 - jepsen.util 274 :ok :add 7364 INFO [2022-01-10 08:39:45,219] jepsen worker 821 - jepsen.util 821 :ok :add 7039 INFO [2022-01-10 08:39:45,219] jepsen worker 422 - jepsen.util 422 :ok :add 7015 INFO [2022-01-10 08:39:45,219] jepsen worker 825 - jepsen.util 1825 :ok :add 7350 INFO [2022-01-10 08:39:45,219] jepsen worker 964 - jepsen.util 964 :ok :add 6790 INFO [2022-01-10 08:39:45,220] jepsen worker 891 - jepsen.util 891 :ok :add 7348 INFO [2022-01-10 08:39:45,220] jepsen worker 487 - jepsen.util 487 :ok :add 6881 INFO [2022-01-10 08:39:45,220] jepsen worker 125 - jepsen.util 1125 :ok :add 7021 INFO [2022-01-10 08:39:45,220] jepsen worker 383 - jepsen.util 383 :invoke :add 7909 INFO [2022-01-10 08:39:45,220] jepsen worker 565 - jepsen.util 565 :ok :add 6899 INFO [2022-01-10 08:39:45,223] jepsen worker 251 - jepsen.util 1251 :ok :add 6979 INFO [2022-01-10 08:39:45,224] jepsen worker 616 - jepsen.util 1616 :invoke :add 7910 INFO [2022-01-10 08:39:45,224] jepsen worker 705 - jepsen.util 705 :ok :add 6961 INFO [2022-01-10 08:39:45,224] jepsen worker 956 - jepsen.util 1956 :ok :add 7284 INFO [2022-01-10 08:39:45,224] jepsen worker 799 - jepsen.util 1799 :ok :add 6955 INFO [2022-01-10 08:39:45,224] jepsen worker 40 - jepsen.util 1040 :ok :add 7314 INFO [2022-01-10 08:39:45,225] jepsen worker 549 - jepsen.util 549 :ok :add 7065 INFO [2022-01-10 08:39:45,225] jepsen worker 5 - jepsen.util 5 :ok :add 7320 INFO [2022-01-10 08:39:45,225] jepsen worker 283 - jepsen.util 1283 :ok :add 7051 INFO [2022-01-10 08:39:45,226] jepsen worker 494 - jepsen.util 494 :invoke :add 7911 INFO [2022-01-10 08:39:45,229] jepsen worker 487 - jepsen.util 487 :invoke :add 7912 INFO [2022-01-10 08:39:45,231] jepsen worker 422 - jepsen.util 422 :invoke :add 7913 INFO [2022-01-10 08:39:45,232] jepsen worker 530 - jepsen.util 1530 :invoke :add 7914 INFO [2022-01-10 08:39:45,234] jepsen worker 169 - jepsen.util 169 :invoke :add 7915 INFO [2022-01-10 08:39:45,236] jepsen worker 845 - jepsen.util 1845 :invoke :add 7916 INFO [2022-01-10 08:39:45,238] jepsen worker 419 - jepsen.util 1419 :invoke :add 7917 INFO [2022-01-10 08:39:45,239] jepsen worker 790 - jepsen.util 1790 :invoke :add 7918 INFO [2022-01-10 08:39:45,241] jepsen worker 964 - jepsen.util 964 :invoke :add 7919 INFO [2022-01-10 08:39:45,244] jepsen worker 283 - jepsen.util 1283 :invoke :add 7920 INFO [2022-01-10 08:39:45,246] jepsen worker 438 - jepsen.util 1438 :invoke :add 7921 INFO [2022-01-10 08:39:45,247] jepsen worker 113 - jepsen.util 1113 :invoke :add 7922 INFO [2022-01-10 08:39:45,249] jepsen worker 125 - jepsen.util 1125 :invoke :add 7923 INFO [2022-01-10 08:39:45,251] jepsen worker 308 - jepsen.util 1308 :invoke :add 7924 INFO [2022-01-10 08:39:45,253] jepsen worker 664 - jepsen.util 1664 :invoke :add 7925 INFO [2022-01-10 08:39:45,254] jepsen worker 457 - jepsen.util 457 :invoke :add 7926 INFO [2022-01-10 08:39:45,256] jepsen worker 683 - jepsen.util 683 :invoke :add 7927 INFO [2022-01-10 08:39:45,258] jepsen worker 251 - jepsen.util 1251 :invoke :add 7928 INFO [2022-01-10 08:39:45,260] jepsen worker 965 - jepsen.util 965 :invoke :add 7929 INFO [2022-01-10 08:39:45,262] jepsen worker 891 - jepsen.util 891 :invoke :add 7930 INFO [2022-01-10 08:39:45,263] jepsen worker 807 - jepsen.util 807 :invoke :add 7931 INFO [2022-01-10 08:39:45,265] jepsen worker 82 - jepsen.util 1082 :invoke :add 7932 INFO [2022-01-10 08:39:45,267] jepsen worker 510 - jepsen.util 1510 :invoke :add 7933 INFO [2022-01-10 08:39:45,270] jepsen worker 340 - jepsen.util 340 :invoke :add 7934 INFO [2022-01-10 08:39:45,272] jepsen worker 367 - jepsen.util 367 :invoke :add 7935 INFO [2022-01-10 08:39:45,274] jepsen worker 100 - jepsen.util 1100 :invoke :add 7936 INFO [2022-01-10 08:39:45,276] jepsen worker 264 - jepsen.util 1264 :invoke :add 7937 INFO [2022-01-10 08:39:45,277] jepsen worker 274 - jepsen.util 274 :invoke :add 7938 INFO [2022-01-10 08:39:45,279] jepsen worker 37 - jepsen.util 37 :invoke :add 7939 INFO [2022-01-10 08:39:45,281] jepsen worker 715 - jepsen.util 715 :invoke :add 7940 INFO [2022-01-10 08:39:45,282] jepsen worker 260 - jepsen.util 1260 :invoke :add 7941 INFO [2022-01-10 08:39:45,283] jepsen worker 812 - jepsen.util 812 :info :add 6699 :RequestTimeoutException INFO [2022-01-10 08:39:45,285] jepsen worker 346 - jepsen.util 346 :invoke :add 7942 INFO [2022-01-10 08:39:45,287] jepsen worker 638 - jepsen.util 1638 :invoke :add 7943 INFO [2022-01-10 08:39:45,288] jepsen worker 799 - jepsen.util 1799 :invoke :add 7944 INFO [2022-01-10 08:39:45,290] jepsen worker 40 - jepsen.util 1040 :invoke :add 7945 INFO [2022-01-10 08:39:45,291] jepsen worker 31 - jepsen.util 31 :info :add 6702 :RequestTimeoutException INFO [2022-01-10 08:39:45,291] jepsen worker 9 - jepsen.util 1009 :invoke :add 7946 INFO [2022-01-10 08:39:45,293] jepsen worker 384 - jepsen.util 384 :info :add 6704 :RequestTimeoutException INFO [2022-01-10 08:39:45,293] jepsen worker 541 - jepsen.util 541 :info :add 6706 :RequestTimeoutException INFO [2022-01-10 08:39:45,294] jepsen worker 31 - jepsen.util 1031 :invoke :add 7947 INFO [2022-01-10 08:39:45,297] jepsen worker 5 - jepsen.util 5 :invoke :add 7948 INFO [2022-01-10 08:39:45,297] jepsen worker 742 - jepsen.util 742 :info :add 6705 :RequestTimeoutException INFO [2022-01-10 08:39:45,300] jepsen worker 742 - jepsen.util 1742 :invoke :add 7949 INFO [2022-01-10 08:39:45,301] jepsen worker 384 - jepsen.util 1384 :invoke :add 7950 INFO [2022-01-10 08:39:45,302] jepsen worker 72 - jepsen.util 72 :info :add 6709 :RequestTimeoutException INFO [2022-01-10 08:39:45,303] jepsen worker 353 - jepsen.util 353 :info :add 6707 :RequestTimeoutException INFO [2022-01-10 08:39:45,303] jepsen worker 261 - jepsen.util 261 :info :add 6708 :RequestTimeoutException INFO [2022-01-10 08:39:45,304] jepsen worker 557 - jepsen.util 557 :info :add 6710 :RequestTimeoutException INFO [2022-01-10 08:39:45,304] jepsen worker 632 - jepsen.util 632 :info :add 6711 :RequestTimeoutException INFO [2022-01-10 08:39:45,304] jepsen worker 543 - jepsen.util 1543 :invoke :add 7951 INFO [2022-01-10 08:39:45,308] jepsen worker 236 - jepsen.util 236 :ok :add 6834 INFO [2022-01-10 08:39:45,309] jepsen worker 956 - jepsen.util 1956 :invoke :add 7952 INFO [2022-01-10 08:39:45,310] jepsen worker 658 - jepsen.util 658 :ok :add 7489 INFO [2022-01-10 08:39:45,311] jepsen worker 554 - jepsen.util 1554 :ok :add 7092 INFO [2022-01-10 08:39:45,311] jepsen worker 141 - jepsen.util 1141 :ok :add 6996 INFO [2022-01-10 08:39:45,311] jepsen worker 690 - jepsen.util 690 :ok :add 7491 INFO [2022-01-10 08:39:45,311] jepsen worker 972 - jepsen.util 972 :info :add 6712 :RequestTimeoutException INFO [2022-01-10 08:39:45,311] jepsen worker 875 - jepsen.util 1875 :ok :add 7273 INFO [2022-01-10 08:39:45,311] jepsen worker 46 - jepsen.util 46 :info :add 6713 :RequestTimeoutException INFO [2022-01-10 08:39:45,313] jepsen worker 57 - jepsen.util 1057 :invoke :add 7953 INFO [2022-01-10 08:39:45,317] jepsen worker 629 - jepsen.util 629 :ok :add 6940 INFO [2022-01-10 08:39:45,317] jepsen worker 107 - jepsen.util 1107 :ok :add 7289 INFO [2022-01-10 08:39:45,317] jepsen worker 407 - jepsen.util 1407 :ok :add 7301 INFO [2022-01-10 08:39:45,317] jepsen worker 865 - jepsen.util 1865 :ok :add 7473 INFO [2022-01-10 08:39:45,317] jepsen worker 684 - jepsen.util 1684 :ok :add 7447 INFO [2022-01-10 08:39:45,317] jepsen worker 327 - jepsen.util 1327 :ok :add 7319 INFO [2022-01-10 08:39:45,317] jepsen worker 731 - jepsen.util 1731 :ok :add 7291 INFO [2022-01-10 08:39:45,317] jepsen worker 611 - jepsen.util 1611 :ok :add 7335 INFO [2022-01-10 08:39:45,321] jepsen worker 373 - jepsen.util 373 :info :add 6718 :RequestTimeoutException INFO [2022-01-10 08:39:45,322] jepsen worker 684 - jepsen.util 1684 :invoke :add 7954 INFO [2022-01-10 08:39:45,325] jepsen worker 825 - jepsen.util 1825 :invoke :add 7955 INFO [2022-01-10 08:39:45,326] jepsen worker 88 - jepsen.util 88 :ok :add 7493 INFO [2022-01-10 08:39:45,326] jepsen worker 20 - jepsen.util 1020 :ok :add 7503 INFO [2022-01-10 08:39:45,326] jepsen worker 874 - jepsen.util 1874 :ok :add 7271 INFO [2022-01-10 08:39:45,326] jepsen worker 298 - jepsen.util 1298 :ok :add 7646 INFO [2022-01-10 08:39:45,326] jepsen worker 157 - jepsen.util 157 :ok :add 7088 INFO [2022-01-10 08:39:45,326] jepsen worker 97 - jepsen.util 1097 :ok :add 7245 INFO [2022-01-10 08:39:45,329] jepsen worker 865 - jepsen.util 1865 :invoke :add 7956 INFO [2022-01-10 08:39:45,332] jepsen worker 690 - jepsen.util 690 :invoke :add 7957 INFO [2022-01-10 08:39:45,335] jepsen worker 463 - jepsen.util 1463 :ok :add 7444 INFO [2022-01-10 08:39:45,335] jepsen worker 356 - jepsen.util 1356 :ok :add 7470 INFO [2022-01-10 08:39:45,335] jepsen worker 343 - jepsen.util 1343 :ok :add 7689 INFO [2022-01-10 08:39:45,335] jepsen worker 228 - jepsen.util 1228 :ok :add 7810 INFO [2022-01-10 08:39:45,335] jepsen worker 211 - jepsen.util 211 :ok :add 7808 INFO [2022-01-10 08:39:45,336] jepsen worker 874 - jepsen.util 1874 :invoke :add 7958 INFO [2022-01-10 08:39:45,336] jepsen worker 915 - jepsen.util 915 :ok :add 7518 INFO [2022-01-10 08:39:45,336] jepsen worker 287 - jepsen.util 1287 :ok :add 7246 INFO [2022-01-10 08:39:45,336] jepsen worker 561 - jepsen.util 561 :ok :add 7490 INFO [2022-01-10 08:39:45,336] jepsen worker 974 - jepsen.util 1974 :ok :add 7488 INFO [2022-01-10 08:39:45,336] jepsen worker 295 - jepsen.util 1295 :ok :add 7137 INFO [2022-01-10 08:39:45,336] jepsen worker 755 - jepsen.util 755 :ok :add 6807 INFO [2022-01-10 08:39:45,337] jepsen worker 937 - jepsen.util 1937 :ok :add 7103 INFO [2022-01-10 08:39:45,342] jepsen worker 575 - jepsen.util 1575 :ok :add 7671 INFO [2022-01-10 08:39:45,343] jepsen worker 549 - jepsen.util 549 :invoke :add 7959 INFO [2022-01-10 08:39:45,348] jepsen worker 327 - jepsen.util 1327 :invoke :add 7960 INFO [2022-01-10 08:39:45,351] jepsen worker 937 - jepsen.util 1937 :invoke :add 7961 INFO [2022-01-10 08:39:45,353] jepsen worker 287 - jepsen.util 1287 :invoke :add 7962 INFO [2022-01-10 08:39:45,355] jepsen worker 844 - jepsen.util 1844 :invoke :add 7963 INFO [2022-01-10 08:39:45,358] jepsen worker 88 - jepsen.util 88 :invoke :add 7964 INFO [2022-01-10 08:39:45,360] jepsen worker 561 - jepsen.util 561 :invoke :add 7965 INFO [2022-01-10 08:39:45,362] jepsen worker 463 - jepsen.util 1463 :invoke :add 7966 INFO [2022-01-10 08:39:45,363] jepsen worker 565 - jepsen.util 565 :invoke :add 7967 INFO [2022-01-10 08:39:45,366] jepsen worker 20 - jepsen.util 1020 :invoke :add 7968 INFO [2022-01-10 08:39:45,367] jepsen worker 72 - jepsen.util 1072 :invoke :add 7969 INFO [2022-01-10 08:39:45,370] jepsen worker 97 - jepsen.util 1097 :invoke :add 7970 INFO [2022-01-10 08:39:45,372] jepsen worker 107 - jepsen.util 1107 :invoke :add 7971 INFO [2022-01-10 08:39:45,374] jepsen worker 611 - jepsen.util 1611 :invoke :add 7972 INFO [2022-01-10 08:39:45,376] jepsen worker 141 - jepsen.util 1141 :invoke :add 7973 INFO [2022-01-10 08:39:45,378] jepsen worker 298 - jepsen.util 1298 :invoke :add 7974 INFO [2022-01-10 08:39:45,380] jepsen worker 343 - jepsen.util 1343 :invoke :add 7975 INFO [2022-01-10 08:39:45,382] jepsen worker 541 - jepsen.util 1541 :invoke :add 7976 INFO [2022-01-10 08:39:45,384] jepsen worker 821 - jepsen.util 821 :invoke :add 7977 INFO [2022-01-10 08:39:45,386] jepsen worker 557 - jepsen.util 1557 :invoke :add 7978 INFO [2022-01-10 08:39:45,388] jepsen worker 373 - jepsen.util 1373 :invoke :add 7979 INFO [2022-01-10 08:39:45,389] jepsen worker 356 - jepsen.util 1356 :invoke :add 7980 INFO [2022-01-10 08:39:45,391] jepsen worker 295 - jepsen.util 1295 :invoke :add 7981 INFO [2022-01-10 08:39:45,393] jepsen worker 705 - jepsen.util 705 :invoke :add 7982 INFO [2022-01-10 08:39:45,395] jepsen worker 915 - jepsen.util 915 :invoke :add 7983 INFO [2022-01-10 08:39:45,397] jepsen worker 157 - jepsen.util 157 :invoke :add 7984 INFO [2022-01-10 08:39:45,399] jepsen worker 753 - jepsen.util 1753 :invoke :add 7985 INFO [2022-01-10 08:39:45,401] jepsen worker 46 - jepsen.util 1046 :invoke :add 7986 INFO [2022-01-10 08:39:45,402] jepsen worker 731 - jepsen.util 1731 :invoke :add 7987 INFO [2022-01-10 08:39:45,404] jepsen worker 521 - jepsen.util 521 :invoke :add 7988 INFO [2022-01-10 08:39:45,405] jepsen worker 812 - jepsen.util 1812 :invoke :add 7989 INFO [2022-01-10 08:39:45,407] jepsen worker 407 - jepsen.util 1407 :invoke :add 7990 INFO [2022-01-10 08:39:45,409] jepsen worker 974 - jepsen.util 1974 :invoke :add 7991 INFO [2022-01-10 08:39:45,412] jepsen worker 229 - jepsen.util 1229 :invoke :add 7992 INFO [2022-01-10 08:39:45,413] jepsen worker 338 - jepsen.util 338 :invoke :add 7993 INFO [2022-01-10 08:39:45,415] jepsen worker 755 - jepsen.util 755 :invoke :add 7994 INFO [2022-01-10 08:39:45,415] jepsen worker 45 - jepsen.util 1045 :ok :add 7502 INFO [2022-01-10 08:39:45,415] jepsen worker 375 - jepsen.util 1375 :ok :add 7091 INFO [2022-01-10 08:39:45,415] jepsen worker 697 - jepsen.util 1697 :ok :add 7492 INFO [2022-01-10 08:39:45,415] jepsen worker 247 - jepsen.util 247 :ok :add 7536 INFO [2022-01-10 08:39:45,417] jepsen worker 261 - jepsen.util 1261 :invoke :add 7995 INFO [2022-01-10 08:39:45,420] jepsen worker 972 - jepsen.util 1972 :invoke :add 7996 INFO [2022-01-10 08:39:45,422] jepsen worker 875 - jepsen.util 1875 :invoke :add 7997 INFO [2022-01-10 08:39:45,424] jepsen worker 697 - jepsen.util 1697 :invoke :add 7998 INFO [2022-01-10 08:39:45,426] jepsen worker 236 - jepsen.util 236 :invoke :add 7999 INFO [2022-01-10 08:39:45,428] jepsen worker 211 - jepsen.util 211 :invoke :add 8000 INFO [2022-01-10 08:39:45,430] jepsen worker 375 - jepsen.util 1375 :invoke :add 8001 INFO [2022-01-10 08:39:45,432] jepsen worker 554 - jepsen.util 1554 :invoke :add 8002 INFO [2022-01-10 08:39:45,434] jepsen worker 45 - jepsen.util 1045 :invoke :add 8003 INFO [2022-01-10 08:39:45,435] jepsen worker 353 - jepsen.util 1353 :invoke :add 8004 INFO [2022-01-10 08:39:45,438] jepsen worker 575 - jepsen.util 1575 :invoke :add 8005 INFO [2022-01-10 08:39:45,440] jepsen worker 247 - jepsen.util 247 :invoke :add 8006 INFO [2022-01-10 08:39:45,442] jepsen worker 629 - jepsen.util 629 :invoke :add 8007 INFO [2022-01-10 08:39:45,444] jepsen worker 658 - jepsen.util 658 :invoke :add 8008 INFO [2022-01-10 08:39:45,446] jepsen worker 671 - jepsen.util 1671 :invoke :add 8009 INFO [2022-01-10 08:39:45,448] jepsen worker 632 - jepsen.util 1632 :invoke :add 8010 INFO [2022-01-10 08:39:45,449] jepsen worker 228 - jepsen.util 1228 :invoke :add 8011 INFO [2022-01-10 08:39:45,456] jepsen worker 769 - jepsen.util 1769 :ok :add 7193 INFO [2022-01-10 08:39:45,456] jepsen worker 397 - jepsen.util 1397 :ok :add 7372 INFO [2022-01-10 08:39:45,458] jepsen worker 769 - jepsen.util 1769 :invoke :add 8012 INFO [2022-01-10 08:39:45,459] jepsen worker 669 - jepsen.util 669 :ok :add 7812 INFO [2022-01-10 08:39:45,460] jepsen worker 425 - jepsen.util 1425 :ok :add 7826 INFO [2022-01-10 08:39:45,460] jepsen worker 856 - jepsen.util 1856 :ok :add 7288 INFO [2022-01-10 08:39:45,460] jepsen worker 429 - jepsen.util 1429 :ok :add 7737 INFO [2022-01-10 08:39:45,460] jepsen worker 397 - jepsen.util 1397 :invoke :add 8013 INFO [2022-01-10 08:39:45,460] jepsen worker 652 - jepsen.util 1652 :ok :add 7300 INFO [2022-01-10 08:39:45,463] jepsen worker 425 - jepsen.util 1425 :invoke :add 8014 INFO [2022-01-10 08:39:45,464] jepsen worker 652 - jepsen.util 1652 :invoke :add 8015 INFO [2022-01-10 08:39:45,466] jepsen worker 856 - jepsen.util 1856 :invoke :add 8016 INFO [2022-01-10 08:39:45,468] jepsen worker 429 - jepsen.util 1429 :invoke :add 8017 INFO [2022-01-10 08:39:45,470] jepsen worker 669 - jepsen.util 669 :invoke :add 8018 INFO [2022-01-10 08:39:45,511] jepsen worker 637 - jepsen.util 637 :info :add 6724 :RequestTimeoutException INFO [2022-01-10 08:39:45,511] jepsen worker 294 - jepsen.util 294 :info :add 6725 :RequestTimeoutException INFO [2022-01-10 08:39:45,512] jepsen worker 357 - jepsen.util 357 :ok :add 7189 INFO [2022-01-10 08:39:45,512] jepsen worker 809 - jepsen.util 809 :ok :add 7191 INFO [2022-01-10 08:39:45,512] jepsen worker 727 - jepsen.util 727 :ok :add 7862 INFO [2022-01-10 08:39:45,512] jepsen worker 901 - jepsen.util 901 :ok :add 7368 INFO [2022-01-10 08:39:45,515] jepsen worker 637 - jepsen.util 1637 :invoke :add 8019 INFO [2022-01-10 08:39:45,516] jepsen worker 102 - jepsen.util 102 :ok :add 7544 INFO [2022-01-10 08:39:45,516] jepsen worker 308 - jepsen.util 1308 :ok :add 7924 INFO [2022-01-10 08:39:45,516] jepsen worker 532 - jepsen.util 1532 :ok :add 7568 INFO [2022-01-10 08:39:45,516] jepsen worker 98 - jepsen.util 98 :ok :add 7898 INFO [2022-01-10 08:39:45,516] jepsen worker 809 - jepsen.util 809 :invoke :add 8020 INFO [2022-01-10 08:39:45,516] jepsen worker 55 - jepsen.util 55 :ok :add 7880 INFO [2022-01-10 08:39:45,517] jepsen worker 242 - jepsen.util 1242 :ok :add 7601 INFO [2022-01-10 08:39:45,517] jepsen worker 519 - jepsen.util 1519 :ok :add 7619 INFO [2022-01-10 08:39:45,517] jepsen worker 466 - jepsen.util 1466 :ok :add 7570 INFO [2022-01-10 08:39:45,517] jepsen worker 275 - jepsen.util 275 :ok :add 7635 INFO [2022-01-10 08:39:45,517] jepsen worker 489 - jepsen.util 489 :info :add 6726 :RequestTimeoutException INFO [2022-01-10 08:39:45,518] jepsen worker 285 - jepsen.util 285 :info :add 6727 :RequestTimeoutException INFO [2022-01-10 08:39:45,521] jepsen worker 830 - jepsen.util 830 :info :add 6728 :RequestTimeoutException INFO [2022-01-10 08:39:45,522] jepsen worker 55 - jepsen.util 55 :invoke :add 8021 INFO [2022-01-10 08:39:45,522] jepsen worker 957 - jepsen.util 1957 :ok :add 7235 INFO [2022-01-10 08:39:45,522] jepsen worker 630 - jepsen.util 630 :ok :add 7636 INFO [2022-01-10 08:39:45,522] jepsen worker 771 - jepsen.util 771 :info :add 6729 :RequestTimeoutException INFO [2022-01-10 08:39:45,522] jepsen worker 718 - jepsen.util 718 :info :add 6732 :RequestTimeoutException INFO [2022-01-10 08:39:45,523] jepsen worker 138 - jepsen.util 138 :info :add 6730 :RequestTimeoutException INFO [2022-01-10 08:39:45,525] jepsen worker 487 - jepsen.util 487 :ok :add 7912 INFO [2022-01-10 08:39:45,525] jepsen worker 553 - jepsen.util 1553 :ok :add 7390 INFO [2022-01-10 08:39:45,525] jepsen worker 3 - jepsen.util 1003 :ok :add 7572 INFO [2022-01-10 08:39:45,526] jepsen worker 213 - jepsen.util 213 :ok :add 7546 INFO [2022-01-10 08:39:45,526] jepsen worker 943 - jepsen.util 1943 :ok :add 7603 INFO [2022-01-10 08:39:45,526] jepsen worker 361 - jepsen.util 1361 :ok :add 7793 INFO [2022-01-10 08:39:45,526] jepsen worker 872 - jepsen.util 872 :ok :add 7637 INFO [2022-01-10 08:39:45,526] jepsen worker 457 - jepsen.util 457 :ok :add 7926 INFO [2022-01-10 08:39:45,527] jepsen worker 242 - jepsen.util 1242 :invoke :add 8022 INFO [2022-01-10 08:39:45,530] jepsen worker 487 - jepsen.util 487 :invoke :add 8023 INFO [2022-01-10 08:39:45,530] jepsen worker 77 - jepsen.util 77 :info :add 6731 :RequestTimeoutException INFO [2022-01-10 08:39:45,532] jepsen worker 275 - jepsen.util 275 :invoke :add 8024 INFO [2022-01-10 08:39:45,533] jepsen worker 326 - jepsen.util 326 :info :add 6733 :RequestTimeoutException INFO [2022-01-10 08:39:45,533] jepsen worker 601 - jepsen.util 601 :info :add 6735 :RequestTimeoutException INFO [2022-01-10 08:39:45,537] jepsen worker 138 - jepsen.util 1138 :invoke :add 8025 INFO [2022-01-10 08:39:45,539] jepsen worker 466 - jepsen.util 1466 :invoke :add 8026 INFO [2022-01-10 08:39:45,540] jepsen worker 213 - jepsen.util 213 :invoke :add 8027 INFO [2022-01-10 08:39:45,542] jepsen worker 574 - jepsen.util 574 :info :add 6740 :RequestTimeoutException INFO [2022-01-10 08:39:45,543] jepsen worker 452 - jepsen.util 452 :info :add 6742 :RequestTimeoutException INFO [2022-01-10 08:39:45,543] jepsen worker 643 - jepsen.util 1643 :ok :add 7593 INFO [2022-01-10 08:39:45,543] jepsen worker 247 - jepsen.util 247 :ok :add 8006 INFO [2022-01-10 08:39:45,543] jepsen worker 727 - jepsen.util 727 :invoke :add 8028 INFO [2022-01-10 08:39:45,544] jepsen worker 544 - jepsen.util 544 :info :add 6741 :RequestTimeoutException INFO [2022-01-10 08:39:45,548] jepsen worker 830 - jepsen.util 1830 :invoke :add 8029 INFO [2022-01-10 08:39:45,550] jepsen worker 3 - jepsen.util 1003 :invoke :add 8030 INFO [2022-01-10 08:39:45,552] jepsen worker 553 - jepsen.util 1553 :invoke :add 8031 INFO [2022-01-10 08:39:45,552] jepsen worker 969 - jepsen.util 969 :info :add 6743 :RequestTimeoutException INFO [2022-01-10 08:39:45,555] jepsen worker 357 - jepsen.util 357 :invoke :add 8032 INFO [2022-01-10 08:39:45,557] jepsen worker 969 - jepsen.util 1969 :invoke :add 8033 INFO [2022-01-10 08:39:45,558] jepsen worker 308 - jepsen.util 1308 :invoke :add 8034 INFO [2022-01-10 08:39:45,560] jepsen worker 957 - jepsen.util 1957 :invoke :add 8035 INFO [2022-01-10 08:39:45,561] jepsen worker 993 - jepsen.util 993 :info :add 6746 :RequestTimeoutException INFO [2022-01-10 08:39:45,562] jepsen worker 98 - jepsen.util 98 :invoke :add 8036 INFO [2022-01-10 08:39:45,563] jepsen worker 378 - jepsen.util 378 :info :add 6745 :RequestTimeoutException INFO [2022-01-10 08:39:45,566] jepsen worker 993 - jepsen.util 1993 :invoke :add 8037 INFO [2022-01-10 08:39:45,568] jepsen worker 643 - jepsen.util 1643 :invoke :add 8038 INFO [2022-01-10 08:39:45,569] jepsen worker 872 - jepsen.util 872 :invoke :add 8039 INFO [2022-01-10 08:39:45,571] jepsen worker 519 - jepsen.util 1519 :invoke :add 8040 INFO [2022-01-10 08:39:45,573] jepsen worker 247 - jepsen.util 247 :invoke :add 8041 INFO [2022-01-10 08:39:45,574] jepsen worker 937 - jepsen.util 1937 :ok :add 7961 INFO [2022-01-10 08:39:45,574] jepsen worker 887 - jepsen.util 1887 :ok :add 7501 INFO [2022-01-10 08:39:45,575] jepsen worker 7 - jepsen.util 7 :ok :add 7519 INFO [2022-01-10 08:39:45,575] jepsen worker 326 - jepsen.util 1326 :invoke :add 8042 INFO [2022-01-10 08:39:45,577] jepsen worker 983 - jepsen.util 983 :info :add 6754 :RequestTimeoutException INFO [2022-01-10 08:39:45,577] jepsen worker 532 - jepsen.util 1532 :invoke :add 8043 INFO [2022-01-10 08:39:45,580] jepsen worker 601 - jepsen.util 1601 :invoke :add 8044 INFO [2022-01-10 08:39:45,581] jepsen worker 59 - jepsen.util 59 :info :add 6759 :RequestTimeoutException INFO [2022-01-10 08:39:45,582] jepsen worker 489 - jepsen.util 1489 :invoke :add 8045 INFO [2022-01-10 08:39:45,583] jepsen worker 527 - jepsen.util 527 :info :add 6758 :RequestTimeoutException INFO [2022-01-10 08:39:45,584] jepsen worker 893 - jepsen.util 893 :info :add 6755 :RequestTimeoutException INFO [2022-01-10 08:39:45,584] jepsen worker 649 - jepsen.util 649 :info :add 6756 :RequestTimeoutException INFO [2022-01-10 08:39:45,587] jepsen worker 893 - jepsen.util 1893 :invoke :add 8046 INFO [2022-01-10 08:39:45,587] jepsen worker 315 - jepsen.util 315 :info :add 6760 :RequestTimeoutException INFO [2022-01-10 08:39:45,589] jepsen worker 102 - jepsen.util 102 :invoke :add 8047 INFO [2022-01-10 08:39:45,590] jepsen worker 190 - jepsen.util 190 :info :add 6763 :RequestTimeoutException INFO [2022-01-10 08:39:45,590] jepsen worker 468 - jepsen.util 468 :info :add 6761 :RequestTimeoutException INFO [2022-01-10 08:39:45,592] jepsen worker 760 - jepsen.util 760 :info :add 6762 :RequestTimeoutException INFO [2022-01-10 08:39:45,594] jepsen worker 527 - jepsen.util 1527 :invoke :add 8048 INFO [2022-01-10 08:39:45,597] jepsen worker 574 - jepsen.util 1574 :invoke :add 8049 INFO [2022-01-10 08:39:45,597] jepsen worker 83 - jepsen.util 83 :info :add 6768 :RequestTimeoutException INFO [2022-01-10 08:39:45,598] jepsen worker 77 - jepsen.util 1077 :invoke :add 8050 INFO [2022-01-10 08:39:45,601] jepsen worker 345 - jepsen.util 345 :info :add 6770 :RequestTimeoutException INFO [2022-01-10 08:39:45,601] jepsen worker 923 - jepsen.util 923 :info :add 6771 :RequestTimeoutException INFO [2022-01-10 08:39:45,601] jepsen worker 544 - jepsen.util 1544 :invoke :add 8051 INFO [2022-01-10 08:39:45,604] jepsen worker 378 - jepsen.util 1378 :invoke :add 8052 INFO [2022-01-10 08:39:45,606] jepsen worker 901 - jepsen.util 901 :invoke :add 8053 INFO [2022-01-10 08:39:45,607] jepsen worker 898 - jepsen.util 898 :info :add 6774 :RequestTimeoutException INFO [2022-01-10 08:39:45,608] jepsen worker 649 - jepsen.util 1649 :invoke :add 8054 INFO [2022-01-10 08:39:45,611] jepsen worker 937 - jepsen.util 1937 :invoke :add 8055 INFO [2022-01-10 08:39:45,613] jepsen worker 760 - jepsen.util 1760 :invoke :add 8056 INFO [2022-01-10 08:39:45,615] jepsen worker 923 - jepsen.util 1923 :invoke :add 8057 INFO [2022-01-10 08:39:45,617] jepsen worker 457 - jepsen.util 457 :invoke :add 8058 INFO [2022-01-10 08:39:45,619] jepsen worker 887 - jepsen.util 1887 :invoke :add 8059 INFO [2022-01-10 08:39:45,620] jepsen worker 294 - jepsen.util 1294 :invoke :add 8060 INFO [2022-01-10 08:39:45,623] jepsen worker 83 - jepsen.util 1083 :invoke :add 8061 INFO [2022-01-10 08:39:45,625] jepsen worker 285 - jepsen.util 1285 :invoke :add 8062 INFO [2022-01-10 08:39:45,627] jepsen worker 361 - jepsen.util 1361 :invoke :add 8063 INFO [2022-01-10 08:39:45,628] jepsen worker 315 - jepsen.util 1315 :invoke :add 8064 INFO [2022-01-10 08:39:45,630] jepsen worker 452 - jepsen.util 1452 :invoke :add 8065 INFO [2022-01-10 08:39:45,632] jepsen worker 345 - jepsen.util 1345 :invoke :add 8066 INFO [2022-01-10 08:39:45,634] jepsen worker 983 - jepsen.util 1983 :invoke :add 8067 INFO [2022-01-10 08:39:45,635] jepsen worker 898 - jepsen.util 1898 :invoke :add 8068 INFO [2022-01-10 08:39:45,637] jepsen worker 630 - jepsen.util 630 :invoke :add 8069 INFO [2022-01-10 08:39:45,639] jepsen worker 190 - jepsen.util 1190 :invoke :add 8070 INFO [2022-01-10 08:39:45,640] jepsen worker 943 - jepsen.util 1943 :invoke :add 8071 INFO [2022-01-10 08:39:45,642] jepsen worker 468 - jepsen.util 1468 :invoke :add 8072 INFO [2022-01-10 08:39:45,644] jepsen worker 7 - jepsen.util 7 :invoke :add 8073 INFO [2022-01-10 08:39:45,645] jepsen worker 59 - jepsen.util 1059 :invoke :add 8074 INFO [2022-01-10 08:39:45,647] jepsen worker 718 - jepsen.util 1718 :invoke :add 8075 INFO [2022-01-10 08:39:45,649] jepsen worker 771 - jepsen.util 1771 :invoke :add 8076 INFO [2022-01-10 08:39:45,674] jepsen worker 844 - jepsen.util 1844 :ok :add 7963 INFO [2022-01-10 08:39:45,674] jepsen worker 606 - jepsen.util 606 :ok :add 7537 INFO [2022-01-10 08:39:45,676] jepsen worker 844 - jepsen.util 1844 :invoke :add 8077 INFO [2022-01-10 08:39:45,677] jepsen worker 132 - jepsen.util 132 :ok :add 7534 INFO [2022-01-10 08:39:45,677] jepsen worker 557 - jepsen.util 1557 :ok :add 7978 INFO [2022-01-10 08:39:45,677] jepsen worker 684 - jepsen.util 1684 :ok :add 7954 INFO [2022-01-10 08:39:45,678] jepsen worker 606 - jepsen.util 606 :invoke :add 8078 INFO [2022-01-10 08:39:45,680] jepsen worker 697 - jepsen.util 1697 :ok :add 7998 INFO [2022-01-10 08:39:45,680] jepsen worker 247 - jepsen.util 247 :ok :add 8041 INFO [2022-01-10 08:39:45,681] jepsen worker 132 - jepsen.util 132 :invoke :add 8079 INFO [2022-01-10 08:39:45,683] jepsen worker 247 - jepsen.util 247 :invoke :add 8080 INFO [2022-01-10 08:39:45,685] jepsen worker 684 - jepsen.util 1684 :invoke :add 8081 INFO [2022-01-10 08:39:45,687] jepsen worker 697 - jepsen.util 1697 :invoke :add 8082 INFO [2022-01-10 08:39:45,687] jepsen worker 844 - jepsen.util 1844 :ok :add 8077 INFO [2022-01-10 08:39:45,688] jepsen worker 557 - jepsen.util 1557 :invoke :add 8083 INFO [2022-01-10 08:39:45,690] jepsen worker 844 - jepsen.util 1844 :invoke :add 8084 INFO [2022-01-10 08:39:45,697] jepsen worker 575 - jepsen.util 1575 :ok :add 8005 INFO [2022-01-10 08:39:45,697] jepsen worker 553 - jepsen.util 1553 :ok :add 8031 INFO [2022-01-10 08:39:45,697] jepsen worker 830 - jepsen.util 1830 :ok :add 8029 INFO [2022-01-10 08:39:45,700] jepsen worker 830 - jepsen.util 1830 :invoke :add 8085 INFO [2022-01-10 08:39:45,701] jepsen worker 553 - jepsen.util 1553 :invoke :add 8086 INFO [2022-01-10 08:39:45,703] jepsen worker 575 - jepsen.util 1575 :invoke :add 8087 INFO [2022-01-10 08:39:45,765] jepsen worker 357 - jepsen.util 357 :ok :add 8032 INFO [2022-01-10 08:39:45,766] jepsen worker 357 - jepsen.util 357 :invoke :add 8088 INFO [2022-01-10 08:39:45,781] jepsen worker 771 - jepsen.util 1771 :ok :add 8076 INFO [2022-01-10 08:39:45,781] jepsen worker 326 - jepsen.util 1326 :ok :add 8042 INFO [2022-01-10 08:39:45,784] jepsen worker 771 - jepsen.util 1771 :invoke :add 8089 INFO [2022-01-10 08:39:45,786] jepsen worker 326 - jepsen.util 1326 :invoke :add 8090 INFO [2022-01-10 08:39:45,823] jepsen worker 449 - jepsen.util 449 :info :add 6775 :RequestTimeoutException INFO [2022-01-10 08:39:45,826] jepsen worker 449 - jepsen.util 1449 :invoke :add 8091 INFO [2022-01-10 08:39:45,831] jepsen worker 417 - jepsen.util 417 :info :add 6776 :RequestTimeoutException INFO [2022-01-10 08:39:45,832] jepsen worker 483 - jepsen.util 483 :info :add 6779 :RequestTimeoutException INFO [2022-01-10 08:39:45,832] jepsen worker 171 - jepsen.util 171 :info :add 6780 :RequestTimeoutException INFO [2022-01-10 08:39:45,833] jepsen worker 417 - jepsen.util 1417 :invoke :add 8092 INFO [2022-01-10 08:39:45,837] jepsen worker 171 - jepsen.util 1171 :invoke :add 8093 INFO [2022-01-10 08:39:45,837] jepsen worker 787 - jepsen.util 787 :info :add 6781 :RequestTimeoutException INFO [2022-01-10 08:39:45,838] jepsen worker 446 - jepsen.util 446 :info :add 6782 :RequestTimeoutException INFO [2022-01-10 08:39:45,838] jepsen worker 483 - jepsen.util 1483 :invoke :add 8094 INFO [2022-01-10 08:39:45,841] jepsen worker 581 - jepsen.util 581 :info :add 6783 :RequestTimeoutException INFO [2022-01-10 08:39:45,841] jepsen worker 787 - jepsen.util 1787 :invoke :add 8095 INFO [2022-01-10 08:39:45,843] jepsen worker 446 - jepsen.util 1446 :invoke :add 8096 INFO [2022-01-10 08:39:45,845] jepsen worker 581 - jepsen.util 1581 :invoke :add 8097 INFO [2022-01-10 08:39:45,853] jepsen worker 166 - jepsen.util 166 :info :add 6788 :RequestTimeoutException INFO [2022-01-10 08:39:45,853] jepsen worker 738 - jepsen.util 738 :info :add 6791 :RequestTimeoutException INFO [2022-01-10 08:39:45,856] jepsen worker 166 - jepsen.util 1166 :invoke :add 8098 INFO [2022-01-10 08:39:45,858] jepsen worker 738 - jepsen.util 1738 :invoke :add 8099 INFO [2022-01-10 08:39:45,861] jepsen worker 71 - jepsen.util 71 :info :add 6792 :RequestTimeoutException INFO [2022-01-10 08:39:45,864] jepsen worker 665 - jepsen.util 665 :info :add 6795 :RequestTimeoutException INFO [2022-01-10 08:39:45,864] jepsen worker 144 - jepsen.util 144 :info :add 6796 :RequestTimeoutException INFO [2022-01-10 08:39:45,866] jepsen worker 71 - jepsen.util 1071 :invoke :add 8100 INFO [2022-01-10 08:39:45,867] jepsen worker 913 - jepsen.util 913 :info :add 6794 :RequestTimeoutException INFO [2022-01-10 08:39:45,867] jepsen worker 546 - jepsen.util 546 :info :add 6797 :RequestTimeoutException INFO [2022-01-10 08:39:45,871] jepsen worker 144 - jepsen.util 1144 :invoke :add 8101 INFO [2022-01-10 08:39:45,871] jepsen worker 862 - jepsen.util 862 :info :add 6798 :RequestTimeoutException INFO [2022-01-10 08:39:45,872] jepsen worker 954 - jepsen.util 954 :info :add 6800 :RequestTimeoutException INFO [2022-01-10 08:39:45,873] jepsen worker 546 - jepsen.util 1546 :invoke :add 8102 INFO [2022-01-10 08:39:45,873] jepsen worker 940 - jepsen.util 940 :info :add 6799 :RequestTimeoutException INFO [2022-01-10 08:39:45,877] jepsen worker 913 - jepsen.util 1913 :invoke :add 8103 INFO [2022-01-10 08:39:45,879] jepsen worker 940 - jepsen.util 1940 :invoke :add 8104 INFO [2022-01-10 08:39:45,881] jepsen worker 862 - jepsen.util 1862 :invoke :add 8105 INFO [2022-01-10 08:39:45,882] jepsen worker 954 - jepsen.util 1954 :invoke :add 8106 INFO [2022-01-10 08:39:45,885] jepsen worker 665 - jepsen.util 1665 :invoke :add 8107 INFO [2022-01-10 08:39:45,899] jepsen worker 581 - jepsen.util 1581 :ok :add 8097 INFO [2022-01-10 08:39:45,901] jepsen worker 581 - jepsen.util 1581 :invoke :add 8108 INFO [2022-01-10 08:39:45,904] jepsen worker 862 - jepsen.util 1862 :ok :add 8105 INFO [2022-01-10 08:39:45,905] jepsen worker 446 - jepsen.util 1446 :ok :add 8096 INFO [2022-01-10 08:39:45,907] jepsen worker 862 - jepsen.util 1862 :invoke :add 8109 INFO [2022-01-10 08:39:45,910] jepsen worker 446 - jepsen.util 1446 :invoke :add 8110 INFO [2022-01-10 08:39:46,033] jepsen worker 231 - jepsen.util 231 :info :add 6801 :RequestTimeoutException INFO [2022-01-10 08:39:46,034] jepsen worker 585 - jepsen.util 585 :info :add 6803 :RequestTimeoutException INFO [2022-01-10 08:39:46,037] jepsen worker 231 - jepsen.util 1231 :invoke :add 8111 INFO [2022-01-10 08:39:46,037] jepsen worker 610 - jepsen.util 610 :info :add 6802 :RequestTimeoutException INFO [2022-01-10 08:39:46,040] jepsen worker 585 - jepsen.util 1585 :invoke :add 8112 INFO [2022-01-10 08:39:46,042] jepsen worker 610 - jepsen.util 1610 :invoke :add 8113 INFO [2022-01-10 08:39:46,056] jepsen worker 788 - jepsen.util 788 :info :add 6810 :RequestTimeoutException INFO [2022-01-10 08:39:46,056] jepsen worker 226 - jepsen.util 226 :info :add 6809 :RequestTimeoutException INFO [2022-01-10 08:39:46,059] jepsen worker 226 - jepsen.util 1226 :invoke :add 8114 INFO [2022-01-10 08:39:46,061] jepsen worker 945 - jepsen.util 945 :info :add 6815 :RequestTimeoutException INFO [2022-01-10 08:39:46,061] jepsen worker 826 - jepsen.util 826 :info :add 6813 :RequestTimeoutException INFO [2022-01-10 08:39:46,061] jepsen worker 788 - jepsen.util 1788 :invoke :add 8115 INFO [2022-01-10 08:39:46,062] jepsen worker 121 - jepsen.util 121 :info :add 6816 :RequestTimeoutException INFO [2022-01-10 08:39:46,063] jepsen worker 430 - jepsen.util 430 :info :add 6814 :RequestTimeoutException INFO [2022-01-10 08:39:46,065] jepsen worker 945 - jepsen.util 1945 :invoke :add 8116 INFO [2022-01-10 08:39:46,067] jepsen worker 826 - jepsen.util 1826 :invoke :add 8117 INFO [2022-01-10 08:39:46,067] jepsen worker 330 - jepsen.util 330 :info :add 6817 :RequestTimeoutException INFO [2022-01-10 08:39:46,068] jepsen worker 121 - jepsen.util 1121 :invoke :add 8118 INFO [2022-01-10 08:39:46,071] jepsen worker 430 - jepsen.util 1430 :invoke :add 8119 INFO [2022-01-10 08:39:46,073] jepsen worker 330 - jepsen.util 1330 :invoke :add 8120 INFO [2022-01-10 08:39:46,213] jepsen worker 232 - jepsen.util 232 :ok :add 6879 INFO [2022-01-10 08:39:46,213] jepsen worker 843 - jepsen.util 1843 :ok :add 7238 INFO [2022-01-10 08:39:46,213] jepsen worker 85 - jepsen.util 85 :ok :add 6861 INFO [2022-01-10 08:39:46,213] jepsen worker 99 - jepsen.util 99 :ok :add 6855 INFO [2022-01-10 08:39:46,213] jepsen worker 938 - jepsen.util 938 :ok :add 7149 INFO [2022-01-10 08:39:46,219] jepsen worker 99 - jepsen.util 99 :invoke :add 8121 INFO [2022-01-10 08:39:46,221] jepsen worker 232 - jepsen.util 232 :invoke :add 8122 INFO [2022-01-10 08:39:46,224] jepsen worker 843 - jepsen.util 1843 :invoke :add 8123 INFO [2022-01-10 08:39:46,226] jepsen worker 85 - jepsen.util 85 :invoke :add 8124 INFO [2022-01-10 08:39:46,229] jepsen worker 938 - jepsen.util 938 :invoke :add 8125 INFO [2022-01-10 08:39:46,230] jepsen worker 547 - jepsen.util 547 :info :add 6819 :RequestTimeoutException INFO [2022-01-10 08:39:46,232] jepsen worker 160 - jepsen.util 160 :info :add 6820 :RequestTimeoutException INFO [2022-01-10 08:39:46,234] jepsen worker 547 - jepsen.util 1547 :invoke :add 8126 INFO [2022-01-10 08:39:46,234] jepsen worker 291 - jepsen.util 291 :info :add 6818 :RequestTimeoutException INFO [2022-01-10 08:39:46,234] jepsen worker 22 - jepsen.util 22 :info :add 6822 :RequestTimeoutException INFO [2022-01-10 08:39:46,238] jepsen worker 291 - jepsen.util 1291 :invoke :add 8127 INFO [2022-01-10 08:39:46,240] jepsen worker 22 - jepsen.util 1022 :invoke :add 8128 INFO [2022-01-10 08:39:46,242] jepsen worker 160 - jepsen.util 1160 :invoke :add 8129 INFO [2022-01-10 08:39:46,242] jepsen worker 791 - jepsen.util 791 :info :add 6825 :RequestTimeoutException INFO [2022-01-10 08:39:46,243] jepsen worker 979 - jepsen.util 979 :info :add 6823 :RequestTimeoutException INFO [2022-01-10 08:39:46,247] jepsen worker 979 - jepsen.util 1979 :invoke :add 8130 INFO [2022-01-10 08:39:46,249] jepsen worker 160 - jepsen.util 1160 :ok :add 8129 INFO [2022-01-10 08:39:46,249] jepsen worker 22 - jepsen.util 1022 :ok :add 8128 INFO [2022-01-10 08:39:46,249] jepsen worker 791 - jepsen.util 1791 :invoke :add 8131 INFO [2022-01-10 08:39:46,251] jepsen worker 22 - jepsen.util 1022 :invoke :add 8132 INFO [2022-01-10 08:39:46,254] jepsen worker 137 - jepsen.util 137 :info :add 6827 :RequestTimeoutException INFO [2022-01-10 08:39:46,254] jepsen worker 160 - jepsen.util 1160 :invoke :add 8133 INFO [2022-01-10 08:39:46,256] jepsen worker 791 - jepsen.util 1791 :ok :add 8131 INFO [2022-01-10 08:39:46,257] jepsen worker 137 - jepsen.util 1137 :invoke :add 8134 INFO [2022-01-10 08:39:46,259] jepsen worker 22 - jepsen.util 1022 :ok :add 8132 INFO [2022-01-10 08:39:46,259] jepsen worker 791 - jepsen.util 1791 :invoke :add 8135 INFO [2022-01-10 08:39:46,262] jepsen worker 22 - jepsen.util 1022 :invoke :add 8136 INFO [2022-01-10 08:39:46,263] jepsen worker 160 - jepsen.util 1160 :ok :add 8133 INFO [2022-01-10 08:39:46,266] jepsen worker 160 - jepsen.util 1160 :invoke :add 8137 INFO [2022-01-10 08:39:46,413] jepsen worker 184 - jepsen.util 184 :info :add 6835 :RequestTimeoutException INFO [2022-01-10 08:39:46,414] jepsen worker 772 - jepsen.util 1772 :info :add 6836 :RequestTimeoutException INFO [2022-01-10 08:39:46,417] jepsen worker 184 - jepsen.util 1184 :invoke :add 8138 INFO [2022-01-10 08:39:46,417] jepsen worker 79 - jepsen.util 79 :info :add 6837 :RequestTimeoutException INFO [2022-01-10 08:39:46,419] jepsen worker 772 - jepsen.util 2772 :invoke :add 8139 INFO [2022-01-10 08:39:46,421] jepsen worker 79 - jepsen.util 1079 :invoke :add 8140 INFO [2022-01-10 08:39:46,427] jepsen worker 331 - jepsen.util 331 :info :add 6838 :RequestTimeoutException INFO [2022-01-10 08:39:46,429] jepsen worker 331 - jepsen.util 1331 :invoke :add 8141 INFO [2022-01-10 08:39:46,433] jepsen worker 215 - jepsen.util 215 :info :add 6844 :RequestTimeoutException INFO [2022-01-10 08:39:46,436] jepsen worker 215 - jepsen.util 1215 :invoke :add 8142 INFO [2022-01-10 08:39:46,437] jepsen worker 975 - jepsen.util 975 :info :add 6846 :RequestTimeoutException INFO [2022-01-10 08:39:46,441] jepsen worker 975 - jepsen.util 1975 :invoke :add 8143 INFO [2022-01-10 08:39:46,442] jepsen worker 215 - jepsen.util 1215 :ok :add 8142 INFO [2022-01-10 08:39:46,443] jepsen worker 186 - jepsen.util 1186 :info :add 6847 :RequestTimeoutException INFO [2022-01-10 08:39:46,443] jepsen worker 750 - jepsen.util 750 :info :add 6848 :RequestTimeoutException INFO [2022-01-10 08:39:46,443] jepsen worker 337 - jepsen.util 337 :info :add 6850 :RequestTimeoutException INFO [2022-01-10 08:39:46,443] jepsen worker 603 - jepsen.util 603 :info :add 6849 :RequestTimeoutException INFO [2022-01-10 08:39:46,447] jepsen worker 768 - jepsen.util 768 :info :add 6853 :RequestTimeoutException INFO [2022-01-10 08:39:46,449] jepsen worker 337 - jepsen.util 1337 :invoke :add 8144 INFO [2022-01-10 08:39:46,452] jepsen worker 186 - jepsen.util 2186 :invoke :add 8145 INFO [2022-01-10 08:39:46,453] jepsen worker 525 - jepsen.util 525 :info :add 6851 :RequestTimeoutException INFO [2022-01-10 08:39:46,454] jepsen worker 368 - jepsen.util 368 :info :add 6852 :RequestTimeoutException INFO [2022-01-10 08:39:46,454] jepsen worker 215 - jepsen.util 1215 :invoke :add 8146 INFO [2022-01-10 08:39:46,458] jepsen worker 768 - jepsen.util 1768 :invoke :add 8147 INFO [2022-01-10 08:39:46,459] jepsen worker 368 - jepsen.util 1368 :invoke :add 8148 INFO [2022-01-10 08:39:46,462] jepsen worker 750 - jepsen.util 1750 :invoke :add 8149 INFO [2022-01-10 08:39:46,463] jepsen worker 603 - jepsen.util 1603 :invoke :add 8150 INFO [2022-01-10 08:39:46,465] jepsen worker 525 - jepsen.util 1525 :invoke :add 8151 INFO [2022-01-10 08:39:46,498] jepsen worker 739 - jepsen.util 739 :info :add 6857 :RequestTimeoutException INFO [2022-01-10 08:39:46,501] jepsen worker 739 - jepsen.util 1739 :invoke :add 8152 INFO [2022-01-10 08:39:46,507] jepsen worker 559 - jepsen.util 559 :info :add 6863 :RequestTimeoutException INFO [2022-01-10 08:39:46,507] jepsen worker 53 - jepsen.util 1053 :info :add 6865 :RequestTimeoutException INFO [2022-01-10 08:39:46,510] jepsen worker 413 - jepsen.util 413 :info :add 6867 :RequestTimeoutException INFO [2022-01-10 08:39:46,511] jepsen worker 53 - jepsen.util 2053 :invoke :add 8153 INFO [2022-01-10 08:39:46,513] jepsen worker 732 - jepsen.util 732 :info :add 6866 :RequestTimeoutException INFO [2022-01-10 08:39:46,513] jepsen worker 380 - jepsen.util 380 :info :add 6864 :RequestTimeoutException INFO [2022-01-10 08:39:46,514] jepsen worker 897 - jepsen.util 897 :info :add 6868 :RequestTimeoutException INFO [2022-01-10 08:39:46,515] jepsen worker 559 - jepsen.util 1559 :invoke :add 8154 INFO [2022-01-10 08:39:46,519] jepsen worker 732 - jepsen.util 1732 :invoke :add 8155 INFO [2022-01-10 08:39:46,520] jepsen worker 380 - jepsen.util 1380 :invoke :add 8156 INFO [2022-01-10 08:39:46,522] jepsen worker 75 - jepsen.util 75 :info :add 6869 :RequestTimeoutException INFO [2022-01-10 08:39:46,522] jepsen worker 764 - jepsen.util 764 :info :add 6871 :RequestTimeoutException INFO [2022-01-10 08:39:46,522] jepsen worker 413 - jepsen.util 1413 :invoke :add 8157 INFO [2022-01-10 08:39:46,523] jepsen worker 279 - jepsen.util 279 :info :add 6873 :RequestTimeoutException INFO [2022-01-10 08:39:46,527] jepsen worker 75 - jepsen.util 1075 :invoke :add 8158 INFO [2022-01-10 08:39:46,527] jepsen worker 548 - jepsen.util 548 :info :add 6872 :RequestTimeoutException INFO [2022-01-10 08:39:46,528] jepsen worker 897 - jepsen.util 1897 :invoke :add 8159 INFO [2022-01-10 08:39:46,531] jepsen worker 279 - jepsen.util 1279 :invoke :add 8160 INFO [2022-01-10 08:39:46,533] jepsen worker 548 - jepsen.util 1548 :invoke :add 8161 INFO [2022-01-10 08:39:46,534] jepsen worker 897 - jepsen.util 1897 :ok :add 8159 INFO [2022-01-10 08:39:46,535] jepsen worker 764 - jepsen.util 1764 :invoke :add 8162 INFO [2022-01-10 08:39:46,536] jepsen worker 897 - jepsen.util 1897 :invoke :add 8163 INFO [2022-01-10 08:39:46,601] jepsen worker 694 - jepsen.util 1694 :info :add 6880 :RequestTimeoutException INFO [2022-01-10 08:39:46,604] jepsen worker 694 - jepsen.util 2694 :invoke :add 8164 INFO [2022-01-10 08:39:46,613] jepsen worker 761 - jepsen.util 761 :info :add 6884 :RequestTimeoutException INFO [2022-01-10 08:39:46,617] jepsen worker 761 - jepsen.util 1761 :invoke :add 8165 INFO [2022-01-10 08:39:46,617] jepsen worker 454 - jepsen.util 454 :info :add 6883 :RequestTimeoutException INFO [2022-01-10 08:39:46,621] jepsen worker 454 - jepsen.util 1454 :invoke :add 8166 INFO [2022-01-10 08:39:46,621] jepsen worker 372 - jepsen.util 372 :info :add 6885 :RequestTimeoutException INFO [2022-01-10 08:39:46,622] jepsen worker 370 - jepsen.util 1370 :info :add 6887 :RequestTimeoutException INFO [2022-01-10 08:39:46,623] jepsen worker 566 - jepsen.util 566 :info :add 6889 :RequestTimeoutException INFO [2022-01-10 08:39:46,623] jepsen worker 625 - jepsen.util 625 :info :add 6888 :RequestTimeoutException INFO [2022-01-10 08:39:46,623] jepsen worker 931 - jepsen.util 1931 :info :add 6886 :RequestTimeoutException INFO [2022-01-10 08:39:46,624] jepsen worker 13 - jepsen.util 13 :info :add 6891 :RequestTimeoutException INFO [2022-01-10 08:39:46,627] jepsen worker 801 - jepsen.util 1801 :info :add 6890 :RequestTimeoutException INFO [2022-01-10 08:39:46,629] jepsen worker 372 - jepsen.util 1372 :invoke :add 8167 INFO [2022-01-10 08:39:46,631] jepsen worker 931 - jepsen.util 2931 :invoke :add 8168 INFO [2022-01-10 08:39:46,631] jepsen worker 319 - jepsen.util 319 :info :add 6892 :RequestTimeoutException INFO [2022-01-10 08:39:46,632] jepsen worker 349 - jepsen.util 349 :info :add 6893 :RequestTimeoutException INFO [2022-01-10 08:39:46,632] jepsen worker 370 - jepsen.util 2370 :invoke :add 8169 INFO [2022-01-10 08:39:46,635] jepsen worker 801 - jepsen.util 2801 :invoke :add 8170 INFO [2022-01-10 08:39:46,637] jepsen worker 349 - jepsen.util 1349 :invoke :add 8171 INFO [2022-01-10 08:39:46,639] jepsen worker 625 - jepsen.util 1625 :invoke :add 8172 INFO [2022-01-10 08:39:46,640] jepsen worker 13 - jepsen.util 1013 :invoke :add 8173 INFO [2022-01-10 08:39:46,642] jepsen worker 566 - jepsen.util 1566 :invoke :add 8174 INFO [2022-01-10 08:39:46,644] jepsen worker 319 - jepsen.util 1319 :invoke :add 8175 INFO [2022-01-10 08:39:46,661] jepsen worker 794 - jepsen.util 794 :info :add 6898 :RequestTimeoutException INFO [2022-01-10 08:39:46,664] jepsen worker 794 - jepsen.util 1794 :invoke :add 8176 INFO [2022-01-10 08:39:46,671] jepsen worker 706 - jepsen.util 706 :info :add 6901 :RequestTimeoutException INFO [2022-01-10 08:39:46,672] jepsen worker 810 - jepsen.util 810 :info :add 6902 :RequestTimeoutException INFO [2022-01-10 08:39:46,673] jepsen worker 803 - jepsen.util 803 :info :add 6900 :RequestTimeoutException INFO [2022-01-10 08:39:46,673] jepsen worker 95 - jepsen.util 1095 :info :add 6903 :RequestTimeoutException INFO [2022-01-10 08:39:46,676] jepsen worker 810 - jepsen.util 1810 :invoke :add 8177 INFO [2022-01-10 08:39:46,677] jepsen worker 803 - jepsen.util 1803 :invoke :add 8178 INFO [2022-01-10 08:39:46,679] jepsen worker 706 - jepsen.util 1706 :invoke :add 8179 INFO [2022-01-10 08:39:46,681] jepsen worker 95 - jepsen.util 2095 :invoke :add 8180 INFO [2022-01-10 08:39:46,682] jepsen worker 810 - jepsen.util 1810 :ok :add 8177 INFO [2022-01-10 08:39:46,682] jepsen worker 131 - jepsen.util 131 :info :add 6908 :RequestTimeoutException INFO [2022-01-10 08:39:46,683] jepsen worker 257 - jepsen.util 257 :info :add 6910 :RequestTimeoutException INFO [2022-01-10 08:39:46,684] jepsen worker 810 - jepsen.util 1810 :invoke :add 8181 INFO [2022-01-10 08:39:46,687] jepsen worker 257 - jepsen.util 1257 :invoke :add 8182 INFO [2022-01-10 08:39:46,689] jepsen worker 131 - jepsen.util 1131 :invoke :add 8183 INFO [2022-01-10 08:39:46,691] jepsen worker 882 - jepsen.util 882 :info :add 6914 :RequestTimeoutException INFO [2022-01-10 08:39:46,692] jepsen worker 282 - jepsen.util 1282 :info :add 6912 :RequestTimeoutException INFO [2022-01-10 08:39:46,693] jepsen worker 111 - jepsen.util 111 :info :add 6911 :RequestTimeoutException INFO [2022-01-10 08:39:46,693] jepsen worker 192 - jepsen.util 192 :info :add 6916 :RequestTimeoutException INFO [2022-01-10 08:39:46,693] jepsen worker 882 - jepsen.util 1882 :invoke :add 8184 INFO [2022-01-10 08:39:46,694] jepsen worker 850 - jepsen.util 850 :info :add 6913 :RequestTimeoutException INFO [2022-01-10 08:39:46,697] jepsen worker 578 - jepsen.util 578 :info :add 6915 :RequestTimeoutException INFO [2022-01-10 08:39:46,697] jepsen worker 111 - jepsen.util 1111 :invoke :add 8185 INFO [2022-01-10 08:39:46,699] jepsen worker 578 - jepsen.util 1578 :invoke :add 8186 INFO [2022-01-10 08:39:46,701] jepsen worker 11 - jepsen.util 11 :info :add 6919 :RequestTimeoutException INFO [2022-01-10 08:39:46,702] jepsen worker 850 - jepsen.util 1850 :invoke :add 8187 INFO [2022-01-10 08:39:46,704] jepsen worker 623 - jepsen.util 623 :info :add 6917 :RequestTimeoutException INFO [2022-01-10 08:39:46,704] jepsen worker 282 - jepsen.util 2282 :invoke :add 8188 INFO [2022-01-10 08:39:46,706] jepsen worker 623 - jepsen.util 1623 :invoke :add 8189 INFO [2022-01-10 08:39:46,707] jepsen worker 436 - jepsen.util 1436 :info :add 6918 :RequestTimeoutException INFO [2022-01-10 08:39:46,708] jepsen worker 192 - jepsen.util 1192 :invoke :add 8190 INFO [2022-01-10 08:39:46,710] jepsen worker 602 - jepsen.util 1602 :info :add 6920 :RequestTimeoutException INFO [2022-01-10 08:39:46,710] jepsen worker 685 - jepsen.util 685 :info :add 6921 :RequestTimeoutException INFO [2022-01-10 08:39:46,711] jepsen worker 436 - jepsen.util 2436 :invoke :add 8191 INFO [2022-01-10 08:39:46,714] jepsen worker 685 - jepsen.util 1685 :invoke :add 8192 INFO [2022-01-10 08:39:46,716] jepsen worker 11 - jepsen.util 1011 :invoke :add 8193 INFO [2022-01-10 08:39:46,718] jepsen worker 602 - jepsen.util 2602 :invoke :add 8194 INFO [2022-01-10 08:39:46,791] jepsen worker 442 - jepsen.util 1442 :ok :add 7414 INFO [2022-01-10 08:39:46,791] jepsen worker 989 - jepsen.util 1989 :ok :add 7420 INFO [2022-01-10 08:39:46,791] jepsen worker 921 - jepsen.util 1921 :ok :add 7023 INFO [2022-01-10 08:39:46,791] jepsen worker 805 - jepsen.util 1805 :ok :add 6927 INFO [2022-01-10 08:39:46,791] jepsen worker 538 - jepsen.util 538 :ok :add 7352 INFO [2022-01-10 08:39:46,791] jepsen worker 679 - jepsen.util 1679 :ok :add 7366 INFO [2022-01-10 08:39:46,791] jepsen worker 535 - jepsen.util 1535 :ok :add 7017 INFO [2022-01-10 08:39:46,791] jepsen worker 181 - jepsen.util 181 :ok :add 7438 INFO [2022-01-10 08:39:46,794] jepsen worker 805 - jepsen.util 1805 :invoke :add 8195 INFO [2022-01-10 08:39:46,796] jepsen worker 535 - jepsen.util 1535 :invoke :add 8196 INFO [2022-01-10 08:39:46,798] jepsen worker 679 - jepsen.util 1679 :invoke :add 8197 INFO [2022-01-10 08:39:46,800] jepsen worker 538 - jepsen.util 538 :invoke :add 8198 INFO [2022-01-10 08:39:46,801] jepsen worker 989 - jepsen.util 1989 :invoke :add 8199 INFO [2022-01-10 08:39:46,803] jepsen worker 716 - jepsen.util 1716 :info :add 6922 :RequestTimeoutException INFO [2022-01-10 08:39:46,803] jepsen worker 442 - jepsen.util 1442 :invoke :add 8200 INFO [2022-01-10 08:39:46,805] jepsen worker 679 - jepsen.util 1679 :ok :add 8197 INFO [2022-01-10 08:39:46,805] jepsen worker 716 - jepsen.util 2716 :invoke :add 8201 INFO [2022-01-10 08:39:46,807] jepsen worker 921 - jepsen.util 1921 :invoke :add 8202 INFO [2022-01-10 08:39:46,809] jepsen worker 181 - jepsen.util 181 :invoke :add 8203 INFO [2022-01-10 08:39:46,811] jepsen worker 679 - jepsen.util 1679 :invoke :add 8204 INFO [2022-01-10 08:39:46,811] jepsen worker 299 - jepsen.util 1299 :info :add 6923 :RequestTimeoutException INFO [2022-01-10 08:39:46,812] jepsen worker 350 - jepsen.util 1350 :info :add 6925 :RequestTimeoutException INFO [2022-01-10 08:39:46,813] jepsen worker 406 - jepsen.util 1406 :info :add 6924 :RequestTimeoutException INFO [2022-01-10 08:39:46,815] jepsen worker 299 - jepsen.util 2299 :invoke :add 8205 INFO [2022-01-10 08:39:46,817] jepsen worker 679 - jepsen.util 1679 :ok :add 8204 INFO [2022-01-10 08:39:46,817] jepsen worker 406 - jepsen.util 2406 :invoke :add 8206 INFO [2022-01-10 08:39:46,820] jepsen worker 350 - jepsen.util 2350 :invoke :add 8207 INFO [2022-01-10 08:39:46,822] jepsen worker 679 - jepsen.util 1679 :invoke :add 8208 INFO [2022-01-10 08:39:46,825] jepsen worker 406 - jepsen.util 2406 :ok :add 8206 INFO [2022-01-10 08:39:46,828] jepsen worker 406 - jepsen.util 2406 :invoke :add 8209 INFO [2022-01-10 08:39:46,833] jepsen worker 114 - jepsen.util 1114 :ok :add 7550 INFO [2022-01-10 08:39:46,833] jepsen worker 62 - jepsen.util 62 :ok :add 7548 INFO [2022-01-10 08:39:46,833] jepsen worker 270 - jepsen.util 1270 :ok :add 7222 INFO [2022-01-10 08:39:46,833] jepsen worker 686 - jepsen.util 1686 :ok :add 7216 INFO [2022-01-10 08:39:46,834] jepsen worker 150 - jepsen.util 1150 :ok :add 7153 INFO [2022-01-10 08:39:46,834] jepsen worker 24 - jepsen.util 24 :ok :add 7167 INFO [2022-01-10 08:39:46,834] jepsen worker 496 - jepsen.util 1496 :ok :add 7386 INFO [2022-01-10 08:39:46,837] jepsen worker 150 - jepsen.util 1150 :invoke :add 8210 INFO [2022-01-10 08:39:46,839] jepsen worker 24 - jepsen.util 24 :invoke :add 8211 INFO [2022-01-10 08:39:46,840] jepsen worker 62 - jepsen.util 62 :invoke :add 8212 INFO [2022-01-10 08:39:46,842] jepsen worker 270 - jepsen.util 1270 :invoke :add 8213 INFO [2022-01-10 08:39:46,844] jepsen worker 114 - jepsen.util 1114 :invoke :add 8214 INFO [2022-01-10 08:39:46,846] jepsen worker 686 - jepsen.util 1686 :invoke :add 8215 INFO [2022-01-10 08:39:46,848] jepsen worker 496 - jepsen.util 1496 :invoke :add 8216 INFO [2022-01-10 08:39:46,881] jepsen worker 641 - jepsen.util 1641 :info :add 6935 :RequestTimeoutException INFO [2022-01-10 08:39:46,884] jepsen worker 641 - jepsen.util 2641 :invoke :add 8217 INFO [2022-01-10 08:39:46,887] jepsen worker 246 - jepsen.util 246 :info :add 6934 :RequestTimeoutException INFO [2022-01-10 08:39:46,891] jepsen worker 246 - jepsen.util 1246 :invoke :add 8218 INFO [2022-01-10 08:39:46,892] jepsen worker 94 - jepsen.util 94 :info :add 6936 :RequestTimeoutException INFO [2022-01-10 08:39:46,893] jepsen worker 881 - jepsen.util 881 :info :add 6937 :RequestTimeoutException INFO [2022-01-10 08:39:46,893] jepsen worker 126 - jepsen.util 126 :info :add 6938 :RequestTimeoutException INFO [2022-01-10 08:39:46,897] jepsen worker 443 - jepsen.util 443 :info :add 6939 :RequestTimeoutException INFO [2022-01-10 08:39:46,897] jepsen worker 881 - jepsen.util 1881 :invoke :add 8219 INFO [2022-01-10 08:39:46,900] jepsen worker 443 - jepsen.util 1443 :invoke :add 8220 INFO [2022-01-10 08:39:46,902] jepsen worker 205 - jepsen.util 1205 :info :add 6944 :RequestTimeoutException INFO [2022-01-10 08:39:46,902] jepsen worker 905 - jepsen.util 905 :info :add 6945 :RequestTimeoutException INFO [2022-01-10 08:39:46,902] jepsen worker 126 - jepsen.util 1126 :invoke :add 8221 INFO [2022-01-10 08:39:46,905] jepsen worker 205 - jepsen.util 2205 :invoke :add 8222 INFO [2022-01-10 08:39:46,907] jepsen worker 94 - jepsen.util 1094 :invoke :add 8223 INFO [2022-01-10 08:39:46,907] jepsen worker 332 - jepsen.util 332 :info :add 6947 :RequestTimeoutException INFO [2022-01-10 08:39:46,908] jepsen worker 905 - jepsen.util 1905 :invoke :add 8224 INFO [2022-01-10 08:39:46,911] jepsen worker 877 - jepsen.util 877 :info :add 6946 :RequestTimeoutException INFO [2022-01-10 08:39:46,912] jepsen worker 332 - jepsen.util 1332 :invoke :add 8225 INFO [2022-01-10 08:39:46,914] jepsen worker 877 - jepsen.util 1877 :invoke :add 8226 INFO [2022-01-10 08:39:46,947] jepsen worker 656 - jepsen.util 1656 :info :add 6948 :RequestTimeoutException INFO [2022-01-10 08:39:46,953] jepsen worker 656 - jepsen.util 2656 :invoke :add 8227 INFO [2022-01-10 08:39:46,960] jepsen worker 886 - jepsen.util 1886 :info :add 6949 :RequestTimeoutException INFO [2022-01-10 08:39:46,961] jepsen worker 540 - jepsen.util 1540 :info :add 6951 :RequestTimeoutException INFO [2022-01-10 08:39:46,963] jepsen worker 659 - jepsen.util 1659 :info :add 6950 :RequestTimeoutException INFO [2022-01-10 08:39:46,965] jepsen worker 886 - jepsen.util 2886 :invoke :add 8228 INFO [2022-01-10 08:39:46,968] jepsen worker 659 - jepsen.util 2659 :invoke :add 8229 INFO [2022-01-10 08:39:46,969] jepsen worker 540 - jepsen.util 2540 :invoke :add 8230 INFO [2022-01-10 08:39:47,003] jepsen worker 475 - jepsen.util 1475 :info :add 6953 :RequestTimeoutException INFO [2022-01-10 08:39:47,004] jepsen worker 735 - jepsen.util 1735 :info :add 6952 :RequestTimeoutException INFO [2022-01-10 08:39:47,005] jepsen worker 475 - jepsen.util 2475 :invoke :add 8231 INFO [2022-01-10 08:39:47,008] jepsen worker 735 - jepsen.util 2735 :invoke :add 8232 INFO [2022-01-10 08:39:47,015] jepsen worker 735 - jepsen.util 2735 :ok :add 8232 INFO [2022-01-10 08:39:47,019] jepsen worker 735 - jepsen.util 2735 :invoke :add 8233 INFO [2022-01-10 08:39:47,025] jepsen worker 735 - jepsen.util 2735 :ok :add 8233 INFO [2022-01-10 08:39:47,028] jepsen worker 735 - jepsen.util 2735 :invoke :add 8234 INFO [2022-01-10 08:39:47,031] jepsen worker 385 - jepsen.util 1385 :info :add 6957 :RequestTimeoutException INFO [2022-01-10 08:39:47,033] jepsen worker 500 - jepsen.util 1500 :info :add 6956 :RequestTimeoutException INFO [2022-01-10 08:39:47,037] jepsen worker 385 - jepsen.util 2385 :invoke :add 8235 INFO [2022-01-10 08:39:47,039] jepsen worker 500 - jepsen.util 2500 :invoke :add 8236 INFO [2022-01-10 08:39:47,162] jepsen worker 309 - jepsen.util 1309 :info :add 6960 :RequestTimeoutException INFO [2022-01-10 08:39:47,165] jepsen worker 309 - jepsen.util 2309 :invoke :add 8237 INFO [2022-01-10 08:39:47,190] jepsen worker 951 - jepsen.util 1951 :info :add 6964 :RequestTimeoutException INFO [2022-01-10 08:39:47,193] jepsen worker 951 - jepsen.util 2951 :invoke :add 8238 INFO [2022-01-10 08:39:47,193] jepsen worker 976 - jepsen.util 976 :info :add 6963 :RequestTimeoutException INFO [2022-01-10 08:39:47,196] jepsen worker 976 - jepsen.util 1976 :invoke :add 8239 INFO [2022-01-10 08:39:47,197] jepsen worker 967 - jepsen.util 1967 :info :add 6962 :RequestTimeoutException INFO [2022-01-10 08:39:47,200] jepsen worker 967 - jepsen.util 2967 :invoke :add 8240 INFO [2022-01-10 08:39:47,200] jepsen worker 647 - jepsen.util 647 :info :add 6965 :RequestTimeoutException INFO [2022-01-10 08:39:47,201] jepsen worker 793 - jepsen.util 1793 :info :add 6968 :RequestTimeoutException INFO [2022-01-10 08:39:47,203] jepsen worker 416 - jepsen.util 1416 :info :add 6966 :RequestTimeoutException INFO [2022-01-10 08:39:47,203] jepsen worker 567 - jepsen.util 567 :info :add 6967 :RequestTimeoutException INFO [2022-01-10 08:39:47,203] jepsen worker 593 - jepsen.util 593 :info :add 6969 :RequestTimeoutException INFO [2022-01-10 08:39:47,204] jepsen worker 785 - jepsen.util 1785 :info :add 6970 :RequestTimeoutException INFO [2022-01-10 08:39:47,205] jepsen worker 647 - jepsen.util 1647 :invoke :add 8241 INFO [2022-01-10 08:39:47,207] jepsen worker 967 - jepsen.util 2967 :ok :add 8240 INFO [2022-01-10 08:39:47,209] jepsen worker 967 - jepsen.util 2967 :invoke :add 8242 INFO [2022-01-10 08:39:47,212] jepsen worker 970 - jepsen.util 970 :info :add 6971 :RequestTimeoutException INFO [2022-01-10 08:39:47,212] jepsen worker 741 - jepsen.util 741 :info :add 6972 :RequestTimeoutException INFO [2022-01-10 08:39:47,212] jepsen worker 752 - jepsen.util 1752 :info :add 6973 :RequestTimeoutException INFO [2022-01-10 08:39:47,212] jepsen worker 416 - jepsen.util 2416 :invoke :add 8243 INFO [2022-01-10 08:39:47,213] jepsen worker 793 - jepsen.util 2793 :invoke :add 8244 INFO [2022-01-10 08:39:47,216] jepsen worker 967 - jepsen.util 2967 :ok :add 8242 INFO [2022-01-10 08:39:47,217] jepsen worker 741 - jepsen.util 1741 :invoke :add 8245 INFO [2022-01-10 08:39:47,218] jepsen worker 647 - jepsen.util 1647 :ok :add 8241 INFO [2022-01-10 08:39:47,219] jepsen worker 970 - jepsen.util 1970 :invoke :add 8246 INFO [2022-01-10 08:39:47,221] jepsen worker 647 - jepsen.util 1647 :invoke :add 8247 INFO [2022-01-10 08:39:47,222] jepsen worker 967 - jepsen.util 2967 :invoke :add 8248 INFO [2022-01-10 08:39:47,223] jepsen worker 220 - jepsen.util 220 :info :add 6978 :RequestTimeoutException INFO [2022-01-10 08:39:47,224] jepsen worker 785 - jepsen.util 2785 :invoke :add 8249 INFO [2022-01-10 08:39:47,226] jepsen worker 567 - jepsen.util 1567 :invoke :add 8250 INFO [2022-01-10 08:39:47,228] jepsen worker 593 - jepsen.util 1593 :invoke :add 8251 INFO [2022-01-10 08:39:47,230] jepsen worker 736 - jepsen.util 736 :info :add 6985 :RequestTimeoutException INFO [2022-01-10 08:39:47,230] jepsen worker 220 - jepsen.util 1220 :invoke :add 8252 INFO [2022-01-10 08:39:47,231] jepsen worker 478 - jepsen.util 1478 :info :add 6983 :RequestTimeoutException INFO [2022-01-10 08:39:47,231] jepsen worker 369 - jepsen.util 369 :info :add 6984 :RequestTimeoutException INFO [2022-01-10 08:39:47,233] jepsen worker 752 - jepsen.util 2752 :invoke :add 8253 INFO [2022-01-10 08:39:47,233] jepsen worker 16 - jepsen.util 1016 :info :add 6982 :RequestTimeoutException INFO [2022-01-10 08:39:47,234] jepsen worker 926 - jepsen.util 1926 :info :add 6986 :RequestTimeoutException INFO [2022-01-10 08:39:47,238] jepsen worker 16 - jepsen.util 2016 :invoke :add 8254 INFO [2022-01-10 08:39:47,240] jepsen worker 736 - jepsen.util 1736 :invoke :add 8255 INFO [2022-01-10 08:39:47,240] jepsen worker 241 - jepsen.util 1241 :info :add 6989 :RequestTimeoutException INFO [2022-01-10 08:39:47,241] jepsen worker 478 - jepsen.util 2478 :invoke :add 8256 INFO [2022-01-10 08:39:47,242] jepsen worker 267 - jepsen.util 1267 :info :add 6987 :RequestTimeoutException INFO [2022-01-10 08:39:47,243] jepsen worker 926 - jepsen.util 2926 :invoke :add 8257 INFO [2022-01-10 08:39:47,246] jepsen worker 267 - jepsen.util 2267 :invoke :add 8258 INFO [2022-01-10 08:39:47,247] jepsen worker 271 - jepsen.util 1271 :info :add 6988 :RequestTimeoutException INFO [2022-01-10 08:39:47,249] jepsen worker 241 - jepsen.util 2241 :invoke :add 8259 INFO [2022-01-10 08:39:47,251] jepsen worker 369 - jepsen.util 1369 :invoke :add 8260 INFO [2022-01-10 08:39:47,252] jepsen worker 267 - jepsen.util 2267 :ok :add 8258 INFO [2022-01-10 08:39:47,253] jepsen worker 392 - jepsen.util 1392 :info :add 6990 :RequestTimeoutException INFO [2022-01-10 08:39:47,254] jepsen worker 271 - jepsen.util 2271 :invoke :add 8261 INFO [2022-01-10 08:39:47,257] jepsen worker 267 - jepsen.util 2267 :invoke :add 8262 INFO [2022-01-10 08:39:47,258] jepsen worker 392 - jepsen.util 2392 :invoke :add 8263 INFO [2022-01-10 08:39:47,271] jepsen worker 505 - jepsen.util 1505 :info :add 6991 :RequestTimeoutException INFO [2022-01-10 08:39:47,274] jepsen worker 505 - jepsen.util 2505 :invoke :add 8264 INFO [2022-01-10 08:39:47,275] jepsen worker 545 - jepsen.util 1545 :info :add 6992 :RequestTimeoutException INFO [2022-01-10 08:39:47,278] jepsen worker 545 - jepsen.util 2545 :invoke :add 8265 INFO [2022-01-10 08:39:47,357] jepsen worker 28 - jepsen.util 28 :info :add 6993 :RequestTimeoutException INFO [2022-01-10 08:39:47,361] jepsen worker 666 - jepsen.util 666 :info :add 6998 :RequestTimeoutException INFO [2022-01-10 08:39:47,361] jepsen worker 28 - jepsen.util 1028 :invoke :add 8266 INFO [2022-01-10 08:39:47,364] jepsen worker 666 - jepsen.util 1666 :invoke :add 8267 INFO [2022-01-10 08:39:47,372] jepsen worker 746 - jepsen.util 746 :info :add 6999 :RequestTimeoutException INFO [2022-01-10 08:39:47,376] jepsen worker 746 - jepsen.util 1746 :invoke :add 8268 INFO [2022-01-10 08:39:47,380] jepsen worker 116 - jepsen.util 116 :info :add 7004 :RequestTimeoutException INFO [2022-01-10 08:39:47,381] jepsen worker 758 - jepsen.util 758 :info :add 7008 :RequestTimeoutException INFO [2022-01-10 08:39:47,383] jepsen worker 626 - jepsen.util 626 :info :add 7005 :RequestTimeoutException INFO [2022-01-10 08:39:47,383] jepsen worker 878 - jepsen.util 1878 :info :add 7006 :RequestTimeoutException INFO [2022-01-10 08:39:47,383] jepsen worker 116 - jepsen.util 1116 :invoke :add 8269 INFO [2022-01-10 08:39:47,387] jepsen worker 133 - jepsen.util 1133 :info :add 7007 :RequestTimeoutException INFO [2022-01-10 08:39:47,389] jepsen worker 626 - jepsen.util 1626 :invoke :add 8270 INFO [2022-01-10 08:39:47,390] jepsen worker 802 - jepsen.util 1802 :info :add 7009 :RequestTimeoutException INFO [2022-01-10 08:39:47,391] jepsen worker 600 - jepsen.util 1600 :info :add 7012 :RequestTimeoutException INFO [2022-01-10 08:39:47,392] jepsen worker 650 - jepsen.util 650 :info :add 7013 :RequestTimeoutException INFO [2022-01-10 08:39:47,393] jepsen worker 42 - jepsen.util 1042 :info :add 7014 :RequestTimeoutException INFO [2022-01-10 08:39:47,394] jepsen worker 802 - jepsen.util 2802 :invoke :add 8271 INFO [2022-01-10 08:39:47,397] jepsen worker 950 - jepsen.util 950 :ok :add 7163 INFO [2022-01-10 08:39:47,397] jepsen worker 488 - jepsen.util 488 :info :add 7011 :RequestTimeoutException INFO [2022-01-10 08:39:47,399] jepsen worker 650 - jepsen.util 1650 :invoke :add 8272 INFO [2022-01-10 08:39:47,401] jepsen worker 600 - jepsen.util 2600 :invoke :add 8273 INFO [2022-01-10 08:39:47,403] jepsen worker 488 - jepsen.util 1488 :invoke :add 8274 INFO [2022-01-10 08:39:47,405] jepsen worker 758 - jepsen.util 1758 :invoke :add 8275 INFO [2022-01-10 08:39:47,407] jepsen worker 133 - jepsen.util 2133 :invoke :add 8276 INFO [2022-01-10 08:39:47,409] jepsen worker 950 - jepsen.util 950 :invoke :add 8277 INFO [2022-01-10 08:39:47,411] jepsen worker 878 - jepsen.util 2878 :invoke :add 8278 INFO [2022-01-10 08:39:47,413] jepsen worker 515 - jepsen.util 1515 :info :add 7016 :RequestTimeoutException INFO [2022-01-10 08:39:47,413] jepsen worker 42 - jepsen.util 2042 :invoke :add 8279 INFO [2022-01-10 08:39:47,416] jepsen worker 515 - jepsen.util 2515 :invoke :add 8280 INFO [2022-01-10 08:39:47,474] jepsen worker 488 - jepsen.util 1488 :ok :add 8274 INFO [2022-01-10 08:39:47,474] jepsen worker 758 - jepsen.util 1758 :ok :add 8275 INFO [2022-01-10 08:39:47,476] jepsen worker 488 - jepsen.util 1488 :invoke :add 8281 INFO [2022-01-10 08:39:47,478] jepsen worker 758 - jepsen.util 1758 :invoke :add 8282 INFO [2022-01-10 08:39:47,501] jepsen worker 827 - jepsen.util 1827 :info :add 7020 :RequestTimeoutException INFO [2022-01-10 08:39:47,504] jepsen worker 827 - jepsen.util 2827 :invoke :add 8283 INFO [2022-01-10 08:39:47,511] jepsen worker 15 - jepsen.util 1015 :info :add 7024 :RequestTimeoutException INFO [2022-01-10 08:39:47,512] jepsen worker 507 - jepsen.util 1507 :info :add 7022 :RequestTimeoutException INFO [2022-01-10 08:39:47,513] jepsen worker 555 - jepsen.util 1555 :info :add 7025 :RequestTimeoutException INFO [2022-01-10 08:39:47,513] jepsen worker 15 - jepsen.util 2015 :invoke :add 8284 INFO [2022-01-10 08:39:47,518] jepsen worker 507 - jepsen.util 2507 :invoke :add 8285 INFO [2022-01-10 08:39:47,520] jepsen worker 555 - jepsen.util 2555 :invoke :add 8286 INFO [2022-01-10 08:39:47,521] jepsen worker 552 - jepsen.util 1552 :info :add 7026 :RequestTimeoutException INFO [2022-01-10 08:39:47,523] jepsen worker 806 - jepsen.util 1806 :info :add 7027 :RequestTimeoutException INFO [2022-01-10 08:39:47,523] jepsen worker 387 - jepsen.util 387 :info :add 7029 :RequestTimeoutException INFO [2022-01-10 08:39:47,523] jepsen worker 471 - jepsen.util 471 :info :add 7030 :RequestTimeoutException INFO [2022-01-10 08:39:47,524] jepsen worker 552 - jepsen.util 2552 :invoke :add 8287 INFO [2022-01-10 08:39:47,527] jepsen worker 798 - jepsen.util 798 :info :add 7028 :RequestTimeoutException INFO [2022-01-10 08:39:47,528] jepsen worker 471 - jepsen.util 1471 :invoke :add 8288 INFO [2022-01-10 08:39:47,530] jepsen worker 765 - jepsen.util 765 :info :add 7031 :RequestTimeoutException INFO [2022-01-10 08:39:47,532] jepsen worker 798 - jepsen.util 1798 :invoke :add 8289 INFO [2022-01-10 08:39:47,532] jepsen worker 860 - jepsen.util 1860 :info :add 7032 :RequestTimeoutException INFO [2022-01-10 08:39:47,536] jepsen worker 860 - jepsen.util 2860 :invoke :add 8290 INFO [2022-01-10 08:39:47,536] jepsen worker 118 - jepsen.util 118 :info :add 7033 :RequestTimeoutException INFO [2022-01-10 08:39:47,539] jepsen worker 387 - jepsen.util 1387 :invoke :add 8291 INFO [2022-01-10 08:39:47,542] jepsen worker 765 - jepsen.util 1765 :invoke :add 8292 INFO [2022-01-10 08:39:47,543] jepsen worker 917 - jepsen.util 917 :info :add 7038 :RequestTimeoutException INFO [2022-01-10 08:39:47,543] jepsen worker 797 - jepsen.util 797 :info :add 7040 :RequestTimeoutException INFO [2022-01-10 08:39:47,544] jepsen worker 806 - jepsen.util 2806 :invoke :add 8293 INFO [2022-01-10 08:39:47,548] jepsen worker 917 - jepsen.util 1917 :invoke :add 8294 INFO [2022-01-10 08:39:47,549] jepsen worker 797 - jepsen.util 1797 :invoke :add 8295 INFO [2022-01-10 08:39:47,550] jepsen worker 621 - jepsen.util 621 :info :add 7041 :RequestTimeoutException INFO [2022-01-10 08:39:47,551] jepsen worker 495 - jepsen.util 495 :info :add 7043 :RequestTimeoutException INFO [2022-01-10 08:39:47,551] jepsen worker 118 - jepsen.util 1118 :invoke :add 8296 INFO [2022-01-10 08:39:47,552] jepsen worker 512 - jepsen.util 512 :info :add 7042 :RequestTimeoutException INFO [2022-01-10 08:39:47,557] jepsen worker 512 - jepsen.util 1512 :invoke :add 8297 INFO [2022-01-10 08:39:47,558] jepsen worker 118 - jepsen.util 1118 :ok :add 8296 INFO [2022-01-10 08:39:47,559] jepsen worker 495 - jepsen.util 1495 :invoke :add 8298 INFO [2022-01-10 08:39:47,562] jepsen worker 621 - jepsen.util 1621 :invoke :add 8299 INFO [2022-01-10 08:39:47,564] jepsen worker 118 - jepsen.util 1118 :invoke :add 8300 INFO [2022-01-10 08:39:47,582] jepsen worker 233 - jepsen.util 1233 :ok :add 7465 INFO [2022-01-10 08:39:47,583] jepsen worker 156 - jepsen.util 1156 :ok :add 7287 INFO [2022-01-10 08:39:47,583] jepsen worker 316 - jepsen.util 1316 :ok :add 7317 INFO [2022-01-10 08:39:47,583] jepsen worker 702 - jepsen.util 702 :ok :add 7449 INFO [2022-01-10 08:39:47,585] jepsen worker 233 - jepsen.util 1233 :invoke :add 8301 INFO [2022-01-10 08:39:47,588] jepsen worker 156 - jepsen.util 1156 :invoke :add 8302 INFO [2022-01-10 08:39:47,590] jepsen worker 316 - jepsen.util 1316 :invoke :add 8303 INFO [2022-01-10 08:39:47,592] jepsen worker 702 - jepsen.util 702 :invoke :add 8304 INFO [2022-01-10 08:39:47,684] jepsen worker 388 - jepsen.util 388 :info :add 7048 :RequestTimeoutException INFO [2022-01-10 08:39:47,689] jepsen worker 388 - jepsen.util 1388 :invoke :add 8305 INFO [2022-01-10 08:39:47,692] jepsen worker 262 - jepsen.util 262 :info :add 7050 :RequestTimeoutException INFO [2022-01-10 08:39:47,693] jepsen worker 808 - jepsen.util 808 :info :add 7049 :RequestTimeoutException INFO [2022-01-10 08:39:47,698] jepsen worker 808 - jepsen.util 1808 :invoke :add 8306 INFO [2022-01-10 08:39:47,700] jepsen worker 262 - jepsen.util 1262 :invoke :add 8307 INFO [2022-01-10 08:39:47,707] jepsen worker 262 - jepsen.util 1262 :ok :add 8307 INFO [2022-01-10 08:39:47,712] jepsen worker 262 - jepsen.util 1262 :invoke :add 8308 INFO [2022-01-10 08:39:47,743] jepsen worker 728 - jepsen.util 1728 :info :add 7052 :RequestTimeoutException INFO [2022-01-10 08:39:47,747] jepsen worker 728 - jepsen.util 2728 :invoke :add 8309 INFO [2022-01-10 08:39:47,790] jepsen worker 335 - jepsen.util 1335 :info :add 7054 :RequestTimeoutException INFO [2022-01-10 08:39:47,796] jepsen worker 335 - jepsen.util 2335 :invoke :add 8310 INFO [2022-01-10 08:39:47,797] jepsen worker 48 - jepsen.util 1048 :info :add 7053 :RequestTimeoutException INFO [2022-01-10 08:39:47,802] jepsen worker 48 - jepsen.util 2048 :invoke :add 8311 INFO [2022-01-10 08:39:47,832] jepsen worker 485 - jepsen.util 485 :info :add 7055 :RequestTimeoutException INFO [2022-01-10 08:39:47,837] jepsen worker 485 - jepsen.util 1485 :invoke :add 8312 INFO [2022-01-10 08:39:47,840] jepsen worker 437 - jepsen.util 437 :info :add 7057 :RequestTimeoutException INFO [2022-01-10 08:39:47,841] jepsen worker 204 - jepsen.util 204 :info :add 7056 :RequestTimeoutException INFO [2022-01-10 08:39:47,842] jepsen worker 977 - jepsen.util 977 :info :add 7061 :RequestTimeoutException INFO [2022-01-10 08:39:47,843] jepsen worker 35 - jepsen.util 35 :info :add 7059 :RequestTimeoutException INFO [2022-01-10 08:39:47,843] jepsen worker 87 - jepsen.util 87 :info :add 7060 :RequestTimeoutException INFO [2022-01-10 08:39:47,843] jepsen worker 504 - jepsen.util 504 :info :add 7058 :RequestTimeoutException INFO [2022-01-10 08:39:47,844] jepsen worker 813 - jepsen.util 813 :info :add 7062 :RequestTimeoutException INFO [2022-01-10 08:39:47,851] jepsen worker 342 - jepsen.util 342 :info :add 7063 :RequestTimeoutException INFO [2022-01-10 08:39:47,851] jepsen worker 813 - jepsen.util 1813 :invoke :add 8313 INFO [2022-01-10 08:39:47,853] jepsen worker 997 - jepsen.util 1997 :info :add 7066 :RequestTimeoutException INFO [2022-01-10 08:39:47,858] jepsen worker 87 - jepsen.util 1087 :invoke :add 8314 INFO [2022-01-10 08:39:47,860] jepsen worker 342 - jepsen.util 1342 :invoke :add 8315 INFO [2022-01-10 08:39:47,863] jepsen worker 320 - jepsen.util 320 :info :add 7067 :RequestTimeoutException INFO [2022-01-10 08:39:47,864] jepsen worker 35 - jepsen.util 1035 :invoke :add 8316 INFO [2022-01-10 08:39:47,867] jepsen worker 204 - jepsen.util 1204 :invoke :add 8317 INFO [2022-01-10 08:39:47,870] jepsen worker 320 - jepsen.util 1320 :invoke :add 8318 INFO [2022-01-10 08:39:47,873] jepsen worker 437 - jepsen.util 1437 :invoke :add 8319 INFO [2022-01-10 08:39:47,876] jepsen worker 997 - jepsen.util 2997 :invoke :add 8320 INFO [2022-01-10 08:39:47,877] jepsen worker 221 - jepsen.util 221 :info :add 7074 :RequestTimeoutException INFO [2022-01-10 08:39:47,878] jepsen worker 977 - jepsen.util 1977 :invoke :add 8321 INFO [2022-01-10 08:39:47,881] jepsen worker 857 - jepsen.util 857 :info :add 7075 :RequestTimeoutException INFO [2022-01-10 08:39:47,882] jepsen worker 300 - jepsen.util 300 :info :add 7077 :RequestTimeoutException INFO [2022-01-10 08:39:47,882] jepsen worker 829 - jepsen.util 1829 :info :add 7079 :RequestTimeoutException INFO [2022-01-10 08:39:47,883] jepsen worker 221 - jepsen.util 1221 :invoke :add 8322 INFO [2022-01-10 08:39:47,884] jepsen worker 56 - jepsen.util 1056 :info :add 7076 :RequestTimeoutException INFO [2022-01-10 08:39:47,887] jepsen worker 497 - jepsen.util 497 :info :add 7078 :RequestTimeoutException INFO [2022-01-10 08:39:47,887] jepsen worker 108 - jepsen.util 1108 :info :add 7081 :RequestTimeoutException INFO [2022-01-10 08:39:47,890] jepsen worker 263 - jepsen.util 263 :info :add 7082 :RequestTimeoutException INFO [2022-01-10 08:39:47,891] jepsen worker 280 - jepsen.util 280 :info :add 7080 :RequestTimeoutException INFO [2022-01-10 08:39:47,892] jepsen worker 516 - jepsen.util 516 :info :add 7083 :RequestTimeoutException INFO [2022-01-10 08:39:47,892] jepsen worker 108 - jepsen.util 2108 :invoke :add 8323 INFO [2022-01-10 08:39:47,894] jepsen worker 290 - jepsen.util 1290 :info :add 7085 :RequestTimeoutException INFO [2022-01-10 08:39:47,899] jepsen worker 290 - jepsen.util 2290 :invoke :add 8324 INFO [2022-01-10 08:39:47,902] jepsen worker 504 - jepsen.util 1504 :invoke :add 8325 INFO [2022-01-10 08:39:47,903] jepsen worker 523 - jepsen.util 523 :info :add 7087 :RequestTimeoutException INFO [2022-01-10 08:39:47,905] jepsen worker 56 - jepsen.util 2056 :invoke :add 8326 INFO [2022-01-10 08:39:47,909] jepsen worker 300 - jepsen.util 1300 :invoke :add 8327 INFO [2022-01-10 08:39:47,912] jepsen worker 516 - jepsen.util 1516 :invoke :add 8328 INFO [2022-01-10 08:39:47,914] jepsen worker 497 - jepsen.util 1497 :invoke :add 8329 INFO [2022-01-10 08:39:47,917] jepsen worker 263 - jepsen.util 1263 :invoke :add 8330 INFO [2022-01-10 08:39:47,918] jepsen worker 516 - jepsen.util 1516 :ok :add 8328 INFO [2022-01-10 08:39:47,920] jepsen worker 280 - jepsen.util 1280 :invoke :add 8331 INFO [2022-01-10 08:39:47,923] jepsen worker 857 - jepsen.util 1857 :invoke :add 8332 INFO [2022-01-10 08:39:47,926] jepsen worker 829 - jepsen.util 2829 :invoke :add 8333 INFO [2022-01-10 08:39:47,928] jepsen worker 523 - jepsen.util 1523 :invoke :add 8334 INFO [2022-01-10 08:39:47,929] jepsen worker 857 - jepsen.util 1857 :ok :add 8332 INFO [2022-01-10 08:39:47,931] jepsen worker 516 - jepsen.util 1516 :invoke :add 8335 INFO [2022-01-10 08:39:47,935] jepsen worker 857 - jepsen.util 1857 :invoke :add 8336 INFO [2022-01-10 08:39:48,003] jepsen worker 609 - jepsen.util 1609 :info :add 7094 :RequestTimeoutException INFO [2022-01-10 08:39:48,008] jepsen worker 609 - jepsen.util 2609 :invoke :add 8337 INFO [2022-01-10 08:39:48,011] jepsen worker 627 - jepsen.util 1627 :info :add 7095 :RequestTimeoutException INFO [2022-01-10 08:39:48,017] jepsen worker 627 - jepsen.util 2627 :invoke :add 8338 INFO [2022-01-10 08:39:48,023] jepsen worker 244 - jepsen.util 1244 :info :add 7096 :RequestTimeoutException INFO [2022-01-10 08:39:48,027] jepsen worker 244 - jepsen.util 2244 :invoke :add 8339 INFO [2022-01-10 08:39:48,032] jepsen worker 266 - jepsen.util 1266 :info :add 7097 :RequestTimeoutException INFO [2022-01-10 08:39:48,037] jepsen worker 916 - jepsen.util 1916 :info :add 7098 :RequestTimeoutException INFO [2022-01-10 08:39:48,037] jepsen worker 266 - jepsen.util 2266 :invoke :add 8340 INFO [2022-01-10 08:39:48,041] jepsen worker 916 - jepsen.util 2916 :invoke :add 8341 INFO [2022-01-10 08:39:48,045] jepsen worker 266 - jepsen.util 2266 :ok :add 8340 INFO [2022-01-10 08:39:48,048] jepsen worker 266 - jepsen.util 2266 :invoke :add 8342 INFO [2022-01-10 08:39:48,053] jepsen worker 266 - jepsen.util 2266 :ok :add 8342 INFO [2022-01-10 08:39:48,058] jepsen worker 266 - jepsen.util 2266 :invoke :add 8343 INFO [2022-01-10 08:39:48,065] jepsen worker 266 - jepsen.util 2266 :ok :add 8343 INFO [2022-01-10 08:39:48,069] jepsen worker 266 - jepsen.util 2266 :invoke :add 8344 INFO [2022-01-10 08:39:48,102] jepsen worker 222 - jepsen.util 222 :info :add 7104 :RequestTimeoutException INFO [2022-01-10 08:39:48,108] jepsen worker 222 - jepsen.util 1222 :invoke :add 8345 INFO [2022-01-10 08:39:48,111] jepsen worker 604 - jepsen.util 604 :info :add 7107 :RequestTimeoutException INFO [2022-01-10 08:39:48,111] jepsen worker 861 - jepsen.util 861 :info :add 7108 :RequestTimeoutException INFO [2022-01-10 08:39:48,112] jepsen worker 43 - jepsen.util 43 :info :add 7106 :RequestTimeoutException INFO [2022-01-10 08:39:48,114] jepsen worker 120 - jepsen.util 120 :info :add 7105 :RequestTimeoutException INFO [2022-01-10 08:39:48,120] jepsen worker 43 - jepsen.util 1043 :invoke :add 8346 INFO [2022-01-10 08:39:48,122] jepsen worker 639 - jepsen.util 639 :info :add 7110 :RequestTimeoutException INFO [2022-01-10 08:39:48,122] jepsen worker 459 - jepsen.util 459 :info :add 7109 :RequestTimeoutException INFO [2022-01-10 08:39:48,123] jepsen worker 861 - jepsen.util 1861 :invoke :add 8347 INFO [2022-01-10 08:39:48,127] jepsen worker 871 - jepsen.util 871 :info :add 7112 :RequestTimeoutException INFO [2022-01-10 08:39:48,128] jepsen worker 892 - jepsen.util 1892 :info :add 7113 :RequestTimeoutException INFO [2022-01-10 08:39:48,129] jepsen worker 604 - jepsen.util 1604 :invoke :add 8348 INFO [2022-01-10 08:39:48,136] jepsen worker 120 - jepsen.util 1120 :invoke :add 8349 INFO [2022-01-10 08:39:48,141] jepsen worker 892 - jepsen.util 2892 :invoke :add 8350 INFO [2022-01-10 08:39:48,145] jepsen worker 871 - jepsen.util 1871 :invoke :add 8351 INFO [2022-01-10 08:39:48,148] jepsen worker 459 - jepsen.util 1459 :invoke :add 8352 INFO [2022-01-10 08:39:48,152] jepsen worker 639 - jepsen.util 1639 :invoke :add 8353 INFO [2022-01-10 08:39:48,163] jepsen worker 701 - jepsen.util 1701 :info :add 7115 :RequestTimeoutException INFO [2022-01-10 08:39:48,170] jepsen worker 701 - jepsen.util 2701 :invoke :add 8354 INFO [2022-01-10 08:39:48,253] jepsen worker 868 - jepsen.util 1868 :info :add 7121 :RequestTimeoutException INFO [2022-01-10 08:39:48,257] jepsen worker 868 - jepsen.util 2868 :invoke :add 8355 INFO [2022-01-10 08:39:48,263] jepsen worker 194 - jepsen.util 1194 :info :add 7124 :RequestTimeoutException INFO [2022-01-10 08:39:48,267] jepsen worker 194 - jepsen.util 2194 :invoke :add 8356 INFO [2022-01-10 08:39:48,270] jepsen worker 278 - jepsen.util 1278 :info :add 7126 :RequestTimeoutException INFO [2022-01-10 08:39:48,274] jepsen worker 278 - jepsen.util 2278 :invoke :add 8357 INFO [2022-01-10 08:39:48,277] jepsen worker 660 - jepsen.util 1660 :info :add 7125 :RequestTimeoutException INFO [2022-01-10 08:39:48,283] jepsen worker 660 - jepsen.util 2660 :invoke :add 8358 INFO [2022-01-10 08:39:48,293] jepsen worker 393 - jepsen.util 1393 :info :add 7127 :RequestTimeoutException INFO [2022-01-10 08:39:48,296] jepsen worker 393 - jepsen.util 2393 :invoke :add 8359 INFO [2022-01-10 08:39:48,332] jepsen worker 123 - jepsen.util 1123 :info :add 7128 :RequestTimeoutException INFO [2022-01-10 08:39:48,332] jepsen worker 81 - jepsen.util 1081 :info :add 7130 :RequestTimeoutException INFO [2022-01-10 08:39:48,334] jepsen worker 571 - jepsen.util 1571 :info :add 7129 :RequestTimeoutException INFO [2022-01-10 08:39:48,336] jepsen worker 322 - jepsen.util 1322 :info :add 7131 :RequestTimeoutException INFO [2022-01-10 08:39:48,339] jepsen worker 123 - jepsen.util 2123 :invoke :add 8360 INFO [2022-01-10 08:39:48,342] jepsen worker 571 - jepsen.util 2571 :invoke :add 8361 INFO [2022-01-10 08:39:48,344] jepsen worker 322 - jepsen.util 2322 :invoke :add 8362 INFO [2022-01-10 08:39:48,347] jepsen worker 81 - jepsen.util 2081 :invoke :add 8363 INFO [2022-01-10 08:39:48,351] jepsen worker 235 - jepsen.util 1235 :info :add 7132 :RequestTimeoutException INFO [2022-01-10 08:39:48,356] jepsen worker 235 - jepsen.util 2235 :invoke :add 8364 INFO [2022-01-10 08:39:48,372] jepsen worker 928 - jepsen.util 1928 :info :add 7133 :RequestTimeoutException INFO [2022-01-10 08:39:48,377] jepsen worker 928 - jepsen.util 2928 :invoke :add 8365 INFO [2022-01-10 08:39:48,498] jepsen worker 467 - jepsen.util 1467 :ok :add 7464 INFO [2022-01-10 08:39:48,498] jepsen worker 411 - jepsen.util 1411 :ok :add 7448 INFO [2022-01-10 08:39:48,499] jepsen worker 577 - jepsen.util 1577 :info :add 7155 :RequestTimeoutException INFO [2022-01-10 08:39:48,499] jepsen worker 73 - jepsen.util 73 :info :add 7140 :RequestTimeoutException INFO [2022-01-10 08:39:48,500] jepsen worker 272 - jepsen.util 1272 :info :add 7139 :RequestTimeoutException INFO [2022-01-10 08:39:48,500] jepsen worker 620 - jepsen.util 1620 :info :add 7148 :RequestTimeoutException INFO [2022-01-10 08:39:48,500] jepsen worker 19 - jepsen.util 1019 :info :add 7156 :RequestTimeoutException INFO [2022-01-10 08:39:48,500] jepsen worker 852 - jepsen.util 852 :info :add 7141 :RequestTimeoutException INFO [2022-01-10 08:39:48,500] jepsen worker 415 - jepsen.util 415 :info :add 7142 :RequestTimeoutException INFO [2022-01-10 08:39:48,500] jepsen worker 374 - jepsen.util 374 :info :add 7152 :RequestTimeoutException INFO [2022-01-10 08:39:48,500] jepsen worker 473 - jepsen.util 473 :info :add 7151 :RequestTimeoutException INFO [2022-01-10 08:39:48,500] jepsen worker 424 - jepsen.util 424 :info :add 7143 :RequestTimeoutException INFO [2022-01-10 08:39:48,501] jepsen worker 458 - jepsen.util 1458 :info :add 7154 :RequestTimeoutException INFO [2022-01-10 08:39:48,501] jepsen worker 173 - jepsen.util 173 :info :add 7150 :RequestTimeoutException INFO [2022-01-10 08:39:48,501] jepsen worker 318 - jepsen.util 318 :info :add 7157 :RequestTimeoutException INFO [2022-01-10 08:39:48,501] jepsen worker 250 - jepsen.util 1250 :ok :add 7322 INFO [2022-01-10 08:39:48,501] jepsen worker 306 - jepsen.util 1306 :ok :add 7286 INFO [2022-01-10 08:39:48,502] jepsen worker 51 - jepsen.util 1051 :ok :add 7316 INFO [2022-01-10 08:39:48,510] jepsen worker 434 - jepsen.util 434 :info :add 7158 :RequestTimeoutException INFO [2022-01-10 08:39:48,513] jepsen worker 869 - jepsen.util 1869 :info :add 7162 :RequestTimeoutException INFO [2022-01-10 08:39:48,514] jepsen worker 96 - jepsen.util 96 :info :add 7159 :RequestTimeoutException INFO [2022-01-10 08:39:48,514] jepsen worker 655 - jepsen.util 655 :info :add 7161 :RequestTimeoutException INFO [2022-01-10 08:39:48,517] jepsen worker 129 - jepsen.util 129 :info :add 7160 :RequestTimeoutException INFO [2022-01-10 08:39:48,520] jepsen worker 655 - jepsen.util 1655 :invoke :add 8366 INFO [2022-01-10 08:39:48,522] jepsen worker 473 - jepsen.util 1473 :invoke :add 8367 INFO [2022-01-10 08:39:48,523] jepsen worker 933 - jepsen.util 933 :info :add 7165 :RequestTimeoutException INFO [2022-01-10 08:39:48,523] jepsen worker 948 - jepsen.util 948 :info :add 7166 :RequestTimeoutException INFO [2022-01-10 08:39:48,524] jepsen worker 467 - jepsen.util 1467 :invoke :add 8368 INFO [2022-01-10 08:39:48,528] jepsen worker 577 - jepsen.util 2577 :invoke :add 8369 INFO [2022-01-10 08:39:48,529] jepsen worker 869 - jepsen.util 2869 :invoke :add 8370 INFO [2022-01-10 08:39:48,531] jepsen worker 852 - jepsen.util 1852 :invoke :add 8371 INFO [2022-01-10 08:39:48,533] jepsen worker 173 - jepsen.util 1173 :invoke :add 8372 INFO [2022-01-10 08:39:48,535] jepsen worker 424 - jepsen.util 1424 :invoke :add 8373 INFO [2022-01-10 08:39:48,537] jepsen worker 96 - jepsen.util 1096 :invoke :add 8374 INFO [2022-01-10 08:39:48,538] jepsen worker 250 - jepsen.util 1250 :invoke :add 8375 INFO [2022-01-10 08:39:48,540] jepsen worker 415 - jepsen.util 1415 :invoke :add 8376 INFO [2022-01-10 08:39:48,541] jepsen worker 933 - jepsen.util 1933 :invoke :add 8377 INFO [2022-01-10 08:39:48,542] jepsen worker 182 - jepsen.util 182 :info :add 7172 :RequestTimeoutException INFO [2022-01-10 08:39:48,544] jepsen worker 434 - jepsen.util 1434 :invoke :add 8378 INFO [2022-01-10 08:39:48,544] jepsen worker 310 - jepsen.util 1310 :info :add 7174 :RequestTimeoutException INFO [2022-01-10 08:39:48,544] jepsen worker 96 - jepsen.util 1096 :ok :add 8374 INFO [2022-01-10 08:39:48,547] jepsen worker 19 - jepsen.util 2019 :invoke :add 8379 INFO [2022-01-10 08:39:48,549] jepsen worker 310 - jepsen.util 2310 :invoke :add 8380 INFO [2022-01-10 08:39:48,550] jepsen worker 933 - jepsen.util 1933 :ok :add 8377 INFO [2022-01-10 08:39:48,551] jepsen worker 458 - jepsen.util 2458 :invoke :add 8381 INFO [2022-01-10 08:39:48,553] jepsen worker 411 - jepsen.util 1411 :invoke :add 8382 INFO [2022-01-10 08:39:48,555] jepsen worker 318 - jepsen.util 1318 :invoke :add 8383 INFO [2022-01-10 08:39:48,556] jepsen worker 129 - jepsen.util 1129 :invoke :add 8384 INFO [2022-01-10 08:39:48,559] jepsen worker 96 - jepsen.util 1096 :invoke :add 8385 INFO [2022-01-10 08:39:48,561] jepsen worker 948 - jepsen.util 1948 :invoke :add 8386 INFO [2022-01-10 08:39:48,563] jepsen worker 51 - jepsen.util 1051 :invoke :add 8387 INFO [2022-01-10 08:39:48,564] jepsen worker 620 - jepsen.util 2620 :invoke :add 8388 INFO [2022-01-10 08:39:48,566] jepsen worker 933 - jepsen.util 1933 :invoke :add 8389 INFO [2022-01-10 08:39:48,568] jepsen worker 374 - jepsen.util 1374 :invoke :add 8390 INFO [2022-01-10 08:39:48,570] jepsen worker 73 - jepsen.util 1073 :invoke :add 8391 INFO [2022-01-10 08:39:48,571] jepsen worker 182 - jepsen.util 1182 :invoke :add 8392 INFO [2022-01-10 08:39:48,573] jepsen worker 272 - jepsen.util 2272 :invoke :add 8393 INFO [2022-01-10 08:39:48,575] jepsen worker 306 - jepsen.util 1306 :invoke :add 8394 INFO [2022-01-10 08:39:48,580] jepsen worker 306 - jepsen.util 1306 :ok :add 8394 INFO [2022-01-10 08:39:48,583] jepsen worker 89 - jepsen.util 1089 :info :add 7175 :RequestTimeoutException INFO [2022-01-10 08:39:48,583] jepsen worker 306 - jepsen.util 1306 :invoke :add 8395 INFO [2022-01-10 08:39:48,585] jepsen worker 89 - jepsen.util 2089 :invoke :add 8396 INFO [2022-01-10 08:39:48,607] jepsen worker 508 - jepsen.util 1508 :info :add 7176 :RequestTimeoutException INFO [2022-01-10 08:39:48,611] jepsen worker 106 - jepsen.util 1106 :info :add 7177 :RequestTimeoutException INFO [2022-01-10 08:39:48,611] jepsen worker 508 - jepsen.util 2508 :invoke :add 8397 INFO [2022-01-10 08:39:48,615] jepsen worker 106 - jepsen.util 2106 :invoke :add 8398 INFO [2022-01-10 08:39:48,623] jepsen worker 551 - jepsen.util 1551 :info :add 7178 :RequestTimeoutException INFO [2022-01-10 08:39:48,626] jepsen worker 551 - jepsen.util 2551 :invoke :add 8399 INFO [2022-01-10 08:39:48,632] jepsen worker 998 - jepsen.util 1998 :info :add 7179 :RequestTimeoutException INFO [2022-01-10 08:39:48,635] jepsen worker 998 - jepsen.util 2998 :invoke :add 8400 INFO [2022-01-10 08:39:48,652] jepsen worker 645 - jepsen.util 1645 :info :add 7181 :RequestTimeoutException INFO [2022-01-10 08:39:48,653] jepsen worker 608 - jepsen.util 1608 :info :add 7180 :RequestTimeoutException INFO [2022-01-10 08:39:48,654] jepsen worker 645 - jepsen.util 2645 :invoke :add 8401 INFO [2022-01-10 08:39:48,657] jepsen worker 608 - jepsen.util 2608 :invoke :add 8402 INFO [2022-01-10 08:39:48,657] jepsen worker 249 - jepsen.util 1249 :info :add 7182 :RequestTimeoutException INFO [2022-01-10 08:39:48,660] jepsen worker 249 - jepsen.util 2249 :invoke :add 8403 INFO [2022-01-10 08:39:48,660] jepsen worker 800 - jepsen.util 1800 :info :add 7183 :RequestTimeoutException INFO [2022-01-10 08:39:48,664] jepsen worker 800 - jepsen.util 2800 :invoke :add 8404 INFO [2022-01-10 08:39:48,681] jepsen worker 633 - jepsen.util 633 :info :add 7184 :RequestTimeoutException INFO [2022-01-10 08:39:48,683] jepsen worker 362 - jepsen.util 362 :info :add 7185 :RequestTimeoutException INFO [2022-01-10 08:39:48,683] jepsen worker 668 - jepsen.util 668 :info :add 7186 :RequestTimeoutException INFO [2022-01-10 08:39:48,684] jepsen worker 633 - jepsen.util 1633 :invoke :add 8405 INFO [2022-01-10 08:39:48,687] jepsen worker 668 - jepsen.util 1668 :invoke :add 8406 INFO [2022-01-10 08:39:48,689] jepsen worker 362 - jepsen.util 1362 :invoke :add 8407 INFO [2022-01-10 08:39:48,693] jepsen worker 633 - jepsen.util 1633 :ok :add 8405 INFO [2022-01-10 08:39:48,695] jepsen worker 633 - jepsen.util 1633 :invoke :add 8408 INFO [2022-01-10 08:39:48,732] jepsen worker 550 - jepsen.util 1550 :info :add 7194 :RequestTimeoutException INFO [2022-01-10 08:39:48,733] jepsen worker 595 - jepsen.util 1595 :info :add 7195 :RequestTimeoutException INFO [2022-01-10 08:39:48,734] jepsen worker 550 - jepsen.util 2550 :invoke :add 8409 INFO [2022-01-10 08:39:48,736] jepsen worker 595 - jepsen.util 2595 :invoke :add 8410 INFO [2022-01-10 08:39:48,737] jepsen worker 234 - jepsen.util 1234 :info :add 7197 :RequestTimeoutException INFO [2022-01-10 08:39:48,740] jepsen worker 234 - jepsen.util 2234 :invoke :add 8411 INFO [2022-01-10 08:39:48,740] jepsen worker 692 - jepsen.util 1692 :info :add 7198 :RequestTimeoutException INFO [2022-01-10 08:39:48,742] jepsen worker 991 - jepsen.util 1991 :info :add 7196 :RequestTimeoutException INFO [2022-01-10 08:39:48,744] jepsen worker 692 - jepsen.util 2692 :invoke :add 8412 INFO [2022-01-10 08:39:48,746] jepsen worker 991 - jepsen.util 2991 :invoke :add 8413 INFO [2022-01-10 08:39:48,823] jepsen worker 427 - jepsen.util 1427 :info :add 7199 :RequestTimeoutException INFO [2022-01-10 08:39:48,826] jepsen worker 427 - jepsen.util 2427 :invoke :add 8414 INFO [2022-01-10 08:39:48,861] jepsen worker 953 - jepsen.util 1953 :info :add 7204 :RequestTimeoutException INFO [2022-01-10 08:39:48,863] jepsen worker 201 - jepsen.util 1201 :info :add 7205 :RequestTimeoutException INFO [2022-01-10 08:39:48,864] jepsen worker 953 - jepsen.util 2953 :invoke :add 8415 INFO [2022-01-10 08:39:48,866] jepsen worker 201 - jepsen.util 2201 :invoke :add 8416 INFO [2022-01-10 08:39:48,883] jepsen worker 767 - jepsen.util 767 :info :add 7206 :RequestTimeoutException INFO [2022-01-10 08:39:48,883] jepsen worker 193 - jepsen.util 193 :info :add 7208 :RequestTimeoutException INFO [2022-01-10 08:39:48,884] jepsen worker 455 - jepsen.util 455 :info :add 7209 :RequestTimeoutException INFO [2022-01-10 08:39:48,884] jepsen worker 325 - jepsen.util 325 :info :add 7207 :RequestTimeoutException INFO [2022-01-10 08:39:48,888] jepsen worker 767 - jepsen.util 1767 :invoke :add 8417 INFO [2022-01-10 08:39:48,889] jepsen worker 455 - jepsen.util 1455 :invoke :add 8418 INFO [2022-01-10 08:39:48,891] jepsen worker 493 - jepsen.util 493 :info :add 7211 :RequestTimeoutException INFO [2022-01-10 08:39:48,891] jepsen worker 325 - jepsen.util 1325 :invoke :add 8419 INFO [2022-01-10 08:39:48,892] jepsen worker 219 - jepsen.util 219 :info :add 7210 :RequestTimeoutException INFO [2022-01-10 08:39:48,893] jepsen worker 193 - jepsen.util 1193 :invoke :add 8420 INFO [2022-01-10 08:39:48,896] jepsen worker 493 - jepsen.util 1493 :invoke :add 8421 INFO [2022-01-10 08:39:48,898] jepsen worker 219 - jepsen.util 1219 :invoke :add 8422 INFO [2022-01-10 08:39:48,941] jepsen worker 91 - jepsen.util 1091 :info :add 7212 :RequestTimeoutException INFO [2022-01-10 08:39:48,944] jepsen worker 756 - jepsen.util 1756 :info :add 7213 :RequestTimeoutException INFO [2022-01-10 08:39:48,944] jepsen worker 91 - jepsen.util 2091 :invoke :add 8423 INFO [2022-01-10 08:39:48,947] jepsen worker 756 - jepsen.util 2756 :invoke :add 8424 INFO [2022-01-10 08:39:48,952] jepsen worker 448 - jepsen.util 1448 :info :add 7214 :RequestTimeoutException INFO [2022-01-10 08:39:48,955] jepsen worker 448 - jepsen.util 2448 :invoke :add 8425 INFO [2022-01-10 08:39:48,963] jepsen worker 44 - jepsen.util 1044 :info :add 7217 :RequestTimeoutException INFO [2022-01-10 08:39:48,966] jepsen worker 44 - jepsen.util 2044 :invoke :add 8426 INFO [2022-01-10 08:39:48,992] jepsen worker 258 - jepsen.util 1258 :info :add 7220 :RequestTimeoutException INFO [2022-01-10 08:39:48,995] jepsen worker 258 - jepsen.util 2258 :invoke :add 8427 INFO [2022-01-10 08:39:49,073] jepsen worker 714 - jepsen.util 1714 :info :add 7223 :RequestTimeoutException INFO [2022-01-10 08:39:49,076] jepsen worker 714 - jepsen.util 2714 :invoke :add 8428 INFO [2022-01-10 08:39:49,081] jepsen worker 899 - jepsen.util 1899 :info :add 7224 :RequestTimeoutException INFO [2022-01-10 08:39:49,082] jepsen worker 381 - jepsen.util 1381 :info :add 7226 :RequestTimeoutException INFO [2022-01-10 08:39:49,083] jepsen worker 899 - jepsen.util 2899 :invoke :add 8429 INFO [2022-01-10 08:39:49,084] jepsen worker 522 - jepsen.util 1522 :info :add 7225 :RequestTimeoutException INFO [2022-01-10 08:39:49,085] jepsen worker 381 - jepsen.util 2381 :invoke :add 8430 INFO [2022-01-10 08:39:49,088] jepsen worker 522 - jepsen.util 2522 :invoke :add 8431 INFO [2022-01-10 08:39:49,091] jepsen worker 381 - jepsen.util 2381 :ok :add 8430 INFO [2022-01-10 08:39:49,094] jepsen worker 749 - jepsen.util 1749 :ok :add 7522 INFO [2022-01-10 08:39:49,094] jepsen worker 911 - jepsen.util 1911 :ok :add 7250 INFO [2022-01-10 08:39:49,094] jepsen worker 835 - jepsen.util 835 :ok :add 7516 INFO [2022-01-10 08:39:49,094] jepsen worker 556 - jepsen.util 556 :ok :add 7486 INFO [2022-01-10 08:39:49,094] jepsen worker 381 - jepsen.util 2381 :invoke :add 8432 INFO [2022-01-10 08:39:49,097] jepsen worker 911 - jepsen.util 1911 :invoke :add 8433 INFO [2022-01-10 08:39:49,098] jepsen worker 749 - jepsen.util 1749 :invoke :add 8434 INFO [2022-01-10 08:39:49,099] jepsen worker 636 - jepsen.util 636 :ok :add 7351 INFO [2022-01-10 08:39:49,099] jepsen worker 381 - jepsen.util 2381 :ok :add 8432 INFO [2022-01-10 08:39:49,100] jepsen worker 399 - jepsen.util 1399 :ok :add 7423 INFO [2022-01-10 08:39:49,100] jepsen worker 820 - jepsen.util 1820 :ok :add 7417 INFO [2022-01-10 08:39:49,100] jepsen worker 835 - jepsen.util 835 :invoke :add 8435 INFO [2022-01-10 08:39:49,103] jepsen worker 399 - jepsen.util 1399 :invoke :add 8436 INFO [2022-01-10 08:39:49,104] jepsen worker 636 - jepsen.util 636 :invoke :add 8437 INFO [2022-01-10 08:39:49,106] jepsen worker 381 - jepsen.util 2381 :invoke :add 8438 INFO [2022-01-10 08:39:49,108] jepsen worker 556 - jepsen.util 556 :invoke :add 8439 INFO [2022-01-10 08:39:49,109] jepsen worker 820 - jepsen.util 1820 :invoke :add 8440 INFO [2022-01-10 08:39:49,115] jepsen worker 820 - jepsen.util 1820 :ok :add 8440 INFO [2022-01-10 08:39:49,118] jepsen worker 820 - jepsen.util 1820 :invoke :add 8441 INFO [2022-01-10 08:39:49,120] jepsen worker 128 - jepsen.util 1128 :info :add 7228 :RequestTimeoutException INFO [2022-01-10 08:39:49,122] jepsen worker 820 - jepsen.util 1820 :ok :add 8441 INFO [2022-01-10 08:39:49,122] jepsen worker 128 - jepsen.util 2128 :invoke :add 8442 INFO [2022-01-10 08:39:49,123] jepsen worker 786 - jepsen.util 1786 :info :add 7229 :RequestTimeoutException INFO [2022-01-10 08:39:49,127] jepsen worker 786 - jepsen.util 2786 :invoke :add 8443 INFO [2022-01-10 08:39:49,128] jepsen worker 128 - jepsen.util 2128 :ok :add 8442 INFO [2022-01-10 08:39:49,129] jepsen worker 820 - jepsen.util 1820 :invoke :add 8444 INFO [2022-01-10 08:39:49,130] jepsen worker 128 - jepsen.util 2128 :invoke :add 8445 INFO [2022-01-10 08:39:49,211] jepsen worker 139 - jepsen.util 1139 :info :add 7230 :RequestTimeoutException INFO [2022-01-10 08:39:49,214] jepsen worker 139 - jepsen.util 2139 :invoke :add 8446 INFO [2022-01-10 08:39:49,216] jepsen worker 536 - jepsen.util 1536 :info :add 7231 :RequestTimeoutException INFO [2022-01-10 08:39:49,219] jepsen worker 536 - jepsen.util 2536 :invoke :add 8447 INFO [2022-01-10 08:39:49,222] jepsen worker 10 - jepsen.util 1010 :info :add 7232 :RequestTimeoutException INFO [2022-01-10 08:39:49,226] jepsen worker 10 - jepsen.util 2010 :invoke :add 8448 INFO [2022-01-10 08:39:49,227] jepsen worker 297 - jepsen.util 1297 :info :add 7233 :RequestTimeoutException INFO [2022-01-10 08:39:49,230] jepsen worker 297 - jepsen.util 2297 :invoke :add 8449 INFO [2022-01-10 08:39:49,283] jepsen worker 984 - jepsen.util 1984 :info :add 7239 :RequestTimeoutException INFO [2022-01-10 08:39:49,285] jepsen worker 984 - jepsen.util 2984 :invoke :add 8450 INFO [2022-01-10 08:39:49,286] jepsen worker 305 - jepsen.util 1305 :info :add 7240 :RequestTimeoutException INFO [2022-01-10 08:39:49,289] jepsen worker 305 - jepsen.util 2305 :invoke :add 8451 INFO [2022-01-10 08:39:49,290] jepsen worker 312 - jepsen.util 1312 :info :add 7241 :RequestTimeoutException INFO [2022-01-10 08:39:49,293] jepsen worker 312 - jepsen.util 2312 :invoke :add 8452 INFO [2022-01-10 08:39:49,294] jepsen worker 63 - jepsen.util 1063 :info :add 7242 :RequestTimeoutException INFO [2022-01-10 08:39:49,299] jepsen worker 63 - jepsen.util 2063 :invoke :add 8453 INFO [2022-01-10 08:39:49,302] jepsen worker 341 - jepsen.util 1341 :info :add 7243 :RequestTimeoutException INFO [2022-01-10 08:39:49,304] jepsen worker 341 - jepsen.util 2341 :invoke :add 8454 INFO [2022-01-10 08:39:49,311] jepsen worker 828 - jepsen.util 1828 :info :add 7249 :RequestTimeoutException INFO [2022-01-10 08:39:49,313] jepsen worker 922 - jepsen.util 1922 :info :add 7248 :RequestTimeoutException INFO [2022-01-10 08:39:49,314] jepsen worker 828 - jepsen.util 2828 :invoke :add 8455 INFO [2022-01-10 08:39:49,317] jepsen worker 922 - jepsen.util 2922 :invoke :add 8456 INFO [2022-01-10 08:39:49,343] jepsen worker 117 - jepsen.util 1117 :info :add 7251 :RequestTimeoutException INFO [2022-01-10 08:39:49,346] jepsen worker 117 - jepsen.util 2117 :invoke :add 8457 INFO [2022-01-10 08:39:49,351] jepsen worker 952 - jepsen.util 952 :info :add 7252 :RequestTimeoutException INFO [2022-01-10 08:39:49,353] jepsen worker 754 - jepsen.util 754 :info :add 7253 :RequestTimeoutException INFO [2022-01-10 08:39:49,353] jepsen worker 952 - jepsen.util 1952 :invoke :add 8458 INFO [2022-01-10 08:39:49,356] jepsen worker 754 - jepsen.util 1754 :invoke :add 8459 INFO [2022-01-10 08:39:49,357] jepsen worker 179 - jepsen.util 1179 :info :add 7254 :RequestTimeoutException INFO [2022-01-10 08:39:49,359] jepsen worker 179 - jepsen.util 2179 :invoke :add 8460 INFO [2022-01-10 08:39:49,370] jepsen worker 412 - jepsen.util 1412 :info :add 7255 :RequestTimeoutException INFO [2022-01-10 08:39:49,371] jepsen worker 678 - jepsen.util 678 :info :add 7257 :RequestTimeoutException INFO [2022-01-10 08:39:49,373] jepsen worker 680 - jepsen.util 680 :info :add 7256 :RequestTimeoutException INFO [2022-01-10 08:39:49,374] jepsen worker 90 - jepsen.util 90 :info :add 7258 :RequestTimeoutException INFO [2022-01-10 08:39:49,374] jepsen worker 412 - jepsen.util 2412 :invoke :add 8461 INFO [2022-01-10 08:39:49,377] jepsen worker 831 - jepsen.util 831 :info :add 7260 :RequestTimeoutException INFO [2022-01-10 08:39:49,377] jepsen worker 680 - jepsen.util 1680 :invoke :add 8462 INFO [2022-01-10 08:39:49,380] jepsen worker 831 - jepsen.util 1831 :invoke :add 8463 INFO [2022-01-10 08:39:49,381] jepsen worker 90 - jepsen.util 1090 :invoke :add 8464 INFO [2022-01-10 08:39:49,382] jepsen worker 612 - jepsen.util 612 :info :add 7259 :RequestTimeoutException INFO [2022-01-10 08:39:49,383] jepsen worker 717 - jepsen.util 717 :info :add 7261 :RequestTimeoutException INFO [2022-01-10 08:39:49,383] jepsen worker 12 - jepsen.util 12 :info :add 7262 :RequestTimeoutException INFO [2022-01-10 08:39:49,383] jepsen worker 155 - jepsen.util 155 :info :add 7263 :RequestTimeoutException INFO [2022-01-10 08:39:49,384] jepsen worker 482 - jepsen.util 482 :info :add 7264 :RequestTimeoutException INFO [2022-01-10 08:39:49,384] jepsen worker 678 - jepsen.util 1678 :invoke :add 8465 INFO [2022-01-10 08:39:49,389] jepsen worker 155 - jepsen.util 1155 :invoke :add 8466 INFO [2022-01-10 08:39:49,391] jepsen worker 717 - jepsen.util 1717 :invoke :add 8467 INFO [2022-01-10 08:39:49,393] jepsen worker 12 - jepsen.util 1012 :invoke :add 8468 INFO [2022-01-10 08:39:49,393] jepsen worker 253 - jepsen.util 1253 :info :add 7265 :RequestTimeoutException INFO [2022-01-10 08:39:49,397] jepsen worker 721 - jepsen.util 721 :info :add 7266 :RequestTimeoutException INFO [2022-01-10 08:39:49,397] jepsen worker 253 - jepsen.util 2253 :invoke :add 8469 INFO [2022-01-10 08:39:49,399] jepsen worker 482 - jepsen.util 1482 :invoke :add 8470 INFO [2022-01-10 08:39:49,401] jepsen worker 612 - jepsen.util 1612 :invoke :add 8471 INFO [2022-01-10 08:39:49,403] jepsen worker 721 - jepsen.util 1721 :invoke :add 8472 INFO [2022-01-10 08:39:49,412] jepsen worker 837 - jepsen.util 1837 :info :add 7274 :RequestTimeoutException INFO [2022-01-10 08:39:49,414] jepsen worker 720 - jepsen.util 1720 :info :add 7275 :RequestTimeoutException INFO [2022-01-10 08:39:49,416] jepsen worker 837 - jepsen.util 2837 :invoke :add 8473 INFO [2022-01-10 08:39:49,417] jepsen worker 962 - jepsen.util 1962 :info :add 7277 :RequestTimeoutException INFO [2022-01-10 08:39:49,419] jepsen worker 720 - jepsen.util 2720 :invoke :add 8474 INFO [2022-01-10 08:39:49,421] jepsen worker 962 - jepsen.util 2962 :invoke :add 8475 INFO [2022-01-10 08:39:49,422] jepsen worker 883 - jepsen.util 1883 :info :add 7276 :RequestTimeoutException INFO [2022-01-10 08:39:49,425] jepsen worker 883 - jepsen.util 2883 :invoke :add 8476 INFO [2022-01-10 08:39:49,430] jepsen worker 677 - jepsen.util 1677 :info :add 7278 :RequestTimeoutException INFO [2022-01-10 08:39:49,433] jepsen worker 480 - jepsen.util 1480 :info :add 7279 :RequestTimeoutException INFO [2022-01-10 08:39:49,433] jepsen worker 677 - jepsen.util 2677 :invoke :add 8477 INFO [2022-01-10 08:39:49,436] jepsen worker 480 - jepsen.util 2480 :invoke :add 8478 INFO [2022-01-10 08:39:49,441] jepsen worker 65 - jepsen.util 1065 :info :add 7280 :RequestTimeoutException INFO [2022-01-10 08:39:49,442] jepsen worker 784 - jepsen.util 1784 :info :add 7282 :RequestTimeoutException INFO [2022-01-10 08:39:49,443] jepsen worker 252 - jepsen.util 1252 :info :add 7281 :RequestTimeoutException INFO [2022-01-10 08:39:49,446] jepsen worker 65 - jepsen.util 2065 :invoke :add 8479 INFO [2022-01-10 08:39:49,447] jepsen worker 773 - jepsen.util 1773 :info :add 7283 :RequestTimeoutException INFO [2022-01-10 08:39:49,448] jepsen worker 784 - jepsen.util 2784 :invoke :add 8480 INFO [2022-01-10 08:39:49,450] jepsen worker 773 - jepsen.util 2773 :invoke :add 8481 INFO [2022-01-10 08:39:49,452] jepsen worker 252 - jepsen.util 2252 :invoke :add 8482 INFO [2022-01-10 08:39:49,463] jepsen worker 836 - jepsen.util 1836 :info :add 7285 :RequestTimeoutException INFO [2022-01-10 08:39:49,466] jepsen worker 836 - jepsen.util 2836 :invoke :add 8483 INFO [2022-01-10 08:39:49,513] jepsen worker 759 - jepsen.util 1759 :info :add 7294 :RequestTimeoutException INFO [2022-01-10 08:39:49,516] jepsen worker 759 - jepsen.util 2759 :invoke :add 8484 INFO [2022-01-10 08:39:49,517] jepsen worker 269 - jepsen.util 1269 :info :add 7295 :RequestTimeoutException INFO [2022-01-10 08:39:49,520] jepsen worker 269 - jepsen.util 2269 :invoke :add 8485 INFO [2022-01-10 08:39:49,520] jepsen worker 992 - jepsen.util 1992 :info :add 7296 :RequestTimeoutException INFO [2022-01-10 08:39:49,523] jepsen worker 985 - jepsen.util 1985 :info :add 7297 :RequestTimeoutException INFO [2022-01-10 08:39:49,524] jepsen worker 992 - jepsen.util 2992 :invoke :add 8486 INFO [2022-01-10 08:39:49,526] jepsen worker 985 - jepsen.util 2985 :invoke :add 8487 INFO [2022-01-10 08:39:49,531] jepsen worker 433 - jepsen.util 1433 :info :add 7298 :RequestTimeoutException INFO [2022-01-10 08:39:49,533] jepsen worker 723 - jepsen.util 1723 :info :add 7299 :RequestTimeoutException INFO [2022-01-10 08:39:49,535] jepsen worker 723 - jepsen.util 2723 :invoke :add 8488 INFO [2022-01-10 08:39:49,537] jepsen worker 433 - jepsen.util 2433 :invoke :add 8489 INFO [2022-01-10 08:39:49,572] jepsen worker 428 - jepsen.util 1428 :info :add 7304 :RequestTimeoutException INFO [2022-01-10 08:39:49,573] jepsen worker 673 - jepsen.util 1673 :info :add 7303 :RequestTimeoutException INFO [2022-01-10 08:39:49,574] jepsen worker 942 - jepsen.util 1942 :info :add 7305 :RequestTimeoutException INFO [2022-01-10 08:39:49,578] jepsen worker 428 - jepsen.util 2428 :invoke :add 8490 INFO [2022-01-10 08:39:49,581] jepsen worker 942 - jepsen.util 2942 :invoke :add 8491 INFO [2022-01-10 08:39:49,584] jepsen worker 673 - jepsen.util 2673 :invoke :add 8492 INFO [2022-01-10 08:39:49,596] jepsen worker 177 - jepsen.util 177 :ok :add 7520 INFO [2022-01-10 08:39:49,596] jepsen worker 904 - jepsen.util 1904 :ok :add 7484 INFO [2022-01-10 08:39:49,596] jepsen worker 34 - jepsen.util 1034 :ok :add 7538 INFO [2022-01-10 08:39:49,596] jepsen worker 462 - jepsen.util 462 :ok :add 7514 INFO [2022-01-10 08:39:49,602] jepsen worker 675 - jepsen.util 1675 :info :add 7306 :RequestTimeoutException INFO [2022-01-10 08:39:49,603] jepsen worker 462 - jepsen.util 462 :invoke :add 8493 INFO [2022-01-10 08:39:49,607] jepsen worker 503 - jepsen.util 1503 :info :add 7307 :RequestTimeoutException INFO [2022-01-10 08:39:49,608] jepsen worker 675 - jepsen.util 2675 :invoke :add 8494 INFO [2022-01-10 08:39:49,612] jepsen worker 503 - jepsen.util 2503 :invoke :add 8495 INFO [2022-01-10 08:39:49,614] jepsen worker 481 - jepsen.util 481 :ok :add 7811 INFO [2022-01-10 08:39:49,615] jepsen worker 177 - jepsen.util 177 :invoke :add 8496 INFO [2022-01-10 08:39:49,618] jepsen worker 481 - jepsen.util 481 :invoke :add 8497 INFO [2022-01-10 08:39:49,619] jepsen worker 457 - jepsen.util 457 :ok :add 8058 INFO [2022-01-10 08:39:49,619] jepsen worker 396 - jepsen.util 1396 :ok :add 7393 INFO [2022-01-10 08:39:49,619] jepsen worker 664 - jepsen.util 1664 :ok :add 7925 INFO [2022-01-10 08:39:49,619] jepsen worker 566 - jepsen.util 1566 :ok :add 8174 INFO [2022-01-10 08:39:49,619] jepsen worker 502 - jepsen.util 1502 :ok :add 7401 INFO [2022-01-10 08:39:49,619] jepsen worker 955 - jepsen.util 1955 :ok :add 7419 INFO [2022-01-10 08:39:49,619] jepsen worker 362 - jepsen.util 1362 :ok :add 8407 INFO [2022-01-10 08:39:49,620] jepsen worker 236 - jepsen.util 236 :ok :add 7999 INFO [2022-01-10 08:39:49,620] jepsen worker 75 - jepsen.util 1075 :ok :add 8158 INFO [2022-01-10 08:39:49,620] jepsen worker 299 - jepsen.util 2299 :ok :add 8205 INFO [2022-01-10 08:39:49,621] jepsen worker 64 - jepsen.util 1064 :info :add 7310 :RequestTimeoutException INFO [2022-01-10 08:39:49,622] jepsen worker 402 - jepsen.util 1402 :ok :add 7591 INFO [2022-01-10 08:39:49,622] jepsen worker 711 - jepsen.util 1711 :info :add 7312 :RequestTimeoutException INFO [2022-01-10 08:39:49,622] jepsen worker 383 - jepsen.util 383 :ok :add 7909 INFO [2022-01-10 08:39:49,628] jepsen worker 797 - jepsen.util 1797 :ok :add 8295 INFO [2022-01-10 08:39:49,633] jepsen worker 302 - jepsen.util 1302 :ok :add 7700 INFO [2022-01-10 08:39:49,633] jepsen worker 79 - jepsen.util 1079 :ok :add 8140 INFO [2022-01-10 08:39:49,634] jepsen worker 277 - jepsen.util 1277 :info :add 7309 :RequestTimeoutException INFO [2022-01-10 08:39:49,633] jepsen worker 944 - jepsen.util 944 :ok :add 7881 INFO [2022-01-10 08:39:49,633] jepsen worker 61 - jepsen.util 61 :ok :add 7855 INFO [2022-01-10 08:39:49,634] jepsen worker 365 - jepsen.util 1365 :info :add 7308 :RequestTimeoutException INFO [2022-01-10 08:39:49,633] jepsen worker 306 - jepsen.util 1306 :ok :add 8395 INFO [2022-01-10 08:39:49,633] jepsen worker 203 - jepsen.util 203 :ok :add 7545 INFO [2022-01-10 08:39:49,633] jepsen worker 34 - jepsen.util 1034 :invoke :add 8498 INFO [2022-01-10 08:39:49,633] jepsen worker 176 - jepsen.util 1176 :ok :add 7589 INFO [2022-01-10 08:39:49,633] jepsen worker 659 - jepsen.util 2659 :ok :add 8229 INFO [2022-01-10 08:39:49,633] jepsen worker 727 - jepsen.util 727 :ok :add 8028 INFO [2022-01-10 08:39:49,638] jepsen worker 317 - jepsen.util 1317 :info :add 7311 :RequestTimeoutException INFO [2022-01-10 08:39:49,639] jepsen worker 130 - jepsen.util 1130 :info :add 7313 :RequestTimeoutException INFO [2022-01-10 08:39:49,643] jepsen worker 817 - jepsen.util 1817 :info :add 7315 :RequestTimeoutException INFO [2022-01-10 08:39:49,662] jepsen worker 280 - jepsen.util 1280 :ok :add 8331 INFO [2022-01-10 08:39:49,662] jepsen worker 904 - jepsen.util 1904 :invoke :add 8499 INFO [2022-01-10 08:39:49,664] jepsen worker 277 - jepsen.util 2277 :invoke :add 8500 INFO [2022-01-10 08:39:49,666] jepsen worker 890 - jepsen.util 890 :ok :add 7825 INFO [2022-01-10 08:39:49,666] jepsen worker 302 - jepsen.util 1302 :invoke :add 8501 INFO [2022-01-10 08:39:49,666] jepsen worker 39 - jepsen.util 1039 :ok :add 7879 INFO [2022-01-10 08:39:49,668] jepsen worker 383 - jepsen.util 383 :invoke :add 8502 INFO [2022-01-10 08:39:49,670] jepsen worker 317 - jepsen.util 2317 :invoke :add 8503 INFO [2022-01-10 08:39:49,673] jepsen worker 39 - jepsen.util 1039 :invoke :add 8504 INFO [2022-01-10 08:39:49,673] jepsen worker 876 - jepsen.util 1876 :ok :add 7768 INFO [2022-01-10 08:39:49,673] jepsen worker 47 - jepsen.util 1047 :ok :add 7688 INFO [2022-01-10 08:39:49,675] jepsen worker 306 - jepsen.util 1306 :invoke :add 8505 INFO [2022-01-10 08:39:49,677] jepsen worker 280 - jepsen.util 1280 :invoke :add 8506 INFO [2022-01-10 08:39:49,680] jepsen worker 362 - jepsen.util 1362 :invoke :add 8507 INFO [2022-01-10 08:39:49,681] jepsen worker 711 - jepsen.util 2711 :invoke :add 8508 INFO [2022-01-10 08:39:49,683] jepsen worker 955 - jepsen.util 1955 :invoke :add 8509 INFO [2022-01-10 08:39:49,685] jepsen worker 176 - jepsen.util 1176 :invoke :add 8510 INFO [2022-01-10 08:39:49,686] jepsen worker 64 - jepsen.util 2064 :invoke :add 8511 INFO [2022-01-10 08:39:49,688] jepsen worker 944 - jepsen.util 944 :invoke :add 8512 INFO [2022-01-10 08:39:49,690] jepsen worker 203 - jepsen.util 203 :invoke :add 8513 INFO [2022-01-10 08:39:49,692] jepsen worker 502 - jepsen.util 1502 :invoke :add 8514 INFO [2022-01-10 08:39:49,693] jepsen worker 653 - jepsen.util 1653 :info :add 7323 :RequestTimeoutException INFO [2022-01-10 08:39:49,694] jepsen worker 890 - jepsen.util 890 :invoke :add 8515 INFO [2022-01-10 08:39:49,695] jepsen worker 481 - jepsen.util 481 :ok :add 8497 INFO [2022-01-10 08:39:49,695] jepsen worker 963 - jepsen.util 963 :ok :add 7668 INFO [2022-01-10 08:39:49,695] jepsen worker 848 - jepsen.util 1848 :ok :add 7670 INFO [2022-01-10 08:39:49,697] jepsen worker 876 - jepsen.util 1876 :invoke :add 8516 INFO [2022-01-10 08:39:49,699] jepsen worker 145 - jepsen.util 1145 :ok :add 7644 INFO [2022-01-10 08:39:49,699] jepsen worker 825 - jepsen.util 1825 :ok :add 7955 INFO [2022-01-10 08:39:49,699] jepsen worker 373 - jepsen.util 1373 :ok :add 7979 INFO [2022-01-10 08:39:49,700] jepsen worker 797 - jepsen.util 1797 :invoke :add 8517 INFO [2022-01-10 08:39:49,701] jepsen worker 573 - jepsen.util 1573 :info :add 7324 :RequestTimeoutException INFO [2022-01-10 08:39:49,702] jepsen worker 914 - jepsen.util 1914 :info :add 7326 :RequestTimeoutException INFO [2022-01-10 08:39:49,703] jepsen worker 402 - jepsen.util 1402 :invoke :add 8518 INFO [2022-01-10 08:39:49,704] jepsen worker 751 - jepsen.util 1751 :info :add 7325 :RequestTimeoutException INFO [2022-01-10 08:39:49,707] jepsen worker 165 - jepsen.util 1165 :info :add 7327 :RequestTimeoutException INFO [2022-01-10 08:39:49,708] jepsen worker 659 - jepsen.util 2659 :invoke :add 8519 INFO [2022-01-10 08:39:49,710] jepsen worker 236 - jepsen.util 236 :invoke :add 8520 INFO [2022-01-10 08:39:49,711] jepsen worker 713 - jepsen.util 1713 :info :add 7328 :RequestTimeoutException INFO [2022-01-10 08:39:49,712] jepsen worker 573 - jepsen.util 2573 :invoke :add 8521 INFO [2022-01-10 08:39:49,713] jepsen worker 804 - jepsen.util 1804 :info :add 7329 :RequestTimeoutException INFO [2022-01-10 08:39:49,715] jepsen worker 396 - jepsen.util 1396 :invoke :add 8522 INFO [2022-01-10 08:39:49,718] jepsen worker 848 - jepsen.util 1848 :invoke :add 8523 INFO [2022-01-10 08:39:49,720] jepsen worker 481 - jepsen.util 481 :invoke :add 8524 INFO [2022-01-10 08:39:49,721] jepsen worker 167 - jepsen.util 1167 :info :add 7330 :RequestTimeoutException INFO [2022-01-10 08:39:49,722] jepsen worker 457 - jepsen.util 457 :invoke :add 8525 INFO [2022-01-10 08:39:49,722] jepsen worker 239 - jepsen.util 1239 :info :add 7332 :RequestTimeoutException INFO [2022-01-10 08:39:49,723] jepsen worker 6 - jepsen.util 1006 :info :add 7331 :RequestTimeoutException INFO [2022-01-10 08:39:49,727] jepsen worker 587 - jepsen.util 1587 :info :add 7333 :RequestTimeoutException INFO [2022-01-10 08:39:49,727] jepsen worker 299 - jepsen.util 2299 :invoke :add 8526 INFO [2022-01-10 08:39:49,730] jepsen worker 720 - jepsen.util 2720 :ok :add 8474 INFO [2022-01-10 08:39:49,730] jepsen worker 705 - jepsen.util 705 :ok :add 7982 INFO [2022-01-10 08:39:49,730] jepsen worker 409 - jepsen.util 1409 :ok :add 7703 INFO [2022-01-10 08:39:49,730] jepsen worker 808 - jepsen.util 1808 :ok :add 8306 INFO [2022-01-10 08:39:49,730] jepsen worker 365 - jepsen.util 2365 :invoke :add 8527 INFO [2022-01-10 08:39:49,730] jepsen worker 532 - jepsen.util 1532 :ok :add 8043 INFO [2022-01-10 08:39:49,735] jepsen worker 653 - jepsen.util 2653 :invoke :add 8528 INFO [2022-01-10 08:39:49,737] jepsen worker 21 - jepsen.util 21 :ok :add 7790 INFO [2022-01-10 08:39:49,737] jepsen worker 164 - jepsen.util 164 :ok :add 7788 INFO [2022-01-10 08:39:49,738] jepsen worker 461 - jepsen.util 1461 :ok :add 7569 INFO [2022-01-10 08:39:49,738] jepsen worker 145 - jepsen.util 1145 :invoke :add 8529 INFO [2022-01-10 08:39:49,740] jepsen worker 239 - jepsen.util 2239 :invoke :add 8530 INFO [2022-01-10 08:39:49,741] jepsen worker 21 - jepsen.util 21 :invoke :add 8531 INFO [2022-01-10 08:39:49,742] jepsen worker 356 - jepsen.util 1356 :ok :add 7980 INFO [2022-01-10 08:39:49,742] jepsen worker 702 - jepsen.util 702 :ok :add 8304 INFO [2022-01-10 08:39:49,742] jepsen worker 887 - jepsen.util 1887 :ok :add 8059 INFO [2022-01-10 08:39:49,743] jepsen worker 999 - jepsen.util 999 :ok :add 7824 INFO [2022-01-10 08:39:49,743] jepsen worker 76 - jepsen.util 1076 :ok :add 7735 INFO [2022-01-10 08:39:49,743] jepsen worker 210 - jepsen.util 1210 :ok :add 7719 INFO [2022-01-10 08:39:49,743] jepsen worker 570 - jepsen.util 1570 :ok :add 7701 INFO [2022-01-10 08:39:49,743] jepsen worker 917 - jepsen.util 1917 :ok :add 8294 INFO [2022-01-10 08:39:49,743] jepsen worker 718 - jepsen.util 1718 :ok :add 8075 INFO [2022-01-10 08:39:49,743] jepsen worker 75 - jepsen.util 1075 :invoke :add 8532 INFO [2022-01-10 08:39:49,743] jepsen worker 607 - jepsen.util 607 :ok :add 7691 INFO [2022-01-10 08:39:49,743] jepsen worker 263 - jepsen.util 1263 :ok :add 8330 INFO [2022-01-10 08:39:49,747] jepsen worker 566 - jepsen.util 1566 :invoke :add 8533 INFO [2022-01-10 08:39:49,749] jepsen worker 887 - jepsen.util 1887 :invoke :add 8534 INFO [2022-01-10 08:39:49,750] jepsen worker 263 - jepsen.util 1263 :invoke :add 8535 INFO [2022-01-10 08:39:49,752] jepsen worker 130 - jepsen.util 2130 :invoke :add 8536 INFO [2022-01-10 08:39:49,753] jepsen worker 389 - jepsen.util 389 :ok :add 7882 INFO [2022-01-10 08:39:49,753] jepsen worker 975 - jepsen.util 1975 :ok :add 8143 INFO [2022-01-10 08:39:49,753] jepsen worker 89 - jepsen.util 2089 :ok :add 8396 INFO [2022-01-10 08:39:49,754] jepsen worker 587 - jepsen.util 2587 :invoke :add 8537 INFO [2022-01-10 08:39:49,755] jepsen worker 629 - jepsen.util 629 :ok :add 8007 INFO [2022-01-10 08:39:49,755] jepsen worker 969 - jepsen.util 1969 :ok :add 8033 INFO [2022-01-10 08:39:49,755] jepsen worker 800 - jepsen.util 2800 :ok :add 8404 INFO [2022-01-10 08:39:49,756] jepsen worker 718 - jepsen.util 1718 :invoke :add 8538 INFO [2022-01-10 08:39:49,756] jepsen worker 358 - jepsen.util 358 :ok :add 7747 INFO [2022-01-10 08:39:49,756] jepsen worker 415 - jepsen.util 1415 :ok :add 8376 INFO [2022-01-10 08:39:49,756] jepsen worker 360 - jepsen.util 360 :ok :add 7773 INFO [2022-01-10 08:39:49,756] jepsen worker 982 - jepsen.util 1982 :ok :add 7592 INFO [2022-01-10 08:39:49,756] jepsen worker 474 - jepsen.util 474 :ok :add 7856 INFO [2022-01-10 08:39:49,759] jepsen worker 474 - jepsen.util 474 :invoke :add 8539 INFO [2022-01-10 08:39:49,760] jepsen worker 167 - jepsen.util 2167 :invoke :add 8540 INFO [2022-01-10 08:39:49,762] jepsen worker 210 - jepsen.util 1210 :invoke :add 8541 INFO [2022-01-10 08:39:49,763] jepsen worker 982 - jepsen.util 1982 :invoke :add 8542 INFO [2022-01-10 08:39:49,766] jepsen worker 664 - jepsen.util 1664 :invoke :add 8543 INFO [2022-01-10 08:39:49,766] jepsen worker 950 - jepsen.util 950 :ok :add 8277 INFO [2022-01-10 08:39:49,766] jepsen worker 690 - jepsen.util 690 :ok :add 7957 INFO [2022-01-10 08:39:49,766] jepsen worker 954 - jepsen.util 1954 :ok :add 8106 INFO [2022-01-10 08:39:49,766] jepsen worker 306 - jepsen.util 1306 :ok :add 8505 INFO [2022-01-10 08:39:49,766] jepsen worker 416 - jepsen.util 2416 :ok :add 8243 INFO [2022-01-10 08:39:49,767] jepsen worker 817 - jepsen.util 2817 :invoke :add 8544 INFO [2022-01-10 08:39:49,770] jepsen worker 532 - jepsen.util 1532 :invoke :add 8545 INFO [2022-01-10 08:39:49,771] jepsen worker 865 - jepsen.util 1865 :ok :add 7956 INFO [2022-01-10 08:39:49,771] jepsen worker 133 - jepsen.util 2133 :ok :add 8276 INFO [2022-01-10 08:39:49,771] jepsen worker 109 - jepsen.util 1109 :ok :add 7673 INFO [2022-01-10 08:39:49,771] jepsen worker 239 - jepsen.util 2239 :ok :add 8530 INFO [2022-01-10 08:39:49,771] jepsen worker 665 - jepsen.util 1665 :ok :add 8107 INFO [2022-01-10 08:39:49,771] jepsen worker 287 - jepsen.util 1287 :ok :add 7962 INFO [2022-01-10 08:39:49,771] jepsen worker 653 - jepsen.util 2653 :ok :add 8528 INFO [2022-01-10 08:39:49,772] jepsen worker 629 - jepsen.util 629 :invoke :add 8546 INFO [2022-01-10 08:39:49,772] jepsen worker 712 - jepsen.util 1712 :info :add 7338 :RequestTimeoutException INFO [2022-01-10 08:39:49,776] jepsen worker 47 - jepsen.util 1047 :invoke :add 8547 INFO [2022-01-10 08:39:49,777] jepsen worker 306 - jepsen.util 1306 :invoke :add 8548 INFO [2022-01-10 08:39:49,779] jepsen worker 616 - jepsen.util 1616 :ok :add 7910 INFO [2022-01-10 08:39:49,779] jepsen worker 895 - jepsen.util 1895 :ok :add 7789 INFO [2022-01-10 08:39:49,779] jepsen worker 598 - jepsen.util 1598 :ok :add 7791 INFO [2022-01-10 08:39:49,779] jepsen worker 331 - jepsen.util 1331 :ok :add 8141 INFO [2022-01-10 08:39:49,779] jepsen worker 661 - jepsen.util 1661 :ok :add 7908 INFO [2022-01-10 08:39:49,779] jepsen worker 886 - jepsen.util 2886 :ok :add 8228 INFO [2022-01-10 08:39:49,779] jepsen worker 319 - jepsen.util 1319 :ok :add 8175 INFO [2022-01-10 08:39:49,779] jepsen worker 705 - jepsen.util 705 :invoke :add 8549 INFO [2022-01-10 08:39:49,779] jepsen worker 540 - jepsen.util 2540 :ok :add 8230 INFO [2022-01-10 08:39:49,780] jepsen worker 26 - jepsen.util 1026 :info :add 7340 :RequestTimeoutException INFO [2022-01-10 08:39:49,782] jepsen worker 859 - jepsen.util 1859 :ok :add 7645 INFO [2022-01-10 08:39:49,782] jepsen worker 787 - jepsen.util 1787 :ok :add 8095 INFO [2022-01-10 08:39:49,783] jepsen worker 327 - jepsen.util 1327 :ok :add 7960 INFO [2022-01-10 08:39:49,783] jepsen worker 280 - jepsen.util 1280 :ok :add 8506 INFO [2022-01-10 08:39:49,783] jepsen worker 75 - jepsen.util 1075 :ok :add 8532 INFO [2022-01-10 08:39:49,783] jepsen worker 287 - jepsen.util 1287 :invoke :add 8550 INFO [2022-01-10 08:39:49,786] jepsen worker 727 - jepsen.util 727 :invoke :add 8551 INFO [2022-01-10 08:39:49,787] jepsen worker 804 - jepsen.util 2804 :invoke :add 8552 INFO [2022-01-10 08:39:49,789] jepsen worker 865 - jepsen.util 1865 :invoke :add 8553 INFO [2022-01-10 08:39:49,791] jepsen worker 358 - jepsen.util 358 :invoke :add 8554 INFO [2022-01-10 08:39:49,791] jepsen worker 189 - jepsen.util 1189 :info :add 7342 :RequestTimeoutException INFO [2022-01-10 08:39:49,792] jepsen worker 950 - jepsen.util 950 :invoke :add 8555 INFO [2022-01-10 08:39:49,793] jepsen worker 353 - jepsen.util 1353 :ok :add 8004 INFO [2022-01-10 08:39:49,793] jepsen worker 584 - jepsen.util 1584 :ok :add 7861 INFO [2022-01-10 08:39:49,793] jepsen worker 916 - jepsen.util 2916 :ok :add 8341 INFO [2022-01-10 08:39:49,793] jepsen worker 911 - jepsen.util 1911 :ok :add 8433 INFO [2022-01-10 08:39:49,793] jepsen worker 3 - jepsen.util 1003 :ok :add 8030 INFO [2022-01-10 08:39:49,793] jepsen worker 529 - jepsen.util 1529 :ok :add 7744 INFO [2022-01-10 08:39:49,793] jepsen worker 602 - jepsen.util 2602 :ok :add 8194 INFO [2022-01-10 08:39:49,793] jepsen worker 393 - jepsen.util 2393 :ok :add 8359 INFO [2022-01-10 08:39:49,793] jepsen worker 775 - jepsen.util 1775 :info :add 7341 :RequestTimeoutException INFO [2022-01-10 08:39:49,797] jepsen worker 712 - jepsen.util 2712 :invoke :add 8556 INFO [2022-01-10 08:39:49,798] jepsen worker 202 - jepsen.util 202 :ok :add 7860 INFO [2022-01-10 08:39:49,798] jepsen worker 668 - jepsen.util 1668 :ok :add 8406 INFO [2022-01-10 08:39:49,798] jepsen worker 255 - jepsen.util 255 :ok :add 7769 INFO [2022-01-10 08:39:49,798] jepsen worker 674 - jepsen.util 674 :ok :add 7771 INFO [2022-01-10 08:39:49,798] jepsen worker 777 - jepsen.util 777 :ok :add 7878 INFO [2022-01-10 08:39:49,798] jepsen worker 54 - jepsen.util 54 :ok :add 7854 INFO [2022-01-10 08:39:49,798] jepsen worker 660 - jepsen.util 2660 :ok :add 8358 INFO [2022-01-10 08:39:49,799] jepsen worker 805 - jepsen.util 1805 :ok :add 8195 INFO [2022-01-10 08:39:49,799] jepsen worker 800 - jepsen.util 2800 :invoke :add 8557 INFO [2022-01-10 08:39:49,801] jepsen worker 709 - jepsen.util 709 :info :add 7343 :RequestTimeoutException INFO [2022-01-10 08:39:49,802] jepsen worker 665 - jepsen.util 1665 :invoke :add 8558 INFO [2022-01-10 08:39:49,803] jepsen worker 431 - jepsen.util 1431 :ok :add 7863 INFO [2022-01-10 08:39:49,803] jepsen worker 522 - jepsen.util 2522 :ok :add 8431 INFO [2022-01-10 08:39:49,803] jepsen worker 535 - jepsen.util 1535 :ok :add 8196 INFO [2022-01-10 08:39:49,803] jepsen worker 899 - jepsen.util 2899 :ok :add 8429 INFO [2022-01-10 08:39:49,804] jepsen worker 415 - jepsen.util 1415 :invoke :add 8559 INFO [2022-01-10 08:39:49,807] jepsen worker 598 - jepsen.util 1598 :invoke :add 8560 INFO [2022-01-10 08:39:49,809] jepsen worker 914 - jepsen.util 2914 :invoke :add 8561 INFO [2022-01-10 08:39:49,810] jepsen worker 54 - jepsen.util 54 :invoke :add 8562 INFO [2022-01-10 08:39:49,812] jepsen worker 331 - jepsen.util 1331 :invoke :add 8563 INFO [2022-01-10 08:39:49,813] jepsen worker 833 - jepsen.util 833 :info :add 7349 :RequestTimeoutException INFO [2022-01-10 08:39:49,814] jepsen worker 787 - jepsen.util 1787 :invoke :add 8564 INFO [2022-01-10 08:39:49,816] jepsen worker 594 - jepsen.util 594 :ok :add 7736 INFO [2022-01-10 08:39:49,816] jepsen worker 700 - jepsen.util 1700 :ok :add 7813 INFO [2022-01-10 08:39:49,816] jepsen worker 353 - jepsen.util 1353 :invoke :add 8565 INFO [2022-01-10 08:39:49,816] jepsen worker 362 - jepsen.util 1362 :ok :add 8507 INFO [2022-01-10 08:39:49,816] jepsen worker 754 - jepsen.util 1754 :ok :add 8459 INFO [2022-01-10 08:39:49,816] jepsen worker 566 - jepsen.util 1566 :ok :add 8533 INFO [2022-01-10 08:39:49,816] jepsen worker 483 - jepsen.util 1483 :ok :add 8094 INFO [2022-01-10 08:39:49,816] jepsen worker 241 - jepsen.util 2241 :ok :add 8259 INFO [2022-01-10 08:39:49,817] jepsen worker 512 - jepsen.util 1512 :ok :add 8297 INFO [2022-01-10 08:39:49,817] jepsen worker 69 - jepsen.util 1069 :ok :add 7702 INFO [2022-01-10 08:39:49,817] jepsen worker 534 - jepsen.util 534 :ok :add 7692 INFO [2022-01-10 08:39:49,817] jepsen worker 979 - jepsen.util 1979 :ok :add 8130 INFO [2022-01-10 08:39:49,817] jepsen worker 237 - jepsen.util 237 :ok :add 7827 INFO [2022-01-10 08:39:49,817] jepsen worker 915 - jepsen.util 915 :ok :add 7983 INFO [2022-01-10 08:39:49,817] jepsen worker 940 - jepsen.util 1940 :ok :add 8104 INFO [2022-01-10 08:39:49,817] jepsen worker 829 - jepsen.util 2829 :ok :add 8333 INFO [2022-01-10 08:39:49,821] jepsen worker 899 - jepsen.util 2899 :invoke :add 8566 INFO [2022-01-10 08:39:49,823] jepsen worker 709 - jepsen.util 1709 :invoke :add 8567 INFO [2022-01-10 08:39:49,824] jepsen worker 239 - jepsen.util 2239 :invoke :add 8568 INFO [2022-01-10 08:39:49,826] jepsen worker 661 - jepsen.util 1661 :invoke :add 8569 INFO [2022-01-10 08:39:49,827] jepsen worker 23 - jepsen.util 23 :info :add 7353 :RequestTimeoutException INFO [2022-01-10 08:39:49,828] jepsen worker 362 - jepsen.util 1362 :invoke :add 8570 INFO [2022-01-10 08:39:49,829] jepsen worker 146 - jepsen.util 146 :ok :add 7883 INFO [2022-01-10 08:39:49,829] jepsen worker 390 - jepsen.util 390 :ok :add 7792 INFO [2022-01-10 08:39:49,829] jepsen worker 422 - jepsen.util 422 :ok :add 7913 INFO [2022-01-10 08:39:49,829] jepsen worker 794 - jepsen.util 1794 :ok :add 8176 INFO [2022-01-10 08:39:49,829] jepsen worker 683 - jepsen.util 683 :ok :add 7927 INFO [2022-01-10 08:39:49,830] jepsen worker 350 - jepsen.util 2350 :ok :add 8207 INFO [2022-01-10 08:39:49,830] jepsen worker 966 - jepsen.util 1966 :info :add 7354 :RequestTimeoutException INFO [2022-01-10 08:39:49,831] jepsen worker 255 - jepsen.util 255 :invoke :add 8571 INFO [2022-01-10 08:39:49,831] jepsen worker 183 - jepsen.util 183 :info :add 7355 :RequestTimeoutException INFO [2022-01-10 08:39:49,832] jepsen worker 634 - jepsen.util 634 :info :add 7356 :RequestTimeoutException INFO [2022-01-10 08:39:49,833] jepsen worker 906 - jepsen.util 1906 :info :add 7357 :RequestTimeoutException INFO [2022-01-10 08:39:49,837] jepsen worker 75 - jepsen.util 1075 :invoke :add 8572 INFO [2022-01-10 08:39:49,838] jepsen worker 416 - jepsen.util 2416 :invoke :add 8573 INFO [2022-01-10 08:39:49,840] jepsen worker 859 - jepsen.util 1859 :invoke :add 8574 INFO [2022-01-10 08:39:49,841] jepsen worker 782 - jepsen.util 1782 :info :add 7358 :RequestTimeoutException INFO [2022-01-10 08:39:49,842] jepsen worker 657 - jepsen.util 1657 :info :add 7360 :RequestTimeoutException INFO [2022-01-10 08:39:49,842] jepsen worker 602 - jepsen.util 2602 :invoke :add 8575 INFO [2022-01-10 08:39:49,843] jepsen worker 519 - jepsen.util 1519 :ok :add 8040 INFO [2022-01-10 08:39:49,843] jepsen worker 152 - jepsen.util 1152 :ok :add 7809 INFO [2022-01-10 08:39:49,843] jepsen worker 537 - jepsen.util 537 :info :add 7361 :RequestTimeoutException INFO [2022-01-10 08:39:49,843] jepsen worker 388 - jepsen.util 1388 :ok :add 8305 INFO [2022-01-10 08:39:49,843] jepsen worker 677 - jepsen.util 2677 :ok :add 8477 INFO [2022-01-10 08:39:49,844] jepsen worker 497 - jepsen.util 1497 :ok :add 8329 INFO [2022-01-10 08:39:49,844] jepsen worker 786 - jepsen.util 2786 :ok :add 8443 INFO [2022-01-10 08:39:49,844] jepsen worker 699 - jepsen.util 1699 :info :add 7359 :RequestTimeoutException INFO [2022-01-10 08:39:49,844] jepsen worker 295 - jepsen.util 1295 :ok :add 7981 INFO [2022-01-10 08:39:49,844] jepsen worker 59 - jepsen.util 1059 :ok :add 8074 INFO [2022-01-10 08:39:49,847] jepsen worker 941 - jepsen.util 1941 :info :add 7362 :RequestTimeoutException INFO [2022-01-10 08:39:49,848] jepsen worker 461 - jepsen.util 1461 :invoke :add 8576 INFO [2022-01-10 08:39:49,851] jepsen worker 162 - jepsen.util 1162 :info :add 7363 :RequestTimeoutException INFO [2022-01-10 08:39:49,851] jepsen worker 537 - jepsen.util 1537 :invoke :add 8577 INFO [2022-01-10 08:39:49,855] jepsen worker 356 - jepsen.util 1356 :invoke :add 8578 INFO [2022-01-10 08:39:49,856] jepsen worker 350 - jepsen.util 2350 :invoke :add 8579 INFO [2022-01-10 08:39:49,858] jepsen worker 202 - jepsen.util 202 :invoke :add 8580 INFO [2022-01-10 08:39:49,859] jepsen worker 475 - jepsen.util 2475 :ok :add 8231 INFO [2022-01-10 08:39:49,860] jepsen worker 512 - jepsen.util 1512 :invoke :add 8581 INFO [2022-01-10 08:39:49,861] jepsen worker 683 - jepsen.util 683 :invoke :add 8582 INFO [2022-01-10 08:39:49,863] jepsen worker 50 - jepsen.util 1050 :info :add 7365 :RequestTimeoutException INFO [2022-01-10 08:39:49,864] jepsen worker 61 - jepsen.util 61 :invoke :add 8583 INFO [2022-01-10 08:39:49,864] jepsen worker 377 - jepsen.util 1377 :ok :add 7899 INFO [2022-01-10 08:39:49,864] jepsen worker 494 - jepsen.util 494 :ok :add 7911 INFO [2022-01-10 08:39:49,865] jepsen worker 664 - jepsen.util 1664 :ok :add 8543 INFO [2022-01-10 08:39:49,866] jepsen worker 570 - jepsen.util 1570 :invoke :add 8584 INFO [2022-01-10 08:39:49,868] jepsen worker 389 - jepsen.util 389 :invoke :add 8585 INFO [2022-01-10 08:39:49,870] jepsen worker 775 - jepsen.util 2775 :invoke :add 8586 INFO [2022-01-10 08:39:49,872] jepsen worker 969 - jepsen.util 1969 :invoke :add 8587 INFO [2022-01-10 08:39:49,873] jepsen worker 714 - jepsen.util 2714 :ok :add 8428 INFO [2022-01-10 08:39:49,873] jepsen worker 388 - jepsen.util 1388 :invoke :add 8588 INFO [2022-01-10 08:39:49,876] jepsen worker 133 - jepsen.util 2133 :invoke :add 8589 INFO [2022-01-10 08:39:49,878] jepsen worker 475 - jepsen.util 2475 :invoke :add 8590 INFO [2022-01-10 08:39:49,880] jepsen worker 664 - jepsen.util 1664 :invoke :add 8591 INFO [2022-01-10 08:39:49,880] jepsen worker 145 - jepsen.util 1145 :ok :add 8529 INFO [2022-01-10 08:39:49,880] jepsen worker 21 - jepsen.util 21 :ok :add 8531 INFO [2022-01-10 08:39:49,881] jepsen worker 675 - jepsen.util 2675 :ok :add 8494 INFO [2022-01-10 08:39:49,881] jepsen worker 39 - jepsen.util 1039 :ok :add 8504 INFO [2022-01-10 08:39:49,881] jepsen worker 76 - jepsen.util 1076 :invoke :add 8592 INFO [2022-01-10 08:39:49,884] jepsen worker 915 - jepsen.util 915 :invoke :add 8593 INFO [2022-01-10 08:39:49,884] jepsen worker 167 - jepsen.util 2167 :ok :add 8540 INFO [2022-01-10 08:39:49,886] jepsen worker 390 - jepsen.util 390 :invoke :add 8594 INFO [2022-01-10 08:39:49,887] jepsen worker 177 - jepsen.util 177 :ok :add 8496 INFO [2022-01-10 08:39:49,887] jepsen worker 250 - jepsen.util 1250 :ok :add 8375 INFO [2022-01-10 08:39:49,888] jepsen worker 660 - jepsen.util 2660 :invoke :add 8595 INFO [2022-01-10 08:39:49,891] jepsen worker 373 - jepsen.util 1373 :invoke :add 8596 INFO [2022-01-10 08:39:49,893] jepsen worker 161 - jepsen.util 1161 :info :add 7375 :RequestTimeoutException INFO [2022-01-10 08:39:49,893] jepsen worker 280 - jepsen.util 1280 :invoke :add 8597 INFO [2022-01-10 08:39:49,894] jepsen worker 823 - jepsen.util 1823 :info :add 7374 :RequestTimeoutException INFO [2022-01-10 08:39:49,894] jepsen worker 393 - jepsen.util 2393 :invoke :add 8598 INFO [2022-01-10 08:39:49,897] jepsen worker 167 - jepsen.util 2167 :invoke :add 8599 INFO [2022-01-10 08:39:49,898] jepsen worker 280 - jepsen.util 1280 :ok :add 8597 INFO [2022-01-10 08:39:49,899] jepsen worker 941 - jepsen.util 2941 :invoke :add 8600 INFO [2022-01-10 08:39:49,901] jepsen worker 422 - jepsen.util 422 :invoke :add 8601 INFO [2022-01-10 08:39:49,902] jepsen worker 660 - jepsen.util 2660 :ok :add 8595 INFO [2022-01-10 08:39:49,903] jepsen worker 517 - jepsen.util 1517 :info :add 7378 :RequestTimeoutException INFO [2022-01-10 08:39:49,903] jepsen worker 250 - jepsen.util 1250 :invoke :add 8602 INFO [2022-01-10 08:39:49,904] jepsen worker 962 - jepsen.util 2962 :ok :add 8475 INFO [2022-01-10 08:39:49,906] jepsen worker 660 - jepsen.util 2660 :invoke :add 8603 INFO [2022-01-10 08:39:49,907] jepsen worker 818 - jepsen.util 1818 :info :add 7376 :RequestTimeoutException INFO [2022-01-10 08:39:49,908] jepsen worker 39 - jepsen.util 1039 :invoke :add 8604 INFO [2022-01-10 08:39:49,910] jepsen worker 591 - jepsen.util 1591 :info :add 7377 :RequestTimeoutException INFO [2022-01-10 08:39:49,910] jepsen worker 829 - jepsen.util 2829 :invoke :add 8605 INFO [2022-01-10 08:39:49,910] jepsen worker 415 - jepsen.util 1415 :ok :add 8559 INFO [2022-01-10 08:39:49,911] jepsen worker 508 - jepsen.util 2508 :ok :add 8397 INFO [2022-01-10 08:39:49,912] jepsen worker 602 - jepsen.util 2602 :ok :add 8575 INFO [2022-01-10 08:39:49,913] jepsen worker 69 - jepsen.util 1069 :invoke :add 8606 INFO [2022-01-10 08:39:49,913] jepsen worker 617 - jepsen.util 1617 :info :add 7379 :RequestTimeoutException INFO [2022-01-10 08:39:49,914] jepsen worker 531 - jepsen.util 1531 :info :add 7380 :RequestTimeoutException INFO [2022-01-10 08:39:49,917] jepsen worker 818 - jepsen.util 2818 :invoke :add 8607 INFO [2022-01-10 08:39:49,918] jepsen worker 952 - jepsen.util 1952 :ok :add 8458 INFO [2022-01-10 08:39:49,919] jepsen worker 237 - jepsen.util 237 :invoke :add 8608 INFO [2022-01-10 08:39:49,921] jepsen worker 782 - jepsen.util 2782 :invoke :add 8609 INFO [2022-01-10 08:39:49,923] jepsen worker 360 - jepsen.util 360 :invoke :add 8610 INFO [2022-01-10 08:39:49,923] jepsen worker 52 - jepsen.util 1052 :info :add 7381 :RequestTimeoutException INFO [2022-01-10 08:39:49,924] jepsen worker 883 - jepsen.util 2883 :ok :add 8476 INFO [2022-01-10 08:39:49,925] jepsen worker 911 - jepsen.util 1911 :invoke :add 8611 INFO [2022-01-10 08:39:49,927] jepsen worker 58 - jepsen.util 1058 :info :add 7382 :RequestTimeoutException INFO [2022-01-10 08:39:49,929] jepsen worker 833 - jepsen.util 1833 :invoke :add 8612 INFO [2022-01-10 08:39:49,931] jepsen worker 29 - jepsen.util 1029 :info :add 7383 :RequestTimeoutException INFO [2022-01-10 08:39:49,931] jepsen worker 703 - jepsen.util 1703 :info :add 7385 :RequestTimeoutException INFO [2022-01-10 08:39:49,931] jepsen worker 183 - jepsen.util 1183 :invoke :add 8613 INFO [2022-01-10 08:39:49,933] jepsen worker 927 - jepsen.util 1927 :info :add 7384 :RequestTimeoutException INFO [2022-01-10 08:39:49,933] jepsen worker 829 - jepsen.util 2829 :ok :add 8605 INFO [2022-01-10 08:39:49,933] jepsen worker 859 - jepsen.util 1859 :ok :add 8574 INFO [2022-01-10 08:39:49,933] jepsen worker 665 - jepsen.util 1665 :ok :add 8558 INFO [2022-01-10 08:39:49,936] jepsen worker 751 - jepsen.util 2751 :invoke :add 8614 INFO [2022-01-10 08:39:49,937] jepsen worker 503 - jepsen.util 2503 :ok :add 8495 INFO [2022-01-10 08:39:49,938] jepsen worker 177 - jepsen.util 177 :invoke :add 8615 INFO [2022-01-10 08:39:49,941] jepsen worker 79 - jepsen.util 1079 :invoke :add 8616 INFO [2022-01-10 08:39:49,942] jepsen worker 165 - jepsen.util 2165 :invoke :add 8617 INFO [2022-01-10 08:39:49,944] jepsen worker 808 - jepsen.util 1808 :invoke :add 8618 INFO [2022-01-10 08:39:49,945] jepsen worker 69 - jepsen.util 1069 :ok :add 8606 INFO [2022-01-10 08:39:49,945] jepsen worker 537 - jepsen.util 1537 :ok :add 8577 INFO [2022-01-10 08:39:49,947] jepsen worker 602 - jepsen.util 2602 :invoke :add 8619 INFO [2022-01-10 08:39:49,947] jepsen worker 982 - jepsen.util 1982 :ok :add 8542 INFO [2022-01-10 08:39:49,948] jepsen worker 461 - jepsen.util 1461 :ok :add 8576 INFO [2022-01-10 08:39:49,948] jepsen worker 818 - jepsen.util 2818 :ok :add 8607 INFO [2022-01-10 08:39:49,949] jepsen worker 390 - jepsen.util 390 :ok :add 8594 INFO [2022-01-10 08:39:49,950] jepsen worker 700 - jepsen.util 1700 :invoke :add 8620 INFO [2022-01-10 08:39:49,952] jepsen worker 900 - jepsen.util 1900 :info :add 7387 :RequestTimeoutException INFO [2022-01-10 08:39:49,953] jepsen worker 675 - jepsen.util 2675 :invoke :add 8621 INFO [2022-01-10 08:39:49,955] jepsen worker 534 - jepsen.util 534 :invoke :add 8622 INFO [2022-01-10 08:39:49,956] jepsen worker 373 - jepsen.util 1373 :ok :add 8596 INFO [2022-01-10 08:39:49,957] jepsen worker 668 - jepsen.util 1668 :invoke :add 8623 INFO [2022-01-10 08:39:49,960] jepsen worker 634 - jepsen.util 1634 :invoke :add 8624 INFO [2022-01-10 08:39:49,963] jepsen worker 690 - jepsen.util 690 :invoke :add 8625 INFO [2022-01-10 08:39:49,964] jepsen worker 210 - jepsen.util 1210 :ok :add 8541 INFO [2022-01-10 08:39:49,964] jepsen worker 39 - jepsen.util 1039 :ok :add 8604 INFO [2022-01-10 08:39:49,964] jepsen worker 829 - jepsen.util 2829 :invoke :add 8626 INFO [2022-01-10 08:39:49,966] jepsen worker 963 - jepsen.util 963 :invoke :add 8627 INFO [2022-01-10 08:39:49,968] jepsen worker 607 - jepsen.util 607 :invoke :add 8628 INFO [2022-01-10 08:39:49,970] jepsen worker 161 - jepsen.util 2161 :invoke :add 8629 INFO [2022-01-10 08:39:49,971] jepsen worker 805 - jepsen.util 1805 :invoke :add 8630 INFO [2022-01-10 08:39:49,973] jepsen worker 164 - jepsen.util 164 :invoke :add 8631 INFO [2022-01-10 08:39:49,975] jepsen worker 917 - jepsen.util 1917 :invoke :add 8632 INFO [2022-01-10 08:39:49,976] jepsen worker 859 - jepsen.util 1859 :invoke :add 8633 INFO [2022-01-10 08:39:49,977] jepsen worker 580 - jepsen.util 1580 :info :add 7394 :RequestTimeoutException INFO [2022-01-10 08:39:49,978] jepsen worker 537 - jepsen.util 1537 :invoke :add 8634 INFO [2022-01-10 08:39:49,980] jepsen worker 599 - jepsen.util 1599 :info :add 7395 :RequestTimeoutException INFO [2022-01-10 08:39:49,982] jepsen worker 591 - jepsen.util 2591 :invoke :add 8635 INFO [2022-01-10 08:39:49,983] jepsen worker 163 - jepsen.util 1163 :info :add 7396 :RequestTimeoutException INFO [2022-01-10 08:39:49,984] jepsen worker 540 - jepsen.util 2540 :invoke :add 8636 INFO [2022-01-10 08:39:49,987] jepsen worker 906 - jepsen.util 2906 :invoke :add 8637 INFO [2022-01-10 08:39:49,989] jepsen worker 146 - jepsen.util 146 :invoke :add 8638 INFO [2022-01-10 08:39:49,991] jepsen worker 927 - jepsen.util 2927 :invoke :add 8639 INFO [2022-01-10 08:39:49,991] jepsen worker 323 - jepsen.util 1323 :info :add 7397 :RequestTimeoutException INFO [2022-01-10 08:39:49,993] jepsen worker 617 - jepsen.util 2617 :invoke :add 8640 INFO [2022-01-10 08:39:49,996] jepsen worker 409 - jepsen.util 1409 :invoke :add 8641 INFO [2022-01-10 08:39:49,998] jepsen worker 720 - jepsen.util 2720 :invoke :add 8642 INFO [2022-01-10 08:39:50,000] jepsen worker 979 - jepsen.util 1979 :invoke :add 8643 INFO [2022-01-10 08:39:50,002] jepsen worker 517 - jepsen.util 2517 :invoke :add 8644 INFO [2022-01-10 08:39:50,004] jepsen worker 59 - jepsen.util 1059 :invoke :add 8645 INFO [2022-01-10 08:39:50,006] jepsen worker 280 - jepsen.util 1280 :invoke :add 8646 INFO [2022-01-10 08:39:50,008] jepsen worker 674 - jepsen.util 674 :invoke :add 8647 INFO [2022-01-10 08:39:50,012] jepsen worker 584 - jepsen.util 1584 :invoke :add 8648 INFO [2022-01-10 08:39:50,014] jepsen worker 390 - jepsen.util 390 :invoke :add 8649 INFO [2022-01-10 08:39:50,016] jepsen worker 319 - jepsen.util 1319 :invoke :add 8650 INFO [2022-01-10 08:39:50,017] jepsen worker 754 - jepsen.util 1754 :invoke :add 8651 INFO [2022-01-10 08:39:50,019] jepsen worker 327 - jepsen.util 1327 :invoke :add 8652 INFO [2022-01-10 08:39:50,021] jepsen worker 23 - jepsen.util 1023 :invoke :add 8653 INFO [2022-01-10 08:39:50,021] jepsen worker 961 - jepsen.util 961 :info :add 7399 :RequestTimeoutException INFO [2022-01-10 08:39:50,021] jepsen worker 158 - jepsen.util 158 :info :add 7398 :RequestTimeoutException INFO [2022-01-10 08:39:50,024] jepsen worker 210 - jepsen.util 1210 :invoke :add 8654 INFO [2022-01-10 08:39:50,026] jepsen worker 531 - jepsen.util 2531 :invoke :add 8655 INFO [2022-01-10 08:39:50,028] jepsen worker 461 - jepsen.util 1461 :invoke :add 8656 INFO [2022-01-10 08:39:50,030] jepsen worker 519 - jepsen.util 1519 :invoke :add 8657 INFO [2022-01-10 08:39:50,031] jepsen worker 497 - jepsen.util 1497 :invoke :add 8658 INFO [2022-01-10 08:39:50,033] jepsen worker 93 - jepsen.util 1093 :info :add 7404 :RequestTimeoutException INFO [2022-01-10 08:39:50,033] jepsen worker 677 - jepsen.util 2677 :invoke :add 8659 INFO [2022-01-10 08:39:50,036] jepsen worker 699 - jepsen.util 2699 :invoke :add 8660 INFO [2022-01-10 08:39:50,038] jepsen worker 503 - jepsen.util 2503 :invoke :add 8661 INFO [2022-01-10 08:39:50,039] jepsen worker 6 - jepsen.util 2006 :invoke :add 8662 INFO [2022-01-10 08:39:50,041] jepsen worker 187 - jepsen.util 1187 :info :add 7405 :RequestTimeoutException INFO [2022-01-10 08:39:50,041] jepsen worker 431 - jepsen.util 1431 :invoke :add 8663 INFO [2022-01-10 08:39:50,043] jepsen worker 535 - jepsen.util 1535 :invoke :add 8664 INFO [2022-01-10 08:39:50,045] jepsen worker 529 - jepsen.util 1529 :invoke :add 8665 INFO [2022-01-10 08:39:50,047] jepsen worker 703 - jepsen.util 2703 :invoke :add 8666 INFO [2022-01-10 08:39:50,049] jepsen worker 599 - jepsen.util 2599 :invoke :add 8667 INFO [2022-01-10 08:39:50,051] jepsen worker 900 - jepsen.util 2900 :invoke :add 8668 INFO [2022-01-10 08:39:50,052] jepsen worker 295 - jepsen.util 1295 :invoke :add 8669 INFO [2022-01-10 08:39:50,054] jepsen worker 152 - jepsen.util 1152 :invoke :add 8670 INFO [2022-01-10 08:39:50,056] jepsen worker 163 - jepsen.util 2163 :invoke :add 8671 INFO [2022-01-10 08:39:50,058] jepsen worker 966 - jepsen.util 2966 :invoke :add 8672 INFO [2022-01-10 08:39:50,060] jepsen worker 145 - jepsen.util 1145 :invoke :add 8673 INFO [2022-01-10 08:39:50,061] jepsen worker 522 - jepsen.util 2522 :invoke :add 8674 INFO [2022-01-10 08:39:50,063] jepsen worker 594 - jepsen.util 594 :invoke :add 8675 INFO [2022-01-10 08:39:50,065] jepsen worker 26 - jepsen.util 2026 :invoke :add 8676 INFO [2022-01-10 08:39:50,066] jepsen worker 582 - jepsen.util 1582 :ok :add 7532 INFO [2022-01-10 08:39:50,067] jepsen worker 916 - jepsen.util 2916 :invoke :add 8677 INFO [2022-01-10 08:39:50,069] jepsen worker 3 - jepsen.util 1003 :invoke :add 8678 INFO [2022-01-10 08:39:50,071] jepsen worker 961 - jepsen.util 1961 :invoke :add 8679 INFO [2022-01-10 08:39:50,073] jepsen worker 29 - jepsen.util 2029 :invoke :add 8680 INFO [2022-01-10 08:39:50,074] jepsen worker 616 - jepsen.util 1616 :invoke :add 8681 INFO [2022-01-10 08:39:50,076] jepsen worker 786 - jepsen.util 2786 :invoke :add 8682 INFO [2022-01-10 08:39:50,078] jepsen worker 58 - jepsen.util 2058 :invoke :add 8683 INFO [2022-01-10 08:39:50,079] jepsen worker 823 - jepsen.util 2823 :invoke :add 8684 INFO [2022-01-10 08:39:50,081] jepsen worker 377 - jepsen.util 1377 :invoke :add 8685 INFO [2022-01-10 08:39:50,082] jepsen worker 363 - jepsen.util 1363 :info :add 7407 :RequestTimeoutException INFO [2022-01-10 08:39:50,083] jepsen worker 777 - jepsen.util 777 :invoke :add 8686 INFO [2022-01-10 08:39:50,083] jepsen worker 935 - jepsen.util 1935 :info :add 7406 :RequestTimeoutException INFO [2022-01-10 08:39:50,085] jepsen worker 50 - jepsen.util 2050 :invoke :add 8687 INFO [2022-01-10 08:39:50,087] jepsen worker 580 - jepsen.util 2580 :invoke :add 8688 INFO [2022-01-10 08:39:50,087] jepsen worker 988 - jepsen.util 1988 :info :add 7409 :RequestTimeoutException INFO [2022-01-10 08:39:50,089] jepsen worker 954 - jepsen.util 1954 :invoke :add 8689 INFO [2022-01-10 08:39:50,091] jepsen worker 908 - jepsen.util 1908 :info :add 7408 :RequestTimeoutException INFO [2022-01-10 08:39:50,091] jepsen worker 566 - jepsen.util 1566 :invoke :add 8690 INFO [2022-01-10 08:39:50,091] jepsen worker 311 - jepsen.util 1311 :info :add 7411 :RequestTimeoutException INFO [2022-01-10 08:39:50,093] jepsen worker 689 - jepsen.util 1689 :info :add 7410 :RequestTimeoutException INFO [2022-01-10 08:39:50,093] jepsen worker 352 - jepsen.util 1352 :info :add 7412 :RequestTimeoutException INFO [2022-01-10 08:39:50,094] jepsen worker 158 - jepsen.util 1158 :invoke :add 8691 INFO [2022-01-10 08:39:50,097] jepsen worker 702 - jepsen.util 702 :invoke :add 8692 INFO [2022-01-10 08:39:50,098] jepsen worker 883 - jepsen.util 2883 :invoke :add 8693 INFO [2022-01-10 08:39:50,100] jepsen worker 180 - jepsen.util 1180 :info :add 7415 :RequestTimeoutException INFO [2022-01-10 08:39:50,100] jepsen worker 653 - jepsen.util 2653 :invoke :add 8694 INFO [2022-01-10 08:39:50,101] jepsen worker 161 - jepsen.util 2161 :ok :add 8629 INFO [2022-01-10 08:39:50,102] jepsen worker 864 - jepsen.util 1864 :info :add 7413 :RequestTimeoutException INFO [2022-01-10 08:39:50,103] jepsen worker 323 - jepsen.util 2323 :invoke :add 8695 INFO [2022-01-10 08:39:50,104] jepsen worker 164 - jepsen.util 164 :ok :add 8631 INFO [2022-01-10 08:39:50,106] jepsen worker 89 - jepsen.util 2089 :invoke :add 8696 INFO [2022-01-10 08:39:50,108] jepsen worker 794 - jepsen.util 1794 :invoke :add 8697 INFO [2022-01-10 08:39:50,108] jepsen worker 859 - jepsen.util 1859 :ok :add 8633 INFO [2022-01-10 08:39:50,109] jepsen worker 352 - jepsen.util 2352 :invoke :add 8698 INFO [2022-01-10 08:39:50,111] jepsen worker 999 - jepsen.util 999 :invoke :add 8699 INFO [2022-01-10 08:39:50,113] jepsen worker 288 - jepsen.util 1288 :info :add 7416 :RequestTimeoutException INFO [2022-01-10 08:39:50,113] jepsen worker 908 - jepsen.util 2908 :invoke :add 8700 INFO [2022-01-10 08:39:50,115] jepsen worker 617 - jepsen.util 2617 :ok :add 8640 INFO [2022-01-10 08:39:50,115] jepsen worker 497 - jepsen.util 1497 :ok :add 8658 INFO [2022-01-10 08:39:50,115] jepsen worker 522 - jepsen.util 2522 :ok :add 8674 INFO [2022-01-10 08:39:50,115] jepsen worker 886 - jepsen.util 2886 :invoke :add 8701 INFO [2022-01-10 08:39:50,115] jepsen worker 607 - jepsen.util 607 :ok :add 8628 INFO [2022-01-10 08:39:50,115] jepsen worker 805 - jepsen.util 1805 :ok :add 8630 INFO [2022-01-10 08:39:50,117] jepsen worker 917 - jepsen.util 1917 :ok :add 8632 INFO [2022-01-10 08:39:50,119] jepsen worker 288 - jepsen.util 2288 :invoke :add 8702 INFO [2022-01-10 08:39:50,120] jepsen worker 720 - jepsen.util 2720 :ok :add 8642 INFO [2022-01-10 08:39:50,120] jepsen worker 26 - jepsen.util 2026 :ok :add 8676 INFO [2022-01-10 08:39:50,121] jepsen worker 508 - jepsen.util 2508 :invoke :add 8703 INFO [2022-01-10 08:39:50,121] jepsen worker 409 - jepsen.util 1409 :ok :add 8641 INFO [2022-01-10 08:39:50,121] jepsen worker 677 - jepsen.util 2677 :ok :add 8659 INFO [2022-01-10 08:39:50,121] jepsen worker 594 - jepsen.util 594 :ok :add 8675 INFO [2022-01-10 08:39:50,123] jepsen worker 164 - jepsen.util 164 :invoke :add 8704 INFO [2022-01-10 08:39:50,125] jepsen worker 373 - jepsen.util 1373 :invoke :add 8705 INFO [2022-01-10 08:39:50,128] jepsen worker 677 - jepsen.util 2677 :invoke :add 8706 INFO [2022-01-10 08:39:50,129] jepsen worker 689 - jepsen.util 2689 :invoke :add 8707 INFO [2022-01-10 08:39:50,130] jepsen worker 979 - jepsen.util 1979 :ok :add 8643 INFO [2022-01-10 08:39:50,130] jepsen worker 558 - jepsen.util 1558 :info :add 7421 :RequestTimeoutException INFO [2022-01-10 08:39:50,130] jepsen worker 916 - jepsen.util 2916 :ok :add 8677 INFO [2022-01-10 08:39:50,131] jepsen worker 975 - jepsen.util 1975 :invoke :add 8708 INFO [2022-01-10 08:39:50,134] jepsen worker 323 - jepsen.util 2323 :ok :add 8695 INFO [2022-01-10 08:39:50,134] jepsen worker 373 - jepsen.util 1373 :ok :add 8705 INFO [2022-01-10 08:39:50,134] jepsen worker 409 - jepsen.util 1409 :invoke :add 8709 INFO [2022-01-10 08:39:50,137] jepsen worker 582 - jepsen.util 1582 :invoke :add 8710 INFO [2022-01-10 08:39:50,138] jepsen worker 653 - jepsen.util 2653 :ok :add 8694 INFO [2022-01-10 08:39:50,139] jepsen worker 164 - jepsen.util 164 :ok :add 8704 INFO [2022-01-10 08:39:50,139] jepsen worker 617 - jepsen.util 2617 :invoke :add 8711 INFO [2022-01-10 08:39:50,140] jepsen worker 794 - jepsen.util 1794 :ok :add 8697 INFO [2022-01-10 08:39:50,140] jepsen worker 689 - jepsen.util 2689 :ok :add 8707 INFO [2022-01-10 08:39:50,141] jepsen worker 109 - jepsen.util 1109 :invoke :add 8712 INFO [2022-01-10 08:39:50,144] jepsen worker 689 - jepsen.util 2689 :invoke :add 8713 INFO [2022-01-10 08:39:50,144] jepsen worker 115 - jepsen.util 115 :ok :add 7439 INFO [2022-01-10 08:39:50,145] jepsen worker 89 - jepsen.util 2089 :ok :add 8696 INFO [2022-01-10 08:39:50,145] jepsen worker 677 - jepsen.util 2677 :ok :add 8706 INFO [2022-01-10 08:39:50,146] jepsen worker 363 - jepsen.util 2363 :invoke :add 8714 INFO [2022-01-10 08:39:50,149] jepsen worker 982 - jepsen.util 1982 :invoke :add 8715 INFO [2022-01-10 08:39:50,150] jepsen worker 952 - jepsen.util 1952 :invoke :add 8716 INFO [2022-01-10 08:39:50,152] jepsen worker 89 - jepsen.util 2089 :invoke :add 8717 INFO [2022-01-10 08:39:50,153] jepsen worker 0 - jepsen.util 1000 :info :add 7422 :RequestTimeoutException INFO [2022-01-10 08:39:50,154] jepsen worker 859 - jepsen.util 1859 :invoke :add 8718 INFO [2022-01-10 08:39:50,156] jepsen worker 714 - jepsen.util 2714 :invoke :add 8719 INFO [2022-01-10 08:39:50,158] jepsen worker 311 - jepsen.util 2311 :invoke :add 8720 INFO [2022-01-10 08:39:50,160] jepsen worker 323 - jepsen.util 2323 :invoke :add 8721 INFO [2022-01-10 08:39:50,162] jepsen worker 677 - jepsen.util 2677 :invoke :add 8722 INFO [2022-01-10 08:39:50,163] jepsen worker 38 - jepsen.util 1038 :info :add 7424 :RequestTimeoutException INFO [2022-01-10 08:39:50,164] jepsen worker 21 - jepsen.util 21 :invoke :add 8723 INFO [2022-01-10 08:39:50,166] jepsen worker 415 - jepsen.util 1415 :invoke :add 8724 INFO [2022-01-10 08:39:50,168] jepsen worker 93 - jepsen.util 2093 :invoke :add 8725 INFO [2022-01-10 08:39:50,170] jepsen worker 164 - jepsen.util 164 :invoke :add 8726 INFO [2022-01-10 08:39:50,171] jepsen worker 187 - jepsen.util 2187 :invoke :add 8727 INFO [2022-01-10 08:39:50,173] jepsen worker 642 - jepsen.util 1642 :info :add 7425 :RequestTimeoutException INFO [2022-01-10 08:39:50,173] jepsen worker 864 - jepsen.util 2864 :invoke :add 8728 INFO [2022-01-10 08:39:50,175] jepsen worker 0 - jepsen.util 2000 :invoke :add 8729 INFO [2022-01-10 08:39:50,177] jepsen worker 628 - jepsen.util 1628 :info :add 7426 :RequestTimeoutException INFO [2022-01-10 08:39:50,177] jepsen worker 653 - jepsen.util 2653 :invoke :add 8730 INFO [2022-01-10 08:39:50,180] jepsen worker 726 - jepsen.util 1726 :info :add 7428 :RequestTimeoutException INFO [2022-01-10 08:39:50,180] jepsen worker 52 - jepsen.util 2052 :invoke :add 8731 INFO [2022-01-10 08:39:50,182] jepsen worker 719 - jepsen.util 719 :info :add 7427 :RequestTimeoutException INFO [2022-01-10 08:39:50,183] jepsen worker 589 - jepsen.util 1589 :info :add 7429 :RequestTimeoutException INFO [2022-01-10 08:39:50,183] jepsen worker 864 - jepsen.util 2864 :ok :add 8728 INFO [2022-01-10 08:39:50,184] jepsen worker 988 - jepsen.util 2988 :invoke :add 8732 INFO [2022-01-10 08:39:50,187] jepsen worker 241 - jepsen.util 2241 :invoke :add 8733 INFO [2022-01-10 08:39:50,189] jepsen worker 713 - jepsen.util 2713 :invoke :add 8734 INFO [2022-01-10 08:39:50,190] jepsen worker 0 - jepsen.util 2000 :ok :add 8729 INFO [2022-01-10 08:39:50,191] jepsen worker 522 - jepsen.util 2522 :invoke :add 8735 INFO [2022-01-10 08:39:50,191] jepsen worker 743 - jepsen.util 1743 :info :add 7430 :RequestTimeoutException INFO [2022-01-10 08:39:50,192] jepsen worker 653 - jepsen.util 2653 :ok :add 8730 INFO [2022-01-10 08:39:50,193] jepsen worker 162 - jepsen.util 2162 :invoke :add 8736 INFO [2022-01-10 08:39:50,193] jepsen worker 314 - jepsen.util 1314 :info :add 7431 :RequestTimeoutException INFO [2022-01-10 08:39:50,194] jepsen worker 52 - jepsen.util 2052 :ok :add 8731 INFO [2022-01-10 08:39:50,196] jepsen worker 988 - jepsen.util 2988 :ok :add 8732 INFO [2022-01-10 08:39:50,197] jepsen worker 115 - jepsen.util 115 :invoke :add 8737 INFO [2022-01-10 08:39:50,198] jepsen worker 241 - jepsen.util 2241 :ok :add 8733 INFO [2022-01-10 08:39:50,199] jepsen worker 373 - jepsen.util 1373 :invoke :add 8738 INFO [2022-01-10 08:39:50,200] jepsen worker 794 - jepsen.util 1794 :invoke :add 8739 INFO [2022-01-10 08:39:50,203] jepsen worker 653 - jepsen.util 2653 :invoke :add 8740 INFO [2022-01-10 08:39:50,205] jepsen worker 719 - jepsen.util 1719 :invoke :add 8741 INFO [2022-01-10 08:39:50,207] jepsen worker 38 - jepsen.util 2038 :invoke :add 8742 INFO [2022-01-10 08:39:50,208] jepsen worker 743 - jepsen.util 2743 :invoke :add 8743 INFO [2022-01-10 08:39:50,210] jepsen worker 0 - jepsen.util 2000 :invoke :add 8744 INFO [2022-01-10 08:39:50,211] jepsen worker 241 - jepsen.util 2241 :invoke :add 8745 INFO [2022-01-10 08:39:50,212] jepsen worker 653 - jepsen.util 2653 :ok :add 8740 INFO [2022-01-10 08:39:50,212] jepsen worker 719 - jepsen.util 1719 :ok :add 8741 INFO [2022-01-10 08:39:50,213] jepsen worker 26 - jepsen.util 2026 :invoke :add 8746 INFO [2022-01-10 08:39:50,214] jepsen worker 38 - jepsen.util 2038 :ok :add 8742 INFO [2022-01-10 08:39:50,216] jepsen worker 642 - jepsen.util 2642 :invoke :add 8747 INFO [2022-01-10 08:39:50,216] jepsen worker 743 - jepsen.util 2743 :ok :add 8743 INFO [2022-01-10 08:39:50,218] jepsen worker 864 - jepsen.util 2864 :invoke :add 8748 INFO [2022-01-10 08:39:50,220] jepsen worker 189 - jepsen.util 2189 :invoke :add 8749 INFO [2022-01-10 08:39:50,221] jepsen worker 805 - jepsen.util 1805 :invoke :add 8750 INFO [2022-01-10 08:39:50,223] jepsen worker 719 - jepsen.util 1719 :invoke :add 8751 INFO [2022-01-10 08:39:50,225] jepsen worker 818 - jepsen.util 2818 :invoke :add 8752 INFO [2022-01-10 08:39:50,227] jepsen worker 743 - jepsen.util 2743 :invoke :add 8753 INFO [2022-01-10 08:39:50,228] jepsen worker 497 - jepsen.util 1497 :invoke :add 8754 INFO [2022-01-10 08:39:50,230] jepsen worker 52 - jepsen.util 2052 :invoke :add 8755 INFO [2022-01-10 08:39:50,232] jepsen worker 720 - jepsen.util 2720 :invoke :add 8756 INFO [2022-01-10 08:39:50,234] jepsen worker 962 - jepsen.util 2962 :invoke :add 8757 INFO [2022-01-10 08:39:50,236] jepsen worker 665 - jepsen.util 1665 :invoke :add 8758 INFO [2022-01-10 08:39:50,237] jepsen worker 940 - jepsen.util 1940 :invoke :add 8759 INFO [2022-01-10 08:39:50,239] jepsen worker 988 - jepsen.util 2988 :invoke :add 8760 INFO [2022-01-10 08:39:50,241] jepsen worker 558 - jepsen.util 2558 :invoke :add 8761 INFO [2022-01-10 08:39:50,242] jepsen worker 607 - jepsen.util 607 :invoke :add 8762 INFO [2022-01-10 08:39:50,242] jepsen worker 940 - jepsen.util 1940 :ok :add 8759 INFO [2022-01-10 08:39:50,244] jepsen worker 665 - jepsen.util 1665 :ok :add 8758 INFO [2022-01-10 08:39:50,244] jepsen worker 825 - jepsen.util 1825 :invoke :add 8763 INFO [2022-01-10 08:39:50,246] jepsen worker 940 - jepsen.util 1940 :invoke :add 8764 INFO [2022-01-10 08:39:50,247] jepsen worker 917 - jepsen.util 1917 :invoke :add 8765 INFO [2022-01-10 08:39:50,250] jepsen worker 895 - jepsen.util 1895 :invoke :add 8766 INFO [2022-01-10 08:39:50,251] jepsen worker 653 - jepsen.util 2653 :invoke :add 8767 INFO [2022-01-10 08:39:50,253] jepsen worker 39 - jepsen.util 1039 :invoke :add 8768 INFO [2022-01-10 08:39:50,255] jepsen worker 594 - jepsen.util 594 :invoke :add 8769 INFO [2022-01-10 08:39:50,256] jepsen worker 979 - jepsen.util 1979 :invoke :add 8770 INFO [2022-01-10 08:39:50,258] jepsen worker 665 - jepsen.util 1665 :invoke :add 8771 INFO [2022-01-10 08:39:50,260] jepsen worker 483 - jepsen.util 1483 :invoke :add 8772 INFO [2022-01-10 08:39:50,261] jepsen worker 916 - jepsen.util 2916 :invoke :add 8773 INFO [2022-01-10 08:39:50,263] jepsen worker 628 - jepsen.util 2628 :invoke :add 8774 INFO [2022-01-10 08:39:50,265] jepsen worker 38 - jepsen.util 2038 :invoke :add 8775 INFO [2022-01-10 08:39:50,267] jepsen worker 494 - jepsen.util 494 :invoke :add 8776 INFO [2022-01-10 08:39:50,269] jepsen worker 935 - jepsen.util 2935 :invoke :add 8777 INFO [2022-01-10 08:39:50,271] jepsen worker 92 - jepsen.util 92 :ok :add 7631 INFO [2022-01-10 08:39:50,271] jepsen worker 38 - jepsen.util 2038 :ok :add 8775 INFO [2022-01-10 08:39:50,272] jepsen worker 657 - jepsen.util 2657 :invoke :add 8778 INFO [2022-01-10 08:39:50,274] jepsen worker 69 - jepsen.util 1069 :invoke :add 8779 INFO [2022-01-10 08:39:50,275] jepsen worker 628 - jepsen.util 2628 :ok :add 8774 INFO [2022-01-10 08:39:50,275] jepsen worker 180 - jepsen.util 2180 :invoke :add 8780 INFO [2022-01-10 08:39:50,276] jepsen worker 707 - jepsen.util 707 :ok :add 7540 INFO [2022-01-10 08:39:50,276] jepsen worker 781 - jepsen.util 1781 :ok :add 7558 INFO [2022-01-10 08:39:50,276] jepsen worker 354 - jepsen.util 1354 :ok :add 7629 INFO [2022-01-10 08:39:50,277] jepsen worker 498 - jepsen.util 1498 :ok :add 7605 INFO [2022-01-10 08:39:50,277] jepsen worker 935 - jepsen.util 2935 :ok :add 8777 INFO [2022-01-10 08:39:50,278] jepsen worker 314 - jepsen.util 2314 :invoke :add 8781 INFO [2022-01-10 08:39:50,280] jepsen worker 494 - jepsen.util 494 :ok :add 8776 INFO [2022-01-10 08:39:50,282] jepsen worker 161 - jepsen.util 2161 :invoke :add 8782 INFO [2022-01-10 08:39:50,284] jepsen worker 781 - jepsen.util 1781 :invoke :add 8783 INFO [2022-01-10 08:39:50,286] jepsen worker 589 - jepsen.util 2589 :invoke :add 8784 INFO [2022-01-10 08:39:50,289] jepsen worker 935 - jepsen.util 2935 :invoke :add 8785 INFO [2022-01-10 08:39:50,291] jepsen worker 726 - jepsen.util 2726 :invoke :add 8786 INFO [2022-01-10 08:39:50,294] jepsen worker 494 - jepsen.util 494 :invoke :add 8787 INFO [2022-01-10 08:39:50,295] jepsen worker 38 - jepsen.util 2038 :invoke :add 8788 INFO [2022-01-10 08:39:50,297] jepsen worker 223 - jepsen.util 223 :info :add 7432 :RequestTimeoutException INFO [2022-01-10 08:39:50,297] jepsen worker 934 - jepsen.util 934 :info :add 7433 :RequestTimeoutException INFO [2022-01-10 08:39:50,298] jepsen worker 628 - jepsen.util 2628 :invoke :add 8789 INFO [2022-01-10 08:39:50,301] jepsen worker 354 - jepsen.util 1354 :invoke :add 8790 INFO [2022-01-10 08:39:50,303] jepsen worker 934 - jepsen.util 1934 :invoke :add 8791 INFO [2022-01-10 08:39:50,305] jepsen worker 707 - jepsen.util 707 :invoke :add 8792 INFO [2022-01-10 08:39:50,307] jepsen worker 223 - jepsen.util 1223 :invoke :add 8793 INFO [2022-01-10 08:39:50,308] jepsen worker 92 - jepsen.util 92 :invoke :add 8794 INFO [2022-01-10 08:39:50,311] jepsen worker 498 - jepsen.util 1498 :invoke :add 8795 INFO [2022-01-10 08:39:50,317] jepsen worker 492 - jepsen.util 492 :info :add 7440 :RequestTimeoutException INFO [2022-01-10 08:39:50,321] jepsen worker 492 - jepsen.util 1492 :invoke :add 8796 INFO [2022-01-10 08:39:50,323] jepsen worker 92 - jepsen.util 92 :ok :add 8794 INFO [2022-01-10 08:39:50,326] jepsen worker 92 - jepsen.util 92 :invoke :add 8797 INFO [2022-01-10 08:39:50,327] jepsen worker 498 - jepsen.util 1498 :ok :add 8795 INFO [2022-01-10 08:39:50,331] jepsen worker 498 - jepsen.util 1498 :invoke :add 8798 INFO [2022-01-10 08:39:50,345] jepsen worker 492 - jepsen.util 1492 :ok :add 8796 INFO [2022-01-10 08:39:50,345] jepsen worker 92 - jepsen.util 92 :ok :add 8797 INFO [2022-01-10 08:39:50,348] jepsen worker 92 - jepsen.util 92 :invoke :add 8799 INFO [2022-01-10 08:39:50,350] jepsen worker 492 - jepsen.util 1492 :invoke :add 8800 INFO [2022-01-10 08:39:50,362] jepsen worker 903 - jepsen.util 1903 :info :add 7441 :RequestTimeoutException INFO [2022-01-10 08:39:50,365] jepsen worker 903 - jepsen.util 2903 :invoke :add 8801 INFO [2022-01-10 08:39:50,367] jepsen worker 122 - jepsen.util 1122 :info :add 7442 :RequestTimeoutException INFO [2022-01-10 08:39:50,370] jepsen worker 122 - jepsen.util 2122 :invoke :add 8802 INFO [2022-01-10 08:39:50,371] jepsen worker 766 - jepsen.util 1766 :info :add 7443 :RequestTimeoutException INFO [2022-01-10 08:39:50,373] jepsen worker 766 - jepsen.util 2766 :invoke :add 8803 INFO [2022-01-10 08:39:50,467] jepsen worker 894 - jepsen.util 1894 :ok :add 7549 INFO [2022-01-10 08:39:50,468] jepsen worker 894 - jepsen.util 1894 :invoke :add 8804 INFO [2022-01-10 08:39:50,472] jepsen worker 178 - jepsen.util 1178 :ok :add 7517 INFO [2022-01-10 08:39:50,472] jepsen worker 420 - jepsen.util 1420 :ok :add 7487 INFO [2022-01-10 08:39:50,472] jepsen worker 745 - jepsen.util 1745 :ok :add 7523 INFO [2022-01-10 08:39:50,475] jepsen worker 745 - jepsen.util 1745 :invoke :add 8805 INFO [2022-01-10 08:39:50,477] jepsen worker 178 - jepsen.util 1178 :invoke :add 8806 INFO [2022-01-10 08:39:50,479] jepsen worker 420 - jepsen.util 1420 :invoke :add 8807 INFO [2022-01-10 08:39:50,542] jepsen worker 470 - jepsen.util 470 :info :add 7450 :RequestTimeoutException INFO [2022-01-10 08:39:50,543] jepsen worker 949 - jepsen.util 949 :info :add 7451 :RequestTimeoutException INFO [2022-01-10 08:39:50,546] jepsen worker 949 - jepsen.util 1949 :invoke :add 8808 INFO [2022-01-10 08:39:50,547] jepsen worker 470 - jepsen.util 1470 :invoke :add 8809 INFO [2022-01-10 08:39:50,550] jepsen worker 847 - jepsen.util 847 :info :add 7454 :RequestTimeoutException INFO [2022-01-10 08:39:50,553] jepsen worker 418 - jepsen.util 418 :info :add 7452 :RequestTimeoutException INFO [2022-01-10 08:39:50,554] jepsen worker 847 - jepsen.util 1847 :invoke :add 8810 INFO [2022-01-10 08:39:50,556] jepsen worker 418 - jepsen.util 1418 :invoke :add 8811 INFO [2022-01-10 08:39:50,560] jepsen worker 421 - jepsen.util 1421 :info :add 7455 :RequestTimeoutException INFO [2022-01-10 08:39:50,562] jepsen worker 590 - jepsen.util 590 :info :add 7456 :RequestTimeoutException INFO [2022-01-10 08:39:50,563] jepsen worker 912 - jepsen.util 912 :info :add 7457 :RequestTimeoutException INFO [2022-01-10 08:39:50,563] jepsen worker 421 - jepsen.util 2421 :invoke :add 8812 INFO [2022-01-10 08:39:50,564] jepsen worker 824 - jepsen.util 824 :info :add 7458 :RequestTimeoutException INFO [2022-01-10 08:39:50,568] jepsen worker 912 - jepsen.util 1912 :invoke :add 8813 INFO [2022-01-10 08:39:50,569] jepsen worker 590 - jepsen.util 1590 :invoke :add 8814 INFO [2022-01-10 08:39:50,572] jepsen worker 824 - jepsen.util 1824 :invoke :add 8815 INFO [2022-01-10 08:39:50,579] jepsen worker 824 - jepsen.util 1824 :ok :add 8815 INFO [2022-01-10 08:39:50,579] jepsen worker 590 - jepsen.util 1590 :ok :add 8814 INFO [2022-01-10 08:39:50,582] jepsen worker 824 - jepsen.util 1824 :invoke :add 8816 INFO [2022-01-10 08:39:50,584] jepsen worker 590 - jepsen.util 1590 :invoke :add 8817 INFO [2022-01-10 08:39:50,588] jepsen worker 824 - jepsen.util 1824 :ok :add 8816 INFO [2022-01-10 08:39:50,590] jepsen worker 590 - jepsen.util 1590 :ok :add 8817 INFO [2022-01-10 08:39:50,590] jepsen worker 824 - jepsen.util 1824 :invoke :add 8818 INFO [2022-01-10 08:39:50,593] jepsen worker 590 - jepsen.util 1590 :invoke :add 8819 INFO [2022-01-10 08:39:50,631] jepsen worker 959 - jepsen.util 1959 :info :add 7459 :RequestTimeoutException INFO [2022-01-10 08:39:50,633] jepsen worker 740 - jepsen.util 1740 :info :add 7460 :RequestTimeoutException INFO [2022-01-10 08:39:50,635] jepsen worker 959 - jepsen.util 2959 :invoke :add 8820 INFO [2022-01-10 08:39:50,637] jepsen worker 740 - jepsen.util 2740 :invoke :add 8821 INFO [2022-01-10 08:39:50,641] jepsen worker 884 - jepsen.util 1884 :info :add 7461 :RequestTimeoutException INFO [2022-01-10 08:39:50,643] jepsen worker 379 - jepsen.util 1379 :info :add 7462 :RequestTimeoutException INFO [2022-01-10 08:39:50,643] jepsen worker 884 - jepsen.util 2884 :invoke :add 8822 INFO [2022-01-10 08:39:50,646] jepsen worker 379 - jepsen.util 2379 :invoke :add 8823 INFO [2022-01-10 08:39:50,742] jepsen worker 730 - jepsen.util 1730 :info :add 7463 :RequestTimeoutException INFO [2022-01-10 08:39:50,745] jepsen worker 730 - jepsen.util 2730 :invoke :add 8824 INFO [2022-01-10 08:39:50,763] jepsen worker 651 - jepsen.util 1651 :info :add 7466 :RequestTimeoutException INFO [2022-01-10 08:39:50,766] jepsen worker 651 - jepsen.util 2651 :invoke :add 8825 INFO [2022-01-10 08:39:50,771] jepsen worker 644 - jepsen.util 1644 :info :add 7467 :RequestTimeoutException INFO [2022-01-10 08:39:50,774] jepsen worker 644 - jepsen.util 2644 :invoke :add 8826 INFO [2022-01-10 08:39:50,810] jepsen worker 959 - jepsen.util 2959 :ok :add 8820 INFO [2022-01-10 08:39:50,812] jepsen worker 740 - jepsen.util 2740 :ok :add 8821 INFO [2022-01-10 08:39:50,812] jepsen worker 959 - jepsen.util 2959 :invoke :add 8827 INFO [2022-01-10 08:39:50,813] jepsen worker 884 - jepsen.util 2884 :ok :add 8822 INFO [2022-01-10 08:39:50,814] jepsen worker 740 - jepsen.util 2740 :invoke :add 8828 INFO [2022-01-10 08:39:50,815] jepsen worker 379 - jepsen.util 2379 :ok :add 8823 INFO [2022-01-10 08:39:50,816] jepsen worker 884 - jepsen.util 2884 :invoke :add 8829 INFO [2022-01-10 08:39:50,818] jepsen worker 379 - jepsen.util 2379 :invoke :add 8830 INFO [2022-01-10 08:39:50,827] jepsen worker 980 - jepsen.util 980 :info :add 7474 :RequestTimeoutException INFO [2022-01-10 08:39:50,830] jepsen worker 980 - jepsen.util 1980 :invoke :add 8831 INFO [2022-01-10 08:39:50,840] jepsen worker 153 - jepsen.util 1153 :info :add 7475 :RequestTimeoutException INFO [2022-01-10 08:39:50,843] jepsen worker 646 - jepsen.util 1646 :info :add 7476 :RequestTimeoutException INFO [2022-01-10 08:39:50,844] jepsen worker 153 - jepsen.util 2153 :invoke :add 8832 INFO [2022-01-10 08:39:50,846] jepsen worker 646 - jepsen.util 2646 :invoke :add 8833 INFO [2022-01-10 08:39:51,111] jepsen worker 958 - jepsen.util 1958 :info :add 7477 :RequestTimeoutException INFO [2022-01-10 08:39:51,114] jepsen worker 854 - jepsen.util 1854 :info :add 7478 :RequestTimeoutException INFO [2022-01-10 08:39:51,114] jepsen worker 958 - jepsen.util 2958 :invoke :add 8834 INFO [2022-01-10 08:39:51,117] jepsen worker 854 - jepsen.util 2854 :invoke :add 8835 INFO [2022-01-10 08:39:51,157] jepsen worker 533 - jepsen.util 533 :info :add 7479 :RequestTimeoutException INFO [2022-01-10 08:39:51,161] jepsen worker 533 - jepsen.util 1533 :invoke :add 8836 INFO [2022-01-10 08:39:51,161] jepsen worker 224 - jepsen.util 224 :info :add 7480 :RequestTimeoutException INFO [2022-01-10 08:39:51,164] jepsen worker 414 - jepsen.util 414 :info :add 7481 :RequestTimeoutException INFO [2022-01-10 08:39:51,165] jepsen worker 224 - jepsen.util 1224 :invoke :add 8837 INFO [2022-01-10 08:39:51,168] jepsen worker 414 - jepsen.util 1414 :invoke :add 8838 INFO [2022-01-10 08:39:51,175] jepsen worker 414 - jepsen.util 1414 :ok :add 8838 INFO [2022-01-10 08:39:51,178] jepsen worker 414 - jepsen.util 1414 :invoke :add 8839 INFO [2022-01-10 08:39:51,184] jepsen worker 414 - jepsen.util 1414 :ok :add 8839 INFO [2022-01-10 08:39:51,187] jepsen worker 414 - jepsen.util 1414 :invoke :add 8840 INFO [2022-01-10 08:39:51,221] jepsen worker 245 - jepsen.util 245 :info :add 7482 :RequestTimeoutException INFO [2022-01-10 08:39:51,224] jepsen worker 245 - jepsen.util 1245 :invoke :add 8841 INFO [2022-01-10 08:39:51,227] jepsen worker 135 - jepsen.util 135 :info :add 7483 :RequestTimeoutException INFO [2022-01-10 08:39:51,231] jepsen worker 135 - jepsen.util 1135 :invoke :add 8842 INFO [2022-01-10 08:39:51,257] jepsen worker 796 - jepsen.util 796 :info :add 7494 :RequestTimeoutException INFO [2022-01-10 08:39:51,257] jepsen worker 313 - jepsen.util 313 :info :add 7496 :RequestTimeoutException INFO [2022-01-10 08:39:51,257] jepsen worker 779 - jepsen.util 1779 :info :add 7495 :RequestTimeoutException INFO [2022-01-10 08:39:51,261] jepsen worker 796 - jepsen.util 1796 :invoke :add 8843 INFO [2022-01-10 08:39:51,262] jepsen worker 779 - jepsen.util 2779 :invoke :add 8844 INFO [2022-01-10 08:39:51,264] jepsen worker 313 - jepsen.util 1313 :invoke :add 8845 INFO [2022-01-10 08:39:51,267] jepsen worker 30 - jepsen.util 1030 :info :add 7497 :RequestTimeoutException INFO [2022-01-10 08:39:51,269] jepsen worker 30 - jepsen.util 2030 :invoke :add 8846 INFO [2022-01-10 08:39:51,293] jepsen worker 592 - jepsen.util 592 :info :add 7498 :RequestTimeoutException INFO [2022-01-10 08:39:51,296] jepsen worker 592 - jepsen.util 1592 :invoke :add 8847 INFO [2022-01-10 08:39:51,301] jepsen worker 172 - jepsen.util 172 :info :add 7499 :RequestTimeoutException INFO [2022-01-10 08:39:51,305] jepsen worker 172 - jepsen.util 1172 :invoke :add 8848 INFO [2022-01-10 08:39:51,310] jepsen worker 172 - jepsen.util 1172 :ok :add 8848 INFO [2022-01-10 08:39:51,314] jepsen worker 172 - jepsen.util 1172 :invoke :add 8849 INFO [2022-01-10 08:39:51,318] jepsen worker 172 - jepsen.util 1172 :ok :add 8849 INFO [2022-01-10 08:39:51,320] jepsen worker 172 - jepsen.util 1172 :invoke :add 8850 INFO [2022-01-10 08:39:51,326] jepsen worker 172 - jepsen.util 1172 :ok :add 8850 INFO [2022-01-10 08:39:51,328] jepsen worker 172 - jepsen.util 1172 :invoke :add 8851 INFO [2022-01-10 08:39:51,335] jepsen worker 172 - jepsen.util 1172 :ok :add 8851 INFO [2022-01-10 08:39:51,338] jepsen worker 172 - jepsen.util 1172 :invoke :add 8852 INFO [2022-01-10 08:39:51,346] jepsen worker 172 - jepsen.util 1172 :ok :add 8852 INFO [2022-01-10 08:39:51,351] jepsen worker 172 - jepsen.util 1172 :invoke :add 8853 INFO [2022-01-10 08:39:51,357] jepsen worker 172 - jepsen.util 1172 :ok :add 8853 INFO [2022-01-10 08:39:51,359] jepsen worker 172 - jepsen.util 1172 :invoke :add 8854 INFO [2022-01-10 08:39:51,490] jepsen worker 910 - jepsen.util 1910 :ok :add 7819 INFO [2022-01-10 08:39:51,492] jepsen worker 910 - jepsen.util 1910 :invoke :add 8855 INFO [2022-01-10 08:39:51,627] jepsen worker 858 - jepsen.util 858 :info :add 7505 :RequestTimeoutException INFO [2022-01-10 08:39:51,630] jepsen worker 858 - jepsen.util 1858 :invoke :add 8856 INFO [2022-01-10 08:39:51,631] jepsen worker 710 - jepsen.util 710 :info :add 7504 :RequestTimeoutException INFO [2022-01-10 08:39:51,633] jepsen worker 698 - jepsen.util 698 :info :add 7507 :RequestTimeoutException INFO [2022-01-10 08:39:51,635] jepsen worker 710 - jepsen.util 1710 :invoke :add 8857 INFO [2022-01-10 08:39:51,638] jepsen worker 698 - jepsen.util 1698 :invoke :add 8858 INFO [2022-01-10 08:39:51,639] jepsen worker 435 - jepsen.util 435 :info :add 7509 :RequestTimeoutException INFO [2022-01-10 08:39:51,640] jepsen worker 238 - jepsen.util 1238 :info :add 7506 :RequestTimeoutException INFO [2022-01-10 08:39:51,643] jepsen worker 783 - jepsen.util 783 :info :add 7508 :RequestTimeoutException INFO [2022-01-10 08:39:51,643] jepsen worker 499 - jepsen.util 499 :info :add 7510 :RequestTimeoutException INFO [2022-01-10 08:39:51,643] jepsen worker 453 - jepsen.util 453 :info :add 7511 :RequestTimeoutException INFO [2022-01-10 08:39:51,644] jepsen worker 238 - jepsen.util 2238 :invoke :add 8859 INFO [2022-01-10 08:39:51,647] jepsen worker 8 - jepsen.util 8 :info :add 7512 :RequestTimeoutException INFO [2022-01-10 08:39:51,648] jepsen worker 435 - jepsen.util 1435 :invoke :add 8860 INFO [2022-01-10 08:39:51,650] jepsen worker 104 - jepsen.util 1104 :info :add 7513 :RequestTimeoutException INFO [2022-01-10 08:39:51,651] jepsen worker 783 - jepsen.util 1783 :invoke :add 8861 INFO [2022-01-10 08:39:51,653] jepsen worker 499 - jepsen.util 1499 :invoke :add 8862 INFO [2022-01-10 08:39:51,655] jepsen worker 8 - jepsen.util 1008 :invoke :add 8863 INFO [2022-01-10 08:39:51,657] jepsen worker 453 - jepsen.util 1453 :invoke :add 8864 INFO [2022-01-10 08:39:51,659] jepsen worker 104 - jepsen.util 2104 :invoke :add 8865 INFO [2022-01-10 08:39:51,663] jepsen worker 453 - jepsen.util 1453 :ok :add 8864 INFO [2022-01-10 08:39:51,665] jepsen worker 453 - jepsen.util 1453 :invoke :add 8866 INFO [2022-01-10 08:39:51,665] jepsen worker 104 - jepsen.util 2104 :ok :add 8865 INFO [2022-01-10 08:39:51,669] jepsen worker 104 - jepsen.util 2104 :invoke :add 8867 INFO [2022-01-10 08:39:51,670] jepsen worker 112 - jepsen.util 1112 :info :add 7515 :RequestTimeoutException INFO [2022-01-10 08:39:51,674] jepsen worker 453 - jepsen.util 1453 :ok :add 8866 INFO [2022-01-10 08:39:51,674] jepsen worker 104 - jepsen.util 2104 :ok :add 8867 INFO [2022-01-10 08:39:51,675] jepsen worker 112 - jepsen.util 2112 :invoke :add 8868 INFO [2022-01-10 08:39:51,677] jepsen worker 453 - jepsen.util 1453 :invoke :add 8869 INFO [2022-01-10 08:39:51,679] jepsen worker 104 - jepsen.util 2104 :invoke :add 8870 INFO [2022-01-10 08:39:51,782] jepsen worker 86 - jepsen.util 1086 :info :add 7521 :RequestTimeoutException INFO [2022-01-10 08:39:51,784] jepsen worker 86 - jepsen.util 2086 :invoke :add 8871 INFO [2022-01-10 08:39:51,797] jepsen worker 563 - jepsen.util 1563 :info :add 7524 :RequestTimeoutException INFO [2022-01-10 08:39:51,799] jepsen worker 563 - jepsen.util 2563 :invoke :add 8872 INFO [2022-01-10 08:39:51,800] jepsen worker 968 - jepsen.util 1968 :info :add 7525 :RequestTimeoutException INFO [2022-01-10 08:39:51,803] jepsen worker 968 - jepsen.util 2968 :invoke :add 8873 INFO [2022-01-10 08:39:51,961] jepsen worker 995 - jepsen.util 1995 :info :add 7527 :RequestTimeoutException INFO [2022-01-10 08:39:51,963] jepsen worker 66 - jepsen.util 1066 :info :add 7528 :RequestTimeoutException INFO [2022-01-10 08:39:51,964] jepsen worker 995 - jepsen.util 2995 :invoke :add 8874 INFO [2022-01-10 08:39:51,967] jepsen worker 66 - jepsen.util 2066 :invoke :add 8875 INFO [2022-01-10 08:39:51,973] jepsen worker 520 - jepsen.util 1520 :info :add 7529 :RequestTimeoutException INFO [2022-01-10 08:39:51,975] jepsen worker 520 - jepsen.util 2520 :invoke :add 8876 INFO [2022-01-10 08:39:51,987] jepsen worker 216 - jepsen.util 1216 :info :add 7530 :RequestTimeoutException INFO [2022-01-10 08:39:51,990] jepsen worker 216 - jepsen.util 2216 :invoke :add 8877 INFO [2022-01-10 08:39:51,990] jepsen worker 256 - jepsen.util 1256 :info :add 7531 :RequestTimeoutException INFO [2022-01-10 08:39:51,993] jepsen worker 256 - jepsen.util 2256 :invoke :add 8878 INFO [2022-01-10 08:39:52,053] jepsen worker 506 - jepsen.util 506 :info :add 7533 :RequestTimeoutException INFO [2022-01-10 08:39:52,058] jepsen worker 506 - jepsen.util 1506 :invoke :add 8879 INFO [2022-01-10 08:39:52,081] jepsen worker 371 - jepsen.util 371 :info :add 7539 :RequestTimeoutException INFO [2022-01-10 08:39:52,085] jepsen worker 371 - jepsen.util 1371 :invoke :add 8880 INFO [2022-01-10 08:39:52,087] jepsen worker 564 - jepsen.util 564 :info :add 7541 :RequestTimeoutException INFO [2022-01-10 08:39:52,090] jepsen worker 564 - jepsen.util 1564 :invoke :add 8881 INFO [2022-01-10 08:39:52,091] jepsen worker 579 - jepsen.util 579 :info :add 7542 :RequestTimeoutException INFO [2022-01-10 08:39:52,093] jepsen worker 303 - jepsen.util 303 :info :add 7543 :RequestTimeoutException INFO [2022-01-10 08:39:52,094] jepsen worker 579 - jepsen.util 1579 :invoke :add 8882 INFO [2022-01-10 08:39:52,096] jepsen worker 303 - jepsen.util 1303 :invoke :add 8883 INFO [2022-01-10 08:39:52,173] jepsen worker 321 - jepsen.util 321 :info :add 7551 :RequestTimeoutException INFO [2022-01-10 08:39:52,176] jepsen worker 321 - jepsen.util 1321 :invoke :add 8884 INFO [2022-01-10 08:39:52,329] jepsen worker 321 - jepsen.util 1321 :ok :add 8884 INFO [2022-01-10 08:39:52,331] jepsen worker 321 - jepsen.util 1321 :invoke :add 8885 INFO [2022-01-10 08:39:52,337] jepsen worker 432 - jepsen.util 1432 :info :add 7552 :RequestTimeoutException INFO [2022-01-10 08:39:52,337] jepsen worker 206 - jepsen.util 206 :info :add 7553 :RequestTimeoutException INFO [2022-01-10 08:39:52,338] jepsen worker 321 - jepsen.util 1321 :ok :add 8885 INFO [2022-01-10 08:39:52,340] jepsen worker 562 - jepsen.util 562 :info :add 7554 :RequestTimeoutException INFO [2022-01-10 08:39:52,341] jepsen worker 321 - jepsen.util 1321 :invoke :add 8886 INFO [2022-01-10 08:39:52,344] jepsen worker 206 - jepsen.util 1206 :invoke :add 8887 INFO [2022-01-10 08:39:52,346] jepsen worker 432 - jepsen.util 2432 :invoke :add 8888 INFO [2022-01-10 08:39:52,347] jepsen worker 321 - jepsen.util 1321 :ok :add 8886 INFO [2022-01-10 08:39:52,347] jepsen worker 562 - jepsen.util 1562 :invoke :add 8889 INFO [2022-01-10 08:39:52,350] jepsen worker 321 - jepsen.util 1321 :invoke :add 8890 INFO [2022-01-10 08:39:52,351] jepsen worker 849 - jepsen.util 1849 :info :add 7557 :RequestTimeoutException INFO [2022-01-10 08:39:52,351] jepsen worker 596 - jepsen.util 596 :info :add 7555 :RequestTimeoutException INFO [2022-01-10 08:39:52,352] jepsen worker 206 - jepsen.util 1206 :ok :add 8887 INFO [2022-01-10 08:39:52,353] jepsen worker 248 - jepsen.util 248 :info :add 7556 :RequestTimeoutException INFO [2022-01-10 08:39:52,355] jepsen worker 849 - jepsen.util 2849 :invoke :add 8891 INFO [2022-01-10 08:39:52,357] jepsen worker 206 - jepsen.util 1206 :invoke :add 8892 INFO [2022-01-10 08:39:52,359] jepsen worker 248 - jepsen.util 1248 :invoke :add 8893 INFO [2022-01-10 08:39:52,361] jepsen worker 364 - jepsen.util 1364 :info :add 7559 :RequestTimeoutException INFO [2022-01-10 08:39:52,361] jepsen worker 596 - jepsen.util 1596 :invoke :add 8894 INFO [2022-01-10 08:39:52,363] jepsen worker 307 - jepsen.util 1307 :info :add 7560 :RequestTimeoutException INFO [2022-01-10 08:39:52,363] jepsen worker 364 - jepsen.util 2364 :invoke :add 8895 INFO [2022-01-10 08:39:52,365] jepsen worker 307 - jepsen.util 2307 :invoke :add 8896 INFO [2022-01-10 08:39:52,372] jepsen worker 198 - jepsen.util 1198 :info :add 7561 :RequestTimeoutException INFO [2022-01-10 08:39:52,376] jepsen worker 198 - jepsen.util 2198 :invoke :add 8897 INFO [2022-01-10 08:39:52,377] jepsen worker 334 - jepsen.util 1334 :info :add 7562 :RequestTimeoutException INFO [2022-01-10 08:39:52,380] jepsen worker 816 - jepsen.util 1816 :info :add 7563 :RequestTimeoutException INFO [2022-01-10 08:39:52,381] jepsen worker 334 - jepsen.util 2334 :invoke :add 8898 INFO [2022-01-10 08:39:52,384] jepsen worker 816 - jepsen.util 2816 :invoke :add 8899 INFO [2022-01-10 08:39:52,471] jepsen worker 501 - jepsen.util 1501 :info :add 7565 :RequestTimeoutException INFO [2022-01-10 08:39:52,472] jepsen worker 811 - jepsen.util 1811 :info :add 7567 :RequestTimeoutException INFO [2022-01-10 08:39:52,473] jepsen worker 197 - jepsen.util 1197 :info :add 7564 :RequestTimeoutException INFO [2022-01-10 08:39:52,473] jepsen worker 230 - jepsen.util 1230 :info :add 7566 :RequestTimeoutException INFO [2022-01-10 08:39:52,477] jepsen worker 197 - jepsen.util 2197 :invoke :add 8900 INFO [2022-01-10 08:39:52,479] jepsen worker 501 - jepsen.util 2501 :invoke :add 8901 INFO [2022-01-10 08:39:52,481] jepsen worker 811 - jepsen.util 2811 :invoke :add 8902 INFO [2022-01-10 08:39:52,483] jepsen worker 230 - jepsen.util 2230 :invoke :add 8903 INFO [2022-01-10 08:39:52,507] jepsen worker 49 - jepsen.util 1049 :info :add 7574 :RequestTimeoutException INFO [2022-01-10 08:39:52,510] jepsen worker 49 - jepsen.util 2049 :invoke :add 8904 INFO [2022-01-10 08:39:52,511] jepsen worker 687 - jepsen.util 687 :info :add 7576 :RequestTimeoutException INFO [2022-01-10 08:39:52,511] jepsen worker 286 - jepsen.util 1286 :info :add 7577 :RequestTimeoutException INFO [2022-01-10 08:39:52,512] jepsen worker 681 - jepsen.util 681 :info :add 7575 :RequestTimeoutException INFO [2022-01-10 08:39:52,513] jepsen worker 841 - jepsen.util 841 :info :add 7579 :RequestTimeoutException INFO [2022-01-10 08:39:52,513] jepsen worker 842 - jepsen.util 842 :info :add 7578 :RequestTimeoutException INFO [2022-01-10 08:39:52,516] jepsen worker 870 - jepsen.util 870 :info :add 7580 :RequestTimeoutException INFO [2022-01-10 08:39:52,517] jepsen worker 286 - jepsen.util 2286 :invoke :add 8905 INFO [2022-01-10 08:39:52,519] jepsen worker 687 - jepsen.util 1687 :invoke :add 8906 INFO [2022-01-10 08:39:52,521] jepsen worker 870 - jepsen.util 1870 :invoke :add 8907 INFO [2022-01-10 08:39:52,523] jepsen worker 681 - jepsen.util 1681 :invoke :add 8908 INFO [2022-01-10 08:39:52,525] jepsen worker 841 - jepsen.util 1841 :invoke :add 8909 INFO [2022-01-10 08:39:52,527] jepsen worker 842 - jepsen.util 1842 :invoke :add 8910 INFO [2022-01-10 08:39:52,593] jepsen worker 136 - jepsen.util 1136 :info :add 7581 :RequestTimeoutException INFO [2022-01-10 08:39:52,596] jepsen worker 136 - jepsen.util 2136 :invoke :add 8911 INFO [2022-01-10 08:39:52,621] jepsen worker 265 - jepsen.util 1265 :info :add 7582 :RequestTimeoutException INFO [2022-01-10 08:39:52,622] jepsen worker 333 - jepsen.util 1333 :info :add 7584 :RequestTimeoutException INFO [2022-01-10 08:39:52,623] jepsen worker 265 - jepsen.util 2265 :invoke :add 8912 INFO [2022-01-10 08:39:52,624] jepsen worker 404 - jepsen.util 1404 :info :add 7583 :RequestTimeoutException INFO [2022-01-10 08:39:52,626] jepsen worker 333 - jepsen.util 2333 :invoke :add 8913 INFO [2022-01-10 08:39:52,628] jepsen worker 404 - jepsen.util 2404 :invoke :add 8914 INFO [2022-01-10 08:39:52,634] jepsen worker 404 - jepsen.util 2404 :ok :add 8914 INFO [2022-01-10 08:39:52,637] jepsen worker 404 - jepsen.util 2404 :invoke :add 8915 INFO [2022-01-10 08:39:52,641] jepsen worker 404 - jepsen.util 2404 :ok :add 8915 INFO [2022-01-10 08:39:52,643] jepsen worker 404 - jepsen.util 2404 :invoke :add 8916 INFO [2022-01-10 08:39:52,648] jepsen worker 404 - jepsen.util 2404 :ok :add 8916 INFO [2022-01-10 08:39:52,650] jepsen worker 404 - jepsen.util 2404 :invoke :add 8917 INFO [2022-01-10 08:39:52,657] jepsen worker 404 - jepsen.util 2404 :ok :add 8917 INFO [2022-01-10 08:39:52,659] jepsen worker 404 - jepsen.util 2404 :invoke :add 8918 INFO [2022-01-10 08:39:52,674] jepsen worker 119 - jepsen.util 119 :ok :add 7630 INFO [2022-01-10 08:39:52,676] jepsen worker 119 - jepsen.util 119 :invoke :add 8919 INFO [2022-01-10 08:39:52,727] jepsen worker 688 - jepsen.util 1688 :info :add 7585 :RequestTimeoutException INFO [2022-01-10 08:39:52,731] jepsen worker 688 - jepsen.util 2688 :invoke :add 8920 INFO [2022-01-10 08:39:52,736] jepsen worker 688 - jepsen.util 2688 :ok :add 8920 INFO [2022-01-10 08:39:52,739] jepsen worker 688 - jepsen.util 2688 :invoke :add 8921 INFO [2022-01-10 08:39:52,745] jepsen worker 688 - jepsen.util 2688 :ok :add 8921 INFO [2022-01-10 08:39:52,748] jepsen worker 688 - jepsen.util 2688 :invoke :add 8922 INFO [2022-01-10 08:39:52,754] jepsen worker 688 - jepsen.util 2688 :ok :add 8922 INFO [2022-01-10 08:39:52,757] jepsen worker 688 - jepsen.util 2688 :invoke :add 8923 INFO [2022-01-10 08:39:52,760] jepsen worker 924 - jepsen.util 1924 :info :add 7586 :RequestTimeoutException INFO [2022-01-10 08:39:52,763] jepsen worker 688 - jepsen.util 2688 :ok :add 8923 INFO [2022-01-10 08:39:52,766] jepsen worker 924 - jepsen.util 2924 :invoke :add 8924 INFO [2022-01-10 08:39:52,768] jepsen worker 108 - jepsen.util 2108 :ok :add 8323 INFO [2022-01-10 08:39:52,768] jepsen worker 338 - jepsen.util 338 :ok :add 7993 INFO [2022-01-10 08:39:52,768] jepsen worker 688 - jepsen.util 2688 :invoke :add 8925 INFO [2022-01-10 08:39:52,772] jepsen worker 338 - jepsen.util 338 :invoke :add 8926 INFO [2022-01-10 08:39:52,774] jepsen worker 108 - jepsen.util 2108 :invoke :add 8927 INFO [2022-01-10 08:39:52,783] jepsen worker 867 - jepsen.util 1867 :info :add 7587 :RequestTimeoutException INFO [2022-01-10 08:39:52,786] jepsen worker 867 - jepsen.util 2867 :invoke :add 8928 INFO [2022-01-10 08:39:52,961] jepsen worker 366 - jepsen.util 1366 :info :add 7594 :RequestTimeoutException INFO [2022-01-10 08:39:52,964] jepsen worker 747 - jepsen.util 1747 :info :add 7595 :RequestTimeoutException INFO [2022-01-10 08:39:52,964] jepsen worker 366 - jepsen.util 2366 :invoke :add 8929 INFO [2022-01-10 08:39:52,968] jepsen worker 747 - jepsen.util 2747 :invoke :add 8930 INFO [2022-01-10 08:39:52,992] jepsen worker 725 - jepsen.util 1725 :info :add 7596 :RequestTimeoutException INFO [2022-01-10 08:39:52,995] jepsen worker 725 - jepsen.util 2725 :invoke :add 8931 INFO [2022-01-10 08:39:52,997] jepsen worker 4 - jepsen.util 1004 :info :add 7597 :RequestTimeoutException INFO [2022-01-10 08:39:52,999] jepsen worker 4 - jepsen.util 2004 :invoke :add 8932 INFO [2022-01-10 08:39:53,000] jepsen worker 484 - jepsen.util 1484 :info :add 7598 :RequestTimeoutException INFO [2022-01-10 08:39:53,003] jepsen worker 863 - jepsen.util 1863 :info :add 7599 :RequestTimeoutException INFO [2022-01-10 08:39:53,003] jepsen worker 484 - jepsen.util 2484 :invoke :add 8933 INFO [2022-01-10 08:39:53,005] jepsen worker 863 - jepsen.util 2863 :invoke :add 8934 INFO [2022-01-10 08:39:53,051] jepsen worker 347 - jepsen.util 1347 :info :add 7604 :RequestTimeoutException INFO [2022-01-10 08:39:53,054] jepsen worker 347 - jepsen.util 2347 :invoke :add 8935 INFO [2022-01-10 08:39:53,061] jepsen worker 846 - jepsen.util 1846 :info :add 7606 :RequestTimeoutException INFO [2022-01-10 08:39:53,064] jepsen worker 846 - jepsen.util 2846 :invoke :add 8936 INFO [2022-01-10 08:39:53,094] jepsen worker 614 - jepsen.util 1614 :info :add 7607 :RequestTimeoutException INFO [2022-01-10 08:39:53,097] jepsen worker 514 - jepsen.util 1514 :info :add 7609 :RequestTimeoutException INFO [2022-01-10 08:39:53,097] jepsen worker 614 - jepsen.util 2614 :invoke :add 8937 INFO [2022-01-10 08:39:53,099] jepsen worker 514 - jepsen.util 2514 :invoke :add 8938 INFO [2022-01-10 08:39:53,100] jepsen worker 196 - jepsen.util 1196 :info :add 7610 :RequestTimeoutException INFO [2022-01-10 08:39:53,103] jepsen worker 301 - jepsen.util 1301 :info :add 7608 :RequestTimeoutException INFO [2022-01-10 08:39:53,104] jepsen worker 196 - jepsen.util 2196 :invoke :add 8939 INFO [2022-01-10 08:39:53,106] jepsen worker 514 - jepsen.util 2514 :ok :add 8938 INFO [2022-01-10 08:39:53,107] jepsen worker 301 - jepsen.util 2301 :invoke :add 8940 WARN [2022-01-10 08:39:53,109] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","encode_duration_us":363,"operation_id":"0x202e","timeout_ms":10000,"total_duration_us":10093305},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000001/00000000CB02B145","last_local_socket":"172.23.104.79:41914","encode_duration_us":271,"operation_id":"0x2030","timeout_ms":10000,"total_duration_us":10081294},{"operation_name":"insert","last_dispatch_duration_us":9753402,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000001/00000000CB02B145","last_local_socket":"172.23.104.79:41914","total_dispatch_duration_us":9753402,"total_server_duration_us":1038170,"encode_duration_us":264,"operation_id":"0x2032","timeout_ms":10000,"last_server_duration_us":1038170,"total_duration_us":10078303},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","encode_duration_us":268,"operation_id":"0x2039","timeout_ms":10000,"total_duration_us":10066110},{"operation_name":"insert","last_dispatch_duration_us":10532879,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":10532879,"total_server_duration_us":4785326,"encode_duration_us":267,"operation_id":"0x20ed","timeout_ms":10000,"last_server_duration_us":4785326,"total_duration_us":10020272},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","encode_duration_us":269,"operation_id":"0x1f70","timeout_ms":10000,"total_duration_us":10010644},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","encode_duration_us":261,"operation_id":"0x2128","timeout_ms":10000,"total_duration_us":10010606},{"operation_name":"insert","last_dispatch_duration_us":5807162,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":5807162,"total_server_duration_us":3565894,"encode_duration_us":264,"operation_id":"0x1ee4","timeout_ms":10000,"last_server_duration_us":3565894,"total_duration_us":10010433},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","encode_duration_us":349,"operation_id":"0x1fc5","timeout_ms":10000,"total_duration_us":10010344},{"operation_name":"insert","last_dispatch_duration_us":10134678,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000001/00000000CB02B145","last_local_socket":"172.23.104.79:41914","total_dispatch_duration_us":10134678,"total_server_duration_us":5116429,"encode_duration_us":378,"operation_id":"0x1e0d","timeout_ms":10000,"last_server_duration_us":5116429,"total_duration_us":10010266}],"total_count":183}} WARN [2022-01-10 08:39:53,109] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":20306744,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20306744,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":9009789,"encode_duration_us":266,"operation_id":"0x1c46","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10009491},{"operation_name":"insert","last_dispatch_duration_us":10134678,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10134678,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":5116429,"encode_duration_us":378,"operation_id":"0x1e0d","timeout_ms":10000,"last_server_duration_us":5116429,"total_duration_us":10010266},{"operation_name":"insert","last_dispatch_duration_us":15414964,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15414964,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":7594986,"encode_duration_us":263,"operation_id":"0x1c9b","timeout_ms":10000,"last_server_duration_us":7594986,"total_duration_us":10009698},{"operation_name":"insert","last_dispatch_duration_us":19168425,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19168425,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":7804179,"encode_duration_us":316,"operation_id":"0x1c1d","timeout_ms":10000,"last_server_duration_us":7804179,"total_duration_us":10010830},{"operation_name":"insert","last_dispatch_duration_us":20238625,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20238625,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":9014029,"encode_duration_us":258,"operation_id":"0x1983","timeout_ms":10000,"last_server_duration_us":9014029,"total_duration_us":10010429},{"operation_name":"insert","last_dispatch_duration_us":16987052,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16987052,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":5913480,"encode_duration_us":494,"operation_id":"0x191a","timeout_ms":10000,"last_server_duration_us":5913480,"total_duration_us":10009754},{"operation_name":"insert","last_dispatch_duration_us":14278375,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14278375,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":3101513,"encode_duration_us":269,"operation_id":"0x1b32","timeout_ms":10000,"last_server_duration_us":3101513,"total_duration_us":10011111},{"operation_name":"insert","last_dispatch_duration_us":9753402,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9753402,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":1038170,"encode_duration_us":264,"operation_id":"0x2032","timeout_ms":10000,"last_server_duration_us":1038170,"total_duration_us":10078303},{"operation_name":"insert","last_dispatch_duration_us":10532879,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10532879,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":4785326,"encode_duration_us":267,"operation_id":"0x20ed","timeout_ms":10000,"last_server_duration_us":4785326,"total_duration_us":10020272},{"operation_name":"insert","last_dispatch_duration_us":5807162,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":5807162,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":3565894,"encode_duration_us":264,"operation_id":"0x1ee4","timeout_ms":10000,"last_server_duration_us":3565894,"total_duration_us":10010433}],"total_count":69}} INFO [2022-01-10 08:39:53,110] jepsen worker 514 - jepsen.util 2514 :invoke :add 8941 INFO [2022-01-10 08:39:53,110] jepsen worker 196 - jepsen.util 2196 :ok :add 8939 INFO [2022-01-10 08:39:53,113] jepsen worker 174 - jepsen.util 1174 :info :add 7611 :RequestTimeoutException INFO [2022-01-10 08:39:53,114] jepsen worker 196 - jepsen.util 2196 :invoke :add 8942 INFO [2022-01-10 08:39:53,117] jepsen worker 174 - jepsen.util 2174 :invoke :add 8943 INFO [2022-01-10 08:39:53,121] jepsen worker 254 - jepsen.util 1254 :info :add 7612 :RequestTimeoutException INFO [2022-01-10 08:39:53,123] jepsen worker 814 - jepsen.util 1814 :info :add 7613 :RequestTimeoutException INFO [2022-01-10 08:39:53,124] jepsen worker 254 - jepsen.util 2254 :invoke :add 8944 INFO [2022-01-10 08:39:53,126] jepsen worker 814 - jepsen.util 2814 :invoke :add 8945 INFO [2022-01-10 08:39:53,132] jepsen worker 618 - jepsen.util 1618 :info :add 7614 :RequestTimeoutException INFO [2022-01-10 08:39:53,135] jepsen worker 618 - jepsen.util 2618 :invoke :add 8946 INFO [2022-01-10 08:39:53,137] jepsen worker 336 - jepsen.util 1336 :info :add 7615 :RequestTimeoutException INFO [2022-01-10 08:39:53,141] jepsen worker 336 - jepsen.util 2336 :invoke :add 8947 INFO [2022-01-10 08:39:53,161] jepsen worker 472 - jepsen.util 1472 :info :add 7616 :RequestTimeoutException INFO [2022-01-10 08:39:53,163] jepsen worker 472 - jepsen.util 2472 :invoke :add 8948 INFO [2022-01-10 08:39:53,164] jepsen worker 407 - jepsen.util 1407 :ok :add 7990 INFO [2022-01-10 08:39:53,166] jepsen worker 748 - jepsen.util 1748 :ok :add 7725 INFO [2022-01-10 08:39:53,167] jepsen worker 407 - jepsen.util 1407 :invoke :add 8949 INFO [2022-01-10 08:39:53,169] jepsen worker 748 - jepsen.util 1748 :invoke :add 8950 INFO [2022-01-10 08:39:53,173] jepsen worker 391 - jepsen.util 1391 :info :add 7617 :RequestTimeoutException INFO [2022-01-10 08:39:53,176] jepsen worker 472 - jepsen.util 2472 :ok :add 8948 INFO [2022-01-10 08:39:53,176] jepsen worker 748 - jepsen.util 1748 :ok :add 8950 INFO [2022-01-10 08:39:53,176] jepsen worker 391 - jepsen.util 2391 :invoke :add 8951 INFO [2022-01-10 08:39:53,177] jepsen worker 780 - jepsen.util 1780 :info :add 7621 :RequestTimeoutException INFO [2022-01-10 08:39:53,179] jepsen worker 748 - jepsen.util 1748 :invoke :add 8952 INFO [2022-01-10 08:39:53,180] jepsen worker 795 - jepsen.util 1795 :info :add 7622 :RequestTimeoutException INFO [2022-01-10 08:39:53,182] jepsen worker 780 - jepsen.util 2780 :invoke :add 8953 INFO [2022-01-10 08:39:53,182] jepsen worker 971 - jepsen.util 1971 :info :add 7620 :RequestTimeoutException INFO [2022-01-10 08:39:53,183] jepsen worker 407 - jepsen.util 1407 :ok :add 8949 INFO [2022-01-10 08:39:53,186] jepsen worker 472 - jepsen.util 2472 :invoke :add 8954 INFO [2022-01-10 08:39:53,187] jepsen worker 391 - jepsen.util 2391 :ok :add 8951 INFO [2022-01-10 08:39:53,189] jepsen worker 795 - jepsen.util 2795 :invoke :add 8955 INFO [2022-01-10 08:39:53,190] jepsen worker 971 - jepsen.util 2971 :invoke :add 8956 INFO [2022-01-10 08:39:53,193] jepsen worker 391 - jepsen.util 2391 :invoke :add 8957 INFO [2022-01-10 08:39:53,195] jepsen worker 407 - jepsen.util 1407 :invoke :add 8958 INFO [2022-01-10 08:39:53,266] jepsen worker 799 - jepsen.util 1799 :ok :add 7944 INFO [2022-01-10 08:39:53,266] jepsen worker 788 - jepsen.util 1788 :ok :add 8115 INFO [2022-01-10 08:39:53,271] jepsen worker 788 - jepsen.util 1788 :invoke :add 8959 INFO [2022-01-10 08:39:53,273] jepsen worker 799 - jepsen.util 1799 :invoke :add 8960 INFO [2022-01-10 08:39:53,313] jepsen worker 440 - jepsen.util 1440 :info :add 7623 :RequestTimeoutException INFO [2022-01-10 08:39:53,316] jepsen worker 440 - jepsen.util 2440 :invoke :add 8961 INFO [2022-01-10 08:39:53,321] jepsen worker 889 - jepsen.util 1889 :info :add 7624 :RequestTimeoutException INFO [2022-01-10 08:39:53,324] jepsen worker 889 - jepsen.util 2889 :invoke :add 8962 INFO [2022-01-10 08:39:53,325] jepsen worker 780 - jepsen.util 2780 :ok :add 8953 INFO [2022-01-10 08:39:53,325] jepsen worker 748 - jepsen.util 1748 :ok :add 8952 INFO [2022-01-10 08:39:53,328] jepsen worker 780 - jepsen.util 2780 :invoke :add 8963 INFO [2022-01-10 08:39:53,330] jepsen worker 748 - jepsen.util 1748 :invoke :add 8964 INFO [2022-01-10 08:39:53,335] jepsen worker 748 - jepsen.util 1748 :ok :add 8964 INFO [2022-01-10 08:39:53,338] jepsen worker 748 - jepsen.util 1748 :invoke :add 8965 INFO [2022-01-10 08:39:53,343] jepsen worker 748 - jepsen.util 1748 :ok :add 8965 INFO [2022-01-10 08:39:53,345] jepsen worker 748 - jepsen.util 1748 :invoke :add 8966 INFO [2022-01-10 08:39:53,351] jepsen worker 748 - jepsen.util 1748 :ok :add 8966 INFO [2022-01-10 08:39:53,352] jepsen worker 218 - jepsen.util 1218 :info :add 7626 :RequestTimeoutException INFO [2022-01-10 08:39:53,353] jepsen worker 168 - jepsen.util 1168 :info :add 7625 :RequestTimeoutException INFO [2022-01-10 08:39:53,354] jepsen worker 218 - jepsen.util 2218 :invoke :add 8967 INFO [2022-01-10 08:39:53,357] jepsen worker 613 - jepsen.util 1613 :info :add 7627 :RequestTimeoutException INFO [2022-01-10 08:39:53,357] jepsen worker 748 - jepsen.util 1748 :invoke :add 8968 INFO [2022-01-10 08:39:53,359] jepsen worker 168 - jepsen.util 2168 :invoke :add 8969 INFO [2022-01-10 08:39:53,360] jepsen worker 218 - jepsen.util 2218 :ok :add 8967 INFO [2022-01-10 08:39:53,360] jepsen worker 469 - jepsen.util 1469 :info :add 7628 :RequestTimeoutException INFO [2022-01-10 08:39:53,361] jepsen worker 613 - jepsen.util 2613 :invoke :add 8970 INFO [2022-01-10 08:39:53,364] jepsen worker 218 - jepsen.util 2218 :invoke :add 8971 INFO [2022-01-10 08:39:53,365] jepsen worker 469 - jepsen.util 2469 :invoke :add 8972 INFO [2022-01-10 08:39:53,393] jepsen worker 199 - jepsen.util 199 :info :add 7632 :RequestTimeoutException INFO [2022-01-10 08:39:53,393] jepsen worker 18 - jepsen.util 18 :info :add 7633 :RequestTimeoutException INFO [2022-01-10 08:39:53,397] jepsen worker 199 - jepsen.util 1199 :invoke :add 8973 INFO [2022-01-10 08:39:53,399] jepsen worker 18 - jepsen.util 1018 :invoke :add 8974 INFO [2022-01-10 08:39:53,491] jepsen worker 67 - jepsen.util 67 :info :add 7638 :RequestTimeoutException INFO [2022-01-10 08:39:53,494] jepsen worker 67 - jepsen.util 1067 :invoke :add 8975 INFO [2022-01-10 08:39:53,523] jepsen worker 293 - jepsen.util 293 :info :add 7639 :RequestTimeoutException INFO [2022-01-10 08:39:53,526] jepsen worker 293 - jepsen.util 1293 :invoke :add 8976 INFO [2022-01-10 08:39:53,531] jepsen worker 640 - jepsen.util 640 :info :add 7640 :RequestTimeoutException INFO [2022-01-10 08:39:53,534] jepsen worker 640 - jepsen.util 1640 :invoke :add 8977 INFO [2022-01-10 08:39:53,571] jepsen worker 722 - jepsen.util 722 :info :add 7641 :RequestTimeoutException INFO [2022-01-10 08:39:53,574] jepsen worker 722 - jepsen.util 1722 :invoke :add 8978 INFO [2022-01-10 08:39:53,592] jepsen worker 667 - jepsen.util 1667 :info :add 7643 :RequestTimeoutException INFO [2022-01-10 08:39:53,594] jepsen worker 939 - jepsen.util 1939 :info :add 7642 :RequestTimeoutException INFO [2022-01-10 08:39:53,596] jepsen worker 667 - jepsen.util 2667 :invoke :add 8979 INFO [2022-01-10 08:39:53,598] jepsen worker 939 - jepsen.util 2939 :invoke :add 8980 INFO [2022-01-10 08:39:53,603] jepsen worker 822 - jepsen.util 1822 :info :add 7648 :RequestTimeoutException INFO [2022-01-10 08:39:53,607] jepsen worker 822 - jepsen.util 2822 :invoke :add 8981 INFO [2022-01-10 08:39:53,611] jepsen worker 408 - jepsen.util 1408 :info :add 7651 :RequestTimeoutException INFO [2022-01-10 08:39:53,612] jepsen worker 14 - jepsen.util 1014 :info :add 7649 :RequestTimeoutException INFO [2022-01-10 08:39:53,613] jepsen worker 729 - jepsen.util 1729 :info :add 7652 :RequestTimeoutException INFO [2022-01-10 08:39:53,617] jepsen worker 879 - jepsen.util 1879 :info :add 7650 :RequestTimeoutException INFO [2022-01-10 08:39:53,618] jepsen worker 14 - jepsen.util 2014 :invoke :add 8982 INFO [2022-01-10 08:39:53,621] jepsen worker 408 - jepsen.util 2408 :invoke :add 8983 INFO [2022-01-10 08:39:53,623] jepsen worker 729 - jepsen.util 2729 :invoke :add 8984 INFO [2022-01-10 08:39:53,625] jepsen worker 879 - jepsen.util 2879 :invoke :add 8985 INFO [2022-01-10 08:39:53,630] jepsen worker 729 - jepsen.util 2729 :ok :add 8984 INFO [2022-01-10 08:39:53,632] jepsen worker 729 - jepsen.util 2729 :invoke :add 8986 INFO [2022-01-10 08:39:53,633] jepsen worker 879 - jepsen.util 2879 :ok :add 8985 INFO [2022-01-10 08:39:53,636] jepsen worker 879 - jepsen.util 2879 :invoke :add 8987 INFO [2022-01-10 08:39:53,637] jepsen worker 729 - jepsen.util 2729 :ok :add 8986 INFO [2022-01-10 08:39:53,640] jepsen worker 729 - jepsen.util 2729 :invoke :add 8988 INFO [2022-01-10 08:39:53,642] jepsen worker 526 - jepsen.util 526 :info :add 7653 :RequestTimeoutException INFO [2022-01-10 08:39:53,642] jepsen worker 879 - jepsen.util 2879 :ok :add 8987 INFO [2022-01-10 08:39:53,645] jepsen worker 879 - jepsen.util 2879 :invoke :add 8989 INFO [2022-01-10 08:39:53,647] jepsen worker 526 - jepsen.util 1526 :invoke :add 8990 INFO [2022-01-10 08:39:53,711] jepsen worker 491 - jepsen.util 1491 :info :add 7654 :RequestTimeoutException INFO [2022-01-10 08:39:53,714] jepsen worker 491 - jepsen.util 2491 :invoke :add 8991 INFO [2022-01-10 08:39:53,723] jepsen worker 511 - jepsen.util 1511 :info :add 7655 :RequestTimeoutException INFO [2022-01-10 08:39:53,728] jepsen worker 511 - jepsen.util 2511 :invoke :add 8992 INFO [2022-01-10 08:39:53,731] jepsen worker 182 - jepsen.util 1182 :ok :add 8392 INFO [2022-01-10 08:39:53,731] jepsen worker 500 - jepsen.util 2500 :ok :add 8236 INFO [2022-01-10 08:39:53,732] jepsen worker 834 - jepsen.util 1834 :info :add 7656 :RequestTimeoutException INFO [2022-01-10 08:39:53,735] jepsen worker 182 - jepsen.util 1182 :invoke :add 8993 INFO [2022-01-10 08:39:53,737] jepsen worker 329 - jepsen.util 1329 :info :add 7657 :RequestTimeoutException INFO [2022-01-10 08:39:53,738] jepsen worker 500 - jepsen.util 2500 :invoke :add 8994 INFO [2022-01-10 08:39:53,740] jepsen worker 329 - jepsen.util 2329 :invoke :add 8995 INFO [2022-01-10 08:39:53,743] jepsen worker 834 - jepsen.util 2834 :invoke :add 8996 INFO [2022-01-10 08:39:53,883] jepsen worker 441 - jepsen.util 1441 :info :add 7658 :RequestTimeoutException INFO [2022-01-10 08:39:53,886] jepsen worker 441 - jepsen.util 2441 :invoke :add 8997 INFO [2022-01-10 08:39:53,891] jepsen worker 919 - jepsen.util 919 :info :add 7660 :RequestTimeoutException INFO [2022-01-10 08:39:53,891] jepsen worker 936 - jepsen.util 936 :info :add 7661 :RequestTimeoutException INFO [2022-01-10 08:39:53,892] jepsen worker 832 - jepsen.util 1832 :info :add 7662 :RequestTimeoutException INFO [2022-01-10 08:39:53,894] jepsen worker 324 - jepsen.util 324 :info :add 7659 :RequestTimeoutException INFO [2022-01-10 08:39:53,898] jepsen worker 324 - jepsen.util 1324 :invoke :add 8998 INFO [2022-01-10 08:39:53,901] jepsen worker 919 - jepsen.util 1919 :invoke :add 8999 INFO [2022-01-10 08:39:53,902] jepsen worker 832 - jepsen.util 2832 :invoke :add 9000 INFO [2022-01-10 08:39:53,903] jepsen worker 476 - jepsen.util 476 :info :add 7664 :RequestTimeoutException INFO [2022-01-10 08:39:53,905] jepsen worker 936 - jepsen.util 1936 :invoke :add 9001 INFO [2022-01-10 08:39:53,907] jepsen worker 476 - jepsen.util 1476 :invoke :add 9002 INFO [2022-01-10 08:39:53,911] jepsen worker 259 - jepsen.util 259 :info :add 7663 :RequestTimeoutException INFO [2022-01-10 08:39:53,914] jepsen worker 259 - jepsen.util 1259 :invoke :add 9003 INFO [2022-01-10 08:39:53,942] jepsen worker 734 - jepsen.util 734 :info :add 7665 :RequestTimeoutException INFO [2022-01-10 08:39:53,945] jepsen worker 734 - jepsen.util 1734 :invoke :add 9004 INFO [2022-01-10 08:39:53,950] jepsen worker 873 - jepsen.util 873 :info :add 7666 :RequestTimeoutException INFO [2022-01-10 08:39:53,952] jepsen worker 918 - jepsen.util 1918 :info :add 7667 :RequestTimeoutException INFO [2022-01-10 08:39:53,953] jepsen worker 873 - jepsen.util 1873 :invoke :add 9005 INFO [2022-01-10 08:39:53,956] jepsen worker 918 - jepsen.util 2918 :invoke :add 9006 INFO [2022-01-10 08:39:53,982] jepsen worker 986 - jepsen.util 1986 :info :add 7674 :RequestTimeoutException INFO [2022-01-10 08:39:53,985] jepsen worker 986 - jepsen.util 2986 :invoke :add 9007 INFO [2022-01-10 08:39:53,997] jepsen worker 227 - jepsen.util 1227 :info :add 7675 :RequestTimeoutException INFO [2022-01-10 08:39:53,999] jepsen worker 227 - jepsen.util 2227 :invoke :add 9008 INFO [2022-01-10 08:39:54,006] jepsen worker 227 - jepsen.util 2227 :ok :add 9008 INFO [2022-01-10 08:39:54,009] jepsen worker 227 - jepsen.util 2227 :invoke :add 9009 INFO [2022-01-10 08:39:54,014] jepsen worker 227 - jepsen.util 2227 :ok :add 9009 INFO [2022-01-10 08:39:54,018] jepsen worker 227 - jepsen.util 2227 :invoke :add 9010 INFO [2022-01-10 08:39:54,023] jepsen worker 227 - jepsen.util 2227 :ok :add 9010 INFO [2022-01-10 08:39:54,026] jepsen worker 227 - jepsen.util 2227 :invoke :add 9011 INFO [2022-01-10 08:39:54,032] jepsen worker 227 - jepsen.util 2227 :ok :add 9011 INFO [2022-01-10 08:39:54,034] jepsen worker 227 - jepsen.util 2227 :invoke :add 9012 INFO [2022-01-10 08:39:54,040] jepsen worker 227 - jepsen.util 2227 :ok :add 9012 INFO [2022-01-10 08:39:54,043] jepsen worker 227 - jepsen.util 2227 :invoke :add 9013 INFO [2022-01-10 08:39:54,049] jepsen worker 227 - jepsen.util 2227 :ok :add 9013 INFO [2022-01-10 08:39:54,052] jepsen worker 227 - jepsen.util 2227 :invoke :add 9014 INFO [2022-01-10 08:39:54,060] jepsen worker 217 - jepsen.util 1217 :info :add 7676 :RequestTimeoutException INFO [2022-01-10 08:39:54,061] jepsen worker 243 - jepsen.util 1243 :info :add 7678 :RequestTimeoutException INFO [2022-01-10 08:39:54,063] jepsen worker 925 - jepsen.util 1925 :info :add 7677 :RequestTimeoutException INFO [2022-01-10 08:39:54,064] jepsen worker 243 - jepsen.util 2243 :invoke :add 9015 INFO [2022-01-10 08:39:54,066] jepsen worker 217 - jepsen.util 2217 :invoke :add 9016 INFO [2022-01-10 08:39:54,069] jepsen worker 925 - jepsen.util 2925 :invoke :add 9017 INFO [2022-01-10 08:39:54,072] jepsen worker 539 - jepsen.util 1539 :info :add 7680 :RequestTimeoutException INFO [2022-01-10 08:39:54,073] jepsen worker 447 - jepsen.util 1447 :info :add 7679 :RequestTimeoutException INFO [2022-01-10 08:39:54,075] jepsen worker 539 - jepsen.util 2539 :invoke :add 9018 INFO [2022-01-10 08:39:54,077] jepsen worker 447 - jepsen.util 2447 :invoke :add 9019 INFO [2022-01-10 08:39:54,077] jepsen worker 902 - jepsen.util 1902 :info :add 7681 :RequestTimeoutException INFO [2022-01-10 08:39:54,079] jepsen worker 902 - jepsen.util 2902 :invoke :add 9020 INFO [2022-01-10 08:39:54,080] jepsen worker 103 - jepsen.util 1103 :info :add 7682 :RequestTimeoutException INFO [2022-01-10 08:39:54,082] jepsen worker 631 - jepsen.util 1631 :info :add 7683 :RequestTimeoutException INFO [2022-01-10 08:39:54,083] jepsen worker 103 - jepsen.util 2103 :invoke :add 9021 INFO [2022-01-10 08:39:54,085] jepsen worker 631 - jepsen.util 2631 :invoke :add 9022 INFO [2022-01-10 08:39:54,091] jepsen worker 170 - jepsen.util 1170 :info :add 7684 :RequestTimeoutException INFO [2022-01-10 08:39:54,093] jepsen worker 569 - jepsen.util 1569 :info :add 7685 :RequestTimeoutException INFO [2022-01-10 08:39:54,094] jepsen worker 170 - jepsen.util 2170 :invoke :add 9023 INFO [2022-01-10 08:39:54,097] jepsen worker 151 - jepsen.util 1151 :info :add 7687 :RequestTimeoutException INFO [2022-01-10 08:39:54,097] jepsen worker 569 - jepsen.util 2569 :invoke :add 9024 INFO [2022-01-10 08:39:54,099] jepsen worker 151 - jepsen.util 2151 :invoke :add 9025 INFO [2022-01-10 08:39:54,102] jepsen worker 110 - jepsen.util 1110 :info :add 7686 :RequestTimeoutException INFO [2022-01-10 08:39:54,103] jepsen worker 569 - jepsen.util 2569 :ok :add 9024 INFO [2022-01-10 08:39:54,104] jepsen worker 110 - jepsen.util 2110 :invoke :add 9026 INFO [2022-01-10 08:39:54,106] jepsen worker 151 - jepsen.util 2151 :ok :add 9025 INFO [2022-01-10 08:39:54,106] jepsen worker 569 - jepsen.util 2569 :invoke :add 9027 INFO [2022-01-10 08:39:54,109] jepsen worker 151 - jepsen.util 2151 :invoke :add 9028 INFO [2022-01-10 08:39:54,113] jepsen worker 110 - jepsen.util 2110 :ok :add 9026 INFO [2022-01-10 08:39:54,115] jepsen worker 110 - jepsen.util 2110 :invoke :add 9029 INFO [2022-01-10 08:39:54,116] jepsen worker 569 - jepsen.util 2569 :ok :add 9027 INFO [2022-01-10 08:39:54,119] jepsen worker 569 - jepsen.util 2569 :invoke :add 9030 INFO [2022-01-10 08:39:54,182] jepsen worker 696 - jepsen.util 696 :info :add 7694 :RequestTimeoutException INFO [2022-01-10 08:39:54,186] jepsen worker 696 - jepsen.util 1696 :invoke :add 9031 INFO [2022-01-10 08:39:54,191] jepsen worker 973 - jepsen.util 1973 :info :add 7695 :RequestTimeoutException INFO [2022-01-10 08:39:54,191] jepsen worker 284 - jepsen.util 284 :info :add 7696 :RequestTimeoutException INFO [2022-01-10 08:39:54,191] jepsen worker 159 - jepsen.util 1159 :info :add 7698 :RequestTimeoutException INFO [2022-01-10 08:39:54,193] jepsen worker 101 - jepsen.util 101 :info :add 7697 :RequestTimeoutException INFO [2022-01-10 08:39:54,195] jepsen worker 159 - jepsen.util 2159 :invoke :add 9032 INFO [2022-01-10 08:39:54,197] jepsen worker 973 - jepsen.util 2973 :invoke :add 9033 INFO [2022-01-10 08:39:54,199] jepsen worker 284 - jepsen.util 1284 :invoke :add 9034 INFO [2022-01-10 08:39:54,201] jepsen worker 101 - jepsen.util 1101 :invoke :add 9035 INFO [2022-01-10 08:39:54,205] jepsen worker 273 - jepsen.util 1273 :ok :add 7870 INFO [2022-01-10 08:39:54,205] jepsen worker 770 - jepsen.util 770 :ok :add 7761 INFO [2022-01-10 08:39:54,207] jepsen worker 770 - jepsen.util 770 :invoke :add 9036 INFO [2022-01-10 08:39:54,209] jepsen worker 273 - jepsen.util 1273 :invoke :add 9037 INFO [2022-01-10 08:39:54,211] jepsen worker 32 - jepsen.util 1032 :ok :add 7887 INFO [2022-01-10 08:39:54,213] jepsen worker 38 - jepsen.util 2038 :ok :add 8788 INFO [2022-01-10 08:39:54,214] jepsen worker 32 - jepsen.util 1032 :invoke :add 9038 INFO [2022-01-10 08:39:54,217] jepsen worker 38 - jepsen.util 2038 :invoke :add 9039 INFO [2022-01-10 08:39:54,293] jepsen worker 140 - jepsen.util 140 :info :add 7699 :RequestTimeoutException INFO [2022-01-10 08:39:54,296] jepsen worker 140 - jepsen.util 1140 :invoke :add 9040 INFO [2022-01-10 08:39:54,297] jepsen worker 479 - jepsen.util 479 :info :add 7704 :RequestTimeoutException INFO [2022-01-10 08:39:54,301] jepsen worker 479 - jepsen.util 1479 :invoke :add 9041 INFO [2022-01-10 08:39:54,301] jepsen worker 68 - jepsen.util 68 :info :add 7705 :RequestTimeoutException INFO [2022-01-10 08:39:54,302] jepsen worker 672 - jepsen.util 672 :info :add 7706 :RequestTimeoutException INFO [2022-01-10 08:39:54,306] jepsen worker 68 - jepsen.util 1068 :invoke :add 9042 INFO [2022-01-10 08:39:54,307] jepsen worker 672 - jepsen.util 1672 :invoke :add 9043 INFO [2022-01-10 08:39:54,322] jepsen worker 464 - jepsen.util 1464 :info :add 7707 :RequestTimeoutException INFO [2022-01-10 08:39:54,325] jepsen worker 464 - jepsen.util 2464 :invoke :add 9044 INFO [2022-01-10 08:39:54,327] jepsen worker 426 - jepsen.util 1426 :info :add 7708 :RequestTimeoutException INFO [2022-01-10 08:39:54,329] jepsen worker 426 - jepsen.util 2426 :invoke :add 9045 INFO [2022-01-10 08:39:54,331] jepsen worker 209 - jepsen.util 1209 :info :add 7709 :RequestTimeoutException INFO [2022-01-10 08:39:54,331] jepsen worker 932 - jepsen.util 1932 :info :add 7711 :RequestTimeoutException INFO [2022-01-10 08:39:54,332] jepsen worker 586 - jepsen.util 1586 :info :add 7710 :RequestTimeoutException INFO [2022-01-10 08:39:54,334] jepsen worker 209 - jepsen.util 2209 :invoke :add 9046 INFO [2022-01-10 08:39:54,336] jepsen worker 586 - jepsen.util 2586 :invoke :add 9047 INFO [2022-01-10 08:39:54,339] jepsen worker 932 - jepsen.util 2932 :invoke :add 9048 INFO [2022-01-10 08:39:54,350] jepsen worker 929 - jepsen.util 929 :info :add 7713 :RequestTimeoutException INFO [2022-01-10 08:39:54,351] jepsen worker 708 - jepsen.util 1708 :info :add 7712 :RequestTimeoutException INFO [2022-01-10 08:39:54,354] jepsen worker 929 - jepsen.util 1929 :invoke :add 9049 INFO [2022-01-10 08:39:54,355] jepsen worker 708 - jepsen.util 2708 :invoke :add 9050 INFO [2022-01-10 08:39:54,361] jepsen worker 662 - jepsen.util 662 :info :add 7714 :RequestTimeoutException INFO [2022-01-10 08:39:54,361] jepsen worker 148 - jepsen.util 148 :info :add 7715 :RequestTimeoutException INFO [2022-01-10 08:39:54,365] jepsen worker 148 - jepsen.util 1148 :invoke :add 9051 INFO [2022-01-10 08:39:54,366] jepsen worker 662 - jepsen.util 1662 :invoke :add 9052 INFO [2022-01-10 08:39:54,380] jepsen worker 142 - jepsen.util 142 :info :add 7717 :RequestTimeoutException INFO [2022-01-10 08:39:54,381] jepsen worker 946 - jepsen.util 946 :info :add 7716 :RequestTimeoutException INFO [2022-01-10 08:39:54,381] jepsen worker 395 - jepsen.util 1395 :info :add 7720 :RequestTimeoutException INFO [2022-01-10 08:39:54,385] jepsen worker 142 - jepsen.util 1142 :invoke :add 9053 INFO [2022-01-10 08:39:54,387] jepsen worker 946 - jepsen.util 1946 :invoke :add 9054 INFO [2022-01-10 08:39:54,389] jepsen worker 395 - jepsen.util 2395 :invoke :add 9055 INFO [2022-01-10 08:39:54,391] jepsen worker 195 - jepsen.util 1195 :info :add 7721 :RequestTimeoutException INFO [2022-01-10 08:39:54,392] jepsen worker 339 - jepsen.util 339 :info :add 7724 :RequestTimeoutException INFO [2022-01-10 08:39:54,392] jepsen worker 398 - jepsen.util 1398 :info :add 7722 :RequestTimeoutException INFO [2022-01-10 08:39:54,394] jepsen worker 188 - jepsen.util 1188 :info :add 7723 :RequestTimeoutException INFO [2022-01-10 08:39:54,396] jepsen worker 946 - jepsen.util 1946 :ok :add 9054 INFO [2022-01-10 08:39:54,397] jepsen worker 395 - jepsen.util 2395 :ok :add 9055 INFO [2022-01-10 08:39:54,399] jepsen worker 195 - jepsen.util 2195 :invoke :add 9056 INFO [2022-01-10 08:39:54,401] jepsen worker 191 - jepsen.util 1191 :info :add 7726 :RequestTimeoutException INFO [2022-01-10 08:39:54,401] jepsen worker 946 - jepsen.util 1946 :invoke :add 9057 INFO [2022-01-10 08:39:54,402] jepsen worker 78 - jepsen.util 78 :info :add 7728 :RequestTimeoutException INFO [2022-01-10 08:39:54,404] jepsen worker 191 - jepsen.util 2191 :invoke :add 9058 INFO [2022-01-10 08:39:54,407] jepsen worker 395 - jepsen.util 2395 :invoke :add 9059 INFO [2022-01-10 08:39:54,407] jepsen worker 920 - jepsen.util 1920 :info :add 7727 :RequestTimeoutException INFO [2022-01-10 08:39:54,408] jepsen worker 78 - jepsen.util 1078 :invoke :add 9060 INFO [2022-01-10 08:39:54,409] jepsen worker 195 - jepsen.util 2195 :ok :add 9056 INFO [2022-01-10 08:39:54,410] jepsen worker 348 - jepsen.util 1348 :info :add 7729 :RequestTimeoutException INFO [2022-01-10 08:39:54,410] jepsen worker 946 - jepsen.util 1946 :ok :add 9057 INFO [2022-01-10 08:39:54,410] jepsen worker 444 - jepsen.util 1444 :info :add 7731 :RequestTimeoutException INFO [2022-01-10 08:39:54,411] jepsen worker 398 - jepsen.util 2398 :invoke :add 9061 INFO [2022-01-10 08:39:54,412] jepsen worker 622 - jepsen.util 622 :info :add 7730 :RequestTimeoutException INFO [2022-01-10 08:39:54,413] jepsen worker 978 - jepsen.util 1978 :info :add 7732 :RequestTimeoutException INFO [2022-01-10 08:39:54,417] jepsen worker 978 - jepsen.util 2978 :invoke :add 9062 INFO [2022-01-10 08:39:54,418] jepsen worker 398 - jepsen.util 2398 :ok :add 9061 INFO [2022-01-10 08:39:54,419] jepsen worker 348 - jepsen.util 2348 :invoke :add 9063 INFO [2022-01-10 08:39:54,421] jepsen worker 444 - jepsen.util 2444 :invoke :add 9064 INFO [2022-01-10 08:39:54,421] jepsen worker 78 - jepsen.util 1078 :ok :add 9060 INFO [2022-01-10 08:39:54,421] jepsen worker 513 - jepsen.util 513 :info :add 7733 :RequestTimeoutException INFO [2022-01-10 08:39:54,425] jepsen worker 920 - jepsen.util 2920 :invoke :add 9066 INFO [2022-01-10 08:39:54,427] jepsen worker 946 - jepsen.util 1946 :invoke :add 9067 INFO [2022-01-10 08:39:54,429] jepsen worker 398 - jepsen.util 2398 :invoke :add 9068 INFO [2022-01-10 08:39:54,430] jepsen worker 978 - jepsen.util 2978 :ok :add 9062 INFO [2022-01-10 08:39:54,430] jepsen worker 348 - jepsen.util 2348 :ok :add 9063 INFO [2022-01-10 08:39:54,430] jepsen worker 622 - jepsen.util 1622 :invoke :add 9065 INFO [2022-01-10 08:39:54,430] jepsen worker 513 - jepsen.util 1513 :invoke :add 9069 INFO [2022-01-10 08:39:54,432] jepsen worker 25 - jepsen.util 1025 :info :add 7738 :RequestTimeoutException INFO [2022-01-10 08:39:54,433] jepsen worker 348 - jepsen.util 2348 :invoke :add 9070 INFO [2022-01-10 08:39:54,435] jepsen worker 188 - jepsen.util 2188 :invoke :add 9071 INFO [2022-01-10 08:39:54,437] jepsen worker 624 - jepsen.util 1624 :info :add 7739 :RequestTimeoutException INFO [2022-01-10 08:39:54,437] jepsen worker 25 - jepsen.util 2025 :invoke :add 9072 INFO [2022-01-10 08:39:54,440] jepsen worker 477 - jepsen.util 1477 :info :add 7741 :RequestTimeoutException INFO [2022-01-10 08:39:54,440] jepsen worker 339 - jepsen.util 1339 :invoke :add 9073 INFO [2022-01-10 08:39:54,442] jepsen worker 733 - jepsen.util 733 :info :add 7740 :RequestTimeoutException INFO [2022-01-10 08:39:54,442] jepsen worker 776 - jepsen.util 776 :info :add 7742 :RequestTimeoutException INFO [2022-01-10 08:39:54,443] jepsen worker 195 - jepsen.util 2195 :invoke :add 9074 INFO [2022-01-10 08:39:54,446] jepsen worker 978 - jepsen.util 2978 :invoke :add 9075 INFO [2022-01-10 08:39:54,448] jepsen worker 776 - jepsen.util 1776 :invoke :add 9076 INFO [2022-01-10 08:39:54,450] jepsen worker 733 - jepsen.util 1733 :invoke :add 9077 INFO [2022-01-10 08:39:54,451] jepsen worker 460 - jepsen.util 1460 :info :add 7743 :RequestTimeoutException INFO [2022-01-10 08:39:54,452] jepsen worker 78 - jepsen.util 1078 :invoke :add 9078 INFO [2022-01-10 08:39:54,454] jepsen worker 624 - jepsen.util 2624 :invoke :add 9079 INFO [2022-01-10 08:39:54,456] jepsen worker 460 - jepsen.util 2460 :invoke :add 9080 INFO [2022-01-10 08:39:54,458] jepsen worker 477 - jepsen.util 2477 :invoke :add 9081 INFO [2022-01-10 08:39:54,459] jepsen worker 78 - jepsen.util 1078 :ok :add 9078 INFO [2022-01-10 08:39:54,462] jepsen worker 624 - jepsen.util 2624 :ok :add 9079 INFO [2022-01-10 08:39:54,462] jepsen worker 78 - jepsen.util 1078 :invoke :add 9082 INFO [2022-01-10 08:39:54,462] jepsen worker 947 - jepsen.util 947 :info :add 7749 :RequestTimeoutException INFO [2022-01-10 08:39:54,462] jepsen worker 17 - jepsen.util 1017 :info :add 7750 :RequestTimeoutException INFO [2022-01-10 08:39:54,462] jepsen worker 588 - jepsen.util 588 :info :add 7751 :RequestTimeoutException INFO [2022-01-10 08:39:54,463] jepsen worker 572 - jepsen.util 1572 :info :add 7748 :RequestTimeoutException INFO [2022-01-10 08:39:54,467] jepsen worker 460 - jepsen.util 2460 :ok :add 9080 INFO [2022-01-10 08:39:54,468] jepsen worker 572 - jepsen.util 2572 :invoke :add 9083 INFO [2022-01-10 08:39:54,468] jepsen worker 477 - jepsen.util 2477 :ok :add 9081 INFO [2022-01-10 08:39:54,469] jepsen worker 460 - jepsen.util 2460 :invoke :add 9084 INFO [2022-01-10 08:39:54,471] jepsen worker 490 - jepsen.util 1490 :info :add 7752 :RequestTimeoutException INFO [2022-01-10 08:39:54,471] jepsen worker 896 - jepsen.util 1896 :info :add 7753 :RequestTimeoutException INFO [2022-01-10 08:39:54,472] jepsen worker 907 - jepsen.util 907 :info :add 7754 :RequestTimeoutException INFO [2022-01-10 08:39:54,472] jepsen worker 947 - jepsen.util 1947 :invoke :add 9085 INFO [2022-01-10 08:39:54,473] jepsen worker 386 - jepsen.util 386 :info :add 7755 :RequestTimeoutException INFO [2022-01-10 08:39:54,474] jepsen worker 78 - jepsen.util 1078 :ok :add 9082 INFO [2022-01-10 08:39:54,477] jepsen worker 225 - jepsen.util 225 :info :add 7758 :RequestTimeoutException INFO [2022-01-10 08:39:54,477] jepsen worker 490 - jepsen.util 2490 :invoke :add 9086 INFO [2022-01-10 08:39:54,480] jepsen worker 619 - jepsen.util 1619 :info :add 7759 :RequestTimeoutException INFO [2022-01-10 08:39:54,480] jepsen worker 78 - jepsen.util 1078 :invoke :add 9087 INFO [2022-01-10 08:39:54,480] jepsen worker 382 - jepsen.util 1382 :info :add 7756 :RequestTimeoutException INFO [2022-01-10 08:39:54,481] jepsen worker 208 - jepsen.util 208 :info :add 7757 :RequestTimeoutException INFO [2022-01-10 08:39:54,481] jepsen worker 853 - jepsen.util 853 :info :add 7760 :RequestTimeoutException INFO [2022-01-10 08:39:54,485] jepsen worker 907 - jepsen.util 1907 :invoke :add 9088 INFO [2022-01-10 08:39:54,487] jepsen worker 619 - jepsen.util 2619 :invoke :add 9089 INFO [2022-01-10 08:39:54,489] jepsen worker 853 - jepsen.util 1853 :invoke :add 9090 INFO [2022-01-10 08:39:54,490] jepsen worker 524 - jepsen.util 1524 :info :add 7762 :RequestTimeoutException INFO [2022-01-10 08:39:54,491] jepsen worker 200 - jepsen.util 1200 :info :add 7764 :RequestTimeoutException INFO [2022-01-10 08:39:54,491] jepsen worker 292 - jepsen.util 1292 :info :add 7765 :RequestTimeoutException INFO [2022-01-10 08:39:54,491] jepsen worker 208 - jepsen.util 1208 :invoke :add 9091 INFO [2022-01-10 08:39:54,492] jepsen worker 693 - jepsen.util 1693 :info :add 7763 :RequestTimeoutException INFO [2022-01-10 08:39:54,492] jepsen worker 588 - jepsen.util 1588 :invoke :add 9092 INFO [2022-01-10 08:39:54,497] jepsen worker 524 - jepsen.util 2524 :invoke :add 9093 INFO [2022-01-10 08:39:54,499] jepsen worker 896 - jepsen.util 2896 :invoke :add 9094 INFO [2022-01-10 08:39:54,501] jepsen worker 693 - jepsen.util 2693 :invoke :add 9095 INFO [2022-01-10 08:39:54,502] jepsen worker 154 - jepsen.util 154 :info :add 7766 :RequestTimeoutException INFO [2022-01-10 08:39:54,503] jepsen worker 456 - jepsen.util 1456 :info :add 7767 :RequestTimeoutException INFO [2022-01-10 08:39:54,503] jepsen worker 477 - jepsen.util 2477 :invoke :add 9096 INFO [2022-01-10 08:39:54,506] jepsen worker 154 - jepsen.util 1154 :invoke :add 9097 INFO [2022-01-10 08:39:54,508] jepsen worker 386 - jepsen.util 1386 :invoke :add 9098 INFO [2022-01-10 08:39:54,510] jepsen worker 225 - jepsen.util 1225 :invoke :add 9099 WARN [2022-01-10 08:39:54,511] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":10426834,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10426834,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":7838113,"encode_duration_us":278,"operation_id":"0x2061","timeout_ms":10000,"last_server_duration_us":7838113,"total_duration_us":10008667},{"operation_name":"insert","last_dispatch_duration_us":15098574,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15098574,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":5385169,"encode_duration_us":299,"operation_id":"0x1a72","timeout_ms":10000,"last_server_duration_us":5385169,"total_duration_us":10008961},{"operation_name":"insert","last_dispatch_duration_us":13782192,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13782192,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":7876114,"encode_duration_us":268,"operation_id":"0x1e44","timeout_ms":10000,"last_server_duration_us":7876114,"total_duration_us":10009587},{"operation_name":"insert","last_dispatch_duration_us":17598294,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17598294,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":4603783,"encode_duration_us":305,"operation_id":"0x1e07","timeout_ms":10000,"last_server_duration_us":4603783,"total_duration_us":10010009},{"operation_name":"insert","last_dispatch_duration_us":17216459,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17216459,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":3405941,"encode_duration_us":259,"operation_id":"0x1ef6","timeout_ms":10000,"last_server_duration_us":3405941,"total_duration_us":10009028},{"operation_name":"insert","last_dispatch_duration_us":18038016,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18038016,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":9009789,"encode_duration_us":267,"operation_id":"0x1bd9","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10009934},{"operation_name":"insert","last_dispatch_duration_us":11708763,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11708763,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":8880867,"encode_duration_us":265,"operation_id":"0x2037","timeout_ms":10000,"last_server_duration_us":8880867,"total_duration_us":10068481},{"operation_name":"insert","last_dispatch_duration_us":15795602,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15795602,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":3875842,"encode_duration_us":290,"operation_id":"0x202f","timeout_ms":10000,"last_server_duration_us":3875842,"total_duration_us":10084545},{"operation_name":"insert","last_dispatch_duration_us":11839348,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11839348,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":9016150,"encode_duration_us":264,"operation_id":"0x203b","timeout_ms":10000,"last_server_duration_us":9016150,"total_duration_us":10062100},{"operation_name":"insert","last_dispatch_duration_us":14176017,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14176017,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":6474640,"encode_duration_us":308,"operation_id":"0x1d72","timeout_ms":10000,"last_server_duration_us":6474640,"total_duration_us":10010796}],"total_count":58}} WARN [2022-01-10 08:39:54,511] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":15795602,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000002/00000000F4655C1A","last_local_socket":"172.23.104.79:41954","total_dispatch_duration_us":15795602,"total_server_duration_us":3875842,"encode_duration_us":290,"operation_id":"0x202f","timeout_ms":10000,"last_server_duration_us":3875842,"total_duration_us":10084545},{"operation_name":"insert","last_dispatch_duration_us":11708763,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","total_dispatch_duration_us":11708763,"total_server_duration_us":8880867,"encode_duration_us":265,"operation_id":"0x2037","timeout_ms":10000,"last_server_duration_us":8880867,"total_duration_us":10068481},{"operation_name":"insert","last_dispatch_duration_us":11839348,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","total_dispatch_duration_us":11839348,"total_server_duration_us":9016150,"encode_duration_us":264,"operation_id":"0x203b","timeout_ms":10000,"last_server_duration_us":9016150,"total_duration_us":10062100},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","encode_duration_us":258,"operation_id":"0x2001","timeout_ms":10000,"total_duration_us":10011000},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000002/00000000F4655C1A","last_local_socket":"172.23.104.79:41954","encode_duration_us":280,"operation_id":"0x2020","timeout_ms":10000,"total_duration_us":10010693},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","encode_duration_us":270,"operation_id":"0x22c7","timeout_ms":10000,"total_duration_us":10010457},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000002/00000000F4655C1A","last_local_socket":"172.23.104.79:41954","encode_duration_us":266,"operation_id":"0x20a5","timeout_ms":10000,"total_duration_us":10010439},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000002/00000000F4655C1A","last_local_socket":"172.23.104.79:41954","encode_duration_us":279,"operation_id":"0x1f49","timeout_ms":10000,"total_duration_us":10010426},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","encode_duration_us":281,"operation_id":"0x1fe4","timeout_ms":10000,"total_duration_us":10010268},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","encode_duration_us":281,"operation_id":"0x2255","timeout_ms":10000,"total_duration_us":10010150}],"total_count":188}} WARN [2022-01-10 08:39:54,511] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":18050142,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18050142,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9009789,"encode_duration_us":282,"operation_id":"0x1d84","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10009712},{"operation_name":"insert","last_dispatch_duration_us":4897260,"last_local_id":"2B99390F00000004/00000000EF8F24DE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":4897260,"last_local_socket":"172.23.104.79:51736","total_server_duration_us":4895192,"encode_duration_us":294,"operation_id":"0x204f","timeout_ms":10000,"last_server_duration_us":4895192,"total_duration_us":10009740},{"operation_name":"insert","last_dispatch_duration_us":14891393,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14891393,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":1166968,"encode_duration_us":259,"operation_id":"0x1e0c","timeout_ms":10000,"last_server_duration_us":1166968,"total_duration_us":10010053},{"operation_name":"insert","last_dispatch_duration_us":19427618,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19427618,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":8575515,"encode_duration_us":281,"operation_id":"0x1d37","timeout_ms":10000,"last_server_duration_us":8575515,"total_duration_us":10010043},{"operation_name":"insert","last_dispatch_duration_us":19851089,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19851089,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9016150,"encode_duration_us":394,"operation_id":"0x1c28","timeout_ms":10000,"last_server_duration_us":9016150,"total_duration_us":10009822},{"operation_name":"insert","last_dispatch_duration_us":18131326,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18131326,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":7416491,"encode_duration_us":268,"operation_id":"0x1c45","timeout_ms":10000,"last_server_duration_us":7416491,"total_duration_us":10010904},{"operation_name":"insert","last_dispatch_duration_us":18295523,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18295523,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":3419970,"encode_duration_us":262,"operation_id":"0x1b06","timeout_ms":10000,"last_server_duration_us":3419970,"total_duration_us":10010544},{"operation_name":"insert","last_dispatch_duration_us":11756791,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11756791,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":7422347,"encode_duration_us":462,"operation_id":"0x1ddd","timeout_ms":10000,"last_server_duration_us":7422347,"total_duration_us":10010067},{"operation_name":"insert","last_dispatch_duration_us":14250686,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14250686,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":3755219,"encode_duration_us":267,"operation_id":"0x2031","timeout_ms":10000,"last_server_duration_us":3755219,"total_duration_us":10079232},{"operation_name":"insert","last_dispatch_duration_us":17772701,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17772701,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":2840359,"encode_duration_us":265,"operation_id":"0x1ae4","timeout_ms":10000,"last_server_duration_us":2840359,"total_duration_us":10012570}],"total_count":83}} WARN [2022-01-10 08:39:54,511] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","encode_duration_us":276,"operation_id":"0x2256","timeout_ms":10000,"total_duration_us":10011130},{"operation_name":"insert","last_dispatch_duration_us":11960546,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","total_dispatch_duration_us":11960546,"total_server_duration_us":448733,"encode_duration_us":522,"operation_id":"0x1ffd","timeout_ms":10000,"last_server_duration_us":448733,"total_duration_us":10010964},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","encode_duration_us":265,"operation_id":"0x20d1","timeout_ms":10000,"total_duration_us":10010624},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","encode_duration_us":265,"operation_id":"0x2265","timeout_ms":10000,"total_duration_us":10010582},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","encode_duration_us":339,"operation_id":"0x1eb7","timeout_ms":10000,"total_duration_us":10010464},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","encode_duration_us":609,"operation_id":"0x1e0e","timeout_ms":10000,"total_duration_us":10010452},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","encode_duration_us":288,"operation_id":"0x20ef","timeout_ms":10000,"total_duration_us":10010448},{"operation_name":"insert","last_dispatch_duration_us":18639494,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","total_dispatch_duration_us":18639494,"total_server_duration_us":4632501,"encode_duration_us":259,"operation_id":"0x1e72","timeout_ms":10000,"last_server_duration_us":4632501,"total_duration_us":10010418},{"operation_name":"insert","encode_duration_us":275,"operation_id":"0x22c3","timeout_ms":10000,"total_duration_us":10010354},{"operation_name":"insert","last_dispatch_duration_us":8531142,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","total_dispatch_duration_us":8531142,"total_server_duration_us":6813572,"encode_duration_us":481,"operation_id":"0x1fd7","timeout_ms":10000,"last_server_duration_us":6813572,"total_duration_us":10010054}],"total_count":196}} WARN [2022-01-10 08:39:54,511] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":14250686,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","total_dispatch_duration_us":14250686,"total_server_duration_us":3755219,"encode_duration_us":267,"operation_id":"0x2031","timeout_ms":10000,"last_server_duration_us":3755219,"total_duration_us":10079232},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","encode_duration_us":270,"operation_id":"0x203a","timeout_ms":10000,"total_duration_us":10064064},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","encode_duration_us":274,"operation_id":"0x203d","timeout_ms":10000,"total_duration_us":10054215},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":330,"operation_id":"0x20bb","timeout_ms":10000,"total_duration_us":10010594},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","encode_duration_us":270,"operation_id":"0x22c0","timeout_ms":10000,"total_duration_us":10010382},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","encode_duration_us":273,"operation_id":"0x22f4","timeout_ms":10000,"total_duration_us":10010333},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","encode_duration_us":277,"operation_id":"0x2272","timeout_ms":10000,"total_duration_us":10010296},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","encode_duration_us":265,"operation_id":"0x227e","timeout_ms":10000,"total_duration_us":10010236},{"operation_name":"insert","last_dispatch_duration_us":11756791,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":11756791,"total_server_duration_us":7422347,"encode_duration_us":462,"operation_id":"0x1ddd","timeout_ms":10000,"last_server_duration_us":7422347,"total_duration_us":10010067},{"operation_name":"insert","last_dispatch_duration_us":14891393,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","total_dispatch_duration_us":14891393,"total_server_duration_us":1166968,"encode_duration_us":259,"operation_id":"0x1e0c","timeout_ms":10000,"last_server_duration_us":1166968,"total_duration_us":10010053}],"total_count":174}} WARN [2022-01-10 08:39:54,511] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":11032100,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11032100,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":4696591,"encode_duration_us":382,"operation_id":"0x1b4d","timeout_ms":10000,"last_server_duration_us":4696591,"total_duration_us":10009372},{"operation_name":"insert","last_dispatch_duration_us":18766748,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18766748,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":9007669,"encode_duration_us":262,"operation_id":"0x1b69","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10009890},{"operation_name":"insert","last_dispatch_duration_us":11231962,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11231962,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":4717501,"encode_duration_us":302,"operation_id":"0x1898","timeout_ms":10000,"last_server_duration_us":4717501,"total_duration_us":10009844},{"operation_name":"insert","last_dispatch_duration_us":11739725,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11739725,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":447846,"encode_duration_us":395,"operation_id":"0x2021","timeout_ms":10000,"last_server_duration_us":447846,"total_duration_us":10009958},{"operation_name":"insert","last_dispatch_duration_us":17721657,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17721657,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":7755383,"encode_duration_us":382,"operation_id":"0x1b02","timeout_ms":10000,"last_server_duration_us":7755383,"total_duration_us":10009914},{"operation_name":"insert","last_dispatch_duration_us":12154866,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12154866,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":4846232,"encode_duration_us":314,"operation_id":"0x1c91","timeout_ms":10000,"last_server_duration_us":4846232,"total_duration_us":10017388},{"operation_name":"insert","last_dispatch_duration_us":8531142,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":8531142,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":6813572,"encode_duration_us":481,"operation_id":"0x1fd7","timeout_ms":10000,"last_server_duration_us":6813572,"total_duration_us":10010054},{"operation_name":"insert","last_dispatch_duration_us":18639494,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18639494,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":4632501,"encode_duration_us":259,"operation_id":"0x1e72","timeout_ms":10000,"last_server_duration_us":4632501,"total_duration_us":10010418},{"operation_name":"insert","last_dispatch_duration_us":11257166,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11257166,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":6959174,"encode_duration_us":327,"operation_id":"0x1a37","timeout_ms":10000,"last_server_duration_us":6959174,"total_duration_us":10010706},{"operation_name":"insert","last_dispatch_duration_us":11960546,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11960546,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":448733,"encode_duration_us":522,"operation_id":"0x1ffd","timeout_ms":10000,"last_server_duration_us":448733,"total_duration_us":10010964}],"total_count":65}} INFO [2022-01-10 08:39:54,512] jepsen worker 382 - jepsen.util 2382 :invoke :add 9100 INFO [2022-01-10 08:39:54,514] jepsen worker 456 - jepsen.util 2456 :invoke :add 9101 INFO [2022-01-10 08:39:54,516] jepsen worker 292 - jepsen.util 2292 :invoke :add 9102 INFO [2022-01-10 08:39:54,518] jepsen worker 624 - jepsen.util 2624 :invoke :add 9103 INFO [2022-01-10 08:39:54,519] jepsen worker 200 - jepsen.util 2200 :invoke :add 9104 INFO [2022-01-10 08:39:54,521] jepsen worker 17 - jepsen.util 2017 :invoke :add 9105 INFO [2022-01-10 08:39:54,527] jepsen worker 127 - jepsen.util 127 :info :add 7776 :RequestTimeoutException INFO [2022-01-10 08:39:54,529] jepsen worker 127 - jepsen.util 1127 :invoke :add 9106 INFO [2022-01-10 08:39:54,531] jepsen worker 840 - jepsen.util 840 :info :add 7778 :RequestTimeoutException INFO [2022-01-10 08:39:54,531] jepsen worker 105 - jepsen.util 1105 :info :add 7777 :RequestTimeoutException INFO [2022-01-10 08:39:54,531] jepsen worker 134 - jepsen.util 1134 :info :add 7774 :RequestTimeoutException INFO [2022-01-10 08:39:54,532] jepsen worker 909 - jepsen.util 1909 :info :add 7775 :RequestTimeoutException INFO [2022-01-10 08:39:54,536] jepsen worker 105 - jepsen.util 2105 :invoke :add 9107 INFO [2022-01-10 08:39:54,538] jepsen worker 134 - jepsen.util 2134 :invoke :add 9108 INFO [2022-01-10 08:39:54,540] jepsen worker 840 - jepsen.util 1840 :invoke :add 9109 INFO [2022-01-10 08:39:54,541] jepsen worker 744 - jepsen.util 1744 :info :add 7779 :RequestTimeoutException INFO [2022-01-10 08:39:54,541] jepsen worker 654 - jepsen.util 1654 :info :add 7781 :RequestTimeoutException INFO [2022-01-10 08:39:54,543] jepsen worker 670 - jepsen.util 670 :info :add 7783 :RequestTimeoutException INFO [2022-01-10 08:39:54,543] jepsen worker 450 - jepsen.util 1450 :info :add 7780 :RequestTimeoutException INFO [2022-01-10 08:39:54,543] jepsen worker 648 - jepsen.util 1648 :info :add 7782 :RequestTimeoutException INFO [2022-01-10 08:39:54,543] jepsen worker 909 - jepsen.util 2909 :invoke :add 9110 INFO [2022-01-10 08:39:54,549] jepsen worker 450 - jepsen.util 2450 :invoke :add 9111 INFO [2022-01-10 08:39:54,551] jepsen worker 670 - jepsen.util 1670 :invoke :add 9112 INFO [2022-01-10 08:39:54,552] jepsen worker 648 - jepsen.util 2648 :invoke :add 9113 INFO [2022-01-10 08:39:54,554] jepsen worker 654 - jepsen.util 2654 :invoke :add 9114 INFO [2022-01-10 08:39:54,556] jepsen worker 744 - jepsen.util 2744 :invoke :add 9115 INFO [2022-01-10 08:39:54,557] jepsen worker 851 - jepsen.util 851 :info :add 7784 :RequestTimeoutException INFO [2022-01-10 08:39:54,557] jepsen worker 289 - jepsen.util 1289 :info :add 7785 :RequestTimeoutException INFO [2022-01-10 08:39:54,557] jepsen worker 486 - jepsen.util 486 :info :add 7786 :RequestTimeoutException INFO [2022-01-10 08:39:54,562] jepsen worker 147 - jepsen.util 147 :info :add 7787 :RequestTimeoutException INFO [2022-01-10 08:39:54,562] jepsen worker 289 - jepsen.util 2289 :invoke :add 9116 INFO [2022-01-10 08:39:54,565] jepsen worker 147 - jepsen.util 1147 :invoke :add 9117 INFO [2022-01-10 08:39:54,567] jepsen worker 486 - jepsen.util 1486 :invoke :add 9118 INFO [2022-01-10 08:39:54,569] jepsen worker 851 - jepsen.util 1851 :invoke :add 9119 INFO [2022-01-10 08:39:54,571] jepsen worker 695 - jepsen.util 1695 :info :add 7795 :RequestTimeoutException INFO [2022-01-10 08:39:54,573] jepsen worker 27 - jepsen.util 27 :info :add 7796 :RequestTimeoutException INFO [2022-01-10 08:39:54,574] jepsen worker 695 - jepsen.util 2695 :invoke :add 9120 INFO [2022-01-10 08:39:54,576] jepsen worker 27 - jepsen.util 1027 :invoke :add 9121 INFO [2022-01-10 08:39:54,577] jepsen worker 1 - jepsen.util 1001 :info :add 7794 :RequestTimeoutException INFO [2022-01-10 08:39:54,580] jepsen worker 1 - jepsen.util 2001 :invoke :add 9122 INFO [2022-01-10 08:39:54,581] jepsen worker 355 - jepsen.util 1355 :info :add 7797 :RequestTimeoutException INFO [2022-01-10 08:39:54,581] jepsen worker 240 - jepsen.util 240 :info :add 7800 :RequestTimeoutException INFO [2022-01-10 08:39:54,582] jepsen worker 296 - jepsen.util 1296 :info :add 7798 :RequestTimeoutException INFO [2022-01-10 08:39:54,583] jepsen worker 757 - jepsen.util 757 :info :add 7801 :RequestTimeoutException INFO [2022-01-10 08:39:54,586] jepsen worker 2 - jepsen.util 2 :info :add 7799 :RequestTimeoutException INFO [2022-01-10 08:39:54,587] jepsen worker 296 - jepsen.util 2296 :invoke :add 9123 INFO [2022-01-10 08:39:54,590] jepsen worker 240 - jepsen.util 1240 :invoke :add 9124 INFO [2022-01-10 08:39:54,591] jepsen worker 124 - jepsen.util 1124 :info :add 7802 :RequestTimeoutException INFO [2022-01-10 08:39:54,591] jepsen worker 214 - jepsen.util 1214 :info :add 7803 :RequestTimeoutException INFO [2022-01-10 08:39:54,592] jepsen worker 757 - jepsen.util 1757 :invoke :add 9125 INFO [2022-01-10 08:39:54,594] jepsen worker 185 - jepsen.util 1185 :info :add 7804 :RequestTimeoutException INFO [2022-01-10 08:39:54,594] jepsen worker 276 - jepsen.util 1276 :info :add 7805 :RequestTimeoutException INFO [2022-01-10 08:39:54,597] jepsen worker 465 - jepsen.util 1465 :info :add 7806 :RequestTimeoutException INFO [2022-01-10 08:39:54,599] jepsen worker 355 - jepsen.util 2355 :invoke :add 9126 INFO [2022-01-10 08:39:54,601] jepsen worker 214 - jepsen.util 2214 :invoke :add 9127 INFO [2022-01-10 08:39:54,602] jepsen worker 212 - jepsen.util 212 :info :add 7807 :RequestTimeoutException INFO [2022-01-10 08:39:54,603] jepsen worker 185 - jepsen.util 2185 :invoke :add 9128 INFO [2022-01-10 08:39:54,606] jepsen worker 276 - jepsen.util 2276 :invoke :add 9129 INFO [2022-01-10 08:39:54,608] jepsen worker 212 - jepsen.util 1212 :invoke :add 9130 INFO [2022-01-10 08:39:54,610] jepsen worker 465 - jepsen.util 2465 :invoke :add 9131 INFO [2022-01-10 08:39:54,612] jepsen worker 2 - jepsen.util 1002 :invoke :add 9132 WARN [2022-01-10 08:39:54,612] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","encode_duration_us":273,"operation_id":"0x20ec","timeout_ms":10000,"total_duration_us":10024580},{"operation_name":"insert","last_dispatch_duration_us":12505508,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":12505508,"total_server_duration_us":9006609,"encode_duration_us":294,"operation_id":"0x1e5a","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":10010675},{"operation_name":"insert","encode_duration_us":315,"operation_id":"0x21e8","timeout_ms":10000,"total_duration_us":10010427},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","encode_duration_us":318,"operation_id":"0x2242","timeout_ms":10000,"total_duration_us":10010404},{"operation_name":"insert","last_dispatch_duration_us":9641760,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":9641760,"total_server_duration_us":4595819,"encode_duration_us":262,"operation_id":"0x21fe","timeout_ms":10000,"last_server_duration_us":4595819,"total_duration_us":10010332},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","encode_duration_us":294,"operation_id":"0x22ca","timeout_ms":10000,"total_duration_us":10010208},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","encode_duration_us":304,"operation_id":"0x2134","timeout_ms":10000,"total_duration_us":10010188},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","encode_duration_us":263,"operation_id":"0x2309","timeout_ms":10000,"total_duration_us":10010074},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","encode_duration_us":259,"operation_id":"0x2194","timeout_ms":10000,"total_duration_us":10010029},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","encode_duration_us":298,"operation_id":"0x211d","timeout_ms":10000,"total_duration_us":10009438}],"total_count":177}} WARN [2022-01-10 08:39:54,612] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":12063743,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12063743,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":3467852,"encode_duration_us":322,"operation_id":"0x1f5c","timeout_ms":10000,"last_server_duration_us":3467852,"total_duration_us":10009367},{"operation_name":"insert","last_dispatch_duration_us":12033138,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12033138,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":3466440,"encode_duration_us":268,"operation_id":"0x1e06","timeout_ms":10000,"last_server_duration_us":3466440,"total_duration_us":10009375},{"operation_name":"insert","last_dispatch_duration_us":14401574,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14401574,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":6755322,"encode_duration_us":308,"operation_id":"0x1c48","timeout_ms":10000,"last_server_duration_us":6755322,"total_duration_us":10009695},{"operation_name":"insert","last_dispatch_duration_us":17529711,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17529711,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":8967433,"encode_duration_us":277,"operation_id":"0x1cbf","timeout_ms":10000,"last_server_duration_us":8967433,"total_duration_us":10009418},{"operation_name":"insert","last_dispatch_duration_us":9641760,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9641760,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":4595819,"encode_duration_us":262,"operation_id":"0x21fe","timeout_ms":10000,"last_server_duration_us":4595819,"total_duration_us":10010332},{"operation_name":"insert","last_dispatch_duration_us":11683817,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11683817,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":6534623,"encode_duration_us":315,"operation_id":"0x19aa","timeout_ms":10000,"last_server_duration_us":6534623,"total_duration_us":10009790},{"operation_name":"insert","last_dispatch_duration_us":16666570,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16666570,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9010849,"encode_duration_us":272,"operation_id":"0x1c44","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10010280},{"operation_name":"insert","last_dispatch_duration_us":20142896,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":20142896,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":3939742,"encode_duration_us":272,"operation_id":"0x1d87","timeout_ms":10000,"last_server_duration_us":3939742,"total_duration_us":10010352},{"operation_name":"insert","last_dispatch_duration_us":17034345,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17034345,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9011909,"encode_duration_us":299,"operation_id":"0x1982","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10010464},{"operation_name":"insert","last_dispatch_duration_us":12505508,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12505508,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9006609,"encode_duration_us":294,"operation_id":"0x1e5a","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":10010675}],"total_count":69}} WARN [2022-01-10 08:39:54,612] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":18555923,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18555923,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9002371,"encode_duration_us":308,"operation_id":"0x1ac5","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10008943},{"operation_name":"insert","last_dispatch_duration_us":12140443,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12140443,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9014029,"encode_duration_us":458,"operation_id":"0x2004","timeout_ms":10000,"last_server_duration_us":9014029,"total_duration_us":10008970},{"operation_name":"insert","last_dispatch_duration_us":17957425,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17957425,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9002371,"encode_duration_us":267,"operation_id":"0x1bf9","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10009430},{"operation_name":"insert","last_dispatch_duration_us":18577752,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18577752,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9015090,"encode_duration_us":262,"operation_id":"0x1ac0","timeout_ms":10000,"last_server_duration_us":9015090,"total_duration_us":10009617},{"operation_name":"insert","last_dispatch_duration_us":10182908,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10182908,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":7000064,"encode_duration_us":449,"operation_id":"0x1f04","timeout_ms":10000,"last_server_duration_us":7000064,"total_duration_us":10009046},{"operation_name":"insert","last_dispatch_duration_us":10283977,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10283977,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":1712100,"encode_duration_us":258,"operation_id":"0x1ed7","timeout_ms":10000,"last_server_duration_us":1712100,"total_duration_us":10010202},{"operation_name":"insert","last_dispatch_duration_us":10711549,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10711549,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":8127052,"encode_duration_us":343,"operation_id":"0x1fc9","timeout_ms":10000,"last_server_duration_us":8127052,"total_duration_us":10009941},{"operation_name":"insert","last_dispatch_duration_us":10209839,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10209839,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":8788365,"encode_duration_us":267,"operation_id":"0x20c2","timeout_ms":10000,"last_server_duration_us":8788365,"total_duration_us":10010415},{"operation_name":"insert","last_dispatch_duration_us":10396567,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10396567,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":7295915,"encode_duration_us":272,"operation_id":"0x1f59","timeout_ms":10000,"last_server_duration_us":7295915,"total_duration_us":10010058},{"operation_name":"insert","last_dispatch_duration_us":15103863,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15103863,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9008729,"encode_duration_us":282,"operation_id":"0x1cff","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10010915}],"total_count":63}} WARN [2022-01-10 08:39:54,612] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":330,"operation_id":"0x203e","timeout_ms":10000,"total_duration_us":10014129},{"operation_name":"insert","encode_duration_us":275,"operation_id":"0x1f99","timeout_ms":10000,"total_duration_us":10011164},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":284,"operation_id":"0x2017","timeout_ms":10000,"total_duration_us":10011152},{"operation_name":"insert","encode_duration_us":266,"operation_id":"0x1e9c","timeout_ms":10000,"total_duration_us":10010424},{"operation_name":"insert","last_dispatch_duration_us":10209839,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":10209839,"total_server_duration_us":8788365,"encode_duration_us":267,"operation_id":"0x20c2","timeout_ms":10000,"last_server_duration_us":8788365,"total_duration_us":10010415},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":291,"operation_id":"0x22c1","timeout_ms":10000,"total_duration_us":10010407},{"operation_name":"insert","last_dispatch_duration_us":10283977,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":10283977,"total_server_duration_us":1712100,"encode_duration_us":258,"operation_id":"0x1ed7","timeout_ms":10000,"last_server_duration_us":1712100,"total_duration_us":10010202},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","encode_duration_us":312,"operation_id":"0x22b7","timeout_ms":10000,"total_duration_us":10010179},{"operation_name":"insert","last_dispatch_duration_us":10396567,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":10396567,"total_server_duration_us":7295915,"encode_duration_us":272,"operation_id":"0x1f59","timeout_ms":10000,"last_server_duration_us":7295915,"total_duration_us":10010058},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000005/0000000013738E6A","last_local_socket":"172.23.104.79:41944","encode_duration_us":264,"operation_id":"0x1f2d","timeout_ms":10000,"total_duration_us":10010040}],"total_count":217}} INFO [2022-01-10 08:39:54,613] jepsen worker 542 - jepsen.util 542 :info :add 7815 :RequestTimeoutException INFO [2022-01-10 08:39:54,613] jepsen worker 124 - jepsen.util 2124 :invoke :add 9133 INFO [2022-01-10 08:39:54,616] jepsen worker 542 - jepsen.util 1542 :invoke :add 9134 INFO [2022-01-10 08:39:54,617] jepsen worker 528 - jepsen.util 528 :info :add 7814 :RequestTimeoutException INFO [2022-01-10 08:39:54,620] jepsen worker 528 - jepsen.util 1528 :invoke :add 9135 INFO [2022-01-10 08:39:54,622] jepsen worker 175 - jepsen.util 175 :info :add 7816 :RequestTimeoutException INFO [2022-01-10 08:39:54,622] jepsen worker 70 - jepsen.util 70 :info :add 7817 :RequestTimeoutException INFO [2022-01-10 08:39:54,622] jepsen worker 207 - jepsen.util 1207 :info :add 7821 :RequestTimeoutException INFO [2022-01-10 08:39:54,623] jepsen worker 359 - jepsen.util 1359 :info :add 7818 :RequestTimeoutException INFO [2022-01-10 08:39:54,626] jepsen worker 175 - jepsen.util 1175 :invoke :add 9136 INFO [2022-01-10 08:39:54,627] jepsen worker 328 - jepsen.util 328 :info :add 7820 :RequestTimeoutException INFO [2022-01-10 08:39:54,627] jepsen worker 518 - jepsen.util 1518 :info :add 7822 :RequestTimeoutException INFO [2022-01-10 08:39:54,628] jepsen worker 359 - jepsen.util 2359 :invoke :add 9137 INFO [2022-01-10 08:39:54,631] jepsen worker 84 - jepsen.util 1084 :info :add 7823 :RequestTimeoutException INFO [2022-01-10 08:39:54,631] jepsen worker 207 - jepsen.util 2207 :invoke :add 9138 INFO [2022-01-10 08:39:54,634] jepsen worker 328 - jepsen.util 1328 :invoke :add 9139 INFO [2022-01-10 08:39:54,635] jepsen worker 70 - jepsen.util 1070 :invoke :add 9140 INFO [2022-01-10 08:39:54,637] jepsen worker 518 - jepsen.util 2518 :invoke :add 9141 INFO [2022-01-10 08:39:54,638] jepsen worker 84 - jepsen.util 2084 :invoke :add 9142 INFO [2022-01-10 08:39:54,640] jepsen worker 403 - jepsen.util 403 :info :add 7831 :RequestTimeoutException INFO [2022-01-10 08:39:54,641] jepsen worker 990 - jepsen.util 990 :info :add 7828 :RequestTimeoutException INFO [2022-01-10 08:39:54,641] jepsen worker 763 - jepsen.util 763 :info :add 7829 :RequestTimeoutException INFO [2022-01-10 08:39:54,641] jepsen worker 724 - jepsen.util 1724 :info :add 7830 :RequestTimeoutException INFO [2022-01-10 08:39:54,643] jepsen worker 60 - jepsen.util 60 :info :add 7832 :RequestTimeoutException INFO [2022-01-10 08:39:54,643] jepsen worker 403 - jepsen.util 1403 :invoke :add 9143 INFO [2022-01-10 08:39:54,647] jepsen worker 60 - jepsen.util 1060 :invoke :add 9144 INFO [2022-01-10 08:39:54,649] jepsen worker 724 - jepsen.util 2724 :invoke :add 9145 INFO [2022-01-10 08:39:54,650] jepsen worker 405 - jepsen.util 1405 :info :add 7837 :RequestTimeoutException INFO [2022-01-10 08:39:54,651] jepsen worker 866 - jepsen.util 1866 :info :add 7833 :RequestTimeoutException INFO [2022-01-10 08:39:54,651] jepsen worker 401 - jepsen.util 401 :info :add 7834 :RequestTimeoutException INFO [2022-01-10 08:39:54,651] jepsen worker 994 - jepsen.util 1994 :info :add 7838 :RequestTimeoutException INFO [2022-01-10 08:39:54,651] jepsen worker 763 - jepsen.util 1763 :invoke :add 9146 INFO [2022-01-10 08:39:54,652] jepsen worker 568 - jepsen.util 568 :info :add 7836 :RequestTimeoutException INFO [2022-01-10 08:39:54,653] jepsen worker 663 - jepsen.util 1663 :info :add 7835 :RequestTimeoutException INFO [2022-01-10 08:39:54,657] jepsen worker 568 - jepsen.util 1568 :invoke :add 9147 INFO [2022-01-10 08:39:54,659] jepsen worker 663 - jepsen.util 2663 :invoke :add 9148 INFO [2022-01-10 08:39:54,660] jepsen worker 981 - jepsen.util 981 :info :add 7839 :RequestTimeoutException INFO [2022-01-10 08:39:54,660] jepsen worker 36 - jepsen.util 36 :info :add 7841 :RequestTimeoutException INFO [2022-01-10 08:39:54,661] jepsen worker 994 - jepsen.util 2994 :invoke :add 9149 INFO [2022-01-10 08:39:54,661] jepsen worker 410 - jepsen.util 1410 :info :add 7842 :RequestTimeoutException INFO [2022-01-10 08:39:54,661] jepsen worker 583 - jepsen.util 1583 :info :add 7844 :RequestTimeoutException INFO [2022-01-10 08:39:54,662] jepsen worker 691 - jepsen.util 691 :info :add 7840 :RequestTimeoutException INFO [2022-01-10 08:39:54,663] jepsen worker 597 - jepsen.util 1597 :info :add 7843 :RequestTimeoutException INFO [2022-01-10 08:39:54,663] jepsen worker 885 - jepsen.util 1885 :info :add 7845 :RequestTimeoutException INFO [2022-01-10 08:39:54,669] jepsen worker 691 - jepsen.util 1691 :invoke :add 9150 INFO [2022-01-10 08:39:54,671] jepsen worker 880 - jepsen.util 1880 :info :add 7846 :RequestTimeoutException INFO [2022-01-10 08:39:54,671] jepsen worker 36 - jepsen.util 1036 :invoke :add 9151 INFO [2022-01-10 08:39:54,672] jepsen worker 400 - jepsen.util 1400 :info :add 7849 :RequestTimeoutException INFO [2022-01-10 08:39:54,673] jepsen worker 405 - jepsen.util 2405 :invoke :add 9152 INFO [2022-01-10 08:39:54,674] jepsen worker 839 - jepsen.util 839 :info :add 7847 :RequestTimeoutException INFO [2022-01-10 08:39:54,674] jepsen worker 789 - jepsen.util 1789 :info :add 7848 :RequestTimeoutException INFO [2022-01-10 08:39:54,674] jepsen worker 930 - jepsen.util 1930 :info :add 7850 :RequestTimeoutException INFO [2022-01-10 08:39:54,674] jepsen worker 351 - jepsen.util 351 :info :add 7851 :RequestTimeoutException INFO [2022-01-10 08:39:54,680] jepsen worker 400 - jepsen.util 2400 :invoke :add 9153 INFO [2022-01-10 08:39:54,681] jepsen worker 80 - jepsen.util 1080 :info :add 7852 :RequestTimeoutException INFO [2022-01-10 08:39:54,682] jepsen worker 880 - jepsen.util 2880 :invoke :add 9154 INFO [2022-01-10 08:39:54,684] jepsen worker 268 - jepsen.util 1268 :info :add 7853 :RequestTimeoutException INFO [2022-01-10 08:39:54,685] jepsen worker 930 - jepsen.util 2930 :invoke :add 9155 INFO [2022-01-10 08:39:54,685] jepsen worker 572 - jepsen.util 2572 :ok :add 9083 INFO [2022-01-10 08:39:54,688] jepsen worker 981 - jepsen.util 1981 :invoke :add 9156 INFO [2022-01-10 08:39:54,691] jepsen worker 33 - jepsen.util 33 :info :add 7858 :RequestTimeoutException INFO [2022-01-10 08:39:54,692] jepsen worker 670 - jepsen.util 1670 :ok :add 9112 INFO [2022-01-10 08:39:54,692] jepsen worker 355 - jepsen.util 2355 :ok :add 9126 INFO [2022-01-10 08:39:54,692] jepsen worker 605 - jepsen.util 605 :info :add 7859 :RequestTimeoutException INFO [2022-01-10 08:39:54,693] jepsen worker 401 - jepsen.util 1401 :invoke :add 9157 INFO [2022-01-10 08:39:54,695] jepsen worker 630 - jepsen.util 630 :ok :add 8069 INFO [2022-01-10 08:39:54,695] jepsen worker 757 - jepsen.util 1757 :ok :add 9125 INFO [2022-01-10 08:39:54,696] jepsen worker 139 - jepsen.util 2139 :ok :add 8446 INFO [2022-01-10 08:39:54,696] jepsen worker 508 - jepsen.util 2508 :ok :add 8703 INFO [2022-01-10 08:39:54,697] jepsen worker 450 - jepsen.util 2450 :ok :add 9111 INFO [2022-01-10 08:39:54,698] jepsen worker 225 - jepsen.util 1225 :ok :add 9099 INFO [2022-01-10 08:39:54,698] jepsen worker 840 - jepsen.util 1840 :ok :add 9109 INFO [2022-01-10 08:39:54,701] jepsen worker 445 - jepsen.util 1445 :info :add 7864 :RequestTimeoutException INFO [2022-01-10 08:39:54,706] jepsen worker 909 - jepsen.util 2909 :ok :add 9110 INFO [2022-01-10 08:39:54,706] jepsen worker 386 - jepsen.util 1386 :ok :add 9098 INFO [2022-01-10 08:39:54,706] jepsen worker 134 - jepsen.util 2134 :ok :add 9108 INFO [2022-01-10 08:39:54,706] jepsen worker 240 - jepsen.util 1240 :ok :add 9124 INFO [2022-01-10 08:39:54,707] jepsen worker 445 - jepsen.util 2445 :invoke :add 9158 INFO [2022-01-10 08:39:54,708] jepsen worker 648 - jepsen.util 2648 :ok :add 9113 INFO [2022-01-10 08:39:54,708] jepsen worker 214 - jepsen.util 2214 :ok :add 9127 INFO [2022-01-10 08:39:54,711] jepsen worker 33 - jepsen.util 1033 :invoke :add 9159 INFO [2022-01-10 08:39:54,714] jepsen worker 268 - jepsen.util 2268 :invoke :add 9160 INFO [2022-01-10 08:39:54,716] jepsen worker 240 - jepsen.util 1240 :invoke :add 9161 INFO [2022-01-10 08:39:54,719] jepsen worker 355 - jepsen.util 2355 :invoke :add 9162 INFO [2022-01-10 08:39:54,722] jepsen worker 139 - jepsen.util 2139 :invoke :add 9163 INFO [2022-01-10 08:39:54,724] jepsen worker 80 - jepsen.util 2080 :invoke :add 9164 INFO [2022-01-10 08:39:54,727] jepsen worker 605 - jepsen.util 1605 :invoke :add 9165 INFO [2022-01-10 08:39:54,729] jepsen worker 866 - jepsen.util 2866 :invoke :add 9166 INFO [2022-01-10 08:39:54,731] jepsen worker 990 - jepsen.util 1990 :invoke :add 9167 INFO [2022-01-10 08:39:54,733] jepsen worker 597 - jepsen.util 2597 :invoke :add 9168 INFO [2022-01-10 08:39:54,735] jepsen worker 909 - jepsen.util 2909 :invoke :add 9169 INFO [2022-01-10 08:39:54,738] jepsen worker 450 - jepsen.util 2450 :invoke :add 9170 INFO [2022-01-10 08:39:54,741] jepsen worker 386 - jepsen.util 1386 :invoke :add 9171 INFO [2022-01-10 08:39:54,742] jepsen worker 789 - jepsen.util 2789 :invoke :add 9172 INFO [2022-01-10 08:39:54,744] jepsen worker 572 - jepsen.util 2572 :invoke :add 9173 INFO [2022-01-10 08:39:54,746] jepsen worker 225 - jepsen.util 1225 :invoke :add 9174 INFO [2022-01-10 08:39:54,747] jepsen worker 648 - jepsen.util 2648 :invoke :add 9175 INFO [2022-01-10 08:39:54,749] jepsen worker 670 - jepsen.util 1670 :invoke :add 9176 INFO [2022-01-10 08:39:54,752] jepsen worker 840 - jepsen.util 1840 :invoke :add 9177 INFO [2022-01-10 08:39:54,755] jepsen worker 630 - jepsen.util 630 :invoke :add 9178 INFO [2022-01-10 08:39:54,757] jepsen worker 410 - jepsen.util 2410 :invoke :add 9179 INFO [2022-01-10 08:39:54,759] jepsen worker 583 - jepsen.util 2583 :invoke :add 9180 INFO [2022-01-10 08:39:54,761] jepsen worker 508 - jepsen.util 2508 :invoke :add 9181 INFO [2022-01-10 08:39:54,764] jepsen worker 757 - jepsen.util 1757 :invoke :add 9182 INFO [2022-01-10 08:39:54,767] jepsen worker 839 - jepsen.util 1839 :invoke :add 9183 INFO [2022-01-10 08:39:54,769] jepsen worker 134 - jepsen.util 2134 :invoke :add 9184 INFO [2022-01-10 08:39:54,772] jepsen worker 885 - jepsen.util 2885 :invoke :add 9185 INFO [2022-01-10 08:39:54,775] jepsen worker 351 - jepsen.util 1351 :invoke :add 9186 INFO [2022-01-10 08:39:54,776] jepsen worker 214 - jepsen.util 2214 :invoke :add 9187 INFO [2022-01-10 08:39:54,795] jepsen worker 930 - jepsen.util 2930 :ok :add 9155 INFO [2022-01-10 08:39:54,795] jepsen worker 880 - jepsen.util 2880 :ok :add 9154 INFO [2022-01-10 08:39:54,799] jepsen worker 981 - jepsen.util 1981 :ok :add 9156 INFO [2022-01-10 08:39:54,799] jepsen worker 355 - jepsen.util 2355 :ok :add 9162 INFO [2022-01-10 08:39:54,799] jepsen worker 930 - jepsen.util 2930 :invoke :add 9188 INFO [2022-01-10 08:39:54,801] jepsen worker 355 - jepsen.util 2355 :invoke :add 9189 INFO [2022-01-10 08:39:54,804] jepsen worker 880 - jepsen.util 2880 :invoke :add 9190 INFO [2022-01-10 08:39:54,805] jepsen worker 139 - jepsen.util 2139 :ok :add 9163 INFO [2022-01-10 08:39:54,805] jepsen worker 401 - jepsen.util 1401 :ok :add 9157 INFO [2022-01-10 08:39:54,807] jepsen worker 981 - jepsen.util 1981 :invoke :add 9191 INFO [2022-01-10 08:39:54,810] jepsen worker 401 - jepsen.util 1401 :invoke :add 9192 INFO [2022-01-10 08:39:54,812] jepsen worker 376 - jepsen.util 1376 :info :add 7866 :RequestTimeoutException INFO [2022-01-10 08:39:54,813] jepsen worker 762 - jepsen.util 1762 :info :add 7865 :RequestTimeoutException INFO [2022-01-10 08:39:54,813] jepsen worker 704 - jepsen.util 1704 :info :add 7867 :RequestTimeoutException INFO [2022-01-10 08:39:54,814] jepsen worker 139 - jepsen.util 2139 :invoke :add 9193 INFO [2022-01-10 08:39:54,815] jepsen worker 240 - jepsen.util 1240 :ok :add 9161 INFO [2022-01-10 08:39:54,815] jepsen worker 410 - jepsen.util 2410 :ok :add 9179 INFO [2022-01-10 08:39:54,818] jepsen worker 630 - jepsen.util 630 :ok :add 9178 INFO [2022-01-10 08:39:54,818] jepsen worker 268 - jepsen.util 2268 :ok :add 9160 INFO [2022-01-10 08:39:54,821] jepsen worker 583 - jepsen.util 2583 :ok :add 9180 INFO [2022-01-10 08:39:54,822] jepsen worker 704 - jepsen.util 2704 :invoke :add 9194 INFO [2022-01-10 08:39:54,823] jepsen worker 149 - jepsen.util 1149 :info :add 7868 :RequestTimeoutException INFO [2022-01-10 08:39:54,823] jepsen worker 508 - jepsen.util 2508 :ok :add 9181 INFO [2022-01-10 08:39:54,824] jepsen worker 344 - jepsen.util 1344 :info :add 7871 :RequestTimeoutException INFO [2022-01-10 08:39:54,825] jepsen worker 757 - jepsen.util 1757 :ok :add 9182 INFO [2022-01-10 08:39:54,825] jepsen worker 376 - jepsen.util 2376 :invoke :add 9195 INFO [2022-01-10 08:39:54,827] jepsen worker 635 - jepsen.util 1635 :info :add 7869 :RequestTimeoutException INFO [2022-01-10 08:39:54,828] jepsen worker 839 - jepsen.util 1839 :ok :add 9183 INFO [2022-01-10 08:39:54,833] jepsen worker 762 - jepsen.util 2762 :invoke :add 9196 INFO [2022-01-10 08:39:54,836] jepsen worker 240 - jepsen.util 1240 :invoke :add 9197 INFO [2022-01-10 08:39:54,838] jepsen worker 508 - jepsen.util 2508 :invoke :add 9198 INFO [2022-01-10 08:39:54,841] jepsen worker 410 - jepsen.util 2410 :invoke :add 9199 INFO [2022-01-10 08:39:54,841] jepsen worker 996 - jepsen.util 1996 :info :add 7872 :RequestTimeoutException INFO [2022-01-10 08:39:54,843] jepsen worker 635 - jepsen.util 2635 :invoke :add 9200 INFO [2022-01-10 08:39:54,847] jepsen worker 996 - jepsen.util 2996 :invoke :add 9201 INFO [2022-01-10 08:39:54,848] jepsen worker 508 - jepsen.util 2508 :ok :add 9198 INFO [2022-01-10 08:39:54,849] jepsen worker 410 - jepsen.util 2410 :ok :add 9199 INFO [2022-01-10 08:39:54,849] jepsen worker 149 - jepsen.util 2149 :invoke :add 9202 INFO [2022-01-10 08:39:54,852] jepsen worker 508 - jepsen.util 2508 :invoke :add 9203 INFO [2022-01-10 08:39:54,855] jepsen worker 757 - jepsen.util 1757 :invoke :add 9204 INFO [2022-01-10 08:39:54,858] jepsen worker 268 - jepsen.util 2268 :invoke :add 9205 INFO [2022-01-10 08:39:54,860] jepsen worker 583 - jepsen.util 2583 :invoke :add 9206 INFO [2022-01-10 08:39:54,864] jepsen worker 839 - jepsen.util 1839 :invoke :add 9207 INFO [2022-01-10 08:39:54,865] jepsen worker 344 - jepsen.util 2344 :invoke :add 9208 INFO [2022-01-10 08:39:54,867] jepsen worker 410 - jepsen.util 2410 :invoke :add 9209 INFO [2022-01-10 08:39:54,869] jepsen worker 630 - jepsen.util 630 :invoke :add 9210 INFO [2022-01-10 08:39:54,874] jepsen worker 344 - jepsen.util 2344 :ok :add 9208 INFO [2022-01-10 08:39:54,875] jepsen worker 410 - jepsen.util 2410 :ok :add 9209 INFO [2022-01-10 08:39:54,876] jepsen worker 344 - jepsen.util 2344 :invoke :add 9211 INFO [2022-01-10 08:39:54,878] jepsen worker 410 - jepsen.util 2410 :invoke :add 9212 INFO [2022-01-10 08:39:54,882] jepsen worker 630 - jepsen.util 630 :ok :add 9210 INFO [2022-01-10 08:39:54,884] jepsen worker 630 - jepsen.util 630 :invoke :add 9213 INFO [2022-01-10 08:39:54,888] jepsen worker 344 - jepsen.util 2344 :ok :add 9211 INFO [2022-01-10 08:39:54,893] jepsen worker 344 - jepsen.util 2344 :invoke :add 9214 INFO [2022-01-10 08:39:54,899] jepsen worker 410 - jepsen.util 2410 :ok :add 9212 INFO [2022-01-10 08:39:54,899] jepsen worker 630 - jepsen.util 630 :ok :add 9213 INFO [2022-01-10 08:39:54,902] jepsen worker 410 - jepsen.util 2410 :invoke :add 9215 INFO [2022-01-10 08:39:54,906] jepsen worker 630 - jepsen.util 630 :invoke :add 9216 INFO [2022-01-10 08:39:55,005] jepsen worker 815 - jepsen.util 815 :info :add 7873 :RequestTimeoutException INFO [2022-01-10 08:39:55,005] jepsen worker 987 - jepsen.util 987 :info :add 7874 :RequestTimeoutException INFO [2022-01-10 08:39:55,007] jepsen worker 838 - jepsen.util 838 :info :add 7875 :RequestTimeoutException INFO [2022-01-10 08:39:55,008] jepsen worker 987 - jepsen.util 1987 :invoke :add 9217 INFO [2022-01-10 08:39:55,010] jepsen worker 819 - jepsen.util 819 :info :add 7877 :RequestTimeoutException INFO [2022-01-10 08:39:55,011] jepsen worker 815 - jepsen.util 1815 :invoke :add 9218 INFO [2022-01-10 08:39:55,014] jepsen worker 838 - jepsen.util 1838 :invoke :add 9219 INFO [2022-01-10 08:39:55,015] jepsen worker 819 - jepsen.util 1819 :invoke :add 9220 INFO [2022-01-10 08:39:55,017] jepsen worker 778 - jepsen.util 1778 :info :add 7876 :RequestTimeoutException INFO [2022-01-10 08:39:55,020] jepsen worker 778 - jepsen.util 2778 :invoke :add 9221 INFO [2022-01-10 08:39:55,027] jepsen worker 792 - jepsen.util 1792 :info :add 7885 :RequestTimeoutException INFO [2022-01-10 08:39:55,029] jepsen worker 792 - jepsen.util 2792 :invoke :add 9222 INFO [2022-01-10 08:39:55,031] jepsen worker 615 - jepsen.util 1615 :info :add 7886 :RequestTimeoutException INFO [2022-01-10 08:39:55,032] jepsen worker 394 - jepsen.util 394 :info :add 7884 :RequestTimeoutException INFO [2022-01-10 08:39:55,033] jepsen worker 615 - jepsen.util 2615 :invoke :add 9223 INFO [2022-01-10 08:39:55,035] jepsen worker 394 - jepsen.util 1394 :invoke :add 9224 INFO [2022-01-10 08:39:55,041] jepsen worker 394 - jepsen.util 1394 :ok :add 9224 INFO [2022-01-10 08:39:55,044] jepsen worker 394 - jepsen.util 1394 :invoke :add 9225 INFO [2022-01-10 08:39:55,049] jepsen worker 394 - jepsen.util 1394 :ok :add 9225 INFO [2022-01-10 08:39:55,051] jepsen worker 394 - jepsen.util 1394 :invoke :add 9226 INFO [2022-01-10 08:39:55,135] jepsen worker 394 - jepsen.util 1394 :ok :add 9226 INFO [2022-01-10 08:39:55,137] jepsen worker 394 - jepsen.util 1394 :invoke :add 9227 INFO [2022-01-10 08:39:55,141] jepsen worker 394 - jepsen.util 1394 :ok :add 9227 INFO [2022-01-10 08:39:55,143] jepsen worker 394 - jepsen.util 1394 :invoke :add 9228 INFO [2022-01-10 08:39:55,160] jepsen worker 423 - jepsen.util 423 :info :add 7888 :RequestTimeoutException INFO [2022-01-10 08:39:55,163] jepsen worker 423 - jepsen.util 1423 :invoke :add 9229 INFO [2022-01-10 08:39:55,171] jepsen worker 509 - jepsen.util 1509 :info :add 7889 :RequestTimeoutException INFO [2022-01-10 08:39:55,172] jepsen worker 41 - jepsen.util 41 :info :add 7890 :RequestTimeoutException INFO [2022-01-10 08:39:55,173] jepsen worker 560 - jepsen.util 1560 :info :add 7891 :RequestTimeoutException INFO [2022-01-10 08:39:55,174] jepsen worker 509 - jepsen.util 2509 :invoke :add 9230 INFO [2022-01-10 08:39:55,177] jepsen worker 41 - jepsen.util 1041 :invoke :add 9231 INFO [2022-01-10 08:39:55,177] jepsen worker 451 - jepsen.util 451 :info :add 7892 :RequestTimeoutException INFO [2022-01-10 08:39:55,179] jepsen worker 451 - jepsen.util 1451 :invoke :add 9232 INFO [2022-01-10 08:39:55,180] jepsen worker 576 - jepsen.util 576 :info :add 7893 :RequestTimeoutException INFO [2022-01-10 08:39:55,181] jepsen worker 560 - jepsen.util 2560 :invoke :add 9233 INFO [2022-01-10 08:39:55,181] jepsen worker 737 - jepsen.util 737 :info :add 7895 :RequestTimeoutException INFO [2022-01-10 08:39:55,182] jepsen worker 304 - jepsen.util 304 :info :add 7896 :RequestTimeoutException INFO [2022-01-10 08:39:55,183] jepsen worker 281 - jepsen.util 281 :info :add 7894 :RequestTimeoutException INFO [2022-01-10 08:39:55,185] jepsen worker 576 - jepsen.util 1576 :invoke :add 9234 INFO [2022-01-10 08:39:55,187] jepsen worker 737 - jepsen.util 1737 :invoke :add 9235 INFO [2022-01-10 08:39:55,190] jepsen worker 304 - jepsen.util 1304 :invoke :add 9236 INFO [2022-01-10 08:39:55,191] jepsen worker 281 - jepsen.util 1281 :invoke :add 9237 INFO [2022-01-10 08:39:55,203] jepsen worker 676 - jepsen.util 1676 :info :add 7897 :RequestTimeoutException INFO [2022-01-10 08:39:55,205] jepsen worker 676 - jepsen.util 2676 :invoke :add 9238 INFO [2022-01-10 08:39:55,207] jepsen worker 960 - jepsen.util 960 :info :add 7901 :RequestTimeoutException INFO [2022-01-10 08:39:55,210] jepsen worker 960 - jepsen.util 1960 :invoke :add 9239 INFO [2022-01-10 08:39:55,212] jepsen worker 439 - jepsen.util 1439 :info :add 7904 :RequestTimeoutException INFO [2022-01-10 08:39:55,212] jepsen worker 888 - jepsen.util 1888 :info :add 7900 :RequestTimeoutException INFO [2022-01-10 08:39:55,212] jepsen worker 682 - jepsen.util 1682 :info :add 7902 :RequestTimeoutException INFO [2022-01-10 08:39:55,215] jepsen worker 888 - jepsen.util 2888 :invoke :add 9240 INFO [2022-01-10 08:39:55,217] jepsen worker 682 - jepsen.util 2682 :invoke :add 9241 INFO [2022-01-10 08:39:55,218] jepsen worker 774 - jepsen.util 774 :info :add 7903 :RequestTimeoutException INFO [2022-01-10 08:39:55,218] jepsen worker 74 - jepsen.util 1074 :info :add 7906 :RequestTimeoutException INFO [2022-01-10 08:39:55,219] jepsen worker 439 - jepsen.util 2439 :invoke :add 9242 INFO [2022-01-10 08:39:55,221] jepsen worker 143 - jepsen.util 1143 :info :add 7905 :RequestTimeoutException INFO [2022-01-10 08:39:55,221] jepsen worker 774 - jepsen.util 1774 :invoke :add 9243 INFO [2022-01-10 08:39:55,223] jepsen worker 855 - jepsen.util 855 :info :add 7907 :RequestTimeoutException INFO [2022-01-10 08:39:55,225] jepsen worker 74 - jepsen.util 2074 :invoke :add 9244 INFO [2022-01-10 08:39:55,227] jepsen worker 143 - jepsen.util 2143 :invoke :add 9245 INFO [2022-01-10 08:39:55,229] jepsen worker 855 - jepsen.util 1855 :invoke :add 9246 INFO [2022-01-10 08:39:55,241] jepsen worker 419 - jepsen.util 1419 :info :add 7917 :RequestTimeoutException INFO [2022-01-10 08:39:55,242] jepsen worker 169 - jepsen.util 169 :info :add 7915 :RequestTimeoutException INFO [2022-01-10 08:39:55,243] jepsen worker 530 - jepsen.util 1530 :info :add 7914 :RequestTimeoutException INFO [2022-01-10 08:39:55,243] jepsen worker 790 - jepsen.util 1790 :info :add 7918 :RequestTimeoutException INFO [2022-01-10 08:39:55,245] jepsen worker 419 - jepsen.util 2419 :invoke :add 9247 INFO [2022-01-10 08:39:55,246] jepsen worker 845 - jepsen.util 1845 :info :add 7916 :RequestTimeoutException INFO [2022-01-10 08:39:55,249] jepsen worker 845 - jepsen.util 2845 :invoke :add 9248 INFO [2022-01-10 08:39:55,251] jepsen worker 790 - jepsen.util 2790 :invoke :add 9249 INFO [2022-01-10 08:39:55,252] jepsen worker 113 - jepsen.util 1113 :info :add 7922 :RequestTimeoutException INFO [2022-01-10 08:39:55,252] jepsen worker 964 - jepsen.util 964 :info :add 7919 :RequestTimeoutException INFO [2022-01-10 08:39:55,252] jepsen worker 530 - jepsen.util 2530 :invoke :add 9250 INFO [2022-01-10 08:39:55,254] jepsen worker 283 - jepsen.util 1283 :info :add 7920 :RequestTimeoutException INFO [2022-01-10 08:39:55,254] jepsen worker 125 - jepsen.util 1125 :info :add 7923 :RequestTimeoutException INFO [2022-01-10 08:39:55,254] jepsen worker 438 - jepsen.util 1438 :info :add 7921 :RequestTimeoutException INFO [2022-01-10 08:39:55,255] jepsen worker 113 - jepsen.util 2113 :invoke :add 9251 INFO [2022-01-10 08:39:55,259] jepsen worker 125 - jepsen.util 2125 :invoke :add 9252 INFO [2022-01-10 08:39:55,261] jepsen worker 964 - jepsen.util 1964 :invoke :add 9253 INFO [2022-01-10 08:39:55,263] jepsen worker 169 - jepsen.util 1169 :invoke :add 9254 INFO [2022-01-10 08:39:55,266] jepsen worker 283 - jepsen.util 2283 :invoke :add 9255 INFO [2022-01-10 08:39:55,267] jepsen worker 251 - jepsen.util 1251 :info :add 7928 :RequestTimeoutException INFO [2022-01-10 08:39:55,269] jepsen worker 438 - jepsen.util 2438 :invoke :add 9256 INFO [2022-01-10 08:39:55,271] jepsen worker 965 - jepsen.util 965 :info :add 7929 :RequestTimeoutException INFO [2022-01-10 08:39:55,271] jepsen worker 891 - jepsen.util 891 :info :add 7930 :RequestTimeoutException INFO [2022-01-10 08:39:55,272] jepsen worker 807 - jepsen.util 807 :info :add 7931 :RequestTimeoutException INFO [2022-01-10 08:39:55,272] jepsen worker 510 - jepsen.util 1510 :info :add 7933 :RequestTimeoutException INFO [2022-01-10 08:39:55,272] jepsen worker 251 - jepsen.util 2251 :invoke :add 9257 INFO [2022-01-10 08:39:55,273] jepsen worker 169 - jepsen.util 1169 :ok :add 9254 INFO [2022-01-10 08:39:55,273] jepsen worker 82 - jepsen.util 1082 :info :add 7932 :RequestTimeoutException INFO [2022-01-10 08:39:55,277] jepsen worker 340 - jepsen.util 340 :info :add 7934 :RequestTimeoutException INFO [2022-01-10 08:39:55,277] jepsen worker 438 - jepsen.util 2438 :ok :add 9256 INFO [2022-01-10 08:39:55,277] jepsen worker 100 - jepsen.util 1100 :info :add 7936 :RequestTimeoutException INFO [2022-01-10 08:39:55,279] jepsen worker 283 - jepsen.util 2283 :ok :add 9255 INFO [2022-01-10 08:39:55,280] jepsen worker 264 - jepsen.util 1264 :info :add 7937 :RequestTimeoutException INFO [2022-01-10 08:39:55,280] jepsen worker 274 - jepsen.util 274 :info :add 7938 :RequestTimeoutException INFO [2022-01-10 08:39:55,281] jepsen worker 367 - jepsen.util 367 :info :add 7935 :RequestTimeoutException INFO [2022-01-10 08:39:55,282] jepsen worker 251 - jepsen.util 2251 :ok :add 9257 INFO [2022-01-10 08:39:55,286] jepsen worker 340 - jepsen.util 1340 :invoke :add 9258 INFO [2022-01-10 08:39:55,287] jepsen worker 346 - jepsen.util 346 :info :add 7942 :RequestTimeoutException INFO [2022-01-10 08:39:55,287] jepsen worker 274 - jepsen.util 1274 :invoke :add 9259 INFO [2022-01-10 08:39:55,290] jepsen worker 37 - jepsen.util 37 :info :add 7939 :RequestTimeoutException INFO [2022-01-10 08:39:55,290] jepsen worker 100 - jepsen.util 2100 :invoke :add 9260 INFO [2022-01-10 08:39:55,291] jepsen worker 638 - jepsen.util 1638 :info :add 7943 :RequestTimeoutException INFO [2022-01-10 08:39:55,292] jepsen worker 715 - jepsen.util 715 :info :add 7940 :RequestTimeoutException INFO [2022-01-10 08:39:55,293] jepsen worker 9 - jepsen.util 1009 :info :add 7946 :RequestTimeoutException INFO [2022-01-10 08:39:55,293] jepsen worker 807 - jepsen.util 1807 :invoke :add 9261 INFO [2022-01-10 08:39:55,293] jepsen worker 260 - jepsen.util 1260 :info :add 7941 :RequestTimeoutException INFO [2022-01-10 08:39:55,297] jepsen worker 715 - jepsen.util 1715 :invoke :add 9262 INFO [2022-01-10 08:39:55,299] jepsen worker 807 - jepsen.util 1807 :ok :add 9261 INFO [2022-01-10 08:39:55,299] jepsen worker 100 - jepsen.util 2100 :ok :add 9260 INFO [2022-01-10 08:39:55,299] jepsen worker 264 - jepsen.util 2264 :invoke :add 9263 INFO [2022-01-10 08:39:55,300] jepsen worker 40 - jepsen.util 1040 :info :add 7945 :RequestTimeoutException INFO [2022-01-10 08:39:55,302] jepsen worker 31 - jepsen.util 1031 :info :add 7947 :RequestTimeoutException INFO [2022-01-10 08:39:55,303] jepsen worker 5 - jepsen.util 5 :info :add 7948 :RequestTimeoutException INFO [2022-01-10 08:39:55,303] jepsen worker 965 - jepsen.util 1965 :invoke :add 9264 INFO [2022-01-10 08:39:55,306] jepsen worker 510 - jepsen.util 2510 :invoke :add 9265 INFO [2022-01-10 08:39:55,307] jepsen worker 742 - jepsen.util 1742 :info :add 7949 :RequestTimeoutException INFO [2022-01-10 08:39:55,308] jepsen worker 260 - jepsen.util 2260 :invoke :add 9266 INFO [2022-01-10 08:39:55,310] jepsen worker 384 - jepsen.util 1384 :info :add 7950 :RequestTimeoutException INFO [2022-01-10 08:39:55,310] jepsen worker 715 - jepsen.util 1715 :ok :add 9262 INFO [2022-01-10 08:39:55,310] jepsen worker 264 - jepsen.util 2264 :ok :add 9263 INFO [2022-01-10 08:39:55,311] jepsen worker 31 - jepsen.util 2031 :invoke :add 9267 INFO [2022-01-10 08:39:55,313] jepsen worker 543 - jepsen.util 1543 :info :add 7951 :RequestTimeoutException INFO [2022-01-10 08:39:55,314] jepsen worker 384 - jepsen.util 2384 :invoke :add 9268 INFO [2022-01-10 08:39:55,316] jepsen worker 346 - jepsen.util 1346 :invoke :add 9269 INFO [2022-01-10 08:39:55,319] jepsen worker 169 - jepsen.util 1169 :invoke :add 9270 INFO [2022-01-10 08:39:55,320] jepsen worker 956 - jepsen.util 1956 :info :add 7952 :RequestTimeoutException INFO [2022-01-10 08:39:55,321] jepsen worker 283 - jepsen.util 2283 :invoke :add 9271 INFO [2022-01-10 08:39:55,323] jepsen worker 57 - jepsen.util 1057 :info :add 7953 :RequestTimeoutException INFO [2022-01-10 08:39:55,324] jepsen worker 251 - jepsen.util 2251 :invoke :add 9272 INFO [2022-01-10 08:39:55,326] jepsen worker 807 - jepsen.util 1807 :invoke :add 9273 INFO [2022-01-10 08:39:55,328] jepsen worker 40 - jepsen.util 2040 :invoke :add 9274 INFO [2022-01-10 08:39:55,330] jepsen worker 5 - jepsen.util 1005 :invoke :add 9275 INFO [2022-01-10 08:39:55,332] jepsen worker 367 - jepsen.util 1367 :invoke :add 9276 INFO [2022-01-10 08:39:55,333] jepsen worker 638 - jepsen.util 2638 :invoke :add 9277 INFO [2022-01-10 08:39:55,335] jepsen worker 543 - jepsen.util 2543 :invoke :add 9278 INFO [2022-01-10 08:39:55,337] jepsen worker 891 - jepsen.util 1891 :invoke :add 9279 INFO [2022-01-10 08:39:55,339] jepsen worker 264 - jepsen.util 2264 :invoke :add 9280 INFO [2022-01-10 08:39:55,342] jepsen worker 57 - jepsen.util 2057 :invoke :add 9281 INFO [2022-01-10 08:39:55,343] jepsen worker 543 - jepsen.util 2543 :ok :add 9278 INFO [2022-01-10 08:39:55,344] jepsen worker 438 - jepsen.util 2438 :invoke :add 9282 INFO [2022-01-10 08:39:55,347] jepsen worker 37 - jepsen.util 1037 :invoke :add 9283 INFO [2022-01-10 08:39:55,347] jepsen worker 874 - jepsen.util 1874 :info :add 7958 :RequestTimeoutException INFO [2022-01-10 08:39:55,348] jepsen worker 891 - jepsen.util 1891 :ok :add 9279 INFO [2022-01-10 08:39:55,350] jepsen worker 9 - jepsen.util 2009 :invoke :add 9284 INFO [2022-01-10 08:39:55,352] jepsen worker 549 - jepsen.util 549 :info :add 7959 :RequestTimeoutException INFO [2022-01-10 08:39:55,352] jepsen worker 715 - jepsen.util 1715 :invoke :add 9285 INFO [2022-01-10 08:39:55,353] jepsen worker 264 - jepsen.util 2264 :ok :add 9280 INFO [2022-01-10 08:39:55,353] jepsen worker 57 - jepsen.util 2057 :ok :add 9281 INFO [2022-01-10 08:39:55,356] jepsen worker 37 - jepsen.util 1037 :ok :add 9283 INFO [2022-01-10 08:39:55,358] jepsen worker 543 - jepsen.util 2543 :invoke :add 9286 INFO [2022-01-10 08:39:55,360] jepsen worker 88 - jepsen.util 88 :info :add 7964 :RequestTimeoutException INFO [2022-01-10 08:39:55,361] jepsen worker 264 - jepsen.util 2264 :invoke :add 9287 INFO [2022-01-10 08:39:55,362] jepsen worker 438 - jepsen.util 2438 :ok :add 9282 INFO [2022-01-10 08:39:55,366] jepsen worker 37 - jepsen.util 1037 :invoke :add 9288 INFO [2022-01-10 08:39:55,367] jepsen worker 561 - jepsen.util 561 :info :add 7965 :RequestTimeoutException INFO [2022-01-10 08:39:55,369] jepsen worker 57 - jepsen.util 2057 :invoke :add 9289 INFO [2022-01-10 08:39:55,371] jepsen worker 565 - jepsen.util 565 :info :add 7967 :RequestTimeoutException INFO [2022-01-10 08:39:55,372] jepsen worker 72 - jepsen.util 1072 :info :add 7969 :RequestTimeoutException INFO [2022-01-10 08:39:55,372] jepsen worker 97 - jepsen.util 1097 :info :add 7970 :RequestTimeoutException INFO [2022-01-10 08:39:55,373] jepsen worker 463 - jepsen.util 1463 :info :add 7966 :RequestTimeoutException INFO [2022-01-10 08:39:55,373] jepsen worker 20 - jepsen.util 1020 :info :add 7968 :RequestTimeoutException INFO [2022-01-10 08:39:55,373] jepsen worker 438 - jepsen.util 2438 :invoke :add 9290 INFO [2022-01-10 08:39:55,377] jepsen worker 107 - jepsen.util 1107 :info :add 7971 :RequestTimeoutException INFO [2022-01-10 08:39:55,383] jepsen worker 611 - jepsen.util 1611 :info :add 7972 :RequestTimeoutException INFO [2022-01-10 08:39:55,384] jepsen worker 343 - jepsen.util 1343 :info :add 7975 :RequestTimeoutException INFO [2022-01-10 08:39:55,384] jepsen worker 561 - jepsen.util 1561 :invoke :add 9291 INFO [2022-01-10 08:39:55,384] jepsen worker 298 - jepsen.util 1298 :info :add 7974 :RequestTimeoutException INFO [2022-01-10 08:39:55,384] jepsen worker 541 - jepsen.util 1541 :info :add 7976 :RequestTimeoutException INFO [2022-01-10 08:39:55,385] jepsen worker 459 - jepsen.util 1459 :ok :add 8352 INFO [2022-01-10 08:39:55,386] jepsen worker 72 - jepsen.util 2072 :invoke :add 9292 INFO [2022-01-10 08:39:55,388] jepsen worker 141 - jepsen.util 1141 :info :add 7973 :RequestTimeoutException INFO [2022-01-10 08:39:55,390] jepsen worker 821 - jepsen.util 821 :info :add 7977 :RequestTimeoutException INFO [2022-01-10 08:39:55,396] jepsen worker 100 - jepsen.util 2100 :invoke :add 9293 INFO [2022-01-10 08:39:55,398] jepsen worker 541 - jepsen.util 2541 :invoke :add 9294 INFO [2022-01-10 08:39:55,401] jepsen worker 82 - jepsen.util 2082 :invoke :add 9295 INFO [2022-01-10 08:39:55,402] jepsen worker 753 - jepsen.util 1753 :info :add 7985 :RequestTimeoutException INFO [2022-01-10 08:39:55,403] jepsen worker 141 - jepsen.util 2141 :invoke :add 9296 INFO [2022-01-10 08:39:55,407] jepsen worker 157 - jepsen.util 157 :info :add 7984 :RequestTimeoutException INFO [2022-01-10 08:39:55,408] jepsen worker 611 - jepsen.util 2611 :invoke :add 9297 INFO [2022-01-10 08:39:55,411] jepsen worker 46 - jepsen.util 1046 :info :add 7986 :RequestTimeoutException INFO [2022-01-10 08:39:55,412] jepsen worker 874 - jepsen.util 2874 :invoke :add 9298 INFO [2022-01-10 08:39:55,412] jepsen worker 731 - jepsen.util 1731 :info :add 7987 :RequestTimeoutException INFO [2022-01-10 08:39:55,412] jepsen worker 812 - jepsen.util 1812 :info :add 7989 :RequestTimeoutException INFO [2022-01-10 08:39:55,413] jepsen worker 521 - jepsen.util 521 :info :add 7988 :RequestTimeoutException INFO [2022-01-10 08:39:55,413] jepsen worker 974 - jepsen.util 1974 :info :add 7991 :RequestTimeoutException INFO [2022-01-10 08:39:55,417] jepsen worker 755 - jepsen.util 755 :info :add 7994 :RequestTimeoutException INFO [2022-01-10 08:39:55,419] jepsen worker 874 - jepsen.util 2874 :ok :add 9298 INFO [2022-01-10 08:39:55,422] jepsen worker 229 - jepsen.util 1229 :info :add 7992 :RequestTimeoutException INFO [2022-01-10 08:39:55,423] jepsen worker 261 - jepsen.util 1261 :info :add 7995 :RequestTimeoutException INFO [2022-01-10 08:39:55,425] jepsen worker 891 - jepsen.util 1891 :invoke :add 9299 INFO [2022-01-10 08:39:55,432] jepsen worker 972 - jepsen.util 1972 :info :add 7996 :RequestTimeoutException INFO [2022-01-10 08:39:55,432] jepsen worker 211 - jepsen.util 211 :info :add 8000 :RequestTimeoutException INFO [2022-01-10 08:39:55,433] jepsen worker 375 - jepsen.util 1375 :info :add 8001 :RequestTimeoutException INFO [2022-01-10 08:39:55,433] jepsen worker 875 - jepsen.util 1875 :info :add 7997 :RequestTimeoutException INFO [2022-01-10 08:39:55,433] jepsen worker 107 - jepsen.util 2107 :invoke :add 9300 INFO [2022-01-10 08:39:55,436] jepsen worker 891 - jepsen.util 1891 :ok :add 9299 INFO [2022-01-10 08:39:55,437] jepsen worker 554 - jepsen.util 1554 :info :add 8002 :RequestTimeoutException INFO [2022-01-10 08:39:55,440] jepsen worker 45 - jepsen.util 1045 :info :add 8003 :RequestTimeoutException INFO [2022-01-10 08:39:55,441] jepsen worker 972 - jepsen.util 2972 :invoke :add 9301 INFO [2022-01-10 08:39:55,445] jepsen worker 812 - jepsen.util 2812 :invoke :add 9302 INFO [2022-01-10 08:39:55,448] jepsen worker 157 - jepsen.util 1157 :invoke :add 9303 INFO [2022-01-10 08:39:55,450] jepsen worker 658 - jepsen.util 658 :info :add 8008 :RequestTimeoutException INFO [2022-01-10 08:39:55,450] jepsen worker 671 - jepsen.util 1671 :info :add 8009 :RequestTimeoutException INFO [2022-01-10 08:39:55,450] jepsen worker 459 - jepsen.util 1459 :invoke :add 9304 INFO [2022-01-10 08:39:55,451] jepsen worker 228 - jepsen.util 1228 :info :add 8011 :RequestTimeoutException INFO [2022-01-10 08:39:55,453] jepsen worker 632 - jepsen.util 1632 :info :add 8010 :RequestTimeoutException INFO [2022-01-10 08:39:55,458] jepsen worker 565 - jepsen.util 1565 :invoke :add 9305 INFO [2022-01-10 08:39:55,460] jepsen worker 88 - jepsen.util 1088 :invoke :add 9306 INFO [2022-01-10 08:39:55,462] jepsen worker 769 - jepsen.util 1769 :info :add 8012 :RequestTimeoutException INFO [2022-01-10 08:39:55,463] jepsen worker 397 - jepsen.util 1397 :info :add 8013 :RequestTimeoutException INFO [2022-01-10 08:39:55,463] jepsen worker 211 - jepsen.util 1211 :invoke :add 9307 INFO [2022-01-10 08:39:55,468] jepsen worker 874 - jepsen.util 2874 :invoke :add 9308 INFO [2022-01-10 08:39:55,472] jepsen worker 228 - jepsen.util 2228 :invoke :add 9309 INFO [2022-01-10 08:39:55,472] jepsen worker 425 - jepsen.util 1425 :info :add 8014 :RequestTimeoutException INFO [2022-01-10 08:39:55,473] jepsen worker 652 - jepsen.util 1652 :info :add 8015 :RequestTimeoutException INFO [2022-01-10 08:39:55,473] jepsen worker 856 - jepsen.util 1856 :info :add 8016 :RequestTimeoutException INFO [2022-01-10 08:39:55,473] jepsen worker 429 - jepsen.util 1429 :info :add 8017 :RequestTimeoutException INFO [2022-01-10 08:39:55,475] jepsen worker 343 - jepsen.util 2343 :invoke :add 9310 INFO [2022-01-10 08:39:55,478] jepsen worker 874 - jepsen.util 2874 :ok :add 9308 INFO [2022-01-10 08:39:55,479] jepsen worker 228 - jepsen.util 2228 :ok :add 9309 INFO [2022-01-10 08:39:55,482] jepsen worker 669 - jepsen.util 669 :info :add 8018 :RequestTimeoutException INFO [2022-01-10 08:39:55,483] jepsen worker 343 - jepsen.util 2343 :ok :add 9310 INFO [2022-01-10 08:39:55,484] jepsen worker 753 - jepsen.util 2753 :invoke :add 9311 INFO [2022-01-10 08:39:55,487] jepsen worker 229 - jepsen.util 2229 :invoke :add 9312 INFO [2022-01-10 08:39:55,488] jepsen worker 753 - jepsen.util 2753 :ok :add 9311 INFO [2022-01-10 08:39:55,491] jepsen worker 20 - jepsen.util 2020 :invoke :add 9313 INFO [2022-01-10 08:39:55,494] jepsen worker 554 - jepsen.util 2554 :invoke :add 9314 INFO [2022-01-10 08:39:55,497] jepsen worker 669 - jepsen.util 1669 :invoke :add 9315 INFO [2022-01-10 08:39:55,500] jepsen worker 97 - jepsen.util 2097 :invoke :add 9316 INFO [2022-01-10 08:39:55,503] jepsen worker 228 - jepsen.util 2228 :invoke :add 9317 INFO [2022-01-10 08:39:55,505] jepsen worker 343 - jepsen.util 2343 :invoke :add 9318 INFO [2022-01-10 08:39:55,508] jepsen worker 298 - jepsen.util 2298 :invoke :add 9319 INFO [2022-01-10 08:39:55,511] jepsen worker 549 - jepsen.util 1549 :invoke :add 9320 INFO [2022-01-10 08:39:55,514] jepsen worker 658 - jepsen.util 1658 :invoke :add 9321 INFO [2022-01-10 08:39:55,516] jepsen worker 671 - jepsen.util 2671 :invoke :add 9322 INFO [2022-01-10 08:39:55,519] jepsen worker 874 - jepsen.util 2874 :invoke :add 9323 INFO [2022-01-10 08:39:55,521] jepsen worker 637 - jepsen.util 1637 :info :add 8019 :RequestTimeoutException INFO [2022-01-10 08:39:55,522] jepsen worker 632 - jepsen.util 2632 :invoke :add 9324 INFO [2022-01-10 08:39:55,524] jepsen worker 809 - jepsen.util 809 :info :add 8020 :RequestTimeoutException INFO [2022-01-10 08:39:55,526] jepsen worker 463 - jepsen.util 2463 :invoke :add 9325 INFO [2022-01-10 08:39:55,531] jepsen worker 55 - jepsen.util 55 :info :add 8021 :RequestTimeoutException INFO [2022-01-10 08:39:55,531] jepsen worker 755 - jepsen.util 1755 :invoke :add 9326 INFO [2022-01-10 08:39:55,533] jepsen worker 487 - jepsen.util 487 :info :add 8023 :RequestTimeoutException INFO [2022-01-10 08:39:55,534] jepsen worker 242 - jepsen.util 1242 :info :add 8022 :RequestTimeoutException INFO [2022-01-10 08:39:55,535] jepsen worker 753 - jepsen.util 2753 :invoke :add 9327 INFO [2022-01-10 08:39:55,541] jepsen worker 856 - jepsen.util 2856 :invoke :add 9328 INFO [2022-01-10 08:39:55,543] jepsen worker 275 - jepsen.util 275 :info :add 8024 :RequestTimeoutException INFO [2022-01-10 08:39:55,543] jepsen worker 466 - jepsen.util 1466 :info :add 8026 :RequestTimeoutException INFO [2022-01-10 08:39:55,543] jepsen worker 213 - jepsen.util 213 :info :add 8027 :RequestTimeoutException INFO [2022-01-10 08:39:55,543] jepsen worker 138 - jepsen.util 1138 :info :add 8025 :RequestTimeoutException INFO [2022-01-10 08:39:55,544] jepsen worker 956 - jepsen.util 2956 :invoke :add 9329 INFO [2022-01-10 08:39:55,547] jepsen worker 413 - jepsen.util 1413 :ok :add 8157 INFO [2022-01-10 08:39:55,552] jepsen worker 229 - jepsen.util 2229 :ok :add 9312 INFO [2022-01-10 08:39:55,553] jepsen worker 487 - jepsen.util 1487 :invoke :add 9330 INFO [2022-01-10 08:39:55,555] jepsen worker 20 - jepsen.util 2020 :ok :add 9313 INFO [2022-01-10 08:39:55,555] jepsen worker 753 - jepsen.util 2753 :ok :add 9327 INFO [2022-01-10 08:39:55,557] jepsen worker 275 - jepsen.util 1275 :invoke :add 9331 INFO [2022-01-10 08:39:55,560] jepsen worker 463 - jepsen.util 2463 :ok :add 9325 INFO [2022-01-10 08:39:55,560] jepsen worker 632 - jepsen.util 2632 :ok :add 9324 INFO [2022-01-10 08:39:55,560] jepsen worker 213 - jepsen.util 1213 :invoke :add 9332 INFO [2022-01-10 08:39:55,563] jepsen worker 308 - jepsen.util 1308 :info :add 8034 :RequestTimeoutException INFO [2022-01-10 08:39:55,563] jepsen worker 957 - jepsen.util 1957 :info :add 8035 :RequestTimeoutException INFO [2022-01-10 08:39:55,563] jepsen worker 769 - jepsen.util 2769 :invoke :add 9333 INFO [2022-01-10 08:39:55,566] jepsen worker 755 - jepsen.util 1755 :ok :add 9326 INFO [2022-01-10 08:39:55,569] jepsen worker 397 - jepsen.util 2397 :invoke :add 9334 INFO [2022-01-10 08:39:55,570] jepsen worker 993 - jepsen.util 1993 :info :add 8037 :RequestTimeoutException INFO [2022-01-10 08:39:55,571] jepsen worker 98 - jepsen.util 98 :info :add 8036 :RequestTimeoutException INFO [2022-01-10 08:39:55,572] jepsen worker 872 - jepsen.util 872 :info :add 8039 :RequestTimeoutException INFO [2022-01-10 08:39:55,572] jepsen worker 521 - jepsen.util 1521 :invoke :add 9335 INFO [2022-01-10 08:39:55,573] jepsen worker 643 - jepsen.util 1643 :info :add 8038 :RequestTimeoutException INFO [2022-01-10 08:39:55,580] jepsen worker 875 - jepsen.util 2875 :invoke :add 9336 INFO [2022-01-10 08:39:55,584] jepsen worker 643 - jepsen.util 2643 :invoke :add 9337 INFO [2022-01-10 08:39:55,587] jepsen worker 755 - jepsen.util 1755 :invoke :add 9338 INFO [2022-01-10 08:39:55,589] jepsen worker 375 - jepsen.util 2375 :invoke :add 9339 INFO [2022-01-10 08:39:55,591] jepsen worker 601 - jepsen.util 1601 :info :add 8044 :RequestTimeoutException INFO [2022-01-10 08:39:55,592] jepsen worker 893 - jepsen.util 1893 :info :add 8046 :RequestTimeoutException INFO [2022-01-10 08:39:55,593] jepsen worker 102 - jepsen.util 102 :info :add 8047 :RequestTimeoutException INFO [2022-01-10 08:39:55,593] jepsen worker 489 - jepsen.util 1489 :info :add 8045 :RequestTimeoutException INFO [2022-01-10 08:39:55,593] jepsen worker 753 - jepsen.util 2753 :invoke :add 9340 INFO [2022-01-10 08:39:55,597] jepsen worker 527 - jepsen.util 1527 :info :add 8048 :RequestTimeoutException INFO [2022-01-10 08:39:55,601] jepsen worker 574 - jepsen.util 1574 :info :add 8049 :RequestTimeoutException INFO [2022-01-10 08:39:55,602] jepsen worker 957 - jepsen.util 2957 :invoke :add 9341 INFO [2022-01-10 08:39:55,603] jepsen worker 77 - jepsen.util 1077 :info :add 8050 :RequestTimeoutException INFO [2022-01-10 08:39:55,608] jepsen worker 731 - jepsen.util 2731 :invoke :add 9342 INFO [2022-01-10 08:39:55,610] jepsen worker 463 - jepsen.util 2463 :invoke :add 9343 INFO [2022-01-10 08:39:55,611] jepsen worker 544 - jepsen.util 1544 :info :add 8051 :RequestTimeoutException INFO [2022-01-10 08:39:55,612] jepsen worker 649 - jepsen.util 1649 :info :add 8054 :RequestTimeoutException INFO [2022-01-10 08:39:55,613] jepsen worker 466 - jepsen.util 2466 :invoke :add 9344 INFO [2022-01-10 08:39:55,613] jepsen worker 901 - jepsen.util 901 :info :add 8053 :RequestTimeoutException INFO [2022-01-10 08:39:55,614] jepsen worker 378 - jepsen.util 1378 :info :add 8052 :RequestTimeoutException INFO [2022-01-10 08:39:55,617] jepsen worker 760 - jepsen.util 1760 :info :add 8056 :RequestTimeoutException INFO [2022-01-10 08:39:55,620] jepsen worker 923 - jepsen.util 1923 :info :add 8057 :RequestTimeoutException INFO [2022-01-10 08:39:55,621] jepsen worker 77 - jepsen.util 2077 :invoke :add 9345 INFO [2022-01-10 08:39:55,622] jepsen worker 937 - jepsen.util 1937 :info :add 8055 :RequestTimeoutException INFO [2022-01-10 08:39:55,623] jepsen worker 294 - jepsen.util 1294 :info :add 8060 :RequestTimeoutException INFO [2022-01-10 08:39:55,628] jepsen worker 937 - jepsen.util 2937 :invoke :add 9346 INFO [2022-01-10 08:39:55,631] jepsen worker 83 - jepsen.util 1083 :info :add 8061 :RequestTimeoutException INFO [2022-01-10 08:39:55,632] jepsen worker 893 - jepsen.util 2893 :invoke :add 9347 INFO [2022-01-10 08:39:55,633] jepsen worker 285 - jepsen.util 1285 :info :add 8062 :RequestTimeoutException INFO [2022-01-10 08:39:55,635] jepsen worker 637 - jepsen.util 2637 :invoke :add 9348 INFO [2022-01-10 08:39:55,638] jepsen worker 452 - jepsen.util 1452 :info :add 8065 :RequestTimeoutException INFO [2022-01-10 08:39:55,639] jepsen worker 361 - jepsen.util 1361 :info :add 8063 :RequestTimeoutException INFO [2022-01-10 08:39:55,639] jepsen worker 315 - jepsen.util 1315 :info :add 8064 :RequestTimeoutException INFO [2022-01-10 08:39:55,640] jepsen worker 574 - jepsen.util 2574 :invoke :add 9349 INFO [2022-01-10 08:39:55,640] jepsen worker 345 - jepsen.util 1345 :info :add 8066 :RequestTimeoutException INFO [2022-01-10 08:39:55,641] jepsen worker 898 - jepsen.util 1898 :info :add 8068 :RequestTimeoutException INFO [2022-01-10 08:39:55,643] jepsen worker 983 - jepsen.util 1983 :info :add 8067 :RequestTimeoutException INFO [2022-01-10 08:39:55,644] jepsen worker 190 - jepsen.util 1190 :info :add 8070 :RequestTimeoutException INFO [2022-01-10 08:39:55,647] jepsen worker 943 - jepsen.util 1943 :info :add 8071 :RequestTimeoutException INFO [2022-01-10 08:39:55,652] jepsen worker 468 - jepsen.util 1468 :info :add 8072 :RequestTimeoutException INFO [2022-01-10 08:39:55,653] jepsen worker 943 - jepsen.util 2943 :invoke :add 9350 INFO [2022-01-10 08:39:55,653] jepsen worker 7 - jepsen.util 7 :info :add 8073 :RequestTimeoutException INFO [2022-01-10 08:39:55,657] jepsen worker 898 - jepsen.util 2898 :invoke :add 9351 INFO [2022-01-10 08:39:55,659] jepsen worker 345 - jepsen.util 2345 :invoke :add 9352 INFO [2022-01-10 08:39:55,662] jepsen worker 760 - jepsen.util 2760 :invoke :add 9353 INFO [2022-01-10 08:39:55,664] jepsen worker 20 - jepsen.util 2020 :invoke :add 9354 INFO [2022-01-10 08:39:55,668] jepsen worker 742 - jepsen.util 2742 :invoke :add 9355 INFO [2022-01-10 08:39:55,670] jepsen worker 468 - jepsen.util 2468 :invoke :add 9356 INFO [2022-01-10 08:39:55,671] jepsen worker 20 - jepsen.util 2020 :ok :add 9354 INFO [2022-01-10 08:39:55,673] jepsen worker 452 - jepsen.util 2452 :invoke :add 9357 INFO [2022-01-10 08:39:55,681] jepsen worker 452 - jepsen.util 2452 :ok :add 9357 INFO [2022-01-10 08:39:55,681] jepsen worker 468 - jepsen.util 2468 :ok :add 9356 INFO [2022-01-10 08:39:55,681] jepsen worker 742 - jepsen.util 2742 :ok :add 9355 INFO [2022-01-10 08:39:55,683] jepsen worker 489 - jepsen.util 2489 :invoke :add 9358 INFO [2022-01-10 08:39:55,687] jepsen worker 132 - jepsen.util 132 :info :add 8079 :RequestTimeoutException INFO [2022-01-10 08:39:55,687] jepsen worker 606 - jepsen.util 606 :info :add 8078 :RequestTimeoutException INFO [2022-01-10 08:39:55,688] jepsen worker 98 - jepsen.util 1098 :invoke :add 9359 INFO [2022-01-10 08:39:55,692] jepsen worker 247 - jepsen.util 247 :info :add 8080 :RequestTimeoutException INFO [2022-01-10 08:39:55,693] jepsen worker 684 - jepsen.util 1684 :info :add 8081 :RequestTimeoutException INFO [2022-01-10 08:39:55,693] jepsen worker 557 - jepsen.util 1557 :info :add 8083 :RequestTimeoutException INFO [2022-01-10 08:39:55,693] jepsen worker 844 - jepsen.util 1844 :info :add 8084 :RequestTimeoutException INFO [2022-01-10 08:39:55,693] jepsen worker 697 - jepsen.util 1697 :info :add 8082 :RequestTimeoutException INFO [2022-01-10 08:39:55,695] jepsen worker 315 - jepsen.util 2315 :invoke :add 9360 INFO [2022-01-10 08:39:55,702] jepsen worker 830 - jepsen.util 1830 :info :add 8085 :RequestTimeoutException INFO [2022-01-10 08:39:55,703] jepsen worker 315 - jepsen.util 2315 :ok :add 9360 INFO [2022-01-10 08:39:55,703] jepsen worker 553 - jepsen.util 1553 :info :add 8086 :RequestTimeoutException INFO [2022-01-10 08:39:55,704] jepsen worker 821 - jepsen.util 1821 :invoke :add 9361 INFO [2022-01-10 08:39:55,709] jepsen worker 229 - jepsen.util 2229 :invoke :add 9362 INFO [2022-01-10 08:39:55,711] jepsen worker 901 - jepsen.util 1901 :invoke :add 9363 INFO [2022-01-10 08:39:55,713] jepsen worker 575 - jepsen.util 1575 :info :add 8087 :RequestTimeoutException INFO [2022-01-10 08:39:55,714] jepsen worker 821 - jepsen.util 1821 :ok :add 9361 INFO [2022-01-10 08:39:55,715] jepsen worker 285 - jepsen.util 2285 :invoke :add 9364 INFO [2022-01-10 08:39:55,718] jepsen worker 229 - jepsen.util 2229 :ok :add 9362 INFO [2022-01-10 08:39:55,720] jepsen worker 575 - jepsen.util 2575 :invoke :add 9365 INFO [2022-01-10 08:39:55,723] jepsen worker 974 - jepsen.util 2974 :invoke :add 9366 INFO [2022-01-10 08:39:55,724] jepsen worker 901 - jepsen.util 1901 :ok :add 9363 INFO [2022-01-10 08:39:55,725] jepsen worker 557 - jepsen.util 2557 :invoke :add 9367 INFO [2022-01-10 08:39:55,728] jepsen worker 652 - jepsen.util 2652 :invoke :add 9368 INFO [2022-01-10 08:39:55,731] jepsen worker 294 - jepsen.util 2294 :invoke :add 9369 INFO [2022-01-10 08:39:55,733] jepsen worker 7 - jepsen.util 1007 :invoke :add 9370 INFO [2022-01-10 08:39:55,736] jepsen worker 844 - jepsen.util 2844 :invoke :add 9371 INFO [2022-01-10 08:39:55,739] jepsen worker 425 - jepsen.util 2425 :invoke :add 9372 INFO [2022-01-10 08:39:55,742] jepsen worker 413 - jepsen.util 1413 :invoke :add 9373 INFO [2022-01-10 08:39:55,745] jepsen worker 83 - jepsen.util 2083 :invoke :add 9374 INFO [2022-01-10 08:39:55,748] jepsen worker 993 - jepsen.util 2993 :invoke :add 9375 INFO [2022-01-10 08:39:55,750] jepsen worker 46 - jepsen.util 2046 :invoke :add 9376 INFO [2022-01-10 08:39:55,753] jepsen worker 527 - jepsen.util 2527 :invoke :add 9377 INFO [2022-01-10 08:39:55,755] jepsen worker 308 - jepsen.util 2308 :invoke :add 9378 INFO [2022-01-10 08:39:55,759] jepsen worker 809 - jepsen.util 1809 :invoke :add 9379 INFO [2022-01-10 08:39:55,762] jepsen worker 544 - jepsen.util 2544 :invoke :add 9380 INFO [2022-01-10 08:39:55,764] jepsen worker 308 - jepsen.util 2308 :ok :add 9378 INFO [2022-01-10 08:39:55,765] jepsen worker 872 - jepsen.util 1872 :invoke :add 9381 INFO [2022-01-10 08:39:55,768] jepsen worker 809 - jepsen.util 1809 :ok :add 9379 INFO [2022-01-10 08:39:55,769] jepsen worker 190 - jepsen.util 2190 :invoke :add 9382 INFO [2022-01-10 08:39:55,773] jepsen worker 315 - jepsen.util 2315 :invoke :add 9383 INFO [2022-01-10 08:39:55,773] jepsen worker 357 - jepsen.util 357 :info :add 8088 :RequestTimeoutException INFO [2022-01-10 08:39:55,774] jepsen worker 872 - jepsen.util 1872 :ok :add 9381 INFO [2022-01-10 08:39:55,774] jepsen worker 544 - jepsen.util 2544 :ok :add 9380 INFO [2022-01-10 08:39:55,775] jepsen worker 247 - jepsen.util 1247 :invoke :add 9384 INFO [2022-01-10 08:39:55,780] jepsen worker 132 - jepsen.util 1132 :invoke :add 9385 INFO [2022-01-10 08:39:55,783] jepsen worker 315 - jepsen.util 2315 :ok :add 9383 INFO [2022-01-10 08:39:55,783] jepsen worker 553 - jepsen.util 2553 :invoke :add 9386 INFO [2022-01-10 08:39:55,783] jepsen worker 190 - jepsen.util 2190 :ok :add 9382 INFO [2022-01-10 08:39:55,787] jepsen worker 872 - jepsen.util 1872 :invoke :add 9387 INFO [2022-01-10 08:39:55,789] jepsen worker 649 - jepsen.util 2649 :invoke :add 9388 INFO [2022-01-10 08:39:55,792] jepsen worker 697 - jepsen.util 2697 :invoke :add 9389 INFO [2022-01-10 08:39:55,793] jepsen worker 771 - jepsen.util 1771 :info :add 8089 :RequestTimeoutException INFO [2022-01-10 08:39:55,793] jepsen worker 326 - jepsen.util 1326 :info :add 8090 :RequestTimeoutException INFO [2022-01-10 08:39:55,795] jepsen worker 632 - jepsen.util 2632 :invoke :add 9390 INFO [2022-01-10 08:39:55,801] jepsen worker 601 - jepsen.util 2601 :invoke :add 9391 INFO [2022-01-10 08:39:55,804] jepsen worker 771 - jepsen.util 2771 :invoke :add 9392 INFO [2022-01-10 08:39:55,807] jepsen worker 20 - jepsen.util 2020 :invoke :add 9393 INFO [2022-01-10 08:39:55,812] jepsen worker 923 - jepsen.util 2923 :invoke :add 9394 INFO [2022-01-10 08:39:55,812] jepsen worker 462 - jepsen.util 462 :ok :add 8493 INFO [2022-01-10 08:39:55,816] jepsen worker 684 - jepsen.util 2684 :invoke :add 9395 INFO [2022-01-10 08:39:55,820] jepsen worker 326 - jepsen.util 2326 :invoke :add 9396 INFO [2022-01-10 08:39:55,823] jepsen worker 468 - jepsen.util 2468 :invoke :add 9397 INFO [2022-01-10 08:39:55,825] jepsen worker 830 - jepsen.util 2830 :invoke :add 9398 INFO [2022-01-10 08:39:55,828] jepsen worker 261 - jepsen.util 2261 :invoke :add 9399 INFO [2022-01-10 08:39:55,830] jepsen worker 229 - jepsen.util 2229 :invoke :add 9400 INFO [2022-01-10 08:39:55,831] jepsen worker 449 - jepsen.util 1449 :info :add 8091 :RequestTimeoutException INFO [2022-01-10 08:39:55,833] jepsen worker 452 - jepsen.util 2452 :invoke :add 9401 INFO [2022-01-10 08:39:55,834] jepsen worker 261 - jepsen.util 2261 :ok :add 9399 INFO [2022-01-10 08:39:55,838] jepsen worker 544 - jepsen.util 2544 :invoke :add 9402 INFO [2022-01-10 08:39:55,839] jepsen worker 830 - jepsen.util 2830 :ok :add 9398 INFO [2022-01-10 08:39:55,841] jepsen worker 983 - jepsen.util 2983 :invoke :add 9403 INFO [2022-01-10 08:39:55,843] jepsen worker 171 - jepsen.util 1171 :info :add 8093 :RequestTimeoutException INFO [2022-01-10 08:39:55,843] jepsen worker 417 - jepsen.util 1417 :info :add 8092 :RequestTimeoutException INFO [2022-01-10 08:39:55,844] jepsen worker 315 - jepsen.util 2315 :invoke :add 9404 INFO [2022-01-10 08:39:55,849] jepsen worker 821 - jepsen.util 1821 :invoke :add 9405 INFO [2022-01-10 08:39:55,852] jepsen worker 45 - jepsen.util 2045 :invoke :add 9406 INFO [2022-01-10 08:39:55,855] jepsen worker 901 - jepsen.util 1901 :invoke :add 9407 INFO [2022-01-10 08:39:55,858] jepsen worker 55 - jepsen.util 1055 :invoke :add 9408 INFO [2022-01-10 08:39:55,862] jepsen worker 606 - jepsen.util 1606 :invoke :add 9409 INFO [2022-01-10 08:39:55,862] jepsen worker 738 - jepsen.util 1738 :info :add 8099 :RequestTimeoutException INFO [2022-01-10 08:39:55,863] jepsen worker 166 - jepsen.util 1166 :info :add 8098 :RequestTimeoutException INFO [2022-01-10 08:39:55,865] jepsen worker 429 - jepsen.util 2429 :invoke :add 9410 INFO [2022-01-10 08:39:55,866] jepsen worker 55 - jepsen.util 1055 :ok :add 9408 INFO [2022-01-10 08:39:55,868] jepsen worker 606 - jepsen.util 1606 :ok :add 9409 INFO [2022-01-10 08:39:55,872] jepsen worker 102 - jepsen.util 1102 :invoke :add 9411 INFO [2022-01-10 08:39:55,875] jepsen worker 171 - jepsen.util 2171 :invoke :add 9412 INFO [2022-01-10 08:39:55,875] jepsen worker 429 - jepsen.util 2429 :ok :add 9410 INFO [2022-01-10 08:39:55,877] jepsen worker 71 - jepsen.util 1071 :info :add 8100 :RequestTimeoutException INFO [2022-01-10 08:39:55,878] jepsen worker 830 - jepsen.util 2830 :invoke :add 9413 INFO [2022-01-10 08:39:55,880] jepsen worker 144 - jepsen.util 1144 :info :add 8101 :RequestTimeoutException INFO [2022-01-10 08:39:55,881] jepsen worker 913 - jepsen.util 1913 :info :add 8103 :RequestTimeoutException INFO [2022-01-10 08:39:55,883] jepsen worker 546 - jepsen.util 1546 :info :add 8102 :RequestTimeoutException INFO [2022-01-10 08:39:55,883] jepsen worker 738 - jepsen.util 2738 :invoke :add 9414 INFO [2022-01-10 08:39:55,884] jepsen worker 102 - jepsen.util 1102 :ok :add 9411 INFO [2022-01-10 08:39:55,888] jepsen worker 171 - jepsen.util 2171 :ok :add 9412 INFO [2022-01-10 08:39:55,888] jepsen worker 830 - jepsen.util 2830 :ok :add 9413 INFO [2022-01-10 08:39:55,891] jepsen worker 809 - jepsen.util 1809 :invoke :add 9415 INFO [2022-01-10 08:39:55,897] jepsen worker 71 - jepsen.util 2071 :invoke :add 9416 INFO [2022-01-10 08:39:55,900] jepsen worker 606 - jepsen.util 1606 :invoke :add 9417 INFO [2022-01-10 08:39:55,903] jepsen worker 138 - jepsen.util 2138 :invoke :add 9418 INFO [2022-01-10 08:39:55,906] jepsen worker 166 - jepsen.util 2166 :invoke :add 9419 INFO [2022-01-10 08:39:55,908] jepsen worker 55 - jepsen.util 1055 :invoke :add 9420 INFO [2022-01-10 08:39:55,911] jepsen worker 581 - jepsen.util 1581 :info :add 8108 :RequestTimeoutException INFO [2022-01-10 08:39:55,911] jepsen worker 144 - jepsen.util 2144 :invoke :add 9421 INFO [2022-01-10 08:39:55,912] jepsen worker 862 - jepsen.util 1862 :info :add 8109 :RequestTimeoutException INFO [2022-01-10 08:39:55,913] jepsen worker 446 - jepsen.util 1446 :info :add 8110 :RequestTimeoutException INFO [2022-01-10 08:39:55,918] jepsen worker 102 - jepsen.util 1102 :invoke :add 9422 INFO [2022-01-10 08:39:55,921] jepsen worker 242 - jepsen.util 2242 :invoke :add 9423 INFO [2022-01-10 08:39:55,924] jepsen worker 546 - jepsen.util 2546 :invoke :add 9424 INFO [2022-01-10 08:39:55,927] jepsen worker 891 - jepsen.util 1891 :invoke :add 9425 INFO [2022-01-10 08:39:55,930] jepsen worker 190 - jepsen.util 2190 :invoke :add 9426 INFO [2022-01-10 08:39:55,933] jepsen worker 913 - jepsen.util 2913 :invoke :add 9427 INFO [2022-01-10 08:39:55,935] jepsen worker 891 - jepsen.util 1891 :ok :add 9425 INFO [2022-01-10 08:39:55,935] jepsen worker 546 - jepsen.util 2546 :ok :add 9424 INFO [2022-01-10 08:39:55,936] jepsen worker 742 - jepsen.util 2742 :invoke :add 9428 INFO [2022-01-10 08:39:55,940] jepsen worker 190 - jepsen.util 2190 :ok :add 9426 INFO [2022-01-10 08:39:55,940] jepsen worker 171 - jepsen.util 2171 :invoke :add 9429 INFO [2022-01-10 08:39:55,944] jepsen worker 462 - jepsen.util 462 :invoke :add 9430 INFO [2022-01-10 08:39:55,944] jepsen worker 913 - jepsen.util 2913 :ok :add 9427 INFO [2022-01-10 08:39:55,946] jepsen worker 261 - jepsen.util 2261 :invoke :add 9431 INFO [2022-01-10 08:39:55,949] jepsen worker 361 - jepsen.util 2361 :invoke :add 9432 INFO [2022-01-10 08:39:55,952] jepsen worker 417 - jepsen.util 2417 :invoke :add 9433 INFO [2022-01-10 08:39:55,955] jepsen worker 446 - jepsen.util 2446 :invoke :add 9434 INFO [2022-01-10 08:39:55,958] jepsen worker 913 - jepsen.util 2913 :invoke :add 9435 INFO [2022-01-10 08:39:55,960] jepsen worker 378 - jepsen.util 2378 :invoke :add 9436 INFO [2022-01-10 08:39:55,963] jepsen worker 190 - jepsen.util 2190 :invoke :add 9437 INFO [2022-01-10 08:39:55,965] jepsen worker 429 - jepsen.util 2429 :invoke :add 9438 INFO [2022-01-10 08:39:55,968] jepsen worker 357 - jepsen.util 1357 :invoke :add 9439 INFO [2022-01-10 08:39:55,971] jepsen worker 308 - jepsen.util 2308 :invoke :add 9440 INFO [2022-01-10 08:39:55,973] jepsen worker 862 - jepsen.util 2862 :invoke :add 9441 INFO [2022-01-10 08:39:55,976] jepsen worker 449 - jepsen.util 2449 :invoke :add 9442 INFO [2022-01-10 08:39:55,979] jepsen worker 581 - jepsen.util 2581 :invoke :add 9443 INFO [2022-01-10 08:39:55,981] jepsen worker 546 - jepsen.util 2546 :invoke :add 9444 INFO [2022-01-10 08:39:55,984] jepsen worker 830 - jepsen.util 2830 :invoke :add 9445 INFO [2022-01-10 08:39:55,986] jepsen worker 891 - jepsen.util 1891 :invoke :add 9446 INFO [2022-01-10 08:39:56,043] jepsen worker 610 - jepsen.util 1610 :info :add 8113 :RequestTimeoutException INFO [2022-01-10 08:39:56,043] jepsen worker 231 - jepsen.util 1231 :info :add 8111 :RequestTimeoutException INFO [2022-01-10 08:39:56,049] jepsen worker 610 - jepsen.util 2610 :invoke :add 9447 INFO [2022-01-10 08:39:56,051] jepsen worker 585 - jepsen.util 1585 :info :add 8112 :RequestTimeoutException INFO [2022-01-10 08:39:56,052] jepsen worker 231 - jepsen.util 2231 :invoke :add 9448 INFO [2022-01-10 08:39:56,056] jepsen worker 585 - jepsen.util 2585 :invoke :add 9449 INFO [2022-01-10 08:39:56,062] jepsen worker 226 - jepsen.util 1226 :info :add 8114 :RequestTimeoutException INFO [2022-01-10 08:39:56,065] jepsen worker 226 - jepsen.util 2226 :invoke :add 9450 INFO [2022-01-10 08:39:56,070] jepsen worker 945 - jepsen.util 1945 :info :add 8116 :RequestTimeoutException INFO [2022-01-10 08:39:56,071] jepsen worker 121 - jepsen.util 1121 :info :add 8118 :RequestTimeoutException INFO [2022-01-10 08:39:56,073] jepsen worker 826 - jepsen.util 1826 :info :add 8117 :RequestTimeoutException INFO [2022-01-10 08:39:56,074] jepsen worker 430 - jepsen.util 1430 :info :add 8119 :RequestTimeoutException INFO [2022-01-10 08:39:56,078] jepsen worker 121 - jepsen.util 2121 :invoke :add 9451 INFO [2022-01-10 08:39:56,081] jepsen worker 945 - jepsen.util 2945 :invoke :add 9452 INFO [2022-01-10 08:39:56,082] jepsen worker 330 - jepsen.util 1330 :info :add 8120 :RequestTimeoutException INFO [2022-01-10 08:39:56,084] jepsen worker 826 - jepsen.util 2826 :invoke :add 9453 INFO [2022-01-10 08:39:56,088] jepsen worker 330 - jepsen.util 2330 :invoke :add 9454 INFO [2022-01-10 08:39:56,091] jepsen worker 430 - jepsen.util 2430 :invoke :add 9455 INFO [2022-01-10 08:39:56,095] jepsen worker 330 - jepsen.util 2330 :ok :add 9454 INFO [2022-01-10 08:39:56,097] jepsen worker 430 - jepsen.util 2430 :ok :add 9455 INFO [2022-01-10 08:39:56,100] jepsen worker 330 - jepsen.util 2330 :invoke :add 9456 INFO [2022-01-10 08:39:56,103] jepsen worker 430 - jepsen.util 2430 :invoke :add 9457 INFO [2022-01-10 08:39:56,106] jepsen worker 330 - jepsen.util 2330 :ok :add 9456 INFO [2022-01-10 08:39:56,108] jepsen worker 430 - jepsen.util 2430 :ok :add 9457 INFO [2022-01-10 08:39:56,109] jepsen worker 330 - jepsen.util 2330 :invoke :add 9458 INFO [2022-01-10 08:39:56,112] jepsen worker 430 - jepsen.util 2430 :invoke :add 9459 INFO [2022-01-10 08:39:56,221] jepsen worker 99 - jepsen.util 99 :info :add 8121 :RequestTimeoutException INFO [2022-01-10 08:39:56,223] jepsen worker 232 - jepsen.util 232 :info :add 8122 :RequestTimeoutException INFO [2022-01-10 08:39:56,228] jepsen worker 99 - jepsen.util 1099 :invoke :add 9460 INFO [2022-01-10 08:39:56,231] jepsen worker 232 - jepsen.util 1232 :invoke :add 9461 INFO [2022-01-10 08:39:56,233] jepsen worker 843 - jepsen.util 1843 :info :add 8123 :RequestTimeoutException INFO [2022-01-10 08:39:56,234] jepsen worker 85 - jepsen.util 85 :info :add 8124 :RequestTimeoutException INFO [2022-01-10 08:39:56,237] jepsen worker 547 - jepsen.util 1547 :info :add 8126 :RequestTimeoutException INFO [2022-01-10 08:39:56,237] jepsen worker 99 - jepsen.util 1099 :ok :add 9460 INFO [2022-01-10 08:39:56,240] jepsen worker 938 - jepsen.util 938 :info :add 8125 :RequestTimeoutException INFO [2022-01-10 08:39:56,240] jepsen worker 291 - jepsen.util 1291 :info :add 8127 :RequestTimeoutException INFO [2022-01-10 08:39:56,240] jepsen worker 85 - jepsen.util 1085 :invoke :add 9462 INFO [2022-01-10 08:39:56,241] jepsen worker 232 - jepsen.util 1232 :ok :add 9461 INFO [2022-01-10 08:39:56,247] jepsen worker 232 - jepsen.util 1232 :invoke :add 9463 INFO [2022-01-10 08:39:56,248] jepsen worker 85 - jepsen.util 1085 :ok :add 9462 INFO [2022-01-10 08:39:56,252] jepsen worker 547 - jepsen.util 2547 :invoke :add 9464 INFO [2022-01-10 08:39:56,255] jepsen worker 85 - jepsen.util 1085 :invoke :add 9465 INFO [2022-01-10 08:39:56,258] jepsen worker 843 - jepsen.util 2843 :invoke :add 9466 INFO [2022-01-10 08:39:56,260] jepsen worker 938 - jepsen.util 1938 :invoke :add 9467 INFO [2022-01-10 08:39:56,262] jepsen worker 791 - jepsen.util 1791 :info :add 8135 :RequestTimeoutException INFO [2022-01-10 08:39:56,263] jepsen worker 291 - jepsen.util 2291 :invoke :add 9468 INFO [2022-01-10 08:39:56,267] jepsen worker 137 - jepsen.util 1137 :info :add 8134 :RequestTimeoutException INFO [2022-01-10 08:39:56,267] jepsen worker 791 - jepsen.util 2791 :invoke :add 9469 INFO [2022-01-10 08:39:56,268] jepsen worker 232 - jepsen.util 1232 :ok :add 9463 INFO [2022-01-10 08:39:56,271] jepsen worker 160 - jepsen.util 1160 :info :add 8137 :RequestTimeoutException INFO [2022-01-10 08:39:56,273] jepsen worker 22 - jepsen.util 1022 :info :add 8136 :RequestTimeoutException INFO [2022-01-10 08:39:56,273] jepsen worker 99 - jepsen.util 1099 :invoke :add 9470 INFO [2022-01-10 08:39:56,279] jepsen worker 22 - jepsen.util 2022 :invoke :add 9471 INFO [2022-01-10 08:39:56,281] jepsen worker 137 - jepsen.util 2137 :invoke :add 9472 INFO [2022-01-10 08:39:56,285] jepsen worker 160 - jepsen.util 2160 :invoke :add 9473 INFO [2022-01-10 08:39:56,287] jepsen worker 232 - jepsen.util 1232 :invoke :add 9474 INFO [2022-01-10 08:39:56,420] jepsen worker 184 - jepsen.util 1184 :info :add 8138 :RequestTimeoutException INFO [2022-01-10 08:39:56,423] jepsen worker 772 - jepsen.util 2772 :info :add 8139 :RequestTimeoutException INFO [2022-01-10 08:39:56,427] jepsen worker 772 - jepsen.util 3772 :invoke :add 9475 INFO [2022-01-10 08:39:56,430] jepsen worker 184 - jepsen.util 2184 :invoke :add 9476 INFO [2022-01-10 08:39:56,460] jepsen worker 337 - jepsen.util 1337 :info :add 8144 :RequestTimeoutException INFO [2022-01-10 08:39:56,461] jepsen worker 768 - jepsen.util 1768 :info :add 8147 :RequestTimeoutException INFO [2022-01-10 08:39:56,462] jepsen worker 215 - jepsen.util 1215 :info :add 8146 :RequestTimeoutException INFO [2022-01-10 08:39:56,463] jepsen worker 186 - jepsen.util 2186 :info :add 8145 :RequestTimeoutException INFO [2022-01-10 08:39:56,463] jepsen worker 368 - jepsen.util 1368 :info :add 8148 :RequestTimeoutException INFO [2022-01-10 08:39:56,467] jepsen worker 603 - jepsen.util 1603 :info :add 8150 :RequestTimeoutException INFO [2022-01-10 08:39:56,470] jepsen worker 525 - jepsen.util 1525 :info :add 8151 :RequestTimeoutException INFO [2022-01-10 08:39:56,471] jepsen worker 337 - jepsen.util 2337 :invoke :add 9477 INFO [2022-01-10 08:39:56,472] jepsen worker 750 - jepsen.util 1750 :info :add 8149 :RequestTimeoutException INFO [2022-01-10 08:39:56,477] jepsen worker 750 - jepsen.util 2750 :invoke :add 9478 INFO [2022-01-10 08:39:56,480] jepsen worker 603 - jepsen.util 2603 :invoke :add 9479 INFO [2022-01-10 08:39:56,482] jepsen worker 186 - jepsen.util 3186 :invoke :add 9480 INFO [2022-01-10 08:39:56,485] jepsen worker 368 - jepsen.util 2368 :invoke :add 9481 INFO [2022-01-10 08:39:56,488] jepsen worker 215 - jepsen.util 2215 :invoke :add 9482 INFO [2022-01-10 08:39:56,491] jepsen worker 525 - jepsen.util 2525 :invoke :add 9483 INFO [2022-01-10 08:39:56,494] jepsen worker 768 - jepsen.util 2768 :invoke :add 9484 INFO [2022-01-10 08:39:56,513] jepsen worker 739 - jepsen.util 1739 :info :add 8152 :RequestTimeoutException INFO [2022-01-10 08:39:56,517] jepsen worker 739 - jepsen.util 2739 :invoke :add 9485 INFO [2022-01-10 08:39:56,521] jepsen worker 53 - jepsen.util 2053 :info :add 8153 :RequestTimeoutException INFO [2022-01-10 08:39:56,523] jepsen worker 732 - jepsen.util 1732 :info :add 8155 :RequestTimeoutException INFO [2022-01-10 08:39:56,523] jepsen worker 559 - jepsen.util 1559 :info :add 8154 :RequestTimeoutException INFO [2022-01-10 08:39:56,527] jepsen worker 380 - jepsen.util 1380 :info :add 8156 :RequestTimeoutException INFO [2022-01-10 08:39:56,529] jepsen worker 732 - jepsen.util 2732 :invoke :add 9486 INFO [2022-01-10 08:39:56,533] jepsen worker 559 - jepsen.util 2559 :invoke :add 9487 INFO [2022-01-10 08:39:56,533] jepsen worker 279 - jepsen.util 1279 :info :add 8160 :RequestTimeoutException INFO [2022-01-10 08:39:56,536] jepsen worker 380 - jepsen.util 2380 :invoke :add 9488 INFO [2022-01-10 08:39:56,537] jepsen worker 764 - jepsen.util 1764 :info :add 8162 :RequestTimeoutException INFO [2022-01-10 08:39:56,541] jepsen worker 897 - jepsen.util 1897 :info :add 8163 :RequestTimeoutException INFO [2022-01-10 08:39:56,541] jepsen worker 279 - jepsen.util 2279 :invoke :add 9489 INFO [2022-01-10 08:39:56,542] jepsen worker 548 - jepsen.util 1548 :info :add 8161 :RequestTimeoutException INFO [2022-01-10 08:39:56,544] jepsen worker 897 - jepsen.util 2897 :invoke :add 9490 INFO [2022-01-10 08:39:56,548] jepsen worker 548 - jepsen.util 2548 :invoke :add 9491 INFO [2022-01-10 08:39:56,551] jepsen worker 764 - jepsen.util 2764 :invoke :add 9492 INFO [2022-01-10 08:39:56,553] jepsen worker 53 - jepsen.util 3053 :invoke :add 9493 INFO [2022-01-10 08:39:56,569] jepsen worker 694 - jepsen.util 2694 :ok :add 8164 INFO [2022-01-10 08:39:56,569] jepsen worker 686 - jepsen.util 1686 :ok :add 8215 INFO [2022-01-10 08:39:56,569] jepsen worker 976 - jepsen.util 1976 :ok :add 8239 INFO [2022-01-10 08:39:56,573] jepsen worker 976 - jepsen.util 1976 :invoke :add 9494 INFO [2022-01-10 08:39:56,576] jepsen worker 694 - jepsen.util 2694 :invoke :add 9495 INFO [2022-01-10 08:39:56,579] jepsen worker 686 - jepsen.util 1686 :invoke :add 9496 INFO [2022-01-10 08:39:56,623] jepsen worker 761 - jepsen.util 1761 :info :add 8165 :RequestTimeoutException INFO [2022-01-10 08:39:56,627] jepsen worker 761 - jepsen.util 2761 :invoke :add 9497 INFO [2022-01-10 08:39:56,631] jepsen worker 454 - jepsen.util 1454 :info :add 8166 :RequestTimeoutException INFO [2022-01-10 08:39:56,633] jepsen worker 372 - jepsen.util 1372 :info :add 8167 :RequestTimeoutException INFO [2022-01-10 08:39:56,637] jepsen worker 370 - jepsen.util 2370 :info :add 8169 :RequestTimeoutException INFO [2022-01-10 08:39:56,637] jepsen worker 454 - jepsen.util 2454 :invoke :add 9498 INFO [2022-01-10 08:39:56,640] jepsen worker 801 - jepsen.util 2801 :info :add 8170 :RequestTimeoutException INFO [2022-01-10 08:39:56,641] jepsen worker 372 - jepsen.util 2372 :invoke :add 9499 INFO [2022-01-10 08:39:56,641] jepsen worker 625 - jepsen.util 1625 :info :add 8172 :RequestTimeoutException INFO [2022-01-10 08:39:56,642] jepsen worker 931 - jepsen.util 2931 :info :add 8168 :RequestTimeoutException INFO [2022-01-10 08:39:56,643] jepsen worker 349 - jepsen.util 1349 :info :add 8171 :RequestTimeoutException INFO [2022-01-10 08:39:56,643] jepsen worker 13 - jepsen.util 1013 :info :add 8173 :RequestTimeoutException INFO [2022-01-10 08:39:56,649] jepsen worker 370 - jepsen.util 3370 :invoke :add 9500 INFO [2022-01-10 08:39:56,651] jepsen worker 349 - jepsen.util 2349 :invoke :add 9501 INFO [2022-01-10 08:39:56,653] jepsen worker 931 - jepsen.util 3931 :invoke :add 9502 INFO [2022-01-10 08:39:56,656] jepsen worker 801 - jepsen.util 3801 :invoke :add 9503 INFO [2022-01-10 08:39:56,659] jepsen worker 625 - jepsen.util 2625 :invoke :add 9504 INFO [2022-01-10 08:39:56,661] jepsen worker 13 - jepsen.util 2013 :invoke :add 9505 INFO [2022-01-10 08:39:56,685] jepsen worker 706 - jepsen.util 1706 :info :add 8179 :RequestTimeoutException INFO [2022-01-10 08:39:56,686] jepsen worker 803 - jepsen.util 1803 :info :add 8178 :RequestTimeoutException INFO [2022-01-10 08:39:56,692] jepsen worker 95 - jepsen.util 2095 :info :add 8180 :RequestTimeoutException INFO [2022-01-10 08:39:56,693] jepsen worker 810 - jepsen.util 1810 :info :add 8181 :RequestTimeoutException INFO [2022-01-10 08:39:56,694] jepsen worker 706 - jepsen.util 2706 :invoke :add 9506 INFO [2022-01-10 08:39:56,696] jepsen worker 257 - jepsen.util 1257 :info :add 8182 :RequestTimeoutException INFO [2022-01-10 08:39:56,698] jepsen worker 95 - jepsen.util 3095 :invoke :add 9507 INFO [2022-01-10 08:39:56,700] jepsen worker 131 - jepsen.util 1131 :info :add 8183 :RequestTimeoutException INFO [2022-01-10 08:39:56,701] jepsen worker 111 - jepsen.util 1111 :info :add 8185 :RequestTimeoutException INFO [2022-01-10 08:39:56,703] jepsen worker 882 - jepsen.util 1882 :info :add 8184 :RequestTimeoutException INFO [2022-01-10 08:39:56,704] jepsen worker 578 - jepsen.util 1578 :info :add 8186 :RequestTimeoutException INFO [2022-01-10 08:39:56,706] jepsen worker 803 - jepsen.util 2803 :invoke :add 9508 INFO [2022-01-10 08:39:56,707] jepsen worker 282 - jepsen.util 2282 :info :add 8188 :RequestTimeoutException INFO [2022-01-10 08:39:56,710] jepsen worker 623 - jepsen.util 1623 :info :add 8189 :RequestTimeoutException INFO [2022-01-10 08:39:56,711] jepsen worker 282 - jepsen.util 3282 :invoke :add 9509 INFO [2022-01-10 08:39:56,712] jepsen worker 850 - jepsen.util 1850 :info :add 8187 :RequestTimeoutException INFO [2022-01-10 08:39:56,713] jepsen worker 192 - jepsen.util 1192 :info :add 8190 :RequestTimeoutException INFO [2022-01-10 08:39:56,718] jepsen worker 111 - jepsen.util 2111 :invoke :add 9510 INFO [2022-01-10 08:39:56,721] jepsen worker 192 - jepsen.util 2192 :invoke :add 9511 INFO [2022-01-10 08:39:56,722] jepsen worker 436 - jepsen.util 2436 :info :add 8191 :RequestTimeoutException INFO [2022-01-10 08:39:56,722] jepsen worker 11 - jepsen.util 1011 :info :add 8193 :RequestTimeoutException INFO [2022-01-10 08:39:56,723] jepsen worker 685 - jepsen.util 1685 :info :add 8192 :RequestTimeoutException INFO [2022-01-10 08:39:56,724] jepsen worker 257 - jepsen.util 2257 :invoke :add 9512 INFO [2022-01-10 08:39:56,730] jepsen worker 623 - jepsen.util 2623 :invoke :add 9513 INFO [2022-01-10 08:39:56,732] jepsen worker 436 - jepsen.util 3436 :invoke :add 9514 INFO [2022-01-10 08:39:56,735] jepsen worker 882 - jepsen.util 2882 :invoke :add 9515 INFO [2022-01-10 08:39:56,737] jepsen worker 850 - jepsen.util 2850 :invoke :add 9516 INFO [2022-01-10 08:39:56,740] jepsen worker 685 - jepsen.util 2685 :invoke :add 9517 INFO [2022-01-10 08:39:56,742] jepsen worker 810 - jepsen.util 2810 :invoke :add 9518 INFO [2022-01-10 08:39:56,745] jepsen worker 11 - jepsen.util 2011 :invoke :add 9519 INFO [2022-01-10 08:39:56,747] jepsen worker 131 - jepsen.util 2131 :invoke :add 9520 INFO [2022-01-10 08:39:56,750] jepsen worker 578 - jepsen.util 2578 :invoke :add 9521 INFO [2022-01-10 08:39:56,807] jepsen worker 989 - jepsen.util 1989 :info :add 8199 :RequestTimeoutException INFO [2022-01-10 08:39:56,807] jepsen worker 442 - jepsen.util 1442 :info :add 8200 :RequestTimeoutException INFO [2022-01-10 08:39:56,810] jepsen worker 538 - jepsen.util 538 :info :add 8198 :RequestTimeoutException INFO [2022-01-10 08:39:56,810] jepsen worker 716 - jepsen.util 2716 :info :add 8201 :RequestTimeoutException INFO [2022-01-10 08:39:56,812] jepsen worker 989 - jepsen.util 2989 :invoke :add 9522 INFO [2022-01-10 08:39:56,817] jepsen worker 442 - jepsen.util 2442 :invoke :add 9523 INFO [2022-01-10 08:39:56,818] jepsen worker 181 - jepsen.util 181 :info :add 8203 :RequestTimeoutException INFO [2022-01-10 08:39:56,818] jepsen worker 921 - jepsen.util 1921 :info :add 8202 :RequestTimeoutException INFO [2022-01-10 08:39:56,819] jepsen worker 716 - jepsen.util 3716 :invoke :add 9524 INFO [2022-01-10 08:39:56,823] jepsen worker 921 - jepsen.util 2921 :invoke :add 9525 INFO [2022-01-10 08:39:56,826] jepsen worker 181 - jepsen.util 1181 :invoke :add 9526 INFO [2022-01-10 08:39:56,830] jepsen worker 538 - jepsen.util 1538 :invoke :add 9527 INFO [2022-01-10 08:39:56,831] jepsen worker 679 - jepsen.util 1679 :info :add 8208 :RequestTimeoutException INFO [2022-01-10 08:39:56,832] jepsen worker 406 - jepsen.util 2406 :info :add 8209 :RequestTimeoutException INFO [2022-01-10 08:39:56,839] jepsen worker 679 - jepsen.util 2679 :invoke :add 9528 INFO [2022-01-10 08:39:56,841] jepsen worker 406 - jepsen.util 3406 :invoke :add 9529 INFO [2022-01-10 08:39:56,843] jepsen worker 150 - jepsen.util 1150 :info :add 8210 :RequestTimeoutException INFO [2022-01-10 08:39:56,843] jepsen worker 24 - jepsen.util 24 :info :add 8211 :RequestTimeoutException INFO [2022-01-10 08:39:56,846] jepsen worker 62 - jepsen.util 62 :info :add 8212 :RequestTimeoutException INFO [2022-01-10 08:39:56,847] jepsen worker 114 - jepsen.util 1114 :info :add 8214 :RequestTimeoutException INFO [2022-01-10 08:39:56,848] jepsen worker 24 - jepsen.util 1024 :invoke :add 9530 INFO [2022-01-10 08:39:56,851] jepsen worker 270 - jepsen.util 1270 :info :add 8213 :RequestTimeoutException INFO [2022-01-10 08:39:56,853] jepsen worker 496 - jepsen.util 1496 :info :add 8216 :RequestTimeoutException INFO [2022-01-10 08:39:56,855] jepsen worker 150 - jepsen.util 2150 :invoke :add 9531 INFO [2022-01-10 08:39:56,858] jepsen worker 496 - jepsen.util 2496 :invoke :add 9532 INFO [2022-01-10 08:39:56,860] jepsen worker 62 - jepsen.util 1062 :invoke :add 9533 INFO [2022-01-10 08:39:56,863] jepsen worker 114 - jepsen.util 2114 :invoke :add 9534 INFO [2022-01-10 08:39:56,866] jepsen worker 270 - jepsen.util 2270 :invoke :add 9535 INFO [2022-01-10 08:39:56,892] jepsen worker 641 - jepsen.util 2641 :info :add 8217 :RequestTimeoutException INFO [2022-01-10 08:39:56,895] jepsen worker 641 - jepsen.util 3641 :invoke :add 9536 INFO [2022-01-10 08:39:56,897] jepsen worker 246 - jepsen.util 1246 :info :add 8218 :RequestTimeoutException INFO [2022-01-10 08:39:56,900] jepsen worker 881 - jepsen.util 1881 :info :add 8219 :RequestTimeoutException INFO [2022-01-10 08:39:56,902] jepsen worker 246 - jepsen.util 2246 :invoke :add 9537 INFO [2022-01-10 08:39:56,903] jepsen worker 443 - jepsen.util 1443 :info :add 8220 :RequestTimeoutException INFO [2022-01-10 08:39:56,906] jepsen worker 443 - jepsen.util 2443 :invoke :add 9538 INFO [2022-01-10 08:39:56,909] jepsen worker 881 - jepsen.util 2881 :invoke :add 9539 INFO [2022-01-10 08:39:56,911] jepsen worker 126 - jepsen.util 1126 :info :add 8221 :RequestTimeoutException INFO [2022-01-10 08:39:56,912] jepsen worker 94 - jepsen.util 1094 :info :add 8223 :RequestTimeoutException INFO [2022-01-10 08:39:56,913] jepsen worker 205 - jepsen.util 2205 :info :add 8222 :RequestTimeoutException INFO [2022-01-10 08:39:56,917] jepsen worker 905 - jepsen.util 1905 :info :add 8224 :RequestTimeoutException INFO [2022-01-10 08:39:56,918] jepsen worker 205 - jepsen.util 3205 :invoke :add 9540 INFO [2022-01-10 08:39:56,920] jepsen worker 332 - jepsen.util 1332 :info :add 8225 :RequestTimeoutException INFO [2022-01-10 08:39:56,922] jepsen worker 877 - jepsen.util 1877 :info :add 8226 :RequestTimeoutException INFO [2022-01-10 08:39:56,923] jepsen worker 94 - jepsen.util 2094 :invoke :add 9541 INFO [2022-01-10 08:39:56,929] jepsen worker 332 - jepsen.util 2332 :invoke :add 9542 INFO [2022-01-10 08:39:56,932] jepsen worker 905 - jepsen.util 2905 :invoke :add 9543 INFO [2022-01-10 08:39:56,935] jepsen worker 877 - jepsen.util 2877 :invoke :add 9544 INFO [2022-01-10 08:39:56,938] jepsen worker 126 - jepsen.util 2126 :invoke :add 9545 INFO [2022-01-10 08:39:56,961] jepsen worker 656 - jepsen.util 2656 :info :add 8227 :RequestTimeoutException INFO [2022-01-10 08:39:56,966] jepsen worker 656 - jepsen.util 3656 :invoke :add 9546 INFO [2022-01-10 08:39:57,032] jepsen worker 735 - jepsen.util 2735 :info :add 8234 :RequestTimeoutException INFO [2022-01-10 08:39:57,038] jepsen worker 735 - jepsen.util 3735 :invoke :add 9547 INFO [2022-01-10 08:39:57,041] jepsen worker 385 - jepsen.util 2385 :info :add 8235 :RequestTimeoutException INFO [2022-01-10 08:39:57,045] jepsen worker 385 - jepsen.util 3385 :invoke :add 9548 INFO [2022-01-10 08:39:57,076] jepsen worker 603 - jepsen.util 2603 :ok :add 9479 INFO [2022-01-10 08:39:57,080] jepsen worker 603 - jepsen.util 2603 :invoke :add 9549 INFO [2022-01-10 08:39:57,172] jepsen worker 309 - jepsen.util 2309 :info :add 8237 :RequestTimeoutException INFO [2022-01-10 08:39:57,176] jepsen worker 309 - jepsen.util 3309 :invoke :add 9550 INFO [2022-01-10 08:39:57,197] jepsen worker 951 - jepsen.util 2951 :info :add 8238 :RequestTimeoutException INFO [2022-01-10 08:39:57,203] jepsen worker 951 - jepsen.util 3951 :invoke :add 9551 INFO [2022-01-10 08:39:57,223] jepsen worker 793 - jepsen.util 2793 :info :add 8244 :RequestTimeoutException INFO [2022-01-10 08:39:57,227] jepsen worker 741 - jepsen.util 1741 :info :add 8245 :RequestTimeoutException INFO [2022-01-10 08:39:57,227] jepsen worker 793 - jepsen.util 3793 :invoke :add 9552 INFO [2022-01-10 08:39:57,230] jepsen worker 970 - jepsen.util 1970 :info :add 8246 :RequestTimeoutException INFO [2022-01-10 08:39:57,231] jepsen worker 741 - jepsen.util 2741 :invoke :add 9553 INFO [2022-01-10 08:39:57,231] jepsen worker 567 - jepsen.util 1567 :info :add 8250 :RequestTimeoutException INFO [2022-01-10 08:39:57,232] jepsen worker 220 - jepsen.util 1220 :info :add 8252 :RequestTimeoutException INFO [2022-01-10 08:39:57,232] jepsen worker 647 - jepsen.util 1647 :info :add 8247 :RequestTimeoutException INFO [2022-01-10 08:39:57,232] jepsen worker 967 - jepsen.util 2967 :info :add 8248 :RequestTimeoutException INFO [2022-01-10 08:39:57,232] jepsen worker 785 - jepsen.util 2785 :info :add 8249 :RequestTimeoutException INFO [2022-01-10 08:39:57,234] jepsen worker 593 - jepsen.util 1593 :info :add 8251 :RequestTimeoutException INFO [2022-01-10 08:39:57,237] jepsen worker 752 - jepsen.util 2752 :info :add 8253 :RequestTimeoutException INFO [2022-01-10 08:39:57,241] jepsen worker 16 - jepsen.util 2016 :info :add 8254 :RequestTimeoutException INFO [2022-01-10 08:39:57,243] jepsen worker 967 - jepsen.util 3967 :invoke :add 9554 INFO [2022-01-10 08:39:57,243] jepsen worker 736 - jepsen.util 1736 :info :add 8255 :RequestTimeoutException INFO [2022-01-10 08:39:57,248] jepsen worker 647 - jepsen.util 2647 :invoke :add 9555 INFO [2022-01-10 08:39:57,251] jepsen worker 478 - jepsen.util 2478 :info :add 8256 :RequestTimeoutException INFO [2022-01-10 08:39:57,251] jepsen worker 736 - jepsen.util 2736 :invoke :add 9556 INFO [2022-01-10 08:39:57,254] jepsen worker 926 - jepsen.util 2926 :info :add 8257 :RequestTimeoutException INFO [2022-01-10 08:39:57,256] jepsen worker 752 - jepsen.util 3752 :invoke :add 9557 INFO [2022-01-10 08:39:57,260] jepsen worker 220 - jepsen.util 2220 :invoke :add 9558 INFO [2022-01-10 08:39:57,260] jepsen worker 369 - jepsen.util 1369 :info :add 8260 :RequestTimeoutException INFO [2022-01-10 08:39:57,260] jepsen worker 392 - jepsen.util 2392 :info :add 8263 :RequestTimeoutException INFO [2022-01-10 08:39:57,262] jepsen worker 267 - jepsen.util 2267 :info :add 8262 :RequestTimeoutException INFO [2022-01-10 08:39:57,262] jepsen worker 271 - jepsen.util 2271 :info :add 8261 :RequestTimeoutException INFO [2022-01-10 08:39:57,268] jepsen worker 785 - jepsen.util 3785 :invoke :add 9559 INFO [2022-01-10 08:39:57,271] jepsen worker 970 - jepsen.util 2970 :invoke :add 9560 INFO [2022-01-10 08:39:57,274] jepsen worker 926 - jepsen.util 3926 :invoke :add 9561 INFO [2022-01-10 08:39:57,277] jepsen worker 392 - jepsen.util 3392 :invoke :add 9562 INFO [2022-01-10 08:39:57,279] jepsen worker 593 - jepsen.util 2593 :invoke :add 9563 INFO [2022-01-10 08:39:57,282] jepsen worker 545 - jepsen.util 2545 :info :add 8265 :RequestTimeoutException INFO [2022-01-10 08:39:57,283] jepsen worker 567 - jepsen.util 2567 :invoke :add 9564 INFO [2022-01-10 08:39:57,283] jepsen worker 505 - jepsen.util 2505 :info :add 8264 :RequestTimeoutException INFO [2022-01-10 08:39:57,288] jepsen worker 16 - jepsen.util 3016 :invoke :add 9565 INFO [2022-01-10 08:39:57,291] jepsen worker 505 - jepsen.util 3505 :invoke :add 9566 INFO [2022-01-10 08:39:57,293] jepsen worker 369 - jepsen.util 2369 :invoke :add 9567 INFO [2022-01-10 08:39:57,296] jepsen worker 478 - jepsen.util 3478 :invoke :add 9568 INFO [2022-01-10 08:39:57,298] jepsen worker 267 - jepsen.util 3267 :invoke :add 9569 INFO [2022-01-10 08:39:57,301] jepsen worker 545 - jepsen.util 3545 :invoke :add 9570 INFO [2022-01-10 08:39:57,304] jepsen worker 271 - jepsen.util 3271 :invoke :add 9571 INFO [2022-01-10 08:39:57,367] jepsen worker 28 - jepsen.util 1028 :info :add 8266 :RequestTimeoutException INFO [2022-01-10 08:39:57,370] jepsen worker 666 - jepsen.util 1666 :info :add 8267 :RequestTimeoutException INFO [2022-01-10 08:39:57,377] jepsen worker 666 - jepsen.util 2666 :invoke :add 9572 INFO [2022-01-10 08:39:57,380] jepsen worker 28 - jepsen.util 2028 :invoke :add 9573 INFO [2022-01-10 08:39:57,383] jepsen worker 746 - jepsen.util 1746 :info :add 8268 :RequestTimeoutException INFO [2022-01-10 08:39:57,390] jepsen worker 746 - jepsen.util 2746 :invoke :add 9574 INFO [2022-01-10 08:39:57,391] jepsen worker 116 - jepsen.util 1116 :info :add 8269 :RequestTimeoutException INFO [2022-01-10 08:39:57,394] jepsen worker 626 - jepsen.util 1626 :info :add 8270 :RequestTimeoutException INFO [2022-01-10 08:39:57,397] jepsen worker 626 - jepsen.util 2626 :invoke :add 9575 INFO [2022-01-10 08:39:57,400] jepsen worker 116 - jepsen.util 2116 :invoke :add 9576 INFO [2022-01-10 08:39:57,402] jepsen worker 802 - jepsen.util 2802 :info :add 8271 :RequestTimeoutException INFO [2022-01-10 08:39:57,406] jepsen worker 802 - jepsen.util 3802 :invoke :add 9577 INFO [2022-01-10 08:39:57,408] jepsen worker 650 - jepsen.util 1650 :info :add 8272 :RequestTimeoutException INFO [2022-01-10 08:39:57,410] jepsen worker 600 - jepsen.util 2600 :info :add 8273 :RequestTimeoutException INFO [2022-01-10 08:39:57,412] jepsen worker 650 - jepsen.util 2650 :invoke :add 9578 INFO [2022-01-10 08:39:57,416] jepsen worker 600 - jepsen.util 3600 :invoke :add 9579 INFO [2022-01-10 08:39:57,417] jepsen worker 42 - jepsen.util 2042 :info :add 8279 :RequestTimeoutException INFO [2022-01-10 08:39:57,420] jepsen worker 515 - jepsen.util 2515 :info :add 8280 :RequestTimeoutException INFO [2022-01-10 08:39:57,423] jepsen worker 878 - jepsen.util 2878 :info :add 8278 :RequestTimeoutException INFO [2022-01-10 08:39:57,425] jepsen worker 515 - jepsen.util 3515 :invoke :add 9580 INFO [2022-01-10 08:39:57,429] jepsen worker 42 - jepsen.util 3042 :invoke :add 9581 INFO [2022-01-10 08:39:57,432] jepsen worker 878 - jepsen.util 3878 :invoke :add 9582 INFO [2022-01-10 08:39:57,481] jepsen worker 758 - jepsen.util 1758 :info :add 8282 :RequestTimeoutException INFO [2022-01-10 08:39:57,483] jepsen worker 488 - jepsen.util 1488 :info :add 8281 :RequestTimeoutException INFO [2022-01-10 08:39:57,484] jepsen worker 758 - jepsen.util 2758 :invoke :add 9583 INFO [2022-01-10 08:39:57,489] jepsen worker 488 - jepsen.util 2488 :invoke :add 9584 INFO [2022-01-10 08:39:57,513] jepsen worker 827 - jepsen.util 2827 :info :add 8283 :RequestTimeoutException INFO [2022-01-10 08:39:57,514] jepsen worker 716 - jepsen.util 3716 :ok :add 9524 INFO [2022-01-10 08:39:57,514] jepsen worker 803 - jepsen.util 2803 :ok :add 9508 INFO [2022-01-10 08:39:57,515] jepsen worker 186 - jepsen.util 3186 :ok :add 9480 INFO [2022-01-10 08:39:57,515] jepsen worker 454 - jepsen.util 2454 :ok :add 9498 INFO [2022-01-10 08:39:57,515] jepsen worker 111 - jepsen.util 2111 :ok :add 9510 INFO [2022-01-10 08:39:57,520] jepsen worker 827 - jepsen.util 3827 :invoke :add 9585 INFO [2022-01-10 08:39:57,521] jepsen worker 15 - jepsen.util 2015 :info :add 8284 :RequestTimeoutException INFO [2022-01-10 08:39:57,522] jepsen worker 555 - jepsen.util 2555 :info :add 8286 :RequestTimeoutException INFO [2022-01-10 08:39:57,522] jepsen worker 111 - jepsen.util 2111 :invoke :add 9586 INFO [2022-01-10 08:39:57,524] jepsen worker 507 - jepsen.util 2507 :info :add 8285 :RequestTimeoutException INFO [2022-01-10 08:39:57,527] jepsen worker 552 - jepsen.util 2552 :info :add 8287 :RequestTimeoutException INFO [2022-01-10 08:39:57,528] jepsen worker 555 - jepsen.util 3555 :invoke :add 9587 INFO [2022-01-10 08:39:57,530] jepsen worker 471 - jepsen.util 1471 :info :add 8288 :RequestTimeoutException INFO [2022-01-10 08:39:57,531] jepsen worker 716 - jepsen.util 3716 :invoke :add 9588 INFO [2022-01-10 08:39:57,534] jepsen worker 803 - jepsen.util 2803 :invoke :add 9589 INFO [2022-01-10 08:39:57,537] jepsen worker 15 - jepsen.util 3015 :invoke :add 9590 INFO [2022-01-10 08:39:57,540] jepsen worker 507 - jepsen.util 3507 :invoke :add 9591 INFO [2022-01-10 08:39:57,541] jepsen worker 860 - jepsen.util 2860 :info :add 8290 :RequestTimeoutException INFO [2022-01-10 08:39:57,543] jepsen worker 387 - jepsen.util 1387 :info :add 8291 :RequestTimeoutException INFO [2022-01-10 08:39:57,543] jepsen worker 798 - jepsen.util 1798 :info :add 8289 :RequestTimeoutException INFO [2022-01-10 08:39:57,543] jepsen worker 552 - jepsen.util 3552 :invoke :add 9592 INFO [2022-01-10 08:39:57,547] jepsen worker 806 - jepsen.util 2806 :info :add 8293 :RequestTimeoutException INFO [2022-01-10 08:39:57,551] jepsen worker 387 - jepsen.util 2387 :invoke :add 9593 INFO [2022-01-10 08:39:57,552] jepsen worker 765 - jepsen.util 1765 :info :add 8292 :RequestTimeoutException INFO [2022-01-10 08:39:57,554] jepsen worker 860 - jepsen.util 3860 :invoke :add 9594 INFO [2022-01-10 08:39:57,557] jepsen worker 765 - jepsen.util 2765 :invoke :add 9595 INFO [2022-01-10 08:39:57,560] jepsen worker 471 - jepsen.util 2471 :invoke :add 9596 INFO [2022-01-10 08:39:57,562] jepsen worker 495 - jepsen.util 1495 :info :add 8298 :RequestTimeoutException INFO [2022-01-10 08:39:57,562] jepsen worker 186 - jepsen.util 3186 :invoke :add 9597 INFO [2022-01-10 08:39:57,566] jepsen worker 806 - jepsen.util 3806 :invoke :add 9598 INFO [2022-01-10 08:39:57,566] jepsen worker 621 - jepsen.util 1621 :info :add 8299 :RequestTimeoutException INFO [2022-01-10 08:39:57,568] jepsen worker 798 - jepsen.util 2798 :invoke :add 9599 INFO [2022-01-10 08:39:57,571] jepsen worker 118 - jepsen.util 1118 :info :add 8300 :RequestTimeoutException INFO [2022-01-10 08:39:57,572] jepsen worker 454 - jepsen.util 2454 :invoke :add 9600 INFO [2022-01-10 08:39:57,576] jepsen worker 621 - jepsen.util 2621 :invoke :add 9601 INFO [2022-01-10 08:39:57,579] jepsen worker 495 - jepsen.util 2495 :invoke :add 9602 INFO [2022-01-10 08:39:57,582] jepsen worker 118 - jepsen.util 2118 :invoke :add 9603 INFO [2022-01-10 08:39:57,590] jepsen worker 233 - jepsen.util 1233 :info :add 8301 :RequestTimeoutException INFO [2022-01-10 08:39:57,594] jepsen worker 316 - jepsen.util 1316 :info :add 8303 :RequestTimeoutException INFO [2022-01-10 08:39:57,594] jepsen worker 156 - jepsen.util 1156 :info :add 8302 :RequestTimeoutException INFO [2022-01-10 08:39:57,595] jepsen worker 233 - jepsen.util 2233 :invoke :add 9604 INFO [2022-01-10 08:39:57,600] jepsen worker 316 - jepsen.util 2316 :invoke :add 9605 INFO [2022-01-10 08:39:57,603] jepsen worker 156 - jepsen.util 2156 :invoke :add 9606 INFO [2022-01-10 08:39:57,721] jepsen worker 262 - jepsen.util 1262 :info :add 8308 :RequestTimeoutException INFO [2022-01-10 08:39:57,726] jepsen worker 262 - jepsen.util 2262 :invoke :add 9607 INFO [2022-01-10 08:39:57,753] jepsen worker 728 - jepsen.util 2728 :info :add 8309 :RequestTimeoutException INFO [2022-01-10 08:39:57,759] jepsen worker 728 - jepsen.util 3728 :invoke :add 9608 INFO [2022-01-10 08:39:57,802] jepsen worker 335 - jepsen.util 2335 :info :add 8310 :RequestTimeoutException INFO [2022-01-10 08:39:57,807] jepsen worker 335 - jepsen.util 3335 :invoke :add 9609 INFO [2022-01-10 08:39:57,807] jepsen worker 48 - jepsen.util 2048 :info :add 8311 :RequestTimeoutException INFO [2022-01-10 08:39:57,814] jepsen worker 48 - jepsen.util 3048 :invoke :add 9610 INFO [2022-01-10 08:39:57,840] jepsen worker 485 - jepsen.util 1485 :info :add 8312 :RequestTimeoutException INFO [2022-01-10 08:39:57,845] jepsen worker 485 - jepsen.util 2485 :invoke :add 9611 INFO [2022-01-10 08:39:57,861] jepsen worker 87 - jepsen.util 1087 :info :add 8314 :RequestTimeoutException INFO [2022-01-10 08:39:57,863] jepsen worker 813 - jepsen.util 1813 :info :add 8313 :RequestTimeoutException INFO [2022-01-10 08:39:57,864] jepsen worker 342 - jepsen.util 1342 :info :add 8315 :RequestTimeoutException INFO [2022-01-10 08:39:57,869] jepsen worker 342 - jepsen.util 2342 :invoke :add 9612 INFO [2022-01-10 08:39:57,872] jepsen worker 35 - jepsen.util 1035 :info :add 8316 :RequestTimeoutException INFO [2022-01-10 08:39:57,873] jepsen worker 87 - jepsen.util 2087 :invoke :add 9613 INFO [2022-01-10 08:39:57,877] jepsen worker 204 - jepsen.util 1204 :info :add 8317 :RequestTimeoutException INFO [2022-01-10 08:39:57,877] jepsen worker 813 - jepsen.util 2813 :invoke :add 9614 INFO [2022-01-10 08:39:57,880] jepsen worker 320 - jepsen.util 1320 :info :add 8318 :RequestTimeoutException INFO [2022-01-10 08:39:57,881] jepsen worker 997 - jepsen.util 2997 :info :add 8320 :RequestTimeoutException INFO [2022-01-10 08:39:57,881] jepsen worker 35 - jepsen.util 2035 :invoke :add 9615 INFO [2022-01-10 08:39:57,883] jepsen worker 977 - jepsen.util 1977 :info :add 8321 :RequestTimeoutException INFO [2022-01-10 08:39:57,883] jepsen worker 437 - jepsen.util 1437 :info :add 8319 :RequestTimeoutException INFO [2022-01-10 08:39:57,890] jepsen worker 320 - jepsen.util 2320 :invoke :add 9616 INFO [2022-01-10 08:39:57,892] jepsen worker 221 - jepsen.util 1221 :info :add 8322 :RequestTimeoutException INFO [2022-01-10 08:39:57,893] jepsen worker 204 - jepsen.util 2204 :invoke :add 9617 INFO [2022-01-10 08:39:57,896] jepsen worker 997 - jepsen.util 3997 :invoke :add 9618 INFO [2022-01-10 08:39:57,899] jepsen worker 437 - jepsen.util 2437 :invoke :add 9619 INFO [2022-01-10 08:39:57,901] jepsen worker 977 - jepsen.util 2977 :invoke :add 9620 INFO [2022-01-10 08:39:57,904] jepsen worker 221 - jepsen.util 2221 :invoke :add 9621 INFO [2022-01-10 08:39:57,910] jepsen worker 290 - jepsen.util 2290 :info :add 8324 :RequestTimeoutException INFO [2022-01-10 08:39:57,911] jepsen worker 56 - jepsen.util 2056 :info :add 8326 :RequestTimeoutException INFO [2022-01-10 08:39:57,913] jepsen worker 504 - jepsen.util 1504 :info :add 8325 :RequestTimeoutException INFO [2022-01-10 08:39:57,913] jepsen worker 300 - jepsen.util 1300 :info :add 8327 :RequestTimeoutException INFO [2022-01-10 08:39:57,918] jepsen worker 56 - jepsen.util 3056 :invoke :add 9622 INFO [2022-01-10 08:39:57,921] jepsen worker 300 - jepsen.util 2300 :invoke :add 9623 INFO [2022-01-10 08:39:57,925] jepsen worker 504 - jepsen.util 2504 :invoke :add 9624 INFO [2022-01-10 08:39:57,928] jepsen worker 290 - jepsen.util 3290 :invoke :add 9625 INFO [2022-01-10 08:39:57,932] jepsen worker 523 - jepsen.util 1523 :info :add 8334 :RequestTimeoutException INFO [2022-01-10 08:39:57,938] jepsen worker 523 - jepsen.util 2523 :invoke :add 9626 INFO [2022-01-10 08:39:57,941] jepsen worker 857 - jepsen.util 1857 :info :add 8336 :RequestTimeoutException INFO [2022-01-10 08:39:57,942] jepsen worker 516 - jepsen.util 1516 :info :add 8335 :RequestTimeoutException INFO [2022-01-10 08:39:57,948] jepsen worker 516 - jepsen.util 2516 :invoke :add 9627 INFO [2022-01-10 08:39:57,951] jepsen worker 857 - jepsen.util 2857 :invoke :add 9628 INFO [2022-01-10 08:39:57,962] jepsen worker 257 - jepsen.util 2257 :ok :add 9512 INFO [2022-01-10 08:39:57,961] jepsen worker 215 - jepsen.util 2215 :ok :add 9482 INFO [2022-01-10 08:39:57,962] jepsen worker 181 - jepsen.util 1181 :ok :add 9526 INFO [2022-01-10 08:39:57,966] jepsen worker 257 - jepsen.util 2257 :invoke :add 9629 INFO [2022-01-10 08:39:57,968] jepsen worker 215 - jepsen.util 2215 :invoke :add 9630 INFO [2022-01-10 08:39:57,971] jepsen worker 181 - jepsen.util 1181 :invoke :add 9631 INFO [2022-01-10 08:39:58,016] jepsen worker 609 - jepsen.util 2609 :info :add 8337 :RequestTimeoutException INFO [2022-01-10 08:39:58,020] jepsen worker 627 - jepsen.util 2627 :info :add 8338 :RequestTimeoutException INFO [2022-01-10 08:39:58,021] jepsen worker 609 - jepsen.util 3609 :invoke :add 9632 INFO [2022-01-10 08:39:58,025] jepsen worker 627 - jepsen.util 3627 :invoke :add 9633 INFO [2022-01-10 08:39:58,033] jepsen worker 244 - jepsen.util 2244 :info :add 8339 :RequestTimeoutException INFO [2022-01-10 08:39:58,037] jepsen worker 244 - jepsen.util 3244 :invoke :add 9634 INFO [2022-01-10 08:39:58,071] jepsen worker 266 - jepsen.util 2266 :info :add 8344 :RequestTimeoutException INFO [2022-01-10 08:39:58,077] jepsen worker 266 - jepsen.util 3266 :invoke :add 9635 INFO [2022-01-10 08:39:58,112] jepsen worker 222 - jepsen.util 1222 :info :add 8345 :RequestTimeoutException INFO [2022-01-10 08:39:58,116] jepsen worker 222 - jepsen.util 2222 :invoke :add 9636 INFO [2022-01-10 08:39:58,128] jepsen worker 43 - jepsen.util 1043 :info :add 8346 :RequestTimeoutException INFO [2022-01-10 08:39:58,130] jepsen worker 861 - jepsen.util 1861 :info :add 8347 :RequestTimeoutException INFO [2022-01-10 08:39:58,133] jepsen worker 604 - jepsen.util 1604 :info :add 8348 :RequestTimeoutException INFO [2022-01-10 08:39:58,136] jepsen worker 861 - jepsen.util 2861 :invoke :add 9637 INFO [2022-01-10 08:39:58,140] jepsen worker 43 - jepsen.util 2043 :invoke :add 9638 INFO [2022-01-10 08:39:58,141] jepsen worker 120 - jepsen.util 1120 :info :add 8349 :RequestTimeoutException INFO [2022-01-10 08:39:58,143] jepsen worker 892 - jepsen.util 2892 :info :add 8350 :RequestTimeoutException INFO [2022-01-10 08:39:58,144] jepsen worker 604 - jepsen.util 2604 :invoke :add 9639 INFO [2022-01-10 08:39:58,149] jepsen worker 892 - jepsen.util 3892 :invoke :add 9640 INFO [2022-01-10 08:39:58,152] jepsen worker 871 - jepsen.util 1871 :info :add 8351 :RequestTimeoutException INFO [2022-01-10 08:39:58,152] jepsen worker 120 - jepsen.util 2120 :invoke :add 9641 INFO [2022-01-10 08:39:58,156] jepsen worker 871 - jepsen.util 2871 :invoke :add 9642 INFO [2022-01-10 08:39:58,160] jepsen worker 639 - jepsen.util 1639 :info :add 8353 :RequestTimeoutException INFO [2022-01-10 08:39:58,165] jepsen worker 639 - jepsen.util 2639 :invoke :add 9643 INFO [2022-01-10 08:39:58,173] jepsen worker 701 - jepsen.util 2701 :info :add 8354 :RequestTimeoutException INFO [2022-01-10 08:39:58,178] jepsen worker 701 - jepsen.util 3701 :invoke :add 9644 INFO [2022-01-10 08:39:58,219] jepsen worker 750 - jepsen.util 2750 :ok :add 9478 INFO [2022-01-10 08:39:58,224] jepsen worker 750 - jepsen.util 2750 :invoke :add 9645 INFO [2022-01-10 08:39:58,261] jepsen worker 868 - jepsen.util 2868 :info :add 8355 :RequestTimeoutException INFO [2022-01-10 08:39:58,266] jepsen worker 868 - jepsen.util 3868 :invoke :add 9646 INFO [2022-01-10 08:39:58,274] jepsen worker 194 - jepsen.util 2194 :info :add 8356 :RequestTimeoutException INFO [2022-01-10 08:39:58,280] jepsen worker 194 - jepsen.util 3194 :invoke :add 9647 INFO [2022-01-10 08:39:58,282] jepsen worker 278 - jepsen.util 2278 :info :add 8357 :RequestTimeoutException INFO [2022-01-10 08:39:58,286] jepsen worker 278 - jepsen.util 3278 :invoke :add 9648 INFO [2022-01-10 08:39:58,324] jepsen worker 899 - jepsen.util 2899 :ok :add 8566 INFO [2022-01-10 08:39:58,324] jepsen worker 804 - jepsen.util 2804 :ok :add 8552 INFO [2022-01-10 08:39:58,333] jepsen worker 804 - jepsen.util 2804 :invoke :add 9649 INFO [2022-01-10 08:39:58,336] jepsen worker 899 - jepsen.util 2899 :invoke :add 9650 INFO [2022-01-10 08:39:58,343] jepsen worker 123 - jepsen.util 2123 :info :add 8360 :RequestTimeoutException INFO [2022-01-10 08:39:58,349] jepsen worker 123 - jepsen.util 3123 :invoke :add 9651 INFO [2022-01-10 08:39:58,351] jepsen worker 571 - jepsen.util 2571 :info :add 8361 :RequestTimeoutException INFO [2022-01-10 08:39:58,352] jepsen worker 81 - jepsen.util 2081 :info :add 8363 :RequestTimeoutException INFO [2022-01-10 08:39:58,353] jepsen worker 322 - jepsen.util 2322 :info :add 8362 :RequestTimeoutException INFO [2022-01-10 08:39:58,354] jepsen worker 571 - jepsen.util 3571 :invoke :add 9652 INFO [2022-01-10 08:39:58,359] jepsen worker 322 - jepsen.util 3322 :invoke :add 9653 INFO [2022-01-10 08:39:58,362] jepsen worker 81 - jepsen.util 3081 :invoke :add 9654 INFO [2022-01-10 08:39:58,367] jepsen worker 235 - jepsen.util 2235 :info :add 8364 :RequestTimeoutException INFO [2022-01-10 08:39:58,372] jepsen worker 235 - jepsen.util 3235 :invoke :add 9655 INFO [2022-01-10 08:39:58,381] jepsen worker 928 - jepsen.util 2928 :info :add 8365 :RequestTimeoutException INFO [2022-01-10 08:39:58,386] jepsen worker 928 - jepsen.util 3928 :invoke :add 9656 INFO [2022-01-10 08:39:58,403] jepsen worker 282 - jepsen.util 3282 :ok :add 9509 INFO [2022-01-10 08:39:58,404] jepsen worker 921 - jepsen.util 2921 :ok :add 9525 INFO [2022-01-10 08:39:58,404] jepsen worker 368 - jepsen.util 2368 :ok :add 9481 INFO [2022-01-10 08:39:58,404] jepsen worker 372 - jepsen.util 2372 :ok :add 9499 INFO [2022-01-10 08:39:58,405] jepsen worker 192 - jepsen.util 2192 :ok :add 9511 INFO [2022-01-10 08:39:58,407] jepsen worker 921 - jepsen.util 2921 :invoke :add 9657 INFO [2022-01-10 08:39:58,411] jepsen worker 372 - jepsen.util 2372 :invoke :add 9658 INFO [2022-01-10 08:39:58,413] jepsen worker 282 - jepsen.util 3282 :invoke :add 9659 INFO [2022-01-10 08:39:58,416] jepsen worker 192 - jepsen.util 2192 :invoke :add 9660 INFO [2022-01-10 08:39:58,418] jepsen worker 368 - jepsen.util 2368 :invoke :add 9661 INFO [2022-01-10 08:39:58,449] jepsen worker 525 - jepsen.util 2525 :ok :add 9483 INFO [2022-01-10 08:39:58,449] jepsen worker 623 - jepsen.util 2623 :ok :add 9513 INFO [2022-01-10 08:39:58,449] jepsen worker 928 - jepsen.util 3928 :ok :add 9656 INFO [2022-01-10 08:39:58,450] jepsen worker 538 - jepsen.util 1538 :ok :add 9527 INFO [2022-01-10 08:39:58,455] jepsen worker 538 - jepsen.util 1538 :invoke :add 9662 INFO [2022-01-10 08:39:58,458] jepsen worker 623 - jepsen.util 2623 :invoke :add 9663 INFO [2022-01-10 08:39:58,461] jepsen worker 525 - jepsen.util 2525 :invoke :add 9664 INFO [2022-01-10 08:39:58,463] jepsen worker 928 - jepsen.util 3928 :invoke :add 9665 INFO [2022-01-10 08:39:58,523] jepsen worker 655 - jepsen.util 1655 :info :add 8366 :RequestTimeoutException INFO [2022-01-10 08:39:58,527] jepsen worker 467 - jepsen.util 1467 :info :add 8368 :RequestTimeoutException INFO [2022-01-10 08:39:58,528] jepsen worker 448 - jepsen.util 2448 :ok :add 8425 INFO [2022-01-10 08:39:58,531] jepsen worker 473 - jepsen.util 1473 :info :add 8367 :RequestTimeoutException INFO [2022-01-10 08:39:58,532] jepsen worker 869 - jepsen.util 2869 :info :add 8370 :RequestTimeoutException INFO [2022-01-10 08:39:58,533] jepsen worker 577 - jepsen.util 2577 :info :add 8369 :RequestTimeoutException INFO [2022-01-10 08:39:58,533] jepsen worker 655 - jepsen.util 2655 :invoke :add 9666 INFO [2022-01-10 08:39:58,537] jepsen worker 852 - jepsen.util 1852 :info :add 8371 :RequestTimeoutException INFO [2022-01-10 08:39:58,540] jepsen worker 173 - jepsen.util 1173 :info :add 8372 :RequestTimeoutException INFO [2022-01-10 08:39:58,542] jepsen worker 869 - jepsen.util 3869 :invoke :add 9667 INFO [2022-01-10 08:39:58,542] jepsen worker 424 - jepsen.util 1424 :info :add 8373 :RequestTimeoutException INFO [2022-01-10 08:39:58,546] jepsen worker 434 - jepsen.util 1434 :info :add 8378 :RequestTimeoutException INFO [2022-01-10 08:39:58,547] jepsen worker 424 - jepsen.util 2424 :invoke :add 9668 INFO [2022-01-10 08:39:58,550] jepsen worker 19 - jepsen.util 2019 :info :add 8379 :RequestTimeoutException INFO [2022-01-10 08:39:58,551] jepsen worker 577 - jepsen.util 3577 :invoke :add 9669 INFO [2022-01-10 08:39:58,553] jepsen worker 310 - jepsen.util 2310 :info :add 8380 :RequestTimeoutException INFO [2022-01-10 08:39:58,557] jepsen worker 411 - jepsen.util 1411 :info :add 8382 :RequestTimeoutException INFO [2022-01-10 08:39:58,557] jepsen worker 173 - jepsen.util 2173 :invoke :add 9670 INFO [2022-01-10 08:39:58,561] jepsen worker 458 - jepsen.util 2458 :info :add 8381 :RequestTimeoutException INFO [2022-01-10 08:39:58,562] jepsen worker 96 - jepsen.util 1096 :info :add 8385 :RequestTimeoutException INFO [2022-01-10 08:39:58,562] jepsen worker 411 - jepsen.util 2411 :invoke :add 9671 INFO [2022-01-10 08:39:58,562] jepsen worker 129 - jepsen.util 1129 :info :add 8384 :RequestTimeoutException INFO [2022-01-10 08:39:58,563] jepsen worker 318 - jepsen.util 1318 :info :add 8383 :RequestTimeoutException INFO [2022-01-10 08:39:58,567] jepsen worker 948 - jepsen.util 1948 :info :add 8386 :RequestTimeoutException INFO [2022-01-10 08:39:58,570] jepsen worker 620 - jepsen.util 2620 :info :add 8388 :RequestTimeoutException INFO [2022-01-10 08:39:58,570] jepsen worker 318 - jepsen.util 2318 :invoke :add 9672 INFO [2022-01-10 08:39:58,571] jepsen worker 51 - jepsen.util 1051 :info :add 8387 :RequestTimeoutException INFO [2022-01-10 08:39:58,572] jepsen worker 933 - jepsen.util 1933 :info :add 8389 :RequestTimeoutException INFO [2022-01-10 08:39:58,577] jepsen worker 374 - jepsen.util 1374 :info :add 8390 :RequestTimeoutException INFO [2022-01-10 08:39:58,577] jepsen worker 448 - jepsen.util 2448 :invoke :add 9673 INFO [2022-01-10 08:39:58,581] jepsen worker 73 - jepsen.util 1073 :info :add 8391 :RequestTimeoutException INFO [2022-01-10 08:39:58,582] jepsen worker 458 - jepsen.util 3458 :invoke :add 9674 INFO [2022-01-10 08:39:58,582] jepsen worker 272 - jepsen.util 2272 :info :add 8393 :RequestTimeoutException INFO [2022-01-10 08:39:58,588] jepsen worker 948 - jepsen.util 2948 :invoke :add 9675 INFO [2022-01-10 08:39:58,592] jepsen worker 51 - jepsen.util 2051 :invoke :add 9676 INFO [2022-01-10 08:39:58,595] jepsen worker 272 - jepsen.util 3272 :invoke :add 9677 INFO [2022-01-10 08:39:58,598] jepsen worker 19 - jepsen.util 3019 :invoke :add 9678 INFO [2022-01-10 08:39:58,601] jepsen worker 374 - jepsen.util 2374 :invoke :add 9679 INFO [2022-01-10 08:39:58,605] jepsen worker 73 - jepsen.util 2073 :invoke :add 9680 INFO [2022-01-10 08:39:58,608] jepsen worker 467 - jepsen.util 2467 :invoke :add 9681 INFO [2022-01-10 08:39:58,608] jepsen worker 253 - jepsen.util 2253 :info :add 8469 :SyncWriteAmbiguous INFO [2022-01-10 08:39:58,614] jepsen worker 96 - jepsen.util 2096 :invoke :add 9682 INFO [2022-01-10 08:39:58,617] jepsen worker 253 - jepsen.util 3253 :invoke :add 9683 INFO [2022-01-10 08:39:58,620] jepsen worker 129 - jepsen.util 2129 :invoke :add 9684 INFO [2022-01-10 08:39:58,623] jepsen worker 106 - jepsen.util 2106 :info :add 8398 :RequestTimeoutException INFO [2022-01-10 08:39:58,624] jepsen worker 852 - jepsen.util 2852 :invoke :add 9685 INFO [2022-01-10 08:39:58,628] jepsen worker 106 - jepsen.util 3106 :invoke :add 9686 INFO [2022-01-10 08:39:58,631] jepsen worker 551 - jepsen.util 2551 :info :add 8399 :RequestTimeoutException INFO [2022-01-10 08:39:58,631] jepsen worker 933 - jepsen.util 2933 :invoke :add 9687 INFO [2022-01-10 08:39:58,634] jepsen worker 942 - jepsen.util 2942 :info :add 8491 :SyncWriteAmbiguous INFO [2022-01-10 08:39:58,634] jepsen worker 837 - jepsen.util 2837 :info :add 8473 :SyncWriteAmbiguous INFO [2022-01-10 08:39:58,636] jepsen worker 551 - jepsen.util 3551 :invoke :add 9688 INFO [2022-01-10 08:39:58,640] jepsen worker 620 - jepsen.util 3620 :invoke :add 9689 INFO [2022-01-10 08:39:58,643] jepsen worker 310 - jepsen.util 3310 :invoke :add 9690 INFO [2022-01-10 08:39:58,644] jepsen worker 998 - jepsen.util 2998 :info :add 8400 :RequestTimeoutException INFO [2022-01-10 08:39:58,645] jepsen worker 473 - jepsen.util 2473 :invoke :add 9691 INFO [2022-01-10 08:39:58,649] jepsen worker 942 - jepsen.util 3942 :invoke :add 9692 INFO [2022-01-10 08:39:58,651] jepsen worker 998 - jepsen.util 3998 :invoke :add 9693 INFO [2022-01-10 08:39:58,654] jepsen worker 434 - jepsen.util 2434 :invoke :add 9694 INFO [2022-01-10 08:39:58,657] jepsen worker 837 - jepsen.util 3837 :invoke :add 9695 INFO [2022-01-10 08:39:58,663] jepsen worker 645 - jepsen.util 2645 :info :add 8401 :RequestTimeoutException INFO [2022-01-10 08:39:58,666] jepsen worker 645 - jepsen.util 3645 :invoke :add 9696 INFO [2022-01-10 08:39:58,667] jepsen worker 608 - jepsen.util 2608 :info :add 8402 :RequestTimeoutException INFO [2022-01-10 08:39:58,671] jepsen worker 249 - jepsen.util 2249 :info :add 8403 :RequestTimeoutException INFO [2022-01-10 08:39:58,672] jepsen worker 608 - jepsen.util 3608 :invoke :add 9697 INFO [2022-01-10 08:39:58,675] jepsen worker 249 - jepsen.util 3249 :invoke :add 9698 INFO [2022-01-10 08:39:58,701] jepsen worker 633 - jepsen.util 1633 :info :add 8408 :RequestTimeoutException INFO [2022-01-10 08:39:58,706] jepsen worker 633 - jepsen.util 2633 :invoke :add 9699 INFO [2022-01-10 08:39:58,736] jepsen worker 550 - jepsen.util 2550 :info :add 8409 :RequestTimeoutException INFO [2022-01-10 08:39:58,740] jepsen worker 595 - jepsen.util 2595 :info :add 8410 :RequestTimeoutException INFO [2022-01-10 08:39:58,741] jepsen worker 550 - jepsen.util 3550 :invoke :add 9700 INFO [2022-01-10 08:39:58,743] jepsen worker 234 - jepsen.util 2234 :info :add 8411 :RequestTimeoutException INFO [2022-01-10 08:39:58,746] jepsen worker 595 - jepsen.util 3595 :invoke :add 9701 INFO [2022-01-10 08:39:58,749] jepsen worker 234 - jepsen.util 3234 :invoke :add 9702 INFO [2022-01-10 08:39:58,751] jepsen worker 692 - jepsen.util 2692 :info :add 8412 :RequestTimeoutException INFO [2022-01-10 08:39:58,753] jepsen worker 991 - jepsen.util 2991 :info :add 8413 :RequestTimeoutException INFO [2022-01-10 08:39:58,758] jepsen worker 692 - jepsen.util 3692 :invoke :add 9703 INFO [2022-01-10 08:39:58,761] jepsen worker 991 - jepsen.util 3991 :invoke :add 9704 INFO [2022-01-10 08:39:58,832] jepsen worker 427 - jepsen.util 2427 :info :add 8414 :RequestTimeoutException INFO [2022-01-10 08:39:58,836] jepsen worker 427 - jepsen.util 3427 :invoke :add 9705 INFO [2022-01-10 08:39:58,867] jepsen worker 953 - jepsen.util 2953 :info :add 8415 :RequestTimeoutException INFO [2022-01-10 08:39:58,870] jepsen worker 201 - jepsen.util 2201 :info :add 8416 :RequestTimeoutException INFO [2022-01-10 08:39:58,871] jepsen worker 953 - jepsen.util 3953 :invoke :add 9706 INFO [2022-01-10 08:39:58,873] jepsen worker 504 - jepsen.util 2504 :ok :add 9624 INFO [2022-01-10 08:39:58,873] jepsen worker 926 - jepsen.util 3926 :ok :add 9561 INFO [2022-01-10 08:39:58,873] jepsen worker 647 - jepsen.util 2647 :ok :add 9555 INFO [2022-01-10 08:39:58,873] jepsen worker 48 - jepsen.util 3048 :ok :add 9610 INFO [2022-01-10 08:39:58,873] jepsen worker 728 - jepsen.util 3728 :ok :add 9608 INFO [2022-01-10 08:39:58,873] jepsen worker 600 - jepsen.util 3600 :ok :add 9579 INFO [2022-01-10 08:39:58,875] jepsen worker 201 - jepsen.util 3201 :invoke :add 9707 INFO [2022-01-10 08:39:58,879] jepsen worker 728 - jepsen.util 3728 :invoke :add 9708 INFO [2022-01-10 08:39:58,882] jepsen worker 504 - jepsen.util 2504 :invoke :add 9709 INFO [2022-01-10 08:39:58,884] jepsen worker 48 - jepsen.util 3048 :invoke :add 9710 INFO [2022-01-10 08:39:58,887] jepsen worker 926 - jepsen.util 3926 :invoke :add 9711 INFO [2022-01-10 08:39:58,890] jepsen worker 647 - jepsen.util 2647 :invoke :add 9712 INFO [2022-01-10 08:39:58,891] jepsen worker 455 - jepsen.util 1455 :info :add 8418 :RequestTimeoutException INFO [2022-01-10 08:39:58,893] jepsen worker 600 - jepsen.util 3600 :invoke :add 9713 INFO [2022-01-10 08:39:58,893] jepsen worker 767 - jepsen.util 1767 :info :add 8417 :RequestTimeoutException INFO [2022-01-10 08:39:58,893] jepsen worker 325 - jepsen.util 1325 :info :add 8419 :RequestTimeoutException INFO [2022-01-10 08:39:58,900] jepsen worker 767 - jepsen.util 2767 :invoke :add 9714 INFO [2022-01-10 08:39:58,900] jepsen worker 219 - jepsen.util 1219 :info :add 8422 :RequestTimeoutException INFO [2022-01-10 08:39:58,902] jepsen worker 193 - jepsen.util 1193 :info :add 8420 :RequestTimeoutException INFO [2022-01-10 08:39:58,902] jepsen worker 455 - jepsen.util 2455 :invoke :add 9715 INFO [2022-01-10 08:39:58,907] jepsen worker 493 - jepsen.util 1493 :info :add 8421 :RequestTimeoutException INFO [2022-01-10 08:39:58,907] jepsen worker 325 - jepsen.util 2325 :invoke :add 9716 INFO [2022-01-10 08:39:58,911] jepsen worker 193 - jepsen.util 2193 :invoke :add 9717 INFO [2022-01-10 08:39:58,913] jepsen worker 493 - jepsen.util 2493 :invoke :add 9718 INFO [2022-01-10 08:39:58,916] jepsen worker 219 - jepsen.util 2219 :invoke :add 9719 INFO [2022-01-10 08:39:58,951] jepsen worker 756 - jepsen.util 2756 :info :add 8424 :RequestTimeoutException INFO [2022-01-10 08:39:58,953] jepsen worker 91 - jepsen.util 2091 :info :add 8423 :RequestTimeoutException INFO [2022-01-10 08:39:58,955] jepsen worker 756 - jepsen.util 3756 :invoke :add 9720 INFO [2022-01-10 08:39:58,959] jepsen worker 91 - jepsen.util 3091 :invoke :add 9721 INFO [2022-01-10 08:39:58,968] jepsen worker 916 - jepsen.util 2916 :ok :add 8773 INFO [2022-01-10 08:39:58,968] jepsen worker 76 - jepsen.util 1076 :ok :add 8592 INFO [2022-01-10 08:39:58,972] jepsen worker 44 - jepsen.util 2044 :info :add 8426 :RequestTimeoutException INFO [2022-01-10 08:39:58,972] jepsen worker 76 - jepsen.util 1076 :invoke :add 9722 INFO [2022-01-10 08:39:58,976] jepsen worker 44 - jepsen.util 3044 :invoke :add 9723 INFO [2022-01-10 08:39:58,979] jepsen worker 916 - jepsen.util 2916 :invoke :add 9724 INFO [2022-01-10 08:39:59,007] jepsen worker 258 - jepsen.util 2258 :info :add 8427 :RequestTimeoutException INFO [2022-01-10 08:39:59,012] jepsen worker 258 - jepsen.util 3258 :invoke :add 9725 INFO [2022-01-10 08:39:59,103] jepsen worker 749 - jepsen.util 1749 :info :add 8434 :RequestTimeoutException INFO [2022-01-10 08:39:59,108] jepsen worker 749 - jepsen.util 2749 :invoke :add 9726 INFO [2022-01-10 08:39:59,109] jepsen worker 867 - jepsen.util 2867 :ok :add 8928 INFO [2022-01-10 08:39:59,109] jepsen worker 787 - jepsen.util 1787 :ok :add 8564 INFO [2022-01-10 08:39:59,111] jepsen worker 636 - jepsen.util 636 :info :add 8437 :RequestTimeoutException INFO [2022-01-10 08:39:59,112] jepsen worker 835 - jepsen.util 835 :info :add 8435 :RequestTimeoutException INFO [2022-01-10 08:39:59,112] jepsen worker 399 - jepsen.util 1399 :info :add 8436 :RequestTimeoutException INFO [2022-01-10 08:39:59,112] jepsen worker 556 - jepsen.util 556 :info :add 8439 :RequestTimeoutException INFO [2022-01-10 08:39:59,112] jepsen worker 381 - jepsen.util 2381 :info :add 8438 :RequestTimeoutException INFO [2022-01-10 08:39:59,121] jepsen worker 835 - jepsen.util 1835 :invoke :add 9727 INFO [2022-01-10 08:39:59,123] jepsen worker 867 - jepsen.util 2867 :invoke :add 9728 INFO [2022-01-10 08:39:59,126] jepsen worker 556 - jepsen.util 1556 :invoke :add 9729 INFO [2022-01-10 08:39:59,129] jepsen worker 787 - jepsen.util 1787 :invoke :add 9730 INFO [2022-01-10 08:39:59,131] jepsen worker 636 - jepsen.util 1636 :invoke :add 9731 INFO [2022-01-10 08:39:59,133] jepsen worker 128 - jepsen.util 2128 :info :add 8445 :RequestTimeoutException INFO [2022-01-10 08:39:59,134] jepsen worker 820 - jepsen.util 1820 :info :add 8444 :RequestTimeoutException INFO [2022-01-10 08:39:59,134] jepsen worker 381 - jepsen.util 3381 :invoke :add 9732 INFO [2022-01-10 08:39:59,139] jepsen worker 128 - jepsen.util 3128 :invoke :add 9733 INFO [2022-01-10 08:39:59,142] jepsen worker 820 - jepsen.util 2820 :invoke :add 9734 INFO [2022-01-10 08:39:59,144] jepsen worker 399 - jepsen.util 2399 :invoke :add 9735 INFO [2022-01-10 08:39:59,223] jepsen worker 536 - jepsen.util 2536 :info :add 8447 :RequestTimeoutException INFO [2022-01-10 08:39:59,227] jepsen worker 536 - jepsen.util 3536 :invoke :add 9736 INFO [2022-01-10 08:39:59,231] jepsen worker 10 - jepsen.util 2010 :info :add 8448 :RequestTimeoutException INFO [2022-01-10 08:39:59,233] jepsen worker 297 - jepsen.util 2297 :info :add 8449 :RequestTimeoutException INFO [2022-01-10 08:39:59,238] jepsen worker 10 - jepsen.util 3010 :invoke :add 9737 INFO [2022-01-10 08:39:59,240] jepsen worker 297 - jepsen.util 3297 :invoke :add 9738 INFO [2022-01-10 08:39:59,263] jepsen worker 286 - jepsen.util 2286 :ok :add 8905 INFO [2022-01-10 08:39:59,269] jepsen worker 286 - jepsen.util 2286 :invoke :add 9739 INFO [2022-01-10 08:39:59,292] jepsen worker 984 - jepsen.util 2984 :info :add 8450 :RequestTimeoutException INFO [2022-01-10 08:39:59,297] jepsen worker 305 - jepsen.util 2305 :info :add 8451 :RequestTimeoutException INFO [2022-01-10 08:39:59,300] jepsen worker 312 - jepsen.util 2312 :info :add 8452 :RequestTimeoutException INFO [2022-01-10 08:39:59,301] jepsen worker 984 - jepsen.util 3984 :invoke :add 9740 INFO [2022-01-10 08:39:59,303] jepsen worker 63 - jepsen.util 2063 :info :add 8453 :RequestTimeoutException INFO [2022-01-10 08:39:59,303] jepsen worker 0 - jepsen.util 2000 :info :add 8744 :SyncWriteAmbiguous INFO [2022-01-10 08:39:59,311] jepsen worker 341 - jepsen.util 2341 :info :add 8454 :RequestTimeoutException INFO [2022-01-10 08:39:59,311] jepsen worker 63 - jepsen.util 3063 :invoke :add 9741 INFO [2022-01-10 08:39:59,314] jepsen worker 263 - jepsen.util 1263 :ok :add 8535 INFO [2022-01-10 08:39:59,316] jepsen worker 433 - jepsen.util 2433 :ok :add 8489 INFO [2022-01-10 08:39:59,316] jepsen worker 302 - jepsen.util 1302 :ok :add 8501 INFO [2022-01-10 08:39:59,317] jepsen worker 305 - jepsen.util 3305 :invoke :add 9742 INFO [2022-01-10 08:39:59,317] jepsen worker 26 - jepsen.util 2026 :info :add 8746 :SyncWriteAmbiguous INFO [2022-01-10 08:39:59,318] jepsen worker 665 - jepsen.util 1665 :info :add 8771 :SyncWriteAmbiguous INFO [2022-01-10 08:39:59,322] jepsen worker 922 - jepsen.util 2922 :info :add 8456 :RequestTimeoutException INFO [2022-01-10 08:39:59,323] jepsen worker 828 - jepsen.util 2828 :info :add 8455 :RequestTimeoutException INFO [2022-01-10 08:39:59,331] jepsen worker 828 - jepsen.util 3828 :invoke :add 9743 INFO [2022-01-10 08:39:59,334] jepsen worker 26 - jepsen.util 3026 :invoke :add 9744 INFO [2022-01-10 08:39:59,337] jepsen worker 665 - jepsen.util 2665 :invoke :add 9745 INFO [2022-01-10 08:39:59,341] jepsen worker 312 - jepsen.util 3312 :invoke :add 9746 INFO [2022-01-10 08:39:59,344] jepsen worker 263 - jepsen.util 1263 :invoke :add 9747 INFO [2022-01-10 08:39:59,347] jepsen worker 302 - jepsen.util 1302 :invoke :add 9748 INFO [2022-01-10 08:39:59,350] jepsen worker 433 - jepsen.util 2433 :invoke :add 9749 INFO [2022-01-10 08:39:59,353] jepsen worker 922 - jepsen.util 3922 :invoke :add 9750 INFO [2022-01-10 08:39:59,355] jepsen worker 0 - jepsen.util 3000 :invoke :add 9751 INFO [2022-01-10 08:39:59,357] jepsen worker 117 - jepsen.util 2117 :info :add 8457 :RequestTimeoutException INFO [2022-01-10 08:39:59,358] jepsen worker 341 - jepsen.util 3341 :invoke :add 9752 INFO [2022-01-10 08:39:59,363] jepsen worker 117 - jepsen.util 3117 :invoke :add 9753 INFO [2022-01-10 08:39:59,371] jepsen worker 179 - jepsen.util 2179 :info :add 8460 :RequestTimeoutException INFO [2022-01-10 08:39:59,375] jepsen worker 179 - jepsen.util 3179 :invoke :add 9754 INFO [2022-01-10 08:39:59,382] jepsen worker 680 - jepsen.util 1680 :info :add 8462 :RequestTimeoutException INFO [2022-01-10 08:39:59,383] jepsen worker 412 - jepsen.util 2412 :info :add 8461 :RequestTimeoutException INFO [2022-01-10 08:39:59,386] jepsen worker 831 - jepsen.util 1831 :info :add 8463 :RequestTimeoutException INFO [2022-01-10 08:39:59,389] jepsen worker 412 - jepsen.util 3412 :invoke :add 9755 INFO [2022-01-10 08:39:59,390] jepsen worker 90 - jepsen.util 1090 :info :add 8464 :RequestTimeoutException INFO [2022-01-10 08:39:59,391] jepsen worker 155 - jepsen.util 1155 :info :add 8466 :RequestTimeoutException INFO [2022-01-10 08:39:59,392] jepsen worker 678 - jepsen.util 1678 :info :add 8465 :RequestTimeoutException INFO [2022-01-10 08:39:59,393] jepsen worker 717 - jepsen.util 1717 :info :add 8467 :RequestTimeoutException INFO [2022-01-10 08:39:59,398] jepsen worker 831 - jepsen.util 2831 :invoke :add 9756 INFO [2022-01-10 08:39:59,400] jepsen worker 680 - jepsen.util 2680 :invoke :add 9757 INFO [2022-01-10 08:39:59,402] jepsen worker 12 - jepsen.util 1012 :info :add 8468 :RequestTimeoutException INFO [2022-01-10 08:39:59,403] jepsen worker 612 - jepsen.util 1612 :info :add 8471 :RequestTimeoutException INFO [2022-01-10 08:39:59,403] jepsen worker 678 - jepsen.util 2678 :invoke :add 9758 INFO [2022-01-10 08:39:59,408] jepsen worker 612 - jepsen.util 2612 :invoke :add 9759 INFO [2022-01-10 08:39:59,410] jepsen worker 482 - jepsen.util 1482 :info :add 8470 :RequestTimeoutException INFO [2022-01-10 08:39:59,411] jepsen worker 721 - jepsen.util 1721 :info :add 8472 :RequestTimeoutException INFO [2022-01-10 08:39:59,412] jepsen worker 717 - jepsen.util 2717 :invoke :add 9760 INFO [2022-01-10 08:39:59,417] jepsen worker 155 - jepsen.util 2155 :invoke :add 9761 INFO [2022-01-10 08:39:59,419] jepsen worker 90 - jepsen.util 2090 :invoke :add 9762 INFO [2022-01-10 08:39:59,422] jepsen worker 482 - jepsen.util 2482 :invoke :add 9763 INFO [2022-01-10 08:39:59,425] jepsen worker 12 - jepsen.util 2012 :invoke :add 9764 INFO [2022-01-10 08:39:59,427] jepsen worker 721 - jepsen.util 2721 :invoke :add 9765 INFO [2022-01-10 08:39:59,441] jepsen worker 480 - jepsen.util 2480 :info :add 8478 :RequestTimeoutException INFO [2022-01-10 08:39:59,447] jepsen worker 480 - jepsen.util 3480 :invoke :add 9766 INFO [2022-01-10 08:39:59,452] jepsen worker 784 - jepsen.util 2784 :info :add 8480 :RequestTimeoutException INFO [2022-01-10 08:39:59,453] jepsen worker 65 - jepsen.util 2065 :info :add 8479 :RequestTimeoutException INFO [2022-01-10 08:39:59,457] jepsen worker 773 - jepsen.util 2773 :info :add 8481 :RequestTimeoutException INFO [2022-01-10 08:39:59,458] jepsen worker 65 - jepsen.util 3065 :invoke :add 9767 INFO [2022-01-10 08:39:59,460] jepsen worker 252 - jepsen.util 2252 :info :add 8482 :RequestTimeoutException INFO [2022-01-10 08:39:59,462] jepsen worker 773 - jepsen.util 3773 :invoke :add 9768 INFO [2022-01-10 08:39:59,467] jepsen worker 252 - jepsen.util 3252 :invoke :add 9769 INFO [2022-01-10 08:39:59,470] jepsen worker 784 - jepsen.util 3784 :invoke :add 9770 INFO [2022-01-10 08:39:59,473] jepsen worker 836 - jepsen.util 2836 :info :add 8483 :RequestTimeoutException INFO [2022-01-10 08:39:59,477] jepsen worker 836 - jepsen.util 3836 :invoke :add 9771 INFO [2022-01-10 08:39:59,522] jepsen worker 759 - jepsen.util 2759 :info :add 8484 :RequestTimeoutException INFO [2022-01-10 08:39:59,523] jepsen worker 269 - jepsen.util 2269 :info :add 8485 :RequestTimeoutException INFO [2022-01-10 08:39:59,525] jepsen worker 759 - jepsen.util 3759 :invoke :add 9772 INFO [2022-01-10 08:39:59,529] jepsen worker 269 - jepsen.util 3269 :invoke :add 9773 INFO [2022-01-10 08:39:59,532] jepsen worker 992 - jepsen.util 2992 :info :add 8486 :RequestTimeoutException INFO [2022-01-10 08:39:59,535] jepsen worker 992 - jepsen.util 3992 :invoke :add 9774 INFO [2022-01-10 08:39:59,537] jepsen worker 985 - jepsen.util 2985 :info :add 8487 :RequestTimeoutException INFO [2022-01-10 08:39:59,540] jepsen worker 723 - jepsen.util 2723 :info :add 8488 :RequestTimeoutException INFO [2022-01-10 08:39:59,541] jepsen worker 985 - jepsen.util 3985 :invoke :add 9775 INFO [2022-01-10 08:39:59,544] jepsen worker 723 - jepsen.util 3723 :invoke :add 9776 INFO [2022-01-10 08:39:59,581] jepsen worker 428 - jepsen.util 2428 :info :add 8490 :RequestTimeoutException INFO [2022-01-10 08:39:59,587] jepsen worker 428 - jepsen.util 3428 :invoke :add 9777 INFO [2022-01-10 08:39:59,592] jepsen worker 673 - jepsen.util 2673 :info :add 8492 :RequestTimeoutException INFO [2022-01-10 08:39:59,596] jepsen worker 673 - jepsen.util 3673 :invoke :add 9778 INFO [2022-01-10 08:39:59,643] jepsen worker 34 - jepsen.util 1034 :info :add 8498 :RequestTimeoutException INFO [2022-01-10 08:39:59,648] jepsen worker 34 - jepsen.util 2034 :invoke :add 9779 INFO [2022-01-10 08:39:59,650] jepsen worker 970 - jepsen.util 2970 :ok :add 9560 INFO [2022-01-10 08:39:59,650] jepsen worker 650 - jepsen.util 2650 :ok :add 9578 INFO [2022-01-10 08:39:59,650] jepsen worker 967 - jepsen.util 3967 :ok :add 9554 INFO [2022-01-10 08:39:59,651] jepsen worker 290 - jepsen.util 3290 :ok :add 9625 INFO [2022-01-10 08:39:59,651] jepsen worker 485 - jepsen.util 2485 :ok :add 9611 INFO [2022-01-10 08:39:59,651] jepsen worker 335 - jepsen.util 3335 :ok :add 9609 INFO [2022-01-10 08:39:59,655] jepsen worker 290 - jepsen.util 3290 :invoke :add 9780 INFO [2022-01-10 08:39:59,657] jepsen worker 335 - jepsen.util 3335 :invoke :add 9781 INFO [2022-01-10 08:39:59,660] jepsen worker 650 - jepsen.util 2650 :invoke :add 9782 INFO [2022-01-10 08:39:59,663] jepsen worker 970 - jepsen.util 2970 :invoke :add 9783 INFO [2022-01-10 08:39:59,665] jepsen worker 967 - jepsen.util 3967 :invoke :add 9784 INFO [2022-01-10 08:39:59,667] jepsen worker 904 - jepsen.util 1904 :info :add 8499 :RequestTimeoutException INFO [2022-01-10 08:39:59,668] jepsen worker 485 - jepsen.util 2485 :invoke :add 9785 INFO [2022-01-10 08:39:59,671] jepsen worker 904 - jepsen.util 2904 :invoke :add 9786 INFO [2022-01-10 08:39:59,672] jepsen worker 383 - jepsen.util 383 :info :add 8502 :RequestTimeoutException INFO [2022-01-10 08:39:59,672] jepsen worker 277 - jepsen.util 2277 :info :add 8500 :RequestTimeoutException INFO [2022-01-10 08:39:59,679] jepsen worker 383 - jepsen.util 1383 :invoke :add 9787 INFO [2022-01-10 08:39:59,681] jepsen worker 317 - jepsen.util 2317 :info :add 8503 :RequestTimeoutException INFO [2022-01-10 08:39:59,682] jepsen worker 277 - jepsen.util 3277 :invoke :add 9788 INFO [2022-01-10 08:39:59,683] jepsen worker 711 - jepsen.util 2711 :info :add 8508 :RequestTimeoutException INFO [2022-01-10 08:39:59,688] jepsen worker 711 - jepsen.util 3711 :invoke :add 9789 INFO [2022-01-10 08:39:59,691] jepsen worker 317 - jepsen.util 3317 :invoke :add 9790 INFO [2022-01-10 08:39:59,691] jepsen worker 944 - jepsen.util 944 :info :add 8512 :RequestTimeoutException INFO [2022-01-10 08:39:59,692] jepsen worker 955 - jepsen.util 1955 :info :add 8509 :RequestTimeoutException INFO [2022-01-10 08:39:59,692] jepsen worker 64 - jepsen.util 2064 :info :add 8511 :RequestTimeoutException INFO [2022-01-10 08:39:59,693] jepsen worker 176 - jepsen.util 1176 :info :add 8510 :RequestTimeoutException INFO [2022-01-10 08:39:59,699] jepsen worker 890 - jepsen.util 890 :info :add 8515 :RequestTimeoutException INFO [2022-01-10 08:39:59,700] jepsen worker 203 - jepsen.util 203 :info :add 8513 :RequestTimeoutException INFO [2022-01-10 08:39:59,702] jepsen worker 502 - jepsen.util 1502 :info :add 8514 :RequestTimeoutException INFO [2022-01-10 08:39:59,703] jepsen worker 797 - jepsen.util 1797 :info :add 8517 :RequestTimeoutException INFO [2022-01-10 08:39:59,703] jepsen worker 876 - jepsen.util 1876 :info :add 8516 :RequestTimeoutException INFO [2022-01-10 08:39:59,707] jepsen worker 402 - jepsen.util 1402 :info :add 8518 :RequestTimeoutException INFO [2022-01-10 08:39:59,712] jepsen worker 659 - jepsen.util 2659 :info :add 8519 :RequestTimeoutException INFO [2022-01-10 08:39:59,712] jepsen worker 890 - jepsen.util 1890 :invoke :add 9791 INFO [2022-01-10 08:39:59,712] jepsen worker 236 - jepsen.util 236 :info :add 8520 :RequestTimeoutException INFO [2022-01-10 08:39:59,717] jepsen worker 573 - jepsen.util 2573 :info :add 8521 :RequestTimeoutException INFO [2022-01-10 08:39:59,718] jepsen worker 502 - jepsen.util 2502 :invoke :add 9792 INFO [2022-01-10 08:39:59,720] jepsen worker 848 - jepsen.util 1848 :info :add 8523 :RequestTimeoutException INFO [2022-01-10 08:39:59,722] jepsen worker 396 - jepsen.util 1396 :info :add 8522 :RequestTimeoutException INFO [2022-01-10 08:39:59,722] jepsen worker 659 - jepsen.util 3659 :invoke :add 9793 INFO [2022-01-10 08:39:59,722] jepsen worker 481 - jepsen.util 481 :info :add 8524 :RequestTimeoutException INFO [2022-01-10 08:39:59,730] jepsen worker 848 - jepsen.util 2848 :invoke :add 9794 INFO [2022-01-10 08:39:59,732] jepsen worker 457 - jepsen.util 457 :info :add 8525 :RequestTimeoutException INFO [2022-01-10 08:39:59,732] jepsen worker 236 - jepsen.util 1236 :invoke :add 9795 INFO [2022-01-10 08:39:59,733] jepsen worker 299 - jepsen.util 2299 :info :add 8526 :RequestTimeoutException INFO [2022-01-10 08:39:59,737] jepsen worker 396 - jepsen.util 2396 :invoke :add 9796 INFO [2022-01-10 08:39:59,739] jepsen worker 299 - jepsen.util 3299 :invoke :add 9797 INFO [2022-01-10 08:39:59,740] jepsen worker 365 - jepsen.util 2365 :info :add 8527 :RequestTimeoutException INFO [2022-01-10 08:39:59,741] jepsen worker 955 - jepsen.util 2955 :invoke :add 9798 INFO [2022-01-10 08:39:59,745] jepsen worker 573 - jepsen.util 3573 :invoke :add 9799 INFO [2022-01-10 08:39:59,748] jepsen worker 876 - jepsen.util 2876 :invoke :add 9800 INFO [2022-01-10 08:39:59,750] jepsen worker 481 - jepsen.util 1481 :invoke :add 9801 INFO [2022-01-10 08:39:59,753] jepsen worker 365 - jepsen.util 3365 :invoke :add 9802 INFO [2022-01-10 08:39:59,756] jepsen worker 64 - jepsen.util 3064 :invoke :add 9803 INFO [2022-01-10 08:39:59,757] jepsen worker 887 - jepsen.util 1887 :info :add 8534 :RequestTimeoutException INFO [2022-01-10 08:39:59,759] jepsen worker 797 - jepsen.util 2797 :invoke :add 9804 INFO [2022-01-10 08:39:59,760] jepsen worker 718 - jepsen.util 1718 :info :add 8538 :RequestTimeoutException INFO [2022-01-10 08:39:59,760] jepsen worker 474 - jepsen.util 474 :info :add 8539 :RequestTimeoutException INFO [2022-01-10 08:39:59,762] jepsen worker 587 - jepsen.util 2587 :info :add 8537 :RequestTimeoutException INFO [2022-01-10 08:39:59,762] jepsen worker 402 - jepsen.util 2402 :invoke :add 9805 INFO [2022-01-10 08:39:59,764] jepsen worker 130 - jepsen.util 2130 :info :add 8536 :RequestTimeoutException INFO [2022-01-10 08:39:59,769] jepsen worker 718 - jepsen.util 2718 :invoke :add 9806 INFO [2022-01-10 08:39:59,770] jepsen worker 817 - jepsen.util 2817 :info :add 8544 :RequestTimeoutException INFO [2022-01-10 08:39:59,772] jepsen worker 474 - jepsen.util 1474 :invoke :add 9807 INFO [2022-01-10 08:39:59,776] jepsen worker 532 - jepsen.util 1532 :info :add 8545 :RequestTimeoutException INFO [2022-01-10 08:39:59,777] jepsen worker 203 - jepsen.util 1203 :invoke :add 9808 INFO [2022-01-10 08:39:59,777] jepsen worker 629 - jepsen.util 629 :info :add 8546 :RequestTimeoutException INFO [2022-01-10 08:39:59,780] jepsen worker 47 - jepsen.util 1047 :info :add 8547 :RequestTimeoutException INFO [2022-01-10 08:39:59,781] jepsen worker 306 - jepsen.util 1306 :info :add 8548 :RequestTimeoutException INFO [2022-01-10 08:39:59,781] jepsen worker 176 - jepsen.util 2176 :invoke :add 9809 INFO [2022-01-10 08:39:59,787] jepsen worker 944 - jepsen.util 1944 :invoke :add 9810 INFO [2022-01-10 08:39:59,787] jepsen worker 287 - jepsen.util 1287 :info :add 8550 :RequestTimeoutException INFO [2022-01-10 08:39:59,789] jepsen worker 47 - jepsen.util 2047 :invoke :add 9811 INFO [2022-01-10 08:39:59,790] jepsen worker 705 - jepsen.util 705 :info :add 8549 :RequestTimeoutException INFO [2022-01-10 08:39:59,791] jepsen worker 727 - jepsen.util 727 :info :add 8551 :RequestTimeoutException INFO [2022-01-10 08:39:59,793] jepsen worker 306 - jepsen.util 2306 :invoke :add 9812 INFO [2022-01-10 08:39:59,793] jepsen worker 358 - jepsen.util 358 :info :add 8554 :RequestTimeoutException INFO [2022-01-10 08:39:59,794] jepsen worker 865 - jepsen.util 1865 :info :add 8553 :RequestTimeoutException INFO [2022-01-10 08:39:59,800] jepsen worker 950 - jepsen.util 950 :info :add 8555 :RequestTimeoutException INFO [2022-01-10 08:39:59,800] jepsen worker 358 - jepsen.util 1358 :invoke :add 9813 INFO [2022-01-10 08:39:59,801] jepsen worker 712 - jepsen.util 2712 :info :add 8556 :RequestTimeoutException INFO [2022-01-10 08:39:59,802] jepsen worker 800 - jepsen.util 2800 :info :add 8557 :RequestTimeoutException INFO [2022-01-10 08:39:59,803] jepsen worker 287 - jepsen.util 2287 :invoke :add 9814 INFO [2022-01-10 08:39:59,808] jepsen worker 130 - jepsen.util 3130 :invoke :add 9815 INFO [2022-01-10 08:39:59,811] jepsen worker 532 - jepsen.util 2532 :invoke :add 9816 INFO [2022-01-10 08:39:59,813] jepsen worker 598 - jepsen.util 1598 :info :add 8560 :RequestTimeoutException INFO [2022-01-10 08:39:59,813] jepsen worker 914 - jepsen.util 2914 :info :add 8561 :RequestTimeoutException INFO [2022-01-10 08:39:59,814] jepsen worker 705 - jepsen.util 1705 :invoke :add 9817 INFO [2022-01-10 08:39:59,817] jepsen worker 54 - jepsen.util 54 :info :add 8562 :RequestTimeoutException INFO [2022-01-10 08:39:59,819] jepsen worker 865 - jepsen.util 2865 :invoke :add 9818 INFO [2022-01-10 08:39:59,823] jepsen worker 914 - jepsen.util 3914 :invoke :add 9819 INFO [2022-01-10 08:39:59,824] jepsen worker 331 - jepsen.util 1331 :info :add 8563 :RequestTimeoutException INFO [2022-01-10 08:39:59,826] jepsen worker 800 - jepsen.util 3800 :invoke :add 9820 INFO [2022-01-10 08:39:59,826] jepsen worker 353 - jepsen.util 1353 :info :add 8565 :RequestTimeoutException INFO [2022-01-10 08:39:59,830] jepsen worker 515 - jepsen.util 3515 :ok :add 9580 INFO [2022-01-10 08:39:59,831] jepsen worker 806 - jepsen.util 3806 :ok :add 9598 INFO [2022-01-10 08:39:59,831] jepsen worker 239 - jepsen.util 2239 :info :add 8568 :RequestTimeoutException INFO [2022-01-10 08:39:59,832] jepsen worker 817 - jepsen.util 3817 :invoke :add 9821 INFO [2022-01-10 08:39:59,832] jepsen worker 709 - jepsen.util 1709 :info :add 8567 :RequestTimeoutException INFO [2022-01-10 08:39:59,833] jepsen worker 362 - jepsen.util 1362 :info :add 8570 :RequestTimeoutException INFO [2022-01-10 08:39:59,834] jepsen worker 661 - jepsen.util 1661 :info :add 8569 :RequestTimeoutException INFO [2022-01-10 08:39:59,834] jepsen worker 255 - jepsen.util 255 :info :add 8571 :RequestTimeoutException INFO [2022-01-10 08:39:59,841] jepsen worker 661 - jepsen.util 2661 :invoke :add 9822 INFO [2022-01-10 08:39:59,843] jepsen worker 75 - jepsen.util 1075 :info :add 8572 :RequestTimeoutException INFO [2022-01-10 08:39:59,843] jepsen worker 416 - jepsen.util 2416 :info :add 8573 :RequestTimeoutException INFO [2022-01-10 08:39:59,845] jepsen worker 598 - jepsen.util 2598 :invoke :add 9823 INFO [2022-01-10 08:39:59,845] jepsen worker 119 - jepsen.util 119 :ok :add 8919 INFO [2022-01-10 08:39:59,845] jepsen worker 873 - jepsen.util 1873 :ok :add 9005 INFO [2022-01-10 08:39:59,850] jepsen worker 709 - jepsen.util 2709 :invoke :add 9824 INFO [2022-01-10 08:39:59,852] jepsen worker 255 - jepsen.util 1255 :invoke :add 9825 INFO [2022-01-10 08:39:59,855] jepsen worker 587 - jepsen.util 3587 :invoke :add 9826 INFO [2022-01-10 08:39:59,857] jepsen worker 356 - jepsen.util 1356 :info :add 8578 :RequestTimeoutException INFO [2022-01-10 08:39:59,858] jepsen worker 239 - jepsen.util 3239 :invoke :add 9827 INFO [2022-01-10 08:39:59,860] jepsen worker 202 - jepsen.util 202 :info :add 8580 :RequestTimeoutException INFO [2022-01-10 08:39:59,862] jepsen worker 727 - jepsen.util 1727 :invoke :add 9828 INFO [2022-01-10 08:39:59,863] jepsen worker 512 - jepsen.util 1512 :info :add 8581 :RequestTimeoutException INFO [2022-01-10 08:39:59,863] jepsen worker 350 - jepsen.util 2350 :info :add 8579 :RequestTimeoutException INFO [2022-01-10 08:39:59,863] jepsen worker 683 - jepsen.util 683 :info :add 8582 :RequestTimeoutException INFO [2022-01-10 08:39:59,869] jepsen worker 356 - jepsen.util 2356 :invoke :add 9829 INFO [2022-01-10 08:39:59,870] jepsen worker 570 - jepsen.util 1570 :info :add 8584 :RequestTimeoutException INFO [2022-01-10 08:39:59,871] jepsen worker 61 - jepsen.util 61 :info :add 8583 :RequestTimeoutException INFO [2022-01-10 08:39:59,872] jepsen worker 806 - jepsen.util 3806 :invoke :add 9830 INFO [2022-01-10 08:39:59,877] jepsen worker 389 - jepsen.util 389 :info :add 8585 :RequestTimeoutException INFO [2022-01-10 08:39:59,877] jepsen worker 969 - jepsen.util 1969 :info :add 8587 :RequestTimeoutException INFO [2022-01-10 08:39:59,877] jepsen worker 362 - jepsen.util 2362 :invoke :add 9831 INFO [2022-01-10 08:39:59,880] jepsen worker 388 - jepsen.util 1388 :info :add 8588 :RequestTimeoutException INFO [2022-01-10 08:39:59,880] jepsen worker 475 - jepsen.util 2475 :info :add 8590 :RequestTimeoutException INFO [2022-01-10 08:39:59,880] jepsen worker 775 - jepsen.util 2775 :info :add 8586 :RequestTimeoutException INFO [2022-01-10 08:39:59,882] jepsen worker 664 - jepsen.util 1664 :info :add 8591 :RequestTimeoutException INFO [2022-01-10 08:39:59,883] jepsen worker 457 - jepsen.util 1457 :invoke :add 9832 INFO [2022-01-10 08:39:59,884] jepsen worker 133 - jepsen.util 2133 :info :add 8589 :RequestTimeoutException INFO [2022-01-10 08:39:59,891] jepsen worker 915 - jepsen.util 915 :info :add 8593 :RequestTimeoutException INFO [2022-01-10 08:39:59,891] jepsen worker 475 - jepsen.util 3475 :invoke :add 9833 INFO [2022-01-10 08:39:59,895] jepsen worker 629 - jepsen.util 1629 :invoke :add 9834 INFO [2022-01-10 08:39:59,898] jepsen worker 873 - jepsen.util 1873 :invoke :add 9835 INFO [2022-01-10 08:39:59,900] jepsen worker 393 - jepsen.util 2393 :info :add 8598 :RequestTimeoutException INFO [2022-01-10 08:39:59,900] jepsen worker 61 - jepsen.util 1061 :invoke :add 9836 INFO [2022-01-10 08:39:59,903] jepsen worker 941 - jepsen.util 2941 :info :add 8600 :RequestTimeoutException INFO [2022-01-10 08:39:59,904] jepsen worker 416 - jepsen.util 3416 :invoke :add 9837 INFO [2022-01-10 08:39:59,907] jepsen worker 167 - jepsen.util 2167 :info :add 8599 :RequestTimeoutException INFO [2022-01-10 08:39:59,907] jepsen worker 422 - jepsen.util 422 :info :add 8601 :RequestTimeoutException INFO [2022-01-10 08:39:59,908] jepsen worker 969 - jepsen.util 2969 :invoke :add 9838 INFO [2022-01-10 08:39:59,910] jepsen worker 250 - jepsen.util 1250 :info :add 8602 :RequestTimeoutException INFO [2022-01-10 08:39:59,914] jepsen worker 167 - jepsen.util 3167 :invoke :add 9839 INFO [2022-01-10 08:39:59,916] jepsen worker 660 - jepsen.util 2660 :info :add 8603 :RequestTimeoutException INFO [2022-01-10 08:39:59,917] jepsen worker 887 - jepsen.util 2887 :invoke :add 9840 INFO [2022-01-10 08:39:59,921] jepsen worker 75 - jepsen.util 2075 :invoke :add 9841 INFO [2022-01-10 08:39:59,924] jepsen worker 237 - jepsen.util 237 :info :add 8608 :RequestTimeoutException INFO [2022-01-10 08:39:59,924] jepsen worker 202 - jepsen.util 1202 :invoke :add 9842 INFO [2022-01-10 08:39:59,924] jepsen worker 782 - jepsen.util 2782 :info :add 8609 :RequestTimeoutException INFO [2022-01-10 08:39:59,929] jepsen worker 570 - jepsen.util 2570 :invoke :add 9843 INFO [2022-01-10 08:39:59,931] jepsen worker 353 - jepsen.util 2353 :invoke :add 9844 INFO [2022-01-10 08:39:59,932] jepsen worker 833 - jepsen.util 1833 :info :add 8612 :RequestTimeoutException INFO [2022-01-10 08:39:59,935] jepsen worker 360 - jepsen.util 360 :info :add 8610 :RequestTimeoutException INFO [2022-01-10 08:39:59,935] jepsen worker 393 - jepsen.util 3393 :invoke :add 9845 INFO [2022-01-10 08:39:59,935] jepsen worker 911 - jepsen.util 1911 :info :add 8611 :RequestTimeoutException INFO [2022-01-10 08:39:59,937] jepsen worker 183 - jepsen.util 1183 :info :add 8613 :RequestTimeoutException INFO [2022-01-10 08:39:59,940] jepsen worker 751 - jepsen.util 2751 :info :add 8614 :RequestTimeoutException INFO [2022-01-10 08:39:59,944] jepsen worker 512 - jepsen.util 2512 :invoke :add 9846 INFO [2022-01-10 08:39:59,947] jepsen worker 782 - jepsen.util 3782 :invoke :add 9847 INFO [2022-01-10 08:39:59,947] jepsen worker 177 - jepsen.util 177 :info :add 8615 :RequestTimeoutException INFO [2022-01-10 08:39:59,951] jepsen worker 79 - jepsen.util 1079 :info :add 8616 :RequestTimeoutException INFO [2022-01-10 08:39:59,952] jepsen worker 660 - jepsen.util 3660 :invoke :add 9848 INFO [2022-01-10 08:39:59,953] jepsen worker 165 - jepsen.util 2165 :info :add 8617 :RequestTimeoutException INFO [2022-01-10 08:39:59,953] jepsen worker 700 - jepsen.util 1700 :info :add 8620 :RequestTimeoutException INFO [2022-01-10 08:39:59,953] jepsen worker 808 - jepsen.util 1808 :info :add 8618 :RequestTimeoutException INFO [2022-01-10 08:39:59,957] jepsen worker 602 - jepsen.util 2602 :info :add 8619 :RequestTimeoutException INFO [2022-01-10 08:39:59,957] jepsen worker 675 - jepsen.util 2675 :info :add 8621 :RequestTimeoutException INFO [2022-01-10 08:39:59,960] jepsen worker 534 - jepsen.util 534 :info :add 8622 :RequestTimeoutException INFO [2022-01-10 08:39:59,963] jepsen worker 668 - jepsen.util 1668 :info :add 8623 :RequestTimeoutException INFO [2022-01-10 08:39:59,965] jepsen worker 360 - jepsen.util 1360 :invoke :add 9849 INFO [2022-01-10 08:39:59,967] jepsen worker 690 - jepsen.util 690 :info :add 8625 :RequestTimeoutException INFO [2022-01-10 08:39:59,969] jepsen worker 775 - jepsen.util 3775 :invoke :add 9850 INFO [2022-01-10 08:39:59,971] jepsen worker 634 - jepsen.util 1634 :info :add 8624 :RequestTimeoutException INFO [2022-01-10 08:39:59,974] jepsen worker 829 - jepsen.util 2829 :info :add 8626 :RequestTimeoutException INFO [2022-01-10 08:39:59,974] jepsen worker 963 - jepsen.util 963 :info :add 8627 :RequestTimeoutException INFO [2022-01-10 08:39:59,974] jepsen worker 388 - jepsen.util 2388 :invoke :add 9851 INFO [2022-01-10 08:39:59,980] jepsen worker 515 - jepsen.util 3515 :invoke :add 9852 INFO [2022-01-10 08:39:59,983] jepsen worker 133 - jepsen.util 3133 :invoke :add 9853 INFO [2022-01-10 08:39:59,986] jepsen worker 177 - jepsen.util 1177 :invoke :add 9854 INFO [2022-01-10 08:39:59,987] jepsen worker 537 - jepsen.util 1537 :info :add 8634 :RequestTimeoutException INFO [2022-01-10 08:39:59,987] jepsen worker 540 - jepsen.util 2540 :info :add 8636 :RequestTimeoutException INFO [2022-01-10 08:39:59,989] jepsen worker 950 - jepsen.util 1950 :invoke :add 9855 INFO [2022-01-10 08:39:59,992] jepsen worker 591 - jepsen.util 2591 :info :add 8635 :RequestTimeoutException INFO [2022-01-10 08:39:59,994] jepsen worker 911 - jepsen.util 2911 :invoke :add 9856 INFO [2022-01-10 08:39:59,997] jepsen worker 906 - jepsen.util 2906 :info :add 8637 :RequestTimeoutException INFO [2022-01-10 08:39:59,998] jepsen worker 683 - jepsen.util 1683 :invoke :add 9857 INFO [2022-01-10 08:40:00,000] jepsen worker 146 - jepsen.util 146 :info :add 8638 :RequestTimeoutException INFO [2022-01-10 08:40:00,000] jepsen worker 927 - jepsen.util 2927 :info :add 8639 :RequestTimeoutException INFO [2022-01-10 08:40:00,002] jepsen worker 183 - jepsen.util 2183 :invoke :add 9858 INFO [2022-01-10 08:40:00,007] jepsen worker 59 - jepsen.util 1059 :info :add 8645 :RequestTimeoutException INFO [2022-01-10 08:40:00,007] jepsen worker 237 - jepsen.util 1237 :invoke :add 9859 INFO [2022-01-10 08:40:00,011] jepsen worker 517 - jepsen.util 2517 :info :add 8644 :RequestTimeoutException INFO [2022-01-10 08:40:00,011] jepsen worker 833 - jepsen.util 2833 :invoke :add 9860 INFO [2022-01-10 08:40:00,013] jepsen worker 280 - jepsen.util 1280 :info :add 8646 :RequestTimeoutException INFO [2022-01-10 08:40:00,013] jepsen worker 674 - jepsen.util 674 :info :add 8647 :RequestTimeoutException INFO [2022-01-10 08:40:00,014] jepsen worker 584 - jepsen.util 1584 :info :add 8648 :RequestTimeoutException INFO [2022-01-10 08:40:00,015] jepsen worker 808 - jepsen.util 2808 :invoke :add 9861 INFO [2022-01-10 08:40:00,021] jepsen worker 390 - jepsen.util 390 :info :add 8649 :RequestTimeoutException INFO [2022-01-10 08:40:00,021] jepsen worker 540 - jepsen.util 3540 :invoke :add 9862 INFO [2022-01-10 08:40:00,023] jepsen worker 23 - jepsen.util 1023 :info :add 8653 :RequestTimeoutException INFO [2022-01-10 08:40:00,023] jepsen worker 754 - jepsen.util 1754 :info :add 8651 :RequestTimeoutException INFO [2022-01-10 08:40:00,027] jepsen worker 319 - jepsen.util 1319 :info :add 8650 :RequestTimeoutException INFO [2022-01-10 08:40:00,027] jepsen worker 327 - jepsen.util 1327 :info :add 8652 :RequestTimeoutException INFO [2022-01-10 08:40:00,029] jepsen worker 517 - jepsen.util 3517 :invoke :add 9863 INFO [2022-01-10 08:40:00,030] jepsen worker 210 - jepsen.util 1210 :info :add 8654 :RequestTimeoutException INFO [2022-01-10 08:40:00,030] jepsen worker 461 - jepsen.util 1461 :info :add 8656 :RequestTimeoutException INFO [2022-01-10 08:40:00,032] jepsen worker 531 - jepsen.util 2531 :info :add 8655 :RequestTimeoutException INFO [2022-01-10 08:40:00,033] jepsen worker 519 - jepsen.util 1519 :info :add 8657 :RequestTimeoutException INFO [2022-01-10 08:40:00,037] jepsen worker 534 - jepsen.util 1534 :invoke :add 9864 INFO [2022-01-10 08:40:00,040] jepsen worker 54 - jepsen.util 1054 :invoke :add 9865 INFO [2022-01-10 08:40:00,041] jepsen worker 503 - jepsen.util 2503 :info :add 8661 :RequestTimeoutException INFO [2022-01-10 08:40:00,042] jepsen worker 963 - jepsen.util 1963 :invoke :add 9866 INFO [2022-01-10 08:40:00,044] jepsen worker 431 - jepsen.util 1431 :info :add 8663 :RequestTimeoutException INFO [2022-01-10 08:40:00,047] jepsen worker 699 - jepsen.util 2699 :info :add 8660 :RequestTimeoutException INFO [2022-01-10 08:40:00,047] jepsen worker 535 - jepsen.util 1535 :info :add 8664 :RequestTimeoutException INFO [2022-01-10 08:40:00,048] jepsen worker 165 - jepsen.util 3165 :invoke :add 9867 INFO [2022-01-10 08:40:00,050] jepsen worker 6 - jepsen.util 2006 :info :add 8662 :RequestTimeoutException INFO [2022-01-10 08:40:00,050] jepsen worker 529 - jepsen.util 1529 :info :add 8665 :RequestTimeoutException INFO [2022-01-10 08:40:00,051] jepsen worker 599 - jepsen.util 2599 :info :add 8667 :RequestTimeoutException INFO [2022-01-10 08:40:00,053] jepsen worker 703 - jepsen.util 2703 :info :add 8666 :RequestTimeoutException INFO [2022-01-10 08:40:00,053] jepsen worker 900 - jepsen.util 2900 :info :add 8668 :RequestTimeoutException INFO [2022-01-10 08:40:00,057] jepsen worker 295 - jepsen.util 1295 :info :add 8669 :RequestTimeoutException INFO [2022-01-10 08:40:00,060] jepsen worker 152 - jepsen.util 1152 :info :add 8670 :RequestTimeoutException INFO [2022-01-10 08:40:00,061] jepsen worker 537 - jepsen.util 2537 :invoke :add 9868 INFO [2022-01-10 08:40:00,062] jepsen worker 163 - jepsen.util 2163 :info :add 8671 :RequestTimeoutException INFO [2022-01-10 08:40:00,066] jepsen worker 829 - jepsen.util 3829 :invoke :add 9869 INFO [2022-01-10 08:40:00,067] jepsen worker 966 - jepsen.util 2966 :info :add 8672 :RequestTimeoutException INFO [2022-01-10 08:40:00,067] jepsen worker 145 - jepsen.util 1145 :info :add 8673 :RequestTimeoutException INFO [2022-01-10 08:40:00,069] jepsen worker 327 - jepsen.util 2327 :invoke :add 9870 INFO [2022-01-10 08:40:00,073] jepsen worker 961 - jepsen.util 1961 :info :add 8679 :RequestTimeoutException INFO [2022-01-10 08:40:00,073] jepsen worker 3 - jepsen.util 1003 :info :add 8678 :RequestTimeoutException INFO [2022-01-10 08:40:00,074] jepsen worker 119 - jepsen.util 119 :invoke :add 9871 INFO [2022-01-10 08:40:00,079] jepsen worker 961 - jepsen.util 2961 :invoke :add 9872 INFO [2022-01-10 08:40:00,080] jepsen worker 616 - jepsen.util 1616 :info :add 8681 :RequestTimeoutException INFO [2022-01-10 08:40:00,081] jepsen worker 29 - jepsen.util 2029 :info :add 8680 :RequestTimeoutException INFO [2022-01-10 08:40:00,082] jepsen worker 823 - jepsen.util 2823 :info :add 8684 :RequestTimeoutException INFO [2022-01-10 08:40:00,082] jepsen worker 675 - jepsen.util 3675 :invoke :add 9873 INFO [2022-01-10 08:40:00,084] jepsen worker 58 - jepsen.util 2058 :info :add 8683 :RequestTimeoutException INFO [2022-01-10 08:40:00,084] jepsen worker 377 - jepsen.util 1377 :info :add 8685 :RequestTimeoutException INFO [2022-01-10 08:40:00,087] jepsen worker 786 - jepsen.util 2786 :info :add 8682 :RequestTimeoutException INFO [2022-01-10 08:40:00,087] jepsen worker 777 - jepsen.util 777 :info :add 8686 :RequestTimeoutException INFO [2022-01-10 08:40:00,090] jepsen worker 580 - jepsen.util 2580 :info :add 8688 :RequestTimeoutException INFO [2022-01-10 08:40:00,093] jepsen worker 390 - jepsen.util 1390 :invoke :add 9874 INFO [2022-01-10 08:40:00,097] jepsen worker 754 - jepsen.util 2754 :invoke :add 9875 INFO [2022-01-10 08:40:00,097] jepsen worker 50 - jepsen.util 2050 :info :add 8687 :RequestTimeoutException INFO [2022-01-10 08:40:00,098] jepsen worker 954 - jepsen.util 1954 :info :add 8689 :RequestTimeoutException INFO [2022-01-10 08:40:00,099] jepsen worker 163 - jepsen.util 3163 :invoke :add 9876 INFO [2022-01-10 08:40:00,100] jepsen worker 883 - jepsen.util 2883 :info :add 8693 :RequestTimeoutException INFO [2022-01-10 08:40:00,102] jepsen worker 566 - jepsen.util 1566 :info :add 8690 :RequestTimeoutException INFO [2022-01-10 08:40:00,103] jepsen worker 158 - jepsen.util 1158 :info :add 8691 :RequestTimeoutException INFO [2022-01-10 08:40:00,107] jepsen worker 702 - jepsen.util 702 :info :add 8692 :RequestTimeoutException INFO [2022-01-10 08:40:00,108] jepsen worker 584 - jepsen.util 2584 :invoke :add 9877 INFO [2022-01-10 08:40:00,112] jepsen worker 422 - jepsen.util 1422 :invoke :add 9878 INFO [2022-01-10 08:40:00,113] jepsen worker 352 - jepsen.util 2352 :info :add 8698 :RequestTimeoutException INFO [2022-01-10 08:40:00,114] jepsen worker 999 - jepsen.util 999 :info :add 8699 :RequestTimeoutException INFO [2022-01-10 08:40:00,114] jepsen worker 146 - jepsen.util 1146 :invoke :add 9879 INFO [2022-01-10 08:40:00,119] jepsen worker 519 - jepsen.util 2519 :invoke :add 9880 INFO [2022-01-10 08:40:00,122] jepsen worker 908 - jepsen.util 2908 :info :add 8700 :RequestTimeoutException INFO [2022-01-10 08:40:00,122] jepsen worker 295 - jepsen.util 2295 :invoke :add 9881 INFO [2022-01-10 08:40:00,123] jepsen worker 886 - jepsen.util 2886 :info :add 8701 :RequestTimeoutException INFO [2022-01-10 08:40:00,127] jepsen worker 288 - jepsen.util 2288 :info :add 8702 :RequestTimeoutException INFO [2022-01-10 08:40:00,127] jepsen worker 23 - jepsen.util 2023 :invoke :add 9882 INFO [2022-01-10 08:40:00,131] jepsen worker 377 - jepsen.util 2377 :invoke :add 9883 INFO [2022-01-10 08:40:00,134] jepsen worker 668 - jepsen.util 2668 :invoke :add 9884 INFO [2022-01-10 08:40:00,137] jepsen worker 975 - jepsen.util 1975 :info :add 8708 :RequestTimeoutException INFO [2022-01-10 08:40:00,139] jepsen worker 431 - jepsen.util 2431 :invoke :add 9885 INFO [2022-01-10 08:40:00,142] jepsen worker 582 - jepsen.util 1582 :info :add 8710 :RequestTimeoutException INFO [2022-01-10 08:40:00,143] jepsen worker 617 - jepsen.util 2617 :info :add 8711 :RequestTimeoutException INFO [2022-01-10 08:40:00,143] jepsen worker 906 - jepsen.util 3906 :invoke :add 9886 INFO [2022-01-10 08:40:00,144] jepsen worker 409 - jepsen.util 1409 :info :add 8709 :RequestTimeoutException INFO [2022-01-10 08:40:00,144] jepsen worker 109 - jepsen.util 1109 :info :add 8712 :RequestTimeoutException INFO [2022-01-10 08:40:00,150] jepsen worker 689 - jepsen.util 2689 :info :add 8713 :RequestTimeoutException INFO [2022-01-10 08:40:00,152] jepsen worker 3 - jepsen.util 2003 :invoke :add 9887 INFO [2022-01-10 08:40:00,153] jepsen worker 363 - jepsen.util 2363 :info :add 8714 :RequestTimeoutException INFO [2022-01-10 08:40:00,155] jepsen worker 941 - jepsen.util 3941 :invoke :add 9888 INFO [2022-01-10 08:40:00,156] jepsen worker 89 - jepsen.util 2089 :info :add 8717 :RequestTimeoutException INFO [2022-01-10 08:40:00,160] jepsen worker 982 - jepsen.util 1982 :info :add 8715 :RequestTimeoutException INFO [2022-01-10 08:40:00,161] jepsen worker 952 - jepsen.util 1952 :info :add 8716 :RequestTimeoutException INFO [2022-01-10 08:40:00,161] jepsen worker 859 - jepsen.util 1859 :info :add 8718 :RequestTimeoutException INFO [2022-01-10 08:40:00,161] jepsen worker 714 - jepsen.util 2714 :info :add 8719 :RequestTimeoutException INFO [2022-01-10 08:40:00,161] jepsen worker 311 - jepsen.util 2311 :info :add 8720 :RequestTimeoutException INFO [2022-01-10 08:40:00,161] jepsen worker 288 - jepsen.util 3288 :invoke :add 9889 INFO [2022-01-10 08:40:00,167] jepsen worker 21 - jepsen.util 21 :info :add 8723 :RequestTimeoutException INFO [2022-01-10 08:40:00,169] jepsen worker 580 - jepsen.util 3580 :invoke :add 9890 INFO [2022-01-10 08:40:00,171] jepsen worker 323 - jepsen.util 2323 :info :add 8721 :RequestTimeoutException INFO [2022-01-10 08:40:00,173] jepsen worker 677 - jepsen.util 2677 :info :add 8722 :RequestTimeoutException INFO [2022-01-10 08:40:00,173] jepsen worker 952 - jepsen.util 2952 :invoke :add 9891 INFO [2022-01-10 08:40:00,173] jepsen worker 415 - jepsen.util 1415 :info :add 8724 :RequestTimeoutException INFO [2022-01-10 08:40:00,173] jepsen worker 93 - jepsen.util 2093 :info :add 8725 :RequestTimeoutException INFO [2022-01-10 08:40:00,177] jepsen worker 164 - jepsen.util 164 :info :add 8726 :RequestTimeoutException INFO [2022-01-10 08:40:00,182] jepsen worker 187 - jepsen.util 2187 :info :add 8727 :RequestTimeoutException INFO [2022-01-10 08:40:00,183] jepsen worker 712 - jepsen.util 3712 :invoke :add 9892 INFO [2022-01-10 08:40:00,189] jepsen worker 363 - jepsen.util 3363 :invoke :add 9893 INFO [2022-01-10 08:40:00,191] jepsen worker 999 - jepsen.util 1999 :invoke :add 9894 INFO [2022-01-10 08:40:00,193] jepsen worker 713 - jepsen.util 2713 :info :add 8734 :RequestTimeoutException INFO [2022-01-10 08:40:00,193] jepsen worker 522 - jepsen.util 2522 :info :add 8735 :RequestTimeoutException INFO [2022-01-10 08:40:00,194] jepsen worker 59 - jepsen.util 2059 :invoke :add 9895 INFO [2022-01-10 08:40:00,199] jepsen worker 634 - jepsen.util 2634 :invoke :add 9896 INFO [2022-01-10 08:40:00,202] jepsen worker 162 - jepsen.util 2162 :info :add 8736 :RequestTimeoutException INFO [2022-01-10 08:40:00,202] jepsen worker 966 - jepsen.util 3966 :invoke :add 9897 INFO [2022-01-10 08:40:00,207] jepsen worker 162 - jepsen.util 3162 :invoke :add 9898 INFO [2022-01-10 08:40:00,207] jepsen worker 115 - jepsen.util 115 :info :add 8737 :RequestTimeoutException INFO [2022-01-10 08:40:00,208] jepsen worker 373 - jepsen.util 1373 :info :add 8738 :RequestTimeoutException INFO [2022-01-10 08:40:00,210] jepsen worker 152 - jepsen.util 2152 :invoke :add 9899 INFO [2022-01-10 08:40:00,210] jepsen worker 794 - jepsen.util 1794 :info :add 8739 :RequestTimeoutException INFO [2022-01-10 08:40:00,217] jepsen worker 352 - jepsen.util 3352 :invoke :add 9900 INFO [2022-01-10 08:40:00,217] jepsen worker 241 - jepsen.util 2241 :info :add 8745 :RequestTimeoutException INFO [2022-01-10 08:40:00,220] jepsen worker 529 - jepsen.util 2529 :invoke :add 9901 INFO [2022-01-10 08:40:00,221] jepsen worker 642 - jepsen.util 2642 :info :add 8747 :RequestTimeoutException INFO [2022-01-10 08:40:00,223] jepsen worker 189 - jepsen.util 2189 :info :add 8749 :RequestTimeoutException INFO [2022-01-10 08:40:00,224] jepsen worker 677 - jepsen.util 3677 :invoke :add 9902 INFO [2022-01-10 08:40:00,227] jepsen worker 864 - jepsen.util 2864 :info :add 8748 :RequestTimeoutException INFO [2022-01-10 08:40:00,227] jepsen worker 719 - jepsen.util 1719 :info :add 8751 :RequestTimeoutException INFO [2022-01-10 08:40:00,229] jepsen worker 954 - jepsen.util 2954 :invoke :add 9903 INFO [2022-01-10 08:40:00,231] jepsen worker 805 - jepsen.util 1805 :info :add 8750 :RequestTimeoutException INFO [2022-01-10 08:40:00,231] jepsen worker 818 - jepsen.util 2818 :info :add 8752 :RequestTimeoutException INFO [2022-01-10 08:40:00,233] jepsen worker 52 - jepsen.util 2052 :info :add 8755 :RequestTimeoutException INFO [2022-01-10 08:40:00,233] jepsen worker 743 - jepsen.util 2743 :info :add 8753 :RequestTimeoutException INFO [2022-01-10 08:40:00,237] jepsen worker 497 - jepsen.util 1497 :info :add 8754 :RequestTimeoutException INFO [2022-01-10 08:40:00,237] jepsen worker 962 - jepsen.util 2962 :info :add 8757 :RequestTimeoutException INFO [2022-01-10 08:40:00,239] jepsen worker 886 - jepsen.util 3886 :invoke :add 9904 INFO [2022-01-10 08:40:00,241] jepsen worker 988 - jepsen.util 2988 :info :add 8760 :RequestTimeoutException INFO [2022-01-10 08:40:00,242] jepsen worker 720 - jepsen.util 2720 :info :add 8756 :RequestTimeoutException INFO [2022-01-10 08:40:00,246] jepsen worker 301 - jepsen.util 2301 :ok :add 8940 INFO [2022-01-10 08:40:00,247] jepsen worker 407 - jepsen.util 1407 :ok :add 8958 INFO [2022-01-10 08:40:00,247] jepsen worker 825 - jepsen.util 1825 :info :add 8763 :RequestTimeoutException INFO [2022-01-10 08:40:00,247] jepsen worker 805 - jepsen.util 2805 :invoke :add 9905 INFO [2022-01-10 08:40:00,250] jepsen worker 607 - jepsen.util 607 :info :add 8762 :RequestTimeoutException INFO [2022-01-10 08:40:00,250] jepsen worker 917 - jepsen.util 1917 :info :add 8765 :RequestTimeoutException INFO [2022-01-10 08:40:00,252] jepsen worker 558 - jepsen.util 2558 :info :add 8761 :RequestTimeoutException INFO [2022-01-10 08:40:00,252] jepsen worker 280 - jepsen.util 2280 :invoke :add 9906 INFO [2022-01-10 08:40:00,253] jepsen worker 653 - jepsen.util 2653 :info :add 8767 :RequestTimeoutException INFO [2022-01-10 08:40:00,254] jepsen worker 940 - jepsen.util 1940 :info :add 8764 :RequestTimeoutException INFO [2022-01-10 08:40:00,254] jepsen worker 895 - jepsen.util 1895 :info :add 8766 :RequestTimeoutException INFO [2022-01-10 08:40:00,257] jepsen worker 39 - jepsen.util 1039 :info :add 8768 :RequestTimeoutException INFO [2022-01-10 08:40:00,263] jepsen worker 594 - jepsen.util 594 :info :add 8769 :RequestTimeoutException INFO [2022-01-10 08:40:00,263] jepsen worker 979 - jepsen.util 1979 :info :add 8770 :RequestTimeoutException INFO [2022-01-10 08:40:00,263] jepsen worker 115 - jepsen.util 1115 :invoke :add 9907 INFO [2022-01-10 08:40:00,266] jepsen worker 582 - jepsen.util 2582 :invoke :add 9908 INFO [2022-01-10 08:40:00,266] jepsen worker 483 - jepsen.util 1483 :info :add 8772 :RequestTimeoutException INFO [2022-01-10 08:40:00,274] jepsen worker 823 - jepsen.util 3823 :invoke :add 9909 INFO [2022-01-10 08:40:00,275] jepsen worker 373 - jepsen.util 2373 :invoke :add 9910 INFO [2022-01-10 08:40:00,277] jepsen worker 415 - jepsen.util 2415 :invoke :add 9911 INFO [2022-01-10 08:40:00,280] jepsen worker 69 - jepsen.util 1069 :info :add 8779 :RequestTimeoutException INFO [2022-01-10 08:40:00,280] jepsen worker 895 - jepsen.util 2895 :invoke :add 9912 INFO [2022-01-10 08:40:00,281] jepsen worker 180 - jepsen.util 2180 :info :add 8780 :RequestTimeoutException INFO [2022-01-10 08:40:00,283] jepsen worker 657 - jepsen.util 2657 :info :add 8778 :RequestTimeoutException INFO [2022-01-10 08:40:00,283] jepsen worker 940 - jepsen.util 2940 :invoke :add 9913 INFO [2022-01-10 08:40:00,284] jepsen worker 314 - jepsen.util 2314 :info :add 8781 :RequestTimeoutException INFO [2022-01-10 08:40:00,284] jepsen worker 161 - jepsen.util 2161 :info :add 8782 :RequestTimeoutException INFO [2022-01-10 08:40:00,291] jepsen worker 39 - jepsen.util 2039 :invoke :add 9914 INFO [2022-01-10 08:40:00,292] jepsen worker 589 - jepsen.util 2589 :info :add 8784 :RequestTimeoutException INFO [2022-01-10 08:40:00,293] jepsen worker 781 - jepsen.util 1781 :info :add 8783 :RequestTimeoutException INFO [2022-01-10 08:40:00,295] jepsen worker 616 - jepsen.util 2616 :invoke :add 9915 INFO [2022-01-10 08:40:00,297] jepsen worker 726 - jepsen.util 2726 :info :add 8786 :RequestTimeoutException INFO [2022-01-10 08:40:00,301] jepsen worker 935 - jepsen.util 2935 :info :add 8785 :RequestTimeoutException INFO [2022-01-10 08:40:00,301] jepsen worker 494 - jepsen.util 494 :info :add 8787 :RequestTimeoutException INFO [2022-01-10 08:40:00,301] jepsen worker 409 - jepsen.util 2409 :invoke :add 9916 INFO [2022-01-10 08:40:00,303] jepsen worker 354 - jepsen.util 1354 :info :add 8790 :RequestTimeoutException INFO [2022-01-10 08:40:00,306] jepsen worker 628 - jepsen.util 2628 :info :add 8789 :RequestTimeoutException INFO [2022-01-10 08:40:00,307] jepsen worker 558 - jepsen.util 3558 :invoke :add 9917 INFO [2022-01-10 08:40:00,310] jepsen worker 934 - jepsen.util 1934 :info :add 8791 :RequestTimeoutException INFO [2022-01-10 08:40:00,311] jepsen worker 319 - jepsen.util 2319 :invoke :add 9918 INFO [2022-01-10 08:40:00,313] jepsen worker 707 - jepsen.util 707 :info :add 8792 :RequestTimeoutException INFO [2022-01-10 08:40:00,313] jepsen worker 223 - jepsen.util 1223 :info :add 8793 :RequestTimeoutException INFO [2022-01-10 08:40:00,315] jepsen worker 591 - jepsen.util 3591 :invoke :add 9919 INFO [2022-01-10 08:40:00,321] jepsen worker 786 - jepsen.util 3786 :invoke :add 9920 INFO [2022-01-10 08:40:00,324] jepsen worker 975 - jepsen.util 2975 :invoke :add 9921 INFO [2022-01-10 08:40:00,326] jepsen worker 589 - jepsen.util 3589 :invoke :add 9922 INFO [2022-01-10 08:40:00,329] jepsen worker 599 - jepsen.util 3599 :invoke :add 9923 INFO [2022-01-10 08:40:00,332] jepsen worker 250 - jepsen.util 2250 :invoke :add 9924 INFO [2022-01-10 08:40:00,333] jepsen worker 498 - jepsen.util 1498 :info :add 8798 :RequestTimeoutException INFO [2022-01-10 08:40:00,334] jepsen worker 29 - jepsen.util 3029 :invoke :add 9925 INFO [2022-01-10 08:40:00,338] jepsen worker 21 - jepsen.util 1021 :invoke :add 9926 INFO [2022-01-10 08:40:00,340] jepsen worker 389 - jepsen.util 1389 :invoke :add 9927 INFO [2022-01-10 08:40:00,343] jepsen worker 934 - jepsen.util 2934 :invoke :add 9928 INFO [2022-01-10 08:40:00,345] jepsen worker 189 - jepsen.util 3189 :invoke :add 9929 INFO [2022-01-10 08:40:00,349] jepsen worker 707 - jepsen.util 1707 :invoke :add 9930 INFO [2022-01-10 08:40:00,352] jepsen worker 664 - jepsen.util 2664 :invoke :add 9931 INFO [2022-01-10 08:40:00,353] jepsen worker 92 - jepsen.util 92 :info :add 8799 :RequestTimeoutException INFO [2022-01-10 08:40:00,354] jepsen worker 988 - jepsen.util 3988 :invoke :add 9932 INFO [2022-01-10 08:40:00,359] jepsen worker 628 - jepsen.util 3628 :invoke :add 9933 INFO [2022-01-10 08:40:00,361] jepsen worker 492 - jepsen.util 1492 :info :add 8800 :RequestTimeoutException INFO [2022-01-10 08:40:00,361] jepsen worker 794 - jepsen.util 2794 :invoke :add 9934 INFO [2022-01-10 08:40:00,365] jepsen worker 777 - jepsen.util 1777 :invoke :add 9935 INFO [2022-01-10 08:40:00,368] jepsen worker 350 - jepsen.util 3350 :invoke :add 9936 INFO [2022-01-10 08:40:00,371] jepsen worker 503 - jepsen.util 3503 :invoke :add 9937 INFO [2022-01-10 08:40:00,372] jepsen worker 122 - jepsen.util 2122 :info :add 8802 :RequestTimeoutException INFO [2022-01-10 08:40:00,373] jepsen worker 109 - jepsen.util 2109 :invoke :add 9938 INFO [2022-01-10 08:40:00,374] jepsen worker 903 - jepsen.util 2903 :info :add 8801 :RequestTimeoutException INFO [2022-01-10 08:40:00,377] jepsen worker 766 - jepsen.util 2766 :info :add 8803 :RequestTimeoutException INFO [2022-01-10 08:40:00,379] jepsen worker 864 - jepsen.util 3864 :invoke :add 9939 INFO [2022-01-10 08:40:00,383] jepsen worker 908 - jepsen.util 3908 :invoke :add 9940 INFO [2022-01-10 08:40:00,385] jepsen worker 311 - jepsen.util 3311 :invoke :add 9941 INFO [2022-01-10 08:40:00,387] jepsen worker 187 - jepsen.util 3187 :invoke :add 9942 INFO [2022-01-10 08:40:00,390] jepsen worker 653 - jepsen.util 3653 :invoke :add 9943 INFO [2022-01-10 08:40:00,393] jepsen worker 859 - jepsen.util 2859 :invoke :add 9944 INFO [2022-01-10 08:40:00,395] jepsen worker 617 - jepsen.util 3617 :invoke :add 9945 INFO [2022-01-10 08:40:00,398] jepsen worker 719 - jepsen.util 2719 :invoke :add 9946 INFO [2022-01-10 08:40:00,400] jepsen worker 982 - jepsen.util 2982 :invoke :add 9947 INFO [2022-01-10 08:40:00,403] jepsen worker 962 - jepsen.util 3962 :invoke :add 9948 INFO [2022-01-10 08:40:00,406] jepsen worker 781 - jepsen.util 2781 :invoke :add 9949 INFO [2022-01-10 08:40:00,409] jepsen worker 602 - jepsen.util 3602 :invoke :add 9950 INFO [2022-01-10 08:40:00,413] jepsen worker 690 - jepsen.util 1690 :invoke :add 9951 INFO [2022-01-10 08:40:00,416] jepsen worker 323 - jepsen.util 3323 :invoke :add 9952 INFO [2022-01-10 08:40:00,419] jepsen worker 89 - jepsen.util 3089 :invoke :add 9953 INFO [2022-01-10 08:40:00,422] jepsen worker 164 - jepsen.util 1164 :invoke :add 9954 INFO [2022-01-10 08:40:00,426] jepsen worker 657 - jepsen.util 3657 :invoke :add 9955 INFO [2022-01-10 08:40:00,429] jepsen worker 702 - jepsen.util 1702 :invoke :add 9956 INFO [2022-01-10 08:40:00,431] jepsen worker 87 - jepsen.util 2087 :ok :add 9613 INFO [2022-01-10 08:40:00,431] jepsen worker 392 - jepsen.util 3392 :ok :add 9562 INFO [2022-01-10 08:40:00,431] jepsen worker 736 - jepsen.util 2736 :ok :add 9556 INFO [2022-01-10 08:40:00,431] jepsen worker 516 - jepsen.util 2516 :ok :add 9627 INFO [2022-01-10 08:40:00,432] jepsen worker 58 - jepsen.util 3058 :invoke :add 9957 INFO [2022-01-10 08:40:00,436] jepsen worker 331 - jepsen.util 2331 :invoke :add 9958 INFO [2022-01-10 08:40:00,439] jepsen worker 699 - jepsen.util 3699 :invoke :add 9959 INFO [2022-01-10 08:40:00,441] jepsen worker 979 - jepsen.util 2979 :invoke :add 9960 INFO [2022-01-10 08:40:00,444] jepsen worker 935 - jepsen.util 3935 :invoke :add 9961 INFO [2022-01-10 08:40:00,446] jepsen worker 689 - jepsen.util 3689 :invoke :add 9962 INFO [2022-01-10 08:40:00,449] jepsen worker 494 - jepsen.util 1494 :invoke :add 9963 INFO [2022-01-10 08:40:00,452] jepsen worker 461 - jepsen.util 2461 :invoke :add 9964 INFO [2022-01-10 08:40:00,454] jepsen worker 818 - jepsen.util 3818 :invoke :add 9965 INFO [2022-01-10 08:40:00,457] jepsen worker 883 - jepsen.util 3883 :invoke :add 9966 INFO [2022-01-10 08:40:00,460] jepsen worker 743 - jepsen.util 3743 :invoke :add 9967 INFO [2022-01-10 08:40:00,463] jepsen worker 535 - jepsen.util 2535 :invoke :add 9968 INFO [2022-01-10 08:40:00,465] jepsen worker 210 - jepsen.util 2210 :invoke :add 9969 INFO [2022-01-10 08:40:00,468] jepsen worker 483 - jepsen.util 2483 :invoke :add 9970 INFO [2022-01-10 08:40:00,471] jepsen worker 566 - jepsen.util 2566 :invoke :add 9971 INFO [2022-01-10 08:40:00,472] jepsen worker 894 - jepsen.util 1894 :info :add 8804 :RequestTimeoutException INFO [2022-01-10 08:40:00,474] jepsen worker 714 - jepsen.util 3714 :invoke :add 9972 INFO [2022-01-10 08:40:00,478] jepsen worker 736 - jepsen.util 2736 :invoke :add 9973 INFO [2022-01-10 08:40:00,480] jepsen worker 178 - jepsen.util 1178 :info :add 8806 :RequestTimeoutException INFO [2022-01-10 08:40:00,481] jepsen worker 79 - jepsen.util 2079 :invoke :add 9974 INFO [2022-01-10 08:40:00,482] jepsen worker 745 - jepsen.util 1745 :info :add 8805 :RequestTimeoutException INFO [2022-01-10 08:40:00,482] jepsen worker 420 - jepsen.util 1420 :info :add 8807 :RequestTimeoutException INFO [2022-01-10 08:40:00,487] jepsen worker 178 - jepsen.util 2178 :invoke :add 9975 INFO [2022-01-10 08:40:00,490] jepsen worker 751 - jepsen.util 3751 :invoke :add 9976 INFO [2022-01-10 08:40:00,493] jepsen worker 301 - jepsen.util 2301 :invoke :add 9977 INFO [2022-01-10 08:40:00,495] jepsen worker 713 - jepsen.util 3713 :invoke :add 9978 INFO [2022-01-10 08:40:00,498] jepsen worker 241 - jepsen.util 3241 :invoke :add 9979 INFO [2022-01-10 08:40:00,500] jepsen worker 703 - jepsen.util 3703 :invoke :add 9980 INFO [2022-01-10 08:40:00,503] jepsen worker 314 - jepsen.util 3314 :invoke :add 9981 INFO [2022-01-10 08:40:00,506] jepsen worker 915 - jepsen.util 1915 :invoke :add 9982 INFO [2022-01-10 08:40:00,509] jepsen worker 392 - jepsen.util 3392 :invoke :add 9983 INFO [2022-01-10 08:40:00,512] jepsen worker 180 - jepsen.util 3180 :invoke :add 9984 INFO [2022-01-10 08:40:00,514] jepsen worker 69 - jepsen.util 2069 :invoke :add 9985 INFO [2022-01-10 08:40:00,517] jepsen worker 726 - jepsen.util 3726 :invoke :add 9986 INFO [2022-01-10 08:40:00,521] jepsen worker 122 - jepsen.util 3122 :invoke :add 9987 INFO [2022-01-10 08:40:00,523] jepsen worker 52 - jepsen.util 3052 :invoke :add 9988 INFO [2022-01-10 08:40:00,527] jepsen worker 674 - jepsen.util 1674 :invoke :add 9989 INFO [2022-01-10 08:40:00,530] jepsen worker 700 - jepsen.util 2700 :invoke :add 9990 INFO [2022-01-10 08:40:00,533] jepsen worker 492 - jepsen.util 2492 :invoke :add 9991 INFO [2022-01-10 08:40:00,536] jepsen worker 878 - jepsen.util 3878 :ok :add 9582 INFO [2022-01-10 08:40:00,536] jepsen worker 766 - jepsen.util 3766 :invoke :add 9992 INFO [2022-01-10 08:40:00,539] jepsen worker 516 - jepsen.util 2516 :invoke :add 9993 INFO [2022-01-10 08:40:00,542] jepsen worker 93 - jepsen.util 3093 :invoke :add 9994 INFO [2022-01-10 08:40:00,544] jepsen worker 522 - jepsen.util 3522 :invoke :add 9995 INFO [2022-01-10 08:40:00,547] jepsen worker 642 - jepsen.util 3642 :invoke :add 9996 INFO [2022-01-10 08:40:00,550] jepsen worker 354 - jepsen.util 2354 :invoke :add 9997 INFO [2022-01-10 08:40:00,550] jepsen worker 949 - jepsen.util 1949 :info :add 8808 :RequestTimeoutException INFO [2022-01-10 08:40:00,553] jepsen worker 470 - jepsen.util 1470 :info :add 8809 :RequestTimeoutException INFO [2022-01-10 08:40:00,555] jepsen worker 594 - jepsen.util 1594 :invoke :add 9998 INFO [2022-01-10 08:40:00,558] jepsen worker 470 - jepsen.util 2470 :invoke :add 9999 INFO [2022-01-10 08:40:00,561] jepsen worker 158 - jepsen.util 2158 :invoke :add 10000 INFO [2022-01-10 08:40:00,562] jepsen worker 847 - jepsen.util 1847 :info :add 8810 :RequestTimeoutException INFO [2022-01-10 08:40:00,563] jepsen worker 418 - jepsen.util 1418 :info :add 8811 :RequestTimeoutException INFO [2022-01-10 08:40:00,563] jepsen worker 745 - jepsen.util 2745 :invoke :add 10001 INFO [2022-01-10 08:40:00,568] jepsen worker 498 - jepsen.util 2498 :invoke :add 10002 INFO [2022-01-10 08:40:00,570] jepsen worker 497 - jepsen.util 2497 :invoke :add 10003 INFO [2022-01-10 08:40:00,572] jepsen worker 421 - jepsen.util 2421 :info :add 8812 :RequestTimeoutException INFO [2022-01-10 08:40:00,573] jepsen worker 825 - jepsen.util 2825 :invoke :add 10004 INFO [2022-01-10 08:40:00,577] jepsen worker 912 - jepsen.util 1912 :info :add 8813 :RequestTimeoutException INFO [2022-01-10 08:40:00,577] jepsen worker 420 - jepsen.util 2420 :invoke :add 10005 INFO [2022-01-10 08:40:00,581] jepsen worker 607 - jepsen.util 1607 :invoke :add 10006 INFO [2022-01-10 08:40:00,584] jepsen worker 6 - jepsen.util 3006 :invoke :add 10007 INFO [2022-01-10 08:40:00,587] jepsen worker 903 - jepsen.util 3903 :invoke :add 10008 INFO [2022-01-10 08:40:00,589] jepsen worker 418 - jepsen.util 2418 :invoke :add 10009 INFO [2022-01-10 08:40:00,592] jepsen worker 223 - jepsen.util 2223 :invoke :add 10010 INFO [2022-01-10 08:40:00,593] jepsen worker 824 - jepsen.util 1824 :info :add 8818 :RequestTimeoutException INFO [2022-01-10 08:40:00,594] jepsen worker 917 - jepsen.util 2917 :invoke :add 10011 INFO [2022-01-10 08:40:00,598] jepsen worker 145 - jepsen.util 2145 :invoke :add 10012 INFO [2022-01-10 08:40:00,600] jepsen worker 590 - jepsen.util 1590 :info :add 8819 :RequestTimeoutException INFO [2022-01-10 08:40:00,601] jepsen worker 900 - jepsen.util 3900 :invoke :add 10013 INFO [2022-01-10 08:40:00,605] jepsen worker 50 - jepsen.util 3050 :invoke :add 10014 INFO [2022-01-10 08:40:00,607] jepsen worker 720 - jepsen.util 3720 :invoke :add 10015 INFO [2022-01-10 08:40:00,610] jepsen worker 407 - jepsen.util 1407 :invoke :add 10016 INFO [2022-01-10 08:40:00,613] jepsen worker 87 - jepsen.util 2087 :invoke :add 10017 INFO [2022-01-10 08:40:00,616] jepsen worker 894 - jepsen.util 2894 :invoke :add 10018 INFO [2022-01-10 08:40:00,618] jepsen worker 847 - jepsen.util 2847 :invoke :add 10019 INFO [2022-01-10 08:40:00,621] jepsen worker 92 - jepsen.util 1092 :invoke :add 10020 INFO [2022-01-10 08:40:00,624] jepsen worker 161 - jepsen.util 3161 :invoke :add 10021 INFO [2022-01-10 08:40:00,627] jepsen worker 878 - jepsen.util 3878 :invoke :add 10022 INFO [2022-01-10 08:40:00,630] jepsen worker 590 - jepsen.util 2590 :invoke :add 10023 INFO [2022-01-10 08:40:00,632] jepsen worker 421 - jepsen.util 3421 :invoke :add 10024 INFO [2022-01-10 08:40:00,636] jepsen worker 927 - jepsen.util 3927 :invoke :add 10025 INFO [2022-01-10 08:40:00,639] jepsen worker 912 - jepsen.util 2912 :invoke :add 10026 INFO [2022-01-10 08:40:00,641] jepsen worker 531 - jepsen.util 3531 :invoke :add 10027 INFO [2022-01-10 08:40:00,644] jepsen worker 824 - jepsen.util 2824 :invoke :add 10028 INFO [2022-01-10 08:40:00,647] jepsen worker 949 - jepsen.util 2949 :invoke :add 10029 INFO [2022-01-10 08:40:00,715] jepsen worker 973 - jepsen.util 2973 :ok :add 9033 INFO [2022-01-10 08:40:00,721] jepsen worker 973 - jepsen.util 2973 :invoke :add 10030 INFO [2022-01-10 08:40:00,751] jepsen worker 730 - jepsen.util 2730 :info :add 8824 :RequestTimeoutException INFO [2022-01-10 08:40:00,757] jepsen worker 730 - jepsen.util 3730 :invoke :add 10031 INFO [2022-01-10 08:40:00,773] jepsen worker 651 - jepsen.util 2651 :info :add 8825 :RequestTimeoutException INFO [2022-01-10 08:40:00,778] jepsen worker 651 - jepsen.util 3651 :invoke :add 10032 INFO [2022-01-10 08:40:00,781] jepsen worker 644 - jepsen.util 2644 :info :add 8826 :RequestTimeoutException INFO [2022-01-10 08:40:00,785] jepsen worker 644 - jepsen.util 3644 :invoke :add 10033 INFO [2022-01-10 08:40:00,821] jepsen worker 959 - jepsen.util 2959 :info :add 8827 :RequestTimeoutException INFO [2022-01-10 08:40:00,825] jepsen worker 959 - jepsen.util 3959 :invoke :add 10034 INFO [2022-01-10 08:40:00,827] jepsen worker 884 - jepsen.util 2884 :info :add 8829 :RequestTimeoutException INFO [2022-01-10 08:40:00,827] jepsen worker 740 - jepsen.util 2740 :info :add 8828 :RequestTimeoutException INFO [2022-01-10 08:40:00,827] jepsen worker 379 - jepsen.util 2379 :info :add 8830 :RequestTimeoutException INFO [2022-01-10 08:40:00,833] jepsen worker 980 - jepsen.util 1980 :info :add 8831 :RequestTimeoutException INFO [2022-01-10 08:40:00,834] jepsen worker 740 - jepsen.util 3740 :invoke :add 10035 INFO [2022-01-10 08:40:00,837] jepsen worker 884 - jepsen.util 3884 :invoke :add 10036 INFO [2022-01-10 08:40:00,840] jepsen worker 379 - jepsen.util 3379 :invoke :add 10037 INFO [2022-01-10 08:40:00,843] jepsen worker 980 - jepsen.util 2980 :invoke :add 10038 INFO [2022-01-10 08:40:00,852] jepsen worker 153 - jepsen.util 2153 :info :add 8832 :RequestTimeoutException INFO [2022-01-10 08:40:00,856] jepsen worker 153 - jepsen.util 3153 :invoke :add 10039 INFO [2022-01-10 08:40:00,857] jepsen worker 646 - jepsen.util 2646 :info :add 8833 :RequestTimeoutException INFO [2022-01-10 08:40:00,861] jepsen worker 646 - jepsen.util 3646 :invoke :add 10040 INFO [2022-01-10 08:40:01,030] jepsen worker 235 - jepsen.util 3235 :ok :add 9655 INFO [2022-01-10 08:40:01,030] jepsen worker 374 - jepsen.util 2374 :ok :add 9679 INFO [2022-01-10 08:40:01,034] jepsen worker 235 - jepsen.util 3235 :invoke :add 10041 INFO [2022-01-10 08:40:01,037] jepsen worker 374 - jepsen.util 2374 :invoke :add 10042 INFO [2022-01-10 08:40:01,120] jepsen worker 958 - jepsen.util 2958 :info :add 8834 :RequestTimeoutException INFO [2022-01-10 08:40:01,123] jepsen worker 854 - jepsen.util 2854 :info :add 8835 :RequestTimeoutException INFO [2022-01-10 08:40:01,123] jepsen worker 958 - jepsen.util 3958 :invoke :add 10043 INFO [2022-01-10 08:40:01,126] jepsen worker 854 - jepsen.util 3854 :invoke :add 10044 INFO [2022-01-10 08:40:01,143] jepsen worker 530 - jepsen.util 2530 :ok :add 9250 INFO [2022-01-10 08:40:01,149] jepsen worker 530 - jepsen.util 2530 :invoke :add 10045 INFO [2022-01-10 08:40:01,171] jepsen worker 533 - jepsen.util 1533 :info :add 8836 :RequestTimeoutException INFO [2022-01-10 08:40:01,173] jepsen worker 224 - jepsen.util 1224 :info :add 8837 :RequestTimeoutException INFO [2022-01-10 08:40:01,178] jepsen worker 224 - jepsen.util 2224 :invoke :add 10046 INFO [2022-01-10 08:40:01,183] jepsen worker 533 - jepsen.util 2533 :invoke :add 10047 INFO [2022-01-10 08:40:01,192] jepsen worker 414 - jepsen.util 1414 :info :add 8840 :RequestTimeoutException INFO [2022-01-10 08:40:01,198] jepsen worker 414 - jepsen.util 2414 :invoke :add 10048 INFO [2022-01-10 08:40:01,227] jepsen worker 245 - jepsen.util 1245 :info :add 8841 :RequestTimeoutException INFO [2022-01-10 08:40:01,231] jepsen worker 245 - jepsen.util 2245 :invoke :add 10049 INFO [2022-01-10 08:40:01,239] jepsen worker 342 - jepsen.util 2342 :ok :add 9612 INFO [2022-01-10 08:40:01,239] jepsen worker 954 - jepsen.util 2954 :ok :add 9903 INFO [2022-01-10 08:40:01,239] jepsen worker 593 - jepsen.util 2593 :ok :add 9563 INFO [2022-01-10 08:40:01,239] jepsen worker 752 - jepsen.util 3752 :ok :add 9557 INFO [2022-01-10 08:40:01,239] jepsen worker 523 - jepsen.util 2523 :ok :add 9626 INFO [2022-01-10 08:40:01,239] jepsen worker 503 - jepsen.util 3503 :ok :add 9937 INFO [2022-01-10 08:40:01,240] jepsen worker 135 - jepsen.util 1135 :info :add 8842 :RequestTimeoutException INFO [2022-01-10 08:40:01,242] jepsen worker 758 - jepsen.util 2758 :ok :add 9583 INFO [2022-01-10 08:40:01,242] jepsen worker 782 - jepsen.util 3782 :ok :add 9847 INFO [2022-01-10 08:40:01,242] jepsen worker 675 - jepsen.util 3675 :ok :add 9873 INFO [2022-01-10 08:40:01,249] jepsen worker 593 - jepsen.util 2593 :invoke :add 10050 INFO [2022-01-10 08:40:01,253] jepsen worker 758 - jepsen.util 2758 :invoke :add 10051 INFO [2022-01-10 08:40:01,256] jepsen worker 782 - jepsen.util 3782 :invoke :add 10052 INFO [2022-01-10 08:40:01,259] jepsen worker 342 - jepsen.util 2342 :invoke :add 10053 INFO [2022-01-10 08:40:01,263] jepsen worker 503 - jepsen.util 3503 :invoke :add 10054 INFO [2022-01-10 08:40:01,263] jepsen worker 796 - jepsen.util 1796 :info :add 8843 :RequestTimeoutException INFO [2022-01-10 08:40:01,265] jepsen worker 523 - jepsen.util 2523 :invoke :add 10055 INFO [2022-01-10 08:40:01,269] jepsen worker 796 - jepsen.util 2796 :invoke :add 10056 INFO [2022-01-10 08:40:01,271] jepsen worker 779 - jepsen.util 2779 :info :add 8844 :RequestTimeoutException INFO [2022-01-10 08:40:01,272] jepsen worker 675 - jepsen.util 3675 :invoke :add 10057 INFO [2022-01-10 08:40:01,272] jepsen worker 30 - jepsen.util 2030 :info :add 8846 :RequestTimeoutException INFO [2022-01-10 08:40:01,274] jepsen worker 313 - jepsen.util 1313 :info :add 8845 :RequestTimeoutException INFO [2022-01-10 08:40:01,278] jepsen worker 30 - jepsen.util 3030 :invoke :add 10058 INFO [2022-01-10 08:40:01,281] jepsen worker 954 - jepsen.util 2954 :invoke :add 10059 INFO [2022-01-10 08:40:01,284] jepsen worker 779 - jepsen.util 3779 :invoke :add 10060 INFO [2022-01-10 08:40:01,286] jepsen worker 135 - jepsen.util 2135 :invoke :add 10061 INFO [2022-01-10 08:40:01,288] jepsen worker 752 - jepsen.util 3752 :invoke :add 10062 INFO [2022-01-10 08:40:01,291] jepsen worker 313 - jepsen.util 2313 :invoke :add 10063 INFO [2022-01-10 08:40:01,307] jepsen worker 592 - jepsen.util 1592 :info :add 8847 :RequestTimeoutException INFO [2022-01-10 08:40:01,311] jepsen worker 592 - jepsen.util 2592 :invoke :add 10064 INFO [2022-01-10 08:40:01,360] jepsen worker 172 - jepsen.util 1172 :info :add 8854 :RequestTimeoutException INFO [2022-01-10 08:40:01,364] jepsen worker 172 - jepsen.util 2172 :invoke :add 10065 INFO [2022-01-10 08:40:01,497] jepsen worker 910 - jepsen.util 1910 :info :add 8855 :RequestTimeoutException INFO [2022-01-10 08:40:01,504] jepsen worker 910 - jepsen.util 2910 :invoke :add 10066 INFO [2022-01-10 08:40:01,508] jepsen worker 42 - jepsen.util 3042 :ok :add 9581 INFO [2022-01-10 08:40:01,509] jepsen worker 798 - jepsen.util 2798 :ok :add 9599 INFO [2022-01-10 08:40:01,509] jepsen worker 393 - jepsen.util 3393 :ok :add 9845 INFO [2022-01-10 08:40:01,515] jepsen worker 42 - jepsen.util 3042 :invoke :add 10067 INFO [2022-01-10 08:40:01,517] jepsen worker 798 - jepsen.util 2798 :invoke :add 10068 INFO [2022-01-10 08:40:01,520] jepsen worker 393 - jepsen.util 3393 :invoke :add 10069 INFO [2022-01-10 08:40:01,633] jepsen worker 858 - jepsen.util 1858 :info :add 8856 :RequestTimeoutException INFO [2022-01-10 08:40:01,638] jepsen worker 858 - jepsen.util 2858 :invoke :add 10070 INFO [2022-01-10 08:40:01,641] jepsen worker 710 - jepsen.util 1710 :info :add 8857 :RequestTimeoutException INFO [2022-01-10 08:40:01,644] jepsen worker 698 - jepsen.util 1698 :info :add 8858 :RequestTimeoutException INFO [2022-01-10 08:40:01,644] jepsen worker 710 - jepsen.util 2710 :invoke :add 10071 INFO [2022-01-10 08:40:01,648] jepsen worker 698 - jepsen.util 2698 :invoke :add 10072 INFO [2022-01-10 08:40:01,652] jepsen worker 238 - jepsen.util 2238 :info :add 8859 :RequestTimeoutException INFO [2022-01-10 08:40:01,657] jepsen worker 435 - jepsen.util 1435 :info :add 8860 :RequestTimeoutException INFO [2022-01-10 08:40:01,657] jepsen worker 238 - jepsen.util 3238 :invoke :add 10073 INFO [2022-01-10 08:40:01,660] jepsen worker 783 - jepsen.util 1783 :info :add 8861 :RequestTimeoutException INFO [2022-01-10 08:40:01,661] jepsen worker 8 - jepsen.util 1008 :info :add 8863 :RequestTimeoutException INFO [2022-01-10 08:40:01,661] jepsen worker 435 - jepsen.util 2435 :invoke :add 10074 INFO [2022-01-10 08:40:01,663] jepsen worker 499 - jepsen.util 1499 :info :add 8862 :RequestTimeoutException INFO [2022-01-10 08:40:01,669] jepsen worker 783 - jepsen.util 2783 :invoke :add 10075 INFO [2022-01-10 08:40:01,672] jepsen worker 8 - jepsen.util 2008 :invoke :add 10076 INFO [2022-01-10 08:40:01,675] jepsen worker 499 - jepsen.util 2499 :invoke :add 10077 INFO [2022-01-10 08:40:01,679] jepsen worker 112 - jepsen.util 2112 :info :add 8868 :RequestTimeoutException INFO [2022-01-10 08:40:01,682] jepsen worker 104 - jepsen.util 2104 :info :add 8870 :RequestTimeoutException INFO [2022-01-10 08:40:01,683] jepsen worker 112 - jepsen.util 3112 :invoke :add 10078 INFO [2022-01-10 08:40:01,683] jepsen worker 453 - jepsen.util 1453 :info :add 8869 :RequestTimeoutException INFO [2022-01-10 08:40:01,687] jepsen worker 104 - jepsen.util 3104 :invoke :add 10079 INFO [2022-01-10 08:40:01,690] jepsen worker 453 - jepsen.util 2453 :invoke :add 10080 INFO [2022-01-10 08:40:01,739] jepsen worker 395 - jepsen.util 2395 :ok :add 9059 INFO [2022-01-10 08:40:01,739] jepsen worker 218 - jepsen.util 2218 :ok :add 8971 INFO [2022-01-10 08:40:01,744] jepsen worker 218 - jepsen.util 2218 :invoke :add 10081 INFO [2022-01-10 08:40:01,748] jepsen worker 395 - jepsen.util 2395 :invoke :add 10082 INFO [2022-01-10 08:40:01,790] jepsen worker 86 - jepsen.util 2086 :info :add 8871 :RequestTimeoutException INFO [2022-01-10 08:40:01,794] jepsen worker 86 - jepsen.util 3086 :invoke :add 10083 INFO [2022-01-10 08:40:01,797] jepsen worker 50 - jepsen.util 3050 :ok :add 10014 INFO [2022-01-10 08:40:01,797] jepsen worker 623 - jepsen.util 2623 :ok :add 9663 INFO [2022-01-10 08:40:01,797] jepsen worker 921 - jepsen.util 2921 :ok :add 9657 INFO [2022-01-10 08:40:01,801] jepsen worker 623 - jepsen.util 2623 :invoke :add 10084 INFO [2022-01-10 08:40:01,803] jepsen worker 563 - jepsen.util 2563 :info :add 8872 :RequestTimeoutException INFO [2022-01-10 08:40:01,803] jepsen worker 921 - jepsen.util 2921 :invoke :add 10085 INFO [2022-01-10 08:40:01,807] jepsen worker 50 - jepsen.util 3050 :invoke :add 10086 INFO [2022-01-10 08:40:01,811] jepsen worker 563 - jepsen.util 3563 :invoke :add 10087 INFO [2022-01-10 08:40:01,811] jepsen worker 968 - jepsen.util 2968 :info :add 8873 :RequestTimeoutException INFO [2022-01-10 08:40:01,817] jepsen worker 968 - jepsen.util 3968 :invoke :add 10088 INFO [2022-01-10 08:40:01,972] jepsen worker 66 - jepsen.util 2066 :info :add 8875 :RequestTimeoutException INFO [2022-01-10 08:40:01,973] jepsen worker 995 - jepsen.util 2995 :info :add 8874 :RequestTimeoutException INFO [2022-01-10 08:40:01,977] jepsen worker 520 - jepsen.util 2520 :info :add 8876 :RequestTimeoutException INFO [2022-01-10 08:40:01,977] jepsen worker 995 - jepsen.util 3995 :invoke :add 10089 INFO [2022-01-10 08:40:01,981] jepsen worker 66 - jepsen.util 3066 :invoke :add 10090 INFO [2022-01-10 08:40:01,984] jepsen worker 520 - jepsen.util 3520 :invoke :add 10091 INFO [2022-01-10 08:40:02,000] jepsen worker 216 - jepsen.util 2216 :info :add 8877 :RequestTimeoutException INFO [2022-01-10 08:40:02,003] jepsen worker 256 - jepsen.util 2256 :info :add 8878 :RequestTimeoutException INFO [2022-01-10 08:40:02,004] jepsen worker 216 - jepsen.util 3216 :invoke :add 10092 INFO [2022-01-10 08:40:02,008] jepsen worker 256 - jepsen.util 3256 :invoke :add 10093 INFO [2022-01-10 08:40:02,062] jepsen worker 506 - jepsen.util 1506 :info :add 8879 :RequestTimeoutException INFO [2022-01-10 08:40:02,067] jepsen worker 506 - jepsen.util 2506 :invoke :add 10094 INFO [2022-01-10 08:40:02,092] jepsen worker 564 - jepsen.util 1564 :info :add 8881 :RequestTimeoutException INFO [2022-01-10 08:40:02,093] jepsen worker 371 - jepsen.util 1371 :info :add 8880 :RequestTimeoutException INFO [2022-01-10 08:40:02,097] jepsen worker 579 - jepsen.util 1579 :info :add 8882 :RequestTimeoutException INFO [2022-01-10 08:40:02,098] jepsen worker 564 - jepsen.util 2564 :invoke :add 10095 INFO [2022-01-10 08:40:02,100] jepsen worker 303 - jepsen.util 1303 :info :add 8883 :RequestTimeoutException INFO [2022-01-10 08:40:02,102] jepsen worker 579 - jepsen.util 2579 :invoke :add 10096 INFO [2022-01-10 08:40:02,106] jepsen worker 303 - jepsen.util 2303 :invoke :add 10097 INFO [2022-01-10 08:40:02,109] jepsen worker 371 - jepsen.util 2371 :invoke :add 10098 INFO [2022-01-10 08:40:02,151] jepsen worker nemesis - jepsen.util :nemesis :info :rebalance-cluster nil INFO [2022-01-10 08:40:02,151] jepsen worker nemesis - jepsen.util :nemesis :info :rebalance-cluster :rebalanced INFO [2022-01-10 08:40:02,156] jepsen worker 467 - jepsen.util 2467 :ok :add 9681 INFO [2022-01-10 08:40:02,156] jepsen worker 504 - jepsen.util 2504 :ok :add 9709 INFO [2022-01-10 08:40:02,156] jepsen worker 258 - jepsen.util 3258 :ok :add 9725 INFO [2022-01-10 08:40:02,156] jepsen worker 633 - jepsen.util 2633 :ok :add 9699 INFO [2022-01-10 08:40:02,156] jepsen worker 52 - jepsen.util 3052 :ok :add 9988 INFO [2022-01-10 08:40:02,157] jepsen worker 629 - jepsen.util 1629 :ok :add 9834 INFO [2022-01-10 08:40:02,157] jepsen worker 700 - jepsen.util 2700 :ok :add 9990 INFO [2022-01-10 08:40:02,162] jepsen worker 504 - jepsen.util 2504 :invoke :add 10099 INFO [2022-01-10 08:40:02,165] jepsen worker 700 - jepsen.util 2700 :invoke :add 10100 INFO [2022-01-10 08:40:02,168] jepsen worker 633 - jepsen.util 2633 :invoke :add 10101 INFO [2022-01-10 08:40:02,170] jepsen worker 629 - jepsen.util 1629 :invoke :add 10102 INFO [2022-01-10 08:40:02,173] jepsen worker 258 - jepsen.util 3258 :invoke :add 10103 INFO [2022-01-10 08:40:02,176] jepsen worker 52 - jepsen.util 3052 :invoke :add 10104 INFO [2022-01-10 08:40:02,178] jepsen worker 467 - jepsen.util 2467 :invoke :add 10105 INFO [2022-01-10 08:40:02,343] jepsen worker 408 - jepsen.util 2408 :ok :add 8983 INFO [2022-01-10 08:40:02,348] jepsen worker 408 - jepsen.util 2408 :invoke :add 10106 INFO [2022-01-10 08:40:02,353] jepsen worker 562 - jepsen.util 1562 :info :add 8889 :RequestTimeoutException INFO [2022-01-10 08:40:02,353] jepsen worker 321 - jepsen.util 1321 :info :add 8890 :RequestTimeoutException INFO [2022-01-10 08:40:02,353] jepsen worker 432 - jepsen.util 2432 :info :add 8888 :RequestTimeoutException INFO [2022-01-10 08:40:02,357] jepsen worker 849 - jepsen.util 2849 :info :add 8891 :RequestTimeoutException INFO [2022-01-10 08:40:02,361] jepsen worker 206 - jepsen.util 1206 :info :add 8892 :RequestTimeoutException INFO [2022-01-10 08:40:02,363] jepsen worker 432 - jepsen.util 3432 :invoke :add 10107 INFO [2022-01-10 08:40:02,363] jepsen worker 596 - jepsen.util 1596 :info :add 8894 :RequestTimeoutException INFO [2022-01-10 08:40:02,368] jepsen worker 849 - jepsen.util 3849 :invoke :add 10108 INFO [2022-01-10 08:40:02,370] jepsen worker 248 - jepsen.util 1248 :info :add 8893 :RequestTimeoutException INFO [2022-01-10 08:40:02,370] jepsen worker 206 - jepsen.util 2206 :invoke :add 10109 INFO [2022-01-10 08:40:02,371] jepsen worker 364 - jepsen.util 2364 :info :add 8895 :RequestTimeoutException INFO [2022-01-10 08:40:02,374] jepsen worker 307 - jepsen.util 2307 :info :add 8896 :RequestTimeoutException INFO [2022-01-10 08:40:02,375] jepsen worker 596 - jepsen.util 2596 :invoke :add 10110 INFO [2022-01-10 08:40:02,380] jepsen worker 248 - jepsen.util 2248 :invoke :add 10111 INFO [2022-01-10 08:40:02,382] jepsen worker 198 - jepsen.util 2198 :info :add 8897 :RequestTimeoutException INFO [2022-01-10 08:40:02,383] jepsen worker 364 - jepsen.util 3364 :invoke :add 10112 INFO [2022-01-10 08:40:02,387] jepsen worker 334 - jepsen.util 2334 :info :add 8898 :RequestTimeoutException INFO [2022-01-10 08:40:02,388] jepsen worker 562 - jepsen.util 2562 :invoke :add 10113 INFO [2022-01-10 08:40:02,390] jepsen worker 816 - jepsen.util 2816 :info :add 8899 :RequestTimeoutException INFO [2022-01-10 08:40:02,394] jepsen worker 334 - jepsen.util 3334 :invoke :add 10114 INFO [2022-01-10 08:40:02,397] jepsen worker 321 - jepsen.util 2321 :invoke :add 10115 INFO [2022-01-10 08:40:02,400] jepsen worker 307 - jepsen.util 3307 :invoke :add 10116 INFO [2022-01-10 08:40:02,403] jepsen worker 816 - jepsen.util 3816 :invoke :add 10117 INFO [2022-01-10 08:40:02,405] jepsen worker 198 - jepsen.util 3198 :invoke :add 10118 INFO [2022-01-10 08:40:02,481] jepsen worker 501 - jepsen.util 2501 :info :add 8901 :RequestTimeoutException INFO [2022-01-10 08:40:02,483] jepsen worker 811 - jepsen.util 2811 :info :add 8902 :RequestTimeoutException INFO [2022-01-10 08:40:02,483] jepsen worker 197 - jepsen.util 2197 :info :add 8900 :RequestTimeoutException INFO [2022-01-10 08:40:02,486] jepsen worker 501 - jepsen.util 3501 :invoke :add 10119 INFO [2022-01-10 08:40:02,490] jepsen worker 197 - jepsen.util 3197 :invoke :add 10120 INFO [2022-01-10 08:40:02,492] jepsen worker 230 - jepsen.util 2230 :info :add 8903 :RequestTimeoutException INFO [2022-01-10 08:40:02,493] jepsen worker 811 - jepsen.util 3811 :invoke :add 10121 INFO [2022-01-10 08:40:02,498] jepsen worker 230 - jepsen.util 3230 :invoke :add 10122 INFO [2022-01-10 08:40:02,517] jepsen worker 49 - jepsen.util 2049 :info :add 8904 :RequestTimeoutException INFO [2022-01-10 08:40:02,523] jepsen worker 687 - jepsen.util 1687 :info :add 8906 :RequestTimeoutException INFO [2022-01-10 08:40:02,523] jepsen worker 49 - jepsen.util 3049 :invoke :add 10123 INFO [2022-01-10 08:40:02,527] jepsen worker 687 - jepsen.util 2687 :invoke :add 10124 INFO [2022-01-10 08:40:02,531] jepsen worker 870 - jepsen.util 1870 :info :add 8907 :RequestTimeoutException INFO [2022-01-10 08:40:02,532] jepsen worker 841 - jepsen.util 1841 :info :add 8909 :RequestTimeoutException INFO [2022-01-10 08:40:02,533] jepsen worker 681 - jepsen.util 1681 :info :add 8908 :RequestTimeoutException INFO [2022-01-10 08:40:02,537] jepsen worker 842 - jepsen.util 1842 :info :add 8910 :RequestTimeoutException INFO [2022-01-10 08:40:02,541] jepsen worker 842 - jepsen.util 2842 :invoke :add 10125 INFO [2022-01-10 08:40:02,544] jepsen worker 681 - jepsen.util 2681 :invoke :add 10126 INFO [2022-01-10 08:40:02,547] jepsen worker 870 - jepsen.util 2870 :invoke :add 10127 INFO [2022-01-10 08:40:02,551] jepsen worker 841 - jepsen.util 2841 :invoke :add 10128 INFO [2022-01-10 08:40:02,600] jepsen worker 136 - jepsen.util 2136 :info :add 8911 :RequestTimeoutException INFO [2022-01-10 08:40:02,604] jepsen worker 136 - jepsen.util 3136 :invoke :add 10129 INFO [2022-01-10 08:40:02,631] jepsen worker 333 - jepsen.util 2333 :info :add 8913 :RequestTimeoutException INFO [2022-01-10 08:40:02,633] jepsen worker 265 - jepsen.util 2265 :info :add 8912 :RequestTimeoutException INFO [2022-01-10 08:40:02,638] jepsen worker 265 - jepsen.util 3265 :invoke :add 10130 INFO [2022-01-10 08:40:02,641] jepsen worker 333 - jepsen.util 3333 :invoke :add 10131 INFO [2022-01-10 08:40:02,663] jepsen worker 404 - jepsen.util 2404 :info :add 8918 :RequestTimeoutException INFO [2022-01-10 08:40:02,670] jepsen worker 404 - jepsen.util 3404 :invoke :add 10132 INFO [2022-01-10 08:40:02,690] jepsen worker 491 - jepsen.util 2491 :ok :add 8991 INFO [2022-01-10 08:40:02,692] jepsen worker 491 - jepsen.util 2491 :invoke :add 10133 INFO [2022-01-10 08:40:02,772] jepsen worker 688 - jepsen.util 2688 :info :add 8925 :RequestTimeoutException INFO [2022-01-10 08:40:02,776] jepsen worker 688 - jepsen.util 3688 :invoke :add 10134 INFO [2022-01-10 08:40:02,777] jepsen worker 924 - jepsen.util 2924 :info :add 8924 :RequestTimeoutException INFO [2022-01-10 08:40:02,777] jepsen worker 338 - jepsen.util 338 :info :add 8926 :RequestTimeoutException INFO [2022-01-10 08:40:02,778] jepsen worker 108 - jepsen.util 2108 :info :add 8927 :RequestTimeoutException INFO [2022-01-10 08:40:02,785] jepsen worker 108 - jepsen.util 3108 :invoke :add 10135 INFO [2022-01-10 08:40:02,788] jepsen worker 924 - jepsen.util 3924 :invoke :add 10136 INFO [2022-01-10 08:40:02,790] jepsen worker 338 - jepsen.util 1338 :invoke :add 10137 INFO [2022-01-10 08:40:02,972] jepsen worker 747 - jepsen.util 2747 :info :add 8930 :RequestTimeoutException INFO [2022-01-10 08:40:02,973] jepsen worker 366 - jepsen.util 2366 :info :add 8929 :RequestTimeoutException INFO [2022-01-10 08:40:02,979] jepsen worker 747 - jepsen.util 3747 :invoke :add 10138 INFO [2022-01-10 08:40:02,981] jepsen worker 366 - jepsen.util 3366 :invoke :add 10139 INFO [2022-01-10 08:40:03,002] jepsen worker 4 - jepsen.util 2004 :info :add 8932 :RequestTimeoutException INFO [2022-01-10 08:40:03,003] jepsen worker 725 - jepsen.util 2725 :info :add 8931 :RequestTimeoutException INFO [2022-01-10 08:40:03,007] jepsen worker 484 - jepsen.util 2484 :info :add 8933 :RequestTimeoutException INFO [2022-01-10 08:40:03,010] jepsen worker 863 - jepsen.util 2863 :info :add 8934 :RequestTimeoutException INFO [2022-01-10 08:40:03,011] jepsen worker 4 - jepsen.util 3004 :invoke :add 10140 INFO [2022-01-10 08:40:03,015] jepsen worker 725 - jepsen.util 3725 :invoke :add 10141 INFO [2022-01-10 08:40:03,017] jepsen worker 863 - jepsen.util 3863 :invoke :add 10142 INFO [2022-01-10 08:40:03,020] jepsen worker 484 - jepsen.util 3484 :invoke :add 10143 INFO [2022-01-10 08:40:03,063] jepsen worker 347 - jepsen.util 2347 :info :add 8935 :RequestTimeoutException INFO [2022-01-10 08:40:03,070] jepsen worker 347 - jepsen.util 3347 :invoke :add 10144 INFO [2022-01-10 08:40:03,071] jepsen worker 846 - jepsen.util 2846 :info :add 8936 :RequestTimeoutException INFO [2022-01-10 08:40:03,076] jepsen worker 846 - jepsen.util 3846 :invoke :add 10145 INFO [2022-01-10 08:40:03,104] jepsen worker 614 - jepsen.util 2614 :info :add 8937 :RequestTimeoutException INFO [2022-01-10 08:40:03,109] jepsen worker 614 - jepsen.util 3614 :invoke :add 10146 INFO [2022-01-10 08:40:03,112] jepsen worker 514 - jepsen.util 2514 :info :add 8941 :RequestTimeoutException INFO [2022-01-10 08:40:03,112] jepsen worker 192 - jepsen.util 2192 :ok :add 9660 INFO [2022-01-10 08:40:03,112] jepsen worker 19 - jepsen.util 3019 :ok :add 9678 INFO [2022-01-10 08:40:03,112] jepsen worker 590 - jepsen.util 2590 :ok :add 10023 INFO [2022-01-10 08:40:03,114] jepsen worker 617 - jepsen.util 3617 :ok :add 9945 INFO [2022-01-10 08:40:03,114] jepsen worker 210 - jepsen.util 2210 :ok :add 9969 INFO [2022-01-10 08:40:03,114] jepsen worker 81 - jepsen.util 3081 :ok :add 9654 INFO [2022-01-10 08:40:03,117] jepsen worker 196 - jepsen.util 2196 :info :add 8942 :RequestTimeoutException INFO [2022-01-10 08:40:03,118] jepsen worker 514 - jepsen.util 3514 :invoke :add 10147 INFO [2022-01-10 08:40:03,122] jepsen worker 19 - jepsen.util 3019 :invoke :add 10148 INFO [2022-01-10 08:40:03,123] jepsen worker 174 - jepsen.util 2174 :info :add 8943 :RequestTimeoutException WARN [2022-01-10 08:40:03,125] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","encode_duration_us":288,"operation_id":"0x232b","timeout_ms":10000,"total_duration_us":10010907},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","encode_duration_us":285,"operation_id":"0x27b2","timeout_ms":10000,"total_duration_us":10010648},{"operation_name":"insert","encode_duration_us":304,"operation_id":"0x259f","timeout_ms":10000,"total_duration_us":10010532},{"operation_name":"insert","last_dispatch_duration_us":8102906,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":8102906,"total_server_duration_us":2241641,"encode_duration_us":270,"operation_id":"0x24e6","timeout_ms":10000,"last_server_duration_us":2241641,"total_duration_us":10010523},{"operation_name":"insert","encode_duration_us":298,"operation_id":"0x2439","timeout_ms":10000,"total_duration_us":10010509},{"operation_name":"insert","last_dispatch_duration_us":9922722,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000001/00000000CB02B145","last_local_socket":"172.23.104.79:41914","total_dispatch_duration_us":9922722,"total_server_duration_us":6639462,"encode_duration_us":255,"operation_id":"0x24cd","timeout_ms":10000,"last_server_duration_us":6639462,"total_duration_us":10010420},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000001/00000000CB02B145","last_local_socket":"172.23.104.79:41914","encode_duration_us":258,"operation_id":"0x24df","timeout_ms":10000,"total_duration_us":10010374},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000001/00000000CB02B145","last_local_socket":"172.23.104.79:41914","encode_duration_us":252,"operation_id":"0x241a","timeout_ms":10000,"total_duration_us":10010246},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","encode_duration_us":252,"operation_id":"0x23fb","timeout_ms":10000,"total_duration_us":10009954},{"operation_name":"insert","encode_duration_us":265,"operation_id":"0x22cb","timeout_ms":10000,"total_duration_us":10009865}],"total_count":159}} INFO [2022-01-10 08:40:03,125] jepsen worker 81 - jepsen.util 3081 :invoke :add 10149 WARN [2022-01-10 08:40:03,125] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":17455421,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17455421,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":9004490,"encode_duration_us":264,"operation_id":"0x1f5d","timeout_ms":10000,"last_server_duration_us":9004490,"total_duration_us":10010173},{"operation_name":"insert","last_dispatch_duration_us":21568314,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":21568314,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":9016150,"encode_duration_us":298,"operation_id":"0x1f18","timeout_ms":10000,"last_server_duration_us":9016150,"total_duration_us":10010209},{"operation_name":"insert","last_dispatch_duration_us":8102906,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":8102906,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":2241641,"encode_duration_us":270,"operation_id":"0x24e6","timeout_ms":10000,"last_server_duration_us":2241641,"total_duration_us":10010523},{"operation_name":"insert","last_dispatch_duration_us":17335886,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17335886,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":787633,"encode_duration_us":349,"operation_id":"0x1fc5","timeout_ms":10000,"last_server_duration_us":787633,"total_duration_us":10010344},{"operation_name":"insert","last_dispatch_duration_us":9922722,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9922722,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":6639462,"encode_duration_us":255,"operation_id":"0x24cd","timeout_ms":10000,"last_server_duration_us":6639462,"total_duration_us":10010420},{"operation_name":"insert","last_dispatch_duration_us":21411046,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":21411046,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":5589470,"encode_duration_us":268,"operation_id":"0x2039","timeout_ms":10000,"last_server_duration_us":5589470,"total_duration_us":10066110},{"operation_name":"insert","last_dispatch_duration_us":18676351,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18676351,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":4247388,"encode_duration_us":261,"operation_id":"0x2128","timeout_ms":10000,"last_server_duration_us":4247388,"total_duration_us":10010606},{"operation_name":"insert","last_dispatch_duration_us":19710084,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19710084,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":3862524,"encode_duration_us":363,"operation_id":"0x202e","timeout_ms":10000,"last_server_duration_us":3862524,"total_duration_us":10093305},{"operation_name":"insert","last_dispatch_duration_us":13207587,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13207587,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":4494400,"encode_duration_us":271,"operation_id":"0x2030","timeout_ms":10000,"last_server_duration_us":4494400,"total_duration_us":10081294},{"operation_name":"insert","last_dispatch_duration_us":17008059,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17008059,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":8009733,"encode_duration_us":269,"operation_id":"0x1f70","timeout_ms":10000,"last_server_duration_us":8009733,"total_duration_us":10010644}],"total_count":119}} INFO [2022-01-10 08:40:03,129] jepsen worker 174 - jepsen.util 3174 :invoke :add 10150 INFO [2022-01-10 08:40:03,131] jepsen worker 254 - jepsen.util 2254 :info :add 8944 :RequestTimeoutException INFO [2022-01-10 08:40:03,132] jepsen worker 196 - jepsen.util 3196 :invoke :add 10151 INFO [2022-01-10 08:40:03,133] jepsen worker 814 - jepsen.util 2814 :info :add 8945 :RequestTimeoutException INFO [2022-01-10 08:40:03,136] jepsen worker 192 - jepsen.util 2192 :invoke :add 10152 INFO [2022-01-10 08:40:03,139] jepsen worker 814 - jepsen.util 3814 :invoke :add 10153 INFO [2022-01-10 08:40:03,142] jepsen worker 617 - jepsen.util 3617 :invoke :add 10154 INFO [2022-01-10 08:40:03,142] jepsen worker 618 - jepsen.util 2618 :info :add 8946 :RequestTimeoutException INFO [2022-01-10 08:40:03,144] jepsen worker 210 - jepsen.util 2210 :invoke :add 10155 INFO [2022-01-10 08:40:03,147] jepsen worker 336 - jepsen.util 2336 :info :add 8947 :RequestTimeoutException INFO [2022-01-10 08:40:03,148] jepsen worker 618 - jepsen.util 3618 :invoke :add 10156 INFO [2022-01-10 08:40:03,152] jepsen worker 590 - jepsen.util 2590 :invoke :add 10157 INFO [2022-01-10 08:40:03,155] jepsen worker 254 - jepsen.util 3254 :invoke :add 10158 INFO [2022-01-10 08:40:03,159] jepsen worker 336 - jepsen.util 3336 :invoke :add 10159 INFO [2022-01-10 08:40:03,191] jepsen worker 472 - jepsen.util 2472 :info :add 8954 :RequestTimeoutException INFO [2022-01-10 08:40:03,193] jepsen worker 795 - jepsen.util 2795 :info :add 8955 :RequestTimeoutException INFO [2022-01-10 08:40:03,195] jepsen worker 472 - jepsen.util 3472 :invoke :add 10160 INFO [2022-01-10 08:40:03,199] jepsen worker 795 - jepsen.util 3795 :invoke :add 10161 INFO [2022-01-10 08:40:03,202] jepsen worker 971 - jepsen.util 2971 :info :add 8956 :RequestTimeoutException INFO [2022-01-10 08:40:03,203] jepsen worker 391 - jepsen.util 2391 :info :add 8957 :RequestTimeoutException INFO [2022-01-10 08:40:03,207] jepsen worker 391 - jepsen.util 3391 :invoke :add 10162 INFO [2022-01-10 08:40:03,210] jepsen worker 971 - jepsen.util 3971 :invoke :add 10163 INFO [2022-01-10 08:40:03,277] jepsen worker 788 - jepsen.util 1788 :info :add 8959 :RequestTimeoutException INFO [2022-01-10 08:40:03,280] jepsen worker 799 - jepsen.util 1799 :info :add 8960 :RequestTimeoutException INFO [2022-01-10 08:40:03,281] jepsen worker 788 - jepsen.util 2788 :invoke :add 10164 INFO [2022-01-10 08:40:03,284] jepsen worker 799 - jepsen.util 2799 :invoke :add 10165 INFO [2022-01-10 08:40:03,327] jepsen worker 440 - jepsen.util 2440 :info :add 8961 :RequestTimeoutException INFO [2022-01-10 08:40:03,330] jepsen worker 889 - jepsen.util 2889 :info :add 8962 :RequestTimeoutException INFO [2022-01-10 08:40:03,331] jepsen worker 440 - jepsen.util 3440 :invoke :add 10166 INFO [2022-01-10 08:40:03,331] jepsen worker 780 - jepsen.util 2780 :info :add 8963 :RequestTimeoutException INFO [2022-01-10 08:40:03,337] jepsen worker 780 - jepsen.util 3780 :invoke :add 10167 INFO [2022-01-10 08:40:03,340] jepsen worker 889 - jepsen.util 3889 :invoke :add 10168 INFO [2022-01-10 08:40:03,361] jepsen worker 168 - jepsen.util 2168 :info :add 8969 :RequestTimeoutException INFO [2022-01-10 08:40:03,362] jepsen worker 748 - jepsen.util 1748 :info :add 8968 :RequestTimeoutException INFO [2022-01-10 08:40:03,364] jepsen worker 613 - jepsen.util 2613 :info :add 8970 :RequestTimeoutException INFO [2022-01-10 08:40:03,369] jepsen worker 748 - jepsen.util 2748 :invoke :add 10169 INFO [2022-01-10 08:40:03,371] jepsen worker 168 - jepsen.util 3168 :invoke :add 10170 INFO [2022-01-10 08:40:03,372] jepsen worker 469 - jepsen.util 2469 :info :add 8972 :RequestTimeoutException INFO [2022-01-10 08:40:03,374] jepsen worker 613 - jepsen.util 3613 :invoke :add 10171 INFO [2022-01-10 08:40:03,379] jepsen worker 469 - jepsen.util 3469 :invoke :add 10172 INFO [2022-01-10 08:40:03,407] jepsen worker 199 - jepsen.util 1199 :info :add 8973 :RequestTimeoutException INFO [2022-01-10 08:40:03,410] jepsen worker 18 - jepsen.util 1018 :info :add 8974 :RequestTimeoutException INFO [2022-01-10 08:40:03,414] jepsen worker 199 - jepsen.util 2199 :invoke :add 10173 INFO [2022-01-10 08:40:03,417] jepsen worker 18 - jepsen.util 2018 :invoke :add 10174 INFO [2022-01-10 08:40:03,503] jepsen worker 67 - jepsen.util 1067 :info :add 8975 :RequestTimeoutException INFO [2022-01-10 08:40:03,508] jepsen worker 67 - jepsen.util 2067 :invoke :add 10175 INFO [2022-01-10 08:40:03,531] jepsen worker 293 - jepsen.util 1293 :info :add 8976 :RequestTimeoutException INFO [2022-01-10 08:40:03,536] jepsen worker 293 - jepsen.util 2293 :invoke :add 10176 INFO [2022-01-10 08:40:03,543] jepsen worker 640 - jepsen.util 1640 :info :add 8977 :RequestTimeoutException INFO [2022-01-10 08:40:03,549] jepsen worker 640 - jepsen.util 2640 :invoke :add 10177 INFO [2022-01-10 08:40:03,582] jepsen worker 722 - jepsen.util 1722 :info :add 8978 :RequestTimeoutException INFO [2022-01-10 08:40:03,587] jepsen worker 722 - jepsen.util 2722 :invoke :add 10178 INFO [2022-01-10 08:40:03,591] jepsen worker 538 - jepsen.util 1538 :ok :add 9662 INFO [2022-01-10 08:40:03,592] jepsen worker 736 - jepsen.util 2736 :ok :add 9973 INFO [2022-01-10 08:40:03,592] jepsen worker 161 - jepsen.util 3161 :ok :add 10021 INFO [2022-01-10 08:40:03,592] jepsen worker 720 - jepsen.util 3720 :ok :add 10015 INFO [2022-01-10 08:40:03,598] jepsen worker 161 - jepsen.util 3161 :invoke :add 10179 INFO [2022-01-10 08:40:03,600] jepsen worker 538 - jepsen.util 1538 :invoke :add 10180 INFO [2022-01-10 08:40:03,603] jepsen worker 736 - jepsen.util 2736 :invoke :add 10181 INFO [2022-01-10 08:40:03,606] jepsen worker 720 - jepsen.util 3720 :invoke :add 10182 INFO [2022-01-10 08:40:03,607] jepsen worker 667 - jepsen.util 2667 :info :add 8979 :RequestTimeoutException INFO [2022-01-10 08:40:03,610] jepsen worker 939 - jepsen.util 2939 :info :add 8980 :RequestTimeoutException INFO [2022-01-10 08:40:03,612] jepsen worker 667 - jepsen.util 3667 :invoke :add 10183 INFO [2022-01-10 08:40:03,612] jepsen worker 822 - jepsen.util 2822 :info :add 8981 :RequestTimeoutException INFO [2022-01-10 08:40:03,618] jepsen worker 939 - jepsen.util 3939 :invoke :add 10184 INFO [2022-01-10 08:40:03,621] jepsen worker 14 - jepsen.util 2014 :info :add 8982 :RequestTimeoutException INFO [2022-01-10 08:40:03,621] jepsen worker 822 - jepsen.util 3822 :invoke :add 10185 INFO [2022-01-10 08:40:03,626] jepsen worker 14 - jepsen.util 3014 :invoke :add 10186 INFO [2022-01-10 08:40:03,642] jepsen worker 729 - jepsen.util 2729 :info :add 8988 :RequestTimeoutException INFO [2022-01-10 08:40:03,646] jepsen worker 879 - jepsen.util 2879 :info :add 8989 :RequestTimeoutException INFO [2022-01-10 08:40:03,647] jepsen worker 729 - jepsen.util 3729 :invoke :add 10187 INFO [2022-01-10 08:40:03,650] jepsen worker 526 - jepsen.util 1526 :info :add 8990 :RequestTimeoutException INFO [2022-01-10 08:40:03,651] jepsen worker 879 - jepsen.util 3879 :invoke :add 10188 INFO [2022-01-10 08:40:03,655] jepsen worker 526 - jepsen.util 2526 :invoke :add 10189 INFO [2022-01-10 08:40:03,731] jepsen worker 511 - jepsen.util 2511 :info :add 8992 :RequestTimeoutException INFO [2022-01-10 08:40:03,736] jepsen worker 511 - jepsen.util 3511 :invoke :add 10190 INFO [2022-01-10 08:40:03,743] jepsen worker 500 - jepsen.util 2500 :info :add 8994 :RequestTimeoutException INFO [2022-01-10 08:40:03,743] jepsen worker 329 - jepsen.util 2329 :info :add 8995 :RequestTimeoutException INFO [2022-01-10 08:40:03,744] jepsen worker 182 - jepsen.util 1182 :info :add 8993 :RequestTimeoutException INFO [2022-01-10 08:40:03,751] jepsen worker 182 - jepsen.util 2182 :invoke :add 10191 INFO [2022-01-10 08:40:03,752] jepsen worker 834 - jepsen.util 2834 :info :add 8996 :RequestTimeoutException INFO [2022-01-10 08:40:03,753] jepsen worker 329 - jepsen.util 3329 :invoke :add 10192 INFO [2022-01-10 08:40:03,756] jepsen worker 834 - jepsen.util 3834 :invoke :add 10193 INFO [2022-01-10 08:40:03,758] jepsen worker 397 - jepsen.util 2397 :ok :add 9334 INFO [2022-01-10 08:40:03,759] jepsen worker 500 - jepsen.util 3500 :invoke :add 10194 INFO [2022-01-10 08:40:03,761] jepsen worker 188 - jepsen.util 2188 :ok :add 9071 INFO [2022-01-10 08:40:03,763] jepsen worker 397 - jepsen.util 2397 :invoke :add 10195 INFO [2022-01-10 08:40:03,766] jepsen worker 188 - jepsen.util 2188 :invoke :add 10196 INFO [2022-01-10 08:40:03,897] jepsen worker 441 - jepsen.util 2441 :info :add 8997 :RequestTimeoutException INFO [2022-01-10 08:40:03,900] jepsen worker 324 - jepsen.util 1324 :info :add 8998 :RequestTimeoutException INFO [2022-01-10 08:40:03,903] jepsen worker 919 - jepsen.util 1919 :info :add 8999 :RequestTimeoutException INFO [2022-01-10 08:40:03,903] jepsen worker 441 - jepsen.util 3441 :invoke :add 10197 INFO [2022-01-10 08:40:03,909] jepsen worker 324 - jepsen.util 2324 :invoke :add 10198 INFO [2022-01-10 08:40:03,911] jepsen worker 832 - jepsen.util 2832 :info :add 9000 :RequestTimeoutException INFO [2022-01-10 08:40:03,912] jepsen worker 476 - jepsen.util 1476 :info :add 9002 :RequestTimeoutException INFO [2022-01-10 08:40:03,912] jepsen worker 919 - jepsen.util 2919 :invoke :add 10199 INFO [2022-01-10 08:40:03,913] jepsen worker 936 - jepsen.util 1936 :info :add 9001 :RequestTimeoutException INFO [2022-01-10 08:40:03,917] jepsen worker 259 - jepsen.util 1259 :info :add 9003 :RequestTimeoutException INFO [2022-01-10 08:40:03,919] jepsen worker 476 - jepsen.util 2476 :invoke :add 10200 INFO [2022-01-10 08:40:03,923] jepsen worker 936 - jepsen.util 2936 :invoke :add 10201 INFO [2022-01-10 08:40:03,925] jepsen worker 832 - jepsen.util 3832 :invoke :add 10202 INFO [2022-01-10 08:40:03,927] jepsen worker 259 - jepsen.util 2259 :invoke :add 10203 INFO [2022-01-10 08:40:03,950] jepsen worker 734 - jepsen.util 1734 :info :add 9004 :RequestTimeoutException INFO [2022-01-10 08:40:03,953] jepsen worker 734 - jepsen.util 2734 :invoke :add 10204 INFO [2022-01-10 08:40:03,961] jepsen worker 918 - jepsen.util 2918 :info :add 9006 :RequestTimeoutException INFO [2022-01-10 08:40:03,966] jepsen worker 918 - jepsen.util 3918 :invoke :add 10205 INFO [2022-01-10 08:40:03,993] jepsen worker 986 - jepsen.util 2986 :info :add 9007 :RequestTimeoutException INFO [2022-01-10 08:40:03,999] jepsen worker 986 - jepsen.util 3986 :invoke :add 10206 INFO [2022-01-10 08:40:04,012] jepsen worker 600 - jepsen.util 3600 :ok :add 9713 INFO [2022-01-10 08:40:04,013] jepsen worker 835 - jepsen.util 1835 :ok :add 9727 INFO [2022-01-10 08:40:04,013] jepsen worker 253 - jepsen.util 3253 :ok :add 9683 INFO [2022-01-10 08:40:04,013] jepsen worker 365 - jepsen.util 3365 :ok :add 9802 INFO [2022-01-10 08:40:04,017] jepsen worker 64 - jepsen.util 3064 :ok :add 9803 INFO [2022-01-10 08:40:04,017] jepsen worker 96 - jepsen.util 2096 :ok :add 9682 INFO [2022-01-10 08:40:04,017] jepsen worker 749 - jepsen.util 2749 :ok :add 9726 INFO [2022-01-10 08:40:04,017] jepsen worker 516 - jepsen.util 2516 :ok :add 9993 INFO [2022-01-10 08:40:04,017] jepsen worker 647 - jepsen.util 2647 :ok :add 9712 INFO [2022-01-10 08:40:04,019] jepsen worker 365 - jepsen.util 3365 :invoke :add 10207 INFO [2022-01-10 08:40:04,022] jepsen worker 196 - jepsen.util 3196 :ok :add 10151 INFO [2022-01-10 08:40:04,024] jepsen worker 96 - jepsen.util 2096 :invoke :add 10208 INFO [2022-01-10 08:40:04,028] jepsen worker 647 - jepsen.util 2647 :invoke :add 10209 INFO [2022-01-10 08:40:04,031] jepsen worker 196 - jepsen.util 3196 :invoke :add 10210 INFO [2022-01-10 08:40:04,033] jepsen worker 835 - jepsen.util 1835 :invoke :add 10211 INFO [2022-01-10 08:40:04,037] jepsen worker 749 - jepsen.util 2749 :invoke :add 10212 INFO [2022-01-10 08:40:04,039] jepsen worker 600 - jepsen.util 3600 :invoke :add 10213 INFO [2022-01-10 08:40:04,043] jepsen worker 253 - jepsen.util 3253 :invoke :add 10214 INFO [2022-01-10 08:40:04,045] jepsen worker 516 - jepsen.util 2516 :invoke :add 10215 INFO [2022-01-10 08:40:04,048] jepsen worker 64 - jepsen.util 3064 :invoke :add 10216 INFO [2022-01-10 08:40:04,062] jepsen worker 227 - jepsen.util 2227 :info :add 9014 :RequestTimeoutException INFO [2022-01-10 08:40:04,067] jepsen worker 243 - jepsen.util 2243 :info :add 9015 :RequestTimeoutException INFO [2022-01-10 08:40:04,068] jepsen worker 227 - jepsen.util 3227 :invoke :add 10217 INFO [2022-01-10 08:40:04,070] jepsen worker 217 - jepsen.util 2217 :info :add 9016 :RequestTimeoutException INFO [2022-01-10 08:40:04,073] jepsen worker 925 - jepsen.util 2925 :info :add 9017 :RequestTimeoutException INFO [2022-01-10 08:40:04,073] jepsen worker 243 - jepsen.util 3243 :invoke :add 10218 INFO [2022-01-10 08:40:04,077] jepsen worker 217 - jepsen.util 3217 :invoke :add 10219 INFO [2022-01-10 08:40:04,080] jepsen worker 925 - jepsen.util 3925 :invoke :add 10220 INFO [2022-01-10 08:40:04,081] jepsen worker 539 - jepsen.util 2539 :info :add 9018 :RequestTimeoutException INFO [2022-01-10 08:40:04,082] jepsen worker 902 - jepsen.util 2902 :info :add 9020 :RequestTimeoutException INFO [2022-01-10 08:40:04,084] jepsen worker 447 - jepsen.util 2447 :info :add 9019 :RequestTimeoutException INFO [2022-01-10 08:40:04,087] jepsen worker 103 - jepsen.util 2103 :info :add 9021 :RequestTimeoutException INFO [2022-01-10 08:40:04,088] jepsen worker 902 - jepsen.util 3902 :invoke :add 10221 INFO [2022-01-10 08:40:04,090] jepsen worker 631 - jepsen.util 2631 :info :add 9022 :RequestTimeoutException INFO [2022-01-10 08:40:04,093] jepsen worker 447 - jepsen.util 3447 :invoke :add 10222 INFO [2022-01-10 08:40:04,097] jepsen worker 103 - jepsen.util 3103 :invoke :add 10223 INFO [2022-01-10 08:40:04,099] jepsen worker 631 - jepsen.util 3631 :invoke :add 10224 INFO [2022-01-10 08:40:04,102] jepsen worker 539 - jepsen.util 3539 :invoke :add 10225 INFO [2022-01-10 08:40:04,103] jepsen worker 170 - jepsen.util 2170 :info :add 9023 :RequestTimeoutException INFO [2022-01-10 08:40:04,108] jepsen worker 170 - jepsen.util 3170 :invoke :add 10226 INFO [2022-01-10 08:40:04,114] jepsen worker 151 - jepsen.util 2151 :info :add 9028 :RequestTimeoutException INFO [2022-01-10 08:40:04,117] jepsen worker 151 - jepsen.util 3151 :invoke :add 10227 INFO [2022-01-10 08:40:04,121] jepsen worker 569 - jepsen.util 2569 :info :add 9030 :RequestTimeoutException INFO [2022-01-10 08:40:04,122] jepsen worker 110 - jepsen.util 2110 :info :add 9029 :RequestTimeoutException INFO [2022-01-10 08:40:04,128] jepsen worker 110 - jepsen.util 3110 :invoke :add 10228 INFO [2022-01-10 08:40:04,131] jepsen worker 569 - jepsen.util 3569 :invoke :add 10229 INFO [2022-01-10 08:40:04,197] jepsen worker 696 - jepsen.util 1696 :info :add 9031 :RequestTimeoutException INFO [2022-01-10 08:40:04,200] jepsen worker 159 - jepsen.util 2159 :info :add 9032 :RequestTimeoutException INFO [2022-01-10 08:40:04,201] jepsen worker 284 - jepsen.util 1284 :info :add 9034 :RequestTimeoutException INFO [2022-01-10 08:40:04,202] jepsen worker 696 - jepsen.util 2696 :invoke :add 10230 INFO [2022-01-10 08:40:04,203] jepsen worker 101 - jepsen.util 1101 :info :add 9035 :RequestTimeoutException INFO [2022-01-10 08:40:04,207] jepsen worker 159 - jepsen.util 3159 :invoke :add 10231 INFO [2022-01-10 08:40:04,209] jepsen worker 101 - jepsen.util 2101 :invoke :add 10232 INFO [2022-01-10 08:40:04,210] jepsen worker 770 - jepsen.util 770 :info :add 9036 :RequestTimeoutException INFO [2022-01-10 08:40:04,212] jepsen worker 284 - jepsen.util 2284 :invoke :add 10233 INFO [2022-01-10 08:40:04,215] jepsen worker 770 - jepsen.util 1770 :invoke :add 10234 INFO [2022-01-10 08:40:04,220] jepsen worker 273 - jepsen.util 1273 :info :add 9037 :RequestTimeoutException INFO [2022-01-10 08:40:04,220] jepsen worker 38 - jepsen.util 2038 :info :add 9039 :RequestTimeoutException INFO [2022-01-10 08:40:04,223] jepsen worker 32 - jepsen.util 1032 :info :add 9038 :RequestTimeoutException INFO [2022-01-10 08:40:04,228] jepsen worker 273 - jepsen.util 2273 :invoke :add 10235 INFO [2022-01-10 08:40:04,231] jepsen worker 32 - jepsen.util 2032 :invoke :add 10236 INFO [2022-01-10 08:40:04,234] jepsen worker 38 - jepsen.util 3038 :invoke :add 10237 INFO [2022-01-10 08:40:04,302] jepsen worker 140 - jepsen.util 1140 :info :add 9040 :RequestTimeoutException INFO [2022-01-10 08:40:04,306] jepsen worker 140 - jepsen.util 2140 :invoke :add 10238 INFO [2022-01-10 08:40:04,307] jepsen worker 479 - jepsen.util 1479 :info :add 9041 :RequestTimeoutException INFO [2022-01-10 08:40:04,310] jepsen worker 68 - jepsen.util 1068 :info :add 9042 :RequestTimeoutException INFO [2022-01-10 08:40:04,313] jepsen worker 672 - jepsen.util 1672 :info :add 9043 :RequestTimeoutException INFO [2022-01-10 08:40:04,314] jepsen worker 68 - jepsen.util 2068 :invoke :add 10239 INFO [2022-01-10 08:40:04,318] jepsen worker 479 - jepsen.util 2479 :invoke :add 10240 INFO [2022-01-10 08:40:04,321] jepsen worker 672 - jepsen.util 2672 :invoke :add 10241 INFO [2022-01-10 08:40:04,331] jepsen worker 464 - jepsen.util 2464 :info :add 9044 :RequestTimeoutException INFO [2022-01-10 08:40:04,333] jepsen worker 426 - jepsen.util 2426 :info :add 9045 :RequestTimeoutException INFO [2022-01-10 08:40:04,338] jepsen worker 464 - jepsen.util 3464 :invoke :add 10242 INFO [2022-01-10 08:40:04,340] jepsen worker 426 - jepsen.util 3426 :invoke :add 10243 INFO [2022-01-10 08:40:04,342] jepsen worker 209 - jepsen.util 2209 :info :add 9046 :RequestTimeoutException INFO [2022-01-10 08:40:04,347] jepsen worker 209 - jepsen.util 3209 :invoke :add 10244 INFO [2022-01-10 08:40:04,347] jepsen worker 586 - jepsen.util 2586 :info :add 9047 :RequestTimeoutException INFO [2022-01-10 08:40:04,350] jepsen worker 932 - jepsen.util 2932 :info :add 9048 :RequestTimeoutException INFO [2022-01-10 08:40:04,352] jepsen worker 586 - jepsen.util 3586 :invoke :add 10245 INFO [2022-01-10 08:40:04,356] jepsen worker 932 - jepsen.util 3932 :invoke :add 10246 INFO [2022-01-10 08:40:04,361] jepsen worker 708 - jepsen.util 2708 :info :add 9050 :RequestTimeoutException INFO [2022-01-10 08:40:04,363] jepsen worker 929 - jepsen.util 1929 :info :add 9049 :RequestTimeoutException INFO [2022-01-10 08:40:04,366] jepsen worker 929 - jepsen.util 2929 :invoke :add 10247 INFO [2022-01-10 08:40:04,368] jepsen worker 708 - jepsen.util 3708 :invoke :add 10248 INFO [2022-01-10 08:40:04,371] jepsen worker 662 - jepsen.util 1662 :info :add 9052 :RequestTimeoutException INFO [2022-01-10 08:40:04,374] jepsen worker 148 - jepsen.util 1148 :info :add 9051 :RequestTimeoutException INFO [2022-01-10 08:40:04,375] jepsen worker 662 - jepsen.util 2662 :invoke :add 10249 INFO [2022-01-10 08:40:04,379] jepsen worker 148 - jepsen.util 2148 :invoke :add 10250 INFO [2022-01-10 08:40:04,387] jepsen worker 142 - jepsen.util 1142 :info :add 9053 :RequestTimeoutException INFO [2022-01-10 08:40:04,392] jepsen worker 142 - jepsen.util 2142 :invoke :add 10251 INFO [2022-01-10 08:40:04,413] jepsen worker 191 - jepsen.util 2191 :info :add 9058 :RequestTimeoutException INFO [2022-01-10 08:40:04,419] jepsen worker 191 - jepsen.util 3191 :invoke :add 10252 INFO [2022-01-10 08:40:04,430] jepsen worker 946 - jepsen.util 1946 :info :add 9067 :RequestTimeoutException INFO [2022-01-10 08:40:04,431] jepsen worker 444 - jepsen.util 2444 :info :add 9064 :RequestTimeoutException INFO [2022-01-10 08:40:04,432] jepsen worker 622 - jepsen.util 1622 :info :add 9065 :RequestTimeoutException INFO [2022-01-10 08:40:04,433] jepsen worker 920 - jepsen.util 2920 :info :add 9066 :RequestTimeoutException INFO [2022-01-10 08:40:04,437] jepsen worker 398 - jepsen.util 2398 :info :add 9068 :RequestTimeoutException INFO [2022-01-10 08:40:04,437] jepsen worker 513 - jepsen.util 1513 :info :add 9069 :RequestTimeoutException INFO [2022-01-10 08:40:04,438] jepsen worker 920 - jepsen.util 3920 :invoke :add 10253 INFO [2022-01-10 08:40:04,443] jepsen worker 348 - jepsen.util 2348 :info :add 9070 :RequestTimeoutException INFO [2022-01-10 08:40:04,443] jepsen worker 25 - jepsen.util 2025 :info :add 9072 :RequestTimeoutException INFO [2022-01-10 08:40:04,445] jepsen worker 444 - jepsen.util 3444 :invoke :add 10254 INFO [2022-01-10 08:40:04,451] jepsen worker 339 - jepsen.util 1339 :info :add 9073 :RequestTimeoutException INFO [2022-01-10 08:40:04,451] jepsen worker 195 - jepsen.util 2195 :info :add 9074 :RequestTimeoutException INFO [2022-01-10 08:40:04,451] jepsen worker 978 - jepsen.util 2978 :info :add 9075 :RequestTimeoutException INFO [2022-01-10 08:40:04,452] jepsen worker 733 - jepsen.util 1733 :info :add 9077 :RequestTimeoutException INFO [2022-01-10 08:40:04,453] jepsen worker 513 - jepsen.util 2513 :invoke :add 10255 INFO [2022-01-10 08:40:04,454] jepsen worker 776 - jepsen.util 1776 :info :add 9076 :RequestTimeoutException INFO [2022-01-10 08:40:04,461] jepsen worker 776 - jepsen.util 2776 :invoke :add 10256 INFO [2022-01-10 08:40:04,464] jepsen worker 978 - jepsen.util 3978 :invoke :add 10257 INFO [2022-01-10 08:40:04,467] jepsen worker 339 - jepsen.util 2339 :invoke :add 10258 INFO [2022-01-10 08:40:04,470] jepsen worker 195 - jepsen.util 3195 :invoke :add 10259 INFO [2022-01-10 08:40:04,472] jepsen worker 733 - jepsen.util 2733 :invoke :add 10260 INFO [2022-01-10 08:40:04,475] jepsen worker 398 - jepsen.util 3398 :invoke :add 10261 INFO [2022-01-10 08:40:04,477] jepsen worker 946 - jepsen.util 2946 :invoke :add 10262 INFO [2022-01-10 08:40:04,480] jepsen worker 622 - jepsen.util 2622 :invoke :add 10263 INFO [2022-01-10 08:40:04,480] jepsen worker 460 - jepsen.util 2460 :info :add 9084 :RequestTimeoutException INFO [2022-01-10 08:40:04,481] jepsen worker 490 - jepsen.util 2490 :info :add 9086 :RequestTimeoutException INFO [2022-01-10 08:40:04,482] jepsen worker 78 - jepsen.util 1078 :info :add 9087 :RequestTimeoutException INFO [2022-01-10 08:40:04,483] jepsen worker 947 - jepsen.util 1947 :info :add 9085 :RequestTimeoutException INFO [2022-01-10 08:40:04,487] jepsen worker 907 - jepsen.util 1907 :info :add 9088 :RequestTimeoutException INFO [2022-01-10 08:40:04,488] jepsen worker 460 - jepsen.util 3460 :invoke :add 10264 INFO [2022-01-10 08:40:04,491] jepsen worker 619 - jepsen.util 2619 :info :add 9089 :RequestTimeoutException INFO [2022-01-10 08:40:04,492] jepsen worker 947 - jepsen.util 2947 :invoke :add 10265 INFO [2022-01-10 08:40:04,493] jepsen worker 853 - jepsen.util 1853 :info :add 9090 :RequestTimeoutException INFO [2022-01-10 08:40:04,497] jepsen worker 853 - jepsen.util 2853 :invoke :add 10266 INFO [2022-01-10 08:40:04,500] jepsen worker 348 - jepsen.util 3348 :invoke :add 10267 INFO [2022-01-10 08:40:04,501] jepsen worker 208 - jepsen.util 1208 :info :add 9091 :RequestTimeoutException INFO [2022-01-10 08:40:04,502] jepsen worker 524 - jepsen.util 2524 :info :add 9093 :RequestTimeoutException INFO [2022-01-10 08:40:04,503] jepsen worker 25 - jepsen.util 3025 :invoke :add 10268 INFO [2022-01-10 08:40:04,503] jepsen worker 588 - jepsen.util 1588 :info :add 9092 :RequestTimeoutException INFO [2022-01-10 08:40:04,507] jepsen worker 896 - jepsen.util 2896 :info :add 9094 :RequestTimeoutException INFO [2022-01-10 08:40:04,509] jepsen worker 490 - jepsen.util 3490 :invoke :add 10269 INFO [2022-01-10 08:40:04,510] jepsen worker 693 - jepsen.util 2693 :info :add 9095 :RequestTimeoutException INFO [2022-01-10 08:40:04,513] jepsen worker 477 - jepsen.util 2477 :info :add 9096 :RequestTimeoutException INFO [2022-01-10 08:40:04,514] jepsen worker 154 - jepsen.util 1154 :info :add 9097 :RequestTimeoutException INFO [2022-01-10 08:40:04,516] jepsen worker 588 - jepsen.util 2588 :invoke :add 10270 INFO [2022-01-10 08:40:04,517] jepsen worker 456 - jepsen.util 2456 :info :add 9101 :RequestTimeoutException INFO [2022-01-10 08:40:04,520] jepsen worker 292 - jepsen.util 2292 :info :add 9102 :RequestTimeoutException INFO [2022-01-10 08:40:04,522] jepsen worker 382 - jepsen.util 2382 :info :add 9100 :RequestTimeoutException INFO [2022-01-10 08:40:04,523] jepsen worker 619 - jepsen.util 3619 :invoke :add 10271 INFO [2022-01-10 08:40:04,523] jepsen worker 200 - jepsen.util 2200 :info :add 9104 :RequestTimeoutException WARN [2022-01-10 08:40:04,526] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":18511938,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18511938,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":9002371,"encode_duration_us":263,"operation_id":"0x20cc","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10009493},{"operation_name":"insert","last_dispatch_duration_us":12665786,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12665786,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":3863263,"encode_duration_us":270,"operation_id":"0x2158","timeout_ms":10000,"last_server_duration_us":3863263,"total_duration_us":10009624},{"operation_name":"insert","last_dispatch_duration_us":10249041,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10249041,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":987956,"encode_duration_us":253,"operation_id":"0x26a1","timeout_ms":10000,"last_server_duration_us":987956,"total_duration_us":10009770},{"operation_name":"insert","last_dispatch_duration_us":17184558,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17184558,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9008729,"encode_duration_us":273,"operation_id":"0x22f4","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10010333},{"operation_name":"insert","last_dispatch_duration_us":17546684,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17546684,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":3312579,"encode_duration_us":265,"operation_id":"0x227e","timeout_ms":10000,"last_server_duration_us":3312579,"total_duration_us":10010236},{"operation_name":"insert","last_dispatch_duration_us":17784319,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17784319,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9016150,"encode_duration_us":270,"operation_id":"0x22c0","timeout_ms":10000,"last_server_duration_us":9016150,"total_duration_us":10010382},{"operation_name":"insert","last_dispatch_duration_us":11872480,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11872480,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":3045828,"encode_duration_us":277,"operation_id":"0x2272","timeout_ms":10000,"last_server_duration_us":3045828,"total_duration_us":10010296},{"operation_name":"insert","last_dispatch_duration_us":14630752,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14630752,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":6100822,"encode_duration_us":274,"operation_id":"0x203d","timeout_ms":10000,"last_server_duration_us":6100822,"total_duration_us":10054215},{"operation_name":"insert","last_dispatch_duration_us":16023687,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16023687,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":7159830,"encode_duration_us":270,"operation_id":"0x203a","timeout_ms":10000,"last_server_duration_us":7159830,"total_duration_us":10064064},{"operation_name":"insert","last_dispatch_duration_us":18551289,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18551289,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":9002371,"encode_duration_us":330,"operation_id":"0x20bb","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10010594}],"total_count":66}} INFO [2022-01-10 08:40:04,526] jepsen worker 624 - jepsen.util 2624 :info :add 9103 :RequestTimeoutException WARN [2022-01-10 08:40:04,527] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":14492671,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14492671,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":6159311,"encode_duration_us":473,"operation_id":"0x2555","timeout_ms":10000,"last_server_duration_us":6159311,"total_duration_us":10008907},{"operation_name":"insert","last_dispatch_duration_us":20188658,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20188658,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":1574865,"encode_duration_us":269,"operation_id":"0x208a","timeout_ms":10000,"last_server_duration_us":1574865,"total_duration_us":10009302},{"operation_name":"insert","last_dispatch_duration_us":20759165,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":20759165,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":9022513,"encode_duration_us":295,"operation_id":"0x1fe0","timeout_ms":10000,"last_server_duration_us":9022513,"total_duration_us":10009474},{"operation_name":"insert","last_dispatch_duration_us":22759460,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":22759460,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":1585988,"encode_duration_us":267,"operation_id":"0x1ef1","timeout_ms":10000,"last_server_duration_us":1585988,"total_duration_us":10009887},{"operation_name":"insert","last_dispatch_duration_us":13062939,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13062939,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":9011909,"encode_duration_us":270,"operation_id":"0x2201","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10009790},{"operation_name":"insert","last_dispatch_duration_us":9945116,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9945116,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":9001311,"encode_duration_us":265,"operation_id":"0x2265","timeout_ms":10000,"last_server_duration_us":9001311,"total_duration_us":10010582},{"operation_name":"insert","last_dispatch_duration_us":13483746,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13483746,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":9002371,"encode_duration_us":260,"operation_id":"0x2469","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10010003},{"operation_name":"insert","last_dispatch_duration_us":23408278,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":23408278,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":9010849,"encode_duration_us":609,"operation_id":"0x1e0e","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10010452},{"operation_name":"insert","last_dispatch_duration_us":13479361,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13479361,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":9007669,"encode_duration_us":253,"operation_id":"0x26b7","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10011143},{"operation_name":"insert","last_dispatch_duration_us":22369974,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":22369974,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":9006609,"encode_duration_us":339,"operation_id":"0x1eb7","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":10010464}],"total_count":61}} WARN [2022-01-10 08:40:04,527] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","encode_duration_us":292,"operation_id":"0x247b","timeout_ms":10000,"total_duration_us":10011661},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","encode_duration_us":254,"operation_id":"0x2576","timeout_ms":10000,"total_duration_us":10011099},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":330,"operation_id":"0x289a","timeout_ms":10000,"total_duration_us":10010938},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":260,"operation_id":"0x2751","timeout_ms":10000,"total_duration_us":10010768},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","encode_duration_us":255,"operation_id":"0x2438","timeout_ms":10000,"total_duration_us":10010728},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":441,"operation_id":"0x23b9","timeout_ms":10000,"total_duration_us":10010562},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":299,"operation_id":"0x25b5","timeout_ms":10000,"total_duration_us":10010356},{"operation_name":"insert","encode_duration_us":253,"operation_id":"0x279b","timeout_ms":10000,"total_duration_us":10010319},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":260,"operation_id":"0x24f6","timeout_ms":10000,"total_duration_us":10010258},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","encode_duration_us":260,"operation_id":"0x2473","timeout_ms":10000,"total_duration_us":10010125}],"total_count":180}} WARN [2022-01-10 08:40:04,527] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","encode_duration_us":453,"operation_id":"0x27ed","timeout_ms":10000,"total_duration_us":10011913},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","encode_duration_us":379,"operation_id":"0x2701","timeout_ms":10000,"total_duration_us":10011567},{"operation_name":"insert","last_dispatch_duration_us":13479361,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","total_dispatch_duration_us":13479361,"total_server_duration_us":9007669,"encode_duration_us":253,"operation_id":"0x26b7","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10011143},{"operation_name":"insert","encode_duration_us":250,"operation_id":"0x26d2","timeout_ms":10000,"total_duration_us":10010684},{"operation_name":"insert","last_dispatch_duration_us":13483746,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","total_dispatch_duration_us":13483746,"total_server_duration_us":9002371,"encode_duration_us":260,"operation_id":"0x2469","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10010003},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","encode_duration_us":478,"operation_id":"0x23c3","timeout_ms":10000,"total_duration_us":10009798},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","encode_duration_us":253,"operation_id":"0x2864","timeout_ms":10000,"total_duration_us":10009534},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","encode_duration_us":261,"operation_id":"0x2702","timeout_ms":10000,"total_duration_us":10009472},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","encode_duration_us":255,"operation_id":"0x23e0","timeout_ms":10000,"total_duration_us":10009294},{"operation_name":"insert","encode_duration_us":269,"operation_id":"0x249b","timeout_ms":10000,"total_duration_us":10009236}],"total_count":187}} INFO [2022-01-10 08:40:04,531] jepsen worker 17 - jepsen.util 2017 :info :add 9105 :RequestTimeoutException INFO [2022-01-10 08:40:04,531] jepsen worker 208 - jepsen.util 2208 :invoke :add 10272 WARN [2022-01-10 08:40:04,532] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000002/00000000F4655C1A","last_local_socket":"172.23.104.79:41954","encode_duration_us":255,"operation_id":"0x244e","timeout_ms":10000,"total_duration_us":10011109},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000002/00000000F4655C1A","last_local_socket":"172.23.104.79:41954","encode_duration_us":260,"operation_id":"0x2631","timeout_ms":10000,"total_duration_us":10010916},{"operation_name":"insert","encode_duration_us":293,"operation_id":"0x2668","timeout_ms":10000,"total_duration_us":10010750},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","encode_duration_us":259,"operation_id":"0x23ec","timeout_ms":10000,"total_duration_us":10010748},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000002/00000000F4655C1A","last_local_socket":"172.23.104.79:41954","encode_duration_us":294,"operation_id":"0x236f","timeout_ms":10000,"total_duration_us":10010710},{"operation_name":"insert","encode_duration_us":262,"operation_id":"0x27ce","timeout_ms":10000,"total_duration_us":10010520},{"operation_name":"insert","last_dispatch_duration_us":18051408,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","total_dispatch_duration_us":18051408,"total_server_duration_us":1960422,"encode_duration_us":259,"operation_id":"0x2380","timeout_ms":10000,"last_server_duration_us":1960422,"total_duration_us":10010413},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","encode_duration_us":260,"operation_id":"0x2707","timeout_ms":10000,"total_duration_us":10010199},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000002/00000000F4655C1A","last_local_socket":"172.23.104.79:41954","encode_duration_us":295,"operation_id":"0x25ff","timeout_ms":10000,"total_duration_us":10010144},{"operation_name":"insert","last_dispatch_duration_us":15704125,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","total_dispatch_duration_us":15704125,"total_server_duration_us":997083,"encode_duration_us":451,"operation_id":"0x24af","timeout_ms":10000,"last_server_duration_us":997083,"total_duration_us":10010100}],"total_count":181}} WARN [2022-01-10 08:40:04,532] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":22456444,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":22456444,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":9009789,"encode_duration_us":266,"operation_id":"0x1f3f","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10009707},{"operation_name":"insert","last_dispatch_duration_us":22503413,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":22503413,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":1148366,"encode_duration_us":283,"operation_id":"0x1e8a","timeout_ms":10000,"last_server_duration_us":1148366,"total_duration_us":10009791},{"operation_name":"insert","last_dispatch_duration_us":15704125,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":15704125,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":997083,"encode_duration_us":451,"operation_id":"0x24af","timeout_ms":10000,"last_server_duration_us":997083,"total_duration_us":10010100},{"operation_name":"insert","last_dispatch_duration_us":25879500,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25879500,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":6574434,"encode_duration_us":281,"operation_id":"0x1fe4","timeout_ms":10000,"last_server_duration_us":6574434,"total_duration_us":10010268},{"operation_name":"insert","last_dispatch_duration_us":22136677,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":22136677,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":9007669,"encode_duration_us":259,"operation_id":"0x1f62","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10009947},{"operation_name":"insert","last_dispatch_duration_us":22437593,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":22437593,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":9008729,"encode_duration_us":279,"operation_id":"0x1f49","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10010426},{"operation_name":"insert","last_dispatch_duration_us":20994396,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":20994396,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":8981189,"encode_duration_us":280,"operation_id":"0x2020","timeout_ms":10000,"last_server_duration_us":8981189,"total_duration_us":10010693},{"operation_name":"insert","last_dispatch_duration_us":16518331,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16518331,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":2079955,"encode_duration_us":266,"operation_id":"0x20a5","timeout_ms":10000,"last_server_duration_us":2079955,"total_duration_us":10010439},{"operation_name":"insert","last_dispatch_duration_us":22138152,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":22138152,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":3058537,"encode_duration_us":258,"operation_id":"0x2001","timeout_ms":10000,"last_server_duration_us":3058537,"total_duration_us":10011000},{"operation_name":"insert","last_dispatch_duration_us":18051408,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18051408,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":1960422,"encode_duration_us":259,"operation_id":"0x2380","timeout_ms":10000,"last_server_duration_us":1960422,"total_duration_us":10010413}],"total_count":61}} INFO [2022-01-10 08:40:04,534] jepsen worker 127 - jepsen.util 1127 :info :add 9106 :RequestTimeoutException INFO [2022-01-10 08:40:04,535] jepsen worker 907 - jepsen.util 2907 :invoke :add 10273 INFO [2022-01-10 08:40:04,539] jepsen worker 524 - jepsen.util 3524 :invoke :add 10274 INFO [2022-01-10 08:40:04,542] jepsen worker 105 - jepsen.util 2105 :info :add 9107 :RequestTimeoutException INFO [2022-01-10 08:40:04,542] jepsen worker 896 - jepsen.util 3896 :invoke :add 10275 INFO [2022-01-10 08:40:04,547] jepsen worker 624 - jepsen.util 3624 :invoke :add 10276 INFO [2022-01-10 08:40:04,549] jepsen worker 292 - jepsen.util 3292 :invoke :add 10277 INFO [2022-01-10 08:40:04,552] jepsen worker 200 - jepsen.util 3200 :invoke :add 10278 INFO [2022-01-10 08:40:04,555] jepsen worker 477 - jepsen.util 3477 :invoke :add 10279 INFO [2022-01-10 08:40:04,557] jepsen worker 654 - jepsen.util 2654 :info :add 9114 :RequestTimeoutException INFO [2022-01-10 08:40:04,558] jepsen worker 78 - jepsen.util 2078 :invoke :add 10280 INFO [2022-01-10 08:40:04,560] jepsen worker 744 - jepsen.util 2744 :info :add 9115 :RequestTimeoutException INFO [2022-01-10 08:40:04,562] jepsen worker 654 - jepsen.util 3654 :invoke :add 10281 INFO [2022-01-10 08:40:04,566] jepsen worker 154 - jepsen.util 2154 :invoke :add 10282 INFO [2022-01-10 08:40:04,569] jepsen worker 105 - jepsen.util 3105 :invoke :add 10283 INFO [2022-01-10 08:40:04,571] jepsen worker 147 - jepsen.util 1147 :info :add 9117 :RequestTimeoutException INFO [2022-01-10 08:40:04,572] jepsen worker 17 - jepsen.util 3017 :invoke :add 10284 INFO [2022-01-10 08:40:04,572] jepsen worker 851 - jepsen.util 1851 :info :add 9119 :RequestTimeoutException INFO [2022-01-10 08:40:04,573] jepsen worker 289 - jepsen.util 2289 :info :add 9116 :RequestTimeoutException INFO [2022-01-10 08:40:04,574] jepsen worker 486 - jepsen.util 1486 :info :add 9118 :RequestTimeoutException INFO [2022-01-10 08:40:04,577] jepsen worker 695 - jepsen.util 2695 :info :add 9120 :RequestTimeoutException INFO [2022-01-10 08:40:04,579] jepsen worker 289 - jepsen.util 3289 :invoke :add 10285 INFO [2022-01-10 08:40:04,581] jepsen worker 27 - jepsen.util 1027 :info :add 9121 :RequestTimeoutException INFO [2022-01-10 08:40:04,582] jepsen worker 1 - jepsen.util 2001 :info :add 9122 :RequestTimeoutException INFO [2022-01-10 08:40:04,584] jepsen worker 744 - jepsen.util 3744 :invoke :add 10286 INFO [2022-01-10 08:40:04,588] jepsen worker 1 - jepsen.util 3001 :invoke :add 10287 INFO [2022-01-10 08:40:04,591] jepsen worker 695 - jepsen.util 3695 :invoke :add 10288 INFO [2022-01-10 08:40:04,591] jepsen worker 296 - jepsen.util 2296 :info :add 9123 :RequestTimeoutException INFO [2022-01-10 08:40:04,593] jepsen worker 851 - jepsen.util 2851 :invoke :add 10289 INFO [2022-01-10 08:40:04,597] jepsen worker 693 - jepsen.util 3693 :invoke :add 10290 INFO [2022-01-10 08:40:04,600] jepsen worker 456 - jepsen.util 3456 :invoke :add 10291 INFO [2022-01-10 08:40:04,603] jepsen worker 127 - jepsen.util 2127 :invoke :add 10292 INFO [2022-01-10 08:40:04,607] jepsen worker 382 - jepsen.util 3382 :invoke :add 10293 INFO [2022-01-10 08:40:04,610] jepsen worker 486 - jepsen.util 2486 :invoke :add 10294 INFO [2022-01-10 08:40:04,611] jepsen worker 276 - jepsen.util 2276 :info :add 9129 :RequestTimeoutException INFO [2022-01-10 08:40:04,612] jepsen worker 465 - jepsen.util 2465 :info :add 9131 :RequestTimeoutException INFO [2022-01-10 08:40:04,612] jepsen worker 27 - jepsen.util 2027 :invoke :add 10295 INFO [2022-01-10 08:40:04,613] jepsen worker 185 - jepsen.util 2185 :info :add 9128 :RequestTimeoutException INFO [2022-01-10 08:40:04,614] jepsen worker 212 - jepsen.util 1212 :info :add 9130 :RequestTimeoutException INFO [2022-01-10 08:40:04,616] jepsen worker 2 - jepsen.util 1002 :info :add 9132 :RequestTimeoutException INFO [2022-01-10 08:40:04,620] jepsen worker 124 - jepsen.util 2124 :info :add 9133 :RequestTimeoutException INFO [2022-01-10 08:40:04,623] jepsen worker 542 - jepsen.util 1542 :info :add 9134 :RequestTimeoutException INFO [2022-01-10 08:40:04,623] jepsen worker 2 - jepsen.util 2002 :invoke :add 10296 WARN [2022-01-10 08:40:04,628] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":21068245,"last_local_id":"2B99390F00000005/0000000013738E6A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":21068245,"last_local_socket":"172.23.104.79:41944","total_server_duration_us":9002371,"encode_duration_us":339,"operation_id":"0x1faf","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10009640},{"operation_name":"insert","last_dispatch_duration_us":12363393,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12363393,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9010849,"encode_duration_us":257,"operation_id":"0x2421","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10009654},{"operation_name":"insert","last_dispatch_duration_us":21898112,"last_local_id":"2B99390F00000005/0000000013738E6A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":21898112,"last_local_socket":"172.23.104.79:41944","total_server_duration_us":9012969,"encode_duration_us":264,"operation_id":"0x1f2d","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10010040},{"operation_name":"insert","last_dispatch_duration_us":19780942,"last_local_id":"2B99390F00000005/0000000013738E6A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":19780942,"last_local_socket":"172.23.104.79:41944","total_server_duration_us":6235249,"encode_duration_us":455,"operation_id":"0x1ea4","timeout_ms":10000,"last_server_duration_us":6235249,"total_duration_us":10009987},{"operation_name":"insert","last_dispatch_duration_us":14861416,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14861416,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9011909,"encode_duration_us":307,"operation_id":"0x25e4","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10009898},{"operation_name":"insert","last_dispatch_duration_us":11891586,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11891586,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":1694875,"encode_duration_us":446,"operation_id":"0x2629","timeout_ms":10000,"last_server_duration_us":1694875,"total_duration_us":10010768},{"operation_name":"insert","last_dispatch_duration_us":17570773,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17570773,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9007669,"encode_duration_us":330,"operation_id":"0x203e","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10014129},{"operation_name":"insert","last_dispatch_duration_us":11858889,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11858889,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":6038099,"encode_duration_us":291,"operation_id":"0x22c1","timeout_ms":10000,"last_server_duration_us":6038099,"total_duration_us":10010407},{"operation_name":"insert","last_dispatch_duration_us":14055688,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14055688,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":8407042,"encode_duration_us":251,"operation_id":"0x23cc","timeout_ms":10000,"last_server_duration_us":8407042,"total_duration_us":10011132},{"operation_name":"insert","last_dispatch_duration_us":17637295,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17637295,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9009789,"encode_duration_us":284,"operation_id":"0x2017","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10011152}],"total_count":87}} WARN [2022-01-10 08:40:04,629] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":14055688,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":14055688,"total_server_duration_us":8407042,"encode_duration_us":251,"operation_id":"0x23cc","timeout_ms":10000,"last_server_duration_us":8407042,"total_duration_us":10011132},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":255,"operation_id":"0x274e","timeout_ms":10000,"total_duration_us":10011058},{"operation_name":"insert","encode_duration_us":270,"operation_id":"0x2809","timeout_ms":10000,"total_duration_us":10010967},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":253,"operation_id":"0x2729","timeout_ms":10000,"total_duration_us":10010938},{"operation_name":"insert","last_dispatch_duration_us":11891586,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","total_dispatch_duration_us":11891586,"total_server_duration_us":1694875,"encode_duration_us":446,"operation_id":"0x2629","timeout_ms":10000,"last_server_duration_us":1694875,"total_duration_us":10010768},{"operation_name":"insert","encode_duration_us":279,"operation_id":"0x28de","timeout_ms":10000,"total_duration_us":10010739},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","encode_duration_us":279,"operation_id":"0x2900","timeout_ms":10000,"total_duration_us":10010688},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":261,"operation_id":"0x2683","timeout_ms":10000,"total_duration_us":10010604},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":257,"operation_id":"0x2623","timeout_ms":10000,"total_duration_us":10010447},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","encode_duration_us":292,"operation_id":"0x240f","timeout_ms":10000,"total_duration_us":10010297}],"total_count":190}} INFO [2022-01-10 08:40:04,629] jepsen worker 542 - jepsen.util 2542 :invoke :add 10297 INFO [2022-01-10 08:40:04,631] jepsen worker 528 - jepsen.util 1528 :info :add 9135 :RequestTimeoutException INFO [2022-01-10 08:40:04,632] jepsen worker 359 - jepsen.util 2359 :info :add 9137 :RequestTimeoutException INFO [2022-01-10 08:40:04,633] jepsen worker 124 - jepsen.util 3124 :invoke :add 10298 INFO [2022-01-10 08:40:04,634] jepsen worker 175 - jepsen.util 1175 :info :add 9136 :RequestTimeoutException INFO [2022-01-10 08:40:04,637] jepsen worker 207 - jepsen.util 2207 :info :add 9138 :RequestTimeoutException WARN [2022-01-10 08:40:04,638] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","encode_duration_us":256,"operation_id":"0x291d","timeout_ms":10000,"total_duration_us":10010993},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","encode_duration_us":319,"operation_id":"0x276a","timeout_ms":10000,"total_duration_us":10010925},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","encode_duration_us":291,"operation_id":"0x27b9","timeout_ms":10000,"total_duration_us":10010888},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","encode_duration_us":386,"operation_id":"0x28ca","timeout_ms":10000,"total_duration_us":10010838},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","encode_duration_us":273,"operation_id":"0x2906","timeout_ms":10000,"total_duration_us":10010688},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","encode_duration_us":254,"operation_id":"0x254a","timeout_ms":10000,"total_duration_us":10010666},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","encode_duration_us":260,"operation_id":"0x26c4","timeout_ms":10000,"total_duration_us":10010513},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","encode_duration_us":257,"operation_id":"0x28c4","timeout_ms":10000,"total_duration_us":10010456},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","encode_duration_us":297,"operation_id":"0x2672","timeout_ms":10000,"total_duration_us":10010451},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","encode_duration_us":256,"operation_id":"0x271d","timeout_ms":10000,"total_duration_us":10010423}],"total_count":178}} WARN [2022-01-10 08:40:04,638] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":10211327,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":10211327,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":2689646,"encode_duration_us":347,"operation_id":"0x25aa","timeout_ms":10000,"last_server_duration_us":2689646,"total_duration_us":10009626},{"operation_name":"insert","last_dispatch_duration_us":8009425,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":8009425,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":2518971,"encode_duration_us":263,"operation_id":"0x260c","timeout_ms":10000,"last_server_duration_us":2518971,"total_duration_us":10009698},{"operation_name":"insert","last_dispatch_duration_us":16589322,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16589322,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":9014029,"encode_duration_us":260,"operation_id":"0x2508","timeout_ms":10000,"last_server_duration_us":9014029,"total_duration_us":10009849},{"operation_name":"insert","last_dispatch_duration_us":11071378,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11071378,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":1181672,"encode_duration_us":263,"operation_id":"0x2309","timeout_ms":10000,"last_server_duration_us":1181672,"total_duration_us":10010074},{"operation_name":"insert","last_dispatch_duration_us":12820688,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12820688,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":4788566,"encode_duration_us":262,"operation_id":"0x24bf","timeout_ms":10000,"last_server_duration_us":4788566,"total_duration_us":10009774},{"operation_name":"insert","last_dispatch_duration_us":10508642,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10508642,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":1913563,"encode_duration_us":254,"operation_id":"0x23c9","timeout_ms":10000,"last_server_duration_us":1913563,"total_duration_us":10009923},{"operation_name":"insert","last_dispatch_duration_us":13187231,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13187231,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9009789,"encode_duration_us":307,"operation_id":"0x23ff","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10009909},{"operation_name":"insert","last_dispatch_duration_us":17815846,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17815846,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":9020392,"encode_duration_us":304,"operation_id":"0x2134","timeout_ms":10000,"last_server_duration_us":9020392,"total_duration_us":10010188},{"operation_name":"insert","last_dispatch_duration_us":14043295,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14043295,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9003430,"encode_duration_us":318,"operation_id":"0x2242","timeout_ms":10000,"last_server_duration_us":9003430,"total_duration_us":10010404},{"operation_name":"insert","last_dispatch_duration_us":17796820,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17796820,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":7498649,"encode_duration_us":294,"operation_id":"0x22ca","timeout_ms":10000,"last_server_duration_us":7498649,"total_duration_us":10010208}],"total_count":97}} INFO [2022-01-10 08:40:04,639] jepsen worker 528 - jepsen.util 2528 :invoke :add 10299 INFO [2022-01-10 08:40:04,640] jepsen worker 328 - jepsen.util 1328 :info :add 9139 :RequestTimeoutException INFO [2022-01-10 08:40:04,641] jepsen worker 518 - jepsen.util 2518 :info :add 9141 :RequestTimeoutException INFO [2022-01-10 08:40:04,642] jepsen worker 70 - jepsen.util 1070 :info :add 9140 :RequestTimeoutException INFO [2022-01-10 08:40:04,643] jepsen worker 84 - jepsen.util 2084 :info :add 9142 :RequestTimeoutException INFO [2022-01-10 08:40:04,648] jepsen worker 518 - jepsen.util 3518 :invoke :add 10300 INFO [2022-01-10 08:40:04,652] jepsen worker 403 - jepsen.util 1403 :info :add 9143 :RequestTimeoutException INFO [2022-01-10 08:40:04,652] jepsen worker 212 - jepsen.util 2212 :invoke :add 10301 INFO [2022-01-10 08:40:04,656] jepsen worker 147 - jepsen.util 2147 :invoke :add 10302 INFO [2022-01-10 08:40:04,657] jepsen worker 60 - jepsen.util 1060 :info :add 9144 :RequestTimeoutException INFO [2022-01-10 08:40:04,659] jepsen worker 276 - jepsen.util 3276 :invoke :add 10303 INFO [2022-01-10 08:40:04,660] jepsen worker 724 - jepsen.util 2724 :info :add 9145 :RequestTimeoutException INFO [2022-01-10 08:40:04,661] jepsen worker 568 - jepsen.util 1568 :info :add 9147 :RequestTimeoutException INFO [2022-01-10 08:40:04,662] jepsen worker 70 - jepsen.util 2070 :invoke :add 10304 INFO [2022-01-10 08:40:04,663] jepsen worker 763 - jepsen.util 1763 :info :add 9146 :RequestTimeoutException INFO [2022-01-10 08:40:04,664] jepsen worker 663 - jepsen.util 2663 :info :add 9148 :RequestTimeoutException INFO [2022-01-10 08:40:04,671] jepsen worker 84 - jepsen.util 3084 :invoke :add 10305 INFO [2022-01-10 08:40:04,672] jepsen worker 994 - jepsen.util 2994 :info :add 9149 :RequestTimeoutException INFO [2022-01-10 08:40:04,674] jepsen worker 175 - jepsen.util 2175 :invoke :add 10306 INFO [2022-01-10 08:40:04,676] jepsen worker 691 - jepsen.util 1691 :info :add 9150 :RequestTimeoutException INFO [2022-01-10 08:40:04,678] jepsen worker 568 - jepsen.util 2568 :invoke :add 10307 INFO [2022-01-10 08:40:04,680] jepsen worker 36 - jepsen.util 1036 :info :add 9151 :RequestTimeoutException INFO [2022-01-10 08:40:04,682] jepsen worker 724 - jepsen.util 3724 :invoke :add 10308 INFO [2022-01-10 08:40:04,683] jepsen worker 405 - jepsen.util 2405 :info :add 9152 :RequestTimeoutException INFO [2022-01-10 08:40:04,688] jepsen worker 405 - jepsen.util 3405 :invoke :add 10309 INFO [2022-01-10 08:40:04,691] jepsen worker 465 - jepsen.util 3465 :invoke :add 10310 INFO [2022-01-10 08:40:04,691] jepsen worker 400 - jepsen.util 2400 :info :add 9153 :RequestTimeoutException INFO [2022-01-10 08:40:04,694] jepsen worker 185 - jepsen.util 3185 :invoke :add 10311 INFO [2022-01-10 08:40:04,695] jepsen worker 955 - jepsen.util 2955 :ok :add 9798 INFO [2022-01-10 08:40:04,695] jepsen worker 288 - jepsen.util 3288 :ok :add 9889 INFO [2022-01-10 08:40:04,695] jepsen worker 529 - jepsen.util 2529 :ok :add 9901 INFO [2022-01-10 08:40:04,695] jepsen worker 952 - jepsen.util 2952 :ok :add 9891 INFO [2022-01-10 08:40:04,695] jepsen worker 42 - jepsen.util 3042 :ok :add 10067 INFO [2022-01-10 08:40:04,695] jepsen worker 591 - jepsen.util 3591 :ok :add 9919 INFO [2022-01-10 08:40:04,695] jepsen worker 290 - jepsen.util 3290 :ok :add 9780 INFO [2022-01-10 08:40:04,702] jepsen worker 994 - jepsen.util 3994 :invoke :add 10312 INFO [2022-01-10 08:40:04,705] jepsen worker 359 - jepsen.util 3359 :invoke :add 10313 INFO [2022-01-10 08:40:04,708] jepsen worker 952 - jepsen.util 2952 :invoke :add 10314 INFO [2022-01-10 08:40:04,711] jepsen worker 60 - jepsen.util 2060 :invoke :add 10315 INFO [2022-01-10 08:40:04,717] jepsen worker 445 - jepsen.util 2445 :info :add 9158 :RequestTimeoutException INFO [2022-01-10 08:40:04,719] jepsen worker 663 - jepsen.util 3663 :invoke :add 10316 INFO [2022-01-10 08:40:04,721] jepsen worker 33 - jepsen.util 1033 :info :add 9159 :RequestTimeoutException INFO [2022-01-10 08:40:04,724] jepsen worker 207 - jepsen.util 3207 :invoke :add 10317 INFO [2022-01-10 08:40:04,728] jepsen worker 296 - jepsen.util 3296 :invoke :add 10318 INFO [2022-01-10 08:40:04,731] jepsen worker 42 - jepsen.util 3042 :invoke :add 10319 INFO [2022-01-10 08:40:04,732] jepsen worker 80 - jepsen.util 2080 :info :add 9164 :RequestTimeoutException INFO [2022-01-10 08:40:04,732] jepsen worker 990 - jepsen.util 1990 :info :add 9167 :RequestTimeoutException INFO [2022-01-10 08:40:04,734] jepsen worker 290 - jepsen.util 3290 :invoke :add 10320 INFO [2022-01-10 08:40:04,736] jepsen worker 605 - jepsen.util 1605 :info :add 9165 :RequestTimeoutException INFO [2022-01-10 08:40:04,740] jepsen worker 866 - jepsen.util 2866 :info :add 9166 :RequestTimeoutException INFO [2022-01-10 08:40:04,741] jepsen worker 597 - jepsen.util 2597 :info :add 9168 :RequestTimeoutException INFO [2022-01-10 08:40:04,741] jepsen worker 450 - jepsen.util 2450 :info :add 9170 :RequestTimeoutException INFO [2022-01-10 08:40:04,741] jepsen worker 691 - jepsen.util 2691 :invoke :add 10321 INFO [2022-01-10 08:40:04,743] jepsen worker 909 - jepsen.util 2909 :info :add 9169 :RequestTimeoutException INFO [2022-01-10 08:40:04,749] jepsen worker 990 - jepsen.util 2990 :invoke :add 10322 INFO [2022-01-10 08:40:04,751] jepsen worker 386 - jepsen.util 1386 :info :add 9171 :RequestTimeoutException INFO [2022-01-10 08:40:04,751] jepsen worker 572 - jepsen.util 2572 :info :add 9173 :RequestTimeoutException INFO [2022-01-10 08:40:04,752] jepsen worker 648 - jepsen.util 2648 :info :add 9175 :RequestTimeoutException INFO [2022-01-10 08:40:04,753] jepsen worker 400 - jepsen.util 3400 :invoke :add 10323 INFO [2022-01-10 08:40:04,753] jepsen worker 789 - jepsen.util 2789 :info :add 9172 :RequestTimeoutException INFO [2022-01-10 08:40:04,753] jepsen worker 225 - jepsen.util 1225 :info :add 9174 :RequestTimeoutException INFO [2022-01-10 08:40:04,754] jepsen worker 670 - jepsen.util 1670 :info :add 9176 :RequestTimeoutException INFO [2022-01-10 08:40:04,760] jepsen worker 840 - jepsen.util 1840 :info :add 9177 :RequestTimeoutException INFO [2022-01-10 08:40:04,765] jepsen worker 597 - jepsen.util 3597 :invoke :add 10324 INFO [2022-01-10 08:40:04,768] jepsen worker 591 - jepsen.util 3591 :invoke :add 10325 INFO [2022-01-10 08:40:04,771] jepsen worker 33 - jepsen.util 2033 :invoke :add 10326 INFO [2022-01-10 08:40:04,774] jepsen worker 529 - jepsen.util 2529 :invoke :add 10327 INFO [2022-01-10 08:40:04,777] jepsen worker 572 - jepsen.util 3572 :invoke :add 10328 INFO [2022-01-10 08:40:04,778] jepsen worker 134 - jepsen.util 2134 :info :add 9184 :RequestTimeoutException INFO [2022-01-10 08:40:04,779] jepsen worker 445 - jepsen.util 3445 :invoke :add 10329 INFO [2022-01-10 08:40:04,780] jepsen worker 214 - jepsen.util 2214 :info :add 9187 :RequestTimeoutException INFO [2022-01-10 08:40:04,781] jepsen worker 351 - jepsen.util 1351 :info :add 9186 :RequestTimeoutException INFO [2022-01-10 08:40:04,783] jepsen worker 885 - jepsen.util 2885 :info :add 9185 :RequestTimeoutException INFO [2022-01-10 08:40:04,788] jepsen worker 955 - jepsen.util 2955 :invoke :add 10330 INFO [2022-01-10 08:40:04,791] jepsen worker 134 - jepsen.util 3134 :invoke :add 10331 INFO [2022-01-10 08:40:04,793] jepsen worker 450 - jepsen.util 3450 :invoke :add 10332 INFO [2022-01-10 08:40:04,796] jepsen worker 225 - jepsen.util 2225 :invoke :add 10333 INFO [2022-01-10 08:40:04,798] jepsen worker 763 - jepsen.util 2763 :invoke :add 10334 INFO [2022-01-10 08:40:04,801] jepsen worker 909 - jepsen.util 3909 :invoke :add 10335 INFO [2022-01-10 08:40:04,802] jepsen worker 930 - jepsen.util 2930 :info :add 9188 :RequestTimeoutException INFO [2022-01-10 08:40:04,804] jepsen worker 80 - jepsen.util 3080 :invoke :add 10336 INFO [2022-01-10 08:40:04,807] jepsen worker 355 - jepsen.util 2355 :info :add 9189 :RequestTimeoutException INFO [2022-01-10 08:40:04,809] jepsen worker 328 - jepsen.util 2328 :invoke :add 10337 INFO [2022-01-10 08:40:04,813] jepsen worker 930 - jepsen.util 3930 :invoke :add 10338 INFO [2022-01-10 08:40:04,813] jepsen worker 880 - jepsen.util 2880 :info :add 9190 :RequestTimeoutException INFO [2022-01-10 08:40:04,816] jepsen worker 605 - jepsen.util 2605 :invoke :add 10339 INFO [2022-01-10 08:40:04,817] jepsen worker 981 - jepsen.util 1981 :info :add 9191 :RequestTimeoutException INFO [2022-01-10 08:40:04,820] jepsen worker 139 - jepsen.util 2139 :info :add 9193 :RequestTimeoutException INFO [2022-01-10 08:40:04,820] jepsen worker 401 - jepsen.util 1401 :info :add 9192 :RequestTimeoutException INFO [2022-01-10 08:40:04,823] jepsen worker 840 - jepsen.util 2840 :invoke :add 10340 INFO [2022-01-10 08:40:04,828] jepsen worker 36 - jepsen.util 2036 :invoke :add 10341 INFO [2022-01-10 08:40:04,832] jepsen worker 376 - jepsen.util 2376 :info :add 9195 :RequestTimeoutException INFO [2022-01-10 08:40:04,833] jepsen worker 139 - jepsen.util 3139 :invoke :add 10342 INFO [2022-01-10 08:40:04,834] jepsen worker 704 - jepsen.util 2704 :info :add 9194 :RequestTimeoutException INFO [2022-01-10 08:40:04,837] jepsen worker 762 - jepsen.util 2762 :info :add 9196 :RequestTimeoutException INFO [2022-01-10 08:40:04,838] jepsen worker 376 - jepsen.util 3376 :invoke :add 10343 INFO [2022-01-10 08:40:04,843] jepsen worker 789 - jepsen.util 3789 :invoke :add 10344 INFO [2022-01-10 08:40:04,843] jepsen worker 240 - jepsen.util 1240 :info :add 9197 :RequestTimeoutException INFO [2022-01-10 08:40:04,845] jepsen worker 670 - jepsen.util 2670 :invoke :add 10345 INFO [2022-01-10 08:40:04,849] jepsen worker 288 - jepsen.util 3288 :invoke :add 10346 INFO [2022-01-10 08:40:04,850] jepsen worker 635 - jepsen.util 2635 :info :add 9200 :RequestTimeoutException INFO [2022-01-10 08:40:04,852] jepsen worker 351 - jepsen.util 2351 :invoke :add 10347 INFO [2022-01-10 08:40:04,853] jepsen worker 996 - jepsen.util 2996 :info :add 9201 :RequestTimeoutException INFO [2022-01-10 08:40:04,855] jepsen worker 648 - jepsen.util 3648 :invoke :add 10348 INFO [2022-01-10 08:40:04,859] jepsen worker 240 - jepsen.util 2240 :invoke :add 10349 INFO [2022-01-10 08:40:04,860] jepsen worker 508 - jepsen.util 2508 :info :add 9203 :RequestTimeoutException INFO [2022-01-10 08:40:04,861] jepsen worker 149 - jepsen.util 2149 :info :add 9202 :RequestTimeoutException INFO [2022-01-10 08:40:04,861] jepsen worker 635 - jepsen.util 3635 :invoke :add 10350 INFO [2022-01-10 08:40:04,862] jepsen worker 757 - jepsen.util 1757 :info :add 9204 :RequestTimeoutException INFO [2022-01-10 08:40:04,863] jepsen worker 268 - jepsen.util 2268 :info :add 9205 :RequestTimeoutException INFO [2022-01-10 08:40:04,867] jepsen worker 583 - jepsen.util 2583 :info :add 9206 :RequestTimeoutException INFO [2022-01-10 08:40:04,869] jepsen worker 355 - jepsen.util 3355 :invoke :add 10351 INFO [2022-01-10 08:40:04,871] jepsen worker 839 - jepsen.util 1839 :info :add 9207 :RequestTimeoutException INFO [2022-01-10 08:40:04,873] jepsen worker 386 - jepsen.util 2386 :invoke :add 10352 INFO [2022-01-10 08:40:04,877] jepsen worker 214 - jepsen.util 3214 :invoke :add 10353 INFO [2022-01-10 08:40:04,880] jepsen worker 996 - jepsen.util 3996 :invoke :add 10354 INFO [2022-01-10 08:40:04,883] jepsen worker 839 - jepsen.util 2839 :invoke :add 10355 INFO [2022-01-10 08:40:04,885] jepsen worker 880 - jepsen.util 3880 :invoke :add 10356 INFO [2022-01-10 08:40:04,888] jepsen worker 583 - jepsen.util 3583 :invoke :add 10357 INFO [2022-01-10 08:40:04,890] jepsen worker 401 - jepsen.util 2401 :invoke :add 10358 INFO [2022-01-10 08:40:04,893] jepsen worker 508 - jepsen.util 3508 :invoke :add 10359 INFO [2022-01-10 08:40:04,896] jepsen worker 866 - jepsen.util 3866 :invoke :add 10360 INFO [2022-01-10 08:40:04,900] jepsen worker 762 - jepsen.util 3762 :invoke :add 10361 INFO [2022-01-10 08:40:04,902] jepsen worker 268 - jepsen.util 3268 :invoke :add 10362 INFO [2022-01-10 08:40:04,903] jepsen worker 344 - jepsen.util 2344 :info :add 9214 :RequestTimeoutException INFO [2022-01-10 08:40:04,905] jepsen worker 757 - jepsen.util 2757 :invoke :add 10363 INFO [2022-01-10 08:40:04,909] jepsen worker 704 - jepsen.util 3704 :invoke :add 10364 INFO [2022-01-10 08:40:04,912] jepsen worker 410 - jepsen.util 2410 :info :add 9215 :RequestTimeoutException INFO [2022-01-10 08:40:04,912] jepsen worker 630 - jepsen.util 630 :info :add 9216 :RequestTimeoutException INFO [2022-01-10 08:40:04,913] jepsen worker 403 - jepsen.util 2403 :invoke :add 10365 INFO [2022-01-10 08:40:04,919] jepsen worker 410 - jepsen.util 3410 :invoke :add 10366 INFO [2022-01-10 08:40:04,923] jepsen worker 630 - jepsen.util 1630 :invoke :add 10367 INFO [2022-01-10 08:40:04,926] jepsen worker 981 - jepsen.util 2981 :invoke :add 10368 INFO [2022-01-10 08:40:04,929] jepsen worker 344 - jepsen.util 3344 :invoke :add 10369 INFO [2022-01-10 08:40:04,933] jepsen worker 885 - jepsen.util 3885 :invoke :add 10370 INFO [2022-01-10 08:40:04,935] jepsen worker 149 - jepsen.util 3149 :invoke :add 10371 INFO [2022-01-10 08:40:04,966] jepsen worker 782 - jepsen.util 3782 :ok :add 10052 INFO [2022-01-10 08:40:04,966] jepsen worker 335 - jepsen.util 3335 :ok :add 9781 INFO [2022-01-10 08:40:04,966] jepsen worker 580 - jepsen.util 3580 :ok :add 9890 INFO [2022-01-10 08:40:04,966] jepsen worker 573 - jepsen.util 3573 :ok :add 9799 INFO [2022-01-10 08:40:04,966] jepsen worker 910 - jepsen.util 2910 :ok :add 10066 INFO [2022-01-10 08:40:04,971] jepsen worker 910 - jepsen.util 2910 :invoke :add 10372 INFO [2022-01-10 08:40:04,975] jepsen worker 335 - jepsen.util 3335 :invoke :add 10373 INFO [2022-01-10 08:40:04,978] jepsen worker 580 - jepsen.util 3580 :invoke :add 10374 INFO [2022-01-10 08:40:04,980] jepsen worker 573 - jepsen.util 3573 :invoke :add 10375 INFO [2022-01-10 08:40:04,983] jepsen worker 782 - jepsen.util 3782 :invoke :add 10376 INFO [2022-01-10 08:40:05,012] jepsen worker 987 - jepsen.util 1987 :info :add 9217 :RequestTimeoutException INFO [2022-01-10 08:40:05,016] jepsen worker 987 - jepsen.util 2987 :invoke :add 10377 INFO [2022-01-10 08:40:05,017] jepsen worker 815 - jepsen.util 1815 :info :add 9218 :RequestTimeoutException INFO [2022-01-10 08:40:05,020] jepsen worker 838 - jepsen.util 1838 :info :add 9219 :RequestTimeoutException INFO [2022-01-10 08:40:05,022] jepsen worker 815 - jepsen.util 2815 :invoke :add 10378 INFO [2022-01-10 08:40:05,022] jepsen worker 819 - jepsen.util 1819 :info :add 9220 :RequestTimeoutException INFO [2022-01-10 08:40:05,027] jepsen worker 819 - jepsen.util 2819 :invoke :add 10379 INFO [2022-01-10 08:40:05,030] jepsen worker 838 - jepsen.util 2838 :invoke :add 10380 INFO [2022-01-10 08:40:05,032] jepsen worker 778 - jepsen.util 2778 :info :add 9221 :RequestTimeoutException INFO [2022-01-10 08:40:05,033] jepsen worker 792 - jepsen.util 2792 :info :add 9222 :RequestTimeoutException INFO [2022-01-10 08:40:05,038] jepsen worker 792 - jepsen.util 3792 :invoke :add 10381 INFO [2022-01-10 08:40:05,041] jepsen worker 778 - jepsen.util 3778 :invoke :add 10382 INFO [2022-01-10 08:40:05,042] jepsen worker 615 - jepsen.util 2615 :info :add 9223 :RequestTimeoutException INFO [2022-01-10 08:40:05,047] jepsen worker 615 - jepsen.util 3615 :invoke :add 10383 INFO [2022-01-10 08:40:05,147] jepsen worker 394 - jepsen.util 1394 :info :add 9228 :RequestTimeoutException INFO [2022-01-10 08:40:05,153] jepsen worker 394 - jepsen.util 2394 :invoke :add 10384 INFO [2022-01-10 08:40:05,170] jepsen worker 423 - jepsen.util 1423 :info :add 9229 :RequestTimeoutException INFO [2022-01-10 08:40:05,177] jepsen worker 423 - jepsen.util 2423 :invoke :add 10385 INFO [2022-01-10 08:40:05,181] jepsen worker 41 - jepsen.util 1041 :info :add 9231 :RequestTimeoutException INFO [2022-01-10 08:40:05,183] jepsen worker 509 - jepsen.util 2509 :info :add 9230 :RequestTimeoutException INFO [2022-01-10 08:40:05,189] jepsen worker 509 - jepsen.util 3509 :invoke :add 10386 INFO [2022-01-10 08:40:05,183] jepsen worker 451 - jepsen.util 1451 :info :add 9232 :RequestTimeoutException INFO [2022-01-10 08:40:05,187] jepsen worker 576 - jepsen.util 1576 :info :add 9234 :RequestTimeoutException INFO [2022-01-10 08:40:05,190] jepsen worker 737 - jepsen.util 1737 :info :add 9235 :RequestTimeoutException INFO [2022-01-10 08:40:05,192] jepsen worker 560 - jepsen.util 2560 :info :add 9233 :RequestTimeoutException INFO [2022-01-10 08:40:05,193] jepsen worker 304 - jepsen.util 1304 :info :add 9236 :RequestTimeoutException INFO [2022-01-10 08:40:05,199] jepsen worker 576 - jepsen.util 2576 :invoke :add 10387 INFO [2022-01-10 08:40:05,201] jepsen worker 281 - jepsen.util 1281 :info :add 9237 :RequestTimeoutException INFO [2022-01-10 08:40:05,203] jepsen worker 560 - jepsen.util 3560 :invoke :add 10388 INFO [2022-01-10 08:40:05,208] jepsen worker 281 - jepsen.util 2281 :invoke :add 10389 INFO [2022-01-10 08:40:05,211] jepsen worker 451 - jepsen.util 2451 :invoke :add 10390 INFO [2022-01-10 08:40:05,212] jepsen worker 960 - jepsen.util 1960 :info :add 9239 :RequestTimeoutException INFO [2022-01-10 08:40:05,212] jepsen worker 970 - jepsen.util 2970 :ok :add 9783 INFO [2022-01-10 08:40:05,212] jepsen worker 677 - jepsen.util 3677 :ok :add 9902 INFO [2022-01-10 08:40:05,212] jepsen worker 593 - jepsen.util 2593 :ok :add 10050 INFO [2022-01-10 08:40:05,213] jepsen worker 712 - jepsen.util 3712 :ok :add 9892 INFO [2022-01-10 08:40:05,213] jepsen worker 676 - jepsen.util 2676 :info :add 9238 :RequestTimeoutException INFO [2022-01-10 08:40:05,216] jepsen worker 737 - jepsen.util 2737 :invoke :add 10391 INFO [2022-01-10 08:40:05,216] jepsen worker 888 - jepsen.util 2888 :info :add 9240 :RequestTimeoutException INFO [2022-01-10 08:40:05,220] jepsen worker 682 - jepsen.util 2682 :info :add 9241 :RequestTimeoutException INFO [2022-01-10 08:40:05,223] jepsen worker 439 - jepsen.util 2439 :info :add 9242 :RequestTimeoutException INFO [2022-01-10 08:40:05,225] jepsen worker 304 - jepsen.util 2304 :invoke :add 10392 INFO [2022-01-10 08:40:05,229] jepsen worker 970 - jepsen.util 2970 :invoke :add 10393 INFO [2022-01-10 08:40:05,231] jepsen worker 774 - jepsen.util 1774 :info :add 9243 :RequestTimeoutException INFO [2022-01-10 08:40:05,232] jepsen worker 888 - jepsen.util 3888 :invoke :add 10394 INFO [2022-01-10 08:40:05,232] jepsen worker 143 - jepsen.util 2143 :info :add 9245 :RequestTimeoutException INFO [2022-01-10 08:40:05,233] jepsen worker 74 - jepsen.util 2074 :info :add 9244 :RequestTimeoutException INFO [2022-01-10 08:40:05,237] jepsen worker 855 - jepsen.util 1855 :info :add 9246 :RequestTimeoutException INFO [2022-01-10 08:40:05,239] jepsen worker 439 - jepsen.util 3439 :invoke :add 10395 INFO [2022-01-10 08:40:05,243] jepsen worker 712 - jepsen.util 3712 :invoke :add 10396 INFO [2022-01-10 08:40:05,245] jepsen worker 676 - jepsen.util 3676 :invoke :add 10397 INFO [2022-01-10 08:40:05,249] jepsen worker 855 - jepsen.util 2855 :invoke :add 10398 INFO [2022-01-10 08:40:05,250] jepsen worker 419 - jepsen.util 2419 :info :add 9247 :RequestTimeoutException INFO [2022-01-10 08:40:05,252] jepsen worker 593 - jepsen.util 2593 :invoke :add 10399 INFO [2022-01-10 08:40:05,252] jepsen worker 845 - jepsen.util 2845 :info :add 9248 :RequestTimeoutException INFO [2022-01-10 08:40:05,257] jepsen worker 682 - jepsen.util 3682 :invoke :add 10400 INFO [2022-01-10 08:40:05,259] jepsen worker 143 - jepsen.util 3143 :invoke :add 10401 INFO [2022-01-10 08:40:05,261] jepsen worker 790 - jepsen.util 2790 :info :add 9249 :RequestTimeoutException INFO [2022-01-10 08:40:05,262] jepsen worker 774 - jepsen.util 2774 :invoke :add 10402 INFO [2022-01-10 08:40:05,262] jepsen worker 113 - jepsen.util 2113 :info :add 9251 :RequestTimeoutException INFO [2022-01-10 08:40:05,267] jepsen worker 125 - jepsen.util 2125 :info :add 9252 :RequestTimeoutException INFO [2022-01-10 08:40:05,267] jepsen worker 677 - jepsen.util 3677 :invoke :add 10403 INFO [2022-01-10 08:40:05,270] jepsen worker 964 - jepsen.util 1964 :info :add 9253 :RequestTimeoutException INFO [2022-01-10 08:40:05,272] jepsen worker 845 - jepsen.util 3845 :invoke :add 10404 INFO [2022-01-10 08:40:05,276] jepsen worker 790 - jepsen.util 3790 :invoke :add 10405 INFO [2022-01-10 08:40:05,278] jepsen worker 964 - jepsen.util 2964 :invoke :add 10406 INFO [2022-01-10 08:40:05,280] jepsen worker 74 - jepsen.util 3074 :invoke :add 10407 INFO [2022-01-10 08:40:05,283] jepsen worker 419 - jepsen.util 3419 :invoke :add 10408 INFO [2022-01-10 08:40:05,285] jepsen worker 125 - jepsen.util 3125 :invoke :add 10409 INFO [2022-01-10 08:40:05,288] jepsen worker 41 - jepsen.util 2041 :invoke :add 10410 INFO [2022-01-10 08:40:05,290] jepsen worker 274 - jepsen.util 1274 :info :add 9259 :RequestTimeoutException INFO [2022-01-10 08:40:05,291] jepsen worker 960 - jepsen.util 2960 :invoke :add 10411 INFO [2022-01-10 08:40:05,295] jepsen worker 113 - jepsen.util 3113 :invoke :add 10412 INFO [2022-01-10 08:40:05,296] jepsen worker 340 - jepsen.util 1340 :info :add 9258 :RequestTimeoutException INFO [2022-01-10 08:40:05,297] jepsen worker 274 - jepsen.util 2274 :invoke :add 10413 INFO [2022-01-10 08:40:05,301] jepsen worker 340 - jepsen.util 2340 :invoke :add 10414 INFO [2022-01-10 08:40:05,307] jepsen worker 965 - jepsen.util 1965 :info :add 9264 :RequestTimeoutException INFO [2022-01-10 08:40:05,310] jepsen worker 510 - jepsen.util 2510 :info :add 9265 :RequestTimeoutException INFO [2022-01-10 08:40:05,312] jepsen worker 965 - jepsen.util 2965 :invoke :add 10415 INFO [2022-01-10 08:40:05,312] jepsen worker 260 - jepsen.util 2260 :info :add 9266 :RequestTimeoutException INFO [2022-01-10 08:40:05,315] jepsen worker 510 - jepsen.util 3510 :invoke :add 10416 INFO [2022-01-10 08:40:05,318] jepsen worker 260 - jepsen.util 3260 :invoke :add 10417 INFO [2022-01-10 08:40:05,321] jepsen worker 31 - jepsen.util 2031 :info :add 9267 :RequestTimeoutException INFO [2022-01-10 08:40:05,323] jepsen worker 346 - jepsen.util 1346 :info :add 9269 :RequestTimeoutException INFO [2022-01-10 08:40:05,323] jepsen worker 169 - jepsen.util 1169 :info :add 9270 :RequestTimeoutException INFO [2022-01-10 08:40:05,323] jepsen worker 384 - jepsen.util 2384 :info :add 9268 :RequestTimeoutException INFO [2022-01-10 08:40:05,324] jepsen worker 31 - jepsen.util 3031 :invoke :add 10418 INFO [2022-01-10 08:40:05,331] jepsen worker 384 - jepsen.util 3384 :invoke :add 10419 INFO [2022-01-10 08:40:05,331] jepsen worker 283 - jepsen.util 2283 :info :add 9271 :RequestTimeoutException INFO [2022-01-10 08:40:05,331] jepsen worker 807 - jepsen.util 1807 :info :add 9273 :RequestTimeoutException INFO [2022-01-10 08:40:05,332] jepsen worker 251 - jepsen.util 2251 :info :add 9272 :RequestTimeoutException INFO [2022-01-10 08:40:05,334] jepsen worker 346 - jepsen.util 2346 :invoke :add 10420 INFO [2022-01-10 08:40:05,336] jepsen worker 40 - jepsen.util 2040 :info :add 9274 :RequestTimeoutException INFO [2022-01-10 08:40:05,340] jepsen worker 5 - jepsen.util 1005 :info :add 9275 :RequestTimeoutException INFO [2022-01-10 08:40:05,341] jepsen worker 251 - jepsen.util 3251 :invoke :add 10421 INFO [2022-01-10 08:40:05,341] jepsen worker 638 - jepsen.util 2638 :info :add 9277 :RequestTimeoutException INFO [2022-01-10 08:40:05,342] jepsen worker 367 - jepsen.util 1367 :info :add 9276 :RequestTimeoutException INFO [2022-01-10 08:40:05,348] jepsen worker 367 - jepsen.util 2367 :invoke :add 10422 INFO [2022-01-10 08:40:05,350] jepsen worker 169 - jepsen.util 2169 :invoke :add 10423 INFO [2022-01-10 08:40:05,353] jepsen worker 9 - jepsen.util 2009 :info :add 9284 :RequestTimeoutException INFO [2022-01-10 08:40:05,354] jepsen worker 283 - jepsen.util 3283 :invoke :add 10424 INFO [2022-01-10 08:40:05,358] jepsen worker 40 - jepsen.util 3040 :invoke :add 10425 INFO [2022-01-10 08:40:05,360] jepsen worker 9 - jepsen.util 3009 :invoke :add 10426 INFO [2022-01-10 08:40:05,363] jepsen worker 715 - jepsen.util 1715 :info :add 9285 :RequestTimeoutException INFO [2022-01-10 08:40:05,363] jepsen worker 543 - jepsen.util 2543 :info :add 9286 :RequestTimeoutException INFO [2022-01-10 08:40:05,364] jepsen worker 5 - jepsen.util 2005 :invoke :add 10427 INFO [2022-01-10 08:40:05,368] jepsen worker 715 - jepsen.util 2715 :invoke :add 10428 INFO [2022-01-10 08:40:05,370] jepsen worker 37 - jepsen.util 1037 :info :add 9288 :RequestTimeoutException INFO [2022-01-10 08:40:05,372] jepsen worker 264 - jepsen.util 2264 :info :add 9287 :RequestTimeoutException INFO [2022-01-10 08:40:05,372] jepsen worker 543 - jepsen.util 3543 :invoke :add 10429 INFO [2022-01-10 08:40:05,376] jepsen worker 57 - jepsen.util 2057 :info :add 9289 :RequestTimeoutException INFO [2022-01-10 08:40:05,377] jepsen worker 807 - jepsen.util 2807 :invoke :add 10430 INFO [2022-01-10 08:40:05,382] jepsen worker 37 - jepsen.util 2037 :invoke :add 10431 INFO [2022-01-10 08:40:05,384] jepsen worker 438 - jepsen.util 2438 :info :add 9290 :RequestTimeoutException INFO [2022-01-10 08:40:05,384] jepsen worker 264 - jepsen.util 3264 :invoke :add 10432 INFO [2022-01-10 08:40:05,388] jepsen worker 57 - jepsen.util 3057 :invoke :add 10433 INFO [2022-01-10 08:40:05,391] jepsen worker 561 - jepsen.util 1561 :info :add 9291 :RequestTimeoutException INFO [2022-01-10 08:40:05,391] jepsen worker 638 - jepsen.util 3638 :invoke :add 10434 INFO [2022-01-10 08:40:05,393] jepsen worker 72 - jepsen.util 2072 :info :add 9292 :RequestTimeoutException INFO [2022-01-10 08:40:05,394] jepsen worker 561 - jepsen.util 2561 :invoke :add 10435 INFO [2022-01-10 08:40:05,398] jepsen worker 72 - jepsen.util 3072 :invoke :add 10436 INFO [2022-01-10 08:40:05,401] jepsen worker 438 - jepsen.util 3438 :invoke :add 10437 INFO [2022-01-10 08:40:05,403] jepsen worker 541 - jepsen.util 2541 :info :add 9294 :RequestTimeoutException INFO [2022-01-10 08:40:05,403] jepsen worker 100 - jepsen.util 2100 :info :add 9293 :RequestTimeoutException INFO [2022-01-10 08:40:05,407] jepsen worker 82 - jepsen.util 2082 :info :add 9295 :RequestTimeoutException INFO [2022-01-10 08:40:05,408] jepsen worker 100 - jepsen.util 3100 :invoke :add 10438 INFO [2022-01-10 08:40:05,410] jepsen worker 141 - jepsen.util 2141 :info :add 9296 :RequestTimeoutException INFO [2022-01-10 08:40:05,412] jepsen worker 541 - jepsen.util 3541 :invoke :add 10439 INFO [2022-01-10 08:40:05,413] jepsen worker 611 - jepsen.util 2611 :info :add 9297 :RequestTimeoutException INFO [2022-01-10 08:40:05,417] jepsen worker 141 - jepsen.util 3141 :invoke :add 10440 INFO [2022-01-10 08:40:05,420] jepsen worker 611 - jepsen.util 3611 :invoke :add 10441 INFO [2022-01-10 08:40:05,423] jepsen worker 82 - jepsen.util 3082 :invoke :add 10442 INFO [2022-01-10 08:40:05,444] jepsen worker 107 - jepsen.util 2107 :info :add 9300 :RequestTimeoutException INFO [2022-01-10 08:40:05,447] jepsen worker 812 - jepsen.util 2812 :info :add 9302 :RequestTimeoutException INFO [2022-01-10 08:40:05,449] jepsen worker 107 - jepsen.util 3107 :invoke :add 10443 INFO [2022-01-10 08:40:05,450] jepsen worker 157 - jepsen.util 1157 :info :add 9303 :RequestTimeoutException INFO [2022-01-10 08:40:05,451] jepsen worker 972 - jepsen.util 2972 :info :add 9301 :RequestTimeoutException INFO [2022-01-10 08:40:05,455] jepsen worker 972 - jepsen.util 3972 :invoke :add 10444 INFO [2022-01-10 08:40:05,457] jepsen worker 459 - jepsen.util 1459 :info :add 9304 :RequestTimeoutException INFO [2022-01-10 08:40:05,458] jepsen worker 812 - jepsen.util 3812 :invoke :add 10445 INFO [2022-01-10 08:40:05,461] jepsen worker 459 - jepsen.util 2459 :invoke :add 10446 INFO [2022-01-10 08:40:05,462] jepsen worker 565 - jepsen.util 1565 :info :add 9305 :RequestTimeoutException INFO [2022-01-10 08:40:05,464] jepsen worker 157 - jepsen.util 2157 :invoke :add 10447 INFO [2022-01-10 08:40:05,468] jepsen worker 565 - jepsen.util 2565 :invoke :add 10448 INFO [2022-01-10 08:40:05,471] jepsen worker 88 - jepsen.util 1088 :info :add 9306 :RequestTimeoutException INFO [2022-01-10 08:40:05,473] jepsen worker 211 - jepsen.util 1211 :info :add 9307 :RequestTimeoutException INFO [2022-01-10 08:40:05,475] jepsen worker 88 - jepsen.util 2088 :invoke :add 10449 INFO [2022-01-10 08:40:05,479] jepsen worker 211 - jepsen.util 2211 :invoke :add 10450 INFO [2022-01-10 08:40:05,503] jepsen worker 669 - jepsen.util 1669 :info :add 9315 :RequestTimeoutException INFO [2022-01-10 08:40:05,503] jepsen worker 554 - jepsen.util 2554 :info :add 9314 :RequestTimeoutException INFO [2022-01-10 08:40:05,507] jepsen worker 97 - jepsen.util 2097 :info :add 9316 :RequestTimeoutException INFO [2022-01-10 08:40:05,507] jepsen worker 343 - jepsen.util 2343 :info :add 9318 :RequestTimeoutException INFO [2022-01-10 08:40:05,509] jepsen worker 669 - jepsen.util 2669 :invoke :add 10451 INFO [2022-01-10 08:40:05,510] jepsen worker 298 - jepsen.util 2298 :info :add 9319 :RequestTimeoutException INFO [2022-01-10 08:40:05,512] jepsen worker 228 - jepsen.util 2228 :info :add 9317 :RequestTimeoutException INFO [2022-01-10 08:40:05,513] jepsen worker 549 - jepsen.util 1549 :info :add 9320 :RequestTimeoutException INFO [2022-01-10 08:40:05,518] jepsen worker 554 - jepsen.util 3554 :invoke :add 10452 INFO [2022-01-10 08:40:05,521] jepsen worker 658 - jepsen.util 1658 :info :add 9321 :RequestTimeoutException INFO [2022-01-10 08:40:05,521] jepsen worker 874 - jepsen.util 2874 :info :add 9323 :RequestTimeoutException INFO [2022-01-10 08:40:05,521] jepsen worker 228 - jepsen.util 3228 :invoke :add 10453 INFO [2022-01-10 08:40:05,524] jepsen worker 671 - jepsen.util 2671 :info :add 9322 :RequestTimeoutException INFO [2022-01-10 08:40:05,528] jepsen worker 658 - jepsen.util 2658 :invoke :add 10454 INFO [2022-01-10 08:40:05,531] jepsen worker 298 - jepsen.util 3298 :invoke :add 10455 INFO [2022-01-10 08:40:05,534] jepsen worker 671 - jepsen.util 3671 :invoke :add 10456 INFO [2022-01-10 08:40:05,537] jepsen worker 874 - jepsen.util 3874 :invoke :add 10457 INFO [2022-01-10 08:40:05,539] jepsen worker 343 - jepsen.util 3343 :invoke :add 10458 INFO [2022-01-10 08:40:05,543] jepsen worker 549 - jepsen.util 2549 :invoke :add 10459 INFO [2022-01-10 08:40:05,545] jepsen worker 97 - jepsen.util 3097 :invoke :add 10460 INFO [2022-01-10 08:40:05,551] jepsen worker 956 - jepsen.util 2956 :info :add 9329 :RequestTimeoutException INFO [2022-01-10 08:40:05,553] jepsen worker 856 - jepsen.util 2856 :info :add 9328 :RequestTimeoutException INFO [2022-01-10 08:40:05,555] jepsen worker 956 - jepsen.util 3956 :invoke :add 10461 INFO [2022-01-10 08:40:05,559] jepsen worker 856 - jepsen.util 3856 :invoke :add 10462 INFO [2022-01-10 08:40:05,562] jepsen worker 275 - jepsen.util 1275 :info :add 9331 :RequestTimeoutException INFO [2022-01-10 08:40:05,563] jepsen worker 487 - jepsen.util 1487 :info :add 9330 :RequestTimeoutException INFO [2022-01-10 08:40:05,565] jepsen worker 275 - jepsen.util 2275 :invoke :add 10463 INFO [2022-01-10 08:40:05,566] jepsen worker 213 - jepsen.util 1213 :info :add 9332 :RequestTimeoutException INFO [2022-01-10 08:40:05,570] jepsen worker 487 - jepsen.util 2487 :invoke :add 10464 INFO [2022-01-10 08:40:05,571] jepsen worker 769 - jepsen.util 2769 :info :add 9333 :RequestTimeoutException INFO [2022-01-10 08:40:05,574] jepsen worker 213 - jepsen.util 2213 :invoke :add 10465 INFO [2022-01-10 08:40:05,577] jepsen worker 769 - jepsen.util 3769 :invoke :add 10466 INFO [2022-01-10 08:40:05,578] jepsen worker 327 - jepsen.util 2327 :ok :add 9870 INFO [2022-01-10 08:40:05,578] jepsen worker 50 - jepsen.util 3050 :ok :add 10086 INFO [2022-01-10 08:40:05,578] jepsen worker 537 - jepsen.util 2537 :ok :add 9868 INFO [2022-01-10 08:40:05,579] jepsen worker 412 - jepsen.util 3412 :ok :add 9755 INFO [2022-01-10 08:40:05,579] jepsen worker 155 - jepsen.util 2155 :ok :add 9761 INFO [2022-01-10 08:40:05,579] jepsen worker 34 - jepsen.util 2034 :ok :add 9779 INFO [2022-01-10 08:40:05,581] jepsen worker 521 - jepsen.util 1521 :info :add 9335 :RequestTimeoutException INFO [2022-01-10 08:40:05,583] jepsen worker 875 - jepsen.util 2875 :info :add 9336 :RequestTimeoutException INFO [2022-01-10 08:40:05,587] jepsen worker 155 - jepsen.util 2155 :invoke :add 10467 INFO [2022-01-10 08:40:05,590] jepsen worker 521 - jepsen.util 2521 :invoke :add 10468 INFO [2022-01-10 08:40:05,590] jepsen worker 755 - jepsen.util 1755 :info :add 9338 :RequestTimeoutException INFO [2022-01-10 08:40:05,592] jepsen worker 643 - jepsen.util 2643 :info :add 9337 :RequestTimeoutException INFO [2022-01-10 08:40:05,593] jepsen worker 412 - jepsen.util 3412 :invoke :add 10469 INFO [2022-01-10 08:40:05,597] jepsen worker 375 - jepsen.util 2375 :info :add 9339 :RequestTimeoutException INFO [2022-01-10 08:40:05,599] jepsen worker 643 - jepsen.util 3643 :invoke :add 10470 INFO [2022-01-10 08:40:05,600] jepsen worker 753 - jepsen.util 2753 :info :add 9340 :RequestTimeoutException INFO [2022-01-10 08:40:05,605] jepsen worker 537 - jepsen.util 2537 :invoke :add 10471 INFO [2022-01-10 08:40:05,608] jepsen worker 34 - jepsen.util 2034 :invoke :add 10472 INFO [2022-01-10 08:40:05,610] jepsen worker 957 - jepsen.util 2957 :info :add 9341 :RequestTimeoutException INFO [2022-01-10 08:40:05,611] jepsen worker 50 - jepsen.util 3050 :invoke :add 10473 INFO [2022-01-10 08:40:05,613] jepsen worker 731 - jepsen.util 2731 :info :add 9342 :RequestTimeoutException INFO [2022-01-10 08:40:05,614] jepsen worker 327 - jepsen.util 2327 :invoke :add 10474 INFO [2022-01-10 08:40:05,616] jepsen worker 463 - jepsen.util 2463 :info :add 9343 :RequestTimeoutException INFO [2022-01-10 08:40:05,622] jepsen worker 875 - jepsen.util 3875 :invoke :add 10475 INFO [2022-01-10 08:40:05,623] jepsen worker 77 - jepsen.util 2077 :info :add 9345 :RequestTimeoutException INFO [2022-01-10 08:40:05,624] jepsen worker 466 - jepsen.util 2466 :info :add 9344 :RequestTimeoutException INFO [2022-01-10 08:40:05,625] jepsen worker 375 - jepsen.util 3375 :invoke :add 10476 INFO [2022-01-10 08:40:05,630] jepsen worker 753 - jepsen.util 3753 :invoke :add 10477 INFO [2022-01-10 08:40:05,632] jepsen worker 937 - jepsen.util 2937 :info :add 9346 :RequestTimeoutException INFO [2022-01-10 08:40:05,633] jepsen worker 77 - jepsen.util 3077 :invoke :add 10478 INFO [2022-01-10 08:40:05,637] jepsen worker 893 - jepsen.util 2893 :info :add 9347 :RequestTimeoutException INFO [2022-01-10 08:40:05,637] jepsen worker 731 - jepsen.util 3731 :invoke :add 10479 INFO [2022-01-10 08:40:05,640] jepsen worker 755 - jepsen.util 2755 :invoke :add 10480 INFO [2022-01-10 08:40:05,640] jepsen worker 637 - jepsen.util 2637 :info :add 9348 :RequestTimeoutException INFO [2022-01-10 08:40:05,643] jepsen worker 893 - jepsen.util 3893 :invoke :add 10481 INFO [2022-01-10 08:40:05,643] jepsen worker 574 - jepsen.util 2574 :info :add 9349 :RequestTimeoutException INFO [2022-01-10 08:40:05,647] jepsen worker 637 - jepsen.util 3637 :invoke :add 10482 INFO [2022-01-10 08:40:05,650] jepsen worker 574 - jepsen.util 3574 :invoke :add 10483 INFO [2022-01-10 08:40:05,653] jepsen worker 957 - jepsen.util 3957 :invoke :add 10484 INFO [2022-01-10 08:40:05,655] jepsen worker 466 - jepsen.util 3466 :invoke :add 10485 INFO [2022-01-10 08:40:05,658] jepsen worker 937 - jepsen.util 3937 :invoke :add 10486 INFO [2022-01-10 08:40:05,661] jepsen worker 463 - jepsen.util 3463 :invoke :add 10487 INFO [2022-01-10 08:40:05,661] jepsen worker 943 - jepsen.util 2943 :info :add 9350 :RequestTimeoutException INFO [2022-01-10 08:40:05,662] jepsen worker 345 - jepsen.util 2345 :info :add 9352 :RequestTimeoutException INFO [2022-01-10 08:40:05,663] jepsen worker 898 - jepsen.util 2898 :info :add 9351 :RequestTimeoutException INFO [2022-01-10 08:40:05,667] jepsen worker 760 - jepsen.util 2760 :info :add 9353 :RequestTimeoutException INFO [2022-01-10 08:40:05,669] jepsen worker 345 - jepsen.util 3345 :invoke :add 10488 INFO [2022-01-10 08:40:05,673] jepsen worker 943 - jepsen.util 3943 :invoke :add 10489 INFO [2022-01-10 08:40:05,676] jepsen worker 760 - jepsen.util 3760 :invoke :add 10490 INFO [2022-01-10 08:40:05,678] jepsen worker 898 - jepsen.util 3898 :invoke :add 10491 INFO [2022-01-10 08:40:05,693] jepsen worker 98 - jepsen.util 1098 :info :add 9359 :RequestTimeoutException INFO [2022-01-10 08:40:05,693] jepsen worker 489 - jepsen.util 2489 :info :add 9358 :RequestTimeoutException INFO [2022-01-10 08:40:05,698] jepsen worker 489 - jepsen.util 3489 :invoke :add 10492 INFO [2022-01-10 08:40:05,701] jepsen worker 98 - jepsen.util 2098 :invoke :add 10493 INFO [2022-01-10 08:40:05,721] jepsen worker 285 - jepsen.util 2285 :info :add 9364 :RequestTimeoutException INFO [2022-01-10 08:40:05,724] jepsen worker 575 - jepsen.util 2575 :info :add 9365 :RequestTimeoutException INFO [2022-01-10 08:40:05,728] jepsen worker 285 - jepsen.util 3285 :invoke :add 10494 INFO [2022-01-10 08:40:05,730] jepsen worker 652 - jepsen.util 2652 :info :add 9368 :RequestTimeoutException INFO [2022-01-10 08:40:05,731] jepsen worker 575 - jepsen.util 3575 :invoke :add 10495 INFO [2022-01-10 08:40:05,732] jepsen worker 974 - jepsen.util 2974 :info :add 9366 :RequestTimeoutException INFO [2022-01-10 08:40:05,733] jepsen worker 294 - jepsen.util 2294 :info :add 9369 :RequestTimeoutException INFO [2022-01-10 08:40:05,735] jepsen worker 652 - jepsen.util 3652 :invoke :add 10496 INFO [2022-01-10 08:40:05,737] jepsen worker 557 - jepsen.util 2557 :info :add 9367 :RequestTimeoutException INFO [2022-01-10 08:40:05,739] jepsen worker 974 - jepsen.util 3974 :invoke :add 10497 INFO [2022-01-10 08:40:05,740] jepsen worker 7 - jepsen.util 1007 :info :add 9370 :RequestTimeoutException INFO [2022-01-10 08:40:05,742] jepsen worker 425 - jepsen.util 2425 :info :add 9372 :RequestTimeoutException INFO [2022-01-10 08:40:05,743] jepsen worker 294 - jepsen.util 3294 :invoke :add 10498 INFO [2022-01-10 08:40:05,743] jepsen worker 844 - jepsen.util 2844 :info :add 9371 :RequestTimeoutException INFO [2022-01-10 08:40:05,747] jepsen worker 83 - jepsen.util 2083 :info :add 9374 :RequestTimeoutException INFO [2022-01-10 08:40:05,748] jepsen worker 844 - jepsen.util 3844 :invoke :add 10499 INFO [2022-01-10 08:40:05,750] jepsen worker 413 - jepsen.util 1413 :info :add 9373 :RequestTimeoutException INFO [2022-01-10 08:40:05,751] jepsen worker 993 - jepsen.util 2993 :info :add 9375 :RequestTimeoutException INFO [2022-01-10 08:40:05,752] jepsen worker 7 - jepsen.util 2007 :invoke :add 10500 INFO [2022-01-10 08:40:05,753] jepsen worker 46 - jepsen.util 2046 :info :add 9376 :RequestTimeoutException INFO [2022-01-10 08:40:05,758] jepsen worker 46 - jepsen.util 3046 :invoke :add 10501 INFO [2022-01-10 08:40:05,761] jepsen worker 425 - jepsen.util 3425 :invoke :add 10502 INFO [2022-01-10 08:40:05,761] jepsen worker 527 - jepsen.util 2527 :info :add 9377 :RequestTimeoutException INFO [2022-01-10 08:40:05,763] jepsen worker 413 - jepsen.util 2413 :invoke :add 10503 INFO [2022-01-10 08:40:05,767] jepsen worker 83 - jepsen.util 3083 :invoke :add 10504 INFO [2022-01-10 08:40:05,769] jepsen worker 993 - jepsen.util 3993 :invoke :add 10505 INFO [2022-01-10 08:40:05,772] jepsen worker 557 - jepsen.util 3557 :invoke :add 10506 INFO [2022-01-10 08:40:05,774] jepsen worker 527 - jepsen.util 3527 :invoke :add 10507 INFO [2022-01-10 08:40:05,781] jepsen worker 247 - jepsen.util 1247 :info :add 9384 :RequestTimeoutException INFO [2022-01-10 08:40:05,783] jepsen worker 132 - jepsen.util 1132 :info :add 9385 :RequestTimeoutException INFO [2022-01-10 08:40:05,788] jepsen worker 247 - jepsen.util 2247 :invoke :add 10508 INFO [2022-01-10 08:40:05,790] jepsen worker 872 - jepsen.util 1872 :info :add 9387 :RequestTimeoutException INFO [2022-01-10 08:40:05,791] jepsen worker 132 - jepsen.util 2132 :invoke :add 10509 INFO [2022-01-10 08:40:05,792] jepsen worker 553 - jepsen.util 2553 :info :add 9386 :RequestTimeoutException INFO [2022-01-10 08:40:05,796] jepsen worker 553 - jepsen.util 3553 :invoke :add 10510 INFO [2022-01-10 08:40:05,797] jepsen worker 649 - jepsen.util 2649 :info :add 9388 :RequestTimeoutException INFO [2022-01-10 08:40:05,798] jepsen worker 872 - jepsen.util 2872 :invoke :add 10511 INFO [2022-01-10 08:40:05,800] jepsen worker 697 - jepsen.util 2697 :info :add 9389 :RequestTimeoutException INFO [2022-01-10 08:40:05,802] jepsen worker 649 - jepsen.util 3649 :invoke :add 10512 INFO [2022-01-10 08:40:05,802] jepsen worker 632 - jepsen.util 2632 :info :add 9390 :RequestTimeoutException INFO [2022-01-10 08:40:05,807] jepsen worker 697 - jepsen.util 3697 :invoke :add 10513 INFO [2022-01-10 08:40:05,810] jepsen worker 632 - jepsen.util 3632 :invoke :add 10514 INFO [2022-01-10 08:40:05,811] jepsen worker 771 - jepsen.util 2771 :info :add 9392 :RequestTimeoutException INFO [2022-01-10 08:40:05,811] jepsen worker 20 - jepsen.util 2020 :info :add 9393 :RequestTimeoutException INFO [2022-01-10 08:40:05,813] jepsen worker 601 - jepsen.util 2601 :info :add 9391 :RequestTimeoutException INFO [2022-01-10 08:40:05,813] jepsen worker 923 - jepsen.util 2923 :info :add 9394 :RequestTimeoutException INFO [2022-01-10 08:40:05,818] jepsen worker 20 - jepsen.util 3020 :invoke :add 10515 INFO [2022-01-10 08:40:05,821] jepsen worker 601 - jepsen.util 3601 :invoke :add 10516 INFO [2022-01-10 08:40:05,822] jepsen worker 684 - jepsen.util 2684 :info :add 9395 :RequestTimeoutException INFO [2022-01-10 08:40:05,824] jepsen worker 771 - jepsen.util 3771 :invoke :add 10517 INFO [2022-01-10 08:40:05,827] jepsen worker 326 - jepsen.util 2326 :info :add 9396 :RequestTimeoutException INFO [2022-01-10 08:40:05,828] jepsen worker 684 - jepsen.util 3684 :invoke :add 10518 INFO [2022-01-10 08:40:05,833] jepsen worker 468 - jepsen.util 2468 :info :add 9397 :RequestTimeoutException INFO [2022-01-10 08:40:05,833] jepsen worker 326 - jepsen.util 3326 :invoke :add 10519 INFO [2022-01-10 08:40:05,833] jepsen worker 229 - jepsen.util 2229 :info :add 9400 :RequestTimeoutException INFO [2022-01-10 08:40:05,837] jepsen worker 229 - jepsen.util 3229 :invoke :add 10520 INFO [2022-01-10 08:40:05,840] jepsen worker 923 - jepsen.util 3923 :invoke :add 10521 INFO [2022-01-10 08:40:05,841] jepsen worker 452 - jepsen.util 2452 :info :add 9401 :RequestTimeoutException INFO [2022-01-10 08:40:05,843] jepsen worker 468 - jepsen.util 3468 :invoke :add 10522 INFO [2022-01-10 08:40:05,847] jepsen worker 452 - jepsen.util 3452 :invoke :add 10523 INFO [2022-01-10 08:40:05,847] jepsen worker 544 - jepsen.util 2544 :info :add 9402 :RequestTimeoutException INFO [2022-01-10 08:40:05,847] jepsen worker 315 - jepsen.util 2315 :info :add 9404 :RequestTimeoutException INFO [2022-01-10 08:40:05,852] jepsen worker 983 - jepsen.util 2983 :info :add 9403 :RequestTimeoutException INFO [2022-01-10 08:40:05,855] jepsen worker 315 - jepsen.util 3315 :invoke :add 10524 INFO [2022-01-10 08:40:05,860] jepsen worker 983 - jepsen.util 3983 :invoke :add 10525 INFO [2022-01-10 08:40:05,860] jepsen worker 821 - jepsen.util 1821 :info :add 9405 :RequestTimeoutException INFO [2022-01-10 08:40:05,863] jepsen worker 901 - jepsen.util 1901 :info :add 9407 :RequestTimeoutException INFO [2022-01-10 08:40:05,863] jepsen worker 544 - jepsen.util 3544 :invoke :add 10526 INFO [2022-01-10 08:40:05,863] jepsen worker 45 - jepsen.util 2045 :info :add 9406 :RequestTimeoutException INFO [2022-01-10 08:40:05,869] jepsen worker 901 - jepsen.util 2901 :invoke :add 10527 INFO [2022-01-10 08:40:05,872] jepsen worker 45 - jepsen.util 3045 :invoke :add 10528 INFO [2022-01-10 08:40:05,874] jepsen worker 821 - jepsen.util 2821 :invoke :add 10529 INFO [2022-01-10 08:40:05,894] jepsen worker 738 - jepsen.util 2738 :info :add 9414 :RequestTimeoutException INFO [2022-01-10 08:40:05,898] jepsen worker 738 - jepsen.util 3738 :invoke :add 10530 INFO [2022-01-10 08:40:05,903] jepsen worker 809 - jepsen.util 1809 :info :add 9415 :RequestTimeoutException INFO [2022-01-10 08:40:05,903] jepsen worker 71 - jepsen.util 2071 :info :add 9416 :RequestTimeoutException INFO [2022-01-10 08:40:05,906] jepsen worker 606 - jepsen.util 1606 :info :add 9417 :RequestTimeoutException INFO [2022-01-10 08:40:05,907] jepsen worker 138 - jepsen.util 2138 :info :add 9418 :RequestTimeoutException INFO [2022-01-10 08:40:05,909] jepsen worker 809 - jepsen.util 2809 :invoke :add 10531 INFO [2022-01-10 08:40:05,910] jepsen worker 166 - jepsen.util 2166 :info :add 9419 :RequestTimeoutException INFO [2022-01-10 08:40:05,912] jepsen worker 55 - jepsen.util 1055 :info :add 9420 :RequestTimeoutException INFO [2022-01-10 08:40:05,916] jepsen worker 55 - jepsen.util 2055 :invoke :add 10532 INFO [2022-01-10 08:40:05,919] jepsen worker 138 - jepsen.util 3138 :invoke :add 10533 INFO [2022-01-10 08:40:05,922] jepsen worker 71 - jepsen.util 3071 :invoke :add 10534 INFO [2022-01-10 08:40:05,922] jepsen worker 144 - jepsen.util 2144 :info :add 9421 :RequestTimeoutException INFO [2022-01-10 08:40:05,923] jepsen worker 102 - jepsen.util 1102 :info :add 9422 :RequestTimeoutException INFO [2022-01-10 08:40:05,925] jepsen worker 606 - jepsen.util 2606 :invoke :add 10535 INFO [2022-01-10 08:40:05,931] jepsen worker 166 - jepsen.util 3166 :invoke :add 10536 INFO [2022-01-10 08:40:05,931] jepsen worker 242 - jepsen.util 2242 :info :add 9423 :RequestTimeoutException INFO [2022-01-10 08:40:05,937] jepsen worker 102 - jepsen.util 2102 :invoke :add 10537 INFO [2022-01-10 08:40:05,939] jepsen worker 242 - jepsen.util 3242 :invoke :add 10538 INFO [2022-01-10 08:40:05,940] jepsen worker 742 - jepsen.util 2742 :info :add 9428 :RequestTimeoutException INFO [2022-01-10 08:40:05,942] jepsen worker 144 - jepsen.util 3144 :invoke :add 10539 INFO [2022-01-10 08:40:05,946] jepsen worker 742 - jepsen.util 3742 :invoke :add 10540 INFO [2022-01-10 08:40:05,951] jepsen worker 171 - jepsen.util 2171 :info :add 9429 :RequestTimeoutException INFO [2022-01-10 08:40:05,953] jepsen worker 261 - jepsen.util 2261 :info :add 9431 :RequestTimeoutException INFO [2022-01-10 08:40:05,953] jepsen worker 462 - jepsen.util 462 :info :add 9430 :RequestTimeoutException INFO [2022-01-10 08:40:05,956] jepsen worker 361 - jepsen.util 2361 :info :add 9432 :RequestTimeoutException INFO [2022-01-10 08:40:05,959] jepsen worker 462 - jepsen.util 1462 :invoke :add 10541 INFO [2022-01-10 08:40:05,960] jepsen worker 417 - jepsen.util 2417 :info :add 9433 :RequestTimeoutException INFO [2022-01-10 08:40:05,962] jepsen worker 913 - jepsen.util 2913 :info :add 9435 :RequestTimeoutException INFO [2022-01-10 08:40:05,963] jepsen worker 446 - jepsen.util 2446 :info :add 9434 :RequestTimeoutException INFO [2022-01-10 08:40:05,964] jepsen worker 171 - jepsen.util 3171 :invoke :add 10542 INFO [2022-01-10 08:40:05,970] jepsen worker 429 - jepsen.util 2429 :info :add 9438 :RequestTimeoutException INFO [2022-01-10 08:40:05,971] jepsen worker 378 - jepsen.util 2378 :info :add 9436 :RequestTimeoutException INFO [2022-01-10 08:40:05,972] jepsen worker 261 - jepsen.util 3261 :invoke :add 10543 INFO [2022-01-10 08:40:05,973] jepsen worker 190 - jepsen.util 2190 :info :add 9437 :RequestTimeoutException INFO [2022-01-10 08:40:05,973] jepsen worker 357 - jepsen.util 1357 :info :add 9439 :RequestTimeoutException INFO [2022-01-10 08:40:05,974] jepsen worker 308 - jepsen.util 2308 :info :add 9440 :RequestTimeoutException INFO [2022-01-10 08:40:05,982] jepsen worker 190 - jepsen.util 3190 :invoke :add 10544 INFO [2022-01-10 08:40:05,983] jepsen worker 862 - jepsen.util 2862 :info :add 9441 :RequestTimeoutException INFO [2022-01-10 08:40:05,983] jepsen worker 546 - jepsen.util 2546 :info :add 9444 :RequestTimeoutException INFO [2022-01-10 08:40:05,984] jepsen worker 913 - jepsen.util 3913 :invoke :add 10545 INFO [2022-01-10 08:40:05,987] jepsen worker 449 - jepsen.util 2449 :info :add 9442 :RequestTimeoutException INFO [2022-01-10 08:40:05,990] jepsen worker 429 - jepsen.util 3429 :invoke :add 10546 INFO [2022-01-10 08:40:05,990] jepsen worker 581 - jepsen.util 2581 :info :add 9443 :RequestTimeoutException INFO [2022-01-10 08:40:05,991] jepsen worker 830 - jepsen.util 2830 :info :add 9445 :RequestTimeoutException INFO [2022-01-10 08:40:05,992] jepsen worker 891 - jepsen.util 1891 :info :add 9446 :RequestTimeoutException INFO [2022-01-10 08:40:05,997] jepsen worker 308 - jepsen.util 3308 :invoke :add 10547 INFO [2022-01-10 08:40:06,000] jepsen worker 378 - jepsen.util 3378 :invoke :add 10548 INFO [2022-01-10 08:40:06,003] jepsen worker 891 - jepsen.util 2891 :invoke :add 10549 INFO [2022-01-10 08:40:06,006] jepsen worker 446 - jepsen.util 3446 :invoke :add 10550 INFO [2022-01-10 08:40:06,008] jepsen worker 830 - jepsen.util 3830 :invoke :add 10551 INFO [2022-01-10 08:40:06,011] jepsen worker 361 - jepsen.util 3361 :invoke :add 10552 INFO [2022-01-10 08:40:06,014] jepsen worker 581 - jepsen.util 3581 :invoke :add 10553 INFO [2022-01-10 08:40:06,017] jepsen worker 449 - jepsen.util 3449 :invoke :add 10554 INFO [2022-01-10 08:40:06,020] jepsen worker 862 - jepsen.util 3862 :invoke :add 10555 INFO [2022-01-10 08:40:06,022] jepsen worker 417 - jepsen.util 3417 :invoke :add 10556 INFO [2022-01-10 08:40:06,025] jepsen worker 546 - jepsen.util 3546 :invoke :add 10557 INFO [2022-01-10 08:40:06,027] jepsen worker 357 - jepsen.util 2357 :invoke :add 10558 INFO [2022-01-10 08:40:06,052] jepsen worker 610 - jepsen.util 2610 :info :add 9447 :RequestTimeoutException INFO [2022-01-10 08:40:06,056] jepsen worker 610 - jepsen.util 3610 :invoke :add 10559 INFO [2022-01-10 08:40:06,061] jepsen worker 231 - jepsen.util 2231 :info :add 9448 :RequestTimeoutException INFO [2022-01-10 08:40:06,064] jepsen worker 585 - jepsen.util 2585 :info :add 9449 :RequestTimeoutException INFO [2022-01-10 08:40:06,066] jepsen worker 231 - jepsen.util 3231 :invoke :add 10560 INFO [2022-01-10 08:40:06,071] jepsen worker 585 - jepsen.util 3585 :invoke :add 10561 INFO [2022-01-10 08:40:06,072] jepsen worker 226 - jepsen.util 2226 :info :add 9450 :RequestTimeoutException INFO [2022-01-10 08:40:06,077] jepsen worker 226 - jepsen.util 3226 :invoke :add 10562 INFO [2022-01-10 08:40:06,087] jepsen worker 121 - jepsen.util 2121 :info :add 9451 :RequestTimeoutException INFO [2022-01-10 08:40:06,090] jepsen worker 945 - jepsen.util 2945 :info :add 9452 :RequestTimeoutException INFO [2022-01-10 08:40:06,092] jepsen worker 121 - jepsen.util 3121 :invoke :add 10563 INFO [2022-01-10 08:40:06,093] jepsen worker 826 - jepsen.util 2826 :info :add 9453 :RequestTimeoutException INFO [2022-01-10 08:40:06,095] jepsen worker 945 - jepsen.util 3945 :invoke :add 10564 INFO [2022-01-10 08:40:06,100] jepsen worker 826 - jepsen.util 3826 :invoke :add 10565 INFO [2022-01-10 08:40:06,111] jepsen worker 330 - jepsen.util 2330 :info :add 9458 :RequestTimeoutException INFO [2022-01-10 08:40:06,113] jepsen worker 430 - jepsen.util 2430 :info :add 9459 :RequestTimeoutException INFO [2022-01-10 08:40:06,118] jepsen worker 330 - jepsen.util 3330 :invoke :add 10566 INFO [2022-01-10 08:40:06,120] jepsen worker 430 - jepsen.util 3430 :invoke :add 10567 INFO [2022-01-10 08:40:06,261] jepsen worker 85 - jepsen.util 1085 :info :add 9465 :RequestTimeoutException INFO [2022-01-10 08:40:06,261] jepsen worker 843 - jepsen.util 2843 :info :add 9466 :RequestTimeoutException INFO [2022-01-10 08:40:06,263] jepsen worker 547 - jepsen.util 2547 :info :add 9464 :RequestTimeoutException INFO [2022-01-10 08:40:06,263] jepsen worker 938 - jepsen.util 1938 :info :add 9467 :RequestTimeoutException INFO [2022-01-10 08:40:06,272] jepsen worker 291 - jepsen.util 2291 :info :add 9468 :RequestTimeoutException INFO [2022-01-10 08:40:06,275] jepsen worker 843 - jepsen.util 3843 :invoke :add 10568 INFO [2022-01-10 08:40:06,277] jepsen worker 791 - jepsen.util 2791 :info :add 9469 :RequestTimeoutException INFO [2022-01-10 08:40:06,279] jepsen worker 291 - jepsen.util 3291 :invoke :add 10569 INFO [2022-01-10 08:40:06,282] jepsen worker 99 - jepsen.util 1099 :info :add 9470 :RequestTimeoutException INFO [2022-01-10 08:40:06,284] jepsen worker 85 - jepsen.util 2085 :invoke :add 10570 INFO [2022-01-10 08:40:06,288] jepsen worker 938 - jepsen.util 2938 :invoke :add 10571 INFO [2022-01-10 08:40:06,290] jepsen worker 22 - jepsen.util 2022 :info :add 9471 :RequestTimeoutException INFO [2022-01-10 08:40:06,291] jepsen worker 160 - jepsen.util 2160 :info :add 9473 :RequestTimeoutException INFO [2022-01-10 08:40:06,291] jepsen worker 791 - jepsen.util 3791 :invoke :add 10572 INFO [2022-01-10 08:40:06,293] jepsen worker 137 - jepsen.util 2137 :info :add 9472 :RequestTimeoutException INFO [2022-01-10 08:40:06,297] jepsen worker 232 - jepsen.util 1232 :info :add 9474 :RequestTimeoutException INFO [2022-01-10 08:40:06,297] jepsen worker 547 - jepsen.util 3547 :invoke :add 10573 INFO [2022-01-10 08:40:06,302] jepsen worker 22 - jepsen.util 3022 :invoke :add 10574 INFO [2022-01-10 08:40:06,305] jepsen worker 232 - jepsen.util 2232 :invoke :add 10575 INFO [2022-01-10 08:40:06,308] jepsen worker 160 - jepsen.util 3160 :invoke :add 10576 INFO [2022-01-10 08:40:06,310] jepsen worker 99 - jepsen.util 2099 :invoke :add 10577 INFO [2022-01-10 08:40:06,313] jepsen worker 137 - jepsen.util 3137 :invoke :add 10578 INFO [2022-01-10 08:40:06,399] jepsen worker 481 - jepsen.util 1481 :ok :add 9801 INFO [2022-01-10 08:40:06,399] jepsen worker 914 - jepsen.util 3914 :ok :add 9819 INFO [2022-01-10 08:40:06,400] jepsen worker 73 - jepsen.util 2073 :ok :add 9680 INFO [2022-01-10 08:40:06,400] jepsen worker 48 - jepsen.util 3048 :ok :add 9710 INFO [2022-01-10 08:40:06,400] jepsen worker 249 - jepsen.util 3249 :ok :add 9698 INFO [2022-01-10 08:40:06,400] jepsen worker 780 - jepsen.util 3780 :ok :add 10167 INFO [2022-01-10 08:40:06,400] jepsen worker 916 - jepsen.util 2916 :ok :add 9724 INFO [2022-01-10 08:40:06,400] jepsen worker 674 - jepsen.util 1674 :ok :add 9989 INFO [2022-01-10 08:40:06,401] jepsen worker 728 - jepsen.util 3728 :ok :add 9708 INFO [2022-01-10 08:40:06,408] jepsen worker 674 - jepsen.util 1674 :invoke :add 10579 INFO [2022-01-10 08:40:06,411] jepsen worker 48 - jepsen.util 3048 :invoke :add 10580 INFO [2022-01-10 08:40:06,414] jepsen worker 481 - jepsen.util 1481 :invoke :add 10581 INFO [2022-01-10 08:40:06,417] jepsen worker 914 - jepsen.util 3914 :invoke :add 10582 INFO [2022-01-10 08:40:06,420] jepsen worker 780 - jepsen.util 3780 :invoke :add 10583 INFO [2022-01-10 08:40:06,423] jepsen worker 728 - jepsen.util 3728 :invoke :add 10584 INFO [2022-01-10 08:40:06,426] jepsen worker 916 - jepsen.util 2916 :invoke :add 10585 INFO [2022-01-10 08:40:06,429] jepsen worker 249 - jepsen.util 3249 :invoke :add 10586 INFO [2022-01-10 08:40:06,432] jepsen worker 73 - jepsen.util 2073 :invoke :add 10587 INFO [2022-01-10 08:40:06,432] jepsen worker 184 - jepsen.util 2184 :info :add 9476 :RequestTimeoutException INFO [2022-01-10 08:40:06,434] jepsen worker 772 - jepsen.util 3772 :info :add 9475 :RequestTimeoutException INFO [2022-01-10 08:40:06,440] jepsen worker 772 - jepsen.util 4772 :invoke :add 10588 INFO [2022-01-10 08:40:06,443] jepsen worker 184 - jepsen.util 3184 :invoke :add 10589 INFO [2022-01-10 08:40:06,477] jepsen worker 337 - jepsen.util 2337 :info :add 9477 :RequestTimeoutException INFO [2022-01-10 08:40:06,481] jepsen worker 337 - jepsen.util 3337 :invoke :add 10590 INFO [2022-01-10 08:40:06,500] jepsen worker 768 - jepsen.util 2768 :info :add 9484 :RequestTimeoutException INFO [2022-01-10 08:40:06,505] jepsen worker 768 - jepsen.util 3768 :invoke :add 10591 INFO [2022-01-10 08:40:06,523] jepsen worker 739 - jepsen.util 2739 :info :add 9485 :RequestTimeoutException INFO [2022-01-10 08:40:06,527] jepsen worker 739 - jepsen.util 3739 :invoke :add 10592 INFO [2022-01-10 08:40:06,531] jepsen worker 732 - jepsen.util 2732 :info :add 9486 :RequestTimeoutException INFO [2022-01-10 08:40:06,537] jepsen worker 732 - jepsen.util 3732 :invoke :add 10593 INFO [2022-01-10 08:40:06,542] jepsen worker 380 - jepsen.util 2380 :info :add 9488 :RequestTimeoutException INFO [2022-01-10 08:40:06,543] jepsen worker 559 - jepsen.util 2559 :info :add 9487 :RequestTimeoutException INFO [2022-01-10 08:40:06,547] jepsen worker 279 - jepsen.util 2279 :info :add 9489 :RequestTimeoutException INFO [2022-01-10 08:40:06,548] jepsen worker 559 - jepsen.util 3559 :invoke :add 10594 INFO [2022-01-10 08:40:06,550] jepsen worker 897 - jepsen.util 2897 :info :add 9490 :RequestTimeoutException INFO [2022-01-10 08:40:06,552] jepsen worker 279 - jepsen.util 3279 :invoke :add 10595 INFO [2022-01-10 08:40:06,553] jepsen worker 548 - jepsen.util 2548 :info :add 9491 :RequestTimeoutException INFO [2022-01-10 08:40:06,556] jepsen worker 380 - jepsen.util 3380 :invoke :add 10596 INFO [2022-01-10 08:40:06,560] jepsen worker 897 - jepsen.util 3897 :invoke :add 10597 INFO [2022-01-10 08:40:06,561] jepsen worker 764 - jepsen.util 2764 :info :add 9492 :RequestTimeoutException INFO [2022-01-10 08:40:06,562] jepsen worker 548 - jepsen.util 3548 :invoke :add 10598 INFO [2022-01-10 08:40:06,564] jepsen worker 53 - jepsen.util 3053 :info :add 9493 :RequestTimeoutException INFO [2022-01-10 08:40:06,569] jepsen worker 764 - jepsen.util 3764 :invoke :add 10599 INFO [2022-01-10 08:40:06,571] jepsen worker 53 - jepsen.util 4053 :invoke :add 10600 INFO [2022-01-10 08:40:06,581] jepsen worker 976 - jepsen.util 1976 :info :add 9494 :RequestTimeoutException INFO [2022-01-10 08:40:06,581] jepsen worker 694 - jepsen.util 2694 :info :add 9495 :RequestTimeoutException INFO [2022-01-10 08:40:06,585] jepsen worker 976 - jepsen.util 2976 :invoke :add 10601 INFO [2022-01-10 08:40:06,587] jepsen worker 694 - jepsen.util 3694 :invoke :add 10602 INFO [2022-01-10 08:40:06,590] jepsen worker 686 - jepsen.util 1686 :info :add 9496 :RequestTimeoutException INFO [2022-01-10 08:40:06,594] jepsen worker 686 - jepsen.util 2686 :invoke :add 10603 INFO [2022-01-10 08:40:06,632] jepsen worker 761 - jepsen.util 2761 :info :add 9497 :RequestTimeoutException INFO [2022-01-10 08:40:06,637] jepsen worker 761 - jepsen.util 3761 :invoke :add 10604 INFO [2022-01-10 08:40:06,653] jepsen worker 370 - jepsen.util 3370 :info :add 9500 :RequestTimeoutException INFO [2022-01-10 08:40:06,657] jepsen worker 370 - jepsen.util 4370 :invoke :add 10605 INFO [2022-01-10 08:40:06,661] jepsen worker 349 - jepsen.util 2349 :info :add 9501 :RequestTimeoutException INFO [2022-01-10 08:40:06,662] jepsen worker 801 - jepsen.util 3801 :info :add 9503 :RequestTimeoutException INFO [2022-01-10 08:40:06,663] jepsen worker 931 - jepsen.util 3931 :info :add 9502 :RequestTimeoutException INFO [2022-01-10 08:40:06,667] jepsen worker 625 - jepsen.util 2625 :info :add 9504 :RequestTimeoutException INFO [2022-01-10 08:40:06,669] jepsen worker 931 - jepsen.util 4931 :invoke :add 10606 INFO [2022-01-10 08:40:06,670] jepsen worker 13 - jepsen.util 2013 :info :add 9505 :RequestTimeoutException INFO [2022-01-10 08:40:06,672] jepsen worker 801 - jepsen.util 4801 :invoke :add 10607 INFO [2022-01-10 08:40:06,676] jepsen worker 349 - jepsen.util 3349 :invoke :add 10608 INFO [2022-01-10 08:40:06,679] jepsen worker 13 - jepsen.util 3013 :invoke :add 10609 INFO [2022-01-10 08:40:06,681] jepsen worker 625 - jepsen.util 3625 :invoke :add 10610 INFO [2022-01-10 08:40:06,701] jepsen worker 95 - jepsen.util 3095 :info :add 9507 :RequestTimeoutException INFO [2022-01-10 08:40:06,703] jepsen worker 706 - jepsen.util 2706 :info :add 9506 :RequestTimeoutException INFO [2022-01-10 08:40:06,705] jepsen worker 95 - jepsen.util 4095 :invoke :add 10611 INFO [2022-01-10 08:40:06,709] jepsen worker 706 - jepsen.util 3706 :invoke :add 10612 INFO [2022-01-10 08:40:06,734] jepsen worker 173 - jepsen.util 2173 :ok :add 9670 INFO [2022-01-10 08:40:06,739] jepsen worker 173 - jepsen.util 2173 :invoke :add 10613 INFO [2022-01-10 08:40:06,743] jepsen worker 882 - jepsen.util 2882 :info :add 9515 :RequestTimeoutException INFO [2022-01-10 08:40:06,743] jepsen worker 436 - jepsen.util 3436 :info :add 9514 :RequestTimeoutException INFO [2022-01-10 08:40:06,746] jepsen worker 850 - jepsen.util 2850 :info :add 9516 :RequestTimeoutException INFO [2022-01-10 08:40:06,750] jepsen worker 882 - jepsen.util 3882 :invoke :add 10614 INFO [2022-01-10 08:40:06,750] jepsen worker 685 - jepsen.util 2685 :info :add 9517 :RequestTimeoutException INFO [2022-01-10 08:40:06,751] jepsen worker 11 - jepsen.util 2011 :info :add 9519 :RequestTimeoutException INFO [2022-01-10 08:40:06,752] jepsen worker 578 - jepsen.util 2578 :info :add 9521 :RequestTimeoutException INFO [2022-01-10 08:40:06,753] jepsen worker 810 - jepsen.util 2810 :info :add 9518 :RequestTimeoutException INFO [2022-01-10 08:40:06,753] jepsen worker 131 - jepsen.util 2131 :info :add 9520 :RequestTimeoutException INFO [2022-01-10 08:40:06,753] jepsen worker 436 - jepsen.util 4436 :invoke :add 10615 INFO [2022-01-10 08:40:06,761] jepsen worker 810 - jepsen.util 3810 :invoke :add 10616 INFO [2022-01-10 08:40:06,764] jepsen worker 850 - jepsen.util 3850 :invoke :add 10617 INFO [2022-01-10 08:40:06,767] jepsen worker 11 - jepsen.util 3011 :invoke :add 10618 INFO [2022-01-10 08:40:06,769] jepsen worker 131 - jepsen.util 3131 :invoke :add 10619 INFO [2022-01-10 08:40:06,773] jepsen worker 578 - jepsen.util 3578 :invoke :add 10620 INFO [2022-01-10 08:40:06,775] jepsen worker 685 - jepsen.util 3685 :invoke :add 10621 INFO [2022-01-10 08:40:06,817] jepsen worker 989 - jepsen.util 2989 :info :add 9522 :RequestTimeoutException INFO [2022-01-10 08:40:06,820] jepsen worker 442 - jepsen.util 2442 :info :add 9523 :RequestTimeoutException INFO [2022-01-10 08:40:06,821] jepsen worker 989 - jepsen.util 3989 :invoke :add 10622 INFO [2022-01-10 08:40:06,824] jepsen worker 442 - jepsen.util 3442 :invoke :add 10623 INFO [2022-01-10 08:40:06,847] jepsen worker 679 - jepsen.util 2679 :info :add 9528 :RequestTimeoutException INFO [2022-01-10 08:40:06,850] jepsen worker 406 - jepsen.util 3406 :info :add 9529 :RequestTimeoutException INFO [2022-01-10 08:40:06,852] jepsen worker 679 - jepsen.util 3679 :invoke :add 10624 INFO [2022-01-10 08:40:06,853] jepsen worker 24 - jepsen.util 1024 :info :add 9530 :RequestTimeoutException INFO [2022-01-10 08:40:06,856] jepsen worker 406 - jepsen.util 4406 :invoke :add 10625 INFO [2022-01-10 08:40:06,859] jepsen worker 24 - jepsen.util 2024 :invoke :add 10626 INFO [2022-01-10 08:40:06,861] jepsen worker 150 - jepsen.util 2150 :info :add 9531 :RequestTimeoutException INFO [2022-01-10 08:40:06,862] jepsen worker 62 - jepsen.util 1062 :info :add 9533 :RequestTimeoutException INFO [2022-01-10 08:40:06,864] jepsen worker 496 - jepsen.util 2496 :info :add 9532 :RequestTimeoutException INFO [2022-01-10 08:40:06,866] jepsen worker 150 - jepsen.util 3150 :invoke :add 10627 INFO [2022-01-10 08:40:06,867] jepsen worker 114 - jepsen.util 2114 :info :add 9534 :RequestTimeoutException INFO [2022-01-10 08:40:06,870] jepsen worker 270 - jepsen.util 2270 :info :add 9535 :RequestTimeoutException INFO [2022-01-10 08:40:06,870] jepsen worker 62 - jepsen.util 2062 :invoke :add 10628 INFO [2022-01-10 08:40:06,874] jepsen worker 496 - jepsen.util 3496 :invoke :add 10629 INFO [2022-01-10 08:40:06,877] jepsen worker 270 - jepsen.util 3270 :invoke :add 10630 INFO [2022-01-10 08:40:06,880] jepsen worker 114 - jepsen.util 3114 :invoke :add 10631 INFO [2022-01-10 08:40:06,903] jepsen worker 641 - jepsen.util 3641 :info :add 9536 :RequestTimeoutException INFO [2022-01-10 08:40:06,906] jepsen worker 641 - jepsen.util 4641 :invoke :add 10632 INFO [2022-01-10 08:40:06,911] jepsen worker 246 - jepsen.util 2246 :info :add 9537 :RequestTimeoutException INFO [2022-01-10 08:40:06,911] jepsen worker 881 - jepsen.util 2881 :info :add 9539 :RequestTimeoutException INFO [2022-01-10 08:40:06,914] jepsen worker 443 - jepsen.util 2443 :info :add 9538 :RequestTimeoutException INFO [2022-01-10 08:40:06,917] jepsen worker 443 - jepsen.util 3443 :invoke :add 10633 INFO [2022-01-10 08:40:06,920] jepsen worker 881 - jepsen.util 3881 :invoke :add 10634 INFO [2022-01-10 08:40:06,923] jepsen worker 246 - jepsen.util 3246 :invoke :add 10635 INFO [2022-01-10 08:40:06,927] jepsen worker 205 - jepsen.util 3205 :info :add 9540 :RequestTimeoutException INFO [2022-01-10 08:40:06,930] jepsen worker 94 - jepsen.util 2094 :info :add 9541 :RequestTimeoutException INFO [2022-01-10 08:40:06,932] jepsen worker 205 - jepsen.util 4205 :invoke :add 10636 INFO [2022-01-10 08:40:06,933] jepsen worker 332 - jepsen.util 2332 :info :add 9542 :RequestTimeoutException INFO [2022-01-10 08:40:06,934] jepsen worker 831 - jepsen.util 2831 :ok :add 9756 INFO [2022-01-10 08:40:06,934] jepsen worker 90 - jepsen.util 2090 :ok :add 9762 INFO [2022-01-10 08:40:06,939] jepsen worker 332 - jepsen.util 3332 :invoke :add 10637 INFO [2022-01-10 08:40:06,941] jepsen worker 905 - jepsen.util 2905 :info :add 9543 :RequestTimeoutException INFO [2022-01-10 08:40:06,942] jepsen worker 94 - jepsen.util 3094 :invoke :add 10638 INFO [2022-01-10 08:40:06,942] jepsen worker 126 - jepsen.util 2126 :info :add 9545 :RequestTimeoutException INFO [2022-01-10 08:40:06,943] jepsen worker 877 - jepsen.util 2877 :info :add 9544 :RequestTimeoutException INFO [2022-01-10 08:40:06,949] jepsen worker 831 - jepsen.util 2831 :invoke :add 10639 INFO [2022-01-10 08:40:06,951] jepsen worker 126 - jepsen.util 3126 :invoke :add 10640 INFO [2022-01-10 08:40:06,954] jepsen worker 905 - jepsen.util 3905 :invoke :add 10641 INFO [2022-01-10 08:40:06,956] jepsen worker 90 - jepsen.util 2090 :invoke :add 10642 INFO [2022-01-10 08:40:06,959] jepsen worker 877 - jepsen.util 3877 :invoke :add 10643 INFO [2022-01-10 08:40:06,977] jepsen worker 656 - jepsen.util 3656 :info :add 9546 :RequestTimeoutException INFO [2022-01-10 08:40:06,981] jepsen worker 656 - jepsen.util 4656 :invoke :add 10644 INFO [2022-01-10 08:40:07,041] jepsen worker 735 - jepsen.util 3735 :info :add 9547 :RequestTimeoutException INFO [2022-01-10 08:40:07,045] jepsen worker 735 - jepsen.util 4735 :invoke :add 10645 INFO [2022-01-10 08:40:07,052] jepsen worker 385 - jepsen.util 3385 :info :add 9548 :RequestTimeoutException INFO [2022-01-10 08:40:07,057] jepsen worker 385 - jepsen.util 4385 :invoke :add 10646 INFO [2022-01-10 08:40:07,083] jepsen worker 603 - jepsen.util 2603 :info :add 9549 :RequestTimeoutException INFO [2022-01-10 08:40:07,088] jepsen worker 603 - jepsen.util 3603 :invoke :add 10647 INFO [2022-01-10 08:40:07,182] jepsen worker 309 - jepsen.util 3309 :info :add 9550 :RequestTimeoutException INFO [2022-01-10 08:40:07,187] jepsen worker 309 - jepsen.util 4309 :invoke :add 10648 INFO [2022-01-10 08:40:07,214] jepsen worker 951 - jepsen.util 3951 :info :add 9551 :RequestTimeoutException INFO [2022-01-10 08:40:07,218] jepsen worker 951 - jepsen.util 4951 :invoke :add 10649 INFO [2022-01-10 08:40:07,232] jepsen worker 793 - jepsen.util 3793 :info :add 9552 :RequestTimeoutException INFO [2022-01-10 08:40:07,237] jepsen worker 741 - jepsen.util 2741 :info :add 9553 :RequestTimeoutException INFO [2022-01-10 08:40:07,238] jepsen worker 793 - jepsen.util 4793 :invoke :add 10650 INFO [2022-01-10 08:40:07,242] jepsen worker 741 - jepsen.util 3741 :invoke :add 10651 INFO [2022-01-10 08:40:07,262] jepsen worker 220 - jepsen.util 2220 :info :add 9558 :RequestTimeoutException INFO [2022-01-10 08:40:07,266] jepsen worker 220 - jepsen.util 3220 :invoke :add 10652 INFO [2022-01-10 08:40:07,277] jepsen worker 785 - jepsen.util 3785 :info :add 9559 :RequestTimeoutException INFO [2022-01-10 08:40:07,282] jepsen worker 785 - jepsen.util 4785 :invoke :add 10653 INFO [2022-01-10 08:40:07,292] jepsen worker 567 - jepsen.util 2567 :info :add 9564 :RequestTimeoutException INFO [2022-01-10 08:40:07,295] jepsen worker 567 - jepsen.util 3567 :invoke :add 10654 INFO [2022-01-10 08:40:07,296] jepsen worker 16 - jepsen.util 3016 :info :add 9565 :RequestTimeoutException INFO [2022-01-10 08:40:07,300] jepsen worker 505 - jepsen.util 3505 :info :add 9566 :RequestTimeoutException INFO [2022-01-10 08:40:07,301] jepsen worker 478 - jepsen.util 3478 :info :add 9568 :RequestTimeoutException INFO [2022-01-10 08:40:07,302] jepsen worker 16 - jepsen.util 4016 :invoke :add 10655 INFO [2022-01-10 08:40:07,303] jepsen worker 369 - jepsen.util 2369 :info :add 9567 :RequestTimeoutException INFO [2022-01-10 08:40:07,304] jepsen worker 267 - jepsen.util 3267 :info :add 9569 :RequestTimeoutException INFO [2022-01-10 08:40:07,307] jepsen worker 271 - jepsen.util 3271 :info :add 9571 :RequestTimeoutException INFO [2022-01-10 08:40:07,308] jepsen worker 478 - jepsen.util 4478 :invoke :add 10656 INFO [2022-01-10 08:40:07,312] jepsen worker 545 - jepsen.util 3545 :info :add 9570 :RequestTimeoutException INFO [2022-01-10 08:40:07,313] jepsen worker 505 - jepsen.util 4505 :invoke :add 10657 INFO [2022-01-10 08:40:07,319] jepsen worker 271 - jepsen.util 4271 :invoke :add 10658 INFO [2022-01-10 08:40:07,321] jepsen worker 545 - jepsen.util 4545 :invoke :add 10659 INFO [2022-01-10 08:40:07,324] jepsen worker 369 - jepsen.util 3369 :invoke :add 10660 INFO [2022-01-10 08:40:07,326] jepsen worker 267 - jepsen.util 4267 :invoke :add 10661 INFO [2022-01-10 08:40:07,380] jepsen worker 666 - jepsen.util 2666 :info :add 9572 :RequestTimeoutException INFO [2022-01-10 08:40:07,383] jepsen worker 28 - jepsen.util 2028 :info :add 9573 :RequestTimeoutException INFO [2022-01-10 08:40:07,387] jepsen worker 28 - jepsen.util 3028 :invoke :add 10662 INFO [2022-01-10 08:40:07,390] jepsen worker 666 - jepsen.util 3666 :invoke :add 10663 INFO [2022-01-10 08:40:07,401] jepsen worker 746 - jepsen.util 2746 :info :add 9574 :RequestTimeoutException INFO [2022-01-10 08:40:07,402] jepsen worker 116 - jepsen.util 2116 :info :add 9576 :RequestTimeoutException INFO [2022-01-10 08:40:07,403] jepsen worker 626 - jepsen.util 2626 :info :add 9575 :RequestTimeoutException INFO [2022-01-10 08:40:07,408] jepsen worker 116 - jepsen.util 3116 :invoke :add 10664 INFO [2022-01-10 08:40:07,410] jepsen worker 512 - jepsen.util 2512 :ok :add 9846 INFO [2022-01-10 08:40:07,410] jepsen worker 961 - jepsen.util 2961 :ok :add 9872 INFO [2022-01-10 08:40:07,410] jepsen worker 482 - jepsen.util 2482 :ok :add 9763 INFO [2022-01-10 08:40:07,410] jepsen worker 680 - jepsen.util 2680 :ok :add 9757 INFO [2022-01-10 08:40:07,411] jepsen worker 623 - jepsen.util 2623 :ok :add 10084 INFO [2022-01-10 08:40:07,412] jepsen worker 746 - jepsen.util 3746 :invoke :add 10665 INFO [2022-01-10 08:40:07,416] jepsen worker 626 - jepsen.util 3626 :invoke :add 10666 INFO [2022-01-10 08:40:07,417] jepsen worker 802 - jepsen.util 3802 :info :add 9577 :RequestTimeoutException INFO [2022-01-10 08:40:07,419] jepsen worker 680 - jepsen.util 2680 :invoke :add 10667 INFO [2022-01-10 08:40:07,424] jepsen worker 482 - jepsen.util 2482 :invoke :add 10668 INFO [2022-01-10 08:40:07,426] jepsen worker 802 - jepsen.util 4802 :invoke :add 10669 INFO [2022-01-10 08:40:07,430] jepsen worker 623 - jepsen.util 2623 :invoke :add 10670 INFO [2022-01-10 08:40:07,432] jepsen worker 512 - jepsen.util 2512 :invoke :add 10671 INFO [2022-01-10 08:40:07,435] jepsen worker 961 - jepsen.util 2961 :invoke :add 10672 INFO [2022-01-10 08:40:07,500] jepsen worker 488 - jepsen.util 2488 :info :add 9584 :RequestTimeoutException INFO [2022-01-10 08:40:07,504] jepsen worker 488 - jepsen.util 3488 :invoke :add 10673 INFO [2022-01-10 08:40:07,523] jepsen worker 827 - jepsen.util 3827 :info :add 9585 :RequestTimeoutException INFO [2022-01-10 08:40:07,526] jepsen worker 111 - jepsen.util 2111 :info :add 9586 :RequestTimeoutException INFO [2022-01-10 08:40:07,529] jepsen worker 827 - jepsen.util 4827 :invoke :add 10674 INFO [2022-01-10 08:40:07,531] jepsen worker 555 - jepsen.util 3555 :info :add 9587 :RequestTimeoutException INFO [2022-01-10 08:40:07,533] jepsen worker 716 - jepsen.util 3716 :info :add 9588 :RequestTimeoutException INFO [2022-01-10 08:40:07,533] jepsen worker 111 - jepsen.util 3111 :invoke :add 10675 INFO [2022-01-10 08:40:07,537] jepsen worker 716 - jepsen.util 4716 :invoke :add 10676 INFO [2022-01-10 08:40:07,541] jepsen worker 555 - jepsen.util 4555 :invoke :add 10677 INFO [2022-01-10 08:40:07,543] jepsen worker 507 - jepsen.util 3507 :info :add 9591 :RequestTimeoutException INFO [2022-01-10 08:40:07,543] jepsen worker 803 - jepsen.util 2803 :info :add 9589 :RequestTimeoutException INFO [2022-01-10 08:40:07,544] jepsen worker 15 - jepsen.util 3015 :info :add 9590 :RequestTimeoutException INFO [2022-01-10 08:40:07,547] jepsen worker 552 - jepsen.util 3552 :info :add 9592 :RequestTimeoutException INFO [2022-01-10 08:40:07,549] jepsen worker 803 - jepsen.util 3803 :invoke :add 10678 INFO [2022-01-10 08:40:07,552] jepsen worker 552 - jepsen.util 4552 :invoke :add 10679 INFO [2022-01-10 08:40:07,555] jepsen worker 507 - jepsen.util 4507 :invoke :add 10680 INFO [2022-01-10 08:40:07,557] jepsen worker 15 - jepsen.util 4015 :invoke :add 10681 INFO [2022-01-10 08:40:07,560] jepsen worker 387 - jepsen.util 2387 :info :add 9593 :RequestTimeoutException INFO [2022-01-10 08:40:07,561] jepsen worker 765 - jepsen.util 2765 :info :add 9595 :RequestTimeoutException INFO [2022-01-10 08:40:07,562] jepsen worker 860 - jepsen.util 3860 :info :add 9594 :RequestTimeoutException INFO [2022-01-10 08:40:07,564] jepsen worker 387 - jepsen.util 3387 :invoke :add 10682 INFO [2022-01-10 08:40:07,569] jepsen worker 471 - jepsen.util 2471 :info :add 9596 :RequestTimeoutException INFO [2022-01-10 08:40:07,570] jepsen worker 765 - jepsen.util 3765 :invoke :add 10683 INFO [2022-01-10 08:40:07,571] jepsen worker 186 - jepsen.util 3186 :info :add 9597 :RequestTimeoutException INFO [2022-01-10 08:40:07,574] jepsen worker 186 - jepsen.util 4186 :invoke :add 10684 INFO [2022-01-10 08:40:07,577] jepsen worker 454 - jepsen.util 2454 :info :add 9600 :RequestTimeoutException INFO [2022-01-10 08:40:07,577] jepsen worker 860 - jepsen.util 4860 :invoke :add 10685 INFO [2022-01-10 08:40:07,580] jepsen worker 621 - jepsen.util 2621 :info :add 9601 :RequestTimeoutException INFO [2022-01-10 08:40:07,581] jepsen worker 454 - jepsen.util 3454 :invoke :add 10686 INFO [2022-01-10 08:40:07,583] jepsen worker 495 - jepsen.util 2495 :info :add 9602 :RequestTimeoutException INFO [2022-01-10 08:40:07,584] jepsen worker 621 - jepsen.util 3621 :invoke :add 10687 INFO [2022-01-10 08:40:07,588] jepsen worker 495 - jepsen.util 3495 :invoke :add 10688 INFO [2022-01-10 08:40:07,590] jepsen worker 471 - jepsen.util 3471 :invoke :add 10689 INFO [2022-01-10 08:40:07,591] jepsen worker 118 - jepsen.util 2118 :info :add 9603 :RequestTimeoutException INFO [2022-01-10 08:40:07,596] jepsen worker 118 - jepsen.util 3118 :invoke :add 10690 INFO [2022-01-10 08:40:07,602] jepsen worker 316 - jepsen.util 2316 :info :add 9605 :RequestTimeoutException INFO [2022-01-10 08:40:07,604] jepsen worker 233 - jepsen.util 2233 :info :add 9604 :RequestTimeoutException INFO [2022-01-10 08:40:07,606] jepsen worker 316 - jepsen.util 3316 :invoke :add 10691 INFO [2022-01-10 08:40:07,607] jepsen worker 156 - jepsen.util 2156 :info :add 9606 :RequestTimeoutException INFO [2022-01-10 08:40:07,611] jepsen worker 156 - jepsen.util 3156 :invoke :add 10692 INFO [2022-01-10 08:40:07,614] jepsen worker 233 - jepsen.util 3233 :invoke :add 10693 INFO [2022-01-10 08:40:07,626] jepsen worker 636 - jepsen.util 1636 :ok :add 9731 INFO [2022-01-10 08:40:07,633] jepsen worker 636 - jepsen.util 1636 :invoke :add 10694 INFO [2022-01-10 08:40:07,730] jepsen worker 262 - jepsen.util 2262 :info :add 9607 :RequestTimeoutException INFO [2022-01-10 08:40:07,735] jepsen worker 262 - jepsen.util 3262 :invoke :add 10695 INFO [2022-01-10 08:40:07,742] jepsen worker 179 - jepsen.util 3179 :ok :add 9754 INFO [2022-01-10 08:40:07,742] jepsen worker 673 - jepsen.util 3673 :ok :add 9778 INFO [2022-01-10 08:40:07,742] jepsen worker 717 - jepsen.util 2717 :ok :add 9760 INFO [2022-01-10 08:40:07,747] jepsen worker 673 - jepsen.util 3673 :invoke :add 10696 INFO [2022-01-10 08:40:07,749] jepsen worker 179 - jepsen.util 3179 :invoke :add 10697 INFO [2022-01-10 08:40:07,753] jepsen worker 717 - jepsen.util 2717 :invoke :add 10698 INFO [2022-01-10 08:40:07,883] jepsen worker 813 - jepsen.util 2813 :info :add 9614 :RequestTimeoutException INFO [2022-01-10 08:40:07,887] jepsen worker 813 - jepsen.util 3813 :invoke :add 10699 INFO [2022-01-10 08:40:07,891] jepsen worker 35 - jepsen.util 2035 :info :add 9615 :RequestTimeoutException INFO [2022-01-10 08:40:07,893] jepsen worker 320 - jepsen.util 2320 :info :add 9616 :RequestTimeoutException INFO [2022-01-10 08:40:07,898] jepsen worker 35 - jepsen.util 3035 :invoke :add 10700 INFO [2022-01-10 08:40:07,901] jepsen worker 320 - jepsen.util 3320 :invoke :add 10701 INFO [2022-01-10 08:40:07,902] jepsen worker 204 - jepsen.util 2204 :info :add 9617 :RequestTimeoutException INFO [2022-01-10 08:40:07,906] jepsen worker 204 - jepsen.util 3204 :invoke :add 10702 INFO [2022-01-10 08:40:07,907] jepsen worker 997 - jepsen.util 3997 :info :add 9618 :RequestTimeoutException INFO [2022-01-10 08:40:07,910] jepsen worker 437 - jepsen.util 2437 :info :add 9619 :RequestTimeoutException INFO [2022-01-10 08:40:07,911] jepsen worker 221 - jepsen.util 2221 :info :add 9621 :RequestTimeoutException INFO [2022-01-10 08:40:07,911] jepsen worker 997 - jepsen.util 4997 :invoke :add 10703 INFO [2022-01-10 08:40:07,913] jepsen worker 977 - jepsen.util 2977 :info :add 9620 :RequestTimeoutException INFO [2022-01-10 08:40:07,917] jepsen worker 221 - jepsen.util 3221 :invoke :add 10704 INFO [2022-01-10 08:40:07,919] jepsen worker 977 - jepsen.util 3977 :invoke :add 10705 INFO [2022-01-10 08:40:07,923] jepsen worker 437 - jepsen.util 3437 :invoke :add 10706 INFO [2022-01-10 08:40:07,923] jepsen worker 56 - jepsen.util 3056 :info :add 9622 :RequestTimeoutException INFO [2022-01-10 08:40:07,929] jepsen worker 56 - jepsen.util 4056 :invoke :add 10707 INFO [2022-01-10 08:40:07,932] jepsen worker 300 - jepsen.util 2300 :info :add 9623 :RequestTimeoutException INFO [2022-01-10 08:40:07,935] jepsen worker 300 - jepsen.util 3300 :invoke :add 10708 INFO [2022-01-10 08:40:07,954] jepsen worker 857 - jepsen.util 2857 :info :add 9628 :RequestTimeoutException INFO [2022-01-10 08:40:07,958] jepsen worker 857 - jepsen.util 3857 :invoke :add 10709 INFO [2022-01-10 08:40:07,972] jepsen worker 215 - jepsen.util 2215 :info :add 9630 :RequestTimeoutException INFO [2022-01-10 08:40:07,975] jepsen worker 257 - jepsen.util 2257 :info :add 9629 :RequestTimeoutException INFO [2022-01-10 08:40:07,975] jepsen worker 181 - jepsen.util 1181 :info :add 9631 :RequestTimeoutException INFO [2022-01-10 08:40:07,980] jepsen worker 181 - jepsen.util 2181 :invoke :add 10710 INFO [2022-01-10 08:40:07,983] jepsen worker 257 - jepsen.util 3257 :invoke :add 10711 INFO [2022-01-10 08:40:07,986] jepsen worker 215 - jepsen.util 3215 :invoke :add 10712 INFO [2022-01-10 08:40:08,027] jepsen worker 627 - jepsen.util 3627 :info :add 9633 :RequestTimeoutException INFO [2022-01-10 08:40:08,031] jepsen worker 627 - jepsen.util 4627 :invoke :add 10713 INFO [2022-01-10 08:40:08,032] jepsen worker 609 - jepsen.util 3609 :info :add 9632 :RequestTimeoutException INFO [2022-01-10 08:40:08,038] jepsen worker 609 - jepsen.util 4609 :invoke :add 10714 INFO [2022-01-10 08:40:08,040] jepsen worker 244 - jepsen.util 3244 :info :add 9634 :RequestTimeoutException INFO [2022-01-10 08:40:08,045] jepsen worker 244 - jepsen.util 4244 :invoke :add 10715 INFO [2022-01-10 08:40:08,083] jepsen worker 266 - jepsen.util 3266 :info :add 9635 :RequestTimeoutException INFO [2022-01-10 08:40:08,089] jepsen worker 266 - jepsen.util 4266 :invoke :add 10716 INFO [2022-01-10 08:40:08,121] jepsen worker 222 - jepsen.util 2222 :info :add 9636 :RequestTimeoutException INFO [2022-01-10 08:40:08,125] jepsen worker 222 - jepsen.util 3222 :invoke :add 10717 INFO [2022-01-10 08:40:08,143] jepsen worker 861 - jepsen.util 2861 :info :add 9637 :RequestTimeoutException INFO [2022-01-10 08:40:08,146] jepsen worker 604 - jepsen.util 2604 :info :add 9639 :RequestTimeoutException INFO [2022-01-10 08:40:08,150] jepsen worker 604 - jepsen.util 3604 :invoke :add 10718 INFO [2022-01-10 08:40:08,151] jepsen worker 878 - jepsen.util 3878 :ok :add 10022 INFO [2022-01-10 08:40:08,152] jepsen worker 859 - jepsen.util 2859 :ok :add 9944 INFO [2022-01-10 08:40:08,152] jepsen worker 368 - jepsen.util 2368 :ok :add 9661 INFO [2022-01-10 08:40:08,152] jepsen worker 43 - jepsen.util 2043 :info :add 9638 :RequestTimeoutException INFO [2022-01-10 08:40:08,152] jepsen worker 535 - jepsen.util 2535 :ok :add 9968 INFO [2022-01-10 08:40:08,154] jepsen worker 861 - jepsen.util 3861 :invoke :add 10719 INFO [2022-01-10 08:40:08,160] jepsen worker 859 - jepsen.util 2859 :invoke :add 10720 INFO [2022-01-10 08:40:08,160] jepsen worker 892 - jepsen.util 3892 :info :add 9640 :RequestTimeoutException INFO [2022-01-10 08:40:08,161] jepsen worker 871 - jepsen.util 2871 :info :add 9642 :RequestTimeoutException INFO [2022-01-10 08:40:08,162] jepsen worker 878 - jepsen.util 3878 :invoke :add 10721 INFO [2022-01-10 08:40:08,163] jepsen worker 120 - jepsen.util 2120 :info :add 9641 :RequestTimeoutException INFO [2022-01-10 08:40:08,169] jepsen worker 892 - jepsen.util 4892 :invoke :add 10722 INFO [2022-01-10 08:40:08,172] jepsen worker 43 - jepsen.util 3043 :invoke :add 10723 INFO [2022-01-10 08:40:08,174] jepsen worker 639 - jepsen.util 2639 :info :add 9643 :RequestTimeoutException INFO [2022-01-10 08:40:08,174] jepsen worker 535 - jepsen.util 2535 :invoke :add 10724 INFO [2022-01-10 08:40:08,179] jepsen worker 639 - jepsen.util 3639 :invoke :add 10725 INFO [2022-01-10 08:40:08,181] jepsen worker 871 - jepsen.util 3871 :invoke :add 10726 INFO [2022-01-10 08:40:08,182] jepsen worker 701 - jepsen.util 3701 :info :add 9644 :RequestTimeoutException INFO [2022-01-10 08:40:08,184] jepsen worker 120 - jepsen.util 3120 :invoke :add 10727 INFO [2022-01-10 08:40:08,188] jepsen worker 701 - jepsen.util 4701 :invoke :add 10728 INFO [2022-01-10 08:40:08,191] jepsen worker 368 - jepsen.util 2368 :invoke :add 10729 INFO [2022-01-10 08:40:08,230] jepsen worker 750 - jepsen.util 2750 :info :add 9645 :RequestTimeoutException INFO [2022-01-10 08:40:08,235] jepsen worker 750 - jepsen.util 3750 :invoke :add 10730 INFO [2022-01-10 08:40:08,277] jepsen worker 868 - jepsen.util 3868 :info :add 9646 :RequestTimeoutException INFO [2022-01-10 08:40:08,282] jepsen worker 868 - jepsen.util 4868 :invoke :add 10731 INFO [2022-01-10 08:40:08,290] jepsen worker 194 - jepsen.util 3194 :info :add 9647 :RequestTimeoutException INFO [2022-01-10 08:40:08,293] jepsen worker 278 - jepsen.util 3278 :info :add 9648 :RequestTimeoutException INFO [2022-01-10 08:40:08,297] jepsen worker 194 - jepsen.util 4194 :invoke :add 10732 INFO [2022-01-10 08:40:08,299] jepsen worker 278 - jepsen.util 4278 :invoke :add 10733 INFO [2022-01-10 08:40:08,343] jepsen worker 804 - jepsen.util 2804 :info :add 9649 :RequestTimeoutException INFO [2022-01-10 08:40:08,343] jepsen worker 899 - jepsen.util 2899 :info :add 9650 :RequestTimeoutException INFO [2022-01-10 08:40:08,351] jepsen worker 899 - jepsen.util 3899 :invoke :add 10734 INFO [2022-01-10 08:40:08,352] jepsen worker 123 - jepsen.util 3123 :info :add 9651 :RequestTimeoutException INFO [2022-01-10 08:40:08,353] jepsen worker 804 - jepsen.util 3804 :invoke :add 10735 INFO [2022-01-10 08:40:08,357] jepsen worker 123 - jepsen.util 4123 :invoke :add 10736 INFO [2022-01-10 08:40:08,362] jepsen worker 322 - jepsen.util 3322 :info :add 9653 :RequestTimeoutException INFO [2022-01-10 08:40:08,363] jepsen worker 571 - jepsen.util 3571 :info :add 9652 :RequestTimeoutException INFO [2022-01-10 08:40:08,367] jepsen worker 322 - jepsen.util 4322 :invoke :add 10737 INFO [2022-01-10 08:40:08,370] jepsen worker 571 - jepsen.util 4571 :invoke :add 10738 INFO [2022-01-10 08:40:08,420] jepsen worker 372 - jepsen.util 2372 :info :add 9658 :RequestTimeoutException INFO [2022-01-10 08:40:08,422] jepsen worker 282 - jepsen.util 3282 :info :add 9659 :RequestTimeoutException INFO [2022-01-10 08:40:08,428] jepsen worker 372 - jepsen.util 3372 :invoke :add 10739 INFO [2022-01-10 08:40:08,431] jepsen worker 282 - jepsen.util 4282 :invoke :add 10740 INFO [2022-01-10 08:40:08,467] jepsen worker 525 - jepsen.util 2525 :info :add 9664 :RequestTimeoutException INFO [2022-01-10 08:40:08,471] jepsen worker 525 - jepsen.util 3525 :invoke :add 10741 INFO [2022-01-10 08:40:08,472] jepsen worker 928 - jepsen.util 3928 :info :add 9665 :RequestTimeoutException INFO [2022-01-10 08:40:08,478] jepsen worker 928 - jepsen.util 4928 :invoke :add 10742 INFO [2022-01-10 08:40:08,626] jepsen worker 172 - jepsen.util 2172 :ok :add 10065 INFO [2022-01-10 08:40:08,626] jepsen worker 650 - jepsen.util 2650 :ok :add 9782 INFO [2022-01-10 08:40:08,626] jepsen worker 363 - jepsen.util 3363 :ok :add 9893 INFO [2022-01-10 08:40:08,627] jepsen worker 424 - jepsen.util 2424 :info :add 9668 :RequestTimeoutException INFO [2022-01-10 08:40:08,627] jepsen worker 129 - jepsen.util 2129 :info :add 9684 :RequestTimeoutException INFO [2022-01-10 08:40:08,627] jepsen worker 948 - jepsen.util 2948 :info :add 9675 :RequestTimeoutException INFO [2022-01-10 08:40:08,627] jepsen worker 411 - jepsen.util 2411 :info :add 9671 :RequestTimeoutException INFO [2022-01-10 08:40:08,627] jepsen worker 655 - jepsen.util 2655 :info :add 9666 :RequestTimeoutException INFO [2022-01-10 08:40:08,628] jepsen worker 318 - jepsen.util 2318 :info :add 9672 :RequestTimeoutException INFO [2022-01-10 08:40:08,628] jepsen worker 272 - jepsen.util 3272 :info :add 9677 :RequestTimeoutException INFO [2022-01-10 08:40:08,629] jepsen worker 577 - jepsen.util 3577 :info :add 9669 :RequestTimeoutException INFO [2022-01-10 08:40:08,629] jepsen worker 869 - jepsen.util 3869 :info :add 9667 :RequestTimeoutException INFO [2022-01-10 08:40:08,629] jepsen worker 51 - jepsen.util 2051 :info :add 9676 :RequestTimeoutException INFO [2022-01-10 08:40:08,629] jepsen worker 448 - jepsen.util 2448 :info :add 9673 :RequestTimeoutException INFO [2022-01-10 08:40:08,629] jepsen worker 458 - jepsen.util 3458 :info :add 9674 :RequestTimeoutException INFO [2022-01-10 08:40:08,630] jepsen worker 926 - jepsen.util 3926 :ok :add 9711 INFO [2022-01-10 08:40:08,630] jepsen worker 576 - jepsen.util 2576 :ok :add 10387 INFO [2022-01-10 08:40:08,630] jepsen worker 826 - jepsen.util 3826 :ok :add 10565 INFO [2022-01-10 08:40:08,631] jepsen worker 852 - jepsen.util 2852 :info :add 9685 :RequestTimeoutException INFO [2022-01-10 08:40:08,634] jepsen worker 106 - jepsen.util 3106 :info :add 9686 :RequestTimeoutException INFO [2022-01-10 08:40:08,642] jepsen worker 933 - jepsen.util 2933 :info :add 9687 :RequestTimeoutException INFO [2022-01-10 08:40:08,645] jepsen worker 458 - jepsen.util 4458 :invoke :add 10743 INFO [2022-01-10 08:40:08,647] jepsen worker 129 - jepsen.util 3129 :invoke :add 10744 INFO [2022-01-10 08:40:08,647] jepsen worker 551 - jepsen.util 3551 :info :add 9688 :RequestTimeoutException INFO [2022-01-10 08:40:08,648] jepsen worker 948 - jepsen.util 3948 :invoke :add 10745 INFO [2022-01-10 08:40:08,650] jepsen worker 620 - jepsen.util 3620 :info :add 9689 :RequestTimeoutException INFO [2022-01-10 08:40:08,651] jepsen worker 650 - jepsen.util 2650 :invoke :add 10746 INFO [2022-01-10 08:40:08,651] jepsen worker 473 - jepsen.util 2473 :info :add 9691 :RequestTimeoutException INFO [2022-01-10 08:40:08,652] jepsen worker 310 - jepsen.util 3310 :info :add 9690 :RequestTimeoutException INFO [2022-01-10 08:40:08,653] jepsen worker 942 - jepsen.util 3942 :info :add 9692 :RequestTimeoutException INFO [2022-01-10 08:40:08,653] jepsen worker 826 - jepsen.util 3826 :invoke :add 10747 INFO [2022-01-10 08:40:08,656] jepsen worker 51 - jepsen.util 3051 :invoke :add 10748 INFO [2022-01-10 08:40:08,656] jepsen worker 434 - jepsen.util 2434 :info :add 9694 :RequestTimeoutException INFO [2022-01-10 08:40:08,658] jepsen worker 310 - jepsen.util 4310 :invoke :add 10749 INFO [2022-01-10 08:40:08,660] jepsen worker 837 - jepsen.util 3837 :info :add 9695 :RequestTimeoutException INFO [2022-01-10 08:40:08,660] jepsen worker 448 - jepsen.util 3448 :invoke :add 10750 INFO [2022-01-10 08:40:08,662] jepsen worker 998 - jepsen.util 3998 :info :add 9693 :RequestTimeoutException INFO [2022-01-10 08:40:08,663] jepsen worker 318 - jepsen.util 3318 :invoke :add 10751 INFO [2022-01-10 08:40:08,665] jepsen worker 576 - jepsen.util 2576 :invoke :add 10752 INFO [2022-01-10 08:40:08,667] jepsen worker 106 - jepsen.util 4106 :invoke :add 10753 INFO [2022-01-10 08:40:08,669] jepsen worker 551 - jepsen.util 4551 :invoke :add 10754 INFO [2022-01-10 08:40:08,670] jepsen worker 998 - jepsen.util 4998 :invoke :add 10755 INFO [2022-01-10 08:40:08,672] jepsen worker 172 - jepsen.util 2172 :invoke :add 10756 INFO [2022-01-10 08:40:08,673] jepsen worker 645 - jepsen.util 3645 :info :add 9696 :RequestTimeoutException INFO [2022-01-10 08:40:08,674] jepsen worker 933 - jepsen.util 3933 :invoke :add 10757 INFO [2022-01-10 08:40:08,676] jepsen worker 272 - jepsen.util 4272 :invoke :add 10758 INFO [2022-01-10 08:40:08,678] jepsen worker 942 - jepsen.util 4942 :invoke :add 10759 INFO [2022-01-10 08:40:08,679] jepsen worker 837 - jepsen.util 4837 :invoke :add 10760 INFO [2022-01-10 08:40:08,681] jepsen worker 608 - jepsen.util 3608 :info :add 9697 :RequestTimeoutException INFO [2022-01-10 08:40:08,681] jepsen worker 424 - jepsen.util 3424 :invoke :add 10761 INFO [2022-01-10 08:40:08,684] jepsen worker 926 - jepsen.util 3926 :invoke :add 10762 INFO [2022-01-10 08:40:08,685] jepsen worker 608 - jepsen.util 4608 :invoke :add 10763 INFO [2022-01-10 08:40:08,687] jepsen worker 620 - jepsen.util 4620 :invoke :add 10764 INFO [2022-01-10 08:40:08,689] jepsen worker 852 - jepsen.util 3852 :invoke :add 10765 INFO [2022-01-10 08:40:08,691] jepsen worker 869 - jepsen.util 4869 :invoke :add 10766 INFO [2022-01-10 08:40:08,692] jepsen worker 655 - jepsen.util 3655 :invoke :add 10767 INFO [2022-01-10 08:40:08,694] jepsen worker 473 - jepsen.util 3473 :invoke :add 10768 INFO [2022-01-10 08:40:08,696] jepsen worker 434 - jepsen.util 3434 :invoke :add 10769 INFO [2022-01-10 08:40:08,697] jepsen worker 645 - jepsen.util 4645 :invoke :add 10770 INFO [2022-01-10 08:40:08,699] jepsen worker 577 - jepsen.util 4577 :invoke :add 10771 INFO [2022-01-10 08:40:08,700] jepsen worker 386 - jepsen.util 2386 :ok :add 10352 INFO [2022-01-10 08:40:08,701] jepsen worker 411 - jepsen.util 3411 :invoke :add 10772 INFO [2022-01-10 08:40:08,703] jepsen worker 386 - jepsen.util 2386 :invoke :add 10773 INFO [2022-01-10 08:40:08,705] jepsen worker 363 - jepsen.util 3363 :invoke :add 10774 INFO [2022-01-10 08:40:08,747] jepsen worker 550 - jepsen.util 3550 :info :add 9700 :RequestTimeoutException INFO [2022-01-10 08:40:08,749] jepsen worker 550 - jepsen.util 4550 :invoke :add 10775 INFO [2022-01-10 08:40:08,750] jepsen worker 595 - jepsen.util 3595 :info :add 9701 :RequestTimeoutException INFO [2022-01-10 08:40:08,752] jepsen worker 234 - jepsen.util 3234 :info :add 9702 :RequestTimeoutException INFO [2022-01-10 08:40:08,753] jepsen worker 595 - jepsen.util 4595 :invoke :add 10776 INFO [2022-01-10 08:40:08,755] jepsen worker 234 - jepsen.util 4234 :invoke :add 10777 INFO [2022-01-10 08:40:08,761] jepsen worker 692 - jepsen.util 3692 :info :add 9703 :RequestTimeoutException INFO [2022-01-10 08:40:08,763] jepsen worker 991 - jepsen.util 3991 :info :add 9704 :RequestTimeoutException INFO [2022-01-10 08:40:08,764] jepsen worker 692 - jepsen.util 4692 :invoke :add 10778 INFO [2022-01-10 08:40:08,766] jepsen worker 991 - jepsen.util 4991 :invoke :add 10779 INFO [2022-01-10 08:40:08,842] jepsen worker 427 - jepsen.util 3427 :info :add 9705 :RequestTimeoutException INFO [2022-01-10 08:40:08,845] jepsen worker 427 - jepsen.util 4427 :invoke :add 10780 INFO [2022-01-10 08:40:08,877] jepsen worker 953 - jepsen.util 3953 :info :add 9706 :RequestTimeoutException INFO [2022-01-10 08:40:08,880] jepsen worker 201 - jepsen.util 3201 :info :add 9707 :RequestTimeoutException INFO [2022-01-10 08:40:08,881] jepsen worker 953 - jepsen.util 4953 :invoke :add 10781 INFO [2022-01-10 08:40:08,883] jepsen worker 201 - jepsen.util 4201 :invoke :add 10782 INFO [2022-01-10 08:40:08,903] jepsen worker 767 - jepsen.util 2767 :info :add 9714 :RequestTimeoutException INFO [2022-01-10 08:40:08,906] jepsen worker 767 - jepsen.util 3767 :invoke :add 10783 INFO [2022-01-10 08:40:08,911] jepsen worker 455 - jepsen.util 2455 :info :add 9715 :RequestTimeoutException INFO [2022-01-10 08:40:08,914] jepsen worker 325 - jepsen.util 2325 :info :add 9716 :RequestTimeoutException INFO [2022-01-10 08:40:08,915] jepsen worker 455 - jepsen.util 3455 :invoke :add 10784 INFO [2022-01-10 08:40:08,917] jepsen worker 493 - jepsen.util 2493 :info :add 9718 :RequestTimeoutException INFO [2022-01-10 08:40:08,918] jepsen worker 325 - jepsen.util 3325 :invoke :add 10785 INFO [2022-01-10 08:40:08,920] jepsen worker 219 - jepsen.util 2219 :info :add 9719 :RequestTimeoutException INFO [2022-01-10 08:40:08,921] jepsen worker 493 - jepsen.util 3493 :invoke :add 10786 INFO [2022-01-10 08:40:08,922] jepsen worker 193 - jepsen.util 2193 :info :add 9717 :RequestTimeoutException INFO [2022-01-10 08:40:08,923] jepsen worker 219 - jepsen.util 3219 :invoke :add 10787 INFO [2022-01-10 08:40:08,926] jepsen worker 193 - jepsen.util 3193 :invoke :add 10788 INFO [2022-01-10 08:40:08,961] jepsen worker 91 - jepsen.util 3091 :info :add 9721 :RequestTimeoutException INFO [2022-01-10 08:40:08,963] jepsen worker 756 - jepsen.util 3756 :info :add 9720 :RequestTimeoutException INFO [2022-01-10 08:40:08,964] jepsen worker 91 - jepsen.util 4091 :invoke :add 10789 INFO [2022-01-10 08:40:08,966] jepsen worker 756 - jepsen.util 4756 :invoke :add 10790 INFO [2022-01-10 08:40:08,984] jepsen worker 76 - jepsen.util 1076 :info :add 9722 :RequestTimeoutException INFO [2022-01-10 08:40:08,984] jepsen worker 44 - jepsen.util 3044 :info :add 9723 :RequestTimeoutException INFO [2022-01-10 08:40:08,988] jepsen worker 76 - jepsen.util 2076 :invoke :add 10791 INFO [2022-01-10 08:40:08,989] jepsen worker 44 - jepsen.util 4044 :invoke :add 10792 INFO [2022-01-10 08:40:09,130] jepsen worker 867 - jepsen.util 2867 :info :add 9728 :RequestTimeoutException INFO [2022-01-10 08:40:09,134] jepsen worker 556 - jepsen.util 1556 :info :add 9729 :RequestTimeoutException INFO [2022-01-10 08:40:09,134] jepsen worker 867 - jepsen.util 3867 :invoke :add 10793 INFO [2022-01-10 08:40:09,137] jepsen worker 556 - jepsen.util 2556 :invoke :add 10794 INFO [2022-01-10 08:40:09,140] jepsen worker 787 - jepsen.util 1787 :info :add 9730 :RequestTimeoutException INFO [2022-01-10 08:40:09,142] jepsen worker 128 - jepsen.util 3128 :info :add 9733 :RequestTimeoutException INFO [2022-01-10 08:40:09,143] jepsen worker 381 - jepsen.util 3381 :info :add 9732 :RequestTimeoutException INFO [2022-01-10 08:40:09,144] jepsen worker 787 - jepsen.util 2787 :invoke :add 10795 INFO [2022-01-10 08:40:09,147] jepsen worker 128 - jepsen.util 4128 :invoke :add 10796 INFO [2022-01-10 08:40:09,147] jepsen worker 820 - jepsen.util 2820 :info :add 9734 :RequestTimeoutException INFO [2022-01-10 08:40:09,148] jepsen worker 381 - jepsen.util 4381 :invoke :add 10797 INFO [2022-01-10 08:40:09,149] jepsen worker 399 - jepsen.util 2399 :info :add 9735 :RequestTimeoutException INFO [2022-01-10 08:40:09,151] jepsen worker 820 - jepsen.util 3820 :invoke :add 10798 INFO [2022-01-10 08:40:09,153] jepsen worker 399 - jepsen.util 3399 :invoke :add 10799 INFO [2022-01-10 08:40:09,233] jepsen worker 536 - jepsen.util 3536 :info :add 9736 :RequestTimeoutException INFO [2022-01-10 08:40:09,236] jepsen worker 536 - jepsen.util 4536 :invoke :add 10800 INFO [2022-01-10 08:40:09,241] jepsen worker 10 - jepsen.util 3010 :info :add 9737 :RequestTimeoutException INFO [2022-01-10 08:40:09,243] jepsen worker 297 - jepsen.util 3297 :info :add 9738 :RequestTimeoutException INFO [2022-01-10 08:40:09,245] jepsen worker 10 - jepsen.util 4010 :invoke :add 10801 INFO [2022-01-10 08:40:09,248] jepsen worker 297 - jepsen.util 4297 :invoke :add 10802 INFO [2022-01-10 08:40:09,280] jepsen worker 286 - jepsen.util 2286 :info :add 9739 :RequestTimeoutException INFO [2022-01-10 08:40:09,283] jepsen worker 286 - jepsen.util 3286 :invoke :add 10803 INFO [2022-01-10 08:40:09,312] jepsen worker 984 - jepsen.util 3984 :info :add 9740 :RequestTimeoutException INFO [2022-01-10 08:40:09,314] jepsen worker 984 - jepsen.util 4984 :invoke :add 10804 INFO [2022-01-10 08:40:09,316] jepsen worker 63 - jepsen.util 3063 :info :add 9741 :RequestTimeoutException INFO [2022-01-10 08:40:09,319] jepsen worker 63 - jepsen.util 4063 :invoke :add 10805 INFO [2022-01-10 08:40:09,320] jepsen worker 305 - jepsen.util 3305 :info :add 9742 :RequestTimeoutException INFO [2022-01-10 08:40:09,323] jepsen worker 305 - jepsen.util 4305 :invoke :add 10806 INFO [2022-01-10 08:40:09,333] jepsen worker 828 - jepsen.util 3828 :info :add 9743 :RequestTimeoutException INFO [2022-01-10 08:40:09,335] jepsen worker 828 - jepsen.util 4828 :invoke :add 10807 INFO [2022-01-10 08:40:09,341] jepsen worker 26 - jepsen.util 3026 :info :add 9744 :RequestTimeoutException INFO [2022-01-10 08:40:09,343] jepsen worker 665 - jepsen.util 2665 :info :add 9745 :RequestTimeoutException INFO [2022-01-10 08:40:09,344] jepsen worker 26 - jepsen.util 4026 :invoke :add 10808 INFO [2022-01-10 08:40:09,346] jepsen worker 665 - jepsen.util 3665 :invoke :add 10809 INFO [2022-01-10 08:40:09,350] jepsen worker 263 - jepsen.util 1263 :info :add 9747 :RequestTimeoutException INFO [2022-01-10 08:40:09,352] jepsen worker 302 - jepsen.util 1302 :info :add 9748 :RequestTimeoutException INFO [2022-01-10 08:40:09,352] jepsen worker 312 - jepsen.util 3312 :info :add 9746 :RequestTimeoutException INFO [2022-01-10 08:40:09,353] jepsen worker 433 - jepsen.util 2433 :info :add 9749 :RequestTimeoutException INFO [2022-01-10 08:40:09,353] jepsen worker 263 - jepsen.util 2263 :invoke :add 10810 INFO [2022-01-10 08:40:09,357] jepsen worker 433 - jepsen.util 3433 :invoke :add 10811 INFO [2022-01-10 08:40:09,357] jepsen worker 922 - jepsen.util 3922 :info :add 9750 :RequestTimeoutException INFO [2022-01-10 08:40:09,359] jepsen worker 312 - jepsen.util 4312 :invoke :add 10812 INFO [2022-01-10 08:40:09,360] jepsen worker 0 - jepsen.util 3000 :info :add 9751 :RequestTimeoutException INFO [2022-01-10 08:40:09,361] jepsen worker 922 - jepsen.util 4922 :invoke :add 10813 INFO [2022-01-10 08:40:09,363] jepsen worker 341 - jepsen.util 3341 :info :add 9752 :RequestTimeoutException INFO [2022-01-10 08:40:09,363] jepsen worker 0 - jepsen.util 4000 :invoke :add 10814 INFO [2022-01-10 08:40:09,365] jepsen worker 302 - jepsen.util 2302 :invoke :add 10815 INFO [2022-01-10 08:40:09,367] jepsen worker 341 - jepsen.util 4341 :invoke :add 10816 INFO [2022-01-10 08:40:09,371] jepsen worker 117 - jepsen.util 3117 :info :add 9753 :RequestTimeoutException INFO [2022-01-10 08:40:09,374] jepsen worker 117 - jepsen.util 4117 :invoke :add 10817 INFO [2022-01-10 08:40:09,411] jepsen worker 612 - jepsen.util 2612 :info :add 9759 :RequestTimeoutException INFO [2022-01-10 08:40:09,413] jepsen worker 678 - jepsen.util 2678 :info :add 9758 :RequestTimeoutException INFO [2022-01-10 08:40:09,414] jepsen worker 612 - jepsen.util 3612 :invoke :add 10818 INFO [2022-01-10 08:40:09,416] jepsen worker 678 - jepsen.util 3678 :invoke :add 10819 INFO [2022-01-10 08:40:09,431] jepsen worker 721 - jepsen.util 2721 :info :add 9765 :RequestTimeoutException INFO [2022-01-10 08:40:09,432] jepsen worker 12 - jepsen.util 2012 :info :add 9764 :RequestTimeoutException INFO [2022-01-10 08:40:09,433] jepsen worker 721 - jepsen.util 3721 :invoke :add 10820 INFO [2022-01-10 08:40:09,436] jepsen worker 12 - jepsen.util 3012 :invoke :add 10821 INFO [2022-01-10 08:40:09,454] jepsen worker 480 - jepsen.util 3480 :info :add 9766 :RequestTimeoutException INFO [2022-01-10 08:40:09,457] jepsen worker 480 - jepsen.util 4480 :invoke :add 10822 INFO [2022-01-10 08:40:09,462] jepsen worker 65 - jepsen.util 3065 :info :add 9767 :RequestTimeoutException INFO [2022-01-10 08:40:09,465] jepsen worker 65 - jepsen.util 4065 :invoke :add 10823 INFO [2022-01-10 08:40:09,467] jepsen worker 773 - jepsen.util 3773 :info :add 9768 :RequestTimeoutException INFO [2022-01-10 08:40:09,469] jepsen worker 773 - jepsen.util 4773 :invoke :add 10824 INFO [2022-01-10 08:40:09,470] jepsen worker 252 - jepsen.util 3252 :info :add 9769 :RequestTimeoutException INFO [2022-01-10 08:40:09,472] jepsen worker 252 - jepsen.util 4252 :invoke :add 10825 INFO [2022-01-10 08:40:09,474] jepsen worker 784 - jepsen.util 3784 :info :add 9770 :RequestTimeoutException INFO [2022-01-10 08:40:09,476] jepsen worker 784 - jepsen.util 4784 :invoke :add 10826 INFO [2022-01-10 08:40:09,481] jepsen worker 836 - jepsen.util 3836 :info :add 9771 :RequestTimeoutException INFO [2022-01-10 08:40:09,483] jepsen worker 836 - jepsen.util 4836 :invoke :add 10827 INFO [2022-01-10 08:40:09,532] jepsen worker 269 - jepsen.util 3269 :info :add 9773 :RequestTimeoutException INFO [2022-01-10 08:40:09,533] jepsen worker 759 - jepsen.util 3759 :info :add 9772 :RequestTimeoutException INFO [2022-01-10 08:40:09,535] jepsen worker 759 - jepsen.util 4759 :invoke :add 10828 INFO [2022-01-10 08:40:09,537] jepsen worker 269 - jepsen.util 4269 :invoke :add 10829 INFO [2022-01-10 08:40:09,547] jepsen worker 992 - jepsen.util 3992 :info :add 9774 :RequestTimeoutException INFO [2022-01-10 08:40:09,550] jepsen worker 985 - jepsen.util 3985 :info :add 9775 :RequestTimeoutException INFO [2022-01-10 08:40:09,550] jepsen worker 992 - jepsen.util 4992 :invoke :add 10830 INFO [2022-01-10 08:40:09,552] jepsen worker 985 - jepsen.util 4985 :invoke :add 10831 INFO [2022-01-10 08:40:09,552] jepsen worker 723 - jepsen.util 3723 :info :add 9776 :RequestTimeoutException INFO [2022-01-10 08:40:09,556] jepsen worker 723 - jepsen.util 4723 :invoke :add 10832 INFO [2022-01-10 08:40:09,591] jepsen worker 428 - jepsen.util 3428 :info :add 9777 :RequestTimeoutException INFO [2022-01-10 08:40:09,593] jepsen worker 428 - jepsen.util 4428 :invoke :add 10833 INFO [2022-01-10 08:40:09,670] jepsen worker 967 - jepsen.util 3967 :info :add 9784 :RequestTimeoutException INFO [2022-01-10 08:40:09,672] jepsen worker 485 - jepsen.util 2485 :info :add 9785 :RequestTimeoutException INFO [2022-01-10 08:40:09,673] jepsen worker 967 - jepsen.util 4967 :invoke :add 10834 INFO [2022-01-10 08:40:09,675] jepsen worker 485 - jepsen.util 3485 :invoke :add 10835 INFO [2022-01-10 08:40:09,677] jepsen worker 904 - jepsen.util 2904 :info :add 9786 :RequestTimeoutException INFO [2022-01-10 08:40:09,679] jepsen worker 904 - jepsen.util 3904 :invoke :add 10836 INFO [2022-01-10 08:40:09,690] jepsen worker 383 - jepsen.util 1383 :info :add 9787 :RequestTimeoutException INFO [2022-01-10 08:40:09,691] jepsen worker 711 - jepsen.util 3711 :info :add 9789 :RequestTimeoutException INFO [2022-01-10 08:40:09,692] jepsen worker 277 - jepsen.util 3277 :info :add 9788 :RequestTimeoutException INFO [2022-01-10 08:40:09,693] jepsen worker 317 - jepsen.util 3317 :info :add 9790 :RequestTimeoutException INFO [2022-01-10 08:40:09,694] jepsen worker 383 - jepsen.util 2383 :invoke :add 10837 INFO [2022-01-10 08:40:09,697] jepsen worker 277 - jepsen.util 4277 :invoke :add 10838 INFO [2022-01-10 08:40:09,698] jepsen worker 317 - jepsen.util 4317 :invoke :add 10839 INFO [2022-01-10 08:40:09,701] jepsen worker 711 - jepsen.util 4711 :invoke :add 10840 INFO [2022-01-10 08:40:09,722] jepsen worker 890 - jepsen.util 1890 :info :add 9791 :RequestTimeoutException INFO [2022-01-10 08:40:09,725] jepsen worker 890 - jepsen.util 2890 :invoke :add 10841 INFO [2022-01-10 08:40:09,727] jepsen worker 502 - jepsen.util 2502 :info :add 9792 :RequestTimeoutException INFO [2022-01-10 08:40:09,729] jepsen worker 502 - jepsen.util 3502 :invoke :add 10842 INFO [2022-01-10 08:40:09,731] jepsen worker 659 - jepsen.util 3659 :info :add 9793 :RequestTimeoutException INFO [2022-01-10 08:40:09,732] jepsen worker 848 - jepsen.util 2848 :info :add 9794 :RequestTimeoutException INFO [2022-01-10 08:40:09,733] jepsen worker 659 - jepsen.util 4659 :invoke :add 10843 INFO [2022-01-10 08:40:09,736] jepsen worker 848 - jepsen.util 3848 :invoke :add 10844 INFO [2022-01-10 08:40:09,741] jepsen worker 236 - jepsen.util 1236 :info :add 9795 :RequestTimeoutException INFO [2022-01-10 08:40:09,741] jepsen worker 299 - jepsen.util 3299 :info :add 9797 :RequestTimeoutException INFO [2022-01-10 08:40:09,743] jepsen worker 396 - jepsen.util 2396 :info :add 9796 :RequestTimeoutException INFO [2022-01-10 08:40:09,743] jepsen worker 236 - jepsen.util 2236 :invoke :add 10845 INFO [2022-01-10 08:40:09,746] jepsen worker 299 - jepsen.util 4299 :invoke :add 10846 INFO [2022-01-10 08:40:09,747] jepsen worker 396 - jepsen.util 3396 :invoke :add 10847 INFO [2022-01-10 08:40:09,753] jepsen worker 876 - jepsen.util 2876 :info :add 9800 :RequestTimeoutException INFO [2022-01-10 08:40:09,756] jepsen worker 876 - jepsen.util 3876 :invoke :add 10848 INFO [2022-01-10 08:40:09,767] jepsen worker 797 - jepsen.util 2797 :info :add 9804 :RequestTimeoutException INFO [2022-01-10 08:40:09,769] jepsen worker 797 - jepsen.util 3797 :invoke :add 10849 INFO [2022-01-10 08:40:09,770] jepsen worker 402 - jepsen.util 2402 :info :add 9805 :RequestTimeoutException INFO [2022-01-10 08:40:09,772] jepsen worker 718 - jepsen.util 2718 :info :add 9806 :RequestTimeoutException INFO [2022-01-10 08:40:09,773] jepsen worker 402 - jepsen.util 3402 :invoke :add 10850 INFO [2022-01-10 08:40:09,775] jepsen worker 718 - jepsen.util 3718 :invoke :add 10851 INFO [2022-01-10 08:40:09,781] jepsen worker 474 - jepsen.util 1474 :info :add 9807 :RequestTimeoutException INFO [2022-01-10 08:40:09,783] jepsen worker 203 - jepsen.util 1203 :info :add 9808 :RequestTimeoutException INFO [2022-01-10 08:40:09,784] jepsen worker 474 - jepsen.util 2474 :invoke :add 10852 INFO [2022-01-10 08:40:09,787] jepsen worker 203 - jepsen.util 2203 :invoke :add 10853 INFO [2022-01-10 08:40:09,792] jepsen worker 176 - jepsen.util 2176 :info :add 9809 :RequestTimeoutException INFO [2022-01-10 08:40:09,795] jepsen worker 176 - jepsen.util 3176 :invoke :add 10854 INFO [2022-01-10 08:40:09,797] jepsen worker 944 - jepsen.util 1944 :info :add 9810 :RequestTimeoutException INFO [2022-01-10 08:40:09,799] jepsen worker 944 - jepsen.util 2944 :invoke :add 10855 INFO [2022-01-10 08:40:09,800] jepsen worker 47 - jepsen.util 2047 :info :add 9811 :RequestTimeoutException INFO [2022-01-10 08:40:09,802] jepsen worker 306 - jepsen.util 2306 :info :add 9812 :RequestTimeoutException INFO [2022-01-10 08:40:09,803] jepsen worker 47 - jepsen.util 3047 :invoke :add 10856 INFO [2022-01-10 08:40:09,805] jepsen worker 306 - jepsen.util 3306 :invoke :add 10857 INFO [2022-01-10 08:40:09,811] jepsen worker 358 - jepsen.util 1358 :info :add 9813 :RequestTimeoutException INFO [2022-01-10 08:40:09,812] jepsen worker 130 - jepsen.util 3130 :info :add 9815 :RequestTimeoutException INFO [2022-01-10 08:40:09,813] jepsen worker 287 - jepsen.util 2287 :info :add 9814 :RequestTimeoutException INFO [2022-01-10 08:40:09,814] jepsen worker 130 - jepsen.util 4130 :invoke :add 10858 INFO [2022-01-10 08:40:09,816] jepsen worker 287 - jepsen.util 3287 :invoke :add 10859 INFO [2022-01-10 08:40:09,817] jepsen worker 532 - jepsen.util 2532 :info :add 9816 :RequestTimeoutException INFO [2022-01-10 08:40:09,819] jepsen worker 358 - jepsen.util 2358 :invoke :add 10860 INFO [2022-01-10 08:40:09,820] jepsen worker 705 - jepsen.util 1705 :info :add 9817 :RequestTimeoutException INFO [2022-01-10 08:40:09,821] jepsen worker 532 - jepsen.util 3532 :invoke :add 10861 INFO [2022-01-10 08:40:09,823] jepsen worker 865 - jepsen.util 2865 :info :add 9818 :RequestTimeoutException INFO [2022-01-10 08:40:09,824] jepsen worker 705 - jepsen.util 2705 :invoke :add 10862 INFO [2022-01-10 08:40:09,826] jepsen worker 865 - jepsen.util 3865 :invoke :add 10863 INFO [2022-01-10 08:40:09,833] jepsen worker 800 - jepsen.util 3800 :info :add 9820 :RequestTimeoutException INFO [2022-01-10 08:40:09,835] jepsen worker 800 - jepsen.util 4800 :invoke :add 10864 INFO [2022-01-10 08:40:09,842] jepsen worker 817 - jepsen.util 3817 :info :add 9821 :RequestTimeoutException INFO [2022-01-10 08:40:09,844] jepsen worker 817 - jepsen.util 4817 :invoke :add 10865 INFO [2022-01-10 08:40:09,847] jepsen worker 661 - jepsen.util 2661 :info :add 9822 :RequestTimeoutException INFO [2022-01-10 08:40:09,850] jepsen worker 598 - jepsen.util 2598 :info :add 9823 :RequestTimeoutException INFO [2022-01-10 08:40:09,850] jepsen worker 661 - jepsen.util 3661 :invoke :add 10866 INFO [2022-01-10 08:40:09,852] jepsen worker 709 - jepsen.util 2709 :info :add 9824 :RequestTimeoutException INFO [2022-01-10 08:40:09,852] jepsen worker 598 - jepsen.util 3598 :invoke :add 10867 INFO [2022-01-10 08:40:09,855] jepsen worker 709 - jepsen.util 3709 :invoke :add 10868 INFO [2022-01-10 08:40:09,861] jepsen worker 255 - jepsen.util 1255 :info :add 9825 :RequestTimeoutException INFO [2022-01-10 08:40:09,862] jepsen worker 239 - jepsen.util 3239 :info :add 9827 :RequestTimeoutException INFO [2022-01-10 08:40:09,863] jepsen worker 587 - jepsen.util 3587 :info :add 9826 :RequestTimeoutException INFO [2022-01-10 08:40:09,864] jepsen worker 239 - jepsen.util 4239 :invoke :add 10869 INFO [2022-01-10 08:40:09,867] jepsen worker 255 - jepsen.util 2255 :invoke :add 10870 INFO [2022-01-10 08:40:09,867] jepsen worker 727 - jepsen.util 1727 :info :add 9828 :RequestTimeoutException INFO [2022-01-10 08:40:09,868] jepsen worker 587 - jepsen.util 4587 :invoke :add 10871 INFO [2022-01-10 08:40:09,871] jepsen worker 727 - jepsen.util 2727 :invoke :add 10872 INFO [2022-01-10 08:40:09,881] jepsen worker 356 - jepsen.util 2356 :info :add 9829 :RequestTimeoutException INFO [2022-01-10 08:40:09,882] jepsen worker 806 - jepsen.util 3806 :info :add 9830 :RequestTimeoutException INFO [2022-01-10 08:40:09,883] jepsen worker 362 - jepsen.util 2362 :info :add 9831 :RequestTimeoutException INFO [2022-01-10 08:40:09,883] jepsen worker 356 - jepsen.util 3356 :invoke :add 10873 INFO [2022-01-10 08:40:09,887] jepsen worker 806 - jepsen.util 4806 :invoke :add 10874 INFO [2022-01-10 08:40:09,888] jepsen worker 362 - jepsen.util 3362 :invoke :add 10875 INFO [2022-01-10 08:40:09,891] jepsen worker 457 - jepsen.util 1457 :info :add 9832 :RequestTimeoutException INFO [2022-01-10 08:40:09,893] jepsen worker 475 - jepsen.util 3475 :info :add 9833 :RequestTimeoutException INFO [2022-01-10 08:40:09,895] jepsen worker 457 - jepsen.util 2457 :invoke :add 10876 INFO [2022-01-10 08:40:09,897] jepsen worker 475 - jepsen.util 4475 :invoke :add 10877 INFO [2022-01-10 08:40:09,903] jepsen worker 873 - jepsen.util 1873 :info :add 9835 :RequestTimeoutException INFO [2022-01-10 08:40:09,906] jepsen worker 873 - jepsen.util 2873 :invoke :add 10878 INFO [2022-01-10 08:40:09,907] jepsen worker 61 - jepsen.util 1061 :info :add 9836 :RequestTimeoutException INFO [2022-01-10 08:40:09,910] jepsen worker 416 - jepsen.util 3416 :info :add 9837 :RequestTimeoutException INFO [2022-01-10 08:40:09,910] jepsen worker 61 - jepsen.util 2061 :invoke :add 10879 INFO [2022-01-10 08:40:09,912] jepsen worker 416 - jepsen.util 4416 :invoke :add 10880 INFO [2022-01-10 08:40:09,913] jepsen worker 969 - jepsen.util 2969 :info :add 9838 :RequestTimeoutException INFO [2022-01-10 08:40:09,916] jepsen worker 969 - jepsen.util 3969 :invoke :add 10881 INFO [2022-01-10 08:40:09,921] jepsen worker 167 - jepsen.util 3167 :info :add 9839 :RequestTimeoutException INFO [2022-01-10 08:40:09,923] jepsen worker 887 - jepsen.util 2887 :info :add 9840 :RequestTimeoutException INFO [2022-01-10 08:40:09,925] jepsen worker 167 - jepsen.util 4167 :invoke :add 10882 INFO [2022-01-10 08:40:09,927] jepsen worker 202 - jepsen.util 1202 :info :add 9842 :RequestTimeoutException INFO [2022-01-10 08:40:09,927] jepsen worker 887 - jepsen.util 3887 :invoke :add 10883 INFO [2022-01-10 08:40:09,930] jepsen worker 202 - jepsen.util 2202 :invoke :add 10884 INFO [2022-01-10 08:40:09,931] jepsen worker 75 - jepsen.util 2075 :info :add 9841 :RequestTimeoutException INFO [2022-01-10 08:40:09,934] jepsen worker 75 - jepsen.util 3075 :invoke :add 10885 INFO [2022-01-10 08:40:09,940] jepsen worker 570 - jepsen.util 2570 :info :add 9843 :RequestTimeoutException INFO [2022-01-10 08:40:09,942] jepsen worker 353 - jepsen.util 2353 :info :add 9844 :RequestTimeoutException INFO [2022-01-10 08:40:09,942] jepsen worker 570 - jepsen.util 3570 :invoke :add 10886 INFO [2022-01-10 08:40:09,944] jepsen worker 353 - jepsen.util 3353 :invoke :add 10887 INFO [2022-01-10 08:40:09,956] jepsen worker 660 - jepsen.util 3660 :info :add 9848 :RequestTimeoutException INFO [2022-01-10 08:40:09,959] jepsen worker 660 - jepsen.util 4660 :invoke :add 10888 INFO [2022-01-10 08:40:09,970] jepsen worker 360 - jepsen.util 1360 :info :add 9849 :RequestTimeoutException INFO [2022-01-10 08:40:09,972] jepsen worker 360 - jepsen.util 2360 :invoke :add 10889 INFO [2022-01-10 08:40:09,973] jepsen worker 775 - jepsen.util 3775 :info :add 9850 :RequestTimeoutException INFO [2022-01-10 08:40:09,976] jepsen worker 775 - jepsen.util 4775 :invoke :add 10890 INFO [2022-01-10 08:40:09,981] jepsen worker 388 - jepsen.util 2388 :info :add 9851 :RequestTimeoutException INFO [2022-01-10 08:40:09,983] jepsen worker 388 - jepsen.util 3388 :invoke :add 10891 INFO [2022-01-10 08:40:09,991] jepsen worker 515 - jepsen.util 3515 :info :add 9852 :RequestTimeoutException INFO [2022-01-10 08:40:09,992] jepsen worker 177 - jepsen.util 1177 :info :add 9854 :RequestTimeoutException INFO [2022-01-10 08:40:09,994] jepsen worker 133 - jepsen.util 3133 :info :add 9853 :RequestTimeoutException INFO [2022-01-10 08:40:09,997] jepsen worker 177 - jepsen.util 2177 :invoke :add 10892 INFO [2022-01-10 08:40:09,997] jepsen worker 950 - jepsen.util 1950 :info :add 9855 :RequestTimeoutException INFO [2022-01-10 08:40:09,998] jepsen worker 133 - jepsen.util 4133 :invoke :add 10893 INFO [2022-01-10 08:40:10,000] jepsen worker 911 - jepsen.util 2911 :info :add 9856 :RequestTimeoutException INFO [2022-01-10 08:40:10,001] jepsen worker 950 - jepsen.util 2950 :invoke :add 10894 INFO [2022-01-10 08:40:10,003] jepsen worker 683 - jepsen.util 1683 :info :add 9857 :RequestTimeoutException INFO [2022-01-10 08:40:10,004] jepsen worker 911 - jepsen.util 3911 :invoke :add 10895 INFO [2022-01-10 08:40:10,006] jepsen worker 683 - jepsen.util 2683 :invoke :add 10896 INFO [2022-01-10 08:40:10,009] jepsen worker 515 - jepsen.util 4515 :invoke :add 10897 INFO [2022-01-10 08:40:10,011] jepsen worker 183 - jepsen.util 2183 :info :add 9858 :RequestTimeoutException INFO [2022-01-10 08:40:10,013] jepsen worker 237 - jepsen.util 1237 :info :add 9859 :RequestTimeoutException INFO [2022-01-10 08:40:10,016] jepsen worker 183 - jepsen.util 3183 :invoke :add 10898 INFO [2022-01-10 08:40:10,019] jepsen worker 237 - jepsen.util 2237 :invoke :add 10899 INFO [2022-01-10 08:40:10,022] jepsen worker 833 - jepsen.util 2833 :info :add 9860 :RequestTimeoutException INFO [2022-01-10 08:40:10,026] jepsen worker 833 - jepsen.util 3833 :invoke :add 10900 INFO [2022-01-10 08:40:10,027] jepsen worker 808 - jepsen.util 2808 :info :add 9861 :RequestTimeoutException INFO [2022-01-10 08:40:10,029] jepsen worker 540 - jepsen.util 3540 :info :add 9862 :RequestTimeoutException INFO [2022-01-10 08:40:10,030] jepsen worker 808 - jepsen.util 3808 :invoke :add 10901 INFO [2022-01-10 08:40:10,033] jepsen worker 540 - jepsen.util 4540 :invoke :add 10902 INFO [2022-01-10 08:40:10,033] jepsen worker 517 - jepsen.util 3517 :info :add 9863 :RequestTimeoutException INFO [2022-01-10 08:40:10,037] jepsen worker 517 - jepsen.util 4517 :invoke :add 10903 INFO [2022-01-10 08:40:10,041] jepsen worker 534 - jepsen.util 1534 :info :add 9864 :RequestTimeoutException INFO [2022-01-10 08:40:10,043] jepsen worker 54 - jepsen.util 1054 :info :add 9865 :RequestTimeoutException INFO [2022-01-10 08:40:10,044] jepsen worker 534 - jepsen.util 2534 :invoke :add 10904 INFO [2022-01-10 08:40:10,046] jepsen worker 54 - jepsen.util 2054 :invoke :add 10905 INFO [2022-01-10 08:40:10,052] jepsen worker 963 - jepsen.util 1963 :info :add 9866 :RequestTimeoutException INFO [2022-01-10 08:40:10,055] jepsen worker 963 - jepsen.util 2963 :invoke :add 10906 INFO [2022-01-10 08:40:10,057] jepsen worker 165 - jepsen.util 3165 :info :add 9867 :RequestTimeoutException INFO [2022-01-10 08:40:10,059] jepsen worker 165 - jepsen.util 4165 :invoke :add 10907 INFO [2022-01-10 08:40:10,073] jepsen worker 829 - jepsen.util 3829 :info :add 9869 :RequestTimeoutException INFO [2022-01-10 08:40:10,076] jepsen worker 829 - jepsen.util 4829 :invoke :add 10908 INFO [2022-01-10 08:40:10,082] jepsen worker 119 - jepsen.util 119 :info :add 9871 :RequestTimeoutException INFO [2022-01-10 08:40:10,084] jepsen worker 119 - jepsen.util 1119 :invoke :add 10909 INFO [2022-01-10 08:40:10,096] jepsen worker 390 - jepsen.util 1390 :info :add 9874 :RequestTimeoutException INFO [2022-01-10 08:40:10,099] jepsen worker 390 - jepsen.util 2390 :invoke :add 10910 INFO [2022-01-10 08:40:10,100] jepsen worker 754 - jepsen.util 2754 :info :add 9875 :RequestTimeoutException INFO [2022-01-10 08:40:10,103] jepsen worker 163 - jepsen.util 3163 :info :add 9876 :RequestTimeoutException INFO [2022-01-10 08:40:10,103] jepsen worker 754 - jepsen.util 3754 :invoke :add 10911 INFO [2022-01-10 08:40:10,106] jepsen worker 163 - jepsen.util 4163 :invoke :add 10912 INFO [2022-01-10 08:40:10,111] jepsen worker 584 - jepsen.util 2584 :info :add 9877 :RequestTimeoutException INFO [2022-01-10 08:40:10,113] jepsen worker 422 - jepsen.util 1422 :info :add 9878 :RequestTimeoutException INFO [2022-01-10 08:40:10,114] jepsen worker 584 - jepsen.util 3584 :invoke :add 10913 INFO [2022-01-10 08:40:10,117] jepsen worker 422 - jepsen.util 2422 :invoke :add 10914 INFO [2022-01-10 08:40:10,122] jepsen worker 146 - jepsen.util 1146 :info :add 9879 :RequestTimeoutException INFO [2022-01-10 08:40:10,125] jepsen worker 146 - jepsen.util 2146 :invoke :add 10915 INFO [2022-01-10 08:40:10,127] jepsen worker 519 - jepsen.util 2519 :info :add 9880 :RequestTimeoutException INFO [2022-01-10 08:40:10,129] jepsen worker 519 - jepsen.util 3519 :invoke :add 10916 INFO [2022-01-10 08:40:10,130] jepsen worker 295 - jepsen.util 2295 :info :add 9881 :RequestTimeoutException INFO [2022-01-10 08:40:10,133] jepsen worker 23 - jepsen.util 2023 :info :add 9882 :RequestTimeoutException INFO [2022-01-10 08:40:10,133] jepsen worker 295 - jepsen.util 3295 :invoke :add 10917 INFO [2022-01-10 08:40:10,136] jepsen worker 23 - jepsen.util 3023 :invoke :add 10918 INFO [2022-01-10 08:40:10,141] jepsen worker 377 - jepsen.util 2377 :info :add 9883 :RequestTimeoutException INFO [2022-01-10 08:40:10,142] jepsen worker 431 - jepsen.util 2431 :info :add 9885 :RequestTimeoutException INFO [2022-01-10 08:40:10,143] jepsen worker 377 - jepsen.util 3377 :invoke :add 10919 INFO [2022-01-10 08:40:10,143] jepsen worker 668 - jepsen.util 2668 :info :add 9884 :RequestTimeoutException INFO [2022-01-10 08:40:10,146] jepsen worker 668 - jepsen.util 3668 :invoke :add 10920 INFO [2022-01-10 08:40:10,147] jepsen worker 906 - jepsen.util 3906 :info :add 9886 :RequestTimeoutException INFO [2022-01-10 08:40:10,148] jepsen worker 431 - jepsen.util 3431 :invoke :add 10921 INFO [2022-01-10 08:40:10,151] jepsen worker 906 - jepsen.util 4906 :invoke :add 10922 INFO [2022-01-10 08:40:10,161] jepsen worker 3 - jepsen.util 2003 :info :add 9887 :RequestTimeoutException INFO [2022-01-10 08:40:10,162] jepsen worker 941 - jepsen.util 3941 :info :add 9888 :RequestTimeoutException INFO [2022-01-10 08:40:10,164] jepsen worker 3 - jepsen.util 3003 :invoke :add 10923 INFO [2022-01-10 08:40:10,166] jepsen worker 941 - jepsen.util 4941 :invoke :add 10924 INFO [2022-01-10 08:40:10,201] jepsen worker 59 - jepsen.util 2059 :info :add 9895 :RequestTimeoutException INFO [2022-01-10 08:40:10,203] jepsen worker 999 - jepsen.util 1999 :info :add 9894 :RequestTimeoutException INFO [2022-01-10 08:40:10,203] jepsen worker 59 - jepsen.util 3059 :invoke :add 10925 INFO [2022-01-10 08:40:10,204] jepsen worker 634 - jepsen.util 2634 :info :add 9896 :RequestTimeoutException INFO [2022-01-10 08:40:10,207] jepsen worker 634 - jepsen.util 3634 :invoke :add 10926 INFO [2022-01-10 08:40:10,209] jepsen worker 999 - jepsen.util 2999 :invoke :add 10927 INFO [2022-01-10 08:40:10,212] jepsen worker 966 - jepsen.util 3966 :info :add 9897 :RequestTimeoutException INFO [2022-01-10 08:40:10,215] jepsen worker 966 - jepsen.util 4966 :invoke :add 10928 INFO [2022-01-10 08:40:10,217] jepsen worker 162 - jepsen.util 3162 :info :add 9898 :RequestTimeoutException INFO [2022-01-10 08:40:10,219] jepsen worker 162 - jepsen.util 4162 :invoke :add 10929 INFO [2022-01-10 08:40:10,220] jepsen worker 152 - jepsen.util 2152 :info :add 9899 :RequestTimeoutException INFO [2022-01-10 08:40:10,222] jepsen worker 352 - jepsen.util 3352 :info :add 9900 :RequestTimeoutException INFO [2022-01-10 08:40:10,223] jepsen worker 152 - jepsen.util 3152 :invoke :add 10930 INFO [2022-01-10 08:40:10,225] jepsen worker 352 - jepsen.util 4352 :invoke :add 10931 INFO [2022-01-10 08:40:10,246] jepsen worker 886 - jepsen.util 3886 :info :add 9904 :RequestTimeoutException INFO [2022-01-10 08:40:10,249] jepsen worker 886 - jepsen.util 4886 :invoke :add 10932 INFO [2022-01-10 08:40:10,250] jepsen worker 805 - jepsen.util 2805 :info :add 9905 :RequestTimeoutException INFO [2022-01-10 08:40:10,253] jepsen worker 805 - jepsen.util 3805 :invoke :add 10933 INFO [2022-01-10 08:40:10,263] jepsen worker 280 - jepsen.util 2280 :info :add 9906 :RequestTimeoutException INFO [2022-01-10 08:40:10,266] jepsen worker 280 - jepsen.util 3280 :invoke :add 10934 INFO [2022-01-10 08:40:10,271] jepsen worker 115 - jepsen.util 1115 :info :add 9907 :RequestTimeoutException INFO [2022-01-10 08:40:10,274] jepsen worker 582 - jepsen.util 2582 :info :add 9908 :RequestTimeoutException INFO [2022-01-10 08:40:10,274] jepsen worker 115 - jepsen.util 2115 :invoke :add 10935 INFO [2022-01-10 08:40:10,277] jepsen worker 582 - jepsen.util 3582 :invoke :add 10936 INFO [2022-01-10 08:40:10,277] jepsen worker 373 - jepsen.util 2373 :info :add 9910 :RequestTimeoutException INFO [2022-01-10 08:40:10,280] jepsen worker 415 - jepsen.util 2415 :info :add 9911 :RequestTimeoutException INFO [2022-01-10 08:40:10,280] jepsen worker 373 - jepsen.util 3373 :invoke :add 10937 INFO [2022-01-10 08:40:10,282] jepsen worker 823 - jepsen.util 3823 :info :add 9909 :RequestTimeoutException INFO [2022-01-10 08:40:10,282] jepsen worker 895 - jepsen.util 2895 :info :add 9912 :RequestTimeoutException INFO [2022-01-10 08:40:10,282] jepsen worker 415 - jepsen.util 3415 :invoke :add 10938 INFO [2022-01-10 08:40:10,285] jepsen worker 823 - jepsen.util 4823 :invoke :add 10939 INFO [2022-01-10 08:40:10,287] jepsen worker 895 - jepsen.util 3895 :invoke :add 10940 INFO [2022-01-10 08:40:10,291] jepsen worker 940 - jepsen.util 2940 :info :add 9913 :RequestTimeoutException INFO [2022-01-10 08:40:10,293] jepsen worker 39 - jepsen.util 2039 :info :add 9914 :RequestTimeoutException INFO [2022-01-10 08:40:10,293] jepsen worker 940 - jepsen.util 3940 :invoke :add 10941 INFO [2022-01-10 08:40:10,295] jepsen worker 39 - jepsen.util 3039 :invoke :add 10942 INFO [2022-01-10 08:40:10,302] jepsen worker 616 - jepsen.util 2616 :info :add 9915 :RequestTimeoutException INFO [2022-01-10 08:40:10,305] jepsen worker 616 - jepsen.util 3616 :invoke :add 10943 INFO [2022-01-10 08:40:10,306] jepsen worker 409 - jepsen.util 2409 :info :add 9916 :RequestTimeoutException INFO [2022-01-10 08:40:10,309] jepsen worker 409 - jepsen.util 3409 :invoke :add 10944 INFO [2022-01-10 08:40:10,310] jepsen worker 558 - jepsen.util 3558 :info :add 9917 :RequestTimeoutException INFO [2022-01-10 08:40:10,313] jepsen worker 558 - jepsen.util 4558 :invoke :add 10945 INFO [2022-01-10 08:40:10,323] jepsen worker 786 - jepsen.util 3786 :info :add 9920 :RequestTimeoutException INFO [2022-01-10 08:40:10,323] jepsen worker 319 - jepsen.util 2319 :info :add 9918 :RequestTimeoutException INFO [2022-01-10 08:40:10,326] jepsen worker 319 - jepsen.util 3319 :invoke :add 10946 INFO [2022-01-10 08:40:10,327] jepsen worker 786 - jepsen.util 4786 :invoke :add 10947 INFO [2022-01-10 08:40:10,332] jepsen worker 975 - jepsen.util 2975 :info :add 9921 :RequestTimeoutException INFO [2022-01-10 08:40:10,334] jepsen worker 975 - jepsen.util 3975 :invoke :add 10948 INFO [2022-01-10 08:40:10,336] jepsen worker 704 - jepsen.util 3704 :ok :add 10364 INFO [2022-01-10 08:40:10,337] jepsen worker 589 - jepsen.util 3589 :info :add 9922 :RequestTimeoutException INFO [2022-01-10 08:40:10,339] jepsen worker 704 - jepsen.util 3704 :invoke :add 10949 INFO [2022-01-10 08:40:10,340] jepsen worker 599 - jepsen.util 3599 :info :add 9923 :RequestTimeoutException INFO [2022-01-10 08:40:10,341] jepsen worker 29 - jepsen.util 3029 :info :add 9925 :RequestTimeoutException INFO [2022-01-10 08:40:10,341] jepsen worker 589 - jepsen.util 4589 :invoke :add 10950 INFO [2022-01-10 08:40:10,342] jepsen worker 389 - jepsen.util 1389 :info :add 9927 :RequestTimeoutException INFO [2022-01-10 08:40:10,343] jepsen worker 250 - jepsen.util 2250 :info :add 9924 :RequestTimeoutException INFO [2022-01-10 08:40:10,343] jepsen worker 21 - jepsen.util 1021 :info :add 9926 :RequestTimeoutException INFO [2022-01-10 08:40:10,346] jepsen worker 250 - jepsen.util 3250 :invoke :add 10951 INFO [2022-01-10 08:40:10,347] jepsen worker 934 - jepsen.util 2934 :info :add 9928 :RequestTimeoutException INFO [2022-01-10 08:40:10,348] jepsen worker 599 - jepsen.util 4599 :invoke :add 10952 INFO [2022-01-10 08:40:10,350] jepsen worker 934 - jepsen.util 3934 :invoke :add 10953 INFO [2022-01-10 08:40:10,350] jepsen worker 189 - jepsen.util 3189 :info :add 9929 :RequestTimeoutException INFO [2022-01-10 08:40:10,351] jepsen worker 29 - jepsen.util 4029 :invoke :add 10954 INFO [2022-01-10 08:40:10,353] jepsen worker 707 - jepsen.util 1707 :info :add 9930 :RequestTimeoutException INFO [2022-01-10 08:40:10,354] jepsen worker 189 - jepsen.util 4189 :invoke :add 10955 INFO [2022-01-10 08:40:10,356] jepsen worker 707 - jepsen.util 2707 :invoke :add 10956 INFO [2022-01-10 08:40:10,358] jepsen worker 21 - jepsen.util 2021 :invoke :add 10957 INFO [2022-01-10 08:40:10,359] jepsen worker 389 - jepsen.util 2389 :invoke :add 10958 INFO [2022-01-10 08:40:10,361] jepsen worker 664 - jepsen.util 2664 :info :add 9931 :RequestTimeoutException INFO [2022-01-10 08:40:10,362] jepsen worker 628 - jepsen.util 3628 :info :add 9933 :RequestTimeoutException INFO [2022-01-10 08:40:10,364] jepsen worker 988 - jepsen.util 3988 :info :add 9932 :RequestTimeoutException INFO [2022-01-10 08:40:10,365] jepsen worker 664 - jepsen.util 3664 :invoke :add 10959 INFO [2022-01-10 08:40:10,367] jepsen worker 794 - jepsen.util 2794 :info :add 9934 :RequestTimeoutException INFO [2022-01-10 08:40:10,368] jepsen worker 628 - jepsen.util 4628 :invoke :add 10960 INFO [2022-01-10 08:40:10,370] jepsen worker 988 - jepsen.util 4988 :invoke :add 10961 INFO [2022-01-10 08:40:10,370] jepsen worker 777 - jepsen.util 1777 :info :add 9935 :RequestTimeoutException INFO [2022-01-10 08:40:10,372] jepsen worker 794 - jepsen.util 3794 :invoke :add 10962 INFO [2022-01-10 08:40:10,373] jepsen worker 350 - jepsen.util 3350 :info :add 9936 :RequestTimeoutException INFO [2022-01-10 08:40:10,374] jepsen worker 777 - jepsen.util 2777 :invoke :add 10963 INFO [2022-01-10 08:40:10,376] jepsen worker 350 - jepsen.util 4350 :invoke :add 10964 INFO [2022-01-10 08:40:10,382] jepsen worker 864 - jepsen.util 3864 :info :add 9939 :RequestTimeoutException INFO [2022-01-10 08:40:10,383] jepsen worker 109 - jepsen.util 2109 :info :add 9938 :RequestTimeoutException INFO [2022-01-10 08:40:10,386] jepsen worker 109 - jepsen.util 3109 :invoke :add 10965 INFO [2022-01-10 08:40:10,387] jepsen worker 908 - jepsen.util 3908 :info :add 9940 :RequestTimeoutException INFO [2022-01-10 08:40:10,388] jepsen worker 864 - jepsen.util 4864 :invoke :add 10966 INFO [2022-01-10 08:40:10,390] jepsen worker 311 - jepsen.util 3311 :info :add 9941 :RequestTimeoutException INFO [2022-01-10 08:40:10,391] jepsen worker 908 - jepsen.util 4908 :invoke :add 10967 INFO [2022-01-10 08:40:10,392] jepsen worker 187 - jepsen.util 3187 :info :add 9942 :RequestTimeoutException INFO [2022-01-10 08:40:10,393] jepsen worker 311 - jepsen.util 4311 :invoke :add 10968 INFO [2022-01-10 08:40:10,396] jepsen worker 187 - jepsen.util 4187 :invoke :add 10969 INFO [2022-01-10 08:40:10,401] jepsen worker 653 - jepsen.util 3653 :info :add 9943 :RequestTimeoutException INFO [2022-01-10 08:40:10,404] jepsen worker 653 - jepsen.util 4653 :invoke :add 10970 INFO [2022-01-10 08:40:10,407] jepsen worker 719 - jepsen.util 2719 :info :add 9946 :RequestTimeoutException INFO [2022-01-10 08:40:10,409] jepsen worker 719 - jepsen.util 3719 :invoke :add 10971 INFO [2022-01-10 08:40:10,410] jepsen worker 982 - jepsen.util 2982 :info :add 9947 :RequestTimeoutException INFO [2022-01-10 08:40:10,411] jepsen worker 781 - jepsen.util 2781 :info :add 9949 :RequestTimeoutException INFO [2022-01-10 08:40:10,413] jepsen worker 982 - jepsen.util 3982 :invoke :add 10972 INFO [2022-01-10 08:40:10,413] jepsen worker 602 - jepsen.util 3602 :info :add 9950 :RequestTimeoutException INFO [2022-01-10 08:40:10,413] jepsen worker 962 - jepsen.util 3962 :info :add 9948 :RequestTimeoutException INFO [2022-01-10 08:40:10,416] jepsen worker 602 - jepsen.util 4602 :invoke :add 10973 INFO [2022-01-10 08:40:10,418] jepsen worker 781 - jepsen.util 3781 :invoke :add 10974 INFO [2022-01-10 08:40:10,420] jepsen worker 962 - jepsen.util 4962 :invoke :add 10975 INFO [2022-01-10 08:40:10,422] jepsen worker 690 - jepsen.util 1690 :info :add 9951 :RequestTimeoutException INFO [2022-01-10 08:40:10,424] jepsen worker 690 - jepsen.util 2690 :invoke :add 10976 INFO [2022-01-10 08:40:10,427] jepsen worker 323 - jepsen.util 3323 :info :add 9952 :RequestTimeoutException INFO [2022-01-10 08:40:10,429] jepsen worker 89 - jepsen.util 3089 :info :add 9953 :RequestTimeoutException INFO [2022-01-10 08:40:10,431] jepsen worker 657 - jepsen.util 3657 :info :add 9955 :RequestTimeoutException INFO [2022-01-10 08:40:10,431] jepsen worker 323 - jepsen.util 4323 :invoke :add 10977 INFO [2022-01-10 08:40:10,432] jepsen worker 164 - jepsen.util 1164 :info :add 9954 :RequestTimeoutException INFO [2022-01-10 08:40:10,433] jepsen worker 702 - jepsen.util 1702 :info :add 9956 :RequestTimeoutException INFO [2022-01-10 08:40:10,434] jepsen worker 657 - jepsen.util 4657 :invoke :add 10978 INFO [2022-01-10 08:40:10,436] jepsen worker 164 - jepsen.util 2164 :invoke :add 10979 INFO [2022-01-10 08:40:10,438] jepsen worker 89 - jepsen.util 4089 :invoke :add 10980 INFO [2022-01-10 08:40:10,440] jepsen worker 702 - jepsen.util 2702 :invoke :add 10981 INFO [2022-01-10 08:40:10,442] jepsen worker 58 - jepsen.util 3058 :info :add 9957 :RequestTimeoutException INFO [2022-01-10 08:40:10,445] jepsen worker 58 - jepsen.util 4058 :invoke :add 10982 INFO [2022-01-10 08:40:10,446] jepsen worker 331 - jepsen.util 2331 :info :add 9958 :RequestTimeoutException INFO [2022-01-10 08:40:10,449] jepsen worker 331 - jepsen.util 3331 :invoke :add 10983 INFO [2022-01-10 08:40:10,450] jepsen worker 699 - jepsen.util 3699 :info :add 9959 :RequestTimeoutException INFO [2022-01-10 08:40:10,451] jepsen worker 935 - jepsen.util 3935 :info :add 9961 :RequestTimeoutException INFO [2022-01-10 08:40:10,451] jepsen worker 494 - jepsen.util 1494 :info :add 9963 :RequestTimeoutException INFO [2022-01-10 08:40:10,452] jepsen worker 979 - jepsen.util 2979 :info :add 9960 :RequestTimeoutException INFO [2022-01-10 08:40:10,453] jepsen worker 689 - jepsen.util 3689 :info :add 9962 :RequestTimeoutException INFO [2022-01-10 08:40:10,456] jepsen worker 494 - jepsen.util 2494 :invoke :add 10984 INFO [2022-01-10 08:40:10,457] jepsen worker 461 - jepsen.util 2461 :info :add 9964 :RequestTimeoutException INFO [2022-01-10 08:40:10,458] jepsen worker 935 - jepsen.util 4935 :invoke :add 10985 INFO [2022-01-10 08:40:10,460] jepsen worker 818 - jepsen.util 3818 :info :add 9965 :RequestTimeoutException INFO [2022-01-10 08:40:10,460] jepsen worker 979 - jepsen.util 3979 :invoke :add 10986 INFO [2022-01-10 08:40:10,463] jepsen worker 883 - jepsen.util 3883 :info :add 9966 :RequestTimeoutException INFO [2022-01-10 08:40:10,463] jepsen worker 699 - jepsen.util 4699 :invoke :add 10987 INFO [2022-01-10 08:40:10,466] jepsen worker 461 - jepsen.util 3461 :invoke :add 10988 INFO [2022-01-10 08:40:10,468] jepsen worker 689 - jepsen.util 4689 :invoke :add 10989 INFO [2022-01-10 08:40:10,469] jepsen worker 883 - jepsen.util 4883 :invoke :add 10990 INFO [2022-01-10 08:40:10,471] jepsen worker 743 - jepsen.util 3743 :info :add 9967 :RequestTimeoutException INFO [2022-01-10 08:40:10,471] jepsen worker 818 - jepsen.util 4818 :invoke :add 10991 INFO [2022-01-10 08:40:10,473] jepsen worker 743 - jepsen.util 4743 :invoke :add 10992 INFO [2022-01-10 08:40:10,477] jepsen worker 483 - jepsen.util 2483 :info :add 9970 :RequestTimeoutException INFO [2022-01-10 08:40:10,480] jepsen worker 566 - jepsen.util 2566 :info :add 9971 :RequestTimeoutException INFO [2022-01-10 08:40:10,481] jepsen worker 483 - jepsen.util 3483 :invoke :add 10993 INFO [2022-01-10 08:40:10,483] jepsen worker 714 - jepsen.util 3714 :info :add 9972 :RequestTimeoutException INFO [2022-01-10 08:40:10,484] jepsen worker 566 - jepsen.util 3566 :invoke :add 10994 INFO [2022-01-10 08:40:10,487] jepsen worker 714 - jepsen.util 4714 :invoke :add 10995 INFO [2022-01-10 08:40:10,491] jepsen worker 79 - jepsen.util 2079 :info :add 9974 :RequestTimeoutException INFO [2022-01-10 08:40:10,492] jepsen worker 751 - jepsen.util 3751 :info :add 9976 :RequestTimeoutException INFO [2022-01-10 08:40:10,494] jepsen worker 178 - jepsen.util 2178 :info :add 9975 :RequestTimeoutException INFO [2022-01-10 08:40:10,495] jepsen worker 751 - jepsen.util 4751 :invoke :add 10996 INFO [2022-01-10 08:40:10,496] jepsen worker 713 - jepsen.util 3713 :info :add 9978 :RequestTimeoutException INFO [2022-01-10 08:40:10,499] jepsen worker 713 - jepsen.util 4713 :invoke :add 10997 INFO [2022-01-10 08:40:10,500] jepsen worker 241 - jepsen.util 3241 :info :add 9979 :RequestTimeoutException INFO [2022-01-10 08:40:10,500] jepsen worker 301 - jepsen.util 2301 :info :add 9977 :RequestTimeoutException INFO [2022-01-10 08:40:10,501] jepsen worker 178 - jepsen.util 3178 :invoke :add 10998 INFO [2022-01-10 08:40:10,503] jepsen worker 703 - jepsen.util 3703 :info :add 9980 :RequestTimeoutException INFO [2022-01-10 08:40:10,505] jepsen worker 301 - jepsen.util 3301 :invoke :add 10999 INFO [2022-01-10 08:40:10,508] jepsen worker 79 - jepsen.util 3079 :invoke :add 11000 INFO [2022-01-10 08:40:10,509] jepsen worker 241 - jepsen.util 4241 :invoke :add 11001 INFO [2022-01-10 08:40:10,511] jepsen worker 314 - jepsen.util 3314 :info :add 9981 :RequestTimeoutException INFO [2022-01-10 08:40:10,511] jepsen worker 392 - jepsen.util 3392 :info :add 9983 :RequestTimeoutException INFO [2022-01-10 08:40:10,511] jepsen worker 703 - jepsen.util 4703 :invoke :add 11002 INFO [2022-01-10 08:40:10,513] jepsen worker 915 - jepsen.util 1915 :info :add 9982 :RequestTimeoutException INFO [2022-01-10 08:40:10,515] jepsen worker 314 - jepsen.util 4314 :invoke :add 11003 INFO [2022-01-10 08:40:10,517] jepsen worker 69 - jepsen.util 2069 :info :add 9985 :RequestTimeoutException INFO [2022-01-10 08:40:10,517] jepsen worker 392 - jepsen.util 4392 :invoke :add 11004 INFO [2022-01-10 08:40:10,519] jepsen worker 69 - jepsen.util 3069 :invoke :add 11005 INFO [2022-01-10 08:40:10,520] jepsen worker 726 - jepsen.util 3726 :info :add 9986 :RequestTimeoutException INFO [2022-01-10 08:40:10,521] jepsen worker 915 - jepsen.util 2915 :invoke :add 11006 INFO [2022-01-10 08:40:10,522] jepsen worker 180 - jepsen.util 3180 :info :add 9984 :RequestTimeoutException INFO [2022-01-10 08:40:10,523] jepsen worker 726 - jepsen.util 4726 :invoke :add 11007 INFO [2022-01-10 08:40:10,523] jepsen worker 122 - jepsen.util 3122 :info :add 9987 :RequestTimeoutException INFO [2022-01-10 08:40:10,526] jepsen worker 122 - jepsen.util 4122 :invoke :add 11008 INFO [2022-01-10 08:40:10,528] jepsen worker 180 - jepsen.util 4180 :invoke :add 11009 INFO [2022-01-10 08:40:10,536] jepsen worker 492 - jepsen.util 2492 :info :add 9991 :RequestTimeoutException INFO [2022-01-10 08:40:10,539] jepsen worker 492 - jepsen.util 3492 :invoke :add 11010 INFO [2022-01-10 08:40:10,540] jepsen worker 766 - jepsen.util 3766 :info :add 9992 :RequestTimeoutException INFO [2022-01-10 08:40:10,543] jepsen worker 766 - jepsen.util 4766 :invoke :add 11011 INFO [2022-01-10 08:40:10,551] jepsen worker 522 - jepsen.util 3522 :info :add 9995 :RequestTimeoutException INFO [2022-01-10 08:40:10,552] jepsen worker 354 - jepsen.util 2354 :info :add 9997 :RequestTimeoutException INFO [2022-01-10 08:40:10,553] jepsen worker 93 - jepsen.util 3093 :info :add 9994 :RequestTimeoutException INFO [2022-01-10 08:40:10,553] jepsen worker 642 - jepsen.util 3642 :info :add 9996 :RequestTimeoutException INFO [2022-01-10 08:40:10,553] jepsen worker 522 - jepsen.util 4522 :invoke :add 11012 INFO [2022-01-10 08:40:10,557] jepsen worker 594 - jepsen.util 1594 :info :add 9998 :RequestTimeoutException INFO [2022-01-10 08:40:10,557] jepsen worker 354 - jepsen.util 3354 :invoke :add 11013 INFO [2022-01-10 08:40:10,559] jepsen worker 93 - jepsen.util 4093 :invoke :add 11014 INFO [2022-01-10 08:40:10,559] jepsen worker 470 - jepsen.util 2470 :info :add 9999 :RequestTimeoutException INFO [2022-01-10 08:40:10,561] jepsen worker 594 - jepsen.util 2594 :invoke :add 11015 INFO [2022-01-10 08:40:10,563] jepsen worker 158 - jepsen.util 2158 :info :add 10000 :RequestTimeoutException INFO [2022-01-10 08:40:10,563] jepsen worker 642 - jepsen.util 4642 :invoke :add 11016 INFO [2022-01-10 08:40:10,566] jepsen worker 470 - jepsen.util 3470 :invoke :add 11017 INFO [2022-01-10 08:40:10,568] jepsen worker 158 - jepsen.util 3158 :invoke :add 11018 INFO [2022-01-10 08:40:10,571] jepsen worker 745 - jepsen.util 2745 :info :add 10001 :RequestTimeoutException INFO [2022-01-10 08:40:10,572] jepsen worker 497 - jepsen.util 2497 :info :add 10003 :RequestTimeoutException INFO [2022-01-10 08:40:10,573] jepsen worker 498 - jepsen.util 2498 :info :add 10002 :RequestTimeoutException INFO [2022-01-10 08:40:10,574] jepsen worker 497 - jepsen.util 3497 :invoke :add 11019 INFO [2022-01-10 08:40:10,576] jepsen worker 745 - jepsen.util 3745 :invoke :add 11020 INFO [2022-01-10 08:40:10,577] jepsen worker 825 - jepsen.util 2825 :info :add 10004 :RequestTimeoutException INFO [2022-01-10 08:40:10,578] jepsen worker 498 - jepsen.util 3498 :invoke :add 11021 INFO [2022-01-10 08:40:10,580] jepsen worker 420 - jepsen.util 2420 :info :add 10005 :RequestTimeoutException INFO [2022-01-10 08:40:10,580] jepsen worker 825 - jepsen.util 3825 :invoke :add 11022 INFO [2022-01-10 08:40:10,583] jepsen worker 420 - jepsen.util 3420 :invoke :add 11023 INFO [2022-01-10 08:40:10,591] jepsen worker 6 - jepsen.util 3006 :info :add 10007 :RequestTimeoutException INFO [2022-01-10 08:40:10,592] jepsen worker 418 - jepsen.util 2418 :info :add 10009 :RequestTimeoutException INFO [2022-01-10 08:40:10,593] jepsen worker 607 - jepsen.util 1607 :info :add 10006 :RequestTimeoutException INFO [2022-01-10 08:40:10,593] jepsen worker 903 - jepsen.util 3903 :info :add 10008 :RequestTimeoutException INFO [2022-01-10 08:40:10,596] jepsen worker 418 - jepsen.util 3418 :invoke :add 11024 INFO [2022-01-10 08:40:10,597] jepsen worker 223 - jepsen.util 2223 :info :add 10010 :RequestTimeoutException INFO [2022-01-10 08:40:10,598] jepsen worker 607 - jepsen.util 2607 :invoke :add 11025 INFO [2022-01-10 08:40:10,600] jepsen worker 903 - jepsen.util 4903 :invoke :add 11026 INFO [2022-01-10 08:40:10,600] jepsen worker 917 - jepsen.util 2917 :info :add 10011 :RequestTimeoutException INFO [2022-01-10 08:40:10,602] jepsen worker 6 - jepsen.util 4006 :invoke :add 11027 INFO [2022-01-10 08:40:10,602] jepsen worker 145 - jepsen.util 2145 :info :add 10012 :RequestTimeoutException INFO [2022-01-10 08:40:10,605] jepsen worker 223 - jepsen.util 3223 :invoke :add 11028 INFO [2022-01-10 08:40:10,606] jepsen worker 145 - jepsen.util 3145 :invoke :add 11029 INFO [2022-01-10 08:40:10,608] jepsen worker 917 - jepsen.util 3917 :invoke :add 11030 INFO [2022-01-10 08:40:10,611] jepsen worker 900 - jepsen.util 3900 :info :add 10013 :RequestTimeoutException INFO [2022-01-10 08:40:10,614] jepsen worker 900 - jepsen.util 4900 :invoke :add 11031 INFO [2022-01-10 08:40:10,617] jepsen worker 407 - jepsen.util 1407 :info :add 10016 :RequestTimeoutException INFO [2022-01-10 08:40:10,620] jepsen worker 87 - jepsen.util 2087 :info :add 10017 :RequestTimeoutException INFO [2022-01-10 08:40:10,620] jepsen worker 847 - jepsen.util 2847 :info :add 10019 :RequestTimeoutException INFO [2022-01-10 08:40:10,621] jepsen worker 407 - jepsen.util 2407 :invoke :add 11032 INFO [2022-01-10 08:40:10,623] jepsen worker 92 - jepsen.util 1092 :info :add 10020 :RequestTimeoutException INFO [2022-01-10 08:40:10,624] jepsen worker 847 - jepsen.util 3847 :invoke :add 11033 INFO [2022-01-10 08:40:10,626] jepsen worker 87 - jepsen.util 3087 :invoke :add 11034 INFO [2022-01-10 08:40:10,626] jepsen worker 894 - jepsen.util 2894 :info :add 10018 :RequestTimeoutException INFO [2022-01-10 08:40:10,629] jepsen worker 92 - jepsen.util 2092 :invoke :add 11035 INFO [2022-01-10 08:40:10,631] jepsen worker 894 - jepsen.util 3894 :invoke :add 11036 INFO [2022-01-10 08:40:10,642] jepsen worker 421 - jepsen.util 3421 :info :add 10024 :RequestTimeoutException INFO [2022-01-10 08:40:10,645] jepsen worker 421 - jepsen.util 4421 :invoke :add 11037 INFO [2022-01-10 08:40:10,647] jepsen worker 927 - jepsen.util 3927 :info :add 10025 :RequestTimeoutException INFO [2022-01-10 08:40:10,649] jepsen worker 927 - jepsen.util 4927 :invoke :add 11038 INFO [2022-01-10 08:40:10,651] jepsen worker 912 - jepsen.util 2912 :info :add 10026 :RequestTimeoutException INFO [2022-01-10 08:40:10,651] jepsen worker 824 - jepsen.util 2824 :info :add 10028 :RequestTimeoutException INFO [2022-01-10 08:40:10,653] jepsen worker 531 - jepsen.util 3531 :info :add 10027 :RequestTimeoutException INFO [2022-01-10 08:40:10,653] jepsen worker 949 - jepsen.util 2949 :info :add 10029 :RequestTimeoutException INFO [2022-01-10 08:40:10,656] jepsen worker 949 - jepsen.util 3949 :invoke :add 11039 INFO [2022-01-10 08:40:10,657] jepsen worker 912 - jepsen.util 3912 :invoke :add 11040 INFO [2022-01-10 08:40:10,659] jepsen worker 824 - jepsen.util 3824 :invoke :add 11041 INFO [2022-01-10 08:40:10,661] jepsen worker 531 - jepsen.util 4531 :invoke :add 11042 INFO [2022-01-10 08:40:10,723] jepsen worker 973 - jepsen.util 2973 :info :add 10030 :RequestTimeoutException INFO [2022-01-10 08:40:10,726] jepsen worker 973 - jepsen.util 3973 :invoke :add 11043 INFO [2022-01-10 08:40:10,762] jepsen worker 730 - jepsen.util 3730 :info :add 10031 :RequestTimeoutException INFO [2022-01-10 08:40:10,765] jepsen worker 730 - jepsen.util 4730 :invoke :add 11044 INFO [2022-01-10 08:40:10,784] jepsen worker 581 - jepsen.util 3581 :ok :add 10553 INFO [2022-01-10 08:40:10,787] jepsen worker 651 - jepsen.util 3651 :info :add 10032 :RequestTimeoutException INFO [2022-01-10 08:40:10,788] jepsen worker 581 - jepsen.util 3581 :invoke :add 11045 INFO [2022-01-10 08:40:10,789] jepsen worker 86 - jepsen.util 3086 :ok :add 10083 INFO [2022-01-10 08:40:10,791] jepsen worker 644 - jepsen.util 3644 :info :add 10033 :RequestTimeoutException INFO [2022-01-10 08:40:10,792] jepsen worker 651 - jepsen.util 4651 :invoke :add 11046 INFO [2022-01-10 08:40:10,795] jepsen worker 644 - jepsen.util 4644 :invoke :add 11047 INFO [2022-01-10 08:40:10,798] jepsen worker 86 - jepsen.util 3086 :invoke :add 11048 INFO [2022-01-10 08:40:10,833] jepsen worker 959 - jepsen.util 3959 :info :add 10034 :RequestTimeoutException INFO [2022-01-10 08:40:10,837] jepsen worker 959 - jepsen.util 4959 :invoke :add 11049 INFO [2022-01-10 08:40:10,841] jepsen worker 740 - jepsen.util 3740 :info :add 10035 :RequestTimeoutException INFO [2022-01-10 08:40:10,842] jepsen worker 379 - jepsen.util 3379 :info :add 10037 :RequestTimeoutException INFO [2022-01-10 08:40:10,843] jepsen worker 884 - jepsen.util 3884 :info :add 10036 :RequestTimeoutException INFO [2022-01-10 08:40:10,846] jepsen worker 379 - jepsen.util 4379 :invoke :add 11050 INFO [2022-01-10 08:40:10,847] jepsen worker 980 - jepsen.util 2980 :info :add 10038 :RequestTimeoutException INFO [2022-01-10 08:40:10,848] jepsen worker 740 - jepsen.util 4740 :invoke :add 11051 INFO [2022-01-10 08:40:10,850] jepsen worker 884 - jepsen.util 4884 :invoke :add 11052 INFO [2022-01-10 08:40:10,851] jepsen worker 980 - jepsen.util 3980 :invoke :add 11053 INFO [2022-01-10 08:40:10,860] jepsen worker 153 - jepsen.util 3153 :info :add 10039 :RequestTimeoutException INFO [2022-01-10 08:40:10,862] jepsen worker 153 - jepsen.util 4153 :invoke :add 11054 INFO [2022-01-10 08:40:10,863] jepsen worker 646 - jepsen.util 3646 :info :add 10040 :RequestTimeoutException INFO [2022-01-10 08:40:10,865] jepsen worker 646 - jepsen.util 4646 :invoke :add 11055 INFO [2022-01-10 08:40:11,040] jepsen worker 235 - jepsen.util 3235 :info :add 10041 :RequestTimeoutException INFO [2022-01-10 08:40:11,041] jepsen worker 374 - jepsen.util 2374 :info :add 10042 :RequestTimeoutException INFO [2022-01-10 08:40:11,045] jepsen worker 374 - jepsen.util 3374 :invoke :add 11056 INFO [2022-01-10 08:40:11,047] jepsen worker 235 - jepsen.util 4235 :invoke :add 11057 INFO [2022-01-10 08:40:11,132] jepsen worker 958 - jepsen.util 3958 :info :add 10043 :RequestTimeoutException INFO [2022-01-10 08:40:11,133] jepsen worker 854 - jepsen.util 3854 :info :add 10044 :RequestTimeoutException INFO [2022-01-10 08:40:11,135] jepsen worker 958 - jepsen.util 4958 :invoke :add 11058 INFO [2022-01-10 08:40:11,138] jepsen worker 854 - jepsen.util 4854 :invoke :add 11059 INFO [2022-01-10 08:40:11,154] jepsen worker 530 - jepsen.util 2530 :info :add 10045 :RequestTimeoutException INFO [2022-01-10 08:40:11,157] jepsen worker 530 - jepsen.util 3530 :invoke :add 11060 INFO [2022-01-10 08:40:11,182] jepsen worker 224 - jepsen.util 2224 :info :add 10046 :RequestTimeoutException INFO [2022-01-10 08:40:11,184] jepsen worker 224 - jepsen.util 3224 :invoke :add 11061 INFO [2022-01-10 08:40:11,187] jepsen worker 533 - jepsen.util 2533 :info :add 10047 :RequestTimeoutException INFO [2022-01-10 08:40:11,191] jepsen worker 533 - jepsen.util 3533 :invoke :add 11062 INFO [2022-01-10 08:40:11,200] jepsen worker 414 - jepsen.util 2414 :info :add 10048 :RequestTimeoutException INFO [2022-01-10 08:40:11,203] jepsen worker 414 - jepsen.util 3414 :invoke :add 11063 INFO [2022-01-10 08:40:11,243] jepsen worker 245 - jepsen.util 2245 :info :add 10049 :RequestTimeoutException INFO [2022-01-10 08:40:11,246] jepsen worker 245 - jepsen.util 3245 :invoke :add 11064 INFO [2022-01-10 08:40:11,257] jepsen worker 758 - jepsen.util 2758 :info :add 10051 :RequestTimeoutException INFO [2022-01-10 08:40:11,261] jepsen worker 758 - jepsen.util 3758 :invoke :add 11065 INFO [2022-01-10 08:40:11,266] jepsen worker 342 - jepsen.util 2342 :info :add 10053 :RequestTimeoutException INFO [2022-01-10 08:40:11,269] jepsen worker 342 - jepsen.util 3342 :invoke :add 11066 INFO [2022-01-10 08:40:11,271] jepsen worker 796 - jepsen.util 2796 :info :add 10056 :RequestTimeoutException INFO [2022-01-10 08:40:11,271] jepsen worker 503 - jepsen.util 3503 :info :add 10054 :RequestTimeoutException INFO [2022-01-10 08:40:11,273] jepsen worker 523 - jepsen.util 2523 :info :add 10055 :RequestTimeoutException INFO [2022-01-10 08:40:11,276] jepsen worker 796 - jepsen.util 3796 :invoke :add 11067 INFO [2022-01-10 08:40:11,278] jepsen worker 503 - jepsen.util 4503 :invoke :add 11068 INFO [2022-01-10 08:40:11,280] jepsen worker 523 - jepsen.util 3523 :invoke :add 11069 INFO [2022-01-10 08:40:11,282] jepsen worker 675 - jepsen.util 3675 :info :add 10057 :RequestTimeoutException INFO [2022-01-10 08:40:11,284] jepsen worker 30 - jepsen.util 3030 :info :add 10058 :RequestTimeoutException INFO [2022-01-10 08:40:11,285] jepsen worker 675 - jepsen.util 4675 :invoke :add 11070 INFO [2022-01-10 08:40:11,289] jepsen worker 30 - jepsen.util 4030 :invoke :add 11071 INFO [2022-01-10 08:40:11,292] jepsen worker 779 - jepsen.util 3779 :info :add 10060 :RequestTimeoutException INFO [2022-01-10 08:40:11,292] jepsen worker 954 - jepsen.util 2954 :info :add 10059 :RequestTimeoutException INFO [2022-01-10 08:40:11,293] jepsen worker 752 - jepsen.util 3752 :info :add 10062 :RequestTimeoutException INFO [2022-01-10 08:40:11,295] jepsen worker 779 - jepsen.util 4779 :invoke :add 11072 INFO [2022-01-10 08:40:11,297] jepsen worker 752 - jepsen.util 4752 :invoke :add 11073 INFO [2022-01-10 08:40:11,297] jepsen worker 135 - jepsen.util 2135 :info :add 10061 :RequestTimeoutException INFO [2022-01-10 08:40:11,299] jepsen worker 954 - jepsen.util 3954 :invoke :add 11074 INFO [2022-01-10 08:40:11,300] jepsen worker 313 - jepsen.util 2313 :info :add 10063 :RequestTimeoutException INFO [2022-01-10 08:40:11,301] jepsen worker 135 - jepsen.util 3135 :invoke :add 11075 INFO [2022-01-10 08:40:11,304] jepsen worker 313 - jepsen.util 3313 :invoke :add 11076 INFO [2022-01-10 08:40:11,312] jepsen worker 592 - jepsen.util 2592 :info :add 10064 :RequestTimeoutException INFO [2022-01-10 08:40:11,315] jepsen worker 592 - jepsen.util 3592 :invoke :add 11077 INFO [2022-01-10 08:40:11,381] jepsen worker 669 - jepsen.util 2669 :ok :add 10451 INFO [2022-01-10 08:40:11,383] jepsen worker 663 - jepsen.util 3663 :ok :add 10316 INFO [2022-01-10 08:40:11,383] jepsen worker 669 - jepsen.util 2669 :invoke :add 11078 INFO [2022-01-10 08:40:11,385] jepsen worker 663 - jepsen.util 3663 :invoke :add 11079 INFO [2022-01-10 08:40:11,528] jepsen worker 798 - jepsen.util 2798 :info :add 10068 :RequestTimeoutException INFO [2022-01-10 08:40:11,531] jepsen worker 798 - jepsen.util 3798 :invoke :add 11080 INFO [2022-01-10 08:40:11,531] jepsen worker 393 - jepsen.util 3393 :info :add 10069 :RequestTimeoutException INFO [2022-01-10 08:40:11,534] jepsen worker 393 - jepsen.util 4393 :invoke :add 11081 INFO [2022-01-10 08:40:11,550] jepsen worker 289 - jepsen.util 3289 :ok :add 10285 INFO [2022-01-10 08:40:11,551] jepsen worker 605 - jepsen.util 2605 :ok :add 10339 INFO [2022-01-10 08:40:11,551] jepsen worker 691 - jepsen.util 2691 :ok :add 10321 INFO [2022-01-10 08:40:11,554] jepsen worker 289 - jepsen.util 3289 :invoke :add 11082 INFO [2022-01-10 08:40:11,556] jepsen worker 605 - jepsen.util 2605 :invoke :add 11083 INFO [2022-01-10 08:40:11,558] jepsen worker 691 - jepsen.util 2691 :invoke :add 11084 INFO [2022-01-10 08:40:11,607] jepsen worker 640 - jepsen.util 2640 :ok :add 10177 INFO [2022-01-10 08:40:11,611] jepsen worker 640 - jepsen.util 2640 :invoke :add 11085 INFO [2022-01-10 08:40:11,630] jepsen worker 348 - jepsen.util 3348 :ok :add 10267 INFO [2022-01-10 08:40:11,631] jepsen worker 811 - jepsen.util 3811 :ok :add 10121 INFO [2022-01-10 08:40:11,631] jepsen worker 321 - jepsen.util 2321 :ok :add 10115 INFO [2022-01-10 08:40:11,631] jepsen worker 460 - jepsen.util 3460 :ok :add 10264 INFO [2022-01-10 08:40:11,632] jepsen worker 921 - jepsen.util 2921 :ok :add 10085 INFO [2022-01-10 08:40:11,633] jepsen worker 348 - jepsen.util 3348 :invoke :add 11086 INFO [2022-01-10 08:40:11,636] jepsen worker 460 - jepsen.util 3460 :invoke :add 11087 INFO [2022-01-10 08:40:11,638] jepsen worker 921 - jepsen.util 2921 :invoke :add 11088 INFO [2022-01-10 08:40:11,639] jepsen worker 811 - jepsen.util 3811 :invoke :add 11089 INFO [2022-01-10 08:40:11,641] jepsen worker 321 - jepsen.util 2321 :invoke :add 11090 INFO [2022-01-10 08:40:11,642] jepsen worker 858 - jepsen.util 2858 :info :add 10070 :RequestTimeoutException INFO [2022-01-10 08:40:11,646] jepsen worker 710 - jepsen.util 2710 :info :add 10071 :RequestTimeoutException INFO [2022-01-10 08:40:11,648] jepsen worker 858 - jepsen.util 3858 :invoke :add 11091 INFO [2022-01-10 08:40:11,650] jepsen worker 710 - jepsen.util 3710 :invoke :add 11092 INFO [2022-01-10 08:40:11,651] jepsen worker 698 - jepsen.util 2698 :info :add 10072 :RequestTimeoutException INFO [2022-01-10 08:40:11,654] jepsen worker 698 - jepsen.util 3698 :invoke :add 11093 INFO [2022-01-10 08:40:11,662] jepsen worker 238 - jepsen.util 3238 :info :add 10073 :RequestTimeoutException INFO [2022-01-10 08:40:11,666] jepsen worker 238 - jepsen.util 4238 :invoke :add 11094 INFO [2022-01-10 08:40:11,671] jepsen worker 435 - jepsen.util 2435 :info :add 10074 :RequestTimeoutException INFO [2022-01-10 08:40:11,673] jepsen worker 783 - jepsen.util 2783 :info :add 10075 :RequestTimeoutException INFO [2022-01-10 08:40:11,676] jepsen worker 435 - jepsen.util 3435 :invoke :add 11095 INFO [2022-01-10 08:40:11,677] jepsen worker 783 - jepsen.util 3783 :invoke :add 11096 INFO [2022-01-10 08:40:11,682] jepsen worker 8 - jepsen.util 2008 :info :add 10076 :RequestTimeoutException INFO [2022-01-10 08:40:11,685] jepsen worker 8 - jepsen.util 3008 :invoke :add 11097 INFO [2022-01-10 08:40:11,687] jepsen worker 499 - jepsen.util 2499 :info :add 10077 :RequestTimeoutException INFO [2022-01-10 08:40:11,690] jepsen worker 499 - jepsen.util 3499 :invoke :add 11098 INFO [2022-01-10 08:40:11,690] jepsen worker 112 - jepsen.util 3112 :info :add 10078 :RequestTimeoutException INFO [2022-01-10 08:40:11,693] jepsen worker 104 - jepsen.util 3104 :info :add 10079 :RequestTimeoutException INFO [2022-01-10 08:40:11,694] jepsen worker 112 - jepsen.util 4112 :invoke :add 11099 INFO [2022-01-10 08:40:11,696] jepsen worker 104 - jepsen.util 4104 :invoke :add 11100 INFO [2022-01-10 08:40:11,701] jepsen worker 453 - jepsen.util 2453 :info :add 10080 :RequestTimeoutException INFO [2022-01-10 08:40:11,703] jepsen worker 453 - jepsen.util 3453 :invoke :add 11101 INFO [2022-01-10 08:40:11,753] jepsen worker 218 - jepsen.util 2218 :info :add 10081 :RequestTimeoutException INFO [2022-01-10 08:40:11,753] jepsen worker 395 - jepsen.util 2395 :info :add 10082 :RequestTimeoutException INFO [2022-01-10 08:40:11,757] jepsen worker 395 - jepsen.util 3395 :invoke :add 11102 INFO [2022-01-10 08:40:11,758] jepsen worker 218 - jepsen.util 3218 :invoke :add 11103 INFO [2022-01-10 08:40:11,822] jepsen worker 563 - jepsen.util 3563 :info :add 10087 :RequestTimeoutException INFO [2022-01-10 08:40:11,825] jepsen worker 563 - jepsen.util 4563 :invoke :add 11104 INFO [2022-01-10 08:40:11,827] jepsen worker 968 - jepsen.util 3968 :info :add 10088 :RequestTimeoutException INFO [2022-01-10 08:40:11,830] jepsen worker 968 - jepsen.util 4968 :invoke :add 11105 INFO [2022-01-10 08:40:11,858] jepsen worker 681 - jepsen.util 2681 :ok :add 10126 INFO [2022-01-10 08:40:11,858] jepsen worker 364 - jepsen.util 3364 :ok :add 10112 INFO [2022-01-10 08:40:11,860] jepsen worker 681 - jepsen.util 2681 :invoke :add 11106 INFO [2022-01-10 08:40:11,860] jepsen worker 622 - jepsen.util 2622 :ok :add 10263 INFO [2022-01-10 08:40:11,862] jepsen worker 364 - jepsen.util 3364 :invoke :add 11107 INFO [2022-01-10 08:40:11,864] jepsen worker 622 - jepsen.util 2622 :invoke :add 11108 INFO [2022-01-10 08:40:11,950] jepsen worker 447 - jepsen.util 3447 :ok :add 10222 INFO [2022-01-10 08:40:11,951] jepsen worker 446 - jepsen.util 3446 :ok :add 10550 INFO [2022-01-10 08:40:11,953] jepsen worker 447 - jepsen.util 3447 :invoke :add 11109 INFO [2022-01-10 08:40:11,955] jepsen worker 446 - jepsen.util 3446 :invoke :add 11110 INFO [2022-01-10 08:40:11,981] jepsen worker 995 - jepsen.util 3995 :info :add 10089 :RequestTimeoutException INFO [2022-01-10 08:40:11,983] jepsen worker 995 - jepsen.util 4995 :invoke :add 11111 INFO [2022-01-10 08:40:11,991] jepsen worker 520 - jepsen.util 3520 :info :add 10091 :RequestTimeoutException INFO [2022-01-10 08:40:11,992] jepsen worker 66 - jepsen.util 3066 :info :add 10090 :RequestTimeoutException INFO [2022-01-10 08:40:11,996] jepsen worker 66 - jepsen.util 4066 :invoke :add 11112 INFO [2022-01-10 08:40:11,998] jepsen worker 520 - jepsen.util 4520 :invoke :add 11113 INFO [2022-01-10 08:40:12,012] jepsen worker 256 - jepsen.util 3256 :info :add 10093 :RequestTimeoutException INFO [2022-01-10 08:40:12,013] jepsen worker 216 - jepsen.util 3216 :info :add 10092 :RequestTimeoutException INFO [2022-01-10 08:40:12,015] jepsen worker 256 - jepsen.util 4256 :invoke :add 11114 INFO [2022-01-10 08:40:12,017] jepsen worker 216 - jepsen.util 4216 :invoke :add 11115 INFO [2022-01-10 08:40:12,077] jepsen worker 506 - jepsen.util 2506 :info :add 10094 :RequestTimeoutException INFO [2022-01-10 08:40:12,081] jepsen worker 506 - jepsen.util 3506 :invoke :add 11116 INFO [2022-01-10 08:40:12,100] jepsen worker 564 - jepsen.util 2564 :info :add 10095 :RequestTimeoutException INFO [2022-01-10 08:40:12,103] jepsen worker 564 - jepsen.util 3564 :invoke :add 11117 INFO [2022-01-10 08:40:12,111] jepsen worker 303 - jepsen.util 2303 :info :add 10097 :RequestTimeoutException INFO [2022-01-10 08:40:12,112] jepsen worker 579 - jepsen.util 2579 :info :add 10096 :RequestTimeoutException INFO [2022-01-10 08:40:12,113] jepsen worker 371 - jepsen.util 2371 :info :add 10098 :RequestTimeoutException INFO [2022-01-10 08:40:12,114] jepsen worker 303 - jepsen.util 3303 :invoke :add 11118 INFO [2022-01-10 08:40:12,117] jepsen worker 579 - jepsen.util 3579 :invoke :add 11119 INFO [2022-01-10 08:40:12,119] jepsen worker 371 - jepsen.util 3371 :invoke :add 11120 INFO [2022-01-10 08:40:12,159] jepsen worker nemesis - jepsen.generator.interpreter Stopping test INFO [2022-01-10 08:40:12,167] jepsen worker 504 - jepsen.util 2504 :info :add 10099 :RequestTimeoutException INFO [2022-01-10 08:40:12,172] jepsen worker 700 - jepsen.util 2700 :info :add 10100 :RequestTimeoutException INFO [2022-01-10 08:40:12,172] jepsen worker 633 - jepsen.util 2633 :info :add 10101 :RequestTimeoutException INFO [2022-01-10 08:40:12,173] jepsen worker 629 - jepsen.util 1629 :info :add 10102 :RequestTimeoutException INFO [2022-01-10 08:40:12,181] jepsen worker 52 - jepsen.util 3052 :info :add 10104 :RequestTimeoutException INFO [2022-01-10 08:40:12,182] jepsen worker 258 - jepsen.util 3258 :info :add 10103 :RequestTimeoutException INFO [2022-01-10 08:40:12,182] jepsen worker 467 - jepsen.util 2467 :info :add 10105 :RequestTimeoutException INFO [2022-01-10 08:40:12,327] jepsen worker 134 - jepsen.util 3134 :ok :add 10331 INFO [2022-01-10 08:40:12,354] jepsen worker 408 - jepsen.util 2408 :info :add 10106 :RequestTimeoutException INFO [2022-01-10 08:40:12,372] jepsen worker 432 - jepsen.util 3432 :info :add 10107 :RequestTimeoutException INFO [2022-01-10 08:40:12,377] jepsen worker 849 - jepsen.util 3849 :info :add 10108 :RequestTimeoutException INFO [2022-01-10 08:40:12,380] jepsen worker 206 - jepsen.util 2206 :info :add 10109 :RequestTimeoutException INFO [2022-01-10 08:40:12,382] jepsen worker 596 - jepsen.util 2596 :info :add 10110 :RequestTimeoutException INFO [2022-01-10 08:40:12,391] jepsen worker 248 - jepsen.util 2248 :info :add 10111 :RequestTimeoutException INFO [2022-01-10 08:40:12,392] jepsen worker 562 - jepsen.util 2562 :info :add 10113 :RequestTimeoutException INFO [2022-01-10 08:40:12,397] jepsen worker 334 - jepsen.util 3334 :info :add 10114 :RequestTimeoutException INFO [2022-01-10 08:40:12,403] jepsen worker 307 - jepsen.util 3307 :info :add 10116 :RequestTimeoutException INFO [2022-01-10 08:40:12,411] jepsen worker 816 - jepsen.util 3816 :info :add 10117 :RequestTimeoutException INFO [2022-01-10 08:40:12,414] jepsen worker 198 - jepsen.util 3198 :info :add 10118 :RequestTimeoutException INFO [2022-01-10 08:40:12,418] jepsen worker 191 - jepsen.util 3191 :ok :add 10252 INFO [2022-01-10 08:40:12,418] jepsen worker 957 - jepsen.util 3957 :ok :add 10484 INFO [2022-01-10 08:40:12,418] jepsen worker 229 - jepsen.util 3229 :ok :add 10520 INFO [2022-01-10 08:40:12,418] jepsen worker 242 - jepsen.util 3242 :ok :add 10538 INFO [2022-01-10 08:40:12,418] jepsen worker 632 - jepsen.util 3632 :ok :add 10514 INFO [2022-01-10 08:40:12,419] jepsen worker 49 - jepsen.util 3049 :ok :add 10123 INFO [2022-01-10 08:40:12,419] jepsen worker 853 - jepsen.util 2853 :ok :add 10266 INFO [2022-01-10 08:40:12,492] jepsen worker 501 - jepsen.util 3501 :info :add 10119 :RequestTimeoutException INFO [2022-01-10 08:40:12,497] jepsen worker 197 - jepsen.util 3197 :info :add 10120 :RequestTimeoutException INFO [2022-01-10 08:40:12,503] jepsen worker 230 - jepsen.util 3230 :info :add 10122 :RequestTimeoutException INFO [2022-01-10 08:40:12,533] jepsen worker 687 - jepsen.util 2687 :info :add 10124 :RequestTimeoutException INFO [2022-01-10 08:40:12,550] jepsen worker 870 - jepsen.util 2870 :info :add 10127 :RequestTimeoutException INFO [2022-01-10 08:40:12,552] jepsen worker 842 - jepsen.util 2842 :info :add 10125 :RequestTimeoutException INFO [2022-01-10 08:40:12,553] jepsen worker 841 - jepsen.util 2841 :info :add 10128 :RequestTimeoutException INFO [2022-01-10 08:40:12,611] jepsen worker 136 - jepsen.util 3136 :info :add 10129 :RequestTimeoutException INFO [2022-01-10 08:40:12,643] jepsen worker 265 - jepsen.util 3265 :info :add 10130 :RequestTimeoutException INFO [2022-01-10 08:40:12,652] jepsen worker 333 - jepsen.util 3333 :info :add 10131 :RequestTimeoutException INFO [2022-01-10 08:40:12,677] jepsen worker 404 - jepsen.util 3404 :info :add 10132 :RequestTimeoutException INFO [2022-01-10 08:40:12,703] jepsen worker 491 - jepsen.util 2491 :info :add 10133 :RequestTimeoutException INFO [2022-01-10 08:40:12,780] jepsen worker 688 - jepsen.util 3688 :info :add 10134 :RequestTimeoutException INFO [2022-01-10 08:40:12,791] jepsen worker 924 - jepsen.util 3924 :info :add 10136 :RequestTimeoutException INFO [2022-01-10 08:40:12,792] jepsen worker 108 - jepsen.util 3108 :info :add 10135 :RequestTimeoutException INFO [2022-01-10 08:40:12,793] jepsen worker 338 - jepsen.util 1338 :info :add 10137 :RequestTimeoutException INFO [2022-01-10 08:40:12,972] jepsen worker 463 - jepsen.util 3463 :ok :add 10487 INFO [2022-01-10 08:40:12,972] jepsen worker 452 - jepsen.util 3452 :ok :add 10523 INFO [2022-01-10 08:40:12,972] jepsen worker 626 - jepsen.util 3626 :ok :add 10666 INFO [2022-01-10 08:40:12,972] jepsen worker 662 - jepsen.util 2662 :ok :add 10249 INFO [2022-01-10 08:40:12,972] jepsen worker 771 - jepsen.util 3771 :ok :add 10517 INFO [2022-01-10 08:40:12,982] jepsen worker 747 - jepsen.util 3747 :info :add 10138 :RequestTimeoutException INFO [2022-01-10 08:40:12,987] jepsen worker 366 - jepsen.util 3366 :info :add 10139 :RequestTimeoutException INFO [2022-01-10 08:40:13,020] jepsen worker 4 - jepsen.util 3004 :info :add 10140 :RequestTimeoutException INFO [2022-01-10 08:40:13,021] jepsen worker 863 - jepsen.util 3863 :info :add 10142 :RequestTimeoutException INFO [2022-01-10 08:40:13,022] jepsen worker 725 - jepsen.util 3725 :info :add 10141 :RequestTimeoutException INFO [2022-01-10 08:40:13,024] jepsen worker 484 - jepsen.util 3484 :info :add 10143 :RequestTimeoutException INFO [2022-01-10 08:40:13,072] jepsen worker 347 - jepsen.util 3347 :info :add 10144 :RequestTimeoutException INFO [2022-01-10 08:40:13,087] jepsen worker 846 - jepsen.util 3846 :info :add 10145 :RequestTimeoutException INFO [2022-01-10 08:40:13,120] jepsen worker 614 - jepsen.util 3614 :info :add 10146 :RequestTimeoutException INFO [2022-01-10 08:40:13,123] jepsen worker 514 - jepsen.util 3514 :info :add 10147 :RequestTimeoutException INFO [2022-01-10 08:40:13,127] jepsen worker 81 - jepsen.util 3081 :info :add 10149 :RequestTimeoutException INFO [2022-01-10 08:40:13,131] jepsen worker 174 - jepsen.util 3174 :info :add 10150 :RequestTimeoutException INFO [2022-01-10 08:40:13,133] jepsen worker 19 - jepsen.util 3019 :info :add 10148 :RequestTimeoutException INFO [2022-01-10 08:40:13,140] jepsen worker 192 - jepsen.util 2192 :info :add 10152 :RequestTimeoutException INFO [2022-01-10 08:40:13,142] jepsen worker 814 - jepsen.util 3814 :info :add 10153 :RequestTimeoutException INFO [2022-01-10 08:40:13,152] jepsen worker 617 - jepsen.util 3617 :info :add 10154 :RequestTimeoutException INFO [2022-01-10 08:40:13,153] jepsen worker 210 - jepsen.util 2210 :info :add 10155 :RequestTimeoutException INFO [2022-01-10 08:40:13,156] jepsen worker 618 - jepsen.util 3618 :info :add 10156 :RequestTimeoutException INFO [2022-01-10 08:40:13,160] jepsen worker 254 - jepsen.util 3254 :info :add 10158 :RequestTimeoutException INFO [2022-01-10 08:40:13,163] jepsen worker 336 - jepsen.util 3336 :info :add 10159 :RequestTimeoutException INFO [2022-01-10 08:40:13,164] jepsen worker 590 - jepsen.util 2590 :info :add 10157 :RequestTimeoutException INFO [2022-01-10 08:40:13,201] jepsen worker 472 - jepsen.util 3472 :info :add 10160 :RequestTimeoutException INFO [2022-01-10 08:40:13,203] jepsen worker 795 - jepsen.util 3795 :info :add 10161 :RequestTimeoutException INFO [2022-01-10 08:40:13,212] jepsen worker 391 - jepsen.util 3391 :info :add 10162 :RequestTimeoutException INFO [2022-01-10 08:40:13,216] jepsen worker 971 - jepsen.util 3971 :info :add 10163 :RequestTimeoutException WARN [2022-01-10 08:40:13,242] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":16929300,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":16929300,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":9009789,"encode_duration_us":252,"operation_id":"0x23fb","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10009954},{"operation_name":"insert","last_dispatch_duration_us":14105721,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14105721,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":8966375,"encode_duration_us":293,"operation_id":"0x2bc5","timeout_ms":10000,"last_server_duration_us":8966375,"total_duration_us":10010048},{"operation_name":"insert","last_dispatch_duration_us":11697449,"last_local_id":"2B99390F00000001/000000005D894E7E","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":11697449,"last_local_socket":"172.23.104.79:51762","total_server_duration_us":2995216,"encode_duration_us":294,"operation_id":"0x2cc3","timeout_ms":10000,"last_server_duration_us":2995216,"total_duration_us":10010075},{"operation_name":"insert","last_dispatch_duration_us":13717486,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13717486,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":9010849,"encode_duration_us":252,"operation_id":"0x241a","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10010246},{"operation_name":"insert","last_dispatch_duration_us":9940593,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9940593,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":2289963,"encode_duration_us":256,"operation_id":"0x2982","timeout_ms":10000,"last_server_duration_us":2289963,"total_duration_us":10010588},{"operation_name":"insert","last_dispatch_duration_us":14144997,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14144997,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":9012969,"encode_duration_us":312,"operation_id":"0x2a50","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10010508},{"operation_name":"insert","last_dispatch_duration_us":10233797,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10233797,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":3067916,"encode_duration_us":567,"operation_id":"0x2bf5","timeout_ms":10000,"last_server_duration_us":3067916,"total_duration_us":10063545},{"operation_name":"insert","last_dispatch_duration_us":14431939,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":14431939,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":9005550,"encode_duration_us":285,"operation_id":"0x27b2","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10010648},{"operation_name":"insert","last_dispatch_duration_us":17642646,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17642646,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":9007669,"encode_duration_us":258,"operation_id":"0x24df","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10010374},{"operation_name":"insert","last_dispatch_duration_us":11068689,"last_local_id":"2B99390F00000001/000000005D894E7E","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":11068689,"last_local_socket":"172.23.104.79:51762","total_server_duration_us":2739283,"encode_duration_us":258,"operation_id":"0x2d16","timeout_ms":10000,"last_server_duration_us":2739283,"total_duration_us":10010968}],"total_count":117}} WARN [2022-01-10 08:40:13,242] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":10233797,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":10233797,"total_server_duration_us":3067916,"encode_duration_us":567,"operation_id":"0x2bf5","timeout_ms":10000,"last_server_duration_us":3067916,"total_duration_us":10063545},{"operation_name":"insert","last_dispatch_duration_us":11068689,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000001/000000005D894E7E","last_local_socket":"172.23.104.79:51762","total_dispatch_duration_us":11068689,"total_server_duration_us":2739283,"encode_duration_us":258,"operation_id":"0x2d16","timeout_ms":10000,"last_server_duration_us":2739283,"total_duration_us":10010968},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","encode_duration_us":432,"operation_id":"0x2a95","timeout_ms":10000,"total_duration_us":10010779},{"operation_name":"insert","encode_duration_us":332,"operation_id":"0x2d7a","timeout_ms":10000,"total_duration_us":10010774},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000001/00000000CB02B145","last_local_socket":"172.23.104.79:41914","encode_duration_us":312,"operation_id":"0x2cf5","timeout_ms":10000,"total_duration_us":10010751},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000001/00000000CB02B145","last_local_socket":"172.23.104.79:41914","encode_duration_us":314,"operation_id":"0x2d8f","timeout_ms":10000,"total_duration_us":10010668},{"operation_name":"insert","last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000001/000000005D894E7E","last_local_socket":"172.23.104.79:51762","encode_duration_us":373,"operation_id":"0x2db1","timeout_ms":10000,"total_duration_us":10010631},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000001/00000000CB02B145","last_local_socket":"172.23.104.79:41914","encode_duration_us":303,"operation_id":"0x2bb0","timeout_ms":10000,"total_duration_us":10010625},{"operation_name":"insert","last_dispatch_duration_us":9940593,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":9940593,"total_server_duration_us":2289963,"encode_duration_us":256,"operation_id":"0x2982","timeout_ms":10000,"last_server_duration_us":2289963,"total_duration_us":10010588},{"operation_name":"insert","last_dispatch_duration_us":14144997,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","total_dispatch_duration_us":14144997,"total_server_duration_us":9012969,"encode_duration_us":312,"operation_id":"0x2a50","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10010508}],"total_count":171}} INFO [2022-01-10 08:40:13,291] jepsen worker 788 - jepsen.util 2788 :info :add 10164 :RequestTimeoutException INFO [2022-01-10 08:40:13,293] jepsen worker 799 - jepsen.util 2799 :info :add 10165 :RequestTimeoutException INFO [2022-01-10 08:40:13,341] jepsen worker 440 - jepsen.util 3440 :info :add 10166 :RequestTimeoutException INFO [2022-01-10 08:40:13,352] jepsen worker 889 - jepsen.util 3889 :info :add 10168 :RequestTimeoutException INFO [2022-01-10 08:40:13,377] jepsen worker 748 - jepsen.util 2748 :info :add 10169 :RequestTimeoutException INFO [2022-01-10 08:40:13,380] jepsen worker 168 - jepsen.util 3168 :info :add 10170 :RequestTimeoutException INFO [2022-01-10 08:40:13,383] jepsen worker 613 - jepsen.util 3613 :info :add 10171 :RequestTimeoutException INFO [2022-01-10 08:40:13,391] jepsen worker 469 - jepsen.util 3469 :info :add 10172 :RequestTimeoutException INFO [2022-01-10 08:40:13,422] jepsen worker 18 - jepsen.util 2018 :info :add 10174 :RequestTimeoutException INFO [2022-01-10 08:40:13,423] jepsen worker 199 - jepsen.util 2199 :info :add 10173 :RequestTimeoutException INFO [2022-01-10 08:40:13,517] jepsen worker 67 - jepsen.util 2067 :info :add 10175 :RequestTimeoutException INFO [2022-01-10 08:40:13,523] jepsen worker 455 - jepsen.util 3455 :ok :add 10784 INFO [2022-01-10 08:40:13,540] jepsen worker 293 - jepsen.util 2293 :info :add 10176 :RequestTimeoutException INFO [2022-01-10 08:40:13,591] jepsen worker 722 - jepsen.util 2722 :info :add 10178 :RequestTimeoutException INFO [2022-01-10 08:40:13,601] jepsen worker 161 - jepsen.util 3161 :info :add 10179 :RequestTimeoutException INFO [2022-01-10 08:40:13,601] jepsen worker 538 - jepsen.util 1538 :info :add 10180 :RequestTimeoutException INFO [2022-01-10 08:40:13,611] jepsen worker 736 - jepsen.util 2736 :info :add 10181 :RequestTimeoutException INFO [2022-01-10 08:40:13,612] jepsen worker 720 - jepsen.util 3720 :info :add 10182 :RequestTimeoutException INFO [2022-01-10 08:40:13,622] jepsen worker 939 - jepsen.util 3939 :info :add 10184 :RequestTimeoutException INFO [2022-01-10 08:40:13,623] jepsen worker 667 - jepsen.util 3667 :info :add 10183 :RequestTimeoutException INFO [2022-01-10 08:40:13,627] jepsen worker 822 - jepsen.util 3822 :info :add 10185 :RequestTimeoutException INFO [2022-01-10 08:40:13,630] jepsen worker 14 - jepsen.util 3014 :info :add 10186 :RequestTimeoutException INFO [2022-01-10 08:40:13,631] jepsen worker 251 - jepsen.util 3251 :ok :add 10421 INFO [2022-01-10 08:40:13,631] jepsen worker 965 - jepsen.util 2965 :ok :add 10415 INFO [2022-01-10 08:40:13,631] jepsen worker 214 - jepsen.util 3214 :ok :add 10353 INFO [2022-01-10 08:40:13,631] jepsen worker 620 - jepsen.util 4620 :ok :add 10764 INFO [2022-01-10 08:40:13,632] jepsen worker 541 - jepsen.util 3541 :ok :add 10439 INFO [2022-01-10 08:40:13,632] jepsen worker 630 - jepsen.util 1630 :ok :add 10367 INFO [2022-01-10 08:40:13,652] jepsen worker 729 - jepsen.util 3729 :info :add 10187 :RequestTimeoutException INFO [2022-01-10 08:40:13,661] jepsen worker 879 - jepsen.util 3879 :info :add 10188 :RequestTimeoutException INFO [2022-01-10 08:40:13,663] jepsen worker 526 - jepsen.util 2526 :info :add 10189 :RequestTimeoutException INFO [2022-01-10 08:40:13,742] jepsen worker 511 - jepsen.util 3511 :info :add 10190 :RequestTimeoutException INFO [2022-01-10 08:40:13,757] jepsen worker 182 - jepsen.util 2182 :info :add 10191 :RequestTimeoutException INFO [2022-01-10 08:40:13,760] jepsen worker 329 - jepsen.util 3329 :info :add 10192 :RequestTimeoutException INFO [2022-01-10 08:40:13,761] jepsen worker 500 - jepsen.util 3500 :info :add 10194 :RequestTimeoutException INFO [2022-01-10 08:40:13,763] jepsen worker 834 - jepsen.util 3834 :info :add 10193 :RequestTimeoutException INFO [2022-01-10 08:40:13,770] jepsen worker 188 - jepsen.util 2188 :info :add 10196 :RequestTimeoutException INFO [2022-01-10 08:40:13,773] jepsen worker 397 - jepsen.util 2397 :info :add 10195 :RequestTimeoutException INFO [2022-01-10 08:40:13,912] jepsen worker 324 - jepsen.util 2324 :info :add 10198 :RequestTimeoutException INFO [2022-01-10 08:40:13,913] jepsen worker 441 - jepsen.util 3441 :info :add 10197 :RequestTimeoutException INFO [2022-01-10 08:40:13,916] jepsen worker 919 - jepsen.util 2919 :info :add 10199 :RequestTimeoutException INFO [2022-01-10 08:40:13,930] jepsen worker 476 - jepsen.util 2476 :info :add 10200 :RequestTimeoutException INFO [2022-01-10 08:40:13,931] jepsen worker 832 - jepsen.util 3832 :info :add 10202 :RequestTimeoutException INFO [2022-01-10 08:40:13,933] jepsen worker 936 - jepsen.util 2936 :info :add 10201 :RequestTimeoutException INFO [2022-01-10 08:40:13,934] jepsen worker 259 - jepsen.util 2259 :info :add 10203 :RequestTimeoutException INFO [2022-01-10 08:40:13,944] jepsen worker 310 - jepsen.util 4310 :ok :add 10749 INFO [2022-01-10 08:40:13,945] jepsen worker 346 - jepsen.util 2346 :ok :add 10420 INFO [2022-01-10 08:40:13,945] jepsen worker 340 - jepsen.util 2340 :ok :add 10414 INFO [2022-01-10 08:40:13,945] jepsen worker 100 - jepsen.util 3100 :ok :add 10438 INFO [2022-01-10 08:40:13,962] jepsen worker 734 - jepsen.util 2734 :info :add 10204 :RequestTimeoutException INFO [2022-01-10 08:40:13,977] jepsen worker 918 - jepsen.util 3918 :info :add 10205 :RequestTimeoutException INFO [2022-01-10 08:40:14,010] jepsen worker 986 - jepsen.util 3986 :info :add 10206 :RequestTimeoutException INFO [2022-01-10 08:40:14,023] jepsen worker 365 - jepsen.util 3365 :info :add 10207 :RequestTimeoutException INFO [2022-01-10 08:40:14,027] jepsen worker 96 - jepsen.util 2096 :info :add 10208 :RequestTimeoutException INFO [2022-01-10 08:40:14,032] jepsen worker 647 - jepsen.util 2647 :info :add 10209 :RequestTimeoutException INFO [2022-01-10 08:40:14,033] jepsen worker 196 - jepsen.util 3196 :info :add 10210 :RequestTimeoutException INFO [2022-01-10 08:40:14,040] jepsen worker 835 - jepsen.util 1835 :info :add 10211 :RequestTimeoutException INFO [2022-01-10 08:40:14,043] jepsen worker 600 - jepsen.util 3600 :info :add 10213 :RequestTimeoutException INFO [2022-01-10 08:40:14,047] jepsen worker 749 - jepsen.util 2749 :info :add 10212 :RequestTimeoutException INFO [2022-01-10 08:40:14,050] jepsen worker 253 - jepsen.util 3253 :info :add 10214 :RequestTimeoutException INFO [2022-01-10 08:40:14,051] jepsen worker 516 - jepsen.util 2516 :info :add 10215 :RequestTimeoutException INFO [2022-01-10 08:40:14,052] jepsen worker 64 - jepsen.util 3064 :info :add 10216 :RequestTimeoutException INFO [2022-01-10 08:40:14,057] jepsen worker 930 - jepsen.util 3930 :ok :add 10338 INFO [2022-01-10 08:40:14,057] jepsen worker 222 - jepsen.util 3222 :ok :add 10717 INFO [2022-01-10 08:40:14,057] jepsen worker 554 - jepsen.util 3554 :ok :add 10452 INFO [2022-01-10 08:40:14,057] jepsen worker 952 - jepsen.util 2952 :ok :add 10314 INFO [2022-01-10 08:40:14,057] jepsen worker 17 - jepsen.util 3017 :ok :add 10284 INFO [2022-01-10 08:40:14,057] jepsen worker 769 - jepsen.util 3769 :ok :add 10466 INFO [2022-01-10 08:40:14,057] jepsen worker 290 - jepsen.util 3290 :ok :add 10320 INFO [2022-01-10 08:40:14,073] jepsen worker 227 - jepsen.util 3227 :info :add 10217 :RequestTimeoutException INFO [2022-01-10 08:40:14,081] jepsen worker 243 - jepsen.util 3243 :info :add 10218 :RequestTimeoutException INFO [2022-01-10 08:40:14,081] jepsen worker 82 - jepsen.util 3082 :ok :add 10442 INFO [2022-01-10 08:40:14,082] jepsen worker 925 - jepsen.util 3925 :info :add 10220 :RequestTimeoutException INFO [2022-01-10 08:40:14,083] jepsen worker 217 - jepsen.util 3217 :info :add 10219 :RequestTimeoutException INFO [2022-01-10 08:40:14,097] jepsen worker 902 - jepsen.util 3902 :info :add 10221 :RequestTimeoutException INFO [2022-01-10 08:40:14,101] jepsen worker 631 - jepsen.util 3631 :info :add 10224 :RequestTimeoutException INFO [2022-01-10 08:40:14,103] jepsen worker 103 - jepsen.util 3103 :info :add 10223 :RequestTimeoutException INFO [2022-01-10 08:40:14,112] jepsen worker 170 - jepsen.util 3170 :info :add 10226 :RequestTimeoutException INFO [2022-01-10 08:40:14,113] jepsen worker 539 - jepsen.util 3539 :info :add 10225 :RequestTimeoutException INFO [2022-01-10 08:40:14,127] jepsen worker 151 - jepsen.util 3151 :info :add 10227 :RequestTimeoutException INFO [2022-01-10 08:40:14,130] jepsen worker 110 - jepsen.util 3110 :info :add 10228 :RequestTimeoutException INFO [2022-01-10 08:40:14,133] jepsen worker 569 - jepsen.util 3569 :info :add 10229 :RequestTimeoutException INFO [2022-01-10 08:40:14,211] jepsen worker 696 - jepsen.util 2696 :info :add 10230 :RequestTimeoutException INFO [2022-01-10 08:40:14,212] jepsen worker 101 - jepsen.util 2101 :info :add 10232 :RequestTimeoutException INFO [2022-01-10 08:40:14,213] jepsen worker 159 - jepsen.util 3159 :info :add 10231 :RequestTimeoutException INFO [2022-01-10 08:40:14,217] jepsen worker 284 - jepsen.util 2284 :info :add 10233 :RequestTimeoutException INFO [2022-01-10 08:40:14,220] jepsen worker 770 - jepsen.util 1770 :info :add 10234 :RequestTimeoutException INFO [2022-01-10 08:40:14,233] jepsen worker 273 - jepsen.util 2273 :info :add 10235 :RequestTimeoutException INFO [2022-01-10 08:40:14,241] jepsen worker 32 - jepsen.util 2032 :info :add 10236 :RequestTimeoutException INFO [2022-01-10 08:40:14,243] jepsen worker 38 - jepsen.util 3038 :info :add 10237 :RequestTimeoutException INFO [2022-01-10 08:40:14,312] jepsen worker 140 - jepsen.util 2140 :info :add 10238 :RequestTimeoutException INFO [2022-01-10 08:40:14,317] jepsen worker 68 - jepsen.util 2068 :info :add 10239 :RequestTimeoutException INFO [2022-01-10 08:40:14,320] jepsen worker 479 - jepsen.util 2479 :info :add 10240 :RequestTimeoutException INFO [2022-01-10 08:40:14,333] jepsen worker 672 - jepsen.util 2672 :info :add 10241 :RequestTimeoutException INFO [2022-01-10 08:40:14,341] jepsen worker 464 - jepsen.util 3464 :info :add 10242 :RequestTimeoutException INFO [2022-01-10 08:40:14,344] jepsen worker 426 - jepsen.util 3426 :info :add 10243 :RequestTimeoutException INFO [2022-01-10 08:40:14,352] jepsen worker 209 - jepsen.util 3209 :info :add 10244 :RequestTimeoutException INFO [2022-01-10 08:40:14,356] jepsen worker 586 - jepsen.util 3586 :info :add 10245 :RequestTimeoutException INFO [2022-01-10 08:40:14,360] jepsen worker 932 - jepsen.util 3932 :info :add 10246 :RequestTimeoutException INFO [2022-01-10 08:40:14,371] jepsen worker 708 - jepsen.util 3708 :info :add 10248 :RequestTimeoutException INFO [2022-01-10 08:40:14,373] jepsen worker 929 - jepsen.util 2929 :info :add 10247 :RequestTimeoutException INFO [2022-01-10 08:40:14,381] jepsen worker 148 - jepsen.util 2148 :info :add 10250 :RequestTimeoutException INFO [2022-01-10 08:40:14,397] jepsen worker 142 - jepsen.util 2142 :info :add 10251 :RequestTimeoutException INFO [2022-01-10 08:40:14,443] jepsen worker 920 - jepsen.util 3920 :info :add 10253 :RequestTimeoutException INFO [2022-01-10 08:40:14,451] jepsen worker 444 - jepsen.util 3444 :info :add 10254 :RequestTimeoutException INFO [2022-01-10 08:40:14,463] jepsen worker 513 - jepsen.util 2513 :info :add 10255 :RequestTimeoutException INFO [2022-01-10 08:40:14,464] jepsen worker 403 - jepsen.util 2403 :ok :add 10365 INFO [2022-01-10 08:40:14,464] jepsen worker 355 - jepsen.util 3355 :ok :add 10351 INFO [2022-01-10 08:40:14,467] jepsen worker 978 - jepsen.util 3978 :info :add 10257 :RequestTimeoutException INFO [2022-01-10 08:40:14,469] jepsen worker 240 - jepsen.util 2240 :ok :add 10349 INFO [2022-01-10 08:40:14,469] jepsen worker 576 - jepsen.util 2576 :ok :add 10752 INFO [2022-01-10 08:40:14,469] jepsen worker 659 - jepsen.util 4659 :ok :add 10843 INFO [2022-01-10 08:40:14,469] jepsen worker 73 - jepsen.util 2073 :ok :add 10587 INFO [2022-01-10 08:40:14,470] jepsen worker 339 - jepsen.util 2339 :info :add 10258 :RequestTimeoutException INFO [2022-01-10 08:40:14,472] jepsen worker 776 - jepsen.util 2776 :info :add 10256 :RequestTimeoutException INFO [2022-01-10 08:40:14,472] jepsen worker 195 - jepsen.util 3195 :info :add 10259 :RequestTimeoutException INFO [2022-01-10 08:40:14,481] jepsen worker 733 - jepsen.util 2733 :info :add 10260 :RequestTimeoutException INFO [2022-01-10 08:40:14,482] jepsen worker 946 - jepsen.util 2946 :info :add 10262 :RequestTimeoutException INFO [2022-01-10 08:40:14,483] jepsen worker 398 - jepsen.util 3398 :info :add 10261 :RequestTimeoutException INFO [2022-01-10 08:40:14,503] jepsen worker 947 - jepsen.util 2947 :info :add 10265 :RequestTimeoutException INFO [2022-01-10 08:40:14,512] jepsen worker 25 - jepsen.util 3025 :info :add 10268 :RequestTimeoutException INFO [2022-01-10 08:40:14,517] jepsen worker 490 - jepsen.util 3490 :info :add 10269 :RequestTimeoutException INFO [2022-01-10 08:40:14,520] jepsen worker 588 - jepsen.util 2588 :info :add 10270 :RequestTimeoutException INFO [2022-01-10 08:40:14,533] jepsen worker 619 - jepsen.util 3619 :info :add 10271 :RequestTimeoutException INFO [2022-01-10 08:40:14,541] jepsen worker 208 - jepsen.util 2208 :info :add 10272 :RequestTimeoutException INFO [2022-01-10 08:40:14,542] jepsen worker 524 - jepsen.util 3524 :info :add 10274 :RequestTimeoutException INFO [2022-01-10 08:40:14,543] jepsen worker 907 - jepsen.util 2907 :info :add 10273 :RequestTimeoutException WARN [2022-01-10 08:40:14,546] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":13626125,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","total_dispatch_duration_us":13626125,"total_server_duration_us":9006609,"encode_duration_us":316,"operation_id":"0x2bf1","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":10086197},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","encode_duration_us":256,"operation_id":"0x2bf7","timeout_ms":10000,"total_duration_us":10050911},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000003/000000001BBE5521","last_local_socket":"172.23.104.79:46178","encode_duration_us":427,"operation_id":"0x2bf8","timeout_ms":10000,"total_duration_us":10045878},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","encode_duration_us":304,"operation_id":"0x2c2e","timeout_ms":10000,"total_duration_us":10010966},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","encode_duration_us":316,"operation_id":"0x2e4d","timeout_ms":10000,"total_duration_us":10010924},{"operation_name":"insert","last_dispatch_duration_us":15210361,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","total_dispatch_duration_us":15210361,"total_server_duration_us":9015090,"encode_duration_us":263,"operation_id":"0x2945","timeout_ms":10000,"last_server_duration_us":9015090,"total_duration_us":10010812},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","encode_duration_us":367,"operation_id":"0x2ea9","timeout_ms":10000,"total_duration_us":10010685},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","encode_duration_us":279,"operation_id":"0x299f","timeout_ms":10000,"total_duration_us":10010278},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","encode_duration_us":349,"operation_id":"0x2b31","timeout_ms":10000,"total_duration_us":10010272},{"operation_name":"insert","encode_duration_us":429,"operation_id":"0x29b5","timeout_ms":10000,"total_duration_us":10010182}],"total_count":182}} INFO [2022-01-10 08:40:14,547] jepsen worker 896 - jepsen.util 3896 :info :add 10275 :RequestTimeoutException WARN [2022-01-10 08:40:14,547] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":12909008,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12909008,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":6484774,"encode_duration_us":392,"operation_id":"0x2a25","timeout_ms":10000,"last_server_duration_us":6484774,"total_duration_us":10009513},{"operation_name":"insert","last_dispatch_duration_us":18038796,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18038796,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":9014029,"encode_duration_us":310,"operation_id":"0x2a3b","timeout_ms":10000,"last_server_duration_us":9014029,"total_duration_us":10009589},{"operation_name":"insert","last_dispatch_duration_us":18034364,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18034364,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":9001311,"encode_duration_us":343,"operation_id":"0x2a52","timeout_ms":10000,"last_server_duration_us":9001311,"total_duration_us":10009645},{"operation_name":"insert","last_dispatch_duration_us":19394438,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19394438,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":9007669,"encode_duration_us":478,"operation_id":"0x23c3","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10009798},{"operation_name":"insert","last_dispatch_duration_us":15210361,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15210361,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":9015090,"encode_duration_us":263,"operation_id":"0x2945","timeout_ms":10000,"last_server_duration_us":9015090,"total_duration_us":10010812},{"operation_name":"insert","last_dispatch_duration_us":18007837,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18007837,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":9007669,"encode_duration_us":379,"operation_id":"0x2701","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10011567},{"operation_name":"insert","last_dispatch_duration_us":27261796,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":27261796,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":9005550,"encode_duration_us":265,"operation_id":"0x20d1","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10010624},{"operation_name":"insert","last_dispatch_duration_us":13626125,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13626125,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":9006609,"encode_duration_us":316,"operation_id":"0x2bf1","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":10086197},{"operation_name":"insert","last_dispatch_duration_us":27077379,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":27077379,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":9004490,"encode_duration_us":288,"operation_id":"0x20ef","timeout_ms":10000,"last_server_duration_us":9004490,"total_duration_us":10010448},{"operation_name":"insert","last_dispatch_duration_us":29713679,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":29713679,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":714329,"encode_duration_us":276,"operation_id":"0x2256","timeout_ms":10000,"last_server_duration_us":714329,"total_duration_us":10011130}],"total_count":121}} INFO [2022-01-10 08:40:14,550] jepsen worker 624 - jepsen.util 3624 :info :add 10276 :RequestTimeoutException INFO [2022-01-10 08:40:14,553] jepsen worker 292 - jepsen.util 3292 :info :add 10277 :RequestTimeoutException INFO [2022-01-10 08:40:14,561] jepsen worker 200 - jepsen.util 3200 :info :add 10278 :RequestTimeoutException INFO [2022-01-10 08:40:14,562] jepsen worker 78 - jepsen.util 2078 :info :add 10280 :RequestTimeoutException INFO [2022-01-10 08:40:14,563] jepsen worker 477 - jepsen.util 3477 :info :add 10279 :RequestTimeoutException INFO [2022-01-10 08:40:14,567] jepsen worker 654 - jepsen.util 3654 :info :add 10281 :RequestTimeoutException INFO [2022-01-10 08:40:14,570] jepsen worker 154 - jepsen.util 2154 :info :add 10282 :RequestTimeoutException INFO [2022-01-10 08:40:14,572] jepsen worker 105 - jepsen.util 3105 :info :add 10283 :RequestTimeoutException INFO [2022-01-10 08:40:14,592] jepsen worker 744 - jepsen.util 3744 :info :add 10286 :RequestTimeoutException INFO [2022-01-10 08:40:14,597] jepsen worker 1 - jepsen.util 3001 :info :add 10287 :RequestTimeoutException INFO [2022-01-10 08:40:14,600] jepsen worker 695 - jepsen.util 3695 :info :add 10288 :RequestTimeoutException INFO [2022-01-10 08:40:14,600] jepsen worker 693 - jepsen.util 3693 :info :add 10290 :RequestTimeoutException INFO [2022-01-10 08:40:14,603] jepsen worker 456 - jepsen.util 3456 :info :add 10291 :RequestTimeoutException INFO [2022-01-10 08:40:14,603] jepsen worker 851 - jepsen.util 2851 :info :add 10289 :RequestTimeoutException INFO [2022-01-10 08:40:14,612] jepsen worker 127 - jepsen.util 2127 :info :add 10292 :RequestTimeoutException INFO [2022-01-10 08:40:14,617] jepsen worker 382 - jepsen.util 3382 :info :add 10293 :RequestTimeoutException INFO [2022-01-10 08:40:14,620] jepsen worker 486 - jepsen.util 2486 :info :add 10294 :RequestTimeoutException INFO [2022-01-10 08:40:14,622] jepsen worker 27 - jepsen.util 2027 :info :add 10295 :RequestTimeoutException INFO [2022-01-10 08:40:14,631] jepsen worker 2 - jepsen.util 2002 :info :add 10296 :RequestTimeoutException INFO [2022-01-10 08:40:14,633] jepsen worker 55 - jepsen.util 2055 :ok :add 10532 INFO [2022-01-10 08:40:14,634] jepsen worker 542 - jepsen.util 2542 :info :add 10297 :RequestTimeoutException WARN [2022-01-10 08:40:14,634] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":26302226,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":26302226,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":9004490,"encode_duration_us":323,"operation_id":"0x2136","timeout_ms":10000,"last_server_duration_us":9004490,"total_duration_us":10009280},{"operation_name":"insert","last_dispatch_duration_us":29569822,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":29569822,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":9004490,"encode_duration_us":272,"operation_id":"0x1f06","timeout_ms":10000,"last_server_duration_us":9004490,"total_duration_us":10009352},{"operation_name":"insert","last_dispatch_duration_us":22197276,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":22197276,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":9002371,"encode_duration_us":475,"operation_id":"0x25f5","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10009491},{"operation_name":"insert","last_dispatch_duration_us":25349906,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":25349906,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":9008729,"encode_duration_us":289,"operation_id":"0x2310","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10009749},{"operation_name":"insert","last_dispatch_duration_us":22525209,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":22525209,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":9010849,"encode_duration_us":252,"operation_id":"0x2554","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10009397},{"operation_name":"insert","last_dispatch_duration_us":28363457,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":28363457,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":9009789,"encode_duration_us":333,"operation_id":"0x1fcc","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10009808},{"operation_name":"insert","last_dispatch_duration_us":28683238,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":28683238,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":9012969,"encode_duration_us":295,"operation_id":"0x1fa8","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10009906},{"operation_name":"insert","last_dispatch_duration_us":9852559,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":9852559,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":4447192,"encode_duration_us":258,"operation_id":"0x2d53","timeout_ms":10000,"last_server_duration_us":4447192,"total_duration_us":10010632},{"operation_name":"insert","last_dispatch_duration_us":24483404,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":24483404,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":9005550,"encode_duration_us":259,"operation_id":"0x23ec","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10010748},{"operation_name":"insert","last_dispatch_duration_us":25644075,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25644075,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":7878116,"encode_duration_us":311,"operation_id":"0x20d0","timeout_ms":10000,"last_server_duration_us":7878116,"total_duration_us":10009961}],"total_count":59}} WARN [2022-01-10 08:40:14,634] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","encode_duration_us":433,"operation_id":"0x2bf0","timeout_ms":10000,"total_duration_us":10093066},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","encode_duration_us":311,"operation_id":"0x2bf6","timeout_ms":10000,"total_duration_us":10056559},{"operation_name":"insert","encode_duration_us":263,"operation_id":"0x298c","timeout_ms":10000,"total_duration_us":10011048},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","encode_duration_us":305,"operation_id":"0x29bd","timeout_ms":10000,"total_duration_us":10011044},{"operation_name":"insert","encode_duration_us":270,"operation_id":"0x29d0","timeout_ms":10000,"total_duration_us":10010644},{"operation_name":"insert","last_dispatch_duration_us":9852559,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","total_dispatch_duration_us":9852559,"total_server_duration_us":4447192,"encode_duration_us":258,"operation_id":"0x2d53","timeout_ms":10000,"last_server_duration_us":4447192,"total_duration_us":10010632},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","encode_duration_us":261,"operation_id":"0x2977","timeout_ms":10000,"total_duration_us":10010475},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","encode_duration_us":341,"operation_id":"0x2a60","timeout_ms":10000,"total_duration_us":10010404},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","encode_duration_us":309,"operation_id":"0x2ccb","timeout_ms":10000,"total_duration_us":10010395},{"operation_name":"insert","encode_duration_us":345,"operation_id":"0x2de8","timeout_ms":10000,"total_duration_us":10010378}],"total_count":170}} INFO [2022-01-10 08:40:14,642] jepsen worker 124 - jepsen.util 3124 :info :add 10298 :RequestTimeoutException INFO [2022-01-10 08:40:14,663] jepsen worker 212 - jepsen.util 2212 :info :add 10301 :RequestTimeoutException INFO [2022-01-10 08:40:14,663] jepsen worker 518 - jepsen.util 3518 :info :add 10300 :RequestTimeoutException INFO [2022-01-10 08:40:14,663] jepsen worker 147 - jepsen.util 2147 :info :add 10302 :RequestTimeoutException INFO [2022-01-10 08:40:14,664] jepsen worker 276 - jepsen.util 3276 :info :add 10303 :RequestTimeoutException INFO [2022-01-10 08:40:14,664] jepsen worker 528 - jepsen.util 2528 :info :add 10299 :RequestTimeoutException WARN [2022-01-10 08:40:14,664] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":308,"operation_id":"0x2bf3","timeout_ms":10000,"total_duration_us":10076662},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","encode_duration_us":311,"operation_id":"0x2bfa","timeout_ms":10000,"total_duration_us":10035532},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":316,"operation_id":"0x2de5","timeout_ms":10000,"total_duration_us":10010883},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":311,"operation_id":"0x2cd4","timeout_ms":10000,"total_duration_us":10010778},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","encode_duration_us":255,"operation_id":"0x2c77","timeout_ms":10000,"total_duration_us":10010710},{"operation_name":"insert","last_dispatch_duration_us":11927891,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000005/0000000013738E6A","last_local_socket":"172.23.104.79:41944","total_dispatch_duration_us":11927891,"total_server_duration_us":904638,"encode_duration_us":348,"operation_id":"0x2a7d","timeout_ms":10000,"last_server_duration_us":904638,"total_duration_us":10010577},{"operation_name":"insert","last_dispatch_duration_us":12661231,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","total_dispatch_duration_us":12661231,"total_server_duration_us":5437223,"encode_duration_us":469,"operation_id":"0x2c06","timeout_ms":10000,"last_server_duration_us":5437223,"total_duration_us":10010521},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","encode_duration_us":311,"operation_id":"0x2e41","timeout_ms":10000,"total_duration_us":10010352},{"operation_name":"insert","encode_duration_us":304,"operation_id":"0x2ac8","timeout_ms":10000,"total_duration_us":10010284},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000005/0000000013738E6A","last_local_socket":"172.23.104.79:41944","encode_duration_us":328,"operation_id":"0x2d8d","timeout_ms":10000,"total_duration_us":10010249}],"total_count":162}} WARN [2022-01-10 08:40:14,664] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","encode_duration_us":324,"operation_id":"0x2bf2","timeout_ms":10000,"total_duration_us":10078580},{"operation_name":"insert","encode_duration_us":316,"operation_id":"0x2bf9","timeout_ms":10000,"total_duration_us":10037204},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","encode_duration_us":312,"operation_id":"0x2e6a","timeout_ms":10000,"total_duration_us":10010746},{"operation_name":"insert","encode_duration_us":254,"operation_id":"0x29dc","timeout_ms":10000,"total_duration_us":10010732},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":320,"operation_id":"0x2bc2","timeout_ms":10000,"total_duration_us":10010718},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":303,"operation_id":"0x2bb3","timeout_ms":10000,"total_duration_us":10010569},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":299,"operation_id":"0x2c05","timeout_ms":10000,"total_duration_us":10010523},{"operation_name":"insert","last_dispatch_duration_us":10599562,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000004/00000000EF8F24DE","last_local_socket":"172.23.104.79:51736","total_dispatch_duration_us":10599562,"total_server_duration_us":4259713,"encode_duration_us":377,"operation_id":"0x2def","timeout_ms":10000,"last_server_duration_us":4259713,"total_duration_us":10010486},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":304,"operation_id":"0x2aad","timeout_ms":10000,"total_duration_us":10010357},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":255,"operation_id":"0x2985","timeout_ms":10000,"total_duration_us":10010284}],"total_count":168}} WARN [2022-01-10 08:40:14,664] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":16538420,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16538420,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9002371,"encode_duration_us":285,"operation_id":"0x26d6","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10008810},{"operation_name":"insert","last_dispatch_duration_us":20617912,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20617912,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9005550,"encode_duration_us":260,"operation_id":"0x1f75","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10008907},{"operation_name":"insert","last_dispatch_duration_us":17912065,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17912065,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9007669,"encode_duration_us":263,"operation_id":"0x244a","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10009097},{"operation_name":"insert","last_dispatch_duration_us":17918783,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17918783,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9008729,"encode_duration_us":293,"operation_id":"0x24e3","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10009772},{"operation_name":"insert","last_dispatch_duration_us":20172328,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":20172328,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":8512301,"encode_duration_us":298,"operation_id":"0x250c","timeout_ms":10000,"last_server_duration_us":8512301,"total_duration_us":10009814},{"operation_name":"insert","last_dispatch_duration_us":16546117,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16546117,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":5312143,"encode_duration_us":262,"operation_id":"0x247c","timeout_ms":10000,"last_server_duration_us":5312143,"total_duration_us":10009943},{"operation_name":"insert","last_dispatch_duration_us":25228598,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":25228598,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":1822908,"encode_duration_us":260,"operation_id":"0x24f6","timeout_ms":10000,"last_server_duration_us":1822908,"total_duration_us":10010258},{"operation_name":"insert","last_dispatch_duration_us":10599562,"last_local_id":"2B99390F00000004/00000000EF8F24DE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10599562,"last_local_socket":"172.23.104.79:51736","total_server_duration_us":4259713,"encode_duration_us":377,"operation_id":"0x2def","timeout_ms":10000,"last_server_duration_us":4259713,"total_duration_us":10010486},{"operation_name":"insert","last_dispatch_duration_us":21873905,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":21873905,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":4178343,"encode_duration_us":292,"operation_id":"0x247b","timeout_ms":10000,"last_server_duration_us":4178343,"total_duration_us":10011661},{"operation_name":"insert","last_dispatch_duration_us":22074360,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":22074360,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":4193645,"encode_duration_us":255,"operation_id":"0x2438","timeout_ms":10000,"last_server_duration_us":4193645,"total_duration_us":10010728}],"total_count":57}} INFO [2022-01-10 08:40:14,672] jepsen worker 70 - jepsen.util 2070 :info :add 10304 :RequestTimeoutException INFO [2022-01-10 08:40:14,676] jepsen worker 84 - jepsen.util 3084 :info :add 10305 :RequestTimeoutException INFO [2022-01-10 08:40:14,680] jepsen worker 175 - jepsen.util 2175 :info :add 10306 :RequestTimeoutException INFO [2022-01-10 08:40:14,683] jepsen worker 568 - jepsen.util 2568 :info :add 10307 :RequestTimeoutException INFO [2022-01-10 08:40:14,691] jepsen worker 724 - jepsen.util 3724 :info :add 10308 :RequestTimeoutException INFO [2022-01-10 08:40:14,693] jepsen worker 405 - jepsen.util 3405 :info :add 10309 :RequestTimeoutException INFO [2022-01-10 08:40:14,697] jepsen worker 185 - jepsen.util 3185 :info :add 10311 :RequestTimeoutException INFO [2022-01-10 08:40:14,702] jepsen worker 465 - jepsen.util 3465 :info :add 10310 :RequestTimeoutException INFO [2022-01-10 08:40:14,710] jepsen worker 994 - jepsen.util 3994 :info :add 10312 :RequestTimeoutException INFO [2022-01-10 08:40:14,713] jepsen worker 359 - jepsen.util 3359 :info :add 10313 :RequestTimeoutException INFO [2022-01-10 08:40:14,721] jepsen worker 60 - jepsen.util 2060 :info :add 10315 :RequestTimeoutException INFO [2022-01-10 08:40:14,732] jepsen worker 207 - jepsen.util 3207 :info :add 10317 :RequestTimeoutException INFO [2022-01-10 08:40:14,733] jepsen worker 42 - jepsen.util 3042 :info :add 10319 :RequestTimeoutException WARN [2022-01-10 08:40:14,734] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","encode_duration_us":305,"operation_id":"0x2bfb","timeout_ms":10000,"total_duration_us":10031976},{"operation_name":"insert","last_dispatch_duration_us":11999311,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","total_dispatch_duration_us":11999311,"total_server_duration_us":4216642,"encode_duration_us":261,"operation_id":"0x2981","timeout_ms":10000,"last_server_duration_us":4216642,"total_duration_us":10011122},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","encode_duration_us":318,"operation_id":"0x2d8e","timeout_ms":10000,"total_duration_us":10010942},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","encode_duration_us":330,"operation_id":"0x2e96","timeout_ms":10000,"total_duration_us":10010898},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","encode_duration_us":302,"operation_id":"0x2ac9","timeout_ms":10000,"total_duration_us":10010830},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","encode_duration_us":346,"operation_id":"0x2c36","timeout_ms":10000,"total_duration_us":10010808},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","encode_duration_us":309,"operation_id":"0x2e90","timeout_ms":10000,"total_duration_us":10010801},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","encode_duration_us":257,"operation_id":"0x2cb4","timeout_ms":10000,"total_duration_us":10010759},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","encode_duration_us":312,"operation_id":"0x2aa3","timeout_ms":10000,"total_duration_us":10010670},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","encode_duration_us":258,"operation_id":"0x2999","timeout_ms":10000,"total_duration_us":10010468}],"total_count":182}} WARN [2022-01-10 08:40:14,735] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":17820562,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17820562,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":9011909,"encode_duration_us":297,"operation_id":"0x2672","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10010451},{"operation_name":"insert","last_dispatch_duration_us":16880183,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16880183,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9018271,"encode_duration_us":257,"operation_id":"0x28c4","timeout_ms":10000,"last_server_duration_us":9018271,"total_duration_us":10010456},{"operation_name":"insert","last_dispatch_duration_us":14545219,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14545219,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":9008729,"encode_duration_us":386,"operation_id":"0x28ca","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10010838},{"operation_name":"insert","last_dispatch_duration_us":14467839,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14467839,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":9004490,"encode_duration_us":254,"operation_id":"0x254a","timeout_ms":10000,"last_server_duration_us":9004490,"total_duration_us":10010666},{"operation_name":"insert","last_dispatch_duration_us":16344673,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16344673,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9005550,"encode_duration_us":260,"operation_id":"0x26c4","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10010513},{"operation_name":"insert","last_dispatch_duration_us":12891913,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12891913,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":6485696,"encode_duration_us":256,"operation_id":"0x291d","timeout_ms":10000,"last_server_duration_us":6485696,"total_duration_us":10010993},{"operation_name":"insert","last_dispatch_duration_us":16069992,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16069992,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9007669,"encode_duration_us":291,"operation_id":"0x27b9","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10010888},{"operation_name":"insert","last_dispatch_duration_us":13916483,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13916483,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9006609,"encode_duration_us":273,"operation_id":"0x2906","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":10010688},{"operation_name":"insert","last_dispatch_duration_us":10774489,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":10774489,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":5811064,"encode_duration_us":319,"operation_id":"0x276a","timeout_ms":10000,"last_server_duration_us":5811064,"total_duration_us":10010925},{"operation_name":"insert","last_dispatch_duration_us":11999311,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11999311,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":4216642,"encode_duration_us":261,"operation_id":"0x2981","timeout_ms":10000,"last_server_duration_us":4216642,"total_duration_us":10011122}],"total_count":139}} INFO [2022-01-10 08:40:14,737] jepsen worker 296 - jepsen.util 3296 :info :add 10318 :RequestTimeoutException INFO [2022-01-10 08:40:14,752] jepsen worker 990 - jepsen.util 2990 :info :add 10322 :RequestTimeoutException INFO [2022-01-10 08:40:14,761] jepsen worker 400 - jepsen.util 3400 :info :add 10323 :RequestTimeoutException WARN [2022-01-10 08:40:14,765] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":22912716,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":22912716,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":4056032,"encode_duration_us":298,"operation_id":"0x2660","timeout_ms":10000,"last_server_duration_us":4056032,"total_duration_us":10009900},{"operation_name":"insert","last_dispatch_duration_us":17163012,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17163012,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9012969,"encode_duration_us":275,"operation_id":"0x23aa","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10009926},{"operation_name":"insert","last_dispatch_duration_us":24018546,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":24018546,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":8465794,"encode_duration_us":312,"operation_id":"0x22b7","timeout_ms":10000,"last_server_duration_us":8465794,"total_duration_us":10010179},{"operation_name":"insert","last_dispatch_duration_us":11518199,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":11518199,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":5494643,"encode_duration_us":260,"operation_id":"0x291c","timeout_ms":10000,"last_server_duration_us":5494643,"total_duration_us":10010082},{"operation_name":"insert","last_dispatch_duration_us":22837182,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":22837182,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9005550,"encode_duration_us":292,"operation_id":"0x240f","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10010297},{"operation_name":"insert","last_dispatch_duration_us":12661231,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":12661231,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":5437223,"encode_duration_us":469,"operation_id":"0x2c06","timeout_ms":10000,"last_server_duration_us":5437223,"total_duration_us":10010521},{"operation_name":"insert","last_dispatch_duration_us":11927891,"last_local_id":"2B99390F00000005/0000000013738E6A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11927891,"last_local_socket":"172.23.104.79:41944","total_server_duration_us":904638,"encode_duration_us":348,"operation_id":"0x2a7d","timeout_ms":10000,"last_server_duration_us":904638,"total_duration_us":10010577},{"operation_name":"insert","last_dispatch_duration_us":15546366,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15546366,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9006609,"encode_duration_us":253,"operation_id":"0x2729","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":10010938},{"operation_name":"insert","last_dispatch_duration_us":17261401,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17261401,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9018271,"encode_duration_us":257,"operation_id":"0x2623","timeout_ms":10000,"last_server_duration_us":9018271,"total_duration_us":10010447},{"operation_name":"insert","last_dispatch_duration_us":13517629,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13517629,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":7613725,"encode_duration_us":255,"operation_id":"0x274e","timeout_ms":10000,"last_server_duration_us":7613725,"total_duration_us":10011058}],"total_count":124}} INFO [2022-01-10 08:40:14,765] jepsen worker 635 - jepsen.util 3635 :ok :add 10350 INFO [2022-01-10 08:40:14,765] jepsen worker 655 - jepsen.util 3655 :ok :add 10767 INFO [2022-01-10 08:40:14,765] jepsen worker 106 - jepsen.util 4106 :ok :add 10753 INFO [2022-01-10 08:40:14,766] jepsen worker 249 - jepsen.util 3249 :ok :add 10586 INFO [2022-01-10 08:40:14,766] jepsen worker 648 - jepsen.util 3648 :ok :add 10348 INFO [2022-01-10 08:40:14,766] jepsen worker 367 - jepsen.util 2367 :ok :add 10422 INFO [2022-01-10 08:40:14,766] jepsen worker 510 - jepsen.util 3510 :ok :add 10416 INFO [2022-01-10 08:40:14,771] jepsen worker 591 - jepsen.util 3591 :info :add 10325 :RequestTimeoutException INFO [2022-01-10 08:40:14,773] jepsen worker 597 - jepsen.util 3597 :info :add 10324 :RequestTimeoutException INFO [2022-01-10 08:40:14,780] jepsen worker 529 - jepsen.util 2529 :info :add 10327 :RequestTimeoutException INFO [2022-01-10 08:40:14,782] jepsen worker 33 - jepsen.util 2033 :info :add 10326 :RequestTimeoutException INFO [2022-01-10 08:40:14,786] jepsen worker 572 - jepsen.util 3572 :info :add 10328 :RequestTimeoutException INFO [2022-01-10 08:40:14,790] jepsen worker 445 - jepsen.util 3445 :info :add 10329 :RequestTimeoutException INFO [2022-01-10 08:40:14,797] jepsen worker 955 - jepsen.util 2955 :info :add 10330 :RequestTimeoutException INFO [2022-01-10 08:40:14,801] jepsen worker 450 - jepsen.util 3450 :info :add 10332 :RequestTimeoutException INFO [2022-01-10 08:40:14,801] jepsen worker 763 - jepsen.util 2763 :info :add 10334 :RequestTimeoutException INFO [2022-01-10 08:40:14,803] jepsen worker 225 - jepsen.util 2225 :info :add 10333 :RequestTimeoutException INFO [2022-01-10 08:40:14,807] jepsen worker 909 - jepsen.util 3909 :info :add 10335 :RequestTimeoutException INFO [2022-01-10 08:40:14,810] jepsen worker 80 - jepsen.util 3080 :info :add 10336 :RequestTimeoutException INFO [2022-01-10 08:40:14,812] jepsen worker 328 - jepsen.util 2328 :info :add 10337 :RequestTimeoutException INFO [2022-01-10 08:40:14,832] jepsen worker 840 - jepsen.util 2840 :info :add 10340 :RequestTimeoutException INFO [2022-01-10 08:40:14,837] jepsen worker 36 - jepsen.util 2036 :info :add 10341 :RequestTimeoutException INFO [2022-01-10 08:40:14,840] jepsen worker 139 - jepsen.util 3139 :info :add 10342 :RequestTimeoutException INFO [2022-01-10 08:40:14,843] jepsen worker 376 - jepsen.util 3376 :info :add 10343 :RequestTimeoutException INFO [2022-01-10 08:40:14,851] jepsen worker 789 - jepsen.util 3789 :info :add 10344 :RequestTimeoutException INFO [2022-01-10 08:40:14,852] jepsen worker 288 - jepsen.util 3288 :info :add 10346 :RequestTimeoutException INFO [2022-01-10 08:40:14,853] jepsen worker 670 - jepsen.util 2670 :info :add 10345 :RequestTimeoutException INFO [2022-01-10 08:40:14,862] jepsen worker 351 - jepsen.util 2351 :info :add 10347 :RequestTimeoutException INFO [2022-01-10 08:40:14,886] jepsen worker 996 - jepsen.util 3996 :info :add 10354 :RequestTimeoutException INFO [2022-01-10 08:40:14,890] jepsen worker 839 - jepsen.util 2839 :info :add 10355 :RequestTimeoutException INFO [2022-01-10 08:40:14,891] jepsen worker 583 - jepsen.util 3583 :info :add 10357 :RequestTimeoutException INFO [2022-01-10 08:40:14,893] jepsen worker 880 - jepsen.util 3880 :info :add 10356 :RequestTimeoutException INFO [2022-01-10 08:40:14,893] jepsen worker 401 - jepsen.util 2401 :info :add 10358 :RequestTimeoutException INFO [2022-01-10 08:40:14,902] jepsen worker 508 - jepsen.util 3508 :info :add 10359 :RequestTimeoutException INFO [2022-01-10 08:40:14,907] jepsen worker 866 - jepsen.util 3866 :info :add 10360 :RequestTimeoutException INFO [2022-01-10 08:40:14,911] jepsen worker 762 - jepsen.util 3762 :info :add 10361 :RequestTimeoutException INFO [2022-01-10 08:40:14,911] jepsen worker 757 - jepsen.util 2757 :info :add 10363 :RequestTimeoutException INFO [2022-01-10 08:40:14,913] jepsen worker 268 - jepsen.util 3268 :info :add 10362 :RequestTimeoutException INFO [2022-01-10 08:40:14,927] jepsen worker 410 - jepsen.util 3410 :info :add 10366 :RequestTimeoutException INFO [2022-01-10 08:40:14,931] jepsen worker 344 - jepsen.util 3344 :info :add 10369 :RequestTimeoutException INFO [2022-01-10 08:40:14,932] jepsen worker 981 - jepsen.util 2981 :info :add 10368 :RequestTimeoutException INFO [2022-01-10 08:40:14,942] jepsen worker 149 - jepsen.util 3149 :info :add 10371 :RequestTimeoutException INFO [2022-01-10 08:40:14,943] jepsen worker 885 - jepsen.util 3885 :info :add 10370 :RequestTimeoutException INFO [2022-01-10 08:40:14,974] jepsen worker 910 - jepsen.util 2910 :info :add 10372 :RequestTimeoutException INFO [2022-01-10 08:40:14,981] jepsen worker 335 - jepsen.util 3335 :info :add 10373 :RequestTimeoutException INFO [2022-01-10 08:40:14,983] jepsen worker 580 - jepsen.util 3580 :info :add 10374 :RequestTimeoutException INFO [2022-01-10 08:40:14,990] jepsen worker 573 - jepsen.util 3573 :info :add 10375 :RequestTimeoutException INFO [2022-01-10 08:40:14,991] jepsen worker 126 - jepsen.util 3126 :ok :add 10640 INFO [2022-01-10 08:40:14,993] jepsen worker 782 - jepsen.util 3782 :info :add 10376 :RequestTimeoutException INFO [2022-01-10 08:40:15,027] jepsen worker 987 - jepsen.util 2987 :info :add 10377 :RequestTimeoutException INFO [2022-01-10 08:40:15,030] jepsen worker 815 - jepsen.util 2815 :info :add 10378 :RequestTimeoutException INFO [2022-01-10 08:40:15,033] jepsen worker 819 - jepsen.util 2819 :info :add 10379 :RequestTimeoutException INFO [2022-01-10 08:40:15,041] jepsen worker 838 - jepsen.util 2838 :info :add 10380 :RequestTimeoutException INFO [2022-01-10 08:40:15,044] jepsen worker 792 - jepsen.util 3792 :info :add 10381 :RequestTimeoutException INFO [2022-01-10 08:40:15,052] jepsen worker 778 - jepsen.util 3778 :info :add 10382 :RequestTimeoutException INFO [2022-01-10 08:40:15,056] jepsen worker 615 - jepsen.util 3615 :info :add 10383 :RequestTimeoutException INFO [2022-01-10 08:40:15,161] jepsen worker 394 - jepsen.util 2394 :info :add 10384 :RequestTimeoutException INFO [2022-01-10 08:40:15,166] jepsen worker 515 - jepsen.util 4515 :ok :add 10897 INFO [2022-01-10 08:40:15,166] jepsen worker 805 - jepsen.util 3805 :ok :add 10933 INFO [2022-01-10 08:40:15,166] jepsen worker 423 - jepsen.util 2423 :ok :add 10385 INFO [2022-01-10 08:40:15,167] jepsen worker 430 - jepsen.util 3430 :ok :add 10567 INFO [2022-01-10 08:40:15,167] jepsen worker 989 - jepsen.util 3989 :ok :add 10622 INFO [2022-01-10 08:40:15,167] jepsen worker 810 - jepsen.util 3810 :ok :add 10616 INFO [2022-01-10 08:40:15,191] jepsen worker 509 - jepsen.util 3509 :info :add 10386 :RequestTimeoutException INFO [2022-01-10 08:40:15,212] jepsen worker 560 - jepsen.util 3560 :info :add 10388 :RequestTimeoutException INFO [2022-01-10 08:40:15,217] jepsen worker 281 - jepsen.util 2281 :info :add 10389 :RequestTimeoutException INFO [2022-01-10 08:40:15,220] jepsen worker 451 - jepsen.util 2451 :info :add 10390 :RequestTimeoutException INFO [2022-01-10 08:40:15,222] jepsen worker 737 - jepsen.util 2737 :info :add 10391 :RequestTimeoutException INFO [2022-01-10 08:40:15,231] jepsen worker 304 - jepsen.util 2304 :info :add 10392 :RequestTimeoutException INFO [2022-01-10 08:40:15,240] jepsen worker 970 - jepsen.util 2970 :info :add 10393 :RequestTimeoutException INFO [2022-01-10 08:40:15,242] jepsen worker 439 - jepsen.util 3439 :info :add 10395 :RequestTimeoutException INFO [2022-01-10 08:40:15,243] jepsen worker 888 - jepsen.util 3888 :info :add 10394 :RequestTimeoutException INFO [2022-01-10 08:40:15,246] jepsen worker 712 - jepsen.util 3712 :info :add 10396 :RequestTimeoutException INFO [2022-01-10 08:40:15,247] jepsen worker 676 - jepsen.util 3676 :info :add 10397 :RequestTimeoutException INFO [2022-01-10 08:40:15,255] jepsen worker 505 - jepsen.util 4505 :ok :add 10657 INFO [2022-01-10 08:40:15,255] jepsen worker 666 - jepsen.util 3666 :ok :add 10663 INFO [2022-01-10 08:40:15,260] jepsen worker 855 - jepsen.util 2855 :info :add 10398 :RequestTimeoutException INFO [2022-01-10 08:40:15,261] jepsen worker 143 - jepsen.util 3143 :info :add 10401 :RequestTimeoutException INFO [2022-01-10 08:40:15,263] jepsen worker 682 - jepsen.util 3682 :info :add 10400 :RequestTimeoutException INFO [2022-01-10 08:40:15,264] jepsen worker 593 - jepsen.util 2593 :info :add 10399 :RequestTimeoutException INFO [2022-01-10 08:40:15,273] jepsen worker 774 - jepsen.util 2774 :info :add 10402 :RequestTimeoutException INFO [2022-01-10 08:40:15,273] jepsen worker 677 - jepsen.util 3677 :info :add 10403 :RequestTimeoutException INFO [2022-01-10 08:40:15,280] jepsen worker 964 - jepsen.util 2964 :info :add 10406 :RequestTimeoutException INFO [2022-01-10 08:40:15,282] jepsen worker 845 - jepsen.util 3845 :info :add 10404 :RequestTimeoutException INFO [2022-01-10 08:40:15,282] jepsen worker 74 - jepsen.util 3074 :info :add 10407 :RequestTimeoutException INFO [2022-01-10 08:40:15,286] jepsen worker 790 - jepsen.util 3790 :info :add 10405 :RequestTimeoutException INFO [2022-01-10 08:40:15,291] jepsen worker 419 - jepsen.util 3419 :info :add 10408 :RequestTimeoutException INFO [2022-01-10 08:40:15,292] jepsen worker 41 - jepsen.util 2041 :info :add 10410 :RequestTimeoutException INFO [2022-01-10 08:40:15,294] jepsen worker 125 - jepsen.util 3125 :info :add 10409 :RequestTimeoutException INFO [2022-01-10 08:40:15,297] jepsen worker 960 - jepsen.util 2960 :info :add 10411 :RequestTimeoutException INFO [2022-01-10 08:40:15,300] jepsen worker 113 - jepsen.util 3113 :info :add 10412 :RequestTimeoutException INFO [2022-01-10 08:40:15,303] jepsen worker 274 - jepsen.util 2274 :info :add 10413 :RequestTimeoutException INFO [2022-01-10 08:40:15,326] jepsen worker 260 - jepsen.util 3260 :info :add 10417 :RequestTimeoutException INFO [2022-01-10 08:40:15,330] jepsen worker 31 - jepsen.util 3031 :info :add 10418 :RequestTimeoutException INFO [2022-01-10 08:40:15,332] jepsen worker 384 - jepsen.util 3384 :info :add 10419 :RequestTimeoutException INFO [2022-01-10 08:40:15,357] jepsen worker 169 - jepsen.util 2169 :info :add 10423 :RequestTimeoutException INFO [2022-01-10 08:40:15,360] jepsen worker 283 - jepsen.util 3283 :info :add 10424 :RequestTimeoutException INFO [2022-01-10 08:40:15,363] jepsen worker 40 - jepsen.util 3040 :info :add 10425 :RequestTimeoutException INFO [2022-01-10 08:40:15,371] jepsen worker 9 - jepsen.util 3009 :info :add 10426 :RequestTimeoutException INFO [2022-01-10 08:40:15,372] jepsen worker 715 - jepsen.util 2715 :info :add 10428 :RequestTimeoutException INFO [2022-01-10 08:40:15,374] jepsen worker 5 - jepsen.util 2005 :info :add 10427 :RequestTimeoutException INFO [2022-01-10 08:40:15,376] jepsen worker 543 - jepsen.util 3543 :info :add 10429 :RequestTimeoutException INFO [2022-01-10 08:40:15,380] jepsen worker 807 - jepsen.util 2807 :info :add 10430 :RequestTimeoutException INFO [2022-01-10 08:40:15,391] jepsen worker 264 - jepsen.util 3264 :info :add 10432 :RequestTimeoutException INFO [2022-01-10 08:40:15,393] jepsen worker 37 - jepsen.util 2037 :info :add 10431 :RequestTimeoutException INFO [2022-01-10 08:40:15,393] jepsen worker 57 - jepsen.util 3057 :info :add 10433 :RequestTimeoutException INFO [2022-01-10 08:40:15,397] jepsen worker 561 - jepsen.util 2561 :info :add 10435 :RequestTimeoutException INFO [2022-01-10 08:40:15,400] jepsen worker 72 - jepsen.util 3072 :info :add 10436 :RequestTimeoutException INFO [2022-01-10 08:40:15,402] jepsen worker 638 - jepsen.util 3638 :info :add 10434 :RequestTimeoutException INFO [2022-01-10 08:40:15,403] jepsen worker 438 - jepsen.util 3438 :info :add 10437 :RequestTimeoutException INFO [2022-01-10 08:40:15,422] jepsen worker 141 - jepsen.util 3141 :info :add 10440 :RequestTimeoutException INFO [2022-01-10 08:40:15,427] jepsen worker 611 - jepsen.util 3611 :info :add 10441 :RequestTimeoutException INFO [2022-01-10 08:40:15,453] jepsen worker 107 - jepsen.util 3107 :info :add 10443 :RequestTimeoutException INFO [2022-01-10 08:40:15,461] jepsen worker 972 - jepsen.util 3972 :info :add 10444 :RequestTimeoutException INFO [2022-01-10 08:40:15,463] jepsen worker 812 - jepsen.util 3812 :info :add 10445 :RequestTimeoutException INFO [2022-01-10 08:40:15,467] jepsen worker 157 - jepsen.util 2157 :info :add 10447 :RequestTimeoutException INFO [2022-01-10 08:40:15,470] jepsen worker 565 - jepsen.util 2565 :info :add 10448 :RequestTimeoutException INFO [2022-01-10 08:40:15,472] jepsen worker 459 - jepsen.util 2459 :info :add 10446 :RequestTimeoutException INFO [2022-01-10 08:40:15,483] jepsen worker 88 - jepsen.util 2088 :info :add 10449 :RequestTimeoutException INFO [2022-01-10 08:40:15,491] jepsen worker 211 - jepsen.util 2211 :info :add 10450 :RequestTimeoutException INFO [2022-01-10 08:40:15,506] jepsen worker 641 - jepsen.util 4641 :ok :add 10632 INFO [2022-01-10 08:40:15,506] jepsen worker 6 - jepsen.util 4006 :ok :add 11027 INFO [2022-01-10 08:40:15,527] jepsen worker 228 - jepsen.util 3228 :info :add 10453 :RequestTimeoutException INFO [2022-01-10 08:40:15,530] jepsen worker 658 - jepsen.util 2658 :info :add 10454 :RequestTimeoutException INFO [2022-01-10 08:40:15,541] jepsen worker 671 - jepsen.util 3671 :info :add 10456 :RequestTimeoutException INFO [2022-01-10 08:40:15,542] jepsen worker 343 - jepsen.util 3343 :info :add 10458 :RequestTimeoutException INFO [2022-01-10 08:40:15,543] jepsen worker 298 - jepsen.util 3298 :info :add 10455 :RequestTimeoutException INFO [2022-01-10 08:40:15,543] jepsen worker 874 - jepsen.util 3874 :info :add 10457 :RequestTimeoutException INFO [2022-01-10 08:40:15,547] jepsen worker 549 - jepsen.util 2549 :info :add 10459 :RequestTimeoutException INFO [2022-01-10 08:40:15,551] jepsen worker 97 - jepsen.util 3097 :info :add 10460 :RequestTimeoutException INFO [2022-01-10 08:40:15,560] jepsen worker 856 - jepsen.util 3856 :info :add 10462 :RequestTimeoutException INFO [2022-01-10 08:40:15,562] jepsen worker 956 - jepsen.util 3956 :info :add 10461 :RequestTimeoutException INFO [2022-01-10 08:40:15,572] jepsen worker 487 - jepsen.util 2487 :info :add 10464 :RequestTimeoutException INFO [2022-01-10 08:40:15,573] jepsen worker 275 - jepsen.util 2275 :info :add 10463 :RequestTimeoutException INFO [2022-01-10 08:40:15,576] jepsen worker 213 - jepsen.util 2213 :info :add 10465 :RequestTimeoutException INFO [2022-01-10 08:40:15,592] jepsen worker 155 - jepsen.util 2155 :info :add 10467 :RequestTimeoutException INFO [2022-01-10 08:40:15,593] jepsen worker 521 - jepsen.util 2521 :info :add 10468 :RequestTimeoutException INFO [2022-01-10 08:40:15,601] jepsen worker 643 - jepsen.util 3643 :info :add 10470 :RequestTimeoutException INFO [2022-01-10 08:40:15,602] jepsen worker 412 - jepsen.util 3412 :info :add 10469 :RequestTimeoutException INFO [2022-01-10 08:40:15,611] jepsen worker 537 - jepsen.util 2537 :info :add 10471 :RequestTimeoutException INFO [2022-01-10 08:40:15,612] jepsen worker 34 - jepsen.util 2034 :info :add 10472 :RequestTimeoutException INFO [2022-01-10 08:40:15,614] jepsen worker 50 - jepsen.util 3050 :info :add 10473 :RequestTimeoutException INFO [2022-01-10 08:40:15,620] jepsen worker 327 - jepsen.util 2327 :info :add 10474 :RequestTimeoutException INFO [2022-01-10 08:40:15,632] jepsen worker 375 - jepsen.util 3375 :info :add 10476 :RequestTimeoutException INFO [2022-01-10 08:40:15,633] jepsen worker 875 - jepsen.util 3875 :info :add 10475 :RequestTimeoutException INFO [2022-01-10 08:40:15,636] jepsen worker 753 - jepsen.util 3753 :info :add 10477 :RequestTimeoutException INFO [2022-01-10 08:40:15,640] jepsen worker 77 - jepsen.util 3077 :info :add 10478 :RequestTimeoutException INFO [2022-01-10 08:40:15,642] jepsen worker 731 - jepsen.util 3731 :info :add 10479 :RequestTimeoutException INFO [2022-01-10 08:40:15,651] jepsen worker 755 - jepsen.util 2755 :info :add 10480 :RequestTimeoutException INFO [2022-01-10 08:40:15,651] jepsen worker 637 - jepsen.util 3637 :info :add 10482 :RequestTimeoutException INFO [2022-01-10 08:40:15,653] jepsen worker 893 - jepsen.util 3893 :info :add 10481 :RequestTimeoutException INFO [2022-01-10 08:40:15,657] jepsen worker 574 - jepsen.util 3574 :info :add 10483 :RequestTimeoutException INFO [2022-01-10 08:40:15,661] jepsen worker 937 - jepsen.util 3937 :info :add 10486 :RequestTimeoutException INFO [2022-01-10 08:40:15,667] jepsen worker 466 - jepsen.util 3466 :info :add 10485 :RequestTimeoutException INFO [2022-01-10 08:40:15,672] jepsen worker 345 - jepsen.util 3345 :info :add 10488 :RequestTimeoutException INFO [2022-01-10 08:40:15,677] jepsen worker 943 - jepsen.util 3943 :info :add 10489 :RequestTimeoutException INFO [2022-01-10 08:40:15,680] jepsen worker 760 - jepsen.util 3760 :info :add 10490 :RequestTimeoutException INFO [2022-01-10 08:40:15,682] jepsen worker 898 - jepsen.util 3898 :info :add 10491 :RequestTimeoutException INFO [2022-01-10 08:40:15,701] jepsen worker 489 - jepsen.util 3489 :info :add 10492 :RequestTimeoutException INFO [2022-01-10 08:40:15,703] jepsen worker 98 - jepsen.util 2098 :info :add 10493 :RequestTimeoutException INFO [2022-01-10 08:40:15,731] jepsen worker 285 - jepsen.util 3285 :info :add 10494 :RequestTimeoutException INFO [2022-01-10 08:40:15,737] jepsen worker 575 - jepsen.util 3575 :info :add 10495 :RequestTimeoutException INFO [2022-01-10 08:40:15,740] jepsen worker 652 - jepsen.util 3652 :info :add 10496 :RequestTimeoutException INFO [2022-01-10 08:40:15,743] jepsen worker 974 - jepsen.util 3974 :info :add 10497 :RequestTimeoutException INFO [2022-01-10 08:40:15,751] jepsen worker 294 - jepsen.util 3294 :info :add 10498 :RequestTimeoutException INFO [2022-01-10 08:40:15,753] jepsen worker 844 - jepsen.util 3844 :info :add 10499 :RequestTimeoutException INFO [2022-01-10 08:40:15,762] jepsen worker 7 - jepsen.util 2007 :info :add 10500 :RequestTimeoutException INFO [2022-01-10 08:40:15,767] jepsen worker 46 - jepsen.util 3046 :info :add 10501 :RequestTimeoutException INFO [2022-01-10 08:40:15,771] jepsen worker 83 - jepsen.util 3083 :info :add 10504 :RequestTimeoutException INFO [2022-01-10 08:40:15,771] jepsen worker 425 - jepsen.util 3425 :info :add 10502 :RequestTimeoutException INFO [2022-01-10 08:40:15,773] jepsen worker 413 - jepsen.util 2413 :info :add 10503 :RequestTimeoutException INFO [2022-01-10 08:40:15,774] jepsen worker 993 - jepsen.util 3993 :info :add 10505 :RequestTimeoutException INFO [2022-01-10 08:40:15,777] jepsen worker 527 - jepsen.util 3527 :info :add 10507 :RequestTimeoutException INFO [2022-01-10 08:40:15,783] jepsen worker 557 - jepsen.util 3557 :info :add 10506 :RequestTimeoutException INFO [2022-01-10 08:40:15,790] jepsen worker 247 - jepsen.util 2247 :info :add 10508 :RequestTimeoutException INFO [2022-01-10 08:40:15,793] jepsen worker 132 - jepsen.util 2132 :info :add 10509 :RequestTimeoutException INFO [2022-01-10 08:40:15,801] jepsen worker 553 - jepsen.util 3553 :info :add 10510 :RequestTimeoutException INFO [2022-01-10 08:40:15,803] jepsen worker 872 - jepsen.util 2872 :info :add 10511 :RequestTimeoutException INFO [2022-01-10 08:40:15,812] jepsen worker 649 - jepsen.util 3649 :info :add 10512 :RequestTimeoutException INFO [2022-01-10 08:40:15,817] jepsen worker 697 - jepsen.util 3697 :info :add 10513 :RequestTimeoutException INFO [2022-01-10 08:40:15,823] jepsen worker 20 - jepsen.util 3020 :info :add 10515 :RequestTimeoutException INFO [2022-01-10 08:40:15,831] jepsen worker 601 - jepsen.util 3601 :info :add 10516 :RequestTimeoutException INFO [2022-01-10 08:40:15,832] jepsen worker 684 - jepsen.util 3684 :info :add 10518 :RequestTimeoutException INFO [2022-01-10 08:40:15,836] jepsen worker 326 - jepsen.util 3326 :info :add 10519 :RequestTimeoutException INFO [2022-01-10 08:40:15,843] jepsen worker 923 - jepsen.util 3923 :info :add 10521 :RequestTimeoutException INFO [2022-01-10 08:40:15,851] jepsen worker 468 - jepsen.util 3468 :info :add 10522 :RequestTimeoutException INFO [2022-01-10 08:40:15,862] jepsen worker 315 - jepsen.util 3315 :info :add 10524 :RequestTimeoutException INFO [2022-01-10 08:40:15,867] jepsen worker 983 - jepsen.util 3983 :info :add 10525 :RequestTimeoutException INFO [2022-01-10 08:40:15,870] jepsen worker 544 - jepsen.util 3544 :info :add 10526 :RequestTimeoutException INFO [2022-01-10 08:40:15,873] jepsen worker 901 - jepsen.util 2901 :info :add 10527 :RequestTimeoutException INFO [2022-01-10 08:40:15,881] jepsen worker 45 - jepsen.util 3045 :info :add 10528 :RequestTimeoutException INFO [2022-01-10 08:40:15,884] jepsen worker 821 - jepsen.util 2821 :info :add 10529 :RequestTimeoutException INFO [2022-01-10 08:40:15,893] jepsen worker 908 - jepsen.util 4908 :ok :add 10967 INFO [2022-01-10 08:40:15,893] jepsen worker 138 - jepsen.util 3138 :ok :add 10533 INFO [2022-01-10 08:40:15,893] jepsen worker 716 - jepsen.util 4716 :ok :add 10676 INFO [2022-01-10 08:40:15,902] jepsen worker 738 - jepsen.util 3738 :info :add 10530 :RequestTimeoutException INFO [2022-01-10 08:40:15,917] jepsen worker 809 - jepsen.util 2809 :info :add 10531 :RequestTimeoutException INFO [2022-01-10 08:40:15,931] jepsen worker 71 - jepsen.util 3071 :info :add 10534 :RequestTimeoutException INFO [2022-01-10 08:40:15,933] jepsen worker 606 - jepsen.util 2606 :info :add 10535 :RequestTimeoutException INFO [2022-01-10 08:40:15,942] jepsen worker 166 - jepsen.util 3166 :info :add 10536 :RequestTimeoutException INFO [2022-01-10 08:40:15,946] jepsen worker 102 - jepsen.util 2102 :info :add 10537 :RequestTimeoutException INFO [2022-01-10 08:40:15,951] jepsen worker 742 - jepsen.util 3742 :info :add 10540 :RequestTimeoutException INFO [2022-01-10 08:40:15,953] jepsen worker 144 - jepsen.util 3144 :info :add 10539 :RequestTimeoutException INFO [2022-01-10 08:40:15,963] jepsen worker 462 - jepsen.util 1462 :info :add 10541 :RequestTimeoutException INFO [2022-01-10 08:40:15,972] jepsen worker 171 - jepsen.util 3171 :info :add 10542 :RequestTimeoutException INFO [2022-01-10 08:40:15,977] jepsen worker 261 - jepsen.util 3261 :info :add 10543 :RequestTimeoutException INFO [2022-01-10 08:40:15,988] jepsen worker 548 - jepsen.util 3548 :ok :add 10598 INFO [2022-01-10 08:40:15,990] jepsen worker 190 - jepsen.util 3190 :info :add 10544 :RequestTimeoutException INFO [2022-01-10 08:40:15,992] jepsen worker 913 - jepsen.util 3913 :info :add 10545 :RequestTimeoutException INFO [2022-01-10 08:40:16,001] jepsen worker 429 - jepsen.util 3429 :info :add 10546 :RequestTimeoutException INFO [2022-01-10 08:40:16,002] jepsen worker 378 - jepsen.util 3378 :info :add 10548 :RequestTimeoutException INFO [2022-01-10 08:40:16,003] jepsen worker 308 - jepsen.util 3308 :info :add 10547 :RequestTimeoutException INFO [2022-01-10 08:40:16,007] jepsen worker 891 - jepsen.util 2891 :info :add 10549 :RequestTimeoutException INFO [2022-01-10 08:40:16,013] jepsen worker 830 - jepsen.util 3830 :info :add 10551 :RequestTimeoutException INFO [2022-01-10 08:40:16,020] jepsen worker 361 - jepsen.util 3361 :info :add 10552 :RequestTimeoutException INFO [2022-01-10 08:40:16,022] jepsen worker 449 - jepsen.util 3449 :info :add 10554 :RequestTimeoutException INFO [2022-01-10 08:40:16,027] jepsen worker 862 - jepsen.util 3862 :info :add 10555 :RequestTimeoutException INFO [2022-01-10 08:40:16,030] jepsen worker 417 - jepsen.util 3417 :info :add 10556 :RequestTimeoutException INFO [2022-01-10 08:40:16,031] jepsen worker 357 - jepsen.util 2357 :info :add 10558 :RequestTimeoutException INFO [2022-01-10 08:40:16,032] jepsen worker 546 - jepsen.util 3546 :info :add 10557 :RequestTimeoutException INFO [2022-01-10 08:40:16,063] jepsen worker 610 - jepsen.util 3610 :info :add 10559 :RequestTimeoutException INFO [2022-01-10 08:40:16,072] jepsen worker 231 - jepsen.util 3231 :info :add 10560 :RequestTimeoutException INFO [2022-01-10 08:40:16,077] jepsen worker 585 - jepsen.util 3585 :info :add 10561 :RequestTimeoutException INFO [2022-01-10 08:40:16,080] jepsen worker 226 - jepsen.util 3226 :info :add 10562 :RequestTimeoutException INFO [2022-01-10 08:40:16,101] jepsen worker 945 - jepsen.util 3945 :info :add 10564 :RequestTimeoutException INFO [2022-01-10 08:40:16,102] jepsen worker 121 - jepsen.util 3121 :info :add 10563 :RequestTimeoutException INFO [2022-01-10 08:40:16,122] jepsen worker 330 - jepsen.util 3330 :info :add 10566 :RequestTimeoutException INFO [2022-01-10 08:40:16,280] jepsen worker 843 - jepsen.util 3843 :info :add 10568 :RequestTimeoutException INFO [2022-01-10 08:40:16,283] jepsen worker 291 - jepsen.util 3291 :info :add 10569 :RequestTimeoutException INFO [2022-01-10 08:40:16,291] jepsen worker 85 - jepsen.util 2085 :info :add 10570 :RequestTimeoutException INFO [2022-01-10 08:40:16,293] jepsen worker 938 - jepsen.util 2938 :info :add 10571 :RequestTimeoutException INFO [2022-01-10 08:40:16,302] jepsen worker 791 - jepsen.util 3791 :info :add 10572 :RequestTimeoutException INFO [2022-01-10 08:40:16,306] jepsen worker 547 - jepsen.util 3547 :info :add 10573 :RequestTimeoutException INFO [2022-01-10 08:40:16,310] jepsen worker 22 - jepsen.util 3022 :info :add 10574 :RequestTimeoutException INFO [2022-01-10 08:40:16,311] jepsen worker 160 - jepsen.util 3160 :info :add 10576 :RequestTimeoutException INFO [2022-01-10 08:40:16,313] jepsen worker 232 - jepsen.util 2232 :info :add 10575 :RequestTimeoutException INFO [2022-01-10 08:40:16,313] jepsen worker 99 - jepsen.util 2099 :info :add 10577 :RequestTimeoutException INFO [2022-01-10 08:40:16,322] jepsen worker 137 - jepsen.util 3137 :info :add 10578 :RequestTimeoutException INFO [2022-01-10 08:40:16,414] jepsen worker 674 - jepsen.util 1674 :info :add 10579 :RequestTimeoutException INFO [2022-01-10 08:40:16,414] jepsen worker 48 - jepsen.util 3048 :info :add 10580 :RequestTimeoutException INFO [2022-01-10 08:40:16,421] jepsen worker 481 - jepsen.util 1481 :info :add 10581 :RequestTimeoutException INFO [2022-01-10 08:40:16,421] jepsen worker 914 - jepsen.util 3914 :info :add 10582 :RequestTimeoutException INFO [2022-01-10 08:40:16,423] jepsen worker 780 - jepsen.util 3780 :info :add 10583 :RequestTimeoutException INFO [2022-01-10 08:40:16,433] jepsen worker 728 - jepsen.util 3728 :info :add 10584 :RequestTimeoutException INFO [2022-01-10 08:40:16,433] jepsen worker 916 - jepsen.util 2916 :info :add 10585 :RequestTimeoutException INFO [2022-01-10 08:40:16,447] jepsen worker 772 - jepsen.util 4772 :info :add 10588 :RequestTimeoutException INFO [2022-01-10 08:40:16,450] jepsen worker 184 - jepsen.util 3184 :info :add 10589 :RequestTimeoutException INFO [2022-01-10 08:40:16,464] jepsen worker 609 - jepsen.util 4609 :ok :add 10714 INFO [2022-01-10 08:40:16,464] jepsen worker 269 - jepsen.util 4269 :ok :add 10829 INFO [2022-01-10 08:40:16,464] jepsen worker 63 - jepsen.util 4063 :ok :add 10805 INFO [2022-01-10 08:40:16,464] jepsen worker 571 - jepsen.util 4571 :ok :add 10738 INFO [2022-01-10 08:40:16,464] jepsen worker 859 - jepsen.util 2859 :ok :add 10720 INFO [2022-01-10 08:40:16,464] jepsen worker 186 - jepsen.util 4186 :ok :add 10684 INFO [2022-01-10 08:40:16,464] jepsen worker 985 - jepsen.util 4985 :ok :add 10831 INFO [2022-01-10 08:40:16,464] jepsen worker 858 - jepsen.util 3858 :ok :add 11091 INFO [2022-01-10 08:40:16,493] jepsen worker 337 - jepsen.util 3337 :info :add 10590 :RequestTimeoutException INFO [2022-01-10 08:40:16,511] jepsen worker 768 - jepsen.util 3768 :info :add 10591 :RequestTimeoutException INFO [2022-01-10 08:40:16,533] jepsen worker 739 - jepsen.util 3739 :info :add 10592 :RequestTimeoutException INFO [2022-01-10 08:40:16,541] jepsen worker 732 - jepsen.util 3732 :info :add 10593 :RequestTimeoutException INFO [2022-01-10 08:40:16,557] jepsen worker 559 - jepsen.util 3559 :info :add 10594 :RequestTimeoutException INFO [2022-01-10 08:40:16,560] jepsen worker 279 - jepsen.util 3279 :info :add 10595 :RequestTimeoutException INFO [2022-01-10 08:40:16,563] jepsen worker 380 - jepsen.util 3380 :info :add 10596 :RequestTimeoutException INFO [2022-01-10 08:40:16,571] jepsen worker 897 - jepsen.util 3897 :info :add 10597 :RequestTimeoutException INFO [2022-01-10 08:40:16,572] jepsen worker 764 - jepsen.util 3764 :info :add 10599 :RequestTimeoutException INFO [2022-01-10 08:40:16,576] jepsen worker 53 - jepsen.util 4053 :info :add 10600 :RequestTimeoutException INFO [2022-01-10 08:40:16,590] jepsen worker 976 - jepsen.util 2976 :info :add 10601 :RequestTimeoutException INFO [2022-01-10 08:40:16,592] jepsen worker 694 - jepsen.util 3694 :info :add 10602 :RequestTimeoutException INFO [2022-01-10 08:40:16,601] jepsen worker 686 - jepsen.util 2686 :info :add 10603 :RequestTimeoutException INFO [2022-01-10 08:40:16,644] jepsen worker 761 - jepsen.util 3761 :info :add 10604 :RequestTimeoutException INFO [2022-01-10 08:40:16,662] jepsen worker 370 - jepsen.util 4370 :info :add 10605 :RequestTimeoutException INFO [2022-01-10 08:40:16,677] jepsen worker 931 - jepsen.util 4931 :info :add 10606 :RequestTimeoutException INFO [2022-01-10 08:40:16,680] jepsen worker 801 - jepsen.util 4801 :info :add 10607 :RequestTimeoutException INFO [2022-01-10 08:40:16,681] jepsen worker 13 - jepsen.util 3013 :info :add 10609 :RequestTimeoutException INFO [2022-01-10 08:40:16,683] jepsen worker 349 - jepsen.util 3349 :info :add 10608 :RequestTimeoutException INFO [2022-01-10 08:40:16,683] jepsen worker 625 - jepsen.util 3625 :info :add 10610 :RequestTimeoutException INFO [2022-01-10 08:40:16,712] jepsen worker 95 - jepsen.util 4095 :info :add 10611 :RequestTimeoutException INFO [2022-01-10 08:40:16,716] jepsen worker 706 - jepsen.util 3706 :info :add 10612 :RequestTimeoutException INFO [2022-01-10 08:40:16,750] jepsen worker 173 - jepsen.util 2173 :info :add 10613 :RequestTimeoutException INFO [2022-01-10 08:40:16,760] jepsen worker 882 - jepsen.util 3882 :info :add 10614 :RequestTimeoutException INFO [2022-01-10 08:40:16,763] jepsen worker 436 - jepsen.util 4436 :info :add 10615 :RequestTimeoutException INFO [2022-01-10 08:40:16,772] jepsen worker 11 - jepsen.util 3011 :info :add 10618 :RequestTimeoutException INFO [2022-01-10 08:40:16,773] jepsen worker 850 - jepsen.util 3850 :info :add 10617 :RequestTimeoutException INFO [2022-01-10 08:40:16,776] jepsen worker 131 - jepsen.util 3131 :info :add 10619 :RequestTimeoutException INFO [2022-01-10 08:40:16,780] jepsen worker 578 - jepsen.util 3578 :info :add 10620 :RequestTimeoutException INFO [2022-01-10 08:40:16,782] jepsen worker 685 - jepsen.util 3685 :info :add 10621 :RequestTimeoutException INFO [2022-01-10 08:40:16,834] jepsen worker 442 - jepsen.util 3442 :info :add 10623 :RequestTimeoutException INFO [2022-01-10 08:40:16,862] jepsen worker 679 - jepsen.util 3679 :info :add 10624 :RequestTimeoutException INFO [2022-01-10 08:40:16,867] jepsen worker 406 - jepsen.util 4406 :info :add 10625 :RequestTimeoutException INFO [2022-01-10 08:40:16,870] jepsen worker 24 - jepsen.util 2024 :info :add 10626 :RequestTimeoutException INFO [2022-01-10 08:40:16,872] jepsen worker 150 - jepsen.util 3150 :info :add 10627 :RequestTimeoutException INFO [2022-01-10 08:40:16,881] jepsen worker 62 - jepsen.util 2062 :info :add 10628 :RequestTimeoutException INFO [2022-01-10 08:40:16,882] jepsen worker 270 - jepsen.util 3270 :info :add 10630 :RequestTimeoutException INFO [2022-01-10 08:40:16,883] jepsen worker 496 - jepsen.util 3496 :info :add 10629 :RequestTimeoutException INFO [2022-01-10 08:40:16,887] jepsen worker 114 - jepsen.util 3114 :info :add 10631 :RequestTimeoutException INFO [2022-01-10 08:40:16,922] jepsen worker 443 - jepsen.util 3443 :info :add 10633 :RequestTimeoutException INFO [2022-01-10 08:40:16,931] jepsen worker 881 - jepsen.util 3881 :info :add 10634 :RequestTimeoutException INFO [2022-01-10 08:40:16,933] jepsen worker 246 - jepsen.util 3246 :info :add 10635 :RequestTimeoutException INFO [2022-01-10 08:40:16,942] jepsen worker 205 - jepsen.util 4205 :info :add 10636 :RequestTimeoutException INFO [2022-01-10 08:40:16,947] jepsen worker 332 - jepsen.util 3332 :info :add 10637 :RequestTimeoutException INFO [2022-01-10 08:40:16,951] jepsen worker 94 - jepsen.util 3094 :info :add 10638 :RequestTimeoutException INFO [2022-01-10 08:40:16,956] jepsen worker 831 - jepsen.util 2831 :info :add 10639 :RequestTimeoutException INFO [2022-01-10 08:40:16,961] jepsen worker 905 - jepsen.util 3905 :info :add 10641 :RequestTimeoutException INFO [2022-01-10 08:40:16,963] jepsen worker 877 - jepsen.util 3877 :info :add 10643 :RequestTimeoutException INFO [2022-01-10 08:40:16,967] jepsen worker 90 - jepsen.util 2090 :info :add 10642 :RequestTimeoutException INFO [2022-01-10 08:40:16,992] jepsen worker 656 - jepsen.util 4656 :info :add 10644 :RequestTimeoutException INFO [2022-01-10 08:40:17,057] jepsen worker 735 - jepsen.util 4735 :info :add 10645 :RequestTimeoutException INFO [2022-01-10 08:40:17,060] jepsen worker 385 - jepsen.util 4385 :info :add 10646 :RequestTimeoutException INFO [2022-01-10 08:40:17,093] jepsen worker 603 - jepsen.util 3603 :info :add 10647 :RequestTimeoutException INFO [2022-01-10 08:40:17,191] jepsen worker 309 - jepsen.util 4309 :info :add 10648 :RequestTimeoutException INFO [2022-01-10 08:40:17,223] jepsen worker 951 - jepsen.util 4951 :info :add 10649 :RequestTimeoutException INFO [2022-01-10 08:40:17,242] jepsen worker 793 - jepsen.util 4793 :info :add 10650 :RequestTimeoutException INFO [2022-01-10 08:40:17,247] jepsen worker 741 - jepsen.util 3741 :info :add 10651 :RequestTimeoutException INFO [2022-01-10 08:40:17,270] jepsen worker 220 - jepsen.util 3220 :info :add 10652 :RequestTimeoutException INFO [2022-01-10 08:40:17,293] jepsen worker 785 - jepsen.util 4785 :info :add 10653 :RequestTimeoutException INFO [2022-01-10 08:40:17,301] jepsen worker 828 - jepsen.util 4828 :ok :add 10807 INFO [2022-01-10 08:40:17,301] jepsen worker 218 - jepsen.util 3218 :ok :add 11103 INFO [2022-01-10 08:40:17,302] jepsen worker 266 - jepsen.util 4266 :ok :add 10716 INFO [2022-01-10 08:40:17,302] jepsen worker 567 - jepsen.util 3567 :info :add 10654 :RequestTimeoutException INFO [2022-01-10 08:40:17,302] jepsen worker 454 - jepsen.util 3454 :ok :add 10686 INFO [2022-01-10 08:40:17,302] jepsen worker 892 - jepsen.util 4892 :ok :add 10722 INFO [2022-01-10 08:40:17,302] jepsen worker 428 - jepsen.util 4428 :ok :add 10833 INFO [2022-01-10 08:40:17,305] jepsen worker 219 - jepsen.util 3219 :ok :add 10787 INFO [2022-01-10 08:40:17,305] jepsen worker 894 - jepsen.util 3894 :ok :add 11036 INFO [2022-01-10 08:40:17,312] jepsen worker 478 - jepsen.util 4478 :info :add 10656 :RequestTimeoutException INFO [2022-01-10 08:40:17,314] jepsen worker 16 - jepsen.util 4016 :info :add 10655 :RequestTimeoutException INFO [2022-01-10 08:40:17,330] jepsen worker 271 - jepsen.util 4271 :info :add 10658 :RequestTimeoutException INFO [2022-01-10 08:40:17,331] jepsen worker 369 - jepsen.util 3369 :info :add 10660 :RequestTimeoutException INFO [2022-01-10 08:40:17,332] jepsen worker 545 - jepsen.util 4545 :info :add 10659 :RequestTimeoutException INFO [2022-01-10 08:40:17,333] jepsen worker 267 - jepsen.util 4267 :info :add 10661 :RequestTimeoutException INFO [2022-01-10 08:40:17,392] jepsen worker 28 - jepsen.util 3028 :info :add 10662 :RequestTimeoutException INFO [2022-01-10 08:40:17,420] jepsen worker 116 - jepsen.util 3116 :info :add 10664 :RequestTimeoutException INFO [2022-01-10 08:40:17,423] jepsen worker 746 - jepsen.util 3746 :info :add 10665 :RequestTimeoutException INFO [2022-01-10 08:40:17,430] jepsen worker 680 - jepsen.util 2680 :info :add 10667 :RequestTimeoutException INFO [2022-01-10 08:40:17,431] jepsen worker 482 - jepsen.util 2482 :info :add 10668 :RequestTimeoutException INFO [2022-01-10 08:40:17,432] jepsen worker 857 - jepsen.util 3857 :ok :add 10709 INFO [2022-01-10 08:40:17,434] jepsen worker 802 - jepsen.util 4802 :info :add 10669 :RequestTimeoutException INFO [2022-01-10 08:40:17,440] jepsen worker 623 - jepsen.util 2623 :info :add 10670 :RequestTimeoutException INFO [2022-01-10 08:40:17,443] jepsen worker 512 - jepsen.util 2512 :info :add 10671 :RequestTimeoutException INFO [2022-01-10 08:40:17,443] jepsen worker 961 - jepsen.util 2961 :info :add 10672 :RequestTimeoutException INFO [2022-01-10 08:40:17,512] jepsen worker 488 - jepsen.util 3488 :info :add 10673 :RequestTimeoutException INFO [2022-01-10 08:40:17,536] jepsen worker 827 - jepsen.util 4827 :info :add 10674 :RequestTimeoutException INFO [2022-01-10 08:40:17,540] jepsen worker 111 - jepsen.util 3111 :info :add 10675 :RequestTimeoutException INFO [2022-01-10 08:40:17,551] jepsen worker 555 - jepsen.util 4555 :info :add 10677 :RequestTimeoutException INFO [2022-01-10 08:40:17,553] jepsen worker 803 - jepsen.util 3803 :info :add 10678 :RequestTimeoutException INFO [2022-01-10 08:40:17,557] jepsen worker 507 - jepsen.util 4507 :info :add 10680 :RequestTimeoutException INFO [2022-01-10 08:40:17,560] jepsen worker 458 - jepsen.util 4458 :ok :add 10743 INFO [2022-01-10 08:40:17,561] jepsen worker 15 - jepsen.util 4015 :info :add 10681 :RequestTimeoutException INFO [2022-01-10 08:40:17,562] jepsen worker 552 - jepsen.util 4552 :info :add 10679 :RequestTimeoutException INFO [2022-01-10 08:40:17,564] jepsen worker 474 - jepsen.util 2474 :ok :add 10852 INFO [2022-01-10 08:40:17,573] jepsen worker 387 - jepsen.util 3387 :info :add 10682 :RequestTimeoutException INFO [2022-01-10 08:40:17,581] jepsen worker 765 - jepsen.util 3765 :info :add 10683 :RequestTimeoutException INFO [2022-01-10 08:40:17,582] jepsen worker 860 - jepsen.util 4860 :info :add 10685 :RequestTimeoutException INFO [2022-01-10 08:40:17,590] jepsen worker 621 - jepsen.util 3621 :info :add 10687 :RequestTimeoutException INFO [2022-01-10 08:40:17,593] jepsen worker 495 - jepsen.util 3495 :info :add 10688 :RequestTimeoutException INFO [2022-01-10 08:40:17,601] jepsen worker 471 - jepsen.util 3471 :info :add 10689 :RequestTimeoutException INFO [2022-01-10 08:40:17,603] jepsen worker 118 - jepsen.util 3118 :info :add 10690 :RequestTimeoutException INFO [2022-01-10 08:40:17,612] jepsen worker 316 - jepsen.util 3316 :info :add 10691 :RequestTimeoutException INFO [2022-01-10 08:40:17,617] jepsen worker 156 - jepsen.util 3156 :info :add 10692 :RequestTimeoutException INFO [2022-01-10 08:40:17,620] jepsen worker 233 - jepsen.util 3233 :info :add 10693 :RequestTimeoutException INFO [2022-01-10 08:40:17,641] jepsen worker 636 - jepsen.util 1636 :info :add 10694 :RequestTimeoutException INFO [2022-01-10 08:40:17,743] jepsen worker 262 - jepsen.util 3262 :info :add 10695 :RequestTimeoutException INFO [2022-01-10 08:40:17,753] jepsen worker 673 - jepsen.util 3673 :info :add 10696 :RequestTimeoutException INFO [2022-01-10 08:40:17,753] jepsen worker 179 - jepsen.util 3179 :info :add 10697 :RequestTimeoutException INFO [2022-01-10 08:40:17,763] jepsen worker 717 - jepsen.util 2717 :info :add 10698 :RequestTimeoutException INFO [2022-01-10 08:40:17,891] jepsen worker 813 - jepsen.util 3813 :info :add 10699 :RequestTimeoutException INFO [2022-01-10 08:40:17,903] jepsen worker 35 - jepsen.util 3035 :info :add 10700 :RequestTimeoutException INFO [2022-01-10 08:40:17,912] jepsen worker 320 - jepsen.util 3320 :info :add 10701 :RequestTimeoutException INFO [2022-01-10 08:40:17,917] jepsen worker 204 - jepsen.util 3204 :info :add 10702 :RequestTimeoutException INFO [2022-01-10 08:40:17,920] jepsen worker 997 - jepsen.util 4997 :info :add 10703 :RequestTimeoutException INFO [2022-01-10 08:40:17,921] jepsen worker 977 - jepsen.util 3977 :info :add 10705 :RequestTimeoutException INFO [2022-01-10 08:40:17,923] jepsen worker 221 - jepsen.util 3221 :info :add 10704 :RequestTimeoutException INFO [2022-01-10 08:40:17,932] jepsen worker 56 - jepsen.util 4056 :info :add 10707 :RequestTimeoutException INFO [2022-01-10 08:40:17,933] jepsen worker 437 - jepsen.util 3437 :info :add 10706 :RequestTimeoutException INFO [2022-01-10 08:40:17,947] jepsen worker 300 - jepsen.util 3300 :info :add 10708 :RequestTimeoutException INFO [2022-01-10 08:40:17,983] jepsen worker 181 - jepsen.util 2181 :info :add 10710 :RequestTimeoutException INFO [2022-01-10 08:40:17,991] jepsen worker 257 - jepsen.util 3257 :info :add 10711 :RequestTimeoutException INFO [2022-01-10 08:40:17,993] jepsen worker 215 - jepsen.util 3215 :info :add 10712 :RequestTimeoutException INFO [2022-01-10 08:40:18,024] jepsen worker 566 - jepsen.util 3566 :ok :add 10994 INFO [2022-01-10 08:40:18,029] jepsen worker 878 - jepsen.util 3878 :ok :add 10721 INFO [2022-01-10 08:40:18,029] jepsen worker 372 - jepsen.util 3372 :ok :add 10739 INFO [2022-01-10 08:40:18,042] jepsen worker 627 - jepsen.util 4627 :info :add 10713 :RequestTimeoutException INFO [2022-01-10 08:40:18,051] jepsen worker 244 - jepsen.util 4244 :info :add 10715 :RequestTimeoutException INFO [2022-01-10 08:40:18,154] jepsen worker 604 - jepsen.util 3604 :info :add 10718 :RequestTimeoutException INFO [2022-01-10 08:40:18,162] jepsen worker 861 - jepsen.util 3861 :info :add 10719 :RequestTimeoutException INFO [2022-01-10 08:40:18,180] jepsen worker 43 - jepsen.util 3043 :info :add 10723 :RequestTimeoutException INFO [2022-01-10 08:40:18,183] jepsen worker 535 - jepsen.util 2535 :info :add 10724 :RequestTimeoutException INFO [2022-01-10 08:40:18,183] jepsen worker 639 - jepsen.util 3639 :info :add 10725 :RequestTimeoutException INFO [2022-01-10 08:40:18,191] jepsen worker 871 - jepsen.util 3871 :info :add 10726 :RequestTimeoutException INFO [2022-01-10 08:40:18,192] jepsen worker 701 - jepsen.util 4701 :info :add 10728 :RequestTimeoutException INFO [2022-01-10 08:40:18,194] jepsen worker 368 - jepsen.util 2368 :info :add 10729 :RequestTimeoutException INFO [2022-01-10 08:40:18,194] jepsen worker 120 - jepsen.util 3120 :info :add 10727 :RequestTimeoutException INFO [2022-01-10 08:40:18,247] jepsen worker 750 - jepsen.util 3750 :info :add 10730 :RequestTimeoutException INFO [2022-01-10 08:40:18,263] jepsen worker 690 - jepsen.util 2690 :ok :add 10976 INFO [2022-01-10 08:40:18,290] jepsen worker 868 - jepsen.util 4868 :info :add 10731 :RequestTimeoutException INFO [2022-01-10 08:40:18,301] jepsen worker 278 - jepsen.util 4278 :info :add 10733 :RequestTimeoutException INFO [2022-01-10 08:40:18,303] jepsen worker 194 - jepsen.util 4194 :info :add 10732 :RequestTimeoutException INFO [2022-01-10 08:40:18,354] jepsen worker 899 - jepsen.util 3899 :info :add 10734 :RequestTimeoutException INFO [2022-01-10 08:40:18,362] jepsen worker 804 - jepsen.util 3804 :info :add 10735 :RequestTimeoutException INFO [2022-01-10 08:40:18,367] jepsen worker 123 - jepsen.util 4123 :info :add 10736 :RequestTimeoutException INFO [2022-01-10 08:40:18,370] jepsen worker 322 - jepsen.util 4322 :info :add 10737 :RequestTimeoutException INFO [2022-01-10 08:40:18,433] jepsen worker 282 - jepsen.util 4282 :info :add 10740 :RequestTimeoutException INFO [2022-01-10 08:40:18,482] jepsen worker 525 - jepsen.util 3525 :info :add 10741 :RequestTimeoutException INFO [2022-01-10 08:40:18,486] jepsen worker 928 - jepsen.util 4928 :info :add 10742 :RequestTimeoutException INFO [2022-01-10 08:40:18,569] jepsen worker 903 - jepsen.util 4903 :ok :add 11026 INFO [2022-01-10 08:40:18,651] jepsen worker 948 - jepsen.util 3948 :info :add 10745 :RequestTimeoutException INFO [2022-01-10 08:40:18,653] jepsen worker 129 - jepsen.util 3129 :info :add 10744 :RequestTimeoutException INFO [2022-01-10 08:40:18,660] jepsen worker 650 - jepsen.util 2650 :info :add 10746 :RequestTimeoutException INFO [2022-01-10 08:40:18,663] jepsen worker 51 - jepsen.util 3051 :info :add 10748 :RequestTimeoutException INFO [2022-01-10 08:40:18,663] jepsen worker 826 - jepsen.util 3826 :info :add 10747 :RequestTimeoutException INFO [2022-01-10 08:40:18,667] jepsen worker 448 - jepsen.util 3448 :info :add 10750 :RequestTimeoutException INFO [2022-01-10 08:40:18,670] jepsen worker 318 - jepsen.util 3318 :info :add 10751 :RequestTimeoutException INFO [2022-01-10 08:40:18,671] jepsen worker 551 - jepsen.util 4551 :info :add 10754 :RequestTimeoutException INFO [2022-01-10 08:40:18,673] jepsen worker 998 - jepsen.util 4998 :info :add 10755 :RequestTimeoutException INFO [2022-01-10 08:40:18,680] jepsen worker 942 - jepsen.util 4942 :info :add 10759 :RequestTimeoutException INFO [2022-01-10 08:40:18,681] jepsen worker 172 - jepsen.util 2172 :info :add 10756 :RequestTimeoutException INFO [2022-01-10 08:40:18,682] jepsen worker 933 - jepsen.util 3933 :info :add 10757 :RequestTimeoutException INFO [2022-01-10 08:40:18,683] jepsen worker 837 - jepsen.util 4837 :info :add 10760 :RequestTimeoutException INFO [2022-01-10 08:40:18,687] jepsen worker 272 - jepsen.util 4272 :info :add 10758 :RequestTimeoutException INFO [2022-01-10 08:40:18,691] jepsen worker 424 - jepsen.util 3424 :info :add 10761 :RequestTimeoutException INFO [2022-01-10 08:40:18,693] jepsen worker 608 - jepsen.util 4608 :info :add 10763 :RequestTimeoutException INFO [2022-01-10 08:40:18,693] jepsen worker 926 - jepsen.util 3926 :info :add 10762 :RequestTimeoutException INFO [2022-01-10 08:40:18,696] jepsen worker 852 - jepsen.util 3852 :info :add 10765 :RequestTimeoutException INFO [2022-01-10 08:40:18,700] jepsen worker 869 - jepsen.util 4869 :info :add 10766 :RequestTimeoutException INFO [2022-01-10 08:40:18,701] jepsen worker 473 - jepsen.util 3473 :info :add 10768 :RequestTimeoutException INFO [2022-01-10 08:40:18,702] jepsen worker 645 - jepsen.util 4645 :info :add 10770 :RequestTimeoutException INFO [2022-01-10 08:40:18,703] jepsen worker 434 - jepsen.util 3434 :info :add 10769 :RequestTimeoutException INFO [2022-01-10 08:40:18,707] jepsen worker 577 - jepsen.util 4577 :info :add 10771 :RequestTimeoutException INFO [2022-01-10 08:40:18,710] jepsen worker 411 - jepsen.util 3411 :info :add 10772 :RequestTimeoutException INFO [2022-01-10 08:40:18,710] jepsen worker 363 - jepsen.util 3363 :info :add 10774 :RequestTimeoutException INFO [2022-01-10 08:40:18,713] jepsen worker 386 - jepsen.util 2386 :info :add 10773 :RequestTimeoutException INFO [2022-01-10 08:40:18,753] jepsen worker 550 - jepsen.util 4550 :info :add 10775 :RequestTimeoutException INFO [2022-01-10 08:40:18,761] jepsen worker 595 - jepsen.util 4595 :info :add 10776 :RequestTimeoutException INFO [2022-01-10 08:40:18,763] jepsen worker 234 - jepsen.util 4234 :info :add 10777 :RequestTimeoutException INFO [2022-01-10 08:40:18,772] jepsen worker 692 - jepsen.util 4692 :info :add 10778 :RequestTimeoutException INFO [2022-01-10 08:40:18,777] jepsen worker 991 - jepsen.util 4991 :info :add 10779 :RequestTimeoutException INFO [2022-01-10 08:40:18,850] jepsen worker 427 - jepsen.util 4427 :info :add 10780 :RequestTimeoutException INFO [2022-01-10 08:40:18,882] jepsen worker 953 - jepsen.util 4953 :info :add 10781 :RequestTimeoutException INFO [2022-01-10 08:40:18,891] jepsen worker 201 - jepsen.util 4201 :info :add 10782 :RequestTimeoutException INFO [2022-01-10 08:40:18,913] jepsen worker 767 - jepsen.util 3767 :info :add 10783 :RequestTimeoutException INFO [2022-01-10 08:40:18,927] jepsen worker 325 - jepsen.util 3325 :info :add 10785 :RequestTimeoutException INFO [2022-01-10 08:40:18,930] jepsen worker 493 - jepsen.util 3493 :info :add 10786 :RequestTimeoutException INFO [2022-01-10 08:40:18,931] jepsen worker 193 - jepsen.util 3193 :info :add 10788 :RequestTimeoutException INFO [2022-01-10 08:40:18,972] jepsen worker 756 - jepsen.util 4756 :info :add 10790 :RequestTimeoutException INFO [2022-01-10 08:40:18,973] jepsen worker 91 - jepsen.util 4091 :info :add 10789 :RequestTimeoutException INFO [2022-01-10 08:40:18,997] jepsen worker 76 - jepsen.util 2076 :info :add 10791 :RequestTimeoutException INFO [2022-01-10 08:40:19,000] jepsen worker 44 - jepsen.util 4044 :info :add 10792 :RequestTimeoutException INFO [2022-01-10 08:40:19,141] jepsen worker 556 - jepsen.util 2556 :info :add 10794 :RequestTimeoutException INFO [2022-01-10 08:40:19,142] jepsen worker 607 - jepsen.util 2607 :ok :add 11025 INFO [2022-01-10 08:40:19,142] jepsen worker 180 - jepsen.util 4180 :ok :add 11009 INFO [2022-01-10 08:40:19,143] jepsen worker 867 - jepsen.util 3867 :info :add 10793 :RequestTimeoutException INFO [2022-01-10 08:40:19,152] jepsen worker 128 - jepsen.util 4128 :info :add 10796 :RequestTimeoutException INFO [2022-01-10 08:40:19,153] jepsen worker 787 - jepsen.util 2787 :info :add 10795 :RequestTimeoutException INFO [2022-01-10 08:40:19,157] jepsen worker 381 - jepsen.util 4381 :info :add 10797 :RequestTimeoutException INFO [2022-01-10 08:40:19,160] jepsen worker 820 - jepsen.util 3820 :info :add 10798 :RequestTimeoutException INFO [2022-01-10 08:40:19,163] jepsen worker 399 - jepsen.util 3399 :info :add 10799 :RequestTimeoutException INFO [2022-01-10 08:40:19,241] jepsen worker 536 - jepsen.util 4536 :info :add 10800 :RequestTimeoutException INFO [2022-01-10 08:40:19,252] jepsen worker 297 - jepsen.util 4297 :info :add 10802 :RequestTimeoutException INFO [2022-01-10 08:40:19,254] jepsen worker 10 - jepsen.util 4010 :info :add 10801 :RequestTimeoutException INFO [2022-01-10 08:40:19,287] jepsen worker 286 - jepsen.util 3286 :info :add 10803 :RequestTimeoutException INFO [2022-01-10 08:40:19,320] jepsen worker 984 - jepsen.util 4984 :info :add 10804 :RequestTimeoutException INFO [2022-01-10 08:40:19,331] jepsen worker 305 - jepsen.util 4305 :info :add 10806 :RequestTimeoutException INFO [2022-01-10 08:40:19,352] jepsen worker 26 - jepsen.util 4026 :info :add 10808 :RequestTimeoutException INFO [2022-01-10 08:40:19,357] jepsen worker 665 - jepsen.util 3665 :info :add 10809 :RequestTimeoutException INFO [2022-01-10 08:40:19,360] jepsen worker 263 - jepsen.util 2263 :info :add 10810 :RequestTimeoutException INFO [2022-01-10 08:40:19,360] jepsen worker 312 - jepsen.util 4312 :info :add 10812 :RequestTimeoutException INFO [2022-01-10 08:40:19,363] jepsen worker 433 - jepsen.util 3433 :info :add 10811 :RequestTimeoutException INFO [2022-01-10 08:40:19,363] jepsen worker 922 - jepsen.util 4922 :info :add 10813 :RequestTimeoutException INFO [2022-01-10 08:40:19,370] jepsen worker 341 - jepsen.util 4341 :info :add 10816 :RequestTimeoutException INFO [2022-01-10 08:40:19,372] jepsen worker 0 - jepsen.util 4000 :info :add 10814 :RequestTimeoutException INFO [2022-01-10 08:40:19,377] jepsen worker 302 - jepsen.util 2302 :info :add 10815 :RequestTimeoutException INFO [2022-01-10 08:40:19,383] jepsen worker 117 - jepsen.util 4117 :info :add 10817 :RequestTimeoutException INFO [2022-01-10 08:40:19,421] jepsen worker 612 - jepsen.util 3612 :info :add 10818 :RequestTimeoutException INFO [2022-01-10 08:40:19,423] jepsen worker 678 - jepsen.util 3678 :info :add 10819 :RequestTimeoutException INFO [2022-01-10 08:40:19,442] jepsen worker 721 - jepsen.util 3721 :info :add 10820 :RequestTimeoutException INFO [2022-01-10 08:40:19,447] jepsen worker 12 - jepsen.util 3012 :info :add 10821 :RequestTimeoutException INFO [2022-01-10 08:40:19,460] jepsen worker 480 - jepsen.util 4480 :info :add 10822 :RequestTimeoutException INFO [2022-01-10 08:40:19,471] jepsen worker 773 - jepsen.util 4773 :info :add 10824 :RequestTimeoutException INFO [2022-01-10 08:40:19,473] jepsen worker 65 - jepsen.util 4065 :info :add 10823 :RequestTimeoutException INFO [2022-01-10 08:40:19,482] jepsen worker 784 - jepsen.util 4784 :info :add 10826 :RequestTimeoutException INFO [2022-01-10 08:40:19,483] jepsen worker 252 - jepsen.util 4252 :info :add 10825 :RequestTimeoutException INFO [2022-01-10 08:40:19,486] jepsen worker 836 - jepsen.util 4836 :info :add 10827 :RequestTimeoutException INFO [2022-01-10 08:40:19,540] jepsen worker 759 - jepsen.util 4759 :info :add 10828 :RequestTimeoutException INFO [2022-01-10 08:40:19,561] jepsen worker 992 - jepsen.util 4992 :info :add 10830 :RequestTimeoutException INFO [2022-01-10 08:40:19,562] jepsen worker 723 - jepsen.util 4723 :info :add 10832 :RequestTimeoutException INFO [2022-01-10 08:40:19,680] jepsen worker 967 - jepsen.util 4967 :info :add 10834 :RequestTimeoutException INFO [2022-01-10 08:40:19,681] jepsen worker 904 - jepsen.util 3904 :info :add 10836 :RequestTimeoutException INFO [2022-01-10 08:40:19,683] jepsen worker 485 - jepsen.util 3485 :info :add 10835 :RequestTimeoutException INFO [2022-01-10 08:40:19,702] jepsen worker 277 - jepsen.util 4277 :info :add 10838 :RequestTimeoutException INFO [2022-01-10 08:40:19,703] jepsen worker 383 - jepsen.util 2383 :info :add 10837 :RequestTimeoutException INFO [2022-01-10 08:40:19,707] jepsen worker 317 - jepsen.util 4317 :info :add 10839 :RequestTimeoutException INFO [2022-01-10 08:40:19,710] jepsen worker 711 - jepsen.util 4711 :info :add 10840 :RequestTimeoutException INFO [2022-01-10 08:40:19,731] jepsen worker 502 - jepsen.util 3502 :info :add 10842 :RequestTimeoutException INFO [2022-01-10 08:40:19,732] jepsen worker 890 - jepsen.util 2890 :info :add 10841 :RequestTimeoutException INFO [2022-01-10 08:40:19,742] jepsen worker 848 - jepsen.util 3848 :info :add 10844 :RequestTimeoutException INFO [2022-01-10 08:40:19,747] jepsen worker 236 - jepsen.util 2236 :info :add 10845 :RequestTimeoutException INFO [2022-01-10 08:40:19,750] jepsen worker 299 - jepsen.util 4299 :info :add 10846 :RequestTimeoutException INFO [2022-01-10 08:40:19,753] jepsen worker 396 - jepsen.util 3396 :info :add 10847 :RequestTimeoutException INFO [2022-01-10 08:40:19,761] jepsen worker 876 - jepsen.util 3876 :info :add 10848 :RequestTimeoutException INFO [2022-01-10 08:40:19,773] jepsen worker 797 - jepsen.util 3797 :info :add 10849 :RequestTimeoutException INFO [2022-01-10 08:40:19,781] jepsen worker 402 - jepsen.util 3402 :info :add 10850 :RequestTimeoutException INFO [2022-01-10 08:40:19,786] jepsen worker 718 - jepsen.util 3718 :info :add 10851 :RequestTimeoutException INFO [2022-01-10 08:40:19,792] jepsen worker 203 - jepsen.util 2203 :info :add 10853 :RequestTimeoutException INFO [2022-01-10 08:40:19,801] jepsen worker 176 - jepsen.util 3176 :info :add 10854 :RequestTimeoutException INFO [2022-01-10 08:40:19,804] jepsen worker 944 - jepsen.util 2944 :info :add 10855 :RequestTimeoutException INFO [2022-01-10 08:40:19,807] jepsen worker 306 - jepsen.util 3306 :info :add 10857 :RequestTimeoutException INFO [2022-01-10 08:40:19,812] jepsen worker 47 - jepsen.util 3047 :info :add 10856 :RequestTimeoutException INFO [2022-01-10 08:40:19,820] jepsen worker 130 - jepsen.util 4130 :info :add 10858 :RequestTimeoutException INFO [2022-01-10 08:40:19,821] jepsen worker 358 - jepsen.util 2358 :info :add 10860 :RequestTimeoutException INFO [2022-01-10 08:40:19,823] jepsen worker 287 - jepsen.util 3287 :info :add 10859 :RequestTimeoutException INFO [2022-01-10 08:40:19,823] jepsen worker 532 - jepsen.util 3532 :info :add 10861 :RequestTimeoutException INFO [2022-01-10 08:40:19,832] jepsen worker 705 - jepsen.util 2705 :info :add 10862 :RequestTimeoutException INFO [2022-01-10 08:40:19,836] jepsen worker 865 - jepsen.util 3865 :info :add 10863 :RequestTimeoutException INFO [2022-01-10 08:40:19,840] jepsen worker 800 - jepsen.util 4800 :info :add 10864 :RequestTimeoutException INFO [2022-01-10 08:40:19,853] jepsen worker 817 - jepsen.util 4817 :info :add 10865 :RequestTimeoutException INFO [2022-01-10 08:40:19,861] jepsen worker 661 - jepsen.util 3661 :info :add 10866 :RequestTimeoutException INFO [2022-01-10 08:40:19,862] jepsen worker 709 - jepsen.util 3709 :info :add 10868 :RequestTimeoutException INFO [2022-01-10 08:40:19,864] jepsen worker 598 - jepsen.util 3598 :info :add 10867 :RequestTimeoutException INFO [2022-01-10 08:40:19,867] jepsen worker 239 - jepsen.util 4239 :info :add 10869 :RequestTimeoutException INFO [2022-01-10 08:40:19,870] jepsen worker 255 - jepsen.util 2255 :info :add 10870 :RequestTimeoutException INFO [2022-01-10 08:40:19,873] jepsen worker 587 - jepsen.util 4587 :info :add 10871 :RequestTimeoutException INFO [2022-01-10 08:40:19,881] jepsen worker 727 - jepsen.util 2727 :info :add 10872 :RequestTimeoutException INFO [2022-01-10 08:40:19,892] jepsen worker 806 - jepsen.util 4806 :info :add 10874 :RequestTimeoutException INFO [2022-01-10 08:40:19,893] jepsen worker 356 - jepsen.util 3356 :info :add 10873 :RequestTimeoutException INFO [2022-01-10 08:40:19,896] jepsen worker 362 - jepsen.util 3362 :info :add 10875 :RequestTimeoutException INFO [2022-01-10 08:40:19,901] jepsen worker 457 - jepsen.util 2457 :info :add 10876 :RequestTimeoutException INFO [2022-01-10 08:40:19,903] jepsen worker 475 - jepsen.util 4475 :info :add 10877 :RequestTimeoutException INFO [2022-01-10 08:40:19,911] jepsen worker 873 - jepsen.util 2873 :info :add 10878 :RequestTimeoutException INFO [2022-01-10 08:40:19,913] jepsen worker 61 - jepsen.util 2061 :info :add 10879 :RequestTimeoutException INFO [2022-01-10 08:40:19,922] jepsen worker 416 - jepsen.util 4416 :info :add 10880 :RequestTimeoutException INFO [2022-01-10 08:40:19,927] jepsen worker 969 - jepsen.util 3969 :info :add 10881 :RequestTimeoutException INFO [2022-01-10 08:40:19,930] jepsen worker 167 - jepsen.util 4167 :info :add 10882 :RequestTimeoutException INFO [2022-01-10 08:40:19,933] jepsen worker 887 - jepsen.util 3887 :info :add 10883 :RequestTimeoutException INFO [2022-01-10 08:40:19,941] jepsen worker 202 - jepsen.util 2202 :info :add 10884 :RequestTimeoutException INFO [2022-01-10 08:40:19,943] jepsen worker 75 - jepsen.util 3075 :info :add 10885 :RequestTimeoutException INFO [2022-01-10 08:40:19,947] jepsen worker 353 - jepsen.util 3353 :info :add 10887 :RequestTimeoutException INFO [2022-01-10 08:40:19,952] jepsen worker 570 - jepsen.util 3570 :info :add 10886 :RequestTimeoutException INFO [2022-01-10 08:40:19,970] jepsen worker 660 - jepsen.util 4660 :info :add 10888 :RequestTimeoutException INFO [2022-01-10 08:40:19,981] jepsen worker 775 - jepsen.util 4775 :info :add 10890 :RequestTimeoutException INFO [2022-01-10 08:40:19,983] jepsen worker 360 - jepsen.util 2360 :info :add 10889 :RequestTimeoutException INFO [2022-01-10 08:40:19,993] jepsen worker 388 - jepsen.util 3388 :info :add 10891 :RequestTimeoutException INFO [2022-01-10 08:40:20,002] jepsen worker 177 - jepsen.util 2177 :info :add 10892 :RequestTimeoutException INFO [2022-01-10 08:40:20,006] jepsen worker 133 - jepsen.util 4133 :info :add 10893 :RequestTimeoutException INFO [2022-01-10 08:40:20,010] jepsen worker 950 - jepsen.util 2950 :info :add 10894 :RequestTimeoutException INFO [2022-01-10 08:40:20,011] jepsen worker 683 - jepsen.util 2683 :info :add 10896 :RequestTimeoutException INFO [2022-01-10 08:40:20,013] jepsen worker 911 - jepsen.util 3911 :info :add 10895 :RequestTimeoutException INFO [2022-01-10 08:40:20,021] jepsen worker 183 - jepsen.util 3183 :info :add 10898 :RequestTimeoutException INFO [2022-01-10 08:40:20,027] jepsen worker 237 - jepsen.util 2237 :info :add 10899 :RequestTimeoutException INFO [2022-01-10 08:40:20,030] jepsen worker 833 - jepsen.util 3833 :info :add 10900 :RequestTimeoutException INFO [2022-01-10 08:40:20,033] jepsen worker 808 - jepsen.util 3808 :info :add 10901 :RequestTimeoutException INFO [2022-01-10 08:40:20,041] jepsen worker 540 - jepsen.util 4540 :info :add 10902 :RequestTimeoutException INFO [2022-01-10 08:40:20,044] jepsen worker 517 - jepsen.util 4517 :info :add 10903 :RequestTimeoutException INFO [2022-01-10 08:40:20,052] jepsen worker 534 - jepsen.util 2534 :info :add 10904 :RequestTimeoutException INFO [2022-01-10 08:40:20,056] jepsen worker 54 - jepsen.util 2054 :info :add 10905 :RequestTimeoutException INFO [2022-01-10 08:40:20,060] jepsen worker 963 - jepsen.util 2963 :info :add 10906 :RequestTimeoutException INFO [2022-01-10 08:40:20,063] jepsen worker 165 - jepsen.util 4165 :info :add 10907 :RequestTimeoutException INFO [2022-01-10 08:40:20,081] jepsen worker 829 - jepsen.util 4829 :info :add 10908 :RequestTimeoutException INFO [2022-01-10 08:40:20,093] jepsen worker 119 - jepsen.util 1119 :info :add 10909 :RequestTimeoutException INFO [2022-01-10 08:40:20,102] jepsen worker 390 - jepsen.util 2390 :info :add 10910 :RequestTimeoutException INFO [2022-01-10 08:40:20,107] jepsen worker 754 - jepsen.util 3754 :info :add 10911 :RequestTimeoutException INFO [2022-01-10 08:40:20,111] jepsen worker 163 - jepsen.util 4163 :info :add 10912 :RequestTimeoutException INFO [2022-01-10 08:40:20,121] jepsen worker 422 - jepsen.util 2422 :info :add 10914 :RequestTimeoutException INFO [2022-01-10 08:40:20,122] jepsen worker 584 - jepsen.util 3584 :info :add 10913 :RequestTimeoutException INFO [2022-01-10 08:40:20,131] jepsen worker 519 - jepsen.util 3519 :info :add 10916 :RequestTimeoutException INFO [2022-01-10 08:40:20,133] jepsen worker 146 - jepsen.util 2146 :info :add 10915 :RequestTimeoutException INFO [2022-01-10 08:40:20,137] jepsen worker 295 - jepsen.util 3295 :info :add 10917 :RequestTimeoutException INFO [2022-01-10 08:40:20,140] jepsen worker 23 - jepsen.util 3023 :info :add 10918 :RequestTimeoutException INFO [2022-01-10 08:40:20,151] jepsen worker 668 - jepsen.util 3668 :info :add 10920 :RequestTimeoutException INFO [2022-01-10 08:40:20,153] jepsen worker 377 - jepsen.util 3377 :info :add 10919 :RequestTimeoutException INFO [2022-01-10 08:40:20,154] jepsen worker 431 - jepsen.util 3431 :info :add 10921 :RequestTimeoutException INFO [2022-01-10 08:40:20,162] jepsen worker 906 - jepsen.util 4906 :info :add 10922 :RequestTimeoutException INFO [2022-01-10 08:40:20,162] jepsen worker 389 - jepsen.util 2389 :ok :add 10958 INFO [2022-01-10 08:40:20,163] jepsen worker 503 - jepsen.util 4503 :ok :add 11068 INFO [2022-01-10 08:40:20,167] jepsen worker 3 - jepsen.util 3003 :info :add 10923 :RequestTimeoutException INFO [2022-01-10 08:40:20,170] jepsen worker 941 - jepsen.util 4941 :info :add 10924 :RequestTimeoutException INFO [2022-01-10 08:40:20,211] jepsen worker 634 - jepsen.util 3634 :info :add 10926 :RequestTimeoutException INFO [2022-01-10 08:40:20,212] jepsen worker 59 - jepsen.util 3059 :info :add 10925 :RequestTimeoutException INFO [2022-01-10 08:40:20,214] jepsen worker 999 - jepsen.util 2999 :info :add 10927 :RequestTimeoutException INFO [2022-01-10 08:40:20,222] jepsen worker 966 - jepsen.util 4966 :info :add 10928 :RequestTimeoutException INFO [2022-01-10 08:40:20,227] jepsen worker 162 - jepsen.util 4162 :info :add 10929 :RequestTimeoutException INFO [2022-01-10 08:40:20,230] jepsen worker 152 - jepsen.util 3152 :info :add 10930 :RequestTimeoutException INFO [2022-01-10 08:40:20,233] jepsen worker 352 - jepsen.util 4352 :info :add 10931 :RequestTimeoutException INFO [2022-01-10 08:40:20,251] jepsen worker 886 - jepsen.util 4886 :info :add 10932 :RequestTimeoutException INFO [2022-01-10 08:40:20,272] jepsen worker 280 - jepsen.util 3280 :info :add 10934 :RequestTimeoutException INFO [2022-01-10 08:40:20,277] jepsen worker 115 - jepsen.util 2115 :info :add 10935 :RequestTimeoutException INFO [2022-01-10 08:40:20,280] jepsen worker 582 - jepsen.util 3582 :info :add 10936 :RequestTimeoutException INFO [2022-01-10 08:40:20,283] jepsen worker 373 - jepsen.util 3373 :info :add 10937 :RequestTimeoutException INFO [2022-01-10 08:40:20,291] jepsen worker 415 - jepsen.util 3415 :info :add 10938 :RequestTimeoutException INFO [2022-01-10 08:40:20,292] jepsen worker 895 - jepsen.util 3895 :info :add 10940 :RequestTimeoutException INFO [2022-01-10 08:40:20,293] jepsen worker 823 - jepsen.util 4823 :info :add 10939 :RequestTimeoutException INFO [2022-01-10 08:40:20,296] jepsen worker 940 - jepsen.util 3940 :info :add 10941 :RequestTimeoutException INFO [2022-01-10 08:40:20,300] jepsen worker 39 - jepsen.util 3039 :info :add 10942 :RequestTimeoutException INFO [2022-01-10 08:40:20,311] jepsen worker 409 - jepsen.util 3409 :info :add 10944 :RequestTimeoutException INFO [2022-01-10 08:40:20,313] jepsen worker 616 - jepsen.util 3616 :info :add 10943 :RequestTimeoutException INFO [2022-01-10 08:40:20,323] jepsen worker 558 - jepsen.util 4558 :info :add 10945 :RequestTimeoutException INFO [2022-01-10 08:40:20,332] jepsen worker 319 - jepsen.util 3319 :info :add 10946 :RequestTimeoutException INFO [2022-01-10 08:40:20,337] jepsen worker 786 - jepsen.util 4786 :info :add 10947 :RequestTimeoutException INFO [2022-01-10 08:40:20,340] jepsen worker 975 - jepsen.util 3975 :info :add 10948 :RequestTimeoutException INFO [2022-01-10 08:40:20,343] jepsen worker 589 - jepsen.util 4589 :info :add 10950 :RequestTimeoutException INFO [2022-01-10 08:40:20,343] jepsen worker 704 - jepsen.util 3704 :info :add 10949 :RequestTimeoutException INFO [2022-01-10 08:40:20,351] jepsen worker 250 - jepsen.util 3250 :info :add 10951 :RequestTimeoutException INFO [2022-01-10 08:40:20,352] jepsen worker 934 - jepsen.util 3934 :info :add 10953 :RequestTimeoutException INFO [2022-01-10 08:40:20,353] jepsen worker 599 - jepsen.util 4599 :info :add 10952 :RequestTimeoutException INFO [2022-01-10 08:40:20,357] jepsen worker 29 - jepsen.util 4029 :info :add 10954 :RequestTimeoutException INFO [2022-01-10 08:40:20,360] jepsen worker 189 - jepsen.util 4189 :info :add 10955 :RequestTimeoutException INFO [2022-01-10 08:40:20,361] jepsen worker 21 - jepsen.util 2021 :info :add 10957 :RequestTimeoutException INFO [2022-01-10 08:40:20,363] jepsen worker 707 - jepsen.util 2707 :info :add 10956 :RequestTimeoutException INFO [2022-01-10 08:40:20,372] jepsen worker 664 - jepsen.util 3664 :info :add 10959 :RequestTimeoutException INFO [2022-01-10 08:40:20,377] jepsen worker 628 - jepsen.util 4628 :info :add 10960 :RequestTimeoutException INFO [2022-01-10 08:40:20,380] jepsen worker 988 - jepsen.util 4988 :info :add 10961 :RequestTimeoutException INFO [2022-01-10 08:40:20,381] jepsen worker 777 - jepsen.util 2777 :info :add 10963 :RequestTimeoutException INFO [2022-01-10 08:40:20,383] jepsen worker 794 - jepsen.util 3794 :info :add 10962 :RequestTimeoutException INFO [2022-01-10 08:40:20,383] jepsen worker 350 - jepsen.util 4350 :info :add 10964 :RequestTimeoutException INFO [2022-01-10 08:40:20,392] jepsen worker 109 - jepsen.util 3109 :info :add 10965 :RequestTimeoutException INFO [2022-01-10 08:40:20,397] jepsen worker 864 - jepsen.util 4864 :info :add 10966 :RequestTimeoutException INFO [2022-01-10 08:40:20,401] jepsen worker 187 - jepsen.util 4187 :info :add 10969 :RequestTimeoutException INFO [2022-01-10 08:40:20,402] jepsen worker 311 - jepsen.util 4311 :info :add 10968 :RequestTimeoutException INFO [2022-01-10 08:40:20,412] jepsen worker 719 - jepsen.util 3719 :info :add 10971 :RequestTimeoutException INFO [2022-01-10 08:40:20,413] jepsen worker 653 - jepsen.util 4653 :info :add 10970 :RequestTimeoutException INFO [2022-01-10 08:40:20,417] jepsen worker 982 - jepsen.util 3982 :info :add 10972 :RequestTimeoutException INFO [2022-01-10 08:40:20,420] jepsen worker 602 - jepsen.util 4602 :info :add 10973 :RequestTimeoutException INFO [2022-01-10 08:40:20,423] jepsen worker 781 - jepsen.util 3781 :info :add 10974 :RequestTimeoutException INFO [2022-01-10 08:40:20,431] jepsen worker 962 - jepsen.util 4962 :info :add 10975 :RequestTimeoutException INFO [2022-01-10 08:40:20,437] jepsen worker 657 - jepsen.util 4657 :info :add 10978 :RequestTimeoutException INFO [2022-01-10 08:40:20,440] jepsen worker 164 - jepsen.util 2164 :info :add 10979 :RequestTimeoutException INFO [2022-01-10 08:40:20,442] jepsen worker 323 - jepsen.util 4323 :info :add 10977 :RequestTimeoutException INFO [2022-01-10 08:40:20,443] jepsen worker 89 - jepsen.util 4089 :info :add 10980 :RequestTimeoutException INFO [2022-01-10 08:40:20,451] jepsen worker 702 - jepsen.util 2702 :info :add 10981 :RequestTimeoutException INFO [2022-01-10 08:40:20,452] jepsen worker 331 - jepsen.util 3331 :info :add 10983 :RequestTimeoutException INFO [2022-01-10 08:40:20,453] jepsen worker 58 - jepsen.util 4058 :info :add 10982 :RequestTimeoutException INFO [2022-01-10 08:40:20,460] jepsen worker 935 - jepsen.util 4935 :info :add 10985 :RequestTimeoutException INFO [2022-01-10 08:40:20,462] jepsen worker 979 - jepsen.util 3979 :info :add 10986 :RequestTimeoutException INFO [2022-01-10 08:40:20,467] jepsen worker 494 - jepsen.util 2494 :info :add 10984 :RequestTimeoutException INFO [2022-01-10 08:40:20,471] jepsen worker 699 - jepsen.util 4699 :info :add 10987 :RequestTimeoutException INFO [2022-01-10 08:40:20,472] jepsen worker 689 - jepsen.util 4689 :info :add 10989 :RequestTimeoutException INFO [2022-01-10 08:40:20,473] jepsen worker 461 - jepsen.util 3461 :info :add 10988 :RequestTimeoutException INFO [2022-01-10 08:40:20,476] jepsen worker 883 - jepsen.util 4883 :info :add 10990 :RequestTimeoutException INFO [2022-01-10 08:40:20,480] jepsen worker 818 - jepsen.util 4818 :info :add 10991 :RequestTimeoutException INFO [2022-01-10 08:40:20,483] jepsen worker 743 - jepsen.util 4743 :info :add 10992 :RequestTimeoutException INFO [2022-01-10 08:40:20,491] jepsen worker 483 - jepsen.util 3483 :info :add 10993 :RequestTimeoutException INFO [2022-01-10 08:40:20,492] jepsen worker 714 - jepsen.util 4714 :info :add 10995 :RequestTimeoutException INFO [2022-01-10 08:40:20,497] jepsen worker 751 - jepsen.util 4751 :info :add 10996 :RequestTimeoutException INFO [2022-01-10 08:40:20,510] jepsen worker 713 - jepsen.util 4713 :info :add 10997 :RequestTimeoutException INFO [2022-01-10 08:40:20,511] jepsen worker 301 - jepsen.util 3301 :info :add 10999 :RequestTimeoutException INFO [2022-01-10 08:40:20,512] jepsen worker 241 - jepsen.util 4241 :info :add 11001 :RequestTimeoutException INFO [2022-01-10 08:40:20,512] jepsen worker 178 - jepsen.util 3178 :info :add 10998 :RequestTimeoutException INFO [2022-01-10 08:40:20,513] jepsen worker 79 - jepsen.util 3079 :info :add 11000 :RequestTimeoutException INFO [2022-01-10 08:40:20,516] jepsen worker 703 - jepsen.util 4703 :info :add 11002 :RequestTimeoutException INFO [2022-01-10 08:40:20,520] jepsen worker 314 - jepsen.util 4314 :info :add 11003 :RequestTimeoutException INFO [2022-01-10 08:40:20,521] jepsen worker 69 - jepsen.util 3069 :info :add 11005 :RequestTimeoutException INFO [2022-01-10 08:40:20,523] jepsen worker 392 - jepsen.util 4392 :info :add 11004 :RequestTimeoutException INFO [2022-01-10 08:40:20,523] jepsen worker 915 - jepsen.util 2915 :info :add 11006 :RequestTimeoutException INFO [2022-01-10 08:40:20,532] jepsen worker 726 - jepsen.util 4726 :info :add 11007 :RequestTimeoutException INFO [2022-01-10 08:40:20,537] jepsen worker 122 - jepsen.util 4122 :info :add 11008 :RequestTimeoutException INFO [2022-01-10 08:40:20,543] jepsen worker 492 - jepsen.util 3492 :info :add 11010 :RequestTimeoutException INFO [2022-01-10 08:40:20,550] jepsen worker 766 - jepsen.util 4766 :info :add 11011 :RequestTimeoutException INFO [2022-01-10 08:40:20,562] jepsen worker 354 - jepsen.util 3354 :info :add 11013 :RequestTimeoutException INFO [2022-01-10 08:40:20,563] jepsen worker 522 - jepsen.util 4522 :info :add 11012 :RequestTimeoutException INFO [2022-01-10 08:40:20,567] jepsen worker 93 - jepsen.util 4093 :info :add 11014 :RequestTimeoutException INFO [2022-01-10 08:40:20,570] jepsen worker 594 - jepsen.util 2594 :info :add 11015 :RequestTimeoutException INFO [2022-01-10 08:40:20,571] jepsen worker 470 - jepsen.util 3470 :info :add 11017 :RequestTimeoutException INFO [2022-01-10 08:40:20,573] jepsen worker 642 - jepsen.util 4642 :info :add 11016 :RequestTimeoutException INFO [2022-01-10 08:40:20,573] jepsen worker 158 - jepsen.util 3158 :info :add 11018 :RequestTimeoutException INFO [2022-01-10 08:40:20,580] jepsen worker 498 - jepsen.util 3498 :info :add 11021 :RequestTimeoutException INFO [2022-01-10 08:40:20,582] jepsen worker 497 - jepsen.util 3497 :info :add 11019 :RequestTimeoutException INFO [2022-01-10 08:40:20,583] jepsen worker 825 - jepsen.util 3825 :info :add 11022 :RequestTimeoutException INFO [2022-01-10 08:40:20,587] jepsen worker 745 - jepsen.util 3745 :info :add 11020 :RequestTimeoutException INFO [2022-01-10 08:40:20,591] jepsen worker 420 - jepsen.util 3420 :info :add 11023 :RequestTimeoutException INFO [2022-01-10 08:40:20,604] jepsen worker 418 - jepsen.util 3418 :info :add 11024 :RequestTimeoutException INFO [2022-01-10 08:40:20,611] jepsen worker 145 - jepsen.util 3145 :info :add 11029 :RequestTimeoutException INFO [2022-01-10 08:40:20,612] jepsen worker 223 - jepsen.util 3223 :info :add 11028 :RequestTimeoutException INFO [2022-01-10 08:40:20,613] jepsen worker 917 - jepsen.util 3917 :info :add 11030 :RequestTimeoutException INFO [2022-01-10 08:40:20,622] jepsen worker 900 - jepsen.util 4900 :info :add 11031 :RequestTimeoutException INFO [2022-01-10 08:40:20,626] jepsen worker 407 - jepsen.util 2407 :info :add 11032 :RequestTimeoutException INFO [2022-01-10 08:40:20,630] jepsen worker 847 - jepsen.util 3847 :info :add 11033 :RequestTimeoutException INFO [2022-01-10 08:40:20,631] jepsen worker 92 - jepsen.util 2092 :info :add 11035 :RequestTimeoutException INFO [2022-01-10 08:40:20,633] jepsen worker 87 - jepsen.util 3087 :info :add 11034 :RequestTimeoutException INFO [2022-01-10 08:40:20,652] jepsen worker 421 - jepsen.util 4421 :info :add 11037 :RequestTimeoutException INFO [2022-01-10 08:40:20,657] jepsen worker 927 - jepsen.util 4927 :info :add 11038 :RequestTimeoutException INFO [2022-01-10 08:40:20,660] jepsen worker 949 - jepsen.util 3949 :info :add 11039 :RequestTimeoutException INFO [2022-01-10 08:40:20,662] jepsen worker 912 - jepsen.util 3912 :info :add 11040 :RequestTimeoutException INFO [2022-01-10 08:40:20,663] jepsen worker 531 - jepsen.util 4531 :info :add 11042 :RequestTimeoutException INFO [2022-01-10 08:40:20,675] jepsen worker 824 - jepsen.util 3824 :info :add 11041 :RequestTimeoutException INFO [2022-01-10 08:40:20,732] jepsen worker 973 - jepsen.util 3973 :info :add 11043 :RequestTimeoutException INFO [2022-01-10 08:40:20,766] jepsen worker 730 - jepsen.util 4730 :info :add 11044 :RequestTimeoutException INFO [2022-01-10 08:40:20,793] jepsen worker 581 - jepsen.util 3581 :info :add 11045 :RequestTimeoutException INFO [2022-01-10 08:40:20,800] jepsen worker 651 - jepsen.util 4651 :info :add 11046 :RequestTimeoutException INFO [2022-01-10 08:40:20,803] jepsen worker 644 - jepsen.util 4644 :info :add 11047 :RequestTimeoutException INFO [2022-01-10 08:40:20,803] jepsen worker 86 - jepsen.util 3086 :info :add 11048 :RequestTimeoutException INFO [2022-01-10 08:40:20,841] jepsen worker 959 - jepsen.util 4959 :info :add 11049 :RequestTimeoutException INFO [2022-01-10 08:40:20,852] jepsen worker 740 - jepsen.util 4740 :info :add 11051 :RequestTimeoutException INFO [2022-01-10 08:40:20,853] jepsen worker 379 - jepsen.util 4379 :info :add 11050 :RequestTimeoutException INFO [2022-01-10 08:40:20,857] jepsen worker 884 - jepsen.util 4884 :info :add 11052 :RequestTimeoutException INFO [2022-01-10 08:40:20,860] jepsen worker 980 - jepsen.util 3980 :info :add 11053 :RequestTimeoutException INFO [2022-01-10 08:40:20,871] jepsen worker 646 - jepsen.util 4646 :info :add 11055 :RequestTimeoutException INFO [2022-01-10 08:40:20,873] jepsen worker 153 - jepsen.util 4153 :info :add 11054 :RequestTimeoutException INFO [2022-01-10 08:40:21,052] jepsen worker 235 - jepsen.util 4235 :info :add 11057 :RequestTimeoutException INFO [2022-01-10 08:40:21,053] jepsen worker 374 - jepsen.util 3374 :info :add 11056 :RequestTimeoutException INFO [2022-01-10 08:40:21,140] jepsen worker 854 - jepsen.util 4854 :info :add 11059 :RequestTimeoutException INFO [2022-01-10 08:40:21,147] jepsen worker 958 - jepsen.util 4958 :info :add 11058 :RequestTimeoutException INFO [2022-01-10 08:40:21,162] jepsen worker 530 - jepsen.util 3530 :info :add 11060 :RequestTimeoutException INFO [2022-01-10 08:40:21,191] jepsen worker 224 - jepsen.util 3224 :info :add 11061 :RequestTimeoutException INFO [2022-01-10 08:40:21,193] jepsen worker 533 - jepsen.util 3533 :info :add 11062 :RequestTimeoutException INFO [2022-01-10 08:40:21,212] jepsen worker 414 - jepsen.util 3414 :info :add 11063 :RequestTimeoutException INFO [2022-01-10 08:40:21,257] jepsen worker 245 - jepsen.util 3245 :info :add 11064 :RequestTimeoutException INFO [2022-01-10 08:40:21,270] jepsen worker 758 - jepsen.util 3758 :info :add 11065 :RequestTimeoutException INFO [2022-01-10 08:40:21,272] jepsen worker 342 - jepsen.util 3342 :info :add 11066 :RequestTimeoutException INFO [2022-01-10 08:40:21,281] jepsen worker 796 - jepsen.util 3796 :info :add 11067 :RequestTimeoutException INFO [2022-01-10 08:40:21,282] jepsen worker 523 - jepsen.util 3523 :info :add 11069 :RequestTimeoutException INFO [2022-01-10 08:40:21,287] jepsen worker 675 - jepsen.util 4675 :info :add 11070 :RequestTimeoutException INFO [2022-01-10 08:40:21,300] jepsen worker 30 - jepsen.util 4030 :info :add 11071 :RequestTimeoutException INFO [2022-01-10 08:40:21,301] jepsen worker 752 - jepsen.util 4752 :info :add 11073 :RequestTimeoutException INFO [2022-01-10 08:40:21,303] jepsen worker 779 - jepsen.util 4779 :info :add 11072 :RequestTimeoutException INFO [2022-01-10 08:40:21,303] jepsen worker 954 - jepsen.util 3954 :info :add 11074 :RequestTimeoutException INFO [2022-01-10 08:40:21,307] jepsen worker 313 - jepsen.util 3313 :info :add 11076 :RequestTimeoutException INFO [2022-01-10 08:40:21,312] jepsen worker 135 - jepsen.util 3135 :info :add 11075 :RequestTimeoutException INFO [2022-01-10 08:40:21,320] jepsen worker 592 - jepsen.util 3592 :info :add 11077 :RequestTimeoutException INFO [2022-01-10 08:40:21,394] jepsen worker 669 - jepsen.util 2669 :info :add 11078 :RequestTimeoutException INFO [2022-01-10 08:40:21,394] jepsen worker 663 - jepsen.util 3663 :info :add 11079 :RequestTimeoutException INFO [2022-01-10 08:40:21,533] jepsen worker 798 - jepsen.util 3798 :info :add 11080 :RequestTimeoutException INFO [2022-01-10 08:40:21,541] jepsen worker 393 - jepsen.util 4393 :info :add 11081 :RequestTimeoutException INFO [2022-01-10 08:40:21,560] jepsen worker 691 - jepsen.util 2691 :info :add 11084 :RequestTimeoutException INFO [2022-01-10 08:40:21,564] jepsen worker 289 - jepsen.util 3289 :info :add 11082 :RequestTimeoutException INFO [2022-01-10 08:40:21,564] jepsen worker 605 - jepsen.util 2605 :info :add 11083 :RequestTimeoutException INFO [2022-01-10 08:40:21,613] jepsen worker 640 - jepsen.util 2640 :info :add 11085 :RequestTimeoutException INFO [2022-01-10 08:40:21,640] jepsen worker 460 - jepsen.util 3460 :info :add 11087 :RequestTimeoutException INFO [2022-01-10 08:40:21,641] jepsen worker 921 - jepsen.util 2921 :info :add 11088 :RequestTimeoutException INFO [2022-01-10 08:40:21,643] jepsen worker 348 - jepsen.util 3348 :info :add 11086 :RequestTimeoutException INFO [2022-01-10 08:40:21,650] jepsen worker 811 - jepsen.util 3811 :info :add 11089 :RequestTimeoutException INFO [2022-01-10 08:40:21,650] jepsen worker 321 - jepsen.util 2321 :info :add 11090 :RequestTimeoutException INFO [2022-01-10 08:40:21,652] jepsen worker 710 - jepsen.util 3710 :info :add 11092 :RequestTimeoutException INFO [2022-01-10 08:40:21,657] jepsen worker 698 - jepsen.util 3698 :info :add 11093 :RequestTimeoutException INFO [2022-01-10 08:40:21,670] jepsen worker 238 - jepsen.util 4238 :info :add 11094 :RequestTimeoutException INFO [2022-01-10 08:40:21,681] jepsen worker 783 - jepsen.util 3783 :info :add 11096 :RequestTimeoutException INFO [2022-01-10 08:40:21,683] jepsen worker 435 - jepsen.util 3435 :info :add 11095 :RequestTimeoutException INFO [2022-01-10 08:40:21,692] jepsen worker 499 - jepsen.util 3499 :info :add 11098 :RequestTimeoutException INFO [2022-01-10 08:40:21,694] jepsen worker 8 - jepsen.util 3008 :info :add 11097 :RequestTimeoutException INFO [2022-01-10 08:40:21,697] jepsen worker 112 - jepsen.util 4112 :info :add 11099 :RequestTimeoutException INFO [2022-01-10 08:40:21,700] jepsen worker 104 - jepsen.util 4104 :info :add 11100 :RequestTimeoutException INFO [2022-01-10 08:40:21,712] jepsen worker 453 - jepsen.util 3453 :info :add 11101 :RequestTimeoutException INFO [2022-01-10 08:40:21,761] jepsen worker 395 - jepsen.util 3395 :info :add 11102 :RequestTimeoutException INFO [2022-01-10 08:40:21,832] jepsen worker 563 - jepsen.util 4563 :info :add 11104 :RequestTimeoutException INFO [2022-01-10 08:40:21,837] jepsen worker 968 - jepsen.util 4968 :info :add 11105 :RequestTimeoutException INFO [2022-01-10 08:40:21,866] jepsen worker 681 - jepsen.util 2681 :info :add 11106 :RequestTimeoutException INFO [2022-01-10 08:40:21,866] jepsen worker 622 - jepsen.util 2622 :info :add 11108 :RequestTimeoutException INFO [2022-01-10 08:40:21,873] jepsen worker 364 - jepsen.util 3364 :info :add 11107 :RequestTimeoutException INFO [2022-01-10 08:40:21,960] jepsen worker 446 - jepsen.util 3446 :info :add 11110 :RequestTimeoutException INFO [2022-01-10 08:40:21,960] jepsen worker 447 - jepsen.util 3447 :info :add 11109 :RequestTimeoutException INFO [2022-01-10 08:40:21,991] jepsen worker 995 - jepsen.util 4995 :info :add 11111 :RequestTimeoutException INFO [2022-01-10 08:40:22,001] jepsen worker 520 - jepsen.util 4520 :info :add 11113 :RequestTimeoutException INFO [2022-01-10 08:40:22,003] jepsen worker 66 - jepsen.util 4066 :info :add 11112 :RequestTimeoutException INFO [2022-01-10 08:40:22,022] jepsen worker 216 - jepsen.util 4216 :info :add 11115 :RequestTimeoutException INFO [2022-01-10 08:40:22,023] jepsen worker 256 - jepsen.util 4256 :info :add 11114 :RequestTimeoutException INFO [2022-01-10 08:40:22,086] jepsen worker 506 - jepsen.util 3506 :info :add 11116 :RequestTimeoutException INFO [2022-01-10 08:40:22,110] jepsen worker 564 - jepsen.util 3564 :info :add 11117 :RequestTimeoutException INFO [2022-01-10 08:40:22,121] jepsen worker 579 - jepsen.util 3579 :info :add 11119 :RequestTimeoutException INFO [2022-01-10 08:40:22,123] jepsen worker 303 - jepsen.util 3303 :info :add 11118 :RequestTimeoutException INFO [2022-01-10 08:40:22,124] jepsen worker 371 - jepsen.util 3371 :info :add 11120 :RequestTimeoutException INFO [2022-01-10 08:40:22,128] jepsen worker 817 - jepsen.util 5817 :invoke :read 11120 WARN [2022-01-10 08:40:23,182] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":9969661,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","total_dispatch_duration_us":9969661,"total_server_duration_us":4109780,"encode_duration_us":386,"operation_id":"0x2ec8","timeout_ms":10000,"last_server_duration_us":4109780,"total_duration_us":10010823},{"operation_name":"insert","last_dispatch_duration_us":13797637,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000001/00000000CB02B145","last_local_socket":"172.23.104.79:41914","total_dispatch_duration_us":13797637,"total_server_duration_us":5040013,"encode_duration_us":312,"operation_id":"0x2fb6","timeout_ms":10000,"last_server_duration_us":5040013,"total_duration_us":10010778},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","encode_duration_us":370,"operation_id":"0x3055","timeout_ms":10000,"total_duration_us":10010768},{"operation_name":"insert","last_dispatch_duration_us":15274504,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000001/000000005D894E7E","last_local_socket":"172.23.104.79:51762","total_dispatch_duration_us":15274504,"total_server_duration_us":6453471,"encode_duration_us":353,"operation_id":"0x2fd4","timeout_ms":10000,"last_server_duration_us":6453471,"total_duration_us":10010651},{"operation_name":"insert","last_dispatch_duration_us":14988221,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000001/000000005D894E7E","last_local_socket":"172.23.104.79:51762","total_dispatch_duration_us":14988221,"total_server_duration_us":6446115,"encode_duration_us":309,"operation_id":"0x300d","timeout_ms":10000,"last_server_duration_us":6446115,"total_duration_us":10010413},{"operation_name":"insert","last_dispatch_duration_us":12043715,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000001/000000005D894E7E","last_local_socket":"172.23.104.79:51762","total_dispatch_duration_us":12043715,"total_server_duration_us":699606,"encode_duration_us":300,"operation_id":"0x2e44","timeout_ms":10000,"last_server_duration_us":699606,"total_duration_us":10010359},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","encode_duration_us":295,"operation_id":"0x3291","timeout_ms":10000,"total_duration_us":10010332},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","encode_duration_us":264,"operation_id":"0x3253","timeout_ms":10000,"total_duration_us":10010188},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_local_socket":"172.23.104.79:59920","encode_duration_us":316,"operation_id":"0x30af","timeout_ms":10000,"total_duration_us":10010096},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000001/000000005A213BC7","last_local_socket":"172.23.104.79:46170","encode_duration_us":272,"operation_id":"0x322f","timeout_ms":10000,"total_duration_us":10010052}],"total_count":152}} WARN [2022-01-10 08:40:23,182] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":16323767,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16323767,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":9007669,"encode_duration_us":338,"operation_id":"0x2d22","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10010288},{"operation_name":"insert","last_dispatch_duration_us":12043715,"last_local_id":"2B99390F00000001/000000005D894E7E","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":12043715,"last_local_socket":"172.23.104.79:51762","total_server_duration_us":699606,"encode_duration_us":300,"operation_id":"0x2e44","timeout_ms":10000,"last_server_duration_us":699606,"total_duration_us":10010359},{"operation_name":"insert","last_dispatch_duration_us":16464147,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16464147,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":9004490,"encode_duration_us":305,"operation_id":"0x2af8","timeout_ms":10000,"last_server_duration_us":9004490,"total_duration_us":10010475},{"operation_name":"insert","last_dispatch_duration_us":18897883,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":18897883,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":9004490,"encode_duration_us":303,"operation_id":"0x2bb0","timeout_ms":10000,"last_server_duration_us":9004490,"total_duration_us":10010625},{"operation_name":"insert","last_dispatch_duration_us":14988221,"last_local_id":"2B99390F00000001/000000005D894E7E","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":14988221,"last_local_socket":"172.23.104.79:51762","total_server_duration_us":6446115,"encode_duration_us":309,"operation_id":"0x300d","timeout_ms":10000,"last_server_duration_us":6446115,"total_duration_us":10010413},{"operation_name":"insert","last_dispatch_duration_us":15657917,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15657917,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":8221633,"encode_duration_us":432,"operation_id":"0x2a95","timeout_ms":10000,"last_server_duration_us":8221633,"total_duration_us":10010779},{"operation_name":"insert","last_dispatch_duration_us":15274504,"last_local_id":"2B99390F00000001/000000005D894E7E","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":15274504,"last_local_socket":"172.23.104.79:51762","total_server_duration_us":6453471,"encode_duration_us":353,"operation_id":"0x2fd4","timeout_ms":10000,"last_server_duration_us":6453471,"total_duration_us":10010651},{"operation_name":"insert","last_dispatch_duration_us":14037343,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14037343,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":5069857,"encode_duration_us":312,"operation_id":"0x2cf5","timeout_ms":10000,"last_server_duration_us":5069857,"total_duration_us":10010751},{"operation_name":"insert","last_dispatch_duration_us":9969661,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9969661,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":4109780,"encode_duration_us":386,"operation_id":"0x2ec8","timeout_ms":10000,"last_server_duration_us":4109780,"total_duration_us":10010823},{"operation_name":"insert","last_dispatch_duration_us":13797637,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13797637,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":5040013,"encode_duration_us":312,"operation_id":"0x2fb6","timeout_ms":10000,"last_server_duration_us":5040013,"total_duration_us":10010778}],"total_count":142}} WARN [2022-01-10 08:40:24,588] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":15287811,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15287811,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":6361777,"encode_duration_us":309,"operation_id":"0x2c55","timeout_ms":10000,"last_server_duration_us":6361777,"total_duration_us":10010153},{"operation_name":"insert","last_dispatch_duration_us":17958592,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17958592,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":9015090,"encode_duration_us":322,"operation_id":"0x300a","timeout_ms":10000,"last_server_duration_us":9015090,"total_duration_us":10010281},{"operation_name":"insert","last_dispatch_duration_us":17909015,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17909015,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":9007669,"encode_duration_us":308,"operation_id":"0x2fad","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10010292},{"operation_name":"insert","last_dispatch_duration_us":34012515,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":34012515,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":9016150,"encode_duration_us":441,"operation_id":"0x23b9","timeout_ms":10000,"last_server_duration_us":9016150,"total_duration_us":10010562},{"operation_name":"insert","last_dispatch_duration_us":29468967,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":29468967,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":840244,"encode_duration_us":255,"operation_id":"0x2985","timeout_ms":10000,"last_server_duration_us":840244,"total_duration_us":10010284},{"operation_name":"insert","last_dispatch_duration_us":17962005,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17962005,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":9009789,"encode_duration_us":312,"operation_id":"0x2e6a","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10010746},{"operation_name":"insert","last_dispatch_duration_us":32928438,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":32928438,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":8998133,"encode_duration_us":299,"operation_id":"0x25b5","timeout_ms":10000,"last_server_duration_us":8998133,"total_duration_us":10010356},{"operation_name":"insert","last_dispatch_duration_us":17499576,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17499576,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9010849,"encode_duration_us":324,"operation_id":"0x2bf2","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10078580},{"operation_name":"insert","last_dispatch_duration_us":30182814,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":30182814,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":1251785,"encode_duration_us":260,"operation_id":"0x2751","timeout_ms":10000,"last_server_duration_us":1251785,"total_duration_us":10010768},{"operation_name":"insert","last_dispatch_duration_us":26938935,"last_local_id":"2B99390F00000004/000000001581771D","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":26938935,"last_local_socket":"172.23.104.79:46242","total_server_duration_us":9010849,"encode_duration_us":254,"operation_id":"0x2576","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10011099}],"total_count":181}} WARN [2022-01-10 08:40:24,648] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","encode_duration_us":452,"operation_id":"0x327e","timeout_ms":10000,"total_duration_us":10014279},{"operation_name":"insert","last_dispatch_duration_us":12548746,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000002/0000000056958F5B","last_local_socket":"172.23.104.79:51728","total_dispatch_duration_us":12548746,"total_server_duration_us":6370006,"encode_duration_us":326,"operation_id":"0x2fb1","timeout_ms":10000,"last_server_duration_us":6370006,"total_duration_us":10010843},{"operation_name":"insert","last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000002/0000000056958F5B","last_local_socket":"172.23.104.79:51728","encode_duration_us":298,"operation_id":"0x323c","timeout_ms":10000,"total_duration_us":10010829},{"operation_name":"insert","encode_duration_us":346,"operation_id":"0x305c","timeout_ms":10000,"total_duration_us":10010749},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000002/00000000118F2C63","last_local_socket":"172.23.104.79:59942","encode_duration_us":285,"operation_id":"0x31c9","timeout_ms":10000,"total_duration_us":10010641},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","encode_duration_us":343,"operation_id":"0x30c0","timeout_ms":10000,"total_duration_us":10010179},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","encode_duration_us":332,"operation_id":"0x3087","timeout_ms":10000,"total_duration_us":10010139},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","encode_duration_us":303,"operation_id":"0x3242","timeout_ms":10000,"total_duration_us":10010124},{"operation_name":"insert","last_dispatch_duration_us":13105129,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_local_socket":"172.23.104.79:46216","total_dispatch_duration_us":13105129,"total_server_duration_us":452,"encode_duration_us":524,"operation_id":"0x3014","timeout_ms":10000,"last_server_duration_us":452,"total_duration_us":10010032},{"operation_name":"insert","encode_duration_us":366,"operation_id":"0x30cd","timeout_ms":10000,"total_duration_us":10010029}],"total_count":136}} WARN [2022-01-10 08:40:24,648] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":17629945,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17629945,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":9009789,"encode_duration_us":314,"operation_id":"0x2abb","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10009925},{"operation_name":"insert","last_dispatch_duration_us":33558422,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":33558422,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":808427,"encode_duration_us":260,"operation_id":"0x2510","timeout_ms":10000,"last_server_duration_us":808427,"total_duration_us":10010065},{"operation_name":"insert","last_dispatch_duration_us":33552644,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":33552644,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":7377504,"encode_duration_us":372,"operation_id":"0x2200","timeout_ms":10000,"last_server_duration_us":7377504,"total_duration_us":10010042},{"operation_name":"insert","last_dispatch_duration_us":17723627,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17723627,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":9014029,"encode_duration_us":309,"operation_id":"0x2ccb","timeout_ms":10000,"last_server_duration_us":9014029,"total_duration_us":10010395},{"operation_name":"insert","last_dispatch_duration_us":33304245,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":33304245,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":555734,"encode_duration_us":259,"operation_id":"0x254b","timeout_ms":10000,"last_server_duration_us":555734,"total_duration_us":10010085},{"operation_name":"insert","last_dispatch_duration_us":39814683,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":39814683,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":1343363,"encode_duration_us":294,"operation_id":"0x236f","timeout_ms":10000,"last_server_duration_us":1343363,"total_duration_us":10010710},{"operation_name":"insert","last_dispatch_duration_us":17669510,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17669510,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":9010849,"encode_duration_us":319,"operation_id":"0x2dda","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10010051},{"operation_name":"insert","last_dispatch_duration_us":34670524,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":34670524,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":1433309,"encode_duration_us":270,"operation_id":"0x22c7","timeout_ms":10000,"last_server_duration_us":1433309,"total_duration_us":10010457},{"operation_name":"insert","last_dispatch_duration_us":12548746,"last_local_id":"2B99390F00000002/0000000056958F5B","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":12548746,"last_local_socket":"172.23.104.79:51728","total_server_duration_us":6370006,"encode_duration_us":326,"operation_id":"0x2fb1","timeout_ms":10000,"last_server_duration_us":6370006,"total_duration_us":10010843},{"operation_name":"insert","last_dispatch_duration_us":33428283,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":33428283,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":663895,"encode_duration_us":260,"operation_id":"0x2631","timeout_ms":10000,"last_server_duration_us":663895,"total_duration_us":10010916}],"total_count":102}} WARN [2022-01-10 08:40:24,688] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":6391097,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":6391097,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":1425558,"encode_duration_us":338,"operation_id":"0x30df","timeout_ms":10000,"last_server_duration_us":1425558,"total_duration_us":10009878},{"operation_name":"insert","last_dispatch_duration_us":5715878,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":5715878,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":289840,"encode_duration_us":327,"operation_id":"0x32e1","timeout_ms":10000,"last_server_duration_us":289840,"total_duration_us":10009965},{"operation_name":"insert","last_dispatch_duration_us":26457065,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":26457065,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":9018271,"encode_duration_us":425,"operation_id":"0x2aa4","timeout_ms":10000,"last_server_duration_us":9018271,"total_duration_us":10010103},{"operation_name":"insert","last_dispatch_duration_us":17731793,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17731793,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":9010849,"encode_duration_us":349,"operation_id":"0x2b31","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10010272},{"operation_name":"insert","last_dispatch_duration_us":11370751,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":11370751,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":3355639,"encode_duration_us":614,"operation_id":"0x2e7c","timeout_ms":10000,"last_server_duration_us":3355639,"total_duration_us":10010032},{"operation_name":"insert","last_dispatch_duration_us":17496433,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17496433,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":9002371,"encode_duration_us":304,"operation_id":"0x2c2e","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10010966},{"operation_name":"insert","last_dispatch_duration_us":15896401,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15896401,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":9008729,"encode_duration_us":316,"operation_id":"0x2e4d","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10010924},{"operation_name":"insert","last_dispatch_duration_us":15390039,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15390039,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":7749417,"encode_duration_us":390,"operation_id":"0x30a5","timeout_ms":10000,"last_server_duration_us":7749417,"total_duration_us":10010721},{"operation_name":"insert","last_dispatch_duration_us":9134723,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9134723,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":1796664,"encode_duration_us":367,"operation_id":"0x2ea9","timeout_ms":10000,"last_server_duration_us":1796664,"total_duration_us":10010685},{"operation_name":"insert","last_dispatch_duration_us":14614941,"last_local_id":"2B99390F00000003/00000000919EB30B","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14614941,"last_local_socket":"172.23.104.79:41940","total_server_duration_us":8717169,"encode_duration_us":495,"operation_id":"0x2f75","timeout_ms":10000,"last_server_duration_us":8717169,"total_duration_us":10010786}],"total_count":113}} WARN [2022-01-10 08:40:24,688] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":14981953,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000005/0000000013738E6A","last_local_socket":"172.23.104.79:41944","total_dispatch_duration_us":14981953,"total_server_duration_us":7491788,"encode_duration_us":308,"operation_id":"0x2f05","timeout_ms":10000,"last_server_duration_us":7491788,"total_duration_us":10024410},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":334,"operation_id":"0x3102","timeout_ms":10000,"total_duration_us":10011013},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","encode_duration_us":324,"operation_id":"0x3175","timeout_ms":10000,"total_duration_us":10010723},{"operation_name":"insert","encode_duration_us":337,"operation_id":"0x3092","timeout_ms":10000,"total_duration_us":10010593},{"operation_name":"insert","last_dispatch_duration_us":8734921,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000005/0000000013738E6A","last_local_socket":"172.23.104.79:41944","total_dispatch_duration_us":8734921,"total_server_duration_us":2141134,"encode_duration_us":282,"operation_id":"0x3295","timeout_ms":10000,"last_server_duration_us":2141134,"total_duration_us":10010540},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","encode_duration_us":298,"operation_id":"0x31c6","timeout_ms":10000,"total_duration_us":10010422},{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000005/0000000086155695","last_local_socket":"172.23.104.79:46186","encode_duration_us":322,"operation_id":"0x32a1","timeout_ms":10000,"total_duration_us":10010420},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000005/000000009E68EEB4","last_local_socket":"172.23.104.79:59936","encode_duration_us":293,"operation_id":"0x317e","timeout_ms":10000,"total_duration_us":10010312},{"operation_name":"insert","last_dispatch_duration_us":14522753,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000005/0000000013738E6A","last_local_socket":"172.23.104.79:41944","total_dispatch_duration_us":14522753,"total_server_duration_us":5806666,"encode_duration_us":351,"operation_id":"0x30e9","timeout_ms":10000,"last_server_duration_us":5806666,"total_duration_us":10010211},{"operation_name":"insert","last_dispatch_duration_us":12776955,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000005/0000000013738E6A","last_local_socket":"172.23.104.79:41944","total_dispatch_duration_us":12776955,"total_server_duration_us":6207179,"encode_duration_us":341,"operation_id":"0x2f45","timeout_ms":10000,"last_server_duration_us":6207179,"total_duration_us":10010174}],"total_count":135}} WARN [2022-01-10 08:40:24,688] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":14614941,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","total_dispatch_duration_us":14614941,"total_server_duration_us":8717169,"encode_duration_us":495,"operation_id":"0x2f75","timeout_ms":10000,"last_server_duration_us":8717169,"total_duration_us":10010786},{"operation_name":"insert","last_dispatch_duration_us":15390039,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","total_dispatch_duration_us":15390039,"total_server_duration_us":7749417,"encode_duration_us":390,"operation_id":"0x30a5","timeout_ms":10000,"last_server_duration_us":7749417,"total_duration_us":10010721},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","encode_duration_us":263,"operation_id":"0x2f6a","timeout_ms":10000,"total_duration_us":10010454},{"operation_name":"insert","encode_duration_us":355,"operation_id":"0x2f78","timeout_ms":10000,"total_duration_us":10010351},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","encode_duration_us":270,"operation_id":"0x31b8","timeout_ms":10000,"total_duration_us":10010314},{"operation_name":"insert","encode_duration_us":322,"operation_id":"0x30c1","timeout_ms":10000,"total_duration_us":10009978},{"operation_name":"insert","last_dispatch_duration_us":5715878,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","total_dispatch_duration_us":5715878,"total_server_duration_us":289840,"encode_duration_us":327,"operation_id":"0x32e1","timeout_ms":10000,"last_server_duration_us":289840,"total_duration_us":10009965},{"operation_name":"insert","last_dispatch_duration_us":6391097,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","total_dispatch_duration_us":6391097,"total_server_duration_us":1425558,"encode_duration_us":338,"operation_id":"0x30df","timeout_ms":10000,"last_server_duration_us":1425558,"total_duration_us":10009878},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000003/00000000056F6839","last_local_socket":"172.23.104.79:59952","encode_duration_us":350,"operation_id":"0x30e7","timeout_ms":10000,"total_duration_us":10009818},{"operation_name":"insert","last_dispatch_duration_us":12786071,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000003/00000000919EB30B","last_local_socket":"172.23.104.79:41940","total_dispatch_duration_us":12786071,"total_server_duration_us":5854233,"encode_duration_us":340,"operation_id":"0x2ed9","timeout_ms":10000,"last_server_duration_us":5854233,"total_duration_us":10009753}],"total_count":149}} WARN [2022-01-10 08:40:24,688] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000004/00000000EF8F24DE","last_local_socket":"172.23.104.79:51736","encode_duration_us":264,"operation_id":"0x323e","timeout_ms":10000,"total_duration_us":10010979},{"operation_name":"insert","last_dispatch_duration_us":7639635,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":7639635,"total_server_duration_us":391,"encode_duration_us":384,"operation_id":"0x30e2","timeout_ms":10000,"last_server_duration_us":391,"total_duration_us":10010755},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":308,"operation_id":"0x2f23","timeout_ms":10000,"total_duration_us":10010521},{"operation_name":"insert","last_dispatch_duration_us":17909015,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":17909015,"total_server_duration_us":9007669,"encode_duration_us":308,"operation_id":"0x2fad","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10010292},{"operation_name":"insert","last_dispatch_duration_us":17958592,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":17958592,"total_server_duration_us":9015090,"encode_duration_us":322,"operation_id":"0x300a","timeout_ms":10000,"last_server_duration_us":9015090,"total_duration_us":10010281},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","encode_duration_us":397,"operation_id":"0x31ef","timeout_ms":10000,"total_duration_us":10010159},{"operation_name":"insert","last_dispatch_duration_us":17247901,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":17247901,"total_server_duration_us":8288014,"encode_duration_us":304,"operation_id":"0x2fa1","timeout_ms":10000,"last_server_duration_us":8288014,"total_duration_us":10010141},{"operation_name":"insert","last_dispatch_duration_us":8671433,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":8671433,"total_server_duration_us":1782272,"encode_duration_us":334,"operation_id":"0x311c","timeout_ms":10000,"last_server_duration_us":1782272,"total_duration_us":10010130},{"operation_name":"insert","last_dispatch_duration_us":17961748,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":17961748,"total_server_duration_us":9008729,"encode_duration_us":318,"operation_id":"0x303a","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10010116},{"operation_name":"insert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","encode_duration_us":327,"operation_id":"0x30ee","timeout_ms":10000,"total_duration_us":10009984}],"total_count":164}} WARN [2022-01-10 08:40:24,748] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000006/00000000C173A901","last_local_socket":"172.23.104.79:46230","encode_duration_us":276,"operation_id":"0x2f06","timeout_ms":10000,"total_duration_us":10014434},{"operation_name":"insert","last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000006/0000000067A97380","last_local_socket":"172.23.104.79:51746","encode_duration_us":364,"operation_id":"0x30ba","timeout_ms":10000,"total_duration_us":10010738},{"operation_name":"insert","last_dispatch_duration_us":13955717,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","total_dispatch_duration_us":13955717,"total_server_duration_us":1073947,"encode_duration_us":416,"operation_id":"0x2f74","timeout_ms":10000,"last_server_duration_us":1073947,"total_duration_us":10010644},{"operation_name":"insert","last_dispatch_duration_us":12920723,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","total_dispatch_duration_us":12920723,"total_server_duration_us":9009789,"encode_duration_us":406,"operation_id":"0x3072","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10010571},{"operation_name":"insert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_local_socket":"172.23.104.79:59910","encode_duration_us":307,"operation_id":"0x30ae","timeout_ms":10000,"total_duration_us":10010395},{"operation_name":"insert","last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000006/0000000067A97380","last_local_socket":"172.23.104.79:51746","encode_duration_us":332,"operation_id":"0x30bd","timeout_ms":10000,"total_duration_us":10010258},{"operation_name":"insert","last_dispatch_duration_us":15195248,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","total_dispatch_duration_us":15195248,"total_server_duration_us":2390933,"encode_duration_us":310,"operation_id":"0x307f","timeout_ms":10000,"last_server_duration_us":2390933,"total_duration_us":10010223},{"operation_name":"insert","last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000006/0000000067A97380","last_local_socket":"172.23.104.79:51746","encode_duration_us":263,"operation_id":"0x32c9","timeout_ms":10000,"total_duration_us":10010223},{"operation_name":"insert","last_dispatch_duration_us":13479677,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","total_dispatch_duration_us":13479677,"total_server_duration_us":2152653,"encode_duration_us":265,"operation_id":"0x31cd","timeout_ms":10000,"last_server_duration_us":2152653,"total_duration_us":10010200},{"operation_name":"insert","last_dispatch_duration_us":12243579,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000006/00000000873EBBE0","last_local_socket":"172.23.104.79:41916","total_dispatch_duration_us":12243579,"total_server_duration_us":8414248,"encode_duration_us":320,"operation_id":"0x2f46","timeout_ms":10000,"last_server_duration_us":8414248,"total_duration_us":10010187}],"total_count":133}} WARN [2022-01-10 08:40:24,748] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":15652856,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15652856,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9010849,"encode_duration_us":298,"operation_id":"0x2baf","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10010247},{"operation_name":"insert","last_dispatch_duration_us":17612131,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":17612131,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":9011909,"encode_duration_us":258,"operation_id":"0x2999","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10010468},{"operation_name":"insert","last_dispatch_duration_us":12920723,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12920723,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":9009789,"encode_duration_us":406,"operation_id":"0x3072","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10010571},{"operation_name":"insert","last_dispatch_duration_us":20793524,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":20793524,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":9011909,"encode_duration_us":312,"operation_id":"0x2aa3","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10010670},{"operation_name":"insert","last_dispatch_duration_us":19463582,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":19463582,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":6389226,"encode_duration_us":346,"operation_id":"0x2c36","timeout_ms":10000,"last_server_duration_us":6389226,"total_duration_us":10010808},{"operation_name":"insert","last_dispatch_duration_us":13955717,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":13955717,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":1073947,"encode_duration_us":416,"operation_id":"0x2f74","timeout_ms":10000,"last_server_duration_us":1073947,"total_duration_us":10010644},{"operation_name":"insert","last_dispatch_duration_us":12781876,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12781876,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":9015090,"encode_duration_us":257,"operation_id":"0x2cb4","timeout_ms":10000,"last_server_duration_us":9015090,"total_duration_us":10010759},{"operation_name":"insert","last_dispatch_duration_us":15115958,"last_local_id":"2B99390F00000006/00000000873EBBE0","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15115958,"last_local_socket":"172.23.104.79:41916","total_server_duration_us":9005550,"encode_duration_us":318,"operation_id":"0x2d8e","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10010942},{"operation_name":"insert","last_dispatch_duration_us":15210852,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15210852,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9003430,"encode_duration_us":330,"operation_id":"0x2e96","timeout_ms":10000,"last_server_duration_us":9003430,"total_duration_us":10010898},{"operation_name":"insert","last_dispatch_duration_us":20694988,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":20694988,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":6051516,"encode_duration_us":302,"operation_id":"0x2ac9","timeout_ms":10000,"last_server_duration_us":6051516,"total_duration_us":10010830}],"total_count":126}} WARN [2022-01-10 08:40:24,788] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":8346078,"last_local_id":"2B99390F00000005/0000000013738E6A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":8346078,"last_local_socket":"172.23.104.79:41944","total_server_duration_us":3252555,"encode_duration_us":325,"operation_id":"0x307e","timeout_ms":10000,"last_server_duration_us":3252555,"total_duration_us":10010151},{"operation_name":"insert","last_dispatch_duration_us":12776955,"last_local_id":"2B99390F00000005/0000000013738E6A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":12776955,"last_local_socket":"172.23.104.79:41944","total_server_duration_us":6207179,"encode_duration_us":341,"operation_id":"0x2f45","timeout_ms":10000,"last_server_duration_us":6207179,"total_duration_us":10010174},{"operation_name":"insert","last_dispatch_duration_us":17945054,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17945054,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9012969,"encode_duration_us":257,"operation_id":"0x2733","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10010252},{"operation_name":"insert","last_dispatch_duration_us":14522753,"last_local_id":"2B99390F00000005/0000000013738E6A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14522753,"last_local_socket":"172.23.104.79:41944","total_server_duration_us":5806666,"encode_duration_us":351,"operation_id":"0x30e9","timeout_ms":10000,"last_server_duration_us":5806666,"total_duration_us":10010211},{"operation_name":"insert","last_dispatch_duration_us":8734921,"last_local_id":"2B99390F00000005/0000000013738E6A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":8734921,"last_local_socket":"172.23.104.79:41944","total_server_duration_us":2141134,"encode_duration_us":282,"operation_id":"0x3295","timeout_ms":10000,"last_server_duration_us":2141134,"total_duration_us":10010540},{"operation_name":"insert","last_dispatch_duration_us":17617057,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17617057,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9016150,"encode_duration_us":261,"operation_id":"0x2683","timeout_ms":10000,"last_server_duration_us":9016150,"total_duration_us":10010604},{"operation_name":"insert","last_dispatch_duration_us":11627114,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":11627114,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":2622812,"encode_duration_us":255,"operation_id":"0x2c77","timeout_ms":10000,"last_server_duration_us":2622812,"total_duration_us":10010710},{"operation_name":"insert","last_dispatch_duration_us":16140875,"last_local_id":"2B99390F00000005/0000000013738E6A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16140875,"last_local_socket":"172.23.104.79:41944","total_server_duration_us":9006609,"encode_duration_us":328,"operation_id":"0x2d8d","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":10010249},{"operation_name":"insert","last_dispatch_duration_us":16272902,"last_local_id":"2B99390F00000005/0000000013738E6A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16272902,"last_local_socket":"172.23.104.79:41944","total_server_duration_us":9007669,"encode_duration_us":298,"operation_id":"0x2dc6","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10010241},{"operation_name":"insert","last_dispatch_duration_us":14981953,"last_local_id":"2B99390F00000005/0000000013738E6A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14981953,"last_local_socket":"172.23.104.79:41944","total_server_duration_us":7491788,"encode_duration_us":308,"operation_id":"0x2f05","timeout_ms":10000,"last_server_duration_us":7491788,"total_duration_us":10024410}],"total_count":118}} WARN [2022-01-10 08:40:33,225] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":14587253,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14587253,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":5806666,"encode_duration_us":357,"operation_id":"0x2f47","timeout_ms":10000,"last_server_duration_us":5806666,"total_duration_us":10009691},{"operation_name":"insert","last_dispatch_duration_us":10400696,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":10400696,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":4183696,"encode_duration_us":320,"operation_id":"0x31ce","timeout_ms":10000,"last_server_duration_us":4183696,"total_duration_us":10009815},{"operation_name":"insert","last_dispatch_duration_us":15482222,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":15482222,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":9012969,"encode_duration_us":308,"operation_id":"0x2d4c","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10009972},{"operation_name":"insert","last_dispatch_duration_us":18982911,"last_local_id":"2B99390F00000001/000000005D894E7E","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":18982911,"last_local_socket":"172.23.104.79:51762","total_server_duration_us":9003430,"encode_duration_us":325,"operation_id":"0x2ee3","timeout_ms":10000,"last_server_duration_us":9003430,"total_duration_us":10009821},{"operation_name":"insert","last_dispatch_duration_us":17366621,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17366621,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":9008729,"encode_duration_us":358,"operation_id":"0x2f07","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10009908},{"operation_name":"insert","last_dispatch_duration_us":14839253,"last_local_id":"2B99390F00000001/00000000CB02B145","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":14839253,"last_local_socket":"172.23.104.79:41914","total_server_duration_us":5825145,"encode_duration_us":314,"operation_id":"0x2d8f","timeout_ms":10000,"last_server_duration_us":5825145,"total_duration_us":10010668},{"operation_name":"insert","last_dispatch_duration_us":13232512,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13232512,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":9010849,"encode_duration_us":286,"operation_id":"0x2e35","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10010252},{"operation_name":"insert","last_dispatch_duration_us":13557524,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13557524,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":9015090,"encode_duration_us":316,"operation_id":"0x30af","timeout_ms":10000,"last_server_duration_us":9015090,"total_duration_us":10010096},{"operation_name":"insert","last_dispatch_duration_us":13416670,"last_local_id":"2B99390F00000001/00000000DF3AF7AF","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13416670,"last_local_socket":"172.23.104.79:59920","total_server_duration_us":973922,"encode_duration_us":295,"operation_id":"0x3291","timeout_ms":10000,"last_server_duration_us":973922,"total_duration_us":10010332},{"operation_name":"insert","last_dispatch_duration_us":22243147,"last_local_id":"2B99390F00000001/000000005D894E7E","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":22243147,"last_local_socket":"172.23.104.79:51762","total_server_duration_us":9011909,"encode_duration_us":373,"operation_id":"0x2db1","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10010631}],"total_count":78}} WARN [2022-01-10 08:40:34,664] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":23866407,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":23866407,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":5162346,"encode_duration_us":332,"operation_id":"0x30d3","timeout_ms":10000,"last_server_duration_us":5162346,"total_duration_us":10009944},{"operation_name":"insert","last_dispatch_duration_us":23481191,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":23481191,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":9009789,"encode_duration_us":312,"operation_id":"0x2a3a","timeout_ms":10000,"last_server_duration_us":9009789,"total_duration_us":10010066},{"operation_name":"insert","last_dispatch_duration_us":37411717,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":37411717,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":4875582,"encode_duration_us":295,"operation_id":"0x25ff","timeout_ms":10000,"last_server_duration_us":4875582,"total_duration_us":10010144},{"operation_name":"insert","last_dispatch_duration_us":17755403,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17755403,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":9007669,"encode_duration_us":332,"operation_id":"0x3087","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10010139},{"operation_name":"insert","last_dispatch_duration_us":12279961,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":12279961,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":2536875,"encode_duration_us":303,"operation_id":"0x3242","timeout_ms":10000,"last_server_duration_us":2536875,"total_duration_us":10010124},{"operation_name":"insert","last_dispatch_duration_us":20858325,"last_local_id":"2B99390F00000002/0000000056958F5B","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":20858325,"last_local_socket":"172.23.104.79:51728","total_server_duration_us":1598160,"encode_duration_us":298,"operation_id":"0x323c","timeout_ms":10000,"last_server_duration_us":1598160,"total_duration_us":10010829},{"operation_name":"insert","last_dispatch_duration_us":43537737,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":43537737,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":9017210,"encode_duration_us":281,"operation_id":"0x2255","timeout_ms":10000,"last_server_duration_us":9017210,"total_duration_us":10010150},{"operation_name":"insert","last_dispatch_duration_us":17679413,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17679413,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":8932552,"encode_duration_us":343,"operation_id":"0x30c0","timeout_ms":10000,"last_server_duration_us":8932552,"total_duration_us":10010179},{"operation_name":"insert","last_dispatch_duration_us":18617316,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18617316,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":9006609,"encode_duration_us":327,"operation_id":"0x2e71","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":10010208},{"operation_name":"insert","last_dispatch_duration_us":42616248,"last_local_id":"2B99390F00000002/00000000F4655C1A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":42616248,"last_local_socket":"172.23.104.79:41954","total_server_duration_us":4992914,"encode_duration_us":255,"operation_id":"0x244e","timeout_ms":10000,"last_server_duration_us":4992914,"total_duration_us":10011109}],"total_count":99}} WARN [2022-01-10 08:40:34,728] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":35921162,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":35921162,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":4528363,"encode_duration_us":267,"operation_id":"0x270e","timeout_ms":10000,"last_server_duration_us":4528363,"total_duration_us":10005401},{"operation_name":"insert","last_dispatch_duration_us":26993054,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":26993054,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":9014029,"encode_duration_us":312,"operation_id":"0x2b07","timeout_ms":10000,"last_server_duration_us":9014029,"total_duration_us":10005659},{"operation_name":"insert","last_dispatch_duration_us":23126898,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":23126898,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":5103937,"encode_duration_us":314,"operation_id":"0x2eeb","timeout_ms":10000,"last_server_duration_us":5103937,"total_duration_us":10006812},{"operation_name":"insert","last_dispatch_duration_us":35413867,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":35413867,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":4306842,"encode_duration_us":260,"operation_id":"0x279e","timeout_ms":10000,"last_server_duration_us":4306842,"total_duration_us":10007204},{"operation_name":"insert","last_dispatch_duration_us":33512747,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":33512747,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":4520455,"encode_duration_us":266,"operation_id":"0x2858","timeout_ms":10000,"last_server_duration_us":4520455,"total_duration_us":10007432},{"operation_name":"insert","last_dispatch_duration_us":36362765,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":36362765,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":6970575,"encode_duration_us":319,"operation_id":"0x282d","timeout_ms":10000,"last_server_duration_us":6970575,"total_duration_us":10008262},{"operation_name":"insert","last_dispatch_duration_us":37792790,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":37792790,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":7253332,"encode_duration_us":453,"operation_id":"0x27ed","timeout_ms":10000,"last_server_duration_us":7253332,"total_duration_us":10011913},{"operation_name":"insert","last_dispatch_duration_us":26988036,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":26988036,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":9002371,"encode_duration_us":296,"operation_id":"0x2ccc","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10009642},{"operation_name":"insert","last_dispatch_duration_us":23107405,"last_local_id":"2B99390F00000003/000000001BBE5521","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":23107405,"last_local_socket":"172.23.104.79:46178","total_server_duration_us":5102272,"encode_duration_us":318,"operation_id":"0x2df4","timeout_ms":10000,"last_server_duration_us":5102272,"total_duration_us":10007869},{"operation_name":"insert","last_dispatch_duration_us":36514199,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":36514199,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":8150399,"encode_duration_us":268,"operation_id":"0x2881","timeout_ms":10000,"last_server_duration_us":8150399,"total_duration_us":10007454}],"total_count":32}} WARN [2022-01-10 08:40:34,728] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":4333106,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":4333106,"total_server_duration_us":11,"operation_id":"0x3453","timeout_ms":2000,"last_server_duration_us":11,"total_duration_us":2025006},{"operation_name":"get","last_dispatch_duration_us":4330770,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":4330770,"total_server_duration_us":11,"operation_id":"0x3454","timeout_ms":2000,"last_server_duration_us":11,"total_duration_us":2022176},{"operation_name":"get","last_dispatch_duration_us":4327826,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B99390F00000004/0000000065A879A1","last_local_socket":"172.23.104.79:59932","total_dispatch_duration_us":4327826,"total_server_duration_us":11,"operation_id":"0x3455","timeout_ms":2000,"last_server_duration_us":11,"total_duration_us":2019164},{"operation_name":"get","last_dispatch_duration_us":5951443,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B99390F00000004/00000000EF8F24DE","last_local_socket":"172.23.104.79:51736","total_dispatch_duration_us":5951443,"total_server_duration_us":15,"operation_id":"0x3456","timeout_ms":2000,"last_server_duration_us":15,"total_duration_us":2015435},{"operation_name":"get","last_dispatch_duration_us":8148320,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","total_dispatch_duration_us":8148320,"total_server_duration_us":15,"operation_id":"0x3457","timeout_ms":2000,"last_server_duration_us":15,"total_duration_us":2012226},{"operation_name":"get","last_dispatch_duration_us":8171211,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","total_dispatch_duration_us":8171211,"total_server_duration_us":27,"operation_id":"0x344f","timeout_ms":2000,"last_server_duration_us":27,"total_duration_us":2010784},{"operation_name":"get","last_dispatch_duration_us":2124796,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","total_dispatch_duration_us":2124796,"total_server_duration_us":15,"operation_id":"0x3530","timeout_ms":2000,"last_server_duration_us":15,"total_duration_us":2010719},{"operation_name":"get","last_dispatch_duration_us":6045814,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":6045814,"total_server_duration_us":27,"operation_id":"0x346a","timeout_ms":2000,"last_server_duration_us":27,"total_duration_us":2010647},{"operation_name":"get","last_dispatch_duration_us":5162295,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","total_dispatch_duration_us":5162295,"total_server_duration_us":27,"operation_id":"0x34c2","timeout_ms":2000,"last_server_duration_us":27,"total_duration_us":2010136},{"operation_name":"get","last_dispatch_duration_us":5113157,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B99390F00000004/000000004F1F4C7A","last_local_socket":"172.23.104.79:41918","total_dispatch_duration_us":5113157,"total_server_duration_us":15,"operation_id":"0x34d5","timeout_ms":2000,"last_server_duration_us":15,"total_duration_us":2010117}],"total_count":72}} WARN [2022-01-10 08:40:34,729] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":23089991,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":23089991,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":40057,"encode_duration_us":264,"operation_id":"0x2eda","timeout_ms":10000,"last_server_duration_us":40057,"total_duration_us":10009980},{"operation_name":"insert","last_dispatch_duration_us":23091676,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":23091676,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":49719,"encode_duration_us":327,"operation_id":"0x30ee","timeout_ms":10000,"last_server_duration_us":49719,"total_duration_us":10009984},{"operation_name":"insert","last_dispatch_duration_us":17451654,"last_local_id":"2B99390F00000004/0000000065A879A1","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":17451654,"last_local_socket":"172.23.104.79:59932","total_server_duration_us":9011909,"encode_duration_us":397,"operation_id":"0x31ef","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10010159},{"operation_name":"insert","last_dispatch_duration_us":34518243,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":34518243,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":43618,"encode_duration_us":299,"operation_id":"0x2c05","timeout_ms":10000,"last_server_duration_us":43618,"total_duration_us":10010523},{"operation_name":"insert","last_dispatch_duration_us":37204560,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":37204560,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":4997038,"encode_duration_us":304,"operation_id":"0x2aad","timeout_ms":10000,"last_server_duration_us":4997038,"total_duration_us":10010357},{"operation_name":"insert","last_dispatch_duration_us":35217481,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":35217481,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":37036,"encode_duration_us":303,"operation_id":"0x2bb3","timeout_ms":10000,"last_server_duration_us":37036,"total_duration_us":10010569},{"operation_name":"insert","last_dispatch_duration_us":15510037,"last_local_id":"2B99390F00000004/00000000EF8F24DE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":15510037,"last_local_socket":"172.23.104.79:51736","total_server_duration_us":6602270,"encode_duration_us":264,"operation_id":"0x323e","timeout_ms":10000,"last_server_duration_us":6602270,"total_duration_us":10010979},{"operation_name":"insert","last_dispatch_duration_us":34981150,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":34981150,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":19809,"encode_duration_us":320,"operation_id":"0x2bc2","timeout_ms":10000,"last_server_duration_us":19809,"total_duration_us":10010718},{"operation_name":"insert","last_dispatch_duration_us":34899579,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":34899579,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":4808837,"encode_duration_us":330,"operation_id":"0x289a","timeout_ms":10000,"last_server_duration_us":4808837,"total_duration_us":10010938},{"operation_name":"insert","last_dispatch_duration_us":22834850,"last_local_id":"2B99390F00000004/000000004F1F4C7A","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":22834850,"last_local_socket":"172.23.104.79:41918","total_server_duration_us":58999,"encode_duration_us":308,"operation_id":"0x2f23","timeout_ms":10000,"last_server_duration_us":58999,"total_duration_us":10010521}],"total_count":220}} WARN [2022-01-10 08:40:34,764] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":22403756,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":22403756,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":9017210,"encode_duration_us":303,"operation_id":"0x2d21","timeout_ms":10000,"last_server_duration_us":9017210,"total_duration_us":10010003},{"operation_name":"insert","last_dispatch_duration_us":22425196,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":22425196,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":9005550,"encode_duration_us":339,"operation_id":"0x2e42","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10010035},{"operation_name":"insert","last_dispatch_duration_us":13161960,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":13161960,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":7092675,"encode_duration_us":353,"operation_id":"0x2f26","timeout_ms":10000,"last_server_duration_us":7092675,"total_duration_us":10010118},{"operation_name":"insert","last_dispatch_duration_us":29250856,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":29250856,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":9002371,"encode_duration_us":574,"operation_id":"0x2b1c","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10010322},{"operation_name":"insert","last_dispatch_duration_us":21400419,"last_local_id":"2B99390F00000006/0000000067A97380","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":21400419,"last_local_socket":"172.23.104.79:51746","total_server_duration_us":1779612,"encode_duration_us":332,"operation_id":"0x30bd","timeout_ms":10000,"last_server_duration_us":1779612,"total_duration_us":10010258},{"operation_name":"insert","last_dispatch_duration_us":14768525,"last_local_id":"2B99390F00000006/00000000B3A5B8A6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":14768525,"last_local_socket":"172.23.104.79:59910","total_server_duration_us":9001311,"encode_duration_us":307,"operation_id":"0x30ae","timeout_ms":10000,"last_server_duration_us":9001311,"total_duration_us":10010395},{"operation_name":"insert","last_dispatch_duration_us":23869102,"last_local_id":"2B99390F00000006/0000000067A97380","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":23869102,"last_local_socket":"172.23.104.79:51746","total_server_duration_us":4238154,"encode_duration_us":364,"operation_id":"0x30ba","timeout_ms":10000,"last_server_duration_us":4238154,"total_duration_us":10010738},{"operation_name":"insert","last_dispatch_duration_us":27989380,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":27989380,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":8905112,"encode_duration_us":276,"operation_id":"0x2f06","timeout_ms":10000,"last_server_duration_us":8905112,"total_duration_us":10014434},{"operation_name":"insert","last_dispatch_duration_us":24994418,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":24994418,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":9007669,"encode_duration_us":309,"operation_id":"0x2e90","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10010801},{"operation_name":"insert","last_dispatch_duration_us":27249112,"last_local_id":"2B99390F00000006/00000000C173A901","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":27249112,"last_local_socket":"172.23.104.79:46230","total_server_duration_us":9011909,"encode_duration_us":305,"operation_id":"0x2bfb","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10031976}],"total_count":48}} WARN [2022-01-10 08:40:34,828] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":9703062,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":9703062,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":898664,"encode_duration_us":279,"operation_id":"0x31a8","timeout_ms":10000,"last_server_duration_us":898664,"total_duration_us":10010164},{"operation_name":"insert","last_dispatch_duration_us":16470698,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16470698,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":7763341,"encode_duration_us":293,"operation_id":"0x317e","timeout_ms":10000,"last_server_duration_us":7763341,"total_duration_us":10010312},{"operation_name":"insert","last_dispatch_duration_us":18046577,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":18046577,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9015090,"encode_duration_us":311,"operation_id":"0x2e41","timeout_ms":10000,"last_server_duration_us":9015090,"total_duration_us":10010352},{"operation_name":"insert","last_dispatch_duration_us":13885012,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":13885012,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":5187465,"encode_duration_us":324,"operation_id":"0x3175","timeout_ms":10000,"last_server_duration_us":5187465,"total_duration_us":10010723},{"operation_name":"insert","last_dispatch_duration_us":26662915,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":26662915,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":8781025,"encode_duration_us":279,"operation_id":"0x2900","timeout_ms":10000,"last_server_duration_us":8781025,"total_duration_us":10010688},{"operation_name":"insert","last_dispatch_duration_us":21492991,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":21492991,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9005550,"encode_duration_us":316,"operation_id":"0x2de5","timeout_ms":10000,"last_server_duration_us":9005550,"total_duration_us":10010883},{"operation_name":"insert","last_dispatch_duration_us":18527541,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":18527541,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9008729,"encode_duration_us":308,"operation_id":"0x2bf3","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10076662},{"operation_name":"insert","last_dispatch_duration_us":24443360,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":24443360,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":9015090,"encode_duration_us":311,"operation_id":"0x2cd4","timeout_ms":10000,"last_server_duration_us":9015090,"total_duration_us":10010778},{"operation_name":"insert","last_dispatch_duration_us":16500338,"last_local_id":"2B99390F00000005/000000009E68EEB4","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16500338,"last_local_socket":"172.23.104.79:59936","total_server_duration_us":7764336,"encode_duration_us":334,"operation_id":"0x3102","timeout_ms":10000,"last_server_duration_us":7764336,"total_duration_us":10011013},{"operation_name":"insert","last_dispatch_duration_us":17959150,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17959150,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9011909,"encode_duration_us":311,"operation_id":"0x2bfa","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10035532}],"total_count":83}} WARN [2022-01-10 08:40:43,254] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":24169770,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":24169770,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":9001311,"encode_duration_us":354,"operation_id":"0x2f4d","timeout_ms":10000,"last_server_duration_us":9001311,"total_duration_us":10005315},{"operation_name":"insert","last_dispatch_duration_us":24183216,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":24183216,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":9016150,"encode_duration_us":278,"operation_id":"0x2e70","timeout_ms":10000,"last_server_duration_us":9016150,"total_duration_us":10007881},{"operation_name":"insert","last_dispatch_duration_us":24182830,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":24182830,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":9011909,"encode_duration_us":370,"operation_id":"0x3055","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10010768},{"operation_name":"insert","last_dispatch_duration_us":24083691,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":24083691,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":8913551,"encode_duration_us":374,"operation_id":"0x2e91","timeout_ms":10000,"last_server_duration_us":8913551,"total_duration_us":10009310},{"operation_name":"insert","last_dispatch_duration_us":21061936,"last_local_id":"2B99390F00000001/000000005A213BC7","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":28283690,"last_local_socket":"172.23.104.79:46170","total_server_duration_us":4107879,"encode_duration_us":273,"operation_id":"0x3265","timeout_ms":10000,"last_server_duration_us":4107503,"total_duration_us":10008805}],"total_count":5}} WARN [2022-01-10 08:40:44,656] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":31715894,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":31715894,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":55303,"encode_duration_us":569,"operation_id":"0x2fca","timeout_ms":10000,"last_server_duration_us":55303,"total_duration_us":10009353},{"operation_name":"insert","last_dispatch_duration_us":40556416,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":40556416,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":6400220,"encode_duration_us":314,"operation_id":"0x2a2a","timeout_ms":10000,"last_server_duration_us":6400220,"total_duration_us":10009356},{"operation_name":"insert","last_dispatch_duration_us":42339996,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":42339996,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":7309486,"encode_duration_us":259,"operation_id":"0x294c","timeout_ms":10000,"last_server_duration_us":7309486,"total_duration_us":10010179},{"operation_name":"insert","last_dispatch_duration_us":25628250,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25628250,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":20362,"encode_duration_us":434,"operation_id":"0x32be","timeout_ms":10000,"last_server_duration_us":20362,"total_duration_us":10009426},{"operation_name":"insert","last_dispatch_duration_us":25273150,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25273150,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":16848,"encode_duration_us":350,"operation_id":"0x30e7","timeout_ms":10000,"last_server_duration_us":16848,"total_duration_us":10009818},{"operation_name":"insert","last_dispatch_duration_us":32039879,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":32039879,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":32840,"encode_duration_us":263,"operation_id":"0x2f6a","timeout_ms":10000,"last_server_duration_us":32840,"total_duration_us":10010454},{"operation_name":"insert","last_dispatch_duration_us":41867133,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":41867133,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":7083104,"encode_duration_us":279,"operation_id":"0x299f","timeout_ms":10000,"last_server_duration_us":7083104,"total_duration_us":10010278},{"operation_name":"insert","last_dispatch_duration_us":25406616,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":25406616,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":48449,"encode_duration_us":270,"operation_id":"0x31b8","timeout_ms":10000,"last_server_duration_us":48449,"total_duration_us":10010314},{"operation_name":"insert","last_dispatch_duration_us":33317488,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":33317488,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":80113,"encode_duration_us":363,"operation_id":"0x2e8d","timeout_ms":10000,"last_server_duration_us":80113,"total_duration_us":10009612},{"operation_name":"insert","last_dispatch_duration_us":38592481,"last_local_id":"2B99390F00000003/00000000056F6839","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":38592481,"last_local_socket":"172.23.104.79:59952","total_server_duration_us":53494,"encode_duration_us":256,"operation_id":"0x2bf7","timeout_ms":10000,"last_server_duration_us":53494,"total_duration_us":10050911}],"total_count":90}} WARN [2022-01-10 08:40:44,676] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":41258713,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":41258713,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":56401,"encode_duration_us":341,"operation_id":"0x2ac2","timeout_ms":10000,"last_server_duration_us":56401,"total_duration_us":10009961},{"operation_name":"insert","last_dispatch_duration_us":42376111,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":42376111,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":234781,"encode_duration_us":297,"operation_id":"0x299e","timeout_ms":10000,"last_server_duration_us":234781,"total_duration_us":10010013},{"operation_name":"insert","last_dispatch_duration_us":46904779,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":46904779,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":3907737,"encode_duration_us":260,"operation_id":"0x2707","timeout_ms":10000,"last_server_duration_us":3907737,"total_duration_us":10010199},{"operation_name":"insert","last_dispatch_duration_us":45939724,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":45939724,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":3158305,"encode_duration_us":254,"operation_id":"0x2770","timeout_ms":10000,"last_server_duration_us":3158305,"total_duration_us":10010023},{"operation_name":"insert","last_dispatch_duration_us":37431686,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":37431686,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":55061,"encode_duration_us":324,"operation_id":"0x2ca4","timeout_ms":10000,"last_server_duration_us":55061,"total_duration_us":10010339},{"operation_name":"insert","last_dispatch_duration_us":41606306,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":41606306,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":63310,"encode_duration_us":341,"operation_id":"0x2a60","timeout_ms":10000,"last_server_duration_us":63310,"total_duration_us":10010404},{"operation_name":"insert","last_dispatch_duration_us":42281572,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":42281572,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":24755,"encode_duration_us":261,"operation_id":"0x2977","timeout_ms":10000,"last_server_duration_us":24755,"total_duration_us":10010475},{"operation_name":"insert","last_dispatch_duration_us":19991523,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":19991523,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":164724,"encode_duration_us":285,"operation_id":"0x31c9","timeout_ms":10000,"last_server_duration_us":164724,"total_duration_us":10010641},{"operation_name":"insert","last_dispatch_duration_us":42258240,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":42258240,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":223440,"encode_duration_us":305,"operation_id":"0x29bd","timeout_ms":10000,"last_server_duration_us":223440,"total_duration_us":10011044},{"operation_name":"insert","last_dispatch_duration_us":38630869,"last_local_id":"2B99390F00000002/00000000118F2C63","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":38630869,"last_local_socket":"172.23.104.79:59942","total_server_duration_us":29137,"encode_duration_us":311,"operation_id":"0x2bf6","timeout_ms":10000,"last_server_duration_us":29137,"total_duration_us":10056559}],"total_count":109}} WARN [2022-01-10 08:40:44,755] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":1180748,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":1180748,"total_server_duration_us":1179439,"operation_id":"0x3b18","timeout_ms":2000,"last_server_duration_us":1179439,"total_duration_us":1180990},{"operation_name":"get","last_dispatch_duration_us":1180515,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":1180515,"total_server_duration_us":1178993,"operation_id":"0x3b22","timeout_ms":2000,"last_server_duration_us":1178993,"total_duration_us":1180674},{"operation_name":"get","last_dispatch_duration_us":1080457,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":1080457,"total_server_duration_us":1078670,"operation_id":"0x3acb","timeout_ms":2000,"last_server_duration_us":1078670,"total_duration_us":1080879},{"operation_name":"get","last_dispatch_duration_us":1039583,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":1039583,"total_server_duration_us":1038170,"operation_id":"0x359a","timeout_ms":2000,"last_server_duration_us":1038170,"total_duration_us":1039792},{"operation_name":"get","last_dispatch_duration_us":1038996,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":1038996,"total_server_duration_us":1037747,"operation_id":"0x35a5","timeout_ms":2000,"last_server_duration_us":1037747,"total_duration_us":1039191},{"operation_name":"get","last_dispatch_duration_us":1026223,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":1026223,"total_server_duration_us":1023835,"operation_id":"0x3596","timeout_ms":2000,"last_server_duration_us":1023835,"total_duration_us":1026407},{"operation_name":"get","last_dispatch_duration_us":826035,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":826035,"total_server_duration_us":824845,"operation_id":"0x3b14","timeout_ms":2000,"last_server_duration_us":824845,"total_duration_us":826286},{"operation_name":"get","last_dispatch_duration_us":825709,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":825709,"total_server_duration_us":824079,"operation_id":"0x3b26","timeout_ms":2000,"last_server_duration_us":824079,"total_duration_us":825858},{"operation_name":"get","last_dispatch_duration_us":783998,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":783998,"total_server_duration_us":783126,"operation_id":"0x3ae9","timeout_ms":2000,"last_server_duration_us":783126,"total_duration_us":784329},{"operation_name":"get","last_dispatch_duration_us":783571,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":783571,"total_server_duration_us":782376,"operation_id":"0x3aeb","timeout_ms":2000,"last_server_duration_us":782376,"total_duration_us":783920}],"total_count":28}} WARN [2022-01-10 08:40:44,777] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":26496166,"last_local_id":"2B99390F00000006/0000000067A97380","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":26496166,"last_local_socket":"172.23.104.79:51746","total_server_duration_us":7625569,"encode_duration_us":307,"operation_id":"0x30f8","timeout_ms":10000,"last_server_duration_us":7625569,"total_duration_us":10007538},{"operation_name":"insert","last_dispatch_duration_us":28268672,"last_local_id":"2B99390F00000006/0000000067A97380","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":28268672,"last_local_socket":"172.23.104.79:51746","total_server_duration_us":4372100,"encode_duration_us":270,"operation_id":"0x3289","timeout_ms":10000,"last_server_duration_us":4372100,"total_duration_us":10007780},{"operation_name":"insert","last_dispatch_duration_us":25958948,"last_local_id":"2B99390F00000006/0000000067A97380","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":25958948,"last_local_socket":"172.23.104.79:51746","total_server_duration_us":1278923,"encode_duration_us":263,"operation_id":"0x31d3","timeout_ms":10000,"last_server_duration_us":1278923,"total_duration_us":10007808},{"operation_name":"insert","last_dispatch_duration_us":28920666,"last_local_id":"2B99390F00000006/0000000067A97380","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":28920666,"last_local_socket":"172.23.104.79:51746","total_server_duration_us":3170531,"encode_duration_us":264,"operation_id":"0x314c","timeout_ms":10000,"last_server_duration_us":3170531,"total_duration_us":10008364},{"operation_name":"insert","last_dispatch_duration_us":24473841,"last_local_id":"2B99390F00000006/0000000067A97380","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":24473841,"last_local_socket":"172.23.104.79:51746","total_server_duration_us":1065382,"encode_duration_us":300,"operation_id":"0x32ae","timeout_ms":10000,"last_server_duration_us":1065382,"total_duration_us":10010024},{"operation_name":"insert","last_dispatch_duration_us":26044193,"last_local_id":"2B99390F00000006/0000000067A97380","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":26044193,"last_local_socket":"172.23.104.79:51746","total_server_duration_us":65123,"encode_duration_us":261,"operation_id":"0x3138","timeout_ms":10000,"last_server_duration_us":65123,"total_duration_us":10008955},{"operation_name":"insert","last_dispatch_duration_us":27386884,"last_local_id":"2B99390F00000006/0000000067A97380","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":27386884,"last_local_socket":"172.23.104.79:51746","total_server_duration_us":2412680,"encode_duration_us":267,"operation_id":"0x319d","timeout_ms":10000,"last_server_duration_us":2412680,"total_duration_us":10008826},{"operation_name":"insert","last_dispatch_duration_us":24251138,"last_local_id":"2B99390F00000006/0000000067A97380","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":24251138,"last_local_socket":"172.23.104.79:51746","total_server_duration_us":63568,"encode_duration_us":335,"operation_id":"0x3252","timeout_ms":10000,"last_server_duration_us":63568,"total_duration_us":10009912},{"operation_name":"insert","last_dispatch_duration_us":23637210,"last_local_id":"2B99390F00000006/0000000067A97380","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":23637210,"last_local_socket":"172.23.104.79:51746","total_server_duration_us":576644,"encode_duration_us":313,"operation_id":"0x32ca","timeout_ms":10000,"last_server_duration_us":576644,"total_duration_us":10008574},{"operation_name":"insert","last_dispatch_duration_us":28010263,"last_local_id":"2B99390F00000006/0000000067A97380","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":28010263,"last_local_socket":"172.23.104.79:51746","total_server_duration_us":4949289,"encode_duration_us":263,"operation_id":"0x32c9","timeout_ms":10000,"last_server_duration_us":4949289,"total_duration_us":10010223}],"total_count":23}} WARN [2022-01-10 08:40:44,855] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":17833933,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17833933,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9011909,"encode_duration_us":320,"operation_id":"0x313d","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10006355},{"operation_name":"insert","last_dispatch_duration_us":17570996,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17570996,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9011909,"encode_duration_us":299,"operation_id":"0x31d8","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10006574},{"operation_name":"insert","last_dispatch_duration_us":17815496,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17815496,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9004490,"encode_duration_us":268,"operation_id":"0x3263","timeout_ms":10000,"last_server_duration_us":9004490,"total_duration_us":10007234},{"operation_name":"insert","last_dispatch_duration_us":17912405,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17912405,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9003430,"encode_duration_us":259,"operation_id":"0x327b","timeout_ms":10000,"last_server_duration_us":9003430,"total_duration_us":10006799},{"operation_name":"insert","last_dispatch_duration_us":17581175,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17581175,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9012969,"encode_duration_us":283,"operation_id":"0x3151","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10008287},{"operation_name":"insert","last_dispatch_duration_us":17657914,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17657914,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9011909,"encode_duration_us":298,"operation_id":"0x31c6","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":10010422},{"operation_name":"insert","last_dispatch_duration_us":17706916,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17706916,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9010849,"encode_duration_us":282,"operation_id":"0x322d","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10008501},{"operation_name":"insert","last_dispatch_duration_us":17966834,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17966834,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9002371,"encode_duration_us":262,"operation_id":"0x3269","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":10009546},{"operation_name":"insert","last_dispatch_duration_us":26339782,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":26339782,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":8725532,"encode_duration_us":310,"operation_id":"0x303b","timeout_ms":10000,"last_server_duration_us":8725532,"total_duration_us":10008303},{"operation_name":"insert","last_dispatch_duration_us":17823170,"last_local_id":"2B99390F00000005/0000000086155695","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17823170,"last_local_socket":"172.23.104.79:46186","total_server_duration_us":9001311,"encode_duration_us":322,"operation_id":"0x32a1","timeout_ms":10000,"last_server_duration_us":9001311,"total_duration_us":10010420}],"total_count":21}} WARN [2022-01-10 08:40:54,689] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"insert","last_dispatch_duration_us":25587687,"last_local_id":"2B99390F00000002/00000000ECD3FB8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":25587687,"last_local_socket":"172.23.104.79:46216","total_server_duration_us":6913651,"encode_duration_us":266,"operation_id":"0x3254","timeout_ms":10000,"last_server_duration_us":6913651,"total_duration_us":10004477}],"total_count":1}} WARN [2022-01-10 08:40:54,780] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":1261360,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":1261360,"total_server_duration_us":1259579,"operation_id":"0x3b5e","timeout_ms":2000,"last_server_duration_us":1259579,"total_duration_us":1261490},{"operation_name":"get","last_dispatch_duration_us":1261201,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":1261201,"total_server_duration_us":1259579,"operation_id":"0x3b60","timeout_ms":2000,"last_server_duration_us":1259579,"total_duration_us":1261295},{"operation_name":"get","last_dispatch_duration_us":1260789,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":1260789,"total_server_duration_us":1258661,"operation_id":"0x3b69","timeout_ms":2000,"last_server_duration_us":1258661,"total_duration_us":1260996},{"operation_name":"get","last_dispatch_duration_us":1260519,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B99390F00000004/000000001581771D","last_local_socket":"172.23.104.79:46242","total_dispatch_duration_us":1260519,"total_server_duration_us":1258661,"operation_id":"0x3b70","timeout_ms":2000,"last_server_duration_us":1258661,"total_duration_us":1260625}],"total_count":4}} INFO [2022-01-10 08:41:03,326] jepsen worker 817 - jepsen.util 5817 :ok :read (0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 2880 2881 2882 2883 2884 2885 2886 2887 2888 2889 2890 2891 2892 2893 2894 2895 2896 2897 2898 2899 2900 2901 2902 2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2925 2926 2927 2928 2929 2930 2931 2932 2933 2934 2935 2936 2937 2938 2939 2940 2941 2942 2943 2944 2945 2946 2947 2948 2949 2950 2951 2952 2953 2954 2955 2956 2957 2958 2959 2960 2961 2962 2963 2964 2965 2966 2967 2968 2969 2970 2971 2972 2973 2974 2975 2976 2977 2978 2979 2980 2981 2982 2983 2984 2985 2986 2987 2988 2989 2990 2991 2992 2993 2994 2995 2996 2997 2998 2999 3000 3001 3002 3003 3004 3005 3006 3007 3008 3009 3010 3011 3012 3013 3014 3015 3016 3017 3018 3019 3020 3021 3022 3023 3024 3025 3026 3027 3028 3029 3030 3031 3032 3033 3034 3035 3036 3037 3038 3039 3040 3041 3042 3043 3044 3045 3046 3047 3048 3049 3050 3051 3052 3053 3054 3055 3056 3057 3058 3059 3060 3061 3062 3063 3064 3065 3066 3067 3068 3069 3070 3071 3072 3073 3074 3075 3076 3077 3078 3079 3080 3081 3082 3083 3084 3085 3086 3087 3088 3089 3090 3091 3092 3093 3094 3095 3096 3097 3098 3099 3100 3101 3102 3103 3104 3105 3106 3107 3108 3109 3110 3111 3112 3113 3114 3115 3116 3117 3118 3119 3120 3121 3122 3123 3124 3125 3126 3127 3128 3129 3130 3131 3132 3133 3134 3135 3136 3137 3138 3139 3140 3141 3142 3143 3144 3145 3146 3147 3148 3149 3150 3151 3152 3153 3154 3155 3156 3157 3158 3159 3160 3161 3162 3163 3164 3165 3166 3167 3168 3169 3170 3171 3172 3173 3174 3175 3176 3177 3178 3179 3180 3181 3182 3183 3184 3185 3186 3187 3188 3189 3190 3191 3192 3193 3194 3195 3196 3197 3198 3199 3200 3201 3202 3203 3204 3205 3206 3207 3208 3209 3210 3211 3212 3213 3214 3215 3216 3217 3218 3219 3220 3221 3222 3223 3224 3225 3226 3227 3228 3229 3230 3231 3232 3233 3234 3235 3236 3237 3238 3239 3240 3241 3242 3243 3244 3245 3246 3247 3248 3249 3250 3251 3252 3253 3254 3255 3256 3257 3258 3259 3260 3261 3262 3263 3264 3265 3266 3267 3268 3269 3270 3271 3272 3273 3274 3275 3276 3277 3278 3279 3280 3281 3282 3283 3284 3285 3286 3287 3288 3289 3290 3291 3292 3293 3294 3295 3296 3297 3298 3299 3300 3301 3302 3303 3304 3305 3306 3307 3308 3309 3310 3311 3312 3313 3314 3315 3316 3317 3318 3319 3320 3321 3322 3323 3324 3325 3326 3327 3328 3329 3330 3331 3332 3333 3334 3335 3336 3337 3338 3339 3340 3341 3342 3343 3344 3345 3346 3347 3348 3349 3350 3351 3352 3353 3354 3355 3356 3357 3358 3359 3360 3361 3362 3363 3364 3365 3366 3367 3368 3369 3370 3371 3372 3373 3374 3375 3376 3377 3378 3379 3380 3381 3382 3383 3384 3385 3386 3387 3388 3389 3390 3391 3392 3393 3394 3395 3396 3397 3398 3399 3400 3401 3402 3403 3404 3405 3406 3407 3408 3409 3410 3411 3412 3413 3414 3415 3416 3417 3418 3419 3420 3421 3422 3423 3424 3425 3426 3427 3428 3429 3430 3431 3432 3433 3434 3435 3436 3437 3438 3439 3440 3441 3442 3443 3444 3445 3446 3447 3448 3449 3450 3451 3452 3453 3454 3455 3456 3457 3458 3459 3460 3461 3462 3463 3464 3465 3466 3467 3468 3469 3470 3471 3472 3473 3474 3475 3476 3477 3478 3479 3480 3481 3482 3483 3484 3485 3486 3487 3488 3489 3490 3491 3492 3493 3494 3495 3496 3497 3498 3499 3500 3501 3502 3503 3504 3505 3506 3507 3508 3509 3510 3511 3512 3513 3514 3515 3516 3517 3518 3519 3520 3521 3522 3523 3524 3525 3526 3527 3528 3529 3530 3531 3532 3533 3534 3535 3536 3537 3538 3539 3540 3541 3542 3543 3544 3545 3546 3547 3548 3549 3550 3551 3552 3553 3554 3555 3556 3557 3558 3559 3560 3561 3562 3563 3564 3565 3566 3567 3568 3569 3570 3571 3572 3573 3574 3575 3576 3577 3578 3579 3580 3581 3582 3583 3584 3585 3586 3587 3588 3589 3590 3591 3592 3593 3594 3595 3596 3597 3598 3599 3600 3601 3602 3603 3604 3605 3606 3607 3608 3609 3610 3611 3612 3613 3614 3615 3616 3617 3618 3619 3620 3621 3622 3623 3624 3625 3626 3627 3628 3629 3630 3631 3632 3633 3634 3635 3636 3637 3638 3639 3640 3641 3642 3643 3644 3645 3646 3647 3648 3649 3650 3651 3652 3653 3654 3655 3656 3657 3658 3659 3660 3661 3662 3663 3664 3665 3666 3667 3668 3669 3670 3671 3672 3673 3674 3675 3676 3677 3678 3679 3680 3681 3682 3683 3684 3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 3696 3697 3698 3699 3700 3701 3702 3703 3704 3705 3706 3707 3708 3709 3710 3711 3712 3713 3714 3715 3716 3717 3718 3719 3720 3721 3722 3723 3724 3725 3726 3727 3728 3729 3730 3731 3732 3733 3734 3735 3736 3737 3738 3739 3740 3741 3742 3743 3744 3745 3746 3747 3748 3749 3750 3751 3752 3753 3754 3755 3756 3757 3758 3759 3760 3761 3762 3763 3764 3765 3766 3767 3768 3769 3770 3771 3772 3773 3774 3775 3776 3777 3778 3779 3780 3781 3782 3783 3784 3785 3786 3787 3788 3789 3790 3791 3792 3793 3794 3795 3796 3797 3798 3799 3800 3801 3802 3803 3804 3805 3806 3807 3808 3809 3810 3811 3812 3813 3814 3815 3816 3817 3818 3819 3820 3821 3822 3823 3824 3825 3826 3827 3828 3829 3830 3831 3832 3833 3834 3835 3836 3837 3838 3839 3840 3841 3842 3843 3844 3845 3846 3847 3848 3849 3850 3851 3852 3853 3854 3855 3856 3857 3858 3859 3860 3861 3862 3863 3864 3865 3866 3867 3868 3869 3870 3871 3872 3873 3874 3875 3876 3877 3878 3879 3880 3881 3882 3883 3884 3885 3886 3887 3888 3889 3890 3891 3892 3893 3894 3895 3896 3897 3898 3899 3900 3901 3902 3903 3904 3905 3906 3907 3908 3909 3910 3911 3912 3913 3914 3915 3916 3917 3918 3919 3920 3921 3922 3923 3924 3925 3926 3927 3928 3929 3930 3931 3932 3933 3934 3935 3936 3937 3938 3939 3940 3941 3942 3943 3944 3945 3946 3947 3948 3949 3950 3951 3952 3953 3954 3955 3956 3957 3958 3959 3960 3961 3962 3963 3964 3965 3966 3967 3968 3969 3970 3971 3972 3973 3974 3975 3976 3977 3978 3979 3980 3981 3982 3983 3984 3985 3986 3987 3988 3989 3990 3991 3992 3993 3994 3995 3996 3997 3998 3999 4000 4001 4002 4003 4004 4005 4006 4007 4008 4009 4010 4011 4012 4013 4014 4015 4016 4017 4018 4019 4020 4021 4022 4023 4024 4025 4026 4027 4028 4029 4030 4031 4032 4033 4034 4035 4036 4037 4038 4039 4040 4041 4042 4043 4044 4045 4046 4047 4048 4049 4050 4051 4052 4053 4054 4055 4056 4057 4058 4059 4060 4061 4062 4063 4064 4065 4066 4067 4068 4069 4070 4071 4072 4073 4074 4075 4076 4077 4078 4079 4080 4081 4082 4083 4084 4085 4086 4087 4088 4089 4090 4091 4092 4093 4094 4095 4096 4097 4098 4099 4100 4101 4102 4103 4104 4105 4106 4107 4108 4109 4110 4111 4112 4113 4114 4115 4116 4117 4118 4119 4120 4121 4122 4123 4124 4125 4126 4127 4128 4129 4130 4131 4132 4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 4157 4158 4159 4160 4161 4162 4163 4164 4165 4166 4167 4168 4169 4170 4171 4172 4173 4174 4175 4176 4177 4178 4179 4180 4181 4182 4183 4184 4185 4186 4187 4188 4189 4190 4191 4192 4193 4194 4195 4196 4197 4198 4199 4200 4201 4202 4203 4204 4205 4206 4207 4208 4209 4210 4211 4212 4213 4214 4215 4216 4217 4218 4219 4220 4221 4222 4223 4224 4225 4226 4227 4228 4229 4230 4231 4232 4233 4234 4235 4236 4237 4238 4239 4240 4241 4242 4243 4244 4245 4246 4247 4248 4249 4250 4251 4252 4253 4254 4255 4256 4257 4258 4259 4260 4261 4262 4263 4264 4265 4266 4267 4268 4269 4270 4271 4272 4273 4274 4275 4276 4277 4278 4279 4280 4281 4282 4283 4284 4285 4286 4287 4288 4289 4290 4291 4292 4293 4294 4295 4296 4297 4298 4299 4300 4301 4302 4303 4304 4305 4306 4307 4308 4309 4310 4311 4312 4313 4314 4315 4316 4317 4318 4319 4320 4321 4322 4323 4324 4325 4326 4327 4328 4329 4330 4331 4332 4333 4334 4335 4336 4337 4338 4339 4340 4341 4342 4343 4344 4345 4346 4347 4348 4349 4350 4351 4352 4353 4354 4355 4356 4357 4358 4359 4360 4361 4362 4363 4364 4365 4366 4367 4368 4369 4370 4371 4372 4373 4374 4375 4376 4377 4378 4379 4380 4381 4382 4383 4384 4385 4386 4387 4388 4389 4390 4391 4392 4393 4394 4395 4396 4397 4398 4399 4400 4401 4402 4403 4404 4405 4406 4407 4408 4409 4410 4411 4412 4413 4414 4415 4416 4417 4418 4419 4420 4421 4422 4423 4424 4425 4426 4427 4428 4429 4430 4431 4432 4433 4434 4435 4436 4437 4438 4439 4440 4441 4442 4443 4444 4445 4446 4447 4448 4449 4450 4451 4452 4453 4454 4455 4456 4457 4458 4459 4460 4461 4462 4463 4464 4465 4466 4467 4468 4469 4470 4471 4472 4473 4474 4475 4476 4477 4478 4479 4480 4481 4482 4483 4484 4485 4486 4487 4488 4489 4490 4491 4492 4493 4494 4495 4496 4497 4498 4499 4500 4501 4502 4503 4504 4505 4506 4507 4508 4509 4510 4511 4512 4513 4514 4515 4516 4517 4518 4519 4520 4521 4522 4523 4524 4525 4526 4527 4528 4529 4530 4531 4532 4533 4534 4535 4536 4537 4538 4539 4540 4541 4542 4543 4544 4545 4546 4547 4548 4549 4550 4551 4552 4553 4554 4555 4556 4557 4558 4559 4560 4561 4562 4563 4564 4565 4566 4567 4568 4569 4570 4571 4572 4573 4574 4575 4576 4577 4578 4579 4580 4581 4582 4583 4584 4585 4586 4587 4588 4589 4590 4591 4592 4593 4594 4595 4596 4597 4598 4599 4600 4601 4602 4603 4604 4605 4606 4607 4608 4609 4610 4611 4612 4613 4614 4615 4616 4617 4618 4619 4620 4621 4622 4623 4624 4625 4626 4627 4628 4629 4630 4631 4632 4633 4634 4635 4636 4637 4638 4639 4640 4641 4642 4643 4644 4645 4646 4647 4648 4649 4650 4651 4652 4653 4654 4655 4656 4657 4658 4659 4660 4661 4662 4663 4664 4665 4666 4667 4668 4669 4670 4671 4672 4673 4674 4675 4676 4677 4678 4679 4680 4681 4682 4683 4684 4685 4686 4687 4688 4689 4690 4691 4692 4693 4694 4695 4696 4697 4698 4699 4700 4701 4702 4703 4704 4705 4706 4707 4708 4709 4710 4711 4712 4713 4714 4715 4716 4717 4718 4719 4720 4721 4722 4723 4724 4725 4726 4727 4728 4729 4730 4731 4732 4733 4734 4735 4736 4737 4738 4739 4740 4741 4742 4743 4744 4745 4746 4747 4748 4749 4750 4751 4752 4753 4754 4755 4756 4757 4758 4759 4760 4761 4762 4763 4764 4765 4766 4767 4768 4769 4770 4771 4772 4773 4775 4776 4777 4778 4779 4780 4781 4782 4783 4784 4785 4786 4787 4788 4789 4790 4791 4792 4793 4794 4796 4797 4798 4799 4800 4801 4802 4803 4804 4805 4806 4807 4808 4809 4810 4811 4812 4813 4814 4816 4817 4818 4819 4821 4822 4823 4824 4825 4826 4827 4828 4829 4830 4831 4832 4833 4834 4835 4836 4837 4840 4841 4842 4843 4844 4845 4846 4847 4848 4849 4850 4852 4853 4854 4855 4856 4857 4858 4859 4860 4861 4862 4863 4864 4865 4866 4867 4868 4869 4870 4871 4872 4873 4874 4875 4876 4877 4878 4879 4880 4881 4882 4883 4885 4887 4888 4889 4890 4891 4892 4893 4894 4895 4896 4897 4898 4899 4900 4901 4902 4903 4904 4905 4906 4907 4908 4909 4910 4911 4912 4913 4915 4917 4918 4919 4921 4923 4924 4925 4926 4927 4928 4929 4930 4931 4932 4933 4934 4935 4936 4937 4939 4940 4941 4942 4943 4944 4945 4946 4947 4949 4951 4952 4954 4955 4956 4957 4958 4959 4960 4961 4962 4963 4965 4966 4967 4968 4969 4970 4971 4972 4973 4974 4975 4976 4977 4978 4979 4980 4981 4982 4983 4984 4986 4987 4988 4989 4990 4991 4992 4993 4994 4995 4996 4997 4998 4999 5000 5001 5002 5003 5004 5005 5006 5007 5011 5013 5014 5015 5016 5017 5018 5019 5020 5021 5022 5023 5027 5028 5029 5030 5031 5032 5033 5034 5035 5036 5037 5038 5039 5040 5041 5042 5043 5044 5045 5046 5047 5048 5049 5050 5051 5052 5053 5055 5056 5058 5059 5061 5062 5064 5065 5066 5067 5068 5069 5070 5071 5072 5073 5074 5075 5076 5077 5079 5082 5083 5084 5085 5086 5087 5088 5089 5090 5091 5092 5093 5094 5095 5096 5097 5100 5101 5102 5103 5104 5105 5106 5107 5112 5113 5114 5115 5116 5117 5118 5119 5120 5121 5122 5123 5124 5126 5127 5128 5129 5130 5131 5132 5133 5134 5135 5136 5137 5138 5139 5140 5141 5142 5143 5144 5145 5146 5147 5148 5149 5150 5151 5152 5153 5154 5155 5156 5158 5159 5160 5161 5162 5164 5165 5166 5167 5168 5169 5170 5171 5172 5173 5174 5175 5176 5177 5178 5179 5183 5184 5185 5186 5187 5188 5189 5190 5191 5192 5193 5194 5195 5196 5197 5200 5201 5202 5203 5204 5205 5206 5207 5209 5210 5211 5212 5213 5214 5215 5216 5217 5218 5219 5220 5221 5222 5223 5224 5225 5226 5227 5228 5229 5230 5231 5232 5233 5234 5235 5236 5237 5238 5239 5240 5241 5242 5243 5244 5245 5246 5247 5248 5249 5250 5251 5252 5253 5255 5256 5257 5258 5259 5260 5262 5263 5264 5265 5266 5267 5268 5269 5270 5271 5272 5273 5274 5275 5276 5277 5280 5282 5283 5284 5285 5286 5287 5288 5289 5290 5291 5292 5293 5294 5295 5296 5297 5298 5299 5300 5301 5302 5303 5304 5305 5306 5307 5308 5309 5310 5313 5314 5315 5316 5317 5318 5319 5320 5321 5322 5323 5324 5326 5327 5328 5329 5330 5331 5332 5333 5334 5335 5336 5337 5338 5339 5340 5341 5342 5343 5344 5345 5346 5347 5348 5349 5350 5351 5352 5353 5356 5358 5359 5360 5362 5364 5365 5366 5367 5368 5369 5370 5371 5372 5373 5374 5375 5376 5377 5381 5382 5383 5384 5385 5386 5387 5388 5389 5390 5391 5392 5393 5394 5395 5396 5397 5398 5399 5400 5401 5402 5403 5404 5405 5406 5407 5408 5410 5414 5415 5416 5417 5418 5419 5420 5421 5422 5423 5428 5429 5430 5431 5432 5433 5434 5435 5436 5437 5438 5439 5440 5441 5442 5443 5444 5445 5446 5447 5448 5449 5450 5451 5452 5453 5454 5455 5456 5457 5458 5459 5461 5462 5464 5465 5466 5468 5469 5470 5471 5472 5473 5474 5475 5476 5477 5478 5479 5480 5481 5482 5484 5485 5486 5487 5488 5489 5490 5491 5492 5493 5494 5495 5496 5497 5498 5499 5500 5501 5502 5503 5504 5505 5506 5507 5508 5510 5514 5515 5516 5517 5518 5519 5520 5521 5522 5523 5524 5525 5528 5529 5530 5531 5532 5533 5534 5535 5536 5537 5538 5539 5540 5541 5542 5543 5544 5545 5546 5547 5548 5549 5550 5551 5552 5553 5555 5557 5558 5559 5560 5561 5563 5564 5565 5566 5567 5568 5569 5570 5571 5572 5573 5574 5575 5576 5577 5579 5580 5581 5582 5583 5584 5585 5586 5587 5588 5589 5590 5591 5592 5593 5594 5595 5596 5597 5598 5600 5601 5602 5603 5604 5605 5606 5607 5608 5609 5610 5614 5615 5616 5617 5618 5619 5620 5621 5622 5623 5624 5625 5626 5628 5629 5630 5631 5632 5633 5634 5635 5636 5637 5638 5639 5640 5641 5642 5643 5644 5645 5646 5647 5648 5649 5650 5651 5652 5653 5655 5657 5658 5659 5662 5663 5664 5665 5666 5667 5668 5669 5670 5671 5672 5673 5674 5675 5676 5677 5678 5679 5682 5683 5684 5685 5686 5687 5688 5689 5690 5691 5692 5693 5694 5695 5696 5697 5698 5700 5701 5702 5703 5704 5705 5706 5707 5708 5709 5710 5711 5713 5714 5715 5716 5717 5718 5719 5720 5721 5722 5723 5724 5725 5726 5728 5729 5730 5731 5732 5733 5734 5735 5736 5737 5738 5739 5740 5741 5742 5743 5744 5745 5746 5747 5748 5749 5750 5751 5752 5753 5755 5757 5758 5759 5760 5761 5764 5765 5766 5767 5768 5769 5770 5771 5772 5773 5774 5775 5776 5777 5778 5779 5780 5783 5784 5785 5786 5787 5788 5789 5790 5791 5792 5793 5794 5795 5796 5797 5798 5799 5800 5801 5802 5804 5805 5806 5808 5809 5810 5811 5812 5813 5814 5815 5816 5817 5820 5821 5822 5823 5824 5825 5826 5827 5828 5829 5830 5831 5832 5833 5835 5836 5838 5839 5840 5841 5842 5843 5845 5848 5849 5850 5851 5852 5853 5854 5855 5856 5857 5858 5859 5860 5861 5862 5863 5864 5865 5866 5867 5870 5871 5874 5875 5876 5877 5878 5879 5880 5881 5882 5883 5884 5885 5886 5887 5888 5889 5890 5891 5893 5894 5895 5896 5897 5898 5899 5900 5901 5902 5903 5904 5905 5906 5907 5908 5909 5910 5911 5912 5913 5914 5915 5916 5917 5919 5920 5921 5922 5923 5924 5925 5926 5927 5928 5929 5930 5931 5932 5933 5935 5936 5937 5938 5939 5940 5941 5942 5943 5944 5945 5948 5949 5950 5951 5952 5953 5954 5955 5956 5957 5958 5959 5960 5961 5962 5963 5964 5965 5966 5967 5970 5972 5974 5976 5977 5978 5979 5980 5981 5982 5983 5984 5985 5986 5987 5989 5994 5995 5996 5997 5998 5999 6000 6001 6002 6003 6004 6005 6006 6007 6008 6009 6010 6011 6012 6013 6014 6015 6016 6017 6018 6019 6020 6021 6022 6023 6024 6025 6026 6027 6028 6031 6032 6033 6034 6035 6036 6037 6038 6039 6040 6042 6043 6044 6045 6046 6047 6048 6049 6050 6051 6052 6053 6054 6055 6056 6057 6060 6061 6062 6063 6064 6065 6066 6067 6068 6069 6070 6071 6072 6073 6076 6078 6079 6080 6081 6082 6083 6084 6085 6086 6087 6088 6089 6091 6092 6093 6094 6095 6098 6099 6100 6101 6102 6103 6106 6108 6109 6110 6111 6112 6113 6114 6115 6116 6117 6118 6119 6120 6121 6122 6123 6124 6125 6126 6127 6133 6134 6135 6136 6137 6138 6139 6140 6141 6143 6144 6145 6146 6147 6148 6149 6150 6151 6152 6153 6154 6155 6156 6157 6158 6159 6160 6161 6162 6163 6164 6165 6166 6167 6168 6169 6170 6171 6172 6173 6174 6175 6177 6178 6179 6180 6181 6182 6183 6184 6185 6186 6187 6188 6189 6190 6191 6192 6193 6194 6195 6196 6198 6199 6200 6201 6202 6203 6204 6207 6208 6209 6210 6211 6212 6213 6214 6215 6216 6217 6218 6219 6220 6221 6222 6223 6224 6225 6226 6227 6228 6230 6231 6233 6234 6235 6236 6237 6238 6239 6244 6245 6246 6247 6248 6249 6250 6251 6252 6253 6254 6255 6256 6257 6260 6261 6262 6263 6264 6265 6266 6267 6268 6269 6270 6271 6272 6273 6278 6279 6280 6281 6282 6283 6284 6285 6286 6287 6288 6289 6290 6291 6292 6293 6296 6297 6298 6299 6300 6301 6302 6303 6308 6309 6310 6311 6312 6313 6314 6315 6316 6317 6318 6319 6320 6321 6322 6323 6324 6326 6327 6330 6333 6334 6335 6336 6337 6338 6339 6340 6341 6342 6343 6344 6345 6346 6347 6348 6349 6350 6351 6352 6353 6354 6355 6356 6357 6358 6359 6360 6361 6362 6363 6364 6365 6366 6367 6368 6369 6370 6371 6372 6373 6374 6376 6377 6378 6379 6380 6381 6382 6383 6384 6385 6386 6387 6388 6389 6390 6391 6392 6393 6395 6397 6398 6399 6400 6401 6402 6403 6404 6405 6406 6407 6408 6409 6410 6411 6412 6413 6414 6418 6419 6420 6424 6425 6426 6427 6428 6429 6430 6431 6432 6433 6434 6435 6436 6437 6441 6442 6444 6445 6446 6447 6448 6450 6452 6454 6455 6456 6457 6458 6460 6462 6464 6466 6468 6469 6470 6471 6472 6473 6476 6478 6479 6480 6481 6482 6483 6485 6486 6487 6488 6489 6490 6491 6492 6493 6496 6497 6498 6499 6500 6501 6502 6503 6506 6508 6510 6511 6512 6513 6515 6516 6517 6518 6519 6521 6522 6523 6524 6525 6526 6527 6528 6530 6531 6533 6534 6535 6536 6537 6539 6541 6543 6544 6545 6546 6547 6552 6554 6556 6557 6559 6562 6563 6569 6571 6572 6573 6574 6575 6576 6578 6580 6581 6582 6586 6588 6589 6590 6591 6592 6593 6596 6598 6599 6600 6601 6602 6603 6605 6607 6609 6611 6612 6614 6615 6617 6618 6619 6620 6621 6622 6624 6631 6633 6634 6635 6636 6637 6638 6639 6642 6643 6644 6645 6646 6647 6648 6649 6650 6651 6652 6653 6656 6658 6662 6664 6665 6666 6667 6668 6669 6670 6671 6672 6673 6674 6675 6676 6677 6680 6681 6683 6684 6685 6686 6687 6688 6689 6690 6691 6692 6693 6696 6697 6699 6700 6701 6702 6703 6704 6705 6709 6710 6713 6714 6715 6716 6717 6718 6719 6720 6721 6722 6723 6725 6727 6728 6729 6732 6733 6734 6735 6736 6737 6738 6739 6740 6743 6744 6745 6746 6747 6748 6749 6750 6751 6752 6753 6754 6755 6756 6757 6759 6760 6761 6762 6763 6764 6765 6766 6767 6768 6769 6770 6771 6772 6773 6774 6776 6777 6778 6779 6780 6784 6785 6786 6787 6788 6789 6790 6791 6792 6793 6800 6802 6804 6805 6806 6807 6808 6809 6810 6811 6812 6813 6814 6821 6822 6823 6824 6825 6826 6827 6828 6829 6830 6831 6832 6833 6834 6836 6838 6839 6840 6841 6842 6843 6844 6845 6846 6847 6849 6850 6851 6854 6855 6856 6857 6858 6859 6860 6861 6862 6863 6864 6865 6867 6868 6869 6870 6871 6872 6873 6874 6875 6876 6877 6878 6879 6880 6881 6882 6883 6884 6891 6894 6895 6896 6897 6898 6899 6904 6905 6906 6907 6908 6909 6910 6911 6912 6913 6915 6916 6920 6921 6922 6924 6925 6926 6927 6928 6929 6930 6931 6932 6933 6940 6941 6942 6943 6945 6947 6948 6949 6950 6951 6954 6955 6956 6957 6958 6959 6960 6961 6962 6963 6964 6965 6967 6973 6974 6975 6976 6977 6978 6979 6980 6981 6982 6983 6984 6986 6992 6994 6995 6996 6997 6999 7000 7001 7002 7003 7005 7010 7014 7015 7016 7017 7018 7019 7020 7021 7022 7023 7029 7034 7035 7036 7037 7038 7039 7043 7044 7045 7046 7047 7048 7049 7050 7051 7052 7053 7054 7055 7061 7064 7065 7066 7067 7068 7069 7070 7071 7072 7073 7077 7078 7080 7084 7085 7086 7087 7088 7089 7090 7091 7092 7093 7096 7099 7100 7101 7102 7103 7105 7106 7108 7109 7110 7111 7114 7115 7116 7117 7118 7119 7120 7121 7122 7123 7124 7127 7128 7129 7131 7132 7134 7135 7136 7137 7138 7139 7140 7141 7142 7143 7144 7145 7146 7147 7148 7149 7150 7151 7153 7154 7157 7158 7159 7161 7163 7164 7165 7166 7167 7168 7169 7170 7171 7172 7173 7174 7175 7176 7179 7181 7184 7185 7186 7187 7188 7189 7190 7191 7192 7193 7195 7199 7200 7201 7202 7203 7204 7205 7206 7208 7209 7211 7212 7214 7215 7216 7217 7218 7219 7220 7221 7222 7223 7224 7225 7226 7227 7229 7230 7232 7233 7234 7235 7236 7237 7238 7239 7241 7243 7244 7245 7246 7247 7248 7250 7251 7252 7256 7257 7258 7264 7265 7266 7267 7268 7269 7270 7271 7272 7273 7278 7279 7280 7283 7284 7285 7286 7287 7288 7289 7290 7291 7292 7293 7294 7298 7300 7301 7302 7303 7306 7309 7310 7313 7314 7315 7316 7317 7318 7319 7320 7321 7322 7323 7324 7327 7332 7334 7335 7336 7337 7338 7339 7342 7343 7344 7345 7346 7347 7348 7349 7350 7351 7352 7353 7354 7356 7357 7361 7362 7363 7364 7365 7366 7367 7368 7369 7370 7371 7372 7373 7376 7380 7381 7383 7384 7385 7386 7387 7388 7389 7390 7391 7392 7393 7395 7396 7398 7399 7400 7401 7402 7403 7405 7409 7413 7414 7415 7417 7418 7419 7420 7421 7423 7425 7434 7435 7436 7437 7438 7439 7444 7445 7446 7447 7448 7449 7453 7454 7460 7462 7464 7465 7468 7469 7470 7471 7472 7473 7477 7481 7483 7484 7485 7486 7487 7488 7489 7490 7491 7492 7493 7498 7500 7501 7502 7503 7508 7510 7511 7512 7514 7515 7516 7517 7518 7519 7520 7521 7522 7523 7525 7526 7527 7532 7534 7535 7536 7537 7538 7539 7540 7543 7544 7545 7546 7547 7548 7549 7550 7551 7552 7556 7558 7559 7560 7562 7565 7566 7567 7568 7569 7570 7571 7572 7573 7575 7580 7583 7587 7588 7589 7590 7591 7592 7593 7594 7596 7600 7601 7602 7603 7605 7606 7608 7610 7613 7614 7615 7616 7617 7618 7619 7620 7621 7622 7625 7626 7627 7629 7630 7631 7632 7633 7634 7635 7636 7637 7638 7640 7642 7644 7645 7646 7647 7650 7651 7652 7654 7657 7658 7660 7662 7663 7666 7668 7669 7670 7671 7672 7673 7674 7678 7683 7685 7688 7689 7690 7691 7692 7693 7696 7698 7700 7701 7702 7703 7705 7711 7712 7714 7715 7716 7718 7719 7722 7725 7726 7727 7732 7733 7734 7735 7736 7737 7738 7743 7744 7745 7746 7747 7752 7754 7756 7760 7761 7762 7764 7768 7769 7770 7771 7772 7773 7774 7775 7777 7778 7780 7781 7782 7783 7784 7788 7789 7790 7791 7792 7793 7795 7797 7798 7799 7800 7803 7804 7808 7809 7810 7811 7812 7813 7816 7817 7818 7819 7823 7824 7825 7826 7827 7831 7833 7834 7838 7839 7841 7842 7847 7848 7850 7851 7852 7853 7854 7855 7856 7857 7860 7861 7862 7863 7865 7866 7870 7873 7875 7876 7877 7878 7879 7880 7881 7882 7883 7886 7887 7888 7889 7891 7893 7896 7897 7898 7899 7900 7901 7902 7903 7907 7908 7909 7910 7911 7912 7913 7915 7917 7918 7922 7924 7925 7926 7927 7934 7936 7943 7944 7946 7947 7950 7954 7955 7956 7957 7959 7960 7961 7962 7963 7964 7967 7969 7972 7975 7976 7977 7978 7979 7980 7981 7982 7983 7987 7988 7990 7993 7996 7998 7999 8001 8004 8005 8006 8007 8008 8011 8018 8019 8022 8024 8026 8027 8028 8029 8030 8031 8032 8033 8035 8036 8039 8040 8041 8042 8043 8044 8051 8053 8054 8056 8058 8059 8062 8063 8064 8067 8069 8070 8074 8075 8076 8077 8082 8083 8085 8087 8088 8090 8091 8094 8095 8096 8097 8101 8102 8103 8104 8105 8106 8107 8108 8111 8112 8113 8114 8115 8117 8118 8120 8121 8123 8125 8126 8127 8128 8129 8130 8131 8132 8133 8136 8137 8140 8141 8142 8143 8146 8147 8148 8151 8152 8153 8154 8157 8158 8159 8160 8164 8167 8168 8172 8174 8175 8176 8177 8181 8185 8186 8187 8188 8189 8191 8192 8194 8195 8196 8197 8198 8199 8204 8205 8206 8207 8211 8213 8214 8215 8216 8220 8221 8222 8225 8228 8229 8230 8231 8232 8233 8235 8236 8237 8238 8239 8240 8241 8242 8243 8247 8250 8251 8254 8255 8257 8258 8259 8260 8261 8263 8265 8266 8269 8271 8274 8275 8276 8277 8279 8280 8286 8289 8290 8293 8294 8295 8296 8297 8300 8303 8304 8305 8306 8307 8316 8319 8320 8322 8323 8324 8327 8328 8329 8330 8331 8332 8333 8334 8335 8336 8338 8340 8341 8342 8343 8344 8345 8349 8351 8352 8358 8359 8360 8367 8370 8373 8374 8375 8376 8377 8378 8383 8385 8386 8387 8389 8391 8392 8393 8394 8395 8396 8397 8401 8402 8403 8404 8405 8406 8407 8410 8413 8418 8425 8427 8428 8429 8430 8431 8432 8433 8435 8436 8438 8439 8440 8441 8442 8443 8444 8445 8446 8448 8452 8457 8458 8459 8463 8464 8466 8467 8468 8471 8472 8474 8475 8476 8477 8483 8484 8485 8489 8490 8492 8493 8494 8495 8496 8497 8501 8502 8503 8504 8505 8506 8507 8513 8514 8515 8516 8517 8519 8521 8523 8527 8528 8529 8530 8531 8532 8533 8535 8537 8539 8540 8541 8542 8543 8548 8551 8552 8553 8558 8559 8564 8566 8568 8570 8574 8575 8576 8577 8580 8589 8591 8592 8593 8594 8595 8596 8597 8598 8604 8605 8606 8607 8616 8620 8623 8628 8629 8630 8631 8632 8633 8635 8640 8641 8642 8643 8644 8648 8651 8652 8653 8656 8658 8659 8662 8665 8666 8667 8671 8672 8674 8675 8676 8677 8681 8685 8687 8689 8690 8694 8695 8696 8697 8698 8700 8703 8704 8705 8706 8707 8709 8710 8712 8715 8717 8718 8720 8721 8723 8728 8729 8730 8731 8732 8733 8734 8736 8739 8740 8741 8742 8743 8747 8749 8752 8754 8755 8758 8759 8764 8766 8770 8773 8774 8775 8776 8777 8779 8781 8788 8794 8795 8796 8797 8804 8805 8806 8807 8809 8810 8812 8814 8815 8816 8817 8820 8821 8822 8823 8824 8825 8826 8828 8831 8834 8837 8838 8839 8840 8848 8849 8850 8851 8852 8853 8854 8857 8858 8860 8863 8864 8865 8866 8867 8872 8874 8876 8877 8880 8881 8882 8883 8884 8885 8886 8887 8894 8895 8896 8899 8905 8906 8907 8908 8909 8910 8914 8915 8916 8917 8919 8920 8921 8922 8923 8925 8928 8930 8931 8938 8939 8940 8941 8943 8944 8948 8949 8950 8951 8952 8953 8954 8956 8957 8958 8959 8960 8962 8963 8964 8965 8966 8967 8971 8975 8976 8977 8978 8983 8984 8985 8986 8987 8990 8991 8994 8995 8996 8998 9000 9001 9002 9005 9006 9008 9009 9010 9011 9012 9013 9014 9015 9016 9017 9018 9019 9020 9021 9022 9024 9025 9026 9027 9031 9033 9034 9035 9038 9039 9044 9045 9047 9048 9050 9052 9054 9055 9056 9057 9059 9060 9061 9062 9063 9064 9065 9066 9067 9071 9072 9073 9078 9079 9080 9081 9082 9083 9084 9086 9087 9088 9089 9091 9092 9098 9099 9100 9101 9103 9104 9108 9109 9110 9111 9112 9113 9114 9118 9120 9124 9125 9126 9127 9128 9130 9133 9134 9135 9136 9137 9138 9142 9144 9145 9146 9147 9148 9149 9150 9151 9154 9155 9156 9157 9160 9161 9162 9163 9164 9165 9168 9169 9170 9171 9172 9173 9178 9179 9180 9181 9182 9183 9185 9187 9190 9198 9199 9202 9207 9208 9209 9210 9211 9212 9213 9214 9215 9217 9223 9224 9225 9226 9227 9229 9232 9234 9235 9236 9237 9239 9241 9242 9244 9245 9249 9250 9251 9253 9254 9255 9256 9257 9259 9260 9261 9262 9263 9265 9268 9270 9271 9273 9275 9276 9278 9279 9280 9281 9282 9283 9285 9287 9290 9291 9292 9298 9299 9300 9303 9308 9309 9310 9311 9312 9313 9315 9317 9324 9325 9326 9327 9334 9335 9336 9344 9345 9346 9347 9349 9352 9353 9354 9355 9356 9357 9360 9361 9362 9363 9364 9366 9367 9368 9369 9370 9371 9372 9373 9374 9378 9379 9380 9381 9382 9383 9384 9385 9386 9389 9391 9392 9393 9398 9399 9401 9403 9408 9409 9410 9411 9412 9413 9415 9416 9420 9421 9422 9424 9425 9426 9427 9435 9436 9438 9439 9443 9444 9446 9450 9454 9455 9456 9457 9458 9460 9461 9462 9463 9464 9466 9468 9470 9471 9473 9478 9479 9480 9481 9482 9483 9485 9487 9488 9489 9490 9491 9492 9493 9495 9498 9499 9500 9501 9502 9504 9505 9508 9509 9510 9511 9512 9513 9515 9517 9518 9519 9521 9523 9524 9525 9526 9527 9529 9530 9534 9535 9537 9539 9544 9545 9546 9547 9549 9552 9553 9554 9555 9556 9557 9560 9561 9562 9563 9564 9565 9567 9568 9569 9573 9578 9579 9580 9581 9582 9583 9584 9585 9586 9588 9589 9590 9591 9598 9599 9603 9608 9609 9610 9611 9612 9613 9617 9618 9622 9623 9624 9625 9626 9627 9634 9635 9636 9637 9638 9640 9641 9642 9644 9645 9646 9648 9650 9652 9653 9654 9655 9656 9657 9658 9660 9661 9662 9663 9664 9670 9671 9672 9673 9678 9679 9680 9681 9682 9683 9686 9687 9688 9690 9691 9692 9693 9698 9699 9702 9703 9706 9708 9709 9710 9711 9712 9713 9714 9717 9718 9719 9720 9724 9725 9726 9727 9728 9730 9731 9737 9740 9744 9745 9746 9747 9748 9749 9751 9752 9753 9754 9755 9756 9757 9760 9761 9762 9763 9764 9766 9767 9769 9771 9772 9773 9774 9776 9778 9779 9780 9781 9782 9783 9785 9788 9789 9790 9792 9793 9798 9799 9800 9801 9802 9803 9804 9806 9808 9810 9811 9813 9815 9818 9819 9821 9825 9826 9827 9830 9831 9834 9835 9836 9837 9840 9841 9842 9843 9844 9845 9846 9847 9849 9854 9855 9857 9860 9862 9863 9868 9869 9870 9871 9872 9873 9874 9878 9879 9880 9881 9882 9883 9886 9888 9889 9890 9891 9892 9893 9894 9897 9898 9899 9900 9901 9902 9903 9905 9906 9908 9910 9911 9912 9913 9918 9919 9925 9926 9928 9929 9933 9934 9935 9936 9937 9940 9941 9942 9944 9945 9946 9947 9948 9951 9955 9956 9957 9958 9959 9961 9962 9963 9965 9967 9968 9969 9970 9971 9972 9973 9976 9977 9979 9981 9982 9983 9985 9987 9988 9989 9990 9991 9992 9993 9997 9998 10001 10007 10008 10009 10010 10014 10015 10016 10017 10020 10021 10022 10023 10024 10027 10028 10029 10030 10031 10037 10038 10039 10042 10043 10047 10048 10050 10051 10052 10053 10055 10057 10058 10060 10063 10064 10065 10066 10067 10068 10074 10075 10079 10081 10082 10083 10084 10085 10086 10087 10091 10092 10093 10095 10096 10097 10098 10099 10104 10105 10106 10107 10108 10109 10110 10112 10113 10115 10116 10117 10121 10122 10123 10125 10126 10127 10128 10129 10130 10131 10135 10136 10139 10140 10141 10142 10143 10148 10149 10150 10151 10152 10154 10155 10156 10159 10162 10163 10164 10165 10166 10167 10174 10176 10177 10180 10182 10183 10184 10185 10186 10187 10193 10194 10195 10196 10197 10198 10199 10201 10204 10207 10209 10210 10211 10212 10213 10214 10215 10217 10220 10221 10222 10223 10225 10226 10227 10228 10229 10230 10231 10233 10236 10238 10239 10240 10241 10242 10243 10248 10249 10250 10252 10253 10255 10256 10257 10258 10259 10261 10262 10263 10264 10266 10267 10273 10277 10280 10281 10282 10283 10284 10285 10286 10287 10288 10291 10295 10296 10297 10298 10299 10300 10303 10307 10310 10311 10312 10313 10314 10315 10316 10317 10320 10321 10322 10323 10326 10329 10331 10333 10338 10339 10340 10342 10346 10348 10349 10350 10351 10352 10353 10354 10356 10358 10359 10360 10361 10362 10364 10365 10366 10367 10374 10376 10378 10379 10381 10382 10383 10384 10385 10387 10392 10394 10397 10398 10399 10400 10404 10406 10409 10410 10411 10412 10413 10414 10415 10416 10420 10421 10422 10424 10425 10427 10428 10430 10433 10434 10438 10439 10442 10448 10449 10450 10451 10452 10453 10454 10455 10457 10460 10461 10463 10464 10465 10466 10467 10475 10478 10481 10482 10483 10484 10485 10486 10487 10497 10505 10508 10509 10511 10512 10513 10514 10515 10516 10517 10520 10521 10522 10523 10525 10526 10527 10529 10530 10532 10533 10538 10539 10541 10542 10549 10550 10551 10552 10553 10554 10555 10556 10557 10559 10560 10561 10562 10563 10565 10566 10567 10570 10573 10574 10575 10577 10579 10580 10583 10584 10586 10587 10594 10596 10598 10599 10604 10607 10610 10611 10612 10614 10615 10616 10617 10619 10620 10621 10622 10623 10625 10626 10627 10629 10631 10632 10633 10638 10639 10640 10643 10649 10650 10651 10653 10655 10656 10657 10661 10662 10663 10664 10665 10666 10667 10670 10676 10677 10678 10679 10681 10682 10684 10685 10686 10687 10689 10693 10694 10696 10697 10698 10702 10705 10706 10707 10708 10709 10710 10712 10713 10714 10715 10716 10717 10720 10721 10722 10723 10724 10725 10727 10729 10730 10733 10736 10738 10739 10740 10741 10742 10743 10749 10751 10752 10753 10755 10757 10758 10759 10760 10762 10763 10764 10765 10766 10767 10775 10776 10777 10778 10779 10780 10781 10783 10784 10785 10786 10787 10794 10795 10796 10798 10799 10801 10802 10803 10804 10805 10806 10807 10809 10814 10815 10819 10820 10821 10823 10828 10829 10831 10832 10833 10834 10835 10837 10838 10839 10840 10841 10842 10843 10844 10845 10846 10847 10848 10849 10850 10851 10852 10853 10854 10857 10858 10859 10860 10863 10864 10866 10867 10868 10869 10870 10871 10873 10874 10876 10877 10878 10884 10885 10887 10888 10889 10891 10892 10893 10894 10895 10896 10897 10900 10901 10903 10905 10906 10907 10908 10914 10915 10918 10919 10920 10921 10929 10930 10931 10932 10933 10934 10935 10936 10937 10938 10939 10940 10941 10942 10943 10945 10946 10947 10949 10951 10952 10958 10964 10966 10967 10968 10970 10971 10972 10973 10974 10975 10976 10977 10981 10984 10987 10988 10989 10990 10991 10992 10994 10995 10996 10997 10999 11000 11002 11003 11004 11006 11007 11008 11009 11011 11012 11016 11017 11018 11023 11024 11025 11026 11027 11028 11030 11031 11032 11033 11034 11035 11036 11037 11040 11042 11043 11044 11045 11046 11047 11051 11054 11055 11056 11057 11058 11059 11060 11061 11062 11068 11070 11071 11072 11073 11074 11075 11076 11077 11078 11079 11081 11082 11083 11088 11089 11090 11091 11092 11093 11094 11095 11097 11098 11099 11100 11101 11102 11103 11104 11105 11106 11107 11108 11109 11110 11111 11112 11113 11116 11118 11119) INFO [2022-01-10 08:41:03,395] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][5023us] Closed bucket "default" {"coreId":"0x2b99390f00000005"} INFO [2022-01-10 08:41:03,395] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][1087us] Node disconnected {"coreId":"0x2b99390f00000005","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 08:41:03,395] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][145us] Node disconnected {"coreId":"0x2b99390f00000005","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 08:41:03,395] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][210us] Node disconnected {"coreId":"0x2b99390f00000005","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 08:41:03,495] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][845us] Closed bucket "default" {"coreId":"0x2b99390f00000003"} INFO [2022-01-10 08:41:03,495] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][195us] Node disconnected {"coreId":"0x2b99390f00000003","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 08:41:03,495] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][129us] Node disconnected {"coreId":"0x2b99390f00000005","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 08:41:03,495] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][72ms] Completed shutdown and closed all open buckets {"coreId":"0x2b99390f00000005"} INFO [2022-01-10 08:41:03,496] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][741us] Closed bucket "default" {"coreId":"0x2b99390f00000001"} INFO [2022-01-10 08:41:03,496] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][130us] Node disconnected {"coreId":"0x2b99390f00000001","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 08:41:03,496] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][212us] Node disconnected {"coreId":"0x2b99390f00000001","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 08:41:03,496] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][167us] Node disconnected {"coreId":"0x2b99390f00000001","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 08:41:03,496] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][167us] Node disconnected {"coreId":"0x2b99390f00000001","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 08:41:03,497] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][26ms] Completed shutdown and closed all open buckets {"coreId":"0x2b99390f00000001"} INFO [2022-01-10 08:41:03,501] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][993us] Closed bucket "default" {"coreId":"0x2b99390f00000002"} INFO [2022-01-10 08:41:03,501] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][226us] Node disconnected {"coreId":"0x2b99390f00000002","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 08:41:03,501] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][153us] Node disconnected {"coreId":"0x2b99390f00000002","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 08:41:03,502] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][200us] Node disconnected {"coreId":"0x2b99390f00000002","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 08:41:03,502] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][246us] Node disconnected {"coreId":"0x2b99390f00000002","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 08:41:03,502] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][26ms] Completed shutdown and closed all open buckets {"coreId":"0x2b99390f00000002"} INFO [2022-01-10 08:41:03,502] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][811us] Closed bucket "default" {"coreId":"0x2b99390f00000006"} INFO [2022-01-10 08:41:03,502] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][142us] Node disconnected {"coreId":"0x2b99390f00000006","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 08:41:03,502] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][132us] Node disconnected {"coreId":"0x2b99390f00000006","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 08:41:03,502] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][131us] Node disconnected {"coreId":"0x2b99390f00000006","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 08:41:03,502] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][164us] Node disconnected {"coreId":"0x2b99390f00000006","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 08:41:03,502] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][21ms] Completed shutdown and closed all open buckets {"coreId":"0x2b99390f00000006"} INFO [2022-01-10 08:41:03,605] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][789us] Closed bucket "default" {"coreId":"0x2b99390f00000004"} INFO [2022-01-10 08:41:03,605] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][126us] Node disconnected {"coreId":"0x2b99390f00000004","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 08:41:03,605] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][125us] Node disconnected {"coreId":"0x2b99390f00000004","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 08:41:03,606] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][121us] Node disconnected {"coreId":"0x2b99390f00000004","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 08:41:03,606] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][181us] Node disconnected {"coreId":"0x2b99390f00000004","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 08:41:03,606] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][25ms] Completed shutdown and closed all open buckets {"coreId":"0x2b99390f00000004"} INFO [2022-01-10 08:41:03,606] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][140us] Node disconnected {"coreId":"0x2b99390f00000003","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 08:41:03,606] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][153us] Node disconnected {"coreId":"0x2b99390f00000003","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 08:41:03,607] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][132us] Node disconnected {"coreId":"0x2b99390f00000003","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 08:41:03,607] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][23ms] Completed shutdown and closed all open buckets {"coreId":"0x2b99390f00000003"} INFO [2022-01-10 08:41:03,889] jepsen test runner - jepsen.core Run complete, writing INFO [2022-01-10 08:41:03,895] jepsen fressian - couchbase.core Not writing fressian INFO [2022-01-10 08:41:04,604] jepsen test runner - jepsen.core Snarfing log files INFO [2022-01-10 08:41:04,913] jepsen node 172.23.106.61 - couchbase.util Running cbcollect_info on 172.23.106.61 INFO [2022-01-10 08:41:04,914] jepsen node 172.23.105.187 - couchbase.util Running cbcollect_info on 172.23.105.187 INFO [2022-01-10 08:41:04,914] jepsen node 172.23.107.228 - couchbase.util Running cbcollect_info on 172.23.107.228 INFO [2022-01-10 08:41:04,914] jepsen node 172.23.107.227 - couchbase.util Running cbcollect_info on 172.23.107.227 INFO [2022-01-10 08:46:13,209] jepsen node 172.23.106.61 - couchbase.util Creating hashtable dump on 172.23.106.61 INFO [2022-01-10 08:46:14,609] jepsen node 172.23.105.187 - couchbase.util Creating hashtable dump on 172.23.105.187 INFO [2022-01-10 08:46:15,512] jepsen node 172.23.107.227 - couchbase.util Creating hashtable dump on 172.23.107.227 INFO [2022-01-10 08:46:15,609] jepsen node 172.23.107.228 - couchbase.util Creating hashtable dump on 172.23.107.228 INFO [2022-01-10 08:46:25,827] jepsen node 172.23.106.61 - couchbase.util Collecting tcp packet capture INFO [2022-01-10 08:46:27,025] jepsen node 172.23.105.187 - couchbase.util Collecting tcp packet capture INFO [2022-01-10 08:46:27,329] jepsen node 172.23.107.227 - couchbase.util Collecting tcp packet capture INFO [2022-01-10 08:46:27,825] jepsen node 172.23.107.228 - couchbase.util Collecting tcp packet capture INFO [2022-01-10 08:47:00,080] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/goxdcr_cprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/goxdcr_cprof.log INFO [2022-01-10 08:47:00,382] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_access.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_access.log INFO [2022-01-10 08:47:01,878] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_info.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_info.log INFO [2022-01-10 08:47:03,630] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.fts.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.fts.log INFO [2022-01-10 08:47:03,687] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_warn.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_warn.log INFO [2022-01-10 08:47:05,124] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.couchdb.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.couchdb.log INFO [2022-01-10 08:47:06,493] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/goxdcr_remote_clusters.json to cbcollect_info_ns_1@172.23.106.61_20220110-164105/goxdcr_remote_clusters.json INFO [2022-01-10 08:47:06,713] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/stats.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/stats.log INFO [2022-01-10 08:47:07,833] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.indexer.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.indexer.log INFO [2022-01-10 08:47:08,237] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/memcached.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/memcached.log INFO [2022-01-10 08:47:09,234] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_access.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_access.log INFO [2022-01-10 08:47:09,858] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/users.dets to cbcollect_info_ns_1@172.23.107.227_20220110-164105/users.dets INFO [2022-01-10 08:47:10,389] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.indexer.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.indexer.log INFO [2022-01-10 08:47:10,568] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/projector_pprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/projector_pprof.log INFO [2022-01-10 08:47:11,208] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/stats_snapshot/01FS2E422MMV4QPB3VQ2V69XT9/tombstones to cbcollect_info_ns_1@172.23.107.227_20220110-164105/stats_snapshot/01FS2E422MMV4QPB3VQ2V69XT9/tombstones INFO [2022-01-10 08:47:11,674] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/stats_snapshot/01FS2E41CS7ZW6T7BNHJYKRYNX/meta.json to cbcollect_info_ns_1@172.23.105.187_20220110-164105/stats_snapshot/01FS2E41CS7ZW6T7BNHJYKRYNX/meta.json INFO [2022-01-10 08:47:11,742] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_shutdown.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_shutdown.log INFO [2022-01-10 08:47:11,935] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/event_log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/event_log INFO [2022-01-10 08:47:12,549] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.http_access.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.http_access.log INFO [2022-01-10 08:47:13,040] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.mapreduce_errors.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.mapreduce_errors.log INFO [2022-01-10 08:47:13,069] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.debug.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.debug.log INFO [2022-01-10 08:47:13,311] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/projector_mprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/projector_mprof.log INFO [2022-01-10 08:47:13,859] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.indexer.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.indexer.log INFO [2022-01-10 08:47:14,379] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/mdocs.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/mdocs.log INFO [2022-01-10 08:47:14,509] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.backup_service.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.backup_service.log INFO [2022-01-10 08:47:14,679] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.backup_service.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.backup_service.log INFO [2022-01-10 08:47:15,200] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ddocs.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ddocs.log INFO [2022-01-10 08:47:15,704] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/couchstore_local.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/couchstore_local.log INFO [2022-01-10 08:47:15,816] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/couchbase.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/couchbase.log INFO [2022-01-10 08:47:16,016] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.query.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.query.log INFO [2022-01-10 08:47:16,542] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/mdocs.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/mdocs.log INFO [2022-01-10 08:47:17,047] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/systemd_journal.gz to cbcollect_info_ns_1@172.23.105.187_20220110-164105/systemd_journal.gz INFO [2022-01-10 08:47:17,376] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/stats_snapshot/01FS2E408GCBGTVP4T4B1HAHAW/tombstones to cbcollect_info_ns_1@172.23.106.61_20220110-164105/stats_snapshot/01FS2E408GCBGTVP4T4B1HAHAW/tombstones INFO [2022-01-10 08:47:17,398] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_dcpdebug.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_dcpdebug.log INFO [2022-01-10 08:47:17,897] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.http_access_internal.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.http_access_internal.log INFO [2022-01-10 08:47:18,718] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.info.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.info.log INFO [2022-01-10 08:47:18,739] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.query.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.query.log INFO [2022-01-10 08:47:19,235] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.query.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.query.log INFO [2022-01-10 08:47:20,093] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/master_events.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/master_events.log INFO [2022-01-10 08:47:20,096] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/syslog.tar.gz to cbcollect_info_ns_1@172.23.107.228_20220110-164105/syslog.tar.gz INFO [2022-01-10 08:47:20,581] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.debug.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.debug.log INFO [2022-01-10 08:47:20,632] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/projector_cprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/projector_cprof.log INFO [2022-01-10 08:47:21,456] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.error.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.error.log INFO [2022-01-10 08:47:21,473] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.mapreduce_errors.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.mapreduce_errors.log INFO [2022-01-10 08:47:21,946] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.backup_service.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.backup_service.log INFO [2022-01-10 08:47:21,974] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/projector_pprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/projector_pprof.log INFO [2022-01-10 08:47:22,835] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.goxdcr.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.goxdcr.log INFO [2022-01-10 08:47:22,853] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_trace.json to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_trace.json INFO [2022-01-10 08:47:23,265] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/stats_snapshot/01FS2E422MMV4QPB3VQ2V69XT9/meta.json to cbcollect_info_ns_1@172.23.107.227_20220110-164105/stats_snapshot/01FS2E422MMV4QPB3VQ2V69XT9/meta.json INFO [2022-01-10 08:47:23,335] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/Couchbase-172.23.105.187.pcap.1641832708.gz to Couchbase-172.23.105.187.pcap.1641832708.gz INFO [2022-01-10 08:47:24,161] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/goxdcr_replications.json to cbcollect_info_ns_1@172.23.106.61_20220110-164105/goxdcr_replications.json INFO [2022-01-10 08:47:24,163] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_log INFO [2022-01-10 08:47:24,606] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.mapreduce_errors.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.mapreduce_errors.log INFO [2022-01-10 08:47:25,507] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.error.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.error.log INFO [2022-01-10 08:47:25,512] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.prometheus.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.prometheus.log INFO [2022-01-10 08:47:25,971] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_info.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_info.log INFO [2022-01-10 08:47:26,904] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.metakv.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.metakv.log INFO [2022-01-10 08:47:26,919] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/diag.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/diag.log INFO [2022-01-10 08:47:27,271] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.xdcr_target.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.xdcr_target.log INFO [2022-01-10 08:47:28,186] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_info.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_info.log INFO [2022-01-10 08:47:28,213] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.babysitter.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.babysitter.log INFO [2022-01-10 08:47:28,437] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.views.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.views.log INFO [2022-01-10 08:47:29,465] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/stats.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/stats.log INFO [2022-01-10 08:47:29,700] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/rebalance_report_20220110T163901.json to cbcollect_info_ns_1@172.23.106.61_20220110-164105/rebalance_report_20220110T163901.json INFO [2022-01-10 08:47:29,914] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.info.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.info.log INFO [2022-01-10 08:47:30,882] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/Couchbase-172.23.107.228.pcap.1641832706.gz to Couchbase-172.23.107.228.pcap.1641832706.gz INFO [2022-01-10 08:47:31,023] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ddocs.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ddocs.log INFO [2022-01-10 08:47:31,286] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/goxdcr_replications.json to cbcollect_info_ns_1@172.23.107.227_20220110-164105/goxdcr_replications.json INFO [2022-01-10 08:47:32,379] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/Couchbase-172.23.106.61.pcap.1641832706.gz to Couchbase-172.23.106.61.pcap.1641832706.gz INFO [2022-01-10 08:47:32,628] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_cbas_debug.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_cbas_debug.log INFO [2022-01-10 08:47:34,006] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/systemd_journal.gz to cbcollect_info_ns_1@172.23.107.227_20220110-164105/systemd_journal.gz INFO [2022-01-10 08:47:37,727] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.ns_couchdb.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.ns_couchdb.log INFO [2022-01-10 08:47:39,123] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/couchbase.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/couchbase.log INFO [2022-01-10 08:47:40,800] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_log INFO [2022-01-10 08:47:42,153] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.eventing.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.eventing.log INFO [2022-01-10 08:47:43,521] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/event_log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/event_log INFO [2022-01-10 08:47:44,916] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/master_events.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/master_events.log INFO [2022-01-10 08:47:46,290] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.babysitter.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.babysitter.log INFO [2022-01-10 08:47:47,329] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.projector.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.projector.log INFO [2022-01-10 08:47:48,491] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.goxdcr.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.goxdcr.log INFO [2022-01-10 08:47:49,854] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/goxdcr_pprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/goxdcr_pprof.log INFO [2022-01-10 08:47:50,664] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/couchbase.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/couchbase.log INFO [2022-01-10 08:47:51,329] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.couchdb.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.couchdb.log INFO [2022-01-10 08:47:52,201] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.http_access_internal.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.http_access_internal.log INFO [2022-01-10 08:47:52,686] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_trace.json to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_trace.json INFO [2022-01-10 08:47:53,548] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.couchdb.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.couchdb.log INFO [2022-01-10 08:47:54,035] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_debug.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_debug.log INFO [2022-01-10 08:47:54,932] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/goxdcr_replications.json to cbcollect_info_ns_1@172.23.105.187_20220110-164105/goxdcr_replications.json INFO [2022-01-10 08:47:55,410] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.reports.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.reports.log INFO [2022-01-10 08:47:55,930] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/chronicle_logs.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/chronicle_logs.log INFO [2022-01-10 08:47:56,267] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/projector_mprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/projector_mprof.log INFO [2022-01-10 08:47:56,757] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/stats_snapshot/01FS2E422MMV4QPB3VQ2V69XT9/chunks/000001 to cbcollect_info_ns_1@172.23.107.227_20220110-164105/stats_snapshot/01FS2E422MMV4QPB3VQ2V69XT9/chunks/000001 INFO [2022-01-10 08:47:57,288] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.xdcr_target.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.xdcr_target.log INFO [2022-01-10 08:47:57,623] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/stats_snapshot/01FS2E41CS7ZW6T7BNHJYKRYNX/index to cbcollect_info_ns_1@172.23.105.187_20220110-164105/stats_snapshot/01FS2E41CS7ZW6T7BNHJYKRYNX/index INFO [2022-01-10 08:47:58,163] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ini.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ini.log INFO [2022-01-10 08:47:58,237] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/goxdcr_cprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/goxdcr_cprof.log INFO [2022-01-10 08:47:58,651] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/stats_snapshot/01FS2E41Y316WQ7WKZ2K0MJ1YA/meta.json to cbcollect_info_ns_1@172.23.107.228_20220110-164105/stats_snapshot/01FS2E41Y316WQ7WKZ2K0MJ1YA/meta.json INFO [2022-01-10 08:47:58,981] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_error.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_error.log INFO [2022-01-10 08:47:59,516] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.http_access.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.http_access.log INFO [2022-01-10 08:47:59,554] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/Couchbase-172.23.107.227.pcap.1641832706.gz to Couchbase-172.23.107.227.pcap.1641832706.gz INFO [2022-01-10 08:47:59,975] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_cbas_debug.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_cbas_debug.log INFO [2022-01-10 08:48:00,327] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_access.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_access.log INFO [2022-01-10 08:48:00,845] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/mdocs.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/mdocs.log INFO [2022-01-10 08:48:01,314] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_warn.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_warn.log INFO [2022-01-10 08:48:01,646] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_debug.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_debug.log INFO [2022-01-10 08:48:02,211] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.prometheus.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.prometheus.log INFO [2022-01-10 08:48:02,679] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/users.dets to cbcollect_info_ns_1@172.23.107.228_20220110-164105/users.dets INFO [2022-01-10 08:48:03,039] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/memcached.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/memcached.log INFO [2022-01-10 08:48:03,607] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.babysitter.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.babysitter.log INFO [2022-01-10 08:48:04,106] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/stats_snapshot/01FS2E41Y316WQ7WKZ2K0MJ1YA/chunks/000001 to cbcollect_info_ns_1@172.23.107.228_20220110-164105/stats_snapshot/01FS2E41Y316WQ7WKZ2K0MJ1YA/chunks/000001 INFO [2022-01-10 08:48:04,722] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.query.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.query.log INFO [2022-01-10 08:48:04,973] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.xdcr_target.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.xdcr_target.log INFO [2022-01-10 08:48:05,494] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_error.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_error.log INFO [2022-01-10 08:48:06,123] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/goxdcr_pprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/goxdcr_pprof.log INFO [2022-01-10 08:48:06,407] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.eventing.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.eventing.log INFO [2022-01-10 08:48:06,851] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/systemd_journal.gz to cbcollect_info_ns_1@172.23.107.228_20220110-164105/systemd_journal.gz INFO [2022-01-10 08:48:07,475] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/syslog.tar.gz to cbcollect_info_ns_1@172.23.105.187_20220110-164105/syslog.tar.gz INFO [2022-01-10 08:48:07,787] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/cbcollect_info.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/cbcollect_info.log INFO [2022-01-10 08:48:08,995] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_dcpdebug.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_dcpdebug.log INFO [2022-01-10 08:48:09,170] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/stats.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/stats.log INFO [2022-01-10 08:48:10,435] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.xdcr_target.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.xdcr_target.log INFO [2022-01-10 08:48:10,461] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.json_rpc.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.json_rpc.log INFO [2022-01-10 08:48:10,578] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/memcached.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/memcached.log INFO [2022-01-10 08:48:11,804] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.indexer_stats.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.indexer_stats.log INFO [2022-01-10 08:48:11,818] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.prometheus.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.prometheus.log INFO [2022-01-10 08:48:12,413] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_dcpdebug.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_dcpdebug.log INFO [2022-01-10 08:48:13,170] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/event_log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/event_log INFO [2022-01-10 08:48:13,192] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ini.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ini.log INFO [2022-01-10 08:48:13,740] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.projector.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.projector.log INFO [2022-01-10 08:48:14,527] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/rebalance_report_20220110T163901.json to cbcollect_info_ns_1@172.23.107.228_20220110-164105/rebalance_report_20220110T163901.json INFO [2022-01-10 08:48:14,592] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/chronicle_logs.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/chronicle_logs.log INFO [2022-01-10 08:48:15,096] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.http_access_internal.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.http_access_internal.log INFO [2022-01-10 08:48:15,879] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.projector.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.projector.log INFO [2022-01-10 08:48:15,952] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/goxdcr_mprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/goxdcr_mprof.log INFO [2022-01-10 08:48:16,436] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_cbas_debug.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_cbas_debug.log INFO [2022-01-10 08:48:17,240] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.fts.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.fts.log INFO [2022-01-10 08:48:17,356] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_log INFO [2022-01-10 08:48:17,832] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/systemd_journal.gz to cbcollect_info_ns_1@172.23.106.61_20220110-164105/systemd_journal.gz INFO [2022-01-10 08:48:18,566] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ddocs.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ddocs.log INFO [2022-01-10 08:48:18,767] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/cbcollect_info.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/cbcollect_info.log INFO [2022-01-10 08:48:19,926] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/projector_cprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/projector_cprof.log INFO [2022-01-10 08:48:20,093] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.indexer.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.indexer.log INFO [2022-01-10 08:48:21,287] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.http_access_internal.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.http_access_internal.log INFO [2022-01-10 08:48:21,447] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/projector_pprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/projector_pprof.log INFO [2022-01-10 08:48:21,483] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/projector_cprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/projector_cprof.log INFO [2022-01-10 08:48:22,452] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ini.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ini.log INFO [2022-01-10 08:48:22,624] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/users.dets to cbcollect_info_ns_1@172.23.106.61_20220110-164105/users.dets INFO [2022-01-10 08:48:22,800] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/event_log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/event_log INFO [2022-01-10 08:48:23,768] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.reports.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.reports.log INFO [2022-01-10 08:48:23,954] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.views.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.views.log INFO [2022-01-10 08:48:24,154] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.fts.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.fts.log INFO [2022-01-10 08:48:24,819] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_warn.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_warn.log INFO [2022-01-10 08:48:25,099] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/mdocs.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/mdocs.log INFO [2022-01-10 08:48:25,298] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/couchbase.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/couchbase.log INFO [2022-01-10 08:48:25,502] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/master_events.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/master_events.log INFO [2022-01-10 08:48:26,159] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/rebalance_report_20220110T163901.json to cbcollect_info_ns_1@172.23.107.227_20220110-164105/rebalance_report_20220110T163901.json INFO [2022-01-10 08:48:26,442] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/couchstore_local.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/couchstore_local.log INFO [2022-01-10 08:48:26,829] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_cbas_debug.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_cbas_debug.log INFO [2022-01-10 08:48:26,871] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_error.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_error.log INFO [2022-01-10 08:48:27,491] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/projector_mprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/projector_mprof.log INFO [2022-01-10 08:48:27,772] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/kv_trace.json to cbcollect_info_ns_1@172.23.107.228_20220110-164105/kv_trace.json INFO [2022-01-10 08:48:28,159] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.eventing.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.eventing.log INFO [2022-01-10 08:48:28,222] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.metakv.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.metakv.log INFO [2022-01-10 08:48:28,827] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/couchstore_local.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/couchstore_local.log INFO [2022-01-10 08:48:29,237] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/stats_snapshot/01FS2E41Y316WQ7WKZ2K0MJ1YA/tombstones to cbcollect_info_ns_1@172.23.107.228_20220110-164105/stats_snapshot/01FS2E41Y316WQ7WKZ2K0MJ1YA/tombstones INFO [2022-01-10 08:48:29,490] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ddocs.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ddocs.log INFO [2022-01-10 08:48:29,547] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/stats_snapshot/01FS2E408GCBGTVP4T4B1HAHAW/meta.json to cbcollect_info_ns_1@172.23.106.61_20220110-164105/stats_snapshot/01FS2E408GCBGTVP4T4B1HAHAW/meta.json INFO [2022-01-10 08:48:30,182] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_dcpdebug.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_dcpdebug.log INFO [2022-01-10 08:48:30,610] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.mapreduce_errors.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.mapreduce_errors.log INFO [2022-01-10 08:48:30,886] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.goxdcr.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.goxdcr.log INFO [2022-01-10 08:48:30,908] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_log INFO [2022-01-10 08:48:31,502] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/hashtable_dump.txt to hashtable_dump.txt INFO [2022-01-10 08:48:31,931] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/goxdcr_mprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/goxdcr_mprof.log INFO [2022-01-10 08:48:32,232] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/hashtable_dump.txt to hashtable_dump.txt INFO [2022-01-10 08:48:32,233] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.json_rpc.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.json_rpc.log INFO [2022-01-10 08:48:32,840] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/cbcollect_info.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/cbcollect_info.log INFO [2022-01-10 08:48:33,287] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/goxdcr_pprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/goxdcr_pprof.log INFO [2022-01-10 08:48:33,547] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.info.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.info.log INFO [2022-01-10 08:48:33,577] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_shutdown.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_shutdown.log INFO [2022-01-10 08:48:34,161] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/stats_snapshot/01FS2E422MMV4QPB3VQ2V69XT9/index to cbcollect_info_ns_1@172.23.107.227_20220110-164105/stats_snapshot/01FS2E422MMV4QPB3VQ2V69XT9/index INFO [2022-01-10 08:48:34,616] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.goxdcr.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.goxdcr.log INFO [2022-01-10 08:48:34,906] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/stats_snapshot/01FS2E41CS7ZW6T7BNHJYKRYNX/chunks/000001 to cbcollect_info_ns_1@172.23.105.187_20220110-164105/stats_snapshot/01FS2E41CS7ZW6T7BNHJYKRYNX/chunks/000001 INFO [2022-01-10 08:48:34,914] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/syslog.tar.gz to cbcollect_info_ns_1@172.23.106.61_20220110-164105/syslog.tar.gz INFO [2022-01-10 08:48:35,559] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/goxdcr_mprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/goxdcr_mprof.log INFO [2022-01-10 08:48:35,964] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/projector_pprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/projector_pprof.log INFO [2022-01-10 08:48:36,303] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.indexer_stats.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.indexer_stats.log INFO [2022-01-10 08:48:36,351] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.ns_couchdb.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.ns_couchdb.log INFO [2022-01-10 08:48:36,888] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ini.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ini.log INFO [2022-01-10 08:48:37,303] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/goxdcr_remote_clusters.json to cbcollect_info_ns_1@172.23.107.228_20220110-164105/goxdcr_remote_clusters.json INFO [2022-01-10 08:48:37,611] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.debug.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.debug.log INFO [2022-01-10 08:48:37,692] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_warn.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_warn.log INFO [2022-01-10 08:48:38,292] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.stats.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.stats.log INFO [2022-01-10 08:48:38,654] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/projector_mprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/projector_mprof.log INFO [2022-01-10 08:48:39,019] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/goxdcr_mprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/goxdcr_mprof.log INFO [2022-01-10 08:48:39,033] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.babysitter.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.babysitter.log INFO [2022-01-10 08:48:39,644] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/chronicle_logs.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/chronicle_logs.log INFO [2022-01-10 08:48:39,949] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/hashtable_dump.txt to hashtable_dump.txt INFO [2022-01-10 08:48:40,339] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.json_rpc.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.json_rpc.log INFO [2022-01-10 08:48:40,363] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/diag.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/diag.log INFO [2022-01-10 08:48:40,968] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.prometheus.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.prometheus.log INFO [2022-01-10 08:48:41,324] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/stats_snapshot/01FS2E41Y316WQ7WKZ2K0MJ1YA/index to cbcollect_info_ns_1@172.23.107.228_20220110-164105/stats_snapshot/01FS2E41Y316WQ7WKZ2K0MJ1YA/index INFO [2022-01-10 08:48:41,660] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/.ignore to .ignore INFO [2022-01-10 08:48:41,808] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.reports.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.reports.log INFO [2022-01-10 08:48:42,317] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/kv_trace.json to cbcollect_info_ns_1@172.23.107.227_20220110-164105/kv_trace.json INFO [2022-01-10 08:48:42,696] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/master_events.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/master_events.log INFO [2022-01-10 08:48:43,059] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.stats.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.stats.log INFO [2022-01-10 08:48:43,161] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_trace.json to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_trace.json INFO [2022-01-10 08:48:43,830] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/.ignore to .ignore INFO [2022-01-10 08:48:44,150] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.http_access.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.http_access.log INFO [2022-01-10 08:48:44,373] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.debug.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.debug.log INFO [2022-01-10 08:48:44,485] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_info.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_info.log INFO [2022-01-10 08:48:45,162] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/goxdcr_remote_clusters.json to cbcollect_info_ns_1@172.23.107.227_20220110-164105/goxdcr_remote_clusters.json INFO [2022-01-10 08:48:45,459] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.ns_couchdb.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.ns_couchdb.log INFO [2022-01-10 08:48:45,752] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_debug.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_debug.log INFO [2022-01-10 08:48:45,818] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/hashtable_dump.txt to hashtable_dump.txt INFO [2022-01-10 08:48:46,499] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/diag.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/diag.log INFO [2022-01-10 08:48:46,852] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/cbcollect_info.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/cbcollect_info.log INFO [2022-01-10 08:48:47,122] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_trace.json to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.analytics_trace.json INFO [2022-01-10 08:48:47,139] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/rebalance_report_20220110T163901.json to cbcollect_info_ns_1@172.23.105.187_20220110-164105/rebalance_report_20220110T163901.json INFO [2022-01-10 08:48:47,942] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.error.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.error.log INFO [2022-01-10 08:48:48,196] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/goxdcr_replications.json to cbcollect_info_ns_1@172.23.107.228_20220110-164105/goxdcr_replications.json INFO [2022-01-10 08:48:48,474] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/chronicle_logs.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/chronicle_logs.log INFO [2022-01-10 08:48:48,486] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/stats_snapshot/01FS2E41CS7ZW6T7BNHJYKRYNX/tombstones to cbcollect_info_ns_1@172.23.105.187_20220110-164105/stats_snapshot/01FS2E41CS7ZW6T7BNHJYKRYNX/tombstones INFO [2022-01-10 08:48:49,243] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_shutdown.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_shutdown.log INFO [2022-01-10 08:48:49,507] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.info.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.info.log INFO [2022-01-10 08:48:49,813] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.reports.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.reports.log INFO [2022-01-10 08:48:49,814] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/goxdcr_remote_clusters.json to cbcollect_info_ns_1@172.23.105.187_20220110-164105/goxdcr_remote_clusters.json INFO [2022-01-10 08:48:50,567] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.indexer_stats.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.indexer_stats.log INFO [2022-01-10 08:48:50,846] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.stats.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.stats.log INFO [2022-01-10 08:48:51,166] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/stats.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/stats.log INFO [2022-01-10 08:48:51,184] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/goxdcr_pprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/goxdcr_pprof.log INFO [2022-01-10 08:48:51,896] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.fts.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.fts.log INFO [2022-01-10 08:48:52,194] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/.ignore to .ignore INFO [2022-01-10 08:48:52,527] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/kv_trace.json to cbcollect_info_ns_1@172.23.106.61_20220110-164105/kv_trace.json INFO [2022-01-10 08:48:52,540] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_shutdown.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.analytics_shutdown.log INFO [2022-01-10 08:48:53,195] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/projector_cprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/projector_cprof.log INFO [2022-01-10 08:48:53,499] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/goxdcr_cprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/goxdcr_cprof.log INFO [2022-01-10 08:48:53,923] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/.ignore to .ignore INFO [2022-01-10 08:48:54,042] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/stats_snapshot/01FS2E408GCBGTVP4T4B1HAHAW/chunks/000001 to cbcollect_info_ns_1@172.23.106.61_20220110-164105/stats_snapshot/01FS2E408GCBGTVP4T4B1HAHAW/chunks/000001 INFO [2022-01-10 08:48:54,540] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.backup_service.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.backup_service.log INFO [2022-01-10 08:48:54,807] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.views.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.views.log INFO [2022-01-10 08:48:55,254] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.ns_couchdb.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.ns_couchdb.log INFO [2022-01-10 08:48:55,396] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/couchstore_local.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/couchstore_local.log INFO [2022-01-10 08:48:55,875] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.metakv.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.metakv.log INFO [2022-01-10 08:48:56,138] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.couchdb.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.couchdb.log INFO [2022-01-10 08:48:56,638] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/kv_trace.json to cbcollect_info_ns_1@172.23.105.187_20220110-164105/kv_trace.json INFO [2022-01-10 08:48:56,691] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/stats_snapshot/01FS2E408GCBGTVP4T4B1HAHAW/index to cbcollect_info_ns_1@172.23.106.61_20220110-164105/stats_snapshot/01FS2E408GCBGTVP4T4B1HAHAW/index INFO [2022-01-10 08:48:57,239] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/syslog.tar.gz to cbcollect_info_ns_1@172.23.107.227_20220110-164105/syslog.tar.gz INFO [2022-01-10 08:48:57,575] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_access.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_access.log INFO [2022-01-10 08:48:58,082] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.indexer_stats.log to cbcollect_info_ns_1@172.23.106.61_20220110-164105/ns_server.indexer_stats.log INFO [2022-01-10 08:48:58,248] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.views.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.views.log INFO [2022-01-10 08:48:58,703] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_error.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.analytics_error.log INFO [2022-01-10 08:48:58,938] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_debug.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.analytics_debug.log INFO [2022-01-10 08:48:59,562] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.error.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.error.log INFO [2022-01-10 08:49:00,002] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.json_rpc.log to cbcollect_info_ns_1@172.23.107.227_20220110-164105/ns_server.json_rpc.log INFO [2022-01-10 08:49:00,206] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/diag.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/diag.log INFO [2022-01-10 08:49:00,852] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.stats.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.stats.log INFO [2022-01-10 08:49:01,640] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.eventing.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/ns_server.eventing.log INFO [2022-01-10 08:49:02,169] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.metakv.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.metakv.log INFO [2022-01-10 08:49:02,934] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-164105/memcached.log to cbcollect_info_ns_1@172.23.107.228_20220110-164105/memcached.log INFO [2022-01-10 08:49:03,478] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/users.dets to cbcollect_info_ns_1@172.23.105.187_20220110-164105/users.dets INFO [2022-01-10 08:49:04,762] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.http_access.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.http_access.log INFO [2022-01-10 08:49:06,070] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.projector.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/ns_server.projector.log INFO [2022-01-10 08:49:07,360] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-164105/goxdcr_cprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-164105/goxdcr_cprof.log INFO [2022-01-10 08:49:08,546] jepsen test runner - jepsen.core Snarfing log files WARN [2022-01-10 08:49:08,548] jepsen node 172.23.106.61 - couchbase.core Ignoring duplicate log collection request WARN [2022-01-10 08:49:08,548] jepsen node 172.23.105.187 - couchbase.core Ignoring duplicate log collection request WARN [2022-01-10 08:49:08,548] jepsen node 172.23.107.228 - couchbase.core Ignoring duplicate log collection request WARN [2022-01-10 08:49:08,548] jepsen node 172.23.107.227 - couchbase.core Ignoring duplicate log collection request INFO [2022-01-10 08:49:08,552] jepsen node 172.23.105.187 - couchbase.util Tearing down couchbase node INFO [2022-01-10 08:49:08,552] jepsen node 172.23.107.228 - couchbase.util Tearing down couchbase node INFO [2022-01-10 08:49:08,552] jepsen node 172.23.106.61 - couchbase.util Tearing down couchbase node INFO [2022-01-10 08:49:08,552] jepsen node 172.23.107.227 - couchbase.util Tearing down couchbase node INFO [2022-01-10 08:49:12,579] jepsen node 172.23.105.187 - couchbase.util Teardown Complete INFO [2022-01-10 08:49:12,784] jepsen node 172.23.107.228 - couchbase.util Teardown Complete INFO [2022-01-10 08:49:12,989] jepsen node 172.23.107.227 - couchbase.util Teardown Complete INFO [2022-01-10 08:49:13,193] jepsen node 172.23.106.61 - couchbase.util Teardown Complete INFO [2022-01-10 08:49:13,196] jepsen test runner - jepsen.core Analyzing... INFO [2022-01-10 08:49:16,290] jepsen test runner - jepsen.core Analysis complete INFO [2022-01-10 08:49:16,292] jepsen fressian - couchbase.core Not writing fressian INFO [2022-01-10 08:49:16,297] jepsen results - jepsen.store Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T083805.000-0800/results.edn INFO [2022-01-10 08:49:16,617] jepsen test runner - jepsen.core {:timeline {:valid? true}, :sanity {:valid? :unknown, :error "Test aborted due to nemesis exception"}, :set {:ok-count 9140, :valid? true, :lost-count 0, :lost "#{}", :acknowledged-count 7263, :recovered "#{4770 4772 4811 4813 4855 4857 4863 4967 4983..4984 4999 5085 5115 5124 5139 5255 5260 5282 5299 5309 5326 5348 5352..5353 5367 5387 5414 5421 5448..5449 5451 5457 5464 5481 5514 5516 5520..5523 5538 5551 5553 5579 5581 5583 5587 5616 5622 5638 5648..5649 5652 5662 5665 5682 5684..5685 5687 5716 5720 5722..5723 5726 5748 5751 5753 5757 5760 5764 5778..5780 5783 5799 5801 5805 5828 5830 5832..5833 5836 5839..5841 5843 5845 5850 5859 5864 5870..5871 5874 5876..5877 5888..5890 5893 5900..5904 5906 5917 5919 5923 5928..5929 5935..5937 5944..5945 5949 5951 5965 5985 5995 5997 6001 6003..6007 6028 6031..6034 6037 6042..6043 6046..6047 6068..6070 6072..6073 6076 6082 6091 6093..6095 6102..6103 6106 6112 6133 6135 6140..6141 6143 6145 6147 6157..6159 6162..6163 6169 6173..6175 6177 6193..6196 6204 6207 6212 6226..6228 6230 6233 6236 6263 6270 6273 6289..6291 6293 6296..6297 6330 6333..6334 6340..6343 6355 6358..6359 6362 6371 6376..6378 6380 6393 6399 6403..6405 6407 6413 6428..6431 6433 6441..6442 6458 6476 6496..6497 6506 6525 6528 6530..6531 6533 6535 6541 6543 6547 6552 6554 6556..6557 6559 6562..6563 6574..6575 6578 6580 6589 6593 6596 6602 6605 6607 6609 6611..6612 6624 6631 6633..6634 6636 6642..6645 6656 6658 6662 6668 6670 6674..6677 6680..6681 6683 6696..6697 6699 6702 6704..6705 6709..6710 6713 6718 6725 6727..6729 6732..6733 6735 6740 6743 6745..6746 6754..6756 6759..6763 6768 6770..6771 6774 6776 6779..6780 6788 6791..6792 6800 6802 6809..6810 6813..6814 6822..6823 6825 6827 6836 6838 6844 6846..6847 6849..6851 6857 6863..6865 6867..6869 6871..6873 6880 6883..6884 6891 6898 6908 6910..6913 6915..6916 6920..6922 6924..6925 6945 6947..6951 6956..6957 6960 6962..6965 6967 6973 6978 6982..6984 6986 6992 6999 7005 7014 7016 7020 7022 7029 7038 7043 7048..7050 7052..7055 7061 7066..7067 7077..7078 7080 7085 7087 7096 7105..7106 7108..7110 7115 7121 7124 7127..7129 7131..7132 7139..7143 7148 7150..7151 7154 7157..7159 7161 7165..7166 7172 7174..7176 7179 7181 7184..7186 7195 7199 7204..7206 7208..7209 7211..7212 7214 7217 7220 7223..7226 7229..7230 7232..7233 7239 7241 7243 7248 7251..7252 7256..7258 7264..7266 7278..7280 7283 7285 7294 7298 7303 7306 7309..7310 7313 7315 7323..7324 7327 7332 7338 7342..7343 7349 7353..7354 7356..7357 7361..7363 7365 7376 7380..7381 7383..7385 7387 7395..7396 7398..7399 7405 7409 7413 7415 7421 7425 7454 7460 7462 7477 7481 7483 7498 7508 7510..7512 7515 7521 7525 7527 7539 7543 7551..7552 7556 7559..7560 7562 7565..7567 7575 7580 7583 7587 7594 7596 7606 7608 7610 7613..7617 7620..7622 7625..7627 7632..7633 7638 7640 7642 7650..7652 7654 7657..7658 7660 7662..7663 7666 7674 7678 7683 7685 7696 7698 7705 7711..7712 7714..7716 7722 7726..7727 7732..7733 7738 7743 7752 7754 7756 7760 7762 7764 7774..7775 7777..7778 7780..7784 7795 7797..7800 7803..7804 7816..7818 7823 7831 7833..7834 7838..7839 7841..7842 7847..7848 7850..7853 7865..7866 7873 7875..7877 7886 7888..7889 7891 7893 7896..7897 7900..7903 7907 7915 7917..7918 7922 7934 7936 7943 7946..7947 7950 7959 7964 7967 7969 7972 7975..7977 7987..7988 7996 8001 8008 8011 8018..8019 8022 8024 8026..8027 8035..8036 8039 8044 8051 8053..8054 8056 8062..8064 8067 8070 8082..8083 8085 8087..8088 8090..8091 8101..8103 8108 8111..8114 8117..8118 8120..8121 8123 8125..8127 8136..8137 8146..8148 8151..8154 8160 8167..8168 8172 8181 8185..8189 8191..8192 8198..8199 8211 8213..8214 8216 8220..8222 8225 8235 8237..8238 8247 8250..8251 8254..8255 8257 8260..8261 8263 8265..8266 8269 8271 8279..8280 8286 8289..8290 8293 8300 8303 8316 8319..8320 8322 8324 8327 8334..8336 8338 8344..8345 8349 8351 8360 8367 8370 8373 8378 8383 8385..8387 8389 8391 8393 8401..8403 8410 8413 8418 8427 8435..8436 8438..8439 8444..8445 8448 8452 8457 8463..8464 8466..8468 8471..8472 8483..8485 8490 8492 8502..8503 8513..8517 8519 8521 8523 8527 8537 8539 8548 8551 8553 8568 8570 8580 8589 8591 8593 8598 8616 8620 8623 8635 8644 8648 8651..8653 8656 8662 8665..8667 8671..8672 8681 8685 8687 8689..8690 8698 8700 8709..8710 8712 8715 8717..8718 8720..8721 8723 8734 8736 8739 8747 8749 8752 8754..8755 8764 8766 8770 8779 8781 8804..8807 8809..8810 8812 8824..8826 8828 8831 8834 8837 8840 8854 8857..8858 8860 8863 8872 8874 8876..8877 8880..8883 8894..8896 8899 8906..8910 8925 8930..8931 8941 8943..8944 8954 8956..8957 8959..8960 8962..8963 8975..8978 8990 8994..8996 8998 9000..9002 9006 9014..9022 9031 9034..9035 9038..9039 9044..9045 9047..9048 9050 9052 9064..9067 9072..9073 9084 9086..9089 9091..9092 9100..9101 9103..9104 9114 9118 9120 9128 9130 9133..9138 9142 9144..9151 9164..9165 9168..9173 9185 9187 9190 9202 9207 9214..9215 9217 9223 9229 9232 9234..9237 9239 9241..9242 9244..9245 9249 9251 9253 9259 9265 9268 9270..9271 9273 9275..9276 9285 9287 9290..9292 9300 9303 9315 9317 9335..9336 9344..9347 9349 9352..9353 9364 9366..9374 9384..9386 9389 9391..9393 9401 9403 9415..9416 9420..9422 9435..9436 9438..9439 9443..9444 9446 9450 9458 9464 9466 9468 9470..9471 9473 9485 9487..9493 9495 9500..9502 9504..9505 9515 9517..9519 9521 9523 9529..9530 9534..9535 9537 9539 9544..9547 9549 9552..9553 9564..9565 9567..9569 9573 9584..9586 9588..9591 9603 9617..9618 9622..9623 9634..9638 9640..9642 9644..9646 9648 9650 9652..9653 9658 9664 9671..9673 9686..9688 9690..9693 9702..9703 9706 9714 9717..9720 9728 9730 9737 9740 9744..9749 9751..9753 9764 9766..9767 9769 9771..9774 9776 9785 9788..9790 9792..9793 9800 9804 9806 9808 9810..9811 9813 9815 9818 9821 9825..9827 9830..9831 9835..9837 9840..9844 9849 9854..9855 9857 9860 9862..9863 9869 9871 9874 9878..9883 9886 9888 9894 9897..9900 9905..9906 9908 9910..9913 9918 9925..9926 9928..9929 9933..9936 9940..9942 9946..9948 9951 9955..9959 9961..9963 9965 9967 9970..9972 9976..9977 9979 9981..9983 9985 9987 9991..9992 9997..9998 10001 10007..10010 10016..10017 10020 10024 10027..10031 10037..10039 10042..10043 10047..10048 10051 10053 10055 10057..10058 10060 10063..10064 10068 10074..10075 10079 10081..10082 10087 10091..10093 10095..10099 10104..10110 10113 10116..10117 10122 10125 10127..10131 10135..10136 10139..10143 10148..10150 10152 10154..10156 10159 10162..10166 10174 10176 10180 10182..10187 10193..10199 10201 10204 10207 10209..10215 10217 10220..10221 10223 10225..10231 10233 10236 10238..10243 10248 10250 10253 10255..10259 10261..10262 10273 10277 10280..10283 10286..10288 10291 10295..10300 10303 10307 10310..10313 10315 10317 10322..10323 10326 10329 10333 10340 10342 10346 10354 10356 10358..10362 10366 10374 10376 10378..10379 10381..10384 10392 10394 10397..10400 10404 10406 10409..10413 10424..10425 10427..10428 10430 10433..10434 10448..10450 10453..10455 10457 10460..10461 10463..10465 10467 10475 10478 10481..10483 10485..10486 10497 10505 10508..10509 10511..10513 10515..10516 10521..10522 10525..10527 10529..10530 10539 10541..10542 10549 10551..10552 10554..10557 10559..10563 10566 10570 10573..10575 10577 10579..10580 10583..10584 10594 10596 10599 10604 10607 10610..10612 10614..10615 10617 10619..10621 10623 10625..10627 10629 10631 10633 10638..10639 10643 10649..10651 10653 10655..10656 10661..10662 10664..10665 10667 10670 10677..10679 10681..10682 10685 10687 10689 10693..10694 10696..10698 10702 10705..10708 10710 10712..10713 10715 10723..10725 10727 10729..10730 10733 10736 10740..10742 10751 10755 10757..10760 10762..10763 10765..10766 10775..10781 10783 10785..10786 10794..10796 10798..10799 10801..10804 10806 10809 10814..10815 10819..10821 10823 10828 10832 10834..10835 10837..10842 10844..10851 10853..10854 10857..10860 10863..10864 10866..10871 10873..10874 10876..10878 10884..10885 10887..10889 10891..10896 10900..10901 10903 10905..10908 10914..10915 10918..10921 10929..10932 10934..10943 10945..10947 10949 10951..10952 10964 10966 10968 10970..10975 10977 10981 10984 10987..10992 10995..10997 10999..11000 11002..11004 11006..11008 11011..11012 11016..11018 11023..11024 11028 11030..11035 11037 11040 11042..11047 11051 11054..11062 11070..11079 11081..11083 11088..11090 11092..11095 11097..11102 11104..11113 11116 11118..11119}", :ok "#{0..4773 4775..4794 4796..4814 4816..4819 4821..4837 4840..4850 4852..4883 4885 4887..4913 4915 4917..4919 4921 4923..4937 4939..4947 4949 4951..4952 4954..4963 4965..4984 4986..5007 5011 5013..5023 5027..5053 5055..5056 5058..5059 5061..5062 5064..5077 5079 5082..5097 5100..5107 5112..5124 5126..5156 5158..5162 5164..5179 5183..5197 5200..5207 5209..5253 5255..5260 5262..5277 5280 5282..5310 5313..5324 5326..5353 5356 5358..5360 5362 5364..5377 5381..5408 5410 5414..5423 5428..5459 5461..5462 5464..5466 5468..5482 5484..5508 5510 5514..5525 5528..5553 5555 5557..5561 5563..5577 5579..5598 5600..5610 5614..5626 5628..5653 5655 5657..5659 5662..5679 5682..5698 5700..5711 5713..5726 5728..5753 5755 5757..5761 5764..5780 5783..5802 5804..5806 5808..5817 5820..5833 5835..5836 5838..5843 5845 5848..5867 5870..5871 5874..5891 5893..5917 5919..5933 5935..5945 5948..5967 5970 5972 5974 5976..5987 5989 5994..6028 6031..6040 6042..6057 6060..6073 6076 6078..6089 6091..6095 6098..6103 6106 6108..6127 6133..6141 6143..6175 6177..6196 6198..6204 6207..6228 6230..6231 6233..6239 6244..6257 6260..6273 6278..6293 6296..6303 6308..6324 6326..6327 6330 6333..6374 6376..6393 6395 6397..6414 6418..6420 6424..6437 6441..6442 6444..6448 6450 6452 6454..6458 6460 6462 6464 6466 6468..6473 6476 6478..6483 6485..6493 6496..6503 6506 6508 6510..6513 6515..6519 6521..6528 6530..6531 6533..6537 6539 6541 6543..6547 6552 6554 6556..6557 6559 6562..6563 6569 6571..6576 6578 6580..6582 6586 6588..6593 6596 6598..6603 6605 6607 6609 6611..6612 6614..6615 6617..6622 6624 6631 6633..6639 6642..6653 6656 6658 6662 6664..6677 6680..6681 6683..6693 6696..6697 6699..6705 6709..6710 6713..6723 6725 6727..6729 6732..6740 6743..6757 6759..6774 6776..6780 6784..6793 6800 6802 6804..6814 6821..6834 6836 6838..6847 6849..6851 6854..6865 6867..6884 6891 6894..6899 6904..6913 6915..6916 6920..6922 6924..6933 6940..6943 6945 6947..6951 6954..6965 6967 6973..6984 6986 6992 6994..6997 6999..7003 7005 7010 7014..7023 7029 7034..7039 7043..7055 7061 7064..7073 7077..7078 7080 7084..7093 7096 7099..7103 7105..7106 7108..7111 7114..7124 7127..7129 7131..7132 7134..7151 7153..7154 7157..7159 7161 7163..7176 7179 7181 7184..7193 7195 7199..7206 7208..7209 7211..7212 7214..7227 7229..7230 7232..7239 7241 7243..7248 7250..7252 7256..7258 7264..7273 7278..7280 7283..7294 7298 7300..7303 7306 7309..7310 7313..7324 7327 7332 7334..7339 7342..7354 7356..7357 7361..7373 7376 7380..7381 7383..7393 7395..7396 7398..7403 7405 7409 7413..7415 7417..7421 7423 7425 7434..7439 7444..7449 7453..7454 7460 7462 7464..7465 7468..7473 7477 7481 7483..7493 7498 7500..7503 7508 7510..7512 7514..7523 7525..7527 7532 7534..7540 7543..7552 7556 7558..7560 7562 7565..7573 7575 7580 7583 7587..7594 7596 7600..7603 7605..7606 7608 7610 7613..7622 7625..7627 7629..7638 7640 7642 7644..7647 7650..7652 7654 7657..7658 7660 7662..7663 7666 7668..7674 7678 7683 7685 7688..7693 7696 7698 7700..7703 7705 7711..7712 7714..7716 7718..7719 7722 7725..7727 7732..7738 7743..7747 7752 7754 7756 7760..7762 7764 7768..7775 7777..7778 7780..7784 7788..7793 7795 7797..7800 7803..7804 7808..7813 7816..7819 7823..7827 7831 7833..7834 7838..7839 7841..7842 7847..7848 7850..7857 7860..7863 7865..7866 7870 7873 7875..7883 7886..7889 7891 7893 7896..7903 7907..7913 7915 7917..7918 7922 7924..7927 7934 7936 7943..7944 7946..7947 7950 7954..7957 7959..7964 7967 7969 7972 7975..7983 7987..7988 7990 7993 7996 7998..7999 8001 8004..8008 8011 8018..8019 8022 8024 8026..8033 8035..8036 8039..8044 8051 8053..8054 8056 8058..8059 8062..8064 8067 8069..8070 8074..8077 8082..8083 8085 8087..8088 8090..8091 8094..8097 8101..8108 8111..8115 8117..8118 8120..8121 8123 8125..8133 8136..8137 8140..8143 8146..8148 8151..8154 8157..8160 8164 8167..8168 8172 8174..8177 8181 8185..8189 8191..8192 8194..8199 8204..8207 8211 8213..8216 8220..8222 8225 8228..8233 8235..8243 8247 8250..8251 8254..8255 8257..8261 8263 8265..8266 8269 8271 8274..8277 8279..8280 8286 8289..8290 8293..8297 8300 8303..8307 8316 8319..8320 8322..8324 8327..8336 8338 8340..8345 8349 8351..8352 8358..8360 8367 8370 8373..8378 8383 8385..8387 8389 8391..8397 8401..8407 8410 8413 8418 8425 8427..8433 8435..8436 8438..8446 8448 8452 8457..8459 8463..8464 8466..8468 8471..8472 8474..8477 8483..8485 8489..8490 8492..8497 8501..8507 8513..8517 8519 8521 8523 8527..8533 8535 8537 8539..8543 8548 8551..8553 8558..8559 8564 8566 8568 8570 8574..8577 8580 8589 8591..8598 8604..8607 8616 8620 8623 8628..8633 8635 8640..8644 8648 8651..8653 8656 8658..8659 8662 8665..8667 8671..8672 8674..8677 8681 8685 8687 8689..8690 8694..8698 8700 8703..8707 8709..8710 8712 8715 8717..8718 8720..8721 8723 8728..8734 8736 8739..8743 8747 8749 8752 8754..8755 8758..8759 8764 8766 8770 8773..8777 8779 8781 8788 8794..8797 8804..8807 8809..8810 8812 8814..8817 8820..8826 8828 8831 8834 8837..8840 8848..8854 8857..8858 8860 8863..8867 8872 8874 8876..8877 8880..8887 8894..8896 8899 8905..8910 8914..8917 8919..8923 8925 8928 8930..8931 8938..8941 8943..8944 8948..8954 8956..8960 8962..8967 8971 8975..8978 8983..8987 8990..8991 8994..8996 8998 9000..9002 9005..9006 9008..9022 9024..9027 9031 9033..9035 9038..9039 9044..9045 9047..9048 9050 9052 9054..9057 9059..9067 9071..9073 9078..9084 9086..9089 9091..9092 9098..9101 9103..9104 9108..9114 9118 9120 9124..9128 9130 9133..9138 9142 9144..9151 9154..9157 9160..9165 9168..9173 9178..9183 9185 9187 9190 9198..9199 9202 9207..9215 9217 9223..9227 9229 9232 9234..9237 9239 9241..9242 9244..9245 9249..9251 9253..9257 9259..9263 9265 9268 9270..9271 9273 9275..9276 9278..9283 9285 9287 9290..9292 9298..9300 9303 9308..9313 9315 9317 9324..9327 9334..9336 9344..9347 9349 9352..9357 9360..9364 9366..9374 9378..9386 9389 9391..9393 9398..9399 9401 9403 9408..9413 9415..9416 9420..9422 9424..9427 9435..9436 9438..9439 9443..9444 9446 9450 9454..9458 9460..9464 9466 9468 9470..9471 9473 9478..9483 9485 9487..9493 9495 9498..9502 9504..9505 9508..9513 9515 9517..9519 9521 9523..9527 9529..9530 9534..9535 9537 9539 9544..9547 9549 9552..9557 9560..9565 9567..9569 9573 9578..9586 9588..9591 9598..9599 9603 9608..9613 9617..9618 9622..9627 9634..9638 9640..9642 9644..9646 9648 9650 9652..9658 9660..9664 9670..9673 9678..9683 9686..9688 9690..9693 9698..9699 9702..9703 9706 9708..9714 9717..9720 9724..9728 9730..9731 9737 9740 9744..9749 9751..9757 9760..9764 9766..9767 9769 9771..9774 9776 9778..9783 9785 9788..9790 9792..9793 9798..9804 9806 9808 9810..9811 9813 9815 9818..9819 9821 9825..9827 9830..9831 9834..9837 9840..9847 9849 9854..9855 9857 9860 9862..9863 9868..9874 9878..9883 9886 9888..9894 9897..9903 9905..9906 9908 9910..9913 9918..9919 9925..9926 9928..9929 9933..9937 9940..9942 9944..9948 9951 9955..9959 9961..9963 9965 9967..9973 9976..9977 9979 9981..9983 9985 9987..9993 9997..9998 10001 10007..10010 10014..10017 10020..10024 10027..10031 10037..10039 10042..10043 10047..10048 10050..10053 10055 10057..10058 10060 10063..10068 10074..10075 10079 10081..10087 10091..10093 10095..10099 10104..10110 10112..10113 10115..10117 10121..10123 10125..10131 10135..10136 10139..10143 10148..10152 10154..10156 10159 10162..10167 10174 10176..10177 10180 10182..10187 10193..10199 10201 10204 10207 10209..10215 10217 10220..10223 10225..10231 10233 10236 10238..10243 10248..10250 10252..10253 10255..10259 10261..10264 10266..10267 10273 10277 10280..10288 10291 10295..10300 10303 10307 10310..10317 10320..10323 10326 10329 10331 10333 10338..10340 10342 10346 10348..10354 10356 10358..10362 10364..10367 10374 10376 10378..10379 10381..10385 10387 10392 10394 10397..10400 10404 10406 10409..10416 10420..10422 10424..10425 10427..10428 10430 10433..10434 10438..10439 10442 10448..10455 10457 10460..10461 10463..10467 10475 10478 10481..10487 10497 10505 10508..10509 10511..10517 10520..10523 10525..10527 10529..10530 10532..10533 10538..10539 10541..10542 10549..10557 10559..10563 10565..10567 10570 10573..10575 10577 10579..10580 10583..10584 10586..10587 10594 10596 10598..10599 10604 10607 10610..10612 10614..10617 10619..10623 10625..10627 10629 10631..10633 10638..10640 10643 10649..10651 10653 10655..10657 10661..10667 10670 10676..10679 10681..10682 10684..10687 10689 10693..10694 10696..10698 10702 10705..10710 10712..10717 10720..10725 10727 10729..10730 10733 10736 10738..10743 10749 10751..10753 10755 10757..10760 10762..10767 10775..10781 10783..10787 10794..10796 10798..10799 10801..10807 10809 10814..10815 10819..10821 10823 10828..10829 10831..10835 10837..10854 10857..10860 10863..10864 10866..10871 10873..10874 10876..10878 10884..10885 10887..10889 10891..10897 10900..10901 10903 10905..10908 10914..10915 10918..10921 10929..10943 10945..10947 10949 10951..10952 10958 10964 10966..10968 10970..10977 10981 10984 10987..10992 10994..10997 10999..11000 11002..11004 11006..11009 11011..11012 11016..11018 11023..11028 11030..11037 11040 11042..11047 11051 11054..11062 11068 11070..11079 11081..11083 11088..11095 11097..11113 11116 11118..11119}", :attempt-count 11121, :unexpected "#{}", :unexpected-count 0, :recovered-count 1877}, :valid? :unknown} Errors occurred during analysis, but no anomalies found. ಠ~ಠ Jepsen failed with unknown failure Test crashed! ###### Current Test Report ######### pass: 13 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 13/15 = 86.66% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #16 Test params: workload=set-kill,scenario=kill-memcached,replicas=1,no-autofailover,durability=0:0:0:100,eviction-policy=value Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:0:100 --eviction-policy=value --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-16.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 14 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 14/16 = 87.50% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #17 Test params: workload=set-kill,scenario=kill-ns-server,replicas=2,no-autofailover,durability=0:100:0:0,eviction-policy=value Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=kill-ns-server --replicas=2 --no-autofailover --durability=0:100:0:0 --eviction-policy=value --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-17.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 15 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 15/17 = 88.23% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #18 Test params: workload=set-kill,scenario=kill-babysitter,replicas=2,no-autofailover,durability=0:100:0:0,eviction-policy=value Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=kill-babysitter --replicas=2 --no-autofailover --durability=0:100:0:0 --eviction-policy=value --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-18.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 16 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 16/18 = 88.88% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #19 Test params: workload=set-kill,scenario=hard-reboot,replicas=1,no-autofailover,durability=0:0:100:0,eviction-policy=value Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=hard-reboot --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=value --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-19.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 17 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 17/19 = 89.47% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #20 Test params: workload=set-kill,scenario=suspend-process,process-to-suspend=memcached,process-suspend-time=10,replicas=2,autofailover,durability=0:0:100:0,eviction-policy=value Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=suspend-process --process-to-suspend=memcached --process-suspend-time=10 --replicas=2 --autofailover --durability=0:0:100:0 --eviction-policy=value --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-20.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 18 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 18/20 = 90.00% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #21 Test params: workload=set-kill,scenario=kill-memcached-on-slow-disk,replicas=2,no-autofailover,durability=0:0:0:100,eviction-policy=value,manipulate-disks Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=kill-memcached-on-slow-disk --replicas=2 --no-autofailover --durability=0:0:0:100 --eviction-policy=value --manipulate-disks --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-21.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 19 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 19/21 = 90.47% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #22 Test params: workload=set-kill,scenario=kill-memcached-on-slow-disk,replicas=2,no-autofailover,durability=0:0:0:100,eviction-policy=full,manipulate-disks Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=kill-memcached-on-slow-disk --replicas=2 --no-autofailover --durability=0:0:0:100 --eviction-policy=full --manipulate-disks --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-22.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 20 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 20/22 = 90.90% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #23 Test params: workload=register,replicas=2,no-autofailover,rate=0,durability=50:50:0:0,bucket-type=ephemeral,cas Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=register --replicas=2 --no-autofailover --rate=0 --durability=50:50:0:0 --bucket-type=ephemeral --cas --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-23.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 21 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 21/23 = 91.30% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #24 Test params: workload=failover,failover-type=hard,recovery-type=delta,replicas=2,no-autofailover,disrupt-count=1,rate=0,durability=0:100:0:0,bucket-type=ephemeral,cas Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=failover --failover-type=hard --recovery-type=delta --replicas=2 --no-autofailover --disrupt-count=1 --rate=0 --durability=0:100:0:0 --bucket-type=ephemeral --cas --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-24.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 22 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 22/24 = 91.66% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #25 Test params: workload=rebalance,scenario=sequential-rebalance-out-in,replicas=2,no-autofailover,disrupt-count=1,rate=0,durability=0:100:0:0,bucket-type=ephemeral,cas Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=rebalance --scenario=sequential-rebalance-out-in --replicas=2 --no-autofailover --disrupt-count=1 --rate=0 --durability=0:100:0:0 --bucket-type=ephemeral --cas --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-25.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 23 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 23/25 = 92.00% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #26 Test params: workload=set-kill,scenario=kill-memcached,replicas=2,no-autofailover,durability=0:100:0:0,bucket-type=ephemeral Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=kill-memcached --replicas=2 --no-autofailover --durability=0:100:0:0 --bucket-type=ephemeral --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-26.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 24 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 24/26 = 92.30% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #27 Test params: workload=set-kill,scenario=kill-memcached,replicas=1,no-autofailover,durability=0:100:0:0,bucket-type=ephemeral,use-json-docs,use-subdoc Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:100:0:0 --bucket-type=ephemeral --use-json-docs --use-subdoc --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-27.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 25 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 25/27 = 92.59% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #28 Test params: workload=set-kill,scenario=kill-ns-server,replicas=2,no-autofailover,durability=0:100:0:0,bucket-type=ephemeral,use-json-docs,doc-padding-size=64 Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=kill-ns-server --replicas=2 --no-autofailover --durability=0:100:0:0 --bucket-type=ephemeral --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-28.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 26 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 26/28 = 92.85% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #29 Test params: workload=set-kill,scenario=kill-babysitter,replicas=2,no-autofailover,durability=0:100:0:0,bucket-type=ephemeral Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=set-kill --scenario=kill-babysitter --replicas=2 --no-autofailover --durability=0:100:0:0 --bucket-type=ephemeral --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-29.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 27 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 27/29 = 93.10% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #30 Test params: workload=disk-failure,manipulate-disks,replicas=1,durability=0:100:0:0,recovery-type=full,eviction-policy=value,use-json-docs,use-subdoc Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=disk-failure --manipulate-disks --replicas=1 --durability=0:100:0:0 --recovery-type=full --eviction-policy=value --use-json-docs --use-subdoc --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-30.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 28 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 28/30 = 93.33% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #31 Test params: workload=disk-failure,manipulate-disks,replicas=1,durability=0:0:100:0,recovery-type=delta,eviction-policy=full,cas Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=disk-failure --manipulate-disks --replicas=1 --durability=0:0:100:0 --recovery-type=delta --eviction-policy=full --cas --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-31.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 29 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 29/31 = 93.54% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #32 Test params: workload=disk-failure,manipulate-disks,replicas=1,durability=0:0:0:100,recovery-type=delta,eviction-policy=full,cas Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=disk-failure --manipulate-disks --replicas=1 --durability=0:0:0:100 --recovery-type=delta --eviction-policy=full --cas --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-32.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 30 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 30/32 = 93.75% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #33 Test params: workload=disk-failure,manipulate-disks,disrupt-count=2,replicas=2,durability=0:0:0:100,recovery-type=delta,eviction-policy=full Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=disk-failure --manipulate-disks --disrupt-count=2 --replicas=2 --durability=0:0:0:100 --recovery-type=delta --eviction-policy=full --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-33.log file to use error_msg_from_memcached.log Test passed! ###### Current Test Report ######### pass: 31 unknown: 0 fail: 0 crash: 2 memcached errors (not reported as total): 0 31/33 = 93.93% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ################################################################ Running test: #34 Test params: workload=kill-n-disk-failure,manipulate-disks,replicas=1,durability=0:0:0:100,recovery-type=delta,eviction-policy=full Test command: lein trampoline run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=kill-n-disk-failure --manipulate-disks --replicas=1 --durability=0:0:0:100 --recovery-type=delta --eviction-policy=full --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 &> jepsen-output-34.log file to use error_msg_from_memcached_filtered.log 12:50:16.976 [main] INFO jepsen.cli - Test options: {:scopes nil, :hashdump true, :cycles 1, :ssh-private-key "./resources/my.key", :enable-memcached-debug-log-level true, :bucket-type :couchbase, :persist-to 0, :collect-core-files false, :manipulate-disks true, :concurrency 4, :pool-size 6, :autoreprovision-maxnodes 1, :leave-db-running? false, :use-checker nil, :skip-teardown false, :logging-json? false, :dcp-set-read false, :replicate-to 0, :cluster-run false, :cas false, :ssh {:dummy? false, :username "root", :password "root", :strict-host-key-checking false, :private-key-path "./resources/my.key"}, :net-interface "eth0", :collections nil, :use-subdoc false, :argv ("test" "--nodes-file" "./nodes" "--username" "root" "--ssh-private-key=./resources/my.key" "--workload=kill-n-disk-failure" "--manipulate-disks" "--replicas=1" "--durability=0:0:0:100" "--recovery-type=delta" "--eviction-policy=full" "--enable-tcp-capture" "--hashdump" "--enable-memcached-debug-log-level" "--net-interface=eth0"), :durability (0 0 0 100), :disable-out-of-order-execution false, :nodes ["172.23.105.187" "172.23.107.228" "172.23.106.61" "172.23.107.227"], :test-count 1, :install-path "/opt/couchbase", :doc-count 30, :process-suspend-time 10, :perf-graphs false, :replicas 1, :storage-backend :couchstore, :cbcollect true, :connect-timeout 10, :autoreprovision true, :disrupt-count 1, :collect-data-files false, :doc-threads 3, :kv-timeout #object[java.time.Duration 0x361483eb "PT2S"], :disable-auto-compaction false, :time-limit 60, :use-json-docs false, :eviction-policy "fullEviction", :workload "kill-n-disk-failure", :recovery-type :delta, :enable-tcp-capture true} INFO [2022-01-10 12:50:17,216] jepsen test runner - jepsen.core Test version 7a53fac81fdd4c21e7b4a3d63887911ad58baa52 (plus uncommitted changes) INFO [2022-01-10 12:50:17,218] jepsen test runner - jepsen.core Command line: lein run test --nodes-file ./nodes --username root --ssh-private-key=./resources/my.key --workload=kill-n-disk-failure --manipulate-disks --replicas=1 --durability=0:0:0:100 --recovery-type=delta --eviction-policy=full --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 INFO [2022-01-10 12:50:17,310] jepsen test runner - jepsen.core Running test: {:scopes nil :remote #jepsen.control.retry.Remote{:remote #jepsen.control.scp.Remote{:cmd-remote #jepsen.control.clj_ssh.Remote{:concurrency-limit 8, :conn-spec nil, :session nil, :semaphore nil}, :conn-spec nil}, :conn nil} :hashdump true :cycles 1 :ssh-private-key "./resources/my.key" :enable-memcached-debug-log-level true :bucket-type :couchbase :persist-to 0 :collect-core-files false :manipulate-disks true :concurrency 90 :pool-size 6 :autoreprovision-maxnodes 1 :db #object[couchbase.core$couchbase_remote$reify__3667 "0x3c4e17ea" "couchbase.core$couchbase_remote$reify__3667@3c4e17ea"] :leave-db-running? false :use-checker nil :skip-teardown false :name "Couchbase" :logging-json? false :dcp-set-read false :start-time #object[org.joda.time.DateTime "0x1f6ba47b" "2022-01-10T12:50:17.000-08:00"] :net #object[jepsen.net$reify__9402 "0x58be583e" "jepsen.net$reify__9402@58be583e"] :client #couchbase.clients.NewRegisterClient{:cluster nil, :bucket nil, :collection nil, :env nil} :replicate-to 0 :barrier #object[java.util.concurrent.CyclicBarrier "0x401002ce" "java.util.concurrent.CyclicBarrier@401002ce"] :cluster-run false :cas false :pure-generators true :ssh {:dummy? false, :username "root", :password "root", :strict-host-key-checking false, :private-key-path "./resources/my.key"} :net-interface "eth0" :collections nil :checker #object[jepsen.checker$compose$reify__8903 "0x42257df7" "jepsen.checker$compose$reify__8903@42257df7"] :use-subdoc false :argv ("test" "--nodes-file" "./nodes" "--username" "root" "--ssh-private-key=./resources/my.key" "--workload=kill-n-disk-failure" "--manipulate-disks" "--replicas=1" "--durability=0:0:0:100" "--recovery-type=delta" "--eviction-policy=full" "--enable-tcp-capture" "--hashdump" "--enable-memcached-debug-log-level" "--net-interface=eth0") :db-intialized #object[clojure.lang.Atom "0x4b4bc73d" {:status :ready, :val false}] :disrupt-time 20 :nemesis #unprintable "couchbase.nemesis$couchbase$reify__3984@12aa2cca" :durability (0 0 0 100) :disable-out-of-order-execution false :workload-type :register :nodes ["172.23.105.187" "172.23.107.228" "172.23.106.61" "172.23.107.227"] :test-count 1 :install-path "/opt/couchbase" :doc-count 30 :process-suspend-time 10 :perf-graphs false :replicas 1 :storage-backend :couchstore :cbcollect true :connect-timeout 10 :autoreprovision true :disrupt-count 1 :generator (couchbase.workload.Stopable {:gen (jepsen.generator.Any {:gens [(jepsen.generator.OnThreads {:f #{:nemesis}, :gen [{:type :sleep, :value 10} {:type :info, :f :fail-disk, :targeter #object[couchbase.nemesis$target_first_node "0xb53ce18" "couchbase.nemesis$target_first_node@b53ce18"], :target-count 1} {:type :sleep, :value 2} {:type :info, :f :kill-process, :kill-process :memcached, :targeter #object[couchbase.nemesis$target_first_node "0xb53ce18" "couchbase.nemesis$target_first_node@b53ce18"], :target-count 1} {:type :sleep, :value 15} {:type :stop-test}]}) (jepsen.generator.OnThreads {:f #object[clojure.core$complement$fn__5686 "0x7e7987b8" "clojure.core$complement$fn__5686@7e7987b8"], :gen (jepsen.generator.OnThreads {:f #object[clojure.core$complement$fn__5686 "0x6bcfb561" "clojure.core$complement$fn__5686@6bcfb561"], :gen (jepsen.independent.ConcurrentGenerator {:n 3, :fgen #object[couchbase.workload$register_client_gen$fn__3426 "0x49f9758" "couchbase.workload$register_client_gen$fn__3426@49f9758"], :group->threads nil, :thread->group nil, :keys (0 1 2 3 4 5 6 7 ...), :gens nil})})})]})}) :collect-data-files false :doc-threads 3 :kv-timeout #object[java.time.Duration "0x361483eb" "PT2S"] :disable-auto-compaction false :os #object[jepsen.os$reify__2507 "0x209b4b35" "jepsen.os$reify__2507@209b4b35"] :time-limit 60 :use-json-docs false :eviction-policy "fullEviction" :workload "kill-n-disk-failure" :recovery-type :delta :enable-tcp-capture true} INFO [2022-01-10 12:50:18,333] jepsen test runner - jepsen.db Tearing down DB INFO [2022-01-10 12:50:18,341] jepsen node 172.23.106.61 - couchbase.util Tearing down couchbase node INFO [2022-01-10 12:50:18,341] jepsen node 172.23.107.228 - couchbase.util Tearing down couchbase node INFO [2022-01-10 12:50:18,341] jepsen node 172.23.105.187 - couchbase.util Tearing down couchbase node INFO [2022-01-10 12:50:18,342] jepsen node 172.23.107.227 - couchbase.util Tearing down couchbase node INFO [2022-01-10 12:50:20,594] jepsen node 172.23.106.61 - couchbase.util Teardown Complete INFO [2022-01-10 12:50:20,808] jepsen node 172.23.107.227 - couchbase.util Teardown Complete INFO [2022-01-10 12:50:21,383] jepsen node 172.23.105.187 - couchbase.util Teardown Complete INFO [2022-01-10 12:50:21,593] jepsen node 172.23.107.228 - couchbase.util Teardown Complete INFO [2022-01-10 12:50:21,597] jepsen test runner - jepsen.db Setting up DB INFO [2022-01-10 12:50:21,600] jepsen node 172.23.107.228 - couchbase.util Setting up Couchbase Server INFO [2022-01-10 12:50:21,600] jepsen node 172.23.107.227 - couchbase.util Setting up Couchbase Server INFO [2022-01-10 12:50:21,600] jepsen node 172.23.106.61 - couchbase.util Setting up Couchbase Server INFO [2022-01-10 12:50:21,600] jepsen node 172.23.105.187 - couchbase.util Setting up Couchbase Server INFO [2022-01-10 12:50:21,600] jepsen node 172.23.107.227 - couchbase.util Setting up data paths INFO [2022-01-10 12:50:21,600] jepsen node 172.23.105.187 - couchbase.util Setting up data paths INFO [2022-01-10 12:50:21,600] jepsen node 172.23.107.228 - couchbase.util Setting up data paths INFO [2022-01-10 12:50:21,601] jepsen node 172.23.106.61 - couchbase.util Setting up data paths INFO [2022-01-10 12:51:32,098] jepsen node 172.23.106.61 - couchbase.util Changing permissions for /opt/couchbase INFO [2022-01-10 12:51:32,301] jepsen node 172.23.106.61 - couchbase.util Starting daemon INFO [2022-01-10 12:51:43,010] jepsen node 172.23.107.227 - couchbase.util Changing permissions for /opt/couchbase INFO [2022-01-10 12:51:43,212] jepsen node 172.23.107.227 - couchbase.util Starting daemon INFO [2022-01-10 12:51:43,613] jepsen node 172.23.107.228 - couchbase.util Changing permissions for /opt/couchbase INFO [2022-01-10 12:51:44,215] jepsen node 172.23.107.228 - couchbase.util Starting daemon INFO [2022-01-10 12:51:50,721] jepsen node 172.23.105.187 - couchbase.util Changing permissions for /opt/couchbase INFO [2022-01-10 12:51:50,923] jepsen node 172.23.105.187 - couchbase.util Starting daemon INFO [2022-01-10 12:52:04,513] jepsen node 172.23.107.228 - couchbase.util Daemon started INFO [2022-01-10 12:52:04,616] jepsen node 172.23.107.228 - couchbase.util packet dump file name Couchbase-172.23.107.228.pcap INFO [2022-01-10 12:52:04,972] jepsen node 172.23.106.61 - couchbase.util Daemon started INFO [2022-01-10 12:52:05,075] jepsen node 172.23.106.61 - couchbase.util packet dump file name Couchbase-172.23.106.61.pcap INFO [2022-01-10 12:52:05,471] jepsen node 172.23.107.227 - couchbase.util Daemon started INFO [2022-01-10 12:52:05,574] jepsen node 172.23.107.227 - couchbase.util packet dump file name Couchbase-172.23.107.227.pcap INFO [2022-01-10 12:52:07,175] jepsen node 172.23.105.187 - couchbase.util Daemon started INFO [2022-01-10 12:52:07,277] jepsen node 172.23.105.187 - couchbase.util packet dump file name Couchbase-172.23.105.187.pcap INFO [2022-01-10 12:52:07,480] jepsen test runner - jepsen.db Setting up primary 172.23.105.187 INFO [2022-01-10 12:52:07,482] jepsen node 172.23.105.187 - couchbase.util Creating couchbase cluster from 172.23.105.187 INFO [2022-01-10 12:52:07,714] jepsen node 172.23.105.187 - couchbase.util Adding node 172.23.107.228 to cluster INFO [2022-01-10 12:52:21,754] jepsen node 172.23.105.187 - couchbase.util Adding node 172.23.106.61 to cluster INFO [2022-01-10 12:52:28,055] jepsen node 172.23.105.187 - couchbase.util Adding node 172.23.107.227 to cluster INFO [2022-01-10 12:52:33,811] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {} INFO [2022-01-10 12:52:33,812] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 0 INFO [2022-01-10 12:52:33,813] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 12:52:34,834] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 12:52:34,835] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 1 INFO [2022-01-10 12:52:34,835] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 0 INFO [2022-01-10 12:52:35,841] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 12:52:35,842] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 2 INFO [2022-01-10 12:52:35,842] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 1 INFO [2022-01-10 12:52:36,848] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 12:52:36,849] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 3 INFO [2022-01-10 12:52:36,849] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 2 INFO [2022-01-10 12:52:37,855] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 12:52:37,856] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 4 INFO [2022-01-10 12:52:37,856] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 3 INFO [2022-01-10 12:52:38,863] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 12:52:38,864] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 5 INFO [2022-01-10 12:52:38,864] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 4 INFO [2022-01-10 12:52:39,873] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 12:52:39,873] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 6 INFO [2022-01-10 12:52:39,874] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 5 INFO [2022-01-10 12:52:40,882] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 0}, :ns_1@172.23.107.227 {:progress 0}, :ns_1@172.23.105.187 {:progress 0}, :ns_1@172.23.107.228 {:progress 0}} INFO [2022-01-10 12:52:40,883] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 7 INFO [2022-01-10 12:52:40,883] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 6 INFO [2022-01-10 12:52:41,892] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status running, :ns_1@172.23.106.61 {:progress 1}, :ns_1@172.23.107.227 {:progress 1}, :ns_1@172.23.105.187 {:progress 1}, :ns_1@172.23.107.228 {:progress 1}} INFO [2022-01-10 12:52:41,892] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 8 INFO [2022-01-10 12:52:41,893] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 6 INFO [2022-01-10 12:52:42,899] jepsen node 172.23.105.187 - couchbase.util Rebalance status: {:status none} INFO [2022-01-10 12:52:42,900] jepsen node 172.23.105.187 - couchbase.util Rebalance check count: 9 INFO [2022-01-10 12:52:42,900] jepsen node 172.23.105.187 - couchbase.util Rebalance stuck count: 6 INFO [2022-01-10 12:52:42,900] jepsen node 172.23.105.187 - couchbase.util Rebalance complete INFO [2022-01-10 12:52:42,993] jepsen node 172.23.105.187 - couchbase.util Waiting for bucket warmup to complete... INFO [2022-01-10 12:52:43,012] jepsen node 172.23.105.187 - couchbase.util Setting memcached log level to debug INFO [2022-01-10 12:52:43,042] jepsen node 172.23.105.187 - couchbase.util Setup complete INFO [2022-01-10 12:52:43,047] jepsen test runner - jepsen.core Relative time begins now INFO [2022-01-10 12:52:43,055] jepsen node 172.23.107.228 - couchbase.cbclients Opening new client INFO [2022-01-10 12:52:45,329] jepsen node 172.23.107.228 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 12:52:45,330] jepsen node 172.23.107.228 - couchbase.cbclients Opening new client INFO [2022-01-10 12:52:45,364] jepsen node 172.23.107.228 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 12:52:45,364] jepsen node 172.23.107.228 - couchbase.cbclients Opening new client INFO [2022-01-10 12:52:45,397] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0x2b94f89700000001","seedNodes":[{"address":"172.23.105.187"}]} INFO [2022-01-10 12:52:45,400] jepsen node 172.23.107.228 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 12:52:45,400] jepsen node 172.23.107.228 - couchbase.cbclients Opening new client INFO [2022-01-10 12:52:45,412] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000001","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 12:52:45,443] jepsen node 172.23.107.228 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 12:52:45,444] jepsen node 172.23.107.228 - couchbase.cbclients Opening new client INFO [2022-01-10 12:52:45,446] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0x2b94f89700000002","seedNodes":[{"address":"172.23.106.61"}]} INFO [2022-01-10 12:52:45,451] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000002","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 12:52:45,479] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0x2b94f89700000003","seedNodes":[{"address":"172.23.107.227"}]} INFO [2022-01-10 12:52:45,480] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000003","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 12:52:45,494] jepsen node 172.23.107.228 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 12:52:45,497] jepsen node 172.23.107.228 - couchbase.cbclients Opening new client INFO [2022-01-10 12:52:45,508] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0x2b94f89700000004","seedNodes":[{"address":"172.23.107.228"}]} INFO [2022-01-10 12:52:45,508] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000004","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 12:52:45,525] jepsen node 172.23.107.228 - couchbase.cbclients Checking value of 'com.couchbase.unorderedExecutionEnabled' : nil INFO [2022-01-10 12:52:45,582] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0x2b94f89700000005","seedNodes":[{"address":"172.23.107.228"}]} INFO [2022-01-10 12:52:45,583] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000005","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 12:52:45,610] cb-events - com.couchbase.core [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.3","clientGitHash":"${buildNumber}","coreVersion":"2.2.3","coreGitHash":"${buildNumber}","userAgent":"couchbase-java/3.2.3 (Linux 4.15.0-143-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_312-8u312-b07-0ubuntu1~18.04-b07)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"configIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2000,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryThresholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0x2b94f89700000006","seedNodes":[{"address":"172.23.107.227"}]} INFO [2022-01-10 12:52:45,611] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000006","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 12:52:46,081] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000003","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 12:52:46,082] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000003","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 12:52:46,084] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000005","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 12:52:46,084] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000005","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 12:52:46,111] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000004","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 12:52:46,112] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000004","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 12:52:46,112] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000006","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 12:52:46,112] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000004","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 12:52:46,112] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000006","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 12:52:46,159] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000002","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 12:52:46,159] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000002","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 12:52:46,163] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000002","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 12:52:46,163] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][781ms] Opened bucket "default" {"coreId":"0x2b94f89700000002"} INFO [2022-01-10 12:52:46,184] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][704ms] Opened bucket "default" {"coreId":"0x2b94f89700000003"} INFO [2022-01-10 12:52:46,185] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000005","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 12:52:46,185] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000003","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 12:52:46,185] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][610ms] Opened bucket "default" {"coreId":"0x2b94f89700000005"} INFO [2022-01-10 12:52:46,212] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][682ms] Opened bucket "default" {"coreId":"0x2b94f89700000004"} INFO [2022-01-10 12:52:46,213] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000006","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 12:52:46,219] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000001","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 12:52:46,220] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000001","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 12:52:46,220] cb-events - com.couchbase.node [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0x2b94f89700000001","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 12:52:46,221] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][909ms] Opened bucket "default" {"coreId":"0x2b94f89700000001"} INFO [2022-01-10 12:52:46,228] cb-events - com.couchbase.core [com.couchbase.core][BucketOpenedEvent][668ms] Opened bucket "default" {"coreId":"0x2b94f89700000006"} INFO [2022-01-10 12:52:50,733] jepsen worker 26 - jepsen.util 26 :invoke :write [8 26] INFO [2022-01-10 12:52:50,756] jepsen worker 28 - jepsen.util 28 :invoke :write [9 39] INFO [2022-01-10 12:52:50,774] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:50,790] jepsen worker 72 - jepsen.util 72 :invoke :write [24 8] INFO [2022-01-10 12:52:50,803] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:52:50,811] jepsen worker 14 - jepsen.util 14 :invoke :write [4 5] INFO [2022-01-10 12:52:50,819] jepsen worker 9 - jepsen.util 9 :invoke :write [3 23] INFO [2022-01-10 12:52:50,834] jepsen worker 6 - jepsen.util 6 :invoke :write [2 21] INFO [2022-01-10 12:52:50,844] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:50,848] jepsen worker 81 - jepsen.util 81 :ok :read [27 :nil] INFO [2022-01-10 12:52:50,850] jepsen worker 67 - jepsen.util 67 :ok :read [22 :nil] INFO [2022-01-10 12:52:50,851] jepsen worker 16 - jepsen.util 16 :ok :read [5 :nil] INFO [2022-01-10 12:52:50,858] jepsen worker 7 - jepsen.util 7 :invoke :write [2 7] INFO [2022-01-10 12:52:50,870] jepsen worker 23 - jepsen.util 23 :invoke :write [7 48] INFO [2022-01-10 12:52:50,877] jepsen worker 51 - jepsen.util 51 :invoke :write [17 25] INFO [2022-01-10 12:52:50,887] jepsen worker 73 - jepsen.util 73 :invoke :write [24 43] INFO [2022-01-10 12:52:50,894] jepsen worker 21 - jepsen.util 21 :invoke :write [7 36] INFO [2022-01-10 12:52:50,901] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:50,912] jepsen worker 11 - jepsen.util 11 :ok :read [3 :nil] INFO [2022-01-10 12:52:50,914] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:50,919] jepsen worker 85 - jepsen.util 85 :ok :read [28 :nil] INFO [2022-01-10 12:52:50,924] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:52:50,929] jepsen worker 86 - jepsen.util 86 :ok :read [28 :nil] INFO [2022-01-10 12:52:50,934] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:50,939] jepsen worker 60 - jepsen.util 60 :ok :read [20 :nil] INFO [2022-01-10 12:52:50,946] jepsen worker 13 - jepsen.util 13 :invoke :write [4 25] INFO [2022-01-10 12:52:50,955] jepsen worker 55 - jepsen.util 55 :invoke :write [18 17] INFO [2022-01-10 12:52:50,961] jepsen worker 22 - jepsen.util 22 :invoke :write [7 42] INFO [2022-01-10 12:52:50,967] jepsen worker 29 - jepsen.util 29 :invoke :write [9 0] INFO [2022-01-10 12:52:50,973] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:52:50,979] jepsen worker 54 - jepsen.util 54 :ok :read [18 :nil] INFO [2022-01-10 12:52:50,981] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:50,987] jepsen worker 0 - jepsen.util 0 :ok :read [0 :nil] INFO [2022-01-10 12:52:50,993] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:51,000] jepsen worker 4 - jepsen.util 4 :ok :read [1 :nil] INFO [2022-01-10 12:52:51,005] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:52:51,012] jepsen worker 53 - jepsen.util 53 :ok :read [17 :nil] INFO [2022-01-10 12:52:51,017] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:51,023] jepsen worker 83 - jepsen.util 83 :ok :read [27 :nil] INFO [2022-01-10 12:52:51,029] jepsen worker 71 - jepsen.util 71 :invoke :write [23 16] INFO [2022-01-10 12:52:51,036] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:51,041] jepsen worker 2 - jepsen.util 2 :ok :read [0 :nil] INFO [2022-01-10 12:52:51,044] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:52:51,050] jepsen worker 76 - jepsen.util 76 :ok :read [25 :nil] INFO [2022-01-10 12:52:51,053] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:51,059] jepsen worker 74 - jepsen.util 74 :ok :read [24 :nil] INFO [2022-01-10 12:52:51,062] jepsen worker 35 - jepsen.util 35 :invoke :write [11 32] INFO [2022-01-10 12:52:51,068] jepsen worker 69 - jepsen.util 69 :invoke :write [23 31] INFO [2022-01-10 12:52:51,075] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:51,081] jepsen worker 58 - jepsen.util 58 :ok :read [19 :nil] INFO [2022-01-10 12:52:51,084] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:51,090] jepsen worker 11 - jepsen.util 11 :ok :read [3 :nil] INFO [2022-01-10 12:52:51,093] jepsen worker 17 - jepsen.util 17 :invoke :write [5 15] INFO [2022-01-10 12:52:51,100] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:51,105] jepsen worker 74 - jepsen.util 74 :ok :read [24 :nil] INFO [2022-01-10 12:52:51,108] jepsen worker 11 - jepsen.util 11 :invoke :write [3 41] INFO [2022-01-10 12:52:51,114] jepsen worker 82 - jepsen.util 82 :invoke :write [27 13] INFO [2022-01-10 12:52:51,120] jepsen worker 48 - jepsen.util 48 :invoke :write [16 0] INFO [2022-01-10 12:52:51,127] jepsen worker 33 - jepsen.util 33 :invoke :write [11 22] INFO [2022-01-10 12:52:51,133] jepsen worker 78 - jepsen.util 78 :invoke :write [26 12] INFO [2022-01-10 12:52:51,139] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:51,145] jepsen worker 58 - jepsen.util 58 :ok :read [19 :nil] INFO [2022-01-10 12:52:51,146] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:51,150] jepsen worker 74 - jepsen.util 74 :ok :read [24 :nil] INFO [2022-01-10 12:52:51,151] jepsen worker 88 - jepsen.util 88 :invoke :write [29 15] INFO [2022-01-10 12:52:51,157] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:52:51,165] jepsen worker 30 - jepsen.util 30 :ok :read [10 :nil] INFO [2022-01-10 12:52:51,167] jepsen worker 12 - jepsen.util 12 :invoke :write [4 10] INFO [2022-01-10 12:52:51,171] jepsen worker 83 - jepsen.util 83 :invoke :write [27 28] INFO [2022-01-10 12:52:51,176] jepsen worker 52 - jepsen.util 52 :invoke :write [17 26] INFO [2022-01-10 12:52:51,181] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:51,186] jepsen worker 2 - jepsen.util 2 :ok :read [0 :nil] INFO [2022-01-10 12:52:51,188] jepsen worker 25 - jepsen.util 25 :invoke :write [8 0] INFO [2022-01-10 12:52:51,193] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:52:51,199] jepsen worker 1 - jepsen.util 1 :ok :read [0 :nil] INFO [2022-01-10 12:52:51,205] jepsen worker 81 - jepsen.util 81 :invoke :write [27 11] INFO [2022-01-10 12:52:51,211] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:52:51,217] jepsen worker 53 - jepsen.util 53 :ok :read [17 :nil] INFO [2022-01-10 12:52:51,222] jepsen worker 41 - jepsen.util 41 :invoke :write [13 39] INFO [2022-01-10 12:52:51,229] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 12:52:51,234] jepsen worker 80 - jepsen.util 80 :ok :read [26 :nil] INFO [2022-01-10 12:52:51,237] jepsen worker 24 - jepsen.util 24 :invoke :write [8 29] INFO [2022-01-10 12:52:51,246] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:52:51,251] jepsen worker 47 - jepsen.util 47 :ok :read [15 :nil] INFO [2022-01-10 12:52:51,256] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:52:51,262] jepsen worker 70 - jepsen.util 70 :ok :read [23 :nil] INFO [2022-01-10 12:52:51,266] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:52:51,271] jepsen worker 76 - jepsen.util 76 :ok :read [25 :nil] INFO [2022-01-10 12:52:51,275] jepsen worker 62 - jepsen.util 62 :invoke :write [20 46] INFO [2022-01-10 12:52:51,281] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:52:51,287] jepsen worker 66 - jepsen.util 66 :ok :read [22 :nil] INFO [2022-01-10 12:52:51,291] jepsen worker 10 - jepsen.util 10 :invoke :read [3 nil] INFO [2022-01-10 12:52:51,297] jepsen worker 10 - jepsen.util 10 :ok :read [3 :nil] INFO [2022-01-10 12:52:51,300] jepsen worker 20 - jepsen.util 20 :invoke :read [6 nil] INFO [2022-01-10 12:52:51,306] jepsen worker 20 - jepsen.util 20 :ok :read [6 :nil] INFO [2022-01-10 12:52:51,309] jepsen worker 85 - jepsen.util 85 :invoke :write [28 6] INFO [2022-01-10 12:52:51,314] jepsen worker 59 - jepsen.util 59 :invoke :write [19 31] INFO [2022-01-10 12:52:51,320] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:51,331] jepsen worker 84 - jepsen.util 84 :ok :read [28 :nil] INFO [2022-01-10 12:52:51,334] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:51,339] jepsen worker 50 - jepsen.util 50 :ok :read [16 :nil] INFO [2022-01-10 12:52:51,341] jepsen worker 79 - jepsen.util 79 :invoke :write [26 10] INFO [2022-01-10 12:52:51,346] jepsen worker 58 - jepsen.util 58 :invoke :write [19 26] INFO [2022-01-10 12:52:51,352] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:51,358] jepsen worker 89 - jepsen.util 89 :ok :read [29 :nil] INFO [2022-01-10 12:52:51,362] jepsen worker 4 - jepsen.util 4 :invoke :write [1 42] INFO [2022-01-10 12:52:51,368] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:52:51,373] jepsen worker 1 - jepsen.util 1 :ok :read [0 :nil] INFO [2022-01-10 12:52:51,377] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:52:51,382] jepsen worker 53 - jepsen.util 53 :ok :read [17 :nil] INFO [2022-01-10 12:52:51,386] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:51,392] jepsen worker 74 - jepsen.util 74 :ok :read [24 :nil] INFO [2022-01-10 12:52:51,394] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:52:51,399] jepsen worker 54 - jepsen.util 54 :ok :read [18 :nil] INFO [2022-01-10 12:52:51,400] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:51,405] jepsen worker 84 - jepsen.util 84 :ok :read [28 :nil] INFO [2022-01-10 12:52:51,406] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:52:51,410] jepsen worker 70 - jepsen.util 70 :ok :read [23 :nil] INFO [2022-01-10 12:52:51,412] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:51,419] jepsen worker 63 - jepsen.util 63 :ok :read [21 :nil] INFO [2022-01-10 12:52:51,420] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 12:52:51,423] jepsen worker 80 - jepsen.util 80 :ok :read [26 :nil] INFO [2022-01-10 12:52:51,424] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:52:51,427] jepsen worker 53 - jepsen.util 53 :ok :read [17 :nil] INFO [2022-01-10 12:52:51,428] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:51,430] jepsen worker 50 - jepsen.util 50 :ok :read [16 :nil] INFO [2022-01-10 12:52:51,434] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:52:51,437] jepsen worker 66 - jepsen.util 66 :ok :read [22 :nil] INFO [2022-01-10 12:52:51,438] jepsen worker 2 - jepsen.util 2 :invoke :write [0 21] INFO [2022-01-10 12:52:51,443] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:51,447] jepsen worker 84 - jepsen.util 84 :ok :read [28 :nil] INFO [2022-01-10 12:52:51,448] jepsen worker 86 - jepsen.util 86 :invoke :write [28 39] INFO [2022-01-10 12:52:51,455] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:52:51,458] jepsen worker 1 - jepsen.util 1 :ok :read [0 :nil] INFO [2022-01-10 12:52:51,460] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:52:51,463] jepsen worker 40 - jepsen.util 40 :ok :read [13 :nil] INFO [2022-01-10 12:52:51,466] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:51,469] jepsen worker 84 - jepsen.util 84 :ok :read [28 :nil] INFO [2022-01-10 12:52:51,471] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 12:52:51,474] jepsen worker 80 - jepsen.util 80 :ok :read [26 :nil] INFO [2022-01-10 12:52:51,478] jepsen worker 70 - jepsen.util 70 :invoke :write [23 22] INFO [2022-01-10 12:52:51,484] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:51,488] jepsen worker 31 - jepsen.util 31 :ok :read [10 :nil] INFO [2022-01-10 12:52:51,493] jepsen worker 3 - jepsen.util 3 :invoke :write [1 9] INFO [2022-01-10 12:52:51,502] jepsen worker 75 - jepsen.util 75 :invoke :write [25 20] INFO [2022-01-10 12:52:51,509] jepsen worker 40 - jepsen.util 40 :invoke :write [13 28] INFO [2022-01-10 12:52:51,516] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:51,519] jepsen worker 39 - jepsen.util 39 :ok :read [13 :nil] INFO [2022-01-10 12:52:51,522] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:52:51,525] jepsen worker 76 - jepsen.util 76 :ok :read [25 :nil] INFO [2022-01-10 12:52:51,528] jepsen worker 53 - jepsen.util 53 :invoke :write [17 17] INFO [2022-01-10 12:52:51,533] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:51,536] jepsen worker 63 - jepsen.util 63 :ok :read [21 :nil] INFO [2022-01-10 12:52:51,539] jepsen worker 46 - jepsen.util 46 :invoke :write [15 33] INFO [2022-01-10 12:52:51,544] jepsen worker 49 - jepsen.util 49 :invoke :write [16 4] INFO [2022-01-10 12:52:51,548] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:51,552] jepsen worker 0 - jepsen.util 0 :ok :read [0 :nil] INFO [2022-01-10 12:52:51,553] jepsen worker 84 - jepsen.util 84 :invoke :write [28 29] INFO [2022-01-10 12:52:51,558] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:52:51,561] jepsen worker 5 - jepsen.util 5 :ok :read [1 :nil] INFO [2022-01-10 12:52:51,562] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:51,565] jepsen worker 60 - jepsen.util 60 :ok :read [20 :nil] INFO [2022-01-10 12:52:51,567] jepsen worker 57 - jepsen.util 57 :invoke :write [19 21] INFO [2022-01-10 12:52:51,571] jepsen worker 8 - jepsen.util 8 :invoke :write [2 25] INFO [2022-01-10 12:52:51,574] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:52:51,577] jepsen worker 19 - jepsen.util 19 :ok :read [6 :nil] INFO [2022-01-10 12:52:51,577] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:51,580] jepsen worker 27 - jepsen.util 27 :ok :read [9 :nil] INFO [2022-01-10 12:52:51,582] jepsen worker 77 - jepsen.util 77 :invoke :write [25 0] INFO [2022-01-10 12:52:51,585] jepsen worker 5 - jepsen.util 5 :invoke :write [1 16] INFO [2022-01-10 12:52:51,588] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:51,590] jepsen worker 60 - jepsen.util 60 :ok :read [20 :nil] INFO [2022-01-10 12:52:51,592] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:52:51,595] jepsen worker 68 - jepsen.util 68 :ok :read [22 :nil] INFO [2022-01-10 12:52:51,595] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:51,598] jepsen worker 50 - jepsen.util 50 :ok :read [16 :nil] INFO [2022-01-10 12:52:51,598] jepsen worker 15 - jepsen.util 15 :invoke :write [5 10] INFO [2022-01-10 12:52:51,601] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:52:51,604] jepsen worker 65 - jepsen.util 65 :ok :read [21 :nil] INFO [2022-01-10 12:52:51,604] jepsen worker 44 - jepsen.util 44 :invoke :write [14 11] INFO [2022-01-10 12:52:51,608] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:51,611] jepsen worker 61 - jepsen.util 61 :ok :read [20 :nil] INFO [2022-01-10 12:52:51,613] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:52:51,617] jepsen worker 54 - jepsen.util 54 :ok :read [18 :nil] INFO [2022-01-10 12:52:51,618] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:52:51,621] jepsen worker 76 - jepsen.util 76 :ok :read [25 :nil] INFO [2022-01-10 12:52:51,624] jepsen worker 36 - jepsen.util 36 :invoke :write [12 40] INFO [2022-01-10 12:52:51,629] jepsen worker 67 - jepsen.util 67 :invoke :write [22 16] INFO [2022-01-10 12:52:51,633] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:52:51,636] jepsen worker 34 - jepsen.util 34 :ok :read [11 :nil] INFO [2022-01-10 12:52:51,637] jepsen worker 10 - jepsen.util 10 :invoke :write [3 43] INFO [2022-01-10 12:52:51,642] jepsen worker 42 - jepsen.util 42 :invoke :write [14 27] INFO [2022-01-10 12:52:51,646] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:51,650] jepsen worker 27 - jepsen.util 27 :ok :read [9 :nil] INFO [2022-01-10 12:52:51,652] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:51,656] jepsen worker 61 - jepsen.util 61 :ok :read [20 :nil] INFO [2022-01-10 12:52:51,658] jepsen worker 18 - jepsen.util 18 :invoke :write [6 12] INFO [2022-01-10 12:52:51,663] jepsen worker 34 - jepsen.util 34 :invoke :write [11 34] INFO [2022-01-10 12:52:51,666] jepsen worker 80 - jepsen.util 80 :invoke :write [26 36] INFO [2022-01-10 12:52:51,669] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:52:51,672] jepsen worker 54 - jepsen.util 54 :ok :read [18 :nil] INFO [2022-01-10 12:52:51,673] jepsen worker 1 - jepsen.util 1 :invoke :write [0 12] INFO [2022-01-10 12:52:51,678] jepsen worker 60 - jepsen.util 60 :invoke :write [20 28] INFO [2022-01-10 12:52:51,680] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:51,682] jepsen worker 39 - jepsen.util 39 :ok :read [13 :nil] INFO [2022-01-10 12:52:51,683] jepsen worker 43 - jepsen.util 43 :invoke :write [14 26] INFO [2022-01-10 12:52:51,685] jepsen worker 89 - jepsen.util 89 :invoke :write [29 43] INFO [2022-01-10 12:52:51,687] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:51,689] jepsen worker 31 - jepsen.util 31 :invoke :write [10 20] INFO [2022-01-10 12:52:51,690] jepsen worker 61 - jepsen.util 61 :ok :read [20 :nil] INFO [2022-01-10 12:52:51,692] jepsen worker 50 - jepsen.util 50 :invoke :write [16 23] INFO [2022-01-10 12:52:51,696] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:51,699] jepsen worker 39 - jepsen.util 39 :invoke :write [13 15] INFO [2022-01-10 12:52:51,699] jepsen worker 16 - jepsen.util 16 :ok :read [5 :nil] INFO [2022-01-10 12:52:51,701] jepsen worker 61 - jepsen.util 61 :invoke :write [20 11] INFO [2022-01-10 12:52:51,704] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 12:52:51,707] jepsen worker 54 - jepsen.util 54 :invoke :write [18 12] INFO [2022-01-10 12:52:51,708] jepsen worker 56 - jepsen.util 56 :ok :read [18 :nil] INFO [2022-01-10 12:52:51,709] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:51,712] jepsen worker 87 - jepsen.util 87 :ok :read [29 :nil] INFO [2022-01-10 12:52:51,714] jepsen worker 32 - jepsen.util 32 :invoke :write [10 49] INFO [2022-01-10 12:52:51,717] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:51,721] jepsen worker 87 - jepsen.util 87 :ok :read [29 :nil] INFO [2022-01-10 12:52:51,721] jepsen worker 76 - jepsen.util 76 :invoke :write [25 43] INFO [2022-01-10 12:52:51,723] jepsen worker 27 - jepsen.util 27 :invoke :write [9 0] INFO [2022-01-10 12:52:51,725] jepsen worker 74 - jepsen.util 74 :invoke :write [24 42] INFO [2022-01-10 12:52:51,728] jepsen worker 16 - jepsen.util 16 :invoke :write [5 2] INFO [2022-01-10 12:52:51,730] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:52:51,733] jepsen worker 19 - jepsen.util 19 :ok :read [6 :nil] INFO [2022-01-10 12:52:51,734] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:52:51,734] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:51,737] jepsen worker 64 - jepsen.util 64 :ok :read [21 :nil] INFO [2022-01-10 12:52:51,737] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:52:51,738] jepsen worker 63 - jepsen.util 63 :ok :read [21 :nil] INFO [2022-01-10 12:52:51,740] jepsen worker 37 - jepsen.util 37 :invoke :write [12 0] INFO [2022-01-10 12:52:51,741] jepsen worker 65 - jepsen.util 65 :ok :read [21 :nil] INFO [2022-01-10 12:52:51,741] jepsen worker 87 - jepsen.util 87 :invoke :write [29 21] INFO [2022-01-10 12:52:51,744] jepsen worker 20 - jepsen.util 20 :invoke :write [6 2] INFO [2022-01-10 12:52:51,746] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:52:51,748] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:52:51,749] jepsen worker 30 - jepsen.util 30 :ok :read [10 :nil] INFO [2022-01-10 12:52:51,749] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:52:51,750] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:51,751] jepsen worker 19 - jepsen.util 19 :ok :read [6 :nil] INFO [2022-01-10 12:52:51,752] jepsen worker 47 - jepsen.util 47 :ok :read [15 :nil] INFO [2022-01-10 12:52:51,752] jepsen worker 63 - jepsen.util 63 :invoke :write [21 25] INFO [2022-01-10 12:52:51,754] jepsen worker 30 - jepsen.util 30 :invoke :write [10 23] INFO [2022-01-10 12:52:51,755] jepsen worker 38 - jepsen.util 38 :ok :read [12 :nil] INFO [2022-01-10 12:52:51,756] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:52:51,758] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:52:51,758] jepsen worker 65 - jepsen.util 65 :ok :read [21 :nil] INFO [2022-01-10 12:52:51,759] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:51,760] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:51,761] jepsen worker 64 - jepsen.util 64 :ok :read [21 :nil] INFO [2022-01-10 12:52:51,761] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:52:51,762] jepsen worker 0 - jepsen.util 0 :ok :read [0 :nil] INFO [2022-01-10 12:52:51,764] jepsen worker 68 - jepsen.util 68 :ok :read [22 :nil] INFO [2022-01-10 12:52:51,764] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:52:51,765] jepsen worker 38 - jepsen.util 38 :ok :read [12 :nil] INFO [2022-01-10 12:52:51,765] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:52:51,767] jepsen worker 65 - jepsen.util 65 :invoke :write [21 37] INFO [2022-01-10 12:52:51,768] jepsen worker 47 - jepsen.util 47 :ok :read [15 :nil] INFO [2022-01-10 12:52:51,768] jepsen worker 64 - jepsen.util 64 :ok :read [21 :nil] INFO [2022-01-10 12:52:51,769] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:51,770] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:52:51,771] jepsen worker 56 - jepsen.util 56 :invoke :write [18 25] INFO [2022-01-10 12:52:51,772] jepsen worker 45 - jepsen.util 45 :ok :read [15 :nil] INFO [2022-01-10 12:52:51,773] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:52:51,774] jepsen worker 64 - jepsen.util 64 :ok :read [21 :nil] INFO [2022-01-10 12:52:51,775] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:52:51,777] jepsen worker 68 - jepsen.util 68 :invoke :write [22 8] INFO [2022-01-10 12:52:51,777] jepsen worker 47 - jepsen.util 47 :ok :read [15 :nil] INFO [2022-01-10 12:52:51,778] jepsen worker 19 - jepsen.util 19 :ok :read [6 :nil] INFO [2022-01-10 12:52:51,779] jepsen worker 19 - jepsen.util 19 :invoke :write [6 21] INFO [2022-01-10 12:52:51,779] jepsen worker 66 - jepsen.util 66 :invoke :write [22 19] INFO [2022-01-10 12:52:51,780] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:51,781] jepsen worker 47 - jepsen.util 47 :invoke :write [15 9] INFO [2022-01-10 12:52:51,782] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:52:51,783] jepsen worker 38 - jepsen.util 38 :ok :read [12 :nil] INFO [2022-01-10 12:52:51,783] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:51,785] jepsen worker 0 - jepsen.util 0 :invoke :write [0 30] INFO [2022-01-10 12:52:51,786] jepsen worker 45 - jepsen.util 45 :ok :read [15 :nil] INFO [2022-01-10 12:52:51,786] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:51,786] jepsen worker 64 - jepsen.util 64 :ok :read [21 :nil] INFO [2022-01-10 12:52:51,788] jepsen worker 64 - jepsen.util 64 :invoke :write [21 4] INFO [2022-01-10 12:52:51,788] jepsen worker 38 - jepsen.util 38 :ok :read [12 :nil] INFO [2022-01-10 12:52:51,789] jepsen worker 45 - jepsen.util 45 :invoke :write [15 34] INFO [2022-01-10 12:52:51,791] jepsen worker 38 - jepsen.util 38 :invoke :write [12 40] INFO [2022-01-10 12:52:51,856] jepsen worker 14 - jepsen.util 14 :ok :write [4 5] INFO [2022-01-10 12:52:51,857] jepsen worker 14 - jepsen.util 14 :invoke :write [4 22] INFO [2022-01-10 12:52:51,860] jepsen worker 71 - jepsen.util 71 :ok :write [23 16] INFO [2022-01-10 12:52:51,862] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:51,865] jepsen worker 78 - jepsen.util 78 :ok :write [26 12] INFO [2022-01-10 12:52:51,866] jepsen worker 78 - jepsen.util 78 :invoke :write [26 35] INFO [2022-01-10 12:52:51,867] jepsen worker 71 - jepsen.util 71 :ok :read [23 16] INFO [2022-01-10 12:52:51,868] jepsen worker 71 - jepsen.util 71 :invoke :write [23 33] INFO [2022-01-10 12:52:52,219] jepsen worker 4 - jepsen.util 4 :ok :write [1 42] INFO [2022-01-10 12:52:52,221] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:52,224] jepsen worker 4 - jepsen.util 4 :ok :read [1 42] INFO [2022-01-10 12:52:52,224] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:52,227] jepsen worker 4 - jepsen.util 4 :ok :read [1 42] INFO [2022-01-10 12:52:52,227] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:52,229] jepsen worker 4 - jepsen.util 4 :ok :read [1 42] INFO [2022-01-10 12:52:52,230] jepsen worker 4 - jepsen.util 4 :invoke :write [1 9] INFO [2022-01-10 12:52:52,266] jepsen worker 36 - jepsen.util 36 :ok :write [12 40] INFO [2022-01-10 12:52:52,266] jepsen worker 36 - jepsen.util 36 :invoke :write [12 43] INFO [2022-01-10 12:52:52,424] jepsen worker 31 - jepsen.util 31 :ok :write [10 20] INFO [2022-01-10 12:52:52,426] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:52,428] jepsen worker 31 - jepsen.util 31 :ok :read [10 20] INFO [2022-01-10 12:52:52,429] jepsen worker 31 - jepsen.util 31 :invoke :write [10 9] INFO [2022-01-10 12:52:52,483] jepsen worker 9 - jepsen.util 9 :ok :write [3 23] INFO [2022-01-10 12:52:52,484] jepsen worker 9 - jepsen.util 9 :invoke :write [3 0] INFO [2022-01-10 12:52:52,751] jepsen worker 17 - jepsen.util 17 :ok :write [5 15] INFO [2022-01-10 12:52:52,752] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 12:52:52,754] jepsen worker 17 - jepsen.util 17 :ok :read [5 15] INFO [2022-01-10 12:52:52,756] jepsen worker 17 - jepsen.util 17 :invoke :write [5 10] INFO [2022-01-10 12:52:52,757] jepsen worker 82 - jepsen.util 82 :ok :write [27 13] INFO [2022-01-10 12:52:52,759] jepsen worker 82 - jepsen.util 82 :invoke :write [27 45] INFO [2022-01-10 12:52:52,767] jepsen worker 48 - jepsen.util 48 :ok :write [16 0] INFO [2022-01-10 12:52:52,768] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:52,769] jepsen worker 48 - jepsen.util 48 :ok :read [16 0] INFO [2022-01-10 12:52:52,770] jepsen worker 48 - jepsen.util 48 :invoke :write [16 13] INFO [2022-01-10 12:52:52,772] jepsen worker 85 - jepsen.util 85 :ok :write [28 6] INFO [2022-01-10 12:52:52,773] jepsen worker 88 - jepsen.util 88 :ok :write [29 15] INFO [2022-01-10 12:52:52,773] jepsen worker 85 - jepsen.util 85 :invoke :write [28 43] INFO [2022-01-10 12:52:52,774] jepsen worker 88 - jepsen.util 88 :invoke :write [29 12] INFO [2022-01-10 12:52:52,782] jepsen worker 41 - jepsen.util 41 :ok :write [13 39] INFO [2022-01-10 12:52:52,783] jepsen worker 41 - jepsen.util 41 :invoke :write [13 25] INFO [2022-01-10 12:52:52,791] jepsen worker 4 - jepsen.util 4 :ok :write [1 9] INFO [2022-01-10 12:52:52,792] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:52,793] jepsen worker 2 - jepsen.util 2 :ok :write [0 21] INFO [2022-01-10 12:52:52,794] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:52,796] jepsen worker 2 - jepsen.util 2 :ok :read [0 21] INFO [2022-01-10 12:52:52,797] jepsen worker 2 - jepsen.util 2 :invoke :write [0 40] INFO [2022-01-10 12:52:52,799] jepsen worker 36 - jepsen.util 36 :ok :write [12 43] INFO [2022-01-10 12:52:52,800] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:52,803] jepsen worker 85 - jepsen.util 85 :ok :write [28 43] INFO [2022-01-10 12:52:52,804] jepsen worker 85 - jepsen.util 85 :invoke :write [28 26] INFO [2022-01-10 12:52:52,806] jepsen worker 4 - jepsen.util 4 :ok :read [1 9] INFO [2022-01-10 12:52:52,807] jepsen worker 36 - jepsen.util 36 :ok :read [12 43] INFO [2022-01-10 12:52:52,808] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:52,809] jepsen worker 36 - jepsen.util 36 :invoke :write [12 7] INFO [2022-01-10 12:52:52,810] jepsen worker 4 - jepsen.util 4 :ok :read [1 9] INFO [2022-01-10 12:52:52,811] jepsen worker 4 - jepsen.util 4 :invoke :write [1 11] INFO [2022-01-10 12:52:52,826] jepsen worker 85 - jepsen.util 85 :ok :write [28 26] INFO [2022-01-10 12:52:52,827] jepsen worker 89 - jepsen.util 89 :ok :write [29 43] INFO [2022-01-10 12:52:52,827] jepsen worker 85 - jepsen.util 85 :invoke :write [28 36] INFO [2022-01-10 12:52:52,828] jepsen worker 89 - jepsen.util 89 :invoke :write [29 12] INFO [2022-01-10 12:52:52,834] jepsen worker 39 - jepsen.util 39 :ok :write [13 15] INFO [2022-01-10 12:52:52,835] jepsen worker 39 - jepsen.util 39 :invoke :write [13 46] INFO [2022-01-10 12:52:52,835] jepsen worker 36 - jepsen.util 36 :ok :write [12 7] INFO [2022-01-10 12:52:52,837] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:52,839] jepsen worker 36 - jepsen.util 36 :ok :read [12 7] INFO [2022-01-10 12:52:52,840] jepsen worker 36 - jepsen.util 36 :invoke :write [12 9] INFO [2022-01-10 12:52:52,841] jepsen worker 4 - jepsen.util 4 :ok :write [1 11] INFO [2022-01-10 12:52:52,842] jepsen worker 2 - jepsen.util 2 :ok :write [0 40] INFO [2022-01-10 12:52:52,842] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:52,843] jepsen worker 2 - jepsen.util 2 :invoke :write [0 35] INFO [2022-01-10 12:52:52,844] jepsen worker 4 - jepsen.util 4 :ok :read [1 11] INFO [2022-01-10 12:52:52,845] jepsen worker 4 - jepsen.util 4 :invoke :write [1 10] INFO [2022-01-10 12:52:52,849] jepsen worker 89 - jepsen.util 89 :ok :write [29 12] INFO [2022-01-10 12:52:52,850] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:52,851] jepsen worker 89 - jepsen.util 89 :ok :read [29 12] INFO [2022-01-10 12:52:52,852] jepsen worker 89 - jepsen.util 89 :invoke :write [29 29] INFO [2022-01-10 12:52:52,854] jepsen worker 36 - jepsen.util 36 :ok :write [12 9] INFO [2022-01-10 12:52:52,855] jepsen worker 39 - jepsen.util 39 :ok :write [13 46] INFO [2022-01-10 12:52:52,856] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:52,856] jepsen worker 39 - jepsen.util 39 :invoke :write [13 32] INFO [2022-01-10 12:52:52,857] jepsen worker 85 - jepsen.util 85 :ok :write [28 36] INFO [2022-01-10 12:52:52,857] jepsen worker 36 - jepsen.util 36 :ok :read [12 9] INFO [2022-01-10 12:52:52,858] jepsen worker 85 - jepsen.util 85 :invoke :write [28 5] INFO [2022-01-10 12:52:52,859] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:52,860] jepsen worker 36 - jepsen.util 36 :ok :read [12 9] INFO [2022-01-10 12:52:52,862] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:52,862] jepsen worker 4 - jepsen.util 4 :ok :write [1 10] INFO [2022-01-10 12:52:52,863] jepsen worker 4 - jepsen.util 4 :invoke :write [1 1] INFO [2022-01-10 12:52:52,863] jepsen worker 36 - jepsen.util 36 :ok :read [12 9] INFO [2022-01-10 12:52:52,864] jepsen worker 2 - jepsen.util 2 :ok :write [0 35] INFO [2022-01-10 12:52:52,864] jepsen worker 36 - jepsen.util 36 :invoke :write [12 37] INFO [2022-01-10 12:52:52,865] jepsen worker 2 - jepsen.util 2 :invoke :write [0 23] INFO [2022-01-10 12:52:52,872] jepsen worker 89 - jepsen.util 89 :ok :write [29 29] INFO [2022-01-10 12:52:52,873] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:52,875] jepsen worker 89 - jepsen.util 89 :ok :read [29 29] INFO [2022-01-10 12:52:52,876] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:52,878] jepsen worker 89 - jepsen.util 89 :ok :read [29 29] INFO [2022-01-10 12:52:52,879] jepsen worker 39 - jepsen.util 39 :ok :write [13 32] INFO [2022-01-10 12:52:52,880] jepsen worker 89 - jepsen.util 89 :invoke :write [29 28] INFO [2022-01-10 12:52:52,880] jepsen worker 85 - jepsen.util 85 :ok :write [28 5] INFO [2022-01-10 12:52:52,881] jepsen worker 39 - jepsen.util 39 :invoke :write [13 48] INFO [2022-01-10 12:52:52,882] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:52,885] jepsen worker 85 - jepsen.util 85 :ok :read [28 5] INFO [2022-01-10 12:52:52,885] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:52,887] jepsen worker 85 - jepsen.util 85 :ok :read [28 5] INFO [2022-01-10 12:52:52,888] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:52,888] jepsen worker 4 - jepsen.util 4 :ok :write [1 1] INFO [2022-01-10 12:52:52,889] jepsen worker 2 - jepsen.util 2 :ok :write [0 23] INFO [2022-01-10 12:52:52,889] jepsen worker 4 - jepsen.util 4 :invoke :write [1 19] INFO [2022-01-10 12:52:52,890] jepsen worker 85 - jepsen.util 85 :ok :read [28 5] INFO [2022-01-10 12:52:52,890] jepsen worker 2 - jepsen.util 2 :invoke :write [0 43] INFO [2022-01-10 12:52:52,891] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:52,892] jepsen worker 85 - jepsen.util 85 :ok :read [28 5] INFO [2022-01-10 12:52:52,893] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:52,895] jepsen worker 85 - jepsen.util 85 :ok :read [28 5] INFO [2022-01-10 12:52:52,896] jepsen worker 85 - jepsen.util 85 :invoke :write [28 32] INFO [2022-01-10 12:52:52,939] jepsen worker 67 - jepsen.util 67 :ok :write [22 16] INFO [2022-01-10 12:52:52,940] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:52,942] jepsen worker 67 - jepsen.util 67 :ok :read [22 16] INFO [2022-01-10 12:52:52,943] jepsen worker 67 - jepsen.util 67 :invoke :write [22 10] INFO [2022-01-10 12:52:52,998] jepsen worker 36 - jepsen.util 36 :ok :write [12 37] INFO [2022-01-10 12:52:53,000] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,001] jepsen worker 36 - jepsen.util 36 :ok :read [12 37] INFO [2022-01-10 12:52:53,002] jepsen worker 39 - jepsen.util 39 :ok :write [13 48] INFO [2022-01-10 12:52:53,003] jepsen worker 85 - jepsen.util 85 :ok :write [28 32] INFO [2022-01-10 12:52:53,003] jepsen worker 36 - jepsen.util 36 :invoke :write [12 29] INFO [2022-01-10 12:52:53,004] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:53,005] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:53,007] jepsen worker 85 - jepsen.util 85 :ok :read [28 32] INFO [2022-01-10 12:52:53,007] jepsen worker 39 - jepsen.util 39 :ok :read [13 48] INFO [2022-01-10 12:52:53,008] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:53,009] jepsen worker 39 - jepsen.util 39 :invoke :write [13 1] INFO [2022-01-10 12:52:53,010] jepsen worker 89 - jepsen.util 89 :ok :write [29 28] INFO [2022-01-10 12:52:53,010] jepsen worker 4 - jepsen.util 4 :ok :write [1 19] INFO [2022-01-10 12:52:53,011] jepsen worker 85 - jepsen.util 85 :ok :read [28 32] INFO [2022-01-10 12:52:53,012] jepsen worker 89 - jepsen.util 89 :invoke :write [29 6] INFO [2022-01-10 12:52:53,015] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:53,016] jepsen worker 85 - jepsen.util 85 :invoke :write [28 33] INFO [2022-01-10 12:52:53,017] jepsen worker 4 - jepsen.util 4 :ok :read [1 19] INFO [2022-01-10 12:52:53,018] jepsen worker 36 - jepsen.util 36 :ok :write [12 29] INFO [2022-01-10 12:52:53,019] jepsen worker 4 - jepsen.util 4 :invoke :write [1 5] INFO [2022-01-10 12:52:53,020] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,023] jepsen worker 36 - jepsen.util 36 :ok :read [12 29] INFO [2022-01-10 12:52:53,024] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,025] jepsen worker 2 - jepsen.util 2 :ok :write [0 43] INFO [2022-01-10 12:52:53,025] jepsen worker 36 - jepsen.util 36 :ok :read [12 29] INFO [2022-01-10 12:52:53,026] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,026] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,027] jepsen worker 2 - jepsen.util 2 :ok :read [0 43] INFO [2022-01-10 12:52:53,028] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,028] jepsen worker 36 - jepsen.util 36 :ok :read [12 29] INFO [2022-01-10 12:52:53,029] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,030] jepsen worker 2 - jepsen.util 2 :ok :read [0 43] INFO [2022-01-10 12:52:53,031] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,031] jepsen worker 36 - jepsen.util 36 :ok :read [12 29] INFO [2022-01-10 12:52:53,032] jepsen worker 2 - jepsen.util 2 :ok :read [0 43] INFO [2022-01-10 12:52:53,032] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,033] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,034] jepsen worker 36 - jepsen.util 36 :ok :read [12 29] INFO [2022-01-10 12:52:53,035] jepsen worker 36 - jepsen.util 36 :invoke :write [12 23] INFO [2022-01-10 12:52:53,035] jepsen worker 2 - jepsen.util 2 :ok :read [0 43] INFO [2022-01-10 12:52:53,037] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,048] jepsen worker 89 - jepsen.util 89 :ok :write [29 6] INFO [2022-01-10 12:52:53,049] jepsen worker 2 - jepsen.util 2 :ok :read [0 43] INFO [2022-01-10 12:52:53,049] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,051] jepsen worker 89 - jepsen.util 89 :ok :read [29 6] INFO [2022-01-10 12:52:53,052] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,052] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,053] jepsen worker 39 - jepsen.util 39 :ok :write [13 1] INFO [2022-01-10 12:52:53,054] jepsen worker 89 - jepsen.util 89 :ok :read [29 6] INFO [2022-01-10 12:52:53,055] jepsen worker 39 - jepsen.util 39 :invoke :write [13 30] INFO [2022-01-10 12:52:53,056] jepsen worker 89 - jepsen.util 89 :invoke :write [29 27] INFO [2022-01-10 12:52:53,056] jepsen worker 85 - jepsen.util 85 :ok :write [28 33] INFO [2022-01-10 12:52:53,058] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:53,059] jepsen worker 2 - jepsen.util 2 :ok :read [0 43] INFO [2022-01-10 12:52:53,059] jepsen worker 85 - jepsen.util 85 :ok :read [28 33] INFO [2022-01-10 12:52:53,060] jepsen worker 2 - jepsen.util 2 :invoke :write [0 19] INFO [2022-01-10 12:52:53,061] jepsen worker 85 - jepsen.util 85 :invoke :write [28 28] INFO [2022-01-10 12:52:53,065] jepsen worker 28 - jepsen.util 28 :ok :write [9 39] INFO [2022-01-10 12:52:53,065] jepsen worker 4 - jepsen.util 4 :ok :write [1 5] INFO [2022-01-10 12:52:53,067] jepsen worker 28 - jepsen.util 28 :invoke :read [9 nil] INFO [2022-01-10 12:52:53,069] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:53,071] jepsen worker 4 - jepsen.util 4 :ok :read [1 5] INFO [2022-01-10 12:52:53,073] jepsen worker 4 - jepsen.util 4 :invoke :write [1 46] INFO [2022-01-10 12:52:53,074] jepsen worker 36 - jepsen.util 36 :ok :write [12 23] INFO [2022-01-10 12:52:53,075] jepsen worker 36 - jepsen.util 36 :invoke :write [12 24] INFO [2022-01-10 12:52:53,084] jepsen worker 85 - jepsen.util 85 :ok :write [28 28] INFO [2022-01-10 12:52:53,085] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:53,087] jepsen worker 85 - jepsen.util 85 :ok :read [28 28] INFO [2022-01-10 12:52:53,088] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:53,088] jepsen worker 39 - jepsen.util 39 :ok :write [13 30] INFO [2022-01-10 12:52:53,089] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:53,089] jepsen worker 85 - jepsen.util 85 :ok :read [28 28] INFO [2022-01-10 12:52:53,091] jepsen worker 85 - jepsen.util 85 :invoke :write [28 38] INFO [2022-01-10 12:52:53,091] jepsen worker 39 - jepsen.util 39 :ok :read [13 30] INFO [2022-01-10 12:52:53,092] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:53,094] jepsen worker 39 - jepsen.util 39 :ok :read [13 30] INFO [2022-01-10 12:52:53,095] jepsen worker 36 - jepsen.util 36 :ok :write [12 24] INFO [2022-01-10 12:52:53,095] jepsen worker 39 - jepsen.util 39 :invoke :write [13 23] INFO [2022-01-10 12:52:53,097] jepsen worker 36 - jepsen.util 36 :invoke :write [12 6] INFO [2022-01-10 12:52:53,097] jepsen worker 89 - jepsen.util 89 :ok :write [29 27] INFO [2022-01-10 12:52:53,098] jepsen worker 4 - jepsen.util 4 :ok :write [1 46] INFO [2022-01-10 12:52:53,098] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,100] jepsen worker 89 - jepsen.util 89 :ok :read [29 27] INFO [2022-01-10 12:52:53,101] jepsen worker 4 - jepsen.util 4 :invoke :write [1 18] INFO [2022-01-10 12:52:53,102] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,102] jepsen worker 85 - jepsen.util 85 :ok :write [28 38] INFO [2022-01-10 12:52:53,103] jepsen worker 2 - jepsen.util 2 :ok :write [0 19] INFO [2022-01-10 12:52:53,104] jepsen worker 85 - jepsen.util 85 :invoke :write [28 43] INFO [2022-01-10 12:52:53,104] jepsen worker 89 - jepsen.util 89 :ok :read [29 27] INFO [2022-01-10 12:52:53,105] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,106] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,107] jepsen worker 2 - jepsen.util 2 :ok :read [0 19] INFO [2022-01-10 12:52:53,108] jepsen worker 89 - jepsen.util 89 :ok :read [29 27] INFO [2022-01-10 12:52:53,109] jepsen worker 2 - jepsen.util 2 :invoke :write [0 31] INFO [2022-01-10 12:52:53,110] jepsen worker 36 - jepsen.util 36 :ok :write [12 6] INFO [2022-01-10 12:52:53,110] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,111] jepsen worker 36 - jepsen.util 36 :invoke :write [12 22] INFO [2022-01-10 12:52:53,112] jepsen worker 89 - jepsen.util 89 :ok :read [29 27] INFO [2022-01-10 12:52:53,113] jepsen worker 41 - jepsen.util 41 :ok :write [13 25] INFO [2022-01-10 12:52:53,113] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,114] jepsen worker 41 - jepsen.util 41 :invoke :write [13 24] INFO [2022-01-10 12:52:53,115] jepsen worker 89 - jepsen.util 89 :ok :read [29 27] INFO [2022-01-10 12:52:53,117] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,117] jepsen worker 4 - jepsen.util 4 :ok :write [1 18] INFO [2022-01-10 12:52:53,118] jepsen worker 4 - jepsen.util 4 :invoke :write [1 5] INFO [2022-01-10 12:52:53,118] jepsen worker 89 - jepsen.util 89 :ok :read [29 27] INFO [2022-01-10 12:52:53,120] jepsen worker 89 - jepsen.util 89 :invoke :write [29 43] INFO [2022-01-10 12:52:53,125] jepsen worker 2 - jepsen.util 2 :ok :write [0 31] INFO [2022-01-10 12:52:53,126] jepsen worker 85 - jepsen.util 85 :ok :write [28 43] INFO [2022-01-10 12:52:53,127] jepsen worker 85 - jepsen.util 85 :invoke :write [28 16] INFO [2022-01-10 12:52:53,128] jepsen worker 2 - jepsen.util 2 :invoke :write [0 4] INFO [2022-01-10 12:52:53,129] jepsen worker 36 - jepsen.util 36 :ok :write [12 22] INFO [2022-01-10 12:52:53,131] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,132] jepsen worker 36 - jepsen.util 36 :ok :read [12 22] INFO [2022-01-10 12:52:53,133] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,135] jepsen worker 36 - jepsen.util 36 :ok :read [12 22] INFO [2022-01-10 12:52:53,136] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,137] jepsen worker 36 - jepsen.util 36 :ok :read [12 22] INFO [2022-01-10 12:52:53,139] jepsen worker 36 - jepsen.util 36 :invoke :write [12 46] INFO [2022-01-10 12:52:53,195] jepsen worker 39 - jepsen.util 39 :ok :write [13 23] INFO [2022-01-10 12:52:53,195] jepsen worker 4 - jepsen.util 4 :ok :write [1 5] INFO [2022-01-10 12:52:53,196] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:53,198] jepsen worker 4 - jepsen.util 4 :ok :read [1 5] INFO [2022-01-10 12:52:53,199] jepsen worker 39 - jepsen.util 39 :invoke :write [13 7] INFO [2022-01-10 12:52:53,199] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:53,201] jepsen worker 4 - jepsen.util 4 :ok :read [1 5] INFO [2022-01-10 12:52:53,201] jepsen worker 85 - jepsen.util 85 :ok :write [28 16] INFO [2022-01-10 12:52:53,202] jepsen worker 89 - jepsen.util 89 :ok :write [29 43] INFO [2022-01-10 12:52:53,203] jepsen worker 4 - jepsen.util 4 :invoke :write [1 1] INFO [2022-01-10 12:52:53,204] jepsen worker 85 - jepsen.util 85 :invoke :write [28 40] INFO [2022-01-10 12:52:53,205] jepsen worker 89 - jepsen.util 89 :invoke :write [29 46] INFO [2022-01-10 12:52:53,210] jepsen worker 2 - jepsen.util 2 :ok :write [0 4] INFO [2022-01-10 12:52:53,211] jepsen worker 36 - jepsen.util 36 :ok :write [12 46] INFO [2022-01-10 12:52:53,212] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,212] jepsen worker 2 - jepsen.util 2 :invoke :write [0 26] INFO [2022-01-10 12:52:53,213] jepsen worker 36 - jepsen.util 36 :ok :read [12 46] INFO [2022-01-10 12:52:53,215] jepsen worker 36 - jepsen.util 36 :invoke :write [12 46] INFO [2022-01-10 12:52:53,218] jepsen worker 39 - jepsen.util 39 :ok :write [13 7] INFO [2022-01-10 12:52:53,219] jepsen worker 39 - jepsen.util 39 :invoke :write [13 12] INFO [2022-01-10 12:52:53,223] jepsen worker 85 - jepsen.util 85 :ok :write [28 40] INFO [2022-01-10 12:52:53,225] jepsen worker 4 - jepsen.util 4 :ok :write [1 1] INFO [2022-01-10 12:52:53,225] jepsen worker 85 - jepsen.util 85 :invoke :write [28 17] INFO [2022-01-10 12:52:53,227] jepsen worker 89 - jepsen.util 89 :ok :write [29 46] INFO [2022-01-10 12:52:53,227] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:53,228] jepsen worker 89 - jepsen.util 89 :invoke :write [29 21] INFO [2022-01-10 12:52:53,229] jepsen worker 4 - jepsen.util 4 :ok :read [1 1] INFO [2022-01-10 12:52:53,230] jepsen worker 4 - jepsen.util 4 :invoke :write [1 23] INFO [2022-01-10 12:52:53,231] jepsen worker 36 - jepsen.util 36 :ok :write [12 46] INFO [2022-01-10 12:52:53,232] jepsen worker 36 - jepsen.util 36 :invoke :write [12 22] INFO [2022-01-10 12:52:53,235] jepsen worker 2 - jepsen.util 2 :ok :write [0 26] INFO [2022-01-10 12:52:53,236] jepsen worker 2 - jepsen.util 2 :invoke :write [0 30] INFO [2022-01-10 12:52:53,245] jepsen worker 85 - jepsen.util 85 :ok :write [28 17] INFO [2022-01-10 12:52:53,246] jepsen worker 85 - jepsen.util 85 :invoke :write [28 34] INFO [2022-01-10 12:52:53,358] jepsen worker 72 - jepsen.util 72 :ok :write [24 8] INFO [2022-01-10 12:52:53,359] jepsen worker 72 - jepsen.util 72 :invoke :write [24 21] INFO [2022-01-10 12:52:53,537] jepsen worker 4 - jepsen.util 4 :ok :write [1 23] INFO [2022-01-10 12:52:53,538] jepsen worker 41 - jepsen.util 41 :ok :write [13 24] INFO [2022-01-10 12:52:53,539] jepsen worker 4 - jepsen.util 4 :invoke :write [1 32] INFO [2022-01-10 12:52:53,540] jepsen worker 41 - jepsen.util 41 :invoke :write [13 42] INFO [2022-01-10 12:52:53,543] jepsen worker 36 - jepsen.util 36 :ok :write [12 22] INFO [2022-01-10 12:52:53,544] jepsen worker 89 - jepsen.util 89 :ok :write [29 21] INFO [2022-01-10 12:52:53,544] jepsen worker 36 - jepsen.util 36 :invoke :write [12 18] INFO [2022-01-10 12:52:53,546] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,548] jepsen worker 89 - jepsen.util 89 :ok :read [29 21] INFO [2022-01-10 12:52:53,548] jepsen worker 85 - jepsen.util 85 :ok :write [28 34] INFO [2022-01-10 12:52:53,553] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,554] jepsen worker 4 - jepsen.util 4 :ok :write [1 32] INFO [2022-01-10 12:52:53,554] jepsen worker 89 - jepsen.util 89 :ok :read [29 21] INFO [2022-01-10 12:52:53,555] jepsen worker 2 - jepsen.util 2 :ok :write [0 30] INFO [2022-01-10 12:52:53,555] jepsen worker 85 - jepsen.util 85 :invoke :write [28 47] INFO [2022-01-10 12:52:53,557] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:53,558] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,558] jepsen worker 41 - jepsen.util 41 :ok :write [13 42] INFO [2022-01-10 12:52:53,558] jepsen worker 4 - jepsen.util 4 :ok :read [1 32] INFO [2022-01-10 12:52:53,559] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,559] jepsen worker 2 - jepsen.util 2 :ok :read [0 30] INFO [2022-01-10 12:52:53,560] jepsen worker 36 - jepsen.util 36 :ok :write [12 18] INFO [2022-01-10 12:52:53,560] jepsen worker 89 - jepsen.util 89 :ok :read [29 21] INFO [2022-01-10 12:52:53,562] jepsen worker 41 - jepsen.util 41 :invoke :write [13 41] INFO [2022-01-10 12:52:53,563] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,564] jepsen worker 89 - jepsen.util 89 :invoke :write [29 8] INFO [2022-01-10 12:52:53,565] jepsen worker 2 - jepsen.util 2 :ok :read [0 30] INFO [2022-01-10 12:52:53,567] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:53,568] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,568] jepsen worker 4 - jepsen.util 4 :ok :read [1 32] INFO [2022-01-10 12:52:53,569] jepsen worker 2 - jepsen.util 2 :invoke :write [0 44] INFO [2022-01-10 12:52:53,570] jepsen worker 36 - jepsen.util 36 :ok :read [12 18] INFO [2022-01-10 12:52:53,570] jepsen worker 4 - jepsen.util 4 :invoke :write [1 41] INFO [2022-01-10 12:52:53,571] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,572] jepsen worker 36 - jepsen.util 36 :ok :read [12 18] INFO [2022-01-10 12:52:53,573] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,574] jepsen worker 85 - jepsen.util 85 :ok :write [28 47] INFO [2022-01-10 12:52:53,575] jepsen worker 85 - jepsen.util 85 :invoke :write [28 9] INFO [2022-01-10 12:52:53,575] jepsen worker 36 - jepsen.util 36 :ok :read [12 18] INFO [2022-01-10 12:52:53,575] jepsen worker 41 - jepsen.util 41 :ok :write [13 41] INFO [2022-01-10 12:52:53,576] jepsen worker 41 - jepsen.util 41 :invoke :write [13 30] INFO [2022-01-10 12:52:53,577] jepsen worker 63 - jepsen.util 63 :ok :write [21 25] INFO [2022-01-10 12:52:53,578] jepsen worker 36 - jepsen.util 36 :invoke :write [12 35] INFO [2022-01-10 12:52:53,579] jepsen worker 63 - jepsen.util 63 :invoke :write [21 6] INFO [2022-01-10 12:52:53,583] jepsen worker 4 - jepsen.util 4 :ok :write [1 41] INFO [2022-01-10 12:52:53,583] jepsen worker 89 - jepsen.util 89 :ok :write [29 8] INFO [2022-01-10 12:52:53,585] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:53,586] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,586] jepsen worker 4 - jepsen.util 4 :ok :read [1 41] INFO [2022-01-10 12:52:53,587] jepsen worker 89 - jepsen.util 89 :ok :read [29 8] INFO [2022-01-10 12:52:53,588] jepsen worker 4 - jepsen.util 4 :invoke :write [1 29] INFO [2022-01-10 12:52:53,589] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,590] jepsen worker 89 - jepsen.util 89 :ok :read [29 8] INFO [2022-01-10 12:52:53,592] jepsen worker 89 - jepsen.util 89 :invoke :write [29 11] INFO [2022-01-10 12:52:53,676] jepsen worker 51 - jepsen.util 51 :ok :write [17 25] INFO [2022-01-10 12:52:53,677] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:53,679] jepsen worker 2 - jepsen.util 2 :ok :write [0 44] INFO [2022-01-10 12:52:53,679] jepsen worker 2 - jepsen.util 2 :invoke :write [0 6] INFO [2022-01-10 12:52:53,682] jepsen worker 14 - jepsen.util 14 :ok :write [4 22] INFO [2022-01-10 12:52:53,683] jepsen worker 41 - jepsen.util 41 :ok :write [13 30] INFO [2022-01-10 12:52:53,684] jepsen worker 51 - jepsen.util 51 :ok :read [17 25] INFO [2022-01-10 12:52:53,684] jepsen worker 14 - jepsen.util 14 :invoke :write [4 30] INFO [2022-01-10 12:52:53,686] jepsen worker 51 - jepsen.util 51 :invoke :write [17 11] INFO [2022-01-10 12:52:53,686] jepsen worker 85 - jepsen.util 85 :ok :write [28 9] INFO [2022-01-10 12:52:53,686] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:53,688] jepsen worker 85 - jepsen.util 85 :invoke :write [28 8] INFO [2022-01-10 12:52:53,688] jepsen worker 41 - jepsen.util 41 :ok :read [13 30] INFO [2022-01-10 12:52:53,689] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:53,691] jepsen worker 41 - jepsen.util 41 :ok :read [13 30] INFO [2022-01-10 12:52:53,691] jepsen worker 89 - jepsen.util 89 :ok :write [29 11] INFO [2022-01-10 12:52:53,692] jepsen worker 71 - jepsen.util 71 :ok :write [23 33] INFO [2022-01-10 12:52:53,693] jepsen worker 89 - jepsen.util 89 :invoke :write [29 16] INFO [2022-01-10 12:52:53,694] jepsen worker 41 - jepsen.util 41 :invoke :write [13 14] INFO [2022-01-10 12:52:53,695] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:53,699] jepsen worker 71 - jepsen.util 71 :ok :read [23 33] INFO [2022-01-10 12:52:53,700] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:53,701] jepsen worker 36 - jepsen.util 36 :ok :write [12 35] INFO [2022-01-10 12:52:53,702] jepsen worker 78 - jepsen.util 78 :ok :write [26 35] INFO [2022-01-10 12:52:53,702] jepsen worker 2 - jepsen.util 2 :ok :write [0 6] INFO [2022-01-10 12:52:53,703] jepsen worker 71 - jepsen.util 71 :ok :read [23 33] INFO [2022-01-10 12:52:53,703] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,704] jepsen worker 36 - jepsen.util 36 :ok :read [12 35] INFO [2022-01-10 12:52:53,705] jepsen worker 71 - jepsen.util 71 :invoke :write [23 34] INFO [2022-01-10 12:52:53,706] jepsen worker 36 - jepsen.util 36 :invoke :write [12 39] INFO [2022-01-10 12:52:53,707] jepsen worker 4 - jepsen.util 4 :ok :write [1 29] INFO [2022-01-10 12:52:53,707] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:53,708] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,709] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:53,710] jepsen worker 78 - jepsen.util 78 :ok :read [26 35] INFO [2022-01-10 12:52:53,710] jepsen worker 2 - jepsen.util 2 :ok :read [0 6] INFO [2022-01-10 12:52:53,710] jepsen worker 4 - jepsen.util 4 :ok :read [1 29] INFO [2022-01-10 12:52:53,711] jepsen worker 78 - jepsen.util 78 :invoke :write [26 18] INFO [2022-01-10 12:52:53,712] jepsen worker 4 - jepsen.util 4 :invoke :write [1 8] INFO [2022-01-10 12:52:53,713] jepsen worker 88 - jepsen.util 88 :ok :write [29 12] INFO [2022-01-10 12:52:53,713] jepsen worker 2 - jepsen.util 2 :invoke :write [0 28] INFO [2022-01-10 12:52:53,715] jepsen worker 88 - jepsen.util 88 :invoke :write [29 1] INFO [2022-01-10 12:52:53,722] jepsen worker 41 - jepsen.util 41 :ok :write [13 14] INFO [2022-01-10 12:52:53,723] jepsen worker 41 - jepsen.util 41 :invoke :write [13 46] INFO [2022-01-10 12:52:53,725] jepsen worker 85 - jepsen.util 85 :ok :write [28 8] INFO [2022-01-10 12:52:53,726] jepsen worker 85 - jepsen.util 85 :invoke :write [28 39] INFO [2022-01-10 12:52:53,732] jepsen worker 89 - jepsen.util 89 :ok :write [29 16] INFO [2022-01-10 12:52:53,733] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,735] jepsen worker 89 - jepsen.util 89 :ok :read [29 16] INFO [2022-01-10 12:52:53,735] jepsen worker 36 - jepsen.util 36 :ok :write [12 39] INFO [2022-01-10 12:52:53,736] jepsen worker 2 - jepsen.util 2 :ok :write [0 28] INFO [2022-01-10 12:52:53,736] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,736] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,737] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,737] jepsen worker 36 - jepsen.util 36 :ok :read [12 39] INFO [2022-01-10 12:52:53,738] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,738] jepsen worker 2 - jepsen.util 2 :ok :read [0 28] INFO [2022-01-10 12:52:53,740] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,742] jepsen worker 2 - jepsen.util 2 :ok :read [0 28] INFO [2022-01-10 12:52:53,742] jepsen worker 89 - jepsen.util 89 :ok :read [29 16] INFO [2022-01-10 12:52:53,743] jepsen worker 36 - jepsen.util 36 :ok :read [12 39] INFO [2022-01-10 12:52:53,743] jepsen worker 41 - jepsen.util 41 :ok :write [13 46] INFO [2022-01-10 12:52:53,743] jepsen worker 2 - jepsen.util 2 :invoke :write [0 27] INFO [2022-01-10 12:52:53,744] jepsen worker 36 - jepsen.util 36 :invoke :write [12 47] INFO [2022-01-10 12:52:53,746] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:53,746] jepsen worker 89 - jepsen.util 89 :invoke :write [29 35] INFO [2022-01-10 12:52:53,748] jepsen worker 41 - jepsen.util 41 :ok :read [13 46] INFO [2022-01-10 12:52:53,748] jepsen worker 41 - jepsen.util 41 :invoke :write [13 46] INFO [2022-01-10 12:52:53,749] jepsen worker 4 - jepsen.util 4 :ok :write [1 8] INFO [2022-01-10 12:52:53,750] jepsen worker 4 - jepsen.util 4 :invoke :write [1 34] INFO [2022-01-10 12:52:53,754] jepsen worker 88 - jepsen.util 88 :ok :write [29 1] INFO [2022-01-10 12:52:53,755] jepsen worker 88 - jepsen.util 88 :invoke :write [29 0] INFO [2022-01-10 12:52:53,756] jepsen worker 85 - jepsen.util 85 :ok :write [28 39] INFO [2022-01-10 12:52:53,757] jepsen worker 85 - jepsen.util 85 :invoke :write [28 15] INFO [2022-01-10 12:52:53,761] jepsen worker 36 - jepsen.util 36 :ok :write [12 47] INFO [2022-01-10 12:52:53,762] jepsen worker 41 - jepsen.util 41 :ok :write [13 46] INFO [2022-01-10 12:52:53,763] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,764] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:53,764] jepsen worker 36 - jepsen.util 36 :ok :read [12 47] INFO [2022-01-10 12:52:53,765] jepsen worker 41 - jepsen.util 41 :ok :read [13 46] INFO [2022-01-10 12:52:53,765] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,766] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:53,766] jepsen worker 2 - jepsen.util 2 :ok :write [0 27] INFO [2022-01-10 12:52:53,767] jepsen worker 36 - jepsen.util 36 :ok :read [12 47] INFO [2022-01-10 12:52:53,768] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,768] jepsen worker 41 - jepsen.util 41 :ok :read [13 46] INFO [2022-01-10 12:52:53,769] jepsen worker 36 - jepsen.util 36 :invoke :write [12 41] INFO [2022-01-10 12:52:53,769] jepsen worker 2 - jepsen.util 2 :ok :read [0 27] INFO [2022-01-10 12:52:53,769] jepsen worker 41 - jepsen.util 41 :invoke :write [13 8] INFO [2022-01-10 12:52:53,770] jepsen worker 2 - jepsen.util 2 :invoke :write [0 35] INFO [2022-01-10 12:52:53,772] jepsen worker 4 - jepsen.util 4 :ok :write [1 34] INFO [2022-01-10 12:52:53,772] jepsen worker 4 - jepsen.util 4 :invoke :write [1 2] INFO [2022-01-10 12:52:53,774] jepsen worker 89 - jepsen.util 89 :ok :write [29 35] INFO [2022-01-10 12:52:53,775] jepsen worker 89 - jepsen.util 89 :invoke :write [29 9] INFO [2022-01-10 12:52:53,779] jepsen worker 85 - jepsen.util 85 :ok :write [28 15] INFO [2022-01-10 12:52:53,780] jepsen worker 85 - jepsen.util 85 :invoke :write [28 18] INFO [2022-01-10 12:52:53,786] jepsen worker 2 - jepsen.util 2 :ok :write [0 35] INFO [2022-01-10 12:52:53,787] jepsen worker 2 - jepsen.util 2 :invoke :write [0 45] INFO [2022-01-10 12:52:53,789] jepsen worker 4 - jepsen.util 4 :ok :write [1 2] INFO [2022-01-10 12:52:53,790] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:53,793] jepsen worker 4 - jepsen.util 4 :ok :read [1 2] INFO [2022-01-10 12:52:53,794] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:53,795] jepsen worker 4 - jepsen.util 4 :ok :read [1 2] INFO [2022-01-10 12:52:53,796] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:53,798] jepsen worker 4 - jepsen.util 4 :ok :read [1 2] INFO [2022-01-10 12:52:53,799] jepsen worker 4 - jepsen.util 4 :invoke :write [1 3] INFO [2022-01-10 12:52:53,805] jepsen worker 51 - jepsen.util 51 :ok :write [17 11] INFO [2022-01-10 12:52:53,806] jepsen worker 51 - jepsen.util 51 :invoke :write [17 21] INFO [2022-01-10 12:52:53,812] jepsen worker 12 - jepsen.util 12 :ok :write [4 10] INFO [2022-01-10 12:52:53,812] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:53,814] jepsen worker 12 - jepsen.util 12 :ok :read [4 10] INFO [2022-01-10 12:52:53,814] jepsen worker 12 - jepsen.util 12 :invoke :write [4 14] INFO [2022-01-10 12:52:53,818] jepsen worker 69 - jepsen.util 69 :ok :write [23 31] INFO [2022-01-10 12:52:53,819] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:53,821] jepsen worker 69 - jepsen.util 69 :ok :read [23 31] INFO [2022-01-10 12:52:53,822] jepsen worker 69 - jepsen.util 69 :invoke :write [23 1] INFO [2022-01-10 12:52:53,825] jepsen worker 78 - jepsen.util 78 :ok :write [26 18] INFO [2022-01-10 12:52:53,826] jepsen worker 78 - jepsen.util 78 :invoke :write [26 39] INFO [2022-01-10 12:52:53,832] jepsen worker 51 - jepsen.util 51 :ok :write [17 21] INFO [2022-01-10 12:52:53,833] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:53,834] jepsen worker 51 - jepsen.util 51 :ok :read [17 21] INFO [2022-01-10 12:52:53,835] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:53,836] jepsen worker 51 - jepsen.util 51 :ok :read [17 21] INFO [2022-01-10 12:52:53,837] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:53,838] jepsen worker 12 - jepsen.util 12 :ok :write [4 14] INFO [2022-01-10 12:52:53,838] jepsen worker 51 - jepsen.util 51 :ok :read [17 21] INFO [2022-01-10 12:52:53,839] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:53,840] jepsen worker 51 - jepsen.util 51 :invoke :write [17 3] INFO [2022-01-10 12:52:53,842] jepsen worker 12 - jepsen.util 12 :ok :read [4 14] INFO [2022-01-10 12:52:53,843] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:53,844] jepsen worker 12 - jepsen.util 12 :ok :read [4 14] INFO [2022-01-10 12:52:53,844] jepsen worker 71 - jepsen.util 71 :ok :write [23 34] INFO [2022-01-10 12:52:53,845] jepsen worker 71 - jepsen.util 71 :invoke :write [23 49] INFO [2022-01-10 12:52:53,846] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:53,847] jepsen worker 12 - jepsen.util 12 :ok :read [4 14] INFO [2022-01-10 12:52:53,848] jepsen worker 12 - jepsen.util 12 :invoke :write [4 13] INFO [2022-01-10 12:52:53,850] jepsen worker 78 - jepsen.util 78 :ok :write [26 39] INFO [2022-01-10 12:52:53,852] jepsen worker 78 - jepsen.util 78 :invoke :write [26 6] INFO [2022-01-10 12:52:53,855] jepsen worker 51 - jepsen.util 51 :ok :write [17 3] INFO [2022-01-10 12:52:53,856] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:53,858] jepsen worker 51 - jepsen.util 51 :ok :read [17 3] INFO [2022-01-10 12:52:53,859] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:53,859] jepsen worker 6 - jepsen.util 6 :ok :write [2 21] INFO [2022-01-10 12:52:53,860] jepsen worker 51 - jepsen.util 51 :ok :read [17 3] INFO [2022-01-10 12:52:53,860] jepsen worker 6 - jepsen.util 6 :invoke :write [2 1] INFO [2022-01-10 12:52:53,860] jepsen worker 12 - jepsen.util 12 :ok :write [4 13] INFO [2022-01-10 12:52:53,861] jepsen worker 12 - jepsen.util 12 :invoke :write [4 23] INFO [2022-01-10 12:52:53,862] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:53,863] jepsen worker 51 - jepsen.util 51 :ok :read [17 3] INFO [2022-01-10 12:52:53,864] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:53,866] jepsen worker 23 - jepsen.util 23 :ok :write [7 48] INFO [2022-01-10 12:52:53,866] jepsen worker 51 - jepsen.util 51 :ok :read [17 3] INFO [2022-01-10 12:52:53,866] jepsen worker 71 - jepsen.util 71 :ok :write [23 49] INFO [2022-01-10 12:52:53,867] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:53,868] jepsen worker 51 - jepsen.util 51 :invoke :write [17 19] INFO [2022-01-10 12:52:53,868] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:53,870] jepsen worker 71 - jepsen.util 71 :ok :read [23 49] INFO [2022-01-10 12:52:53,870] jepsen worker 78 - jepsen.util 78 :ok :write [26 6] INFO [2022-01-10 12:52:53,871] jepsen worker 71 - jepsen.util 71 :invoke :write [23 29] INFO [2022-01-10 12:52:53,871] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:53,872] jepsen worker 35 - jepsen.util 35 :ok :write [11 32] INFO [2022-01-10 12:52:53,873] jepsen worker 78 - jepsen.util 78 :ok :read [26 6] INFO [2022-01-10 12:52:53,873] jepsen worker 35 - jepsen.util 35 :invoke :write [11 42] INFO [2022-01-10 12:52:53,874] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:53,876] jepsen worker 78 - jepsen.util 78 :ok :read [26 6] INFO [2022-01-10 12:52:53,876] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:53,878] jepsen worker 78 - jepsen.util 78 :ok :read [26 6] INFO [2022-01-10 12:52:53,879] jepsen worker 78 - jepsen.util 78 :invoke :write [26 31] INFO [2022-01-10 12:52:53,882] jepsen worker 62 - jepsen.util 62 :ok :write [20 46] INFO [2022-01-10 12:52:53,883] jepsen worker 62 - jepsen.util 62 :invoke :write [20 2] INFO [2022-01-10 12:52:53,887] jepsen worker 12 - jepsen.util 12 :ok :write [4 23] INFO [2022-01-10 12:52:53,888] jepsen worker 12 - jepsen.util 12 :invoke :write [4 3] INFO [2022-01-10 12:52:53,889] jepsen worker 41 - jepsen.util 41 :ok :write [13 8] INFO [2022-01-10 12:52:53,889] jepsen worker 36 - jepsen.util 36 :ok :write [12 41] INFO [2022-01-10 12:52:53,890] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:53,891] jepsen worker 75 - jepsen.util 75 :ok :write [25 20] INFO [2022-01-10 12:52:53,891] jepsen worker 69 - jepsen.util 69 :ok :write [23 1] INFO [2022-01-10 12:52:53,891] jepsen worker 36 - jepsen.util 36 :invoke :write [12 12] INFO [2022-01-10 12:52:53,892] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:53,892] jepsen worker 41 - jepsen.util 41 :ok :read [13 8] INFO [2022-01-10 12:52:53,893] jepsen worker 75 - jepsen.util 75 :invoke :write [25 18] INFO [2022-01-10 12:52:53,893] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:53,894] jepsen worker 69 - jepsen.util 69 :ok :read [23 1] INFO [2022-01-10 12:52:53,895] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:53,895] jepsen worker 41 - jepsen.util 41 :ok :read [13 8] INFO [2022-01-10 12:52:53,895] jepsen worker 85 - jepsen.util 85 :ok :write [28 18] INFO [2022-01-10 12:52:53,896] jepsen worker 41 - jepsen.util 41 :invoke :write [13 22] INFO [2022-01-10 12:52:53,896] jepsen worker 69 - jepsen.util 69 :ok :read [23 1] INFO [2022-01-10 12:52:53,897] jepsen worker 85 - jepsen.util 85 :invoke :write [28 48] INFO [2022-01-10 12:52:53,898] jepsen worker 69 - jepsen.util 69 :invoke :write [23 5] INFO [2022-01-10 12:52:53,898] jepsen worker 89 - jepsen.util 89 :ok :write [29 9] INFO [2022-01-10 12:52:53,900] jepsen worker 89 - jepsen.util 89 :invoke :write [29 11] INFO [2022-01-10 12:52:53,900] jepsen worker 44 - jepsen.util 44 :ok :write [14 11] INFO [2022-01-10 12:52:53,901] jepsen worker 23 - jepsen.util 23 :ok :read [7 48] INFO [2022-01-10 12:52:53,902] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:53,903] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:53,903] jepsen worker 44 - jepsen.util 44 :ok :read [14 11] INFO [2022-01-10 12:52:53,904] jepsen worker 78 - jepsen.util 78 :ok :write [26 31] INFO [2022-01-10 12:52:53,904] jepsen worker 4 - jepsen.util 4 :ok :write [1 3] INFO [2022-01-10 12:52:53,904] jepsen worker 23 - jepsen.util 23 :ok :read [7 48] INFO [2022-01-10 12:52:53,904] jepsen worker 2 - jepsen.util 2 :ok :write [0 45] INFO [2022-01-10 12:52:53,905] jepsen worker 78 - jepsen.util 78 :invoke :write [26 28] INFO [2022-01-10 12:52:53,907] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:53,908] jepsen worker 4 - jepsen.util 4 :invoke :write [1 14] INFO [2022-01-10 12:52:53,908] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:53,909] jepsen worker 2 - jepsen.util 2 :ok :read [0 45] INFO [2022-01-10 12:52:53,909] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:53,910] jepsen worker 44 - jepsen.util 44 :ok :read [14 11] INFO [2022-01-10 12:52:53,910] jepsen worker 2 - jepsen.util 2 :invoke :write [0 35] INFO [2022-01-10 12:52:53,910] jepsen worker 23 - jepsen.util 23 :ok :read [7 48] INFO [2022-01-10 12:52:53,911] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:53,911] jepsen worker 17 - jepsen.util 17 :ok :write [5 10] INFO [2022-01-10 12:52:53,912] jepsen worker 36 - jepsen.util 36 :ok :write [12 12] INFO [2022-01-10 12:52:53,912] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:53,913] jepsen worker 44 - jepsen.util 44 :ok :read [14 11] INFO [2022-01-10 12:52:53,913] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:53,913] jepsen worker 51 - jepsen.util 51 :ok :write [17 19] INFO [2022-01-10 12:52:53,914] jepsen worker 23 - jepsen.util 23 :ok :read [7 48] INFO [2022-01-10 12:52:53,914] jepsen worker 36 - jepsen.util 36 :ok :read [12 12] INFO [2022-01-10 12:52:53,915] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:53,916] jepsen worker 23 - jepsen.util 23 :invoke :write [7 2] INFO [2022-01-10 12:52:53,917] jepsen worker 44 - jepsen.util 44 :ok :read [14 11] INFO [2022-01-10 12:52:53,917] jepsen worker 82 - jepsen.util 82 :ok :write [27 45] INFO [2022-01-10 12:52:53,917] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:53,917] jepsen worker 85 - jepsen.util 85 :ok :write [28 48] INFO [2022-01-10 12:52:53,918] jepsen worker 36 - jepsen.util 36 :invoke :write [12 5] INFO [2022-01-10 12:52:53,919] jepsen worker 51 - jepsen.util 51 :ok :read [17 19] INFO [2022-01-10 12:52:53,921] jepsen worker 85 - jepsen.util 85 :invoke :write [28 39] INFO [2022-01-10 12:52:53,922] jepsen worker 48 - jepsen.util 48 :ok :write [16 13] INFO [2022-01-10 12:52:53,922] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 12:52:53,923] jepsen worker 17 - jepsen.util 17 :ok :read [5 10] INFO [2022-01-10 12:52:53,924] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:53,925] jepsen worker 12 - jepsen.util 12 :ok :write [4 3] INFO [2022-01-10 12:52:53,925] jepsen worker 44 - jepsen.util 44 :ok :read [14 11] INFO [2022-01-10 12:52:53,925] jepsen worker 82 - jepsen.util 82 :invoke :write [27 11] INFO [2022-01-10 12:52:53,927] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:53,929] jepsen worker 4 - jepsen.util 4 :ok :write [1 14] INFO [2022-01-10 12:52:53,929] jepsen worker 67 - jepsen.util 67 :ok :write [22 10] INFO [2022-01-10 12:52:53,929] jepsen worker 12 - jepsen.util 12 :ok :read [4 3] INFO [2022-01-10 12:52:53,929] jepsen worker 44 - jepsen.util 44 :invoke :write [14 16] INFO [2022-01-10 12:52:53,931] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:53,931] jepsen worker 41 - jepsen.util 41 :ok :write [13 22] INFO [2022-01-10 12:52:53,932] jepsen worker 26 - jepsen.util 26 :ok :write [8 26] INFO [2022-01-10 12:52:53,932] jepsen worker 51 - jepsen.util 51 :invoke :write [17 33] INFO [2022-01-10 12:52:53,933] jepsen worker 12 - jepsen.util 12 :ok :read [4 3] INFO [2022-01-10 12:52:53,934] jepsen worker 7 - jepsen.util 7 :ok :write [2 7] INFO [2022-01-10 12:52:53,934] jepsen worker 4 - jepsen.util 4 :invoke :write [1 5] INFO [2022-01-10 12:52:53,935] jepsen worker 71 - jepsen.util 71 :ok :write [23 29] INFO [2022-01-10 12:52:53,937] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:53,937] jepsen worker 59 - jepsen.util 59 :ok :write [19 31] INFO [2022-01-10 12:52:53,938] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:53,938] jepsen worker 41 - jepsen.util 41 :ok :read [13 22] INFO [2022-01-10 12:52:53,940] jepsen worker 71 - jepsen.util 71 :invoke :write [23 15] INFO [2022-01-10 12:52:53,940] jepsen worker 48 - jepsen.util 48 :ok :read [16 13] INFO [2022-01-10 12:52:53,941] jepsen worker 17 - jepsen.util 17 :invoke :write [5 9] INFO [2022-01-10 12:52:53,943] jepsen worker 36 - jepsen.util 36 :ok :write [12 5] INFO [2022-01-10 12:52:53,943] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:53,944] jepsen worker 46 - jepsen.util 46 :ok :write [15 33] INFO [2022-01-10 12:52:53,945] jepsen worker 89 - jepsen.util 89 :ok :write [29 11] INFO [2022-01-10 12:52:53,945] jepsen worker 59 - jepsen.util 59 :ok :read [19 31] INFO [2022-01-10 12:52:53,945] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:53,945] jepsen worker 21 - jepsen.util 21 :ok :write [7 36] INFO [2022-01-10 12:52:53,947] jepsen worker 12 - jepsen.util 12 :ok :read [4 3] INFO [2022-01-10 12:52:53,947] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:52:53,948] jepsen worker 67 - jepsen.util 67 :invoke :write [22 28] INFO [2022-01-10 12:52:53,948] jepsen worker 7 - jepsen.util 7 :ok :read [2 7] INFO [2022-01-10 12:52:53,950] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,952] jepsen worker 33 - jepsen.util 33 :ok :write [11 22] INFO [2022-01-10 12:52:53,952] jepsen worker 89 - jepsen.util 89 :ok :read [29 11] INFO [2022-01-10 12:52:53,952] jepsen worker 0 - jepsen.util 0 :ok :write [0 30] INFO [2022-01-10 12:52:53,952] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:53,953] jepsen worker 78 - jepsen.util 78 :ok :write [26 28] INFO [2022-01-10 12:52:53,953] jepsen worker 18 - jepsen.util 18 :ok :write [6 12] INFO [2022-01-10 12:52:53,954] jepsen worker 59 - jepsen.util 59 :ok :read [19 31] INFO [2022-01-10 12:52:53,956] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:53,958] jepsen worker 18 - jepsen.util 18 :ok :read [6 12] INFO [2022-01-10 12:52:53,958] jepsen worker 48 - jepsen.util 48 :invoke :write [16 40] INFO [2022-01-10 12:52:53,960] jepsen worker 51 - jepsen.util 51 :ok :write [17 33] INFO [2022-01-10 12:52:53,961] jepsen worker 78 - jepsen.util 78 :invoke :write [26 19] INFO [2022-01-10 12:52:53,962] jepsen worker 31 - jepsen.util 31 :ok :write [10 9] INFO [2022-01-10 12:52:53,962] jepsen worker 28 - jepsen.util 28 :ok :read [9 39] INFO [2022-01-10 12:52:53,964] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:53,965] jepsen worker 62 - jepsen.util 62 :ok :write [20 2] INFO [2022-01-10 12:52:53,966] jepsen worker 33 - jepsen.util 33 :ok :read [11 22] INFO [2022-01-10 12:52:53,967] jepsen worker 28 - jepsen.util 28 :invoke :read [9 nil] INFO [2022-01-10 12:52:53,969] jepsen worker 9 - jepsen.util 9 :ok :write [3 0] INFO [2022-01-10 12:52:53,969] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:53,971] jepsen worker 46 - jepsen.util 46 :ok :read [15 33] INFO [2022-01-10 12:52:53,971] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:52:53,973] jepsen worker 7 - jepsen.util 7 :ok :read [2 7] INFO [2022-01-10 12:52:53,973] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:52:53,975] jepsen worker 62 - jepsen.util 62 :ok :read [20 2] INFO [2022-01-10 12:52:53,976] jepsen worker 26 - jepsen.util 26 :invoke :write [8 0] INFO [2022-01-10 12:52:53,978] jepsen worker 43 - jepsen.util 43 :ok :write [14 26] INFO [2022-01-10 12:52:53,979] jepsen worker 59 - jepsen.util 59 :invoke :write [19 19] INFO [2022-01-10 12:52:53,981] jepsen worker 33 - jepsen.util 33 :invoke :write [11 39] INFO [2022-01-10 12:52:53,983] jepsen worker 64 - jepsen.util 64 :ok :write [21 4] INFO [2022-01-10 12:52:53,983] jepsen worker 12 - jepsen.util 12 :invoke :write [4 37] INFO [2022-01-10 12:52:53,986] jepsen worker 51 - jepsen.util 51 :invoke :write [17 7] INFO [2022-01-10 12:52:53,988] jepsen worker 17 - jepsen.util 17 :ok :write [5 9] INFO [2022-01-10 12:52:53,989] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:53,990] jepsen worker 89 - jepsen.util 89 :ok :read [29 11] INFO [2022-01-10 12:52:53,991] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 12:52:53,992] jepsen worker 82 - jepsen.util 82 :ok :write [27 11] INFO [2022-01-10 12:52:53,993] jepsen worker 17 - jepsen.util 17 :ok :read [5 9] INFO [2022-01-10 12:52:53,993] jepsen worker 59 - jepsen.util 59 :ok :write [19 19] INFO [2022-01-10 12:52:53,994] jepsen worker 46 - jepsen.util 46 :invoke :write [15 21] INFO [2022-01-10 12:52:53,996] jepsen worker 48 - jepsen.util 48 :ok :write [16 40] INFO [2022-01-10 12:52:53,997] jepsen worker 45 - jepsen.util 45 :ok :write [15 34] INFO [2022-01-10 12:52:53,998] jepsen worker 59 - jepsen.util 59 :invoke :write [19 12] INFO [2022-01-10 12:52:54,000] jepsen worker 67 - jepsen.util 67 :ok :write [22 28] INFO [2022-01-10 12:52:54,000] jepsen worker 17 - jepsen.util 17 :invoke :write [5 19] INFO [2022-01-10 12:52:54,002] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:54,004] jepsen worker 18 - jepsen.util 18 :ok :read [6 12] INFO [2022-01-10 12:52:54,004] jepsen worker 55 - jepsen.util 55 :ok :write [18 17] INFO [2022-01-10 12:52:54,005] jepsen worker 6 - jepsen.util 6 :ok :write [2 1] INFO [2022-01-10 12:52:54,006] jepsen worker 82 - jepsen.util 82 :invoke :write [27 28] INFO [2022-01-10 12:52:54,009] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:52:54,009] jepsen worker 72 - jepsen.util 72 :ok :write [24 21] INFO [2022-01-10 12:52:54,010] jepsen worker 27 - jepsen.util 27 :ok :write [9 0] INFO [2022-01-10 12:52:54,010] jepsen worker 62 - jepsen.util 62 :ok :read [20 2] INFO [2022-01-10 12:52:54,012] jepsen worker 64 - jepsen.util 64 :invoke :write [21 5] INFO [2022-01-10 12:52:54,014] jepsen worker 67 - jepsen.util 67 :invoke :write [22 33] INFO [2022-01-10 12:52:54,016] jepsen worker 62 - jepsen.util 62 :invoke :write [20 15] INFO [2022-01-10 12:52:54,017] jepsen worker 35 - jepsen.util 35 :ok :write [11 42] INFO [2022-01-10 12:52:54,019] jepsen worker 26 - jepsen.util 26 :ok :write [8 0] INFO [2022-01-10 12:52:54,019] jepsen worker 55 - jepsen.util 55 :invoke :write [18 15] INFO [2022-01-10 12:52:54,022] jepsen worker 26 - jepsen.util 26 :invoke :write [8 47] INFO [2022-01-10 12:52:54,024] jepsen worker 31 - jepsen.util 31 :invoke :write [10 24] INFO [2022-01-10 12:52:54,026] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:54,027] jepsen worker 59 - jepsen.util 59 :ok :write [19 12] INFO [2022-01-10 12:52:54,028] jepsen worker 18 - jepsen.util 18 :ok :read [6 12] INFO [2022-01-10 12:52:54,028] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:52:54,030] jepsen worker 21 - jepsen.util 21 :ok :read [7 36] INFO [2022-01-10 12:52:54,032] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:54,033] jepsen worker 59 - jepsen.util 59 :ok :read [19 12] INFO [2022-01-10 12:52:54,035] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] WARN [2022-01-10 12:52:54,036] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":616046,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000001/00000000100998FD","last_local_socket":"172.23.104.79:58632","total_dispatch_duration_us":630161,"total_server_duration_us":117274,"encode_duration_us":1163,"operation_id":"0x1ad","timeout_ms":10000,"last_server_duration_us":115283,"total_duration_us":2747527},{"operation_name":"upsert","last_dispatch_duration_us":2604846,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000001/000000005D716E8E","last_local_socket":"172.23.104.79:34680","total_dispatch_duration_us":2604846,"total_server_duration_us":2604026,"encode_duration_us":149,"operation_id":"0x1c9","timeout_ms":10000,"last_server_duration_us":2604026,"total_duration_us":2605317},{"operation_name":"upsert","last_dispatch_duration_us":1834165,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000001/00000000100998FD","last_local_socket":"172.23.104.79:58632","total_dispatch_duration_us":1834165,"total_server_duration_us":1833127,"encode_duration_us":159,"operation_id":"0x25e","timeout_ms":10000,"last_server_duration_us":1833127,"total_duration_us":1834590},{"operation_name":"upsert","last_dispatch_duration_us":1309118,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":1309118,"total_server_duration_us":1308174,"encode_duration_us":192,"operation_id":"0x215","timeout_ms":10000,"last_server_duration_us":1308174,"total_duration_us":1309657},{"operation_name":"upsert","last_dispatch_duration_us":728705,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000001/00000000100998FD","last_local_socket":"172.23.104.79:58632","total_dispatch_duration_us":728705,"total_server_duration_us":727728,"encode_duration_us":214,"operation_id":"0x1b6","timeout_ms":10000,"last_server_duration_us":727728,"total_duration_us":729154}],"total_count":5}} INFO [2022-01-10 12:52:54,037] jepsen worker 9 - jepsen.util 9 :ok :read [3 0] INFO [2022-01-10 12:52:54,038] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:54,040] jepsen worker 0 - jepsen.util 0 :ok :read [0 30] INFO [2022-01-10 12:52:54,042] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:54,044] jepsen worker 59 - jepsen.util 59 :ok :read [19 12] INFO [2022-01-10 12:52:54,044] jepsen worker 45 - jepsen.util 45 :invoke :write [15 7] INFO [2022-01-10 12:52:54,047] jepsen worker 43 - jepsen.util 43 :invoke :write [14 7] INFO [2022-01-10 12:52:54,050] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:52:54,052] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:52:54,053] jepsen worker 41 - jepsen.util 41 :invoke :write [13 16] INFO [2022-01-10 12:52:54,056] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:54,058] jepsen worker 9 - jepsen.util 9 :ok :read [3 0] INFO [2022-01-10 12:52:54,058] jepsen worker 48 - jepsen.util 48 :invoke :write [16 30] INFO [2022-01-10 12:52:54,060] jepsen worker 18 - jepsen.util 18 :invoke :write [6 9] INFO [2022-01-10 12:52:54,061] jepsen worker 35 - jepsen.util 35 :invoke :write [11 11] INFO [2022-01-10 12:52:54,063] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:52:54,065] jepsen worker 21 - jepsen.util 21 :ok :read [7 36] INFO [2022-01-10 12:52:54,065] jepsen worker 0 - jepsen.util 0 :invoke :write [0 19] INFO [2022-01-10 12:52:54,067] jepsen worker 58 - jepsen.util 58 :ok :write [19 26] INFO [2022-01-10 12:52:54,067] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:52:54,069] jepsen worker 72 - jepsen.util 72 :ok :read [24 21] INFO [2022-01-10 12:52:54,069] jepsen worker 21 - jepsen.util 21 :invoke :write [7 0] INFO [2022-01-10 12:52:54,071] jepsen worker 59 - jepsen.util 59 :invoke :write [19 0] INFO [2022-01-10 12:52:54,072] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:52:54,074] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:54,074] jepsen worker 72 - jepsen.util 72 :ok :read [24 21] INFO [2022-01-10 12:52:54,075] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:54,075] jepsen worker 36 - jepsen.util 36 :ok :read [12 5] INFO [2022-01-10 12:52:54,076] jepsen worker 89 - jepsen.util 89 :ok :read [29 11] INFO [2022-01-10 12:52:54,077] jepsen worker 58 - jepsen.util 58 :invoke :write [19 33] INFO [2022-01-10 12:52:54,077] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:54,078] jepsen worker 27 - jepsen.util 27 :ok :read [9 0] INFO [2022-01-10 12:52:54,079] jepsen worker 59 - jepsen.util 59 :ok :write [19 0] INFO [2022-01-10 12:52:54,079] jepsen worker 36 - jepsen.util 36 :invoke :write [12 0] INFO [2022-01-10 12:52:54,081] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:54,082] jepsen worker 59 - jepsen.util 59 :invoke :write [19 19] INFO [2022-01-10 12:52:54,082] jepsen worker 89 - jepsen.util 89 :ok :read [29 11] INFO [2022-01-10 12:52:54,083] jepsen worker 27 - jepsen.util 27 :invoke :write [9 41] INFO [2022-01-10 12:52:54,084] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:54,085] jepsen worker 6 - jepsen.util 6 :invoke :write [2 28] INFO [2022-01-10 12:52:54,086] jepsen worker 9 - jepsen.util 9 :ok :read [3 0] INFO [2022-01-10 12:52:54,087] jepsen worker 72 - jepsen.util 72 :invoke :write [24 19] INFO [2022-01-10 12:52:54,088] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:54,089] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:52:54,090] jepsen worker 9 - jepsen.util 9 :ok :read [3 0] INFO [2022-01-10 12:52:54,090] jepsen worker 89 - jepsen.util 89 :invoke :write [29 10] INFO [2022-01-10 12:52:54,091] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:52:54,091] jepsen worker 9 - jepsen.util 9 :invoke :write [3 6] INFO [2022-01-10 12:52:54,092] jepsen worker 7 - jepsen.util 7 :invoke :write [2 46] INFO [2022-01-10 12:52:54,093] jepsen worker 27 - jepsen.util 27 :ok :write [9 41] INFO [2022-01-10 12:52:54,094] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:54,095] jepsen worker 27 - jepsen.util 27 :ok :read [9 41] INFO [2022-01-10 12:52:54,096] jepsen worker 27 - jepsen.util 27 :invoke :write [9 48] INFO [2022-01-10 12:52:54,098] jepsen worker 59 - jepsen.util 59 :ok :write [19 19] INFO [2022-01-10 12:52:54,099] jepsen worker 59 - jepsen.util 59 :invoke :write [19 40] INFO [2022-01-10 12:52:54,106] jepsen worker 27 - jepsen.util 27 :ok :write [9 48] INFO [2022-01-10 12:52:54,107] jepsen worker 58 - jepsen.util 58 :ok :write [19 33] INFO [2022-01-10 12:52:54,107] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:54,108] jepsen worker 58 - jepsen.util 58 :invoke :write [19 46] INFO [2022-01-10 12:52:54,109] jepsen worker 27 - jepsen.util 27 :ok :read [9 48] INFO [2022-01-10 12:52:54,109] jepsen worker 27 - jepsen.util 27 :invoke :write [9 35] INFO [2022-01-10 12:52:54,119] jepsen worker 59 - jepsen.util 59 :ok :write [19 40] INFO [2022-01-10 12:52:54,120] jepsen worker 59 - jepsen.util 59 :invoke :write [19 4] INFO [2022-01-10 12:52:54,122] jepsen worker 27 - jepsen.util 27 :ok :write [9 35] INFO [2022-01-10 12:52:54,122] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:54,124] jepsen worker 27 - jepsen.util 27 :ok :read [9 35] INFO [2022-01-10 12:52:54,124] jepsen worker 27 - jepsen.util 27 :invoke :write [9 7] INFO [2022-01-10 12:52:54,135] jepsen worker 85 - jepsen.util 85 :ok :write [28 39] INFO [2022-01-10 12:52:54,135] jepsen worker 77 - jepsen.util 77 :ok :write [25 0] INFO [2022-01-10 12:52:54,135] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:54,136] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:54,137] jepsen worker 85 - jepsen.util 85 :ok :read [28 39] INFO [2022-01-10 12:52:54,138] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:54,138] jepsen worker 77 - jepsen.util 77 :ok :read [25 0] INFO [2022-01-10 12:52:54,138] jepsen worker 88 - jepsen.util 88 :ok :write [29 0] INFO [2022-01-10 12:52:54,139] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:54,139] jepsen worker 85 - jepsen.util 85 :ok :read [28 39] INFO [2022-01-10 12:52:54,139] jepsen worker 88 - jepsen.util 88 :invoke :write [29 24] INFO [2022-01-10 12:52:54,140] jepsen worker 77 - jepsen.util 77 :ok :read [25 0] INFO [2022-01-10 12:52:54,140] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:54,140] jepsen worker 77 - jepsen.util 77 :invoke :write [25 19] INFO [2022-01-10 12:52:54,141] jepsen worker 85 - jepsen.util 85 :ok :read [28 39] INFO [2022-01-10 12:52:54,142] jepsen worker 85 - jepsen.util 85 :invoke :write [28 19] INFO [2022-01-10 12:52:54,145] jepsen worker 4 - jepsen.util 4 :ok :write [1 5] INFO [2022-01-10 12:52:54,145] jepsen worker 4 - jepsen.util 4 :invoke :write [1 40] INFO [2022-01-10 12:52:54,146] jepsen worker 23 - jepsen.util 23 :ok :write [7 2] INFO [2022-01-10 12:52:54,147] jepsen worker 23 - jepsen.util 23 :invoke :write [7 3] INFO [2022-01-10 12:52:54,152] jepsen worker 36 - jepsen.util 36 :ok :write [12 0] INFO [2022-01-10 12:52:54,153] jepsen worker 2 - jepsen.util 2 :ok :write [0 35] INFO [2022-01-10 12:52:54,153] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:54,153] jepsen worker 2 - jepsen.util 2 :invoke :write [0 30] INFO [2022-01-10 12:52:54,154] jepsen worker 36 - jepsen.util 36 :ok :read [12 0] INFO [2022-01-10 12:52:54,155] jepsen worker 36 - jepsen.util 36 :invoke :write [12 31] INFO [2022-01-10 12:52:54,158] jepsen worker 71 - jepsen.util 71 :ok :write [23 15] INFO [2022-01-10 12:52:54,159] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:54,160] jepsen worker 71 - jepsen.util 71 :ok :read [23 15] INFO [2022-01-10 12:52:54,162] jepsen worker 71 - jepsen.util 71 :invoke :write [23 27] INFO [2022-01-10 12:52:54,165] jepsen worker 78 - jepsen.util 78 :ok :write [26 19] INFO [2022-01-10 12:52:54,167] jepsen worker 55 - jepsen.util 55 :ok :write [18 15] INFO [2022-01-10 12:52:54,167] jepsen worker 78 - jepsen.util 78 :invoke :write [26 37] INFO [2022-01-10 12:52:54,168] jepsen worker 63 - jepsen.util 63 :ok :write [21 6] INFO [2022-01-10 12:52:54,168] jepsen worker 55 - jepsen.util 55 :invoke :write [18 13] INFO [2022-01-10 12:52:54,170] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:54,171] jepsen worker 62 - jepsen.util 62 :ok :write [20 15] INFO [2022-01-10 12:52:54,171] jepsen worker 63 - jepsen.util 63 :ok :read [21 6] INFO [2022-01-10 12:52:54,172] jepsen worker 51 - jepsen.util 51 :ok :write [17 7] INFO [2022-01-10 12:52:54,173] jepsen worker 12 - jepsen.util 12 :ok :write [4 37] INFO [2022-01-10 12:52:54,173] jepsen worker 41 - jepsen.util 41 :ok :write [13 16] INFO [2022-01-10 12:52:54,174] jepsen worker 62 - jepsen.util 62 :invoke :write [20 15] INFO [2022-01-10 12:52:54,176] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:54,177] jepsen worker 12 - jepsen.util 12 :ok :read [4 37] INFO [2022-01-10 12:52:54,178] jepsen worker 63 - jepsen.util 63 :invoke :write [21 26] INFO [2022-01-10 12:52:54,178] jepsen worker 46 - jepsen.util 46 :ok :write [15 21] INFO [2022-01-10 12:52:54,179] jepsen worker 41 - jepsen.util 41 :invoke :write [13 10] INFO [2022-01-10 12:52:54,180] jepsen worker 32 - jepsen.util 32 :ok :write [10 49] INFO [2022-01-10 12:52:54,180] jepsen worker 44 - jepsen.util 44 :ok :write [14 16] INFO [2022-01-10 12:52:54,181] jepsen worker 28 - jepsen.util 28 :ok :read [9 35] INFO [2022-01-10 12:52:54,182] jepsen worker 51 - jepsen.util 51 :invoke :write [17 11] INFO [2022-01-10 12:52:54,184] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:54,185] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:54,185] jepsen worker 46 - jepsen.util 46 :ok :read [15 21] INFO [2022-01-10 12:52:54,186] jepsen worker 32 - jepsen.util 32 :invoke :read [10 nil] INFO [2022-01-10 12:52:54,187] jepsen worker 17 - jepsen.util 17 :ok :write [5 19] INFO [2022-01-10 12:52:54,187] jepsen worker 72 - jepsen.util 72 :ok :write [24 19] INFO [2022-01-10 12:52:54,187] jepsen worker 12 - jepsen.util 12 :ok :read [4 37] INFO [2022-01-10 12:52:54,187] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:54,187] jepsen worker 32 - jepsen.util 32 :ok :read [10 49] INFO [2022-01-10 12:52:54,188] jepsen worker 9 - jepsen.util 9 :ok :write [3 6] INFO [2022-01-10 12:52:54,188] jepsen worker 46 - jepsen.util 46 :ok :read [15 21] INFO [2022-01-10 12:52:54,189] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:52:54,190] jepsen worker 88 - jepsen.util 88 :ok :write [29 24] INFO [2022-01-10 12:52:54,191] jepsen worker 72 - jepsen.util 72 :ok :read [24 19] INFO [2022-01-10 12:52:54,192] jepsen worker 17 - jepsen.util 17 :invoke :write [5 48] INFO [2022-01-10 12:52:54,192] jepsen worker 26 - jepsen.util 26 :ok :write [8 47] INFO [2022-01-10 12:52:54,195] jepsen worker 72 - jepsen.util 72 :invoke :write [24 10] INFO [2022-01-10 12:52:54,195] jepsen worker 82 - jepsen.util 82 :ok :write [27 28] INFO [2022-01-10 12:52:54,196] jepsen worker 85 - jepsen.util 85 :ok :write [28 19] INFO [2022-01-10 12:52:54,197] jepsen worker 9 - jepsen.util 9 :invoke :write [3 9] INFO [2022-01-10 12:52:54,199] jepsen worker 4 - jepsen.util 4 :ok :write [1 40] INFO [2022-01-10 12:52:54,199] jepsen worker 48 - jepsen.util 48 :ok :write [16 30] INFO [2022-01-10 12:52:54,200] jepsen worker 46 - jepsen.util 46 :invoke :write [15 31] INFO [2022-01-10 12:52:54,200] jepsen worker 18 - jepsen.util 18 :ok :write [6 9] INFO [2022-01-10 12:52:54,202] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:54,204] jepsen worker 82 - jepsen.util 82 :ok :read [27 28] INFO [2022-01-10 12:52:54,205] jepsen worker 67 - jepsen.util 67 :ok :write [22 33] INFO [2022-01-10 12:52:54,205] jepsen worker 88 - jepsen.util 88 :invoke :write [29 38] INFO [2022-01-10 12:52:54,208] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:54,210] jepsen worker 44 - jepsen.util 44 :ok :read [14 16] INFO [2022-01-10 12:52:54,210] jepsen worker 28 - jepsen.util 28 :invoke :write [9 49] INFO [2022-01-10 12:52:54,212] jepsen worker 6 - jepsen.util 6 :ok :write [2 28] INFO [2022-01-10 12:52:54,213] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:54,214] jepsen worker 82 - jepsen.util 82 :ok :read [27 28] INFO [2022-01-10 12:52:54,214] jepsen worker 4 - jepsen.util 4 :invoke :write [1 20] INFO [2022-01-10 12:52:54,215] jepsen worker 2 - jepsen.util 2 :ok :write [0 30] INFO [2022-01-10 12:52:54,216] jepsen worker 71 - jepsen.util 71 :ok :write [23 27] INFO [2022-01-10 12:52:54,216] jepsen worker 26 - jepsen.util 26 :invoke :write [8 44] INFO [2022-01-10 12:52:54,219] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:54,221] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:54,222] jepsen worker 85 - jepsen.util 85 :ok :read [28 19] INFO [2022-01-10 12:52:54,222] jepsen worker 67 - jepsen.util 67 :invoke :write [22 49] INFO [2022-01-10 12:52:54,222] jepsen worker 82 - jepsen.util 82 :ok :read [27 28] INFO [2022-01-10 12:52:54,224] jepsen worker 12 - jepsen.util 12 :invoke :write [4 48] INFO [2022-01-10 12:52:54,224] jepsen worker 78 - jepsen.util 78 :ok :write [26 37] INFO [2022-01-10 12:52:54,225] jepsen worker 33 - jepsen.util 33 :ok :write [11 39] INFO [2022-01-10 12:52:54,225] jepsen worker 85 - jepsen.util 85 :invoke :write [28 19] INFO [2022-01-10 12:52:54,226] jepsen worker 41 - jepsen.util 41 :ok :write [13 10] INFO [2022-01-10 12:52:54,228] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:54,229] jepsen worker 44 - jepsen.util 44 :ok :read [14 16] INFO [2022-01-10 12:52:54,229] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:54,230] jepsen worker 36 - jepsen.util 36 :ok :write [12 31] INFO [2022-01-10 12:52:54,231] jepsen worker 41 - jepsen.util 41 :ok :read [13 10] INFO [2022-01-10 12:52:54,231] jepsen worker 78 - jepsen.util 78 :invoke :write [26 34] INFO [2022-01-10 12:52:54,233] jepsen worker 62 - jepsen.util 62 :ok :write [20 15] INFO [2022-01-10 12:52:54,233] jepsen worker 71 - jepsen.util 71 :invoke :write [23 29] INFO [2022-01-10 12:52:54,235] jepsen worker 6 - jepsen.util 6 :invoke :write [2 12] INFO [2022-01-10 12:52:54,237] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:54,238] jepsen worker 77 - jepsen.util 77 :ok :write [25 19] INFO [2022-01-10 12:52:54,238] jepsen worker 18 - jepsen.util 18 :ok :read [6 9] INFO [2022-01-10 12:52:54,239] jepsen worker 33 - jepsen.util 33 :invoke :write [11 36] INFO [2022-01-10 12:52:54,241] jepsen worker 36 - jepsen.util 36 :invoke :write [12 27] INFO [2022-01-10 12:52:54,243] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:54,244] jepsen worker 77 - jepsen.util 77 :ok :read [25 19] INFO [2022-01-10 12:52:54,244] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:54,245] jepsen worker 23 - jepsen.util 23 :ok :write [7 3] INFO [2022-01-10 12:52:54,246] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:52:54,246] jepsen worker 41 - jepsen.util 41 :ok :read [13 10] INFO [2022-01-10 12:52:54,248] jepsen worker 62 - jepsen.util 62 :ok :read [20 15] INFO [2022-01-10 12:52:54,248] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:54,249] jepsen worker 48 - jepsen.util 48 :ok :read [16 30] INFO [2022-01-10 12:52:54,249] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:54,251] jepsen worker 82 - jepsen.util 82 :ok :read [27 28] INFO [2022-01-10 12:52:54,251] jepsen worker 2 - jepsen.util 2 :invoke :write [0 3] INFO [2022-01-10 12:52:54,253] jepsen worker 32 - jepsen.util 32 :invoke :write [10 38] INFO [2022-01-10 12:52:54,254] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:54,255] jepsen worker 51 - jepsen.util 51 :ok :write [17 11] INFO [2022-01-10 12:52:54,256] jepsen worker 23 - jepsen.util 23 :ok :read [7 3] INFO [2022-01-10 12:52:54,256] jepsen worker 48 - jepsen.util 48 :invoke :write [16 10] INFO [2022-01-10 12:52:54,258] jepsen worker 77 - jepsen.util 77 :invoke :write [25 33] INFO [2022-01-10 12:52:54,259] jepsen worker 17 - jepsen.util 17 :ok :write [5 48] INFO [2022-01-10 12:52:54,260] jepsen worker 82 - jepsen.util 82 :invoke :write [27 47] INFO [2022-01-10 12:52:54,261] jepsen worker 59 - jepsen.util 59 :ok :write [19 4] INFO [2022-01-10 12:52:54,261] jepsen worker 27 - jepsen.util 27 :ok :write [9 7] INFO [2022-01-10 12:52:54,262] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:54,263] jepsen worker 44 - jepsen.util 44 :ok :read [14 16] INFO [2022-01-10 12:52:54,263] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:54,264] jepsen worker 89 - jepsen.util 89 :ok :write [29 10] INFO [2022-01-10 12:52:54,264] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:54,266] jepsen worker 18 - jepsen.util 18 :ok :read [6 9] INFO [2022-01-10 12:52:54,266] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:52:54,266] jepsen worker 23 - jepsen.util 23 :ok :read [7 3] INFO [2022-01-10 12:52:54,267] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:54,267] jepsen worker 63 - jepsen.util 63 :ok :write [21 26] INFO [2022-01-10 12:52:54,268] jepsen worker 62 - jepsen.util 62 :ok :read [20 15] INFO [2022-01-10 12:52:54,269] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 12:52:54,269] jepsen worker 44 - jepsen.util 44 :ok :read [14 16] INFO [2022-01-10 12:52:54,269] jepsen worker 12 - jepsen.util 12 :ok :write [4 48] INFO [2022-01-10 12:52:54,270] jepsen worker 55 - jepsen.util 55 :ok :write [18 13] INFO [2022-01-10 12:52:54,271] jepsen worker 17 - jepsen.util 17 :ok :read [5 48] INFO [2022-01-10 12:52:54,271] jepsen worker 41 - jepsen.util 41 :invoke :write [13 14] INFO [2022-01-10 12:52:54,274] jepsen worker 62 - jepsen.util 62 :invoke :write [20 22] INFO [2022-01-10 12:52:54,276] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:54,278] jepsen worker 46 - jepsen.util 46 :ok :write [15 31] INFO [2022-01-10 12:52:54,278] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:54,278] jepsen worker 23 - jepsen.util 23 :ok :read [7 3] INFO [2022-01-10 12:52:54,279] jepsen worker 27 - jepsen.util 27 :ok :read [9 7] INFO [2022-01-10 12:52:54,279] jepsen worker 0 - jepsen.util 0 :ok :write [0 19] INFO [2022-01-10 12:52:54,280] jepsen worker 67 - jepsen.util 67 :ok :write [22 49] INFO [2022-01-10 12:52:54,280] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:54,282] jepsen worker 18 - jepsen.util 18 :ok :read [6 9] INFO [2022-01-10 12:52:54,284] jepsen worker 43 - jepsen.util 43 :ok :write [14 7] INFO [2022-01-10 12:52:54,284] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:54,286] jepsen worker 31 - jepsen.util 31 :ok :write [10 24] INFO [2022-01-10 12:52:54,287] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 12:52:54,288] jepsen worker 17 - jepsen.util 17 :ok :read [5 48] INFO [2022-01-10 12:52:54,288] jepsen worker 6 - jepsen.util 6 :ok :write [2 12] INFO [2022-01-10 12:52:54,289] jepsen worker 59 - jepsen.util 59 :ok :read [19 4] INFO [2022-01-10 12:52:54,289] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:54,291] jepsen worker 46 - jepsen.util 46 :ok :read [15 31] INFO [2022-01-10 12:52:54,291] jepsen worker 41 - jepsen.util 41 :ok :write [13 14] INFO [2022-01-10 12:52:54,293] jepsen worker 63 - jepsen.util 63 :invoke :write [21 41] INFO [2022-01-10 12:52:54,297] jepsen worker 85 - jepsen.util 85 :ok :write [28 19] INFO [2022-01-10 12:52:54,298] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:54,299] jepsen worker 72 - jepsen.util 72 :ok :write [24 10] INFO [2022-01-10 12:52:54,299] jepsen worker 46 - jepsen.util 46 :ok :read [15 31] INFO [2022-01-10 12:52:54,300] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:54,302] jepsen worker 67 - jepsen.util 67 :ok :read [22 49] INFO [2022-01-10 12:52:54,302] jepsen worker 35 - jepsen.util 35 :ok :write [11 11] INFO [2022-01-10 12:52:54,303] jepsen worker 3 - jepsen.util 3 :ok :write [1 9] INFO [2022-01-10 12:52:54,305] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:54,305] jepsen worker 62 - jepsen.util 62 :ok :write [20 22] INFO [2022-01-10 12:52:54,306] jepsen worker 9 - jepsen.util 9 :ok :write [3 9] INFO [2022-01-10 12:52:54,306] jepsen worker 46 - jepsen.util 46 :ok :read [15 31] INFO [2022-01-10 12:52:54,308] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:54,310] jepsen worker 27 - jepsen.util 27 :ok :read [9 7] INFO [2022-01-10 12:52:54,311] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:52:54,312] jepsen worker 71 - jepsen.util 71 :ok :write [23 29] INFO [2022-01-10 12:52:54,313] jepsen worker 26 - jepsen.util 26 :ok :write [8 44] INFO [2022-01-10 12:52:54,313] jepsen worker 62 - jepsen.util 62 :ok :read [20 22] INFO [2022-01-10 12:52:54,313] jepsen worker 75 - jepsen.util 75 :ok :write [25 18] INFO [2022-01-10 12:52:54,314] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:54,316] jepsen worker 23 - jepsen.util 23 :ok :read [7 3] INFO [2022-01-10 12:52:54,318] jepsen worker 27 - jepsen.util 27 :invoke :write [9 3] INFO [2022-01-10 12:52:54,319] jepsen worker 78 - jepsen.util 78 :ok :write [26 34] INFO [2022-01-10 12:52:54,320] jepsen worker 21 - jepsen.util 21 :ok :write [7 0] INFO [2022-01-10 12:52:54,321] jepsen worker 58 - jepsen.util 58 :ok :write [19 46] INFO [2022-01-10 12:52:54,321] jepsen worker 55 - jepsen.util 55 :invoke :write [18 15] INFO [2022-01-10 12:52:54,325] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:54,326] jepsen worker 36 - jepsen.util 36 :ok :write [12 27] INFO [2022-01-10 12:52:54,326] jepsen worker 63 - jepsen.util 63 :ok :write [21 41] INFO [2022-01-10 12:52:54,329] jepsen worker 26 - jepsen.util 26 :ok :read [8 44] INFO [2022-01-10 12:52:54,329] jepsen worker 75 - jepsen.util 75 :invoke :write [25 25] INFO [2022-01-10 12:52:54,334] jepsen worker 48 - jepsen.util 48 :ok :write [16 10] INFO [2022-01-10 12:52:54,335] jepsen worker 28 - jepsen.util 28 :ok :write [9 49] INFO [2022-01-10 12:52:54,335] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 12:52:54,336] jepsen worker 6 - jepsen.util 6 :ok :read [2 12] INFO [2022-01-10 12:52:54,341] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:54,341] jepsen worker 88 - jepsen.util 88 :ok :write [29 38] INFO [2022-01-10 12:52:54,342] jepsen worker 43 - jepsen.util 43 :ok :read [14 7] INFO [2022-01-10 12:52:54,345] jepsen worker 2 - jepsen.util 2 :ok :write [0 3] INFO [2022-01-10 12:52:54,347] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:52:54,348] jepsen worker 35 - jepsen.util 35 :ok :read [11 11] INFO [2022-01-10 12:52:54,352] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:52:54,354] jepsen worker 3 - jepsen.util 3 :ok :read [1 9] INFO [2022-01-10 12:52:54,355] jepsen worker 13 - jepsen.util 13 :ok :write [4 25] INFO [2022-01-10 12:52:54,357] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:52:54,359] jepsen worker 72 - jepsen.util 72 :ok :read [24 10] INFO [2022-01-10 12:52:54,362] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:54,363] jepsen worker 18 - jepsen.util 18 :ok :read [6 9] INFO [2022-01-10 12:52:54,365] jepsen worker 32 - jepsen.util 32 :ok :write [10 38] INFO [2022-01-10 12:52:54,365] jepsen worker 39 - jepsen.util 39 :ok :write [13 12] INFO [2022-01-10 12:52:54,366] jepsen worker 6 - jepsen.util 6 :invoke :write [2 36] INFO [2022-01-10 12:52:54,369] jepsen worker 82 - jepsen.util 82 :ok :write [27 47] INFO [2022-01-10 12:52:54,369] jepsen worker 27 - jepsen.util 27 :ok :write [9 3] INFO [2022-01-10 12:52:54,371] jepsen worker 26 - jepsen.util 26 :invoke :write [8 26] INFO [2022-01-10 12:52:54,375] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:54,377] jepsen worker 71 - jepsen.util 71 :ok :read [23 29] INFO [2022-01-10 12:52:54,379] jepsen worker 18 - jepsen.util 18 :invoke :write [6 13] INFO [2022-01-10 12:52:54,379] jepsen worker 4 - jepsen.util 4 :ok :write [1 20] INFO [2022-01-10 12:52:54,381] jepsen worker 26 - jepsen.util 26 :ok :write [8 26] INFO [2022-01-10 12:52:54,383] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:54,385] jepsen worker 51 - jepsen.util 51 :ok :read [17 11] INFO [2022-01-10 12:52:54,385] jepsen worker 45 - jepsen.util 45 :ok :write [15 7] INFO [2022-01-10 12:52:54,387] jepsen worker 17 - jepsen.util 17 :invoke :write [5 25] INFO [2022-01-10 12:52:54,391] jepsen worker 39 - jepsen.util 39 :invoke :write [13 41] INFO [2022-01-10 12:52:54,395] jepsen worker 33 - jepsen.util 33 :ok :write [11 36] INFO [2022-01-10 12:52:54,396] jepsen worker 85 - jepsen.util 85 :invoke :write [28 14] INFO [2022-01-10 12:52:54,397] jepsen worker 18 - jepsen.util 18 :ok :write [6 13] INFO [2022-01-10 12:52:54,399] jepsen worker 6 - jepsen.util 6 :ok :write [2 36] INFO [2022-01-10 12:52:54,400] jepsen worker 55 - jepsen.util 55 :ok :write [18 15] INFO [2022-01-10 12:52:54,400] jepsen worker 61 - jepsen.util 61 :ok :write [20 11] INFO [2022-01-10 12:52:54,401] jepsen worker 46 - jepsen.util 46 :invoke :write [15 48] INFO [2022-01-10 12:52:54,402] jepsen worker 24 - jepsen.util 24 :ok :write [8 29] INFO [2022-01-10 12:52:54,404] jepsen worker 52 - jepsen.util 52 :ok :write [17 26] INFO [2022-01-10 12:52:54,407] jepsen worker 71 - jepsen.util 71 :invoke :write [23 17] INFO [2022-01-10 12:52:54,408] jepsen worker 77 - jepsen.util 77 :ok :write [25 33] INFO [2022-01-10 12:52:54,410] jepsen worker 87 - jepsen.util 87 :ok :write [29 21] INFO [2022-01-10 12:52:54,412] jepsen worker 89 - jepsen.util 89 :invoke :write [29 13] INFO [2022-01-10 12:52:54,416] jepsen worker 30 - jepsen.util 30 :ok :write [10 23] INFO [2022-01-10 12:52:54,417] jepsen worker 33 - jepsen.util 33 :invoke :write [11 3] INFO [2022-01-10 12:52:54,421] jepsen worker 61 - jepsen.util 61 :invoke :write [20 29] INFO [2022-01-10 12:52:54,422] jepsen worker 16 - jepsen.util 16 :ok :write [5 2] INFO [2022-01-10 12:52:54,424] jepsen worker 39 - jepsen.util 39 :ok :write [13 41] INFO [2022-01-10 12:52:54,424] jepsen worker 85 - jepsen.util 85 :ok :write [28 14] INFO [2022-01-10 12:52:54,425] jepsen worker 30 - jepsen.util 30 :invoke :write [10 46] INFO [2022-01-10 12:52:54,429] jepsen worker 22 - jepsen.util 22 :ok :write [7 42] INFO [2022-01-10 12:52:54,430] jepsen worker 85 - jepsen.util 85 :invoke :write [28 28] INFO [2022-01-10 12:52:54,433] jepsen worker 42 - jepsen.util 42 :ok :write [14 27] INFO [2022-01-10 12:52:54,434] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:54,437] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:52:54,437] jepsen worker 71 - jepsen.util 71 :ok :write [23 17] INFO [2022-01-10 12:52:54,439] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:54,439] jepsen worker 33 - jepsen.util 33 :ok :write [11 3] INFO [2022-01-10 12:52:54,440] jepsen worker 38 - jepsen.util 38 :ok :write [12 40] INFO [2022-01-10 12:52:54,440] jepsen worker 48 - jepsen.util 48 :ok :read [16 10] INFO [2022-01-10 12:52:54,443] jepsen worker 26 - jepsen.util 26 :invoke :write [8 24] INFO [2022-01-10 12:52:54,447] jepsen worker 8 - jepsen.util 8 :ok :write [2 25] INFO [2022-01-10 12:52:54,447] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:54,449] jepsen worker 0 - jepsen.util 0 :ok :read [0 3] INFO [2022-01-10 12:52:54,451] jepsen worker 61 - jepsen.util 61 :ok :write [20 29] INFO [2022-01-10 12:52:54,453] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:54,454] jepsen worker 77 - jepsen.util 77 :ok :read [25 33] INFO [2022-01-10 12:52:54,456] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:52:54,458] jepsen worker 52 - jepsen.util 52 :ok :read [17 26] INFO [2022-01-10 12:52:54,458] jepsen worker 68 - jepsen.util 68 :ok :write [22 8] INFO [2022-01-10 12:52:54,461] jepsen worker 67 - jepsen.util 67 :invoke :write [22 13] INFO [2022-01-10 12:52:54,462] jepsen worker 17 - jepsen.util 17 :ok :write [5 25] INFO [2022-01-10 12:52:54,465] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:54,467] jepsen worker 77 - jepsen.util 77 :ok :read [25 33] INFO [2022-01-10 12:52:54,470] jepsen worker 33 - jepsen.util 33 :invoke :write [11 49] INFO [2022-01-10 12:52:54,474] jepsen worker 32 - jepsen.util 32 :invoke :read [10 nil] INFO [2022-01-10 12:52:54,476] jepsen worker 32 - jepsen.util 32 :ok :read [10 23] INFO [2022-01-10 12:52:54,480] jepsen worker 63 - jepsen.util 63 :invoke :write [21 24] INFO [2022-01-10 12:52:54,484] jepsen worker 59 - jepsen.util 59 :invoke :write [19 34] INFO [2022-01-10 12:52:54,489] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:54,490] jepsen worker 18 - jepsen.util 18 :ok :read [6 13] INFO [2022-01-10 12:52:54,492] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:54,493] jepsen worker 58 - jepsen.util 58 :ok :read [19 46] INFO [2022-01-10 12:52:54,495] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:54,497] jepsen worker 48 - jepsen.util 48 :ok :read [16 10] INFO [2022-01-10 12:52:54,499] jepsen worker 44 - jepsen.util 44 :invoke :write [14 34] INFO [2022-01-10 12:52:54,502] jepsen worker 6 - jepsen.util 6 :invoke :write [2 45] INFO [2022-01-10 12:52:54,506] jepsen worker 27 - jepsen.util 27 :invoke :write [9 23] INFO [2022-01-10 12:52:54,509] jepsen worker 87 - jepsen.util 87 :invoke :write [29 10] INFO [2022-01-10 12:52:54,513] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:54,514] jepsen worker 82 - jepsen.util 82 :ok :read [27 47] INFO [2022-01-10 12:52:54,517] jepsen worker 55 - jepsen.util 55 :invoke :write [18 15] INFO [2022-01-10 12:52:54,521] jepsen worker 35 - jepsen.util 35 :invoke :write [11 30] INFO [2022-01-10 12:52:54,524] jepsen worker 82 - jepsen.util 82 :invoke :write [27 19] INFO [2022-01-10 12:52:54,527] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:54,529] jepsen worker 0 - jepsen.util 0 :ok :read [0 3] INFO [2022-01-10 12:52:54,531] jepsen worker 22 - jepsen.util 22 :invoke :write [7 12] INFO [2022-01-10 12:52:54,534] jepsen worker 78 - jepsen.util 78 :invoke :write [26 42] INFO [2022-01-10 12:52:54,538] jepsen worker 18 - jepsen.util 18 :invoke :write [6 7] INFO [2022-01-10 12:52:54,541] jepsen worker 12 - jepsen.util 12 :invoke :write [4 38] INFO [2022-01-10 12:52:54,543] jepsen worker 71 - jepsen.util 71 :invoke :write [23 26] INFO [2022-01-10 12:52:54,546] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:52:54,547] jepsen worker 72 - jepsen.util 72 :ok :read [24 10] INFO [2022-01-10 12:52:54,549] jepsen worker 88 - jepsen.util 88 :invoke :read [29 nil] INFO [2022-01-10 12:52:54,550] jepsen worker 88 - jepsen.util 88 :ok :read [29 21] INFO [2022-01-10 12:52:54,552] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:52:54,553] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:52:54,555] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:54,556] jepsen worker 77 - jepsen.util 77 :ok :read [25 33] INFO [2022-01-10 12:52:54,558] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 12:52:54,560] jepsen worker 17 - jepsen.util 17 :ok :read [5 25] INFO [2022-01-10 12:52:54,561] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:52:54,562] jepsen worker 13 - jepsen.util 13 :ok :read [4 25] INFO [2022-01-10 12:52:54,564] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:54,565] jepsen worker 77 - jepsen.util 77 :ok :read [25 33] INFO [2022-01-10 12:52:54,567] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:54,568] jepsen worker 2 - jepsen.util 2 :ok :read [0 3] INFO [2022-01-10 12:52:54,570] jepsen worker 68 - jepsen.util 68 :invoke :write [22 37] INFO [2022-01-10 12:52:54,573] jepsen worker 17 - jepsen.util 17 :invoke :write [5 29] INFO [2022-01-10 12:52:54,576] jepsen worker 77 - jepsen.util 77 :invoke :write [25 19] INFO [2022-01-10 12:52:54,579] jepsen worker 45 - jepsen.util 45 :invoke :write [15 13] INFO [2022-01-10 12:52:54,581] jepsen worker 46 - jepsen.util 46 :ok :write [15 48] INFO [2022-01-10 12:52:54,581] jepsen worker 29 - jepsen.util 29 :ok :write [9 0] INFO [2022-01-10 12:52:54,582] jepsen worker 16 - jepsen.util 16 :invoke :write [5 37] INFO [2022-01-10 12:52:54,583] jepsen worker 85 - jepsen.util 85 :ok :write [28 28] INFO [2022-01-10 12:52:54,585] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:54,585] jepsen worker 89 - jepsen.util 89 :ok :write [29 13] INFO [2022-01-10 12:52:54,586] jepsen worker 4 - jepsen.util 4 :ok :read [1 20] INFO [2022-01-10 12:52:54,588] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:54,589] jepsen worker 48 - jepsen.util 48 :ok :read [16 10] INFO [2022-01-10 12:52:54,591] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:52:54,592] jepsen worker 75 - jepsen.util 75 :ok :write [25 25] INFO [2022-01-10 12:52:54,592] jepsen worker 72 - jepsen.util 72 :ok :read [24 10] INFO [2022-01-10 12:52:54,595] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:54,596] jepsen worker 51 - jepsen.util 51 :ok :read [17 26] INFO [2022-01-10 12:52:54,599] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:54,600] jepsen worker 85 - jepsen.util 85 :ok :read [28 28] INFO [2022-01-10 12:52:54,603] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:54,603] jepsen worker 5 - jepsen.util 5 :ok :write [1 16] INFO [2022-01-10 12:52:54,604] jepsen worker 4 - jepsen.util 4 :ok :read [1 16] INFO [2022-01-10 12:52:54,607] jepsen worker 52 - jepsen.util 52 :invoke :write [17 24] INFO [2022-01-10 12:52:54,611] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:52:54,612] jepsen worker 62 - jepsen.util 62 :ok :read [20 29] INFO [2022-01-10 12:52:54,613] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:54,615] jepsen worker 36 - jepsen.util 36 :ok :read [12 40] INFO [2022-01-10 12:52:54,616] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:54,618] jepsen worker 86 - jepsen.util 86 :ok :write [28 39] INFO [2022-01-10 12:52:54,618] jepsen worker 48 - jepsen.util 48 :ok :read [16 10] INFO [2022-01-10 12:52:54,620] jepsen worker 8 - jepsen.util 8 :invoke :write [2 16] INFO [2022-01-10 12:52:54,622] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:52:54,624] jepsen worker 5 - jepsen.util 5 :ok :read [1 16] INFO [2022-01-10 12:52:54,625] jepsen worker 31 - jepsen.util 31 :invoke :write [10 25] INFO [2022-01-10 12:52:54,627] jepsen worker 87 - jepsen.util 87 :ok :write [29 10] INFO [2022-01-10 12:52:54,628] jepsen worker 13 - jepsen.util 13 :invoke :write [4 35] INFO [2022-01-10 12:52:54,631] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:52:54,633] jepsen worker 24 - jepsen.util 24 :ok :read [8 29] INFO [2022-01-10 12:52:54,634] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:54,635] jepsen worker 69 - jepsen.util 69 :ok :write [23 5] INFO [2022-01-10 12:52:54,635] jepsen worker 46 - jepsen.util 46 :ok :read [15 48] INFO [2022-01-10 12:52:54,636] jepsen worker 6 - jepsen.util 6 :ok :write [2 45] INFO [2022-01-10 12:52:54,636] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:54,637] jepsen worker 51 - jepsen.util 51 :ok :read [17 26] INFO [2022-01-10 12:52:54,639] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:54,641] jepsen worker 48 - jepsen.util 48 :ok :read [16 10] INFO [2022-01-10 12:52:54,642] jepsen worker 62 - jepsen.util 62 :invoke :write [20 41] INFO [2022-01-10 12:52:54,644] jepsen worker 42 - jepsen.util 42 :invoke :write [14 10] INFO [2022-01-10 12:52:54,647] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:54,648] jepsen worker 53 - jepsen.util 53 :ok :write [17 17] INFO [2022-01-10 12:52:54,648] jepsen worker 48 - jepsen.util 48 :ok :read [16 10] INFO [2022-01-10 12:52:54,649] jepsen worker 67 - jepsen.util 67 :ok :write [22 13] INFO [2022-01-10 12:52:54,650] jepsen worker 2 - jepsen.util 2 :invoke :write [0 33] INFO [2022-01-10 12:52:54,653] jepsen worker 17 - jepsen.util 17 :ok :write [5 29] INFO [2022-01-10 12:52:54,653] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:54,654] jepsen worker 78 - jepsen.util 78 :ok :write [26 42] INFO [2022-01-10 12:52:54,655] jepsen worker 48 - jepsen.util 48 :ok :read [16 10] INFO [2022-01-10 12:52:54,656] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:54,658] jepsen worker 43 - jepsen.util 43 :ok :read [14 27] INFO [2022-01-10 12:52:54,659] jepsen worker 67 - jepsen.util 67 :invoke :write [22 45] INFO [2022-01-10 12:52:54,660] jepsen worker 33 - jepsen.util 33 :ok :write [11 49] INFO [2022-01-10 12:52:54,662] jepsen worker 61 - jepsen.util 61 :invoke :write [20 32] INFO [2022-01-10 12:52:54,662] jepsen worker 12 - jepsen.util 12 :ok :write [4 38] INFO [2022-01-10 12:52:54,664] jepsen worker 0 - jepsen.util 0 :invoke :write [0 13] INFO [2022-01-10 12:52:54,667] jepsen worker 32 - jepsen.util 32 :invoke :write [10 36] INFO [2022-01-10 12:52:54,667] jepsen worker 44 - jepsen.util 44 :ok :write [14 34] INFO [2022-01-10 12:52:54,673] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:54,673] jepsen worker 51 - jepsen.util 51 :invoke :write [17 18] INFO [2022-01-10 12:52:54,675] jepsen worker 78 - jepsen.util 78 :ok :read [26 42] INFO [2022-01-10 12:52:54,676] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:54,677] jepsen worker 36 - jepsen.util 36 :ok :read [12 40] INFO [2022-01-10 12:52:54,679] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:54,680] jepsen worker 4 - jepsen.util 4 :ok :read [1 16] INFO [2022-01-10 12:52:54,681] jepsen worker 36 - jepsen.util 36 :invoke :write [12 15] INFO [2022-01-10 12:52:54,684] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:54,686] jepsen worker 9 - jepsen.util 9 :ok :read [3 9] INFO [2022-01-10 12:52:54,687] jepsen worker 78 - jepsen.util 78 :invoke :write [26 31] INFO [2022-01-10 12:52:54,689] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:54,691] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:52:54,693] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:52:54,694] jepsen worker 75 - jepsen.util 75 :ok :read [25 25] INFO [2022-01-10 12:52:54,696] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 12:52:54,697] jepsen worker 17 - jepsen.util 17 :ok :read [5 29] INFO [2022-01-10 12:52:54,699] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:54,700] jepsen worker 12 - jepsen.util 12 :ok :read [4 38] INFO [2022-01-10 12:52:54,701] jepsen worker 75 - jepsen.util 75 :invoke :write [25 16] INFO [2022-01-10 12:52:54,705] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:54,706] jepsen worker 9 - jepsen.util 9 :ok :read [3 9] INFO [2022-01-10 12:52:54,708] jepsen worker 41 - jepsen.util 41 :invoke :write [13 7] INFO [2022-01-10 12:52:54,710] jepsen worker 17 - jepsen.util 17 :invoke :write [5 40] INFO [2022-01-10 12:52:54,712] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:54,714] jepsen worker 46 - jepsen.util 46 :ok :read [15 48] INFO [2022-01-10 12:52:54,715] jepsen worker 55 - jepsen.util 55 :ok :write [18 15] INFO [2022-01-10 12:52:54,715] jepsen worker 26 - jepsen.util 26 :ok :write [8 24] INFO [2022-01-10 12:52:54,715] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:52:54,717] jepsen worker 29 - jepsen.util 29 :ok :read [9 0] INFO [2022-01-10 12:52:54,718] jepsen worker 46 - jepsen.util 46 :invoke :write [15 10] INFO [2022-01-10 12:52:54,721] jepsen worker 12 - jepsen.util 12 :invoke :write [4 16] INFO [2022-01-10 12:52:54,723] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:52:54,724] jepsen worker 55 - jepsen.util 55 :ok :read [18 15] INFO [2022-01-10 12:52:54,725] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:52:54,727] jepsen worker 72 - jepsen.util 72 :ok :read [24 10] INFO [2022-01-10 12:52:54,727] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:54,729] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:52:54,731] jepsen worker 33 - jepsen.util 33 :invoke :write [11 39] INFO [2022-01-10 12:52:54,733] jepsen worker 29 - jepsen.util 29 :invoke :write [9 26] INFO [2022-01-10 12:52:54,735] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:52:54,737] jepsen worker 24 - jepsen.util 24 :ok :read [8 24] INFO [2022-01-10 12:52:54,738] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:52:54,739] jepsen worker 21 - jepsen.util 21 :ok :read [7 42] INFO [2022-01-10 12:52:54,741] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:54,742] jepsen worker 85 - jepsen.util 85 :ok :read [28 39] INFO [2022-01-10 12:52:54,743] jepsen worker 86 - jepsen.util 86 :invoke :write [28 0] INFO [2022-01-10 12:52:54,745] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:52:54,746] jepsen worker 55 - jepsen.util 55 :ok :read [18 15] INFO [2022-01-10 12:52:54,748] jepsen worker 72 - jepsen.util 72 :invoke :write [24 10] INFO [2022-01-10 12:52:54,750] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:54,751] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:52:54,752] jepsen worker 88 - jepsen.util 88 :invoke :write [29 23] INFO [2022-01-10 12:52:54,755] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:54,756] jepsen worker 58 - jepsen.util 58 :ok :read [19 46] INFO [2022-01-10 12:52:54,757] jepsen worker 53 - jepsen.util 53 :invoke :write [17 17] INFO [2022-01-10 12:52:54,759] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:54,760] jepsen worker 69 - jepsen.util 69 :ok :read [23 5] INFO [2022-01-10 12:52:54,760] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:54,762] jepsen worker 48 - jepsen.util 48 :ok :read [16 10] INFO [2022-01-10 12:52:54,763] jepsen worker 43 - jepsen.util 43 :invoke :write [14 24] INFO [2022-01-10 12:52:54,765] jepsen worker 38 - jepsen.util 38 :invoke :write [12 4] INFO [2022-01-10 12:52:54,767] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:54,768] jepsen worker 36 - jepsen.util 36 :ok :write [12 15] INFO [2022-01-10 12:52:54,768] jepsen worker 85 - jepsen.util 85 :ok :read [28 39] INFO [2022-01-10 12:52:54,769] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:52:54,771] jepsen worker 24 - jepsen.util 24 :ok :read [8 24] INFO [2022-01-10 12:52:54,772] jepsen worker 28 - jepsen.util 28 :invoke :write [9 13] INFO [2022-01-10 12:52:54,773] jepsen worker 84 - jepsen.util 84 :ok :write [28 29] INFO [2022-01-10 12:52:54,775] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:52:54,777] jepsen worker 55 - jepsen.util 55 :ok :read [18 15] INFO [2022-01-10 12:52:54,778] jepsen worker 40 - jepsen.util 40 :ok :write [13 28] INFO [2022-01-10 12:52:54,778] jepsen worker 71 - jepsen.util 71 :ok :write [23 26] INFO [2022-01-10 12:52:54,779] jepsen worker 6 - jepsen.util 6 :invoke :write [2 31] INFO [2022-01-10 12:52:54,780] jepsen worker 82 - jepsen.util 82 :ok :write [27 19] INFO [2022-01-10 12:52:54,781] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:54,783] jepsen worker 44 - jepsen.util 44 :ok :read [14 34] INFO [2022-01-10 12:52:54,784] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:54,785] jepsen worker 52 - jepsen.util 52 :ok :write [17 24] INFO [2022-01-10 12:52:54,785] jepsen worker 23 - jepsen.util 23 :ok :read [7 42] INFO [2022-01-10 12:52:54,786] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:54,787] jepsen worker 89 - jepsen.util 89 :ok :read [29 10] INFO [2022-01-10 12:52:54,788] jepsen worker 22 - jepsen.util 22 :ok :write [7 12] INFO [2022-01-10 12:52:54,789] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:54,790] jepsen worker 78 - jepsen.util 78 :ok :write [26 31] INFO [2022-01-10 12:52:54,790] jepsen worker 2 - jepsen.util 2 :ok :write [0 33] INFO [2022-01-10 12:52:54,791] jepsen worker 69 - jepsen.util 69 :ok :read [23 26] INFO [2022-01-10 12:52:54,791] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:54,793] jepsen worker 87 - jepsen.util 87 :ok :read [29 10] INFO [2022-01-10 12:52:54,794] jepsen worker 77 - jepsen.util 77 :ok :write [25 19] INFO [2022-01-10 12:52:54,794] jepsen worker 38 - jepsen.util 38 :ok :write [12 4] INFO [2022-01-10 12:52:54,795] jepsen worker 24 - jepsen.util 24 :invoke :write [8 11] INFO [2022-01-10 12:52:54,798] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:54,800] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:54,801] jepsen worker 26 - jepsen.util 26 :ok :read [8 24] INFO [2022-01-10 12:52:54,802] jepsen worker 8 - jepsen.util 8 :ok :write [2 16] INFO [2022-01-10 12:52:54,802] jepsen worker 85 - jepsen.util 85 :ok :read [28 29] INFO [2022-01-10 12:52:54,802] jepsen worker 13 - jepsen.util 13 :ok :write [4 35] INFO [2022-01-10 12:52:54,804] jepsen worker 78 - jepsen.util 78 :invoke :write [26 37] INFO [2022-01-10 12:52:54,807] jepsen worker 62 - jepsen.util 62 :ok :write [20 41] INFO [2022-01-10 12:52:54,809] jepsen worker 82 - jepsen.util 82 :invoke :write [27 5] INFO [2022-01-10 12:52:54,809] jepsen worker 86 - jepsen.util 86 :ok :write [28 0] INFO [2022-01-10 12:52:54,811] jepsen worker 88 - jepsen.util 88 :ok :write [29 23] INFO [2022-01-10 12:52:54,812] jepsen worker 26 - jepsen.util 26 :invoke :write [8 5] INFO [2022-01-10 12:52:54,814] jepsen worker 41 - jepsen.util 41 :ok :write [13 7] INFO [2022-01-10 12:52:54,815] jepsen worker 67 - jepsen.util 67 :ok :write [22 45] INFO [2022-01-10 12:52:54,815] jepsen worker 21 - jepsen.util 21 :invoke :write [7 20] INFO [2022-01-10 12:52:54,819] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:54,821] jepsen worker 67 - jepsen.util 67 :ok :read [22 45] INFO [2022-01-10 12:52:54,821] jepsen worker 13 - jepsen.util 13 :invoke :write [4 16] INFO [2022-01-10 12:52:54,822] jepsen worker 51 - jepsen.util 51 :ok :write [17 18] INFO [2022-01-10 12:52:54,823] jepsen worker 16 - jepsen.util 16 :ok :write [5 37] INFO [2022-01-10 12:52:54,824] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:54,825] jepsen worker 9 - jepsen.util 9 :ok :read [3 9] INFO [2022-01-10 12:52:54,826] jepsen worker 86 - jepsen.util 86 :invoke :write [28 23] INFO [2022-01-10 12:52:54,827] jepsen worker 78 - jepsen.util 78 :ok :write [26 37] INFO [2022-01-10 12:52:54,828] jepsen worker 35 - jepsen.util 35 :ok :write [11 30] INFO [2022-01-10 12:52:54,829] jepsen worker 85 - jepsen.util 85 :invoke :write [28 14] INFO [2022-01-10 12:52:54,832] jepsen worker 27 - jepsen.util 27 :ok :write [9 23] INFO [2022-01-10 12:52:54,832] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:52:54,833] jepsen worker 30 - jepsen.util 30 :ok :write [10 46] INFO [2022-01-10 12:52:54,833] jepsen worker 62 - jepsen.util 62 :ok :read [20 41] INFO [2022-01-10 12:52:54,835] jepsen worker 89 - jepsen.util 89 :invoke :write [29 10] INFO [2022-01-10 12:52:54,836] jepsen worker 42 - jepsen.util 42 :ok :write [14 10] INFO [2022-01-10 12:52:54,839] jepsen worker 13 - jepsen.util 13 :ok :write [4 16] INFO [2022-01-10 12:52:54,839] jepsen worker 19 - jepsen.util 19 :ok :write [6 21] INFO [2022-01-10 12:52:54,840] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:54,841] jepsen worker 69 - jepsen.util 69 :ok :read [23 26] INFO [2022-01-10 12:52:54,843] jepsen worker 44 - jepsen.util 44 :invoke :write [14 41] INFO [2022-01-10 12:52:54,844] jepsen worker 65 - jepsen.util 65 :ok :write [21 37] INFO [2022-01-10 12:52:54,844] jepsen worker 49 - jepsen.util 49 :ok :write [16 4] INFO [2022-01-10 12:52:54,845] jepsen worker 86 - jepsen.util 86 :ok :write [28 23] INFO [2022-01-10 12:52:54,846] jepsen worker 78 - jepsen.util 78 :invoke :write [26 35] INFO [2022-01-10 12:52:54,847] jepsen worker 1 - jepsen.util 1 :ok :write [0 12] INFO [2022-01-10 12:52:54,849] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:54,851] jepsen worker 67 - jepsen.util 67 :ok :read [22 45] INFO [2022-01-10 12:52:54,851] jepsen worker 73 - jepsen.util 73 :ok :write [24 43] INFO [2022-01-10 12:52:54,853] jepsen worker 82 - jepsen.util 82 :ok :write [27 5] INFO [2022-01-10 12:52:54,854] jepsen worker 9 - jepsen.util 9 :invoke :write [3 16] INFO [2022-01-10 12:52:54,854] jepsen worker 53 - jepsen.util 53 :ok :write [17 17] INFO [2022-01-10 12:52:54,855] jepsen worker 59 - jepsen.util 59 :ok :write [19 34] INFO [2022-01-10 12:52:54,859] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:54,859] jepsen worker 21 - jepsen.util 21 :ok :write [7 20] INFO [2022-01-10 12:52:54,860] jepsen worker 69 - jepsen.util 69 :ok :read [23 26] INFO [2022-01-10 12:52:54,862] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:52:54,863] jepsen worker 3 - jepsen.util 3 :ok :read [1 16] INFO [2022-01-10 12:52:54,863] jepsen worker 28 - jepsen.util 28 :ok :write [9 13] INFO [2022-01-10 12:52:54,864] jepsen worker 45 - jepsen.util 45 :ok :write [15 13] INFO [2022-01-10 12:52:54,865] jepsen worker 88 - jepsen.util 88 :invoke :read [29 nil] INFO [2022-01-10 12:52:54,867] jepsen worker 75 - jepsen.util 75 :ok :write [25 16] INFO [2022-01-10 12:52:54,867] jepsen worker 88 - jepsen.util 88 :ok :read [29 23] INFO [2022-01-10 12:52:54,867] jepsen worker 89 - jepsen.util 89 :ok :write [29 10] INFO [2022-01-10 12:52:54,870] jepsen worker 55 - jepsen.util 55 :invoke :write [18 31] INFO [2022-01-10 12:52:54,871] jepsen worker 24 - jepsen.util 24 :ok :write [8 11] INFO [2022-01-10 12:52:54,873] jepsen worker 78 - jepsen.util 78 :ok :write [26 35] INFO [2022-01-10 12:52:54,874] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:54,875] jepsen worker 6 - jepsen.util 6 :ok :write [2 31] INFO [2022-01-10 12:52:54,875] jepsen worker 11 - jepsen.util 11 :ok :write [3 41] INFO [2022-01-10 12:52:54,876] jepsen worker 16 - jepsen.util 16 :ok :read [5 37] INFO [2022-01-10 12:52:54,876] jepsen worker 0 - jepsen.util 0 :ok :write [0 13] INFO [2022-01-10 12:52:54,878] jepsen worker 32 - jepsen.util 32 :ok :write [10 36] INFO [2022-01-10 12:52:54,880] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:54,881] jepsen worker 60 - jepsen.util 60 :ok :write [20 28] INFO [2022-01-10 12:52:54,882] jepsen worker 18 - jepsen.util 18 :ok :write [6 7] INFO [2022-01-10 12:52:54,882] jepsen worker 69 - jepsen.util 69 :ok :read [23 26] INFO [2022-01-10 12:52:54,883] jepsen worker 54 - jepsen.util 54 :ok :write [18 12] INFO [2022-01-10 12:52:54,883] jepsen worker 52 - jepsen.util 52 :invoke :write [17 36] INFO [2022-01-10 12:52:54,884] jepsen worker 85 - jepsen.util 85 :ok :write [28 14] INFO [2022-01-10 12:52:54,889] jepsen worker 73 - jepsen.util 73 :invoke :write [24 44] INFO [2022-01-10 12:52:54,892] jepsen worker 72 - jepsen.util 72 :ok :write [24 10] INFO [2022-01-10 12:52:54,893] jepsen worker 54 - jepsen.util 54 :invoke :write [18 15] INFO [2022-01-10 12:52:54,897] jepsen worker 11 - jepsen.util 11 :invoke :write [3 15] INFO [2022-01-10 12:52:54,898] jepsen worker 33 - jepsen.util 33 :ok :write [11 39] INFO [2022-01-10 12:52:54,899] jepsen worker 68 - jepsen.util 68 :ok :write [22 37] INFO [2022-01-10 12:52:54,901] jepsen worker 65 - jepsen.util 65 :invoke :write [21 16] INFO [2022-01-10 12:52:54,903] jepsen worker 43 - jepsen.util 43 :ok :write [14 24] INFO [2022-01-10 12:52:54,906] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:54,906] jepsen worker 26 - jepsen.util 26 :ok :write [8 5] INFO [2022-01-10 12:52:54,907] jepsen worker 71 - jepsen.util 71 :ok :read [23 26] INFO [2022-01-10 12:52:54,909] jepsen worker 9 - jepsen.util 9 :ok :write [3 16] INFO [2022-01-10 12:52:54,910] jepsen worker 58 - jepsen.util 58 :invoke :write [19 30] INFO [2022-01-10 12:52:54,913] jepsen worker 9 - jepsen.util 9 :invoke :write [3 6] INFO [2022-01-10 12:52:54,914] jepsen worker 81 - jepsen.util 81 :ok :write [27 11] INFO [2022-01-10 12:52:54,916] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:54,918] jepsen worker 18 - jepsen.util 18 :ok :read [6 7] INFO [2022-01-10 12:52:54,920] jepsen worker 38 - jepsen.util 38 :invoke :write [12 29] INFO [2022-01-10 12:52:54,923] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:54,925] jepsen worker 2 - jepsen.util 2 :ok :read [0 13] INFO [2022-01-10 12:52:54,926] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:54,928] jepsen worker 36 - jepsen.util 36 :ok :read [12 4] INFO [2022-01-10 12:52:54,929] jepsen worker 73 - jepsen.util 73 :ok :write [24 44] INFO [2022-01-10 12:52:54,930] jepsen worker 29 - jepsen.util 29 :ok :write [9 26] INFO [2022-01-10 12:52:54,930] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:54,932] jepsen worker 89 - jepsen.util 89 :ok :read [29 10] INFO [2022-01-10 12:52:54,934] jepsen worker 71 - jepsen.util 71 :invoke :write [23 4] INFO [2022-01-10 12:52:54,936] jepsen worker 58 - jepsen.util 58 :ok :write [19 30] INFO [2022-01-10 12:52:54,937] jepsen worker 13 - jepsen.util 13 :invoke :write [4 22] INFO [2022-01-10 12:52:54,940] jepsen worker 46 - jepsen.util 46 :ok :write [15 10] INFO [2022-01-10 12:52:54,941] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:52:54,943] jepsen worker 35 - jepsen.util 35 :ok :read [11 39] INFO [2022-01-10 12:52:54,943] jepsen worker 17 - jepsen.util 17 :ok :write [5 40] INFO [2022-01-10 12:52:54,944] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:54,944] jepsen worker 65 - jepsen.util 65 :ok :write [21 16] INFO [2022-01-10 12:52:54,946] jepsen worker 69 - jepsen.util 69 :ok :read [23 26] INFO [2022-01-10 12:52:54,948] jepsen worker 18 - jepsen.util 18 :invoke :write [6 16] INFO [2022-01-10 12:52:54,952] jepsen worker 3 - jepsen.util 3 :invoke :write [1 8] INFO [2022-01-10 12:52:54,955] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:52:54,956] jepsen worker 73 - jepsen.util 73 :ok :read [24 44] INFO [2022-01-10 12:52:54,958] jepsen worker 62 - jepsen.util 62 :invoke :write [20 27] INFO [2022-01-10 12:52:54,961] jepsen worker 60 - jepsen.util 60 :invoke :write [20 3] INFO [2022-01-10 12:52:54,964] jepsen worker 48 - jepsen.util 48 :invoke :write [16 0] INFO [2022-01-10 12:52:54,968] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:54,969] jepsen worker 16 - jepsen.util 16 :ok :read [5 40] INFO [2022-01-10 12:52:54,971] jepsen worker 49 - jepsen.util 49 :invoke :write [16 17] INFO [2022-01-10 12:52:54,974] jepsen worker 32 - jepsen.util 32 :invoke :read [10 nil] INFO [2022-01-10 12:52:54,975] jepsen worker 32 - jepsen.util 32 :ok :read [10 36] INFO [2022-01-10 12:52:54,977] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:52:54,978] jepsen worker 72 - jepsen.util 72 :ok :read [24 44] INFO [2022-01-10 12:52:54,980] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:54,981] jepsen worker 77 - jepsen.util 77 :ok :read [25 16] INFO [2022-01-10 12:52:54,983] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:52:54,985] jepsen worker 5 - jepsen.util 5 :ok :read [1 16] INFO [2022-01-10 12:52:54,986] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:52:54,988] jepsen worker 65 - jepsen.util 65 :ok :read [21 16] INFO [2022-01-10 12:52:54,990] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:54,991] jepsen worker 69 - jepsen.util 69 :ok :read [23 26] INFO [2022-01-10 12:52:54,993] jepsen worker 32 - jepsen.util 32 :invoke :write [10 27] INFO [2022-01-10 12:52:54,996] jepsen worker 2 - jepsen.util 2 :invoke :write [0 46] INFO [2022-01-10 12:52:54,999] jepsen worker 68 - jepsen.util 68 :invoke :write [22 23] INFO [2022-01-10 12:52:55,002] jepsen worker 78 - jepsen.util 78 :invoke :write [26 13] INFO [2022-01-10 12:52:55,005] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:55,007] jepsen worker 58 - jepsen.util 58 :ok :read [19 30] INFO [2022-01-10 12:52:55,009] jepsen worker 46 - jepsen.util 46 :invoke :write [15 46] INFO [2022-01-10 12:52:55,012] jepsen worker 59 - jepsen.util 59 :invoke :write [19 12] INFO [2022-01-10 12:52:55,014] jepsen worker 88 - jepsen.util 88 :invoke :read [29 nil] INFO [2022-01-10 12:52:55,016] jepsen worker 88 - jepsen.util 88 :ok :read [29 10] INFO [2022-01-10 12:52:55,018] jepsen worker 19 - jepsen.util 19 :invoke :write [6 41] INFO [2022-01-10 12:52:55,020] jepsen worker 0 - jepsen.util 0 :invoke :write [0 49] INFO [2022-01-10 12:52:55,023] jepsen worker 42 - jepsen.util 42 :invoke :write [14 1] INFO [2022-01-10 12:52:55,027] jepsen worker 1 - jepsen.util 1 :invoke :write [0 24] INFO [2022-01-10 12:52:55,029] jepsen worker 53 - jepsen.util 53 :invoke :write [17 48] INFO [2022-01-10 12:52:55,032] jepsen worker 26 - jepsen.util 26 :invoke :write [8 34] INFO [2022-01-10 12:52:55,035] jepsen worker 55 - jepsen.util 55 :ok :write [18 31] INFO [2022-01-10 12:52:55,035] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:55,036] jepsen worker 52 - jepsen.util 52 :ok :write [17 36] INFO [2022-01-10 12:52:55,037] jepsen worker 84 - jepsen.util 84 :ok :read [28 14] INFO [2022-01-10 12:52:55,039] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:52:55,040] jepsen worker 65 - jepsen.util 65 :ok :read [21 16] INFO [2022-01-10 12:52:55,043] jepsen worker 55 - jepsen.util 55 :invoke :write [18 4] INFO [2022-01-10 12:52:55,045] jepsen worker 12 - jepsen.util 12 :ok :write [4 16] INFO [2022-01-10 12:52:55,046] jepsen worker 44 - jepsen.util 44 :ok :write [14 41] INFO [2022-01-10 12:52:55,048] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:55,050] jepsen worker 2 - jepsen.util 2 :ok :write [0 46] INFO [2022-01-10 12:52:55,051] jepsen worker 45 - jepsen.util 45 :ok :read [15 10] INFO [2022-01-10 12:52:55,052] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:55,053] jepsen worker 12 - jepsen.util 12 :ok :read [4 16] INFO [2022-01-10 12:52:55,055] jepsen worker 24 - jepsen.util 24 :invoke :write [8 42] INFO [2022-01-10 12:52:55,056] jepsen worker 11 - jepsen.util 11 :ok :write [3 15] INFO [2022-01-10 12:52:55,061] jepsen worker 38 - jepsen.util 38 :ok :write [12 29] INFO [2022-01-10 12:52:55,062] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:55,062] jepsen worker 82 - jepsen.util 82 :invoke :write [27 3] INFO [2022-01-10 12:52:55,063] jepsen worker 85 - jepsen.util 85 :ok :read [28 14] INFO [2022-01-10 12:52:55,064] jepsen worker 1 - jepsen.util 1 :ok :write [0 24] INFO [2022-01-10 12:52:55,065] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:55,067] jepsen worker 58 - jepsen.util 58 :ok :read [19 30] INFO [2022-01-10 12:52:55,067] jepsen worker 71 - jepsen.util 71 :ok :write [23 4] INFO [2022-01-10 12:52:55,068] jepsen worker 22 - jepsen.util 22 :invoke :read [7 nil] INFO [2022-01-10 12:52:55,069] jepsen worker 22 - jepsen.util 22 :ok :read [7 20] INFO [2022-01-10 12:52:55,072] jepsen worker 75 - jepsen.util 75 :invoke :write [25 21] INFO [2022-01-10 12:52:55,075] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:52:55,077] jepsen worker 81 - jepsen.util 81 :ok :read [27 11] INFO [2022-01-10 12:52:55,077] jepsen worker 3 - jepsen.util 3 :ok :write [1 8] INFO [2022-01-10 12:52:55,079] jepsen worker 0 - jepsen.util 0 :ok :write [0 49] INFO [2022-01-10 12:52:55,080] jepsen worker 85 - jepsen.util 85 :invoke :write [28 29] INFO [2022-01-10 12:52:55,082] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 12:52:55,084] jepsen worker 6 - jepsen.util 6 :ok :read [2 31] INFO [2022-01-10 12:52:55,085] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:55,087] jepsen worker 43 - jepsen.util 43 :ok :read [14 41] INFO [2022-01-10 12:52:55,088] jepsen worker 77 - jepsen.util 77 :invoke :write [25 17] INFO [2022-01-10 12:52:55,091] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:55,092] jepsen worker 44 - jepsen.util 44 :ok :read [14 41] INFO [2022-01-10 12:52:55,094] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:55,096] jepsen worker 84 - jepsen.util 84 :ok :read [28 14] INFO [2022-01-10 12:52:55,097] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:55,098] jepsen worker 36 - jepsen.util 36 :ok :read [12 29] INFO [2022-01-10 12:52:55,099] jepsen worker 28 - jepsen.util 28 :invoke :read [9 nil] INFO [2022-01-10 12:52:55,100] jepsen worker 59 - jepsen.util 59 :ok :write [19 12] INFO [2022-01-10 12:52:55,101] jepsen worker 28 - jepsen.util 28 :ok :read [9 26] INFO [2022-01-10 12:52:55,101] jepsen worker 55 - jepsen.util 55 :ok :write [18 4] INFO [2022-01-10 12:52:55,103] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:52:55,104] jepsen worker 3 - jepsen.util 3 :ok :read [1 8] INFO [2022-01-10 12:52:55,106] jepsen worker 62 - jepsen.util 62 :ok :write [20 27] INFO [2022-01-10 12:52:55,106] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 12:52:55,107] jepsen worker 31 - jepsen.util 31 :ok :write [10 25] INFO [2022-01-10 12:52:55,109] jepsen worker 6 - jepsen.util 6 :ok :read [2 31] INFO [2022-01-10 12:52:55,110] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:55,112] jepsen worker 44 - jepsen.util 44 :ok :read [14 41] INFO [2022-01-10 12:52:55,112] jepsen worker 63 - jepsen.util 63 :ok :write [21 24] INFO [2022-01-10 12:52:55,112] jepsen worker 48 - jepsen.util 48 :ok :write [16 0] INFO [2022-01-10 12:52:55,113] jepsen worker 88 - jepsen.util 88 :invoke :write [29 10] INFO [2022-01-10 12:52:55,117] jepsen worker 41 - jepsen.util 41 :invoke :write [13 47] INFO [2022-01-10 12:52:55,119] jepsen worker 18 - jepsen.util 18 :ok :write [6 16] INFO [2022-01-10 12:52:55,120] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:55,122] jepsen worker 45 - jepsen.util 45 :ok :read [15 10] INFO [2022-01-10 12:52:55,122] jepsen worker 68 - jepsen.util 68 :ok :write [22 23] INFO [2022-01-10 12:52:55,123] jepsen worker 85 - jepsen.util 85 :ok :write [28 29] INFO [2022-01-10 12:52:55,123] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:55,124] jepsen worker 88 - jepsen.util 88 :ok :write [29 10] INFO [2022-01-10 12:52:55,124] jepsen worker 74 - jepsen.util 74 :ok :write [24 42] INFO [2022-01-10 12:52:55,125] jepsen worker 18 - jepsen.util 18 :ok :read [6 16] INFO [2022-01-10 12:52:55,127] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:55,128] jepsen worker 42 - jepsen.util 42 :ok :write [14 1] INFO [2022-01-10 12:52:55,128] jepsen worker 69 - jepsen.util 69 :ok :read [23 4] INFO [2022-01-10 12:52:55,129] jepsen worker 78 - jepsen.util 78 :ok :write [26 13] INFO [2022-01-10 12:52:55,130] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:55,131] jepsen worker 47 - jepsen.util 47 :ok :write [15 9] INFO [2022-01-10 12:52:55,132] jepsen worker 11 - jepsen.util 11 :ok :read [3 15] INFO [2022-01-10 12:52:55,132] jepsen worker 41 - jepsen.util 41 :ok :write [13 47] INFO [2022-01-10 12:52:55,134] jepsen worker 63 - jepsen.util 63 :invoke :write [21 2] INFO [2022-01-10 12:52:55,135] jepsen worker 82 - jepsen.util 82 :ok :write [27 3] INFO [2022-01-10 12:52:55,137] jepsen worker 26 - jepsen.util 26 :ok :write [8 34] INFO [2022-01-10 12:52:55,138] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:55,138] jepsen worker 75 - jepsen.util 75 :ok :write [25 21] INFO [2022-01-10 12:52:55,139] jepsen worker 33 - jepsen.util 33 :ok :read [11 39] INFO [2022-01-10 12:52:55,142] jepsen worker 86 - jepsen.util 86 :invoke :write [28 27] INFO [2022-01-10 12:52:55,143] jepsen worker 9 - jepsen.util 9 :ok :write [3 6] INFO [2022-01-10 12:52:55,144] jepsen worker 53 - jepsen.util 53 :ok :write [17 48] INFO [2022-01-10 12:52:55,146] jepsen worker 48 - jepsen.util 48 :invoke :write [16 23] INFO [2022-01-10 12:52:55,150] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:55,151] jepsen worker 13 - jepsen.util 13 :ok :write [4 22] INFO [2022-01-10 12:52:55,151] jepsen worker 38 - jepsen.util 38 :ok :read [12 29] INFO [2022-01-10 12:52:55,153] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:55,153] jepsen worker 60 - jepsen.util 60 :ok :write [20 3] INFO [2022-01-10 12:52:55,154] jepsen worker 0 - jepsen.util 0 :ok :read [0 49] INFO [2022-01-10 12:52:55,157] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:55,159] jepsen worker 58 - jepsen.util 58 :ok :read [19 12] INFO [2022-01-10 12:52:55,161] jepsen worker 63 - jepsen.util 63 :ok :write [21 2] INFO [2022-01-10 12:52:55,162] jepsen worker 17 - jepsen.util 17 :invoke :write [5 20] INFO [2022-01-10 12:52:55,164] jepsen worker 19 - jepsen.util 19 :ok :write [6 41] INFO [2022-01-10 12:52:55,167] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:55,168] jepsen worker 16 - jepsen.util 16 :ok :read [5 40] INFO [2022-01-10 12:52:55,171] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:52:55,172] jepsen worker 77 - jepsen.util 77 :ok :write [25 17] INFO [2022-01-10 12:52:55,173] jepsen worker 81 - jepsen.util 81 :ok :read [27 3] INFO [2022-01-10 12:52:55,175] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:52:55,176] jepsen worker 47 - jepsen.util 47 :ok :read [15 9] INFO [2022-01-10 12:52:55,178] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:55,180] jepsen worker 69 - jepsen.util 69 :ok :read [23 4] INFO [2022-01-10 12:52:55,182] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:55,184] jepsen worker 11 - jepsen.util 11 :ok :read [3 6] INFO [2022-01-10 12:52:55,185] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:55,187] jepsen worker 9 - jepsen.util 9 :ok :read [3 6] INFO [2022-01-10 12:52:55,190] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:52:55,190] jepsen worker 24 - jepsen.util 24 :ok :write [8 42] INFO [2022-01-10 12:52:55,191] jepsen worker 5 - jepsen.util 5 :ok :read [1 8] INFO [2022-01-10 12:52:55,191] jepsen worker 32 - jepsen.util 32 :ok :write [10 27] INFO [2022-01-10 12:52:55,194] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:55,195] jepsen worker 82 - jepsen.util 82 :ok :read [27 3] INFO [2022-01-10 12:52:55,200] jepsen worker 47 - jepsen.util 47 :invoke :write [15 13] INFO [2022-01-10 12:52:55,205] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:55,206] jepsen worker 77 - jepsen.util 77 :ok :read [25 17] INFO [2022-01-10 12:52:55,209] jepsen worker 53 - jepsen.util 53 :invoke :write [17 37] INFO [2022-01-10 12:52:55,212] jepsen worker 54 - jepsen.util 54 :ok :write [18 15] INFO [2022-01-10 12:52:55,213] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:55,215] jepsen worker 60 - jepsen.util 60 :ok :read [20 3] INFO [2022-01-10 12:52:55,217] jepsen worker 46 - jepsen.util 46 :ok :write [15 46] INFO [2022-01-10 12:52:55,217] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:52:55,219] jepsen worker 24 - jepsen.util 24 :ok :read [8 42] INFO [2022-01-10 12:52:55,222] jepsen worker 6 - jepsen.util 6 :invoke :write [2 17] INFO [2022-01-10 12:52:55,226] jepsen worker 30 - jepsen.util 30 :invoke :write [10 45] INFO [2022-01-10 12:52:55,228] jepsen worker 53 - jepsen.util 53 :ok :write [17 37] INFO [2022-01-10 12:52:55,231] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:52:55,232] jepsen worker 47 - jepsen.util 47 :ok :write [15 13] INFO [2022-01-10 12:52:55,233] jepsen worker 62 - jepsen.util 62 :ok :read [20 3] INFO [2022-01-10 12:52:55,236] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:55,237] jepsen worker 69 - jepsen.util 69 :ok :read [23 4] INFO [2022-01-10 12:52:55,240] jepsen worker 79 - jepsen.util 79 :ok :write [26 10] INFO [2022-01-10 12:52:55,241] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 12:52:55,242] jepsen worker 42 - jepsen.util 42 :ok :read [14 1] INFO [2022-01-10 12:52:55,245] jepsen worker 30 - jepsen.util 30 :ok :write [10 45] INFO [2022-01-10 12:52:55,246] jepsen worker 19 - jepsen.util 19 :invoke :write [6 48] INFO [2022-01-10 12:52:55,250] jepsen worker 47 - jepsen.util 47 :invoke :write [15 41] INFO [2022-01-10 12:52:55,252] jepsen worker 86 - jepsen.util 86 :ok :write [28 27] INFO [2022-01-10 12:52:55,254] jepsen worker 49 - jepsen.util 49 :ok :write [16 17] INFO [2022-01-10 12:52:55,254] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:55,254] jepsen worker 61 - jepsen.util 61 :ok :write [20 32] INFO [2022-01-10 12:52:55,255] jepsen worker 38 - jepsen.util 38 :ok :read [12 29] INFO [2022-01-10 12:52:55,256] jepsen worker 57 - jepsen.util 57 :ok :write [19 21] INFO [2022-01-10 12:52:55,257] jepsen worker 17 - jepsen.util 17 :ok :write [5 20] INFO [2022-01-10 12:52:55,258] jepsen worker 19 - jepsen.util 19 :ok :write [6 48] INFO [2022-01-10 12:52:55,258] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:55,260] jepsen worker 82 - jepsen.util 82 :ok :read [27 3] INFO [2022-01-10 12:52:55,263] jepsen worker 9 - jepsen.util 9 :invoke :write [3 10] INFO [2022-01-10 12:52:55,263] jepsen worker 37 - jepsen.util 37 :ok :write [12 0] INFO [2022-01-10 12:52:55,265] jepsen worker 7 - jepsen.util 7 :ok :write [2 46] INFO [2022-01-10 12:52:55,266] jepsen worker 47 - jepsen.util 47 :ok :write [15 41] INFO [2022-01-10 12:52:55,267] jepsen worker 26 - jepsen.util 26 :invoke :write [8 48] INFO [2022-01-10 12:52:55,271] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:55,273] jepsen worker 69 - jepsen.util 69 :ok :read [23 4] INFO [2022-01-10 12:52:55,274] jepsen worker 48 - jepsen.util 48 :ok :write [16 23] INFO [2022-01-10 12:52:55,275] jepsen worker 37 - jepsen.util 37 :invoke :write [12 2] INFO [2022-01-10 12:52:55,277] jepsen worker 9 - jepsen.util 9 :ok :write [3 10] INFO [2022-01-10 12:52:55,279] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 12:52:55,281] jepsen worker 17 - jepsen.util 17 :ok :read [5 20] INFO [2022-01-10 12:52:55,283] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:52:55,284] jepsen worker 8 - jepsen.util 8 :ok :read [2 46] INFO [2022-01-10 12:52:55,284] jepsen worker 37 - jepsen.util 37 :ok :write [12 2] INFO [2022-01-10 12:52:55,287] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:52:55,289] jepsen worker 3 - jepsen.util 3 :ok :read [1 8] INFO [2022-01-10 12:52:55,290] jepsen worker 26 - jepsen.util 26 :ok :write [8 48] INFO [2022-01-10 12:52:55,291] jepsen worker 6 - jepsen.util 6 :ok :write [2 17] INFO [2022-01-10 12:52:55,291] jepsen worker 74 - jepsen.util 74 :invoke :write [24 2] INFO [2022-01-10 12:52:55,293] jepsen worker 14 - jepsen.util 14 :ok :write [4 30] INFO [2022-01-10 12:52:55,295] jepsen worker 30 - jepsen.util 30 :invoke :write [10 18] INFO [2022-01-10 12:52:55,300] jepsen worker 10 - jepsen.util 10 :ok :write [3 43] INFO [2022-01-10 12:52:55,300] jepsen worker 19 - jepsen.util 19 :invoke :write [6 43] INFO [2022-01-10 12:52:55,304] jepsen worker 86 - jepsen.util 86 :invoke :write [28 19] INFO [2022-01-10 12:52:55,305] jepsen worker 74 - jepsen.util 74 :ok :write [24 2] INFO [2022-01-10 12:52:55,307] jepsen worker 37 - jepsen.util 37 :invoke :write [12 13] INFO [2022-01-10 12:52:55,311] jepsen worker 46 - jepsen.util 46 :invoke :write [15 40] INFO [2022-01-10 12:52:55,311] jepsen worker 30 - jepsen.util 30 :ok :write [10 18] INFO [2022-01-10 12:52:55,314] jepsen worker 79 - jepsen.util 79 :invoke :write [26 36] INFO [2022-01-10 12:52:55,318] jepsen worker 67 - jepsen.util 67 :invoke :write [22 27] INFO [2022-01-10 12:52:55,321] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:55,322] jepsen worker 19 - jepsen.util 19 :ok :write [6 43] INFO [2022-01-10 12:52:55,323] jepsen worker 71 - jepsen.util 71 :ok :read [23 4] INFO [2022-01-10 12:52:55,325] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:55,326] jepsen worker 67 - jepsen.util 67 :ok :write [22 27] INFO [2022-01-10 12:52:55,326] jepsen worker 84 - jepsen.util 84 :ok :read [28 27] INFO [2022-01-10 12:52:55,329] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:52:55,330] jepsen worker 29 - jepsen.util 29 :ok :read [9 26] INFO [2022-01-10 12:52:55,333] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:55,335] jepsen worker 48 - jepsen.util 48 :ok :read [16 23] INFO [2022-01-10 12:52:55,336] jepsen worker 46 - jepsen.util 46 :ok :write [15 40] INFO [2022-01-10 12:52:55,337] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:52:55,338] jepsen worker 21 - jepsen.util 21 :ok :read [7 20] INFO [2022-01-10 12:52:55,341] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:55,342] jepsen worker 0 - jepsen.util 0 :ok :read [0 49] INFO [2022-01-10 12:52:55,345] jepsen worker 46 - jepsen.util 46 :invoke :write [15 38] INFO [2022-01-10 12:52:55,349] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:55,351] jepsen worker 48 - jepsen.util 48 :ok :read [16 23] INFO [2022-01-10 12:52:55,354] jepsen worker 3 - jepsen.util 3 :invoke :write [1 3] INFO [2022-01-10 12:52:55,358] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:55,359] jepsen worker 71 - jepsen.util 71 :ok :read [23 4] INFO [2022-01-10 12:52:55,361] jepsen worker 35 - jepsen.util 35 :invoke :write [11 27] INFO [2022-01-10 12:52:55,366] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:52:55,367] jepsen worker 53 - jepsen.util 53 :ok :read [17 37] INFO [2022-01-10 12:52:55,369] jepsen worker 71 - jepsen.util 71 :invoke :write [23 15] INFO [2022-01-10 12:52:55,373] jepsen worker 16 - jepsen.util 16 :invoke :write [5 6] INFO [2022-01-10 12:52:55,378] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:55,379] jepsen worker 87 - jepsen.util 87 :ok :read [29 10] WARN [2022-01-10 12:52:55,381] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12609,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","total_dispatch_duration_us":1294453,"total_server_duration_us":13293,"encode_duration_us":207,"operation_id":"0x1d1","timeout_ms":10000,"last_server_duration_us":11865,"total_duration_us":3898232},{"operation_name":"upsert","last_dispatch_duration_us":124008,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":1503606,"total_server_duration_us":124916,"encode_duration_us":222,"operation_id":"0x1a2","timeout_ms":10000,"last_server_duration_us":123371,"total_duration_us":3612972},{"operation_name":"upsert","last_dispatch_duration_us":376554,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":1980055,"total_server_duration_us":376921,"encode_duration_us":161,"operation_id":"0x247","timeout_ms":10000,"last_server_duration_us":375981,"total_duration_us":3075673},{"operation_name":"upsert","last_dispatch_duration_us":1456019,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000002/00000000F0645BC3","last_local_socket":"172.23.104.79:34668","total_dispatch_duration_us":1466956,"total_server_duration_us":43977,"encode_duration_us":2901,"operation_id":"0x196","timeout_ms":10000,"last_server_duration_us":42961,"total_duration_us":3071554},{"operation_name":"upsert","last_dispatch_duration_us":49382,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000002/00000000F0645BC3","last_local_socket":"172.23.104.79:34668","total_dispatch_duration_us":1293042,"total_server_duration_us":49461,"encode_duration_us":205,"operation_id":"0x206","timeout_ms":10000,"last_server_duration_us":48449,"total_duration_us":2874537},{"operation_name":"upsert","last_dispatch_duration_us":9176,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":106758,"total_server_duration_us":9506,"encode_duration_us":218,"operation_id":"0x1d2","timeout_ms":10000,"last_server_duration_us":8069,"total_duration_us":2720240},{"operation_name":"upsert","last_dispatch_duration_us":2564903,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":2564903,"total_server_duration_us":2564143,"encode_duration_us":725,"operation_id":"0x190","timeout_ms":10000,"last_server_duration_us":2564143,"total_duration_us":2565509},{"operation_name":"upsert","last_dispatch_duration_us":2387021,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000002/00000000F0645BC3","last_local_socket":"172.23.104.79:34668","total_dispatch_duration_us":2387021,"total_server_duration_us":2386112,"encode_duration_us":209,"operation_id":"0x1f6","timeout_ms":10000,"last_server_duration_us":2386112,"total_duration_us":2387591},{"operation_name":"upsert","last_dispatch_duration_us":2292785,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":2292785,"total_server_duration_us":2291147,"encode_duration_us":288,"operation_id":"0x21b","timeout_ms":10000,"last_server_duration_us":2291147,"total_duration_us":2293509},{"operation_name":"upsert","last_dispatch_duration_us":1822443,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","total_dispatch_duration_us":1822443,"total_server_duration_us":1820759,"encode_duration_us":137,"operation_id":"0x25f","timeout_ms":10000,"last_server_duration_us":1820759,"total_duration_us":1822808}],"total_count":16}} INFO [2022-01-10 12:52:55,382] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:55,383] jepsen worker 4 - jepsen.util 4 :ok :read [1 8] INFO [2022-01-10 12:52:55,386] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:52:55,388] jepsen worker 57 - jepsen.util 57 :ok :read [19 21] INFO [2022-01-10 12:52:55,391] jepsen worker 89 - jepsen.util 89 :invoke :write [29 19] INFO [2022-01-10 12:52:55,395] jepsen worker 65 - jepsen.util 65 :invoke :write [21 38] INFO [2022-01-10 12:52:55,401] jepsen worker 17 - jepsen.util 17 :invoke :write [5 29] WARN [2022-01-10 12:52:55,404] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":49736,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":67122,"total_server_duration_us":50683,"encode_duration_us":338,"operation_id":"0x1c1","timeout_ms":10000,"last_server_duration_us":49024,"total_duration_us":3707510},{"operation_name":"upsert","last_dispatch_duration_us":28873,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":44305,"total_server_duration_us":29794,"encode_duration_us":163,"operation_id":"0x22b","timeout_ms":10000,"last_server_duration_us":27945,"total_duration_us":3174894},{"operation_name":"upsert","last_dispatch_duration_us":33837,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":55373,"total_server_duration_us":34852,"encode_duration_us":206,"operation_id":"0x1eb","timeout_ms":10000,"last_server_duration_us":33230,"total_duration_us":3168716},{"operation_name":"upsert","last_dispatch_duration_us":425402,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":438293,"total_server_duration_us":426063,"encode_duration_us":149,"operation_id":"0x24e","timeout_ms":10000,"last_server_duration_us":424743,"total_duration_us":3059772},{"operation_name":"upsert","last_dispatch_duration_us":2992975,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000003/000000000DBBABF1","last_local_socket":"172.23.104.79:34702","total_dispatch_duration_us":2992975,"total_server_duration_us":2991899,"encode_duration_us":934,"operation_id":"0x197","timeout_ms":10000,"last_server_duration_us":2991899,"total_duration_us":2993560},{"operation_name":"upsert","last_dispatch_duration_us":22470,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":37739,"total_server_duration_us":23388,"encode_duration_us":140,"operation_id":"0x23f","timeout_ms":10000,"last_server_duration_us":21731,"total_duration_us":2660843},{"operation_name":"upsert","last_dispatch_duration_us":25463,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":43467,"total_server_duration_us":25767,"encode_duration_us":178,"operation_id":"0x25b","timeout_ms":10000,"last_server_duration_us":24669,"total_duration_us":2648148},{"operation_name":"upsert","last_dispatch_duration_us":2294467,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000003/000000000DBBABF1","last_local_socket":"172.23.104.79:34702","total_dispatch_duration_us":2294467,"total_server_duration_us":2293516,"encode_duration_us":194,"operation_id":"0x210","timeout_ms":10000,"last_server_duration_us":2293516,"total_duration_us":2294983},{"operation_name":"upsert","last_dispatch_duration_us":1194033,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000003/000000000DBBABF1","last_local_socket":"172.23.104.79:34702","total_dispatch_duration_us":1204975,"total_server_duration_us":79505,"encode_duration_us":191,"operation_id":"0x222","timeout_ms":10000,"last_server_duration_us":78273,"total_duration_us":2294678},{"operation_name":"upsert","last_dispatch_duration_us":1654546,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":1654546,"total_server_duration_us":1653424,"encode_duration_us":1349,"operation_id":"0x1b0","timeout_ms":10000,"last_server_duration_us":1653424,"total_duration_us":1655173}],"total_count":15}} INFO [2022-01-10 12:52:55,405] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:55,407] jepsen worker 60 - jepsen.util 60 :ok :read [20 32] INFO [2022-01-10 12:52:55,409] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:55,410] jepsen worker 61 - jepsen.util 61 :ok :read [20 32] INFO [2022-01-10 12:52:55,415] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:52:55,416] jepsen worker 73 - jepsen.util 73 :ok :read [24 2] INFO [2022-01-10 12:52:55,419] jepsen worker 78 - jepsen.util 78 :invoke :write [26 39] INFO [2022-01-10 12:52:55,423] jepsen worker 45 - jepsen.util 45 :invoke :write [15 48] INFO [2022-01-10 12:52:55,429] jepsen worker 57 - jepsen.util 57 :invoke :write [19 16] INFO [2022-01-10 12:52:55,431] jepsen worker 54 - jepsen.util 54 :invoke :write [18 42] INFO [2022-01-10 12:52:55,434] jepsen worker 68 - jepsen.util 68 :invoke :write [22 32] INFO [2022-01-10 12:52:55,437] jepsen worker 57 - jepsen.util 57 :ok :write [19 16] INFO [2022-01-10 12:52:55,438] jepsen worker 47 - jepsen.util 47 :invoke :write [15 44] INFO [2022-01-10 12:52:55,441] jepsen worker 57 - jepsen.util 57 :invoke :write [19 30] INFO [2022-01-10 12:52:55,445] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:55,446] jepsen worker 26 - jepsen.util 26 :ok :read [8 48] INFO [2022-01-10 12:52:55,448] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:55,448] jepsen worker 57 - jepsen.util 57 :ok :write [19 30] INFO [2022-01-10 12:52:55,451] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:52:55,452] jepsen worker 5 - jepsen.util 5 :ok :read [1 8] INFO [2022-01-10 12:52:55,455] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:52:55,456] jepsen worker 8 - jepsen.util 8 :ok :read [2 17] INFO [2022-01-10 12:52:55,458] jepsen worker 38 - jepsen.util 38 :invoke :write [12 9] INFO [2022-01-10 12:52:55,461] jepsen worker 69 - jepsen.util 69 :invoke :write [23 16] INFO [2022-01-10 12:52:55,465] jepsen worker 86 - jepsen.util 86 :ok :write [28 19] INFO [2022-01-10 12:52:55,465] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:55,467] jepsen worker 23 - jepsen.util 23 :ok :read [7 20] INFO [2022-01-10 12:52:55,469] jepsen worker 42 - jepsen.util 42 :invoke :write [14 47] INFO [2022-01-10 12:52:55,470] jepsen worker 37 - jepsen.util 37 :ok :write [12 13] INFO [2022-01-10 12:52:55,471] jepsen worker 89 - jepsen.util 89 :ok :write [29 19] INFO [2022-01-10 12:52:55,473] jepsen worker 84 - jepsen.util 84 :invoke :write [28 24] INFO [2022-01-10 12:52:55,475] jepsen worker 54 - jepsen.util 54 :ok :write [18 42] INFO [2022-01-10 12:52:55,477] jepsen worker 4 - jepsen.util 4 :invoke :write [1 48] INFO [2022-01-10 12:52:55,480] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:55,482] jepsen worker 87 - jepsen.util 87 :ok :read [29 19] INFO [2022-01-10 12:52:55,483] jepsen worker 79 - jepsen.util 79 :ok :write [26 36] INFO [2022-01-10 12:52:55,484] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:55,488] jepsen worker 3 - jepsen.util 3 :ok :write [1 3] INFO [2022-01-10 12:52:55,489] jepsen worker 30 - jepsen.util 30 :invoke :write [10 28] INFO [2022-01-10 12:52:55,492] jepsen worker 0 - jepsen.util 0 :invoke :write [0 34] INFO [2022-01-10 12:52:55,496] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:55,497] jepsen worker 12 - jepsen.util 12 :ok :read [4 30] INFO [2022-01-10 12:52:55,499] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:55,500] jepsen worker 59 - jepsen.util 59 :ok :read [19 30] INFO [2022-01-10 12:52:55,501] jepsen worker 84 - jepsen.util 84 :ok :write [28 24] INFO [2022-01-10 12:52:55,501] jepsen worker 0 - jepsen.util 0 :ok :write [0 34] INFO [2022-01-10 12:52:55,503] jepsen worker 67 - jepsen.util 67 :invoke :write [22 5] WARN [2022-01-10 12:52:55,504] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":374544,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":1839503,"total_server_duration_us":375499,"encode_duration_us":177,"operation_id":"0x199","timeout_ms":10000,"last_server_duration_us":373787,"total_duration_us":3963149},{"operation_name":"upsert","last_dispatch_duration_us":28463,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":292179,"total_server_duration_us":30186,"encode_duration_us":236,"operation_id":"0x19f","timeout_ms":10000,"last_server_duration_us":27945,"total_duration_us":3408232},{"operation_name":"upsert","last_dispatch_duration_us":138157,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":1291335,"total_server_duration_us":138509,"encode_duration_us":200,"operation_id":"0x231","timeout_ms":10000,"last_server_duration_us":137364,"total_duration_us":3397960},{"operation_name":"upsert","last_dispatch_duration_us":144622,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":1256798,"total_server_duration_us":144424,"encode_duration_us":134,"operation_id":"0x251","timeout_ms":10000,"last_server_duration_us":143504,"total_duration_us":3349217},{"operation_name":"upsert","last_dispatch_duration_us":147143,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000005/0000000090CEECC6","last_local_socket":"172.23.104.79:48368","total_dispatch_duration_us":160539,"total_server_duration_us":148448,"encode_duration_us":156,"operation_id":"0x200","timeout_ms":10000,"last_server_duration_us":146433,"total_duration_us":3298969},{"operation_name":"upsert","last_dispatch_duration_us":131811,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000005/0000000090CEECC6","last_local_socket":"172.23.104.79:48368","total_dispatch_duration_us":144049,"total_server_duration_us":132617,"encode_duration_us":144,"operation_id":"0x1f7","timeout_ms":10000,"last_server_duration_us":130988,"total_duration_us":3268753},{"operation_name":"upsert","last_dispatch_duration_us":67838,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":90012,"total_server_duration_us":68217,"encode_duration_us":203,"operation_id":"0x1bd","timeout_ms":10000,"last_server_duration_us":66431,"total_duration_us":3226789},{"operation_name":"upsert","last_dispatch_duration_us":3147774,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000005/000000006FA84E55","last_local_socket":"172.23.104.79:34638","total_dispatch_duration_us":3147774,"total_server_duration_us":3146776,"encode_duration_us":28023,"operation_id":"0x18b","timeout_ms":10000,"last_server_duration_us":3146776,"total_duration_us":3152444},{"operation_name":"upsert","last_dispatch_duration_us":43744,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":55534,"total_server_duration_us":43932,"encode_duration_us":222,"operation_id":"0x238","timeout_ms":10000,"last_server_duration_us":42743,"total_duration_us":2667980},{"operation_name":"upsert","last_dispatch_duration_us":2621393,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000005/0000000090CEECC6","last_local_socket":"172.23.104.79:48368","total_dispatch_duration_us":2621393,"total_server_duration_us":2620304,"encode_duration_us":134,"operation_id":"0x1ce","timeout_ms":10000,"last_server_duration_us":2620304,"total_duration_us":2621803}],"total_count":19}} INFO [2022-01-10 12:52:55,505] jepsen worker 38 - jepsen.util 38 :ok :write [12 9] INFO [2022-01-10 12:52:55,506] jepsen worker 28 - jepsen.util 28 :invoke :read [9 nil] INFO [2022-01-10 12:52:55,507] jepsen worker 28 - jepsen.util 28 :ok :read [9 26] INFO [2022-01-10 12:52:55,509] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:52:55,510] jepsen worker 55 - jepsen.util 55 :ok :read [18 42] INFO [2022-01-10 12:52:55,512] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:55,513] jepsen worker 31 - jepsen.util 31 :ok :read [10 18] INFO [2022-01-10 12:52:55,514] jepsen worker 4 - jepsen.util 4 :ok :write [1 48] INFO [2022-01-10 12:52:55,515] jepsen worker 62 - jepsen.util 62 :invoke :write [20 0] INFO [2022-01-10 12:52:55,518] jepsen worker 33 - jepsen.util 33 :invoke :write [11 38] INFO [2022-01-10 12:52:55,520] jepsen worker 30 - jepsen.util 30 :ok :write [10 28] INFO [2022-01-10 12:52:55,524] jepsen worker 58 - jepsen.util 58 :invoke :write [19 31] INFO [2022-01-10 12:52:55,528] jepsen worker 39 - jepsen.util 39 :invoke :write [13 15] INFO [2022-01-10 12:52:55,532] jepsen worker 58 - jepsen.util 58 :ok :write [19 31] INFO [2022-01-10 12:52:55,533] jepsen worker 86 - jepsen.util 86 :invoke :write [28 43] INFO [2022-01-10 12:52:55,536] jepsen worker 2 - jepsen.util 2 :invoke :write [0 11] WARN [2022-01-10 12:52:55,537] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":67627,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":81079,"total_server_duration_us":68571,"encode_duration_us":280,"operation_id":"0x220","timeout_ms":10000,"last_server_duration_us":66957,"total_duration_us":3202516},{"operation_name":"upsert","last_dispatch_duration_us":17512,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":31888,"total_server_duration_us":18468,"encode_duration_us":159,"operation_id":"0x1c5","timeout_ms":10000,"last_server_duration_us":16775,"total_duration_us":3163496},{"operation_name":"upsert","last_dispatch_duration_us":2795689,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000004/00000000C4B4FBBF","last_local_socket":"172.23.104.79:58638","total_dispatch_duration_us":2795689,"total_server_duration_us":2794457,"encode_duration_us":1375,"operation_id":"0x198","timeout_ms":10000,"last_server_duration_us":2794457,"total_duration_us":2796134},{"operation_name":"upsert","last_dispatch_duration_us":76343,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":88734,"total_server_duration_us":77299,"encode_duration_us":176,"operation_id":"0x229","timeout_ms":10000,"last_server_duration_us":75616,"total_duration_us":2698459},{"operation_name":"upsert","last_dispatch_duration_us":1027892,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000004/00000000C4B4FBBF","last_local_socket":"172.23.104.79:58638","total_dispatch_duration_us":1041943,"total_server_duration_us":130830,"encode_duration_us":188,"operation_id":"0x1bb","timeout_ms":10000,"last_server_duration_us":129240,"total_duration_us":2643805},{"operation_name":"upsert","last_dispatch_duration_us":2403180,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":2403180,"total_server_duration_us":2402400,"encode_duration_us":149,"operation_id":"0x1ff","timeout_ms":10000,"last_server_duration_us":2402400,"total_duration_us":2403522},{"operation_name":"upsert","last_dispatch_duration_us":1185570,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":1196205,"total_server_duration_us":67522,"encode_duration_us":115,"operation_id":"0x230","timeout_ms":10000,"last_server_duration_us":66300,"total_duration_us":2286461},{"operation_name":"upsert","last_dispatch_duration_us":1113440,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":1123861,"total_server_duration_us":53861,"encode_duration_us":156,"operation_id":"0x25a","timeout_ms":10000,"last_server_duration_us":52777,"total_duration_us":2206643},{"operation_name":"upsert","last_dispatch_duration_us":47877,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000004/0000000048226CCA","last_local_socket":"172.23.104.79:48384","total_dispatch_duration_us":59465,"total_server_duration_us":48213,"encode_duration_us":143,"operation_id":"0x254","timeout_ms":10000,"last_server_duration_us":46965,"total_duration_us":2166578},{"operation_name":"upsert","last_dispatch_duration_us":1823604,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":1823604,"total_server_duration_us":1822908,"encode_duration_us":164,"operation_id":"0x23e","timeout_ms":10000,"last_server_duration_us":1822908,"total_duration_us":1823984}],"total_count":18}} INFO [2022-01-10 12:52:55,539] jepsen worker 43 - jepsen.util 43 :invoke :write [14 33] INFO [2022-01-10 12:52:55,540] jepsen worker 39 - jepsen.util 39 :ok :write [13 15] INFO [2022-01-10 12:52:55,543] jepsen worker 26 - jepsen.util 26 :invoke :write [8 45] WARN [2022-01-10 12:52:55,543] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":86477,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000006/0000000044D7A75E","last_local_socket":"172.23.104.79:48380","total_dispatch_duration_us":99998,"total_server_duration_us":86905,"encode_duration_us":257,"operation_id":"0x202","timeout_ms":10000,"last_server_duration_us":85742,"total_duration_us":3218650},{"operation_name":"upsert","last_dispatch_duration_us":142488,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":1540648,"total_server_duration_us":142761,"encode_duration_us":218,"operation_id":"0x1fd","timeout_ms":10000,"last_server_duration_us":141686,"total_duration_us":3118822},{"operation_name":"upsert","last_dispatch_duration_us":918532,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":935528,"total_server_duration_us":72857,"encode_duration_us":138,"operation_id":"0x19a","timeout_ms":10000,"last_server_duration_us":71366,"total_duration_us":3050342},{"operation_name":"upsert","last_dispatch_duration_us":3048215,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":3048215,"total_server_duration_us":3047165,"encode_duration_us":134,"operation_id":"0x1a0","timeout_ms":10000,"last_server_duration_us":3047165,"total_duration_us":3048947},{"operation_name":"upsert","last_dispatch_duration_us":3023017,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":3023017,"total_server_duration_us":3022475,"encode_duration_us":1096,"operation_id":"0x194","timeout_ms":10000,"last_server_duration_us":3022475,"total_duration_us":3023703},{"operation_name":"upsert","last_dispatch_duration_us":97211,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":1437216,"total_server_duration_us":99321,"encode_duration_us":181,"operation_id":"0x20a","timeout_ms":10000,"last_server_duration_us":96403,"total_duration_us":3017199},{"operation_name":"upsert","last_dispatch_duration_us":1216011,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":1226408,"total_server_duration_us":79304,"encode_duration_us":132,"operation_id":"0x1b5","timeout_ms":10000,"last_server_duration_us":78132,"total_duration_us":2823652},{"operation_name":"upsert","last_dispatch_duration_us":2808618,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":2808618,"total_server_duration_us":2807355,"encode_duration_us":161,"operation_id":"0x1ac","timeout_ms":10000,"last_server_duration_us":2807355,"total_duration_us":2809052},{"operation_name":"upsert","last_dispatch_duration_us":41367,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000006/0000000044D7A75E","last_local_socket":"172.23.104.79:48380","total_dispatch_duration_us":51992,"total_server_duration_us":42061,"encode_duration_us":218,"operation_id":"0x24d","timeout_ms":10000,"last_server_duration_us":40695,"total_duration_us":2680520},{"operation_name":"upsert","last_dispatch_duration_us":1373924,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000006/000000001E9CFABF","last_local_socket":"172.23.104.79:40226","total_dispatch_duration_us":1382587,"total_server_duration_us":219834,"encode_duration_us":386,"operation_id":"0x22d","timeout_ms":10000,"last_server_duration_us":218622,"total_duration_us":2465631}],"total_count":14}} INFO [2022-01-10 12:52:55,547] jepsen worker 9 - jepsen.util 9 :invoke :write [3 14] INFO [2022-01-10 12:52:55,548] jepsen worker 86 - jepsen.util 86 :ok :write [28 43] INFO [2022-01-10 12:52:55,549] jepsen worker 2 - jepsen.util 2 :ok :write [0 11] INFO [2022-01-10 12:52:55,550] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:55,552] jepsen worker 18 - jepsen.util 18 :ok :read [6 43] INFO [2022-01-10 12:52:55,554] jepsen worker 59 - jepsen.util 59 :invoke :write [19 25] INFO [2022-01-10 12:52:55,555] jepsen worker 50 - jepsen.util 50 :ok :write [16 23] INFO [2022-01-10 12:52:55,558] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:52:55,560] jepsen worker 13 - jepsen.util 13 :ok :read [4 30] INFO [2022-01-10 12:52:55,561] jepsen worker 83 - jepsen.util 83 :ok :write [27 28] INFO [2022-01-10 12:52:55,562] jepsen worker 46 - jepsen.util 46 :ok :write [15 38] INFO [2022-01-10 12:52:55,562] jepsen worker 9 - jepsen.util 9 :ok :write [3 14] INFO [2022-01-10 12:52:55,563] jepsen worker 57 - jepsen.util 57 :invoke :write [19 25] INFO [2022-01-10 12:52:55,564] jepsen worker 63 - jepsen.util 63 :ok :read [21 2] INFO [2022-01-10 12:52:55,566] jepsen worker 59 - jepsen.util 59 :ok :write [19 25] INFO [2022-01-10 12:52:55,566] jepsen worker 71 - jepsen.util 71 :ok :write [23 15] INFO [2022-01-10 12:52:55,568] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:52:55,569] jepsen worker 1 - jepsen.util 1 :ok :read [0 11] INFO [2022-01-10 12:52:55,572] jepsen worker 35 - jepsen.util 35 :ok :write [11 27] INFO [2022-01-10 12:52:55,572] jepsen worker 65 - jepsen.util 65 :ok :write [21 38] INFO [2022-01-10 12:52:55,573] jepsen worker 30 - jepsen.util 30 :invoke :write [10 39] INFO [2022-01-10 12:52:55,574] jepsen worker 25 - jepsen.util 25 :ok :write [8 0] INFO [2022-01-10 12:52:55,578] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:52:55,580] jepsen worker 81 - jepsen.util 81 :ok :read [27 28] INFO [2022-01-10 12:52:55,581] jepsen worker 82 - jepsen.util 82 :invoke :write [27 25] INFO [2022-01-10 12:52:55,581] jepsen worker 76 - jepsen.util 76 :ok :write [25 43] INFO [2022-01-10 12:52:55,582] jepsen worker 60 - jepsen.util 60 :ok :read [20 32] INFO [2022-01-10 12:52:55,583] jepsen worker 78 - jepsen.util 78 :ok :write [26 39] INFO [2022-01-10 12:52:55,584] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:55,586] jepsen worker 58 - jepsen.util 58 :ok :read [19 25] INFO [2022-01-10 12:52:55,588] jepsen worker 86 - jepsen.util 86 :invoke :write [28 49] INFO [2022-01-10 12:52:55,589] jepsen worker 16 - jepsen.util 16 :ok :write [5 6] INFO [2022-01-10 12:52:55,591] jepsen worker 65 - jepsen.util 65 :invoke :write [21 27] INFO [2022-01-10 12:52:55,595] jepsen worker 52 - jepsen.util 52 :invoke :write [17 16] INFO [2022-01-10 12:52:55,595] jepsen worker 68 - jepsen.util 68 :ok :write [22 32] INFO [2022-01-10 12:52:55,595] jepsen worker 86 - jepsen.util 86 :ok :write [28 49] INFO [2022-01-10 12:52:55,598] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:55,600] jepsen worker 16 - jepsen.util 16 :ok :read [5 6] INFO [2022-01-10 12:52:55,600] jepsen worker 69 - jepsen.util 69 :ok :write [23 16] INFO [2022-01-10 12:52:55,601] jepsen worker 42 - jepsen.util 42 :ok :write [14 47] INFO [2022-01-10 12:52:55,602] jepsen worker 76 - jepsen.util 76 :invoke :write [25 36] INFO [2022-01-10 12:52:55,606] jepsen worker 0 - jepsen.util 0 :invoke :write [0 42] INFO [2022-01-10 12:52:55,608] jepsen worker 62 - jepsen.util 62 :ok :write [20 0] INFO [2022-01-10 12:52:55,611] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:55,612] jepsen worker 63 - jepsen.util 63 :ok :read [21 38] INFO [2022-01-10 12:52:55,615] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:52:55,616] jepsen worker 0 - jepsen.util 0 :ok :write [0 42] INFO [2022-01-10 12:52:55,616] jepsen worker 75 - jepsen.util 75 :ok :read [25 43] INFO [2022-01-10 12:52:55,619] jepsen worker 18 - jepsen.util 18 :invoke :write [6 32] INFO [2022-01-10 12:52:55,622] jepsen worker 82 - jepsen.util 82 :ok :write [27 25] INFO [2022-01-10 12:52:55,623] jepsen worker 52 - jepsen.util 52 :ok :write [17 16] INFO [2022-01-10 12:52:55,623] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:55,624] jepsen worker 59 - jepsen.util 59 :ok :read [19 25] INFO [2022-01-10 12:52:55,626] jepsen worker 33 - jepsen.util 33 :ok :write [11 38] INFO [2022-01-10 12:52:55,627] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:52:55,628] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:52:55,630] jepsen worker 76 - jepsen.util 76 :ok :write [25 36] INFO [2022-01-10 12:52:55,631] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:55,632] jepsen worker 46 - jepsen.util 46 :ok :read [15 38] INFO [2022-01-10 12:52:55,634] jepsen worker 16 - jepsen.util 16 :invoke :write [5 9] INFO [2022-01-10 12:52:55,638] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:55,639] jepsen worker 63 - jepsen.util 63 :ok :read [21 38] INFO [2022-01-10 12:52:55,641] jepsen worker 67 - jepsen.util 67 :ok :write [22 5] INFO [2022-01-10 12:52:55,642] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:55,643] jepsen worker 43 - jepsen.util 43 :ok :write [14 33] INFO [2022-01-10 12:52:55,643] jepsen worker 58 - jepsen.util 58 :ok :read [19 25] INFO [2022-01-10 12:52:55,645] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:52:55,647] jepsen worker 68 - jepsen.util 68 :ok :read [22 5] INFO [2022-01-10 12:52:55,650] jepsen worker 78 - jepsen.util 78 :invoke :write [26 12] INFO [2022-01-10 12:52:55,653] jepsen worker 53 - jepsen.util 53 :invoke :write [17 40] INFO [2022-01-10 12:52:55,657] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:52:55,658] jepsen worker 54 - jepsen.util 54 :ok :read [18 42] INFO [2022-01-10 12:52:55,659] jepsen worker 78 - jepsen.util 78 :ok :write [26 12] INFO [2022-01-10 12:52:55,660] jepsen worker 67 - jepsen.util 67 :invoke :write [22 27] INFO [2022-01-10 12:52:55,664] jepsen worker 53 - jepsen.util 53 :ok :write [17 40] INFO [2022-01-10 12:52:55,664] jepsen worker 63 - jepsen.util 63 :invoke :write [21 5] INFO [2022-01-10 12:52:55,667] jepsen worker 16 - jepsen.util 16 :ok :write [5 9] INFO [2022-01-10 12:52:55,668] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:55,670] jepsen worker 74 - jepsen.util 74 :ok :read [24 2] INFO [2022-01-10 12:52:55,672] jepsen worker 10 - jepsen.util 10 :invoke :write [3 20] INFO [2022-01-10 12:52:55,676] jepsen worker 72 - jepsen.util 72 :invoke :write [24 40] INFO [2022-01-10 12:52:55,679] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:55,681] jepsen worker 60 - jepsen.util 60 :ok :read [20 0] INFO [2022-01-10 12:52:55,683] jepsen worker 55 - jepsen.util 55 :invoke :write [18 8] INFO [2022-01-10 12:52:55,687] jepsen worker 54 - jepsen.util 54 :invoke :write [18 35] INFO [2022-01-10 12:52:55,690] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:55,691] jepsen worker 55 - jepsen.util 55 :ok :write [18 8] INFO [2022-01-10 12:52:55,692] jepsen worker 10 - jepsen.util 10 :ok :write [3 20] INFO [2022-01-10 12:52:55,692] jepsen worker 84 - jepsen.util 84 :ok :read [28 49] INFO [2022-01-10 12:52:55,694] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:55,695] jepsen worker 23 - jepsen.util 23 :ok :read [7 20] INFO [2022-01-10 12:52:55,696] jepsen worker 45 - jepsen.util 45 :ok :write [15 48] INFO [2022-01-10 12:52:55,699] jepsen worker 81 - jepsen.util 81 :invoke :write [27 7] INFO [2022-01-10 12:52:55,702] jepsen worker 54 - jepsen.util 54 :ok :write [18 35] INFO [2022-01-10 12:52:55,703] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:52:55,705] jepsen worker 20 - jepsen.util 20 :ok :write [6 2] INFO [2022-01-10 12:52:55,705] jepsen worker 73 - jepsen.util 73 :ok :read [24 2] INFO [2022-01-10 12:52:55,708] jepsen worker 78 - jepsen.util 78 :invoke :write [26 34] INFO [2022-01-10 12:52:55,711] jepsen worker 57 - jepsen.util 57 :ok :write [19 25] INFO [2022-01-10 12:52:55,711] jepsen worker 68 - jepsen.util 68 :invoke :write [22 17] INFO [2022-01-10 12:52:55,715] jepsen worker 61 - jepsen.util 61 :invoke :write [20 19] INFO [2022-01-10 12:52:55,716] jepsen worker 78 - jepsen.util 78 :ok :write [26 34] INFO [2022-01-10 12:52:55,719] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:55,719] jepsen worker 65 - jepsen.util 65 :ok :write [21 27] INFO [2022-01-10 12:52:55,721] jepsen worker 74 - jepsen.util 74 :ok :read [24 2] INFO [2022-01-10 12:52:55,723] jepsen worker 21 - jepsen.util 21 :invoke :write [7 6] INFO [2022-01-10 12:52:55,724] jepsen worker 56 - jepsen.util 56 :ok :write [18 25] INFO [2022-01-10 12:52:55,724] jepsen worker 30 - jepsen.util 30 :ok :write [10 39] INFO [2022-01-10 12:52:55,728] jepsen worker 58 - jepsen.util 58 :invoke :write [19 49] INFO [2022-01-10 12:52:55,732] jepsen worker 26 - jepsen.util 26 :ok :write [8 45] INFO [2022-01-10 12:52:55,732] jepsen worker 61 - jepsen.util 61 :ok :write [20 19] INFO [2022-01-10 12:52:55,734] jepsen worker 56 - jepsen.util 56 :invoke :write [18 24] INFO [2022-01-10 12:52:55,735] jepsen worker 72 - jepsen.util 72 :ok :write [24 40] INFO [2022-01-10 12:52:55,738] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:52:55,739] jepsen worker 35 - jepsen.util 35 :ok :read [11 38] INFO [2022-01-10 12:52:55,742] jepsen worker 21 - jepsen.util 21 :ok :write [7 6] INFO [2022-01-10 12:52:55,744] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:52:55,745] jepsen worker 53 - jepsen.util 53 :ok :read [17 40] INFO [2022-01-10 12:52:55,746] jepsen worker 67 - jepsen.util 67 :ok :write [22 27] INFO [2022-01-10 12:52:55,748] jepsen worker 54 - jepsen.util 54 :invoke :write [18 49] INFO [2022-01-10 12:52:55,750] jepsen worker 18 - jepsen.util 18 :ok :write [6 32] INFO [2022-01-10 12:52:55,751] jepsen worker 35 - jepsen.util 35 :invoke :write [11 42] INFO [2022-01-10 12:52:55,755] jepsen worker 75 - jepsen.util 75 :invoke :write [25 35] INFO [2022-01-10 12:52:55,755] jepsen worker 58 - jepsen.util 58 :ok :write [19 49] INFO [2022-01-10 12:52:55,755] jepsen worker 81 - jepsen.util 81 :ok :write [27 7] INFO [2022-01-10 12:52:55,757] jepsen worker 63 - jepsen.util 63 :ok :write [21 5] INFO [2022-01-10 12:52:55,759] jepsen worker 4 - jepsen.util 4 :invoke :write [1 45] INFO [2022-01-10 12:52:55,762] jepsen worker 62 - jepsen.util 62 :invoke :write [20 19] INFO [2022-01-10 12:52:55,764] jepsen worker 68 - jepsen.util 68 :ok :write [22 17] INFO [2022-01-10 12:52:55,766] jepsen worker 37 - jepsen.util 37 :invoke :write [12 35] INFO [2022-01-10 12:52:55,769] jepsen worker 29 - jepsen.util 29 :invoke :write [9 40] INFO [2022-01-10 12:52:55,771] jepsen worker 15 - jepsen.util 15 :ok :write [5 10] INFO [2022-01-10 12:52:55,774] jepsen worker 1 - jepsen.util 1 :invoke :write [0 26] INFO [2022-01-10 12:52:55,775] jepsen worker 47 - jepsen.util 47 :ok :write [15 44] INFO [2022-01-10 12:52:55,779] jepsen worker 35 - jepsen.util 35 :ok :write [11 42] INFO [2022-01-10 12:52:55,779] jepsen worker 33 - jepsen.util 33 :invoke :write [11 41] INFO [2022-01-10 12:52:55,784] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:55,785] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:52:55,787] jepsen worker 65 - jepsen.util 65 :invoke :write [21 31] INFO [2022-01-10 12:52:55,792] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:55,792] jepsen worker 29 - jepsen.util 29 :ok :write [9 40] INFO [2022-01-10 12:52:55,794] jepsen worker 36 - jepsen.util 36 :ok :read [12 9] INFO [2022-01-10 12:52:55,796] jepsen worker 17 - jepsen.util 17 :ok :write [5 29] INFO [2022-01-10 12:52:55,798] jepsen worker 50 - jepsen.util 50 :invoke :write [16 20] INFO [2022-01-10 12:52:55,799] jepsen worker 33 - jepsen.util 33 :ok :write [11 41] INFO [2022-01-10 12:52:55,799] jepsen worker 65 - jepsen.util 65 :ok :write [21 31] INFO [2022-01-10 12:52:55,803] jepsen worker 72 - jepsen.util 72 :invoke :write [24 1] INFO [2022-01-10 12:52:55,807] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:52:55,808] jepsen worker 68 - jepsen.util 68 :ok :read [22 17] INFO [2022-01-10 12:52:55,810] jepsen worker 20 - jepsen.util 20 :invoke :write [6 2] INFO [2022-01-10 12:52:55,813] jepsen worker 79 - jepsen.util 79 :invoke :write [26 7] INFO [2022-01-10 12:52:55,817] jepsen worker 65 - jepsen.util 65 :invoke :write [21 5] INFO [2022-01-10 12:52:55,820] jepsen worker 2 - jepsen.util 2 :invoke :write [0 28] INFO [2022-01-10 12:52:55,824] jepsen worker 22 - jepsen.util 22 :invoke :read [7 nil] INFO [2022-01-10 12:52:55,827] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:55,829] jepsen worker 69 - jepsen.util 69 :ok :read [23 16] INFO [2022-01-10 12:52:55,831] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:55,832] jepsen worker 77 - jepsen.util 77 :ok :read [25 36] INFO [2022-01-10 12:52:55,834] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:55,835] jepsen worker 78 - jepsen.util 78 :ok :read [26 34] INFO [2022-01-10 12:52:55,838] jepsen worker 23 - jepsen.util 23 :invoke :write [7 20] INFO [2022-01-10 12:52:55,842] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:55,843] jepsen worker 9 - jepsen.util 9 :ok :read [3 20] INFO [2022-01-10 12:52:55,845] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 12:52:55,845] jepsen worker 56 - jepsen.util 56 :ok :write [18 24] INFO [2022-01-10 12:52:55,847] jepsen worker 4 - jepsen.util 4 :ok :write [1 45] INFO [2022-01-10 12:52:55,849] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:52:55,850] jepsen worker 86 - jepsen.util 86 :ok :read [28 49] INFO [2022-01-10 12:52:55,852] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:55,853] jepsen worker 0 - jepsen.util 0 :ok :read [0 42] INFO [2022-01-10 12:52:55,855] jepsen worker 75 - jepsen.util 75 :ok :write [25 35] INFO [2022-01-10 12:52:55,856] jepsen worker 56 - jepsen.util 56 :invoke :write [18 32] INFO [2022-01-10 12:52:55,859] jepsen worker 37 - jepsen.util 37 :ok :write [12 35] INFO [2022-01-10 12:52:55,860] jepsen worker 29 - jepsen.util 29 :invoke :write [9 12] INFO [2022-01-10 12:52:55,861] jepsen worker 62 - jepsen.util 62 :ok :write [20 19] INFO [2022-01-10 12:52:55,861] jepsen worker 22 - jepsen.util 22 :ok :read [7 6] INFO [2022-01-10 12:52:55,861] jepsen worker 42 - jepsen.util 42 :ok :read [14 33] INFO [2022-01-10 12:52:55,864] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:55,973] jepsen worker 79 - jepsen.util 79 :ok :write [26 7] INFO [2022-01-10 12:52:55,974] jepsen worker 23 - jepsen.util 23 :ok :write [7 20] INFO [2022-01-10 12:52:55,974] jepsen worker 2 - jepsen.util 2 :ok :write [0 28] INFO [2022-01-10 12:52:55,974] jepsen worker 20 - jepsen.util 20 :ok :write [6 2] INFO [2022-01-10 12:52:55,976] jepsen worker 1 - jepsen.util 1 :ok :write [0 26] INFO [2022-01-10 12:52:55,976] jepsen worker 0 - jepsen.util 0 :ok :read [0 26] INFO [2022-01-10 12:52:55,977] jepsen worker 50 - jepsen.util 50 :ok :write [16 20] INFO [2022-01-10 12:52:55,977] jepsen worker 29 - jepsen.util 29 :ok :write [9 12] INFO [2022-01-10 12:52:55,977] jepsen worker 56 - jepsen.util 56 :ok :write [18 32] INFO [2022-01-10 12:52:55,977] jepsen worker 72 - jepsen.util 72 :ok :write [24 1] INFO [2022-01-10 12:52:55,978] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:55,978] jepsen worker 65 - jepsen.util 65 :ok :write [21 5] INFO [2022-01-10 12:52:55,981] jepsen worker 43 - jepsen.util 43 :ok :read [14 33] INFO [2022-01-10 12:52:55,981] jepsen worker 37 - jepsen.util 37 :invoke :write [12 0] INFO [2022-01-10 12:52:55,983] jepsen worker 54 - jepsen.util 54 :ok :write [18 49] INFO [2022-01-10 12:52:55,984] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:52:55,984] jepsen worker 34 - jepsen.util 34 :ok :write [11 34] INFO [2022-01-10 12:52:55,985] jepsen worker 86 - jepsen.util 86 :ok :read [28 49] INFO [2022-01-10 12:52:55,987] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:55,988] jepsen worker 71 - jepsen.util 71 :ok :read [23 16] INFO [2022-01-10 12:52:55,989] jepsen worker 37 - jepsen.util 37 :ok :write [12 0] INFO [2022-01-10 12:52:55,990] jepsen worker 7 - jepsen.util 7 :invoke :write [2 17] INFO [2022-01-10 12:52:55,994] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:52:55,995] jepsen worker 72 - jepsen.util 72 :ok :read [24 1] INFO [2022-01-10 12:52:55,996] jepsen worker 11 - jepsen.util 11 :invoke :write [3 48] INFO [2022-01-10 12:52:55,998] jepsen worker 7 - jepsen.util 7 :ok :write [2 17] INFO [2022-01-10 12:52:55,999] jepsen worker 39 - jepsen.util 39 :invoke :write [13 20] INFO [2022-01-10 12:52:56,002] jepsen worker 9 - jepsen.util 9 :invoke :write [3 32] INFO [2022-01-10 12:52:56,004] jepsen worker 32 - jepsen.util 32 :invoke :write [10 30] INFO [2022-01-10 12:52:56,006] jepsen worker 11 - jepsen.util 11 :ok :write [3 48] INFO [2022-01-10 12:52:56,006] jepsen worker 45 - jepsen.util 45 :invoke :write [15 30] INFO [2022-01-10 12:52:56,009] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:56,010] jepsen worker 83 - jepsen.util 83 :ok :read [27 7] INFO [2022-01-10 12:52:56,010] jepsen worker 39 - jepsen.util 39 :ok :write [13 20] INFO [2022-01-10 12:52:56,011] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:52:56,013] jepsen worker 1 - jepsen.util 1 :ok :read [0 28] INFO [2022-01-10 12:52:56,014] jepsen worker 5 - jepsen.util 5 :invoke :write [1 23] INFO [2022-01-10 12:52:56,017] jepsen worker 22 - jepsen.util 22 :invoke :read [7 nil] INFO [2022-01-10 12:52:56,017] jepsen worker 45 - jepsen.util 45 :ok :write [15 30] INFO [2022-01-10 12:52:56,017] jepsen worker 32 - jepsen.util 32 :ok :write [10 30] INFO [2022-01-10 12:52:56,018] jepsen worker 22 - jepsen.util 22 :ok :read [7 20] INFO [2022-01-10 12:52:56,020] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:52:56,021] jepsen worker 7 - jepsen.util 7 :ok :read [2 17] INFO [2022-01-10 12:52:56,021] jepsen worker 9 - jepsen.util 9 :ok :write [3 32] INFO [2022-01-10 12:52:56,024] jepsen worker 58 - jepsen.util 58 :invoke :write [19 23] INFO [2022-01-10 12:52:56,025] jepsen worker 5 - jepsen.util 5 :ok :write [1 23] INFO [2022-01-10 12:52:56,026] jepsen worker 81 - jepsen.util 81 :invoke :write [27 42] INFO [2022-01-10 12:52:56,029] jepsen worker 74 - jepsen.util 74 :invoke :write [24 12] INFO [2022-01-10 12:52:56,033] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:52:56,034] jepsen worker 7 - jepsen.util 7 :ok :read [2 17] INFO [2022-01-10 12:52:56,035] jepsen worker 58 - jepsen.util 58 :ok :write [19 23] INFO [2022-01-10 12:52:56,035] jepsen worker 81 - jepsen.util 81 :ok :write [27 42] INFO [2022-01-10 12:52:56,037] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:52:56,038] jepsen worker 8 - jepsen.util 8 :ok :read [2 17] INFO [2022-01-10 12:52:56,040] jepsen worker 74 - jepsen.util 74 :ok :write [24 12] INFO [2022-01-10 12:52:56,040] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:52:56,042] jepsen worker 73 - jepsen.util 73 :ok :read [24 12] INFO [2022-01-10 12:52:56,044] jepsen worker 44 - jepsen.util 44 :invoke :write [14 16] INFO [2022-01-10 12:52:56,047] jepsen worker 2 - jepsen.util 2 :invoke :write [0 13] INFO [2022-01-10 12:52:56,048] jepsen worker 70 - jepsen.util 70 :ok :write [23 22] INFO [2022-01-10 12:52:56,049] jepsen worker 71 - jepsen.util 71 :invoke :write [23 41] INFO [2022-01-10 12:52:56,052] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:52:56,053] jepsen worker 73 - jepsen.util 73 :ok :read [24 12] INFO [2022-01-10 12:52:56,054] jepsen worker 44 - jepsen.util 44 :ok :write [14 16] INFO [2022-01-10 12:52:56,054] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:56,055] jepsen worker 33 - jepsen.util 33 :ok :read [11 34] INFO [2022-01-10 12:52:56,057] jepsen worker 14 - jepsen.util 14 :invoke :write [4 3] INFO [2022-01-10 12:52:56,060] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:52:56,062] jepsen worker 52 - jepsen.util 52 :ok :read [17 40] INFO [2022-01-10 12:52:56,062] jepsen worker 2 - jepsen.util 2 :ok :write [0 13] INFO [2022-01-10 12:52:56,063] jepsen worker 69 - jepsen.util 69 :invoke :write [23 2] INFO [2022-01-10 12:52:56,066] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:52:56,067] jepsen worker 15 - jepsen.util 15 :ok :read [5 29] INFO [2022-01-10 12:52:56,069] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:56,070] jepsen worker 48 - jepsen.util 48 :ok :read [16 20] INFO [2022-01-10 12:52:56,071] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:56,073] jepsen worker 63 - jepsen.util 63 :ok :read [21 5] INFO [2022-01-10 12:52:56,074] jepsen worker 22 - jepsen.util 22 :invoke :write [7 23] INFO [2022-01-10 12:52:56,077] jepsen worker 17 - jepsen.util 17 :invoke :write [5 32] INFO [2022-01-10 12:52:56,080] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:52:56,081] jepsen worker 13 - jepsen.util 13 :ok :read [4 30] INFO [2022-01-10 12:52:56,083] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:56,084] jepsen worker 33 - jepsen.util 33 :ok :read [11 34] INFO [2022-01-10 12:52:56,087] jepsen worker 36 - jepsen.util 36 :invoke :write [12 12] INFO [2022-01-10 12:52:56,089] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:56,091] jepsen worker 60 - jepsen.util 60 :ok :read [20 19] INFO [2022-01-10 12:52:56,092] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:56,094] jepsen worker 74 - jepsen.util 74 :ok :read [24 12] INFO [2022-01-10 12:52:56,095] jepsen worker 36 - jepsen.util 36 :ok :write [12 12] INFO [2022-01-10 12:52:56,095] jepsen worker 40 - jepsen.util 40 :invoke :write [13 2] INFO [2022-01-10 12:52:56,098] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:52:56,099] jepsen worker 34 - jepsen.util 34 :ok :read [11 34] INFO [2022-01-10 12:52:56,101] jepsen worker 70 - jepsen.util 70 :invoke :write [23 32] INFO [2022-01-10 12:52:56,103] jepsen worker 40 - jepsen.util 40 :ok :write [13 2] INFO [2022-01-10 12:52:56,104] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:56,105] jepsen worker 48 - jepsen.util 48 :ok :read [16 20] INFO [2022-01-10 12:52:56,107] jepsen worker 23 - jepsen.util 23 :invoke :write [7 8] INFO [2022-01-10 12:52:56,110] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:52:56,111] jepsen worker 47 - jepsen.util 47 :ok :read [15 30] INFO [2022-01-10 12:52:56,112] jepsen worker 1 - jepsen.util 1 :invoke :write [0 26] INFO [2022-01-10 12:52:56,115] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:56,116] jepsen worker 39 - jepsen.util 39 :ok :read [13 2] INFO [2022-01-10 12:52:56,117] jepsen worker 30 - jepsen.util 30 :invoke :write [10 10] INFO [2022-01-10 12:52:56,119] jepsen worker 1 - jepsen.util 1 :ok :write [0 26] INFO [2022-01-10 12:52:56,120] jepsen worker 35 - jepsen.util 35 :invoke :write [11 9] INFO [2022-01-10 12:52:56,124] jepsen worker 46 - jepsen.util 46 :invoke :write [15 35] INFO [2022-01-10 12:52:56,126] jepsen worker 30 - jepsen.util 30 :ok :write [10 10] INFO [2022-01-10 12:52:56,126] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:52:56,128] jepsen worker 72 - jepsen.util 72 :ok :read [24 12] INFO [2022-01-10 12:52:56,129] jepsen worker 24 - jepsen.util 24 :invoke :write [8 40] INFO [2022-01-10 12:52:56,132] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:52:56,133] jepsen worker 3 - jepsen.util 3 :ok :read [1 23] INFO [2022-01-10 12:52:56,134] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:52:56,135] jepsen worker 40 - jepsen.util 40 :ok :read [13 2] INFO [2022-01-10 12:52:56,136] jepsen worker 51 - jepsen.util 51 :invoke :write [17 20] INFO [2022-01-10 12:52:56,139] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:56,140] jepsen worker 44 - jepsen.util 44 :ok :read [14 16] INFO [2022-01-10 12:52:56,141] jepsen worker 68 - jepsen.util 68 :invoke :write [22 44] INFO [2022-01-10 12:52:56,144] jepsen worker 15 - jepsen.util 15 :invoke :write [5 41] INFO [2022-01-10 12:52:56,146] jepsen worker 5 - jepsen.util 5 :invoke :write [1 7] INFO [2022-01-10 12:52:56,148] jepsen worker 44 - jepsen.util 44 :invoke :write [14 7] INFO [2022-01-10 12:52:56,152] jepsen worker 18 - jepsen.util 18 :invoke :write [6 36] INFO [2022-01-10 12:52:56,154] jepsen worker 5 - jepsen.util 5 :ok :write [1 7] INFO [2022-01-10 12:52:56,155] jepsen worker 37 - jepsen.util 37 :invoke :write [12 42] INFO [2022-01-10 12:52:56,157] jepsen worker 4 - jepsen.util 4 :invoke :write [1 39] INFO [2022-01-10 12:52:56,160] jepsen worker 57 - jepsen.util 57 :invoke :write [19 17] INFO [2022-01-10 12:52:56,162] jepsen worker 38 - jepsen.util 38 :invoke :write [12 41] INFO [2022-01-10 12:52:56,165] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:52:56,165] jepsen worker 37 - jepsen.util 37 :ok :write [12 42] INFO [2022-01-10 12:52:56,166] jepsen worker 81 - jepsen.util 81 :ok :read [27 42] INFO [2022-01-10 12:52:56,167] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:56,168] jepsen worker 11 - jepsen.util 11 :ok :read [3 32] INFO [2022-01-10 12:52:56,169] jepsen worker 4 - jepsen.util 4 :ok :write [1 39] INFO [2022-01-10 12:52:56,170] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:52:56,171] jepsen worker 75 - jepsen.util 75 :ok :read [25 35] INFO [2022-01-10 12:52:56,172] jepsen worker 8 - jepsen.util 8 :invoke :write [2 4] INFO [2022-01-10 12:52:56,173] jepsen worker 57 - jepsen.util 57 :ok :write [19 17] INFO [2022-01-10 12:52:56,174] jepsen worker 37 - jepsen.util 37 :invoke :write [12 39] INFO [2022-01-10 12:52:56,177] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:56,178] jepsen worker 41 - jepsen.util 41 :ok :read [13 2] INFO [2022-01-10 12:52:56,179] jepsen worker 16 - jepsen.util 16 :invoke :write [5 45] INFO [2022-01-10 12:52:56,181] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:52:56,183] jepsen worker 21 - jepsen.util 21 :ok :read [7 20] INFO [2022-01-10 12:52:56,184] jepsen worker 50 - jepsen.util 50 :invoke :write [16 21] INFO [2022-01-10 12:52:56,186] jepsen worker 3 - jepsen.util 3 :invoke :write [1 19] INFO [2022-01-10 12:52:56,188] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:56,190] jepsen worker 74 - jepsen.util 74 :ok :read [24 12] INFO [2022-01-10 12:52:56,190] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:56,192] jepsen worker 26 - jepsen.util 26 :ok :read [8 45] INFO [2022-01-10 12:52:56,193] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:52:56,194] jepsen worker 86 - jepsen.util 86 :ok :read [28 49] INFO [2022-01-10 12:52:56,197] jepsen worker 73 - jepsen.util 73 :invoke :write [24 42] INFO [2022-01-10 12:52:56,197] jepsen worker 38 - jepsen.util 38 :ok :write [12 41] INFO [2022-01-10 12:52:56,200] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:56,201] jepsen worker 3 - jepsen.util 3 :ok :write [1 19] INFO [2022-01-10 12:52:56,202] jepsen worker 63 - jepsen.util 63 :ok :read [21 5] INFO [2022-01-10 12:52:56,202] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:52:56,204] jepsen worker 21 - jepsen.util 21 :ok :read [7 20] INFO [2022-01-10 12:52:56,205] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:56,206] jepsen worker 38 - jepsen.util 38 :ok :read [12 41] INFO [2022-01-10 12:52:56,207] jepsen worker 36 - jepsen.util 36 :invoke :write [12 40] INFO [2022-01-10 12:52:56,210] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:52:56,210] jepsen worker 37 - jepsen.util 37 :ok :write [12 39] INFO [2022-01-10 12:52:56,211] jepsen worker 21 - jepsen.util 21 :ok :read [7 20] INFO [2022-01-10 12:52:56,213] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:56,214] jepsen worker 26 - jepsen.util 26 :ok :read [8 45] INFO [2022-01-10 12:52:56,216] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:56,217] jepsen worker 39 - jepsen.util 39 :ok :read [13 2] INFO [2022-01-10 12:52:56,219] jepsen worker 25 - jepsen.util 25 :invoke :write [8 32] INFO [2022-01-10 12:52:56,222] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:56,223] jepsen worker 87 - jepsen.util 87 :ok :read [29 19] INFO [2022-01-10 12:52:56,223] jepsen worker 22 - jepsen.util 22 :ok :write [7 23] INFO [2022-01-10 12:52:56,223] jepsen worker 71 - jepsen.util 71 :ok :write [23 41] INFO [2022-01-10 12:52:56,224] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:56,226] jepsen worker 45 - jepsen.util 45 :ok :read [15 30] INFO [2022-01-10 12:52:56,227] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:56,228] jepsen worker 27 - jepsen.util 27 :ok :read [9 12] INFO [2022-01-10 12:52:56,229] jepsen worker 14 - jepsen.util 14 :ok :write [4 3] INFO [2022-01-10 12:52:56,230] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:56,230] jepsen worker 17 - jepsen.util 17 :ok :write [5 32] INFO [2022-01-10 12:52:56,231] jepsen worker 45 - jepsen.util 45 :ok :read [15 30] INFO [2022-01-10 12:52:56,233] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:56,234] jepsen worker 26 - jepsen.util 26 :ok :read [8 45] INFO [2022-01-10 12:52:56,235] jepsen worker 64 - jepsen.util 64 :ok :write [21 5] INFO [2022-01-10 12:52:56,236] jepsen worker 66 - jepsen.util 66 :ok :write [22 19] INFO [2022-01-10 12:52:56,237] jepsen worker 31 - jepsen.util 31 :invoke :write [10 8] INFO [2022-01-10 12:52:56,242] jepsen worker 46 - jepsen.util 46 :ok :write [15 35] INFO [2022-01-10 12:52:56,243] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:56,244] jepsen worker 35 - jepsen.util 35 :ok :write [11 9] INFO [2022-01-10 12:52:56,244] jepsen worker 85 - jepsen.util 85 :ok :read [28 49] INFO [2022-01-10 12:52:56,246] jepsen worker 24 - jepsen.util 24 :ok :write [8 40] INFO [2022-01-10 12:52:56,247] jepsen worker 3 - jepsen.util 3 :invoke :write [1 24] INFO [2022-01-10 12:52:56,251] jepsen worker 44 - jepsen.util 44 :ok :write [14 7] INFO [2022-01-10 12:52:56,251] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:52:56,252] jepsen worker 51 - jepsen.util 51 :ok :write [17 20] INFO [2022-01-10 12:52:56,252] jepsen worker 14 - jepsen.util 14 :ok :read [4 3] INFO [2022-01-10 12:52:56,254] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:56,255] jepsen worker 74 - jepsen.util 74 :ok :read [24 12] INFO [2022-01-10 12:52:56,257] jepsen worker 8 - jepsen.util 8 :ok :write [2 4] INFO [2022-01-10 12:52:56,257] jepsen worker 77 - jepsen.util 77 :invoke :write [25 33] INFO [2022-01-10 12:52:56,259] jepsen worker 18 - jepsen.util 18 :ok :write [6 36] INFO [2022-01-10 12:52:56,261] jepsen worker 29 - jepsen.util 29 :invoke :write [9 13] INFO [2022-01-10 12:52:56,264] jepsen worker 43 - jepsen.util 43 :invoke :write [14 43] INFO [2022-01-10 12:52:56,264] jepsen worker 50 - jepsen.util 50 :ok :write [16 21] INFO [2022-01-10 12:52:56,265] jepsen worker 73 - jepsen.util 73 :ok :write [24 42] INFO [2022-01-10 12:52:56,267] jepsen worker 49 - jepsen.util 49 :invoke :write [16 26] INFO [2022-01-10 12:52:56,270] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 12:52:56,271] jepsen worker 6 - jepsen.util 6 :ok :read [2 4] INFO [2022-01-10 12:52:56,272] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:56,273] jepsen worker 63 - jepsen.util 63 :ok :read [21 5] INFO [2022-01-10 12:52:56,275] jepsen worker 37 - jepsen.util 37 :invoke :write [12 25] INFO [2022-01-10 12:52:56,278] jepsen worker 21 - jepsen.util 21 :invoke :write [7 6] INFO [2022-01-10 12:52:56,280] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:56,281] jepsen worker 48 - jepsen.util 48 :ok :read [16 21] INFO [2022-01-10 12:52:56,283] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:56,284] jepsen worker 39 - jepsen.util 39 :ok :read [13 2] INFO [2022-01-10 12:52:56,286] jepsen worker 50 - jepsen.util 50 :invoke :write [16 23] INFO [2022-01-10 12:52:56,288] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:56,289] jepsen worker 48 - jepsen.util 48 :ok :read [16 21] INFO [2022-01-10 12:52:56,291] jepsen worker 46 - jepsen.util 46 :invoke :write [15 42] INFO [2022-01-10 12:52:56,293] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:56,295] jepsen worker 2 - jepsen.util 2 :ok :read [0 26] INFO [2022-01-10 12:52:56,296] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:56,297] jepsen worker 71 - jepsen.util 71 :ok :read [23 41] INFO [2022-01-10 12:52:56,299] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:52:56,300] jepsen worker 52 - jepsen.util 52 :ok :read [17 20] INFO [2022-01-10 12:52:56,301] jepsen worker 12 - jepsen.util 12 :invoke :write [4 33] INFO [2022-01-10 12:52:56,304] jepsen worker 32 - jepsen.util 32 :invoke :write [10 33] INFO [2022-01-10 12:52:56,306] jepsen worker 89 - jepsen.util 89 :invoke :write [29 34] INFO [2022-01-10 12:52:56,309] jepsen worker 56 - jepsen.util 56 :invoke :write [18 5] INFO [2022-01-10 12:52:56,311] jepsen worker 71 - jepsen.util 71 :invoke :write [23 29] INFO [2022-01-10 12:52:56,314] jepsen worker 38 - jepsen.util 38 :invoke :write [12 41] INFO [2022-01-10 12:52:56,317] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:56,318] jepsen worker 48 - jepsen.util 48 :ok :read [16 21] INFO [2022-01-10 12:52:56,319] jepsen worker 24 - jepsen.util 24 :invoke :write [8 8] INFO [2022-01-10 12:52:56,320] jepsen worker 36 - jepsen.util 36 :ok :write [12 40] INFO [2022-01-10 12:52:56,321] jepsen worker 88 - jepsen.util 88 :invoke :write [29 28] INFO [2022-01-10 12:52:56,324] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:52:56,325] jepsen worker 62 - jepsen.util 62 :ok :read [20 19] INFO [2022-01-10 12:52:56,326] jepsen worker 3 - jepsen.util 3 :ok :write [1 24] INFO [2022-01-10 12:52:56,326] jepsen worker 4 - jepsen.util 4 :invoke :write [1 21] INFO [2022-01-10 12:52:56,329] jepsen worker 65 - jepsen.util 65 :invoke :write [21 41] INFO [2022-01-10 12:52:56,329] jepsen worker 68 - jepsen.util 68 :ok :write [22 44] INFO [2022-01-10 12:52:56,331] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:52:56,332] jepsen worker 30 - jepsen.util 30 :ok :read [10 10] INFO [2022-01-10 12:52:56,334] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:56,335] jepsen worker 78 - jepsen.util 78 :ok :read [26 7] INFO [2022-01-10 12:52:56,337] jepsen worker 53 - jepsen.util 53 :invoke :write [17 49] INFO [2022-01-10 12:52:56,338] jepsen worker 16 - jepsen.util 16 :ok :write [5 45] INFO [2022-01-10 12:52:56,340] jepsen worker 31 - jepsen.util 31 :ok :write [10 8] INFO [2022-01-10 12:52:56,340] jepsen worker 13 - jepsen.util 13 :invoke :write [4 4] INFO [2022-01-10 12:52:56,343] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:52:56,344] jepsen worker 66 - jepsen.util 66 :ok :read [22 44] INFO [2022-01-10 12:52:56,346] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:56,346] jepsen worker 46 - jepsen.util 46 :ok :write [15 42] INFO [2022-01-10 12:52:56,347] jepsen worker 87 - jepsen.util 87 :ok :read [29 19] INFO [2022-01-10 12:52:56,350] jepsen worker 49 - jepsen.util 49 :ok :write [16 26] INFO [2022-01-10 12:52:56,353] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:56,353] jepsen worker 83 - jepsen.util 83 :invoke :write [27 27] INFO [2022-01-10 12:52:56,353] jepsen worker 25 - jepsen.util 25 :ok :write [8 32] INFO [2022-01-10 12:52:56,354] jepsen worker 74 - jepsen.util 74 :ok :read [24 42] INFO [2022-01-10 12:52:56,356] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:56,357] jepsen worker 16 - jepsen.util 16 :ok :read [5 45] INFO [2022-01-10 12:52:56,357] jepsen worker 69 - jepsen.util 69 :ok :write [23 2] INFO [2022-01-10 12:52:56,358] jepsen worker 29 - jepsen.util 29 :ok :write [9 13] INFO [2022-01-10 12:52:56,358] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:56,358] jepsen worker 77 - jepsen.util 77 :ok :write [25 33] INFO [2022-01-10 12:52:56,360] jepsen worker 78 - jepsen.util 78 :ok :read [26 7] INFO [2022-01-10 12:52:56,361] jepsen worker 63 - jepsen.util 63 :invoke :write [21 17] INFO [2022-01-10 12:52:56,363] jepsen worker 54 - jepsen.util 54 :invoke :write [18 46] INFO [2022-01-10 12:52:56,365] jepsen worker 43 - jepsen.util 43 :ok :write [14 43] INFO [2022-01-10 12:52:56,365] jepsen worker 12 - jepsen.util 12 :ok :write [4 33] INFO [2022-01-10 12:52:56,366] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:56,368] jepsen worker 16 - jepsen.util 16 :ok :read [5 45] INFO [2022-01-10 12:52:56,369] jepsen worker 65 - jepsen.util 65 :ok :write [21 41] INFO [2022-01-10 12:52:56,370] jepsen worker 66 - jepsen.util 66 :invoke :write [22 45] INFO [2022-01-10 12:52:56,372] jepsen worker 21 - jepsen.util 21 :ok :write [7 6] INFO [2022-01-10 12:52:56,373] jepsen worker 38 - jepsen.util 38 :ok :write [12 41] INFO [2022-01-10 12:52:56,373] jepsen worker 10 - jepsen.util 10 :invoke :read [3 nil] INFO [2022-01-10 12:52:56,374] jepsen worker 50 - jepsen.util 50 :ok :write [16 23] INFO [2022-01-10 12:52:56,375] jepsen worker 10 - jepsen.util 10 :ok :read [3 32] INFO [2022-01-10 12:52:56,377] jepsen worker 68 - jepsen.util 68 :invoke :write [22 3] INFO [2022-01-10 12:52:56,379] jepsen worker 89 - jepsen.util 89 :ok :write [29 34] INFO [2022-01-10 12:52:56,380] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:56,380] jepsen worker 4 - jepsen.util 4 :ok :write [1 21] INFO [2022-01-10 12:52:56,381] jepsen worker 83 - jepsen.util 83 :ok :write [27 27] INFO [2022-01-10 12:52:56,381] jepsen worker 2 - jepsen.util 2 :ok :read [0 26] INFO [2022-01-10 12:52:56,382] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:52:56,384] jepsen worker 3 - jepsen.util 3 :ok :read [1 21] INFO [2022-01-10 12:52:56,385] jepsen worker 21 - jepsen.util 21 :invoke :write [7 31] INFO [2022-01-10 12:52:56,385] jepsen worker 53 - jepsen.util 53 :ok :write [17 49] INFO [2022-01-10 12:52:56,385] jepsen worker 56 - jepsen.util 56 :ok :write [18 5] INFO [2022-01-10 12:52:56,386] jepsen worker 24 - jepsen.util 24 :ok :write [8 8] INFO [2022-01-10 12:52:56,388] jepsen worker 35 - jepsen.util 35 :invoke :write [11 12] INFO [2022-01-10 12:52:56,391] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:52:56,392] jepsen worker 32 - jepsen.util 32 :ok :write [10 33] INFO [2022-01-10 12:52:56,392] jepsen worker 40 - jepsen.util 40 :ok :read [13 2] INFO [2022-01-10 12:52:56,395] jepsen worker 5 - jepsen.util 5 :invoke :write [1 48] INFO [2022-01-10 12:52:56,398] jepsen worker 80 - jepsen.util 80 :ok :write [26 36] INFO [2022-01-10 12:52:56,398] jepsen worker 51 - jepsen.util 51 :invoke :write [17 3] INFO [2022-01-10 12:52:56,401] jepsen worker 22 - jepsen.util 22 :invoke :read [7 nil] INFO [2022-01-10 12:52:56,403] jepsen worker 22 - jepsen.util 22 :ok :read [7 6] INFO [2022-01-10 12:52:56,404] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:56,405] jepsen worker 43 - jepsen.util 43 :ok :read [14 43] INFO [2022-01-10 12:52:56,406] jepsen worker 69 - jepsen.util 69 :invoke :write [23 28] INFO [2022-01-10 12:52:56,409] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:52:56,410] jepsen worker 53 - jepsen.util 53 :ok :read [17 49] INFO [2022-01-10 12:52:56,411] jepsen worker 88 - jepsen.util 88 :ok :write [29 28] INFO [2022-01-10 12:52:56,411] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:56,413] jepsen worker 89 - jepsen.util 89 :ok :read [29 28] INFO [2022-01-10 12:52:56,413] jepsen worker 83 - jepsen.util 83 :invoke :write [27 15] INFO [2022-01-10 12:52:56,416] jepsen worker 0 - jepsen.util 0 :invoke :write [0 35] INFO [2022-01-10 12:52:56,417] jepsen worker 37 - jepsen.util 37 :ok :write [12 25] INFO [2022-01-10 12:52:56,417] jepsen worker 54 - jepsen.util 54 :ok :write [18 46] INFO [2022-01-10 12:52:56,419] jepsen worker 24 - jepsen.util 24 :invoke :write [8 16] INFO [2022-01-10 12:52:56,422] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:56,423] jepsen worker 5 - jepsen.util 5 :ok :write [1 48] INFO [2022-01-10 12:52:56,423] jepsen worker 87 - jepsen.util 87 :ok :read [29 28] INFO [2022-01-10 12:52:56,424] jepsen worker 46 - jepsen.util 46 :invoke :write [15 0] INFO [2022-01-10 12:52:56,427] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:56,428] jepsen worker 12 - jepsen.util 12 :ok :read [4 33] INFO [2022-01-10 12:52:56,429] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:56,431] jepsen worker 26 - jepsen.util 26 :ok :read [8 8] INFO [2022-01-10 12:52:56,432] jepsen worker 50 - jepsen.util 50 :invoke :write [16 4] INFO [2022-01-10 12:52:56,434] jepsen worker 73 - jepsen.util 73 :invoke :write [24 26] INFO [2022-01-10 12:52:56,435] jepsen worker 0 - jepsen.util 0 :ok :write [0 35] INFO [2022-01-10 12:52:56,437] jepsen worker 86 - jepsen.util 86 :invoke :write [28 19] INFO [2022-01-10 12:52:56,440] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:56,441] jepsen worker 87 - jepsen.util 87 :ok :read [29 28] INFO [2022-01-10 12:52:56,442] jepsen worker 49 - jepsen.util 49 :invoke :write [16 19] INFO [2022-01-10 12:52:56,444] jepsen worker 36 - jepsen.util 36 :invoke :write [12 26] INFO [2022-01-10 12:52:56,447] jepsen worker 44 - jepsen.util 44 :invoke :write [14 44] INFO [2022-01-10 12:52:56,449] jepsen worker 86 - jepsen.util 86 :ok :write [28 19] INFO [2022-01-10 12:52:56,449] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:52:56,451] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:52:56,452] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:56,453] jepsen worker 11 - jepsen.util 11 :ok :read [3 32] INFO [2022-01-10 12:52:56,454] jepsen worker 36 - jepsen.util 36 :ok :write [12 26] INFO [2022-01-10 12:52:56,455] jepsen worker 88 - jepsen.util 88 :invoke :write [29 47] INFO [2022-01-10 12:52:56,457] jepsen worker 42 - jepsen.util 42 :invoke :write [14 5] INFO [2022-01-10 12:52:56,460] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:56,461] jepsen worker 67 - jepsen.util 67 :ok :read [22 44] INFO [2022-01-10 12:52:56,462] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:56,463] jepsen worker 48 - jepsen.util 48 :ok :read [16 23] INFO [2022-01-10 12:52:56,465] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:52:56,465] jepsen worker 88 - jepsen.util 88 :ok :write [29 47] INFO [2022-01-10 12:52:56,466] jepsen worker 72 - jepsen.util 72 :ok :read [24 42] INFO [2022-01-10 12:52:56,468] jepsen worker 65 - jepsen.util 65 :invoke :write [21 24] INFO [2022-01-10 12:52:56,470] jepsen worker 25 - jepsen.util 25 :invoke :write [8 20] INFO [2022-01-10 12:52:56,472] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:52:56,474] jepsen worker 30 - jepsen.util 30 :ok :read [10 33] INFO [2022-01-10 12:52:56,476] jepsen worker 53 - jepsen.util 53 :invoke :write [17 43] INFO [2022-01-10 12:52:56,478] jepsen worker 67 - jepsen.util 67 :invoke :write [22 31] INFO [2022-01-10 12:52:56,480] jepsen worker 72 - jepsen.util 72 :invoke :write [24 46] INFO [2022-01-10 12:52:56,483] jepsen worker 60 - jepsen.util 60 :invoke :write [20 21] INFO [2022-01-10 12:52:56,485] jepsen worker 47 - jepsen.util 47 :invoke :write [15 41] INFO [2022-01-10 12:52:56,488] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:56,489] jepsen worker 77 - jepsen.util 77 :ok :read [25 33] INFO [2022-01-10 12:52:56,490] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:52:56,491] jepsen worker 64 - jepsen.util 64 :ok :read [21 41] INFO [2022-01-10 12:52:56,492] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:56,493] jepsen worker 85 - jepsen.util 85 :ok :read [28 19] INFO [2022-01-10 12:52:56,495] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:56,496] jepsen worker 45 - jepsen.util 45 :ok :read [15 42] INFO [2022-01-10 12:52:56,497] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:56,498] jepsen worker 12 - jepsen.util 12 :ok :read [4 33] INFO [2022-01-10 12:52:56,499] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:56,500] jepsen worker 74 - jepsen.util 74 :ok :read [24 42] INFO [2022-01-10 12:52:56,502] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:52:56,503] jepsen worker 5 - jepsen.util 5 :ok :read [1 48] INFO [2022-01-10 12:52:56,504] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:52:56,505] jepsen worker 64 - jepsen.util 64 :ok :read [21 41] INFO [2022-01-10 12:52:56,507] jepsen worker 6 - jepsen.util 6 :invoke :write [2 37] INFO [2022-01-10 12:52:56,509] jepsen worker 89 - jepsen.util 89 :invoke :write [29 18] INFO [2022-01-10 12:52:56,512] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:56,513] jepsen worker 36 - jepsen.util 36 :ok :read [12 26] INFO [2022-01-10 12:52:56,514] jepsen worker 79 - jepsen.util 79 :invoke :write [26 16] INFO [2022-01-10 12:52:56,517] jepsen worker 17 - jepsen.util 17 :invoke :write [5 40] INFO [2022-01-10 12:52:56,519] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:52:56,521] jepsen worker 89 - jepsen.util 89 :ok :write [29 18] INFO [2022-01-10 12:52:56,521] jepsen worker 52 - jepsen.util 52 :ok :read [17 49] INFO [2022-01-10 12:52:56,525] jepsen worker 0 - jepsen.util 0 :invoke :write [0 4] INFO [2022-01-10 12:52:56,529] jepsen worker 86 - jepsen.util 86 :invoke :write [28 14] INFO [2022-01-10 12:52:56,531] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:56,532] jepsen worker 0 - jepsen.util 0 :ok :write [0 4] INFO [2022-01-10 12:52:56,532] jepsen worker 58 - jepsen.util 58 :ok :read [19 17] INFO [2022-01-10 12:52:56,534] jepsen worker 13 - jepsen.util 13 :ok :write [4 4] INFO [2022-01-10 12:52:56,534] jepsen worker 74 - jepsen.util 74 :invoke :write [24 40] INFO [2022-01-10 12:52:56,537] jepsen worker 84 - jepsen.util 84 :invoke :write [28 16] INFO [2022-01-10 12:52:56,538] jepsen worker 86 - jepsen.util 86 :ok :write [28 14] INFO [2022-01-10 12:52:56,540] jepsen worker 21 - jepsen.util 21 :ok :write [7 31] INFO [2022-01-10 12:52:56,540] jepsen worker 63 - jepsen.util 63 :ok :write [21 17] INFO [2022-01-10 12:52:56,540] jepsen worker 16 - jepsen.util 16 :invoke :write [5 22] INFO [2022-01-10 12:52:56,544] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:52:56,546] jepsen worker 71 - jepsen.util 71 :ok :write [23 29] INFO [2022-01-10 12:52:56,546] jepsen worker 54 - jepsen.util 54 :ok :read [18 46] INFO [2022-01-10 12:52:56,548] jepsen worker 66 - jepsen.util 66 :ok :write [22 45] INFO [2022-01-10 12:52:56,548] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:52:56,549] jepsen worker 84 - jepsen.util 84 :ok :write [28 16] INFO [2022-01-10 12:52:56,549] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:52:56,551] jepsen worker 24 - jepsen.util 24 :ok :write [8 16] INFO [2022-01-10 12:52:56,551] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:52:56,552] jepsen worker 29 - jepsen.util 29 :ok :read [9 13] INFO [2022-01-10 12:52:56,553] jepsen worker 50 - jepsen.util 50 :ok :write [16 4] INFO [2022-01-10 12:52:56,554] jepsen worker 43 - jepsen.util 43 :invoke :write [14 38] INFO [2022-01-10 12:52:56,556] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:56,557] jepsen worker 51 - jepsen.util 51 :ok :write [17 3] INFO [2022-01-10 12:52:56,557] jepsen worker 48 - jepsen.util 48 :ok :read [16 4] INFO [2022-01-10 12:52:56,558] jepsen worker 83 - jepsen.util 83 :ok :write [27 15] INFO [2022-01-10 12:52:56,560] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:56,561] jepsen worker 79 - jepsen.util 79 :ok :write [26 16] INFO [2022-01-10 12:52:56,562] jepsen worker 77 - jepsen.util 77 :ok :read [25 33] INFO [2022-01-10 12:52:56,564] jepsen worker 39 - jepsen.util 39 :invoke :write [13 0] INFO [2022-01-10 12:52:56,564] jepsen worker 35 - jepsen.util 35 :ok :write [11 12] INFO [2022-01-10 12:52:56,566] jepsen worker 46 - jepsen.util 46 :ok :write [15 0] INFO [2022-01-10 12:52:56,567] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:52:56,568] jepsen worker 52 - jepsen.util 52 :ok :read [17 3] INFO [2022-01-10 12:52:56,571] jepsen worker 66 - jepsen.util 66 :invoke :write [22 37] INFO [2022-01-10 12:52:56,572] jepsen worker 44 - jepsen.util 44 :ok :write [14 44] INFO [2022-01-10 12:52:56,572] jepsen worker 73 - jepsen.util 73 :ok :write [24 26] INFO [2022-01-10 12:52:56,575] jepsen worker 75 - jepsen.util 75 :invoke :write [25 27] INFO [2022-01-10 12:52:56,578] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:52:56,578] jepsen worker 39 - jepsen.util 39 :ok :write [13 0] INFO [2022-01-10 12:52:56,579] jepsen worker 15 - jepsen.util 15 :ok :write [5 41] INFO [2022-01-10 12:52:56,580] jepsen worker 34 - jepsen.util 34 :ok :read [11 12] INFO [2022-01-10 12:52:56,581] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:56,583] jepsen worker 48 - jepsen.util 48 :ok :read [16 4] INFO [2022-01-10 12:52:56,584] jepsen worker 60 - jepsen.util 60 :ok :write [20 21] INFO [2022-01-10 12:52:56,586] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:52:56,587] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:52:56,589] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:56,591] jepsen worker 71 - jepsen.util 71 :ok :read [23 29] INFO [2022-01-10 12:52:56,591] jepsen worker 6 - jepsen.util 6 :ok :write [2 37] INFO [2022-01-10 12:52:56,593] jepsen worker 14 - jepsen.util 14 :invoke :write [4 40] INFO [2022-01-10 12:52:56,597] jepsen worker 88 - jepsen.util 88 :invoke :read [29 nil] INFO [2022-01-10 12:52:56,597] jepsen worker 25 - jepsen.util 25 :ok :write [8 20] INFO [2022-01-10 12:52:56,598] jepsen worker 88 - jepsen.util 88 :ok :read [29 18] INFO [2022-01-10 12:52:56,599] jepsen worker 73 - jepsen.util 73 :invoke :write [24 3] INFO [2022-01-10 12:52:56,602] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:56,603] jepsen worker 53 - jepsen.util 53 :ok :write [17 43] INFO [2022-01-10 12:52:56,603] jepsen worker 78 - jepsen.util 78 :ok :read [26 16] INFO [2022-01-10 12:52:56,603] jepsen worker 68 - jepsen.util 68 :ok :write [22 3] INFO [2022-01-10 12:52:56,604] jepsen worker 79 - jepsen.util 79 :invoke :write [26 41] INFO [2022-01-10 12:52:56,608] jepsen worker 6 - jepsen.util 6 :invoke :write [2 31] INFO [2022-01-10 12:52:56,610] jepsen worker 10 - jepsen.util 10 :invoke :read [3 nil] INFO [2022-01-10 12:52:56,611] jepsen worker 10 - jepsen.util 10 :ok :read [3 32] INFO [2022-01-10 12:52:56,612] jepsen worker 47 - jepsen.util 47 :ok :write [15 41] INFO [2022-01-10 12:52:56,613] jepsen worker 1 - jepsen.util 1 :invoke :write [0 37] INFO [2022-01-10 12:52:56,615] jepsen worker 62 - jepsen.util 62 :invoke :write [20 18] INFO [2022-01-10 12:52:56,617] jepsen worker 74 - jepsen.util 74 :ok :write [24 40] INFO [2022-01-10 12:52:56,618] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:56,620] jepsen worker 85 - jepsen.util 85 :ok :read [28 16] INFO [2022-01-10 12:52:56,621] jepsen worker 43 - jepsen.util 43 :ok :write [14 38] INFO [2022-01-10 12:52:56,621] jepsen worker 65 - jepsen.util 65 :ok :write [21 24] INFO [2022-01-10 12:52:56,621] jepsen worker 76 - jepsen.util 76 :invoke :write [25 48] INFO [2022-01-10 12:52:56,622] jepsen worker 1 - jepsen.util 1 :ok :write [0 37] INFO [2022-01-10 12:52:56,624] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:56,626] jepsen worker 77 - jepsen.util 77 :ok :read [25 33] INFO [2022-01-10 12:52:56,626] jepsen worker 69 - jepsen.util 69 :ok :write [23 28] INFO [2022-01-10 12:52:56,627] jepsen worker 24 - jepsen.util 24 :invoke :write [8 1] INFO [2022-01-10 12:52:56,628] jepsen worker 75 - jepsen.util 75 :ok :write [25 27] INFO [2022-01-10 12:52:56,630] jepsen worker 45 - jepsen.util 45 :invoke :write [15 16] INFO [2022-01-10 12:52:56,633] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:56,634] jepsen worker 16 - jepsen.util 16 :ok :write [5 22] INFO [2022-01-10 12:52:56,634] jepsen worker 44 - jepsen.util 44 :ok :read [14 38] INFO [2022-01-10 12:52:56,635] jepsen worker 20 - jepsen.util 20 :invoke :write [6 10] INFO [2022-01-10 12:52:56,637] jepsen worker 14 - jepsen.util 14 :ok :write [4 40] INFO [2022-01-10 12:52:56,638] jepsen worker 37 - jepsen.util 37 :invoke :write [12 37] INFO [2022-01-10 12:52:56,640] jepsen worker 62 - jepsen.util 62 :ok :write [20 18] INFO [2022-01-10 12:52:56,641] jepsen worker 60 - jepsen.util 60 :invoke :write [20 29] INFO [2022-01-10 12:52:56,644] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:56,645] jepsen worker 31 - jepsen.util 31 :ok :read [10 33] INFO [2022-01-10 12:52:56,647] jepsen worker 40 - jepsen.util 40 :invoke :write [13 34] INFO [2022-01-10 12:52:56,647] jepsen worker 6 - jepsen.util 6 :ok :write [2 31] INFO [2022-01-10 12:52:56,648] jepsen worker 37 - jepsen.util 37 :ok :write [12 37] INFO [2022-01-10 12:52:56,650] jepsen worker 79 - jepsen.util 79 :ok :write [26 41] INFO [2022-01-10 12:52:56,650] jepsen worker 4 - jepsen.util 4 :invoke :write [1 14] INFO [2022-01-10 12:52:56,653] jepsen worker 66 - jepsen.util 66 :ok :write [22 37] INFO [2022-01-10 12:52:56,653] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:52:56,654] jepsen worker 45 - jepsen.util 45 :ok :write [15 16] INFO [2022-01-10 12:52:56,655] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:52:56,656] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:56,657] jepsen worker 69 - jepsen.util 69 :ok :read [23 28] INFO [2022-01-10 12:52:56,658] jepsen worker 63 - jepsen.util 63 :invoke :write [21 26] INFO [2022-01-10 12:52:56,660] jepsen worker 49 - jepsen.util 49 :ok :write [16 19] INFO [2022-01-10 12:52:56,661] jepsen worker 40 - jepsen.util 40 :ok :write [13 34] INFO [2022-01-10 12:52:56,662] jepsen worker 19 - jepsen.util 19 :invoke :write [6 15] INFO [2022-01-10 12:52:56,664] jepsen worker 73 - jepsen.util 73 :ok :write [24 3] INFO [2022-01-10 12:52:56,666] jepsen worker 42 - jepsen.util 42 :ok :write [14 5] INFO [2022-01-10 12:52:56,666] jepsen worker 88 - jepsen.util 88 :invoke :read [29 nil] INFO [2022-01-10 12:52:56,667] jepsen worker 4 - jepsen.util 4 :ok :write [1 14] INFO [2022-01-10 12:52:56,667] jepsen worker 63 - jepsen.util 63 :ok :write [21 26] INFO [2022-01-10 12:52:56,668] jepsen worker 88 - jepsen.util 88 :ok :read [29 18] INFO [2022-01-10 12:52:56,670] jepsen worker 39 - jepsen.util 39 :invoke :write [13 31] INFO [2022-01-10 12:52:56,672] jepsen worker 76 - jepsen.util 76 :ok :write [25 48] INFO [2022-01-10 12:52:56,674] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:56,675] jepsen worker 9 - jepsen.util 9 :ok :read [3 32] INFO [2022-01-10 12:52:56,677] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 12:52:56,678] jepsen worker 6 - jepsen.util 6 :ok :read [2 31] INFO [2022-01-10 12:52:56,679] jepsen worker 24 - jepsen.util 24 :ok :write [8 1] INFO [2022-01-10 12:52:56,679] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:56,680] jepsen worker 39 - jepsen.util 39 :ok :write [13 31] INFO [2022-01-10 12:52:56,680] jepsen worker 48 - jepsen.util 48 :ok :read [16 19] INFO [2022-01-10 12:52:56,684] jepsen worker 33 - jepsen.util 33 :invoke :write [11 27] INFO [2022-01-10 12:52:56,685] jepsen worker 60 - jepsen.util 60 :ok :write [20 29] INFO [2022-01-10 12:52:56,687] jepsen worker 82 - jepsen.util 82 :invoke :write [27 37] INFO [2022-01-10 12:52:56,690] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:56,691] jepsen worker 58 - jepsen.util 58 :ok :read [19 17] INFO [2022-01-10 12:52:56,692] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:56,693] jepsen worker 20 - jepsen.util 20 :ok :write [6 10] INFO [2022-01-10 12:52:56,693] jepsen worker 0 - jepsen.util 0 :ok :read [0 37] INFO [2022-01-10 12:52:56,696] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:52:56,697] jepsen worker 81 - jepsen.util 81 :ok :read [27 15] INFO [2022-01-10 12:52:56,699] jepsen worker 46 - jepsen.util 46 :invoke :write [15 41] INFO [2022-01-10 12:52:56,703] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:52:56,704] jepsen worker 21 - jepsen.util 21 :ok :read [7 31] INFO [2022-01-10 12:52:56,704] jepsen worker 67 - jepsen.util 67 :ok :write [22 31] INFO [2022-01-10 12:52:56,706] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:56,707] jepsen worker 38 - jepsen.util 38 :ok :read [12 37] INFO [2022-01-10 12:52:56,708] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:56,710] jepsen worker 16 - jepsen.util 16 :ok :read [5 22] INFO [2022-01-10 12:52:56,711] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 12:52:56,712] jepsen worker 42 - jepsen.util 42 :ok :read [14 5] INFO [2022-01-10 12:52:56,713] jepsen worker 46 - jepsen.util 46 :ok :write [15 41] INFO [2022-01-10 12:52:56,714] jepsen worker 1 - jepsen.util 1 :invoke :write [0 26] INFO [2022-01-10 12:52:56,716] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:52:56,718] jepsen worker 86 - jepsen.util 86 :ok :read [28 16] INFO [2022-01-10 12:52:56,719] jepsen worker 10 - jepsen.util 10 :invoke :write [3 7] INFO [2022-01-10 12:52:56,721] jepsen worker 76 - jepsen.util 76 :invoke :write [25 32] INFO [2022-01-10 12:52:56,723] jepsen worker 79 - jepsen.util 79 :invoke :write [26 19] INFO [2022-01-10 12:52:56,726] jepsen worker 66 - jepsen.util 66 :invoke :write [22 15] INFO [2022-01-10 12:52:56,728] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:56,729] jepsen worker 78 - jepsen.util 78 :ok :read [26 41] INFO [2022-01-10 12:52:56,730] jepsen worker 44 - jepsen.util 44 :invoke :write [14 38] INFO [2022-01-10 12:52:56,732] jepsen worker 61 - jepsen.util 61 :invoke :write [20 35] INFO [2022-01-10 12:52:56,735] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:52:56,736] jepsen worker 5 - jepsen.util 5 :ok :read [1 14] INFO [2022-01-10 12:52:56,737] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:56,738] jepsen worker 45 - jepsen.util 45 :ok :read [15 41] INFO [2022-01-10 12:52:56,739] jepsen worker 81 - jepsen.util 81 :invoke :write [27 42] INFO [2022-01-10 12:52:56,741] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:56,742] jepsen worker 2 - jepsen.util 2 :ok :read [0 37] INFO [2022-01-10 12:52:56,744] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:52:56,745] jepsen worker 68 - jepsen.util 68 :ok :read [22 31] INFO [2022-01-10 12:52:56,746] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:52:56,747] jepsen worker 7 - jepsen.util 7 :ok :read [2 31] INFO [2022-01-10 12:52:56,748] jepsen worker 62 - jepsen.util 62 :invoke :write [20 12] INFO [2022-01-10 12:52:56,751] jepsen worker 49 - jepsen.util 49 :invoke :read [16 nil] INFO [2022-01-10 12:52:56,752] jepsen worker 49 - jepsen.util 49 :ok :read [16 19] INFO [2022-01-10 12:52:56,753] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:52:56,754] jepsen worker 35 - jepsen.util 35 :ok :read [11 12] INFO [2022-01-10 12:52:56,755] jepsen worker 36 - jepsen.util 36 :invoke :write [12 7] INFO [2022-01-10 12:52:56,758] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:56,759] jepsen worker 60 - jepsen.util 60 :ok :read [20 29] INFO [2022-01-10 12:52:56,760] jepsen worker 80 - jepsen.util 80 :invoke :write [26 49] INFO [2022-01-10 12:52:56,762] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:56,764] jepsen worker 78 - jepsen.util 78 :ok :read [26 41] INFO [2022-01-10 12:52:56,765] jepsen worker 40 - jepsen.util 40 :invoke :write [13 9] INFO [2022-01-10 12:52:56,767] jepsen worker 22 - jepsen.util 22 :invoke :write [7 29] INFO [2022-01-10 12:52:56,769] jepsen worker 52 - jepsen.util 52 :invoke :write [17 14] INFO [2022-01-10 12:52:56,772] jepsen worker 38 - jepsen.util 38 :invoke :write [12 15] INFO [2022-01-10 12:52:56,774] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 12:52:56,775] jepsen worker 42 - jepsen.util 42 :ok :read [14 5] INFO [2022-01-10 12:52:56,778] jepsen worker 25 - jepsen.util 25 :invoke :read [8 nil] INFO [2022-01-10 12:52:56,779] jepsen worker 25 - jepsen.util 25 :ok :read [8 1] INFO [2022-01-10 12:52:56,780] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:56,781] jepsen worker 67 - jepsen.util 67 :ok :read [22 31] INFO [2022-01-10 12:52:56,783] jepsen worker 74 - jepsen.util 74 :invoke :write [24 33] INFO [2022-01-10 12:52:56,785] jepsen worker 21 - jepsen.util 21 :invoke :write [7 29] INFO [2022-01-10 12:52:56,787] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:56,789] jepsen worker 9 - jepsen.util 9 :ok :read [3 32] INFO [2022-01-10 12:52:56,790] jepsen worker 20 - jepsen.util 20 :invoke :write [6 43] INFO [2022-01-10 12:52:56,792] jepsen worker 34 - jepsen.util 34 :invoke :write [11 34] INFO [2022-01-10 12:52:56,795] jepsen worker 14 - jepsen.util 14 :invoke :write [4 47] INFO [2022-01-10 12:52:56,797] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:56,798] jepsen worker 78 - jepsen.util 78 :ok :read [26 41] INFO [2022-01-10 12:52:56,799] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:56,800] jepsen worker 63 - jepsen.util 63 :ok :read [21 26] INFO [2022-01-10 12:52:56,801] jepsen worker 0 - jepsen.util 0 :invoke :write [0 15] INFO [2022-01-10 12:52:56,804] jepsen worker 32 - jepsen.util 32 :invoke :write [10 48] INFO [2022-01-10 12:52:56,806] jepsen worker 65 - jepsen.util 65 :invoke :write [21 49] INFO [2022-01-10 12:52:56,809] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:56,811] jepsen worker 18 - jepsen.util 18 :ok :read [6 10] INFO [2022-01-10 12:52:56,812] jepsen worker 37 - jepsen.util 37 :invoke :write [12 0] INFO [2022-01-10 12:52:56,813] jepsen worker 10 - jepsen.util 10 :ok :write [3 7] INFO [2022-01-10 12:52:56,815] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:56,817] jepsen worker 78 - jepsen.util 78 :ok :read [26 41] INFO [2022-01-10 12:52:56,818] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 12:52:56,819] jepsen worker 6 - jepsen.util 6 :ok :read [2 31] INFO [2022-01-10 12:52:56,821] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:56,822] jepsen worker 60 - jepsen.util 60 :ok :read [20 29] INFO [2022-01-10 12:52:56,823] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:52:56,825] jepsen worker 29 - jepsen.util 29 :ok :read [9 13] INFO [2022-01-10 12:52:56,825] jepsen worker 36 - jepsen.util 36 :ok :write [12 7] INFO [2022-01-10 12:52:56,827] jepsen worker 13 - jepsen.util 13 :invoke :write [4 8] INFO [2022-01-10 12:52:56,831] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:56,832] jepsen worker 36 - jepsen.util 36 :ok :read [12 7] INFO [2022-01-10 12:52:56,832] jepsen worker 1 - jepsen.util 1 :ok :write [0 26] INFO [2022-01-10 12:52:56,833] jepsen worker 76 - jepsen.util 76 :ok :write [25 32] INFO [2022-01-10 12:52:56,834] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:56,836] jepsen worker 67 - jepsen.util 67 :ok :read [22 31] INFO [2022-01-10 12:52:56,838] jepsen worker 40 - jepsen.util 40 :ok :write [13 9] INFO [2022-01-10 12:52:56,840] jepsen worker 27 - jepsen.util 27 :invoke :write [9 14] INFO [2022-01-10 12:52:56,842] jepsen worker 32 - jepsen.util 32 :ok :write [10 48] INFO [2022-01-10 12:52:56,845] jepsen worker 68 - jepsen.util 68 :invoke :write [22 24] INFO [2022-01-10 12:52:56,848] jepsen worker 0 - jepsen.util 0 :ok :write [0 15] INFO [2022-01-10 12:52:56,848] jepsen worker 88 - jepsen.util 88 :invoke :write [29 27] INFO [2022-01-10 12:52:56,852] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:56,852] jepsen worker 37 - jepsen.util 37 :ok :write [12 0] INFO [2022-01-10 12:52:56,854] jepsen worker 83 - jepsen.util 83 :ok :read [27 15] INFO [2022-01-10 12:52:56,855] jepsen worker 27 - jepsen.util 27 :ok :write [9 14] INFO [2022-01-10 12:52:56,855] jepsen worker 35 - jepsen.util 35 :invoke :write [11 40] INFO [2022-01-10 12:52:56,858] jepsen worker 67 - jepsen.util 67 :invoke :write [22 9] INFO [2022-01-10 12:52:56,860] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 12:52:56,862] jepsen worker 56 - jepsen.util 56 :ok :read [18 46] INFO [2022-01-10 12:52:56,863] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:52:56,864] jepsen worker 64 - jepsen.util 64 :ok :read [21 26] INFO [2022-01-10 12:52:56,864] jepsen worker 88 - jepsen.util 88 :ok :write [29 27] INFO [2022-01-10 12:52:56,865] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:56,866] jepsen worker 60 - jepsen.util 60 :ok :read [20 29] INFO [2022-01-10 12:52:56,867] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:56,868] jepsen worker 50 - jepsen.util 50 :ok :read [16 19] INFO [2022-01-10 12:52:56,869] jepsen worker 36 - jepsen.util 36 :invoke :write [12 17] INFO [2022-01-10 12:52:56,872] jepsen worker 53 - jepsen.util 53 :invoke :write [17 34] INFO [2022-01-10 12:52:56,874] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:56,875] jepsen worker 12 - jepsen.util 12 :ok :read [4 40] INFO [2022-01-10 12:52:56,876] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:52:56,877] jepsen worker 37 - jepsen.util 37 :ok :read [12 0] INFO [2022-01-10 12:52:56,878] jepsen worker 36 - jepsen.util 36 :ok :write [12 17] INFO [2022-01-10 12:52:56,879] jepsen worker 2 - jepsen.util 2 :invoke :write [0 48] INFO [2022-01-10 12:52:56,882] jepsen worker 37 - jepsen.util 37 :invoke :write [12 3] INFO [2022-01-10 12:52:56,885] jepsen worker 75 - jepsen.util 75 :invoke :write [25 8] INFO [2022-01-10 12:52:56,888] jepsen worker 2 - jepsen.util 2 :ok :write [0 48] INFO [2022-01-10 12:52:56,888] jepsen worker 18 - jepsen.util 18 :invoke :write [6 33] INFO [2022-01-10 12:52:56,891] jepsen worker 78 - jepsen.util 78 :invoke :write [26 34] INFO [2022-01-10 12:52:56,891] jepsen worker 37 - jepsen.util 37 :ok :write [12 3] INFO [2022-01-10 12:52:56,894] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:52:56,896] jepsen worker 47 - jepsen.util 47 :ok :read [15 41] INFO [2022-01-10 12:52:56,897] jepsen worker 9 - jepsen.util 9 :invoke :write [3 13] INFO [2022-01-10 12:52:56,900] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:56,901] jepsen worker 36 - jepsen.util 36 :ok :read [12 3] INFO [2022-01-10 12:52:56,903] jepsen worker 32 - jepsen.util 32 :invoke :write [10 28] INFO [2022-01-10 12:52:56,905] jepsen worker 9 - jepsen.util 9 :ok :write [3 13] INFO [2022-01-10 12:52:56,906] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:56,907] jepsen worker 26 - jepsen.util 26 :ok :read [8 1] INFO [2022-01-10 12:52:56,909] jepsen worker 31 - jepsen.util 31 :invoke :write [10 7] INFO [2022-01-10 12:52:56,911] jepsen worker 32 - jepsen.util 32 :ok :write [10 28] INFO [2022-01-10 12:52:56,911] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:52:56,912] jepsen worker 15 - jepsen.util 15 :ok :read [5 22] INFO [2022-01-10 12:52:56,914] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:56,915] jepsen worker 43 - jepsen.util 43 :ok :read [14 5] INFO [2022-01-10 12:52:56,917] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:52:56,918] jepsen worker 30 - jepsen.util 30 :ok :read [10 28] INFO [2022-01-10 12:52:56,921] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 12:52:56,922] jepsen worker 42 - jepsen.util 42 :ok :read [14 5] INFO [2022-01-10 12:52:56,924] jepsen worker 31 - jepsen.util 31 :ok :write [10 7] INFO [2022-01-10 12:52:56,924] jepsen worker 36 - jepsen.util 36 :invoke :write [12 23] INFO [2022-01-10 12:52:56,927] jepsen worker 9 - jepsen.util 9 :invoke :write [3 18] INFO [2022-01-10 12:52:56,930] jepsen worker 89 - jepsen.util 89 :invoke :write [29 19] INFO [2022-01-10 12:52:56,933] jepsen worker 36 - jepsen.util 36 :ok :write [12 23] INFO [2022-01-10 12:52:56,933] jepsen worker 64 - jepsen.util 64 :invoke :write [21 31] INFO [2022-01-10 12:52:56,937] jepsen worker 10 - jepsen.util 10 :invoke :write [3 7] INFO [2022-01-10 12:52:56,939] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:56,940] jepsen worker 77 - jepsen.util 77 :ok :read [25 32] INFO [2022-01-10 12:52:56,941] jepsen worker 89 - jepsen.util 89 :ok :write [29 19] INFO [2022-01-10 12:52:56,941] jepsen worker 9 - jepsen.util 9 :ok :write [3 18] INFO [2022-01-10 12:52:56,942] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:56,944] jepsen worker 51 - jepsen.util 51 :ok :read [17 43] INFO [2022-01-10 12:52:56,944] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:56,946] jepsen worker 27 - jepsen.util 27 :ok :read [9 14] INFO [2022-01-10 12:52:56,947] jepsen worker 5 - jepsen.util 5 :invoke :write [1 30] INFO [2022-01-10 12:52:56,950] jepsen worker 48 - jepsen.util 48 :invoke :write [16 48] INFO [2022-01-10 12:52:56,953] jepsen worker 10 - jepsen.util 10 :ok :write [3 7] INFO [2022-01-10 12:52:56,953] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:56,955] jepsen worker 60 - jepsen.util 60 :ok :read [20 29] INFO [2022-01-10 12:52:56,957] jepsen worker 73 - jepsen.util 73 :invoke :write [24 14] INFO [2022-01-10 12:52:56,960] jepsen worker 5 - jepsen.util 5 :ok :write [1 30] INFO [2022-01-10 12:52:56,960] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:52:56,961] jepsen worker 76 - jepsen.util 76 :ok :read [25 32] INFO [2022-01-10 12:52:56,962] jepsen worker 5 - jepsen.util 5 :invoke :write [1 35] INFO [2022-01-10 12:52:56,965] jepsen worker 38 - jepsen.util 38 :ok :write [12 15] INFO [2022-01-10 12:52:56,965] jepsen worker 45 - jepsen.util 45 :invoke :write [15 41] INFO [2022-01-10 12:52:56,968] jepsen worker 60 - jepsen.util 60 :invoke :write [20 29] INFO [2022-01-10 12:52:56,970] jepsen worker 76 - jepsen.util 76 :invoke :write [25 3] INFO [2022-01-10 12:52:56,972] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:56,973] jepsen worker 77 - jepsen.util 77 :ok :read [25 32] INFO [2022-01-10 12:52:56,974] jepsen worker 5 - jepsen.util 5 :ok :write [1 35] INFO [2022-01-10 12:52:56,974] jepsen worker 27 - jepsen.util 27 :invoke :write [9 24] INFO [2022-01-10 12:52:56,977] jepsen worker 83 - jepsen.util 83 :invoke :write [27 36] INFO [2022-01-10 12:52:56,979] jepsen worker 43 - jepsen.util 43 :invoke :write [14 39] INFO [2022-01-10 12:52:56,981] jepsen worker 51 - jepsen.util 51 :invoke :write [17 46] INFO [2022-01-10 12:52:56,984] jepsen worker 49 - jepsen.util 49 :invoke :write [16 16] INFO [2022-01-10 12:52:56,984] jepsen worker 27 - jepsen.util 27 :ok :write [9 24] INFO [2022-01-10 12:52:56,985] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:52:56,987] jepsen worker 1 - jepsen.util 1 :ok :read [0 48] INFO [2022-01-10 12:52:56,988] jepsen worker 24 - jepsen.util 24 :invoke :write [8 31] INFO [2022-01-10 12:52:56,990] jepsen worker 0 - jepsen.util 0 :invoke :write [0 49] INFO [2022-01-10 12:52:56,993] jepsen worker 8 - jepsen.util 8 :invoke :write [2 30] INFO [2022-01-10 12:52:56,995] jepsen worker 42 - jepsen.util 42 :invoke :write [14 35] INFO [2022-01-10 12:52:56,997] jepsen worker 23 - jepsen.util 23 :ok :write [7 8] INFO [2022-01-10 12:52:56,998] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 12:52:56,998] jepsen worker 0 - jepsen.util 0 :ok :write [0 49] INFO [2022-01-10 12:52:56,999] jepsen worker 56 - jepsen.util 56 :ok :read [18 46] INFO [2022-01-10 12:52:57,001] jepsen worker 36 - jepsen.util 36 :invoke :write [12 33] INFO [2022-01-10 12:52:57,001] jepsen worker 65 - jepsen.util 65 :ok :write [21 49] INFO [2022-01-10 12:52:57,004] jepsen worker 59 - jepsen.util 59 :invoke :write [19 6] INFO [2022-01-10 12:52:57,004] jepsen worker 61 - jepsen.util 61 :ok :write [20 35] INFO [2022-01-10 12:52:57,006] jepsen worker 9 - jepsen.util 9 :invoke :write [3 4] INFO [2022-01-10 12:52:57,008] jepsen worker 24 - jepsen.util 24 :ok :write [8 31] INFO [2022-01-10 12:52:57,008] jepsen worker 41 - jepsen.util 41 :invoke :write [13 47] INFO [2022-01-10 12:52:57,009] jepsen worker 33 - jepsen.util 33 :ok :write [11 27] INFO [2022-01-10 12:52:57,012] jepsen worker 30 - jepsen.util 30 :invoke :write [10 3] INFO [2022-01-10 12:52:57,014] jepsen worker 36 - jepsen.util 36 :ok :write [12 33] INFO [2022-01-10 12:52:57,015] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:57,017] jepsen worker 89 - jepsen.util 89 :ok :read [29 19] INFO [2022-01-10 12:52:57,018] jepsen worker 16 - jepsen.util 16 :invoke :write [5 49] INFO [2022-01-10 12:52:57,020] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:57,022] jepsen worker 33 - jepsen.util 33 :ok :read [11 27] INFO [2022-01-10 12:52:57,023] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:57,024] jepsen worker 50 - jepsen.util 50 :ok :read [16 19] INFO [2022-01-10 12:52:57,025] jepsen worker 41 - jepsen.util 41 :ok :write [13 47] INFO [2022-01-10 12:52:57,026] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:57,028] jepsen worker 36 - jepsen.util 36 :ok :read [12 33] INFO [2022-01-10 12:52:57,029] jepsen worker 27 - jepsen.util 27 :invoke :write [9 7] INFO [2022-01-10 12:52:57,032] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:57,033] jepsen worker 39 - jepsen.util 39 :ok :read [13 47] INFO [2022-01-10 12:52:57,035] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:57,037] jepsen worker 11 - jepsen.util 11 :ok :read [3 7] INFO [2022-01-10 12:52:57,038] jepsen worker 24 - jepsen.util 24 :invoke :write [8 20] INFO [2022-01-10 12:52:57,041] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:52:57,042] jepsen worker 54 - jepsen.util 54 :ok :read [18 46] INFO [2022-01-10 12:52:57,043] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:57,044] jepsen worker 61 - jepsen.util 61 :ok :read [20 35] INFO [2022-01-10 12:52:57,045] jepsen worker 4 - jepsen.util 4 :invoke :write [1 40] INFO [2022-01-10 12:52:57,048] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:57,049] jepsen worker 11 - jepsen.util 11 :ok :read [3 7] INFO [2022-01-10 12:52:57,050] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:52:57,052] jepsen worker 47 - jepsen.util 47 :ok :read [15 41] INFO [2022-01-10 12:52:57,053] jepsen worker 41 - jepsen.util 41 :invoke :write [13 46] INFO [2022-01-10 12:52:57,053] jepsen worker 4 - jepsen.util 4 :ok :write [1 40] INFO [2022-01-10 12:52:57,055] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:57,056] jepsen worker 89 - jepsen.util 89 :ok :read [29 19] INFO [2022-01-10 12:52:57,058] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:57,059] jepsen worker 69 - jepsen.util 69 :ok :read [23 28] INFO [2022-01-10 12:52:57,060] jepsen worker 36 - jepsen.util 36 :invoke :write [12 38] INFO [2022-01-10 12:52:57,061] jepsen worker 41 - jepsen.util 41 :ok :write [13 46] INFO [2022-01-10 12:52:57,062] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:57,064] jepsen worker 46 - jepsen.util 46 :ok :read [15 41] INFO [2022-01-10 12:52:57,065] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:57,066] jepsen worker 63 - jepsen.util 63 :ok :read [21 49] INFO [2022-01-10 12:52:57,068] jepsen worker 28 - jepsen.util 28 :invoke :read [9 nil] INFO [2022-01-10 12:52:57,068] jepsen worker 36 - jepsen.util 36 :ok :write [12 38] INFO [2022-01-10 12:52:57,069] jepsen worker 28 - jepsen.util 28 :ok :read [9 24] INFO [2022-01-10 12:52:57,071] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:57,072] jepsen worker 87 - jepsen.util 87 :ok :read [29 19] INFO [2022-01-10 12:52:57,073] jepsen worker 33 - jepsen.util 33 :invoke :write [11 18] INFO [2022-01-10 12:52:57,076] jepsen worker 37 - jepsen.util 37 :invoke :write [12 36] INFO [2022-01-10 12:52:57,078] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:57,079] jepsen worker 0 - jepsen.util 0 :ok :read [0 49] INFO [2022-01-10 12:52:57,081] jepsen worker 50 - jepsen.util 50 :invoke :write [16 10] INFO [2022-01-10 12:52:57,082] jepsen worker 37 - jepsen.util 37 :ok :write [12 36] INFO [2022-01-10 12:52:57,084] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 12:52:57,085] jepsen worker 56 - jepsen.util 56 :ok :read [18 46] INFO [2022-01-10 12:52:57,087] jepsen worker 23 - jepsen.util 23 :invoke :write [7 14] INFO [2022-01-10 12:52:57,089] jepsen worker 88 - jepsen.util 88 :invoke :read [29 nil] INFO [2022-01-10 12:52:57,090] jepsen worker 88 - jepsen.util 88 :ok :read [29 19] INFO [2022-01-10 12:52:57,091] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:52:57,092] jepsen worker 15 - jepsen.util 15 :ok :read [5 22] INFO [2022-01-10 12:52:57,093] jepsen worker 41 - jepsen.util 41 :invoke :write [13 3] INFO [2022-01-10 12:52:57,095] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:52:57,097] jepsen worker 57 - jepsen.util 57 :ok :read [19 17] INFO [2022-01-10 12:52:57,098] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:57,099] jepsen worker 87 - jepsen.util 87 :ok :read [29 19] INFO [2022-01-10 12:52:57,100] jepsen worker 5 - jepsen.util 5 :invoke :write [1 22] INFO [2022-01-10 12:52:57,101] jepsen worker 41 - jepsen.util 41 :ok :write [13 3] INFO [2022-01-10 12:52:57,102] jepsen worker 65 - jepsen.util 65 :invoke :write [21 9] INFO [2022-01-10 12:52:57,105] jepsen worker 89 - jepsen.util 89 :invoke :write [29 32] INFO [2022-01-10 12:52:57,107] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:57,108] jepsen worker 46 - jepsen.util 46 :ok :read [15 41] INFO [2022-01-10 12:52:57,109] jepsen worker 5 - jepsen.util 5 :ok :write [1 22] INFO [2022-01-10 12:52:57,110] jepsen worker 26 - jepsen.util 26 :invoke :write [8 46] INFO [2022-01-10 12:52:57,112] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:52:57,113] jepsen worker 47 - jepsen.util 47 :ok :read [15 41] INFO [2022-01-10 12:52:57,113] jepsen worker 25 - jepsen.util 25 :invoke :read [8 nil] INFO [2022-01-10 12:52:57,114] jepsen worker 89 - jepsen.util 89 :ok :write [29 32] INFO [2022-01-10 12:52:57,115] jepsen worker 25 - jepsen.util 25 :ok :read [8 31] INFO [2022-01-10 12:52:57,115] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:57,116] jepsen worker 77 - jepsen.util 77 :ok :read [25 32] INFO [2022-01-10 12:52:57,117] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:52:57,118] jepsen worker 5 - jepsen.util 5 :ok :read [1 22] INFO [2022-01-10 12:52:57,120] jepsen worker 25 - jepsen.util 25 :invoke :write [8 21] INFO [2022-01-10 12:52:57,123] jepsen worker 4 - jepsen.util 4 :invoke :write [1 26] INFO [2022-01-10 12:52:57,125] jepsen worker 27 - jepsen.util 27 :ok :write [9 7] INFO [2022-01-10 12:52:57,127] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:57,128] jepsen worker 0 - jepsen.util 0 :ok :read [0 49] INFO [2022-01-10 12:52:57,131] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:57,132] jepsen worker 70 - jepsen.util 70 :ok :write [23 32] INFO [2022-01-10 12:52:57,132] jepsen worker 46 - jepsen.util 46 :ok :read [15 41] INFO [2022-01-10 12:52:57,132] jepsen worker 59 - jepsen.util 59 :ok :write [19 6] INFO [2022-01-10 12:52:57,133] jepsen worker 4 - jepsen.util 4 :ok :write [1 26] INFO [2022-01-10 12:52:57,135] jepsen worker 12 - jepsen.util 12 :invoke :write [4 12] INFO [2022-01-10 12:52:57,135] jepsen worker 82 - jepsen.util 82 :ok :write [27 37] INFO [2022-01-10 12:52:57,135] jepsen worker 9 - jepsen.util 9 :ok :write [3 4] INFO [2022-01-10 12:52:57,138] jepsen worker 84 - jepsen.util 84 :invoke :write [28 21] INFO [2022-01-10 12:52:57,139] jepsen worker 19 - jepsen.util 19 :ok :write [6 15] INFO [2022-01-10 12:52:57,141] jepsen worker 28 - jepsen.util 28 :invoke :read [9 nil] INFO [2022-01-10 12:52:57,142] jepsen worker 28 - jepsen.util 28 :ok :read [9 7] INFO [2022-01-10 12:52:57,143] jepsen worker 17 - jepsen.util 17 :ok :write [5 40] INFO [2022-01-10 12:52:57,143] jepsen worker 45 - jepsen.util 45 :ok :write [15 41] INFO [2022-01-10 12:52:57,144] jepsen worker 31 - jepsen.util 31 :invoke :write [10 12] INFO [2022-01-10 12:52:57,147] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:57,147] jepsen worker 66 - jepsen.util 66 :ok :write [22 15] INFO [2022-01-10 12:52:57,148] jepsen worker 79 - jepsen.util 79 :ok :write [26 19] INFO [2022-01-10 12:52:57,148] jepsen worker 61 - jepsen.util 61 :ok :read [20 35] INFO [2022-01-10 12:52:57,149] jepsen worker 30 - jepsen.util 30 :ok :write [10 3] INFO [2022-01-10 12:52:57,151] jepsen worker 38 - jepsen.util 38 :invoke :write [12 10] INFO [2022-01-10 12:52:57,153] jepsen worker 44 - jepsen.util 44 :ok :write [14 38] INFO [2022-01-10 12:52:57,153] jepsen worker 79 - jepsen.util 79 :invoke :read [26 nil] INFO [2022-01-10 12:52:57,155] jepsen worker 79 - jepsen.util 79 :ok :read [26 19] INFO [2022-01-10 12:52:57,155] jepsen worker 52 - jepsen.util 52 :ok :write [17 14] INFO [2022-01-10 12:52:57,156] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:57,158] jepsen worker 82 - jepsen.util 82 :ok :read [27 37] INFO [2022-01-10 12:52:57,159] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:52:57,160] jepsen worker 75 - jepsen.util 75 :ok :write [25 8] INFO [2022-01-10 12:52:57,160] jepsen worker 55 - jepsen.util 55 :ok :read [18 46] INFO [2022-01-10 12:52:57,161] jepsen worker 74 - jepsen.util 74 :ok :write [24 33] INFO [2022-01-10 12:52:57,162] jepsen worker 63 - jepsen.util 63 :invoke :write [21 26] INFO [2022-01-10 12:52:57,166] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:57,166] jepsen worker 48 - jepsen.util 48 :ok :write [16 48] INFO [2022-01-10 12:52:57,167] jepsen worker 14 - jepsen.util 14 :ok :write [4 47] INFO [2022-01-10 12:52:57,167] jepsen worker 82 - jepsen.util 82 :ok :read [27 37] INFO [2022-01-10 12:52:57,170] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:52:57,171] jepsen worker 75 - jepsen.util 75 :ok :read [25 8] INFO [2022-01-10 12:52:57,171] jepsen worker 64 - jepsen.util 64 :ok :write [21 31] INFO [2022-01-10 12:52:57,173] jepsen worker 8 - jepsen.util 8 :ok :write [2 30] INFO [2022-01-10 12:52:57,174] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:57,176] jepsen worker 0 - jepsen.util 0 :ok :read [0 49] INFO [2022-01-10 12:52:57,176] jepsen worker 24 - jepsen.util 24 :ok :write [8 20] INFO [2022-01-10 12:52:57,178] jepsen worker 79 - jepsen.util 79 :invoke :read [26 nil] INFO [2022-01-10 12:52:57,179] jepsen worker 22 - jepsen.util 22 :ok :write [7 29] INFO [2022-01-10 12:52:57,179] jepsen worker 79 - jepsen.util 79 :ok :read [26 19] INFO [2022-01-10 12:52:57,181] jepsen worker 77 - jepsen.util 77 :invoke :write [25 14] INFO [2022-01-10 12:52:57,184] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:57,185] jepsen worker 9 - jepsen.util 9 :ok :read [3 4] INFO [2022-01-10 12:52:57,185] jepsen worker 60 - jepsen.util 60 :ok :write [20 29] INFO [2022-01-10 12:52:57,188] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:57,189] jepsen worker 82 - jepsen.util 82 :ok :read [27 37] INFO [2022-01-10 12:52:57,191] jepsen worker 89 - jepsen.util 89 :invoke :write [29 31] INFO [2022-01-10 12:52:57,191] jepsen worker 35 - jepsen.util 35 :ok :write [11 40] INFO [2022-01-10 12:52:57,194] jepsen worker 8 - jepsen.util 8 :invoke :write [2 0] INFO [2022-01-10 12:52:57,197] jepsen worker 83 - jepsen.util 83 :ok :write [27 36] INFO [2022-01-10 12:52:57,198] jepsen worker 27 - jepsen.util 27 :invoke :write [9 20] INFO [2022-01-10 12:52:57,201] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:57,203] jepsen worker 39 - jepsen.util 39 :ok :read [13 3] INFO [2022-01-10 12:52:57,205] jepsen worker 40 - jepsen.util 40 :invoke :write [13 15] INFO [2022-01-10 12:52:57,207] jepsen worker 68 - jepsen.util 68 :ok :write [22 24] INFO [2022-01-10 12:52:57,209] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:57,210] jepsen worker 60 - jepsen.util 60 :ok :read [20 29] INFO [2022-01-10 12:52:57,214] jepsen worker 70 - jepsen.util 70 :invoke :write [23 6] INFO [2022-01-10 12:52:57,215] jepsen worker 63 - jepsen.util 63 :ok :write [21 26] INFO [2022-01-10 12:52:57,219] jepsen worker 29 - jepsen.util 29 :invoke :write [9 48] INFO [2022-01-10 12:52:57,220] jepsen worker 25 - jepsen.util 25 :ok :write [8 21] INFO [2022-01-10 12:52:57,222] jepsen worker 60 - jepsen.util 60 :invoke :write [20 35] INFO [2022-01-10 12:52:57,224] jepsen worker 86 - jepsen.util 86 :invoke :write [28 9] INFO [2022-01-10 12:52:57,227] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:52:57,228] jepsen worker 35 - jepsen.util 35 :ok :read [11 40] INFO [2022-01-10 12:52:57,230] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:57,231] jepsen worker 82 - jepsen.util 82 :ok :read [27 36] INFO [2022-01-10 12:52:57,232] jepsen worker 18 - jepsen.util 18 :ok :write [6 33] INFO [2022-01-10 12:52:57,234] jepsen worker 75 - jepsen.util 75 :invoke :write [25 34] INFO [2022-01-10 12:52:57,237] jepsen worker 56 - jepsen.util 56 :invoke :write [18 29] INFO [2022-01-10 12:52:57,240] jepsen worker 14 - jepsen.util 14 :invoke :write [4 1] INFO [2022-01-10 12:52:57,243] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:52:57,244] jepsen worker 47 - jepsen.util 47 :ok :read [15 41] INFO [2022-01-10 12:52:57,246] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:52:57,247] jepsen worker 54 - jepsen.util 54 :ok :read [18 46] INFO [2022-01-10 12:52:57,249] jepsen worker 28 - jepsen.util 28 :invoke :write [9 44] INFO [2022-01-10 12:52:57,252] jepsen worker 74 - jepsen.util 74 :invoke :write [24 37] INFO [2022-01-10 12:52:57,255] jepsen worker 79 - jepsen.util 79 :invoke :read [26 nil] INFO [2022-01-10 12:52:57,257] jepsen worker 79 - jepsen.util 79 :ok :read [26 19] INFO [2022-01-10 12:52:57,258] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:57,259] jepsen worker 74 - jepsen.util 74 :ok :write [24 37] INFO [2022-01-10 12:52:57,260] jepsen worker 82 - jepsen.util 82 :ok :read [27 36] INFO [2022-01-10 12:52:57,264] jepsen worker 15 - jepsen.util 15 :invoke :write [5 34] INFO [2022-01-10 12:52:57,268] jepsen worker 54 - jepsen.util 54 :invoke :write [18 4] INFO [2022-01-10 12:52:57,271] jepsen worker 43 - jepsen.util 43 :ok :write [14 39] INFO [2022-01-10 12:52:57,273] jepsen worker 18 - jepsen.util 18 :invoke :write [6 41] INFO [2022-01-10 12:52:57,275] jepsen worker 73 - jepsen.util 73 :ok :write [24 14] INFO [2022-01-10 12:52:57,275] jepsen worker 84 - jepsen.util 84 :ok :write [28 21] INFO [2022-01-10 12:52:57,278] jepsen worker 77 - jepsen.util 77 :ok :write [25 14] INFO [2022-01-10 12:52:57,279] jepsen worker 30 - jepsen.util 30 :invoke :write [10 29] INFO [2022-01-10 12:52:57,281] jepsen worker 78 - jepsen.util 78 :ok :write [26 34] INFO [2022-01-10 12:52:57,284] jepsen worker 77 - jepsen.util 77 :invoke :write [25 31] INFO [2022-01-10 12:52:57,284] jepsen worker 23 - jepsen.util 23 :ok :write [7 14] INFO [2022-01-10 12:52:57,288] jepsen worker 31 - jepsen.util 31 :ok :write [10 12] INFO [2022-01-10 12:52:57,288] jepsen worker 35 - jepsen.util 35 :invoke :write [11 6] INFO [2022-01-10 12:52:57,288] jepsen worker 16 - jepsen.util 16 :ok :write [5 49] INFO [2022-01-10 12:52:57,291] jepsen worker 60 - jepsen.util 60 :ok :write [20 35] INFO [2022-01-10 12:52:57,291] jepsen worker 87 - jepsen.util 87 :invoke :write [29 49] INFO [2022-01-10 12:52:57,293] jepsen worker 38 - jepsen.util 38 :ok :write [12 10] INFO [2022-01-10 12:52:57,294] jepsen worker 24 - jepsen.util 24 :invoke :write [8 43] INFO [2022-01-10 12:52:57,297] jepsen worker 89 - jepsen.util 89 :ok :write [29 31] INFO [2022-01-10 12:52:57,298] jepsen worker 16 - jepsen.util 16 :invoke :write [5 29] INFO [2022-01-10 12:52:57,298] jepsen worker 50 - jepsen.util 50 :ok :write [16 10] INFO [2022-01-10 12:52:57,299] jepsen worker 51 - jepsen.util 51 :ok :write [17 46] INFO [2022-01-10 12:52:57,301] jepsen worker 10 - jepsen.util 10 :invoke :write [3 46] INFO [2022-01-10 12:52:57,303] jepsen worker 33 - jepsen.util 33 :ok :write [11 18] INFO [2022-01-10 12:52:57,305] jepsen worker 68 - jepsen.util 68 :invoke :write [22 10] INFO [2022-01-10 12:52:57,305] jepsen worker 56 - jepsen.util 56 :ok :write [18 29] INFO [2022-01-10 12:52:57,308] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:57,308] jepsen worker 12 - jepsen.util 12 :ok :write [4 12] INFO [2022-01-10 12:52:57,309] jepsen worker 33 - jepsen.util 33 :ok :read [11 18] INFO [2022-01-10 12:52:57,311] jepsen worker 56 - jepsen.util 56 :invoke :write [18 46] INFO [2022-01-10 12:52:57,311] jepsen worker 8 - jepsen.util 8 :ok :write [2 0] INFO [2022-01-10 12:52:57,313] jepsen worker 70 - jepsen.util 70 :ok :write [23 6] INFO [2022-01-10 12:52:57,314] jepsen worker 33 - jepsen.util 33 :invoke :write [11 42] INFO [2022-01-10 12:52:57,316] jepsen worker 27 - jepsen.util 27 :ok :write [9 20] INFO [2022-01-10 12:52:57,317] jepsen worker 40 - jepsen.util 40 :ok :write [13 15] INFO [2022-01-10 12:52:57,317] jepsen worker 75 - jepsen.util 75 :ok :write [25 34] INFO [2022-01-10 12:52:57,318] jepsen worker 86 - jepsen.util 86 :ok :write [28 9] INFO [2022-01-10 12:52:57,318] jepsen worker 47 - jepsen.util 47 :invoke :write [15 19] INFO [2022-01-10 12:52:57,320] jepsen worker 22 - jepsen.util 22 :invoke :read [7 nil] INFO [2022-01-10 12:52:57,322] jepsen worker 42 - jepsen.util 42 :ok :write [14 35] INFO [2022-01-10 12:52:57,322] jepsen worker 22 - jepsen.util 22 :ok :read [7 14] INFO [2022-01-10 12:52:57,324] jepsen worker 11 - jepsen.util 11 :invoke :write [3 3] INFO [2022-01-10 12:52:57,328] jepsen worker 15 - jepsen.util 15 :ok :write [5 34] INFO [2022-01-10 12:52:57,328] jepsen worker 32 - jepsen.util 32 :invoke :read [10 nil] INFO [2022-01-10 12:52:57,329] jepsen worker 32 - jepsen.util 32 :ok :read [10 12] INFO [2022-01-10 12:52:57,331] jepsen worker 22 - jepsen.util 22 :invoke :write [7 18] INFO [2022-01-10 12:52:57,335] jepsen worker 58 - jepsen.util 58 :invoke :write [19 31] INFO [2022-01-10 12:52:57,336] jepsen worker 87 - jepsen.util 87 :ok :write [29 49] INFO [2022-01-10 12:52:57,338] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:52:57,339] jepsen worker 5 - jepsen.util 5 :ok :read [1 26] INFO [2022-01-10 12:52:57,339] jepsen worker 35 - jepsen.util 35 :ok :write [11 6] INFO [2022-01-10 12:52:57,343] jepsen worker 66 - jepsen.util 66 :invoke :write [22 30] INFO [2022-01-10 12:52:57,343] jepsen worker 68 - jepsen.util 68 :ok :write [22 10] INFO [2022-01-10 12:52:57,343] jepsen worker 81 - jepsen.util 81 :ok :write [27 42] INFO [2022-01-10 12:52:57,347] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:57,348] jepsen worker 60 - jepsen.util 60 :ok :read [20 35] INFO [2022-01-10 12:52:57,352] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:57,353] jepsen worker 14 - jepsen.util 14 :ok :write [4 1] INFO [2022-01-10 12:52:57,353] jepsen worker 12 - jepsen.util 12 :ok :read [4 1] INFO [2022-01-10 12:52:57,355] jepsen worker 62 - jepsen.util 62 :ok :write [20 12] INFO [2022-01-10 12:52:57,356] jepsen worker 55 - jepsen.util 55 :invoke :write [18 27] INFO [2022-01-10 12:52:57,360] jepsen worker 42 - jepsen.util 42 :invoke :write [14 31] INFO [2022-01-10 12:52:57,361] jepsen worker 77 - jepsen.util 77 :ok :write [25 31] INFO [2022-01-10 12:52:57,365] jepsen worker 84 - jepsen.util 84 :invoke :write [28 7] INFO [2022-01-10 12:52:57,366] jepsen worker 80 - jepsen.util 80 :ok :write [26 49] INFO [2022-01-10 12:52:57,368] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:57,369] jepsen worker 85 - jepsen.util 85 :ok :read [28 9] INFO [2022-01-10 12:52:57,372] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:57,373] jepsen worker 50 - jepsen.util 50 :ok :read [16 10] INFO [2022-01-10 12:52:57,375] jepsen worker 68 - jepsen.util 68 :invoke :write [22 34] INFO [2022-01-10 12:52:57,378] jepsen worker 2 - jepsen.util 2 :invoke :write [0 43] INFO [2022-01-10 12:52:57,380] jepsen worker 84 - jepsen.util 84 :ok :write [28 7] INFO [2022-01-10 12:52:57,381] jepsen worker 63 - jepsen.util 63 :invoke :write [21 8] INFO [2022-01-10 12:52:57,384] jepsen worker 31 - jepsen.util 31 :invoke :write [10 12] INFO [2022-01-10 12:52:57,386] jepsen worker 2 - jepsen.util 2 :ok :write [0 43] INFO [2022-01-10 12:52:57,387] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:52:57,388] jepsen worker 5 - jepsen.util 5 :ok :read [1 26] INFO [2022-01-10 12:52:57,391] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:52:57,392] jepsen worker 52 - jepsen.util 52 :ok :read [17 46] INFO [2022-01-10 12:52:57,394] jepsen worker 18 - jepsen.util 18 :ok :write [6 41] INFO [2022-01-10 12:52:57,394] jepsen worker 32 - jepsen.util 32 :invoke :write [10 16] INFO [2022-01-10 12:52:57,397] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:57,398] jepsen worker 65 - jepsen.util 65 :ok :write [21 9] INFO [2022-01-10 12:52:57,398] jepsen worker 51 - jepsen.util 51 :ok :read [17 46] INFO [2022-01-10 12:52:57,400] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 12:52:57,401] jepsen worker 17 - jepsen.util 17 :ok :read [5 34] INFO [2022-01-10 12:52:57,403] jepsen worker 56 - jepsen.util 56 :ok :write [18 46] INFO [2022-01-10 12:52:57,403] jepsen worker 30 - jepsen.util 30 :ok :write [10 29] INFO [2022-01-10 12:52:57,404] jepsen worker 88 - jepsen.util 88 :invoke :read [29 nil] INFO [2022-01-10 12:52:57,405] jepsen worker 88 - jepsen.util 88 :ok :read [29 49] INFO [2022-01-10 12:52:57,408] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:57,409] jepsen worker 4 - jepsen.util 4 :ok :read [1 26] INFO [2022-01-10 12:52:57,411] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 12:52:57,412] jepsen worker 17 - jepsen.util 17 :ok :read [5 34] INFO [2022-01-10 12:52:57,413] jepsen worker 26 - jepsen.util 26 :ok :write [8 46] INFO [2022-01-10 12:52:57,414] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:52:57,414] jepsen worker 28 - jepsen.util 28 :ok :write [9 44] INFO [2022-01-10 12:52:57,415] jepsen worker 10 - jepsen.util 10 :ok :write [3 46] INFO [2022-01-10 12:52:57,416] jepsen worker 65 - jepsen.util 65 :ok :read [21 9] INFO [2022-01-10 12:52:57,418] jepsen worker 88 - jepsen.util 88 :invoke :write [29 25] INFO [2022-01-10 12:52:57,422] jepsen worker 55 - jepsen.util 55 :ok :write [18 27] INFO [2022-01-10 12:52:57,422] jepsen worker 89 - jepsen.util 89 :invoke :write [29 16] INFO [2022-01-10 12:52:57,424] jepsen worker 47 - jepsen.util 47 :ok :write [15 19] INFO [2022-01-10 12:52:57,426] jepsen worker 88 - jepsen.util 88 :ok :write [29 25] INFO [2022-01-10 12:52:57,428] jepsen worker 35 - jepsen.util 35 :invoke :write [11 32] INFO [2022-01-10 12:52:57,432] jepsen worker 58 - jepsen.util 58 :ok :write [19 31] INFO [2022-01-10 12:52:57,432] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:57,433] jepsen worker 83 - jepsen.util 83 :ok :read [27 42] INFO [2022-01-10 12:52:57,435] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:57,436] jepsen worker 82 - jepsen.util 82 :ok :read [27 42] INFO [2022-01-10 12:52:57,438] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:57,439] jepsen worker 39 - jepsen.util 39 :ok :read [13 15] INFO [2022-01-10 12:52:57,441] jepsen worker 64 - jepsen.util 64 :invoke :write [21 3] INFO [2022-01-10 12:52:57,441] jepsen worker 89 - jepsen.util 89 :ok :write [29 16] INFO [2022-01-10 12:52:57,443] jepsen worker 32 - jepsen.util 32 :ok :write [10 16] INFO [2022-01-10 12:52:57,444] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:57,445] jepsen worker 74 - jepsen.util 74 :ok :read [24 14] INFO [2022-01-10 12:52:57,447] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:57,448] jepsen worker 87 - jepsen.util 87 :ok :read [29 16] INFO [2022-01-10 12:52:57,449] jepsen worker 11 - jepsen.util 11 :ok :write [3 3] INFO [2022-01-10 12:52:57,450] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 12:52:57,451] jepsen worker 54 - jepsen.util 54 :ok :write [18 4] INFO [2022-01-10 12:52:57,452] jepsen worker 80 - jepsen.util 80 :ok :read [26 49] INFO [2022-01-10 12:52:57,454] jepsen worker 23 - jepsen.util 23 :invoke :write [7 14] INFO [2022-01-10 12:52:57,457] jepsen worker 83 - jepsen.util 83 :invoke :write [27 41] INFO [2022-01-10 12:52:57,460] jepsen worker 15 - jepsen.util 15 :invoke :write [5 36] INFO [2022-01-10 12:52:57,463] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:57,464] jepsen worker 0 - jepsen.util 0 :ok :read [0 43] INFO [2022-01-10 12:52:57,466] jepsen worker 8 - jepsen.util 8 :invoke :write [2 7] INFO [2022-01-10 12:52:57,467] jepsen worker 31 - jepsen.util 31 :ok :write [10 12] INFO [2022-01-10 12:52:57,469] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:57,470] jepsen worker 26 - jepsen.util 26 :ok :read [8 46] INFO [2022-01-10 12:52:57,474] jepsen worker 10 - jepsen.util 10 :invoke :write [3 28] INFO [2022-01-10 12:52:57,478] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:57,479] jepsen worker 44 - jepsen.util 44 :ok :read [14 35] INFO [2022-01-10 12:52:57,481] jepsen worker 10 - jepsen.util 10 :ok :write [3 28] INFO [2022-01-10 12:52:57,481] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:52:57,482] jepsen worker 75 - jepsen.util 75 :ok :read [25 31] INFO [2022-01-10 12:52:57,485] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:57,486] jepsen worker 51 - jepsen.util 51 :ok :read [17 46] INFO [2022-01-10 12:52:57,488] jepsen worker 4 - jepsen.util 4 :invoke :write [1 36] INFO [2022-01-10 12:52:57,492] jepsen worker 79 - jepsen.util 79 :invoke :write [26 2] INFO [2022-01-10 12:52:57,495] jepsen worker 4 - jepsen.util 4 :ok :write [1 36] INFO [2022-01-10 12:52:57,496] jepsen worker 5 - jepsen.util 5 :invoke :write [1 9] INFO [2022-01-10 12:52:57,500] jepsen worker 57 - jepsen.util 57 :invoke :write [19 15] INFO [2022-01-10 12:52:57,504] jepsen worker 5 - jepsen.util 5 :ok :write [1 9] INFO [2022-01-10 12:52:57,505] jepsen worker 48 - jepsen.util 48 :invoke :write [16 23] INFO [2022-01-10 12:52:57,507] jepsen worker 55 - jepsen.util 55 :invoke :write [18 40] INFO [2022-01-10 12:52:57,509] jepsen worker 57 - jepsen.util 57 :ok :write [19 15] INFO [2022-01-10 12:52:57,510] jepsen worker 16 - jepsen.util 16 :ok :write [5 29] INFO [2022-01-10 12:52:57,510] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:57,512] jepsen worker 27 - jepsen.util 27 :ok :read [9 44] INFO [2022-01-10 12:52:57,514] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:52:57,515] jepsen worker 22 - jepsen.util 22 :ok :write [7 18] INFO [2022-01-10 12:52:57,515] jepsen worker 65 - jepsen.util 65 :ok :read [21 9] INFO [2022-01-10 12:52:57,517] jepsen worker 56 - jepsen.util 56 :invoke :write [18 31] INFO [2022-01-10 12:52:57,520] jepsen worker 33 - jepsen.util 33 :ok :write [11 42] INFO [2022-01-10 12:52:57,521] jepsen worker 69 - jepsen.util 69 :invoke :write [23 27] INFO [2022-01-10 12:52:57,521] jepsen worker 13 - jepsen.util 13 :ok :write [4 8] INFO [2022-01-10 12:52:57,525] jepsen worker 66 - jepsen.util 66 :ok :write [22 30] INFO [2022-01-10 12:52:57,525] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:57,526] jepsen worker 4 - jepsen.util 4 :ok :read [1 9] INFO [2022-01-10 12:52:57,526] jepsen worker 63 - jepsen.util 63 :ok :write [21 8] INFO [2022-01-10 12:52:57,528] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:57,529] jepsen worker 38 - jepsen.util 38 :ok :read [12 10] INFO [2022-01-10 12:52:57,530] jepsen worker 83 - jepsen.util 83 :ok :write [27 41] INFO [2022-01-10 12:52:57,531] jepsen worker 24 - jepsen.util 24 :ok :write [8 43] INFO [2022-01-10 12:52:57,531] jepsen worker 58 - jepsen.util 58 :invoke :write [19 15] INFO [2022-01-10 12:52:57,534] jepsen worker 45 - jepsen.util 45 :invoke :write [15 27] INFO [2022-01-10 12:52:57,537] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:52:57,538] jepsen worker 66 - jepsen.util 66 :ok :read [22 30] INFO [2022-01-10 12:52:57,538] jepsen worker 29 - jepsen.util 29 :ok :write [9 48] INFO [2022-01-10 12:52:57,539] jepsen worker 84 - jepsen.util 84 :invoke :write [28 7] INFO [2022-01-10 12:52:57,539] jepsen worker 53 - jepsen.util 53 :ok :write [17 34] INFO [2022-01-10 12:52:57,540] jepsen worker 58 - jepsen.util 58 :ok :write [19 15] INFO [2022-01-10 12:52:57,541] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:57,542] jepsen worker 50 - jepsen.util 50 :ok :read [16 10] INFO [2022-01-10 12:52:57,544] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:52:57,544] jepsen worker 42 - jepsen.util 42 :ok :write [14 31] INFO [2022-01-10 12:52:57,545] jepsen worker 24 - jepsen.util 24 :ok :read [8 43] INFO [2022-01-10 12:52:57,546] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:52:57,547] jepsen worker 47 - jepsen.util 47 :ok :read [15 19] INFO [2022-01-10 12:52:57,548] jepsen worker 8 - jepsen.util 8 :ok :write [2 7] INFO [2022-01-10 12:52:57,549] jepsen worker 24 - jepsen.util 24 :invoke :write [8 42] INFO [2022-01-10 12:52:57,551] jepsen worker 79 - jepsen.util 79 :ok :write [26 2] INFO [2022-01-10 12:52:57,553] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:57,554] jepsen worker 61 - jepsen.util 61 :ok :read [20 12] INFO [2022-01-10 12:52:57,555] jepsen worker 48 - jepsen.util 48 :ok :write [16 23] INFO [2022-01-10 12:52:57,556] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:52:57,558] jepsen worker 89 - jepsen.util 89 :ok :read [29 16] INFO [2022-01-10 12:52:57,560] jepsen worker 61 - jepsen.util 61 :invoke :write [20 39] INFO [2022-01-10 12:52:57,560] jepsen worker 15 - jepsen.util 15 :ok :write [5 36] INFO [2022-01-10 12:52:57,563] jepsen worker 22 - jepsen.util 22 :invoke :read [7 nil] INFO [2022-01-10 12:52:57,564] jepsen worker 24 - jepsen.util 24 :ok :write [8 42] INFO [2022-01-10 12:52:57,564] jepsen worker 22 - jepsen.util 22 :ok :read [7 18] INFO [2022-01-10 12:52:57,566] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:52:57,567] jepsen worker 3 - jepsen.util 3 :ok :read [1 9] INFO [2022-01-10 12:52:57,568] jepsen worker 45 - jepsen.util 45 :ok :write [15 27] INFO [2022-01-10 12:52:57,569] jepsen worker 35 - jepsen.util 35 :ok :write [11 32] INFO [2022-01-10 12:52:57,570] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:52:57,571] jepsen worker 70 - jepsen.util 70 :ok :read [23 6] INFO [2022-01-10 12:52:57,573] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:52:57,574] jepsen worker 73 - jepsen.util 73 :ok :read [24 14] INFO [2022-01-10 12:52:57,577] jepsen worker 68 - jepsen.util 68 :ok :write [22 34] INFO [2022-01-10 12:52:57,577] jepsen worker 25 - jepsen.util 25 :invoke :write [8 9] INFO [2022-01-10 12:52:57,581] jepsen worker 2 - jepsen.util 2 :invoke :write [0 42] INFO [2022-01-10 12:52:57,584] jepsen worker 64 - jepsen.util 64 :ok :write [21 3] INFO [2022-01-10 12:52:57,584] jepsen worker 16 - jepsen.util 16 :invoke :write [5 31] INFO [2022-01-10 12:52:57,588] jepsen worker 88 - jepsen.util 88 :invoke :write [29 37] INFO [2022-01-10 12:52:57,591] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:52:57,592] jepsen worker 70 - jepsen.util 70 :ok :read [23 6] INFO [2022-01-10 12:52:57,595] jepsen worker 37 - jepsen.util 37 :invoke :write [12 23] INFO [2022-01-10 12:52:57,597] jepsen worker 72 - jepsen.util 72 :ok :write [24 46] INFO [2022-01-10 12:52:57,599] jepsen worker 28 - jepsen.util 28 :invoke :write [9 25] INFO [2022-01-10 12:52:57,603] jepsen worker 62 - jepsen.util 62 :invoke :write [20 48] INFO [2022-01-10 12:52:57,604] jepsen worker 16 - jepsen.util 16 :ok :write [5 31] INFO [2022-01-10 12:52:57,606] jepsen worker 70 - jepsen.util 70 :invoke :write [23 33] INFO [2022-01-10 12:52:57,609] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:57,610] jepsen worker 41 - jepsen.util 41 :ok :read [13 15] INFO [2022-01-10 12:52:57,613] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:57,614] jepsen worker 51 - jepsen.util 51 :ok :read [17 34] INFO [2022-01-10 12:52:57,616] jepsen worker 75 - jepsen.util 75 :invoke :write [25 0] INFO [2022-01-10 12:52:57,620] jepsen worker 41 - jepsen.util 41 :invoke :write [13 24] INFO [2022-01-10 12:52:57,623] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:57,624] jepsen worker 77 - jepsen.util 77 :ok :read [25 31] INFO [2022-01-10 12:52:57,626] jepsen worker 8 - jepsen.util 8 :invoke :write [2 34] INFO [2022-01-10 12:52:57,629] jepsen worker 87 - jepsen.util 87 :invoke :write [29 40] INFO [2022-01-10 12:52:57,633] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:57,635] jepsen worker 48 - jepsen.util 48 :ok :read [16 23] INFO [2022-01-10 12:52:57,637] jepsen worker 58 - jepsen.util 58 :invoke :write [19 8] INFO [2022-01-10 12:52:57,638] jepsen worker 55 - jepsen.util 55 :ok :write [18 40] INFO [2022-01-10 12:52:57,640] jepsen worker 27 - jepsen.util 27 :invoke :write [9 21] INFO [2022-01-10 12:52:57,641] jepsen worker 69 - jepsen.util 69 :ok :write [23 27] INFO [2022-01-10 12:52:57,644] jepsen worker 23 - jepsen.util 23 :ok :write [7 14] INFO [2022-01-10 12:52:57,644] jepsen worker 12 - jepsen.util 12 :invoke :write [4 20] INFO [2022-01-10 12:52:57,648] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:52:57,649] jepsen worker 24 - jepsen.util 24 :ok :read [8 42] INFO [2022-01-10 12:52:57,649] jepsen worker 84 - jepsen.util 84 :ok :write [28 7] INFO [2022-01-10 12:52:57,651] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:57,652] jepsen worker 39 - jepsen.util 39 :ok :read [13 15] INFO [2022-01-10 12:52:57,655] jepsen worker 71 - jepsen.util 71 :invoke :write [23 6] INFO [2022-01-10 12:52:57,655] jepsen worker 76 - jepsen.util 76 :ok :write [25 3] INFO [2022-01-10 12:52:57,658] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:57,659] jepsen worker 77 - jepsen.util 77 :ok :read [25 3] INFO [2022-01-10 12:52:57,661] jepsen worker 61 - jepsen.util 61 :ok :write [20 39] INFO [2022-01-10 12:52:57,661] jepsen worker 70 - jepsen.util 70 :ok :write [23 33] INFO [2022-01-10 12:52:57,662] jepsen worker 7 - jepsen.util 7 :invoke :write [2 34] INFO [2022-01-10 12:52:57,665] jepsen worker 82 - jepsen.util 82 :invoke :write [27 23] INFO [2022-01-10 12:52:57,666] jepsen worker 12 - jepsen.util 12 :ok :write [4 20] INFO [2022-01-10 12:52:57,667] jepsen worker 2 - jepsen.util 2 :ok :write [0 42] INFO [2022-01-10 12:52:57,668] jepsen worker 16 - jepsen.util 16 :invoke :write [5 44] INFO [2022-01-10 12:52:57,672] jepsen worker 61 - jepsen.util 61 :invoke :write [20 35] INFO [2022-01-10 12:52:57,675] jepsen worker 66 - jepsen.util 66 :invoke :write [22 35] INFO [2022-01-10 12:52:57,679] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:52:57,680] jepsen worker 1 - jepsen.util 1 :ok :read [0 42] INFO [2022-01-10 12:52:57,680] jepsen worker 71 - jepsen.util 71 :ok :write [23 6] INFO [2022-01-10 12:52:57,682] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:52:57,683] jepsen worker 64 - jepsen.util 64 :ok :read [21 3] INFO [2022-01-10 12:52:57,685] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:57,686] jepsen worker 44 - jepsen.util 44 :ok :read [14 31] INFO [2022-01-10 12:52:57,688] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:52:57,689] jepsen worker 52 - jepsen.util 52 :ok :read [17 34] INFO [2022-01-10 12:52:57,690] jepsen worker 75 - jepsen.util 75 :ok :write [25 0] INFO [2022-01-10 12:52:57,692] jepsen worker 79 - jepsen.util 79 :invoke :read [26 nil] INFO [2022-01-10 12:52:57,692] jepsen worker 62 - jepsen.util 62 :ok :write [20 48] INFO [2022-01-10 12:52:57,693] jepsen worker 79 - jepsen.util 79 :ok :read [26 2] INFO [2022-01-10 12:52:57,694] jepsen worker 24 - jepsen.util 24 :invoke :write [8 22] INFO [2022-01-10 12:52:57,697] jepsen worker 19 - jepsen.util 19 :invoke :write [6 46] INFO [2022-01-10 12:52:57,699] jepsen worker 74 - jepsen.util 74 :invoke :write [24 44] INFO [2022-01-10 12:52:57,701] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:52:57,703] jepsen worker 40 - jepsen.util 40 :ok :read [13 15] INFO [2022-01-10 12:52:57,704] jepsen worker 2 - jepsen.util 2 :invoke :write [0 27] INFO [2022-01-10 12:52:57,704] jepsen worker 61 - jepsen.util 61 :ok :write [20 35] INFO [2022-01-10 12:52:57,706] jepsen worker 11 - jepsen.util 11 :invoke :write [3 19] INFO [2022-01-10 12:52:57,709] jepsen worker 25 - jepsen.util 25 :ok :write [8 9] INFO [2022-01-10 12:52:57,709] jepsen worker 15 - jepsen.util 15 :invoke :write [5 42] INFO [2022-01-10 12:52:57,712] jepsen worker 50 - jepsen.util 50 :invoke :write [16 15] INFO [2022-01-10 12:52:57,714] jepsen worker 56 - jepsen.util 56 :ok :write [18 31] INFO [2022-01-10 12:52:57,716] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:52:57,717] jepsen worker 35 - jepsen.util 35 :ok :read [11 32] INFO [2022-01-10 12:52:57,718] jepsen worker 74 - jepsen.util 74 :ok :write [24 44] INFO [2022-01-10 12:52:57,718] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:57,720] jepsen worker 85 - jepsen.util 85 :ok :read [28 7] INFO [2022-01-10 12:52:57,722] jepsen worker 89 - jepsen.util 89 :invoke :write [29 49] INFO [2022-01-10 12:52:57,724] jepsen worker 88 - jepsen.util 88 :ok :write [29 37] INFO [2022-01-10 12:52:57,725] jepsen worker 37 - jepsen.util 37 :ok :write [12 23] INFO [2022-01-10 12:52:57,725] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:57,726] jepsen worker 19 - jepsen.util 19 :ok :write [6 46] INFO [2022-01-10 12:52:57,726] jepsen worker 45 - jepsen.util 45 :ok :read [15 27] INFO [2022-01-10 12:52:57,728] jepsen worker 28 - jepsen.util 28 :ok :write [9 25] INFO [2022-01-10 12:52:57,729] jepsen worker 12 - jepsen.util 12 :invoke :write [4 41] INFO [2022-01-10 12:52:57,731] jepsen worker 11 - jepsen.util 11 :ok :write [3 19] INFO [2022-01-10 12:52:57,731] jepsen worker 58 - jepsen.util 58 :ok :write [19 8] INFO [2022-01-10 12:52:57,733] jepsen worker 68 - jepsen.util 68 :invoke :write [22 14] INFO [2022-01-10 12:52:57,733] jepsen worker 16 - jepsen.util 16 :ok :write [5 44] INFO [2022-01-10 12:52:57,736] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:57,737] jepsen worker 18 - jepsen.util 18 :ok :read [6 46] INFO [2022-01-10 12:52:57,738] jepsen worker 24 - jepsen.util 24 :ok :write [8 22] INFO [2022-01-10 12:52:57,740] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:52:57,741] jepsen worker 41 - jepsen.util 41 :ok :write [13 24] INFO [2022-01-10 12:52:57,741] jepsen worker 14 - jepsen.util 14 :ok :read [4 20] INFO [2022-01-10 12:52:57,743] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:57,745] jepsen worker 43 - jepsen.util 43 :ok :read [14 31] INFO [2022-01-10 12:52:57,746] jepsen worker 8 - jepsen.util 8 :ok :write [2 34] INFO [2022-01-10 12:52:57,747] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:57,748] jepsen worker 71 - jepsen.util 71 :ok :read [23 6] INFO [2022-01-10 12:52:57,750] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 12:52:57,751] jepsen worker 82 - jepsen.util 82 :ok :write [27 23] INFO [2022-01-10 12:52:57,751] jepsen worker 56 - jepsen.util 56 :ok :read [18 31] INFO [2022-01-10 12:52:57,754] jepsen worker 88 - jepsen.util 88 :invoke :read [29 nil] INFO [2022-01-10 12:52:57,755] jepsen worker 88 - jepsen.util 88 :ok :read [29 37] INFO [2022-01-10 12:52:57,757] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:57,757] jepsen worker 2 - jepsen.util 2 :ok :write [0 27] INFO [2022-01-10 12:52:57,758] jepsen worker 16 - jepsen.util 16 :ok :read [5 44] INFO [2022-01-10 12:52:57,760] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:52:57,761] jepsen worker 55 - jepsen.util 55 :ok :read [18 31] INFO [2022-01-10 12:52:57,761] jepsen worker 66 - jepsen.util 66 :ok :write [22 35] INFO [2022-01-10 12:52:57,763] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:57,764] jepsen worker 59 - jepsen.util 59 :ok :read [19 8] INFO [2022-01-10 12:52:57,766] jepsen worker 50 - jepsen.util 50 :ok :write [16 15] INFO [2022-01-10 12:52:57,766] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:52:57,767] jepsen worker 35 - jepsen.util 35 :ok :read [11 32] INFO [2022-01-10 12:52:57,770] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:57,771] jepsen worker 26 - jepsen.util 26 :ok :read [8 22] INFO [2022-01-10 12:52:57,772] jepsen worker 89 - jepsen.util 89 :ok :write [29 49] INFO [2022-01-10 12:52:57,773] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:52:57,774] jepsen worker 55 - jepsen.util 55 :ok :read [18 31] INFO [2022-01-10 12:52:57,776] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:52:57,776] jepsen worker 27 - jepsen.util 27 :ok :write [9 21] INFO [2022-01-10 12:52:57,777] jepsen worker 8 - jepsen.util 8 :ok :read [2 34] INFO [2022-01-10 12:52:57,779] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:57,783] jepsen worker 33 - jepsen.util 33 :ok :read [11 32] INFO [2022-01-10 12:52:57,783] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:57,784] jepsen worker 69 - jepsen.util 69 :ok :read [23 6] INFO [2022-01-10 12:52:57,786] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:52:57,787] jepsen worker 8 - jepsen.util 8 :ok :read [2 34] INFO [2022-01-10 12:52:57,789] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:57,791] jepsen worker 23 - jepsen.util 23 :ok :read [7 14] INFO [2022-01-10 12:52:57,795] jepsen worker 80 - jepsen.util 80 :invoke :write [26 32] INFO [2022-01-10 12:52:57,796] jepsen worker 76 - jepsen.util 76 :invoke :write [25 39] INFO [2022-01-10 12:52:57,799] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:52:57,800] jepsen worker 14 - jepsen.util 14 :ok :read [4 20] INFO [2022-01-10 12:52:57,802] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:52:57,803] jepsen worker 70 - jepsen.util 70 :ok :read [23 6] INFO [2022-01-10 12:52:57,805] jepsen worker 52 - jepsen.util 52 :invoke :write [17 28] INFO [2022-01-10 12:52:57,808] jepsen worker 33 - jepsen.util 33 :invoke :write [11 42] INFO [2022-01-10 12:52:57,811] jepsen worker 88 - jepsen.util 88 :invoke :read [29 nil] INFO [2022-01-10 12:52:57,812] jepsen worker 88 - jepsen.util 88 :ok :read [29 49] INFO [2022-01-10 12:52:57,813] jepsen worker 87 - jepsen.util 87 :ok :write [29 40] INFO [2022-01-10 12:52:57,814] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:57,815] jepsen worker 83 - jepsen.util 83 :ok :read [27 23] INFO [2022-01-10 12:52:57,818] jepsen worker 41 - jepsen.util 41 :invoke :write [13 38] INFO [2022-01-10 12:52:57,821] jepsen worker 53 - jepsen.util 53 :invoke :write [17 8] INFO [2022-01-10 12:52:57,823] jepsen worker 37 - jepsen.util 37 :invoke :write [12 25] INFO [2022-01-10 12:52:57,825] jepsen worker 41 - jepsen.util 41 :ok :write [13 38] INFO [2022-01-10 12:52:57,825] jepsen worker 66 - jepsen.util 66 :invoke :write [22 35] INFO [2022-01-10 12:52:57,827] jepsen worker 69 - jepsen.util 69 :invoke :write [23 48] INFO [2022-01-10 12:52:57,829] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:57,831] jepsen worker 11 - jepsen.util 11 :ok :read [3 19] INFO [2022-01-10 12:52:57,832] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:57,833] jepsen worker 38 - jepsen.util 38 :ok :read [12 23] INFO [2022-01-10 12:52:57,834] jepsen worker 37 - jepsen.util 37 :ok :write [12 25] INFO [2022-01-10 12:52:57,835] jepsen worker 25 - jepsen.util 25 :invoke :read [8 nil] INFO [2022-01-10 12:52:57,836] jepsen worker 25 - jepsen.util 25 :ok :read [8 22] INFO [2022-01-10 12:52:57,837] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:57,838] jepsen worker 31 - jepsen.util 31 :ok :read [10 12] INFO [2022-01-10 12:52:57,839] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:52:57,840] jepsen worker 65 - jepsen.util 65 :ok :read [21 3] INFO [2022-01-10 12:52:57,842] jepsen worker 64 - jepsen.util 64 :invoke :write [21 26] INFO [2022-01-10 12:52:57,845] jepsen worker 37 - jepsen.util 37 :invoke :write [12 9] INFO [2022-01-10 12:52:57,848] jepsen worker 29 - jepsen.util 29 :invoke :write [9 0] INFO [2022-01-10 12:52:57,851] jepsen worker 73 - jepsen.util 73 :invoke :write [24 4] INFO [2022-01-10 12:52:57,855] jepsen worker 17 - jepsen.util 17 :invoke :write [5 24] INFO [2022-01-10 12:52:57,855] jepsen worker 37 - jepsen.util 37 :ok :write [12 9] INFO [2022-01-10 12:52:57,856] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:52:57,857] jepsen worker 29 - jepsen.util 29 :ok :write [9 0] INFO [2022-01-10 12:52:57,858] jepsen worker 75 - jepsen.util 75 :ok :read [25 0] INFO [2022-01-10 12:52:57,859] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:52:57,860] jepsen worker 14 - jepsen.util 14 :ok :read [4 20] INFO [2022-01-10 12:52:57,862] jepsen worker 9 - jepsen.util 9 :invoke :write [3 28] INFO [2022-01-10 12:52:57,865] jepsen worker 51 - jepsen.util 51 :invoke :write [17 3] INFO [2022-01-10 12:52:57,868] jepsen worker 6 - jepsen.util 6 :invoke :write [2 12] INFO [2022-01-10 12:52:57,869] jepsen worker 9 - jepsen.util 9 :ok :write [3 28] INFO [2022-01-10 12:52:57,871] jepsen worker 84 - jepsen.util 84 :invoke :write [28 41] INFO [2022-01-10 12:52:57,873] jepsen worker 8 - jepsen.util 8 :invoke :write [2 25] INFO [2022-01-10 12:52:57,875] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 12:52:57,876] jepsen worker 56 - jepsen.util 56 :ok :read [18 31] INFO [2022-01-10 12:52:57,878] jepsen worker 28 - jepsen.util 28 :invoke :read [9 nil] INFO [2022-01-10 12:52:57,879] jepsen worker 84 - jepsen.util 84 :ok :write [28 41] INFO [2022-01-10 12:52:57,879] jepsen worker 28 - jepsen.util 28 :ok :read [9 0] INFO [2022-01-10 12:52:57,880] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:57,882] jepsen worker 39 - jepsen.util 39 :ok :read [13 38] INFO [2022-01-10 12:52:57,883] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:52:57,884] jepsen worker 14 - jepsen.util 14 :ok :read [4 20] INFO [2022-01-10 12:52:57,885] jepsen worker 45 - jepsen.util 45 :invoke :write [15 21] INFO [2022-01-10 12:52:57,887] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:57,889] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:52:57,890] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:52:57,891] jepsen worker 86 - jepsen.util 86 :ok :read [28 41] INFO [2022-01-10 12:52:57,893] jepsen worker 3 - jepsen.util 3 :invoke :write [1 4] INFO [2022-01-10 12:52:57,896] jepsen worker 77 - jepsen.util 77 :invoke :write [25 35] INFO [2022-01-10 12:52:57,899] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:57,900] jepsen worker 60 - jepsen.util 60 :ok :read [20 35] INFO [2022-01-10 12:52:57,901] jepsen worker 3 - jepsen.util 3 :ok :write [1 4] INFO [2022-01-10 12:52:57,902] jepsen worker 30 - jepsen.util 30 :invoke :write [10 27] INFO [2022-01-10 12:52:57,905] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:52:57,906] jepsen worker 13 - jepsen.util 13 :ok :read [4 20] INFO [2022-01-10 12:52:57,907] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:57,908] jepsen worker 46 - jepsen.util 46 :ok :read [15 27] INFO [2022-01-10 12:52:57,909] jepsen worker 78 - jepsen.util 78 :invoke :write [26 40] INFO [2022-01-10 12:52:57,909] jepsen worker 12 - jepsen.util 12 :ok :write [4 41] INFO [2022-01-10 12:52:57,911] jepsen worker 55 - jepsen.util 55 :invoke :write [18 14] INFO [2022-01-10 12:52:57,912] jepsen worker 30 - jepsen.util 30 :ok :write [10 27] INFO [2022-01-10 12:52:57,914] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 12:52:57,915] jepsen worker 42 - jepsen.util 42 :ok :read [14 31] INFO [2022-01-10 12:52:57,916] jepsen worker 44 - jepsen.util 44 :invoke :write [14 45] INFO [2022-01-10 12:52:57,916] jepsen worker 15 - jepsen.util 15 :ok :write [5 42] INFO [2022-01-10 12:52:57,918] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:52:57,919] jepsen worker 80 - jepsen.util 80 :ok :write [26 32] INFO [2022-01-10 12:52:57,919] jepsen worker 5 - jepsen.util 5 :ok :read [1 4] INFO [2022-01-10 12:52:57,921] jepsen worker 10 - jepsen.util 10 :invoke :write [3 0] INFO [2022-01-10 12:52:57,924] jepsen worker 7 - jepsen.util 7 :ok :write [2 34] INFO [2022-01-10 12:52:57,925] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 12:52:57,925] jepsen worker 52 - jepsen.util 52 :ok :write [17 28] INFO [2022-01-10 12:52:57,927] jepsen worker 56 - jepsen.util 56 :ok :read [18 31] INFO [2022-01-10 12:52:57,929] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:52:57,930] jepsen worker 69 - jepsen.util 69 :ok :write [23 48] INFO [2022-01-10 12:52:57,930] jepsen worker 75 - jepsen.util 75 :ok :read [25 0] INFO [2022-01-10 12:52:57,932] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:52:57,933] jepsen worker 14 - jepsen.util 14 :ok :read [4 41] INFO [2022-01-10 12:52:57,934] jepsen worker 64 - jepsen.util 64 :ok :write [21 26] INFO [2022-01-10 12:52:57,935] jepsen worker 88 - jepsen.util 88 :invoke :write [29 19] INFO [2022-01-10 12:52:57,935] jepsen worker 68 - jepsen.util 68 :ok :write [22 14] INFO [2022-01-10 12:52:57,937] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:57,938] jepsen worker 73 - jepsen.util 73 :ok :write [24 4] INFO [2022-01-10 12:52:57,938] jepsen worker 63 - jepsen.util 63 :ok :read [21 26] INFO [2022-01-10 12:52:57,940] jepsen worker 81 - jepsen.util 81 :invoke :write [27 19] INFO [2022-01-10 12:52:57,942] jepsen worker 20 - jepsen.util 20 :ok :write [6 43] INFO [2022-01-10 12:52:57,943] jepsen worker 54 - jepsen.util 54 :invoke :write [18 40] INFO [2022-01-10 12:52:57,945] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:57,946] jepsen worker 45 - jepsen.util 45 :ok :write [15 21] INFO [2022-01-10 12:52:57,947] jepsen worker 61 - jepsen.util 61 :ok :read [20 35] INFO [2022-01-10 12:52:57,947] jepsen worker 76 - jepsen.util 76 :ok :write [25 39] INFO [2022-01-10 12:52:57,948] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:57,949] jepsen worker 69 - jepsen.util 69 :ok :read [23 48] INFO [2022-01-10 12:52:57,951] jepsen worker 68 - jepsen.util 68 :invoke :write [22 10] INFO [2022-01-10 12:52:57,953] jepsen worker 79 - jepsen.util 79 :invoke :write [26 20] INFO [2022-01-10 12:52:57,955] jepsen worker 33 - jepsen.util 33 :ok :write [11 42] INFO [2022-01-10 12:52:57,955] jepsen worker 78 - jepsen.util 78 :ok :write [26 40] INFO [2022-01-10 12:52:57,956] jepsen worker 7 - jepsen.util 7 :invoke :write [2 37] INFO [2022-01-10 12:52:57,958] jepsen worker 76 - jepsen.util 76 :invoke :write [25 36] INFO [2022-01-10 12:52:57,960] jepsen worker 38 - jepsen.util 38 :invoke :write [12 28] INFO [2022-01-10 12:52:57,962] jepsen worker 21 - jepsen.util 21 :ok :write [7 29] INFO [2022-01-10 12:52:57,963] jepsen worker 13 - jepsen.util 13 :invoke :write [4 18] INFO [2022-01-10 12:52:57,965] jepsen worker 26 - jepsen.util 26 :invoke :write [8 17] INFO [2022-01-10 12:52:57,967] jepsen worker 83 - jepsen.util 83 :invoke :write [27 3] INFO [2022-01-10 12:52:57,968] jepsen worker 44 - jepsen.util 44 :ok :write [14 45] INFO [2022-01-10 12:52:57,970] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:57,971] jepsen worker 45 - jepsen.util 45 :ok :read [15 21] INFO [2022-01-10 12:52:57,972] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:52:57,974] jepsen worker 52 - jepsen.util 52 :ok :read [17 28] INFO [2022-01-10 12:52:57,974] jepsen worker 17 - jepsen.util 17 :ok :write [5 24] INFO [2022-01-10 12:52:57,975] jepsen worker 29 - jepsen.util 29 :invoke :write [9 13] INFO [2022-01-10 12:52:57,978] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:57,979] jepsen worker 48 - jepsen.util 48 :ok :read [16 15] INFO [2022-01-10 12:52:57,980] jepsen worker 6 - jepsen.util 6 :ok :write [2 12] INFO [2022-01-10 12:52:57,981] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:57,984] jepsen worker 82 - jepsen.util 82 :ok :read [27 23] INFO [2022-01-10 12:52:57,986] jepsen worker 89 - jepsen.util 89 :invoke :write [29 4] INFO [2022-01-10 12:52:57,988] jepsen worker 67 - jepsen.util 67 :ok :write [22 9] INFO [2022-01-10 12:52:57,990] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:57,991] jepsen worker 82 - jepsen.util 82 :ok :read [27 23] INFO [2022-01-10 12:52:57,992] jepsen worker 81 - jepsen.util 81 :ok :write [27 19] INFO [2022-01-10 12:52:57,992] jepsen worker 6 - jepsen.util 6 :invoke :write [2 4] INFO [2022-01-10 12:52:57,995] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:57,996] jepsen worker 60 - jepsen.util 60 :ok :read [20 35] INFO [2022-01-10 12:52:57,998] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:57,999] jepsen worker 33 - jepsen.util 33 :ok :read [11 42] INFO [2022-01-10 12:52:58,000] jepsen worker 40 - jepsen.util 40 :invoke :write [13 1] INFO [2022-01-10 12:52:58,003] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:58,004] jepsen worker 33 - jepsen.util 33 :ok :read [11 42] INFO [2022-01-10 12:52:58,006] jepsen worker 74 - jepsen.util 74 :invoke :write [24 38] INFO [2022-01-10 12:52:58,009] jepsen worker 52 - jepsen.util 52 :invoke :write [17 15] INFO [2022-01-10 12:52:58,011] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:52:58,013] jepsen worker 14 - jepsen.util 14 :ok :read [4 41] INFO [2022-01-10 12:52:58,015] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:58,016] jepsen worker 67 - jepsen.util 67 :ok :read [22 9] INFO [2022-01-10 12:52:58,017] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:58,018] jepsen worker 9 - jepsen.util 9 :ok :read [3 28] INFO [2022-01-10 12:52:58,019] jepsen worker 12 - jepsen.util 12 :invoke :write [4 2] INFO [2022-01-10 12:52:58,022] jepsen worker 7 - jepsen.util 7 :ok :write [2 37] INFO [2022-01-10 12:52:58,023] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:58,024] jepsen worker 67 - jepsen.util 67 :ok :read [22 9] INFO [2022-01-10 12:52:58,025] jepsen worker 72 - jepsen.util 72 :invoke :write [24 2] INFO [2022-01-10 12:52:58,027] jepsen worker 68 - jepsen.util 68 :ok :write [22 10] INFO [2022-01-10 12:52:58,028] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:58,029] jepsen worker 46 - jepsen.util 46 :ok :read [15 21] INFO [2022-01-10 12:52:58,031] jepsen worker 83 - jepsen.util 83 :ok :write [27 3] INFO [2022-01-10 12:52:58,031] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:58,034] jepsen worker 82 - jepsen.util 82 :ok :read [27 3] INFO [2022-01-10 12:52:58,034] jepsen worker 22 - jepsen.util 22 :invoke :read [7 nil] INFO [2022-01-10 12:52:58,037] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:58,038] jepsen worker 33 - jepsen.util 33 :ok :read [11 42] INFO [2022-01-10 12:52:58,038] jepsen worker 22 - jepsen.util 22 :ok :read [7 29] INFO [2022-01-10 12:52:58,040] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:52:58,041] jepsen worker 75 - jepsen.util 75 :ok :read [25 39] INFO [2022-01-10 12:52:58,042] jepsen worker 48 - jepsen.util 48 :invoke :write [16 29] INFO [2022-01-10 12:52:58,045] jepsen worker 50 - jepsen.util 50 :invoke :write [16 28] INFO [2022-01-10 12:52:58,047] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:52:58,048] jepsen worker 57 - jepsen.util 57 :ok :read [19 8] INFO [2022-01-10 12:52:58,049] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:52:58,050] jepsen worker 6 - jepsen.util 6 :ok :write [2 4] INFO [2022-01-10 12:52:58,050] jepsen worker 30 - jepsen.util 30 :ok :read [10 27] INFO [2022-01-10 12:52:58,052] jepsen worker 35 - jepsen.util 35 :invoke :write [11 6] INFO [2022-01-10 12:52:58,055] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:58,057] jepsen worker 59 - jepsen.util 59 :ok :read [19 8] INFO [2022-01-10 12:52:58,058] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 12:52:58,059] jepsen worker 48 - jepsen.util 48 :ok :write [16 29] INFO [2022-01-10 12:52:58,059] jepsen worker 56 - jepsen.util 56 :ok :read [18 31] INFO [2022-01-10 12:52:58,060] jepsen worker 61 - jepsen.util 61 :invoke :write [20 41] INFO [2022-01-10 12:52:58,063] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:52:58,064] jepsen worker 57 - jepsen.util 57 :ok :read [19 8] INFO [2022-01-10 12:52:58,066] jepsen worker 81 - jepsen.util 81 :invoke :write [27 45] INFO [2022-01-10 12:52:58,069] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:52:58,070] jepsen worker 14 - jepsen.util 14 :ok :read [4 41] INFO [2022-01-10 12:52:58,071] jepsen worker 24 - jepsen.util 24 :invoke :write [8 32] INFO [2022-01-10 12:52:58,074] jepsen worker 19 - jepsen.util 19 :invoke :write [6 26] INFO [2022-01-10 12:52:58,077] jepsen worker 80 - jepsen.util 80 :invoke :write [26 0] INFO [2022-01-10 12:52:58,080] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:52:58,081] jepsen worker 62 - jepsen.util 62 :ok :read [20 35] INFO [2022-01-10 12:52:58,082] jepsen worker 33 - jepsen.util 33 :invoke :write [11 41] INFO [2022-01-10 12:52:58,084] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:52:58,085] jepsen worker 3 - jepsen.util 3 :ok :read [1 4] INFO [2022-01-10 12:52:58,087] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:58,088] jepsen worker 78 - jepsen.util 78 :ok :read [26 40] INFO [2022-01-10 12:52:58,089] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:58,090] jepsen worker 43 - jepsen.util 43 :ok :read [14 45] INFO [2022-01-10 12:52:58,091] jepsen worker 46 - jepsen.util 46 :invoke :write [15 37] INFO [2022-01-10 12:52:58,094] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:52:58,095] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 12:52:58,096] jepsen worker 51 - jepsen.util 51 :ok :write [17 3] INFO [2022-01-10 12:52:58,096] jepsen worker 55 - jepsen.util 55 :ok :write [18 14] INFO [2022-01-10 12:52:58,096] jepsen worker 44 - jepsen.util 44 :invoke :write [14 7] INFO [2022-01-10 12:52:58,097] jepsen worker 88 - jepsen.util 88 :ok :write [29 19] INFO [2022-01-10 12:52:58,099] jepsen worker 25 - jepsen.util 25 :invoke :write [8 46] INFO [2022-01-10 12:52:58,102] jepsen worker 5 - jepsen.util 5 :invoke :write [1 12] INFO [2022-01-10 12:52:58,102] jepsen worker 77 - jepsen.util 77 :ok :write [25 35] INFO [2022-01-10 12:52:58,103] jepsen worker 29 - jepsen.util 29 :ok :write [9 13] INFO [2022-01-10 12:52:58,103] jepsen worker 10 - jepsen.util 10 :ok :write [3 0] INFO [2022-01-10 12:52:58,105] jepsen worker 48 - jepsen.util 48 :invoke :write [16 6] INFO [2022-01-10 12:52:58,107] jepsen worker 73 - jepsen.util 73 :invoke :write [24 24] INFO [2022-01-10 12:52:58,109] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 12:52:58,110] jepsen worker 74 - jepsen.util 74 :ok :write [24 38] INFO [2022-01-10 12:52:58,111] jepsen worker 56 - jepsen.util 56 :ok :read [18 14] INFO [2022-01-10 12:52:58,112] jepsen worker 41 - jepsen.util 41 :invoke :write [13 40] INFO [2022-01-10 12:52:58,113] jepsen worker 44 - jepsen.util 44 :ok :write [14 7] INFO [2022-01-10 12:52:58,114] jepsen worker 79 - jepsen.util 79 :ok :write [26 20] INFO [2022-01-10 12:52:58,115] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:58,116] jepsen worker 4 - jepsen.util 4 :ok :read [1 4] INFO [2022-01-10 12:52:58,117] jepsen worker 19 - jepsen.util 19 :ok :write [6 26] INFO [2022-01-10 12:52:58,118] jepsen worker 79 - jepsen.util 79 :invoke :read [26 nil] INFO [2022-01-10 12:52:58,119] jepsen worker 40 - jepsen.util 40 :ok :write [13 1] INFO [2022-01-10 12:52:58,119] jepsen worker 79 - jepsen.util 79 :ok :read [26 20] INFO [2022-01-10 12:52:58,120] jepsen worker 28 - jepsen.util 28 :invoke :write [9 44] INFO [2022-01-10 12:52:58,121] jepsen worker 38 - jepsen.util 38 :ok :write [12 28] INFO [2022-01-10 12:52:58,123] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:52:58,123] jepsen worker 46 - jepsen.util 46 :ok :write [15 37] INFO [2022-01-10 12:52:58,124] jepsen worker 86 - jepsen.util 86 :ok :read [28 41] INFO [2022-01-10 12:52:58,125] jepsen worker 61 - jepsen.util 61 :ok :write [20 41] INFO [2022-01-10 12:52:58,125] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:58,126] jepsen worker 38 - jepsen.util 38 :ok :read [12 28] INFO [2022-01-10 12:52:58,127] jepsen worker 86 - jepsen.util 86 :invoke :write [28 5] INFO [2022-01-10 12:52:58,130] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:58,130] jepsen worker 13 - jepsen.util 13 :ok :write [4 18] INFO [2022-01-10 12:52:58,131] jepsen worker 82 - jepsen.util 82 :ok :read [27 3] INFO [2022-01-10 12:52:58,132] jepsen worker 26 - jepsen.util 26 :ok :write [8 17] INFO [2022-01-10 12:52:58,132] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:58,133] jepsen worker 83 - jepsen.util 83 :ok :read [27 3] INFO [2022-01-10 12:52:58,136] jepsen worker 46 - jepsen.util 46 :invoke :write [15 23] INFO [2022-01-10 12:52:58,138] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:58,139] jepsen worker 69 - jepsen.util 69 :ok :read [23 48] INFO [2022-01-10 12:52:58,140] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:58,141] jepsen worker 85 - jepsen.util 85 :ok :read [28 41] INFO [2022-01-10 12:52:58,143] jepsen worker 43 - jepsen.util 43 :invoke :write [14 45] INFO [2022-01-10 12:52:58,144] jepsen worker 73 - jepsen.util 73 :ok :write [24 24] INFO [2022-01-10 12:52:58,145] jepsen worker 68 - jepsen.util 68 :invoke :write [22 37] INFO [2022-01-10 12:52:58,147] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:58,149] jepsen worker 71 - jepsen.util 71 :ok :read [23 48] INFO [2022-01-10 12:52:58,149] jepsen worker 83 - jepsen.util 83 :invoke :write [27 13] INFO [2022-01-10 12:52:58,149] jepsen worker 76 - jepsen.util 76 :ok :write [25 36] INFO [2022-01-10 12:52:58,152] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:58,152] jepsen worker 52 - jepsen.util 52 :ok :write [17 15] INFO [2022-01-10 12:52:58,153] jepsen worker 59 - jepsen.util 59 :ok :read [19 8] INFO [2022-01-10 12:52:58,155] jepsen worker 51 - jepsen.util 51 :invoke :write [17 18] INFO [2022-01-10 12:52:58,155] jepsen worker 80 - jepsen.util 80 :ok :write [26 0] INFO [2022-01-10 12:52:58,155] jepsen worker 5 - jepsen.util 5 :ok :write [1 12] INFO [2022-01-10 12:52:58,159] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:58,160] jepsen worker 35 - jepsen.util 35 :ok :write [11 6] INFO [2022-01-10 12:52:58,160] jepsen worker 46 - jepsen.util 46 :ok :write [15 23] INFO [2022-01-10 12:52:58,160] jepsen worker 58 - jepsen.util 58 :ok :read [19 8] INFO [2022-01-10 12:52:58,162] jepsen worker 76 - jepsen.util 76 :invoke :write [25 40] INFO [2022-01-10 12:52:58,163] jepsen worker 50 - jepsen.util 50 :ok :write [16 28] INFO [2022-01-10 12:52:58,165] jepsen worker 5 - jepsen.util 5 :invoke :write [1 27] INFO [2022-01-10 12:52:58,165] jepsen worker 54 - jepsen.util 54 :ok :write [18 40] INFO [2022-01-10 12:52:58,168] jepsen worker 2 - jepsen.util 2 :invoke :write [0 10] INFO [2022-01-10 12:52:58,170] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:52:58,170] jepsen worker 81 - jepsen.util 81 :ok :write [27 45] INFO [2022-01-10 12:52:58,171] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:52:58,172] jepsen worker 86 - jepsen.util 86 :ok :write [28 5] INFO [2022-01-10 12:52:58,173] jepsen worker 82 - jepsen.util 82 :invoke :write [27 21] INFO [2022-01-10 12:52:58,176] jepsen worker 43 - jepsen.util 43 :ok :write [14 45] INFO [2022-01-10 12:52:58,176] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:58,177] jepsen worker 71 - jepsen.util 71 :ok :read [23 48] INFO [2022-01-10 12:52:58,179] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:58,179] jepsen worker 89 - jepsen.util 89 :ok :write [29 4] INFO [2022-01-10 12:52:58,180] jepsen worker 69 - jepsen.util 69 :ok :read [23 48] INFO [2022-01-10 12:52:58,181] jepsen worker 81 - jepsen.util 81 :invoke :write [27 24] INFO [2022-01-10 12:52:58,184] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:52:58,185] jepsen worker 28 - jepsen.util 28 :ok :write [9 44] INFO [2022-01-10 12:52:58,185] jepsen worker 73 - jepsen.util 73 :ok :read [24 24] INFO [2022-01-10 12:52:58,187] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 12:52:58,188] jepsen worker 42 - jepsen.util 42 :ok :read [14 45] INFO [2022-01-10 12:52:58,189] jepsen worker 41 - jepsen.util 41 :ok :write [13 40] INFO [2022-01-10 12:52:58,189] jepsen worker 5 - jepsen.util 5 :ok :write [1 27] INFO [2022-01-10 12:52:58,191] jepsen worker 67 - jepsen.util 67 :invoke :write [22 17] INFO [2022-01-10 12:52:58,193] jepsen worker 15 - jepsen.util 15 :invoke :write [5 6] INFO [2022-01-10 12:52:58,195] jepsen worker 16 - jepsen.util 16 :invoke :write [5 37] INFO [2022-01-10 12:52:58,197] jepsen worker 23 - jepsen.util 23 :invoke :write [7 40] INFO [2022-01-10 12:52:58,198] jepsen worker 66 - jepsen.util 66 :ok :write [22 35] INFO [2022-01-10 12:52:58,199] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:52:58,201] jepsen worker 54 - jepsen.util 54 :ok :read [18 40] INFO [2022-01-10 12:52:58,202] jepsen worker 17 - jepsen.util 17 :invoke :write [5 16] INFO [2022-01-10 12:52:58,204] jepsen worker 26 - jepsen.util 26 :invoke :write [8 13] INFO [2022-01-10 12:52:58,207] jepsen worker 52 - jepsen.util 52 :invoke :write [17 26] INFO [2022-01-10 12:52:58,209] jepsen worker 13 - jepsen.util 13 :invoke :write [4 42] INFO [2022-01-10 12:52:58,211] jepsen worker 14 - jepsen.util 14 :invoke :write [4 15] INFO [2022-01-10 12:52:58,213] jepsen worker 39 - jepsen.util 39 :invoke :write [13 42] INFO [2022-01-10 12:52:58,216] jepsen worker 63 - jepsen.util 63 :invoke :write [21 45] INFO [2022-01-10 12:52:58,218] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:52:58,219] jepsen worker 2 - jepsen.util 2 :ok :write [0 10] INFO [2022-01-10 12:52:58,219] jepsen worker 21 - jepsen.util 21 :ok :read [7 29] INFO [2022-01-10 12:52:58,222] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:58,223] jepsen worker 50 - jepsen.util 50 :ok :read [16 28] INFO [2022-01-10 12:52:58,225] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:58,226] jepsen worker 27 - jepsen.util 27 :ok :read [9 44] INFO [2022-01-10 12:52:58,227] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:58,228] jepsen worker 43 - jepsen.util 43 :ok :read [14 45] INFO [2022-01-10 12:52:58,230] jepsen worker 41 - jepsen.util 41 :invoke :write [13 21] INFO [2022-01-10 12:52:58,232] jepsen worker 84 - jepsen.util 84 :invoke :write [28 1] INFO [2022-01-10 12:52:58,234] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:58,235] jepsen worker 50 - jepsen.util 50 :ok :read [16 28] INFO [2022-01-10 12:52:58,236] jepsen worker 44 - jepsen.util 44 :invoke :write [14 30] INFO [2022-01-10 12:52:58,239] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:58,240] jepsen worker 2 - jepsen.util 2 :ok :read [0 10] INFO [2022-01-10 12:52:58,241] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:58,241] jepsen worker 84 - jepsen.util 84 :ok :write [28 1] INFO [2022-01-10 12:52:58,242] jepsen worker 50 - jepsen.util 50 :ok :read [16 28] INFO [2022-01-10 12:52:58,243] jepsen worker 61 - jepsen.util 61 :invoke :write [20 5] INFO [2022-01-10 12:52:58,244] jepsen worker 39 - jepsen.util 39 :ok :write [13 42] INFO [2022-01-10 12:52:58,245] jepsen worker 89 - jepsen.util 89 :invoke :write [29 16] INFO [2022-01-10 12:52:58,247] jepsen worker 20 - jepsen.util 20 :invoke :read [6 nil] INFO [2022-01-10 12:52:58,248] jepsen worker 20 - jepsen.util 20 :ok :read [6 26] INFO [2022-01-10 12:52:58,249] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:52:58,250] jepsen worker 40 - jepsen.util 40 :ok :read [13 42] INFO [2022-01-10 12:52:58,251] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 12:52:58,252] jepsen worker 6 - jepsen.util 6 :ok :read [2 4] INFO [2022-01-10 12:52:58,253] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:52:58,254] jepsen worker 19 - jepsen.util 19 :ok :read [6 26] INFO [2022-01-10 12:52:58,255] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:58,256] jepsen worker 43 - jepsen.util 43 :ok :read [14 45] INFO [2022-01-10 12:52:58,257] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:58,258] jepsen worker 85 - jepsen.util 85 :ok :read [28 1] INFO [2022-01-10 12:52:58,260] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:52:58,261] jepsen worker 35 - jepsen.util 35 :ok :read [11 6] INFO [2022-01-10 12:52:58,262] jepsen worker 41 - jepsen.util 41 :ok :write [13 21] INFO [2022-01-10 12:52:58,263] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:52:58,264] jepsen worker 86 - jepsen.util 86 :ok :read [28 1] INFO [2022-01-10 12:52:58,266] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:58,267] jepsen worker 74 - jepsen.util 74 :ok :read [24 24] INFO [2022-01-10 12:52:58,269] jepsen worker 35 - jepsen.util 35 :invoke :write [11 23] INFO [2022-01-10 12:52:58,269] jepsen worker 89 - jepsen.util 89 :ok :write [29 16] INFO [2022-01-10 12:52:58,272] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:58,273] jepsen worker 43 - jepsen.util 43 :ok :read [14 45] INFO [2022-01-10 12:52:58,275] jepsen worker 40 - jepsen.util 40 :invoke :write [13 33] INFO [2022-01-10 12:52:58,278] jepsen worker 7 - jepsen.util 7 :invoke :write [2 49] INFO [2022-01-10 12:52:58,280] jepsen worker 58 - jepsen.util 58 :invoke :write [19 40] INFO [2022-01-10 12:52:58,281] jepsen worker 24 - jepsen.util 24 :ok :write [8 32] INFO [2022-01-10 12:52:58,283] jepsen worker 30 - jepsen.util 30 :invoke :write [10 35] INFO [2022-01-10 12:52:58,284] jepsen worker 40 - jepsen.util 40 :ok :write [13 33] INFO [2022-01-10 12:52:58,285] jepsen worker 33 - jepsen.util 33 :ok :write [11 41] INFO [2022-01-10 12:52:58,286] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:58,287] jepsen worker 50 - jepsen.util 50 :ok :read [16 28] INFO [2022-01-10 12:52:58,288] jepsen worker 12 - jepsen.util 12 :ok :write [4 2] INFO [2022-01-10 12:52:58,289] jepsen worker 21 - jepsen.util 21 :invoke :write [7 47] INFO [2022-01-10 12:52:58,292] jepsen worker 76 - jepsen.util 76 :ok :write [25 40] INFO [2022-01-10 12:52:58,292] jepsen worker 5 - jepsen.util 5 :invoke :write [1 0] INFO [2022-01-10 12:52:58,293] jepsen worker 72 - jepsen.util 72 :ok :write [24 2] INFO [2022-01-10 12:52:58,293] jepsen worker 30 - jepsen.util 30 :ok :write [10 35] INFO [2022-01-10 12:52:58,295] jepsen worker 22 - jepsen.util 22 :invoke :write [7 0] INFO [2022-01-10 12:52:58,298] jepsen worker 51 - jepsen.util 51 :ok :write [17 18] INFO [2022-01-10 12:52:58,298] jepsen worker 48 - jepsen.util 48 :ok :write [16 6] INFO [2022-01-10 12:52:58,299] jepsen worker 79 - jepsen.util 79 :invoke :read [26 nil] INFO [2022-01-10 12:52:58,299] jepsen worker 58 - jepsen.util 58 :ok :write [19 40] INFO [2022-01-10 12:52:58,300] jepsen worker 79 - jepsen.util 79 :ok :read [26 0] INFO [2022-01-10 12:52:58,301] jepsen worker 42 - jepsen.util 42 :invoke :write [14 11] INFO [2022-01-10 12:52:58,304] jepsen worker 54 - jepsen.util 54 :invoke :write [18 40] INFO [2022-01-10 12:52:58,304] jepsen worker 5 - jepsen.util 5 :ok :write [1 0] INFO [2022-01-10 12:52:58,305] jepsen worker 83 - jepsen.util 83 :ok :write [27 13] INFO [2022-01-10 12:52:58,307] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:52:58,309] jepsen worker 73 - jepsen.util 73 :ok :read [24 2] INFO [2022-01-10 12:52:58,310] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:58,311] jepsen worker 41 - jepsen.util 41 :ok :read [13 33] INFO [2022-01-10 12:52:58,312] jepsen worker 44 - jepsen.util 44 :ok :write [14 30] INFO [2022-01-10 12:52:58,312] jepsen worker 54 - jepsen.util 54 :ok :write [18 40] INFO [2022-01-10 12:52:58,314] jepsen worker 31 - jepsen.util 31 :invoke :write [10 14] INFO [2022-01-10 12:52:58,317] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:58,318] jepsen worker 48 - jepsen.util 48 :ok :read [16 6] INFO [2022-01-10 12:52:58,319] jepsen worker 63 - jepsen.util 63 :ok :write [21 45] INFO [2022-01-10 12:52:58,319] jepsen worker 8 - jepsen.util 8 :ok :write [2 25] INFO [2022-01-10 12:52:58,320] jepsen worker 62 - jepsen.util 62 :invoke :write [20 12] INFO [2022-01-10 12:52:58,323] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:58,324] jepsen worker 51 - jepsen.util 51 :ok :read [17 18] INFO [2022-01-10 12:52:58,325] jepsen worker 8 - jepsen.util 8 :invoke :write [2 5] INFO [2022-01-10 12:52:58,327] jepsen worker 15 - jepsen.util 15 :ok :write [5 6] INFO [2022-01-10 12:52:58,327] jepsen worker 26 - jepsen.util 26 :ok :write [8 13] INFO [2022-01-10 12:52:58,328] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 12:52:58,329] jepsen worker 6 - jepsen.util 6 :ok :read [2 25] INFO [2022-01-10 12:52:58,330] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:58,331] jepsen worker 2 - jepsen.util 2 :ok :read [0 10] INFO [2022-01-10 12:52:58,332] jepsen worker 20 - jepsen.util 20 :invoke :read [6 nil] INFO [2022-01-10 12:52:58,333] jepsen worker 20 - jepsen.util 20 :ok :read [6 26] INFO [2022-01-10 12:52:58,334] jepsen worker 23 - jepsen.util 23 :ok :write [7 40] INFO [2022-01-10 12:52:58,335] jepsen worker 46 - jepsen.util 46 :invoke :write [15 40] INFO [2022-01-10 12:52:58,337] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:58,338] jepsen worker 12 - jepsen.util 12 :ok :read [4 2] INFO [2022-01-10 12:52:58,339] jepsen worker 14 - jepsen.util 14 :ok :write [4 15] INFO [2022-01-10 12:52:58,340] jepsen worker 31 - jepsen.util 31 :ok :write [10 14] INFO [2022-01-10 12:52:58,340] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:58,341] jepsen worker 23 - jepsen.util 23 :ok :read [7 40] INFO [2022-01-10 12:52:58,342] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:52:58,343] jepsen worker 67 - jepsen.util 67 :ok :write [22 17] INFO [2022-01-10 12:52:58,344] jepsen worker 15 - jepsen.util 15 :ok :read [5 6] INFO [2022-01-10 12:52:58,344] jepsen worker 44 - jepsen.util 44 :invoke :write [14 6] INFO [2022-01-10 12:52:58,347] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:58,348] jepsen worker 11 - jepsen.util 11 :ok :read [3 0] INFO [2022-01-10 12:52:58,348] jepsen worker 61 - jepsen.util 61 :ok :write [20 5] INFO [2022-01-10 12:52:58,350] jepsen worker 52 - jepsen.util 52 :ok :write [17 26] INFO [2022-01-10 12:52:58,351] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:52:58,352] jepsen worker 14 - jepsen.util 14 :ok :read [4 15] INFO [2022-01-10 12:52:58,353] jepsen worker 1 - jepsen.util 1 :invoke :write [0 34] INFO [2022-01-10 12:52:58,355] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:58,356] jepsen worker 35 - jepsen.util 35 :ok :write [11 23] INFO [2022-01-10 12:52:58,357] jepsen worker 67 - jepsen.util 67 :ok :read [22 17] INFO [2022-01-10 12:52:58,357] jepsen worker 69 - jepsen.util 69 :invoke :write [23 23] INFO [2022-01-10 12:52:58,360] jepsen worker 55 - jepsen.util 55 :invoke :write [18 39] INFO [2022-01-10 12:52:58,363] jepsen worker 83 - jepsen.util 83 :invoke :write [27 10] INFO [2022-01-10 12:52:58,365] jepsen worker 28 - jepsen.util 28 :invoke :write [9 39] INFO [2022-01-10 12:52:58,367] jepsen worker 1 - jepsen.util 1 :ok :write [0 34] INFO [2022-01-10 12:52:58,369] jepsen worker 46 - jepsen.util 46 :ok :write [15 40] INFO [2022-01-10 12:52:58,369] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:58,370] jepsen worker 26 - jepsen.util 26 :ok :read [8 13] INFO [2022-01-10 12:52:58,371] jepsen worker 60 - jepsen.util 60 :invoke :write [20 42] INFO [2022-01-10 12:52:58,374] jepsen worker 67 - jepsen.util 67 :invoke :write [22 31] INFO [2022-01-10 12:52:58,374] jepsen worker 82 - jepsen.util 82 :ok :write [27 21] INFO [2022-01-10 12:52:58,376] jepsen worker 47 - jepsen.util 47 :invoke :write [15 38] INFO [2022-01-10 12:52:58,379] jepsen worker 43 - jepsen.util 43 :invoke :write [14 9] INFO [2022-01-10 12:52:58,379] jepsen worker 42 - jepsen.util 42 :ok :write [14 11] INFO [2022-01-10 12:52:58,381] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:58,382] jepsen worker 74 - jepsen.util 74 :ok :read [24 2] INFO [2022-01-10 12:52:58,383] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:58,384] jepsen worker 50 - jepsen.util 50 :ok :read [16 6] INFO [2022-01-10 12:52:58,385] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:58,387] jepsen worker 12 - jepsen.util 12 :ok :read [4 15] INFO [2022-01-10 12:52:58,387] jepsen worker 69 - jepsen.util 69 :ok :write [23 23] INFO [2022-01-10 12:52:58,387] jepsen worker 7 - jepsen.util 7 :ok :write [2 49] INFO [2022-01-10 12:52:58,388] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:58,389] jepsen worker 82 - jepsen.util 82 :ok :read [27 21] INFO [2022-01-10 12:52:58,391] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:58,392] jepsen worker 74 - jepsen.util 74 :ok :read [24 2] INFO [2022-01-10 12:52:58,393] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:58,394] jepsen worker 33 - jepsen.util 33 :ok :read [11 23] INFO [2022-01-10 12:52:58,396] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:58,396] jepsen worker 16 - jepsen.util 16 :ok :write [5 37] INFO [2022-01-10 12:52:58,397] jepsen worker 61 - jepsen.util 61 :ok :read [20 5] INFO [2022-01-10 12:52:58,399] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:58,400] jepsen worker 51 - jepsen.util 51 :ok :read [17 26] INFO [2022-01-10 12:52:58,401] jepsen worker 21 - jepsen.util 21 :ok :write [7 47] INFO [2022-01-10 12:52:58,402] jepsen worker 15 - jepsen.util 15 :invoke :write [5 6] INFO [2022-01-10 12:52:58,404] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:58,405] jepsen worker 9 - jepsen.util 9 :ok :read [3 0] INFO [2022-01-10 12:52:58,407] jepsen worker 59 - jepsen.util 59 :invoke :write [19 44] INFO [2022-01-10 12:52:58,407] jepsen worker 13 - jepsen.util 13 :ok :write [4 42] INFO [2022-01-10 12:52:58,409] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:58,409] jepsen worker 67 - jepsen.util 67 :ok :write [22 31] INFO [2022-01-10 12:52:58,410] jepsen worker 16 - jepsen.util 16 :ok :read [5 37] INFO [2022-01-10 12:52:58,411] jepsen worker 46 - jepsen.util 46 :invoke :write [15 43] INFO [2022-01-10 12:52:58,413] jepsen worker 53 - jepsen.util 53 :ok :write [17 8] INFO [2022-01-10 12:52:58,414] jepsen worker 11 - jepsen.util 11 :invoke :write [3 3] INFO [2022-01-10 12:52:58,415] jepsen worker 62 - jepsen.util 62 :ok :write [20 12] INFO [2022-01-10 12:52:58,416] jepsen worker 67 - jepsen.util 67 :invoke :write [22 4] INFO [2022-01-10 12:52:58,418] jepsen worker 66 - jepsen.util 66 :invoke :write [22 40] INFO [2022-01-10 12:52:58,420] jepsen worker 56 - jepsen.util 56 :invoke :write [18 3] INFO [2022-01-10 12:52:58,422] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:52:58,423] jepsen worker 34 - jepsen.util 34 :ok :write [11 34] INFO [2022-01-10 12:52:58,424] jepsen worker 19 - jepsen.util 19 :ok :read [6 26] INFO [2022-01-10 12:52:58,425] jepsen worker 45 - jepsen.util 45 :invoke :write [15 1] INFO [2022-01-10 12:52:58,430] jepsen worker 83 - jepsen.util 83 :ok :write [27 10] INFO [2022-01-10 12:52:58,430] jepsen worker 10 - jepsen.util 10 :invoke :write [3 4] INFO [2022-01-10 12:52:58,433] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:58,434] jepsen worker 84 - jepsen.util 84 :ok :read [28 1] INFO [2022-01-10 12:52:58,435] jepsen worker 43 - jepsen.util 43 :ok :write [14 9] INFO [2022-01-10 12:52:58,437] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:58,438] jepsen worker 83 - jepsen.util 83 :ok :read [27 10] INFO [2022-01-10 12:52:58,440] jepsen worker 73 - jepsen.util 73 :invoke :write [24 27] INFO [2022-01-10 12:52:58,442] jepsen worker 8 - jepsen.util 8 :ok :write [2 5] INFO [2022-01-10 12:52:58,444] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:52:58,445] jepsen worker 72 - jepsen.util 72 :ok :read [24 2] INFO [2022-01-10 12:52:58,448] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:52:58,449] jepsen worker 15 - jepsen.util 15 :ok :write [5 6] INFO [2022-01-10 12:52:58,449] jepsen worker 19 - jepsen.util 19 :ok :read [6 26] INFO [2022-01-10 12:52:58,452] jepsen worker 34 - jepsen.util 34 :invoke :write [11 35] INFO [2022-01-10 12:52:58,457] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:58,457] jepsen worker 67 - jepsen.util 67 :ok :write [22 4] INFO [2022-01-10 12:52:58,458] jepsen worker 39 - jepsen.util 39 :ok :read [13 33] INFO [2022-01-10 12:52:58,461] jepsen worker 79 - jepsen.util 79 :invoke :read [26 nil] INFO [2022-01-10 12:52:58,462] jepsen worker 79 - jepsen.util 79 :ok :read [26 0] INFO [2022-01-10 12:52:58,465] jepsen worker 20 - jepsen.util 20 :invoke :read [6 nil] INFO [2022-01-10 12:52:58,466] jepsen worker 20 - jepsen.util 20 :ok :read [6 26] INFO [2022-01-10 12:52:58,467] jepsen worker 34 - jepsen.util 34 :ok :write [11 35] INFO [2022-01-10 12:52:58,468] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:58,470] jepsen worker 50 - jepsen.util 50 :ok :read [16 6] INFO [2022-01-10 12:52:58,472] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:58,473] jepsen worker 77 - jepsen.util 77 :ok :read [25 40] INFO [2022-01-10 12:52:58,476] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:52:58,477] jepsen worker 35 - jepsen.util 35 :ok :read [11 35] INFO [2022-01-10 12:52:58,479] jepsen worker 87 - jepsen.util 87 :invoke :write [29 29] INFO [2022-01-10 12:52:58,482] jepsen worker 26 - jepsen.util 26 :invoke :write [8 11] INFO [2022-01-10 12:52:58,485] jepsen worker 82 - jepsen.util 82 :invoke :write [27 20] INFO [2022-01-10 12:52:58,489] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:58,490] jepsen worker 38 - jepsen.util 38 :ok :read [12 28] INFO [2022-01-10 12:52:58,492] jepsen worker 66 - jepsen.util 66 :ok :write [22 40] INFO [2022-01-10 12:52:58,493] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:58,495] jepsen worker 55 - jepsen.util 55 :ok :write [18 39] INFO [2022-01-10 12:52:58,497] jepsen worker 51 - jepsen.util 51 :invoke :write [17 21] INFO [2022-01-10 12:52:58,499] jepsen worker 47 - jepsen.util 47 :ok :write [15 38] INFO [2022-01-10 12:52:58,499] jepsen worker 28 - jepsen.util 28 :ok :write [9 39] INFO [2022-01-10 12:52:58,500] jepsen worker 78 - jepsen.util 78 :invoke :write [26 30] INFO [2022-01-10 12:52:58,503] jepsen worker 27 - jepsen.util 27 :invoke :write [9 26] INFO [2022-01-10 12:52:58,506] jepsen worker 52 - jepsen.util 52 :invoke :write [17 7] INFO [2022-01-10 12:52:58,507] jepsen worker 25 - jepsen.util 25 :ok :write [8 46] INFO [2022-01-10 12:52:58,508] jepsen worker 67 - jepsen.util 67 :invoke :write [22 32] INFO [2022-01-10 12:52:58,510] jepsen worker 59 - jepsen.util 59 :ok :write [19 44] INFO [2022-01-10 12:52:58,512] jepsen worker 50 - jepsen.util 50 :invoke :write [16 23] INFO [2022-01-10 12:52:58,514] jepsen worker 48 - jepsen.util 48 :ok :read [16 6] INFO [2022-01-10 12:52:58,515] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:52:58,517] jepsen worker 53 - jepsen.util 53 :ok :read [17 8] INFO [2022-01-10 12:52:58,519] jepsen worker 73 - jepsen.util 73 :ok :write [24 27] INFO [2022-01-10 12:52:58,520] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:52:58,521] jepsen worker 57 - jepsen.util 57 :ok :read [19 44] INFO [2022-01-10 12:52:58,524] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:58,525] jepsen worker 31 - jepsen.util 31 :ok :read [10 14] INFO [2022-01-10 12:52:58,525] jepsen worker 51 - jepsen.util 51 :ok :write [17 21] INFO [2022-01-10 12:52:58,526] jepsen worker 22 - jepsen.util 22 :ok :write [7 0] INFO [2022-01-10 12:52:58,527] jepsen worker 83 - jepsen.util 83 :invoke :write [27 15] INFO [2022-01-10 12:52:58,527] jepsen worker 26 - jepsen.util 26 :ok :write [8 11] INFO [2022-01-10 12:52:58,529] jepsen worker 15 - jepsen.util 15 :invoke :write [5 21] INFO [2022-01-10 12:52:58,530] jepsen worker 11 - jepsen.util 11 :ok :write [3 3] INFO [2022-01-10 12:52:58,531] jepsen worker 46 - jepsen.util 46 :ok :write [15 43] INFO [2022-01-10 12:52:58,532] jepsen worker 79 - jepsen.util 79 :invoke :write [26 8] INFO [2022-01-10 12:52:58,532] jepsen worker 60 - jepsen.util 60 :ok :write [20 42] INFO [2022-01-10 12:52:58,534] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:58,536] jepsen worker 26 - jepsen.util 26 :ok :read [8 11] INFO [2022-01-10 12:52:58,537] jepsen worker 20 - jepsen.util 20 :invoke :write [6 33] INFO [2022-01-10 12:52:58,537] jepsen worker 44 - jepsen.util 44 :ok :write [14 6] INFO [2022-01-10 12:52:58,538] jepsen worker 78 - jepsen.util 78 :ok :write [26 30] INFO [2022-01-10 12:52:58,539] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:58,541] jepsen worker 60 - jepsen.util 60 :ok :read [20 42] INFO [2022-01-10 12:52:58,543] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:58,545] jepsen worker 46 - jepsen.util 46 :ok :read [15 43] INFO [2022-01-10 12:52:58,547] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:58,548] jepsen worker 56 - jepsen.util 56 :ok :write [18 3] INFO [2022-01-10 12:52:58,548] jepsen worker 10 - jepsen.util 10 :ok :write [3 4] INFO [2022-01-10 12:52:58,548] jepsen worker 58 - jepsen.util 58 :ok :read [19 44] INFO [2022-01-10 12:52:58,549] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:58,551] jepsen worker 31 - jepsen.util 31 :ok :read [10 14] INFO [2022-01-10 12:52:58,553] jepsen worker 19 - jepsen.util 19 :invoke :write [6 20] INFO [2022-01-10 12:52:58,557] jepsen worker 58 - jepsen.util 58 :invoke :write [19 15] INFO [2022-01-10 12:52:58,560] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:52:58,561] jepsen worker 53 - jepsen.util 53 :ok :read [17 21] INFO [2022-01-10 12:52:58,563] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:58,564] jepsen worker 77 - jepsen.util 77 :ok :read [25 40] INFO [2022-01-10 12:52:58,566] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:58,567] jepsen worker 63 - jepsen.util 63 :ok :read [21 45] INFO [2022-01-10 12:52:58,568] jepsen worker 53 - jepsen.util 53 :invoke :write [17 7] INFO [2022-01-10 12:52:58,571] jepsen worker 78 - jepsen.util 78 :invoke :write [26 41] INFO [2022-01-10 12:52:58,573] jepsen worker 88 - jepsen.util 88 :invoke :write [29 41] INFO [2022-01-10 12:52:58,576] jepsen worker 16 - jepsen.util 16 :invoke :write [5 27] INFO [2022-01-10 12:52:58,578] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:58,579] jepsen worker 61 - jepsen.util 61 :ok :read [20 42] INFO [2022-01-10 12:52:58,581] jepsen worker 18 - jepsen.util 18 :invoke :write [6 18] INFO [2022-01-10 12:52:58,584] jepsen worker 31 - jepsen.util 31 :invoke :write [10 34] INFO [2022-01-10 12:52:58,587] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 12:52:58,588] jepsen worker 6 - jepsen.util 6 :ok :read [2 5] INFO [2022-01-10 12:52:58,590] jepsen worker 51 - jepsen.util 51 :invoke :write [17 7] INFO [2022-01-10 12:52:58,591] jepsen worker 31 - jepsen.util 31 :ok :write [10 34] INFO [2022-01-10 12:52:58,593] jepsen worker 42 - jepsen.util 42 :invoke :write [14 39] INFO [2022-01-10 12:52:58,596] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:52:58,597] jepsen worker 14 - jepsen.util 14 :ok :read [4 42] INFO [2022-01-10 12:52:58,598] jepsen worker 12 - jepsen.util 12 :invoke :write [4 16] INFO [2022-01-10 12:52:58,600] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:58,601] jepsen worker 48 - jepsen.util 48 :ok :read [16 6] INFO [2022-01-10 12:52:58,603] jepsen worker 60 - jepsen.util 60 :invoke :write [20 46] INFO [2022-01-10 12:52:58,606] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:52:58,607] jepsen worker 77 - jepsen.util 77 :ok :read [25 40] INFO [2022-01-10 12:52:58,608] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:52:58,609] jepsen worker 30 - jepsen.util 30 :ok :read [10 34] INFO [2022-01-10 12:52:58,610] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:52:58,611] jepsen worker 70 - jepsen.util 70 :ok :read [23 23] INFO [2022-01-10 12:52:58,612] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:58,613] jepsen worker 71 - jepsen.util 71 :ok :read [23 23] INFO [2022-01-10 12:52:58,615] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:52:58,616] jepsen worker 66 - jepsen.util 66 :ok :read [22 40] INFO [2022-01-10 12:52:58,617] jepsen worker 57 - jepsen.util 57 :invoke :write [19 33] INFO [2022-01-10 12:52:58,620] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:58,622] jepsen worker 11 - jepsen.util 11 :ok :read [3 4] INFO [2022-01-10 12:52:58,622] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:52:58,623] jepsen worker 87 - jepsen.util 87 :ok :write [29 29] INFO [2022-01-10 12:52:58,623] jepsen worker 73 - jepsen.util 73 :ok :read [24 27] INFO [2022-01-10 12:52:58,625] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:52:58,626] jepsen worker 55 - jepsen.util 55 :ok :read [18 3] INFO [2022-01-10 12:52:58,627] jepsen worker 5 - jepsen.util 5 :invoke :write [1 18] INFO [2022-01-10 12:52:58,629] jepsen worker 87 - jepsen.util 87 :invoke :write [29 11] INFO [2022-01-10 12:52:58,632] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:52:58,633] jepsen worker 75 - jepsen.util 75 :ok :read [25 40] INFO [2022-01-10 12:52:58,633] jepsen worker 47 - jepsen.util 47 :invoke :write [15 42] INFO [2022-01-10 12:52:58,636] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:52:58,637] jepsen worker 73 - jepsen.util 73 :ok :read [24 27] INFO [2022-01-10 12:52:58,639] jepsen worker 28 - jepsen.util 28 :invoke :read [9 nil] INFO [2022-01-10 12:52:58,640] jepsen worker 28 - jepsen.util 28 :ok :read [9 39] INFO [2022-01-10 12:52:58,641] jepsen worker 62 - jepsen.util 62 :invoke :write [20 42] INFO [2022-01-10 12:52:58,643] jepsen worker 14 - jepsen.util 14 :invoke :write [4 7] INFO [2022-01-10 12:52:58,646] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:58,647] jepsen worker 74 - jepsen.util 74 :ok :read [24 27] INFO [2022-01-10 12:52:58,648] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:58,649] jepsen worker 11 - jepsen.util 11 :ok :read [3 4] INFO [2022-01-10 12:52:58,650] jepsen worker 2 - jepsen.util 2 :invoke :write [0 39] INFO [2022-01-10 12:52:58,652] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 12:52:58,653] jepsen worker 6 - jepsen.util 6 :ok :read [2 5] INFO [2022-01-10 12:52:58,655] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:52:58,656] jepsen worker 21 - jepsen.util 21 :ok :read [7 0] INFO [2022-01-10 12:52:58,659] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:58,660] jepsen worker 46 - jepsen.util 46 :ok :read [15 43] INFO [2022-01-10 12:52:58,662] jepsen worker 4 - jepsen.util 4 :invoke :write [1 49] INFO [2022-01-10 12:52:58,664] jepsen worker 38 - jepsen.util 38 :invoke :write [12 26] INFO [2022-01-10 12:52:58,667] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:58,668] jepsen worker 59 - jepsen.util 59 :ok :read [19 44] INFO [2022-01-10 12:52:58,669] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:52:58,670] jepsen worker 13 - jepsen.util 13 :ok :read [4 42] INFO [2022-01-10 12:52:58,671] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:52:58,672] jepsen worker 8 - jepsen.util 8 :ok :read [2 5] INFO [2022-01-10 12:52:58,673] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:58,674] jepsen worker 71 - jepsen.util 71 :ok :read [23 23] INFO [2022-01-10 12:52:58,675] jepsen worker 32 - jepsen.util 32 :invoke :read [10 nil] INFO [2022-01-10 12:52:58,677] jepsen worker 32 - jepsen.util 32 :ok :read [10 34] INFO [2022-01-10 12:52:58,678] jepsen worker 33 - jepsen.util 33 :invoke :write [11 8] INFO [2022-01-10 12:52:58,680] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:52:58,681] jepsen worker 35 - jepsen.util 35 :ok :read [11 35] INFO [2022-01-10 12:52:58,683] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:58,684] jepsen worker 48 - jepsen.util 48 :ok :read [16 6] INFO [2022-01-10 12:52:58,685] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:58,686] jepsen worker 0 - jepsen.util 0 :ok :read [0 34] INFO [2022-01-10 12:52:58,688] jepsen worker 43 - jepsen.util 43 :invoke :write [14 38] INFO [2022-01-10 12:52:58,690] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:58,691] jepsen worker 59 - jepsen.util 59 :ok :read [19 44] INFO [2022-01-10 12:52:58,692] jepsen worker 44 - jepsen.util 44 :invoke :write [14 10] INFO [2022-01-10 12:52:58,695] jepsen worker 80 - jepsen.util 80 :invoke :write [26 48] INFO [2022-01-10 12:52:58,697] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:52:58,698] jepsen worker 1 - jepsen.util 1 :ok :read [0 34] INFO [2022-01-10 12:52:58,700] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:58,701] jepsen worker 84 - jepsen.util 84 :ok :read [28 1] INFO [2022-01-10 12:52:58,702] jepsen worker 85 - jepsen.util 85 :invoke :write [28 6] INFO [2022-01-10 12:52:58,705] jepsen worker 13 - jepsen.util 13 :invoke :write [4 20] INFO [2022-01-10 12:52:58,707] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:52:58,708] jepsen worker 24 - jepsen.util 24 :ok :read [8 11] INFO [2022-01-10 12:52:58,708] jepsen worker 45 - jepsen.util 45 :ok :write [15 1] INFO [2022-01-10 12:52:58,709] jepsen worker 42 - jepsen.util 42 :ok :write [14 39] INFO [2022-01-10 12:52:58,710] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:52:58,711] jepsen worker 8 - jepsen.util 8 :ok :read [2 5] INFO [2022-01-10 12:52:58,712] jepsen worker 20 - jepsen.util 20 :ok :write [6 33] INFO [2022-01-10 12:52:58,714] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:52:58,715] jepsen worker 37 - jepsen.util 37 :ok :read [12 28] INFO [2022-01-10 12:52:58,717] jepsen worker 79 - jepsen.util 79 :ok :write [26 8] INFO [2022-01-10 12:52:58,717] jepsen worker 60 - jepsen.util 60 :ok :write [20 46] INFO [2022-01-10 12:52:58,718] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:58,719] jepsen worker 48 - jepsen.util 48 :ok :read [16 6] INFO [2022-01-10 12:52:58,721] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:52:58,722] jepsen worker 24 - jepsen.util 24 :ok :read [8 11] INFO [2022-01-10 12:52:58,723] jepsen worker 53 - jepsen.util 53 :ok :write [17 7] INFO [2022-01-10 12:52:58,724] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:58,725] jepsen worker 74 - jepsen.util 74 :ok :read [24 27] INFO [2022-01-10 12:52:58,727] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 12:52:58,728] jepsen worker 42 - jepsen.util 42 :ok :read [14 39] INFO [2022-01-10 12:52:58,728] jepsen worker 44 - jepsen.util 44 :ok :write [14 10] INFO [2022-01-10 12:52:58,728] jepsen worker 12 - jepsen.util 12 :ok :write [4 16] INFO [2022-01-10 12:52:58,730] jepsen worker 60 - jepsen.util 60 :invoke :write [20 1] INFO [2022-01-10 12:52:58,734] jepsen worker 10 - jepsen.util 10 :invoke :write [3 10] INFO [2022-01-10 12:52:58,738] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:52:58,740] jepsen worker 76 - jepsen.util 76 :ok :read [25 40] INFO [2022-01-10 12:52:58,743] jepsen worker 79 - jepsen.util 79 :invoke :read [26 nil] INFO [2022-01-10 12:52:58,744] jepsen worker 79 - jepsen.util 79 :ok :read [26 8] INFO [2022-01-10 12:52:58,745] jepsen worker 80 - jepsen.util 80 :ok :write [26 48] INFO [2022-01-10 12:52:58,747] jepsen worker 35 - jepsen.util 35 :invoke :write [11 11] INFO [2022-01-10 12:52:58,750] jepsen worker 3 - jepsen.util 3 :invoke :write [1 24] INFO [2022-01-10 12:52:58,750] jepsen worker 60 - jepsen.util 60 :ok :write [20 1] INFO [2022-01-10 12:52:58,752] jepsen worker 52 - jepsen.util 52 :ok :write [17 7] INFO [2022-01-10 12:52:58,753] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:58,754] jepsen worker 84 - jepsen.util 84 :ok :read [28 1] INFO [2022-01-10 12:52:58,755] jepsen worker 43 - jepsen.util 43 :ok :write [14 38] INFO [2022-01-10 12:52:58,756] jepsen worker 25 - jepsen.util 25 :invoke :write [8 47] INFO [2022-01-10 12:52:58,756] jepsen worker 10 - jepsen.util 10 :ok :write [3 10] INFO [2022-01-10 12:52:58,758] jepsen worker 13 - jepsen.util 13 :ok :write [4 20] INFO [2022-01-10 12:52:58,760] jepsen worker 20 - jepsen.util 20 :invoke :write [6 28] INFO [2022-01-10 12:52:58,763] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:58,763] jepsen worker 62 - jepsen.util 62 :ok :write [20 42] INFO [2022-01-10 12:52:58,764] jepsen worker 19 - jepsen.util 19 :ok :write [6 20] INFO [2022-01-10 12:52:58,764] jepsen worker 11 - jepsen.util 11 :ok :read [3 10] INFO [2022-01-10 12:52:58,766] jepsen worker 7 - jepsen.util 7 :invoke :write [2 4] INFO [2022-01-10 12:52:58,769] jepsen worker 56 - jepsen.util 56 :invoke :write [18 12] INFO [2022-01-10 12:52:58,771] jepsen worker 59 - jepsen.util 59 :invoke :write [19 0] INFO [2022-01-10 12:52:58,771] jepsen worker 47 - jepsen.util 47 :ok :write [15 42] INFO [2022-01-10 12:52:58,774] jepsen worker 82 - jepsen.util 82 :ok :write [27 20] INFO [2022-01-10 12:52:58,774] jepsen worker 58 - jepsen.util 58 :ok :write [19 15] INFO [2022-01-10 12:52:58,775] jepsen worker 19 - jepsen.util 19 :invoke :write [6 45] INFO [2022-01-10 12:52:58,777] jepsen worker 78 - jepsen.util 78 :ok :write [26 41] INFO [2022-01-10 12:52:58,778] jepsen worker 54 - jepsen.util 54 :invoke :write [18 16] INFO [2022-01-10 12:52:58,780] jepsen worker 31 - jepsen.util 31 :invoke :write [10 6] INFO [2022-01-10 12:52:58,782] jepsen worker 5 - jepsen.util 5 :ok :write [1 18] INFO [2022-01-10 12:52:58,783] jepsen worker 67 - jepsen.util 67 :ok :write [22 32] INFO [2022-01-10 12:52:58,783] jepsen worker 56 - jepsen.util 56 :ok :write [18 12] INFO [2022-01-10 12:52:58,783] jepsen worker 22 - jepsen.util 22 :invoke :write [7 30] INFO [2022-01-10 12:52:58,787] jepsen worker 38 - jepsen.util 38 :ok :write [12 26] INFO [2022-01-10 12:52:58,790] jepsen worker 25 - jepsen.util 25 :ok :write [8 47] INFO [2022-01-10 12:52:58,791] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:58,791] jepsen worker 27 - jepsen.util 27 :ok :write [9 26] INFO [2022-01-10 12:52:58,793] jepsen worker 85 - jepsen.util 85 :ok :write [28 6] INFO [2022-01-10 12:52:58,795] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:58,796] jepsen worker 48 - jepsen.util 48 :ok :read [16 6] INFO [2022-01-10 12:52:58,798] jepsen worker 50 - jepsen.util 50 :ok :write [16 23] INFO [2022-01-10 12:52:58,799] jepsen worker 89 - jepsen.util 89 :invoke :write [29 5] INFO [2022-01-10 12:52:58,801] jepsen worker 59 - jepsen.util 59 :ok :write [19 0] INFO [2022-01-10 12:52:58,802] jepsen worker 20 - jepsen.util 20 :ok :write [6 28] INFO [2022-01-10 12:52:58,803] jepsen worker 54 - jepsen.util 54 :ok :write [18 16] INFO [2022-01-10 12:52:58,803] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:52:58,804] jepsen worker 22 - jepsen.util 22 :ok :write [7 30] INFO [2022-01-10 12:52:58,804] jepsen worker 76 - jepsen.util 76 :ok :read [25 40] INFO [2022-01-10 12:52:58,806] jepsen worker 17 - jepsen.util 17 :ok :write [5 16] INFO [2022-01-10 12:52:58,808] jepsen worker 3 - jepsen.util 3 :ok :write [1 24] INFO [2022-01-10 12:52:58,809] jepsen worker 65 - jepsen.util 65 :invoke :write [21 5] INFO [2022-01-10 12:52:58,809] jepsen worker 78 - jepsen.util 78 :ok :read [26 41] INFO [2022-01-10 12:52:58,809] jepsen worker 31 - jepsen.util 31 :ok :write [10 6] INFO [2022-01-10 12:52:58,813] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:52:58,814] jepsen worker 87 - jepsen.util 87 :ok :write [29 11] INFO [2022-01-10 12:52:58,814] jepsen worker 59 - jepsen.util 59 :ok :read [19 0] INFO [2022-01-10 12:52:58,816] jepsen worker 51 - jepsen.util 51 :ok :write [17 7] INFO [2022-01-10 12:52:58,818] jepsen worker 17 - jepsen.util 17 :invoke :write [5 29] INFO [2022-01-10 12:52:58,820] jepsen worker 2 - jepsen.util 2 :ok :write [0 39] INFO [2022-01-10 12:52:58,824] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:58,825] jepsen worker 63 - jepsen.util 63 :ok :read [21 45] INFO [2022-01-10 12:52:58,829] jepsen worker 33 - jepsen.util 33 :ok :write [11 8] INFO [2022-01-10 12:52:58,829] jepsen worker 6 - jepsen.util 6 :invoke :write [2 41] INFO [2022-01-10 12:52:58,832] jepsen worker 20 - jepsen.util 20 :invoke :read [6 nil] INFO [2022-01-10 12:52:58,833] jepsen worker 19 - jepsen.util 19 :ok :write [6 45] INFO [2022-01-10 12:52:58,833] jepsen worker 7 - jepsen.util 7 :ok :write [2 4] INFO [2022-01-10 12:52:58,834] jepsen worker 20 - jepsen.util 20 :ok :read [6 45] INFO [2022-01-10 12:52:58,837] jepsen worker 59 - jepsen.util 59 :invoke :write [19 27] INFO [2022-01-10 12:52:58,841] jepsen worker 65 - jepsen.util 65 :ok :write [21 5] INFO [2022-01-10 12:52:58,842] jepsen worker 60 - jepsen.util 60 :invoke :write [20 46] INFO [2022-01-10 12:52:58,846] jepsen worker 4 - jepsen.util 4 :ok :write [1 49] INFO [2022-01-10 12:52:58,847] jepsen worker 84 - jepsen.util 84 :invoke :write [28 31] INFO [2022-01-10 12:52:58,852] jepsen worker 59 - jepsen.util 59 :ok :write [19 27] INFO [2022-01-10 12:52:58,852] jepsen worker 14 - jepsen.util 14 :ok :write [4 7] INFO [2022-01-10 12:52:58,852] jepsen worker 52 - jepsen.util 52 :invoke :write [17 6] INFO [2022-01-10 12:52:58,857] jepsen worker 19 - jepsen.util 19 :invoke :write [6 21] INFO [2022-01-10 12:52:58,861] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:58,862] jepsen worker 17 - jepsen.util 17 :ok :write [5 29] INFO [2022-01-10 12:52:58,863] jepsen worker 82 - jepsen.util 82 :ok :read [27 20] INFO [2022-01-10 12:52:58,863] jepsen worker 60 - jepsen.util 60 :ok :write [20 46] INFO [2022-01-10 12:52:58,864] jepsen worker 84 - jepsen.util 84 :ok :write [28 31] INFO [2022-01-10 12:52:58,866] jepsen worker 54 - jepsen.util 54 :invoke :write [18 45] INFO [2022-01-10 12:52:58,869] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:58,869] jepsen worker 89 - jepsen.util 89 :ok :write [29 5] INFO [2022-01-10 12:52:58,871] jepsen worker 85 - jepsen.util 85 :ok :read [28 31] INFO [2022-01-10 12:52:58,873] jepsen worker 20 - jepsen.util 20 :invoke :write [6 37] INFO [2022-01-10 12:52:58,874] jepsen worker 52 - jepsen.util 52 :ok :write [17 6] INFO [2022-01-10 12:52:58,877] jepsen worker 82 - jepsen.util 82 :invoke :write [27 31] INFO [2022-01-10 12:52:58,878] jepsen worker 35 - jepsen.util 35 :ok :write [11 11] INFO [2022-01-10 12:52:58,879] jepsen worker 19 - jepsen.util 19 :ok :write [6 21] INFO [2022-01-10 12:52:58,881] jepsen worker 83 - jepsen.util 83 :ok :write [27 15] INFO [2022-01-10 12:52:58,881] jepsen worker 22 - jepsen.util 22 :invoke :read [7 nil] INFO [2022-01-10 12:52:58,882] jepsen worker 22 - jepsen.util 22 :ok :read [7 30] INFO [2022-01-10 12:52:58,884] jepsen worker 84 - jepsen.util 84 :invoke :write [28 21] INFO [2022-01-10 12:52:58,888] jepsen worker 64 - jepsen.util 64 :invoke :write [21 25] INFO [2022-01-10 12:52:58,888] jepsen worker 6 - jepsen.util 6 :ok :write [2 41] INFO [2022-01-10 12:52:58,891] jepsen worker 74 - jepsen.util 74 :invoke :write [24 24] INFO [2022-01-10 12:52:58,893] jepsen worker 37 - jepsen.util 37 :invoke :write [12 15] INFO [2022-01-10 12:52:58,896] jepsen worker 16 - jepsen.util 16 :ok :write [5 27] INFO [2022-01-10 12:52:58,896] jepsen worker 20 - jepsen.util 20 :ok :write [6 37] INFO [2022-01-10 12:52:58,896] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:52:58,898] jepsen worker 19 - jepsen.util 19 :ok :read [6 37] INFO [2022-01-10 12:52:58,899] jepsen worker 35 - jepsen.util 35 :invoke :write [11 15] INFO [2022-01-10 12:52:58,901] jepsen worker 64 - jepsen.util 64 :ok :write [21 25] INFO [2022-01-10 12:52:58,902] jepsen worker 2 - jepsen.util 2 :invoke :write [0 26] INFO [2022-01-10 12:52:58,905] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:58,906] jepsen worker 31 - jepsen.util 31 :ok :read [10 6] INFO [2022-01-10 12:52:58,907] jepsen worker 74 - jepsen.util 74 :ok :write [24 24] INFO [2022-01-10 12:52:58,907] jepsen worker 82 - jepsen.util 82 :ok :write [27 31] INFO [2022-01-10 12:52:58,907] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:58,908] jepsen worker 41 - jepsen.util 41 :ok :read [13 33] INFO [2022-01-10 12:52:58,910] jepsen worker 24 - jepsen.util 24 :invoke :write [8 36] INFO [2022-01-10 12:52:58,912] jepsen worker 18 - jepsen.util 18 :ok :write [6 18] INFO [2022-01-10 12:52:58,913] jepsen worker 59 - jepsen.util 59 :invoke :write [19 11] INFO [2022-01-10 12:52:58,915] jepsen worker 71 - jepsen.util 71 :invoke :write [23 24] INFO [2022-01-10 12:52:58,917] jepsen worker 35 - jepsen.util 35 :ok :write [11 15] INFO [2022-01-10 12:52:58,918] jepsen worker 65 - jepsen.util 65 :invoke :write [21 37] INFO [2022-01-10 12:52:58,921] jepsen worker 48 - jepsen.util 48 :invoke :write [16 32] INFO [2022-01-10 12:52:58,923] jepsen worker 58 - jepsen.util 58 :invoke :write [19 6] INFO [2022-01-10 12:52:58,926] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:58,927] jepsen worker 24 - jepsen.util 24 :ok :write [8 36] INFO [2022-01-10 12:52:58,927] jepsen worker 59 - jepsen.util 59 :ok :write [19 11] INFO [2022-01-10 12:52:58,927] jepsen worker 69 - jepsen.util 69 :ok :read [23 23] INFO [2022-01-10 12:52:58,929] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:52:58,930] jepsen worker 55 - jepsen.util 55 :ok :read [18 16] INFO [2022-01-10 12:52:58,931] jepsen worker 22 - jepsen.util 22 :invoke :write [7 22] INFO [2022-01-10 12:52:58,934] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:58,934] jepsen worker 48 - jepsen.util 48 :ok :write [16 32] INFO [2022-01-10 12:52:58,935] jepsen worker 36 - jepsen.util 36 :ok :read [12 26] INFO [2022-01-10 12:52:58,936] jepsen worker 9 - jepsen.util 9 :invoke :write [3 37] INFO [2022-01-10 12:52:58,939] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:58,940] jepsen worker 82 - jepsen.util 82 :ok :read [27 31] INFO [2022-01-10 12:52:58,941] jepsen worker 58 - jepsen.util 58 :ok :write [19 6] INFO [2022-01-10 12:52:58,941] jepsen worker 10 - jepsen.util 10 :invoke :write [3 47] INFO [2022-01-10 12:52:58,944] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:58,945] jepsen worker 44 - jepsen.util 44 :ok :read [14 38] INFO [2022-01-10 12:52:58,946] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:58,947] jepsen worker 65 - jepsen.util 65 :ok :write [21 37] INFO [2022-01-10 12:52:58,947] jepsen worker 23 - jepsen.util 23 :ok :read [7 30] INFO [2022-01-10 12:52:58,949] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:52:58,950] jepsen worker 55 - jepsen.util 55 :ok :read [18 16] INFO [2022-01-10 12:52:58,951] jepsen worker 89 - jepsen.util 89 :invoke :write [29 37] INFO [2022-01-10 12:52:58,953] jepsen worker 60 - jepsen.util 60 :invoke :write [20 37] INFO [2022-01-10 12:52:58,956] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 12:52:58,957] jepsen worker 56 - jepsen.util 56 :ok :read [18 16] INFO [2022-01-10 12:52:58,958] jepsen worker 27 - jepsen.util 27 :invoke :write [9 45] INFO [2022-01-10 12:52:58,960] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 12:52:58,961] jepsen worker 80 - jepsen.util 80 :ok :read [26 41] INFO [2022-01-10 12:52:58,963] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:52:58,964] jepsen worker 13 - jepsen.util 13 :ok :read [4 7] INFO [2022-01-10 12:52:58,965] jepsen worker 27 - jepsen.util 27 :ok :write [9 45] INFO [2022-01-10 12:52:58,966] jepsen worker 55 - jepsen.util 55 :invoke :write [18 12] INFO [2022-01-10 12:52:58,969] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:58,970] jepsen worker 51 - jepsen.util 51 :ok :read [17 6] INFO [2022-01-10 12:52:58,972] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:58,973] jepsen worker 78 - jepsen.util 78 :ok :read [26 41] INFO [2022-01-10 12:52:58,975] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:58,976] jepsen worker 43 - jepsen.util 43 :ok :read [14 38] INFO [2022-01-10 12:52:58,978] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:58,979] jepsen worker 12 - jepsen.util 12 :ok :read [4 7] INFO [2022-01-10 12:52:58,981] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:52:58,981] jepsen worker 54 - jepsen.util 54 :ok :write [18 45] INFO [2022-01-10 12:52:58,982] jepsen worker 87 - jepsen.util 87 :ok :read [29 5] INFO [2022-01-10 12:52:58,984] jepsen worker 11 - jepsen.util 11 :invoke :write [3 47] INFO [2022-01-10 12:52:58,987] jepsen worker 84 - jepsen.util 84 :ok :write [28 21] INFO [2022-01-10 12:52:58,988] jepsen worker 88 - jepsen.util 88 :ok :write [29 41] INFO [2022-01-10 12:52:58,989] jepsen worker 77 - jepsen.util 77 :invoke :write [25 3] INFO [2022-01-10 12:52:58,991] jepsen worker 37 - jepsen.util 37 :ok :write [12 15] INFO [2022-01-10 12:52:58,991] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:58,993] jepsen worker 83 - jepsen.util 83 :ok :read [27 31] INFO [2022-01-10 12:52:58,994] jepsen worker 2 - jepsen.util 2 :ok :write [0 26] INFO [2022-01-10 12:52:58,995] jepsen worker 67 - jepsen.util 67 :invoke :write [22 2] INFO [2022-01-10 12:52:58,997] jepsen worker 79 - jepsen.util 79 :invoke :write [26 6] INFO [2022-01-10 12:52:58,998] jepsen worker 71 - jepsen.util 71 :ok :write [23 24] INFO [2022-01-10 12:52:58,998] jepsen worker 22 - jepsen.util 22 :ok :write [7 22] INFO [2022-01-10 12:52:59,000] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:59,001] jepsen worker 83 - jepsen.util 83 :ok :read [27 31] INFO [2022-01-10 12:52:59,003] jepsen worker 9 - jepsen.util 9 :ok :write [3 37] INFO [2022-01-10 12:52:59,003] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:59,004] jepsen worker 82 - jepsen.util 82 :ok :read [27 31] INFO [2022-01-10 12:52:59,006] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:52:59,007] jepsen worker 50 - jepsen.util 50 :ok :read [16 32] INFO [2022-01-10 12:52:59,008] jepsen worker 71 - jepsen.util 71 :invoke :write [23 2] INFO [2022-01-10 12:52:59,011] jepsen worker 83 - jepsen.util 83 :invoke :write [27 23] INFO [2022-01-10 12:52:59,014] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 12:52:59,014] jepsen worker 89 - jepsen.util 89 :ok :write [29 37] INFO [2022-01-10 12:52:59,015] jepsen worker 56 - jepsen.util 56 :ok :read [18 45] INFO [2022-01-10 12:52:59,016] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:59,017] jepsen worker 61 - jepsen.util 61 :ok :read [20 46] INFO [2022-01-10 12:52:59,019] jepsen worker 51 - jepsen.util 51 :invoke :write [17 18] INFO [2022-01-10 12:52:59,022] jepsen worker 72 - jepsen.util 72 :invoke :write [24 3] INFO [2022-01-10 12:52:59,024] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:59,025] jepsen worker 82 - jepsen.util 82 :ok :read [27 31] INFO [2022-01-10 12:52:59,027] jepsen worker 42 - jepsen.util 42 :invoke :write [14 23] INFO [2022-01-10 12:52:59,029] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:52:59,030] jepsen worker 13 - jepsen.util 13 :ok :read [4 7] INFO [2022-01-10 12:52:59,032] jepsen worker 32 - jepsen.util 32 :invoke :read [10 nil] INFO [2022-01-10 12:52:59,033] jepsen worker 32 - jepsen.util 32 :ok :read [10 6] INFO [2022-01-10 12:52:59,035] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:52:59,036] jepsen worker 8 - jepsen.util 8 :ok :read [2 41] INFO [2022-01-10 12:52:59,038] jepsen worker 28 - jepsen.util 28 :invoke :read [9 nil] INFO [2022-01-10 12:52:59,038] jepsen worker 28 - jepsen.util 28 :ok :read [9 45] INFO [2022-01-10 12:52:59,040] jepsen worker 53 - jepsen.util 53 :invoke :write [17 1] INFO [2022-01-10 12:52:59,042] jepsen worker 5 - jepsen.util 5 :invoke :write [1 22] INFO [2022-01-10 12:52:59,045] jepsen worker 45 - jepsen.util 45 :invoke :write [15 25] INFO [2022-01-10 12:52:59,048] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:52:59,049] jepsen worker 40 - jepsen.util 40 :ok :read [13 33] INFO [2022-01-10 12:52:59,051] jepsen worker 5 - jepsen.util 5 :ok :write [1 22] INFO [2022-01-10 12:52:59,051] jepsen worker 9 - jepsen.util 9 :invoke :write [3 9] INFO [2022-01-10 12:52:59,053] jepsen worker 31 - jepsen.util 31 :invoke :write [10 17] INFO [2022-01-10 12:52:59,056] jepsen worker 35 - jepsen.util 35 :invoke :write [11 44] INFO [2022-01-10 12:52:59,059] jepsen worker 59 - jepsen.util 59 :invoke :write [19 8] INFO [2022-01-10 12:52:59,060] jepsen worker 60 - jepsen.util 60 :ok :write [20 37] INFO [2022-01-10 12:52:59,061] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:59,062] jepsen worker 71 - jepsen.util 71 :ok :write [23 2] INFO [2022-01-10 12:52:59,063] jepsen worker 61 - jepsen.util 61 :ok :read [20 37] INFO [2022-01-10 12:52:59,064] jepsen worker 32 - jepsen.util 32 :invoke :write [10 2] INFO [2022-01-10 12:52:59,066] jepsen worker 56 - jepsen.util 56 :invoke :write [18 21] INFO [2022-01-10 12:52:59,067] jepsen worker 77 - jepsen.util 77 :ok :write [25 3] INFO [2022-01-10 12:52:59,068] jepsen worker 79 - jepsen.util 79 :ok :write [26 6] INFO [2022-01-10 12:52:59,070] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:52:59,071] jepsen worker 66 - jepsen.util 66 :ok :read [22 32] INFO [2022-01-10 12:52:59,072] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:59,073] jepsen worker 4 - jepsen.util 4 :ok :read [1 22] INFO [2022-01-10 12:52:59,074] jepsen worker 51 - jepsen.util 51 :ok :write [17 18] INFO [2022-01-10 12:52:59,075] jepsen worker 17 - jepsen.util 17 :invoke :write [5 12] INFO [2022-01-10 12:52:59,078] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:52:59,080] jepsen worker 34 - jepsen.util 34 :ok :read [11 15] INFO [2022-01-10 12:52:59,080] jepsen worker 67 - jepsen.util 67 :ok :write [22 2] INFO [2022-01-10 12:52:59,080] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:52:59,081] jepsen worker 47 - jepsen.util 47 :ok :read [15 42] INFO [2022-01-10 12:52:59,084] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:59,085] jepsen worker 82 - jepsen.util 82 :ok :read [27 31] INFO [2022-01-10 12:52:59,086] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:52:59,087] jepsen worker 0 - jepsen.util 0 :ok :read [0 26] INFO [2022-01-10 12:52:59,088] jepsen worker 83 - jepsen.util 83 :ok :write [27 23] INFO [2022-01-10 12:52:59,089] jepsen worker 69 - jepsen.util 69 :invoke :write [23 31] INFO [2022-01-10 12:52:59,092] jepsen worker 40 - jepsen.util 40 :invoke :write [13 47] INFO [2022-01-10 12:52:59,094] jepsen worker 42 - jepsen.util 42 :ok :write [14 23] INFO [2022-01-10 12:52:59,095] jepsen worker 47 - jepsen.util 47 :invoke :write [15 35] INFO [2022-01-10 12:52:59,098] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:59,099] jepsen worker 39 - jepsen.util 39 :ok :read [13 33] INFO [2022-01-10 12:52:59,099] jepsen worker 53 - jepsen.util 53 :ok :write [17 1] INFO [2022-01-10 12:52:59,100] jepsen worker 35 - jepsen.util 35 :ok :write [11 44] INFO [2022-01-10 12:52:59,100] jepsen worker 28 - jepsen.util 28 :invoke :read [9 nil] INFO [2022-01-10 12:52:59,101] jepsen worker 28 - jepsen.util 28 :ok :read [9 45] INFO [2022-01-10 12:52:59,103] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:59,104] jepsen worker 44 - jepsen.util 44 :ok :read [14 23] INFO [2022-01-10 12:52:59,105] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:52:59,106] jepsen worker 69 - jepsen.util 69 :ok :write [23 31] INFO [2022-01-10 12:52:59,106] jepsen worker 53 - jepsen.util 53 :ok :read [17 1] INFO [2022-01-10 12:52:59,108] jepsen worker 58 - jepsen.util 58 :invoke :write [19 46] INFO [2022-01-10 12:52:59,110] jepsen worker 86 - jepsen.util 86 :invoke :write [28 20] INFO [2022-01-10 12:52:59,113] jepsen worker 69 - jepsen.util 69 :invoke :write [23 8] INFO [2022-01-10 12:52:59,115] jepsen worker 54 - jepsen.util 54 :invoke :write [18 37] INFO [2022-01-10 12:52:59,118] jepsen worker 17 - jepsen.util 17 :ok :write [5 12] INFO [2022-01-10 12:52:59,118] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:59,119] jepsen worker 16 - jepsen.util 16 :ok :read [5 12] INFO [2022-01-10 12:52:59,121] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:52:59,122] jepsen worker 13 - jepsen.util 13 :ok :read [4 7] INFO [2022-01-10 12:52:59,122] jepsen worker 69 - jepsen.util 69 :ok :write [23 8] INFO [2022-01-10 12:52:59,124] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:59,125] jepsen worker 61 - jepsen.util 61 :ok :read [20 37] INFO [2022-01-10 12:52:59,126] jepsen worker 46 - jepsen.util 46 :invoke :write [15 33] INFO [2022-01-10 12:52:59,128] jepsen worker 14 - jepsen.util 14 :invoke :write [4 35] INFO [2022-01-10 12:52:59,131] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:52:59,132] jepsen worker 29 - jepsen.util 29 :ok :read [9 45] INFO [2022-01-10 12:52:59,134] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:52:59,135] jepsen worker 51 - jepsen.util 51 :ok :read [17 1] INFO [2022-01-10 12:52:59,137] jepsen worker 14 - jepsen.util 14 :ok :write [4 35] INFO [2022-01-10 12:52:59,138] jepsen worker 24 - jepsen.util 24 :invoke :write [8 42] INFO [2022-01-10 12:52:59,139] jepsen worker 55 - jepsen.util 55 :ok :write [18 12] INFO [2022-01-10 12:52:59,140] jepsen worker 11 - jepsen.util 11 :ok :write [3 47] INFO [2022-01-10 12:52:59,142] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:52:59,143] jepsen worker 41 - jepsen.util 41 :ok :read [13 33] INFO [2022-01-10 12:52:59,145] jepsen worker 37 - jepsen.util 37 :invoke :write [12 17] INFO [2022-01-10 12:52:59,147] jepsen worker 72 - jepsen.util 72 :ok :write [24 3] INFO [2022-01-10 12:52:59,148] jepsen worker 78 - jepsen.util 78 :invoke :write [26 24] INFO [2022-01-10 12:52:59,150] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:52:59,151] jepsen worker 29 - jepsen.util 29 :ok :read [9 45] INFO [2022-01-10 12:52:59,152] jepsen worker 54 - jepsen.util 54 :ok :write [18 37] INFO [2022-01-10 12:52:59,153] jepsen worker 41 - jepsen.util 41 :invoke :write [13 36] INFO [2022-01-10 12:52:59,154] jepsen worker 45 - jepsen.util 45 :ok :write [15 25] INFO [2022-01-10 12:52:59,155] jepsen worker 21 - jepsen.util 21 :invoke :write [7 5] INFO [2022-01-10 12:52:59,157] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:52:59,159] jepsen worker 11 - jepsen.util 11 :ok :read [3 47] INFO [2022-01-10 12:52:59,160] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:52:59,160] jepsen worker 78 - jepsen.util 78 :ok :write [26 24] INFO [2022-01-10 12:52:59,161] jepsen worker 1 - jepsen.util 1 :ok :read [0 26] INFO [2022-01-10 12:52:59,161] jepsen worker 31 - jepsen.util 31 :ok :write [10 17] INFO [2022-01-10 12:52:59,163] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:52:59,164] jepsen worker 52 - jepsen.util 52 :ok :read [17 1] INFO [2022-01-10 12:52:59,164] jepsen worker 21 - jepsen.util 21 :ok :write [7 5] INFO [2022-01-10 12:52:59,165] jepsen worker 48 - jepsen.util 48 :invoke :write [16 27] INFO [2022-01-10 12:52:59,168] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:59,169] jepsen worker 59 - jepsen.util 59 :ok :write [19 8] INFO [2022-01-10 12:52:59,169] jepsen worker 4 - jepsen.util 4 :ok :read [1 22] INFO [2022-01-10 12:52:59,172] jepsen worker 9 - jepsen.util 9 :ok :write [3 9] INFO [2022-01-10 12:52:59,173] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:52:59,174] jepsen worker 62 - jepsen.util 62 :ok :read [20 37] INFO [2022-01-10 12:52:59,177] jepsen worker 70 - jepsen.util 70 :invoke :write [23 42] INFO [2022-01-10 12:52:59,177] jepsen worker 24 - jepsen.util 24 :ok :write [8 42] INFO [2022-01-10 12:52:59,180] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:52:59,181] jepsen worker 24 - jepsen.util 24 :ok :read [8 42] INFO [2022-01-10 12:52:59,184] jepsen worker 56 - jepsen.util 56 :ok :write [18 21] INFO [2022-01-10 12:52:59,184] jepsen worker 55 - jepsen.util 55 :invoke :write [18 40] INFO [2022-01-10 12:52:59,188] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:59,189] jepsen worker 33 - jepsen.util 33 :ok :read [11 44] INFO [2022-01-10 12:52:59,189] jepsen worker 70 - jepsen.util 70 :ok :write [23 42] INFO [2022-01-10 12:52:59,190] jepsen worker 46 - jepsen.util 46 :ok :write [15 33] INFO [2022-01-10 12:52:59,191] jepsen worker 73 - jepsen.util 73 :invoke :write [24 32] INFO [2022-01-10 12:52:59,193] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:59,194] jepsen worker 16 - jepsen.util 16 :ok :read [5 12] INFO [2022-01-10 12:52:59,197] jepsen worker 32 - jepsen.util 32 :ok :write [10 2] INFO [2022-01-10 12:52:59,197] jepsen worker 61 - jepsen.util 61 :invoke :write [20 15] INFO [2022-01-10 12:52:59,198] jepsen worker 55 - jepsen.util 55 :ok :write [18 40] INFO [2022-01-10 12:52:59,199] jepsen worker 24 - jepsen.util 24 :invoke :write [8 38] INFO [2022-01-10 12:52:59,202] jepsen worker 50 - jepsen.util 50 :invoke :write [16 19] INFO [2022-01-10 12:52:59,204] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:59,205] jepsen worker 23 - jepsen.util 23 :ok :read [7 5] INFO [2022-01-10 12:52:59,207] jepsen worker 65 - jepsen.util 65 :invoke :write [21 27] INFO [2022-01-10 12:52:59,209] jepsen worker 61 - jepsen.util 61 :ok :write [20 15] INFO [2022-01-10 12:52:59,209] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:59,210] jepsen worker 83 - jepsen.util 83 :ok :read [27 23] INFO [2022-01-10 12:52:59,211] jepsen worker 36 - jepsen.util 36 :invoke :write [12 17] INFO [2022-01-10 12:52:59,214] jepsen worker 52 - jepsen.util 52 :invoke :write [17 46] INFO [2022-01-10 12:52:59,215] jepsen worker 24 - jepsen.util 24 :ok :write [8 38] INFO [2022-01-10 12:52:59,216] jepsen worker 46 - jepsen.util 46 :invoke :write [15 48] INFO [2022-01-10 12:52:59,218] jepsen worker 16 - jepsen.util 16 :invoke :write [5 42] INFO [2022-01-10 12:52:59,219] jepsen worker 47 - jepsen.util 47 :ok :write [15 35] INFO [2022-01-10 12:52:59,222] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:52:59,223] jepsen worker 76 - jepsen.util 76 :ok :read [25 3] INFO [2022-01-10 12:52:59,224] jepsen worker 73 - jepsen.util 73 :ok :write [24 32] INFO [2022-01-10 12:52:59,224] jepsen worker 34 - jepsen.util 34 :invoke :write [11 23] INFO [2022-01-10 12:52:59,227] jepsen worker 89 - jepsen.util 89 :invoke :write [29 24] INFO [2022-01-10 12:52:59,229] jepsen worker 17 - jepsen.util 17 :invoke :read [5 nil] INFO [2022-01-10 12:52:59,230] jepsen worker 17 - jepsen.util 17 :ok :read [5 12] INFO [2022-01-10 12:52:59,232] jepsen worker 23 - jepsen.util 23 :invoke :write [7 8] INFO [2022-01-10 12:52:59,235] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:52:59,236] jepsen worker 8 - jepsen.util 8 :ok :read [2 41] INFO [2022-01-10 12:52:59,238] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:59,239] jepsen worker 83 - jepsen.util 83 :ok :read [27 23] INFO [2022-01-10 12:52:59,240] jepsen worker 26 - jepsen.util 26 :invoke :write [8 20] INFO [2022-01-10 12:52:59,243] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:52:59,244] jepsen worker 21 - jepsen.util 21 :ok :read [7 5] INFO [2022-01-10 12:52:59,246] jepsen worker 87 - jepsen.util 87 :invoke :write [29 12] INFO [2022-01-10 12:52:59,248] jepsen worker 0 - jepsen.util 0 :invoke :write [0 31] INFO [2022-01-10 12:52:59,250] jepsen worker 53 - jepsen.util 53 :invoke :write [17 30] INFO [2022-01-10 12:52:59,253] jepsen worker 73 - jepsen.util 73 :invoke :write [24 4] INFO [2022-01-10 12:52:59,255] jepsen worker 71 - jepsen.util 71 :invoke :write [23 0] INFO [2022-01-10 12:52:59,258] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 12:52:59,259] jepsen worker 42 - jepsen.util 42 :ok :read [14 23] INFO [2022-01-10 12:52:59,260] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:52:59,261] jepsen worker 55 - jepsen.util 55 :ok :read [18 40] INFO [2022-01-10 12:52:59,263] jepsen worker 25 - jepsen.util 25 :invoke :read [8 nil] INFO [2022-01-10 12:52:59,264] jepsen worker 86 - jepsen.util 86 :ok :write [28 20] INFO [2022-01-10 12:52:59,265] jepsen worker 52 - jepsen.util 52 :ok :write [17 46] INFO [2022-01-10 12:52:59,265] jepsen worker 25 - jepsen.util 25 :ok :read [8 38] INFO [2022-01-10 12:52:59,266] jepsen worker 7 - jepsen.util 7 :invoke :write [2 41] INFO [2022-01-10 12:52:59,268] jepsen worker 85 - jepsen.util 85 :invoke :write [28 47] INFO [2022-01-10 12:52:59,269] jepsen worker 65 - jepsen.util 65 :ok :write [21 27] INFO [2022-01-10 12:52:59,271] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 12:52:59,272] jepsen worker 80 - jepsen.util 80 :ok :read [26 24] INFO [2022-01-10 12:52:59,272] jepsen worker 37 - jepsen.util 37 :ok :write [12 17] INFO [2022-01-10 12:52:59,273] jepsen worker 49 - jepsen.util 49 :ok :write [16 16] INFO [2022-01-10 12:52:59,273] jepsen worker 23 - jepsen.util 23 :ok :write [7 8] INFO [2022-01-10 12:52:59,274] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:59,276] jepsen worker 82 - jepsen.util 82 :ok :read [27 23] INFO [2022-01-10 12:52:59,277] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:59,278] jepsen worker 58 - jepsen.util 58 :ok :write [19 46] INFO [2022-01-10 12:52:59,278] jepsen worker 60 - jepsen.util 60 :ok :read [20 15] INFO [2022-01-10 12:52:59,280] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:52:59,281] jepsen worker 14 - jepsen.util 14 :ok :read [4 35] INFO [2022-01-10 12:52:59,281] jepsen worker 46 - jepsen.util 46 :ok :write [15 48] INFO [2022-01-10 12:52:59,283] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:59,283] jepsen worker 26 - jepsen.util 26 :ok :write [8 20] INFO [2022-01-10 12:52:59,284] jepsen worker 60 - jepsen.util 60 :ok :read [20 15] INFO [2022-01-10 12:52:59,284] jepsen worker 85 - jepsen.util 85 :ok :write [28 47] INFO [2022-01-10 12:52:59,285] jepsen worker 78 - jepsen.util 78 :invoke :write [26 45] INFO [2022-01-10 12:52:59,286] jepsen worker 73 - jepsen.util 73 :ok :write [24 4] INFO [2022-01-10 12:52:59,288] jepsen worker 71 - jepsen.util 71 :ok :write [23 0] INFO [2022-01-10 12:52:59,289] jepsen worker 40 - jepsen.util 40 :ok :write [13 47] INFO [2022-01-10 12:52:59,289] jepsen worker 11 - jepsen.util 11 :invoke :write [3 21] INFO [2022-01-10 12:52:59,291] jepsen worker 36 - jepsen.util 36 :ok :write [12 17] INFO [2022-01-10 12:52:59,292] jepsen worker 53 - jepsen.util 53 :ok :write [17 30] INFO [2022-01-10 12:52:59,292] jepsen worker 10 - jepsen.util 10 :ok :write [3 47] INFO [2022-01-10 12:52:59,293] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:59,294] jepsen worker 4 - jepsen.util 4 :ok :read [1 22] INFO [2022-01-10 12:52:59,298] jepsen worker 13 - jepsen.util 13 :invoke :write [4 23] INFO [2022-01-10 12:52:59,301] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:52:59,302] jepsen worker 70 - jepsen.util 70 :ok :read [23 0] INFO [2022-01-10 12:52:59,302] jepsen worker 34 - jepsen.util 34 :ok :write [11 23] INFO [2022-01-10 12:52:59,304] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:59,305] jepsen worker 46 - jepsen.util 46 :ok :read [15 48] INFO [2022-01-10 12:52:59,306] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:52:59,307] jepsen worker 86 - jepsen.util 86 :ok :read [28 47] INFO [2022-01-10 12:52:59,309] jepsen worker 6 - jepsen.util 6 :invoke :write [2 39] INFO [2022-01-10 12:52:59,311] jepsen worker 58 - jepsen.util 58 :invoke :write [19 36] INFO [2022-01-10 12:52:59,314] jepsen worker 79 - jepsen.util 79 :invoke :write [26 3] INFO [2022-01-10 12:52:59,316] jepsen worker 3 - jepsen.util 3 :invoke :write [1 10] INFO [2022-01-10 12:52:59,320] jepsen worker 88 - jepsen.util 88 :invoke :read [29 nil] INFO [2022-01-10 12:52:59,321] jepsen worker 88 - jepsen.util 88 :ok :read [29 37] INFO [2022-01-10 12:52:59,324] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:59,325] jepsen worker 36 - jepsen.util 36 :ok :read [12 17] INFO [2022-01-10 12:52:59,328] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:52:59,329] jepsen worker 82 - jepsen.util 82 :ok :read [27 23] INFO [2022-01-10 12:52:59,331] jepsen worker 22 - jepsen.util 22 :invoke :read [7 nil] INFO [2022-01-10 12:52:59,332] jepsen worker 22 - jepsen.util 22 :ok :read [7 8] INFO [2022-01-10 12:52:59,333] jepsen worker 4 - jepsen.util 4 :invoke :write [1 47] INFO [2022-01-10 12:52:59,336] jepsen worker 24 - jepsen.util 24 :invoke :write [8 37] INFO [2022-01-10 12:52:59,338] jepsen worker 35 - jepsen.util 35 :invoke :write [11 40] INFO [2022-01-10 12:52:59,340] jepsen worker 5 - jepsen.util 5 :invoke :write [1 2] INFO [2022-01-10 12:52:59,342] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:59,342] jepsen worker 16 - jepsen.util 16 :ok :write [5 42] INFO [2022-01-10 12:52:59,343] jepsen worker 45 - jepsen.util 45 :ok :read [15 48] INFO [2022-01-10 12:52:59,344] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:52:59,345] jepsen worker 19 - jepsen.util 19 :ok :read [6 18] INFO [2022-01-10 12:52:59,347] jepsen worker 20 - jepsen.util 20 :invoke :read [6 nil] INFO [2022-01-10 12:52:59,348] jepsen worker 20 - jepsen.util 20 :ok :read [6 18] INFO [2022-01-10 12:52:59,350] jepsen worker 88 - jepsen.util 88 :invoke :write [29 28] INFO [2022-01-10 12:52:59,352] jepsen worker 89 - jepsen.util 89 :ok :write [29 24] INFO [2022-01-10 12:52:59,352] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:59,355] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:52:59,355] jepsen worker 33 - jepsen.util 33 :ok :read [11 23] INFO [2022-01-10 12:52:59,356] jepsen worker 73 - jepsen.util 73 :ok :read [24 4] INFO [2022-01-10 12:52:59,358] jepsen worker 28 - jepsen.util 28 :invoke :write [9 19] INFO [2022-01-10 12:52:59,359] jepsen worker 68 - jepsen.util 68 :ok :write [22 37] INFO [2022-01-10 12:52:59,360] jepsen worker 36 - jepsen.util 36 :invoke :write [12 46] INFO [2022-01-10 12:52:59,363] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:52:59,364] jepsen worker 21 - jepsen.util 21 :ok :read [7 8] INFO [2022-01-10 12:52:59,365] jepsen worker 0 - jepsen.util 0 :ok :write [0 31] INFO [2022-01-10 12:52:59,366] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:52:59,367] jepsen worker 70 - jepsen.util 70 :ok :read [23 0] INFO [2022-01-10 12:52:59,369] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:52:59,370] jepsen worker 61 - jepsen.util 61 :ok :read [20 15] INFO [2022-01-10 12:52:59,372] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:52:59,373] jepsen worker 9 - jepsen.util 9 :ok :read [3 47] INFO [2022-01-10 12:52:59,373] jepsen worker 7 - jepsen.util 7 :ok :write [2 41] INFO [2022-01-10 12:52:59,374] jepsen worker 17 - jepsen.util 17 :invoke :write [5 22] INFO [2022-01-10 12:52:59,377] jepsen worker 82 - jepsen.util 82 :invoke :write [27 35] INFO [2022-01-10 12:52:59,379] jepsen worker 66 - jepsen.util 66 :invoke :write [22 48] INFO [2022-01-10 12:52:59,379] jepsen worker 48 - jepsen.util 48 :ok :write [16 27] INFO [2022-01-10 12:52:59,380] jepsen worker 58 - jepsen.util 58 :ok :write [19 36] INFO [2022-01-10 12:52:59,383] jepsen worker 40 - jepsen.util 40 :invoke :write [13 25] INFO [2022-01-10 12:52:59,385] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:59,386] jepsen worker 39 - jepsen.util 39 :ok :read [13 47] INFO [2022-01-10 12:52:59,387] jepsen worker 81 - jepsen.util 81 :ok :write [27 24] INFO [2022-01-10 12:52:59,388] jepsen worker 38 - jepsen.util 38 :invoke :write [12 42] INFO [2022-01-10 12:52:59,391] jepsen worker 59 - jepsen.util 59 :invoke :write [19 35] INFO [2022-01-10 12:52:59,394] jepsen worker 8 - jepsen.util 8 :invoke :write [2 41] INFO [2022-01-10 12:52:59,398] jepsen worker 39 - jepsen.util 39 :invoke :write [13 9] INFO [2022-01-10 12:52:59,402] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:52:59,403] jepsen worker 16 - jepsen.util 16 :ok :read [5 42] INFO [2022-01-10 12:52:59,403] jepsen worker 35 - jepsen.util 35 :ok :write [11 40] INFO [2022-01-10 12:52:59,405] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:59,406] jepsen worker 44 - jepsen.util 44 :ok :read [14 23] INFO [2022-01-10 12:52:59,408] jepsen worker 59 - jepsen.util 59 :ok :write [19 35] INFO [2022-01-10 12:52:59,409] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:59,410] jepsen worker 63 - jepsen.util 63 :ok :read [21 27] INFO [2022-01-10 12:52:59,413] jepsen worker 14 - jepsen.util 14 :invoke :write [4 34] INFO [2022-01-10 12:52:59,416] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:52:59,417] jepsen worker 37 - jepsen.util 37 :ok :read [12 17] INFO [2022-01-10 12:52:59,419] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:52:59,420] jepsen worker 76 - jepsen.util 76 :ok :read [25 3] INFO [2022-01-10 12:52:59,421] jepsen worker 28 - jepsen.util 28 :ok :write [9 19] INFO [2022-01-10 12:52:59,423] jepsen worker 51 - jepsen.util 51 :invoke :write [17 30] INFO [2022-01-10 12:52:59,426] jepsen worker 89 - jepsen.util 89 :invoke :write [29 21] INFO [2022-01-10 12:52:59,429] jepsen worker 47 - jepsen.util 47 :invoke :write [15 28] INFO [2022-01-10 12:52:59,433] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:59,434] jepsen worker 71 - jepsen.util 71 :ok :read [23 0] INFO [2022-01-10 12:52:59,435] jepsen worker 78 - jepsen.util 78 :ok :write [26 45] INFO [2022-01-10 12:52:59,435] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:52:59,437] jepsen worker 68 - jepsen.util 68 :ok :read [22 37] INFO [2022-01-10 12:52:59,438] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:59,439] jepsen worker 67 - jepsen.util 67 :ok :read [22 37] INFO [2022-01-10 12:52:59,441] jepsen worker 1 - jepsen.util 1 :invoke :write [0 35] INFO [2022-01-10 12:52:59,441] jepsen worker 41 - jepsen.util 41 :ok :write [13 36] INFO [2022-01-10 12:52:59,443] jepsen worker 25 - jepsen.util 25 :invoke :read [8 nil] INFO [2022-01-10 12:52:59,446] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:59,447] jepsen worker 25 - jepsen.util 25 :ok :read [8 20] INFO [2022-01-10 12:52:59,447] jepsen worker 84 - jepsen.util 84 :ok :read [28 47] INFO [2022-01-10 12:52:59,448] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:59,449] jepsen worker 45 - jepsen.util 45 :ok :read [15 48] INFO [2022-01-10 12:52:59,450] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:52:59,451] jepsen worker 76 - jepsen.util 76 :ok :read [25 3] INFO [2022-01-10 12:52:59,451] jepsen worker 88 - jepsen.util 88 :ok :write [29 28] INFO [2022-01-10 12:52:59,453] jepsen worker 70 - jepsen.util 70 :invoke :write [23 38] INFO [2022-01-10 12:52:59,453] jepsen worker 36 - jepsen.util 36 :ok :write [12 46] INFO [2022-01-10 12:52:59,455] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:59,456] jepsen worker 43 - jepsen.util 43 :ok :read [14 23] INFO [2022-01-10 12:52:59,458] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:52:59,459] jepsen worker 27 - jepsen.util 27 :ok :read [9 19] INFO [2022-01-10 12:52:59,460] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:52:59,461] jepsen worker 29 - jepsen.util 29 :ok :read [9 19] INFO [2022-01-10 12:52:59,462] jepsen worker 26 - jepsen.util 26 :invoke :write [8 38] INFO [2022-01-10 12:52:59,465] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:59,467] jepsen worker 31 - jepsen.util 31 :ok :read [10 2] INFO [2022-01-10 12:52:59,468] jepsen worker 69 - jepsen.util 69 :invoke :write [23 17] INFO [2022-01-10 12:52:59,471] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:52:59,471] jepsen worker 11 - jepsen.util 11 :ok :write [3 21] INFO [2022-01-10 12:52:59,472] jepsen worker 1 - jepsen.util 1 :ok :write [0 35] INFO [2022-01-10 12:52:59,472] jepsen worker 40 - jepsen.util 40 :ok :write [13 25] INFO [2022-01-10 12:52:59,472] jepsen worker 54 - jepsen.util 54 :ok :read [18 40] INFO [2022-01-10 12:52:59,473] jepsen worker 88 - jepsen.util 88 :invoke :write [29 18] INFO [2022-01-10 12:52:59,476] jepsen worker 7 - jepsen.util 7 :invoke :write [2 19] INFO [2022-01-10 12:52:59,478] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:59,479] jepsen worker 43 - jepsen.util 43 :ok :read [14 23] INFO [2022-01-10 12:52:59,480] jepsen worker 89 - jepsen.util 89 :ok :write [29 21] INFO [2022-01-10 12:52:59,480] jepsen worker 46 - jepsen.util 46 :invoke :write [15 47] INFO [2022-01-10 12:52:59,483] jepsen worker 42 - jepsen.util 42 :invoke :write [14 37] INFO [2022-01-10 12:52:59,485] jepsen worker 64 - jepsen.util 64 :invoke :write [21 39] INFO [2022-01-10 12:52:59,488] jepsen worker 37 - jepsen.util 37 :invoke :write [12 24] INFO [2022-01-10 12:52:59,490] jepsen worker 48 - jepsen.util 48 :invoke :write [16 26] INFO [2022-01-10 12:52:59,492] jepsen worker 3 - jepsen.util 3 :ok :write [1 10] INFO [2022-01-10 12:52:59,493] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:59,494] jepsen worker 39 - jepsen.util 39 :ok :write [13 9] INFO [2022-01-10 12:52:59,495] jepsen worker 31 - jepsen.util 31 :ok :read [10 2] INFO [2022-01-10 12:52:59,497] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:52:59,498] jepsen worker 43 - jepsen.util 43 :ok :read [14 23] INFO [2022-01-10 12:52:59,499] jepsen worker 88 - jepsen.util 88 :ok :write [29 18] INFO [2022-01-10 12:52:59,500] jepsen worker 23 - jepsen.util 23 :invoke :write [7 29] INFO [2022-01-10 12:52:59,503] jepsen worker 38 - jepsen.util 38 :ok :write [12 42] INFO [2022-01-10 12:52:59,504] jepsen worker 49 - jepsen.util 49 :invoke :write [16 30] INFO [2022-01-10 12:52:59,506] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:59,508] jepsen worker 71 - jepsen.util 71 :ok :read [23 0] INFO [2022-01-10 12:52:59,508] jepsen worker 43 - jepsen.util 43 :invoke :write [14 49] INFO [2022-01-10 12:52:59,511] jepsen worker 29 - jepsen.util 29 :invoke :write [9 22] INFO [2022-01-10 12:52:59,514] jepsen worker 28 - jepsen.util 28 :invoke :read [9 nil] INFO [2022-01-10 12:52:59,515] jepsen worker 28 - jepsen.util 28 :ok :read [9 19] INFO [2022-01-10 12:52:59,517] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:52:59,518] jepsen worker 67 - jepsen.util 67 :ok :read [22 37] INFO [2022-01-10 12:52:59,518] jepsen worker 6 - jepsen.util 6 :ok :write [2 39] INFO [2022-01-10 12:52:59,519] jepsen worker 13 - jepsen.util 13 :ok :write [4 23] INFO [2022-01-10 12:52:59,519] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:59,520] jepsen worker 12 - jepsen.util 12 :ok :read [4 23] INFO [2022-01-10 12:52:59,522] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:52:59,523] jepsen worker 86 - jepsen.util 86 :ok :read [28 47] INFO [2022-01-10 12:52:59,524] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 12:52:59,524] jepsen worker 24 - jepsen.util 24 :ok :write [8 37] INFO [2022-01-10 12:52:59,525] jepsen worker 29 - jepsen.util 29 :ok :write [9 22] INFO [2022-01-10 12:52:59,525] jepsen worker 80 - jepsen.util 80 :ok :read [26 45] INFO [2022-01-10 12:52:59,527] jepsen worker 17 - jepsen.util 17 :ok :write [5 22] INFO [2022-01-10 12:52:59,527] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:52:59,527] jepsen worker 4 - jepsen.util 4 :ok :write [1 47] INFO [2022-01-10 12:52:59,528] jepsen worker 81 - jepsen.util 81 :ok :read [27 24] INFO [2022-01-10 12:52:59,529] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:52:59,531] jepsen worker 71 - jepsen.util 71 :ok :read [23 0] INFO [2022-01-10 12:52:59,532] jepsen worker 78 - jepsen.util 78 :invoke :write [26 30] INFO [2022-01-10 12:52:59,533] jepsen worker 37 - jepsen.util 37 :ok :write [12 24] INFO [2022-01-10 12:52:59,535] jepsen worker 71 - jepsen.util 71 :invoke :write [23 43] INFO [2022-01-10 12:52:59,537] jepsen worker 19 - jepsen.util 19 :invoke :write [6 46] INFO [2022-01-10 12:52:59,539] jepsen worker 31 - jepsen.util 31 :invoke :write [10 13] INFO [2022-01-10 12:52:59,541] jepsen worker 25 - jepsen.util 25 :invoke :read [8 nil] INFO [2022-01-10 12:52:59,542] jepsen worker 25 - jepsen.util 25 :ok :read [8 37] INFO [2022-01-10 12:52:59,543] jepsen worker 22 - jepsen.util 22 :invoke :write [7 39] INFO [2022-01-10 12:52:59,545] jepsen worker 4 - jepsen.util 4 :invoke :write [1 21] INFO [2022-01-10 12:52:59,548] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:52:59,549] jepsen worker 12 - jepsen.util 12 :ok :read [4 23] INFO [2022-01-10 12:52:59,550] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:59,551] jepsen worker 45 - jepsen.util 45 :ok :read [15 48] INFO [2022-01-10 12:52:59,552] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:59,553] jepsen worker 83 - jepsen.util 83 :ok :read [27 24] INFO [2022-01-10 12:52:59,554] jepsen worker 31 - jepsen.util 31 :ok :write [10 13] INFO [2022-01-10 12:52:59,554] jepsen worker 24 - jepsen.util 24 :invoke :write [8 9] INFO [2022-01-10 12:52:59,557] jepsen worker 88 - jepsen.util 88 :invoke :write [29 32] INFO [2022-01-10 12:52:59,557] jepsen worker 4 - jepsen.util 4 :ok :write [1 21] INFO [2022-01-10 12:52:59,559] jepsen worker 67 - jepsen.util 67 :invoke :write [22 16] INFO [2022-01-10 12:52:59,561] jepsen worker 34 - jepsen.util 34 :invoke :write [11 2] INFO [2022-01-10 12:52:59,564] jepsen worker 6 - jepsen.util 6 :invoke :write [2 11] INFO [2022-01-10 12:52:59,565] jepsen worker 88 - jepsen.util 88 :ok :write [29 32] INFO [2022-01-10 12:52:59,565] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:59,567] jepsen worker 39 - jepsen.util 39 :ok :read [13 9] INFO [2022-01-10 12:52:59,568] jepsen worker 73 - jepsen.util 73 :invoke :write [24 3] INFO [2022-01-10 12:52:59,570] jepsen worker 1 - jepsen.util 1 :invoke :write [0 39] INFO [2022-01-10 12:52:59,572] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 12:52:59,574] jepsen worker 80 - jepsen.util 80 :ok :read [26 45] INFO [2022-01-10 12:52:59,575] jepsen worker 20 - jepsen.util 20 :invoke :write [6 11] INFO [2022-01-10 12:52:59,577] jepsen worker 87 - jepsen.util 87 :ok :write [29 12] INFO [2022-01-10 12:52:59,578] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:52:59,579] jepsen worker 13 - jepsen.util 13 :ok :read [4 23] INFO [2022-01-10 12:52:59,580] jepsen worker 17 - jepsen.util 17 :invoke :write [5 42] INFO [2022-01-10 12:52:59,582] jepsen worker 30 - jepsen.util 30 :invoke :write [10 41] INFO [2022-01-10 12:52:59,582] jepsen worker 1 - jepsen.util 1 :ok :write [0 39] INFO [2022-01-10 12:52:59,584] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:59,585] jepsen worker 18 - jepsen.util 18 :ok :read [6 18] INFO [2022-01-10 12:52:59,588] jepsen worker 25 - jepsen.util 25 :invoke :read [8 nil] INFO [2022-01-10 12:52:59,589] jepsen worker 25 - jepsen.util 25 :ok :read [8 37] INFO [2022-01-10 12:52:59,590] jepsen worker 58 - jepsen.util 58 :invoke :write [19 46] INFO [2022-01-10 12:52:59,592] jepsen worker 88 - jepsen.util 88 :invoke :write [29 0] INFO [2022-01-10 12:52:59,594] jepsen worker 30 - jepsen.util 30 :ok :write [10 41] INFO [2022-01-10 12:52:59,595] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:59,596] jepsen worker 60 - jepsen.util 60 :ok :read [20 15] INFO [2022-01-10 12:52:59,598] jepsen worker 21 - jepsen.util 21 :invoke :write [7 8] INFO [2022-01-10 12:52:59,599] jepsen worker 58 - jepsen.util 58 :ok :write [19 46] INFO [2022-01-10 12:52:59,601] jepsen worker 52 - jepsen.util 52 :invoke :write [17 48] INFO [2022-01-10 12:52:59,603] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:52:59,603] jepsen worker 88 - jepsen.util 88 :ok :write [29 0] INFO [2022-01-10 12:52:59,604] jepsen worker 84 - jepsen.util 84 :ok :read [28 47] INFO [2022-01-10 12:52:59,605] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 12:52:59,606] jepsen worker 80 - jepsen.util 80 :ok :read [26 45] INFO [2022-01-10 12:52:59,607] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:59,608] jepsen worker 83 - jepsen.util 83 :ok :read [27 24] INFO [2022-01-10 12:52:59,609] jepsen worker 72 - jepsen.util 72 :invoke :write [24 43] INFO [2022-01-10 12:52:59,611] jepsen worker 56 - jepsen.util 56 :invoke :read [18 nil] INFO [2022-01-10 12:52:59,612] jepsen worker 56 - jepsen.util 56 :ok :read [18 40] INFO [2022-01-10 12:52:59,613] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:59,615] jepsen worker 18 - jepsen.util 18 :ok :read [6 18] INFO [2022-01-10 12:52:59,615] jepsen worker 16 - jepsen.util 16 :invoke :write [5 35] INFO [2022-01-10 12:52:59,617] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:59,618] jepsen worker 44 - jepsen.util 44 :ok :read [14 23] INFO [2022-01-10 12:52:59,619] jepsen worker 25 - jepsen.util 25 :invoke :read [8 nil] INFO [2022-01-10 12:52:59,620] jepsen worker 25 - jepsen.util 25 :ok :read [8 37] INFO [2022-01-10 12:52:59,621] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:59,622] jepsen worker 39 - jepsen.util 39 :ok :read [13 9] INFO [2022-01-10 12:52:59,623] jepsen worker 65 - jepsen.util 65 :invoke :write [21 45] INFO [2022-01-10 12:52:59,625] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:59,626] jepsen worker 44 - jepsen.util 44 :ok :read [14 23] INFO [2022-01-10 12:52:59,627] jepsen worker 88 - jepsen.util 88 :invoke :write [29 16] INFO [2022-01-10 12:52:59,629] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:59,630] jepsen worker 74 - jepsen.util 74 :ok :read [24 4] INFO [2022-01-10 12:52:59,631] jepsen worker 75 - jepsen.util 75 :invoke :write [25 37] INFO [2022-01-10 12:52:59,633] jepsen worker 18 - jepsen.util 18 :invoke :write [6 49] INFO [2022-01-10 12:52:59,635] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:52:59,636] jepsen worker 63 - jepsen.util 63 :ok :read [21 27] INFO [2022-01-10 12:52:59,637] jepsen worker 25 - jepsen.util 25 :invoke :write [8 45] INFO [2022-01-10 12:52:59,639] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:52:59,640] jepsen worker 44 - jepsen.util 44 :ok :read [14 23] INFO [2022-01-10 12:52:59,642] jepsen worker 51 - jepsen.util 51 :ok :write [17 30] INFO [2022-01-10 12:52:59,642] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 12:52:59,642] jepsen worker 50 - jepsen.util 50 :ok :write [16 19] INFO [2022-01-10 12:52:59,643] jepsen worker 80 - jepsen.util 80 :ok :read [26 45] INFO [2022-01-10 12:52:59,645] jepsen worker 38 - jepsen.util 38 :invoke :write [12 37] INFO [2022-01-10 12:52:59,647] jepsen worker 47 - jepsen.util 47 :ok :write [15 28] INFO [2022-01-10 12:52:59,647] jepsen worker 82 - jepsen.util 82 :ok :write [27 35] INFO [2022-01-10 12:52:59,647] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:52:59,648] jepsen worker 74 - jepsen.util 74 :ok :read [24 4] INFO [2022-01-10 12:52:59,650] jepsen worker 3 - jepsen.util 3 :invoke :write [1 17] INFO [2022-01-10 12:52:59,652] jepsen worker 79 - jepsen.util 79 :ok :write [26 3] INFO [2022-01-10 12:52:59,653] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:59,654] jepsen worker 31 - jepsen.util 31 :ok :read [10 41] INFO [2022-01-10 12:52:59,654] jepsen worker 34 - jepsen.util 34 :ok :write [11 2] INFO [2022-01-10 12:52:59,655] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:59,657] jepsen worker 70 - jepsen.util 70 :ok :write [23 38] INFO [2022-01-10 12:52:59,657] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:52:59,657] jepsen worker 4 - jepsen.util 4 :ok :read [1 21] INFO [2022-01-10 12:52:59,658] jepsen worker 76 - jepsen.util 76 :ok :read [25 3] INFO [2022-01-10 12:52:59,659] jepsen worker 54 - jepsen.util 54 :invoke :write [18 39] INFO [2022-01-10 12:52:59,661] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:52:59,662] jepsen worker 66 - jepsen.util 66 :ok :write [22 48] INFO [2022-01-10 12:52:59,663] jepsen worker 58 - jepsen.util 58 :ok :read [19 46] INFO [2022-01-10 12:52:59,663] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:59,663] jepsen worker 64 - jepsen.util 64 :ok :write [21 39] INFO [2022-01-10 12:52:59,664] jepsen worker 85 - jepsen.util 85 :ok :read [28 47] INFO [2022-01-10 12:52:59,665] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:52:59,665] jepsen worker 42 - jepsen.util 42 :ok :write [14 37] INFO [2022-01-10 12:52:59,667] jepsen worker 37 - jepsen.util 37 :ok :read [12 24] INFO [2022-01-10 12:52:59,668] jepsen worker 40 - jepsen.util 40 :invoke :write [13 12] INFO [2022-01-10 12:52:59,671] jepsen worker 23 - jepsen.util 23 :ok :write [7 29] INFO [2022-01-10 12:52:59,671] jepsen worker 60 - jepsen.util 60 :invoke :write [20 45] INFO [2022-01-10 12:52:59,673] jepsen worker 54 - jepsen.util 54 :ok :write [18 39] INFO [2022-01-10 12:52:59,673] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:52:59,675] jepsen worker 14 - jepsen.util 14 :ok :write [4 34] INFO [2022-01-10 12:52:59,676] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:52:59,676] jepsen worker 7 - jepsen.util 7 :ok :write [2 19] INFO [2022-01-10 12:52:59,677] jepsen worker 70 - jepsen.util 70 :ok :read [23 38] INFO [2022-01-10 12:52:59,678] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:52:59,680] jepsen worker 53 - jepsen.util 53 :ok :read [17 30] INFO [2022-01-10 12:52:59,680] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:59,681] jepsen worker 23 - jepsen.util 23 :ok :read [7 29] INFO [2022-01-10 12:52:59,681] jepsen worker 26 - jepsen.util 26 :ok :write [8 38] INFO [2022-01-10 12:52:59,682] jepsen worker 17 - jepsen.util 17 :ok :write [5 42] INFO [2022-01-10 12:52:59,682] jepsen worker 7 - jepsen.util 7 :invoke :write [2 20] INFO [2022-01-10 12:52:59,685] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:52:59,686] jepsen worker 19 - jepsen.util 19 :ok :write [6 46] INFO [2022-01-10 12:52:59,686] jepsen worker 45 - jepsen.util 45 :ok :read [15 28] INFO [2022-01-10 12:52:59,687] jepsen worker 51 - jepsen.util 51 :invoke :write [17 19] INFO [2022-01-10 12:52:59,690] jepsen worker 42 - jepsen.util 42 :invoke :write [14 4] INFO [2022-01-10 12:52:59,692] jepsen worker 73 - jepsen.util 73 :ok :write [24 3] INFO [2022-01-10 12:52:59,692] jepsen worker 89 - jepsen.util 89 :invoke :write [29 9] INFO [2022-01-10 12:52:59,694] jepsen worker 41 - jepsen.util 41 :invoke :write [13 2] INFO [2022-01-10 12:52:59,696] jepsen worker 75 - jepsen.util 75 :ok :write [25 37] INFO [2022-01-10 12:52:59,697] jepsen worker 56 - jepsen.util 56 :invoke :write [18 9] INFO [2022-01-10 12:52:59,699] jepsen worker 52 - jepsen.util 52 :ok :write [17 48] INFO [2022-01-10 12:52:59,699] jepsen worker 73 - jepsen.util 73 :invoke :write [24 21] INFO [2022-01-10 12:52:59,701] jepsen worker 87 - jepsen.util 87 :invoke :write [29 17] INFO [2022-01-10 12:52:59,702] jepsen worker 46 - jepsen.util 46 :ok :write [15 47] INFO [2022-01-10 12:52:59,703] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:59,705] jepsen worker 26 - jepsen.util 26 :ok :read [8 38] INFO [2022-01-10 12:52:59,706] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:52:59,707] jepsen worker 33 - jepsen.util 33 :ok :read [11 2] INFO [2022-01-10 12:52:59,708] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:52:59,709] jepsen worker 85 - jepsen.util 85 :ok :read [28 47] INFO [2022-01-10 12:52:59,710] jepsen worker 79 - jepsen.util 79 :invoke :read [26 nil] INFO [2022-01-10 12:52:59,712] jepsen worker 69 - jepsen.util 69 :ok :write [23 17] INFO [2022-01-10 12:52:59,712] jepsen worker 79 - jepsen.util 79 :ok :read [26 3] INFO [2022-01-10 12:52:59,713] jepsen worker 55 - jepsen.util 55 :invoke :write [18 16] INFO [2022-01-10 12:52:59,716] jepsen worker 63 - jepsen.util 63 :invoke :write [21 31] INFO [2022-01-10 12:52:59,718] jepsen worker 65 - jepsen.util 65 :ok :write [21 45] INFO [2022-01-10 12:52:59,718] jepsen worker 0 - jepsen.util 0 :invoke :write [0 47] INFO [2022-01-10 12:52:59,721] jepsen worker 52 - jepsen.util 52 :invoke :write [17 18] INFO [2022-01-10 12:52:59,723] jepsen worker 30 - jepsen.util 30 :invoke :write [10 24] INFO [2022-01-10 12:52:59,726] jepsen worker 11 - jepsen.util 11 :invoke :write [3 2] INFO [2022-01-10 12:52:59,728] jepsen worker 25 - jepsen.util 25 :ok :write [8 45] INFO [2022-01-10 12:52:59,728] jepsen worker 43 - jepsen.util 43 :ok :write [14 49] INFO [2022-01-10 12:52:59,728] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:52:59,729] jepsen worker 26 - jepsen.util 26 :ok :read [8 45] INFO [2022-01-10 12:52:59,731] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:52:59,732] jepsen worker 54 - jepsen.util 54 :ok :read [18 39] INFO [2022-01-10 12:52:59,733] jepsen worker 74 - jepsen.util 74 :invoke :write [24 42] INFO [2022-01-10 12:52:59,734] jepsen worker 21 - jepsen.util 21 :ok :write [7 8] INFO [2022-01-10 12:52:59,735] jepsen worker 23 - jepsen.util 23 :invoke :write [7 7] INFO [2022-01-10 12:52:59,737] jepsen worker 18 - jepsen.util 18 :ok :write [6 49] INFO [2022-01-10 12:52:59,738] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:52:59,738] jepsen worker 88 - jepsen.util 88 :ok :write [29 16] INFO [2022-01-10 12:52:59,739] jepsen worker 38 - jepsen.util 38 :ok :write [12 37] INFO [2022-01-10 12:52:59,739] jepsen worker 39 - jepsen.util 39 :ok :read [13 9] INFO [2022-01-10 12:52:59,740] jepsen worker 60 - jepsen.util 60 :ok :write [20 45] INFO [2022-01-10 12:52:59,741] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:59,742] jepsen worker 3 - jepsen.util 3 :ok :write [1 17] INFO [2022-01-10 12:52:59,743] jepsen worker 69 - jepsen.util 69 :ok :read [23 17] INFO [2022-01-10 12:52:59,743] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:52:59,744] jepsen worker 48 - jepsen.util 48 :ok :write [16 26] INFO [2022-01-10 12:52:59,744] jepsen worker 70 - jepsen.util 70 :ok :read [23 17] INFO [2022-01-10 12:52:59,747] jepsen worker 58 - jepsen.util 58 :invoke :write [19 39] INFO [2022-01-10 12:52:59,750] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:59,751] jepsen worker 48 - jepsen.util 48 :ok :read [16 26] INFO [2022-01-10 12:52:59,752] jepsen worker 51 - jepsen.util 51 :ok :write [17 19] INFO [2022-01-10 12:52:59,753] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:52:59,754] jepsen worker 35 - jepsen.util 35 :ok :read [11 2] INFO [2022-01-10 12:52:59,757] jepsen worker 14 - jepsen.util 14 :invoke :write [4 44] INFO [2022-01-10 12:52:59,760] jepsen worker 32 - jepsen.util 32 :invoke :write [10 28] INFO [2022-01-10 12:52:59,761] jepsen worker 71 - jepsen.util 71 :ok :write [23 43] INFO [2022-01-10 12:52:59,762] jepsen worker 67 - jepsen.util 67 :ok :write [22 16] INFO [2022-01-10 12:52:59,763] jepsen worker 10 - jepsen.util 10 :invoke :write [3 14] INFO [2022-01-10 12:52:59,765] jepsen worker 40 - jepsen.util 40 :ok :write [13 12] INFO [2022-01-10 12:52:59,766] jepsen worker 85 - jepsen.util 85 :invoke :write [28 4] INFO [2022-01-10 12:52:59,767] jepsen worker 78 - jepsen.util 78 :ok :write [26 30] INFO [2022-01-10 12:52:59,769] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:59,770] jepsen worker 7 - jepsen.util 7 :ok :write [2 20] INFO [2022-01-10 12:52:59,770] jepsen worker 60 - jepsen.util 60 :ok :read [20 45] INFO [2022-01-10 12:52:59,771] jepsen worker 47 - jepsen.util 47 :invoke :write [15 44] INFO [2022-01-10 12:52:59,773] jepsen worker 16 - jepsen.util 16 :ok :write [5 35] INFO [2022-01-10 12:52:59,774] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:52:59,775] jepsen worker 69 - jepsen.util 69 :ok :read [23 43] INFO [2022-01-10 12:52:59,776] jepsen worker 76 - jepsen.util 76 :invoke :write [25 22] INFO [2022-01-10 12:52:59,778] jepsen worker 68 - jepsen.util 68 :invoke :write [22 43] INFO [2022-01-10 12:52:59,781] jepsen worker 44 - jepsen.util 44 :invoke :write [14 11] INFO [2022-01-10 12:52:59,783] jepsen worker 34 - jepsen.util 34 :invoke :write [11 10] INFO [2022-01-10 12:52:59,785] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:59,786] jepsen worker 31 - jepsen.util 31 :ok :read [10 41] INFO [2022-01-10 12:52:59,787] jepsen worker 17 - jepsen.util 17 :invoke :write [5 6] INFO [2022-01-10 12:52:59,789] jepsen worker 4 - jepsen.util 4 :invoke :write [1 31] INFO [2022-01-10 12:52:59,791] jepsen worker 67 - jepsen.util 67 :invoke :write [22 45] INFO [2022-01-10 12:52:59,793] jepsen worker 13 - jepsen.util 13 :invoke :write [4 27] INFO [2022-01-10 12:52:59,796] jepsen worker 3 - jepsen.util 3 :invoke :write [1 10] INFO [2022-01-10 12:52:59,797] jepsen worker 86 - jepsen.util 86 :invoke :write [28 44] INFO [2022-01-10 12:52:59,799] jepsen worker 54 - jepsen.util 54 :invoke :write [18 47] INFO [2022-01-10 12:52:59,801] jepsen worker 33 - jepsen.util 33 :invoke :write [11 11] INFO [2022-01-10 12:52:59,803] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:52:59,804] jepsen worker 78 - jepsen.util 78 :ok :read [26 30] INFO [2022-01-10 12:52:59,805] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:52:59,806] jepsen worker 31 - jepsen.util 31 :ok :read [10 41] INFO [2022-01-10 12:52:59,807] jepsen worker 59 - jepsen.util 59 :invoke :write [19 7] INFO [2022-01-10 12:52:59,809] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:59,810] jepsen worker 46 - jepsen.util 46 :ok :read [15 47] INFO [2022-01-10 12:52:59,811] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:59,812] jepsen worker 60 - jepsen.util 60 :ok :read [20 45] INFO [2022-01-10 12:52:59,812] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:59,813] jepsen worker 36 - jepsen.util 36 :ok :read [12 37] INFO [2022-01-10 12:52:59,814] jepsen worker 31 - jepsen.util 31 :invoke :write [10 12] INFO [2022-01-10 12:52:59,816] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 12:52:59,817] jepsen worker 80 - jepsen.util 80 :ok :read [26 30] INFO [2022-01-10 12:52:59,818] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:52:59,819] jepsen worker 83 - jepsen.util 83 :ok :read [27 35] INFO [2022-01-10 12:52:59,820] jepsen worker 81 - jepsen.util 81 :invoke :write [27 15] INFO [2022-01-10 12:52:59,822] jepsen worker 27 - jepsen.util 27 :invoke :write [9 49] INFO [2022-01-10 12:52:59,824] jepsen worker 51 - jepsen.util 51 :invoke :write [17 39] INFO [2022-01-10 12:52:59,825] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:52:59,826] jepsen worker 40 - jepsen.util 40 :ok :read [13 12] INFO [2022-01-10 12:52:59,827] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:52:59,828] jepsen worker 1 - jepsen.util 1 :ok :read [0 39] INFO [2022-01-10 12:52:59,829] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:52:59,830] jepsen worker 18 - jepsen.util 18 :ok :read [6 49] INFO [2022-01-10 12:52:59,831] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:52:59,832] jepsen worker 7 - jepsen.util 7 :ok :read [2 20] INFO [2022-01-10 12:52:59,833] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:52:59,834] jepsen worker 37 - jepsen.util 37 :ok :read [12 37] INFO [2022-01-10 12:52:59,835] jepsen worker 69 - jepsen.util 69 :invoke :write [23 14] INFO [2022-01-10 12:52:59,837] jepsen worker 79 - jepsen.util 79 :invoke :write [26 35] INFO [2022-01-10 12:52:59,839] jepsen worker 84 - jepsen.util 84 :invoke :write [28 18] INFO [2022-01-10 12:52:59,841] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:52:59,842] jepsen worker 37 - jepsen.util 37 :ok :read [12 37] INFO [2022-01-10 12:52:59,843] jepsen worker 7 - jepsen.util 7 :invoke :write [2 17] INFO [2022-01-10 12:52:59,845] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:52:59,846] jepsen worker 36 - jepsen.util 36 :ok :read [12 37] INFO [2022-01-10 12:52:59,847] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:52:59,848] jepsen worker 46 - jepsen.util 46 :ok :read [15 47] INFO [2022-01-10 12:52:59,849] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:52:59,850] jepsen worker 70 - jepsen.util 70 :ok :read [23 43] INFO [2022-01-10 12:52:59,850] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:59,851] jepsen worker 2 - jepsen.util 2 :ok :read [0 39] INFO [2022-01-10 12:52:59,852] jepsen worker 53 - jepsen.util 53 :invoke :write [17 36] INFO [2022-01-10 12:52:59,854] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:52:59,855] jepsen worker 38 - jepsen.util 38 :ok :read [12 37] INFO [2022-01-10 12:52:59,856] jepsen worker 19 - jepsen.util 19 :invoke :write [6 9] INFO [2022-01-10 12:52:59,857] jepsen worker 83 - jepsen.util 83 :invoke :write [27 22] INFO [2022-01-10 12:52:59,859] jepsen worker 45 - jepsen.util 45 :invoke :write [15 39] INFO [2022-01-10 12:52:59,861] jepsen worker 21 - jepsen.util 21 :invoke :write [7 7] INFO [2022-01-10 12:52:59,863] jepsen worker 35 - jepsen.util 35 :invoke :write [11 2] INFO [2022-01-10 12:52:59,866] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:52:59,867] jepsen worker 2 - jepsen.util 2 :ok :read [0 39] INFO [2022-01-10 12:52:59,868] jepsen worker 9 - jepsen.util 9 :invoke :write [3 31] INFO [2022-01-10 12:52:59,871] jepsen worker 36 - jepsen.util 36 :invoke :write [12 0] INFO [2022-01-10 12:52:59,873] jepsen worker 78 - jepsen.util 78 :invoke :write [26 35] INFO [2022-01-10 12:52:59,876] jepsen worker 26 - jepsen.util 26 :invoke :write [8 28] INFO [2022-01-10 12:52:59,951] jepsen worker 23 - jepsen.util 23 :ok :write [7 7] INFO [2022-01-10 12:52:59,952] jepsen worker 42 - jepsen.util 42 :ok :write [14 4] INFO [2022-01-10 12:52:59,952] jepsen worker 11 - jepsen.util 11 :ok :write [3 2] INFO [2022-01-10 12:52:59,952] jepsen worker 20 - jepsen.util 20 :ok :write [6 11] INFO [2022-01-10 12:52:59,951] jepsen worker 36 - jepsen.util 36 :ok :write [12 0] INFO [2022-01-10 12:52:59,952] jepsen worker 30 - jepsen.util 30 :ok :write [10 24] INFO [2022-01-10 12:52:59,951] jepsen worker 0 - jepsen.util 0 :ok :write [0 47] INFO [2022-01-10 12:52:59,952] jepsen worker 57 - jepsen.util 57 :ok :write [19 33] INFO [2022-01-10 12:52:59,952] jepsen worker 73 - jepsen.util 73 :ok :write [24 21] INFO [2022-01-10 12:52:59,953] jepsen worker 87 - jepsen.util 87 :ok :write [29 17] INFO [2022-01-10 12:52:59,953] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:52:59,953] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:52:59,954] jepsen worker 47 - jepsen.util 47 :ok :write [15 44] INFO [2022-01-10 12:52:59,954] jepsen worker 4 - jepsen.util 4 :ok :write [1 31] INFO [2022-01-10 12:52:59,952] jepsen worker 63 - jepsen.util 63 :ok :write [21 31] INFO [2022-01-10 12:52:59,952] jepsen worker 24 - jepsen.util 24 :ok :write [8 9] INFO [2022-01-10 12:52:59,954] jepsen worker 56 - jepsen.util 56 :ok :write [18 9] INFO [2022-01-10 12:52:59,956] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:52:59,957] jepsen worker 70 - jepsen.util 70 :ok :read [23 43] INFO [2022-01-10 12:52:59,957] jepsen worker 60 - jepsen.util 60 :ok :read [20 45] INFO [2022-01-10 12:52:59,959] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:52:59,960] jepsen worker 68 - jepsen.util 68 :ok :write [22 43] INFO [2022-01-10 12:52:59,960] jepsen worker 23 - jepsen.util 23 :ok :read [7 7] INFO [2022-01-10 12:52:59,961] jepsen worker 11 - jepsen.util 11 :invoke :write [3 18] INFO [2022-01-10 12:52:59,962] jepsen worker 52 - jepsen.util 52 :ok :write [17 18] INFO [2022-01-10 12:52:59,964] jepsen worker 28 - jepsen.util 28 :invoke :write [9 1] INFO [2022-01-10 12:52:59,966] jepsen worker 41 - jepsen.util 41 :ok :write [13 2] INFO [2022-01-10 12:52:59,966] jepsen worker 20 - jepsen.util 20 :invoke :write [6 20] INFO [2022-01-10 12:52:59,969] jepsen worker 14 - jepsen.util 14 :ok :write [4 44] INFO [2022-01-10 12:52:59,969] jepsen worker 30 - jepsen.util 30 :invoke :write [10 38] INFO [2022-01-10 12:52:59,971] jepsen worker 76 - jepsen.util 76 :ok :write [25 22] INFO [2022-01-10 12:52:59,972] jepsen worker 80 - jepsen.util 80 :invoke :write [26 26] INFO [2022-01-10 12:52:59,974] jepsen worker 7 - jepsen.util 7 :ok :write [2 17] INFO [2022-01-10 12:52:59,974] jepsen worker 55 - jepsen.util 55 :ok :write [18 16] INFO [2022-01-10 12:52:59,975] jepsen worker 69 - jepsen.util 69 :ok :write [23 14] INFO [2022-01-10 12:52:59,976] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:52:59,977] jepsen worker 57 - jepsen.util 57 :ok :read [19 33] INFO [2022-01-10 12:52:59,978] jepsen worker 17 - jepsen.util 17 :ok :write [5 6] INFO [2022-01-10 12:52:59,978] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:52:59,979] jepsen worker 52 - jepsen.util 52 :ok :read [17 18] INFO [2022-01-10 12:52:59,980] jepsen worker 79 - jepsen.util 79 :ok :write [26 35] INFO [2022-01-10 12:52:59,980] jepsen worker 57 - jepsen.util 57 :invoke :write [19 32] INFO [2022-01-10 12:52:59,982] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:52:59,983] jepsen worker 4 - jepsen.util 4 :ok :read [1 31] INFO [2022-01-10 12:52:59,984] jepsen worker 50 - jepsen.util 50 :invoke :write [16 16] INFO [2022-01-10 12:52:59,984] jepsen worker 34 - jepsen.util 34 :ok :write [11 10] INFO [2022-01-10 12:52:59,986] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:52:59,986] jepsen worker 85 - jepsen.util 85 :ok :write [28 4] INFO [2022-01-10 12:52:59,987] jepsen worker 48 - jepsen.util 48 :ok :read [16 30] INFO [2022-01-10 12:52:59,988] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:52:59,989] jepsen worker 7 - jepsen.util 7 :ok :read [2 17] INFO [2022-01-10 12:52:59,990] jepsen worker 81 - jepsen.util 81 :ok :write [27 15] INFO [2022-01-10 12:52:59,991] jepsen worker 52 - jepsen.util 52 :invoke :write [17 44] INFO [2022-01-10 12:52:59,992] jepsen worker 59 - jepsen.util 59 :ok :write [19 7] INFO [2022-01-10 12:52:59,992] jepsen worker 31 - jepsen.util 31 :ok :write [10 12] INFO [2022-01-10 12:52:59,993] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:52:59,994] jepsen worker 24 - jepsen.util 24 :ok :read [8 9] INFO [2022-01-10 12:52:59,995] jepsen worker 7 - jepsen.util 7 :invoke :write [2 15] INFO [2022-01-10 12:52:59,996] jepsen worker 27 - jepsen.util 27 :ok :write [9 49] INFO [2022-01-10 12:52:59,998] jepsen worker 9 - jepsen.util 9 :ok :write [3 31] INFO [2022-01-10 12:52:59,998] jepsen worker 39 - jepsen.util 39 :invoke :write [13 10] INFO [2022-01-10 12:53:00,000] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:53:00,001] jepsen worker 46 - jepsen.util 46 :ok :read [15 44] INFO [2022-01-10 12:53:00,002] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:53:00,003] jepsen worker 85 - jepsen.util 85 :ok :read [28 4] INFO [2022-01-10 12:53:00,004] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:00,005] jepsen worker 1 - jepsen.util 1 :ok :read [0 47] INFO [2022-01-10 12:53:00,005] jepsen worker 57 - jepsen.util 57 :ok :write [19 32] INFO [2022-01-10 12:53:00,006] jepsen worker 46 - jepsen.util 46 :invoke :write [15 25] INFO [2022-01-10 12:53:00,008] jepsen worker 5 - jepsen.util 5 :ok :write [1 2] INFO [2022-01-10 12:53:00,008] jepsen worker 60 - jepsen.util 60 :invoke :write [20 0] INFO [2022-01-10 12:53:00,011] jepsen worker 34 - jepsen.util 34 :invoke :write [11 49] INFO [2022-01-10 12:53:00,013] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:00,014] jepsen worker 75 - jepsen.util 75 :ok :read [25 22] INFO [2022-01-10 12:53:00,016] jepsen worker 79 - jepsen.util 79 :invoke :read [26 nil] INFO [2022-01-10 12:53:00,017] jepsen worker 79 - jepsen.util 79 :ok :read [26 35] INFO [2022-01-10 12:53:00,018] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:00,019] jepsen worker 27 - jepsen.util 27 :ok :read [9 49] INFO [2022-01-10 12:53:00,021] jepsen worker 48 - jepsen.util 48 :invoke :write [16 41] INFO [2022-01-10 12:53:00,022] jepsen worker 39 - jepsen.util 39 :ok :write [13 10] INFO [2022-01-10 12:53:00,024] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:53:00,025] jepsen worker 82 - jepsen.util 82 :ok :read [27 15] INFO [2022-01-10 12:53:00,027] jepsen worker 2 - jepsen.util 2 :invoke :write [0 30] INFO [2022-01-10 12:53:00,030] jepsen worker 85 - jepsen.util 85 :invoke :write [28 0] INFO [2022-01-10 12:53:00,030] jepsen worker 84 - jepsen.util 84 :ok :write [28 18] INFO [2022-01-10 12:53:00,032] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,033] jepsen worker 4 - jepsen.util 4 :ok :read [1 2] INFO [2022-01-10 12:53:00,035] jepsen worker 36 - jepsen.util 36 :invoke :write [12 30] INFO [2022-01-10 12:53:00,036] jepsen worker 30 - jepsen.util 30 :ok :write [10 38] INFO [2022-01-10 12:53:00,038] jepsen worker 82 - jepsen.util 82 :invoke :write [27 6] INFO [2022-01-10 12:53:00,039] jepsen worker 54 - jepsen.util 54 :ok :write [18 47] INFO [2022-01-10 12:53:00,040] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:53:00,041] jepsen worker 65 - jepsen.util 65 :ok :read [21 31] INFO [2022-01-10 12:53:00,042] jepsen worker 11 - jepsen.util 11 :ok :write [3 18] INFO [2022-01-10 12:53:00,042] jepsen worker 71 - jepsen.util 71 :invoke :write [23 42] INFO [2022-01-10 12:53:00,044] jepsen worker 76 - jepsen.util 76 :invoke :write [25 34] INFO [2022-01-10 12:53:00,046] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,046] jepsen worker 19 - jepsen.util 19 :ok :write [6 9] INFO [2022-01-10 12:53:00,047] jepsen worker 21 - jepsen.util 21 :ok :write [7 7] INFO [2022-01-10 12:53:00,047] jepsen worker 3 - jepsen.util 3 :ok :write [1 10] INFO [2022-01-10 12:53:00,047] jepsen worker 5 - jepsen.util 5 :ok :read [1 10] INFO [2022-01-10 12:53:00,047] jepsen worker 28 - jepsen.util 28 :ok :write [9 1] INFO [2022-01-10 12:53:00,048] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:53:00,049] jepsen worker 14 - jepsen.util 14 :ok :read [4 44] INFO [2022-01-10 12:53:00,050] jepsen worker 24 - jepsen.util 24 :invoke :write [8 36] INFO [2022-01-10 12:53:00,051] jepsen worker 26 - jepsen.util 26 :ok :write [8 28] INFO [2022-01-10 12:53:00,052] jepsen worker 44 - jepsen.util 44 :ok :write [14 11] INFO [2022-01-10 12:53:00,052] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:00,053] jepsen worker 68 - jepsen.util 68 :ok :read [22 43] INFO [2022-01-10 12:53:00,053] jepsen worker 89 - jepsen.util 89 :ok :write [29 9] INFO [2022-01-10 12:53:00,054] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:00,056] jepsen worker 57 - jepsen.util 57 :ok :read [19 32] INFO [2022-01-10 12:53:00,057] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:00,058] jepsen worker 81 - jepsen.util 81 :ok :read [27 15] INFO [2022-01-10 12:53:00,058] jepsen worker 74 - jepsen.util 74 :ok :write [24 42] INFO [2022-01-10 12:53:00,058] jepsen worker 85 - jepsen.util 85 :ok :write [28 0] INFO [2022-01-10 12:53:00,059] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:00,061] jepsen worker 50 - jepsen.util 50 :ok :write [16 16] INFO [2022-01-10 12:53:00,062] jepsen worker 57 - jepsen.util 57 :ok :read [19 32] INFO [2022-01-10 12:53:00,062] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,063] jepsen worker 38 - jepsen.util 38 :ok :read [12 0] INFO [2022-01-10 12:53:00,064] jepsen worker 36 - jepsen.util 36 :ok :write [12 30] INFO [2022-01-10 12:53:00,064] jepsen worker 21 - jepsen.util 21 :invoke :write [7 25] INFO [2022-01-10 12:53:00,065] jepsen worker 45 - jepsen.util 45 :ok :write [15 39] INFO [2022-01-10 12:53:00,068] jepsen worker 0 - jepsen.util 0 :invoke :write [0 38] INFO [2022-01-10 12:53:00,068] jepsen worker 2 - jepsen.util 2 :ok :write [0 30] INFO [2022-01-10 12:53:00,069] jepsen worker 67 - jepsen.util 67 :ok :write [22 45] INFO [2022-01-10 12:53:00,070] jepsen worker 53 - jepsen.util 53 :ok :write [17 36] INFO [2022-01-10 12:53:00,070] jepsen worker 81 - jepsen.util 81 :invoke :write [27 2] INFO [2022-01-10 12:53:00,073] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:00,074] jepsen worker 23 - jepsen.util 23 :ok :read [7 7] INFO [2022-01-10 12:53:00,075] jepsen worker 65 - jepsen.util 65 :invoke :write [21 1] INFO [2022-01-10 12:53:00,075] jepsen worker 76 - jepsen.util 76 :ok :write [25 34] INFO [2022-01-10 12:53:00,077] jepsen worker 13 - jepsen.util 13 :ok :write [4 27] INFO [2022-01-10 12:53:00,077] jepsen worker 68 - jepsen.util 68 :invoke :write [22 44] INFO [2022-01-10 12:53:00,079] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:00,080] jepsen worker 57 - jepsen.util 57 :ok :read [19 32] INFO [2022-01-10 12:53:00,081] jepsen worker 7 - jepsen.util 7 :ok :write [2 15] INFO [2022-01-10 12:53:00,082] jepsen worker 16 - jepsen.util 16 :invoke :write [5 17] INFO [2022-01-10 12:53:00,082] jepsen worker 71 - jepsen.util 71 :ok :write [23 42] INFO [2022-01-10 12:53:00,084] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:00,085] jepsen worker 27 - jepsen.util 27 :ok :read [9 1] INFO [2022-01-10 12:53:00,086] jepsen worker 17 - jepsen.util 17 :invoke :write [5 48] INFO [2022-01-10 12:53:00,087] jepsen worker 0 - jepsen.util 0 :ok :write [0 38] INFO [2022-01-10 12:53:00,088] jepsen worker 88 - jepsen.util 88 :invoke :read [29 nil] INFO [2022-01-10 12:53:00,089] jepsen worker 78 - jepsen.util 78 :ok :write [26 35] INFO [2022-01-10 12:53:00,089] jepsen worker 88 - jepsen.util 88 :ok :read [29 9] INFO [2022-01-10 12:53:00,090] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:00,091] jepsen worker 7 - jepsen.util 7 :ok :read [2 15] INFO [2022-01-10 12:53:00,093] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:53:00,094] jepsen worker 26 - jepsen.util 26 :ok :read [8 28] INFO [2022-01-10 12:53:00,095] jepsen worker 35 - jepsen.util 35 :ok :write [11 2] INFO [2022-01-10 12:53:00,096] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:53:00,096] jepsen worker 20 - jepsen.util 20 :ok :write [6 20] INFO [2022-01-10 12:53:00,097] jepsen worker 64 - jepsen.util 64 :ok :read [21 31] INFO [2022-01-10 12:53:00,099] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:53:00,100] jepsen worker 26 - jepsen.util 26 :ok :read [8 28] INFO [2022-01-10 12:53:00,102] jepsen worker 83 - jepsen.util 83 :ok :write [27 22] INFO [2022-01-10 12:53:00,103] jepsen worker 58 - jepsen.util 58 :ok :write [19 39] INFO [2022-01-10 12:53:00,103] jepsen worker 24 - jepsen.util 24 :ok :write [8 36] INFO [2022-01-10 12:53:00,103] jepsen worker 62 - jepsen.util 62 :invoke :write [20 6] INFO [2022-01-10 12:53:00,107] jepsen worker 57 - jepsen.util 57 :invoke :write [19 41] INFO [2022-01-10 12:53:00,108] jepsen worker 60 - jepsen.util 60 :ok :write [20 0] INFO [2022-01-10 12:53:00,110] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:53:00,112] jepsen worker 35 - jepsen.util 35 :ok :read [11 2] INFO [2022-01-10 12:53:00,113] jepsen worker 83 - jepsen.util 83 :invoke :read [27 nil] INFO [2022-01-10 12:53:00,114] jepsen worker 83 - jepsen.util 83 :ok :read [27 22] INFO [2022-01-10 12:53:00,115] jepsen worker 21 - jepsen.util 21 :ok :write [7 25] INFO [2022-01-10 12:53:00,115] jepsen worker 19 - jepsen.util 19 :invoke :write [6 49] INFO [2022-01-10 12:53:00,117] jepsen worker 57 - jepsen.util 57 :ok :write [19 41] INFO [2022-01-10 12:53:00,117] jepsen worker 46 - jepsen.util 46 :ok :write [15 25] INFO [2022-01-10 12:53:00,118] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:00,119] jepsen worker 32 - jepsen.util 32 :ok :write [10 28] INFO [2022-01-10 12:53:00,120] jepsen worker 7 - jepsen.util 7 :ok :read [2 15] INFO [2022-01-10 12:53:00,121] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:00,122] jepsen worker 75 - jepsen.util 75 :ok :read [25 34] INFO [2022-01-10 12:53:00,124] jepsen worker 57 - jepsen.util 57 :invoke :write [19 44] INFO [2022-01-10 12:53:00,126] jepsen worker 51 - jepsen.util 51 :ok :write [17 39] INFO [2022-01-10 12:53:00,126] jepsen worker 4 - jepsen.util 4 :invoke :write [1 0] INFO [2022-01-10 12:53:00,127] jepsen worker 10 - jepsen.util 10 :ok :write [3 14] INFO [2022-01-10 12:53:00,129] jepsen worker 1 - jepsen.util 1 :invoke :write [0 9] INFO [2022-01-10 12:53:00,131] jepsen worker 49 - jepsen.util 49 :ok :write [16 30] INFO [2022-01-10 12:53:00,131] jepsen worker 65 - jepsen.util 65 :ok :write [21 1] INFO [2022-01-10 12:53:00,133] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:00,134] jepsen worker 7 - jepsen.util 7 :ok :read [2 15] INFO [2022-01-10 12:53:00,137] jepsen worker 68 - jepsen.util 68 :ok :write [22 44] INFO [2022-01-10 12:53:00,137] jepsen worker 55 - jepsen.util 55 :invoke :write [18 9] INFO [2022-01-10 12:53:00,139] jepsen worker 57 - jepsen.util 57 :ok :write [19 44] INFO [2022-01-10 12:53:00,140] jepsen worker 20 - jepsen.util 20 :invoke :write [6 2] INFO [2022-01-10 12:53:00,144] jepsen worker 49 - jepsen.util 49 :invoke :read [16 nil] INFO [2022-01-10 12:53:00,145] jepsen worker 49 - jepsen.util 49 :ok :read [16 30] INFO [2022-01-10 12:53:00,147] jepsen worker 55 - jepsen.util 55 :ok :write [18 9] INFO [2022-01-10 12:53:00,147] jepsen worker 46 - jepsen.util 46 :invoke :write [15 49] INFO [2022-01-10 12:53:00,150] jepsen worker 71 - jepsen.util 71 :invoke :write [23 44] INFO [2022-01-10 12:53:00,152] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:53:00,153] jepsen worker 14 - jepsen.util 14 :ok :read [4 27] INFO [2022-01-10 12:53:00,154] jepsen worker 16 - jepsen.util 16 :ok :write [5 17] INFO [2022-01-10 12:53:00,155] jepsen worker 19 - jepsen.util 19 :ok :write [6 49] INFO [2022-01-10 12:53:00,155] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:00,156] jepsen worker 74 - jepsen.util 74 :ok :read [24 42] INFO [2022-01-10 12:53:00,158] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:00,159] jepsen worker 78 - jepsen.util 78 :ok :read [26 35] INFO [2022-01-10 12:53:00,160] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:53:00,161] jepsen worker 19 - jepsen.util 19 :ok :read [6 49] INFO [2022-01-10 12:53:00,163] jepsen worker 70 - jepsen.util 70 :invoke :write [23 35] INFO [2022-01-10 12:53:00,164] jepsen worker 34 - jepsen.util 34 :ok :write [11 49] INFO [2022-01-10 12:53:00,166] jepsen worker 23 - jepsen.util 23 :invoke :write [7 32] INFO [2022-01-10 12:53:00,168] jepsen worker 16 - jepsen.util 16 :invoke :write [5 10] INFO [2022-01-10 12:53:00,169] jepsen worker 46 - jepsen.util 46 :ok :write [15 49] INFO [2022-01-10 12:53:00,169] jepsen worker 81 - jepsen.util 81 :ok :write [27 2] INFO [2022-01-10 12:53:00,171] jepsen worker 58 - jepsen.util 58 :invoke :write [19 13] INFO [2022-01-10 12:53:00,174] jepsen worker 83 - jepsen.util 83 :invoke :write [27 24] INFO [2022-01-10 12:53:00,177] jepsen worker 62 - jepsen.util 62 :ok :write [20 6] INFO [2022-01-10 12:53:00,177] jepsen worker 75 - jepsen.util 75 :invoke :write [25 0] INFO [2022-01-10 12:53:00,181] jepsen worker 3 - jepsen.util 3 :invoke :write [1 14] INFO [2022-01-10 12:53:00,183] jepsen worker 84 - jepsen.util 84 :invoke :write [28 17] INFO [2022-01-10 12:53:00,185] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:53:00,186] jepsen worker 35 - jepsen.util 35 :ok :read [11 49] INFO [2022-01-10 12:53:00,187] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:53:00,188] jepsen worker 9 - jepsen.util 9 :ok :read [3 14] INFO [2022-01-10 12:53:00,190] jepsen worker 26 - jepsen.util 26 :invoke :write [8 45] INFO [2022-01-10 12:53:00,193] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,194] jepsen worker 41 - jepsen.util 41 :ok :read [13 10] INFO [2022-01-10 12:53:00,195] jepsen worker 85 - jepsen.util 85 :invoke :write [28 14] INFO [2022-01-10 12:53:00,197] jepsen worker 38 - jepsen.util 38 :invoke :write [12 22] INFO [2022-01-10 12:53:00,199] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:00,200] jepsen worker 44 - jepsen.util 44 :ok :read [14 11] INFO [2022-01-10 12:53:00,201] jepsen worker 35 - jepsen.util 35 :invoke :write [11 6] INFO [2022-01-10 12:53:00,204] jepsen worker 74 - jepsen.util 74 :invoke :write [24 47] INFO [2022-01-10 12:53:00,205] jepsen worker 23 - jepsen.util 23 :ok :write [7 32] INFO [2022-01-10 12:53:00,206] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:00,207] jepsen worker 47 - jepsen.util 47 :ok :read [15 49] INFO [2022-01-10 12:53:00,208] jepsen worker 71 - jepsen.util 71 :ok :write [23 44] INFO [2022-01-10 12:53:00,208] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:00,209] jepsen worker 87 - jepsen.util 87 :ok :read [29 9] INFO [2022-01-10 12:53:00,210] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,211] jepsen worker 37 - jepsen.util 37 :ok :read [12 30] INFO [2022-01-10 12:53:00,212] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,214] jepsen worker 62 - jepsen.util 62 :ok :read [20 6] INFO [2022-01-10 12:53:00,215] jepsen worker 80 - jepsen.util 80 :ok :write [26 26] INFO [2022-01-10 12:53:00,215] jepsen worker 77 - jepsen.util 77 :invoke :read [25 nil] INFO [2022-01-10 12:53:00,216] jepsen worker 77 - jepsen.util 77 :ok :read [25 34] INFO [2022-01-10 12:53:00,217] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:00,218] jepsen worker 71 - jepsen.util 71 :ok :read [23 44] INFO [2022-01-10 12:53:00,219] jepsen worker 31 - jepsen.util 31 :invoke :write [10 9] INFO [2022-01-10 12:53:00,220] jepsen worker 20 - jepsen.util 20 :ok :write [6 2] INFO [2022-01-10 12:53:00,221] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,223] jepsen worker 39 - jepsen.util 39 :ok :read [13 10] INFO [2022-01-10 12:53:00,224] jepsen worker 19 - jepsen.util 19 :invoke :write [6 24] INFO [2022-01-10 12:53:00,224] jepsen worker 52 - jepsen.util 52 :ok :write [17 44] INFO [2022-01-10 12:53:00,226] jepsen worker 21 - jepsen.util 21 :invoke :write [7 46] INFO [2022-01-10 12:53:00,228] jepsen worker 61 - jepsen.util 61 :invoke :write [20 25] INFO [2022-01-10 12:53:00,230] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:00,232] jepsen worker 81 - jepsen.util 81 :ok :read [27 2] INFO [2022-01-10 12:53:00,233] jepsen worker 20 - jepsen.util 20 :invoke :write [6 13] INFO [2022-01-10 12:53:00,234] jepsen worker 26 - jepsen.util 26 :ok :write [8 45] INFO [2022-01-10 12:53:00,235] jepsen worker 24 - jepsen.util 24 :invoke :write [8 36] INFO [2022-01-10 12:53:00,237] jepsen worker 32 - jepsen.util 32 :invoke :read [10 nil] INFO [2022-01-10 12:53:00,238] jepsen worker 32 - jepsen.util 32 :ok :read [10 28] INFO [2022-01-10 12:53:00,239] jepsen worker 34 - jepsen.util 34 :invoke :write [11 36] INFO [2022-01-10 12:53:00,241] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:00,242] jepsen worker 69 - jepsen.util 69 :ok :read [23 44] INFO [2022-01-10 12:53:00,243] jepsen worker 73 - jepsen.util 73 :invoke :write [24 34] INFO [2022-01-10 12:53:00,245] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,246] jepsen worker 74 - jepsen.util 74 :ok :write [24 47] INFO [2022-01-10 12:53:00,246] jepsen worker 36 - jepsen.util 36 :ok :read [12 30] INFO [2022-01-10 12:53:00,247] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,248] jepsen worker 37 - jepsen.util 37 :ok :read [12 30] INFO [2022-01-10 12:53:00,250] jepsen worker 30 - jepsen.util 30 :invoke :write [10 47] INFO [2022-01-10 12:53:00,252] jepsen worker 18 - jepsen.util 18 :invoke :write [6 22] INFO [2022-01-10 12:53:00,254] jepsen worker 53 - jepsen.util 53 :invoke :write [17 5] INFO [2022-01-10 12:53:00,256] jepsen worker 14 - jepsen.util 14 :invoke :write [4 40] INFO [2022-01-10 12:53:00,258] jepsen worker 9 - jepsen.util 9 :invoke :write [3 23] INFO [2022-01-10 12:53:00,260] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,261] jepsen worker 39 - jepsen.util 39 :ok :read [13 10] INFO [2022-01-10 12:53:00,262] jepsen worker 57 - jepsen.util 57 :invoke :write [19 29] INFO [2022-01-10 12:53:00,264] jepsen worker 89 - jepsen.util 89 :invoke :write [29 13] INFO [2022-01-10 12:53:00,266] jepsen worker 79 - jepsen.util 79 :invoke :read [26 nil] INFO [2022-01-10 12:53:00,267] jepsen worker 79 - jepsen.util 79 :ok :read [26 26] INFO [2022-01-10 12:53:00,268] jepsen worker 27 - jepsen.util 27 :invoke :write [9 20] INFO [2022-01-10 12:53:00,270] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,271] jepsen worker 62 - jepsen.util 62 :ok :read [20 6] INFO [2022-01-10 12:53:00,272] jepsen worker 2 - jepsen.util 2 :invoke :write [0 18] INFO [2022-01-10 12:53:00,274] jepsen worker 37 - jepsen.util 37 :invoke :write [12 31] INFO [2022-01-10 12:53:00,276] jepsen worker 10 - jepsen.util 10 :invoke :read [3 nil] INFO [2022-01-10 12:53:00,277] jepsen worker 10 - jepsen.util 10 :ok :read [3 14] INFO [2022-01-10 12:53:00,278] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:53:00,279] jepsen worker 54 - jepsen.util 54 :ok :read [18 9] INFO [2022-01-10 12:53:00,280] jepsen worker 81 - jepsen.util 81 :invoke :write [27 6] INFO [2022-01-10 12:53:00,282] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:00,283] jepsen worker 29 - jepsen.util 29 :ok :read [9 1] INFO [2022-01-10 12:53:00,284] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,285] jepsen worker 60 - jepsen.util 60 :ok :read [20 6] INFO [2022-01-10 12:53:00,286] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:00,287] jepsen worker 52 - jepsen.util 52 :ok :read [17 44] INFO [2022-01-10 12:53:00,288] jepsen worker 39 - jepsen.util 39 :invoke :write [13 17] INFO [2022-01-10 12:53:00,290] jepsen worker 10 - jepsen.util 10 :invoke :read [3 nil] INFO [2022-01-10 12:53:00,291] jepsen worker 10 - jepsen.util 10 :ok :read [3 14] INFO [2022-01-10 12:53:00,292] jepsen worker 41 - jepsen.util 41 :invoke :write [13 43] INFO [2022-01-10 12:53:00,294] jepsen worker 10 - jepsen.util 10 :invoke :write [3 25] INFO [2022-01-10 12:53:00,296] jepsen worker 59 - jepsen.util 59 :invoke :write [19 18] INFO [2022-01-10 12:53:00,297] jepsen worker 6 - jepsen.util 6 :ok :write [2 11] INFO [2022-01-10 12:53:00,298] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:00,299] jepsen worker 23 - jepsen.util 23 :ok :read [7 32] INFO [2022-01-10 12:53:00,300] jepsen worker 4 - jepsen.util 4 :ok :write [1 0] INFO [2022-01-10 12:53:00,300] jepsen worker 32 - jepsen.util 32 :invoke :write [10 0] INFO [2022-01-10 12:53:00,302] jepsen worker 48 - jepsen.util 48 :ok :write [16 41] INFO [2022-01-10 12:53:00,303] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:00,303] jepsen worker 74 - jepsen.util 74 :ok :read [24 47] INFO [2022-01-10 12:53:00,304] jepsen worker 58 - jepsen.util 58 :ok :write [19 13] INFO [2022-01-10 12:53:00,305] jepsen worker 29 - jepsen.util 29 :invoke :write [9 38] INFO [2022-01-10 12:53:00,307] jepsen worker 88 - jepsen.util 88 :invoke :write [29 21] INFO [2022-01-10 12:53:00,309] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:53:00,310] jepsen worker 86 - jepsen.util 86 :ok :write [28 44] INFO [2022-01-10 12:53:00,310] jepsen worker 64 - jepsen.util 64 :ok :read [21 1] INFO [2022-01-10 12:53:00,311] jepsen worker 1 - jepsen.util 1 :ok :write [0 9] INFO [2022-01-10 12:53:00,311] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,312] jepsen worker 40 - jepsen.util 40 :ok :read [13 10] INFO [2022-01-10 12:53:00,313] jepsen worker 75 - jepsen.util 75 :ok :write [25 0] INFO [2022-01-10 12:53:00,313] jepsen worker 78 - jepsen.util 78 :invoke :write [26 32] INFO [2022-01-10 12:53:00,314] jepsen worker 70 - jepsen.util 70 :ok :write [23 35] INFO [2022-01-10 12:53:00,316] jepsen worker 74 - jepsen.util 74 :invoke :write [24 19] INFO [2022-01-10 12:53:00,317] jepsen worker 38 - jepsen.util 38 :ok :write [12 22] INFO [2022-01-10 12:53:00,317] jepsen worker 21 - jepsen.util 21 :ok :write [7 46] INFO [2022-01-10 12:53:00,317] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:00,319] jepsen worker 23 - jepsen.util 23 :ok :read [7 46] INFO [2022-01-10 12:53:00,319] jepsen worker 86 - jepsen.util 86 :invoke :write [28 10] INFO [2022-01-10 12:53:00,320] jepsen worker 17 - jepsen.util 17 :ok :write [5 48] INFO [2022-01-10 12:53:00,321] jepsen worker 67 - jepsen.util 67 :invoke :write [22 22] INFO [2022-01-10 12:53:00,322] jepsen worker 61 - jepsen.util 61 :ok :write [20 25] INFO [2022-01-10 12:53:00,323] jepsen worker 46 - jepsen.util 46 :invoke :write [15 20] INFO [2022-01-10 12:53:00,325] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:53:00,326] jepsen worker 45 - jepsen.util 45 :ok :read [15 49] INFO [2022-01-10 12:53:00,327] jepsen worker 35 - jepsen.util 35 :ok :write [11 6] INFO [2022-01-10 12:53:00,327] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,328] jepsen worker 4 - jepsen.util 4 :ok :read [1 0] INFO [2022-01-10 12:53:00,329] jepsen worker 53 - jepsen.util 53 :ok :write [17 5] INFO [2022-01-10 12:53:00,330] jepsen worker 21 - jepsen.util 21 :invoke :write [7 24] INFO [2022-01-10 12:53:00,330] jepsen worker 83 - jepsen.util 83 :ok :write [27 24] INFO [2022-01-10 12:53:00,332] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,333] jepsen worker 4 - jepsen.util 4 :ok :read [1 0] INFO [2022-01-10 12:53:00,334] jepsen worker 79 - jepsen.util 79 :invoke :write [26 14] INFO [2022-01-10 12:53:00,334] jepsen worker 14 - jepsen.util 14 :ok :write [4 40] INFO [2022-01-10 12:53:00,336] jepsen worker 40 - jepsen.util 40 :invoke :write [13 21] INFO [2022-01-10 12:53:00,337] jepsen worker 89 - jepsen.util 89 :ok :write [29 13] INFO [2022-01-10 12:53:00,338] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:00,339] jepsen worker 68 - jepsen.util 68 :ok :read [22 44] INFO [2022-01-10 12:53:00,340] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,341] jepsen worker 61 - jepsen.util 61 :ok :read [20 25] INFO [2022-01-10 12:53:00,343] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:00,344] jepsen worker 39 - jepsen.util 39 :ok :write [13 17] INFO [2022-01-10 12:53:00,345] jepsen worker 27 - jepsen.util 27 :ok :write [9 20] INFO [2022-01-10 12:53:00,345] jepsen worker 47 - jepsen.util 47 :ok :read [15 49] INFO [2022-01-10 12:53:00,346] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:00,349] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,350] jepsen worker 4 - jepsen.util 4 :ok :read [1 0] INFO [2022-01-10 12:53:00,350] jepsen worker 50 - jepsen.util 50 :ok :read [16 41] INFO [2022-01-10 12:53:00,351] jepsen worker 31 - jepsen.util 31 :ok :write [10 9] INFO [2022-01-10 12:53:00,352] jepsen worker 17 - jepsen.util 17 :invoke :write [5 36] INFO [2022-01-10 12:53:00,355] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,355] jepsen worker 86 - jepsen.util 86 :ok :write [28 10] INFO [2022-01-10 12:53:00,356] jepsen worker 36 - jepsen.util 36 :ok :read [12 22] INFO [2022-01-10 12:53:00,357] jepsen worker 31 - jepsen.util 31 :invoke :write [10 4] INFO [2022-01-10 12:53:00,359] jepsen worker 1 - jepsen.util 1 :invoke :write [0 49] INFO [2022-01-10 12:53:00,359] jepsen worker 19 - jepsen.util 19 :ok :write [6 24] INFO [2022-01-10 12:53:00,360] jepsen worker 2 - jepsen.util 2 :ok :write [0 18] INFO [2022-01-10 12:53:00,361] jepsen worker 80 - jepsen.util 80 :invoke :read [26 nil] INFO [2022-01-10 12:53:00,362] jepsen worker 80 - jepsen.util 80 :ok :read [26 26] INFO [2022-01-10 12:53:00,364] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:00,364] jepsen worker 78 - jepsen.util 78 :ok :write [26 32] INFO [2022-01-10 12:53:00,365] jepsen worker 51 - jepsen.util 51 :ok :read [17 5] INFO [2022-01-10 12:53:00,365] jepsen worker 24 - jepsen.util 24 :ok :write [8 36] INFO [2022-01-10 12:53:00,366] jepsen worker 64 - jepsen.util 64 :invoke :write [21 49] INFO [2022-01-10 12:53:00,368] jepsen worker 60 - jepsen.util 60 :invoke :write [20 8] INFO [2022-01-10 12:53:00,370] jepsen worker 73 - jepsen.util 73 :ok :write [24 34] INFO [2022-01-10 12:53:00,370] jepsen worker 37 - jepsen.util 37 :ok :write [12 31] INFO [2022-01-10 12:53:00,370] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:00,371] jepsen worker 75 - jepsen.util 75 :ok :read [25 0] INFO [2022-01-10 12:53:00,373] jepsen worker 11 - jepsen.util 11 :invoke :write [3 38] INFO [2022-01-10 12:53:00,375] jepsen worker 19 - jepsen.util 19 :invoke :write [6 34] INFO [2022-01-10 12:53:00,375] jepsen worker 9 - jepsen.util 9 :ok :write [3 23] INFO [2022-01-10 12:53:00,377] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:53:00,378] jepsen worker 73 - jepsen.util 73 :ok :read [24 34] INFO [2022-01-10 12:53:00,379] jepsen worker 52 - jepsen.util 52 :invoke :write [17 48] INFO [2022-01-10 12:53:00,381] jepsen worker 50 - jepsen.util 50 :invoke :write [16 2] INFO [2022-01-10 12:53:00,382] jepsen worker 59 - jepsen.util 59 :ok :write [19 18] INFO [2022-01-10 12:53:00,383] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,384] jepsen worker 61 - jepsen.util 61 :ok :read [20 25] INFO [2022-01-10 12:53:00,386] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,386] jepsen worker 88 - jepsen.util 88 :ok :write [29 21] INFO [2022-01-10 12:53:00,387] jepsen worker 46 - jepsen.util 46 :ok :write [15 20] INFO [2022-01-10 12:53:00,388] jepsen worker 54 - jepsen.util 54 :invoke :write [18 39] INFO [2022-01-10 12:53:00,388] jepsen worker 36 - jepsen.util 36 :ok :read [12 31] INFO [2022-01-10 12:53:00,390] jepsen worker 40 - jepsen.util 40 :ok :write [13 21] INFO [2022-01-10 12:53:00,390] jepsen worker 39 - jepsen.util 39 :invoke :write [13 19] INFO [2022-01-10 12:53:00,392] jepsen worker 23 - jepsen.util 23 :invoke :write [7 28] INFO [2022-01-10 12:53:00,394] jepsen worker 84 - jepsen.util 84 :ok :write [28 17] INFO [2022-01-10 12:53:00,394] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:00,395] jepsen worker 30 - jepsen.util 30 :ok :write [10 47] INFO [2022-01-10 12:53:00,395] jepsen worker 71 - jepsen.util 71 :ok :read [23 35] INFO [2022-01-10 12:53:00,396] jepsen worker 29 - jepsen.util 29 :ok :write [9 38] INFO [2022-01-10 12:53:00,396] jepsen worker 25 - jepsen.util 25 :invoke :write [8 20] INFO [2022-01-10 12:53:00,398] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:53:00,399] jepsen worker 3 - jepsen.util 3 :ok :write [1 14] INFO [2022-01-10 12:53:00,400] jepsen worker 73 - jepsen.util 73 :ok :read [24 34] INFO [2022-01-10 12:53:00,400] jepsen worker 54 - jepsen.util 54 :ok :write [18 39] INFO [2022-01-10 12:53:00,401] jepsen worker 1 - jepsen.util 1 :ok :write [0 49] INFO [2022-01-10 12:53:00,401] jepsen worker 7 - jepsen.util 7 :invoke :write [2 19] INFO [2022-01-10 12:53:00,403] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 12:53:00,405] jepsen worker 6 - jepsen.util 6 :ok :read [2 11] INFO [2022-01-10 12:53:00,405] jepsen worker 88 - jepsen.util 88 :invoke :write [29 16] INFO [2022-01-10 12:53:00,408] jepsen worker 45 - jepsen.util 45 :invoke :write [15 43] INFO [2022-01-10 12:53:00,410] jepsen worker 43 - jepsen.util 43 :invoke :write [14 24] INFO [2022-01-10 12:53:00,412] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,412] jepsen worker 39 - jepsen.util 39 :ok :write [13 19] INFO [2022-01-10 12:53:00,413] jepsen worker 40 - jepsen.util 40 :ok :read [13 19] INFO [2022-01-10 12:53:00,414] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:53:00,415] jepsen worker 73 - jepsen.util 73 :ok :read [24 34] INFO [2022-01-10 12:53:00,416] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,417] jepsen worker 3 - jepsen.util 3 :ok :read [1 14] INFO [2022-01-10 12:53:00,418] jepsen worker 2 - jepsen.util 2 :invoke :write [0 4] INFO [2022-01-10 12:53:00,421] jepsen worker 66 - jepsen.util 66 :invoke :write [22 36] INFO [2022-01-10 12:53:00,423] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:00,424] jepsen worker 47 - jepsen.util 47 :ok :read [15 20] INFO [2022-01-10 12:53:00,425] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,427] jepsen worker 62 - jepsen.util 62 :ok :read [20 25] INFO [2022-01-10 12:53:00,428] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:00,429] jepsen worker 13 - jepsen.util 13 :ok :read [4 40] INFO [2022-01-10 12:53:00,429] jepsen worker 88 - jepsen.util 88 :ok :write [29 16] INFO [2022-01-10 12:53:00,430] jepsen worker 70 - jepsen.util 70 :invoke :write [23 19] INFO [2022-01-10 12:53:00,432] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:53:00,433] jepsen worker 9 - jepsen.util 9 :ok :read [3 23] INFO [2022-01-10 12:53:00,434] jepsen worker 86 - jepsen.util 86 :invoke :write [28 2] INFO [2022-01-10 12:53:00,436] jepsen worker 2 - jepsen.util 2 :ok :write [0 4] INFO [2022-01-10 12:53:00,437] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:53:00,438] jepsen worker 24 - jepsen.util 24 :ok :read [8 36] INFO [2022-01-10 12:53:00,439] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:53:00,440] jepsen worker 9 - jepsen.util 9 :ok :read [3 23] INFO [2022-01-10 12:53:00,441] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:53:00,442] jepsen worker 35 - jepsen.util 35 :ok :read [11 6] INFO [2022-01-10 12:53:00,443] jepsen worker 65 - jepsen.util 65 :invoke :write [21 41] INFO [2022-01-10 12:53:00,445] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:00,446] jepsen worker 86 - jepsen.util 86 :ok :write [28 2] INFO [2022-01-10 12:53:00,447] jepsen worker 78 - jepsen.util 78 :ok :read [26 32] INFO [2022-01-10 12:53:00,448] jepsen worker 39 - jepsen.util 39 :invoke :write [13 34] INFO [2022-01-10 12:53:00,450] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 12:53:00,451] jepsen worker 42 - jepsen.util 42 :ok :read [14 11] INFO [2022-01-10 12:53:00,452] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:00,453] jepsen worker 68 - jepsen.util 68 :ok :read [22 44] INFO [2022-01-10 12:53:00,455] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,456] jepsen worker 39 - jepsen.util 39 :ok :write [13 34] INFO [2022-01-10 12:53:00,456] jepsen worker 4 - jepsen.util 4 :ok :read [1 14] INFO [2022-01-10 12:53:00,457] jepsen worker 67 - jepsen.util 67 :ok :write [22 22] INFO [2022-01-10 12:53:00,458] jepsen worker 28 - jepsen.util 28 :invoke :write [9 25] INFO [2022-01-10 12:53:00,461] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:00,462] jepsen worker 55 - jepsen.util 55 :ok :read [18 39] INFO [2022-01-10 12:53:00,464] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,465] jepsen worker 37 - jepsen.util 37 :ok :read [12 31] INFO [2022-01-10 12:53:00,465] jepsen worker 21 - jepsen.util 21 :ok :write [7 24] INFO [2022-01-10 12:53:00,466] jepsen worker 89 - jepsen.util 89 :invoke :write [29 24] INFO [2022-01-10 12:53:00,469] jepsen worker 75 - jepsen.util 75 :invoke :write [25 12] INFO [2022-01-10 12:53:00,471] jepsen worker 55 - jepsen.util 55 :invoke :write [18 14] INFO [2022-01-10 12:53:00,472] jepsen worker 81 - jepsen.util 81 :ok :write [27 6] INFO [2022-01-10 12:53:00,473] jepsen worker 79 - jepsen.util 79 :ok :write [26 14] INFO [2022-01-10 12:53:00,474] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:53:00,475] jepsen worker 14 - jepsen.util 14 :ok :read [4 40] INFO [2022-01-10 12:53:00,476] jepsen worker 13 - jepsen.util 13 :invoke :write [4 45] INFO [2022-01-10 12:53:00,476] jepsen worker 89 - jepsen.util 89 :ok :write [29 24] INFO [2022-01-10 12:53:00,478] jepsen worker 34 - jepsen.util 34 :ok :write [11 36] INFO [2022-01-10 12:53:00,479] jepsen worker 48 - jepsen.util 48 :invoke :write [16 17] INFO [2022-01-10 12:53:00,481] jepsen worker 40 - jepsen.util 40 :invoke :write [13 29] INFO [2022-01-10 12:53:00,483] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,484] jepsen worker 62 - jepsen.util 62 :ok :read [20 25] INFO [2022-01-10 12:53:00,485] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:53:00,486] jepsen worker 16 - jepsen.util 16 :ok :write [5 10] INFO [2022-01-10 12:53:00,486] jepsen worker 63 - jepsen.util 63 :ok :read [21 1] INFO [2022-01-10 12:53:00,488] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,489] jepsen worker 3 - jepsen.util 3 :ok :read [1 14] INFO [2022-01-10 12:53:00,491] jepsen worker 60 - jepsen.util 60 :ok :write [20 8] INFO [2022-01-10 12:53:00,491] jepsen worker 16 - jepsen.util 16 :invoke :write [5 35] INFO [2022-01-10 12:53:00,493] jepsen worker 52 - jepsen.util 52 :ok :write [17 48] INFO [2022-01-10 12:53:00,494] jepsen worker 58 - jepsen.util 58 :invoke :write [19 18] INFO [2022-01-10 12:53:00,496] jepsen worker 49 - jepsen.util 49 :invoke :read [16 nil] INFO [2022-01-10 12:53:00,497] jepsen worker 50 - jepsen.util 50 :ok :write [16 2] INFO [2022-01-10 12:53:00,498] jepsen worker 49 - jepsen.util 49 :ok :read [16 2] INFO [2022-01-10 12:53:00,498] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:00,499] jepsen worker 76 - jepsen.util 76 :ok :read [25 0] INFO [2022-01-10 12:53:00,500] jepsen worker 19 - jepsen.util 19 :ok :write [6 34] INFO [2022-01-10 12:53:00,500] jepsen worker 9 - jepsen.util 9 :invoke :write [3 42] INFO [2022-01-10 12:53:00,503] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 12:53:00,503] jepsen worker 41 - jepsen.util 41 :ok :write [13 43] INFO [2022-01-10 12:53:00,503] jepsen worker 70 - jepsen.util 70 :ok :write [23 19] INFO [2022-01-10 12:53:00,504] jepsen worker 42 - jepsen.util 42 :ok :read [14 11] INFO [2022-01-10 12:53:00,505] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:00,505] jepsen worker 7 - jepsen.util 7 :ok :write [2 19] INFO [2022-01-10 12:53:00,506] jepsen worker 81 - jepsen.util 81 :ok :read [27 6] INFO [2022-01-10 12:53:00,507] jepsen worker 25 - jepsen.util 25 :ok :write [8 20] INFO [2022-01-10 12:53:00,509] jepsen worker 55 - jepsen.util 55 :ok :write [18 14] INFO [2022-01-10 12:53:00,509] jepsen worker 53 - jepsen.util 53 :invoke :write [17 4] INFO [2022-01-10 12:53:00,509] jepsen worker 85 - jepsen.util 85 :ok :write [28 14] INFO [2022-01-10 12:53:00,512] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:00,513] jepsen worker 43 - jepsen.util 43 :ok :write [14 24] INFO [2022-01-10 12:53:00,513] jepsen worker 34 - jepsen.util 34 :ok :read [11 36] INFO [2022-01-10 12:53:00,513] jepsen worker 28 - jepsen.util 28 :ok :write [9 25] INFO [2022-01-10 12:53:00,515] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:53:00,516] jepsen worker 64 - jepsen.util 64 :ok :write [21 49] INFO [2022-01-10 12:53:00,517] jepsen worker 63 - jepsen.util 63 :ok :read [21 49] INFO [2022-01-10 12:53:00,518] jepsen worker 13 - jepsen.util 13 :ok :write [4 45] INFO [2022-01-10 12:53:00,519] jepsen worker 12 - jepsen.util 12 :invoke :write [4 33] INFO [2022-01-10 12:53:00,520] jepsen worker 40 - jepsen.util 40 :ok :write [13 29] INFO [2022-01-10 12:53:00,522] jepsen worker 66 - jepsen.util 66 :ok :write [22 36] INFO [2022-01-10 12:53:00,523] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,523] jepsen worker 74 - jepsen.util 74 :ok :write [24 19] INFO [2022-01-10 12:53:00,524] jepsen worker 4 - jepsen.util 4 :ok :read [1 14] INFO [2022-01-10 12:53:00,525] jepsen worker 53 - jepsen.util 53 :ok :write [17 4] INFO [2022-01-10 12:53:00,526] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:00,527] jepsen worker 27 - jepsen.util 27 :ok :read [9 25] INFO [2022-01-10 12:53:00,529] jepsen worker 23 - jepsen.util 23 :ok :write [7 28] INFO [2022-01-10 12:53:00,529] jepsen worker 11 - jepsen.util 11 :ok :write [3 38] INFO [2022-01-10 12:53:00,530] jepsen worker 41 - jepsen.util 41 :invoke :write [13 4] INFO [2022-01-10 12:53:00,532] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:53:00,533] jepsen worker 63 - jepsen.util 63 :ok :read [21 49] INFO [2022-01-10 12:53:00,534] jepsen worker 75 - jepsen.util 75 :ok :write [25 12] INFO [2022-01-10 12:53:00,535] jepsen worker 85 - jepsen.util 85 :invoke :write [28 27] INFO [2022-01-10 12:53:00,535] jepsen worker 57 - jepsen.util 57 :ok :write [19 29] INFO [2022-01-10 12:53:00,537] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:53:00,538] jepsen worker 35 - jepsen.util 35 :ok :read [11 36] INFO [2022-01-10 12:53:00,540] jepsen worker 49 - jepsen.util 49 :invoke :read [16 nil] INFO [2022-01-10 12:53:00,540] jepsen worker 45 - jepsen.util 45 :ok :write [15 43] INFO [2022-01-10 12:53:00,540] jepsen worker 49 - jepsen.util 49 :ok :read [16 2] INFO [2022-01-10 12:53:00,542] jepsen worker 46 - jepsen.util 46 :invoke :write [15 21] INFO [2022-01-10 12:53:00,544] jepsen worker 86 - jepsen.util 86 :invoke :write [28 5] INFO [2022-01-10 12:53:00,546] jepsen worker 31 - jepsen.util 31 :ok :write [10 4] INFO [2022-01-10 12:53:00,546] jepsen worker 64 - jepsen.util 64 :invoke :write [21 17] INFO [2022-01-10 12:53:00,549] jepsen worker 89 - jepsen.util 89 :invoke :write [29 39] INFO [2022-01-10 12:53:00,551] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:00,552] jepsen worker 7 - jepsen.util 7 :ok :read [2 19] INFO [2022-01-10 12:53:00,553] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:00,554] jepsen worker 23 - jepsen.util 23 :ok :read [7 28] INFO [2022-01-10 12:53:00,556] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:53:00,557] jepsen worker 45 - jepsen.util 45 :ok :read [15 43] INFO [2022-01-10 12:53:00,558] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:00,559] jepsen worker 74 - jepsen.util 74 :ok :read [24 19] INFO [2022-01-10 12:53:00,560] jepsen worker 16 - jepsen.util 16 :ok :write [5 35] INFO [2022-01-10 12:53:00,561] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,562] jepsen worker 36 - jepsen.util 36 :ok :read [12 31] INFO [2022-01-10 12:53:00,563] jepsen worker 12 - jepsen.util 12 :ok :write [4 33] INFO [2022-01-10 12:53:00,563] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:00,564] jepsen worker 55 - jepsen.util 55 :ok :read [18 14] INFO [2022-01-10 12:53:00,566] jepsen worker 45 - jepsen.util 45 :invoke :write [15 25] INFO [2022-01-10 12:53:00,568] jepsen worker 11 - jepsen.util 11 :invoke :write [3 43] INFO [2022-01-10 12:53:00,570] jepsen worker 51 - jepsen.util 51 :invoke :write [17 5] INFO [2022-01-10 12:53:00,573] jepsen worker 44 - jepsen.util 44 :invoke :write [14 19] INFO [2022-01-10 12:53:00,575] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:53:00,576] jepsen worker 14 - jepsen.util 14 :ok :read [4 33] INFO [2022-01-10 12:53:00,577] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:00,578] jepsen worker 48 - jepsen.util 48 :ok :write [16 17] INFO [2022-01-10 12:53:00,579] jepsen worker 23 - jepsen.util 23 :ok :read [7 28] INFO [2022-01-10 12:53:00,580] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:53:00,581] jepsen worker 53 - jepsen.util 53 :ok :read [17 4] INFO [2022-01-10 12:53:00,583] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:00,585] jepsen worker 81 - jepsen.util 81 :ok :read [27 6] INFO [2022-01-10 12:53:00,587] jepsen worker 16 - jepsen.util 16 :invoke :write [5 28] INFO [2022-01-10 12:53:00,589] jepsen worker 52 - jepsen.util 52 :invoke :write [17 27] INFO [2022-01-10 12:53:00,592] jepsen worker 38 - jepsen.util 38 :invoke :write [12 32] INFO [2022-01-10 12:53:00,593] jepsen worker 46 - jepsen.util 46 :ok :write [15 21] INFO [2022-01-10 12:53:00,593] jepsen worker 20 - jepsen.util 20 :ok :write [6 13] INFO [2022-01-10 12:53:00,596] jepsen worker 84 - jepsen.util 84 :invoke :write [28 20] INFO [2022-01-10 12:53:00,596] jepsen worker 64 - jepsen.util 64 :ok :write [21 17] INFO [2022-01-10 12:53:00,598] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:00,599] jepsen worker 55 - jepsen.util 55 :ok :read [18 14] INFO [2022-01-10 12:53:00,600] jepsen worker 61 - jepsen.util 61 :invoke :write [20 24] INFO [2022-01-10 12:53:00,603] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:00,604] jepsen worker 13 - jepsen.util 13 :ok :read [4 33] INFO [2022-01-10 12:53:00,605] jepsen worker 36 - jepsen.util 36 :invoke :write [12 35] INFO [2022-01-10 12:53:00,608] jepsen worker 63 - jepsen.util 63 :invoke :write [21 23] INFO [2022-01-10 12:53:00,611] jepsen worker 13 - jepsen.util 13 :invoke :write [4 13] INFO [2022-01-10 12:53:00,613] jepsen worker 42 - jepsen.util 42 :invoke :read [14 nil] INFO [2022-01-10 12:53:00,615] jepsen worker 42 - jepsen.util 42 :ok :read [14 24] INFO [2022-01-10 12:53:00,615] jepsen worker 6 - jepsen.util 6 :invoke :read [2 nil] INFO [2022-01-10 12:53:00,617] jepsen worker 6 - jepsen.util 6 :ok :read [2 19] INFO [2022-01-10 12:53:00,618] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:00,619] jepsen worker 30 - jepsen.util 30 :ok :read [10 4] INFO [2022-01-10 12:53:00,620] jepsen worker 42 - jepsen.util 42 :invoke :write [14 49] INFO [2022-01-10 12:53:00,620] jepsen worker 45 - jepsen.util 45 :ok :write [15 25] INFO [2022-01-10 12:53:00,622] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,623] jepsen worker 5 - jepsen.util 5 :ok :read [1 14] INFO [2022-01-10 12:53:00,624] jepsen worker 53 - jepsen.util 53 :invoke :write [17 29] INFO [2022-01-10 12:53:00,627] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:00,628] jepsen worker 23 - jepsen.util 23 :ok :read [7 28] INFO [2022-01-10 12:53:00,629] jepsen worker 25 - jepsen.util 25 :invoke :write [8 18] INFO [2022-01-10 12:53:00,631] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:00,632] jepsen worker 81 - jepsen.util 81 :ok :read [27 6] INFO [2022-01-10 12:53:00,632] jepsen worker 63 - jepsen.util 63 :ok :write [21 23] INFO [2022-01-10 12:53:00,633] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,634] jepsen worker 37 - jepsen.util 37 :ok :read [12 31] INFO [2022-01-10 12:53:00,636] jepsen worker 83 - jepsen.util 83 :invoke :write [27 44] INFO [2022-01-10 12:53:00,639] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:53:00,640] jepsen worker 14 - jepsen.util 14 :ok :read [4 33] INFO [2022-01-10 12:53:00,641] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:00,642] jepsen worker 30 - jepsen.util 30 :ok :read [10 4] INFO [2022-01-10 12:53:00,643] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:00,644] jepsen worker 34 - jepsen.util 34 :ok :read [11 36] INFO [2022-01-10 12:53:00,645] jepsen worker 87 - jepsen.util 87 :invoke :write [29 19] INFO [2022-01-10 12:53:00,647] jepsen worker 25 - jepsen.util 25 :ok :write [8 18] INFO [2022-01-10 12:53:00,647] jepsen worker 73 - jepsen.util 73 :invoke :read [24 nil] INFO [2022-01-10 12:53:00,648] jepsen worker 73 - jepsen.util 73 :ok :read [24 19] INFO [2022-01-10 12:53:00,649] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:53:00,650] jepsen worker 48 - jepsen.util 48 :ok :read [16 17] INFO [2022-01-10 12:53:00,651] jepsen worker 35 - jepsen.util 35 :invoke :read [11 nil] INFO [2022-01-10 12:53:00,652] jepsen worker 35 - jepsen.util 35 :ok :read [11 36] INFO [2022-01-10 12:53:00,653] jepsen worker 2 - jepsen.util 2 :invoke :write [0 25] INFO [2022-01-10 12:53:00,655] jepsen worker 88 - jepsen.util 88 :invoke :read [29 nil] INFO [2022-01-10 12:53:00,656] jepsen worker 88 - jepsen.util 88 :ok :read [29 24] INFO [2022-01-10 12:53:00,656] jepsen worker 0 - jepsen.util 0 :invoke :write [0 43] INFO [2022-01-10 12:53:00,658] jepsen worker 43 - jepsen.util 43 :invoke :write [14 2] INFO [2022-01-10 12:53:00,660] jepsen worker 66 - jepsen.util 66 :invoke :write [22 12] INFO [2022-01-10 12:53:00,662] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,663] jepsen worker 60 - jepsen.util 60 :ok :read [20 8] INFO [2022-01-10 12:53:00,664] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:00,665] jepsen worker 30 - jepsen.util 30 :ok :read [10 4] INFO [2022-01-10 12:53:00,665] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:53:00,666] jepsen worker 12 - jepsen.util 12 :ok :read [4 33] INFO [2022-01-10 12:53:00,667] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,668] jepsen worker 40 - jepsen.util 40 :ok :read [13 29] INFO [2022-01-10 12:53:00,669] jepsen worker 24 - jepsen.util 24 :invoke :write [8 15] INFO [2022-01-10 12:53:00,671] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:00,672] jepsen worker 55 - jepsen.util 55 :ok :read [18 14] INFO [2022-01-10 12:53:00,673] jepsen worker 12 - jepsen.util 12 :invoke :read [4 nil] INFO [2022-01-10 12:53:00,674] jepsen worker 12 - jepsen.util 12 :ok :read [4 33] INFO [2022-01-10 12:53:00,675] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:53:00,676] jepsen worker 21 - jepsen.util 21 :ok :read [7 28] INFO [2022-01-10 12:53:00,677] jepsen worker 50 - jepsen.util 50 :invoke :write [16 34] INFO [2022-01-10 12:53:00,677] jepsen worker 24 - jepsen.util 24 :ok :write [8 15] INFO [2022-01-10 12:53:00,679] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:53:00,680] jepsen worker 19 - jepsen.util 19 :ok :read [6 13] INFO [2022-01-10 12:53:00,681] jepsen worker 37 - jepsen.util 37 :invoke :write [12 20] INFO [2022-01-10 12:53:00,683] jepsen worker 12 - jepsen.util 12 :invoke :write [4 29] INFO [2022-01-10 12:53:00,684] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:00,685] jepsen worker 68 - jepsen.util 68 :ok :read [22 36] INFO [2022-01-10 12:53:00,686] jepsen worker 45 - jepsen.util 45 :invoke :write [15 30] INFO [2022-01-10 12:53:00,688] jepsen worker 19 - jepsen.util 19 :invoke :write [6 31] INFO [2022-01-10 12:53:00,690] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:00,692] jepsen worker 1 - jepsen.util 1 :ok :read [0 4] INFO [2022-01-10 12:53:00,692] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,693] jepsen worker 9 - jepsen.util 9 :ok :write [3 42] INFO [2022-01-10 12:53:00,694] jepsen worker 81 - jepsen.util 81 :invoke :write [27 39] INFO [2022-01-10 12:53:00,695] jepsen worker 60 - jepsen.util 60 :ok :read [20 8] INFO [2022-01-10 12:53:00,695] jepsen worker 44 - jepsen.util 44 :ok :write [14 19] INFO [2022-01-10 12:53:00,695] jepsen worker 45 - jepsen.util 45 :ok :write [15 30] INFO [2022-01-10 12:53:00,696] jepsen worker 58 - jepsen.util 58 :ok :write [19 18] INFO [2022-01-10 12:53:00,696] jepsen worker 57 - jepsen.util 57 :invoke :write [19 1] INFO [2022-01-10 12:53:00,698] jepsen worker 16 - jepsen.util 16 :ok :write [5 28] INFO [2022-01-10 12:53:00,699] jepsen worker 58 - jepsen.util 58 :invoke :write [19 33] INFO [2022-01-10 12:53:00,701] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,701] jepsen worker 41 - jepsen.util 41 :ok :write [13 4] INFO [2022-01-10 12:53:00,702] jepsen worker 40 - jepsen.util 40 :ok :read [13 4] INFO [2022-01-10 12:53:00,703] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:53:00,703] jepsen worker 85 - jepsen.util 85 :ok :write [28 27] INFO [2022-01-10 12:53:00,704] jepsen worker 21 - jepsen.util 21 :ok :read [7 28] INFO [2022-01-10 12:53:00,704] jepsen worker 19 - jepsen.util 19 :ok :write [6 31] INFO [2022-01-10 12:53:00,705] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,706] jepsen worker 32 - jepsen.util 32 :ok :write [10 0] INFO [2022-01-10 12:53:00,706] jepsen worker 50 - jepsen.util 50 :ok :write [16 34] INFO [2022-01-10 12:53:00,706] jepsen worker 62 - jepsen.util 62 :ok :read [20 8] INFO [2022-01-10 12:53:00,707] jepsen worker 9 - jepsen.util 9 :invoke :read [3 nil] INFO [2022-01-10 12:53:00,708] jepsen worker 9 - jepsen.util 9 :ok :read [3 42] INFO [2022-01-10 12:53:00,708] jepsen worker 22 - jepsen.util 22 :ok :write [7 39] INFO [2022-01-10 12:53:00,709] jepsen worker 23 - jepsen.util 23 :invoke :write [7 37] INFO [2022-01-10 12:53:00,711] jepsen worker 13 - jepsen.util 13 :ok :write [4 13] INFO [2022-01-10 12:53:00,712] jepsen worker 16 - jepsen.util 16 :invoke :write [5 28] INFO [2022-01-10 12:53:00,714] jepsen worker 1 - jepsen.util 1 :invoke :write [0 44] INFO [2022-01-10 12:53:00,714] jepsen worker 82 - jepsen.util 82 :ok :write [27 6] INFO [2022-01-10 12:53:00,716] jepsen worker 24 - jepsen.util 24 :invoke :write [8 43] INFO [2022-01-10 12:53:00,717] jepsen worker 51 - jepsen.util 51 :ok :write [17 5] INFO [2022-01-10 12:53:00,718] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:53:00,720] jepsen worker 57 - jepsen.util 57 :ok :write [19 1] INFO [2022-01-10 12:53:00,720] jepsen worker 82 - jepsen.util 82 :ok :read [27 6] INFO [2022-01-10 12:53:00,720] jepsen worker 89 - jepsen.util 89 :ok :write [29 39] INFO [2022-01-10 12:53:00,721] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:00,722] jepsen worker 71 - jepsen.util 71 :ok :read [23 19] INFO [2022-01-10 12:53:00,722] jepsen worker 38 - jepsen.util 38 :ok :write [12 32] INFO [2022-01-10 12:53:00,723] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:00,724] jepsen worker 44 - jepsen.util 44 :ok :read [14 19] INFO [2022-01-10 12:53:00,725] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:00,726] jepsen worker 66 - jepsen.util 66 :ok :write [22 12] INFO [2022-01-10 12:53:00,727] jepsen worker 34 - jepsen.util 34 :ok :read [11 36] INFO [2022-01-10 12:53:00,728] jepsen worker 61 - jepsen.util 61 :ok :write [20 24] INFO [2022-01-10 12:53:00,728] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:00,729] jepsen worker 44 - jepsen.util 44 :ok :read [14 19] INFO [2022-01-10 12:53:00,730] jepsen worker 76 - jepsen.util 76 :invoke :write [25 12] INFO [2022-01-10 12:53:00,733] jepsen worker 2 - jepsen.util 2 :ok :write [0 25] INFO [2022-01-10 12:53:00,733] jepsen worker 84 - jepsen.util 84 :ok :write [28 20] INFO [2022-01-10 12:53:00,733] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,734] jepsen worker 38 - jepsen.util 38 :ok :read [12 32] INFO [2022-01-10 12:53:00,736] jepsen worker 20 - jepsen.util 20 :invoke :read [6 nil] INFO [2022-01-10 12:53:00,737] jepsen worker 20 - jepsen.util 20 :ok :read [6 31] INFO [2022-01-10 12:53:00,738] jepsen worker 43 - jepsen.util 43 :ok :write [14 2] INFO [2022-01-10 12:53:00,740] jepsen worker nemesis - jepsen.util :nemesis :info :fail-disk nil INFO [2022-01-10 12:53:00,741] jepsen worker 58 - jepsen.util 58 :ok :write [19 33] INFO [2022-01-10 12:53:00,742] jepsen worker 7 - jepsen.util 7 :invoke :write [2 26] INFO [2022-01-10 12:53:00,745] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:53:00,748] jepsen worker 67 - jepsen.util 67 :invoke :write [22 11] INFO [2022-01-10 12:53:00,750] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:00,751] jepsen worker 51 - jepsen.util 51 :ok :read [17 5] INFO [2022-01-10 12:53:00,752] jepsen worker 46 - jepsen.util 46 :ok :read [15 30] INFO [2022-01-10 12:53:00,753] jepsen worker 29 - jepsen.util 29 :invoke :write [9 19] INFO [2022-01-10 12:53:00,755] jepsen worker 79 - jepsen.util 79 :invoke :write [26 33] INFO [2022-01-10 12:53:00,759] jepsen worker 2 - jepsen.util 2 :invoke :write [0 37] INFO [2022-01-10 12:53:00,761] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:53:00,763] jepsen worker 21 - jepsen.util 21 :ok :read [7 39] INFO [2022-01-10 12:53:00,764] jepsen worker 43 - jepsen.util 43 :invoke :write [14 4] INFO [2022-01-10 12:53:00,767] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,768] jepsen worker 61 - jepsen.util 61 :ok :read [20 24] INFO [2022-01-10 12:53:00,768] jepsen worker 87 - jepsen.util 87 :ok :write [29 19] INFO [2022-01-10 12:53:00,769] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:00,770] jepsen worker 13 - jepsen.util 13 :ok :read [4 13] INFO [2022-01-10 12:53:00,770] jepsen worker 37 - jepsen.util 37 :ok :write [12 20] INFO [2022-01-10 12:53:00,771] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:53:00,772] jepsen worker 19 - jepsen.util 19 :ok :read [6 31] INFO [2022-01-10 12:53:00,774] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:00,775] jepsen worker 13 - jepsen.util 13 :ok :read [4 13] INFO [2022-01-10 12:53:00,776] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:00,777] jepsen worker 74 - jepsen.util 74 :ok :read [24 19] INFO [2022-01-10 12:53:00,778] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,779] jepsen worker 40 - jepsen.util 40 :ok :read [13 4] INFO [2022-01-10 12:53:00,780] jepsen worker 11 - jepsen.util 11 :ok :write [3 43] INFO [2022-01-10 12:53:00,780] jepsen worker 21 - jepsen.util 21 :invoke :write [7 12] INFO [2022-01-10 12:53:00,783] jepsen worker 9 - jepsen.util 9 :invoke :write [3 4] INFO [2022-01-10 12:53:00,786] jepsen worker 22 - jepsen.util 22 :invoke :read [7 nil] INFO [2022-01-10 12:53:00,787] jepsen worker 22 - jepsen.util 22 :ok :read [7 39] INFO [2022-01-10 12:53:00,788] jepsen worker 73 - jepsen.util 73 :invoke :write [24 28] INFO [2022-01-10 12:53:00,790] jepsen worker 35 - jepsen.util 35 :invoke :write [11 27] INFO [2022-01-10 12:53:00,792] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:53:00,794] jepsen worker 89 - jepsen.util 89 :ok :read [29 19] INFO [2022-01-10 12:53:00,795] jepsen worker 51 - jepsen.util 51 :invoke :write [17 29] INFO [2022-01-10 12:53:00,797] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:00,798] jepsen worker 44 - jepsen.util 44 :ok :read [14 2] INFO [2022-01-10 12:53:00,799] jepsen worker 59 - jepsen.util 59 :invoke :read [19 nil] INFO [2022-01-10 12:53:00,800] jepsen worker 59 - jepsen.util 59 :ok :read [19 33] INFO [2022-01-10 12:53:00,801] jepsen worker 74 - jepsen.util 74 :invoke :write [24 39] INFO [2022-01-10 12:53:00,804] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,805] jepsen worker 60 - jepsen.util 60 :ok :read [20 24] INFO [2022-01-10 12:53:00,806] jepsen worker 30 - jepsen.util 30 :invoke :write [10 35] INFO [2022-01-10 12:53:00,809] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:00,810] jepsen worker 27 - jepsen.util 27 :ok :read [9 25] INFO [2022-01-10 12:53:00,812] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:53:00,813] jepsen worker 48 - jepsen.util 48 :ok :read [16 34] INFO [2022-01-10 12:53:00,814] jepsen worker 60 - jepsen.util 60 :invoke :write [20 46] INFO [2022-01-10 12:53:00,816] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:00,817] jepsen worker 84 - jepsen.util 84 :ok :read [28 20] INFO [2022-01-10 12:53:00,818] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:00,819] jepsen worker 78 - jepsen.util 78 :ok :read [26 14] INFO [2022-01-10 12:53:00,821] jepsen worker 66 - jepsen.util 66 :invoke :write [22 16] INFO [2022-01-10 12:53:00,823] jepsen worker 6 - jepsen.util 6 :invoke :write [2 43] INFO [2022-01-10 12:53:00,826] jepsen worker 57 - jepsen.util 57 :invoke :write [19 19] INFO [2022-01-10 12:53:00,828] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,828] jepsen worker 12 - jepsen.util 12 :ok :write [4 29] INFO [2022-01-10 12:53:00,829] jepsen worker 41 - jepsen.util 41 :ok :read [13 4] INFO [2022-01-10 12:53:00,830] jepsen worker 32 - jepsen.util 32 :invoke :write [10 45] INFO [2022-01-10 12:53:00,831] jepsen worker 23 - jepsen.util 23 :ok :write [7 37] INFO [2022-01-10 12:53:00,832] jepsen worker 31 - jepsen.util 31 :invoke :read [10 nil] INFO [2022-01-10 12:53:00,833] jepsen worker 31 - jepsen.util 31 :ok :read [10 0] INFO [2022-01-10 12:53:00,833] jepsen worker 24 - jepsen.util 24 :ok :write [8 43] INFO [2022-01-10 12:53:00,835] jepsen worker 26 - jepsen.util 26 :invoke :write [8 17] INFO [2022-01-10 12:53:00,837] jepsen worker 55 - jepsen.util 55 :invoke :write [18 38] INFO [2022-01-10 12:53:00,840] jepsen worker 85 - jepsen.util 85 :invoke :write [28 26] INFO [2022-01-10 12:53:00,842] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:00,843] jepsen worker 34 - jepsen.util 34 :ok :read [11 36] INFO [2022-01-10 12:53:00,844] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:00,845] jepsen worker 47 - jepsen.util 47 :ok :read [15 30] INFO [2022-01-10 12:53:00,846] jepsen worker 14 - jepsen.util 14 :invoke :write [4 31] INFO [2022-01-10 12:53:00,848] jepsen worker 11 - jepsen.util 11 :invoke :write [3 30] INFO [2022-01-10 12:53:00,850] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:53:00,851] jepsen worker 58 - jepsen.util 58 :ok :read [19 33] INFO [2022-01-10 12:53:00,852] jepsen worker 82 - jepsen.util 82 :invoke :read [27 nil] INFO [2022-01-10 12:53:00,853] jepsen worker 82 - jepsen.util 82 :ok :read [27 6] INFO [2022-01-10 12:53:00,854] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:00,855] jepsen worker 68 - jepsen.util 68 :ok :read [22 12] INFO [2022-01-10 12:53:00,856] jepsen worker 61 - jepsen.util 61 :invoke :write [20 19] INFO [2022-01-10 12:53:00,858] jepsen worker 70 - jepsen.util 70 :invoke :read [23 nil] INFO [2022-01-10 12:53:00,859] jepsen worker 70 - jepsen.util 70 :ok :read [23 19] INFO [2022-01-10 12:53:00,861] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:53:00,861] jepsen worker 54 - jepsen.util 54 :ok :read [18 14] INFO [2022-01-10 12:53:00,862] jepsen worker 22 - jepsen.util 22 :invoke :write [7 39] INFO [2022-01-10 12:53:00,864] jepsen worker 58 - jepsen.util 58 :invoke :write [19 34] INFO [2022-01-10 12:53:00,866] jepsen worker 77 - jepsen.util 77 :invoke :write [25 14] INFO [2022-01-10 12:53:00,868] jepsen worker 87 - jepsen.util 87 :invoke :write [29 44] INFO [2022-01-10 12:53:00,870] jepsen worker 88 - jepsen.util 88 :invoke :write [29 7] INFO [2022-01-10 12:53:00,872] jepsen worker 68 - jepsen.util 68 :invoke :write [22 41] INFO [2022-01-10 12:53:00,874] jepsen worker 31 - jepsen.util 31 :invoke :write [10 26] INFO [2022-01-10 12:53:00,876] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,877] jepsen worker 4 - jepsen.util 4 :ok :read [1 14] INFO [2022-01-10 12:53:00,877] jepsen worker 82 - jepsen.util 82 :invoke :write [27 39] INFO [2022-01-10 12:53:00,879] jepsen worker 84 - jepsen.util 84 :invoke :write [28 47] INFO [2022-01-10 12:53:00,881] jepsen worker 13 - jepsen.util 13 :invoke :write [4 34] INFO [2022-01-10 12:53:00,882] jepsen worker 19 - jepsen.util 19 :invoke :read [6 nil] INFO [2022-01-10 12:53:00,883] jepsen worker 19 - jepsen.util 19 :ok :read [6 31] INFO [2022-01-10 12:53:00,884] jepsen worker 12 - jepsen.util 12 :invoke :write [4 38] INFO [2022-01-10 12:53:00,886] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,887] jepsen worker 62 - jepsen.util 62 :ok :read [20 24] INFO [2022-01-10 12:53:00,887] jepsen worker 19 - jepsen.util 19 :invoke :write [6 32] INFO [2022-01-10 12:53:00,889] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:00,890] jepsen worker 23 - jepsen.util 23 :ok :read [7 37] INFO [2022-01-10 12:53:00,890] jepsen worker 80 - jepsen.util 80 :invoke :write [26 0] INFO [2022-01-10 12:53:00,892] jepsen worker 23 - jepsen.util 23 :invoke :write [7 12] INFO [2022-01-10 12:53:00,894] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:00,895] jepsen worker 59 - jepsen.util 59 :invoke :write [19 43] INFO [2022-01-10 12:53:00,895] jepsen worker 44 - jepsen.util 44 :ok :read [14 2] INFO [2022-01-10 12:53:00,896] jepsen worker 45 - jepsen.util 45 :invoke :read [15 nil] INFO [2022-01-10 12:53:00,898] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,898] jepsen worker 37 - jepsen.util 37 :ok :read [12 20] INFO [2022-01-10 12:53:00,899] jepsen worker 45 - jepsen.util 45 :ok :read [15 30] INFO [2022-01-10 12:53:00,899] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:00,900] jepsen worker 47 - jepsen.util 47 :ok :read [15 30] INFO [2022-01-10 12:53:00,900] jepsen worker 56 - jepsen.util 56 :invoke :write [18 11] INFO [2022-01-10 12:53:00,902] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:00,903] jepsen worker 34 - jepsen.util 34 :ok :read [11 36] INFO [2022-01-10 12:53:00,903] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:00,904] jepsen worker 62 - jepsen.util 62 :ok :read [20 24] INFO [2022-01-10 12:53:00,905] jepsen worker 49 - jepsen.util 49 :invoke :write [16 3] INFO [2022-01-10 12:53:00,907] jepsen worker 20 - jepsen.util 20 :invoke :write [6 31] INFO [2022-01-10 12:53:00,908] jepsen worker 44 - jepsen.util 44 :invoke :write [14 21] INFO [2022-01-10 12:53:00,910] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,911] jepsen worker 39 - jepsen.util 39 :ok :read [13 4] INFO [2022-01-10 12:53:00,911] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:53:00,912] jepsen worker 89 - jepsen.util 89 :ok :read [29 19] INFO [2022-01-10 12:53:00,912] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:53:00,913] jepsen worker 48 - jepsen.util 48 :ok :read [16 34] INFO [2022-01-10 12:53:00,913] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:00,915] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:53:00,916] jepsen worker 63 - jepsen.util 63 :ok :read [21 23] INFO [2022-01-10 12:53:00,916] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:00,917] jepsen worker 78 - jepsen.util 78 :ok :read [26 14] INFO [2022-01-10 12:53:00,917] jepsen worker 47 - jepsen.util 47 :invoke :write [15 32] INFO [2022-01-10 12:53:00,919] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:53:00,920] jepsen worker 89 - jepsen.util 89 :ok :read [29 19] INFO [2022-01-10 12:53:00,920] jepsen worker 62 - jepsen.util 62 :invoke :write [20 7] INFO [2022-01-10 12:53:00,921] jepsen worker 78 - jepsen.util 78 :invoke :write [26 19] INFO [2022-01-10 12:53:00,922] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,923] jepsen worker 4 - jepsen.util 4 :ok :read [1 14] INFO [2022-01-10 12:53:00,923] jepsen worker 75 - jepsen.util 75 :invoke :write [25 10] INFO [2022-01-10 12:53:00,924] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,925] jepsen worker 5 - jepsen.util 5 :ok :read [1 14] INFO [2022-01-10 12:53:00,925] jepsen worker 54 - jepsen.util 54 :invoke :read [18 nil] INFO [2022-01-10 12:53:00,926] jepsen worker 54 - jepsen.util 54 :ok :read [18 14] INFO [2022-01-10 12:53:00,926] jepsen worker 46 - jepsen.util 46 :invoke :read [15 nil] INFO [2022-01-10 12:53:00,928] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:00,928] jepsen worker 46 - jepsen.util 46 :ok :read [15 30] INFO [2022-01-10 12:53:00,929] jepsen worker 69 - jepsen.util 69 :ok :read [23 19] INFO [2022-01-10 12:53:00,929] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:53:00,930] jepsen worker 63 - jepsen.util 63 :ok :read [21 23] INFO [2022-01-10 12:53:00,930] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:53:00,931] jepsen worker 48 - jepsen.util 48 :ok :read [16 34] INFO [2022-01-10 12:53:00,931] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:53:00,932] jepsen worker 24 - jepsen.util 24 :ok :read [8 43] INFO [2022-01-10 12:53:00,932] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,933] jepsen worker 41 - jepsen.util 41 :ok :read [13 4] INFO [2022-01-10 12:53:00,933] jepsen worker 71 - jepsen.util 71 :invoke :write [23 31] INFO [2022-01-10 12:53:00,935] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:53:00,936] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:53:00,937] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,937] jepsen worker 48 - jepsen.util 48 :ok :read [16 34] INFO [2022-01-10 12:53:00,938] jepsen worker 37 - jepsen.util 37 :ok :read [12 20] INFO [2022-01-10 12:53:00,938] jepsen worker 24 - jepsen.util 24 :ok :read [8 43] INFO [2022-01-10 12:53:00,938] jepsen worker 25 - jepsen.util 25 :invoke :read [8 nil] INFO [2022-01-10 12:53:00,939] jepsen worker 25 - jepsen.util 25 :ok :read [8 43] INFO [2022-01-10 12:53:00,939] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:00,940] jepsen worker 69 - jepsen.util 69 :ok :read [23 19] INFO [2022-01-10 12:53:00,941] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:53:00,942] jepsen worker 89 - jepsen.util 89 :ok :read [29 19] INFO [2022-01-10 12:53:00,942] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:53:00,943] jepsen worker 1 - jepsen.util 1 :ok :write [0 44] INFO [2022-01-10 12:53:00,943] jepsen worker 24 - jepsen.util 24 :ok :read [8 43] INFO [2022-01-10 12:53:00,944] jepsen worker 54 - jepsen.util 54 :invoke :write [18 32] INFO [2022-01-10 12:53:00,945] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,946] jepsen worker 4 - jepsen.util 4 :ok :read [1 14] INFO [2022-01-10 12:53:00,946] jepsen worker 27 - jepsen.util 27 :invoke :write [9 25] INFO [2022-01-10 12:53:00,948] jepsen worker 89 - jepsen.util 89 :invoke :write [29 19] INFO [2022-01-10 12:53:00,949] jepsen worker 45 - jepsen.util 45 :invoke :write [15 35] INFO [2022-01-10 12:53:00,950] jepsen worker 9 - jepsen.util 9 :ok :write [3 4] INFO [2022-01-10 12:53:00,950] jepsen worker 70 - jepsen.util 70 :invoke :write [23 11] INFO [2022-01-10 12:53:00,953] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,954] jepsen worker 37 - jepsen.util 37 :ok :read [12 20] INFO [2022-01-10 12:53:00,954] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,955] jepsen worker 40 - jepsen.util 40 :ok :read [13 4] INFO [2022-01-10 12:53:00,955] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,956] jepsen worker 36 - jepsen.util 36 :ok :write [12 35] INFO [2022-01-10 12:53:00,956] jepsen worker 34 - jepsen.util 34 :invoke :write [11 7] INFO [2022-01-10 12:53:00,956] jepsen worker 37 - jepsen.util 37 :ok :read [12 35] INFO [2022-01-10 12:53:00,957] jepsen worker 29 - jepsen.util 29 :ok :write [9 19] INFO [2022-01-10 12:53:00,957] jepsen worker 9 - jepsen.util 9 :invoke :write [3 35] INFO [2022-01-10 12:53:00,959] jepsen worker 37 - jepsen.util 37 :invoke :write [12 19] INFO [2022-01-10 12:53:00,961] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:00,962] jepsen worker 5 - jepsen.util 5 :invoke :write [1 14] INFO [2022-01-10 12:53:00,962] jepsen worker 50 - jepsen.util 50 :ok :read [16 34] INFO [2022-01-10 12:53:00,963] jepsen worker 53 - jepsen.util 53 :ok :write [17 29] INFO [2022-01-10 12:53:00,963] jepsen worker 81 - jepsen.util 81 :ok :write [27 39] INFO [2022-01-10 12:53:00,963] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:53:00,964] jepsen worker 64 - jepsen.util 64 :ok :read [21 23] INFO [2022-01-10 12:53:00,965] jepsen worker 81 - jepsen.util 81 :invoke :write [27 3] INFO [2022-01-10 12:53:00,966] jepsen worker 28 - jepsen.util 28 :invoke :write [9 24] INFO [2022-01-10 12:53:00,967] jepsen worker 87 - jepsen.util 87 :ok :write [29 44] INFO [2022-01-10 12:53:00,967] jepsen worker 46 - jepsen.util 46 :invoke :write [15 26] INFO [2022-01-10 12:53:00,968] jepsen worker 72 - jepsen.util 72 :ok :write [24 43] INFO [2022-01-10 12:53:00,969] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:53:00,969] jepsen worker 16 - jepsen.util 16 :ok :write [5 28] INFO [2022-01-10 12:53:00,969] jepsen worker 63 - jepsen.util 63 :ok :read [21 23] INFO [2022-01-10 12:53:00,970] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,971] jepsen worker 4 - jepsen.util 4 :ok :read [1 14] INFO [2022-01-10 12:53:00,971] jepsen worker 72 - jepsen.util 72 :invoke :read [24 nil] INFO [2022-01-10 12:53:00,972] jepsen worker 72 - jepsen.util 72 :ok :read [24 43] INFO [2022-01-10 12:53:00,972] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:00,973] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,973] jepsen worker 1 - jepsen.util 1 :ok :read [0 44] INFO [2022-01-10 12:53:00,974] jepsen worker 41 - jepsen.util 41 :ok :read [13 4] INFO [2022-01-10 12:53:00,974] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:00,975] jepsen worker 85 - jepsen.util 85 :ok :write [28 26] INFO [2022-01-10 12:53:00,975] jepsen worker 65 - jepsen.util 65 :ok :write [21 41] INFO [2022-01-10 12:53:00,975] jepsen worker 39 - jepsen.util 39 :ok :read [13 4] INFO [2022-01-10 12:53:00,976] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:00,976] jepsen worker 67 - jepsen.util 67 :ok :write [22 11] INFO [2022-01-10 12:53:00,977] jepsen worker 36 - jepsen.util 36 :ok :read [12 35] INFO [2022-01-10 12:53:00,977] jepsen worker 24 - jepsen.util 24 :invoke :read [8 nil] INFO [2022-01-10 12:53:00,978] jepsen worker 24 - jepsen.util 24 :ok :read [8 43] INFO [2022-01-10 12:53:00,979] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:00,980] jepsen worker 1 - jepsen.util 1 :ok :read [0 44] INFO [2022-01-10 12:53:00,980] jepsen worker 79 - jepsen.util 79 :ok :write [26 33] INFO [2022-01-10 12:53:00,980] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:00,981] jepsen worker 67 - jepsen.util 67 :ok :read [22 11] INFO [2022-01-10 12:53:00,981] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:00,983] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:00,984] jepsen worker 1 - jepsen.util 1 :ok :read [0 44] INFO [2022-01-10 12:53:00,984] jepsen worker 69 - jepsen.util 69 :invoke :write [23 40] INFO [2022-01-10 12:53:00,984] jepsen worker 3 - jepsen.util 3 :ok :read [1 14] INFO [2022-01-10 12:53:00,985] jepsen worker 60 - jepsen.util 60 :ok :write [20 46] INFO [2022-01-10 12:53:00,985] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:53:00,987] jepsen worker 48 - jepsen.util 48 :ok :read [16 34] INFO [2022-01-10 12:53:00,987] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:00,988] jepsen worker 14 - jepsen.util 14 :ok :write [4 31] INFO [2022-01-10 12:53:00,988] jepsen worker 50 - jepsen.util 50 :ok :read [16 34] INFO [2022-01-10 12:53:00,989] jepsen worker 72 - jepsen.util 72 :invoke :write [24 3] INFO [2022-01-10 12:53:00,990] jepsen worker 48 - jepsen.util 48 :invoke :write [16 23] INFO [2022-01-10 12:53:00,991] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:53:00,993] jepsen worker 85 - jepsen.util 85 :ok :read [28 26] INFO [2022-01-10 12:53:00,993] jepsen worker 79 - jepsen.util 79 :invoke :write [26 12] INFO [2022-01-10 12:53:00,993] jepsen worker 76 - jepsen.util 76 :ok :write [25 12] INFO [2022-01-10 12:53:00,994] jepsen worker 0 - jepsen.util 0 :ok :write [0 43] INFO [2022-01-10 12:53:00,994] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:53:00,996] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:00,997] jepsen worker 53 - jepsen.util 53 :invoke :write [17 14] INFO [2022-01-10 12:53:00,997] jepsen worker 76 - jepsen.util 76 :ok :read [25 12] INFO [2022-01-10 12:53:00,997] jepsen worker 63 - jepsen.util 63 :ok :read [21 41] INFO [2022-01-10 12:53:00,998] jepsen worker 37 - jepsen.util 37 :ok :write [12 19] INFO [2022-01-10 12:53:00,998] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:00,999] jepsen worker 67 - jepsen.util 67 :ok :read [22 11] INFO [2022-01-10 12:53:01,000] jepsen worker 14 - jepsen.util 14 :invoke :read [4 nil] INFO [2022-01-10 12:53:01,000] jepsen worker 43 - jepsen.util 43 :ok :write [14 4] INFO [2022-01-10 12:53:01,001] jepsen worker 14 - jepsen.util 14 :ok :read [4 31] INFO [2022-01-10 12:53:01,001] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:01,002] jepsen worker 60 - jepsen.util 60 :ok :read [20 46] INFO [2022-01-10 12:53:01,003] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:01,004] jepsen worker 4 - jepsen.util 4 :ok :read [1 14] INFO [2022-01-10 12:53:01,004] jepsen worker 71 - jepsen.util 71 :ok :write [23 31] INFO [2022-01-10 12:53:01,004] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,006] jepsen worker 16 - jepsen.util 16 :ok :read [5 28] INFO [2022-01-10 12:53:01,006] jepsen worker 14 - jepsen.util 14 :invoke :write [4 29] INFO [2022-01-10 12:53:01,007] jepsen worker 7 - jepsen.util 7 :ok :write [2 26] INFO [2022-01-10 12:53:01,008] jepsen worker 0 - jepsen.util 0 :invoke :write [0 27] INFO [2022-01-10 12:53:01,009] jepsen worker 51 - jepsen.util 51 :ok :write [17 29] INFO [2022-01-10 12:53:01,009] jepsen worker 65 - jepsen.util 65 :invoke :write [21 7] INFO [2022-01-10 12:53:01,010] jepsen worker 89 - jepsen.util 89 :ok :write [29 19] INFO [2022-01-10 12:53:01,011] jepsen worker 5 - jepsen.util 5 :ok :write [1 14] INFO [2022-01-10 12:53:01,012] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:01,012] jepsen worker 35 - jepsen.util 35 :ok :write [11 27] INFO [2022-01-10 12:53:01,013] jepsen worker 50 - jepsen.util 50 :ok :read [16 34] INFO [2022-01-10 12:53:01,014] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,015] jepsen worker 16 - jepsen.util 16 :ok :read [5 28] INFO [2022-01-10 12:53:01,016] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:01,017] jepsen worker 84 - jepsen.util 84 :ok :write [28 47] INFO [2022-01-10 12:53:01,018] jepsen worker 51 - jepsen.util 51 :ok :read [17 29] INFO [2022-01-10 12:53:01,019] jepsen worker 67 - jepsen.util 67 :invoke :write [22 44] INFO [2022-01-10 12:53:01,020] jepsen worker 21 - jepsen.util 21 :ok :write [7 12] INFO [2022-01-10 12:53:01,020] jepsen worker 50 - jepsen.util 50 :invoke :write [16 4] INFO [2022-01-10 12:53:01,022] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:01,023] jepsen worker 60 - jepsen.util 60 :ok :read [20 46] INFO [2022-01-10 12:53:01,024] jepsen worker 71 - jepsen.util 71 :invoke :write [23 45] INFO [2022-01-10 12:53:01,026] jepsen worker 5 - jepsen.util 5 :invoke :write [1 4] INFO [2022-01-10 12:53:01,026] jepsen worker 78 - jepsen.util 78 :ok :write [26 19] INFO [2022-01-10 12:53:01,027] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,028] jepsen worker 49 - jepsen.util 49 :ok :write [16 3] INFO [2022-01-10 12:53:01,028] jepsen worker 89 - jepsen.util 89 :ok :read [29 19] INFO [2022-01-10 12:53:01,029] jepsen worker 25 - jepsen.util 25 :invoke :write [8 3] INFO [2022-01-10 12:53:01,031] jepsen worker 35 - jepsen.util 35 :invoke :write [11 32] INFO [2022-01-10 12:53:01,032] jepsen worker 13 - jepsen.util 13 :ok :write [4 34] INFO [2022-01-10 12:53:01,032] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:01,032] jepsen worker 81 - jepsen.util 81 :ok :write [27 3] INFO [2022-01-10 12:53:01,033] jepsen worker 36 - jepsen.util 36 :ok :read [12 19] INFO [2022-01-10 12:53:01,034] jepsen worker 24 - jepsen.util 24 :invoke :write [8 18] INFO [2022-01-10 12:53:01,035] jepsen worker 0 - jepsen.util 0 :ok :write [0 27] INFO [2022-01-10 12:53:01,036] jepsen worker 39 - jepsen.util 39 :invoke :write [13 43] INFO [2022-01-10 12:53:01,038] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,039] jepsen worker 29 - jepsen.util 29 :ok :read [9 19] INFO [2022-01-10 12:53:01,039] jepsen worker 7 - jepsen.util 7 :invoke :write [2 32] INFO [2022-01-10 12:53:01,040] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:01,041] jepsen worker 81 - jepsen.util 81 :invoke :write [27 19] INFO [2022-01-10 12:53:01,043] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,044] jepsen worker 64 - jepsen.util 64 :ok :read [21 41] INFO [2022-01-10 12:53:01,044] jepsen worker 29 - jepsen.util 29 :invoke :write [9 14] INFO [2022-01-10 12:53:01,046] jepsen worker 40 - jepsen.util 40 :invoke :write [13 14] INFO [2022-01-10 12:53:01,047] jepsen worker 70 - jepsen.util 70 :ok :write [23 11] INFO [2022-01-10 12:53:01,047] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,047] jepsen worker 68 - jepsen.util 68 :ok :write [22 41] INFO [2022-01-10 12:53:01,048] jepsen worker 16 - jepsen.util 16 :ok :read [5 28] INFO [2022-01-10 12:53:01,048] jepsen worker 51 - jepsen.util 51 :invoke :write [17 34] INFO [2022-01-10 12:53:01,050] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,052] jepsen worker 63 - jepsen.util 63 :ok :read [21 41] INFO [2022-01-10 12:53:01,052] jepsen worker 62 - jepsen.util 62 :ok :write [20 7] INFO [2022-01-10 12:53:01,053] jepsen worker 89 - jepsen.util 89 :invoke :write [29 47] INFO [2022-01-10 12:53:01,053] jepsen worker 53 - jepsen.util 53 :ok :write [17 14] INFO [2022-01-10 12:53:01,055] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:53:01,057] jepsen worker 76 - jepsen.util 76 :invoke :write [25 21] INFO [2022-01-10 12:53:01,057] jepsen worker 37 - jepsen.util 37 :ok :read [12 13] INFO [2022-01-10 12:53:01,059] jepsen worker 75 - jepsen.util 75 :ok :write [25 10] INFO [2022-01-10 12:53:01,059] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,060] jepsen worker 16 - jepsen.util 16 :ok :read [5 28] INFO [2022-01-10 12:53:01,060] jepsen worker 75 - jepsen.util 75 :invoke :write [25 5] INFO [2022-01-10 12:53:01,061] jepsen worker 5 - jepsen.util 5 :ok :write [1 4] INFO [2022-01-10 12:53:01,062] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,063] jepsen worker 43 - jepsen.util 43 :ok :read [14 4] INFO [2022-01-10 12:53:01,063] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:53:01,064] jepsen worker 5 - jepsen.util 5 :ok :read [1 4] INFO [2022-01-10 12:53:01,065] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:01,066] jepsen worker 44 - jepsen.util 44 :ok :write [14 21] INFO [2022-01-10 12:53:01,066] jepsen worker 62 - jepsen.util 62 :ok :read [20 7] INFO [2022-01-10 12:53:01,066] jepsen worker 85 - jepsen.util 85 :invoke :write [28 6] INFO [2022-01-10 12:53:01,068] jepsen worker 84 - jepsen.util 84 :invoke :write [28 31] INFO [2022-01-10 12:53:01,069] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,071] jepsen worker 16 - jepsen.util 16 :ok :read [5 28] INFO [2022-01-10 12:53:01,071] jepsen worker 49 - jepsen.util 49 :invoke :read [16 nil] INFO [2022-01-10 12:53:01,072] jepsen worker 49 - jepsen.util 49 :ok :read [16 3] INFO [2022-01-10 12:53:01,072] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,073] jepsen worker 64 - jepsen.util 64 :ok :read [21 41] INFO [2022-01-10 12:53:01,073] jepsen worker 49 - jepsen.util 49 :invoke :read [16 nil] INFO [2022-01-10 12:53:01,073] jepsen worker 39 - jepsen.util 39 :ok :write [13 43] INFO [2022-01-10 12:53:01,074] jepsen worker 49 - jepsen.util 49 :ok :read [16 3] INFO [2022-01-10 12:53:01,075] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,075] jepsen worker 79 - jepsen.util 79 :ok :write [26 12] INFO [2022-01-10 12:53:01,076] jepsen worker 16 - jepsen.util 16 :ok :read [5 28] INFO [2022-01-10 12:53:01,077] jepsen worker 62 - jepsen.util 62 :invoke :write [20 47] INFO [2022-01-10 12:53:01,078] jepsen worker 16 - jepsen.util 16 :invoke :write [5 26] INFO [2022-01-10 12:53:01,080] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,080] jepsen worker 34 - jepsen.util 34 :ok :write [11 7] INFO [2022-01-10 12:53:01,080] jepsen worker 7 - jepsen.util 7 :ok :write [2 32] INFO [2022-01-10 12:53:01,081] jepsen worker 63 - jepsen.util 63 :ok :read [21 41] INFO [2022-01-10 12:53:01,081] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:01,082] jepsen worker 68 - jepsen.util 68 :ok :read [22 41] INFO [2022-01-10 12:53:01,082] jepsen worker 13 - jepsen.util 13 :invoke :write [4 34] INFO [2022-01-10 12:53:01,083] jepsen worker 14 - jepsen.util 14 :ok :write [4 29] INFO [2022-01-10 12:53:01,084] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:53:01,085] jepsen worker 53 - jepsen.util 53 :ok :read [17 14] INFO [2022-01-10 12:53:01,085] jepsen worker 34 - jepsen.util 34 :invoke :write [11 25] INFO [2022-01-10 12:53:01,087] jepsen worker 60 - jepsen.util 60 :invoke :write [20 3] INFO [2022-01-10 12:53:01,088] jepsen worker 22 - jepsen.util 22 :ok :write [7 39] INFO [2022-01-10 12:53:01,088] jepsen worker 3 - jepsen.util 3 :invoke :write [1 31] INFO [2022-01-10 12:53:01,089] jepsen worker 88 - jepsen.util 88 :ok :write [29 7] INFO [2022-01-10 12:53:01,090] jepsen worker 63 - jepsen.util 63 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,091] jepsen worker 63 - jepsen.util 63 :ok :read [21 41] INFO [2022-01-10 12:53:01,091] jepsen worker 78 - jepsen.util 78 :invoke :write [26 4] INFO [2022-01-10 12:53:01,093] jepsen worker 7 - jepsen.util 7 :invoke :write [2 35] INFO [2022-01-10 12:53:01,095] jepsen worker 43 - jepsen.util 43 :invoke :write [14 40] INFO [2022-01-10 12:53:01,095] jepsen worker 48 - jepsen.util 48 :ok :write [16 23] INFO [2022-01-10 12:53:01,096] jepsen worker 49 - jepsen.util 49 :invoke :write [16 49] INFO [2022-01-10 12:53:01,097] jepsen worker 22 - jepsen.util 22 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,098] jepsen worker 2 - jepsen.util 2 :ok :write [0 37] INFO [2022-01-10 12:53:01,098] jepsen worker 1 - jepsen.util 1 :invoke :write [0 16] INFO [2022-01-10 12:53:01,099] jepsen worker 22 - jepsen.util 22 :ok :read [7 39] INFO [2022-01-10 12:53:01,100] jepsen worker 21 - jepsen.util 21 :invoke :write [7 26] INFO [2022-01-10 12:53:01,101] jepsen worker 14 - jepsen.util 14 :invoke :write [4 4] INFO [2022-01-10 12:53:01,102] jepsen worker 4 - jepsen.util 4 :invoke :write [1 16] INFO [2022-01-10 12:53:01,103] jepsen worker 22 - jepsen.util 22 :invoke :write [7 40] INFO [2022-01-10 12:53:01,103] jepsen worker 81 - jepsen.util 81 :ok :write [27 19] INFO [2022-01-10 12:53:01,104] jepsen worker 40 - jepsen.util 40 :ok :write [13 14] INFO [2022-01-10 12:53:01,104] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,105] jepsen worker 41 - jepsen.util 41 :ok :read [13 14] INFO [2022-01-10 12:53:01,105] jepsen worker 40 - jepsen.util 40 :invoke :write [13 48] INFO [2022-01-10 12:53:01,106] jepsen worker 2 - jepsen.util 2 :invoke :read [0 nil] INFO [2022-01-10 12:53:01,107] jepsen worker 67 - jepsen.util 67 :ok :write [22 44] INFO [2022-01-10 12:53:01,107] jepsen worker 2 - jepsen.util 2 :ok :read [0 37] INFO [2022-01-10 12:53:01,108] jepsen worker 68 - jepsen.util 68 :invoke :write [22 4] INFO [2022-01-10 12:53:01,109] jepsen worker 5 - jepsen.util 5 :invoke :read [1 nil] INFO [2022-01-10 12:53:01,110] jepsen worker 70 - jepsen.util 70 :invoke :write [23 43] INFO [2022-01-10 12:53:01,110] jepsen worker 5 - jepsen.util 5 :ok :read [1 4] INFO [2022-01-10 12:53:01,111] jepsen worker 81 - jepsen.util 81 :invoke :write [27 26] INFO [2022-01-10 12:53:01,112] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:53:01,113] jepsen worker 37 - jepsen.util 37 :ok :read [12 13] INFO [2022-01-10 12:53:01,113] jepsen worker 79 - jepsen.util 79 :invoke :write [26 24] INFO [2022-01-10 12:53:01,114] jepsen worker 75 - jepsen.util 75 :ok :write [25 5] INFO [2022-01-10 12:53:01,114] jepsen worker 87 - jepsen.util 87 :invoke :write [29 8] INFO [2022-01-10 12:53:01,115] jepsen worker 5 - jepsen.util 5 :invoke :write [1 18] INFO [2022-01-10 12:53:01,116] jepsen worker 44 - jepsen.util 44 :invoke :write [14 24] INFO [2022-01-10 12:53:01,117] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:53:01,118] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:01,118] jepsen worker 53 - jepsen.util 53 :ok :read [17 14] INFO [2022-01-10 12:53:01,118] jepsen worker 37 - jepsen.util 37 :invoke :write [12 49] INFO [2022-01-10 12:53:01,119] jepsen worker 67 - jepsen.util 67 :ok :read [22 44] INFO [2022-01-10 12:53:01,119] jepsen worker 63 - jepsen.util 63 :invoke :write [21 34] INFO [2022-01-10 12:53:01,120] jepsen worker 88 - jepsen.util 88 :invoke :write [29 14] INFO [2022-01-10 12:53:01,121] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,122] jepsen worker 2 - jepsen.util 2 :invoke :write [0 17] INFO [2022-01-10 12:53:01,122] jepsen worker 64 - jepsen.util 64 :ok :read [21 41] INFO [2022-01-10 12:53:01,123] jepsen worker 75 - jepsen.util 75 :invoke :write [25 28] INFO [2022-01-10 12:53:01,123] jepsen worker 35 - jepsen.util 35 :ok :write [11 32] INFO [2022-01-10 12:53:01,124] jepsen worker 67 - jepsen.util 67 :invoke :write [22 19] INFO [2022-01-10 12:53:01,124] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:53:01,125] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:01,125] jepsen worker 7 - jepsen.util 7 :ok :write [2 35] INFO [2022-01-10 12:53:01,125] jepsen worker 48 - jepsen.util 48 :ok :read [16 23] INFO [2022-01-10 12:53:01,126] jepsen worker 53 - jepsen.util 53 :invoke :write [17 8] INFO [2022-01-10 12:53:01,126] jepsen worker 36 - jepsen.util 36 :ok :read [12 13] INFO [2022-01-10 12:53:01,127] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:01,128] jepsen worker 0 - jepsen.util 0 :invoke :write [0 5] INFO [2022-01-10 12:53:01,128] jepsen worker 7 - jepsen.util 7 :ok :read [2 35] INFO [2022-01-10 12:53:01,128] jepsen worker 35 - jepsen.util 35 :invoke :write [11 34] INFO [2022-01-10 12:53:01,129] jepsen worker 7 - jepsen.util 7 :invoke :write [2 19] INFO [2022-01-10 12:53:01,129] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:53:01,130] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,130] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:01,131] jepsen worker 39 - jepsen.util 39 :ok :read [13 14] INFO [2022-01-10 12:53:01,131] jepsen worker 48 - jepsen.util 48 :ok :read [16 23] INFO [2022-01-10 12:53:01,131] jepsen worker 41 - jepsen.util 41 :invoke :write [13 19] INFO [2022-01-10 12:53:01,131] jepsen worker 16 - jepsen.util 16 :ok :write [5 26] INFO [2022-01-10 12:53:01,132] jepsen worker 48 - jepsen.util 48 :invoke :read [16 nil] INFO [2022-01-10 12:53:01,132] jepsen worker 16 - jepsen.util 16 :invoke :write [5 11] INFO [2022-01-10 12:53:01,133] jepsen worker 48 - jepsen.util 48 :ok :read [16 23] INFO [2022-01-10 12:53:01,133] jepsen worker 36 - jepsen.util 36 :invoke :write [12 6] INFO [2022-01-10 12:53:01,134] jepsen worker 48 - jepsen.util 48 :invoke :write [16 19] INFO [2022-01-10 12:53:01,134] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,135] jepsen worker 64 - jepsen.util 64 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,135] jepsen worker 39 - jepsen.util 39 :ok :read [13 14] INFO [2022-01-10 12:53:01,136] jepsen worker 39 - jepsen.util 39 :invoke :write [13 0] INFO [2022-01-10 12:53:01,136] jepsen worker 64 - jepsen.util 64 :ok :read [21 41] INFO [2022-01-10 12:53:01,137] jepsen worker 64 - jepsen.util 64 :invoke :write [21 47] INFO [2022-01-10 12:53:01,140] jepsen worker 49 - jepsen.util 49 :ok :write [16 49] INFO [2022-01-10 12:53:01,140] jepsen worker 49 - jepsen.util 49 :invoke :write [16 24] INFO [2022-01-10 12:53:01,145] jepsen worker 81 - jepsen.util 81 :ok :write [27 26] INFO [2022-01-10 12:53:01,146] jepsen worker 81 - jepsen.util 81 :invoke :write [27 13] INFO [2022-01-10 12:53:01,151] jepsen worker 68 - jepsen.util 68 :ok :write [22 4] INFO [2022-01-10 12:53:01,152] jepsen worker 68 - jepsen.util 68 :invoke :write [22 49] INFO [2022-01-10 12:53:01,158] jepsen worker 34 - jepsen.util 34 :ok :write [11 25] INFO [2022-01-10 12:53:01,159] jepsen worker 34 - jepsen.util 34 :invoke :write [11 4] INFO [2022-01-10 12:53:01,176] jepsen worker 7 - jepsen.util 7 :ok :write [2 19] INFO [2022-01-10 12:53:01,176] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:01,177] jepsen worker 7 - jepsen.util 7 :ok :read [2 19] INFO [2022-01-10 12:53:01,178] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:01,179] jepsen worker 7 - jepsen.util 7 :ok :read [2 19] INFO [2022-01-10 12:53:01,179] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:01,180] jepsen worker 7 - jepsen.util 7 :ok :read [2 19] INFO [2022-01-10 12:53:01,180] jepsen worker 16 - jepsen.util 16 :ok :write [5 11] INFO [2022-01-10 12:53:01,180] jepsen worker 7 - jepsen.util 7 :invoke :write [2 40] INFO [2022-01-10 12:53:01,181] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,182] jepsen worker 16 - jepsen.util 16 :ok :read [5 11] INFO [2022-01-10 12:53:01,182] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,183] jepsen worker 16 - jepsen.util 16 :ok :read [5 11] INFO [2022-01-10 12:53:01,184] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,184] jepsen worker 16 - jepsen.util 16 :ok :read [5 11] INFO [2022-01-10 12:53:01,185] jepsen worker 16 - jepsen.util 16 :invoke :write [5 27] INFO [2022-01-10 12:53:01,205] jepsen worker 71 - jepsen.util 71 :ok :write [23 45] INFO [2022-01-10 12:53:01,206] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:01,207] jepsen worker 71 - jepsen.util 71 :ok :read [23 45] INFO [2022-01-10 12:53:01,208] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:01,209] jepsen worker 62 - jepsen.util 62 :ok :write [20 47] INFO [2022-01-10 12:53:01,209] jepsen worker 71 - jepsen.util 71 :ok :read [23 45] INFO [2022-01-10 12:53:01,209] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:01,210] jepsen worker 51 - jepsen.util 51 :ok :write [17 34] INFO [2022-01-10 12:53:01,210] jepsen worker 71 - jepsen.util 71 :invoke :write [23 28] INFO [2022-01-10 12:53:01,210] jepsen worker 62 - jepsen.util 62 :ok :read [20 47] INFO [2022-01-10 12:53:01,210] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:01,211] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:01,212] jepsen worker 51 - jepsen.util 51 :ok :read [17 34] INFO [2022-01-10 12:53:01,212] jepsen worker 62 - jepsen.util 62 :ok :read [20 47] INFO [2022-01-10 12:53:01,212] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:01,213] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:01,213] jepsen worker 3 - jepsen.util 3 :ok :write [1 31] INFO [2022-01-10 12:53:01,213] jepsen worker 51 - jepsen.util 51 :ok :read [17 34] INFO [2022-01-10 12:53:01,214] jepsen worker 62 - jepsen.util 62 :ok :read [20 47] INFO [2022-01-10 12:53:01,214] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:01,215] jepsen worker 3 - jepsen.util 3 :ok :read [1 31] INFO [2022-01-10 12:53:01,215] jepsen worker 51 - jepsen.util 51 :invoke :write [17 22] INFO [2022-01-10 12:53:01,215] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:01,216] jepsen worker 89 - jepsen.util 89 :ok :write [29 47] INFO [2022-01-10 12:53:01,216] jepsen worker 3 - jepsen.util 3 :invoke :write [1 47] INFO [2022-01-10 12:53:01,217] jepsen worker 62 - jepsen.util 62 :ok :read [20 47] INFO [2022-01-10 12:53:01,217] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,218] jepsen worker 62 - jepsen.util 62 :invoke :write [20 7] INFO [2022-01-10 12:53:01,218] jepsen worker 89 - jepsen.util 89 :ok :read [29 47] INFO [2022-01-10 12:53:01,219] jepsen worker 89 - jepsen.util 89 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,221] jepsen worker 85 - jepsen.util 85 :ok :write [28 6] INFO [2022-01-10 12:53:01,221] jepsen worker 89 - jepsen.util 89 :ok :read [29 47] INFO [2022-01-10 12:53:01,222] jepsen worker 89 - jepsen.util 89 :invoke :write [29 33] INFO [2022-01-10 12:53:01,222] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:53:01,224] jepsen worker 85 - jepsen.util 85 :ok :read [28 6] INFO [2022-01-10 12:53:01,224] jepsen worker 85 - jepsen.util 85 :invoke :write [28 6] INFO [2022-01-10 12:53:01,225] jepsen worker 78 - jepsen.util 78 :ok :write [26 4] INFO [2022-01-10 12:53:01,226] jepsen worker 21 - jepsen.util 21 :ok :write [7 26] INFO [2022-01-10 12:53:01,226] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:01,227] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,227] jepsen worker 78 - jepsen.util 78 :ok :read [26 4] INFO [2022-01-10 12:53:01,228] jepsen worker 21 - jepsen.util 21 :ok :read [7 26] INFO [2022-01-10 12:53:01,228] jepsen worker 78 - jepsen.util 78 :invoke :write [26 42] INFO [2022-01-10 12:53:01,228] jepsen worker 21 - jepsen.util 21 :invoke :write [7 32] INFO [2022-01-10 12:53:01,231] jepsen worker 43 - jepsen.util 43 :ok :write [14 40] INFO [2022-01-10 12:53:01,231] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,232] jepsen worker 43 - jepsen.util 43 :ok :read [14 40] INFO [2022-01-10 12:53:01,233] jepsen worker 43 - jepsen.util 43 :invoke :write [14 3] INFO [2022-01-10 12:53:01,233] jepsen worker 13 - jepsen.util 13 :ok :write [4 34] INFO [2022-01-10 12:53:01,234] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:01,235] jepsen worker 13 - jepsen.util 13 :ok :read [4 34] INFO [2022-01-10 12:53:01,235] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:01,236] jepsen worker 13 - jepsen.util 13 :ok :read [4 34] INFO [2022-01-10 12:53:01,237] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:01,237] jepsen worker 37 - jepsen.util 37 :ok :write [12 49] INFO [2022-01-10 12:53:01,238] jepsen worker 13 - jepsen.util 13 :ok :read [4 34] INFO [2022-01-10 12:53:01,238] jepsen worker 1 - jepsen.util 1 :ok :write [0 16] INFO [2022-01-10 12:53:01,238] jepsen worker 37 - jepsen.util 37 :invoke :read [12 nil] INFO [2022-01-10 12:53:01,239] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:01,239] jepsen worker 37 - jepsen.util 37 :ok :read [12 49] INFO [2022-01-10 12:53:01,239] jepsen worker 13 - jepsen.util 13 :invoke :write [4 47] INFO [2022-01-10 12:53:01,239] jepsen worker 1 - jepsen.util 1 :ok :read [0 16] INFO [2022-01-10 12:53:01,240] jepsen worker 37 - jepsen.util 37 :invoke :write [12 40] INFO [2022-01-10 12:53:01,240] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:01,241] jepsen worker 1 - jepsen.util 1 :ok :read [0 16] INFO [2022-01-10 12:53:01,242] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:01,242] jepsen worker 1 - jepsen.util 1 :ok :read [0 16] INFO [2022-01-10 12:53:01,243] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:01,244] jepsen worker 1 - jepsen.util 1 :ok :read [0 16] INFO [2022-01-10 12:53:01,244] jepsen worker 1 - jepsen.util 1 :invoke :write [0 35] INFO [2022-01-10 12:53:01,246] jepsen worker 40 - jepsen.util 40 :ok :write [13 48] INFO [2022-01-10 12:53:01,247] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,248] jepsen worker 40 - jepsen.util 40 :ok :read [13 48] INFO [2022-01-10 12:53:01,248] jepsen worker 40 - jepsen.util 40 :invoke :write [13 45] INFO [2022-01-10 12:53:01,251] jepsen worker 76 - jepsen.util 76 :ok :write [25 21] INFO [2022-01-10 12:53:01,251] jepsen worker 76 - jepsen.util 76 :invoke :write [25 44] INFO [2022-01-10 12:53:01,296] jepsen worker 55 - jepsen.util 55 :ok :write [18 38] INFO [2022-01-10 12:53:01,296] jepsen worker 55 - jepsen.util 55 :invoke :write [18 47] INFO [2022-01-10 12:53:01,297] jepsen worker 49 - jepsen.util 49 :ok :write [16 24] INFO [2022-01-10 12:53:01,297] jepsen worker 30 - jepsen.util 30 :ok :write [10 35] INFO [2022-01-10 12:53:01,298] jepsen worker 49 - jepsen.util 49 :invoke :write [16 25] INFO [2022-01-10 12:53:01,299] jepsen worker 30 - jepsen.util 30 :invoke :write [10 47] INFO [2022-01-10 12:53:01,304] jepsen worker 81 - jepsen.util 81 :ok :write [27 13] INFO [2022-01-10 12:53:01,304] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:01,305] jepsen worker 27 - jepsen.util 27 :ok :write [9 25] INFO [2022-01-10 12:53:01,305] jepsen worker 81 - jepsen.util 81 :ok :read [27 13] INFO [2022-01-10 12:53:01,306] jepsen worker 27 - jepsen.util 27 :invoke :write [9 21] INFO [2022-01-10 12:53:01,306] jepsen worker 3 - jepsen.util 3 :ok :write [1 47] INFO [2022-01-10 12:53:01,307] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:01,307] jepsen worker 18 - jepsen.util 18 :ok :write [6 22] INFO [2022-01-10 12:53:01,308] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:01,308] jepsen worker 81 - jepsen.util 81 :ok :read [27 13] INFO [2022-01-10 12:53:01,308] jepsen worker 18 - jepsen.util 18 :invoke :write [6 32] INFO [2022-01-10 12:53:01,309] jepsen worker 3 - jepsen.util 3 :ok :read [1 47] INFO [2022-01-10 12:53:01,309] jepsen worker 81 - jepsen.util 81 :invoke :write [27 9] INFO [2022-01-10 12:53:01,310] jepsen worker 3 - jepsen.util 3 :invoke :write [1 29] INFO [2022-01-10 12:53:01,310] jepsen worker 85 - jepsen.util 85 :ok :write [28 6] INFO [2022-01-10 12:53:01,310] jepsen worker 67 - jepsen.util 67 :ok :write [22 19] INFO [2022-01-10 12:53:01,311] jepsen worker 85 - jepsen.util 85 :invoke :write [28 20] INFO [2022-01-10 12:53:01,312] jepsen worker 67 - jepsen.util 67 :invoke :write [22 24] INFO [2022-01-10 12:53:01,314] jepsen worker 55 - jepsen.util 55 :ok :write [18 47] INFO [2022-01-10 12:53:01,315] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:01,315] jepsen worker 57 - jepsen.util 57 :ok :write [19 19] INFO [2022-01-10 12:53:01,316] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,316] jepsen worker 55 - jepsen.util 55 :ok :read [18 47] INFO [2022-01-10 12:53:01,317] jepsen worker 57 - jepsen.util 57 :ok :read [19 19] INFO [2022-01-10 12:53:01,317] jepsen worker 55 - jepsen.util 55 :invoke :write [18 24] INFO [2022-01-10 12:53:01,318] jepsen worker 57 - jepsen.util 57 :invoke :write [19 10] INFO [2022-01-10 12:53:01,318] jepsen worker 36 - jepsen.util 36 :ok :write [12 6] INFO [2022-01-10 12:53:01,319] jepsen worker 26 - jepsen.util 26 :ok :write [8 17] INFO [2022-01-10 12:53:01,319] jepsen worker 34 - jepsen.util 34 :ok :write [11 4] INFO [2022-01-10 12:53:01,319] jepsen worker 36 - jepsen.util 36 :invoke :write [12 27] INFO [2022-01-10 12:53:01,321] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:01,321] jepsen worker 26 - jepsen.util 26 :invoke :write [8 14] INFO [2022-01-10 12:53:01,322] jepsen worker 34 - jepsen.util 34 :ok :read [11 4] INFO [2022-01-10 12:53:01,323] jepsen worker 34 - jepsen.util 34 :invoke :write [11 9] INFO [2022-01-10 12:53:01,324] jepsen worker 3 - jepsen.util 3 :ok :write [1 29] INFO [2022-01-10 12:53:01,324] jepsen worker 7 - jepsen.util 7 :ok :write [2 40] INFO [2022-01-10 12:53:01,324] jepsen worker 3 - jepsen.util 3 :invoke :write [1 26] INFO [2022-01-10 12:53:01,325] jepsen worker 7 - jepsen.util 7 :invoke :write [2 13] INFO [2022-01-10 12:53:01,328] jepsen worker 16 - jepsen.util 16 :ok :write [5 27] INFO [2022-01-10 12:53:01,328] jepsen worker 85 - jepsen.util 85 :ok :write [28 20] INFO [2022-01-10 12:53:01,329] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:53:01,330] jepsen worker 16 - jepsen.util 16 :invoke :write [5 37] INFO [2022-01-10 12:53:01,330] jepsen worker 85 - jepsen.util 85 :ok :read [28 20] INFO [2022-01-10 12:53:01,331] jepsen worker 85 - jepsen.util 85 :invoke :write [28 42] INFO [2022-01-10 12:53:01,336] jepsen worker 87 - jepsen.util 87 :ok :write [29 8] INFO [2022-01-10 12:53:01,337] jepsen worker 87 - jepsen.util 87 :invoke :write [29 48] INFO [2022-01-10 12:53:01,340] jepsen worker 1 - jepsen.util 1 :ok :write [0 35] INFO [2022-01-10 12:53:01,341] jepsen worker 1 - jepsen.util 1 :invoke :write [0 41] INFO [2022-01-10 12:53:01,341] jepsen worker 36 - jepsen.util 36 :ok :write [12 27] INFO [2022-01-10 12:53:01,342] jepsen worker 36 - jepsen.util 36 :invoke :write [12 44] INFO [2022-01-10 12:53:01,346] jepsen worker 40 - jepsen.util 40 :ok :write [13 45] INFO [2022-01-10 12:53:01,347] jepsen worker 40 - jepsen.util 40 :invoke :write [13 16] INFO [2022-01-10 12:53:01,348] jepsen worker 3 - jepsen.util 3 :ok :write [1 26] INFO [2022-01-10 12:53:01,349] jepsen worker 3 - jepsen.util 3 :invoke :write [1 0] INFO [2022-01-10 12:53:01,353] jepsen worker 85 - jepsen.util 85 :ok :write [28 42] INFO [2022-01-10 12:53:01,354] jepsen worker 85 - jepsen.util 85 :invoke :write [28 35] INFO [2022-01-10 12:53:01,359] jepsen worker 36 - jepsen.util 36 :ok :write [12 44] INFO [2022-01-10 12:53:01,359] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:01,360] jepsen worker 36 - jepsen.util 36 :ok :read [12 44] INFO [2022-01-10 12:53:01,361] jepsen worker 36 - jepsen.util 36 :invoke :write [12 36] INFO [2022-01-10 12:53:01,364] jepsen worker 3 - jepsen.util 3 :ok :write [1 0] INFO [2022-01-10 12:53:01,365] jepsen worker 3 - jepsen.util 3 :invoke :write [1 45] INFO [2022-01-10 12:53:01,366] jepsen worker 71 - jepsen.util 71 :ok :write [23 28] INFO [2022-01-10 12:53:01,367] jepsen worker 71 - jepsen.util 71 :invoke :write [23 25] INFO [2022-01-10 12:53:01,368] jepsen worker 62 - jepsen.util 62 :ok :write [20 7] INFO [2022-01-10 12:53:01,368] jepsen worker 85 - jepsen.util 85 :ok :write [28 35] INFO [2022-01-10 12:53:01,368] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:01,369] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:53:01,369] jepsen worker 62 - jepsen.util 62 :ok :read [20 7] INFO [2022-01-10 12:53:01,370] jepsen worker 62 - jepsen.util 62 :invoke :write [20 20] INFO [2022-01-10 12:53:01,371] jepsen worker 85 - jepsen.util 85 :ok :read [28 35] INFO [2022-01-10 12:53:01,371] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:53:01,372] jepsen worker 27 - jepsen.util 27 :ok :write [9 21] INFO [2022-01-10 12:53:01,372] jepsen worker 51 - jepsen.util 51 :ok :write [17 22] INFO [2022-01-10 12:53:01,372] jepsen worker 85 - jepsen.util 85 :ok :read [28 35] INFO [2022-01-10 12:53:01,373] jepsen worker 51 - jepsen.util 51 :invoke :write [17 1] INFO [2022-01-10 12:53:01,374] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,374] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:53:01,375] jepsen worker 36 - jepsen.util 36 :ok :write [12 36] INFO [2022-01-10 12:53:01,375] jepsen worker 21 - jepsen.util 21 :ok :write [7 32] INFO [2022-01-10 12:53:01,375] jepsen worker 27 - jepsen.util 27 :ok :read [9 21] INFO [2022-01-10 12:53:01,376] jepsen worker 85 - jepsen.util 85 :ok :read [28 35] INFO [2022-01-10 12:53:01,376] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,377] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,377] jepsen worker 21 - jepsen.util 21 :ok :read [7 32] INFO [2022-01-10 12:53:01,378] jepsen worker 36 - jepsen.util 36 :invoke :write [12 32] INFO [2022-01-10 12:53:01,378] jepsen worker 78 - jepsen.util 78 :ok :write [26 42] INFO [2022-01-10 12:53:01,378] jepsen worker 27 - jepsen.util 27 :ok :read [9 21] INFO [2022-01-10 12:53:01,379] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,380] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,380] jepsen worker 27 - jepsen.util 27 :ok :read [9 21] INFO [2022-01-10 12:53:01,380] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:53:01,381] jepsen worker 3 - jepsen.util 3 :ok :write [1 45] INFO [2022-01-10 12:53:01,381] jepsen worker 21 - jepsen.util 21 :ok :read [7 32] INFO [2022-01-10 12:53:01,382] jepsen worker 85 - jepsen.util 85 :ok :read [28 35] INFO [2022-01-10 12:53:01,382] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,383] jepsen worker 13 - jepsen.util 13 :ok :write [4 47] INFO [2022-01-10 12:53:01,383] jepsen worker 27 - jepsen.util 27 :ok :read [9 21] INFO [2022-01-10 12:53:01,383] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,384] jepsen worker 21 - jepsen.util 21 :ok :read [7 32] INFO [2022-01-10 12:53:01,385] jepsen worker 85 - jepsen.util 85 :invoke :read [28 nil] INFO [2022-01-10 12:53:01,386] jepsen worker 43 - jepsen.util 43 :ok :write [14 3] INFO [2022-01-10 12:53:01,387] jepsen worker 85 - jepsen.util 85 :ok :read [28 35] INFO [2022-01-10 12:53:01,389] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:01,391] jepsen worker 78 - jepsen.util 78 :ok :read [26 42] INFO [2022-01-10 12:53:01,391] jepsen worker 85 - jepsen.util 85 :invoke :write [28 15] INFO [2022-01-10 12:53:01,392] jepsen worker 3 - jepsen.util 3 :invoke :write [1 20] INFO [2022-01-10 12:53:01,393] jepsen worker 78 - jepsen.util 78 :invoke :write [26 11] INFO [2022-01-10 12:53:01,394] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:01,394] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,395] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,395] jepsen worker 13 - jepsen.util 13 :ok :read [4 47] INFO [2022-01-10 12:53:01,395] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,395] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:01,396] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,396] jepsen worker 36 - jepsen.util 36 :ok :write [12 32] INFO [2022-01-10 12:53:01,396] jepsen worker 21 - jepsen.util 21 :ok :read [7 32] INFO [2022-01-10 12:53:01,397] jepsen worker 27 - jepsen.util 27 :ok :read [9 21] INFO [2022-01-10 12:53:01,397] jepsen worker 76 - jepsen.util 76 :ok :write [25 44] INFO [2022-01-10 12:53:01,397] jepsen worker 13 - jepsen.util 13 :invoke :write [4 1] INFO [2022-01-10 12:53:01,397] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:01,398] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:01,398] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:01,398] jepsen worker 76 - jepsen.util 76 :ok :read [25 44] INFO [2022-01-10 12:53:01,399] jepsen worker 36 - jepsen.util 36 :ok :read [12 32] INFO [2022-01-10 12:53:01,399] jepsen worker 21 - jepsen.util 21 :invoke :write [7 42] INFO [2022-01-10 12:53:01,400] jepsen worker 27 - jepsen.util 27 :invoke :write [9 22] INFO [2022-01-10 12:53:01,400] jepsen worker 71 - jepsen.util 71 :ok :write [23 25] INFO [2022-01-10 12:53:01,400] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:01,401] jepsen worker 87 - jepsen.util 87 :ok :write [29 48] INFO [2022-01-10 12:53:01,401] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,402] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,402] jepsen worker 76 - jepsen.util 76 :ok :read [25 44] INFO [2022-01-10 12:53:01,402] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:01,403] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:01,403] jepsen worker 87 - jepsen.util 87 :ok :read [29 48] INFO [2022-01-10 12:53:01,403] jepsen worker 76 - jepsen.util 76 :invoke :write [25 41] INFO [2022-01-10 12:53:01,403] jepsen worker 71 - jepsen.util 71 :ok :read [23 25] INFO [2022-01-10 12:53:01,404] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,404] jepsen worker 87 - jepsen.util 87 :invoke :write [29 43] INFO [2022-01-10 12:53:01,405] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:01,405] jepsen worker 71 - jepsen.util 71 :invoke :write [23 41] INFO [2022-01-10 12:53:01,405] jepsen worker 36 - jepsen.util 36 :invoke :write [12 14] INFO [2022-01-10 12:53:01,406] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,407] jepsen worker 1 - jepsen.util 1 :ok :write [0 41] INFO [2022-01-10 12:53:01,407] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:01,408] jepsen worker 51 - jepsen.util 51 :ok :write [17 1] INFO [2022-01-10 12:53:01,408] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:01,408] jepsen worker 49 - jepsen.util 49 :ok :write [16 25] INFO [2022-01-10 12:53:01,409] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,410] jepsen worker 1 - jepsen.util 1 :ok :read [0 41] INFO [2022-01-10 12:53:01,410] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:01,410] jepsen worker 49 - jepsen.util 49 :invoke :write [16 14] INFO [2022-01-10 12:53:01,411] jepsen worker 51 - jepsen.util 51 :invoke :write [17 29] INFO [2022-01-10 12:53:01,412] jepsen worker 43 - jepsen.util 43 :invoke :write [14 28] INFO [2022-01-10 12:53:01,413] jepsen worker 1 - jepsen.util 1 :invoke :write [0 41] INFO [2022-01-10 12:53:01,413] jepsen worker 78 - jepsen.util 78 :ok :write [26 11] INFO [2022-01-10 12:53:01,413] jepsen worker 78 - jepsen.util 78 :invoke :write [26 44] INFO [2022-01-10 12:53:01,414] jepsen worker 3 - jepsen.util 3 :ok :write [1 20] INFO [2022-01-10 12:53:01,415] jepsen worker 3 - jepsen.util 3 :invoke :write [1 12] INFO [2022-01-10 12:53:01,416] jepsen worker 81 - jepsen.util 81 :ok :write [27 9] INFO [2022-01-10 12:53:01,417] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:01,418] jepsen worker 40 - jepsen.util 40 :ok :write [13 16] INFO [2022-01-10 12:53:01,418] jepsen worker 81 - jepsen.util 81 :ok :read [27 9] INFO [2022-01-10 12:53:01,419] jepsen worker 40 - jepsen.util 40 :invoke :write [13 30] INFO [2022-01-10 12:53:01,419] jepsen worker 81 - jepsen.util 81 :invoke :write [27 4] INFO [2022-01-10 12:53:01,423] jepsen worker 67 - jepsen.util 67 :ok :write [22 24] INFO [2022-01-10 12:53:01,423] jepsen worker 13 - jepsen.util 13 :ok :write [4 1] INFO [2022-01-10 12:53:01,424] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:01,425] jepsen worker 67 - jepsen.util 67 :invoke :write [22 24] INFO [2022-01-10 12:53:01,426] jepsen worker 13 - jepsen.util 13 :ok :read [4 1] INFO [2022-01-10 12:53:01,427] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:01,428] jepsen worker 13 - jepsen.util 13 :ok :read [4 1] INFO [2022-01-10 12:53:01,429] jepsen worker 13 - jepsen.util 13 :invoke :write [4 7] INFO [2022-01-10 12:53:01,431] jepsen worker 34 - jepsen.util 34 :ok :write [11 9] INFO [2022-01-10 12:53:01,431] jepsen worker 47 - jepsen.util 47 :ok :write [15 32] INFO [2022-01-10 12:53:01,432] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:01,433] jepsen worker 47 - jepsen.util 47 :invoke :write [15 29] INFO [2022-01-10 12:53:01,434] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:53:01,434] jepsen worker 34 - jepsen.util 34 :invoke :write [11 9] INFO [2022-01-10 12:53:01,436] jepsen worker 84 - jepsen.util 84 :ok :write [28 31] INFO [2022-01-10 12:53:01,436] jepsen worker 87 - jepsen.util 87 :ok :write [29 43] INFO [2022-01-10 12:53:01,436] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:01,437] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,437] jepsen worker 84 - jepsen.util 84 :ok :read [28 31] INFO [2022-01-10 12:53:01,438] jepsen worker 84 - jepsen.util 84 :invoke :write [28 16] INFO [2022-01-10 12:53:01,439] jepsen worker 87 - jepsen.util 87 :ok :read [29 43] INFO [2022-01-10 12:53:01,439] jepsen worker 55 - jepsen.util 55 :ok :write [18 24] INFO [2022-01-10 12:53:01,439] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,440] jepsen worker 55 - jepsen.util 55 :invoke :write [18 24] INFO [2022-01-10 12:53:01,440] jepsen worker 7 - jepsen.util 7 :ok :write [2 13] INFO [2022-01-10 12:53:01,440] jepsen worker 87 - jepsen.util 87 :ok :read [29 43] INFO [2022-01-10 12:53:01,441] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:01,441] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,442] jepsen worker 7 - jepsen.util 7 :ok :read [2 13] INFO [2022-01-10 12:53:01,443] jepsen worker 87 - jepsen.util 87 :ok :read [29 43] INFO [2022-01-10 12:53:01,443] jepsen worker 7 - jepsen.util 7 :invoke :write [2 2] INFO [2022-01-10 12:53:01,443] jepsen worker 36 - jepsen.util 36 :ok :write [12 14] INFO [2022-01-10 12:53:01,444] jepsen worker 1 - jepsen.util 1 :ok :write [0 41] INFO [2022-01-10 12:53:01,444] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,445] jepsen worker 36 - jepsen.util 36 :invoke :write [12 8] INFO [2022-01-10 12:53:01,445] jepsen worker 87 - jepsen.util 87 :ok :read [29 43] INFO [2022-01-10 12:53:01,445] jepsen worker 1 - jepsen.util 1 :invoke :write [0 13] INFO [2022-01-10 12:53:01,446] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,447] jepsen worker 87 - jepsen.util 87 :ok :read [29 43] INFO [2022-01-10 12:53:01,448] jepsen worker 87 - jepsen.util 87 :invoke :write [29 44] INFO [2022-01-10 12:53:01,448] jepsen worker 11 - jepsen.util 11 :ok :write [3 30] INFO [2022-01-10 12:53:01,449] jepsen worker 11 - jepsen.util 11 :invoke :write [3 2] INFO [2022-01-10 12:53:01,449] jepsen worker 16 - jepsen.util 16 :ok :write [5 37] INFO [2022-01-10 12:53:01,449] jepsen worker 27 - jepsen.util 27 :ok :write [9 22] INFO [2022-01-10 12:53:01,450] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,451] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,451] jepsen worker 27 - jepsen.util 27 :ok :read [9 22] INFO [2022-01-10 12:53:01,451] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,452] jepsen worker 16 - jepsen.util 16 :ok :read [5 37] INFO [2022-01-10 12:53:01,452] jepsen worker 27 - jepsen.util 27 :ok :read [9 22] INFO [2022-01-10 12:53:01,452] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,453] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,453] jepsen worker 16 - jepsen.util 16 :ok :read [5 37] INFO [2022-01-10 12:53:01,454] jepsen worker 27 - jepsen.util 27 :ok :read [9 22] INFO [2022-01-10 12:53:01,454] jepsen worker 62 - jepsen.util 62 :ok :write [20 20] INFO [2022-01-10 12:53:01,454] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,455] jepsen worker 27 - jepsen.util 27 :invoke :write [9 48] INFO [2022-01-10 12:53:01,455] jepsen worker 16 - jepsen.util 16 :ok :read [5 37] INFO [2022-01-10 12:53:01,455] jepsen worker 62 - jepsen.util 62 :invoke :write [20 39] INFO [2022-01-10 12:53:01,456] jepsen worker 16 - jepsen.util 16 :invoke :write [5 14] INFO [2022-01-10 12:53:01,456] jepsen worker 74 - jepsen.util 74 :ok :write [24 39] INFO [2022-01-10 12:53:01,457] jepsen worker 74 - jepsen.util 74 :invoke :write [24 17] INFO [2022-01-10 12:53:01,461] jepsen worker 40 - jepsen.util 40 :ok :write [13 30] INFO [2022-01-10 12:53:01,461] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,462] jepsen worker 40 - jepsen.util 40 :ok :read [13 30] INFO [2022-01-10 12:53:01,463] jepsen worker 65 - jepsen.util 65 :ok :write [21 7] INFO [2022-01-10 12:53:01,463] jepsen worker 40 - jepsen.util 40 :invoke :write [13 13] INFO [2022-01-10 12:53:01,463] jepsen worker 21 - jepsen.util 21 :ok :write [7 42] INFO [2022-01-10 12:53:01,463] jepsen worker 65 - jepsen.util 65 :invoke :write [21 35] INFO [2022-01-10 12:53:01,464] jepsen worker 21 - jepsen.util 21 :invoke :write [7 18] INFO [2022-01-10 12:53:01,468] jepsen worker 30 - jepsen.util 30 :ok :write [10 47] INFO [2022-01-10 12:53:01,469] jepsen worker 30 - jepsen.util 30 :invoke :write [10 4] INFO [2022-01-10 12:53:01,474] jepsen worker 18 - jepsen.util 18 :ok :write [6 32] INFO [2022-01-10 12:53:01,475] jepsen worker 43 - jepsen.util 43 :ok :write [14 28] INFO [2022-01-10 12:53:01,475] jepsen worker 18 - jepsen.util 18 :invoke :write [6 7] INFO [2022-01-10 12:53:01,475] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,476] jepsen worker 43 - jepsen.util 43 :ok :read [14 28] INFO [2022-01-10 12:53:01,477] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,478] jepsen worker 43 - jepsen.util 43 :ok :read [14 28] INFO [2022-01-10 12:53:01,479] jepsen worker 43 - jepsen.util 43 :invoke :write [14 2] INFO [2022-01-10 12:53:01,480] jepsen worker 57 - jepsen.util 57 :ok :write [19 10] INFO [2022-01-10 12:53:01,481] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,481] jepsen worker 76 - jepsen.util 76 :ok :write [25 41] INFO [2022-01-10 12:53:01,482] jepsen worker 57 - jepsen.util 57 :ok :read [19 10] INFO [2022-01-10 12:53:01,482] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:01,483] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,483] jepsen worker 76 - jepsen.util 76 :ok :read [25 41] INFO [2022-01-10 12:53:01,484] jepsen worker 57 - jepsen.util 57 :ok :read [19 10] INFO [2022-01-10 12:53:01,484] jepsen worker 76 - jepsen.util 76 :invoke :write [25 44] INFO [2022-01-10 12:53:01,484] jepsen worker 26 - jepsen.util 26 :ok :write [8 14] INFO [2022-01-10 12:53:01,485] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,485] jepsen worker 26 - jepsen.util 26 :invoke :write [8 7] INFO [2022-01-10 12:53:01,486] jepsen worker 57 - jepsen.util 57 :ok :read [19 10] INFO [2022-01-10 12:53:01,487] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,487] jepsen worker 47 - jepsen.util 47 :ok :write [15 29] INFO [2022-01-10 12:53:01,488] jepsen worker 49 - jepsen.util 49 :ok :write [16 14] INFO [2022-01-10 12:53:01,488] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:01,488] jepsen worker 57 - jepsen.util 57 :ok :read [19 10] INFO [2022-01-10 12:53:01,489] jepsen worker 47 - jepsen.util 47 :ok :read [15 29] INFO [2022-01-10 12:53:01,489] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,490] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:01,490] jepsen worker 57 - jepsen.util 57 :ok :read [19 10] INFO [2022-01-10 12:53:01,491] jepsen worker 27 - jepsen.util 27 :ok :write [9 48] INFO [2022-01-10 12:53:01,491] jepsen worker 49 - jepsen.util 49 :invoke :write [16 12] INFO [2022-01-10 12:53:01,491] jepsen worker 47 - jepsen.util 47 :ok :read [15 29] INFO [2022-01-10 12:53:01,492] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,492] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:01,493] jepsen worker 27 - jepsen.util 27 :ok :read [9 48] INFO [2022-01-10 12:53:01,493] jepsen worker 47 - jepsen.util 47 :ok :read [15 29] INFO [2022-01-10 12:53:01,494] jepsen worker 57 - jepsen.util 57 :invoke :write [19 20] INFO [2022-01-10 12:53:01,494] jepsen worker 81 - jepsen.util 81 :ok :write [27 4] INFO [2022-01-10 12:53:01,494] jepsen worker 27 - jepsen.util 27 :invoke :write [9 24] INFO [2022-01-10 12:53:01,495] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:01,496] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:01,496] jepsen worker 81 - jepsen.util 81 :ok :read [27 4] INFO [2022-01-10 12:53:01,497] jepsen worker 47 - jepsen.util 47 :ok :read [15 29] INFO [2022-01-10 12:53:01,497] jepsen worker 81 - jepsen.util 81 :invoke :write [27 4] INFO [2022-01-10 12:53:01,497] jepsen worker 47 - jepsen.util 47 :invoke :write [15 0] INFO [2022-01-10 12:53:01,501] jepsen worker 67 - jepsen.util 67 :ok :write [22 24] INFO [2022-01-10 12:53:01,501] jepsen worker 67 - jepsen.util 67 :invoke :write [22 35] INFO [2022-01-10 12:53:01,502] jepsen worker 74 - jepsen.util 74 :ok :write [24 17] INFO [2022-01-10 12:53:01,502] jepsen worker 74 - jepsen.util 74 :invoke :write [24 21] INFO [2022-01-10 12:53:01,507] jepsen worker 65 - jepsen.util 65 :ok :write [21 35] INFO [2022-01-10 12:53:01,508] jepsen worker 65 - jepsen.util 65 :invoke :write [21 37] INFO [2022-01-10 12:53:01,508] jepsen worker 27 - jepsen.util 27 :ok :write [9 24] INFO [2022-01-10 12:53:01,508] jepsen worker 34 - jepsen.util 34 :ok :write [11 9] INFO [2022-01-10 12:53:01,509] jepsen worker 27 - jepsen.util 27 :invoke :write [9 42] INFO [2022-01-10 12:53:01,509] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:01,511] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:53:01,511] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:01,512] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:53:01,513] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:01,514] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:53:01,514] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:01,515] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:53:01,515] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:01,516] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:53:01,517] jepsen worker 34 - jepsen.util 34 :invoke :write [11 43] INFO [2022-01-10 12:53:01,542] jepsen worker 7 - jepsen.util 7 :ok :write [2 2] INFO [2022-01-10 12:53:01,543] jepsen worker 7 - jepsen.util 7 :invoke :write [2 15] INFO [2022-01-10 12:53:01,548] jepsen worker 20 - jepsen.util 20 :ok :write [6 31] INFO [2022-01-10 12:53:01,548] jepsen worker 16 - jepsen.util 16 :ok :write [5 14] INFO [2022-01-10 12:53:01,549] jepsen worker 20 - jepsen.util 20 :invoke :write [6 2] INFO [2022-01-10 12:53:01,550] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,551] jepsen worker 16 - jepsen.util 16 :ok :read [5 14] INFO [2022-01-10 12:53:01,551] jepsen worker 16 - jepsen.util 16 :invoke :write [5 45] INFO [2022-01-10 12:53:01,553] jepsen worker 27 - jepsen.util 27 :ok :write [9 42] INFO [2022-01-10 12:53:01,554] jepsen worker 27 - jepsen.util 27 :invoke :write [9 16] INFO [2022-01-10 12:53:01,555] jepsen worker 57 - jepsen.util 57 :ok :write [19 20] INFO [2022-01-10 12:53:01,555] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,557] jepsen worker 57 - jepsen.util 57 :ok :read [19 20] INFO [2022-01-10 12:53:01,557] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,558] jepsen worker 57 - jepsen.util 57 :ok :read [19 20] INFO [2022-01-10 12:53:01,559] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,560] jepsen worker 26 - jepsen.util 26 :ok :write [8 7] INFO [2022-01-10 12:53:01,560] jepsen worker 57 - jepsen.util 57 :ok :read [19 20] INFO [2022-01-10 12:53:01,560] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:53:01,561] jepsen worker 57 - jepsen.util 57 :invoke :write [19 47] INFO [2022-01-10 12:53:01,564] jepsen worker 26 - jepsen.util 26 :ok :read [8 7] INFO [2022-01-10 12:53:01,565] jepsen worker 47 - jepsen.util 47 :ok :write [15 0] INFO [2022-01-10 12:53:01,565] jepsen worker 26 - jepsen.util 26 :invoke :write [8 13] INFO [2022-01-10 12:53:01,565] jepsen worker 47 - jepsen.util 47 :invoke :write [15 31] INFO [2022-01-10 12:53:01,578] jepsen worker 71 - jepsen.util 71 :ok :write [23 41] INFO [2022-01-10 12:53:01,578] jepsen worker 74 - jepsen.util 74 :ok :write [24 21] INFO [2022-01-10 12:53:01,578] jepsen worker 71 - jepsen.util 71 :invoke :write [23 44] INFO [2022-01-10 12:53:01,579] jepsen worker 27 - jepsen.util 27 :ok :write [9 16] INFO [2022-01-10 12:53:01,579] jepsen worker 74 - jepsen.util 74 :invoke :write [24 45] INFO [2022-01-10 12:53:01,579] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,581] jepsen worker 27 - jepsen.util 27 :ok :read [9 16] INFO [2022-01-10 12:53:01,581] jepsen worker 55 - jepsen.util 55 :ok :write [18 24] INFO [2022-01-10 12:53:01,581] jepsen worker 55 - jepsen.util 55 :invoke :write [18 21] INFO [2022-01-10 12:53:01,582] jepsen worker 27 - jepsen.util 27 :invoke :write [9 9] INFO [2022-01-10 12:53:01,585] jepsen worker 51 - jepsen.util 51 :ok :write [17 29] INFO [2022-01-10 12:53:01,586] jepsen worker 51 - jepsen.util 51 :invoke :write [17 49] INFO [2022-01-10 12:53:01,587] jepsen worker 65 - jepsen.util 65 :ok :write [21 37] INFO [2022-01-10 12:53:01,587] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,588] jepsen worker 1 - jepsen.util 1 :ok :write [0 13] INFO [2022-01-10 12:53:01,588] jepsen worker 1 - jepsen.util 1 :invoke :write [0 48] INFO [2022-01-10 12:53:01,588] jepsen worker 65 - jepsen.util 65 :ok :read [21 37] INFO [2022-01-10 12:53:01,590] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,590] jepsen worker 65 - jepsen.util 65 :ok :read [21 37] INFO [2022-01-10 12:53:01,591] jepsen worker 65 - jepsen.util 65 :invoke :write [21 2] INFO [2022-01-10 12:53:01,592] jepsen worker 78 - jepsen.util 78 :ok :write [26 44] INFO [2022-01-10 12:53:01,592] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:01,593] jepsen worker 87 - jepsen.util 87 :ok :write [29 44] INFO [2022-01-10 12:53:01,594] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,594] jepsen worker 78 - jepsen.util 78 :ok :read [26 44] INFO [2022-01-10 12:53:01,595] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:01,596] jepsen worker 20 - jepsen.util 20 :ok :write [6 2] INFO [2022-01-10 12:53:01,596] jepsen worker 78 - jepsen.util 78 :ok :read [26 44] INFO [2022-01-10 12:53:01,597] jepsen worker 20 - jepsen.util 20 :invoke :read [6 nil] INFO [2022-01-10 12:53:01,597] jepsen worker 3 - jepsen.util 3 :ok :write [1 12] INFO [2022-01-10 12:53:01,597] jepsen worker 78 - jepsen.util 78 :invoke :write [26 40] INFO [2022-01-10 12:53:01,598] jepsen worker 20 - jepsen.util 20 :ok :read [6 2] INFO [2022-01-10 12:53:01,598] jepsen worker 3 - jepsen.util 3 :invoke :write [1 6] INFO [2022-01-10 12:53:01,598] jepsen worker 20 - jepsen.util 20 :invoke :read [6 nil] INFO [2022-01-10 12:53:01,599] jepsen worker 62 - jepsen.util 62 :ok :write [20 39] INFO [2022-01-10 12:53:01,599] jepsen worker 20 - jepsen.util 20 :ok :read [6 2] INFO [2022-01-10 12:53:01,600] jepsen worker 62 - jepsen.util 62 :invoke :write [20 23] INFO [2022-01-10 12:53:01,600] jepsen worker 20 - jepsen.util 20 :invoke :write [6 24] INFO [2022-01-10 12:53:01,602] jepsen worker 81 - jepsen.util 81 :ok :write [27 4] INFO [2022-01-10 12:53:01,603] jepsen worker 81 - jepsen.util 81 :invoke :write [27 23] INFO [2022-01-10 12:53:01,604] jepsen worker 13 - jepsen.util 13 :ok :write [4 7] INFO [2022-01-10 12:53:01,604] jepsen worker 27 - jepsen.util 27 :ok :write [9 9] INFO [2022-01-10 12:53:01,604] jepsen worker 57 - jepsen.util 57 :ok :write [19 47] INFO [2022-01-10 12:53:01,605] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:01,605] jepsen worker 40 - jepsen.util 40 :ok :write [13 13] INFO [2022-01-10 12:53:01,606] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,607] jepsen worker 57 - jepsen.util 57 :ok :read [19 47] INFO [2022-01-10 12:53:01,607] jepsen worker 27 - jepsen.util 27 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,608] jepsen worker 27 - jepsen.util 27 :ok :read [9 9] INFO [2022-01-10 12:53:01,608] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,608] jepsen worker 67 - jepsen.util 67 :ok :write [22 35] INFO [2022-01-10 12:53:01,609] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,609] jepsen worker 40 - jepsen.util 40 :ok :read [13 13] INFO [2022-01-10 12:53:01,609] jepsen worker 27 - jepsen.util 27 :invoke :write [9 26] INFO [2022-01-10 12:53:01,609] jepsen worker 57 - jepsen.util 57 :ok :read [19 47] INFO [2022-01-10 12:53:01,610] jepsen worker 40 - jepsen.util 40 :invoke :write [13 44] INFO [2022-01-10 12:53:01,611] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,611] jepsen worker 67 - jepsen.util 67 :invoke :write [22 28] INFO [2022-01-10 12:53:01,612] jepsen worker 57 - jepsen.util 57 :ok :read [19 47] INFO [2022-01-10 12:53:01,613] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,614] jepsen worker 57 - jepsen.util 57 :ok :read [19 47] INFO [2022-01-10 12:53:01,614] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,615] jepsen worker 23 - jepsen.util 23 :ok :write [7 12] INFO [2022-01-10 12:53:01,616] jepsen worker 57 - jepsen.util 57 :ok :read [19 47] INFO [2022-01-10 12:53:01,616] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,616] jepsen worker 26 - jepsen.util 26 :ok :write [8 13] INFO [2022-01-10 12:53:01,616] jepsen worker 84 - jepsen.util 84 :ok :write [28 16] INFO [2022-01-10 12:53:01,616] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,617] jepsen worker 23 - jepsen.util 23 :ok :read [7 12] INFO [2022-01-10 12:53:01,617] jepsen worker 84 - jepsen.util 84 :invoke :write [28 18] INFO [2022-01-10 12:53:01,617] jepsen worker 57 - jepsen.util 57 :ok :read [19 47] INFO [2022-01-10 12:53:01,618] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,618] jepsen worker 26 - jepsen.util 26 :invoke :write [8 16] INFO [2022-01-10 12:53:01,619] jepsen worker 23 - jepsen.util 23 :ok :read [7 12] INFO [2022-01-10 12:53:01,619] jepsen worker 57 - jepsen.util 57 :invoke :write [19 37] INFO [2022-01-10 12:53:01,620] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,621] jepsen worker 34 - jepsen.util 34 :ok :write [11 43] INFO [2022-01-10 12:53:01,622] jepsen worker 23 - jepsen.util 23 :ok :read [7 12] INFO [2022-01-10 12:53:01,622] jepsen worker 43 - jepsen.util 43 :ok :write [14 2] INFO [2022-01-10 12:53:01,622] jepsen worker 36 - jepsen.util 36 :ok :write [12 8] INFO [2022-01-10 12:53:01,622] jepsen worker 34 - jepsen.util 34 :invoke :write [11 27] INFO [2022-01-10 12:53:01,622] jepsen worker 87 - jepsen.util 87 :ok :read [29 44] INFO [2022-01-10 12:53:01,622] jepsen worker 13 - jepsen.util 13 :ok :read [4 7] INFO [2022-01-10 12:53:01,623] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,623] jepsen worker 47 - jepsen.util 47 :ok :write [15 31] INFO [2022-01-10 12:53:01,623] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,624] jepsen worker 43 - jepsen.util 43 :ok :read [14 2] INFO [2022-01-10 12:53:01,624] jepsen worker 23 - jepsen.util 23 :ok :read [7 12] INFO [2022-01-10 12:53:01,624] jepsen worker 47 - jepsen.util 47 :invoke :write [15 6] INFO [2022-01-10 12:53:01,625] jepsen worker 36 - jepsen.util 36 :invoke :write [12 40] INFO [2022-01-10 12:53:01,626] jepsen worker 23 - jepsen.util 23 :invoke :write [7 16] INFO [2022-01-10 12:53:01,627] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:01,628] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,628] jepsen worker 11 - jepsen.util 11 :ok :write [3 2] INFO [2022-01-10 12:53:01,628] jepsen worker 76 - jepsen.util 76 :ok :write [25 44] INFO [2022-01-10 12:53:01,629] jepsen worker 74 - jepsen.util 74 :ok :write [24 45] INFO [2022-01-10 12:53:01,630] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,630] jepsen worker 13 - jepsen.util 13 :ok :read [4 7] INFO [2022-01-10 12:53:01,630] jepsen worker 87 - jepsen.util 87 :ok :read [29 44] INFO [2022-01-10 12:53:01,631] jepsen worker 76 - jepsen.util 76 :invoke :write [25 43] INFO [2022-01-10 12:53:01,631] jepsen worker 43 - jepsen.util 43 :ok :read [14 2] INFO [2022-01-10 12:53:01,632] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:01,632] jepsen worker 55 - jepsen.util 55 :ok :write [18 21] INFO [2022-01-10 12:53:01,632] jepsen worker 7 - jepsen.util 7 :ok :write [2 15] INFO [2022-01-10 12:53:01,633] jepsen worker 74 - jepsen.util 74 :invoke :write [24 38] INFO [2022-01-10 12:53:01,633] jepsen worker 30 - jepsen.util 30 :ok :write [10 4] INFO [2022-01-10 12:53:01,633] jepsen worker 11 - jepsen.util 11 :ok :read [3 2] INFO [2022-01-10 12:53:01,634] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:01,635] jepsen worker 13 - jepsen.util 13 :invoke :write [4 13] INFO [2022-01-10 12:53:01,635] jepsen worker 55 - jepsen.util 55 :ok :read [18 21] INFO [2022-01-10 12:53:01,636] jepsen worker 30 - jepsen.util 30 :invoke :write [10 17] INFO [2022-01-10 12:53:01,636] jepsen worker 65 - jepsen.util 65 :ok :write [21 2] INFO [2022-01-10 12:53:01,637] jepsen worker 55 - jepsen.util 55 :invoke :write [18 33] INFO [2022-01-10 12:53:01,637] jepsen worker 71 - jepsen.util 71 :ok :write [23 44] INFO [2022-01-10 12:53:01,637] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,638] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:01,639] jepsen worker 65 - jepsen.util 65 :ok :read [21 2] INFO [2022-01-10 12:53:01,639] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,639] jepsen worker 7 - jepsen.util 7 :ok :read [2 15] INFO [2022-01-10 12:53:01,639] jepsen worker 11 - jepsen.util 11 :invoke :write [3 22] INFO [2022-01-10 12:53:01,640] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:01,640] jepsen worker 43 - jepsen.util 43 :ok :read [14 2] INFO [2022-01-10 12:53:01,640] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:01,641] jepsen worker 7 - jepsen.util 7 :ok :read [2 15] INFO [2022-01-10 12:53:01,641] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,641] jepsen worker 7 - jepsen.util 7 :invoke :write [2 41] INFO [2022-01-10 12:53:01,642] jepsen worker 71 - jepsen.util 71 :ok :read [23 44] INFO [2022-01-10 12:53:01,642] jepsen worker 87 - jepsen.util 87 :invoke :write [29 18] INFO [2022-01-10 12:53:01,642] jepsen worker 43 - jepsen.util 43 :ok :read [14 2] INFO [2022-01-10 12:53:01,642] jepsen worker 71 - jepsen.util 71 :invoke :write [23 45] INFO [2022-01-10 12:53:01,643] jepsen worker 51 - jepsen.util 51 :ok :write [17 49] INFO [2022-01-10 12:53:01,643] jepsen worker 20 - jepsen.util 20 :ok :write [6 24] INFO [2022-01-10 12:53:01,643] jepsen worker 65 - jepsen.util 65 :invoke :write [21 27] INFO [2022-01-10 12:53:01,644] jepsen worker 51 - jepsen.util 51 :invoke :write [17 0] INFO [2022-01-10 12:53:01,644] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,645] jepsen worker 20 - jepsen.util 20 :invoke :read [6 nil] INFO [2022-01-10 12:53:01,646] jepsen worker 43 - jepsen.util 43 :ok :read [14 2] INFO [2022-01-10 12:53:01,646] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,646] jepsen worker 20 - jepsen.util 20 :ok :read [6 24] INFO [2022-01-10 12:53:01,647] jepsen worker 43 - jepsen.util 43 :ok :read [14 2] INFO [2022-01-10 12:53:01,647] jepsen worker 20 - jepsen.util 20 :invoke :write [6 30] INFO [2022-01-10 12:53:01,648] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:01,649] jepsen worker 43 - jepsen.util 43 :ok :read [14 2] INFO [2022-01-10 12:53:01,649] jepsen worker 78 - jepsen.util 78 :ok :write [26 40] INFO [2022-01-10 12:53:01,649] jepsen worker 43 - jepsen.util 43 :invoke :write [14 5] INFO [2022-01-10 12:53:01,650] jepsen worker 78 - jepsen.util 78 :invoke :write [26 18] INFO [2022-01-10 12:53:01,656] jepsen worker 26 - jepsen.util 26 :ok :write [8 16] INFO [2022-01-10 12:53:01,656] jepsen worker 26 - jepsen.util 26 :invoke :write [8 26] INFO [2022-01-10 12:53:01,660] jepsen worker 47 - jepsen.util 47 :ok :write [15 6] INFO [2022-01-10 12:53:01,660] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:01,661] jepsen worker 47 - jepsen.util 47 :ok :read [15 6] INFO [2022-01-10 12:53:01,662] jepsen worker 47 - jepsen.util 47 :invoke :write [15 13] INFO [2022-01-10 12:53:01,692] jepsen worker 55 - jepsen.util 55 :ok :write [18 33] INFO [2022-01-10 12:53:01,692] jepsen worker 55 - jepsen.util 55 :invoke :write [18 18] INFO [2022-01-10 12:53:01,693] jepsen worker 11 - jepsen.util 11 :ok :write [3 22] INFO [2022-01-10 12:53:01,693] jepsen worker 11 - jepsen.util 11 :invoke :write [3 23] INFO [2022-01-10 12:53:01,699] jepsen worker 30 - jepsen.util 30 :ok :write [10 17] INFO [2022-01-10 12:53:01,700] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:01,701] jepsen worker 30 - jepsen.util 30 :ok :read [10 17] INFO [2022-01-10 12:53:01,701] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:01,702] jepsen worker 30 - jepsen.util 30 :ok :read [10 17] INFO [2022-01-10 12:53:01,703] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:01,703] jepsen worker 30 - jepsen.util 30 :ok :read [10 17] INFO [2022-01-10 12:53:01,704] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:01,705] jepsen worker 30 - jepsen.util 30 :ok :read [10 17] INFO [2022-01-10 12:53:01,705] jepsen worker 30 - jepsen.util 30 :invoke :write [10 47] INFO [2022-01-10 12:53:01,734] jepsen worker 55 - jepsen.util 55 :ok :write [18 18] INFO [2022-01-10 12:53:01,734] jepsen worker 11 - jepsen.util 11 :ok :write [3 23] INFO [2022-01-10 12:53:01,734] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:01,735] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:01,735] jepsen worker 55 - jepsen.util 55 :ok :read [18 18] INFO [2022-01-10 12:53:01,736] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:01,736] jepsen worker 11 - jepsen.util 11 :ok :read [3 23] INFO [2022-01-10 12:53:01,737] jepsen worker 55 - jepsen.util 55 :ok :read [18 18] INFO [2022-01-10 12:53:01,737] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:01,737] jepsen worker 55 - jepsen.util 55 :invoke :write [18 36] INFO [2022-01-10 12:53:01,739] jepsen worker 11 - jepsen.util 11 :ok :read [3 23] INFO [2022-01-10 12:53:01,739] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:01,740] jepsen worker 30 - jepsen.util 30 :ok :write [10 47] INFO [2022-01-10 12:53:01,740] jepsen worker 11 - jepsen.util 11 :ok :read [3 23] INFO [2022-01-10 12:53:01,741] jepsen worker 30 - jepsen.util 30 :invoke :write [10 30] INFO [2022-01-10 12:53:01,741] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:01,742] jepsen worker 11 - jepsen.util 11 :ok :read [3 23] INFO [2022-01-10 12:53:01,743] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:01,744] jepsen worker 11 - jepsen.util 11 :ok :read [3 23] INFO [2022-01-10 12:53:01,745] jepsen worker 11 - jepsen.util 11 :invoke :write [3 46] INFO [2022-01-10 12:53:01,746] jepsen worker 55 - jepsen.util 55 :ok :write [18 36] INFO [2022-01-10 12:53:01,747] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:01,748] jepsen worker 55 - jepsen.util 55 :ok :read [18 36] INFO [2022-01-10 12:53:01,748] jepsen worker 55 - jepsen.util 55 :invoke :write [18 32] INFO [2022-01-10 12:53:01,753] jepsen worker 30 - jepsen.util 30 :ok :write [10 30] INFO [2022-01-10 12:53:01,754] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:01,755] jepsen worker 30 - jepsen.util 30 :ok :read [10 30] INFO [2022-01-10 12:53:01,756] jepsen worker 30 - jepsen.util 30 :invoke :write [10 14] INFO [2022-01-10 12:53:01,760] jepsen worker 1 - jepsen.util 1 :ok :write [0 48] INFO [2022-01-10 12:53:01,761] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:01,761] jepsen worker 11 - jepsen.util 11 :ok :write [3 46] INFO [2022-01-10 12:53:01,762] jepsen worker 1 - jepsen.util 1 :ok :read [0 48] INFO [2022-01-10 12:53:01,762] jepsen worker 11 - jepsen.util 11 :invoke :write [3 21] INFO [2022-01-10 12:53:01,762] jepsen worker 55 - jepsen.util 55 :ok :write [18 32] INFO [2022-01-10 12:53:01,762] jepsen worker 1 - jepsen.util 1 :invoke :write [0 26] INFO [2022-01-10 12:53:01,763] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:01,764] jepsen worker 55 - jepsen.util 55 :ok :read [18 32] INFO [2022-01-10 12:53:01,764] jepsen worker 55 - jepsen.util 55 :invoke :write [18 48] INFO [2022-01-10 12:53:01,769] jepsen worker 3 - jepsen.util 3 :ok :write [1 6] INFO [2022-01-10 12:53:01,769] jepsen worker 3 - jepsen.util 3 :invoke :write [1 28] INFO [2022-01-10 12:53:01,770] jepsen worker 30 - jepsen.util 30 :ok :write [10 14] INFO [2022-01-10 12:53:01,770] jepsen worker 30 - jepsen.util 30 :invoke :write [10 37] INFO [2022-01-10 12:53:01,776] jepsen worker 57 - jepsen.util 57 :ok :write [19 37] INFO [2022-01-10 12:53:01,777] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,778] jepsen worker 57 - jepsen.util 57 :ok :read [19 37] INFO [2022-01-10 12:53:01,778] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,779] jepsen worker 87 - jepsen.util 87 :ok :write [29 18] INFO [2022-01-10 12:53:01,780] jepsen worker 57 - jepsen.util 57 :ok :read [19 37] INFO [2022-01-10 12:53:01,780] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,780] jepsen worker 57 - jepsen.util 57 :invoke :write [19 35] INFO [2022-01-10 12:53:01,781] jepsen worker 11 - jepsen.util 11 :ok :write [3 21] INFO [2022-01-10 12:53:01,781] jepsen worker 55 - jepsen.util 55 :ok :write [18 48] INFO [2022-01-10 12:53:01,781] jepsen worker 87 - jepsen.util 87 :ok :read [29 18] INFO [2022-01-10 12:53:01,782] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:01,783] jepsen worker 55 - jepsen.util 55 :invoke :write [18 4] INFO [2022-01-10 12:53:01,783] jepsen worker 11 - jepsen.util 11 :ok :read [3 21] INFO [2022-01-10 12:53:01,783] jepsen worker 87 - jepsen.util 87 :invoke :write [29 1] INFO [2022-01-10 12:53:01,784] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:01,785] jepsen worker 11 - jepsen.util 11 :ok :read [3 21] INFO [2022-01-10 12:53:01,785] jepsen worker 84 - jepsen.util 84 :ok :write [28 18] INFO [2022-01-10 12:53:01,785] jepsen worker 16 - jepsen.util 16 :ok :write [5 45] INFO [2022-01-10 12:53:01,785] jepsen worker 11 - jepsen.util 11 :invoke :write [3 38] INFO [2022-01-10 12:53:01,786] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,787] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:01,788] jepsen worker 84 - jepsen.util 84 :ok :read [28 18] INFO [2022-01-10 12:53:01,789] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:01,790] jepsen worker 84 - jepsen.util 84 :ok :read [28 18] INFO [2022-01-10 12:53:01,791] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:01,791] jepsen worker 50 - jepsen.util 50 :ok :write [16 4] INFO [2022-01-10 12:53:01,792] jepsen worker 84 - jepsen.util 84 :ok :read [28 18] INFO [2022-01-10 12:53:01,792] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:01,792] jepsen worker 30 - jepsen.util 30 :ok :write [10 37] INFO [2022-01-10 12:53:01,792] jepsen worker 74 - jepsen.util 74 :ok :write [24 38] INFO [2022-01-10 12:53:01,793] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:01,793] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:01,794] jepsen worker 84 - jepsen.util 84 :ok :read [28 18] INFO [2022-01-10 12:53:01,794] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:01,794] jepsen worker 40 - jepsen.util 40 :ok :write [13 44] INFO [2022-01-10 12:53:01,795] jepsen worker 74 - jepsen.util 74 :ok :read [24 38] INFO [2022-01-10 12:53:01,795] jepsen worker 84 - jepsen.util 84 :invoke :write [28 33] INFO [2022-01-10 12:53:01,795] jepsen worker 30 - jepsen.util 30 :ok :read [10 37] INFO [2022-01-10 12:53:01,795] jepsen worker 36 - jepsen.util 36 :ok :write [12 40] INFO [2022-01-10 12:53:01,796] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,797] jepsen worker 36 - jepsen.util 36 :invoke :write [12 11] INFO [2022-01-10 12:53:01,797] jepsen worker 40 - jepsen.util 40 :ok :read [13 44] INFO [2022-01-10 12:53:01,797] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:01,797] jepsen worker 62 - jepsen.util 62 :ok :write [20 23] INFO [2022-01-10 12:53:01,797] jepsen worker 13 - jepsen.util 13 :ok :write [4 13] INFO [2022-01-10 12:53:01,798] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:01,798] jepsen worker 30 - jepsen.util 30 :ok :read [10 37] INFO [2022-01-10 12:53:01,799] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,799] jepsen worker 74 - jepsen.util 74 :ok :read [24 38] INFO [2022-01-10 12:53:01,799] jepsen worker 3 - jepsen.util 3 :ok :write [1 28] INFO [2022-01-10 12:53:01,800] jepsen worker 62 - jepsen.util 62 :invoke :write [20 16] INFO [2022-01-10 12:53:01,800] jepsen worker 40 - jepsen.util 40 :ok :read [13 44] INFO [2022-01-10 12:53:01,800] jepsen worker 74 - jepsen.util 74 :invoke :write [24 45] INFO [2022-01-10 12:53:01,801] jepsen worker 55 - jepsen.util 55 :ok :write [18 4] INFO [2022-01-10 12:53:01,801] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:01,802] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,803] jepsen worker 55 - jepsen.util 55 :invoke :write [18 7] INFO [2022-01-10 12:53:01,803] jepsen worker 40 - jepsen.util 40 :ok :read [13 44] INFO [2022-01-10 12:53:01,803] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:01,804] jepsen worker 13 - jepsen.util 13 :invoke :write [4 46] INFO [2022-01-10 12:53:01,804] jepsen worker 30 - jepsen.util 30 :ok :read [10 37] INFO [2022-01-10 12:53:01,804] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,804] jepsen worker 57 - jepsen.util 57 :ok :write [19 35] INFO [2022-01-10 12:53:01,805] jepsen worker 71 - jepsen.util 71 :ok :write [23 45] INFO [2022-01-10 12:53:01,805] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:01,805] jepsen worker 81 - jepsen.util 81 :ok :write [27 23] INFO [2022-01-10 12:53:01,805] jepsen worker 29 - jepsen.util 29 :ok :write [9 14] INFO [2022-01-10 12:53:01,805] jepsen worker 40 - jepsen.util 40 :ok :read [13 44] INFO [2022-01-10 12:53:01,806] jepsen worker 30 - jepsen.util 30 :ok :read [10 37] INFO [2022-01-10 12:53:01,806] jepsen worker 71 - jepsen.util 71 :invoke :write [23 11] INFO [2022-01-10 12:53:01,807] jepsen worker 40 - jepsen.util 40 :invoke :write [13 3] INFO [2022-01-10 12:53:01,808] jepsen worker 57 - jepsen.util 57 :invoke :write [19 45] INFO [2022-01-10 12:53:01,809] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:01,809] jepsen worker 29 - jepsen.util 29 :invoke :write [9 22] INFO [2022-01-10 12:53:01,810] jepsen worker 81 - jepsen.util 81 :ok :read [27 23] INFO [2022-01-10 12:53:01,810] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:01,811] jepsen worker 81 - jepsen.util 81 :invoke :write [27 39] INFO [2022-01-10 12:53:01,811] jepsen worker 30 - jepsen.util 30 :ok :read [10 37] INFO [2022-01-10 12:53:01,812] jepsen worker 84 - jepsen.util 84 :ok :write [28 33] INFO [2022-01-10 12:53:01,812] jepsen worker 30 - jepsen.util 30 :invoke :write [10 26] INFO [2022-01-10 12:53:01,813] jepsen worker 84 - jepsen.util 84 :invoke :write [28 16] INFO [2022-01-10 12:53:01,814] jepsen worker 67 - jepsen.util 67 :ok :write [22 28] INFO [2022-01-10 12:53:01,815] jepsen worker 65 - jepsen.util 65 :ok :write [21 27] INFO [2022-01-10 12:53:01,815] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,816] jepsen worker 67 - jepsen.util 67 :invoke :write [22 9] INFO [2022-01-10 12:53:01,817] jepsen worker 65 - jepsen.util 65 :ok :read [21 27] INFO [2022-01-10 12:53:01,817] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,818] jepsen worker 36 - jepsen.util 36 :ok :write [12 11] INFO [2022-01-10 12:53:01,818] jepsen worker 65 - jepsen.util 65 :ok :read [21 27] INFO [2022-01-10 12:53:01,819] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:01,819] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,820] jepsen worker 36 - jepsen.util 36 :ok :read [12 11] INFO [2022-01-10 12:53:01,820] jepsen worker 23 - jepsen.util 23 :ok :write [7 16] INFO [2022-01-10 12:53:01,820] jepsen worker 65 - jepsen.util 65 :ok :read [21 27] INFO [2022-01-10 12:53:01,820] jepsen worker 51 - jepsen.util 51 :ok :write [17 0] INFO [2022-01-10 12:53:01,821] jepsen worker 36 - jepsen.util 36 :invoke :write [12 14] INFO [2022-01-10 12:53:01,822] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,823] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:01,823] jepsen worker 23 - jepsen.util 23 :ok :read [7 16] INFO [2022-01-10 12:53:01,824] jepsen worker 65 - jepsen.util 65 :invoke :write [21 25] INFO [2022-01-10 12:53:01,824] jepsen worker 51 - jepsen.util 51 :ok :read [17 0] INFO [2022-01-10 12:53:01,824] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,825] jepsen worker 51 - jepsen.util 51 :invoke :write [17 49] INFO [2022-01-10 12:53:01,825] jepsen worker 23 - jepsen.util 23 :ok :read [7 16] INFO [2022-01-10 12:53:01,826] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,827] jepsen worker 23 - jepsen.util 23 :ok :read [7 16] INFO [2022-01-10 12:53:01,828] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,828] jepsen worker 23 - jepsen.util 23 :ok :read [7 16] INFO [2022-01-10 12:53:01,829] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:01,830] jepsen worker 23 - jepsen.util 23 :ok :read [7 16] INFO [2022-01-10 12:53:01,830] jepsen worker 23 - jepsen.util 23 :invoke :write [7 29] INFO [2022-01-10 12:53:01,835] jepsen worker 34 - jepsen.util 34 :ok :write [11 27] INFO [2022-01-10 12:53:01,836] jepsen worker 34 - jepsen.util 34 :invoke :write [11 37] INFO [2022-01-10 12:53:01,836] jepsen worker 18 - jepsen.util 18 :ok :write [6 7] INFO [2022-01-10 12:53:01,837] jepsen worker 18 - jepsen.util 18 :invoke :write [6 27] INFO [2022-01-10 12:53:01,842] jepsen worker 78 - jepsen.util 78 :ok :write [26 18] INFO [2022-01-10 12:53:01,842] jepsen worker 3 - jepsen.util 3 :ok :read [1 28] INFO [2022-01-10 12:53:01,842] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:01,843] jepsen worker 3 - jepsen.util 3 :invoke :write [1 33] INFO [2022-01-10 12:53:01,844] jepsen worker 78 - jepsen.util 78 :ok :read [26 18] INFO [2022-01-10 12:53:01,844] jepsen worker 43 - jepsen.util 43 :ok :write [14 5] INFO [2022-01-10 12:53:01,845] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:01,845] jepsen worker 1 - jepsen.util 1 :ok :write [0 26] INFO [2022-01-10 12:53:01,845] jepsen worker 43 - jepsen.util 43 :invoke :write [14 30] INFO [2022-01-10 12:53:01,845] jepsen worker 26 - jepsen.util 26 :ok :write [8 26] INFO [2022-01-10 12:53:01,845] jepsen worker 78 - jepsen.util 78 :ok :read [26 18] INFO [2022-01-10 12:53:01,846] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:01,846] jepsen worker 16 - jepsen.util 16 :ok :read [5 45] INFO [2022-01-10 12:53:01,846] jepsen worker 50 - jepsen.util 50 :ok :read [16 4] INFO [2022-01-10 12:53:01,847] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:53:01,847] jepsen worker 1 - jepsen.util 1 :ok :read [0 26] INFO [2022-01-10 12:53:01,848] jepsen worker 16 - jepsen.util 16 :invoke :write [5 11] INFO [2022-01-10 12:53:01,848] jepsen worker 26 - jepsen.util 26 :ok :read [8 26] INFO [2022-01-10 12:53:01,848] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:01,849] jepsen worker 26 - jepsen.util 26 :invoke :write [8 23] INFO [2022-01-10 12:53:01,849] jepsen worker 50 - jepsen.util 50 :ok :read [16 4] INFO [2022-01-10 12:53:01,850] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:01,850] jepsen worker 50 - jepsen.util 50 :invoke :write [16 17] INFO [2022-01-10 12:53:01,851] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:01,851] jepsen worker 78 - jepsen.util 78 :ok :read [26 18] INFO [2022-01-10 12:53:01,852] jepsen worker 76 - jepsen.util 76 :ok :write [25 43] INFO [2022-01-10 12:53:01,852] jepsen worker 78 - jepsen.util 78 :invoke :write [26 45] INFO [2022-01-10 12:53:01,852] jepsen worker 1 - jepsen.util 1 :ok :read [0 26] INFO [2022-01-10 12:53:01,853] jepsen worker 1 - jepsen.util 1 :invoke :write [0 16] INFO [2022-01-10 12:53:01,853] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:01,854] jepsen worker 87 - jepsen.util 87 :ok :write [29 1] INFO [2022-01-10 12:53:01,854] jepsen worker 84 - jepsen.util 84 :ok :write [28 16] INFO [2022-01-10 12:53:01,854] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:01,855] jepsen worker 76 - jepsen.util 76 :ok :read [25 43] INFO [2022-01-10 12:53:01,855] jepsen worker 84 - jepsen.util 84 :invoke :write [28 49] INFO [2022-01-10 12:53:01,855] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:01,856] jepsen worker 87 - jepsen.util 87 :ok :read [29 1] INFO [2022-01-10 12:53:01,856] jepsen worker 87 - jepsen.util 87 :invoke :write [29 24] INFO [2022-01-10 12:53:01,857] jepsen worker 47 - jepsen.util 47 :ok :write [15 13] INFO [2022-01-10 12:53:01,857] jepsen worker 76 - jepsen.util 76 :ok :read [25 43] INFO [2022-01-10 12:53:01,857] jepsen worker 47 - jepsen.util 47 :invoke :write [15 33] INFO [2022-01-10 12:53:01,857] jepsen worker 76 - jepsen.util 76 :invoke :write [25 16] INFO [2022-01-10 12:53:01,861] jepsen worker 7 - jepsen.util 7 :ok :write [2 41] INFO [2022-01-10 12:53:01,861] jepsen worker 7 - jepsen.util 7 :invoke :write [2 33] INFO [2022-01-10 12:53:01,866] jepsen worker 36 - jepsen.util 36 :ok :write [12 14] INFO [2022-01-10 12:53:01,867] jepsen worker 36 - jepsen.util 36 :invoke :write [12 12] INFO [2022-01-10 12:53:01,869] jepsen worker 16 - jepsen.util 16 :ok :write [5 11] INFO [2022-01-10 12:53:01,870] jepsen worker 13 - jepsen.util 13 :ok :write [4 46] INFO [2022-01-10 12:53:01,870] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:01,870] jepsen worker 13 - jepsen.util 13 :invoke :write [4 49] INFO [2022-01-10 12:53:01,871] jepsen worker 16 - jepsen.util 16 :ok :read [5 11] INFO [2022-01-10 12:53:01,872] jepsen worker 16 - jepsen.util 16 :invoke :write [5 36] INFO [2022-01-10 12:53:01,873] jepsen worker 40 - jepsen.util 40 :ok :write [13 3] INFO [2022-01-10 12:53:01,873] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,874] jepsen worker 40 - jepsen.util 40 :ok :read [13 3] INFO [2022-01-10 12:53:01,875] jepsen worker 50 - jepsen.util 50 :ok :write [16 17] INFO [2022-01-10 12:53:01,875] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,875] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:01,876] jepsen worker 40 - jepsen.util 40 :ok :read [13 3] INFO [2022-01-10 12:53:01,876] jepsen worker 50 - jepsen.util 50 :ok :read [16 17] INFO [2022-01-10 12:53:01,876] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,877] jepsen worker 50 - jepsen.util 50 :invoke :write [16 3] INFO [2022-01-10 12:53:01,877] jepsen worker 40 - jepsen.util 40 :ok :read [13 3] INFO [2022-01-10 12:53:01,877] jepsen worker 71 - jepsen.util 71 :ok :write [23 11] INFO [2022-01-10 12:53:01,878] jepsen worker 71 - jepsen.util 71 :invoke :write [23 43] INFO [2022-01-10 12:53:01,878] jepsen worker 62 - jepsen.util 62 :ok :write [20 16] INFO [2022-01-10 12:53:01,878] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,879] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:01,879] jepsen worker 40 - jepsen.util 40 :ok :read [13 3] INFO [2022-01-10 12:53:01,880] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:01,880] jepsen worker 62 - jepsen.util 62 :ok :read [20 16] INFO [2022-01-10 12:53:01,880] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:01,881] jepsen worker 40 - jepsen.util 40 :ok :read [13 3] INFO [2022-01-10 12:53:01,881] jepsen worker 62 - jepsen.util 62 :ok :read [20 16] INFO [2022-01-10 12:53:01,881] jepsen worker 40 - jepsen.util 40 :invoke :write [13 0] INFO [2022-01-10 12:53:01,881] jepsen worker 62 - jepsen.util 62 :invoke :read [20 nil] INFO [2022-01-10 12:53:01,882] jepsen worker 62 - jepsen.util 62 :ok :read [20 16] INFO [2022-01-10 12:53:01,883] jepsen worker 62 - jepsen.util 62 :invoke :write [20 35] INFO [2022-01-10 12:53:01,899] jepsen worker 81 - jepsen.util 81 :ok :write [27 39] INFO [2022-01-10 12:53:01,900] jepsen worker 81 - jepsen.util 81 :invoke :write [27 7] INFO [2022-01-10 12:53:01,905] jepsen worker 67 - jepsen.util 67 :ok :write [22 9] INFO [2022-01-10 12:53:01,906] jepsen worker 67 - jepsen.util 67 :invoke :write [22 48] INFO [2022-01-10 12:53:01,913] jepsen worker 23 - jepsen.util 23 :ok :write [7 29] INFO [2022-01-10 12:53:01,914] jepsen worker 23 - jepsen.util 23 :invoke :write [7 27] INFO [2022-01-10 12:53:01,916] jepsen worker 11 - jepsen.util 11 :ok :write [3 38] INFO [2022-01-10 12:53:01,917] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:01,918] jepsen worker 11 - jepsen.util 11 :ok :read [3 38] INFO [2022-01-10 12:53:01,919] jepsen worker 11 - jepsen.util 11 :invoke :write [3 1] INFO [2022-01-10 12:53:01,919] jepsen worker 87 - jepsen.util 87 :ok :write [29 24] INFO [2022-01-10 12:53:01,920] jepsen worker 34 - jepsen.util 34 :ok :write [11 37] INFO [2022-01-10 12:53:01,920] jepsen worker 87 - jepsen.util 87 :invoke :write [29 5] INFO [2022-01-10 12:53:01,920] jepsen worker 34 - jepsen.util 34 :invoke :write [11 5] INFO [2022-01-10 12:53:01,921] jepsen worker 29 - jepsen.util 29 :ok :write [9 22] INFO [2022-01-10 12:53:01,922] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,923] jepsen worker 29 - jepsen.util 29 :ok :read [9 22] INFO [2022-01-10 12:53:01,923] jepsen worker 74 - jepsen.util 74 :ok :write [24 45] INFO [2022-01-10 12:53:01,924] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,924] jepsen worker 74 - jepsen.util 74 :invoke :write [24 35] INFO [2022-01-10 12:53:01,925] jepsen worker 29 - jepsen.util 29 :ok :read [9 22] INFO [2022-01-10 12:53:01,926] jepsen worker 29 - jepsen.util 29 :invoke :write [9 0] INFO [2022-01-10 12:53:01,927] jepsen worker 43 - jepsen.util 43 :ok :write [14 30] INFO [2022-01-10 12:53:01,927] jepsen worker 43 - jepsen.util 43 :invoke :write [14 18] INFO [2022-01-10 12:53:01,930] jepsen worker 55 - jepsen.util 55 :ok :write [18 7] INFO [2022-01-10 12:53:01,930] jepsen worker 55 - jepsen.util 55 :invoke :write [18 15] INFO [2022-01-10 12:53:01,933] jepsen worker 30 - jepsen.util 30 :ok :write [10 26] INFO [2022-01-10 12:53:01,934] jepsen worker 30 - jepsen.util 30 :invoke :write [10 28] INFO [2022-01-10 12:53:01,942] jepsen worker 57 - jepsen.util 57 :ok :write [19 45] INFO [2022-01-10 12:53:01,943] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,944] jepsen worker 57 - jepsen.util 57 :ok :read [19 45] INFO [2022-01-10 12:53:01,945] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,946] jepsen worker 57 - jepsen.util 57 :ok :read [19 45] INFO [2022-01-10 12:53:01,946] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,947] jepsen worker 57 - jepsen.util 57 :ok :read [19 45] INFO [2022-01-10 12:53:01,948] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,949] jepsen worker 65 - jepsen.util 65 :ok :write [21 25] INFO [2022-01-10 12:53:01,949] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:53:01,950] jepsen worker 57 - jepsen.util 57 :ok :read [19 45] INFO [2022-01-10 12:53:01,950] jepsen worker 65 - jepsen.util 65 :ok :read [21 25] INFO [2022-01-10 12:53:01,951] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,951] jepsen worker 65 - jepsen.util 65 :invoke :write [21 40] INFO [2022-01-10 12:53:01,952] jepsen worker 57 - jepsen.util 57 :ok :read [19 45] INFO [2022-01-10 12:53:01,953] jepsen worker 57 - jepsen.util 57 :invoke :write [19 46] INFO [2022-01-10 12:53:01,955] jepsen worker 18 - jepsen.util 18 :ok :write [6 27] INFO [2022-01-10 12:53:01,956] jepsen worker 18 - jepsen.util 18 :invoke :write [6 38] INFO [2022-01-10 12:53:01,961] jepsen worker 26 - jepsen.util 26 :ok :write [8 23] INFO [2022-01-10 12:53:01,961] jepsen worker 26 - jepsen.util 26 :invoke :write [8 26] INFO [2022-01-10 12:53:01,963] jepsen worker 29 - jepsen.util 29 :ok :write [9 0] INFO [2022-01-10 12:53:01,963] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,964] jepsen worker 29 - jepsen.util 29 :ok :read [9 0] INFO [2022-01-10 12:53:01,965] jepsen worker 29 - jepsen.util 29 :invoke :write [9 48] INFO [2022-01-10 12:53:01,968] jepsen worker 47 - jepsen.util 47 :ok :write [15 33] INFO [2022-01-10 12:53:01,968] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:01,970] jepsen worker 47 - jepsen.util 47 :ok :read [15 33] INFO [2022-01-10 12:53:01,970] jepsen worker 47 - jepsen.util 47 :invoke :write [15 1] INFO [2022-01-10 12:53:01,975] jepsen worker 29 - jepsen.util 29 :ok :write [9 48] INFO [2022-01-10 12:53:01,976] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,977] jepsen worker 29 - jepsen.util 29 :ok :read [9 48] INFO [2022-01-10 12:53:01,977] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,978] jepsen worker 29 - jepsen.util 29 :ok :read [9 48] INFO [2022-01-10 12:53:01,979] jepsen worker 29 - jepsen.util 29 :invoke :write [9 24] INFO [2022-01-10 12:53:01,988] jepsen worker 29 - jepsen.util 29 :ok :write [9 24] INFO [2022-01-10 12:53:01,989] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,990] jepsen worker 29 - jepsen.util 29 :ok :read [9 24] INFO [2022-01-10 12:53:01,990] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,991] jepsen worker 29 - jepsen.util 29 :ok :read [9 24] INFO [2022-01-10 12:53:01,992] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:01,993] jepsen worker 29 - jepsen.util 29 :ok :read [9 24] INFO [2022-01-10 12:53:01,993] jepsen worker 29 - jepsen.util 29 :invoke :write [9 26] INFO [2022-01-10 12:53:01,993] jepsen worker 58 - jepsen.util 58 :ok :write [19 34] INFO [2022-01-10 12:53:01,994] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:53:01,995] jepsen worker 58 - jepsen.util 58 :ok :read [19 34] INFO [2022-01-10 12:53:01,995] jepsen worker 58 - jepsen.util 58 :invoke :write [19 47] INFO [2022-01-10 12:53:02,001] jepsen worker 51 - jepsen.util 51 :ok :write [17 49] INFO [2022-01-10 12:53:02,002] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:02,003] jepsen worker 51 - jepsen.util 51 :ok :read [17 49] INFO [2022-01-10 12:53:02,003] jepsen worker 1 - jepsen.util 1 :ok :write [0 16] INFO [2022-01-10 12:53:02,004] jepsen worker 1 - jepsen.util 1 :invoke :write [0 34] INFO [2022-01-10 12:53:02,004] jepsen worker 51 - jepsen.util 51 :invoke :write [17 38] INFO [2022-01-10 12:53:02,004] jepsen worker 29 - jepsen.util 29 :ok :write [9 26] INFO [2022-01-10 12:53:02,005] jepsen worker 29 - jepsen.util 29 :invoke :write [9 24] INFO [2022-01-10 12:53:02,009] jepsen worker 78 - jepsen.util 78 :ok :write [26 45] INFO [2022-01-10 12:53:02,010] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:02,011] jepsen worker 76 - jepsen.util 76 :ok :write [25 16] INFO [2022-01-10 12:53:02,012] jepsen worker 78 - jepsen.util 78 :ok :read [26 45] INFO [2022-01-10 12:53:02,012] jepsen worker 76 - jepsen.util 76 :invoke :write [25 29] INFO [2022-01-10 12:53:02,012] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:02,013] jepsen worker 78 - jepsen.util 78 :ok :read [26 45] INFO [2022-01-10 12:53:02,014] jepsen worker 78 - jepsen.util 78 :invoke :write [26 27] INFO [2022-01-10 12:53:02,017] jepsen worker 3 - jepsen.util 3 :ok :write [1 33] INFO [2022-01-10 12:53:02,017] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:02,018] jepsen worker 40 - jepsen.util 40 :ok :write [13 0] INFO [2022-01-10 12:53:02,018] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:02,019] jepsen worker 40 - jepsen.util 40 :invoke :write [13 25] INFO [2022-01-10 12:53:02,019] jepsen worker 3 - jepsen.util 3 :invoke :write [1 44] INFO [2022-01-10 12:53:02,019] jepsen worker 29 - jepsen.util 29 :ok :write [9 24] INFO [2022-01-10 12:53:02,020] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:02,021] jepsen worker 29 - jepsen.util 29 :ok :read [9 24] INFO [2022-01-10 12:53:02,021] jepsen worker 29 - jepsen.util 29 :invoke :write [9 17] INFO [2022-01-10 12:53:02,025] jepsen worker 58 - jepsen.util 58 :ok :write [19 47] INFO [2022-01-10 12:53:02,026] jepsen worker 84 - jepsen.util 84 :ok :write [28 49] INFO [2022-01-10 12:53:02,026] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:53:02,026] jepsen worker 84 - jepsen.util 84 :invoke :write [28 7] INFO [2022-01-10 12:53:02,027] jepsen worker 58 - jepsen.util 58 :ok :read [19 47] INFO [2022-01-10 12:53:02,027] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:53:02,028] jepsen worker 58 - jepsen.util 58 :ok :read [19 47] INFO [2022-01-10 12:53:02,029] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:53:02,029] jepsen worker 58 - jepsen.util 58 :ok :read [19 47] INFO [2022-01-10 12:53:02,030] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:53:02,031] jepsen worker 58 - jepsen.util 58 :ok :read [19 47] INFO [2022-01-10 12:53:02,031] jepsen worker 36 - jepsen.util 36 :ok :write [12 12] INFO [2022-01-10 12:53:02,031] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:53:02,032] jepsen worker 36 - jepsen.util 36 :invoke :write [12 3] INFO [2022-01-10 12:53:02,033] jepsen worker 58 - jepsen.util 58 :ok :read [19 47] INFO [2022-01-10 12:53:02,033] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:53:02,034] jepsen worker 58 - jepsen.util 58 :ok :read [19 47] INFO [2022-01-10 12:53:02,034] jepsen worker 58 - jepsen.util 58 :invoke :write [19 34] INFO [2022-01-10 12:53:02,038] jepsen worker 13 - jepsen.util 13 :ok :write [4 49] INFO [2022-01-10 12:53:02,039] jepsen worker 13 - jepsen.util 13 :invoke :write [4 43] INFO [2022-01-10 12:53:02,039] jepsen worker 87 - jepsen.util 87 :ok :write [29 5] INFO [2022-01-10 12:53:02,040] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:02,041] jepsen worker 87 - jepsen.util 87 :ok :read [29 5] INFO [2022-01-10 12:53:02,041] jepsen worker 87 - jepsen.util 87 :invoke :write [29 49] INFO [2022-01-10 12:53:02,042] jepsen worker 29 - jepsen.util 29 :ok :write [9 17] INFO [2022-01-10 12:53:02,042] jepsen worker 29 - jepsen.util 29 :invoke :write [9 27] INFO [2022-01-10 12:53:02,060] jepsen worker 58 - jepsen.util 58 :ok :write [19 34] INFO [2022-01-10 12:53:02,061] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:53:02,061] jepsen worker 58 - jepsen.util 58 :ok :read [19 34] INFO [2022-01-10 12:53:02,062] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:53:02,063] jepsen worker 58 - jepsen.util 58 :ok :read [19 34] INFO [2022-01-10 12:53:02,064] jepsen worker 58 - jepsen.util 58 :invoke :write [19 7] INFO [2022-01-10 12:53:02,064] jepsen worker 55 - jepsen.util 55 :ok :write [18 15] INFO [2022-01-10 12:53:02,065] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:02,065] jepsen worker 11 - jepsen.util 11 :ok :write [3 1] INFO [2022-01-10 12:53:02,066] jepsen worker 55 - jepsen.util 55 :ok :read [18 15] INFO [2022-01-10 12:53:02,066] jepsen worker 11 - jepsen.util 11 :invoke :write [3 20] INFO [2022-01-10 12:53:02,067] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:02,068] jepsen worker 55 - jepsen.util 55 :ok :read [18 15] INFO [2022-01-10 12:53:02,068] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:02,069] jepsen worker 55 - jepsen.util 55 :ok :read [18 15] INFO [2022-01-10 12:53:02,070] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:02,071] jepsen worker 1 - jepsen.util 1 :ok :write [0 34] INFO [2022-01-10 12:53:02,071] jepsen worker 7 - jepsen.util 7 :ok :write [2 33] INFO [2022-01-10 12:53:02,071] jepsen worker 55 - jepsen.util 55 :ok :read [18 15] INFO [2022-01-10 12:53:02,072] jepsen worker 30 - jepsen.util 30 :ok :write [10 28] INFO [2022-01-10 12:53:02,072] jepsen worker 71 - jepsen.util 71 :ok :write [23 43] INFO [2022-01-10 12:53:02,072] jepsen worker 55 - jepsen.util 55 :invoke :write [18 1] INFO [2022-01-10 12:53:02,072] jepsen worker 29 - jepsen.util 29 :ok :write [9 27] INFO [2022-01-10 12:53:02,073] jepsen worker 71 - jepsen.util 71 :invoke :write [23 4] INFO [2022-01-10 12:53:02,073] jepsen worker 7 - jepsen.util 7 :invoke :write [2 38] INFO [2022-01-10 12:53:02,074] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:02,075] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:02,075] jepsen worker 30 - jepsen.util 30 :ok :read [10 28] INFO [2022-01-10 12:53:02,075] jepsen worker 1 - jepsen.util 1 :invoke :write [0 2] INFO [2022-01-10 12:53:02,075] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:02,075] jepsen worker 29 - jepsen.util 29 :ok :read [9 27] INFO [2022-01-10 12:53:02,076] jepsen worker 30 - jepsen.util 30 :ok :read [10 28] INFO [2022-01-10 12:53:02,076] jepsen worker 29 - jepsen.util 29 :invoke :write [9 0] INFO [2022-01-10 12:53:02,077] jepsen worker 30 - jepsen.util 30 :invoke :write [10 38] INFO [2022-01-10 12:53:02,080] jepsen worker 74 - jepsen.util 74 :ok :write [24 35] INFO [2022-01-10 12:53:02,080] jepsen worker 57 - jepsen.util 57 :ok :write [19 46] INFO [2022-01-10 12:53:02,081] jepsen worker 16 - jepsen.util 16 :ok :write [5 36] INFO [2022-01-10 12:53:02,081] jepsen worker 74 - jepsen.util 74 :invoke :write [24 0] INFO [2022-01-10 12:53:02,081] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:02,082] jepsen worker 16 - jepsen.util 16 :invoke :write [5 47] INFO [2022-01-10 12:53:02,082] jepsen worker 57 - jepsen.util 57 :ok :read [19 46] INFO [2022-01-10 12:53:02,083] jepsen worker 57 - jepsen.util 57 :invoke :write [19 14] INFO [2022-01-10 12:53:02,088] jepsen worker 65 - jepsen.util 65 :ok :write [21 40] INFO [2022-01-10 12:53:02,088] jepsen worker 50 - jepsen.util 50 :ok :write [16 3] INFO [2022-01-10 12:53:02,089] jepsen worker 65 - jepsen.util 65 :invoke :write [21 43] INFO [2022-01-10 12:53:02,089] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,089] jepsen worker 40 - jepsen.util 40 :ok :write [13 25] INFO [2022-01-10 12:53:02,090] jepsen worker 50 - jepsen.util 50 :ok :read [16 3] INFO [2022-01-10 12:53:02,090] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:02,091] jepsen worker 50 - jepsen.util 50 :invoke :write [16 18] INFO [2022-01-10 12:53:02,091] jepsen worker 40 - jepsen.util 40 :ok :read [13 25] INFO [2022-01-10 12:53:02,092] jepsen worker 40 - jepsen.util 40 :invoke :write [13 35] INFO [2022-01-10 12:53:02,100] jepsen worker 18 - jepsen.util 18 :ok :write [6 38] INFO [2022-01-10 12:53:02,101] jepsen worker 62 - jepsen.util 62 :ok :write [20 35] INFO [2022-01-10 12:53:02,101] jepsen worker 18 - jepsen.util 18 :invoke :write [6 46] INFO [2022-01-10 12:53:02,101] jepsen worker 62 - jepsen.util 62 :invoke :write [20 2] INFO [2022-01-10 12:53:02,102] jepsen worker 87 - jepsen.util 87 :ok :write [29 49] INFO [2022-01-10 12:53:02,102] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:02,104] jepsen worker 3 - jepsen.util 3 :ok :write [1 44] INFO [2022-01-10 12:53:02,104] jepsen worker 87 - jepsen.util 87 :ok :read [29 49] INFO [2022-01-10 12:53:02,105] jepsen worker 3 - jepsen.util 3 :invoke :write [1 24] INFO [2022-01-10 12:53:02,105] jepsen worker 26 - jepsen.util 26 :ok :write [8 26] INFO [2022-01-10 12:53:02,105] jepsen worker 87 - jepsen.util 87 :invoke :write [29 3] INFO [2022-01-10 12:53:02,106] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:53:02,107] jepsen worker 26 - jepsen.util 26 :ok :read [8 26] INFO [2022-01-10 12:53:02,107] jepsen worker 26 - jepsen.util 26 :invoke :write [8 37] INFO [2022-01-10 12:53:02,108] jepsen worker 55 - jepsen.util 55 :ok :write [18 1] INFO [2022-01-10 12:53:02,108] jepsen worker 55 - jepsen.util 55 :invoke :write [18 26] INFO [2022-01-10 12:53:02,109] jepsen worker 81 - jepsen.util 81 :ok :write [27 7] INFO [2022-01-10 12:53:02,110] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,111] jepsen worker 81 - jepsen.util 81 :ok :read [27 7] INFO [2022-01-10 12:53:02,111] jepsen worker 81 - jepsen.util 81 :invoke :write [27 15] INFO [2022-01-10 12:53:02,112] jepsen worker 47 - jepsen.util 47 :ok :write [15 1] INFO [2022-01-10 12:53:02,112] jepsen worker 47 - jepsen.util 47 :invoke :write [15 1] INFO [2022-01-10 12:53:02,114] jepsen worker 84 - jepsen.util 84 :ok :write [28 7] INFO [2022-01-10 12:53:02,114] jepsen worker 67 - jepsen.util 67 :ok :write [22 48] INFO [2022-01-10 12:53:02,115] jepsen worker 84 - jepsen.util 84 :invoke :write [28 49] INFO [2022-01-10 12:53:02,115] jepsen worker 67 - jepsen.util 67 :invoke :write [22 3] INFO [2022-01-10 12:53:02,120] jepsen worker 36 - jepsen.util 36 :ok :write [12 3] INFO [2022-01-10 12:53:02,121] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,122] jepsen worker 51 - jepsen.util 51 :ok :write [17 38] INFO [2022-01-10 12:53:02,122] jepsen worker 36 - jepsen.util 36 :ok :read [12 3] INFO [2022-01-10 12:53:02,122] jepsen worker 51 - jepsen.util 51 :invoke :write [17 47] INFO [2022-01-10 12:53:02,123] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,124] jepsen worker 36 - jepsen.util 36 :ok :read [12 3] INFO [2022-01-10 12:53:02,124] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,125] jepsen worker 36 - jepsen.util 36 :ok :read [12 3] INFO [2022-01-10 12:53:02,126] jepsen worker 23 - jepsen.util 23 :ok :write [7 27] INFO [2022-01-10 12:53:02,126] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,127] jepsen worker 78 - jepsen.util 78 :ok :write [26 27] INFO [2022-01-10 12:53:02,127] jepsen worker 23 - jepsen.util 23 :invoke :write [7 47] INFO [2022-01-10 12:53:02,127] jepsen worker 36 - jepsen.util 36 :ok :read [12 3] INFO [2022-01-10 12:53:02,127] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:02,128] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,129] jepsen worker 78 - jepsen.util 78 :ok :read [26 27] INFO [2022-01-10 12:53:02,129] jepsen worker 36 - jepsen.util 36 :ok :read [12 3] INFO [2022-01-10 12:53:02,130] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:02,130] jepsen worker 36 - jepsen.util 36 :invoke :write [12 26] INFO [2022-01-10 12:53:02,131] jepsen worker 78 - jepsen.util 78 :ok :read [26 27] INFO [2022-01-10 12:53:02,131] jepsen worker 34 - jepsen.util 34 :ok :write [11 5] INFO [2022-01-10 12:53:02,131] jepsen worker 13 - jepsen.util 13 :ok :write [4 43] INFO [2022-01-10 12:53:02,131] jepsen worker 78 - jepsen.util 78 :invoke :write [26 42] INFO [2022-01-10 12:53:02,132] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:02,132] jepsen worker 34 - jepsen.util 34 :invoke :write [11 42] INFO [2022-01-10 12:53:02,133] jepsen worker 13 - jepsen.util 13 :ok :read [4 43] INFO [2022-01-10 12:53:02,133] jepsen worker 13 - jepsen.util 13 :invoke :write [4 49] INFO [2022-01-10 12:53:02,137] jepsen worker 11 - jepsen.util 11 :ok :write [3 20] INFO [2022-01-10 12:53:02,138] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:02,139] jepsen worker 11 - jepsen.util 11 :ok :read [3 20] INFO [2022-01-10 12:53:02,139] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:02,140] jepsen worker 11 - jepsen.util 11 :ok :read [3 20] INFO [2022-01-10 12:53:02,141] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:02,142] jepsen worker 11 - jepsen.util 11 :ok :read [3 20] INFO [2022-01-10 12:53:02,143] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:02,143] jepsen worker 43 - jepsen.util 43 :ok :write [14 18] INFO [2022-01-10 12:53:02,144] jepsen worker 11 - jepsen.util 11 :ok :read [3 20] INFO [2022-01-10 12:53:02,144] jepsen worker 43 - jepsen.util 43 :invoke :write [14 33] INFO [2022-01-10 12:53:02,144] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:02,145] jepsen worker 11 - jepsen.util 11 :ok :read [3 20] INFO [2022-01-10 12:53:02,146] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:02,146] jepsen worker 55 - jepsen.util 55 :ok :write [18 26] INFO [2022-01-10 12:53:02,146] jepsen worker 55 - jepsen.util 55 :invoke :write [18 36] INFO [2022-01-10 12:53:02,147] jepsen worker 11 - jepsen.util 11 :ok :read [3 20] INFO [2022-01-10 12:53:02,147] jepsen worker 74 - jepsen.util 74 :ok :write [24 0] INFO [2022-01-10 12:53:02,147] jepsen worker 69 - jepsen.util 69 :ok :write [23 40] INFO [2022-01-10 12:53:02,147] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:02,148] jepsen worker 76 - jepsen.util 76 :ok :write [25 29] INFO [2022-01-10 12:53:02,148] jepsen worker 74 - jepsen.util 74 :invoke :write [24 27] INFO [2022-01-10 12:53:02,149] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:02,149] jepsen worker 11 - jepsen.util 11 :ok :read [3 20] INFO [2022-01-10 12:53:02,149] jepsen worker 69 - jepsen.util 69 :invoke :write [23 10] INFO [2022-01-10 12:53:02,150] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:02,150] jepsen worker 76 - jepsen.util 76 :ok :read [25 29] INFO [2022-01-10 12:53:02,151] jepsen worker 11 - jepsen.util 11 :ok :read [3 20] INFO [2022-01-10 12:53:02,151] jepsen worker 76 - jepsen.util 76 :invoke :write [25 11] INFO [2022-01-10 12:53:02,151] jepsen worker 65 - jepsen.util 65 :ok :write [21 43] INFO [2022-01-10 12:53:02,151] jepsen worker 11 - jepsen.util 11 :invoke :write [3 26] INFO [2022-01-10 12:53:02,152] jepsen worker 65 - jepsen.util 65 :invoke :read [21 nil] INFO [2022-01-10 12:53:02,153] jepsen worker 65 - jepsen.util 65 :ok :read [21 43] INFO [2022-01-10 12:53:02,153] jepsen worker 65 - jepsen.util 65 :invoke :write [21 45] INFO [2022-01-10 12:53:02,153] jepsen worker 30 - jepsen.util 30 :ok :write [10 38] INFO [2022-01-10 12:53:02,154] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:02,155] jepsen worker 30 - jepsen.util 30 :ok :read [10 38] INFO [2022-01-10 12:53:02,155] jepsen worker 30 - jepsen.util 30 :invoke :write [10 17] INFO [2022-01-10 12:53:02,157] jepsen worker 18 - jepsen.util 18 :ok :write [6 46] INFO [2022-01-10 12:53:02,158] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:53:02,159] jepsen worker 18 - jepsen.util 18 :ok :read [6 46] INFO [2022-01-10 12:53:02,160] jepsen worker 18 - jepsen.util 18 :invoke :write [6 47] INFO [2022-01-10 12:53:02,165] jepsen worker 26 - jepsen.util 26 :ok :write [8 37] INFO [2022-01-10 12:53:02,165] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:53:02,167] jepsen worker 26 - jepsen.util 26 :ok :read [8 37] INFO [2022-01-10 12:53:02,167] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:53:02,168] jepsen worker 55 - jepsen.util 55 :ok :write [18 36] INFO [2022-01-10 12:53:02,168] jepsen worker 26 - jepsen.util 26 :ok :read [8 37] INFO [2022-01-10 12:53:02,168] jepsen worker 55 - jepsen.util 55 :invoke :write [18 29] INFO [2022-01-10 12:53:02,169] jepsen worker 26 - jepsen.util 26 :invoke :write [8 11] INFO [2022-01-10 12:53:02,171] jepsen worker 47 - jepsen.util 47 :ok :write [15 1] INFO [2022-01-10 12:53:02,172] jepsen worker 47 - jepsen.util 47 :invoke :write [15 1] INFO [2022-01-10 12:53:02,173] jepsen worker 62 - jepsen.util 62 :ok :write [20 2] INFO [2022-01-10 12:53:02,174] jepsen worker 62 - jepsen.util 62 :invoke :write [20 6] INFO [2022-01-10 12:53:02,178] jepsen worker 51 - jepsen.util 51 :ok :write [17 47] INFO [2022-01-10 12:53:02,178] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:02,180] jepsen worker 51 - jepsen.util 51 :ok :read [17 47] INFO [2022-01-10 12:53:02,180] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:02,181] jepsen worker 51 - jepsen.util 51 :ok :read [17 47] INFO [2022-01-10 12:53:02,182] jepsen worker 51 - jepsen.util 51 :invoke :write [17 39] INFO [2022-01-10 12:53:02,183] jepsen worker 55 - jepsen.util 55 :ok :write [18 29] INFO [2022-01-10 12:53:02,184] jepsen worker 55 - jepsen.util 55 :invoke :write [18 43] INFO [2022-01-10 12:53:02,186] jepsen worker 78 - jepsen.util 78 :ok :write [26 42] INFO [2022-01-10 12:53:02,186] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:02,187] jepsen worker 78 - jepsen.util 78 :ok :read [26 42] INFO [2022-01-10 12:53:02,188] jepsen worker 78 - jepsen.util 78 :invoke :write [26 34] INFO [2022-01-10 12:53:02,189] jepsen worker 74 - jepsen.util 74 :ok :write [24 27] INFO [2022-01-10 12:53:02,190] jepsen worker 74 - jepsen.util 74 :invoke :write [24 16] INFO [2022-01-10 12:53:02,193] jepsen worker 13 - jepsen.util 13 :ok :write [4 49] INFO [2022-01-10 12:53:02,194] jepsen worker 13 - jepsen.util 13 :invoke :write [4 37] INFO [2022-01-10 12:53:02,200] jepsen worker 55 - jepsen.util 55 :ok :write [18 43] INFO [2022-01-10 12:53:02,200] jepsen worker 11 - jepsen.util 11 :ok :write [3 26] INFO [2022-01-10 12:53:02,200] jepsen worker 55 - jepsen.util 55 :invoke :write [18 26] INFO [2022-01-10 12:53:02,201] jepsen worker 11 - jepsen.util 11 :invoke :write [3 36] INFO [2022-01-10 12:53:02,203] jepsen worker 7 - jepsen.util 7 :ok :write [2 38] INFO [2022-01-10 12:53:02,203] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:02,204] jepsen worker 7 - jepsen.util 7 :ok :read [2 38] INFO [2022-01-10 12:53:02,205] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:02,205] jepsen worker 7 - jepsen.util 7 :ok :read [2 38] INFO [2022-01-10 12:53:02,206] jepsen worker 7 - jepsen.util 7 :invoke :write [2 16] INFO [2022-01-10 12:53:02,206] jepsen worker 57 - jepsen.util 57 :ok :write [19 14] INFO [2022-01-10 12:53:02,207] jepsen worker 57 - jepsen.util 57 :invoke :read [19 nil] INFO [2022-01-10 12:53:02,208] jepsen worker 57 - jepsen.util 57 :ok :read [19 14] INFO [2022-01-10 12:53:02,209] jepsen worker 69 - jepsen.util 69 :ok :write [23 10] INFO [2022-01-10 12:53:02,209] jepsen worker 29 - jepsen.util 29 :ok :write [9 0] INFO [2022-01-10 12:53:02,209] jepsen worker 57 - jepsen.util 57 :invoke :write [19 35] INFO [2022-01-10 12:53:02,210] jepsen worker 29 - jepsen.util 29 :invoke :write [9 2] INFO [2022-01-10 12:53:02,210] jepsen worker 69 - jepsen.util 69 :invoke :write [23 17] INFO [2022-01-10 12:53:02,214] jepsen worker 55 - jepsen.util 55 :ok :write [18 26] INFO [2022-01-10 12:53:02,214] jepsen worker 55 - jepsen.util 55 :invoke :write [18 16] INFO [2022-01-10 12:53:02,215] jepsen worker 3 - jepsen.util 3 :ok :write [1 24] INFO [2022-01-10 12:53:02,215] jepsen worker 30 - jepsen.util 30 :ok :write [10 17] INFO [2022-01-10 12:53:02,215] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 12:53:02,216] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:02,217] jepsen worker 30 - jepsen.util 30 :ok :read [10 17] INFO [2022-01-10 12:53:02,217] jepsen worker 30 - jepsen.util 30 :invoke :write [10 31] INFO [2022-01-10 12:53:02,218] jepsen worker 1 - jepsen.util 1 :ok :write [0 2] INFO [2022-01-10 12:53:02,218] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:02,219] jepsen worker 1 - jepsen.util 1 :ok :read [0 2] INFO [2022-01-10 12:53:02,220] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:02,221] jepsen worker 1 - jepsen.util 1 :ok :read [0 2] INFO [2022-01-10 12:53:02,221] jepsen worker 1 - jepsen.util 1 :invoke :write [0 1] INFO [2022-01-10 12:53:02,222] jepsen worker 84 - jepsen.util 84 :ok :write [28 49] INFO [2022-01-10 12:53:02,223] jepsen worker 16 - jepsen.util 16 :ok :write [5 47] INFO [2022-01-10 12:53:02,223] jepsen worker 84 - jepsen.util 84 :invoke :write [28 43] INFO [2022-01-10 12:53:02,223] jepsen worker 16 - jepsen.util 16 :invoke :write [5 39] INFO [2022-01-10 12:53:02,229] jepsen worker 55 - jepsen.util 55 :ok :write [18 16] INFO [2022-01-10 12:53:02,230] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:02,230] jepsen worker 36 - jepsen.util 36 :ok :write [12 26] INFO [2022-01-10 12:53:02,231] jepsen worker 55 - jepsen.util 55 :ok :read [18 16] INFO [2022-01-10 12:53:02,231] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,231] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:02,232] jepsen worker 36 - jepsen.util 36 :ok :read [12 26] INFO [2022-01-10 12:53:02,232] jepsen worker 50 - jepsen.util 50 :ok :write [16 18] INFO [2022-01-10 12:53:02,232] jepsen worker 55 - jepsen.util 55 :ok :read [18 16] INFO [2022-01-10 12:53:02,232] jepsen worker 36 - jepsen.util 36 :invoke :write [12 33] INFO [2022-01-10 12:53:02,233] jepsen worker 55 - jepsen.util 55 :invoke :write [18 16] INFO [2022-01-10 12:53:02,234] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,234] jepsen worker 50 - jepsen.util 50 :ok :read [16 18] INFO [2022-01-10 12:53:02,235] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,236] jepsen worker 57 - jepsen.util 57 :ok :write [19 35] INFO [2022-01-10 12:53:02,236] jepsen worker 50 - jepsen.util 50 :ok :read [16 18] INFO [2022-01-10 12:53:02,237] jepsen worker 50 - jepsen.util 50 :invoke :write [16 40] INFO [2022-01-10 12:53:02,238] jepsen worker 57 - jepsen.util 57 :invoke :write [19 28] INFO [2022-01-10 12:53:02,251] jepsen worker 40 - jepsen.util 40 :ok :write [13 35] INFO [2022-01-10 12:53:02,251] jepsen worker 11 - jepsen.util 11 :ok :write [3 36] INFO [2022-01-10 12:53:02,251] jepsen worker 40 - jepsen.util 40 :invoke :write [13 2] INFO [2022-01-10 12:53:02,252] jepsen worker 11 - jepsen.util 11 :invoke :write [3 2] INFO [2022-01-10 12:53:02,254] jepsen worker 55 - jepsen.util 55 :ok :write [18 16] INFO [2022-01-10 12:53:02,255] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:02,256] jepsen worker 55 - jepsen.util 55 :ok :read [18 16] INFO [2022-01-10 12:53:02,257] jepsen worker 87 - jepsen.util 87 :ok :write [29 3] INFO [2022-01-10 12:53:02,257] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:02,257] jepsen worker 87 - jepsen.util 87 :invoke :write [29 25] INFO [2022-01-10 12:53:02,258] jepsen worker 55 - jepsen.util 55 :ok :read [18 16] INFO [2022-01-10 12:53:02,258] jepsen worker 55 - jepsen.util 55 :invoke :write [18 42] INFO [2022-01-10 12:53:02,269] jepsen worker 3 - jepsen.util 3 :ok :write [1 40] INFO [2022-01-10 12:53:02,270] jepsen worker 3 - jepsen.util 3 :invoke :write [1 4] INFO [2022-01-10 12:53:02,272] jepsen worker 81 - jepsen.util 81 :ok :write [27 15] INFO [2022-01-10 12:53:02,273] jepsen worker 81 - jepsen.util 81 :invoke :write [27 19] INFO [2022-01-10 12:53:02,276] jepsen worker 58 - jepsen.util 58 :ok :write [19 7] INFO [2022-01-10 12:53:02,276] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:53:02,277] jepsen worker 30 - jepsen.util 30 :ok :write [10 31] INFO [2022-01-10 12:53:02,278] jepsen worker 58 - jepsen.util 58 :ok :read [19 7] INFO [2022-01-10 12:53:02,278] jepsen worker 30 - jepsen.util 30 :invoke :write [10 2] INFO [2022-01-10 12:53:02,279] jepsen worker 58 - jepsen.util 58 :invoke :write [19 23] INFO [2022-01-10 12:53:02,279] jepsen worker 67 - jepsen.util 67 :ok :write [22 3] INFO [2022-01-10 12:53:02,280] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,281] jepsen worker 67 - jepsen.util 67 :ok :read [22 3] INFO [2022-01-10 12:53:02,281] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,283] jepsen worker 67 - jepsen.util 67 :ok :read [22 3] INFO [2022-01-10 12:53:02,283] jepsen worker 84 - jepsen.util 84 :ok :write [28 43] INFO [2022-01-10 12:53:02,284] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,284] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:02,285] jepsen worker 67 - jepsen.util 67 :ok :read [22 3] INFO [2022-01-10 12:53:02,285] jepsen worker 84 - jepsen.util 84 :ok :read [28 43] INFO [2022-01-10 12:53:02,285] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,286] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:02,286] jepsen worker 36 - jepsen.util 36 :ok :write [12 33] INFO [2022-01-10 12:53:02,286] jepsen worker 67 - jepsen.util 67 :ok :read [22 3] INFO [2022-01-10 12:53:02,287] jepsen worker 84 - jepsen.util 84 :ok :read [28 43] INFO [2022-01-10 12:53:02,287] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,288] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,288] jepsen worker 36 - jepsen.util 36 :ok :read [12 33] INFO [2022-01-10 12:53:02,288] jepsen worker 84 - jepsen.util 84 :invoke :write [28 11] INFO [2022-01-10 12:53:02,289] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,289] jepsen worker 67 - jepsen.util 67 :ok :read [22 3] INFO [2022-01-10 12:53:02,290] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,290] jepsen worker 36 - jepsen.util 36 :ok :read [12 33] INFO [2022-01-10 12:53:02,291] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,291] jepsen worker 67 - jepsen.util 67 :ok :read [22 3] INFO [2022-01-10 12:53:02,292] jepsen worker 36 - jepsen.util 36 :ok :read [12 33] INFO [2022-01-10 12:53:02,292] jepsen worker 67 - jepsen.util 67 :invoke :write [22 0] INFO [2022-01-10 12:53:02,293] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,293] jepsen worker 36 - jepsen.util 36 :ok :read [12 33] INFO [2022-01-10 12:53:02,294] jepsen worker 36 - jepsen.util 36 :invoke :write [12 3] INFO [2022-01-10 12:53:02,297] jepsen worker 29 - jepsen.util 29 :ok :write [9 2] INFO [2022-01-10 12:53:02,297] jepsen worker 3 - jepsen.util 3 :ok :write [1 4] INFO [2022-01-10 12:53:02,297] jepsen worker 29 - jepsen.util 29 :invoke :read [9 nil] INFO [2022-01-10 12:53:02,298] jepsen worker 3 - jepsen.util 3 :invoke :write [1 22] INFO [2022-01-10 12:53:02,298] jepsen worker 29 - jepsen.util 29 :ok :read [9 2] INFO [2022-01-10 12:53:02,299] jepsen worker 29 - jepsen.util 29 :invoke :write [9 44] INFO [2022-01-10 12:53:02,303] jepsen worker 0 - jepsen.util 0 :ok :write [0 5] INFO [2022-01-10 12:53:02,304] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:02,305] jepsen worker 0 - jepsen.util 0 :ok :read [0 5] INFO [2022-01-10 12:53:02,305] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:02,306] jepsen worker 0 - jepsen.util 0 :ok :read [0 5] INFO [2022-01-10 12:53:02,307] jepsen worker 0 - jepsen.util 0 :invoke :write [0 43] INFO [2022-01-10 12:53:02,307] jepsen worker 65 - jepsen.util 65 :ok :write [21 45] INFO [2022-01-10 12:53:02,307] jepsen worker 23 - jepsen.util 23 :ok :write [7 47] INFO [2022-01-10 12:53:02,308] jepsen worker 65 - jepsen.util 65 :invoke :write [21 36] INFO [2022-01-10 12:53:02,308] jepsen worker 23 - jepsen.util 23 :invoke :write [7 39] INFO [2022-01-10 12:53:02,312] jepsen worker 18 - jepsen.util 18 :ok :write [6 47] INFO [2022-01-10 12:53:02,312] jepsen worker 18 - jepsen.util 18 :invoke :write [6 13] INFO [2022-01-10 12:53:02,315] jepsen worker 26 - jepsen.util 26 :ok :write [8 11] INFO [2022-01-10 12:53:02,316] jepsen worker 26 - jepsen.util 26 :invoke :write [8 0] INFO [2022-01-10 12:53:02,317] jepsen worker 34 - jepsen.util 34 :ok :write [11 42] INFO [2022-01-10 12:53:02,318] jepsen worker 34 - jepsen.util 34 :invoke :write [11 24] INFO [2022-01-10 12:53:02,320] jepsen worker 47 - jepsen.util 47 :ok :write [15 1] INFO [2022-01-10 12:53:02,321] jepsen worker 47 - jepsen.util 47 :invoke :write [15 14] INFO [2022-01-10 12:53:02,324] jepsen worker 40 - jepsen.util 40 :ok :write [13 2] INFO [2022-01-10 12:53:02,325] jepsen worker 51 - jepsen.util 51 :ok :write [17 39] INFO [2022-01-10 12:53:02,325] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:02,326] jepsen worker 51 - jepsen.util 51 :invoke :write [17 9] INFO [2022-01-10 12:53:02,327] jepsen worker 40 - jepsen.util 40 :ok :read [13 2] INFO [2022-01-10 12:53:02,327] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:02,328] jepsen worker 40 - jepsen.util 40 :ok :read [13 2] INFO [2022-01-10 12:53:02,328] jepsen worker 78 - jepsen.util 78 :ok :write [26 34] INFO [2022-01-10 12:53:02,329] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:02,329] jepsen worker 43 - jepsen.util 43 :ok :write [14 33] INFO [2022-01-10 12:53:02,329] jepsen worker 78 - jepsen.util 78 :invoke :write [26 33] INFO [2022-01-10 12:53:02,330] jepsen worker 40 - jepsen.util 40 :ok :read [13 2] INFO [2022-01-10 12:53:02,330] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:02,330] jepsen worker 87 - jepsen.util 87 :ok :write [29 25] INFO [2022-01-10 12:53:02,331] jepsen worker 40 - jepsen.util 40 :invoke :write [13 49] INFO [2022-01-10 12:53:02,331] jepsen worker 43 - jepsen.util 43 :ok :read [14 33] INFO [2022-01-10 12:53:02,331] jepsen worker 87 - jepsen.util 87 :invoke :write [29 37] INFO [2022-01-10 12:53:02,332] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:02,333] jepsen worker 43 - jepsen.util 43 :ok :read [14 33] INFO [2022-01-10 12:53:02,333] jepsen worker 74 - jepsen.util 74 :ok :write [24 16] INFO [2022-01-10 12:53:02,334] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:02,334] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:02,335] jepsen worker 43 - jepsen.util 43 :ok :read [14 33] INFO [2022-01-10 12:53:02,335] jepsen worker 74 - jepsen.util 74 :ok :read [24 16] INFO [2022-01-10 12:53:02,335] jepsen worker 76 - jepsen.util 76 :ok :write [25 11] INFO [2022-01-10 12:53:02,335] jepsen worker 43 - jepsen.util 43 :invoke :write [14 34] INFO [2022-01-10 12:53:02,336] jepsen worker 76 - jepsen.util 76 :invoke :write [25 10] INFO [2022-01-10 12:53:02,336] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:02,337] jepsen worker 74 - jepsen.util 74 :ok :read [24 16] INFO [2022-01-10 12:53:02,337] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:02,337] jepsen worker 13 - jepsen.util 13 :ok :write [4 37] INFO [2022-01-10 12:53:02,338] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:02,338] jepsen worker 74 - jepsen.util 74 :ok :read [24 16] INFO [2022-01-10 12:53:02,339] jepsen worker 74 - jepsen.util 74 :invoke :write [24 33] INFO [2022-01-10 12:53:02,339] jepsen worker 13 - jepsen.util 13 :ok :read [4 37] INFO [2022-01-10 12:53:02,340] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:02,341] jepsen worker 13 - jepsen.util 13 :ok :read [4 37] INFO [2022-01-10 12:53:02,341] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:02,342] jepsen worker 60 - jepsen.util 60 :ok :write [20 3] INFO [2022-01-10 12:53:02,342] jepsen worker 13 - jepsen.util 13 :ok :read [4 37] INFO [2022-01-10 12:53:02,343] jepsen worker 60 - jepsen.util 60 :invoke :write [20 49] INFO [2022-01-10 12:53:02,343] jepsen worker 69 - jepsen.util 69 :ok :write [23 17] INFO [2022-01-10 12:53:02,343] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:02,344] jepsen worker 69 - jepsen.util 69 :invoke :write [23 2] INFO [2022-01-10 12:53:02,344] jepsen worker 13 - jepsen.util 13 :ok :read [4 37] INFO [2022-01-10 12:53:02,345] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:02,346] jepsen worker 13 - jepsen.util 13 :ok :read [4 37] INFO [2022-01-10 12:53:02,347] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:02,348] jepsen worker 13 - jepsen.util 13 :ok :read [4 37] INFO [2022-01-10 12:53:02,348] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:02,349] jepsen worker 13 - jepsen.util 13 :ok :read [4 37] INFO [2022-01-10 12:53:02,350] jepsen worker 13 - jepsen.util 13 :invoke :write [4 10] INFO [2022-01-10 12:53:02,350] jepsen worker 7 - jepsen.util 7 :ok :write [2 16] INFO [2022-01-10 12:53:02,351] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:02,352] jepsen worker 7 - jepsen.util 7 :ok :read [2 16] INFO [2022-01-10 12:53:02,353] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:02,354] jepsen worker 7 - jepsen.util 7 :ok :read [2 16] INFO [2022-01-10 12:53:02,355] jepsen worker 7 - jepsen.util 7 :invoke :write [2 6] INFO [2022-01-10 12:53:02,359] jepsen worker 16 - jepsen.util 16 :ok :write [5 39] INFO [2022-01-10 12:53:02,360] jepsen worker 16 - jepsen.util 16 :invoke :write [5 5] INFO [2022-01-10 12:53:02,364] jepsen worker 50 - jepsen.util 50 :ok :write [16 40] INFO [2022-01-10 12:53:02,365] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,366] jepsen worker 50 - jepsen.util 50 :ok :read [16 40] INFO [2022-01-10 12:53:02,367] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,368] jepsen worker 50 - jepsen.util 50 :ok :read [16 40] INFO [2022-01-10 12:53:02,369] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,370] jepsen worker 81 - jepsen.util 81 :ok :write [27 19] INFO [2022-01-10 12:53:02,370] jepsen worker 50 - jepsen.util 50 :ok :read [16 40] INFO [2022-01-10 12:53:02,371] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,371] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,372] jepsen worker 81 - jepsen.util 81 :ok :read [27 19] INFO [2022-01-10 12:53:02,372] jepsen worker 50 - jepsen.util 50 :ok :read [16 40] INFO [2022-01-10 12:53:02,373] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,373] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:53:02,374] jepsen worker 81 - jepsen.util 81 :ok :read [27 19] INFO [2022-01-10 12:53:02,374] jepsen worker 81 - jepsen.util 81 :invoke :write [27 35] INFO [2022-01-10 12:53:02,375] jepsen worker 11 - jepsen.util 11 :ok :write [3 2] INFO [2022-01-10 12:53:02,376] jepsen worker 11 - jepsen.util 11 :invoke :write [3 30] INFO [2022-01-10 12:53:02,376] jepsen worker 67 - jepsen.util 67 :ok :write [22 0] INFO [2022-01-10 12:53:02,377] jepsen worker 67 - jepsen.util 67 :invoke :write [22 30] INFO [2022-01-10 12:53:02,380] jepsen worker 55 - jepsen.util 55 :ok :write [18 42] INFO [2022-01-10 12:53:02,381] jepsen worker 58 - jepsen.util 58 :ok :write [19 23] INFO [2022-01-10 12:53:02,381] jepsen worker 55 - jepsen.util 55 :invoke :write [18 2] INFO [2022-01-10 12:53:02,382] jepsen worker 58 - jepsen.util 58 :invoke :read [19 nil] INFO [2022-01-10 12:53:02,383] jepsen worker 58 - jepsen.util 58 :ok :read [19 23] INFO [2022-01-10 12:53:02,383] jepsen worker 58 - jepsen.util 58 :invoke :write [19 26] INFO [2022-01-10 12:53:02,385] jepsen worker 29 - jepsen.util 29 :ok :write [9 44] INFO [2022-01-10 12:53:02,385] jepsen worker 34 - jepsen.util 34 :ok :write [11 24] INFO [2022-01-10 12:53:02,386] jepsen worker 29 - jepsen.util 29 :invoke :write [9 0] INFO [2022-01-10 12:53:02,387] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:02,388] jepsen worker 34 - jepsen.util 34 :ok :read [11 24] INFO [2022-01-10 12:53:02,388] jepsen worker 34 - jepsen.util 34 :invoke :write [11 13] INFO [2022-01-10 12:53:02,390] jepsen worker 30 - jepsen.util 30 :ok :write [10 2] INFO [2022-01-10 12:53:02,390] jepsen worker 30 - jepsen.util 30 :invoke :write [10 38] INFO [2022-01-10 12:53:02,410] jepsen worker 65 - jepsen.util 65 :ok :write [21 36] INFO [2022-01-10 12:53:02,411] jepsen worker 65 - jepsen.util 65 :invoke :write [21 13] INFO [2022-01-10 12:53:02,415] jepsen worker 18 - jepsen.util 18 :ok :write [6 13] INFO [2022-01-10 12:53:02,416] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:53:02,417] jepsen worker 7 - jepsen.util 7 :ok :write [2 6] INFO [2022-01-10 12:53:02,417] jepsen worker 18 - jepsen.util 18 :ok :read [6 13] INFO [2022-01-10 12:53:02,418] jepsen worker 7 - jepsen.util 7 :invoke :write [2 15] INFO [2022-01-10 12:53:02,418] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:53:02,419] jepsen worker 18 - jepsen.util 18 :ok :read [6 13] INFO [2022-01-10 12:53:02,420] jepsen worker 18 - jepsen.util 18 :invoke :write [6 6] INFO [2022-01-10 12:53:02,424] jepsen worker 16 - jepsen.util 16 :ok :write [5 5] INFO [2022-01-10 12:53:02,424] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:02,425] jepsen worker 16 - jepsen.util 16 :ok :read [5 5] INFO [2022-01-10 12:53:02,426] jepsen worker 26 - jepsen.util 26 :ok :write [8 0] INFO [2022-01-10 12:53:02,426] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:02,427] jepsen worker 26 - jepsen.util 26 :invoke :write [8 37] INFO [2022-01-10 12:53:02,427] jepsen worker 29 - jepsen.util 29 :ok :write [9 0] INFO [2022-01-10 12:53:02,428] jepsen worker 29 - jepsen.util 29 :invoke :write [9 33] INFO [2022-01-10 12:53:02,428] jepsen worker 16 - jepsen.util 16 :ok :read [5 5] INFO [2022-01-10 12:53:02,428] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:53:02,429] jepsen worker 16 - jepsen.util 16 :invoke :write [5 31] INFO [2022-01-10 12:53:02,430] jepsen worker 50 - jepsen.util 50 :invoke :write [16 26] INFO [2022-01-10 12:53:02,432] jepsen worker 84 - jepsen.util 84 :ok :write [28 11] INFO [2022-01-10 12:53:02,433] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:02,434] jepsen worker 47 - jepsen.util 47 :ok :write [15 14] INFO [2022-01-10 12:53:02,434] jepsen worker 84 - jepsen.util 84 :ok :read [28 11] INFO [2022-01-10 12:53:02,435] jepsen worker 81 - jepsen.util 81 :ok :write [27 35] INFO [2022-01-10 12:53:02,435] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:02,436] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,436] jepsen worker 84 - jepsen.util 84 :ok :read [28 11] INFO [2022-01-10 12:53:02,436] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:02,437] jepsen worker 81 - jepsen.util 81 :ok :read [27 35] INFO [2022-01-10 12:53:02,437] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:02,438] jepsen worker 47 - jepsen.util 47 :ok :read [15 14] INFO [2022-01-10 12:53:02,438] jepsen worker 81 - jepsen.util 81 :invoke :write [27 8] INFO [2022-01-10 12:53:02,438] jepsen worker 84 - jepsen.util 84 :ok :read [28 11] INFO [2022-01-10 12:53:02,438] jepsen worker 36 - jepsen.util 36 :ok :write [12 3] INFO [2022-01-10 12:53:02,439] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:02,440] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,440] jepsen worker 84 - jepsen.util 84 :ok :read [28 11] INFO [2022-01-10 12:53:02,440] jepsen worker 67 - jepsen.util 67 :ok :write [22 30] INFO [2022-01-10 12:53:02,440] jepsen worker 47 - jepsen.util 47 :invoke :write [15 9] INFO [2022-01-10 12:53:02,441] jepsen worker 36 - jepsen.util 36 :ok :read [12 3] INFO [2022-01-10 12:53:02,441] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,441] jepsen worker 23 - jepsen.util 23 :ok :write [7 39] INFO [2022-01-10 12:53:02,441] jepsen worker 84 - jepsen.util 84 :invoke :write [28 7] INFO [2022-01-10 12:53:02,442] jepsen worker 67 - jepsen.util 67 :ok :read [22 30] INFO [2022-01-10 12:53:02,442] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,443] jepsen worker 23 - jepsen.util 23 :invoke :write [7 21] INFO [2022-01-10 12:53:02,443] jepsen worker 36 - jepsen.util 36 :ok :read [12 3] INFO [2022-01-10 12:53:02,443] jepsen worker 67 - jepsen.util 67 :invoke :write [22 0] INFO [2022-01-10 12:53:02,444] jepsen worker 36 - jepsen.util 36 :invoke :write [12 39] INFO [2022-01-10 12:53:02,451] jepsen worker 74 - jepsen.util 74 :ok :write [24 33] INFO [2022-01-10 12:53:02,451] jepsen worker 34 - jepsen.util 34 :ok :write [11 13] INFO [2022-01-10 12:53:02,451] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:02,452] jepsen worker 34 - jepsen.util 34 :invoke :write [11 9] INFO [2022-01-10 12:53:02,452] jepsen worker 3 - jepsen.util 3 :ok :write [1 22] INFO [2022-01-10 12:53:02,452] jepsen worker 74 - jepsen.util 74 :ok :read [24 33] INFO [2022-01-10 12:53:02,453] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:02,453] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:02,454] jepsen worker 3 - jepsen.util 3 :ok :read [1 22] INFO [2022-01-10 12:53:02,454] jepsen worker 74 - jepsen.util 74 :ok :read [24 33] INFO [2022-01-10 12:53:02,454] jepsen worker 3 - jepsen.util 3 :invoke :write [1 49] INFO [2022-01-10 12:53:02,455] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:02,456] jepsen worker 74 - jepsen.util 74 :ok :read [24 33] INFO [2022-01-10 12:53:02,456] jepsen worker 29 - jepsen.util 29 :ok :write [9 33] INFO [2022-01-10 12:53:02,457] jepsen worker 74 - jepsen.util 74 :invoke :write [24 19] INFO [2022-01-10 12:53:02,457] jepsen worker 0 - jepsen.util 0 :ok :write [0 43] INFO [2022-01-10 12:53:02,457] jepsen worker 29 - jepsen.util 29 :invoke :write [9 28] INFO [2022-01-10 12:53:02,458] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:02,459] jepsen worker 0 - jepsen.util 0 :ok :read [0 43] INFO [2022-01-10 12:53:02,459] jepsen worker 7 - jepsen.util 7 :ok :write [2 15] INFO [2022-01-10 12:53:02,460] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:02,460] jepsen worker 40 - jepsen.util 40 :ok :write [13 49] INFO [2022-01-10 12:53:02,460] jepsen worker 51 - jepsen.util 51 :ok :write [17 9] INFO [2022-01-10 12:53:02,460] jepsen worker 7 - jepsen.util 7 :invoke :write [2 15] INFO [2022-01-10 12:53:02,461] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:02,461] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:02,461] jepsen worker 0 - jepsen.util 0 :ok :read [0 43] INFO [2022-01-10 12:53:02,462] jepsen worker 40 - jepsen.util 40 :ok :read [13 49] INFO [2022-01-10 12:53:02,462] jepsen worker 0 - jepsen.util 0 :invoke :write [0 13] INFO [2022-01-10 12:53:02,462] jepsen worker 51 - jepsen.util 51 :ok :read [17 9] INFO [2022-01-10 12:53:02,463] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:02,463] jepsen worker 51 - jepsen.util 51 :invoke :write [17 38] INFO [2022-01-10 12:53:02,464] jepsen worker 40 - jepsen.util 40 :ok :read [13 49] INFO [2022-01-10 12:53:02,464] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:02,465] jepsen worker 40 - jepsen.util 40 :ok :read [13 49] INFO [2022-01-10 12:53:02,465] jepsen worker 40 - jepsen.util 40 :invoke :write [13 26] INFO [2022-01-10 12:53:02,473] jepsen worker 87 - jepsen.util 87 :ok :write [29 37] INFO [2022-01-10 12:53:02,473] jepsen worker 16 - jepsen.util 16 :ok :write [5 31] INFO [2022-01-10 12:53:02,473] jepsen worker 87 - jepsen.util 87 :invoke :write [29 37] INFO [2022-01-10 12:53:02,474] jepsen worker 16 - jepsen.util 16 :invoke :write [5 24] INFO [2022-01-10 12:53:02,475] jepsen worker 78 - jepsen.util 78 :ok :write [26 33] INFO [2022-01-10 12:53:02,475] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:02,476] jepsen worker 78 - jepsen.util 78 :ok :read [26 33] INFO [2022-01-10 12:53:02,477] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:02,478] jepsen worker 78 - jepsen.util 78 :ok :read [26 33] INFO [2022-01-10 12:53:02,478] jepsen worker 78 - jepsen.util 78 :invoke :write [26 23] INFO [2022-01-10 12:53:02,482] jepsen worker 58 - jepsen.util 58 :ok :write [19 26] INFO [2022-01-10 12:53:02,482] jepsen worker 58 - jepsen.util 58 :invoke :write [19 48] INFO [2022-01-10 12:53:02,505] jepsen worker 50 - jepsen.util 50 :ok :write [16 26] INFO [2022-01-10 12:53:02,506] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,507] jepsen worker 50 - jepsen.util 50 :ok :read [16 26] INFO [2022-01-10 12:53:02,507] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,508] jepsen worker 50 - jepsen.util 50 :ok :read [16 26] INFO [2022-01-10 12:53:02,508] jepsen worker 43 - jepsen.util 43 :ok :write [14 34] INFO [2022-01-10 12:53:02,509] jepsen worker 50 - jepsen.util 50 :invoke :write [16 18] INFO [2022-01-10 12:53:02,509] jepsen worker 13 - jepsen.util 13 :ok :write [4 10] INFO [2022-01-10 12:53:02,510] jepsen worker 65 - jepsen.util 65 :ok :write [21 13] INFO [2022-01-10 12:53:02,510] jepsen worker 81 - jepsen.util 81 :ok :write [27 8] INFO [2022-01-10 12:53:02,510] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:02,510] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:02,511] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,511] jepsen worker 43 - jepsen.util 43 :ok :read [14 34] INFO [2022-01-10 12:53:02,511] jepsen worker 65 - jepsen.util 65 :invoke :write [21 19] INFO [2022-01-10 12:53:02,512] jepsen worker 43 - jepsen.util 43 :invoke :write [14 24] INFO [2022-01-10 12:53:02,512] jepsen worker 13 - jepsen.util 13 :ok :read [4 10] INFO [2022-01-10 12:53:02,512] jepsen worker 81 - jepsen.util 81 :ok :read [27 8] INFO [2022-01-10 12:53:02,512] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:02,513] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,513] jepsen worker 13 - jepsen.util 13 :ok :read [4 10] INFO [2022-01-10 12:53:02,513] jepsen worker 81 - jepsen.util 81 :ok :read [27 8] INFO [2022-01-10 12:53:02,514] jepsen worker 13 - jepsen.util 13 :invoke :write [4 8] INFO [2022-01-10 12:53:02,514] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,515] jepsen worker 81 - jepsen.util 81 :ok :read [27 8] INFO [2022-01-10 12:53:02,515] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,516] jepsen worker 81 - jepsen.util 81 :ok :read [27 8] INFO [2022-01-10 12:53:02,516] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,517] jepsen worker 81 - jepsen.util 81 :ok :read [27 8] INFO [2022-01-10 12:53:02,518] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,518] jepsen worker 81 - jepsen.util 81 :ok :read [27 8] INFO [2022-01-10 12:53:02,519] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,524] jepsen worker 81 - jepsen.util 81 :ok :read [27 8] INFO [2022-01-10 12:53:02,524] jepsen worker 81 - jepsen.util 81 :invoke :write [27 23] INFO [2022-01-10 12:53:02,530] jepsen worker 18 - jepsen.util 18 :ok :write [6 6] INFO [2022-01-10 12:53:02,531] jepsen worker 18 - jepsen.util 18 :invoke :read [6 nil] INFO [2022-01-10 12:53:02,533] jepsen worker 18 - jepsen.util 18 :ok :read [6 6] INFO [2022-01-10 12:53:02,533] jepsen worker 76 - jepsen.util 76 :ok :write [25 10] INFO [2022-01-10 12:53:02,533] jepsen worker 67 - jepsen.util 67 :ok :write [22 0] INFO [2022-01-10 12:53:02,533] jepsen worker 69 - jepsen.util 69 :ok :write [23 2] INFO [2022-01-10 12:53:02,534] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:02,535] jepsen worker 76 - jepsen.util 76 :ok :read [25 10] INFO [2022-01-10 12:53:02,535] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:02,535] jepsen worker 26 - jepsen.util 26 :ok :write [8 37] INFO [2022-01-10 12:53:02,536] jepsen worker 18 - jepsen.util 18 :invoke :write [6 8] INFO [2022-01-10 12:53:02,536] jepsen worker 69 - jepsen.util 69 :ok :read [23 2] INFO [2022-01-10 12:53:02,536] jepsen worker 26 - jepsen.util 26 :invoke :read [8 nil] INFO [2022-01-10 12:53:02,537] jepsen worker 67 - jepsen.util 67 :invoke :write [22 30] INFO [2022-01-10 12:53:02,537] jepsen worker 26 - jepsen.util 26 :ok :read [8 37] INFO [2022-01-10 12:53:02,538] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:02,538] jepsen worker 26 - jepsen.util 26 :invoke :write [8 25] INFO [2022-01-10 12:53:02,538] jepsen worker 76 - jepsen.util 76 :ok :read [25 10] INFO [2022-01-10 12:53:02,539] jepsen worker 47 - jepsen.util 47 :ok :write [15 9] INFO [2022-01-10 12:53:02,539] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:02,540] jepsen worker 47 - jepsen.util 47 :invoke :write [15 10] INFO [2022-01-10 12:53:02,540] jepsen worker 69 - jepsen.util 69 :ok :read [23 2] INFO [2022-01-10 12:53:02,540] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:02,541] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:02,541] jepsen worker 76 - jepsen.util 76 :ok :read [25 10] INFO [2022-01-10 12:53:02,541] jepsen worker 60 - jepsen.util 60 :ok :write [20 49] INFO [2022-01-10 12:53:02,542] jepsen worker 69 - jepsen.util 69 :ok :read [23 2] INFO [2022-01-10 12:53:02,543] jepsen worker 60 - jepsen.util 60 :invoke :write [20 18] INFO [2022-01-10 12:53:02,543] jepsen worker 74 - jepsen.util 74 :ok :write [24 19] INFO [2022-01-10 12:53:02,543] jepsen worker 69 - jepsen.util 69 :invoke :write [23 12] INFO [2022-01-10 12:53:02,544] jepsen worker 74 - jepsen.util 74 :invoke :write [24 37] INFO [2022-01-10 12:53:02,545] jepsen worker 76 - jepsen.util 76 :invoke :write [25 1] INFO [2022-01-10 12:53:02,545] jepsen worker 11 - jepsen.util 11 :ok :write [3 30] INFO [2022-01-10 12:53:02,545] jepsen worker 29 - jepsen.util 29 :ok :write [9 28] INFO [2022-01-10 12:53:02,546] jepsen worker 29 - jepsen.util 29 :invoke :write [9 1] INFO [2022-01-10 12:53:02,547] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:02,548] jepsen worker 11 - jepsen.util 11 :ok :read [3 30] INFO [2022-01-10 12:53:02,548] jepsen worker 55 - jepsen.util 55 :ok :write [18 2] INFO [2022-01-10 12:53:02,549] jepsen worker 55 - jepsen.util 55 :invoke :write [18 46] INFO [2022-01-10 12:53:02,549] jepsen worker 11 - jepsen.util 11 :invoke :read [3 nil] INFO [2022-01-10 12:53:02,549] jepsen worker 58 - jepsen.util 58 :ok :write [19 48] INFO [2022-01-10 12:53:02,550] jepsen worker 58 - jepsen.util 58 :invoke :write [19 45] INFO [2022-01-10 12:53:02,550] jepsen worker 11 - jepsen.util 11 :ok :read [3 30] INFO [2022-01-10 12:53:02,551] jepsen worker 11 - jepsen.util 11 :invoke :write [3 41] INFO [2022-01-10 12:53:02,564] jepsen worker 30 - jepsen.util 30 :ok :write [10 38] INFO [2022-01-10 12:53:02,565] jepsen worker 34 - jepsen.util 34 :ok :write [11 9] INFO [2022-01-10 12:53:02,565] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:02,565] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:02,566] jepsen worker 30 - jepsen.util 30 :ok :read [10 38] INFO [2022-01-10 12:53:02,566] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:53:02,567] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:02,567] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:02,567] jepsen worker 30 - jepsen.util 30 :ok :read [10 38] INFO [2022-01-10 12:53:02,568] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:02,568] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:53:02,568] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:02,568] jepsen worker 30 - jepsen.util 30 :ok :read [10 38] INFO [2022-01-10 12:53:02,569] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:02,569] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:53:02,570] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:02,570] jepsen worker 30 - jepsen.util 30 :ok :read [10 38] INFO [2022-01-10 12:53:02,570] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:53:02,570] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:02,571] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:02,571] jepsen worker 30 - jepsen.util 30 :ok :read [10 38] INFO [2022-01-10 12:53:02,572] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:53:02,572] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:02,572] jepsen worker 34 - jepsen.util 34 :invoke :write [11 37] INFO [2022-01-10 12:53:02,572] jepsen worker 30 - jepsen.util 30 :ok :read [10 38] INFO [2022-01-10 12:53:02,573] jepsen worker 30 - jepsen.util 30 :invoke :read [10 nil] INFO [2022-01-10 12:53:02,574] jepsen worker 30 - jepsen.util 30 :ok :read [10 38] INFO [2022-01-10 12:53:02,574] jepsen worker 30 - jepsen.util 30 :invoke :write [10 43] INFO [2022-01-10 12:53:02,586] jepsen worker 23 - jepsen.util 23 :ok :write [7 21] INFO [2022-01-10 12:53:02,586] jepsen worker 65 - jepsen.util 65 :ok :write [21 19] INFO [2022-01-10 12:53:02,587] jepsen worker 84 - jepsen.util 84 :ok :write [28 7] INFO [2022-01-10 12:53:02,587] jepsen worker 23 - jepsen.util 23 :invoke :write [7 13] INFO [2022-01-10 12:53:02,588] jepsen worker 65 - jepsen.util 65 :invoke :write [21 4] INFO [2022-01-10 12:53:02,588] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:02,589] jepsen worker 84 - jepsen.util 84 :ok :read [28 7] INFO [2022-01-10 12:53:02,590] jepsen worker 84 - jepsen.util 84 :invoke :write [28 29] INFO [2022-01-10 12:53:02,592] jepsen worker 0 - jepsen.util 0 :ok :write [0 13] INFO [2022-01-10 12:53:02,592] jepsen worker 18 - jepsen.util 18 :ok :write [6 8] INFO [2022-01-10 12:53:02,592] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:02,593] jepsen worker 36 - jepsen.util 36 :ok :write [12 39] INFO [2022-01-10 12:53:02,593] jepsen worker 18 - jepsen.util 18 :invoke :write [6 38] INFO [2022-01-10 12:53:02,593] jepsen worker 0 - jepsen.util 0 :ok :read [0 13] INFO [2022-01-10 12:53:02,594] jepsen worker 36 - jepsen.util 36 :invoke :write [12 36] INFO [2022-01-10 12:53:02,594] jepsen worker 0 - jepsen.util 0 :invoke :write [0 28] INFO [2022-01-10 12:53:02,601] jepsen worker 3 - jepsen.util 3 :ok :write [1 49] INFO [2022-01-10 12:53:02,602] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:02,602] jepsen worker 26 - jepsen.util 26 :ok :write [8 25] INFO [2022-01-10 12:53:02,603] jepsen worker 26 - jepsen.util 26 :invoke :write [8 40] INFO [2022-01-10 12:53:02,603] jepsen worker 3 - jepsen.util 3 :ok :read [1 49] INFO [2022-01-10 12:53:02,604] jepsen worker 3 - jepsen.util 3 :invoke :write [1 41] INFO [2022-01-10 12:53:02,604] jepsen worker 43 - jepsen.util 43 :ok :write [14 24] INFO [2022-01-10 12:53:02,605] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:02,606] jepsen worker 43 - jepsen.util 43 :ok :read [14 24] INFO [2022-01-10 12:53:02,607] jepsen worker 43 - jepsen.util 43 :invoke :write [14 18] INFO [2022-01-10 12:53:02,609] jepsen worker 51 - jepsen.util 51 :ok :write [17 38] INFO [2022-01-10 12:53:02,610] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:02,611] jepsen worker 51 - jepsen.util 51 :ok :read [17 38] INFO [2022-01-10 12:53:02,611] jepsen worker 51 - jepsen.util 51 :invoke :write [17 18] INFO [2022-01-10 12:53:02,613] jepsen worker 76 - jepsen.util 76 :ok :write [25 1] INFO [2022-01-10 12:53:02,613] jepsen worker 47 - jepsen.util 47 :ok :write [15 10] INFO [2022-01-10 12:53:02,614] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:02,614] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:02,615] jepsen worker 76 - jepsen.util 76 :ok :read [25 1] INFO [2022-01-10 12:53:02,615] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:02,615] jepsen worker 47 - jepsen.util 47 :ok :read [15 10] INFO [2022-01-10 12:53:02,616] jepsen worker 47 - jepsen.util 47 :invoke :read [15 nil] INFO [2022-01-10 12:53:02,616] jepsen worker 78 - jepsen.util 78 :ok :write [26 23] INFO [2022-01-10 12:53:02,616] jepsen worker 76 - jepsen.util 76 :ok :read [25 1] INFO [2022-01-10 12:53:02,616] jepsen worker 55 - jepsen.util 55 :ok :write [18 46] INFO [2022-01-10 12:53:02,617] jepsen worker 47 - jepsen.util 47 :ok :read [15 10] INFO [2022-01-10 12:53:02,617] jepsen worker 78 - jepsen.util 78 :invoke :write [26 33] INFO [2022-01-10 12:53:02,617] jepsen worker 55 - jepsen.util 55 :invoke :read [18 nil] INFO [2022-01-10 12:53:02,618] jepsen worker 47 - jepsen.util 47 :invoke :write [15 45] INFO [2022-01-10 12:53:02,618] jepsen worker 76 - jepsen.util 76 :invoke :write [25 27] INFO [2022-01-10 12:53:02,619] jepsen worker 55 - jepsen.util 55 :ok :read [18 46] INFO [2022-01-10 12:53:02,619] jepsen worker 55 - jepsen.util 55 :invoke :write [18 12] INFO [2022-01-10 12:53:02,625] jepsen worker 13 - jepsen.util 13 :ok :write [4 8] INFO [2022-01-10 12:53:02,626] jepsen worker 60 - jepsen.util 60 :ok :write [20 18] INFO [2022-01-10 12:53:02,626] jepsen worker 13 - jepsen.util 13 :invoke :write [4 17] INFO [2022-01-10 12:53:02,627] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:02,627] jepsen worker 60 - jepsen.util 60 :ok :read [20 18] INFO [2022-01-10 12:53:02,628] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:02,629] jepsen worker 60 - jepsen.util 60 :ok :read [20 18] INFO [2022-01-10 12:53:02,630] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:02,631] jepsen worker 60 - jepsen.util 60 :ok :read [20 18] INFO [2022-01-10 12:53:02,631] jepsen worker 69 - jepsen.util 69 :ok :write [23 12] INFO [2022-01-10 12:53:02,631] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:02,632] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:02,632] jepsen worker 60 - jepsen.util 60 :ok :read [20 18] INFO [2022-01-10 12:53:02,633] jepsen worker 60 - jepsen.util 60 :invoke :write [20 42] INFO [2022-01-10 12:53:02,633] jepsen worker 69 - jepsen.util 69 :ok :read [23 12] INFO [2022-01-10 12:53:02,634] jepsen worker 69 - jepsen.util 69 :invoke :write [23 28] INFO [2022-01-10 12:53:02,643] jepsen worker 74 - jepsen.util 74 :ok :write [24 37] INFO [2022-01-10 12:53:02,643] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:02,646] jepsen worker 74 - jepsen.util 74 :ok :read [24 37] INFO [2022-01-10 12:53:02,646] jepsen worker 74 - jepsen.util 74 :invoke :read [24 nil] INFO [2022-01-10 12:53:02,647] jepsen worker 74 - jepsen.util 74 :ok :read [24 37] INFO [2022-01-10 12:53:02,648] jepsen worker 74 - jepsen.util 74 :invoke :write [24 34] INFO [2022-01-10 12:53:02,680] jepsen worker 40 - jepsen.util 40 :ok :write [13 26] INFO [2022-01-10 12:53:02,681] jepsen worker 40 - jepsen.util 40 :invoke :write [13 15] INFO [2022-01-10 12:53:02,686] jepsen worker 7 - jepsen.util 7 :ok :write [2 15] INFO [2022-01-10 12:53:02,687] jepsen worker 7 - jepsen.util 7 :invoke :write [2 16] INFO [2022-01-10 12:53:02,693] jepsen worker 16 - jepsen.util 16 :ok :write [5 24] INFO [2022-01-10 12:53:02,693] jepsen worker 84 - jepsen.util 84 :ok :write [28 29] INFO [2022-01-10 12:53:02,694] jepsen worker 16 - jepsen.util 16 :invoke :write [5 31] INFO [2022-01-10 12:53:02,695] jepsen worker 84 - jepsen.util 84 :invoke :write [28 4] INFO [2022-01-10 12:53:02,700] jepsen worker 36 - jepsen.util 36 :ok :write [12 36] INFO [2022-01-10 12:53:02,700] jepsen worker 87 - jepsen.util 87 :ok :write [29 37] INFO [2022-01-10 12:53:02,701] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:02,702] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,703] jepsen worker 87 - jepsen.util 87 :ok :read [29 37] INFO [2022-01-10 12:53:02,703] jepsen worker 36 - jepsen.util 36 :ok :read [12 36] INFO [2022-01-10 12:53:02,703] jepsen worker 87 - jepsen.util 87 :invoke :write [29 26] INFO [2022-01-10 12:53:02,704] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,705] jepsen worker 36 - jepsen.util 36 :ok :read [12 36] INFO [2022-01-10 12:53:02,705] jepsen worker 3 - jepsen.util 3 :ok :write [1 41] INFO [2022-01-10 12:53:02,706] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,707] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:02,707] jepsen worker 36 - jepsen.util 36 :ok :read [12 36] INFO [2022-01-10 12:53:02,707] jepsen worker 3 - jepsen.util 3 :ok :read [1 41] INFO [2022-01-10 12:53:02,708] jepsen worker 50 - jepsen.util 50 :ok :write [16 18] INFO [2022-01-10 12:53:02,708] jepsen worker 36 - jepsen.util 36 :invoke :write [12 11] INFO [2022-01-10 12:53:02,709] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:02,709] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,709] jepsen worker 84 - jepsen.util 84 :ok :write [28 4] INFO [2022-01-10 12:53:02,710] jepsen worker 3 - jepsen.util 3 :ok :read [1 41] INFO [2022-01-10 12:53:02,710] jepsen worker 50 - jepsen.util 50 :ok :read [16 18] INFO [2022-01-10 12:53:02,711] jepsen worker 3 - jepsen.util 3 :invoke :write [1 45] INFO [2022-01-10 12:53:02,712] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,712] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:02,712] jepsen worker 50 - jepsen.util 50 :ok :read [16 18] INFO [2022-01-10 12:53:02,713] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:53:02,713] jepsen worker 84 - jepsen.util 84 :ok :read [28 4] INFO [2022-01-10 12:53:02,715] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:02,715] jepsen worker 84 - jepsen.util 84 :ok :read [28 4] INFO [2022-01-10 12:53:02,716] jepsen worker 84 - jepsen.util 84 :invoke :write [28 46] INFO [2022-01-10 12:53:02,717] jepsen worker 81 - jepsen.util 81 :ok :write [27 23] INFO [2022-01-10 12:53:02,717] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,718] jepsen worker 81 - jepsen.util 81 :ok :read [27 23] INFO [2022-01-10 12:53:02,719] jepsen worker 81 - jepsen.util 81 :invoke :write [27 3] INFO [2022-01-10 12:53:02,724] jepsen worker 3 - jepsen.util 3 :ok :write [1 45] INFO [2022-01-10 12:53:02,724] jepsen worker 67 - jepsen.util 67 :ok :write [22 30] INFO [2022-01-10 12:53:02,725] jepsen worker 3 - jepsen.util 3 :invoke :write [1 9] INFO [2022-01-10 12:53:02,726] jepsen worker 67 - jepsen.util 67 :invoke :write [22 4] INFO [2022-01-10 12:53:02,728] jepsen worker 36 - jepsen.util 36 :ok :write [12 11] INFO [2022-01-10 12:53:02,729] jepsen worker 36 - jepsen.util 36 :invoke :write [12 23] INFO [2022-01-10 12:53:02,730] jepsen worker 84 - jepsen.util 84 :ok :write [28 46] INFO [2022-01-10 12:53:02,731] jepsen worker 84 - jepsen.util 84 :invoke :write [28 11] INFO [2022-01-10 12:53:02,734] jepsen worker 29 - jepsen.util 29 :ok :write [9 1] INFO [2022-01-10 12:53:02,735] jepsen worker 29 - jepsen.util 29 :invoke :write [9 47] INFO [2022-01-10 12:53:02,738] jepsen worker 34 - jepsen.util 34 :ok :write [11 37] INFO [2022-01-10 12:53:02,738] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:02,739] jepsen worker 34 - jepsen.util 34 :ok :read [11 37] INFO [2022-01-10 12:53:02,740] jepsen worker 0 - jepsen.util 0 :ok :write [0 28] INFO [2022-01-10 12:53:02,740] jepsen worker 34 - jepsen.util 34 :invoke :write [11 40] INFO [2022-01-10 12:53:02,741] jepsen worker 3 - jepsen.util 3 :ok :write [1 9] INFO [2022-01-10 12:53:02,741] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:02,742] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:02,742] jepsen worker 0 - jepsen.util 0 :ok :read [0 28] INFO [2022-01-10 12:53:02,743] jepsen worker 3 - jepsen.util 3 :ok :read [1 9] INFO [2022-01-10 12:53:02,744] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:02,745] jepsen worker 0 - jepsen.util 0 :ok :read [0 28] INFO [2022-01-10 12:53:02,745] jepsen worker 3 - jepsen.util 3 :invoke :write [1 49] INFO [2022-01-10 12:53:02,745] jepsen worker 0 - jepsen.util 0 :invoke :write [0 25] INFO [2022-01-10 12:53:02,746] jepsen worker 36 - jepsen.util 36 :ok :write [12 23] INFO [2022-01-10 12:53:02,747] jepsen worker 36 - jepsen.util 36 :invoke :write [12 23] INFO [2022-01-10 12:53:02,749] jepsen worker 40 - jepsen.util 40 :ok :write [13 15] INFO [2022-01-10 12:53:02,749] jepsen worker 84 - jepsen.util 84 :ok :write [28 11] INFO [2022-01-10 12:53:02,750] jepsen worker 40 - jepsen.util 40 :invoke :write [13 27] INFO [2022-01-10 12:53:02,751] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:02,752] jepsen worker 84 - jepsen.util 84 :ok :read [28 11] INFO [2022-01-10 12:53:02,752] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:02,753] jepsen worker 84 - jepsen.util 84 :ok :read [28 11] INFO [2022-01-10 12:53:02,754] jepsen worker 84 - jepsen.util 84 :invoke :write [28 29] INFO [2022-01-10 12:53:02,761] jepsen worker 3 - jepsen.util 3 :ok :write [1 49] INFO [2022-01-10 12:53:02,762] jepsen worker 3 - jepsen.util 3 :invoke :write [1 21] INFO [2022-01-10 12:53:02,765] jepsen worker 23 - jepsen.util 23 :ok :write [7 13] INFO [2022-01-10 12:53:02,766] jepsen worker 36 - jepsen.util 36 :ok :write [12 23] INFO [2022-01-10 12:53:02,766] jepsen worker 87 - jepsen.util 87 :ok :write [29 26] INFO [2022-01-10 12:53:02,766] jepsen worker 23 - jepsen.util 23 :invoke :write [7 31] INFO [2022-01-10 12:53:02,767] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,768] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:02,768] jepsen worker 36 - jepsen.util 36 :ok :read [12 23] INFO [2022-01-10 12:53:02,769] jepsen worker 87 - jepsen.util 87 :ok :read [29 26] INFO [2022-01-10 12:53:02,769] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,769] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:02,770] jepsen worker 36 - jepsen.util 36 :ok :read [12 23] INFO [2022-01-10 12:53:02,770] jepsen worker 87 - jepsen.util 87 :ok :read [29 26] INFO [2022-01-10 12:53:02,771] jepsen worker 36 - jepsen.util 36 :invoke :write [12 44] INFO [2022-01-10 12:53:02,771] jepsen worker 87 - jepsen.util 87 :invoke :write [29 43] INFO [2022-01-10 12:53:02,771] jepsen worker 43 - jepsen.util 43 :ok :write [14 18] INFO [2022-01-10 12:53:02,772] jepsen worker 43 - jepsen.util 43 :invoke :write [14 38] INFO [2022-01-10 12:53:02,775] jepsen worker 51 - jepsen.util 51 :ok :write [17 18] INFO [2022-01-10 12:53:02,776] jepsen worker 51 - jepsen.util 51 :invoke :write [17 43] INFO [2022-01-10 12:53:02,780] jepsen worker 76 - jepsen.util 76 :ok :write [25 27] INFO [2022-01-10 12:53:02,780] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:02,782] jepsen worker 76 - jepsen.util 76 :ok :read [25 27] INFO [2022-01-10 12:53:02,782] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:02,783] jepsen worker 76 - jepsen.util 76 :ok :read [25 27] INFO [2022-01-10 12:53:02,784] jepsen worker 76 - jepsen.util 76 :invoke :write [25 17] INFO [2022-01-10 12:53:02,787] jepsen worker 78 - jepsen.util 78 :ok :write [26 33] INFO [2022-01-10 12:53:02,788] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:02,788] jepsen worker 60 - jepsen.util 60 :ok :write [20 42] INFO [2022-01-10 12:53:02,789] jepsen worker 78 - jepsen.util 78 :ok :read [26 33] INFO [2022-01-10 12:53:02,789] jepsen worker 60 - jepsen.util 60 :invoke :write [20 23] INFO [2022-01-10 12:53:02,789] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:02,790] jepsen worker 78 - jepsen.util 78 :ok :read [26 33] INFO [2022-01-10 12:53:02,790] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:02,791] jepsen worker 78 - jepsen.util 78 :ok :read [26 33] INFO [2022-01-10 12:53:02,792] jepsen worker 78 - jepsen.util 78 :invoke :write [26 29] INFO [2022-01-10 12:53:02,792] jepsen worker 13 - jepsen.util 13 :ok :write [4 17] INFO [2022-01-10 12:53:02,793] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:02,794] jepsen worker 13 - jepsen.util 13 :ok :read [4 17] INFO [2022-01-10 12:53:02,794] jepsen worker 7 - jepsen.util 7 :ok :write [2 16] INFO [2022-01-10 12:53:02,794] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:02,795] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:02,795] jepsen worker 0 - jepsen.util 0 :ok :write [0 25] INFO [2022-01-10 12:53:02,795] jepsen worker 7 - jepsen.util 7 :ok :read [2 16] INFO [2022-01-10 12:53:02,796] jepsen worker 13 - jepsen.util 13 :ok :read [4 17] INFO [2022-01-10 12:53:02,796] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:02,796] jepsen worker 13 - jepsen.util 13 :invoke :write [4 23] INFO [2022-01-10 12:53:02,797] jepsen worker 0 - jepsen.util 0 :ok :read [0 25] INFO [2022-01-10 12:53:02,797] jepsen worker 7 - jepsen.util 7 :invoke :write [2 42] INFO [2022-01-10 12:53:02,797] jepsen worker 69 - jepsen.util 69 :ok :write [23 28] INFO [2022-01-10 12:53:02,798] jepsen worker 0 - jepsen.util 0 :invoke :write [0 8] INFO [2022-01-10 12:53:02,798] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:02,799] jepsen worker 69 - jepsen.util 69 :ok :read [23 28] INFO [2022-01-10 12:53:02,800] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:02,800] jepsen worker 16 - jepsen.util 16 :ok :write [5 31] INFO [2022-01-10 12:53:02,800] jepsen worker 69 - jepsen.util 69 :ok :read [23 28] INFO [2022-01-10 12:53:02,801] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:02,801] jepsen worker 69 - jepsen.util 69 :invoke :write [23 4] INFO [2022-01-10 12:53:02,802] jepsen worker 16 - jepsen.util 16 :ok :read [5 31] INFO [2022-01-10 12:53:02,802] jepsen worker 16 - jepsen.util 16 :invoke :write [5 46] INFO [2022-01-10 12:53:02,803] jepsen worker 40 - jepsen.util 40 :ok :write [13 27] INFO [2022-01-10 12:53:02,803] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:02,804] jepsen worker 40 - jepsen.util 40 :ok :read [13 27] INFO [2022-01-10 12:53:02,804] jepsen worker 40 - jepsen.util 40 :invoke :write [13 7] INFO [2022-01-10 12:53:02,804] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:53:02,805] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,806] jepsen worker 50 - jepsen.util 50 :ok :read [16 30] INFO [2022-01-10 12:53:02,807] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,808] jepsen worker 50 - jepsen.util 50 :ok :read [16 30] INFO [2022-01-10 12:53:02,808] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,810] jepsen worker 50 - jepsen.util 50 :ok :read [16 30] INFO [2022-01-10 12:53:02,810] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:02,811] jepsen worker 81 - jepsen.util 81 :ok :write [27 3] INFO [2022-01-10 12:53:02,811] jepsen worker 50 - jepsen.util 50 :ok :read [16 30] INFO [2022-01-10 12:53:02,811] jepsen worker 81 - jepsen.util 81 :invoke :write [27 40] INFO [2022-01-10 12:53:02,812] jepsen worker 50 - jepsen.util 50 :invoke :write [16 34] INFO [2022-01-10 12:53:02,817] jepsen worker 67 - jepsen.util 67 :ok :write [22 4] INFO [2022-01-10 12:53:02,817] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,818] jepsen worker 67 - jepsen.util 67 :ok :read [22 4] INFO [2022-01-10 12:53:02,819] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,820] jepsen worker 67 - jepsen.util 67 :ok :read [22 4] INFO [2022-01-10 12:53:02,820] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,821] jepsen worker 67 - jepsen.util 67 :ok :read [22 4] INFO [2022-01-10 12:53:02,822] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,822] jepsen worker 67 - jepsen.util 67 :ok :read [22 4] INFO [2022-01-10 12:53:02,823] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,824] jepsen worker 67 - jepsen.util 67 :ok :read [22 4] INFO [2022-01-10 12:53:02,824] jepsen worker 34 - jepsen.util 34 :ok :write [11 40] INFO [2022-01-10 12:53:02,824] jepsen worker 67 - jepsen.util 67 :invoke :write [22 37] INFO [2022-01-10 12:53:02,825] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:02,826] jepsen worker 34 - jepsen.util 34 :ok :read [11 40] INFO [2022-01-10 12:53:02,826] jepsen worker 34 - jepsen.util 34 :invoke :write [11 34] INFO [2022-01-10 12:53:02,848] jepsen worker 7 - jepsen.util 7 :ok :write [2 42] INFO [2022-01-10 12:53:02,848] jepsen worker 7 - jepsen.util 7 :invoke :write [2 31] INFO [2022-01-10 12:53:02,853] jepsen worker 16 - jepsen.util 16 :ok :write [5 46] INFO [2022-01-10 12:53:02,854] jepsen worker 16 - jepsen.util 16 :invoke :write [5 25] INFO [2022-01-10 12:53:02,858] jepsen worker 50 - jepsen.util 50 :ok :write [16 34] INFO [2022-01-10 12:53:02,859] jepsen worker 50 - jepsen.util 50 :invoke :write [16 7] INFO [2022-01-10 12:53:02,862] jepsen worker 81 - jepsen.util 81 :ok :write [27 40] INFO [2022-01-10 12:53:02,863] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,864] jepsen worker 81 - jepsen.util 81 :ok :read [27 40] INFO [2022-01-10 12:53:02,864] jepsen worker 81 - jepsen.util 81 :invoke :write [27 13] INFO [2022-01-10 12:53:02,867] jepsen worker 67 - jepsen.util 67 :ok :write [22 37] INFO [2022-01-10 12:53:02,867] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,868] jepsen worker 67 - jepsen.util 67 :ok :read [22 37] INFO [2022-01-10 12:53:02,869] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:02,869] jepsen worker 67 - jepsen.util 67 :ok :read [22 37] INFO [2022-01-10 12:53:02,870] jepsen worker 67 - jepsen.util 67 :invoke :write [22 24] INFO [2022-01-10 12:53:02,871] jepsen worker 34 - jepsen.util 34 :ok :write [11 34] INFO [2022-01-10 12:53:02,872] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:02,873] jepsen worker 34 - jepsen.util 34 :ok :read [11 34] INFO [2022-01-10 12:53:02,873] jepsen worker 34 - jepsen.util 34 :invoke :write [11 11] INFO [2022-01-10 12:53:02,877] jepsen worker 7 - jepsen.util 7 :ok :write [2 31] INFO [2022-01-10 12:53:02,877] jepsen worker 7 - jepsen.util 7 :invoke :write [2 22] INFO [2022-01-10 12:53:02,882] jepsen worker 16 - jepsen.util 16 :ok :write [5 25] INFO [2022-01-10 12:53:02,883] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:02,884] jepsen worker 16 - jepsen.util 16 :ok :read [5 25] INFO [2022-01-10 12:53:02,884] jepsen worker 16 - jepsen.util 16 :invoke :write [5 41] INFO [2022-01-10 12:53:02,886] jepsen worker 50 - jepsen.util 50 :ok :write [16 7] INFO [2022-01-10 12:53:02,887] jepsen worker 50 - jepsen.util 50 :invoke :write [16 2] INFO [2022-01-10 12:53:02,890] jepsen worker 81 - jepsen.util 81 :ok :write [27 13] INFO [2022-01-10 12:53:02,891] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,892] jepsen worker 81 - jepsen.util 81 :ok :read [27 13] INFO [2022-01-10 12:53:02,892] jepsen worker 84 - jepsen.util 84 :ok :write [28 29] INFO [2022-01-10 12:53:02,893] jepsen worker 81 - jepsen.util 81 :invoke :write [27 10] INFO [2022-01-10 12:53:02,893] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:02,894] jepsen worker 84 - jepsen.util 84 :ok :read [28 29] INFO [2022-01-10 12:53:02,895] jepsen worker 84 - jepsen.util 84 :invoke :write [28 39] INFO [2022-01-10 12:53:02,896] jepsen worker 3 - jepsen.util 3 :ok :write [1 21] INFO [2022-01-10 12:53:02,896] jepsen worker 67 - jepsen.util 67 :ok :write [22 24] INFO [2022-01-10 12:53:02,896] jepsen worker 3 - jepsen.util 3 :invoke :write [1 25] INFO [2022-01-10 12:53:02,897] jepsen worker 67 - jepsen.util 67 :invoke :write [22 3] INFO [2022-01-10 12:53:02,897] jepsen worker 23 - jepsen.util 23 :ok :write [7 31] INFO [2022-01-10 12:53:02,898] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:02,899] jepsen worker 23 - jepsen.util 23 :ok :read [7 31] INFO [2022-01-10 12:53:02,899] jepsen worker 23 - jepsen.util 23 :invoke :write [7 45] INFO [2022-01-10 12:53:02,899] jepsen worker 34 - jepsen.util 34 :ok :write [11 11] INFO [2022-01-10 12:53:02,900] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:02,901] jepsen worker 34 - jepsen.util 34 :ok :read [11 11] INFO [2022-01-10 12:53:02,902] jepsen worker 34 - jepsen.util 34 :invoke :write [11 19] INFO [2022-01-10 12:53:02,903] jepsen worker 43 - jepsen.util 43 :ok :write [14 38] INFO [2022-01-10 12:53:02,904] jepsen worker 43 - jepsen.util 43 :invoke :write [14 21] INFO [2022-01-10 12:53:02,906] jepsen worker 36 - jepsen.util 36 :ok :write [12 44] INFO [2022-01-10 12:53:02,906] jepsen worker 7 - jepsen.util 7 :ok :write [2 22] INFO [2022-01-10 12:53:02,907] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:02,907] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:02,908] jepsen worker 36 - jepsen.util 36 :ok :read [12 44] INFO [2022-01-10 12:53:02,908] jepsen worker 7 - jepsen.util 7 :ok :read [2 22] INFO [2022-01-10 12:53:02,909] jepsen worker 36 - jepsen.util 36 :invoke :write [12 42] INFO [2022-01-10 12:53:02,909] jepsen worker 87 - jepsen.util 87 :ok :write [29 43] INFO [2022-01-10 12:53:02,909] jepsen worker 7 - jepsen.util 7 :invoke :write [2 1] INFO [2022-01-10 12:53:02,910] jepsen worker 87 - jepsen.util 87 :invoke :write [29 1] INFO [2022-01-10 12:53:02,912] jepsen worker 16 - jepsen.util 16 :ok :write [5 41] INFO [2022-01-10 12:53:02,913] jepsen worker 16 - jepsen.util 16 :invoke :write [5 6] INFO [2022-01-10 12:53:02,916] jepsen worker 50 - jepsen.util 50 :ok :write [16 2] INFO [2022-01-10 12:53:02,917] jepsen worker 50 - jepsen.util 50 :invoke :write [16 46] INFO [2022-01-10 12:53:02,920] jepsen worker 81 - jepsen.util 81 :ok :write [27 10] INFO [2022-01-10 12:53:02,921] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:02,922] jepsen worker 81 - jepsen.util 81 :ok :read [27 10] INFO [2022-01-10 12:53:02,923] jepsen worker 81 - jepsen.util 81 :invoke :write [27 4] INFO [2022-01-10 12:53:02,928] jepsen worker 67 - jepsen.util 67 :ok :write [22 3] INFO [2022-01-10 12:53:02,929] jepsen worker 67 - jepsen.util 67 :invoke :write [22 14] INFO [2022-01-10 12:53:02,932] jepsen worker 34 - jepsen.util 34 :ok :write [11 19] INFO [2022-01-10 12:53:02,932] jepsen worker 34 - jepsen.util 34 :invoke :write [11 48] INFO [2022-01-10 12:53:02,941] jepsen worker 7 - jepsen.util 7 :ok :write [2 1] INFO [2022-01-10 12:53:02,942] jepsen worker 7 - jepsen.util 7 :invoke :write [2 48] INFO [2022-01-10 12:53:03,069] jepsen worker 16 - jepsen.util 16 :ok :write [5 6] INFO [2022-01-10 12:53:03,070] jepsen worker 16 - jepsen.util 16 :invoke :write [5 41] INFO [2022-01-10 12:53:03,074] jepsen worker 50 - jepsen.util 50 :ok :write [16 46] INFO [2022-01-10 12:53:03,075] jepsen worker 50 - jepsen.util 50 :invoke :write [16 25] INFO [2022-01-10 12:53:03,079] jepsen worker 81 - jepsen.util 81 :ok :write [27 4] INFO [2022-01-10 12:53:03,080] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,081] jepsen worker 81 - jepsen.util 81 :ok :read [27 4] INFO [2022-01-10 12:53:03,081] jepsen worker 81 - jepsen.util 81 :invoke :write [27 14] INFO [2022-01-10 12:53:03,084] jepsen worker 67 - jepsen.util 67 :ok :write [22 14] INFO [2022-01-10 12:53:03,085] jepsen worker 67 - jepsen.util 67 :invoke :write [22 36] INFO [2022-01-10 12:53:03,089] jepsen worker 34 - jepsen.util 34 :ok :write [11 48] INFO [2022-01-10 12:53:03,090] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,091] jepsen worker 34 - jepsen.util 34 :ok :read [11 48] INFO [2022-01-10 12:53:03,091] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,092] jepsen worker 34 - jepsen.util 34 :ok :read [11 48] INFO [2022-01-10 12:53:03,093] jepsen worker 34 - jepsen.util 34 :invoke :write [11 12] INFO [2022-01-10 12:53:03,093] jepsen worker 7 - jepsen.util 7 :ok :write [2 48] INFO [2022-01-10 12:53:03,094] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:03,095] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:03,095] jepsen worker 7 - jepsen.util 7 :invoke :write [2 4] INFO [2022-01-10 12:53:03,098] jepsen worker 16 - jepsen.util 16 :ok :write [5 41] INFO [2022-01-10 12:53:03,099] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,100] jepsen worker 16 - jepsen.util 16 :ok :read [5 41] INFO [2022-01-10 12:53:03,100] jepsen worker 16 - jepsen.util 16 :invoke :write [5 49] INFO [2022-01-10 12:53:03,105] jepsen worker 50 - jepsen.util 50 :ok :write [16 25] INFO [2022-01-10 12:53:03,106] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,106] jepsen worker 50 - jepsen.util 50 :ok :read [16 25] INFO [2022-01-10 12:53:03,107] jepsen worker 50 - jepsen.util 50 :invoke :write [16 44] INFO [2022-01-10 12:53:03,111] jepsen worker 81 - jepsen.util 81 :ok :write [27 14] INFO [2022-01-10 12:53:03,111] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,112] jepsen worker 81 - jepsen.util 81 :ok :read [27 14] INFO [2022-01-10 12:53:03,113] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,113] jepsen worker 81 - jepsen.util 81 :ok :read [27 14] INFO [2022-01-10 12:53:03,114] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,115] jepsen worker 81 - jepsen.util 81 :ok :read [27 14] INFO [2022-01-10 12:53:03,115] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,116] jepsen worker 81 - jepsen.util 81 :ok :read [27 14] INFO [2022-01-10 12:53:03,116] jepsen worker 81 - jepsen.util 81 :invoke :write [27 45] INFO [2022-01-10 12:53:03,117] jepsen worker 67 - jepsen.util 67 :ok :write [22 36] INFO [2022-01-10 12:53:03,117] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,118] jepsen worker 67 - jepsen.util 67 :ok :read [22 36] INFO [2022-01-10 12:53:03,119] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,120] jepsen worker 67 - jepsen.util 67 :ok :read [22 36] INFO [2022-01-10 12:53:03,120] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,121] jepsen worker 67 - jepsen.util 67 :ok :read [22 36] INFO [2022-01-10 12:53:03,122] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,123] jepsen worker 67 - jepsen.util 67 :ok :read [22 36] INFO [2022-01-10 12:53:03,123] jepsen worker 34 - jepsen.util 34 :ok :write [11 12] INFO [2022-01-10 12:53:03,123] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,124] jepsen worker 34 - jepsen.util 34 :invoke :write [11 18] INFO [2022-01-10 12:53:03,125] jepsen worker 67 - jepsen.util 67 :ok :read [22 36] INFO [2022-01-10 12:53:03,125] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,126] jepsen worker 67 - jepsen.util 67 :ok :read [22 36] INFO [2022-01-10 12:53:03,127] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,127] jepsen worker 67 - jepsen.util 67 :ok :read [22 36] INFO [2022-01-10 12:53:03,128] jepsen worker 7 - jepsen.util 7 :ok :write [2 4] INFO [2022-01-10 12:53:03,128] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,128] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:03,129] jepsen worker 67 - jepsen.util 67 :ok :read [22 36] INFO [2022-01-10 12:53:03,129] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:03,130] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,130] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:03,130] jepsen worker 67 - jepsen.util 67 :ok :read [22 36] INFO [2022-01-10 12:53:03,131] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:03,131] jepsen worker 67 - jepsen.util 67 :invoke :write [22 38] INFO [2022-01-10 12:53:03,131] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:03,132] jepsen worker 16 - jepsen.util 16 :ok :write [5 49] INFO [2022-01-10 12:53:03,132] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:03,133] jepsen worker 16 - jepsen.util 16 :invoke :write [5 41] INFO [2022-01-10 12:53:03,133] jepsen worker 7 - jepsen.util 7 :invoke :write [2 38] INFO [2022-01-10 12:53:03,137] jepsen worker 50 - jepsen.util 50 :ok :write [16 44] INFO [2022-01-10 12:53:03,138] jepsen worker 50 - jepsen.util 50 :invoke :write [16 22] INFO [2022-01-10 12:53:03,143] jepsen worker 81 - jepsen.util 81 :ok :write [27 45] INFO [2022-01-10 12:53:03,143] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,144] jepsen worker 81 - jepsen.util 81 :ok :read [27 45] INFO [2022-01-10 12:53:03,145] jepsen worker 81 - jepsen.util 81 :invoke :write [27 37] INFO [2022-01-10 12:53:03,153] jepsen worker 34 - jepsen.util 34 :ok :write [11 18] INFO [2022-01-10 12:53:03,153] jepsen worker 34 - jepsen.util 34 :invoke :write [11 47] INFO [2022-01-10 12:53:03,219] jepsen worker 76 - jepsen.util 76 :ok :write [25 17] INFO [2022-01-10 12:53:03,220] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:03,221] jepsen worker 51 - jepsen.util 51 :ok :write [17 43] INFO [2022-01-10 12:53:03,221] jepsen worker 76 - jepsen.util 76 :ok :read [25 17] INFO [2022-01-10 12:53:03,222] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:03,223] jepsen worker 76 - jepsen.util 76 :invoke :write [25 40] INFO [2022-01-10 12:53:03,223] jepsen worker 51 - jepsen.util 51 :ok :read [17 43] INFO [2022-01-10 12:53:03,224] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:03,225] jepsen worker 51 - jepsen.util 51 :ok :read [17 43] INFO [2022-01-10 12:53:03,225] jepsen worker 51 - jepsen.util 51 :invoke :write [17 28] INFO [2022-01-10 12:53:03,226] jepsen worker 60 - jepsen.util 60 :ok :write [20 23] INFO [2022-01-10 12:53:03,227] jepsen worker 60 - jepsen.util 60 :invoke :write [20 30] INFO [2022-01-10 12:53:03,228] jepsen worker 78 - jepsen.util 78 :ok :write [26 29] INFO [2022-01-10 12:53:03,228] jepsen worker 78 - jepsen.util 78 :invoke :write [26 23] INFO [2022-01-10 12:53:03,232] jepsen worker 0 - jepsen.util 0 :ok :write [0 8] INFO [2022-01-10 12:53:03,232] jepsen worker 0 - jepsen.util 0 :invoke :write [0 21] INFO [2022-01-10 12:53:03,236] jepsen worker 13 - jepsen.util 13 :ok :write [4 23] INFO [2022-01-10 12:53:03,237] jepsen worker 13 - jepsen.util 13 :invoke :write [4 15] INFO [2022-01-10 12:53:03,238] jepsen worker 40 - jepsen.util 40 :ok :write [13 7] INFO [2022-01-10 12:53:03,239] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:03,240] jepsen worker 40 - jepsen.util 40 :ok :read [13 7] INFO [2022-01-10 12:53:03,241] jepsen worker 40 - jepsen.util 40 :invoke :write [13 35] INFO [2022-01-10 12:53:03,244] jepsen worker 23 - jepsen.util 23 :ok :write [7 45] INFO [2022-01-10 12:53:03,244] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:03,245] jepsen worker 69 - jepsen.util 69 :ok :write [23 4] INFO [2022-01-10 12:53:03,245] jepsen worker 23 - jepsen.util 23 :ok :read [7 45] INFO [2022-01-10 12:53:03,245] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:03,246] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:03,247] jepsen worker 69 - jepsen.util 69 :ok :read [23 4] INFO [2022-01-10 12:53:03,247] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:03,247] jepsen worker 23 - jepsen.util 23 :ok :read [7 45] INFO [2022-01-10 12:53:03,248] jepsen worker 43 - jepsen.util 43 :ok :write [14 21] INFO [2022-01-10 12:53:03,248] jepsen worker 69 - jepsen.util 69 :ok :read [23 4] INFO [2022-01-10 12:53:03,249] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:03,249] jepsen worker 43 - jepsen.util 43 :ok :read [14 21] INFO [2022-01-10 12:53:03,250] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:03,250] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:03,251] jepsen worker 23 - jepsen.util 23 :ok :read [7 45] INFO [2022-01-10 12:53:03,251] jepsen worker 43 - jepsen.util 43 :ok :read [14 21] INFO [2022-01-10 12:53:03,252] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:03,252] jepsen worker 84 - jepsen.util 84 :ok :write [28 39] INFO [2022-01-10 12:53:03,252] jepsen worker 43 - jepsen.util 43 :invoke :write [14 28] INFO [2022-01-10 12:53:03,253] jepsen worker 69 - jepsen.util 69 :ok :read [23 4] INFO [2022-01-10 12:53:03,253] jepsen worker 84 - jepsen.util 84 :invoke :write [28 37] INFO [2022-01-10 12:53:03,254] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:03,255] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:03,255] jepsen worker 69 - jepsen.util 69 :ok :read [23 4] INFO [2022-01-10 12:53:03,256] jepsen worker 23 - jepsen.util 23 :ok :read [7 45] INFO [2022-01-10 12:53:03,256] jepsen worker 69 - jepsen.util 69 :invoke :write [23 33] INFO [2022-01-10 12:53:03,257] jepsen worker 23 - jepsen.util 23 :invoke :write [7 11] INFO [2022-01-10 12:53:03,259] jepsen worker 87 - jepsen.util 87 :ok :write [29 1] INFO [2022-01-10 12:53:03,259] jepsen worker 3 - jepsen.util 3 :ok :write [1 25] INFO [2022-01-10 12:53:03,260] jepsen worker 87 - jepsen.util 87 :invoke :write [29 6] INFO [2022-01-10 12:53:03,260] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:03,261] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:03,262] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:03,262] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:03,263] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:03,264] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:03,265] jepsen worker 3 - jepsen.util 3 :invoke :write [1 15] INFO [2022-01-10 12:53:03,265] jepsen worker 36 - jepsen.util 36 :ok :write [12 42] INFO [2022-01-10 12:53:03,265] jepsen worker 36 - jepsen.util 36 :invoke :write [12 3] INFO [2022-01-10 12:53:03,274] jepsen worker 0 - jepsen.util 0 :ok :write [0 21] INFO [2022-01-10 12:53:03,275] jepsen worker 0 - jepsen.util 0 :invoke :write [0 4] INFO [2022-01-10 12:53:03,280] jepsen worker 41 - jepsen.util 41 :ok :write [13 19] INFO [2022-01-10 12:53:03,280] jepsen worker 41 - jepsen.util 41 :invoke :write [13 26] INFO [2022-01-10 12:53:03,294] jepsen worker 84 - jepsen.util 84 :ok :write [28 37] INFO [2022-01-10 12:53:03,294] jepsen worker 84 - jepsen.util 84 :invoke :write [28 38] INFO [2022-01-10 12:53:03,295] jepsen worker 87 - jepsen.util 87 :ok :write [29 6] INFO [2022-01-10 12:53:03,296] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:03,297] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:03,297] jepsen worker 87 - jepsen.util 87 :invoke :write [29 7] INFO [2022-01-10 12:53:03,298] jepsen worker 3 - jepsen.util 3 :ok :write [1 15] INFO [2022-01-10 12:53:03,298] jepsen worker 3 - jepsen.util 3 :invoke :write [1 8] INFO [2022-01-10 12:53:03,303] jepsen worker 36 - jepsen.util 36 :ok :write [12 3] INFO [2022-01-10 12:53:03,303] jepsen worker 36 - jepsen.util 36 :invoke :write [12 40] INFO [2022-01-10 12:53:03,307] jepsen worker 0 - jepsen.util 0 :ok :write [0 4] INFO [2022-01-10 12:53:03,307] jepsen worker 0 - jepsen.util 0 :invoke :write [0 43] INFO [2022-01-10 12:53:03,309] jepsen worker 16 - jepsen.util 16 :ok :write [5 41] INFO [2022-01-10 12:53:03,309] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,309] jepsen worker 51 - jepsen.util 51 :ok :write [17 28] INFO [2022-01-10 12:53:03,310] jepsen worker 16 - jepsen.util 16 :ok :read [5 41] INFO [2022-01-10 12:53:03,310] jepsen worker 51 - jepsen.util 51 :invoke :write [17 36] INFO [2022-01-10 12:53:03,311] jepsen worker 16 - jepsen.util 16 :invoke :write [5 11] INFO [2022-01-10 12:53:03,314] jepsen worker 67 - jepsen.util 67 :ok :write [22 38] INFO [2022-01-10 12:53:03,315] jepsen worker 67 - jepsen.util 67 :invoke :write [22 28] INFO [2022-01-10 12:53:03,316] jepsen worker 78 - jepsen.util 78 :ok :write [26 23] INFO [2022-01-10 12:53:03,317] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:03,317] jepsen worker 78 - jepsen.util 78 :ok :read [26 23] INFO [2022-01-10 12:53:03,318] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:03,319] jepsen worker 78 - jepsen.util 78 :ok :read [26 23] INFO [2022-01-10 12:53:03,319] jepsen worker 7 - jepsen.util 7 :ok :write [2 38] INFO [2022-01-10 12:53:03,319] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:03,320] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:03,320] jepsen worker 78 - jepsen.util 78 :ok :read [26 23] INFO [2022-01-10 12:53:03,320] jepsen worker 7 - jepsen.util 7 :ok :read [2 38] INFO [2022-01-10 12:53:03,321] jepsen worker 13 - jepsen.util 13 :ok :write [4 15] INFO [2022-01-10 12:53:03,321] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:03,321] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:03,322] jepsen worker 78 - jepsen.util 78 :ok :read [26 23] INFO [2022-01-10 12:53:03,322] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:03,322] jepsen worker 13 - jepsen.util 13 :ok :read [4 15] INFO [2022-01-10 12:53:03,322] jepsen worker 78 - jepsen.util 78 :invoke :write [26 4] INFO [2022-01-10 12:53:03,323] jepsen worker 7 - jepsen.util 7 :ok :read [2 38] INFO [2022-01-10 12:53:03,323] jepsen worker 13 - jepsen.util 13 :invoke :write [4 36] INFO [2022-01-10 12:53:03,323] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:03,325] jepsen worker 7 - jepsen.util 7 :ok :read [2 38] INFO [2022-01-10 12:53:03,325] jepsen worker 7 - jepsen.util 7 :invoke :write [2 5] INFO [2022-01-10 12:53:03,333] jepsen worker 69 - jepsen.util 69 :ok :write [23 33] INFO [2022-01-10 12:53:03,333] jepsen worker 50 - jepsen.util 50 :ok :write [16 22] INFO [2022-01-10 12:53:03,334] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:03,334] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,335] jepsen worker 69 - jepsen.util 69 :ok :read [23 33] INFO [2022-01-10 12:53:03,335] jepsen worker 50 - jepsen.util 50 :ok :read [16 22] INFO [2022-01-10 12:53:03,335] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:03,336] jepsen worker 50 - jepsen.util 50 :invoke :write [16 10] INFO [2022-01-10 12:53:03,336] jepsen worker 69 - jepsen.util 69 :ok :read [23 33] INFO [2022-01-10 12:53:03,336] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:03,337] jepsen worker 69 - jepsen.util 69 :ok :read [23 33] INFO [2022-01-10 12:53:03,338] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:03,339] jepsen worker 69 - jepsen.util 69 :ok :read [23 33] INFO [2022-01-10 12:53:03,339] jepsen worker 81 - jepsen.util 81 :ok :write [27 37] INFO [2022-01-10 12:53:03,339] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:03,340] jepsen worker 81 - jepsen.util 81 :invoke :write [27 12] INFO [2022-01-10 12:53:03,340] jepsen worker 69 - jepsen.util 69 :ok :read [23 33] INFO [2022-01-10 12:53:03,340] jepsen worker 69 - jepsen.util 69 :invoke :write [23 41] INFO [2022-01-10 12:53:03,345] jepsen worker 34 - jepsen.util 34 :ok :write [11 47] INFO [2022-01-10 12:53:03,345] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,346] jepsen worker 34 - jepsen.util 34 :ok :read [11 47] INFO [2022-01-10 12:53:03,347] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,348] jepsen worker 34 - jepsen.util 34 :ok :read [11 47] INFO [2022-01-10 12:53:03,348] jepsen worker 34 - jepsen.util 34 :invoke :write [11 12] INFO [2022-01-10 12:53:03,353] jepsen worker 76 - jepsen.util 76 :ok :write [25 40] INFO [2022-01-10 12:53:03,353] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:03,354] jepsen worker 76 - jepsen.util 76 :ok :read [25 40] INFO [2022-01-10 12:53:03,354] jepsen worker 76 - jepsen.util 76 :invoke :write [25 11] INFO [2022-01-10 12:53:03,358] jepsen worker 60 - jepsen.util 60 :ok :write [20 30] INFO [2022-01-10 12:53:03,358] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,359] jepsen worker 60 - jepsen.util 60 :ok :read [20 30] INFO [2022-01-10 12:53:03,359] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,360] jepsen worker 60 - jepsen.util 60 :ok :read [20 30] INFO [2022-01-10 12:53:03,361] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,361] jepsen worker 60 - jepsen.util 60 :ok :read [20 30] INFO [2022-01-10 12:53:03,362] jepsen worker 60 - jepsen.util 60 :invoke :write [20 4] INFO [2022-01-10 12:53:03,362] jepsen worker 23 - jepsen.util 23 :ok :write [7 11] INFO [2022-01-10 12:53:03,363] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:03,364] jepsen worker 23 - jepsen.util 23 :ok :read [7 11] INFO [2022-01-10 12:53:03,364] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:03,365] jepsen worker 23 - jepsen.util 23 :ok :read [7 11] INFO [2022-01-10 12:53:03,366] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:03,366] jepsen worker 23 - jepsen.util 23 :ok :read [7 11] INFO [2022-01-10 12:53:03,367] jepsen worker 23 - jepsen.util 23 :invoke :write [7 37] INFO [2022-01-10 12:53:03,367] jepsen worker 43 - jepsen.util 43 :ok :write [14 28] INFO [2022-01-10 12:53:03,368] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:03,369] jepsen worker 43 - jepsen.util 43 :ok :read [14 28] INFO [2022-01-10 12:53:03,369] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:03,370] jepsen worker 43 - jepsen.util 43 :ok :read [14 28] INFO [2022-01-10 12:53:03,370] jepsen worker 43 - jepsen.util 43 :invoke :write [14 39] INFO [2022-01-10 12:53:03,374] jepsen worker 16 - jepsen.util 16 :ok :write [5 11] INFO [2022-01-10 12:53:03,375] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,376] jepsen worker 16 - jepsen.util 16 :ok :read [5 11] INFO [2022-01-10 12:53:03,376] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,377] jepsen worker 16 - jepsen.util 16 :ok :read [5 11] INFO [2022-01-10 12:53:03,378] jepsen worker 16 - jepsen.util 16 :invoke :write [5 21] INFO [2022-01-10 12:53:03,378] jepsen worker 67 - jepsen.util 67 :ok :write [22 28] INFO [2022-01-10 12:53:03,379] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,380] jepsen worker 67 - jepsen.util 67 :ok :read [22 28] INFO [2022-01-10 12:53:03,381] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,381] jepsen worker 7 - jepsen.util 7 :ok :write [2 5] INFO [2022-01-10 12:53:03,382] jepsen worker 67 - jepsen.util 67 :ok :read [22 28] INFO [2022-01-10 12:53:03,382] jepsen worker 7 - jepsen.util 7 :invoke :write [2 45] INFO [2022-01-10 12:53:03,383] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,384] jepsen worker 67 - jepsen.util 67 :ok :read [22 28] INFO [2022-01-10 12:53:03,384] jepsen worker 67 - jepsen.util 67 :invoke :write [22 31] INFO [2022-01-10 12:53:03,388] jepsen worker 50 - jepsen.util 50 :ok :write [16 10] INFO [2022-01-10 12:53:03,388] jepsen worker 50 - jepsen.util 50 :invoke :write [16 49] INFO [2022-01-10 12:53:03,392] jepsen worker 81 - jepsen.util 81 :ok :write [27 12] INFO [2022-01-10 12:53:03,392] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,393] jepsen worker 81 - jepsen.util 81 :ok :read [27 12] INFO [2022-01-10 12:53:03,394] jepsen worker 81 - jepsen.util 81 :invoke :write [27 6] INFO [2022-01-10 12:53:03,395] jepsen worker 34 - jepsen.util 34 :ok :write [11 12] INFO [2022-01-10 12:53:03,395] jepsen worker 34 - jepsen.util 34 :invoke :write [11 24] INFO [2022-01-10 12:53:03,413] jepsen worker 16 - jepsen.util 16 :ok :write [5 21] INFO [2022-01-10 12:53:03,413] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,414] jepsen worker 16 - jepsen.util 16 :ok :read [5 21] INFO [2022-01-10 12:53:03,415] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,416] jepsen worker 16 - jepsen.util 16 :ok :read [5 21] INFO [2022-01-10 12:53:03,416] jepsen worker 16 - jepsen.util 16 :invoke :write [5 42] INFO [2022-01-10 12:53:03,417] jepsen worker 67 - jepsen.util 67 :ok :write [22 31] INFO [2022-01-10 12:53:03,418] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,419] jepsen worker 67 - jepsen.util 67 :ok :read [22 31] INFO [2022-01-10 12:53:03,420] jepsen worker 67 - jepsen.util 67 :invoke :write [22 10] INFO [2022-01-10 12:53:03,421] jepsen worker 7 - jepsen.util 7 :ok :write [2 45] INFO [2022-01-10 12:53:03,421] jepsen worker 7 - jepsen.util 7 :invoke :write [2 32] INFO [2022-01-10 12:53:03,426] jepsen worker 50 - jepsen.util 50 :ok :write [16 49] INFO [2022-01-10 12:53:03,426] jepsen worker 50 - jepsen.util 50 :invoke :write [16 8] INFO [2022-01-10 12:53:03,429] jepsen worker 81 - jepsen.util 81 :ok :write [27 6] INFO [2022-01-10 12:53:03,430] jepsen worker 81 - jepsen.util 81 :invoke :write [27 19] INFO [2022-01-10 12:53:03,433] jepsen worker 34 - jepsen.util 34 :ok :write [11 24] INFO [2022-01-10 12:53:03,433] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,434] jepsen worker 34 - jepsen.util 34 :ok :read [11 24] INFO [2022-01-10 12:53:03,435] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,436] jepsen worker 34 - jepsen.util 34 :ok :read [11 24] INFO [2022-01-10 12:53:03,437] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,438] jepsen worker 34 - jepsen.util 34 :ok :read [11 24] INFO [2022-01-10 12:53:03,438] jepsen worker 16 - jepsen.util 16 :ok :write [5 42] INFO [2022-01-10 12:53:03,439] jepsen worker 34 - jepsen.util 34 :invoke :write [11 31] INFO [2022-01-10 12:53:03,439] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,440] jepsen worker 16 - jepsen.util 16 :ok :read [5 42] INFO [2022-01-10 12:53:03,441] jepsen worker 16 - jepsen.util 16 :invoke :write [5 6] INFO [2022-01-10 12:53:03,443] jepsen worker 67 - jepsen.util 67 :ok :write [22 10] INFO [2022-01-10 12:53:03,443] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,444] jepsen worker 67 - jepsen.util 67 :ok :read [22 10] INFO [2022-01-10 12:53:03,444] jepsen worker 84 - jepsen.util 84 :ok :write [28 38] INFO [2022-01-10 12:53:03,445] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,445] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:03,446] jepsen worker 67 - jepsen.util 67 :ok :read [22 10] INFO [2022-01-10 12:53:03,446] jepsen worker 41 - jepsen.util 41 :ok :write [13 26] INFO [2022-01-10 12:53:03,446] jepsen worker 84 - jepsen.util 84 :ok :read [28 38] INFO [2022-01-10 12:53:03,447] jepsen worker 67 - jepsen.util 67 :invoke :write [22 25] INFO [2022-01-10 12:53:03,447] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:03,448] jepsen worker 41 - jepsen.util 41 :invoke :write [13 7] INFO [2022-01-10 12:53:03,448] jepsen worker 7 - jepsen.util 7 :ok :write [2 32] INFO [2022-01-10 12:53:03,448] jepsen worker 84 - jepsen.util 84 :ok :read [28 38] INFO [2022-01-10 12:53:03,449] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:03,449] jepsen worker 3 - jepsen.util 3 :ok :write [1 8] INFO [2022-01-10 12:53:03,449] jepsen worker 84 - jepsen.util 84 :invoke :write [28 12] INFO [2022-01-10 12:53:03,450] jepsen worker 7 - jepsen.util 7 :ok :read [2 32] INFO [2022-01-10 12:53:03,450] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:03,450] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:03,451] jepsen worker 7 - jepsen.util 7 :ok :read [2 32] INFO [2022-01-10 12:53:03,451] jepsen worker 3 - jepsen.util 3 :ok :read [1 8] INFO [2022-01-10 12:53:03,452] jepsen worker 7 - jepsen.util 7 :invoke :write [2 24] INFO [2022-01-10 12:53:03,452] jepsen worker 3 - jepsen.util 3 :invoke :write [1 20] INFO [2022-01-10 12:53:03,454] jepsen worker 87 - jepsen.util 87 :ok :write [29 7] INFO [2022-01-10 12:53:03,454] jepsen worker 36 - jepsen.util 36 :ok :write [12 40] INFO [2022-01-10 12:53:03,455] jepsen worker 50 - jepsen.util 50 :ok :write [16 8] INFO [2022-01-10 12:53:03,455] jepsen worker 36 - jepsen.util 36 :invoke :write [12 49] INFO [2022-01-10 12:53:03,456] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:03,456] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,457] jepsen worker 87 - jepsen.util 87 :ok :read [29 7] INFO [2022-01-10 12:53:03,457] jepsen worker 50 - jepsen.util 50 :ok :read [16 8] INFO [2022-01-10 12:53:03,457] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:03,458] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,458] jepsen worker 87 - jepsen.util 87 :ok :read [29 7] INFO [2022-01-10 12:53:03,459] jepsen worker 50 - jepsen.util 50 :ok :read [16 8] INFO [2022-01-10 12:53:03,459] jepsen worker 87 - jepsen.util 87 :invoke :write [29 30] INFO [2022-01-10 12:53:03,460] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,461] jepsen worker 50 - jepsen.util 50 :ok :read [16 8] INFO [2022-01-10 12:53:03,461] jepsen worker 0 - jepsen.util 0 :ok :write [0 43] INFO [2022-01-10 12:53:03,461] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,462] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:03,462] jepsen worker 50 - jepsen.util 50 :ok :read [16 8] INFO [2022-01-10 12:53:03,463] jepsen worker 0 - jepsen.util 0 :ok :read [0 43] INFO [2022-01-10 12:53:03,463] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,463] jepsen worker 51 - jepsen.util 51 :ok :write [17 36] INFO [2022-01-10 12:53:03,463] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:03,464] jepsen worker 50 - jepsen.util 50 :ok :read [16 8] INFO [2022-01-10 12:53:03,464] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:03,464] jepsen worker 0 - jepsen.util 0 :ok :read [0 43] INFO [2022-01-10 12:53:03,465] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,465] jepsen worker 51 - jepsen.util 51 :ok :read [17 36] INFO [2022-01-10 12:53:03,465] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:03,465] jepsen worker 50 - jepsen.util 50 :ok :read [16 8] INFO [2022-01-10 12:53:03,466] jepsen worker 76 - jepsen.util 76 :ok :write [25 11] INFO [2022-01-10 12:53:03,466] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:03,466] jepsen worker 0 - jepsen.util 0 :ok :read [0 43] INFO [2022-01-10 12:53:03,467] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:03,467] jepsen worker 51 - jepsen.util 51 :ok :read [17 36] INFO [2022-01-10 12:53:03,467] jepsen worker 50 - jepsen.util 50 :invoke :write [16 28] INFO [2022-01-10 12:53:03,468] jepsen worker 0 - jepsen.util 0 :ok :read [0 43] INFO [2022-01-10 12:53:03,468] jepsen worker 51 - jepsen.util 51 :invoke :write [17 10] INFO [2022-01-10 12:53:03,468] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:03,469] jepsen worker 76 - jepsen.util 76 :invoke :write [25 11] INFO [2022-01-10 12:53:03,469] jepsen worker 0 - jepsen.util 0 :ok :read [0 43] INFO [2022-01-10 12:53:03,470] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:03,471] jepsen worker 0 - jepsen.util 0 :ok :read [0 43] INFO [2022-01-10 12:53:03,472] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:03,472] jepsen worker 78 - jepsen.util 78 :ok :write [26 4] INFO [2022-01-10 12:53:03,472] jepsen worker 60 - jepsen.util 60 :ok :write [20 4] INFO [2022-01-10 12:53:03,472] jepsen worker 0 - jepsen.util 0 :ok :read [0 43] INFO [2022-01-10 12:53:03,473] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,474] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:03,474] jepsen worker 60 - jepsen.util 60 :ok :read [20 4] INFO [2022-01-10 12:53:03,474] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:03,475] jepsen worker 78 - jepsen.util 78 :ok :read [26 4] INFO [2022-01-10 12:53:03,475] jepsen worker 60 - jepsen.util 60 :invoke :write [20 14] INFO [2022-01-10 12:53:03,475] jepsen worker 0 - jepsen.util 0 :ok :read [0 43] INFO [2022-01-10 12:53:03,475] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:03,476] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:03,476] jepsen worker 78 - jepsen.util 78 :ok :read [26 4] INFO [2022-01-10 12:53:03,477] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:03,477] jepsen worker 0 - jepsen.util 0 :ok :read [0 43] INFO [2022-01-10 12:53:03,478] jepsen worker 0 - jepsen.util 0 :invoke :write [0 19] INFO [2022-01-10 12:53:03,478] jepsen worker 78 - jepsen.util 78 :ok :read [26 4] INFO [2022-01-10 12:53:03,479] jepsen worker 78 - jepsen.util 78 :invoke :write [26 7] INFO [2022-01-10 12:53:03,483] jepsen worker 23 - jepsen.util 23 :ok :write [7 37] INFO [2022-01-10 12:53:03,483] jepsen worker 13 - jepsen.util 13 :ok :write [4 36] INFO [2022-01-10 12:53:03,483] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:03,484] jepsen worker 13 - jepsen.util 13 :invoke :write [4 35] INFO [2022-01-10 12:53:03,484] jepsen worker 23 - jepsen.util 23 :ok :read [7 37] INFO [2022-01-10 12:53:03,485] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:03,486] jepsen worker 23 - jepsen.util 23 :ok :read [7 37] INFO [2022-01-10 12:53:03,486] jepsen worker 23 - jepsen.util 23 :invoke :write [7 35] INFO [2022-01-10 12:53:03,552] jepsen worker 81 - jepsen.util 81 :ok :write [27 19] INFO [2022-01-10 12:53:03,553] jepsen worker 81 - jepsen.util 81 :invoke :write [27 20] INFO [2022-01-10 12:53:03,553] jepsen worker 43 - jepsen.util 43 :ok :write [14 39] INFO [2022-01-10 12:53:03,554] jepsen worker 43 - jepsen.util 43 :invoke :write [14 8] INFO [2022-01-10 12:53:03,555] jepsen worker 69 - jepsen.util 69 :ok :write [23 41] INFO [2022-01-10 12:53:03,556] jepsen worker 69 - jepsen.util 69 :invoke :write [23 9] INFO [2022-01-10 12:53:03,560] jepsen worker 34 - jepsen.util 34 :ok :write [11 31] INFO [2022-01-10 12:53:03,561] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,561] jepsen worker 34 - jepsen.util 34 :ok :read [11 31] INFO [2022-01-10 12:53:03,562] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,563] jepsen worker 34 - jepsen.util 34 :ok :read [11 31] INFO [2022-01-10 12:53:03,564] jepsen worker 34 - jepsen.util 34 :invoke :write [11 38] INFO [2022-01-10 12:53:03,566] jepsen worker 3 - jepsen.util 3 :ok :write [1 20] INFO [2022-01-10 12:53:03,566] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:03,568] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:53:03,568] jepsen worker 16 - jepsen.util 16 :ok :write [5 6] INFO [2022-01-10 12:53:03,568] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:03,569] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,569] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:53:03,569] jepsen worker 84 - jepsen.util 84 :ok :write [28 12] INFO [2022-01-10 12:53:03,570] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:03,570] jepsen worker 16 - jepsen.util 16 :ok :read [5 6] INFO [2022-01-10 12:53:03,570] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:03,570] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:53:03,571] jepsen worker 16 - jepsen.util 16 :invoke :write [5 14] INFO [2022-01-10 12:53:03,571] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:03,571] jepsen worker 84 - jepsen.util 84 :ok :read [28 12] INFO [2022-01-10 12:53:03,571] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:03,572] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:53:03,572] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:03,572] jepsen worker 84 - jepsen.util 84 :ok :read [28 12] INFO [2022-01-10 12:53:03,573] jepsen worker 84 - jepsen.util 84 :invoke :write [28 22] INFO [2022-01-10 12:53:03,573] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:53:03,574] jepsen worker 3 - jepsen.util 3 :invoke :write [1 38] INFO [2022-01-10 12:53:03,576] jepsen worker 76 - jepsen.util 76 :ok :write [25 11] INFO [2022-01-10 12:53:03,576] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:03,577] jepsen worker 76 - jepsen.util 76 :ok :read [25 11] INFO [2022-01-10 12:53:03,578] jepsen worker 51 - jepsen.util 51 :ok :write [17 10] INFO [2022-01-10 12:53:03,578] jepsen worker 66 - jepsen.util 66 :ok :write [22 16] INFO [2022-01-10 12:53:03,578] jepsen worker 76 - jepsen.util 76 :invoke :write [25 36] INFO [2022-01-10 12:53:03,579] jepsen worker 36 - jepsen.util 36 :ok :write [12 49] INFO [2022-01-10 12:53:03,579] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:03,579] jepsen worker 36 - jepsen.util 36 :invoke :write [12 10] INFO [2022-01-10 12:53:03,580] jepsen worker 51 - jepsen.util 51 :ok :read [17 10] INFO [2022-01-10 12:53:03,580] jepsen worker 66 - jepsen.util 66 :invoke :write [22 22] INFO [2022-01-10 12:53:03,581] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:03,581] jepsen worker 51 - jepsen.util 51 :ok :read [17 10] INFO [2022-01-10 12:53:03,582] jepsen worker 61 - jepsen.util 61 :ok :write [20 19] INFO [2022-01-10 12:53:03,582] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:03,583] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,583] jepsen worker 51 - jepsen.util 51 :ok :read [17 10] INFO [2022-01-10 12:53:03,583] jepsen worker 51 - jepsen.util 51 :invoke :write [17 43] INFO [2022-01-10 12:53:03,584] jepsen worker 61 - jepsen.util 61 :ok :read [20 19] INFO [2022-01-10 12:53:03,584] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,585] jepsen worker 41 - jepsen.util 41 :ok :write [13 7] INFO [2022-01-10 12:53:03,585] jepsen worker 61 - jepsen.util 61 :ok :read [20 19] INFO [2022-01-10 12:53:03,586] jepsen worker 41 - jepsen.util 41 :invoke :write [13 43] INFO [2022-01-10 12:53:03,587] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,587] jepsen worker 61 - jepsen.util 61 :ok :read [20 19] INFO [2022-01-10 12:53:03,588] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,588] jepsen worker 78 - jepsen.util 78 :ok :write [26 7] INFO [2022-01-10 12:53:03,589] jepsen worker 78 - jepsen.util 78 :invoke :write [26 2] INFO [2022-01-10 12:53:03,589] jepsen worker 61 - jepsen.util 61 :ok :read [20 19] INFO [2022-01-10 12:53:03,590] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,590] jepsen worker 23 - jepsen.util 23 :ok :write [7 35] INFO [2022-01-10 12:53:03,591] jepsen worker 61 - jepsen.util 61 :ok :read [20 19] INFO [2022-01-10 12:53:03,591] jepsen worker 23 - jepsen.util 23 :invoke :write [7 13] INFO [2022-01-10 12:53:03,592] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,593] jepsen worker 13 - jepsen.util 13 :ok :write [4 35] INFO [2022-01-10 12:53:03,594] jepsen worker 7 - jepsen.util 7 :ok :write [2 24] INFO [2022-01-10 12:53:03,594] jepsen worker 61 - jepsen.util 61 :ok :read [20 19] INFO [2022-01-10 12:53:03,594] jepsen worker 13 - jepsen.util 13 :invoke :write [4 2] INFO [2022-01-10 12:53:03,595] jepsen worker 43 - jepsen.util 43 :ok :write [14 8] INFO [2022-01-10 12:53:03,595] jepsen worker 61 - jepsen.util 61 :invoke :write [20 27] INFO [2022-01-10 12:53:03,595] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:03,596] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:03,596] jepsen worker 43 - jepsen.util 43 :ok :read [14 8] INFO [2022-01-10 12:53:03,597] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:03,597] jepsen worker 7 - jepsen.util 7 :ok :read [2 24] INFO [2022-01-10 12:53:03,597] jepsen worker 50 - jepsen.util 50 :ok :write [16 28] INFO [2022-01-10 12:53:03,597] jepsen worker 7 - jepsen.util 7 :invoke :write [2 6] INFO [2022-01-10 12:53:03,598] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,598] jepsen worker 43 - jepsen.util 43 :ok :read [14 8] INFO [2022-01-10 12:53:03,598] jepsen worker 43 - jepsen.util 43 :invoke :write [14 46] INFO [2022-01-10 12:53:03,599] jepsen worker 50 - jepsen.util 50 :ok :read [16 28] INFO [2022-01-10 12:53:03,599] jepsen worker 50 - jepsen.util 50 :invoke :write [16 15] INFO [2022-01-10 12:53:03,600] jepsen worker 76 - jepsen.util 76 :ok :write [25 36] INFO [2022-01-10 12:53:03,600] jepsen worker 76 - jepsen.util 76 :invoke :write [25 4] INFO [2022-01-10 12:53:03,600] jepsen worker 69 - jepsen.util 69 :ok :write [23 9] INFO [2022-01-10 12:53:03,601] jepsen worker 69 - jepsen.util 69 :invoke :write [23 1] INFO [2022-01-10 12:53:03,604] jepsen worker 87 - jepsen.util 87 :ok :write [29 30] INFO [2022-01-10 12:53:03,605] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:03,606] jepsen worker 87 - jepsen.util 87 :ok :read [29 30] INFO [2022-01-10 12:53:03,607] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:03,607] jepsen worker 87 - jepsen.util 87 :ok :read [29 30] INFO [2022-01-10 12:53:03,608] jepsen worker 60 - jepsen.util 60 :ok :write [20 14] INFO [2022-01-10 12:53:03,608] jepsen worker 0 - jepsen.util 0 :ok :write [0 19] INFO [2022-01-10 12:53:03,609] jepsen worker 0 - jepsen.util 0 :invoke :write [0 10] INFO [2022-01-10 12:53:03,609] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,610] jepsen worker 87 - jepsen.util 87 :invoke :write [29 22] INFO [2022-01-10 12:53:03,611] jepsen worker 60 - jepsen.util 60 :ok :read [20 14] INFO [2022-01-10 12:53:03,611] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,612] jepsen worker 81 - jepsen.util 81 :ok :write [27 20] INFO [2022-01-10 12:53:03,612] jepsen worker 60 - jepsen.util 60 :ok :read [20 14] INFO [2022-01-10 12:53:03,612] jepsen worker 81 - jepsen.util 81 :invoke :write [27 48] INFO [2022-01-10 12:53:03,613] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:03,614] jepsen worker 60 - jepsen.util 60 :ok :read [20 14] INFO [2022-01-10 12:53:03,615] jepsen worker 60 - jepsen.util 60 :invoke :write [20 33] INFO [2022-01-10 12:53:03,617] jepsen worker 34 - jepsen.util 34 :ok :write [11 38] INFO [2022-01-10 12:53:03,618] jepsen worker 34 - jepsen.util 34 :invoke :write [11 44] INFO [2022-01-10 12:53:03,620] jepsen worker 16 - jepsen.util 16 :ok :write [5 14] INFO [2022-01-10 12:53:03,620] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,621] jepsen worker 16 - jepsen.util 16 :ok :read [5 14] INFO [2022-01-10 12:53:03,622] jepsen worker 16 - jepsen.util 16 :invoke :write [5 18] INFO [2022-01-10 12:53:03,624] jepsen worker 66 - jepsen.util 66 :ok :write [22 22] INFO [2022-01-10 12:53:03,625] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,625] jepsen worker 66 - jepsen.util 66 :ok :read [22 22] INFO [2022-01-10 12:53:03,626] jepsen worker 66 - jepsen.util 66 :invoke :write [22 47] INFO [2022-01-10 12:53:03,631] jepsen worker 7 - jepsen.util 7 :ok :write [2 6] INFO [2022-01-10 12:53:03,632] jepsen worker 7 - jepsen.util 7 :invoke :write [2 36] INFO [2022-01-10 12:53:03,741] jepsen worker 50 - jepsen.util 50 :ok :write [16 15] INFO [2022-01-10 12:53:03,742] jepsen worker 50 - jepsen.util 50 :invoke :write [16 24] INFO [2022-01-10 12:53:03,752] jepsen worker 81 - jepsen.util 81 :ok :write [27 48] INFO [2022-01-10 12:53:03,752] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,753] jepsen worker 81 - jepsen.util 81 :ok :read [27 48] INFO [2022-01-10 12:53:03,754] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,755] jepsen worker 81 - jepsen.util 81 :ok :read [27 48] INFO [2022-01-10 12:53:03,755] jepsen worker 81 - jepsen.util 81 :invoke :write [27 15] INFO [2022-01-10 12:53:03,756] jepsen worker 34 - jepsen.util 34 :ok :write [11 44] INFO [2022-01-10 12:53:03,756] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,757] jepsen worker 34 - jepsen.util 34 :ok :read [11 44] INFO [2022-01-10 12:53:03,758] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,759] jepsen worker 34 - jepsen.util 34 :ok :read [11 44] INFO [2022-01-10 12:53:03,760] jepsen worker 34 - jepsen.util 34 :invoke :write [11 19] INFO [2022-01-10 12:53:03,761] jepsen worker 16 - jepsen.util 16 :ok :write [5 18] INFO [2022-01-10 12:53:03,762] jepsen worker 16 - jepsen.util 16 :invoke :write [5 22] INFO [2022-01-10 12:53:03,765] jepsen worker 66 - jepsen.util 66 :ok :write [22 47] INFO [2022-01-10 12:53:03,766] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,767] jepsen worker 66 - jepsen.util 66 :ok :read [22 47] INFO [2022-01-10 12:53:03,767] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,768] jepsen worker 66 - jepsen.util 66 :ok :read [22 47] INFO [2022-01-10 12:53:03,769] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,770] jepsen worker 66 - jepsen.util 66 :ok :read [22 47] INFO [2022-01-10 12:53:03,770] jepsen worker 7 - jepsen.util 7 :ok :write [2 36] INFO [2022-01-10 12:53:03,770] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,771] jepsen worker 7 - jepsen.util 7 :invoke :write [2 25] INFO [2022-01-10 12:53:03,771] jepsen worker 66 - jepsen.util 66 :ok :read [22 47] INFO [2022-01-10 12:53:03,772] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,772] jepsen worker 66 - jepsen.util 66 :ok :read [22 47] INFO [2022-01-10 12:53:03,773] jepsen worker 66 - jepsen.util 66 :invoke :write [22 28] INFO [2022-01-10 12:53:03,775] jepsen worker 50 - jepsen.util 50 :ok :write [16 24] INFO [2022-01-10 12:53:03,776] jepsen worker 50 - jepsen.util 50 :invoke :write [16 39] INFO [2022-01-10 12:53:03,779] jepsen worker 81 - jepsen.util 81 :ok :write [27 15] INFO [2022-01-10 12:53:03,779] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,781] jepsen worker 81 - jepsen.util 81 :ok :read [27 15] INFO [2022-01-10 12:53:03,781] jepsen worker 81 - jepsen.util 81 :invoke :write [27 49] INFO [2022-01-10 12:53:03,782] jepsen worker 34 - jepsen.util 34 :ok :write [11 19] INFO [2022-01-10 12:53:03,783] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,784] jepsen worker 34 - jepsen.util 34 :ok :read [11 19] INFO [2022-01-10 12:53:03,784] jepsen worker 34 - jepsen.util 34 :invoke :write [11 0] INFO [2022-01-10 12:53:03,788] jepsen worker 16 - jepsen.util 16 :ok :write [5 22] INFO [2022-01-10 12:53:03,788] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,789] jepsen worker 16 - jepsen.util 16 :ok :read [5 22] INFO [2022-01-10 12:53:03,790] jepsen worker 16 - jepsen.util 16 :invoke :write [5 29] INFO [2022-01-10 12:53:03,792] jepsen worker 68 - jepsen.util 68 :ok :write [22 49] INFO [2022-01-10 12:53:03,793] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,793] jepsen worker 68 - jepsen.util 68 :ok :read [22 49] INFO [2022-01-10 12:53:03,794] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,795] jepsen worker 68 - jepsen.util 68 :ok :read [22 49] INFO [2022-01-10 12:53:03,795] jepsen worker 7 - jepsen.util 7 :ok :write [2 25] INFO [2022-01-10 12:53:03,796] jepsen worker 68 - jepsen.util 68 :invoke :write [22 6] INFO [2022-01-10 12:53:03,796] jepsen worker 7 - jepsen.util 7 :invoke :write [2 32] INFO [2022-01-10 12:53:03,799] jepsen worker 50 - jepsen.util 50 :ok :write [16 39] INFO [2022-01-10 12:53:03,800] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,801] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:03,802] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,802] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:03,803] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,804] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:03,804] jepsen worker 81 - jepsen.util 81 :ok :write [27 49] INFO [2022-01-10 12:53:03,804] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,805] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,805] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:03,805] jepsen worker 81 - jepsen.util 81 :ok :read [27 49] INFO [2022-01-10 12:53:03,806] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:03,806] jepsen worker 81 - jepsen.util 81 :invoke :write [27 8] INFO [2022-01-10 12:53:03,807] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:03,807] jepsen worker 50 - jepsen.util 50 :invoke :write [16 10] INFO [2022-01-10 12:53:03,809] jepsen worker 34 - jepsen.util 34 :ok :write [11 0] INFO [2022-01-10 12:53:03,810] jepsen worker 34 - jepsen.util 34 :invoke :write [11 47] INFO [2022-01-10 12:53:03,815] jepsen worker 16 - jepsen.util 16 :ok :write [5 29] INFO [2022-01-10 12:53:03,815] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,816] jepsen worker 16 - jepsen.util 16 :ok :read [5 29] INFO [2022-01-10 12:53:03,817] jepsen worker 16 - jepsen.util 16 :invoke :write [5 16] INFO [2022-01-10 12:53:03,819] jepsen worker 68 - jepsen.util 68 :ok :write [22 6] INFO [2022-01-10 12:53:03,819] jepsen worker 68 - jepsen.util 68 :invoke :write [22 20] INFO [2022-01-10 12:53:03,824] jepsen worker 7 - jepsen.util 7 :ok :write [2 32] INFO [2022-01-10 12:53:03,825] jepsen worker 7 - jepsen.util 7 :invoke :write [2 15] INFO [2022-01-10 12:53:03,830] jepsen worker 50 - jepsen.util 50 :ok :write [16 10] INFO [2022-01-10 12:53:03,830] jepsen worker 50 - jepsen.util 50 :invoke :write [16 18] INFO [2022-01-10 12:53:03,834] jepsen worker 81 - jepsen.util 81 :ok :write [27 8] INFO [2022-01-10 12:53:03,835] jepsen worker 81 - jepsen.util 81 :invoke :write [27 8] INFO [2022-01-10 12:53:03,839] jepsen worker 34 - jepsen.util 34 :ok :write [11 47] INFO [2022-01-10 12:53:03,839] jepsen worker 34 - jepsen.util 34 :invoke :write [11 12] INFO [2022-01-10 12:53:03,849] jepsen worker 68 - jepsen.util 68 :ok :write [22 20] INFO [2022-01-10 12:53:03,849] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,850] jepsen worker 16 - jepsen.util 16 :ok :write [5 16] INFO [2022-01-10 12:53:03,850] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,851] jepsen worker 68 - jepsen.util 68 :ok :read [22 20] INFO [2022-01-10 12:53:03,851] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,852] jepsen worker 7 - jepsen.util 7 :ok :write [2 15] INFO [2022-01-10 12:53:03,852] jepsen worker 16 - jepsen.util 16 :ok :read [5 16] INFO [2022-01-10 12:53:03,853] jepsen worker 7 - jepsen.util 7 :invoke :write [2 8] INFO [2022-01-10 12:53:03,853] jepsen worker 68 - jepsen.util 68 :ok :read [22 20] INFO [2022-01-10 12:53:03,853] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,854] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:03,855] jepsen worker 68 - jepsen.util 68 :ok :read [22 20] INFO [2022-01-10 12:53:03,855] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,855] jepsen worker 16 - jepsen.util 16 :ok :read [5 16] INFO [2022-01-10 12:53:03,856] jepsen worker 68 - jepsen.util 68 :ok :read [22 20] INFO [2022-01-10 12:53:03,856] jepsen worker 16 - jepsen.util 16 :invoke :write [5 16] INFO [2022-01-10 12:53:03,857] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,857] jepsen worker 68 - jepsen.util 68 :ok :read [22 20] INFO [2022-01-10 12:53:03,858] jepsen worker 68 - jepsen.util 68 :invoke :write [22 19] INFO [2022-01-10 12:53:03,859] jepsen worker 50 - jepsen.util 50 :ok :write [16 18] INFO [2022-01-10 12:53:03,859] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:53:03,862] jepsen worker 81 - jepsen.util 81 :ok :write [27 8] INFO [2022-01-10 12:53:03,863] jepsen worker 81 - jepsen.util 81 :invoke :write [27 49] INFO [2022-01-10 12:53:03,866] jepsen worker 34 - jepsen.util 34 :ok :write [11 12] INFO [2022-01-10 12:53:03,867] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,868] jepsen worker 34 - jepsen.util 34 :ok :read [11 12] INFO [2022-01-10 12:53:03,869] jepsen worker 34 - jepsen.util 34 :invoke :write [11 9] INFO [2022-01-10 12:53:03,876] jepsen worker 16 - jepsen.util 16 :ok :write [5 16] INFO [2022-01-10 12:53:03,876] jepsen worker 16 - jepsen.util 16 :invoke :write [5 2] INFO [2022-01-10 12:53:03,879] jepsen worker 7 - jepsen.util 7 :ok :write [2 8] INFO [2022-01-10 12:53:03,880] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:03,880] jepsen worker 7 - jepsen.util 7 :ok :read [2 8] INFO [2022-01-10 12:53:03,881] jepsen worker 68 - jepsen.util 68 :ok :write [22 19] INFO [2022-01-10 12:53:03,881] jepsen worker 7 - jepsen.util 7 :invoke :write [2 34] INFO [2022-01-10 12:53:03,882] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,883] jepsen worker 68 - jepsen.util 68 :ok :read [22 19] INFO [2022-01-10 12:53:03,884] jepsen worker 68 - jepsen.util 68 :invoke :write [22 16] INFO [2022-01-10 12:53:03,884] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:53:03,885] jepsen worker 50 - jepsen.util 50 :invoke :write [16 38] INFO [2022-01-10 12:53:03,888] jepsen worker 81 - jepsen.util 81 :ok :write [27 49] INFO [2022-01-10 12:53:03,889] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,890] jepsen worker 81 - jepsen.util 81 :ok :read [27 49] INFO [2022-01-10 12:53:03,890] jepsen worker 81 - jepsen.util 81 :invoke :write [27 36] INFO [2022-01-10 12:53:03,892] jepsen worker 34 - jepsen.util 34 :ok :write [11 9] INFO [2022-01-10 12:53:03,893] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,894] jepsen worker 34 - jepsen.util 34 :ok :read [11 9] INFO [2022-01-10 12:53:03,894] jepsen worker 34 - jepsen.util 34 :invoke :write [11 32] INFO [2022-01-10 12:53:03,898] jepsen worker 16 - jepsen.util 16 :ok :write [5 2] INFO [2022-01-10 12:53:03,899] jepsen worker 16 - jepsen.util 16 :invoke :write [5 17] INFO [2022-01-10 12:53:03,903] jepsen worker 7 - jepsen.util 7 :ok :write [2 34] INFO [2022-01-10 12:53:03,903] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:03,904] jepsen worker 7 - jepsen.util 7 :ok :read [2 34] INFO [2022-01-10 12:53:03,905] jepsen worker 7 - jepsen.util 7 :invoke :write [2 5] INFO [2022-01-10 12:53:03,908] jepsen worker 68 - jepsen.util 68 :ok :write [22 16] INFO [2022-01-10 12:53:03,909] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,910] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:03,910] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,911] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:03,912] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,912] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:03,913] jepsen worker 68 - jepsen.util 68 :invoke :write [22 18] INFO [2022-01-10 12:53:03,913] jepsen worker 50 - jepsen.util 50 :ok :write [16 38] INFO [2022-01-10 12:53:03,914] jepsen worker 50 - jepsen.util 50 :invoke :write [16 20] INFO [2022-01-10 12:53:03,916] jepsen worker 81 - jepsen.util 81 :ok :write [27 36] INFO [2022-01-10 12:53:03,916] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:03,917] jepsen worker 81 - jepsen.util 81 :ok :read [27 36] INFO [2022-01-10 12:53:03,918] jepsen worker 81 - jepsen.util 81 :invoke :write [27 15] INFO [2022-01-10 12:53:03,920] jepsen worker 34 - jepsen.util 34 :ok :write [11 32] INFO [2022-01-10 12:53:03,921] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:03,922] jepsen worker 34 - jepsen.util 34 :ok :read [11 32] INFO [2022-01-10 12:53:03,923] jepsen worker 34 - jepsen.util 34 :invoke :write [11 32] INFO [2022-01-10 12:53:03,926] jepsen worker 16 - jepsen.util 16 :ok :write [5 17] INFO [2022-01-10 12:53:03,926] jepsen worker 16 - jepsen.util 16 :invoke :write [5 39] INFO [2022-01-10 12:53:03,931] jepsen worker 7 - jepsen.util 7 :ok :write [2 5] INFO [2022-01-10 12:53:03,931] jepsen worker 7 - jepsen.util 7 :invoke :write [2 10] INFO [2022-01-10 12:53:03,937] jepsen worker 68 - jepsen.util 68 :ok :write [22 18] INFO [2022-01-10 12:53:03,938] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:03,939] jepsen worker 68 - jepsen.util 68 :ok :read [22 18] INFO [2022-01-10 12:53:03,939] jepsen worker 68 - jepsen.util 68 :invoke :write [22 1] INFO [2022-01-10 12:53:04,121] jepsen worker 50 - jepsen.util 50 :ok :write [16 20] INFO [2022-01-10 12:53:04,121] jepsen worker 50 - jepsen.util 50 :invoke :write [16 5] INFO [2022-01-10 12:53:04,127] jepsen worker 81 - jepsen.util 81 :ok :write [27 15] INFO [2022-01-10 12:53:04,127] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:04,128] jepsen worker 81 - jepsen.util 81 :ok :read [27 15] INFO [2022-01-10 12:53:04,128] jepsen worker 81 - jepsen.util 81 :invoke :write [27 14] INFO [2022-01-10 12:53:04,131] jepsen worker 34 - jepsen.util 34 :ok :write [11 32] INFO [2022-01-10 12:53:04,132] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:04,133] jepsen worker 34 - jepsen.util 34 :ok :read [11 32] INFO [2022-01-10 12:53:04,134] jepsen worker 34 - jepsen.util 34 :invoke :write [11 16] INFO [2022-01-10 12:53:04,135] jepsen worker 16 - jepsen.util 16 :ok :write [5 39] INFO [2022-01-10 12:53:04,136] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:04,137] jepsen worker 16 - jepsen.util 16 :ok :read [5 39] INFO [2022-01-10 12:53:04,137] jepsen worker 16 - jepsen.util 16 :invoke :write [5 10] INFO [2022-01-10 12:53:04,140] jepsen worker 7 - jepsen.util 7 :ok :write [2 10] INFO [2022-01-10 12:53:04,141] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,142] jepsen worker 7 - jepsen.util 7 :ok :read [2 10] INFO [2022-01-10 12:53:04,142] jepsen worker 7 - jepsen.util 7 :invoke :write [2 3] INFO [2022-01-10 12:53:04,145] jepsen worker 68 - jepsen.util 68 :ok :write [22 1] INFO [2022-01-10 12:53:04,146] jepsen worker 68 - jepsen.util 68 :invoke :write [22 47] INFO [2022-01-10 12:53:04,150] jepsen worker 50 - jepsen.util 50 :ok :write [16 5] INFO [2022-01-10 12:53:04,150] jepsen worker 50 - jepsen.util 50 :invoke :write [16 4] INFO [2022-01-10 12:53:04,154] jepsen worker 3 - jepsen.util 3 :ok :write [1 38] INFO [2022-01-10 12:53:04,155] jepsen worker 3 - jepsen.util 3 :invoke :write [1 14] INFO [2022-01-10 12:53:04,157] jepsen worker 81 - jepsen.util 81 :ok :write [27 14] INFO [2022-01-10 12:53:04,157] jepsen worker 81 - jepsen.util 81 :invoke :write [27 43] INFO [2022-01-10 12:53:04,160] jepsen worker 41 - jepsen.util 41 :ok :write [13 43] INFO [2022-01-10 12:53:04,160] jepsen worker 41 - jepsen.util 41 :invoke :write [13 2] WARN [2022-01-10 12:53:04,161] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":209397,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":228584,"total_server_duration_us":210429,"encode_duration_us":118,"operation_id":"0x1bc","timeout_ms":10000,"last_server_duration_us":208479,"total_duration_us":4389084},{"operation_name":"upsert","last_dispatch_duration_us":19684,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":33802,"total_server_duration_us":20469,"encode_duration_us":155,"operation_id":"0x20e","timeout_ms":10000,"last_server_duration_us":19030,"total_duration_us":4171981},{"operation_name":"upsert","last_dispatch_duration_us":203687,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":218173,"total_server_duration_us":204268,"encode_duration_us":139,"operation_id":"0x226","timeout_ms":10000,"last_server_duration_us":203167,"total_duration_us":3861706},{"operation_name":"upsert","last_dispatch_duration_us":102981,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":119806,"total_server_duration_us":103908,"encode_duration_us":186,"operation_id":"0x1b2","timeout_ms":10000,"last_server_duration_us":102169,"total_duration_us":3765784},{"operation_name":"upsert","last_dispatch_duration_us":44077,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":56752,"total_server_duration_us":44788,"encode_duration_us":165,"operation_id":"0x205","timeout_ms":10000,"last_server_duration_us":43508,"total_duration_us":3687793},{"operation_name":"upsert","last_dispatch_duration_us":37280,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000001/000000005D716E8E","last_local_socket":"172.23.104.79:34680","total_dispatch_duration_us":851114,"total_server_duration_us":38226,"encode_duration_us":1032,"operation_id":"0x1a1","timeout_ms":10000,"last_server_duration_us":36525,"total_duration_us":3465870},{"operation_name":"upsert","last_dispatch_duration_us":34577,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":47263,"total_server_duration_us":34542,"encode_duration_us":160,"operation_id":"0x21f","timeout_ms":10000,"last_server_duration_us":33328,"total_duration_us":3172409},{"operation_name":"upsert","last_dispatch_duration_us":91326,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000001/00000000100998FD","last_local_socket":"172.23.104.79:58632","total_dispatch_duration_us":693690,"total_server_duration_us":93154,"encode_duration_us":236,"operation_id":"0x1f5","timeout_ms":10000,"last_server_duration_us":90631,"total_duration_us":2808650},{"operation_name":"upsert","last_dispatch_duration_us":41613,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000001/000000005D716E8E","last_local_socket":"172.23.104.79:34680","total_dispatch_duration_us":693526,"total_server_duration_us":42178,"encode_duration_us":212,"operation_id":"0x218","timeout_ms":10000,"last_server_duration_us":40589,"total_duration_us":2790360},{"operation_name":"upsert","last_dispatch_duration_us":80690,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":92346,"total_server_duration_us":80924,"encode_duration_us":178,"operation_id":"0x232","timeout_ms":10000,"last_server_duration_us":79971,"total_duration_us":2692894}],"total_count":21}} INFO [2022-01-10 12:53:04,163] jepsen worker 34 - jepsen.util 34 :ok :write [11 16] INFO [2022-01-10 12:53:04,164] jepsen worker 34 - jepsen.util 34 :invoke :write [11 20] INFO [2022-01-10 12:53:04,168] jepsen worker 16 - jepsen.util 16 :ok :write [5 10] INFO [2022-01-10 12:53:04,169] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:04,170] jepsen worker 16 - jepsen.util 16 :ok :read [5 10] INFO [2022-01-10 12:53:04,170] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:04,171] jepsen worker 16 - jepsen.util 16 :ok :read [5 10] INFO [2022-01-10 12:53:04,172] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:04,173] jepsen worker 16 - jepsen.util 16 :ok :read [5 10] INFO [2022-01-10 12:53:04,173] jepsen worker 7 - jepsen.util 7 :ok :write [2 3] INFO [2022-01-10 12:53:04,174] jepsen worker 16 - jepsen.util 16 :invoke :write [5 28] INFO [2022-01-10 12:53:04,175] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,176] jepsen worker 7 - jepsen.util 7 :ok :read [2 3] INFO [2022-01-10 12:53:04,176] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,177] jepsen worker 7 - jepsen.util 7 :ok :read [2 3] INFO [2022-01-10 12:53:04,177] jepsen worker 68 - jepsen.util 68 :ok :write [22 47] INFO [2022-01-10 12:53:04,178] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,179] jepsen worker 7 - jepsen.util 7 :ok :read [2 3] INFO [2022-01-10 12:53:04,179] jepsen worker 68 - jepsen.util 68 :invoke :write [22 32] INFO [2022-01-10 12:53:04,180] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,181] jepsen worker 7 - jepsen.util 7 :ok :read [2 3] INFO [2022-01-10 12:53:04,181] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,182] jepsen worker 50 - jepsen.util 50 :ok :write [16 4] INFO [2022-01-10 12:53:04,182] jepsen worker 7 - jepsen.util 7 :ok :read [2 3] INFO [2022-01-10 12:53:04,183] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,184] jepsen worker 50 - jepsen.util 50 :invoke :write [16 24] INFO [2022-01-10 12:53:04,184] jepsen worker 7 - jepsen.util 7 :ok :read [2 3] INFO [2022-01-10 12:53:04,185] jepsen worker 7 - jepsen.util 7 :invoke :write [2 29] INFO [2022-01-10 12:53:04,188] jepsen worker 81 - jepsen.util 81 :ok :write [27 43] INFO [2022-01-10 12:53:04,189] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:04,190] jepsen worker 81 - jepsen.util 81 :ok :read [27 43] INFO [2022-01-10 12:53:04,190] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:04,191] jepsen worker 81 - jepsen.util 81 :ok :read [27 43] INFO [2022-01-10 12:53:04,192] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:04,192] jepsen worker 81 - jepsen.util 81 :ok :read [27 43] INFO [2022-01-10 12:53:04,193] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:04,194] jepsen worker 81 - jepsen.util 81 :ok :read [27 43] INFO [2022-01-10 12:53:04,194] jepsen worker 81 - jepsen.util 81 :invoke :write [27 31] INFO [2022-01-10 12:53:04,197] jepsen worker 34 - jepsen.util 34 :ok :write [11 20] INFO [2022-01-10 12:53:04,198] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:04,199] jepsen worker 34 - jepsen.util 34 :ok :read [11 20] INFO [2022-01-10 12:53:04,200] jepsen worker 34 - jepsen.util 34 :invoke :write [11 43] INFO [2022-01-10 12:53:04,201] jepsen worker 16 - jepsen.util 16 :ok :write [5 28] INFO [2022-01-10 12:53:04,202] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:04,203] jepsen worker 16 - jepsen.util 16 :ok :read [5 28] INFO [2022-01-10 12:53:04,203] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:04,204] jepsen worker 16 - jepsen.util 16 :ok :read [5 28] INFO [2022-01-10 12:53:04,205] jepsen worker 16 - jepsen.util 16 :invoke :read [5 nil] INFO [2022-01-10 12:53:04,205] jepsen worker 16 - jepsen.util 16 :ok :read [5 28] INFO [2022-01-10 12:53:04,206] jepsen worker 7 - jepsen.util 7 :ok :write [2 29] INFO [2022-01-10 12:53:04,206] jepsen worker 16 - jepsen.util 16 :invoke :write [5 26] INFO [2022-01-10 12:53:04,207] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,208] jepsen worker 7 - jepsen.util 7 :ok :read [2 29] INFO [2022-01-10 12:53:04,208] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,209] jepsen worker 7 - jepsen.util 7 :ok :read [2 29] INFO [2022-01-10 12:53:04,210] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,210] jepsen worker 68 - jepsen.util 68 :ok :write [22 32] INFO [2022-01-10 12:53:04,211] jepsen worker 7 - jepsen.util 7 :ok :read [2 29] INFO [2022-01-10 12:53:04,211] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:04,212] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:53:04,212] jepsen worker 7 - jepsen.util 7 :invoke :write [2 44] INFO [2022-01-10 12:53:04,213] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:04,213] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:53:04,214] jepsen worker 68 - jepsen.util 68 :invoke :write [22 44] INFO [2022-01-10 12:53:04,216] jepsen worker 50 - jepsen.util 50 :ok :write [16 24] INFO [2022-01-10 12:53:04,217] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:04,218] jepsen worker 50 - jepsen.util 50 :ok :read [16 24] INFO [2022-01-10 12:53:04,218] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:04,219] jepsen worker 50 - jepsen.util 50 :ok :read [16 24] INFO [2022-01-10 12:53:04,220] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:04,220] jepsen worker 50 - jepsen.util 50 :ok :read [16 24] INFO [2022-01-10 12:53:04,221] jepsen worker 81 - jepsen.util 81 :ok :write [27 31] INFO [2022-01-10 12:53:04,221] jepsen worker 50 - jepsen.util 50 :invoke :write [16 2] INFO [2022-01-10 12:53:04,222] jepsen worker 81 - jepsen.util 81 :invoke :write [27 2] INFO [2022-01-10 12:53:04,226] jepsen worker 34 - jepsen.util 34 :ok :write [11 43] INFO [2022-01-10 12:53:04,227] jepsen worker 34 - jepsen.util 34 :invoke :write [11 13] INFO [2022-01-10 12:53:04,304] jepsen worker 15 - jepsen.util 15 :ok :write [5 21] INFO [2022-01-10 12:53:04,304] jepsen worker 15 - jepsen.util 15 :invoke :write [5 26] INFO [2022-01-10 12:53:04,307] jepsen worker 7 - jepsen.util 7 :ok :write [2 44] INFO [2022-01-10 12:53:04,308] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,309] jepsen worker 7 - jepsen.util 7 :ok :read [2 44] INFO [2022-01-10 12:53:04,309] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,310] jepsen worker 7 - jepsen.util 7 :ok :read [2 44] INFO [2022-01-10 12:53:04,311] jepsen worker 7 - jepsen.util 7 :invoke :write [2 44] INFO [2022-01-10 12:53:04,312] jepsen worker 68 - jepsen.util 68 :ok :write [22 44] INFO [2022-01-10 12:53:04,313] jepsen worker 68 - jepsen.util 68 :invoke :write [22 26] INFO [2022-01-10 12:53:04,315] jepsen worker 50 - jepsen.util 50 :ok :write [16 2] INFO [2022-01-10 12:53:04,316] jepsen worker 50 - jepsen.util 50 :invoke :write [16 43] INFO [2022-01-10 12:53:04,320] jepsen worker 81 - jepsen.util 81 :ok :write [27 2] INFO [2022-01-10 12:53:04,320] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:04,321] jepsen worker 81 - jepsen.util 81 :ok :read [27 2] INFO [2022-01-10 12:53:04,322] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:04,322] jepsen worker 34 - jepsen.util 34 :ok :write [11 13] INFO [2022-01-10 12:53:04,323] jepsen worker 81 - jepsen.util 81 :ok :read [27 2] INFO [2022-01-10 12:53:04,323] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:04,324] jepsen worker 34 - jepsen.util 34 :ok :read [11 13] INFO [2022-01-10 12:53:04,324] jepsen worker 81 - jepsen.util 81 :invoke :write [27 36] INFO [2022-01-10 12:53:04,325] jepsen worker 34 - jepsen.util 34 :invoke :write [11 24] INFO [2022-01-10 12:53:04,327] jepsen worker 15 - jepsen.util 15 :ok :write [5 26] INFO [2022-01-10 12:53:04,328] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:04,329] jepsen worker 15 - jepsen.util 15 :ok :read [5 26] INFO [2022-01-10 12:53:04,330] jepsen worker 7 - jepsen.util 7 :ok :write [2 44] INFO [2022-01-10 12:53:04,330] jepsen worker 15 - jepsen.util 15 :invoke :write [5 10] INFO [2022-01-10 12:53:04,330] jepsen worker 7 - jepsen.util 7 :invoke :write [2 0] INFO [2022-01-10 12:53:04,335] jepsen worker 68 - jepsen.util 68 :ok :write [22 26] INFO [2022-01-10 12:53:04,336] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:04,337] jepsen worker 68 - jepsen.util 68 :ok :read [22 26] INFO [2022-01-10 12:53:04,338] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:04,339] jepsen worker 68 - jepsen.util 68 :ok :read [22 26] INFO [2022-01-10 12:53:04,339] jepsen worker 68 - jepsen.util 68 :invoke :write [22 2] INFO [2022-01-10 12:53:04,340] jepsen worker 50 - jepsen.util 50 :ok :write [16 43] INFO [2022-01-10 12:53:04,340] jepsen worker 50 - jepsen.util 50 :invoke :write [16 25] INFO [2022-01-10 12:53:04,348] jepsen worker 81 - jepsen.util 81 :ok :write [27 36] INFO [2022-01-10 12:53:04,349] jepsen worker 81 - jepsen.util 81 :invoke :write [27 0] INFO [2022-01-10 12:53:04,353] jepsen worker 34 - jepsen.util 34 :ok :write [11 24] INFO [2022-01-10 12:53:04,354] jepsen worker 34 - jepsen.util 34 :invoke :write [11 17] INFO [2022-01-10 12:53:04,355] jepsen worker 7 - jepsen.util 7 :ok :write [2 0] INFO [2022-01-10 12:53:04,355] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,356] jepsen worker 7 - jepsen.util 7 :ok :read [2 0] INFO [2022-01-10 12:53:04,357] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,357] jepsen worker 15 - jepsen.util 15 :ok :write [5 10] INFO [2022-01-10 12:53:04,357] jepsen worker 7 - jepsen.util 7 :ok :read [2 0] INFO [2022-01-10 12:53:04,357] jepsen worker 15 - jepsen.util 15 :invoke :write [5 46] INFO [2022-01-10 12:53:04,358] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,359] jepsen worker 7 - jepsen.util 7 :ok :read [2 0] INFO [2022-01-10 12:53:04,359] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,360] jepsen worker 7 - jepsen.util 7 :ok :read [2 0] INFO [2022-01-10 12:53:04,361] jepsen worker 7 - jepsen.util 7 :invoke :write [2 21] INFO [2022-01-10 12:53:04,362] jepsen worker 66 - jepsen.util 66 :ok :write [22 28] INFO [2022-01-10 12:53:04,363] jepsen worker 66 - jepsen.util 66 :invoke :write [22 19] INFO [2022-01-10 12:53:04,368] jepsen worker 50 - jepsen.util 50 :ok :write [16 25] INFO [2022-01-10 12:53:04,369] jepsen worker 50 - jepsen.util 50 :invoke :write [16 8] INFO [2022-01-10 12:53:04,374] jepsen worker 81 - jepsen.util 81 :ok :write [27 0] INFO [2022-01-10 12:53:04,375] jepsen worker 81 - jepsen.util 81 :invoke :write [27 8] INFO [2022-01-10 12:53:04,378] jepsen worker 34 - jepsen.util 34 :ok :write [11 17] INFO [2022-01-10 12:53:04,380] jepsen worker 34 - jepsen.util 34 :invoke :write [11 32] INFO [2022-01-10 12:53:04,382] jepsen worker 15 - jepsen.util 15 :ok :write [5 46] INFO [2022-01-10 12:53:04,383] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:04,384] jepsen worker 15 - jepsen.util 15 :ok :read [5 46] INFO [2022-01-10 12:53:04,384] jepsen worker 15 - jepsen.util 15 :invoke :write [5 4] INFO [2022-01-10 12:53:04,388] jepsen worker 7 - jepsen.util 7 :ok :write [2 21] INFO [2022-01-10 12:53:04,389] jepsen worker 7 - jepsen.util 7 :invoke :write [2 42] INFO [2022-01-10 12:53:04,392] jepsen worker 66 - jepsen.util 66 :ok :write [22 19] INFO [2022-01-10 12:53:04,393] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:04,394] jepsen worker 66 - jepsen.util 66 :ok :read [22 19] INFO [2022-01-10 12:53:04,395] jepsen worker 66 - jepsen.util 66 :invoke :write [22 33] INFO [2022-01-10 12:53:04,398] jepsen worker 50 - jepsen.util 50 :ok :write [16 8] INFO [2022-01-10 12:53:04,399] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:04,400] jepsen worker 50 - jepsen.util 50 :ok :read [16 8] INFO [2022-01-10 12:53:04,401] jepsen worker 50 - jepsen.util 50 :invoke :write [16 48] INFO [2022-01-10 12:53:04,405] jepsen worker 81 - jepsen.util 81 :ok :write [27 8] INFO [2022-01-10 12:53:04,406] jepsen worker 81 - jepsen.util 81 :invoke :write [27 42] INFO [2022-01-10 12:53:04,411] jepsen worker 16 - jepsen.util 16 :ok :write [5 26] INFO [2022-01-10 12:53:04,412] jepsen worker 16 - jepsen.util 16 :invoke :write [5 26] INFO [2022-01-10 12:53:04,417] jepsen worker 34 - jepsen.util 34 :ok :write [11 32] INFO [2022-01-10 12:53:04,418] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:04,419] jepsen worker 34 - jepsen.util 34 :ok :read [11 32] INFO [2022-01-10 12:53:04,419] jepsen worker 34 - jepsen.util 34 :invoke :write [11 7] INFO [2022-01-10 12:53:04,422] jepsen worker 7 - jepsen.util 7 :ok :write [2 42] INFO [2022-01-10 12:53:04,423] jepsen worker 7 - jepsen.util 7 :invoke :write [2 0] INFO [2022-01-10 12:53:04,426] jepsen worker 66 - jepsen.util 66 :ok :write [22 33] INFO [2022-01-10 12:53:04,427] jepsen worker 66 - jepsen.util 66 :invoke :write [22 6] INFO [2022-01-10 12:53:04,432] jepsen worker 50 - jepsen.util 50 :ok :write [16 48] INFO [2022-01-10 12:53:04,433] jepsen worker 50 - jepsen.util 50 :invoke :write [16 44] INFO [2022-01-10 12:53:04,441] jepsen worker 81 - jepsen.util 81 :ok :write [27 42] INFO [2022-01-10 12:53:04,442] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:04,443] jepsen worker 81 - jepsen.util 81 :ok :read [27 42] INFO [2022-01-10 12:53:04,443] jepsen worker 81 - jepsen.util 81 :invoke :write [27 48] INFO [2022-01-10 12:53:04,453] jepsen worker 15 - jepsen.util 15 :ok :write [5 4] INFO [2022-01-10 12:53:04,453] jepsen worker 15 - jepsen.util 15 :invoke :write [5 43] INFO [2022-01-10 12:53:04,457] jepsen worker 34 - jepsen.util 34 :ok :write [11 7] INFO [2022-01-10 12:53:04,458] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:04,459] jepsen worker 34 - jepsen.util 34 :ok :read [11 7] INFO [2022-01-10 12:53:04,459] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:04,460] jepsen worker 7 - jepsen.util 7 :ok :write [2 0] INFO [2022-01-10 12:53:04,460] jepsen worker 34 - jepsen.util 34 :ok :read [11 7] INFO [2022-01-10 12:53:04,461] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,461] jepsen worker 34 - jepsen.util 34 :invoke :write [11 46] INFO [2022-01-10 12:53:04,462] jepsen worker 7 - jepsen.util 7 :ok :read [2 0] INFO [2022-01-10 12:53:04,463] jepsen worker 7 - jepsen.util 7 :invoke :write [2 25] INFO [2022-01-10 12:53:04,471] jepsen worker 66 - jepsen.util 66 :ok :write [22 6] INFO [2022-01-10 12:53:04,472] jepsen worker 66 - jepsen.util 66 :invoke :write [22 3] INFO [2022-01-10 12:53:04,478] jepsen worker 50 - jepsen.util 50 :ok :write [16 44] INFO [2022-01-10 12:53:04,478] jepsen worker 50 - jepsen.util 50 :invoke :write [16 25] INFO [2022-01-10 12:53:04,482] jepsen worker 81 - jepsen.util 81 :ok :write [27 48] INFO [2022-01-10 12:53:04,483] jepsen worker 81 - jepsen.util 81 :invoke :write [27 8] INFO [2022-01-10 12:53:04,488] jepsen worker 15 - jepsen.util 15 :ok :write [5 43] INFO [2022-01-10 12:53:04,489] jepsen worker 15 - jepsen.util 15 :invoke :write [5 35] INFO [2022-01-10 12:53:04,499] jepsen worker 34 - jepsen.util 34 :ok :write [11 46] INFO [2022-01-10 12:53:04,500] jepsen worker 34 - jepsen.util 34 :invoke :write [11 44] INFO [2022-01-10 12:53:04,503] jepsen worker 7 - jepsen.util 7 :ok :write [2 25] INFO [2022-01-10 12:53:04,503] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,504] jepsen worker 84 - jepsen.util 84 :ok :write [28 22] INFO [2022-01-10 12:53:04,504] jepsen worker 7 - jepsen.util 7 :ok :read [2 25] INFO [2022-01-10 12:53:04,504] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:04,505] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:04,505] jepsen worker 84 - jepsen.util 84 :ok :read [28 22] INFO [2022-01-10 12:53:04,505] jepsen worker 7 - jepsen.util 7 :ok :read [2 25] INFO [2022-01-10 12:53:04,506] jepsen worker 84 - jepsen.util 84 :invoke :write [28 20] INFO [2022-01-10 12:53:04,506] jepsen worker 7 - jepsen.util 7 :invoke :write [2 1] INFO [2022-01-10 12:53:04,523] jepsen worker 66 - jepsen.util 66 :ok :write [22 3] INFO [2022-01-10 12:53:04,523] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:04,524] jepsen worker 66 - jepsen.util 66 :ok :read [22 3] INFO [2022-01-10 12:53:04,525] jepsen worker 66 - jepsen.util 66 :invoke :write [22 13] INFO [2022-01-10 12:53:05,021] jepsen worker 23 - jepsen.util 23 :ok :write [7 13] INFO [2022-01-10 12:53:05,022] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:05,024] jepsen worker 50 - jepsen.util 50 :ok :write [16 25] INFO [2022-01-10 12:53:05,025] jepsen worker 43 - jepsen.util 43 :ok :write [14 46] INFO [2022-01-10 12:53:05,026] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:05,026] jepsen worker 23 - jepsen.util 23 :ok :read [7 13] INFO [2022-01-10 12:53:05,026] jepsen worker 51 - jepsen.util 51 :ok :write [17 43] INFO [2022-01-10 12:53:05,027] jepsen worker 50 - jepsen.util 50 :ok :read [16 25] INFO [2022-01-10 12:53:05,027] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:05,028] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,028] jepsen worker 23 - jepsen.util 23 :ok :read [7 13] INFO [2022-01-10 12:53:05,028] jepsen worker 51 - jepsen.util 51 :invoke :write [17 41] INFO [2022-01-10 12:53:05,029] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:05,029] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:05,029] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:05,030] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,030] jepsen worker 50 - jepsen.util 50 :ok :read [16 25] INFO [2022-01-10 12:53:05,030] jepsen worker 23 - jepsen.util 23 :ok :read [7 13] INFO [2022-01-10 12:53:05,030] jepsen worker 50 - jepsen.util 50 :invoke :write [16 4] INFO [2022-01-10 12:53:05,031] jepsen worker 81 - jepsen.util 81 :ok :write [27 8] INFO [2022-01-10 12:53:05,031] jepsen worker 23 - jepsen.util 23 :invoke :write [7 41] INFO [2022-01-10 12:53:05,031] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:05,032] jepsen worker 81 - jepsen.util 81 :invoke :write [27 33] INFO [2022-01-10 12:53:05,032] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,032] jepsen worker 76 - jepsen.util 76 :ok :write [25 4] INFO [2022-01-10 12:53:05,033] jepsen worker 36 - jepsen.util 36 :ok :write [12 10] INFO [2022-01-10 12:53:05,033] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,033] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:05,034] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,034] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,035] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:05,036] jepsen worker 43 - jepsen.util 43 :invoke :write [14 40] INFO [2022-01-10 12:53:05,036] jepsen worker 87 - jepsen.util 87 :ok :write [29 22] INFO [2022-01-10 12:53:05,037] jepsen worker 87 - jepsen.util 87 :invoke :write [29 35] INFO [2022-01-10 12:53:05,037] jepsen worker 15 - jepsen.util 15 :ok :write [5 35] INFO [2022-01-10 12:53:05,038] jepsen worker 15 - jepsen.util 15 :invoke :write [5 26] INFO [2022-01-10 12:53:05,046] jepsen worker 61 - jepsen.util 61 :ok :write [20 27] INFO [2022-01-10 12:53:05,046] jepsen worker 78 - jepsen.util 78 :ok :write [26 2] INFO [2022-01-10 12:53:05,047] jepsen worker 34 - jepsen.util 34 :ok :write [11 44] INFO [2022-01-10 12:53:05,047] jepsen worker 76 - jepsen.util 76 :ok :read [25 4] INFO [2022-01-10 12:53:05,048] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:05,048] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,049] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,049] jepsen worker 76 - jepsen.util 76 :ok :read [25 4] INFO [2022-01-10 12:53:05,049] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,050] jepsen worker 61 - jepsen.util 61 :ok :read [20 27] INFO [2022-01-10 12:53:05,050] jepsen worker 76 - jepsen.util 76 :invoke :write [25 4] INFO [2022-01-10 12:53:05,051] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,051] jepsen worker 34 - jepsen.util 34 :ok :read [11 44] INFO [2022-01-10 12:53:05,052] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,052] jepsen worker 34 - jepsen.util 34 :ok :read [11 44] INFO [2022-01-10 12:53:05,053] jepsen worker 61 - jepsen.util 61 :ok :read [20 27] INFO [2022-01-10 12:53:05,053] jepsen worker 34 - jepsen.util 34 :invoke :write [11 11] INFO [2022-01-10 12:53:05,053] jepsen worker 7 - jepsen.util 7 :ok :write [2 1] INFO [2022-01-10 12:53:05,053] jepsen worker 0 - jepsen.util 0 :ok :write [0 10] INFO [2022-01-10 12:53:05,054] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,054] jepsen worker 0 - jepsen.util 0 :invoke :write [0 5] INFO [2022-01-10 12:53:05,055] jepsen worker 13 - jepsen.util 13 :ok :write [4 2] INFO [2022-01-10 12:53:05,055] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:05,055] jepsen worker 61 - jepsen.util 61 :ok :read [20 27] INFO [2022-01-10 12:53:05,056] jepsen worker 13 - jepsen.util 13 :invoke :write [4 4] INFO [2022-01-10 12:53:05,056] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,057] jepsen worker 66 - jepsen.util 66 :ok :write [22 13] INFO [2022-01-10 12:53:05,057] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:53:05,057] jepsen worker 61 - jepsen.util 61 :ok :read [20 27] INFO [2022-01-10 12:53:05,057] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,058] jepsen worker 36 - jepsen.util 36 :ok :read [12 10] INFO [2022-01-10 12:53:05,058] jepsen worker 61 - jepsen.util 61 :invoke :write [20 49] INFO [2022-01-10 12:53:05,058] jepsen worker 66 - jepsen.util 66 :ok :read [22 13] INFO [2022-01-10 12:53:05,059] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:05,060] jepsen worker 36 - jepsen.util 36 :invoke :write [12 42] INFO [2022-01-10 12:53:05,060] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,061] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:53:05,061] jepsen worker 66 - jepsen.util 66 :ok :read [22 13] INFO [2022-01-10 12:53:05,061] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:05,062] jepsen worker 41 - jepsen.util 41 :ok :write [13 2] INFO [2022-01-10 12:53:05,062] jepsen worker 50 - jepsen.util 50 :ok :write [16 4] INFO [2022-01-10 12:53:05,062] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,062] jepsen worker 69 - jepsen.util 69 :ok :write [23 1] INFO [2022-01-10 12:53:05,062] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:53:05,063] jepsen worker 66 - jepsen.util 66 :ok :read [22 13] INFO [2022-01-10 12:53:05,063] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:05,064] jepsen worker 69 - jepsen.util 69 :invoke :write [23 34] INFO [2022-01-10 12:53:05,065] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,065] jepsen worker 41 - jepsen.util 41 :ok :read [13 2] INFO [2022-01-10 12:53:05,065] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:05,066] jepsen worker 66 - jepsen.util 66 :ok :read [22 13] INFO [2022-01-10 12:53:05,066] jepsen worker 50 - jepsen.util 50 :invoke :write [16 2] INFO [2022-01-10 12:53:05,067] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:05,067] jepsen worker 66 - jepsen.util 66 :invoke :write [22 41] INFO [2022-01-10 12:53:05,068] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:53:05,068] jepsen worker 23 - jepsen.util 23 :ok :write [7 41] INFO [2022-01-10 12:53:05,068] jepsen worker 41 - jepsen.util 41 :ok :read [13 2] INFO [2022-01-10 12:53:05,068] jepsen worker 7 - jepsen.util 7 :invoke :write [2 9] INFO [2022-01-10 12:53:05,069] jepsen worker 23 - jepsen.util 23 :invoke :write [7 37] INFO [2022-01-10 12:53:05,069] jepsen worker 41 - jepsen.util 41 :invoke :write [13 28] INFO [2022-01-10 12:53:05,072] jepsen worker 81 - jepsen.util 81 :ok :write [27 33] INFO [2022-01-10 12:53:05,073] jepsen worker 81 - jepsen.util 81 :invoke :write [27 15] INFO [2022-01-10 12:53:05,073] jepsen worker 3 - jepsen.util 3 :ok :write [1 14] INFO [2022-01-10 12:53:05,074] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,075] jepsen worker 78 - jepsen.util 78 :ok :read [26 2] INFO [2022-01-10 12:53:05,075] jepsen worker 78 - jepsen.util 78 :invoke :write [26 2] INFO [2022-01-10 12:53:05,075] jepsen worker 3 - jepsen.util 3 :ok :read [1 14] INFO [2022-01-10 12:53:05,075] jepsen worker 44 - jepsen.util 44 :ok :write [14 24] INFO [2022-01-10 12:53:05,076] jepsen worker 3 - jepsen.util 3 :invoke :write [1 36] INFO [2022-01-10 12:53:05,076] jepsen worker 44 - jepsen.util 44 :invoke :write [14 10] INFO [2022-01-10 12:53:05,081] jepsen worker 15 - jepsen.util 15 :ok :write [5 26] INFO [2022-01-10 12:53:05,082] jepsen worker 15 - jepsen.util 15 :invoke :write [5 47] INFO [2022-01-10 12:53:05,082] jepsen worker 84 - jepsen.util 84 :ok :write [28 20] INFO [2022-01-10 12:53:05,083] jepsen worker 84 - jepsen.util 84 :invoke :write [28 23] INFO [2022-01-10 12:53:05,085] jepsen worker 51 - jepsen.util 51 :ok :write [17 41] INFO [2022-01-10 12:53:05,086] jepsen worker 51 - jepsen.util 51 :invoke :write [17 37] INFO [2022-01-10 12:53:05,090] jepsen worker 87 - jepsen.util 87 :ok :write [29 35] INFO [2022-01-10 12:53:05,091] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,092] jepsen worker 87 - jepsen.util 87 :ok :read [29 35] INFO [2022-01-10 12:53:05,092] jepsen worker 36 - jepsen.util 36 :ok :write [12 42] INFO [2022-01-10 12:53:05,093] jepsen worker 87 - jepsen.util 87 :invoke :write [29 7] INFO [2022-01-10 12:53:05,094] jepsen worker 36 - jepsen.util 36 :invoke :write [12 11] INFO [2022-01-10 12:53:05,103] jepsen worker 0 - jepsen.util 0 :ok :write [0 5] INFO [2022-01-10 12:53:05,104] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,105] jepsen worker 0 - jepsen.util 0 :ok :read [0 5] INFO [2022-01-10 12:53:05,106] jepsen worker 13 - jepsen.util 13 :ok :write [4 4] INFO [2022-01-10 12:53:05,106] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,106] jepsen worker 13 - jepsen.util 13 :invoke :write [4 30] INFO [2022-01-10 12:53:05,107] jepsen worker 0 - jepsen.util 0 :ok :read [0 5] INFO [2022-01-10 12:53:05,108] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,109] jepsen worker 0 - jepsen.util 0 :ok :read [0 5] INFO [2022-01-10 12:53:05,110] jepsen worker 0 - jepsen.util 0 :invoke :write [0 23] INFO [2022-01-10 12:53:05,110] jepsen worker nemesis - jepsen.util :nemesis :info :fail-disk ("172.23.105.187") INFO [2022-01-10 12:53:05,120] jepsen worker 3 - jepsen.util 3 :ok :write [1 36] INFO [2022-01-10 12:53:05,121] jepsen worker 3 - jepsen.util 3 :invoke :write [1 25] INFO [2022-01-10 12:53:05,123] jepsen worker 41 - jepsen.util 41 :ok :write [13 28] INFO [2022-01-10 12:53:05,124] jepsen worker 41 - jepsen.util 41 :invoke :write [13 21] INFO [2022-01-10 12:53:05,128] jepsen worker 84 - jepsen.util 84 :ok :write [28 23] INFO [2022-01-10 12:53:05,129] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,130] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:05,130] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,131] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:05,131] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,132] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:05,133] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,134] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:05,135] jepsen worker 84 - jepsen.util 84 :invoke :write [28 30] INFO [2022-01-10 12:53:05,137] jepsen worker 36 - jepsen.util 36 :ok :write [12 11] INFO [2022-01-10 12:53:05,138] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,139] jepsen worker 36 - jepsen.util 36 :ok :read [12 11] INFO [2022-01-10 12:53:05,139] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,140] jepsen worker 36 - jepsen.util 36 :ok :read [12 11] INFO [2022-01-10 12:53:05,141] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,141] jepsen worker 36 - jepsen.util 36 :ok :read [12 11] INFO [2022-01-10 12:53:05,142] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,143] jepsen worker 36 - jepsen.util 36 :ok :read [12 11] INFO [2022-01-10 12:53:05,143] jepsen worker 36 - jepsen.util 36 :invoke :write [12 45] INFO [2022-01-10 12:53:05,145] jepsen worker 3 - jepsen.util 3 :ok :write [1 25] INFO [2022-01-10 12:53:05,146] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,146] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:05,147] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,148] jepsen worker 87 - jepsen.util 87 :ok :write [29 7] INFO [2022-01-10 12:53:05,148] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:05,149] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,150] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:05,150] jepsen worker 60 - jepsen.util 60 :ok :write [20 33] INFO [2022-01-10 12:53:05,150] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,151] jepsen worker 60 - jepsen.util 60 :invoke :write [20 23] INFO [2022-01-10 12:53:05,151] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:05,152] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,153] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:05,153] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,154] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,154] jepsen worker 0 - jepsen.util 0 :ok :write [0 23] INFO [2022-01-10 12:53:05,154] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:05,154] jepsen worker 0 - jepsen.util 0 :invoke :write [0 38] INFO [2022-01-10 12:53:05,155] jepsen worker 87 - jepsen.util 87 :ok :read [29 7] INFO [2022-01-10 12:53:05,155] jepsen worker 3 - jepsen.util 3 :invoke :write [1 27] INFO [2022-01-10 12:53:05,155] jepsen worker 84 - jepsen.util 84 :ok :write [28 30] INFO [2022-01-10 12:53:05,156] jepsen worker 69 - jepsen.util 69 :ok :write [23 34] INFO [2022-01-10 12:53:05,156] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,157] jepsen worker 84 - jepsen.util 84 :invoke :write [28 4] INFO [2022-01-10 12:53:05,157] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,157] jepsen worker 34 - jepsen.util 34 :ok :write [11 11] INFO [2022-01-10 12:53:05,158] jepsen worker 87 - jepsen.util 87 :ok :read [29 7] INFO [2022-01-10 12:53:05,158] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,158] jepsen worker 69 - jepsen.util 69 :ok :read [23 34] INFO [2022-01-10 12:53:05,158] jepsen worker 41 - jepsen.util 41 :ok :write [13 21] INFO [2022-01-10 12:53:05,159] jepsen worker 87 - jepsen.util 87 :invoke :write [29 29] INFO [2022-01-10 12:53:05,159] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:05,159] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,159] jepsen worker 34 - jepsen.util 34 :ok :read [11 11] INFO [2022-01-10 12:53:05,160] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,160] jepsen worker 69 - jepsen.util 69 :ok :read [23 34] INFO [2022-01-10 12:53:05,160] jepsen worker 41 - jepsen.util 41 :ok :read [13 21] INFO [2022-01-10 12:53:05,161] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:05,161] jepsen worker 34 - jepsen.util 34 :ok :read [11 11] INFO [2022-01-10 12:53:05,161] jepsen worker 78 - jepsen.util 78 :ok :write [26 2] INFO [2022-01-10 12:53:05,161] jepsen worker 69 - jepsen.util 69 :invoke :write [23 28] INFO [2022-01-10 12:53:05,162] jepsen worker 41 - jepsen.util 41 :ok :read [13 21] INFO [2022-01-10 12:53:05,163] jepsen worker 78 - jepsen.util 78 :invoke :write [26 26] INFO [2022-01-10 12:53:05,163] jepsen worker 36 - jepsen.util 36 :ok :write [12 45] INFO [2022-01-10 12:53:05,163] jepsen worker 41 - jepsen.util 41 :invoke :write [13 18] INFO [2022-01-10 12:53:05,164] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,164] jepsen worker 36 - jepsen.util 36 :invoke :write [12 5] INFO [2022-01-10 12:53:05,165] jepsen worker 34 - jepsen.util 34 :ok :read [11 11] INFO [2022-01-10 12:53:05,166] jepsen worker 34 - jepsen.util 34 :invoke :write [11 36] INFO [2022-01-10 12:53:05,166] jepsen worker 8 - jepsen.util 8 :ok :write [2 41] INFO [2022-01-10 12:53:05,166] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:05,167] jepsen worker 8 - jepsen.util 8 :ok :read [2 41] INFO [2022-01-10 12:53:05,167] jepsen worker 51 - jepsen.util 51 :ok :write [17 37] INFO [2022-01-10 12:53:05,168] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:05,168] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:05,169] jepsen worker 51 - jepsen.util 51 :ok :read [17 37] INFO [2022-01-10 12:53:05,170] jepsen worker 8 - jepsen.util 8 :ok :read [2 41] INFO [2022-01-10 12:53:05,170] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:05,170] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:05,171] jepsen worker 51 - jepsen.util 51 :ok :read [17 37] INFO [2022-01-10 12:53:05,171] jepsen worker 8 - jepsen.util 8 :ok :read [2 41] INFO [2022-01-10 12:53:05,171] jepsen worker 76 - jepsen.util 76 :ok :write [25 4] INFO [2022-01-10 12:53:05,172] jepsen worker 51 - jepsen.util 51 :invoke :write [17 9] INFO [2022-01-10 12:53:05,172] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:05,173] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,173] jepsen worker 8 - jepsen.util 8 :ok :read [2 41] INFO [2022-01-10 12:53:05,174] jepsen worker 76 - jepsen.util 76 :ok :read [25 4] INFO [2022-01-10 12:53:05,174] jepsen worker 8 - jepsen.util 8 :invoke :write [2 21] INFO [2022-01-10 12:53:05,175] jepsen worker 76 - jepsen.util 76 :invoke :write [25 18] INFO [2022-01-10 12:53:05,175] jepsen worker 3 - jepsen.util 3 :ok :write [1 27] INFO [2022-01-10 12:53:05,176] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,176] jepsen worker 66 - jepsen.util 66 :ok :write [22 41] INFO [2022-01-10 12:53:05,176] jepsen worker 13 - jepsen.util 13 :ok :write [4 30] INFO [2022-01-10 12:53:05,177] jepsen worker 66 - jepsen.util 66 :invoke :write [22 22] INFO [2022-01-10 12:53:05,177] jepsen worker 3 - jepsen.util 3 :ok :read [1 27] INFO [2022-01-10 12:53:05,177] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:05,178] jepsen worker 3 - jepsen.util 3 :invoke :write [1 13] INFO [2022-01-10 12:53:05,178] jepsen worker 13 - jepsen.util 13 :ok :read [4 30] INFO [2022-01-10 12:53:05,179] jepsen worker 13 - jepsen.util 13 :invoke :write [4 33] INFO [2022-01-10 12:53:05,180] jepsen worker 0 - jepsen.util 0 :ok :write [0 38] INFO [2022-01-10 12:53:05,180] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,181] jepsen worker 0 - jepsen.util 0 :ok :read [0 38] INFO [2022-01-10 12:53:05,182] jepsen worker 69 - jepsen.util 69 :ok :write [23 28] INFO [2022-01-10 12:53:05,182] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,183] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,183] jepsen worker 50 - jepsen.util 50 :ok :write [16 2] INFO [2022-01-10 12:53:05,183] jepsen worker 0 - jepsen.util 0 :ok :read [0 38] INFO [2022-01-10 12:53:05,184] jepsen worker 69 - jepsen.util 69 :ok :read [23 28] INFO [2022-01-10 12:53:05,184] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:05,184] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,185] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,185] jepsen worker 50 - jepsen.util 50 :ok :read [16 2] INFO [2022-01-10 12:53:05,185] jepsen worker 84 - jepsen.util 84 :ok :write [28 4] INFO [2022-01-10 12:53:05,185] jepsen worker 69 - jepsen.util 69 :ok :read [23 28] INFO [2022-01-10 12:53:05,186] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,186] jepsen worker 0 - jepsen.util 0 :ok :read [0 38] INFO [2022-01-10 12:53:05,186] jepsen worker 41 - jepsen.util 41 :ok :write [13 18] INFO [2022-01-10 12:53:05,187] jepsen worker 69 - jepsen.util 69 :ok :read [23 28] INFO [2022-01-10 12:53:05,187] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,187] jepsen worker 23 - jepsen.util 23 :ok :write [7 37] INFO [2022-01-10 12:53:05,188] jepsen worker 0 - jepsen.util 0 :invoke :write [0 22] INFO [2022-01-10 12:53:05,188] jepsen worker 84 - jepsen.util 84 :ok :read [28 4] INFO [2022-01-10 12:53:05,189] jepsen worker 50 - jepsen.util 50 :invoke :write [16 12] INFO [2022-01-10 12:53:05,189] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,190] jepsen worker 69 - jepsen.util 69 :invoke :write [23 47] INFO [2022-01-10 12:53:05,190] jepsen worker 78 - jepsen.util 78 :ok :write [26 26] INFO [2022-01-10 12:53:05,190] jepsen worker 84 - jepsen.util 84 :ok :read [28 4] INFO [2022-01-10 12:53:05,191] jepsen worker 41 - jepsen.util 41 :invoke :write [13 42] INFO [2022-01-10 12:53:05,191] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:05,192] jepsen worker 78 - jepsen.util 78 :invoke :write [26 37] INFO [2022-01-10 12:53:05,192] jepsen worker 23 - jepsen.util 23 :ok :read [7 37] INFO [2022-01-10 12:53:05,192] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,193] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:05,193] jepsen worker 84 - jepsen.util 84 :ok :read [28 4] INFO [2022-01-10 12:53:05,194] jepsen worker 84 - jepsen.util 84 :invoke :write [28 19] INFO [2022-01-10 12:53:05,194] jepsen worker 87 - jepsen.util 87 :ok :write [29 29] INFO [2022-01-10 12:53:05,194] jepsen worker 23 - jepsen.util 23 :ok :read [7 37] INFO [2022-01-10 12:53:05,195] jepsen worker 87 - jepsen.util 87 :invoke :write [29 29] INFO [2022-01-10 12:53:05,195] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:05,196] jepsen worker 23 - jepsen.util 23 :ok :read [7 37] INFO [2022-01-10 12:53:05,197] jepsen worker 36 - jepsen.util 36 :ok :write [12 5] INFO [2022-01-10 12:53:05,197] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:05,198] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,198] jepsen worker 23 - jepsen.util 23 :ok :read [7 37] INFO [2022-01-10 12:53:05,199] jepsen worker 36 - jepsen.util 36 :ok :read [12 5] INFO [2022-01-10 12:53:05,199] jepsen worker 23 - jepsen.util 23 :invoke :write [7 21] INFO [2022-01-10 12:53:05,199] jepsen worker 81 - jepsen.util 81 :ok :write [27 15] INFO [2022-01-10 12:53:05,200] jepsen worker 36 - jepsen.util 36 :invoke :write [12 17] INFO [2022-01-10 12:53:05,200] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:05,201] jepsen worker 81 - jepsen.util 81 :ok :read [27 15] INFO [2022-01-10 12:53:05,202] jepsen worker 81 - jepsen.util 81 :invoke :write [27 49] INFO [2022-01-10 12:53:05,204] jepsen worker 51 - jepsen.util 51 :ok :write [17 9] INFO [2022-01-10 12:53:05,205] jepsen worker 51 - jepsen.util 51 :invoke :write [17 15] INFO [2022-01-10 12:53:05,207] jepsen worker 44 - jepsen.util 44 :ok :write [14 10] INFO [2022-01-10 12:53:05,207] jepsen worker 44 - jepsen.util 44 :invoke :write [14 22] INFO [2022-01-10 12:53:05,212] jepsen worker 0 - jepsen.util 0 :ok :write [0 22] INFO [2022-01-10 12:53:05,212] jepsen worker 3 - jepsen.util 3 :ok :write [1 13] INFO [2022-01-10 12:53:05,213] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,213] jepsen worker 3 - jepsen.util 3 :invoke :write [1 19] INFO [2022-01-10 12:53:05,214] jepsen worker 0 - jepsen.util 0 :ok :read [0 22] INFO [2022-01-10 12:53:05,214] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,215] jepsen worker 0 - jepsen.util 0 :ok :read [0 22] INFO [2022-01-10 12:53:05,216] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,216] jepsen worker 15 - jepsen.util 15 :ok :write [5 47] INFO [2022-01-10 12:53:05,217] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,217] jepsen worker 0 - jepsen.util 0 :ok :read [0 22] INFO [2022-01-10 12:53:05,217] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,218] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:05,218] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,219] jepsen worker 13 - jepsen.util 13 :ok :write [4 33] INFO [2022-01-10 12:53:05,219] jepsen worker 0 - jepsen.util 0 :ok :read [0 22] INFO [2022-01-10 12:53:05,219] jepsen worker 0 - jepsen.util 0 :invoke :write [0 47] INFO [2022-01-10 12:53:05,219] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:05,220] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:05,220] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,221] jepsen worker 13 - jepsen.util 13 :ok :read [4 33] INFO [2022-01-10 12:53:05,221] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:05,221] jepsen worker 60 - jepsen.util 60 :ok :write [20 23] INFO [2022-01-10 12:53:05,222] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,222] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:05,223] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,223] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:05,223] jepsen worker 13 - jepsen.util 13 :ok :read [4 33] INFO [2022-01-10 12:53:05,223] jepsen worker 15 - jepsen.util 15 :invoke :write [5 18] INFO [2022-01-10 12:53:05,223] jepsen worker 60 - jepsen.util 60 :ok :read [20 23] INFO [2022-01-10 12:53:05,224] jepsen worker 13 - jepsen.util 13 :invoke :write [4 9] INFO [2022-01-10 12:53:05,224] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,225] jepsen worker 60 - jepsen.util 60 :ok :read [20 23] INFO [2022-01-10 12:53:05,225] jepsen worker 69 - jepsen.util 69 :ok :write [23 47] INFO [2022-01-10 12:53:05,225] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,226] jepsen worker 41 - jepsen.util 41 :ok :write [13 42] INFO [2022-01-10 12:53:05,226] jepsen worker 69 - jepsen.util 69 :ok :read [23 47] INFO [2022-01-10 12:53:05,226] jepsen worker 60 - jepsen.util 60 :invoke :write [20 14] INFO [2022-01-10 12:53:05,227] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:05,227] jepsen worker 69 - jepsen.util 69 :invoke :write [23 48] INFO [2022-01-10 12:53:05,228] jepsen worker 41 - jepsen.util 41 :ok :read [13 42] INFO [2022-01-10 12:53:05,229] jepsen worker 41 - jepsen.util 41 :invoke :write [13 30] INFO [2022-01-10 12:53:05,232] jepsen worker 34 - jepsen.util 34 :ok :write [11 36] INFO [2022-01-10 12:53:05,233] jepsen worker 87 - jepsen.util 87 :ok :write [29 29] INFO [2022-01-10 12:53:05,233] jepsen worker 34 - jepsen.util 34 :invoke :write [11 46] INFO [2022-01-10 12:53:05,233] jepsen worker 87 - jepsen.util 87 :invoke :write [29 33] INFO [2022-01-10 12:53:05,235] jepsen worker 84 - jepsen.util 84 :ok :write [28 19] INFO [2022-01-10 12:53:05,235] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,237] jepsen worker 84 - jepsen.util 84 :ok :read [28 19] INFO [2022-01-10 12:53:05,237] jepsen worker 84 - jepsen.util 84 :invoke :write [28 35] INFO [2022-01-10 12:53:05,239] jepsen worker 78 - jepsen.util 78 :ok :write [26 37] INFO [2022-01-10 12:53:05,239] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:05,240] jepsen worker 78 - jepsen.util 78 :ok :read [26 37] INFO [2022-01-10 12:53:05,241] jepsen worker 8 - jepsen.util 8 :ok :write [2 21] INFO [2022-01-10 12:53:05,241] jepsen worker 78 - jepsen.util 78 :invoke :write [26 26] INFO [2022-01-10 12:53:05,241] jepsen worker 8 - jepsen.util 8 :invoke :write [2 16] INFO [2022-01-10 12:53:05,245] jepsen worker 36 - jepsen.util 36 :ok :write [12 17] INFO [2022-01-10 12:53:05,245] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,246] jepsen worker 36 - jepsen.util 36 :ok :read [12 17] INFO [2022-01-10 12:53:05,247] jepsen worker 76 - jepsen.util 76 :ok :write [25 18] INFO [2022-01-10 12:53:05,247] jepsen worker 36 - jepsen.util 36 :invoke :write [12 40] INFO [2022-01-10 12:53:05,247] jepsen worker 0 - jepsen.util 0 :ok :write [0 47] INFO [2022-01-10 12:53:05,248] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,248] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,249] jepsen worker 76 - jepsen.util 76 :ok :read [25 18] INFO [2022-01-10 12:53:05,249] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,250] jepsen worker 0 - jepsen.util 0 :ok :read [0 47] INFO [2022-01-10 12:53:05,250] jepsen worker 76 - jepsen.util 76 :ok :read [25 18] INFO [2022-01-10 12:53:05,250] jepsen worker 0 - jepsen.util 0 :invoke :write [0 20] INFO [2022-01-10 12:53:05,251] jepsen worker 76 - jepsen.util 76 :invoke :write [25 13] INFO [2022-01-10 12:53:05,256] jepsen worker 66 - jepsen.util 66 :ok :write [22 22] INFO [2022-01-10 12:53:05,257] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,258] jepsen worker 66 - jepsen.util 66 :ok :read [22 22] INFO [2022-01-10 12:53:05,259] jepsen worker 66 - jepsen.util 66 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,259] jepsen worker 66 - jepsen.util 66 :ok :read [22 22] INFO [2022-01-10 12:53:05,260] jepsen worker 50 - jepsen.util 50 :ok :write [16 12] INFO [2022-01-10 12:53:05,260] jepsen worker 66 - jepsen.util 66 :invoke :write [22 24] INFO [2022-01-10 12:53:05,261] jepsen worker 50 - jepsen.util 50 :invoke :write [16 38] INFO [2022-01-10 12:53:05,264] jepsen worker 23 - jepsen.util 23 :ok :write [7 21] INFO [2022-01-10 12:53:05,265] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:05,266] jepsen worker 23 - jepsen.util 23 :ok :read [7 21] INFO [2022-01-10 12:53:05,267] jepsen worker 23 - jepsen.util 23 :invoke :write [7 42] INFO [2022-01-10 12:53:05,272] jepsen worker 81 - jepsen.util 81 :ok :write [27 49] INFO [2022-01-10 12:53:05,273] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:05,274] jepsen worker 81 - jepsen.util 81 :ok :read [27 49] INFO [2022-01-10 12:53:05,275] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:05,275] jepsen worker 44 - jepsen.util 44 :ok :write [14 22] INFO [2022-01-10 12:53:05,275] jepsen worker 81 - jepsen.util 81 :ok :read [27 49] INFO [2022-01-10 12:53:05,276] jepsen worker 44 - jepsen.util 44 :invoke :write [14 28] INFO [2022-01-10 12:53:05,277] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:05,278] jepsen worker 81 - jepsen.util 81 :ok :read [27 49] INFO [2022-01-10 12:53:05,278] jepsen worker 81 - jepsen.util 81 :invoke :write [27 8] INFO [2022-01-10 12:53:05,281] jepsen worker 15 - jepsen.util 15 :ok :write [5 18] INFO [2022-01-10 12:53:05,282] jepsen worker 15 - jepsen.util 15 :invoke :write [5 40] INFO [2022-01-10 12:53:05,289] jepsen worker 34 - jepsen.util 34 :ok :write [11 46] INFO [2022-01-10 12:53:05,289] jepsen worker 34 - jepsen.util 34 :invoke :write [11 22] INFO [2022-01-10 12:53:05,294] jepsen worker 8 - jepsen.util 8 :ok :write [2 16] INFO [2022-01-10 12:53:05,294] jepsen worker 8 - jepsen.util 8 :invoke :write [2 1] INFO [2022-01-10 12:53:05,303] jepsen worker 66 - jepsen.util 66 :ok :write [22 24] INFO [2022-01-10 12:53:05,303] jepsen worker 66 - jepsen.util 66 :invoke :write [22 43] INFO [2022-01-10 12:53:05,306] jepsen worker 50 - jepsen.util 50 :ok :write [16 38] INFO [2022-01-10 12:53:05,307] jepsen worker 50 - jepsen.util 50 :invoke :write [16 8] INFO [2022-01-10 12:53:05,375] jepsen worker 51 - jepsen.util 51 :ok :write [17 15] INFO [2022-01-10 12:53:05,375] jepsen worker 60 - jepsen.util 60 :ok :write [20 14] INFO [2022-01-10 12:53:05,376] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,376] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:05,376] jepsen worker 60 - jepsen.util 60 :ok :read [20 14] INFO [2022-01-10 12:53:05,377] jepsen worker 60 - jepsen.util 60 :invoke :write [20 38] INFO [2022-01-10 12:53:05,377] jepsen worker 51 - jepsen.util 51 :ok :read [17 15] INFO [2022-01-10 12:53:05,378] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:05,379] jepsen worker 51 - jepsen.util 51 :ok :read [17 15] INFO [2022-01-10 12:53:05,380] jepsen worker 51 - jepsen.util 51 :invoke :write [17 44] INFO [2022-01-10 12:53:05,382] jepsen worker 41 - jepsen.util 41 :ok :write [13 30] INFO [2022-01-10 12:53:05,383] jepsen worker 3 - jepsen.util 3 :ok :write [1 19] INFO [2022-01-10 12:53:05,383] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:05,383] jepsen worker 3 - jepsen.util 3 :invoke :write [1 24] INFO [2022-01-10 12:53:05,388] jepsen worker 81 - jepsen.util 81 :ok :write [27 8] INFO [2022-01-10 12:53:05,388] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:05,389] jepsen worker 81 - jepsen.util 81 :ok :read [27 8] INFO [2022-01-10 12:53:05,389] jepsen worker 87 - jepsen.util 87 :ok :write [29 33] INFO [2022-01-10 12:53:05,391] jepsen worker 13 - jepsen.util 13 :ok :write [4 9] INFO [2022-01-10 12:53:05,391] jepsen worker 81 - jepsen.util 81 :invoke :write [27 27] INFO [2022-01-10 12:53:05,392] jepsen worker 13 - jepsen.util 13 :invoke :write [4 37] INFO [2022-01-10 12:53:05,392] jepsen worker 87 - jepsen.util 87 :invoke :write [29 1] INFO [2022-01-10 12:53:05,393] jepsen worker 15 - jepsen.util 15 :ok :write [5 40] INFO [2022-01-10 12:53:05,394] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,395] jepsen worker 15 - jepsen.util 15 :ok :read [5 40] INFO [2022-01-10 12:53:05,396] jepsen worker 15 - jepsen.util 15 :invoke :write [5 46] INFO [2022-01-10 12:53:05,397] jepsen worker 69 - jepsen.util 69 :ok :write [23 48] INFO [2022-01-10 12:53:05,398] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,398] jepsen worker 76 - jepsen.util 76 :ok :write [25 13] INFO [2022-01-10 12:53:05,399] jepsen worker 69 - jepsen.util 69 :ok :read [23 48] INFO [2022-01-10 12:53:05,399] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,399] jepsen worker 69 - jepsen.util 69 :invoke :write [23 4] INFO [2022-01-10 12:53:05,400] jepsen worker 76 - jepsen.util 76 :ok :read [25 13] INFO [2022-01-10 12:53:05,401] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,401] jepsen worker 34 - jepsen.util 34 :ok :write [11 22] INFO [2022-01-10 12:53:05,401] jepsen worker 76 - jepsen.util 76 :ok :read [25 13] INFO [2022-01-10 12:53:05,402] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,403] jepsen worker 76 - jepsen.util 76 :invoke :write [25 17] INFO [2022-01-10 12:53:05,403] jepsen worker 34 - jepsen.util 34 :ok :read [11 22] INFO [2022-01-10 12:53:05,403] jepsen worker 0 - jepsen.util 0 :ok :write [0 20] INFO [2022-01-10 12:53:05,404] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,404] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,406] jepsen worker 0 - jepsen.util 0 :ok :read [0 20] INFO [2022-01-10 12:53:05,406] jepsen worker 84 - jepsen.util 84 :ok :write [28 35] INFO [2022-01-10 12:53:05,406] jepsen worker 34 - jepsen.util 34 :ok :read [11 22] INFO [2022-01-10 12:53:05,406] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,407] jepsen worker 8 - jepsen.util 8 :ok :write [2 1] INFO [2022-01-10 12:53:05,407] jepsen worker 0 - jepsen.util 0 :ok :read [0 20] INFO [2022-01-10 12:53:05,407] jepsen worker 34 - jepsen.util 34 :invoke :write [11 10] INFO [2022-01-10 12:53:05,408] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,409] jepsen worker 78 - jepsen.util 78 :ok :write [26 26] INFO [2022-01-10 12:53:05,409] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:05,409] jepsen worker 23 - jepsen.util 23 :ok :write [7 42] INFO [2022-01-10 12:53:05,409] jepsen worker 84 - jepsen.util 84 :ok :read [28 35] INFO [2022-01-10 12:53:05,411] jepsen worker 84 - jepsen.util 84 :invoke :write [28 23] INFO [2022-01-10 12:53:05,411] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,412] jepsen worker 8 - jepsen.util 8 :ok :read [2 1] INFO [2022-01-10 12:53:05,412] jepsen worker 78 - jepsen.util 78 :invoke :write [26 0] INFO [2022-01-10 12:53:05,412] jepsen worker 0 - jepsen.util 0 :ok :read [0 20] INFO [2022-01-10 12:53:05,412] jepsen worker 66 - jepsen.util 66 :ok :write [22 43] INFO [2022-01-10 12:53:05,413] jepsen worker 8 - jepsen.util 8 :invoke :write [2 42] INFO [2022-01-10 12:53:05,414] jepsen worker 23 - jepsen.util 23 :invoke :write [7 9] INFO [2022-01-10 12:53:05,415] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,415] jepsen worker 66 - jepsen.util 66 :invoke :write [22 49] INFO [2022-01-10 12:53:05,415] jepsen worker 0 - jepsen.util 0 :ok :read [0 20] INFO [2022-01-10 12:53:05,416] jepsen worker 0 - jepsen.util 0 :invoke :write [0 17] INFO [2022-01-10 12:53:05,416] jepsen worker 44 - jepsen.util 44 :ok :write [14 28] INFO [2022-01-10 12:53:05,417] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,418] jepsen worker 36 - jepsen.util 36 :ok :write [12 40] INFO [2022-01-10 12:53:05,418] jepsen worker 44 - jepsen.util 44 :ok :read [14 28] INFO [2022-01-10 12:53:05,418] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,419] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,419] jepsen worker 36 - jepsen.util 36 :ok :read [12 40] INFO [2022-01-10 12:53:05,420] jepsen worker 44 - jepsen.util 44 :ok :read [14 28] INFO [2022-01-10 12:53:05,420] jepsen worker 50 - jepsen.util 50 :ok :write [16 8] INFO [2022-01-10 12:53:05,420] jepsen worker 36 - jepsen.util 36 :invoke :write [12 28] INFO [2022-01-10 12:53:05,421] jepsen worker 50 - jepsen.util 50 :invoke :write [16 28] INFO [2022-01-10 12:53:05,421] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,421] jepsen worker 60 - jepsen.util 60 :ok :write [20 38] INFO [2022-01-10 12:53:05,422] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,422] jepsen worker 44 - jepsen.util 44 :ok :read [14 28] INFO [2022-01-10 12:53:05,423] jepsen worker 60 - jepsen.util 60 :ok :read [20 38] INFO [2022-01-10 12:53:05,423] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,423] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,424] jepsen worker 44 - jepsen.util 44 :ok :read [14 28] INFO [2022-01-10 12:53:05,424] jepsen worker 60 - jepsen.util 60 :ok :read [20 38] INFO [2022-01-10 12:53:05,424] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,425] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,426] jepsen worker 51 - jepsen.util 51 :ok :write [17 44] INFO [2022-01-10 12:53:05,426] jepsen worker 60 - jepsen.util 60 :ok :read [20 38] INFO [2022-01-10 12:53:05,427] jepsen worker 51 - jepsen.util 51 :invoke :write [17 18] INFO [2022-01-10 12:53:05,427] jepsen worker 44 - jepsen.util 44 :ok :read [14 28] INFO [2022-01-10 12:53:05,427] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,427] jepsen worker 44 - jepsen.util 44 :invoke :write [14 30] INFO [2022-01-10 12:53:05,428] jepsen worker 60 - jepsen.util 60 :ok :read [20 38] INFO [2022-01-10 12:53:05,429] jepsen worker 60 - jepsen.util 60 :invoke :write [20 29] INFO [2022-01-10 12:53:05,433] jepsen worker 81 - jepsen.util 81 :ok :write [27 27] INFO [2022-01-10 12:53:05,434] jepsen worker 3 - jepsen.util 3 :ok :write [1 24] INFO [2022-01-10 12:53:05,434] jepsen worker 76 - jepsen.util 76 :ok :write [25 17] INFO [2022-01-10 12:53:05,434] jepsen worker 81 - jepsen.util 81 :invoke :write [27 48] INFO [2022-01-10 12:53:05,435] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,435] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,436] jepsen worker 3 - jepsen.util 3 :ok :read [1 24] INFO [2022-01-10 12:53:05,436] jepsen worker 76 - jepsen.util 76 :ok :read [25 17] INFO [2022-01-10 12:53:05,436] jepsen worker 3 - jepsen.util 3 :invoke :write [1 28] INFO [2022-01-10 12:53:05,437] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,437] jepsen worker 76 - jepsen.util 76 :ok :read [25 17] INFO [2022-01-10 12:53:05,438] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,439] jepsen worker 87 - jepsen.util 87 :ok :write [29 1] INFO [2022-01-10 12:53:05,439] jepsen worker 76 - jepsen.util 76 :ok :read [25 17] INFO [2022-01-10 12:53:05,440] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,440] jepsen worker 87 - jepsen.util 87 :invoke :write [29 47] INFO [2022-01-10 12:53:05,441] jepsen worker 76 - jepsen.util 76 :ok :read [25 17] INFO [2022-01-10 12:53:05,441] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,442] jepsen worker 76 - jepsen.util 76 :ok :read [25 17] INFO [2022-01-10 12:53:05,442] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,443] jepsen worker 15 - jepsen.util 15 :ok :write [5 46] INFO [2022-01-10 12:53:05,443] jepsen worker 76 - jepsen.util 76 :ok :read [25 17] INFO [2022-01-10 12:53:05,444] jepsen worker 13 - jepsen.util 13 :ok :write [4 37] INFO [2022-01-10 12:53:05,444] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,444] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:05,445] jepsen worker 15 - jepsen.util 15 :ok :read [5 46] INFO [2022-01-10 12:53:05,445] jepsen worker 76 - jepsen.util 76 :invoke :write [25 28] INFO [2022-01-10 12:53:05,445] jepsen worker 13 - jepsen.util 13 :ok :read [4 37] INFO [2022-01-10 12:53:05,446] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,446] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:05,447] jepsen worker 15 - jepsen.util 15 :ok :read [5 46] INFO [2022-01-10 12:53:05,447] jepsen worker 13 - jepsen.util 13 :ok :read [4 37] INFO [2022-01-10 12:53:05,447] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,448] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:05,448] jepsen worker 15 - jepsen.util 15 :ok :read [5 46] INFO [2022-01-10 12:53:05,449] jepsen worker 13 - jepsen.util 13 :ok :read [4 37] INFO [2022-01-10 12:53:05,449] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,449] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:05,450] jepsen worker 15 - jepsen.util 15 :ok :read [5 46] INFO [2022-01-10 12:53:05,450] jepsen worker 13 - jepsen.util 13 :ok :read [4 37] INFO [2022-01-10 12:53:05,450] jepsen worker 15 - jepsen.util 15 :invoke :write [5 48] INFO [2022-01-10 12:53:05,451] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:05,451] jepsen worker 69 - jepsen.util 69 :ok :write [23 4] INFO [2022-01-10 12:53:05,452] jepsen worker 13 - jepsen.util 13 :ok :read [4 37] INFO [2022-01-10 12:53:05,452] jepsen worker 0 - jepsen.util 0 :ok :write [0 17] INFO [2022-01-10 12:53:05,452] jepsen worker 13 - jepsen.util 13 :invoke :write [4 8] INFO [2022-01-10 12:53:05,453] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,453] jepsen worker 69 - jepsen.util 69 :invoke :write [23 47] INFO [2022-01-10 12:53:05,454] jepsen worker 0 - jepsen.util 0 :ok :read [0 17] INFO [2022-01-10 12:53:05,454] jepsen worker 0 - jepsen.util 0 :invoke :write [0 30] INFO [2022-01-10 12:53:05,458] jepsen worker 34 - jepsen.util 34 :ok :write [11 10] INFO [2022-01-10 12:53:05,458] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,459] jepsen worker 34 - jepsen.util 34 :ok :read [11 10] INFO [2022-01-10 12:53:05,460] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,461] jepsen worker 34 - jepsen.util 34 :ok :read [11 10] WARN [2022-01-10 12:53:05,461] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":230627,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":1352712,"total_server_duration_us":143503,"encode_duration_us":138,"operation_id":"0x239","timeout_ms":10000,"last_server_duration_us":142231,"total_duration_us":3959429},{"operation_name":"upsert","last_dispatch_duration_us":9595,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000004/00000000C4B4FBBF","last_local_socket":"172.23.104.79:58638","total_dispatch_duration_us":829209,"total_server_duration_us":10556,"encode_duration_us":143,"operation_id":"0x24a","timeout_ms":10000,"last_server_duration_us":9002,"total_duration_us":3951705},{"operation_name":"upsert","last_dispatch_duration_us":218073,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":231973,"total_server_duration_us":219096,"encode_duration_us":256,"operation_id":"0x22f","timeout_ms":10000,"last_server_duration_us":217316,"total_duration_us":3859087},{"operation_name":"upsert","last_dispatch_duration_us":108114,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":117462,"total_server_duration_us":109195,"encode_duration_us":35,"operation_id":"0xd81","timeout_ms":10000,"last_server_duration_us":107592,"total_duration_us":2725485},{"operation_name":"upsert","last_dispatch_duration_us":446656,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":452106,"total_server_duration_us":103446,"encode_duration_us":32,"operation_id":"0x1334","timeout_ms":10000,"last_server_duration_us":102800,"total_duration_us":1534941},{"operation_name":"upsert","last_dispatch_duration_us":1432759,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":1434061,"total_server_duration_us":1431553,"encode_duration_us":67,"operation_id":"0x1322","timeout_ms":10000,"last_server_duration_us":1431369,"total_duration_us":1450792},{"operation_name":"upsert","last_dispatch_duration_us":1444281,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000004/0000000048226CCA","last_local_socket":"172.23.104.79:48384","total_dispatch_duration_us":1444281,"total_server_duration_us":1443510,"encode_duration_us":54,"operation_id":"0x132e","timeout_ms":10000,"last_server_duration_us":1443510,"total_duration_us":1444403},{"operation_name":"upsert","last_dispatch_duration_us":1442091,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000004/00000000C4B4FBBF","last_local_socket":"172.23.104.79:58638","total_dispatch_duration_us":1442091,"total_server_duration_us":1441078,"encode_duration_us":51,"operation_id":"0x1318","timeout_ms":10000,"last_server_duration_us":1441078,"total_duration_us":1442188},{"operation_name":"upsert","last_dispatch_duration_us":1431563,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":1431563,"total_server_duration_us":1430400,"encode_duration_us":39,"operation_id":"0x132a","timeout_ms":10000,"last_server_duration_us":1430400,"total_duration_us":1431684},{"operation_name":"upsert","last_dispatch_duration_us":167992,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":176883,"total_server_duration_us":168288,"encode_duration_us":49,"operation_id":"0xe1a","timeout_ms":10000,"last_server_duration_us":167439,"total_duration_us":1254433}],"total_count":15}} INFO [2022-01-10 12:53:05,461] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,462] jepsen worker 84 - jepsen.util 84 :ok :write [28 23] INFO [2022-01-10 12:53:05,462] jepsen worker 84 - jepsen.util 84 :invoke :write [28 3] INFO [2022-01-10 12:53:05,463] jepsen worker 34 - jepsen.util 34 :ok :read [11 10] WARN [2022-01-10 12:53:05,463] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":53705,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":77295,"total_server_duration_us":55515,"encode_duration_us":239,"operation_id":"0x21d","timeout_ms":10000,"last_server_duration_us":53135,"total_duration_us":4730814},{"operation_name":"upsert","last_dispatch_duration_us":322097,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000003/000000000DBBABF1","last_local_socket":"172.23.104.79:34702","total_dispatch_duration_us":332513,"total_server_duration_us":52022,"encode_duration_us":32,"operation_id":"0xe32","timeout_ms":10000,"last_server_duration_us":50301,"total_duration_us":3959190},{"operation_name":"upsert","last_dispatch_duration_us":15622,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":31908,"total_server_duration_us":16972,"encode_duration_us":241,"operation_id":"0x217","timeout_ms":10000,"last_server_duration_us":14989,"total_duration_us":3661303},{"operation_name":"upsert","last_dispatch_duration_us":1429412,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000003/000000000DBBABF1","last_local_socket":"172.23.104.79:34702","total_dispatch_duration_us":1429412,"total_server_duration_us":1427978,"encode_duration_us":41,"operation_id":"0x131f","timeout_ms":10000,"last_server_duration_us":1427978,"total_duration_us":1429537},{"operation_name":"upsert","last_dispatch_duration_us":1426104,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000003/000000000DBBABF1","last_local_socket":"172.23.104.79:34702","total_dispatch_duration_us":1426104,"total_server_duration_us":1425074,"encode_duration_us":31,"operation_id":"0x1328","timeout_ms":10000,"last_server_duration_us":1425074,"total_duration_us":1426213},{"operation_name":"upsert","last_dispatch_duration_us":113129,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":121273,"total_server_duration_us":113826,"encode_duration_us":62,"operation_id":"0x94e","timeout_ms":10000,"last_server_duration_us":112475,"total_duration_us":1205197},{"operation_name":"upsert","last_dispatch_duration_us":323109,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000003/000000000DBBABF1","last_local_socket":"172.23.104.79:34702","total_dispatch_duration_us":330685,"total_server_duration_us":323544,"encode_duration_us":113,"operation_id":"0x661","timeout_ms":10000,"last_server_duration_us":322535,"total_duration_us":889372},{"operation_name":"upsert","last_dispatch_duration_us":151472,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000003/000000000DBBABF1","last_local_socket":"172.23.104.79:34702","total_dispatch_duration_us":160804,"total_server_duration_us":151782,"encode_duration_us":33,"operation_id":"0xd95","timeout_ms":10000,"last_server_duration_us":150873,"total_duration_us":723272},{"operation_name":"upsert","last_dispatch_duration_us":449047,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":455348,"total_server_duration_us":449273,"encode_duration_us":83,"operation_id":"0x707","timeout_ms":10000,"last_server_duration_us":448437,"total_duration_us":625510},{"operation_name":"upsert","last_dispatch_duration_us":42365,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":49646,"total_server_duration_us":42768,"encode_duration_us":90,"operation_id":"0x75f","timeout_ms":10000,"last_server_duration_us":41767,"total_duration_us":605463}],"total_count":14}} INFO [2022-01-10 12:53:05,463] jepsen worker 7 - jepsen.util 7 :ok :write [2 9] WARN [2022-01-10 12:53:05,463] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":98603,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000002/00000000F0645BC3","last_local_socket":"172.23.104.79:34668","total_dispatch_duration_us":114426,"total_server_duration_us":99253,"encode_duration_us":39,"operation_id":"0xb1b","timeout_ms":10000,"last_server_duration_us":97483,"total_duration_us":5771055},{"operation_name":"upsert","last_dispatch_duration_us":138849,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":314705,"total_server_duration_us":139858,"encode_duration_us":277,"operation_id":"0x24f","timeout_ms":10000,"last_server_duration_us":138260,"total_duration_us":4456166},{"operation_name":"upsert","last_dispatch_duration_us":35251,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","total_dispatch_duration_us":908172,"total_server_duration_us":40213,"encode_duration_us":154,"operation_id":"0x237","timeout_ms":10000,"last_server_duration_us":34215,"total_duration_us":3522422},{"operation_name":"upsert","last_dispatch_duration_us":130915,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":140079,"total_server_duration_us":131494,"encode_duration_us":41,"operation_id":"0xd71","timeout_ms":10000,"last_server_duration_us":130287,"total_duration_us":2756479},{"operation_name":"upsert","last_dispatch_duration_us":42604,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":50761,"total_server_duration_us":43315,"encode_duration_us":38,"operation_id":"0xe46","timeout_ms":10000,"last_server_duration_us":42091,"total_duration_us":2148648},{"operation_name":"upsert","last_dispatch_duration_us":230016,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":237764,"total_server_duration_us":230504,"encode_duration_us":77,"operation_id":"0xb79","timeout_ms":10000,"last_server_duration_us":229415,"total_duration_us":1358634},{"operation_name":"upsert","last_dispatch_duration_us":76942,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000002/00000000F0645BC3","last_local_socket":"172.23.104.79:34668","total_dispatch_duration_us":84694,"total_server_duration_us":77375,"encode_duration_us":101,"operation_id":"0x3fd","timeout_ms":10000,"last_server_duration_us":76173,"total_duration_us":1172738},{"operation_name":"upsert","last_dispatch_duration_us":45819,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":53856,"total_server_duration_us":46075,"encode_duration_us":31,"operation_id":"0xd85","timeout_ms":10000,"last_server_duration_us":45278,"total_duration_us":1128485},{"operation_name":"upsert","last_dispatch_duration_us":28957,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":36613,"total_server_duration_us":28854,"encode_duration_us":61,"operation_id":"0x6f5","timeout_ms":10000,"last_server_duration_us":28036,"total_duration_us":1116057},{"operation_name":"upsert","last_dispatch_duration_us":489701,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":495165,"total_server_duration_us":489883,"encode_duration_us":33,"operation_id":"0xc6b","timeout_ms":10000,"last_server_duration_us":489137,"total_duration_us":1055287}],"total_count":17}} INFO [2022-01-10 12:53:05,463] jepsen worker 34 - jepsen.util 34 :invoke :write [11 20] INFO [2022-01-10 12:53:05,464] jepsen worker 7 - jepsen.util 7 :invoke :write [2 30] INFO [2022-01-10 12:53:05,467] jepsen worker 78 - jepsen.util 78 :ok :write [26 0] INFO [2022-01-10 12:53:05,468] jepsen worker 78 - jepsen.util 78 :invoke :write [26 22] INFO [2022-01-10 12:53:05,476] jepsen worker 36 - jepsen.util 36 :ok :write [12 28] INFO [2022-01-10 12:53:05,476] jepsen worker 36 - jepsen.util 36 :invoke :write [12 34] INFO [2022-01-10 12:53:05,481] jepsen worker 87 - jepsen.util 87 :ok :write [29 47] INFO [2022-01-10 12:53:05,482] jepsen worker 87 - jepsen.util 87 :invoke :write [29 31] INFO [2022-01-10 12:53:05,484] jepsen worker 3 - jepsen.util 3 :ok :write [1 28] INFO [2022-01-10 12:53:05,485] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,486] jepsen worker 3 - jepsen.util 3 :ok :read [1 28] INFO [2022-01-10 12:53:05,486] jepsen worker 3 - jepsen.util 3 :invoke :write [1 32] INFO [2022-01-10 12:53:05,491] jepsen worker 0 - jepsen.util 0 :ok :write [0 30] INFO [2022-01-10 12:53:05,491] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,492] jepsen worker 0 - jepsen.util 0 :ok :read [0 30] INFO [2022-01-10 12:53:05,492] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,493] jepsen worker 0 - jepsen.util 0 :ok :read [0 30] INFO [2022-01-10 12:53:05,494] jepsen worker 0 - jepsen.util 0 :invoke :write [0 30] INFO [2022-01-10 12:53:05,502] jepsen worker 84 - jepsen.util 84 :ok :write [28 3] INFO [2022-01-10 12:53:05,502] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,503] jepsen worker 84 - jepsen.util 84 :ok :read [28 3] INFO [2022-01-10 12:53:05,504] jepsen worker 84 - jepsen.util 84 :invoke :write [28 46] INFO [2022-01-10 12:53:05,506] jepsen worker 87 - jepsen.util 87 :ok :write [29 31] INFO [2022-01-10 12:53:05,507] jepsen worker 87 - jepsen.util 87 :invoke :write [29 36] INFO [2022-01-10 12:53:05,509] jepsen worker 36 - jepsen.util 36 :ok :write [12 34] INFO [2022-01-10 12:53:05,510] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,511] jepsen worker 36 - jepsen.util 36 :ok :read [12 34] INFO [2022-01-10 12:53:05,511] jepsen worker 36 - jepsen.util 36 :invoke :write [12 41] INFO [2022-01-10 12:53:05,513] jepsen worker 0 - jepsen.util 0 :ok :write [0 30] INFO [2022-01-10 12:53:05,513] jepsen worker 0 - jepsen.util 0 :invoke :write [0 14] INFO [2022-01-10 12:53:05,514] jepsen worker 3 - jepsen.util 3 :ok :write [1 32] INFO [2022-01-10 12:53:05,515] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,516] jepsen worker 3 - jepsen.util 3 :ok :read [1 32] INFO [2022-01-10 12:53:05,517] jepsen worker 3 - jepsen.util 3 :invoke :write [1 41] INFO [2022-01-10 12:53:05,523] jepsen worker 84 - jepsen.util 84 :ok :write [28 46] INFO [2022-01-10 12:53:05,523] jepsen worker 87 - jepsen.util 87 :ok :write [29 36] INFO [2022-01-10 12:53:05,523] jepsen worker 87 - jepsen.util 87 :invoke :write [29 15] INFO [2022-01-10 12:53:05,524] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,525] jepsen worker 0 - jepsen.util 0 :ok :write [0 14] INFO [2022-01-10 12:53:05,525] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,526] jepsen worker 0 - jepsen.util 0 :ok :read [0 14] INFO [2022-01-10 12:53:05,527] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,527] jepsen worker 36 - jepsen.util 36 :ok :write [12 41] INFO [2022-01-10 12:53:05,528] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,528] jepsen worker 0 - jepsen.util 0 :ok :read [0 14] INFO [2022-01-10 12:53:05,528] jepsen worker 0 - jepsen.util 0 :invoke :write [0 22] INFO [2022-01-10 12:53:05,529] jepsen worker 36 - jepsen.util 36 :ok :read [12 41] INFO [2022-01-10 12:53:05,529] jepsen worker 36 - jepsen.util 36 :invoke :write [12 47] INFO [2022-01-10 12:53:05,534] jepsen worker 3 - jepsen.util 3 :ok :write [1 41] INFO [2022-01-10 12:53:05,535] jepsen worker 3 - jepsen.util 3 :invoke :write [1 34] INFO [2022-01-10 12:53:05,541] jepsen worker 87 - jepsen.util 87 :ok :write [29 15] INFO [2022-01-10 12:53:05,542] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,543] jepsen worker 87 - jepsen.util 87 :ok :read [29 15] INFO [2022-01-10 12:53:05,543] jepsen worker 87 - jepsen.util 87 :invoke :write [29 48] INFO [2022-01-10 12:53:05,546] jepsen worker 36 - jepsen.util 36 :ok :write [12 47] INFO [2022-01-10 12:53:05,546] jepsen worker 36 - jepsen.util 36 :invoke :write [12 22] INFO [2022-01-10 12:53:05,546] jepsen worker 0 - jepsen.util 0 :ok :write [0 22] INFO [2022-01-10 12:53:05,547] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,548] jepsen worker 0 - jepsen.util 0 :ok :read [0 22] INFO [2022-01-10 12:53:05,548] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,548] jepsen worker 23 - jepsen.util 23 :ok :write [7 9] INFO [2022-01-10 12:53:05,549] jepsen worker 23 - jepsen.util 23 :invoke :write [7 45] INFO [2022-01-10 12:53:05,549] jepsen worker 0 - jepsen.util 0 :ok :read [0 22] INFO [2022-01-10 12:53:05,550] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,551] jepsen worker 0 - jepsen.util 0 :ok :read [0 22] INFO [2022-01-10 12:53:05,551] jepsen worker 0 - jepsen.util 0 :invoke :write [0 42] INFO [2022-01-10 12:53:05,551] jepsen worker 51 - jepsen.util 51 :ok :write [17 18] INFO [2022-01-10 12:53:05,552] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:05,553] jepsen worker 51 - jepsen.util 51 :ok :read [17 18] INFO [2022-01-10 12:53:05,554] jepsen worker 51 - jepsen.util 51 :invoke :write [17 38] INFO [2022-01-10 12:53:05,554] jepsen worker 3 - jepsen.util 3 :ok :write [1 34] INFO [2022-01-10 12:53:05,556] jepsen worker 3 - jepsen.util 3 :invoke :write [1 41] INFO [2022-01-10 12:53:05,558] jepsen worker 68 - jepsen.util 68 :ok :write [22 2] INFO [2022-01-10 12:53:05,558] jepsen worker 84 - jepsen.util 84 :ok :read [28 46] INFO [2022-01-10 12:53:05,559] jepsen worker 13 - jepsen.util 13 :ok :write [4 8] INFO [2022-01-10 12:53:05,559] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,560] jepsen worker 68 - jepsen.util 68 :invoke :write [22 10] INFO [2022-01-10 12:53:05,560] jepsen worker 84 - jepsen.util 84 :ok :read [28 46] INFO [2022-01-10 12:53:05,560] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:05,561] jepsen worker 84 - jepsen.util 84 :invoke :write [28 42] INFO [2022-01-10 12:53:05,561] jepsen worker 13 - jepsen.util 13 :ok :read [4 8] INFO [2022-01-10 12:53:05,562] jepsen worker 13 - jepsen.util 13 :invoke :write [4 16] INFO [2022-01-10 12:53:05,563] jepsen worker 44 - jepsen.util 44 :ok :write [14 30] INFO [2022-01-10 12:53:05,563] jepsen worker 69 - jepsen.util 69 :ok :write [23 47] INFO [2022-01-10 12:53:05,563] jepsen worker 44 - jepsen.util 44 :invoke :write [14 25] WARN [2022-01-10 12:53:05,564] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":12167,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":1452767,"total_server_duration_us":13190,"encode_duration_us":187,"operation_id":"0x1f0","timeout_ms":10000,"last_server_duration_us":11614,"total_duration_us":4568578},{"operation_name":"upsert","last_dispatch_duration_us":167855,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":1273720,"total_server_duration_us":168377,"encode_duration_us":170,"operation_id":"0x1bf","timeout_ms":10000,"last_server_duration_us":167050,"total_duration_us":4385081},{"operation_name":"upsert","last_dispatch_duration_us":8071,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":1154270,"total_server_duration_us":9607,"encode_duration_us":218,"operation_id":"0x21c","timeout_ms":10000,"last_server_duration_us":7437,"total_duration_us":4319893},{"operation_name":"upsert","last_dispatch_duration_us":26403,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000006/0000000044D7A75E","last_local_socket":"172.23.104.79:48380","total_dispatch_duration_us":35926,"total_server_duration_us":27207,"encode_duration_us":52,"operation_id":"0xe51","timeout_ms":10000,"last_server_duration_us":25888,"total_duration_us":2640072},{"operation_name":"upsert","last_dispatch_duration_us":27198,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":37418,"total_server_duration_us":26991,"encode_duration_us":59,"operation_id":"0x76f","timeout_ms":10000,"last_server_duration_us":26418,"total_duration_us":1630336},{"operation_name":"upsert","last_dispatch_duration_us":122581,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000006/0000000044D7A75E","last_local_socket":"172.23.104.79:48380","total_dispatch_duration_us":129289,"total_server_duration_us":123076,"encode_duration_us":81,"operation_id":"0x941","timeout_ms":10000,"last_server_duration_us":122007,"total_duration_us":1213218},{"operation_name":"upsert","last_dispatch_duration_us":76103,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":84989,"total_server_duration_us":76616,"encode_duration_us":88,"operation_id":"0x769","timeout_ms":10000,"last_server_duration_us":75478,"total_duration_us":1176175},{"operation_name":"upsert","last_dispatch_duration_us":455545,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":462315,"total_server_duration_us":455865,"encode_duration_us":94,"operation_id":"0x65f","timeout_ms":10000,"last_server_duration_us":454967,"total_duration_us":1029784},{"operation_name":"upsert","last_dispatch_duration_us":930356,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000006/0000000044D7A75E","last_local_socket":"172.23.104.79:48380","total_dispatch_duration_us":930356,"total_server_duration_us":929507,"encode_duration_us":33,"operation_id":"0x130e","timeout_ms":10000,"last_server_duration_us":929507,"total_duration_us":930492},{"operation_name":"upsert","last_dispatch_duration_us":170830,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":178046,"total_server_duration_us":171060,"encode_duration_us":71,"operation_id":"0xb66","timeout_ms":10000,"last_server_duration_us":170173,"total_duration_us":733088}],"total_count":15}} INFO [2022-01-10 12:53:05,564] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,565] jepsen worker 36 - jepsen.util 36 :ok :write [12 22] INFO [2022-01-10 12:53:05,565] jepsen worker 69 - jepsen.util 69 :ok :read [23 47] INFO [2022-01-10 12:53:05,566] jepsen worker 36 - jepsen.util 36 :invoke :write [12 11] INFO [2022-01-10 12:53:05,566] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,568] jepsen worker 69 - jepsen.util 69 :ok :read [23 47] INFO [2022-01-10 12:53:05,568] jepsen worker 69 - jepsen.util 69 :invoke :write [23 47] INFO [2022-01-10 12:53:05,569] jepsen worker 60 - jepsen.util 60 :ok :write [20 29] INFO [2022-01-10 12:53:05,569] jepsen worker 87 - jepsen.util 87 :ok :write [29 48] INFO [2022-01-10 12:53:05,570] jepsen worker 78 - jepsen.util 78 :ok :write [26 22] INFO [2022-01-10 12:53:05,570] jepsen worker 87 - jepsen.util 87 :invoke :write [29 44] INFO [2022-01-10 12:53:05,571] jepsen worker 78 - jepsen.util 78 :invoke :write [26 36] INFO [2022-01-10 12:53:05,571] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,572] jepsen worker 60 - jepsen.util 60 :ok :read [20 29] INFO [2022-01-10 12:53:05,573] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,574] jepsen worker 60 - jepsen.util 60 :ok :read [20 29] INFO [2022-01-10 12:53:05,575] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,576] jepsen worker 60 - jepsen.util 60 :ok :read [20 29] INFO [2022-01-10 12:53:05,576] jepsen worker 50 - jepsen.util 50 :ok :write [16 28] INFO [2022-01-10 12:53:05,577] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:05,578] jepsen worker 60 - jepsen.util 60 :invoke :write [20 28] INFO [2022-01-10 12:53:05,578] jepsen worker 50 - jepsen.util 50 :ok :read [16 28] INFO [2022-01-10 12:53:05,578] jepsen worker 0 - jepsen.util 0 :ok :write [0 42] INFO [2022-01-10 12:53:05,579] jepsen worker 50 - jepsen.util 50 :invoke :write [16 4] INFO [2022-01-10 12:53:05,579] jepsen worker 51 - jepsen.util 51 :ok :write [17 38] INFO [2022-01-10 12:53:05,579] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,580] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:05,580] jepsen worker 0 - jepsen.util 0 :ok :read [0 42] INFO [2022-01-10 12:53:05,581] jepsen worker 51 - jepsen.util 51 :ok :read [17 38] INFO [2022-01-10 12:53:05,581] jepsen worker 76 - jepsen.util 76 :ok :write [25 28] INFO [2022-01-10 12:53:05,581] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,582] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:05,582] jepsen worker 0 - jepsen.util 0 :ok :read [0 42] INFO [2022-01-10 12:53:05,583] jepsen worker 51 - jepsen.util 51 :ok :read [17 38] INFO [2022-01-10 12:53:05,583] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,584] jepsen worker 76 - jepsen.util 76 :ok :read [25 28] INFO [2022-01-10 12:53:05,584] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,584] jepsen worker 51 - jepsen.util 51 :invoke :write [17 13] INFO [2022-01-10 12:53:05,585] jepsen worker 0 - jepsen.util 0 :ok :read [0 42] INFO [2022-01-10 12:53:05,585] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,586] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,586] jepsen worker 76 - jepsen.util 76 :ok :read [25 28] INFO [2022-01-10 12:53:05,587] jepsen worker 81 - jepsen.util 81 :ok :write [27 48] INFO [2022-01-10 12:53:05,587] jepsen worker 3 - jepsen.util 3 :ok :write [1 41] INFO [2022-01-10 12:53:05,587] jepsen worker 0 - jepsen.util 0 :ok :read [0 42] INFO [2022-01-10 12:53:05,587] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,588] jepsen worker 76 - jepsen.util 76 :ok :read [25 28] INFO [2022-01-10 12:53:05,588] jepsen worker 0 - jepsen.util 0 :invoke :write [0 20] INFO [2022-01-10 12:53:05,589] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,589] jepsen worker 87 - jepsen.util 87 :ok :write [29 44] INFO [2022-01-10 12:53:05,590] jepsen worker 3 - jepsen.util 3 :invoke :write [1 37] INFO [2022-01-10 12:53:05,590] jepsen worker 76 - jepsen.util 76 :ok :read [25 28] INFO [2022-01-10 12:53:05,591] jepsen worker 81 - jepsen.util 81 :invoke :write [27 27] INFO [2022-01-10 12:53:05,592] jepsen worker 87 - jepsen.util 87 :invoke :write [29 35] INFO [2022-01-10 12:53:05,592] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,593] jepsen worker 76 - jepsen.util 76 :ok :read [25 28] INFO [2022-01-10 12:53:05,593] jepsen worker 15 - jepsen.util 15 :ok :write [5 48] INFO [2022-01-10 12:53:05,594] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,595] jepsen worker 76 - jepsen.util 76 :invoke :write [25 31] INFO [2022-01-10 12:53:05,595] jepsen worker 15 - jepsen.util 15 :ok :read [5 48] INFO [2022-01-10 12:53:05,596] jepsen worker 13 - jepsen.util 13 :ok :write [4 16] INFO [2022-01-10 12:53:05,596] jepsen worker 34 - jepsen.util 34 :ok :write [11 20] INFO [2022-01-10 12:53:05,596] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,597] jepsen worker 13 - jepsen.util 13 :invoke :write [4 37] INFO [2022-01-10 12:53:05,597] jepsen worker 15 - jepsen.util 15 :ok :read [5 48] INFO [2022-01-10 12:53:05,598] jepsen worker 34 - jepsen.util 34 :invoke :write [11 48] INFO [2022-01-10 12:53:05,599] jepsen worker 15 - jepsen.util 15 :invoke :write [5 11] INFO [2022-01-10 12:53:05,602] jepsen worker 0 - jepsen.util 0 :ok :write [0 20] INFO [2022-01-10 12:53:05,603] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,603] jepsen worker 69 - jepsen.util 69 :ok :write [23 47] INFO [2022-01-10 12:53:05,604] jepsen worker 69 - jepsen.util 69 :invoke :write [23 27] INFO [2022-01-10 12:53:05,604] jepsen worker 0 - jepsen.util 0 :ok :read [0 20] INFO [2022-01-10 12:53:05,604] jepsen worker 7 - jepsen.util 7 :ok :write [2 30] INFO [2022-01-10 12:53:05,604] jepsen worker 7 - jepsen.util 7 :invoke :write [2 18] INFO [2022-01-10 12:53:05,605] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,606] jepsen worker 0 - jepsen.util 0 :ok :read [0 20] INFO [2022-01-10 12:53:05,606] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,607] jepsen worker 0 - jepsen.util 0 :ok :read [0 20] INFO [2022-01-10 12:53:05,608] jepsen worker 0 - jepsen.util 0 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,609] jepsen worker 0 - jepsen.util 0 :ok :read [0 20] INFO [2022-01-10 12:53:05,609] jepsen worker 0 - jepsen.util 0 :invoke :write [0 30] INFO [2022-01-10 12:53:05,611] jepsen worker 23 - jepsen.util 23 :ok :write [7 45] INFO [2022-01-10 12:53:05,611] jepsen worker 23 - jepsen.util 23 :invoke :write [7 39] INFO [2022-01-10 12:53:05,618] jepsen worker 68 - jepsen.util 68 :ok :write [22 10] INFO [2022-01-10 12:53:05,618] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,619] jepsen worker 68 - jepsen.util 68 :ok :read [22 10] INFO [2022-01-10 12:53:05,620] jepsen worker 68 - jepsen.util 68 :invoke :write [22 19] INFO [2022-01-10 12:53:05,620] jepsen worker 44 - jepsen.util 44 :ok :write [14 25] INFO [2022-01-10 12:53:05,621] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,621] jepsen worker 44 - jepsen.util 44 :ok :read [14 25] INFO [2022-01-10 12:53:05,622] jepsen worker 44 - jepsen.util 44 :invoke :write [14 20] INFO [2022-01-10 12:53:05,625] jepsen worker 60 - jepsen.util 60 :ok :write [20 28] INFO [2022-01-10 12:53:05,625] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,626] jepsen worker 60 - jepsen.util 60 :ok :read [20 28] INFO [2022-01-10 12:53:05,627] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,627] jepsen worker 60 - jepsen.util 60 :ok :read [20 28] INFO [2022-01-10 12:53:05,628] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,629] jepsen worker 60 - jepsen.util 60 :ok :read [20 28] INFO [2022-01-10 12:53:05,629] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,630] jepsen worker 60 - jepsen.util 60 :ok :read [20 28] INFO [2022-01-10 12:53:05,630] jepsen worker 50 - jepsen.util 50 :ok :write [16 4] INFO [2022-01-10 12:53:05,630] jepsen worker 60 - jepsen.util 60 :invoke :write [20 1] INFO [2022-01-10 12:53:05,631] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:05,632] jepsen worker 50 - jepsen.util 50 :ok :read [16 4] INFO [2022-01-10 12:53:05,632] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:05,633] jepsen worker 76 - jepsen.util 76 :ok :write [25 31] INFO [2022-01-10 12:53:05,633] jepsen worker 76 - jepsen.util 76 :invoke :write [25 23] INFO [2022-01-10 12:53:05,633] jepsen worker 50 - jepsen.util 50 :ok :read [16 4] INFO [2022-01-10 12:53:05,634] jepsen worker 50 - jepsen.util 50 :invoke :write [16 38] INFO [2022-01-10 12:53:05,639] jepsen worker 81 - jepsen.util 81 :ok :write [27 27] INFO [2022-01-10 12:53:05,639] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:05,640] jepsen worker 81 - jepsen.util 81 :ok :read [27 27] INFO [2022-01-10 12:53:05,640] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:05,641] jepsen worker 81 - jepsen.util 81 :ok :read [27 27] INFO [2022-01-10 12:53:05,642] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:05,642] jepsen worker 15 - jepsen.util 15 :ok :write [5 11] INFO [2022-01-10 12:53:05,642] jepsen worker 81 - jepsen.util 81 :ok :read [27 27] INFO [2022-01-10 12:53:05,643] jepsen worker 15 - jepsen.util 15 :invoke :write [5 49] INFO [2022-01-10 12:53:05,643] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:05,644] jepsen worker 81 - jepsen.util 81 :ok :read [27 27] INFO [2022-01-10 12:53:05,644] jepsen worker 81 - jepsen.util 81 :invoke :write [27 9] INFO [2022-01-10 12:53:05,645] jepsen worker 33 - jepsen.util 33 :ok :write [11 11] INFO [2022-01-10 12:53:05,646] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,647] jepsen worker 33 - jepsen.util 33 :ok :read [11 11] INFO [2022-01-10 12:53:05,647] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,648] jepsen worker 33 - jepsen.util 33 :ok :read [11 11] INFO [2022-01-10 12:53:05,648] jepsen worker 33 - jepsen.util 33 :invoke :write [11 13] INFO [2022-01-10 12:53:05,650] jepsen worker 7 - jepsen.util 7 :ok :write [2 18] INFO [2022-01-10 12:53:05,651] jepsen worker 7 - jepsen.util 7 :invoke :write [2 15] WARN [2022-01-10 12:53:05,663] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":187118,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000005/0000000090CEECC6","last_local_socket":"172.23.104.79:48368","total_dispatch_duration_us":195582,"total_server_duration_us":186529,"encode_duration_us":72,"operation_id":"0x7b2","timeout_ms":10000,"last_server_duration_us":185749,"total_duration_us":2289335},{"operation_name":"upsert","last_dispatch_duration_us":1460212,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":1460212,"total_server_duration_us":1459113,"encode_duration_us":50,"operation_id":"0x1321","timeout_ms":10000,"last_server_duration_us":1459113,"total_duration_us":1460325},{"operation_name":"upsert","last_dispatch_duration_us":1453206,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":1453206,"total_server_duration_us":1452278,"encode_duration_us":44,"operation_id":"0x1313","timeout_ms":10000,"last_server_duration_us":1452278,"total_duration_us":1453321},{"operation_name":"upsert","last_dispatch_duration_us":1425851,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":1425851,"total_server_duration_us":1425074,"encode_duration_us":52,"operation_id":"0x1330","timeout_ms":10000,"last_server_duration_us":1425074,"total_duration_us":1425974},{"operation_name":"upsert","last_dispatch_duration_us":124319,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":131358,"total_server_duration_us":124549,"encode_duration_us":120,"operation_id":"0x77c","timeout_ms":10000,"last_server_duration_us":123713,"total_duration_us":693469},{"operation_name":"upsert","last_dispatch_duration_us":479907,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":485974,"total_server_duration_us":479958,"encode_duration_us":69,"operation_id":"0xd69","timeout_ms":10000,"last_server_duration_us":479354,"total_duration_us":654509},{"operation_name":"upsert","last_dispatch_duration_us":63930,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000005/0000000090CEECC6","last_local_socket":"172.23.104.79:48368","total_dispatch_duration_us":71562,"total_server_duration_us":64005,"encode_duration_us":52,"operation_id":"0xb47","timeout_ms":10000,"last_server_duration_us":63182,"total_duration_us":626673},{"operation_name":"upsert","last_dispatch_duration_us":513211,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":513211,"total_server_duration_us":512707,"encode_duration_us":43,"operation_id":"0xda7","timeout_ms":10000,"last_server_duration_us":512707,"total_duration_us":513366}],"total_count":8}} INFO [2022-01-10 12:53:05,742] jepsen worker 68 - jepsen.util 68 :ok :write [22 19] INFO [2022-01-10 12:53:05,742] jepsen worker 84 - jepsen.util 84 :ok :write [28 42] INFO [2022-01-10 12:53:05,742] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,743] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,743] jepsen worker 87 - jepsen.util 87 :ok :write [29 35] INFO [2022-01-10 12:53:05,744] jepsen worker 68 - jepsen.util 68 :ok :read [22 19] INFO [2022-01-10 12:53:05,744] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,744] jepsen worker 36 - jepsen.util 36 :ok :write [12 11] INFO [2022-01-10 12:53:05,745] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,745] jepsen worker 87 - jepsen.util 87 :ok :read [29 35] INFO [2022-01-10 12:53:05,745] jepsen worker 84 - jepsen.util 84 :ok :read [28 42] INFO [2022-01-10 12:53:05,745] jepsen worker 36 - jepsen.util 36 :invoke :write [12 43] INFO [2022-01-10 12:53:05,746] jepsen worker 68 - jepsen.util 68 :ok :read [22 19] INFO [2022-01-10 12:53:05,746] jepsen worker 87 - jepsen.util 87 :invoke :write [29 6] INFO [2022-01-10 12:53:05,746] jepsen worker 84 - jepsen.util 84 :invoke :write [28 29] INFO [2022-01-10 12:53:05,747] jepsen worker 50 - jepsen.util 50 :ok :write [16 38] INFO [2022-01-10 12:53:05,747] jepsen worker 68 - jepsen.util 68 :invoke :write [22 23] INFO [2022-01-10 12:53:05,748] jepsen worker 50 - jepsen.util 50 :invoke :write [16 41] INFO [2022-01-10 12:53:05,749] jepsen worker 78 - jepsen.util 78 :ok :write [26 36] INFO [2022-01-10 12:53:05,749] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:05,750] jepsen worker 78 - jepsen.util 78 :ok :read [26 36] INFO [2022-01-10 12:53:05,751] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:05,752] jepsen worker 78 - jepsen.util 78 :ok :read [26 36] INFO [2022-01-10 12:53:05,752] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:05,753] jepsen worker 51 - jepsen.util 51 :ok :write [17 13] INFO [2022-01-10 12:53:05,754] jepsen worker 78 - jepsen.util 78 :ok :read [26 36] INFO [2022-01-10 12:53:05,754] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:05,754] jepsen worker 0 - jepsen.util 0 :ok :write [0 30] INFO [2022-01-10 12:53:05,754] jepsen worker 78 - jepsen.util 78 :invoke :write [26 37] INFO [2022-01-10 12:53:05,755] jepsen worker 0 - jepsen.util 0 :invoke :write [0 8] INFO [2022-01-10 12:53:05,755] jepsen worker 51 - jepsen.util 51 :ok :read [17 13] INFO [2022-01-10 12:53:05,756] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:05,757] jepsen worker 51 - jepsen.util 51 :ok :read [17 13] INFO [2022-01-10 12:53:05,757] jepsen worker 51 - jepsen.util 51 :invoke :write [17 21] INFO [2022-01-10 12:53:05,758] jepsen worker 23 - jepsen.util 23 :ok :write [7 39] INFO [2022-01-10 12:53:05,758] jepsen worker 23 - jepsen.util 23 :invoke :write [7 34] INFO [2022-01-10 12:53:05,761] jepsen worker 81 - jepsen.util 81 :ok :write [27 9] INFO [2022-01-10 12:53:05,761] jepsen worker 81 - jepsen.util 81 :invoke :write [27 17] INFO [2022-01-10 12:53:05,762] jepsen worker 44 - jepsen.util 44 :ok :write [14 20] INFO [2022-01-10 12:53:05,762] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,763] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:05,764] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,765] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:05,765] jepsen worker 44 - jepsen.util 44 :invoke :write [14 36] INFO [2022-01-10 12:53:05,767] jepsen worker 3 - jepsen.util 3 :ok :write [1 37] INFO [2022-01-10 12:53:05,767] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,767] jepsen worker 60 - jepsen.util 60 :ok :write [20 1] INFO [2022-01-10 12:53:05,768] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,769] jepsen worker 60 - jepsen.util 60 :ok :read [20 1] INFO [2022-01-10 12:53:05,769] jepsen worker 3 - jepsen.util 3 :ok :read [1 37] INFO [2022-01-10 12:53:05,769] jepsen worker 15 - jepsen.util 15 :ok :write [5 49] INFO [2022-01-10 12:53:05,769] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,770] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,770] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,771] jepsen worker 3 - jepsen.util 3 :ok :read [1 37] INFO [2022-01-10 12:53:05,771] jepsen worker 15 - jepsen.util 15 :ok :read [5 49] INFO [2022-01-10 12:53:05,771] jepsen worker 13 - jepsen.util 13 :ok :write [4 37] INFO [2022-01-10 12:53:05,771] jepsen worker 60 - jepsen.util 60 :ok :read [20 1] INFO [2022-01-10 12:53:05,771] jepsen worker 15 - jepsen.util 15 :invoke :write [5 40] INFO [2022-01-10 12:53:05,772] jepsen worker 13 - jepsen.util 13 :invoke :write [4 45] INFO [2022-01-10 12:53:05,772] jepsen worker 3 - jepsen.util 3 :invoke :write [1 35] INFO [2022-01-10 12:53:05,773] jepsen worker 60 - jepsen.util 60 :invoke :write [20 2] INFO [2022-01-10 12:53:05,775] jepsen worker 34 - jepsen.util 34 :ok :write [11 48] INFO [2022-01-10 12:53:05,776] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,777] jepsen worker 34 - jepsen.util 34 :ok :read [11 48] INFO [2022-01-10 12:53:05,778] jepsen worker 34 - jepsen.util 34 :invoke :write [11 2] INFO [2022-01-10 12:53:05,778] jepsen worker 76 - jepsen.util 76 :ok :write [25 23] INFO [2022-01-10 12:53:05,778] jepsen worker 69 - jepsen.util 69 :ok :write [23 27] INFO [2022-01-10 12:53:05,778] jepsen worker 76 - jepsen.util 76 :invoke :write [25 7] INFO [2022-01-10 12:53:05,779] jepsen worker 7 - jepsen.util 7 :ok :write [2 15] INFO [2022-01-10 12:53:05,779] jepsen worker 69 - jepsen.util 69 :invoke :write [23 23] INFO [2022-01-10 12:53:05,779] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:05,780] jepsen worker 7 - jepsen.util 7 :ok :read [2 15] INFO [2022-01-10 12:53:05,781] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:05,781] jepsen worker 36 - jepsen.util 36 :ok :write [12 43] INFO [2022-01-10 12:53:05,782] jepsen worker 7 - jepsen.util 7 :ok :read [2 15] INFO [2022-01-10 12:53:05,782] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,782] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:05,783] jepsen worker 68 - jepsen.util 68 :ok :write [22 23] INFO [2022-01-10 12:53:05,783] jepsen worker 36 - jepsen.util 36 :ok :read [12 43] INFO [2022-01-10 12:53:05,783] jepsen worker 7 - jepsen.util 7 :ok :read [2 15] INFO [2022-01-10 12:53:05,784] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,784] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,785] jepsen worker 68 - jepsen.util 68 :ok :read [22 23] INFO [2022-01-10 12:53:05,785] jepsen worker 7 - jepsen.util 7 :invoke :write [2 7] INFO [2022-01-10 12:53:05,785] jepsen worker 36 - jepsen.util 36 :ok :read [12 43] INFO [2022-01-10 12:53:05,786] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,786] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,787] jepsen worker 68 - jepsen.util 68 :ok :read [22 23] INFO [2022-01-10 12:53:05,787] jepsen worker 36 - jepsen.util 36 :ok :read [12 43] INFO [2022-01-10 12:53:05,787] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,788] jepsen worker 36 - jepsen.util 36 :invoke :write [12 6] INFO [2022-01-10 12:53:05,789] jepsen worker 84 - jepsen.util 84 :ok :write [28 29] INFO [2022-01-10 12:53:05,789] jepsen worker 68 - jepsen.util 68 :ok :read [22 23] INFO [2022-01-10 12:53:05,789] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,790] jepsen worker 78 - jepsen.util 78 :ok :write [26 37] INFO [2022-01-10 12:53:05,790] jepsen worker 84 - jepsen.util 84 :ok :read [28 29] INFO [2022-01-10 12:53:05,791] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,791] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:05,792] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,792] jepsen worker 78 - jepsen.util 78 :ok :read [26 37] INFO [2022-01-10 12:53:05,792] jepsen worker 68 - jepsen.util 68 :ok :read [22 23] INFO [2022-01-10 12:53:05,793] jepsen worker 84 - jepsen.util 84 :ok :read [28 29] INFO [2022-01-10 12:53:05,793] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:05,794] jepsen worker 87 - jepsen.util 87 :ok :write [29 6] INFO [2022-01-10 12:53:05,794] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,794] jepsen worker 78 - jepsen.util 78 :ok :read [26 37] INFO [2022-01-10 12:53:05,795] jepsen worker 51 - jepsen.util 51 :ok :write [17 21] INFO [2022-01-10 12:53:05,795] jepsen worker 84 - jepsen.util 84 :invoke :write [28 6] INFO [2022-01-10 12:53:05,795] jepsen worker 68 - jepsen.util 68 :ok :read [22 23] INFO [2022-01-10 12:53:05,796] jepsen worker 68 - jepsen.util 68 :invoke :write [22 38] INFO [2022-01-10 12:53:05,797] jepsen worker 51 - jepsen.util 51 :invoke :write [17 49] INFO [2022-01-10 12:53:05,798] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,798] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:05,799] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:05,799] jepsen worker 78 - jepsen.util 78 :ok :read [26 37] INFO [2022-01-10 12:53:05,800] jepsen worker 3 - jepsen.util 3 :ok :write [1 35] INFO [2022-01-10 12:53:05,800] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,801] jepsen worker 78 - jepsen.util 78 :invoke :write [26 23] INFO [2022-01-10 12:53:05,801] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:05,802] jepsen worker 50 - jepsen.util 50 :ok :write [16 41] INFO [2022-01-10 12:53:05,802] jepsen worker 87 - jepsen.util 87 :invoke :write [29 41] INFO [2022-01-10 12:53:05,803] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,803] jepsen worker 50 - jepsen.util 50 :invoke :write [16 15] INFO [2022-01-10 12:53:05,804] jepsen worker 3 - jepsen.util 3 :ok :read [1 35] INFO [2022-01-10 12:53:05,804] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,805] jepsen worker 13 - jepsen.util 13 :ok :write [4 45] INFO [2022-01-10 12:53:05,805] jepsen worker 3 - jepsen.util 3 :ok :read [1 35] INFO [2022-01-10 12:53:05,806] jepsen worker 13 - jepsen.util 13 :invoke :write [4 36] INFO [2022-01-10 12:53:05,806] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,808] jepsen worker 3 - jepsen.util 3 :ok :read [1 35] INFO [2022-01-10 12:53:05,808] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,809] jepsen worker 0 - jepsen.util 0 :ok :write [0 8] INFO [2022-01-10 12:53:05,809] jepsen worker 3 - jepsen.util 3 :ok :read [1 35] INFO [2022-01-10 12:53:05,809] jepsen worker 0 - jepsen.util 0 :invoke :write [0 2] INFO [2022-01-10 12:53:05,810] jepsen worker 3 - jepsen.util 3 :invoke :write [1 35] INFO [2022-01-10 12:53:05,810] jepsen worker 23 - jepsen.util 23 :ok :write [7 34] INFO [2022-01-10 12:53:05,810] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:05,811] jepsen worker 23 - jepsen.util 23 :ok :read [7 34] INFO [2022-01-10 12:53:05,812] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:05,812] jepsen worker 81 - jepsen.util 81 :ok :write [27 17] INFO [2022-01-10 12:53:05,812] jepsen worker 23 - jepsen.util 23 :ok :read [7 34] INFO [2022-01-10 12:53:05,813] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:05,813] jepsen worker 23 - jepsen.util 23 :invoke :write [7 19] INFO [2022-01-10 12:53:05,814] jepsen worker 69 - jepsen.util 69 :ok :write [23 23] INFO [2022-01-10 12:53:05,814] jepsen worker 81 - jepsen.util 81 :ok :read [27 17] INFO [2022-01-10 12:53:05,814] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,815] jepsen worker 81 - jepsen.util 81 :invoke :write [27 3] INFO [2022-01-10 12:53:05,815] jepsen worker 69 - jepsen.util 69 :ok :read [23 23] INFO [2022-01-10 12:53:05,816] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,817] jepsen worker 69 - jepsen.util 69 :ok :read [23 23] INFO [2022-01-10 12:53:05,817] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,818] jepsen worker 15 - jepsen.util 15 :ok :write [5 40] INFO [2022-01-10 12:53:05,818] jepsen worker 69 - jepsen.util 69 :ok :read [23 23] INFO [2022-01-10 12:53:05,818] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,819] jepsen worker 15 - jepsen.util 15 :invoke :write [5 47] INFO [2022-01-10 12:53:05,819] jepsen worker 69 - jepsen.util 69 :ok :read [23 23] INFO [2022-01-10 12:53:05,820] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,821] jepsen worker 69 - jepsen.util 69 :ok :read [23 23] INFO [2022-01-10 12:53:05,821] jepsen worker 44 - jepsen.util 44 :ok :write [14 36] INFO [2022-01-10 12:53:05,821] jepsen worker 69 - jepsen.util 69 :invoke :write [23 35] INFO [2022-01-10 12:53:05,822] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,823] jepsen worker 44 - jepsen.util 44 :ok :read [14 36] INFO [2022-01-10 12:53:05,823] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,824] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:05,825] jepsen worker 38 - jepsen.util 38 :invoke :write [12 41] INFO [2022-01-10 12:53:05,825] jepsen worker 34 - jepsen.util 34 :ok :write [11 2] INFO [2022-01-10 12:53:05,826] jepsen worker 44 - jepsen.util 44 :ok :read [14 36] INFO [2022-01-10 12:53:05,826] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:05,827] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:05,827] jepsen worker 34 - jepsen.util 34 :ok :read [11 2] INFO [2022-01-10 12:53:05,828] jepsen worker 44 - jepsen.util 44 :ok :read [14 36] INFO [2022-01-10 12:53:05,828] jepsen worker 34 - jepsen.util 34 :invoke :write [11 22] INFO [2022-01-10 12:53:05,828] jepsen worker 44 - jepsen.util 44 :invoke :write [14 31] INFO [2022-01-10 12:53:05,830] jepsen worker 60 - jepsen.util 60 :ok :write [20 2] INFO [2022-01-10 12:53:05,831] jepsen worker 60 - jepsen.util 60 :invoke :write [20 0] INFO [2022-01-10 12:53:05,834] jepsen worker 7 - jepsen.util 7 :ok :write [2 7] INFO [2022-01-10 12:53:05,834] jepsen worker 7 - jepsen.util 7 :invoke :write [2 46] INFO [2022-01-10 12:53:05,836] jepsen worker 84 - jepsen.util 84 :ok :write [28 6] INFO [2022-01-10 12:53:05,836] jepsen worker 84 - jepsen.util 84 :invoke :write [28 6] INFO [2022-01-10 12:53:05,838] jepsen worker 75 - jepsen.util 75 :ok :write [25 28] INFO [2022-01-10 12:53:05,839] jepsen worker 75 - jepsen.util 75 :invoke :write [25 18] INFO [2022-01-10 12:53:05,840] jepsen worker 68 - jepsen.util 68 :ok :write [22 38] INFO [2022-01-10 12:53:05,840] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,841] jepsen worker 68 - jepsen.util 68 :ok :read [22 38] INFO [2022-01-10 12:53:05,842] jepsen worker 78 - jepsen.util 78 :ok :write [26 23] INFO [2022-01-10 12:53:05,842] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:05,842] jepsen worker 78 - jepsen.util 78 :invoke :write [26 7] INFO [2022-01-10 12:53:05,843] jepsen worker 68 - jepsen.util 68 :ok :read [22 38] INFO [2022-01-10 12:53:05,844] jepsen worker 68 - jepsen.util 68 :invoke :write [22 49] INFO [2022-01-10 12:53:05,844] jepsen worker 87 - jepsen.util 87 :ok :write [29 41] INFO [2022-01-10 12:53:05,845] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,845] jepsen worker 87 - jepsen.util 87 :ok :read [29 41] INFO [2022-01-10 12:53:05,846] jepsen worker 87 - jepsen.util 87 :invoke :write [29 19] INFO [2022-01-10 12:53:05,847] jepsen worker 50 - jepsen.util 50 :ok :write [16 15] INFO [2022-01-10 12:53:05,848] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:05,849] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:05,850] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:05,850] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:05,851] jepsen worker 50 - jepsen.util 50 :invoke :write [16 20] INFO [2022-01-10 12:53:05,852] jepsen worker 39 - jepsen.util 39 :ok :write [13 0] INFO [2022-01-10 12:53:05,852] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:05,852] jepsen worker 51 - jepsen.util 51 :ok :write [17 49] INFO [2022-01-10 12:53:05,853] jepsen worker 51 - jepsen.util 51 :invoke :write [17 23] INFO [2022-01-10 12:53:05,853] jepsen worker 39 - jepsen.util 39 :ok :read [13 0] INFO [2022-01-10 12:53:05,854] jepsen worker 39 - jepsen.util 39 :invoke :write [13 25] INFO [2022-01-10 12:53:05,857] jepsen worker 81 - jepsen.util 81 :ok :write [27 3] INFO [2022-01-10 12:53:05,858] jepsen worker 81 - jepsen.util 81 :invoke :write [27 37] INFO [2022-01-10 12:53:05,859] jepsen worker 3 - jepsen.util 3 :ok :write [1 35] INFO [2022-01-10 12:53:05,860] jepsen worker 3 - jepsen.util 3 :invoke :write [1 41] INFO [2022-01-10 12:53:05,861] jepsen worker 0 - jepsen.util 0 :ok :write [0 2] INFO [2022-01-10 12:53:05,861] jepsen worker 15 - jepsen.util 15 :ok :write [5 47] INFO [2022-01-10 12:53:05,862] jepsen worker 0 - jepsen.util 0 :invoke :write [0 23] INFO [2022-01-10 12:53:05,863] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,864] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:05,864] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:05,865] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:05,866] jepsen worker 15 - jepsen.util 15 :invoke :write [5 7] INFO [2022-01-10 12:53:05,869] jepsen worker 13 - jepsen.util 13 :ok :write [4 36] INFO [2022-01-10 12:53:05,869] jepsen worker 13 - jepsen.util 13 :invoke :write [4 19] INFO [2022-01-10 12:53:05,870] jepsen worker 23 - jepsen.util 23 :ok :write [7 19] INFO [2022-01-10 12:53:05,870] jepsen worker 33 - jepsen.util 33 :ok :write [11 13] INFO [2022-01-10 12:53:05,871] jepsen worker 23 - jepsen.util 23 :invoke :write [7 22] INFO [2022-01-10 12:53:05,871] jepsen worker 33 - jepsen.util 33 :invoke :write [11 20] INFO [2022-01-10 12:53:05,875] jepsen worker 44 - jepsen.util 44 :ok :write [14 31] INFO [2022-01-10 12:53:05,876] jepsen worker 69 - jepsen.util 69 :ok :write [23 35] INFO [2022-01-10 12:53:05,876] jepsen worker 44 - jepsen.util 44 :invoke :write [14 45] INFO [2022-01-10 12:53:05,877] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,878] jepsen worker 69 - jepsen.util 69 :ok :read [23 35] INFO [2022-01-10 12:53:05,878] jepsen worker 69 - jepsen.util 69 :invoke :write [23 31] INFO [2022-01-10 12:53:05,883] jepsen worker 60 - jepsen.util 60 :ok :write [20 0] INFO [2022-01-10 12:53:05,883] jepsen worker 7 - jepsen.util 7 :ok :write [2 46] INFO [2022-01-10 12:53:05,883] jepsen worker 38 - jepsen.util 38 :ok :write [12 41] INFO [2022-01-10 12:53:05,884] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:05,884] jepsen worker 7 - jepsen.util 7 :invoke :write [2 46] INFO [2022-01-10 12:53:05,885] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,885] jepsen worker 60 - jepsen.util 60 :ok :read [20 0] INFO [2022-01-10 12:53:05,885] jepsen worker 60 - jepsen.util 60 :invoke :write [20 11] INFO [2022-01-10 12:53:05,886] jepsen worker 38 - jepsen.util 38 :ok :read [12 41] INFO [2022-01-10 12:53:05,886] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,887] jepsen worker 38 - jepsen.util 38 :ok :read [12 41] INFO [2022-01-10 12:53:05,888] jepsen worker 38 - jepsen.util 38 :invoke :write [12 31] INFO [2022-01-10 12:53:05,890] jepsen worker 84 - jepsen.util 84 :ok :write [28 6] INFO [2022-01-10 12:53:05,891] jepsen worker 84 - jepsen.util 84 :invoke :write [28 0] INFO [2022-01-10 12:53:05,897] jepsen worker 87 - jepsen.util 87 :ok :write [29 19] INFO [2022-01-10 12:53:05,898] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,899] jepsen worker 87 - jepsen.util 87 :ok :read [29 19] INFO [2022-01-10 12:53:05,899] jepsen worker 78 - jepsen.util 78 :ok :write [26 7] INFO [2022-01-10 12:53:05,899] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,900] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:05,901] jepsen worker 87 - jepsen.util 87 :ok :read [29 19] INFO [2022-01-10 12:53:05,901] jepsen worker 78 - jepsen.util 78 :ok :read [26 7] INFO [2022-01-10 12:53:05,901] jepsen worker 87 - jepsen.util 87 :invoke :write [29 43] INFO [2022-01-10 12:53:05,902] jepsen worker 78 - jepsen.util 78 :invoke :write [26 25] INFO [2022-01-10 12:53:05,905] jepsen worker 51 - jepsen.util 51 :ok :write [17 23] INFO [2022-01-10 12:53:05,906] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:05,907] jepsen worker 51 - jepsen.util 51 :ok :read [17 23] INFO [2022-01-10 12:53:05,907] jepsen worker 39 - jepsen.util 39 :ok :write [13 25] INFO [2022-01-10 12:53:05,908] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:05,908] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:05,909] jepsen worker 51 - jepsen.util 51 :ok :read [17 23] INFO [2022-01-10 12:53:05,909] jepsen worker 39 - jepsen.util 39 :ok :read [13 25] INFO [2022-01-10 12:53:05,909] jepsen worker 51 - jepsen.util 51 :invoke :write [17 44] INFO [2022-01-10 12:53:05,910] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:05,911] jepsen worker 39 - jepsen.util 39 :ok :read [13 25] INFO [2022-01-10 12:53:05,911] jepsen worker 3 - jepsen.util 3 :ok :write [1 41] INFO [2022-01-10 12:53:05,911] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:05,912] jepsen worker 3 - jepsen.util 3 :invoke :write [1 26] INFO [2022-01-10 12:53:05,912] jepsen worker 1 - jepsen.util 1 :ok :write [0 1] INFO [2022-01-10 12:53:05,912] jepsen worker 39 - jepsen.util 39 :ok :read [13 25] INFO [2022-01-10 12:53:05,913] jepsen worker 1 - jepsen.util 1 :invoke :write [0 39] INFO [2022-01-10 12:53:05,913] jepsen worker 39 - jepsen.util 39 :invoke :write [13 48] INFO [2022-01-10 12:53:05,916] jepsen worker 13 - jepsen.util 13 :ok :write [4 19] INFO [2022-01-10 12:53:05,917] jepsen worker 13 - jepsen.util 13 :invoke :write [4 11] INFO [2022-01-10 12:53:05,925] jepsen worker 38 - jepsen.util 38 :ok :write [12 31] INFO [2022-01-10 12:53:05,925] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,926] jepsen worker 38 - jepsen.util 38 :ok :read [12 31] INFO [2022-01-10 12:53:05,927] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:05,928] jepsen worker 38 - jepsen.util 38 :ok :read [12 31] INFO [2022-01-10 12:53:05,928] jepsen worker 38 - jepsen.util 38 :invoke :write [12 31] INFO [2022-01-10 12:53:05,931] jepsen worker 84 - jepsen.util 84 :ok :write [28 0] INFO [2022-01-10 12:53:05,932] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,933] jepsen worker 84 - jepsen.util 84 :ok :read [28 0] INFO [2022-01-10 12:53:05,933] jepsen worker 87 - jepsen.util 87 :ok :write [29 43] INFO [2022-01-10 12:53:05,934] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,934] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,935] jepsen worker 39 - jepsen.util 39 :ok :write [13 48] INFO [2022-01-10 12:53:05,935] jepsen worker 84 - jepsen.util 84 :ok :read [28 0] INFO [2022-01-10 12:53:05,935] jepsen worker 87 - jepsen.util 87 :ok :read [29 43] INFO [2022-01-10 12:53:05,936] jepsen worker 84 - jepsen.util 84 :invoke :write [28 6] INFO [2022-01-10 12:53:05,936] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,937] jepsen worker 39 - jepsen.util 39 :invoke :write [13 28] INFO [2022-01-10 12:53:05,937] jepsen worker 87 - jepsen.util 87 :ok :read [29 43] INFO [2022-01-10 12:53:05,938] jepsen worker 87 - jepsen.util 87 :invoke :write [29 14] INFO [2022-01-10 12:53:05,939] jepsen worker 1 - jepsen.util 1 :ok :write [0 39] INFO [2022-01-10 12:53:05,940] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,941] jepsen worker 1 - jepsen.util 1 :ok :read [0 39] INFO [2022-01-10 12:53:05,942] jepsen worker 1 - jepsen.util 1 :invoke :write [0 3] INFO [2022-01-10 12:53:05,942] jepsen worker 3 - jepsen.util 3 :ok :write [1 26] INFO [2022-01-10 12:53:05,943] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,944] jepsen worker 3 - jepsen.util 3 :ok :read [1 26] INFO [2022-01-10 12:53:05,945] jepsen worker 3 - jepsen.util 3 :invoke :write [1 47] INFO [2022-01-10 12:53:05,949] jepsen worker 39 - jepsen.util 39 :ok :write [13 28] INFO [2022-01-10 12:53:05,949] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:05,950] jepsen worker 39 - jepsen.util 39 :ok :read [13 28] INFO [2022-01-10 12:53:05,950] jepsen worker 39 - jepsen.util 39 :invoke :write [13 45] INFO [2022-01-10 12:53:05,953] jepsen worker 38 - jepsen.util 38 :ok :write [12 31] INFO [2022-01-10 12:53:05,954] jepsen worker 38 - jepsen.util 38 :invoke :write [12 23] INFO [2022-01-10 12:53:05,958] jepsen worker 1 - jepsen.util 1 :ok :write [0 3] INFO [2022-01-10 12:53:05,958] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,959] jepsen worker 1 - jepsen.util 1 :ok :read [0 3] INFO [2022-01-10 12:53:05,960] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,961] jepsen worker 1 - jepsen.util 1 :ok :read [0 3] INFO [2022-01-10 12:53:05,961] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,961] jepsen worker 87 - jepsen.util 87 :ok :write [29 14] INFO [2022-01-10 12:53:05,962] jepsen worker 1 - jepsen.util 1 :ok :read [0 3] INFO [2022-01-10 12:53:05,962] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,962] jepsen worker 84 - jepsen.util 84 :ok :write [28 6] INFO [2022-01-10 12:53:05,962] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,963] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:05,963] jepsen worker 87 - jepsen.util 87 :ok :read [29 14] INFO [2022-01-10 12:53:05,963] jepsen worker 1 - jepsen.util 1 :ok :read [0 3] INFO [2022-01-10 12:53:05,964] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:05,964] jepsen worker 84 - jepsen.util 84 :ok :read [28 6] INFO [2022-01-10 12:53:05,964] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,964] jepsen worker 84 - jepsen.util 84 :invoke :write [28 9] INFO [2022-01-10 12:53:05,965] jepsen worker 1 - jepsen.util 1 :ok :read [0 3] INFO [2022-01-10 12:53:05,965] jepsen worker 3 - jepsen.util 3 :ok :write [1 47] INFO [2022-01-10 12:53:05,965] jepsen worker 87 - jepsen.util 87 :ok :read [29 14] INFO [2022-01-10 12:53:05,965] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,966] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,966] jepsen worker 1 - jepsen.util 1 :ok :read [0 3] INFO [2022-01-10 12:53:05,967] jepsen worker 3 - jepsen.util 3 :ok :read [1 47] INFO [2022-01-10 12:53:05,967] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:05,967] jepsen worker 87 - jepsen.util 87 :invoke :write [29 7] INFO [2022-01-10 12:53:05,967] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:05,968] jepsen worker 1 - jepsen.util 1 :ok :read [0 3] INFO [2022-01-10 12:53:05,968] jepsen worker 3 - jepsen.util 3 :ok :read [1 47] INFO [2022-01-10 12:53:05,968] jepsen worker 1 - jepsen.util 1 :invoke :write [0 1] INFO [2022-01-10 12:53:05,969] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 12:53:05,978] jepsen worker 69 - jepsen.util 69 :ok :write [23 31] INFO [2022-01-10 12:53:05,978] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:05,979] jepsen worker 69 - jepsen.util 69 :ok :read [23 31] INFO [2022-01-10 12:53:05,980] jepsen worker 75 - jepsen.util 75 :ok :write [25 18] INFO [2022-01-10 12:53:05,980] jepsen worker 69 - jepsen.util 69 :invoke :write [23 24] INFO [2022-01-10 12:53:05,980] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,981] jepsen worker 75 - jepsen.util 75 :ok :read [25 18] INFO [2022-01-10 12:53:05,981] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,982] jepsen worker 78 - jepsen.util 78 :ok :write [26 25] INFO [2022-01-10 12:53:05,982] jepsen worker 75 - jepsen.util 75 :ok :read [25 18] INFO [2022-01-10 12:53:05,982] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:05,983] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,983] jepsen worker 78 - jepsen.util 78 :ok :read [26 25] INFO [2022-01-10 12:53:05,984] jepsen worker 75 - jepsen.util 75 :ok :read [25 18] INFO [2022-01-10 12:53:05,984] jepsen worker 78 - jepsen.util 78 :invoke :write [26 31] INFO [2022-01-10 12:53:05,984] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,985] jepsen worker 75 - jepsen.util 75 :ok :read [25 18] INFO [2022-01-10 12:53:05,986] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:05,986] jepsen worker 68 - jepsen.util 68 :ok :write [22 49] INFO [2022-01-10 12:53:05,987] jepsen worker 68 - jepsen.util 68 :invoke :write [22 9] INFO [2022-01-10 12:53:05,987] jepsen worker 75 - jepsen.util 75 :ok :read [25 18] INFO [2022-01-10 12:53:05,988] jepsen worker 75 - jepsen.util 75 :invoke :write [25 41] INFO [2022-01-10 12:53:05,988] jepsen worker 51 - jepsen.util 51 :ok :write [17 44] INFO [2022-01-10 12:53:05,988] jepsen worker 51 - jepsen.util 51 :invoke :write [17 1] INFO [2022-01-10 12:53:05,992] jepsen worker 50 - jepsen.util 50 :ok :write [16 20] INFO [2022-01-10 12:53:05,992] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:05,993] jepsen worker 13 - jepsen.util 13 :ok :write [4 11] INFO [2022-01-10 12:53:05,993] jepsen worker 50 - jepsen.util 50 :ok :read [16 20] INFO [2022-01-10 12:53:05,994] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:05,994] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:05,994] jepsen worker 13 - jepsen.util 13 :ok :read [4 11] INFO [2022-01-10 12:53:05,995] jepsen worker 50 - jepsen.util 50 :ok :read [16 20] INFO [2022-01-10 12:53:05,995] jepsen worker 13 - jepsen.util 13 :invoke :write [4 22] INFO [2022-01-10 12:53:05,995] jepsen worker 50 - jepsen.util 50 :invoke :write [16 0] INFO [2022-01-10 12:53:05,998] jepsen worker 81 - jepsen.util 81 :ok :write [27 37] INFO [2022-01-10 12:53:05,998] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:05,998] jepsen worker 69 - jepsen.util 69 :ok :write [23 24] INFO [2022-01-10 12:53:05,999] jepsen worker 69 - jepsen.util 69 :invoke :write [23 36] INFO [2022-01-10 12:53:05,999] jepsen worker 81 - jepsen.util 81 :ok :read [27 37] INFO [2022-01-10 12:53:06,000] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,001] jepsen worker 81 - jepsen.util 81 :ok :read [27 37] INFO [2022-01-10 12:53:06,001] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,002] jepsen worker 81 - jepsen.util 81 :ok :read [27 37] INFO [2022-01-10 12:53:06,002] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,003] jepsen worker 15 - jepsen.util 15 :ok :write [5 7] INFO [2022-01-10 12:53:06,003] jepsen worker 81 - jepsen.util 81 :ok :read [27 37] INFO [2022-01-10 12:53:06,003] jepsen worker 15 - jepsen.util 15 :invoke :write [5 4] INFO [2022-01-10 12:53:06,003] jepsen worker 78 - jepsen.util 78 :ok :write [26 31] INFO [2022-01-10 12:53:06,004] jepsen worker 81 - jepsen.util 81 :invoke :write [27 17] INFO [2022-01-10 12:53:06,004] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:06,005] jepsen worker 78 - jepsen.util 78 :ok :read [26 31] INFO [2022-01-10 12:53:06,005] jepsen worker 78 - jepsen.util 78 :invoke :write [26 11] INFO [2022-01-10 12:53:06,009] jepsen worker 23 - jepsen.util 23 :ok :write [7 22] INFO [2022-01-10 12:53:06,010] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,011] jepsen worker 23 - jepsen.util 23 :ok :read [7 22] INFO [2022-01-10 12:53:06,011] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,012] jepsen worker 23 - jepsen.util 23 :ok :read [7 22] INFO [2022-01-10 12:53:06,013] jepsen worker 23 - jepsen.util 23 :invoke :write [7 24] INFO [2022-01-10 12:53:06,014] jepsen worker 51 - jepsen.util 51 :ok :write [17 1] INFO [2022-01-10 12:53:06,015] jepsen worker 51 - jepsen.util 51 :invoke :write [17 0] INFO [2022-01-10 12:53:06,018] jepsen worker 33 - jepsen.util 33 :ok :write [11 20] INFO [2022-01-10 12:53:06,018] jepsen worker 33 - jepsen.util 33 :invoke :write [11 6] INFO [2022-01-10 12:53:06,024] jepsen worker 44 - jepsen.util 44 :ok :write [14 45] INFO [2022-01-10 12:53:06,025] jepsen worker 44 - jepsen.util 44 :invoke :write [14 43] INFO [2022-01-10 12:53:06,030] jepsen worker 60 - jepsen.util 60 :ok :write [20 11] INFO [2022-01-10 12:53:06,030] jepsen worker 60 - jepsen.util 60 :invoke :write [20 47] INFO [2022-01-10 12:53:06,036] jepsen worker 7 - jepsen.util 7 :ok :write [2 46] INFO [2022-01-10 12:53:06,037] jepsen worker 7 - jepsen.util 7 :invoke :write [2 47] INFO [2022-01-10 12:53:06,040] jepsen worker 75 - jepsen.util 75 :ok :write [25 41] INFO [2022-01-10 12:53:06,041] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:06,042] jepsen worker 75 - jepsen.util 75 :ok :read [25 41] INFO [2022-01-10 12:53:06,042] jepsen worker 75 - jepsen.util 75 :invoke :write [25 5] INFO [2022-01-10 12:53:06,057] jepsen worker 39 - jepsen.util 39 :ok :write [13 45] INFO [2022-01-10 12:53:06,057] jepsen worker 39 - jepsen.util 39 :invoke :write [13 21] INFO [2022-01-10 12:53:06,061] jepsen worker 38 - jepsen.util 38 :ok :write [12 23] INFO [2022-01-10 12:53:06,061] jepsen worker 38 - jepsen.util 38 :invoke :write [12 6] INFO [2022-01-10 12:53:06,063] jepsen worker 1 - jepsen.util 1 :ok :write [0 1] INFO [2022-01-10 12:53:06,064] jepsen worker 1 - jepsen.util 1 :invoke :write [0 9] INFO [2022-01-10 12:53:06,066] jepsen worker 84 - jepsen.util 84 :ok :write [28 9] INFO [2022-01-10 12:53:06,067] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:06,068] jepsen worker 84 - jepsen.util 84 :ok :read [28 9] INFO [2022-01-10 12:53:06,068] jepsen worker 87 - jepsen.util 87 :ok :write [29 7] INFO [2022-01-10 12:53:06,069] jepsen worker 84 - jepsen.util 84 :invoke :write [28 7] INFO [2022-01-10 12:53:06,069] jepsen worker 87 - jepsen.util 87 :invoke :write [29 4] INFO [2022-01-10 12:53:06,073] jepsen worker 68 - jepsen.util 68 :ok :write [22 9] INFO [2022-01-10 12:53:06,073] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,074] jepsen worker 68 - jepsen.util 68 :ok :read [22 9] INFO [2022-01-10 12:53:06,075] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,076] jepsen worker 68 - jepsen.util 68 :ok :read [22 9] INFO [2022-01-10 12:53:06,076] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,077] jepsen worker 3 - jepsen.util 3 :ok :write [1 40] INFO [2022-01-10 12:53:06,077] jepsen worker 50 - jepsen.util 50 :ok :write [16 0] INFO [2022-01-10 12:53:06,078] jepsen worker 68 - jepsen.util 68 :ok :read [22 9] INFO [2022-01-10 12:53:06,078] jepsen worker 3 - jepsen.util 3 :invoke :write [1 14] INFO [2022-01-10 12:53:06,078] jepsen worker 68 - jepsen.util 68 :invoke :write [22 35] INFO [2022-01-10 12:53:06,079] jepsen worker 50 - jepsen.util 50 :invoke :write [16 46] INFO [2022-01-10 12:53:06,083] jepsen worker 81 - jepsen.util 81 :ok :write [27 17] INFO [2022-01-10 12:53:06,084] jepsen worker 81 - jepsen.util 81 :invoke :write [27 7] INFO [2022-01-10 12:53:06,088] jepsen worker 15 - jepsen.util 15 :ok :write [5 4] INFO [2022-01-10 12:53:06,088] jepsen worker 15 - jepsen.util 15 :invoke :write [5 47] INFO [2022-01-10 12:53:06,095] jepsen worker 33 - jepsen.util 33 :ok :write [11 6] INFO [2022-01-10 12:53:06,096] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:06,096] jepsen worker 33 - jepsen.util 33 :ok :read [11 6] INFO [2022-01-10 12:53:06,097] jepsen worker 33 - jepsen.util 33 :invoke :write [11 9] INFO [2022-01-10 12:53:06,100] jepsen worker 7 - jepsen.util 7 :ok :write [2 47] INFO [2022-01-10 12:53:06,101] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:06,102] jepsen worker 7 - jepsen.util 7 :ok :read [2 47] INFO [2022-01-10 12:53:06,103] jepsen worker 7 - jepsen.util 7 :invoke :write [2 35] INFO [2022-01-10 12:53:06,121] jepsen worker 68 - jepsen.util 68 :ok :write [22 35] INFO [2022-01-10 12:53:06,121] jepsen worker 68 - jepsen.util 68 :invoke :write [22 3] INFO [2022-01-10 12:53:06,126] jepsen worker 50 - jepsen.util 50 :ok :write [16 46] INFO [2022-01-10 12:53:06,126] jepsen worker 50 - jepsen.util 50 :invoke :write [16 25] INFO [2022-01-10 12:53:06,132] jepsen worker 81 - jepsen.util 81 :ok :write [27 7] INFO [2022-01-10 12:53:06,133] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,134] jepsen worker 81 - jepsen.util 81 :ok :read [27 7] INFO [2022-01-10 12:53:06,135] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,135] jepsen worker 81 - jepsen.util 81 :ok :read [27 7] INFO [2022-01-10 12:53:06,136] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,137] jepsen worker 81 - jepsen.util 81 :ok :read [27 7] INFO [2022-01-10 12:53:06,137] jepsen worker 13 - jepsen.util 13 :ok :write [4 22] INFO [2022-01-10 12:53:06,138] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,138] jepsen worker 15 - jepsen.util 15 :ok :write [5 47] INFO [2022-01-10 12:53:06,138] jepsen worker 81 - jepsen.util 81 :ok :read [27 7] INFO [2022-01-10 12:53:06,139] jepsen worker 13 - jepsen.util 13 :invoke :write [4 16] INFO [2022-01-10 12:53:06,140] jepsen worker 15 - jepsen.util 15 :invoke :write [5 6] INFO [2022-01-10 12:53:06,140] jepsen worker 23 - jepsen.util 23 :ok :write [7 24] INFO [2022-01-10 12:53:06,141] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,141] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,141] jepsen worker 81 - jepsen.util 81 :ok :read [27 7] INFO [2022-01-10 12:53:06,142] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:06,142] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,143] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,143] jepsen worker 81 - jepsen.util 81 :ok :read [27 7] INFO [2022-01-10 12:53:06,144] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:06,144] jepsen worker 69 - jepsen.util 69 :ok :write [23 36] INFO [2022-01-10 12:53:06,144] jepsen worker 81 - jepsen.util 81 :invoke :write [27 29] INFO [2022-01-10 12:53:06,145] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,145] jepsen worker 33 - jepsen.util 33 :ok :write [11 9] INFO [2022-01-10 12:53:06,146] jepsen worker 69 - jepsen.util 69 :invoke :write [23 19] INFO [2022-01-10 12:53:06,146] jepsen worker 33 - jepsen.util 33 :invoke :write [11 45] INFO [2022-01-10 12:53:06,146] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:06,147] jepsen worker 44 - jepsen.util 44 :ok :write [14 43] INFO [2022-01-10 12:53:06,147] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,148] jepsen worker 23 - jepsen.util 23 :invoke :write [7 30] INFO [2022-01-10 12:53:06,148] jepsen worker 44 - jepsen.util 44 :ok :read [14 43] INFO [2022-01-10 12:53:06,149] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,149] jepsen worker 7 - jepsen.util 7 :ok :write [2 35] INFO [2022-01-10 12:53:06,150] jepsen worker 44 - jepsen.util 44 :ok :read [14 43] INFO [2022-01-10 12:53:06,150] jepsen worker 7 - jepsen.util 7 :invoke :write [2 16] INFO [2022-01-10 12:53:06,151] jepsen worker 44 - jepsen.util 44 :invoke :write [14 14] INFO [2022-01-10 12:53:06,152] jepsen worker 78 - jepsen.util 78 :ok :write [26 11] INFO [2022-01-10 12:53:06,152] jepsen worker 78 - jepsen.util 78 :invoke :write [26 37] INFO [2022-01-10 12:53:06,153] jepsen worker 68 - jepsen.util 68 :ok :write [22 3] INFO [2022-01-10 12:53:06,154] jepsen worker 60 - jepsen.util 60 :ok :write [20 47] INFO [2022-01-10 12:53:06,154] jepsen worker 68 - jepsen.util 68 :invoke :write [22 42] INFO [2022-01-10 12:53:06,154] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:06,155] jepsen worker 60 - jepsen.util 60 :ok :read [20 47] INFO [2022-01-10 12:53:06,156] jepsen worker 60 - jepsen.util 60 :invoke :write [20 37] INFO [2022-01-10 12:53:06,159] jepsen worker 51 - jepsen.util 51 :ok :write [17 0] INFO [2022-01-10 12:53:06,159] jepsen worker 50 - jepsen.util 50 :ok :write [16 25] INFO [2022-01-10 12:53:06,160] jepsen worker 50 - jepsen.util 50 :invoke :write [16 1] INFO [2022-01-10 12:53:06,160] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:06,161] jepsen worker 75 - jepsen.util 75 :ok :write [25 5] INFO [2022-01-10 12:53:06,161] jepsen worker 51 - jepsen.util 51 :ok :read [17 0] INFO [2022-01-10 12:53:06,162] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:06,162] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:06,163] jepsen worker 75 - jepsen.util 75 :ok :read [25 5] INFO [2022-01-10 12:53:06,163] jepsen worker 51 - jepsen.util 51 :ok :read [17 0] INFO [2022-01-10 12:53:06,164] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:06,164] jepsen worker 38 - jepsen.util 38 :ok :write [12 6] INFO [2022-01-10 12:53:06,165] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:06,165] jepsen worker 75 - jepsen.util 75 :ok :read [25 5] INFO [2022-01-10 12:53:06,165] jepsen worker 51 - jepsen.util 51 :ok :read [17 0] INFO [2022-01-10 12:53:06,165] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:06,166] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:06,167] jepsen worker 38 - jepsen.util 38 :ok :read [12 6] INFO [2022-01-10 12:53:06,167] jepsen worker 75 - jepsen.util 75 :invoke :write [25 3] INFO [2022-01-10 12:53:06,167] jepsen worker 51 - jepsen.util 51 :ok :read [17 0] INFO [2022-01-10 12:53:06,168] jepsen worker 39 - jepsen.util 39 :ok :write [13 21] INFO [2022-01-10 12:53:06,168] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:06,169] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,170] jepsen worker 51 - jepsen.util 51 :invoke :write [17 6] INFO [2022-01-10 12:53:06,170] jepsen worker 39 - jepsen.util 39 :ok :read [13 21] INFO [2022-01-10 12:53:06,170] jepsen worker 15 - jepsen.util 15 :ok :write [5 6] INFO [2022-01-10 12:53:06,171] jepsen worker 39 - jepsen.util 39 :invoke :write [13 39] INFO [2022-01-10 12:53:06,171] jepsen worker 84 - jepsen.util 84 :ok :write [28 7] INFO [2022-01-10 12:53:06,171] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:06,172] jepsen worker 84 - jepsen.util 84 :invoke :write [28 36] INFO [2022-01-10 12:53:06,173] jepsen worker 15 - jepsen.util 15 :ok :read [5 6] INFO [2022-01-10 12:53:06,173] jepsen worker 15 - jepsen.util 15 :invoke :write [5 35] INFO [2022-01-10 12:53:06,175] jepsen worker 1 - jepsen.util 1 :ok :write [0 9] INFO [2022-01-10 12:53:06,175] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,176] jepsen worker 1 - jepsen.util 1 :ok :read [0 9] INFO [2022-01-10 12:53:06,177] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,177] jepsen worker 4 - jepsen.util 4 :ok :write [1 16] INFO [2022-01-10 12:53:06,178] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,178] jepsen worker 1 - jepsen.util 1 :ok :read [0 9] INFO [2022-01-10 12:53:06,178] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,179] jepsen worker 4 - jepsen.util 4 :ok :read [1 16] INFO [2022-01-10 12:53:06,179] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,179] jepsen worker 1 - jepsen.util 1 :ok :read [0 9] INFO [2022-01-10 12:53:06,180] jepsen worker 4 - jepsen.util 4 :ok :read [1 16] INFO [2022-01-10 12:53:06,180] jepsen worker 1 - jepsen.util 1 :invoke :write [0 7] INFO [2022-01-10 12:53:06,181] jepsen worker 33 - jepsen.util 33 :ok :write [11 45] INFO [2022-01-10 12:53:06,181] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,182] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:06,182] jepsen worker 13 - jepsen.util 13 :ok :write [4 16] INFO [2022-01-10 12:53:06,182] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,182] jepsen worker 33 - jepsen.util 33 :ok :read [11 45] INFO [2022-01-10 12:53:06,183] jepsen worker 87 - jepsen.util 87 :ok :write [29 4] INFO [2022-01-10 12:53:06,183] jepsen worker 4 - jepsen.util 4 :ok :read [1 16] INFO [2022-01-10 12:53:06,184] jepsen worker 33 - jepsen.util 33 :invoke :write [11 2] INFO [2022-01-10 12:53:06,183] jepsen worker 81 - jepsen.util 81 :ok :write [27 29] INFO [2022-01-10 12:53:06,184] jepsen worker 13 - jepsen.util 13 :ok :read [4 16] INFO [2022-01-10 12:53:06,185] jepsen worker 13 - jepsen.util 13 :invoke :write [4 5] INFO [2022-01-10 12:53:06,185] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,186] jepsen worker 69 - jepsen.util 69 :ok :write [23 19] INFO [2022-01-10 12:53:06,186] jepsen worker 4 - jepsen.util 4 :invoke :write [1 0] INFO [2022-01-10 12:53:06,186] jepsen worker 87 - jepsen.util 87 :invoke :write [29 36] INFO [2022-01-10 12:53:06,186] jepsen worker 81 - jepsen.util 81 :ok :read [27 29] INFO [2022-01-10 12:53:06,187] jepsen worker 81 - jepsen.util 81 :invoke :write [27 26] INFO [2022-01-10 12:53:06,187] jepsen worker 69 - jepsen.util 69 :invoke :write [23 22] INFO [2022-01-10 12:53:06,188] jepsen worker 7 - jepsen.util 7 :ok :write [2 16] INFO [2022-01-10 12:53:06,189] jepsen worker 7 - jepsen.util 7 :invoke :write [2 1] INFO [2022-01-10 12:53:06,193] jepsen worker 78 - jepsen.util 78 :ok :write [26 37] INFO [2022-01-10 12:53:06,194] jepsen worker 78 - jepsen.util 78 :invoke :write [26 1] INFO [2022-01-10 12:53:06,197] jepsen worker 23 - jepsen.util 23 :ok :write [7 30] INFO [2022-01-10 12:53:06,197] jepsen worker 23 - jepsen.util 23 :invoke :write [7 35] INFO [2022-01-10 12:53:06,199] jepsen worker 44 - jepsen.util 44 :ok :write [14 14] INFO [2022-01-10 12:53:06,199] jepsen worker 51 - jepsen.util 51 :ok :write [17 6] INFO [2022-01-10 12:53:06,200] jepsen worker 51 - jepsen.util 51 :invoke :write [17 6] INFO [2022-01-10 12:53:06,200] jepsen worker 44 - jepsen.util 44 :invoke :write [14 41] INFO [2022-01-10 12:53:06,205] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:06,205] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:06,206] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:06,207] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:06,208] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:06,209] jepsen worker 38 - jepsen.util 38 :invoke :write [12 24] INFO [2022-01-10 12:53:06,213] jepsen worker 84 - jepsen.util 84 :ok :write [28 36] INFO [2022-01-10 12:53:06,214] jepsen worker 84 - jepsen.util 84 :invoke :write [28 42] INFO [2022-01-10 12:53:06,217] jepsen worker 3 - jepsen.util 3 :ok :write [1 14] INFO [2022-01-10 12:53:06,217] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,218] jepsen worker 39 - jepsen.util 39 :ok :write [13 39] INFO [2022-01-10 12:53:06,218] jepsen worker 3 - jepsen.util 3 :ok :read [1 14] INFO [2022-01-10 12:53:06,218] jepsen worker 39 - jepsen.util 39 :invoke :write [13 4] INFO [2022-01-10 12:53:06,219] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,220] jepsen worker 3 - jepsen.util 3 :ok :read [1 14] INFO [2022-01-10 12:53:06,221] jepsen worker 3 - jepsen.util 3 :invoke :write [1 28] INFO [2022-01-10 12:53:06,225] jepsen worker 1 - jepsen.util 1 :ok :write [0 7] INFO [2022-01-10 12:53:06,226] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,227] jepsen worker 1 - jepsen.util 1 :ok :read [0 7] INFO [2022-01-10 12:53:06,227] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,228] jepsen worker 1 - jepsen.util 1 :ok :read [0 7] INFO [2022-01-10 12:53:06,228] jepsen worker 1 - jepsen.util 1 :invoke :write [0 24] INFO [2022-01-10 12:53:06,231] jepsen worker 87 - jepsen.util 87 :ok :write [29 36] INFO [2022-01-10 12:53:06,232] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,233] jepsen worker 87 - jepsen.util 87 :ok :read [29 36] INFO [2022-01-10 12:53:06,234] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,234] jepsen worker 87 - jepsen.util 87 :ok :read [29 36] INFO [2022-01-10 12:53:06,235] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,236] jepsen worker 87 - jepsen.util 87 :ok :read [29 36] INFO [2022-01-10 12:53:06,236] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,237] jepsen worker 87 - jepsen.util 87 :ok :read [29 36] INFO [2022-01-10 12:53:06,238] jepsen worker 87 - jepsen.util 87 :invoke :write [29 39] INFO [2022-01-10 12:53:06,244] jepsen worker 38 - jepsen.util 38 :ok :write [12 24] INFO [2022-01-10 12:53:06,244] jepsen worker 38 - jepsen.util 38 :invoke :write [12 10] INFO [2022-01-10 12:53:06,250] jepsen worker 39 - jepsen.util 39 :ok :write [13 4] INFO [2022-01-10 12:53:06,250] jepsen worker 84 - jepsen.util 84 :ok :write [28 42] INFO [2022-01-10 12:53:06,251] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:06,251] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,252] jepsen worker 84 - jepsen.util 84 :ok :read [28 42] INFO [2022-01-10 12:53:06,252] jepsen worker 39 - jepsen.util 39 :ok :read [13 4] INFO [2022-01-10 12:53:06,252] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,253] jepsen worker 84 - jepsen.util 84 :invoke :write [28 23] INFO [2022-01-10 12:53:06,253] jepsen worker 39 - jepsen.util 39 :ok :read [13 4] INFO [2022-01-10 12:53:06,254] jepsen worker 39 - jepsen.util 39 :invoke :write [13 49] INFO [2022-01-10 12:53:06,254] jepsen worker 4 - jepsen.util 4 :ok :write [1 0] INFO [2022-01-10 12:53:06,255] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,256] jepsen worker 4 - jepsen.util 4 :ok :read [1 0] INFO [2022-01-10 12:53:06,256] jepsen worker 4 - jepsen.util 4 :invoke :write [1 16] INFO [2022-01-10 12:53:06,257] jepsen worker 1 - jepsen.util 1 :ok :write [0 24] INFO [2022-01-10 12:53:06,258] jepsen worker 1 - jepsen.util 1 :invoke :write [0 14] INFO [2022-01-10 12:53:06,260] jepsen worker 38 - jepsen.util 38 :ok :write [12 10] INFO [2022-01-10 12:53:06,261] jepsen worker 38 - jepsen.util 38 :invoke :write [12 37] INFO [2022-01-10 12:53:06,263] jepsen worker 87 - jepsen.util 87 :ok :write [29 39] INFO [2022-01-10 12:53:06,264] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,265] jepsen worker 87 - jepsen.util 87 :ok :read [29 39] INFO [2022-01-10 12:53:06,265] jepsen worker 87 - jepsen.util 87 :invoke :write [29 6] INFO [2022-01-10 12:53:06,265] jepsen worker 84 - jepsen.util 84 :ok :write [28 23] INFO [2022-01-10 12:53:06,266] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:06,267] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:06,268] jepsen worker 84 - jepsen.util 84 :invoke :write [28 10] INFO [2022-01-10 12:53:06,270] jepsen worker 39 - jepsen.util 39 :ok :write [13 49] INFO [2022-01-10 12:53:06,271] jepsen worker 39 - jepsen.util 39 :invoke :write [13 21] INFO [2022-01-10 12:53:06,271] jepsen worker 4 - jepsen.util 4 :ok :write [1 16] INFO [2022-01-10 12:53:06,272] jepsen worker 4 - jepsen.util 4 :invoke :write [1 5] INFO [2022-01-10 12:53:06,274] jepsen worker 1 - jepsen.util 1 :ok :write [0 14] INFO [2022-01-10 12:53:06,275] jepsen worker 1 - jepsen.util 1 :invoke :write [0 14] INFO [2022-01-10 12:53:06,280] jepsen worker 38 - jepsen.util 38 :ok :write [12 37] INFO [2022-01-10 12:53:06,280] jepsen worker 87 - jepsen.util 87 :ok :write [29 6] INFO [2022-01-10 12:53:06,280] jepsen worker 87 - jepsen.util 87 :invoke :write [29 11] INFO [2022-01-10 12:53:06,281] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:06,282] jepsen worker 38 - jepsen.util 38 :ok :read [12 37] INFO [2022-01-10 12:53:06,283] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:06,283] jepsen worker 38 - jepsen.util 38 :ok :read [12 37] INFO [2022-01-10 12:53:06,284] jepsen worker 38 - jepsen.util 38 :invoke :write [12 32] INFO [2022-01-10 12:53:06,286] jepsen worker 39 - jepsen.util 39 :ok :write [13 21] INFO [2022-01-10 12:53:06,286] jepsen worker 39 - jepsen.util 39 :invoke :write [13 22] INFO [2022-01-10 12:53:06,289] jepsen worker 84 - jepsen.util 84 :ok :write [28 10] INFO [2022-01-10 12:53:06,289] jepsen worker 13 - jepsen.util 13 :ok :write [4 5] INFO [2022-01-10 12:53:06,289] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:06,290] jepsen worker 13 - jepsen.util 13 :invoke :write [4 2] INFO [2022-01-10 12:53:06,290] jepsen worker 84 - jepsen.util 84 :ok :read [28 10] INFO [2022-01-10 12:53:06,291] jepsen worker 84 - jepsen.util 84 :invoke :write [28 23] INFO [2022-01-10 12:53:06,291] jepsen worker 1 - jepsen.util 1 :ok :write [0 14] INFO [2022-01-10 12:53:06,291] jepsen worker 1 - jepsen.util 1 :invoke :write [0 14] INFO [2022-01-10 12:53:06,294] jepsen worker 3 - jepsen.util 3 :ok :write [1 28] INFO [2022-01-10 12:53:06,295] jepsen worker 3 - jepsen.util 3 :invoke :write [1 22] INFO [2022-01-10 12:53:06,295] jepsen worker 60 - jepsen.util 60 :ok :write [20 37] INFO [2022-01-10 12:53:06,296] jepsen worker 60 - jepsen.util 60 :invoke :write [20 3] INFO [2022-01-10 12:53:06,298] jepsen worker 69 - jepsen.util 69 :ok :write [23 22] INFO [2022-01-10 12:53:06,299] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:06,300] jepsen worker 69 - jepsen.util 69 :ok :read [23 22] INFO [2022-01-10 12:53:06,300] jepsen worker 69 - jepsen.util 69 :invoke :write [23 33] INFO [2022-01-10 12:53:06,306] jepsen worker 68 - jepsen.util 68 :ok :write [22 42] INFO [2022-01-10 12:53:06,306] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,307] jepsen worker 68 - jepsen.util 68 :ok :read [22 42] INFO [2022-01-10 12:53:06,307] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,308] jepsen worker 68 - jepsen.util 68 :ok :read [22 42] INFO [2022-01-10 12:53:06,309] jepsen worker 68 - jepsen.util 68 :invoke :write [22 32] INFO [2022-01-10 12:53:06,309] jepsen worker 78 - jepsen.util 78 :ok :write [26 1] INFO [2022-01-10 12:53:06,310] jepsen worker 78 - jepsen.util 78 :invoke :write [26 13] INFO [2022-01-10 12:53:06,312] jepsen worker 50 - jepsen.util 50 :ok :write [16 1] INFO [2022-01-10 12:53:06,313] jepsen worker 50 - jepsen.util 50 :invoke :write [16 13] INFO [2022-01-10 12:53:06,317] jepsen worker 51 - jepsen.util 51 :ok :write [17 6] INFO [2022-01-10 12:53:06,318] jepsen worker 51 - jepsen.util 51 :invoke :write [17 35] INFO [2022-01-10 12:53:06,320] jepsen worker 75 - jepsen.util 75 :ok :write [25 3] INFO [2022-01-10 12:53:06,321] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:06,322] jepsen worker 75 - jepsen.util 75 :ok :read [25 3] INFO [2022-01-10 12:53:06,323] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:06,324] jepsen worker 75 - jepsen.util 75 :ok :read [25 3] INFO [2022-01-10 12:53:06,324] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:06,325] jepsen worker 13 - jepsen.util 13 :ok :write [4 2] INFO [2022-01-10 12:53:06,325] jepsen worker 75 - jepsen.util 75 :ok :read [25 3] INFO [2022-01-10 12:53:06,325] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,326] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:06,327] jepsen worker 13 - jepsen.util 13 :ok :read [4 2] INFO [2022-01-10 12:53:06,327] jepsen worker 75 - jepsen.util 75 :ok :read [25 3] INFO [2022-01-10 12:53:06,327] jepsen worker 13 - jepsen.util 13 :invoke :write [4 9] INFO [2022-01-10 12:53:06,328] jepsen worker 75 - jepsen.util 75 :invoke :write [25 28] INFO [2022-01-10 12:53:06,331] jepsen worker 15 - jepsen.util 15 :ok :write [5 35] INFO [2022-01-10 12:53:06,331] jepsen worker 69 - jepsen.util 69 :ok :write [23 33] INFO [2022-01-10 12:53:06,332] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:06,332] jepsen worker 69 - jepsen.util 69 :invoke :write [23 46] INFO [2022-01-10 12:53:06,333] jepsen worker 15 - jepsen.util 15 :ok :read [5 35] INFO [2022-01-10 12:53:06,333] jepsen worker 15 - jepsen.util 15 :invoke :write [5 19] INFO [2022-01-10 12:53:06,401] jepsen worker 87 - jepsen.util 87 :ok :write [29 11] INFO [2022-01-10 12:53:06,401] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,402] jepsen worker 87 - jepsen.util 87 :ok :read [29 11] INFO [2022-01-10 12:53:06,402] jepsen worker 87 - jepsen.util 87 :invoke :write [29 10] INFO [2022-01-10 12:53:06,405] jepsen worker 38 - jepsen.util 38 :ok :write [12 32] INFO [2022-01-10 12:53:06,406] jepsen worker 39 - jepsen.util 39 :ok :write [13 22] INFO [2022-01-10 12:53:06,406] jepsen worker 38 - jepsen.util 38 :invoke :write [12 7] INFO [2022-01-10 12:53:06,406] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,407] jepsen worker 39 - jepsen.util 39 :ok :read [13 22] INFO [2022-01-10 12:53:06,408] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,409] jepsen worker 39 - jepsen.util 39 :ok :read [13 22] INFO [2022-01-10 12:53:06,409] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,410] jepsen worker 39 - jepsen.util 39 :ok :read [13 22] INFO [2022-01-10 12:53:06,410] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,411] jepsen worker 84 - jepsen.util 84 :ok :write [28 23] INFO [2022-01-10 12:53:06,411] jepsen worker 1 - jepsen.util 1 :ok :write [0 14] INFO [2022-01-10 12:53:06,411] jepsen worker 39 - jepsen.util 39 :ok :read [13 22] INFO [2022-01-10 12:53:06,411] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,412] jepsen worker 39 - jepsen.util 39 :invoke :write [13 36] INFO [2022-01-10 12:53:06,412] jepsen worker 1 - jepsen.util 1 :ok :read [0 14] INFO [2022-01-10 12:53:06,412] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:06,413] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,413] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:06,414] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:06,414] jepsen worker 1 - jepsen.util 1 :ok :read [0 14] INFO [2022-01-10 12:53:06,415] jepsen worker 1 - jepsen.util 1 :invoke :write [0 39] INFO [2022-01-10 12:53:06,415] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:06,415] jepsen worker 84 - jepsen.util 84 :invoke :write [28 28] INFO [2022-01-10 12:53:06,418] jepsen worker 3 - jepsen.util 3 :ok :write [1 22] INFO [2022-01-10 12:53:06,419] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,420] jepsen worker 3 - jepsen.util 3 :ok :read [1 22] INFO [2022-01-10 12:53:06,420] jepsen worker 3 - jepsen.util 3 :invoke :write [1 37] INFO [2022-01-10 12:53:06,487] jepsen worker 78 - jepsen.util 78 :ok :write [26 13] INFO [2022-01-10 12:53:06,488] jepsen worker 78 - jepsen.util 78 :invoke :write [26 11] INFO [2022-01-10 12:53:06,488] jepsen worker 33 - jepsen.util 33 :ok :write [11 2] INFO [2022-01-10 12:53:06,489] jepsen worker 33 - jepsen.util 33 :invoke :write [11 29] INFO [2022-01-10 12:53:06,496] jepsen worker 51 - jepsen.util 51 :ok :write [17 35] INFO [2022-01-10 12:53:06,497] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:06,497] jepsen worker 81 - jepsen.util 81 :ok :write [27 26] INFO [2022-01-10 12:53:06,498] jepsen worker 51 - jepsen.util 51 :ok :read [17 35] INFO [2022-01-10 12:53:06,498] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,498] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:06,499] jepsen worker 81 - jepsen.util 81 :ok :read [27 26] INFO [2022-01-10 12:53:06,499] jepsen worker 51 - jepsen.util 51 :ok :read [17 35] INFO [2022-01-10 12:53:06,499] jepsen worker 81 - jepsen.util 81 :invoke :write [27 25] INFO [2022-01-10 12:53:06,500] jepsen worker 51 - jepsen.util 51 :invoke :write [17 13] INFO [2022-01-10 12:53:06,505] jepsen worker 7 - jepsen.util 7 :ok :write [2 1] INFO [2022-01-10 12:53:06,505] jepsen worker 13 - jepsen.util 13 :ok :write [4 9] INFO [2022-01-10 12:53:06,505] jepsen worker 7 - jepsen.util 7 :invoke :write [2 7] INFO [2022-01-10 12:53:06,506] jepsen worker 13 - jepsen.util 13 :invoke :write [4 11] INFO [2022-01-10 12:53:06,512] jepsen worker 69 - jepsen.util 69 :ok :write [23 46] INFO [2022-01-10 12:53:06,513] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:06,513] jepsen worker 23 - jepsen.util 23 :ok :write [7 35] INFO [2022-01-10 12:53:06,514] jepsen worker 69 - jepsen.util 69 :ok :read [23 46] INFO [2022-01-10 12:53:06,514] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,514] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:06,515] jepsen worker 23 - jepsen.util 23 :ok :read [7 35] INFO [2022-01-10 12:53:06,515] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,515] jepsen worker 69 - jepsen.util 69 :ok :read [23 46] INFO [2022-01-10 12:53:06,516] jepsen worker 69 - jepsen.util 69 :invoke :write [23 38] INFO [2022-01-10 12:53:06,516] jepsen worker 23 - jepsen.util 23 :ok :read [7 35] INFO [2022-01-10 12:53:06,517] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,518] jepsen worker 23 - jepsen.util 23 :ok :read [7 35] INFO [2022-01-10 12:53:06,519] jepsen worker 23 - jepsen.util 23 :invoke :write [7 40] INFO [2022-01-10 12:53:06,523] jepsen worker 44 - jepsen.util 44 :ok :write [14 41] INFO [2022-01-10 12:53:06,524] jepsen worker 44 - jepsen.util 44 :invoke :write [14 6] INFO [2022-01-10 12:53:06,560] jepsen worker 87 - jepsen.util 87 :ok :write [29 10] INFO [2022-01-10 12:53:06,561] jepsen worker 87 - jepsen.util 87 :invoke :write [29 34] INFO [2022-01-10 12:53:06,562] jepsen worker 38 - jepsen.util 38 :ok :write [12 7] INFO [2022-01-10 12:53:06,563] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:06,564] jepsen worker 38 - jepsen.util 38 :ok :read [12 7] INFO [2022-01-10 12:53:06,565] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:06,565] jepsen worker 39 - jepsen.util 39 :ok :write [13 36] INFO [2022-01-10 12:53:06,565] jepsen worker 39 - jepsen.util 39 :invoke :write [13 0] INFO [2022-01-10 12:53:06,566] jepsen worker 38 - jepsen.util 38 :ok :read [12 7] INFO [2022-01-10 12:53:06,566] jepsen worker 38 - jepsen.util 38 :invoke :write [12 14] INFO [2022-01-10 12:53:06,570] jepsen worker 84 - jepsen.util 84 :ok :write [28 28] INFO [2022-01-10 12:53:06,571] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:06,572] jepsen worker 84 - jepsen.util 84 :ok :read [28 28] INFO [2022-01-10 12:53:06,573] jepsen worker 1 - jepsen.util 1 :ok :write [0 39] INFO [2022-01-10 12:53:06,573] jepsen worker 84 - jepsen.util 84 :invoke :write [28 37] INFO [2022-01-10 12:53:06,574] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,574] jepsen worker 3 - jepsen.util 3 :ok :write [1 37] INFO [2022-01-10 12:53:06,575] jepsen worker 1 - jepsen.util 1 :ok :read [0 39] INFO [2022-01-10 12:53:06,575] jepsen worker 1 - jepsen.util 1 :invoke :write [0 16] INFO [2022-01-10 12:53:06,576] jepsen worker 3 - jepsen.util 3 :invoke :write [1 8] INFO [2022-01-10 12:53:06,582] jepsen worker 78 - jepsen.util 78 :ok :write [26 11] INFO [2022-01-10 12:53:06,583] jepsen worker 78 - jepsen.util 78 :invoke :write [26 37] INFO [2022-01-10 12:53:06,584] jepsen worker 87 - jepsen.util 87 :ok :write [29 34] INFO [2022-01-10 12:53:06,585] jepsen worker 87 - jepsen.util 87 :invoke :write [29 16] INFO [2022-01-10 12:53:06,589] jepsen worker 39 - jepsen.util 39 :ok :write [13 0] INFO [2022-01-10 12:53:06,590] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,591] jepsen worker 39 - jepsen.util 39 :ok :read [13 0] INFO [2022-01-10 12:53:06,592] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,592] jepsen worker 39 - jepsen.util 39 :ok :read [13 0] INFO [2022-01-10 12:53:06,593] jepsen worker 39 - jepsen.util 39 :invoke :write [13 1] INFO [2022-01-10 12:53:06,594] jepsen worker 1 - jepsen.util 1 :ok :write [0 16] INFO [2022-01-10 12:53:06,594] jepsen worker 1 - jepsen.util 1 :invoke :write [0 13] INFO [2022-01-10 12:53:06,599] jepsen worker 87 - jepsen.util 87 :ok :write [29 16] INFO [2022-01-10 12:53:06,600] jepsen worker 87 - jepsen.util 87 :invoke :write [29 16] INFO [2022-01-10 12:53:06,602] jepsen worker 39 - jepsen.util 39 :ok :write [13 1] INFO [2022-01-10 12:53:06,603] jepsen worker 39 - jepsen.util 39 :invoke :write [13 39] INFO [2022-01-10 12:53:06,607] jepsen worker 38 - jepsen.util 38 :ok :write [12 14] INFO [2022-01-10 12:53:06,608] jepsen worker 38 - jepsen.util 38 :invoke :write [12 14] INFO [2022-01-10 12:53:06,610] jepsen worker 1 - jepsen.util 1 :ok :write [0 13] INFO [2022-01-10 12:53:06,611] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,612] jepsen worker 1 - jepsen.util 1 :ok :read [0 13] INFO [2022-01-10 12:53:06,612] jepsen worker 1 - jepsen.util 1 :invoke :write [0 17] INFO [2022-01-10 12:53:06,613] jepsen worker 84 - jepsen.util 84 :ok :write [28 37] INFO [2022-01-10 12:53:06,614] jepsen worker 84 - jepsen.util 84 :invoke :write [28 43] INFO [2022-01-10 12:53:06,617] jepsen worker 87 - jepsen.util 87 :ok :write [29 16] INFO [2022-01-10 12:53:06,617] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,618] jepsen worker 87 - jepsen.util 87 :ok :read [29 16] INFO [2022-01-10 12:53:06,619] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,619] jepsen worker 4 - jepsen.util 4 :ok :write [1 5] INFO [2022-01-10 12:53:06,620] jepsen worker 87 - jepsen.util 87 :ok :read [29 16] INFO [2022-01-10 12:53:06,621] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,622] jepsen worker 87 - jepsen.util 87 :invoke :write [29 45] INFO [2022-01-10 12:53:06,623] jepsen worker 60 - jepsen.util 60 :ok :write [20 3] INFO [2022-01-10 12:53:06,623] jepsen worker 51 - jepsen.util 51 :ok :write [17 13] INFO [2022-01-10 12:53:06,624] jepsen worker 4 - jepsen.util 4 :ok :read [1 5] INFO [2022-01-10 12:53:06,624] jepsen worker 60 - jepsen.util 60 :invoke :write [20 31] INFO [2022-01-10 12:53:06,625] jepsen worker 51 - jepsen.util 51 :invoke :write [17 22] INFO [2022-01-10 12:53:06,626] jepsen worker 39 - jepsen.util 39 :ok :write [13 39] INFO [2022-01-10 12:53:06,626] jepsen worker 4 - jepsen.util 4 :invoke :write [1 2] INFO [2022-01-10 12:53:06,626] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,627] jepsen worker 39 - jepsen.util 39 :ok :read [13 39] INFO [2022-01-10 12:53:06,628] jepsen worker 13 - jepsen.util 13 :ok :write [4 11] INFO [2022-01-10 12:53:06,628] jepsen worker 39 - jepsen.util 39 :invoke :write [13 47] INFO [2022-01-10 12:53:06,629] jepsen worker 13 - jepsen.util 13 :invoke :write [4 20] INFO [2022-01-10 12:53:06,631] jepsen worker 68 - jepsen.util 68 :ok :write [22 32] INFO [2022-01-10 12:53:06,631] jepsen worker 1 - jepsen.util 1 :ok :write [0 17] INFO [2022-01-10 12:53:06,632] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,632] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,633] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:53:06,633] jepsen worker 1 - jepsen.util 1 :ok :read [0 17] INFO [2022-01-10 12:53:06,633] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,634] jepsen worker 38 - jepsen.util 38 :ok :write [12 14] INFO [2022-01-10 12:53:06,634] jepsen worker 1 - jepsen.util 1 :invoke :write [0 40] INFO [2022-01-10 12:53:06,634] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:53:06,635] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:06,635] jepsen worker 50 - jepsen.util 50 :ok :write [16 13] INFO [2022-01-10 12:53:06,635] jepsen worker 68 - jepsen.util 68 :invoke :write [22 48] INFO [2022-01-10 12:53:06,636] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:06,636] jepsen worker 38 - jepsen.util 38 :ok :read [12 14] INFO [2022-01-10 12:53:06,637] jepsen worker 69 - jepsen.util 69 :ok :write [23 38] INFO [2022-01-10 12:53:06,637] jepsen worker 87 - jepsen.util 87 :ok :write [29 45] INFO [2022-01-10 12:53:06,637] jepsen worker 38 - jepsen.util 38 :invoke :write [12 2] INFO [2022-01-10 12:53:06,637] jepsen worker 50 - jepsen.util 50 :ok :read [16 13] INFO [2022-01-10 12:53:06,638] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:06,639] jepsen worker 69 - jepsen.util 69 :invoke :write [23 27] INFO [2022-01-10 12:53:06,639] jepsen worker 87 - jepsen.util 87 :invoke :write [29 3] INFO [2022-01-10 12:53:06,640] jepsen worker 50 - jepsen.util 50 :ok :read [16 13] INFO [2022-01-10 12:53:06,641] jepsen worker 50 - jepsen.util 50 :invoke :write [16 39] INFO [2022-01-10 12:53:06,641] jepsen worker 84 - jepsen.util 84 :ok :write [28 43] INFO [2022-01-10 12:53:06,641] jepsen worker 78 - jepsen.util 78 :ok :write [26 37] INFO [2022-01-10 12:53:06,641] jepsen worker 75 - jepsen.util 75 :ok :write [25 28] INFO [2022-01-10 12:53:06,642] jepsen worker 78 - jepsen.util 78 :invoke :write [26 16] INFO [2022-01-10 12:53:06,643] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:06,643] jepsen worker 84 - jepsen.util 84 :invoke :write [28 23] INFO [2022-01-10 12:53:06,644] jepsen worker 75 - jepsen.util 75 :ok :read [25 28] INFO [2022-01-10 12:53:06,644] jepsen worker 75 - jepsen.util 75 :invoke :write [25 13] INFO [2022-01-10 12:53:06,647] jepsen worker 3 - jepsen.util 3 :ok :write [1 8] INFO [2022-01-10 12:53:06,647] jepsen worker 3 - jepsen.util 3 :invoke :write [1 41] INFO [2022-01-10 12:53:06,652] jepsen worker 15 - jepsen.util 15 :ok :write [5 19] INFO [2022-01-10 12:53:06,652] jepsen worker 15 - jepsen.util 15 :invoke :write [5 1] INFO [2022-01-10 12:53:06,653] jepsen worker 51 - jepsen.util 51 :ok :write [17 22] INFO [2022-01-10 12:53:06,653] jepsen worker 51 - jepsen.util 51 :invoke :write [17 16] INFO [2022-01-10 12:53:06,657] jepsen worker 13 - jepsen.util 13 :ok :write [4 20] INFO [2022-01-10 12:53:06,658] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,658] jepsen worker 39 - jepsen.util 39 :ok :write [13 47] INFO [2022-01-10 12:53:06,659] jepsen worker 39 - jepsen.util 39 :invoke :write [13 34] INFO [2022-01-10 12:53:06,659] jepsen worker 13 - jepsen.util 13 :ok :read [4 20] INFO [2022-01-10 12:53:06,659] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,660] jepsen worker 33 - jepsen.util 33 :ok :write [11 29] INFO [2022-01-10 12:53:06,661] jepsen worker 13 - jepsen.util 13 :ok :read [4 20] INFO [2022-01-10 12:53:06,661] jepsen worker 33 - jepsen.util 33 :invoke :write [11 19] INFO [2022-01-10 12:53:06,661] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,662] jepsen worker 13 - jepsen.util 13 :ok :read [4 20] INFO [2022-01-10 12:53:06,663] jepsen worker 13 - jepsen.util 13 :invoke :write [4 21] INFO [2022-01-10 12:53:06,664] jepsen worker 38 - jepsen.util 38 :ok :write [12 2] INFO [2022-01-10 12:53:06,664] jepsen worker 1 - jepsen.util 1 :ok :write [0 40] INFO [2022-01-10 12:53:06,664] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:06,665] jepsen worker 1 - jepsen.util 1 :invoke :write [0 48] INFO [2022-01-10 12:53:06,665] jepsen worker 38 - jepsen.util 38 :ok :read [12 2] INFO [2022-01-10 12:53:06,666] jepsen worker 38 - jepsen.util 38 :invoke :write [12 6] INFO [2022-01-10 12:53:06,669] jepsen worker 81 - jepsen.util 81 :ok :write [27 25] INFO [2022-01-10 12:53:06,670] jepsen worker 81 - jepsen.util 81 :invoke :write [27 42] INFO [2022-01-10 12:53:06,670] jepsen worker 69 - jepsen.util 69 :ok :write [23 27] INFO [2022-01-10 12:53:06,671] jepsen worker 69 - jepsen.util 69 :invoke :write [23 46] INFO [2022-01-10 12:53:06,672] jepsen worker 87 - jepsen.util 87 :ok :write [29 3] INFO [2022-01-10 12:53:06,672] jepsen worker 87 - jepsen.util 87 :invoke :write [29 27] INFO [2022-01-10 12:53:06,678] jepsen worker 78 - jepsen.util 78 :ok :write [26 16] INFO [2022-01-10 12:53:06,678] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:06,678] jepsen worker 6 - jepsen.util 6 :ok :write [2 43] INFO [2022-01-10 12:53:06,679] jepsen worker 6 - jepsen.util 6 :invoke :write [2 10] INFO [2022-01-10 12:53:06,679] jepsen worker 78 - jepsen.util 78 :ok :read [26 16] INFO [2022-01-10 12:53:06,680] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:06,681] jepsen worker 78 - jepsen.util 78 :ok :read [26 16] INFO [2022-01-10 12:53:06,681] jepsen worker 78 - jepsen.util 78 :invoke :write [26 33] INFO [2022-01-10 12:53:06,685] jepsen worker 39 - jepsen.util 39 :ok :write [13 34] INFO [2022-01-10 12:53:06,685] jepsen worker 84 - jepsen.util 84 :ok :write [28 23] INFO [2022-01-10 12:53:06,686] jepsen worker 23 - jepsen.util 23 :ok :write [7 40] INFO [2022-01-10 12:53:06,686] jepsen worker 84 - jepsen.util 84 :invoke :write [28 6] INFO [2022-01-10 12:53:06,686] jepsen worker 39 - jepsen.util 39 :invoke :write [13 32] INFO [2022-01-10 12:53:06,687] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,688] jepsen worker 23 - jepsen.util 23 :ok :read [7 40] INFO [2022-01-10 12:53:06,689] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,690] jepsen worker 23 - jepsen.util 23 :ok :read [7 40] INFO [2022-01-10 12:53:06,690] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,691] jepsen worker 23 - jepsen.util 23 :ok :read [7 40] INFO [2022-01-10 12:53:06,692] jepsen worker 23 - jepsen.util 23 :invoke :write [7 18] INFO [2022-01-10 12:53:06,692] jepsen worker 4 - jepsen.util 4 :ok :write [1 2] INFO [2022-01-10 12:53:06,693] jepsen worker 44 - jepsen.util 44 :ok :write [14 6] INFO [2022-01-10 12:53:06,693] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,694] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,694] jepsen worker 4 - jepsen.util 4 :ok :read [1 2] INFO [2022-01-10 12:53:06,695] jepsen worker 44 - jepsen.util 44 :ok :read [14 6] INFO [2022-01-10 12:53:06,695] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,695] jepsen worker 1 - jepsen.util 1 :ok :write [0 48] INFO [2022-01-10 12:53:06,696] jepsen worker 1 - jepsen.util 1 :invoke :write [0 33] INFO [2022-01-10 12:53:06,696] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,697] jepsen worker 4 - jepsen.util 4 :ok :read [1 2] INFO [2022-01-10 12:53:06,697] jepsen worker 44 - jepsen.util 44 :ok :read [14 6] INFO [2022-01-10 12:53:06,697] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,698] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,698] jepsen worker 4 - jepsen.util 4 :ok :read [1 2] INFO [2022-01-10 12:53:06,699] jepsen worker 44 - jepsen.util 44 :ok :read [14 6] INFO [2022-01-10 12:53:06,699] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,699] jepsen worker 60 - jepsen.util 60 :ok :write [20 31] INFO [2022-01-10 12:53:06,700] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,700] jepsen worker 60 - jepsen.util 60 :invoke :write [20 0] INFO [2022-01-10 12:53:06,700] jepsen worker 4 - jepsen.util 4 :ok :read [1 2] INFO [2022-01-10 12:53:06,701] jepsen worker 44 - jepsen.util 44 :ok :read [14 6] INFO [2022-01-10 12:53:06,701] jepsen worker 51 - jepsen.util 51 :ok :write [17 16] INFO [2022-01-10 12:53:06,701] jepsen worker 4 - jepsen.util 4 :invoke :write [1 49] INFO [2022-01-10 12:53:06,702] jepsen worker 44 - jepsen.util 44 :invoke :write [14 25] INFO [2022-01-10 12:53:06,703] jepsen worker 51 - jepsen.util 51 :invoke :write [17 14] INFO [2022-01-10 12:53:06,707] jepsen worker 13 - jepsen.util 13 :ok :write [4 21] INFO [2022-01-10 12:53:06,707] jepsen worker 68 - jepsen.util 68 :ok :write [22 48] INFO [2022-01-10 12:53:06,708] jepsen worker 87 - jepsen.util 87 :ok :write [29 27] INFO [2022-01-10 12:53:06,708] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,709] jepsen worker 87 - jepsen.util 87 :invoke :write [29 22] INFO [2022-01-10 12:53:06,709] jepsen worker 68 - jepsen.util 68 :ok :read [22 48] INFO [2022-01-10 12:53:06,709] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,710] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,710] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:06,711] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,711] jepsen worker 68 - jepsen.util 68 :ok :read [22 48] INFO [2022-01-10 12:53:06,711] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,712] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:06,712] jepsen worker 68 - jepsen.util 68 :ok :read [22 48] INFO [2022-01-10 12:53:06,712] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,713] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,713] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:06,713] jepsen worker 68 - jepsen.util 68 :ok :read [22 48] INFO [2022-01-10 12:53:06,714] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,714] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,715] jepsen worker 38 - jepsen.util 38 :ok :write [12 6] INFO [2022-01-10 12:53:06,715] jepsen worker 39 - jepsen.util 39 :ok :write [13 32] INFO [2022-01-10 12:53:06,715] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:06,715] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,715] jepsen worker 68 - jepsen.util 68 :ok :read [22 48] INFO [2022-01-10 12:53:06,716] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,717] jepsen worker 39 - jepsen.util 39 :ok :read [13 32] INFO [2022-01-10 12:53:06,717] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,717] jepsen worker 68 - jepsen.util 68 :ok :read [22 48] INFO [2022-01-10 12:53:06,717] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,718] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:06,718] jepsen worker 50 - jepsen.util 50 :ok :write [16 39] INFO [2022-01-10 12:53:06,718] jepsen worker 38 - jepsen.util 38 :invoke :write [12 9] INFO [2022-01-10 12:53:06,718] jepsen worker 39 - jepsen.util 39 :ok :read [13 32] INFO [2022-01-10 12:53:06,719] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,719] jepsen worker 50 - jepsen.util 50 :invoke :write [16 18] INFO [2022-01-10 12:53:06,720] jepsen worker 68 - jepsen.util 68 :ok :read [22 48] INFO [2022-01-10 12:53:06,720] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,720] jepsen worker 39 - jepsen.util 39 :invoke :write [13 33] INFO [2022-01-10 12:53:06,721] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:06,721] jepsen worker 68 - jepsen.util 68 :invoke :write [22 10] INFO [2022-01-10 12:53:06,721] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,721] jepsen worker 1 - jepsen.util 1 :ok :write [0 33] INFO [2022-01-10 12:53:06,722] jepsen worker 1 - jepsen.util 1 :invoke :write [0 39] INFO [2022-01-10 12:53:06,722] jepsen worker 69 - jepsen.util 69 :ok :write [23 46] INFO [2022-01-10 12:53:06,722] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:06,722] jepsen worker 75 - jepsen.util 75 :ok :write [25 13] INFO [2022-01-10 12:53:06,723] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:06,723] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:06,723] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,724] jepsen worker 69 - jepsen.util 69 :ok :read [23 46] INFO [2022-01-10 12:53:06,724] jepsen worker 75 - jepsen.util 75 :ok :read [25 13] INFO [2022-01-10 12:53:06,724] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:06,725] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:06,725] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:06,725] jepsen worker 13 - jepsen.util 13 :invoke :write [4 38] INFO [2022-01-10 12:53:06,726] jepsen worker 75 - jepsen.util 75 :ok :read [25 13] INFO [2022-01-10 12:53:06,726] jepsen worker 75 - jepsen.util 75 :invoke :write [25 26] INFO [2022-01-10 12:53:06,726] jepsen worker 69 - jepsen.util 69 :ok :read [23 46] INFO [2022-01-10 12:53:06,727] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:06,728] jepsen worker 69 - jepsen.util 69 :ok :read [23 46] INFO [2022-01-10 12:53:06,728] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:06,729] jepsen worker 78 - jepsen.util 78 :ok :write [26 33] INFO [2022-01-10 12:53:06,729] jepsen worker 69 - jepsen.util 69 :ok :read [23 46] INFO [2022-01-10 12:53:06,730] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:06,730] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:06,731] jepsen worker 78 - jepsen.util 78 :ok :read [26 33] INFO [2022-01-10 12:53:06,731] jepsen worker 69 - jepsen.util 69 :ok :read [23 46] INFO [2022-01-10 12:53:06,731] jepsen worker 78 - jepsen.util 78 :invoke :write [26 4] INFO [2022-01-10 12:53:06,732] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:06,733] jepsen worker 69 - jepsen.util 69 :ok :read [23 46] INFO [2022-01-10 12:53:06,733] jepsen worker 15 - jepsen.util 15 :ok :write [5 1] INFO [2022-01-10 12:53:06,733] jepsen worker 69 - jepsen.util 69 :invoke :read [23 nil] INFO [2022-01-10 12:53:06,734] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:06,734] jepsen worker 69 - jepsen.util 69 :ok :read [23 46] INFO [2022-01-10 12:53:06,735] jepsen worker 15 - jepsen.util 15 :ok :read [5 1] INFO [2022-01-10 12:53:06,735] jepsen worker 69 - jepsen.util 69 :invoke :write [23 14] INFO [2022-01-10 12:53:06,735] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:06,735] jepsen worker 87 - jepsen.util 87 :ok :write [29 22] INFO [2022-01-10 12:53:06,736] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,736] jepsen worker 15 - jepsen.util 15 :ok :read [5 1] INFO [2022-01-10 12:53:06,737] jepsen worker 15 - jepsen.util 15 :invoke :write [5 10] INFO [2022-01-10 12:53:06,737] jepsen worker 87 - jepsen.util 87 :ok :read [29 22] INFO [2022-01-10 12:53:06,738] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,738] jepsen worker 84 - jepsen.util 84 :ok :write [28 6] INFO [2022-01-10 12:53:06,738] jepsen worker 84 - jepsen.util 84 :invoke :write [28 47] INFO [2022-01-10 12:53:06,738] jepsen worker 87 - jepsen.util 87 :ok :read [29 22] INFO [2022-01-10 12:53:06,739] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,740] jepsen worker 87 - jepsen.util 87 :ok :read [29 22] INFO [2022-01-10 12:53:06,741] jepsen worker 87 - jepsen.util 87 :invoke :write [29 41] INFO [2022-01-10 12:53:06,742] jepsen worker 33 - jepsen.util 33 :ok :write [11 19] INFO [2022-01-10 12:53:06,742] jepsen worker 39 - jepsen.util 39 :ok :write [13 33] INFO [2022-01-10 12:53:06,743] jepsen worker 33 - jepsen.util 33 :invoke :write [11 27] INFO [2022-01-10 12:53:06,743] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,744] jepsen worker 39 - jepsen.util 39 :ok :read [13 33] INFO [2022-01-10 12:53:06,744] jepsen worker 3 - jepsen.util 3 :ok :write [1 41] INFO [2022-01-10 12:53:06,745] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,745] jepsen worker 3 - jepsen.util 3 :invoke :write [1 48] INFO [2022-01-10 12:53:06,746] jepsen worker 39 - jepsen.util 39 :ok :read [13 33] INFO [2022-01-10 12:53:06,746] jepsen worker 39 - jepsen.util 39 :invoke :write [13 48] INFO [2022-01-10 12:53:06,750] jepsen worker 81 - jepsen.util 81 :ok :write [27 42] INFO [2022-01-10 12:53:06,750] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,750] jepsen worker 51 - jepsen.util 51 :ok :write [17 14] INFO [2022-01-10 12:53:06,751] jepsen worker 1 - jepsen.util 1 :ok :write [0 39] INFO [2022-01-10 12:53:06,751] jepsen worker 81 - jepsen.util 81 :ok :read [27 42] INFO [2022-01-10 12:53:06,751] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:06,752] jepsen worker 1 - jepsen.util 1 :invoke :write [0 28] INFO [2022-01-10 12:53:06,752] jepsen worker 51 - jepsen.util 51 :ok :read [17 14] INFO [2022-01-10 12:53:06,752] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,753] jepsen worker 51 - jepsen.util 51 :invoke :write [17 25] INFO [2022-01-10 12:53:06,753] jepsen worker 81 - jepsen.util 81 :ok :read [27 42] INFO [2022-01-10 12:53:06,754] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,755] jepsen worker 81 - jepsen.util 81 :ok :read [27 42] INFO [2022-01-10 12:53:06,755] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,756] jepsen worker 7 - jepsen.util 7 :ok :write [2 7] INFO [2022-01-10 12:53:06,756] jepsen worker 81 - jepsen.util 81 :ok :read [27 42] INFO [2022-01-10 12:53:06,757] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:06,757] jepsen worker 81 - jepsen.util 81 :invoke :write [27 10] INFO [2022-01-10 12:53:06,758] jepsen worker 7 - jepsen.util 7 :ok :read [2 7] INFO [2022-01-10 12:53:06,758] jepsen worker 7 - jepsen.util 7 :invoke :write [2 3] INFO [2022-01-10 12:53:06,759] jepsen worker 13 - jepsen.util 13 :ok :write [4 38] INFO [2022-01-10 12:53:06,759] jepsen worker 13 - jepsen.util 13 :invoke :write [4 2] INFO [2022-01-10 12:53:06,763] jepsen worker 87 - jepsen.util 87 :ok :write [29 41] INFO [2022-01-10 12:53:06,763] jepsen worker 87 - jepsen.util 87 :invoke :write [29 12] INFO [2022-01-10 12:53:06,766] jepsen worker 23 - jepsen.util 23 :ok :write [7 18] INFO [2022-01-10 12:53:06,767] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,768] jepsen worker 23 - jepsen.util 23 :ok :read [7 18] INFO [2022-01-10 12:53:06,768] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,769] jepsen worker 23 - jepsen.util 23 :ok :read [7 18] INFO [2022-01-10 12:53:06,769] jepsen worker 38 - jepsen.util 38 :ok :write [12 9] INFO [2022-01-10 12:53:06,770] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,770] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:06,771] jepsen worker 23 - jepsen.util 23 :ok :read [7 18] INFO [2022-01-10 12:53:06,771] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,771] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:06,772] jepsen worker 39 - jepsen.util 39 :ok :write [13 48] INFO [2022-01-10 12:53:06,772] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:06,772] jepsen worker 23 - jepsen.util 23 :ok :read [7 18] INFO [2022-01-10 12:53:06,773] jepsen worker 39 - jepsen.util 39 :invoke :write [13 25] INFO [2022-01-10 12:53:06,773] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:06,774] jepsen worker 23 - jepsen.util 23 :ok :read [7 18] INFO [2022-01-10 12:53:06,775] jepsen worker 23 - jepsen.util 23 :invoke :write [7 47] INFO [2022-01-10 12:53:06,775] jepsen worker 44 - jepsen.util 44 :ok :write [14 25] INFO [2022-01-10 12:53:06,775] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,777] jepsen worker 44 - jepsen.util 44 :ok :read [14 25] INFO [2022-01-10 12:53:06,777] jepsen worker 71 - jepsen.util 71 :ok :write [23 4] INFO [2022-01-10 12:53:06,777] jepsen worker 44 - jepsen.util 44 :invoke :write [14 34] INFO [2022-01-10 12:53:06,778] jepsen worker 71 - jepsen.util 71 :invoke :write [23 3] INFO [2022-01-10 12:53:06,784] jepsen worker 60 - jepsen.util 60 :ok :write [20 0] INFO [2022-01-10 12:53:06,785] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:06,786] jepsen worker 60 - jepsen.util 60 :ok :read [20 0] INFO [2022-01-10 12:53:06,786] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:06,787] jepsen worker 60 - jepsen.util 60 :ok :read [20 0] INFO [2022-01-10 12:53:06,788] jepsen worker 60 - jepsen.util 60 :invoke :write [20 37] INFO [2022-01-10 12:53:06,792] jepsen worker 68 - jepsen.util 68 :ok :write [22 10] INFO [2022-01-10 12:53:06,795] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:06,796] jepsen worker 68 - jepsen.util 68 :ok :read [22 10] INFO [2022-01-10 12:53:06,797] jepsen worker 68 - jepsen.util 68 :invoke :write [22 30] INFO [2022-01-10 12:53:06,798] jepsen worker 50 - jepsen.util 50 :ok :write [16 18] INFO [2022-01-10 12:53:06,799] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:06,800] jepsen worker 50 - jepsen.util 50 :ok :read [16 18] INFO [2022-01-10 12:53:06,801] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:06,801] jepsen worker 50 - jepsen.util 50 :ok :read [16 18] INFO [2022-01-10 12:53:06,802] jepsen worker 75 - jepsen.util 75 :ok :write [25 26] INFO [2022-01-10 12:53:06,802] jepsen worker 50 - jepsen.util 50 :invoke :write [16 34] INFO [2022-01-10 12:53:06,803] jepsen worker 75 - jepsen.util 75 :invoke :write [25 23] INFO [2022-01-10 12:53:06,808] jepsen worker 15 - jepsen.util 15 :ok :write [5 10] INFO [2022-01-10 12:53:06,809] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:06,810] jepsen worker 15 - jepsen.util 15 :ok :read [5 10] INFO [2022-01-10 12:53:06,810] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:06,811] jepsen worker 15 - jepsen.util 15 :ok :read [5 10] INFO [2022-01-10 12:53:06,811] jepsen worker 15 - jepsen.util 15 :invoke :write [5 6] INFO [2022-01-10 12:53:06,812] jepsen worker 33 - jepsen.util 33 :ok :write [11 27] INFO [2022-01-10 12:53:06,813] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:06,814] jepsen worker 33 - jepsen.util 33 :ok :read [11 27] INFO [2022-01-10 12:53:06,814] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:06,815] jepsen worker 33 - jepsen.util 33 :ok :read [11 27] INFO [2022-01-10 12:53:06,816] jepsen worker 33 - jepsen.util 33 :invoke :write [11 2] INFO [2022-01-10 12:53:06,918] jepsen worker 81 - jepsen.util 81 :ok :write [27 10] INFO [2022-01-10 12:53:06,919] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:06,920] jepsen worker 81 - jepsen.util 81 :ok :read [27 10] INFO [2022-01-10 12:53:06,920] jepsen worker 78 - jepsen.util 78 :ok :write [26 4] INFO [2022-01-10 12:53:06,921] jepsen worker 81 - jepsen.util 81 :invoke :write [27 6] INFO [2022-01-10 12:53:06,921] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:06,922] jepsen worker 78 - jepsen.util 78 :ok :read [26 4] INFO [2022-01-10 12:53:06,923] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:06,924] jepsen worker 1 - jepsen.util 1 :ok :write [0 28] INFO [2022-01-10 12:53:06,924] jepsen worker 78 - jepsen.util 78 :ok :read [26 4] INFO [2022-01-10 12:53:06,924] jepsen worker 1 - jepsen.util 1 :invoke :write [0 6] INFO [2022-01-10 12:53:06,925] jepsen worker 84 - jepsen.util 84 :ok :write [28 47] INFO [2022-01-10 12:53:06,925] jepsen worker 78 - jepsen.util 78 :invoke :write [26 6] INFO [2022-01-10 12:53:06,925] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:06,926] jepsen worker 23 - jepsen.util 23 :ok :write [7 47] INFO [2022-01-10 12:53:06,926] jepsen worker 84 - jepsen.util 84 :ok :read [28 47] INFO [2022-01-10 12:53:06,926] jepsen worker 23 - jepsen.util 23 :invoke :write [7 17] INFO [2022-01-10 12:53:06,927] jepsen worker 84 - jepsen.util 84 :invoke :write [28 33] INFO [2022-01-10 12:53:06,929] jepsen worker 3 - jepsen.util 3 :ok :write [1 48] INFO [2022-01-10 12:53:06,930] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,931] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:06,931] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,932] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:06,933] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,933] jepsen worker 7 - jepsen.util 7 :ok :write [2 3] INFO [2022-01-10 12:53:06,933] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:06,934] jepsen worker 7 - jepsen.util 7 :invoke :write [2 10] INFO [2022-01-10 12:53:06,934] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,935] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:06,936] jepsen worker 3 - jepsen.util 3 :invoke :write [1 48] INFO [2022-01-10 12:53:06,936] jepsen worker 44 - jepsen.util 44 :ok :write [14 34] INFO [2022-01-10 12:53:06,937] jepsen worker 51 - jepsen.util 51 :ok :write [17 25] INFO [2022-01-10 12:53:06,938] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,938] jepsen worker 51 - jepsen.util 51 :invoke :write [17 39] INFO [2022-01-10 12:53:06,939] jepsen worker 87 - jepsen.util 87 :ok :write [29 12] INFO [2022-01-10 12:53:06,940] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,940] jepsen worker 44 - jepsen.util 44 :ok :read [14 34] INFO [2022-01-10 12:53:06,941] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:06,941] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,941] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,942] jepsen worker 60 - jepsen.util 60 :ok :write [20 37] INFO [2022-01-10 12:53:06,942] jepsen worker 44 - jepsen.util 44 :ok :read [14 34] INFO [2022-01-10 12:53:06,942] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:06,943] jepsen worker 60 - jepsen.util 60 :invoke :write [20 18] INFO [2022-01-10 12:53:06,943] jepsen worker 39 - jepsen.util 39 :ok :write [13 25] INFO [2022-01-10 12:53:06,944] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,945] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,945] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:06,945] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,945] jepsen worker 39 - jepsen.util 39 :ok :read [13 25] INFO [2022-01-10 12:53:06,946] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:06,946] jepsen worker 44 - jepsen.util 44 :ok :read [14 34] INFO [2022-01-10 12:53:06,947] jepsen worker 13 - jepsen.util 13 :ok :write [4 2] INFO [2022-01-10 12:53:06,947] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:06,947] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,948] jepsen worker 87 - jepsen.util 87 :invoke :write [29 25] INFO [2022-01-10 12:53:06,948] jepsen worker 39 - jepsen.util 39 :ok :read [13 25] INFO [2022-01-10 12:53:06,949] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,950] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,950] jepsen worker 13 - jepsen.util 13 :ok :read [4 2] INFO [2022-01-10 12:53:06,950] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,951] jepsen worker 39 - jepsen.util 39 :ok :read [13 25] INFO [2022-01-10 12:53:06,951] jepsen worker 68 - jepsen.util 68 :ok :write [22 30] INFO [2022-01-10 12:53:06,951] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:06,952] jepsen worker 44 - jepsen.util 44 :ok :read [14 34] INFO [2022-01-10 12:53:06,952] jepsen worker 68 - jepsen.util 68 :invoke :write [22 25] INFO [2022-01-10 12:53:06,952] jepsen worker 13 - jepsen.util 13 :ok :read [4 2] INFO [2022-01-10 12:53:06,952] jepsen worker 39 - jepsen.util 39 :invoke :write [13 30] INFO [2022-01-10 12:53:06,953] jepsen worker 75 - jepsen.util 75 :ok :write [25 23] INFO [2022-01-10 12:53:06,953] jepsen worker 13 - jepsen.util 13 :invoke :write [4 40] INFO [2022-01-10 12:53:06,954] jepsen worker 75 - jepsen.util 75 :invoke :write [25 0] INFO [2022-01-10 12:53:06,955] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,956] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:06,957] jepsen worker 44 - jepsen.util 44 :ok :read [14 34] INFO [2022-01-10 12:53:06,958] jepsen worker 38 - jepsen.util 38 :invoke :write [12 24] INFO [2022-01-10 12:53:06,959] jepsen worker 50 - jepsen.util 50 :ok :write [16 34] INFO [2022-01-10 12:53:06,959] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,960] jepsen worker 50 - jepsen.util 50 :invoke :write [16 33] INFO [2022-01-10 12:53:06,961] jepsen worker 71 - jepsen.util 71 :ok :write [23 3] INFO [2022-01-10 12:53:06,962] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:06,963] jepsen worker 44 - jepsen.util 44 :ok :read [14 34] INFO [2022-01-10 12:53:06,963] jepsen worker 71 - jepsen.util 71 :ok :read [23 3] INFO [2022-01-10 12:53:06,963] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:06,964] jepsen worker 71 - jepsen.util 71 :invoke :write [23 11] INFO [2022-01-10 12:53:06,964] jepsen worker 44 - jepsen.util 44 :ok :read [14 34] INFO [2022-01-10 12:53:06,965] jepsen worker 44 - jepsen.util 44 :invoke :write [14 49] INFO [2022-01-10 12:53:06,965] jepsen worker 15 - jepsen.util 15 :ok :write [5 6] INFO [2022-01-10 12:53:06,966] jepsen worker 15 - jepsen.util 15 :invoke :write [5 23] INFO [2022-01-10 12:53:06,974] jepsen worker 84 - jepsen.util 84 :ok :write [28 33] INFO [2022-01-10 12:53:06,974] jepsen worker 84 - jepsen.util 84 :invoke :write [28 18] INFO [2022-01-10 12:53:06,975] jepsen worker 33 - jepsen.util 33 :ok :write [11 2] INFO [2022-01-10 12:53:06,975] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:06,976] jepsen worker 33 - jepsen.util 33 :ok :read [11 2] INFO [2022-01-10 12:53:06,977] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:06,978] jepsen worker 33 - jepsen.util 33 :ok :read [11 2] INFO [2022-01-10 12:53:06,978] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:06,979] jepsen worker 33 - jepsen.util 33 :ok :read [11 2] INFO [2022-01-10 12:53:06,979] jepsen worker 33 - jepsen.util 33 :invoke :write [11 13] INFO [2022-01-10 12:53:06,980] jepsen worker 1 - jepsen.util 1 :ok :write [0 6] INFO [2022-01-10 12:53:06,980] jepsen worker 3 - jepsen.util 3 :ok :write [1 48] INFO [2022-01-10 12:53:06,980] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,981] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:06,981] jepsen worker 1 - jepsen.util 1 :ok :read [0 6] INFO [2022-01-10 12:53:06,982] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,982] jepsen worker 78 - jepsen.util 78 :ok :write [26 6] INFO [2022-01-10 12:53:06,982] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:06,983] jepsen worker 1 - jepsen.util 1 :ok :read [0 6] INFO [2022-01-10 12:53:06,983] jepsen worker 78 - jepsen.util 78 :invoke :write [26 45] INFO [2022-01-10 12:53:06,984] jepsen worker 3 - jepsen.util 3 :invoke :write [1 16] INFO [2022-01-10 12:53:06,984] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:06,985] jepsen worker 1 - jepsen.util 1 :ok :read [0 6] INFO [2022-01-10 12:53:06,986] jepsen worker 1 - jepsen.util 1 :invoke :write [0 38] INFO [2022-01-10 12:53:06,989] jepsen worker 81 - jepsen.util 81 :ok :write [27 6] INFO [2022-01-10 12:53:06,989] jepsen worker 81 - jepsen.util 81 :invoke :write [27 47] INFO [2022-01-10 12:53:06,990] jepsen worker 51 - jepsen.util 51 :ok :write [17 39] INFO [2022-01-10 12:53:06,991] jepsen worker 87 - jepsen.util 87 :ok :write [29 25] INFO [2022-01-10 12:53:06,991] jepsen worker 51 - jepsen.util 51 :invoke :write [17 24] INFO [2022-01-10 12:53:06,991] jepsen worker 87 - jepsen.util 87 :invoke :write [29 21] INFO [2022-01-10 12:53:06,994] jepsen worker 39 - jepsen.util 39 :ok :write [13 30] INFO [2022-01-10 12:53:06,995] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:06,996] jepsen worker 39 - jepsen.util 39 :ok :read [13 30] INFO [2022-01-10 12:53:06,996] jepsen worker 39 - jepsen.util 39 :invoke :write [13 28] INFO [2022-01-10 12:53:07,004] jepsen worker 38 - jepsen.util 38 :ok :write [12 24] INFO [2022-01-10 12:53:07,005] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,006] jepsen worker 38 - jepsen.util 38 :ok :read [12 24] INFO [2022-01-10 12:53:07,007] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,008] jepsen worker 38 - jepsen.util 38 :ok :read [12 24] INFO [2022-01-10 12:53:07,008] jepsen worker 38 - jepsen.util 38 :invoke :write [12 19] INFO [2022-01-10 12:53:07,014] jepsen worker 84 - jepsen.util 84 :ok :write [28 18] INFO [2022-01-10 12:53:07,015] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,016] jepsen worker 84 - jepsen.util 84 :ok :read [28 18] INFO [2022-01-10 12:53:07,017] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,017] jepsen worker 3 - jepsen.util 3 :ok :write [1 16] INFO [2022-01-10 12:53:07,017] jepsen worker 1 - jepsen.util 1 :ok :write [0 38] INFO [2022-01-10 12:53:07,018] jepsen worker 84 - jepsen.util 84 :ok :read [28 18] INFO [2022-01-10 12:53:07,018] jepsen worker 3 - jepsen.util 3 :invoke :write [1 5] INFO [2022-01-10 12:53:07,019] jepsen worker 84 - jepsen.util 84 :invoke :write [28 45] INFO [2022-01-10 12:53:07,019] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,020] jepsen worker 1 - jepsen.util 1 :ok :read [0 38] INFO [2022-01-10 12:53:07,021] jepsen worker 1 - jepsen.util 1 :invoke :write [0 43] INFO [2022-01-10 12:53:07,032] jepsen worker 87 - jepsen.util 87 :ok :write [29 21] INFO [2022-01-10 12:53:07,033] jepsen worker 87 - jepsen.util 87 :invoke :write [29 15] INFO [2022-01-10 12:53:07,034] jepsen worker 39 - jepsen.util 39 :ok :write [13 28] INFO [2022-01-10 12:53:07,034] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,035] jepsen worker 39 - jepsen.util 39 :ok :read [13 28] INFO [2022-01-10 12:53:07,036] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,037] jepsen worker 39 - jepsen.util 39 :ok :read [13 28] INFO [2022-01-10 12:53:07,038] jepsen worker 1 - jepsen.util 1 :ok :write [0 43] INFO [2022-01-10 12:53:07,038] jepsen worker 39 - jepsen.util 39 :invoke :write [13 13] INFO [2022-01-10 12:53:07,038] jepsen worker 38 - jepsen.util 38 :ok :write [12 19] INFO [2022-01-10 12:53:07,038] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,039] jepsen worker 1 - jepsen.util 1 :ok :read [0 43] INFO [2022-01-10 12:53:07,039] jepsen worker 38 - jepsen.util 38 :invoke :write [12 25] INFO [2022-01-10 12:53:07,040] jepsen worker 1 - jepsen.util 1 :invoke :write [0 46] INFO [2022-01-10 12:53:07,044] jepsen worker 87 - jepsen.util 87 :ok :write [29 15] INFO [2022-01-10 12:53:07,045] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,046] jepsen worker 87 - jepsen.util 87 :ok :read [29 15] INFO [2022-01-10 12:53:07,046] jepsen worker 87 - jepsen.util 87 :invoke :write [29 38] INFO [2022-01-10 12:53:07,050] jepsen worker 84 - jepsen.util 84 :ok :write [28 45] INFO [2022-01-10 12:53:07,051] jepsen worker 84 - jepsen.util 84 :invoke :write [28 30] INFO [2022-01-10 12:53:07,054] jepsen worker 3 - jepsen.util 3 :ok :write [1 5] INFO [2022-01-10 12:53:07,054] jepsen worker 1 - jepsen.util 1 :ok :write [0 46] INFO [2022-01-10 12:53:07,054] jepsen worker 1 - jepsen.util 1 :invoke :write [0 25] INFO [2022-01-10 12:53:07,055] jepsen worker 39 - jepsen.util 39 :ok :write [13 13] INFO [2022-01-10 12:53:07,055] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,056] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,056] jepsen worker 3 - jepsen.util 3 :ok :read [1 5] INFO [2022-01-10 12:53:07,057] jepsen worker 39 - jepsen.util 39 :ok :read [13 13] INFO [2022-01-10 12:53:07,057] jepsen worker 3 - jepsen.util 3 :invoke :write [1 42] INFO [2022-01-10 12:53:07,058] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,059] jepsen worker 39 - jepsen.util 39 :ok :read [13 13] INFO [2022-01-10 12:53:07,059] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,060] jepsen worker 39 - jepsen.util 39 :ok :read [13 13] INFO [2022-01-10 12:53:07,061] jepsen worker 39 - jepsen.util 39 :invoke :write [13 45] INFO [2022-01-10 12:53:07,061] jepsen worker 38 - jepsen.util 38 :ok :write [12 25] INFO [2022-01-10 12:53:07,062] jepsen worker 87 - jepsen.util 87 :ok :write [29 38] INFO [2022-01-10 12:53:07,062] jepsen worker 38 - jepsen.util 38 :invoke :write [12 42] INFO [2022-01-10 12:53:07,063] jepsen worker 87 - jepsen.util 87 :invoke :write [29 9] INFO [2022-01-10 12:53:07,066] jepsen worker 1 - jepsen.util 1 :ok :write [0 25] INFO [2022-01-10 12:53:07,067] jepsen worker 1 - jepsen.util 1 :invoke :write [0 16] INFO [2022-01-10 12:53:07,071] jepsen worker 84 - jepsen.util 84 :ok :write [28 30] INFO [2022-01-10 12:53:07,072] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,073] jepsen worker 84 - jepsen.util 84 :ok :read [28 30] INFO [2022-01-10 12:53:07,073] jepsen worker 84 - jepsen.util 84 :invoke :write [28 5] INFO [2022-01-10 12:53:07,075] jepsen worker 3 - jepsen.util 3 :ok :write [1 42] INFO [2022-01-10 12:53:07,076] jepsen worker 3 - jepsen.util 3 :invoke :write [1 35] INFO [2022-01-10 12:53:07,080] jepsen worker 87 - jepsen.util 87 :ok :write [29 9] INFO [2022-01-10 12:53:07,080] jepsen worker 38 - jepsen.util 38 :ok :write [12 42] INFO [2022-01-10 12:53:07,081] jepsen worker 38 - jepsen.util 38 :invoke :write [12 17] INFO [2022-01-10 12:53:07,081] jepsen worker 87 - jepsen.util 87 :invoke :write [29 10] INFO [2022-01-10 12:53:07,087] jepsen worker 39 - jepsen.util 39 :ok :write [13 45] INFO [2022-01-10 12:53:07,088] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,090] jepsen worker 39 - jepsen.util 39 :ok :read [13 45] INFO [2022-01-10 12:53:07,090] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,093] jepsen worker 1 - jepsen.util 1 :ok :write [0 16] INFO [2022-01-10 12:53:07,094] jepsen worker 39 - jepsen.util 39 :ok :read [13 45] INFO [2022-01-10 12:53:07,094] jepsen worker 84 - jepsen.util 84 :ok :write [28 5] INFO [2022-01-10 12:53:07,094] jepsen worker 1 - jepsen.util 1 :invoke :write [0 6] INFO [2022-01-10 12:53:07,095] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,095] jepsen worker 3 - jepsen.util 3 :ok :write [1 35] INFO [2022-01-10 12:53:07,095] jepsen worker 84 - jepsen.util 84 :invoke :write [28 5] INFO [2022-01-10 12:53:07,096] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,097] jepsen worker 3 - jepsen.util 3 :ok :read [1 35] INFO [2022-01-10 12:53:07,097] jepsen worker 87 - jepsen.util 87 :ok :write [29 10] INFO [2022-01-10 12:53:07,098] jepsen worker 3 - jepsen.util 3 :invoke :write [1 11] INFO [2022-01-10 12:53:07,099] jepsen worker 87 - jepsen.util 87 :invoke :write [29 18] INFO [2022-01-10 12:53:07,099] jepsen worker 39 - jepsen.util 39 :ok :read [13 45] INFO [2022-01-10 12:53:07,099] jepsen worker 39 - jepsen.util 39 :invoke :write [13 34] INFO [2022-01-10 12:53:07,103] jepsen worker 38 - jepsen.util 38 :ok :write [12 17] INFO [2022-01-10 12:53:07,103] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:07,110] jepsen worker 87 - jepsen.util 87 :ok :write [29 18] INFO [2022-01-10 12:53:07,111] jepsen worker nemesis - jepsen.util :nemesis :info :kill-process nil INFO [2022-01-10 12:53:07,112] jepsen worker 87 - jepsen.util 87 :invoke :write [29 32] INFO [2022-01-10 12:53:07,112] jepsen worker 84 - jepsen.util 84 :ok :write [28 5] INFO [2022-01-10 12:53:07,113] jepsen worker 3 - jepsen.util 3 :ok :write [1 11] INFO [2022-01-10 12:53:07,113] jepsen worker 84 - jepsen.util 84 :invoke :write [28 14] INFO [2022-01-10 12:53:07,114] jepsen worker 3 - jepsen.util 3 :invoke :write [1 37] INFO [2022-01-10 12:53:07,118] jepsen worker 1 - jepsen.util 1 :ok :write [0 6] INFO [2022-01-10 12:53:07,119] jepsen worker 1 - jepsen.util 1 :invoke :write [0 21] INFO [2022-01-10 12:53:07,120] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:07,121] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,122] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:07,123] jepsen worker 38 - jepsen.util 38 :invoke :write [12 24] INFO [2022-01-10 12:53:07,124] jepsen worker 39 - jepsen.util 39 :ok :write [13 34] INFO [2022-01-10 12:53:07,125] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,126] jepsen worker 39 - jepsen.util 39 :ok :read [13 34] INFO [2022-01-10 12:53:07,126] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,127] jepsen worker 87 - jepsen.util 87 :ok :write [29 32] INFO [2022-01-10 12:53:07,128] jepsen worker 39 - jepsen.util 39 :ok :read [13 34] INFO [2022-01-10 12:53:07,128] jepsen worker 87 - jepsen.util 87 :invoke :write [29 1] INFO [2022-01-10 12:53:07,128] jepsen worker 39 - jepsen.util 39 :invoke :write [13 38] INFO [2022-01-10 12:53:07,130] jepsen worker 3 - jepsen.util 3 :ok :write [1 37] INFO [2022-01-10 12:53:07,130] jepsen worker 3 - jepsen.util 3 :invoke :write [1 33] INFO [2022-01-10 12:53:07,131] jepsen worker 84 - jepsen.util 84 :ok :write [28 14] INFO [2022-01-10 12:53:07,131] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,132] jepsen worker 84 - jepsen.util 84 :ok :read [28 14] INFO [2022-01-10 12:53:07,133] jepsen worker 84 - jepsen.util 84 :invoke :write [28 32] INFO [2022-01-10 12:53:07,207] jepsen worker 23 - jepsen.util 23 :ok :write [7 17] INFO [2022-01-10 12:53:07,207] jepsen worker 13 - jepsen.util 13 :ok :write [4 40] INFO [2022-01-10 12:53:07,207] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:07,208] jepsen worker 23 - jepsen.util 23 :invoke :write [7 11] INFO [2022-01-10 12:53:07,208] jepsen worker 13 - jepsen.util 13 :ok :read [4 40] INFO [2022-01-10 12:53:07,209] jepsen worker 13 - jepsen.util 13 :invoke :write [4 14] INFO [2022-01-10 12:53:07,215] jepsen worker 71 - jepsen.util 71 :ok :write [23 11] INFO [2022-01-10 12:53:07,216] jepsen worker 7 - jepsen.util 7 :ok :write [2 10] INFO [2022-01-10 12:53:07,216] jepsen worker 71 - jepsen.util 71 :invoke :write [23 24] INFO [2022-01-10 12:53:07,217] jepsen worker 7 - jepsen.util 7 :invoke :write [2 30] INFO [2022-01-10 12:53:07,223] jepsen worker 78 - jepsen.util 78 :ok :write [26 45] INFO [2022-01-10 12:53:07,223] jepsen worker 38 - jepsen.util 38 :ok :write [12 24] INFO [2022-01-10 12:53:07,224] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:07,224] jepsen worker 44 - jepsen.util 44 :ok :write [14 49] INFO [2022-01-10 12:53:07,224] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,225] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:07,225] jepsen worker 78 - jepsen.util 78 :ok :read [26 45] INFO [2022-01-10 12:53:07,225] jepsen worker 38 - jepsen.util 38 :ok :read [12 24] INFO [2022-01-10 12:53:07,225] jepsen worker 1 - jepsen.util 1 :ok :write [0 21] INFO [2022-01-10 12:53:07,226] jepsen worker 44 - jepsen.util 44 :ok :read [14 49] INFO [2022-01-10 12:53:07,226] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,227] jepsen worker 78 - jepsen.util 78 :invoke :write [26 16] INFO [2022-01-10 12:53:07,227] jepsen worker 1 - jepsen.util 1 :ok :read [0 21] INFO [2022-01-10 12:53:07,228] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:07,228] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,228] jepsen worker 44 - jepsen.util 44 :ok :read [14 49] INFO [2022-01-10 12:53:07,229] jepsen worker 1 - jepsen.util 1 :invoke :write [0 1] INFO [2022-01-10 12:53:07,229] jepsen worker 38 - jepsen.util 38 :ok :read [12 24] INFO [2022-01-10 12:53:07,230] jepsen worker 51 - jepsen.util 51 :ok :write [17 24] INFO [2022-01-10 12:53:07,230] jepsen worker 38 - jepsen.util 38 :invoke :write [12 19] INFO [2022-01-10 12:53:07,231] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:07,231] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:07,232] jepsen worker 3 - jepsen.util 3 :ok :write [1 33] INFO [2022-01-10 12:53:07,232] jepsen worker 51 - jepsen.util 51 :ok :read [17 24] INFO [2022-01-10 12:53:07,232] jepsen worker 60 - jepsen.util 60 :ok :write [20 18] INFO [2022-01-10 12:53:07,232] jepsen worker 44 - jepsen.util 44 :ok :read [14 49] INFO [2022-01-10 12:53:07,232] jepsen worker 39 - jepsen.util 39 :ok :write [13 38] INFO [2022-01-10 12:53:07,233] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:07,233] jepsen worker 44 - jepsen.util 44 :ok :read [14 49] INFO [2022-01-10 12:53:07,234] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:07,235] jepsen worker 44 - jepsen.util 44 :invoke :write [14 32] INFO [2022-01-10 12:53:07,235] jepsen worker 60 - jepsen.util 60 :ok :read [20 18] INFO [2022-01-10 12:53:07,235] jepsen worker 3 - jepsen.util 3 :invoke :write [1 31] INFO [2022-01-10 12:53:07,236] jepsen worker 87 - jepsen.util 87 :ok :write [29 1] INFO [2022-01-10 12:53:07,236] jepsen worker 51 - jepsen.util 51 :invoke :write [17 5] INFO [2022-01-10 12:53:07,237] jepsen worker 60 - jepsen.util 60 :invoke :write [20 42] INFO [2022-01-10 12:53:07,238] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,238] jepsen worker 87 - jepsen.util 87 :invoke :write [29 3] INFO [2022-01-10 12:53:07,239] jepsen worker 39 - jepsen.util 39 :ok :read [13 38] INFO [2022-01-10 12:53:07,239] jepsen worker 39 - jepsen.util 39 :invoke :write [13 44] INFO [2022-01-10 12:53:07,245] jepsen worker 84 - jepsen.util 84 :ok :write [28 32] INFO [2022-01-10 12:53:07,245] jepsen worker 68 - jepsen.util 68 :ok :write [22 25] INFO [2022-01-10 12:53:07,246] jepsen worker 75 - jepsen.util 75 :ok :write [25 0] INFO [2022-01-10 12:53:07,246] jepsen worker 84 - jepsen.util 84 :invoke :write [28 18] INFO [2022-01-10 12:53:07,246] jepsen worker 13 - jepsen.util 13 :ok :write [4 14] INFO [2022-01-10 12:53:07,247] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:07,247] jepsen worker 13 - jepsen.util 13 :invoke :write [4 9] INFO [2022-01-10 12:53:07,248] jepsen worker 75 - jepsen.util 75 :ok :read [25 0] INFO [2022-01-10 12:53:07,248] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:07,248] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:07,249] jepsen worker 75 - jepsen.util 75 :ok :read [25 0] INFO [2022-01-10 12:53:07,250] jepsen worker 68 - jepsen.util 68 :ok :read [22 25] INFO [2022-01-10 12:53:07,250] jepsen worker 75 - jepsen.util 75 :invoke :write [25 23] INFO [2022-01-10 12:53:07,250] jepsen worker 68 - jepsen.util 68 :invoke :write [22 1] INFO [2022-01-10 12:53:07,253] jepsen worker 71 - jepsen.util 71 :ok :write [23 24] INFO [2022-01-10 12:53:07,254] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:07,255] jepsen worker 71 - jepsen.util 71 :ok :read [23 24] INFO [2022-01-10 12:53:07,255] jepsen worker 71 - jepsen.util 71 :invoke :write [23 11] INFO [2022-01-10 12:53:07,256] jepsen worker 1 - jepsen.util 1 :ok :write [0 1] INFO [2022-01-10 12:53:07,256] jepsen worker 50 - jepsen.util 50 :ok :write [16 33] INFO [2022-01-10 12:53:07,256] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,257] jepsen worker 1 - jepsen.util 1 :ok :read [0 1] INFO [2022-01-10 12:53:07,257] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:07,258] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,258] jepsen worker 50 - jepsen.util 50 :ok :read [16 33] INFO [2022-01-10 12:53:07,259] jepsen worker 1 - jepsen.util 1 :ok :read [0 1] INFO [2022-01-10 12:53:07,259] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:07,259] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,259] jepsen worker 50 - jepsen.util 50 :ok :read [16 33] INFO [2022-01-10 12:53:07,260] jepsen worker 50 - jepsen.util 50 :invoke :write [16 39] INFO [2022-01-10 12:53:07,260] jepsen worker 1 - jepsen.util 1 :ok :read [0 1] INFO [2022-01-10 12:53:07,261] jepsen worker 1 - jepsen.util 1 :invoke :write [0 47] INFO [2022-01-10 12:53:07,261] jepsen worker 15 - jepsen.util 15 :ok :write [5 23] INFO [2022-01-10 12:53:07,262] jepsen worker 15 - jepsen.util 15 :invoke :write [5 34] INFO [2022-01-10 12:53:07,263] jepsen worker 78 - jepsen.util 78 :ok :write [26 16] INFO [2022-01-10 12:53:07,264] jepsen worker 78 - jepsen.util 78 :invoke :write [26 26] INFO [2022-01-10 12:53:07,268] jepsen worker 33 - jepsen.util 33 :ok :write [11 13] INFO [2022-01-10 12:53:07,269] jepsen worker 33 - jepsen.util 33 :invoke :write [11 36] INFO [2022-01-10 12:53:07,270] jepsen worker 38 - jepsen.util 38 :ok :write [12 19] INFO [2022-01-10 12:53:07,270] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,271] jepsen worker 38 - jepsen.util 38 :ok :read [12 19] INFO [2022-01-10 12:53:07,272] jepsen worker 38 - jepsen.util 38 :invoke :write [12 7] INFO [2022-01-10 12:53:07,276] jepsen worker 81 - jepsen.util 81 :ok :write [27 47] INFO [2022-01-10 12:53:07,276] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:07,277] jepsen worker 51 - jepsen.util 51 :ok :write [17 5] INFO [2022-01-10 12:53:07,277] jepsen worker 81 - jepsen.util 81 :ok :read [27 47] INFO [2022-01-10 12:53:07,277] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:07,278] jepsen worker 81 - jepsen.util 81 :invoke :write [27 31] INFO [2022-01-10 12:53:07,278] jepsen worker 51 - jepsen.util 51 :ok :read [17 5] INFO [2022-01-10 12:53:07,279] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:07,280] jepsen worker 51 - jepsen.util 51 :ok :read [17 5] INFO [2022-01-10 12:53:07,280] jepsen worker 51 - jepsen.util 51 :invoke :read [17 nil] INFO [2022-01-10 12:53:07,281] jepsen worker 51 - jepsen.util 51 :ok :read [17 5] INFO [2022-01-10 12:53:07,281] jepsen worker 51 - jepsen.util 51 :invoke :write [17 30] INFO [2022-01-10 12:53:07,283] jepsen worker 23 - jepsen.util 23 :ok :write [7 11] INFO [2022-01-10 12:53:07,284] jepsen worker 23 - jepsen.util 23 :invoke :write [7 8] INFO [2022-01-10 12:53:07,286] jepsen worker 13 - jepsen.util 13 :ok :write [4 9] INFO [2022-01-10 12:53:07,286] jepsen worker 13 - jepsen.util 13 :invoke :write [4 31] INFO [2022-01-10 12:53:07,289] jepsen worker 7 - jepsen.util 7 :ok :write [2 30] INFO [2022-01-10 12:53:07,290] jepsen worker 7 - jepsen.util 7 :invoke :write [2 46] INFO [2022-01-10 12:53:07,296] jepsen worker 71 - jepsen.util 71 :ok :write [23 11] INFO [2022-01-10 12:53:07,297] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:07,297] jepsen worker 44 - jepsen.util 44 :ok :write [14 32] INFO [2022-01-10 12:53:07,298] jepsen worker 44 - jepsen.util 44 :invoke :write [14 12] INFO [2022-01-10 12:53:07,298] jepsen worker 71 - jepsen.util 71 :ok :read [23 11] INFO [2022-01-10 12:53:07,299] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:07,300] jepsen worker 71 - jepsen.util 71 :ok :read [23 11] INFO [2022-01-10 12:53:07,300] jepsen worker 71 - jepsen.util 71 :invoke :write [23 0] INFO [2022-01-10 12:53:07,303] jepsen worker 78 - jepsen.util 78 :ok :write [26 26] INFO [2022-01-10 12:53:07,304] jepsen worker 78 - jepsen.util 78 :invoke :write [26 39] INFO [2022-01-10 12:53:07,307] jepsen worker 60 - jepsen.util 60 :ok :write [20 42] INFO [2022-01-10 12:53:07,307] jepsen worker 60 - jepsen.util 60 :invoke :write [20 38] INFO [2022-01-10 12:53:07,315] jepsen worker 51 - jepsen.util 51 :ok :write [17 30] INFO [2022-01-10 12:53:07,316] jepsen worker 51 - jepsen.util 51 :invoke :write [17 18] INFO [2022-01-10 12:53:07,320] jepsen worker 13 - jepsen.util 13 :ok :write [4 31] INFO [2022-01-10 12:53:07,321] jepsen worker 75 - jepsen.util 75 :ok :write [25 23] INFO [2022-01-10 12:53:07,321] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:07,321] jepsen worker 75 - jepsen.util 75 :invoke :write [25 24] INFO [2022-01-10 12:53:07,322] jepsen worker 13 - jepsen.util 13 :ok :read [4 31] INFO [2022-01-10 12:53:07,323] jepsen worker 13 - jepsen.util 13 :invoke :write [4 4] INFO [2022-01-10 12:53:07,327] jepsen worker 71 - jepsen.util 71 :ok :write [23 0] INFO [2022-01-10 12:53:07,327] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:07,328] jepsen worker 71 - jepsen.util 71 :ok :read [23 0] INFO [2022-01-10 12:53:07,329] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:07,329] jepsen worker 71 - jepsen.util 71 :ok :read [23 0] INFO [2022-01-10 12:53:07,330] jepsen worker 71 - jepsen.util 71 :invoke :write [23 0] INFO [2022-01-10 12:53:07,333] jepsen worker 78 - jepsen.util 78 :ok :write [26 39] INFO [2022-01-10 12:53:07,334] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:07,335] jepsen worker 78 - jepsen.util 78 :ok :read [26 39] INFO [2022-01-10 12:53:07,335] jepsen worker 78 - jepsen.util 78 :invoke :write [26 34] INFO [2022-01-10 12:53:07,339] jepsen worker 53 - jepsen.util 53 :ok :write [17 8] INFO [2022-01-10 12:53:07,340] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:53:07,341] jepsen worker 53 - jepsen.util 53 :ok :read [17 8] INFO [2022-01-10 12:53:07,342] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:53:07,343] jepsen worker 53 - jepsen.util 53 :ok :read [17 8] INFO [2022-01-10 12:53:07,343] jepsen worker 53 - jepsen.util 53 :invoke :write [17 20] INFO [2022-01-10 12:53:07,359] jepsen worker 3 - jepsen.util 3 :ok :write [1 31] INFO [2022-01-10 12:53:07,360] jepsen worker 3 - jepsen.util 3 :invoke :write [1 10] INFO [2022-01-10 12:53:07,360] jepsen worker 87 - jepsen.util 87 :ok :write [29 3] INFO [2022-01-10 12:53:07,361] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,362] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:07,363] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,364] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:07,365] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,365] jepsen worker 86 - jepsen.util 86 :ok :write [28 5] INFO [2022-01-10 12:53:07,365] jepsen worker 68 - jepsen.util 68 :ok :write [22 1] INFO [2022-01-10 12:53:07,365] jepsen worker 68 - jepsen.util 68 :invoke :write [22 3] INFO [2022-01-10 12:53:07,365] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:07,366] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,366] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,367] jepsen worker 86 - jepsen.util 86 :ok :read [28 5] INFO [2022-01-10 12:53:07,367] jepsen worker 86 - jepsen.util 86 :invoke :write [28 9] INFO [2022-01-10 12:53:07,368] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:07,368] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,369] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:07,370] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,370] jepsen worker 38 - jepsen.util 38 :ok :write [12 7] INFO [2022-01-10 12:53:07,371] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,371] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:07,371] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,372] jepsen worker 38 - jepsen.util 38 :ok :read [12 7] INFO [2022-01-10 12:53:07,372] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:07,372] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,373] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,373] jepsen worker 38 - jepsen.util 38 :ok :read [12 7] INFO [2022-01-10 12:53:07,374] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:07,374] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,375] jepsen worker 87 - jepsen.util 87 :invoke :write [29 3] INFO [2022-01-10 12:53:07,375] jepsen worker 38 - jepsen.util 38 :ok :read [12 7] INFO [2022-01-10 12:53:07,375] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,376] jepsen worker 38 - jepsen.util 38 :ok :read [12 7] INFO [2022-01-10 12:53:07,376] jepsen worker 3 - jepsen.util 3 :ok :write [1 10] INFO [2022-01-10 12:53:07,377] jepsen worker 38 - jepsen.util 38 :invoke :write [12 7] INFO [2022-01-10 12:53:07,377] jepsen worker 3 - jepsen.util 3 :invoke :write [1 18] INFO [2022-01-10 12:53:07,378] jepsen worker 39 - jepsen.util 39 :ok :write [13 44] INFO [2022-01-10 12:53:07,379] jepsen worker 39 - jepsen.util 39 :invoke :write [13 0] INFO [2022-01-10 12:53:07,381] jepsen worker 86 - jepsen.util 86 :ok :write [28 9] INFO [2022-01-10 12:53:07,381] jepsen worker 86 - jepsen.util 86 :invoke :write [28 4] INFO [2022-01-10 12:53:07,384] jepsen worker 1 - jepsen.util 1 :ok :write [0 47] INFO [2022-01-10 12:53:07,385] jepsen worker 1 - jepsen.util 1 :invoke :write [0 49] INFO [2022-01-10 12:53:07,390] jepsen worker 50 - jepsen.util 50 :ok :write [16 39] INFO [2022-01-10 12:53:07,390] jepsen worker 41 - jepsen.util 41 :fail :read [13 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:07,391] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:53:07,392] jepsen worker 41 - jepsen.util 41 :invoke :write [13 8] INFO [2022-01-10 12:53:07,393] jepsen worker 38 - jepsen.util 38 :ok :write [12 7] INFO [2022-01-10 12:53:07,394] jepsen worker 38 - jepsen.util 38 :invoke :write [12 33] INFO [2022-01-10 12:53:07,398] jepsen worker 3 - jepsen.util 3 :ok :write [1 18] INFO [2022-01-10 12:53:07,399] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,400] jepsen worker 3 - jepsen.util 3 :ok :read [1 18] INFO [2022-01-10 12:53:07,401] jepsen worker 3 - jepsen.util 3 :invoke :write [1 6] INFO [2022-01-10 12:53:07,406] jepsen worker 86 - jepsen.util 86 :ok :write [28 4] INFO [2022-01-10 12:53:07,406] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,412] jepsen worker 38 - jepsen.util 38 :ok :write [12 33] INFO [2022-01-10 12:53:07,413] jepsen worker 38 - jepsen.util 38 :invoke :write [12 3] INFO [2022-01-10 12:53:07,418] jepsen worker 3 - jepsen.util 3 :ok :write [1 6] INFO [2022-01-10 12:53:07,419] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,420] jepsen worker 3 - jepsen.util 3 :ok :read [1 6] INFO [2022-01-10 12:53:07,420] jepsen worker 87 - jepsen.util 87 :ok :write [29 3] INFO [2022-01-10 12:53:07,420] jepsen worker 3 - jepsen.util 3 :invoke :write [1 45] INFO [2022-01-10 12:53:07,421] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,422] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:07,422] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,424] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:07,425] jepsen worker 87 - jepsen.util 87 :invoke :write [29 12] INFO [2022-01-10 12:53:07,429] jepsen worker 38 - jepsen.util 38 :ok :write [12 3] INFO [2022-01-10 12:53:07,430] jepsen worker 38 - jepsen.util 38 :invoke :write [12 39] INFO [2022-01-10 12:53:07,435] jepsen worker 39 - jepsen.util 39 :ok :write [13 0] INFO [2022-01-10 12:53:07,435] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,436] jepsen worker 3 - jepsen.util 3 :ok :write [1 45] INFO [2022-01-10 12:53:07,437] jepsen worker 3 - jepsen.util 3 :invoke :write [1 8] INFO [2022-01-10 12:53:07,440] jepsen worker 84 - jepsen.util 84 :ok :write [28 18] INFO [2022-01-10 12:53:07,441] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,442] jepsen worker 84 - jepsen.util 84 :ok :read [28 18] INFO [2022-01-10 12:53:07,443] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,443] jepsen worker 1 - jepsen.util 1 :ok :write [0 49] INFO [2022-01-10 12:53:07,443] jepsen worker 84 - jepsen.util 84 :ok :read [28 18] INFO [2022-01-10 12:53:07,444] jepsen worker 1 - jepsen.util 1 :invoke :write [0 49] INFO [2022-01-10 12:53:07,444] jepsen worker 84 - jepsen.util 84 :invoke :write [28 2] INFO [2022-01-10 12:53:07,445] jepsen worker 38 - jepsen.util 38 :ok :write [12 39] INFO [2022-01-10 12:53:07,446] jepsen worker 38 - jepsen.util 38 :invoke :write [12 32] INFO [2022-01-10 12:53:07,451] jepsen worker 3 - jepsen.util 3 :ok :write [1 8] INFO [2022-01-10 12:53:07,451] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,452] jepsen worker 3 - jepsen.util 3 :ok :read [1 8] INFO [2022-01-10 12:53:07,453] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,454] jepsen worker 3 - jepsen.util 3 :ok :read [1 8] INFO [2022-01-10 12:53:07,455] jepsen worker 3 - jepsen.util 3 :invoke :write [1 9] INFO [2022-01-10 12:53:07,455] jepsen worker 13 - jepsen.util 13 :ok :write [4 4] INFO [2022-01-10 12:53:07,455] jepsen worker 15 - jepsen.util 15 :ok :write [5 34] INFO [2022-01-10 12:53:07,456] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:07,456] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:07,457] jepsen worker 13 - jepsen.util 13 :ok :read [4 4] INFO [2022-01-10 12:53:07,457] jepsen worker 15 - jepsen.util 15 :ok :read [5 34] INFO [2022-01-10 12:53:07,458] jepsen worker 13 - jepsen.util 13 :invoke :write [4 45] INFO [2022-01-10 12:53:07,458] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:07,459] jepsen worker 87 - jepsen.util 87 :ok :write [29 12] INFO [2022-01-10 12:53:07,460] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,460] jepsen worker 15 - jepsen.util 15 :ok :read [5 34] INFO [2022-01-10 12:53:07,460] jepsen worker 84 - jepsen.util 84 :ok :write [28 2] INFO [2022-01-10 12:53:07,461] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:07,461] jepsen worker 33 - jepsen.util 33 :ok :write [11 36] INFO [2022-01-10 12:53:07,461] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:07,462] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,462] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,463] jepsen worker 33 - jepsen.util 33 :invoke :write [11 3] INFO [2022-01-10 12:53:07,463] jepsen worker 84 - jepsen.util 84 :ok :read [28 2] INFO [2022-01-10 12:53:07,463] jepsen worker 84 - jepsen.util 84 :invoke :write [28 23] INFO [2022-01-10 12:53:07,463] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:07,463] jepsen worker 71 - jepsen.util 71 :ok :write [23 0] INFO [2022-01-10 12:53:07,464] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,464] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:07,465] jepsen worker 15 - jepsen.util 15 :ok :read [5 34] INFO [2022-01-10 12:53:07,465] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:07,465] jepsen worker 71 - jepsen.util 71 :ok :read [23 0] INFO [2022-01-10 12:53:07,466] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:07,467] jepsen worker 71 - jepsen.util 71 :invoke :write [23 47] INFO [2022-01-10 12:53:07,467] jepsen worker 87 - jepsen.util 87 :invoke :write [29 9] INFO [2022-01-10 12:53:07,468] jepsen worker 15 - jepsen.util 15 :ok :read [5 34] INFO [2022-01-10 12:53:07,468] jepsen worker 15 - jepsen.util 15 :invoke :write [5 21] INFO [2022-01-10 12:53:07,469] jepsen worker 81 - jepsen.util 81 :ok :write [27 31] INFO [2022-01-10 12:53:07,469] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:07,472] jepsen worker 78 - jepsen.util 78 :ok :write [26 34] INFO [2022-01-10 12:53:07,473] jepsen worker 86 - jepsen.util 86 :ok :read [28 2] INFO [2022-01-10 12:53:07,472] jepsen worker 39 - jepsen.util 39 :ok :read [13 0] INFO [2022-01-10 12:53:07,473] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:07,474] jepsen worker 86 - jepsen.util 86 :invoke :write [28 24] INFO [2022-01-10 12:53:07,474] jepsen worker 39 - jepsen.util 39 :invoke :write [13 43] INFO [2022-01-10 12:53:07,474] jepsen worker 78 - jepsen.util 78 :ok :read [26 34] INFO [2022-01-10 12:53:07,474] jepsen worker 81 - jepsen.util 81 :ok :read [27 31] INFO [2022-01-10 12:53:07,474] jepsen worker 1 - jepsen.util 1 :ok :write [0 49] INFO [2022-01-10 12:53:07,475] jepsen worker 23 - jepsen.util 23 :ok :write [7 8] INFO [2022-01-10 12:53:07,475] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:07,475] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,476] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:07,476] jepsen worker 81 - jepsen.util 81 :ok :read [27 31] INFO [2022-01-10 12:53:07,476] jepsen worker 23 - jepsen.util 23 :invoke :write [7 28] INFO [2022-01-10 12:53:07,476] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:07,476] jepsen worker 53 - jepsen.util 53 :ok :write [17 20] INFO [2022-01-10 12:53:07,477] jepsen worker 81 - jepsen.util 81 :invoke :write [27 46] INFO [2022-01-10 12:53:07,477] jepsen worker 78 - jepsen.util 78 :ok :read [26 34] INFO [2022-01-10 12:53:07,477] jepsen worker 78 - jepsen.util 78 :invoke :write [26 2] INFO [2022-01-10 12:53:07,478] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,478] jepsen worker 53 - jepsen.util 53 :invoke :write [17 0] INFO [2022-01-10 12:53:07,478] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:07,479] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,480] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:07,480] jepsen worker 1 - jepsen.util 1 :invoke :write [0 11] INFO [2022-01-10 12:53:07,483] jepsen worker 7 - jepsen.util 7 :ok :write [2 46] INFO [2022-01-10 12:53:07,483] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:07,484] jepsen worker 7 - jepsen.util 7 :ok :read [2 46] INFO [2022-01-10 12:53:07,484] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:07,485] jepsen worker 7 - jepsen.util 7 :ok :read [2 46] INFO [2022-01-10 12:53:07,486] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:07,487] jepsen worker 7 - jepsen.util 7 :ok :read [2 46] INFO [2022-01-10 12:53:07,487] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:07,489] jepsen worker 7 - jepsen.util 7 :ok :read [2 46] INFO [2022-01-10 12:53:07,489] jepsen worker 7 - jepsen.util 7 :invoke :write [2 13] INFO [2022-01-10 12:53:07,491] jepsen worker 44 - jepsen.util 44 :ok :write [14 12] INFO [2022-01-10 12:53:07,491] jepsen worker 44 - jepsen.util 44 :invoke :write [14 7] INFO [2022-01-10 12:53:07,499] jepsen worker 60 - jepsen.util 60 :ok :write [20 38] INFO [2022-01-10 12:53:07,500] jepsen worker 60 - jepsen.util 60 :invoke :write [20 16] INFO [2022-01-10 12:53:07,506] jepsen worker 75 - jepsen.util 75 :ok :write [25 24] INFO [2022-01-10 12:53:07,507] jepsen worker 75 - jepsen.util 75 :invoke :write [25 20] INFO [2022-01-10 12:53:07,512] jepsen worker 68 - jepsen.util 68 :ok :write [22 3] INFO [2022-01-10 12:53:07,512] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:07,513] jepsen worker 68 - jepsen.util 68 :ok :read [22 3] INFO [2022-01-10 12:53:07,513] jepsen worker 68 - jepsen.util 68 :invoke :write [22 12] INFO [2022-01-10 12:53:07,613] jepsen worker 38 - jepsen.util 38 :ok :write [12 32] INFO [2022-01-10 12:53:07,614] jepsen worker 38 - jepsen.util 38 :invoke :write [12 6] INFO [2022-01-10 12:53:07,619] jepsen worker 87 - jepsen.util 87 :ok :write [29 9] INFO [2022-01-10 12:53:07,620] jepsen worker 87 - jepsen.util 87 :invoke :write [29 29] INFO [2022-01-10 12:53:07,621] jepsen worker 3 - jepsen.util 3 :ok :write [1 9] INFO [2022-01-10 12:53:07,622] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,623] jepsen worker 3 - jepsen.util 3 :ok :read [1 9] INFO [2022-01-10 12:53:07,623] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,624] jepsen worker 3 - jepsen.util 3 :ok :read [1 9] INFO [2022-01-10 12:53:07,624] jepsen worker 3 - jepsen.util 3 :invoke :write [1 43] INFO [2022-01-10 12:53:07,626] jepsen worker 1 - jepsen.util 1 :ok :write [0 11] INFO [2022-01-10 12:53:07,626] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,627] jepsen worker 1 - jepsen.util 1 :ok :read [0 11] INFO [2022-01-10 12:53:07,627] jepsen worker 1 - jepsen.util 1 :invoke :write [0 39] INFO [2022-01-10 12:53:07,628] jepsen worker 13 - jepsen.util 13 :ok :write [4 45] INFO [2022-01-10 12:53:07,629] jepsen worker 13 - jepsen.util 13 :invoke :write [4 33] INFO [2022-01-10 12:53:07,636] jepsen worker 84 - jepsen.util 84 :ok :write [28 23] INFO [2022-01-10 12:53:07,637] jepsen worker 39 - jepsen.util 39 :ok :write [13 43] INFO [2022-01-10 12:53:07,637] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,637] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,638] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:07,638] jepsen worker 39 - jepsen.util 39 :ok :read [13 43] INFO [2022-01-10 12:53:07,639] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,639] jepsen worker 39 - jepsen.util 39 :invoke :write [13 10] INFO [2022-01-10 12:53:07,640] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:07,640] jepsen worker 84 - jepsen.util 84 :invoke :write [28 7] INFO [2022-01-10 12:53:07,642] jepsen worker 71 - jepsen.util 71 :ok :write [23 47] INFO [2022-01-10 12:53:07,643] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:07,644] jepsen worker 71 - jepsen.util 71 :ok :read [23 47] INFO [2022-01-10 12:53:07,644] jepsen worker 71 - jepsen.util 71 :invoke :write [23 5] INFO [2022-01-10 12:53:07,649] jepsen worker 78 - jepsen.util 78 :ok :write [26 2] INFO [2022-01-10 12:53:07,650] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:07,651] jepsen worker 78 - jepsen.util 78 :ok :read [26 2] INFO [2022-01-10 12:53:07,651] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:07,652] jepsen worker 78 - jepsen.util 78 :ok :read [26 2] INFO [2022-01-10 12:53:07,653] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:07,654] jepsen worker 78 - jepsen.util 78 :ok :read [26 2] INFO [2022-01-10 12:53:07,654] jepsen worker 78 - jepsen.util 78 :invoke :write [26 24] INFO [2022-01-10 12:53:07,657] jepsen worker 87 - jepsen.util 87 :ok :write [29 29] INFO [2022-01-10 12:53:07,657] jepsen worker 87 - jepsen.util 87 :invoke :write [29 37] INFO [2022-01-10 12:53:07,659] jepsen worker 53 - jepsen.util 53 :ok :write [17 0] INFO [2022-01-10 12:53:07,659] jepsen worker 53 - jepsen.util 53 :invoke :write [17 18] INFO [2022-01-10 12:53:07,663] jepsen worker 1 - jepsen.util 1 :ok :write [0 39] INFO [2022-01-10 12:53:07,663] jepsen worker 1 - jepsen.util 1 :invoke :write [0 7] INFO [2022-01-10 12:53:07,666] jepsen worker 38 - jepsen.util 38 :ok :write [12 6] INFO [2022-01-10 12:53:07,667] jepsen worker 38 - jepsen.util 38 :invoke :write [12 34] INFO [2022-01-10 12:53:07,669] jepsen worker 39 - jepsen.util 39 :ok :write [13 10] INFO [2022-01-10 12:53:07,670] jepsen worker 39 - jepsen.util 39 :invoke :write [13 10] INFO [2022-01-10 12:53:07,673] jepsen worker 3 - jepsen.util 3 :ok :write [1 43] INFO [2022-01-10 12:53:07,674] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,674] jepsen worker 87 - jepsen.util 87 :ok :write [29 37] INFO [2022-01-10 12:53:07,675] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,676] jepsen worker 87 - jepsen.util 87 :ok :read [29 37] INFO [2022-01-10 12:53:07,676] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:07,677] jepsen worker 87 - jepsen.util 87 :invoke :write [29 39] INFO [2022-01-10 12:53:07,678] jepsen worker 3 - jepsen.util 3 :invoke :write [1 39] INFO [2022-01-10 12:53:07,682] jepsen worker 1 - jepsen.util 1 :ok :write [0 7] INFO [2022-01-10 12:53:07,683] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,684] jepsen worker 1 - jepsen.util 1 :ok :read [0 7] INFO [2022-01-10 12:53:07,684] jepsen worker 1 - jepsen.util 1 :invoke :write [0 22] INFO [2022-01-10 12:53:07,686] jepsen worker 39 - jepsen.util 39 :ok :write [13 10] INFO [2022-01-10 12:53:07,686] jepsen worker 84 - jepsen.util 84 :ok :write [28 7] INFO [2022-01-10 12:53:07,687] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,688] jepsen worker 39 - jepsen.util 39 :invoke :write [13 34] INFO [2022-01-10 12:53:07,688] jepsen worker 84 - jepsen.util 84 :ok :read [28 7] INFO [2022-01-10 12:53:07,689] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,690] jepsen worker 84 - jepsen.util 84 :ok :read [28 7] INFO [2022-01-10 12:53:07,690] jepsen worker 87 - jepsen.util 87 :ok :write [29 39] INFO [2022-01-10 12:53:07,691] jepsen worker 84 - jepsen.util 84 :invoke :write [28 22] INFO [2022-01-10 12:53:07,691] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,692] jepsen worker 87 - jepsen.util 87 :ok :read [29 39] INFO [2022-01-10 12:53:07,693] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,694] jepsen worker 87 - jepsen.util 87 :ok :read [29 39] INFO [2022-01-10 12:53:07,695] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,695] jepsen worker 87 - jepsen.util 87 :ok :read [29 39] INFO [2022-01-10 12:53:07,696] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,697] jepsen worker 87 - jepsen.util 87 :ok :read [29 39] INFO [2022-01-10 12:53:07,697] jepsen worker 1 - jepsen.util 1 :ok :write [0 22] INFO [2022-01-10 12:53:07,698] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,699] jepsen worker 1 - jepsen.util 1 :invoke :write [0 49] INFO [2022-01-10 12:53:07,699] jepsen worker 87 - jepsen.util 87 :ok :read [29 39] INFO [2022-01-10 12:53:07,700] jepsen worker 87 - jepsen.util 87 :invoke :write [29 15] INFO [2022-01-10 12:53:07,713] jepsen worker 38 - jepsen.util 38 :ok :write [12 34] INFO [2022-01-10 12:53:07,713] jepsen worker 38 - jepsen.util 38 :invoke :write [12 30] INFO [2022-01-10 12:53:07,719] jepsen worker 3 - jepsen.util 3 :ok :write [1 39] INFO [2022-01-10 12:53:07,720] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,721] jepsen worker 3 - jepsen.util 3 :ok :read [1 39] INFO [2022-01-10 12:53:07,722] jepsen worker 3 - jepsen.util 3 :invoke :write [1 42] INFO [2022-01-10 12:53:07,845] jepsen worker 39 - jepsen.util 39 :ok :write [13 34] INFO [2022-01-10 12:53:07,846] jepsen worker 84 - jepsen.util 84 :ok :write [28 22] INFO [2022-01-10 12:53:07,846] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,847] jepsen worker 39 - jepsen.util 39 :ok :read [13 34] INFO [2022-01-10 12:53:07,847] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,848] jepsen worker 39 - jepsen.util 39 :invoke :write [13 46] INFO [2022-01-10 12:53:07,848] jepsen worker 84 - jepsen.util 84 :ok :read [28 22] INFO [2022-01-10 12:53:07,849] jepsen worker 84 - jepsen.util 84 :invoke :write [28 2] INFO [2022-01-10 12:53:07,853] jepsen worker 38 - jepsen.util 38 :ok :write [12 30] INFO [2022-01-10 12:53:07,853] jepsen worker 87 - jepsen.util 87 :ok :write [29 15] INFO [2022-01-10 12:53:07,854] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,854] jepsen worker 87 - jepsen.util 87 :invoke :write [29 13] INFO [2022-01-10 12:53:07,855] jepsen worker 38 - jepsen.util 38 :ok :read [12 30] INFO [2022-01-10 12:53:07,855] jepsen worker 38 - jepsen.util 38 :invoke :write [12 26] INFO [2022-01-10 12:53:07,856] jepsen worker 1 - jepsen.util 1 :ok :write [0 49] INFO [2022-01-10 12:53:07,857] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,859] jepsen worker 3 - jepsen.util 3 :ok :write [1 42] INFO [2022-01-10 12:53:07,859] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,860] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:07,861] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,861] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:07,862] jepsen worker 3 - jepsen.util 3 :invoke :write [1 23] INFO [2022-01-10 12:53:07,863] jepsen worker 39 - jepsen.util 39 :ok :write [13 46] INFO [2022-01-10 12:53:07,864] jepsen worker 39 - jepsen.util 39 :invoke :write [13 34] INFO [2022-01-10 12:53:07,864] jepsen worker 84 - jepsen.util 84 :ok :write [28 2] INFO [2022-01-10 12:53:07,865] jepsen worker 84 - jepsen.util 84 :invoke :write [28 40] INFO [2022-01-10 12:53:07,872] jepsen worker 87 - jepsen.util 87 :ok :write [29 13] INFO [2022-01-10 12:53:07,872] jepsen worker 38 - jepsen.util 38 :ok :write [12 26] INFO [2022-01-10 12:53:07,872] jepsen worker 87 - jepsen.util 87 :invoke :write [29 25] INFO [2022-01-10 12:53:07,873] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,874] jepsen worker 38 - jepsen.util 38 :ok :read [12 26] INFO [2022-01-10 12:53:07,875] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,876] jepsen worker 38 - jepsen.util 38 :ok :read [12 26] INFO [2022-01-10 12:53:07,877] jepsen worker 38 - jepsen.util 38 :invoke :write [12 28] INFO [2022-01-10 12:53:07,879] jepsen worker 3 - jepsen.util 3 :ok :write [1 23] INFO [2022-01-10 12:53:07,880] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,880] jepsen worker 3 - jepsen.util 3 :ok :read [1 23] INFO [2022-01-10 12:53:07,881] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,881] jepsen worker 39 - jepsen.util 39 :ok :write [13 34] INFO [2022-01-10 12:53:07,882] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,882] jepsen worker 3 - jepsen.util 3 :ok :read [1 23] INFO [2022-01-10 12:53:07,882] jepsen worker 84 - jepsen.util 84 :ok :write [28 40] INFO [2022-01-10 12:53:07,882] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,883] jepsen worker 39 - jepsen.util 39 :ok :read [13 34] INFO [2022-01-10 12:53:07,883] jepsen worker 3 - jepsen.util 3 :invoke :write [1 46] INFO [2022-01-10 12:53:07,883] jepsen worker 39 - jepsen.util 39 :invoke :write [13 33] INFO [2022-01-10 12:53:07,883] jepsen worker 84 - jepsen.util 84 :ok :read [28 40] INFO [2022-01-10 12:53:07,884] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,885] jepsen worker 84 - jepsen.util 84 :ok :read [28 40] INFO [2022-01-10 12:53:07,886] jepsen worker 84 - jepsen.util 84 :invoke :write [28 7] INFO [2022-01-10 12:53:07,889] jepsen worker 87 - jepsen.util 87 :ok :write [29 25] INFO [2022-01-10 12:53:07,889] jepsen worker 87 - jepsen.util 87 :invoke :write [29 33] INFO [2022-01-10 12:53:07,890] jepsen worker 38 - jepsen.util 38 :ok :write [12 28] INFO [2022-01-10 12:53:07,891] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,892] jepsen worker 38 - jepsen.util 38 :ok :read [12 28] INFO [2022-01-10 12:53:07,892] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,893] jepsen worker 38 - jepsen.util 38 :ok :read [12 28] INFO [2022-01-10 12:53:07,893] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,894] jepsen worker 3 - jepsen.util 3 :ok :write [1 46] INFO [2022-01-10 12:53:07,894] jepsen worker 38 - jepsen.util 38 :ok :read [12 28] INFO [2022-01-10 12:53:07,895] jepsen worker 3 - jepsen.util 3 :invoke :write [1 21] INFO [2022-01-10 12:53:07,895] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,896] jepsen worker 38 - jepsen.util 38 :ok :read [12 28] INFO [2022-01-10 12:53:07,896] jepsen worker 38 - jepsen.util 38 :invoke :write [12 18] INFO [2022-01-10 12:53:07,898] jepsen worker 39 - jepsen.util 39 :ok :write [13 33] INFO [2022-01-10 12:53:07,898] jepsen worker 39 - jepsen.util 39 :invoke :write [13 9] INFO [2022-01-10 12:53:07,900] jepsen worker 87 - jepsen.util 87 :ok :write [29 33] INFO [2022-01-10 12:53:07,901] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,902] jepsen worker 87 - jepsen.util 87 :ok :read [29 33] INFO [2022-01-10 12:53:07,902] jepsen worker 87 - jepsen.util 87 :invoke :write [29 37] INFO [2022-01-10 12:53:07,904] jepsen worker 84 - jepsen.util 84 :ok :write [28 7] INFO [2022-01-10 12:53:07,905] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:07,906] jepsen worker 84 - jepsen.util 84 :ok :read [28 7] INFO [2022-01-10 12:53:07,906] jepsen worker 84 - jepsen.util 84 :invoke :write [28 1] INFO [2022-01-10 12:53:07,909] jepsen worker 40 - jepsen.util 40 :ok :write [13 35] INFO [2022-01-10 12:53:07,909] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,910] jepsen worker 40 - jepsen.util 40 :ok :read [13 35] INFO [2022-01-10 12:53:07,910] jepsen worker 36 - jepsen.util 36 :ok :write [12 6] INFO [2022-01-10 12:53:07,911] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,911] jepsen worker 36 - jepsen.util 36 :invoke :write [12 29] INFO [2022-01-10 12:53:07,912] jepsen worker 40 - jepsen.util 40 :ok :read [13 35] INFO [2022-01-10 12:53:07,912] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,913] jepsen worker 40 - jepsen.util 40 :ok :read [13 35] INFO [2022-01-10 12:53:07,914] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,915] jepsen worker 40 - jepsen.util 40 :ok :read [13 35] INFO [2022-01-10 12:53:07,916] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,917] jepsen worker 3 - jepsen.util 3 :ok :write [1 21] INFO [2022-01-10 12:53:07,917] jepsen worker 87 - jepsen.util 87 :ok :write [29 37] INFO [2022-01-10 12:53:07,918] jepsen worker 3 - jepsen.util 3 :invoke :write [1 42] INFO [2022-01-10 12:53:07,918] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:53:07,919] jepsen worker 40 - jepsen.util 40 :ok :read [13 35] INFO [2022-01-10 12:53:07,919] jepsen worker 87 - jepsen.util 87 :invoke :write [29 10] INFO [2022-01-10 12:53:07,920] jepsen worker 50 - jepsen.util 50 :invoke :write [16 21] INFO [2022-01-10 12:53:07,920] jepsen worker 40 - jepsen.util 40 :invoke :write [13 13] INFO [2022-01-10 12:53:07,921] jepsen worker 84 - jepsen.util 84 :ok :write [28 1] INFO [2022-01-10 12:53:07,921] jepsen worker 39 - jepsen.util 39 :ok :write [13 9] INFO [2022-01-10 12:53:07,923] jepsen worker 84 - jepsen.util 84 :invoke :write [28 35] INFO [2022-01-10 12:53:07,923] jepsen worker 39 - jepsen.util 39 :invoke :write [13 30] INFO [2022-01-10 12:53:07,924] jepsen worker 15 - jepsen.util 15 :ok :write [5 21] INFO [2022-01-10 12:53:07,925] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:07,926] jepsen worker 13 - jepsen.util 13 :ok :write [4 33] INFO [2022-01-10 12:53:07,926] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:07,926] jepsen worker 15 - jepsen.util 15 :ok :read [5 21] INFO [2022-01-10 12:53:07,927] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,927] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:07,928] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:07,928] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:07,928] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,928] jepsen worker 13 - jepsen.util 13 :ok :read [4 33] INFO [2022-01-10 12:53:07,929] jepsen worker 15 - jepsen.util 15 :ok :read [5 21] INFO [2022-01-10 12:53:07,929] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:07,930] jepsen worker 15 - jepsen.util 15 :invoke :write [5 16] INFO [2022-01-10 12:53:07,930] jepsen worker 33 - jepsen.util 33 :ok :write [11 3] INFO [2022-01-10 12:53:07,930] jepsen worker 36 - jepsen.util 36 :ok :write [12 29] INFO [2022-01-10 12:53:07,931] jepsen worker 13 - jepsen.util 13 :invoke :write [4 9] INFO [2022-01-10 12:53:07,931] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,932] jepsen worker 71 - jepsen.util 71 :ok :write [23 5] INFO [2022-01-10 12:53:07,932] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,933] jepsen worker 36 - jepsen.util 36 :ok :read [12 29] INFO [2022-01-10 12:53:07,933] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:07,933] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:07,934] jepsen worker 71 - jepsen.util 71 :ok :read [23 5] INFO [2022-01-10 12:53:07,934] jepsen worker 36 - jepsen.util 36 :invoke :write [12 15] INFO [2022-01-10 12:53:07,934] jepsen worker 3 - jepsen.util 3 :ok :write [1 42] INFO [2022-01-10 12:53:07,934] jepsen worker 71 - jepsen.util 71 :invoke :write [23 5] INFO [2022-01-10 12:53:07,935] jepsen worker 87 - jepsen.util 87 :ok :write [29 10] INFO [2022-01-10 12:53:07,935] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:07,936] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,937] jepsen worker 33 - jepsen.util 33 :ok :read [11 3] INFO [2022-01-10 12:53:07,937] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,938] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:07,938] jepsen worker 81 - jepsen.util 81 :ok :write [27 46] INFO [2022-01-10 12:53:07,938] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:07,939] jepsen worker 87 - jepsen.util 87 :ok :read [29 10] INFO [2022-01-10 12:53:07,940] jepsen worker 33 - jepsen.util 33 :ok :read [11 3] INFO [2022-01-10 12:53:07,940] jepsen worker 78 - jepsen.util 78 :ok :write [26 24] INFO [2022-01-10 12:53:07,940] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:07,941] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:07,941] jepsen worker 84 - jepsen.util 84 :ok :write [28 35] INFO [2022-01-10 12:53:07,941] jepsen worker 81 - jepsen.util 81 :ok :read [27 46] INFO [2022-01-10 12:53:07,942] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:07,942] jepsen worker 39 - jepsen.util 39 :ok :write [13 30] INFO [2022-01-10 12:53:07,942] jepsen worker 78 - jepsen.util 78 :ok :read [26 24] INFO [2022-01-10 12:53:07,943] jepsen worker 84 - jepsen.util 84 :invoke :write [28 31] INFO [2022-01-10 12:53:07,944] jepsen worker 87 - jepsen.util 87 :ok :read [29 10] INFO [2022-01-10 12:53:07,944] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,945] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:07,945] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:07,946] jepsen worker 33 - jepsen.util 33 :ok :read [11 3] INFO [2022-01-10 12:53:07,947] jepsen worker 23 - jepsen.util 23 :ok :write [7 28] INFO [2022-01-10 12:53:07,947] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,948] jepsen worker 39 - jepsen.util 39 :ok :read [13 30] INFO [2022-01-10 12:53:07,948] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,949] jepsen worker 78 - jepsen.util 78 :invoke :write [26 14] INFO [2022-01-10 12:53:07,949] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:07,950] jepsen worker 53 - jepsen.util 53 :ok :write [17 18] INFO [2022-01-10 12:53:07,950] jepsen worker 38 - jepsen.util 38 :ok :write [12 18] INFO [2022-01-10 12:53:07,950] jepsen worker 39 - jepsen.util 39 :invoke :write [13 30] INFO [2022-01-10 12:53:07,952] jepsen worker 87 - jepsen.util 87 :invoke :write [29 42] INFO [2022-01-10 12:53:07,953] jepsen worker 23 - jepsen.util 23 :invoke :write [7 21] INFO [2022-01-10 12:53:07,954] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:07,955] jepsen worker 33 - jepsen.util 33 :invoke :write [11 6] INFO [2022-01-10 12:53:07,955] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:07,956] jepsen worker 7 - jepsen.util 7 :ok :write [2 13] INFO [2022-01-10 12:53:07,957] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:07,958] jepsen worker 81 - jepsen.util 81 :invoke :write [27 30] INFO [2022-01-10 12:53:07,959] jepsen worker 38 - jepsen.util 38 :ok :read [12 18] INFO [2022-01-10 12:53:07,959] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:07,959] jepsen worker 13 - jepsen.util 13 :ok :write [4 9] INFO [2022-01-10 12:53:07,959] jepsen worker 1 - jepsen.util 1 :invoke :write [0 13] INFO [2022-01-10 12:53:07,960] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:07,960] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:07,960] jepsen worker 7 - jepsen.util 7 :ok :read [2 13] INFO [2022-01-10 12:53:07,961] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:07,962] jepsen worker 13 - jepsen.util 13 :invoke :write [4 49] INFO [2022-01-10 12:53:07,962] jepsen worker 3 - jepsen.util 3 :invoke :write [1 48] INFO [2022-01-10 12:53:07,963] jepsen worker 44 - jepsen.util 44 :ok :write [14 7] INFO [2022-01-10 12:53:07,963] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:07,964] jepsen worker 44 - jepsen.util 44 :invoke :write [14 23] INFO [2022-01-10 12:53:07,964] jepsen worker 71 - jepsen.util 71 :ok :write [23 5] INFO [2022-01-10 12:53:07,964] jepsen worker 7 - jepsen.util 7 :ok :read [2 13] INFO [2022-01-10 12:53:07,964] jepsen worker 53 - jepsen.util 53 :invoke :write [17 3] INFO [2022-01-10 12:53:07,965] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:07,965] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:07,966] jepsen worker 7 - jepsen.util 7 :ok :read [2 13] INFO [2022-01-10 12:53:07,966] jepsen worker 71 - jepsen.util 71 :ok :read [23 5] INFO [2022-01-10 12:53:07,967] jepsen worker 71 - jepsen.util 71 :invoke :write [23 20] INFO [2022-01-10 12:53:07,967] jepsen worker 7 - jepsen.util 7 :invoke :write [2 6] INFO [2022-01-10 12:53:07,970] jepsen worker 60 - jepsen.util 60 :ok :write [20 16] INFO [2022-01-10 12:53:07,970] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:07,971] jepsen worker 60 - jepsen.util 60 :ok :read [20 16] INFO [2022-01-10 12:53:07,971] jepsen worker 60 - jepsen.util 60 :invoke :write [20 17] INFO [2022-01-10 12:53:07,975] jepsen worker 75 - jepsen.util 75 :ok :write [25 20] INFO [2022-01-10 12:53:07,975] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:07,976] jepsen worker 75 - jepsen.util 75 :ok :read [25 20] INFO [2022-01-10 12:53:07,977] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:07,978] jepsen worker 75 - jepsen.util 75 :ok :read [25 20] INFO [2022-01-10 12:53:07,979] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:07,980] jepsen worker 75 - jepsen.util 75 :ok :read [25 20] INFO [2022-01-10 12:53:07,980] jepsen worker 75 - jepsen.util 75 :invoke :write [25 32] INFO [2022-01-10 12:53:07,980] jepsen worker 68 - jepsen.util 68 :ok :write [22 12] INFO [2022-01-10 12:53:07,981] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:07,981] jepsen worker 40 - jepsen.util 40 :ok :write [13 13] INFO [2022-01-10 12:53:07,982] jepsen worker 68 - jepsen.util 68 :ok :read [22 12] INFO [2022-01-10 12:53:07,982] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,982] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:07,983] jepsen worker 40 - jepsen.util 40 :ok :read [13 13] INFO [2022-01-10 12:53:07,983] jepsen worker 68 - jepsen.util 68 :ok :read [22 12] INFO [2022-01-10 12:53:07,984] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,984] jepsen worker 68 - jepsen.util 68 :invoke :write [22 6] INFO [2022-01-10 12:53:07,985] jepsen worker 40 - jepsen.util 40 :ok :read [13 13] INFO [2022-01-10 12:53:07,985] jepsen worker 50 - jepsen.util 50 :ok :write [16 21] INFO [2022-01-10 12:53:07,985] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,986] jepsen worker 50 - jepsen.util 50 :invoke :write [16 34] INFO [2022-01-10 12:53:07,986] jepsen worker 40 - jepsen.util 40 :ok :read [13 13] INFO [2022-01-10 12:53:07,987] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:07,988] jepsen worker 40 - jepsen.util 40 :ok :read [13 13] INFO [2022-01-10 12:53:07,988] jepsen worker 40 - jepsen.util 40 :invoke :write [13 41] INFO [2022-01-10 12:53:08,095] jepsen worker 15 - jepsen.util 15 :ok :write [5 16] INFO [2022-01-10 12:53:08,096] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:08,097] jepsen worker 87 - jepsen.util 87 :ok :write [29 42] INFO [2022-01-10 12:53:08,097] jepsen worker 15 - jepsen.util 15 :ok :read [5 16] INFO [2022-01-10 12:53:08,097] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:08,098] jepsen worker 15 - jepsen.util 15 :invoke :write [5 47] INFO [2022-01-10 12:53:08,098] jepsen worker 87 - jepsen.util 87 :ok :read [29 42] INFO [2022-01-10 12:53:08,099] jepsen worker 78 - jepsen.util 78 :ok :write [26 14] INFO [2022-01-10 12:53:08,099] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:08,099] jepsen worker 34 - jepsen.util 34 :ok :write [11 22] INFO [2022-01-10 12:53:08,099] jepsen worker 78 - jepsen.util 78 :invoke :write [26 40] INFO [2022-01-10 12:53:08,169] jepsen worker 71 - jepsen.util 71 :ok :write [23 20] INFO [2022-01-10 12:53:08,169] jepsen worker 3 - jepsen.util 3 :ok :write [1 48] INFO [2022-01-10 12:53:08,169] jepsen worker 60 - jepsen.util 60 :ok :write [20 17] INFO [2022-01-10 12:53:08,169] jepsen worker 53 - jepsen.util 53 :ok :write [17 3] INFO [2022-01-10 12:53:08,169] jepsen worker 7 - jepsen.util 7 :ok :write [2 6] INFO [2022-01-10 12:53:08,169] jepsen worker 44 - jepsen.util 44 :ok :write [14 23] INFO [2022-01-10 12:53:08,169] jepsen worker 75 - jepsen.util 75 :ok :write [25 32] INFO [2022-01-10 12:53:08,169] jepsen worker 81 - jepsen.util 81 :ok :write [27 30] INFO [2022-01-10 12:53:08,169] jepsen worker 39 - jepsen.util 39 :ok :write [13 30] INFO [2022-01-10 12:53:08,169] jepsen worker 23 - jepsen.util 23 :ok :write [7 21] INFO [2022-01-10 12:53:08,169] jepsen worker 84 - jepsen.util 84 :ok :write [28 31] INFO [2022-01-10 12:53:08,170] jepsen worker 1 - jepsen.util 1 :ok :write [0 13] INFO [2022-01-10 12:53:08,170] jepsen worker 50 - jepsen.util 50 :ok :write [16 34] INFO [2022-01-10 12:53:08,170] jepsen worker 68 - jepsen.util 68 :ok :write [22 6] INFO [2022-01-10 12:53:08,170] jepsen worker 15 - jepsen.util 15 :ok :write [5 47] INFO [2022-01-10 12:53:08,170] jepsen worker 87 - jepsen.util 87 :ok :read [29 42] INFO [2022-01-10 12:53:08,170] jepsen worker 36 - jepsen.util 36 :ok :write [12 15] INFO [2022-01-10 12:53:08,170] jepsen worker 13 - jepsen.util 13 :ok :write [4 49] INFO [2022-01-10 12:53:08,171] jepsen worker 34 - jepsen.util 34 :invoke :write [11 23] INFO [2022-01-10 12:53:08,173] jepsen worker 36 - jepsen.util 36 :invoke :write [12 23] INFO [2022-01-10 12:53:08,174] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:08,175] jepsen worker 87 - jepsen.util 87 :ok :read [29 42] INFO [2022-01-10 12:53:08,175] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:08,177] jepsen worker 75 - jepsen.util 75 :ok :read [25 32] INFO [2022-01-10 12:53:08,177] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:08,178] jepsen worker 13 - jepsen.util 13 :ok :read [4 49] INFO [2022-01-10 12:53:08,178] jepsen worker 23 - jepsen.util 23 :invoke :write [7 25] INFO [2022-01-10 12:53:08,180] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,181] jepsen worker 40 - jepsen.util 40 :ok :write [13 41] INFO [2022-01-10 12:53:08,181] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:08,182] jepsen worker 53 - jepsen.util 53 :invoke :read [17 nil] INFO [2022-01-10 12:53:08,183] jepsen worker 53 - jepsen.util 53 :ok :read [17 3] INFO [2022-01-10 12:53:08,183] jepsen worker 71 - jepsen.util 71 :invoke :write [23 14] INFO [2022-01-10 12:53:08,185] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,186] jepsen worker 68 - jepsen.util 68 :ok :read [22 6] INFO [2022-01-10 12:53:08,186] jepsen worker 1 - jepsen.util 1 :invoke :write [0 36] INFO [2022-01-10 12:53:08,187] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,188] jepsen worker 50 - jepsen.util 50 :ok :read [16 34] INFO [2022-01-10 12:53:08,188] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:08,189] jepsen worker 7 - jepsen.util 7 :ok :read [2 6] INFO [2022-01-10 12:53:08,190] jepsen worker 75 - jepsen.util 75 :invoke :write [25 14] INFO [2022-01-10 12:53:08,191] jepsen worker 50 - jepsen.util 50 :invoke :write [16 27] INFO [2022-01-10 12:53:08,192] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:08,192] jepsen worker 33 - jepsen.util 33 :ok :write [11 6] INFO [2022-01-10 12:53:08,193] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,193] jepsen worker 81 - jepsen.util 81 :ok :read [27 30] INFO [2022-01-10 12:53:08,194] jepsen worker 44 - jepsen.util 44 :invoke :write [14 22] INFO [2022-01-10 12:53:08,194] jepsen worker 68 - jepsen.util 68 :ok :read [22 6] INFO [2022-01-10 12:53:08,195] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:08,195] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:08,196] jepsen worker 7 - jepsen.util 7 :ok :read [2 6] INFO [2022-01-10 12:53:08,196] jepsen worker 13 - jepsen.util 13 :invoke :write [4 31] INFO [2022-01-10 12:53:08,197] jepsen worker 21 - jepsen.util 21 :ok :write [7 18] INFO [2022-01-10 12:53:08,197] jepsen worker 60 - jepsen.util 60 :invoke :read [20 nil] INFO [2022-01-10 12:53:08,198] jepsen worker 60 - jepsen.util 60 :ok :read [20 17] INFO [2022-01-10 12:53:08,199] jepsen worker 84 - jepsen.util 84 :invoke :write [28 7] INFO [2022-01-10 12:53:08,200] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:08,201] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:08,201] jepsen worker 87 - jepsen.util 87 :ok :read [29 42] INFO [2022-01-10 12:53:08,202] jepsen worker 81 - jepsen.util 81 :invoke :write [27 11] INFO [2022-01-10 12:53:08,202] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:08,202] jepsen worker 60 - jepsen.util 60 :invoke :write [20 24] INFO [2022-01-10 12:53:08,203] jepsen worker 43 - jepsen.util 43 :ok :write [14 40] INFO [2022-01-10 12:53:08,203] jepsen worker 78 - jepsen.util 78 :ok :write [26 40] INFO [2022-01-10 12:53:08,204] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:08,205] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:08,205] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:08,206] jepsen worker 87 - jepsen.util 87 :ok :read [29 42] INFO [2022-01-10 12:53:08,206] jepsen worker 53 - jepsen.util 53 :invoke :write [17 22] INFO [2022-01-10 12:53:08,207] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,208] jepsen worker 78 - jepsen.util 78 :ok :read [26 40] INFO [2022-01-10 12:53:08,208] jepsen worker 68 - jepsen.util 68 :invoke :write [22 19] INFO [2022-01-10 12:53:08,208] jepsen worker 67 - jepsen.util 67 :ok :write [22 25] INFO [2022-01-10 12:53:08,209] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:08,209] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:53:08,210] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:08,210] jepsen worker 21 - jepsen.util 21 :ok :read [7 18] INFO [2022-01-10 12:53:08,211] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,211] jepsen worker 52 - jepsen.util 52 :ok :write [17 27] INFO [2022-01-10 12:53:08,212] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,212] jepsen worker 33 - jepsen.util 33 :ok :read [11 6] INFO [2022-01-10 12:53:08,212] jepsen worker 7 - jepsen.util 7 :ok :read [2 6] INFO [2022-01-10 12:53:08,213] jepsen worker 78 - jepsen.util 78 :ok :read [26 40] INFO [2022-01-10 12:53:08,213] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:08,214] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 12:53:08,214] jepsen worker 61 - jepsen.util 61 :ok :write [20 49] INFO [2022-01-10 12:53:08,215] jepsen worker 52 - jepsen.util 52 :ok :read [17 27] INFO [2022-01-10 12:53:08,215] jepsen worker 78 - jepsen.util 78 :invoke :write [26 24] INFO [2022-01-10 12:53:08,216] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,217] jepsen worker 52 - jepsen.util 52 :invoke :write [17 12] INFO [2022-01-10 12:53:08,217] jepsen worker 33 - jepsen.util 33 :ok :read [11 6] INFO [2022-01-10 12:53:08,217] jepsen worker 21 - jepsen.util 21 :invoke :read [7 nil] INFO [2022-01-10 12:53:08,218] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:08,218] jepsen worker 21 - jepsen.util 21 :ok :read [7 18] INFO [2022-01-10 12:53:08,219] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,219] jepsen worker 7 - jepsen.util 7 :ok :read [2 6] INFO [2022-01-10 12:53:08,220] jepsen worker 34 - jepsen.util 34 :ok :write [11 23] INFO [2022-01-10 12:53:08,220] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:08,220] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:08,221] jepsen worker 61 - jepsen.util 61 :ok :read [20 49] INFO [2022-01-10 12:53:08,221] jepsen worker 71 - jepsen.util 71 :ok :write [23 14] INFO [2022-01-10 12:53:08,222] jepsen worker 1 - jepsen.util 1 :ok :write [0 36] INFO [2022-01-10 12:53:08,222] jepsen worker 33 - jepsen.util 33 :invoke :write [11 17] INFO [2022-01-10 12:53:08,224] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:08,225] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:08,225] jepsen worker 40 - jepsen.util 40 :invoke :write [13 43] INFO [2022-01-10 12:53:08,226] jepsen worker 50 - jepsen.util 50 :ok :write [16 27] INFO [2022-01-10 12:53:08,226] jepsen worker 21 - jepsen.util 21 :invoke :write [7 7] INFO [2022-01-10 12:53:08,228] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,229] jepsen worker 61 - jepsen.util 61 :invoke :write [20 2] INFO [2022-01-10 12:53:08,229] jepsen worker 36 - jepsen.util 36 :ok :write [12 23] INFO [2022-01-10 12:53:08,230] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,231] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:08,231] jepsen worker 43 - jepsen.util 43 :ok :read [14 40] INFO [2022-01-10 12:53:08,232] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:08,232] jepsen worker 75 - jepsen.util 75 :ok :write [25 14] INFO [2022-01-10 12:53:08,233] jepsen worker 87 - jepsen.util 87 :ok :read [29 42] INFO [2022-01-10 12:53:08,234] jepsen worker 1 - jepsen.util 1 :invoke :write [0 35] INFO [2022-01-10 12:53:08,235] jepsen worker 71 - jepsen.util 71 :invoke :write [23 10] INFO [2022-01-10 12:53:08,237] jepsen worker 13 - jepsen.util 13 :ok :write [4 31] INFO [2022-01-10 12:53:08,237] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:08,238] jepsen worker 7 - jepsen.util 7 :invoke :write [2 47] INFO [2022-01-10 12:53:08,240] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:08,241] jepsen worker 67 - jepsen.util 67 :invoke :write [22 7] INFO [2022-01-10 12:53:08,241] jepsen worker 13 - jepsen.util 13 :ok :read [4 31] INFO [2022-01-10 12:53:08,241] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:08,242] jepsen worker 87 - jepsen.util 87 :ok :read [29 42] INFO [2022-01-10 12:53:08,242] jepsen worker 15 - jepsen.util 15 :invoke :write [5 44] INFO [2022-01-10 12:53:08,243] jepsen worker 68 - jepsen.util 68 :ok :write [22 19] INFO [2022-01-10 12:53:08,243] jepsen worker 81 - jepsen.util 81 :ok :write [27 11] INFO [2022-01-10 12:53:08,243] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,244] jepsen worker 84 - jepsen.util 84 :ok :write [28 7] INFO [2022-01-10 12:53:08,244] jepsen worker 50 - jepsen.util 50 :ok :read [16 27] INFO [2022-01-10 12:53:08,244] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,245] jepsen worker 68 - jepsen.util 68 :ok :read [22 19] INFO [2022-01-10 12:53:08,246] jepsen worker 50 - jepsen.util 50 :invoke :write [16 20] INFO [2022-01-10 12:53:08,247] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:08,247] jepsen worker 75 - jepsen.util 75 :ok :read [25 14] INFO [2022-01-10 12:53:08,248] jepsen worker 84 - jepsen.util 84 :ok :read [28 7] INFO [2022-01-10 12:53:08,248] jepsen worker 43 - jepsen.util 43 :invoke :write [14 37] INFO [2022-01-10 12:53:08,249] jepsen worker 75 - jepsen.util 75 :invoke :write [25 0] INFO [2022-01-10 12:53:08,249] jepsen worker 23 - jepsen.util 23 :ok :write [7 25] INFO [2022-01-10 12:53:08,250] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,251] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:08,251] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:08,251] jepsen worker 81 - jepsen.util 81 :invoke :write [27 13] INFO [2022-01-10 12:53:08,252] jepsen worker 36 - jepsen.util 36 :ok :read [12 23] INFO [2022-01-10 12:53:08,252] jepsen worker 44 - jepsen.util 44 :ok :write [14 22] INFO [2022-01-10 12:53:08,252] jepsen worker 34 - jepsen.util 34 :invoke :write [11 39] INFO [2022-01-10 12:53:08,253] jepsen worker 60 - jepsen.util 60 :ok :write [20 24] INFO [2022-01-10 12:53:08,253] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,254] jepsen worker 78 - jepsen.util 78 :ok :write [26 24] INFO [2022-01-10 12:53:08,254] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:08,255] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:08,256] jepsen worker 13 - jepsen.util 13 :ok :read [4 31] INFO [2022-01-10 12:53:08,256] jepsen worker 84 - jepsen.util 84 :invoke :write [28 39] INFO [2022-01-10 12:53:08,257] jepsen worker 87 - jepsen.util 87 :invoke :write [29 49] INFO [2022-01-10 12:53:08,258] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,258] jepsen worker 60 - jepsen.util 60 :invoke :write [20 10] INFO [2022-01-10 12:53:08,259] jepsen worker 68 - jepsen.util 68 :ok :read [22 19] INFO [2022-01-10 12:53:08,259] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,260] jepsen worker 44 - jepsen.util 44 :ok :read [14 22] INFO [2022-01-10 12:53:08,260] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:08,261] jepsen worker 53 - jepsen.util 53 :ok :write [17 22] INFO [2022-01-10 12:53:08,261] jepsen worker 23 - jepsen.util 23 :ok :read [7 25] INFO [2022-01-10 12:53:08,261] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,262] jepsen worker 78 - jepsen.util 78 :ok :read [26 24] INFO [2022-01-10 12:53:08,263] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,264] jepsen worker 78 - jepsen.util 78 :invoke :write [26 8] INFO [2022-01-10 12:53:08,264] jepsen worker 44 - jepsen.util 44 :ok :read [14 22] INFO [2022-01-10 12:53:08,265] jepsen worker 39 - jepsen.util 39 :invoke :write [13 35] INFO [2022-01-10 12:53:08,266] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:08,267] jepsen worker 23 - jepsen.util 23 :ok :read [7 25] INFO [2022-01-10 12:53:08,267] jepsen worker 68 - jepsen.util 68 :invoke :write [22 32] INFO [2022-01-10 12:53:08,267] jepsen worker 53 - jepsen.util 53 :invoke :write [17 2] INFO [2022-01-10 12:53:08,268] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:08,269] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:08,269] jepsen worker 36 - jepsen.util 36 :ok :read [12 23] INFO [2022-01-10 12:53:08,269] jepsen worker 3 - jepsen.util 3 :ok :write [1 40] INFO [2022-01-10 12:53:08,269] jepsen worker 13 - jepsen.util 13 :ok :read [4 31] INFO [2022-01-10 12:53:08,269] jepsen worker 23 - jepsen.util 23 :invoke :write [7 36] INFO [2022-01-10 12:53:08,270] jepsen worker 36 - jepsen.util 36 :invoke :write [12 15] INFO [2022-01-10 12:53:08,271] jepsen worker 13 - jepsen.util 13 :invoke :write [4 49] INFO [2022-01-10 12:53:08,271] jepsen worker 3 - jepsen.util 3 :invoke :write [1 39] INFO [2022-01-10 12:53:08,271] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,272] jepsen worker 75 - jepsen.util 75 :ok :write [25 0] INFO [2022-01-10 12:53:08,272] jepsen worker 44 - jepsen.util 44 :ok :read [14 22] INFO [2022-01-10 12:53:08,273] jepsen worker 75 - jepsen.util 75 :invoke :write [25 42] INFO [2022-01-10 12:53:08,273] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,275] jepsen worker 44 - jepsen.util 44 :ok :read [14 22] INFO [2022-01-10 12:53:08,276] jepsen worker 44 - jepsen.util 44 :invoke :write [14 11] INFO [2022-01-10 12:53:08,276] jepsen worker 71 - jepsen.util 71 :ok :write [23 10] INFO [2022-01-10 12:53:08,277] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:08,278] jepsen worker 71 - jepsen.util 71 :ok :read [23 10] INFO [2022-01-10 12:53:08,279] jepsen worker 71 - jepsen.util 71 :invoke :write [23 5] INFO [2022-01-10 12:53:08,280] jepsen worker 21 - jepsen.util 21 :ok :write [7 7] INFO [2022-01-10 12:53:08,281] jepsen worker 21 - jepsen.util 21 :invoke :write [7 6] INFO [2022-01-10 12:53:08,289] jepsen worker 13 - jepsen.util 13 :ok :write [4 49] INFO [2022-01-10 12:53:08,290] jepsen worker 13 - jepsen.util 13 :invoke :write [4 28] INFO [2022-01-10 12:53:08,293] jepsen worker 43 - jepsen.util 43 :ok :write [14 37] INFO [2022-01-10 12:53:08,294] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,296] jepsen worker 43 - jepsen.util 43 :ok :read [14 37] INFO [2022-01-10 12:53:08,296] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,297] jepsen worker 43 - jepsen.util 43 :ok :read [14 37] INFO [2022-01-10 12:53:08,298] jepsen worker 43 - jepsen.util 43 :invoke :write [14 49] INFO [2022-01-10 12:53:08,300] jepsen worker 61 - jepsen.util 61 :ok :write [20 2] INFO [2022-01-10 12:53:08,301] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:08,302] jepsen worker 61 - jepsen.util 61 :ok :read [20 2] INFO [2022-01-10 12:53:08,302] jepsen worker 61 - jepsen.util 61 :invoke :write [20 13] INFO [2022-01-10 12:53:08,304] jepsen worker 78 - jepsen.util 78 :ok :write [26 8] INFO [2022-01-10 12:53:08,305] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,306] jepsen worker 78 - jepsen.util 78 :ok :read [26 8] INFO [2022-01-10 12:53:08,306] jepsen worker 78 - jepsen.util 78 :invoke :write [26 35] INFO [2022-01-10 12:53:08,307] jepsen worker 75 - jepsen.util 75 :ok :write [25 42] INFO [2022-01-10 12:53:08,308] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:08,309] jepsen worker 75 - jepsen.util 75 :ok :read [25 42] INFO [2022-01-10 12:53:08,309] jepsen worker 75 - jepsen.util 75 :invoke :write [25 24] INFO [2022-01-10 12:53:08,311] jepsen worker 52 - jepsen.util 52 :ok :write [17 12] INFO [2022-01-10 12:53:08,311] jepsen worker 52 - jepsen.util 52 :invoke :write [17 23] INFO [2022-01-10 12:53:08,313] jepsen worker 21 - jepsen.util 21 :ok :write [7 6] INFO [2022-01-10 12:53:08,314] jepsen worker 21 - jepsen.util 21 :invoke :write [7 18] INFO [2022-01-10 12:53:08,323] jepsen worker 44 - jepsen.util 44 :ok :write [14 11] INFO [2022-01-10 12:53:08,324] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,325] jepsen worker 71 - jepsen.util 71 :ok :write [23 5] INFO [2022-01-10 12:53:08,325] jepsen worker 44 - jepsen.util 44 :ok :read [14 11] INFO [2022-01-10 12:53:08,325] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,326] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:08,326] jepsen worker 44 - jepsen.util 44 :ok :read [14 11] INFO [2022-01-10 12:53:08,327] jepsen worker 44 - jepsen.util 44 :invoke :write [14 4] INFO [2022-01-10 12:53:08,327] jepsen worker 71 - jepsen.util 71 :ok :read [23 5] INFO [2022-01-10 12:53:08,327] jepsen worker 71 - jepsen.util 71 :invoke :write [23 32] INFO [2022-01-10 12:53:08,332] jepsen worker 13 - jepsen.util 13 :ok :write [4 28] INFO [2022-01-10 12:53:08,333] jepsen worker 13 - jepsen.util 13 :invoke :write [4 2] INFO [2022-01-10 12:53:08,337] jepsen worker 78 - jepsen.util 78 :ok :write [26 35] INFO [2022-01-10 12:53:08,338] jepsen worker 78 - jepsen.util 78 :invoke :write [26 27] INFO [2022-01-10 12:53:08,342] jepsen worker 52 - jepsen.util 52 :ok :write [17 23] INFO [2022-01-10 12:53:08,343] jepsen worker 52 - jepsen.util 52 :invoke :write [17 22] INFO [2022-01-10 12:53:08,377] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:08,378] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:08,378] jepsen worker 38 - jepsen.util 38 :ok :read [12 11] INFO [2022-01-10 12:53:08,379] jepsen worker 33 - jepsen.util 33 :ok :write [11 17] INFO [2022-01-10 12:53:08,379] jepsen worker 38 - jepsen.util 38 :invoke :write [12 32] INFO [2022-01-10 12:53:08,379] jepsen worker 33 - jepsen.util 33 :invoke :write [11 15] INFO [2022-01-10 12:53:08,382] jepsen worker 84 - jepsen.util 84 :ok :write [28 39] INFO [2022-01-10 12:53:08,383] jepsen worker 84 - jepsen.util 84 :invoke :write [28 8] INFO [2022-01-10 12:53:08,383] jepsen worker 61 - jepsen.util 61 :ok :write [20 13] INFO [2022-01-10 12:53:08,384] jepsen worker 61 - jepsen.util 61 :invoke :write [20 24] INFO [2022-01-10 12:53:08,389] jepsen worker 3 - jepsen.util 3 :ok :write [1 39] INFO [2022-01-10 12:53:08,390] jepsen worker 3 - jepsen.util 3 :invoke :write [1 0] INFO [2022-01-10 12:53:08,390] jepsen worker 50 - jepsen.util 50 :ok :write [16 20] INFO [2022-01-10 12:53:08,391] jepsen worker 50 - jepsen.util 50 :invoke :write [16 26] INFO [2022-01-10 12:53:08,392] jepsen worker 75 - jepsen.util 75 :ok :write [25 24] INFO [2022-01-10 12:53:08,392] jepsen worker 71 - jepsen.util 71 :ok :write [23 32] INFO [2022-01-10 12:53:08,394] jepsen worker 36 - jepsen.util 36 :ok :write [12 15] INFO [2022-01-10 12:53:08,396] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:08,397] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:08,397] jepsen worker 13 - jepsen.util 13 :ok :write [4 2] INFO [2022-01-10 12:53:08,397] jepsen worker 40 - jepsen.util 40 :ok :write [13 43] INFO [2022-01-10 12:53:08,397] jepsen worker 71 - jepsen.util 71 :invoke :write [23 18] INFO [2022-01-10 12:53:08,397] jepsen worker 36 - jepsen.util 36 :ok :read [12 15] INFO [2022-01-10 12:53:08,398] jepsen worker 75 - jepsen.util 75 :ok :read [25 24] INFO [2022-01-10 12:53:08,398] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:08,399] jepsen worker 13 - jepsen.util 13 :ok :read [4 2] INFO [2022-01-10 12:53:08,399] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:08,400] jepsen worker 40 - jepsen.util 40 :invoke :write [13 30] INFO [2022-01-10 12:53:08,400] jepsen worker 23 - jepsen.util 23 :ok :write [7 36] INFO [2022-01-10 12:53:08,400] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:08,401] jepsen worker 75 - jepsen.util 75 :ok :read [25 24] INFO [2022-01-10 12:53:08,401] jepsen worker 23 - jepsen.util 23 :invoke :write [7 1] INFO [2022-01-10 12:53:08,401] jepsen worker 13 - jepsen.util 13 :ok :read [4 2] INFO [2022-01-10 12:53:08,401] jepsen worker 78 - jepsen.util 78 :ok :write [26 27] INFO [2022-01-10 12:53:08,401] jepsen worker 1 - jepsen.util 1 :ok :write [0 35] INFO [2022-01-10 12:53:08,401] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:08,402] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:08,403] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:08,403] jepsen worker 1 - jepsen.util 1 :ok :read [0 35] INFO [2022-01-10 12:53:08,404] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,404] jepsen worker 36 - jepsen.util 36 :ok :read [12 15] INFO [2022-01-10 12:53:08,404] jepsen worker 75 - jepsen.util 75 :ok :read [25 24] INFO [2022-01-10 12:53:08,404] jepsen worker 13 - jepsen.util 13 :invoke :write [4 8] INFO [2022-01-10 12:53:08,405] jepsen worker 78 - jepsen.util 78 :ok :read [26 27] INFO [2022-01-10 12:53:08,405] jepsen worker 75 - jepsen.util 75 :invoke :read [25 nil] INFO [2022-01-10 12:53:08,405] jepsen worker 84 - jepsen.util 84 :ok :write [28 8] INFO [2022-01-10 12:53:08,406] jepsen worker 81 - jepsen.util 81 :ok :write [27 13] INFO [2022-01-10 12:53:08,406] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:08,406] jepsen worker 43 - jepsen.util 43 :ok :write [14 49] INFO [2022-01-10 12:53:08,407] jepsen worker 75 - jepsen.util 75 :ok :read [25 24] INFO [2022-01-10 12:53:08,407] jepsen worker 78 - jepsen.util 78 :invoke :write [26 18] INFO [2022-01-10 12:53:08,407] jepsen worker 52 - jepsen.util 52 :ok :write [17 22] INFO [2022-01-10 12:53:08,408] jepsen worker 84 - jepsen.util 84 :ok :read [28 8] INFO [2022-01-10 12:53:08,408] jepsen worker 36 - jepsen.util 36 :invoke :write [12 44] INFO [2022-01-10 12:53:08,409] jepsen worker 1 - jepsen.util 1 :invoke :write [0 42] INFO [2022-01-10 12:53:08,410] jepsen worker 81 - jepsen.util 81 :invoke :write [27 29] INFO [2022-01-10 12:53:08,411] jepsen worker 75 - jepsen.util 75 :invoke :write [25 27] INFO [2022-01-10 12:53:08,412] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:08,413] jepsen worker 52 - jepsen.util 52 :ok :read [17 22] INFO [2022-01-10 12:53:08,413] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:08,413] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:08,414] jepsen worker 84 - jepsen.util 84 :ok :read [28 8] INFO [2022-01-10 12:53:08,414] jepsen worker 43 - jepsen.util 43 :invoke :write [14 46] INFO [2022-01-10 12:53:08,414] jepsen worker 52 - jepsen.util 52 :ok :read [17 22] INFO [2022-01-10 12:53:08,415] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:08,416] jepsen worker 52 - jepsen.util 52 :invoke :write [17 8] INFO [2022-01-10 12:53:08,416] jepsen worker 84 - jepsen.util 84 :ok :read [28 8] INFO [2022-01-10 12:53:08,417] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:08,418] jepsen worker 84 - jepsen.util 84 :ok :read [28 8] INFO [2022-01-10 12:53:08,418] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:08,419] jepsen worker 61 - jepsen.util 61 :ok :write [20 24] INFO [2022-01-10 12:53:08,419] jepsen worker 84 - jepsen.util 84 :ok :read [28 8] INFO [2022-01-10 12:53:08,420] jepsen worker 67 - jepsen.util 67 :ok :write [22 7] INFO [2022-01-10 12:53:08,420] jepsen worker 61 - jepsen.util 61 :invoke :write [20 10] INFO [2022-01-10 12:53:08,420] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,421] jepsen worker 84 - jepsen.util 84 :invoke :write [28 8] INFO [2022-01-10 12:53:08,421] jepsen worker 67 - jepsen.util 67 :ok :read [22 7] INFO [2022-01-10 12:53:08,422] jepsen worker 67 - jepsen.util 67 :invoke :write [22 34] INFO [2022-01-10 12:53:08,428] jepsen worker 15 - jepsen.util 15 :ok :write [5 44] INFO [2022-01-10 12:53:08,429] jepsen worker 15 - jepsen.util 15 :invoke :write [5 37] INFO [2022-01-10 12:53:08,434] jepsen worker 7 - jepsen.util 7 :ok :write [2 47] INFO [2022-01-10 12:53:08,435] jepsen worker 7 - jepsen.util 7 :invoke :write [2 4] INFO [2022-01-10 12:53:08,438] jepsen worker 87 - jepsen.util 87 :ok :write [29 49] INFO [2022-01-10 12:53:08,439] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:08,440] jepsen worker 87 - jepsen.util 87 :ok :read [29 49] INFO [2022-01-10 12:53:08,441] jepsen worker 87 - jepsen.util 87 :invoke :write [29 47] INFO [2022-01-10 12:53:08,442] jepsen worker 33 - jepsen.util 33 :ok :write [11 15] INFO [2022-01-10 12:53:08,443] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,443] jepsen worker 33 - jepsen.util 33 :ok :read [11 15] INFO [2022-01-10 12:53:08,444] jepsen worker 33 - jepsen.util 33 :invoke :write [11 19] INFO [2022-01-10 12:53:08,447] jepsen worker 50 - jepsen.util 50 :ok :write [16 26] INFO [2022-01-10 12:53:08,448] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,449] jepsen worker 50 - jepsen.util 50 :ok :read [16 26] INFO [2022-01-10 12:53:08,449] jepsen worker 50 - jepsen.util 50 :invoke :write [16 43] INFO [2022-01-10 12:53:08,460] jepsen worker 81 - jepsen.util 81 :ok :write [27 29] INFO [2022-01-10 12:53:08,460] jepsen worker 81 - jepsen.util 81 :invoke :write [27 36] INFO [2022-01-10 12:53:08,464] jepsen worker 67 - jepsen.util 67 :ok :write [22 34] INFO [2022-01-10 12:53:08,464] jepsen worker 67 - jepsen.util 67 :invoke :write [22 36] INFO [2022-01-10 12:53:08,468] jepsen worker 15 - jepsen.util 15 :ok :write [5 37] INFO [2022-01-10 12:53:08,469] jepsen worker 15 - jepsen.util 15 :invoke :write [5 45] INFO [2022-01-10 12:53:08,473] jepsen worker 7 - jepsen.util 7 :ok :write [2 4] INFO [2022-01-10 12:53:08,473] jepsen worker 7 - jepsen.util 7 :invoke :write [2 4] INFO [2022-01-10 12:53:08,482] jepsen worker 33 - jepsen.util 33 :ok :write [11 19] INFO [2022-01-10 12:53:08,482] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,483] jepsen worker 33 - jepsen.util 33 :ok :read [11 19] INFO [2022-01-10 12:53:08,484] jepsen worker 33 - jepsen.util 33 :invoke :write [11 0] INFO [2022-01-10 12:53:08,485] jepsen worker 50 - jepsen.util 50 :ok :write [16 43] INFO [2022-01-10 12:53:08,486] jepsen worker 50 - jepsen.util 50 :invoke :write [16 11] INFO [2022-01-10 12:53:08,492] jepsen worker 81 - jepsen.util 81 :ok :write [27 36] INFO [2022-01-10 12:53:08,492] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:08,493] jepsen worker 81 - jepsen.util 81 :ok :read [27 36] INFO [2022-01-10 12:53:08,494] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:08,494] jepsen worker 81 - jepsen.util 81 :ok :read [27 36] INFO [2022-01-10 12:53:08,495] jepsen worker 67 - jepsen.util 67 :ok :write [22 36] INFO [2022-01-10 12:53:08,495] jepsen worker 81 - jepsen.util 81 :invoke :write [27 2] INFO [2022-01-10 12:53:08,496] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,496] jepsen worker 67 - jepsen.util 67 :ok :read [22 36] INFO [2022-01-10 12:53:08,497] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,498] jepsen worker 67 - jepsen.util 67 :ok :read [22 36] INFO [2022-01-10 12:53:08,499] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,499] jepsen worker 67 - jepsen.util 67 :ok :read [22 36] INFO [2022-01-10 12:53:08,500] jepsen worker 15 - jepsen.util 15 :ok :write [5 45] INFO [2022-01-10 12:53:08,500] jepsen worker 67 - jepsen.util 67 :invoke :write [22 16] INFO [2022-01-10 12:53:08,500] jepsen worker 15 - jepsen.util 15 :invoke :write [5 16] INFO [2022-01-10 12:53:08,507] jepsen worker 7 - jepsen.util 7 :ok :write [2 4] INFO [2022-01-10 12:53:08,507] jepsen worker 7 - jepsen.util 7 :invoke :write [2 14] INFO [2022-01-10 12:53:08,511] jepsen worker 33 - jepsen.util 33 :ok :write [11 0] INFO [2022-01-10 12:53:08,511] jepsen worker 3 - jepsen.util 3 :ok :write [1 0] INFO [2022-01-10 12:53:08,512] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,513] jepsen worker 33 - jepsen.util 33 :ok :read [11 0] INFO [2022-01-10 12:53:08,513] jepsen worker 3 - jepsen.util 3 :invoke :write [1 15] INFO [2022-01-10 12:53:08,513] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,514] jepsen worker 33 - jepsen.util 33 :ok :read [11 0] INFO [2022-01-10 12:53:08,514] jepsen worker 50 - jepsen.util 50 :ok :write [16 11] INFO [2022-01-10 12:53:08,515] jepsen worker 33 - jepsen.util 33 :invoke :write [11 13] INFO [2022-01-10 12:53:08,515] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,516] jepsen worker 50 - jepsen.util 50 :ok :read [16 11] INFO [2022-01-10 12:53:08,517] jepsen worker 50 - jepsen.util 50 :invoke :write [16 31] INFO [2022-01-10 12:53:08,517] jepsen worker 71 - jepsen.util 71 :ok :write [23 18] INFO [2022-01-10 12:53:08,517] jepsen worker 40 - jepsen.util 40 :ok :write [13 30] INFO [2022-01-10 12:53:08,518] jepsen worker 71 - jepsen.util 71 :invoke :write [23 47] INFO [2022-01-10 12:53:08,518] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,522] jepsen worker 81 - jepsen.util 81 :ok :write [27 2] INFO [2022-01-10 12:53:08,522] jepsen worker 81 - jepsen.util 81 :invoke :write [27 9] INFO [2022-01-10 12:53:08,522] jepsen worker 40 - jepsen.util 40 :ok :read [13 30] INFO [2022-01-10 12:53:08,523] jepsen worker 40 - jepsen.util 40 :invoke :write [13 16] INFO [2022-01-10 12:53:08,524] jepsen worker 76 - jepsen.util 76 :ok :write [25 7] INFO [2022-01-10 12:53:08,525] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:08,526] jepsen worker 38 - jepsen.util 38 :ok :write [12 32] INFO [2022-01-10 12:53:08,526] jepsen worker 76 - jepsen.util 76 :ok :read [25 7] INFO [2022-01-10 12:53:08,526] jepsen worker 67 - jepsen.util 67 :ok :write [22 16] INFO [2022-01-10 12:53:08,526] jepsen worker 38 - jepsen.util 38 :invoke :write [12 18] INFO [2022-01-10 12:53:08,527] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,527] jepsen worker 76 - jepsen.util 76 :invoke :write [25 7] INFO [2022-01-10 12:53:08,528] jepsen worker 67 - jepsen.util 67 :ok :read [22 16] INFO [2022-01-10 12:53:08,528] jepsen worker 23 - jepsen.util 23 :ok :write [7 1] INFO [2022-01-10 12:53:08,529] jepsen worker 67 - jepsen.util 67 :invoke :write [22 26] INFO [2022-01-10 12:53:08,529] jepsen worker 23 - jepsen.util 23 :invoke :write [7 5] INFO [2022-01-10 12:53:08,532] jepsen worker 15 - jepsen.util 15 :ok :write [5 16] INFO [2022-01-10 12:53:08,533] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:08,534] jepsen worker 1 - jepsen.util 1 :ok :write [0 42] INFO [2022-01-10 12:53:08,534] jepsen worker 13 - jepsen.util 13 :ok :write [4 8] INFO [2022-01-10 12:53:08,534] jepsen worker 15 - jepsen.util 15 :ok :read [5 16] INFO [2022-01-10 12:53:08,535] jepsen worker 13 - jepsen.util 13 :invoke :write [4 10] INFO [2022-01-10 12:53:08,535] jepsen worker 1 - jepsen.util 1 :invoke :write [0 39] INFO [2022-01-10 12:53:08,536] jepsen worker 15 - jepsen.util 15 :invoke :write [5 15] INFO [2022-01-10 12:53:08,541] jepsen worker 43 - jepsen.util 43 :ok :write [14 46] INFO [2022-01-10 12:53:08,541] jepsen worker 7 - jepsen.util 7 :ok :write [2 14] INFO [2022-01-10 12:53:08,541] jepsen worker 78 - jepsen.util 78 :ok :write [26 18] INFO [2022-01-10 12:53:08,541] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,542] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:08,543] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,543] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:08,543] jepsen worker 7 - jepsen.util 7 :ok :read [2 14] INFO [2022-01-10 12:53:08,544] jepsen worker 78 - jepsen.util 78 :ok :read [26 18] INFO [2022-01-10 12:53:08,544] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,544] jepsen worker 7 - jepsen.util 7 :invoke :write [2 6] INFO [2022-01-10 12:53:08,545] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:08,545] jepsen worker 33 - jepsen.util 33 :ok :write [11 13] INFO [2022-01-10 12:53:08,545] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,545] jepsen worker 61 - jepsen.util 61 :ok :write [20 10] INFO [2022-01-10 12:53:08,546] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,546] jepsen worker 78 - jepsen.util 78 :ok :read [26 18] INFO [2022-01-10 12:53:08,546] jepsen worker 33 - jepsen.util 33 :ok :read [11 13] INFO [2022-01-10 12:53:08,546] jepsen worker 43 - jepsen.util 43 :invoke :write [14 2] INFO [2022-01-10 12:53:08,547] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,548] jepsen worker 61 - jepsen.util 61 :invoke :write [20 42] INFO [2022-01-10 12:53:08,548] jepsen worker 50 - jepsen.util 50 :ok :write [16 31] INFO [2022-01-10 12:53:08,548] jepsen worker 33 - jepsen.util 33 :invoke :write [11 4] INFO [2022-01-10 12:53:08,549] jepsen worker 78 - jepsen.util 78 :ok :read [26 18] INFO [2022-01-10 12:53:08,549] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,549] jepsen worker 78 - jepsen.util 78 :invoke :write [26 42] INFO [2022-01-10 12:53:08,550] jepsen worker 52 - jepsen.util 52 :ok :write [17 8] INFO [2022-01-10 12:53:08,550] jepsen worker 87 - jepsen.util 87 :ok :write [29 47] INFO [2022-01-10 12:53:08,550] jepsen worker 50 - jepsen.util 50 :ok :read [16 31] INFO [2022-01-10 12:53:08,551] jepsen worker 50 - jepsen.util 50 :invoke :write [16 1] INFO [2022-01-10 12:53:08,552] jepsen worker 87 - jepsen.util 87 :invoke :write [29 39] INFO [2022-01-10 12:53:08,552] jepsen worker 52 - jepsen.util 52 :invoke :write [17 19] INFO [2022-01-10 12:53:08,557] jepsen worker 81 - jepsen.util 81 :ok :write [27 9] INFO [2022-01-10 12:53:08,558] jepsen worker 81 - jepsen.util 81 :invoke :write [27 0] INFO [2022-01-10 12:53:08,559] jepsen worker 84 - jepsen.util 84 :ok :write [28 8] INFO [2022-01-10 12:53:08,559] jepsen worker 40 - jepsen.util 40 :ok :write [13 16] INFO [2022-01-10 12:53:08,560] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:08,560] jepsen worker 40 - jepsen.util 40 :invoke :write [13 37] INFO [2022-01-10 12:53:08,560] jepsen worker 84 - jepsen.util 84 :ok :read [28 8] INFO [2022-01-10 12:53:08,561] jepsen worker 84 - jepsen.util 84 :invoke :write [28 26] INFO [2022-01-10 12:53:08,563] jepsen worker 76 - jepsen.util 76 :ok :write [25 7] INFO [2022-01-10 12:53:08,564] jepsen worker 76 - jepsen.util 76 :invoke :write [25 24] INFO [2022-01-10 12:53:08,564] jepsen worker 3 - jepsen.util 3 :ok :write [1 15] INFO [2022-01-10 12:53:08,564] jepsen worker 67 - jepsen.util 67 :ok :write [22 26] INFO [2022-01-10 12:53:08,565] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:08,566] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,566] jepsen worker 3 - jepsen.util 3 :ok :read [1 15] INFO [2022-01-10 12:53:08,567] jepsen worker 67 - jepsen.util 67 :ok :read [22 26] INFO [2022-01-10 12:53:08,567] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:08,568] jepsen worker 67 - jepsen.util 67 :invoke :write [22 46] INFO [2022-01-10 12:53:08,568] jepsen worker 3 - jepsen.util 3 :ok :read [1 15] INFO [2022-01-10 12:53:08,568] jepsen worker 3 - jepsen.util 3 :invoke :write [1 47] INFO [2022-01-10 12:53:08,572] jepsen worker 23 - jepsen.util 23 :ok :write [7 5] INFO [2022-01-10 12:53:08,572] jepsen worker 71 - jepsen.util 71 :ok :write [23 47] INFO [2022-01-10 12:53:08,572] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:08,573] jepsen worker 71 - jepsen.util 71 :invoke :write [23 16] INFO [2022-01-10 12:53:08,574] jepsen worker 23 - jepsen.util 23 :ok :read [7 5] INFO [2022-01-10 12:53:08,574] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:08,575] jepsen worker 1 - jepsen.util 1 :ok :write [0 39] INFO [2022-01-10 12:53:08,575] jepsen worker 1 - jepsen.util 1 :invoke :write [0 36] INFO [2022-01-10 12:53:08,576] jepsen worker 23 - jepsen.util 23 :ok :read [7 5] INFO [2022-01-10 12:53:08,576] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:08,577] jepsen worker 23 - jepsen.util 23 :ok :read [7 5] INFO [2022-01-10 12:53:08,578] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:08,579] jepsen worker 15 - jepsen.util 15 :ok :write [5 15] INFO [2022-01-10 12:53:08,579] jepsen worker 23 - jepsen.util 23 :ok :read [7 5] INFO [2022-01-10 12:53:08,579] jepsen worker 15 - jepsen.util 15 :invoke :write [5 26] INFO [2022-01-10 12:53:08,580] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:08,581] jepsen worker 38 - jepsen.util 38 :ok :write [12 18] INFO [2022-01-10 12:53:08,581] jepsen worker 23 - jepsen.util 23 :ok :read [7 5] INFO [2022-01-10 12:53:08,581] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:08,582] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:08,582] jepsen worker 38 - jepsen.util 38 :ok :read [12 18] INFO [2022-01-10 12:53:08,583] jepsen worker 38 - jepsen.util 38 :invoke :write [12 38] INFO [2022-01-10 12:53:08,583] jepsen worker 23 - jepsen.util 23 :ok :read [7 5] INFO [2022-01-10 12:53:08,584] jepsen worker 23 - jepsen.util 23 :invoke :write [7 47] INFO [2022-01-10 12:53:08,586] jepsen worker 13 - jepsen.util 13 :ok :write [4 10] INFO [2022-01-10 12:53:08,587] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:08,587] jepsen worker 13 - jepsen.util 13 :ok :read [4 10] INFO [2022-01-10 12:53:08,588] jepsen worker 43 - jepsen.util 43 :ok :write [14 2] INFO [2022-01-10 12:53:08,589] jepsen worker 43 - jepsen.util 43 :invoke :write [14 10] INFO [2022-01-10 12:53:08,589] jepsen worker 13 - jepsen.util 13 :invoke :write [4 2] INFO [2022-01-10 12:53:08,593] jepsen worker 78 - jepsen.util 78 :ok :write [26 42] INFO [2022-01-10 12:53:08,593] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,594] jepsen worker 78 - jepsen.util 78 :ok :read [26 42] INFO [2022-01-10 12:53:08,595] jepsen worker 7 - jepsen.util 7 :ok :write [2 6] INFO [2022-01-10 12:53:08,595] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,595] jepsen worker 7 - jepsen.util 7 :invoke :write [2 5] INFO [2022-01-10 12:53:08,596] jepsen worker 78 - jepsen.util 78 :ok :read [26 42] INFO [2022-01-10 12:53:08,596] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,597] jepsen worker 78 - jepsen.util 78 :ok :read [26 42] INFO [2022-01-10 12:53:08,598] jepsen worker 78 - jepsen.util 78 :invoke :write [26 21] INFO [2022-01-10 12:53:08,599] jepsen worker 33 - jepsen.util 33 :ok :write [11 4] INFO [2022-01-10 12:53:08,600] jepsen worker 87 - jepsen.util 87 :ok :write [29 39] INFO [2022-01-10 12:53:08,600] jepsen worker 52 - jepsen.util 52 :ok :write [17 19] INFO [2022-01-10 12:53:08,600] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,600] jepsen worker 52 - jepsen.util 52 :invoke :write [17 34] INFO [2022-01-10 12:53:08,601] jepsen worker 33 - jepsen.util 33 :ok :read [11 4] INFO [2022-01-10 12:53:08,601] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:08,601] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,602] jepsen worker 87 - jepsen.util 87 :ok :read [29 39] INFO [2022-01-10 12:53:08,602] jepsen worker 33 - jepsen.util 33 :ok :read [11 4] INFO [2022-01-10 12:53:08,602] jepsen worker 87 - jepsen.util 87 :invoke :write [29 40] INFO [2022-01-10 12:53:08,603] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,604] jepsen worker 33 - jepsen.util 33 :ok :read [11 4] INFO [2022-01-10 12:53:08,604] jepsen worker 33 - jepsen.util 33 :invoke :write [11 38] INFO [2022-01-10 12:53:08,607] jepsen worker 40 - jepsen.util 40 :ok :write [13 37] INFO [2022-01-10 12:53:08,607] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,608] jepsen worker 61 - jepsen.util 61 :ok :write [20 42] INFO [2022-01-10 12:53:08,608] jepsen worker 40 - jepsen.util 40 :ok :read [13 37] INFO [2022-01-10 12:53:08,609] jepsen worker 40 - jepsen.util 40 :invoke :write [13 25] INFO [2022-01-10 12:53:08,609] jepsen worker 61 - jepsen.util 61 :invoke :write [20 37] INFO [2022-01-10 12:53:08,609] jepsen worker 84 - jepsen.util 84 :ok :write [28 26] INFO [2022-01-10 12:53:08,611] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:08,612] jepsen worker 84 - jepsen.util 84 :ok :read [28 26] INFO [2022-01-10 12:53:08,612] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:08,613] jepsen worker 3 - jepsen.util 3 :ok :write [1 47] INFO [2022-01-10 12:53:08,613] jepsen worker 84 - jepsen.util 84 :ok :read [28 26] INFO [2022-01-10 12:53:08,614] jepsen worker 3 - jepsen.util 3 :invoke :write [1 30] INFO [2022-01-10 12:53:08,614] jepsen worker 84 - jepsen.util 84 :invoke :write [28 43] INFO [2022-01-10 12:53:08,615] jepsen worker 50 - jepsen.util 50 :ok :write [16 1] INFO [2022-01-10 12:53:08,616] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,617] jepsen worker 50 - jepsen.util 50 :ok :read [16 1] INFO [2022-01-10 12:53:08,617] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:53:08,618] jepsen worker 1 - jepsen.util 1 :ok :write [0 36] INFO [2022-01-10 12:53:08,618] jepsen worker 1 - jepsen.util 1 :invoke :write [0 48] INFO [2022-01-10 12:53:08,619] jepsen worker 71 - jepsen.util 71 :ok :write [23 16] INFO [2022-01-10 12:53:08,620] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:08,621] jepsen worker 71 - jepsen.util 71 :ok :read [23 16] INFO [2022-01-10 12:53:08,622] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:08,622] jepsen worker 71 - jepsen.util 71 :ok :read [23 16] INFO [2022-01-10 12:53:08,623] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:08,624] jepsen worker 71 - jepsen.util 71 :ok :read [23 16] INFO [2022-01-10 12:53:08,624] jepsen worker 81 - jepsen.util 81 :ok :write [27 0] INFO [2022-01-10 12:53:08,624] jepsen worker 38 - jepsen.util 38 :ok :write [12 38] INFO [2022-01-10 12:53:08,624] jepsen worker 81 - jepsen.util 81 :invoke :write [27 40] INFO [2022-01-10 12:53:08,625] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:08,625] jepsen worker 38 - jepsen.util 38 :invoke :write [12 47] INFO [2022-01-10 12:53:08,626] jepsen worker 71 - jepsen.util 71 :ok :read [23 16] INFO [2022-01-10 12:53:08,626] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:08,627] jepsen worker 71 - jepsen.util 71 :ok :read [23 16] INFO [2022-01-10 12:53:08,628] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:08,629] jepsen worker 76 - jepsen.util 76 :ok :write [25 24] INFO [2022-01-10 12:53:08,629] jepsen worker 71 - jepsen.util 71 :ok :read [23 16] INFO [2022-01-10 12:53:08,629] jepsen worker 76 - jepsen.util 76 :invoke :write [25 26] INFO [2022-01-10 12:53:08,630] jepsen worker 71 - jepsen.util 71 :invoke :write [23 48] INFO [2022-01-10 12:53:08,634] jepsen worker 13 - jepsen.util 13 :ok :write [4 2] INFO [2022-01-10 12:53:08,634] jepsen worker 13 - jepsen.util 13 :invoke :write [4 35] INFO [2022-01-10 12:53:08,634] jepsen worker 67 - jepsen.util 67 :ok :write [22 46] INFO [2022-01-10 12:53:08,635] jepsen worker 67 - jepsen.util 67 :invoke :write [22 42] INFO [2022-01-10 12:53:08,640] jepsen worker 23 - jepsen.util 23 :ok :write [7 47] INFO [2022-01-10 12:53:08,640] jepsen worker 78 - jepsen.util 78 :ok :write [26 21] INFO [2022-01-10 12:53:08,640] jepsen worker 23 - jepsen.util 23 :invoke :write [7 20] INFO [2022-01-10 12:53:08,640] jepsen worker 78 - jepsen.util 78 :invoke :write [26 31] INFO [2022-01-10 12:53:08,645] jepsen worker 52 - jepsen.util 52 :ok :write [17 34] INFO [2022-01-10 12:53:08,646] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:08,647] jepsen worker 52 - jepsen.util 52 :ok :read [17 34] INFO [2022-01-10 12:53:08,647] jepsen worker 52 - jepsen.util 52 :invoke :write [17 20] INFO [2022-01-10 12:53:08,652] jepsen worker 43 - jepsen.util 43 :ok :write [14 10] INFO [2022-01-10 12:53:08,652] jepsen worker 43 - jepsen.util 43 :invoke :write [14 42] INFO [2022-01-10 12:53:08,662] jepsen worker 71 - jepsen.util 71 :ok :write [23 48] INFO [2022-01-10 12:53:08,663] jepsen worker 71 - jepsen.util 71 :invoke :write [23 36] INFO [2022-01-10 12:53:08,669] jepsen worker 13 - jepsen.util 13 :ok :write [4 35] INFO [2022-01-10 12:53:08,670] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:08,670] jepsen worker 13 - jepsen.util 13 :ok :read [4 35] INFO [2022-01-10 12:53:08,671] jepsen worker 13 - jepsen.util 13 :invoke :write [4 21] INFO [2022-01-10 12:53:08,673] jepsen worker 78 - jepsen.util 78 :ok :write [26 31] INFO [2022-01-10 12:53:08,674] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,675] jepsen worker 78 - jepsen.util 78 :ok :read [26 31] INFO [2022-01-10 12:53:08,676] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,677] jepsen worker 78 - jepsen.util 78 :ok :read [26 31] INFO [2022-01-10 12:53:08,677] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,678] jepsen worker 78 - jepsen.util 78 :ok :read [26 31] INFO [2022-01-10 12:53:08,679] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,680] jepsen worker 78 - jepsen.util 78 :ok :read [26 31] INFO [2022-01-10 12:53:08,680] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,681] jepsen worker 78 - jepsen.util 78 :ok :read [26 31] INFO [2022-01-10 12:53:08,681] jepsen worker 78 - jepsen.util 78 :invoke :write [26 20] INFO [2022-01-10 12:53:08,730] jepsen worker 15 - jepsen.util 15 :ok :write [5 26] INFO [2022-01-10 12:53:08,730] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:08,731] jepsen worker 15 - jepsen.util 15 :ok :read [5 26] INFO [2022-01-10 12:53:08,732] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:08,733] jepsen worker 15 - jepsen.util 15 :ok :read [5 26] INFO [2022-01-10 12:53:08,733] jepsen worker 15 - jepsen.util 15 :invoke :write [5 28] INFO [2022-01-10 12:53:08,735] jepsen worker 84 - jepsen.util 84 :ok :write [28 43] INFO [2022-01-10 12:53:08,735] jepsen worker 84 - jepsen.util 84 :invoke :write [28 15] INFO [2022-01-10 12:53:08,742] jepsen worker 3 - jepsen.util 3 :ok :write [1 30] INFO [2022-01-10 12:53:08,743] jepsen worker 3 - jepsen.util 3 :invoke :write [1 13] INFO [2022-01-10 12:53:08,746] jepsen worker 87 - jepsen.util 87 :ok :write [29 40] INFO [2022-01-10 12:53:08,747] jepsen worker 38 - jepsen.util 38 :ok :write [12 47] INFO [2022-01-10 12:53:08,747] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:08,747] jepsen worker 38 - jepsen.util 38 :invoke :write [12 35] INFO [2022-01-10 12:53:08,748] jepsen worker 87 - jepsen.util 87 :ok :read [29 40] INFO [2022-01-10 12:53:08,748] jepsen worker 87 - jepsen.util 87 :invoke :write [29 38] INFO [2022-01-10 12:53:08,752] jepsen worker 40 - jepsen.util 40 :ok :write [13 25] INFO [2022-01-10 12:53:08,753] jepsen worker 40 - jepsen.util 40 :invoke :write [13 4] INFO [2022-01-10 12:53:08,753] jepsen worker 84 - jepsen.util 84 :ok :write [28 15] INFO [2022-01-10 12:53:08,754] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:08,755] jepsen worker 84 - jepsen.util 84 :ok :read [28 15] INFO [2022-01-10 12:53:08,756] jepsen worker 84 - jepsen.util 84 :invoke :write [28 23] INFO [2022-01-10 12:53:08,761] jepsen worker 3 - jepsen.util 3 :ok :write [1 13] INFO [2022-01-10 12:53:08,762] jepsen worker 3 - jepsen.util 3 :invoke :write [1 43] INFO [2022-01-10 12:53:08,767] jepsen worker 1 - jepsen.util 1 :ok :write [0 48] INFO [2022-01-10 12:53:08,768] jepsen worker 7 - jepsen.util 7 :ok :write [2 5] INFO [2022-01-10 12:53:08,768] jepsen worker 38 - jepsen.util 38 :ok :write [12 35] INFO [2022-01-10 12:53:08,768] jepsen worker 1 - jepsen.util 1 :invoke :write [0 44] INFO [2022-01-10 12:53:08,769] jepsen worker 7 - jepsen.util 7 :invoke :write [2 21] INFO [2022-01-10 12:53:08,769] jepsen worker 52 - jepsen.util 52 :ok :write [17 20] INFO [2022-01-10 12:53:08,769] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:08,770] jepsen worker 52 - jepsen.util 52 :invoke :write [17 4] INFO [2022-01-10 12:53:08,771] jepsen worker 38 - jepsen.util 38 :ok :read [12 35] INFO [2022-01-10 12:53:08,771] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:08,772] jepsen worker 38 - jepsen.util 38 :ok :read [12 35] INFO [2022-01-10 12:53:08,773] jepsen worker 38 - jepsen.util 38 :invoke :write [12 42] INFO [2022-01-10 12:53:08,774] jepsen worker 84 - jepsen.util 84 :ok :write [28 23] INFO [2022-01-10 12:53:08,774] jepsen worker 84 - jepsen.util 84 :invoke :write [28 42] INFO [2022-01-10 12:53:08,775] jepsen worker 33 - jepsen.util 33 :ok :write [11 38] INFO [2022-01-10 12:53:08,776] jepsen worker 33 - jepsen.util 33 :invoke :write [11 24] INFO [2022-01-10 12:53:08,777] jepsen worker 71 - jepsen.util 71 :ok :write [23 36] INFO [2022-01-10 12:53:08,777] jepsen worker 71 - jepsen.util 71 :invoke :write [23 48] INFO [2022-01-10 12:53:08,783] jepsen worker 3 - jepsen.util 3 :ok :write [1 43] INFO [2022-01-10 12:53:08,783] jepsen worker 61 - jepsen.util 61 :ok :write [20 37] INFO [2022-01-10 12:53:08,784] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:08,784] jepsen worker 3 - jepsen.util 3 :invoke :write [1 7] INFO [2022-01-10 12:53:08,785] jepsen worker 61 - jepsen.util 61 :ok :read [20 37] INFO [2022-01-10 12:53:08,785] jepsen worker 13 - jepsen.util 13 :ok :write [4 21] INFO [2022-01-10 12:53:08,785] jepsen worker 61 - jepsen.util 61 :invoke :write [20 15] INFO [2022-01-10 12:53:08,786] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:08,787] jepsen worker 38 - jepsen.util 38 :ok :write [12 42] INFO [2022-01-10 12:53:08,787] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:08,787] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:08,788] jepsen worker 38 - jepsen.util 38 :ok :read [12 42] INFO [2022-01-10 12:53:08,788] jepsen worker 13 - jepsen.util 13 :invoke :write [4 39] INFO [2022-01-10 12:53:08,789] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:08,790] jepsen worker 38 - jepsen.util 38 :ok :read [12 42] INFO [2022-01-10 12:53:08,791] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:08,791] jepsen worker 78 - jepsen.util 78 :ok :write [26 20] INFO [2022-01-10 12:53:08,791] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:53:08,791] jepsen worker 38 - jepsen.util 38 :ok :read [12 42] INFO [2022-01-10 12:53:08,792] jepsen worker 78 - jepsen.util 78 :invoke :write [26 21] INFO [2022-01-10 12:53:08,792] jepsen worker 50 - jepsen.util 50 :invoke :write [16 48] INFO [2022-01-10 12:53:08,793] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:08,793] jepsen worker 38 - jepsen.util 38 :ok :read [12 42] INFO [2022-01-10 12:53:08,794] jepsen worker 87 - jepsen.util 87 :ok :write [29 38] INFO [2022-01-10 12:53:08,794] jepsen worker 38 - jepsen.util 38 :invoke :write [12 4] INFO [2022-01-10 12:53:08,794] jepsen worker 84 - jepsen.util 84 :ok :write [28 42] INFO [2022-01-10 12:53:08,795] jepsen worker 87 - jepsen.util 87 :invoke :write [29 49] INFO [2022-01-10 12:53:08,795] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:08,796] jepsen worker 84 - jepsen.util 84 :ok :read [28 42] INFO [2022-01-10 12:53:08,797] jepsen worker 84 - jepsen.util 84 :invoke :write [28 42] INFO [2022-01-10 12:53:08,798] jepsen worker 52 - jepsen.util 52 :ok :write [17 4] INFO [2022-01-10 12:53:08,798] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:08,798] jepsen worker 81 - jepsen.util 81 :ok :write [27 40] INFO [2022-01-10 12:53:08,799] jepsen worker 52 - jepsen.util 52 :ok :read [17 4] INFO [2022-01-10 12:53:08,799] jepsen worker 81 - jepsen.util 81 :invoke :write [27 15] INFO [2022-01-10 12:53:08,800] jepsen worker 40 - jepsen.util 40 :ok :write [13 4] INFO [2022-01-10 12:53:08,800] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:08,801] jepsen worker 40 - jepsen.util 40 :invoke :write [13 45] INFO [2022-01-10 12:53:08,801] jepsen worker 52 - jepsen.util 52 :ok :read [17 4] INFO [2022-01-10 12:53:08,802] jepsen worker 52 - jepsen.util 52 :invoke :write [17 49] INFO [2022-01-10 12:53:08,803] jepsen worker 76 - jepsen.util 76 :ok :write [25 26] INFO [2022-01-10 12:53:08,803] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:08,804] jepsen worker 76 - jepsen.util 76 :ok :read [25 26] INFO [2022-01-10 12:53:08,805] jepsen worker 76 - jepsen.util 76 :invoke :write [25 14] INFO [2022-01-10 12:53:08,805] jepsen worker 1 - jepsen.util 1 :ok :write [0 44] INFO [2022-01-10 12:53:08,805] jepsen worker 1 - jepsen.util 1 :invoke :write [0 27] INFO [2022-01-10 12:53:08,810] jepsen worker 67 - jepsen.util 67 :ok :write [22 42] INFO [2022-01-10 12:53:08,811] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,812] jepsen worker 67 - jepsen.util 67 :ok :read [22 42] INFO [2022-01-10 12:53:08,812] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,813] jepsen worker 67 - jepsen.util 67 :ok :read [22 42] INFO [2022-01-10 12:53:08,814] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,815] jepsen worker 67 - jepsen.util 67 :ok :read [22 42] INFO [2022-01-10 12:53:08,815] jepsen worker 67 - jepsen.util 67 :invoke :write [22 2] INFO [2022-01-10 12:53:08,818] jepsen worker 23 - jepsen.util 23 :ok :write [7 20] INFO [2022-01-10 12:53:08,818] jepsen worker 23 - jepsen.util 23 :invoke :write [7 45] INFO [2022-01-10 12:53:08,823] jepsen worker 43 - jepsen.util 43 :ok :write [14 42] INFO [2022-01-10 12:53:08,823] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,824] jepsen worker 43 - jepsen.util 43 :ok :read [14 42] INFO [2022-01-10 12:53:08,825] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,825] jepsen worker 43 - jepsen.util 43 :ok :read [14 42] INFO [2022-01-10 12:53:08,826] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,827] jepsen worker 43 - jepsen.util 43 :ok :read [14 42] INFO [2022-01-10 12:53:08,828] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:08,828] jepsen worker 15 - jepsen.util 15 :ok :write [5 28] INFO [2022-01-10 12:53:08,829] jepsen worker 43 - jepsen.util 43 :ok :read [14 42] INFO [2022-01-10 12:53:08,829] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:08,829] jepsen worker 43 - jepsen.util 43 :invoke :write [14 33] INFO [2022-01-10 12:53:08,830] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:08,830] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:08,831] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:08,832] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:08,832] jepsen worker 7 - jepsen.util 7 :ok :write [2 21] INFO [2022-01-10 12:53:08,833] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:08,833] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:08,833] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:08,834] jepsen worker 7 - jepsen.util 7 :ok :read [2 21] INFO [2022-01-10 12:53:08,834] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:08,834] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:08,835] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:08,835] jepsen worker 7 - jepsen.util 7 :ok :read [2 21] INFO [2022-01-10 12:53:08,836] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:08,836] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:08,836] jepsen worker 15 - jepsen.util 15 :invoke :write [5 5] INFO [2022-01-10 12:53:08,836] jepsen worker 7 - jepsen.util 7 :ok :read [2 21] INFO [2022-01-10 12:53:08,837] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:08,838] jepsen worker 33 - jepsen.util 33 :ok :write [11 24] INFO [2022-01-10 12:53:08,838] jepsen worker 7 - jepsen.util 7 :ok :read [2 21] INFO [2022-01-10 12:53:08,838] jepsen worker 33 - jepsen.util 33 :invoke :write [11 31] INFO [2022-01-10 12:53:08,839] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:08,840] jepsen worker 7 - jepsen.util 7 :ok :read [2 21] INFO [2022-01-10 12:53:08,840] jepsen worker 7 - jepsen.util 7 :invoke :write [2 1] INFO [2022-01-10 12:53:08,844] jepsen worker 61 - jepsen.util 61 :ok :write [20 15] INFO [2022-01-10 12:53:08,845] jepsen worker 61 - jepsen.util 61 :invoke :write [20 41] INFO [2022-01-10 12:53:08,851] jepsen worker 50 - jepsen.util 50 :ok :write [16 48] INFO [2022-01-10 12:53:08,851] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,852] jepsen worker 50 - jepsen.util 50 :ok :read [16 48] INFO [2022-01-10 12:53:08,853] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,854] jepsen worker 50 - jepsen.util 50 :ok :read [16 48] INFO [2022-01-10 12:53:08,854] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,855] jepsen worker 50 - jepsen.util 50 :ok :read [16 48] INFO [2022-01-10 12:53:08,855] jepsen worker 81 - jepsen.util 81 :ok :write [27 15] INFO [2022-01-10 12:53:08,856] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,856] jepsen worker 81 - jepsen.util 81 :invoke :write [27 11] INFO [2022-01-10 12:53:08,857] jepsen worker 50 - jepsen.util 50 :ok :read [16 48] INFO [2022-01-10 12:53:08,857] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,858] jepsen worker 50 - jepsen.util 50 :ok :read [16 48] INFO [2022-01-10 12:53:08,858] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,859] jepsen worker 50 - jepsen.util 50 :ok :read [16 48] INFO [2022-01-10 12:53:08,860] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,860] jepsen worker 50 - jepsen.util 50 :ok :read [16 48] INFO [2022-01-10 12:53:08,861] jepsen worker 50 - jepsen.util 50 :invoke :write [16 19] INFO [2022-01-10 12:53:08,864] jepsen worker 67 - jepsen.util 67 :ok :write [22 2] INFO [2022-01-10 12:53:08,864] jepsen worker 67 - jepsen.util 67 :invoke :write [22 42] INFO [2022-01-10 12:53:08,875] jepsen worker 15 - jepsen.util 15 :ok :write [5 5] INFO [2022-01-10 12:53:08,875] jepsen worker 15 - jepsen.util 15 :invoke :write [5 38] INFO [2022-01-10 12:53:08,878] jepsen worker 7 - jepsen.util 7 :ok :write [2 1] INFO [2022-01-10 12:53:08,879] jepsen worker 7 - jepsen.util 7 :invoke :write [2 17] INFO [2022-01-10 12:53:08,883] jepsen worker 33 - jepsen.util 33 :ok :write [11 31] INFO [2022-01-10 12:53:08,884] jepsen worker 33 - jepsen.util 33 :invoke :write [11 34] INFO [2022-01-10 12:53:08,893] jepsen worker 50 - jepsen.util 50 :ok :write [16 19] INFO [2022-01-10 12:53:08,894] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:08,895] jepsen worker 50 - jepsen.util 50 :ok :read [16 19] INFO [2022-01-10 12:53:08,895] jepsen worker 50 - jepsen.util 50 :invoke :write [16 28] INFO [2022-01-10 12:53:08,899] jepsen worker 81 - jepsen.util 81 :ok :write [27 11] INFO [2022-01-10 12:53:08,900] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:08,901] jepsen worker 81 - jepsen.util 81 :ok :read [27 11] INFO [2022-01-10 12:53:08,901] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:08,902] jepsen worker 67 - jepsen.util 67 :ok :write [22 42] INFO [2022-01-10 12:53:08,902] jepsen worker 81 - jepsen.util 81 :ok :read [27 11] INFO [2022-01-10 12:53:08,903] jepsen worker 81 - jepsen.util 81 :invoke :write [27 2] INFO [2022-01-10 12:53:08,903] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,904] jepsen worker 67 - jepsen.util 67 :ok :read [22 42] INFO [2022-01-10 12:53:08,904] jepsen worker 67 - jepsen.util 67 :invoke :write [22 37] INFO [2022-01-10 12:53:08,907] jepsen worker 15 - jepsen.util 15 :ok :write [5 38] INFO [2022-01-10 12:53:08,907] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:08,908] jepsen worker 15 - jepsen.util 15 :ok :read [5 38] INFO [2022-01-10 12:53:08,908] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:08,909] jepsen worker 15 - jepsen.util 15 :ok :read [5 38] INFO [2022-01-10 12:53:08,910] jepsen worker 15 - jepsen.util 15 :invoke :write [5 12] INFO [2022-01-10 12:53:08,912] jepsen worker 7 - jepsen.util 7 :ok :write [2 17] INFO [2022-01-10 12:53:08,912] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:08,913] jepsen worker 7 - jepsen.util 7 :ok :read [2 17] INFO [2022-01-10 12:53:08,914] jepsen worker 7 - jepsen.util 7 :invoke :write [2 6] INFO [2022-01-10 12:53:08,915] jepsen worker 71 - jepsen.util 71 :ok :write [23 48] INFO [2022-01-10 12:53:08,915] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:08,916] jepsen worker 71 - jepsen.util 71 :ok :read [23 48] INFO [2022-01-10 12:53:08,917] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:08,917] jepsen worker 71 - jepsen.util 71 :ok :read [23 48] INFO [2022-01-10 12:53:08,918] jepsen worker 71 - jepsen.util 71 :invoke :write [23 16] INFO [2022-01-10 12:53:08,918] jepsen worker 33 - jepsen.util 33 :ok :write [11 34] INFO [2022-01-10 12:53:08,919] jepsen worker 33 - jepsen.util 33 :invoke :write [11 49] INFO [2022-01-10 12:53:08,922] jepsen worker 87 - jepsen.util 87 :ok :write [29 49] INFO [2022-01-10 12:53:08,922] jepsen worker 87 - jepsen.util 87 :invoke :write [29 10] INFO [2022-01-10 12:53:08,923] jepsen worker 3 - jepsen.util 3 :ok :write [1 7] INFO [2022-01-10 12:53:08,923] jepsen worker 3 - jepsen.util 3 :invoke :write [1 46] INFO [2022-01-10 12:53:08,927] jepsen worker 50 - jepsen.util 50 :ok :write [16 28] INFO [2022-01-10 12:53:08,928] jepsen worker 50 - jepsen.util 50 :invoke :write [16 1] INFO [2022-01-10 12:53:08,930] jepsen worker 13 - jepsen.util 13 :ok :write [4 39] INFO [2022-01-10 12:53:08,931] jepsen worker 13 - jepsen.util 13 :invoke :write [4 17] INFO [2022-01-10 12:53:08,932] jepsen worker 81 - jepsen.util 81 :ok :write [27 2] INFO [2022-01-10 12:53:08,932] jepsen worker 40 - jepsen.util 40 :ok :write [13 45] INFO [2022-01-10 12:53:08,933] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:08,933] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,934] jepsen worker 81 - jepsen.util 81 :ok :read [27 2] INFO [2022-01-10 12:53:08,935] jepsen worker 40 - jepsen.util 40 :ok :read [13 45] INFO [2022-01-10 12:53:08,935] jepsen worker 81 - jepsen.util 81 :invoke :write [27 33] INFO [2022-01-10 12:53:08,935] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,936] jepsen worker 40 - jepsen.util 40 :ok :read [13 45] INFO [2022-01-10 12:53:08,936] jepsen worker 38 - jepsen.util 38 :ok :write [12 4] INFO [2022-01-10 12:53:08,936] jepsen worker 76 - jepsen.util 76 :ok :write [25 14] INFO [2022-01-10 12:53:08,937] jepsen worker 38 - jepsen.util 38 :invoke :write [12 20] INFO [2022-01-10 12:53:08,938] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,938] jepsen worker 67 - jepsen.util 67 :ok :write [22 37] INFO [2022-01-10 12:53:08,939] jepsen worker 40 - jepsen.util 40 :ok :read [13 45] INFO [2022-01-10 12:53:08,939] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:08,939] jepsen worker 67 - jepsen.util 67 :invoke :write [22 1] INFO [2022-01-10 12:53:08,940] jepsen worker 76 - jepsen.util 76 :ok :read [25 14] INFO [2022-01-10 12:53:08,940] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,941] jepsen worker 76 - jepsen.util 76 :invoke :write [25 20] INFO [2022-01-10 12:53:08,941] jepsen worker 40 - jepsen.util 40 :ok :read [13 45] INFO [2022-01-10 12:53:08,941] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,942] jepsen worker 1 - jepsen.util 1 :ok :write [0 27] INFO [2022-01-10 12:53:08,943] jepsen worker 40 - jepsen.util 40 :ok :read [13 45] INFO [2022-01-10 12:53:08,943] jepsen worker 1 - jepsen.util 1 :invoke :write [0 18] INFO [2022-01-10 12:53:08,943] jepsen worker 40 - jepsen.util 40 :invoke :write [13 32] INFO [2022-01-10 12:53:08,946] jepsen worker 15 - jepsen.util 15 :ok :write [5 12] INFO [2022-01-10 12:53:08,946] jepsen worker 78 - jepsen.util 78 :ok :write [26 21] INFO [2022-01-10 12:53:08,946] jepsen worker 15 - jepsen.util 15 :invoke :write [5 16] INFO [2022-01-10 12:53:08,947] jepsen worker 23 - jepsen.util 23 :ok :write [7 45] INFO [2022-01-10 12:53:08,947] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:08,948] jepsen worker 23 - jepsen.util 23 :invoke :write [7 40] INFO [2022-01-10 12:53:08,948] jepsen worker 78 - jepsen.util 78 :ok :read [26 21] INFO [2022-01-10 12:53:08,949] jepsen worker 78 - jepsen.util 78 :invoke :write [26 2] INFO [2022-01-10 12:53:08,950] jepsen worker 7 - jepsen.util 7 :ok :write [2 6] INFO [2022-01-10 12:53:08,951] jepsen worker 7 - jepsen.util 7 :invoke :write [2 40] INFO [2022-01-10 12:53:08,953] jepsen worker 43 - jepsen.util 43 :ok :write [14 33] INFO [2022-01-10 12:53:08,954] jepsen worker 43 - jepsen.util 43 :invoke :write [14 12] INFO [2022-01-10 12:53:08,956] jepsen worker 84 - jepsen.util 84 :ok :write [28 42] INFO [2022-01-10 12:53:08,957] jepsen worker 84 - jepsen.util 84 :invoke :write [28 16] INFO [2022-01-10 12:53:08,958] jepsen worker 33 - jepsen.util 33 :ok :write [11 49] INFO [2022-01-10 12:53:08,959] jepsen worker 61 - jepsen.util 61 :ok :write [20 41] INFO [2022-01-10 12:53:08,959] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,960] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:08,960] jepsen worker 33 - jepsen.util 33 :ok :read [11 49] INFO [2022-01-10 12:53:08,960] jepsen worker 61 - jepsen.util 61 :ok :read [20 41] INFO [2022-01-10 12:53:08,960] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,961] jepsen worker 52 - jepsen.util 52 :ok :write [17 49] INFO [2022-01-10 12:53:08,961] jepsen worker 61 - jepsen.util 61 :invoke :write [20 36] INFO [2022-01-10 12:53:08,961] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:08,961] jepsen worker 33 - jepsen.util 33 :ok :read [11 49] INFO [2022-01-10 12:53:08,962] jepsen worker 52 - jepsen.util 52 :ok :read [17 49] INFO [2022-01-10 12:53:08,962] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,963] jepsen worker 87 - jepsen.util 87 :ok :write [29 10] INFO [2022-01-10 12:53:08,963] jepsen worker 52 - jepsen.util 52 :invoke :write [17 28] INFO [2022-01-10 12:53:08,964] jepsen worker 33 - jepsen.util 33 :ok :read [11 49] INFO [2022-01-10 12:53:08,964] jepsen worker 87 - jepsen.util 87 :invoke :write [29 44] INFO [2022-01-10 12:53:08,965] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,966] jepsen worker 33 - jepsen.util 33 :ok :read [11 49] INFO [2022-01-10 12:53:08,967] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:08,968] jepsen worker 33 - jepsen.util 33 :ok :read [11 49] INFO [2022-01-10 12:53:08,968] jepsen worker 33 - jepsen.util 33 :invoke :write [11 26] INFO [2022-01-10 12:53:08,968] jepsen worker 71 - jepsen.util 71 :ok :write [23 16] INFO [2022-01-10 12:53:08,969] jepsen worker 50 - jepsen.util 50 :ok :write [16 1] INFO [2022-01-10 12:53:08,970] jepsen worker 50 - jepsen.util 50 :invoke :write [16 43] INFO [2022-01-10 12:53:08,970] jepsen worker 71 - jepsen.util 71 :invoke :write [23 13] INFO [2022-01-10 12:53:08,971] jepsen worker 40 - jepsen.util 40 :ok :write [13 32] INFO [2022-01-10 12:53:08,972] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,973] jepsen worker 40 - jepsen.util 40 :ok :read [13 32] INFO [2022-01-10 12:53:08,974] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:08,974] jepsen worker 40 - jepsen.util 40 :ok :read [13 32] INFO [2022-01-10 12:53:08,975] jepsen worker 40 - jepsen.util 40 :invoke :write [13 30] INFO [2022-01-10 12:53:08,976] jepsen worker 81 - jepsen.util 81 :ok :write [27 33] INFO [2022-01-10 12:53:08,976] jepsen worker 3 - jepsen.util 3 :ok :write [1 46] INFO [2022-01-10 12:53:08,977] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:08,978] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:08,978] jepsen worker 3 - jepsen.util 3 :ok :read [1 46] INFO [2022-01-10 12:53:08,978] jepsen worker 81 - jepsen.util 81 :ok :read [27 33] INFO [2022-01-10 12:53:08,979] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:08,979] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:08,979] jepsen worker 3 - jepsen.util 3 :ok :read [1 46] INFO [2022-01-10 12:53:08,980] jepsen worker 81 - jepsen.util 81 :ok :read [27 33] INFO [2022-01-10 12:53:08,980] jepsen worker 76 - jepsen.util 76 :ok :write [25 20] INFO [2022-01-10 12:53:08,981] jepsen worker 3 - jepsen.util 3 :invoke :write [1 20] INFO [2022-01-10 12:53:08,981] jepsen worker 81 - jepsen.util 81 :invoke :write [27 48] INFO [2022-01-10 12:53:08,982] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:08,983] jepsen worker 76 - jepsen.util 76 :ok :read [25 20] INFO [2022-01-10 12:53:08,984] jepsen worker 76 - jepsen.util 76 :invoke :write [25 35] INFO [2022-01-10 12:53:08,985] jepsen worker 13 - jepsen.util 13 :ok :write [4 17] INFO [2022-01-10 12:53:08,985] jepsen worker 1 - jepsen.util 1 :ok :write [0 18] INFO [2022-01-10 12:53:08,985] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:08,986] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:08,986] jepsen worker 13 - jepsen.util 13 :ok :read [4 17] INFO [2022-01-10 12:53:08,986] jepsen worker 1 - jepsen.util 1 :ok :read [0 18] INFO [2022-01-10 12:53:08,987] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:08,987] jepsen worker 1 - jepsen.util 1 :invoke :write [0 5] INFO [2022-01-10 12:53:08,988] jepsen worker 13 - jepsen.util 13 :ok :read [4 17] INFO [2022-01-10 12:53:08,989] jepsen worker 13 - jepsen.util 13 :invoke :write [4 20] INFO [2022-01-10 12:53:08,991] jepsen worker 67 - jepsen.util 67 :ok :write [22 1] INFO [2022-01-10 12:53:08,991] jepsen worker 38 - jepsen.util 38 :ok :write [12 20] INFO [2022-01-10 12:53:08,991] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,992] jepsen worker 38 - jepsen.util 38 :invoke :write [12 38] INFO [2022-01-10 12:53:08,992] jepsen worker 67 - jepsen.util 67 :ok :read [22 1] INFO [2022-01-10 12:53:08,993] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:08,993] jepsen worker 67 - jepsen.util 67 :ok :read [22 1] INFO [2022-01-10 12:53:08,994] jepsen worker 67 - jepsen.util 67 :invoke :write [22 13] INFO [2022-01-10 12:53:08,995] jepsen worker 23 - jepsen.util 23 :ok :write [7 40] INFO [2022-01-10 12:53:08,995] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:08,996] jepsen worker 23 - jepsen.util 23 :ok :read [7 40] INFO [2022-01-10 12:53:08,997] jepsen worker 78 - jepsen.util 78 :ok :write [26 2] INFO [2022-01-10 12:53:08,997] jepsen worker 23 - jepsen.util 23 :invoke :write [7 10] INFO [2022-01-10 12:53:08,997] jepsen worker 78 - jepsen.util 78 :invoke :write [26 6] INFO [2022-01-10 12:53:09,003] jepsen worker 15 - jepsen.util 15 :ok :write [5 16] INFO [2022-01-10 12:53:09,004] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:09,004] jepsen worker 15 - jepsen.util 15 :ok :read [5 16] INFO [2022-01-10 12:53:09,005] jepsen worker 15 - jepsen.util 15 :invoke :write [5 43] INFO [2022-01-10 12:53:09,006] jepsen worker 84 - jepsen.util 84 :ok :write [28 16] INFO [2022-01-10 12:53:09,006] jepsen worker 84 - jepsen.util 84 :invoke :write [28 26] INFO [2022-01-10 12:53:09,008] jepsen worker 87 - jepsen.util 87 :ok :write [29 44] INFO [2022-01-10 12:53:09,009] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,009] jepsen worker 87 - jepsen.util 87 :ok :read [29 44] INFO [2022-01-10 12:53:09,010] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,010] jepsen worker 7 - jepsen.util 7 :ok :write [2 40] INFO [2022-01-10 12:53:09,010] jepsen worker 52 - jepsen.util 52 :ok :write [17 28] INFO [2022-01-10 12:53:09,011] jepsen worker 87 - jepsen.util 87 :ok :read [29 44] INFO [2022-01-10 12:53:09,011] jepsen worker 52 - jepsen.util 52 :invoke :write [17 29] INFO [2022-01-10 12:53:09,012] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,012] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,013] jepsen worker 87 - jepsen.util 87 :ok :read [29 44] INFO [2022-01-10 12:53:09,013] jepsen worker 7 - jepsen.util 7 :ok :read [2 40] INFO [2022-01-10 12:53:09,013] jepsen worker 87 - jepsen.util 87 :invoke :write [29 17] INFO [2022-01-10 12:53:09,014] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,015] jepsen worker 7 - jepsen.util 7 :ok :read [2 40] INFO [2022-01-10 12:53:09,015] jepsen worker 7 - jepsen.util 7 :invoke :write [2 32] INFO [2022-01-10 12:53:09,017] jepsen worker 71 - jepsen.util 71 :ok :write [23 13] INFO [2022-01-10 12:53:09,018] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:09,018] jepsen worker 71 - jepsen.util 71 :ok :read [23 13] INFO [2022-01-10 12:53:09,019] jepsen worker 71 - jepsen.util 71 :invoke :write [23 42] INFO [2022-01-10 12:53:09,020] jepsen worker 43 - jepsen.util 43 :ok :write [14 12] INFO [2022-01-10 12:53:09,021] jepsen worker 43 - jepsen.util 43 :invoke :write [14 12] INFO [2022-01-10 12:53:09,023] jepsen worker 3 - jepsen.util 3 :ok :write [1 20] INFO [2022-01-10 12:53:09,024] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,025] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:53:09,026] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,026] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:53:09,027] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,027] jepsen worker 33 - jepsen.util 33 :ok :write [11 26] INFO [2022-01-10 12:53:09,028] jepsen worker 33 - jepsen.util 33 :invoke :write [11 28] INFO [2022-01-10 12:53:09,028] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:53:09,028] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,029] jepsen worker 13 - jepsen.util 13 :ok :write [4 20] INFO [2022-01-10 12:53:09,029] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:53:09,029] jepsen worker 13 - jepsen.util 13 :invoke :write [4 35] INFO [2022-01-10 12:53:09,030] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 12:53:09,034] jepsen worker 61 - jepsen.util 61 :ok :write [20 36] INFO [2022-01-10 12:53:09,035] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:09,036] jepsen worker 61 - jepsen.util 61 :ok :read [20 36] INFO [2022-01-10 12:53:09,036] jepsen worker 61 - jepsen.util 61 :invoke :write [20 42] INFO [2022-01-10 12:53:09,039] jepsen worker 50 - jepsen.util 50 :ok :write [16 43] INFO [2022-01-10 12:53:09,039] jepsen worker 50 - jepsen.util 50 :invoke :write [16 1] INFO [2022-01-10 12:53:09,040] jepsen worker 78 - jepsen.util 78 :ok :write [26 6] INFO [2022-01-10 12:53:09,041] jepsen worker 78 - jepsen.util 78 :invoke :write [26 31] INFO [2022-01-10 12:53:09,047] jepsen worker 76 - jepsen.util 76 :ok :write [25 35] INFO [2022-01-10 12:53:09,048] jepsen worker 76 - jepsen.util 76 :invoke :write [25 45] INFO [2022-01-10 12:53:09,055] jepsen worker 52 - jepsen.util 52 :ok :write [17 29] INFO [2022-01-10 12:53:09,055] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,056] jepsen worker 52 - jepsen.util 52 :ok :read [17 29] INFO [2022-01-10 12:53:09,056] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,057] jepsen worker 52 - jepsen.util 52 :ok :read [17 29] INFO [2022-01-10 12:53:09,058] jepsen worker 52 - jepsen.util 52 :invoke :write [17 38] INFO [2022-01-10 12:53:09,060] jepsen worker 71 - jepsen.util 71 :ok :write [23 42] INFO [2022-01-10 12:53:09,061] jepsen worker 71 - jepsen.util 71 :invoke :write [23 41] INFO [2022-01-10 12:53:09,066] jepsen worker 13 - jepsen.util 13 :ok :write [4 35] INFO [2022-01-10 12:53:09,067] jepsen worker 13 - jepsen.util 13 :invoke :write [4 46] INFO [2022-01-10 12:53:09,134] jepsen worker 38 - jepsen.util 38 :ok :write [12 38] INFO [2022-01-10 12:53:09,135] jepsen worker 40 - jepsen.util 40 :ok :write [13 30] INFO [2022-01-10 12:53:09,135] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,136] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:09,136] jepsen worker 38 - jepsen.util 38 :ok :read [12 38] INFO [2022-01-10 12:53:09,137] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,137] jepsen worker 40 - jepsen.util 40 :ok :read [13 30] INFO [2022-01-10 12:53:09,138] jepsen worker 40 - jepsen.util 40 :invoke :write [13 21] INFO [2022-01-10 12:53:09,138] jepsen worker 38 - jepsen.util 38 :ok :read [12 38] INFO [2022-01-10 12:53:09,139] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,139] jepsen worker 38 - jepsen.util 38 :ok :read [12 38] INFO [2022-01-10 12:53:09,140] jepsen worker 81 - jepsen.util 81 :ok :write [27 48] INFO [2022-01-10 12:53:09,140] jepsen worker 38 - jepsen.util 38 :invoke :write [12 33] INFO [2022-01-10 12:53:09,140] jepsen worker 84 - jepsen.util 84 :ok :write [28 26] INFO [2022-01-10 12:53:09,141] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,141] jepsen worker 84 - jepsen.util 84 :invoke :write [28 30] INFO [2022-01-10 12:53:09,143] jepsen worker 81 - jepsen.util 81 :ok :read [27 48] INFO [2022-01-10 12:53:09,143] jepsen worker 81 - jepsen.util 81 :invoke :write [27 33] INFO [2022-01-10 12:53:09,147] jepsen worker 3 - jepsen.util 3 :ok :write [1 40] INFO [2022-01-10 12:53:09,147] jepsen worker 3 - jepsen.util 3 :invoke :write [1 20] INFO [2022-01-10 12:53:09,147] jepsen worker 1 - jepsen.util 1 :ok :write [0 5] INFO [2022-01-10 12:53:09,148] jepsen worker 1 - jepsen.util 1 :invoke :write [0 6] INFO [2022-01-10 12:53:09,150] jepsen worker 38 - jepsen.util 38 :ok :write [12 33] INFO [2022-01-10 12:53:09,151] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,152] jepsen worker 38 - jepsen.util 38 :ok :read [12 33] INFO [2022-01-10 12:53:09,152] jepsen worker 38 - jepsen.util 38 :invoke :write [12 34] INFO [2022-01-10 12:53:09,157] jepsen worker 84 - jepsen.util 84 :ok :write [28 30] INFO [2022-01-10 12:53:09,157] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:09,158] jepsen worker 84 - jepsen.util 84 :ok :read [28 30] INFO [2022-01-10 12:53:09,159] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:09,160] jepsen worker 84 - jepsen.util 84 :ok :read [28 30] INFO [2022-01-10 12:53:09,160] jepsen worker 87 - jepsen.util 87 :ok :write [29 17] INFO [2022-01-10 12:53:09,160] jepsen worker 84 - jepsen.util 84 :invoke :write [28 9] INFO [2022-01-10 12:53:09,161] jepsen worker 87 - jepsen.util 87 :invoke :write [29 37] INFO [2022-01-10 12:53:09,162] jepsen worker 3 - jepsen.util 3 :ok :write [1 20] INFO [2022-01-10 12:53:09,163] jepsen worker 3 - jepsen.util 3 :invoke :write [1 41] INFO [2022-01-10 12:53:09,164] jepsen worker 78 - jepsen.util 78 :ok :write [26 31] INFO [2022-01-10 12:53:09,164] jepsen worker 78 - jepsen.util 78 :invoke :write [26 5] INFO [2022-01-10 12:53:09,166] jepsen worker 67 - jepsen.util 67 :ok :write [22 13] INFO [2022-01-10 12:53:09,166] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:09,167] jepsen worker 67 - jepsen.util 67 :ok :read [22 13] INFO [2022-01-10 12:53:09,168] jepsen worker 67 - jepsen.util 67 :invoke :write [22 26] INFO [2022-01-10 12:53:09,169] jepsen worker 23 - jepsen.util 23 :ok :write [7 10] INFO [2022-01-10 12:53:09,170] jepsen worker 23 - jepsen.util 23 :invoke :write [7 36] INFO [2022-01-10 12:53:09,171] jepsen worker 38 - jepsen.util 38 :ok :write [12 34] INFO [2022-01-10 12:53:09,171] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,172] jepsen worker 52 - jepsen.util 52 :ok :write [17 38] INFO [2022-01-10 12:53:09,172] jepsen worker 38 - jepsen.util 38 :ok :read [12 34] INFO [2022-01-10 12:53:09,173] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,174] jepsen worker 38 - jepsen.util 38 :invoke :write [12 15] INFO [2022-01-10 12:53:09,174] jepsen worker 52 - jepsen.util 52 :ok :read [17 38] INFO [2022-01-10 12:53:09,175] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,176] jepsen worker 15 - jepsen.util 15 :ok :write [5 43] INFO [2022-01-10 12:53:09,176] jepsen worker 52 - jepsen.util 52 :ok :read [17 38] INFO [2022-01-10 12:53:09,176] jepsen worker 71 - jepsen.util 71 :ok :write [23 41] INFO [2022-01-10 12:53:09,177] jepsen worker 15 - jepsen.util 15 :invoke :write [5 19] INFO [2022-01-10 12:53:09,178] jepsen worker 71 - jepsen.util 71 :invoke :write [23 39] INFO [2022-01-10 12:53:09,179] jepsen worker 52 - jepsen.util 52 :invoke :write [17 2] INFO [2022-01-10 12:53:09,179] jepsen worker 7 - jepsen.util 7 :ok :write [2 32] INFO [2022-01-10 12:53:09,180] jepsen worker 7 - jepsen.util 7 :invoke :write [2 46] INFO [2022-01-10 12:53:09,181] jepsen worker 13 - jepsen.util 13 :ok :write [4 46] INFO [2022-01-10 12:53:09,181] jepsen worker 13 - jepsen.util 13 :invoke :write [4 29] INFO [2022-01-10 12:53:09,183] jepsen worker 3 - jepsen.util 3 :ok :write [1 41] INFO [2022-01-10 12:53:09,183] jepsen worker 40 - jepsen.util 40 :ok :write [13 21] INFO [2022-01-10 12:53:09,184] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,184] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:09,185] jepsen worker 3 - jepsen.util 3 :ok :read [1 41] INFO [2022-01-10 12:53:09,186] jepsen worker 40 - jepsen.util 40 :ok :read [13 21] INFO [2022-01-10 12:53:09,186] jepsen worker 3 - jepsen.util 3 :invoke :write [1 19] INFO [2022-01-10 12:53:09,186] jepsen worker 40 - jepsen.util 40 :invoke :write [13 8] INFO [2022-01-10 12:53:09,187] jepsen worker 43 - jepsen.util 43 :ok :write [14 12] INFO [2022-01-10 12:53:09,187] jepsen worker 43 - jepsen.util 43 :invoke :write [14 46] INFO [2022-01-10 12:53:09,189] jepsen worker 78 - jepsen.util 78 :ok :write [26 5] INFO [2022-01-10 12:53:09,189] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:09,190] jepsen worker 78 - jepsen.util 78 :ok :read [26 5] INFO [2022-01-10 12:53:09,191] jepsen worker 78 - jepsen.util 78 :invoke :write [26 45] INFO [2022-01-10 12:53:09,193] jepsen worker 84 - jepsen.util 84 :ok :write [28 9] INFO [2022-01-10 12:53:09,193] jepsen worker 84 - jepsen.util 84 :invoke :write [28 4] INFO [2022-01-10 12:53:09,193] jepsen worker 33 - jepsen.util 33 :ok :write [11 28] INFO [2022-01-10 12:53:09,194] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:09,195] jepsen worker 1 - jepsen.util 1 :ok :write [0 6] INFO [2022-01-10 12:53:09,195] jepsen worker 33 - jepsen.util 33 :ok :read [11 28] INFO [2022-01-10 12:53:09,195] jepsen worker 38 - jepsen.util 38 :ok :write [12 15] INFO [2022-01-10 12:53:09,195] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:09,196] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,196] jepsen worker 33 - jepsen.util 33 :ok :read [11 28] INFO [2022-01-10 12:53:09,197] jepsen worker 38 - jepsen.util 38 :ok :read [12 15] INFO [2022-01-10 12:53:09,197] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,198] jepsen worker 38 - jepsen.util 38 :invoke :write [12 20] INFO [2022-01-10 12:53:09,198] jepsen worker 1 - jepsen.util 1 :ok :read [0 6] INFO [2022-01-10 12:53:09,198] jepsen worker 33 - jepsen.util 33 :invoke :write [11 32] INFO [2022-01-10 12:53:09,199] jepsen worker 1 - jepsen.util 1 :invoke :write [0 30] INFO [2022-01-10 12:53:09,201] jepsen worker 52 - jepsen.util 52 :ok :write [17 2] INFO [2022-01-10 12:53:09,201] jepsen worker 52 - jepsen.util 52 :invoke :write [17 34] INFO [2022-01-10 12:53:09,201] jepsen worker 87 - jepsen.util 87 :ok :write [29 37] INFO [2022-01-10 12:53:09,202] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,202] jepsen worker 61 - jepsen.util 61 :ok :write [20 42] INFO [2022-01-10 12:53:09,203] jepsen worker 87 - jepsen.util 87 :ok :read [29 37] INFO [2022-01-10 12:53:09,203] jepsen worker 61 - jepsen.util 61 :invoke :write [20 18] INFO [2022-01-10 12:53:09,203] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,205] jepsen worker 87 - jepsen.util 87 :ok :read [29 37] INFO [2022-01-10 12:53:09,205] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,206] jepsen worker 87 - jepsen.util 87 :ok :read [29 37] INFO [2022-01-10 12:53:09,206] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,207] jepsen worker 87 - jepsen.util 87 :ok :read [29 37] INFO [2022-01-10 12:53:09,208] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,209] jepsen worker 87 - jepsen.util 87 :ok :read [29 37] INFO [2022-01-10 12:53:09,209] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,210] jepsen worker 87 - jepsen.util 87 :ok :read [29 37] INFO [2022-01-10 12:53:09,210] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,211] jepsen worker 71 - jepsen.util 71 :ok :write [23 39] INFO [2022-01-10 12:53:09,211] jepsen worker 87 - jepsen.util 87 :ok :read [29 37] INFO [2022-01-10 12:53:09,211] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:09,212] jepsen worker 87 - jepsen.util 87 :invoke :write [29 36] INFO [2022-01-10 12:53:09,212] jepsen worker 71 - jepsen.util 71 :ok :read [23 39] INFO [2022-01-10 12:53:09,212] jepsen worker 50 - jepsen.util 50 :ok :write [16 1] INFO [2022-01-10 12:53:09,213] jepsen worker 50 - jepsen.util 50 :invoke :write [16 22] INFO [2022-01-10 12:53:09,214] jepsen worker 71 - jepsen.util 71 :invoke :write [23 11] INFO [2022-01-10 12:53:09,217] jepsen worker 76 - jepsen.util 76 :ok :write [25 45] INFO [2022-01-10 12:53:09,217] jepsen worker 76 - jepsen.util 76 :invoke :write [25 9] INFO [2022-01-10 12:53:09,222] jepsen worker 40 - jepsen.util 40 :ok :write [13 8] INFO [2022-01-10 12:53:09,223] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:09,224] jepsen worker 40 - jepsen.util 40 :ok :read [13 8] INFO [2022-01-10 12:53:09,225] jepsen worker 40 - jepsen.util 40 :invoke :write [13 34] INFO [2022-01-10 12:53:09,225] jepsen worker 81 - jepsen.util 81 :ok :write [27 33] INFO [2022-01-10 12:53:09,226] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,227] jepsen worker 81 - jepsen.util 81 :ok :read [27 33] INFO [2022-01-10 12:53:09,228] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,230] jepsen worker 81 - jepsen.util 81 :ok :read [27 33] INFO [2022-01-10 12:53:09,230] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,232] jepsen worker 81 - jepsen.util 81 :ok :read [27 33] INFO [2022-01-10 12:53:09,232] jepsen worker 67 - jepsen.util 67 :ok :write [22 26] INFO [2022-01-10 12:53:09,232] jepsen worker 81 - jepsen.util 81 :invoke :write [27 5] INFO [2022-01-10 12:53:09,232] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:09,233] jepsen worker 23 - jepsen.util 23 :ok :write [7 36] INFO [2022-01-10 12:53:09,233] jepsen worker 23 - jepsen.util 23 :invoke :write [7 29] INFO [2022-01-10 12:53:09,233] jepsen worker 67 - jepsen.util 67 :ok :read [22 26] INFO [2022-01-10 12:53:09,234] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:09,235] jepsen worker 67 - jepsen.util 67 :ok :read [22 26] INFO [2022-01-10 12:53:09,236] jepsen worker 67 - jepsen.util 67 :invoke :write [22 28] INFO [2022-01-10 12:53:09,239] jepsen worker 15 - jepsen.util 15 :ok :write [5 19] INFO [2022-01-10 12:53:09,239] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:09,240] jepsen worker 15 - jepsen.util 15 :ok :read [5 19] INFO [2022-01-10 12:53:09,241] jepsen worker 15 - jepsen.util 15 :invoke :write [5 7] INFO [2022-01-10 12:53:09,241] jepsen worker 7 - jepsen.util 7 :ok :write [2 46] INFO [2022-01-10 12:53:09,242] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,243] jepsen worker 7 - jepsen.util 7 :ok :read [2 46] INFO [2022-01-10 12:53:09,243] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,244] jepsen worker 7 - jepsen.util 7 :ok :read [2 46] INFO [2022-01-10 12:53:09,245] jepsen worker 7 - jepsen.util 7 :invoke :write [2 48] INFO [2022-01-10 12:53:09,246] jepsen worker 43 - jepsen.util 43 :ok :write [14 46] INFO [2022-01-10 12:53:09,247] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:09,248] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:09,248] jepsen worker 43 - jepsen.util 43 :invoke :write [14 31] INFO [2022-01-10 12:53:09,250] jepsen worker 33 - jepsen.util 33 :ok :write [11 32] INFO [2022-01-10 12:53:09,251] jepsen worker 33 - jepsen.util 33 :invoke :write [11 11] INFO [2022-01-10 12:53:09,259] jepsen worker 50 - jepsen.util 50 :ok :write [16 22] INFO [2022-01-10 12:53:09,259] jepsen worker 50 - jepsen.util 50 :invoke :write [16 43] INFO [2022-01-10 12:53:09,266] jepsen worker 81 - jepsen.util 81 :ok :write [27 5] INFO [2022-01-10 12:53:09,267] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,268] jepsen worker 81 - jepsen.util 81 :ok :read [27 5] INFO [2022-01-10 12:53:09,268] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,269] jepsen worker 81 - jepsen.util 81 :ok :read [27 5] INFO [2022-01-10 12:53:09,270] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,271] jepsen worker 81 - jepsen.util 81 :ok :read [27 5] INFO [2022-01-10 12:53:09,271] jepsen worker 81 - jepsen.util 81 :invoke :write [27 7] INFO [2022-01-10 12:53:09,272] jepsen worker 67 - jepsen.util 67 :ok :write [22 28] INFO [2022-01-10 12:53:09,272] jepsen worker 67 - jepsen.util 67 :invoke :write [22 12] INFO [2022-01-10 12:53:09,277] jepsen worker 7 - jepsen.util 7 :ok :write [2 48] INFO [2022-01-10 12:53:09,278] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,278] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:09,279] jepsen worker 7 - jepsen.util 7 :invoke :write [2 3] INFO [2022-01-10 12:53:09,283] jepsen worker 15 - jepsen.util 15 :ok :write [5 7] INFO [2022-01-10 12:53:09,284] jepsen worker 15 - jepsen.util 15 :invoke :write [5 28] INFO [2022-01-10 12:53:09,289] jepsen worker 33 - jepsen.util 33 :ok :write [11 11] INFO [2022-01-10 12:53:09,290] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:09,290] jepsen worker 33 - jepsen.util 33 :ok :read [11 11] INFO [2022-01-10 12:53:09,291] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:09,292] jepsen worker 33 - jepsen.util 33 :ok :read [11 11] INFO [2022-01-10 12:53:09,292] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:09,293] jepsen worker 33 - jepsen.util 33 :ok :read [11 11] INFO [2022-01-10 12:53:09,294] jepsen worker 33 - jepsen.util 33 :invoke :write [11 31] INFO [2022-01-10 12:53:09,294] jepsen worker 50 - jepsen.util 50 :ok :write [16 43] INFO [2022-01-10 12:53:09,295] jepsen worker 50 - jepsen.util 50 :invoke :write [16 24] INFO [2022-01-10 12:53:09,299] jepsen worker 81 - jepsen.util 81 :ok :write [27 7] INFO [2022-01-10 12:53:09,300] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,301] jepsen worker 81 - jepsen.util 81 :ok :read [27 7] INFO [2022-01-10 12:53:09,302] jepsen worker 81 - jepsen.util 81 :invoke :write [27 21] INFO [2022-01-10 12:53:09,303] jepsen worker 67 - jepsen.util 67 :ok :write [22 12] INFO [2022-01-10 12:53:09,304] jepsen worker 67 - jepsen.util 67 :invoke :write [22 17] INFO [2022-01-10 12:53:09,306] jepsen worker 7 - jepsen.util 7 :ok :write [2 3] INFO [2022-01-10 12:53:09,307] jepsen worker 7 - jepsen.util 7 :invoke :write [2 42] INFO [2022-01-10 12:53:09,311] jepsen worker 15 - jepsen.util 15 :ok :write [5 28] INFO [2022-01-10 12:53:09,311] jepsen worker 15 - jepsen.util 15 :invoke :write [5 25] INFO [2022-01-10 12:53:09,313] jepsen worker 33 - jepsen.util 33 :ok :write [11 31] INFO [2022-01-10 12:53:09,314] jepsen worker 33 - jepsen.util 33 :invoke :write [11 45] INFO [2022-01-10 12:53:09,319] jepsen worker 50 - jepsen.util 50 :ok :write [16 24] INFO [2022-01-10 12:53:09,320] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,321] jepsen worker 50 - jepsen.util 50 :ok :read [16 24] INFO [2022-01-10 12:53:09,321] jepsen worker 50 - jepsen.util 50 :invoke :write [16 6] INFO [2022-01-10 12:53:09,323] jepsen worker 81 - jepsen.util 81 :ok :write [27 21] INFO [2022-01-10 12:53:09,324] jepsen worker 81 - jepsen.util 81 :invoke :write [27 33] INFO [2022-01-10 12:53:09,324] jepsen worker 13 - jepsen.util 13 :ok :write [4 29] INFO [2022-01-10 12:53:09,325] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:09,326] jepsen worker 13 - jepsen.util 13 :ok :read [4 29] INFO [2022-01-10 12:53:09,327] jepsen worker 13 - jepsen.util 13 :invoke :write [4 5] INFO [2022-01-10 12:53:09,328] jepsen worker 1 - jepsen.util 1 :ok :write [0 30] INFO [2022-01-10 12:53:09,329] jepsen worker 67 - jepsen.util 67 :ok :write [22 17] INFO [2022-01-10 12:53:09,329] jepsen worker 1 - jepsen.util 1 :invoke :write [0 36] INFO [2022-01-10 12:53:09,329] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:09,330] jepsen worker 67 - jepsen.util 67 :ok :read [22 17] INFO [2022-01-10 12:53:09,331] jepsen worker 67 - jepsen.util 67 :invoke :write [22 11] INFO [2022-01-10 12:53:09,331] jepsen worker 7 - jepsen.util 7 :ok :write [2 42] INFO [2022-01-10 12:53:09,332] jepsen worker 87 - jepsen.util 87 :ok :write [29 36] INFO [2022-01-10 12:53:09,332] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,333] jepsen worker 87 - jepsen.util 87 :invoke :write [29 5] INFO [2022-01-10 12:53:09,333] jepsen worker 7 - jepsen.util 7 :ok :read [2 42] INFO [2022-01-10 12:53:09,334] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,335] jepsen worker 3 - jepsen.util 3 :ok :write [1 19] INFO [2022-01-10 12:53:09,335] jepsen worker 7 - jepsen.util 7 :ok :read [2 42] INFO [2022-01-10 12:53:09,335] jepsen worker 3 - jepsen.util 3 :invoke :write [1 39] INFO [2022-01-10 12:53:09,336] jepsen worker 7 - jepsen.util 7 :invoke :write [2 43] INFO [2022-01-10 12:53:09,337] jepsen worker 61 - jepsen.util 61 :ok :write [20 18] INFO [2022-01-10 12:53:09,338] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:09,339] jepsen worker 61 - jepsen.util 61 :ok :read [20 18] INFO [2022-01-10 12:53:09,340] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:09,340] jepsen worker 33 - jepsen.util 33 :ok :write [11 45] INFO [2022-01-10 12:53:09,341] jepsen worker 61 - jepsen.util 61 :ok :read [20 18] INFO [2022-01-10 12:53:09,341] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:09,342] jepsen worker 61 - jepsen.util 61 :invoke :write [20 13] INFO [2022-01-10 12:53:09,342] jepsen worker 78 - jepsen.util 78 :ok :write [26 45] INFO [2022-01-10 12:53:09,342] jepsen worker 76 - jepsen.util 76 :ok :write [25 9] INFO [2022-01-10 12:53:09,342] jepsen worker 33 - jepsen.util 33 :ok :read [11 45] INFO [2022-01-10 12:53:09,343] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:09,344] jepsen worker 78 - jepsen.util 78 :invoke :write [26 6] INFO [2022-01-10 12:53:09,345] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:09,345] jepsen worker 76 - jepsen.util 76 :ok :read [25 9] INFO [2022-01-10 12:53:09,346] jepsen worker 33 - jepsen.util 33 :ok :read [11 45] INFO [2022-01-10 12:53:09,346] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:09,347] jepsen worker 33 - jepsen.util 33 :invoke :write [11 19] INFO [2022-01-10 12:53:09,347] jepsen worker 40 - jepsen.util 40 :ok :write [13 34] INFO [2022-01-10 12:53:09,347] jepsen worker 76 - jepsen.util 76 :ok :read [25 9] INFO [2022-01-10 12:53:09,349] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:09,349] jepsen worker 23 - jepsen.util 23 :ok :write [7 29] INFO [2022-01-10 12:53:09,349] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:09,349] jepsen worker 40 - jepsen.util 40 :ok :read [13 34] INFO [2022-01-10 12:53:09,350] jepsen worker 23 - jepsen.util 23 :invoke :write [7 13] INFO [2022-01-10 12:53:09,350] jepsen worker 76 - jepsen.util 76 :ok :read [25 9] INFO [2022-01-10 12:53:09,350] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:09,351] jepsen worker 40 - jepsen.util 40 :ok :read [13 34] INFO [2022-01-10 12:53:09,351] jepsen worker 76 - jepsen.util 76 :invoke :write [25 1] INFO [2022-01-10 12:53:09,352] jepsen worker 84 - jepsen.util 84 :ok :write [28 4] INFO [2022-01-10 12:53:09,352] jepsen worker 43 - jepsen.util 43 :ok :write [14 31] INFO [2022-01-10 12:53:09,352] jepsen worker 40 - jepsen.util 40 :invoke :write [13 49] INFO [2022-01-10 12:53:09,353] jepsen worker 84 - jepsen.util 84 :invoke :write [28 38] INFO [2022-01-10 12:53:09,353] jepsen worker 43 - jepsen.util 43 :invoke :write [14 41] INFO [2022-01-10 12:53:09,355] jepsen worker 15 - jepsen.util 15 :ok :write [5 25] INFO [2022-01-10 12:53:09,356] jepsen worker 15 - jepsen.util 15 :invoke :write [5 32] INFO [2022-01-10 12:53:09,358] jepsen worker 50 - jepsen.util 50 :ok :write [16 6] INFO [2022-01-10 12:53:09,359] jepsen worker 38 - jepsen.util 38 :ok :write [12 20] INFO [2022-01-10 12:53:09,359] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,360] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,360] jepsen worker 50 - jepsen.util 50 :ok :read [16 6] INFO [2022-01-10 12:53:09,360] jepsen worker 1 - jepsen.util 1 :ok :write [0 36] INFO [2022-01-10 12:53:09,361] jepsen worker 38 - jepsen.util 38 :ok :read [12 20] INFO [2022-01-10 12:53:09,361] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,362] jepsen worker 38 - jepsen.util 38 :invoke :write [12 2] INFO [2022-01-10 12:53:09,362] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,362] jepsen worker 50 - jepsen.util 50 :ok :read [16 6] INFO [2022-01-10 12:53:09,363] jepsen worker 81 - jepsen.util 81 :ok :write [27 33] INFO [2022-01-10 12:53:09,363] jepsen worker 1 - jepsen.util 1 :ok :read [0 36] INFO [2022-01-10 12:53:09,363] jepsen worker 50 - jepsen.util 50 :invoke :write [16 49] INFO [2022-01-10 12:53:09,364] jepsen worker 81 - jepsen.util 81 :invoke :write [27 40] INFO [2022-01-10 12:53:09,364] jepsen worker 52 - jepsen.util 52 :ok :write [17 34] INFO [2022-01-10 12:53:09,365] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,366] jepsen worker 1 - jepsen.util 1 :ok :read [0 36] INFO [2022-01-10 12:53:09,366] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,367] jepsen worker 67 - jepsen.util 67 :ok :write [22 11] INFO [2022-01-10 12:53:09,367] jepsen worker 1 - jepsen.util 1 :invoke :write [0 13] INFO [2022-01-10 12:53:09,367] jepsen worker 52 - jepsen.util 52 :ok :read [17 34] INFO [2022-01-10 12:53:09,367] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:09,368] jepsen worker 71 - jepsen.util 71 :ok :write [23 11] INFO [2022-01-10 12:53:09,368] jepsen worker 87 - jepsen.util 87 :ok :write [29 5] INFO [2022-01-10 12:53:09,368] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,368] jepsen worker 67 - jepsen.util 67 :ok :read [22 11] INFO [2022-01-10 12:53:09,369] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,369] jepsen worker 52 - jepsen.util 52 :ok :read [17 34] INFO [2022-01-10 12:53:09,369] jepsen worker 67 - jepsen.util 67 :invoke :write [22 1] INFO [2022-01-10 12:53:09,370] jepsen worker 87 - jepsen.util 87 :ok :read [29 5] INFO [2022-01-10 12:53:09,370] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:09,371] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,371] jepsen worker 71 - jepsen.util 71 :ok :read [23 11] INFO [2022-01-10 12:53:09,371] jepsen worker 7 - jepsen.util 7 :ok :write [2 43] INFO [2022-01-10 12:53:09,372] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,372] jepsen worker 87 - jepsen.util 87 :ok :read [29 5] INFO [2022-01-10 12:53:09,372] jepsen worker 71 - jepsen.util 71 :invoke :write [23 21] INFO [2022-01-10 12:53:09,372] jepsen worker 52 - jepsen.util 52 :ok :read [17 34] INFO [2022-01-10 12:53:09,373] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,374] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,374] jepsen worker 7 - jepsen.util 7 :ok :read [2 43] INFO [2022-01-10 12:53:09,374] jepsen worker 87 - jepsen.util 87 :invoke :write [29 13] INFO [2022-01-10 12:53:09,375] jepsen worker 52 - jepsen.util 52 :ok :read [17 34] INFO [2022-01-10 12:53:09,375] jepsen worker 7 - jepsen.util 7 :invoke :write [2 20] INFO [2022-01-10 12:53:09,375] jepsen worker 52 - jepsen.util 52 :invoke :write [17 18] INFO [2022-01-10 12:53:09,377] jepsen worker 13 - jepsen.util 13 :ok :write [4 5] INFO [2022-01-10 12:53:09,378] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:09,378] jepsen worker 61 - jepsen.util 61 :ok :write [20 13] INFO [2022-01-10 12:53:09,379] jepsen worker 13 - jepsen.util 13 :ok :read [4 5] INFO [2022-01-10 12:53:09,379] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:09,380] jepsen worker 13 - jepsen.util 13 :invoke :write [4 28] INFO [2022-01-10 12:53:09,380] jepsen worker 61 - jepsen.util 61 :ok :read [20 13] INFO [2022-01-10 12:53:09,380] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:09,381] jepsen worker 61 - jepsen.util 61 :ok :read [20 13] INFO [2022-01-10 12:53:09,382] jepsen worker 61 - jepsen.util 61 :invoke :write [20 6] INFO [2022-01-10 12:53:09,384] jepsen worker 3 - jepsen.util 3 :ok :write [1 39] INFO [2022-01-10 12:53:09,384] jepsen worker 3 - jepsen.util 3 :invoke :write [1 4] INFO [2022-01-10 12:53:09,386] jepsen worker 33 - jepsen.util 33 :ok :write [11 19] INFO [2022-01-10 12:53:09,387] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:09,388] jepsen worker 78 - jepsen.util 78 :ok :write [26 6] INFO [2022-01-10 12:53:09,388] jepsen worker 33 - jepsen.util 33 :ok :read [11 19] INFO [2022-01-10 12:53:09,388] jepsen worker 40 - jepsen.util 40 :ok :write [13 49] INFO [2022-01-10 12:53:09,388] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:09,389] jepsen worker 33 - jepsen.util 33 :invoke :write [11 46] INFO [2022-01-10 12:53:09,389] jepsen worker 78 - jepsen.util 78 :ok :read [26 6] INFO [2022-01-10 12:53:09,389] jepsen worker 40 - jepsen.util 40 :invoke :write [13 12] INFO [2022-01-10 12:53:09,390] jepsen worker 78 - jepsen.util 78 :invoke :write [26 31] INFO [2022-01-10 12:53:09,394] jepsen worker 76 - jepsen.util 76 :ok :write [25 1] INFO [2022-01-10 12:53:09,395] jepsen worker 76 - jepsen.util 76 :invoke :write [25 27] INFO [2022-01-10 12:53:09,395] jepsen worker 84 - jepsen.util 84 :ok :write [28 38] INFO [2022-01-10 12:53:09,396] jepsen worker 84 - jepsen.util 84 :invoke :write [28 43] INFO [2022-01-10 12:53:09,397] jepsen worker 38 - jepsen.util 38 :ok :write [12 2] INFO [2022-01-10 12:53:09,397] jepsen worker 38 - jepsen.util 38 :invoke :write [12 26] INFO [2022-01-10 12:53:09,399] jepsen worker 52 - jepsen.util 52 :ok :write [17 18] INFO [2022-01-10 12:53:09,400] jepsen worker 23 - jepsen.util 23 :ok :write [7 13] INFO [2022-01-10 12:53:09,400] jepsen worker 52 - jepsen.util 52 :invoke :write [17 42] INFO [2022-01-10 12:53:09,401] jepsen worker 23 - jepsen.util 23 :invoke :write [7 16] INFO [2022-01-10 12:53:09,403] jepsen worker 71 - jepsen.util 71 :ok :write [23 21] INFO [2022-01-10 12:53:09,404] jepsen worker 71 - jepsen.util 71 :invoke :write [23 36] INFO [2022-01-10 12:53:09,407] jepsen worker 13 - jepsen.util 13 :ok :write [4 28] INFO [2022-01-10 12:53:09,408] jepsen worker 13 - jepsen.util 13 :invoke :write [4 45] INFO [2022-01-10 12:53:09,409] jepsen worker 43 - jepsen.util 43 :ok :write [14 41] INFO [2022-01-10 12:53:09,409] jepsen worker 43 - jepsen.util 43 :invoke :write [14 49] INFO [2022-01-10 12:53:09,411] jepsen worker 15 - jepsen.util 15 :ok :write [5 32] INFO [2022-01-10 12:53:09,412] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:09,413] jepsen worker 15 - jepsen.util 15 :ok :read [5 32] INFO [2022-01-10 12:53:09,413] jepsen worker 15 - jepsen.util 15 :invoke :write [5 35] INFO [2022-01-10 12:53:09,416] jepsen worker 78 - jepsen.util 78 :ok :write [26 31] INFO [2022-01-10 12:53:09,417] jepsen worker 50 - jepsen.util 50 :ok :write [16 49] INFO [2022-01-10 12:53:09,417] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:09,417] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,417] jepsen worker 1 - jepsen.util 1 :ok :write [0 13] INFO [2022-01-10 12:53:09,418] jepsen worker 78 - jepsen.util 78 :ok :read [26 31] INFO [2022-01-10 12:53:09,418] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,418] jepsen worker 50 - jepsen.util 50 :ok :read [16 49] INFO [2022-01-10 12:53:09,418] jepsen worker 78 - jepsen.util 78 :invoke :write [26 8] INFO [2022-01-10 12:53:09,418] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,419] jepsen worker 1 - jepsen.util 1 :ok :read [0 13] INFO [2022-01-10 12:53:09,419] jepsen worker 1 - jepsen.util 1 :invoke :write [0 3] INFO [2022-01-10 12:53:09,420] jepsen worker 50 - jepsen.util 50 :ok :read [16 49] INFO [2022-01-10 12:53:09,420] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,421] jepsen worker 50 - jepsen.util 50 :ok :read [16 49] INFO [2022-01-10 12:53:09,421] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,422] jepsen worker 50 - jepsen.util 50 :ok :read [16 49] INFO [2022-01-10 12:53:09,423] jepsen worker 50 - jepsen.util 50 :invoke :write [16 11] INFO [2022-01-10 12:53:09,432] jepsen worker 52 - jepsen.util 52 :ok :write [17 42] INFO [2022-01-10 12:53:09,433] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,434] jepsen worker 52 - jepsen.util 52 :ok :read [17 42] INFO [2022-01-10 12:53:09,435] jepsen worker 52 - jepsen.util 52 :invoke :write [17 45] INFO [2022-01-10 12:53:09,438] jepsen worker 71 - jepsen.util 71 :ok :write [23 36] INFO [2022-01-10 12:53:09,439] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:09,440] jepsen worker 71 - jepsen.util 71 :ok :read [23 36] INFO [2022-01-10 12:53:09,440] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:09,441] jepsen worker 71 - jepsen.util 71 :ok :read [23 36] INFO [2022-01-10 12:53:09,442] jepsen worker 71 - jepsen.util 71 :invoke :write [23 7] INFO [2022-01-10 12:53:09,445] jepsen worker 13 - jepsen.util 13 :ok :write [4 45] INFO [2022-01-10 12:53:09,446] jepsen worker 13 - jepsen.util 13 :invoke :write [4 46] INFO [2022-01-10 12:53:09,452] jepsen worker 78 - jepsen.util 78 :ok :write [26 8] INFO [2022-01-10 12:53:09,453] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:09,454] jepsen worker 78 - jepsen.util 78 :ok :read [26 8] INFO [2022-01-10 12:53:09,454] jepsen worker 78 - jepsen.util 78 :invoke :write [26 1] INFO [2022-01-10 12:53:09,485] jepsen worker 81 - jepsen.util 81 :ok :write [27 40] INFO [2022-01-10 12:53:09,486] jepsen worker 81 - jepsen.util 81 :invoke :write [27 44] INFO [2022-01-10 12:53:09,490] jepsen worker 3 - jepsen.util 3 :ok :write [1 4] INFO [2022-01-10 12:53:09,491] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,492] jepsen worker 3 - jepsen.util 3 :ok :read [1 4] INFO [2022-01-10 12:53:09,492] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,493] jepsen worker 67 - jepsen.util 67 :ok :write [22 1] INFO [2022-01-10 12:53:09,493] jepsen worker 3 - jepsen.util 3 :ok :read [1 4] INFO [2022-01-10 12:53:09,493] jepsen worker 67 - jepsen.util 67 :invoke :write [22 12] INFO [2022-01-10 12:53:09,494] jepsen worker 84 - jepsen.util 84 :ok :write [28 43] INFO [2022-01-10 12:53:09,494] jepsen worker 3 - jepsen.util 3 :invoke :write [1 27] INFO [2022-01-10 12:53:09,494] jepsen worker 84 - jepsen.util 84 :invoke :write [28 14] INFO [2022-01-10 12:53:09,500] jepsen worker 38 - jepsen.util 38 :ok :write [12 26] INFO [2022-01-10 12:53:09,501] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,501] jepsen worker 38 - jepsen.util 38 :ok :read [12 26] INFO [2022-01-10 12:53:09,502] jepsen worker 87 - jepsen.util 87 :ok :write [29 13] INFO [2022-01-10 12:53:09,502] jepsen worker 38 - jepsen.util 38 :invoke :write [12 39] INFO [2022-01-10 12:53:09,502] jepsen worker 87 - jepsen.util 87 :invoke :write [29 20] INFO [2022-01-10 12:53:09,506] jepsen worker 3 - jepsen.util 3 :ok :write [1 27] INFO [2022-01-10 12:53:09,507] jepsen worker 3 - jepsen.util 3 :invoke :write [1 31] INFO [2022-01-10 12:53:09,509] jepsen worker 7 - jepsen.util 7 :ok :write [2 20] INFO [2022-01-10 12:53:09,509] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,510] jepsen worker 7 - jepsen.util 7 :ok :read [2 20] INFO [2022-01-10 12:53:09,511] jepsen worker 84 - jepsen.util 84 :ok :write [28 14] INFO [2022-01-10 12:53:09,511] jepsen worker 7 - jepsen.util 7 :invoke :write [2 38] INFO [2022-01-10 12:53:09,511] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:09,512] jepsen worker 84 - jepsen.util 84 :ok :read [28 14] INFO [2022-01-10 12:53:09,513] jepsen worker 84 - jepsen.util 84 :invoke :write [28 9] INFO [2022-01-10 12:53:09,519] jepsen worker 38 - jepsen.util 38 :ok :write [12 39] INFO [2022-01-10 12:53:09,519] jepsen worker 38 - jepsen.util 38 :invoke :write [12 18] INFO [2022-01-10 12:53:09,519] jepsen worker 40 - jepsen.util 40 :ok :write [13 12] INFO [2022-01-10 12:53:09,520] jepsen worker 40 - jepsen.util 40 :invoke :write [13 43] INFO [2022-01-10 12:53:09,524] jepsen worker 3 - jepsen.util 3 :ok :write [1 31] INFO [2022-01-10 12:53:09,525] jepsen worker 3 - jepsen.util 3 :invoke :write [1 41] INFO [2022-01-10 12:53:09,529] jepsen worker 84 - jepsen.util 84 :ok :write [28 9] INFO [2022-01-10 12:53:09,529] jepsen worker 84 - jepsen.util 84 :invoke :write [28 23] INFO [2022-01-10 12:53:09,535] jepsen worker 1 - jepsen.util 1 :ok :write [0 3] INFO [2022-01-10 12:53:09,535] jepsen worker 1 - jepsen.util 1 :invoke :write [0 30] INFO [2022-01-10 12:53:09,535] jepsen worker 38 - jepsen.util 38 :ok :write [12 18] INFO [2022-01-10 12:53:09,536] jepsen worker 38 - jepsen.util 38 :invoke :write [12 44] INFO [2022-01-10 12:53:09,549] jepsen worker 87 - jepsen.util 87 :ok :write [29 20] INFO [2022-01-10 12:53:09,550] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,551] jepsen worker 87 - jepsen.util 87 :ok :read [29 20] INFO [2022-01-10 12:53:09,551] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,552] jepsen worker 87 - jepsen.util 87 :ok :read [29 20] INFO [2022-01-10 12:53:09,552] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,553] jepsen worker 87 - jepsen.util 87 :ok :read [29 20] INFO [2022-01-10 12:53:09,553] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,554] jepsen worker 87 - jepsen.util 87 :ok :read [29 20] INFO [2022-01-10 12:53:09,555] jepsen worker 87 - jepsen.util 87 :invoke :write [29 4] INFO [2022-01-10 12:53:09,557] jepsen worker 40 - jepsen.util 40 :ok :write [13 43] INFO [2022-01-10 12:53:09,558] jepsen worker 3 - jepsen.util 3 :ok :write [1 41] INFO [2022-01-10 12:53:09,558] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:09,558] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,559] jepsen worker 3 - jepsen.util 3 :ok :read [1 41] INFO [2022-01-10 12:53:09,560] jepsen worker 3 - jepsen.util 3 :invoke :write [1 13] INFO [2022-01-10 12:53:09,563] jepsen worker 1 - jepsen.util 1 :ok :write [0 30] INFO [2022-01-10 12:53:09,563] jepsen worker 1 - jepsen.util 1 :invoke :write [0 11] INFO [2022-01-10 12:53:09,571] jepsen worker 61 - jepsen.util 61 :ok :write [20 6] INFO [2022-01-10 12:53:09,571] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:09,572] jepsen worker 61 - jepsen.util 61 :ok :read [20 6] INFO [2022-01-10 12:53:09,573] jepsen worker 61 - jepsen.util 61 :invoke :write [20 23] INFO [2022-01-10 12:53:09,576] jepsen worker 40 - jepsen.util 40 :ok :read [13 43] INFO [2022-01-10 12:53:09,577] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:09,577] jepsen worker 87 - jepsen.util 87 :ok :write [29 4] INFO [2022-01-10 12:53:09,578] jepsen worker 87 - jepsen.util 87 :invoke :write [29 13] INFO [2022-01-10 12:53:09,578] jepsen worker 40 - jepsen.util 40 :ok :read [13 43] INFO [2022-01-10 12:53:09,578] jepsen worker 40 - jepsen.util 40 :invoke :write [13 37] INFO [2022-01-10 12:53:09,578] jepsen worker 1 - jepsen.util 1 :ok :write [0 11] INFO [2022-01-10 12:53:09,579] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,579] jepsen worker 52 - jepsen.util 52 :ok :write [17 45] INFO [2022-01-10 12:53:09,579] jepsen worker 33 - jepsen.util 33 :ok :write [11 46] INFO [2022-01-10 12:53:09,580] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:09,580] jepsen worker 1 - jepsen.util 1 :ok :read [0 11] INFO [2022-01-10 12:53:09,580] jepsen worker 52 - jepsen.util 52 :invoke :write [17 18] INFO [2022-01-10 12:53:09,580] jepsen worker 1 - jepsen.util 1 :invoke :write [0 38] INFO [2022-01-10 12:53:09,581] jepsen worker 33 - jepsen.util 33 :ok :read [11 46] INFO [2022-01-10 12:53:09,581] jepsen worker 33 - jepsen.util 33 :invoke :write [11 12] INFO [2022-01-10 12:53:09,585] jepsen worker 76 - jepsen.util 76 :ok :write [25 27] INFO [2022-01-10 12:53:09,585] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:09,587] jepsen worker 76 - jepsen.util 76 :ok :read [25 27] INFO [2022-01-10 12:53:09,587] jepsen worker 76 - jepsen.util 76 :invoke :write [25 37] INFO [2022-01-10 12:53:09,588] jepsen worker 71 - jepsen.util 71 :ok :write [23 7] INFO [2022-01-10 12:53:09,588] jepsen worker 84 - jepsen.util 84 :ok :write [28 23] INFO [2022-01-10 12:53:09,589] jepsen worker 71 - jepsen.util 71 :invoke :write [23 41] INFO [2022-01-10 12:53:09,589] jepsen worker 84 - jepsen.util 84 :invoke :write [28 35] INFO [2022-01-10 12:53:09,591] jepsen worker 13 - jepsen.util 13 :ok :write [4 46] INFO [2022-01-10 12:53:09,591] jepsen worker 23 - jepsen.util 23 :ok :write [7 16] INFO [2022-01-10 12:53:09,591] jepsen worker 13 - jepsen.util 13 :invoke :write [4 35] INFO [2022-01-10 12:53:09,592] jepsen worker 23 - jepsen.util 23 :invoke :write [7 26] INFO [2022-01-10 12:53:09,596] jepsen worker 38 - jepsen.util 38 :ok :write [12 44] INFO [2022-01-10 12:53:09,596] jepsen worker 38 - jepsen.util 38 :invoke :write [12 32] INFO [2022-01-10 12:53:09,597] jepsen worker 78 - jepsen.util 78 :ok :write [26 1] INFO [2022-01-10 12:53:09,598] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:09,599] jepsen worker 78 - jepsen.util 78 :ok :read [26 1] INFO [2022-01-10 12:53:09,599] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:09,600] jepsen worker 78 - jepsen.util 78 :ok :read [26 1] INFO [2022-01-10 12:53:09,600] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:09,600] jepsen worker 3 - jepsen.util 3 :ok :write [1 13] INFO [2022-01-10 12:53:09,601] jepsen worker 78 - jepsen.util 78 :ok :read [26 1] INFO [2022-01-10 12:53:09,601] jepsen worker 3 - jepsen.util 3 :invoke :write [1 8] INFO [2022-01-10 12:53:09,601] jepsen worker 78 - jepsen.util 78 :invoke :write [26 3] INFO [2022-01-10 12:53:09,603] jepsen worker 43 - jepsen.util 43 :ok :write [14 49] INFO [2022-01-10 12:53:09,604] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:09,605] jepsen worker 43 - jepsen.util 43 :ok :read [14 49] INFO [2022-01-10 12:53:09,605] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:09,606] jepsen worker 43 - jepsen.util 43 :ok :read [14 49] INFO [2022-01-10 12:53:09,607] jepsen worker 43 - jepsen.util 43 :invoke :write [14 21] INFO [2022-01-10 12:53:09,609] jepsen worker 15 - jepsen.util 15 :ok :write [5 35] INFO [2022-01-10 12:53:09,610] jepsen worker 15 - jepsen.util 15 :invoke :write [5 41] INFO [2022-01-10 12:53:09,613] jepsen worker 50 - jepsen.util 50 :ok :write [16 11] INFO [2022-01-10 12:53:09,614] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,615] jepsen worker 50 - jepsen.util 50 :ok :read [16 11] INFO [2022-01-10 12:53:09,615] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,616] jepsen worker 50 - jepsen.util 50 :ok :read [16 11] INFO [2022-01-10 12:53:09,617] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,617] jepsen worker 50 - jepsen.util 50 :ok :read [16 11] INFO [2022-01-10 12:53:09,618] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,619] jepsen worker 50 - jepsen.util 50 :ok :read [16 11] INFO [2022-01-10 12:53:09,619] jepsen worker 81 - jepsen.util 81 :ok :write [27 44] INFO [2022-01-10 12:53:09,619] jepsen worker 50 - jepsen.util 50 :invoke :write [16 22] INFO [2022-01-10 12:53:09,620] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,621] jepsen worker 81 - jepsen.util 81 :ok :read [27 44] INFO [2022-01-10 12:53:09,621] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,622] jepsen worker 81 - jepsen.util 81 :ok :read [27 44] INFO [2022-01-10 12:53:09,623] jepsen worker 81 - jepsen.util 81 :invoke :write [27 47] INFO [2022-01-10 12:53:09,625] jepsen worker 67 - jepsen.util 67 :ok :write [22 12] INFO [2022-01-10 12:53:09,625] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:09,626] jepsen worker 67 - jepsen.util 67 :ok :read [22 12] INFO [2022-01-10 12:53:09,627] jepsen worker 67 - jepsen.util 67 :invoke :write [22 26] INFO [2022-01-10 12:53:09,637] jepsen worker 7 - jepsen.util 7 :ok :write [2 38] INFO [2022-01-10 12:53:09,637] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,638] jepsen worker 7 - jepsen.util 7 :ok :read [2 38] INFO [2022-01-10 12:53:09,639] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,640] jepsen worker 7 - jepsen.util 7 :ok :read [2 38] INFO [2022-01-10 12:53:09,640] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,641] jepsen worker 7 - jepsen.util 7 :ok :read [2 38] INFO [2022-01-10 12:53:09,641] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,642] jepsen worker 7 - jepsen.util 7 :ok :read [2 38] INFO [2022-01-10 12:53:09,643] jepsen worker 7 - jepsen.util 7 :invoke :write [2 17] INFO [2022-01-10 12:53:09,648] jepsen worker 33 - jepsen.util 33 :ok :write [11 12] INFO [2022-01-10 12:53:09,648] jepsen worker 33 - jepsen.util 33 :invoke :write [11 29] INFO [2022-01-10 12:53:09,666] jepsen worker 15 - jepsen.util 15 :ok :write [5 41] INFO [2022-01-10 12:53:09,667] jepsen worker 15 - jepsen.util 15 :invoke :write [5 34] INFO [2022-01-10 12:53:09,671] jepsen worker 50 - jepsen.util 50 :ok :write [16 22] INFO [2022-01-10 12:53:09,672] jepsen worker 50 - jepsen.util 50 :invoke :write [16 47] INFO [2022-01-10 12:53:09,673] jepsen worker 87 - jepsen.util 87 :ok :write [29 13] INFO [2022-01-10 12:53:09,674] jepsen worker 87 - jepsen.util 87 :invoke :write [29 4] INFO [2022-01-10 12:53:09,676] jepsen worker 81 - jepsen.util 81 :ok :write [27 47] INFO [2022-01-10 12:53:09,676] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,677] jepsen worker 81 - jepsen.util 81 :ok :read [27 47] INFO [2022-01-10 12:53:09,678] jepsen worker 81 - jepsen.util 81 :invoke :write [27 47] INFO [2022-01-10 12:53:09,681] jepsen worker 61 - jepsen.util 61 :ok :write [20 23] INFO [2022-01-10 12:53:09,681] jepsen worker 61 - jepsen.util 61 :invoke :write [20 1] INFO [2022-01-10 12:53:09,684] jepsen worker 67 - jepsen.util 67 :ok :write [22 26] INFO [2022-01-10 12:53:09,685] jepsen worker 67 - jepsen.util 67 :invoke :write [22 17] INFO [2022-01-10 12:53:09,686] jepsen worker 1 - jepsen.util 1 :ok :write [0 38] INFO [2022-01-10 12:53:09,687] jepsen worker 7 - jepsen.util 7 :ok :write [2 17] INFO [2022-01-10 12:53:09,687] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,688] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,688] jepsen worker 1 - jepsen.util 1 :ok :read [0 38] INFO [2022-01-10 12:53:09,688] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,689] jepsen worker 7 - jepsen.util 7 :ok :read [2 17] INFO [2022-01-10 12:53:09,689] jepsen worker 1 - jepsen.util 1 :ok :read [0 38] INFO [2022-01-10 12:53:09,690] jepsen worker 7 - jepsen.util 7 :invoke :write [2 14] INFO [2022-01-10 12:53:09,690] jepsen worker 1 - jepsen.util 1 :invoke :write [0 18] INFO [2022-01-10 12:53:09,693] jepsen worker 33 - jepsen.util 33 :ok :write [11 29] INFO [2022-01-10 12:53:09,693] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:09,695] jepsen worker 33 - jepsen.util 33 :ok :read [11 29] INFO [2022-01-10 12:53:09,695] jepsen worker 33 - jepsen.util 33 :invoke :write [11 38] INFO [2022-01-10 12:53:09,696] jepsen worker 52 - jepsen.util 52 :ok :write [17 18] INFO [2022-01-10 12:53:09,696] jepsen worker 52 - jepsen.util 52 :invoke :write [17 33] INFO [2022-01-10 12:53:09,698] jepsen worker 40 - jepsen.util 40 :ok :write [13 37] INFO [2022-01-10 12:53:09,698] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:09,699] jepsen worker 40 - jepsen.util 40 :ok :read [13 37] INFO [2022-01-10 12:53:09,700] jepsen worker 40 - jepsen.util 40 :invoke :write [13 18] INFO [2022-01-10 12:53:09,701] jepsen worker 15 - jepsen.util 15 :ok :write [5 34] INFO [2022-01-10 12:53:09,701] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:09,703] jepsen worker 50 - jepsen.util 50 :ok :write [16 47] INFO [2022-01-10 12:53:09,703] jepsen worker 15 - jepsen.util 15 :ok :read [5 34] INFO [2022-01-10 12:53:09,703] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,704] jepsen worker 15 - jepsen.util 15 :invoke :write [5 18] INFO [2022-01-10 12:53:09,704] jepsen worker 71 - jepsen.util 71 :ok :write [23 41] INFO [2022-01-10 12:53:09,704] jepsen worker 50 - jepsen.util 50 :ok :read [16 47] INFO [2022-01-10 12:53:09,705] jepsen worker 76 - jepsen.util 76 :ok :write [25 37] INFO [2022-01-10 12:53:09,705] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:09,705] jepsen worker 76 - jepsen.util 76 :invoke :write [25 23] INFO [2022-01-10 12:53:09,706] jepsen worker 50 - jepsen.util 50 :invoke :write [16 18] INFO [2022-01-10 12:53:09,706] jepsen worker 71 - jepsen.util 71 :ok :read [23 41] INFO [2022-01-10 12:53:09,707] jepsen worker 71 - jepsen.util 71 :invoke :write [23 24] INFO [2022-01-10 12:53:09,709] jepsen worker 81 - jepsen.util 81 :ok :write [27 47] INFO [2022-01-10 12:53:09,709] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,710] jepsen worker 81 - jepsen.util 81 :ok :read [27 47] INFO [2022-01-10 12:53:09,710] jepsen worker 84 - jepsen.util 84 :ok :write [28 35] INFO [2022-01-10 12:53:09,711] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,711] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:09,712] jepsen worker 23 - jepsen.util 23 :ok :write [7 26] INFO [2022-01-10 12:53:09,712] jepsen worker 81 - jepsen.util 81 :ok :read [27 47] INFO [2022-01-10 12:53:09,712] jepsen worker 84 - jepsen.util 84 :ok :read [28 35] INFO [2022-01-10 12:53:09,713] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:09,713] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:09,713] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,714] jepsen worker 23 - jepsen.util 23 :ok :read [7 26] INFO [2022-01-10 12:53:09,714] jepsen worker 84 - jepsen.util 84 :ok :read [28 35] INFO [2022-01-10 12:53:09,714] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:09,714] jepsen worker 81 - jepsen.util 81 :ok :read [27 47] INFO [2022-01-10 12:53:09,715] jepsen worker 84 - jepsen.util 84 :invoke :write [28 29] INFO [2022-01-10 12:53:09,715] jepsen worker 81 - jepsen.util 81 :invoke :write [27 16] INFO [2022-01-10 12:53:09,715] jepsen worker 23 - jepsen.util 23 :ok :read [7 26] INFO [2022-01-10 12:53:09,716] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:09,716] jepsen worker 13 - jepsen.util 13 :ok :write [4 35] INFO [2022-01-10 12:53:09,717] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:09,717] jepsen worker 23 - jepsen.util 23 :ok :read [7 26] INFO [2022-01-10 12:53:09,717] jepsen worker 43 - jepsen.util 43 :ok :write [14 21] INFO [2022-01-10 12:53:09,717] jepsen worker 23 - jepsen.util 23 :invoke :write [7 41] INFO [2022-01-10 12:53:09,718] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:09,718] jepsen worker 13 - jepsen.util 13 :ok :read [4 35] INFO [2022-01-10 12:53:09,719] jepsen worker 13 - jepsen.util 13 :invoke :write [4 36] INFO [2022-01-10 12:53:09,720] jepsen worker 43 - jepsen.util 43 :ok :read [14 21] INFO [2022-01-10 12:53:09,720] jepsen worker 43 - jepsen.util 43 :invoke :write [14 19] INFO [2022-01-10 12:53:09,724] jepsen worker 87 - jepsen.util 87 :ok :write [29 4] INFO [2022-01-10 12:53:09,724] jepsen worker 87 - jepsen.util 87 :invoke :write [29 49] INFO [2022-01-10 12:53:09,725] jepsen worker 38 - jepsen.util 38 :ok :write [12 32] INFO [2022-01-10 12:53:09,726] jepsen worker 67 - jepsen.util 67 :ok :write [22 17] INFO [2022-01-10 12:53:09,726] jepsen worker 38 - jepsen.util 38 :invoke :write [12 29] INFO [2022-01-10 12:53:09,727] jepsen worker 67 - jepsen.util 67 :invoke :write [22 6] INFO [2022-01-10 12:53:09,730] jepsen worker 7 - jepsen.util 7 :ok :write [2 14] INFO [2022-01-10 12:53:09,731] jepsen worker 7 - jepsen.util 7 :invoke :write [2 28] INFO [2022-01-10 12:53:09,734] jepsen worker 78 - jepsen.util 78 :ok :write [26 3] INFO [2022-01-10 12:53:09,734] jepsen worker 61 - jepsen.util 61 :ok :write [20 1] INFO [2022-01-10 12:53:09,735] jepsen worker 61 - jepsen.util 61 :invoke :write [20 15] INFO [2022-01-10 12:53:09,735] jepsen worker 78 - jepsen.util 78 :invoke :write [26 46] INFO [2022-01-10 12:53:09,741] jepsen worker 33 - jepsen.util 33 :ok :write [11 38] INFO [2022-01-10 12:53:09,741] jepsen worker 33 - jepsen.util 33 :invoke :write [11 18] INFO [2022-01-10 12:53:09,742] jepsen worker 3 - jepsen.util 3 :ok :write [1 8] INFO [2022-01-10 12:53:09,742] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,742] jepsen worker 1 - jepsen.util 1 :ok :write [0 18] INFO [2022-01-10 12:53:09,743] jepsen worker 1 - jepsen.util 1 :invoke :write [0 6] INFO [2022-01-10 12:53:09,743] jepsen worker 3 - jepsen.util 3 :ok :read [1 8] INFO [2022-01-10 12:53:09,744] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,745] jepsen worker 3 - jepsen.util 3 :ok :read [1 8] INFO [2022-01-10 12:53:09,745] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,746] jepsen worker 3 - jepsen.util 3 :ok :read [1 8] INFO [2022-01-10 12:53:09,747] jepsen worker 3 - jepsen.util 3 :invoke :write [1 27] INFO [2022-01-10 12:53:09,748] jepsen worker 15 - jepsen.util 15 :ok :write [5 18] INFO [2022-01-10 12:53:09,749] jepsen worker 15 - jepsen.util 15 :invoke :write [5 24] INFO [2022-01-10 12:53:09,751] jepsen worker 52 - jepsen.util 52 :ok :write [17 33] INFO [2022-01-10 12:53:09,751] jepsen worker 52 - jepsen.util 52 :invoke :write [17 13] INFO [2022-01-10 12:53:09,752] jepsen worker 40 - jepsen.util 40 :ok :write [13 18] INFO [2022-01-10 12:53:09,753] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:09,754] jepsen worker 40 - jepsen.util 40 :ok :read [13 18] INFO [2022-01-10 12:53:09,754] jepsen worker 50 - jepsen.util 50 :ok :write [16 18] INFO [2022-01-10 12:53:09,755] jepsen worker 40 - jepsen.util 40 :invoke :write [13 3] INFO [2022-01-10 12:53:09,755] jepsen worker 50 - jepsen.util 50 :invoke :write [16 43] INFO [2022-01-10 12:53:09,759] jepsen worker 71 - jepsen.util 71 :ok :write [23 24] INFO [2022-01-10 12:53:09,759] jepsen worker 76 - jepsen.util 76 :ok :write [25 23] INFO [2022-01-10 12:53:09,759] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:09,760] jepsen worker 76 - jepsen.util 76 :invoke :write [25 9] INFO [2022-01-10 12:53:09,760] jepsen worker 71 - jepsen.util 71 :ok :read [23 24] INFO [2022-01-10 12:53:09,761] jepsen worker 71 - jepsen.util 71 :invoke :write [23 42] INFO [2022-01-10 12:53:09,766] jepsen worker 81 - jepsen.util 81 :ok :write [27 16] INFO [2022-01-10 12:53:09,766] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,767] jepsen worker 84 - jepsen.util 84 :ok :write [28 29] INFO [2022-01-10 12:53:09,767] jepsen worker 84 - jepsen.util 84 :invoke :write [28 15] INFO [2022-01-10 12:53:09,768] jepsen worker 81 - jepsen.util 81 :ok :read [27 16] INFO [2022-01-10 12:53:09,768] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,769] jepsen worker 23 - jepsen.util 23 :ok :write [7 41] INFO [2022-01-10 12:53:09,769] jepsen worker 81 - jepsen.util 81 :ok :read [27 16] INFO [2022-01-10 12:53:09,770] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,771] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:09,771] jepsen worker 81 - jepsen.util 81 :ok :read [27 16] INFO [2022-01-10 12:53:09,772] jepsen worker 23 - jepsen.util 23 :ok :read [7 41] INFO [2022-01-10 12:53:09,772] jepsen worker 13 - jepsen.util 13 :ok :write [4 36] INFO [2022-01-10 12:53:09,772] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,773] jepsen worker 81 - jepsen.util 81 :ok :read [27 16] INFO [2022-01-10 12:53:09,773] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:09,774] jepsen worker 43 - jepsen.util 43 :ok :write [14 19] INFO [2022-01-10 12:53:09,774] jepsen worker 23 - jepsen.util 23 :ok :read [7 41] INFO [2022-01-10 12:53:09,774] jepsen worker 81 - jepsen.util 81 :invoke :write [27 27] INFO [2022-01-10 12:53:09,775] jepsen worker 23 - jepsen.util 23 :invoke :write [7 40] INFO [2022-01-10 12:53:09,776] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:09,777] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:09,777] jepsen worker 87 - jepsen.util 87 :ok :write [29 49] INFO [2022-01-10 12:53:09,777] jepsen worker 43 - jepsen.util 43 :ok :read [14 19] INFO [2022-01-10 12:53:09,778] jepsen worker 13 - jepsen.util 13 :ok :read [4 36] INFO [2022-01-10 12:53:09,779] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:09,779] jepsen worker 38 - jepsen.util 38 :ok :write [12 29] INFO [2022-01-10 12:53:09,780] jepsen worker 43 - jepsen.util 43 :ok :read [14 19] INFO [2022-01-10 12:53:09,780] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:09,780] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,781] jepsen worker 13 - jepsen.util 13 :ok :read [4 36] INFO [2022-01-10 12:53:09,781] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:09,781] jepsen worker 67 - jepsen.util 67 :ok :write [22 6] INFO [2022-01-10 12:53:09,781] jepsen worker 87 - jepsen.util 87 :ok :read [29 49] INFO [2022-01-10 12:53:09,781] jepsen worker 38 - jepsen.util 38 :invoke :write [12 38] INFO [2022-01-10 12:53:09,781] jepsen worker 43 - jepsen.util 43 :ok :read [14 19] INFO [2022-01-10 12:53:09,782] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:09,783] jepsen worker 13 - jepsen.util 13 :invoke :write [4 17] INFO [2022-01-10 12:53:09,783] jepsen worker 43 - jepsen.util 43 :invoke :write [14 37] INFO [2022-01-10 12:53:09,783] jepsen worker 67 - jepsen.util 67 :ok :read [22 6] INFO [2022-01-10 12:53:09,784] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:09,784] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,785] jepsen worker 67 - jepsen.util 67 :ok :read [22 6] INFO [2022-01-10 12:53:09,785] jepsen worker 87 - jepsen.util 87 :ok :read [29 49] INFO [2022-01-10 12:53:09,785] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:09,786] jepsen worker 87 - jepsen.util 87 :invoke :write [29 10] INFO [2022-01-10 12:53:09,786] jepsen worker 67 - jepsen.util 67 :ok :read [22 6] INFO [2022-01-10 12:53:09,787] jepsen worker 67 - jepsen.util 67 :invoke :write [22 20] INFO [2022-01-10 12:53:09,788] jepsen worker 78 - jepsen.util 78 :ok :write [26 46] INFO [2022-01-10 12:53:09,788] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:09,789] jepsen worker 78 - jepsen.util 78 :ok :read [26 46] INFO [2022-01-10 12:53:09,790] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:09,791] jepsen worker 78 - jepsen.util 78 :ok :read [26 46] INFO [2022-01-10 12:53:09,792] jepsen worker 61 - jepsen.util 61 :ok :write [20 15] INFO [2022-01-10 12:53:09,792] jepsen worker 61 - jepsen.util 61 :invoke :write [20 12] INFO [2022-01-10 12:53:09,793] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:09,794] jepsen worker 78 - jepsen.util 78 :ok :read [26 46] INFO [2022-01-10 12:53:09,794] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:09,795] jepsen worker 78 - jepsen.util 78 :ok :read [26 46] INFO [2022-01-10 12:53:09,796] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:09,797] jepsen worker 78 - jepsen.util 78 :ok :read [26 46] INFO [2022-01-10 12:53:09,797] jepsen worker 78 - jepsen.util 78 :invoke :write [26 4] INFO [2022-01-10 12:53:09,802] jepsen worker 52 - jepsen.util 52 :ok :write [17 13] INFO [2022-01-10 12:53:09,803] jepsen worker 52 - jepsen.util 52 :invoke :write [17 37] INFO [2022-01-10 12:53:09,809] jepsen worker 71 - jepsen.util 71 :ok :write [23 42] INFO [2022-01-10 12:53:09,810] jepsen worker 71 - jepsen.util 71 :invoke :write [23 13] INFO [2022-01-10 12:53:09,815] jepsen worker 76 - jepsen.util 76 :ok :write [25 9] INFO [2022-01-10 12:53:09,816] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:09,817] jepsen worker 76 - jepsen.util 76 :ok :read [25 9] INFO [2022-01-10 12:53:09,817] jepsen worker 76 - jepsen.util 76 :invoke :write [25 48] INFO [2022-01-10 12:53:09,820] jepsen worker 13 - jepsen.util 13 :ok :write [4 17] INFO [2022-01-10 12:53:09,821] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:09,822] jepsen worker 13 - jepsen.util 13 :ok :read [4 17] INFO [2022-01-10 12:53:09,823] jepsen worker 13 - jepsen.util 13 :invoke :write [4 48] INFO [2022-01-10 12:53:09,828] jepsen worker 23 - jepsen.util 23 :ok :write [7 40] INFO [2022-01-10 12:53:09,829] jepsen worker 23 - jepsen.util 23 :invoke :write [7 1] INFO [2022-01-10 12:53:09,833] jepsen worker 78 - jepsen.util 78 :ok :write [26 4] INFO [2022-01-10 12:53:09,834] jepsen worker 78 - jepsen.util 78 :invoke :write [26 30] INFO [2022-01-10 12:53:09,840] jepsen worker 52 - jepsen.util 52 :ok :write [17 37] INFO [2022-01-10 12:53:09,840] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,841] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:09,842] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,842] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:09,843] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,843] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:09,844] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:09,845] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:09,845] jepsen worker 71 - jepsen.util 71 :ok :write [23 13] INFO [2022-01-10 12:53:09,845] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:09,846] jepsen worker 52 - jepsen.util 52 :invoke :write [17 1] INFO [2022-01-10 12:53:09,846] jepsen worker 71 - jepsen.util 71 :ok :read [23 13] INFO [2022-01-10 12:53:09,847] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:09,848] jepsen worker 71 - jepsen.util 71 :ok :read [23 13] INFO [2022-01-10 12:53:09,848] jepsen worker 71 - jepsen.util 71 :invoke :write [23 5] INFO [2022-01-10 12:53:09,850] jepsen worker 13 - jepsen.util 13 :ok :write [4 48] INFO [2022-01-10 12:53:09,850] jepsen worker 13 - jepsen.util 13 :invoke :write [4 31] INFO [2022-01-10 12:53:09,858] jepsen worker 3 - jepsen.util 3 :ok :write [1 27] INFO [2022-01-10 12:53:09,858] jepsen worker 3 - jepsen.util 3 :invoke :write [1 24] INFO [2022-01-10 12:53:09,859] jepsen worker 7 - jepsen.util 7 :ok :write [2 28] INFO [2022-01-10 12:53:09,859] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,860] jepsen worker 7 - jepsen.util 7 :ok :read [2 28] INFO [2022-01-10 12:53:09,860] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:09,861] jepsen worker 7 - jepsen.util 7 :ok :read [2 28] INFO [2022-01-10 12:53:09,861] jepsen worker 7 - jepsen.util 7 :invoke :write [2 6] INFO [2022-01-10 12:53:09,862] jepsen worker 84 - jepsen.util 84 :ok :write [28 15] INFO [2022-01-10 12:53:09,862] jepsen worker 84 - jepsen.util 84 :invoke :write [28 37] INFO [2022-01-10 12:53:09,863] jepsen worker 1 - jepsen.util 1 :ok :write [0 6] INFO [2022-01-10 12:53:09,863] jepsen worker 1 - jepsen.util 1 :invoke :write [0 30] INFO [2022-01-10 12:53:09,869] jepsen worker 33 - jepsen.util 33 :ok :write [11 18] INFO [2022-01-10 12:53:09,870] jepsen worker 33 - jepsen.util 33 :invoke :write [11 34] INFO [2022-01-10 12:53:09,870] jepsen worker 38 - jepsen.util 38 :ok :write [12 38] INFO [2022-01-10 12:53:09,871] jepsen worker 38 - jepsen.util 38 :invoke :write [12 39] INFO [2022-01-10 12:53:09,876] jepsen worker 15 - jepsen.util 15 :ok :write [5 24] INFO [2022-01-10 12:53:09,876] jepsen worker 15 - jepsen.util 15 :invoke :write [5 7] INFO [2022-01-10 12:53:09,877] jepsen worker 3 - jepsen.util 3 :ok :write [1 24] INFO [2022-01-10 12:53:09,878] jepsen worker 3 - jepsen.util 3 :invoke :write [1 37] INFO [2022-01-10 12:53:09,883] jepsen worker 84 - jepsen.util 84 :ok :write [28 37] INFO [2022-01-10 12:53:09,883] jepsen worker 84 - jepsen.util 84 :invoke :write [28 28] INFO [2022-01-10 12:53:09,884] jepsen worker 40 - jepsen.util 40 :ok :write [13 3] INFO [2022-01-10 12:53:09,884] jepsen worker 40 - jepsen.util 40 :invoke :write [13 31] INFO [2022-01-10 12:53:09,886] jepsen worker 38 - jepsen.util 38 :ok :write [12 39] INFO [2022-01-10 12:53:09,887] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,888] jepsen worker 38 - jepsen.util 38 :ok :read [12 39] INFO [2022-01-10 12:53:09,888] jepsen worker 38 - jepsen.util 38 :invoke :write [12 5] INFO [2022-01-10 12:53:09,890] jepsen worker 50 - jepsen.util 50 :ok :write [16 43] INFO [2022-01-10 12:53:09,890] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,891] jepsen worker 50 - jepsen.util 50 :ok :read [16 43] INFO [2022-01-10 12:53:09,892] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,893] jepsen worker 50 - jepsen.util 50 :ok :read [16 43] INFO [2022-01-10 12:53:09,893] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,894] jepsen worker 50 - jepsen.util 50 :ok :read [16 43] INFO [2022-01-10 12:53:09,894] jepsen worker 3 - jepsen.util 3 :ok :write [1 37] INFO [2022-01-10 12:53:09,895] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:09,895] jepsen worker 81 - jepsen.util 81 :ok :write [27 27] INFO [2022-01-10 12:53:09,896] jepsen worker 3 - jepsen.util 3 :invoke :write [1 14] INFO [2022-01-10 12:53:09,896] jepsen worker 50 - jepsen.util 50 :ok :read [16 43] INFO [2022-01-10 12:53:09,896] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:09,897] jepsen worker 50 - jepsen.util 50 :invoke :write [16 7] INFO [2022-01-10 12:53:09,897] jepsen worker 81 - jepsen.util 81 :ok :read [27 27] INFO [2022-01-10 12:53:09,898] jepsen worker 81 - jepsen.util 81 :invoke :write [27 34] INFO [2022-01-10 12:53:09,901] jepsen worker 87 - jepsen.util 87 :ok :write [29 10] INFO [2022-01-10 12:53:09,901] jepsen worker 84 - jepsen.util 84 :ok :write [28 28] INFO [2022-01-10 12:53:09,902] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,903] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:09,903] jepsen worker 87 - jepsen.util 87 :ok :read [29 10] INFO [2022-01-10 12:53:09,904] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,905] jepsen worker 87 - jepsen.util 87 :ok :read [29 10] INFO [2022-01-10 12:53:09,905] jepsen worker 84 - jepsen.util 84 :ok :read [28 28] INFO [2022-01-10 12:53:09,905] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:09,906] jepsen worker 84 - jepsen.util 84 :invoke :write [28 22] INFO [2022-01-10 12:53:09,906] jepsen worker 87 - jepsen.util 87 :ok :read [29 10] INFO [2022-01-10 12:53:09,907] jepsen worker 87 - jepsen.util 87 :invoke :write [29 37] INFO [2022-01-10 12:53:09,910] jepsen worker 38 - jepsen.util 38 :ok :write [12 5] INFO [2022-01-10 12:53:09,911] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,912] jepsen worker 38 - jepsen.util 38 :ok :read [12 5] INFO [2022-01-10 12:53:09,912] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,913] jepsen worker 38 - jepsen.util 38 :ok :read [12 5] INFO [2022-01-10 12:53:09,913] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,914] jepsen worker 38 - jepsen.util 38 :ok :read [12 5] INFO [2022-01-10 12:53:09,915] jepsen worker 38 - jepsen.util 38 :invoke :write [12 29] INFO [2022-01-10 12:53:09,916] jepsen worker 1 - jepsen.util 1 :ok :write [0 30] INFO [2022-01-10 12:53:09,917] jepsen worker 3 - jepsen.util 3 :ok :write [1 14] INFO [2022-01-10 12:53:09,917] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,918] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,919] jepsen worker 3 - jepsen.util 3 :ok :read [1 14] INFO [2022-01-10 12:53:09,919] jepsen worker 1 - jepsen.util 1 :ok :read [0 30] INFO [2022-01-10 12:53:09,919] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:09,920] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,920] jepsen worker 3 - jepsen.util 3 :ok :read [1 14] INFO [2022-01-10 12:53:09,920] jepsen worker 1 - jepsen.util 1 :ok :read [0 30] INFO [2022-01-10 12:53:09,921] jepsen worker 3 - jepsen.util 3 :invoke :write [1 49] INFO [2022-01-10 12:53:09,921] jepsen worker 84 - jepsen.util 84 :ok :write [28 22] INFO [2022-01-10 12:53:09,921] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,922] jepsen worker 84 - jepsen.util 84 :invoke :write [28 34] INFO [2022-01-10 12:53:09,922] jepsen worker 1 - jepsen.util 1 :ok :read [0 30] INFO [2022-01-10 12:53:09,923] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,924] jepsen worker 1 - jepsen.util 1 :ok :read [0 30] INFO [2022-01-10 12:53:09,924] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,925] jepsen worker 1 - jepsen.util 1 :ok :read [0 30] INFO [2022-01-10 12:53:09,925] jepsen worker 1 - jepsen.util 1 :invoke :write [0 34] INFO [2022-01-10 12:53:09,926] jepsen worker 38 - jepsen.util 38 :ok :write [12 29] INFO [2022-01-10 12:53:09,927] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:09,927] jepsen worker 38 - jepsen.util 38 :ok :read [12 29] INFO [2022-01-10 12:53:09,928] jepsen worker 38 - jepsen.util 38 :invoke :write [12 7] INFO [2022-01-10 12:53:09,931] jepsen worker 3 - jepsen.util 3 :ok :write [1 49] INFO [2022-01-10 12:53:09,932] jepsen worker 3 - jepsen.util 3 :invoke :write [1 10] INFO [2022-01-10 12:53:09,937] jepsen worker 43 - jepsen.util 43 :ok :write [14 37] INFO [2022-01-10 12:53:09,937] jepsen worker 84 - jepsen.util 84 :ok :write [28 34] INFO [2022-01-10 12:53:09,937] jepsen worker 40 - jepsen.util 40 :ok :write [13 31] INFO [2022-01-10 12:53:09,938] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:09,939] jepsen worker 43 - jepsen.util 43 :invoke :write [14 12] INFO [2022-01-10 12:53:09,939] jepsen worker 40 - jepsen.util 40 :invoke :write [13 28] INFO [2022-01-10 12:53:09,939] jepsen worker 84 - jepsen.util 84 :ok :read [28 34] INFO [2022-01-10 12:53:09,940] jepsen worker 84 - jepsen.util 84 :invoke :write [28 27] INFO [2022-01-10 12:53:09,944] jepsen worker 67 - jepsen.util 67 :ok :write [22 20] INFO [2022-01-10 12:53:09,944] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:09,945] jepsen worker 67 - jepsen.util 67 :ok :read [22 20] INFO [2022-01-10 12:53:09,945] jepsen worker 38 - jepsen.util 38 :ok :write [12 7] INFO [2022-01-10 12:53:09,946] jepsen worker 67 - jepsen.util 67 :invoke :write [22 38] INFO [2022-01-10 12:53:09,946] jepsen worker 38 - jepsen.util 38 :invoke :write [12 22] INFO [2022-01-10 12:53:09,951] jepsen worker 3 - jepsen.util 3 :ok :write [1 10] INFO [2022-01-10 12:53:09,952] jepsen worker 3 - jepsen.util 3 :invoke :write [1 5] INFO [2022-01-10 12:53:09,957] jepsen worker 87 - jepsen.util 87 :ok :write [29 37] INFO [2022-01-10 12:53:09,958] jepsen worker 87 - jepsen.util 87 :invoke :write [29 34] INFO [2022-01-10 12:53:09,963] jepsen worker 1 - jepsen.util 1 :ok :write [0 34] INFO [2022-01-10 12:53:09,963] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:09,964] jepsen worker 1 - jepsen.util 1 :ok :read [0 34] INFO [2022-01-10 12:53:09,964] jepsen worker 1 - jepsen.util 1 :invoke :write [0 36] INFO [2022-01-10 12:53:10,129] jepsen worker 78 - jepsen.util 78 :ok :write [26 30] INFO [2022-01-10 12:53:10,129] jepsen worker 40 - jepsen.util 40 :ok :write [13 28] INFO [2022-01-10 12:53:10,129] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,130] jepsen worker 40 - jepsen.util 40 :invoke :write [13 37] INFO [2022-01-10 12:53:10,130] jepsen worker 78 - jepsen.util 78 :ok :read [26 30] INFO [2022-01-10 12:53:10,130] jepsen worker 84 - jepsen.util 84 :ok :write [28 27] INFO [2022-01-10 12:53:10,131] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:10,132] jepsen worker 78 - jepsen.util 78 :invoke :write [26 28] INFO [2022-01-10 12:53:10,132] jepsen worker 84 - jepsen.util 84 :ok :read [28 27] INFO [2022-01-10 12:53:10,133] jepsen worker 61 - jepsen.util 61 :ok :write [20 12] INFO [2022-01-10 12:53:10,133] jepsen worker 84 - jepsen.util 84 :invoke :write [28 28] INFO [2022-01-10 12:53:10,134] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,135] jepsen worker 61 - jepsen.util 61 :ok :read [20 12] INFO [2022-01-10 12:53:10,135] jepsen worker 52 - jepsen.util 52 :ok :write [17 1] INFO [2022-01-10 12:53:10,136] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,137] jepsen worker 38 - jepsen.util 38 :ok :write [12 22] INFO [2022-01-10 12:53:10,137] jepsen worker 61 - jepsen.util 61 :invoke :write [20 10] INFO [2022-01-10 12:53:10,137] jepsen worker 52 - jepsen.util 52 :ok :read [17 1] INFO [2022-01-10 12:53:10,138] jepsen worker 38 - jepsen.util 38 :invoke :write [12 46] INFO [2022-01-10 12:53:10,138] jepsen worker 76 - jepsen.util 76 :ok :write [25 48] INFO [2022-01-10 12:53:10,139] jepsen worker 52 - jepsen.util 52 :invoke :write [17 39] INFO [2022-01-10 12:53:10,140] jepsen worker 76 - jepsen.util 76 :invoke :write [25 26] INFO [2022-01-10 12:53:10,142] jepsen worker 71 - jepsen.util 71 :ok :write [23 5] INFO [2022-01-10 12:53:10,142] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,143] jepsen worker 71 - jepsen.util 71 :ok :read [23 5] INFO [2022-01-10 12:53:10,144] jepsen worker 23 - jepsen.util 23 :ok :write [7 1] INFO [2022-01-10 12:53:10,144] jepsen worker 71 - jepsen.util 71 :invoke :write [23 1] INFO [2022-01-10 12:53:10,144] jepsen worker 87 - jepsen.util 87 :ok :write [29 34] INFO [2022-01-10 12:53:10,145] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,145] jepsen worker 23 - jepsen.util 23 :invoke :write [7 10] INFO [2022-01-10 12:53:10,146] jepsen worker 87 - jepsen.util 87 :ok :read [29 34] INFO [2022-01-10 12:53:10,146] jepsen worker 3 - jepsen.util 3 :ok :write [1 5] INFO [2022-01-10 12:53:10,147] jepsen worker 87 - jepsen.util 87 :invoke :write [29 27] INFO [2022-01-10 12:53:10,147] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:10,149] jepsen worker 3 - jepsen.util 3 :ok :read [1 5] INFO [2022-01-10 12:53:10,149] jepsen worker 13 - jepsen.util 13 :ok :write [4 31] INFO [2022-01-10 12:53:10,150] jepsen worker 3 - jepsen.util 3 :invoke :write [1 49] INFO [2022-01-10 12:53:10,151] jepsen worker 13 - jepsen.util 13 :invoke :write [4 34] INFO [2022-01-10 12:53:10,152] jepsen worker 7 - jepsen.util 7 :ok :write [2 6] INFO [2022-01-10 12:53:10,152] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:10,154] jepsen worker 7 - jepsen.util 7 :ok :read [2 6] INFO [2022-01-10 12:53:10,154] jepsen worker 1 - jepsen.util 1 :ok :write [0 36] INFO [2022-01-10 12:53:10,154] jepsen worker 78 - jepsen.util 78 :ok :write [26 28] INFO [2022-01-10 12:53:10,154] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:10,155] jepsen worker 33 - jepsen.util 33 :ok :write [11 34] INFO [2022-01-10 12:53:10,155] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:10,155] jepsen worker 7 - jepsen.util 7 :ok :read [2 6] INFO [2022-01-10 12:53:10,156] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,156] jepsen worker 1 - jepsen.util 1 :ok :read [0 36] INFO [2022-01-10 12:53:10,157] jepsen worker 78 - jepsen.util 78 :invoke :write [26 18] INFO [2022-01-10 12:53:10,157] jepsen worker 33 - jepsen.util 33 :ok :read [11 34] INFO [2022-01-10 12:53:10,157] jepsen worker 1 - jepsen.util 1 :invoke :write [0 1] INFO [2022-01-10 12:53:10,158] jepsen worker 7 - jepsen.util 7 :invoke :write [2 2] INFO [2022-01-10 12:53:10,159] jepsen worker 33 - jepsen.util 33 :invoke :write [11 23] INFO [2022-01-10 12:53:10,162] jepsen worker 84 - jepsen.util 84 :ok :write [28 28] INFO [2022-01-10 12:53:10,162] jepsen worker 84 - jepsen.util 84 :invoke :write [28 26] INFO [2022-01-10 12:53:10,163] jepsen worker 40 - jepsen.util 40 :ok :write [13 37] INFO [2022-01-10 12:53:10,163] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,164] jepsen worker 15 - jepsen.util 15 :ok :write [5 7] INFO [2022-01-10 12:53:10,164] jepsen worker 40 - jepsen.util 40 :ok :read [13 37] INFO [2022-01-10 12:53:10,165] jepsen worker 15 - jepsen.util 15 :invoke :write [5 47] INFO [2022-01-10 12:53:10,165] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,166] jepsen worker 40 - jepsen.util 40 :ok :read [13 37] INFO [2022-01-10 12:53:10,167] jepsen worker 40 - jepsen.util 40 :invoke :write [13 25] INFO [2022-01-10 12:53:10,167] jepsen worker 52 - jepsen.util 52 :ok :write [17 39] INFO [2022-01-10 12:53:10,168] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,169] jepsen worker 52 - jepsen.util 52 :ok :read [17 39] INFO [2022-01-10 12:53:10,169] jepsen worker 50 - jepsen.util 50 :ok :write [16 7] INFO [2022-01-10 12:53:10,170] jepsen worker 52 - jepsen.util 52 :invoke :write [17 40] INFO [2022-01-10 12:53:10,171] jepsen worker 50 - jepsen.util 50 :invoke :write [16 27] INFO [2022-01-10 12:53:10,173] jepsen worker 38 - jepsen.util 38 :ok :write [12 46] INFO [2022-01-10 12:53:10,174] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:10,175] jepsen worker 38 - jepsen.util 38 :ok :read [12 46] INFO [2022-01-10 12:53:10,175] jepsen worker 81 - jepsen.util 81 :ok :write [27 34] INFO [2022-01-10 12:53:10,175] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:10,176] jepsen worker 38 - jepsen.util 38 :ok :read [12 46] INFO [2022-01-10 12:53:10,177] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:10,177] jepsen worker 87 - jepsen.util 87 :ok :write [29 27] INFO [2022-01-10 12:53:10,177] jepsen worker 38 - jepsen.util 38 :invoke :write [12 2] INFO [2022-01-10 12:53:10,177] jepsen worker 81 - jepsen.util 81 :ok :read [27 34] INFO [2022-01-10 12:53:10,178] jepsen worker 71 - jepsen.util 71 :ok :write [23 1] INFO [2022-01-10 12:53:10,178] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,179] jepsen worker 43 - jepsen.util 43 :ok :write [14 12] INFO [2022-01-10 12:53:10,179] jepsen worker 81 - jepsen.util 81 :invoke :write [27 42] INFO [2022-01-10 12:53:10,179] jepsen worker 87 - jepsen.util 87 :ok :read [29 27] INFO [2022-01-10 12:53:10,180] jepsen worker 71 - jepsen.util 71 :invoke :write [23 6] INFO [2022-01-10 12:53:10,181] jepsen worker 43 - jepsen.util 43 :invoke :write [14 2] INFO [2022-01-10 12:53:10,181] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,182] jepsen worker 87 - jepsen.util 87 :ok :read [29 27] INFO [2022-01-10 12:53:10,183] jepsen worker 87 - jepsen.util 87 :invoke :write [29 23] INFO [2022-01-10 12:53:10,184] jepsen worker 3 - jepsen.util 3 :ok :write [1 49] INFO [2022-01-10 12:53:10,185] jepsen worker 3 - jepsen.util 3 :invoke :write [1 20] INFO [2022-01-10 12:53:10,188] jepsen worker 67 - jepsen.util 67 :ok :write [22 38] INFO [2022-01-10 12:53:10,189] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:10,190] jepsen worker 1 - jepsen.util 1 :ok :write [0 1] INFO [2022-01-10 12:53:10,190] jepsen worker 13 - jepsen.util 13 :ok :write [4 34] INFO [2022-01-10 12:53:10,190] jepsen worker 67 - jepsen.util 67 :ok :read [22 38] INFO [2022-01-10 12:53:10,190] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:10,191] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:10,191] jepsen worker 1 - jepsen.util 1 :ok :read [0 1] INFO [2022-01-10 12:53:10,191] jepsen worker 61 - jepsen.util 61 :ok :write [20 10] INFO [2022-01-10 12:53:10,191] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,192] jepsen worker 67 - jepsen.util 67 :ok :read [22 38] INFO [2022-01-10 12:53:10,192] jepsen worker 1 - jepsen.util 1 :invoke :write [0 47] INFO [2022-01-10 12:53:10,192] jepsen worker 13 - jepsen.util 13 :ok :read [4 34] INFO [2022-01-10 12:53:10,192] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:10,193] jepsen worker 61 - jepsen.util 61 :invoke :write [20 31] INFO [2022-01-10 12:53:10,193] jepsen worker 13 - jepsen.util 13 :invoke :write [4 1] INFO [2022-01-10 12:53:10,194] jepsen worker 67 - jepsen.util 67 :ok :read [22 38] INFO [2022-01-10 12:53:10,194] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:10,195] jepsen worker 67 - jepsen.util 67 :ok :read [22 38] INFO [2022-01-10 12:53:10,195] jepsen worker 67 - jepsen.util 67 :invoke :write [22 40] INFO [2022-01-10 12:53:10,196] jepsen worker 40 - jepsen.util 40 :ok :write [13 25] INFO [2022-01-10 12:53:10,196] jepsen worker 40 - jepsen.util 40 :invoke :write [13 0] INFO [2022-01-10 12:53:10,197] jepsen worker 76 - jepsen.util 76 :ok :write [25 26] INFO [2022-01-10 12:53:10,197] jepsen worker 78 - jepsen.util 78 :ok :write [26 18] INFO [2022-01-10 12:53:10,197] jepsen worker 76 - jepsen.util 76 :invoke :write [25 41] INFO [2022-01-10 12:53:10,198] jepsen worker 78 - jepsen.util 78 :invoke :write [26 4] INFO [2022-01-10 12:53:10,200] jepsen worker 87 - jepsen.util 87 :ok :write [29 23] INFO [2022-01-10 12:53:10,201] jepsen worker 87 - jepsen.util 87 :invoke :write [29 4] INFO [2022-01-10 12:53:10,205] jepsen worker 84 - jepsen.util 84 :ok :write [28 26] INFO [2022-01-10 12:53:10,205] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:10,206] jepsen worker 84 - jepsen.util 84 :ok :read [28 26] INFO [2022-01-10 12:53:10,207] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:10,208] jepsen worker 84 - jepsen.util 84 :ok :read [28 26] INFO [2022-01-10 12:53:10,208] jepsen worker 84 - jepsen.util 84 :invoke :write [28 48] INFO [2022-01-10 12:53:10,209] jepsen worker 52 - jepsen.util 52 :ok :write [17 40] INFO [2022-01-10 12:53:10,210] jepsen worker 52 - jepsen.util 52 :invoke :write [17 36] INFO [2022-01-10 12:53:10,213] jepsen worker 1 - jepsen.util 1 :ok :write [0 47] INFO [2022-01-10 12:53:10,214] jepsen worker 1 - jepsen.util 1 :invoke :write [0 41] INFO [2022-01-10 12:53:10,215] jepsen worker 23 - jepsen.util 23 :ok :write [7 10] INFO [2022-01-10 12:53:10,215] jepsen worker 38 - jepsen.util 38 :ok :write [12 2] INFO [2022-01-10 12:53:10,216] jepsen worker 38 - jepsen.util 38 :invoke :write [12 28] INFO [2022-01-10 12:53:10,217] jepsen worker 23 - jepsen.util 23 :invoke :write [7 39] INFO [2022-01-10 12:53:10,224] jepsen worker 71 - jepsen.util 71 :ok :write [23 6] INFO [2022-01-10 12:53:10,224] jepsen worker 40 - jepsen.util 40 :ok :write [13 0] INFO [2022-01-10 12:53:10,225] jepsen worker 71 - jepsen.util 71 :invoke :write [23 30] INFO [2022-01-10 12:53:10,226] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,227] jepsen worker 40 - jepsen.util 40 :ok :read [13 0] INFO [2022-01-10 12:53:10,227] jepsen worker 7 - jepsen.util 7 :ok :write [2 2] INFO [2022-01-10 12:53:10,227] jepsen worker 40 - jepsen.util 40 :invoke :write [13 37] INFO [2022-01-10 12:53:10,228] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:10,228] jepsen worker 3 - jepsen.util 3 :ok :write [1 20] INFO [2022-01-10 12:53:10,229] jepsen worker 7 - jepsen.util 7 :ok :read [2 2] INFO [2022-01-10 12:53:10,229] jepsen worker 3 - jepsen.util 3 :invoke :write [1 0] INFO [2022-01-10 12:53:10,230] jepsen worker 7 - jepsen.util 7 :invoke :write [2 29] INFO [2022-01-10 12:53:10,233] jepsen worker 13 - jepsen.util 13 :ok :write [4 1] INFO [2022-01-10 12:53:10,233] jepsen worker 87 - jepsen.util 87 :ok :write [29 4] INFO [2022-01-10 12:53:10,233] jepsen worker 13 - jepsen.util 13 :invoke :write [4 40] INFO [2022-01-10 12:53:10,234] jepsen worker 87 - jepsen.util 87 :invoke :write [29 6] INFO [2022-01-10 12:53:10,235] jepsen worker 33 - jepsen.util 33 :ok :write [11 23] INFO [2022-01-10 12:53:10,235] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,236] jepsen worker 33 - jepsen.util 33 :ok :read [11 23] INFO [2022-01-10 12:53:10,236] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,237] jepsen worker 33 - jepsen.util 33 :ok :read [11 23] INFO [2022-01-10 12:53:10,238] jepsen worker 33 - jepsen.util 33 :invoke :write [11 3] INFO [2022-01-10 12:53:10,241] jepsen worker 78 - jepsen.util 78 :ok :write [26 4] INFO [2022-01-10 12:53:10,241] jepsen worker 1 - jepsen.util 1 :ok :write [0 41] INFO [2022-01-10 12:53:10,241] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,242] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:10,242] jepsen worker 78 - jepsen.util 78 :ok :read [26 4] INFO [2022-01-10 12:53:10,243] jepsen worker 1 - jepsen.util 1 :ok :read [0 41] INFO [2022-01-10 12:53:10,243] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,243] jepsen worker 1 - jepsen.util 1 :invoke :write [0 36] INFO [2022-01-10 12:53:10,243] jepsen worker 15 - jepsen.util 15 :ok :write [5 47] INFO [2022-01-10 12:53:10,244] jepsen worker 78 - jepsen.util 78 :ok :read [26 4] INFO [2022-01-10 12:53:10,244] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:10,244] jepsen worker 78 - jepsen.util 78 :invoke :write [26 35] INFO [2022-01-10 12:53:10,244] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:10,245] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:10,246] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:10,246] jepsen worker 15 - jepsen.util 15 :invoke :write [5 46] INFO [2022-01-10 12:53:10,248] jepsen worker 84 - jepsen.util 84 :ok :write [28 48] INFO [2022-01-10 12:53:10,248] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:10,249] jepsen worker 84 - jepsen.util 84 :ok :read [28 48] INFO [2022-01-10 12:53:10,249] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:10,250] jepsen worker 50 - jepsen.util 50 :ok :write [16 27] INFO [2022-01-10 12:53:10,250] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,250] jepsen worker 84 - jepsen.util 84 :ok :read [28 48] INFO [2022-01-10 12:53:10,251] jepsen worker 84 - jepsen.util 84 :invoke :write [28 15] INFO [2022-01-10 12:53:10,251] jepsen worker 50 - jepsen.util 50 :ok :read [16 27] INFO [2022-01-10 12:53:10,252] jepsen worker 50 - jepsen.util 50 :invoke :write [16 2] INFO [2022-01-10 12:53:10,252] jepsen worker 52 - jepsen.util 52 :ok :write [17 36] INFO [2022-01-10 12:53:10,253] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,254] jepsen worker 52 - jepsen.util 52 :ok :read [17 36] INFO [2022-01-10 12:53:10,254] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,255] jepsen worker 52 - jepsen.util 52 :ok :read [17 36] INFO [2022-01-10 12:53:10,255] jepsen worker 81 - jepsen.util 81 :ok :write [27 42] INFO [2022-01-10 12:53:10,256] jepsen worker 81 - jepsen.util 81 :invoke :write [27 18] INFO [2022-01-10 12:53:10,256] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,257] jepsen worker 52 - jepsen.util 52 :ok :read [17 36] INFO [2022-01-10 12:53:10,258] jepsen worker 52 - jepsen.util 52 :invoke :write [17 35] INFO [2022-01-10 12:53:10,261] jepsen worker 43 - jepsen.util 43 :ok :write [14 2] INFO [2022-01-10 12:53:10,261] jepsen worker 71 - jepsen.util 71 :ok :write [23 30] INFO [2022-01-10 12:53:10,261] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,262] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,262] jepsen worker 43 - jepsen.util 43 :ok :read [14 2] INFO [2022-01-10 12:53:10,263] jepsen worker 43 - jepsen.util 43 :invoke :write [14 13] INFO [2022-01-10 12:53:10,263] jepsen worker 71 - jepsen.util 71 :ok :read [23 30] INFO [2022-01-10 12:53:10,263] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,264] jepsen worker 71 - jepsen.util 71 :ok :read [23 30] INFO [2022-01-10 12:53:10,265] jepsen worker 71 - jepsen.util 71 :invoke :write [23 9] INFO [2022-01-10 12:53:10,267] jepsen worker 67 - jepsen.util 67 :ok :write [22 40] INFO [2022-01-10 12:53:10,267] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:10,268] jepsen worker 67 - jepsen.util 67 :ok :read [22 40] INFO [2022-01-10 12:53:10,269] jepsen worker 67 - jepsen.util 67 :invoke :write [22 38] INFO [2022-01-10 12:53:10,270] jepsen worker 13 - jepsen.util 13 :ok :write [4 40] INFO [2022-01-10 12:53:10,271] jepsen worker 13 - jepsen.util 13 :invoke :write [4 13] INFO [2022-01-10 12:53:10,274] jepsen worker 61 - jepsen.util 61 :ok :write [20 31] INFO [2022-01-10 12:53:10,275] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,275] jepsen worker 78 - jepsen.util 78 :ok :write [26 35] INFO [2022-01-10 12:53:10,275] jepsen worker 61 - jepsen.util 61 :ok :read [20 31] INFO [2022-01-10 12:53:10,276] jepsen worker 78 - jepsen.util 78 :invoke :write [26 10] INFO [2022-01-10 12:53:10,276] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,277] jepsen worker 61 - jepsen.util 61 :ok :read [20 31] INFO [2022-01-10 12:53:10,277] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,278] jepsen worker 61 - jepsen.util 61 :ok :read [20 31] INFO [2022-01-10 12:53:10,279] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,279] jepsen worker 61 - jepsen.util 61 :ok :read [20 31] INFO [2022-01-10 12:53:10,280] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,281] jepsen worker 61 - jepsen.util 61 :ok :read [20 31] INFO [2022-01-10 12:53:10,281] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,282] jepsen worker 76 - jepsen.util 76 :ok :write [25 41] INFO [2022-01-10 12:53:10,282] jepsen worker 76 - jepsen.util 76 :invoke :write [25 29] INFO [2022-01-10 12:53:10,282] jepsen worker 61 - jepsen.util 61 :ok :read [20 31] INFO [2022-01-10 12:53:10,283] jepsen worker 61 - jepsen.util 61 :invoke :write [20 46] INFO [2022-01-10 12:53:10,288] jepsen worker 52 - jepsen.util 52 :ok :write [17 35] INFO [2022-01-10 12:53:10,289] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,290] jepsen worker 52 - jepsen.util 52 :ok :read [17 35] INFO [2022-01-10 12:53:10,290] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,291] jepsen worker 52 - jepsen.util 52 :ok :read [17 35] INFO [2022-01-10 12:53:10,291] jepsen worker 52 - jepsen.util 52 :invoke :write [17 44] INFO [2022-01-10 12:53:10,291] jepsen worker 23 - jepsen.util 23 :ok :write [7 39] INFO [2022-01-10 12:53:10,292] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:10,293] jepsen worker 23 - jepsen.util 23 :ok :read [7 39] INFO [2022-01-10 12:53:10,293] jepsen worker 23 - jepsen.util 23 :invoke :write [7 49] INFO [2022-01-10 12:53:10,294] jepsen worker 71 - jepsen.util 71 :ok :write [23 9] INFO [2022-01-10 12:53:10,295] jepsen worker 71 - jepsen.util 71 :invoke :write [23 45] INFO [2022-01-10 12:53:10,298] jepsen worker 13 - jepsen.util 13 :ok :write [4 13] INFO [2022-01-10 12:53:10,299] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,300] jepsen worker 13 - jepsen.util 13 :ok :read [4 13] INFO [2022-01-10 12:53:10,300] jepsen worker 13 - jepsen.util 13 :invoke :write [4 20] INFO [2022-01-10 12:53:10,304] jepsen worker 78 - jepsen.util 78 :ok :write [26 10] INFO [2022-01-10 12:53:10,304] jepsen worker 78 - jepsen.util 78 :invoke :write [26 40] INFO [2022-01-10 12:53:10,308] jepsen worker 52 - jepsen.util 52 :ok :write [17 44] INFO [2022-01-10 12:53:10,308] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,309] jepsen worker 52 - jepsen.util 52 :ok :read [17 44] INFO [2022-01-10 12:53:10,310] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,310] jepsen worker 52 - jepsen.util 52 :ok :read [17 44] INFO [2022-01-10 12:53:10,311] jepsen worker 52 - jepsen.util 52 :invoke :write [17 31] INFO [2022-01-10 12:53:10,312] jepsen worker 71 - jepsen.util 71 :ok :write [23 45] INFO [2022-01-10 12:53:10,313] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,314] jepsen worker 71 - jepsen.util 71 :ok :read [23 45] INFO [2022-01-10 12:53:10,314] jepsen worker 71 - jepsen.util 71 :invoke :write [23 48] INFO [2022-01-10 12:53:10,316] jepsen worker 10 - jepsen.util 10 :info :write [3 25] :RequestTimeoutException INFO [2022-01-10 12:53:10,317] jepsen worker 13 - jepsen.util 13 :ok :write [4 20] INFO [2022-01-10 12:53:10,317] jepsen worker 10 - jepsen.util 100 :invoke :read [3 nil] INFO [2022-01-10 12:53:10,318] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,319] jepsen worker 13 - jepsen.util 13 :ok :read [4 20] INFO [2022-01-10 12:53:10,319] jepsen worker 13 - jepsen.util 13 :invoke :write [4 12] INFO [2022-01-10 12:53:10,321] jepsen worker 78 - jepsen.util 78 :ok :write [26 40] INFO [2022-01-10 12:53:10,322] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,323] jepsen worker 78 - jepsen.util 78 :ok :read [26 40] INFO [2022-01-10 12:53:10,323] jepsen worker 78 - jepsen.util 78 :invoke :write [26 17] INFO [2022-01-10 12:53:10,325] jepsen worker 38 - jepsen.util 38 :ok :write [12 28] INFO [2022-01-10 12:53:10,326] jepsen worker 38 - jepsen.util 38 :invoke :write [12 7] INFO [2022-01-10 12:53:10,327] jepsen worker 52 - jepsen.util 52 :ok :write [17 31] INFO [2022-01-10 12:53:10,327] jepsen worker 43 - jepsen.util 43 :ok :write [14 13] INFO [2022-01-10 12:53:10,327] jepsen worker 43 - jepsen.util 43 :invoke :write [14 47] INFO [2022-01-10 12:53:10,328] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,329] jepsen worker 52 - jepsen.util 52 :ok :read [17 31] INFO [2022-01-10 12:53:10,329] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,330] jepsen worker 52 - jepsen.util 52 :ok :read [17 31] INFO [2022-01-10 12:53:10,331] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,332] jepsen worker 52 - jepsen.util 52 :ok :read [17 31] INFO [2022-01-10 12:53:10,332] jepsen worker 3 - jepsen.util 3 :ok :write [1 0] INFO [2022-01-10 12:53:10,332] jepsen worker 40 - jepsen.util 40 :ok :write [13 37] INFO [2022-01-10 12:53:10,333] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,333] jepsen worker 40 - jepsen.util 40 :invoke :write [13 28] INFO [2022-01-10 12:53:10,334] jepsen worker 52 - jepsen.util 52 :ok :read [17 31] INFO [2022-01-10 12:53:10,334] jepsen worker 3 - jepsen.util 3 :invoke :write [1 29] INFO [2022-01-10 12:53:10,334] jepsen worker 71 - jepsen.util 71 :ok :write [23 48] INFO [2022-01-10 12:53:10,335] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,335] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,335] jepsen worker 52 - jepsen.util 52 :ok :read [17 31] INFO [2022-01-10 12:53:10,336] jepsen worker 52 - jepsen.util 52 :invoke :write [17 31] INFO [2022-01-10 12:53:10,336] jepsen worker 71 - jepsen.util 71 :ok :read [23 48] INFO [2022-01-10 12:53:10,337] jepsen worker 71 - jepsen.util 71 :invoke :write [23 13] INFO [2022-01-10 12:53:10,338] jepsen worker 84 - jepsen.util 84 :ok :write [28 15] INFO [2022-01-10 12:53:10,338] jepsen worker 84 - jepsen.util 84 :invoke :write [28 13] INFO [2022-01-10 12:53:10,339] jepsen worker 7 - jepsen.util 7 :ok :write [2 29] INFO [2022-01-10 12:53:10,339] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:10,339] jepsen worker 13 - jepsen.util 13 :ok :write [4 12] INFO [2022-01-10 12:53:10,340] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,340] jepsen worker 7 - jepsen.util 7 :ok :read [2 29] INFO [2022-01-10 12:53:10,340] jepsen worker 7 - jepsen.util 7 :invoke :write [2 15] INFO [2022-01-10 12:53:10,341] jepsen worker 13 - jepsen.util 13 :ok :read [4 12] INFO [2022-01-10 12:53:10,341] jepsen worker 61 - jepsen.util 61 :ok :write [20 46] INFO [2022-01-10 12:53:10,341] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,342] jepsen worker 61 - jepsen.util 61 :invoke :write [20 13] INFO [2022-01-10 12:53:10,342] jepsen worker 13 - jepsen.util 13 :ok :read [4 12] INFO [2022-01-10 12:53:10,343] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,344] jepsen worker 13 - jepsen.util 13 :ok :read [4 12] INFO [2022-01-10 12:53:10,344] jepsen worker 13 - jepsen.util 13 :invoke :write [4 42] INFO [2022-01-10 12:53:10,345] jepsen worker 33 - jepsen.util 33 :ok :write [11 3] INFO [2022-01-10 12:53:10,345] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,346] jepsen worker 33 - jepsen.util 33 :ok :read [11 3] INFO [2022-01-10 12:53:10,347] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,348] jepsen worker 33 - jepsen.util 33 :ok :read [11 3] INFO [2022-01-10 12:53:10,348] jepsen worker 76 - jepsen.util 76 :ok :write [25 29] INFO [2022-01-10 12:53:10,349] jepsen worker 78 - jepsen.util 78 :ok :write [26 17] INFO [2022-01-10 12:53:10,349] jepsen worker 38 - jepsen.util 38 :ok :write [12 7] INFO [2022-01-10 12:53:10,350] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,350] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,351] jepsen worker 33 - jepsen.util 33 :ok :read [11 3] INFO [2022-01-10 12:53:10,351] jepsen worker 76 - jepsen.util 76 :invoke :write [25 14] INFO [2022-01-10 12:53:10,351] jepsen worker 87 - jepsen.util 87 :ok :write [29 6] INFO [2022-01-10 12:53:10,351] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:10,351] jepsen worker 78 - jepsen.util 78 :ok :read [26 17] INFO [2022-01-10 12:53:10,352] jepsen worker 38 - jepsen.util 38 :ok :read [12 7] INFO [2022-01-10 12:53:10,352] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,352] jepsen worker 87 - jepsen.util 87 :invoke :write [29 33] INFO [2022-01-10 12:53:10,353] jepsen worker 33 - jepsen.util 33 :ok :read [11 3] INFO [2022-01-10 12:53:10,353] jepsen worker 78 - jepsen.util 78 :invoke :write [26 2] INFO [2022-01-10 12:53:10,353] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,354] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:10,354] jepsen worker 33 - jepsen.util 33 :ok :read [11 3] INFO [2022-01-10 12:53:10,354] jepsen worker 38 - jepsen.util 38 :ok :read [12 7] INFO [2022-01-10 12:53:10,357] jepsen worker 17 - jepsen.util 17 :info :write [5 36] :RequestTimeoutException INFO [2022-01-10 12:53:10,357] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:10,357] jepsen worker 23 - jepsen.util 23 :ok :write [7 49] INFO [2022-01-10 12:53:10,358] jepsen worker 38 - jepsen.util 38 :ok :read [12 7] INFO [2022-01-10 12:53:10,358] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,358] jepsen worker 52 - jepsen.util 52 :ok :write [17 31] INFO [2022-01-10 12:53:10,359] jepsen worker 1 - jepsen.util 1 :ok :write [0 36] INFO [2022-01-10 12:53:10,359] jepsen worker 38 - jepsen.util 38 :invoke :write [12 8] INFO [2022-01-10 12:53:10,359] jepsen worker 33 - jepsen.util 33 :ok :read [11 3] INFO [2022-01-10 12:53:10,360] jepsen worker 17 - jepsen.util 107 :invoke :read [5 nil] INFO [2022-01-10 12:53:10,361] jepsen worker 33 - jepsen.util 33 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,361] jepsen worker 23 - jepsen.util 23 :invoke :write [7 33] INFO [2022-01-10 12:53:10,361] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,362] jepsen worker 33 - jepsen.util 33 :ok :read [11 3] INFO [2022-01-10 12:53:10,362] jepsen worker 1 - jepsen.util 1 :invoke :write [0 5] INFO [2022-01-10 12:53:10,362] jepsen worker 17 - jepsen.util 107 :ok :read [5 47] INFO [2022-01-10 12:53:10,362] jepsen worker 17 - jepsen.util 107 :invoke :write [5 18] INFO [2022-01-10 12:53:10,362] jepsen worker 52 - jepsen.util 52 :ok :read [17 31] INFO [2022-01-10 12:53:10,363] jepsen worker 33 - jepsen.util 33 :invoke :write [11 36] INFO [2022-01-10 12:53:10,363] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,363] jepsen worker 43 - jepsen.util 43 :ok :write [14 47] INFO [2022-01-10 12:53:10,363] jepsen worker 3 - jepsen.util 3 :ok :write [1 29] INFO [2022-01-10 12:53:10,364] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:10,364] jepsen worker 52 - jepsen.util 52 :ok :read [17 31] INFO [2022-01-10 12:53:10,364] jepsen worker 43 - jepsen.util 43 :invoke :write [14 44] INFO [2022-01-10 12:53:10,365] jepsen worker 52 - jepsen.util 52 :invoke :write [17 33] INFO [2022-01-10 12:53:10,365] jepsen worker 3 - jepsen.util 3 :ok :read [1 29] INFO [2022-01-10 12:53:10,366] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:10,367] jepsen worker 3 - jepsen.util 3 :ok :read [1 29] INFO [2022-01-10 12:53:10,367] jepsen worker 15 - jepsen.util 15 :ok :write [5 46] INFO [2022-01-10 12:53:10,367] jepsen worker 3 - jepsen.util 3 :invoke :write [1 24] INFO [2022-01-10 12:53:10,368] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:10,369] jepsen worker 15 - jepsen.util 15 :ok :read [5 46] INFO [2022-01-10 12:53:10,369] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:10,369] jepsen worker 71 - jepsen.util 71 :ok :write [23 13] INFO [2022-01-10 12:53:10,370] jepsen worker 71 - jepsen.util 71 :invoke :write [23 13] INFO [2022-01-10 12:53:10,370] jepsen worker 15 - jepsen.util 15 :ok :read [5 46] INFO [2022-01-10 12:53:10,371] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:10,372] jepsen worker 15 - jepsen.util 15 :ok :read [5 46] INFO [2022-01-10 12:53:10,372] jepsen worker 15 - jepsen.util 15 :invoke :write [5 30] INFO [2022-01-10 12:53:10,376] jepsen worker 84 - jepsen.util 84 :ok :write [28 13] INFO [2022-01-10 12:53:10,376] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:10,377] jepsen worker 84 - jepsen.util 84 :ok :read [28 13] INFO [2022-01-10 12:53:10,378] jepsen worker 50 - jepsen.util 50 :ok :write [16 2] INFO [2022-01-10 12:53:10,378] jepsen worker 84 - jepsen.util 84 :invoke :write [28 37] INFO [2022-01-10 12:53:10,378] jepsen worker 50 - jepsen.util 50 :invoke :write [16 7] INFO [2022-01-10 12:53:10,381] jepsen worker 13 - jepsen.util 13 :ok :write [4 42] INFO [2022-01-10 12:53:10,381] jepsen worker 61 - jepsen.util 61 :ok :write [20 13] INFO [2022-01-10 12:53:10,382] jepsen worker 13 - jepsen.util 13 :invoke :write [4 5] INFO [2022-01-10 12:53:10,382] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,383] jepsen worker 61 - jepsen.util 61 :ok :read [20 13] INFO [2022-01-10 12:53:10,384] jepsen worker 61 - jepsen.util 61 :invoke :write [20 30] INFO [2022-01-10 12:53:10,386] jepsen worker 78 - jepsen.util 78 :ok :write [26 2] INFO [2022-01-10 12:53:10,387] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,388] jepsen worker 78 - jepsen.util 78 :ok :read [26 2] INFO [2022-01-10 12:53:10,388] jepsen worker 81 - jepsen.util 81 :ok :write [27 18] INFO [2022-01-10 12:53:10,390] jepsen worker 81 - jepsen.util 81 :invoke :write [27 28] INFO [2022-01-10 12:53:10,391] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,392] jepsen worker 78 - jepsen.util 78 :ok :read [26 2] INFO [2022-01-10 12:53:10,393] jepsen worker 38 - jepsen.util 38 :ok :write [12 8] INFO [2022-01-10 12:53:10,394] jepsen worker 78 - jepsen.util 78 :invoke :write [26 1] INFO [2022-01-10 12:53:10,394] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:10,395] jepsen worker 76 - jepsen.util 76 :ok :write [25 14] INFO [2022-01-10 12:53:10,396] jepsen worker 76 - jepsen.util 76 :invoke :write [25 30] INFO [2022-01-10 12:53:10,398] jepsen worker 67 - jepsen.util 67 :ok :write [22 38] INFO [2022-01-10 12:53:10,399] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:10,400] jepsen worker 67 - jepsen.util 67 :ok :read [22 38] INFO [2022-01-10 12:53:10,400] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:10,401] jepsen worker 67 - jepsen.util 67 :ok :read [22 38] INFO [2022-01-10 12:53:10,402] jepsen worker 67 - jepsen.util 67 :invoke :write [22 27] INFO [2022-01-10 12:53:10,404] jepsen worker 23 - jepsen.util 23 :ok :write [7 33] INFO [2022-01-10 12:53:10,404] jepsen worker 23 - jepsen.util 23 :invoke :write [7 19] INFO [2022-01-10 12:53:10,406] jepsen worker 40 - jepsen.util 40 :ok :write [13 28] INFO [2022-01-10 12:53:10,407] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,408] jepsen worker 40 - jepsen.util 40 :ok :read [13 28] INFO [2022-01-10 12:53:10,409] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,409] jepsen worker 40 - jepsen.util 40 :ok :read [13 28] INFO [2022-01-10 12:53:10,410] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,410] jepsen worker 43 - jepsen.util 43 :ok :write [14 44] INFO [2022-01-10 12:53:10,411] jepsen worker 40 - jepsen.util 40 :ok :read [13 28] INFO [2022-01-10 12:53:10,411] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,412] jepsen worker 40 - jepsen.util 40 :invoke :write [13 33] INFO [2022-01-10 12:53:10,412] jepsen worker 43 - jepsen.util 43 :ok :read [14 44] INFO [2022-01-10 12:53:10,412] jepsen worker 7 - jepsen.util 7 :ok :write [2 15] INFO [2022-01-10 12:53:10,413] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,413] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:10,414] jepsen worker 43 - jepsen.util 43 :ok :read [14 44] INFO [2022-01-10 12:53:10,414] jepsen worker 43 - jepsen.util 43 :invoke :write [14 35] INFO [2022-01-10 12:53:10,414] jepsen worker 7 - jepsen.util 7 :ok :read [2 15] INFO [2022-01-10 12:53:10,415] jepsen worker 7 - jepsen.util 7 :invoke :write [2 16] INFO [2022-01-10 12:53:10,417] jepsen worker 34 - jepsen.util 34 :ok :write [11 39] INFO [2022-01-10 12:53:10,417] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,418] jepsen worker 34 - jepsen.util 34 :ok :read [11 39] INFO [2022-01-10 12:53:10,419] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,420] jepsen worker 34 - jepsen.util 34 :ok :read [11 39] INFO [2022-01-10 12:53:10,420] jepsen worker 34 - jepsen.util 34 :invoke :write [11 47] INFO [2022-01-10 12:53:10,421] jepsen worker 87 - jepsen.util 87 :ok :write [29 33] INFO [2022-01-10 12:53:10,422] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,422] jepsen worker 87 - jepsen.util 87 :ok :read [29 33] INFO [2022-01-10 12:53:10,423] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,423] jepsen worker 87 - jepsen.util 87 :ok :read [29 33] INFO [2022-01-10 12:53:10,424] jepsen worker 87 - jepsen.util 87 :invoke :write [29 26] INFO [2022-01-10 12:53:10,425] jepsen worker 1 - jepsen.util 1 :ok :write [0 5] INFO [2022-01-10 12:53:10,426] jepsen worker 1 - jepsen.util 1 :invoke :write [0 29] INFO [2022-01-10 12:53:10,431] jepsen worker 15 - jepsen.util 15 :ok :write [5 30] INFO [2022-01-10 12:53:10,432] jepsen worker 15 - jepsen.util 15 :invoke :write [5 15] INFO [2022-01-10 12:53:10,533] jepsen worker 52 - jepsen.util 52 :ok :write [17 33] INFO [2022-01-10 12:53:10,534] jepsen worker 52 - jepsen.util 52 :invoke :write [17 21] INFO [2022-01-10 12:53:10,539] jepsen worker 3 - jepsen.util 3 :ok :write [1 24] INFO [2022-01-10 12:53:10,539] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:10,540] jepsen worker 3 - jepsen.util 3 :ok :read [1 24] INFO [2022-01-10 12:53:10,540] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:10,541] jepsen worker 3 - jepsen.util 3 :ok :read [1 24] INFO [2022-01-10 12:53:10,542] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:10,543] jepsen worker 3 - jepsen.util 3 :ok :read [1 24] INFO [2022-01-10 12:53:10,543] jepsen worker 3 - jepsen.util 3 :invoke :write [1 22] INFO [2022-01-10 12:53:10,544] jepsen worker 71 - jepsen.util 71 :ok :write [23 13] INFO [2022-01-10 12:53:10,545] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,545] jepsen worker 71 - jepsen.util 71 :ok :read [23 13] INFO [2022-01-10 12:53:10,546] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,547] jepsen worker 71 - jepsen.util 71 :ok :read [23 13] INFO [2022-01-10 12:53:10,547] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,548] jepsen worker 71 - jepsen.util 71 :ok :read [23 13] INFO [2022-01-10 12:53:10,549] jepsen worker 71 - jepsen.util 71 :invoke :write [23 42] INFO [2022-01-10 12:53:10,551] jepsen worker 84 - jepsen.util 84 :ok :write [28 37] INFO [2022-01-10 12:53:10,551] jepsen worker 40 - jepsen.util 40 :ok :write [13 33] INFO [2022-01-10 12:53:10,551] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:10,552] jepsen worker 40 - jepsen.util 40 :invoke :write [13 43] INFO [2022-01-10 12:53:10,552] jepsen worker 84 - jepsen.util 84 :ok :read [28 37] INFO [2022-01-10 12:53:10,553] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:10,554] jepsen worker 84 - jepsen.util 84 :ok :read [28 37] INFO [2022-01-10 12:53:10,554] jepsen worker 84 - jepsen.util 84 :invoke :write [28 28] INFO [2022-01-10 12:53:10,556] jepsen worker 50 - jepsen.util 50 :ok :write [16 7] INFO [2022-01-10 12:53:10,557] jepsen worker 13 - jepsen.util 13 :ok :write [4 5] INFO [2022-01-10 12:53:10,557] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,557] jepsen worker 13 - jepsen.util 13 :invoke :write [4 49] INFO [2022-01-10 12:53:10,558] jepsen worker 50 - jepsen.util 50 :ok :read [16 7] INFO [2022-01-10 12:53:10,558] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,559] jepsen worker 50 - jepsen.util 50 :ok :read [16 7] INFO [2022-01-10 12:53:10,559] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,560] jepsen worker 50 - jepsen.util 50 :ok :read [16 7] INFO [2022-01-10 12:53:10,561] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,562] jepsen worker 50 - jepsen.util 50 :ok :read [16 7] INFO [2022-01-10 12:53:10,562] jepsen worker 61 - jepsen.util 61 :ok :write [20 30] INFO [2022-01-10 12:53:10,562] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,563] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,563] jepsen worker 50 - jepsen.util 50 :ok :read [16 7] INFO [2022-01-10 12:53:10,564] jepsen worker 78 - jepsen.util 78 :ok :write [26 1] INFO [2022-01-10 12:53:10,564] jepsen worker 61 - jepsen.util 61 :ok :read [20 30] INFO [2022-01-10 12:53:10,564] jepsen worker 87 - jepsen.util 87 :ok :write [29 26] INFO [2022-01-10 12:53:10,564] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,565] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,565] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,565] jepsen worker 50 - jepsen.util 50 :ok :read [16 7] INFO [2022-01-10 12:53:10,565] jepsen worker 78 - jepsen.util 78 :ok :read [26 1] INFO [2022-01-10 12:53:10,566] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,566] jepsen worker 87 - jepsen.util 87 :ok :read [29 26] INFO [2022-01-10 12:53:10,566] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,567] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,567] jepsen worker 50 - jepsen.util 50 :ok :read [16 7] INFO [2022-01-10 12:53:10,567] jepsen worker 61 - jepsen.util 61 :ok :read [20 30] INFO [2022-01-10 12:53:10,567] jepsen worker 87 - jepsen.util 87 :invoke :write [29 20] INFO [2022-01-10 12:53:10,568] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,568] jepsen worker 78 - jepsen.util 78 :ok :read [26 1] INFO [2022-01-10 12:53:10,568] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,569] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,569] jepsen worker 81 - jepsen.util 81 :ok :write [27 28] INFO [2022-01-10 12:53:10,569] jepsen worker 50 - jepsen.util 50 :ok :read [16 7] INFO [2022-01-10 12:53:10,569] jepsen worker 61 - jepsen.util 61 :ok :read [20 30] INFO [2022-01-10 12:53:10,570] jepsen worker 78 - jepsen.util 78 :ok :read [26 1] INFO [2022-01-10 12:53:10,570] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:10,571] jepsen worker 81 - jepsen.util 81 :ok :read [27 28] INFO [2022-01-10 12:53:10,571] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:10,571] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,572] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:10,572] jepsen worker 50 - jepsen.util 50 :ok :read [16 7] INFO [2022-01-10 12:53:10,572] jepsen worker 1 - jepsen.util 1 :ok :write [0 29] INFO [2022-01-10 12:53:10,573] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,573] jepsen worker 81 - jepsen.util 81 :ok :read [27 28] INFO [2022-01-10 12:53:10,574] jepsen worker 61 - jepsen.util 61 :ok :read [20 30] INFO [2022-01-10 12:53:10,574] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:10,575] jepsen worker 81 - jepsen.util 81 :ok :read [27 28] INFO [2022-01-10 12:53:10,575] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:10,576] jepsen worker 76 - jepsen.util 76 :ok :write [25 30] INFO [2022-01-10 12:53:10,576] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,576] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:10,576] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:10,577] jepsen worker 61 - jepsen.util 61 :ok :read [20 30] INFO [2022-01-10 12:53:10,577] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,578] jepsen worker 81 - jepsen.util 81 :ok :read [27 28] INFO [2022-01-10 12:53:10,578] jepsen worker 40 - jepsen.util 40 :ok :write [13 43] INFO [2022-01-10 12:53:10,578] jepsen worker 78 - jepsen.util 78 :ok :read [26 1] INFO [2022-01-10 12:53:10,579] jepsen worker 52 - jepsen.util 52 :ok :write [17 21] INFO [2022-01-10 12:53:10,579] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:10,580] jepsen worker 1 - jepsen.util 1 :ok :read [0 29] INFO [2022-01-10 12:53:10,580] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,581] jepsen worker 76 - jepsen.util 76 :ok :read [25 30] INFO [2022-01-10 12:53:10,581] jepsen worker 52 - jepsen.util 52 :invoke :write [17 25] INFO [2022-01-10 12:53:10,582] jepsen worker 1 - jepsen.util 1 :invoke :write [0 17] INFO [2022-01-10 12:53:10,584] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,584] jepsen worker 67 - jepsen.util 67 :ok :write [22 27] INFO [2022-01-10 12:53:10,585] jepsen worker 40 - jepsen.util 40 :invoke :write [13 33] INFO [2022-01-10 12:53:10,585] jepsen worker 61 - jepsen.util 61 :ok :read [20 30] INFO [2022-01-10 12:53:10,586] jepsen worker 76 - jepsen.util 76 :invoke :write [25 0] INFO [2022-01-10 12:53:10,586] jepsen worker 3 - jepsen.util 3 :ok :write [1 22] INFO [2022-01-10 12:53:10,587] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:10,587] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:10,588] jepsen worker 67 - jepsen.util 67 :ok :read [22 27] INFO [2022-01-10 12:53:10,588] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:10,588] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:10,588] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,589] jepsen worker 3 - jepsen.util 3 :ok :read [1 22] INFO [2022-01-10 12:53:10,589] jepsen worker 23 - jepsen.util 23 :ok :write [7 19] INFO [2022-01-10 12:53:10,589] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:10,589] jepsen worker 50 - jepsen.util 50 :ok :read [16 7] INFO [2022-01-10 12:53:10,590] jepsen worker 3 - jepsen.util 3 :invoke :write [1 46] INFO [2022-01-10 12:53:10,590] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:10,591] jepsen worker 23 - jepsen.util 23 :invoke :write [7 4] INFO [2022-01-10 12:53:10,591] jepsen worker 81 - jepsen.util 81 :invoke :write [27 40] INFO [2022-01-10 12:53:10,592] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:10,593] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:10,593] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,593] jepsen worker 67 - jepsen.util 67 :ok :read [22 27] INFO [2022-01-10 12:53:10,594] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:10,594] jepsen worker 61 - jepsen.util 61 :invoke :write [20 14] INFO [2022-01-10 12:53:10,594] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:10,595] jepsen worker 78 - jepsen.util 78 :ok :read [26 1] INFO [2022-01-10 12:53:10,595] jepsen worker 38 - jepsen.util 38 :invoke :write [12 16] INFO [2022-01-10 12:53:10,595] jepsen worker 78 - jepsen.util 78 :invoke :write [26 46] INFO [2022-01-10 12:53:10,595] jepsen worker 67 - jepsen.util 67 :ok :read [22 27] INFO [2022-01-10 12:53:10,596] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,596] jepsen worker 71 - jepsen.util 71 :ok :write [23 42] INFO [2022-01-10 12:53:10,596] jepsen worker 67 - jepsen.util 67 :invoke :write [22 21] INFO [2022-01-10 12:53:10,596] jepsen worker 87 - jepsen.util 87 :ok :write [29 20] INFO [2022-01-10 12:53:10,597] jepsen worker 71 - jepsen.util 71 :invoke :write [23 18] INFO [2022-01-10 12:53:10,597] jepsen worker 50 - jepsen.util 50 :ok :read [16 7] INFO [2022-01-10 12:53:10,597] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,598] jepsen worker 50 - jepsen.util 50 :invoke :write [16 17] INFO [2022-01-10 12:53:10,598] jepsen worker 87 - jepsen.util 87 :ok :read [29 20] INFO [2022-01-10 12:53:10,599] jepsen worker 87 - jepsen.util 87 :invoke :write [29 4] INFO [2022-01-10 12:53:10,600] jepsen worker 43 - jepsen.util 43 :ok :write [14 35] INFO [2022-01-10 12:53:10,601] jepsen worker 43 - jepsen.util 43 :invoke :write [14 33] INFO [2022-01-10 12:53:10,605] jepsen worker 7 - jepsen.util 7 :ok :write [2 16] INFO [2022-01-10 12:53:10,605] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:10,606] jepsen worker 84 - jepsen.util 84 :ok :write [28 28] INFO [2022-01-10 12:53:10,607] jepsen worker 7 - jepsen.util 7 :ok :read [2 16] INFO [2022-01-10 12:53:10,607] jepsen worker 84 - jepsen.util 84 :invoke :write [28 35] INFO [2022-01-10 12:53:10,607] jepsen worker 7 - jepsen.util 7 :invoke :write [2 45] INFO [2022-01-10 12:53:10,610] jepsen worker 40 - jepsen.util 40 :ok :write [13 33] INFO [2022-01-10 12:53:10,611] jepsen worker 13 - jepsen.util 13 :ok :write [4 49] INFO [2022-01-10 12:53:10,611] jepsen worker 40 - jepsen.util 40 :invoke :write [13 36] INFO [2022-01-10 12:53:10,611] jepsen worker 13 - jepsen.util 13 :invoke :write [4 48] INFO [2022-01-10 12:53:10,613] jepsen worker 34 - jepsen.util 34 :ok :write [11 47] INFO [2022-01-10 12:53:10,613] jepsen worker 34 - jepsen.util 34 :invoke :write [11 25] INFO [2022-01-10 12:53:10,618] jepsen worker 1 - jepsen.util 1 :ok :write [0 17] INFO [2022-01-10 12:53:10,619] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:10,619] jepsen worker 78 - jepsen.util 78 :ok :write [26 46] INFO [2022-01-10 12:53:10,619] jepsen worker 78 - jepsen.util 78 :invoke :write [26 17] INFO [2022-01-10 12:53:10,620] jepsen worker 1 - jepsen.util 1 :ok :read [0 17] INFO [2022-01-10 12:53:10,620] jepsen worker 1 - jepsen.util 1 :invoke :write [0 28] INFO [2022-01-10 12:53:10,621] jepsen worker 15 - jepsen.util 15 :ok :write [5 15] INFO [2022-01-10 12:53:10,621] jepsen worker 15 - jepsen.util 15 :invoke :write [5 23] INFO [2022-01-10 12:53:10,624] jepsen worker 42 - jepsen.util 42 :info :write [14 49] :RequestTimeoutException INFO [2022-01-10 12:53:10,625] jepsen worker 42 - jepsen.util 132 :invoke :write [14 48] INFO [2022-01-10 12:53:10,626] jepsen worker 38 - jepsen.util 38 :ok :write [12 16] INFO [2022-01-10 12:53:10,626] jepsen worker 38 - jepsen.util 38 :invoke :write [12 22] INFO [2022-01-10 12:53:10,630] jepsen worker 87 - jepsen.util 87 :ok :write [29 4] INFO [2022-01-10 12:53:10,631] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,632] jepsen worker 87 - jepsen.util 87 :ok :read [29 4] INFO [2022-01-10 12:53:10,633] jepsen worker 52 - jepsen.util 52 :ok :write [17 25] INFO [2022-01-10 12:53:10,633] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,634] jepsen worker 87 - jepsen.util 87 :ok :read [29 4] INFO [2022-01-10 12:53:10,634] jepsen worker 52 - jepsen.util 52 :invoke :write [17 19] INFO [2022-01-10 12:53:10,635] jepsen worker 50 - jepsen.util 50 :ok :write [16 17] INFO [2022-01-10 12:53:10,635] jepsen worker 87 - jepsen.util 87 :invoke :write [29 12] INFO [2022-01-10 12:53:10,635] jepsen worker 61 - jepsen.util 61 :ok :write [20 14] INFO [2022-01-10 12:53:10,636] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,637] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,637] jepsen worker 50 - jepsen.util 50 :ok :read [16 17] INFO [2022-01-10 12:53:10,638] jepsen worker 61 - jepsen.util 61 :ok :read [20 14] INFO [2022-01-10 12:53:10,638] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,639] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,639] jepsen worker 50 - jepsen.util 50 :ok :read [16 17] INFO [2022-01-10 12:53:10,639] jepsen worker 61 - jepsen.util 61 :ok :read [20 14] INFO [2022-01-10 12:53:10,640] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,640] jepsen worker 61 - jepsen.util 61 :invoke :write [20 34] INFO [2022-01-10 12:53:10,641] jepsen worker 3 - jepsen.util 3 :ok :write [1 46] INFO [2022-01-10 12:53:10,642] jepsen worker 3 - jepsen.util 3 :invoke :write [1 48] INFO [2022-01-10 12:53:10,642] jepsen worker 50 - jepsen.util 50 :ok :read [16 17] INFO [2022-01-10 12:53:10,644] jepsen worker 83 - jepsen.util 83 :info :write [27 44] :RequestTimeoutException INFO [2022-01-10 12:53:10,644] jepsen worker 50 - jepsen.util 50 :invoke :write [16 27] INFO [2022-01-10 12:53:10,645] jepsen worker 81 - jepsen.util 81 :ok :write [27 40] INFO [2022-01-10 12:53:10,646] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:10,647] jepsen worker 81 - jepsen.util 81 :ok :read [27 40] INFO [2022-01-10 12:53:10,648] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:10,648] jepsen worker 71 - jepsen.util 71 :ok :write [23 18] INFO [2022-01-10 12:53:10,648] jepsen worker 40 - jepsen.util 40 :ok :write [13 36] INFO [2022-01-10 12:53:10,649] jepsen worker 81 - jepsen.util 81 :ok :read [27 40] INFO [2022-01-10 12:53:10,650] jepsen worker 71 - jepsen.util 71 :invoke :write [23 7] INFO [2022-01-10 12:53:10,650] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:10,651] jepsen worker 81 - jepsen.util 81 :ok :read [27 40] INFO [2022-01-10 12:53:10,651] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,651] jepsen worker 76 - jepsen.util 76 :ok :write [25 0] INFO [2022-01-10 12:53:10,652] jepsen worker 83 - jepsen.util 173 :invoke :write [27 41] INFO [2022-01-10 12:53:10,652] jepsen worker 81 - jepsen.util 81 :invoke :write [27 0] INFO [2022-01-10 12:53:10,652] jepsen worker 40 - jepsen.util 40 :ok :read [13 36] INFO [2022-01-10 12:53:10,653] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,653] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,654] jepsen worker 76 - jepsen.util 76 :ok :read [25 0] INFO [2022-01-10 12:53:10,654] jepsen worker 40 - jepsen.util 40 :ok :read [13 36] INFO [2022-01-10 12:53:10,655] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,655] jepsen worker 84 - jepsen.util 84 :ok :write [28 35] INFO [2022-01-10 12:53:10,655] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,656] jepsen worker 76 - jepsen.util 76 :ok :read [25 0] INFO [2022-01-10 12:53:10,656] jepsen worker 84 - jepsen.util 84 :invoke :write [28 41] INFO [2022-01-10 12:53:10,656] jepsen worker 40 - jepsen.util 40 :ok :read [13 36] INFO [2022-01-10 12:53:10,656] jepsen worker 76 - jepsen.util 76 :invoke :write [25 40] INFO [2022-01-10 12:53:10,657] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,658] jepsen worker 40 - jepsen.util 40 :ok :read [13 36] INFO [2022-01-10 12:53:10,658] jepsen worker 40 - jepsen.util 40 :invoke :write [13 23] INFO [2022-01-10 12:53:10,660] jepsen worker 13 - jepsen.util 13 :ok :write [4 48] INFO [2022-01-10 12:53:10,661] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,662] jepsen worker 67 - jepsen.util 67 :ok :write [22 21] INFO [2022-01-10 12:53:10,662] jepsen worker 13 - jepsen.util 13 :ok :read [4 48] INFO [2022-01-10 12:53:10,663] jepsen worker 67 - jepsen.util 67 :invoke :write [22 42] INFO [2022-01-10 12:53:10,663] jepsen worker 13 - jepsen.util 13 :invoke :write [4 0] INFO [2022-01-10 12:53:10,667] jepsen worker 1 - jepsen.util 1 :ok :write [0 28] INFO [2022-01-10 12:53:10,668] jepsen worker 1 - jepsen.util 1 :invoke :write [0 7] INFO [2022-01-10 12:53:10,669] jepsen worker 78 - jepsen.util 78 :ok :write [26 17] INFO [2022-01-10 12:53:10,669] jepsen worker 23 - jepsen.util 23 :ok :write [7 4] INFO [2022-01-10 12:53:10,670] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:10,671] jepsen worker 78 - jepsen.util 78 :invoke :write [26 39] INFO [2022-01-10 12:53:10,671] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:10,672] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:10,673] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:10,673] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:10,674] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:10,675] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:10,676] jepsen worker 43 - jepsen.util 43 :ok :write [14 33] INFO [2022-01-10 12:53:10,676] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:10,677] jepsen worker 43 - jepsen.util 43 :invoke :write [14 7] INFO [2022-01-10 12:53:10,678] jepsen worker 38 - jepsen.util 38 :ok :write [12 22] INFO [2022-01-10 12:53:10,678] jepsen worker 23 - jepsen.util 23 :invoke :write [7 5] INFO [2022-01-10 12:53:10,679] jepsen worker 87 - jepsen.util 87 :ok :write [29 12] INFO [2022-01-10 12:53:10,679] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:10,679] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,680] jepsen worker 38 - jepsen.util 38 :ok :read [12 22] INFO [2022-01-10 12:53:10,680] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:10,681] jepsen worker 38 - jepsen.util 38 :invoke :write [12 41] INFO [2022-01-10 12:53:10,681] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,682] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:10,683] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,684] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:10,684] jepsen worker 7 - jepsen.util 7 :ok :write [2 45] INFO [2022-01-10 12:53:10,684] jepsen worker 87 - jepsen.util 87 :invoke :write [29 31] INFO [2022-01-10 12:53:10,685] jepsen worker 7 - jepsen.util 7 :invoke :write [2 28] INFO [2022-01-10 12:53:10,686] jepsen worker 52 - jepsen.util 52 :ok :write [17 19] INFO [2022-01-10 12:53:10,687] jepsen worker 52 - jepsen.util 52 :invoke :write [17 41] INFO [2022-01-10 12:53:10,691] jepsen worker 34 - jepsen.util 34 :ok :write [11 25] INFO [2022-01-10 12:53:10,692] jepsen worker 3 - jepsen.util 3 :ok :write [1 48] INFO [2022-01-10 12:53:10,692] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,693] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:10,693] jepsen worker 34 - jepsen.util 34 :ok :read [11 25] INFO [2022-01-10 12:53:10,694] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:10,694] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,694] jepsen worker 3 - jepsen.util 3 :invoke :write [1 8] INFO [2022-01-10 12:53:10,695] jepsen worker 34 - jepsen.util 34 :ok :read [11 25] INFO [2022-01-10 12:53:10,696] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,697] jepsen worker 34 - jepsen.util 34 :ok :read [11 25] INFO [2022-01-10 12:53:10,698] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,698] jepsen worker 71 - jepsen.util 71 :ok :write [23 7] INFO [2022-01-10 12:53:10,698] jepsen worker 34 - jepsen.util 34 :ok :read [11 25] INFO [2022-01-10 12:53:10,699] jepsen worker 15 - jepsen.util 15 :ok :write [5 23] INFO [2022-01-10 12:53:10,699] jepsen worker 15 - jepsen.util 15 :invoke :write [5 15] INFO [2022-01-10 12:53:10,700] jepsen worker 34 - jepsen.util 34 :invoke :write [11 44] INFO [2022-01-10 12:53:10,701] jepsen worker 71 - jepsen.util 71 :invoke :write [23 11] INFO [2022-01-10 12:53:10,704] jepsen worker 61 - jepsen.util 61 :ok :write [20 34] INFO [2022-01-10 12:53:10,705] jepsen worker 84 - jepsen.util 84 :ok :write [28 41] INFO [2022-01-10 12:53:10,705] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,706] jepsen worker 61 - jepsen.util 61 :ok :read [20 34] INFO [2022-01-10 12:53:10,706] jepsen worker 84 - jepsen.util 84 :invoke :write [28 5] INFO [2022-01-10 12:53:10,707] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,708] jepsen worker 61 - jepsen.util 61 :ok :read [20 34] INFO [2022-01-10 12:53:10,708] jepsen worker 61 - jepsen.util 61 :invoke :write [20 3] INFO [2022-01-10 12:53:10,712] jepsen worker 50 - jepsen.util 50 :ok :write [16 27] INFO [2022-01-10 12:53:10,712] jepsen worker 13 - jepsen.util 13 :ok :write [4 0] INFO [2022-01-10 12:53:10,713] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,714] jepsen worker 50 - jepsen.util 50 :ok :read [16 27] INFO [2022-01-10 12:53:10,714] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,715] jepsen worker 13 - jepsen.util 13 :ok :read [4 0] INFO [2022-01-10 12:53:10,715] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,716] jepsen worker 81 - jepsen.util 81 :ok :write [27 0] INFO [2022-01-10 12:53:10,716] jepsen worker 50 - jepsen.util 50 :ok :read [16 27] INFO [2022-01-10 12:53:10,716] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,717] jepsen worker 13 - jepsen.util 13 :ok :read [4 0] INFO [2022-01-10 12:53:10,719] jepsen worker 81 - jepsen.util 81 :invoke :write [27 46] INFO [2022-01-10 12:53:10,719] jepsen worker 78 - jepsen.util 78 :ok :write [26 39] INFO [2022-01-10 12:53:10,720] jepsen worker 13 - jepsen.util 13 :invoke :write [4 37] INFO [2022-01-10 12:53:10,720] jepsen worker 76 - jepsen.util 76 :ok :write [25 40] INFO [2022-01-10 12:53:10,720] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,721] jepsen worker 50 - jepsen.util 50 :invoke :write [16 29] INFO [2022-01-10 12:53:10,721] jepsen worker 78 - jepsen.util 78 :ok :read [26 39] INFO [2022-01-10 12:53:10,721] jepsen worker 76 - jepsen.util 76 :invoke :write [25 45] INFO [2022-01-10 12:53:10,722] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,723] jepsen worker 78 - jepsen.util 78 :ok :read [26 39] INFO [2022-01-10 12:53:10,723] jepsen worker 78 - jepsen.util 78 :invoke :write [26 36] INFO [2022-01-10 12:53:10,731] jepsen worker 52 - jepsen.util 52 :ok :write [17 41] INFO [2022-01-10 12:53:10,732] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,733] jepsen worker 52 - jepsen.util 52 :ok :read [17 41] INFO [2022-01-10 12:53:10,733] jepsen worker 23 - jepsen.util 23 :ok :write [7 5] INFO [2022-01-10 12:53:10,733] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,734] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:10,734] jepsen worker 52 - jepsen.util 52 :ok :read [17 41] INFO [2022-01-10 12:53:10,735] jepsen worker 23 - jepsen.util 23 :ok :read [7 5] INFO [2022-01-10 12:53:10,735] jepsen worker 52 - jepsen.util 52 :invoke :write [17 48] INFO [2022-01-10 12:53:10,736] jepsen worker 23 - jepsen.util 23 :invoke :write [7 1] INFO [2022-01-10 12:53:10,739] jepsen worker 43 - jepsen.util 43 :ok :write [14 7] INFO [2022-01-10 12:53:10,741] jepsen worker 43 - jepsen.util 43 :invoke :write [14 46] INFO [2022-01-10 12:53:10,743] jepsen worker 71 - jepsen.util 71 :ok :write [23 11] INFO [2022-01-10 12:53:10,744] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,745] jepsen worker 71 - jepsen.util 71 :ok :read [23 11] INFO [2022-01-10 12:53:10,746] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,747] jepsen worker 71 - jepsen.util 71 :ok :read [23 11] INFO [2022-01-10 12:53:10,748] jepsen worker 71 - jepsen.util 71 :invoke :write [23 26] INFO [2022-01-10 12:53:10,756] jepsen worker 13 - jepsen.util 13 :ok :write [4 37] INFO [2022-01-10 12:53:10,757] jepsen worker 13 - jepsen.util 13 :invoke :write [4 17] INFO [2022-01-10 12:53:10,761] jepsen worker 78 - jepsen.util 78 :ok :write [26 36] INFO [2022-01-10 12:53:10,762] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,763] jepsen worker 78 - jepsen.util 78 :ok :read [26 36] INFO [2022-01-10 12:53:10,763] jepsen worker 61 - jepsen.util 61 :ok :write [20 3] INFO [2022-01-10 12:53:10,763] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,764] jepsen worker 61 - jepsen.util 61 :invoke :write [20 46] INFO [2022-01-10 12:53:10,764] jepsen worker 78 - jepsen.util 78 :ok :read [26 36] INFO [2022-01-10 12:53:10,765] jepsen worker 78 - jepsen.util 78 :invoke :write [26 47] INFO [2022-01-10 12:53:10,768] jepsen worker 52 - jepsen.util 52 :ok :write [17 48] INFO [2022-01-10 12:53:10,769] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,770] jepsen worker 52 - jepsen.util 52 :ok :read [17 48] INFO [2022-01-10 12:53:10,770] jepsen worker 52 - jepsen.util 52 :invoke :write [17 12] INFO [2022-01-10 12:53:10,777] jepsen worker 71 - jepsen.util 71 :ok :write [23 26] INFO [2022-01-10 12:53:10,778] jepsen worker 71 - jepsen.util 71 :invoke :write [23 19] INFO [2022-01-10 12:53:10,782] jepsen worker 76 - jepsen.util 76 :ok :write [25 45] INFO [2022-01-10 12:53:10,783] jepsen worker 76 - jepsen.util 76 :invoke :write [25 32] INFO [2022-01-10 12:53:10,784] jepsen worker 13 - jepsen.util 13 :ok :write [4 17] INFO [2022-01-10 12:53:10,784] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,785] jepsen worker 13 - jepsen.util 13 :ok :read [4 17] INFO [2022-01-10 12:53:10,786] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,786] jepsen worker 13 - jepsen.util 13 :ok :read [4 17] INFO [2022-01-10 12:53:10,787] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,788] jepsen worker 13 - jepsen.util 13 :ok :read [4 17] INFO [2022-01-10 12:53:10,788] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,789] jepsen worker 13 - jepsen.util 13 :ok :read [4 17] INFO [2022-01-10 12:53:10,790] jepsen worker 13 - jepsen.util 13 :invoke :write [4 17] INFO [2022-01-10 12:53:10,791] jepsen worker 23 - jepsen.util 23 :ok :write [7 1] INFO [2022-01-10 12:53:10,791] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:10,792] jepsen worker 23 - jepsen.util 23 :ok :read [7 1] INFO [2022-01-10 12:53:10,793] jepsen worker 23 - jepsen.util 23 :invoke :write [7 13] INFO [2022-01-10 12:53:10,793] jepsen worker 78 - jepsen.util 78 :ok :write [26 47] INFO [2022-01-10 12:53:10,794] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:10,796] jepsen worker 43 - jepsen.util 43 :ok :write [14 46] INFO [2022-01-10 12:53:10,797] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,799] jepsen worker 73 - jepsen.util 73 :info :write [24 28] :RequestTimeoutException INFO [2022-01-10 12:53:10,800] jepsen worker 73 - jepsen.util 163 :invoke :write [24 9] INFO [2022-01-10 12:53:10,802] jepsen worker 52 - jepsen.util 52 :ok :write [17 12] INFO [2022-01-10 12:53:10,803] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,803] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:10,804] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,804] jepsen worker 52 - jepsen.util 52 :ok :read [17 12] INFO [2022-01-10 12:53:10,805] jepsen worker 61 - jepsen.util 61 :ok :write [20 46] INFO [2022-01-10 12:53:10,805] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:10,805] jepsen worker 52 - jepsen.util 52 :invoke :write [17 6] INFO [2022-01-10 12:53:10,806] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,806] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:10,806] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:10,807] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,807] jepsen worker 61 - jepsen.util 61 :ok :read [20 46] INFO [2022-01-10 12:53:10,808] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:10,808] jepsen worker 61 - jepsen.util 61 :invoke :write [20 2] INFO [2022-01-10 12:53:10,809] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,809] jepsen worker 71 - jepsen.util 71 :ok :write [23 19] INFO [2022-01-10 12:53:10,810] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,810] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:10,810] jepsen worker 40 - jepsen.util 40 :ok :write [13 23] INFO [2022-01-10 12:53:10,810] jepsen worker 71 - jepsen.util 71 :ok :read [23 19] INFO [2022-01-10 12:53:10,810] jepsen worker 43 - jepsen.util 43 :invoke :write [14 0] INFO [2022-01-10 12:53:10,811] jepsen worker 38 - jepsen.util 38 :ok :write [12 41] INFO [2022-01-10 12:53:10,811] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,812] jepsen worker 38 - jepsen.util 38 :invoke :write [12 22] INFO [2022-01-10 12:53:10,812] jepsen worker 40 - jepsen.util 40 :ok :read [13 23] INFO [2022-01-10 12:53:10,812] jepsen worker 76 - jepsen.util 76 :ok :write [25 32] INFO [2022-01-10 12:53:10,813] jepsen worker 71 - jepsen.util 71 :invoke :write [23 30] INFO [2022-01-10 12:53:10,813] jepsen worker 40 - jepsen.util 40 :invoke :write [13 34] INFO [2022-01-10 12:53:10,814] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,815] jepsen worker 13 - jepsen.util 13 :ok :write [4 17] INFO [2022-01-10 12:53:10,815] jepsen worker 76 - jepsen.util 76 :ok :read [25 32] INFO [2022-01-10 12:53:10,816] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,816] jepsen worker 76 - jepsen.util 76 :invoke :write [25 41] INFO [2022-01-10 12:53:10,817] jepsen worker 13 - jepsen.util 13 :ok :read [4 17] INFO [2022-01-10 12:53:10,817] jepsen worker 67 - jepsen.util 67 :ok :write [22 42] INFO [2022-01-10 12:53:10,818] jepsen worker 13 - jepsen.util 13 :invoke :write [4 31] INFO [2022-01-10 12:53:10,818] jepsen worker 67 - jepsen.util 67 :invoke :write [22 48] INFO [2022-01-10 12:53:10,819] jepsen worker 3 - jepsen.util 3 :ok :write [1 8] INFO [2022-01-10 12:53:10,819] jepsen worker 23 - jepsen.util 23 :ok :write [7 13] INFO [2022-01-10 12:53:10,819] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:10,820] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:10,821] jepsen worker 23 - jepsen.util 23 :ok :read [7 13] INFO [2022-01-10 12:53:10,822] jepsen worker 23 - jepsen.util 23 :invoke :write [7 3] INFO [2022-01-10 12:53:10,822] jepsen worker 84 - jepsen.util 84 :ok :write [28 5] INFO [2022-01-10 12:53:10,822] jepsen worker 78 - jepsen.util 78 :ok :read [26 47] INFO [2022-01-10 12:53:10,823] jepsen worker 3 - jepsen.util 3 :ok :read [1 8] INFO [2022-01-10 12:53:10,823] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:10,824] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:10,824] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:10,824] jepsen worker 78 - jepsen.util 78 :invoke :write [26 29] INFO [2022-01-10 12:53:10,824] jepsen worker 1 - jepsen.util 1 :ok :write [0 7] INFO [2022-01-10 12:53:10,825] jepsen worker 3 - jepsen.util 3 :ok :read [1 8] INFO [2022-01-10 12:53:10,825] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:10,826] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:10,826] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:10,826] jepsen worker 42 - jepsen.util 132 :ok :write [14 48] INFO [2022-01-10 12:53:10,826] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:10,827] jepsen worker 84 - jepsen.util 84 :invoke :write [28 20] INFO [2022-01-10 12:53:10,827] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,827] jepsen worker 3 - jepsen.util 3 :ok :read [1 8] INFO [2022-01-10 12:53:10,828] jepsen worker 52 - jepsen.util 52 :ok :write [17 6] INFO [2022-01-10 12:53:10,828] jepsen worker 3 - jepsen.util 3 :invoke :write [1 39] INFO [2022-01-10 12:53:10,828] jepsen worker 42 - jepsen.util 132 :ok :read [14 48] INFO [2022-01-10 12:53:10,828] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,829] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,829] jepsen worker 52 - jepsen.util 52 :ok :read [17 6] INFO [2022-01-10 12:53:10,830] jepsen worker 42 - jepsen.util 132 :ok :read [14 48] INFO [2022-01-10 12:53:10,830] jepsen worker 52 - jepsen.util 52 :invoke :write [17 12] INFO [2022-01-10 12:53:10,830] jepsen worker 42 - jepsen.util 132 :invoke :write [14 32] INFO [2022-01-10 12:53:10,833] jepsen worker 87 - jepsen.util 87 :ok :write [29 31] INFO [2022-01-10 12:53:10,833] jepsen worker 87 - jepsen.util 87 :invoke :write [29 26] INFO [2022-01-10 12:53:10,838] jepsen worker 7 - jepsen.util 7 :ok :write [2 28] INFO [2022-01-10 12:53:10,838] jepsen worker 32 - jepsen.util 32 :info :write [10 45] :RequestTimeoutException INFO [2022-01-10 12:53:10,839] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:10,839] jepsen worker 32 - jepsen.util 122 :invoke :write [10 44] INFO [2022-01-10 12:53:10,840] jepsen worker 7 - jepsen.util 7 :ok :read [2 28] INFO [2022-01-10 12:53:10,840] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:10,841] jepsen worker 7 - jepsen.util 7 :ok :read [2 28] INFO [2022-01-10 12:53:10,841] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:10,842] jepsen worker 34 - jepsen.util 34 :ok :write [11 44] INFO [2022-01-10 12:53:10,843] jepsen worker 7 - jepsen.util 7 :ok :read [2 28] INFO [2022-01-10 12:53:10,843] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,843] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:10,844] jepsen worker 34 - jepsen.util 34 :ok :read [11 44] INFO [2022-01-10 12:53:10,844] jepsen worker 7 - jepsen.util 7 :ok :read [2 28] INFO [2022-01-10 12:53:10,844] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,845] jepsen worker 7 - jepsen.util 7 :invoke :write [2 18] INFO [2022-01-10 12:53:10,845] jepsen worker 34 - jepsen.util 34 :ok :read [11 44] INFO [2022-01-10 12:53:10,845] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,846] jepsen worker 15 - jepsen.util 15 :ok :write [5 15] INFO [2022-01-10 12:53:10,846] jepsen worker 34 - jepsen.util 34 :ok :read [11 44] INFO [2022-01-10 12:53:10,847] jepsen worker 15 - jepsen.util 15 :invoke :write [5 4] INFO [2022-01-10 12:53:10,847] jepsen worker 34 - jepsen.util 34 :invoke :write [11 0] INFO [2022-01-10 12:53:10,852] jepsen worker 50 - jepsen.util 50 :ok :write [16 29] INFO [2022-01-10 12:53:10,852] jepsen worker 1 - jepsen.util 1 :ok :read [0 7] INFO [2022-01-10 12:53:10,853] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,853] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:10,853] jepsen worker 50 - jepsen.util 50 :ok :read [16 29] INFO [2022-01-10 12:53:10,854] jepsen worker 1 - jepsen.util 1 :ok :read [0 7] INFO [2022-01-10 12:53:10,854] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,855] jepsen worker 50 - jepsen.util 50 :ok :read [16 29] INFO [2022-01-10 12:53:10,855] jepsen worker 1 - jepsen.util 1 :invoke :write [0 0] INFO [2022-01-10 12:53:10,856] jepsen worker 50 - jepsen.util 50 :invoke :write [16 10] INFO [2022-01-10 12:53:10,858] jepsen worker 81 - jepsen.util 81 :ok :write [27 46] INFO [2022-01-10 12:53:10,859] jepsen worker 81 - jepsen.util 81 :invoke :write [27 34] INFO [2022-01-10 12:53:10,879] jepsen worker 31 - jepsen.util 31 :info :write [10 26] :RequestTimeoutException INFO [2022-01-10 12:53:10,880] jepsen worker 77 - jepsen.util 77 :info :write [25 14] :RequestTimeoutException INFO [2022-01-10 12:53:10,880] jepsen worker 31 - jepsen.util 121 :invoke :write [10 19] INFO [2022-01-10 12:53:10,881] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,882] jepsen worker 77 - jepsen.util 167 :ok :read [25 32] INFO [2022-01-10 12:53:10,882] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,883] jepsen worker 77 - jepsen.util 167 :ok :read [25 32] INFO [2022-01-10 12:53:10,884] jepsen worker 77 - jepsen.util 167 :invoke :write [25 42] INFO [2022-01-10 12:53:10,886] jepsen worker 82 - jepsen.util 82 :info :write [27 39] :RequestTimeoutException INFO [2022-01-10 12:53:10,887] jepsen worker 82 - jepsen.util 172 :invoke :read [27 nil] INFO [2022-01-10 12:53:10,888] jepsen worker 82 - jepsen.util 172 :ok :read [27 46] INFO [2022-01-10 12:53:10,888] jepsen worker 82 - jepsen.util 172 :invoke :read [27 nil] INFO [2022-01-10 12:53:10,889] jepsen worker 82 - jepsen.util 172 :ok :read [27 46] INFO [2022-01-10 12:53:10,889] jepsen worker 82 - jepsen.util 172 :invoke :write [27 31] INFO [2022-01-10 12:53:10,896] jepsen worker 12 - jepsen.util 12 :info :write [4 38] :RequestTimeoutException INFO [2022-01-10 12:53:10,897] jepsen worker 19 - jepsen.util 19 :info :write [6 32] :RequestTimeoutException INFO [2022-01-10 12:53:10,899] jepsen worker 59 - jepsen.util 59 :info :write [19 43] :RequestTimeoutException INFO [2022-01-10 12:53:10,899] jepsen worker 80 - jepsen.util 80 :info :write [26 0] :RequestTimeoutException INFO [2022-01-10 12:53:10,900] jepsen worker 12 - jepsen.util 102 :invoke :read [4 nil] INFO [2022-01-10 12:53:10,901] jepsen worker 19 - jepsen.util 109 :invoke :write [6 9] INFO [2022-01-10 12:53:10,901] jepsen worker 12 - jepsen.util 102 :ok :read [4 17] INFO [2022-01-10 12:53:10,901] jepsen worker 59 - jepsen.util 149 :invoke :write [19 4] INFO [2022-01-10 12:53:10,902] jepsen worker 80 - jepsen.util 170 :invoke :write [26 13] INFO [2022-01-10 12:53:10,903] jepsen worker 12 - jepsen.util 102 :invoke :write [4 29] INFO [2022-01-10 12:53:10,905] jepsen worker 56 - jepsen.util 56 :info :write [18 11] :RequestTimeoutException INFO [2022-01-10 12:53:10,906] jepsen worker 56 - jepsen.util 146 :invoke :read [18 nil] INFO [2022-01-10 12:53:10,928] jepsen worker 40 - jepsen.util 40 :ok :write [13 34] INFO [2022-01-10 12:53:10,929] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,929] jepsen worker 71 - jepsen.util 71 :ok :write [23 30] INFO [2022-01-10 12:53:10,930] jepsen worker 40 - jepsen.util 40 :ok :read [13 34] INFO [2022-01-10 12:53:10,930] jepsen worker 61 - jepsen.util 61 :ok :write [20 2] INFO [2022-01-10 12:53:10,930] jepsen worker 71 - jepsen.util 71 :invoke :write [23 33] INFO [2022-01-10 12:53:10,931] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,931] jepsen worker 61 - jepsen.util 61 :invoke :write [20 44] INFO [2022-01-10 12:53:10,932] jepsen worker 40 - jepsen.util 40 :ok :read [13 34] INFO [2022-01-10 12:53:10,932] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,933] jepsen worker 40 - jepsen.util 40 :ok :read [13 34] INFO [2022-01-10 12:53:10,934] jepsen worker 40 - jepsen.util 40 :invoke :write [13 43] INFO [2022-01-10 12:53:10,935] jepsen worker 38 - jepsen.util 38 :ok :write [12 22] INFO [2022-01-10 12:53:10,935] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:10,939] jepsen worker 76 - jepsen.util 76 :ok :write [25 41] INFO [2022-01-10 12:53:10,940] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,940] jepsen worker 13 - jepsen.util 13 :ok :write [4 31] INFO [2022-01-10 12:53:10,941] jepsen worker 76 - jepsen.util 76 :ok :read [25 41] INFO [2022-01-10 12:53:10,941] jepsen worker 13 - jepsen.util 13 :invoke :write [4 11] INFO [2022-01-10 12:53:10,942] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,943] jepsen worker 76 - jepsen.util 76 :ok :read [25 41] INFO [2022-01-10 12:53:10,943] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,944] jepsen worker 76 - jepsen.util 76 :ok :read [25 41] INFO [2022-01-10 12:53:10,945] jepsen worker 1 - jepsen.util 1 :ok :write [0 0] INFO [2022-01-10 12:53:10,945] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,946] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:10,946] jepsen worker 76 - jepsen.util 76 :ok :read [25 41] INFO [2022-01-10 12:53:10,946] jepsen worker 3 - jepsen.util 3 :ok :write [1 39] INFO [2022-01-10 12:53:10,947] jepsen worker 1 - jepsen.util 1 :ok :read [0 0] INFO [2022-01-10 12:53:10,947] jepsen worker 3 - jepsen.util 3 :invoke :write [1 24] INFO [2022-01-10 12:53:10,947] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,948] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:10,948] jepsen worker 76 - jepsen.util 76 :ok :read [25 41] INFO [2022-01-10 12:53:10,949] jepsen worker 1 - jepsen.util 1 :ok :read [0 0] INFO [2022-01-10 12:53:10,949] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,949] jepsen worker 67 - jepsen.util 67 :ok :write [22 48] INFO [2022-01-10 12:53:10,949] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:10,950] jepsen worker 76 - jepsen.util 76 :ok :read [25 41] INFO [2022-01-10 12:53:10,950] jepsen worker 1 - jepsen.util 1 :invoke :write [0 18] INFO [2022-01-10 12:53:10,950] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,950] jepsen worker 67 - jepsen.util 67 :ok :read [22 48] INFO [2022-01-10 12:53:10,951] jepsen worker 67 - jepsen.util 67 :invoke :write [22 49] INFO [2022-01-10 12:53:10,951] jepsen worker 76 - jepsen.util 76 :ok :read [25 41] INFO [2022-01-10 12:53:10,952] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,953] jepsen worker 76 - jepsen.util 76 :ok :read [25 41] INFO [2022-01-10 12:53:10,953] jepsen worker 84 - jepsen.util 84 :ok :write [28 20] INFO [2022-01-10 12:53:10,953] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,954] jepsen worker 84 - jepsen.util 84 :invoke :write [28 31] INFO [2022-01-10 12:53:10,956] jepsen worker 23 - jepsen.util 23 :ok :write [7 3] INFO [2022-01-10 12:53:10,956] jepsen worker 45 - jepsen.util 45 :info :write [15 35] :RequestTimeoutException INFO [2022-01-10 12:53:10,956] jepsen worker 54 - jepsen.util 54 :info :write [18 32] :RequestTimeoutException INFO [2022-01-10 12:53:10,956] jepsen worker 76 - jepsen.util 76 :ok :read [25 41] INFO [2022-01-10 12:53:10,958] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,958] jepsen worker 87 - jepsen.util 87 :ok :write [29 26] INFO [2022-01-10 12:53:10,959] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:10,959] jepsen worker 76 - jepsen.util 76 :ok :read [25 41] INFO [2022-01-10 12:53:10,959] jepsen worker 78 - jepsen.util 78 :ok :write [26 29] INFO [2022-01-10 12:53:10,959] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:10,960] jepsen worker 23 - jepsen.util 23 :ok :read [7 3] INFO [2022-01-10 12:53:10,960] jepsen worker 45 - jepsen.util 135 :invoke :write [15 3] INFO [2022-01-10 12:53:10,960] jepsen worker 87 - jepsen.util 87 :ok :read [29 26] INFO [2022-01-10 12:53:10,961] jepsen worker 76 - jepsen.util 76 :invoke :read [25 nil] INFO [2022-01-10 12:53:10,962] jepsen worker 87 - jepsen.util 87 :invoke :write [29 42] INFO [2022-01-10 12:53:10,962] jepsen worker 76 - jepsen.util 76 :ok :read [25 41] INFO [2022-01-10 12:53:10,962] jepsen worker 78 - jepsen.util 78 :invoke :write [26 14] INFO [2022-01-10 12:53:10,963] jepsen worker 76 - jepsen.util 76 :invoke :write [25 38] INFO [2022-01-10 12:53:10,963] jepsen worker 54 - jepsen.util 144 :invoke :write [18 27] INFO [2022-01-10 12:53:10,964] jepsen worker 42 - jepsen.util 132 :ok :write [14 32] INFO [2022-01-10 12:53:10,964] jepsen worker 23 - jepsen.util 23 :invoke :write [7 27] INFO [2022-01-10 12:53:10,964] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,965] jepsen worker 42 - jepsen.util 132 :ok :read [14 32] INFO [2022-01-10 12:53:10,966] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,967] jepsen worker 42 - jepsen.util 132 :ok :read [14 32] INFO [2022-01-10 12:53:10,968] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:10,968] jepsen worker 40 - jepsen.util 40 :ok :write [13 43] INFO [2022-01-10 12:53:10,968] jepsen worker 9 - jepsen.util 9 :info :write [3 35] :RequestTimeoutException INFO [2022-01-10 12:53:10,968] jepsen worker 52 - jepsen.util 52 :ok :write [17 12] INFO [2022-01-10 12:53:10,969] jepsen worker 42 - jepsen.util 132 :ok :read [14 32] INFO [2022-01-10 12:53:10,969] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:10,970] jepsen worker 42 - jepsen.util 132 :invoke :write [14 12] INFO [2022-01-10 12:53:10,970] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:10,971] jepsen worker 40 - jepsen.util 40 :ok :read [13 43] INFO [2022-01-10 12:53:10,971] jepsen worker 9 - jepsen.util 99 :invoke :read [3 nil] INFO [2022-01-10 12:53:10,971] jepsen worker 52 - jepsen.util 52 :ok :read [17 12] INFO [2022-01-10 12:53:10,972] jepsen worker 40 - jepsen.util 40 :invoke :write [13 34] INFO [2022-01-10 12:53:10,972] jepsen worker 7 - jepsen.util 7 :ok :write [2 18] INFO [2022-01-10 12:53:10,972] jepsen worker 52 - jepsen.util 52 :invoke :write [17 38] INFO [2022-01-10 12:53:10,973] jepsen worker 71 - jepsen.util 71 :ok :write [23 33] INFO [2022-01-10 12:53:10,974] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,975] jepsen worker 46 - jepsen.util 46 :info :write [15 26] :RequestTimeoutException INFO [2022-01-10 12:53:10,975] jepsen worker 7 - jepsen.util 7 :invoke :write [2 1] INFO [2022-01-10 12:53:10,976] jepsen worker 46 - jepsen.util 136 :invoke :read [15 nil] INFO [2022-01-10 12:53:10,978] jepsen worker 28 - jepsen.util 28 :info :write [9 24] :RequestTimeoutException INFO [2022-01-10 12:53:10,978] jepsen worker 34 - jepsen.util 34 :ok :write [11 0] INFO [2022-01-10 12:53:10,979] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:10,979] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:10,980] jepsen worker 38 - jepsen.util 38 :invoke :write [12 17] INFO [2022-01-10 12:53:10,980] jepsen worker 34 - jepsen.util 34 :ok :read [11 0] INFO [2022-01-10 12:53:10,981] jepsen worker 34 - jepsen.util 34 :invoke :write [11 30] INFO [2022-01-10 12:53:10,982] jepsen worker 28 - jepsen.util 118 :invoke :write [9 42] INFO [2022-01-10 12:53:10,983] jepsen worker 71 - jepsen.util 71 :ok :read [23 33] INFO [2022-01-10 12:53:10,984] jepsen worker 15 - jepsen.util 15 :ok :write [5 4] INFO [2022-01-10 12:53:10,984] jepsen worker 46 - jepsen.util 136 :ok :read [15 10] INFO [2022-01-10 12:53:10,985] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:10,985] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:10,986] jepsen worker 71 - jepsen.util 71 :ok :read [23 33] INFO [2022-01-10 12:53:10,986] jepsen worker 46 - jepsen.util 136 :invoke :write [15 44] INFO [2022-01-10 12:53:10,986] jepsen worker 1 - jepsen.util 1 :ok :write [0 18] INFO [2022-01-10 12:53:10,986] jepsen worker 13 - jepsen.util 13 :ok :write [4 11] INFO [2022-01-10 12:53:10,986] jepsen worker 15 - jepsen.util 15 :ok :read [5 4] INFO [2022-01-10 12:53:10,987] jepsen worker 71 - jepsen.util 71 :invoke :write [23 35] INFO [2022-01-10 12:53:10,988] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:10,989] jepsen worker 13 - jepsen.util 13 :invoke :write [4 40] INFO [2022-01-10 12:53:10,989] jepsen worker 1 - jepsen.util 1 :ok :read [0 18] INFO [2022-01-10 12:53:10,989] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:10,990] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:10,990] jepsen worker 15 - jepsen.util 15 :ok :read [5 4] INFO [2022-01-10 12:53:10,991] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:10,991] jepsen worker 1 - jepsen.util 1 :ok :read [0 18] INFO [2022-01-10 12:53:10,991] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:10,991] jepsen worker 87 - jepsen.util 87 :ok :write [29 42] INFO [2022-01-10 12:53:10,992] jepsen worker 87 - jepsen.util 87 :invoke :write [29 48] INFO [2022-01-10 12:53:10,993] jepsen worker 15 - jepsen.util 15 :ok :read [5 4] INFO [2022-01-10 12:53:10,993] jepsen worker 1 - jepsen.util 1 :ok :read [0 18] INFO [2022-01-10 12:53:10,993] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:10,994] jepsen worker 1 - jepsen.util 1 :invoke :write [0 44] INFO [2022-01-10 12:53:10,994] jepsen worker 3 - jepsen.util 3 :ok :write [1 24] INFO [2022-01-10 12:53:10,994] jepsen worker 50 - jepsen.util 50 :ok :write [16 10] INFO [2022-01-10 12:53:10,995] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,995] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:10,995] jepsen worker 15 - jepsen.util 15 :ok :read [5 4] INFO [2022-01-10 12:53:10,996] jepsen worker 50 - jepsen.util 50 :ok :read [16 10] INFO [2022-01-10 12:53:10,996] jepsen worker 15 - jepsen.util 15 :invoke :write [5 49] INFO [2022-01-10 12:53:10,996] jepsen worker 3 - jepsen.util 3 :ok :read [1 24] INFO [2022-01-10 12:53:10,996] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:10,997] jepsen worker 3 - jepsen.util 3 :invoke :write [1 44] INFO [2022-01-10 12:53:10,997] jepsen worker 50 - jepsen.util 50 :ok :read [16 10] INFO [2022-01-10 12:53:11,002] jepsen worker 72 - jepsen.util 72 :info :write [24 3] :RequestTimeoutException INFO [2022-01-10 12:53:11,002] jepsen worker 81 - jepsen.util 81 :ok :write [27 34] INFO [2022-01-10 12:53:11,003] jepsen worker 84 - jepsen.util 84 :ok :write [28 31] INFO [2022-01-10 12:53:11,003] jepsen worker 50 - jepsen.util 50 :invoke :write [16 7] INFO [2022-01-10 12:53:11,003] jepsen worker 72 - jepsen.util 162 :invoke :write [24 47] INFO [2022-01-10 12:53:11,005] jepsen worker 78 - jepsen.util 78 :ok :write [26 14] INFO [2022-01-10 12:53:11,005] jepsen worker 81 - jepsen.util 81 :invoke :write [27 23] INFO [2022-01-10 12:53:11,006] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,007] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,007] jepsen worker 84 - jepsen.util 84 :ok :read [28 31] INFO [2022-01-10 12:53:11,008] jepsen worker 84 - jepsen.util 84 :invoke :write [28 43] INFO [2022-01-10 12:53:11,008] jepsen worker 78 - jepsen.util 78 :ok :read [26 14] INFO [2022-01-10 12:53:11,009] jepsen worker 61 - jepsen.util 61 :ok :write [20 44] INFO [2022-01-10 12:53:11,009] jepsen worker 78 - jepsen.util 78 :invoke :write [26 8] INFO [2022-01-10 12:53:11,010] jepsen worker 61 - jepsen.util 61 :invoke :write [20 44] INFO [2022-01-10 12:53:11,012] jepsen worker 52 - jepsen.util 52 :ok :write [17 38] INFO [2022-01-10 12:53:11,012] jepsen worker 40 - jepsen.util 40 :ok :write [13 34] INFO [2022-01-10 12:53:11,013] jepsen worker 52 - jepsen.util 52 :invoke :write [17 16] INFO [2022-01-10 12:53:11,013] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:11,014] jepsen worker 40 - jepsen.util 40 :ok :read [13 34] INFO [2022-01-10 12:53:11,015] jepsen worker 40 - jepsen.util 40 :invoke :write [13 3] INFO [2022-01-10 12:53:11,016] jepsen worker 71 - jepsen.util 71 :ok :write [23 35] INFO [2022-01-10 12:53:11,017] jepsen worker 77 - jepsen.util 167 :ok :write [25 42] INFO [2022-01-10 12:53:11,017] jepsen worker 1 - jepsen.util 1 :ok :write [0 44] INFO [2022-01-10 12:53:11,017] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,018] jepsen worker 1 - jepsen.util 1 :invoke :write [0 21] INFO [2022-01-10 12:53:11,018] jepsen worker 77 - jepsen.util 167 :invoke :write [25 11] INFO [2022-01-10 12:53:11,018] jepsen worker 71 - jepsen.util 71 :ok :read [23 35] INFO [2022-01-10 12:53:11,019] jepsen worker 71 - jepsen.util 71 :invoke :write [23 4] INFO [2022-01-10 12:53:11,022] jepsen worker 87 - jepsen.util 87 :ok :write [29 48] INFO [2022-01-10 12:53:11,022] jepsen worker 87 - jepsen.util 87 :invoke :write [29 16] INFO [2022-01-10 12:53:11,023] jepsen worker 38 - jepsen.util 38 :ok :write [12 17] INFO [2022-01-10 12:53:11,023] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,024] jepsen worker 38 - jepsen.util 38 :ok :read [12 17] INFO [2022-01-10 12:53:11,025] jepsen worker 67 - jepsen.util 67 :ok :write [22 49] INFO [2022-01-10 12:53:11,025] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,026] jepsen worker 67 - jepsen.util 67 :invoke :write [22 37] INFO [2022-01-10 12:53:11,026] jepsen worker 38 - jepsen.util 38 :ok :read [12 17] INFO [2022-01-10 12:53:11,027] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,028] jepsen worker 38 - jepsen.util 38 :ok :read [12 17] INFO [2022-01-10 12:53:11,028] jepsen worker 38 - jepsen.util 38 :invoke :write [12 25] INFO [2022-01-10 12:53:11,029] jepsen worker 13 - jepsen.util 13 :ok :write [4 40] INFO [2022-01-10 12:53:11,029] jepsen worker 13 - jepsen.util 13 :invoke :write [4 7] INFO [2022-01-10 12:53:11,029] jepsen worker 23 - jepsen.util 23 :ok :write [7 27] INFO [2022-01-10 12:53:11,030] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,031] jepsen worker 23 - jepsen.util 23 :ok :read [7 27] INFO [2022-01-10 12:53:11,031] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,032] jepsen worker 23 - jepsen.util 23 :ok :read [7 27] INFO [2022-01-10 12:53:11,032] jepsen worker 23 - jepsen.util 23 :invoke :write [7 2] INFO [2022-01-10 12:53:11,035] jepsen worker 42 - jepsen.util 132 :ok :write [14 12] INFO [2022-01-10 12:53:11,035] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,036] jepsen worker 40 - jepsen.util 40 :ok :write [13 3] INFO [2022-01-10 12:53:11,036] jepsen worker 42 - jepsen.util 132 :ok :read [14 12] INFO [2022-01-10 12:53:11,036] jepsen worker 40 - jepsen.util 40 :invoke :write [13 2] INFO [2022-01-10 12:53:11,038] jepsen worker 3 - jepsen.util 3 :ok :write [1 44] INFO [2022-01-10 12:53:11,038] jepsen worker 25 - jepsen.util 25 :info :write [8 3] :RequestTimeoutException INFO [2022-01-10 12:53:11,039] jepsen worker 42 - jepsen.util 132 :invoke :write [14 39] INFO [2022-01-10 12:53:11,040] jepsen worker 25 - jepsen.util 115 :invoke :write [8 7] INFO [2022-01-10 12:53:11,040] jepsen worker 3 - jepsen.util 3 :invoke :write [1 14] INFO [2022-01-10 12:53:11,041] jepsen worker 1 - jepsen.util 1 :ok :write [0 21] INFO [2022-01-10 12:53:11,042] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:11,043] jepsen worker 1 - jepsen.util 1 :ok :read [0 21] INFO [2022-01-10 12:53:11,043] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:11,044] jepsen worker 7 - jepsen.util 7 :ok :write [2 1] INFO [2022-01-10 12:53:11,045] jepsen worker 24 - jepsen.util 24 :info :write [8 18] :RequestTimeoutException INFO [2022-01-10 12:53:11,045] jepsen worker 1 - jepsen.util 1 :ok :read [0 21] INFO [2022-01-10 12:53:11,046] jepsen worker 84 - jepsen.util 84 :ok :write [28 43] INFO [2022-01-10 12:53:11,046] jepsen worker 24 - jepsen.util 114 :invoke :write [8 21] INFO [2022-01-10 12:53:11,047] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:11,047] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:11,048] jepsen worker 84 - jepsen.util 84 :invoke :write [28 44] INFO [2022-01-10 12:53:11,048] jepsen worker 1 - jepsen.util 1 :ok :read [0 21] INFO [2022-01-10 12:53:11,048] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:53:11,048] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:11,049] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:11,049] jepsen worker 1 - jepsen.util 1 :ok :read [0 21] INFO [2022-01-10 12:53:11,049] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:53:11,050] jepsen worker 1 - jepsen.util 1 :invoke :write [0 35] INFO [2022-01-10 12:53:11,050] jepsen worker 7 - jepsen.util 7 :invoke :write [2 23] INFO [2022-01-10 12:53:11,050] jepsen worker 78 - jepsen.util 78 :ok :write [26 8] INFO [2022-01-10 12:53:11,051] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,051] jepsen worker 34 - jepsen.util 34 :ok :write [11 30] INFO [2022-01-10 12:53:11,052] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:11,052] jepsen worker 78 - jepsen.util 78 :ok :read [26 8] INFO [2022-01-10 12:53:11,053] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,053] jepsen worker 34 - jepsen.util 34 :ok :read [11 30] INFO [2022-01-10 12:53:11,053] jepsen worker 87 - jepsen.util 87 :ok :write [29 16] INFO [2022-01-10 12:53:11,053] jepsen worker 78 - jepsen.util 78 :ok :read [26 8] INFO [2022-01-10 12:53:11,054] jepsen worker 87 - jepsen.util 87 :invoke :write [29 4] INFO [2022-01-10 12:53:11,055] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,055] jepsen worker 34 - jepsen.util 34 :invoke :write [11 8] INFO [2022-01-10 12:53:11,055] jepsen worker 78 - jepsen.util 78 :ok :read [26 8] INFO [2022-01-10 12:53:11,056] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,057] jepsen worker 78 - jepsen.util 78 :ok :read [26 8] INFO [2022-01-10 12:53:11,057] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,058] jepsen worker 78 - jepsen.util 78 :ok :read [26 8] INFO [2022-01-10 12:53:11,058] jepsen worker 52 - jepsen.util 52 :ok :write [17 16] INFO [2022-01-10 12:53:11,059] jepsen worker 78 - jepsen.util 78 :invoke :write [26 5] INFO [2022-01-10 12:53:11,059] jepsen worker 52 - jepsen.util 52 :invoke :write [17 24] INFO [2022-01-10 12:53:11,063] jepsen worker 15 - jepsen.util 15 :ok :write [5 49] INFO [2022-01-10 12:53:11,064] jepsen worker 15 - jepsen.util 15 :invoke :write [5 10] INFO [2022-01-10 12:53:11,067] jepsen worker 50 - jepsen.util 50 :ok :write [16 7] INFO [2022-01-10 12:53:11,067] jepsen worker 50 - jepsen.util 50 :invoke :write [16 39] INFO [2022-01-10 12:53:11,068] jepsen worker 71 - jepsen.util 71 :ok :write [23 4] INFO [2022-01-10 12:53:11,068] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,069] jepsen worker 71 - jepsen.util 71 :ok :read [23 4] INFO [2022-01-10 12:53:11,070] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,070] jepsen worker 81 - jepsen.util 81 :ok :write [27 23] INFO [2022-01-10 12:53:11,071] jepsen worker 71 - jepsen.util 71 :ok :read [23 4] INFO [2022-01-10 12:53:11,071] jepsen worker 81 - jepsen.util 81 :invoke :write [27 6] INFO [2022-01-10 12:53:11,072] jepsen worker 71 - jepsen.util 71 :invoke :write [23 35] INFO [2022-01-10 12:53:11,074] jepsen worker 36 - jepsen.util 36 :ok :write [12 44] INFO [2022-01-10 12:53:11,075] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,076] jepsen worker 36 - jepsen.util 36 :ok :read [12 44] INFO [2022-01-10 12:53:11,077] jepsen worker 61 - jepsen.util 61 :ok :write [20 44] INFO [2022-01-10 12:53:11,077] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,077] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,078] jepsen worker 61 - jepsen.util 61 :ok :read [20 44] INFO [2022-01-10 12:53:11,079] jepsen worker 36 - jepsen.util 36 :ok :read [12 44] INFO [2022-01-10 12:53:11,079] jepsen worker 61 - jepsen.util 61 :invoke :write [20 29] INFO [2022-01-10 12:53:11,079] jepsen worker 36 - jepsen.util 36 :invoke :write [12 45] INFO [2022-01-10 12:53:11,079] jepsen worker 40 - jepsen.util 40 :ok :write [13 2] INFO [2022-01-10 12:53:11,080] jepsen worker 40 - jepsen.util 40 :invoke :write [13 22] INFO [2022-01-10 12:53:11,083] jepsen worker 13 - jepsen.util 13 :ok :write [4 7] INFO [2022-01-10 12:53:11,083] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:11,084] jepsen worker 13 - jepsen.util 13 :ok :read [4 7] INFO [2022-01-10 12:53:11,085] jepsen worker 13 - jepsen.util 13 :invoke :write [4 11] INFO [2022-01-10 12:53:11,086] jepsen worker 1 - jepsen.util 1 :ok :write [0 35] INFO [2022-01-10 12:53:11,087] jepsen worker 77 - jepsen.util 167 :ok :write [25 11] INFO [2022-01-10 12:53:11,087] jepsen worker 1 - jepsen.util 1 :invoke :write [0 6] INFO [2022-01-10 12:53:11,087] jepsen worker 77 - jepsen.util 167 :invoke :write [25 25] INFO [2022-01-10 12:53:11,089] jepsen worker 3 - jepsen.util 3 :ok :write [1 14] INFO [2022-01-10 12:53:11,090] jepsen worker 3 - jepsen.util 3 :invoke :write [1 43] INFO [2022-01-10 12:53:11,095] jepsen worker 67 - jepsen.util 67 :ok :write [22 37] INFO [2022-01-10 12:53:11,096] jepsen worker 67 - jepsen.util 67 :invoke :write [22 34] INFO [2022-01-10 12:53:11,096] jepsen worker 84 - jepsen.util 84 :ok :write [28 44] INFO [2022-01-10 12:53:11,097] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,098] jepsen worker 84 - jepsen.util 84 :ok :read [28 44] INFO [2022-01-10 12:53:11,098] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,099] jepsen worker 84 - jepsen.util 84 :ok :read [28 44] INFO [2022-01-10 12:53:11,100] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,100] jepsen worker 84 - jepsen.util 84 :ok :read [28 44] INFO [2022-01-10 12:53:11,101] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,102] jepsen worker 84 - jepsen.util 84 :ok :read [28 44] INFO [2022-01-10 12:53:11,102] jepsen worker 78 - jepsen.util 78 :ok :write [26 5] INFO [2022-01-10 12:53:11,102] jepsen worker 23 - jepsen.util 23 :ok :write [7 2] INFO [2022-01-10 12:53:11,102] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,103] jepsen worker 23 - jepsen.util 23 :invoke :write [7 35] INFO [2022-01-10 12:53:11,103] jepsen worker 78 - jepsen.util 78 :ok :read [26 5] INFO [2022-01-10 12:53:11,103] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,104] jepsen worker 78 - jepsen.util 78 :invoke :write [26 12] INFO [2022-01-10 12:53:11,104] jepsen worker 84 - jepsen.util 84 :ok :read [28 44] INFO [2022-01-10 12:53:11,106] jepsen worker 14 - jepsen.util 14 :info :write [4 4] :RequestTimeoutException INFO [2022-01-10 12:53:11,107] jepsen worker 84 - jepsen.util 84 :invoke :write [28 33] INFO [2022-01-10 12:53:11,107] jepsen worker 14 - jepsen.util 104 :invoke :write [4 21] INFO [2022-01-10 12:53:11,108] jepsen worker 42 - jepsen.util 132 :ok :write [14 39] INFO [2022-01-10 12:53:11,109] jepsen worker 42 - jepsen.util 132 :invoke :write [14 8] INFO [2022-01-10 12:53:11,109] jepsen worker 52 - jepsen.util 52 :ok :write [17 24] INFO [2022-01-10 12:53:11,110] jepsen worker 52 - jepsen.util 52 :invoke :write [17 13] INFO [2022-01-10 12:53:11,113] jepsen worker 7 - jepsen.util 7 :ok :write [2 23] INFO [2022-01-10 12:53:11,113] jepsen worker 22 - jepsen.util 22 :info :write [7 40] :RequestTimeoutException INFO [2022-01-10 12:53:11,114] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:11,114] jepsen worker 22 - jepsen.util 112 :invoke :write [7 38] INFO [2022-01-10 12:53:11,115] jepsen worker 7 - jepsen.util 7 :ok :read [2 23] INFO [2022-01-10 12:53:11,115] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:11,116] jepsen worker 7 - jepsen.util 7 :ok :read [2 23] INFO [2022-01-10 12:53:11,116] jepsen worker 7 - jepsen.util 7 :invoke :write [2 4] INFO [2022-01-10 12:53:11,119] jepsen worker 71 - jepsen.util 71 :ok :write [23 35] INFO [2022-01-10 12:53:11,119] jepsen worker 70 - jepsen.util 70 :info :write [23 43] :RequestTimeoutException INFO [2022-01-10 12:53:11,120] jepsen worker 71 - jepsen.util 71 :invoke :write [23 1] INFO [2022-01-10 12:53:11,120] jepsen worker 79 - jepsen.util 79 :info :write [26 24] :RequestTimeoutException INFO [2022-01-10 12:53:11,121] jepsen worker 79 - jepsen.util 169 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,122] jepsen worker 70 - jepsen.util 160 :invoke :write [23 11] INFO [2022-01-10 12:53:11,123] jepsen worker 79 - jepsen.util 169 :ok :read [26 5] INFO [2022-01-10 12:53:11,123] jepsen worker 79 - jepsen.util 169 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,125] jepsen worker 63 - jepsen.util 63 :info :write [21 34] :RequestTimeoutException INFO [2022-01-10 12:53:11,125] jepsen worker 15 - jepsen.util 15 :ok :write [5 10] INFO [2022-01-10 12:53:11,126] jepsen worker 79 - jepsen.util 169 :ok :read [26 5] INFO [2022-01-10 12:53:11,126] jepsen worker 63 - jepsen.util 153 :invoke :read [21 nil] INFO [2022-01-10 12:53:11,126] jepsen worker 36 - jepsen.util 36 :ok :write [12 45] INFO [2022-01-10 12:53:11,128] jepsen worker 2 - jepsen.util 2 :info :write [0 17] :RequestTimeoutException INFO [2022-01-10 12:53:11,129] jepsen worker 88 - jepsen.util 88 :info :write [29 14] :RequestTimeoutException INFO [2022-01-10 12:53:11,131] jepsen worker 5 - jepsen.util 5 :info :write [1 18] :RequestTimeoutException INFO [2022-01-10 12:53:11,131] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,131] jepsen worker 13 - jepsen.util 13 :ok :write [4 11] INFO [2022-01-10 12:53:11,132] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:11,133] jepsen worker 36 - jepsen.util 36 :ok :read [12 45] INFO [2022-01-10 12:53:11,134] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,135] jepsen worker 36 - jepsen.util 36 :ok :read [12 45] INFO [2022-01-10 12:53:11,135] jepsen worker 5 - jepsen.util 95 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,136] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:11,138] jepsen worker 36 - jepsen.util 36 :invoke :write [12 12] INFO [2022-01-10 12:53:11,138] jepsen worker 48 - jepsen.util 48 :info :write [16 19] :RequestTimeoutException INFO [2022-01-10 12:53:11,139] jepsen worker 15 - jepsen.util 15 :ok :read [5 10] INFO [2022-01-10 12:53:11,139] jepsen worker 5 - jepsen.util 95 :ok :read [1 14] INFO [2022-01-10 12:53:11,140] jepsen worker 61 - jepsen.util 61 :ok :write [20 29] INFO [2022-01-10 12:53:11,140] jepsen worker 13 - jepsen.util 13 :invoke :write [4 44] INFO [2022-01-10 12:53:11,140] jepsen worker 35 - jepsen.util 35 :info :write [11 34] :RequestTimeoutException INFO [2022-01-10 12:53:11,141] jepsen worker 88 - jepsen.util 178 :invoke :write [29 17] INFO [2022-01-10 12:53:11,143] jepsen worker 64 - jepsen.util 64 :info :write [21 47] :RequestTimeoutException INFO [2022-01-10 12:53:11,144] jepsen worker 5 - jepsen.util 95 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,145] jepsen worker 5 - jepsen.util 95 :ok :read [1 14] INFO [2022-01-10 12:53:11,145] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,146] jepsen worker 77 - jepsen.util 167 :ok :write [25 25] INFO [2022-01-10 12:53:11,146] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:11,147] jepsen worker 64 - jepsen.util 154 :invoke :read [21 nil] INFO [2022-01-10 12:53:11,148] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:11,148] jepsen worker 64 - jepsen.util 154 :ok :read [21 19] INFO [2022-01-10 12:53:11,149] jepsen worker 48 - jepsen.util 138 :invoke :write [16 25] INFO [2022-01-10 12:53:11,149] jepsen worker 15 - jepsen.util 15 :ok :read [5 10] INFO [2022-01-10 12:53:11,149] jepsen worker 78 - jepsen.util 78 :ok :write [26 12] INFO [2022-01-10 12:53:11,150] jepsen worker 61 - jepsen.util 61 :ok :read [20 29] INFO [2022-01-10 12:53:11,150] jepsen worker 5 - jepsen.util 95 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,151] jepsen worker 64 - jepsen.util 154 :invoke :write [21 41] INFO [2022-01-10 12:53:11,151] jepsen worker 5 - jepsen.util 95 :ok :read [1 14] INFO [2022-01-10 12:53:11,152] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,152] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,152] jepsen worker 77 - jepsen.util 167 :ok :read [25 25] INFO [2022-01-10 12:53:11,153] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:11,153] jepsen worker 78 - jepsen.util 78 :ok :read [26 12] INFO [2022-01-10 12:53:11,153] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,154] jepsen worker 15 - jepsen.util 15 :ok :read [5 10] INFO [2022-01-10 12:53:11,154] jepsen worker 5 - jepsen.util 95 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,154] jepsen worker 61 - jepsen.util 61 :ok :read [20 29] INFO [2022-01-10 12:53:11,155] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:11,155] jepsen worker 5 - jepsen.util 95 :ok :read [1 14] INFO [2022-01-10 12:53:11,156] jepsen worker 78 - jepsen.util 78 :invoke :write [26 41] INFO [2022-01-10 12:53:11,156] jepsen worker 5 - jepsen.util 95 :invoke :write [1 38] INFO [2022-01-10 12:53:11,156] jepsen worker 15 - jepsen.util 15 :ok :read [5 10] INFO [2022-01-10 12:53:11,156] jepsen worker 52 - jepsen.util 52 :ok :write [17 13] INFO [2022-01-10 12:53:11,157] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,157] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,157] jepsen worker 77 - jepsen.util 167 :ok :read [25 25] INFO [2022-01-10 12:53:11,158] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:11,158] jepsen worker 61 - jepsen.util 61 :ok :read [20 29] INFO [2022-01-10 12:53:11,159] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,159] jepsen worker 15 - jepsen.util 15 :ok :read [5 10] INFO [2022-01-10 12:53:11,159] jepsen worker 79 - jepsen.util 169 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,160] jepsen worker 77 - jepsen.util 167 :ok :read [25 25] INFO [2022-01-10 12:53:11,160] jepsen worker 61 - jepsen.util 61 :invoke :write [20 32] INFO [2022-01-10 12:53:11,160] jepsen worker 79 - jepsen.util 169 :ok :read [26 12] INFO [2022-01-10 12:53:11,161] jepsen worker 77 - jepsen.util 167 :invoke :write [25 13] INFO [2022-01-10 12:53:11,161] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:11,162] jepsen worker 79 - jepsen.util 169 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,162] jepsen worker 52 - jepsen.util 52 :ok :read [17 13] INFO [2022-01-10 12:53:11,163] jepsen worker 15 - jepsen.util 15 :invoke :write [5 36] INFO [2022-01-10 12:53:11,163] jepsen worker 79 - jepsen.util 169 :ok :read [26 12] INFO [2022-01-10 12:53:11,163] jepsen worker 52 - jepsen.util 52 :invoke :write [17 48] INFO [2022-01-10 12:53:11,164] jepsen worker 79 - jepsen.util 169 :invoke :write [26 32] INFO [2022-01-10 12:53:11,165] jepsen worker 71 - jepsen.util 71 :ok :write [23 1] INFO [2022-01-10 12:53:11,165] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,166] jepsen worker 23 - jepsen.util 23 :ok :write [7 35] INFO [2022-01-10 12:53:11,166] jepsen worker 71 - jepsen.util 71 :ok :read [23 1] INFO [2022-01-10 12:53:11,166] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,167] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,167] jepsen worker 23 - jepsen.util 23 :ok :read [7 35] INFO [2022-01-10 12:53:11,167] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,167] jepsen worker 71 - jepsen.util 71 :ok :read [23 1] INFO [2022-01-10 12:53:11,168] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,168] jepsen worker 23 - jepsen.util 23 :ok :read [7 35] INFO [2022-01-10 12:53:11,169] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,169] jepsen worker 71 - jepsen.util 71 :ok :read [23 1] INFO [2022-01-10 12:53:11,169] jepsen worker 71 - jepsen.util 71 :invoke :write [23 2] INFO [2022-01-10 12:53:11,169] jepsen worker 23 - jepsen.util 23 :ok :read [7 35] INFO [2022-01-10 12:53:11,170] jepsen worker 23 - jepsen.util 23 :invoke :write [7 9] INFO [2022-01-10 12:53:11,173] jepsen worker 42 - jepsen.util 132 :ok :write [14 8] INFO [2022-01-10 12:53:11,174] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,175] jepsen worker 42 - jepsen.util 132 :ok :read [14 8] INFO [2022-01-10 12:53:11,175] jepsen worker 42 - jepsen.util 132 :invoke :write [14 5] INFO [2022-01-10 12:53:11,178] jepsen worker 13 - jepsen.util 13 :ok :write [4 44] INFO [2022-01-10 12:53:11,178] jepsen worker 13 - jepsen.util 13 :invoke :write [4 4] INFO [2022-01-10 12:53:11,226] jepsen worker 89 - jepsen.util 89 :info :write [29 33] :RequestTimeoutException INFO [2022-01-10 12:53:11,226] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,227] jepsen worker 89 - jepsen.util 179 :ok :read [29 16] INFO [2022-01-10 12:53:11,228] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,229] jepsen worker 89 - jepsen.util 179 :ok :read [29 16] INFO [2022-01-10 12:53:11,229] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,230] jepsen worker 89 - jepsen.util 179 :ok :read [29 16] INFO [2022-01-10 12:53:11,231] jepsen worker 89 - jepsen.util 179 :invoke :write [29 45] INFO [2022-01-10 12:53:11,249] jepsen worker 37 - jepsen.util 37 :info :write [12 40] :RequestTimeoutException INFO [2022-01-10 12:53:11,249] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,251] jepsen worker 37 - jepsen.util 127 :ok :read [12 45] INFO [2022-01-10 12:53:11,251] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,252] jepsen worker 37 - jepsen.util 127 :ok :read [12 45] INFO [2022-01-10 12:53:11,253] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,254] jepsen worker 37 - jepsen.util 127 :ok :read [12 45] INFO [2022-01-10 12:53:11,254] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,255] jepsen worker 37 - jepsen.util 127 :ok :read [12 45] INFO [2022-01-10 12:53:11,256] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,257] jepsen worker 37 - jepsen.util 127 :ok :read [12 45] INFO [2022-01-10 12:53:11,257] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,258] jepsen worker 37 - jepsen.util 127 :ok :read [12 45] INFO [2022-01-10 12:53:11,258] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,259] jepsen worker 37 - jepsen.util 127 :ok :read [12 45] INFO [2022-01-10 12:53:11,260] jepsen worker 37 - jepsen.util 127 :invoke :write [12 35] INFO [2022-01-10 12:53:11,316] jepsen worker 78 - jepsen.util 78 :ok :write [26 41] INFO [2022-01-10 12:53:11,317] jepsen worker 78 - jepsen.util 78 :invoke :write [26 15] INFO [2022-01-10 12:53:11,324] jepsen worker 52 - jepsen.util 52 :ok :write [17 48] INFO [2022-01-10 12:53:11,324] jepsen worker 52 - jepsen.util 52 :invoke :write [17 41] INFO [2022-01-10 12:53:11,326] jepsen worker 61 - jepsen.util 61 :ok :write [20 32] INFO [2022-01-10 12:53:11,327] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,328] jepsen worker 61 - jepsen.util 61 :ok :read [20 32] INFO [2022-01-10 12:53:11,329] jepsen worker 61 - jepsen.util 61 :invoke :write [20 28] INFO [2022-01-10 12:53:11,330] jepsen worker 71 - jepsen.util 71 :ok :write [23 2] INFO [2022-01-10 12:53:11,330] jepsen worker 71 - jepsen.util 71 :invoke :write [23 0] INFO [2022-01-10 12:53:11,334] jepsen worker 77 - jepsen.util 167 :ok :write [25 13] INFO [2022-01-10 12:53:11,335] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,336] jepsen worker 77 - jepsen.util 167 :ok :read [25 13] INFO [2022-01-10 12:53:11,336] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,337] jepsen worker 77 - jepsen.util 167 :ok :read [25 13] INFO [2022-01-10 12:53:11,338] jepsen worker 22 - jepsen.util 112 :ok :write [7 38] INFO [2022-01-10 12:53:11,338] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,339] jepsen worker 77 - jepsen.util 167 :ok :read [25 13] INFO [2022-01-10 12:53:11,339] jepsen worker 22 - jepsen.util 112 :invoke :write [7 34] INFO [2022-01-10 12:53:11,340] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,340] jepsen worker 13 - jepsen.util 13 :ok :write [4 4] INFO [2022-01-10 12:53:11,340] jepsen worker 77 - jepsen.util 167 :ok :read [25 13] INFO [2022-01-10 12:53:11,341] jepsen worker 13 - jepsen.util 13 :invoke :write [4 38] INFO [2022-01-10 12:53:11,341] jepsen worker 77 - jepsen.util 167 :invoke :write [25 37] INFO [2022-01-10 12:53:11,343] jepsen worker 42 - jepsen.util 132 :ok :write [14 5] INFO [2022-01-10 12:53:11,344] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,345] jepsen worker 42 - jepsen.util 132 :ok :read [14 5] INFO [2022-01-10 12:53:11,345] jepsen worker 42 - jepsen.util 132 :invoke :write [14 32] INFO [2022-01-10 12:53:11,349] jepsen worker 61 - jepsen.util 61 :ok :write [20 28] INFO [2022-01-10 12:53:11,350] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,351] jepsen worker 61 - jepsen.util 61 :ok :read [20 28] INFO [2022-01-10 12:53:11,351] jepsen worker 61 - jepsen.util 61 :invoke :write [20 47] INFO [2022-01-10 12:53:11,352] jepsen worker 87 - jepsen.util 87 :ok :write [29 4] INFO [2022-01-10 12:53:11,352] jepsen worker 78 - jepsen.util 78 :ok :write [26 15] INFO [2022-01-10 12:53:11,352] jepsen worker 3 - jepsen.util 3 :ok :write [1 43] INFO [2022-01-10 12:53:11,353] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,353] jepsen worker 78 - jepsen.util 78 :invoke :write [26 3] INFO [2022-01-10 12:53:11,354] jepsen worker 87 - jepsen.util 87 :ok :read [29 4] INFO [2022-01-10 12:53:11,354] jepsen worker 87 - jepsen.util 87 :invoke :write [29 23] INFO [2022-01-10 12:53:11,355] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,356] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:11,357] jepsen worker 84 - jepsen.util 84 :ok :write [28 33] INFO [2022-01-10 12:53:11,358] jepsen worker 34 - jepsen.util 34 :ok :write [11 8] INFO [2022-01-10 12:53:11,358] jepsen worker 3 - jepsen.util 3 :invoke :write [1 29] INFO [2022-01-10 12:53:11,358] jepsen worker 34 - jepsen.util 34 :invoke :read [11 nil] INFO [2022-01-10 12:53:11,359] jepsen worker 22 - jepsen.util 112 :ok :write [7 34] INFO [2022-01-10 12:53:11,359] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,359] jepsen worker 34 - jepsen.util 34 :ok :read [11 8] INFO [2022-01-10 12:53:11,359] jepsen worker 52 - jepsen.util 52 :ok :write [17 41] INFO [2022-01-10 12:53:11,360] jepsen worker 22 - jepsen.util 112 :invoke :write [7 28] INFO [2022-01-10 12:53:11,360] jepsen worker 84 - jepsen.util 84 :ok :read [28 33] INFO [2022-01-10 12:53:11,361] jepsen worker 34 - jepsen.util 34 :invoke :write [11 20] INFO [2022-01-10 12:53:11,361] jepsen worker 84 - jepsen.util 84 :invoke :write [28 28] INFO [2022-01-10 12:53:11,362] jepsen worker 52 - jepsen.util 52 :invoke :write [17 5] INFO [2022-01-10 12:53:11,365] jepsen worker 42 - jepsen.util 132 :ok :write [14 32] INFO [2022-01-10 12:53:11,365] jepsen worker 71 - jepsen.util 71 :ok :write [23 0] INFO [2022-01-10 12:53:11,365] jepsen worker 50 - jepsen.util 50 :ok :write [16 39] INFO [2022-01-10 12:53:11,367] jepsen worker 42 - jepsen.util 132 :invoke :write [14 1] INFO [2022-01-10 12:53:11,367] jepsen worker 71 - jepsen.util 71 :invoke :write [23 35] INFO [2022-01-10 12:53:11,368] jepsen worker 36 - jepsen.util 36 :ok :write [12 12] INFO [2022-01-10 12:53:11,368] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:11,369] jepsen worker 36 - jepsen.util 36 :invoke :write [12 21] INFO [2022-01-10 12:53:11,369] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:11,370] jepsen worker 50 - jepsen.util 50 :invoke :write [16 29] INFO [2022-01-10 12:53:11,371] jepsen worker 13 - jepsen.util 13 :ok :write [4 38] INFO [2022-01-10 12:53:11,372] jepsen worker 13 - jepsen.util 13 :invoke :write [4 29] INFO [2022-01-10 12:53:11,373] jepsen worker 77 - jepsen.util 167 :ok :write [25 37] INFO [2022-01-10 12:53:11,374] jepsen worker 77 - jepsen.util 167 :invoke :write [25 46] INFO [2022-01-10 12:53:11,375] jepsen worker 81 - jepsen.util 81 :ok :write [27 6] INFO [2022-01-10 12:53:11,376] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:11,377] jepsen worker 81 - jepsen.util 81 :ok :read [27 6] INFO [2022-01-10 12:53:11,378] jepsen worker 81 - jepsen.util 81 :invoke :write [27 8] INFO [2022-01-10 12:53:11,379] jepsen worker 61 - jepsen.util 61 :ok :write [20 47] INFO [2022-01-10 12:53:11,379] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,380] jepsen worker 61 - jepsen.util 61 :ok :read [20 47] INFO [2022-01-10 12:53:11,381] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,382] jepsen worker 61 - jepsen.util 61 :ok :read [20 47] INFO [2022-01-10 12:53:11,382] jepsen worker 78 - jepsen.util 78 :ok :write [26 3] INFO [2022-01-10 12:53:11,383] jepsen worker 40 - jepsen.util 40 :ok :write [13 22] INFO [2022-01-10 12:53:11,383] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,384] jepsen worker 78 - jepsen.util 78 :ok :read [26 3] INFO [2022-01-10 12:53:11,384] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,385] jepsen worker 78 - jepsen.util 78 :invoke :write [26 25] INFO [2022-01-10 12:53:11,385] jepsen worker 22 - jepsen.util 112 :ok :write [7 28] INFO [2022-01-10 12:53:11,385] jepsen worker 61 - jepsen.util 61 :ok :read [20 47] INFO [2022-01-10 12:53:11,386] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:11,386] jepsen worker 40 - jepsen.util 40 :ok :read [13 22] INFO [2022-01-10 12:53:11,387] jepsen worker 61 - jepsen.util 61 :invoke :write [20 4] INFO [2022-01-10 12:53:11,387] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:11,388] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,389] jepsen worker 40 - jepsen.util 40 :ok :read [13 22] INFO [2022-01-10 12:53:11,389] jepsen worker 22 - jepsen.util 112 :ok :read [7 28] INFO [2022-01-10 12:53:11,389] jepsen worker 22 - jepsen.util 112 :invoke :write [7 15] INFO [2022-01-10 12:53:11,390] jepsen worker 40 - jepsen.util 40 :invoke :write [13 2] INFO [2022-01-10 12:53:11,391] jepsen worker 4 - jepsen.util 4 :ok :write [1 49] INFO [2022-01-10 12:53:11,391] jepsen worker 1 - jepsen.util 1 :ok :write [0 6] INFO [2022-01-10 12:53:11,392] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,392] jepsen worker 42 - jepsen.util 132 :ok :write [14 1] INFO [2022-01-10 12:53:11,393] jepsen worker 1 - jepsen.util 1 :invoke :write [0 37] INFO [2022-01-10 12:53:11,393] jepsen worker 4 - jepsen.util 4 :ok :read [1 49] INFO [2022-01-10 12:53:11,393] jepsen worker 42 - jepsen.util 132 :invoke :write [14 42] INFO [2022-01-10 12:53:11,394] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,395] jepsen worker 4 - jepsen.util 4 :ok :read [1 49] INFO [2022-01-10 12:53:11,396] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,397] jepsen worker 4 - jepsen.util 4 :ok :read [1 49] INFO [2022-01-10 12:53:11,397] jepsen worker 84 - jepsen.util 84 :ok :write [28 28] INFO [2022-01-10 12:53:11,397] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,398] jepsen worker 84 - jepsen.util 84 :invoke :write [28 46] INFO [2022-01-10 12:53:11,398] jepsen worker 4 - jepsen.util 4 :ok :read [1 49] INFO [2022-01-10 12:53:11,400] jepsen worker 85 - jepsen.util 85 :info :write [28 15] :RequestTimeoutException INFO [2022-01-10 12:53:11,400] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,401] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,401] jepsen worker 4 - jepsen.util 4 :ok :read [1 49] INFO [2022-01-10 12:53:11,401] jepsen worker 77 - jepsen.util 167 :ok :write [25 46] INFO [2022-01-10 12:53:11,402] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,402] jepsen worker 67 - jepsen.util 67 :ok :write [22 34] INFO [2022-01-10 12:53:11,402] jepsen worker 85 - jepsen.util 175 :ok :read [28 28] INFO [2022-01-10 12:53:11,403] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,403] jepsen worker 77 - jepsen.util 167 :ok :read [25 46] INFO [2022-01-10 12:53:11,404] jepsen worker 67 - jepsen.util 67 :invoke :write [22 16] INFO [2022-01-10 12:53:11,405] jepsen worker 85 - jepsen.util 175 :invoke :write [28 19] INFO [2022-01-10 12:53:11,405] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,406] jepsen worker 52 - jepsen.util 52 :ok :write [17 5] INFO [2022-01-10 12:53:11,406] jepsen worker 77 - jepsen.util 167 :ok :read [25 46] INFO [2022-01-10 12:53:11,406] jepsen worker 4 - jepsen.util 4 :ok :read [1 49] INFO [2022-01-10 12:53:11,406] jepsen worker 52 - jepsen.util 52 :invoke :write [17 7] INFO [2022-01-10 12:53:11,407] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,407] jepsen worker 77 - jepsen.util 167 :invoke :write [25 19] INFO [2022-01-10 12:53:11,408] jepsen worker 4 - jepsen.util 4 :ok :read [1 49] INFO [2022-01-10 12:53:11,408] jepsen worker 4 - jepsen.util 4 :invoke :write [1 18] INFO [2022-01-10 12:53:11,410] jepsen worker 71 - jepsen.util 71 :ok :write [23 35] INFO [2022-01-10 12:53:11,411] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,412] jepsen worker 71 - jepsen.util 71 :ok :read [23 35] INFO [2022-01-10 12:53:11,412] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,413] jepsen worker 71 - jepsen.util 71 :ok :read [23 35] INFO [2022-01-10 12:53:11,414] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,414] jepsen worker 71 - jepsen.util 71 :ok :read [23 35] INFO [2022-01-10 12:53:11,415] jepsen worker 71 - jepsen.util 71 :invoke :write [23 4] INFO [2022-01-10 12:53:11,416] jepsen worker 61 - jepsen.util 61 :ok :write [20 4] INFO [2022-01-10 12:53:11,416] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,417] jepsen worker 61 - jepsen.util 61 :ok :read [20 4] INFO [2022-01-10 12:53:11,418] jepsen worker 61 - jepsen.util 61 :invoke :write [20 18] INFO [2022-01-10 12:53:11,420] jepsen worker 36 - jepsen.util 36 :ok :write [12 21] INFO [2022-01-10 12:53:11,420] jepsen worker 36 - jepsen.util 36 :invoke :write [12 44] INFO [2022-01-10 12:53:11,422] jepsen worker 22 - jepsen.util 112 :ok :write [7 15] INFO [2022-01-10 12:53:11,423] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,424] jepsen worker 22 - jepsen.util 112 :ok :read [7 15] INFO [2022-01-10 12:53:11,425] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,425] jepsen worker 22 - jepsen.util 112 :ok :read [7 15] INFO [2022-01-10 12:53:11,426] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,427] jepsen worker 22 - jepsen.util 112 :ok :read [7 15] INFO [2022-01-10 12:53:11,427] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,428] jepsen worker 42 - jepsen.util 132 :ok :write [14 42] INFO [2022-01-10 12:53:11,428] jepsen worker 22 - jepsen.util 112 :ok :read [7 15] INFO [2022-01-10 12:53:11,428] jepsen worker 13 - jepsen.util 13 :ok :write [4 29] INFO [2022-01-10 12:53:11,428] jepsen worker 42 - jepsen.util 132 :invoke :write [14 14] INFO [2022-01-10 12:53:11,429] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,429] jepsen worker 13 - jepsen.util 13 :invoke :write [4 22] INFO [2022-01-10 12:53:11,430] jepsen worker 22 - jepsen.util 112 :ok :read [7 15] INFO [2022-01-10 12:53:11,430] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,431] jepsen worker 22 - jepsen.util 112 :ok :read [7 15] INFO [2022-01-10 12:53:11,432] jepsen worker 22 - jepsen.util 112 :invoke :write [7 37] INFO [2022-01-10 12:53:11,435] jepsen worker 77 - jepsen.util 167 :ok :write [25 19] INFO [2022-01-10 12:53:11,435] jepsen worker 77 - jepsen.util 167 :invoke :write [25 15] INFO [2022-01-10 12:53:11,437] jepsen worker 78 - jepsen.util 78 :ok :write [26 25] INFO [2022-01-10 12:53:11,437] jepsen worker 78 - jepsen.util 78 :invoke :write [26 36] INFO [2022-01-10 12:53:11,447] jepsen worker 61 - jepsen.util 61 :ok :write [20 18] INFO [2022-01-10 12:53:11,447] jepsen worker 61 - jepsen.util 61 :invoke :write [20 17] INFO [2022-01-10 12:53:11,450] jepsen worker 22 - jepsen.util 112 :ok :write [7 37] INFO [2022-01-10 12:53:11,451] jepsen worker 22 - jepsen.util 112 :invoke :write [7 14] INFO [2022-01-10 12:53:11,453] jepsen worker 42 - jepsen.util 132 :ok :write [14 14] INFO [2022-01-10 12:53:11,454] jepsen worker 42 - jepsen.util 132 :invoke :write [14 15] INFO [2022-01-10 12:53:11,457] jepsen worker 52 - jepsen.util 52 :ok :write [17 7] INFO [2022-01-10 12:53:11,458] jepsen worker 77 - jepsen.util 167 :ok :write [25 15] INFO [2022-01-10 12:53:11,458] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:11,458] jepsen worker 77 - jepsen.util 167 :invoke :write [25 8] INFO [2022-01-10 12:53:11,459] jepsen worker 52 - jepsen.util 52 :ok :read [17 7] INFO [2022-01-10 12:53:11,459] jepsen worker 52 - jepsen.util 52 :invoke :write [17 1] INFO [2022-01-10 12:53:11,463] jepsen worker 71 - jepsen.util 71 :ok :write [23 4] INFO [2022-01-10 12:53:11,463] jepsen worker 61 - jepsen.util 61 :ok :write [20 17] INFO [2022-01-10 12:53:11,464] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,464] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,465] jepsen worker 71 - jepsen.util 71 :ok :read [23 4] INFO [2022-01-10 12:53:11,465] jepsen worker 61 - jepsen.util 61 :ok :read [20 17] INFO [2022-01-10 12:53:11,465] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,466] jepsen worker 61 - jepsen.util 61 :invoke :write [20 3] INFO [2022-01-10 12:53:11,466] jepsen worker 71 - jepsen.util 71 :ok :read [23 4] INFO [2022-01-10 12:53:11,467] jepsen worker 71 - jepsen.util 71 :invoke :write [23 31] INFO [2022-01-10 12:53:11,469] jepsen worker 22 - jepsen.util 112 :ok :write [7 14] INFO [2022-01-10 12:53:11,470] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,471] jepsen worker 22 - jepsen.util 112 :ok :read [7 14] INFO [2022-01-10 12:53:11,472] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,472] jepsen worker 22 - jepsen.util 112 :ok :read [7 14] INFO [2022-01-10 12:53:11,473] jepsen worker 22 - jepsen.util 112 :invoke :write [7 22] INFO [2022-01-10 12:53:11,474] jepsen worker 44 - jepsen.util 44 :ok :write [14 4] INFO [2022-01-10 12:53:11,475] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,476] jepsen worker 44 - jepsen.util 44 :ok :read [14 4] INFO [2022-01-10 12:53:11,476] jepsen worker 13 - jepsen.util 13 :ok :write [4 22] INFO [2022-01-10 12:53:11,476] jepsen worker 44 - jepsen.util 44 :invoke :write [14 35] INFO [2022-01-10 12:53:11,477] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:11,478] jepsen worker 13 - jepsen.util 13 :ok :read [4 22] INFO [2022-01-10 12:53:11,479] jepsen worker 13 - jepsen.util 13 :invoke :write [4 42] INFO [2022-01-10 12:53:11,479] jepsen worker 77 - jepsen.util 167 :ok :write [25 8] INFO [2022-01-10 12:53:11,480] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,481] jepsen worker 77 - jepsen.util 167 :ok :read [25 8] INFO [2022-01-10 12:53:11,481] jepsen worker 77 - jepsen.util 167 :invoke :write [25 16] INFO [2022-01-10 12:53:11,482] jepsen worker 78 - jepsen.util 78 :ok :write [26 36] INFO [2022-01-10 12:53:11,483] jepsen worker 78 - jepsen.util 78 :invoke :write [26 4] INFO [2022-01-10 12:53:11,485] jepsen worker 61 - jepsen.util 61 :ok :write [20 3] INFO [2022-01-10 12:53:11,486] jepsen worker 61 - jepsen.util 61 :invoke :write [20 10] INFO [2022-01-10 12:53:11,489] jepsen worker 52 - jepsen.util 52 :ok :write [17 1] INFO [2022-01-10 12:53:11,490] jepsen worker 22 - jepsen.util 112 :ok :write [7 22] INFO [2022-01-10 12:53:11,490] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:11,491] jepsen worker 22 - jepsen.util 112 :invoke :write [7 5] INFO [2022-01-10 12:53:11,495] jepsen worker 44 - jepsen.util 44 :ok :write [14 35] INFO [2022-01-10 12:53:11,496] jepsen worker 44 - jepsen.util 44 :invoke :write [14 44] INFO [2022-01-10 12:53:11,498] jepsen worker 71 - jepsen.util 71 :ok :write [23 31] INFO [2022-01-10 12:53:11,498] jepsen worker 49 - jepsen.util 49 :info :write [16 12] :RequestTimeoutException INFO [2022-01-10 12:53:11,499] jepsen worker 77 - jepsen.util 167 :ok :write [25 16] INFO [2022-01-10 12:53:11,499] jepsen worker 71 - jepsen.util 71 :invoke :write [23 22] INFO [2022-01-10 12:53:11,500] jepsen worker 77 - jepsen.util 167 :invoke :write [25 11] INFO [2022-01-10 12:53:11,500] jepsen worker 49 - jepsen.util 139 :invoke :write [16 9] INFO [2022-01-10 12:53:11,505] jepsen worker 13 - jepsen.util 13 :ok :write [4 42] INFO [2022-01-10 12:53:11,505] jepsen worker 13 - jepsen.util 13 :invoke :write [4 7] INFO [2022-01-10 12:53:11,506] jepsen worker 61 - jepsen.util 61 :ok :write [20 10] INFO [2022-01-10 12:53:11,506] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,507] jepsen worker 61 - jepsen.util 61 :ok :read [20 10] INFO [2022-01-10 12:53:11,508] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,508] jepsen worker 61 - jepsen.util 61 :ok :read [20 10] INFO [2022-01-10 12:53:11,509] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,509] jepsen worker 61 - jepsen.util 61 :ok :read [20 10] INFO [2022-01-10 12:53:11,509] jepsen worker 78 - jepsen.util 78 :ok :write [26 4] INFO [2022-01-10 12:53:11,510] jepsen worker 61 - jepsen.util 61 :invoke :write [20 38] INFO [2022-01-10 12:53:11,511] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,511] jepsen worker 22 - jepsen.util 112 :ok :write [7 5] INFO [2022-01-10 12:53:11,511] jepsen worker 78 - jepsen.util 78 :ok :read [26 4] INFO [2022-01-10 12:53:11,512] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,512] jepsen worker 78 - jepsen.util 78 :invoke :write [26 24] INFO [2022-01-10 12:53:11,512] jepsen worker 22 - jepsen.util 112 :ok :read [7 5] INFO [2022-01-10 12:53:11,513] jepsen worker 22 - jepsen.util 112 :invoke :write [7 48] INFO [2022-01-10 12:53:11,514] jepsen worker 44 - jepsen.util 44 :ok :write [14 44] INFO [2022-01-10 12:53:11,515] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,516] jepsen worker 44 - jepsen.util 44 :ok :read [14 44] INFO [2022-01-10 12:53:11,517] jepsen worker 44 - jepsen.util 44 :invoke :write [14 12] INFO [2022-01-10 12:53:11,521] jepsen worker 71 - jepsen.util 71 :ok :write [23 22] INFO [2022-01-10 12:53:11,522] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,523] jepsen worker 71 - jepsen.util 71 :ok :read [23 22] INFO [2022-01-10 12:53:11,523] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,524] jepsen worker 71 - jepsen.util 71 :ok :read [23 22] INFO [2022-01-10 12:53:11,525] jepsen worker 71 - jepsen.util 71 :invoke :write [23 20] INFO [2022-01-10 12:53:11,554] jepsen worker 2 - jepsen.util 92 :ok :read [0 6] INFO [2022-01-10 12:53:11,555] jepsen worker 58 - jepsen.util 58 :info :write [19 45] :SyncWriteAmbiguous INFO [2022-01-10 12:53:11,556] jepsen worker 58 - jepsen.util 148 :invoke :write [19 26] INFO [2022-01-10 12:53:11,557] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:11,557] jepsen worker 63 - jepsen.util 153 :ok :read [21 19] INFO [2022-01-10 12:53:11,557] jepsen worker 11 - jepsen.util 11 :info :write [3 41] :SyncWriteAmbiguous INFO [2022-01-10 12:53:11,558] jepsen worker 63 - jepsen.util 153 :invoke :write [21 4] INFO [2022-01-10 12:53:11,558] jepsen worker 2 - jepsen.util 92 :ok :read [0 6] INFO [2022-01-10 12:53:11,559] jepsen worker 11 - jepsen.util 101 :invoke :read [3 nil] INFO [2022-01-10 12:53:11,559] jepsen worker 2 - jepsen.util 92 :invoke :write [0 26] INFO [2022-01-10 12:53:11,559] jepsen worker 11 - jepsen.util 101 :ok :read [3 30] INFO [2022-01-10 12:53:11,560] jepsen worker 11 - jepsen.util 101 :invoke :read [3 nil] INFO [2022-01-10 12:53:11,561] jepsen worker 11 - jepsen.util 101 :ok :read [3 30] INFO [2022-01-10 12:53:11,561] jepsen worker 11 - jepsen.util 101 :invoke :write [3 33] INFO [2022-01-10 12:53:11,569] jepsen worker 3 - jepsen.util 3 :ok :write [1 29] INFO [2022-01-10 12:53:11,570] jepsen worker 7 - jepsen.util 7 :ok :write [2 4] INFO [2022-01-10 12:53:11,570] jepsen worker 3 - jepsen.util 3 :invoke :write [1 17] INFO [2022-01-10 12:53:11,571] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:11,572] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:11,572] jepsen worker 7 - jepsen.util 7 :invoke :write [2 18] INFO [2022-01-10 12:53:11,574] jepsen worker 15 - jepsen.util 15 :ok :write [5 36] INFO [2022-01-10 12:53:11,575] jepsen worker 15 - jepsen.util 15 :invoke :write [5 44] INFO [2022-01-10 12:53:11,576] jepsen worker 84 - jepsen.util 84 :ok :write [28 46] INFO [2022-01-10 12:53:11,577] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,579] jepsen worker 84 - jepsen.util 84 :ok :read [28 46] INFO [2022-01-10 12:53:11,579] jepsen worker 84 - jepsen.util 84 :invoke :write [28 43] INFO [2022-01-10 12:53:11,581] jepsen worker 87 - jepsen.util 87 :ok :write [29 23] INFO [2022-01-10 12:53:11,582] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,582] jepsen worker 36 - jepsen.util 36 :ok :write [12 44] INFO [2022-01-10 12:53:11,583] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,585] jepsen worker 34 - jepsen.util 34 :ok :write [11 20] INFO [2022-01-10 12:53:11,586] jepsen worker 52 - jepsen.util 52 :ok :read [17 1] INFO [2022-01-10 12:53:11,586] jepsen worker 34 - jepsen.util 34 :invoke :write [11 24] INFO [2022-01-10 12:53:11,586] jepsen worker 36 - jepsen.util 36 :ok :read [12 44] INFO [2022-01-10 12:53:11,586] jepsen worker 87 - jepsen.util 87 :ok :read [29 23] INFO [2022-01-10 12:53:11,587] jepsen worker 52 - jepsen.util 52 :invoke :write [17 31] INFO [2022-01-10 12:53:11,588] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,588] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,590] jepsen worker 30 - jepsen.util 30 :info :write [10 43] :SyncWriteAmbiguous INFO [2022-01-10 12:53:11,590] jepsen worker 9 - jepsen.util 99 :ok :read [3 30] INFO [2022-01-10 12:53:11,591] jepsen worker 36 - jepsen.util 36 :ok :read [12 44] INFO [2022-01-10 12:53:11,591] jepsen worker 87 - jepsen.util 87 :ok :read [29 23] INFO [2022-01-10 12:53:11,591] jepsen worker 30 - jepsen.util 120 :invoke :read [10 nil] INFO [2022-01-10 12:53:11,591] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,592] jepsen worker 9 - jepsen.util 99 :invoke :read [3 nil] INFO [2022-01-10 12:53:11,592] jepsen worker 87 - jepsen.util 87 :invoke :write [29 6] INFO [2022-01-10 12:53:11,592] jepsen worker 36 - jepsen.util 36 :ok :read [12 44] INFO [2022-01-10 12:53:11,593] jepsen worker 9 - jepsen.util 99 :ok :read [3 30] INFO [2022-01-10 12:53:11,593] jepsen worker 36 - jepsen.util 36 :invoke :write [12 20] INFO [2022-01-10 12:53:11,593] jepsen worker 9 - jepsen.util 99 :invoke :write [3 29] INFO [2022-01-10 12:53:11,596] jepsen worker 81 - jepsen.util 81 :ok :write [27 8] INFO [2022-01-10 12:53:11,597] jepsen worker 81 - jepsen.util 81 :invoke :read [27 nil] INFO [2022-01-10 12:53:11,598] jepsen worker 81 - jepsen.util 81 :ok :read [27 8] INFO [2022-01-10 12:53:11,599] jepsen worker 81 - jepsen.util 81 :invoke :write [27 2] INFO [2022-01-10 12:53:11,602] jepsen worker 40 - jepsen.util 40 :ok :write [13 2] INFO [2022-01-10 12:53:11,604] jepsen worker 18 - jepsen.util 18 :info :write [6 38] :SyncWriteAmbiguous INFO [2022-01-10 12:53:11,604] jepsen worker 65 - jepsen.util 65 :info :write [21 4] :SyncWriteAmbiguous INFO [2022-01-10 12:53:11,605] jepsen worker 35 - jepsen.util 125 :ok :read [11 20] INFO [2022-01-10 12:53:11,605] jepsen worker 40 - jepsen.util 40 :invoke :write [13 40] INFO [2022-01-10 12:53:11,607] jepsen worker 26 - jepsen.util 26 :info :write [8 40] :SyncWriteAmbiguous INFO [2022-01-10 12:53:11,607] jepsen worker 1 - jepsen.util 1 :ok :write [0 37] INFO [2022-01-10 12:53:11,607] jepsen worker 18 - jepsen.util 108 :invoke :write [6 16] INFO [2022-01-10 12:53:11,609] jepsen worker 26 - jepsen.util 116 :invoke :write [8 8] INFO [2022-01-10 12:53:11,609] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:11,610] jepsen worker 65 - jepsen.util 155 :invoke :write [21 26] INFO [2022-01-10 12:53:11,610] jepsen worker 35 - jepsen.util 125 :invoke :write [11 48] INFO [2022-01-10 12:53:11,611] jepsen worker 1 - jepsen.util 1 :ok :read [0 37] INFO [2022-01-10 12:53:11,611] jepsen worker 1 - jepsen.util 1 :invoke :write [0 13] INFO [2022-01-10 12:53:11,613] jepsen worker 67 - jepsen.util 67 :ok :write [22 16] INFO [2022-01-10 12:53:11,614] jepsen worker 67 - jepsen.util 67 :invoke :write [22 25] INFO [2022-01-10 12:53:11,615] jepsen worker 27 - jepsen.util 27 :info :write [9 26] :RequestTimeoutException INFO [2022-01-10 12:53:11,616] jepsen worker 27 - jepsen.util 117 :invoke :read [9 nil] INFO [2022-01-10 12:53:11,617] jepsen worker 27 - jepsen.util 117 :ok :read [9 1] INFO [2022-01-10 12:53:11,617] jepsen worker 27 - jepsen.util 117 :invoke :read [9 nil] INFO [2022-01-10 12:53:11,618] jepsen worker 27 - jepsen.util 117 :ok :read [9 1] INFO [2022-01-10 12:53:11,618] jepsen worker 27 - jepsen.util 117 :invoke :read [9 nil] INFO [2022-01-10 12:53:11,619] jepsen worker 27 - jepsen.util 117 :ok :read [9 1] INFO [2022-01-10 12:53:11,620] jepsen worker 27 - jepsen.util 117 :invoke :read [9 nil] INFO [2022-01-10 12:53:11,620] jepsen worker 27 - jepsen.util 117 :ok :read [9 1] INFO [2022-01-10 12:53:11,622] jepsen worker 27 - jepsen.util 117 :invoke :write [9 22] INFO [2022-01-10 12:53:11,622] jepsen worker 47 - jepsen.util 47 :info :write [15 45] :SyncWriteAmbiguous INFO [2022-01-10 12:53:11,622] jepsen worker 47 - jepsen.util 137 :invoke :read [15 nil] INFO [2022-01-10 12:53:11,624] jepsen worker 47 - jepsen.util 137 :ok :read [15 10] INFO [2022-01-10 12:53:11,624] jepsen worker 47 - jepsen.util 137 :invoke :write [15 22] INFO [2022-01-10 12:53:11,629] jepsen worker 56 - jepsen.util 146 :ok :read [18 46] INFO [2022-01-10 12:53:11,630] jepsen worker 55 - jepsen.util 55 :info :write [18 12] :SyncWriteAmbiguous INFO [2022-01-10 12:53:11,631] jepsen worker 56 - jepsen.util 146 :invoke :write [18 3] INFO [2022-01-10 12:53:11,632] jepsen worker 55 - jepsen.util 145 :invoke :read [18 nil] INFO [2022-01-10 12:53:11,633] jepsen worker 55 - jepsen.util 145 :ok :read [18 46] INFO [2022-01-10 12:53:11,633] jepsen worker 55 - jepsen.util 145 :invoke :read [18 nil] INFO [2022-01-10 12:53:11,634] jepsen worker 55 - jepsen.util 145 :ok :read [18 46] INFO [2022-01-10 12:53:11,635] jepsen worker 55 - jepsen.util 145 :invoke :write [18 43] INFO [2022-01-10 12:53:11,653] jepsen worker 74 - jepsen.util 74 :info :write [24 34] :SyncWriteAmbiguous INFO [2022-01-10 12:53:11,654] jepsen worker 74 - jepsen.util 164 :invoke :read [24 nil] INFO [2022-01-10 12:53:11,655] jepsen worker 20 - jepsen.util 20 :info :write [6 30] :RequestTimeoutException INFO [2022-01-10 12:53:11,656] jepsen worker 74 - jepsen.util 164 :ok :read [24 37] INFO [2022-01-10 12:53:11,656] jepsen worker 20 - jepsen.util 110 :invoke :read [6 nil] INFO [2022-01-10 12:53:11,657] jepsen worker 74 - jepsen.util 164 :invoke :read [24 nil] INFO [2022-01-10 12:53:11,657] jepsen worker 20 - jepsen.util 110 :ok :read [6 8] INFO [2022-01-10 12:53:11,657] jepsen worker 74 - jepsen.util 164 :ok :read [24 37] INFO [2022-01-10 12:53:11,658] jepsen worker 74 - jepsen.util 164 :invoke :write [24 20] INFO [2022-01-10 12:53:11,659] jepsen worker 20 - jepsen.util 110 :invoke :read [6 nil] INFO [2022-01-10 12:53:11,660] jepsen worker 20 - jepsen.util 110 :ok :read [6 8] INFO [2022-01-10 12:53:11,660] jepsen worker 20 - jepsen.util 110 :invoke :read [6 nil] INFO [2022-01-10 12:53:11,661] jepsen worker 20 - jepsen.util 110 :ok :read [6 8] INFO [2022-01-10 12:53:11,661] jepsen worker 20 - jepsen.util 110 :invoke :write [6 38] INFO [2022-01-10 12:53:11,683] jepsen worker 77 - jepsen.util 167 :ok :write [25 11] INFO [2022-01-10 12:53:11,683] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,684] jepsen worker 77 - jepsen.util 167 :ok :read [25 11] INFO [2022-01-10 12:53:11,685] jepsen worker 77 - jepsen.util 167 :invoke :write [25 16] INFO [2022-01-10 12:53:11,686] jepsen worker 61 - jepsen.util 61 :ok :write [20 38] INFO [2022-01-10 12:53:11,687] jepsen worker 61 - jepsen.util 61 :invoke :write [20 6] INFO [2022-01-10 12:53:11,688] jepsen worker 78 - jepsen.util 78 :ok :write [26 24] INFO [2022-01-10 12:53:11,688] jepsen worker 78 - jepsen.util 78 :invoke :write [26 25] INFO [2022-01-10 12:53:11,691] jepsen worker 71 - jepsen.util 71 :ok :write [23 20] INFO [2022-01-10 12:53:11,691] jepsen worker 71 - jepsen.util 71 :invoke :write [23 47] INFO [2022-01-10 12:53:11,694] jepsen worker 22 - jepsen.util 112 :ok :write [7 48] INFO [2022-01-10 12:53:11,695] jepsen worker 22 - jepsen.util 112 :invoke :write [7 39] INFO [2022-01-10 12:53:11,702] jepsen worker 44 - jepsen.util 44 :ok :write [14 12] INFO [2022-01-10 12:53:11,702] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,703] jepsen worker 44 - jepsen.util 44 :ok :read [14 12] INFO [2022-01-10 12:53:11,704] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,704] jepsen worker 44 - jepsen.util 44 :ok :read [14 12] INFO [2022-01-10 12:53:11,705] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,705] jepsen worker 44 - jepsen.util 44 :ok :read [14 12] INFO [2022-01-10 12:53:11,706] jepsen worker 44 - jepsen.util 44 :invoke :write [14 9] INFO [2022-01-10 12:53:11,708] jepsen worker 7 - jepsen.util 7 :ok :write [2 18] INFO [2022-01-10 12:53:11,709] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:11,710] jepsen worker 7 - jepsen.util 7 :ok :read [2 18] INFO [2022-01-10 12:53:11,711] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:11,712] jepsen worker 7 - jepsen.util 7 :ok :read [2 18] INFO [2022-01-10 12:53:11,712] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:11,713] jepsen worker 7 - jepsen.util 7 :ok :read [2 18] INFO [2022-01-10 12:53:11,713] jepsen worker 7 - jepsen.util 7 :invoke :write [2 13] INFO [2022-01-10 12:53:11,714] jepsen worker 15 - jepsen.util 15 :ok :write [5 44] INFO [2022-01-10 12:53:11,715] jepsen worker 15 - jepsen.util 15 :invoke :write [5 46] INFO [2022-01-10 12:53:11,718] jepsen worker 35 - jepsen.util 125 :ok :write [11 48] INFO [2022-01-10 12:53:11,718] jepsen worker 87 - jepsen.util 87 :ok :write [29 6] INFO [2022-01-10 12:53:11,719] jepsen worker 35 - jepsen.util 125 :invoke :write [11 44] INFO [2022-01-10 12:53:11,719] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,720] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:11,721] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,722] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:11,722] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,723] jepsen worker 49 - jepsen.util 139 :ok :write [16 9] INFO [2022-01-10 12:53:11,723] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:11,724] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,725] jepsen worker 3 - jepsen.util 3 :ok :write [1 17] INFO [2022-01-10 12:53:11,725] jepsen worker 49 - jepsen.util 139 :invoke :write [16 17] INFO [2022-01-10 12:53:11,725] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:11,725] jepsen worker 40 - jepsen.util 40 :ok :write [13 40] INFO [2022-01-10 12:53:11,726] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,726] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,727] jepsen worker 3 - jepsen.util 3 :ok :read [1 17] INFO [2022-01-10 12:53:11,727] jepsen worker 40 - jepsen.util 40 :invoke :read [13 nil] INFO [2022-01-10 12:53:11,727] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:11,728] jepsen worker 3 - jepsen.util 3 :invoke :write [1 16] INFO [2022-01-10 12:53:11,728] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,728] jepsen worker 40 - jepsen.util 40 :ok :read [13 40] INFO [2022-01-10 12:53:11,729] jepsen worker 40 - jepsen.util 40 :invoke :write [13 14] INFO [2022-01-10 12:53:11,729] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:11,730] jepsen worker 87 - jepsen.util 87 :invoke :write [29 21] INFO [2022-01-10 12:53:11,730] jepsen worker 83 - jepsen.util 173 :ok :write [27 41] INFO [2022-01-10 12:53:11,731] jepsen worker 83 - jepsen.util 173 :invoke :write [27 44] INFO [2022-01-10 12:53:11,732] jepsen worker 1 - jepsen.util 1 :ok :write [0 13] INFO [2022-01-10 12:53:11,732] jepsen worker 1 - jepsen.util 1 :invoke :write [0 13] INFO [2022-01-10 12:53:11,734] jepsen worker 84 - jepsen.util 84 :ok :write [28 43] INFO [2022-01-10 12:53:11,735] jepsen worker 84 - jepsen.util 84 :invoke :write [28 20] INFO [2022-01-10 12:53:11,736] jepsen worker 67 - jepsen.util 67 :ok :write [22 25] INFO [2022-01-10 12:53:11,737] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:11,738] jepsen worker 67 - jepsen.util 67 :ok :read [22 25] INFO [2022-01-10 12:53:11,738] jepsen worker 67 - jepsen.util 67 :invoke :write [22 27] INFO [2022-01-10 12:53:11,739] jepsen worker 36 - jepsen.util 36 :ok :write [12 20] INFO [2022-01-10 12:53:11,739] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,740] jepsen worker 36 - jepsen.util 36 :ok :read [12 20] INFO [2022-01-10 12:53:11,740] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,741] jepsen worker 36 - jepsen.util 36 :ok :read [12 20] INFO [2022-01-10 12:53:11,742] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,743] jepsen worker 36 - jepsen.util 36 :ok :read [12 20] INFO [2022-01-10 12:53:11,743] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,744] jepsen worker 36 - jepsen.util 36 :ok :read [12 20] INFO [2022-01-10 12:53:11,745] jepsen worker 30 - jepsen.util 120 :ok :read [10 38] INFO [2022-01-10 12:53:11,746] jepsen worker 13 - jepsen.util 13 :ok :write [4 7] INFO [2022-01-10 12:53:11,747] jepsen worker 29 - jepsen.util 29 :info :write [9 47] :SyncWriteAmbiguous INFO [2022-01-10 12:53:11,747] jepsen worker 77 - jepsen.util 167 :ok :write [25 16] INFO [2022-01-10 12:53:11,747] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,748] jepsen worker 36 - jepsen.util 36 :ok :read [12 20] INFO [2022-01-10 12:53:11,748] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,749] jepsen worker 29 - jepsen.util 119 :invoke :write [9 23] INFO [2022-01-10 12:53:11,749] jepsen worker 77 - jepsen.util 167 :ok :read [25 16] INFO [2022-01-10 12:53:11,750] jepsen worker 13 - jepsen.util 13 :invoke :write [4 29] INFO [2022-01-10 12:53:11,750] jepsen worker 30 - jepsen.util 120 :invoke :write [10 22] INFO [2022-01-10 12:53:11,751] jepsen worker 36 - jepsen.util 36 :invoke :write [12 22] INFO [2022-01-10 12:53:11,751] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,752] jepsen worker 77 - jepsen.util 167 :ok :read [25 16] INFO [2022-01-10 12:53:11,753] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,753] jepsen worker 61 - jepsen.util 61 :ok :write [20 6] INFO [2022-01-10 12:53:11,753] jepsen worker 52 - jepsen.util 52 :ok :write [17 31] INFO [2022-01-10 12:53:11,753] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,754] jepsen worker 77 - jepsen.util 167 :ok :read [25 16] INFO [2022-01-10 12:53:11,754] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:11,755] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,755] jepsen worker 61 - jepsen.util 61 :ok :read [20 6] INFO [2022-01-10 12:53:11,755] jepsen worker 52 - jepsen.util 52 :ok :read [17 31] INFO [2022-01-10 12:53:11,755] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,756] jepsen worker 77 - jepsen.util 167 :ok :read [25 16] INFO [2022-01-10 12:53:11,756] jepsen worker 52 - jepsen.util 52 :invoke :write [17 38] INFO [2022-01-10 12:53:11,756] jepsen worker 61 - jepsen.util 61 :ok :read [20 6] INFO [2022-01-10 12:53:11,757] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,757] jepsen worker 61 - jepsen.util 61 :invoke :write [20 7] INFO [2022-01-10 12:53:11,759] jepsen worker 22 - jepsen.util 112 :ok :write [7 39] INFO [2022-01-10 12:53:11,759] jepsen worker 77 - jepsen.util 167 :ok :read [25 16] INFO [2022-01-10 12:53:11,759] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,760] jepsen worker 77 - jepsen.util 167 :invoke :write [25 26] INFO [2022-01-10 12:53:11,760] jepsen worker 22 - jepsen.util 112 :ok :read [7 39] INFO [2022-01-10 12:53:11,761] jepsen worker 22 - jepsen.util 112 :invoke :write [7 6] INFO [2022-01-10 12:53:11,763] jepsen worker 44 - jepsen.util 44 :ok :write [14 9] INFO [2022-01-10 12:53:11,763] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,764] jepsen worker 44 - jepsen.util 44 :ok :read [14 9] INFO [2022-01-10 12:53:11,764] jepsen worker 78 - jepsen.util 78 :ok :write [26 25] INFO [2022-01-10 12:53:11,764] jepsen worker 44 - jepsen.util 44 :invoke :write [14 2] INFO [2022-01-10 12:53:11,765] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,766] jepsen worker 78 - jepsen.util 78 :ok :read [26 25] INFO [2022-01-10 12:53:11,766] jepsen worker 78 - jepsen.util 78 :invoke :write [26 30] INFO [2022-01-10 12:53:11,767] jepsen worker 7 - jepsen.util 7 :ok :write [2 13] INFO [2022-01-10 12:53:11,767] jepsen worker 7 - jepsen.util 7 :invoke :write [2 12] INFO [2022-01-10 12:53:11,768] jepsen worker 87 - jepsen.util 87 :ok :write [29 21] INFO [2022-01-10 12:53:11,768] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,770] jepsen worker 87 - jepsen.util 87 :ok :read [29 21] INFO [2022-01-10 12:53:11,770] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,770] jepsen worker 71 - jepsen.util 71 :ok :write [23 47] INFO [2022-01-10 12:53:11,771] jepsen worker 71 - jepsen.util 71 :invoke :write [23 17] INFO [2022-01-10 12:53:11,771] jepsen worker 87 - jepsen.util 87 :ok :read [29 21] INFO [2022-01-10 12:53:11,772] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:11,773] jepsen worker 87 - jepsen.util 87 :ok :read [29 21] INFO [2022-01-10 12:53:11,773] jepsen worker 87 - jepsen.util 87 :invoke :write [29 44] INFO [2022-01-10 12:53:11,774] jepsen worker 41 - jepsen.util 41 :ok :write [13 8] INFO [2022-01-10 12:53:11,775] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:11,776] jepsen worker 15 - jepsen.util 15 :ok :write [5 46] INFO [2022-01-10 12:53:11,776] jepsen worker 41 - jepsen.util 41 :ok :read [13 8] INFO [2022-01-10 12:53:11,776] jepsen worker 3 - jepsen.util 3 :ok :write [1 16] INFO [2022-01-10 12:53:11,776] jepsen worker 15 - jepsen.util 15 :invoke :write [5 2] INFO [2022-01-10 12:53:11,777] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:11,777] jepsen worker 3 - jepsen.util 3 :invoke :write [1 22] INFO [2022-01-10 12:53:11,777] jepsen worker 41 - jepsen.util 41 :ok :read [13 8] INFO [2022-01-10 12:53:11,778] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:11,779] jepsen worker 41 - jepsen.util 41 :ok :read [13 8] INFO [2022-01-10 12:53:11,779] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:11,780] jepsen worker 41 - jepsen.util 41 :ok :read [13 8] INFO [2022-01-10 12:53:11,780] jepsen worker 35 - jepsen.util 125 :ok :write [11 44] INFO [2022-01-10 12:53:11,780] jepsen worker 1 - jepsen.util 1 :ok :write [0 13] INFO [2022-01-10 12:53:11,781] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:11,781] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:11,781] jepsen worker 41 - jepsen.util 41 :ok :read [13 8] INFO [2022-01-10 12:53:11,781] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:11,782] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:11,782] jepsen worker 1 - jepsen.util 1 :ok :read [0 13] INFO [2022-01-10 12:53:11,783] jepsen worker 35 - jepsen.util 125 :ok :read [11 44] INFO [2022-01-10 12:53:11,783] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:11,783] jepsen worker 41 - jepsen.util 41 :ok :read [13 8] INFO [2022-01-10 12:53:11,783] jepsen worker 35 - jepsen.util 125 :invoke :write [11 12] INFO [2022-01-10 12:53:11,784] jepsen worker 41 - jepsen.util 41 :invoke :write [13 47] INFO [2022-01-10 12:53:11,784] jepsen worker 1 - jepsen.util 1 :ok :read [0 13] INFO [2022-01-10 12:53:11,784] jepsen worker 84 - jepsen.util 84 :ok :write [28 20] INFO [2022-01-10 12:53:11,785] jepsen worker 1 - jepsen.util 1 :invoke :write [0 15] INFO [2022-01-10 12:53:11,785] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,786] jepsen worker 84 - jepsen.util 84 :ok :read [28 20] INFO [2022-01-10 12:53:11,787] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,788] jepsen worker 84 - jepsen.util 84 :ok :read [28 20] INFO [2022-01-10 12:53:11,788] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,789] jepsen worker 49 - jepsen.util 139 :ok :write [16 17] INFO [2022-01-10 12:53:11,789] jepsen worker 36 - jepsen.util 36 :ok :write [12 22] INFO [2022-01-10 12:53:11,789] jepsen worker 84 - jepsen.util 84 :ok :read [28 20] INFO [2022-01-10 12:53:11,790] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:11,790] jepsen worker 36 - jepsen.util 36 :invoke :write [12 43] INFO [2022-01-10 12:53:11,791] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,791] jepsen worker 49 - jepsen.util 139 :ok :read [16 17] INFO [2022-01-10 12:53:11,791] jepsen worker 49 - jepsen.util 139 :invoke :write [16 32] INFO [2022-01-10 12:53:11,792] jepsen worker 84 - jepsen.util 84 :ok :read [28 20] INFO [2022-01-10 12:53:11,792] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,793] jepsen worker 83 - jepsen.util 173 :ok :write [27 44] INFO [2022-01-10 12:53:11,793] jepsen worker 84 - jepsen.util 84 :ok :read [28 20] INFO [2022-01-10 12:53:11,794] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:11,794] jepsen worker 84 - jepsen.util 84 :invoke :write [28 36] INFO [2022-01-10 12:53:11,795] jepsen worker 13 - jepsen.util 13 :ok :write [4 29] INFO [2022-01-10 12:53:11,795] jepsen worker 83 - jepsen.util 173 :ok :read [27 44] INFO [2022-01-10 12:53:11,795] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:11,796] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:11,796] jepsen worker 13 - jepsen.util 13 :ok :read [4 29] INFO [2022-01-10 12:53:11,796] jepsen worker 83 - jepsen.util 173 :ok :read [27 44] INFO [2022-01-10 12:53:11,797] jepsen worker 13 - jepsen.util 13 :invoke :write [4 29] INFO [2022-01-10 12:53:11,797] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:11,799] jepsen worker 83 - jepsen.util 173 :ok :read [27 44] INFO [2022-01-10 12:53:11,799] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:11,800] jepsen worker 52 - jepsen.util 52 :ok :write [17 38] INFO [2022-01-10 12:53:11,800] jepsen worker 83 - jepsen.util 173 :ok :read [27 44] INFO [2022-01-10 12:53:11,800] jepsen worker 67 - jepsen.util 67 :ok :write [22 27] INFO [2022-01-10 12:53:11,801] jepsen worker 83 - jepsen.util 173 :invoke :write [27 0] INFO [2022-01-10 12:53:11,801] jepsen worker 52 - jepsen.util 52 :invoke :write [17 41] INFO [2022-01-10 12:53:11,802] jepsen worker 67 - jepsen.util 67 :invoke :write [22 3] INFO [2022-01-10 12:53:11,805] jepsen worker 78 - jepsen.util 78 :ok :write [26 30] INFO [2022-01-10 12:53:11,805] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,807] jepsen worker 78 - jepsen.util 78 :ok :read [26 30] INFO [2022-01-10 12:53:11,807] jepsen worker 77 - jepsen.util 167 :ok :write [25 26] INFO [2022-01-10 12:53:11,808] jepsen worker 77 - jepsen.util 167 :invoke :write [25 48] INFO [2022-01-10 12:53:11,808] jepsen worker 78 - jepsen.util 78 :invoke :write [26 13] INFO [2022-01-10 12:53:11,809] jepsen worker 87 - jepsen.util 87 :ok :write [29 44] INFO [2022-01-10 12:53:11,810] jepsen worker 87 - jepsen.util 87 :invoke :write [29 35] INFO [2022-01-10 12:53:11,813] jepsen worker 71 - jepsen.util 71 :ok :write [23 17] INFO [2022-01-10 12:53:11,813] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,814] jepsen worker 71 - jepsen.util 71 :ok :read [23 17] INFO [2022-01-10 12:53:11,815] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,815] jepsen worker 61 - jepsen.util 61 :ok :write [20 7] INFO [2022-01-10 12:53:11,816] jepsen worker 71 - jepsen.util 71 :ok :read [23 17] INFO [2022-01-10 12:53:11,816] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:11,817] jepsen worker 41 - jepsen.util 41 :ok :write [13 47] INFO [2022-01-10 12:53:11,817] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,817] jepsen worker 61 - jepsen.util 61 :ok :read [20 7] INFO [2022-01-10 12:53:11,818] jepsen worker 71 - jepsen.util 71 :ok :read [23 17] INFO [2022-01-10 12:53:11,818] jepsen worker 41 - jepsen.util 41 :invoke :write [13 48] INFO [2022-01-10 12:53:11,819] jepsen worker 3 - jepsen.util 3 :ok :write [1 22] INFO [2022-01-10 12:53:11,819] jepsen worker 61 - jepsen.util 61 :invoke :write [20 32] INFO [2022-01-10 12:53:11,820] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,820] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,821] jepsen worker 3 - jepsen.util 3 :ok :read [1 22] INFO [2022-01-10 12:53:11,821] jepsen worker 71 - jepsen.util 71 :ok :read [23 17] INFO [2022-01-10 12:53:11,822] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,822] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,823] jepsen worker 3 - jepsen.util 3 :ok :read [1 22] INFO [2022-01-10 12:53:11,823] jepsen worker 22 - jepsen.util 112 :ok :write [7 6] INFO [2022-01-10 12:53:11,823] jepsen worker 71 - jepsen.util 71 :ok :read [23 17] INFO [2022-01-10 12:53:11,824] jepsen worker 22 - jepsen.util 112 :invoke :write [7 35] INFO [2022-01-10 12:53:11,824] jepsen worker 1 - jepsen.util 1 :ok :write [0 15] INFO [2022-01-10 12:53:11,824] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,825] jepsen worker 1 - jepsen.util 1 :invoke :write [0 7] INFO [2022-01-10 12:53:11,826] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,826] jepsen worker 3 - jepsen.util 3 :ok :read [1 22] INFO [2022-01-10 12:53:11,827] jepsen worker 71 - jepsen.util 71 :ok :read [23 17] INFO [2022-01-10 12:53:11,827] jepsen worker 42 - jepsen.util 132 :ok :write [14 15] INFO [2022-01-10 12:53:11,827] jepsen worker 84 - jepsen.util 84 :ok :write [28 36] INFO [2022-01-10 12:53:11,827] jepsen worker 3 - jepsen.util 3 :invoke :write [1 10] INFO [2022-01-10 12:53:11,828] jepsen worker 71 - jepsen.util 71 :invoke :write [23 28] INFO [2022-01-10 12:53:11,829] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,829] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,830] jepsen worker 42 - jepsen.util 132 :ok :read [14 15] INFO [2022-01-10 12:53:11,830] jepsen worker 84 - jepsen.util 84 :ok :read [28 36] INFO [2022-01-10 12:53:11,831] jepsen worker 42 - jepsen.util 132 :invoke :write [14 42] INFO [2022-01-10 12:53:11,831] jepsen worker 7 - jepsen.util 7 :ok :write [2 12] INFO [2022-01-10 12:53:11,831] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,832] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:11,832] jepsen worker 84 - jepsen.util 84 :ok :read [28 36] INFO [2022-01-10 12:53:11,833] jepsen worker 7 - jepsen.util 7 :ok :read [2 12] INFO [2022-01-10 12:53:11,833] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,834] jepsen worker 7 - jepsen.util 7 :invoke :write [2 35] INFO [2022-01-10 12:53:11,834] jepsen worker 36 - jepsen.util 36 :ok :write [12 43] INFO [2022-01-10 12:53:11,835] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,835] jepsen worker 84 - jepsen.util 84 :ok :read [28 36] INFO [2022-01-10 12:53:11,836] jepsen worker 84 - jepsen.util 84 :invoke :write [28 6] INFO [2022-01-10 12:53:11,836] jepsen worker 36 - jepsen.util 36 :ok :read [12 43] INFO [2022-01-10 12:53:11,837] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,838] jepsen worker 36 - jepsen.util 36 :ok :read [12 43] INFO [2022-01-10 12:53:11,838] jepsen worker 36 - jepsen.util 36 :invoke :write [12 44] INFO [2022-01-10 12:53:11,841] jepsen worker 15 - jepsen.util 15 :ok :write [5 2] INFO [2022-01-10 12:53:11,841] jepsen worker 13 - jepsen.util 13 :ok :write [4 29] INFO [2022-01-10 12:53:11,841] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:11,842] jepsen worker 13 - jepsen.util 13 :invoke :write [4 33] INFO [2022-01-10 12:53:11,842] jepsen worker 15 - jepsen.util 15 :ok :read [5 2] INFO [2022-01-10 12:53:11,843] jepsen worker 35 - jepsen.util 125 :ok :write [11 12] INFO [2022-01-10 12:53:11,843] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:11,844] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:11,844] jepsen worker 15 - jepsen.util 15 :ok :read [5 2] INFO [2022-01-10 12:53:11,845] jepsen worker 35 - jepsen.util 125 :ok :read [11 12] INFO [2022-01-10 12:53:11,845] jepsen worker 15 - jepsen.util 15 :invoke :write [5 12] INFO [2022-01-10 12:53:11,845] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:11,846] jepsen worker 52 - jepsen.util 52 :ok :write [17 41] INFO [2022-01-10 12:53:11,846] jepsen worker 35 - jepsen.util 125 :ok :read [11 12] INFO [2022-01-10 12:53:11,847] jepsen worker 52 - jepsen.util 52 :invoke :write [17 43] INFO [2022-01-10 12:53:11,848] jepsen worker 35 - jepsen.util 125 :invoke :write [11 15] INFO [2022-01-10 12:53:11,851] jepsen worker 49 - jepsen.util 139 :ok :write [16 32] INFO [2022-01-10 12:53:11,851] jepsen worker 49 - jepsen.util 139 :invoke :write [16 47] INFO [2022-01-10 12:53:11,854] jepsen worker 78 - jepsen.util 78 :ok :write [26 13] INFO [2022-01-10 12:53:11,854] jepsen worker 78 - jepsen.util 78 :invoke :write [26 17] INFO [2022-01-10 12:53:11,856] jepsen worker 83 - jepsen.util 173 :ok :write [27 0] INFO [2022-01-10 12:53:11,857] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:11,858] jepsen worker 83 - jepsen.util 173 :ok :read [27 0] INFO [2022-01-10 12:53:11,859] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:11,859] jepsen worker 83 - jepsen.util 173 :ok :read [27 0] INFO [2022-01-10 12:53:11,860] jepsen worker 83 - jepsen.util 173 :invoke :write [27 15] INFO [2022-01-10 12:53:11,861] jepsen worker 71 - jepsen.util 71 :ok :write [23 28] INFO [2022-01-10 12:53:11,862] jepsen worker 71 - jepsen.util 71 :invoke :write [23 26] INFO [2022-01-10 12:53:11,862] jepsen worker 87 - jepsen.util 87 :ok :write [29 35] INFO [2022-01-10 12:53:11,863] jepsen worker 87 - jepsen.util 87 :invoke :write [29 10] INFO [2022-01-10 12:53:11,864] jepsen worker 67 - jepsen.util 67 :ok :write [22 3] INFO [2022-01-10 12:53:11,864] jepsen worker 41 - jepsen.util 41 :ok :write [13 48] INFO [2022-01-10 12:53:11,864] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:11,865] jepsen worker 41 - jepsen.util 41 :invoke :write [13 47] INFO [2022-01-10 12:53:11,865] jepsen worker 67 - jepsen.util 67 :ok :read [22 3] INFO [2022-01-10 12:53:11,866] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:11,867] jepsen worker 3 - jepsen.util 3 :ok :write [1 10] INFO [2022-01-10 12:53:11,867] jepsen worker 67 - jepsen.util 67 :ok :read [22 3] INFO [2022-01-10 12:53:11,867] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,868] jepsen worker 67 - jepsen.util 67 :invoke :write [22 7] INFO [2022-01-10 12:53:11,868] jepsen worker 3 - jepsen.util 3 :ok :read [1 10] INFO [2022-01-10 12:53:11,869] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,870] jepsen worker 77 - jepsen.util 167 :ok :write [25 48] INFO [2022-01-10 12:53:11,870] jepsen worker 1 - jepsen.util 1 :ok :write [0 7] INFO [2022-01-10 12:53:11,870] jepsen worker 3 - jepsen.util 3 :ok :read [1 10] INFO [2022-01-10 12:53:11,870] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,871] jepsen worker 1 - jepsen.util 1 :invoke :write [0 3] INFO [2022-01-10 12:53:11,871] jepsen worker 77 - jepsen.util 167 :ok :read [25 48] INFO [2022-01-10 12:53:11,871] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:11,872] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,872] jepsen worker 3 - jepsen.util 3 :ok :read [1 10] INFO [2022-01-10 12:53:11,873] jepsen worker 84 - jepsen.util 84 :ok :write [28 6] INFO [2022-01-10 12:53:11,873] jepsen worker 77 - jepsen.util 167 :ok :read [25 48] INFO [2022-01-10 12:53:11,873] jepsen worker 3 - jepsen.util 3 :invoke :write [1 49] INFO [2022-01-10 12:53:11,873] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:11,874] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:11,874] jepsen worker 84 - jepsen.util 84 :ok :read [28 6] INFO [2022-01-10 12:53:11,875] jepsen worker 77 - jepsen.util 167 :ok :read [25 48] INFO [2022-01-10 12:53:11,875] jepsen worker 84 - jepsen.util 84 :invoke :write [28 14] INFO [2022-01-10 12:53:11,875] jepsen worker 77 - jepsen.util 167 :invoke :write [25 15] INFO [2022-01-10 12:53:11,876] jepsen worker 61 - jepsen.util 61 :ok :write [20 32] INFO [2022-01-10 12:53:11,877] jepsen worker 61 - jepsen.util 61 :invoke :write [20 16] INFO [2022-01-10 12:53:11,878] jepsen worker 36 - jepsen.util 36 :ok :write [12 44] INFO [2022-01-10 12:53:11,879] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,880] jepsen worker 36 - jepsen.util 36 :ok :read [12 44] INFO [2022-01-10 12:53:11,881] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:11,882] jepsen worker 36 - jepsen.util 36 :ok :read [12 44] INFO [2022-01-10 12:53:11,882] jepsen worker 36 - jepsen.util 36 :invoke :write [12 18] INFO [2022-01-10 12:53:11,883] jepsen worker 13 - jepsen.util 13 :ok :write [4 33] INFO [2022-01-10 12:53:11,883] jepsen worker 13 - jepsen.util 13 :invoke :write [4 6] INFO [2022-01-10 12:53:11,884] jepsen worker 22 - jepsen.util 112 :ok :write [7 35] INFO [2022-01-10 12:53:11,885] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,886] jepsen worker 22 - jepsen.util 112 :ok :read [7 35] INFO [2022-01-10 12:53:11,886] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:11,887] jepsen worker 22 - jepsen.util 112 :ok :read [7 35] INFO [2022-01-10 12:53:11,888] jepsen worker 22 - jepsen.util 112 :invoke :write [7 44] INFO [2022-01-10 12:53:11,889] jepsen worker 52 - jepsen.util 52 :ok :write [17 43] INFO [2022-01-10 12:53:11,889] jepsen worker 52 - jepsen.util 52 :invoke :write [17 20] INFO [2022-01-10 12:53:11,891] jepsen worker 42 - jepsen.util 132 :ok :write [14 42] INFO [2022-01-10 12:53:11,892] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,892] jepsen worker 42 - jepsen.util 132 :ok :read [14 42] INFO [2022-01-10 12:53:11,893] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,895] jepsen worker 42 - jepsen.util 132 :ok :read [14 42] INFO [2022-01-10 12:53:11,895] jepsen worker 78 - jepsen.util 78 :ok :write [26 17] INFO [2022-01-10 12:53:11,895] jepsen worker 7 - jepsen.util 7 :ok :write [2 35] INFO [2022-01-10 12:53:11,895] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,896] jepsen worker 78 - jepsen.util 78 :invoke :write [26 5] INFO [2022-01-10 12:53:11,896] jepsen worker 42 - jepsen.util 132 :ok :read [14 42] INFO [2022-01-10 12:53:11,897] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:11,898] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,898] jepsen worker 7 - jepsen.util 7 :ok :read [2 35] INFO [2022-01-10 12:53:11,899] jepsen worker 42 - jepsen.util 132 :ok :read [14 42] INFO [2022-01-10 12:53:11,899] jepsen worker 7 - jepsen.util 7 :invoke :write [2 9] INFO [2022-01-10 12:53:11,899] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,900] jepsen worker 15 - jepsen.util 15 :ok :write [5 12] INFO [2022-01-10 12:53:11,900] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:11,900] jepsen worker 42 - jepsen.util 132 :ok :read [14 42] INFO [2022-01-10 12:53:11,901] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:11,901] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,902] jepsen worker 71 - jepsen.util 71 :ok :write [23 26] INFO [2022-01-10 12:53:11,902] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:11,902] jepsen worker 42 - jepsen.util 132 :ok :read [14 42] INFO [2022-01-10 12:53:11,903] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:11,903] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:11,904] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:11,904] jepsen worker 15 - jepsen.util 15 :invoke :write [5 32] INFO [2022-01-10 12:53:11,905] jepsen worker 42 - jepsen.util 132 :ok :read [14 42] INFO [2022-01-10 12:53:11,905] jepsen worker 71 - jepsen.util 71 :ok :read [23 26] INFO [2022-01-10 12:53:11,905] jepsen worker 71 - jepsen.util 71 :invoke :write [23 18] INFO [2022-01-10 12:53:11,906] jepsen worker 42 - jepsen.util 132 :invoke :write [14 2] INFO [2022-01-10 12:53:11,918] jepsen worker 13 - jepsen.util 13 :ok :write [4 6] INFO [2022-01-10 12:53:11,919] jepsen worker 13 - jepsen.util 13 :invoke :write [4 21] INFO [2022-01-10 12:53:11,923] jepsen worker 52 - jepsen.util 52 :ok :write [17 20] INFO [2022-01-10 12:53:11,923] jepsen worker 52 - jepsen.util 52 :invoke :write [17 12] INFO [2022-01-10 12:53:11,926] jepsen worker 78 - jepsen.util 78 :ok :write [26 5] INFO [2022-01-10 12:53:11,927] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,928] jepsen worker 78 - jepsen.util 78 :ok :read [26 5] INFO [2022-01-10 12:53:11,928] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:11,929] jepsen worker 78 - jepsen.util 78 :ok :read [26 5] INFO [2022-01-10 12:53:11,929] jepsen worker 78 - jepsen.util 78 :invoke :write [26 24] INFO [2022-01-10 12:53:12,073] jepsen worker 3 - jepsen.util 3 :ok :write [1 49] INFO [2022-01-10 12:53:12,073] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 12:53:12,090] jepsen worker 71 - jepsen.util 71 :ok :write [23 18] INFO [2022-01-10 12:53:12,091] jepsen worker 71 - jepsen.util 71 :invoke :write [23 37] INFO [2022-01-10 12:53:12,096] jepsen worker 13 - jepsen.util 13 :ok :write [4 21] INFO [2022-01-10 12:53:12,096] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,097] jepsen worker 77 - jepsen.util 167 :ok :write [25 15] INFO [2022-01-10 12:53:12,097] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:12,097] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,098] jepsen worker 13 - jepsen.util 13 :invoke :write [4 16] INFO [2022-01-10 12:53:12,098] jepsen worker 77 - jepsen.util 167 :ok :read [25 15] INFO [2022-01-10 12:53:12,099] jepsen worker 77 - jepsen.util 167 :invoke :write [25 46] INFO [2022-01-10 12:53:12,102] jepsen worker 52 - jepsen.util 52 :ok :write [17 12] INFO [2022-01-10 12:53:12,102] jepsen worker 52 - jepsen.util 52 :invoke :write [17 13] INFO [2022-01-10 12:53:12,104] jepsen worker 61 - jepsen.util 61 :ok :write [20 16] INFO [2022-01-10 12:53:12,104] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,105] jepsen worker 61 - jepsen.util 61 :ok :read [20 16] INFO [2022-01-10 12:53:12,105] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,106] jepsen worker 61 - jepsen.util 61 :ok :read [20 16] INFO [2022-01-10 12:53:12,107] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,108] jepsen worker 78 - jepsen.util 78 :ok :write [26 24] INFO [2022-01-10 12:53:12,108] jepsen worker 61 - jepsen.util 61 :ok :read [20 16] INFO [2022-01-10 12:53:12,108] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,109] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,109] jepsen worker 78 - jepsen.util 78 :ok :read [26 24] INFO [2022-01-10 12:53:12,110] jepsen worker 78 - jepsen.util 78 :invoke :write [26 19] INFO [2022-01-10 12:53:12,110] jepsen worker 61 - jepsen.util 61 :ok :read [20 16] INFO [2022-01-10 12:53:12,110] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,111] jepsen worker 22 - jepsen.util 112 :ok :write [7 44] INFO [2022-01-10 12:53:12,111] jepsen worker 61 - jepsen.util 61 :ok :read [20 16] INFO [2022-01-10 12:53:12,111] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,111] jepsen worker 61 - jepsen.util 61 :invoke :write [20 23] INFO [2022-01-10 12:53:12,112] jepsen worker 22 - jepsen.util 112 :ok :read [7 44] INFO [2022-01-10 12:53:12,112] jepsen worker 22 - jepsen.util 112 :invoke :write [7 1] INFO [2022-01-10 12:53:12,118] jepsen worker 71 - jepsen.util 71 :ok :write [23 37] INFO [2022-01-10 12:53:12,119] jepsen worker 71 - jepsen.util 71 :invoke :write [23 27] INFO [2022-01-10 12:53:12,120] jepsen worker 43 - jepsen.util 43 :ok :write [14 0] INFO [2022-01-10 12:53:12,121] jepsen worker 43 - jepsen.util 43 :invoke :write [14 38] INFO [2022-01-10 12:53:12,123] jepsen worker 13 - jepsen.util 13 :ok :write [4 16] INFO [2022-01-10 12:53:12,124] jepsen worker 13 - jepsen.util 13 :invoke :write [4 46] INFO [2022-01-10 12:53:12,127] jepsen worker 52 - jepsen.util 52 :ok :write [17 13] INFO [2022-01-10 12:53:12,127] jepsen worker 52 - jepsen.util 52 :invoke :write [17 12] INFO [2022-01-10 12:53:12,132] jepsen worker 78 - jepsen.util 78 :ok :write [26 19] INFO [2022-01-10 12:53:12,133] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,134] jepsen worker 78 - jepsen.util 78 :ok :read [26 19] INFO [2022-01-10 12:53:12,134] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,135] jepsen worker 78 - jepsen.util 78 :ok :read [26 19] INFO [2022-01-10 12:53:12,135] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,136] jepsen worker 78 - jepsen.util 78 :ok :read [26 19] INFO [2022-01-10 12:53:12,137] jepsen worker 78 - jepsen.util 78 :invoke :write [26 29] INFO [2022-01-10 12:53:12,139] jepsen worker 71 - jepsen.util 71 :ok :write [23 27] INFO [2022-01-10 12:53:12,140] jepsen worker 77 - jepsen.util 167 :ok :write [25 46] INFO [2022-01-10 12:53:12,140] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,141] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,141] jepsen worker 71 - jepsen.util 71 :ok :read [23 27] INFO [2022-01-10 12:53:12,142] jepsen worker 71 - jepsen.util 71 :invoke :write [23 34] INFO [2022-01-10 12:53:12,142] jepsen worker 77 - jepsen.util 167 :ok :read [25 46] INFO [2022-01-10 12:53:12,143] jepsen worker 77 - jepsen.util 167 :invoke :write [25 0] INFO [2022-01-10 12:53:12,147] jepsen worker 13 - jepsen.util 13 :ok :write [4 46] INFO [2022-01-10 12:53:12,148] jepsen worker 61 - jepsen.util 61 :ok :write [20 23] INFO [2022-01-10 12:53:12,148] jepsen worker 13 - jepsen.util 13 :invoke :write [4 7] INFO [2022-01-10 12:53:12,149] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,150] jepsen worker 61 - jepsen.util 61 :ok :read [20 23] INFO [2022-01-10 12:53:12,150] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,151] jepsen worker 61 - jepsen.util 61 :ok :read [20 23] INFO [2022-01-10 12:53:12,152] jepsen worker 61 - jepsen.util 61 :invoke :write [20 30] INFO [2022-01-10 12:53:12,155] jepsen worker 52 - jepsen.util 52 :ok :write [17 12] INFO [2022-01-10 12:53:12,156] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,157] jepsen worker 52 - jepsen.util 52 :ok :read [17 12] INFO [2022-01-10 12:53:12,157] jepsen worker 22 - jepsen.util 112 :ok :write [7 1] INFO [2022-01-10 12:53:12,158] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,158] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,159] jepsen worker 52 - jepsen.util 52 :ok :read [17 12] INFO [2022-01-10 12:53:12,159] jepsen worker 22 - jepsen.util 112 :ok :read [7 1] INFO [2022-01-10 12:53:12,160] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,161] jepsen worker 52 - jepsen.util 52 :invoke :write [17 40] INFO [2022-01-10 12:53:12,161] jepsen worker 22 - jepsen.util 112 :ok :read [7 1] INFO [2022-01-10 12:53:12,162] jepsen worker 22 - jepsen.util 112 :invoke :write [7 46] INFO [2022-01-10 12:53:12,166] jepsen worker 78 - jepsen.util 78 :ok :write [26 29] INFO [2022-01-10 12:53:12,166] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,167] jepsen worker 78 - jepsen.util 78 :ok :read [26 29] INFO [2022-01-10 12:53:12,167] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,168] jepsen worker 78 - jepsen.util 78 :ok :read [26 29] INFO [2022-01-10 12:53:12,169] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,169] jepsen worker 78 - jepsen.util 78 :ok :read [26 29] INFO [2022-01-10 12:53:12,170] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,171] jepsen worker 71 - jepsen.util 71 :ok :write [23 34] INFO [2022-01-10 12:53:12,171] jepsen worker 43 - jepsen.util 43 :ok :write [14 38] INFO [2022-01-10 12:53:12,171] jepsen worker 78 - jepsen.util 78 :ok :read [26 29] INFO [2022-01-10 12:53:12,171] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,172] jepsen worker 78 - jepsen.util 78 :invoke :write [26 1] INFO [2022-01-10 12:53:12,172] jepsen worker 43 - jepsen.util 43 :ok :read [14 38] INFO [2022-01-10 12:53:12,172] jepsen worker 71 - jepsen.util 71 :invoke :write [23 40] INFO [2022-01-10 12:53:12,173] jepsen worker 43 - jepsen.util 43 :invoke :write [14 11] INFO [2022-01-10 12:53:12,175] jepsen worker 77 - jepsen.util 167 :ok :write [25 0] INFO [2022-01-10 12:53:12,176] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,177] jepsen worker 35 - jepsen.util 125 :ok :write [11 15] INFO [2022-01-10 12:53:12,177] jepsen worker 77 - jepsen.util 167 :ok :read [25 0] INFO [2022-01-10 12:53:12,177] jepsen worker 35 - jepsen.util 125 :invoke :write [11 43] INFO [2022-01-10 12:53:12,178] jepsen worker 84 - jepsen.util 84 :ok :write [28 14] INFO [2022-01-10 12:53:12,178] jepsen worker 77 - jepsen.util 167 :invoke :write [25 27] INFO [2022-01-10 12:53:12,178] jepsen worker 13 - jepsen.util 13 :ok :write [4 7] INFO [2022-01-10 12:53:12,179] jepsen worker 84 - jepsen.util 84 :invoke :write [28 2] INFO [2022-01-10 12:53:12,179] jepsen worker 13 - jepsen.util 13 :invoke :write [4 26] INFO [2022-01-10 12:53:12,186] jepsen worker 52 - jepsen.util 52 :ok :write [17 40] INFO [2022-01-10 12:53:12,186] jepsen worker 61 - jepsen.util 61 :ok :write [20 30] INFO [2022-01-10 12:53:12,186] jepsen worker 62 - jepsen.util 62 :info :write [20 6] :RequestTimeoutException INFO [2022-01-10 12:53:12,187] jepsen worker 52 - jepsen.util 52 :invoke :write [17 6] INFO [2022-01-10 12:53:12,187] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,188] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,188] jepsen worker 61 - jepsen.util 61 :ok :read [20 30] INFO [2022-01-10 12:53:12,189] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,189] jepsen worker 62 - jepsen.util 152 :ok :read [20 30] INFO [2022-01-10 12:53:12,190] jepsen worker 62 - jepsen.util 152 :invoke :write [20 1] INFO [2022-01-10 12:53:12,190] jepsen worker 61 - jepsen.util 61 :ok :read [20 30] INFO [2022-01-10 12:53:12,190] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,191] jepsen worker 22 - jepsen.util 112 :ok :write [7 46] INFO [2022-01-10 12:53:12,191] jepsen worker 78 - jepsen.util 78 :ok :write [26 1] INFO [2022-01-10 12:53:12,192] jepsen worker 22 - jepsen.util 112 :invoke :write [7 5] INFO [2022-01-10 12:53:12,192] jepsen worker 61 - jepsen.util 61 :ok :read [20 30] INFO [2022-01-10 12:53:12,192] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,193] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,193] jepsen worker 61 - jepsen.util 61 :ok :read [20 30] INFO [2022-01-10 12:53:12,194] jepsen worker 61 - jepsen.util 61 :invoke :write [20 30] INFO [2022-01-10 12:53:12,194] jepsen worker 78 - jepsen.util 78 :ok :read [26 1] INFO [2022-01-10 12:53:12,195] jepsen worker 78 - jepsen.util 78 :invoke :write [26 33] INFO [2022-01-10 12:53:12,195] jepsen worker 43 - jepsen.util 43 :ok :write [14 11] INFO [2022-01-10 12:53:12,196] jepsen worker 43 - jepsen.util 43 :invoke :write [14 14] INFO [2022-01-10 12:53:12,198] jepsen worker 71 - jepsen.util 71 :ok :write [23 40] INFO [2022-01-10 12:53:12,199] jepsen worker 77 - jepsen.util 167 :ok :write [25 27] INFO [2022-01-10 12:53:12,199] jepsen worker 71 - jepsen.util 71 :invoke :write [23 48] INFO [2022-01-10 12:53:12,200] jepsen worker 77 - jepsen.util 167 :invoke :write [25 47] INFO [2022-01-10 12:53:12,204] jepsen worker 13 - jepsen.util 13 :ok :write [4 26] INFO [2022-01-10 12:53:12,204] jepsen worker 13 - jepsen.util 13 :invoke :write [4 9] INFO [2022-01-10 12:53:12,210] jepsen worker 52 - jepsen.util 52 :ok :write [17 6] INFO [2022-01-10 12:53:12,210] jepsen worker 52 - jepsen.util 52 :invoke :write [17 14] INFO [2022-01-10 12:53:12,214] jepsen worker 22 - jepsen.util 112 :ok :write [7 5] INFO [2022-01-10 12:53:12,215] jepsen worker 62 - jepsen.util 152 :ok :write [20 1] INFO [2022-01-10 12:53:12,215] jepsen worker 78 - jepsen.util 78 :ok :write [26 33] INFO [2022-01-10 12:53:12,215] jepsen worker 22 - jepsen.util 112 :invoke :write [7 47] INFO [2022-01-10 12:53:12,216] jepsen worker 62 - jepsen.util 152 :invoke :write [20 13] INFO [2022-01-10 12:53:12,217] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,218] jepsen worker 78 - jepsen.util 78 :ok :read [26 33] INFO [2022-01-10 12:53:12,218] jepsen worker 78 - jepsen.util 78 :invoke :write [26 2] INFO [2022-01-10 12:53:12,223] jepsen worker 71 - jepsen.util 71 :ok :write [23 48] INFO [2022-01-10 12:53:12,224] jepsen worker 71 - jepsen.util 71 :invoke :write [23 26] INFO [2022-01-10 12:53:12,224] jepsen worker 43 - jepsen.util 43 :ok :write [14 14] INFO [2022-01-10 12:53:12,225] jepsen worker 43 - jepsen.util 43 :invoke :write [14 3] INFO [2022-01-10 12:53:12,228] jepsen worker 14 - jepsen.util 104 :ok :write [4 21] INFO [2022-01-10 12:53:12,229] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,230] jepsen worker 14 - jepsen.util 104 :ok :read [4 21] INFO [2022-01-10 12:53:12,230] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,231] jepsen worker 14 - jepsen.util 104 :ok :read [4 21] INFO [2022-01-10 12:53:12,232] jepsen worker 14 - jepsen.util 104 :invoke :write [4 18] INFO [2022-01-10 12:53:12,232] jepsen worker 77 - jepsen.util 167 :ok :write [25 47] INFO [2022-01-10 12:53:12,232] jepsen worker 52 - jepsen.util 52 :ok :write [17 14] INFO [2022-01-10 12:53:12,233] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,234] jepsen worker 52 - jepsen.util 52 :invoke :write [17 1] INFO [2022-01-10 12:53:12,234] jepsen worker 77 - jepsen.util 167 :ok :read [25 47] INFO [2022-01-10 12:53:12,235] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,236] jepsen worker 77 - jepsen.util 167 :ok :read [25 47] INFO [2022-01-10 12:53:12,237] jepsen worker 77 - jepsen.util 167 :invoke :write [25 0] INFO [2022-01-10 12:53:12,238] jepsen worker 78 - jepsen.util 78 :ok :write [26 2] INFO [2022-01-10 12:53:12,238] jepsen worker 22 - jepsen.util 112 :ok :write [7 47] INFO [2022-01-10 12:53:12,238] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,239] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,240] jepsen worker 78 - jepsen.util 78 :ok :read [26 2] INFO [2022-01-10 12:53:12,240] jepsen worker 22 - jepsen.util 112 :ok :read [7 47] INFO [2022-01-10 12:53:12,240] jepsen worker 78 - jepsen.util 78 :invoke :write [26 10] INFO [2022-01-10 12:53:12,241] jepsen worker 22 - jepsen.util 112 :invoke :write [7 15] INFO [2022-01-10 12:53:12,242] jepsen worker 71 - jepsen.util 71 :ok :write [23 26] INFO [2022-01-10 12:53:12,242] jepsen worker 57 - jepsen.util 57 :info :write [19 28] :RequestTimeoutException INFO [2022-01-10 12:53:12,243] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,243] jepsen worker 57 - jepsen.util 147 :invoke :write [19 19] INFO [2022-01-10 12:53:12,244] jepsen worker 71 - jepsen.util 71 :ok :read [23 26] INFO [2022-01-10 12:53:12,245] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,246] jepsen worker 71 - jepsen.util 71 :ok :read [23 26] INFO [2022-01-10 12:53:12,246] jepsen worker 71 - jepsen.util 71 :invoke :write [23 8] INFO [2022-01-10 12:53:12,247] jepsen worker 14 - jepsen.util 104 :ok :write [4 18] INFO [2022-01-10 12:53:12,247] jepsen worker 62 - jepsen.util 152 :ok :write [20 13] INFO [2022-01-10 12:53:12,248] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,248] jepsen worker 62 - jepsen.util 152 :invoke :write [20 36] INFO [2022-01-10 12:53:12,249] jepsen worker 14 - jepsen.util 104 :ok :read [4 18] INFO [2022-01-10 12:53:12,249] jepsen worker 14 - jepsen.util 104 :invoke :write [4 5] INFO [2022-01-10 12:53:12,250] jepsen worker 43 - jepsen.util 43 :ok :write [14 3] INFO [2022-01-10 12:53:12,251] jepsen worker 43 - jepsen.util 43 :invoke :write [14 43] INFO [2022-01-10 12:53:12,253] jepsen worker 52 - jepsen.util 52 :ok :write [17 1] INFO [2022-01-10 12:53:12,253] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,254] jepsen worker 52 - jepsen.util 52 :ok :read [17 1] INFO [2022-01-10 12:53:12,255] jepsen worker 52 - jepsen.util 52 :invoke :write [17 3] INFO [2022-01-10 12:53:12,259] jepsen worker 77 - jepsen.util 167 :ok :write [25 0] INFO [2022-01-10 12:53:12,259] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,260] jepsen worker 77 - jepsen.util 167 :ok :read [25 0] INFO [2022-01-10 12:53:12,260] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,261] jepsen worker 77 - jepsen.util 167 :ok :read [25 0] INFO [2022-01-10 12:53:12,262] jepsen worker 77 - jepsen.util 167 :invoke :write [25 34] INFO [2022-01-10 12:53:12,262] jepsen worker 78 - jepsen.util 78 :ok :write [26 10] INFO [2022-01-10 12:53:12,263] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,264] jepsen worker 78 - jepsen.util 78 :ok :read [26 10] INFO [2022-01-10 12:53:12,264] jepsen worker 22 - jepsen.util 112 :ok :write [7 15] INFO [2022-01-10 12:53:12,265] jepsen worker 78 - jepsen.util 78 :invoke :write [26 28] INFO [2022-01-10 12:53:12,265] jepsen worker 22 - jepsen.util 112 :invoke :write [7 13] INFO [2022-01-10 12:53:12,270] jepsen worker 71 - jepsen.util 71 :ok :write [23 8] INFO [2022-01-10 12:53:12,271] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,272] jepsen worker 62 - jepsen.util 152 :ok :write [20 36] INFO [2022-01-10 12:53:12,272] jepsen worker 71 - jepsen.util 71 :ok :read [23 8] INFO [2022-01-10 12:53:12,272] jepsen worker 62 - jepsen.util 152 :invoke :write [20 31] INFO [2022-01-10 12:53:12,272] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,273] jepsen worker 71 - jepsen.util 71 :ok :read [23 8] INFO [2022-01-10 12:53:12,274] jepsen worker 71 - jepsen.util 71 :invoke :write [23 2] INFO [2022-01-10 12:53:12,276] jepsen worker 43 - jepsen.util 43 :ok :write [14 43] INFO [2022-01-10 12:53:12,276] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,277] jepsen worker 43 - jepsen.util 43 :ok :read [14 43] INFO [2022-01-10 12:53:12,278] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,278] jepsen worker 43 - jepsen.util 43 :ok :read [14 43] INFO [2022-01-10 12:53:12,279] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,280] jepsen worker 43 - jepsen.util 43 :ok :read [14 43] INFO [2022-01-10 12:53:12,280] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,281] jepsen worker 43 - jepsen.util 43 :ok :read [14 43] INFO [2022-01-10 12:53:12,281] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,282] jepsen worker 43 - jepsen.util 43 :ok :read [14 43] INFO [2022-01-10 12:53:12,283] jepsen worker 14 - jepsen.util 104 :ok :write [4 5] INFO [2022-01-10 12:53:12,283] jepsen worker 77 - jepsen.util 167 :ok :write [25 34] INFO [2022-01-10 12:53:12,283] jepsen worker 43 - jepsen.util 43 :invoke :write [14 41] INFO [2022-01-10 12:53:12,283] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,284] jepsen worker 14 - jepsen.util 104 :invoke :write [4 34] INFO [2022-01-10 12:53:12,284] jepsen worker 77 - jepsen.util 167 :ok :read [25 34] INFO [2022-01-10 12:53:12,285] jepsen worker 77 - jepsen.util 167 :invoke :write [25 9] INFO [2022-01-10 12:53:12,290] jepsen worker 52 - jepsen.util 52 :ok :write [17 3] INFO [2022-01-10 12:53:12,291] jepsen worker 52 - jepsen.util 52 :invoke :write [17 30] INFO [2022-01-10 12:53:12,291] jepsen worker 22 - jepsen.util 112 :ok :write [7 13] INFO [2022-01-10 12:53:12,292] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,293] jepsen worker 22 - jepsen.util 112 :ok :read [7 13] INFO [2022-01-10 12:53:12,293] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,295] jepsen worker 22 - jepsen.util 112 :ok :read [7 13] INFO [2022-01-10 12:53:12,295] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,296] jepsen worker 22 - jepsen.util 112 :ok :read [7 13] INFO [2022-01-10 12:53:12,297] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,298] jepsen worker 22 - jepsen.util 112 :ok :read [7 13] INFO [2022-01-10 12:53:12,298] jepsen worker 22 - jepsen.util 112 :invoke :write [7 1] INFO [2022-01-10 12:53:12,299] jepsen worker 62 - jepsen.util 152 :ok :write [20 31] INFO [2022-01-10 12:53:12,300] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,300] jepsen worker 62 - jepsen.util 152 :ok :read [20 31] INFO [2022-01-10 12:53:12,300] jepsen worker 49 - jepsen.util 139 :ok :write [16 47] INFO [2022-01-10 12:53:12,301] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:12,301] jepsen worker 62 - jepsen.util 152 :invoke :write [20 24] INFO [2022-01-10 12:53:12,302] jepsen worker 36 - jepsen.util 36 :ok :write [12 18] INFO [2022-01-10 12:53:12,302] jepsen worker 36 - jepsen.util 36 :invoke :write [12 13] INFO [2022-01-10 12:53:12,302] jepsen worker 49 - jepsen.util 139 :ok :read [16 47] INFO [2022-01-10 12:53:12,303] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:12,304] jepsen worker 49 - jepsen.util 139 :ok :read [16 47] INFO [2022-01-10 12:53:12,304] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:12,305] jepsen worker 49 - jepsen.util 139 :ok :read [16 47] INFO [2022-01-10 12:53:12,306] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:12,306] jepsen worker 49 - jepsen.util 139 :ok :read [16 47] INFO [2022-01-10 12:53:12,307] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:12,308] jepsen worker 49 - jepsen.util 139 :ok :read [16 47] INFO [2022-01-10 12:53:12,308] jepsen worker 49 - jepsen.util 139 :invoke :write [16 29] INFO [2022-01-10 12:53:12,308] jepsen worker 3 - jepsen.util 3 :ok :write [1 40] INFO [2022-01-10 12:53:12,309] jepsen worker 83 - jepsen.util 173 :ok :write [27 15] INFO [2022-01-10 12:53:12,309] jepsen worker 3 - jepsen.util 3 :invoke :write [1 7] INFO [2022-01-10 12:53:12,309] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:12,310] jepsen worker 83 - jepsen.util 173 :ok :read [27 15] INFO [2022-01-10 12:53:12,311] jepsen worker 83 - jepsen.util 173 :invoke :write [27 25] INFO [2022-01-10 12:53:12,318] jepsen worker 87 - jepsen.util 87 :ok :write [29 10] INFO [2022-01-10 12:53:12,318] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:12,319] jepsen worker 84 - jepsen.util 84 :ok :write [28 2] INFO [2022-01-10 12:53:12,319] jepsen worker 87 - jepsen.util 87 :ok :read [29 10] INFO [2022-01-10 12:53:12,319] jepsen worker 84 - jepsen.util 84 :invoke :write [28 7] INFO [2022-01-10 12:53:12,319] jepsen worker 87 - jepsen.util 87 :invoke :write [29 38] INFO [2022-01-10 12:53:12,324] jepsen worker 67 - jepsen.util 67 :ok :write [22 7] INFO [2022-01-10 12:53:12,324] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:12,325] jepsen worker 67 - jepsen.util 67 :ok :read [22 7] INFO [2022-01-10 12:53:12,325] jepsen worker 67 - jepsen.util 67 :invoke :write [22 44] INFO [2022-01-10 12:53:12,329] jepsen worker 10 - jepsen.util 100 :fail :read [3 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:12,329] jepsen worker 10 - jepsen.util 100 :invoke :write [3 35] INFO [2022-01-10 12:53:12,332] jepsen worker 41 - jepsen.util 41 :ok :write [13 47] INFO [2022-01-10 12:53:12,332] jepsen worker 41 - jepsen.util 41 :invoke :write [13 21] INFO [2022-01-10 12:53:12,339] jepsen worker 1 - jepsen.util 1 :ok :write [0 3] INFO [2022-01-10 12:53:12,340] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:12,340] jepsen worker 1 - jepsen.util 1 :ok :read [0 3] INFO [2022-01-10 12:53:12,341] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:12,342] jepsen worker 1 - jepsen.util 1 :ok :read [0 3] INFO [2022-01-10 12:53:12,342] jepsen worker 1 - jepsen.util 1 :invoke :write [0 15] INFO [2022-01-10 12:53:12,346] jepsen worker 7 - jepsen.util 7 :ok :write [2 9] INFO [2022-01-10 12:53:12,346] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:12,347] jepsen worker 7 - jepsen.util 7 :ok :read [2 9] INFO [2022-01-10 12:53:12,348] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:12,349] jepsen worker 7 - jepsen.util 7 :ok :read [2 9] INFO [2022-01-10 12:53:12,350] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:12,350] jepsen worker 15 - jepsen.util 15 :ok :write [5 32] INFO [2022-01-10 12:53:12,351] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:12,351] jepsen worker 7 - jepsen.util 7 :ok :read [2 9] INFO [2022-01-10 12:53:12,351] jepsen worker 7 - jepsen.util 7 :invoke :write [2 39] INFO [2022-01-10 12:53:12,352] jepsen worker 15 - jepsen.util 15 :ok :read [5 32] INFO [2022-01-10 12:53:12,352] jepsen worker 15 - jepsen.util 15 :invoke :write [5 40] INFO [2022-01-10 12:53:12,356] jepsen worker 35 - jepsen.util 125 :ok :write [11 43] INFO [2022-01-10 12:53:12,357] jepsen worker 35 - jepsen.util 125 :invoke :write [11 23] INFO [2022-01-10 12:53:12,361] jepsen worker 49 - jepsen.util 139 :ok :write [16 29] INFO [2022-01-10 12:53:12,362] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:12,363] jepsen worker 49 - jepsen.util 139 :ok :read [16 29] INFO [2022-01-10 12:53:12,363] jepsen worker 49 - jepsen.util 139 :invoke :write [16 25] INFO [2022-01-10 12:53:12,367] jepsen worker 83 - jepsen.util 173 :ok :write [27 25] INFO [2022-01-10 12:53:12,368] jepsen worker 83 - jepsen.util 173 :invoke :write [27 32] INFO [2022-01-10 12:53:12,400] jepsen worker 78 - jepsen.util 78 :ok :write [26 28] INFO [2022-01-10 12:53:12,400] jepsen worker 43 - jepsen.util 43 :ok :write [14 41] INFO [2022-01-10 12:53:12,401] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,401] jepsen worker 78 - jepsen.util 78 :invoke :write [26 44] INFO [2022-01-10 12:53:12,402] jepsen worker 43 - jepsen.util 43 :ok :read [14 41] INFO [2022-01-10 12:53:12,402] jepsen worker 43 - jepsen.util 43 :invoke :write [14 5] INFO [2022-01-10 12:53:12,404] jepsen worker 67 - jepsen.util 67 :ok :write [22 44] INFO [2022-01-10 12:53:12,405] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:12,406] jepsen worker 77 - jepsen.util 167 :ok :write [25 9] INFO [2022-01-10 12:53:12,406] jepsen worker 67 - jepsen.util 67 :ok :read [22 44] INFO [2022-01-10 12:53:12,407] jepsen worker 77 - jepsen.util 167 :invoke :write [25 40] INFO [2022-01-10 12:53:12,407] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:12,408] jepsen worker 71 - jepsen.util 71 :ok :write [23 2] INFO [2022-01-10 12:53:12,408] jepsen worker 67 - jepsen.util 67 :ok :read [22 44] INFO [2022-01-10 12:53:12,409] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,409] jepsen worker 7 - jepsen.util 7 :ok :write [2 39] INFO [2022-01-10 12:53:12,409] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:12,409] jepsen worker 22 - jepsen.util 112 :ok :write [7 1] INFO [2022-01-10 12:53:12,410] jepsen worker 71 - jepsen.util 71 :ok :read [23 2] INFO [2022-01-10 12:53:12,410] jepsen worker 7 - jepsen.util 7 :invoke :write [2 13] INFO [2022-01-10 12:53:12,410] jepsen worker 67 - jepsen.util 67 :ok :read [22 44] INFO [2022-01-10 12:53:12,410] jepsen worker 22 - jepsen.util 112 :invoke :write [7 15] INFO [2022-01-10 12:53:12,411] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:12,411] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,412] jepsen worker 67 - jepsen.util 67 :ok :read [22 44] INFO [2022-01-10 12:53:12,412] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:12,413] jepsen worker 71 - jepsen.util 71 :ok :read [23 2] INFO [2022-01-10 12:53:12,413] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,414] jepsen worker 67 - jepsen.util 67 :ok :read [22 44] INFO [2022-01-10 12:53:12,414] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:12,414] jepsen worker 71 - jepsen.util 71 :ok :read [23 2] INFO [2022-01-10 12:53:12,415] jepsen worker 71 - jepsen.util 71 :invoke :write [23 5] INFO [2022-01-10 12:53:12,415] jepsen worker 14 - jepsen.util 104 :ok :write [4 34] INFO [2022-01-10 12:53:12,415] jepsen worker 67 - jepsen.util 67 :ok :read [22 44] INFO [2022-01-10 12:53:12,416] jepsen worker 67 - jepsen.util 67 :invoke :write [22 44] INFO [2022-01-10 12:53:12,416] jepsen worker 15 - jepsen.util 15 :ok :write [5 40] INFO [2022-01-10 12:53:12,417] jepsen worker 14 - jepsen.util 104 :invoke :write [4 35] INFO [2022-01-10 12:53:12,417] jepsen worker 15 - jepsen.util 15 :invoke :write [5 36] INFO [2022-01-10 12:53:12,420] jepsen worker 52 - jepsen.util 52 :ok :write [17 30] INFO [2022-01-10 12:53:12,420] jepsen worker 62 - jepsen.util 152 :ok :write [20 24] INFO [2022-01-10 12:53:12,421] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,422] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,422] jepsen worker 62 - jepsen.util 152 :ok :read [20 24] INFO [2022-01-10 12:53:12,423] jepsen worker 52 - jepsen.util 52 :ok :read [17 30] INFO [2022-01-10 12:53:12,423] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,423] jepsen worker 35 - jepsen.util 125 :ok :write [11 23] INFO [2022-01-10 12:53:12,424] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,424] jepsen worker 62 - jepsen.util 152 :ok :read [20 24] INFO [2022-01-10 12:53:12,424] jepsen worker 35 - jepsen.util 125 :invoke :write [11 7] INFO [2022-01-10 12:53:12,425] jepsen worker 36 - jepsen.util 36 :ok :write [12 13] INFO [2022-01-10 12:53:12,425] jepsen worker 62 - jepsen.util 152 :invoke :write [20 12] INFO [2022-01-10 12:53:12,425] jepsen worker 52 - jepsen.util 52 :ok :read [17 30] INFO [2022-01-10 12:53:12,425] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:12,426] jepsen worker 52 - jepsen.util 52 :invoke :write [17 46] INFO [2022-01-10 12:53:12,427] jepsen worker 89 - jepsen.util 179 :ok :write [29 45] INFO [2022-01-10 12:53:12,428] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:12,428] jepsen worker 36 - jepsen.util 36 :ok :read [12 13] INFO [2022-01-10 12:53:12,429] jepsen worker 89 - jepsen.util 179 :ok :read [29 45] INFO [2022-01-10 12:53:12,429] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:12,429] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:12,430] jepsen worker 36 - jepsen.util 36 :ok :read [12 13] INFO [2022-01-10 12:53:12,430] jepsen worker 89 - jepsen.util 179 :ok :read [29 45] INFO [2022-01-10 12:53:12,430] jepsen worker 36 - jepsen.util 36 :invoke :write [12 35] INFO [2022-01-10 12:53:12,431] jepsen worker 89 - jepsen.util 179 :invoke :write [29 18] INFO [2022-01-10 12:53:12,431] jepsen worker 3 - jepsen.util 3 :ok :write [1 7] INFO [2022-01-10 12:53:12,432] jepsen worker 49 - jepsen.util 139 :ok :write [16 25] INFO [2022-01-10 12:53:12,432] jepsen worker 3 - jepsen.util 3 :invoke :write [1 17] INFO [2022-01-10 12:53:12,433] jepsen worker 49 - jepsen.util 139 :invoke :write [16 47] INFO [2022-01-10 12:53:12,435] jepsen worker 83 - jepsen.util 173 :ok :write [27 32] INFO [2022-01-10 12:53:12,435] jepsen worker 83 - jepsen.util 173 :invoke :write [27 30] INFO [2022-01-10 12:53:12,436] jepsen worker 41 - jepsen.util 41 :ok :write [13 21] INFO [2022-01-10 12:53:12,437] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,437] jepsen worker 84 - jepsen.util 84 :ok :write [28 7] INFO [2022-01-10 12:53:12,438] jepsen worker 41 - jepsen.util 41 :ok :read [13 21] INFO [2022-01-10 12:53:12,438] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:12,439] jepsen worker 84 - jepsen.util 84 :ok :read [28 7] INFO [2022-01-10 12:53:12,439] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,440] jepsen worker 84 - jepsen.util 84 :invoke :write [28 27] INFO [2022-01-10 12:53:12,440] jepsen worker 41 - jepsen.util 41 :ok :read [13 21] INFO [2022-01-10 12:53:12,441] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,441] jepsen worker 78 - jepsen.util 78 :ok :write [26 44] INFO [2022-01-10 12:53:12,441] jepsen worker 41 - jepsen.util 41 :ok :read [13 21] INFO [2022-01-10 12:53:12,442] jepsen worker 78 - jepsen.util 78 :invoke :write [26 4] INFO [2022-01-10 12:53:12,443] jepsen worker 1 - jepsen.util 1 :ok :write [0 15] INFO [2022-01-10 12:53:12,443] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,443] jepsen worker 1 - jepsen.util 1 :invoke :write [0 27] INFO [2022-01-10 12:53:12,444] jepsen worker 41 - jepsen.util 41 :ok :read [13 21] INFO [2022-01-10 12:53:12,444] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,445] jepsen worker 41 - jepsen.util 41 :ok :read [13 21] INFO [2022-01-10 12:53:12,446] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,447] jepsen worker 43 - jepsen.util 43 :ok :write [14 5] INFO [2022-01-10 12:53:12,447] jepsen worker 67 - jepsen.util 67 :ok :write [22 44] INFO [2022-01-10 12:53:12,447] jepsen worker 41 - jepsen.util 41 :ok :read [13 21] INFO [2022-01-10 12:53:12,447] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,448] jepsen worker 41 - jepsen.util 41 :invoke :write [13 35] INFO [2022-01-10 12:53:12,448] jepsen worker 43 - jepsen.util 43 :ok :read [14 5] INFO [2022-01-10 12:53:12,449] jepsen worker 67 - jepsen.util 67 :invoke :write [22 0] INFO [2022-01-10 12:53:12,449] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,450] jepsen worker 43 - jepsen.util 43 :ok :read [14 5] INFO [2022-01-10 12:53:12,451] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,452] jepsen worker 43 - jepsen.util 43 :ok :read [14 5] INFO [2022-01-10 12:53:12,452] jepsen worker 71 - jepsen.util 71 :ok :write [23 5] INFO [2022-01-10 12:53:12,452] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,453] jepsen worker 71 - jepsen.util 71 :invoke :write [23 4] INFO [2022-01-10 12:53:12,453] jepsen worker 43 - jepsen.util 43 :ok :read [14 5] INFO [2022-01-10 12:53:12,454] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,454] jepsen worker 77 - jepsen.util 167 :ok :write [25 40] INFO [2022-01-10 12:53:12,455] jepsen worker 43 - jepsen.util 43 :ok :read [14 5] INFO [2022-01-10 12:53:12,455] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,456] jepsen worker 43 - jepsen.util 43 :invoke :write [14 13] INFO [2022-01-10 12:53:12,456] jepsen worker 77 - jepsen.util 167 :ok :read [25 40] INFO [2022-01-10 12:53:12,457] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,458] jepsen worker 77 - jepsen.util 167 :ok :read [25 40] INFO [2022-01-10 12:53:12,458] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,459] jepsen worker 14 - jepsen.util 104 :ok :write [4 35] INFO [2022-01-10 12:53:12,459] jepsen worker 77 - jepsen.util 167 :ok :read [25 40] INFO [2022-01-10 12:53:12,460] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,460] jepsen worker 7 - jepsen.util 7 :ok :write [2 13] INFO [2022-01-10 12:53:12,460] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,461] jepsen worker 14 - jepsen.util 104 :ok :read [4 35] INFO [2022-01-10 12:53:12,461] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:12,461] jepsen worker 77 - jepsen.util 167 :ok :read [25 40] INFO [2022-01-10 12:53:12,461] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,461] jepsen worker 7 - jepsen.util 7 :ok :read [2 13] INFO [2022-01-10 12:53:12,462] jepsen worker 77 - jepsen.util 167 :invoke :write [25 29] INFO [2022-01-10 12:53:12,462] jepsen worker 7 - jepsen.util 7 :invoke :write [2 42] INFO [2022-01-10 12:53:12,463] jepsen worker 14 - jepsen.util 104 :ok :read [4 35] INFO [2022-01-10 12:53:12,464] jepsen worker 14 - jepsen.util 104 :invoke :write [4 15] INFO [2022-01-10 12:53:12,467] jepsen worker 52 - jepsen.util 52 :ok :write [17 46] INFO [2022-01-10 12:53:12,467] jepsen worker 87 - jepsen.util 87 :ok :write [29 38] INFO [2022-01-10 12:53:12,468] jepsen worker 87 - jepsen.util 87 :invoke :write [29 6] INFO [2022-01-10 12:53:12,468] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,469] jepsen worker 22 - jepsen.util 112 :ok :write [7 15] INFO [2022-01-10 12:53:12,470] jepsen worker 52 - jepsen.util 52 :ok :read [17 46] INFO [2022-01-10 12:53:12,470] jepsen worker 22 - jepsen.util 112 :invoke :write [7 9] INFO [2022-01-10 12:53:12,470] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,471] jepsen worker 52 - jepsen.util 52 :ok :read [17 46] INFO [2022-01-10 12:53:12,472] jepsen worker 52 - jepsen.util 52 :invoke :write [17 27] INFO [2022-01-10 12:53:12,473] jepsen worker 41 - jepsen.util 41 :ok :write [13 35] INFO [2022-01-10 12:53:12,474] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,475] jepsen worker 41 - jepsen.util 41 :ok :read [13 35] INFO [2022-01-10 12:53:12,475] jepsen worker 41 - jepsen.util 41 :invoke :write [13 15] INFO [2022-01-10 12:53:12,476] jepsen worker 15 - jepsen.util 15 :ok :write [5 36] INFO [2022-01-10 12:53:12,477] jepsen worker 15 - jepsen.util 15 :invoke :write [5 30] INFO [2022-01-10 12:53:12,477] jepsen worker 36 - jepsen.util 36 :ok :write [12 35] INFO [2022-01-10 12:53:12,478] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:12,480] jepsen worker 36 - jepsen.util 36 :ok :read [12 35] INFO [2022-01-10 12:53:12,480] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:12,481] jepsen worker 62 - jepsen.util 152 :ok :write [20 12] INFO [2022-01-10 12:53:12,481] jepsen worker 62 - jepsen.util 152 :invoke :write [20 31] INFO [2022-01-10 12:53:12,481] jepsen worker 36 - jepsen.util 36 :ok :read [12 35] INFO [2022-01-10 12:53:12,482] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:12,482] jepsen worker 1 - jepsen.util 1 :ok :write [0 27] INFO [2022-01-10 12:53:12,483] jepsen worker 1 - jepsen.util 1 :invoke :write [0 30] INFO [2022-01-10 12:53:12,483] jepsen worker 36 - jepsen.util 36 :ok :read [12 35] INFO [2022-01-10 12:53:12,484] jepsen worker 36 - jepsen.util 36 :invoke :write [12 47] INFO [2022-01-10 12:53:12,485] jepsen worker 35 - jepsen.util 125 :ok :write [11 7] INFO [2022-01-10 12:53:12,485] jepsen worker 3 - jepsen.util 3 :ok :write [1 17] INFO [2022-01-10 12:53:12,485] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:12,486] jepsen worker 3 - jepsen.util 3 :invoke :write [1 2] INFO [2022-01-10 12:53:12,487] jepsen worker 35 - jepsen.util 125 :ok :read [11 7] INFO [2022-01-10 12:53:12,487] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:12,488] jepsen worker 35 - jepsen.util 125 :ok :read [11 7] INFO [2022-01-10 12:53:12,488] jepsen worker 35 - jepsen.util 125 :invoke :write [11 37] INFO [2022-01-10 12:53:12,491] jepsen worker 49 - jepsen.util 139 :ok :write [16 47] INFO [2022-01-10 12:53:12,491] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:12,492] jepsen worker 49 - jepsen.util 139 :ok :read [16 47] INFO [2022-01-10 12:53:12,493] jepsen worker 49 - jepsen.util 139 :invoke :write [16 32] INFO [2022-01-10 12:53:12,494] jepsen worker 84 - jepsen.util 84 :ok :write [28 27] INFO [2022-01-10 12:53:12,495] jepsen worker 84 - jepsen.util 84 :invoke :write [28 33] INFO [2022-01-10 12:53:12,495] jepsen worker 83 - jepsen.util 173 :ok :write [27 30] INFO [2022-01-10 12:53:12,496] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:12,497] jepsen worker 83 - jepsen.util 173 :ok :read [27 30] INFO [2022-01-10 12:53:12,497] jepsen worker 83 - jepsen.util 173 :invoke :write [27 3] INFO [2022-01-10 12:53:12,499] jepsen worker 43 - jepsen.util 43 :ok :write [14 13] INFO [2022-01-10 12:53:12,500] jepsen worker 43 - jepsen.util 43 :invoke :write [14 7] INFO [2022-01-10 12:53:12,500] jepsen worker 78 - jepsen.util 78 :ok :write [26 4] INFO [2022-01-10 12:53:12,501] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,502] jepsen worker 78 - jepsen.util 78 :ok :read [26 4] INFO [2022-01-10 12:53:12,502] jepsen worker 78 - jepsen.util 78 :invoke :write [26 32] INFO [2022-01-10 12:53:12,507] jepsen worker 71 - jepsen.util 71 :ok :write [23 4] INFO [2022-01-10 12:53:12,507] jepsen worker 71 - jepsen.util 71 :invoke :write [23 42] INFO [2022-01-10 12:53:12,511] jepsen worker 77 - jepsen.util 167 :ok :write [25 29] INFO [2022-01-10 12:53:12,511] jepsen worker 77 - jepsen.util 167 :invoke :write [25 0] INFO [2022-01-10 12:53:12,513] jepsen worker 14 - jepsen.util 104 :ok :write [4 15] INFO [2022-01-10 12:53:12,513] jepsen worker 14 - jepsen.util 104 :invoke :write [4 16] INFO [2022-01-10 12:53:12,518] jepsen worker 52 - jepsen.util 52 :ok :write [17 27] INFO [2022-01-10 12:53:12,519] jepsen worker 52 - jepsen.util 52 :invoke :write [17 36] INFO [2022-01-10 12:53:12,523] jepsen worker 22 - jepsen.util 112 :ok :write [7 9] INFO [2022-01-10 12:53:12,524] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,525] jepsen worker 22 - jepsen.util 112 :ok :read [7 9] INFO [2022-01-10 12:53:12,525] jepsen worker 22 - jepsen.util 112 :invoke :write [7 9] INFO [2022-01-10 12:53:12,534] jepsen worker 62 - jepsen.util 152 :ok :write [20 31] INFO [2022-01-10 12:53:12,535] jepsen worker 62 - jepsen.util 152 :invoke :write [20 34] INFO [2022-01-10 12:53:12,536] jepsen worker 78 - jepsen.util 78 :ok :write [26 32] INFO [2022-01-10 12:53:12,536] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,537] jepsen worker 78 - jepsen.util 78 :ok :read [26 32] INFO [2022-01-10 12:53:12,538] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,538] jepsen worker 78 - jepsen.util 78 :ok :read [26 32] INFO [2022-01-10 12:53:12,539] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,540] jepsen worker 78 - jepsen.util 78 :ok :read [26 32] INFO [2022-01-10 12:53:12,540] jepsen worker 78 - jepsen.util 78 :invoke :write [26 30] INFO [2022-01-10 12:53:12,542] jepsen worker 71 - jepsen.util 71 :ok :write [23 42] INFO [2022-01-10 12:53:12,542] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,543] jepsen worker 71 - jepsen.util 71 :ok :read [23 42] INFO [2022-01-10 12:53:12,544] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,544] jepsen worker 71 - jepsen.util 71 :ok :read [23 42] INFO [2022-01-10 12:53:12,545] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,545] jepsen worker 14 - jepsen.util 104 :ok :write [4 16] INFO [2022-01-10 12:53:12,546] jepsen worker 71 - jepsen.util 71 :ok :read [23 42] INFO [2022-01-10 12:53:12,546] jepsen worker 14 - jepsen.util 104 :invoke :write [4 40] INFO [2022-01-10 12:53:12,547] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,548] jepsen worker 71 - jepsen.util 71 :ok :read [23 42] INFO [2022-01-10 12:53:12,548] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,549] jepsen worker 71 - jepsen.util 71 :ok :read [23 42] INFO [2022-01-10 12:53:12,549] jepsen worker 52 - jepsen.util 52 :ok :write [17 36] INFO [2022-01-10 12:53:12,550] jepsen worker 71 - jepsen.util 71 :invoke :write [23 33] INFO [2022-01-10 12:53:12,550] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,551] jepsen worker 52 - jepsen.util 52 :ok :read [17 36] INFO [2022-01-10 12:53:12,551] jepsen worker 52 - jepsen.util 52 :invoke :write [17 49] INFO [2022-01-10 12:53:12,552] jepsen worker 78 - jepsen.util 78 :ok :write [26 30] INFO [2022-01-10 12:53:12,552] jepsen worker 43 - jepsen.util 43 :ok :write [14 7] INFO [2022-01-10 12:53:12,553] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,553] jepsen worker 43 - jepsen.util 43 :invoke :write [14 35] INFO [2022-01-10 12:53:12,554] jepsen worker 78 - jepsen.util 78 :ok :read [26 30] INFO [2022-01-10 12:53:12,554] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,555] jepsen worker 78 - jepsen.util 78 :ok :read [26 30] INFO [2022-01-10 12:53:12,556] jepsen worker 78 - jepsen.util 78 :invoke :write [26 10] INFO [2022-01-10 12:53:12,559] jepsen worker 14 - jepsen.util 104 :ok :write [4 40] INFO [2022-01-10 12:53:12,559] jepsen worker 14 - jepsen.util 104 :invoke :write [4 46] INFO [2022-01-10 12:53:12,560] jepsen worker 77 - jepsen.util 167 :ok :write [25 0] INFO [2022-01-10 12:53:12,560] jepsen worker 77 - jepsen.util 167 :invoke :write [25 36] INFO [2022-01-10 12:53:12,563] jepsen worker 71 - jepsen.util 71 :ok :write [23 33] INFO [2022-01-10 12:53:12,563] jepsen worker 52 - jepsen.util 52 :ok :write [17 49] INFO [2022-01-10 12:53:12,564] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,565] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,565] jepsen worker 52 - jepsen.util 52 :ok :read [17 49] INFO [2022-01-10 12:53:12,566] jepsen worker 71 - jepsen.util 71 :ok :read [23 33] INFO [2022-01-10 12:53:12,566] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,567] jepsen worker 22 - jepsen.util 112 :ok :write [7 9] INFO [2022-01-10 12:53:12,567] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,567] jepsen worker 52 - jepsen.util 52 :ok :read [17 49] INFO [2022-01-10 12:53:12,568] jepsen worker 71 - jepsen.util 71 :ok :read [23 33] INFO [2022-01-10 12:53:12,568] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,569] jepsen worker 71 - jepsen.util 71 :invoke :write [23 32] INFO [2022-01-10 12:53:12,569] jepsen worker 22 - jepsen.util 112 :ok :read [7 9] INFO [2022-01-10 12:53:12,569] jepsen worker 52 - jepsen.util 52 :invoke :write [17 37] INFO [2022-01-10 12:53:12,570] jepsen worker 22 - jepsen.util 112 :invoke :write [7 7] INFO [2022-01-10 12:53:12,572] jepsen worker 78 - jepsen.util 78 :ok :write [26 10] INFO [2022-01-10 12:53:12,572] jepsen worker 78 - jepsen.util 78 :invoke :write [26 43] INFO [2022-01-10 12:53:12,574] jepsen worker 62 - jepsen.util 152 :ok :write [20 34] INFO [2022-01-10 12:53:12,574] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,575] jepsen worker 62 - jepsen.util 152 :ok :read [20 34] INFO [2022-01-10 12:53:12,576] jepsen worker 62 - jepsen.util 152 :invoke :write [20 2] INFO [2022-01-10 12:53:12,580] jepsen worker 43 - jepsen.util 43 :ok :write [14 35] INFO [2022-01-10 12:53:12,581] jepsen worker 43 - jepsen.util 43 :invoke :write [14 24] INFO [2022-01-10 12:53:12,581] jepsen worker 14 - jepsen.util 104 :ok :write [4 46] INFO [2022-01-10 12:53:12,582] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,582] jepsen worker 14 - jepsen.util 104 :ok :read [4 46] INFO [2022-01-10 12:53:12,583] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,584] jepsen worker 14 - jepsen.util 104 :ok :read [4 46] INFO [2022-01-10 12:53:12,584] jepsen worker 14 - jepsen.util 104 :invoke :write [4 36] INFO [2022-01-10 12:53:12,587] jepsen worker 77 - jepsen.util 167 :ok :write [25 36] INFO [2022-01-10 12:53:12,587] jepsen worker 77 - jepsen.util 167 :invoke :write [25 34] INFO [2022-01-10 12:53:12,589] jepsen worker 71 - jepsen.util 71 :ok :write [23 32] INFO [2022-01-10 12:53:12,589] jepsen worker 71 - jepsen.util 71 :invoke :write [23 31] INFO [2022-01-10 12:53:12,593] jepsen worker 22 - jepsen.util 112 :ok :write [7 7] INFO [2022-01-10 12:53:12,594] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,595] jepsen worker 22 - jepsen.util 112 :ok :read [7 7] INFO [2022-01-10 12:53:12,595] jepsen worker 22 - jepsen.util 112 :invoke :write [7 13] INFO [2022-01-10 12:53:12,598] jepsen worker 52 - jepsen.util 52 :ok :write [17 37] INFO [2022-01-10 12:53:12,599] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,599] jepsen worker 62 - jepsen.util 152 :ok :write [20 2] INFO [2022-01-10 12:53:12,599] jepsen worker 67 - jepsen.util 67 :ok :write [22 0] INFO [2022-01-10 12:53:12,599] jepsen worker 62 - jepsen.util 152 :invoke :write [20 31] INFO [2022-01-10 12:53:12,600] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:12,600] jepsen worker 67 - jepsen.util 67 :invoke :write [22 42] INFO [2022-01-10 12:53:12,600] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,601] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:12,602] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,603] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:12,603] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,603] jepsen worker 78 - jepsen.util 78 :ok :write [26 43] INFO [2022-01-10 12:53:12,604] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:12,604] jepsen worker 78 - jepsen.util 78 :invoke :write [26 18] INFO [2022-01-10 12:53:12,605] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,606] jepsen worker 36 - jepsen.util 36 :ok :write [12 47] INFO [2022-01-10 12:53:12,606] jepsen worker 43 - jepsen.util 43 :ok :write [14 24] INFO [2022-01-10 12:53:12,606] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:12,606] jepsen worker 7 - jepsen.util 7 :ok :write [2 42] INFO [2022-01-10 12:53:12,607] jepsen worker 36 - jepsen.util 36 :invoke :write [12 32] INFO [2022-01-10 12:53:12,608] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,608] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,608] jepsen worker 43 - jepsen.util 43 :ok :read [14 24] INFO [2022-01-10 12:53:12,609] jepsen worker 14 - jepsen.util 104 :ok :write [4 36] INFO [2022-01-10 12:53:12,609] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:12,610] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,610] jepsen worker 14 - jepsen.util 104 :invoke :write [4 4] INFO [2022-01-10 12:53:12,610] jepsen worker 43 - jepsen.util 43 :ok :read [14 24] INFO [2022-01-10 12:53:12,611] jepsen worker 3 - jepsen.util 3 :ok :write [1 2] INFO [2022-01-10 12:53:12,611] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:12,611] jepsen worker 77 - jepsen.util 167 :ok :write [25 34] INFO [2022-01-10 12:53:12,612] jepsen worker 89 - jepsen.util 179 :ok :write [29 18] INFO [2022-01-10 12:53:12,612] jepsen worker 7 - jepsen.util 7 :ok :read [2 42] INFO [2022-01-10 12:53:12,612] jepsen worker 43 - jepsen.util 43 :invoke :write [14 45] INFO [2022-01-10 12:53:12,613] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:12,614] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,614] jepsen worker 3 - jepsen.util 3 :ok :read [1 2] INFO [2022-01-10 12:53:12,614] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:12,615] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,615] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:12,615] jepsen worker 77 - jepsen.util 167 :ok :read [25 34] INFO [2022-01-10 12:53:12,616] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:12,616] jepsen worker 85 - jepsen.util 175 :ok :write [28 19] INFO [2022-01-10 12:53:12,616] jepsen worker 41 - jepsen.util 41 :ok :write [13 15] INFO [2022-01-10 12:53:12,617] jepsen worker 89 - jepsen.util 179 :ok :read [29 18] INFO [2022-01-10 12:53:12,617] jepsen worker 7 - jepsen.util 7 :ok :read [2 42] INFO [2022-01-10 12:53:12,617] jepsen worker 52 - jepsen.util 52 :invoke :write [17 31] INFO [2022-01-10 12:53:12,618] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:12,619] jepsen worker 89 - jepsen.util 179 :invoke :write [29 9] INFO [2022-01-10 12:53:12,619] jepsen worker 85 - jepsen.util 175 :ok :read [28 19] INFO [2022-01-10 12:53:12,619] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,620] jepsen worker 85 - jepsen.util 175 :invoke :write [28 12] INFO [2022-01-10 12:53:12,620] jepsen worker 77 - jepsen.util 167 :ok :read [25 34] INFO [2022-01-10 12:53:12,621] jepsen worker 3 - jepsen.util 3 :invoke :write [1 27] INFO [2022-01-10 12:53:12,622] jepsen worker 7 - jepsen.util 7 :invoke :write [2 7] INFO [2022-01-10 12:53:12,622] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,623] jepsen worker 77 - jepsen.util 167 :invoke :write [25 41] INFO [2022-01-10 12:53:12,623] jepsen worker 41 - jepsen.util 41 :ok :read [13 15] INFO [2022-01-10 12:53:12,623] jepsen worker 15 - jepsen.util 15 :ok :write [5 30] INFO [2022-01-10 12:53:12,624] jepsen worker 15 - jepsen.util 15 :invoke :write [5 30] INFO [2022-01-10 12:53:12,624] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,625] jepsen worker 41 - jepsen.util 41 :ok :read [13 15] INFO [2022-01-10 12:53:12,625] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,626] jepsen worker 41 - jepsen.util 41 :ok :read [13 15] INFO [2022-01-10 12:53:12,626] jepsen worker 41 - jepsen.util 41 :invoke :write [13 11] INFO [2022-01-10 12:53:12,632] jepsen worker 1 - jepsen.util 1 :ok :write [0 30] INFO [2022-01-10 12:53:12,632] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:12,633] jepsen worker 1 - jepsen.util 1 :ok :read [0 30] INFO [2022-01-10 12:53:12,633] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:12,634] jepsen worker 1 - jepsen.util 1 :ok :read [0 30] INFO [2022-01-10 12:53:12,635] jepsen worker 1 - jepsen.util 1 :invoke :write [0 26] INFO [2022-01-10 12:53:12,638] jepsen worker 35 - jepsen.util 125 :ok :write [11 37] INFO [2022-01-10 12:53:12,638] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:12,639] jepsen worker 35 - jepsen.util 125 :ok :read [11 37] INFO [2022-01-10 12:53:12,640] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:12,640] jepsen worker 35 - jepsen.util 125 :ok :read [11 37] INFO [2022-01-10 12:53:12,641] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:12,642] jepsen worker 35 - jepsen.util 125 :ok :read [11 37] INFO [2022-01-10 12:53:12,642] jepsen worker 49 - jepsen.util 139 :ok :write [16 32] INFO [2022-01-10 12:53:12,642] jepsen worker 35 - jepsen.util 125 :invoke :write [11 8] INFO [2022-01-10 12:53:12,643] jepsen worker 49 - jepsen.util 139 :invoke :write [16 16] INFO [2022-01-10 12:53:12,648] jepsen worker 83 - jepsen.util 173 :ok :write [27 3] INFO [2022-01-10 12:53:12,648] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:12,649] jepsen worker 83 - jepsen.util 173 :ok :read [27 3] INFO [2022-01-10 12:53:12,650] jepsen worker 83 - jepsen.util 173 :invoke :write [27 18] INFO [2022-01-10 12:53:12,735] jepsen worker 22 - jepsen.util 112 :ok :write [7 13] INFO [2022-01-10 12:53:12,736] jepsen worker 22 - jepsen.util 112 :invoke :write [7 41] INFO [2022-01-10 12:53:12,737] jepsen worker 71 - jepsen.util 71 :ok :write [23 31] INFO [2022-01-10 12:53:12,738] jepsen worker 71 - jepsen.util 71 :invoke :write [23 18] INFO [2022-01-10 12:53:12,738] jepsen worker 89 - jepsen.util 179 :ok :write [29 9] INFO [2022-01-10 12:53:12,739] jepsen worker 89 - jepsen.util 179 :invoke :write [29 24] INFO [2022-01-10 12:53:12,741] jepsen worker 62 - jepsen.util 152 :ok :write [20 31] INFO [2022-01-10 12:53:12,742] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,743] jepsen worker 62 - jepsen.util 152 :ok :read [20 31] INFO [2022-01-10 12:53:12,743] jepsen worker 62 - jepsen.util 152 :invoke :write [20 22] INFO [2022-01-10 12:53:12,747] jepsen worker 78 - jepsen.util 78 :ok :write [26 18] INFO [2022-01-10 12:53:12,747] jepsen worker 52 - jepsen.util 52 :ok :write [17 31] INFO [2022-01-10 12:53:12,748] jepsen worker 67 - jepsen.util 67 :ok :write [22 42] INFO [2022-01-10 12:53:12,749] jepsen worker 52 - jepsen.util 52 :invoke :write [17 13] INFO [2022-01-10 12:53:12,749] jepsen worker 78 - jepsen.util 78 :invoke :write [26 48] INFO [2022-01-10 12:53:12,750] jepsen worker 67 - jepsen.util 67 :invoke :write [22 23] INFO [2022-01-10 12:53:12,751] jepsen worker 43 - jepsen.util 43 :ok :write [14 45] INFO [2022-01-10 12:53:12,752] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,753] jepsen worker 43 - jepsen.util 43 :ok :read [14 45] INFO [2022-01-10 12:53:12,753] jepsen worker 43 - jepsen.util 43 :invoke :write [14 19] INFO [2022-01-10 12:53:12,753] jepsen worker 36 - jepsen.util 36 :ok :write [12 32] INFO [2022-01-10 12:53:12,754] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:12,755] jepsen worker 36 - jepsen.util 36 :ok :read [12 32] INFO [2022-01-10 12:53:12,756] jepsen worker 36 - jepsen.util 36 :invoke :write [12 19] INFO [2022-01-10 12:53:12,756] jepsen worker 8 - jepsen.util 8 :ok :write [2 42] INFO [2022-01-10 12:53:12,757] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:12,758] jepsen worker 8 - jepsen.util 8 :ok :read [2 42] INFO [2022-01-10 12:53:12,759] jepsen worker 8 - jepsen.util 8 :invoke :write [2 37] INFO [2022-01-10 12:53:12,759] jepsen worker 41 - jepsen.util 41 :ok :write [13 11] INFO [2022-01-10 12:53:12,760] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,761] jepsen worker 41 - jepsen.util 41 :ok :read [13 11] INFO [2022-01-10 12:53:12,761] jepsen worker 14 - jepsen.util 104 :ok :write [4 4] INFO [2022-01-10 12:53:12,761] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,762] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,762] jepsen worker 41 - jepsen.util 41 :ok :read [13 11] INFO [2022-01-10 12:53:12,763] jepsen worker 14 - jepsen.util 104 :ok :read [4 4] INFO [2022-01-10 12:53:12,763] jepsen worker 41 - jepsen.util 41 :invoke :write [13 9] INFO [2022-01-10 12:53:12,763] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,764] jepsen worker 77 - jepsen.util 167 :ok :write [25 41] INFO [2022-01-10 12:53:12,764] jepsen worker 14 - jepsen.util 104 :ok :read [4 4] INFO [2022-01-10 12:53:12,764] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,765] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,765] jepsen worker 77 - jepsen.util 167 :ok :read [25 41] INFO [2022-01-10 12:53:12,766] jepsen worker 14 - jepsen.util 104 :ok :read [4 4] INFO [2022-01-10 12:53:12,766] jepsen worker 77 - jepsen.util 167 :invoke :write [25 6] INFO [2022-01-10 12:53:12,767] jepsen worker 1 - jepsen.util 1 :ok :write [0 26] INFO [2022-01-10 12:53:12,767] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,767] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:12,768] jepsen worker 14 - jepsen.util 104 :ok :read [4 4] INFO [2022-01-10 12:53:12,768] jepsen worker 1 - jepsen.util 1 :ok :read [0 26] INFO [2022-01-10 12:53:12,768] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,768] jepsen worker 1 - jepsen.util 1 :invoke :write [0 1] INFO [2022-01-10 12:53:12,769] jepsen worker 14 - jepsen.util 104 :ok :read [4 4] INFO [2022-01-10 12:53:12,770] jepsen worker 14 - jepsen.util 104 :invoke :write [4 21] INFO [2022-01-10 12:53:12,770] jepsen worker 15 - jepsen.util 15 :ok :write [5 30] INFO [2022-01-10 12:53:12,770] jepsen worker 3 - jepsen.util 3 :ok :write [1 27] INFO [2022-01-10 12:53:12,771] jepsen worker 15 - jepsen.util 15 :invoke :write [5 39] INFO [2022-01-10 12:53:12,771] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:12,772] jepsen worker 3 - jepsen.util 3 :ok :read [1 27] INFO [2022-01-10 12:53:12,772] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:12,773] jepsen worker 3 - jepsen.util 3 :ok :read [1 27] INFO [2022-01-10 12:53:12,774] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:12,774] jepsen worker 35 - jepsen.util 125 :ok :write [11 8] INFO [2022-01-10 12:53:12,775] jepsen worker 3 - jepsen.util 3 :ok :read [1 27] INFO [2022-01-10 12:53:12,775] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:12,776] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:12,776] jepsen worker 3 - jepsen.util 3 :ok :read [1 27] INFO [2022-01-10 12:53:12,776] jepsen worker 3 - jepsen.util 3 :invoke :write [1 30] INFO [2022-01-10 12:53:12,777] jepsen worker 35 - jepsen.util 125 :ok :read [11 8] INFO [2022-01-10 12:53:12,777] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:12,778] jepsen worker 35 - jepsen.util 125 :ok :read [11 8] INFO [2022-01-10 12:53:12,778] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:12,779] jepsen worker 35 - jepsen.util 125 :ok :read [11 8] INFO [2022-01-10 12:53:12,780] jepsen worker 35 - jepsen.util 125 :invoke :write [11 17] INFO [2022-01-10 12:53:12,780] jepsen worker 85 - jepsen.util 175 :ok :write [28 12] INFO [2022-01-10 12:53:12,780] jepsen worker 49 - jepsen.util 139 :ok :write [16 16] INFO [2022-01-10 12:53:12,781] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:12,781] jepsen worker 49 - jepsen.util 139 :invoke :write [16 0] INFO [2022-01-10 12:53:12,782] jepsen worker 85 - jepsen.util 175 :ok :read [28 12] INFO [2022-01-10 12:53:12,782] jepsen worker 85 - jepsen.util 175 :invoke :write [28 8] INFO [2022-01-10 12:53:12,784] jepsen worker 83 - jepsen.util 173 :ok :write [27 18] INFO [2022-01-10 12:53:12,784] jepsen worker 89 - jepsen.util 179 :ok :write [29 24] INFO [2022-01-10 12:53:12,784] jepsen worker 83 - jepsen.util 173 :invoke :write [27 40] INFO [2022-01-10 12:53:12,785] jepsen worker 89 - jepsen.util 179 :invoke :write [29 44] INFO [2022-01-10 12:53:12,786] jepsen worker 22 - jepsen.util 112 :ok :write [7 41] INFO [2022-01-10 12:53:12,786] jepsen worker 71 - jepsen.util 71 :ok :write [23 18] INFO [2022-01-10 12:53:12,786] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,787] jepsen worker 71 - jepsen.util 71 :invoke :write [23 30] INFO [2022-01-10 12:53:12,787] jepsen worker 22 - jepsen.util 112 :ok :read [7 41] INFO [2022-01-10 12:53:12,788] jepsen worker 22 - jepsen.util 112 :invoke :write [7 44] INFO [2022-01-10 12:53:12,795] jepsen worker 62 - jepsen.util 152 :ok :write [20 22] INFO [2022-01-10 12:53:12,796] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,797] jepsen worker 62 - jepsen.util 152 :ok :read [20 22] INFO [2022-01-10 12:53:12,797] jepsen worker 67 - jepsen.util 67 :ok :write [22 23] INFO [2022-01-10 12:53:12,797] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,798] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:12,798] jepsen worker 78 - jepsen.util 78 :ok :write [26 48] INFO [2022-01-10 12:53:12,798] jepsen worker 62 - jepsen.util 152 :ok :read [20 22] INFO [2022-01-10 12:53:12,798] jepsen worker 67 - jepsen.util 67 :ok :read [22 23] INFO [2022-01-10 12:53:12,799] jepsen worker 62 - jepsen.util 152 :invoke :write [20 16] INFO [2022-01-10 12:53:12,799] jepsen worker 78 - jepsen.util 78 :invoke :write [26 47] INFO [2022-01-10 12:53:12,800] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:12,801] jepsen worker 67 - jepsen.util 67 :ok :read [22 23] INFO [2022-01-10 12:53:12,801] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:12,802] jepsen worker 43 - jepsen.util 43 :ok :write [14 19] INFO [2022-01-10 12:53:12,802] jepsen worker 67 - jepsen.util 67 :ok :read [22 23] INFO [2022-01-10 12:53:12,803] jepsen worker 67 - jepsen.util 67 :invoke :write [22 33] INFO [2022-01-10 12:53:12,803] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,804] jepsen worker 43 - jepsen.util 43 :ok :read [14 19] INFO [2022-01-10 12:53:12,804] jepsen worker 43 - jepsen.util 43 :invoke :write [14 5] INFO [2022-01-10 12:53:12,805] jepsen worker 52 - jepsen.util 52 :ok :write [17 13] INFO [2022-01-10 12:53:12,806] jepsen worker 52 - jepsen.util 52 :invoke :write [17 32] INFO [2022-01-10 12:53:12,815] jepsen worker 77 - jepsen.util 167 :ok :write [25 6] INFO [2022-01-10 12:53:12,815] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,816] jepsen worker 77 - jepsen.util 167 :ok :read [25 6] INFO [2022-01-10 12:53:12,817] jepsen worker 14 - jepsen.util 104 :ok :write [4 21] INFO [2022-01-10 12:53:12,817] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,817] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,818] jepsen worker 77 - jepsen.util 167 :ok :read [25 6] INFO [2022-01-10 12:53:12,818] jepsen worker 14 - jepsen.util 104 :ok :read [4 21] INFO [2022-01-10 12:53:12,818] jepsen worker 77 - jepsen.util 167 :invoke :write [25 16] INFO [2022-01-10 12:53:12,819] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,820] jepsen worker 14 - jepsen.util 104 :ok :read [4 21] INFO [2022-01-10 12:53:12,821] jepsen worker 14 - jepsen.util 104 :invoke :write [4 23] INFO [2022-01-10 12:53:12,837] jepsen worker 22 - jepsen.util 112 :ok :write [7 44] INFO [2022-01-10 12:53:12,837] jepsen worker 71 - jepsen.util 71 :ok :write [23 30] INFO [2022-01-10 12:53:12,838] jepsen worker 22 - jepsen.util 112 :invoke :write [7 44] INFO [2022-01-10 12:53:12,839] jepsen worker 71 - jepsen.util 71 :invoke :write [23 35] INFO [2022-01-10 12:53:12,844] jepsen worker 62 - jepsen.util 152 :ok :write [20 16] INFO [2022-01-10 12:53:12,844] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:12,845] jepsen worker 62 - jepsen.util 152 :ok :read [20 16] INFO [2022-01-10 12:53:12,845] jepsen worker 62 - jepsen.util 152 :invoke :write [20 22] INFO [2022-01-10 12:53:12,847] jepsen worker 78 - jepsen.util 78 :ok :write [26 47] INFO [2022-01-10 12:53:12,848] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,849] jepsen worker 78 - jepsen.util 78 :ok :read [26 47] INFO [2022-01-10 12:53:12,849] jepsen worker 78 - jepsen.util 78 :invoke :write [26 7] INFO [2022-01-10 12:53:12,854] jepsen worker 52 - jepsen.util 52 :ok :write [17 32] INFO [2022-01-10 12:53:12,855] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,856] jepsen worker 52 - jepsen.util 52 :ok :read [17 32] INFO [2022-01-10 12:53:12,856] jepsen worker 52 - jepsen.util 52 :invoke :write [17 13] INFO [2022-01-10 12:53:12,860] jepsen worker 43 - jepsen.util 43 :ok :write [14 5] INFO [2022-01-10 12:53:12,860] jepsen worker 43 - jepsen.util 43 :invoke :write [14 9] INFO [2022-01-10 12:53:12,860] jepsen worker 14 - jepsen.util 104 :ok :write [4 23] INFO [2022-01-10 12:53:12,861] jepsen worker 14 - jepsen.util 104 :invoke :write [4 17] INFO [2022-01-10 12:53:12,867] jepsen worker 77 - jepsen.util 167 :ok :write [25 16] INFO [2022-01-10 12:53:12,867] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,868] jepsen worker 71 - jepsen.util 71 :ok :write [23 35] INFO [2022-01-10 12:53:12,868] jepsen worker 77 - jepsen.util 167 :ok :read [25 16] INFO [2022-01-10 12:53:12,868] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,869] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,869] jepsen worker 71 - jepsen.util 71 :ok :read [23 35] INFO [2022-01-10 12:53:12,870] jepsen worker 77 - jepsen.util 167 :ok :read [25 16] INFO [2022-01-10 12:53:12,870] jepsen worker 71 - jepsen.util 71 :invoke :write [23 3] INFO [2022-01-10 12:53:12,870] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,871] jepsen worker 77 - jepsen.util 167 :ok :read [25 16] INFO [2022-01-10 12:53:12,872] jepsen worker 77 - jepsen.util 167 :invoke :write [25 20] INFO [2022-01-10 12:53:12,874] jepsen worker 22 - jepsen.util 112 :ok :write [7 44] INFO [2022-01-10 12:53:12,875] jepsen worker 78 - jepsen.util 78 :ok :write [26 7] INFO [2022-01-10 12:53:12,875] jepsen worker 22 - jepsen.util 112 :invoke :write [7 14] INFO [2022-01-10 12:53:12,876] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,876] jepsen worker 78 - jepsen.util 78 :ok :read [26 7] INFO [2022-01-10 12:53:12,877] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,878] jepsen worker 78 - jepsen.util 78 :ok :read [26 7] INFO [2022-01-10 12:53:12,879] jepsen worker 78 - jepsen.util 78 :invoke :write [26 16] INFO [2022-01-10 12:53:12,880] jepsen worker 62 - jepsen.util 152 :ok :write [20 22] INFO [2022-01-10 12:53:12,881] jepsen worker 62 - jepsen.util 152 :invoke :write [20 30] INFO [2022-01-10 12:53:12,882] jepsen worker 52 - jepsen.util 52 :ok :write [17 13] INFO [2022-01-10 12:53:12,882] jepsen worker 52 - jepsen.util 52 :invoke :write [17 23] INFO [2022-01-10 12:53:12,886] jepsen worker 43 - jepsen.util 43 :ok :write [14 9] INFO [2022-01-10 12:53:12,886] jepsen worker 36 - jepsen.util 36 :ok :write [12 19] INFO [2022-01-10 12:53:12,886] jepsen worker 14 - jepsen.util 104 :ok :write [4 17] INFO [2022-01-10 12:53:12,887] jepsen worker 43 - jepsen.util 43 :invoke :write [14 46] INFO [2022-01-10 12:53:12,888] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,888] jepsen worker 36 - jepsen.util 36 :invoke :write [12 37] INFO [2022-01-10 12:53:12,888] jepsen worker 41 - jepsen.util 41 :ok :write [13 9] INFO [2022-01-10 12:53:12,889] jepsen worker 3 - jepsen.util 3 :ok :write [1 30] INFO [2022-01-10 12:53:12,889] jepsen worker 14 - jepsen.util 104 :ok :read [4 17] INFO [2022-01-10 12:53:12,889] jepsen worker 41 - jepsen.util 41 :invoke :write [13 42] INFO [2022-01-10 12:53:12,890] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:12,891] jepsen worker 3 - jepsen.util 3 :invoke :write [1 4] INFO [2022-01-10 12:53:12,891] jepsen worker 14 - jepsen.util 104 :ok :read [4 17] INFO [2022-01-10 12:53:12,891] jepsen worker 14 - jepsen.util 104 :invoke :write [4 49] INFO [2022-01-10 12:53:12,893] jepsen worker 77 - jepsen.util 167 :ok :write [25 20] INFO [2022-01-10 12:53:12,893] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,894] jepsen worker 77 - jepsen.util 167 :ok :read [25 20] INFO [2022-01-10 12:53:12,895] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,895] jepsen worker 85 - jepsen.util 175 :ok :write [28 8] INFO [2022-01-10 12:53:12,896] jepsen worker 77 - jepsen.util 167 :ok :read [25 20] INFO [2022-01-10 12:53:12,896] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:12,896] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,897] jepsen worker 85 - jepsen.util 175 :ok :read [28 8] INFO [2022-01-10 12:53:12,897] jepsen worker 71 - jepsen.util 71 :ok :write [23 3] INFO [2022-01-10 12:53:12,897] jepsen worker 8 - jepsen.util 8 :ok :write [2 37] INFO [2022-01-10 12:53:12,897] jepsen worker 77 - jepsen.util 167 :ok :read [25 20] INFO [2022-01-10 12:53:12,898] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:12,899] jepsen worker 71 - jepsen.util 71 :ok :read [23 3] INFO [2022-01-10 12:53:12,899] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:12,900] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:12,900] jepsen worker 22 - jepsen.util 112 :ok :write [7 14] INFO [2022-01-10 12:53:12,900] jepsen worker 8 - jepsen.util 8 :ok :read [2 37] INFO [2022-01-10 12:53:12,900] jepsen worker 71 - jepsen.util 71 :invoke :write [23 20] INFO [2022-01-10 12:53:12,901] jepsen worker 85 - jepsen.util 175 :ok :read [28 8] INFO [2022-01-10 12:53:12,901] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,901] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:12,902] jepsen worker 8 - jepsen.util 8 :invoke :write [2 21] INFO [2022-01-10 12:53:12,902] jepsen worker 77 - jepsen.util 167 :ok :read [25 20] INFO [2022-01-10 12:53:12,902] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:12,902] jepsen worker 85 - jepsen.util 175 :ok :read [28 8] INFO [2022-01-10 12:53:12,903] jepsen worker 85 - jepsen.util 175 :invoke :write [28 34] INFO [2022-01-10 12:53:12,903] jepsen worker 77 - jepsen.util 167 :invoke :write [25 27] INFO [2022-01-10 12:53:12,903] jepsen worker 22 - jepsen.util 112 :ok :read [7 14] INFO [2022-01-10 12:53:12,904] jepsen worker 22 - jepsen.util 112 :invoke :write [7 14] INFO [2022-01-10 12:53:12,905] jepsen worker 1 - jepsen.util 1 :ok :write [0 1] INFO [2022-01-10 12:53:12,905] jepsen worker 78 - jepsen.util 78 :ok :write [26 16] INFO [2022-01-10 12:53:12,906] jepsen worker 1 - jepsen.util 1 :invoke :write [0 8] INFO [2022-01-10 12:53:12,906] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:12,908] jepsen worker 78 - jepsen.util 78 :ok :read [26 16] INFO [2022-01-10 12:53:12,908] jepsen worker 62 - jepsen.util 152 :ok :write [20 30] INFO [2022-01-10 12:53:12,908] jepsen worker 78 - jepsen.util 78 :invoke :write [26 26] INFO [2022-01-10 12:53:12,908] jepsen worker 62 - jepsen.util 152 :invoke :write [20 26] INFO [2022-01-10 12:53:12,913] jepsen worker 43 - jepsen.util 43 :ok :write [14 46] INFO [2022-01-10 12:53:12,913] jepsen worker 15 - jepsen.util 15 :ok :write [5 39] INFO [2022-01-10 12:53:12,913] jepsen worker 52 - jepsen.util 52 :ok :write [17 23] INFO [2022-01-10 12:53:12,913] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:12,914] jepsen worker 43 - jepsen.util 43 :invoke :write [14 37] INFO [2022-01-10 12:53:12,914] jepsen worker 15 - jepsen.util 15 :ok :read [5 39] INFO [2022-01-10 12:53:12,915] jepsen worker 52 - jepsen.util 52 :invoke :write [17 11] INFO [2022-01-10 12:53:12,915] jepsen worker 15 - jepsen.util 15 :invoke :write [5 5] INFO [2022-01-10 12:53:12,918] jepsen worker 77 - jepsen.util 167 :ok :write [25 27] INFO [2022-01-10 12:53:12,919] jepsen worker 77 - jepsen.util 167 :invoke :write [25 35] INFO [2022-01-10 12:53:12,919] jepsen worker 36 - jepsen.util 36 :ok :write [12 37] INFO [2022-01-10 12:53:12,920] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:12,921] jepsen worker 35 - jepsen.util 125 :ok :write [11 17] INFO [2022-01-10 12:53:12,921] jepsen worker 36 - jepsen.util 36 :ok :read [12 37] INFO [2022-01-10 12:53:12,921] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:12,922] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:12,922] jepsen worker 36 - jepsen.util 36 :ok :read [12 37] INFO [2022-01-10 12:53:12,923] jepsen worker 35 - jepsen.util 125 :ok :read [11 17] INFO [2022-01-10 12:53:12,923] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:12,923] jepsen worker 35 - jepsen.util 125 :invoke :write [11 38] INFO [2022-01-10 12:53:12,924] jepsen worker 36 - jepsen.util 36 :ok :read [12 37] INFO [2022-01-10 12:53:12,924] jepsen worker 36 - jepsen.util 36 :invoke :write [12 42] INFO [2022-01-10 12:53:12,926] jepsen worker 14 - jepsen.util 104 :ok :write [4 49] INFO [2022-01-10 12:53:12,926] jepsen worker 49 - jepsen.util 139 :ok :write [16 0] INFO [2022-01-10 12:53:12,926] jepsen worker 14 - jepsen.util 104 :invoke :write [4 43] INFO [2022-01-10 12:53:12,927] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:12,928] jepsen worker 49 - jepsen.util 139 :ok :read [16 0] INFO [2022-01-10 12:53:12,928] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:12,929] jepsen worker 49 - jepsen.util 139 :ok :read [16 0] INFO [2022-01-10 12:53:12,930] jepsen worker 49 - jepsen.util 139 :invoke :write [16 19] INFO [2022-01-10 12:53:12,931] jepsen worker 22 - jepsen.util 112 :ok :write [7 14] INFO [2022-01-10 12:53:12,932] jepsen worker 22 - jepsen.util 112 :invoke :write [7 34] INFO [2022-01-10 12:53:12,933] jepsen worker 3 - jepsen.util 3 :ok :write [1 4] INFO [2022-01-10 12:53:12,934] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:12,934] jepsen worker 83 - jepsen.util 173 :ok :write [27 40] INFO [2022-01-10 12:53:12,935] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:12,935] jepsen worker 3 - jepsen.util 3 :ok :read [1 4] INFO [2022-01-10 12:53:12,936] jepsen worker 3 - jepsen.util 3 :invoke :write [1 3] INFO [2022-01-10 12:53:12,936] jepsen worker 83 - jepsen.util 173 :ok :read [27 40] INFO [2022-01-10 12:53:12,937] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:12,937] jepsen worker 83 - jepsen.util 173 :ok :read [27 40] INFO [2022-01-10 12:53:12,938] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:12,939] jepsen worker 83 - jepsen.util 173 :ok :read [27 40] INFO [2022-01-10 12:53:12,939] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:12,940] jepsen worker 71 - jepsen.util 71 :ok :write [23 20] INFO [2022-01-10 12:53:12,940] jepsen worker 83 - jepsen.util 173 :ok :read [27 40] INFO [2022-01-10 12:53:12,940] jepsen worker 71 - jepsen.util 71 :invoke :write [23 21] INFO [2022-01-10 12:53:12,940] jepsen worker 62 - jepsen.util 152 :ok :write [20 26] INFO [2022-01-10 12:53:12,941] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:12,941] jepsen worker 62 - jepsen.util 152 :invoke :write [20 12] INFO [2022-01-10 12:53:12,942] jepsen worker 83 - jepsen.util 173 :ok :read [27 40] INFO [2022-01-10 12:53:12,942] jepsen worker 83 - jepsen.util 173 :invoke :write [27 33] INFO [2022-01-10 12:53:12,943] jepsen worker 89 - jepsen.util 179 :ok :write [29 44] INFO [2022-01-10 12:53:12,943] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:12,944] jepsen worker 89 - jepsen.util 179 :ok :read [29 44] INFO [2022-01-10 12:53:12,945] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:12,946] jepsen worker 89 - jepsen.util 179 :ok :read [29 44] INFO [2022-01-10 12:53:12,947] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:12,947] jepsen worker 89 - jepsen.util 179 :ok :read [29 44] INFO [2022-01-10 12:53:12,948] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:12,948] jepsen worker 43 - jepsen.util 43 :ok :write [14 37] INFO [2022-01-10 12:53:12,948] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,949] jepsen worker 89 - jepsen.util 179 :ok :read [29 44] INFO [2022-01-10 12:53:12,949] jepsen worker 67 - jepsen.util 67 :ok :write [22 33] INFO [2022-01-10 12:53:12,949] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:12,950] jepsen worker 43 - jepsen.util 43 :ok :read [14 37] INFO [2022-01-10 12:53:12,950] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:12,950] jepsen worker 85 - jepsen.util 175 :ok :write [28 34] INFO [2022-01-10 12:53:12,950] jepsen worker 89 - jepsen.util 179 :ok :read [29 44] INFO [2022-01-10 12:53:12,951] jepsen worker 67 - jepsen.util 67 :ok :read [22 33] INFO [2022-01-10 12:53:12,951] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,952] jepsen worker 85 - jepsen.util 175 :invoke :write [28 26] INFO [2022-01-10 12:53:12,952] jepsen worker 43 - jepsen.util 43 :ok :read [14 37] INFO [2022-01-10 12:53:12,952] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:12,952] jepsen worker 67 - jepsen.util 67 :invoke :write [22 26] INFO [2022-01-10 12:53:12,953] jepsen worker 89 - jepsen.util 179 :ok :read [29 44] INFO [2022-01-10 12:53:12,953] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:12,954] jepsen worker 89 - jepsen.util 179 :invoke :write [29 15] INFO [2022-01-10 12:53:12,954] jepsen worker 43 - jepsen.util 43 :ok :read [14 37] INFO [2022-01-10 12:53:12,955] jepsen worker 78 - jepsen.util 78 :ok :write [26 26] INFO [2022-01-10 12:53:12,955] jepsen worker 43 - jepsen.util 43 :invoke :write [14 17] INFO [2022-01-10 12:53:12,955] jepsen worker 78 - jepsen.util 78 :invoke :write [26 20] INFO [2022-01-10 12:53:12,959] jepsen worker 41 - jepsen.util 41 :ok :write [13 42] INFO [2022-01-10 12:53:12,959] jepsen worker 77 - jepsen.util 167 :ok :write [25 35] INFO [2022-01-10 12:53:12,959] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,960] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:12,960] jepsen worker 41 - jepsen.util 41 :ok :read [13 42] INFO [2022-01-10 12:53:12,961] jepsen worker 77 - jepsen.util 167 :ok :read [25 35] INFO [2022-01-10 12:53:12,961] jepsen worker 52 - jepsen.util 52 :ok :write [17 11] INFO [2022-01-10 12:53:12,961] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,962] jepsen worker 77 - jepsen.util 167 :invoke :write [25 40] INFO [2022-01-10 12:53:12,962] jepsen worker 41 - jepsen.util 41 :ok :read [13 42] INFO [2022-01-10 12:53:12,962] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:12,963] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:12,963] jepsen worker 52 - jepsen.util 52 :ok :read [17 11] INFO [2022-01-10 12:53:12,964] jepsen worker 41 - jepsen.util 41 :ok :read [13 42] INFO [2022-01-10 12:53:12,964] jepsen worker 52 - jepsen.util 52 :invoke :write [17 46] INFO [2022-01-10 12:53:12,964] jepsen worker 41 - jepsen.util 41 :invoke :write [13 20] INFO [2022-01-10 12:53:12,966] jepsen worker 8 - jepsen.util 8 :ok :write [2 21] INFO [2022-01-10 12:53:12,966] jepsen worker 36 - jepsen.util 36 :ok :write [12 42] INFO [2022-01-10 12:53:12,966] jepsen worker 8 - jepsen.util 8 :invoke :write [2 3] INFO [2022-01-10 12:53:12,967] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:12,968] jepsen worker 36 - jepsen.util 36 :ok :read [12 42] INFO [2022-01-10 12:53:12,969] jepsen worker 36 - jepsen.util 36 :invoke :write [12 39] INFO [2022-01-10 12:53:12,972] jepsen worker 1 - jepsen.util 1 :ok :write [0 8] INFO [2022-01-10 12:53:12,973] jepsen worker 14 - jepsen.util 104 :ok :write [4 43] INFO [2022-01-10 12:53:12,973] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:12,974] jepsen worker 1 - jepsen.util 1 :ok :read [0 8] INFO [2022-01-10 12:53:12,974] jepsen worker 14 - jepsen.util 104 :invoke :write [4 35] INFO [2022-01-10 12:53:12,975] jepsen worker 1 - jepsen.util 1 :invoke :write [0 25] INFO [2022-01-10 12:53:12,978] jepsen worker 22 - jepsen.util 112 :ok :write [7 34] INFO [2022-01-10 12:53:12,979] jepsen worker 22 - jepsen.util 112 :invoke :write [7 39] INFO [2022-01-10 12:53:12,981] jepsen worker 3 - jepsen.util 3 :ok :write [1 3] INFO [2022-01-10 12:53:12,982] jepsen worker 15 - jepsen.util 15 :ok :write [5 5] INFO [2022-01-10 12:53:12,982] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:12,983] jepsen worker 3 - jepsen.util 3 :ok :read [1 3] INFO [2022-01-10 12:53:12,983] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:12,984] jepsen worker 15 - jepsen.util 15 :ok :read [5 5] INFO [2022-01-10 12:53:12,984] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:12,985] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:12,985] jepsen worker 3 - jepsen.util 3 :ok :read [1 3] INFO [2022-01-10 12:53:12,986] jepsen worker 15 - jepsen.util 15 :ok :read [5 5] INFO [2022-01-10 12:53:12,986] jepsen worker 71 - jepsen.util 71 :ok :write [23 21] INFO [2022-01-10 12:53:12,986] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:12,987] jepsen worker 71 - jepsen.util 71 :invoke :write [23 49] INFO [2022-01-10 12:53:12,987] jepsen worker 3 - jepsen.util 3 :ok :read [1 3] INFO [2022-01-10 12:53:12,988] jepsen worker 3 - jepsen.util 3 :invoke :write [1 17] INFO [2022-01-10 12:53:12,989] jepsen worker 15 - jepsen.util 15 :invoke :write [5 13] INFO [2022-01-10 12:53:12,990] jepsen worker 35 - jepsen.util 125 :ok :write [11 38] INFO [2022-01-10 12:53:12,991] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:12,992] jepsen worker 35 - jepsen.util 125 :ok :read [11 38] INFO [2022-01-10 12:53:12,993] jepsen worker 85 - jepsen.util 175 :ok :write [28 26] INFO [2022-01-10 12:53:12,993] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:12,994] jepsen worker 35 - jepsen.util 125 :ok :read [11 38] INFO [2022-01-10 12:53:12,994] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:12,995] jepsen worker 35 - jepsen.util 125 :invoke :write [11 8] INFO [2022-01-10 12:53:12,995] jepsen worker 85 - jepsen.util 175 :ok :read [28 26] INFO [2022-01-10 12:53:12,996] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:12,996] jepsen worker 78 - jepsen.util 78 :ok :write [26 20] INFO [2022-01-10 12:53:12,997] jepsen worker 85 - jepsen.util 175 :ok :read [28 26] INFO [2022-01-10 12:53:12,997] jepsen worker 78 - jepsen.util 78 :invoke :write [26 37] INFO [2022-01-10 12:53:12,998] jepsen worker 49 - jepsen.util 139 :ok :write [16 19] INFO [2022-01-10 12:53:12,998] jepsen worker 85 - jepsen.util 175 :invoke :write [28 47] INFO [2022-01-10 12:53:12,999] jepsen worker 49 - jepsen.util 139 :invoke :write [16 38] INFO [2022-01-10 12:53:13,001] jepsen worker 52 - jepsen.util 52 :ok :write [17 46] INFO [2022-01-10 12:53:13,002] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,003] jepsen worker 52 - jepsen.util 52 :ok :read [17 46] INFO [2022-01-10 12:53:13,003] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,005] jepsen worker 52 - jepsen.util 52 :ok :read [17 46] INFO [2022-01-10 12:53:13,005] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,006] jepsen worker 52 - jepsen.util 52 :ok :read [17 46] INFO [2022-01-10 12:53:13,006] jepsen worker 36 - jepsen.util 36 :ok :write [12 39] INFO [2022-01-10 12:53:13,007] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,007] jepsen worker 36 - jepsen.util 36 :invoke :write [12 46] INFO [2022-01-10 12:53:13,008] jepsen worker 52 - jepsen.util 52 :ok :read [17 46] INFO [2022-01-10 12:53:13,008] jepsen worker 83 - jepsen.util 173 :ok :write [27 33] INFO [2022-01-10 12:53:13,008] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,009] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:13,009] jepsen worker 52 - jepsen.util 52 :ok :read [17 46] INFO [2022-01-10 12:53:13,010] jepsen worker 83 - jepsen.util 173 :ok :read [27 33] INFO [2022-01-10 12:53:13,010] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,010] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:13,012] jepsen worker 52 - jepsen.util 52 :ok :read [17 46] INFO [2022-01-10 12:53:13,012] jepsen worker 83 - jepsen.util 173 :ok :read [27 33] INFO [2022-01-10 12:53:13,013] jepsen worker 83 - jepsen.util 173 :invoke :write [27 36] INFO [2022-01-10 12:53:13,013] jepsen worker 52 - jepsen.util 52 :invoke :write [17 5] INFO [2022-01-10 12:53:13,023] jepsen worker 89 - jepsen.util 179 :ok :write [29 15] INFO [2022-01-10 12:53:13,023] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,024] jepsen worker 89 - jepsen.util 179 :ok :read [29 15] INFO [2022-01-10 12:53:13,024] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,025] jepsen worker 89 - jepsen.util 179 :ok :read [29 15] INFO [2022-01-10 12:53:13,026] jepsen worker 41 - jepsen.util 41 :ok :write [13 20] INFO [2022-01-10 12:53:13,026] jepsen worker 89 - jepsen.util 179 :invoke :write [29 47] INFO [2022-01-10 12:53:13,026] jepsen worker 41 - jepsen.util 41 :invoke :write [13 18] INFO [2022-01-10 12:53:13,046] jepsen worker 1 - jepsen.util 1 :ok :write [0 25] INFO [2022-01-10 12:53:13,046] jepsen worker 1 - jepsen.util 1 :invoke :write [0 2] INFO [2022-01-10 12:53:13,107] jepsen worker 62 - jepsen.util 152 :ok :write [20 12] INFO [2022-01-10 12:53:13,107] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:13,108] jepsen worker 62 - jepsen.util 152 :ok :read [20 12] INFO [2022-01-10 12:53:13,108] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:13,108] jepsen worker 14 - jepsen.util 104 :ok :write [4 35] INFO [2022-01-10 12:53:13,109] jepsen worker 14 - jepsen.util 104 :invoke :write [4 5] INFO [2022-01-10 12:53:13,109] jepsen worker 62 - jepsen.util 152 :ok :read [20 12] INFO [2022-01-10 12:53:13,110] jepsen worker 62 - jepsen.util 152 :invoke :write [20 42] INFO [2022-01-10 12:53:13,113] jepsen worker 3 - jepsen.util 3 :ok :write [1 17] INFO [2022-01-10 12:53:13,113] jepsen worker 43 - jepsen.util 43 :ok :write [14 17] INFO [2022-01-10 12:53:13,113] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:13,114] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,114] jepsen worker 43 - jepsen.util 43 :ok :read [14 17] INFO [2022-01-10 12:53:13,115] jepsen worker 3 - jepsen.util 3 :ok :read [1 17] INFO [2022-01-10 12:53:13,115] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,115] jepsen worker 89 - jepsen.util 179 :ok :write [29 47] INFO [2022-01-10 12:53:13,116] jepsen worker 3 - jepsen.util 3 :invoke :write [1 18] INFO [2022-01-10 12:53:13,116] jepsen worker 43 - jepsen.util 43 :ok :read [14 17] INFO [2022-01-10 12:53:13,116] jepsen worker 89 - jepsen.util 179 :invoke :write [29 5] INFO [2022-01-10 12:53:13,116] jepsen worker 43 - jepsen.util 43 :invoke :write [14 14] INFO [2022-01-10 12:53:13,118] jepsen worker 71 - jepsen.util 71 :ok :write [23 49] INFO [2022-01-10 12:53:13,119] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:13,119] jepsen worker 71 - jepsen.util 71 :ok :read [23 49] INFO [2022-01-10 12:53:13,120] jepsen worker 67 - jepsen.util 67 :ok :write [22 26] INFO [2022-01-10 12:53:13,120] jepsen worker 71 - jepsen.util 71 :invoke :write [23 39] INFO [2022-01-10 12:53:13,121] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:13,122] jepsen worker 67 - jepsen.util 67 :ok :read [22 26] INFO [2022-01-10 12:53:13,123] jepsen worker 41 - jepsen.util 41 :ok :write [13 18] INFO [2022-01-10 12:53:13,123] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:13,123] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,124] jepsen worker 77 - jepsen.util 167 :ok :write [25 40] INFO [2022-01-10 12:53:13,124] jepsen worker 67 - jepsen.util 67 :ok :read [22 26] INFO [2022-01-10 12:53:13,124] jepsen worker 41 - jepsen.util 41 :ok :read [13 18] INFO [2022-01-10 12:53:13,124] jepsen worker 77 - jepsen.util 167 :invoke :write [25 2] INFO [2022-01-10 12:53:13,125] jepsen worker 67 - jepsen.util 67 :invoke :write [22 15] INFO [2022-01-10 12:53:13,126] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,127] jepsen worker 41 - jepsen.util 41 :ok :read [13 18] INFO [2022-01-10 12:53:13,127] jepsen worker 85 - jepsen.util 175 :ok :write [28 47] INFO [2022-01-10 12:53:13,127] jepsen worker 41 - jepsen.util 41 :invoke :write [13 32] INFO [2022-01-10 12:53:13,128] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:13,128] jepsen worker 1 - jepsen.util 1 :ok :write [0 2] INFO [2022-01-10 12:53:13,129] jepsen worker 85 - jepsen.util 175 :ok :read [28 47] INFO [2022-01-10 12:53:13,129] jepsen worker 8 - jepsen.util 8 :ok :write [2 3] INFO [2022-01-10 12:53:13,129] jepsen worker 1 - jepsen.util 1 :invoke :write [0 36] INFO [2022-01-10 12:53:13,130] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:13,130] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:13,131] jepsen worker 85 - jepsen.util 175 :ok :read [28 47] INFO [2022-01-10 12:53:13,131] jepsen worker 8 - jepsen.util 8 :ok :read [2 3] INFO [2022-01-10 12:53:13,131] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:13,132] jepsen worker 85 - jepsen.util 175 :invoke :write [28 11] INFO [2022-01-10 12:53:13,132] jepsen worker 8 - jepsen.util 8 :ok :read [2 3] INFO [2022-01-10 12:53:13,133] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:13,134] jepsen worker 8 - jepsen.util 8 :ok :read [2 3] INFO [2022-01-10 12:53:13,135] jepsen worker 78 - jepsen.util 78 :ok :write [26 37] INFO [2022-01-10 12:53:13,135] jepsen worker 8 - jepsen.util 8 :invoke :write [2 30] INFO [2022-01-10 12:53:13,135] jepsen worker 22 - jepsen.util 112 :ok :write [7 39] INFO [2022-01-10 12:53:13,136] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,136] jepsen worker 22 - jepsen.util 112 :invoke :write [7 41] INFO [2022-01-10 12:53:13,136] jepsen worker 89 - jepsen.util 179 :ok :write [29 5] INFO [2022-01-10 12:53:13,137] jepsen worker 78 - jepsen.util 78 :ok :read [26 37] INFO [2022-01-10 12:53:13,137] jepsen worker 89 - jepsen.util 179 :invoke :write [29 40] INFO [2022-01-10 12:53:13,138] jepsen worker 78 - jepsen.util 78 :invoke :write [26 28] INFO [2022-01-10 12:53:13,140] jepsen worker 52 - jepsen.util 52 :ok :write [17 5] INFO [2022-01-10 12:53:13,141] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,142] jepsen worker 52 - jepsen.util 52 :ok :read [17 5] INFO [2022-01-10 12:53:13,142] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,143] jepsen worker 15 - jepsen.util 15 :ok :write [5 13] INFO [2022-01-10 12:53:13,143] jepsen worker 52 - jepsen.util 52 :ok :read [17 5] INFO [2022-01-10 12:53:13,144] jepsen worker 52 - jepsen.util 52 :invoke :write [17 1] INFO [2022-01-10 12:53:13,144] jepsen worker 36 - jepsen.util 36 :ok :write [12 46] INFO [2022-01-10 12:53:13,144] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,145] jepsen worker 36 - jepsen.util 36 :invoke :write [12 27] INFO [2022-01-10 12:53:13,145] jepsen worker 15 - jepsen.util 15 :ok :read [5 13] INFO [2022-01-10 12:53:13,146] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,146] jepsen worker 15 - jepsen.util 15 :ok :read [5 13] INFO [2022-01-10 12:53:13,147] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,148] jepsen worker 15 - jepsen.util 15 :ok :read [5 13] INFO [2022-01-10 12:53:13,148] jepsen worker 15 - jepsen.util 15 :invoke :write [5 8] INFO [2022-01-10 12:53:13,149] jepsen worker 35 - jepsen.util 125 :ok :write [11 8] INFO [2022-01-10 12:53:13,149] jepsen worker 35 - jepsen.util 125 :invoke :write [11 20] INFO [2022-01-10 12:53:13,151] jepsen worker 41 - jepsen.util 41 :ok :write [13 32] INFO [2022-01-10 12:53:13,152] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,153] jepsen worker 41 - jepsen.util 41 :ok :read [13 32] INFO [2022-01-10 12:53:13,153] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,154] jepsen worker 14 - jepsen.util 104 :ok :write [4 5] INFO [2022-01-10 12:53:13,154] jepsen worker 41 - jepsen.util 41 :ok :read [13 32] INFO [2022-01-10 12:53:13,155] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,155] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,156] jepsen worker 14 - jepsen.util 104 :ok :read [4 5] INFO [2022-01-10 12:53:13,156] jepsen worker 14 - jepsen.util 104 :invoke :write [4 43] INFO [2022-01-10 12:53:13,157] jepsen worker 41 - jepsen.util 41 :ok :read [13 32] INFO [2022-01-10 12:53:13,157] jepsen worker 49 - jepsen.util 139 :ok :write [16 38] INFO [2022-01-10 12:53:13,157] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,158] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:13,158] jepsen worker 41 - jepsen.util 41 :ok :read [13 32] INFO [2022-01-10 12:53:13,159] jepsen worker 49 - jepsen.util 139 :ok :read [16 38] INFO [2022-01-10 12:53:13,159] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,160] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:13,160] jepsen worker 41 - jepsen.util 41 :ok :read [13 32] INFO [2022-01-10 12:53:13,160] jepsen worker 1 - jepsen.util 1 :ok :write [0 36] INFO [2022-01-10 12:53:13,160] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,160] jepsen worker 49 - jepsen.util 139 :ok :read [16 38] INFO [2022-01-10 12:53:13,161] jepsen worker 49 - jepsen.util 139 :invoke :write [16 3] INFO [2022-01-10 12:53:13,161] jepsen worker 41 - jepsen.util 41 :ok :read [13 32] INFO [2022-01-10 12:53:13,162] jepsen worker 1 - jepsen.util 1 :invoke :write [0 0] INFO [2022-01-10 12:53:13,162] jepsen worker 3 - jepsen.util 3 :ok :write [1 18] INFO [2022-01-10 12:53:13,162] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,163] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:13,163] jepsen worker 41 - jepsen.util 41 :ok :read [13 32] INFO [2022-01-10 12:53:13,164] jepsen worker 3 - jepsen.util 3 :ok :read [1 18] INFO [2022-01-10 12:53:13,164] jepsen worker 41 - jepsen.util 41 :invoke :write [13 38] INFO [2022-01-10 12:53:13,165] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:13,165] jepsen worker 83 - jepsen.util 173 :ok :write [27 36] INFO [2022-01-10 12:53:13,165] jepsen worker 3 - jepsen.util 3 :ok :read [1 18] INFO [2022-01-10 12:53:13,166] jepsen worker 3 - jepsen.util 3 :invoke :write [1 36] INFO [2022-01-10 12:53:13,167] jepsen worker 83 - jepsen.util 173 :invoke :write [27 0] INFO [2022-01-10 12:53:13,169] jepsen worker 71 - jepsen.util 71 :ok :write [23 39] INFO [2022-01-10 12:53:13,170] jepsen worker 71 - jepsen.util 71 :invoke :write [23 24] INFO [2022-01-10 12:53:13,175] jepsen worker 62 - jepsen.util 152 :ok :write [20 42] INFO [2022-01-10 12:53:13,176] jepsen worker 85 - jepsen.util 175 :ok :write [28 11] INFO [2022-01-10 12:53:13,176] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:13,176] jepsen worker 89 - jepsen.util 179 :ok :write [29 40] INFO [2022-01-10 12:53:13,176] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:13,177] jepsen worker 62 - jepsen.util 152 :ok :read [20 42] INFO [2022-01-10 12:53:13,177] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,177] jepsen worker 85 - jepsen.util 175 :ok :read [28 11] INFO [2022-01-10 12:53:13,177] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:13,178] jepsen worker 89 - jepsen.util 179 :ok :read [29 40] INFO [2022-01-10 12:53:13,178] jepsen worker 85 - jepsen.util 175 :invoke :write [28 47] INFO [2022-01-10 12:53:13,178] jepsen worker 62 - jepsen.util 152 :ok :read [20 42] INFO [2022-01-10 12:53:13,179] jepsen worker 89 - jepsen.util 179 :invoke :write [29 45] INFO [2022-01-10 12:53:13,179] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:13,180] jepsen worker 62 - jepsen.util 152 :ok :read [20 42] INFO [2022-01-10 12:53:13,180] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:13,181] jepsen worker 62 - jepsen.util 152 :ok :read [20 42] INFO [2022-01-10 12:53:13,181] jepsen worker 43 - jepsen.util 43 :ok :write [14 14] INFO [2022-01-10 12:53:13,182] jepsen worker 62 - jepsen.util 152 :invoke :write [20 3] INFO [2022-01-10 12:53:13,182] jepsen worker 43 - jepsen.util 43 :invoke :write [14 1] INFO [2022-01-10 12:53:13,185] jepsen worker 78 - jepsen.util 78 :ok :write [26 28] INFO [2022-01-10 12:53:13,185] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,186] jepsen worker 78 - jepsen.util 78 :ok :read [26 28] INFO [2022-01-10 12:53:13,187] jepsen worker 78 - jepsen.util 78 :invoke :write [26 41] INFO [2022-01-10 12:53:13,191] jepsen worker 67 - jepsen.util 67 :ok :write [22 15] INFO [2022-01-10 12:53:13,192] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:13,193] jepsen worker 67 - jepsen.util 67 :ok :read [22 15] INFO [2022-01-10 12:53:13,193] jepsen worker 67 - jepsen.util 67 :invoke :write [22 2] INFO [2022-01-10 12:53:13,194] jepsen worker 41 - jepsen.util 41 :ok :write [13 38] INFO [2022-01-10 12:53:13,195] jepsen worker 52 - jepsen.util 52 :ok :write [17 1] INFO [2022-01-10 12:53:13,195] jepsen worker 77 - jepsen.util 167 :ok :write [25 2] INFO [2022-01-10 12:53:13,196] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:13,197] jepsen worker 77 - jepsen.util 167 :ok :read [25 2] INFO [2022-01-10 12:53:13,197] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,197] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:13,198] jepsen worker 52 - jepsen.util 52 :ok :read [17 1] INFO [2022-01-10 12:53:13,198] jepsen worker 41 - jepsen.util 41 :invoke :write [13 19] INFO [2022-01-10 12:53:13,198] jepsen worker 77 - jepsen.util 167 :ok :read [25 2] INFO [2022-01-10 12:53:13,199] jepsen worker 52 - jepsen.util 52 :invoke :write [17 22] INFO [2022-01-10 12:53:13,199] jepsen worker 8 - jepsen.util 8 :ok :write [2 30] INFO [2022-01-10 12:53:13,200] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:13,201] jepsen worker 77 - jepsen.util 167 :ok :read [25 2] INFO [2022-01-10 12:53:13,201] jepsen worker 8 - jepsen.util 8 :invoke :write [2 29] INFO [2022-01-10 12:53:13,201] jepsen worker 38 - jepsen.util 38 :ok :write [12 25] INFO [2022-01-10 12:53:13,202] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:13,203] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,203] jepsen worker 77 - jepsen.util 167 :ok :read [25 2] INFO [2022-01-10 12:53:13,204] jepsen worker 38 - jepsen.util 38 :ok :read [12 25] INFO [2022-01-10 12:53:13,204] jepsen worker 22 - jepsen.util 112 :ok :write [7 41] INFO [2022-01-10 12:53:13,204] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:13,205] jepsen worker 77 - jepsen.util 167 :ok :read [25 2] INFO [2022-01-10 12:53:13,205] jepsen worker 22 - jepsen.util 112 :invoke :write [7 7] INFO [2022-01-10 12:53:13,205] jepsen worker 14 - jepsen.util 104 :ok :write [4 43] INFO [2022-01-10 12:53:13,206] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:13,207] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,207] jepsen worker 77 - jepsen.util 167 :ok :read [25 2] INFO [2022-01-10 12:53:13,208] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,208] jepsen worker 14 - jepsen.util 104 :ok :read [4 43] INFO [2022-01-10 12:53:13,208] jepsen worker 77 - jepsen.util 167 :invoke :write [25 33] INFO [2022-01-10 12:53:13,209] jepsen worker 38 - jepsen.util 38 :ok :read [12 25] INFO [2022-01-10 12:53:13,209] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,210] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,210] jepsen worker 14 - jepsen.util 104 :ok :read [4 43] INFO [2022-01-10 12:53:13,211] jepsen worker 38 - jepsen.util 38 :ok :read [12 25] INFO [2022-01-10 12:53:13,211] jepsen worker 14 - jepsen.util 104 :invoke :write [4 20] INFO [2022-01-10 12:53:13,211] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,212] jepsen worker 15 - jepsen.util 15 :ok :write [5 8] INFO [2022-01-10 12:53:13,212] jepsen worker 38 - jepsen.util 38 :ok :read [12 25] INFO [2022-01-10 12:53:13,213] jepsen worker 15 - jepsen.util 15 :invoke :write [5 40] INFO [2022-01-10 12:53:13,213] jepsen worker 38 - jepsen.util 38 :invoke :write [12 3] INFO [2022-01-10 12:53:13,216] jepsen worker 35 - jepsen.util 125 :ok :write [11 20] INFO [2022-01-10 12:53:13,217] jepsen worker 1 - jepsen.util 1 :ok :write [0 0] INFO [2022-01-10 12:53:13,217] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:13,217] jepsen worker 3 - jepsen.util 3 :ok :write [1 36] INFO [2022-01-10 12:53:13,217] jepsen worker 1 - jepsen.util 1 :invoke :write [0 15] INFO [2022-01-10 12:53:13,218] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:13,218] jepsen worker 35 - jepsen.util 125 :ok :read [11 20] INFO [2022-01-10 12:53:13,219] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:13,219] jepsen worker 3 - jepsen.util 3 :ok :read [1 36] INFO [2022-01-10 12:53:13,220] jepsen worker 35 - jepsen.util 125 :ok :read [11 20] INFO [2022-01-10 12:53:13,220] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:13,220] jepsen worker 35 - jepsen.util 125 :invoke :write [11 46] INFO [2022-01-10 12:53:13,221] jepsen worker 3 - jepsen.util 3 :ok :read [1 36] INFO [2022-01-10 12:53:13,221] jepsen worker 3 - jepsen.util 3 :invoke :write [1 12] INFO [2022-01-10 12:53:13,221] jepsen worker 49 - jepsen.util 139 :ok :write [16 3] INFO [2022-01-10 12:53:13,222] jepsen worker 49 - jepsen.util 139 :invoke :write [16 44] INFO [2022-01-10 12:53:13,224] jepsen worker 71 - jepsen.util 71 :ok :write [23 24] INFO [2022-01-10 12:53:13,225] jepsen worker 71 - jepsen.util 71 :invoke :write [23 5] INFO [2022-01-10 12:53:13,227] jepsen worker 83 - jepsen.util 173 :ok :write [27 0] INFO [2022-01-10 12:53:13,228] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:13,229] jepsen worker 85 - jepsen.util 175 :ok :write [28 47] INFO [2022-01-10 12:53:13,229] jepsen worker 62 - jepsen.util 152 :ok :write [20 3] INFO [2022-01-10 12:53:13,229] jepsen worker 83 - jepsen.util 173 :ok :read [27 0] INFO [2022-01-10 12:53:13,230] jepsen worker 85 - jepsen.util 175 :invoke :write [28 13] INFO [2022-01-10 12:53:13,230] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:13,231] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:13,232] jepsen worker 83 - jepsen.util 173 :ok :read [27 0] INFO [2022-01-10 12:53:13,232] jepsen worker 62 - jepsen.util 152 :ok :read [20 3] INFO [2022-01-10 12:53:13,232] jepsen worker 78 - jepsen.util 78 :ok :write [26 41] INFO [2022-01-10 12:53:13,232] jepsen worker 62 - jepsen.util 152 :invoke :write [20 30] INFO [2022-01-10 12:53:13,233] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:13,233] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,234] jepsen worker 83 - jepsen.util 173 :ok :read [27 0] INFO [2022-01-10 12:53:13,234] jepsen worker 78 - jepsen.util 78 :ok :read [26 41] INFO [2022-01-10 12:53:13,234] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:13,235] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,236] jepsen worker 83 - jepsen.util 173 :ok :read [27 0] INFO [2022-01-10 12:53:13,236] jepsen worker 43 - jepsen.util 43 :ok :write [14 1] INFO [2022-01-10 12:53:13,236] jepsen worker 83 - jepsen.util 173 :invoke :write [27 20] INFO [2022-01-10 12:53:13,237] jepsen worker 43 - jepsen.util 43 :invoke :write [14 17] INFO [2022-01-10 12:53:13,237] jepsen worker 52 - jepsen.util 52 :ok :write [17 22] INFO [2022-01-10 12:53:13,238] jepsen worker 78 - jepsen.util 78 :ok :read [26 41] INFO [2022-01-10 12:53:13,238] jepsen worker 52 - jepsen.util 52 :invoke :write [17 5] INFO [2022-01-10 12:53:13,238] jepsen worker 78 - jepsen.util 78 :invoke :write [26 2] INFO [2022-01-10 12:53:13,239] jepsen worker 89 - jepsen.util 179 :ok :write [29 45] INFO [2022-01-10 12:53:13,239] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,240] jepsen worker 89 - jepsen.util 179 :ok :read [29 45] INFO [2022-01-10 12:53:13,241] jepsen worker 89 - jepsen.util 179 :invoke :write [29 6] INFO [2022-01-10 12:53:13,242] jepsen worker 38 - jepsen.util 38 :ok :write [12 3] INFO [2022-01-10 12:53:13,242] jepsen worker 38 - jepsen.util 38 :invoke :write [12 35] INFO [2022-01-10 12:53:13,245] jepsen worker 67 - jepsen.util 67 :ok :write [22 2] INFO [2022-01-10 12:53:13,245] jepsen worker 14 - jepsen.util 104 :ok :write [4 20] INFO [2022-01-10 12:53:13,246] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:13,246] jepsen worker 14 - jepsen.util 104 :invoke :write [4 40] INFO [2022-01-10 12:53:13,247] jepsen worker 77 - jepsen.util 167 :ok :write [25 33] INFO [2022-01-10 12:53:13,247] jepsen worker 67 - jepsen.util 67 :ok :read [22 2] INFO [2022-01-10 12:53:13,247] jepsen worker 77 - jepsen.util 167 :invoke :write [25 13] INFO [2022-01-10 12:53:13,248] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:13,250] jepsen worker 67 - jepsen.util 67 :ok :read [22 2] INFO [2022-01-10 12:53:13,251] jepsen worker 67 - jepsen.util 67 :invoke :write [22 13] INFO [2022-01-10 12:53:13,252] jepsen worker 3 - jepsen.util 3 :ok :write [1 12] INFO [2022-01-10 12:53:13,253] jepsen worker 3 - jepsen.util 3 :invoke :write [1 43] INFO [2022-01-10 12:53:13,255] jepsen worker 41 - jepsen.util 41 :ok :write [13 19] INFO [2022-01-10 12:53:13,256] jepsen worker 22 - jepsen.util 112 :ok :write [7 7] INFO [2022-01-10 12:53:13,256] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,256] jepsen worker 22 - jepsen.util 112 :invoke :write [7 29] INFO [2022-01-10 12:53:13,257] jepsen worker 41 - jepsen.util 41 :ok :read [13 19] INFO [2022-01-10 12:53:13,257] jepsen worker 41 - jepsen.util 41 :invoke :write [13 12] INFO [2022-01-10 12:53:13,261] jepsen worker 71 - jepsen.util 71 :ok :write [23 5] INFO [2022-01-10 12:53:13,261] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:13,262] jepsen worker 71 - jepsen.util 71 :ok :read [23 5] INFO [2022-01-10 12:53:13,263] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:13,264] jepsen worker 71 - jepsen.util 71 :ok :read [23 5] INFO [2022-01-10 12:53:13,264] jepsen worker 71 - jepsen.util 71 :invoke :write [23 48] INFO [2022-01-10 12:53:13,266] jepsen worker 8 - jepsen.util 8 :ok :write [2 29] INFO [2022-01-10 12:53:13,266] jepsen worker 85 - jepsen.util 175 :ok :write [28 13] INFO [2022-01-10 12:53:13,267] jepsen worker 8 - jepsen.util 8 :invoke :write [2 25] INFO [2022-01-10 12:53:13,267] jepsen worker 85 - jepsen.util 175 :invoke :write [28 6] INFO [2022-01-10 12:53:13,270] jepsen worker 78 - jepsen.util 78 :ok :write [26 2] INFO [2022-01-10 12:53:13,270] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,271] jepsen worker 78 - jepsen.util 78 :ok :read [26 2] INFO [2022-01-10 12:53:13,272] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,273] jepsen worker 78 - jepsen.util 78 :ok :read [26 2] INFO [2022-01-10 12:53:13,273] jepsen worker 78 - jepsen.util 78 :invoke :write [26 20] INFO [2022-01-10 12:53:13,273] jepsen worker 15 - jepsen.util 15 :ok :write [5 40] INFO [2022-01-10 12:53:13,274] jepsen worker 15 - jepsen.util 15 :invoke :write [5 23] INFO [2022-01-10 12:53:13,274] jepsen worker 52 - jepsen.util 52 :ok :write [17 5] INFO [2022-01-10 12:53:13,274] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,275] jepsen worker 52 - jepsen.util 52 :ok :read [17 5] INFO [2022-01-10 12:53:13,276] jepsen worker 52 - jepsen.util 52 :invoke :write [17 37] INFO [2022-01-10 12:53:13,278] jepsen worker 38 - jepsen.util 38 :ok :write [12 35] INFO [2022-01-10 12:53:13,279] jepsen worker 38 - jepsen.util 38 :invoke :write [12 49] INFO [2022-01-10 12:53:13,281] jepsen worker 14 - jepsen.util 104 :ok :write [4 40] INFO [2022-01-10 12:53:13,282] jepsen worker 14 - jepsen.util 104 :invoke :write [4 32] INFO [2022-01-10 12:53:13,282] jepsen worker 1 - jepsen.util 1 :ok :write [0 15] INFO [2022-01-10 12:53:13,282] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,283] jepsen worker 3 - jepsen.util 3 :ok :write [1 43] INFO [2022-01-10 12:53:13,283] jepsen worker 1 - jepsen.util 1 :ok :read [0 15] INFO [2022-01-10 12:53:13,284] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:13,285] jepsen worker 1 - jepsen.util 1 :invoke :write [0 38] INFO [2022-01-10 12:53:13,285] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:13,286] jepsen worker 3 - jepsen.util 3 :invoke :write [1 0] INFO [2022-01-10 12:53:13,288] jepsen worker 71 - jepsen.util 71 :ok :write [23 48] INFO [2022-01-10 12:53:13,288] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:13,289] jepsen worker 71 - jepsen.util 71 :ok :read [23 48] INFO [2022-01-10 12:53:13,290] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:13,290] jepsen worker 71 - jepsen.util 71 :ok :read [23 48] INFO [2022-01-10 12:53:13,291] jepsen worker 71 - jepsen.util 71 :invoke :write [23 45] INFO [2022-01-10 12:53:13,291] jepsen worker 62 - jepsen.util 152 :ok :write [20 30] INFO [2022-01-10 12:53:13,292] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:13,293] jepsen worker 85 - jepsen.util 175 :ok :write [28 6] INFO [2022-01-10 12:53:13,293] jepsen worker 62 - jepsen.util 152 :ok :read [20 30] INFO [2022-01-10 12:53:13,294] jepsen worker 85 - jepsen.util 175 :invoke :write [28 30] INFO [2022-01-10 12:53:13,295] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:13,295] jepsen worker 62 - jepsen.util 152 :ok :read [20 30] INFO [2022-01-10 12:53:13,296] jepsen worker 62 - jepsen.util 152 :invoke :write [20 19] INFO [2022-01-10 12:53:13,297] jepsen worker 78 - jepsen.util 78 :ok :write [26 20] INFO [2022-01-10 12:53:13,298] jepsen worker 43 - jepsen.util 43 :ok :write [14 17] INFO [2022-01-10 12:53:13,298] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,299] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,299] jepsen worker 78 - jepsen.util 78 :ok :read [26 20] INFO [2022-01-10 12:53:13,300] jepsen worker 43 - jepsen.util 43 :ok :read [14 17] INFO [2022-01-10 12:53:13,300] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,301] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,301] jepsen worker 78 - jepsen.util 78 :ok :read [26 20] INFO [2022-01-10 12:53:13,301] jepsen worker 43 - jepsen.util 43 :ok :read [14 17] INFO [2022-01-10 12:53:13,302] jepsen worker 78 - jepsen.util 78 :invoke :write [26 31] INFO [2022-01-10 12:53:13,302] jepsen worker 43 - jepsen.util 43 :invoke :write [14 12] INFO [2022-01-10 12:53:13,304] jepsen worker 77 - jepsen.util 167 :ok :write [25 13] INFO [2022-01-10 12:53:13,304] jepsen worker 77 - jepsen.util 167 :invoke :write [25 47] INFO [2022-01-10 12:53:13,305] jepsen worker 52 - jepsen.util 52 :ok :write [17 37] INFO [2022-01-10 12:53:13,305] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,306] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:13,307] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,308] jepsen worker 52 - jepsen.util 52 :ok :read [17 37] INFO [2022-01-10 12:53:13,308] jepsen worker 52 - jepsen.util 52 :invoke :write [17 22] INFO [2022-01-10 12:53:13,312] jepsen worker 14 - jepsen.util 104 :ok :write [4 32] INFO [2022-01-10 12:53:13,313] jepsen worker 14 - jepsen.util 104 :invoke :write [4 17] INFO [2022-01-10 12:53:13,314] jepsen worker 22 - jepsen.util 112 :ok :write [7 29] INFO [2022-01-10 12:53:13,314] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:13,315] jepsen worker 22 - jepsen.util 112 :ok :read [7 29] INFO [2022-01-10 12:53:13,316] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:13,317] jepsen worker 22 - jepsen.util 112 :ok :read [7 29] INFO [2022-01-10 12:53:13,318] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:13,318] jepsen worker 22 - jepsen.util 112 :ok :read [7 29] INFO [2022-01-10 12:53:13,319] jepsen worker 22 - jepsen.util 112 :invoke :write [7 28] INFO [2022-01-10 12:53:13,321] jepsen worker 71 - jepsen.util 71 :ok :write [23 45] INFO [2022-01-10 12:53:13,322] jepsen worker 71 - jepsen.util 71 :invoke :write [23 45] INFO [2022-01-10 12:53:13,401] jepsen worker 38 - jepsen.util 38 :ok :write [12 49] INFO [2022-01-10 12:53:13,402] jepsen worker 38 - jepsen.util 38 :invoke :write [12 24] INFO [2022-01-10 12:53:13,403] jepsen worker 35 - jepsen.util 125 :ok :write [11 46] INFO [2022-01-10 12:53:13,403] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:13,404] jepsen worker 35 - jepsen.util 125 :ok :read [11 46] INFO [2022-01-10 12:53:13,405] jepsen worker 35 - jepsen.util 125 :invoke :write [11 31] INFO [2022-01-10 12:53:13,408] jepsen worker 3 - jepsen.util 3 :ok :write [1 0] INFO [2022-01-10 12:53:13,408] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:13,409] jepsen worker 49 - jepsen.util 139 :ok :write [16 44] INFO [2022-01-10 12:53:13,409] jepsen worker 3 - jepsen.util 3 :ok :read [1 0] INFO [2022-01-10 12:53:13,409] jepsen worker 3 - jepsen.util 3 :invoke :write [1 48] INFO [2022-01-10 12:53:13,410] jepsen worker 49 - jepsen.util 139 :invoke :write [16 10] INFO [2022-01-10 12:53:13,414] jepsen worker 85 - jepsen.util 175 :ok :write [28 30] INFO [2022-01-10 12:53:13,414] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:13,415] jepsen worker 85 - jepsen.util 175 :ok :read [28 30] INFO [2022-01-10 12:53:13,415] jepsen worker 83 - jepsen.util 173 :ok :write [27 20] INFO [2022-01-10 12:53:13,416] jepsen worker 83 - jepsen.util 173 :invoke :write [27 2] INFO [2022-01-10 12:53:13,417] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:13,417] jepsen worker 85 - jepsen.util 175 :ok :read [28 30] INFO [2022-01-10 12:53:13,418] jepsen worker 85 - jepsen.util 175 :invoke :write [28 41] INFO [2022-01-10 12:53:13,422] jepsen worker 89 - jepsen.util 179 :ok :write [29 6] INFO [2022-01-10 12:53:13,423] jepsen worker 89 - jepsen.util 179 :invoke :write [29 48] INFO [2022-01-10 12:53:13,426] jepsen worker 67 - jepsen.util 67 :ok :write [22 13] INFO [2022-01-10 12:53:13,427] jepsen worker 67 - jepsen.util 67 :invoke :write [22 23] INFO [2022-01-10 12:53:13,432] jepsen worker 41 - jepsen.util 41 :ok :write [13 12] INFO [2022-01-10 12:53:13,432] jepsen worker 41 - jepsen.util 41 :invoke :write [13 7] INFO [2022-01-10 12:53:13,437] jepsen worker 8 - jepsen.util 8 :ok :write [2 25] INFO [2022-01-10 12:53:13,437] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:13,438] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:13,439] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:13,440] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:13,440] jepsen worker 8 - jepsen.util 8 :invoke :write [2 24] INFO [2022-01-10 12:53:13,445] jepsen worker 1 - jepsen.util 1 :ok :write [0 38] INFO [2022-01-10 12:53:13,445] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,446] jepsen worker 1 - jepsen.util 1 :ok :read [0 38] INFO [2022-01-10 12:53:13,447] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,447] jepsen worker 1 - jepsen.util 1 :ok :read [0 38] INFO [2022-01-10 12:53:13,448] jepsen worker 1 - jepsen.util 1 :invoke :write [0 23] INFO [2022-01-10 12:53:13,509] jepsen worker 78 - jepsen.util 78 :ok :write [26 31] INFO [2022-01-10 12:53:13,510] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,511] jepsen worker 78 - jepsen.util 78 :ok :read [26 31] INFO [2022-01-10 12:53:13,511] jepsen worker 78 - jepsen.util 78 :invoke :write [26 25] INFO [2022-01-10 12:53:13,512] jepsen worker 15 - jepsen.util 15 :ok :write [5 23] INFO [2022-01-10 12:53:13,512] jepsen worker 15 - jepsen.util 15 :invoke :write [5 34] INFO [2022-01-10 12:53:13,516] jepsen worker 52 - jepsen.util 52 :ok :write [17 22] INFO [2022-01-10 12:53:13,516] jepsen worker 61 - jepsen.util 61 :ok :write [20 30] INFO [2022-01-10 12:53:13,517] jepsen worker 52 - jepsen.util 52 :invoke :write [17 46] INFO [2022-01-10 12:53:13,517] jepsen worker 61 - jepsen.util 61 :invoke :write [20 40] INFO [2022-01-10 12:53:13,519] jepsen worker 89 - jepsen.util 179 :ok :write [29 48] INFO [2022-01-10 12:53:13,520] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,521] jepsen worker 89 - jepsen.util 179 :ok :read [29 48] INFO [2022-01-10 12:53:13,521] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,521] jepsen worker 14 - jepsen.util 104 :ok :write [4 17] INFO [2022-01-10 12:53:13,522] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,522] jepsen worker 89 - jepsen.util 179 :ok :read [29 48] INFO [2022-01-10 12:53:13,522] jepsen worker 43 - jepsen.util 43 :ok :write [14 12] INFO [2022-01-10 12:53:13,523] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,523] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,523] jepsen worker 14 - jepsen.util 104 :ok :read [4 17] INFO [2022-01-10 12:53:13,524] jepsen worker 89 - jepsen.util 179 :ok :read [29 48] INFO [2022-01-10 12:53:13,524] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,524] jepsen worker 43 - jepsen.util 43 :ok :read [14 12] INFO [2022-01-10 12:53:13,525] jepsen worker 89 - jepsen.util 179 :invoke :write [29 8] INFO [2022-01-10 12:53:13,525] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,525] jepsen worker 14 - jepsen.util 104 :ok :read [4 17] INFO [2022-01-10 12:53:13,525] jepsen worker 41 - jepsen.util 41 :ok :write [13 7] INFO [2022-01-10 12:53:13,526] jepsen worker 14 - jepsen.util 104 :invoke :write [4 42] INFO [2022-01-10 12:53:13,526] jepsen worker 43 - jepsen.util 43 :ok :read [14 12] INFO [2022-01-10 12:53:13,526] jepsen worker 41 - jepsen.util 41 :invoke :write [13 38] INFO [2022-01-10 12:53:13,527] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,528] jepsen worker 71 - jepsen.util 71 :ok :write [23 45] INFO [2022-01-10 12:53:13,528] jepsen worker 43 - jepsen.util 43 :ok :read [14 12] INFO [2022-01-10 12:53:13,528] jepsen worker 77 - jepsen.util 167 :ok :write [25 47] INFO [2022-01-10 12:53:13,528] jepsen worker 71 - jepsen.util 71 :invoke :write [23 23] INFO [2022-01-10 12:53:13,529] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:13,529] jepsen worker 43 - jepsen.util 43 :invoke :write [14 31] INFO [2022-01-10 12:53:13,531] jepsen worker 77 - jepsen.util 167 :ok :read [25 47] INFO [2022-01-10 12:53:13,532] jepsen worker 77 - jepsen.util 167 :invoke :write [25 0] INFO [2022-01-10 12:53:13,533] jepsen worker 1 - jepsen.util 1 :ok :write [0 23] INFO [2022-01-10 12:53:13,533] jepsen worker 1 - jepsen.util 1 :invoke :write [0 41] INFO [2022-01-10 12:53:13,534] jepsen worker 22 - jepsen.util 112 :ok :write [7 28] INFO [2022-01-10 12:53:13,535] jepsen worker 22 - jepsen.util 112 :invoke :write [7 9] INFO [2022-01-10 12:53:13,539] jepsen worker 35 - jepsen.util 125 :ok :write [11 31] INFO [2022-01-10 12:53:13,540] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:13,541] jepsen worker 36 - jepsen.util 36 :ok :write [12 27] INFO [2022-01-10 12:53:13,541] jepsen worker 35 - jepsen.util 125 :ok :read [11 31] INFO [2022-01-10 12:53:13,542] jepsen worker 36 - jepsen.util 36 :invoke :write [12 14] INFO [2022-01-10 12:53:13,542] jepsen worker 89 - jepsen.util 179 :ok :write [29 8] INFO [2022-01-10 12:53:13,543] jepsen worker 35 - jepsen.util 125 :invoke :write [11 21] INFO [2022-01-10 12:53:13,543] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,544] jepsen worker 89 - jepsen.util 179 :ok :read [29 8] INFO [2022-01-10 12:53:13,545] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,545] jepsen worker 49 - jepsen.util 139 :ok :write [16 10] INFO [2022-01-10 12:53:13,546] jepsen worker 3 - jepsen.util 3 :ok :write [1 48] INFO [2022-01-10 12:53:13,546] jepsen worker 89 - jepsen.util 179 :ok :read [29 8] INFO [2022-01-10 12:53:13,546] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:13,547] jepsen worker 89 - jepsen.util 179 :invoke :write [29 19] INFO [2022-01-10 12:53:13,547] jepsen worker 49 - jepsen.util 139 :ok :read [16 10] INFO [2022-01-10 12:53:13,547] jepsen worker 3 - jepsen.util 3 :invoke :write [1 44] INFO [2022-01-10 12:53:13,548] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:13,549] jepsen worker 49 - jepsen.util 139 :ok :read [16 10] INFO [2022-01-10 12:53:13,550] jepsen worker 83 - jepsen.util 173 :ok :write [27 2] INFO [2022-01-10 12:53:13,550] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:13,551] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:13,551] jepsen worker 49 - jepsen.util 139 :ok :read [16 10] INFO [2022-01-10 12:53:13,551] jepsen worker 83 - jepsen.util 173 :ok :read [27 2] INFO [2022-01-10 12:53:13,551] jepsen worker 49 - jepsen.util 139 :invoke :write [16 35] INFO [2022-01-10 12:53:13,552] jepsen worker 83 - jepsen.util 173 :invoke :write [27 9] INFO [2022-01-10 12:53:13,553] jepsen worker 85 - jepsen.util 175 :ok :write [28 41] INFO [2022-01-10 12:53:13,553] jepsen worker 67 - jepsen.util 67 :ok :write [22 23] INFO [2022-01-10 12:53:13,554] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:13,554] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:13,554] jepsen worker 85 - jepsen.util 175 :ok :read [28 41] INFO [2022-01-10 12:53:13,555] jepsen worker 67 - jepsen.util 67 :ok :read [22 23] INFO [2022-01-10 12:53:13,555] jepsen worker 85 - jepsen.util 175 :invoke :write [28 22] INFO [2022-01-10 12:53:13,555] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:13,556] jepsen worker 67 - jepsen.util 67 :ok :read [22 23] INFO [2022-01-10 12:53:13,556] jepsen worker 8 - jepsen.util 8 :ok :write [2 24] INFO [2022-01-10 12:53:13,557] jepsen worker 41 - jepsen.util 41 :ok :write [13 38] INFO [2022-01-10 12:53:13,557] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:13,557] jepsen worker 78 - jepsen.util 78 :ok :write [26 25] INFO [2022-01-10 12:53:13,558] jepsen worker 41 - jepsen.util 41 :invoke :write [13 32] INFO [2022-01-10 12:53:13,558] jepsen worker 67 - jepsen.util 67 :ok :read [22 23] INFO [2022-01-10 12:53:13,558] jepsen worker 78 - jepsen.util 78 :invoke :write [26 23] INFO [2022-01-10 12:53:13,559] jepsen worker 8 - jepsen.util 8 :invoke :write [2 0] INFO [2022-01-10 12:53:13,559] jepsen worker 67 - jepsen.util 67 :invoke :write [22 35] INFO [2022-01-10 12:53:13,562] jepsen worker 15 - jepsen.util 15 :ok :write [5 34] INFO [2022-01-10 12:53:13,563] jepsen worker 15 - jepsen.util 15 :invoke :write [5 8] INFO [2022-01-10 12:53:13,565] jepsen worker 52 - jepsen.util 52 :ok :write [17 46] INFO [2022-01-10 12:53:13,566] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,567] jepsen worker 52 - jepsen.util 52 :ok :read [17 46] INFO [2022-01-10 12:53:13,567] jepsen worker 1 - jepsen.util 1 :ok :write [0 41] INFO [2022-01-10 12:53:13,568] jepsen worker 52 - jepsen.util 52 :invoke :write [17 9] INFO [2022-01-10 12:53:13,568] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,569] jepsen worker 1 - jepsen.util 1 :ok :read [0 41] INFO [2022-01-10 12:53:13,569] jepsen worker 1 - jepsen.util 1 :invoke :write [0 23] INFO [2022-01-10 12:53:13,571] jepsen worker 43 - jepsen.util 43 :ok :write [14 31] INFO [2022-01-10 12:53:13,571] jepsen worker 14 - jepsen.util 104 :ok :write [4 42] INFO [2022-01-10 12:53:13,571] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,572] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,572] jepsen worker 43 - jepsen.util 43 :ok :read [14 31] INFO [2022-01-10 12:53:13,573] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,573] jepsen worker 14 - jepsen.util 104 :ok :read [4 42] INFO [2022-01-10 12:53:13,573] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,573] jepsen worker 43 - jepsen.util 43 :ok :read [14 31] INFO [2022-01-10 12:53:13,574] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,574] jepsen worker 14 - jepsen.util 104 :ok :read [4 42] INFO [2022-01-10 12:53:13,575] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,575] jepsen worker 43 - jepsen.util 43 :ok :read [14 31] INFO [2022-01-10 12:53:13,575] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,575] jepsen worker 14 - jepsen.util 104 :ok :read [4 42] INFO [2022-01-10 12:53:13,576] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,576] jepsen worker 43 - jepsen.util 43 :ok :read [14 31] INFO [2022-01-10 12:53:13,576] jepsen worker 43 - jepsen.util 43 :invoke :write [14 4] INFO [2022-01-10 12:53:13,577] jepsen worker 14 - jepsen.util 104 :ok :read [4 42] INFO [2022-01-10 12:53:13,577] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,578] jepsen worker 89 - jepsen.util 179 :ok :write [29 19] INFO [2022-01-10 12:53:13,578] jepsen worker 71 - jepsen.util 71 :ok :write [23 23] INFO [2022-01-10 12:53:13,579] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,579] jepsen worker 71 - jepsen.util 71 :invoke :write [23 15] INFO [2022-01-10 12:53:13,580] jepsen worker 14 - jepsen.util 104 :ok :read [4 42] INFO [2022-01-10 12:53:13,580] jepsen worker 89 - jepsen.util 179 :ok :read [29 19] INFO [2022-01-10 12:53:13,581] jepsen worker 89 - jepsen.util 179 :invoke :write [29 30] INFO [2022-01-10 12:53:13,581] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,582] jepsen worker 14 - jepsen.util 104 :ok :read [4 42] INFO [2022-01-10 12:53:13,582] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,583] jepsen worker 14 - jepsen.util 104 :ok :read [4 42] INFO [2022-01-10 12:53:13,583] jepsen worker 41 - jepsen.util 41 :ok :write [13 32] INFO [2022-01-10 12:53:13,583] jepsen worker 36 - jepsen.util 36 :ok :write [12 14] INFO [2022-01-10 12:53:13,583] jepsen worker 14 - jepsen.util 104 :invoke :write [4 9] INFO [2022-01-10 12:53:13,584] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,584] jepsen worker 41 - jepsen.util 41 :invoke :write [13 25] INFO [2022-01-10 12:53:13,585] jepsen worker 36 - jepsen.util 36 :ok :read [12 14] INFO [2022-01-10 12:53:13,586] jepsen worker 36 - jepsen.util 36 :invoke :write [12 48] INFO [2022-01-10 12:53:13,590] jepsen worker 3 - jepsen.util 3 :ok :write [1 44] INFO [2022-01-10 12:53:13,591] jepsen worker 3 - jepsen.util 3 :invoke :write [1 11] INFO [2022-01-10 12:53:13,596] jepsen worker 85 - jepsen.util 175 :ok :write [28 22] INFO [2022-01-10 12:53:13,597] jepsen worker 85 - jepsen.util 175 :invoke :write [28 13] INFO [2022-01-10 12:53:13,600] jepsen worker 78 - jepsen.util 78 :ok :write [26 23] INFO [2022-01-10 12:53:13,601] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,602] jepsen worker 78 - jepsen.util 78 :ok :read [26 23] INFO [2022-01-10 12:53:13,602] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,603] jepsen worker 1 - jepsen.util 1 :ok :write [0 23] INFO [2022-01-10 12:53:13,603] jepsen worker 78 - jepsen.util 78 :ok :read [26 23] INFO [2022-01-10 12:53:13,603] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,604] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,604] jepsen worker 1 - jepsen.util 1 :ok :read [0 23] INFO [2022-01-10 12:53:13,605] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,605] jepsen worker 78 - jepsen.util 78 :ok :read [26 23] INFO [2022-01-10 12:53:13,605] jepsen worker 78 - jepsen.util 78 :invoke :write [26 37] INFO [2022-01-10 12:53:13,606] jepsen worker 1 - jepsen.util 1 :ok :read [0 23] INFO [2022-01-10 12:53:13,606] jepsen worker 1 - jepsen.util 1 :invoke :write [0 26] INFO [2022-01-10 12:53:13,611] jepsen worker 89 - jepsen.util 179 :ok :write [29 30] INFO [2022-01-10 12:53:13,611] jepsen worker 89 - jepsen.util 179 :invoke :write [29 30] INFO [2022-01-10 12:53:13,614] jepsen worker 41 - jepsen.util 41 :ok :write [13 25] INFO [2022-01-10 12:53:13,614] jepsen worker 41 - jepsen.util 41 :invoke :write [13 46] INFO [2022-01-10 12:53:13,618] jepsen worker 38 - jepsen.util 38 :ok :write [12 24] INFO [2022-01-10 12:53:13,619] jepsen worker 1 - jepsen.util 1 :ok :write [0 26] INFO [2022-01-10 12:53:13,619] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,619] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,620] jepsen worker 38 - jepsen.util 38 :ok :read [12 24] INFO [2022-01-10 12:53:13,620] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,620] jepsen worker 1 - jepsen.util 1 :ok :read [0 26] INFO [2022-01-10 12:53:13,621] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,621] jepsen worker 38 - jepsen.util 38 :ok :read [12 24] INFO [2022-01-10 12:53:13,622] jepsen worker 1 - jepsen.util 1 :ok :read [0 26] INFO [2022-01-10 12:53:13,622] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,622] jepsen worker 1 - jepsen.util 1 :invoke :write [0 12] INFO [2022-01-10 12:53:13,623] jepsen worker 38 - jepsen.util 38 :ok :read [12 24] INFO [2022-01-10 12:53:13,623] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,624] jepsen worker 3 - jepsen.util 3 :ok :write [1 11] INFO [2022-01-10 12:53:13,624] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:13,624] jepsen worker 38 - jepsen.util 38 :ok :read [12 24] INFO [2022-01-10 12:53:13,625] jepsen worker 38 - jepsen.util 38 :invoke :write [12 26] INFO [2022-01-10 12:53:13,625] jepsen worker 3 - jepsen.util 3 :ok :read [1 11] INFO [2022-01-10 12:53:13,626] jepsen worker 3 - jepsen.util 3 :invoke :write [1 33] INFO [2022-01-10 12:53:13,627] jepsen worker 89 - jepsen.util 179 :ok :write [29 30] INFO [2022-01-10 12:53:13,627] jepsen worker 89 - jepsen.util 179 :invoke :write [29 23] INFO [2022-01-10 12:53:13,630] jepsen worker 85 - jepsen.util 175 :ok :write [28 13] INFO [2022-01-10 12:53:13,630] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:13,631] jepsen worker 85 - jepsen.util 175 :ok :read [28 13] INFO [2022-01-10 12:53:13,631] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:13,632] jepsen worker 85 - jepsen.util 175 :ok :read [28 13] INFO [2022-01-10 12:53:13,632] jepsen worker 41 - jepsen.util 41 :ok :write [13 46] INFO [2022-01-10 12:53:13,633] jepsen worker 41 - jepsen.util 41 :invoke :write [13 19] INFO [2022-01-10 12:53:13,633] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:13,634] jepsen worker 85 - jepsen.util 175 :ok :read [28 13] INFO [2022-01-10 12:53:13,635] jepsen worker 85 - jepsen.util 175 :invoke :write [28 19] INFO [2022-01-10 12:53:13,640] jepsen worker 38 - jepsen.util 38 :ok :write [12 26] INFO [2022-01-10 12:53:13,641] jepsen worker 38 - jepsen.util 38 :invoke :write [12 43] INFO [2022-01-10 12:53:13,641] jepsen worker 1 - jepsen.util 1 :ok :write [0 12] INFO [2022-01-10 12:53:13,642] jepsen worker 1 - jepsen.util 1 :invoke :write [0 26] INFO [2022-01-10 12:53:13,647] jepsen worker 3 - jepsen.util 3 :ok :write [1 33] INFO [2022-01-10 12:53:13,648] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:13,648] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:13,649] jepsen worker 89 - jepsen.util 179 :ok :write [29 23] INFO [2022-01-10 12:53:13,649] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:13,649] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,650] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:13,650] jepsen worker 89 - jepsen.util 179 :ok :read [29 23] INFO [2022-01-10 12:53:13,650] jepsen worker 3 - jepsen.util 3 :invoke :write [1 28] INFO [2022-01-10 12:53:13,651] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,651] jepsen worker 89 - jepsen.util 179 :ok :read [29 23] INFO [2022-01-10 12:53:13,652] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,653] jepsen worker 89 - jepsen.util 179 :ok :read [29 23] INFO [2022-01-10 12:53:13,653] jepsen worker 85 - jepsen.util 175 :ok :write [28 19] INFO [2022-01-10 12:53:13,653] jepsen worker 85 - jepsen.util 175 :invoke :write [28 15] INFO [2022-01-10 12:53:13,654] jepsen worker 89 - jepsen.util 179 :invoke :write [29 5] INFO [2022-01-10 12:53:13,654] jepsen worker 41 - jepsen.util 41 :ok :write [13 19] INFO [2022-01-10 12:53:13,655] jepsen worker 41 - jepsen.util 41 :invoke :write [13 19] INFO [2022-01-10 12:53:13,661] jepsen worker 1 - jepsen.util 1 :ok :write [0 26] INFO [2022-01-10 12:53:13,662] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,662] jepsen worker 38 - jepsen.util 38 :ok :write [12 43] INFO [2022-01-10 12:53:13,663] jepsen worker 1 - jepsen.util 1 :ok :read [0 26] INFO [2022-01-10 12:53:13,663] jepsen worker 38 - jepsen.util 38 :invoke :write [12 15] INFO [2022-01-10 12:53:13,664] jepsen worker 1 - jepsen.util 1 :invoke :write [0 32] INFO [2022-01-10 12:53:13,670] jepsen worker 89 - jepsen.util 179 :ok :write [29 5] INFO [2022-01-10 12:53:13,671] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,672] jepsen worker 89 - jepsen.util 179 :ok :read [29 5] INFO [2022-01-10 12:53:13,672] jepsen worker 3 - jepsen.util 3 :ok :write [1 28] INFO [2022-01-10 12:53:13,672] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,672] jepsen worker 3 - jepsen.util 3 :invoke :write [1 48] INFO [2022-01-10 12:53:13,673] jepsen worker 89 - jepsen.util 179 :ok :read [29 5] INFO [2022-01-10 12:53:13,673] jepsen worker 85 - jepsen.util 175 :ok :write [28 15] INFO [2022-01-10 12:53:13,673] jepsen worker 41 - jepsen.util 41 :ok :write [13 19] INFO [2022-01-10 12:53:13,674] jepsen worker 89 - jepsen.util 179 :invoke :write [29 39] INFO [2022-01-10 12:53:13,674] jepsen worker 85 - jepsen.util 175 :invoke :write [28 21] INFO [2022-01-10 12:53:13,675] jepsen worker 41 - jepsen.util 41 :invoke :write [13 40] INFO [2022-01-10 12:53:13,681] jepsen worker 1 - jepsen.util 1 :ok :write [0 32] INFO [2022-01-10 12:53:13,682] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,683] jepsen worker 38 - jepsen.util 38 :ok :write [12 15] INFO [2022-01-10 12:53:13,683] jepsen worker 77 - jepsen.util 167 :ok :write [25 0] INFO [2022-01-10 12:53:13,683] jepsen worker 1 - jepsen.util 1 :ok :read [0 32] INFO [2022-01-10 12:53:13,683] jepsen worker 1 - jepsen.util 1 :invoke :write [0 3] INFO [2022-01-10 12:53:13,684] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,684] jepsen worker 77 - jepsen.util 167 :invoke :write [25 39] INFO [2022-01-10 12:53:13,685] jepsen worker 38 - jepsen.util 38 :ok :read [12 15] INFO [2022-01-10 12:53:13,686] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,687] jepsen worker 38 - jepsen.util 38 :ok :read [12 15] INFO [2022-01-10 12:53:13,687] jepsen worker 38 - jepsen.util 38 :invoke :write [12 39] INFO [2022-01-10 12:53:13,688] jepsen worker 52 - jepsen.util 52 :ok :write [17 9] INFO [2022-01-10 12:53:13,689] jepsen worker 52 - jepsen.util 52 :invoke :write [17 24] INFO [2022-01-10 12:53:13,689] jepsen worker 89 - jepsen.util 179 :ok :write [29 39] INFO [2022-01-10 12:53:13,690] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,691] jepsen worker 3 - jepsen.util 3 :ok :write [1 48] INFO [2022-01-10 12:53:13,691] jepsen worker 89 - jepsen.util 179 :ok :read [29 39] INFO [2022-01-10 12:53:13,691] jepsen worker 3 - jepsen.util 3 :invoke :write [1 31] INFO [2022-01-10 12:53:13,692] jepsen worker 14 - jepsen.util 104 :ok :write [4 9] INFO [2022-01-10 12:53:13,692] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,692] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,693] jepsen worker 89 - jepsen.util 179 :ok :read [29 39] INFO [2022-01-10 12:53:13,693] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,693] jepsen worker 14 - jepsen.util 104 :ok :read [4 9] INFO [2022-01-10 12:53:13,694] jepsen worker 89 - jepsen.util 179 :ok :read [29 39] INFO [2022-01-10 12:53:13,694] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,695] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,695] jepsen worker 41 - jepsen.util 41 :ok :write [13 40] INFO [2022-01-10 12:53:13,695] jepsen worker 14 - jepsen.util 104 :ok :read [4 9] INFO [2022-01-10 12:53:13,695] jepsen worker 71 - jepsen.util 71 :ok :write [23 15] INFO [2022-01-10 12:53:13,696] jepsen worker 89 - jepsen.util 179 :ok :read [29 39] INFO [2022-01-10 12:53:13,696] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,697] jepsen worker 86 - jepsen.util 86 :ok :write [28 24] INFO [2022-01-10 12:53:13,697] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,697] jepsen worker 41 - jepsen.util 41 :ok :read [13 40] INFO [2022-01-10 12:53:13,698] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:13,698] jepsen worker 35 - jepsen.util 125 :ok :write [11 21] INFO [2022-01-10 12:53:13,698] jepsen worker 89 - jepsen.util 179 :ok :read [29 39] INFO [2022-01-10 12:53:13,698] jepsen worker 14 - jepsen.util 104 :invoke :write [4 35] INFO [2022-01-10 12:53:13,698] jepsen worker 71 - jepsen.util 71 :ok :read [23 15] INFO [2022-01-10 12:53:13,699] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,699] jepsen worker 78 - jepsen.util 78 :ok :write [26 37] INFO [2022-01-10 12:53:13,701] jepsen worker 41 - jepsen.util 41 :ok :read [13 40] INFO [2022-01-10 12:53:13,701] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:13,702] jepsen worker 89 - jepsen.util 179 :invoke :write [29 39] INFO [2022-01-10 12:53:13,702] jepsen worker 35 - jepsen.util 125 :ok :read [11 21] INFO [2022-01-10 12:53:13,702] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,703] jepsen worker 41 - jepsen.util 41 :ok :read [13 40] INFO [2022-01-10 12:53:13,704] jepsen worker 86 - jepsen.util 86 :invoke :write [28 5] INFO [2022-01-10 12:53:13,705] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,705] jepsen worker 1 - jepsen.util 1 :ok :write [0 3] INFO [2022-01-10 12:53:13,706] jepsen worker 41 - jepsen.util 41 :ok :read [13 40] INFO [2022-01-10 12:53:13,706] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,706] jepsen worker 38 - jepsen.util 38 :ok :write [12 39] INFO [2022-01-10 12:53:13,707] jepsen worker 52 - jepsen.util 52 :ok :write [17 24] INFO [2022-01-10 12:53:13,707] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:13,708] jepsen worker 1 - jepsen.util 1 :ok :read [0 3] INFO [2022-01-10 12:53:13,708] jepsen worker 49 - jepsen.util 139 :ok :write [16 35] INFO [2022-01-10 12:53:13,710] jepsen worker 49 - jepsen.util 139 :invoke :write [16 49] INFO [2022-01-10 12:53:13,711] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,711] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,712] jepsen worker 78 - jepsen.util 78 :ok :read [26 37] INFO [2022-01-10 12:53:13,712] jepsen worker 1 - jepsen.util 1 :invoke :write [0 0] INFO [2022-01-10 12:53:13,712] jepsen worker 41 - jepsen.util 41 :ok :read [13 40] INFO [2022-01-10 12:53:13,713] jepsen worker 71 - jepsen.util 71 :invoke :write [23 48] INFO [2022-01-10 12:53:13,713] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,713] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,714] jepsen worker 83 - jepsen.util 173 :ok :write [27 9] INFO [2022-01-10 12:53:13,713] jepsen worker 35 - jepsen.util 125 :ok :read [11 21] INFO [2022-01-10 12:53:13,714] jepsen worker 3 - jepsen.util 3 :ok :write [1 31] INFO [2022-01-10 12:53:13,714] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,714] jepsen worker 41 - jepsen.util 41 :ok :read [13 40] INFO [2022-01-10 12:53:13,715] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:13,715] jepsen worker 52 - jepsen.util 52 :ok :read [17 24] INFO [2022-01-10 12:53:13,715] jepsen worker 38 - jepsen.util 38 :ok :read [12 39] INFO [2022-01-10 12:53:13,715] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:13,715] jepsen worker 83 - jepsen.util 173 :ok :read [27 9] INFO [2022-01-10 12:53:13,716] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:13,716] jepsen worker 35 - jepsen.util 125 :ok :read [11 21] INFO [2022-01-10 12:53:13,717] jepsen worker 38 - jepsen.util 38 :invoke :write [12 31] INFO [2022-01-10 12:53:13,717] jepsen worker 3 - jepsen.util 3 :ok :read [1 31] INFO [2022-01-10 12:53:13,717] jepsen worker 89 - jepsen.util 179 :ok :write [29 39] INFO [2022-01-10 12:53:13,718] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:13,718] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:13,719] jepsen worker 52 - jepsen.util 52 :ok :read [17 24] INFO [2022-01-10 12:53:13,719] jepsen worker 83 - jepsen.util 173 :invoke :write [27 46] INFO [2022-01-10 12:53:13,719] jepsen worker 35 - jepsen.util 125 :ok :read [11 21] INFO [2022-01-10 12:53:13,720] jepsen worker 67 - jepsen.util 67 :ok :write [22 35] INFO [2022-01-10 12:53:13,720] jepsen worker 78 - jepsen.util 78 :invoke :write [26 47] INFO [2022-01-10 12:53:13,720] jepsen worker 67 - jepsen.util 67 :invoke :write [22 15] INFO [2022-01-10 12:53:13,721] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:13,721] jepsen worker 14 - jepsen.util 104 :ok :write [4 35] INFO [2022-01-10 12:53:13,722] jepsen worker 89 - jepsen.util 179 :invoke :write [29 42] INFO [2022-01-10 12:53:13,722] jepsen worker 3 - jepsen.util 3 :ok :read [1 31] INFO [2022-01-10 12:53:13,722] jepsen worker 52 - jepsen.util 52 :invoke :write [17 5] INFO [2022-01-10 12:53:13,723] jepsen worker 41 - jepsen.util 41 :invoke :write [13 32] INFO [2022-01-10 12:53:13,724] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:13,724] jepsen worker 3 - jepsen.util 3 :invoke :write [1 17] INFO [2022-01-10 12:53:13,725] jepsen worker 35 - jepsen.util 125 :ok :read [11 21] INFO [2022-01-10 12:53:13,725] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:13,726] jepsen worker 14 - jepsen.util 104 :invoke :write [4 36] INFO [2022-01-10 12:53:13,726] jepsen worker 35 - jepsen.util 125 :ok :read [11 21] INFO [2022-01-10 12:53:13,727] jepsen worker 35 - jepsen.util 125 :invoke :write [11 24] INFO [2022-01-10 12:53:13,727] jepsen worker 8 - jepsen.util 8 :ok :write [2 0] INFO [2022-01-10 12:53:13,728] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:13,729] jepsen worker 8 - jepsen.util 8 :ok :read [2 0] INFO [2022-01-10 12:53:13,729] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:13,730] jepsen worker 8 - jepsen.util 8 :ok :read [2 0] INFO [2022-01-10 12:53:13,731] jepsen worker 71 - jepsen.util 71 :ok :write [23 48] INFO [2022-01-10 12:53:13,731] jepsen worker 8 - jepsen.util 8 :invoke :write [2 9] INFO [2022-01-10 12:53:13,732] jepsen worker 71 - jepsen.util 71 :invoke :write [23 17] INFO [2022-01-10 12:53:13,732] jepsen worker 1 - jepsen.util 1 :ok :write [0 0] INFO [2022-01-10 12:53:13,733] jepsen worker 1 - jepsen.util 1 :invoke :write [0 15] INFO [2022-01-10 12:53:13,735] jepsen worker 85 - jepsen.util 175 :ok :write [28 21] INFO [2022-01-10 12:53:13,736] jepsen worker 85 - jepsen.util 175 :invoke :write [28 7] INFO [2022-01-10 12:53:13,738] jepsen worker 15 - jepsen.util 15 :ok :write [5 8] INFO [2022-01-10 12:53:13,738] jepsen worker 15 - jepsen.util 15 :invoke :write [5 12] INFO [2022-01-10 12:53:13,741] jepsen worker 78 - jepsen.util 78 :ok :write [26 47] INFO [2022-01-10 12:53:13,742] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:13,743] jepsen worker 78 - jepsen.util 78 :ok :read [26 47] INFO [2022-01-10 12:53:13,743] jepsen worker 43 - jepsen.util 43 :ok :write [14 4] INFO [2022-01-10 12:53:13,744] jepsen worker 78 - jepsen.util 78 :invoke :write [26 44] INFO [2022-01-10 12:53:13,744] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,745] jepsen worker 43 - jepsen.util 43 :ok :read [14 4] INFO [2022-01-10 12:53:13,746] jepsen worker 43 - jepsen.util 43 :invoke :write [14 37] INFO [2022-01-10 12:53:13,747] jepsen worker 38 - jepsen.util 38 :ok :write [12 31] INFO [2022-01-10 12:53:13,748] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,749] jepsen worker 38 - jepsen.util 38 :ok :read [12 31] INFO [2022-01-10 12:53:13,750] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,750] jepsen worker 38 - jepsen.util 38 :ok :read [12 31] INFO [2022-01-10 12:53:13,751] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:13,752] jepsen worker 62 - jepsen.util 152 :ok :write [20 19] INFO [2022-01-10 12:53:13,753] jepsen worker 62 - jepsen.util 152 :invoke :write [20 16] INFO [2022-01-10 12:53:13,755] jepsen worker 52 - jepsen.util 52 :ok :write [17 5] INFO [2022-01-10 12:53:13,755] jepsen worker 52 - jepsen.util 52 :invoke :write [17 41] INFO [2022-01-10 12:53:13,759] jepsen worker 77 - jepsen.util 167 :ok :write [25 39] INFO [2022-01-10 12:53:13,759] jepsen worker 77 - jepsen.util 167 :invoke :write [25 18] INFO [2022-01-10 12:53:13,765] jepsen worker 14 - jepsen.util 104 :ok :write [4 36] INFO [2022-01-10 12:53:13,766] jepsen worker 14 - jepsen.util 104 :invoke :write [4 20] INFO [2022-01-10 12:53:13,791] jepsen worker 43 - jepsen.util 43 :ok :write [14 37] INFO [2022-01-10 12:53:13,792] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,793] jepsen worker 43 - jepsen.util 43 :ok :read [14 37] INFO [2022-01-10 12:53:13,793] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,794] jepsen worker 43 - jepsen.util 43 :ok :read [14 37] INFO [2022-01-10 12:53:13,795] jepsen worker 43 - jepsen.util 43 :invoke :write [14 22] INFO [2022-01-10 12:53:13,796] jepsen worker 62 - jepsen.util 152 :ok :write [20 16] INFO [2022-01-10 12:53:13,796] jepsen worker 62 - jepsen.util 152 :invoke :write [20 27] INFO [2022-01-10 12:53:13,846] jepsen worker 3 - jepsen.util 3 :ok :write [1 17] INFO [2022-01-10 12:53:13,846] jepsen worker 3 - jepsen.util 3 :invoke :write [1 43] INFO [2022-01-10 12:53:13,846] jepsen worker 49 - jepsen.util 139 :ok :write [16 49] INFO [2022-01-10 12:53:13,847] jepsen worker 49 - jepsen.util 139 :invoke :write [16 34] INFO [2022-01-10 12:53:13,854] jepsen worker 83 - jepsen.util 173 :ok :write [27 46] INFO [2022-01-10 12:53:13,854] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:13,855] jepsen worker 83 - jepsen.util 173 :ok :read [27 46] INFO [2022-01-10 12:53:13,855] jepsen worker 83 - jepsen.util 173 :invoke :write [27 25] INFO [2022-01-10 12:53:13,863] jepsen worker 89 - jepsen.util 179 :ok :write [29 42] INFO [2022-01-10 12:53:13,864] jepsen worker 89 - jepsen.util 179 :invoke :write [29 15] INFO [2022-01-10 12:53:13,867] jepsen worker 67 - jepsen.util 67 :ok :write [22 15] INFO [2022-01-10 12:53:13,867] jepsen worker 67 - jepsen.util 67 :invoke :write [22 14] INFO [2022-01-10 12:53:13,872] jepsen worker 41 - jepsen.util 41 :ok :write [13 32] INFO [2022-01-10 12:53:13,872] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,873] jepsen worker 41 - jepsen.util 41 :ok :read [13 32] INFO [2022-01-10 12:53:13,873] jepsen worker 41 - jepsen.util 41 :invoke :write [13 13] INFO [2022-01-10 12:53:13,878] jepsen worker 8 - jepsen.util 8 :ok :write [2 9] INFO [2022-01-10 12:53:13,878] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:13,879] jepsen worker 8 - jepsen.util 8 :ok :read [2 9] INFO [2022-01-10 12:53:13,880] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:13,881] jepsen worker 8 - jepsen.util 8 :ok :read [2 9] INFO [2022-01-10 12:53:13,881] jepsen worker 8 - jepsen.util 8 :invoke :write [2 21] INFO [2022-01-10 12:53:13,883] jepsen worker 35 - jepsen.util 125 :ok :write [11 24] INFO [2022-01-10 12:53:13,883] jepsen worker 35 - jepsen.util 125 :invoke :write [11 24] INFO [2022-01-10 12:53:13,888] jepsen worker 1 - jepsen.util 1 :ok :write [0 15] INFO [2022-01-10 12:53:13,889] jepsen worker 1 - jepsen.util 1 :invoke :write [0 12] INFO [2022-01-10 12:53:13,894] jepsen worker 15 - jepsen.util 15 :ok :write [5 12] INFO [2022-01-10 12:53:13,894] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,895] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:13,895] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,896] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:13,897] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,898] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:13,898] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,899] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:13,899] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,900] jepsen worker 49 - jepsen.util 139 :ok :write [16 34] INFO [2022-01-10 12:53:13,900] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:13,900] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:13,900] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,901] jepsen worker 49 - jepsen.util 139 :ok :read [16 34] INFO [2022-01-10 12:53:13,901] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:13,902] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:13,902] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,902] jepsen worker 49 - jepsen.util 139 :ok :read [16 34] INFO [2022-01-10 12:53:13,903] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:13,903] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:13,904] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,904] jepsen worker 49 - jepsen.util 139 :ok :read [16 34] INFO [2022-01-10 12:53:13,905] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:13,905] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:13,905] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,905] jepsen worker 83 - jepsen.util 173 :ok :write [27 25] INFO [2022-01-10 12:53:13,906] jepsen worker 49 - jepsen.util 139 :ok :read [16 34] INFO [2022-01-10 12:53:13,906] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:13,906] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:13,907] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:13,907] jepsen worker 83 - jepsen.util 173 :ok :read [27 25] INFO [2022-01-10 12:53:13,907] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,908] jepsen worker 49 - jepsen.util 139 :ok :read [16 34] INFO [2022-01-10 12:53:13,908] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:13,908] jepsen worker 83 - jepsen.util 173 :invoke :write [27 22] INFO [2022-01-10 12:53:13,909] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:13,909] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:13,910] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:13,910] jepsen worker 77 - jepsen.util 167 :ok :write [25 18] INFO [2022-01-10 12:53:13,910] jepsen worker 49 - jepsen.util 139 :ok :read [16 34] INFO [2022-01-10 12:53:13,911] jepsen worker 15 - jepsen.util 15 :invoke :write [5 33] INFO [2022-01-10 12:53:13,911] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:13,911] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:13,912] jepsen worker 77 - jepsen.util 167 :ok :read [25 18] INFO [2022-01-10 12:53:13,912] jepsen worker 49 - jepsen.util 139 :ok :read [16 34] INFO [2022-01-10 12:53:13,912] jepsen worker 71 - jepsen.util 71 :ok :write [23 17] INFO [2022-01-10 12:53:13,912] jepsen worker 77 - jepsen.util 167 :invoke :write [25 20] INFO [2022-01-10 12:53:13,913] jepsen worker 49 - jepsen.util 139 :invoke :write [16 6] INFO [2022-01-10 12:53:13,913] jepsen worker 71 - jepsen.util 71 :invoke :write [23 6] INFO [2022-01-10 12:53:13,915] jepsen worker 23 - jepsen.util 23 :ok :write [7 9] INFO [2022-01-10 12:53:13,916] jepsen worker 23 - jepsen.util 23 :invoke :write [7 0] INFO [2022-01-10 12:53:13,919] jepsen worker 85 - jepsen.util 175 :ok :write [28 7] INFO [2022-01-10 12:53:13,919] jepsen worker 85 - jepsen.util 175 :invoke :write [28 13] INFO [2022-01-10 12:53:13,922] jepsen worker 43 - jepsen.util 43 :ok :write [14 22] INFO [2022-01-10 12:53:13,923] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,923] jepsen worker 43 - jepsen.util 43 :ok :read [14 22] INFO [2022-01-10 12:53:13,924] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:13,925] jepsen worker 43 - jepsen.util 43 :ok :read [14 22] INFO [2022-01-10 12:53:13,925] jepsen worker 78 - jepsen.util 78 :ok :write [26 44] INFO [2022-01-10 12:53:13,925] jepsen worker 43 - jepsen.util 43 :invoke :write [14 18] INFO [2022-01-10 12:53:13,926] jepsen worker 78 - jepsen.util 78 :invoke :write [26 29] INFO [2022-01-10 12:53:13,926] jepsen worker 62 - jepsen.util 152 :ok :write [20 27] INFO [2022-01-10 12:53:13,926] jepsen worker 62 - jepsen.util 152 :invoke :write [20 23] INFO [2022-01-10 12:53:13,933] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:13,933] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,934] jepsen worker 89 - jepsen.util 179 :ok :write [29 15] INFO [2022-01-10 12:53:13,934] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:13,935] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,935] jepsen worker 38 - jepsen.util 38 :invoke :write [12 9] INFO [2022-01-10 12:53:13,936] jepsen worker 89 - jepsen.util 179 :ok :read [29 15] INFO [2022-01-10 12:53:13,936] jepsen worker 89 - jepsen.util 179 :invoke :write [29 20] INFO [2022-01-10 12:53:13,940] jepsen worker 52 - jepsen.util 52 :ok :write [17 41] INFO [2022-01-10 12:53:13,940] jepsen worker 52 - jepsen.util 52 :invoke :write [17 35] INFO [2022-01-10 12:53:13,941] jepsen worker 41 - jepsen.util 41 :ok :write [13 13] INFO [2022-01-10 12:53:13,942] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,943] jepsen worker 41 - jepsen.util 41 :ok :read [13 13] INFO [2022-01-10 12:53:13,943] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,944] jepsen worker 41 - jepsen.util 41 :ok :read [13 13] INFO [2022-01-10 12:53:13,944] jepsen worker 41 - jepsen.util 41 :invoke :write [13 11] INFO [2022-01-10 12:53:13,945] jepsen worker 14 - jepsen.util 104 :ok :write [4 20] INFO [2022-01-10 12:53:13,946] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:13,947] jepsen worker 1 - jepsen.util 1 :ok :write [0 12] INFO [2022-01-10 12:53:13,947] jepsen worker 14 - jepsen.util 104 :ok :read [4 20] INFO [2022-01-10 12:53:13,947] jepsen worker 1 - jepsen.util 1 :invoke :write [0 43] INFO [2022-01-10 12:53:13,947] jepsen worker 14 - jepsen.util 104 :invoke :write [4 36] INFO [2022-01-10 12:53:13,953] jepsen worker 3 - jepsen.util 3 :ok :write [1 43] INFO [2022-01-10 12:53:13,953] jepsen worker 3 - jepsen.util 3 :invoke :write [1 2] INFO [2022-01-10 12:53:13,965] jepsen worker 85 - jepsen.util 175 :ok :write [28 13] INFO [2022-01-10 12:53:13,965] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:13,966] jepsen worker 85 - jepsen.util 175 :ok :read [28 13] INFO [2022-01-10 12:53:13,967] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:13,967] jepsen worker 85 - jepsen.util 175 :ok :read [28 13] INFO [2022-01-10 12:53:13,968] jepsen worker 85 - jepsen.util 175 :invoke :write [28 15] INFO [2022-01-10 12:53:13,972] jepsen worker 89 - jepsen.util 179 :ok :write [29 20] INFO [2022-01-10 12:53:13,973] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:13,974] jepsen worker 89 - jepsen.util 179 :ok :read [29 20] INFO [2022-01-10 12:53:13,974] jepsen worker 38 - jepsen.util 38 :ok :write [12 9] INFO [2022-01-10 12:53:13,974] jepsen worker 89 - jepsen.util 179 :invoke :write [29 11] INFO [2022-01-10 12:53:13,975] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,975] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:13,976] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,977] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:13,977] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:13,978] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:13,979] jepsen worker 39 - jepsen.util 39 :ok :write [13 35] INFO [2022-01-10 12:53:13,979] jepsen worker 38 - jepsen.util 38 :invoke :write [12 6] INFO [2022-01-10 12:53:13,979] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:13,980] jepsen worker 39 - jepsen.util 39 :ok :read [13 35] INFO [2022-01-10 12:53:13,981] jepsen worker 39 - jepsen.util 39 :invoke :write [13 22] INFO [2022-01-10 12:53:13,985] jepsen worker 1 - jepsen.util 1 :ok :write [0 43] INFO [2022-01-10 12:53:13,986] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,987] jepsen worker 1 - jepsen.util 1 :ok :read [0 43] INFO [2022-01-10 12:53:13,987] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,988] jepsen worker 1 - jepsen.util 1 :ok :read [0 43] INFO [2022-01-10 12:53:13,989] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:13,989] jepsen worker 89 - jepsen.util 179 :ok :write [29 11] INFO [2022-01-10 12:53:13,990] jepsen worker 1 - jepsen.util 1 :ok :read [0 43] INFO [2022-01-10 12:53:13,990] jepsen worker 89 - jepsen.util 179 :invoke :write [29 14] INFO [2022-01-10 12:53:13,990] jepsen worker 1 - jepsen.util 1 :invoke :write [0 24] INFO [2022-01-10 12:53:13,993] jepsen worker 3 - jepsen.util 3 :ok :write [1 2] INFO [2022-01-10 12:53:13,994] jepsen worker 3 - jepsen.util 3 :invoke :write [1 42] INFO [2022-01-10 12:53:13,995] jepsen worker 39 - jepsen.util 39 :ok :write [13 22] INFO [2022-01-10 12:53:13,996] jepsen worker 39 - jepsen.util 39 :invoke :write [13 48] INFO [2022-01-10 12:53:14,002] jepsen worker 85 - jepsen.util 175 :ok :write [28 15] INFO [2022-01-10 12:53:14,003] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:14,004] jepsen worker 85 - jepsen.util 175 :ok :read [28 15] INFO [2022-01-10 12:53:14,004] jepsen worker 1 - jepsen.util 1 :ok :write [0 24] INFO [2022-01-10 12:53:14,004] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:14,005] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:14,005] jepsen worker 85 - jepsen.util 175 :ok :read [28 15] INFO [2022-01-10 12:53:14,006] jepsen worker 1 - jepsen.util 1 :ok :read [0 24] INFO [2022-01-10 12:53:14,006] jepsen worker 85 - jepsen.util 175 :invoke :write [28 13] INFO [2022-01-10 12:53:14,006] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:14,007] jepsen worker 1 - jepsen.util 1 :ok :read [0 24] INFO [2022-01-10 12:53:14,008] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:14,008] jepsen worker 38 - jepsen.util 38 :ok :write [12 6] INFO [2022-01-10 12:53:14,008] jepsen worker 1 - jepsen.util 1 :ok :read [0 24] INFO [2022-01-10 12:53:14,009] jepsen worker 38 - jepsen.util 38 :invoke :write [12 36] INFO [2022-01-10 12:53:14,009] jepsen worker 89 - jepsen.util 179 :ok :write [29 14] INFO [2022-01-10 12:53:14,009] jepsen worker 1 - jepsen.util 1 :invoke :write [0 44] INFO [2022-01-10 12:53:14,010] jepsen worker 89 - jepsen.util 179 :invoke :write [29 38] INFO [2022-01-10 12:53:14,014] jepsen worker 3 - jepsen.util 3 :ok :write [1 42] INFO [2022-01-10 12:53:14,015] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:14,016] jepsen worker 39 - jepsen.util 39 :ok :write [13 48] INFO [2022-01-10 12:53:14,016] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:14,017] jepsen worker 3 - jepsen.util 3 :invoke :write [1 45] INFO [2022-01-10 12:53:14,017] jepsen worker 39 - jepsen.util 39 :invoke :write [13 3] INFO [2022-01-10 12:53:14,020] jepsen worker 1 - jepsen.util 1 :ok :write [0 44] INFO [2022-01-10 12:53:14,021] jepsen worker 1 - jepsen.util 1 :invoke :write [0 0] INFO [2022-01-10 12:53:14,023] jepsen worker 85 - jepsen.util 175 :ok :write [28 13] INFO [2022-01-10 12:53:14,024] jepsen worker 85 - jepsen.util 175 :invoke :write [28 30] INFO [2022-01-10 12:53:14,028] jepsen worker 89 - jepsen.util 179 :ok :write [29 38] INFO [2022-01-10 12:53:14,029] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:14,029] jepsen worker 38 - jepsen.util 38 :ok :write [12 36] INFO [2022-01-10 12:53:14,030] jepsen worker 89 - jepsen.util 179 :ok :read [29 38] INFO [2022-01-10 12:53:14,030] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:14,031] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:14,031] jepsen worker 38 - jepsen.util 38 :ok :read [12 36] INFO [2022-01-10 12:53:14,032] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:14,032] jepsen worker 89 - jepsen.util 179 :ok :read [29 38] INFO [2022-01-10 12:53:14,032] jepsen worker 89 - jepsen.util 179 :invoke :write [29 46] INFO [2022-01-10 12:53:14,033] jepsen worker 38 - jepsen.util 38 :ok :read [12 36] INFO [2022-01-10 12:53:14,033] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:14,034] jepsen worker 38 - jepsen.util 38 :ok :read [12 36] INFO [2022-01-10 12:53:14,035] jepsen worker 38 - jepsen.util 38 :invoke :write [12 30] INFO [2022-01-10 12:53:14,037] jepsen worker 3 - jepsen.util 3 :ok :write [1 45] INFO [2022-01-10 12:53:14,038] jepsen worker 3 - jepsen.util 3 :invoke :write [1 14] INFO [2022-01-10 12:53:14,054] jepsen worker 89 - jepsen.util 179 :ok :write [29 46] INFO [2022-01-10 12:53:14,054] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:14,055] jepsen worker 89 - jepsen.util 179 :ok :read [29 46] INFO [2022-01-10 12:53:14,056] jepsen worker 3 - jepsen.util 3 :ok :write [1 14] INFO [2022-01-10 12:53:14,056] jepsen worker 89 - jepsen.util 179 :invoke :write [29 48] INFO [2022-01-10 12:53:14,056] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:14,057] jepsen worker 3 - jepsen.util 3 :ok :read [1 14] INFO [2022-01-10 12:53:14,058] jepsen worker 3 - jepsen.util 3 :invoke :write [1 12] INFO [2022-01-10 12:53:14,060] jepsen worker 38 - jepsen.util 38 :ok :write [12 30] INFO [2022-01-10 12:53:14,061] jepsen worker 38 - jepsen.util 38 :invoke :write [12 14] INFO [2022-01-10 12:53:14,063] jepsen worker 85 - jepsen.util 175 :ok :write [28 30] INFO [2022-01-10 12:53:14,063] jepsen worker 39 - jepsen.util 39 :ok :write [13 3] INFO [2022-01-10 12:53:14,063] jepsen worker 1 - jepsen.util 1 :ok :write [0 0] INFO [2022-01-10 12:53:14,064] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:14,065] jepsen worker 85 - jepsen.util 175 :ok :read [28 30] INFO [2022-01-10 12:53:14,065] jepsen worker 39 - jepsen.util 39 :invoke :write [13 37] INFO [2022-01-10 12:53:14,066] jepsen worker 1 - jepsen.util 1 :invoke :write [0 26] INFO [2022-01-10 12:53:14,066] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:14,067] jepsen worker 85 - jepsen.util 175 :ok :read [28 30] INFO [2022-01-10 12:53:14,068] jepsen worker 85 - jepsen.util 175 :invoke :write [28 16] INFO [2022-01-10 12:53:14,071] jepsen worker 89 - jepsen.util 179 :ok :write [29 48] INFO [2022-01-10 12:53:14,071] jepsen worker 3 - jepsen.util 3 :ok :write [1 12] INFO [2022-01-10 12:53:14,071] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:14,072] jepsen worker 3 - jepsen.util 3 :invoke :write [1 42] INFO [2022-01-10 12:53:14,072] jepsen worker 89 - jepsen.util 179 :ok :read [29 48] INFO [2022-01-10 12:53:14,073] jepsen worker 89 - jepsen.util 179 :invoke :write [29 29] WARN [2022-01-10 12:53:14,076] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":722,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000001/000000005D716E8E","last_local_socket":"172.23.104.79:34680","total_dispatch_duration_us":18847,"total_server_duration_us":2656,"encode_duration_us":34,"operation_id":"0x109d","timeout_ms":10000,"last_server_duration_us":117,"total_duration_us":10008322},{"operation_name":"upsert","last_dispatch_duration_us":474,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000001/000000005D716E8E","last_local_socket":"172.23.104.79:34680","total_dispatch_duration_us":19075,"total_server_duration_us":3462,"encode_duration_us":42,"operation_id":"0xe26","timeout_ms":10000,"last_server_duration_us":56,"total_duration_us":10008186},{"operation_name":"upsert","last_dispatch_duration_us":524,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":103446,"total_server_duration_us":3166,"encode_duration_us":63,"operation_id":"0xd1d","timeout_ms":10000,"last_server_duration_us":56,"total_duration_us":10005302},{"operation_name":"upsert","last_dispatch_duration_us":6265223,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":6276123,"total_server_duration_us":2121,"encode_duration_us":57,"operation_id":"0xe4e","timeout_ms":10000,"last_server_duration_us":43,"total_duration_us":10004504},{"operation_name":"upsert","last_dispatch_duration_us":519,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":32075,"total_server_duration_us":2313,"encode_duration_us":41,"operation_id":"0x10c0","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10003621},{"operation_name":"upsert","last_dispatch_duration_us":667,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000001/000000005D716E8E","last_local_socket":"172.23.104.79:34680","total_dispatch_duration_us":18826,"total_server_duration_us":3028,"encode_duration_us":79,"operation_id":"0xd16","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10001385},{"operation_name":"upsert","last_dispatch_duration_us":9004367,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":9004367,"total_server_duration_us":9003430,"encode_duration_us":32,"operation_id":"0x1183","timeout_ms":10000,"last_server_duration_us":9003430,"total_duration_us":9004908},{"operation_name":"upsert","last_dispatch_duration_us":102173,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":115890,"total_server_duration_us":103396,"encode_duration_us":75,"operation_id":"0x9e2","timeout_ms":10000,"last_server_duration_us":101696,"total_duration_us":5774073},{"operation_name":"upsert","last_dispatch_duration_us":38462,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":48776,"total_server_duration_us":39603,"encode_duration_us":93,"operation_id":"0x12cd","timeout_ms":10000,"last_server_duration_us":37859,"total_duration_us":4760998},{"operation_name":"upsert","last_dispatch_duration_us":50333,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":61424,"total_server_duration_us":51660,"encode_duration_us":50,"operation_id":"0x10b5","timeout_ms":10000,"last_server_duration_us":49835,"total_duration_us":3690341}],"total_count":17}} INFO [2022-01-10 12:53:14,079] jepsen worker 38 - jepsen.util 38 :ok :write [12 14] INFO [2022-01-10 12:53:14,079] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:14,084] jepsen worker 38 - jepsen.util 38 :ok :read [12 14] INFO [2022-01-10 12:53:14,085] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:14,086] jepsen worker 38 - jepsen.util 38 :ok :read [12 14] INFO [2022-01-10 12:53:14,086] jepsen worker 1 - jepsen.util 1 :ok :write [0 26] INFO [2022-01-10 12:53:14,086] jepsen worker 38 - jepsen.util 38 :invoke :write [12 5] INFO [2022-01-10 12:53:14,087] jepsen worker 39 - jepsen.util 39 :ok :write [13 37] INFO [2022-01-10 12:53:14,087] jepsen worker 1 - jepsen.util 1 :invoke :write [0 30] INFO [2022-01-10 12:53:14,087] jepsen worker 85 - jepsen.util 175 :ok :write [28 16] INFO [2022-01-10 12:53:14,088] jepsen worker 39 - jepsen.util 39 :invoke :write [13 15] INFO [2022-01-10 12:53:14,088] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:14,089] jepsen worker 85 - jepsen.util 175 :ok :read [28 16] INFO [2022-01-10 12:53:14,089] jepsen worker 85 - jepsen.util 175 :invoke :write [28 0] INFO [2022-01-10 12:53:14,093] jepsen worker 3 - jepsen.util 3 :ok :write [1 42] INFO [2022-01-10 12:53:14,093] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:14,094] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:14,095] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:14,096] jepsen worker 89 - jepsen.util 179 :ok :write [29 29] INFO [2022-01-10 12:53:14,096] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:14,096] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:14,097] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:14,097] jepsen worker 89 - jepsen.util 179 :ok :read [29 29] INFO [2022-01-10 12:53:14,097] jepsen worker 89 - jepsen.util 179 :invoke :write [29 15] INFO [2022-01-10 12:53:14,098] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:14,098] jepsen worker 3 - jepsen.util 3 :invoke :write [1 38] INFO [2022-01-10 12:53:14,101] jepsen worker 38 - jepsen.util 38 :ok :write [12 5] INFO [2022-01-10 12:53:14,102] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:14,103] jepsen worker 38 - jepsen.util 38 :ok :read [12 5] INFO [2022-01-10 12:53:14,103] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:14,104] jepsen worker 38 - jepsen.util 38 :ok :read [12 5] INFO [2022-01-10 12:53:14,105] jepsen worker 38 - jepsen.util 38 :invoke :write [12 17] INFO [2022-01-10 12:53:14,108] jepsen worker 85 - jepsen.util 175 :ok :write [28 0] INFO [2022-01-10 12:53:14,109] jepsen worker 85 - jepsen.util 175 :invoke :write [28 30] INFO [2022-01-10 12:53:14,111] jepsen worker 1 - jepsen.util 1 :ok :write [0 30] INFO [2022-01-10 12:53:14,112] jepsen worker 1 - jepsen.util 1 :invoke :write [0 20] INFO [2022-01-10 12:53:14,138] jepsen worker 8 - jepsen.util 8 :ok :write [2 21] INFO [2022-01-10 12:53:14,138] jepsen worker 8 - jepsen.util 8 :invoke :write [2 37] INFO [2022-01-10 12:53:14,144] jepsen worker 35 - jepsen.util 125 :ok :write [11 24] INFO [2022-01-10 12:53:14,144] jepsen worker 35 - jepsen.util 125 :invoke :write [11 46] INFO [2022-01-10 12:53:14,145] jepsen worker 71 - jepsen.util 71 :ok :write [23 6] INFO [2022-01-10 12:53:14,146] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:14,147] jepsen worker 71 - jepsen.util 71 :ok :read [23 6] INFO [2022-01-10 12:53:14,147] jepsen worker 71 - jepsen.util 71 :invoke :write [23 15] INFO [2022-01-10 12:53:14,150] jepsen worker 15 - jepsen.util 15 :ok :write [5 33] INFO [2022-01-10 12:53:14,151] jepsen worker 78 - jepsen.util 78 :ok :write [26 29] INFO [2022-01-10 12:53:14,151] jepsen worker 15 - jepsen.util 15 :invoke :write [5 39] INFO [2022-01-10 12:53:14,151] jepsen worker 78 - jepsen.util 78 :invoke :write [26 21] INFO [2022-01-10 12:53:14,156] jepsen worker 67 - jepsen.util 67 :ok :write [22 14] INFO [2022-01-10 12:53:14,156] jepsen worker 52 - jepsen.util 52 :ok :write [17 35] INFO [2022-01-10 12:53:14,156] jepsen worker 67 - jepsen.util 67 :invoke :write [22 11] INFO [2022-01-10 12:53:14,157] jepsen worker 52 - jepsen.util 52 :invoke :write [17 15] INFO [2022-01-10 12:53:14,160] jepsen worker 49 - jepsen.util 139 :ok :write [16 6] INFO [2022-01-10 12:53:14,161] jepsen worker 49 - jepsen.util 139 :invoke :write [16 32] INFO [2022-01-10 12:53:14,161] jepsen worker 14 - jepsen.util 104 :ok :write [4 36] INFO [2022-01-10 12:53:14,162] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:14,163] jepsen worker 14 - jepsen.util 104 :ok :read [4 36] INFO [2022-01-10 12:53:14,163] jepsen worker 14 - jepsen.util 104 :invoke :write [4 33] INFO [2022-01-10 12:53:14,166] jepsen worker 83 - jepsen.util 173 :ok :write [27 22] INFO [2022-01-10 12:53:14,166] jepsen worker 83 - jepsen.util 173 :invoke :write [27 21] INFO [2022-01-10 12:53:14,167] jepsen worker 71 - jepsen.util 71 :ok :write [23 15] INFO [2022-01-10 12:53:14,167] jepsen worker 71 - jepsen.util 71 :invoke :read [23 nil] INFO [2022-01-10 12:53:14,168] jepsen worker 71 - jepsen.util 71 :ok :read [23 15] INFO [2022-01-10 12:53:14,168] jepsen worker 71 - jepsen.util 71 :invoke :write [23 34] INFO [2022-01-10 12:53:14,171] jepsen worker 78 - jepsen.util 78 :ok :write [26 21] INFO [2022-01-10 12:53:14,171] jepsen worker 77 - jepsen.util 167 :ok :write [25 20] INFO [2022-01-10 12:53:14,172] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:14,172] jepsen worker 77 - jepsen.util 167 :invoke :write [25 23] INFO [2022-01-10 12:53:14,173] jepsen worker 78 - jepsen.util 78 :ok :read [26 21] INFO [2022-01-10 12:53:14,173] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:14,174] jepsen worker 78 - jepsen.util 78 :ok :read [26 21] INFO [2022-01-10 12:53:14,175] jepsen worker 78 - jepsen.util 78 :invoke :write [26 23] INFO [2022-01-10 12:53:14,176] jepsen worker 52 - jepsen.util 52 :ok :write [17 15] INFO [2022-01-10 12:53:14,176] jepsen worker 52 - jepsen.util 52 :invoke :write [17 28] INFO [2022-01-10 12:53:14,179] jepsen worker 23 - jepsen.util 23 :ok :write [7 0] INFO [2022-01-10 12:53:14,180] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:14,180] jepsen worker 14 - jepsen.util 104 :ok :write [4 33] INFO [2022-01-10 12:53:14,181] jepsen worker 14 - jepsen.util 104 :invoke :write [4 28] INFO [2022-01-10 12:53:14,181] jepsen worker 23 - jepsen.util 23 :ok :read [7 0] INFO [2022-01-10 12:53:14,181] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:14,183] jepsen worker 23 - jepsen.util 23 :ok :read [7 0] INFO [2022-01-10 12:53:14,183] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:14,183] jepsen worker 43 - jepsen.util 43 :ok :write [14 18] INFO [2022-01-10 12:53:14,184] jepsen worker 23 - jepsen.util 23 :ok :read [7 0] INFO [2022-01-10 12:53:14,184] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:14,184] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:14,185] jepsen worker 43 - jepsen.util 43 :ok :read [14 18] INFO [2022-01-10 12:53:14,185] jepsen worker 23 - jepsen.util 23 :ok :read [7 0] INFO [2022-01-10 12:53:14,186] jepsen worker 43 - jepsen.util 43 :invoke :write [14 20] INFO [2022-01-10 12:53:14,186] jepsen worker 23 - jepsen.util 23 :invoke :write [7 11] INFO [2022-01-10 12:53:14,189] jepsen worker 62 - jepsen.util 152 :ok :write [20 23] INFO [2022-01-10 12:53:14,189] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:14,190] jepsen worker 62 - jepsen.util 152 :ok :read [20 23] INFO [2022-01-10 12:53:14,191] jepsen worker 62 - jepsen.util 152 :invoke :write [20 14] INFO [2022-01-10 12:53:14,210] jepsen worker 39 - jepsen.util 39 :ok :write [13 15] INFO [2022-01-10 12:53:14,211] jepsen worker 39 - jepsen.util 39 :invoke :write [13 49] INFO [2022-01-10 12:53:14,216] jepsen worker 89 - jepsen.util 179 :ok :write [29 15] INFO [2022-01-10 12:53:14,216] jepsen worker 4 - jepsen.util 4 :ok :write [1 18] INFO [2022-01-10 12:53:14,216] jepsen worker 89 - jepsen.util 179 :invoke :read [29 nil] INFO [2022-01-10 12:53:14,217] jepsen worker 4 - jepsen.util 4 :invoke :write [1 37] INFO [2022-01-10 12:53:14,217] jepsen worker 89 - jepsen.util 179 :ok :read [29 15] INFO [2022-01-10 12:53:14,217] jepsen worker 89 - jepsen.util 179 :invoke :write [29 33] INFO [2022-01-10 12:53:14,221] jepsen worker 38 - jepsen.util 38 :ok :write [12 17] INFO [2022-01-10 12:53:14,221] jepsen worker 38 - jepsen.util 38 :invoke :write [12 30] INFO [2022-01-10 12:53:14,223] jepsen worker 1 - jepsen.util 1 :ok :write [0 20] INFO [2022-01-10 12:53:14,223] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:14,225] jepsen worker 1 - jepsen.util 1 :ok :read [0 20] INFO [2022-01-10 12:53:14,225] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:14,226] jepsen worker 1 - jepsen.util 1 :ok :read [0 20] INFO [2022-01-10 12:53:14,226] jepsen worker 1 - jepsen.util 1 :invoke :write [0 31] INFO [2022-01-10 12:53:14,227] jepsen worker 85 - jepsen.util 175 :ok :write [28 30] INFO [2022-01-10 12:53:14,227] jepsen worker 85 - jepsen.util 175 :invoke :write [28 6] INFO [2022-01-10 12:53:14,230] jepsen worker 8 - jepsen.util 8 :ok :write [2 37] INFO [2022-01-10 12:53:14,231] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,232] jepsen worker 8 - jepsen.util 8 :ok :read [2 37] INFO [2022-01-10 12:53:14,232] jepsen worker 8 - jepsen.util 8 :invoke :write [2 3] INFO [2022-01-10 12:53:14,322] jepsen worker 71 - jepsen.util 71 :ok :write [23 34] INFO [2022-01-10 12:53:14,323] jepsen worker 71 - jepsen.util 71 :invoke :write [23 37] INFO [2022-01-10 12:53:14,325] jepsen worker 35 - jepsen.util 125 :ok :write [11 46] INFO [2022-01-10 12:53:14,325] jepsen worker 35 - jepsen.util 125 :invoke :write [11 13] INFO [2022-01-10 12:53:14,331] jepsen worker 15 - jepsen.util 15 :ok :write [5 39] INFO [2022-01-10 12:53:14,332] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,333] jepsen worker 15 - jepsen.util 15 :ok :read [5 39] INFO [2022-01-10 12:53:14,334] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,335] jepsen worker 15 - jepsen.util 15 :ok :read [5 39] INFO [2022-01-10 12:53:14,335] jepsen worker 15 - jepsen.util 15 :invoke :write [5 25] INFO [2022-01-10 12:53:14,336] jepsen worker 67 - jepsen.util 67 :ok :write [22 11] INFO [2022-01-10 12:53:14,337] jepsen worker 67 - jepsen.util 67 :invoke :write [22 4] INFO [2022-01-10 12:53:14,343] jepsen worker 49 - jepsen.util 139 :ok :write [16 32] INFO [2022-01-10 12:53:14,343] jepsen worker 49 - jepsen.util 139 :invoke :write [16 22] INFO [2022-01-10 12:53:14,347] jepsen worker 83 - jepsen.util 173 :ok :write [27 21] INFO [2022-01-10 12:53:14,347] jepsen worker 83 - jepsen.util 173 :invoke :write [27 31] INFO [2022-01-10 12:53:14,367] jepsen worker 8 - jepsen.util 8 :ok :write [2 3] INFO [2022-01-10 12:53:14,368] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,370] jepsen worker 8 - jepsen.util 8 :ok :read [2 3] INFO [2022-01-10 12:53:14,371] jepsen worker 8 - jepsen.util 8 :invoke :write [2 41] INFO [2022-01-10 12:53:14,373] jepsen worker 35 - jepsen.util 125 :ok :write [11 13] INFO [2022-01-10 12:53:14,374] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,375] jepsen worker 35 - jepsen.util 125 :ok :read [11 13] INFO [2022-01-10 12:53:14,375] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,376] jepsen worker 35 - jepsen.util 125 :ok :read [11 13] INFO [2022-01-10 12:53:14,377] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,377] jepsen worker 35 - jepsen.util 125 :ok :read [11 13] INFO [2022-01-10 12:53:14,378] jepsen worker 35 - jepsen.util 125 :invoke :write [11 29] INFO [2022-01-10 12:53:14,379] jepsen worker 15 - jepsen.util 15 :ok :write [5 25] INFO [2022-01-10 12:53:14,379] jepsen worker 15 - jepsen.util 15 :invoke :write [5 18] INFO [2022-01-10 12:53:14,383] jepsen worker 67 - jepsen.util 67 :ok :write [22 4] INFO [2022-01-10 12:53:14,383] jepsen worker 67 - jepsen.util 67 :invoke :write [22 47] INFO [2022-01-10 12:53:14,386] jepsen worker 49 - jepsen.util 139 :ok :write [16 22] INFO [2022-01-10 12:53:14,387] jepsen worker 49 - jepsen.util 139 :invoke :write [16 3] INFO [2022-01-10 12:53:14,391] jepsen worker 83 - jepsen.util 173 :ok :write [27 31] INFO [2022-01-10 12:53:14,391] jepsen worker 83 - jepsen.util 173 :invoke :write [27 0] INFO [2022-01-10 12:53:14,395] jepsen worker 8 - jepsen.util 8 :ok :write [2 41] INFO [2022-01-10 12:53:14,396] jepsen worker 8 - jepsen.util 8 :invoke :write [2 27] INFO [2022-01-10 12:53:14,400] jepsen worker 35 - jepsen.util 125 :ok :write [11 29] INFO [2022-01-10 12:53:14,401] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,402] jepsen worker 35 - jepsen.util 125 :ok :read [11 29] INFO [2022-01-10 12:53:14,402] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,403] jepsen worker 35 - jepsen.util 125 :ok :read [11 29] INFO [2022-01-10 12:53:14,404] jepsen worker 35 - jepsen.util 125 :invoke :write [11 40] INFO [2022-01-10 12:53:14,407] jepsen worker 15 - jepsen.util 15 :ok :write [5 18] INFO [2022-01-10 12:53:14,407] jepsen worker 15 - jepsen.util 15 :invoke :write [5 20] INFO [2022-01-10 12:53:14,411] jepsen worker 67 - jepsen.util 67 :ok :write [22 47] INFO [2022-01-10 12:53:14,411] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,412] jepsen worker 67 - jepsen.util 67 :ok :read [22 47] INFO [2022-01-10 12:53:14,412] jepsen worker 67 - jepsen.util 67 :invoke :write [22 40] INFO [2022-01-10 12:53:14,414] jepsen worker 49 - jepsen.util 139 :ok :write [16 3] INFO [2022-01-10 12:53:14,415] jepsen worker 49 - jepsen.util 139 :invoke :write [16 37] INFO [2022-01-10 12:53:14,418] jepsen worker 83 - jepsen.util 173 :ok :write [27 0] INFO [2022-01-10 12:53:14,419] jepsen worker 83 - jepsen.util 173 :invoke :write [27 25] INFO [2022-01-10 12:53:14,423] jepsen worker 16 - jepsen.util 16 :info :write [5 26] :RequestTimeoutException INFO [2022-01-10 12:53:14,423] jepsen worker 8 - jepsen.util 8 :ok :write [2 27] INFO [2022-01-10 12:53:14,424] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,424] jepsen worker 16 - jepsen.util 106 :invoke :write [5 29] INFO [2022-01-10 12:53:14,425] jepsen worker 8 - jepsen.util 8 :ok :read [2 27] INFO [2022-01-10 12:53:14,426] jepsen worker 8 - jepsen.util 8 :invoke :write [2 46] INFO [2022-01-10 12:53:14,427] jepsen worker 35 - jepsen.util 125 :ok :write [11 40] INFO [2022-01-10 12:53:14,429] jepsen worker 35 - jepsen.util 125 :invoke :write [11 23] INFO [2022-01-10 12:53:14,433] jepsen worker 15 - jepsen.util 15 :ok :write [5 20] INFO [2022-01-10 12:53:14,434] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,435] jepsen worker 15 - jepsen.util 15 :ok :read [5 20] INFO [2022-01-10 12:53:14,435] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,436] jepsen worker 15 - jepsen.util 15 :ok :read [5 20] INFO [2022-01-10 12:53:14,437] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,437] jepsen worker 67 - jepsen.util 67 :ok :write [22 40] INFO [2022-01-10 12:53:14,437] jepsen worker 15 - jepsen.util 15 :ok :read [5 20] INFO [2022-01-10 12:53:14,438] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,438] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,439] jepsen worker 67 - jepsen.util 67 :ok :read [22 40] INFO [2022-01-10 12:53:14,439] jepsen worker 15 - jepsen.util 15 :ok :read [5 20] INFO [2022-01-10 12:53:14,439] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,440] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,440] jepsen worker 49 - jepsen.util 139 :ok :write [16 37] INFO [2022-01-10 12:53:14,440] jepsen worker 67 - jepsen.util 67 :ok :read [22 40] INFO [2022-01-10 12:53:14,441] jepsen worker 15 - jepsen.util 15 :ok :read [5 20] INFO [2022-01-10 12:53:14,441] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,442] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,442] jepsen worker 49 - jepsen.util 139 :ok :read [16 37] INFO [2022-01-10 12:53:14,442] jepsen worker 67 - jepsen.util 67 :invoke :write [22 15] INFO [2022-01-10 12:53:14,443] jepsen worker 15 - jepsen.util 15 :ok :read [5 20] INFO [2022-01-10 12:53:14,443] jepsen worker 49 - jepsen.util 139 :invoke :write [16 42] INFO [2022-01-10 12:53:14,444] jepsen worker 15 - jepsen.util 15 :invoke :write [5 5] INFO [2022-01-10 12:53:14,444] jepsen worker 83 - jepsen.util 173 :ok :write [27 25] INFO [2022-01-10 12:53:14,445] jepsen worker 83 - jepsen.util 173 :invoke :write [27 29] INFO [2022-01-10 12:53:14,447] jepsen worker 8 - jepsen.util 8 :ok :write [2 46] INFO [2022-01-10 12:53:14,448] jepsen worker 8 - jepsen.util 8 :invoke :write [2 10] INFO [2022-01-10 12:53:14,454] jepsen worker 35 - jepsen.util 125 :ok :write [11 23] INFO [2022-01-10 12:53:14,454] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,455] jepsen worker 35 - jepsen.util 125 :ok :read [11 23] INFO [2022-01-10 12:53:14,456] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,456] jepsen worker 35 - jepsen.util 125 :ok :read [11 23] INFO [2022-01-10 12:53:14,457] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,458] jepsen worker 35 - jepsen.util 125 :ok :read [11 23] INFO [2022-01-10 12:53:14,458] jepsen worker 16 - jepsen.util 106 :ok :write [5 29] INFO [2022-01-10 12:53:14,458] jepsen worker 35 - jepsen.util 125 :invoke :write [11 8] INFO [2022-01-10 12:53:14,459] jepsen worker 16 - jepsen.util 106 :invoke :write [5 42] INFO [2022-01-10 12:53:14,463] jepsen worker 67 - jepsen.util 67 :ok :write [22 15] INFO [2022-01-10 12:53:14,463] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,464] jepsen worker 67 - jepsen.util 67 :ok :read [22 15] INFO [2022-01-10 12:53:14,465] jepsen worker 67 - jepsen.util 67 :invoke :write [22 2] INFO [2022-01-10 12:53:14,467] jepsen worker 49 - jepsen.util 139 :ok :write [16 42] INFO [2022-01-10 12:53:14,468] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,468] jepsen worker 49 - jepsen.util 139 :ok :read [16 42] INFO [2022-01-10 12:53:14,469] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,470] jepsen worker 49 - jepsen.util 139 :ok :read [16 42] INFO [2022-01-10 12:53:14,470] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,471] jepsen worker 83 - jepsen.util 173 :ok :write [27 29] INFO [2022-01-10 12:53:14,471] jepsen worker 49 - jepsen.util 139 :ok :read [16 42] INFO [2022-01-10 12:53:14,472] jepsen worker 83 - jepsen.util 173 :invoke :write [27 16] INFO [2022-01-10 12:53:14,472] jepsen worker 49 - jepsen.util 139 :invoke :write [16 9] INFO [2022-01-10 12:53:14,474] jepsen worker 8 - jepsen.util 8 :ok :write [2 10] INFO [2022-01-10 12:53:14,475] jepsen worker 8 - jepsen.util 8 :invoke :write [2 40] INFO [2022-01-10 12:53:14,481] jepsen worker 35 - jepsen.util 125 :ok :write [11 8] INFO [2022-01-10 12:53:14,481] jepsen worker 35 - jepsen.util 125 :invoke :write [11 26] INFO [2022-01-10 12:53:14,487] jepsen worker 16 - jepsen.util 106 :ok :write [5 42] INFO [2022-01-10 12:53:14,489] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,490] jepsen worker 16 - jepsen.util 106 :ok :read [5 42] INFO [2022-01-10 12:53:14,491] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,492] jepsen worker 16 - jepsen.util 106 :ok :read [5 42] INFO [2022-01-10 12:53:14,493] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,493] jepsen worker 68 - jepsen.util 68 :ok :write [22 32] INFO [2022-01-10 12:53:14,493] jepsen worker 68 - jepsen.util 68 :invoke :write [22 2] INFO [2022-01-10 12:53:14,494] jepsen worker 16 - jepsen.util 106 :ok :read [5 42] INFO [2022-01-10 12:53:14,494] jepsen worker 16 - jepsen.util 106 :invoke :write [5 6] INFO [2022-01-10 12:53:14,497] jepsen worker 49 - jepsen.util 139 :ok :write [16 9] INFO [2022-01-10 12:53:14,498] jepsen worker 49 - jepsen.util 139 :invoke :write [16 46] INFO [2022-01-10 12:53:14,500] jepsen worker 83 - jepsen.util 173 :ok :write [27 16] INFO [2022-01-10 12:53:14,501] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:14,502] jepsen worker 83 - jepsen.util 173 :ok :read [27 16] INFO [2022-01-10 12:53:14,503] jepsen worker 83 - jepsen.util 173 :invoke :write [27 26] INFO [2022-01-10 12:53:14,504] jepsen worker 8 - jepsen.util 8 :ok :write [2 40] INFO [2022-01-10 12:53:14,505] jepsen worker 8 - jepsen.util 8 :invoke :write [2 12] INFO [2022-01-10 12:53:14,509] jepsen worker 35 - jepsen.util 125 :ok :write [11 26] INFO [2022-01-10 12:53:14,511] jepsen worker 35 - jepsen.util 125 :invoke :write [11 34] INFO [2022-01-10 12:53:14,515] jepsen worker 15 - jepsen.util 15 :ok :write [5 5] INFO [2022-01-10 12:53:14,515] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,516] jepsen worker 15 - jepsen.util 15 :ok :read [5 5] INFO [2022-01-10 12:53:14,517] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,518] jepsen worker 15 - jepsen.util 15 :ok :read [5 5] INFO [2022-01-10 12:53:14,518] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,519] jepsen worker 67 - jepsen.util 67 :ok :write [22 2] INFO [2022-01-10 12:53:14,519] jepsen worker 15 - jepsen.util 15 :ok :read [5 5] INFO [2022-01-10 12:53:14,519] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,520] jepsen worker 15 - jepsen.util 15 :invoke :write [5 27] INFO [2022-01-10 12:53:14,520] jepsen worker 67 - jepsen.util 67 :ok :read [22 2] INFO [2022-01-10 12:53:14,521] jepsen worker 67 - jepsen.util 67 :invoke :write [22 15] INFO [2022-01-10 12:53:14,523] jepsen worker 49 - jepsen.util 139 :ok :write [16 46] INFO [2022-01-10 12:53:14,524] jepsen worker 49 - jepsen.util 139 :invoke :write [16 22] INFO [2022-01-10 12:53:14,527] jepsen worker 83 - jepsen.util 173 :ok :write [27 26] INFO [2022-01-10 12:53:14,528] jepsen worker 83 - jepsen.util 173 :invoke :write [27 24] INFO [2022-01-10 12:53:14,531] jepsen worker 8 - jepsen.util 8 :ok :write [2 12] INFO [2022-01-10 12:53:14,532] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,533] jepsen worker 8 - jepsen.util 8 :ok :read [2 12] INFO [2022-01-10 12:53:14,533] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,534] jepsen worker 8 - jepsen.util 8 :ok :read [2 12] INFO [2022-01-10 12:53:14,535] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,535] jepsen worker 8 - jepsen.util 8 :ok :read [2 12] INFO [2022-01-10 12:53:14,536] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,536] jepsen worker 35 - jepsen.util 125 :ok :write [11 34] INFO [2022-01-10 12:53:14,537] jepsen worker 8 - jepsen.util 8 :ok :read [2 12] INFO [2022-01-10 12:53:14,537] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,537] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,538] jepsen worker 35 - jepsen.util 125 :ok :read [11 34] INFO [2022-01-10 12:53:14,538] jepsen worker 8 - jepsen.util 8 :ok :read [2 12] INFO [2022-01-10 12:53:14,538] jepsen worker 35 - jepsen.util 125 :invoke :write [11 36] INFO [2022-01-10 12:53:14,538] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,539] jepsen worker 8 - jepsen.util 8 :ok :read [2 12] INFO [2022-01-10 12:53:14,540] jepsen worker 8 - jepsen.util 8 :invoke :write [2 38] INFO [2022-01-10 12:53:14,672] jepsen worker 77 - jepsen.util 167 :ok :write [25 23] INFO [2022-01-10 12:53:14,672] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:14,673] jepsen worker 78 - jepsen.util 78 :ok :write [26 23] INFO [2022-01-10 12:53:14,674] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:14,675] jepsen worker 78 - jepsen.util 78 :ok :read [26 23] INFO [2022-01-10 12:53:14,676] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:14,676] jepsen worker 23 - jepsen.util 23 :ok :write [7 11] INFO [2022-01-10 12:53:14,676] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:14,677] jepsen worker 78 - jepsen.util 78 :ok :read [26 23] INFO [2022-01-10 12:53:14,677] jepsen worker 23 - jepsen.util 23 :ok :read [7 11] INFO [2022-01-10 12:53:14,678] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:14,679] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:14,679] jepsen worker 23 - jepsen.util 23 :ok :read [7 11] INFO [2022-01-10 12:53:14,679] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:14,680] jepsen worker 43 - jepsen.util 43 :ok :write [14 20] INFO [2022-01-10 12:53:14,680] jepsen worker 78 - jepsen.util 78 :ok :read [26 23] INFO [2022-01-10 12:53:14,680] jepsen worker 23 - jepsen.util 23 :ok :read [7 11] INFO [2022-01-10 12:53:14,681] jepsen worker 43 - jepsen.util 43 :invoke :write [14 19] INFO [2022-01-10 12:53:14,681] jepsen worker 78 - jepsen.util 78 :invoke :write [26 45] INFO [2022-01-10 12:53:14,681] jepsen worker 52 - jepsen.util 52 :ok :write [17 28] INFO [2022-01-10 12:53:14,682] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:14,682] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:14,683] jepsen worker 23 - jepsen.util 23 :ok :read [7 11] INFO [2022-01-10 12:53:14,683] jepsen worker 23 - jepsen.util 23 :invoke :write [7 11] INFO [2022-01-10 12:53:14,684] jepsen worker 62 - jepsen.util 152 :ok :write [20 14] INFO [2022-01-10 12:53:14,684] jepsen worker 77 - jepsen.util 167 :ok :read [25 23] INFO [2022-01-10 12:53:14,684] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:14,685] jepsen worker 77 - jepsen.util 167 :invoke :write [25 46] INFO [2022-01-10 12:53:14,685] jepsen worker 62 - jepsen.util 152 :ok :read [20 14] INFO [2022-01-10 12:53:14,686] jepsen worker 62 - jepsen.util 152 :invoke :write [20 13] INFO [2022-01-10 12:53:14,688] jepsen worker 39 - jepsen.util 39 :ok :write [13 49] INFO [2022-01-10 12:53:14,688] jepsen worker 39 - jepsen.util 39 :invoke :write [13 10] INFO [2022-01-10 12:53:14,690] jepsen worker 14 - jepsen.util 104 :ok :write [4 28] INFO [2022-01-10 12:53:14,691] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:14,692] jepsen worker 89 - jepsen.util 179 :ok :write [29 33] INFO [2022-01-10 12:53:14,692] jepsen worker 14 - jepsen.util 104 :ok :read [4 28] INFO [2022-01-10 12:53:14,693] jepsen worker 14 - jepsen.util 104 :invoke :write [4 16] INFO [2022-01-10 12:53:14,694] jepsen worker 89 - jepsen.util 179 :invoke :write [29 46] INFO [2022-01-10 12:53:14,698] jepsen worker 1 - jepsen.util 1 :ok :write [0 31] INFO [2022-01-10 12:53:14,699] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:14,700] jepsen worker 1 - jepsen.util 1 :ok :read [0 31] INFO [2022-01-10 12:53:14,700] jepsen worker 1 - jepsen.util 1 :invoke :write [0 41] INFO [2022-01-10 12:53:14,701] jepsen worker 4 - jepsen.util 4 :ok :write [1 37] INFO [2022-01-10 12:53:14,702] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:14,702] jepsen worker 52 - jepsen.util 52 :ok :read [17 28] INFO [2022-01-10 12:53:14,702] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:14,703] jepsen worker 4 - jepsen.util 4 :ok :read [1 37] INFO [2022-01-10 12:53:14,703] jepsen worker 4 - jepsen.util 4 :invoke :write [1 23] INFO [2022-01-10 12:53:14,703] jepsen worker 52 - jepsen.util 52 :ok :read [17 28] INFO [2022-01-10 12:53:14,704] jepsen worker 52 - jepsen.util 52 :invoke :write [17 11] INFO [2022-01-10 12:53:14,714] jepsen worker 38 - jepsen.util 38 :ok :write [12 30] INFO [2022-01-10 12:53:14,714] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:14,715] jepsen worker 38 - jepsen.util 38 :ok :read [12 30] INFO [2022-01-10 12:53:14,716] jepsen worker 38 - jepsen.util 38 :invoke :write [12 6] INFO [2022-01-10 12:53:14,717] jepsen worker 85 - jepsen.util 175 :ok :write [28 6] INFO [2022-01-10 12:53:14,718] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:14,719] jepsen worker 85 - jepsen.util 175 :ok :read [28 6] INFO [2022-01-10 12:53:14,720] jepsen worker 16 - jepsen.util 106 :ok :write [5 6] INFO [2022-01-10 12:53:14,720] jepsen worker 85 - jepsen.util 175 :invoke :write [28 38] INFO [2022-01-10 12:53:14,720] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,720] jepsen worker 71 - jepsen.util 71 :ok :write [23 37] INFO [2022-01-10 12:53:14,721] jepsen worker 16 - jepsen.util 106 :ok :read [5 6] INFO [2022-01-10 12:53:14,721] jepsen worker 71 - jepsen.util 71 :invoke :write [23 28] INFO [2022-01-10 12:53:14,722] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,723] jepsen worker 16 - jepsen.util 106 :ok :read [5 6] INFO [2022-01-10 12:53:14,724] jepsen worker 16 - jepsen.util 106 :invoke :write [5 34] INFO [2022-01-10 12:53:14,727] jepsen worker 67 - jepsen.util 67 :ok :write [22 15] INFO [2022-01-10 12:53:14,728] jepsen worker 67 - jepsen.util 67 :invoke :write [22 21] INFO [2022-01-10 12:53:14,737] jepsen worker 49 - jepsen.util 139 :ok :write [16 22] INFO [2022-01-10 12:53:14,738] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,739] jepsen worker 49 - jepsen.util 139 :ok :read [16 22] INFO [2022-01-10 12:53:14,739] jepsen worker 49 - jepsen.util 139 :invoke :write [16 43] INFO [2022-01-10 12:53:14,742] jepsen worker 83 - jepsen.util 173 :ok :write [27 24] INFO [2022-01-10 12:53:14,743] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:14,744] jepsen worker 83 - jepsen.util 173 :ok :read [27 24] INFO [2022-01-10 12:53:14,744] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:14,745] jepsen worker 83 - jepsen.util 173 :ok :read [27 24] INFO [2022-01-10 12:53:14,746] jepsen worker 4 - jepsen.util 4 :ok :write [1 23] INFO [2022-01-10 12:53:14,746] jepsen worker 83 - jepsen.util 173 :invoke :write [27 37] INFO [2022-01-10 12:53:14,747] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:14,747] jepsen worker 8 - jepsen.util 8 :ok :write [2 38] INFO [2022-01-10 12:53:14,748] jepsen worker 4 - jepsen.util 4 :ok :read [1 23] INFO [2022-01-10 12:53:14,748] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,749] jepsen worker 4 - jepsen.util 4 :invoke :write [1 14] INFO [2022-01-10 12:53:14,750] jepsen worker 8 - jepsen.util 8 :ok :read [2 38] INFO [2022-01-10 12:53:14,750] jepsen worker 52 - jepsen.util 52 :ok :write [17 11] INFO [2022-01-10 12:53:14,750] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,751] jepsen worker 8 - jepsen.util 8 :ok :read [2 38] INFO [2022-01-10 12:53:14,752] jepsen worker 52 - jepsen.util 52 :invoke :write [17 27] INFO [2022-01-10 12:53:14,752] jepsen worker 38 - jepsen.util 38 :ok :write [12 6] INFO [2022-01-10 12:53:14,752] jepsen worker 8 - jepsen.util 8 :invoke :write [2 18] INFO [2022-01-10 12:53:14,753] jepsen worker 35 - jepsen.util 125 :ok :write [11 36] INFO [2022-01-10 12:53:14,753] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:14,754] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,754] jepsen worker 38 - jepsen.util 38 :ok :read [12 6] INFO [2022-01-10 12:53:14,755] jepsen worker 35 - jepsen.util 125 :ok :read [11 36] INFO [2022-01-10 12:53:14,755] jepsen worker 38 - jepsen.util 38 :invoke :write [12 12] INFO [2022-01-10 12:53:14,755] jepsen worker 78 - jepsen.util 78 :ok :write [26 45] INFO [2022-01-10 12:53:14,755] jepsen worker 35 - jepsen.util 125 :invoke :write [11 34] INFO [2022-01-10 12:53:14,756] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:14,756] jepsen worker 77 - jepsen.util 167 :ok :write [25 46] INFO [2022-01-10 12:53:14,757] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:14,757] jepsen worker 78 - jepsen.util 78 :ok :read [26 45] INFO [2022-01-10 12:53:14,758] jepsen worker 77 - jepsen.util 167 :ok :read [25 46] INFO [2022-01-10 12:53:14,758] jepsen worker 78 - jepsen.util 78 :invoke :write [26 18] INFO [2022-01-10 12:53:14,758] jepsen worker 77 - jepsen.util 167 :invoke :write [25 13] INFO [2022-01-10 12:53:14,760] jepsen worker 85 - jepsen.util 175 :ok :write [28 38] INFO [2022-01-10 12:53:14,760] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:14,761] jepsen worker 85 - jepsen.util 175 :ok :read [28 38] INFO [2022-01-10 12:53:14,761] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:14,763] jepsen worker 85 - jepsen.util 175 :ok :read [28 38] INFO [2022-01-10 12:53:14,763] jepsen worker 85 - jepsen.util 175 :invoke :write [28 42] INFO [2022-01-10 12:53:14,771] jepsen worker 39 - jepsen.util 39 :ok :write [13 10] INFO [2022-01-10 12:53:14,772] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:14,773] jepsen worker 39 - jepsen.util 39 :ok :read [13 10] INFO [2022-01-10 12:53:14,773] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:14,774] jepsen worker 39 - jepsen.util 39 :ok :read [13 10] INFO [2022-01-10 12:53:14,775] jepsen worker 39 - jepsen.util 39 :invoke :write [13 35] INFO [2022-01-10 12:53:14,783] jepsen worker 4 - jepsen.util 4 :ok :write [1 14] INFO [2022-01-10 12:53:14,784] jepsen worker 4 - jepsen.util 4 :invoke :write [1 20] INFO [2022-01-10 12:53:14,787] jepsen worker 14 - jepsen.util 104 :ok :write [4 16] INFO [2022-01-10 12:53:14,787] jepsen worker 23 - jepsen.util 23 :ok :write [7 11] INFO [2022-01-10 12:53:14,787] jepsen worker 23 - jepsen.util 23 :invoke :write [7 17] INFO [2022-01-10 12:53:14,788] jepsen worker 14 - jepsen.util 104 :invoke :write [4 42] INFO [2022-01-10 12:53:14,792] jepsen worker 70 - jepsen.util 160 :ok :write [23 11] INFO [2022-01-10 12:53:14,792] jepsen worker 43 - jepsen.util 43 :ok :write [14 19] INFO [2022-01-10 12:53:14,793] jepsen worker 70 - jepsen.util 160 :invoke :write [23 32] INFO [2022-01-10 12:53:14,793] jepsen worker 43 - jepsen.util 43 :invoke :write [14 22] INFO [2022-01-10 12:53:14,799] jepsen worker 62 - jepsen.util 152 :ok :write [20 13] INFO [2022-01-10 12:53:14,799] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:14,800] jepsen worker 62 - jepsen.util 152 :ok :read [20 13] INFO [2022-01-10 12:53:14,801] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:14,802] jepsen worker 62 - jepsen.util 152 :ok :read [20 13] INFO [2022-01-10 12:53:14,802] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:14,803] jepsen worker 62 - jepsen.util 152 :ok :read [20 13] INFO [2022-01-10 12:53:14,804] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:14,804] jepsen worker 62 - jepsen.util 152 :ok :read [20 13] INFO [2022-01-10 12:53:14,805] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:14,806] jepsen worker 62 - jepsen.util 152 :ok :read [20 13] INFO [2022-01-10 12:53:14,806] jepsen worker 16 - jepsen.util 106 :ok :write [5 34] INFO [2022-01-10 12:53:14,806] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:14,807] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,807] jepsen worker 62 - jepsen.util 152 :ok :read [20 13] INFO [2022-01-10 12:53:14,807] jepsen worker 16 - jepsen.util 106 :ok :read [5 34] INFO [2022-01-10 12:53:14,808] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:14,808] jepsen worker 16 - jepsen.util 106 :invoke :write [5 2] INFO [2022-01-10 12:53:14,809] jepsen worker 62 - jepsen.util 152 :ok :read [20 13] INFO [2022-01-10 12:53:14,809] jepsen worker 62 - jepsen.util 152 :invoke :write [20 7] INFO [2022-01-10 12:53:14,812] jepsen worker 67 - jepsen.util 67 :ok :write [22 21] INFO [2022-01-10 12:53:14,812] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,813] jepsen worker 67 - jepsen.util 67 :ok :read [22 21] INFO [2022-01-10 12:53:14,814] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,815] jepsen worker 67 - jepsen.util 67 :ok :read [22 21] INFO [2022-01-10 12:53:14,816] jepsen worker 67 - jepsen.util 67 :invoke :write [22 46] INFO [2022-01-10 12:53:14,818] jepsen worker 49 - jepsen.util 139 :ok :write [16 43] INFO [2022-01-10 12:53:14,818] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,819] jepsen worker 49 - jepsen.util 139 :ok :read [16 43] INFO [2022-01-10 12:53:14,820] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,821] jepsen worker 49 - jepsen.util 139 :ok :read [16 43] INFO [2022-01-10 12:53:14,821] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,822] jepsen worker 49 - jepsen.util 139 :ok :read [16 43] INFO [2022-01-10 12:53:14,823] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,823] jepsen worker 83 - jepsen.util 173 :ok :write [27 37] INFO [2022-01-10 12:53:14,823] jepsen worker 49 - jepsen.util 139 :ok :read [16 43] INFO [2022-01-10 12:53:14,824] jepsen worker 83 - jepsen.util 173 :invoke :write [27 5] INFO [2022-01-10 12:53:14,824] jepsen worker 49 - jepsen.util 139 :invoke :write [16 33] INFO [2022-01-10 12:53:14,827] jepsen worker 8 - jepsen.util 8 :ok :write [2 18] INFO [2022-01-10 12:53:14,828] jepsen worker 8 - jepsen.util 8 :invoke :write [2 12] INFO [2022-01-10 12:53:14,831] jepsen worker 35 - jepsen.util 125 :ok :write [11 34] INFO [2022-01-10 12:53:14,831] jepsen worker 35 - jepsen.util 125 :invoke :write [11 1] INFO [2022-01-10 12:53:14,850] jepsen worker 16 - jepsen.util 106 :ok :write [5 2] INFO [2022-01-10 12:53:14,851] jepsen worker 16 - jepsen.util 106 :invoke :write [5 21] INFO [2022-01-10 12:53:14,854] jepsen worker 67 - jepsen.util 67 :ok :write [22 46] INFO [2022-01-10 12:53:14,855] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,856] jepsen worker 67 - jepsen.util 67 :ok :read [22 46] INFO [2022-01-10 12:53:14,857] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,858] jepsen worker 67 - jepsen.util 67 :ok :read [22 46] INFO [2022-01-10 12:53:14,858] jepsen worker 67 - jepsen.util 67 :invoke :write [22 2] INFO [2022-01-10 12:53:14,859] jepsen worker 49 - jepsen.util 139 :ok :write [16 33] INFO [2022-01-10 12:53:14,859] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,860] jepsen worker 49 - jepsen.util 139 :ok :read [16 33] INFO [2022-01-10 12:53:14,861] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,862] jepsen worker 49 - jepsen.util 139 :ok :read [16 33] INFO [2022-01-10 12:53:14,862] jepsen worker 49 - jepsen.util 139 :invoke :write [16 45] INFO [2022-01-10 12:53:14,864] jepsen worker 83 - jepsen.util 173 :ok :write [27 5] INFO [2022-01-10 12:53:14,864] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:14,865] jepsen worker 83 - jepsen.util 173 :ok :read [27 5] INFO [2022-01-10 12:53:14,866] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:14,867] jepsen worker 83 - jepsen.util 173 :ok :read [27 5] INFO [2022-01-10 12:53:14,867] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:14,868] jepsen worker 8 - jepsen.util 8 :ok :write [2 12] INFO [2022-01-10 12:53:14,868] jepsen worker 83 - jepsen.util 173 :ok :read [27 5] INFO [2022-01-10 12:53:14,868] jepsen worker 8 - jepsen.util 8 :invoke :write [2 19] INFO [2022-01-10 12:53:14,869] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:14,870] jepsen worker 83 - jepsen.util 173 :ok :read [27 5] INFO [2022-01-10 12:53:14,871] jepsen worker 83 - jepsen.util 173 :invoke :write [27 24] INFO [2022-01-10 12:53:14,872] jepsen worker 35 - jepsen.util 125 :ok :write [11 1] INFO [2022-01-10 12:53:14,872] jepsen worker 35 - jepsen.util 125 :invoke :write [11 42] INFO [2022-01-10 12:53:14,878] jepsen worker 16 - jepsen.util 106 :ok :write [5 21] INFO [2022-01-10 12:53:14,878] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,879] jepsen worker 16 - jepsen.util 106 :ok :read [5 21] INFO [2022-01-10 12:53:14,880] jepsen worker 16 - jepsen.util 106 :invoke :write [5 18] INFO [2022-01-10 12:53:14,882] jepsen worker 67 - jepsen.util 67 :ok :write [22 2] INFO [2022-01-10 12:53:14,882] jepsen worker 67 - jepsen.util 67 :invoke :write [22 46] INFO [2022-01-10 12:53:14,887] jepsen worker 49 - jepsen.util 139 :ok :write [16 45] INFO [2022-01-10 12:53:14,887] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,888] jepsen worker 49 - jepsen.util 139 :ok :read [16 45] INFO [2022-01-10 12:53:14,889] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,890] jepsen worker 49 - jepsen.util 139 :ok :read [16 45] INFO [2022-01-10 12:53:14,890] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,890] jepsen worker 83 - jepsen.util 173 :ok :write [27 24] INFO [2022-01-10 12:53:14,891] jepsen worker 49 - jepsen.util 139 :ok :read [16 45] INFO [2022-01-10 12:53:14,891] jepsen worker 83 - jepsen.util 173 :invoke :write [27 27] INFO [2022-01-10 12:53:14,891] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,892] jepsen worker 49 - jepsen.util 139 :ok :read [16 45] INFO [2022-01-10 12:53:14,893] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,894] jepsen worker 49 - jepsen.util 139 :ok :read [16 45] INFO [2022-01-10 12:53:14,894] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,895] jepsen worker 49 - jepsen.util 139 :ok :read [16 45] INFO [2022-01-10 12:53:14,896] jepsen worker 8 - jepsen.util 8 :ok :write [2 19] INFO [2022-01-10 12:53:14,896] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,897] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,897] jepsen worker 8 - jepsen.util 8 :ok :read [2 19] INFO [2022-01-10 12:53:14,897] jepsen worker 49 - jepsen.util 139 :ok :read [16 45] INFO [2022-01-10 12:53:14,898] jepsen worker 8 - jepsen.util 8 :invoke :write [2 10] INFO [2022-01-10 12:53:14,898] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,899] jepsen worker 49 - jepsen.util 139 :ok :read [16 45] INFO [2022-01-10 12:53:14,899] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,900] jepsen worker 49 - jepsen.util 139 :ok :read [16 45] INFO [2022-01-10 12:53:14,901] jepsen worker 49 - jepsen.util 139 :invoke :write [16 38] INFO [2022-01-10 12:53:14,901] jepsen worker 35 - jepsen.util 125 :ok :write [11 42] INFO [2022-01-10 12:53:14,901] jepsen worker 35 - jepsen.util 125 :invoke :write [11 40] INFO [2022-01-10 12:53:14,907] jepsen worker 16 - jepsen.util 106 :ok :write [5 18] INFO [2022-01-10 12:53:14,907] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,908] jepsen worker 16 - jepsen.util 106 :ok :read [5 18] INFO [2022-01-10 12:53:14,909] jepsen worker 16 - jepsen.util 106 :invoke :write [5 25] INFO [2022-01-10 12:53:14,910] jepsen worker 87 - jepsen.util 87 :ok :write [29 6] INFO [2022-01-10 12:53:14,911] jepsen worker 52 - jepsen.util 52 :ok :write [17 27] INFO [2022-01-10 12:53:14,911] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:14,912] jepsen worker 67 - jepsen.util 67 :ok :write [22 46] INFO [2022-01-10 12:53:14,912] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:14,912] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:14,912] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,913] jepsen worker 52 - jepsen.util 52 :ok :read [17 27] INFO [2022-01-10 12:53:14,913] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:14,913] jepsen worker 67 - jepsen.util 67 :ok :read [22 46] INFO [2022-01-10 12:53:14,913] jepsen worker 52 - jepsen.util 52 :invoke :write [17 30] INFO [2022-01-10 12:53:14,914] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:14,914] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,914] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:14,915] jepsen worker 67 - jepsen.util 67 :ok :read [22 46] INFO [2022-01-10 12:53:14,915] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:14,915] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,916] jepsen worker 87 - jepsen.util 87 :invoke :write [29 43] INFO [2022-01-10 12:53:14,917] jepsen worker 67 - jepsen.util 67 :ok :read [22 46] INFO [2022-01-10 12:53:14,917] jepsen worker 67 - jepsen.util 67 :invoke :write [22 15] INFO [2022-01-10 12:53:14,920] jepsen worker 49 - jepsen.util 139 :ok :write [16 38] INFO [2022-01-10 12:53:14,921] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,922] jepsen worker 49 - jepsen.util 139 :ok :read [16 38] INFO [2022-01-10 12:53:14,922] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:14,923] jepsen worker 83 - jepsen.util 173 :ok :write [27 27] INFO [2022-01-10 12:53:14,923] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:14,923] jepsen worker 49 - jepsen.util 139 :ok :read [16 38] INFO [2022-01-10 12:53:14,924] jepsen worker 83 - jepsen.util 173 :ok :read [27 27] INFO [2022-01-10 12:53:14,924] jepsen worker 49 - jepsen.util 139 :invoke :write [16 21] INFO [2022-01-10 12:53:14,925] jepsen worker 83 - jepsen.util 173 :invoke :write [27 4] INFO [2022-01-10 12:53:14,927] jepsen worker 8 - jepsen.util 8 :ok :write [2 10] INFO [2022-01-10 12:53:14,927] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,928] jepsen worker 8 - jepsen.util 8 :ok :read [2 10] INFO [2022-01-10 12:53:14,929] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,930] jepsen worker 8 - jepsen.util 8 :ok :read [2 10] INFO [2022-01-10 12:53:14,930] jepsen worker 8 - jepsen.util 8 :invoke :write [2 46] INFO [2022-01-10 12:53:14,931] jepsen worker 35 - jepsen.util 125 :ok :write [11 40] INFO [2022-01-10 12:53:14,932] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,933] jepsen worker 35 - jepsen.util 125 :ok :read [11 40] INFO [2022-01-10 12:53:14,933] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,934] jepsen worker 35 - jepsen.util 125 :ok :read [11 40] INFO [2022-01-10 12:53:14,934] jepsen worker 35 - jepsen.util 125 :invoke :write [11 5] INFO [2022-01-10 12:53:14,936] jepsen worker 16 - jepsen.util 106 :ok :write [5 25] INFO [2022-01-10 12:53:14,936] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,937] jepsen worker 16 - jepsen.util 106 :ok :read [5 25] INFO [2022-01-10 12:53:14,938] jepsen worker 16 - jepsen.util 106 :invoke :write [5 24] INFO [2022-01-10 12:53:14,941] jepsen worker 67 - jepsen.util 67 :ok :write [22 15] INFO [2022-01-10 12:53:14,941] jepsen worker 67 - jepsen.util 67 :invoke :write [22 20] INFO [2022-01-10 12:53:14,944] jepsen worker 49 - jepsen.util 139 :ok :write [16 21] INFO [2022-01-10 12:53:14,945] jepsen worker 49 - jepsen.util 139 :invoke :write [16 30] INFO [2022-01-10 12:53:14,945] jepsen worker 1 - jepsen.util 1 :ok :write [0 41] INFO [2022-01-10 12:53:14,946] jepsen worker 38 - jepsen.util 38 :ok :write [12 12] INFO [2022-01-10 12:53:14,946] jepsen worker 1 - jepsen.util 1 :invoke :write [0 20] INFO [2022-01-10 12:53:14,946] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:14,947] jepsen worker 38 - jepsen.util 38 :ok :read [12 12] INFO [2022-01-10 12:53:14,948] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:14,949] jepsen worker 38 - jepsen.util 38 :ok :read [12 12] INFO [2022-01-10 12:53:14,949] jepsen worker 38 - jepsen.util 38 :invoke :write [12 44] INFO [2022-01-10 12:53:14,950] jepsen worker 83 - jepsen.util 173 :ok :write [27 4] INFO [2022-01-10 12:53:14,950] jepsen worker 83 - jepsen.util 173 :invoke :write [27 32] INFO [2022-01-10 12:53:14,957] jepsen worker 8 - jepsen.util 8 :ok :write [2 46] INFO [2022-01-10 12:53:14,958] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,959] jepsen worker 8 - jepsen.util 8 :ok :read [2 46] INFO [2022-01-10 12:53:14,959] jepsen worker 35 - jepsen.util 125 :ok :write [11 5] INFO [2022-01-10 12:53:14,959] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,960] jepsen worker 35 - jepsen.util 125 :invoke :write [11 7] INFO [2022-01-10 12:53:14,960] jepsen worker 8 - jepsen.util 8 :ok :read [2 46] INFO [2022-01-10 12:53:14,961] jepsen worker 8 - jepsen.util 8 :invoke :write [2 35] INFO [2022-01-10 12:53:14,964] jepsen worker 16 - jepsen.util 106 :ok :write [5 24] INFO [2022-01-10 12:53:14,965] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,966] jepsen worker 16 - jepsen.util 106 :ok :read [5 24] INFO [2022-01-10 12:53:14,966] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,967] jepsen worker 16 - jepsen.util 106 :ok :read [5 24] INFO [2022-01-10 12:53:14,968] jepsen worker 67 - jepsen.util 67 :ok :write [22 20] INFO [2022-01-10 12:53:14,968] jepsen worker 16 - jepsen.util 106 :invoke :write [5 34] INFO [2022-01-10 12:53:14,968] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,969] jepsen worker 67 - jepsen.util 67 :ok :read [22 20] INFO [2022-01-10 12:53:14,970] jepsen worker 67 - jepsen.util 67 :invoke :write [22 19] INFO [2022-01-10 12:53:14,972] jepsen worker 49 - jepsen.util 139 :ok :write [16 30] INFO [2022-01-10 12:53:14,973] jepsen worker 49 - jepsen.util 139 :invoke :write [16 0] INFO [2022-01-10 12:53:14,977] jepsen worker 83 - jepsen.util 173 :ok :write [27 32] INFO [2022-01-10 12:53:14,978] jepsen worker 83 - jepsen.util 173 :invoke :write [27 27] INFO [2022-01-10 12:53:14,983] jepsen worker 8 - jepsen.util 8 :ok :write [2 35] INFO [2022-01-10 12:53:14,983] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:14,984] jepsen worker 8 - jepsen.util 8 :ok :read [2 35] INFO [2022-01-10 12:53:14,985] jepsen worker 8 - jepsen.util 8 :invoke :write [2 4] INFO [2022-01-10 12:53:14,985] jepsen worker 35 - jepsen.util 125 :ok :write [11 7] INFO [2022-01-10 12:53:14,986] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:14,987] jepsen worker 35 - jepsen.util 125 :ok :read [11 7] INFO [2022-01-10 12:53:14,988] jepsen worker 35 - jepsen.util 125 :invoke :write [11 41] INFO [2022-01-10 12:53:14,990] jepsen worker 16 - jepsen.util 106 :ok :write [5 34] INFO [2022-01-10 12:53:14,990] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,991] jepsen worker 16 - jepsen.util 106 :ok :read [5 34] INFO [2022-01-10 12:53:14,992] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:14,993] jepsen worker 77 - jepsen.util 167 :ok :write [25 13] INFO [2022-01-10 12:53:14,993] jepsen worker 16 - jepsen.util 106 :ok :read [5 34] INFO [2022-01-10 12:53:14,993] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:14,994] jepsen worker 16 - jepsen.util 106 :invoke :write [5 25] INFO [2022-01-10 12:53:14,994] jepsen worker 78 - jepsen.util 78 :ok :write [26 18] INFO [2022-01-10 12:53:14,995] jepsen worker 77 - jepsen.util 167 :ok :read [25 13] INFO [2022-01-10 12:53:14,995] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:14,996] jepsen worker 77 - jepsen.util 167 :invoke :write [25 10] INFO [2022-01-10 12:53:14,996] jepsen worker 78 - jepsen.util 78 :ok :read [26 18] INFO [2022-01-10 12:53:14,996] jepsen worker 67 - jepsen.util 67 :ok :write [22 19] INFO [2022-01-10 12:53:14,996] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:14,997] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,998] jepsen worker 78 - jepsen.util 78 :ok :read [26 18] INFO [2022-01-10 12:53:14,998] jepsen worker 67 - jepsen.util 67 :ok :read [22 19] INFO [2022-01-10 12:53:14,998] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:14,999] jepsen worker 85 - jepsen.util 175 :ok :write [28 42] INFO [2022-01-10 12:53:14,999] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:14,999] jepsen worker 39 - jepsen.util 39 :ok :write [13 35] INFO [2022-01-10 12:53:14,999] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:14,999] jepsen worker 78 - jepsen.util 78 :ok :read [26 18] INFO [2022-01-10 12:53:15,000] jepsen worker 39 - jepsen.util 39 :invoke :write [13 45] INFO [2022-01-10 12:53:15,000] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:15,000] jepsen worker 67 - jepsen.util 67 :ok :read [22 19] INFO [2022-01-10 12:53:15,000] jepsen worker 85 - jepsen.util 175 :ok :read [28 42] INFO [2022-01-10 12:53:15,001] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,001] jepsen worker 85 - jepsen.util 175 :invoke :write [28 38] INFO [2022-01-10 12:53:15,001] jepsen worker 78 - jepsen.util 78 :ok :read [26 18] INFO [2022-01-10 12:53:15,002] jepsen worker 67 - jepsen.util 67 :ok :read [22 19] INFO [2022-01-10 12:53:15,002] jepsen worker 78 - jepsen.util 78 :invoke :write [26 15] INFO [2022-01-10 12:53:15,002] jepsen worker 67 - jepsen.util 67 :invoke :write [22 13] INFO [2022-01-10 12:53:15,003] jepsen worker 4 - jepsen.util 4 :ok :write [1 20] INFO [2022-01-10 12:53:15,003] jepsen worker 4 - jepsen.util 4 :invoke :write [1 6] INFO [2022-01-10 12:53:15,004] jepsen worker 49 - jepsen.util 139 :ok :write [16 0] INFO [2022-01-10 12:53:15,005] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:15,005] jepsen worker 49 - jepsen.util 139 :ok :read [16 0] INFO [2022-01-10 12:53:15,006] jepsen worker 23 - jepsen.util 23 :ok :write [7 17] INFO [2022-01-10 12:53:15,006] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:15,006] jepsen worker 23 - jepsen.util 23 :invoke :write [7 25] INFO [2022-01-10 12:53:15,007] jepsen worker 49 - jepsen.util 139 :ok :read [16 0] INFO [2022-01-10 12:53:15,007] jepsen worker 49 - jepsen.util 139 :invoke :write [16 14] INFO [2022-01-10 12:53:15,009] jepsen worker 14 - jepsen.util 104 :ok :write [4 42] INFO [2022-01-10 12:53:15,010] jepsen worker 14 - jepsen.util 104 :invoke :write [4 36] INFO [2022-01-10 12:53:15,012] jepsen worker 43 - jepsen.util 43 :ok :write [14 22] INFO [2022-01-10 12:53:15,012] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:15,013] jepsen worker 43 - jepsen.util 43 :ok :read [14 22] INFO [2022-01-10 12:53:15,013] jepsen worker 43 - jepsen.util 43 :invoke :write [14 35] INFO [2022-01-10 12:53:15,036] jepsen worker 70 - jepsen.util 160 :ok :write [23 32] INFO [2022-01-10 12:53:15,037] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,037] jepsen worker 62 - jepsen.util 152 :ok :write [20 7] INFO [2022-01-10 12:53:15,038] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,038] jepsen worker 70 - jepsen.util 160 :ok :read [23 32] INFO [2022-01-10 12:53:15,039] jepsen worker 62 - jepsen.util 152 :ok :read [20 7] INFO [2022-01-10 12:53:15,039] jepsen worker 70 - jepsen.util 160 :invoke :write [23 32] INFO [2022-01-10 12:53:15,039] jepsen worker 62 - jepsen.util 152 :invoke :write [20 21] INFO [2022-01-10 12:53:15,042] jepsen worker 52 - jepsen.util 52 :ok :write [17 30] INFO [2022-01-10 12:53:15,043] jepsen worker 52 - jepsen.util 52 :invoke :write [17 23] INFO [2022-01-10 12:53:15,044] jepsen worker 87 - jepsen.util 87 :ok :write [29 43] INFO [2022-01-10 12:53:15,044] jepsen worker 87 - jepsen.util 87 :invoke :write [29 11] INFO [2022-01-10 12:53:15,048] jepsen worker 38 - jepsen.util 38 :ok :write [12 44] INFO [2022-01-10 12:53:15,048] jepsen worker 38 - jepsen.util 38 :invoke :write [12 42] INFO [2022-01-10 12:53:15,050] jepsen worker 1 - jepsen.util 1 :ok :write [0 20] INFO [2022-01-10 12:53:15,050] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:15,051] jepsen worker 1 - jepsen.util 1 :ok :read [0 20] INFO [2022-01-10 12:53:15,052] jepsen worker 1 - jepsen.util 1 :invoke :write [0 27] INFO [2022-01-10 12:53:15,052] jepsen worker 78 - jepsen.util 78 :ok :write [26 15] INFO [2022-01-10 12:53:15,053] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:15,053] jepsen worker 77 - jepsen.util 167 :ok :write [25 10] INFO [2022-01-10 12:53:15,054] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:15,054] jepsen worker 78 - jepsen.util 78 :ok :read [26 15] INFO [2022-01-10 12:53:15,055] jepsen worker 77 - jepsen.util 167 :ok :read [25 10] INFO [2022-01-10 12:53:15,055] jepsen worker 78 - jepsen.util 78 :invoke :write [26 42] INFO [2022-01-10 12:53:15,056] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:15,056] jepsen worker 77 - jepsen.util 167 :ok :read [25 10] INFO [2022-01-10 12:53:15,057] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:15,058] jepsen worker 77 - jepsen.util 167 :ok :read [25 10] INFO [2022-01-10 12:53:15,058] jepsen worker 77 - jepsen.util 167 :invoke :write [25 20] INFO [2022-01-10 12:53:15,065] jepsen worker 23 - jepsen.util 23 :ok :write [7 25] INFO [2022-01-10 12:53:15,066] jepsen worker 23 - jepsen.util 23 :invoke :write [7 37] INFO [2022-01-10 12:53:15,066] jepsen worker 14 - jepsen.util 104 :ok :write [4 36] INFO [2022-01-10 12:53:15,067] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,068] jepsen worker 43 - jepsen.util 43 :ok :write [14 35] INFO [2022-01-10 12:53:15,068] jepsen worker 43 - jepsen.util 43 :invoke :write [14 25] INFO [2022-01-10 12:53:15,069] jepsen worker 14 - jepsen.util 104 :ok :read [4 36] INFO [2022-01-10 12:53:15,069] jepsen worker 14 - jepsen.util 104 :invoke :write [4 1] INFO [2022-01-10 12:53:15,071] jepsen worker 62 - jepsen.util 152 :ok :write [20 21] INFO [2022-01-10 12:53:15,071] jepsen worker 62 - jepsen.util 152 :invoke :write [20 19] INFO [2022-01-10 12:53:15,072] jepsen worker 70 - jepsen.util 160 :ok :write [23 32] INFO [2022-01-10 12:53:15,072] jepsen worker 70 - jepsen.util 160 :invoke :write [23 3] INFO [2022-01-10 12:53:15,169] jepsen worker 83 - jepsen.util 173 :ok :write [27 27] INFO [2022-01-10 12:53:15,169] jepsen worker 85 - jepsen.util 175 :ok :write [28 38] INFO [2022-01-10 12:53:15,170] jepsen worker 85 - jepsen.util 175 :invoke :write [28 1] INFO [2022-01-10 12:53:15,170] jepsen worker 83 - jepsen.util 173 :invoke :write [27 14] INFO [2022-01-10 12:53:15,177] jepsen worker 4 - jepsen.util 4 :ok :write [1 6] INFO [2022-01-10 12:53:15,177] jepsen worker 8 - jepsen.util 8 :ok :write [2 4] INFO [2022-01-10 12:53:15,177] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,178] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:15,179] jepsen worker 4 - jepsen.util 4 :ok :read [1 6] INFO [2022-01-10 12:53:15,180] jepsen worker 4 - jepsen.util 4 :invoke :write [1 40] INFO [2022-01-10 12:53:15,181] jepsen worker 8 - jepsen.util 8 :ok :read [2 4] INFO [2022-01-10 12:53:15,182] jepsen worker 8 - jepsen.util 8 :invoke :write [2 47] INFO [2022-01-10 12:53:15,183] jepsen worker 38 - jepsen.util 38 :ok :write [12 42] INFO [2022-01-10 12:53:15,183] jepsen worker 35 - jepsen.util 125 :ok :write [11 41] INFO [2022-01-10 12:53:15,183] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:15,184] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:15,184] jepsen worker 38 - jepsen.util 38 :ok :read [12 42] INFO [2022-01-10 12:53:15,184] jepsen worker 35 - jepsen.util 125 :ok :read [11 41] INFO [2022-01-10 12:53:15,185] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:15,185] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:15,185] jepsen worker 38 - jepsen.util 38 :ok :read [12 42] INFO [2022-01-10 12:53:15,186] jepsen worker 35 - jepsen.util 125 :ok :read [11 41] INFO [2022-01-10 12:53:15,186] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:15,186] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:15,187] jepsen worker 38 - jepsen.util 38 :ok :read [12 42] INFO [2022-01-10 12:53:15,187] jepsen worker 35 - jepsen.util 125 :ok :read [11 41] INFO [2022-01-10 12:53:15,187] jepsen worker 38 - jepsen.util 38 :invoke :write [12 8] INFO [2022-01-10 12:53:15,188] jepsen worker 16 - jepsen.util 106 :ok :write [5 25] INFO [2022-01-10 12:53:15,188] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:15,189] jepsen worker 16 - jepsen.util 106 :invoke :write [5 5] INFO [2022-01-10 12:53:15,189] jepsen worker 35 - jepsen.util 125 :ok :read [11 41] INFO [2022-01-10 12:53:15,190] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:15,190] jepsen worker 35 - jepsen.util 125 :ok :read [11 41] INFO [2022-01-10 12:53:15,191] jepsen worker 35 - jepsen.util 125 :invoke :write [11 40] INFO [2022-01-10 12:53:15,194] jepsen worker 67 - jepsen.util 67 :ok :write [22 13] INFO [2022-01-10 12:53:15,194] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,195] jepsen worker 67 - jepsen.util 67 :ok :read [22 13] INFO [2022-01-10 12:53:15,196] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,197] jepsen worker 67 - jepsen.util 67 :ok :read [22 13] INFO [2022-01-10 12:53:15,197] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,198] jepsen worker 67 - jepsen.util 67 :ok :read [22 13] INFO [2022-01-10 12:53:15,199] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,200] jepsen worker 67 - jepsen.util 67 :ok :read [22 13] INFO [2022-01-10 12:53:15,200] jepsen worker 67 - jepsen.util 67 :invoke :write [22 1] INFO [2022-01-10 12:53:15,201] jepsen worker 39 - jepsen.util 39 :ok :write [13 45] INFO [2022-01-10 12:53:15,201] jepsen worker 39 - jepsen.util 39 :invoke :write [13 41] INFO [2022-01-10 12:53:15,206] jepsen worker 49 - jepsen.util 139 :ok :write [16 14] INFO [2022-01-10 12:53:15,207] jepsen worker 49 - jepsen.util 139 :invoke :write [16 0] INFO [2022-01-10 12:53:15,212] jepsen worker 87 - jepsen.util 87 :ok :write [29 11] INFO [2022-01-10 12:53:15,213] jepsen worker 87 - jepsen.util 87 :invoke :write [29 29] INFO [2022-01-10 12:53:15,216] jepsen worker 1 - jepsen.util 1 :ok :write [0 27] INFO [2022-01-10 12:53:15,216] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:15,217] jepsen worker 1 - jepsen.util 1 :ok :read [0 27] INFO [2022-01-10 12:53:15,218] jepsen worker 1 - jepsen.util 1 :invoke :write [0 44] INFO [2022-01-10 12:53:15,325] jepsen worker 52 - jepsen.util 52 :ok :write [17 23] INFO [2022-01-10 12:53:15,325] jepsen worker 52 - jepsen.util 52 :invoke :write [17 3] INFO [2022-01-10 12:53:15,330] jepsen worker 78 - jepsen.util 78 :ok :write [26 42] INFO [2022-01-10 12:53:15,331] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:15,331] jepsen worker 77 - jepsen.util 167 :ok :write [25 20] INFO [2022-01-10 12:53:15,331] jepsen worker 77 - jepsen.util 167 :invoke :write [25 31] INFO [2022-01-10 12:53:15,332] jepsen worker 78 - jepsen.util 78 :ok :read [26 42] INFO [2022-01-10 12:53:15,332] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:15,333] jepsen worker 78 - jepsen.util 78 :ok :read [26 42] INFO [2022-01-10 12:53:15,333] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:15,334] jepsen worker 78 - jepsen.util 78 :ok :read [26 42] INFO [2022-01-10 12:53:15,335] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:15,335] jepsen worker 14 - jepsen.util 104 :ok :write [4 1] INFO [2022-01-10 12:53:15,335] jepsen worker 78 - jepsen.util 78 :ok :read [26 42] INFO [2022-01-10 12:53:15,336] jepsen worker 14 - jepsen.util 104 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,336] jepsen worker 78 - jepsen.util 78 :invoke :write [26 24] INFO [2022-01-10 12:53:15,337] jepsen worker 14 - jepsen.util 104 :ok :read [4 1] INFO [2022-01-10 12:53:15,337] jepsen worker 14 - jepsen.util 104 :invoke :write [4 19] INFO [2022-01-10 12:53:15,338] jepsen worker 23 - jepsen.util 23 :ok :write [7 37] INFO [2022-01-10 12:53:15,339] jepsen worker 23 - jepsen.util 23 :invoke :write [7 14] INFO [2022-01-10 12:53:15,340] jepsen worker 70 - jepsen.util 160 :ok :write [23 3] INFO [2022-01-10 12:53:15,340] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,341] jepsen worker 70 - jepsen.util 160 :ok :read [23 3] INFO [2022-01-10 12:53:15,341] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,342] jepsen worker 70 - jepsen.util 160 :ok :read [23 3] INFO [2022-01-10 12:53:15,342] jepsen worker 70 - jepsen.util 160 :invoke :write [23 34] INFO [2022-01-10 12:53:15,345] jepsen worker 43 - jepsen.util 43 :ok :write [14 25] INFO [2022-01-10 12:53:15,345] jepsen worker 43 - jepsen.util 43 :invoke :write [14 37] INFO [2022-01-10 12:53:15,350] jepsen worker 62 - jepsen.util 152 :ok :write [20 19] INFO [2022-01-10 12:53:15,351] jepsen worker 62 - jepsen.util 152 :invoke :write [20 23] INFO [2022-01-10 12:53:15,357] jepsen worker 83 - jepsen.util 173 :ok :write [27 14] INFO [2022-01-10 12:53:15,358] jepsen worker 83 - jepsen.util 173 :invoke :write [27 26] INFO [2022-01-10 12:53:15,358] jepsen worker 8 - jepsen.util 8 :ok :write [2 47] INFO [2022-01-10 12:53:15,359] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,360] jepsen worker 8 - jepsen.util 8 :ok :read [2 47] INFO [2022-01-10 12:53:15,360] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,361] jepsen worker 8 - jepsen.util 8 :ok :read [2 47] INFO [2022-01-10 12:53:15,362] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,363] jepsen worker 8 - jepsen.util 8 :ok :read [2 47] INFO [2022-01-10 12:53:15,363] jepsen worker 35 - jepsen.util 125 :ok :write [11 40] INFO [2022-01-10 12:53:15,363] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,364] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:15,364] jepsen worker 8 - jepsen.util 8 :ok :read [2 47] INFO [2022-01-10 12:53:15,365] jepsen worker 35 - jepsen.util 125 :ok :read [11 40] INFO [2022-01-10 12:53:15,365] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,366] jepsen worker 8 - jepsen.util 8 :ok :read [2 47] INFO [2022-01-10 12:53:15,366] jepsen worker 35 - jepsen.util 125 :invoke :write [11 6] INFO [2022-01-10 12:53:15,366] jepsen worker 8 - jepsen.util 8 :invoke :write [2 21] INFO [2022-01-10 12:53:15,367] jepsen worker 16 - jepsen.util 106 :ok :write [5 5] INFO [2022-01-10 12:53:15,368] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:15,368] jepsen worker 16 - jepsen.util 106 :ok :read [5 5] INFO [2022-01-10 12:53:15,369] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:15,370] jepsen worker 16 - jepsen.util 106 :ok :read [5 5] INFO [2022-01-10 12:53:15,370] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:15,370] jepsen worker 67 - jepsen.util 67 :ok :write [22 1] INFO [2022-01-10 12:53:15,371] jepsen worker 16 - jepsen.util 106 :ok :read [5 5] INFO [2022-01-10 12:53:15,371] jepsen worker 67 - jepsen.util 67 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,372] jepsen worker 16 - jepsen.util 106 :invoke :write [5 32] INFO [2022-01-10 12:53:15,372] jepsen worker 67 - jepsen.util 67 :ok :read [22 1] INFO [2022-01-10 12:53:15,373] jepsen worker 67 - jepsen.util 67 :invoke :write [22 18] INFO [2022-01-10 12:53:15,374] jepsen worker 49 - jepsen.util 139 :ok :write [16 0] INFO [2022-01-10 12:53:15,375] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:15,376] jepsen worker 49 - jepsen.util 139 :ok :read [16 0] INFO [2022-01-10 12:53:15,376] jepsen worker 49 - jepsen.util 139 :invoke :write [16 27] INFO [2022-01-10 12:53:15,387] jepsen worker 85 - jepsen.util 175 :ok :write [28 1] INFO [2022-01-10 12:53:15,387] jepsen worker 85 - jepsen.util 175 :invoke :write [28 3] INFO [2022-01-10 12:53:15,390] jepsen worker 39 - jepsen.util 39 :ok :write [13 41] INFO [2022-01-10 12:53:15,390] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,390] jepsen worker 4 - jepsen.util 4 :ok :write [1 40] INFO [2022-01-10 12:53:15,393] jepsen worker 4 - jepsen.util 4 :invoke :write [1 36] INFO [2022-01-10 12:53:15,394] jepsen worker 87 - jepsen.util 87 :ok :write [29 29] INFO [2022-01-10 12:53:15,396] jepsen worker 38 - jepsen.util 38 :ok :write [12 8] INFO [2022-01-10 12:53:15,396] jepsen worker 87 - jepsen.util 87 :invoke :write [29 26] INFO [2022-01-10 12:53:15,396] jepsen worker 83 - jepsen.util 173 :ok :write [27 26] INFO [2022-01-10 12:53:15,397] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:15,397] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:15,397] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,397] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,398] jepsen worker 8 - jepsen.util 8 :ok :write [2 21] INFO [2022-01-10 12:53:15,398] jepsen worker 83 - jepsen.util 173 :ok :read [27 26] INFO [2022-01-10 12:53:15,398] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:15,398] jepsen worker 8 - jepsen.util 8 :invoke :write [2 49] INFO [2022-01-10 12:53:15,398] jepsen worker 38 - jepsen.util 38 :ok :read [12 8] INFO [2022-01-10 12:53:15,399] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,399] jepsen worker 38 - jepsen.util 38 :invoke :write [12 19] INFO [2022-01-10 12:53:15,400] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,401] jepsen worker 83 - jepsen.util 173 :ok :read [27 26] INFO [2022-01-10 12:53:15,401] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:15,401] jepsen worker 52 - jepsen.util 52 :ok :write [17 3] INFO [2022-01-10 12:53:15,401] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,401] jepsen worker 1 - jepsen.util 1 :ok :write [0 44] INFO [2022-01-10 12:53:15,402] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:15,402] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:15,402] jepsen worker 35 - jepsen.util 125 :ok :write [11 6] INFO [2022-01-10 12:53:15,402] jepsen worker 83 - jepsen.util 173 :ok :read [27 26] INFO [2022-01-10 12:53:15,402] jepsen worker 52 - jepsen.util 52 :ok :read [17 3] INFO [2022-01-10 12:53:15,403] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,403] jepsen worker 35 - jepsen.util 125 :invoke :write [11 27] INFO [2022-01-10 12:53:15,403] jepsen worker 1 - jepsen.util 1 :ok :read [0 44] INFO [2022-01-10 12:53:15,404] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:15,404] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,404] jepsen worker 52 - jepsen.util 52 :invoke :write [17 12] INFO [2022-01-10 12:53:15,405] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,405] jepsen worker 83 - jepsen.util 173 :ok :read [27 26] INFO [2022-01-10 12:53:15,405] jepsen worker 77 - jepsen.util 167 :ok :write [25 31] INFO [2022-01-10 12:53:15,405] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:15,405] jepsen worker 78 - jepsen.util 78 :ok :write [26 24] INFO [2022-01-10 12:53:15,406] jepsen worker 83 - jepsen.util 173 :invoke :write [27 20] INFO [2022-01-10 12:53:15,406] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:15,406] jepsen worker 78 - jepsen.util 78 :invoke :write [26 12] INFO [2022-01-10 12:53:15,406] jepsen worker 1 - jepsen.util 1 :ok :read [0 44] INFO [2022-01-10 12:53:15,407] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,407] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:15,408] jepsen worker 1 - jepsen.util 1 :invoke :write [0 8] INFO [2022-01-10 12:53:15,408] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:15,408] jepsen worker 77 - jepsen.util 167 :ok :read [25 31] INFO [2022-01-10 12:53:15,408] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,409] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:15,409] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:15,410] jepsen worker 39 - jepsen.util 39 :invoke :write [13 27] INFO [2022-01-10 12:53:15,410] jepsen worker 23 - jepsen.util 23 :ok :write [7 14] INFO [2022-01-10 12:53:15,410] jepsen worker 77 - jepsen.util 167 :ok :read [25 31] INFO [2022-01-10 12:53:15,411] jepsen worker 23 - jepsen.util 23 :invoke :write [7 15] INFO [2022-01-10 12:53:15,411] jepsen worker 16 - jepsen.util 106 :ok :write [5 32] INFO [2022-01-10 12:53:15,411] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:15,412] jepsen worker 16 - jepsen.util 106 :invoke :write [5 48] INFO [2022-01-10 12:53:15,412] jepsen worker 77 - jepsen.util 167 :ok :read [25 31] INFO [2022-01-10 12:53:15,413] jepsen worker 77 - jepsen.util 167 :invoke :write [25 13] INFO [2022-01-10 12:53:15,413] jepsen worker 13 - jepsen.util 13 :ok :write [4 9] INFO [2022-01-10 12:53:15,414] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,415] jepsen worker 13 - jepsen.util 13 :ok :read [4 9] INFO [2022-01-10 12:53:15,415] jepsen worker 43 - jepsen.util 43 :ok :write [14 37] INFO [2022-01-10 12:53:15,417] jepsen worker 67 - jepsen.util 67 :ok :write [22 18] INFO [2022-01-10 12:53:15,418] jepsen worker 66 - jepsen.util 66 :info :write [22 49] :RequestTimeoutException INFO [2022-01-10 12:53:15,419] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:15,420] jepsen worker 13 - jepsen.util 13 :invoke :write [4 19] INFO [2022-01-10 12:53:15,420] jepsen worker 70 - jepsen.util 160 :ok :write [23 34] INFO [2022-01-10 12:53:15,420] jepsen worker 66 - jepsen.util 156 :invoke :write [22 34] INFO [2022-01-10 12:53:15,420] jepsen worker 43 - jepsen.util 43 :ok :read [14 37] INFO [2022-01-10 12:53:15,421] jepsen worker 67 - jepsen.util 67 :invoke :write [22 48] INFO [2022-01-10 12:53:15,421] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,422] jepsen worker 43 - jepsen.util 43 :invoke :write [14 44] INFO [2022-01-10 12:53:15,422] jepsen worker 62 - jepsen.util 152 :ok :write [20 23] INFO [2022-01-10 12:53:15,422] jepsen worker 70 - jepsen.util 160 :ok :read [23 34] INFO [2022-01-10 12:53:15,423] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,424] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,424] jepsen worker 70 - jepsen.util 160 :ok :read [23 34] INFO [2022-01-10 12:53:15,424] jepsen worker 49 - jepsen.util 139 :ok :write [16 27] INFO [2022-01-10 12:53:15,425] jepsen worker 62 - jepsen.util 152 :ok :read [20 23] INFO [2022-01-10 12:53:15,426] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,428] jepsen worker 70 - jepsen.util 160 :ok :read [23 34] INFO [2022-01-10 12:53:15,428] jepsen worker 85 - jepsen.util 175 :ok :write [28 3] INFO [2022-01-10 12:53:15,428] jepsen worker 83 - jepsen.util 173 :ok :write [27 20] INFO [2022-01-10 12:53:15,428] jepsen worker 49 - jepsen.util 139 :invoke :write [16 22] INFO [2022-01-10 12:53:15,426] jepsen worker 62 - jepsen.util 152 :invoke :write [20 27] INFO [2022-01-10 12:53:15,431] jepsen worker 8 - jepsen.util 8 :ok :write [2 49] INFO [2022-01-10 12:53:15,431] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,432] jepsen worker 8 - jepsen.util 8 :invoke :write [2 2] INFO [2022-01-10 12:53:15,432] jepsen worker 70 - jepsen.util 160 :ok :read [23 34] INFO [2022-01-10 12:53:15,433] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,434] jepsen worker 87 - jepsen.util 87 :ok :write [29 26] INFO [2022-01-10 12:53:15,434] jepsen worker 83 - jepsen.util 173 :invoke :write [27 2] INFO [2022-01-10 12:53:15,434] jepsen worker 85 - jepsen.util 175 :invoke :write [28 0] INFO [2022-01-10 12:53:15,434] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:15,434] jepsen worker 4 - jepsen.util 4 :ok :write [1 36] INFO [2022-01-10 12:53:15,435] jepsen worker 70 - jepsen.util 160 :ok :read [23 34] INFO [2022-01-10 12:53:15,435] jepsen worker 4 - jepsen.util 4 :invoke :write [1 18] INFO [2022-01-10 12:53:15,435] jepsen worker 87 - jepsen.util 87 :ok :read [29 26] INFO [2022-01-10 12:53:15,436] jepsen worker 70 - jepsen.util 160 :invoke :write [23 36] INFO [2022-01-10 12:53:15,436] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:15,439] jepsen worker 87 - jepsen.util 87 :ok :read [29 26] INFO [2022-01-10 12:53:15,440] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:15,440] jepsen worker 38 - jepsen.util 38 :ok :write [12 19] INFO [2022-01-10 12:53:15,440] jepsen worker 35 - jepsen.util 125 :ok :write [11 27] INFO [2022-01-10 12:53:15,441] jepsen worker 38 - jepsen.util 38 :invoke :write [12 29] INFO [2022-01-10 12:53:15,441] jepsen worker 87 - jepsen.util 87 :ok :read [29 26] INFO [2022-01-10 12:53:15,441] jepsen worker 35 - jepsen.util 125 :invoke :write [11 45] INFO [2022-01-10 12:53:15,442] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:15,442] jepsen worker 1 - jepsen.util 1 :ok :write [0 8] INFO [2022-01-10 12:53:15,443] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:15,443] jepsen worker 1 - jepsen.util 1 :ok :read [0 8] INFO [2022-01-10 12:53:15,443] jepsen worker 87 - jepsen.util 87 :ok :read [29 26] INFO [2022-01-10 12:53:15,444] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:15,444] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:15,445] jepsen worker 1 - jepsen.util 1 :ok :read [0 8] INFO [2022-01-10 12:53:15,446] jepsen worker 1 - jepsen.util 1 :invoke :write [0 29] INFO [2022-01-10 12:53:15,446] jepsen worker 87 - jepsen.util 87 :ok :read [29 26] INFO [2022-01-10 12:53:15,446] jepsen worker 52 - jepsen.util 52 :ok :write [17 12] INFO [2022-01-10 12:53:15,447] jepsen worker 77 - jepsen.util 167 :ok :write [25 13] INFO [2022-01-10 12:53:15,447] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:15,448] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:15,448] jepsen worker 52 - jepsen.util 52 :ok :read [17 12] INFO [2022-01-10 12:53:15,448] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:15,449] jepsen worker 87 - jepsen.util 87 :ok :read [29 26] INFO [2022-01-10 12:53:15,449] jepsen worker 52 - jepsen.util 52 :invoke :write [17 6] INFO [2022-01-10 12:53:15,449] jepsen worker 77 - jepsen.util 167 :ok :read [25 13] INFO [2022-01-10 12:53:15,449] jepsen worker 87 - jepsen.util 87 :invoke :write [29 40] INFO [2022-01-10 12:53:15,450] jepsen worker 77 - jepsen.util 167 :invoke :write [25 16] INFO [2022-01-10 12:53:15,450] jepsen worker 39 - jepsen.util 39 :ok :write [13 27] INFO [2022-01-10 12:53:15,450] jepsen worker 78 - jepsen.util 78 :ok :write [26 12] INFO [2022-01-10 12:53:15,451] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,451] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:15,452] jepsen worker 39 - jepsen.util 39 :ok :read [13 27] INFO [2022-01-10 12:53:15,453] jepsen worker 78 - jepsen.util 78 :ok :read [26 12] INFO [2022-01-10 12:53:15,453] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,454] jepsen worker 78 - jepsen.util 78 :invoke :write [26 16] INFO [2022-01-10 12:53:15,454] jepsen worker 39 - jepsen.util 39 :ok :read [13 27] INFO [2022-01-10 12:53:15,454] jepsen worker 16 - jepsen.util 106 :ok :write [5 48] INFO [2022-01-10 12:53:15,455] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,455] jepsen worker 16 - jepsen.util 106 :invoke :write [5 27] INFO [2022-01-10 12:53:15,455] jepsen worker 39 - jepsen.util 39 :ok :read [13 27] INFO [2022-01-10 12:53:15,456] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,456] jepsen worker 39 - jepsen.util 39 :ok :read [13 27] INFO [2022-01-10 12:53:15,457] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,457] jepsen worker 13 - jepsen.util 13 :ok :write [4 19] INFO [2022-01-10 12:53:15,458] jepsen worker 23 - jepsen.util 23 :ok :write [7 15] INFO [2022-01-10 12:53:15,458] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,458] jepsen worker 39 - jepsen.util 39 :ok :read [13 27] INFO [2022-01-10 12:53:15,458] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:15,459] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,459] jepsen worker 23 - jepsen.util 23 :ok :read [7 15] INFO [2022-01-10 12:53:15,459] jepsen worker 13 - jepsen.util 13 :ok :read [4 19] INFO [2022-01-10 12:53:15,460] jepsen worker 39 - jepsen.util 39 :ok :read [13 27] INFO [2022-01-10 12:53:15,460] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:15,460] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,461] jepsen worker 23 - jepsen.util 23 :ok :read [7 15] INFO [2022-01-10 12:53:15,461] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,461] jepsen worker 13 - jepsen.util 13 :ok :read [4 19] INFO [2022-01-10 12:53:15,461] jepsen worker 23 - jepsen.util 23 :invoke :write [7 2] INFO [2022-01-10 12:53:15,462] jepsen worker 39 - jepsen.util 39 :ok :read [13 27] INFO [2022-01-10 12:53:15,462] jepsen worker 13 - jepsen.util 13 :invoke :write [4 14] INFO [2022-01-10 12:53:15,462] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,463] jepsen worker 39 - jepsen.util 39 :ok :read [13 27] INFO [2022-01-10 12:53:15,463] jepsen worker 39 - jepsen.util 39 :invoke :write [13 0] INFO [2022-01-10 12:53:15,464] jepsen worker 70 - jepsen.util 160 :ok :write [23 36] INFO [2022-01-10 12:53:15,464] jepsen worker 66 - jepsen.util 156 :ok :write [22 34] INFO [2022-01-10 12:53:15,464] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,465] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,466] jepsen worker 66 - jepsen.util 156 :ok :read [22 34] INFO [2022-01-10 12:53:15,466] jepsen worker 70 - jepsen.util 160 :ok :read [23 36] INFO [2022-01-10 12:53:15,466] jepsen worker 43 - jepsen.util 43 :ok :write [14 44] INFO [2022-01-10 12:53:15,466] jepsen worker 66 - jepsen.util 156 :invoke :write [22 21] INFO [2022-01-10 12:53:15,467] jepsen worker 43 - jepsen.util 43 :invoke :write [14 31] INFO [2022-01-10 12:53:15,467] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,468] jepsen worker 70 - jepsen.util 160 :ok :read [23 36] INFO [2022-01-10 12:53:15,468] jepsen worker 70 - jepsen.util 160 :invoke :write [23 15] INFO [2022-01-10 12:53:15,470] jepsen worker 85 - jepsen.util 175 :ok :write [28 0] INFO [2022-01-10 12:53:15,470] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:15,471] jepsen worker 85 - jepsen.util 175 :ok :read [28 0] INFO [2022-01-10 12:53:15,471] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:15,473] jepsen worker 85 - jepsen.util 175 :ok :read [28 0] INFO [2022-01-10 12:53:15,473] jepsen worker 85 - jepsen.util 175 :invoke :write [28 29] INFO [2022-01-10 12:53:15,474] jepsen worker 4 - jepsen.util 4 :ok :write [1 18] INFO [2022-01-10 12:53:15,474] jepsen worker 62 - jepsen.util 152 :ok :write [20 27] INFO [2022-01-10 12:53:15,474] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:15,475] jepsen worker 62 - jepsen.util 152 :invoke :write [20 24] INFO [2022-01-10 12:53:15,475] jepsen worker 4 - jepsen.util 4 :ok :read [1 18] INFO [2022-01-10 12:53:15,476] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:15,477] jepsen worker 4 - jepsen.util 4 :ok :read [1 18] INFO [2022-01-10 12:53:15,477] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] WARN [2022-01-10 12:53:15,478] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7134567,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":7148806,"total_server_duration_us":1936,"encode_duration_us":76,"operation_id":"0xc80","timeout_ms":10000,"last_server_duration_us":43,"total_duration_us":10020074},{"operation_name":"upsert","last_dispatch_duration_us":610,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":23083,"total_server_duration_us":3122,"encode_duration_us":61,"operation_id":"0xdbd","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10009900},{"operation_name":"upsert","last_dispatch_duration_us":7064512,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":7076731,"total_server_duration_us":1638,"encode_duration_us":41,"operation_id":"0xd92","timeout_ms":10000,"last_server_duration_us":27,"total_duration_us":10006885},{"operation_name":"upsert","last_dispatch_duration_us":594,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":21061,"total_server_duration_us":2533,"encode_duration_us":90,"operation_id":"0xd94","timeout_ms":10000,"last_server_duration_us":117,"total_duration_us":10006549},{"operation_name":"upsert","last_dispatch_duration_us":1327,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":21655,"total_server_duration_us":2437,"encode_duration_us":88,"operation_id":"0xe6a","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10002111},{"operation_name":"upsert","last_dispatch_duration_us":532,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":20024,"total_server_duration_us":3404,"encode_duration_us":51,"operation_id":"0xe39","timeout_ms":10000,"last_server_duration_us":84,"total_duration_us":10001881},{"operation_name":"upsert","last_dispatch_duration_us":418,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":21925,"total_server_duration_us":3267,"encode_duration_us":65,"operation_id":"0xc9d","timeout_ms":10000,"last_server_duration_us":49,"total_duration_us":10001770},{"operation_name":"upsert","last_dispatch_duration_us":9013055,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":9013055,"total_server_duration_us":9011909,"encode_duration_us":46,"operation_id":"0x1191","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":9013490},{"operation_name":"upsert","last_dispatch_duration_us":120994,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":135886,"total_server_duration_us":122149,"encode_duration_us":33,"operation_id":"0xcf8","timeout_ms":10000,"last_server_duration_us":119804,"total_duration_us":6820543},{"operation_name":"upsert","last_dispatch_duration_us":22779,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":38815,"total_server_duration_us":23835,"encode_duration_us":42,"operation_id":"0x1901","timeout_ms":10000,"last_server_duration_us":22223,"total_duration_us":6222669}],"total_count":21}} WARN [2022-01-10 12:53:15,479] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":845,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000004/00000000C4B4FBBF","last_local_socket":"172.23.104.79:58638","total_dispatch_duration_us":518122,"total_server_duration_us":2564,"encode_duration_us":64,"operation_id":"0xd90","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10009916},{"operation_name":"upsert","last_dispatch_duration_us":614,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":395845,"total_server_duration_us":2950,"encode_duration_us":40,"operation_id":"0xdf9","timeout_ms":10000,"last_server_duration_us":56,"total_duration_us":10008902},{"operation_name":"upsert","last_dispatch_duration_us":653,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":19905,"total_server_duration_us":3018,"encode_duration_us":44,"operation_id":"0xf85","timeout_ms":10000,"last_server_duration_us":84,"total_duration_us":10006541},{"operation_name":"upsert","last_dispatch_duration_us":684,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000004/0000000048226CCA","last_local_socket":"172.23.104.79:48384","total_dispatch_duration_us":23126,"total_server_duration_us":3321,"encode_duration_us":66,"operation_id":"0xd8c","timeout_ms":10000,"last_server_duration_us":145,"total_duration_us":10006054},{"operation_name":"upsert","last_dispatch_duration_us":527,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":21661,"total_server_duration_us":2591,"encode_duration_us":34,"operation_id":"0xdce","timeout_ms":10000,"last_server_duration_us":49,"total_duration_us":10005693},{"operation_name":"upsert","last_dispatch_duration_us":579,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":24625,"total_server_duration_us":3588,"encode_duration_us":71,"operation_id":"0xdc1","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10004725},{"operation_name":"upsert","last_dispatch_duration_us":650,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":18760,"total_server_duration_us":3157,"encode_duration_us":61,"operation_id":"0xf59","timeout_ms":10000,"last_server_duration_us":100,"total_duration_us":10004154},{"operation_name":"upsert","last_dispatch_duration_us":652,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":21910,"total_server_duration_us":2764,"encode_duration_us":53,"operation_id":"0xe36","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10004110},{"operation_name":"upsert","last_dispatch_duration_us":879,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000004/00000000C4B4FBBF","last_local_socket":"172.23.104.79:58638","total_dispatch_duration_us":313805,"total_server_duration_us":2065,"encode_duration_us":31,"operation_id":"0xe24","timeout_ms":10000,"last_server_duration_us":43,"total_duration_us":10002819},{"operation_name":"upsert","last_dispatch_duration_us":869,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000004/00000000C4B4FBBF","last_local_socket":"172.23.104.79:58638","total_dispatch_duration_us":518608,"total_server_duration_us":2614,"encode_duration_us":49,"operation_id":"0xd9b","timeout_ms":10000,"last_server_duration_us":49,"total_duration_us":10002695}],"total_count":14}} INFO [2022-01-10 12:53:15,479] jepsen worker 4 - jepsen.util 4 :ok :read [1 18] INFO [2022-01-10 12:53:15,479] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:15,480] jepsen worker 4 - jepsen.util 4 :ok :read [1 18] INFO [2022-01-10 12:53:15,480] jepsen worker 38 - jepsen.util 38 :ok :write [12 29] INFO [2022-01-10 12:53:15,480] jepsen worker 49 - jepsen.util 139 :ok :write [16 22] INFO [2022-01-10 12:53:15,481] jepsen worker 38 - jepsen.util 38 :invoke :write [12 22] WARN [2022-01-10 12:53:15,481] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":6096929,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":6107448,"total_server_duration_us":1418,"encode_duration_us":57,"operation_id":"0xdde","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10008912},{"operation_name":"upsert","last_dispatch_duration_us":490,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","total_dispatch_duration_us":22368,"total_server_duration_us":2530,"encode_duration_us":41,"operation_id":"0xe37","timeout_ms":10000,"last_server_duration_us":43,"total_duration_us":10007305},{"operation_name":"upsert","last_dispatch_duration_us":4432,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","total_dispatch_duration_us":23379,"total_server_duration_us":2932,"encode_duration_us":52,"operation_id":"0xe79","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10007189},{"operation_name":"upsert","last_dispatch_duration_us":6005120,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":6017619,"total_server_duration_us":1964,"encode_duration_us":66,"operation_id":"0xebe","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10007047},{"operation_name":"upsert","last_dispatch_duration_us":490,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","total_dispatch_duration_us":17103,"total_server_duration_us":2110,"encode_duration_us":42,"operation_id":"0xe2f","timeout_ms":10000,"last_server_duration_us":49,"total_duration_us":10005601},{"operation_name":"upsert","last_dispatch_duration_us":9013269,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":9013269,"total_server_duration_us":9011909,"encode_duration_us":40,"operation_id":"0x1193","timeout_ms":10000,"last_server_duration_us":9011909,"total_duration_us":9013498},{"operation_name":"upsert","last_dispatch_duration_us":9008498,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":9008498,"total_server_duration_us":9007669,"encode_duration_us":57,"operation_id":"0x11ce","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":9009002},{"operation_name":"upsert","last_dispatch_duration_us":9007386,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":9007386,"total_server_duration_us":9006609,"encode_duration_us":62,"operation_id":"0x1197","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":9007811},{"operation_name":"upsert","last_dispatch_duration_us":9001265,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":9001265,"total_server_duration_us":9000252,"encode_duration_us":44,"operation_id":"0x1182","timeout_ms":10000,"last_server_duration_us":9000252,"total_duration_us":9002224},{"operation_name":"upsert","last_dispatch_duration_us":139511,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000002/00000000F0645BC3","last_local_socket":"172.23.104.79:34668","total_dispatch_duration_us":155399,"total_server_duration_us":141450,"encode_duration_us":42,"operation_id":"0x150a","timeout_ms":10000,"last_server_duration_us":138978,"total_duration_us":7343061}],"total_count":16}} INFO [2022-01-10 12:53:15,482] jepsen worker 4 - jepsen.util 4 :invoke :write [1 20] WARN [2022-01-10 12:53:15,482] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":6169586,"last_local_id":"2B94F89700000002/000000004AD6A961","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":6169586,"last_local_socket":"172.23.104.79:48334","total_server_duration_us":43,"operation_id":"0x14f0","timeout_ms":2000,"last_server_duration_us":43,"total_duration_us":2004581}],"total_count":1}} INFO [2022-01-10 12:53:15,483] jepsen worker 49 - jepsen.util 139 :invoke :write [16 0] INFO [2022-01-10 12:53:15,484] jepsen worker 83 - jepsen.util 173 :ok :write [27 2] INFO [2022-01-10 12:53:15,485] jepsen worker 83 - jepsen.util 173 :invoke :write [27 40] INFO [2022-01-10 12:53:15,486] jepsen worker 52 - jepsen.util 52 :ok :write [17 6] INFO [2022-01-10 12:53:15,487] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:15,488] jepsen worker 8 - jepsen.util 8 :ok :write [2 2] INFO [2022-01-10 12:53:15,488] jepsen worker 52 - jepsen.util 52 :ok :read [17 6] INFO [2022-01-10 12:53:15,491] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,492] jepsen worker 52 - jepsen.util 52 :invoke :write [17 43] INFO [2022-01-10 12:53:15,495] jepsen worker 78 - jepsen.util 78 :ok :write [26 16] INFO [2022-01-10 12:53:15,496] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:15,497] jepsen worker 77 - jepsen.util 167 :ok :write [25 16] INFO [2022-01-10 12:53:15,497] jepsen worker 1 - jepsen.util 1 :ok :write [0 29] INFO [2022-01-10 12:53:15,497] jepsen worker 78 - jepsen.util 78 :ok :read [26 16] INFO [2022-01-10 12:53:15,497] jepsen worker 8 - jepsen.util 8 :ok :read [2 2] INFO [2022-01-10 12:53:15,498] jepsen worker 78 - jepsen.util 78 :invoke :write [26 38] INFO [2022-01-10 12:53:15,498] jepsen worker 1 - jepsen.util 1 :invoke :write [0 7] INFO [2022-01-10 12:53:15,500] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,500] jepsen worker 77 - jepsen.util 167 :invoke :write [25 37] INFO [2022-01-10 12:53:15,500] jepsen worker 13 - jepsen.util 13 :ok :write [4 14] INFO [2022-01-10 12:53:15,500] jepsen worker 87 - jepsen.util 87 :ok :write [29 40] INFO [2022-01-10 12:53:15,501] jepsen worker 8 - jepsen.util 8 :ok :read [2 2] INFO [2022-01-10 12:53:15,501] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:15,502] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,502] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,503] jepsen worker 8 - jepsen.util 8 :ok :read [2 2] INFO [2022-01-10 12:53:15,503] jepsen worker 35 - jepsen.util 125 :ok :write [11 45] INFO [2022-01-10 12:53:15,504] jepsen worker 13 - jepsen.util 13 :ok :read [4 14] INFO [2022-01-10 12:53:15,504] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,504] jepsen worker 87 - jepsen.util 87 :ok :read [29 40] INFO [2022-01-10 12:53:15,504] jepsen worker 8 - jepsen.util 8 :ok :read [2 2] INFO [2022-01-10 12:53:15,504] jepsen worker 13 - jepsen.util 13 :invoke :write [4 7] INFO [2022-01-10 12:53:15,505] jepsen worker 70 - jepsen.util 160 :ok :write [23 15] INFO [2022-01-10 12:53:15,505] jepsen worker 35 - jepsen.util 125 :invoke :write [11 28] INFO [2022-01-10 12:53:15,506] jepsen worker 8 - jepsen.util 8 :invoke :write [2 28] INFO [2022-01-10 12:53:15,506] jepsen worker 23 - jepsen.util 23 :ok :write [7 2] INFO [2022-01-10 12:53:15,506] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,507] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:15,507] jepsen worker 70 - jepsen.util 160 :ok :read [23 15] INFO [2022-01-10 12:53:15,508] jepsen worker 23 - jepsen.util 23 :invoke :write [7 21] INFO [2022-01-10 12:53:15,508] jepsen worker 87 - jepsen.util 87 :ok :read [29 40] INFO [2022-01-10 12:53:15,509] jepsen worker 39 - jepsen.util 39 :ok :write [13 0] INFO [2022-01-10 12:53:15,509] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,510] jepsen worker 70 - jepsen.util 160 :ok :read [23 15] INFO [2022-01-10 12:53:15,510] jepsen worker 87 - jepsen.util 87 :invoke :write [29 31] INFO [2022-01-10 12:53:15,510] jepsen worker 85 - jepsen.util 175 :ok :write [28 29] INFO [2022-01-10 12:53:15,511] jepsen worker 70 - jepsen.util 160 :invoke :write [23 42] INFO [2022-01-10 12:53:15,512] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,512] jepsen worker 85 - jepsen.util 175 :invoke :write [28 7] INFO [2022-01-10 12:53:15,513] jepsen worker 39 - jepsen.util 39 :ok :read [13 0] INFO [2022-01-10 12:53:15,514] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,515] jepsen worker 39 - jepsen.util 39 :ok :read [13 0] INFO [2022-01-10 12:53:15,516] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,516] jepsen worker 4 - jepsen.util 4 :ok :write [1 20] INFO [2022-01-10 12:53:15,516] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:15,517] jepsen worker 39 - jepsen.util 39 :ok :read [13 0] INFO [2022-01-10 12:53:15,517] jepsen worker 4 - jepsen.util 4 :ok :read [1 20] INFO [2022-01-10 12:53:15,517] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,517] jepsen worker 43 - jepsen.util 43 :ok :write [14 31] INFO [2022-01-10 12:53:15,518] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:15,518] jepsen worker 16 - jepsen.util 106 :ok :write [5 27] INFO [2022-01-10 12:53:15,518] jepsen worker 39 - jepsen.util 39 :ok :read [13 0] INFO [2022-01-10 12:53:15,519] jepsen worker 43 - jepsen.util 43 :invoke :write [14 3] INFO [2022-01-10 12:53:15,519] jepsen worker 4 - jepsen.util 4 :ok :read [1 20] INFO [2022-01-10 12:53:15,523] jepsen worker 38 - jepsen.util 38 :ok :write [12 22] INFO [2022-01-10 12:53:15,523] jepsen worker 4 - jepsen.util 4 :invoke :write [1 29] INFO [2022-01-10 12:53:15,524] jepsen worker 62 - jepsen.util 152 :ok :write [20 24] INFO [2022-01-10 12:53:15,524] jepsen worker 16 - jepsen.util 106 :invoke :write [5 4] INFO [2022-01-10 12:53:15,525] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,526] jepsen worker 38 - jepsen.util 38 :invoke :write [12 22] INFO [2022-01-10 12:53:15,526] jepsen worker 39 - jepsen.util 39 :ok :read [13 0] INFO [2022-01-10 12:53:15,526] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,526] jepsen worker 66 - jepsen.util 156 :ok :write [22 21] INFO [2022-01-10 12:53:15,527] jepsen worker 66 - jepsen.util 156 :invoke :write [22 47] INFO [2022-01-10 12:53:15,527] jepsen worker 62 - jepsen.util 152 :ok :read [20 24] INFO [2022-01-10 12:53:15,527] jepsen worker 39 - jepsen.util 39 :invoke :write [13 43] INFO [2022-01-10 12:53:15,528] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,529] jepsen worker 52 - jepsen.util 52 :ok :write [17 43] INFO [2022-01-10 12:53:15,529] jepsen worker 62 - jepsen.util 152 :ok :read [20 24] INFO [2022-01-10 12:53:15,529] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,530] jepsen worker 52 - jepsen.util 52 :invoke :write [17 13] INFO [2022-01-10 12:53:15,531] jepsen worker 62 - jepsen.util 152 :ok :read [20 24] INFO [2022-01-10 12:53:15,532] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,533] jepsen worker 62 - jepsen.util 152 :ok :read [20 24] INFO [2022-01-10 12:53:15,533] jepsen worker 62 - jepsen.util 152 :invoke :write [20 6] INFO [2022-01-10 12:53:15,534] jepsen worker 49 - jepsen.util 139 :ok :write [16 0] INFO [2022-01-10 12:53:15,534] jepsen worker 78 - jepsen.util 78 :ok :write [26 38] INFO [2022-01-10 12:53:15,535] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:15,535] jepsen worker 78 - jepsen.util 78 :invoke :write [26 45] INFO [2022-01-10 12:53:15,536] jepsen worker 49 - jepsen.util 139 :ok :read [16 0] INFO [2022-01-10 12:53:15,536] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:15,537] jepsen worker 49 - jepsen.util 139 :ok :read [16 0] INFO [2022-01-10 12:53:15,537] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:15,539] jepsen worker 49 - jepsen.util 139 :ok :read [16 0] INFO [2022-01-10 12:53:15,539] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:15,540] jepsen worker 49 - jepsen.util 139 :ok :read [16 0] INFO [2022-01-10 12:53:15,540] jepsen worker 13 - jepsen.util 13 :ok :write [4 7] INFO [2022-01-10 12:53:15,541] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,541] jepsen worker 49 - jepsen.util 139 :invoke :write [16 14] INFO [2022-01-10 12:53:15,541] jepsen worker 13 - jepsen.util 13 :ok :read [4 7] INFO [2022-01-10 12:53:15,542] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,543] jepsen worker 13 - jepsen.util 13 :ok :read [4 7] INFO [2022-01-10 12:53:15,543] jepsen worker 83 - jepsen.util 173 :ok :write [27 40] INFO [2022-01-10 12:53:15,544] jepsen worker 13 - jepsen.util 13 :invoke :write [4 41] INFO [2022-01-10 12:53:15,545] jepsen worker 83 - jepsen.util 173 :invoke :write [27 4] INFO [2022-01-10 12:53:15,545] jepsen worker 70 - jepsen.util 160 :ok :write [23 42] INFO [2022-01-10 12:53:15,545] jepsen worker 70 - jepsen.util 160 :invoke :write [23 28] INFO [2022-01-10 12:53:15,550] jepsen worker 8 - jepsen.util 8 :ok :write [2 28] INFO [2022-01-10 12:53:15,551] jepsen worker 85 - jepsen.util 175 :ok :write [28 7] INFO [2022-01-10 12:53:15,551] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,551] jepsen worker 85 - jepsen.util 175 :invoke :write [28 2] INFO [2022-01-10 12:53:15,552] jepsen worker 8 - jepsen.util 8 :ok :read [2 28] INFO [2022-01-10 12:53:15,552] jepsen worker 8 - jepsen.util 8 :invoke :write [2 31] INFO [2022-01-10 12:53:15,557] jepsen worker 4 - jepsen.util 4 :ok :write [1 29] INFO [2022-01-10 12:53:15,558] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:15,559] jepsen worker 4 - jepsen.util 4 :ok :read [1 29] INFO [2022-01-10 12:53:15,560] jepsen worker 4 - jepsen.util 4 :invoke :write [1 11] INFO [2022-01-10 12:53:15,564] jepsen worker 38 - jepsen.util 38 :ok :write [12 22] INFO [2022-01-10 12:53:15,564] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:15,565] jepsen worker 38 - jepsen.util 38 :ok :read [12 22] INFO [2022-01-10 12:53:15,565] jepsen worker 38 - jepsen.util 38 :invoke :write [12 5] WARN [2022-01-10 12:53:15,578] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":223798,"last_local_id":"2B94F89700000005/0000000090CEECC6","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":357299,"last_local_socket":"172.23.104.79:48368","total_server_duration_us":227448,"encode_duration_us":63,"operation_id":"0xf1e","timeout_ms":10000,"last_server_duration_us":223220,"total_duration_us":10005050}],"total_count":1}} WARN [2022-01-10 12:53:15,579] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7098657,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000005/000000006FA84E55","last_local_socket":"172.23.104.79:34638","total_dispatch_duration_us":7107394,"total_server_duration_us":1198,"encode_duration_us":32,"operation_id":"0xd86","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10009926},{"operation_name":"upsert","last_dispatch_duration_us":7056342,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":7066566,"total_server_duration_us":1605,"encode_duration_us":34,"operation_id":"0xdc7","timeout_ms":10000,"last_server_duration_us":268,"total_duration_us":10008927},{"operation_name":"upsert","last_dispatch_duration_us":547,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":14465,"total_server_duration_us":1695,"encode_duration_us":80,"operation_id":"0xd60","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10008291},{"operation_name":"upsert","last_dispatch_duration_us":223798,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000005/0000000090CEECC6","last_local_socket":"172.23.104.79:48368","total_dispatch_duration_us":357299,"total_server_duration_us":227448,"encode_duration_us":63,"operation_id":"0xf1e","timeout_ms":10000,"last_server_duration_us":223220,"total_duration_us":10005050},{"operation_name":"upsert","last_dispatch_duration_us":556,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000005/0000000090CEECC6","last_local_socket":"172.23.104.79:48368","total_dispatch_duration_us":18511,"total_server_duration_us":3239,"encode_duration_us":38,"operation_id":"0xe4a","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10002104},{"operation_name":"upsert","last_dispatch_duration_us":647,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000005/0000000090CEECC6","last_local_socket":"172.23.104.79:48368","total_dispatch_duration_us":21017,"total_server_duration_us":3524,"encode_duration_us":48,"operation_id":"0xd97","timeout_ms":10000,"last_server_duration_us":155,"total_duration_us":10001941},{"operation_name":"upsert","last_dispatch_duration_us":9003028,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":9003028,"total_server_duration_us":9001311,"encode_duration_us":61,"operation_id":"0x11b4","timeout_ms":10000,"last_server_duration_us":9001311,"total_duration_us":9003654},{"operation_name":"upsert","last_dispatch_duration_us":9001335,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000005/000000006FA84E55","last_local_socket":"172.23.104.79:34638","total_dispatch_duration_us":9001335,"total_server_duration_us":9000252,"encode_duration_us":70,"operation_id":"0x119b","timeout_ms":10000,"last_server_duration_us":9000252,"total_duration_us":9001869},{"operation_name":"upsert","last_dispatch_duration_us":9001148,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":9001148,"total_server_duration_us":9000252,"encode_duration_us":72,"operation_id":"0x11a7","timeout_ms":10000,"last_server_duration_us":9000252,"total_duration_us":9001568},{"operation_name":"upsert","last_dispatch_duration_us":41415,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":379269,"total_server_duration_us":42168,"encode_duration_us":34,"operation_id":"0xd0a","timeout_ms":10000,"last_server_duration_us":40163,"total_duration_us":7621543}],"total_count":20}} WARN [2022-01-10 12:53:15,582] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":2793123,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":2808692,"total_server_duration_us":1566,"encode_duration_us":47,"operation_id":"0xe31","timeout_ms":10000,"last_server_duration_us":27,"total_duration_us":10011096},{"operation_name":"upsert","last_dispatch_duration_us":854,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":20356,"total_server_duration_us":2885,"encode_duration_us":30,"operation_id":"0xe41","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10009869},{"operation_name":"upsert","last_dispatch_duration_us":673,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000006/000000001E9CFABF","last_local_socket":"172.23.104.79:40226","total_dispatch_duration_us":17853,"total_server_duration_us":2550,"encode_duration_us":37,"operation_id":"0xdcd","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10009498},{"operation_name":"upsert","last_dispatch_duration_us":671,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":25393,"total_server_duration_us":6211,"encode_duration_us":53,"operation_id":"0xdf6","timeout_ms":10000,"last_server_duration_us":108,"total_duration_us":10006811},{"operation_name":"upsert","last_dispatch_duration_us":2803246,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":2817718,"total_server_duration_us":1962,"encode_duration_us":34,"operation_id":"0xe2c","timeout_ms":10000,"last_server_duration_us":84,"total_duration_us":10006403},{"operation_name":"upsert","last_dispatch_duration_us":695,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000006/000000001E9CFABF","last_local_socket":"172.23.104.79:40226","total_dispatch_duration_us":18073,"total_server_duration_us":2531,"encode_duration_us":43,"operation_id":"0xd75","timeout_ms":10000,"last_server_duration_us":100,"total_duration_us":10005826},{"operation_name":"upsert","last_dispatch_duration_us":610,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000006/000000001E9CFABF","last_local_socket":"172.23.104.79:40226","total_dispatch_duration_us":18796,"total_server_duration_us":2461,"encode_duration_us":39,"operation_id":"0xd8a","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10002225},{"operation_name":"upsert","last_dispatch_duration_us":9008515,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":9008515,"total_server_duration_us":9007669,"encode_duration_us":32,"operation_id":"0x11a9","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":9009040},{"operation_name":"upsert","last_dispatch_duration_us":27584,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":50525,"total_server_duration_us":28444,"encode_duration_us":33,"operation_id":"0xf08","timeout_ms":10000,"last_server_duration_us":26418,"total_duration_us":6733015},{"operation_name":"upsert","last_dispatch_duration_us":27446,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000006/0000000044D7A75E","last_local_socket":"172.23.104.79:48380","total_dispatch_duration_us":41894,"total_server_duration_us":28784,"encode_duration_us":28,"operation_id":"0x1a15","timeout_ms":10000,"last_server_duration_us":26507,"total_duration_us":6225673}],"total_count":18}} INFO [2022-01-10 12:53:15,659] jepsen worker 77 - jepsen.util 167 :ok :write [25 37] INFO [2022-01-10 12:53:15,660] jepsen worker 52 - jepsen.util 52 :ok :write [17 13] INFO [2022-01-10 12:53:15,660] jepsen worker 77 - jepsen.util 167 :invoke :write [25 17] INFO [2022-01-10 12:53:15,661] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:15,662] jepsen worker 52 - jepsen.util 52 :ok :read [17 13] INFO [2022-01-10 12:53:15,663] jepsen worker 52 - jepsen.util 52 :invoke :write [17 20] INFO [2022-01-10 12:53:15,665] jepsen worker 78 - jepsen.util 78 :ok :write [26 45] INFO [2022-01-10 12:53:15,665] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:15,666] jepsen worker 78 - jepsen.util 78 :ok :read [26 45] INFO [2022-01-10 12:53:15,667] jepsen worker 78 - jepsen.util 78 :invoke :write [26 0] INFO [2022-01-10 12:53:15,669] jepsen worker 13 - jepsen.util 13 :ok :write [4 41] INFO [2022-01-10 12:53:15,670] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,670] jepsen worker 13 - jepsen.util 13 :ok :read [4 41] INFO [2022-01-10 12:53:15,671] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,672] jepsen worker 13 - jepsen.util 13 :ok :read [4 41] INFO [2022-01-10 12:53:15,673] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,673] jepsen worker 23 - jepsen.util 23 :ok :write [7 21] INFO [2022-01-10 12:53:15,674] jepsen worker 13 - jepsen.util 13 :ok :read [4 41] INFO [2022-01-10 12:53:15,674] jepsen worker 70 - jepsen.util 160 :ok :write [23 28] INFO [2022-01-10 12:53:15,675] jepsen worker 23 - jepsen.util 23 :invoke :write [7 15] INFO [2022-01-10 12:53:15,675] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,676] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,676] jepsen worker 70 - jepsen.util 160 :ok :read [23 28] INFO [2022-01-10 12:53:15,677] jepsen worker 13 - jepsen.util 13 :ok :read [4 41] INFO [2022-01-10 12:53:15,677] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,677] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,678] jepsen worker 70 - jepsen.util 160 :ok :read [23 28] INFO [2022-01-10 12:53:15,678] jepsen worker 13 - jepsen.util 13 :ok :read [4 41] INFO [2022-01-10 12:53:15,679] jepsen worker 43 - jepsen.util 43 :ok :write [14 3] INFO [2022-01-10 12:53:15,679] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,680] jepsen worker 43 - jepsen.util 43 :invoke :write [14 43] INFO [2022-01-10 12:53:15,680] jepsen worker 70 - jepsen.util 160 :ok :read [23 28] INFO [2022-01-10 12:53:15,680] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,681] jepsen worker 13 - jepsen.util 13 :ok :read [4 41] INFO [2022-01-10 12:53:15,681] jepsen worker 87 - jepsen.util 87 :ok :write [29 31] INFO [2022-01-10 12:53:15,682] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,682] jepsen worker 70 - jepsen.util 160 :ok :read [23 28] INFO [2022-01-10 12:53:15,682] jepsen worker 87 - jepsen.util 87 :invoke :write [29 16] INFO [2022-01-10 12:53:15,683] jepsen worker 70 - jepsen.util 160 :invoke :write [23 48] INFO [2022-01-10 12:53:15,684] jepsen worker 13 - jepsen.util 13 :invoke :write [4 24] INFO [2022-01-10 12:53:15,686] jepsen worker 85 - jepsen.util 175 :ok :write [28 2] INFO [2022-01-10 12:53:15,687] jepsen worker 85 - jepsen.util 175 :invoke :write [28 39] INFO [2022-01-10 12:53:15,687] jepsen worker 1 - jepsen.util 1 :ok :write [0 7] INFO [2022-01-10 12:53:15,687] jepsen worker 1 - jepsen.util 1 :invoke :write [0 18] INFO [2022-01-10 12:53:15,688] jepsen worker 62 - jepsen.util 152 :ok :write [20 6] INFO [2022-01-10 12:53:15,689] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,690] jepsen worker 62 - jepsen.util 152 :ok :read [20 6] INFO [2022-01-10 12:53:15,691] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,691] jepsen worker 62 - jepsen.util 152 :ok :read [20 6] INFO [2022-01-10 12:53:15,692] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,693] jepsen worker 62 - jepsen.util 152 :ok :read [20 6] INFO [2022-01-10 12:53:15,693] jepsen worker 4 - jepsen.util 4 :ok :write [1 11] INFO [2022-01-10 12:53:15,694] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:15,694] jepsen worker 62 - jepsen.util 152 :invoke :write [20 47] INFO [2022-01-10 12:53:15,695] jepsen worker 4 - jepsen.util 4 :ok :read [1 11] INFO [2022-01-10 12:53:15,696] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:15,696] jepsen worker 35 - jepsen.util 125 :ok :write [11 28] INFO [2022-01-10 12:53:15,696] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:15,697] jepsen worker 4 - jepsen.util 4 :ok :read [1 11] INFO [2022-01-10 12:53:15,697] jepsen worker 4 - jepsen.util 4 :invoke :write [1 38] INFO [2022-01-10 12:53:15,697] jepsen worker 35 - jepsen.util 125 :ok :read [11 28] INFO [2022-01-10 12:53:15,698] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:15,699] jepsen worker 35 - jepsen.util 125 :ok :read [11 28] INFO [2022-01-10 12:53:15,699] jepsen worker 35 - jepsen.util 125 :invoke :write [11 41] INFO [2022-01-10 12:53:15,700] jepsen worker 38 - jepsen.util 38 :ok :write [12 5] INFO [2022-01-10 12:53:15,701] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:15,702] jepsen worker 39 - jepsen.util 39 :ok :write [13 43] INFO [2022-01-10 12:53:15,702] jepsen worker 38 - jepsen.util 38 :ok :read [12 5] INFO [2022-01-10 12:53:15,702] jepsen worker 39 - jepsen.util 39 :invoke :write [13 25] INFO [2022-01-10 12:53:15,703] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:15,704] jepsen worker 38 - jepsen.util 38 :ok :read [12 5] INFO [2022-01-10 12:53:15,705] jepsen worker 38 - jepsen.util 38 :invoke :write [12 9] INFO [2022-01-10 12:53:15,711] jepsen worker 16 - jepsen.util 106 :ok :write [5 4] INFO [2022-01-10 12:53:15,711] jepsen worker 16 - jepsen.util 106 :invoke :write [5 14] INFO [2022-01-10 12:53:15,718] jepsen worker 66 - jepsen.util 156 :ok :write [22 47] INFO [2022-01-10 12:53:15,718] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,719] jepsen worker 66 - jepsen.util 156 :ok :read [22 47] INFO [2022-01-10 12:53:15,720] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,721] jepsen worker 66 - jepsen.util 156 :ok :read [22 47] INFO [2022-01-10 12:53:15,721] jepsen worker 66 - jepsen.util 156 :invoke :write [22 46] INFO [2022-01-10 12:53:15,724] jepsen worker 49 - jepsen.util 139 :ok :write [16 14] INFO [2022-01-10 12:53:15,724] jepsen worker 49 - jepsen.util 139 :invoke :write [16 30] INFO [2022-01-10 12:53:15,729] jepsen worker 83 - jepsen.util 173 :ok :write [27 4] INFO [2022-01-10 12:53:15,730] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,731] jepsen worker 83 - jepsen.util 173 :ok :read [27 4] INFO [2022-01-10 12:53:15,731] jepsen worker 83 - jepsen.util 173 :invoke :write [27 31] INFO [2022-01-10 12:53:15,735] jepsen worker 8 - jepsen.util 8 :ok :write [2 31] INFO [2022-01-10 12:53:15,736] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,737] jepsen worker 8 - jepsen.util 8 :ok :read [2 31] INFO [2022-01-10 12:53:15,738] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,738] jepsen worker 8 - jepsen.util 8 :ok :read [2 31] INFO [2022-01-10 12:53:15,740] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,741] jepsen worker 8 - jepsen.util 8 :ok :read [2 31] INFO [2022-01-10 12:53:15,741] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,742] jepsen worker 8 - jepsen.util 8 :ok :read [2 31] INFO [2022-01-10 12:53:15,743] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,744] jepsen worker 8 - jepsen.util 8 :ok :read [2 31] INFO [2022-01-10 12:53:15,744] jepsen worker 8 - jepsen.util 8 :invoke :write [2 2] INFO [2022-01-10 12:53:15,747] jepsen worker 35 - jepsen.util 125 :ok :write [11 41] INFO [2022-01-10 12:53:15,748] jepsen worker 35 - jepsen.util 125 :invoke :write [11 22] INFO [2022-01-10 12:53:15,754] jepsen worker 16 - jepsen.util 106 :ok :write [5 14] INFO [2022-01-10 12:53:15,755] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:15,756] jepsen worker 16 - jepsen.util 106 :ok :read [5 14] INFO [2022-01-10 12:53:15,756] jepsen worker 16 - jepsen.util 106 :invoke :write [5 30] INFO [2022-01-10 12:53:15,760] jepsen worker 52 - jepsen.util 52 :ok :write [17 20] INFO [2022-01-10 12:53:15,761] jepsen worker 66 - jepsen.util 156 :ok :write [22 46] INFO [2022-01-10 12:53:15,761] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:15,762] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,762] jepsen worker 52 - jepsen.util 52 :ok :read [17 20] INFO [2022-01-10 12:53:15,762] jepsen worker 66 - jepsen.util 156 :ok :read [22 46] INFO [2022-01-10 12:53:15,763] jepsen worker 52 - jepsen.util 52 :invoke :write [17 46] INFO [2022-01-10 12:53:15,763] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,764] jepsen worker 77 - jepsen.util 167 :ok :write [25 17] INFO [2022-01-10 12:53:15,764] jepsen worker 66 - jepsen.util 156 :ok :read [22 46] INFO [2022-01-10 12:53:15,764] jepsen worker 78 - jepsen.util 78 :ok :write [26 0] INFO [2022-01-10 12:53:15,764] jepsen worker 77 - jepsen.util 167 :invoke :write [25 39] INFO [2022-01-10 12:53:15,765] jepsen worker 66 - jepsen.util 156 :invoke :write [22 37] INFO [2022-01-10 12:53:15,766] jepsen worker 49 - jepsen.util 139 :ok :write [16 30] INFO [2022-01-10 12:53:15,766] jepsen worker 78 - jepsen.util 78 :invoke :write [26 18] INFO [2022-01-10 12:53:15,766] jepsen worker 49 - jepsen.util 139 :invoke :write [16 38] INFO [2022-01-10 12:53:15,770] jepsen worker 13 - jepsen.util 13 :ok :write [4 24] INFO [2022-01-10 12:53:15,770] jepsen worker 23 - jepsen.util 23 :ok :write [7 15] INFO [2022-01-10 12:53:15,770] jepsen worker 13 - jepsen.util 13 :invoke :write [4 40] INFO [2022-01-10 12:53:15,771] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:15,772] jepsen worker 23 - jepsen.util 23 :ok :read [7 15] INFO [2022-01-10 12:53:15,772] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:15,773] jepsen worker 83 - jepsen.util 173 :ok :write [27 31] INFO [2022-01-10 12:53:15,773] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,773] jepsen worker 23 - jepsen.util 23 :ok :read [7 15] INFO [2022-01-10 12:53:15,774] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:15,774] jepsen worker 83 - jepsen.util 173 :ok :read [27 31] INFO [2022-01-10 12:53:15,774] jepsen worker 83 - jepsen.util 173 :invoke :write [27 45] INFO [2022-01-10 12:53:15,775] jepsen worker 23 - jepsen.util 23 :ok :read [7 15] INFO [2022-01-10 12:53:15,775] jepsen worker 23 - jepsen.util 23 :invoke :write [7 0] INFO [2022-01-10 12:53:15,777] jepsen worker 43 - jepsen.util 43 :ok :write [14 43] INFO [2022-01-10 12:53:15,778] jepsen worker 43 - jepsen.util 43 :invoke :write [14 30] INFO [2022-01-10 12:53:15,778] jepsen worker 8 - jepsen.util 8 :ok :write [2 2] INFO [2022-01-10 12:53:15,779] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,779] jepsen worker 70 - jepsen.util 160 :ok :write [23 48] INFO [2022-01-10 12:53:15,779] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,780] jepsen worker 8 - jepsen.util 8 :ok :read [2 2] INFO [2022-01-10 12:53:15,781] jepsen worker 70 - jepsen.util 160 :ok :read [23 48] INFO [2022-01-10 12:53:15,781] jepsen worker 8 - jepsen.util 8 :invoke :write [2 8] INFO [2022-01-10 12:53:15,781] jepsen worker 87 - jepsen.util 87 :ok :write [29 16] INFO [2022-01-10 12:53:15,782] jepsen worker 70 - jepsen.util 160 :invoke :write [23 44] INFO [2022-01-10 12:53:15,783] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:15,783] jepsen worker 35 - jepsen.util 125 :ok :write [11 22] INFO [2022-01-10 12:53:15,783] jepsen worker 35 - jepsen.util 125 :invoke :write [11 47] INFO [2022-01-10 12:53:15,784] jepsen worker 87 - jepsen.util 87 :ok :read [29 16] INFO [2022-01-10 12:53:15,785] jepsen worker 87 - jepsen.util 87 :invoke :write [29 8] INFO [2022-01-10 12:53:15,786] jepsen worker 16 - jepsen.util 106 :ok :write [5 30] INFO [2022-01-10 12:53:15,786] jepsen worker 1 - jepsen.util 1 :ok :write [0 18] INFO [2022-01-10 12:53:15,787] jepsen worker 16 - jepsen.util 106 :invoke :write [5 33] INFO [2022-01-10 12:53:15,787] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:15,788] jepsen worker 1 - jepsen.util 1 :ok :read [0 18] INFO [2022-01-10 12:53:15,789] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:15,790] jepsen worker 1 - jepsen.util 1 :ok :read [0 18] INFO [2022-01-10 12:53:15,790] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:15,791] jepsen worker 85 - jepsen.util 175 :ok :write [28 39] INFO [2022-01-10 12:53:15,791] jepsen worker 62 - jepsen.util 152 :ok :write [20 47] INFO [2022-01-10 12:53:15,791] jepsen worker 1 - jepsen.util 1 :ok :read [0 18] INFO [2022-01-10 12:53:15,792] jepsen worker 66 - jepsen.util 156 :ok :write [22 37] INFO [2022-01-10 12:53:15,793] jepsen worker 85 - jepsen.util 175 :invoke :write [28 21] INFO [2022-01-10 12:53:15,793] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:15,794] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,795] jepsen worker 1 - jepsen.util 1 :ok :read [0 18] INFO [2022-01-10 12:53:15,795] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,795] jepsen worker 1 - jepsen.util 1 :invoke :write [0 27] INFO [2022-01-10 12:53:15,795] jepsen worker 66 - jepsen.util 156 :ok :read [22 37] INFO [2022-01-10 12:53:15,796] jepsen worker 62 - jepsen.util 152 :ok :read [20 47] INFO [2022-01-10 12:53:15,796] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,797] jepsen worker 39 - jepsen.util 39 :ok :write [13 25] INFO [2022-01-10 12:53:15,797] jepsen worker 62 - jepsen.util 152 :invoke :write [20 14] INFO [2022-01-10 12:53:15,797] jepsen worker 4 - jepsen.util 4 :ok :write [1 38] INFO [2022-01-10 12:53:15,798] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,799] jepsen worker 66 - jepsen.util 156 :ok :read [22 37] INFO [2022-01-10 12:53:15,799] jepsen worker 4 - jepsen.util 4 :invoke :write [1 26] INFO [2022-01-10 12:53:15,800] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,802] jepsen worker 77 - jepsen.util 167 :ok :write [25 39] INFO [2022-01-10 12:53:15,802] jepsen worker 66 - jepsen.util 156 :ok :read [22 37] INFO [2022-01-10 12:53:15,802] jepsen worker 39 - jepsen.util 39 :ok :read [13 25] INFO [2022-01-10 12:53:15,802] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:15,803] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,803] jepsen worker 77 - jepsen.util 167 :ok :read [25 39] INFO [2022-01-10 12:53:15,803] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,804] jepsen worker 39 - jepsen.util 39 :ok :read [13 25] INFO [2022-01-10 12:53:15,804] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:15,805] jepsen worker 66 - jepsen.util 156 :ok :read [22 37] INFO [2022-01-10 12:53:15,805] jepsen worker 39 - jepsen.util 39 :invoke :write [13 45] INFO [2022-01-10 12:53:15,805] jepsen worker 77 - jepsen.util 167 :ok :read [25 39] INFO [2022-01-10 12:53:15,805] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,806] jepsen worker 38 - jepsen.util 38 :ok :write [12 9] INFO [2022-01-10 12:53:15,806] jepsen worker 77 - jepsen.util 167 :invoke :write [25 4] INFO [2022-01-10 12:53:15,806] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:15,807] jepsen worker 49 - jepsen.util 139 :ok :write [16 38] INFO [2022-01-10 12:53:15,807] jepsen worker 23 - jepsen.util 23 :ok :write [7 0] INFO [2022-01-10 12:53:15,807] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:15,808] jepsen worker 49 - jepsen.util 139 :invoke :write [16 14] INFO [2022-01-10 12:53:15,808] jepsen worker 66 - jepsen.util 156 :ok :read [22 37] INFO [2022-01-10 12:53:15,808] jepsen worker 38 - jepsen.util 38 :invoke :write [12 27] INFO [2022-01-10 12:53:15,809] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,809] jepsen worker 23 - jepsen.util 23 :invoke :write [7 27] INFO [2022-01-10 12:53:15,810] jepsen worker 66 - jepsen.util 156 :ok :read [22 37] INFO [2022-01-10 12:53:15,810] jepsen worker 66 - jepsen.util 156 :invoke :write [22 49] INFO [2022-01-10 12:53:15,812] jepsen worker 43 - jepsen.util 43 :ok :write [14 30] INFO [2022-01-10 12:53:15,812] jepsen worker 43 - jepsen.util 43 :invoke :write [14 33] INFO [2022-01-10 12:53:15,818] jepsen worker 83 - jepsen.util 173 :ok :write [27 45] INFO [2022-01-10 12:53:15,818] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,819] jepsen worker 52 - jepsen.util 52 :ok :write [17 46] INFO [2022-01-10 12:53:15,819] jepsen worker 52 - jepsen.util 52 :invoke :write [17 1] INFO [2022-01-10 12:53:15,820] jepsen worker 83 - jepsen.util 173 :ok :read [27 45] INFO [2022-01-10 12:53:15,820] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,821] jepsen worker 83 - jepsen.util 173 :ok :read [27 45] INFO [2022-01-10 12:53:15,821] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,822] jepsen worker 83 - jepsen.util 173 :ok :read [27 45] INFO [2022-01-10 12:53:15,823] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,824] jepsen worker 83 - jepsen.util 173 :ok :read [27 45] INFO [2022-01-10 12:53:15,824] jepsen worker 78 - jepsen.util 78 :ok :write [26 18] INFO [2022-01-10 12:53:15,824] jepsen worker 8 - jepsen.util 8 :ok :write [2 8] INFO [2022-01-10 12:53:15,825] jepsen worker 78 - jepsen.util 78 :invoke :write [26 46] INFO [2022-01-10 12:53:15,825] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,826] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,826] jepsen worker 8 - jepsen.util 8 :ok :read [2 8] INFO [2022-01-10 12:53:15,827] jepsen worker 8 - jepsen.util 8 :invoke :write [2 25] INFO [2022-01-10 12:53:15,827] jepsen worker 83 - jepsen.util 173 :ok :read [27 45] INFO [2022-01-10 12:53:15,827] jepsen worker 83 - jepsen.util 173 :invoke :write [27 23] INFO [2022-01-10 12:53:15,828] jepsen worker 13 - jepsen.util 13 :ok :write [4 40] INFO [2022-01-10 12:53:15,829] jepsen worker 13 - jepsen.util 13 :invoke :write [4 47] INFO [2022-01-10 12:53:15,829] jepsen worker 62 - jepsen.util 152 :ok :write [20 14] INFO [2022-01-10 12:53:15,830] jepsen worker 62 - jepsen.util 152 :invoke :write [20 19] INFO [2022-01-10 12:53:15,830] jepsen worker 87 - jepsen.util 87 :ok :write [29 8] INFO [2022-01-10 12:53:15,831] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:15,832] jepsen worker 87 - jepsen.util 87 :ok :read [29 8] INFO [2022-01-10 12:53:15,832] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:15,833] jepsen worker 77 - jepsen.util 167 :ok :write [25 4] INFO [2022-01-10 12:53:15,833] jepsen worker 87 - jepsen.util 87 :ok :read [29 8] INFO [2022-01-10 12:53:15,834] jepsen worker 35 - jepsen.util 125 :ok :write [11 47] INFO [2022-01-10 12:53:15,835] jepsen worker 87 - jepsen.util 87 :invoke :write [29 13] INFO [2022-01-10 12:53:15,836] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:15,836] jepsen worker 77 - jepsen.util 167 :invoke :write [25 24] INFO [2022-01-10 12:53:15,836] jepsen worker 35 - jepsen.util 125 :ok :read [11 47] INFO [2022-01-10 12:53:15,837] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:15,838] jepsen worker 35 - jepsen.util 125 :ok :read [11 47] INFO [2022-01-10 12:53:15,838] jepsen worker 1 - jepsen.util 1 :ok :write [0 27] INFO [2022-01-10 12:53:15,839] jepsen worker 35 - jepsen.util 125 :invoke :write [11 42] INFO [2022-01-10 12:53:15,839] jepsen worker 1 - jepsen.util 1 :invoke :write [0 29] INFO [2022-01-10 12:53:15,841] jepsen worker 70 - jepsen.util 160 :ok :write [23 44] INFO [2022-01-10 12:53:15,841] jepsen worker 23 - jepsen.util 23 :ok :write [7 27] INFO [2022-01-10 12:53:15,841] jepsen worker 70 - jepsen.util 160 :invoke :write [23 4] INFO [2022-01-10 12:53:15,842] jepsen worker 23 - jepsen.util 23 :invoke :write [7 21] INFO [2022-01-10 12:53:15,844] jepsen worker 16 - jepsen.util 106 :ok :write [5 33] INFO [2022-01-10 12:53:15,845] jepsen worker 16 - jepsen.util 106 :invoke :write [5 14] INFO [2022-01-10 12:53:15,847] jepsen worker 43 - jepsen.util 43 :ok :write [14 33] INFO [2022-01-10 12:53:15,848] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:15,849] jepsen worker 43 - jepsen.util 43 :ok :read [14 33] INFO [2022-01-10 12:53:15,849] jepsen worker 43 - jepsen.util 43 :invoke :write [14 19] INFO [2022-01-10 12:53:15,849] jepsen worker 66 - jepsen.util 156 :ok :write [22 49] INFO [2022-01-10 12:53:15,850] jepsen worker 85 - jepsen.util 175 :ok :write [28 21] INFO [2022-01-10 12:53:15,850] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,851] jepsen worker 85 - jepsen.util 175 :invoke :write [28 46] INFO [2022-01-10 12:53:15,851] jepsen worker 66 - jepsen.util 156 :ok :read [22 49] INFO [2022-01-10 12:53:15,852] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,853] jepsen worker 66 - jepsen.util 156 :ok :read [22 49] INFO [2022-01-10 12:53:15,854] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,855] jepsen worker 66 - jepsen.util 156 :ok :read [22 49] INFO [2022-01-10 12:53:15,855] jepsen worker 4 - jepsen.util 4 :ok :write [1 26] INFO [2022-01-10 12:53:15,856] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,856] jepsen worker 4 - jepsen.util 4 :invoke :write [1 15] INFO [2022-01-10 12:53:15,857] jepsen worker 66 - jepsen.util 156 :ok :read [22 49] INFO [2022-01-10 12:53:15,858] jepsen worker 66 - jepsen.util 156 :invoke :write [22 48] INFO [2022-01-10 12:53:15,860] jepsen worker 39 - jepsen.util 39 :ok :write [13 45] INFO [2022-01-10 12:53:15,861] jepsen worker 39 - jepsen.util 39 :invoke :write [13 44] INFO [2022-01-10 12:53:15,865] jepsen worker 0 - jepsen.util 0 :info :write [0 23] :RequestTimeoutException INFO [2022-01-10 12:53:15,865] jepsen worker 38 - jepsen.util 38 :ok :write [12 27] INFO [2022-01-10 12:53:15,866] jepsen worker 62 - jepsen.util 152 :ok :write [20 19] INFO [2022-01-10 12:53:15,866] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:15,866] jepsen worker 49 - jepsen.util 139 :ok :write [16 14] INFO [2022-01-10 12:53:15,867] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:15,867] jepsen worker 0 - jepsen.util 90 :ok :read [0 27] INFO [2022-01-10 12:53:15,867] jepsen worker 62 - jepsen.util 152 :invoke :write [20 24] INFO [2022-01-10 12:53:15,868] jepsen worker 38 - jepsen.util 38 :ok :read [12 27] INFO [2022-01-10 12:53:15,868] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:15,869] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:15,869] jepsen worker 49 - jepsen.util 139 :ok :read [16 14] INFO [2022-01-10 12:53:15,869] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:15,869] jepsen worker 52 - jepsen.util 52 :ok :write [17 1] INFO [2022-01-10 12:53:15,870] jepsen worker 38 - jepsen.util 38 :ok :read [12 27] INFO [2022-01-10 12:53:15,870] jepsen worker 77 - jepsen.util 167 :ok :write [25 24] INFO [2022-01-10 12:53:15,870] jepsen worker 0 - jepsen.util 90 :ok :read [0 27] INFO [2022-01-10 12:53:15,870] jepsen worker 77 - jepsen.util 167 :invoke :write [25 4] INFO [2022-01-10 12:53:15,871] jepsen worker 38 - jepsen.util 38 :invoke :write [12 45] INFO [2022-01-10 12:53:15,872] jepsen worker 49 - jepsen.util 139 :invoke :write [16 5] INFO [2022-01-10 12:53:15,873] jepsen worker 83 - jepsen.util 173 :ok :write [27 23] INFO [2022-01-10 12:53:15,873] jepsen worker 52 - jepsen.util 52 :invoke :write [17 48] INFO [2022-01-10 12:53:15,873] jepsen worker 78 - jepsen.util 78 :ok :write [26 46] INFO [2022-01-10 12:53:15,874] jepsen worker 83 - jepsen.util 173 :invoke :write [27 36] INFO [2022-01-10 12:53:15,874] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:15,875] jepsen worker 78 - jepsen.util 78 :ok :read [26 46] INFO [2022-01-10 12:53:15,875] jepsen worker 0 - jepsen.util 90 :invoke :write [0 4] INFO [2022-01-10 12:53:15,876] jepsen worker 78 - jepsen.util 78 :invoke :write [26 21] INFO [2022-01-10 12:53:15,877] jepsen worker 13 - jepsen.util 13 :ok :write [4 47] INFO [2022-01-10 12:53:15,877] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,878] jepsen worker 13 - jepsen.util 13 :ok :read [4 47] INFO [2022-01-10 12:53:15,879] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:15,880] jepsen worker 13 - jepsen.util 13 :ok :read [4 47] INFO [2022-01-10 12:53:15,880] jepsen worker 13 - jepsen.util 13 :invoke :write [4 40] INFO [2022-01-10 12:53:15,881] jepsen worker 8 - jepsen.util 8 :ok :write [2 25] INFO [2022-01-10 12:53:15,882] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,883] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:15,883] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,884] jepsen worker 70 - jepsen.util 160 :ok :write [23 4] INFO [2022-01-10 12:53:15,884] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,884] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:15,885] jepsen worker 23 - jepsen.util 23 :ok :write [7 21] INFO [2022-01-10 12:53:15,885] jepsen worker 8 - jepsen.util 8 :invoke :write [2 14] INFO [2022-01-10 12:53:15,885] jepsen worker 70 - jepsen.util 160 :ok :read [23 4] INFO [2022-01-10 12:53:15,885] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:15,885] jepsen worker 87 - jepsen.util 87 :ok :write [29 13] INFO [2022-01-10 12:53:15,885] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:15,886] jepsen worker 23 - jepsen.util 23 :ok :read [7 21] INFO [2022-01-10 12:53:15,886] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:15,886] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:15,887] jepsen worker 70 - jepsen.util 160 :ok :read [23 4] INFO [2022-01-10 12:53:15,887] jepsen worker 23 - jepsen.util 23 :ok :read [7 21] INFO [2022-01-10 12:53:15,887] jepsen worker 87 - jepsen.util 87 :ok :read [29 13] INFO [2022-01-10 12:53:15,887] jepsen worker 70 - jepsen.util 160 :invoke :write [23 12] INFO [2022-01-10 12:53:15,888] jepsen worker 23 - jepsen.util 23 :invoke :write [7 4] INFO [2022-01-10 12:53:15,888] jepsen worker 87 - jepsen.util 87 :invoke :write [29 15] INFO [2022-01-10 12:53:15,889] jepsen worker 85 - jepsen.util 175 :ok :write [28 46] INFO [2022-01-10 12:53:15,890] jepsen worker 35 - jepsen.util 125 :ok :write [11 42] INFO [2022-01-10 12:53:15,890] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:15,891] jepsen worker 35 - jepsen.util 125 :invoke :write [11 49] INFO [2022-01-10 12:53:15,891] jepsen worker 85 - jepsen.util 175 :ok :read [28 46] INFO [2022-01-10 12:53:15,891] jepsen worker 4 - jepsen.util 4 :ok :write [1 15] INFO [2022-01-10 12:53:15,891] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:15,892] jepsen worker 4 - jepsen.util 4 :invoke :write [1 21] INFO [2022-01-10 12:53:15,892] jepsen worker 85 - jepsen.util 175 :ok :read [28 46] INFO [2022-01-10 12:53:15,893] jepsen worker 85 - jepsen.util 175 :invoke :write [28 0] INFO [2022-01-10 12:53:15,897] jepsen worker 1 - jepsen.util 1 :ok :write [0 29] INFO [2022-01-10 12:53:15,897] jepsen worker 1 - jepsen.util 1 :invoke :write [0 49] INFO [2022-01-10 12:53:15,899] jepsen worker 38 - jepsen.util 38 :ok :write [12 45] INFO [2022-01-10 12:53:15,899] jepsen worker 38 - jepsen.util 38 :invoke :write [12 40] INFO [2022-01-10 12:53:15,904] jepsen worker 52 - jepsen.util 52 :ok :write [17 48] INFO [2022-01-10 12:53:15,905] jepsen worker 52 - jepsen.util 52 :invoke :write [17 31] INFO [2022-01-10 12:53:15,912] jepsen worker 39 - jepsen.util 39 :ok :write [13 44] INFO [2022-01-10 12:53:15,913] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,914] jepsen worker 39 - jepsen.util 39 :ok :read [13 44] INFO [2022-01-10 12:53:15,915] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,915] jepsen worker 39 - jepsen.util 39 :ok :read [13 44] INFO [2022-01-10 12:53:15,916] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:15,917] jepsen worker 39 - jepsen.util 39 :ok :read [13 44] INFO [2022-01-10 12:53:15,917] jepsen worker 39 - jepsen.util 39 :invoke :write [13 23] INFO [2022-01-10 12:53:15,925] jepsen worker 4 - jepsen.util 4 :ok :write [1 21] INFO [2022-01-10 12:53:15,925] jepsen worker 4 - jepsen.util 4 :invoke :write [1 25] INFO [2022-01-10 12:53:15,930] jepsen worker 85 - jepsen.util 175 :ok :write [28 0] INFO [2022-01-10 12:53:15,931] jepsen worker 85 - jepsen.util 175 :invoke :write [28 43] INFO [2022-01-10 12:53:15,933] jepsen worker 38 - jepsen.util 38 :ok :write [12 40] INFO [2022-01-10 12:53:15,933] jepsen worker 38 - jepsen.util 38 :invoke :write [12 4] INFO [2022-01-10 12:53:15,939] jepsen worker 87 - jepsen.util 87 :ok :write [29 15] INFO [2022-01-10 12:53:15,940] jepsen worker 87 - jepsen.util 87 :invoke :write [29 32] INFO [2022-01-10 12:53:15,940] jepsen worker 4 - jepsen.util 4 :ok :write [1 25] INFO [2022-01-10 12:53:15,941] jepsen worker 4 - jepsen.util 4 :invoke :write [1 38] INFO [2022-01-10 12:53:15,946] jepsen worker 85 - jepsen.util 175 :ok :write [28 43] INFO [2022-01-10 12:53:15,946] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:15,947] jepsen worker 85 - jepsen.util 175 :ok :read [28 43] INFO [2022-01-10 12:53:15,948] jepsen worker 85 - jepsen.util 175 :invoke :write [28 4] INFO [2022-01-10 12:53:15,951] jepsen worker 38 - jepsen.util 38 :ok :write [12 4] INFO [2022-01-10 12:53:15,952] jepsen worker 38 - jepsen.util 38 :invoke :write [12 32] INFO [2022-01-10 12:53:15,952] jepsen worker 13 - jepsen.util 13 :ok :write [4 40] INFO [2022-01-10 12:53:15,953] jepsen worker 13 - jepsen.util 13 :invoke :write [4 49] INFO [2022-01-10 12:53:15,956] jepsen worker 16 - jepsen.util 106 :ok :write [5 14] INFO [2022-01-10 12:53:15,956] jepsen worker 1 - jepsen.util 1 :ok :write [0 49] INFO [2022-01-10 12:53:15,956] jepsen worker 4 - jepsen.util 4 :ok :write [1 38] INFO [2022-01-10 12:53:15,957] jepsen worker 16 - jepsen.util 106 :invoke :write [5 1] INFO [2022-01-10 12:53:15,958] jepsen worker 78 - jepsen.util 78 :ok :write [26 21] INFO [2022-01-10 12:53:15,958] jepsen worker 1 - jepsen.util 1 :invoke :write [0 40] INFO [2022-01-10 12:53:15,958] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:15,959] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:15,959] jepsen worker 4 - jepsen.util 4 :ok :read [1 38] INFO [2022-01-10 12:53:15,960] jepsen worker 78 - jepsen.util 78 :ok :read [26 21] INFO [2022-01-10 12:53:15,960] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:15,960] jepsen worker 43 - jepsen.util 43 :ok :write [14 19] INFO [2022-01-10 12:53:15,961] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:15,961] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:15,961] jepsen worker 4 - jepsen.util 4 :ok :read [1 38] INFO [2022-01-10 12:53:15,962] jepsen worker 85 - jepsen.util 175 :ok :write [28 4] INFO [2022-01-10 12:53:15,962] jepsen worker 78 - jepsen.util 78 :ok :read [26 21] INFO [2022-01-10 12:53:15,962] jepsen worker 43 - jepsen.util 43 :ok :read [14 19] INFO [2022-01-10 12:53:15,963] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:15,964] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:15,964] jepsen worker 70 - jepsen.util 160 :ok :write [23 12] INFO [2022-01-10 12:53:15,964] jepsen worker 4 - jepsen.util 4 :ok :read [1 38] INFO [2022-01-10 12:53:15,965] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:15,965] jepsen worker 43 - jepsen.util 43 :ok :read [14 19] INFO [2022-01-10 12:53:15,965] jepsen worker 85 - jepsen.util 175 :ok :read [28 4] INFO [2022-01-10 12:53:15,966] jepsen worker 66 - jepsen.util 156 :ok :write [22 48] INFO [2022-01-10 12:53:15,966] jepsen worker 43 - jepsen.util 43 :invoke :write [14 34] INFO [2022-01-10 12:53:15,967] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:15,968] jepsen worker 4 - jepsen.util 4 :invoke :write [1 25] INFO [2022-01-10 12:53:15,968] jepsen worker 85 - jepsen.util 175 :ok :read [28 4] INFO [2022-01-10 12:53:15,969] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,969] jepsen worker 66 - jepsen.util 156 :ok :read [22 48] INFO [2022-01-10 12:53:15,969] jepsen worker 78 - jepsen.util 78 :invoke :write [26 28] INFO [2022-01-10 12:53:15,970] jepsen worker 85 - jepsen.util 175 :invoke :write [28 10] INFO [2022-01-10 12:53:15,971] jepsen worker 52 - jepsen.util 52 :ok :write [17 31] INFO [2022-01-10 12:53:15,971] jepsen worker 70 - jepsen.util 160 :invoke :write [23 21] INFO [2022-01-10 12:53:15,972] jepsen worker 52 - jepsen.util 52 :invoke :write [17 4] INFO [2022-01-10 12:53:15,972] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:15,972] jepsen worker 62 - jepsen.util 152 :ok :write [20 24] INFO [2022-01-10 12:53:15,973] jepsen worker 66 - jepsen.util 156 :ok :read [22 48] INFO [2022-01-10 12:53:15,973] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,974] jepsen worker 66 - jepsen.util 156 :invoke :write [22 23] INFO [2022-01-10 12:53:15,974] jepsen worker 62 - jepsen.util 152 :ok :read [20 24] INFO [2022-01-10 12:53:15,975] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,976] jepsen worker 62 - jepsen.util 152 :ok :read [20 24] INFO [2022-01-10 12:53:15,976] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:15,977] jepsen worker 62 - jepsen.util 152 :ok :read [20 24] INFO [2022-01-10 12:53:15,978] jepsen worker 62 - jepsen.util 152 :invoke :write [20 37] INFO [2022-01-10 12:53:15,980] jepsen worker 49 - jepsen.util 139 :ok :write [16 5] INFO [2022-01-10 12:53:15,981] jepsen worker 49 - jepsen.util 139 :invoke :write [16 6] INFO [2022-01-10 12:53:15,987] jepsen worker 77 - jepsen.util 167 :ok :write [25 4] INFO [2022-01-10 12:53:15,987] jepsen worker 77 - jepsen.util 167 :invoke :write [25 16] INFO [2022-01-10 12:53:15,992] jepsen worker 83 - jepsen.util 173 :ok :write [27 36] INFO [2022-01-10 12:53:15,993] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,994] jepsen worker 83 - jepsen.util 173 :ok :read [27 36] INFO [2022-01-10 12:53:15,994] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,995] jepsen worker 83 - jepsen.util 173 :ok :read [27 36] INFO [2022-01-10 12:53:15,995] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,996] jepsen worker 83 - jepsen.util 173 :ok :read [27 36] INFO [2022-01-10 12:53:15,997] jepsen worker 8 - jepsen.util 8 :ok :write [2 14] INFO [2022-01-10 12:53:15,997] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,997] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:15,998] jepsen worker 83 - jepsen.util 173 :ok :read [27 36] INFO [2022-01-10 12:53:15,998] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:15,998] jepsen worker 8 - jepsen.util 8 :ok :read [2 14] INFO [2022-01-10 12:53:15,999] jepsen worker 8 - jepsen.util 8 :invoke :write [2 48] INFO [2022-01-10 12:53:15,999] jepsen worker 83 - jepsen.util 173 :ok :read [27 36] INFO [2022-01-10 12:53:16,000] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:16,001] jepsen worker 83 - jepsen.util 173 :ok :read [27 36] INFO [2022-01-10 12:53:16,001] jepsen worker 83 - jepsen.util 173 :invoke :write [27 9] INFO [2022-01-10 12:53:16,003] jepsen worker 23 - jepsen.util 23 :ok :write [7 4] INFO [2022-01-10 12:53:16,004] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,005] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:16,005] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,006] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:16,007] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,008] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:16,008] jepsen worker 35 - jepsen.util 125 :ok :write [11 49] INFO [2022-01-10 12:53:16,008] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,009] jepsen worker 35 - jepsen.util 125 :invoke :write [11 8] INFO [2022-01-10 12:53:16,009] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:16,010] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,011] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:16,011] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,012] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:16,012] jepsen worker 23 - jepsen.util 23 :invoke :write [7 15] INFO [2022-01-10 12:53:16,063] jepsen worker 38 - jepsen.util 38 :ok :write [12 32] INFO [2022-01-10 12:53:16,064] jepsen worker 39 - jepsen.util 39 :ok :write [13 23] INFO [2022-01-10 12:53:16,064] jepsen worker 38 - jepsen.util 38 :invoke :write [12 30] INFO [2022-01-10 12:53:16,064] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,065] jepsen worker 39 - jepsen.util 39 :ok :read [13 23] INFO [2022-01-10 12:53:16,066] jepsen worker 39 - jepsen.util 39 :invoke :write [13 16] INFO [2022-01-10 12:53:16,066] jepsen worker 13 - jepsen.util 13 :ok :write [4 49] INFO [2022-01-10 12:53:16,068] jepsen worker 13 - jepsen.util 13 :invoke :write [4 28] INFO [2022-01-10 12:53:16,071] jepsen worker 87 - jepsen.util 87 :ok :write [29 32] INFO [2022-01-10 12:53:16,072] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,072] jepsen worker 43 - jepsen.util 43 :ok :write [14 34] INFO [2022-01-10 12:53:16,073] jepsen worker 43 - jepsen.util 43 :invoke :write [14 10] INFO [2022-01-10 12:53:16,073] jepsen worker 87 - jepsen.util 87 :ok :read [29 32] INFO [2022-01-10 12:53:16,074] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,075] jepsen worker 87 - jepsen.util 87 :ok :read [29 32] INFO [2022-01-10 12:53:16,076] jepsen worker 4 - jepsen.util 4 :ok :write [1 25] INFO [2022-01-10 12:53:16,076] jepsen worker 87 - jepsen.util 87 :invoke :write [29 27] INFO [2022-01-10 12:53:16,076] jepsen worker 1 - jepsen.util 1 :ok :write [0 40] INFO [2022-01-10 12:53:16,077] jepsen worker 4 - jepsen.util 4 :invoke :write [1 1] INFO [2022-01-10 12:53:16,078] jepsen worker 1 - jepsen.util 1 :invoke :write [0 35] INFO [2022-01-10 12:53:16,078] jepsen worker 62 - jepsen.util 152 :ok :write [20 37] INFO [2022-01-10 12:53:16,079] jepsen worker 62 - jepsen.util 152 :invoke :write [20 35] INFO [2022-01-10 12:53:16,086] jepsen worker 77 - jepsen.util 167 :ok :write [25 16] INFO [2022-01-10 12:53:16,086] jepsen worker 78 - jepsen.util 78 :ok :write [26 28] INFO [2022-01-10 12:53:16,087] jepsen worker 16 - jepsen.util 106 :ok :write [5 1] INFO [2022-01-10 12:53:16,087] jepsen worker 78 - jepsen.util 78 :invoke :write [26 6] INFO [2022-01-10 12:53:16,088] jepsen worker 77 - jepsen.util 167 :invoke :write [25 41] INFO [2022-01-10 12:53:16,088] jepsen worker 16 - jepsen.util 106 :invoke :write [5 40] INFO [2022-01-10 12:53:16,090] jepsen worker 23 - jepsen.util 23 :ok :write [7 15] INFO [2022-01-10 12:53:16,091] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,092] jepsen worker 23 - jepsen.util 23 :ok :read [7 15] INFO [2022-01-10 12:53:16,092] jepsen worker 66 - jepsen.util 156 :ok :write [22 23] INFO [2022-01-10 12:53:16,093] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,093] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:16,094] jepsen worker 23 - jepsen.util 23 :ok :read [7 15] INFO [2022-01-10 12:53:16,094] jepsen worker 66 - jepsen.util 156 :ok :read [22 23] INFO [2022-01-10 12:53:16,095] jepsen worker 23 - jepsen.util 23 :invoke :write [7 25] INFO [2022-01-10 12:53:16,095] jepsen worker 66 - jepsen.util 156 :invoke :write [22 7] INFO [2022-01-10 12:53:16,097] jepsen worker 49 - jepsen.util 139 :ok :write [16 6] INFO [2022-01-10 12:53:16,097] jepsen worker 85 - jepsen.util 175 :ok :write [28 10] INFO [2022-01-10 12:53:16,098] jepsen worker 85 - jepsen.util 175 :invoke :write [28 32] INFO [2022-01-10 12:53:16,099] jepsen worker 49 - jepsen.util 139 :invoke :write [16 32] INFO [2022-01-10 12:53:16,102] jepsen worker 70 - jepsen.util 160 :ok :write [23 21] INFO [2022-01-10 12:53:16,103] jepsen worker 70 - jepsen.util 160 :invoke :write [23 39] INFO [2022-01-10 12:53:16,104] jepsen worker 43 - jepsen.util 43 :ok :write [14 10] INFO [2022-01-10 12:53:16,105] jepsen worker 83 - jepsen.util 173 :ok :write [27 9] INFO [2022-01-10 12:53:16,105] jepsen worker 43 - jepsen.util 43 :invoke :write [14 26] INFO [2022-01-10 12:53:16,106] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:16,107] jepsen worker 83 - jepsen.util 173 :ok :read [27 9] INFO [2022-01-10 12:53:16,107] jepsen worker 52 - jepsen.util 52 :ok :write [17 4] INFO [2022-01-10 12:53:16,108] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:16,109] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:16,109] jepsen worker 83 - jepsen.util 173 :ok :read [27 9] INFO [2022-01-10 12:53:16,109] jepsen worker 52 - jepsen.util 52 :ok :read [17 4] INFO [2022-01-10 12:53:16,110] jepsen worker 83 - jepsen.util 173 :invoke :write [27 7] INFO [2022-01-10 12:53:16,110] jepsen worker 52 - jepsen.util 52 :invoke :write [17 24] INFO [2022-01-10 12:53:16,112] jepsen worker 38 - jepsen.util 38 :ok :write [12 30] INFO [2022-01-10 12:53:16,113] jepsen worker 38 - jepsen.util 38 :invoke :write [12 23] INFO [2022-01-10 12:53:16,114] jepsen worker 8 - jepsen.util 8 :ok :write [2 48] INFO [2022-01-10 12:53:16,114] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:16,115] jepsen worker 8 - jepsen.util 8 :ok :read [2 48] INFO [2022-01-10 12:53:16,116] jepsen worker 8 - jepsen.util 8 :invoke :write [2 45] INFO [2022-01-10 12:53:16,120] jepsen worker 62 - jepsen.util 152 :ok :write [20 35] INFO [2022-01-10 12:53:16,120] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:16,121] jepsen worker 35 - jepsen.util 125 :ok :write [11 8] INFO [2022-01-10 12:53:16,121] jepsen worker 13 - jepsen.util 13 :ok :write [4 28] INFO [2022-01-10 12:53:16,121] jepsen worker 62 - jepsen.util 152 :ok :read [20 35] INFO [2022-01-10 12:53:16,122] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:16,123] jepsen worker 35 - jepsen.util 125 :ok :read [11 8] INFO [2022-01-10 12:53:16,123] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:16,124] jepsen worker 13 - jepsen.util 13 :ok :read [4 28] INFO [2022-01-10 12:53:16,124] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:16,125] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:16,125] jepsen worker 35 - jepsen.util 125 :ok :read [11 8] INFO [2022-01-10 12:53:16,126] jepsen worker 35 - jepsen.util 125 :invoke :write [11 43] INFO [2022-01-10 12:53:16,126] jepsen worker 13 - jepsen.util 13 :ok :read [4 28] INFO [2022-01-10 12:53:16,127] jepsen worker 62 - jepsen.util 152 :invoke :write [20 46] INFO [2022-01-10 12:53:16,127] jepsen worker 39 - jepsen.util 39 :ok :write [13 16] INFO [2022-01-10 12:53:16,127] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:16,128] jepsen worker 13 - jepsen.util 13 :ok :read [4 28] INFO [2022-01-10 12:53:16,129] jepsen worker 39 - jepsen.util 39 :invoke :write [13 42] INFO [2022-01-10 12:53:16,129] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:16,130] jepsen worker 77 - jepsen.util 167 :ok :write [25 41] INFO [2022-01-10 12:53:16,130] jepsen worker 13 - jepsen.util 13 :ok :read [4 28] INFO [2022-01-10 12:53:16,131] jepsen worker 13 - jepsen.util 13 :invoke :write [4 1] INFO [2022-01-10 12:53:16,132] jepsen worker 77 - jepsen.util 167 :invoke :write [25 13] INFO [2022-01-10 12:53:16,134] jepsen worker 4 - jepsen.util 4 :ok :write [1 1] INFO [2022-01-10 12:53:16,134] jepsen worker 87 - jepsen.util 87 :ok :write [29 27] INFO [2022-01-10 12:53:16,135] jepsen worker 4 - jepsen.util 4 :invoke :write [1 11] INFO [2022-01-10 12:53:16,136] jepsen worker 87 - jepsen.util 87 :invoke :write [29 24] INFO [2022-01-10 12:53:16,140] jepsen worker 78 - jepsen.util 78 :ok :write [26 6] INFO [2022-01-10 12:53:16,140] jepsen worker 78 - jepsen.util 78 :invoke :write [26 40] INFO [2022-01-10 12:53:16,141] jepsen worker 23 - jepsen.util 23 :ok :write [7 25] INFO [2022-01-10 12:53:16,141] jepsen worker 1 - jepsen.util 1 :ok :write [0 35] INFO [2022-01-10 12:53:16,142] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,142] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,142] jepsen worker 1 - jepsen.util 1 :ok :read [0 35] INFO [2022-01-10 12:53:16,143] jepsen worker 23 - jepsen.util 23 :ok :read [7 25] INFO [2022-01-10 12:53:16,143] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,144] jepsen worker 23 - jepsen.util 23 :invoke :write [7 18] INFO [2022-01-10 12:53:16,144] jepsen worker 1 - jepsen.util 1 :ok :read [0 35] INFO [2022-01-10 12:53:16,145] jepsen worker 1 - jepsen.util 1 :invoke :write [0 23] INFO [2022-01-10 12:53:16,148] jepsen worker 16 - jepsen.util 106 :ok :write [5 40] INFO [2022-01-10 12:53:16,148] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:16,149] jepsen worker 16 - jepsen.util 106 :ok :read [5 40] INFO [2022-01-10 12:53:16,150] jepsen worker 85 - jepsen.util 175 :ok :write [28 32] INFO [2022-01-10 12:53:16,150] jepsen worker 16 - jepsen.util 106 :invoke :write [5 39] INFO [2022-01-10 12:53:16,150] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:16,151] jepsen worker 85 - jepsen.util 175 :ok :read [28 32] INFO [2022-01-10 12:53:16,152] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:16,152] jepsen worker 85 - jepsen.util 175 :ok :read [28 32] INFO [2022-01-10 12:53:16,153] jepsen worker 85 - jepsen.util 175 :invoke :write [28 15] INFO [2022-01-10 12:53:16,155] jepsen worker 66 - jepsen.util 156 :ok :write [22 7] INFO [2022-01-10 12:53:16,155] jepsen worker 43 - jepsen.util 43 :ok :write [14 26] INFO [2022-01-10 12:53:16,156] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:16,156] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:16,156] jepsen worker 66 - jepsen.util 156 :ok :read [22 7] INFO [2022-01-10 12:53:16,156] jepsen worker 70 - jepsen.util 160 :ok :write [23 39] INFO [2022-01-10 12:53:16,157] jepsen worker 43 - jepsen.util 43 :ok :read [14 26] INFO [2022-01-10 12:53:16,157] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:16,158] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:16,158] jepsen worker 70 - jepsen.util 160 :ok :read [23 39] INFO [2022-01-10 12:53:16,158] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:16,158] jepsen worker 66 - jepsen.util 156 :ok :read [22 7] INFO [2022-01-10 12:53:16,159] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:16,159] jepsen worker 43 - jepsen.util 43 :ok :read [14 26] INFO [2022-01-10 12:53:16,159] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:16,160] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:16,160] jepsen worker 70 - jepsen.util 160 :ok :read [23 39] INFO [2022-01-10 12:53:16,160] jepsen worker 66 - jepsen.util 156 :ok :read [22 7] INFO [2022-01-10 12:53:16,161] jepsen worker 43 - jepsen.util 43 :ok :read [14 26] INFO [2022-01-10 12:53:16,161] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:16,161] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:16,162] jepsen worker 70 - jepsen.util 160 :ok :read [23 39] INFO [2022-01-10 12:53:16,162] jepsen worker 49 - jepsen.util 139 :ok :write [16 32] INFO [2022-01-10 12:53:16,162] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:16,162] jepsen worker 66 - jepsen.util 156 :ok :read [22 7] INFO [2022-01-10 12:53:16,163] jepsen worker 66 - jepsen.util 156 :invoke :write [22 15] INFO [2022-01-10 12:53:16,163] jepsen worker 43 - jepsen.util 43 :ok :read [14 26] INFO [2022-01-10 12:53:16,163] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:16,164] jepsen worker 43 - jepsen.util 43 :invoke :write [14 40] INFO [2022-01-10 12:53:16,164] jepsen worker 49 - jepsen.util 139 :ok :read [16 32] INFO [2022-01-10 12:53:16,164] jepsen worker 52 - jepsen.util 52 :ok :write [17 24] INFO [2022-01-10 12:53:16,164] jepsen worker 70 - jepsen.util 160 :invoke :write [23 38] INFO [2022-01-10 12:53:16,165] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:16,166] jepsen worker 52 - jepsen.util 52 :invoke :write [17 16] INFO [2022-01-10 12:53:16,166] jepsen worker 49 - jepsen.util 139 :ok :read [16 32] INFO [2022-01-10 12:53:16,167] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:16,168] jepsen worker 49 - jepsen.util 139 :ok :read [16 32] INFO [2022-01-10 12:53:16,169] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:16,169] jepsen worker 83 - jepsen.util 173 :ok :write [27 7] INFO [2022-01-10 12:53:16,169] jepsen worker 49 - jepsen.util 139 :ok :read [16 32] INFO [2022-01-10 12:53:16,171] jepsen worker 83 - jepsen.util 173 :invoke :write [27 27] INFO [2022-01-10 12:53:16,171] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:16,172] jepsen worker 49 - jepsen.util 139 :ok :read [16 32] INFO [2022-01-10 12:53:16,173] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:16,173] jepsen worker 62 - jepsen.util 152 :ok :write [20 46] INFO [2022-01-10 12:53:16,174] jepsen worker 38 - jepsen.util 38 :ok :write [12 23] INFO [2022-01-10 12:53:16,174] jepsen worker 49 - jepsen.util 139 :ok :read [16 32] INFO [2022-01-10 12:53:16,174] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:16,175] jepsen worker 8 - jepsen.util 8 :ok :write [2 45] INFO [2022-01-10 12:53:16,175] jepsen worker 38 - jepsen.util 38 :invoke :write [12 48] INFO [2022-01-10 12:53:16,176] jepsen worker 62 - jepsen.util 152 :ok :read [20 46] INFO [2022-01-10 12:53:16,176] jepsen worker 8 - jepsen.util 8 :invoke :write [2 15] INFO [2022-01-10 12:53:16,177] jepsen worker 49 - jepsen.util 139 :invoke :write [16 27] INFO [2022-01-10 12:53:16,178] jepsen worker 62 - jepsen.util 152 :invoke :write [20 11] INFO [2022-01-10 12:53:16,180] jepsen worker 35 - jepsen.util 125 :ok :write [11 43] INFO [2022-01-10 12:53:16,181] jepsen worker 35 - jepsen.util 125 :invoke :write [11 1] INFO [2022-01-10 12:53:16,182] jepsen worker 13 - jepsen.util 13 :ok :write [4 1] INFO [2022-01-10 12:53:16,182] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:16,183] jepsen worker 13 - jepsen.util 13 :ok :read [4 1] INFO [2022-01-10 12:53:16,184] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:16,185] jepsen worker 77 - jepsen.util 167 :ok :write [25 13] INFO [2022-01-10 12:53:16,185] jepsen worker 13 - jepsen.util 13 :ok :read [4 1] INFO [2022-01-10 12:53:16,186] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:16,186] jepsen worker 39 - jepsen.util 39 :ok :write [13 42] INFO [2022-01-10 12:53:16,186] jepsen worker 13 - jepsen.util 13 :invoke :write [4 7] INFO [2022-01-10 12:53:16,187] jepsen worker 77 - jepsen.util 167 :ok :read [25 13] INFO [2022-01-10 12:53:16,187] jepsen worker 77 - jepsen.util 167 :invoke :write [25 4] INFO [2022-01-10 12:53:16,188] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,189] jepsen worker 39 - jepsen.util 39 :ok :read [13 42] INFO [2022-01-10 12:53:16,190] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,191] jepsen worker 39 - jepsen.util 39 :ok :read [13 42] INFO [2022-01-10 12:53:16,191] jepsen worker 39 - jepsen.util 39 :invoke :write [13 33] INFO [2022-01-10 12:53:16,191] jepsen worker 23 - jepsen.util 23 :ok :write [7 18] INFO [2022-01-10 12:53:16,192] jepsen worker 4 - jepsen.util 4 :ok :write [1 11] INFO [2022-01-10 12:53:16,192] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,192] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,193] jepsen worker 23 - jepsen.util 23 :ok :read [7 18] INFO [2022-01-10 12:53:16,193] jepsen worker 4 - jepsen.util 4 :ok :read [1 11] INFO [2022-01-10 12:53:16,194] jepsen worker 87 - jepsen.util 87 :ok :write [29 24] INFO [2022-01-10 12:53:16,194] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,194] jepsen worker 4 - jepsen.util 4 :invoke :write [1 14] INFO [2022-01-10 12:53:16,194] jepsen worker 23 - jepsen.util 23 :ok :read [7 18] INFO [2022-01-10 12:53:16,195] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,195] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,196] jepsen worker 87 - jepsen.util 87 :ok :read [29 24] INFO [2022-01-10 12:53:16,196] jepsen worker 23 - jepsen.util 23 :ok :read [7 18] INFO [2022-01-10 12:53:16,196] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,197] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,197] jepsen worker 87 - jepsen.util 87 :ok :read [29 24] INFO [2022-01-10 12:53:16,198] jepsen worker 23 - jepsen.util 23 :ok :read [7 18] INFO [2022-01-10 12:53:16,198] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,198] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,199] jepsen worker 87 - jepsen.util 87 :ok :read [29 24] INFO [2022-01-10 12:53:16,199] jepsen worker 1 - jepsen.util 1 :ok :write [0 23] INFO [2022-01-10 12:53:16,199] jepsen worker 23 - jepsen.util 23 :ok :read [7 18] INFO [2022-01-10 12:53:16,200] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,200] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,200] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,200] jepsen worker 87 - jepsen.util 87 :ok :read [29 24] INFO [2022-01-10 12:53:16,201] jepsen worker 1 - jepsen.util 1 :ok :read [0 23] INFO [2022-01-10 12:53:16,201] jepsen worker 23 - jepsen.util 23 :ok :read [7 18] INFO [2022-01-10 12:53:16,201] jepsen worker 87 - jepsen.util 87 :invoke :write [29 15] INFO [2022-01-10 12:53:16,202] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,202] jepsen worker 43 - jepsen.util 43 :ok :write [14 40] INFO [2022-01-10 12:53:16,203] jepsen worker 1 - jepsen.util 1 :invoke :write [0 21] INFO [2022-01-10 12:53:16,203] jepsen worker 23 - jepsen.util 23 :ok :read [7 18] INFO [2022-01-10 12:53:16,203] jepsen worker 43 - jepsen.util 43 :invoke :write [14 37] INFO [2022-01-10 12:53:16,204] jepsen worker 23 - jepsen.util 23 :invoke :write [7 43] INFO [2022-01-10 12:53:16,204] jepsen worker 78 - jepsen.util 78 :ok :write [26 40] INFO [2022-01-10 12:53:16,205] jepsen worker 78 - jepsen.util 78 :invoke :write [26 15] INFO [2022-01-10 12:53:16,207] jepsen worker 16 - jepsen.util 106 :ok :write [5 39] INFO [2022-01-10 12:53:16,207] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:16,208] jepsen worker 16 - jepsen.util 106 :ok :read [5 39] INFO [2022-01-10 12:53:16,209] jepsen worker 16 - jepsen.util 106 :invoke :write [5 31] INFO [2022-01-10 12:53:16,210] jepsen worker 85 - jepsen.util 175 :ok :write [28 15] INFO [2022-01-10 12:53:16,211] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:16,212] jepsen worker 85 - jepsen.util 175 :ok :read [28 15] INFO [2022-01-10 12:53:16,212] jepsen worker 85 - jepsen.util 175 :invoke :write [28 49] INFO [2022-01-10 12:53:16,215] jepsen worker 70 - jepsen.util 160 :ok :write [23 38] INFO [2022-01-10 12:53:16,215] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:16,216] jepsen worker 70 - jepsen.util 160 :ok :read [23 38] INFO [2022-01-10 12:53:16,216] jepsen worker 66 - jepsen.util 156 :ok :write [22 15] INFO [2022-01-10 12:53:16,217] jepsen worker 70 - jepsen.util 160 :invoke :write [23 39] INFO [2022-01-10 12:53:16,217] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:16,218] jepsen worker 66 - jepsen.util 156 :ok :read [22 15] INFO [2022-01-10 12:53:16,219] jepsen worker 66 - jepsen.util 156 :invoke :write [22 45] INFO [2022-01-10 12:53:16,220] jepsen worker 52 - jepsen.util 52 :ok :write [17 16] INFO [2022-01-10 12:53:16,221] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:16,222] jepsen worker 52 - jepsen.util 52 :ok :read [17 16] INFO [2022-01-10 12:53:16,223] jepsen worker 52 - jepsen.util 52 :invoke :write [17 7] INFO [2022-01-10 12:53:16,224] jepsen worker 49 - jepsen.util 139 :ok :write [16 27] INFO [2022-01-10 12:53:16,225] jepsen worker 49 - jepsen.util 139 :invoke :write [16 38] INFO [2022-01-10 12:53:16,226] jepsen worker 38 - jepsen.util 38 :ok :write [12 48] INFO [2022-01-10 12:53:16,227] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,228] jepsen worker 38 - jepsen.util 38 :ok :read [12 48] INFO [2022-01-10 12:53:16,229] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,229] jepsen worker 83 - jepsen.util 173 :ok :write [27 27] INFO [2022-01-10 12:53:16,230] jepsen worker 38 - jepsen.util 38 :ok :read [12 48] INFO [2022-01-10 12:53:16,230] jepsen worker 83 - jepsen.util 173 :invoke :write [27 48] INFO [2022-01-10 12:53:16,231] jepsen worker 62 - jepsen.util 152 :ok :write [20 11] INFO [2022-01-10 12:53:16,231] jepsen worker 38 - jepsen.util 38 :invoke :write [12 31] INFO [2022-01-10 12:53:16,232] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:16,233] jepsen worker 62 - jepsen.util 152 :ok :read [20 11] INFO [2022-01-10 12:53:16,233] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:16,234] jepsen worker 62 - jepsen.util 152 :ok :read [20 11] INFO [2022-01-10 12:53:16,234] jepsen worker 13 - jepsen.util 13 :ok :write [4 7] INFO [2022-01-10 12:53:16,235] jepsen worker 13 - jepsen.util 13 :invoke :write [4 47] INFO [2022-01-10 12:53:16,235] jepsen worker 62 - jepsen.util 152 :invoke :write [20 14] INFO [2022-01-10 12:53:16,239] jepsen worker 8 - jepsen.util 8 :ok :write [2 15] INFO [2022-01-10 12:53:16,240] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:16,241] jepsen worker 8 - jepsen.util 8 :ok :read [2 15] INFO [2022-01-10 12:53:16,241] jepsen worker 4 - jepsen.util 4 :ok :write [1 14] INFO [2022-01-10 12:53:16,241] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:16,242] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,242] jepsen worker 8 - jepsen.util 8 :ok :read [2 15] INFO [2022-01-10 12:53:16,243] jepsen worker 4 - jepsen.util 4 :ok :read [1 14] INFO [2022-01-10 12:53:16,243] jepsen worker 8 - jepsen.util 8 :invoke :write [2 23] INFO [2022-01-10 12:53:16,243] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,244] jepsen worker 4 - jepsen.util 4 :ok :read [1 14] INFO [2022-01-10 12:53:16,245] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,246] jepsen worker 4 - jepsen.util 4 :ok :read [1 14] INFO [2022-01-10 12:53:16,246] jepsen worker 35 - jepsen.util 125 :ok :write [11 1] INFO [2022-01-10 12:53:16,246] jepsen worker 4 - jepsen.util 4 :invoke :write [1 3] INFO [2022-01-10 12:53:16,247] jepsen worker 78 - jepsen.util 78 :ok :write [26 15] INFO [2022-01-10 12:53:16,247] jepsen worker 35 - jepsen.util 125 :invoke :write [11 39] INFO [2022-01-10 12:53:16,247] jepsen worker 78 - jepsen.util 78 :invoke :write [26 16] INFO [2022-01-10 12:53:16,253] jepsen worker 39 - jepsen.util 39 :ok :write [13 33] INFO [2022-01-10 12:53:16,253] jepsen worker 85 - jepsen.util 175 :ok :write [28 49] INFO [2022-01-10 12:53:16,253] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,254] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:16,254] jepsen worker 39 - jepsen.util 39 :ok :read [13 33] INFO [2022-01-10 12:53:16,255] jepsen worker 85 - jepsen.util 175 :ok :read [28 49] INFO [2022-01-10 12:53:16,255] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,255] jepsen worker 85 - jepsen.util 175 :invoke :write [28 33] INFO [2022-01-10 12:53:16,256] jepsen worker 39 - jepsen.util 39 :ok :read [13 33] INFO [2022-01-10 12:53:16,256] jepsen worker 39 - jepsen.util 39 :invoke :write [13 34] INFO [2022-01-10 12:53:16,258] jepsen worker 70 - jepsen.util 160 :ok :write [23 39] INFO [2022-01-10 12:53:16,258] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:16,259] jepsen worker 70 - jepsen.util 160 :ok :read [23 39] INFO [2022-01-10 12:53:16,259] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:16,260] jepsen worker 70 - jepsen.util 160 :ok :read [23 39] INFO [2022-01-10 12:53:16,261] jepsen worker 70 - jepsen.util 160 :invoke :write [23 47] INFO [2022-01-10 12:53:16,261] jepsen worker 87 - jepsen.util 87 :ok :write [29 15] INFO [2022-01-10 12:53:16,262] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,263] jepsen worker 87 - jepsen.util 87 :ok :read [29 15] INFO [2022-01-10 12:53:16,263] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,264] jepsen worker 87 - jepsen.util 87 :ok :read [29 15] INFO [2022-01-10 12:53:16,265] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,265] jepsen worker 87 - jepsen.util 87 :ok :read [29 15] INFO [2022-01-10 12:53:16,266] jepsen worker 1 - jepsen.util 1 :ok :write [0 21] INFO [2022-01-10 12:53:16,266] jepsen worker 87 - jepsen.util 87 :invoke :write [29 12] INFO [2022-01-10 12:53:16,266] jepsen worker 52 - jepsen.util 52 :ok :write [17 7] INFO [2022-01-10 12:53:16,267] jepsen worker 1 - jepsen.util 1 :invoke :write [0 41] INFO [2022-01-10 12:53:16,267] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:16,268] jepsen worker 52 - jepsen.util 52 :ok :read [17 7] INFO [2022-01-10 12:53:16,269] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:16,269] jepsen worker 52 - jepsen.util 52 :ok :read [17 7] INFO [2022-01-10 12:53:16,270] jepsen worker 52 - jepsen.util 52 :invoke :write [17 2] INFO [2022-01-10 12:53:16,274] jepsen worker 36 - jepsen.util 36 :ok :write [12 48] INFO [2022-01-10 12:53:16,274] jepsen worker 36 - jepsen.util 36 :invoke :write [12 20] INFO [2022-01-10 12:53:16,283] jepsen worker 4 - jepsen.util 4 :ok :write [1 3] INFO [2022-01-10 12:53:16,284] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,285] jepsen worker 4 - jepsen.util 4 :ok :read [1 3] INFO [2022-01-10 12:53:16,285] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,286] jepsen worker 4 - jepsen.util 4 :ok :read [1 3] INFO [2022-01-10 12:53:16,287] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,287] jepsen worker 4 - jepsen.util 4 :ok :read [1 3] INFO [2022-01-10 12:53:16,288] jepsen worker 4 - jepsen.util 4 :invoke :write [1 5] INFO [2022-01-10 12:53:16,292] jepsen worker 85 - jepsen.util 175 :ok :write [28 33] INFO [2022-01-10 12:53:16,293] jepsen worker 85 - jepsen.util 175 :invoke :write [28 30] INFO [2022-01-10 12:53:16,298] jepsen worker 39 - jepsen.util 39 :ok :write [13 34] INFO [2022-01-10 12:53:16,299] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,300] jepsen worker 39 - jepsen.util 39 :ok :read [13 34] INFO [2022-01-10 12:53:16,300] jepsen worker 39 - jepsen.util 39 :invoke :write [13 44] INFO [2022-01-10 12:53:16,303] jepsen worker 38 - jepsen.util 38 :ok :write [12 31] INFO [2022-01-10 12:53:16,304] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,305] jepsen worker 38 - jepsen.util 38 :ok :read [12 31] INFO [2022-01-10 12:53:16,305] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,306] jepsen worker 87 - jepsen.util 87 :ok :write [29 12] INFO [2022-01-10 12:53:16,306] jepsen worker 38 - jepsen.util 38 :ok :read [12 31] INFO [2022-01-10 12:53:16,306] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,307] jepsen worker 38 - jepsen.util 38 :invoke :write [12 29] INFO [2022-01-10 12:53:16,307] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:16,308] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,309] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:16,309] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,310] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:16,310] jepsen worker 1 - jepsen.util 1 :ok :write [0 41] INFO [2022-01-10 12:53:16,310] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,311] jepsen worker 1 - jepsen.util 1 :invoke :write [0 0] INFO [2022-01-10 12:53:16,311] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:16,312] jepsen worker 87 - jepsen.util 87 :invoke :write [29 12] INFO [2022-01-10 12:53:16,358] jepsen worker 13 - jepsen.util 13 :ok :write [4 47] INFO [2022-01-10 12:53:16,359] jepsen worker 13 - jepsen.util 13 :invoke :write [4 6] INFO [2022-01-10 12:53:16,360] jepsen worker 77 - jepsen.util 167 :ok :write [25 4] INFO [2022-01-10 12:53:16,360] jepsen worker 77 - jepsen.util 167 :invoke :write [25 5] INFO [2022-01-10 12:53:16,363] jepsen worker 78 - jepsen.util 78 :ok :write [26 16] INFO [2022-01-10 12:53:16,363] jepsen worker 78 - jepsen.util 78 :invoke :write [26 28] INFO [2022-01-10 12:53:16,367] jepsen worker 70 - jepsen.util 160 :ok :write [23 47] INFO [2022-01-10 12:53:16,367] jepsen worker 70 - jepsen.util 160 :invoke :write [23 16] INFO [2022-01-10 12:53:16,368] jepsen worker 23 - jepsen.util 23 :ok :write [7 43] INFO [2022-01-10 12:53:16,368] jepsen worker 23 - jepsen.util 23 :invoke :write [7 13] INFO [2022-01-10 12:53:16,374] jepsen worker 43 - jepsen.util 43 :ok :write [14 37] INFO [2022-01-10 12:53:16,374] jepsen worker 52 - jepsen.util 52 :ok :write [17 2] INFO [2022-01-10 12:53:16,374] jepsen worker 43 - jepsen.util 43 :invoke :write [14 4] INFO [2022-01-10 12:53:16,375] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:16,375] jepsen worker 52 - jepsen.util 52 :ok :read [17 2] INFO [2022-01-10 12:53:16,376] jepsen worker 52 - jepsen.util 52 :invoke :write [17 26] INFO [2022-01-10 12:53:16,379] jepsen worker 16 - jepsen.util 106 :ok :write [5 31] INFO [2022-01-10 12:53:16,380] jepsen worker 16 - jepsen.util 106 :invoke :write [5 33] INFO [2022-01-10 12:53:16,384] jepsen worker 66 - jepsen.util 156 :ok :write [22 45] INFO [2022-01-10 12:53:16,385] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:16,386] jepsen worker 66 - jepsen.util 156 :ok :read [22 45] INFO [2022-01-10 12:53:16,387] jepsen worker 66 - jepsen.util 156 :invoke :write [22 31] INFO [2022-01-10 12:53:16,390] jepsen worker 49 - jepsen.util 139 :ok :write [16 38] INFO [2022-01-10 12:53:16,391] jepsen worker 49 - jepsen.util 139 :invoke :write [16 23] INFO [2022-01-10 12:53:16,395] jepsen worker 62 - jepsen.util 152 :ok :write [20 14] INFO [2022-01-10 12:53:16,395] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:16,396] jepsen worker 62 - jepsen.util 152 :ok :read [20 14] INFO [2022-01-10 12:53:16,396] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:16,397] jepsen worker 62 - jepsen.util 152 :ok :read [20 14] INFO [2022-01-10 12:53:16,398] jepsen worker 62 - jepsen.util 152 :invoke :write [20 46] INFO [2022-01-10 12:53:16,400] jepsen worker 83 - jepsen.util 173 :ok :write [27 48] INFO [2022-01-10 12:53:16,400] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:16,401] jepsen worker 83 - jepsen.util 173 :ok :read [27 48] INFO [2022-01-10 12:53:16,402] jepsen worker 83 - jepsen.util 173 :invoke :write [27 3] INFO [2022-01-10 12:53:16,403] jepsen worker 8 - jepsen.util 8 :ok :write [2 23] INFO [2022-01-10 12:53:16,404] jepsen worker 8 - jepsen.util 8 :invoke :write [2 32] INFO [2022-01-10 12:53:16,408] jepsen worker 35 - jepsen.util 125 :ok :write [11 39] INFO [2022-01-10 12:53:16,409] jepsen worker 35 - jepsen.util 125 :invoke :write [11 32] INFO [2022-01-10 12:53:16,426] jepsen worker 16 - jepsen.util 106 :ok :write [5 33] INFO [2022-01-10 12:53:16,426] jepsen worker 16 - jepsen.util 106 :invoke :write [5 19] INFO [2022-01-10 12:53:16,430] jepsen worker 4 - jepsen.util 4 :ok :write [1 5] INFO [2022-01-10 12:53:16,431] jepsen worker 4 - jepsen.util 4 :invoke :write [1 6] INFO [2022-01-10 12:53:16,432] jepsen worker 66 - jepsen.util 156 :ok :write [22 31] INFO [2022-01-10 12:53:16,432] jepsen worker 66 - jepsen.util 156 :invoke :write [22 27] INFO [2022-01-10 12:53:16,437] jepsen worker 39 - jepsen.util 39 :ok :write [13 44] INFO [2022-01-10 12:53:16,437] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,438] jepsen worker 85 - jepsen.util 175 :ok :write [28 30] INFO [2022-01-10 12:53:16,439] jepsen worker 39 - jepsen.util 39 :ok :read [13 44] INFO [2022-01-10 12:53:16,439] jepsen worker 85 - jepsen.util 175 :invoke :write [28 39] INFO [2022-01-10 12:53:16,439] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,440] jepsen worker 87 - jepsen.util 87 :ok :write [29 12] INFO [2022-01-10 12:53:16,440] jepsen worker 39 - jepsen.util 39 :ok :read [13 44] INFO [2022-01-10 12:53:16,440] jepsen worker 87 - jepsen.util 87 :invoke :write [29 24] INFO [2022-01-10 12:53:16,441] jepsen worker 39 - jepsen.util 39 :invoke :write [13 43] INFO [2022-01-10 12:53:16,443] jepsen worker 36 - jepsen.util 36 :ok :write [12 20] INFO [2022-01-10 12:53:16,444] jepsen worker 36 - jepsen.util 36 :invoke :write [12 33] INFO [2022-01-10 12:53:16,449] jepsen worker 13 - jepsen.util 13 :ok :write [4 6] INFO [2022-01-10 12:53:16,449] jepsen worker 1 - jepsen.util 1 :ok :write [0 0] INFO [2022-01-10 12:53:16,449] jepsen worker 13 - jepsen.util 13 :invoke :write [4 3] INFO [2022-01-10 12:53:16,450] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,450] jepsen worker 1 - jepsen.util 1 :ok :read [0 0] INFO [2022-01-10 12:53:16,451] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,452] jepsen worker 1 - jepsen.util 1 :ok :read [0 0] INFO [2022-01-10 12:53:16,452] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,453] jepsen worker 1 - jepsen.util 1 :ok :read [0 0] INFO [2022-01-10 12:53:16,453] jepsen worker 77 - jepsen.util 167 :ok :write [25 5] INFO [2022-01-10 12:53:16,453] jepsen worker 1 - jepsen.util 1 :invoke :write [0 35] INFO [2022-01-10 12:53:16,454] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:16,455] jepsen worker 77 - jepsen.util 167 :ok :read [25 5] INFO [2022-01-10 12:53:16,455] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:16,456] jepsen worker 77 - jepsen.util 167 :ok :read [25 5] INFO [2022-01-10 12:53:16,456] jepsen worker 77 - jepsen.util 167 :invoke :write [25 0] INFO [2022-01-10 12:53:16,457] jepsen worker 78 - jepsen.util 78 :ok :write [26 28] INFO [2022-01-10 12:53:16,458] jepsen worker 78 - jepsen.util 78 :invoke :write [26 25] INFO [2022-01-10 12:53:16,459] jepsen worker 23 - jepsen.util 23 :ok :write [7 13] INFO [2022-01-10 12:53:16,459] jepsen worker 23 - jepsen.util 23 :invoke :write [7 49] INFO [2022-01-10 12:53:16,466] jepsen worker 43 - jepsen.util 43 :ok :write [14 4] INFO [2022-01-10 12:53:16,466] jepsen worker 70 - jepsen.util 160 :ok :write [23 16] INFO [2022-01-10 12:53:16,466] jepsen worker 70 - jepsen.util 160 :invoke :write [23 32] INFO [2022-01-10 12:53:16,467] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:16,468] jepsen worker 43 - jepsen.util 43 :ok :read [14 4] INFO [2022-01-10 12:53:16,468] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:16,470] jepsen worker 43 - jepsen.util 43 :ok :read [14 4] INFO [2022-01-10 12:53:16,470] jepsen worker 43 - jepsen.util 43 :invoke :write [14 20] INFO [2022-01-10 12:53:16,472] jepsen worker 62 - jepsen.util 152 :ok :write [20 46] INFO [2022-01-10 12:53:16,473] jepsen worker 52 - jepsen.util 52 :ok :write [17 26] INFO [2022-01-10 12:53:16,473] jepsen worker 62 - jepsen.util 152 :invoke :write [20 20] INFO [2022-01-10 12:53:16,473] jepsen worker 52 - jepsen.util 52 :invoke :write [17 39] INFO [2022-01-10 12:53:16,495] jepsen worker 77 - jepsen.util 167 :ok :write [25 0] INFO [2022-01-10 12:53:16,496] jepsen worker 13 - jepsen.util 13 :ok :write [4 3] INFO [2022-01-10 12:53:16,496] jepsen worker 77 - jepsen.util 167 :invoke :write [25 45] INFO [2022-01-10 12:53:16,497] jepsen worker 13 - jepsen.util 13 :invoke :write [4 19] INFO [2022-01-10 12:53:16,500] jepsen worker 23 - jepsen.util 23 :ok :write [7 49] INFO [2022-01-10 12:53:16,501] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,502] jepsen worker 23 - jepsen.util 23 :ok :read [7 49] INFO [2022-01-10 12:53:16,502] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,503] jepsen worker 23 - jepsen.util 23 :ok :read [7 49] INFO [2022-01-10 12:53:16,504] jepsen worker 78 - jepsen.util 78 :ok :write [26 25] INFO [2022-01-10 12:53:16,504] jepsen worker 23 - jepsen.util 23 :invoke :write [7 23] INFO [2022-01-10 12:53:16,504] jepsen worker 78 - jepsen.util 78 :invoke :write [26 5] INFO [2022-01-10 12:53:16,505] jepsen worker 44 - jepsen.util 44 :ok :write [14 2] INFO [2022-01-10 12:53:16,506] jepsen worker 44 - jepsen.util 44 :invoke :write [14 38] INFO [2022-01-10 12:53:16,511] jepsen worker 70 - jepsen.util 160 :ok :write [23 32] INFO [2022-01-10 12:53:16,511] jepsen worker 70 - jepsen.util 160 :invoke :write [23 48] INFO [2022-01-10 12:53:16,513] jepsen worker 62 - jepsen.util 152 :ok :write [20 20] INFO [2022-01-10 12:53:16,513] jepsen worker 62 - jepsen.util 152 :invoke :write [20 29] INFO [2022-01-10 12:53:16,517] jepsen worker 49 - jepsen.util 139 :ok :write [16 23] INFO [2022-01-10 12:53:16,517] jepsen worker 49 - jepsen.util 139 :invoke :write [16 14] INFO [2022-01-10 12:53:16,518] jepsen worker 52 - jepsen.util 52 :ok :write [17 39] INFO [2022-01-10 12:53:16,518] jepsen worker 4 - jepsen.util 4 :ok :write [1 6] INFO [2022-01-10 12:53:16,519] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:16,519] jepsen worker 4 - jepsen.util 4 :invoke :write [1 38] INFO [2022-01-10 12:53:16,520] jepsen worker 52 - jepsen.util 52 :ok :read [17 39] INFO [2022-01-10 12:53:16,520] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:16,521] jepsen worker 77 - jepsen.util 167 :ok :write [25 45] INFO [2022-01-10 12:53:16,521] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:16,522] jepsen worker 52 - jepsen.util 52 :ok :read [17 39] INFO [2022-01-10 12:53:16,522] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:16,522] jepsen worker 77 - jepsen.util 167 :ok :read [25 45] INFO [2022-01-10 12:53:16,523] jepsen worker 52 - jepsen.util 52 :ok :read [17 39] INFO [2022-01-10 12:53:16,523] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:16,523] jepsen worker 85 - jepsen.util 175 :ok :write [28 39] INFO [2022-01-10 12:53:16,523] jepsen worker 83 - jepsen.util 173 :ok :write [27 3] INFO [2022-01-10 12:53:16,524] jepsen worker 77 - jepsen.util 167 :ok :read [25 45] INFO [2022-01-10 12:53:16,524] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:16,525] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:16,525] jepsen worker 83 - jepsen.util 173 :ok :read [27 3] INFO [2022-01-10 12:53:16,525] jepsen worker 13 - jepsen.util 13 :ok :write [4 19] INFO [2022-01-10 12:53:16,525] jepsen worker 85 - jepsen.util 175 :ok :read [28 39] INFO [2022-01-10 12:53:16,526] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:16,527] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:16,527] jepsen worker 52 - jepsen.util 52 :ok :read [17 39] INFO [2022-01-10 12:53:16,527] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:16,527] jepsen worker 77 - jepsen.util 167 :ok :read [25 45] INFO [2022-01-10 12:53:16,528] jepsen worker 52 - jepsen.util 52 :invoke :write [17 1] INFO [2022-01-10 12:53:16,528] jepsen worker 13 - jepsen.util 13 :ok :read [4 19] INFO [2022-01-10 12:53:16,528] jepsen worker 23 - jepsen.util 23 :ok :write [7 23] INFO [2022-01-10 12:53:16,529] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:16,529] jepsen worker 85 - jepsen.util 175 :ok :read [28 39] INFO [2022-01-10 12:53:16,530] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:16,530] jepsen worker 8 - jepsen.util 8 :ok :write [2 32] INFO [2022-01-10 12:53:16,531] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,531] jepsen worker 36 - jepsen.util 36 :ok :write [12 33] INFO [2022-01-10 12:53:16,531] jepsen worker 83 - jepsen.util 173 :ok :read [27 3] INFO [2022-01-10 12:53:16,532] jepsen worker 8 - jepsen.util 8 :invoke :write [2 41] INFO [2022-01-10 12:53:16,532] jepsen worker 23 - jepsen.util 23 :ok :read [7 23] INFO [2022-01-10 12:53:16,532] jepsen worker 36 - jepsen.util 36 :invoke :write [12 6] INFO [2022-01-10 12:53:16,533] jepsen worker 78 - jepsen.util 78 :ok :write [26 5] INFO [2022-01-10 12:53:16,533] jepsen worker 83 - jepsen.util 173 :invoke :write [27 2] INFO [2022-01-10 12:53:16,534] jepsen worker 44 - jepsen.util 44 :ok :write [14 38] INFO [2022-01-10 12:53:16,534] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:16,535] jepsen worker 85 - jepsen.util 175 :invoke :write [28 12] INFO [2022-01-10 12:53:16,535] jepsen worker 78 - jepsen.util 78 :ok :read [26 5] INFO [2022-01-10 12:53:16,536] jepsen worker 77 - jepsen.util 167 :invoke :write [25 22] INFO [2022-01-10 12:53:16,537] jepsen worker 13 - jepsen.util 13 :invoke :write [4 11] INFO [2022-01-10 12:53:16,537] jepsen worker 35 - jepsen.util 125 :ok :write [11 32] INFO [2022-01-10 12:53:16,538] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:16,538] jepsen worker 23 - jepsen.util 23 :invoke :write [7 28] INFO [2022-01-10 12:53:16,539] jepsen worker 78 - jepsen.util 78 :ok :read [26 5] INFO [2022-01-10 12:53:16,539] jepsen worker 35 - jepsen.util 125 :invoke :write [11 18] INFO [2022-01-10 12:53:16,540] jepsen worker 78 - jepsen.util 78 :invoke :write [26 14] INFO [2022-01-10 12:53:16,540] jepsen worker 44 - jepsen.util 44 :invoke :write [14 16] INFO [2022-01-10 12:53:16,541] jepsen worker 16 - jepsen.util 106 :ok :write [5 19] INFO [2022-01-10 12:53:16,542] jepsen worker 16 - jepsen.util 106 :invoke :write [5 24] INFO [2022-01-10 12:53:16,542] jepsen worker 62 - jepsen.util 152 :ok :write [20 29] INFO [2022-01-10 12:53:16,543] jepsen worker 70 - jepsen.util 160 :ok :write [23 48] INFO [2022-01-10 12:53:16,543] jepsen worker 62 - jepsen.util 152 :invoke :write [20 27] INFO [2022-01-10 12:53:16,543] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:16,545] jepsen worker 70 - jepsen.util 160 :ok :read [23 48] INFO [2022-01-10 12:53:16,545] jepsen worker 70 - jepsen.util 160 :invoke :write [23 19] INFO [2022-01-10 12:53:16,548] jepsen worker 66 - jepsen.util 156 :ok :write [22 27] INFO [2022-01-10 12:53:16,549] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:16,550] jepsen worker 66 - jepsen.util 156 :ok :read [22 27] INFO [2022-01-10 12:53:16,551] jepsen worker 66 - jepsen.util 156 :invoke :write [22 41] INFO [2022-01-10 12:53:16,552] jepsen worker 52 - jepsen.util 52 :ok :write [17 1] INFO [2022-01-10 12:53:16,553] jepsen worker 52 - jepsen.util 52 :invoke :write [17 43] INFO [2022-01-10 12:53:16,555] jepsen worker 4 - jepsen.util 4 :ok :write [1 38] INFO [2022-01-10 12:53:16,555] jepsen worker 39 - jepsen.util 39 :ok :write [13 43] INFO [2022-01-10 12:53:16,555] jepsen worker 23 - jepsen.util 23 :ok :write [7 28] INFO [2022-01-10 12:53:16,556] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,556] jepsen worker 39 - jepsen.util 39 :invoke :write [13 41] INFO [2022-01-10 12:53:16,557] jepsen worker 23 - jepsen.util 23 :ok :read [7 28] INFO [2022-01-10 12:53:16,557] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,558] jepsen worker 4 - jepsen.util 4 :ok :read [1 38] INFO [2022-01-10 12:53:16,558] jepsen worker 23 - jepsen.util 23 :invoke :write [7 5] INFO [2022-01-10 12:53:16,559] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,559] jepsen worker 4 - jepsen.util 4 :ok :read [1 38] INFO [2022-01-10 12:53:16,560] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,561] jepsen worker 4 - jepsen.util 4 :ok :read [1 38] INFO [2022-01-10 12:53:16,561] jepsen worker 85 - jepsen.util 175 :ok :write [28 12] INFO [2022-01-10 12:53:16,562] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,562] jepsen worker 85 - jepsen.util 175 :invoke :write [28 39] INFO [2022-01-10 12:53:16,563] jepsen worker 4 - jepsen.util 4 :ok :read [1 38] INFO [2022-01-10 12:53:16,563] jepsen worker 43 - jepsen.util 43 :ok :write [14 20] INFO [2022-01-10 12:53:16,563] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,564] jepsen worker 43 - jepsen.util 43 :invoke :write [14 41] INFO [2022-01-10 12:53:16,565] jepsen worker 4 - jepsen.util 4 :ok :read [1 38] INFO [2022-01-10 12:53:16,566] jepsen worker 4 - jepsen.util 4 :invoke :write [1 45] INFO [2022-01-10 12:53:16,567] jepsen worker 13 - jepsen.util 13 :ok :write [4 11] INFO [2022-01-10 12:53:16,568] jepsen worker 87 - jepsen.util 87 :ok :write [29 24] INFO [2022-01-10 12:53:16,568] jepsen worker 13 - jepsen.util 13 :invoke :write [4 22] INFO [2022-01-10 12:53:16,569] jepsen worker 87 - jepsen.util 87 :invoke :write [29 31] INFO [2022-01-10 12:53:16,572] jepsen worker 1 - jepsen.util 1 :ok :write [0 35] INFO [2022-01-10 12:53:16,573] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,574] jepsen worker 1 - jepsen.util 1 :ok :read [0 35] INFO [2022-01-10 12:53:16,575] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,575] jepsen worker 77 - jepsen.util 167 :ok :write [25 22] INFO [2022-01-10 12:53:16,576] jepsen worker 1 - jepsen.util 1 :ok :read [0 35] INFO [2022-01-10 12:53:16,576] jepsen worker 77 - jepsen.util 167 :invoke :write [25 11] INFO [2022-01-10 12:53:16,576] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,577] jepsen worker 1 - jepsen.util 1 :ok :read [0 35] INFO [2022-01-10 12:53:16,577] jepsen worker 36 - jepsen.util 36 :ok :write [12 6] INFO [2022-01-10 12:53:16,578] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,579] jepsen worker 36 - jepsen.util 36 :invoke :write [12 49] INFO [2022-01-10 12:53:16,579] jepsen worker 1 - jepsen.util 1 :ok :read [0 35] INFO [2022-01-10 12:53:16,580] jepsen worker 1 - jepsen.util 1 :invoke :write [0 18] INFO [2022-01-10 12:53:16,582] jepsen worker 62 - jepsen.util 152 :ok :write [20 27] INFO [2022-01-10 12:53:16,582] jepsen worker 62 - jepsen.util 152 :invoke :write [20 31] INFO [2022-01-10 12:53:16,586] jepsen worker 49 - jepsen.util 139 :ok :write [16 14] INFO [2022-01-10 12:53:16,586] jepsen worker 49 - jepsen.util 139 :invoke :write [16 47] INFO [2022-01-10 12:53:16,587] jepsen worker 78 - jepsen.util 78 :ok :write [26 14] INFO [2022-01-10 12:53:16,587] jepsen worker 78 - jepsen.util 78 :invoke :write [26 16] INFO [2022-01-10 12:53:16,592] jepsen worker 83 - jepsen.util 173 :ok :write [27 2] INFO [2022-01-10 12:53:16,593] jepsen worker 83 - jepsen.util 173 :invoke :write [27 7] INFO [2022-01-10 12:53:16,594] jepsen worker 70 - jepsen.util 160 :ok :write [23 19] INFO [2022-01-10 12:53:16,595] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:16,596] jepsen worker 70 - jepsen.util 160 :ok :read [23 19] INFO [2022-01-10 12:53:16,597] jepsen worker 70 - jepsen.util 160 :invoke :write [23 46] INFO [2022-01-10 12:53:16,597] jepsen worker 8 - jepsen.util 8 :ok :write [2 41] INFO [2022-01-10 12:53:16,598] jepsen worker 8 - jepsen.util 8 :invoke :write [2 12] INFO [2022-01-10 12:53:16,601] jepsen worker 35 - jepsen.util 125 :ok :write [11 18] INFO [2022-01-10 12:53:16,601] jepsen worker 52 - jepsen.util 52 :ok :write [17 43] INFO [2022-01-10 12:53:16,602] jepsen worker 35 - jepsen.util 125 :invoke :write [11 43] INFO [2022-01-10 12:53:16,602] jepsen worker 52 - jepsen.util 52 :invoke :write [17 34] INFO [2022-01-10 12:53:16,604] jepsen worker 16 - jepsen.util 106 :ok :write [5 24] INFO [2022-01-10 12:53:16,605] jepsen worker 16 - jepsen.util 106 :invoke :write [5 27] INFO [2022-01-10 12:53:16,607] jepsen worker 4 - jepsen.util 4 :ok :write [1 45] INFO [2022-01-10 12:53:16,608] jepsen worker 4 - jepsen.util 4 :invoke :write [1 34] INFO [2022-01-10 12:53:16,611] jepsen worker 39 - jepsen.util 39 :ok :write [13 41] INFO [2022-01-10 12:53:16,612] jepsen worker 39 - jepsen.util 39 :invoke :write [13 22] INFO [2022-01-10 12:53:16,613] jepsen worker 85 - jepsen.util 175 :ok :write [28 39] INFO [2022-01-10 12:53:16,614] jepsen worker 85 - jepsen.util 175 :invoke :write [28 44] INFO [2022-01-10 12:53:16,617] jepsen worker 87 - jepsen.util 87 :ok :write [29 31] INFO [2022-01-10 12:53:16,618] jepsen worker 87 - jepsen.util 87 :invoke :write [29 7] INFO [2022-01-10 12:53:16,621] jepsen worker 1 - jepsen.util 1 :ok :write [0 18] INFO [2022-01-10 12:53:16,622] jepsen worker 13 - jepsen.util 13 :ok :write [4 22] INFO [2022-01-10 12:53:16,622] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,622] jepsen worker 13 - jepsen.util 13 :invoke :write [4 28] INFO [2022-01-10 12:53:16,623] jepsen worker 1 - jepsen.util 1 :ok :read [0 18] INFO [2022-01-10 12:53:16,624] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,624] jepsen worker 1 - jepsen.util 1 :ok :read [0 18] INFO [2022-01-10 12:53:16,625] jepsen worker 1 - jepsen.util 1 :invoke :write [0 17] INFO [2022-01-10 12:53:16,628] jepsen worker 36 - jepsen.util 36 :ok :write [12 49] INFO [2022-01-10 12:53:16,628] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,629] jepsen worker 36 - jepsen.util 36 :ok :read [12 49] INFO [2022-01-10 12:53:16,629] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,630] jepsen worker 36 - jepsen.util 36 :ok :read [12 49] INFO [2022-01-10 12:53:16,631] jepsen worker 36 - jepsen.util 36 :invoke :write [12 20] INFO [2022-01-10 12:53:16,643] jepsen worker 39 - jepsen.util 39 :ok :write [13 22] INFO [2022-01-10 12:53:16,643] jepsen worker 39 - jepsen.util 39 :invoke :write [13 39] INFO [2022-01-10 12:53:16,647] jepsen worker 4 - jepsen.util 4 :ok :write [1 34] INFO [2022-01-10 12:53:16,647] jepsen worker 87 - jepsen.util 87 :ok :write [29 7] INFO [2022-01-10 12:53:16,648] jepsen worker 4 - jepsen.util 4 :invoke :write [1 26] INFO [2022-01-10 12:53:16,648] jepsen worker 87 - jepsen.util 87 :invoke :write [29 8] INFO [2022-01-10 12:53:16,649] jepsen worker 1 - jepsen.util 1 :ok :write [0 17] INFO [2022-01-10 12:53:16,650] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,651] jepsen worker 1 - jepsen.util 1 :ok :read [0 17] INFO [2022-01-10 12:53:16,651] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,652] jepsen worker 1 - jepsen.util 1 :ok :read [0 17] INFO [2022-01-10 12:53:16,653] jepsen worker 1 - jepsen.util 1 :invoke :write [0 13] INFO [2022-01-10 12:53:16,654] jepsen worker 85 - jepsen.util 175 :ok :write [28 44] INFO [2022-01-10 12:53:16,655] jepsen worker 39 - jepsen.util 39 :ok :write [13 39] INFO [2022-01-10 12:53:16,655] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:16,656] jepsen worker 85 - jepsen.util 175 :ok :read [28 44] INFO [2022-01-10 12:53:16,656] jepsen worker 39 - jepsen.util 39 :invoke :write [13 22] INFO [2022-01-10 12:53:16,657] jepsen worker 85 - jepsen.util 175 :invoke :write [28 29] INFO [2022-01-10 12:53:16,661] jepsen worker 87 - jepsen.util 87 :ok :write [29 8] INFO [2022-01-10 12:53:16,661] jepsen worker 87 - jepsen.util 87 :invoke :write [29 0] INFO [2022-01-10 12:53:16,666] jepsen worker 36 - jepsen.util 36 :ok :write [12 20] INFO [2022-01-10 12:53:16,666] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,667] jepsen worker 1 - jepsen.util 1 :ok :write [0 13] INFO [2022-01-10 12:53:16,667] jepsen worker 36 - jepsen.util 36 :ok :read [12 20] INFO [2022-01-10 12:53:16,667] jepsen worker 1 - jepsen.util 1 :invoke :write [0 10] INFO [2022-01-10 12:53:16,668] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,669] jepsen worker 36 - jepsen.util 36 :ok :read [12 20] INFO [2022-01-10 12:53:16,669] jepsen worker 36 - jepsen.util 36 :invoke :write [12 4] INFO [2022-01-10 12:53:16,672] jepsen worker 4 - jepsen.util 4 :ok :write [1 26] INFO [2022-01-10 12:53:16,672] jepsen worker 39 - jepsen.util 39 :ok :write [13 22] INFO [2022-01-10 12:53:16,672] jepsen worker 4 - jepsen.util 4 :invoke :write [1 2] INFO [2022-01-10 12:53:16,673] jepsen worker 39 - jepsen.util 39 :invoke :write [13 0] INFO [2022-01-10 12:53:16,678] jepsen worker 87 - jepsen.util 87 :ok :write [29 0] INFO [2022-01-10 12:53:16,678] jepsen worker 85 - jepsen.util 175 :ok :write [28 29] INFO [2022-01-10 12:53:16,678] jepsen worker 87 - jepsen.util 87 :invoke :write [29 22] INFO [2022-01-10 12:53:16,679] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:16,679] jepsen worker 85 - jepsen.util 175 :ok :read [28 29] INFO [2022-01-10 12:53:16,680] jepsen worker 85 - jepsen.util 175 :invoke :write [28 34] INFO [2022-01-10 12:53:16,683] jepsen worker 1 - jepsen.util 1 :ok :write [0 10] INFO [2022-01-10 12:53:16,684] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,684] jepsen worker 36 - jepsen.util 36 :ok :write [12 4] INFO [2022-01-10 12:53:16,685] jepsen worker 1 - jepsen.util 1 :ok :read [0 10] INFO [2022-01-10 12:53:16,685] jepsen worker 36 - jepsen.util 36 :invoke :write [12 46] INFO [2022-01-10 12:53:16,685] jepsen worker 1 - jepsen.util 1 :invoke :write [0 30] INFO [2022-01-10 12:53:16,687] jepsen worker 6 - jepsen.util 6 :info :write [2 10] :RequestTimeoutException INFO [2022-01-10 12:53:16,687] jepsen worker 6 - jepsen.util 96 :invoke :write [2 49] INFO [2022-01-10 12:53:16,715] jepsen worker 66 - jepsen.util 156 :ok :write [22 41] INFO [2022-01-10 12:53:16,716] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:16,717] jepsen worker 66 - jepsen.util 156 :ok :read [22 41] INFO [2022-01-10 12:53:16,718] jepsen worker 66 - jepsen.util 156 :invoke :write [22 25] INFO [2022-01-10 12:53:16,719] jepsen worker 23 - jepsen.util 23 :ok :write [7 5] INFO [2022-01-10 12:53:16,720] jepsen worker 23 - jepsen.util 23 :invoke :write [7 8] INFO [2022-01-10 12:53:16,742] jepsen worker 69 - jepsen.util 69 :info :write [23 14] :RequestTimeoutException INFO [2022-01-10 12:53:16,742] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:16,744] jepsen worker 69 - jepsen.util 159 :ok :read [23 19] INFO [2022-01-10 12:53:16,744] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:16,745] jepsen worker 69 - jepsen.util 159 :ok :read [23 19] INFO [2022-01-10 12:53:16,745] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:16,746] jepsen worker 69 - jepsen.util 159 :ok :read [23 19] INFO [2022-01-10 12:53:16,746] jepsen worker 69 - jepsen.util 159 :invoke :write [23 28] INFO [2022-01-10 12:53:16,765] jepsen worker 39 - jepsen.util 39 :ok :write [13 0] INFO [2022-01-10 12:53:16,766] jepsen worker 39 - jepsen.util 39 :invoke :write [13 15] INFO [2022-01-10 12:53:16,769] jepsen worker 4 - jepsen.util 4 :ok :write [1 2] INFO [2022-01-10 12:53:16,770] jepsen worker 4 - jepsen.util 4 :invoke :write [1 28] INFO [2022-01-10 12:53:16,770] jepsen worker 87 - jepsen.util 87 :ok :write [29 22] INFO [2022-01-10 12:53:16,771] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:16,772] jepsen worker 87 - jepsen.util 87 :ok :read [29 22] INFO [2022-01-10 12:53:16,772] jepsen worker 87 - jepsen.util 87 :invoke :write [29 22] INFO [2022-01-10 12:53:16,775] jepsen worker 78 - jepsen.util 78 :ok :write [26 16] INFO [2022-01-10 12:53:16,776] jepsen worker 78 - jepsen.util 78 :invoke :write [26 23] INFO [2022-01-10 12:53:16,777] jepsen worker 44 - jepsen.util 44 :ok :write [14 16] INFO [2022-01-10 12:53:16,778] jepsen worker 44 - jepsen.util 44 :invoke :write [14 42] INFO [2022-01-10 12:53:16,779] jepsen worker 85 - jepsen.util 175 :ok :write [28 34] INFO [2022-01-10 12:53:16,780] jepsen worker 85 - jepsen.util 175 :invoke :write [28 29] INFO [2022-01-10 12:53:16,780] jepsen worker 1 - jepsen.util 1 :ok :write [0 30] INFO [2022-01-10 12:53:16,781] jepsen worker 77 - jepsen.util 167 :ok :write [25 11] INFO [2022-01-10 12:53:16,781] jepsen worker 70 - jepsen.util 160 :ok :write [23 46] INFO [2022-01-10 12:53:16,782] jepsen worker 1 - jepsen.util 1 :invoke :write [0 36] INFO [2022-01-10 12:53:16,783] jepsen worker 77 - jepsen.util 167 :invoke :write [25 0] INFO [2022-01-10 12:53:16,783] jepsen worker 70 - jepsen.util 160 :invoke :write [23 45] INFO [2022-01-10 12:53:16,786] jepsen worker 62 - jepsen.util 152 :ok :write [20 31] INFO [2022-01-10 12:53:16,787] jepsen worker 36 - jepsen.util 36 :ok :write [12 46] INFO [2022-01-10 12:53:16,787] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:16,788] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,788] jepsen worker 62 - jepsen.util 152 :ok :read [20 31] INFO [2022-01-10 12:53:16,789] jepsen worker 36 - jepsen.util 36 :ok :read [12 46] INFO [2022-01-10 12:53:16,789] jepsen worker 52 - jepsen.util 52 :ok :write [17 34] INFO [2022-01-10 12:53:16,790] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:16,791] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,792] jepsen worker 13 - jepsen.util 13 :ok :write [4 28] INFO [2022-01-10 12:53:16,792] jepsen worker 52 - jepsen.util 52 :invoke :write [17 28] INFO [2022-01-10 12:53:16,792] jepsen worker 62 - jepsen.util 152 :ok :read [20 31] INFO [2022-01-10 12:53:16,792] jepsen worker 36 - jepsen.util 36 :ok :read [12 46] INFO [2022-01-10 12:53:16,792] jepsen worker 13 - jepsen.util 13 :invoke :write [4 14] INFO [2022-01-10 12:53:16,792] jepsen worker 39 - jepsen.util 39 :ok :write [13 15] INFO [2022-01-10 12:53:16,793] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,794] jepsen worker 62 - jepsen.util 152 :invoke :write [20 22] INFO [2022-01-10 12:53:16,794] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,794] jepsen worker 36 - jepsen.util 36 :ok :read [12 46] INFO [2022-01-10 12:53:16,795] jepsen worker 36 - jepsen.util 36 :invoke :write [12 11] INFO [2022-01-10 12:53:16,795] jepsen worker 39 - jepsen.util 39 :ok :read [13 15] INFO [2022-01-10 12:53:16,796] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,797] jepsen worker 39 - jepsen.util 39 :ok :read [13 15] INFO [2022-01-10 12:53:16,797] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,798] jepsen worker 4 - jepsen.util 4 :ok :write [1 28] INFO [2022-01-10 12:53:16,798] jepsen worker 87 - jepsen.util 87 :ok :write [29 22] INFO [2022-01-10 12:53:16,798] jepsen worker 39 - jepsen.util 39 :ok :read [13 15] INFO [2022-01-10 12:53:16,799] jepsen worker 87 - jepsen.util 87 :invoke :write [29 33] INFO [2022-01-10 12:53:16,799] jepsen worker 39 - jepsen.util 39 :invoke :write [13 11] INFO [2022-01-10 12:53:16,800] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,801] jepsen worker 4 - jepsen.util 4 :ok :read [1 28] INFO [2022-01-10 12:53:16,802] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,802] jepsen worker 4 - jepsen.util 4 :ok :read [1 28] INFO [2022-01-10 12:53:16,803] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,804] jepsen worker 4 - jepsen.util 4 :ok :read [1 28] INFO [2022-01-10 12:53:16,804] jepsen worker 4 - jepsen.util 4 :invoke :write [1 48] INFO [2022-01-10 12:53:16,815] jepsen worker 85 - jepsen.util 175 :ok :write [28 29] INFO [2022-01-10 12:53:16,815] jepsen worker 1 - jepsen.util 1 :ok :write [0 36] INFO [2022-01-10 12:53:16,815] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:16,815] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,816] jepsen worker 85 - jepsen.util 175 :ok :read [28 29] INFO [2022-01-10 12:53:16,816] jepsen worker 1 - jepsen.util 1 :ok :read [0 36] INFO [2022-01-10 12:53:16,816] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:16,817] jepsen worker 1 - jepsen.util 1 :invoke :write [0 9] INFO [2022-01-10 12:53:16,817] jepsen worker 85 - jepsen.util 175 :ok :read [28 29] INFO [2022-01-10 12:53:16,817] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:16,818] jepsen worker 85 - jepsen.util 175 :ok :read [28 29] INFO [2022-01-10 12:53:16,818] jepsen worker 85 - jepsen.util 175 :invoke :write [28 5] INFO [2022-01-10 12:53:16,820] jepsen worker 36 - jepsen.util 36 :ok :write [12 11] INFO [2022-01-10 12:53:16,821] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,822] jepsen worker 36 - jepsen.util 36 :ok :read [12 11] INFO [2022-01-10 12:53:16,823] jepsen worker 36 - jepsen.util 36 :invoke :write [12 47] INFO [2022-01-10 12:53:16,826] jepsen worker 39 - jepsen.util 39 :ok :write [13 11] INFO [2022-01-10 12:53:16,827] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,828] jepsen worker 39 - jepsen.util 39 :ok :read [13 11] INFO [2022-01-10 12:53:16,828] jepsen worker 39 - jepsen.util 39 :invoke :write [13 28] INFO [2022-01-10 12:53:16,880] jepsen worker 49 - jepsen.util 139 :ok :write [16 47] INFO [2022-01-10 12:53:16,881] jepsen worker 49 - jepsen.util 139 :invoke :write [16 38] INFO [2022-01-10 12:53:16,882] jepsen worker 78 - jepsen.util 78 :ok :write [26 23] INFO [2022-01-10 12:53:16,882] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:16,883] jepsen worker 78 - jepsen.util 78 :ok :read [26 23] INFO [2022-01-10 12:53:16,884] jepsen worker 78 - jepsen.util 78 :invoke :write [26 26] INFO [2022-01-10 12:53:16,887] jepsen worker 70 - jepsen.util 160 :ok :write [23 45] INFO [2022-01-10 12:53:16,887] jepsen worker 83 - jepsen.util 173 :ok :write [27 7] INFO [2022-01-10 12:53:16,887] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:16,888] jepsen worker 83 - jepsen.util 173 :invoke :write [27 40] INFO [2022-01-10 12:53:16,888] jepsen worker 70 - jepsen.util 160 :ok :read [23 45] INFO [2022-01-10 12:53:16,889] jepsen worker 70 - jepsen.util 160 :invoke :write [23 21] INFO [2022-01-10 12:53:16,889] jepsen worker 52 - jepsen.util 52 :ok :write [17 28] INFO [2022-01-10 12:53:16,890] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:16,891] jepsen worker 52 - jepsen.util 52 :ok :read [17 28] INFO [2022-01-10 12:53:16,891] jepsen worker 52 - jepsen.util 52 :invoke :write [17 46] INFO [2022-01-10 12:53:16,941] jepsen worker 13 - jepsen.util 13 :ok :write [4 14] INFO [2022-01-10 12:53:16,942] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:16,943] jepsen worker 8 - jepsen.util 8 :ok :write [2 12] INFO [2022-01-10 12:53:16,943] jepsen worker 13 - jepsen.util 13 :ok :read [4 14] INFO [2022-01-10 12:53:16,943] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:16,943] jepsen worker 13 - jepsen.util 13 :invoke :write [4 40] INFO [2022-01-10 12:53:16,944] jepsen worker 8 - jepsen.util 8 :ok :read [2 12] INFO [2022-01-10 12:53:16,944] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:16,945] jepsen worker 8 - jepsen.util 8 :ok :read [2 12] INFO [2022-01-10 12:53:16,946] jepsen worker 8 - jepsen.util 8 :invoke :write [2 46] INFO [2022-01-10 12:53:16,948] jepsen worker 78 - jepsen.util 78 :ok :write [26 26] INFO [2022-01-10 12:53:16,949] jepsen worker 78 - jepsen.util 78 :invoke :write [26 4] INFO [2022-01-10 12:53:16,951] jepsen worker 35 - jepsen.util 125 :ok :write [11 43] INFO [2022-01-10 12:53:16,952] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:16,953] jepsen worker 35 - jepsen.util 125 :ok :read [11 43] INFO [2022-01-10 12:53:16,953] jepsen worker 35 - jepsen.util 125 :invoke :write [11 33] INFO [2022-01-10 12:53:16,955] jepsen worker 70 - jepsen.util 160 :ok :write [23 21] INFO [2022-01-10 12:53:16,956] jepsen worker 70 - jepsen.util 160 :invoke :write [23 40] INFO [2022-01-10 12:53:16,958] jepsen worker 16 - jepsen.util 106 :ok :write [5 27] INFO [2022-01-10 12:53:16,959] jepsen worker 16 - jepsen.util 106 :invoke :write [5 47] INFO [2022-01-10 12:53:16,963] jepsen worker 4 - jepsen.util 4 :ok :write [1 48] INFO [2022-01-10 12:53:16,963] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:16,964] jepsen worker 52 - jepsen.util 52 :ok :write [17 46] INFO [2022-01-10 12:53:16,964] jepsen worker 87 - jepsen.util 87 :ok :write [29 33] INFO [2022-01-10 12:53:16,964] jepsen worker 4 - jepsen.util 4 :ok :read [1 48] INFO [2022-01-10 12:53:16,966] jepsen worker 87 - jepsen.util 87 :invoke :write [29 26] INFO [2022-01-10 12:53:16,966] jepsen worker 85 - jepsen.util 175 :ok :write [28 5] INFO [2022-01-10 12:53:16,967] jepsen worker 66 - jepsen.util 156 :ok :write [22 25] INFO [2022-01-10 12:53:16,966] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:16,968] jepsen worker 4 - jepsen.util 4 :invoke :write [1 32] INFO [2022-01-10 12:53:16,969] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:16,970] jepsen worker 85 - jepsen.util 175 :invoke :write [28 5] INFO [2022-01-10 12:53:16,970] jepsen worker 66 - jepsen.util 156 :ok :read [22 25] INFO [2022-01-10 12:53:16,971] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:16,971] jepsen worker 13 - jepsen.util 13 :ok :write [4 40] INFO [2022-01-10 12:53:16,971] jepsen worker 52 - jepsen.util 52 :ok :read [17 46] INFO [2022-01-10 12:53:16,971] jepsen worker 36 - jepsen.util 36 :ok :write [12 47] INFO [2022-01-10 12:53:16,972] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:16,972] jepsen worker 66 - jepsen.util 156 :ok :read [22 25] INFO [2022-01-10 12:53:16,972] jepsen worker 23 - jepsen.util 23 :ok :write [7 8] INFO [2022-01-10 12:53:16,973] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:16,974] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,974] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:16,974] jepsen worker 13 - jepsen.util 13 :ok :read [4 40] INFO [2022-01-10 12:53:16,974] jepsen worker 52 - jepsen.util 52 :ok :read [17 46] INFO [2022-01-10 12:53:16,974] jepsen worker 78 - jepsen.util 78 :ok :write [26 4] INFO [2022-01-10 12:53:16,974] jepsen worker 36 - jepsen.util 36 :ok :read [12 47] INFO [2022-01-10 12:53:16,975] jepsen worker 66 - jepsen.util 156 :invoke :write [22 38] INFO [2022-01-10 12:53:16,975] jepsen worker 23 - jepsen.util 23 :ok :read [7 8] INFO [2022-01-10 12:53:16,976] jepsen worker 78 - jepsen.util 78 :invoke :write [26 18] INFO [2022-01-10 12:53:16,977] jepsen worker 52 - jepsen.util 52 :invoke :write [17 21] INFO [2022-01-10 12:53:16,978] jepsen worker 23 - jepsen.util 23 :invoke :write [7 42] INFO [2022-01-10 12:53:16,978] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:16,979] jepsen worker 1 - jepsen.util 1 :ok :write [0 9] INFO [2022-01-10 12:53:16,979] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,979] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:16,979] jepsen worker 13 - jepsen.util 13 :ok :read [4 40] INFO [2022-01-10 12:53:16,980] jepsen worker 13 - jepsen.util 13 :invoke :write [4 20] INFO [2022-01-10 12:53:16,980] jepsen worker 1 - jepsen.util 1 :ok :read [0 9] INFO [2022-01-10 12:53:16,980] jepsen worker 36 - jepsen.util 36 :ok :read [12 47] INFO [2022-01-10 12:53:16,981] jepsen worker 44 - jepsen.util 44 :ok :write [14 42] INFO [2022-01-10 12:53:16,981] jepsen worker 1 - jepsen.util 1 :invoke :write [0 34] INFO [2022-01-10 12:53:16,981] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:16,981] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:16,982] jepsen worker 44 - jepsen.util 44 :ok :read [14 42] INFO [2022-01-10 12:53:16,983] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:16,983] jepsen worker 44 - jepsen.util 44 :ok :read [14 42] INFO [2022-01-10 12:53:16,984] jepsen worker 44 - jepsen.util 44 :invoke :write [14 43] INFO [2022-01-10 12:53:16,985] jepsen worker 36 - jepsen.util 36 :ok :read [12 47] INFO [2022-01-10 12:53:16,985] jepsen worker 70 - jepsen.util 160 :ok :write [23 40] INFO [2022-01-10 12:53:16,985] jepsen worker 36 - jepsen.util 36 :invoke :write [12 35] INFO [2022-01-10 12:53:16,986] jepsen worker 70 - jepsen.util 160 :invoke :write [23 20] INFO [2022-01-10 12:53:16,987] jepsen worker 39 - jepsen.util 39 :ok :write [13 28] INFO [2022-01-10 12:53:16,987] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:16,988] jepsen worker 39 - jepsen.util 39 :ok :read [13 28] INFO [2022-01-10 12:53:16,989] jepsen worker 39 - jepsen.util 39 :invoke :write [13 4] INFO [2022-01-10 12:53:16,990] jepsen worker 85 - jepsen.util 175 :ok :write [28 5] INFO [2022-01-10 12:53:16,990] jepsen worker 85 - jepsen.util 175 :invoke :write [28 47] INFO [2022-01-10 12:53:16,990] jepsen worker 77 - jepsen.util 167 :ok :write [25 0] INFO [2022-01-10 12:53:16,991] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:16,992] jepsen worker 77 - jepsen.util 167 :ok :read [25 0] INFO [2022-01-10 12:53:16,993] jepsen worker 77 - jepsen.util 167 :invoke :write [25 37] INFO [2022-01-10 12:53:16,997] jepsen worker 13 - jepsen.util 13 :ok :write [4 20] INFO [2022-01-10 12:53:16,997] jepsen worker 13 - jepsen.util 13 :invoke :write [4 46] INFO [2022-01-10 12:53:16,998] jepsen worker 62 - jepsen.util 152 :ok :write [20 22] INFO [2022-01-10 12:53:16,999] jepsen worker 62 - jepsen.util 152 :invoke :write [20 23] INFO [2022-01-10 12:53:17,003] jepsen worker 36 - jepsen.util 36 :ok :write [12 35] INFO [2022-01-10 12:53:17,003] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,004] jepsen worker 36 - jepsen.util 36 :ok :read [12 35] INFO [2022-01-10 12:53:17,004] jepsen worker 36 - jepsen.util 36 :invoke :write [12 47] INFO [2022-01-10 12:53:17,007] jepsen worker 49 - jepsen.util 139 :ok :write [16 38] INFO [2022-01-10 12:53:17,007] jepsen worker 49 - jepsen.util 139 :invoke :write [16 15] INFO [2022-01-10 12:53:17,009] jepsen worker 4 - jepsen.util 4 :ok :write [1 32] INFO [2022-01-10 12:53:17,010] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,011] jepsen worker 4 - jepsen.util 4 :ok :read [1 32] INFO [2022-01-10 12:53:17,011] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,012] jepsen worker 4 - jepsen.util 4 :ok :read [1 32] INFO [2022-01-10 12:53:17,013] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,014] jepsen worker 4 - jepsen.util 4 :ok :read [1 32] INFO [2022-01-10 12:53:17,014] jepsen worker 78 - jepsen.util 78 :ok :write [26 18] INFO [2022-01-10 12:53:17,015] jepsen worker 83 - jepsen.util 173 :ok :write [27 40] INFO [2022-01-10 12:53:17,015] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:17,016] jepsen worker 4 - jepsen.util 4 :invoke :write [1 46] INFO [2022-01-10 12:53:17,016] jepsen worker 78 - jepsen.util 78 :ok :read [26 18] INFO [2022-01-10 12:53:17,017] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:17,017] jepsen worker 78 - jepsen.util 78 :invoke :write [26 39] INFO [2022-01-10 12:53:17,018] jepsen worker 83 - jepsen.util 173 :ok :read [27 40] INFO [2022-01-10 12:53:17,018] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:17,019] jepsen worker 83 - jepsen.util 173 :ok :read [27 40] INFO [2022-01-10 12:53:17,019] jepsen worker 52 - jepsen.util 52 :ok :write [17 21] INFO [2022-01-10 12:53:17,019] jepsen worker 8 - jepsen.util 8 :ok :write [2 46] INFO [2022-01-10 12:53:17,020] jepsen worker 83 - jepsen.util 173 :invoke :write [27 20] INFO [2022-01-10 12:53:17,020] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:17,021] jepsen worker 8 - jepsen.util 8 :invoke :write [2 34] INFO [2022-01-10 12:53:17,022] jepsen worker 87 - jepsen.util 87 :ok :write [29 26] INFO [2022-01-10 12:53:17,023] jepsen worker 52 - jepsen.util 52 :ok :read [17 21] INFO [2022-01-10 12:53:17,023] jepsen worker 87 - jepsen.util 87 :invoke :write [29 33] INFO [2022-01-10 12:53:17,024] jepsen worker 52 - jepsen.util 52 :invoke :write [17 19] INFO [2022-01-10 12:53:17,026] jepsen worker 70 - jepsen.util 160 :ok :write [23 20] INFO [2022-01-10 12:53:17,027] jepsen worker 70 - jepsen.util 160 :invoke :write [23 41] INFO [2022-01-10 12:53:17,027] jepsen worker 35 - jepsen.util 125 :ok :write [11 33] INFO [2022-01-10 12:53:17,028] jepsen worker 35 - jepsen.util 125 :invoke :write [11 25] INFO [2022-01-10 12:53:17,032] jepsen worker 85 - jepsen.util 175 :ok :write [28 47] INFO [2022-01-10 12:53:17,033] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,034] jepsen worker 85 - jepsen.util 175 :ok :read [28 47] INFO [2022-01-10 12:53:17,034] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,035] jepsen worker 1 - jepsen.util 1 :ok :write [0 34] INFO [2022-01-10 12:53:17,035] jepsen worker 85 - jepsen.util 175 :ok :read [28 47] INFO [2022-01-10 12:53:17,036] jepsen worker 1 - jepsen.util 1 :invoke :write [0 22] INFO [2022-01-10 12:53:17,036] jepsen worker 16 - jepsen.util 106 :ok :write [5 47] INFO [2022-01-10 12:53:17,036] jepsen worker 13 - jepsen.util 13 :ok :write [4 46] INFO [2022-01-10 12:53:17,036] jepsen worker 85 - jepsen.util 175 :invoke :write [28 1] INFO [2022-01-10 12:53:17,037] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:17,037] jepsen worker 16 - jepsen.util 106 :invoke :write [5 38] INFO [2022-01-10 12:53:17,038] jepsen worker 13 - jepsen.util 13 :ok :read [4 46] INFO [2022-01-10 12:53:17,039] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:17,040] jepsen worker 13 - jepsen.util 13 :ok :read [4 46] INFO [2022-01-10 12:53:17,040] jepsen worker 66 - jepsen.util 156 :ok :write [22 38] INFO [2022-01-10 12:53:17,041] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:17,041] jepsen worker 66 - jepsen.util 156 :invoke :write [22 10] INFO [2022-01-10 12:53:17,042] jepsen worker 39 - jepsen.util 39 :ok :write [13 4] INFO [2022-01-10 12:53:17,042] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,042] jepsen worker 13 - jepsen.util 13 :ok :read [4 46] INFO [2022-01-10 12:53:17,043] jepsen worker 36 - jepsen.util 36 :ok :write [12 47] INFO [2022-01-10 12:53:17,043] jepsen worker 39 - jepsen.util 39 :ok :read [13 4] INFO [2022-01-10 12:53:17,043] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:17,044] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,044] jepsen worker 13 - jepsen.util 13 :ok :read [4 46] INFO [2022-01-10 12:53:17,045] jepsen worker 36 - jepsen.util 36 :invoke :write [12 43] INFO [2022-01-10 12:53:17,045] jepsen worker 39 - jepsen.util 39 :ok :read [13 4] INFO [2022-01-10 12:53:17,045] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:17,046] jepsen worker 39 - jepsen.util 39 :invoke :write [13 38] INFO [2022-01-10 12:53:17,047] jepsen worker 13 - jepsen.util 13 :ok :read [4 46] INFO [2022-01-10 12:53:17,047] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:17,048] jepsen worker 13 - jepsen.util 13 :ok :read [4 46] INFO [2022-01-10 12:53:17,048] jepsen worker 23 - jepsen.util 23 :ok :write [7 42] INFO [2022-01-10 12:53:17,049] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:17,049] jepsen worker 4 - jepsen.util 4 :ok :write [1 46] INFO [2022-01-10 12:53:17,050] jepsen worker 13 - jepsen.util 13 :invoke :write [4 0] INFO [2022-01-10 12:53:17,050] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,050] jepsen worker 23 - jepsen.util 23 :ok :read [7 42] INFO [2022-01-10 12:53:17,050] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:17,051] jepsen worker 23 - jepsen.util 23 :ok :read [7 42] INFO [2022-01-10 12:53:17,052] jepsen worker 4 - jepsen.util 4 :ok :read [1 46] INFO [2022-01-10 12:53:17,052] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:17,053] jepsen worker 4 - jepsen.util 4 :invoke :write [1 37] INFO [2022-01-10 12:53:17,053] jepsen worker 23 - jepsen.util 23 :ok :read [7 42] INFO [2022-01-10 12:53:17,054] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:17,055] jepsen worker 23 - jepsen.util 23 :ok :read [7 42] INFO [2022-01-10 12:53:17,055] jepsen worker 78 - jepsen.util 78 :ok :write [26 39] INFO [2022-01-10 12:53:17,055] jepsen worker 23 - jepsen.util 23 :invoke :write [7 35] INFO [2022-01-10 12:53:17,056] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:17,057] jepsen worker 78 - jepsen.util 78 :ok :read [26 39] INFO [2022-01-10 12:53:17,057] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:17,058] jepsen worker 78 - jepsen.util 78 :ok :read [26 39] INFO [2022-01-10 12:53:17,058] jepsen worker 44 - jepsen.util 44 :ok :write [14 43] INFO [2022-01-10 12:53:17,059] jepsen worker 78 - jepsen.util 78 :invoke :write [26 14] INFO [2022-01-10 12:53:17,059] jepsen worker 44 - jepsen.util 44 :invoke :write [14 32] INFO [2022-01-10 12:53:17,062] jepsen worker 52 - jepsen.util 52 :ok :write [17 19] INFO [2022-01-10 12:53:17,062] jepsen worker 52 - jepsen.util 52 :invoke :write [17 8] INFO [2022-01-10 12:53:17,064] jepsen worker 77 - jepsen.util 167 :ok :write [25 37] INFO [2022-01-10 12:53:17,065] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:17,065] jepsen worker 77 - jepsen.util 167 :ok :read [25 37] INFO [2022-01-10 12:53:17,066] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:17,067] jepsen worker 77 - jepsen.util 167 :ok :read [25 37] INFO [2022-01-10 12:53:17,067] jepsen worker 70 - jepsen.util 160 :ok :write [23 41] INFO [2022-01-10 12:53:17,067] jepsen worker 77 - jepsen.util 167 :invoke :write [25 4] INFO [2022-01-10 12:53:17,068] jepsen worker 87 - jepsen.util 87 :ok :write [29 33] INFO [2022-01-10 12:53:17,068] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:17,068] jepsen worker 87 - jepsen.util 87 :invoke :write [29 34] INFO [2022-01-10 12:53:17,069] jepsen worker 70 - jepsen.util 160 :ok :read [23 41] INFO [2022-01-10 12:53:17,069] jepsen worker 70 - jepsen.util 160 :invoke :write [23 42] INFO [2022-01-10 12:53:17,072] jepsen worker 62 - jepsen.util 152 :ok :write [20 23] INFO [2022-01-10 12:53:17,072] jepsen worker 62 - jepsen.util 152 :invoke :write [20 45] INFO [2022-01-10 12:53:17,075] jepsen worker 85 - jepsen.util 175 :ok :write [28 1] INFO [2022-01-10 12:53:17,075] jepsen worker 85 - jepsen.util 175 :invoke :write [28 16] INFO [2022-01-10 12:53:17,078] jepsen worker 1 - jepsen.util 1 :ok :write [0 22] INFO [2022-01-10 12:53:17,079] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,079] jepsen worker 13 - jepsen.util 13 :ok :write [4 0] INFO [2022-01-10 12:53:17,079] jepsen worker 1 - jepsen.util 1 :ok :read [0 22] INFO [2022-01-10 12:53:17,080] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:17,080] jepsen worker 49 - jepsen.util 139 :ok :write [16 15] INFO [2022-01-10 12:53:17,080] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,080] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:17,081] jepsen worker 1 - jepsen.util 1 :ok :read [0 22] INFO [2022-01-10 12:53:17,081] jepsen worker 49 - jepsen.util 139 :ok :read [16 15] INFO [2022-01-10 12:53:17,082] jepsen worker 13 - jepsen.util 13 :ok :read [4 0] INFO [2022-01-10 12:53:17,082] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,082] jepsen worker 13 - jepsen.util 13 :invoke :write [4 20] INFO [2022-01-10 12:53:17,083] jepsen worker 1 - jepsen.util 1 :ok :read [0 22] INFO [2022-01-10 12:53:17,083] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:17,083] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,084] jepsen worker 49 - jepsen.util 139 :ok :read [16 15] INFO [2022-01-10 12:53:17,084] jepsen worker 1 - jepsen.util 1 :ok :read [0 22] INFO [2022-01-10 12:53:17,084] jepsen worker 49 - jepsen.util 139 :invoke :write [16 27] INFO [2022-01-10 12:53:17,085] jepsen worker 1 - jepsen.util 1 :invoke :write [0 35] INFO [2022-01-10 12:53:17,087] jepsen worker 36 - jepsen.util 36 :ok :write [12 43] INFO [2022-01-10 12:53:17,088] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,089] jepsen worker 36 - jepsen.util 36 :ok :read [12 43] INFO [2022-01-10 12:53:17,089] jepsen worker 36 - jepsen.util 36 :invoke :write [12 39] INFO [2022-01-10 12:53:17,090] jepsen worker 83 - jepsen.util 173 :ok :write [27 20] INFO [2022-01-10 12:53:17,091] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:17,092] jepsen worker 83 - jepsen.util 173 :ok :read [27 20] INFO [2022-01-10 12:53:17,092] jepsen worker 83 - jepsen.util 173 :invoke :write [27 37] INFO [2022-01-10 12:53:17,095] jepsen worker 8 - jepsen.util 8 :ok :write [2 34] INFO [2022-01-10 12:53:17,095] jepsen worker 8 - jepsen.util 8 :invoke :write [2 47] INFO [2022-01-10 12:53:17,098] jepsen worker 39 - jepsen.util 39 :ok :write [13 38] INFO [2022-01-10 12:53:17,098] jepsen worker 39 - jepsen.util 39 :invoke :write [13 0] INFO [2022-01-10 12:53:17,100] jepsen worker 35 - jepsen.util 125 :ok :write [11 25] INFO [2022-01-10 12:53:17,100] jepsen worker 35 - jepsen.util 125 :invoke :write [11 39] INFO [2022-01-10 12:53:17,103] jepsen worker 87 - jepsen.util 87 :ok :write [29 34] INFO [2022-01-10 12:53:17,104] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,105] jepsen worker 87 - jepsen.util 87 :ok :read [29 34] INFO [2022-01-10 12:53:17,105] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,106] jepsen worker 16 - jepsen.util 106 :ok :write [5 38] INFO [2022-01-10 12:53:17,106] jepsen worker 87 - jepsen.util 87 :ok :read [29 34] INFO [2022-01-10 12:53:17,106] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,107] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:17,107] jepsen worker 87 - jepsen.util 87 :ok :read [29 34] INFO [2022-01-10 12:53:17,108] jepsen worker 16 - jepsen.util 106 :ok :read [5 38] INFO [2022-01-10 12:53:17,108] jepsen worker 87 - jepsen.util 87 :invoke :write [29 1] INFO [2022-01-10 12:53:17,108] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:17,109] jepsen worker 16 - jepsen.util 106 :ok :read [5 38] INFO [2022-01-10 12:53:17,110] jepsen worker 16 - jepsen.util 106 :invoke :write [5 18] INFO [2022-01-10 12:53:17,110] jepsen worker 66 - jepsen.util 156 :ok :write [22 10] INFO [2022-01-10 12:53:17,111] jepsen worker 66 - jepsen.util 156 :invoke :write [22 27] INFO [2022-01-10 12:53:17,115] jepsen worker 23 - jepsen.util 23 :ok :write [7 35] INFO [2022-01-10 12:53:17,116] jepsen worker 23 - jepsen.util 23 :invoke :write [7 39] INFO [2022-01-10 12:53:17,120] jepsen worker 44 - jepsen.util 44 :ok :write [14 32] INFO [2022-01-10 12:53:17,120] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:17,121] jepsen worker 44 - jepsen.util 44 :ok :read [14 32] INFO [2022-01-10 12:53:17,122] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:17,122] jepsen worker 44 - jepsen.util 44 :ok :read [14 32] INFO [2022-01-10 12:53:17,123] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:17,124] jepsen worker 44 - jepsen.util 44 :ok :read [14 32] INFO [2022-01-10 12:53:17,124] jepsen worker 44 - jepsen.util 44 :invoke :write [14 10] INFO [2022-01-10 12:53:17,127] jepsen worker 77 - jepsen.util 167 :ok :write [25 4] INFO [2022-01-10 12:53:17,128] jepsen worker 77 - jepsen.util 167 :invoke :write [25 15] INFO [2022-01-10 12:53:17,138] jepsen worker 49 - jepsen.util 139 :ok :write [16 27] INFO [2022-01-10 12:53:17,139] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:17,140] jepsen worker 49 - jepsen.util 139 :ok :read [16 27] INFO [2022-01-10 12:53:17,140] jepsen worker 49 - jepsen.util 139 :invoke :write [16 39] INFO [2022-01-10 12:53:17,143] jepsen worker 83 - jepsen.util 173 :ok :write [27 37] INFO [2022-01-10 12:53:17,143] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:17,144] jepsen worker 83 - jepsen.util 173 :ok :read [27 37] INFO [2022-01-10 12:53:17,145] jepsen worker 83 - jepsen.util 173 :invoke :write [27 41] INFO [2022-01-10 12:53:17,147] jepsen worker 8 - jepsen.util 8 :ok :write [2 47] INFO [2022-01-10 12:53:17,148] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:17,149] jepsen worker 8 - jepsen.util 8 :ok :read [2 47] INFO [2022-01-10 12:53:17,149] jepsen worker 8 - jepsen.util 8 :invoke :write [2 16] INFO [2022-01-10 12:53:17,152] jepsen worker 35 - jepsen.util 125 :ok :write [11 39] INFO [2022-01-10 12:53:17,153] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:17,154] jepsen worker 35 - jepsen.util 125 :ok :read [11 39] INFO [2022-01-10 12:53:17,155] jepsen worker 35 - jepsen.util 125 :invoke :write [11 20] INFO [2022-01-10 12:53:17,157] jepsen worker 16 - jepsen.util 106 :ok :write [5 18] INFO [2022-01-10 12:53:17,158] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:17,158] jepsen worker 16 - jepsen.util 106 :ok :read [5 18] INFO [2022-01-10 12:53:17,159] jepsen worker 16 - jepsen.util 106 :invoke :read [5 nil] INFO [2022-01-10 12:53:17,160] jepsen worker 16 - jepsen.util 106 :ok :read [5 18] INFO [2022-01-10 12:53:17,160] jepsen worker 16 - jepsen.util 106 :invoke :write [5 2] INFO [2022-01-10 12:53:17,163] jepsen worker 66 - jepsen.util 156 :ok :write [22 27] INFO [2022-01-10 12:53:17,163] jepsen worker 66 - jepsen.util 156 :invoke :write [22 22] INFO [2022-01-10 12:53:17,179] jepsen worker 49 - jepsen.util 139 :ok :write [16 39] INFO [2022-01-10 12:53:17,179] jepsen worker 49 - jepsen.util 139 :invoke :write [16 31] INFO [2022-01-10 12:53:17,180] jepsen worker 62 - jepsen.util 152 :ok :write [20 45] INFO [2022-01-10 12:53:17,181] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:17,182] jepsen worker 62 - jepsen.util 152 :ok :read [20 45] INFO [2022-01-10 12:53:17,182] jepsen worker 4 - jepsen.util 4 :ok :write [1 37] INFO [2022-01-10 12:53:17,182] jepsen worker 62 - jepsen.util 152 :invoke :write [20 44] INFO [2022-01-10 12:53:17,183] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,184] jepsen worker 4 - jepsen.util 4 :ok :read [1 37] INFO [2022-01-10 12:53:17,184] jepsen worker 83 - jepsen.util 173 :ok :write [27 41] INFO [2022-01-10 12:53:17,184] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,184] jepsen worker 1 - jepsen.util 1 :ok :write [0 35] INFO [2022-01-10 12:53:17,185] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:17,185] jepsen worker 4 - jepsen.util 4 :ok :read [1 37] INFO [2022-01-10 12:53:17,185] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,185] jepsen worker 83 - jepsen.util 173 :ok :read [27 41] INFO [2022-01-10 12:53:17,185] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,186] jepsen worker 1 - jepsen.util 1 :ok :read [0 35] INFO [2022-01-10 12:53:17,186] jepsen worker 83 - jepsen.util 173 :invoke :write [27 29] INFO [2022-01-10 12:53:17,186] jepsen worker 4 - jepsen.util 4 :ok :read [1 37] INFO [2022-01-10 12:53:17,186] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,187] jepsen worker 4 - jepsen.util 4 :invoke :write [1 44] INFO [2022-01-10 12:53:17,187] jepsen worker 1 - jepsen.util 1 :ok :read [0 35] INFO [2022-01-10 12:53:17,188] jepsen worker 1 - jepsen.util 1 :invoke :write [0 26] INFO [2022-01-10 12:53:17,188] jepsen worker 8 - jepsen.util 8 :ok :write [2 16] INFO [2022-01-10 12:53:17,188] jepsen worker 8 - jepsen.util 8 :invoke :write [2 41] INFO [2022-01-10 12:53:17,189] jepsen worker 78 - jepsen.util 78 :ok :write [26 14] INFO [2022-01-10 12:53:17,189] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:17,190] jepsen worker 78 - jepsen.util 78 :ok :read [26 14] INFO [2022-01-10 12:53:17,190] jepsen worker 39 - jepsen.util 39 :ok :write [13 0] INFO [2022-01-10 12:53:17,191] jepsen worker 78 - jepsen.util 78 :invoke :write [26 4] INFO [2022-01-10 12:53:17,192] jepsen worker 39 - jepsen.util 39 :invoke :write [13 15] INFO [2022-01-10 12:53:17,195] jepsen worker 52 - jepsen.util 52 :ok :write [17 8] INFO [2022-01-10 12:53:17,195] jepsen worker 52 - jepsen.util 52 :invoke :write [17 30] INFO [2022-01-10 12:53:17,195] jepsen worker 87 - jepsen.util 87 :ok :write [29 1] INFO [2022-01-10 12:53:17,196] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,197] jepsen worker 87 - jepsen.util 87 :ok :read [29 1] INFO [2022-01-10 12:53:17,197] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,198] jepsen worker 87 - jepsen.util 87 :ok :read [29 1] INFO [2022-01-10 12:53:17,198] jepsen worker 87 - jepsen.util 87 :invoke :write [29 10] INFO [2022-01-10 12:53:17,202] jepsen worker 23 - jepsen.util 23 :ok :write [7 39] INFO [2022-01-10 12:53:17,202] jepsen worker 70 - jepsen.util 160 :ok :write [23 42] INFO [2022-01-10 12:53:17,203] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:17,203] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:17,203] jepsen worker 23 - jepsen.util 23 :ok :read [7 39] INFO [2022-01-10 12:53:17,204] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:17,204] jepsen worker 70 - jepsen.util 160 :ok :read [23 42] INFO [2022-01-10 12:53:17,205] jepsen worker 23 - jepsen.util 23 :ok :read [7 39] INFO [2022-01-10 12:53:17,205] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:17,206] jepsen worker 44 - jepsen.util 44 :ok :write [14 10] INFO [2022-01-10 12:53:17,206] jepsen worker 70 - jepsen.util 160 :ok :read [23 42] INFO [2022-01-10 12:53:17,206] jepsen worker 23 - jepsen.util 23 :invoke :write [7 24] INFO [2022-01-10 12:53:17,207] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:17,207] jepsen worker 44 - jepsen.util 44 :invoke :write [14 43] INFO [2022-01-10 12:53:17,207] jepsen worker 70 - jepsen.util 160 :ok :read [23 42] INFO [2022-01-10 12:53:17,208] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:17,209] jepsen worker 70 - jepsen.util 160 :ok :read [23 42] INFO [2022-01-10 12:53:17,209] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:17,210] jepsen worker 70 - jepsen.util 160 :ok :read [23 42] INFO [2022-01-10 12:53:17,210] jepsen worker 85 - jepsen.util 175 :ok :write [28 16] INFO [2022-01-10 12:53:17,210] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:17,211] jepsen worker 85 - jepsen.util 175 :invoke :write [28 14] INFO [2022-01-10 12:53:17,211] jepsen worker 77 - jepsen.util 167 :ok :write [25 15] INFO [2022-01-10 12:53:17,211] jepsen worker 70 - jepsen.util 160 :ok :read [23 42] INFO [2022-01-10 12:53:17,212] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:17,212] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:17,213] jepsen worker 77 - jepsen.util 167 :ok :read [25 15] INFO [2022-01-10 12:53:17,213] jepsen worker 70 - jepsen.util 160 :ok :read [23 42] INFO [2022-01-10 12:53:17,214] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:17,214] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:17,214] jepsen worker 77 - jepsen.util 167 :ok :read [25 15] INFO [2022-01-10 12:53:17,215] jepsen worker 70 - jepsen.util 160 :ok :read [23 42] INFO [2022-01-10 12:53:17,215] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:17,216] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:17,216] jepsen worker 77 - jepsen.util 167 :ok :read [25 15] INFO [2022-01-10 12:53:17,216] jepsen worker 13 - jepsen.util 13 :ok :write [4 20] INFO [2022-01-10 12:53:17,217] jepsen worker 70 - jepsen.util 160 :ok :read [23 42] INFO [2022-01-10 12:53:17,217] jepsen worker 77 - jepsen.util 167 :invoke :write [25 5] INFO [2022-01-10 12:53:17,217] jepsen worker 70 - jepsen.util 160 :invoke :write [23 25] INFO [2022-01-10 12:53:17,218] jepsen worker 13 - jepsen.util 13 :invoke :write [4 6] INFO [2022-01-10 12:53:17,222] jepsen worker 1 - jepsen.util 1 :ok :write [0 26] INFO [2022-01-10 12:53:17,222] jepsen worker 1 - jepsen.util 1 :invoke :write [0 49] INFO [2022-01-10 12:53:17,224] jepsen worker 36 - jepsen.util 36 :ok :write [12 39] INFO [2022-01-10 12:53:17,225] jepsen worker 36 - jepsen.util 36 :invoke :write [12 10] INFO [2022-01-10 12:53:17,226] jepsen worker 39 - jepsen.util 39 :ok :write [13 15] INFO [2022-01-10 12:53:17,227] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,227] jepsen worker 39 - jepsen.util 39 :ok :read [13 15] INFO [2022-01-10 12:53:17,228] jepsen worker 39 - jepsen.util 39 :invoke :write [13 25] INFO [2022-01-10 12:53:17,229] jepsen worker 4 - jepsen.util 4 :ok :write [1 44] INFO [2022-01-10 12:53:17,229] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,230] jepsen worker 4 - jepsen.util 4 :ok :read [1 44] INFO [2022-01-10 12:53:17,231] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,231] jepsen worker 4 - jepsen.util 4 :ok :read [1 44] INFO [2022-01-10 12:53:17,232] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,232] jepsen worker 4 - jepsen.util 4 :ok :read [1 44] INFO [2022-01-10 12:53:17,233] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,234] jepsen worker 4 - jepsen.util 4 :ok :read [1 44] INFO [2022-01-10 12:53:17,234] jepsen worker 4 - jepsen.util 4 :invoke :write [1 1] INFO [2022-01-10 12:53:17,324] jepsen worker 51 - jepsen.util 51 :info :write [17 18] :RequestTimeoutException INFO [2022-01-10 12:53:17,325] jepsen worker 51 - jepsen.util 141 :invoke :write [17 32] INFO [2022-01-10 12:53:17,328] jepsen worker 85 - jepsen.util 175 :ok :write [28 14] INFO [2022-01-10 12:53:17,328] jepsen worker 87 - jepsen.util 87 :ok :write [29 10] INFO [2022-01-10 12:53:17,329] jepsen worker 85 - jepsen.util 175 :invoke :write [28 46] INFO [2022-01-10 12:53:17,329] jepsen worker 87 - jepsen.util 87 :invoke :write [29 5] INFO [2022-01-10 12:53:17,334] jepsen worker 1 - jepsen.util 1 :ok :write [0 49] INFO [2022-01-10 12:53:17,335] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,336] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:17,336] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,337] jepsen worker 1 - jepsen.util 1 :ok :read [0 49] INFO [2022-01-10 12:53:17,338] jepsen worker 1 - jepsen.util 1 :invoke :write [0 45] INFO [2022-01-10 12:53:17,338] jepsen worker 36 - jepsen.util 36 :ok :write [12 10] INFO [2022-01-10 12:53:17,339] jepsen worker 36 - jepsen.util 36 :invoke :write [12 18] INFO [2022-01-10 12:53:17,342] jepsen worker 4 - jepsen.util 4 :ok :write [1 1] INFO [2022-01-10 12:53:17,342] jepsen worker 39 - jepsen.util 39 :ok :write [13 25] INFO [2022-01-10 12:53:17,343] jepsen worker 4 - jepsen.util 4 :invoke :write [1 10] INFO [2022-01-10 12:53:17,344] jepsen worker 39 - jepsen.util 39 :invoke :write [13 35] INFO [2022-01-10 12:53:17,348] jepsen worker 85 - jepsen.util 175 :ok :write [28 46] INFO [2022-01-10 12:53:17,348] jepsen worker 85 - jepsen.util 175 :invoke :write [28 9] INFO [2022-01-10 12:53:17,350] jepsen worker 87 - jepsen.util 87 :ok :write [29 5] INFO [2022-01-10 12:53:17,350] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,351] jepsen worker 87 - jepsen.util 87 :ok :read [29 5] INFO [2022-01-10 12:53:17,352] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,353] jepsen worker 1 - jepsen.util 1 :ok :write [0 45] INFO [2022-01-10 12:53:17,353] jepsen worker 87 - jepsen.util 87 :ok :read [29 5] INFO [2022-01-10 12:53:17,353] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,354] jepsen worker 87 - jepsen.util 87 :invoke :write [29 40] INFO [2022-01-10 12:53:17,354] jepsen worker 36 - jepsen.util 36 :ok :write [12 18] INFO [2022-01-10 12:53:17,354] jepsen worker 1 - jepsen.util 1 :ok :read [0 45] INFO [2022-01-10 12:53:17,355] jepsen worker 1 - jepsen.util 1 :invoke :write [0 0] INFO [2022-01-10 12:53:17,356] jepsen worker 36 - jepsen.util 36 :invoke :write [12 17] INFO [2022-01-10 12:53:17,358] jepsen worker 39 - jepsen.util 39 :ok :write [13 35] INFO [2022-01-10 12:53:17,359] jepsen worker 39 - jepsen.util 39 :invoke :write [13 19] INFO [2022-01-10 12:53:17,361] jepsen worker 4 - jepsen.util 4 :ok :write [1 10] INFO [2022-01-10 12:53:17,362] jepsen worker 4 - jepsen.util 4 :invoke :write [1 27] INFO [2022-01-10 12:53:17,367] jepsen worker 85 - jepsen.util 175 :ok :write [28 9] INFO [2022-01-10 12:53:17,367] jepsen worker 87 - jepsen.util 87 :ok :write [29 40] INFO [2022-01-10 12:53:17,367] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,368] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,368] jepsen worker 85 - jepsen.util 175 :ok :read [28 9] INFO [2022-01-10 12:53:17,369] jepsen worker 87 - jepsen.util 87 :ok :read [29 40] INFO [2022-01-10 12:53:17,369] jepsen worker 85 - jepsen.util 175 :invoke :write [28 35] INFO [2022-01-10 12:53:17,369] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,370] jepsen worker 87 - jepsen.util 87 :ok :read [29 40] INFO [2022-01-10 12:53:17,371] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,371] jepsen worker 1 - jepsen.util 1 :ok :write [0 0] INFO [2022-01-10 12:53:17,372] jepsen worker 87 - jepsen.util 87 :ok :read [29 40] INFO [2022-01-10 12:53:17,372] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,372] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,373] jepsen worker 1 - jepsen.util 1 :ok :read [0 0] INFO [2022-01-10 12:53:17,373] jepsen worker 1 - jepsen.util 1 :invoke :write [0 46] INFO [2022-01-10 12:53:17,374] jepsen worker 36 - jepsen.util 36 :ok :write [12 17] INFO [2022-01-10 12:53:17,375] jepsen worker 87 - jepsen.util 87 :ok :read [29 40] INFO [2022-01-10 12:53:17,375] jepsen worker 36 - jepsen.util 36 :invoke :write [12 41] INFO [2022-01-10 12:53:17,376] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,377] jepsen worker 87 - jepsen.util 87 :ok :read [29 40] INFO [2022-01-10 12:53:17,377] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,378] jepsen worker 39 - jepsen.util 39 :ok :write [13 19] INFO [2022-01-10 12:53:17,379] jepsen worker 87 - jepsen.util 87 :ok :read [29 40] INFO [2022-01-10 12:53:17,379] jepsen worker 39 - jepsen.util 39 :invoke :write [13 28] INFO [2022-01-10 12:53:17,379] jepsen worker 87 - jepsen.util 87 :invoke :write [29 34] INFO [2022-01-10 12:53:17,380] jepsen worker 4 - jepsen.util 4 :ok :write [1 27] INFO [2022-01-10 12:53:17,381] jepsen worker 4 - jepsen.util 4 :invoke :write [1 30] INFO [2022-01-10 12:53:17,389] jepsen worker 85 - jepsen.util 175 :ok :write [28 35] INFO [2022-01-10 12:53:17,390] jepsen worker 1 - jepsen.util 1 :ok :write [0 46] INFO [2022-01-10 12:53:17,391] jepsen worker 1 - jepsen.util 1 :invoke :write [0 7] INFO [2022-01-10 12:53:17,391] jepsen worker 85 - jepsen.util 175 :invoke :write [28 46] INFO [2022-01-10 12:53:17,394] jepsen worker 36 - jepsen.util 36 :ok :write [12 41] INFO [2022-01-10 12:53:17,395] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,396] jepsen worker 36 - jepsen.util 36 :ok :read [12 41] INFO [2022-01-10 12:53:17,396] jepsen worker 36 - jepsen.util 36 :invoke :write [12 31] INFO [2022-01-10 12:53:17,396] jepsen worker 39 - jepsen.util 39 :ok :write [13 28] INFO [2022-01-10 12:53:17,397] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,398] jepsen worker 39 - jepsen.util 39 :ok :read [13 28] INFO [2022-01-10 12:53:17,398] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,399] jepsen worker 39 - jepsen.util 39 :ok :read [13 28] INFO [2022-01-10 12:53:17,399] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,400] jepsen worker 39 - jepsen.util 39 :ok :read [13 28] INFO [2022-01-10 12:53:17,401] jepsen worker 39 - jepsen.util 39 :invoke :write [13 46] INFO [2022-01-10 12:53:17,402] jepsen worker 87 - jepsen.util 87 :ok :write [29 34] INFO [2022-01-10 12:53:17,403] jepsen worker 87 - jepsen.util 87 :invoke :write [29 25] INFO [2022-01-10 12:53:17,404] jepsen worker 4 - jepsen.util 4 :ok :write [1 30] INFO [2022-01-10 12:53:17,405] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,406] jepsen worker 4 - jepsen.util 4 :ok :read [1 30] INFO [2022-01-10 12:53:17,406] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,407] jepsen worker 4 - jepsen.util 4 :ok :read [1 30] INFO [2022-01-10 12:53:17,407] jepsen worker 4 - jepsen.util 4 :invoke :write [1 15] INFO [2022-01-10 12:53:17,414] jepsen worker 1 - jepsen.util 1 :ok :write [0 7] INFO [2022-01-10 12:53:17,414] jepsen worker 85 - jepsen.util 175 :ok :write [28 46] INFO [2022-01-10 12:53:17,415] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,415] jepsen worker 85 - jepsen.util 175 :invoke :write [28 22] INFO [2022-01-10 12:53:17,416] jepsen worker 1 - jepsen.util 1 :ok :read [0 7] INFO [2022-01-10 12:53:17,416] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,417] jepsen worker 1 - jepsen.util 1 :ok :read [0 7] INFO [2022-01-10 12:53:17,418] jepsen worker 1 - jepsen.util 1 :invoke :write [0 14] INFO [2022-01-10 12:53:17,418] jepsen worker 38 - jepsen.util 38 :ok :write [12 29] INFO [2022-01-10 12:53:17,419] jepsen worker 38 - jepsen.util 38 :invoke :write [12 5] INFO [2022-01-10 12:53:17,420] jepsen worker 39 - jepsen.util 39 :ok :write [13 46] INFO [2022-01-10 12:53:17,420] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,421] jepsen worker 39 - jepsen.util 39 :ok :read [13 46] INFO [2022-01-10 12:53:17,422] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,423] jepsen worker 39 - jepsen.util 39 :ok :read [13 46] INFO [2022-01-10 12:53:17,423] jepsen worker 39 - jepsen.util 39 :invoke :write [13 3] INFO [2022-01-10 12:53:17,427] jepsen worker 87 - jepsen.util 87 :ok :write [29 25] INFO [2022-01-10 12:53:17,427] jepsen worker 87 - jepsen.util 87 :invoke :write [29 25] INFO [2022-01-10 12:53:17,427] jepsen worker 4 - jepsen.util 4 :ok :write [1 15] INFO [2022-01-10 12:53:17,428] jepsen worker 4 - jepsen.util 4 :invoke :write [1 28] INFO [2022-01-10 12:53:17,435] jepsen worker 1 - jepsen.util 1 :ok :write [0 14] INFO [2022-01-10 12:53:17,435] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,436] jepsen worker 85 - jepsen.util 175 :ok :write [28 22] INFO [2022-01-10 12:53:17,436] jepsen worker 1 - jepsen.util 1 :ok :read [0 14] INFO [2022-01-10 12:53:17,436] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,437] jepsen worker 85 - jepsen.util 175 :ok :read [28 22] INFO [2022-01-10 12:53:17,437] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,437] jepsen worker 85 - jepsen.util 175 :invoke :write [28 19] INFO [2022-01-10 12:53:17,438] jepsen worker 1 - jepsen.util 1 :ok :read [0 14] INFO [2022-01-10 12:53:17,438] jepsen worker 1 - jepsen.util 1 :invoke :write [0 17] INFO [2022-01-10 12:53:17,441] jepsen worker 38 - jepsen.util 38 :ok :write [12 5] INFO [2022-01-10 12:53:17,441] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,442] jepsen worker 39 - jepsen.util 39 :ok :write [13 3] INFO [2022-01-10 12:53:17,442] jepsen worker 38 - jepsen.util 38 :ok :read [12 5] INFO [2022-01-10 12:53:17,443] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,443] jepsen worker 38 - jepsen.util 38 :invoke :write [12 32] INFO [2022-01-10 12:53:17,444] jepsen worker 39 - jepsen.util 39 :ok :read [13 3] INFO [2022-01-10 12:53:17,444] jepsen worker 39 - jepsen.util 39 :invoke :write [13 4] INFO [2022-01-10 12:53:17,447] jepsen worker 87 - jepsen.util 87 :ok :write [29 25] INFO [2022-01-10 12:53:17,448] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,448] jepsen worker 87 - jepsen.util 87 :ok :read [29 25] INFO [2022-01-10 12:53:17,449] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,450] jepsen worker 4 - jepsen.util 4 :ok :write [1 28] INFO [2022-01-10 12:53:17,450] jepsen worker 87 - jepsen.util 87 :ok :read [29 25] INFO [2022-01-10 12:53:17,450] jepsen worker 4 - jepsen.util 4 :invoke :write [1 36] INFO [2022-01-10 12:53:17,451] jepsen worker 87 - jepsen.util 87 :invoke :write [29 23] INFO [2022-01-10 12:53:17,456] jepsen worker 1 - jepsen.util 1 :ok :write [0 17] INFO [2022-01-10 12:53:17,456] jepsen worker 85 - jepsen.util 175 :ok :write [28 19] INFO [2022-01-10 12:53:17,457] jepsen worker 1 - jepsen.util 1 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,458] jepsen worker 85 - jepsen.util 175 :invoke :write [28 31] INFO [2022-01-10 12:53:17,458] jepsen worker 1 - jepsen.util 1 :ok :read [0 17] INFO [2022-01-10 12:53:17,459] jepsen worker 1 - jepsen.util 1 :invoke :write [0 45] INFO [2022-01-10 12:53:17,462] jepsen worker 38 - jepsen.util 38 :ok :write [12 32] INFO [2022-01-10 12:53:17,462] jepsen worker 39 - jepsen.util 39 :ok :write [13 4] INFO [2022-01-10 12:53:17,462] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,463] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,463] jepsen worker 38 - jepsen.util 38 :ok :read [12 32] INFO [2022-01-10 12:53:17,464] jepsen worker 39 - jepsen.util 39 :ok :read [13 4] INFO [2022-01-10 12:53:17,464] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:17,465] jepsen worker 39 - jepsen.util 39 :invoke :write [13 17] INFO [2022-01-10 12:53:17,467] jepsen worker 87 - jepsen.util 87 :ok :write [29 23] INFO [2022-01-10 12:53:17,468] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,469] jepsen worker 4 - jepsen.util 4 :ok :write [1 36] INFO [2022-01-10 12:53:17,469] jepsen worker 87 - jepsen.util 87 :ok :read [29 23] INFO [2022-01-10 12:53:17,469] jepsen worker 4 - jepsen.util 4 :invoke :write [1 1] INFO [2022-01-10 12:53:17,470] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,471] jepsen worker 87 - jepsen.util 87 :ok :read [29 23] INFO [2022-01-10 12:53:17,472] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,473] jepsen worker 87 - jepsen.util 87 :ok :read [29 23] INFO [2022-01-10 12:53:17,474] jepsen worker 87 - jepsen.util 87 :invoke :write [29 33] INFO [2022-01-10 12:53:17,475] jepsen worker 1 - jepsen.util 1 :ok :write [0 45] INFO [2022-01-10 12:53:17,537] jepsen worker 85 - jepsen.util 175 :ok :write [28 31] INFO [2022-01-10 12:53:17,537] jepsen worker 87 - jepsen.util 87 :ok :write [29 33] INFO [2022-01-10 12:53:17,537] jepsen worker 4 - jepsen.util 4 :ok :write [1 1] INFO [2022-01-10 12:53:17,539] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:17,541] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,541] jepsen worker 39 - jepsen.util 39 :ok :write [13 17] INFO [2022-01-10 12:53:17,542] jepsen worker 38 - jepsen.util 38 :invoke :write [12 24] INFO [2022-01-10 12:53:17,542] jepsen worker 4 - jepsen.util 4 :invoke :write [1 21] INFO [2022-01-10 12:53:17,543] jepsen worker 39 - jepsen.util 39 :invoke :write [13 18] INFO [2022-01-10 12:53:17,543] jepsen worker 85 - jepsen.util 175 :ok :read [28 31] INFO [2022-01-10 12:53:17,544] jepsen worker 1 - jepsen.util 1 :invoke :write [0 40] INFO [2022-01-10 12:53:17,544] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,544] jepsen worker 87 - jepsen.util 87 :invoke :write [29 15] INFO [2022-01-10 12:53:17,545] jepsen worker 85 - jepsen.util 175 :ok :read [28 31] INFO [2022-01-10 12:53:17,545] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,546] jepsen worker 85 - jepsen.util 175 :ok :read [28 31] INFO [2022-01-10 12:53:17,546] jepsen worker 85 - jepsen.util 175 :invoke :write [28 36] INFO [2022-01-10 12:53:17,617] jepsen worker 0 - jepsen.util 90 :ok :write [0 4] INFO [2022-01-10 12:53:17,617] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,618] jepsen worker 0 - jepsen.util 90 :ok :read [0 4] INFO [2022-01-10 12:53:17,619] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,620] jepsen worker 0 - jepsen.util 90 :ok :read [0 4] INFO [2022-01-10 12:53:17,620] jepsen worker 38 - jepsen.util 38 :ok :write [12 24] INFO [2022-01-10 12:53:17,621] jepsen worker 0 - jepsen.util 90 :invoke :write [0 6] INFO [2022-01-10 12:53:17,622] jepsen worker 38 - jepsen.util 38 :invoke :write [12 22] INFO [2022-01-10 12:53:17,622] jepsen worker 39 - jepsen.util 39 :ok :write [13 18] INFO [2022-01-10 12:53:17,623] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,624] jepsen worker 39 - jepsen.util 39 :ok :read [13 18] INFO [2022-01-10 12:53:17,624] jepsen worker 39 - jepsen.util 39 :invoke :write [13 9] INFO [2022-01-10 12:53:17,634] jepsen worker 0 - jepsen.util 90 :ok :write [0 6] INFO [2022-01-10 12:53:17,634] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,635] jepsen worker 85 - jepsen.util 175 :ok :write [28 36] INFO [2022-01-10 12:53:17,635] jepsen worker 0 - jepsen.util 90 :ok :read [0 6] INFO [2022-01-10 12:53:17,636] jepsen worker 85 - jepsen.util 175 :invoke :write [28 39] INFO [2022-01-10 12:53:17,637] jepsen worker 0 - jepsen.util 90 :invoke :write [0 11] INFO [2022-01-10 12:53:17,641] jepsen worker 38 - jepsen.util 38 :ok :write [12 22] INFO [2022-01-10 12:53:17,641] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,641] jepsen worker 39 - jepsen.util 39 :ok :write [13 9] INFO [2022-01-10 12:53:17,642] jepsen worker 38 - jepsen.util 38 :ok :read [12 22] INFO [2022-01-10 12:53:17,642] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,643] jepsen worker 38 - jepsen.util 38 :invoke :write [12 46] INFO [2022-01-10 12:53:17,643] jepsen worker 39 - jepsen.util 39 :ok :read [13 9] INFO [2022-01-10 12:53:17,644] jepsen worker 39 - jepsen.util 39 :invoke :write [13 21] INFO [2022-01-10 12:53:17,647] jepsen worker 0 - jepsen.util 90 :ok :write [0 11] INFO [2022-01-10 12:53:17,648] jepsen worker 0 - jepsen.util 90 :invoke :write [0 26] INFO [2022-01-10 12:53:17,649] jepsen worker 85 - jepsen.util 175 :ok :write [28 39] INFO [2022-01-10 12:53:17,649] jepsen worker 85 - jepsen.util 175 :invoke :write [28 10] INFO [2022-01-10 12:53:17,747] jepsen worker 39 - jepsen.util 39 :ok :write [13 21] INFO [2022-01-10 12:53:17,748] jepsen worker 39 - jepsen.util 39 :invoke :write [13 47] INFO [2022-01-10 12:53:17,748] jepsen worker 38 - jepsen.util 38 :ok :write [12 46] INFO [2022-01-10 12:53:17,749] jepsen worker 38 - jepsen.util 38 :invoke :write [12 25] INFO [2022-01-10 12:53:17,754] jepsen worker 0 - jepsen.util 90 :ok :write [0 26] INFO [2022-01-10 12:53:17,755] jepsen worker 0 - jepsen.util 90 :invoke :write [0 28] INFO [2022-01-10 12:53:17,756] jepsen worker 85 - jepsen.util 175 :ok :write [28 10] INFO [2022-01-10 12:53:17,758] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,759] jepsen worker 85 - jepsen.util 175 :ok :read [28 10] INFO [2022-01-10 12:53:17,759] jepsen worker 85 - jepsen.util 175 :invoke :write [28 22] INFO [2022-01-10 12:53:17,763] jepsen worker 39 - jepsen.util 39 :ok :write [13 47] INFO [2022-01-10 12:53:17,763] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,764] jepsen worker 3 - jepsen.util 3 :ok :write [1 38] INFO [2022-01-10 12:53:17,764] jepsen worker 39 - jepsen.util 39 :ok :read [13 47] INFO [2022-01-10 12:53:17,765] jepsen worker 3 - jepsen.util 3 :invoke :write [1 15] INFO [2022-01-10 12:53:17,765] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,766] jepsen worker 39 - jepsen.util 39 :ok :read [13 47] INFO [2022-01-10 12:53:17,767] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,768] jepsen worker 39 - jepsen.util 39 :ok :read [13 47] INFO [2022-01-10 12:53:17,768] jepsen worker 39 - jepsen.util 39 :invoke :write [13 27] INFO [2022-01-10 12:53:17,769] jepsen worker 0 - jepsen.util 90 :ok :write [0 28] INFO [2022-01-10 12:53:17,769] jepsen worker 0 - jepsen.util 90 :invoke :write [0 24] INFO [2022-01-10 12:53:17,771] jepsen worker 38 - jepsen.util 38 :ok :write [12 25] INFO [2022-01-10 12:53:17,772] jepsen worker 38 - jepsen.util 38 :invoke :write [12 1] INFO [2022-01-10 12:53:17,777] jepsen worker 85 - jepsen.util 175 :ok :write [28 22] INFO [2022-01-10 12:53:17,778] jepsen worker 85 - jepsen.util 175 :invoke :write [28 37] INFO [2022-01-10 12:53:17,780] jepsen worker 0 - jepsen.util 90 :ok :write [0 24] INFO [2022-01-10 12:53:17,780] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,781] jepsen worker 0 - jepsen.util 90 :ok :read [0 24] INFO [2022-01-10 12:53:17,782] jepsen worker 0 - jepsen.util 90 :invoke :write [0 1] INFO [2022-01-10 12:53:17,784] jepsen worker 3 - jepsen.util 3 :ok :write [1 15] INFO [2022-01-10 12:53:17,785] jepsen worker 3 - jepsen.util 3 :invoke :write [1 33] INFO [2022-01-10 12:53:17,786] jepsen worker 39 - jepsen.util 39 :ok :write [13 27] INFO [2022-01-10 12:53:17,787] jepsen worker 39 - jepsen.util 39 :invoke :write [13 4] INFO [2022-01-10 12:53:17,791] jepsen worker 38 - jepsen.util 38 :ok :write [12 1] INFO [2022-01-10 12:53:17,792] jepsen worker 38 - jepsen.util 38 :invoke :write [12 41] INFO [2022-01-10 12:53:17,796] jepsen worker 0 - jepsen.util 90 :ok :write [0 1] INFO [2022-01-10 12:53:17,797] jepsen worker 85 - jepsen.util 175 :ok :write [28 37] INFO [2022-01-10 12:53:17,797] jepsen worker 0 - jepsen.util 90 :invoke :write [0 47] INFO [2022-01-10 12:53:17,798] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,799] jepsen worker 85 - jepsen.util 175 :ok :read [28 37] INFO [2022-01-10 12:53:17,799] jepsen worker 85 - jepsen.util 175 :invoke :write [28 46] INFO [2022-01-10 12:53:17,802] jepsen worker 39 - jepsen.util 39 :ok :write [13 4] INFO [2022-01-10 12:53:17,803] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,804] jepsen worker 39 - jepsen.util 39 :ok :read [13 4] INFO [2022-01-10 12:53:17,804] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,805] jepsen worker 39 - jepsen.util 39 :ok :read [13 4] INFO [2022-01-10 12:53:17,805] jepsen worker 3 - jepsen.util 3 :ok :write [1 33] INFO [2022-01-10 12:53:17,806] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,807] jepsen worker 39 - jepsen.util 39 :ok :read [13 4] INFO [2022-01-10 12:53:17,807] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,807] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,808] jepsen worker 39 - jepsen.util 39 :ok :read [13 4] INFO [2022-01-10 12:53:17,808] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:17,809] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,810] jepsen worker 39 - jepsen.util 39 :invoke :write [13 31] INFO [2022-01-10 12:53:17,810] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:17,811] jepsen worker 3 - jepsen.util 3 :invoke :write [1 47] INFO [2022-01-10 12:53:17,813] jepsen worker 38 - jepsen.util 38 :ok :write [12 41] INFO [2022-01-10 12:53:17,813] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,814] jepsen worker 38 - jepsen.util 38 :ok :read [12 41] INFO [2022-01-10 12:53:17,815] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,815] jepsen worker 0 - jepsen.util 90 :ok :write [0 47] INFO [2022-01-10 12:53:17,816] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,816] jepsen worker 0 - jepsen.util 90 :ok :read [0 47] INFO [2022-01-10 12:53:17,817] jepsen worker 38 - jepsen.util 38 :ok :read [12 41] INFO [2022-01-10 12:53:17,817] jepsen worker 0 - jepsen.util 90 :invoke :write [0 4] INFO [2022-01-10 12:53:17,817] jepsen worker 38 - jepsen.util 38 :invoke :write [12 2] INFO [2022-01-10 12:53:17,822] jepsen worker 85 - jepsen.util 175 :ok :write [28 46] INFO [2022-01-10 12:53:17,823] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,823] jepsen worker 85 - jepsen.util 175 :ok :read [28 46] INFO [2022-01-10 12:53:17,824] jepsen worker 85 - jepsen.util 175 :invoke :write [28 31] INFO [2022-01-10 12:53:17,824] jepsen worker 39 - jepsen.util 39 :ok :write [13 31] INFO [2022-01-10 12:53:17,825] jepsen worker 39 - jepsen.util 39 :invoke :write [13 14] INFO [2022-01-10 12:53:17,833] jepsen worker 3 - jepsen.util 3 :ok :write [1 47] INFO [2022-01-10 12:53:17,833] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,834] jepsen worker 0 - jepsen.util 90 :ok :write [0 4] INFO [2022-01-10 12:53:17,834] jepsen worker 3 - jepsen.util 3 :ok :read [1 47] INFO [2022-01-10 12:53:17,834] jepsen worker 0 - jepsen.util 90 :invoke :write [0 41] INFO [2022-01-10 12:53:17,835] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,835] jepsen worker 3 - jepsen.util 3 :ok :read [1 47] INFO [2022-01-10 12:53:17,836] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,837] jepsen worker 3 - jepsen.util 3 :ok :read [1 47] INFO [2022-01-10 12:53:17,837] jepsen worker 3 - jepsen.util 3 :invoke :write [1 47] INFO [2022-01-10 12:53:17,841] jepsen worker 38 - jepsen.util 38 :ok :write [12 2] INFO [2022-01-10 12:53:17,842] jepsen worker 38 - jepsen.util 38 :invoke :write [12 2] INFO [2022-01-10 12:53:17,844] jepsen worker 39 - jepsen.util 39 :ok :write [13 14] INFO [2022-01-10 12:53:17,845] jepsen worker 39 - jepsen.util 39 :invoke :write [13 38] INFO [2022-01-10 12:53:17,849] jepsen worker 85 - jepsen.util 175 :ok :write [28 31] INFO [2022-01-10 12:53:17,850] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,851] jepsen worker 0 - jepsen.util 90 :ok :write [0 41] INFO [2022-01-10 12:53:17,851] jepsen worker 85 - jepsen.util 175 :ok :read [28 31] INFO [2022-01-10 12:53:17,851] jepsen worker 0 - jepsen.util 90 :invoke :write [0 30] INFO [2022-01-10 12:53:17,852] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,853] jepsen worker 85 - jepsen.util 175 :ok :read [28 31] INFO [2022-01-10 12:53:17,854] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,855] jepsen worker 85 - jepsen.util 175 :ok :read [28 31] INFO [2022-01-10 12:53:17,855] jepsen worker 85 - jepsen.util 175 :invoke :write [28 17] INFO [2022-01-10 12:53:17,860] jepsen worker 3 - jepsen.util 3 :ok :write [1 47] INFO [2022-01-10 12:53:17,860] jepsen worker 3 - jepsen.util 3 :invoke :write [1 38] INFO [2022-01-10 12:53:17,861] jepsen worker 88 - jepsen.util 178 :ok :write [29 17] INFO [2022-01-10 12:53:17,861] jepsen worker 88 - jepsen.util 178 :invoke :write [29 4] INFO [2022-01-10 12:53:17,866] jepsen worker 38 - jepsen.util 38 :ok :write [12 2] INFO [2022-01-10 12:53:17,867] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,867] jepsen worker 39 - jepsen.util 39 :ok :write [13 38] INFO [2022-01-10 12:53:17,868] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,869] jepsen worker 39 - jepsen.util 39 :ok :read [13 38] INFO [2022-01-10 12:53:17,869] jepsen worker 38 - jepsen.util 38 :ok :read [12 2] INFO [2022-01-10 12:53:17,869] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,870] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,870] jepsen worker 39 - jepsen.util 39 :ok :read [13 38] INFO [2022-01-10 12:53:17,871] jepsen worker 38 - jepsen.util 38 :ok :read [12 2] INFO [2022-01-10 12:53:17,871] jepsen worker 86 - jepsen.util 86 :ok :write [28 5] INFO [2022-01-10 12:53:17,871] jepsen worker 39 - jepsen.util 39 :invoke :write [13 18] INFO [2022-01-10 12:53:17,871] jepsen worker 86 - jepsen.util 86 :invoke :write [28 2] INFO [2022-01-10 12:53:17,872] jepsen worker 38 - jepsen.util 38 :invoke :write [12 15] INFO [2022-01-10 12:53:17,874] jepsen worker 0 - jepsen.util 90 :ok :write [0 30] INFO [2022-01-10 12:53:17,874] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,875] jepsen worker 0 - jepsen.util 90 :ok :read [0 30] INFO [2022-01-10 12:53:17,876] jepsen worker 0 - jepsen.util 90 :invoke :write [0 5] INFO [2022-01-10 12:53:17,880] jepsen worker 3 - jepsen.util 3 :ok :write [1 38] INFO [2022-01-10 12:53:17,881] jepsen worker 3 - jepsen.util 3 :invoke :write [1 32] INFO [2022-01-10 12:53:17,882] jepsen worker 88 - jepsen.util 178 :ok :write [29 4] INFO [2022-01-10 12:53:17,883] jepsen worker 88 - jepsen.util 178 :invoke :write [29 22] INFO [2022-01-10 12:53:17,892] jepsen worker 39 - jepsen.util 39 :ok :write [13 18] INFO [2022-01-10 12:53:17,892] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,893] jepsen worker 39 - jepsen.util 39 :ok :read [13 18] INFO [2022-01-10 12:53:17,894] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,894] jepsen worker 86 - jepsen.util 86 :ok :write [28 2] INFO [2022-01-10 12:53:17,895] jepsen worker 38 - jepsen.util 38 :ok :write [12 15] INFO [2022-01-10 12:53:17,895] jepsen worker 39 - jepsen.util 39 :ok :read [13 18] INFO [2022-01-10 12:53:17,895] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,896] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,896] jepsen worker 86 - jepsen.util 86 :ok :read [28 2] INFO [2022-01-10 12:53:17,896] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,896] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,897] jepsen worker 39 - jepsen.util 39 :ok :read [13 18] INFO [2022-01-10 12:53:17,897] jepsen worker 0 - jepsen.util 90 :ok :write [0 5] INFO [2022-01-10 12:53:17,897] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,897] jepsen worker 38 - jepsen.util 38 :ok :read [12 15] INFO [2022-01-10 12:53:17,897] jepsen worker 86 - jepsen.util 86 :ok :read [28 2] INFO [2022-01-10 12:53:17,898] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,898] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,898] jepsen worker 39 - jepsen.util 39 :ok :read [13 18] INFO [2022-01-10 12:53:17,898] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,899] jepsen worker 0 - jepsen.util 90 :ok :read [0 5] INFO [2022-01-10 12:53:17,899] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,899] jepsen worker 0 - jepsen.util 90 :invoke :write [0 22] INFO [2022-01-10 12:53:17,899] jepsen worker 38 - jepsen.util 38 :ok :read [12 15] INFO [2022-01-10 12:53:17,899] jepsen worker 86 - jepsen.util 86 :ok :read [28 2] INFO [2022-01-10 12:53:17,900] jepsen worker 39 - jepsen.util 39 :ok :read [13 18] INFO [2022-01-10 12:53:17,900] jepsen worker 38 - jepsen.util 38 :invoke :write [12 27] INFO [2022-01-10 12:53:17,900] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,900] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,901] jepsen worker 39 - jepsen.util 39 :ok :read [13 18] INFO [2022-01-10 12:53:17,901] jepsen worker 86 - jepsen.util 86 :ok :read [28 2] INFO [2022-01-10 12:53:17,901] jepsen worker 39 - jepsen.util 39 :invoke :write [13 12] INFO [2022-01-10 12:53:17,902] jepsen worker 86 - jepsen.util 86 :invoke :write [28 31] INFO [2022-01-10 12:53:17,903] jepsen worker 88 - jepsen.util 178 :ok :write [29 22] INFO [2022-01-10 12:53:17,904] jepsen worker 88 - jepsen.util 178 :invoke :write [29 20] INFO [2022-01-10 12:53:17,907] jepsen worker 3 - jepsen.util 3 :ok :write [1 32] INFO [2022-01-10 12:53:17,908] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,909] jepsen worker 3 - jepsen.util 3 :ok :read [1 32] INFO [2022-01-10 12:53:17,909] jepsen worker 3 - jepsen.util 3 :invoke :write [1 14] INFO [2022-01-10 12:53:17,917] jepsen worker 85 - jepsen.util 175 :ok :write [28 17] INFO [2022-01-10 12:53:17,917] jepsen worker 0 - jepsen.util 90 :ok :write [0 22] INFO [2022-01-10 12:53:17,917] jepsen worker 38 - jepsen.util 38 :ok :write [12 27] INFO [2022-01-10 12:53:17,918] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:17,918] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,918] jepsen worker 0 - jepsen.util 90 :ok :read [0 22] INFO [2022-01-10 12:53:17,918] jepsen worker 38 - jepsen.util 38 :invoke :write [12 12] INFO [2022-01-10 12:53:17,919] jepsen worker 85 - jepsen.util 175 :ok :read [28 17] INFO [2022-01-10 12:53:17,919] jepsen worker 0 - jepsen.util 90 :invoke :write [0 46] INFO [2022-01-10 12:53:17,919] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,920] jepsen worker 85 - jepsen.util 175 :ok :read [28 17] INFO [2022-01-10 12:53:17,921] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,922] jepsen worker 85 - jepsen.util 175 :ok :read [28 17] INFO [2022-01-10 12:53:17,922] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,923] jepsen worker 85 - jepsen.util 175 :ok :read [28 17] INFO [2022-01-10 12:53:17,924] jepsen worker 85 - jepsen.util 175 :invoke :write [28 8] INFO [2022-01-10 12:53:17,925] jepsen worker 3 - jepsen.util 3 :ok :write [1 14] INFO [2022-01-10 12:53:17,926] jepsen worker 88 - jepsen.util 178 :ok :write [29 20] INFO [2022-01-10 12:53:17,926] jepsen worker 3 - jepsen.util 3 :invoke :write [1 43] INFO [2022-01-10 12:53:17,927] jepsen worker 88 - jepsen.util 178 :invoke :write [29 4] INFO [2022-01-10 12:53:17,929] jepsen worker 39 - jepsen.util 39 :ok :write [13 12] INFO [2022-01-10 12:53:17,930] jepsen worker 39 - jepsen.util 39 :invoke :write [13 10] INFO [2022-01-10 12:53:17,936] jepsen worker 0 - jepsen.util 90 :ok :write [0 46] INFO [2022-01-10 12:53:17,937] jepsen worker 0 - jepsen.util 90 :invoke :write [0 46] INFO [2022-01-10 12:53:17,937] jepsen worker 86 - jepsen.util 86 :ok :write [28 31] INFO [2022-01-10 12:53:17,938] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,939] jepsen worker 86 - jepsen.util 86 :ok :read [28 31] INFO [2022-01-10 12:53:17,940] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,941] jepsen worker 86 - jepsen.util 86 :ok :read [28 31] INFO [2022-01-10 12:53:17,941] jepsen worker 86 - jepsen.util 86 :invoke :read [28 nil] INFO [2022-01-10 12:53:17,943] jepsen worker 86 - jepsen.util 86 :ok :read [28 31] INFO [2022-01-10 12:53:17,943] jepsen worker 38 - jepsen.util 38 :ok :write [12 12] INFO [2022-01-10 12:53:17,943] jepsen worker 86 - jepsen.util 86 :invoke :write [28 15] INFO [2022-01-10 12:53:17,944] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,945] jepsen worker 38 - jepsen.util 38 :ok :read [12 12] INFO [2022-01-10 12:53:17,945] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:17,946] jepsen worker 38 - jepsen.util 38 :ok :read [12 12] INFO [2022-01-10 12:53:17,947] jepsen worker 38 - jepsen.util 38 :invoke :write [12 3] INFO [2022-01-10 12:53:17,948] jepsen worker 88 - jepsen.util 178 :ok :write [29 4] INFO [2022-01-10 12:53:17,948] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,949] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:17,950] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,951] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:17,952] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,952] jepsen worker 3 - jepsen.util 3 :ok :write [1 43] INFO [2022-01-10 12:53:17,953] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:17,953] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,953] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,953] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:17,954] jepsen worker 39 - jepsen.util 39 :ok :write [13 10] INFO [2022-01-10 12:53:17,954] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:17,954] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:17,955] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,955] jepsen worker 39 - jepsen.util 39 :ok :read [13 10] INFO [2022-01-10 12:53:17,956] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:17,956] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:17,956] jepsen worker 39 - jepsen.util 39 :invoke :write [13 15] INFO [2022-01-10 12:53:17,956] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:17,957] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,958] jepsen worker 3 - jepsen.util 3 :invoke :write [1 48] INFO [2022-01-10 12:53:17,958] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:17,959] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,959] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:17,960] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:17,961] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:17,961] jepsen worker 88 - jepsen.util 178 :invoke :write [29 49] INFO [2022-01-10 12:53:17,963] jepsen worker 0 - jepsen.util 90 :ok :write [0 46] INFO [2022-01-10 12:53:17,963] jepsen worker 0 - jepsen.util 90 :invoke :write [0 29] INFO [2022-01-10 12:53:17,964] jepsen worker 38 - jepsen.util 38 :ok :write [12 3] INFO [2022-01-10 12:53:17,964] jepsen worker 38 - jepsen.util 38 :invoke :write [12 46] INFO [2022-01-10 12:53:18,116] jepsen worker 85 - jepsen.util 175 :ok :write [28 8] INFO [2022-01-10 12:53:18,117] jepsen worker 85 - jepsen.util 175 :invoke :write [28 23] INFO [2022-01-10 12:53:18,118] jepsen worker 39 - jepsen.util 39 :ok :write [13 15] INFO [2022-01-10 12:53:18,119] jepsen worker 39 - jepsen.util 39 :invoke :write [13 3] INFO [2022-01-10 12:53:18,119] jepsen worker 3 - jepsen.util 3 :ok :write [1 48] INFO [2022-01-10 12:53:18,119] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,120] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:18,121] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,122] jepsen worker 88 - jepsen.util 178 :ok :write [29 49] INFO [2022-01-10 12:53:18,122] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:18,122] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,123] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,123] jepsen worker 88 - jepsen.util 178 :ok :read [29 49] INFO [2022-01-10 12:53:18,124] jepsen worker 88 - jepsen.util 178 :invoke :write [29 5] INFO [2022-01-10 12:53:18,124] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:18,124] jepsen worker 3 - jepsen.util 3 :invoke :write [1 2] INFO [2022-01-10 12:53:18,129] jepsen worker 38 - jepsen.util 38 :ok :write [12 46] INFO [2022-01-10 12:53:18,130] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,130] jepsen worker 0 - jepsen.util 90 :ok :write [0 29] INFO [2022-01-10 12:53:18,131] jepsen worker 0 - jepsen.util 90 :invoke :write [0 34] INFO [2022-01-10 12:53:18,131] jepsen worker 38 - jepsen.util 38 :ok :read [12 46] INFO [2022-01-10 12:53:18,132] jepsen worker 38 - jepsen.util 38 :invoke :write [12 44] INFO [2022-01-10 12:53:18,134] jepsen worker 85 - jepsen.util 175 :ok :write [28 23] INFO [2022-01-10 12:53:18,135] jepsen worker 85 - jepsen.util 175 :invoke :write [28 11] INFO [2022-01-10 12:53:18,136] jepsen worker 39 - jepsen.util 39 :ok :write [13 3] INFO [2022-01-10 12:53:18,136] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,137] jepsen worker 39 - jepsen.util 39 :ok :read [13 3] INFO [2022-01-10 12:53:18,138] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,139] jepsen worker 39 - jepsen.util 39 :ok :read [13 3] INFO [2022-01-10 12:53:18,139] jepsen worker 39 - jepsen.util 39 :invoke :write [13 32] INFO [2022-01-10 12:53:18,140] jepsen worker 88 - jepsen.util 178 :ok :write [29 5] INFO [2022-01-10 12:53:18,140] jepsen worker 3 - jepsen.util 3 :ok :write [1 2] INFO [2022-01-10 12:53:18,141] jepsen worker 88 - jepsen.util 178 :invoke :write [29 43] INFO [2022-01-10 12:53:18,141] jepsen worker 3 - jepsen.util 3 :invoke :write [1 2] INFO [2022-01-10 12:53:18,148] jepsen worker 0 - jepsen.util 90 :ok :write [0 34] INFO [2022-01-10 12:53:18,148] jepsen worker 0 - jepsen.util 90 :invoke :write [0 22] INFO [2022-01-10 12:53:18,151] jepsen worker 38 - jepsen.util 38 :ok :write [12 44] INFO [2022-01-10 12:53:18,151] jepsen worker 38 - jepsen.util 38 :invoke :write [12 22] INFO [2022-01-10 12:53:18,154] jepsen worker 85 - jepsen.util 175 :ok :write [28 11] INFO [2022-01-10 12:53:18,154] jepsen worker 85 - jepsen.util 175 :invoke :write [28 17] INFO [2022-01-10 12:53:18,156] jepsen worker 39 - jepsen.util 39 :ok :write [13 32] INFO [2022-01-10 12:53:18,156] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,157] jepsen worker 39 - jepsen.util 39 :ok :read [13 32] INFO [2022-01-10 12:53:18,158] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,158] jepsen worker 3 - jepsen.util 3 :ok :write [1 2] INFO [2022-01-10 12:53:18,158] jepsen worker 88 - jepsen.util 178 :ok :write [29 43] INFO [2022-01-10 12:53:18,159] jepsen worker 39 - jepsen.util 39 :ok :read [13 32] INFO [2022-01-10 12:53:18,159] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,159] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,160] jepsen worker 3 - jepsen.util 3 :invoke :write [1 19] INFO [2022-01-10 12:53:18,160] jepsen worker 88 - jepsen.util 178 :ok :read [29 43] INFO [2022-01-10 12:53:18,160] jepsen worker 39 - jepsen.util 39 :ok :read [13 32] INFO [2022-01-10 12:53:18,161] jepsen worker 88 - jepsen.util 178 :invoke :write [29 24] INFO [2022-01-10 12:53:18,161] jepsen worker 39 - jepsen.util 39 :invoke :write [13 17] INFO [2022-01-10 12:53:18,165] jepsen worker 0 - jepsen.util 90 :ok :write [0 22] INFO [2022-01-10 12:53:18,165] jepsen worker 38 - jepsen.util 38 :ok :write [12 22] INFO [2022-01-10 12:53:18,166] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:18,166] jepsen worker 38 - jepsen.util 38 :invoke :write [12 18] INFO [2022-01-10 12:53:18,166] jepsen worker 0 - jepsen.util 90 :ok :read [0 22] INFO [2022-01-10 12:53:18,167] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:18,168] jepsen worker 0 - jepsen.util 90 :ok :read [0 22] INFO [2022-01-10 12:53:18,168] jepsen worker 0 - jepsen.util 90 :invoke :write [0 4] INFO [2022-01-10 12:53:18,236] jepsen worker 35 - jepsen.util 125 :ok :write [11 20] INFO [2022-01-10 12:53:18,237] jepsen worker 35 - jepsen.util 125 :invoke :write [11 30] INFO [2022-01-10 12:53:18,264] jepsen worker 60 - jepsen.util 60 :info :write [20 10] :RequestTimeoutException INFO [2022-01-10 12:53:18,265] jepsen worker 60 - jepsen.util 150 :invoke :write [20 22] INFO [2022-01-10 12:53:18,277] jepsen worker 53 - jepsen.util 53 :info :write [17 2] :RequestTimeoutException INFO [2022-01-10 12:53:18,278] jepsen worker 53 - jepsen.util 143 :invoke :write [17 42] INFO [2022-01-10 12:53:18,311] jepsen worker 85 - jepsen.util 175 :ok :write [28 17] INFO [2022-01-10 12:53:18,311] jepsen worker 85 - jepsen.util 175 :invoke :write [28 47] INFO [2022-01-10 12:53:18,317] jepsen worker 21 - jepsen.util 21 :info :write [7 18] :RequestTimeoutException INFO [2022-01-10 12:53:18,317] jepsen worker 3 - jepsen.util 3 :ok :write [1 19] INFO [2022-01-10 12:53:18,317] jepsen worker 39 - jepsen.util 39 :ok :write [13 17] INFO [2022-01-10 12:53:18,318] jepsen worker 21 - jepsen.util 111 :invoke :read [7 nil] INFO [2022-01-10 12:53:18,318] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,319] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,319] jepsen worker 3 - jepsen.util 3 :ok :read [1 19] INFO [2022-01-10 12:53:18,320] jepsen worker 39 - jepsen.util 39 :ok :read [13 17] INFO [2022-01-10 12:53:18,320] jepsen worker 3 - jepsen.util 3 :invoke :write [1 37] INFO [2022-01-10 12:53:18,320] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,321] jepsen worker 39 - jepsen.util 39 :ok :read [13 17] INFO [2022-01-10 12:53:18,321] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,322] jepsen worker 39 - jepsen.util 39 :ok :read [13 17] INFO [2022-01-10 12:53:18,323] jepsen worker 38 - jepsen.util 38 :ok :write [12 18] INFO [2022-01-10 12:53:18,323] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,323] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,324] jepsen worker 39 - jepsen.util 39 :ok :read [13 17] INFO [2022-01-10 12:53:18,324] jepsen worker 39 - jepsen.util 39 :invoke :write [13 10] INFO [2022-01-10 12:53:18,324] jepsen worker 38 - jepsen.util 38 :ok :read [12 18] INFO [2022-01-10 12:53:18,325] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,325] jepsen worker 88 - jepsen.util 178 :ok :write [29 24] INFO [2022-01-10 12:53:18,326] jepsen worker 38 - jepsen.util 38 :ok :read [12 18] INFO [2022-01-10 12:53:18,326] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,326] jepsen worker 38 - jepsen.util 38 :invoke :write [12 21] INFO [2022-01-10 12:53:18,327] jepsen worker 88 - jepsen.util 178 :ok :read [29 24] INFO [2022-01-10 12:53:18,327] jepsen worker 88 - jepsen.util 178 :invoke :write [29 8] INFO [2022-01-10 12:53:18,329] jepsen worker 85 - jepsen.util 175 :ok :write [28 47] INFO [2022-01-10 12:53:18,330] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:18,331] jepsen worker 85 - jepsen.util 175 :ok :read [28 47] INFO [2022-01-10 12:53:18,331] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:18,331] jepsen worker 0 - jepsen.util 90 :ok :write [0 4] INFO [2022-01-10 12:53:18,332] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:18,332] jepsen worker 85 - jepsen.util 175 :ok :read [28 47] INFO [2022-01-10 12:53:18,332] jepsen worker 85 - jepsen.util 175 :invoke :write [28 5] INFO [2022-01-10 12:53:18,332] jepsen worker 0 - jepsen.util 90 :ok :read [0 4] INFO [2022-01-10 12:53:18,333] jepsen worker 0 - jepsen.util 90 :invoke :write [0 22] INFO [2022-01-10 12:53:18,338] jepsen worker 3 - jepsen.util 3 :ok :write [1 37] INFO [2022-01-10 12:53:18,339] jepsen worker 3 - jepsen.util 3 :invoke :write [1 26] INFO [2022-01-10 12:53:18,343] jepsen worker 38 - jepsen.util 38 :ok :write [12 21] INFO [2022-01-10 12:53:18,343] jepsen worker 39 - jepsen.util 39 :ok :write [13 10] INFO [2022-01-10 12:53:18,344] jepsen worker 38 - jepsen.util 38 :invoke :write [12 42] INFO [2022-01-10 12:53:18,344] jepsen worker 39 - jepsen.util 39 :invoke :write [13 32] INFO [2022-01-10 12:53:18,350] jepsen worker 88 - jepsen.util 178 :ok :write [29 8] INFO [2022-01-10 12:53:18,350] jepsen worker 85 - jepsen.util 175 :ok :write [28 5] INFO [2022-01-10 12:53:18,350] jepsen worker 88 - jepsen.util 178 :invoke :write [29 6] INFO [2022-01-10 12:53:18,351] jepsen worker 85 - jepsen.util 175 :invoke :write [28 19] INFO [2022-01-10 12:53:18,358] jepsen worker 0 - jepsen.util 90 :ok :write [0 22] INFO [2022-01-10 12:53:18,359] jepsen worker 3 - jepsen.util 3 :ok :write [1 26] INFO [2022-01-10 12:53:18,359] jepsen worker 0 - jepsen.util 90 :invoke :write [0 41] INFO [2022-01-10 12:53:18,359] jepsen worker 3 - jepsen.util 3 :invoke :write [1 48] INFO [2022-01-10 12:53:18,365] jepsen worker 39 - jepsen.util 39 :ok :write [13 32] INFO [2022-01-10 12:53:18,366] jepsen worker 38 - jepsen.util 38 :ok :write [12 42] INFO [2022-01-10 12:53:18,366] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,366] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,367] jepsen worker 39 - jepsen.util 39 :ok :read [13 32] INFO [2022-01-10 12:53:18,367] jepsen worker 38 - jepsen.util 38 :ok :read [12 42] INFO [2022-01-10 12:53:18,367] jepsen worker 39 - jepsen.util 39 :invoke :write [13 7] INFO [2022-01-10 12:53:18,368] jepsen worker 38 - jepsen.util 38 :invoke :write [12 32] INFO [2022-01-10 12:53:18,372] jepsen worker 85 - jepsen.util 175 :ok :write [28 19] INFO [2022-01-10 12:53:18,372] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:18,373] jepsen worker 85 - jepsen.util 175 :ok :read [28 19] INFO [2022-01-10 12:53:18,373] jepsen worker 85 - jepsen.util 175 :invoke :write [28 5] INFO [2022-01-10 12:53:18,374] jepsen worker 88 - jepsen.util 178 :ok :write [29 6] INFO [2022-01-10 12:53:18,374] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,375] jepsen worker 88 - jepsen.util 178 :ok :read [29 6] INFO [2022-01-10 12:53:18,376] jepsen worker 88 - jepsen.util 178 :invoke :write [29 9] INFO [2022-01-10 12:53:18,378] jepsen worker 3 - jepsen.util 3 :ok :write [1 48] INFO [2022-01-10 12:53:18,378] jepsen worker 0 - jepsen.util 90 :ok :write [0 41] INFO [2022-01-10 12:53:18,378] jepsen worker 3 - jepsen.util 3 :invoke :write [1 32] INFO [2022-01-10 12:53:18,379] jepsen worker 0 - jepsen.util 90 :invoke :write [0 12] INFO [2022-01-10 12:53:18,383] jepsen worker 38 - jepsen.util 38 :ok :write [12 32] INFO [2022-01-10 12:53:18,383] jepsen worker 38 - jepsen.util 38 :invoke :write [12 20] INFO [2022-01-10 12:53:18,387] jepsen worker 39 - jepsen.util 39 :ok :write [13 7] INFO [2022-01-10 12:53:18,388] jepsen worker 39 - jepsen.util 39 :invoke :write [13 29] INFO [2022-01-10 12:53:18,388] jepsen worker 85 - jepsen.util 175 :ok :write [28 5] INFO [2022-01-10 12:53:18,389] jepsen worker 85 - jepsen.util 175 :invoke :write [28 39] INFO [2022-01-10 12:53:18,394] jepsen worker 88 - jepsen.util 178 :ok :write [29 9] INFO [2022-01-10 12:53:18,395] jepsen worker 88 - jepsen.util 178 :invoke :write [29 30] INFO [2022-01-10 12:53:18,396] jepsen worker 3 - jepsen.util 3 :ok :write [1 32] INFO [2022-01-10 12:53:18,397] jepsen worker 3 - jepsen.util 3 :invoke :write [1 14] INFO [2022-01-10 12:53:18,399] jepsen worker 0 - jepsen.util 90 :ok :write [0 12] INFO [2022-01-10 12:53:18,400] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:18,401] jepsen worker 0 - jepsen.util 90 :ok :read [0 12] INFO [2022-01-10 12:53:18,401] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:18,402] jepsen worker 0 - jepsen.util 90 :ok :read [0 12] INFO [2022-01-10 12:53:18,403] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:18,403] jepsen worker 0 - jepsen.util 90 :ok :read [0 12] INFO [2022-01-10 12:53:18,404] jepsen worker 38 - jepsen.util 38 :ok :write [12 20] INFO [2022-01-10 12:53:18,404] jepsen worker 0 - jepsen.util 90 :invoke :write [0 41] INFO [2022-01-10 12:53:18,405] jepsen worker 38 - jepsen.util 38 :invoke :write [12 35] INFO [2022-01-10 12:53:18,407] jepsen worker 39 - jepsen.util 39 :ok :write [13 29] INFO [2022-01-10 12:53:18,408] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,408] jepsen worker 85 - jepsen.util 175 :ok :write [28 39] INFO [2022-01-10 12:53:18,409] jepsen worker 85 - jepsen.util 175 :invoke :write [28 27] INFO [2022-01-10 12:53:18,409] jepsen worker 39 - jepsen.util 39 :ok :read [13 29] INFO [2022-01-10 12:53:18,410] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,411] jepsen worker 39 - jepsen.util 39 :ok :read [13 29] INFO [2022-01-10 12:53:18,411] jepsen worker 39 - jepsen.util 39 :invoke :write [13 35] INFO [2022-01-10 12:53:18,413] jepsen worker 88 - jepsen.util 178 :ok :write [29 30] INFO [2022-01-10 12:53:18,413] jepsen worker 88 - jepsen.util 178 :invoke :write [29 36] INFO [2022-01-10 12:53:18,416] jepsen worker 3 - jepsen.util 3 :ok :write [1 14] INFO [2022-01-10 12:53:18,417] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,418] jepsen worker 3 - jepsen.util 3 :ok :read [1 14] INFO [2022-01-10 12:53:18,418] jepsen worker 3 - jepsen.util 3 :invoke :write [1 44] INFO [2022-01-10 12:53:18,419] jepsen worker 75 - jepsen.util 75 :info :write [25 27] :RequestTimeoutException INFO [2022-01-10 12:53:18,419] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:18,420] jepsen worker 38 - jepsen.util 38 :ok :write [12 35] INFO [2022-01-10 12:53:18,420] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,421] jepsen worker 75 - jepsen.util 165 :ok :read [25 15] INFO [2022-01-10 12:53:18,421] jepsen worker 75 - jepsen.util 165 :invoke :write [25 26] INFO [2022-01-10 12:53:18,421] jepsen worker 38 - jepsen.util 38 :ok :read [12 35] INFO [2022-01-10 12:53:18,422] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:18,428] jepsen worker 85 - jepsen.util 175 :ok :write [28 27] INFO [2022-01-10 12:53:18,428] jepsen worker 0 - jepsen.util 90 :ok :write [0 41] INFO [2022-01-10 12:53:18,428] jepsen worker 85 - jepsen.util 175 :invoke :write [28 23] INFO [2022-01-10 12:53:18,429] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:18,430] jepsen worker 0 - jepsen.util 90 :ok :read [0 41] INFO [2022-01-10 12:53:18,430] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:18,431] jepsen worker 0 - jepsen.util 90 :ok :read [0 41] INFO [2022-01-10 12:53:18,431] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:18,432] jepsen worker 0 - jepsen.util 90 :ok :read [0 41] INFO [2022-01-10 12:53:18,433] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:18,434] jepsen worker 3 - jepsen.util 3 :ok :write [1 44] INFO [2022-01-10 12:53:18,434] jepsen worker 0 - jepsen.util 90 :ok :read [0 41] INFO [2022-01-10 12:53:18,434] jepsen worker 3 - jepsen.util 3 :invoke :write [1 6] INFO [2022-01-10 12:53:18,435] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:18,435] jepsen worker 88 - jepsen.util 178 :ok :write [29 36] INFO [2022-01-10 12:53:18,435] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,436] jepsen worker 0 - jepsen.util 90 :ok :read [0 41] INFO [2022-01-10 12:53:18,436] jepsen worker 0 - jepsen.util 90 :invoke :write [0 19] INFO [2022-01-10 12:53:18,436] jepsen worker 88 - jepsen.util 178 :ok :read [29 36] INFO [2022-01-10 12:53:18,437] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,438] jepsen worker 88 - jepsen.util 178 :ok :read [29 36] INFO [2022-01-10 12:53:18,438] jepsen worker 88 - jepsen.util 178 :invoke :write [29 49] INFO [2022-01-10 12:53:18,440] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:18,440] jepsen worker 38 - jepsen.util 38 :invoke :write [12 36] INFO [2022-01-10 12:53:18,443] jepsen worker 39 - jepsen.util 39 :ok :write [13 35] INFO [2022-01-10 12:53:18,444] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,445] jepsen worker 39 - jepsen.util 39 :ok :read [13 35] INFO [2022-01-10 12:53:18,445] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,446] jepsen worker 39 - jepsen.util 39 :ok :read [13 35] INFO [2022-01-10 12:53:18,447] jepsen worker 39 - jepsen.util 39 :invoke :write [13 11] INFO [2022-01-10 12:53:18,448] jepsen worker 85 - jepsen.util 175 :ok :write [28 23] INFO [2022-01-10 12:53:18,449] jepsen worker 85 - jepsen.util 175 :invoke :write [28 5] INFO [2022-01-10 12:53:18,449] jepsen worker 0 - jepsen.util 90 :ok :write [0 19] INFO [2022-01-10 12:53:18,450] jepsen worker 0 - jepsen.util 90 :invoke :write [0 38] INFO [2022-01-10 12:53:18,453] jepsen worker 3 - jepsen.util 3 :ok :write [1 6] INFO [2022-01-10 12:53:18,453] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,454] jepsen worker 3 - jepsen.util 3 :ok :read [1 6] INFO [2022-01-10 12:53:18,455] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,455] jepsen worker 3 - jepsen.util 3 :ok :read [1 6] INFO [2022-01-10 12:53:18,456] jepsen worker 3 - jepsen.util 3 :invoke :write [1 18] INFO [2022-01-10 12:53:18,459] jepsen worker 38 - jepsen.util 38 :ok :write [12 36] INFO [2022-01-10 12:53:18,460] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,461] jepsen worker 38 - jepsen.util 38 :ok :read [12 36] INFO [2022-01-10 12:53:18,461] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,461] jepsen worker 88 - jepsen.util 178 :ok :write [29 49] INFO [2022-01-10 12:53:18,462] jepsen worker 88 - jepsen.util 178 :invoke :write [29 26] INFO [2022-01-10 12:53:18,462] jepsen worker 38 - jepsen.util 38 :ok :read [12 36] INFO [2022-01-10 12:53:18,463] jepsen worker 38 - jepsen.util 38 :invoke :write [12 34] INFO [2022-01-10 12:53:18,467] jepsen worker 3 - jepsen.util 3 :ok :write [1 18] INFO [2022-01-10 12:53:18,467] jepsen worker 0 - jepsen.util 90 :ok :write [0 38] INFO [2022-01-10 12:53:18,467] jepsen worker 3 - jepsen.util 3 :invoke :write [1 34] INFO [2022-01-10 12:53:18,468] jepsen worker 0 - jepsen.util 90 :invoke :write [0 9] INFO [2022-01-10 12:53:18,471] jepsen worker 85 - jepsen.util 175 :ok :write [28 5] INFO [2022-01-10 12:53:18,472] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:18,473] jepsen worker 85 - jepsen.util 175 :ok :read [28 5] INFO [2022-01-10 12:53:18,473] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:18,474] jepsen worker 39 - jepsen.util 39 :ok :write [13 11] INFO [2022-01-10 12:53:18,474] jepsen worker 38 - jepsen.util 38 :ok :write [12 34] INFO [2022-01-10 12:53:18,474] jepsen worker 85 - jepsen.util 175 :ok :read [28 5] INFO [2022-01-10 12:53:18,475] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,476] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:18,476] jepsen worker 39 - jepsen.util 39 :ok :read [13 11] INFO [2022-01-10 12:53:18,476] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,477] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,477] jepsen worker 85 - jepsen.util 175 :ok :read [28 5] INFO [2022-01-10 12:53:18,477] jepsen worker 38 - jepsen.util 38 :ok :read [12 34] INFO [2022-01-10 12:53:18,478] jepsen worker 85 - jepsen.util 175 :invoke :write [28 21] INFO [2022-01-10 12:53:18,479] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,480] jepsen worker 38 - jepsen.util 38 :ok :read [12 34] INFO [2022-01-10 12:53:18,481] jepsen worker 38 - jepsen.util 38 :invoke :write [12 23] INFO [2022-01-10 12:53:18,482] jepsen worker 3 - jepsen.util 3 :ok :write [1 34] INFO [2022-01-10 12:53:18,482] jepsen worker 88 - jepsen.util 178 :ok :write [29 26] INFO [2022-01-10 12:53:18,482] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 12:53:18,482] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,483] jepsen worker 39 - jepsen.util 39 :ok :read [13 11] INFO [2022-01-10 12:53:18,483] jepsen worker 88 - jepsen.util 178 :ok :read [29 26] INFO [2022-01-10 12:53:18,483] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,484] jepsen worker 88 - jepsen.util 178 :invoke :write [29 27] INFO [2022-01-10 12:53:18,484] jepsen worker 39 - jepsen.util 39 :ok :read [13 11] INFO [2022-01-10 12:53:18,485] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,486] jepsen worker 39 - jepsen.util 39 :ok :read [13 11] INFO [2022-01-10 12:53:18,487] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,488] jepsen worker 39 - jepsen.util 39 :ok :read [13 11] INFO [2022-01-10 12:53:18,489] jepsen worker 39 - jepsen.util 39 :invoke :write [13 31] INFO [2022-01-10 12:53:18,490] jepsen worker 0 - jepsen.util 90 :ok :write [0 9] INFO [2022-01-10 12:53:18,491] jepsen worker 0 - jepsen.util 90 :invoke :write [0 4] INFO [2022-01-10 12:53:18,494] jepsen worker 38 - jepsen.util 38 :ok :write [12 23] INFO [2022-01-10 12:53:18,495] jepsen worker 38 - jepsen.util 38 :invoke :write [12 36] INFO [2022-01-10 12:53:18,559] jepsen worker 16 - jepsen.util 106 :ok :write [5 2] INFO [2022-01-10 12:53:18,560] jepsen worker 16 - jepsen.util 106 :invoke :write [5 27] INFO [2022-01-10 12:53:18,563] jepsen worker 78 - jepsen.util 78 :ok :write [26 4] INFO [2022-01-10 12:53:18,564] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:18,564] jepsen worker 66 - jepsen.util 156 :ok :write [22 22] INFO [2022-01-10 12:53:18,565] jepsen worker 66 - jepsen.util 156 :invoke :write [22 47] INFO [2022-01-10 12:53:18,565] jepsen worker 78 - jepsen.util 78 :ok :read [26 4] INFO [2022-01-10 12:53:18,565] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:18,566] jepsen worker 78 - jepsen.util 78 :ok :read [26 4] INFO [2022-01-10 12:53:18,567] jepsen worker 78 - jepsen.util 78 :invoke :write [26 32] INFO [2022-01-10 12:53:18,568] jepsen worker 52 - jepsen.util 52 :ok :write [17 30] INFO [2022-01-10 12:53:18,569] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,570] jepsen worker 49 - jepsen.util 139 :ok :write [16 31] INFO [2022-01-10 12:53:18,570] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:18,571] jepsen worker 49 - jepsen.util 139 :ok :read [16 31] INFO [2022-01-10 12:53:18,572] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:18,572] jepsen worker 49 - jepsen.util 139 :ok :read [16 31] INFO [2022-01-10 12:53:18,573] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:18,573] jepsen worker 49 - jepsen.util 139 :ok :read [16 31] INFO [2022-01-10 12:53:18,574] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:18,575] jepsen worker 70 - jepsen.util 160 :ok :write [23 25] INFO [2022-01-10 12:53:18,575] jepsen worker 49 - jepsen.util 139 :ok :read [16 31] INFO [2022-01-10 12:53:18,575] jepsen worker 62 - jepsen.util 152 :ok :write [20 44] INFO [2022-01-10 12:53:18,576] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:18,578] jepsen worker 49 - jepsen.util 139 :ok :read [16 31] INFO [2022-01-10 12:53:18,579] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:18,580] jepsen worker 62 - jepsen.util 152 :invoke :write [20 27] INFO [2022-01-10 12:53:18,581] jepsen worker 49 - jepsen.util 139 :invoke :read [16 nil] INFO [2022-01-10 12:53:18,582] jepsen worker 49 - jepsen.util 139 :ok :read [16 31] INFO [2022-01-10 12:53:18,582] jepsen worker 49 - jepsen.util 139 :invoke :write [16 42] INFO [2022-01-10 12:53:18,583] jepsen worker 70 - jepsen.util 160 :ok :read [23 25] INFO [2022-01-10 12:53:18,583] jepsen worker 70 - jepsen.util 160 :invoke :write [23 5] INFO [2022-01-10 12:53:18,595] jepsen worker 88 - jepsen.util 178 :ok :write [29 27] INFO [2022-01-10 12:53:18,595] jepsen worker 3 - jepsen.util 3 :ok :write [1 40] INFO [2022-01-10 12:53:18,596] jepsen worker 88 - jepsen.util 178 :invoke :write [29 7] INFO [2022-01-10 12:53:18,596] jepsen worker 3 - jepsen.util 3 :invoke :write [1 12] INFO [2022-01-10 12:53:18,601] jepsen worker 85 - jepsen.util 175 :ok :write [28 21] INFO [2022-01-10 12:53:18,602] jepsen worker 85 - jepsen.util 175 :invoke :write [28 10] INFO [2022-01-10 12:53:18,602] jepsen worker 0 - jepsen.util 90 :ok :write [0 4] INFO [2022-01-10 12:53:18,603] jepsen worker 0 - jepsen.util 90 :invoke :write [0 25] INFO [2022-01-10 12:53:18,604] jepsen worker 38 - jepsen.util 38 :ok :write [12 36] INFO [2022-01-10 12:53:18,605] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,606] jepsen worker 38 - jepsen.util 38 :ok :read [12 36] INFO [2022-01-10 12:53:18,607] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,608] jepsen worker 38 - jepsen.util 38 :ok :read [12 36] INFO [2022-01-10 12:53:18,608] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,608] jepsen worker 39 - jepsen.util 39 :ok :write [13 31] INFO [2022-01-10 12:53:18,609] jepsen worker 38 - jepsen.util 38 :ok :read [12 36] INFO [2022-01-10 12:53:18,609] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,610] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,610] jepsen worker 39 - jepsen.util 39 :ok :read [13 31] INFO [2022-01-10 12:53:18,611] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,611] jepsen worker 38 - jepsen.util 38 :ok :read [12 36] INFO [2022-01-10 12:53:18,611] jepsen worker 38 - jepsen.util 38 :invoke :write [12 32] INFO [2022-01-10 12:53:18,611] jepsen worker 39 - jepsen.util 39 :ok :read [13 31] INFO [2022-01-10 12:53:18,612] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,613] jepsen worker 39 - jepsen.util 39 :ok :read [13 31] INFO [2022-01-10 12:53:18,613] jepsen worker 39 - jepsen.util 39 :invoke :write [13 48] INFO [2022-01-10 12:53:18,620] jepsen worker 88 - jepsen.util 178 :ok :write [29 7] INFO [2022-01-10 12:53:18,622] jepsen worker 88 - jepsen.util 178 :invoke :write [29 0] INFO [2022-01-10 12:53:18,623] jepsen worker 3 - jepsen.util 3 :ok :write [1 12] INFO [2022-01-10 12:53:18,624] jepsen worker 3 - jepsen.util 3 :invoke :write [1 12] INFO [2022-01-10 12:53:18,629] jepsen worker 85 - jepsen.util 175 :ok :write [28 10] INFO [2022-01-10 12:53:18,630] jepsen worker 85 - jepsen.util 175 :invoke :write [28 40] INFO [2022-01-10 12:53:18,705] jepsen worker 83 - jepsen.util 173 :ok :write [27 29] INFO [2022-01-10 12:53:18,705] jepsen worker 13 - jepsen.util 13 :ok :write [4 6] INFO [2022-01-10 12:53:18,705] jepsen worker 52 - jepsen.util 52 :ok :read [17 30] INFO [2022-01-10 12:53:18,705] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:18,706] jepsen worker 52 - jepsen.util 52 :invoke :write [17 33] INFO [2022-01-10 12:53:18,706] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:18,706] jepsen worker 13 - jepsen.util 13 :ok :read [4 6] INFO [2022-01-10 12:53:18,707] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:18,708] jepsen worker 13 - jepsen.util 13 :ok :read [4 6] INFO [2022-01-10 12:53:18,708] jepsen worker 13 - jepsen.util 13 :invoke :write [4 23] INFO [2022-01-10 12:53:18,713] jepsen worker 8 - jepsen.util 8 :ok :write [2 41] INFO [2022-01-10 12:53:18,714] jepsen worker 8 - jepsen.util 8 :invoke :write [2 39] INFO [2022-01-10 12:53:18,714] jepsen worker 83 - jepsen.util 173 :ok :read [27 29] INFO [2022-01-10 12:53:18,715] jepsen worker 83 - jepsen.util 173 :invoke :write [27 10] INFO [2022-01-10 12:53:18,715] jepsen worker 78 - jepsen.util 78 :ok :write [26 32] INFO [2022-01-10 12:53:18,716] jepsen worker 78 - jepsen.util 78 :invoke :write [26 35] INFO [2022-01-10 12:53:18,722] jepsen worker 0 - jepsen.util 90 :ok :write [0 25] INFO [2022-01-10 12:53:18,722] jepsen worker 53 - jepsen.util 143 :ok :write [17 42] INFO [2022-01-10 12:53:18,723] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:18,723] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,724] jepsen worker 0 - jepsen.util 90 :ok :read [0 25] INFO [2022-01-10 12:53:18,724] jepsen worker 53 - jepsen.util 143 :ok :read [17 42] INFO [2022-01-10 12:53:18,725] jepsen worker 0 - jepsen.util 90 :invoke :write [0 11] INFO [2022-01-10 12:53:18,725] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,726] jepsen worker 53 - jepsen.util 143 :ok :read [17 42] INFO [2022-01-10 12:53:18,727] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,728] jepsen worker 39 - jepsen.util 39 :ok :write [13 48] INFO [2022-01-10 12:53:18,728] jepsen worker 23 - jepsen.util 23 :ok :write [7 24] INFO [2022-01-10 12:53:18,729] jepsen worker 38 - jepsen.util 38 :ok :write [12 32] INFO [2022-01-10 12:53:18,729] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,729] jepsen worker 53 - jepsen.util 143 :ok :read [17 42] INFO [2022-01-10 12:53:18,730] jepsen worker 39 - jepsen.util 39 :ok :read [13 48] INFO [2022-01-10 12:53:18,730] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,731] jepsen worker 39 - jepsen.util 39 :invoke :write [13 14] INFO [2022-01-10 12:53:18,731] jepsen worker 38 - jepsen.util 38 :ok :read [12 32] INFO [2022-01-10 12:53:18,731] jepsen worker 53 - jepsen.util 143 :invoke :write [17 39] INFO [2022-01-10 12:53:18,732] jepsen worker 70 - jepsen.util 160 :ok :write [23 5] INFO [2022-01-10 12:53:18,732] jepsen worker 38 - jepsen.util 38 :invoke :write [12 30] INFO [2022-01-10 12:53:18,733] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:18,733] jepsen worker 3 - jepsen.util 3 :ok :write [1 12] INFO [2022-01-10 12:53:18,734] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:18,734] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:18,734] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,735] jepsen worker 88 - jepsen.util 178 :ok :write [29 0] INFO [2022-01-10 12:53:18,735] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:18,735] jepsen worker 70 - jepsen.util 160 :ok :read [23 5] INFO [2022-01-10 12:53:18,735] jepsen worker 3 - jepsen.util 3 :ok :read [1 12] INFO [2022-01-10 12:53:18,736] jepsen worker 70 - jepsen.util 160 :invoke :write [23 45] INFO [2022-01-10 12:53:18,736] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:18,736] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,737] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,737] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:18,738] jepsen worker 3 - jepsen.util 3 :ok :read [1 12] INFO [2022-01-10 12:53:18,738] jepsen worker 85 - jepsen.util 175 :ok :write [28 40] INFO [2022-01-10 12:53:18,738] jepsen worker 88 - jepsen.util 178 :ok :read [29 0] INFO [2022-01-10 12:53:18,738] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:18,738] jepsen worker 44 - jepsen.util 44 :ok :write [14 43] INFO [2022-01-10 12:53:18,739] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,739] jepsen worker 3 - jepsen.util 3 :ok :read [1 12] INFO [2022-01-10 12:53:18,739] jepsen worker 44 - jepsen.util 44 :invoke :write [14 23] INFO [2022-01-10 12:53:18,740] jepsen worker 13 - jepsen.util 13 :ok :write [4 23] INFO [2022-01-10 12:53:18,740] jepsen worker 85 - jepsen.util 175 :invoke :write [28 38] INFO [2022-01-10 12:53:18,741] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:18,742] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:18,743] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,744] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:18,744] jepsen worker 3 - jepsen.util 3 :ok :read [1 12] INFO [2022-01-10 12:53:18,745] jepsen worker 13 - jepsen.util 13 :invoke :write [4 43] INFO [2022-01-10 12:53:18,745] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:18,746] jepsen worker 88 - jepsen.util 178 :invoke :write [29 34] INFO [2022-01-10 12:53:18,746] jepsen worker 78 - jepsen.util 78 :ok :write [26 35] INFO [2022-01-10 12:53:18,747] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:18,747] jepsen worker 77 - jepsen.util 167 :ok :write [25 5] INFO [2022-01-10 12:53:18,748] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:18,749] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:18,750] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:18,750] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:18,750] jepsen worker 77 - jepsen.util 167 :invoke :write [25 38] INFO [2022-01-10 12:53:18,751] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:18,751] jepsen worker 78 - jepsen.util 78 :ok :read [26 35] INFO [2022-01-10 12:53:18,751] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:18,752] jepsen worker 3 - jepsen.util 3 :invoke :write [1 35] INFO [2022-01-10 12:53:18,752] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:18,752] jepsen worker 78 - jepsen.util 78 :ok :read [26 35] INFO [2022-01-10 12:53:18,752] jepsen worker 23 - jepsen.util 23 :invoke :write [7 1] INFO [2022-01-10 12:53:18,753] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:18,754] jepsen worker 78 - jepsen.util 78 :ok :read [26 35] INFO [2022-01-10 12:53:18,755] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:18,756] jepsen worker 78 - jepsen.util 78 :ok :read [26 35] INFO [2022-01-10 12:53:18,756] jepsen worker 52 - jepsen.util 52 :ok :write [17 33] INFO [2022-01-10 12:53:18,757] jepsen worker 35 - jepsen.util 125 :ok :write [11 30] INFO [2022-01-10 12:53:18,757] jepsen worker 78 - jepsen.util 78 :invoke :write [26 44] INFO [2022-01-10 12:53:18,757] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,758] jepsen worker 0 - jepsen.util 90 :ok :write [0 11] INFO [2022-01-10 12:53:18,758] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:18,758] jepsen worker 52 - jepsen.util 52 :ok :read [17 33] INFO [2022-01-10 12:53:18,759] jepsen worker 0 - jepsen.util 90 :invoke :write [0 14] INFO [2022-01-10 12:53:18,759] jepsen worker 35 - jepsen.util 125 :ok :read [11 30] INFO [2022-01-10 12:53:18,759] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,760] jepsen worker 35 - jepsen.util 125 :invoke :write [11 6] INFO [2022-01-10 12:53:18,761] jepsen worker 70 - jepsen.util 160 :ok :write [23 45] INFO [2022-01-10 12:53:18,761] jepsen worker 52 - jepsen.util 52 :ok :read [17 33] INFO [2022-01-10 12:53:18,761] jepsen worker 70 - jepsen.util 160 :invoke :write [23 18] INFO [2022-01-10 12:53:18,762] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,763] jepsen worker 52 - jepsen.util 52 :ok :read [17 33] INFO [2022-01-10 12:53:18,764] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,764] jepsen worker 4 - jepsen.util 4 :ok :write [1 21] INFO [2022-01-10 12:53:18,765] jepsen worker 52 - jepsen.util 52 :ok :read [17 33] INFO [2022-01-10 12:53:18,765] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,765] jepsen worker 15 - jepsen.util 15 :ok :write [5 27] INFO [2022-01-10 12:53:18,765] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,766] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:18,767] jepsen worker 15 - jepsen.util 15 :ok :read [5 27] INFO [2022-01-10 12:53:18,767] jepsen worker 39 - jepsen.util 39 :ok :write [13 14] INFO [2022-01-10 12:53:18,767] jepsen worker 4 - jepsen.util 4 :ok :read [1 21] INFO [2022-01-10 12:53:18,767] jepsen worker 15 - jepsen.util 15 :invoke :write [5 33] INFO [2022-01-10 12:53:18,768] jepsen worker 87 - jepsen.util 87 :ok :write [29 15] INFO [2022-01-10 12:53:18,768] jepsen worker 52 - jepsen.util 52 :ok :read [17 33] INFO [2022-01-10 12:53:18,768] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,769] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,769] jepsen worker 4 - jepsen.util 4 :ok :read [1 21] INFO [2022-01-10 12:53:18,770] jepsen worker 39 - jepsen.util 39 :invoke :write [13 41] INFO [2022-01-10 12:53:18,770] jepsen worker 87 - jepsen.util 87 :ok :read [29 15] INFO [2022-01-10 12:53:18,770] jepsen worker 4 - jepsen.util 4 :invoke :write [1 44] INFO [2022-01-10 12:53:18,771] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,771] jepsen worker 38 - jepsen.util 38 :ok :write [12 30] INFO [2022-01-10 12:53:18,771] jepsen worker 87 - jepsen.util 87 :invoke :write [29 13] INFO [2022-01-10 12:53:18,772] jepsen worker 52 - jepsen.util 52 :ok :read [17 33] INFO [2022-01-10 12:53:18,772] jepsen worker 38 - jepsen.util 38 :invoke :write [12 10] INFO [2022-01-10 12:53:18,773] jepsen worker 52 - jepsen.util 52 :invoke :write [17 48] INFO [2022-01-10 12:53:18,776] jepsen worker 66 - jepsen.util 156 :ok :write [22 47] INFO [2022-01-10 12:53:18,777] jepsen worker 66 - jepsen.util 156 :invoke :write [22 31] INFO [2022-01-10 12:53:18,779] jepsen worker 13 - jepsen.util 13 :ok :write [4 43] INFO [2022-01-10 12:53:18,780] jepsen worker 13 - jepsen.util 13 :invoke :write [4 14] INFO [2022-01-10 12:53:18,783] jepsen worker 50 - jepsen.util 50 :ok :write [16 29] INFO [2022-01-10 12:53:18,784] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:18,785] jepsen worker 50 - jepsen.util 50 :ok :read [16 29] INFO [2022-01-10 12:53:18,785] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:18,786] jepsen worker 50 - jepsen.util 50 :ok :read [16 29] INFO [2022-01-10 12:53:18,787] jepsen worker 85 - jepsen.util 175 :ok :write [28 38] INFO [2022-01-10 12:53:18,787] jepsen worker 50 - jepsen.util 50 :invoke :write [16 20] INFO [2022-01-10 12:53:18,788] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:18,789] jepsen worker 85 - jepsen.util 175 :ok :read [28 38] INFO [2022-01-10 12:53:18,789] jepsen worker 60 - jepsen.util 150 :ok :write [20 22] INFO [2022-01-10 12:53:18,789] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:18,790] jepsen worker 60 - jepsen.util 150 :invoke :read [20 nil] INFO [2022-01-10 12:53:18,790] jepsen worker 85 - jepsen.util 175 :ok :read [28 38] INFO [2022-01-10 12:53:18,791] jepsen worker 60 - jepsen.util 150 :ok :read [20 22] INFO [2022-01-10 12:53:18,791] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:18,791] jepsen worker 60 - jepsen.util 150 :invoke :write [20 23] INFO [2022-01-10 12:53:18,792] jepsen worker 85 - jepsen.util 175 :ok :read [28 38] INFO [2022-01-10 12:53:18,792] jepsen worker 85 - jepsen.util 175 :invoke :write [28 30] INFO [2022-01-10 12:53:18,793] jepsen worker 0 - jepsen.util 90 :ok :write [0 14] INFO [2022-01-10 12:53:18,793] jepsen worker 78 - jepsen.util 78 :ok :write [26 44] INFO [2022-01-10 12:53:18,793] jepsen worker 0 - jepsen.util 90 :invoke :write [0 3] INFO [2022-01-10 12:53:18,794] jepsen worker 78 - jepsen.util 78 :invoke :write [26 16] INFO [2022-01-10 12:53:18,797] jepsen worker 83 - jepsen.util 173 :ok :write [27 10] INFO [2022-01-10 12:53:18,798] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:18,799] jepsen worker 83 - jepsen.util 173 :ok :read [27 10] INFO [2022-01-10 12:53:18,799] jepsen worker 87 - jepsen.util 87 :ok :write [29 13] INFO [2022-01-10 12:53:18,799] jepsen worker 83 - jepsen.util 173 :invoke :write [27 34] INFO [2022-01-10 12:53:18,800] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,801] jepsen worker 87 - jepsen.util 87 :ok :read [29 13] INFO [2022-01-10 12:53:18,801] jepsen worker 87 - jepsen.util 87 :invoke :write [29 37] INFO [2022-01-10 12:53:18,801] jepsen worker 53 - jepsen.util 143 :ok :write [17 39] INFO [2022-01-10 12:53:18,802] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,803] jepsen worker 53 - jepsen.util 143 :ok :read [17 39] INFO [2022-01-10 12:53:18,803] jepsen worker 53 - jepsen.util 143 :invoke :write [17 2] INFO [2022-01-10 12:53:18,803] jepsen worker 7 - jepsen.util 7 :ok :write [2 7] INFO [2022-01-10 12:53:18,804] jepsen worker 7 - jepsen.util 7 :invoke :write [2 27] INFO [2022-01-10 12:53:18,807] jepsen worker 39 - jepsen.util 39 :ok :write [13 41] INFO [2022-01-10 12:53:18,808] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,808] jepsen worker 70 - jepsen.util 160 :ok :write [23 18] INFO [2022-01-10 12:53:18,808] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:18,809] jepsen worker 39 - jepsen.util 39 :ok :read [13 41] INFO [2022-01-10 12:53:18,809] jepsen worker 39 - jepsen.util 39 :invoke :write [13 43] INFO [2022-01-10 12:53:18,809] jepsen worker 70 - jepsen.util 160 :ok :read [23 18] INFO [2022-01-10 12:53:18,810] jepsen worker 70 - jepsen.util 160 :invoke :write [23 29] INFO [2022-01-10 12:53:18,817] jepsen worker 3 - jepsen.util 3 :ok :write [1 35] INFO [2022-01-10 12:53:18,817] jepsen worker 3 - jepsen.util 3 :invoke :write [1 47] INFO [2022-01-10 12:53:18,820] jepsen worker 0 - jepsen.util 90 :ok :write [0 3] INFO [2022-01-10 12:53:18,821] jepsen worker 0 - jepsen.util 90 :invoke :write [0 19] INFO [2022-01-10 12:53:18,821] jepsen worker 44 - jepsen.util 44 :ok :write [14 23] INFO [2022-01-10 12:53:18,822] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:18,823] jepsen worker 44 - jepsen.util 44 :ok :read [14 23] INFO [2022-01-10 12:53:18,824] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:18,825] jepsen worker 38 - jepsen.util 38 :ok :write [12 10] INFO [2022-01-10 12:53:18,825] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,826] jepsen worker 44 - jepsen.util 44 :ok :read [14 23] INFO [2022-01-10 12:53:18,826] jepsen worker 38 - jepsen.util 38 :ok :read [12 10] INFO [2022-01-10 12:53:18,827] jepsen worker 44 - jepsen.util 44 :invoke :write [14 9] INFO [2022-01-10 12:53:18,828] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,828] jepsen worker 87 - jepsen.util 87 :ok :write [29 37] INFO [2022-01-10 12:53:18,829] jepsen worker 38 - jepsen.util 38 :ok :read [12 10] INFO [2022-01-10 12:53:18,829] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,830] jepsen worker 38 - jepsen.util 38 :invoke :write [12 9] INFO [2022-01-10 12:53:18,830] jepsen worker 87 - jepsen.util 87 :ok :read [29 37] INFO [2022-01-10 12:53:18,830] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,831] jepsen worker 87 - jepsen.util 87 :ok :read [29 37] INFO [2022-01-10 12:53:18,831] jepsen worker 77 - jepsen.util 167 :ok :write [25 38] INFO [2022-01-10 12:53:18,832] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,832] jepsen worker 77 - jepsen.util 167 :invoke :write [25 14] INFO [2022-01-10 12:53:18,832] jepsen worker 39 - jepsen.util 39 :ok :write [13 43] INFO [2022-01-10 12:53:18,833] jepsen worker 13 - jepsen.util 13 :ok :write [4 14] INFO [2022-01-10 12:53:18,833] jepsen worker 87 - jepsen.util 87 :ok :read [29 37] INFO [2022-01-10 12:53:18,833] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:18,834] jepsen worker 39 - jepsen.util 39 :invoke :write [13 24] INFO [2022-01-10 12:53:18,834] jepsen worker 13 - jepsen.util 13 :ok :read [4 14] INFO [2022-01-10 12:53:18,834] jepsen worker 87 - jepsen.util 87 :invoke :write [29 34] INFO [2022-01-10 12:53:18,835] jepsen worker 13 - jepsen.util 13 :invoke :write [4 8] INFO [2022-01-10 12:53:18,839] jepsen worker 35 - jepsen.util 125 :ok :write [11 6] INFO [2022-01-10 12:53:18,840] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:18,841] jepsen worker 35 - jepsen.util 125 :ok :read [11 6] INFO [2022-01-10 12:53:18,842] jepsen worker 35 - jepsen.util 125 :invoke :write [11 21] INFO [2022-01-10 12:53:18,843] jepsen worker 85 - jepsen.util 175 :ok :write [28 30] INFO [2022-01-10 12:53:18,843] jepsen worker 85 - jepsen.util 175 :invoke :write [28 6] INFO [2022-01-10 12:53:18,846] jepsen worker 0 - jepsen.util 90 :ok :write [0 19] INFO [2022-01-10 12:53:18,846] jepsen worker 78 - jepsen.util 78 :ok :write [26 16] INFO [2022-01-10 12:53:18,846] jepsen worker 0 - jepsen.util 90 :invoke :write [0 1] INFO [2022-01-10 12:53:18,847] jepsen worker 78 - jepsen.util 78 :invoke :write [26 43] INFO [2022-01-10 12:53:18,847] jepsen worker 15 - jepsen.util 15 :ok :write [5 33] INFO [2022-01-10 12:53:18,848] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:18,849] jepsen worker 15 - jepsen.util 15 :ok :read [5 33] INFO [2022-01-10 12:53:18,849] jepsen worker 15 - jepsen.util 15 :invoke :write [5 47] INFO [2022-01-10 12:53:18,853] jepsen worker 66 - jepsen.util 156 :ok :write [22 31] INFO [2022-01-10 12:53:18,853] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:18,854] jepsen worker 66 - jepsen.util 156 :ok :read [22 31] INFO [2022-01-10 12:53:18,855] jepsen worker 53 - jepsen.util 143 :ok :write [17 2] INFO [2022-01-10 12:53:18,855] jepsen worker 87 - jepsen.util 87 :ok :write [29 34] INFO [2022-01-10 12:53:18,855] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:18,856] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,856] jepsen worker 66 - jepsen.util 156 :ok :read [22 31] INFO [2022-01-10 12:53:18,856] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,857] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:18,857] jepsen worker 87 - jepsen.util 87 :ok :read [29 34] INFO [2022-01-10 12:53:18,857] jepsen worker 53 - jepsen.util 143 :ok :read [17 2] INFO [2022-01-10 12:53:18,858] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,858] jepsen worker 66 - jepsen.util 156 :ok :read [22 31] INFO [2022-01-10 12:53:18,858] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:18,859] jepsen worker 66 - jepsen.util 156 :invoke :write [22 19] INFO [2022-01-10 12:53:18,859] jepsen worker 87 - jepsen.util 87 :ok :read [29 34] INFO [2022-01-10 12:53:18,859] jepsen worker 53 - jepsen.util 143 :ok :read [17 2] INFO [2022-01-10 12:53:18,860] jepsen worker 50 - jepsen.util 50 :ok :write [16 20] INFO [2022-01-10 12:53:18,860] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,860] jepsen worker 53 - jepsen.util 143 :invoke :write [17 3] INFO [2022-01-10 12:53:18,860] jepsen worker 50 - jepsen.util 50 :invoke :write [16 19] INFO [2022-01-10 12:53:18,861] jepsen worker 70 - jepsen.util 160 :ok :write [23 29] INFO [2022-01-10 12:53:18,861] jepsen worker 87 - jepsen.util 87 :ok :read [29 34] INFO [2022-01-10 12:53:18,861] jepsen worker 70 - jepsen.util 160 :invoke :write [23 2] INFO [2022-01-10 12:53:18,862] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,863] jepsen worker 87 - jepsen.util 87 :ok :read [29 34] INFO [2022-01-10 12:53:18,864] jepsen worker 87 - jepsen.util 87 :invoke :write [29 46] INFO [2022-01-10 12:53:18,864] jepsen worker 60 - jepsen.util 150 :ok :write [20 23] INFO [2022-01-10 12:53:18,865] jepsen worker 60 - jepsen.util 150 :invoke :read [20 nil] INFO [2022-01-10 12:53:18,865] jepsen worker 60 - jepsen.util 150 :ok :read [20 23] INFO [2022-01-10 12:53:18,866] jepsen worker 60 - jepsen.util 150 :invoke :write [20 11] INFO [2022-01-10 12:53:18,869] jepsen worker 39 - jepsen.util 39 :ok :write [13 24] INFO [2022-01-10 12:53:18,869] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:18,869] jepsen worker 3 - jepsen.util 3 :ok :write [1 47] INFO [2022-01-10 12:53:18,870] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:18,870] jepsen worker 39 - jepsen.util 39 :ok :read [13 24] INFO [2022-01-10 12:53:18,871] jepsen worker 39 - jepsen.util 39 :invoke :write [13 13] INFO [2022-01-10 12:53:18,871] jepsen worker 3 - jepsen.util 3 :ok :read [1 47] INFO [2022-01-10 12:53:18,872] jepsen worker 3 - jepsen.util 3 :invoke :write [1 11] INFO [2022-01-10 12:53:18,874] jepsen worker 83 - jepsen.util 173 :ok :write [27 34] INFO [2022-01-10 12:53:18,874] jepsen worker 38 - jepsen.util 38 :ok :write [12 9] INFO [2022-01-10 12:53:18,875] jepsen worker 83 - jepsen.util 173 :invoke :write [27 34] INFO [2022-01-10 12:53:18,875] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,876] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:18,876] jepsen worker 0 - jepsen.util 90 :ok :write [0 1] INFO [2022-01-10 12:53:18,877] jepsen worker 0 - jepsen.util 90 :invoke :write [0 19] INFO [2022-01-10 12:53:18,877] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,878] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:18,879] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,879] jepsen worker 7 - jepsen.util 7 :ok :write [2 27] INFO [2022-01-10 12:53:18,880] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:18,880] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:18,880] jepsen worker 7 - jepsen.util 7 :ok :read [2 27] INFO [2022-01-10 12:53:18,881] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:18,881] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:18,882] jepsen worker 7 - jepsen.util 7 :ok :read [2 27] INFO [2022-01-10 12:53:18,882] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:18,882] jepsen worker 13 - jepsen.util 13 :ok :write [4 8] INFO [2022-01-10 12:53:18,883] jepsen worker 38 - jepsen.util 38 :invoke :write [12 12] INFO [2022-01-10 12:53:18,884] jepsen worker 7 - jepsen.util 7 :invoke :write [2 47] INFO [2022-01-10 12:53:18,884] jepsen worker 13 - jepsen.util 13 :invoke :write [4 2] INFO [2022-01-10 12:53:18,886] jepsen worker 87 - jepsen.util 87 :ok :write [29 46] INFO [2022-01-10 12:53:18,887] jepsen worker 23 - jepsen.util 23 :ok :write [7 1] INFO [2022-01-10 12:53:18,887] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:18,888] jepsen worker 87 - jepsen.util 87 :ok :read [29 46] INFO [2022-01-10 12:53:18,888] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:18,889] jepsen worker 87 - jepsen.util 87 :invoke :write [29 5] INFO [2022-01-10 12:53:18,889] jepsen worker 23 - jepsen.util 23 :ok :read [7 1] INFO [2022-01-10 12:53:18,890] jepsen worker 23 - jepsen.util 23 :invoke :write [7 11] INFO [2022-01-10 12:53:18,896] jepsen worker 44 - jepsen.util 44 :ok :write [14 9] INFO [2022-01-10 12:53:18,896] jepsen worker 44 - jepsen.util 44 :invoke :write [14 40] INFO [2022-01-10 12:53:18,902] jepsen worker 77 - jepsen.util 167 :ok :write [25 14] INFO [2022-01-10 12:53:18,902] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:18,903] jepsen worker 77 - jepsen.util 167 :ok :read [25 14] INFO [2022-01-10 12:53:18,904] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:18,905] jepsen worker 77 - jepsen.util 167 :ok :read [25 14] INFO [2022-01-10 12:53:18,905] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:18,906] jepsen worker 77 - jepsen.util 167 :ok :read [25 14] INFO [2022-01-10 12:53:18,906] jepsen worker 77 - jepsen.util 167 :invoke :write [25 4] INFO [2022-01-10 12:53:18,907] jepsen worker 35 - jepsen.util 125 :ok :write [11 21] INFO [2022-01-10 12:53:18,907] jepsen worker 35 - jepsen.util 125 :invoke :write [11 9] INFO [2022-01-10 12:53:18,913] jepsen worker 15 - jepsen.util 15 :ok :write [5 47] INFO [2022-01-10 12:53:18,913] jepsen worker 15 - jepsen.util 15 :invoke :write [5 20] INFO [2022-01-10 12:53:18,917] jepsen worker 66 - jepsen.util 156 :ok :write [22 19] INFO [2022-01-10 12:53:18,917] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:18,918] jepsen worker 66 - jepsen.util 156 :ok :read [22 19] INFO [2022-01-10 12:53:18,919] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:18,919] jepsen worker 66 - jepsen.util 156 :ok :read [22 19] INFO [2022-01-10 12:53:18,920] jepsen worker 66 - jepsen.util 156 :invoke :write [22 36] INFO [2022-01-10 12:53:18,921] jepsen worker 50 - jepsen.util 50 :ok :write [16 19] INFO [2022-01-10 12:53:18,922] jepsen worker 50 - jepsen.util 50 :invoke :write [16 46] INFO [2022-01-10 12:53:18,924] jepsen worker 62 - jepsen.util 152 :ok :write [20 27] INFO [2022-01-10 12:53:18,925] jepsen worker 62 - jepsen.util 152 :invoke :write [20 24] INFO [2022-01-10 12:53:18,930] jepsen worker 83 - jepsen.util 173 :ok :write [27 34] INFO [2022-01-10 12:53:18,931] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:18,931] jepsen worker 83 - jepsen.util 173 :ok :read [27 34] INFO [2022-01-10 12:53:18,932] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:18,933] jepsen worker 83 - jepsen.util 173 :ok :read [27 34] INFO [2022-01-10 12:53:18,933] jepsen worker 83 - jepsen.util 173 :invoke :write [27 6] INFO [2022-01-10 12:53:18,934] jepsen worker 7 - jepsen.util 7 :ok :write [2 47] INFO [2022-01-10 12:53:18,935] jepsen worker 7 - jepsen.util 7 :invoke :write [2 41] INFO [2022-01-10 12:53:18,951] jepsen worker 35 - jepsen.util 125 :ok :write [11 9] INFO [2022-01-10 12:53:18,952] jepsen worker 35 - jepsen.util 125 :invoke :write [11 32] INFO [2022-01-10 12:53:19,042] jepsen worker 85 - jepsen.util 175 :ok :write [28 6] INFO [2022-01-10 12:53:19,042] jepsen worker 85 - jepsen.util 175 :invoke :write [28 20] INFO [2022-01-10 12:53:19,043] jepsen worker 39 - jepsen.util 39 :ok :write [13 13] INFO [2022-01-10 12:53:19,044] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:19,044] jepsen worker 39 - jepsen.util 39 :ok :read [13 13] INFO [2022-01-10 12:53:19,045] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:19,046] jepsen worker 39 - jepsen.util 39 :ok :read [13 13] INFO [2022-01-10 12:53:19,046] jepsen worker 39 - jepsen.util 39 :invoke :write [13 4] INFO [2022-01-10 12:53:19,047] jepsen worker 0 - jepsen.util 90 :ok :write [0 19] INFO [2022-01-10 12:53:19,047] jepsen worker 78 - jepsen.util 78 :ok :write [26 43] INFO [2022-01-10 12:53:19,048] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,048] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,048] jepsen worker 0 - jepsen.util 90 :ok :read [0 19] INFO [2022-01-10 12:53:19,049] jepsen worker 78 - jepsen.util 78 :ok :read [26 43] INFO [2022-01-10 12:53:19,049] jepsen worker 0 - jepsen.util 90 :invoke :write [0 6] INFO [2022-01-10 12:53:19,049] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,050] jepsen worker 78 - jepsen.util 78 :ok :read [26 43] INFO [2022-01-10 12:53:19,051] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,051] jepsen worker 87 - jepsen.util 87 :ok :write [29 5] INFO [2022-01-10 12:53:19,051] jepsen worker 78 - jepsen.util 78 :ok :read [26 43] INFO [2022-01-10 12:53:19,052] jepsen worker 87 - jepsen.util 87 :invoke :write [29 23] INFO [2022-01-10 12:53:19,052] jepsen worker 78 - jepsen.util 78 :invoke :write [26 37] INFO [2022-01-10 12:53:19,054] jepsen worker 53 - jepsen.util 143 :ok :write [17 3] INFO [2022-01-10 12:53:19,055] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:19,055] jepsen worker 53 - jepsen.util 143 :ok :read [17 3] INFO [2022-01-10 12:53:19,056] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:19,057] jepsen worker 53 - jepsen.util 143 :ok :read [17 3] INFO [2022-01-10 12:53:19,057] jepsen worker 53 - jepsen.util 143 :invoke :write [17 40] INFO [2022-01-10 12:53:19,060] jepsen worker 23 - jepsen.util 23 :ok :write [7 11] INFO [2022-01-10 12:53:19,060] jepsen worker 70 - jepsen.util 160 :ok :write [23 2] INFO [2022-01-10 12:53:19,060] jepsen worker 23 - jepsen.util 23 :invoke :write [7 9] INFO [2022-01-10 12:53:19,061] jepsen worker 70 - jepsen.util 160 :invoke :write [23 6] INFO [2022-01-10 12:53:19,065] jepsen worker 44 - jepsen.util 44 :ok :write [14 40] INFO [2022-01-10 12:53:19,066] jepsen worker 44 - jepsen.util 44 :invoke :write [14 25] INFO [2022-01-10 12:53:19,066] jepsen worker 3 - jepsen.util 3 :ok :write [1 11] INFO [2022-01-10 12:53:19,066] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,067] jepsen worker 3 - jepsen.util 3 :ok :read [1 11] INFO [2022-01-10 12:53:19,068] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,069] jepsen worker 3 - jepsen.util 3 :ok :read [1 11] INFO [2022-01-10 12:53:19,070] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,070] jepsen worker 77 - jepsen.util 167 :ok :write [25 4] INFO [2022-01-10 12:53:19,071] jepsen worker 3 - jepsen.util 3 :ok :read [1 11] INFO [2022-01-10 12:53:19,071] jepsen worker 77 - jepsen.util 167 :invoke :write [25 20] INFO [2022-01-10 12:53:19,071] jepsen worker 15 - jepsen.util 15 :ok :write [5 20] INFO [2022-01-10 12:53:19,072] jepsen worker 38 - jepsen.util 38 :ok :write [12 12] INFO [2022-01-10 12:53:19,072] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,073] jepsen worker 15 - jepsen.util 15 :ok :read [5 20] INFO [2022-01-10 12:53:19,073] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,074] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,074] jepsen worker 38 - jepsen.util 38 :ok :read [12 12] INFO [2022-01-10 12:53:19,074] jepsen worker 3 - jepsen.util 3 :invoke :write [1 17] INFO [2022-01-10 12:53:19,075] jepsen worker 15 - jepsen.util 15 :ok :read [5 20] INFO [2022-01-10 12:53:19,075] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,076] jepsen worker 15 - jepsen.util 15 :invoke :write [5 48] INFO [2022-01-10 12:53:19,076] jepsen worker 38 - jepsen.util 38 :ok :read [12 12] INFO [2022-01-10 12:53:19,077] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,078] jepsen worker 66 - jepsen.util 156 :ok :write [22 36] INFO [2022-01-10 12:53:19,078] jepsen worker 38 - jepsen.util 38 :ok :read [12 12] INFO [2022-01-10 12:53:19,078] jepsen worker 66 - jepsen.util 156 :invoke :write [22 28] INFO [2022-01-10 12:53:19,079] jepsen worker 13 - jepsen.util 13 :ok :write [4 2] INFO [2022-01-10 12:53:19,079] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,080] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,080] jepsen worker 38 - jepsen.util 38 :ok :read [12 12] INFO [2022-01-10 12:53:19,081] jepsen worker 13 - jepsen.util 13 :ok :read [4 2] INFO [2022-01-10 12:53:19,081] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,082] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,082] jepsen worker 38 - jepsen.util 38 :ok :read [12 12] INFO [2022-01-10 12:53:19,083] jepsen worker 50 - jepsen.util 50 :ok :write [16 46] INFO [2022-01-10 12:53:19,083] jepsen worker 13 - jepsen.util 13 :ok :read [4 2] INFO [2022-01-10 12:53:19,084] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:19,085] jepsen worker 13 - jepsen.util 13 :invoke :write [4 45] INFO [2022-01-10 12:53:19,085] jepsen worker 50 - jepsen.util 50 :ok :read [16 46] INFO [2022-01-10 12:53:19,085] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,086] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:19,086] jepsen worker 38 - jepsen.util 38 :ok :read [12 12] INFO [2022-01-10 12:53:19,087] jepsen worker 50 - jepsen.util 50 :ok :read [16 46] INFO [2022-01-10 12:53:19,087] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,088] jepsen worker 62 - jepsen.util 152 :ok :write [20 24] INFO [2022-01-10 12:53:19,088] jepsen worker 50 - jepsen.util 50 :invoke :write [16 33] INFO [2022-01-10 12:53:19,088] jepsen worker 38 - jepsen.util 38 :ok :read [12 12] INFO [2022-01-10 12:53:19,089] jepsen worker 62 - jepsen.util 152 :invoke :write [20 40] INFO [2022-01-10 12:53:19,089] jepsen worker 38 - jepsen.util 38 :invoke :write [12 16] INFO [2022-01-10 12:53:19,091] jepsen worker 78 - jepsen.util 78 :ok :write [26 37] INFO [2022-01-10 12:53:19,091] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,092] jepsen worker 78 - jepsen.util 78 :ok :read [26 37] INFO [2022-01-10 12:53:19,093] jepsen worker 78 - jepsen.util 78 :invoke :write [26 37] INFO [2022-01-10 12:53:19,096] jepsen worker 83 - jepsen.util 173 :ok :write [27 6] INFO [2022-01-10 12:53:19,097] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:19,097] jepsen worker 83 - jepsen.util 173 :ok :read [27 6] INFO [2022-01-10 12:53:19,097] jepsen worker 53 - jepsen.util 143 :ok :write [17 40] INFO [2022-01-10 12:53:19,099] jepsen worker 83 - jepsen.util 173 :invoke :write [27 46] INFO [2022-01-10 12:53:19,099] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:19,100] jepsen worker 7 - jepsen.util 7 :ok :write [2 41] INFO [2022-01-10 12:53:19,101] jepsen worker 53 - jepsen.util 143 :ok :read [17 40] INFO [2022-01-10 12:53:19,101] jepsen worker 7 - jepsen.util 7 :invoke :write [2 46] INFO [2022-01-10 12:53:19,101] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:19,102] jepsen worker 53 - jepsen.util 143 :ok :read [17 40] INFO [2022-01-10 12:53:19,102] jepsen worker 53 - jepsen.util 143 :invoke :write [17 38] INFO [2022-01-10 12:53:19,106] jepsen worker 70 - jepsen.util 160 :ok :write [23 6] INFO [2022-01-10 12:53:19,107] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:19,108] jepsen worker 70 - jepsen.util 160 :ok :read [23 6] INFO [2022-01-10 12:53:19,109] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:19,110] jepsen worker 35 - jepsen.util 125 :ok :write [11 32] INFO [2022-01-10 12:53:19,110] jepsen worker 70 - jepsen.util 160 :ok :read [23 6] INFO [2022-01-10 12:53:19,111] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,111] jepsen worker 70 - jepsen.util 160 :invoke :write [23 46] INFO [2022-01-10 12:53:19,112] jepsen worker 35 - jepsen.util 125 :ok :read [11 32] INFO [2022-01-10 12:53:19,113] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,114] jepsen worker 35 - jepsen.util 125 :ok :read [11 32] INFO [2022-01-10 12:53:19,114] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,115] jepsen worker 35 - jepsen.util 125 :ok :read [11 32] INFO [2022-01-10 12:53:19,116] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,116] jepsen worker 35 - jepsen.util 125 :ok :read [11 32] INFO [2022-01-10 12:53:19,117] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,118] jepsen worker 23 - jepsen.util 23 :ok :write [7 9] INFO [2022-01-10 12:53:19,118] jepsen worker 35 - jepsen.util 125 :ok :read [11 32] INFO [2022-01-10 12:53:19,118] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:19,119] jepsen worker 35 - jepsen.util 125 :invoke :write [11 7] INFO [2022-01-10 12:53:19,119] jepsen worker 23 - jepsen.util 23 :ok :read [7 9] INFO [2022-01-10 12:53:19,120] jepsen worker 23 - jepsen.util 23 :invoke :write [7 41] INFO [2022-01-10 12:53:19,124] jepsen worker 13 - jepsen.util 13 :ok :write [4 45] INFO [2022-01-10 12:53:19,124] jepsen worker 13 - jepsen.util 13 :invoke :write [4 36] INFO [2022-01-10 12:53:19,124] jepsen worker 44 - jepsen.util 44 :ok :write [14 25] INFO [2022-01-10 12:53:19,125] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:19,126] jepsen worker 44 - jepsen.util 44 :ok :read [14 25] INFO [2022-01-10 12:53:19,126] jepsen worker 44 - jepsen.util 44 :invoke :write [14 45] INFO [2022-01-10 12:53:19,134] jepsen worker 78 - jepsen.util 78 :ok :write [26 37] INFO [2022-01-10 12:53:19,134] jepsen worker 78 - jepsen.util 78 :invoke :write [26 20] INFO [2022-01-10 12:53:19,135] jepsen worker 77 - jepsen.util 167 :ok :write [25 20] INFO [2022-01-10 12:53:19,135] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:19,136] jepsen worker 77 - jepsen.util 167 :ok :read [25 20] INFO [2022-01-10 12:53:19,137] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:19,138] jepsen worker 77 - jepsen.util 167 :ok :read [25 20] INFO [2022-01-10 12:53:19,139] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:19,139] jepsen worker 77 - jepsen.util 167 :ok :read [25 20] INFO [2022-01-10 12:53:19,140] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:19,141] jepsen worker 77 - jepsen.util 167 :ok :read [25 20] INFO [2022-01-10 12:53:19,141] jepsen worker 77 - jepsen.util 167 :invoke :write [25 36] INFO [2022-01-10 12:53:19,142] jepsen worker 53 - jepsen.util 143 :ok :write [17 38] INFO [2022-01-10 12:53:19,143] jepsen worker 53 - jepsen.util 143 :invoke :write [17 14] INFO [2022-01-10 12:53:19,148] jepsen worker 70 - jepsen.util 160 :ok :write [23 46] INFO [2022-01-10 12:53:19,148] jepsen worker 70 - jepsen.util 160 :invoke :write [23 33] INFO [2022-01-10 12:53:19,153] jepsen worker 13 - jepsen.util 13 :ok :write [4 36] INFO [2022-01-10 12:53:19,154] jepsen worker 13 - jepsen.util 13 :invoke :write [4 31] INFO [2022-01-10 12:53:19,160] jepsen worker 85 - jepsen.util 175 :ok :write [28 20] INFO [2022-01-10 12:53:19,160] jepsen worker 39 - jepsen.util 39 :ok :write [13 4] INFO [2022-01-10 12:53:19,161] jepsen worker 39 - jepsen.util 39 :invoke :write [13 49] INFO [2022-01-10 12:53:19,161] jepsen worker 85 - jepsen.util 175 :invoke :write [28 23] INFO [2022-01-10 12:53:19,168] jepsen worker 3 - jepsen.util 3 :ok :write [1 17] INFO [2022-01-10 12:53:19,168] jepsen worker 0 - jepsen.util 90 :ok :write [0 6] INFO [2022-01-10 12:53:19,169] jepsen worker 3 - jepsen.util 3 :invoke :write [1 31] INFO [2022-01-10 12:53:19,169] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,170] jepsen worker 0 - jepsen.util 90 :ok :read [0 6] INFO [2022-01-10 12:53:19,170] jepsen worker 0 - jepsen.util 90 :invoke :write [0 47] INFO [2022-01-10 12:53:19,176] jepsen worker 38 - jepsen.util 38 :ok :write [12 16] INFO [2022-01-10 12:53:19,177] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,179] jepsen worker 87 - jepsen.util 87 :ok :write [29 23] INFO [2022-01-10 12:53:19,179] jepsen worker 38 - jepsen.util 38 :ok :read [12 16] INFO [2022-01-10 12:53:19,180] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,180] jepsen worker 38 - jepsen.util 38 :invoke :write [12 31] INFO [2022-01-10 12:53:19,181] jepsen worker 85 - jepsen.util 175 :ok :write [28 23] INFO [2022-01-10 12:53:19,181] jepsen worker 87 - jepsen.util 87 :ok :read [29 23] INFO [2022-01-10 12:53:19,181] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:19,181] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,182] jepsen worker 85 - jepsen.util 175 :ok :read [28 23] INFO [2022-01-10 12:53:19,182] jepsen worker 85 - jepsen.util 175 :invoke :write [28 1] INFO [2022-01-10 12:53:19,182] jepsen worker 87 - jepsen.util 87 :ok :read [29 23] INFO [2022-01-10 12:53:19,183] jepsen worker 15 - jepsen.util 15 :ok :write [5 48] INFO [2022-01-10 12:53:19,183] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,183] jepsen worker 15 - jepsen.util 15 :invoke :write [5 27] INFO [2022-01-10 12:53:19,184] jepsen worker 87 - jepsen.util 87 :ok :read [29 23] INFO [2022-01-10 12:53:19,184] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,185] jepsen worker 87 - jepsen.util 87 :ok :read [29 23] INFO [2022-01-10 12:53:19,186] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,187] jepsen worker 87 - jepsen.util 87 :ok :read [29 23] INFO [2022-01-10 12:53:19,187] jepsen worker 87 - jepsen.util 87 :invoke :write [29 14] INFO [2022-01-10 12:53:19,189] jepsen worker 3 - jepsen.util 3 :ok :write [1 31] INFO [2022-01-10 12:53:19,190] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,191] jepsen worker 3 - jepsen.util 3 :ok :read [1 31] INFO [2022-01-10 12:53:19,191] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,192] jepsen worker 3 - jepsen.util 3 :ok :read [1 31] INFO [2022-01-10 12:53:19,192] jepsen worker 3 - jepsen.util 3 :invoke :write [1 16] INFO [2022-01-10 12:53:19,194] jepsen worker 36 - jepsen.util 36 :ok :write [12 31] INFO [2022-01-10 12:53:19,195] jepsen worker 36 - jepsen.util 36 :invoke :write [12 2] INFO [2022-01-10 12:53:19,200] jepsen worker 85 - jepsen.util 175 :ok :write [28 1] INFO [2022-01-10 12:53:19,200] jepsen worker 85 - jepsen.util 175 :invoke :write [28 31] INFO [2022-01-10 12:53:19,205] jepsen worker 3 - jepsen.util 3 :ok :write [1 16] INFO [2022-01-10 12:53:19,205] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,206] jepsen worker 3 - jepsen.util 3 :ok :read [1 16] INFO [2022-01-10 12:53:19,207] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,208] jepsen worker 3 - jepsen.util 3 :ok :read [1 16] INFO [2022-01-10 12:53:19,208] jepsen worker 3 - jepsen.util 3 :invoke :write [1 18] INFO [2022-01-10 12:53:19,210] jepsen worker 38 - jepsen.util 38 :ok :write [12 31] INFO [2022-01-10 12:53:19,211] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:19,215] jepsen worker 39 - jepsen.util 39 :ok :write [13 49] INFO [2022-01-10 12:53:19,216] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:19,216] jepsen worker 39 - jepsen.util 39 :ok :read [13 49] INFO [2022-01-10 12:53:19,217] jepsen worker 85 - jepsen.util 175 :ok :write [28 31] INFO [2022-01-10 12:53:19,217] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:19,218] jepsen worker 39 - jepsen.util 39 :ok :read [13 49] INFO [2022-01-10 12:53:19,218] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:19,219] jepsen worker 85 - jepsen.util 175 :ok :read [28 31] INFO [2022-01-10 12:53:19,219] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:19,220] jepsen worker 39 - jepsen.util 39 :ok :read [13 49] INFO [2022-01-10 12:53:19,220] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:19,220] jepsen worker 39 - jepsen.util 39 :invoke :write [13 32] INFO [2022-01-10 12:53:19,220] jepsen worker 85 - jepsen.util 175 :ok :read [28 31] INFO [2022-01-10 12:53:19,221] jepsen worker 3 - jepsen.util 3 :ok :write [1 18] INFO [2022-01-10 12:53:19,221] jepsen worker 85 - jepsen.util 175 :invoke :write [28 29] INFO [2022-01-10 12:53:19,222] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,222] jepsen worker 0 - jepsen.util 90 :ok :write [0 47] INFO [2022-01-10 12:53:19,223] jepsen worker 3 - jepsen.util 3 :ok :read [1 18] INFO [2022-01-10 12:53:19,223] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,223] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,224] jepsen worker 0 - jepsen.util 90 :ok :read [0 47] INFO [2022-01-10 12:53:19,224] jepsen worker 3 - jepsen.util 3 :ok :read [1 18] INFO [2022-01-10 12:53:19,225] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,225] jepsen worker 3 - jepsen.util 3 :invoke :write [1 15] INFO [2022-01-10 12:53:19,226] jepsen worker 0 - jepsen.util 90 :ok :read [0 47] INFO [2022-01-10 12:53:19,226] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,227] jepsen worker 0 - jepsen.util 90 :ok :read [0 47] INFO [2022-01-10 12:53:19,228] jepsen worker 0 - jepsen.util 90 :invoke :write [0 32] INFO [2022-01-10 12:53:19,229] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:19,229] jepsen worker 87 - jepsen.util 87 :ok :write [29 14] INFO [2022-01-10 12:53:19,230] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,231] jepsen worker 38 - jepsen.util 38 :ok :read [12 11] INFO [2022-01-10 12:53:19,231] jepsen worker 87 - jepsen.util 87 :invoke :write [29 16] INFO [2022-01-10 12:53:19,232] jepsen worker 38 - jepsen.util 38 :invoke :write [12 26] INFO [2022-01-10 12:53:19,235] jepsen worker 85 - jepsen.util 175 :ok :write [28 29] INFO [2022-01-10 12:53:19,236] jepsen worker 85 - jepsen.util 175 :invoke :write [28 47] INFO [2022-01-10 12:53:19,240] jepsen worker 66 - jepsen.util 156 :ok :write [22 28] INFO [2022-01-10 12:53:19,240] jepsen worker 78 - jepsen.util 78 :ok :write [26 20] INFO [2022-01-10 12:53:19,240] jepsen worker 66 - jepsen.util 156 :invoke :write [22 11] INFO [2022-01-10 12:53:19,240] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,241] jepsen worker 78 - jepsen.util 78 :ok :read [26 20] INFO [2022-01-10 12:53:19,242] jepsen worker 78 - jepsen.util 78 :invoke :write [26 19] INFO [2022-01-10 12:53:19,243] jepsen worker 53 - jepsen.util 143 :ok :write [17 14] INFO [2022-01-10 12:53:19,243] jepsen worker 3 - jepsen.util 3 :ok :write [1 15] INFO [2022-01-10 12:53:19,244] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:19,244] jepsen worker 3 - jepsen.util 3 :invoke :write [1 18] INFO [2022-01-10 12:53:19,244] jepsen worker 41 - jepsen.util 41 :ok :write [13 11] INFO [2022-01-10 12:53:19,245] jepsen worker 53 - jepsen.util 143 :ok :read [17 14] INFO [2022-01-10 12:53:19,245] jepsen worker 50 - jepsen.util 50 :ok :write [16 33] INFO [2022-01-10 12:53:19,245] jepsen worker 41 - jepsen.util 41 :invoke :write [13 20] INFO [2022-01-10 12:53:19,246] jepsen worker 53 - jepsen.util 143 :invoke :write [17 12] INFO [2022-01-10 12:53:19,246] jepsen worker 50 - jepsen.util 50 :invoke :write [16 22] INFO [2022-01-10 12:53:19,248] jepsen worker 70 - jepsen.util 160 :ok :write [23 33] INFO [2022-01-10 12:53:19,249] jepsen worker 70 - jepsen.util 160 :invoke :write [23 38] INFO [2022-01-10 12:53:19,251] jepsen worker 62 - jepsen.util 152 :ok :write [20 40] INFO [2022-01-10 12:53:19,251] jepsen worker 38 - jepsen.util 38 :ok :write [12 26] INFO [2022-01-10 12:53:19,251] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:19,251] jepsen worker 0 - jepsen.util 90 :ok :write [0 32] INFO [2022-01-10 12:53:19,252] jepsen worker 0 - jepsen.util 90 :invoke :write [0 18] INFO [2022-01-10 12:53:19,252] jepsen worker 13 - jepsen.util 13 :ok :write [4 31] INFO [2022-01-10 12:53:19,252] jepsen worker 62 - jepsen.util 152 :ok :read [20 40] INFO [2022-01-10 12:53:19,253] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,254] jepsen worker 13 - jepsen.util 13 :ok :read [4 31] INFO [2022-01-10 12:53:19,255] jepsen worker 38 - jepsen.util 38 :invoke :write [12 19] INFO [2022-01-10 12:53:19,255] jepsen worker 62 - jepsen.util 152 :invoke :write [20 17] INFO [2022-01-10 12:53:19,256] jepsen worker 13 - jepsen.util 13 :invoke :write [4 19] INFO [2022-01-10 12:53:19,258] jepsen worker 85 - jepsen.util 175 :ok :write [28 47] INFO [2022-01-10 12:53:19,259] jepsen worker 83 - jepsen.util 173 :ok :write [27 46] INFO [2022-01-10 12:53:19,259] jepsen worker 85 - jepsen.util 175 :invoke :write [28 32] INFO [2022-01-10 12:53:19,260] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:19,261] jepsen worker 83 - jepsen.util 173 :ok :read [27 46] INFO [2022-01-10 12:53:19,261] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:19,261] jepsen worker 78 - jepsen.util 78 :ok :write [26 19] INFO [2022-01-10 12:53:19,262] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,262] jepsen worker 83 - jepsen.util 173 :ok :read [27 46] INFO [2022-01-10 12:53:19,262] jepsen worker 87 - jepsen.util 87 :ok :write [29 16] INFO [2022-01-10 12:53:19,263] jepsen worker 78 - jepsen.util 78 :ok :read [26 19] INFO [2022-01-10 12:53:19,263] jepsen worker 7 - jepsen.util 7 :ok :write [2 46] INFO [2022-01-10 12:53:19,264] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,264] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:19,264] jepsen worker 87 - jepsen.util 87 :invoke :write [29 33] INFO [2022-01-10 12:53:19,264] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:19,265] jepsen worker 78 - jepsen.util 78 :ok :read [26 19] INFO [2022-01-10 12:53:19,265] jepsen worker 7 - jepsen.util 7 :ok :read [2 46] INFO [2022-01-10 12:53:19,265] jepsen worker 83 - jepsen.util 173 :ok :read [27 46] INFO [2022-01-10 12:53:19,265] jepsen worker 78 - jepsen.util 78 :invoke :write [26 48] INFO [2022-01-10 12:53:19,266] jepsen worker 83 - jepsen.util 173 :invoke :write [27 47] INFO [2022-01-10 12:53:19,267] jepsen worker 7 - jepsen.util 7 :invoke :write [2 23] INFO [2022-01-10 12:53:19,267] jepsen worker 53 - jepsen.util 143 :ok :write [17 12] INFO [2022-01-10 12:53:19,268] jepsen worker 53 - jepsen.util 143 :invoke :write [17 8] INFO [2022-01-10 12:53:19,272] jepsen worker 35 - jepsen.util 125 :ok :write [11 7] INFO [2022-01-10 12:53:19,272] jepsen worker 35 - jepsen.util 125 :invoke :write [11 22] INFO [2022-01-10 12:53:19,274] jepsen worker 3 - jepsen.util 3 :ok :write [1 18] INFO [2022-01-10 12:53:19,274] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,275] jepsen worker 39 - jepsen.util 39 :ok :write [13 32] INFO [2022-01-10 12:53:19,275] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:19,276] jepsen worker 3 - jepsen.util 3 :ok :read [1 18] INFO [2022-01-10 12:53:19,276] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,277] jepsen worker 39 - jepsen.util 39 :ok :read [13 32] INFO [2022-01-10 12:53:19,277] jepsen worker 3 - jepsen.util 3 :ok :read [1 18] INFO [2022-01-10 12:53:19,277] jepsen worker 39 - jepsen.util 39 :invoke :write [13 11] INFO [2022-01-10 12:53:19,278] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,278] jepsen worker 3 - jepsen.util 3 :ok :read [1 18] INFO [2022-01-10 12:53:19,279] jepsen worker 3 - jepsen.util 3 :invoke :write [1 29] INFO [2022-01-10 12:53:19,280] jepsen worker 70 - jepsen.util 160 :ok :write [23 38] INFO [2022-01-10 12:53:19,280] jepsen worker 70 - jepsen.util 160 :invoke :write [23 15] INFO [2022-01-10 12:53:19,282] jepsen worker 23 - jepsen.util 23 :ok :write [7 41] INFO [2022-01-10 12:53:19,283] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:19,284] jepsen worker 23 - jepsen.util 23 :ok :read [7 41] INFO [2022-01-10 12:53:19,284] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:19,285] jepsen worker 23 - jepsen.util 23 :ok :read [7 41] INFO [2022-01-10 12:53:19,285] jepsen worker 23 - jepsen.util 23 :invoke :write [7 35] INFO [2022-01-10 12:53:19,287] jepsen worker 0 - jepsen.util 90 :ok :write [0 18] INFO [2022-01-10 12:53:19,288] jepsen worker 0 - jepsen.util 90 :invoke :write [0 30] INFO [2022-01-10 12:53:19,290] jepsen worker 44 - jepsen.util 44 :ok :write [14 45] INFO [2022-01-10 12:53:19,291] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:19,292] jepsen worker 44 - jepsen.util 44 :ok :read [14 45] INFO [2022-01-10 12:53:19,292] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:19,293] jepsen worker 44 - jepsen.util 44 :ok :read [14 45] INFO [2022-01-10 12:53:19,294] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:19,295] jepsen worker 44 - jepsen.util 44 :ok :read [14 45] INFO [2022-01-10 12:53:19,295] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:19,296] jepsen worker 44 - jepsen.util 44 :ok :read [14 45] INFO [2022-01-10 12:53:19,297] jepsen worker 44 - jepsen.util 44 :invoke :write [14 47] INFO [2022-01-10 12:53:19,297] jepsen worker 87 - jepsen.util 87 :ok :write [29 33] INFO [2022-01-10 12:53:19,298] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,299] jepsen worker 77 - jepsen.util 167 :ok :write [25 36] INFO [2022-01-10 12:53:19,299] jepsen worker 87 - jepsen.util 87 :ok :read [29 33] INFO [2022-01-10 12:53:19,300] jepsen worker 77 - jepsen.util 167 :invoke :write [25 3] INFO [2022-01-10 12:53:19,300] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,301] jepsen worker 87 - jepsen.util 87 :ok :read [29 33] INFO [2022-01-10 12:53:19,302] jepsen worker 87 - jepsen.util 87 :invoke :write [29 23] INFO [2022-01-10 12:53:19,305] jepsen worker 15 - jepsen.util 15 :ok :write [5 27] INFO [2022-01-10 12:53:19,306] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,307] jepsen worker 15 - jepsen.util 15 :ok :read [5 27] INFO [2022-01-10 12:53:19,307] jepsen worker 15 - jepsen.util 15 :invoke :write [5 24] INFO [2022-01-10 12:53:19,310] jepsen worker 66 - jepsen.util 156 :ok :write [22 11] INFO [2022-01-10 12:53:19,310] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:19,311] jepsen worker 66 - jepsen.util 156 :ok :read [22 11] INFO [2022-01-10 12:53:19,311] jepsen worker 66 - jepsen.util 156 :invoke :write [22 0] INFO [2022-01-10 12:53:19,314] jepsen worker 50 - jepsen.util 50 :ok :write [16 22] INFO [2022-01-10 12:53:19,315] jepsen worker 50 - jepsen.util 50 :invoke :write [16 6] INFO [2022-01-10 12:53:19,319] jepsen worker 62 - jepsen.util 152 :ok :write [20 17] INFO [2022-01-10 12:53:19,320] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:19,321] jepsen worker 62 - jepsen.util 152 :ok :read [20 17] INFO [2022-01-10 12:53:19,321] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:19,322] jepsen worker 62 - jepsen.util 152 :ok :read [20 17] INFO [2022-01-10 12:53:19,322] jepsen worker 62 - jepsen.util 152 :invoke :write [20 17] INFO [2022-01-10 12:53:19,326] jepsen worker 83 - jepsen.util 173 :ok :write [27 47] INFO [2022-01-10 12:53:19,327] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:19,328] jepsen worker 83 - jepsen.util 173 :ok :read [27 47] INFO [2022-01-10 12:53:19,328] jepsen worker 83 - jepsen.util 173 :invoke :write [27 32] INFO [2022-01-10 12:53:19,330] jepsen worker 7 - jepsen.util 7 :ok :write [2 23] INFO [2022-01-10 12:53:19,330] jepsen worker 7 - jepsen.util 7 :invoke :write [2 11] INFO [2022-01-10 12:53:19,335] jepsen worker 35 - jepsen.util 125 :ok :write [11 22] INFO [2022-01-10 12:53:19,335] jepsen worker 35 - jepsen.util 125 :invoke :write [11 40] INFO [2022-01-10 12:53:19,351] jepsen worker 15 - jepsen.util 15 :ok :write [5 24] INFO [2022-01-10 12:53:19,352] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,353] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 12:53:19,354] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,354] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 12:53:19,355] jepsen worker 15 - jepsen.util 15 :invoke :write [5 18] INFO [2022-01-10 12:53:19,355] jepsen worker 66 - jepsen.util 156 :ok :write [22 0] INFO [2022-01-10 12:53:19,356] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:19,357] jepsen worker 66 - jepsen.util 156 :ok :read [22 0] INFO [2022-01-10 12:53:19,358] jepsen worker 66 - jepsen.util 156 :invoke :write [22 27] INFO [2022-01-10 12:53:19,359] jepsen worker 50 - jepsen.util 50 :ok :write [16 6] INFO [2022-01-10 12:53:19,359] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:19,360] jepsen worker 50 - jepsen.util 50 :ok :read [16 6] INFO [2022-01-10 12:53:19,361] jepsen worker 50 - jepsen.util 50 :invoke :write [16 7] INFO [2022-01-10 12:53:19,366] jepsen worker 83 - jepsen.util 173 :ok :write [27 32] INFO [2022-01-10 12:53:19,367] jepsen worker 83 - jepsen.util 173 :invoke :write [27 49] INFO [2022-01-10 12:53:19,371] jepsen worker 7 - jepsen.util 7 :ok :write [2 11] INFO [2022-01-10 12:53:19,371] jepsen worker 7 - jepsen.util 7 :invoke :write [2 8] INFO [2022-01-10 12:53:19,376] jepsen worker 35 - jepsen.util 125 :ok :write [11 40] INFO [2022-01-10 12:53:19,377] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,378] jepsen worker 35 - jepsen.util 125 :ok :read [11 40] INFO [2022-01-10 12:53:19,378] jepsen worker 35 - jepsen.util 125 :invoke :write [11 1] INFO [2022-01-10 12:53:19,382] jepsen worker 15 - jepsen.util 15 :ok :write [5 18] INFO [2022-01-10 12:53:19,382] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,383] jepsen worker 15 - jepsen.util 15 :ok :read [5 18] INFO [2022-01-10 12:53:19,384] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,385] jepsen worker 15 - jepsen.util 15 :ok :read [5 18] INFO [2022-01-10 12:53:19,385] jepsen worker 15 - jepsen.util 15 :invoke :write [5 35] INFO [2022-01-10 12:53:19,387] jepsen worker 66 - jepsen.util 156 :ok :write [22 27] INFO [2022-01-10 12:53:19,389] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:19,390] jepsen worker 66 - jepsen.util 156 :ok :read [22 27] INFO [2022-01-10 12:53:19,391] jepsen worker 50 - jepsen.util 50 :ok :write [16 7] INFO [2022-01-10 12:53:19,391] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:19,392] jepsen worker 66 - jepsen.util 156 :ok :read [22 27] INFO [2022-01-10 12:53:19,392] jepsen worker 50 - jepsen.util 50 :invoke :write [16 26] INFO [2022-01-10 12:53:19,393] jepsen worker 66 - jepsen.util 156 :invoke :write [22 39] INFO [2022-01-10 12:53:19,395] jepsen worker 83 - jepsen.util 173 :ok :write [27 49] INFO [2022-01-10 12:53:19,395] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:19,396] jepsen worker 83 - jepsen.util 173 :ok :read [27 49] INFO [2022-01-10 12:53:19,397] jepsen worker 83 - jepsen.util 173 :invoke :write [27 49] INFO [2022-01-10 12:53:19,399] jepsen worker 7 - jepsen.util 7 :ok :write [2 8] INFO [2022-01-10 12:53:19,399] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:19,400] jepsen worker 7 - jepsen.util 7 :ok :read [2 8] INFO [2022-01-10 12:53:19,401] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:19,402] jepsen worker 7 - jepsen.util 7 :ok :read [2 8] INFO [2022-01-10 12:53:19,402] jepsen worker 7 - jepsen.util 7 :invoke :write [2 14] INFO [2022-01-10 12:53:19,405] jepsen worker 35 - jepsen.util 125 :ok :write [11 1] INFO [2022-01-10 12:53:19,406] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,407] jepsen worker 35 - jepsen.util 125 :ok :read [11 1] INFO [2022-01-10 12:53:19,407] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,408] jepsen worker 35 - jepsen.util 125 :ok :read [11 1] INFO [2022-01-10 12:53:19,409] jepsen worker 35 - jepsen.util 125 :invoke :write [11 41] INFO [2022-01-10 12:53:19,412] jepsen worker 15 - jepsen.util 15 :ok :write [5 35] INFO [2022-01-10 12:53:19,413] jepsen worker 15 - jepsen.util 15 :invoke :write [5 0] INFO [2022-01-10 12:53:19,417] jepsen worker 66 - jepsen.util 156 :ok :write [22 39] INFO [2022-01-10 12:53:19,417] jepsen worker 66 - jepsen.util 156 :invoke :write [22 10] INFO [2022-01-10 12:53:19,420] jepsen worker 50 - jepsen.util 50 :ok :write [16 26] INFO [2022-01-10 12:53:19,420] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:19,422] jepsen worker 50 - jepsen.util 50 :ok :read [16 26] INFO [2022-01-10 12:53:19,422] jepsen worker 50 - jepsen.util 50 :invoke :write [16 44] INFO [2022-01-10 12:53:19,425] jepsen worker 83 - jepsen.util 173 :ok :write [27 49] INFO [2022-01-10 12:53:19,426] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:19,427] jepsen worker 83 - jepsen.util 173 :ok :read [27 49] INFO [2022-01-10 12:53:19,428] jepsen worker 83 - jepsen.util 173 :invoke :write [27 30] INFO [2022-01-10 12:53:19,429] jepsen worker 7 - jepsen.util 7 :ok :write [2 14] INFO [2022-01-10 12:53:19,429] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:19,430] jepsen worker 38 - jepsen.util 38 :ok :write [12 19] INFO [2022-01-10 12:53:19,430] jepsen worker 7 - jepsen.util 7 :ok :read [2 14] INFO [2022-01-10 12:53:19,431] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,431] jepsen worker 41 - jepsen.util 41 :ok :write [13 20] INFO [2022-01-10 12:53:19,432] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:19,432] jepsen worker 38 - jepsen.util 38 :ok :read [12 19] INFO [2022-01-10 12:53:19,432] jepsen worker 41 - jepsen.util 41 :invoke :write [13 18] INFO [2022-01-10 12:53:19,433] jepsen worker 7 - jepsen.util 7 :ok :read [2 14] INFO [2022-01-10 12:53:19,433] jepsen worker 38 - jepsen.util 38 :invoke :write [12 19] INFO [2022-01-10 12:53:19,434] jepsen worker 7 - jepsen.util 7 :invoke :write [2 25] INFO [2022-01-10 12:53:19,435] jepsen worker 35 - jepsen.util 125 :ok :write [11 41] INFO [2022-01-10 12:53:19,436] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,436] jepsen worker 23 - jepsen.util 23 :ok :write [7 35] INFO [2022-01-10 12:53:19,437] jepsen worker 35 - jepsen.util 125 :ok :read [11 41] INFO [2022-01-10 12:53:19,437] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:19,437] jepsen worker 13 - jepsen.util 13 :ok :write [4 19] INFO [2022-01-10 12:53:19,438] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,439] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,439] jepsen worker 35 - jepsen.util 125 :ok :read [11 41] INFO [2022-01-10 12:53:19,440] jepsen worker 13 - jepsen.util 13 :ok :read [4 19] INFO [2022-01-10 12:53:19,440] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,440] jepsen worker 13 - jepsen.util 13 :invoke :write [4 21] INFO [2022-01-10 12:53:19,441] jepsen worker 35 - jepsen.util 125 :ok :read [11 41] INFO [2022-01-10 12:53:19,441] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,442] jepsen worker 35 - jepsen.util 125 :ok :read [11 41] INFO [2022-01-10 12:53:19,443] jepsen worker 35 - jepsen.util 125 :invoke :write [11 17] INFO [2022-01-10 12:53:19,445] jepsen worker 0 - jepsen.util 90 :ok :write [0 30] INFO [2022-01-10 12:53:19,445] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,447] jepsen worker 0 - jepsen.util 90 :ok :read [0 30] INFO [2022-01-10 12:53:19,447] jepsen worker 85 - jepsen.util 175 :ok :write [28 32] INFO [2022-01-10 12:53:19,447] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,448] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:19,448] jepsen worker 0 - jepsen.util 90 :ok :read [0 30] INFO [2022-01-10 12:53:19,449] jepsen worker 85 - jepsen.util 175 :ok :read [28 32] INFO [2022-01-10 12:53:19,449] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,449] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:19,450] jepsen worker 0 - jepsen.util 90 :ok :read [0 30] INFO [2022-01-10 12:53:19,450] jepsen worker 85 - jepsen.util 175 :ok :read [28 32] INFO [2022-01-10 12:53:19,450] jepsen worker 44 - jepsen.util 44 :ok :write [14 47] INFO [2022-01-10 12:53:19,450] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,451] jepsen worker 23 - jepsen.util 23 :ok :read [7 35] INFO [2022-01-10 12:53:19,451] jepsen worker 44 - jepsen.util 44 :invoke :write [14 22] INFO [2022-01-10 12:53:19,451] jepsen worker 0 - jepsen.util 90 :ok :read [0 30] INFO [2022-01-10 12:53:19,452] jepsen worker 23 - jepsen.util 23 :invoke :write [7 40] INFO [2022-01-10 12:53:19,452] jepsen worker 85 - jepsen.util 175 :invoke :write [28 9] INFO [2022-01-10 12:53:19,453] jepsen worker 0 - jepsen.util 90 :invoke :write [0 3] INFO [2022-01-10 12:53:19,453] jepsen worker 78 - jepsen.util 78 :ok :write [26 48] INFO [2022-01-10 12:53:19,454] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,454] jepsen worker 87 - jepsen.util 87 :ok :write [29 23] INFO [2022-01-10 12:53:19,455] jepsen worker 87 - jepsen.util 87 :invoke :write [29 27] INFO [2022-01-10 12:53:19,456] jepsen worker 78 - jepsen.util 78 :ok :read [26 48] INFO [2022-01-10 12:53:19,457] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,458] jepsen worker 78 - jepsen.util 78 :ok :read [26 48] INFO [2022-01-10 12:53:19,458] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,458] jepsen worker 53 - jepsen.util 143 :ok :write [17 8] INFO [2022-01-10 12:53:19,459] jepsen worker 53 - jepsen.util 143 :invoke :write [17 8] INFO [2022-01-10 12:53:19,459] jepsen worker 78 - jepsen.util 78 :ok :read [26 48] INFO [2022-01-10 12:53:19,460] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,461] jepsen worker 78 - jepsen.util 78 :ok :read [26 48] INFO [2022-01-10 12:53:19,461] jepsen worker 78 - jepsen.util 78 :invoke :write [26 12] INFO [2022-01-10 12:53:19,464] jepsen worker 77 - jepsen.util 167 :ok :write [25 3] INFO [2022-01-10 12:53:19,465] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:19,466] jepsen worker 77 - jepsen.util 167 :ok :read [25 3] INFO [2022-01-10 12:53:19,466] jepsen worker 77 - jepsen.util 167 :invoke :write [25 2] INFO [2022-01-10 12:53:19,469] jepsen worker 62 - jepsen.util 152 :ok :write [20 17] INFO [2022-01-10 12:53:19,470] jepsen worker 3 - jepsen.util 3 :ok :write [1 29] INFO [2022-01-10 12:53:19,470] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:19,471] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,471] jepsen worker 62 - jepsen.util 152 :ok :read [20 17] INFO [2022-01-10 12:53:19,472] jepsen worker 62 - jepsen.util 152 :invoke :write [20 25] INFO [2022-01-10 12:53:19,472] jepsen worker 3 - jepsen.util 3 :ok :read [1 29] INFO [2022-01-10 12:53:19,473] jepsen worker 3 - jepsen.util 3 :invoke :write [1 29] INFO [2022-01-10 12:53:19,474] jepsen worker 41 - jepsen.util 41 :ok :write [13 18] INFO [2022-01-10 12:53:19,474] jepsen worker 41 - jepsen.util 41 :invoke :write [13 5] INFO [2022-01-10 12:53:19,478] jepsen worker 70 - jepsen.util 160 :ok :write [23 15] INFO [2022-01-10 12:53:19,479] jepsen worker 70 - jepsen.util 160 :invoke :write [23 2] INFO [2022-01-10 12:53:19,484] jepsen worker 38 - jepsen.util 38 :ok :write [12 19] INFO [2022-01-10 12:53:19,484] jepsen worker 38 - jepsen.util 38 :invoke :write [12 34] INFO [2022-01-10 12:53:19,573] jepsen worker 85 - jepsen.util 175 :ok :write [28 9] INFO [2022-01-10 12:53:19,573] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:19,574] jepsen worker 0 - jepsen.util 90 :ok :write [0 3] INFO [2022-01-10 12:53:19,574] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,574] jepsen worker 85 - jepsen.util 175 :ok :read [28 9] INFO [2022-01-10 12:53:19,575] jepsen worker 0 - jepsen.util 90 :ok :read [0 3] INFO [2022-01-10 12:53:19,575] jepsen worker 15 - jepsen.util 15 :ok :write [5 0] INFO [2022-01-10 12:53:19,575] jepsen worker 85 - jepsen.util 175 :invoke :write [28 25] INFO [2022-01-10 12:53:19,576] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,577] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,577] jepsen worker 0 - jepsen.util 90 :ok :read [0 3] INFO [2022-01-10 12:53:19,577] jepsen worker 15 - jepsen.util 15 :ok :read [5 0] INFO [2022-01-10 12:53:19,578] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,578] jepsen worker 0 - jepsen.util 90 :ok :read [0 3] INFO [2022-01-10 12:53:19,578] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,579] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,579] jepsen worker 15 - jepsen.util 15 :ok :read [5 0] INFO [2022-01-10 12:53:19,581] jepsen worker 3 - jepsen.util 3 :ok :write [1 29] INFO [2022-01-10 12:53:19,581] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,582] jepsen worker 0 - jepsen.util 90 :ok :read [0 3] INFO [2022-01-10 12:53:19,582] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,583] jepsen worker 15 - jepsen.util 15 :ok :read [5 0] INFO [2022-01-10 12:53:19,583] jepsen worker 13 - jepsen.util 13 :ok :write [4 21] INFO [2022-01-10 12:53:19,583] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,583] jepsen worker 0 - jepsen.util 90 :ok :read [0 3] INFO [2022-01-10 12:53:19,583] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,583] jepsen worker 66 - jepsen.util 156 :ok :write [22 10] INFO [2022-01-10 12:53:19,584] jepsen worker 13 - jepsen.util 13 :invoke :write [4 26] INFO [2022-01-10 12:53:19,584] jepsen worker 3 - jepsen.util 3 :ok :read [1 29] INFO [2022-01-10 12:53:19,584] jepsen worker 15 - jepsen.util 15 :ok :read [5 0] INFO [2022-01-10 12:53:19,584] jepsen worker 66 - jepsen.util 156 :invoke :write [22 33] INFO [2022-01-10 12:53:19,585] jepsen worker 87 - jepsen.util 87 :ok :write [29 27] INFO [2022-01-10 12:53:19,585] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,586] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,586] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,586] jepsen worker 15 - jepsen.util 15 :ok :read [5 0] INFO [2022-01-10 12:53:19,587] jepsen worker 37 - jepsen.util 127 :ok :write [12 35] INFO [2022-01-10 12:53:19,587] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,587] jepsen worker 3 - jepsen.util 3 :ok :read [1 29] INFO [2022-01-10 12:53:19,587] jepsen worker 0 - jepsen.util 90 :ok :read [0 3] INFO [2022-01-10 12:53:19,588] jepsen worker 15 - jepsen.util 15 :invoke :write [5 9] INFO [2022-01-10 12:53:19,588] jepsen worker 87 - jepsen.util 87 :ok :read [29 27] INFO [2022-01-10 12:53:19,588] jepsen worker 78 - jepsen.util 78 :ok :write [26 12] INFO [2022-01-10 12:53:19,589] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,589] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,590] jepsen worker 37 - jepsen.util 127 :ok :read [12 35] INFO [2022-01-10 12:53:19,590] jepsen worker 50 - jepsen.util 50 :ok :write [16 44] INFO [2022-01-10 12:53:19,590] jepsen worker 3 - jepsen.util 3 :ok :read [1 29] INFO [2022-01-10 12:53:19,590] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,591] jepsen worker 78 - jepsen.util 78 :ok :read [26 12] INFO [2022-01-10 12:53:19,592] jepsen worker 0 - jepsen.util 90 :invoke :write [0 7] INFO [2022-01-10 12:53:19,592] jepsen worker 41 - jepsen.util 41 :ok :write [13 5] INFO [2022-01-10 12:53:19,592] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,593] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,593] jepsen worker 37 - jepsen.util 127 :ok :read [12 35] INFO [2022-01-10 12:53:19,594] jepsen worker 87 - jepsen.util 87 :invoke :write [29 7] INFO [2022-01-10 12:53:19,594] jepsen worker 78 - jepsen.util 78 :ok :read [26 12] INFO [2022-01-10 12:53:19,594] jepsen worker 83 - jepsen.util 173 :ok :write [27 30] INFO [2022-01-10 12:53:19,595] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,596] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:19,597] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,597] jepsen worker 53 - jepsen.util 143 :ok :write [17 8] INFO [2022-01-10 12:53:19,597] jepsen worker 37 - jepsen.util 127 :ok :read [12 35] INFO [2022-01-10 12:53:19,597] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,598] jepsen worker 85 - jepsen.util 175 :ok :write [28 25] INFO [2022-01-10 12:53:19,598] jepsen worker 3 - jepsen.util 3 :ok :read [1 29] INFO [2022-01-10 12:53:19,598] jepsen worker 78 - jepsen.util 78 :ok :read [26 12] INFO [2022-01-10 12:53:19,598] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:19,598] jepsen worker 41 - jepsen.util 41 :ok :read [13 5] INFO [2022-01-10 12:53:19,599] jepsen worker 53 - jepsen.util 143 :ok :read [17 8] INFO [2022-01-10 12:53:19,600] jepsen worker 78 - jepsen.util 78 :invoke :write [26 35] INFO [2022-01-10 12:53:19,600] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:19,601] jepsen worker 7 - jepsen.util 7 :ok :write [2 25] INFO [2022-01-10 12:53:19,601] jepsen worker 85 - jepsen.util 175 :ok :read [28 25] INFO [2022-01-10 12:53:19,602] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,602] jepsen worker 70 - jepsen.util 160 :ok :write [23 2] INFO [2022-01-10 12:53:19,603] jepsen worker 37 - jepsen.util 127 :ok :read [12 35] INFO [2022-01-10 12:53:19,603] jepsen worker 83 - jepsen.util 173 :invoke :write [27 40] INFO [2022-01-10 12:53:19,604] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:19,605] jepsen worker 53 - jepsen.util 143 :ok :read [17 8] INFO [2022-01-10 12:53:19,606] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:19,606] jepsen worker 53 - jepsen.util 143 :invoke :write [17 33] INFO [2022-01-10 12:53:19,606] jepsen worker 50 - jepsen.util 50 :ok :read [16 44] INFO [2022-01-10 12:53:19,607] jepsen worker 35 - jepsen.util 125 :ok :write [11 17] INFO [2022-01-10 12:53:19,607] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:19,608] jepsen worker 7 - jepsen.util 7 :ok :read [2 25] INFO [2022-01-10 12:53:19,609] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:19,610] jepsen worker 85 - jepsen.util 175 :invoke :write [28 0] INFO [2022-01-10 12:53:19,610] jepsen worker 50 - jepsen.util 50 :ok :read [16 44] INFO [2022-01-10 12:53:19,611] jepsen worker 13 - jepsen.util 13 :ok :write [4 26] INFO [2022-01-10 12:53:19,611] jepsen worker 3 - jepsen.util 3 :invoke :write [1 43] INFO [2022-01-10 12:53:19,612] jepsen worker 50 - jepsen.util 50 :invoke :write [16 3] INFO [2022-01-10 12:53:19,613] jepsen worker 70 - jepsen.util 160 :invoke :write [23 14] INFO [2022-01-10 12:53:19,614] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:19,614] jepsen worker 23 - jepsen.util 23 :ok :write [7 40] INFO [2022-01-10 12:53:19,615] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:19,616] jepsen worker 35 - jepsen.util 125 :ok :read [11 17] INFO [2022-01-10 12:53:19,616] jepsen worker 23 - jepsen.util 23 :invoke :write [7 3] INFO [2022-01-10 12:53:19,616] jepsen worker 41 - jepsen.util 41 :ok :read [13 5] INFO [2022-01-10 12:53:19,616] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,617] jepsen worker 35 - jepsen.util 125 :invoke :write [11 36] INFO [2022-01-10 12:53:19,618] jepsen worker 41 - jepsen.util 41 :invoke :write [13 29] INFO [2022-01-10 12:53:19,618] jepsen worker 13 - jepsen.util 13 :ok :read [4 26] INFO [2022-01-10 12:53:19,618] jepsen worker 38 - jepsen.util 38 :ok :write [12 34] INFO [2022-01-10 12:53:19,618] jepsen worker 7 - jepsen.util 7 :invoke :write [2 11] INFO [2022-01-10 12:53:19,619] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,620] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,620] jepsen worker 38 - jepsen.util 38 :ok :read [12 34] INFO [2022-01-10 12:53:19,621] jepsen worker 44 - jepsen.util 44 :ok :write [14 22] INFO [2022-01-10 12:53:19,621] jepsen worker 13 - jepsen.util 13 :ok :read [4 26] INFO [2022-01-10 12:53:19,621] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,621] jepsen worker 38 - jepsen.util 38 :invoke :write [12 21] INFO [2022-01-10 12:53:19,622] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,622] jepsen worker 37 - jepsen.util 127 :ok :read [12 34] INFO [2022-01-10 12:53:19,622] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:19,623] jepsen worker 13 - jepsen.util 13 :ok :read [4 26] INFO [2022-01-10 12:53:19,623] jepsen worker 37 - jepsen.util 127 :invoke :write [12 17] INFO [2022-01-10 12:53:19,624] jepsen worker 44 - jepsen.util 44 :ok :read [14 22] INFO [2022-01-10 12:53:19,624] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,624] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:19,625] jepsen worker 13 - jepsen.util 13 :ok :read [4 26] INFO [2022-01-10 12:53:19,625] jepsen worker 13 - jepsen.util 13 :invoke :write [4 13] INFO [2022-01-10 12:53:19,626] jepsen worker 44 - jepsen.util 44 :ok :read [14 22] INFO [2022-01-10 12:53:19,626] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:19,627] jepsen worker 44 - jepsen.util 44 :ok :read [14 22] INFO [2022-01-10 12:53:19,627] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:19,628] jepsen worker 78 - jepsen.util 78 :ok :write [26 35] INFO [2022-01-10 12:53:19,628] jepsen worker 87 - jepsen.util 87 :ok :write [29 7] INFO [2022-01-10 12:53:19,628] jepsen worker 44 - jepsen.util 44 :ok :read [14 22] INFO [2022-01-10 12:53:19,629] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,629] jepsen worker 77 - jepsen.util 167 :ok :write [25 2] INFO [2022-01-10 12:53:19,629] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,630] jepsen worker 78 - jepsen.util 78 :ok :read [26 35] INFO [2022-01-10 12:53:19,630] jepsen worker 44 - jepsen.util 44 :invoke :write [14 24] INFO [2022-01-10 12:53:19,631] jepsen worker 87 - jepsen.util 87 :ok :read [29 7] INFO [2022-01-10 12:53:19,631] jepsen worker 78 - jepsen.util 78 :invoke :write [26 17] INFO [2022-01-10 12:53:19,632] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,632] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:19,633] jepsen worker 87 - jepsen.util 87 :ok :read [29 7] INFO [2022-01-10 12:53:19,633] jepsen worker 77 - jepsen.util 167 :ok :read [25 2] INFO [2022-01-10 12:53:19,634] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,634] jepsen worker 77 - jepsen.util 167 :invoke :write [25 5] INFO [2022-01-10 12:53:19,635] jepsen worker 87 - jepsen.util 87 :ok :read [29 7] INFO [2022-01-10 12:53:19,635] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,636] jepsen worker 53 - jepsen.util 143 :ok :write [17 33] INFO [2022-01-10 12:53:19,636] jepsen worker 0 - jepsen.util 90 :ok :write [0 7] INFO [2022-01-10 12:53:19,636] jepsen worker 87 - jepsen.util 87 :ok :read [29 7] INFO [2022-01-10 12:53:19,637] jepsen worker 0 - jepsen.util 90 :invoke :write [0 12] INFO [2022-01-10 12:53:19,637] jepsen worker 87 - jepsen.util 87 :invoke :write [29 39] INFO [2022-01-10 12:53:19,638] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:19,639] jepsen worker 53 - jepsen.util 143 :ok :read [17 33] INFO [2022-01-10 12:53:19,639] jepsen worker 62 - jepsen.util 152 :ok :write [20 25] INFO [2022-01-10 12:53:19,640] jepsen worker 53 - jepsen.util 143 :invoke :write [17 20] INFO [2022-01-10 12:53:19,640] jepsen worker 62 - jepsen.util 152 :invoke :write [20 11] INFO [2022-01-10 12:53:19,645] jepsen worker 15 - jepsen.util 15 :ok :write [5 9] INFO [2022-01-10 12:53:19,645] jepsen worker 41 - jepsen.util 41 :ok :write [13 29] INFO [2022-01-10 12:53:19,645] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,646] jepsen worker 41 - jepsen.util 41 :invoke :write [13 7] INFO [2022-01-10 12:53:19,646] jepsen worker 85 - jepsen.util 175 :ok :write [28 0] INFO [2022-01-10 12:53:19,646] jepsen worker 15 - jepsen.util 15 :ok :read [5 9] INFO [2022-01-10 12:53:19,647] jepsen worker 85 - jepsen.util 175 :invoke :write [28 4] INFO [2022-01-10 12:53:19,647] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,649] jepsen worker 15 - jepsen.util 15 :ok :read [5 9] INFO [2022-01-10 12:53:19,650] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,651] jepsen worker 15 - jepsen.util 15 :ok :read [5 9] INFO [2022-01-10 12:53:19,651] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:19,653] jepsen worker 66 - jepsen.util 156 :ok :write [22 33] INFO [2022-01-10 12:53:19,653] jepsen worker 70 - jepsen.util 160 :ok :write [23 14] INFO [2022-01-10 12:53:19,653] jepsen worker 15 - jepsen.util 15 :ok :read [5 9] INFO [2022-01-10 12:53:19,654] jepsen worker 66 - jepsen.util 156 :invoke :write [22 11] INFO [2022-01-10 12:53:19,655] jepsen worker 15 - jepsen.util 15 :invoke :write [5 9] INFO [2022-01-10 12:53:19,655] jepsen worker 70 - jepsen.util 160 :invoke :write [23 0] INFO [2022-01-10 12:53:19,659] jepsen worker 50 - jepsen.util 50 :ok :write [16 3] INFO [2022-01-10 12:53:19,659] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:19,661] jepsen worker 50 - jepsen.util 50 :ok :read [16 3] INFO [2022-01-10 12:53:19,661] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:19,662] jepsen worker 50 - jepsen.util 50 :ok :read [16 3] INFO [2022-01-10 12:53:19,663] jepsen worker 50 - jepsen.util 50 :invoke :write [16 27] INFO [2022-01-10 12:53:19,663] jepsen worker 13 - jepsen.util 13 :ok :write [4 13] INFO [2022-01-10 12:53:19,664] jepsen worker 87 - jepsen.util 87 :ok :write [29 39] INFO [2022-01-10 12:53:19,664] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,664] jepsen worker 83 - jepsen.util 173 :ok :write [27 40] INFO [2022-01-10 12:53:19,664] jepsen worker 87 - jepsen.util 87 :invoke :write [29 38] INFO [2022-01-10 12:53:19,665] jepsen worker 83 - jepsen.util 173 :invoke :write [27 0] INFO [2022-01-10 12:53:19,665] jepsen worker 13 - jepsen.util 13 :ok :read [4 13] INFO [2022-01-10 12:53:19,666] jepsen worker 13 - jepsen.util 13 :invoke :write [4 44] INFO [2022-01-10 12:53:19,674] jepsen worker 3 - jepsen.util 3 :ok :write [1 43] INFO [2022-01-10 12:53:19,675] jepsen worker 0 - jepsen.util 90 :ok :write [0 12] INFO [2022-01-10 12:53:19,675] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,675] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,676] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:19,677] jepsen worker 3 - jepsen.util 3 :invoke :write [1 23] INFO [2022-01-10 12:53:19,677] jepsen worker 0 - jepsen.util 90 :ok :read [0 12] INFO [2022-01-10 12:53:19,677] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,678] jepsen worker 0 - jepsen.util 90 :ok :read [0 12] INFO [2022-01-10 12:53:19,679] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,680] jepsen worker 0 - jepsen.util 90 :ok :read [0 12] INFO [2022-01-10 12:53:19,680] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,681] jepsen worker 0 - jepsen.util 90 :ok :read [0 12] INFO [2022-01-10 12:53:19,681] jepsen worker 38 - jepsen.util 38 :ok :write [12 21] INFO [2022-01-10 12:53:19,682] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,682] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,682] jepsen worker 0 - jepsen.util 90 :ok :read [0 12] INFO [2022-01-10 12:53:19,683] jepsen worker 38 - jepsen.util 38 :ok :read [12 21] INFO [2022-01-10 12:53:19,683] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,683] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,684] jepsen worker 0 - jepsen.util 90 :ok :read [0 12] INFO [2022-01-10 12:53:19,684] jepsen worker 38 - jepsen.util 38 :ok :read [12 21] INFO [2022-01-10 12:53:19,685] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,685] jepsen worker 38 - jepsen.util 38 :invoke :write [12 15] INFO [2022-01-10 12:53:19,685] jepsen worker 0 - jepsen.util 90 :ok :read [0 12] INFO [2022-01-10 12:53:19,686] jepsen worker 0 - jepsen.util 90 :invoke :write [0 20] INFO [2022-01-10 12:53:19,689] jepsen worker 41 - jepsen.util 41 :ok :write [13 7] INFO [2022-01-10 12:53:19,689] jepsen worker 78 - jepsen.util 78 :ok :write [26 17] INFO [2022-01-10 12:53:19,690] jepsen worker 78 - jepsen.util 78 :invoke :write [26 31] INFO [2022-01-10 12:53:19,690] jepsen worker 41 - jepsen.util 41 :invoke :write [13 44] INFO [2022-01-10 12:53:19,694] jepsen worker 87 - jepsen.util 87 :ok :write [29 38] INFO [2022-01-10 12:53:19,694] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,695] jepsen worker 53 - jepsen.util 143 :ok :write [17 20] INFO [2022-01-10 12:53:19,695] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:19,695] jepsen worker 87 - jepsen.util 87 :ok :read [29 38] INFO [2022-01-10 12:53:19,696] jepsen worker 53 - jepsen.util 143 :ok :read [17 20] INFO [2022-01-10 12:53:19,696] jepsen worker 87 - jepsen.util 87 :invoke :write [29 10] INFO [2022-01-10 12:53:19,697] jepsen worker 53 - jepsen.util 143 :invoke :write [17 14] INFO [2022-01-10 12:53:19,702] jepsen worker 85 - jepsen.util 175 :ok :write [28 4] INFO [2022-01-10 12:53:19,703] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:19,703] jepsen worker 0 - jepsen.util 90 :ok :write [0 20] INFO [2022-01-10 12:53:19,704] jepsen worker 85 - jepsen.util 175 :ok :read [28 4] INFO [2022-01-10 12:53:19,704] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,705] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:19,705] jepsen worker 85 - jepsen.util 175 :ok :read [28 4] INFO [2022-01-10 12:53:19,705] jepsen worker 0 - jepsen.util 90 :ok :read [0 20] INFO [2022-01-10 12:53:19,706] jepsen worker 85 - jepsen.util 175 :invoke :write [28 21] INFO [2022-01-10 12:53:19,706] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,707] jepsen worker 0 - jepsen.util 90 :ok :read [0 20] INFO [2022-01-10 12:53:19,708] jepsen worker 0 - jepsen.util 90 :invoke :write [0 29] INFO [2022-01-10 12:53:19,713] jepsen worker 41 - jepsen.util 41 :ok :write [13 44] INFO [2022-01-10 12:53:19,714] jepsen worker 41 - jepsen.util 41 :invoke :write [13 18] INFO [2022-01-10 12:53:19,719] jepsen worker 87 - jepsen.util 87 :ok :write [29 10] INFO [2022-01-10 12:53:19,719] jepsen worker 87 - jepsen.util 87 :invoke :write [29 16] INFO [2022-01-10 12:53:19,725] jepsen worker 3 - jepsen.util 3 :ok :write [1 23] INFO [2022-01-10 12:53:19,726] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,727] jepsen worker 3 - jepsen.util 3 :ok :read [1 23] INFO [2022-01-10 12:53:19,727] jepsen worker 0 - jepsen.util 90 :ok :write [0 29] INFO [2022-01-10 12:53:19,728] jepsen worker 3 - jepsen.util 3 :invoke :write [1 2] INFO [2022-01-10 12:53:19,728] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,729] jepsen worker 0 - jepsen.util 90 :ok :read [0 29] INFO [2022-01-10 12:53:19,730] jepsen worker 0 - jepsen.util 90 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,731] jepsen worker 0 - jepsen.util 90 :ok :read [0 29] INFO [2022-01-10 12:53:19,731] jepsen worker 0 - jepsen.util 90 :invoke :write [0 2] INFO [2022-01-10 12:53:19,732] jepsen worker 37 - jepsen.util 127 :ok :write [12 17] INFO [2022-01-10 12:53:19,733] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,733] jepsen worker 37 - jepsen.util 127 :ok :read [12 17] INFO [2022-01-10 12:53:19,734] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,735] jepsen worker 37 - jepsen.util 127 :ok :read [12 17] INFO [2022-01-10 12:53:19,736] jepsen worker 37 - jepsen.util 127 :invoke :write [12 34] INFO [2022-01-10 12:53:19,738] jepsen worker 41 - jepsen.util 41 :ok :write [13 18] INFO [2022-01-10 12:53:19,739] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:19,740] jepsen worker 41 - jepsen.util 41 :ok :read [13 18] INFO [2022-01-10 12:53:19,741] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:19,741] jepsen worker 41 - jepsen.util 41 :ok :read [13 18] INFO [2022-01-10 12:53:19,742] jepsen worker 41 - jepsen.util 41 :invoke :write [13 15] INFO [2022-01-10 12:53:19,745] jepsen worker 87 - jepsen.util 87 :ok :write [29 16] INFO [2022-01-10 12:53:19,746] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:19,746] jepsen worker 87 - jepsen.util 87 :ok :read [29 16] INFO [2022-01-10 12:53:19,747] jepsen worker 87 - jepsen.util 87 :invoke :write [29 32] INFO [2022-01-10 12:53:19,754] jepsen worker 2 - jepsen.util 92 :ok :write [0 26] INFO [2022-01-10 12:53:19,755] jepsen worker 85 - jepsen.util 175 :ok :write [28 21] INFO [2022-01-10 12:53:19,755] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,756] jepsen worker 85 - jepsen.util 175 :invoke :write [28 16] INFO [2022-01-10 12:53:19,756] jepsen worker 2 - jepsen.util 92 :ok :read [0 26] INFO [2022-01-10 12:53:19,757] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,758] jepsen worker 2 - jepsen.util 92 :ok :read [0 26] INFO [2022-01-10 12:53:19,759] jepsen worker 2 - jepsen.util 92 :invoke :write [0 17] INFO [2022-01-10 12:53:19,761] jepsen worker 41 - jepsen.util 41 :ok :write [13 15] INFO [2022-01-10 12:53:19,762] jepsen worker 41 - jepsen.util 41 :invoke :write [13 16] INFO [2022-01-10 12:53:19,763] jepsen worker 3 - jepsen.util 3 :ok :write [1 2] INFO [2022-01-10 12:53:19,764] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:19,765] jepsen worker 3 - jepsen.util 3 :ok :read [1 2] INFO [2022-01-10 12:53:19,766] jepsen worker 3 - jepsen.util 3 :invoke :write [1 23] INFO [2022-01-10 12:53:19,767] jepsen worker 87 - jepsen.util 87 :ok :write [29 32] INFO [2022-01-10 12:53:19,768] jepsen worker 87 - jepsen.util 87 :invoke :write [29 17] INFO [2022-01-10 12:53:19,772] jepsen worker 38 - jepsen.util 38 :ok :write [12 15] INFO [2022-01-10 12:53:19,773] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:19,774] jepsen worker 38 - jepsen.util 38 :ok :read [12 15] INFO [2022-01-10 12:53:19,775] jepsen worker 38 - jepsen.util 38 :invoke :write [12 3] INFO [2022-01-10 12:53:19,777] jepsen worker 0 - jepsen.util 90 :ok :write [0 2] INFO [2022-01-10 12:53:19,778] jepsen worker 0 - jepsen.util 90 :invoke :write [0 3] INFO [2022-01-10 12:53:19,804] jepsen worker 70 - jepsen.util 160 :ok :write [23 0] INFO [2022-01-10 12:53:19,805] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:19,806] jepsen worker 70 - jepsen.util 160 :ok :read [23 0] INFO [2022-01-10 12:53:19,806] jepsen worker 70 - jepsen.util 160 :invoke :write [23 39] INFO [2022-01-10 12:53:19,810] jepsen worker 7 - jepsen.util 7 :ok :write [2 11] INFO [2022-01-10 12:53:19,810] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:19,811] jepsen worker 13 - jepsen.util 13 :ok :write [4 44] INFO [2022-01-10 12:53:19,811] jepsen worker 7 - jepsen.util 7 :ok :read [2 11] INFO [2022-01-10 12:53:19,811] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,812] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:19,812] jepsen worker 13 - jepsen.util 13 :ok :read [4 44] INFO [2022-01-10 12:53:19,813] jepsen worker 7 - jepsen.util 7 :ok :read [2 11] INFO [2022-01-10 12:53:19,813] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,813] jepsen worker 7 - jepsen.util 7 :invoke :write [2 35] INFO [2022-01-10 12:53:19,814] jepsen worker 13 - jepsen.util 13 :ok :read [4 44] INFO [2022-01-10 12:53:19,814] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,815] jepsen worker 78 - jepsen.util 78 :ok :write [26 31] INFO [2022-01-10 12:53:19,816] jepsen worker 78 - jepsen.util 78 :invoke :write [26 14] INFO [2022-01-10 12:53:19,816] jepsen worker 13 - jepsen.util 13 :ok :read [4 44] INFO [2022-01-10 12:53:19,817] jepsen worker 13 - jepsen.util 13 :invoke :write [4 22] INFO [2022-01-10 12:53:19,820] jepsen worker 35 - jepsen.util 125 :ok :write [11 36] INFO [2022-01-10 12:53:19,821] jepsen worker 35 - jepsen.util 125 :invoke :write [11 0] INFO [2022-01-10 12:53:19,824] jepsen worker 53 - jepsen.util 143 :ok :write [17 14] INFO [2022-01-10 12:53:19,824] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:19,825] jepsen worker 53 - jepsen.util 143 :ok :read [17 14] INFO [2022-01-10 12:53:19,826] jepsen worker 53 - jepsen.util 143 :invoke :write [17 46] INFO [2022-01-10 12:53:19,828] jepsen worker 23 - jepsen.util 23 :ok :write [7 3] INFO [2022-01-10 12:53:19,828] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:19,829] jepsen worker 23 - jepsen.util 23 :ok :read [7 3] INFO [2022-01-10 12:53:19,830] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:19,831] jepsen worker 23 - jepsen.util 23 :ok :read [7 3] INFO [2022-01-10 12:53:19,832] jepsen worker 23 - jepsen.util 23 :invoke :write [7 22] INFO [2022-01-10 12:53:19,843] jepsen worker 42 - jepsen.util 132 :ok :write [14 2] INFO [2022-01-10 12:53:19,845] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:19,846] jepsen worker 42 - jepsen.util 132 :ok :read [14 2] INFO [2022-01-10 12:53:19,847] jepsen worker 77 - jepsen.util 167 :ok :write [25 5] INFO [2022-01-10 12:53:19,847] jepsen worker 42 - jepsen.util 132 :invoke :write [14 31] INFO [2022-01-10 12:53:19,847] jepsen worker 77 - jepsen.util 167 :invoke :write [25 38] INFO [2022-01-10 12:53:19,856] jepsen worker 62 - jepsen.util 152 :ok :write [20 11] INFO [2022-01-10 12:53:19,857] jepsen worker 62 - jepsen.util 152 :invoke :write [20 26] INFO [2022-01-10 12:53:19,867] jepsen worker 15 - jepsen.util 15 :ok :write [5 9] INFO [2022-01-10 12:53:19,868] jepsen worker 15 - jepsen.util 15 :invoke :write [5 3] INFO [2022-01-10 12:53:19,870] jepsen worker 66 - jepsen.util 156 :ok :write [22 11] INFO [2022-01-10 12:53:19,871] jepsen worker 66 - jepsen.util 156 :invoke :write [22 25] INFO [2022-01-10 12:53:19,875] jepsen worker 50 - jepsen.util 50 :ok :write [16 27] INFO [2022-01-10 12:53:19,876] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:19,877] jepsen worker 50 - jepsen.util 50 :ok :read [16 27] INFO [2022-01-10 12:53:19,878] jepsen worker 50 - jepsen.util 50 :invoke :write [16 19] INFO [2022-01-10 12:53:19,878] jepsen worker 83 - jepsen.util 173 :ok :write [27 0] INFO [2022-01-10 12:53:19,879] jepsen worker 83 - jepsen.util 173 :invoke :write [27 32] INFO [2022-01-10 12:53:19,894] jepsen worker 85 - jepsen.util 175 :ok :write [28 16] INFO [2022-01-10 12:53:19,894] jepsen worker 41 - jepsen.util 41 :ok :write [13 16] INFO [2022-01-10 12:53:19,894] jepsen worker 85 - jepsen.util 175 :invoke :write [28 39] INFO [2022-01-10 12:53:19,895] jepsen worker 41 - jepsen.util 41 :invoke :write [13 42] INFO [2022-01-10 12:53:19,917] jepsen worker 3 - jepsen.util 3 :ok :write [1 23] INFO [2022-01-10 12:53:19,917] jepsen worker 87 - jepsen.util 87 :ok :write [29 17] INFO [2022-01-10 12:53:19,918] jepsen worker 3 - jepsen.util 3 :invoke :write [1 0] INFO [2022-01-10 12:53:19,918] jepsen worker 87 - jepsen.util 87 :invoke :write [29 18] INFO [2022-01-10 12:53:19,922] jepsen worker 2 - jepsen.util 92 :ok :write [0 17] INFO [2022-01-10 12:53:19,923] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,924] jepsen worker 2 - jepsen.util 92 :ok :read [0 17] INFO [2022-01-10 12:53:19,924] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,924] jepsen worker 37 - jepsen.util 127 :ok :write [12 34] INFO [2022-01-10 12:53:19,925] jepsen worker 2 - jepsen.util 92 :ok :read [0 17] INFO [2022-01-10 12:53:19,925] jepsen worker 37 - jepsen.util 127 :invoke :write [12 12] INFO [2022-01-10 12:53:19,926] jepsen worker 23 - jepsen.util 23 :ok :write [7 22] INFO [2022-01-10 12:53:19,926] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:19,927] jepsen worker 23 - jepsen.util 23 :invoke :write [7 29] INFO [2022-01-10 12:53:19,927] jepsen worker 2 - jepsen.util 92 :ok :read [0 17] INFO [2022-01-10 12:53:19,928] jepsen worker 2 - jepsen.util 92 :invoke :write [0 43] INFO [2022-01-10 12:53:19,931] jepsen worker 70 - jepsen.util 160 :ok :write [23 39] INFO [2022-01-10 12:53:19,932] jepsen worker 70 - jepsen.util 160 :invoke :write [23 25] INFO [2022-01-10 12:53:19,933] jepsen worker 42 - jepsen.util 132 :ok :write [14 31] INFO [2022-01-10 12:53:19,934] jepsen worker 42 - jepsen.util 132 :invoke :read [14 nil] INFO [2022-01-10 12:53:19,935] jepsen worker 42 - jepsen.util 132 :ok :read [14 31] INFO [2022-01-10 12:53:19,936] jepsen worker 42 - jepsen.util 132 :invoke :write [14 4] INFO [2022-01-10 12:53:19,939] jepsen worker 77 - jepsen.util 167 :ok :write [25 38] INFO [2022-01-10 12:53:19,940] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:19,940] jepsen worker 13 - jepsen.util 13 :ok :write [4 22] INFO [2022-01-10 12:53:19,941] jepsen worker 77 - jepsen.util 167 :ok :read [25 38] INFO [2022-01-10 12:53:19,941] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:19,941] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:19,942] jepsen worker 13 - jepsen.util 13 :ok :read [4 22] INFO [2022-01-10 12:53:19,942] jepsen worker 77 - jepsen.util 167 :ok :read [25 38] INFO [2022-01-10 12:53:19,942] jepsen worker 13 - jepsen.util 13 :invoke :write [4 43] INFO [2022-01-10 12:53:19,943] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:19,944] jepsen worker 77 - jepsen.util 167 :ok :read [25 38] INFO [2022-01-10 12:53:19,944] jepsen worker 77 - jepsen.util 167 :invoke :write [25 5] INFO [2022-01-10 12:53:19,947] jepsen worker 62 - jepsen.util 152 :ok :write [20 26] INFO [2022-01-10 12:53:19,947] jepsen worker 62 - jepsen.util 152 :invoke :write [20 36] INFO [2022-01-10 12:53:19,950] jepsen worker 78 - jepsen.util 78 :ok :write [26 14] INFO [2022-01-10 12:53:19,950] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:19,951] jepsen worker 78 - jepsen.util 78 :ok :read [26 14] INFO [2022-01-10 12:53:19,952] jepsen worker 78 - jepsen.util 78 :invoke :write [26 4] INFO [2022-01-10 12:53:19,960] jepsen worker 53 - jepsen.util 143 :ok :write [17 46] INFO [2022-01-10 12:53:19,960] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:19,961] jepsen worker 53 - jepsen.util 143 :ok :read [17 46] INFO [2022-01-10 12:53:19,962] jepsen worker 53 - jepsen.util 143 :invoke :write [17 18] INFO [2022-01-10 12:53:19,973] jepsen worker 23 - jepsen.util 23 :ok :write [7 29] INFO [2022-01-10 12:53:19,973] jepsen worker 23 - jepsen.util 23 :invoke :write [7 6] INFO [2022-01-10 12:53:19,976] jepsen worker 44 - jepsen.util 44 :ok :write [14 24] INFO [2022-01-10 12:53:19,977] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:19,978] jepsen worker 44 - jepsen.util 44 :ok :read [14 24] INFO [2022-01-10 12:53:19,978] jepsen worker 44 - jepsen.util 44 :invoke :write [14 35] INFO [2022-01-10 12:53:19,982] jepsen worker 70 - jepsen.util 160 :ok :write [23 25] INFO [2022-01-10 12:53:19,982] jepsen worker 70 - jepsen.util 160 :invoke :write [23 22] INFO [2022-01-10 12:53:19,982] jepsen worker 77 - jepsen.util 167 :ok :write [25 5] INFO [2022-01-10 12:53:19,983] jepsen worker 77 - jepsen.util 167 :invoke :write [25 6] INFO [2022-01-10 12:53:19,989] jepsen worker 62 - jepsen.util 152 :ok :write [20 36] INFO [2022-01-10 12:53:19,989] jepsen worker 62 - jepsen.util 152 :invoke :write [20 9] INFO [2022-01-10 12:53:19,990] jepsen worker 13 - jepsen.util 13 :ok :write [4 43] INFO [2022-01-10 12:53:19,990] jepsen worker 13 - jepsen.util 13 :invoke :write [4 3] INFO [2022-01-10 12:53:20,106] jepsen worker 78 - jepsen.util 78 :ok :write [26 4] INFO [2022-01-10 12:53:20,107] jepsen worker 78 - jepsen.util 78 :invoke :write [26 26] INFO [2022-01-10 12:53:20,112] jepsen worker 23 - jepsen.util 23 :ok :write [7 6] INFO [2022-01-10 12:53:20,112] jepsen worker 53 - jepsen.util 143 :ok :write [17 18] INFO [2022-01-10 12:53:20,113] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:20,114] jepsen worker 53 - jepsen.util 143 :invoke :write [17 35] INFO [2022-01-10 12:53:20,115] jepsen worker 23 - jepsen.util 23 :ok :read [7 6] INFO [2022-01-10 12:53:20,116] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:20,117] jepsen worker 23 - jepsen.util 23 :ok :read [7 6] INFO [2022-01-10 12:53:20,118] jepsen worker 70 - jepsen.util 160 :ok :write [23 22] INFO [2022-01-10 12:53:20,118] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:20,119] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:20,119] jepsen worker 23 - jepsen.util 23 :ok :read [7 6] INFO [2022-01-10 12:53:20,120] jepsen worker 70 - jepsen.util 160 :ok :read [23 22] INFO [2022-01-10 12:53:20,120] jepsen worker 23 - jepsen.util 23 :invoke :write [7 36] INFO [2022-01-10 12:53:20,120] jepsen worker 44 - jepsen.util 44 :ok :write [14 35] INFO [2022-01-10 12:53:20,120] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:20,121] jepsen worker 44 - jepsen.util 44 :invoke :write [14 25] INFO [2022-01-10 12:53:20,121] jepsen worker 70 - jepsen.util 160 :ok :read [23 22] INFO [2022-01-10 12:53:20,122] jepsen worker 70 - jepsen.util 160 :invoke :write [23 46] INFO [2022-01-10 12:53:20,124] jepsen worker 13 - jepsen.util 13 :ok :write [4 3] INFO [2022-01-10 12:53:20,124] jepsen worker 13 - jepsen.util 13 :invoke :write [4 45] INFO [2022-01-10 12:53:20,126] jepsen worker 77 - jepsen.util 167 :ok :write [25 6] INFO [2022-01-10 12:53:20,127] jepsen worker 78 - jepsen.util 78 :ok :write [26 26] INFO [2022-01-10 12:53:20,127] jepsen worker 77 - jepsen.util 167 :invoke :write [25 13] INFO [2022-01-10 12:53:20,128] jepsen worker 78 - jepsen.util 78 :invoke :write [26 1] INFO [2022-01-10 12:53:20,131] jepsen worker 53 - jepsen.util 143 :ok :write [17 35] INFO [2022-01-10 12:53:20,132] jepsen worker 53 - jepsen.util 143 :invoke :write [17 26] INFO [2022-01-10 12:53:20,134] jepsen worker 62 - jepsen.util 152 :ok :write [20 9] INFO [2022-01-10 12:53:20,134] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:20,136] jepsen worker 62 - jepsen.util 152 :ok :read [20 9] INFO [2022-01-10 12:53:20,136] jepsen worker 62 - jepsen.util 152 :invoke :write [20 38] INFO [2022-01-10 12:53:20,138] jepsen worker 70 - jepsen.util 160 :ok :write [23 46] INFO [2022-01-10 12:53:20,138] jepsen worker 70 - jepsen.util 160 :invoke :write [23 43] INFO [2022-01-10 12:53:20,139] jepsen worker 23 - jepsen.util 23 :ok :write [7 36] INFO [2022-01-10 12:53:20,140] jepsen worker 23 - jepsen.util 23 :invoke :write [7 33] INFO [2022-01-10 12:53:20,141] jepsen worker 13 - jepsen.util 13 :ok :write [4 45] INFO [2022-01-10 12:53:20,141] jepsen worker 13 - jepsen.util 13 :invoke :write [4 38] INFO [2022-01-10 12:53:20,145] jepsen worker 44 - jepsen.util 44 :ok :write [14 25] INFO [2022-01-10 12:53:20,145] jepsen worker 78 - jepsen.util 78 :ok :write [26 1] INFO [2022-01-10 12:53:20,146] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:20,147] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:20,147] jepsen worker 44 - jepsen.util 44 :ok :read [14 25] INFO [2022-01-10 12:53:20,148] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:20,148] jepsen worker 78 - jepsen.util 78 :ok :read [26 1] INFO [2022-01-10 12:53:20,148] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:20,149] jepsen worker 44 - jepsen.util 44 :ok :read [14 25] INFO [2022-01-10 12:53:20,149] jepsen worker 44 - jepsen.util 44 :invoke :write [14 2] INFO [2022-01-10 12:53:20,150] jepsen worker 78 - jepsen.util 78 :ok :read [26 1] INFO [2022-01-10 12:53:20,150] jepsen worker 78 - jepsen.util 78 :invoke :write [26 35] INFO [2022-01-10 12:53:20,151] jepsen worker 53 - jepsen.util 143 :ok :write [17 26] INFO [2022-01-10 12:53:20,151] jepsen worker 53 - jepsen.util 143 :invoke :write [17 42] INFO [2022-01-10 12:53:20,153] jepsen worker 77 - jepsen.util 167 :ok :write [25 13] INFO [2022-01-10 12:53:20,153] jepsen worker 77 - jepsen.util 167 :invoke :write [25 21] INFO [2022-01-10 12:53:20,156] jepsen worker 70 - jepsen.util 160 :ok :write [23 43] INFO [2022-01-10 12:53:20,157] jepsen worker 70 - jepsen.util 160 :invoke :write [23 4] INFO [2022-01-10 12:53:20,160] jepsen worker 62 - jepsen.util 152 :ok :write [20 38] INFO [2022-01-10 12:53:20,161] jepsen worker 62 - jepsen.util 152 :invoke :write [20 15] INFO [2022-01-10 12:53:20,165] jepsen worker 13 - jepsen.util 13 :ok :write [4 38] INFO [2022-01-10 12:53:20,165] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:20,166] jepsen worker 13 - jepsen.util 13 :ok :read [4 38] INFO [2022-01-10 12:53:20,166] jepsen worker 23 - jepsen.util 23 :ok :write [7 33] INFO [2022-01-10 12:53:20,167] jepsen worker 13 - jepsen.util 13 :invoke :write [4 25] INFO [2022-01-10 12:53:20,167] jepsen worker 23 - jepsen.util 23 :invoke :write [7 48] INFO [2022-01-10 12:53:20,169] jepsen worker 78 - jepsen.util 78 :ok :write [26 35] INFO [2022-01-10 12:53:20,170] jepsen worker 78 - jepsen.util 78 :invoke :write [26 15] INFO [2022-01-10 12:53:20,172] jepsen worker 44 - jepsen.util 44 :ok :write [14 2] INFO [2022-01-10 12:53:20,172] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:20,173] jepsen worker 44 - jepsen.util 44 :ok :read [14 2] INFO [2022-01-10 12:53:20,174] jepsen worker 53 - jepsen.util 143 :ok :write [17 42] INFO [2022-01-10 12:53:20,174] jepsen worker 44 - jepsen.util 44 :invoke :write [14 29] INFO [2022-01-10 12:53:20,175] jepsen worker 53 - jepsen.util 143 :invoke :write [17 31] INFO [2022-01-10 12:53:20,178] jepsen worker 77 - jepsen.util 167 :ok :write [25 21] INFO [2022-01-10 12:53:20,178] jepsen worker 77 - jepsen.util 167 :invoke :write [25 33] INFO [2022-01-10 12:53:20,181] jepsen worker 70 - jepsen.util 160 :ok :write [23 4] INFO [2022-01-10 12:53:20,182] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:20,183] jepsen worker 70 - jepsen.util 160 :ok :read [23 4] INFO [2022-01-10 12:53:20,184] jepsen worker 70 - jepsen.util 160 :invoke :write [23 47] INFO [2022-01-10 12:53:20,185] jepsen worker 62 - jepsen.util 152 :ok :write [20 15] INFO [2022-01-10 12:53:20,186] jepsen worker 62 - jepsen.util 152 :invoke :write [20 5] INFO [2022-01-10 12:53:20,188] jepsen worker 13 - jepsen.util 13 :ok :write [4 25] INFO [2022-01-10 12:53:20,189] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:20,190] jepsen worker 13 - jepsen.util 13 :ok :read [4 25] INFO [2022-01-10 12:53:20,190] jepsen worker 13 - jepsen.util 13 :invoke :write [4 4] INFO [2022-01-10 12:53:20,207] jepsen worker 78 - jepsen.util 78 :ok :write [26 15] INFO [2022-01-10 12:53:20,207] jepsen worker 53 - jepsen.util 143 :ok :write [17 31] INFO [2022-01-10 12:53:20,208] jepsen worker 77 - jepsen.util 167 :ok :write [25 33] INFO [2022-01-10 12:53:20,208] jepsen worker 23 - jepsen.util 23 :ok :write [7 48] INFO [2022-01-10 12:53:20,208] jepsen worker 44 - jepsen.util 44 :ok :write [14 29] INFO [2022-01-10 12:53:20,209] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:20,210] jepsen worker 53 - jepsen.util 143 :ok :read [17 31] INFO [2022-01-10 12:53:20,210] jepsen worker 77 - jepsen.util 167 :invoke :write [25 2] INFO [2022-01-10 12:53:20,211] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:20,211] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:20,212] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:20,212] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:20,213] jepsen worker 78 - jepsen.util 78 :ok :read [26 15] INFO [2022-01-10 12:53:20,213] jepsen worker 78 - jepsen.util 78 :invoke :write [26 44] INFO [2022-01-10 12:53:20,213] jepsen worker 44 - jepsen.util 44 :ok :read [14 29] INFO [2022-01-10 12:53:20,214] jepsen worker 23 - jepsen.util 23 :ok :read [7 48] INFO [2022-01-10 12:53:20,214] jepsen worker 53 - jepsen.util 143 :ok :read [17 31] INFO [2022-01-10 12:53:20,214] jepsen worker 44 - jepsen.util 44 :invoke :write [14 42] INFO [2022-01-10 12:53:20,214] jepsen worker 70 - jepsen.util 160 :ok :write [23 47] INFO [2022-01-10 12:53:20,215] jepsen worker 23 - jepsen.util 23 :invoke :write [7 49] INFO [2022-01-10 12:53:20,216] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:20,216] jepsen worker 70 - jepsen.util 160 :invoke :write [23 5] INFO [2022-01-10 12:53:20,217] jepsen worker 53 - jepsen.util 143 :ok :read [17 31] INFO [2022-01-10 12:53:20,217] jepsen worker 53 - jepsen.util 143 :invoke :write [17 34] INFO [2022-01-10 12:53:20,218] jepsen worker 62 - jepsen.util 152 :ok :write [20 5] INFO [2022-01-10 12:53:20,218] jepsen worker 62 - jepsen.util 152 :invoke :write [20 13] INFO [2022-01-10 12:53:20,220] jepsen worker 13 - jepsen.util 13 :ok :write [4 4] INFO [2022-01-10 12:53:20,221] jepsen worker 13 - jepsen.util 13 :invoke :write [4 3] INFO [2022-01-10 12:53:20,230] jepsen worker 7 - jepsen.util 7 :ok :write [2 35] INFO [2022-01-10 12:53:20,230] jepsen worker 85 - jepsen.util 175 :ok :write [28 39] INFO [2022-01-10 12:53:20,230] jepsen worker 23 - jepsen.util 23 :ok :write [7 49] INFO [2022-01-10 12:53:20,231] jepsen worker 23 - jepsen.util 23 :invoke :write [7 23] INFO [2022-01-10 12:53:20,232] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:20,232] jepsen worker 7 - jepsen.util 7 :invoke :write [2 18] INFO [2022-01-10 12:53:20,232] jepsen worker 85 - jepsen.util 175 :ok :read [28 39] INFO [2022-01-10 12:53:20,233] jepsen worker 85 - jepsen.util 175 :invoke :write [28 48] INFO [2022-01-10 12:53:20,233] jepsen worker 43 - jepsen.util 43 :ok :write [14 41] INFO [2022-01-10 12:53:20,234] jepsen worker 78 - jepsen.util 78 :ok :write [26 44] INFO [2022-01-10 12:53:20,234] jepsen worker 43 - jepsen.util 43 :invoke :write [14 1] INFO [2022-01-10 12:53:20,235] jepsen worker 78 - jepsen.util 78 :invoke :write [26 0] INFO [2022-01-10 12:53:20,238] jepsen worker 3 - jepsen.util 3 :ok :write [1 0] INFO [2022-01-10 12:53:20,238] jepsen worker 35 - jepsen.util 125 :ok :write [11 0] INFO [2022-01-10 12:53:20,238] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:20,239] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:20,239] jepsen worker 70 - jepsen.util 160 :ok :write [23 5] INFO [2022-01-10 12:53:20,239] jepsen worker 3 - jepsen.util 3 :ok :read [1 0] INFO [2022-01-10 12:53:20,240] jepsen worker 3 - jepsen.util 3 :invoke :write [1 30] INFO [2022-01-10 12:53:20,240] jepsen worker 35 - jepsen.util 125 :ok :read [11 0] INFO [2022-01-10 12:53:20,240] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:20,241] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:20,242] jepsen worker 70 - jepsen.util 160 :ok :read [23 5] INFO [2022-01-10 12:53:20,242] jepsen worker 35 - jepsen.util 125 :ok :read [11 0] INFO [2022-01-10 12:53:20,242] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:20,243] jepsen worker 15 - jepsen.util 15 :ok :write [5 3] INFO [2022-01-10 12:53:20,243] jepsen worker 37 - jepsen.util 127 :ok :write [12 12] INFO [2022-01-10 12:53:20,243] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:20,243] jepsen worker 77 - jepsen.util 167 :ok :write [25 2] INFO [2022-01-10 12:53:20,243] jepsen worker 70 - jepsen.util 160 :ok :read [23 5] INFO [2022-01-10 12:53:20,243] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,244] jepsen worker 35 - jepsen.util 125 :ok :read [11 0] INFO [2022-01-10 12:53:20,244] jepsen worker 37 - jepsen.util 127 :ok :read [12 12] INFO [2022-01-10 12:53:20,245] jepsen worker 15 - jepsen.util 15 :invoke :write [5 14] INFO [2022-01-10 12:53:20,246] jepsen worker 53 - jepsen.util 143 :ok :write [17 34] INFO [2022-01-10 12:53:20,246] jepsen worker 77 - jepsen.util 167 :invoke :write [25 7] INFO [2022-01-10 12:53:20,247] jepsen worker 35 - jepsen.util 125 :invoke :write [11 48] INFO [2022-01-10 12:53:20,248] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,248] jepsen worker 70 - jepsen.util 160 :invoke :write [23 42] INFO [2022-01-10 12:53:20,249] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:20,249] jepsen worker 37 - jepsen.util 127 :ok :read [12 12] INFO [2022-01-10 12:53:20,250] jepsen worker 53 - jepsen.util 143 :ok :read [17 34] INFO [2022-01-10 12:53:20,250] jepsen worker 66 - jepsen.util 156 :ok :write [22 25] INFO [2022-01-10 12:53:20,250] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,250] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:20,251] jepsen worker 37 - jepsen.util 127 :ok :read [12 12] INFO [2022-01-10 12:53:20,251] jepsen worker 53 - jepsen.util 143 :ok :read [17 34] INFO [2022-01-10 12:53:20,251] jepsen worker 66 - jepsen.util 156 :invoke :write [22 14] INFO [2022-01-10 12:53:20,252] jepsen worker 53 - jepsen.util 143 :invoke :write [17 37] INFO [2022-01-10 12:53:20,253] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,254] jepsen worker 37 - jepsen.util 127 :ok :read [12 12] INFO [2022-01-10 12:53:20,255] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,255] jepsen worker 37 - jepsen.util 127 :ok :read [12 12] INFO [2022-01-10 12:53:20,256] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,257] jepsen worker 85 - jepsen.util 175 :ok :write [28 48] INFO [2022-01-10 12:53:20,257] jepsen worker 37 - jepsen.util 127 :ok :read [12 12] INFO [2022-01-10 12:53:20,257] jepsen worker 85 - jepsen.util 175 :invoke :write [28 18] INFO [2022-01-10 12:53:20,258] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,259] jepsen worker 37 - jepsen.util 127 :ok :read [12 12] INFO [2022-01-10 12:53:20,260] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,260] jepsen worker 37 - jepsen.util 127 :ok :read [12 12] INFO [2022-01-10 12:53:20,261] jepsen worker 37 - jepsen.util 127 :invoke :write [12 36] INFO [2022-01-10 12:53:20,325] jepsen worker 21 - jepsen.util 111 :fail :read [7 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:20,325] jepsen worker 21 - jepsen.util 111 :invoke :write [7 44] INFO [2022-01-10 12:53:20,367] jepsen worker 17 - jepsen.util 107 :info :write [5 18] :RequestTimeoutException INFO [2022-01-10 12:53:20,367] jepsen worker 33 - jepsen.util 33 :info :write [11 36] :RequestTimeoutException INFO [2022-01-10 12:53:20,367] jepsen worker 17 - jepsen.util 197 :invoke :read [5 nil] INFO [2022-01-10 12:53:20,368] jepsen worker 33 - jepsen.util 123 :invoke :write [11 33] INFO [2022-01-10 12:53:20,411] jepsen worker 13 - jepsen.util 13 :ok :write [4 3] INFO [2022-01-10 12:53:20,411] jepsen worker 62 - jepsen.util 152 :ok :write [20 13] INFO [2022-01-10 12:53:20,412] jepsen worker 13 - jepsen.util 13 :invoke :write [4 18] INFO [2022-01-10 12:53:20,412] jepsen worker 62 - jepsen.util 152 :invoke :write [20 28] INFO [2022-01-10 12:53:20,416] jepsen worker 78 - jepsen.util 78 :ok :write [26 0] INFO [2022-01-10 12:53:20,417] jepsen worker 78 - jepsen.util 78 :invoke :write [26 43] INFO [2022-01-10 12:53:20,419] jepsen worker 23 - jepsen.util 23 :ok :write [7 23] INFO [2022-01-10 12:53:20,419] jepsen worker 23 - jepsen.util 23 :invoke :write [7 31] INFO [2022-01-10 12:53:20,420] jepsen worker 70 - jepsen.util 160 :ok :write [23 42] INFO [2022-01-10 12:53:20,420] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:20,421] jepsen worker 70 - jepsen.util 160 :ok :read [23 42] INFO [2022-01-10 12:53:20,422] jepsen worker 70 - jepsen.util 160 :invoke :write [23 23] INFO [2022-01-10 12:53:20,424] jepsen worker 3 - jepsen.util 3 :ok :write [1 30] INFO [2022-01-10 12:53:20,424] jepsen worker 53 - jepsen.util 143 :ok :write [17 37] INFO [2022-01-10 12:53:20,424] jepsen worker 50 - jepsen.util 50 :ok :write [16 19] INFO [2022-01-10 12:53:20,424] jepsen worker 3 - jepsen.util 3 :invoke :write [1 22] INFO [2022-01-10 12:53:20,425] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:20,426] jepsen worker 50 - jepsen.util 50 :invoke :write [16 10] INFO [2022-01-10 12:53:20,426] jepsen worker 53 - jepsen.util 143 :ok :read [17 37] INFO [2022-01-10 12:53:20,427] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:20,427] jepsen worker 53 - jepsen.util 143 :ok :read [17 37] INFO [2022-01-10 12:53:20,428] jepsen worker 53 - jepsen.util 143 :invoke :write [17 20] INFO [2022-01-10 12:53:20,429] jepsen worker 44 - jepsen.util 44 :ok :write [14 42] INFO [2022-01-10 12:53:20,430] jepsen worker 37 - jepsen.util 127 :ok :write [12 36] INFO [2022-01-10 12:53:20,430] jepsen worker 44 - jepsen.util 44 :invoke :write [14 24] INFO [2022-01-10 12:53:20,430] jepsen worker 83 - jepsen.util 173 :ok :write [27 32] INFO [2022-01-10 12:53:20,431] jepsen worker 37 - jepsen.util 127 :invoke :write [12 47] INFO [2022-01-10 12:53:20,431] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:20,432] jepsen worker 83 - jepsen.util 173 :ok :read [27 32] INFO [2022-01-10 12:53:20,433] jepsen worker 83 - jepsen.util 173 :invoke :write [27 10] INFO [2022-01-10 12:53:20,436] jepsen worker 41 - jepsen.util 41 :ok :write [13 42] INFO [2022-01-10 12:53:20,437] jepsen worker 85 - jepsen.util 175 :ok :write [28 18] INFO [2022-01-10 12:53:20,437] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,437] jepsen worker 85 - jepsen.util 175 :invoke :write [28 12] INFO [2022-01-10 12:53:20,437] jepsen worker 41 - jepsen.util 41 :ok :read [13 42] INFO [2022-01-10 12:53:20,438] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,439] jepsen worker 41 - jepsen.util 41 :ok :read [13 42] INFO [2022-01-10 12:53:20,439] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,439] jepsen worker 77 - jepsen.util 167 :ok :write [25 7] INFO [2022-01-10 12:53:20,440] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:20,440] jepsen worker 41 - jepsen.util 41 :ok :read [13 42] INFO [2022-01-10 12:53:20,441] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,441] jepsen worker 77 - jepsen.util 167 :ok :read [25 7] INFO [2022-01-10 12:53:20,441] jepsen worker 77 - jepsen.util 167 :invoke :write [25 46] INFO [2022-01-10 12:53:20,441] jepsen worker 41 - jepsen.util 41 :ok :read [13 42] INFO [2022-01-10 12:53:20,442] jepsen worker 41 - jepsen.util 41 :invoke :write [13 27] INFO [2022-01-10 12:53:20,445] jepsen worker 87 - jepsen.util 87 :ok :write [29 18] INFO [2022-01-10 12:53:20,445] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:20,446] jepsen worker 87 - jepsen.util 87 :ok :read [29 18] INFO [2022-01-10 12:53:20,447] jepsen worker 87 - jepsen.util 87 :invoke :write [29 2] INFO [2022-01-10 12:53:20,451] jepsen worker 2 - jepsen.util 92 :ok :write [0 43] INFO [2022-01-10 12:53:20,452] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:20,453] jepsen worker 2 - jepsen.util 92 :ok :read [0 43] INFO [2022-01-10 12:53:20,454] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:20,455] jepsen worker 2 - jepsen.util 92 :ok :read [0 43] INFO [2022-01-10 12:53:20,455] jepsen worker 2 - jepsen.util 92 :invoke :write [0 33] INFO [2022-01-10 12:53:20,458] jepsen worker 7 - jepsen.util 7 :ok :write [2 18] INFO [2022-01-10 12:53:20,458] jepsen worker 7 - jepsen.util 7 :invoke :write [2 21] INFO [2022-01-10 12:53:20,461] jepsen worker 35 - jepsen.util 125 :ok :write [11 48] INFO [2022-01-10 12:53:20,462] jepsen worker 35 - jepsen.util 125 :invoke :write [11 32] INFO [2022-01-10 12:53:20,467] jepsen worker 15 - jepsen.util 15 :ok :write [5 14] INFO [2022-01-10 12:53:20,468] jepsen worker 15 - jepsen.util 15 :invoke :write [5 45] INFO [2022-01-10 12:53:20,471] jepsen worker 66 - jepsen.util 156 :ok :write [22 14] INFO [2022-01-10 12:53:20,472] jepsen worker 66 - jepsen.util 156 :invoke :write [22 26] INFO [2022-01-10 12:53:20,475] jepsen worker 50 - jepsen.util 50 :ok :write [16 10] INFO [2022-01-10 12:53:20,476] jepsen worker 50 - jepsen.util 50 :invoke :write [16 15] INFO [2022-01-10 12:53:20,480] jepsen worker 83 - jepsen.util 173 :ok :write [27 10] INFO [2022-01-10 12:53:20,480] jepsen worker 83 - jepsen.util 173 :invoke :write [27 11] INFO [2022-01-10 12:53:20,496] jepsen worker 7 - jepsen.util 7 :ok :write [2 21] INFO [2022-01-10 12:53:20,496] jepsen worker 7 - jepsen.util 7 :invoke :write [2 29] INFO [2022-01-10 12:53:20,498] jepsen worker 35 - jepsen.util 125 :ok :write [11 32] INFO [2022-01-10 12:53:20,498] jepsen worker 35 - jepsen.util 125 :invoke :write [11 24] INFO [2022-01-10 12:53:20,500] jepsen worker 15 - jepsen.util 15 :ok :write [5 45] INFO [2022-01-10 12:53:20,501] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:20,501] jepsen worker 15 - jepsen.util 15 :ok :read [5 45] INFO [2022-01-10 12:53:20,502] jepsen worker 15 - jepsen.util 15 :invoke :write [5 26] INFO [2022-01-10 12:53:20,505] jepsen worker 66 - jepsen.util 156 :ok :write [22 26] INFO [2022-01-10 12:53:20,505] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:20,506] jepsen worker 66 - jepsen.util 156 :ok :read [22 26] INFO [2022-01-10 12:53:20,506] jepsen worker 66 - jepsen.util 156 :invoke :write [22 3] INFO [2022-01-10 12:53:20,509] jepsen worker 50 - jepsen.util 50 :ok :write [16 15] INFO [2022-01-10 12:53:20,509] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:20,510] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:20,511] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:20,511] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:20,512] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:20,513] jepsen worker 83 - jepsen.util 173 :ok :write [27 11] INFO [2022-01-10 12:53:20,513] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:20,513] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:20,514] jepsen worker 83 - jepsen.util 173 :invoke :write [27 9] INFO [2022-01-10 12:53:20,514] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:20,515] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:20,516] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:20,516] jepsen worker 50 - jepsen.util 50 :invoke :write [16 24] INFO [2022-01-10 12:53:20,518] jepsen worker 7 - jepsen.util 7 :ok :write [2 29] INFO [2022-01-10 12:53:20,518] jepsen worker 7 - jepsen.util 7 :invoke :write [2 35] INFO [2022-01-10 12:53:20,520] jepsen worker 35 - jepsen.util 125 :ok :write [11 24] INFO [2022-01-10 12:53:20,521] jepsen worker 35 - jepsen.util 125 :invoke :write [11 34] INFO [2022-01-10 12:53:20,523] jepsen worker 15 - jepsen.util 15 :ok :write [5 26] INFO [2022-01-10 12:53:20,524] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:20,524] jepsen worker 15 - jepsen.util 15 :ok :read [5 26] INFO [2022-01-10 12:53:20,525] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:20,526] jepsen worker 15 - jepsen.util 15 :ok :read [5 26] INFO [2022-01-10 12:53:20,526] jepsen worker 15 - jepsen.util 15 :invoke :write [5 23] INFO [2022-01-10 12:53:20,528] jepsen worker 66 - jepsen.util 156 :ok :write [22 3] INFO [2022-01-10 12:53:20,529] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:20,529] jepsen worker 66 - jepsen.util 156 :ok :read [22 3] INFO [2022-01-10 12:53:20,530] jepsen worker 66 - jepsen.util 156 :invoke :write [22 14] INFO [2022-01-10 12:53:20,533] jepsen worker 50 - jepsen.util 50 :ok :write [16 24] INFO [2022-01-10 12:53:20,534] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:20,535] jepsen worker 50 - jepsen.util 50 :ok :read [16 24] INFO [2022-01-10 12:53:20,535] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:20,536] jepsen worker 50 - jepsen.util 50 :ok :read [16 24] INFO [2022-01-10 12:53:20,536] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:20,537] jepsen worker 50 - jepsen.util 50 :ok :read [16 24] INFO [2022-01-10 12:53:20,538] jepsen worker 50 - jepsen.util 50 :invoke :write [16 20] INFO [2022-01-10 12:53:20,579] jepsen worker 11 - jepsen.util 101 :info :write [3 33] :SyncWriteAmbiguous INFO [2022-01-10 12:53:20,579] jepsen worker 11 - jepsen.util 191 :invoke :read [3 nil] INFO [2022-01-10 12:53:20,583] jepsen worker 17 - jepsen.util 197 :ok :read [5 26] INFO [2022-01-10 12:53:20,585] jepsen worker 17 - jepsen.util 197 :invoke :write [5 46] INFO [2022-01-10 12:53:20,585] jepsen worker 58 - jepsen.util 148 :info :write [19 26] :SyncWriteAmbiguous INFO [2022-01-10 12:53:20,586] jepsen worker 58 - jepsen.util 238 :invoke :write [19 17] INFO [2022-01-10 12:53:20,586] jepsen worker 62 - jepsen.util 152 :ok :write [20 28] INFO [2022-01-10 12:53:20,586] jepsen worker 13 - jepsen.util 13 :ok :write [4 18] INFO [2022-01-10 12:53:20,587] jepsen worker 13 - jepsen.util 13 :invoke :write [4 8] INFO [2022-01-10 12:53:20,587] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:20,588] jepsen worker 62 - jepsen.util 152 :ok :read [20 28] INFO [2022-01-10 12:53:20,588] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:20,589] jepsen worker 62 - jepsen.util 152 :ok :read [20 28] INFO [2022-01-10 12:53:20,589] jepsen worker 62 - jepsen.util 152 :invoke :write [20 9] INFO [2022-01-10 12:53:20,590] jepsen worker 23 - jepsen.util 23 :ok :write [7 31] INFO [2022-01-10 12:53:20,590] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:20,591] jepsen worker 23 - jepsen.util 23 :ok :read [7 31] INFO [2022-01-10 12:53:20,592] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:20,593] jepsen worker 23 - jepsen.util 23 :ok :read [7 31] INFO [2022-01-10 12:53:20,594] jepsen worker 23 - jepsen.util 23 :invoke :write [7 15] INFO [2022-01-10 12:53:20,595] jepsen worker 44 - jepsen.util 44 :ok :write [14 24] INFO [2022-01-10 12:53:20,596] jepsen worker 44 - jepsen.util 44 :invoke :write [14 45] INFO [2022-01-10 12:53:20,597] jepsen worker 78 - jepsen.util 78 :ok :write [26 43] INFO [2022-01-10 12:53:20,598] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:20,598] jepsen worker 78 - jepsen.util 78 :ok :read [26 43] INFO [2022-01-10 12:53:20,599] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:20,599] jepsen worker 41 - jepsen.util 41 :ok :write [13 27] INFO [2022-01-10 12:53:20,600] jepsen worker 78 - jepsen.util 78 :ok :read [26 43] INFO [2022-01-10 12:53:20,600] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,600] jepsen worker 78 - jepsen.util 78 :invoke :write [26 25] INFO [2022-01-10 12:53:20,601] jepsen worker 41 - jepsen.util 41 :ok :read [13 27] INFO [2022-01-10 12:53:20,601] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,602] jepsen worker 41 - jepsen.util 41 :ok :read [13 27] INFO [2022-01-10 12:53:20,603] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,604] jepsen worker 70 - jepsen.util 160 :ok :write [23 23] INFO [2022-01-10 12:53:20,604] jepsen worker 41 - jepsen.util 41 :ok :read [13 27] INFO [2022-01-10 12:53:20,604] jepsen worker 41 - jepsen.util 41 :invoke :write [13 48] INFO [2022-01-10 12:53:20,605] jepsen worker 70 - jepsen.util 160 :invoke :write [23 15] INFO [2022-01-10 12:53:20,605] jepsen worker 77 - jepsen.util 167 :ok :write [25 46] INFO [2022-01-10 12:53:20,606] jepsen worker 77 - jepsen.util 167 :invoke :write [25 38] INFO [2022-01-10 12:53:20,611] jepsen worker 87 - jepsen.util 87 :ok :write [29 2] INFO [2022-01-10 12:53:20,611] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:20,612] jepsen worker 3 - jepsen.util 3 :ok :write [1 22] INFO [2022-01-10 12:53:20,612] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:20,612] jepsen worker 87 - jepsen.util 87 :ok :read [29 2] INFO [2022-01-10 12:53:20,613] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:20,613] jepsen worker 3 - jepsen.util 3 :ok :read [1 22] INFO [2022-01-10 12:53:20,613] jepsen worker 3 - jepsen.util 3 :invoke :write [1 27] INFO [2022-01-10 12:53:20,614] jepsen worker 87 - jepsen.util 87 :ok :read [29 2] INFO [2022-01-10 12:53:20,614] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:20,615] jepsen worker 87 - jepsen.util 87 :ok :read [29 2] INFO [2022-01-10 12:53:20,617] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:20,619] jepsen worker 2 - jepsen.util 92 :ok :write [0 33] INFO [2022-01-10 12:53:20,620] jepsen worker 87 - jepsen.util 87 :ok :read [29 2] INFO [2022-01-10 12:53:20,620] jepsen worker 2 - jepsen.util 92 :invoke :write [0 36] INFO [2022-01-10 12:53:20,620] jepsen worker 53 - jepsen.util 143 :ok :write [17 20] INFO [2022-01-10 12:53:20,620] jepsen worker 63 - jepsen.util 153 :info :write [21 4] :SyncWriteAmbiguous INFO [2022-01-10 12:53:20,621] jepsen worker 19 - jepsen.util 109 :info :write [6 9] :SyncWriteAmbiguous INFO [2022-01-10 12:53:20,621] jepsen worker 87 - jepsen.util 87 :invoke :write [29 20] INFO [2022-01-10 12:53:20,622] jepsen worker 53 - jepsen.util 143 :invoke :write [17 35] INFO [2022-01-10 12:53:20,623] jepsen worker 63 - jepsen.util 243 :invoke :write [21 23] INFO [2022-01-10 12:53:20,623] jepsen worker 19 - jepsen.util 199 :invoke :write [6 11] INFO [2022-01-10 12:53:20,628] jepsen worker 26 - jepsen.util 116 :info :write [8 8] :SyncWriteAmbiguous INFO [2022-01-10 12:53:20,628] jepsen worker 37 - jepsen.util 127 :ok :write [12 47] INFO [2022-01-10 12:53:20,629] jepsen worker 26 - jepsen.util 206 :invoke :write [8 32] INFO [2022-01-10 12:53:20,630] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,631] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:20,632] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,632] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:20,633] jepsen worker 37 - jepsen.util 127 :invoke :write [12 5] INFO [2022-01-10 12:53:20,634] jepsen worker 85 - jepsen.util 175 :ok :write [28 12] INFO [2022-01-10 12:53:20,635] jepsen worker 85 - jepsen.util 175 :invoke :write [28 41] INFO [2022-01-10 12:53:20,636] jepsen worker 41 - jepsen.util 41 :ok :write [13 48] INFO [2022-01-10 12:53:20,638] jepsen worker 45 - jepsen.util 135 :info :write [15 3] :SyncWriteAmbiguous INFO [2022-01-10 12:53:20,638] jepsen worker 41 - jepsen.util 41 :invoke :write [13 38] INFO [2022-01-10 12:53:20,638] jepsen worker 45 - jepsen.util 225 :invoke :read [15 nil] INFO [2022-01-10 12:53:20,644] jepsen worker 55 - jepsen.util 145 :info :write [18 43] :SyncWriteAmbiguous INFO [2022-01-10 12:53:20,644] jepsen worker 55 - jepsen.util 235 :invoke :read [18 nil] INFO [2022-01-10 12:53:20,648] jepsen worker 87 - jepsen.util 87 :ok :write [29 20] INFO [2022-01-10 12:53:20,648] jepsen worker 87 - jepsen.util 87 :invoke :write [29 18] INFO [2022-01-10 12:53:20,653] jepsen worker 83 - jepsen.util 173 :ok :write [27 9] INFO [2022-01-10 12:53:20,653] jepsen worker 83 - jepsen.util 173 :invoke :write [27 26] INFO [2022-01-10 12:53:20,655] jepsen worker 2 - jepsen.util 92 :ok :write [0 36] INFO [2022-01-10 12:53:20,656] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:20,657] jepsen worker 2 - jepsen.util 92 :ok :read [0 36] INFO [2022-01-10 12:53:20,658] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:20,659] jepsen worker 7 - jepsen.util 7 :ok :write [2 35] INFO [2022-01-10 12:53:20,659] jepsen worker 2 - jepsen.util 92 :ok :read [0 36] INFO [2022-01-10 12:53:20,659] jepsen worker 13 - jepsen.util 13 :ok :write [4 8] INFO [2022-01-10 12:53:20,659] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:20,660] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:20,660] jepsen worker 7 - jepsen.util 7 :ok :read [2 35] INFO [2022-01-10 12:53:20,660] jepsen worker 2 - jepsen.util 92 :invoke :write [0 7] INFO [2022-01-10 12:53:20,660] jepsen worker 7 - jepsen.util 7 :invoke :write [2 0] INFO [2022-01-10 12:53:20,661] jepsen worker 13 - jepsen.util 13 :ok :read [4 8] INFO [2022-01-10 12:53:20,662] jepsen worker 13 - jepsen.util 13 :invoke :write [4 0] INFO [2022-01-10 12:53:20,663] jepsen worker 41 - jepsen.util 41 :ok :write [13 38] INFO [2022-01-10 12:53:20,664] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,665] jepsen worker 41 - jepsen.util 41 :ok :read [13 38] INFO [2022-01-10 12:53:20,665] jepsen worker 35 - jepsen.util 125 :ok :write [11 34] INFO [2022-01-10 12:53:20,666] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,666] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:20,666] jepsen worker 41 - jepsen.util 41 :ok :read [13 38] INFO [2022-01-10 12:53:20,667] jepsen worker 78 - jepsen.util 78 :ok :write [26 25] INFO [2022-01-10 12:53:20,667] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,668] jepsen worker 35 - jepsen.util 125 :ok :read [11 34] INFO [2022-01-10 12:53:20,668] jepsen worker 78 - jepsen.util 78 :invoke :write [26 21] INFO [2022-01-10 12:53:20,668] jepsen worker 41 - jepsen.util 41 :ok :read [13 38] INFO [2022-01-10 12:53:20,669] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:20,669] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,669] jepsen worker 35 - jepsen.util 125 :ok :read [11 34] INFO [2022-01-10 12:53:20,670] jepsen worker 41 - jepsen.util 41 :ok :read [13 38] INFO [2022-01-10 12:53:20,670] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:20,671] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,671] jepsen worker 35 - jepsen.util 125 :ok :read [11 34] INFO [2022-01-10 12:53:20,671] jepsen worker 70 - jepsen.util 160 :ok :write [23 15] INFO [2022-01-10 12:53:20,671] jepsen worker 70 - jepsen.util 160 :invoke :write [23 38] INFO [2022-01-10 12:53:20,672] jepsen worker 35 - jepsen.util 125 :invoke :read [11 nil] INFO [2022-01-10 12:53:20,672] jepsen worker 41 - jepsen.util 41 :ok :read [13 38] INFO [2022-01-10 12:53:20,673] jepsen worker 35 - jepsen.util 125 :ok :read [11 34] INFO [2022-01-10 12:53:20,673] jepsen worker 15 - jepsen.util 15 :ok :write [5 23] INFO [2022-01-10 12:53:20,673] jepsen worker 41 - jepsen.util 41 :invoke :write [13 26] INFO [2022-01-10 12:53:20,674] jepsen worker 3 - jepsen.util 3 :ok :write [1 27] INFO [2022-01-10 12:53:20,674] jepsen worker 35 - jepsen.util 125 :invoke :write [11 22] INFO [2022-01-10 12:53:20,675] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:20,675] jepsen worker 15 - jepsen.util 15 :invoke :write [5 10] INFO [2022-01-10 12:53:20,676] jepsen worker 3 - jepsen.util 3 :ok :read [1 27] INFO [2022-01-10 12:53:20,677] jepsen worker 3 - jepsen.util 3 :invoke :write [1 10] INFO [2022-01-10 12:53:20,679] jepsen worker 87 - jepsen.util 87 :ok :write [29 18] INFO [2022-01-10 12:53:20,679] jepsen worker 87 - jepsen.util 87 :invoke :write [29 4] INFO [2022-01-10 12:53:20,685] jepsen worker 2 - jepsen.util 92 :ok :write [0 7] INFO [2022-01-10 12:53:20,686] jepsen worker 37 - jepsen.util 127 :ok :write [12 5] INFO [2022-01-10 12:53:20,686] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:20,687] jepsen worker 37 - jepsen.util 127 :invoke :write [12 19] INFO [2022-01-10 12:53:20,687] jepsen worker 2 - jepsen.util 92 :ok :read [0 7] INFO [2022-01-10 12:53:20,688] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:20,689] jepsen worker 2 - jepsen.util 92 :ok :read [0 7] INFO [2022-01-10 12:53:20,690] jepsen worker 2 - jepsen.util 92 :invoke :write [0 31] INFO [2022-01-10 12:53:20,692] jepsen worker 41 - jepsen.util 41 :ok :write [13 26] INFO [2022-01-10 12:53:20,692] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,693] jepsen worker 41 - jepsen.util 41 :ok :read [13 26] INFO [2022-01-10 12:53:20,694] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,694] jepsen worker 85 - jepsen.util 175 :ok :write [28 41] INFO [2022-01-10 12:53:20,695] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:20,695] jepsen worker 41 - jepsen.util 41 :ok :read [13 26] INFO [2022-01-10 12:53:20,695] jepsen worker 41 - jepsen.util 41 :invoke :write [13 0] INFO [2022-01-10 12:53:20,696] jepsen worker 85 - jepsen.util 175 :ok :read [28 41] INFO [2022-01-10 12:53:20,696] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:20,697] jepsen worker 85 - jepsen.util 175 :ok :read [28 41] INFO [2022-01-10 12:53:20,697] jepsen worker 87 - jepsen.util 87 :ok :write [29 4] INFO [2022-01-10 12:53:20,697] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:20,698] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:20,698] jepsen worker 85 - jepsen.util 175 :ok :read [28 41] INFO [2022-01-10 12:53:20,699] jepsen worker 85 - jepsen.util 175 :invoke :write [28 2] INFO [2022-01-10 12:53:20,699] jepsen worker 87 - jepsen.util 87 :ok :read [29 4] INFO [2022-01-10 12:53:20,700] jepsen worker 87 - jepsen.util 87 :invoke :write [29 41] INFO [2022-01-10 12:53:20,704] jepsen worker 2 - jepsen.util 92 :ok :write [0 31] INFO [2022-01-10 12:53:20,704] jepsen worker 2 - jepsen.util 92 :invoke :write [0 22] INFO [2022-01-10 12:53:20,710] jepsen worker 41 - jepsen.util 41 :ok :write [13 0] INFO [2022-01-10 12:53:20,710] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,711] jepsen worker 41 - jepsen.util 41 :ok :read [13 0] INFO [2022-01-10 12:53:20,711] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,712] jepsen worker 41 - jepsen.util 41 :ok :read [13 0] INFO [2022-01-10 12:53:20,713] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,714] jepsen worker 87 - jepsen.util 87 :ok :write [29 41] INFO [2022-01-10 12:53:20,714] jepsen worker 41 - jepsen.util 41 :ok :read [13 0] INFO [2022-01-10 12:53:20,714] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:20,715] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,715] jepsen worker 87 - jepsen.util 87 :ok :read [29 41] INFO [2022-01-10 12:53:20,715] jepsen worker 87 - jepsen.util 87 :invoke :write [29 34] INFO [2022-01-10 12:53:20,716] jepsen worker 3 - jepsen.util 3 :ok :write [1 10] INFO [2022-01-10 12:53:20,716] jepsen worker 41 - jepsen.util 41 :ok :read [13 0] INFO [2022-01-10 12:53:20,716] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:20,717] jepsen worker 41 - jepsen.util 41 :invoke :write [13 38] INFO [2022-01-10 12:53:20,717] jepsen worker 3 - jepsen.util 3 :ok :read [1 10] INFO [2022-01-10 12:53:20,718] jepsen worker 3 - jepsen.util 3 :invoke :write [1 39] INFO [2022-01-10 12:53:20,720] jepsen worker 2 - jepsen.util 92 :ok :write [0 22] INFO [2022-01-10 12:53:20,721] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:20,721] jepsen worker 2 - jepsen.util 92 :ok :read [0 22] INFO [2022-01-10 12:53:20,722] jepsen worker 2 - jepsen.util 92 :invoke :write [0 45] INFO [2022-01-10 12:53:20,724] jepsen worker 37 - jepsen.util 127 :ok :write [12 19] INFO [2022-01-10 12:53:20,724] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,725] jepsen worker 37 - jepsen.util 127 :ok :read [12 19] INFO [2022-01-10 12:53:20,726] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,726] jepsen worker 37 - jepsen.util 127 :ok :read [12 19] INFO [2022-01-10 12:53:20,727] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,728] jepsen worker 37 - jepsen.util 127 :ok :read [12 19] INFO [2022-01-10 12:53:20,728] jepsen worker 37 - jepsen.util 127 :invoke :write [12 48] INFO [2022-01-10 12:53:20,730] jepsen worker 85 - jepsen.util 175 :ok :write [28 2] INFO [2022-01-10 12:53:20,731] jepsen worker 85 - jepsen.util 175 :invoke :write [28 29] INFO [2022-01-10 12:53:20,761] jepsen worker 11 - jepsen.util 191 :ok :read [3 30] INFO [2022-01-10 12:53:20,763] jepsen worker 30 - jepsen.util 120 :info :write [10 22] :SyncWriteAmbiguous INFO [2022-01-10 12:53:20,763] jepsen worker 11 - jepsen.util 191 :invoke :write [3 13] INFO [2022-01-10 12:53:20,764] jepsen worker 30 - jepsen.util 210 :invoke :write [10 35] INFO [2022-01-10 12:53:20,806] jepsen worker 73 - jepsen.util 163 :info :write [24 9] :RequestTimeoutException INFO [2022-01-10 12:53:20,806] jepsen worker 27 - jepsen.util 117 :info :write [9 22] :SyncWriteAmbiguous INFO [2022-01-10 12:53:20,807] jepsen worker 73 - jepsen.util 253 :invoke :read [24 nil] INFO [2022-01-10 12:53:20,807] jepsen worker 27 - jepsen.util 207 :invoke :write [9 7] INFO [2022-01-10 12:53:20,808] jepsen worker 73 - jepsen.util 253 :ok :read [24 37] INFO [2022-01-10 12:53:20,809] jepsen worker 73 - jepsen.util 253 :invoke :read [24 nil] INFO [2022-01-10 12:53:20,810] jepsen worker 73 - jepsen.util 253 :ok :read [24 37] INFO [2022-01-10 12:53:20,810] jepsen worker 73 - jepsen.util 253 :invoke :write [24 23] INFO [2022-01-10 12:53:20,815] jepsen worker 66 - jepsen.util 156 :ok :write [22 14] INFO [2022-01-10 12:53:20,815] jepsen worker 66 - jepsen.util 156 :invoke :write [22 19] INFO [2022-01-10 12:53:20,819] jepsen worker 50 - jepsen.util 50 :ok :write [16 20] INFO [2022-01-10 12:53:20,820] jepsen worker 50 - jepsen.util 50 :invoke :write [16 9] INFO [2022-01-10 12:53:20,826] jepsen worker 62 - jepsen.util 152 :ok :write [20 9] INFO [2022-01-10 12:53:20,826] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:20,827] jepsen worker 53 - jepsen.util 143 :ok :write [17 35] INFO [2022-01-10 12:53:20,827] jepsen worker 62 - jepsen.util 152 :ok :read [20 9] INFO [2022-01-10 12:53:20,828] jepsen worker 62 - jepsen.util 152 :invoke :write [20 40] INFO [2022-01-10 12:53:20,829] jepsen worker 53 - jepsen.util 143 :invoke :write [17 43] INFO [2022-01-10 12:53:20,832] jepsen worker 87 - jepsen.util 87 :ok :write [29 34] INFO [2022-01-10 12:53:20,833] jepsen worker 87 - jepsen.util 87 :invoke :write [29 46] INFO [2022-01-10 12:53:20,835] jepsen worker 3 - jepsen.util 3 :ok :write [1 39] INFO [2022-01-10 12:53:20,835] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:20,836] jepsen worker 3 - jepsen.util 3 :ok :read [1 39] INFO [2022-01-10 12:53:20,836] jepsen worker 41 - jepsen.util 41 :ok :write [13 38] INFO [2022-01-10 12:53:20,837] jepsen worker 3 - jepsen.util 3 :invoke :write [1 26] INFO [2022-01-10 12:53:20,838] jepsen worker 41 - jepsen.util 41 :invoke :write [13 5] INFO [2022-01-10 12:53:20,839] jepsen worker 37 - jepsen.util 127 :ok :write [12 48] INFO [2022-01-10 12:53:20,840] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,841] jepsen worker 37 - jepsen.util 127 :ok :read [12 48] INFO [2022-01-10 12:53:20,841] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,842] jepsen worker 37 - jepsen.util 127 :ok :read [12 48] INFO [2022-01-10 12:53:20,843] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,843] jepsen worker 2 - jepsen.util 92 :ok :write [0 45] INFO [2022-01-10 12:53:20,844] jepsen worker 37 - jepsen.util 127 :ok :read [12 48] INFO [2022-01-10 12:53:20,844] jepsen worker 2 - jepsen.util 92 :invoke :write [0 4] INFO [2022-01-10 12:53:20,845] jepsen worker 32 - jepsen.util 122 :info :write [10 44] :RequestTimeoutException INFO [2022-01-10 12:53:20,845] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,846] jepsen worker 37 - jepsen.util 127 :ok :read [12 48] INFO [2022-01-10 12:53:20,846] jepsen worker 85 - jepsen.util 175 :ok :write [28 29] INFO [2022-01-10 12:53:20,846] jepsen worker 32 - jepsen.util 212 :invoke :read [10 nil] INFO [2022-01-10 12:53:20,847] jepsen worker 85 - jepsen.util 175 :invoke :write [28 44] INFO [2022-01-10 12:53:20,847] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,848] jepsen worker 37 - jepsen.util 127 :ok :read [12 48] INFO [2022-01-10 12:53:20,849] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:20,850] jepsen worker 37 - jepsen.util 127 :ok :read [12 48] INFO [2022-01-10 12:53:20,851] jepsen worker 37 - jepsen.util 127 :invoke :write [12 4] INFO [2022-01-10 12:53:20,854] jepsen worker 3 - jepsen.util 3 :ok :write [1 26] INFO [2022-01-10 12:53:20,855] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:20,856] jepsen worker 3 - jepsen.util 3 :ok :read [1 26] INFO [2022-01-10 12:53:20,856] jepsen worker 3 - jepsen.util 3 :invoke :write [1 9] INFO [2022-01-10 12:53:20,864] jepsen worker 37 - jepsen.util 127 :ok :write [12 4] INFO [2022-01-10 12:53:20,865] jepsen worker 37 - jepsen.util 127 :invoke :write [12 45] INFO [2022-01-10 12:53:20,866] jepsen worker 85 - jepsen.util 175 :ok :write [28 44] INFO [2022-01-10 12:53:20,866] jepsen worker 85 - jepsen.util 175 :invoke :write [28 7] INFO [2022-01-10 12:53:20,869] jepsen worker 87 - jepsen.util 87 :ok :write [29 46] INFO [2022-01-10 12:53:20,870] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:20,871] jepsen worker 87 - jepsen.util 87 :ok :read [29 46] INFO [2022-01-10 12:53:20,871] jepsen worker 87 - jepsen.util 87 :invoke :write [29 27] INFO [2022-01-10 12:53:20,872] jepsen worker 3 - jepsen.util 3 :ok :write [1 9] INFO [2022-01-10 12:53:20,872] jepsen worker 3 - jepsen.util 3 :invoke :write [1 18] INFO [2022-01-10 12:53:20,873] jepsen worker 41 - jepsen.util 41 :ok :write [13 5] INFO [2022-01-10 12:53:20,874] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,875] jepsen worker 41 - jepsen.util 41 :ok :read [13 5] INFO [2022-01-10 12:53:20,875] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,876] jepsen worker 41 - jepsen.util 41 :ok :read [13 5] INFO [2022-01-10 12:53:20,877] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,878] jepsen worker 41 - jepsen.util 41 :ok :read [13 5] INFO [2022-01-10 12:53:20,878] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,879] jepsen worker 41 - jepsen.util 41 :ok :read [13 5] INFO [2022-01-10 12:53:20,879] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,880] jepsen worker 41 - jepsen.util 41 :ok :read [13 5] INFO [2022-01-10 12:53:20,880] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,881] jepsen worker 41 - jepsen.util 41 :ok :read [13 5] INFO [2022-01-10 12:53:20,881] jepsen worker 41 - jepsen.util 41 :invoke :write [13 43] INFO [2022-01-10 12:53:20,883] jepsen worker 2 - jepsen.util 92 :ok :write [0 4] INFO [2022-01-10 12:53:20,883] jepsen worker 85 - jepsen.util 175 :ok :write [28 7] INFO [2022-01-10 12:53:20,883] jepsen worker 2 - jepsen.util 92 :invoke :write [0 45] INFO [2022-01-10 12:53:20,884] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:20,884] jepsen worker 85 - jepsen.util 175 :ok :read [28 7] INFO [2022-01-10 12:53:20,885] jepsen worker 85 - jepsen.util 175 :invoke :write [28 3] INFO [2022-01-10 12:53:20,887] jepsen worker 38 - jepsen.util 38 :ok :write [12 3] INFO [2022-01-10 12:53:20,888] jepsen worker 31 - jepsen.util 121 :info :write [10 19] :RequestTimeoutException INFO [2022-01-10 12:53:20,888] jepsen worker 38 - jepsen.util 38 :invoke :write [12 37] INFO [2022-01-10 12:53:20,889] jepsen worker 31 - jepsen.util 211 :invoke :read [10 nil] INFO [2022-01-10 12:53:20,891] jepsen worker 87 - jepsen.util 87 :ok :write [29 27] INFO [2022-01-10 12:53:20,892] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:20,893] jepsen worker 87 - jepsen.util 87 :ok :read [29 27] INFO [2022-01-10 12:53:20,893] jepsen worker 3 - jepsen.util 3 :ok :write [1 18] INFO [2022-01-10 12:53:20,893] jepsen worker 87 - jepsen.util 87 :invoke :write [29 30] INFO [2022-01-10 12:53:20,894] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:20,895] jepsen worker 3 - jepsen.util 3 :ok :read [1 18] INFO [2022-01-10 12:53:20,895] jepsen worker 3 - jepsen.util 3 :invoke :write [1 49] INFO [2022-01-10 12:53:20,896] jepsen worker 39 - jepsen.util 39 :ok :write [13 11] INFO [2022-01-10 12:53:20,897] jepsen worker 39 - jepsen.util 39 :invoke :read [13 nil] INFO [2022-01-10 12:53:20,897] jepsen worker 39 - jepsen.util 39 :ok :read [13 11] INFO [2022-01-10 12:53:20,898] jepsen worker 39 - jepsen.util 39 :invoke :write [13 44] INFO [2022-01-10 12:53:20,900] jepsen worker 82 - jepsen.util 172 :info :write [27 31] :RequestTimeoutException INFO [2022-01-10 12:53:20,900] jepsen worker 82 - jepsen.util 262 :invoke :write [27 16] INFO [2022-01-10 12:53:20,905] jepsen worker 59 - jepsen.util 149 :info :write [19 4] :RequestTimeoutException INFO [2022-01-10 12:53:20,905] jepsen worker 59 - jepsen.util 239 :invoke :write [19 9] INFO [2022-01-10 12:53:20,907] jepsen worker 80 - jepsen.util 170 :info :write [26 13] :RequestTimeoutException INFO [2022-01-10 12:53:20,908] jepsen worker 80 - jepsen.util 260 :invoke :write [26 7] INFO [2022-01-10 12:53:20,911] jepsen worker 12 - jepsen.util 102 :info :write [4 29] :RequestTimeoutException INFO [2022-01-10 12:53:20,912] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:20,915] jepsen worker 12 - jepsen.util 192 :ok :read [4 8] INFO [2022-01-10 12:53:20,916] jepsen worker 12 - jepsen.util 192 :invoke :write [4 28] INFO [2022-01-10 12:53:20,968] jepsen worker 54 - jepsen.util 144 :info :write [18 27] :RequestTimeoutException INFO [2022-01-10 12:53:20,969] jepsen worker 54 - jepsen.util 234 :invoke :read [18 nil] INFO [2022-01-10 12:53:20,970] jepsen worker 54 - jepsen.util 234 :ok :read [18 46] INFO [2022-01-10 12:53:20,971] jepsen worker 54 - jepsen.util 234 :invoke :write [18 1] INFO [2022-01-10 12:53:20,974] jepsen worker 23 - jepsen.util 23 :ok :write [7 15] INFO [2022-01-10 12:53:20,974] jepsen worker 76 - jepsen.util 76 :info :write [25 38] :RequestTimeoutException INFO [2022-01-10 12:53:20,975] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:20,976] jepsen worker 76 - jepsen.util 166 :invoke :write [25 49] INFO [2022-01-10 12:53:20,976] jepsen worker 23 - jepsen.util 23 :ok :read [7 15] INFO [2022-01-10 12:53:20,977] jepsen worker 23 - jepsen.util 23 :invoke :write [7 45] INFO [2022-01-10 12:53:20,981] jepsen worker 44 - jepsen.util 44 :ok :write [14 45] INFO [2022-01-10 12:53:20,982] jepsen worker 44 - jepsen.util 44 :invoke :write [14 11] INFO [2022-01-10 12:53:20,985] jepsen worker 77 - jepsen.util 167 :ok :write [25 38] INFO [2022-01-10 12:53:20,985] jepsen worker 13 - jepsen.util 13 :ok :write [4 0] INFO [2022-01-10 12:53:20,985] jepsen worker 77 - jepsen.util 167 :invoke :write [25 3] INFO [2022-01-10 12:53:20,985] jepsen worker 13 - jepsen.util 13 :invoke :write [4 35] INFO [2022-01-10 12:53:20,987] jepsen worker 28 - jepsen.util 118 :info :write [9 42] :RequestTimeoutException INFO [2022-01-10 12:53:20,988] jepsen worker 28 - jepsen.util 208 :invoke :read [9 nil] INFO [2022-01-10 12:53:20,989] jepsen worker 28 - jepsen.util 208 :ok :read [9 1] INFO [2022-01-10 12:53:20,989] jepsen worker 28 - jepsen.util 208 :invoke :write [9 38] INFO [2022-01-10 12:53:20,990] jepsen worker 78 - jepsen.util 78 :ok :write [26 21] INFO [2022-01-10 12:53:20,991] jepsen worker 78 - jepsen.util 78 :invoke :write [26 44] INFO [2022-01-10 12:53:20,991] jepsen worker 83 - jepsen.util 173 :ok :write [27 26] INFO [2022-01-10 12:53:20,991] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:20,992] jepsen worker 83 - jepsen.util 173 :ok :read [27 26] INFO [2022-01-10 12:53:20,992] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:20,993] jepsen worker 83 - jepsen.util 173 :ok :read [27 26] INFO [2022-01-10 12:53:20,994] jepsen worker 46 - jepsen.util 136 :info :write [15 44] :RequestTimeoutException INFO [2022-01-10 12:53:20,994] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:20,995] jepsen worker 46 - jepsen.util 226 :invoke :write [15 2] INFO [2022-01-10 12:53:20,995] jepsen worker 83 - jepsen.util 173 :ok :read [27 26] INFO [2022-01-10 12:53:20,995] jepsen worker 7 - jepsen.util 7 :ok :write [2 0] INFO [2022-01-10 12:53:20,996] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:20,996] jepsen worker 7 - jepsen.util 7 :invoke :write [2 27] INFO [2022-01-10 12:53:20,997] jepsen worker 83 - jepsen.util 173 :ok :read [27 26] INFO [2022-01-10 12:53:20,997] jepsen worker 83 - jepsen.util 173 :invoke :write [27 6] INFO [2022-01-10 12:53:20,999] jepsen worker 70 - jepsen.util 160 :ok :write [23 38] INFO [2022-01-10 12:53:20,999] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,000] jepsen worker 33 - jepsen.util 123 :ok :write [11 33] INFO [2022-01-10 12:53:21,000] jepsen worker 70 - jepsen.util 160 :ok :read [23 38] INFO [2022-01-10 12:53:21,001] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,001] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,001] jepsen worker 33 - jepsen.util 123 :ok :read [11 33] INFO [2022-01-10 12:53:21,002] jepsen worker 70 - jepsen.util 160 :ok :read [23 38] INFO [2022-01-10 12:53:21,002] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,002] jepsen worker 53 - jepsen.util 143 :ok :write [17 43] INFO [2022-01-10 12:53:21,002] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,003] jepsen worker 53 - jepsen.util 143 :invoke :write [17 36] INFO [2022-01-10 12:53:21,003] jepsen worker 33 - jepsen.util 123 :ok :read [11 33] INFO [2022-01-10 12:53:21,003] jepsen worker 70 - jepsen.util 160 :ok :read [23 38] INFO [2022-01-10 12:53:21,004] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,004] jepsen worker 70 - jepsen.util 160 :invoke :write [23 6] INFO [2022-01-10 12:53:21,005] jepsen worker 33 - jepsen.util 123 :ok :read [11 33] INFO [2022-01-10 12:53:21,005] jepsen worker 33 - jepsen.util 123 :invoke :write [11 27] INFO [2022-01-10 12:53:21,006] jepsen worker 15 - jepsen.util 15 :ok :write [5 10] INFO [2022-01-10 12:53:21,007] jepsen worker 15 - jepsen.util 15 :invoke :write [5 5] INFO [2022-01-10 12:53:21,007] jepsen worker 72 - jepsen.util 162 :info :write [24 47] :RequestTimeoutException INFO [2022-01-10 12:53:21,008] jepsen worker 72 - jepsen.util 252 :invoke :read [24 nil] INFO [2022-01-10 12:53:21,009] jepsen worker 72 - jepsen.util 252 :ok :read [24 37] INFO [2022-01-10 12:53:21,009] jepsen worker 72 - jepsen.util 252 :invoke :write [24 6] INFO [2022-01-10 12:53:21,011] jepsen worker 66 - jepsen.util 156 :ok :write [22 19] INFO [2022-01-10 12:53:21,012] jepsen worker 66 - jepsen.util 156 :invoke :write [22 46] INFO [2022-01-10 12:53:21,016] jepsen worker 50 - jepsen.util 50 :ok :write [16 9] INFO [2022-01-10 12:53:21,017] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:21,018] jepsen worker 50 - jepsen.util 50 :ok :read [16 9] INFO [2022-01-10 12:53:21,018] jepsen worker 50 - jepsen.util 50 :invoke :write [16 23] INFO [2022-01-10 12:53:21,019] jepsen worker 62 - jepsen.util 152 :ok :write [20 40] INFO [2022-01-10 12:53:21,019] jepsen worker 62 - jepsen.util 152 :invoke :write [20 37] INFO [2022-01-10 12:53:21,031] jepsen worker 83 - jepsen.util 173 :ok :write [27 6] INFO [2022-01-10 12:53:21,032] jepsen worker 83 - jepsen.util 173 :invoke :write [27 20] INFO [2022-01-10 12:53:21,034] jepsen worker 7 - jepsen.util 7 :ok :write [2 27] INFO [2022-01-10 12:53:21,035] jepsen worker 7 - jepsen.util 7 :invoke :write [2 4] INFO [2022-01-10 12:53:21,037] jepsen worker 33 - jepsen.util 123 :ok :write [11 27] INFO [2022-01-10 12:53:21,038] jepsen worker 33 - jepsen.util 123 :invoke :write [11 29] INFO [2022-01-10 12:53:21,042] jepsen worker 25 - jepsen.util 115 :info :write [8 7] :RequestTimeoutException INFO [2022-01-10 12:53:21,043] jepsen worker 2 - jepsen.util 92 :ok :write [0 45] INFO [2022-01-10 12:53:21,043] jepsen worker 15 - jepsen.util 15 :ok :write [5 5] INFO [2022-01-10 12:53:21,043] jepsen worker 25 - jepsen.util 205 :invoke :read [8 nil] INFO [2022-01-10 12:53:21,044] jepsen worker 2 - jepsen.util 92 :invoke :write [0 27] INFO [2022-01-10 12:53:21,045] jepsen worker 15 - jepsen.util 15 :invoke :write [5 29] INFO [2022-01-10 12:53:21,045] jepsen worker 25 - jepsen.util 205 :ok :read [8 25] INFO [2022-01-10 12:53:21,045] jepsen worker 85 - jepsen.util 175 :ok :write [28 3] INFO [2022-01-10 12:53:21,046] jepsen worker 85 - jepsen.util 175 :invoke :write [28 25] INFO [2022-01-10 12:53:21,047] jepsen worker 66 - jepsen.util 156 :ok :write [22 46] INFO [2022-01-10 12:53:21,047] jepsen worker 25 - jepsen.util 205 :invoke :read [8 nil] INFO [2022-01-10 12:53:21,047] jepsen worker 87 - jepsen.util 87 :ok :write [29 30] INFO [2022-01-10 12:53:21,048] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,048] jepsen worker 25 - jepsen.util 205 :ok :read [8 25] INFO [2022-01-10 12:53:21,048] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:21,049] jepsen worker 66 - jepsen.util 156 :ok :read [22 46] INFO [2022-01-10 12:53:21,049] jepsen worker 25 - jepsen.util 205 :invoke :write [8 42] INFO [2022-01-10 12:53:21,049] jepsen worker 87 - jepsen.util 87 :ok :read [29 30] INFO [2022-01-10 12:53:21,049] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,050] jepsen worker 87 - jepsen.util 87 :invoke :write [29 17] INFO [2022-01-10 12:53:21,050] jepsen worker 38 - jepsen.util 38 :ok :write [12 37] INFO [2022-01-10 12:53:21,050] jepsen worker 66 - jepsen.util 156 :ok :read [22 46] INFO [2022-01-10 12:53:21,051] jepsen worker 38 - jepsen.util 38 :invoke :write [12 10] INFO [2022-01-10 12:53:21,051] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,052] jepsen worker 50 - jepsen.util 50 :ok :write [16 23] INFO [2022-01-10 12:53:21,052] jepsen worker 50 - jepsen.util 50 :invoke :write [16 22] INFO [2022-01-10 12:53:21,053] jepsen worker 66 - jepsen.util 156 :ok :read [22 46] INFO [2022-01-10 12:53:21,054] jepsen worker 66 - jepsen.util 156 :invoke :write [22 39] INFO [2022-01-10 12:53:21,055] jepsen worker 41 - jepsen.util 41 :ok :write [13 43] INFO [2022-01-10 12:53:21,056] jepsen worker 41 - jepsen.util 41 :invoke :write [13 20] INFO [2022-01-10 12:53:21,058] jepsen worker 24 - jepsen.util 114 :info :write [8 21] :RequestTimeoutException INFO [2022-01-10 12:53:21,059] jepsen worker 24 - jepsen.util 204 :invoke :read [8 nil] INFO [2022-01-10 12:53:21,059] jepsen worker 3 - jepsen.util 3 :ok :write [1 49] INFO [2022-01-10 12:53:21,060] jepsen worker 83 - jepsen.util 173 :ok :write [27 20] INFO [2022-01-10 12:53:21,060] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,060] jepsen worker 24 - jepsen.util 204 :ok :read [8 25] INFO [2022-01-10 12:53:21,061] jepsen worker 83 - jepsen.util 173 :invoke :write [27 46] INFO [2022-01-10 12:53:21,061] jepsen worker 23 - jepsen.util 23 :ok :write [7 45] INFO [2022-01-10 12:53:21,061] jepsen worker 3 - jepsen.util 3 :ok :read [1 49] INFO [2022-01-10 12:53:21,061] jepsen worker 24 - jepsen.util 204 :invoke :read [8 nil] INFO [2022-01-10 12:53:21,062] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,062] jepsen worker 24 - jepsen.util 204 :ok :read [8 25] INFO [2022-01-10 12:53:21,062] jepsen worker 23 - jepsen.util 23 :invoke :write [7 41] INFO [2022-01-10 12:53:21,063] jepsen worker 3 - jepsen.util 3 :ok :read [1 49] INFO [2022-01-10 12:53:21,063] jepsen worker 24 - jepsen.util 204 :invoke :write [8 26] INFO [2022-01-10 12:53:21,063] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,064] jepsen worker 7 - jepsen.util 7 :ok :write [2 4] INFO [2022-01-10 12:53:21,064] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:21,065] jepsen worker 3 - jepsen.util 3 :ok :read [1 49] INFO [2022-01-10 12:53:21,065] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:21,065] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,066] jepsen worker 7 - jepsen.util 7 :invoke :write [2 44] INFO [2022-01-10 12:53:21,066] jepsen worker 3 - jepsen.util 3 :ok :read [1 49] INFO [2022-01-10 12:53:21,067] jepsen worker 13 - jepsen.util 13 :ok :write [4 35] INFO [2022-01-10 12:53:21,067] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,067] jepsen worker 33 - jepsen.util 123 :ok :write [11 29] INFO [2022-01-10 12:53:21,068] jepsen worker 13 - jepsen.util 13 :invoke :write [4 24] INFO [2022-01-10 12:53:21,068] jepsen worker 33 - jepsen.util 123 :invoke :write [11 9] INFO [2022-01-10 12:53:21,068] jepsen worker 3 - jepsen.util 3 :ok :read [1 49] INFO [2022-01-10 12:53:21,068] jepsen worker 44 - jepsen.util 44 :ok :write [14 11] INFO [2022-01-10 12:53:21,069] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,069] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:21,070] jepsen worker 3 - jepsen.util 3 :ok :read [1 49] INFO [2022-01-10 12:53:21,070] jepsen worker 44 - jepsen.util 44 :ok :read [14 11] INFO [2022-01-10 12:53:21,071] jepsen worker 3 - jepsen.util 3 :invoke :write [1 8] INFO [2022-01-10 12:53:21,071] jepsen worker 44 - jepsen.util 44 :invoke :write [14 25] INFO [2022-01-10 12:53:21,075] jepsen worker 78 - jepsen.util 78 :ok :write [26 44] INFO [2022-01-10 12:53:21,076] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,077] jepsen worker 78 - jepsen.util 78 :ok :read [26 44] INFO [2022-01-10 12:53:21,077] jepsen worker 77 - jepsen.util 167 :ok :write [25 3] INFO [2022-01-10 12:53:21,078] jepsen worker 78 - jepsen.util 78 :invoke :write [26 17] INFO [2022-01-10 12:53:21,078] jepsen worker 77 - jepsen.util 167 :invoke :write [25 48] INFO [2022-01-10 12:53:21,083] jepsen worker 70 - jepsen.util 160 :ok :write [23 6] INFO [2022-01-10 12:53:21,083] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,085] jepsen worker 70 - jepsen.util 160 :ok :read [23 6] INFO [2022-01-10 12:53:21,085] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,086] jepsen worker 62 - jepsen.util 152 :ok :write [20 37] INFO [2022-01-10 12:53:21,087] jepsen worker 70 - jepsen.util 160 :ok :read [23 6] INFO [2022-01-10 12:53:21,087] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:21,087] jepsen worker 70 - jepsen.util 160 :invoke :write [23 24] INFO [2022-01-10 12:53:21,087] jepsen worker 62 - jepsen.util 152 :ok :read [20 37] INFO [2022-01-10 12:53:21,088] jepsen worker 62 - jepsen.util 152 :invoke :write [20 19] INFO [2022-01-10 12:53:21,091] jepsen worker 51 - jepsen.util 141 :ok :write [17 32] INFO [2022-01-10 12:53:21,092] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,093] jepsen worker 51 - jepsen.util 141 :ok :read [17 32] INFO [2022-01-10 12:53:21,093] jepsen worker 51 - jepsen.util 141 :invoke :write [17 18] INFO [2022-01-10 12:53:21,094] jepsen worker 2 - jepsen.util 92 :ok :write [0 27] INFO [2022-01-10 12:53:21,094] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:21,095] jepsen worker 2 - jepsen.util 92 :ok :read [0 27] INFO [2022-01-10 12:53:21,096] jepsen worker 2 - jepsen.util 92 :invoke :write [0 11] INFO [2022-01-10 12:53:21,099] jepsen worker 85 - jepsen.util 175 :ok :write [28 25] INFO [2022-01-10 12:53:21,099] jepsen worker 87 - jepsen.util 87 :ok :write [29 17] INFO [2022-01-10 12:53:21,099] jepsen worker 87 - jepsen.util 87 :invoke :write [29 38] INFO [2022-01-10 12:53:21,100] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,101] jepsen worker 85 - jepsen.util 175 :ok :read [28 25] INFO [2022-01-10 12:53:21,102] jepsen worker 85 - jepsen.util 175 :invoke :write [28 7] INFO [2022-01-10 12:53:21,106] jepsen worker 38 - jepsen.util 38 :ok :write [12 10] INFO [2022-01-10 12:53:21,107] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,108] jepsen worker 38 - jepsen.util 38 :ok :read [12 10] INFO [2022-01-10 12:53:21,108] jepsen worker 41 - jepsen.util 41 :ok :write [13 20] INFO [2022-01-10 12:53:21,109] jepsen worker 41 - jepsen.util 41 :invoke :write [13 38] INFO [2022-01-10 12:53:21,110] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,111] jepsen worker 38 - jepsen.util 38 :ok :read [12 10] INFO [2022-01-10 12:53:21,111] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,112] jepsen worker 38 - jepsen.util 38 :ok :read [12 10] INFO [2022-01-10 12:53:21,113] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,114] jepsen worker 38 - jepsen.util 38 :ok :read [12 10] INFO [2022-01-10 12:53:21,114] jepsen worker 3 - jepsen.util 3 :ok :write [1 8] INFO [2022-01-10 12:53:21,115] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,115] jepsen worker 3 - jepsen.util 3 :invoke :write [1 2] INFO [2022-01-10 12:53:21,116] jepsen worker 38 - jepsen.util 38 :ok :read [12 10] INFO [2022-01-10 12:53:21,117] jepsen worker 38 - jepsen.util 38 :invoke :write [12 6] INFO [2022-01-10 12:53:21,139] jepsen worker 2 - jepsen.util 92 :ok :write [0 11] INFO [2022-01-10 12:53:21,140] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:21,141] jepsen worker 85 - jepsen.util 175 :ok :write [28 7] INFO [2022-01-10 12:53:21,142] jepsen worker 2 - jepsen.util 92 :ok :read [0 11] INFO [2022-01-10 12:53:21,142] jepsen worker 85 - jepsen.util 175 :invoke :write [28 18] INFO [2022-01-10 12:53:21,143] jepsen worker 2 - jepsen.util 92 :invoke :write [0 25] INFO [2022-01-10 12:53:21,147] jepsen worker 87 - jepsen.util 87 :ok :write [29 38] INFO [2022-01-10 12:53:21,147] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:21,149] jepsen worker 87 - jepsen.util 87 :ok :read [29 38] INFO [2022-01-10 12:53:21,149] jepsen worker 38 - jepsen.util 38 :ok :write [12 6] INFO [2022-01-10 12:53:21,150] jepsen worker 38 - jepsen.util 38 :invoke :write [12 3] INFO [2022-01-10 12:53:21,150] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:21,151] jepsen worker 87 - jepsen.util 87 :ok :read [29 38] INFO [2022-01-10 12:53:21,152] jepsen worker 87 - jepsen.util 87 :invoke :write [29 37] INFO [2022-01-10 12:53:21,152] jepsen worker 48 - jepsen.util 138 :info :write [16 25] :RequestTimeoutException INFO [2022-01-10 12:53:21,153] jepsen worker 41 - jepsen.util 41 :ok :write [13 38] INFO [2022-01-10 12:53:21,153] jepsen worker 48 - jepsen.util 228 :invoke :write [16 2] INFO [2022-01-10 12:53:21,154] jepsen worker 41 - jepsen.util 41 :invoke :write [13 13] INFO [2022-01-10 12:53:21,156] jepsen worker 64 - jepsen.util 154 :info :write [21 41] :RequestTimeoutException INFO [2022-01-10 12:53:21,157] jepsen worker 64 - jepsen.util 244 :invoke :write [21 25] INFO [2022-01-10 12:53:21,157] jepsen worker 2 - jepsen.util 92 :ok :write [0 25] INFO [2022-01-10 12:53:21,157] jepsen worker 3 - jepsen.util 3 :ok :write [1 2] INFO [2022-01-10 12:53:21,158] jepsen worker 3 - jepsen.util 3 :invoke :write [1 42] INFO [2022-01-10 12:53:21,159] jepsen worker 2 - jepsen.util 92 :invoke :write [0 27] INFO [2022-01-10 12:53:21,165] jepsen worker 5 - jepsen.util 95 :info :write [1 38] :RequestTimeoutException INFO [2022-01-10 12:53:21,166] jepsen worker 5 - jepsen.util 185 :invoke :write [1 24] INFO [2022-01-10 12:53:21,167] jepsen worker 85 - jepsen.util 175 :ok :write [28 18] INFO [2022-01-10 12:53:21,167] jepsen worker 79 - jepsen.util 169 :info :write [26 32] :RequestTimeoutException INFO [2022-01-10 12:53:21,167] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,168] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,168] jepsen worker 85 - jepsen.util 175 :ok :read [28 18] INFO [2022-01-10 12:53:21,169] jepsen worker 79 - jepsen.util 259 :ok :read [26 44] INFO [2022-01-10 12:53:21,169] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,170] jepsen worker 41 - jepsen.util 41 :ok :write [13 13] INFO [2022-01-10 12:53:21,170] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,170] jepsen worker 85 - jepsen.util 175 :ok :read [28 18] INFO [2022-01-10 12:53:21,171] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:21,171] jepsen worker 87 - jepsen.util 87 :ok :write [29 37] INFO [2022-01-10 12:53:21,171] jepsen worker 79 - jepsen.util 259 :ok :read [26 44] INFO [2022-01-10 12:53:21,171] jepsen worker 85 - jepsen.util 175 :invoke :write [28 47] INFO [2022-01-10 12:53:21,172] jepsen worker 41 - jepsen.util 41 :ok :read [13 13] INFO [2022-01-10 12:53:21,172] jepsen worker 79 - jepsen.util 259 :invoke :write [26 34] INFO [2022-01-10 12:53:21,173] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:21,173] jepsen worker 38 - jepsen.util 38 :ok :write [12 3] INFO [2022-01-10 12:53:21,173] jepsen worker 87 - jepsen.util 87 :invoke :write [29 20] INFO [2022-01-10 12:53:21,174] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,174] jepsen worker 41 - jepsen.util 41 :ok :read [13 13] INFO [2022-01-10 12:53:21,174] jepsen worker 41 - jepsen.util 41 :invoke :write [13 10] INFO [2022-01-10 12:53:21,176] jepsen worker 38 - jepsen.util 38 :ok :read [12 3] INFO [2022-01-10 12:53:21,176] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,177] jepsen worker 38 - jepsen.util 38 :ok :read [12 3] INFO [2022-01-10 12:53:21,177] jepsen worker 38 - jepsen.util 38 :invoke :write [12 21] INFO [2022-01-10 12:53:21,178] jepsen worker 2 - jepsen.util 92 :ok :write [0 27] INFO [2022-01-10 12:53:21,178] jepsen worker 2 - jepsen.util 92 :invoke :write [0 43] INFO [2022-01-10 12:53:21,179] jepsen worker 15 - jepsen.util 15 :ok :write [5 29] INFO [2022-01-10 12:53:21,180] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,181] jepsen worker 15 - jepsen.util 15 :ok :read [5 29] INFO [2022-01-10 12:53:21,182] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,182] jepsen worker 15 - jepsen.util 15 :ok :read [5 29] INFO [2022-01-10 12:53:21,183] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,183] jepsen worker 3 - jepsen.util 3 :ok :write [1 42] INFO [2022-01-10 12:53:21,183] jepsen worker 66 - jepsen.util 156 :ok :write [22 39] INFO [2022-01-10 12:53:21,184] jepsen worker 15 - jepsen.util 15 :ok :read [5 29] INFO [2022-01-10 12:53:21,185] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,186] jepsen worker 13 - jepsen.util 13 :ok :write [4 24] INFO [2022-01-10 12:53:21,186] jepsen worker 15 - jepsen.util 15 :invoke :write [5 37] INFO [2022-01-10 12:53:21,186] jepsen worker 66 - jepsen.util 156 :ok :read [22 39] INFO [2022-01-10 12:53:21,188] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:21,188] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,188] jepsen worker 66 - jepsen.util 156 :invoke :write [22 12] INFO [2022-01-10 12:53:21,189] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:21,189] jepsen worker 13 - jepsen.util 13 :ok :read [4 24] INFO [2022-01-10 12:53:21,190] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,190] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:21,190] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:21,191] jepsen worker 13 - jepsen.util 13 :ok :read [4 24] INFO [2022-01-10 12:53:21,191] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,192] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:21,192] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:21,193] jepsen worker 13 - jepsen.util 13 :ok :read [4 24] INFO [2022-01-10 12:53:21,193] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,194] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:21,194] jepsen worker 13 - jepsen.util 13 :invoke :write [4 41] INFO [2022-01-10 12:53:21,195] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,195] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:21,196] jepsen worker 3 - jepsen.util 3 :invoke :write [1 17] INFO [2022-01-10 12:53:21,328] jepsen worker 41 - jepsen.util 41 :ok :write [13 10] INFO [2022-01-10 12:53:21,328] jepsen worker 41 - jepsen.util 41 :invoke :write [13 2] INFO [2022-01-10 12:53:21,329] jepsen worker 85 - jepsen.util 175 :ok :write [28 47] INFO [2022-01-10 12:53:21,329] jepsen worker 50 - jepsen.util 50 :ok :write [16 22] INFO [2022-01-10 12:53:21,330] jepsen worker 85 - jepsen.util 175 :invoke :write [28 14] INFO [2022-01-10 12:53:21,331] jepsen worker 50 - jepsen.util 50 :invoke :write [16 35] INFO [2022-01-10 12:53:21,331] jepsen worker 78 - jepsen.util 78 :ok :write [26 17] INFO [2022-01-10 12:53:21,332] jepsen worker 23 - jepsen.util 23 :ok :write [7 41] INFO [2022-01-10 12:53:21,332] jepsen worker 78 - jepsen.util 78 :invoke :write [26 42] INFO [2022-01-10 12:53:21,333] jepsen worker 23 - jepsen.util 23 :invoke :write [7 24] INFO [2022-01-10 12:53:21,334] jepsen worker 87 - jepsen.util 87 :ok :write [29 20] INFO [2022-01-10 12:53:21,335] jepsen worker 87 - jepsen.util 87 :invoke :write [29 13] INFO [2022-01-10 12:53:21,337] jepsen worker 37 - jepsen.util 127 :ok :write [12 45] INFO [2022-01-10 12:53:21,338] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,338] jepsen worker 70 - jepsen.util 160 :ok :write [23 24] INFO [2022-01-10 12:53:21,339] jepsen worker 37 - jepsen.util 127 :ok :read [12 45] INFO [2022-01-10 12:53:21,339] jepsen worker 70 - jepsen.util 160 :invoke :write [23 16] INFO [2022-01-10 12:53:21,339] jepsen worker 83 - jepsen.util 173 :ok :write [27 46] INFO [2022-01-10 12:53:21,340] jepsen worker 37 - jepsen.util 127 :invoke :write [12 36] INFO [2022-01-10 12:53:21,340] jepsen worker 2 - jepsen.util 92 :ok :write [0 43] INFO [2022-01-10 12:53:21,341] jepsen worker 83 - jepsen.util 173 :invoke :write [27 29] INFO [2022-01-10 12:53:21,342] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:21,343] jepsen worker 7 - jepsen.util 7 :ok :write [2 44] INFO [2022-01-10 12:53:21,343] jepsen worker 5 - jepsen.util 185 :ok :write [1 24] INFO [2022-01-10 12:53:21,343] jepsen worker 2 - jepsen.util 92 :ok :read [0 43] INFO [2022-01-10 12:53:21,344] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:21,344] jepsen worker 7 - jepsen.util 7 :ok :read [2 44] INFO [2022-01-10 12:53:21,345] jepsen worker 2 - jepsen.util 92 :invoke :write [0 37] INFO [2022-01-10 12:53:21,345] jepsen worker 7 - jepsen.util 7 :invoke :write [2 18] INFO [2022-01-10 12:53:21,346] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,347] jepsen worker 51 - jepsen.util 141 :ok :write [17 18] INFO [2022-01-10 12:53:21,347] jepsen worker 33 - jepsen.util 123 :ok :write [11 9] INFO [2022-01-10 12:53:21,348] jepsen worker 5 - jepsen.util 185 :ok :read [1 24] INFO [2022-01-10 12:53:21,348] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,349] jepsen worker 5 - jepsen.util 185 :invoke :write [1 33] INFO [2022-01-10 12:53:21,349] jepsen worker 51 - jepsen.util 141 :ok :read [17 18] INFO [2022-01-10 12:53:21,349] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,350] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,350] jepsen worker 33 - jepsen.util 123 :ok :read [11 9] INFO [2022-01-10 12:53:21,350] jepsen worker 51 - jepsen.util 141 :ok :read [17 18] INFO [2022-01-10 12:53:21,351] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,351] jepsen worker 51 - jepsen.util 141 :invoke :write [17 14] INFO [2022-01-10 12:53:21,352] jepsen worker 33 - jepsen.util 123 :ok :read [11 9] INFO [2022-01-10 12:53:21,352] jepsen worker 33 - jepsen.util 123 :invoke :write [11 35] INFO [2022-01-10 12:53:21,354] jepsen worker 44 - jepsen.util 44 :ok :write [14 25] INFO [2022-01-10 12:53:21,354] jepsen worker 44 - jepsen.util 44 :invoke :write [14 15] INFO [2022-01-10 12:53:21,357] jepsen worker 13 - jepsen.util 13 :ok :write [4 41] INFO [2022-01-10 12:53:21,357] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:21,358] jepsen worker 13 - jepsen.util 13 :ok :read [4 41] INFO [2022-01-10 12:53:21,359] jepsen worker 13 - jepsen.util 13 :invoke :write [4 8] INFO [2022-01-10 12:53:21,359] jepsen worker 85 - jepsen.util 175 :ok :write [28 14] INFO [2022-01-10 12:53:21,359] jepsen worker 41 - jepsen.util 41 :ok :write [13 2] INFO [2022-01-10 12:53:21,360] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,360] jepsen worker 77 - jepsen.util 167 :ok :write [25 48] INFO [2022-01-10 12:53:21,360] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:21,361] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:21,361] jepsen worker 85 - jepsen.util 175 :ok :read [28 14] INFO [2022-01-10 12:53:21,362] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,362] jepsen worker 77 - jepsen.util 167 :ok :read [25 48] INFO [2022-01-10 12:53:21,363] jepsen worker 77 - jepsen.util 167 :invoke :write [25 17] INFO [2022-01-10 12:53:21,363] jepsen worker 85 - jepsen.util 175 :ok :read [28 14] INFO [2022-01-10 12:53:21,363] jepsen worker 85 - jepsen.util 175 :invoke :write [28 22] INFO [2022-01-10 12:53:21,364] jepsen worker 78 - jepsen.util 78 :ok :write [26 42] INFO [2022-01-10 12:53:21,364] jepsen worker 37 - jepsen.util 127 :ok :write [12 36] INFO [2022-01-10 12:53:21,364] jepsen worker 78 - jepsen.util 78 :invoke :write [26 21] INFO [2022-01-10 12:53:21,365] jepsen worker 37 - jepsen.util 127 :invoke :write [12 14] INFO [2022-01-10 12:53:21,368] jepsen worker 62 - jepsen.util 152 :ok :write [20 19] INFO [2022-01-10 12:53:21,368] jepsen worker 62 - jepsen.util 152 :invoke :write [20 1] INFO [2022-01-10 12:53:21,371] jepsen worker 70 - jepsen.util 160 :ok :write [23 16] INFO [2022-01-10 12:53:21,372] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,373] jepsen worker 87 - jepsen.util 87 :ok :write [29 13] INFO [2022-01-10 12:53:21,373] jepsen worker 70 - jepsen.util 160 :ok :read [23 16] INFO [2022-01-10 12:53:21,373] jepsen worker 87 - jepsen.util 87 :invoke :write [29 13] INFO [2022-01-10 12:53:21,374] jepsen worker 70 - jepsen.util 160 :invoke :write [23 21] INFO [2022-01-10 12:53:21,376] jepsen worker 15 - jepsen.util 15 :ok :write [5 37] INFO [2022-01-10 12:53:21,376] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,377] jepsen worker 15 - jepsen.util 15 :ok :read [5 37] INFO [2022-01-10 12:53:21,378] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,379] jepsen worker 15 - jepsen.util 15 :ok :read [5 37] INFO [2022-01-10 12:53:21,379] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,380] jepsen worker 66 - jepsen.util 156 :ok :write [22 12] INFO [2022-01-10 12:53:21,380] jepsen worker 5 - jepsen.util 185 :ok :write [1 33] INFO [2022-01-10 12:53:21,380] jepsen worker 66 - jepsen.util 156 :invoke :write [22 47] INFO [2022-01-10 12:53:21,381] jepsen worker 15 - jepsen.util 15 :ok :read [5 37] INFO [2022-01-10 12:53:21,381] jepsen worker 5 - jepsen.util 185 :invoke :write [1 17] INFO [2022-01-10 12:53:21,381] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,382] jepsen worker 15 - jepsen.util 15 :ok :read [5 37] INFO [2022-01-10 12:53:21,382] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,383] jepsen worker 15 - jepsen.util 15 :ok :read [5 37] INFO [2022-01-10 12:53:21,384] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,384] jepsen worker 15 - jepsen.util 15 :ok :read [5 37] INFO [2022-01-10 12:53:21,385] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,386] jepsen worker 15 - jepsen.util 15 :ok :read [5 37] INFO [2022-01-10 12:53:21,386] jepsen worker 48 - jepsen.util 228 :ok :write [16 2] INFO [2022-01-10 12:53:21,387] jepsen worker 48 - jepsen.util 228 :invoke :write [16 12] INFO [2022-01-10 12:53:21,388] jepsen worker 15 - jepsen.util 15 :invoke :write [5 13] INFO [2022-01-10 12:53:21,388] jepsen worker 51 - jepsen.util 141 :ok :write [17 14] INFO [2022-01-10 12:53:21,389] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,389] jepsen worker 23 - jepsen.util 23 :ok :write [7 24] INFO [2022-01-10 12:53:21,390] jepsen worker 23 - jepsen.util 23 :invoke :write [7 25] INFO [2022-01-10 12:53:21,390] jepsen worker 51 - jepsen.util 141 :ok :read [17 14] INFO [2022-01-10 12:53:21,391] jepsen worker 2 - jepsen.util 92 :ok :write [0 37] INFO [2022-01-10 12:53:21,391] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,391] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:21,392] jepsen worker 51 - jepsen.util 141 :ok :read [17 14] INFO [2022-01-10 12:53:21,392] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,393] jepsen worker 51 - jepsen.util 141 :ok :read [17 14] INFO [2022-01-10 12:53:21,394] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,394] jepsen worker 51 - jepsen.util 141 :ok :read [17 14] INFO [2022-01-10 12:53:21,395] jepsen worker 51 - jepsen.util 141 :invoke :write [17 39] INFO [2022-01-10 12:53:21,397] jepsen worker 83 - jepsen.util 173 :ok :write [27 29] INFO [2022-01-10 12:53:21,398] jepsen worker 83 - jepsen.util 173 :invoke :write [27 13] INFO [2022-01-10 12:53:21,400] jepsen worker 41 - jepsen.util 41 :ok :read [13 2] INFO [2022-01-10 12:53:21,400] jepsen worker 2 - jepsen.util 92 :ok :read [0 37] INFO [2022-01-10 12:53:21,401] jepsen worker 2 - jepsen.util 92 :invoke :write [0 26] INFO [2022-01-10 12:53:21,401] jepsen worker 41 - jepsen.util 41 :invoke :write [13 44] INFO [2022-01-10 12:53:21,473] jepsen worker 13 - jepsen.util 13 :ok :write [4 8] INFO [2022-01-10 12:53:21,473] jepsen worker 87 - jepsen.util 87 :ok :write [29 13] INFO [2022-01-10 12:53:21,474] jepsen worker 87 - jepsen.util 87 :invoke :write [29 39] INFO [2022-01-10 12:53:21,474] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:21,475] jepsen worker 7 - jepsen.util 7 :ok :write [2 18] INFO [2022-01-10 12:53:21,475] jepsen worker 13 - jepsen.util 13 :ok :read [4 8] INFO [2022-01-10 12:53:21,476] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:21,476] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:21,477] jepsen worker 7 - jepsen.util 7 :ok :read [2 18] INFO [2022-01-10 12:53:21,477] jepsen worker 13 - jepsen.util 13 :ok :read [4 8] INFO [2022-01-10 12:53:21,477] jepsen worker 7 - jepsen.util 7 :invoke :write [2 47] INFO [2022-01-10 12:53:21,478] jepsen worker 13 - jepsen.util 13 :invoke :write [4 0] INFO [2022-01-10 12:53:21,480] jepsen worker 33 - jepsen.util 123 :ok :write [11 35] INFO [2022-01-10 12:53:21,481] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,482] jepsen worker 33 - jepsen.util 123 :ok :read [11 35] INFO [2022-01-10 12:53:21,482] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,483] jepsen worker 85 - jepsen.util 175 :ok :write [28 22] INFO [2022-01-10 12:53:21,483] jepsen worker 33 - jepsen.util 123 :ok :read [11 35] INFO [2022-01-10 12:53:21,484] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,484] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,484] jepsen worker 85 - jepsen.util 175 :ok :read [28 22] INFO [2022-01-10 12:53:21,485] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,485] jepsen worker 33 - jepsen.util 123 :ok :read [11 35] INFO [2022-01-10 12:53:21,485] jepsen worker 78 - jepsen.util 78 :ok :write [26 21] INFO [2022-01-10 12:53:21,486] jepsen worker 85 - jepsen.util 175 :ok :read [28 22] INFO [2022-01-10 12:53:21,486] jepsen worker 2 - jepsen.util 92 :ok :write [0 26] INFO [2022-01-10 12:53:21,486] jepsen worker 33 - jepsen.util 123 :invoke :write [11 22] INFO [2022-01-10 12:53:21,487] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:21,487] jepsen worker 85 - jepsen.util 175 :invoke :write [28 31] INFO [2022-01-10 12:53:21,488] jepsen worker 78 - jepsen.util 78 :invoke :write [26 38] INFO [2022-01-10 12:53:21,488] jepsen worker 2 - jepsen.util 92 :ok :read [0 26] INFO [2022-01-10 12:53:21,488] jepsen worker 44 - jepsen.util 44 :ok :write [14 15] INFO [2022-01-10 12:53:21,489] jepsen worker 2 - jepsen.util 92 :invoke :write [0 1] INFO [2022-01-10 12:53:21,490] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:21,490] jepsen worker 44 - jepsen.util 44 :ok :read [14 15] INFO [2022-01-10 12:53:21,491] jepsen worker 44 - jepsen.util 44 :invoke :write [14 19] INFO [2022-01-10 12:53:21,493] jepsen worker 41 - jepsen.util 41 :ok :write [13 44] INFO [2022-01-10 12:53:21,494] jepsen worker 41 - jepsen.util 41 :invoke :write [13 11] INFO [2022-01-10 12:53:21,496] jepsen worker 37 - jepsen.util 127 :ok :write [12 14] INFO [2022-01-10 12:53:21,496] jepsen worker 77 - jepsen.util 167 :ok :write [25 17] INFO [2022-01-10 12:53:21,496] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,497] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:21,497] jepsen worker 37 - jepsen.util 127 :ok :read [12 14] INFO [2022-01-10 12:53:21,498] jepsen worker 37 - jepsen.util 127 :invoke :write [12 49] INFO [2022-01-10 12:53:21,498] jepsen worker 77 - jepsen.util 167 :ok :read [25 17] INFO [2022-01-10 12:53:21,499] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:21,499] jepsen worker 77 - jepsen.util 167 :ok :read [25 17] INFO [2022-01-10 12:53:21,500] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:21,501] jepsen worker 77 - jepsen.util 167 :ok :read [25 17] INFO [2022-01-10 12:53:21,501] jepsen worker 62 - jepsen.util 152 :ok :write [20 1] INFO [2022-01-10 12:53:21,502] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:21,502] jepsen worker 87 - jepsen.util 87 :ok :write [29 39] INFO [2022-01-10 12:53:21,502] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:21,503] jepsen worker 77 - jepsen.util 167 :ok :read [25 17] INFO [2022-01-10 12:53:21,503] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:21,503] jepsen worker 62 - jepsen.util 152 :ok :read [20 1] INFO [2022-01-10 12:53:21,503] jepsen worker 77 - jepsen.util 167 :invoke :read [25 nil] INFO [2022-01-10 12:53:21,504] jepsen worker 87 - jepsen.util 87 :ok :read [29 39] INFO [2022-01-10 12:53:21,504] jepsen worker 62 - jepsen.util 152 :invoke :write [20 17] INFO [2022-01-10 12:53:21,504] jepsen worker 70 - jepsen.util 160 :ok :write [23 21] INFO [2022-01-10 12:53:21,504] jepsen worker 77 - jepsen.util 167 :ok :read [25 17] INFO [2022-01-10 12:53:21,505] jepsen worker 87 - jepsen.util 87 :invoke :write [29 18] INFO [2022-01-10 12:53:21,505] jepsen worker 77 - jepsen.util 167 :invoke :write [25 18] INFO [2022-01-10 12:53:21,506] jepsen worker 70 - jepsen.util 160 :invoke :write [23 11] INFO [2022-01-10 12:53:21,509] jepsen worker 15 - jepsen.util 15 :ok :write [5 13] INFO [2022-01-10 12:53:21,509] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,510] jepsen worker 15 - jepsen.util 15 :ok :read [5 13] INFO [2022-01-10 12:53:21,511] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,512] jepsen worker 15 - jepsen.util 15 :ok :read [5 13] INFO [2022-01-10 12:53:21,512] jepsen worker 2 - jepsen.util 92 :ok :write [0 1] INFO [2022-01-10 12:53:21,512] jepsen worker 5 - jepsen.util 185 :ok :write [1 17] INFO [2022-01-10 12:53:21,513] jepsen worker 66 - jepsen.util 156 :ok :write [22 47] INFO [2022-01-10 12:53:21,514] jepsen worker 2 - jepsen.util 92 :invoke :write [0 9] INFO [2022-01-10 12:53:21,515] jepsen worker 15 - jepsen.util 15 :invoke :write [5 26] INFO [2022-01-10 12:53:21,515] jepsen worker 5 - jepsen.util 185 :invoke :write [1 23] INFO [2022-01-10 12:53:21,516] jepsen worker 66 - jepsen.util 156 :invoke :write [22 11] INFO [2022-01-10 12:53:21,521] jepsen worker 48 - jepsen.util 228 :ok :write [16 12] INFO [2022-01-10 12:53:21,521] jepsen worker 51 - jepsen.util 141 :ok :write [17 39] INFO [2022-01-10 12:53:21,521] jepsen worker 48 - jepsen.util 228 :invoke :write [16 5] INFO [2022-01-10 12:53:21,522] jepsen worker 51 - jepsen.util 141 :invoke :write [17 10] INFO [2022-01-10 12:53:21,523] jepsen worker 23 - jepsen.util 23 :ok :write [7 25] INFO [2022-01-10 12:53:21,524] jepsen worker 23 - jepsen.util 23 :invoke :write [7 28] INFO [2022-01-10 12:53:21,526] jepsen worker 41 - jepsen.util 41 :ok :write [13 11] INFO [2022-01-10 12:53:21,527] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:21,528] jepsen worker 41 - jepsen.util 41 :ok :read [13 11] INFO [2022-01-10 12:53:21,528] jepsen worker 41 - jepsen.util 41 :invoke :write [13 18] INFO [2022-01-10 12:53:21,529] jepsen worker 13 - jepsen.util 13 :ok :write [4 0] INFO [2022-01-10 12:53:21,529] jepsen worker 13 - jepsen.util 13 :invoke :write [4 47] INFO [2022-01-10 12:53:21,531] jepsen worker 83 - jepsen.util 173 :ok :write [27 13] INFO [2022-01-10 12:53:21,532] jepsen worker 83 - jepsen.util 173 :invoke :write [27 38] INFO [2022-01-10 12:53:21,538] jepsen worker 8 - jepsen.util 8 :ok :write [2 39] INFO [2022-01-10 12:53:21,538] jepsen worker 85 - jepsen.util 175 :ok :write [28 31] INFO [2022-01-10 12:53:21,538] jepsen worker 8 - jepsen.util 8 :invoke :write [2 34] INFO [2022-01-10 12:53:21,539] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,540] jepsen worker 87 - jepsen.util 87 :ok :write [29 18] INFO [2022-01-10 12:53:21,540] jepsen worker 85 - jepsen.util 175 :ok :read [28 31] INFO [2022-01-10 12:53:21,541] jepsen worker 87 - jepsen.util 87 :invoke :write [29 15] INFO [2022-01-10 12:53:21,541] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,542] jepsen worker 85 - jepsen.util 175 :ok :read [28 31] INFO [2022-01-10 12:53:21,542] jepsen worker 85 - jepsen.util 175 :invoke :write [28 2] INFO [2022-01-10 12:53:21,543] jepsen worker 33 - jepsen.util 123 :ok :write [11 22] INFO [2022-01-10 12:53:21,544] jepsen worker 33 - jepsen.util 123 :invoke :write [11 37] INFO [2022-01-10 12:53:21,545] jepsen worker 78 - jepsen.util 78 :ok :write [26 38] INFO [2022-01-10 12:53:21,546] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,547] jepsen worker 78 - jepsen.util 78 :ok :read [26 38] INFO [2022-01-10 12:53:21,547] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,548] jepsen worker 44 - jepsen.util 44 :ok :write [14 19] INFO [2022-01-10 12:53:21,548] jepsen worker 78 - jepsen.util 78 :ok :read [26 38] INFO [2022-01-10 12:53:21,549] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,549] jepsen worker 44 - jepsen.util 44 :invoke :write [14 18] INFO [2022-01-10 12:53:21,549] jepsen worker 78 - jepsen.util 78 :ok :read [26 38] INFO [2022-01-10 12:53:21,550] jepsen worker 78 - jepsen.util 78 :invoke :write [26 17] INFO [2022-01-10 12:53:21,553] jepsen worker 37 - jepsen.util 127 :ok :write [12 49] INFO [2022-01-10 12:53:21,553] jepsen worker 2 - jepsen.util 92 :ok :write [0 9] INFO [2022-01-10 12:53:21,554] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,554] jepsen worker 77 - jepsen.util 167 :ok :write [25 18] INFO [2022-01-10 12:53:21,554] jepsen worker 2 - jepsen.util 92 :invoke :write [0 12] INFO [2022-01-10 12:53:21,555] jepsen worker 37 - jepsen.util 127 :ok :read [12 49] INFO [2022-01-10 12:53:21,555] jepsen worker 77 - jepsen.util 167 :invoke :write [25 17] INFO [2022-01-10 12:53:21,556] jepsen worker 37 - jepsen.util 127 :invoke :write [12 30] INFO [2022-01-10 12:53:21,561] jepsen worker 70 - jepsen.util 160 :ok :write [23 11] INFO [2022-01-10 12:53:21,561] jepsen worker 70 - jepsen.util 160 :invoke :write [23 2] INFO [2022-01-10 12:53:21,562] jepsen worker 62 - jepsen.util 152 :ok :write [20 17] INFO [2022-01-10 12:53:21,562] jepsen worker 62 - jepsen.util 152 :invoke :read [20 nil] INFO [2022-01-10 12:53:21,567] jepsen worker 41 - jepsen.util 41 :ok :write [13 18] INFO [2022-01-10 12:53:21,568] jepsen worker 41 - jepsen.util 41 :invoke :write [13 21] INFO [2022-01-10 12:53:21,568] jepsen worker 15 - jepsen.util 15 :ok :write [5 26] INFO [2022-01-10 12:53:21,569] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,570] jepsen worker 15 - jepsen.util 15 :ok :read [5 26] INFO [2022-01-10 12:53:21,570] jepsen worker 5 - jepsen.util 185 :ok :write [1 23] INFO [2022-01-10 12:53:21,570] jepsen worker 15 - jepsen.util 15 :invoke :write [5 36] INFO [2022-01-10 12:53:21,571] jepsen worker 5 - jepsen.util 185 :invoke :write [1 35] INFO [2022-01-10 12:53:21,573] jepsen worker 87 - jepsen.util 87 :ok :write [29 15] INFO [2022-01-10 12:53:21,573] jepsen worker 87 - jepsen.util 87 :invoke :write [29 24] INFO [2022-01-10 12:53:21,576] jepsen worker 66 - jepsen.util 156 :ok :write [22 11] INFO [2022-01-10 12:53:21,577] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,578] jepsen worker 51 - jepsen.util 141 :ok :write [17 10] INFO [2022-01-10 12:53:21,578] jepsen worker 66 - jepsen.util 156 :ok :read [22 11] INFO [2022-01-10 12:53:21,578] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,579] jepsen worker 51 - jepsen.util 141 :ok :read [17 10] INFO [2022-01-10 12:53:21,579] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,579] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,580] jepsen worker 48 - jepsen.util 228 :ok :write [16 5] INFO [2022-01-10 12:53:21,580] jepsen worker 51 - jepsen.util 141 :ok :read [17 10] INFO [2022-01-10 12:53:21,581] jepsen worker 48 - jepsen.util 228 :invoke :write [16 5] INFO [2022-01-10 12:53:21,581] jepsen worker 66 - jepsen.util 156 :ok :read [22 11] INFO [2022-01-10 12:53:21,581] jepsen worker 51 - jepsen.util 141 :invoke :write [17 17] INFO [2022-01-10 12:53:21,582] jepsen worker 66 - jepsen.util 156 :invoke :write [22 41] INFO [2022-01-10 12:53:21,584] jepsen worker 2 - jepsen.util 92 :ok :write [0 12] INFO [2022-01-10 12:53:21,584] jepsen worker 23 - jepsen.util 23 :ok :write [7 28] INFO [2022-01-10 12:53:21,585] jepsen worker 13 - jepsen.util 13 :ok :write [4 47] INFO [2022-01-10 12:53:21,585] jepsen worker 23 - jepsen.util 23 :invoke :write [7 5] INFO [2022-01-10 12:53:21,586] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:21,587] jepsen worker 13 - jepsen.util 13 :invoke :write [4 34] INFO [2022-01-10 12:53:21,587] jepsen worker 2 - jepsen.util 92 :ok :read [0 12] INFO [2022-01-10 12:53:21,588] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:21,588] jepsen worker 2 - jepsen.util 92 :ok :read [0 12] INFO [2022-01-10 12:53:21,589] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:21,590] jepsen worker 2 - jepsen.util 92 :ok :read [0 12] INFO [2022-01-10 12:53:21,591] jepsen worker 2 - jepsen.util 92 :invoke :write [0 14] INFO [2022-01-10 12:53:21,591] jepsen worker 83 - jepsen.util 173 :ok :write [27 38] INFO [2022-01-10 12:53:21,592] jepsen worker 85 - jepsen.util 175 :ok :write [28 2] INFO [2022-01-10 12:53:21,592] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:21,593] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,593] jepsen worker 83 - jepsen.util 173 :ok :read [27 38] INFO [2022-01-10 12:53:21,593] jepsen worker 85 - jepsen.util 175 :ok :read [28 2] INFO [2022-01-10 12:53:21,594] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:21,594] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,595] jepsen worker 83 - jepsen.util 173 :ok :read [27 38] INFO [2022-01-10 12:53:21,595] jepsen worker 85 - jepsen.util 175 :ok :read [28 2] INFO [2022-01-10 12:53:21,595] jepsen worker 8 - jepsen.util 8 :ok :write [2 34] INFO [2022-01-10 12:53:21,595] jepsen worker 83 - jepsen.util 173 :invoke :write [27 4] INFO [2022-01-10 12:53:21,596] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:21,597] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,597] jepsen worker 34 - jepsen.util 34 :info :write [11 24] :RequestTimeoutException INFO [2022-01-10 12:53:21,597] jepsen worker 8 - jepsen.util 8 :ok :read [2 34] INFO [2022-01-10 12:53:21,597] jepsen worker 78 - jepsen.util 78 :ok :write [26 17] INFO [2022-01-10 12:53:21,598] jepsen worker 85 - jepsen.util 175 :ok :read [28 2] INFO [2022-01-10 12:53:21,598] jepsen worker 34 - jepsen.util 124 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,599] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,599] jepsen worker 34 - jepsen.util 124 :ok :read [11 22] INFO [2022-01-10 12:53:21,600] jepsen worker 78 - jepsen.util 78 :invoke :write [26 9] INFO [2022-01-10 12:53:21,600] jepsen worker 85 - jepsen.util 175 :ok :read [28 2] INFO [2022-01-10 12:53:21,600] jepsen worker 8 - jepsen.util 8 :invoke :write [2 33] INFO [2022-01-10 12:53:21,601] jepsen worker 34 - jepsen.util 124 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,601] jepsen worker 85 - jepsen.util 175 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,601] jepsen worker 34 - jepsen.util 124 :ok :read [11 22] INFO [2022-01-10 12:53:21,602] jepsen worker 85 - jepsen.util 175 :ok :read [28 2] INFO [2022-01-10 12:53:21,602] jepsen worker 34 - jepsen.util 124 :invoke :write [11 4] INFO [2022-01-10 12:53:21,602] jepsen worker 85 - jepsen.util 175 :invoke :write [28 23] INFO [2022-01-10 12:53:21,604] jepsen worker 41 - jepsen.util 41 :ok :write [13 21] INFO [2022-01-10 12:53:21,604] jepsen worker 9 - jepsen.util 99 :info :write [3 29] :RequestTimeoutException INFO [2022-01-10 12:53:21,605] jepsen worker 81 - jepsen.util 81 :info :write [27 2] :RequestTimeoutException INFO [2022-01-10 12:53:21,605] jepsen worker 37 - jepsen.util 127 :ok :write [12 30] INFO [2022-01-10 12:53:21,605] jepsen worker 41 - jepsen.util 41 :invoke :write [13 24] INFO [2022-01-10 12:53:21,606] jepsen worker 9 - jepsen.util 189 :invoke :read [3 nil] INFO [2022-01-10 12:53:21,607] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,607] jepsen worker 33 - jepsen.util 123 :ok :write [11 37] INFO [2022-01-10 12:53:21,608] jepsen worker 37 - jepsen.util 127 :ok :read [12 30] INFO [2022-01-10 12:53:21,608] jepsen worker 33 - jepsen.util 123 :invoke :write [11 44] INFO [2022-01-10 12:53:21,609] jepsen worker 37 - jepsen.util 127 :invoke :write [12 36] INFO [2022-01-10 12:53:21,609] jepsen worker 87 - jepsen.util 87 :ok :write [29 24] INFO [2022-01-10 12:53:21,610] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:21,611] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:21,611] jepsen worker 87 - jepsen.util 87 :ok :read [29 24] INFO [2022-01-10 12:53:21,612] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:21,612] jepsen worker 81 - jepsen.util 171 :ok :read [27 38] INFO [2022-01-10 12:53:21,613] jepsen worker 87 - jepsen.util 87 :ok :read [29 24] INFO [2022-01-10 12:53:21,613] jepsen worker 70 - jepsen.util 160 :ok :write [23 2] INFO [2022-01-10 12:53:21,614] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:21,614] jepsen worker 18 - jepsen.util 108 :info :write [6 16] :RequestTimeoutException INFO [2022-01-10 12:53:21,615] jepsen worker 81 - jepsen.util 171 :ok :read [27 38] INFO [2022-01-10 12:53:21,615] jepsen worker 87 - jepsen.util 87 :invoke :write [29 29] INFO [2022-01-10 12:53:21,615] jepsen worker 44 - jepsen.util 44 :ok :write [14 18] INFO [2022-01-10 12:53:21,616] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,616] jepsen worker 44 - jepsen.util 44 :invoke :write [14 23] INFO [2022-01-10 12:53:21,617] jepsen worker 65 - jepsen.util 155 :info :write [21 26] :RequestTimeoutException INFO [2022-01-10 12:53:21,617] jepsen worker 18 - jepsen.util 198 :invoke :write [6 28] INFO [2022-01-10 12:53:21,617] jepsen worker 70 - jepsen.util 160 :ok :read [23 2] INFO [2022-01-10 12:53:21,618] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:21,618] jepsen worker 2 - jepsen.util 92 :ok :write [0 14] INFO [2022-01-10 12:53:21,618] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,618] jepsen worker 81 - jepsen.util 171 :ok :read [27 38] INFO [2022-01-10 12:53:21,619] jepsen worker 65 - jepsen.util 245 :invoke :write [21 23] INFO [2022-01-10 12:53:21,620] jepsen worker 2 - jepsen.util 92 :invoke :write [0 46] INFO [2022-01-10 12:53:21,620] jepsen worker 70 - jepsen.util 160 :ok :read [23 2] INFO [2022-01-10 12:53:21,620] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:21,621] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,621] jepsen worker 81 - jepsen.util 171 :ok :read [27 38] INFO [2022-01-10 12:53:21,621] jepsen worker 5 - jepsen.util 185 :ok :write [1 35] INFO [2022-01-10 12:53:21,621] jepsen worker 70 - jepsen.util 160 :ok :read [23 2] INFO [2022-01-10 12:53:21,622] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:21,622] jepsen worker 5 - jepsen.util 185 :invoke :write [1 24] INFO [2022-01-10 12:53:21,622] jepsen worker 81 - jepsen.util 171 :ok :read [27 38] INFO [2022-01-10 12:53:21,623] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,623] jepsen worker 81 - jepsen.util 171 :invoke :write [27 37] INFO [2022-01-10 12:53:21,623] jepsen worker 70 - jepsen.util 160 :ok :read [23 2] INFO [2022-01-10 12:53:21,624] jepsen worker 70 - jepsen.util 160 :invoke :write [23 19] INFO [2022-01-10 12:53:21,632] jepsen worker 51 - jepsen.util 141 :ok :write [17 17] INFO [2022-01-10 12:53:21,632] jepsen worker 47 - jepsen.util 137 :info :write [15 22] :RequestTimeoutException INFO [2022-01-10 12:53:21,633] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,633] jepsen worker 15 - jepsen.util 15 :ok :write [5 36] INFO [2022-01-10 12:53:21,634] jepsen worker 47 - jepsen.util 227 :invoke :read [15 nil] INFO [2022-01-10 12:53:21,634] jepsen worker 51 - jepsen.util 141 :ok :read [17 17] INFO [2022-01-10 12:53:21,634] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,635] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,635] jepsen worker 47 - jepsen.util 227 :ok :read [15 10] INFO [2022-01-10 12:53:21,635] jepsen worker 15 - jepsen.util 15 :ok :read [5 36] INFO [2022-01-10 12:53:21,636] jepsen worker 51 - jepsen.util 141 :ok :read [17 17] INFO [2022-01-10 12:53:21,636] jepsen worker 47 - jepsen.util 227 :invoke :write [15 26] INFO [2022-01-10 12:53:21,636] jepsen worker 66 - jepsen.util 156 :ok :write [22 41] INFO [2022-01-10 12:53:21,636] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,637] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,637] jepsen worker 56 - jepsen.util 146 :info :write [18 3] :RequestTimeoutException INFO [2022-01-10 12:53:21,637] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,637] jepsen worker 15 - jepsen.util 15 :ok :read [5 36] INFO [2022-01-10 12:53:21,638] jepsen worker 66 - jepsen.util 156 :ok :read [22 41] INFO [2022-01-10 12:53:21,638] jepsen worker 56 - jepsen.util 236 :invoke :write [18 0] INFO [2022-01-10 12:53:21,638] jepsen worker 51 - jepsen.util 141 :ok :read [17 17] INFO [2022-01-10 12:53:21,638] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,639] jepsen worker 48 - jepsen.util 228 :ok :write [16 5] INFO [2022-01-10 12:53:21,639] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,639] jepsen worker 66 - jepsen.util 156 :ok :read [22 41] INFO [2022-01-10 12:53:21,640] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:21,640] jepsen worker 51 - jepsen.util 141 :ok :read [17 17] INFO [2022-01-10 12:53:21,640] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,640] jepsen worker 48 - jepsen.util 228 :ok :read [16 5] INFO [2022-01-10 12:53:21,640] jepsen worker 13 - jepsen.util 13 :ok :write [4 34] INFO [2022-01-10 12:53:21,641] jepsen worker 66 - jepsen.util 156 :ok :read [22 41] INFO [2022-01-10 12:53:21,641] jepsen worker 51 - jepsen.util 141 :invoke :write [17 2] INFO [2022-01-10 12:53:21,642] jepsen worker 15 - jepsen.util 15 :invoke :write [5 42] INFO [2022-01-10 12:53:21,642] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,643] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:21,643] jepsen worker 66 - jepsen.util 156 :ok :read [22 41] INFO [2022-01-10 12:53:21,643] jepsen worker 23 - jepsen.util 23 :ok :write [7 5] INFO [2022-01-10 12:53:21,644] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:21,644] jepsen worker 48 - jepsen.util 228 :ok :read [16 5] INFO [2022-01-10 12:53:21,645] jepsen worker 66 - jepsen.util 156 :invoke :write [22 11] INFO [2022-01-10 12:53:21,645] jepsen worker 13 - jepsen.util 13 :ok :read [4 34] INFO [2022-01-10 12:53:21,646] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:21,646] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:21,647] jepsen worker 48 - jepsen.util 228 :ok :read [16 5] INFO [2022-01-10 12:53:21,647] jepsen worker 13 - jepsen.util 13 :invoke :write [4 27] INFO [2022-01-10 12:53:21,647] jepsen worker 23 - jepsen.util 23 :ok :read [7 5] INFO [2022-01-10 12:53:21,648] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:21,648] jepsen worker 23 - jepsen.util 23 :invoke :write [7 26] INFO [2022-01-10 12:53:21,649] jepsen worker 48 - jepsen.util 228 :ok :read [16 5] INFO [2022-01-10 12:53:21,649] jepsen worker 84 - jepsen.util 84 :ok :write [28 33] INFO [2022-01-10 12:53:21,649] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:21,650] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,650] jepsen worker 48 - jepsen.util 228 :ok :read [16 5] INFO [2022-01-10 12:53:21,651] jepsen worker 84 - jepsen.util 84 :ok :read [28 33] INFO [2022-01-10 12:53:21,651] jepsen worker 48 - jepsen.util 228 :invoke :write [16 33] INFO [2022-01-10 12:53:21,652] jepsen worker 84 - jepsen.util 84 :invoke :write [28 28] INFO [2022-01-10 12:53:21,656] jepsen worker 78 - jepsen.util 78 :ok :write [26 9] INFO [2022-01-10 12:53:21,657] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,658] jepsen worker 78 - jepsen.util 78 :ok :read [26 9] INFO [2022-01-10 12:53:21,659] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,660] jepsen worker 78 - jepsen.util 78 :ok :read [26 9] INFO [2022-01-10 12:53:21,660] jepsen worker 78 - jepsen.util 78 :invoke :write [26 11] INFO [2022-01-10 12:53:21,665] jepsen worker 20 - jepsen.util 110 :info :write [6 38] :RequestTimeoutException INFO [2022-01-10 12:53:21,665] jepsen worker 74 - jepsen.util 164 :info :write [24 20] :RequestTimeoutException INFO [2022-01-10 12:53:21,665] jepsen worker 20 - jepsen.util 200 :invoke :write [6 3] INFO [2022-01-10 12:53:21,666] jepsen worker 74 - jepsen.util 254 :invoke :read [24 nil] INFO [2022-01-10 12:53:21,667] jepsen worker 44 - jepsen.util 44 :ok :write [14 23] INFO [2022-01-10 12:53:21,667] jepsen worker 70 - jepsen.util 160 :ok :write [23 19] INFO [2022-01-10 12:53:21,668] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:21,668] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,669] jepsen worker 44 - jepsen.util 44 :ok :read [14 23] INFO [2022-01-10 12:53:21,669] jepsen worker 70 - jepsen.util 160 :ok :read [23 19] INFO [2022-01-10 12:53:21,669] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,670] jepsen worker 70 - jepsen.util 160 :ok :read [23 19] INFO [2022-01-10 12:53:21,670] jepsen worker 44 - jepsen.util 44 :invoke :write [14 27] INFO [2022-01-10 12:53:21,670] jepsen worker 70 - jepsen.util 160 :invoke :write [23 1] INFO [2022-01-10 12:53:21,681] jepsen worker 51 - jepsen.util 141 :ok :write [17 2] INFO [2022-01-10 12:53:21,681] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,682] jepsen worker 51 - jepsen.util 141 :ok :read [17 2] INFO [2022-01-10 12:53:21,682] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,683] jepsen worker 51 - jepsen.util 141 :ok :read [17 2] INFO [2022-01-10 12:53:21,684] jepsen worker 51 - jepsen.util 141 :invoke :write [17 48] INFO [2022-01-10 12:53:21,687] jepsen worker 13 - jepsen.util 13 :ok :write [4 27] INFO [2022-01-10 12:53:21,687] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:21,688] jepsen worker 13 - jepsen.util 13 :ok :read [4 27] INFO [2022-01-10 12:53:21,688] jepsen worker 13 - jepsen.util 13 :invoke :write [4 40] INFO [2022-01-10 12:53:21,697] jepsen worker 78 - jepsen.util 78 :ok :write [26 11] INFO [2022-01-10 12:53:21,698] jepsen worker 78 - jepsen.util 78 :invoke :write [26 10] INFO [2022-01-10 12:53:21,703] jepsen worker 70 - jepsen.util 160 :ok :write [23 1] INFO [2022-01-10 12:53:21,703] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,704] jepsen worker 70 - jepsen.util 160 :ok :read [23 1] INFO [2022-01-10 12:53:21,704] jepsen worker 23 - jepsen.util 23 :ok :write [7 26] INFO [2022-01-10 12:53:21,704] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,705] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:21,705] jepsen worker 70 - jepsen.util 160 :ok :read [23 1] INFO [2022-01-10 12:53:21,706] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,706] jepsen worker 23 - jepsen.util 23 :ok :read [7 26] INFO [2022-01-10 12:53:21,707] jepsen worker 70 - jepsen.util 160 :ok :read [23 1] INFO [2022-01-10 12:53:21,707] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:21,707] jepsen worker 70 - jepsen.util 160 :invoke :write [23 26] INFO [2022-01-10 12:53:21,708] jepsen worker 23 - jepsen.util 23 :ok :read [7 26] INFO [2022-01-10 12:53:21,708] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:21,709] jepsen worker 23 - jepsen.util 23 :ok :read [7 26] INFO [2022-01-10 12:53:21,710] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:21,710] jepsen worker 51 - jepsen.util 141 :ok :write [17 48] INFO [2022-01-10 12:53:21,710] jepsen worker 44 - jepsen.util 44 :ok :write [14 27] INFO [2022-01-10 12:53:21,710] jepsen worker 51 - jepsen.util 141 :invoke :write [17 20] INFO [2022-01-10 12:53:21,711] jepsen worker 23 - jepsen.util 23 :ok :read [7 26] INFO [2022-01-10 12:53:21,711] jepsen worker 44 - jepsen.util 44 :invoke :write [14 21] INFO [2022-01-10 12:53:21,711] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:21,712] jepsen worker 23 - jepsen.util 23 :ok :read [7 26] INFO [2022-01-10 12:53:21,713] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:21,714] jepsen worker 23 - jepsen.util 23 :ok :read [7 26] INFO [2022-01-10 12:53:21,714] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:21,715] jepsen worker 23 - jepsen.util 23 :ok :read [7 26] INFO [2022-01-10 12:53:21,716] jepsen worker 23 - jepsen.util 23 :invoke :write [7 24] INFO [2022-01-10 12:53:21,720] jepsen worker 13 - jepsen.util 13 :ok :write [4 40] INFO [2022-01-10 12:53:21,721] jepsen worker 13 - jepsen.util 13 :invoke :write [4 45] INFO [2022-01-10 12:53:21,723] jepsen worker 37 - jepsen.util 127 :ok :write [12 36] INFO [2022-01-10 12:53:21,723] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,724] jepsen worker 37 - jepsen.util 127 :ok :read [12 36] INFO [2022-01-10 12:53:21,725] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,725] jepsen worker 37 - jepsen.util 127 :ok :read [12 36] INFO [2022-01-10 12:53:21,726] jepsen worker 83 - jepsen.util 173 :ok :write [27 4] INFO [2022-01-10 12:53:21,726] jepsen worker 37 - jepsen.util 127 :invoke :write [12 5] INFO [2022-01-10 12:53:21,727] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:21,728] jepsen worker 83 - jepsen.util 173 :ok :read [27 4] INFO [2022-01-10 12:53:21,728] jepsen worker 5 - jepsen.util 185 :ok :write [1 24] INFO [2022-01-10 12:53:21,728] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,729] jepsen worker 83 - jepsen.util 173 :invoke :read [27 nil] INFO [2022-01-10 12:53:21,729] jepsen worker 5 - jepsen.util 185 :ok :read [1 24] INFO [2022-01-10 12:53:21,730] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,730] jepsen worker 83 - jepsen.util 173 :ok :read [27 4] INFO [2022-01-10 12:53:21,730] jepsen worker 78 - jepsen.util 78 :ok :write [26 10] INFO [2022-01-10 12:53:21,730] jepsen worker 5 - jepsen.util 185 :ok :read [1 24] INFO [2022-01-10 12:53:21,730] jepsen worker 83 - jepsen.util 173 :invoke :write [27 35] INFO [2022-01-10 12:53:21,731] jepsen worker 5 - jepsen.util 185 :invoke :write [1 4] INFO [2022-01-10 12:53:21,731] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,732] jepsen worker 78 - jepsen.util 78 :ok :read [26 10] INFO [2022-01-10 12:53:21,732] jepsen worker 8 - jepsen.util 8 :ok :write [2 33] INFO [2022-01-10 12:53:21,733] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,734] jepsen worker 8 - jepsen.util 8 :invoke :write [2 34] INFO [2022-01-10 12:53:21,734] jepsen worker 84 - jepsen.util 84 :ok :write [28 28] INFO [2022-01-10 12:53:21,734] jepsen worker 78 - jepsen.util 78 :ok :read [26 10] INFO [2022-01-10 12:53:21,735] jepsen worker 84 - jepsen.util 84 :invoke :write [28 17] INFO [2022-01-10 12:53:21,735] jepsen worker 78 - jepsen.util 78 :invoke :write [26 39] INFO [2022-01-10 12:53:21,737] jepsen worker 40 - jepsen.util 40 :info :write [13 14] :RequestTimeoutException INFO [2022-01-10 12:53:21,738] jepsen worker 40 - jepsen.util 130 :invoke :write [13 3] INFO [2022-01-10 12:53:21,739] jepsen worker 41 - jepsen.util 41 :ok :write [13 24] INFO [2022-01-10 12:53:21,739] jepsen worker 41 - jepsen.util 41 :invoke :write [13 30] INFO [2022-01-10 12:53:21,745] jepsen worker 33 - jepsen.util 123 :ok :write [11 44] INFO [2022-01-10 12:53:21,746] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,747] jepsen worker 33 - jepsen.util 123 :ok :read [11 44] INFO [2022-01-10 12:53:21,747] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,748] jepsen worker 33 - jepsen.util 123 :ok :read [11 44] INFO [2022-01-10 12:53:21,748] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,750] jepsen worker 33 - jepsen.util 123 :ok :read [11 44] INFO [2022-01-10 12:53:21,750] jepsen worker 33 - jepsen.util 123 :invoke :write [11 36] INFO [2022-01-10 12:53:21,751] jepsen worker 87 - jepsen.util 87 :ok :write [29 29] INFO [2022-01-10 12:53:21,751] jepsen worker 87 - jepsen.util 87 :invoke :write [29 46] INFO [2022-01-10 12:53:21,755] jepsen worker 2 - jepsen.util 92 :ok :write [0 46] INFO [2022-01-10 12:53:21,756] jepsen worker 2 - jepsen.util 92 :invoke :write [0 47] INFO [2022-01-10 12:53:21,756] jepsen worker 37 - jepsen.util 127 :ok :write [12 5] INFO [2022-01-10 12:53:21,757] jepsen worker 37 - jepsen.util 127 :invoke :write [12 40] INFO [2022-01-10 12:53:21,757] jepsen worker 29 - jepsen.util 119 :info :write [9 23] :RequestTimeoutException INFO [2022-01-10 12:53:21,758] jepsen worker 29 - jepsen.util 209 :invoke :read [9 nil] INFO [2022-01-10 12:53:21,759] jepsen worker 15 - jepsen.util 15 :ok :write [5 42] INFO [2022-01-10 12:53:21,760] jepsen worker 15 - jepsen.util 15 :invoke :write [5 9] INFO [2022-01-10 12:53:21,765] jepsen worker 66 - jepsen.util 156 :ok :write [22 11] INFO [2022-01-10 12:53:21,765] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,766] jepsen worker 66 - jepsen.util 156 :ok :read [22 11] INFO [2022-01-10 12:53:21,766] jepsen worker 66 - jepsen.util 156 :invoke :write [22 45] INFO [2022-01-10 12:53:21,769] jepsen worker 50 - jepsen.util 50 :ok :write [16 35] INFO [2022-01-10 12:53:21,769] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:21,770] jepsen worker 50 - jepsen.util 50 :ok :read [16 35] INFO [2022-01-10 12:53:21,770] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:21,771] jepsen worker 50 - jepsen.util 50 :ok :read [16 35] INFO [2022-01-10 12:53:21,771] jepsen worker 50 - jepsen.util 50 :invoke :write [16 36] INFO [2022-01-10 12:53:21,779] jepsen worker 40 - jepsen.util 130 :ok :write [13 3] INFO [2022-01-10 12:53:21,780] jepsen worker 40 - jepsen.util 130 :invoke :write [13 27] INFO [2022-01-10 12:53:21,852] jepsen worker 70 - jepsen.util 160 :ok :write [23 26] INFO [2022-01-10 12:53:21,853] jepsen worker 70 - jepsen.util 160 :invoke :write [23 37] INFO [2022-01-10 12:53:21,857] jepsen worker 51 - jepsen.util 141 :ok :write [17 20] INFO [2022-01-10 12:53:21,858] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,858] jepsen worker 51 - jepsen.util 141 :ok :read [17 20] INFO [2022-01-10 12:53:21,859] jepsen worker 51 - jepsen.util 141 :invoke :write [17 33] INFO [2022-01-10 12:53:21,860] jepsen worker 44 - jepsen.util 44 :ok :write [14 21] INFO [2022-01-10 12:53:21,861] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:21,863] jepsen worker 44 - jepsen.util 44 :ok :read [14 21] INFO [2022-01-10 12:53:21,863] jepsen worker 13 - jepsen.util 13 :ok :write [4 45] INFO [2022-01-10 12:53:21,864] jepsen worker 13 - jepsen.util 13 :invoke :write [4 40] INFO [2022-01-10 12:53:21,864] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:21,866] jepsen worker 44 - jepsen.util 44 :ok :read [14 21] INFO [2022-01-10 12:53:21,866] jepsen worker 61 - jepsen.util 61 :ok :write [20 40] INFO [2022-01-10 12:53:21,866] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:21,867] jepsen worker 61 - jepsen.util 61 :invoke :write [20 38] INFO [2022-01-10 12:53:21,867] jepsen worker 44 - jepsen.util 44 :ok :read [14 21] INFO [2022-01-10 12:53:21,868] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:21,869] jepsen worker 44 - jepsen.util 44 :ok :read [14 21] INFO [2022-01-10 12:53:21,869] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:21,870] jepsen worker 22 - jepsen.util 112 :ok :write [7 9] INFO [2022-01-10 12:53:21,871] jepsen worker 87 - jepsen.util 87 :ok :write [29 46] INFO [2022-01-10 12:53:21,871] jepsen worker 5 - jepsen.util 185 :ok :write [1 4] INFO [2022-01-10 12:53:21,871] jepsen worker 44 - jepsen.util 44 :ok :read [14 21] INFO [2022-01-10 12:53:21,871] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:21,872] jepsen worker 87 - jepsen.util 87 :invoke :write [29 18] INFO [2022-01-10 12:53:21,872] jepsen worker 22 - jepsen.util 112 :ok :read [7 9] INFO [2022-01-10 12:53:21,872] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:21,873] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,873] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:21,873] jepsen worker 44 - jepsen.util 44 :ok :read [14 21] INFO [2022-01-10 12:53:21,874] jepsen worker 5 - jepsen.util 185 :ok :read [1 4] INFO [2022-01-10 12:53:21,874] jepsen worker 44 - jepsen.util 44 :invoke :write [14 44] INFO [2022-01-10 12:53:21,875] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,875] jepsen worker 22 - jepsen.util 112 :ok :read [7 9] INFO [2022-01-10 12:53:21,875] jepsen worker 83 - jepsen.util 173 :ok :write [27 35] INFO [2022-01-10 12:53:21,875] jepsen worker 22 - jepsen.util 112 :invoke :write [7 21] INFO [2022-01-10 12:53:21,876] jepsen worker 5 - jepsen.util 185 :ok :read [1 4] INFO [2022-01-10 12:53:21,876] jepsen worker 83 - jepsen.util 173 :invoke :write [27 23] INFO [2022-01-10 12:53:21,876] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,877] jepsen worker 78 - jepsen.util 78 :ok :write [26 39] INFO [2022-01-10 12:53:21,878] jepsen worker 5 - jepsen.util 185 :ok :read [1 4] INFO [2022-01-10 12:53:21,878] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,879] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,879] jepsen worker 78 - jepsen.util 78 :ok :read [26 39] INFO [2022-01-10 12:53:21,879] jepsen worker 2 - jepsen.util 92 :ok :write [0 47] INFO [2022-01-10 12:53:21,879] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,880] jepsen worker 2 - jepsen.util 92 :invoke :write [0 49] INFO [2022-01-10 12:53:21,880] jepsen worker 5 - jepsen.util 185 :ok :read [1 4] INFO [2022-01-10 12:53:21,880] jepsen worker 78 - jepsen.util 78 :ok :read [26 39] INFO [2022-01-10 12:53:21,881] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,881] jepsen worker 78 - jepsen.util 78 :invoke :write [26 13] INFO [2022-01-10 12:53:21,882] jepsen worker 5 - jepsen.util 185 :ok :read [1 4] INFO [2022-01-10 12:53:21,883] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,884] jepsen worker 5 - jepsen.util 185 :ok :read [1 4] INFO [2022-01-10 12:53:21,884] jepsen worker 84 - jepsen.util 84 :ok :write [28 17] INFO [2022-01-10 12:53:21,884] jepsen worker 5 - jepsen.util 185 :invoke :write [1 1] INFO [2022-01-10 12:53:21,885] jepsen worker 40 - jepsen.util 130 :ok :write [13 27] INFO [2022-01-10 12:53:21,885] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,886] jepsen worker 40 - jepsen.util 130 :invoke :write [13 16] INFO [2022-01-10 12:53:21,886] jepsen worker 84 - jepsen.util 84 :ok :read [28 17] INFO [2022-01-10 12:53:21,886] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,887] jepsen worker 84 - jepsen.util 84 :ok :read [28 17] INFO [2022-01-10 12:53:21,888] jepsen worker 8 - jepsen.util 8 :ok :write [2 34] INFO [2022-01-10 12:53:21,888] jepsen worker 84 - jepsen.util 84 :invoke :write [28 32] INFO [2022-01-10 12:53:21,888] jepsen worker 8 - jepsen.util 8 :invoke :write [2 20] INFO [2022-01-10 12:53:21,891] jepsen worker 37 - jepsen.util 127 :ok :write [12 40] INFO [2022-01-10 12:53:21,892] jepsen worker 37 - jepsen.util 127 :invoke :write [12 5] INFO [2022-01-10 12:53:21,894] jepsen worker 33 - jepsen.util 123 :ok :write [11 36] INFO [2022-01-10 12:53:21,894] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,895] jepsen worker 33 - jepsen.util 123 :ok :read [11 36] INFO [2022-01-10 12:53:21,896] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,896] jepsen worker 33 - jepsen.util 123 :ok :read [11 36] INFO [2022-01-10 12:53:21,897] jepsen worker 33 - jepsen.util 123 :invoke :write [11 17] INFO [2022-01-10 12:53:21,898] jepsen worker 70 - jepsen.util 160 :ok :write [23 37] INFO [2022-01-10 12:53:21,898] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,899] jepsen worker 15 - jepsen.util 15 :ok :write [5 9] INFO [2022-01-10 12:53:21,899] jepsen worker 70 - jepsen.util 160 :ok :read [23 37] INFO [2022-01-10 12:53:21,899] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:21,900] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,900] jepsen worker 70 - jepsen.util 160 :ok :read [23 37] INFO [2022-01-10 12:53:21,901] jepsen worker 15 - jepsen.util 15 :ok :read [5 9] INFO [2022-01-10 12:53:21,901] jepsen worker 70 - jepsen.util 160 :invoke :write [23 38] INFO [2022-01-10 12:53:21,901] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,902] jepsen worker 15 - jepsen.util 15 :ok :read [5 9] INFO [2022-01-10 12:53:21,902] jepsen worker 66 - jepsen.util 156 :ok :write [22 45] INFO [2022-01-10 12:53:21,903] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,904] jepsen worker 15 - jepsen.util 15 :ok :read [5 9] INFO [2022-01-10 12:53:21,904] jepsen worker 66 - jepsen.util 156 :invoke :write [22 13] INFO [2022-01-10 12:53:21,904] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:21,905] jepsen worker 87 - jepsen.util 87 :ok :write [29 18] INFO [2022-01-10 12:53:21,906] jepsen worker 51 - jepsen.util 141 :ok :write [17 33] INFO [2022-01-10 12:53:21,906] jepsen worker 15 - jepsen.util 15 :ok :read [5 9] INFO [2022-01-10 12:53:21,906] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:21,907] jepsen worker 50 - jepsen.util 50 :ok :write [16 36] INFO [2022-01-10 12:53:21,907] jepsen worker 15 - jepsen.util 15 :invoke :write [5 35] INFO [2022-01-10 12:53:21,907] jepsen worker 87 - jepsen.util 87 :ok :read [29 18] INFO [2022-01-10 12:53:21,908] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:21,908] jepsen worker 51 - jepsen.util 141 :invoke :write [17 36] INFO [2022-01-10 12:53:21,909] jepsen worker 50 - jepsen.util 50 :ok :read [16 36] INFO [2022-01-10 12:53:21,909] jepsen worker 87 - jepsen.util 87 :invoke :write [29 41] INFO [2022-01-10 12:53:21,910] jepsen worker 50 - jepsen.util 50 :invoke :write [16 49] INFO [2022-01-10 12:53:21,911] jepsen worker 44 - jepsen.util 44 :ok :write [14 44] INFO [2022-01-10 12:53:21,912] jepsen worker 44 - jepsen.util 44 :invoke :write [14 40] INFO [2022-01-10 12:53:21,914] jepsen worker 13 - jepsen.util 13 :ok :write [4 40] INFO [2022-01-10 12:53:21,914] jepsen worker 61 - jepsen.util 61 :ok :write [20 38] INFO [2022-01-10 12:53:21,915] jepsen worker 13 - jepsen.util 13 :invoke :write [4 25] INFO [2022-01-10 12:53:21,915] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:21,916] jepsen worker 61 - jepsen.util 61 :ok :read [20 38] INFO [2022-01-10 12:53:21,917] jepsen worker 61 - jepsen.util 61 :invoke :write [20 8] INFO [2022-01-10 12:53:21,920] jepsen worker 2 - jepsen.util 92 :ok :write [0 49] INFO [2022-01-10 12:53:21,920] jepsen worker 5 - jepsen.util 185 :ok :write [1 1] INFO [2022-01-10 12:53:21,920] jepsen worker 2 - jepsen.util 92 :invoke :write [0 15] INFO [2022-01-10 12:53:21,921] jepsen worker 5 - jepsen.util 185 :invoke :write [1 30] INFO [2022-01-10 12:53:21,925] jepsen worker 22 - jepsen.util 112 :ok :write [7 21] INFO [2022-01-10 12:53:21,925] jepsen worker 22 - jepsen.util 112 :invoke :write [7 2] INFO [2022-01-10 12:53:21,927] jepsen worker 40 - jepsen.util 130 :ok :write [13 16] INFO [2022-01-10 12:53:21,927] jepsen worker 78 - jepsen.util 78 :ok :write [26 13] INFO [2022-01-10 12:53:21,927] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:21,928] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:21,928] jepsen worker 40 - jepsen.util 130 :ok :read [13 16] INFO [2022-01-10 12:53:21,929] jepsen worker 40 - jepsen.util 130 :invoke :write [13 35] INFO [2022-01-10 12:53:21,929] jepsen worker 78 - jepsen.util 78 :ok :read [26 13] INFO [2022-01-10 12:53:21,929] jepsen worker 78 - jepsen.util 78 :invoke :write [26 0] INFO [2022-01-10 12:53:21,930] jepsen worker 83 - jepsen.util 173 :ok :write [27 23] INFO [2022-01-10 12:53:21,930] jepsen worker 87 - jepsen.util 87 :ok :write [29 41] INFO [2022-01-10 12:53:21,931] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:21,932] jepsen worker 83 - jepsen.util 173 :invoke :write [27 9] INFO [2022-01-10 12:53:21,932] jepsen worker 87 - jepsen.util 87 :ok :read [29 41] INFO [2022-01-10 12:53:21,933] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:21,933] jepsen worker 84 - jepsen.util 84 :ok :write [28 32] INFO [2022-01-10 12:53:21,934] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,934] jepsen worker 87 - jepsen.util 87 :ok :read [29 41] INFO [2022-01-10 12:53:21,935] jepsen worker 84 - jepsen.util 84 :ok :read [28 32] INFO [2022-01-10 12:53:21,935] jepsen worker 87 - jepsen.util 87 :invoke :write [29 32] INFO [2022-01-10 12:53:21,935] jepsen worker 8 - jepsen.util 8 :ok :write [2 20] INFO [2022-01-10 12:53:21,935] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,936] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:21,937] jepsen worker 8 - jepsen.util 8 :ok :read [2 20] INFO [2022-01-10 12:53:21,937] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:21,937] jepsen worker 84 - jepsen.util 84 :ok :read [28 32] INFO [2022-01-10 12:53:21,938] jepsen worker 8 - jepsen.util 8 :ok :read [2 20] INFO [2022-01-10 12:53:21,938] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,938] jepsen worker 8 - jepsen.util 8 :invoke :write [2 49] INFO [2022-01-10 12:53:21,939] jepsen worker 84 - jepsen.util 84 :ok :read [28 32] INFO [2022-01-10 12:53:21,939] jepsen worker 37 - jepsen.util 127 :ok :write [12 5] INFO [2022-01-10 12:53:21,939] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,940] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,940] jepsen worker 84 - jepsen.util 84 :ok :read [28 32] INFO [2022-01-10 12:53:21,941] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:21,941] jepsen worker 37 - jepsen.util 127 :ok :read [12 5] INFO [2022-01-10 12:53:21,942] jepsen worker 84 - jepsen.util 84 :ok :read [28 32] INFO [2022-01-10 12:53:21,942] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,942] jepsen worker 33 - jepsen.util 123 :ok :write [11 17] INFO [2022-01-10 12:53:21,942] jepsen worker 84 - jepsen.util 84 :invoke :write [28 12] INFO [2022-01-10 12:53:21,943] jepsen worker 33 - jepsen.util 123 :invoke :write [11 14] INFO [2022-01-10 12:53:21,943] jepsen worker 37 - jepsen.util 127 :ok :read [12 5] INFO [2022-01-10 12:53:21,944] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,945] jepsen worker 37 - jepsen.util 127 :ok :read [12 5] INFO [2022-01-10 12:53:21,945] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,946] jepsen worker 70 - jepsen.util 160 :ok :write [23 38] INFO [2022-01-10 12:53:21,946] jepsen worker 70 - jepsen.util 160 :invoke :write [23 39] INFO [2022-01-10 12:53:21,947] jepsen worker 37 - jepsen.util 127 :ok :read [12 5] INFO [2022-01-10 12:53:21,947] jepsen worker 37 - jepsen.util 127 :invoke :write [12 32] INFO [2022-01-10 12:53:21,949] jepsen worker 15 - jepsen.util 15 :ok :write [5 35] INFO [2022-01-10 12:53:21,950] jepsen worker 15 - jepsen.util 15 :invoke :write [5 32] INFO [2022-01-10 12:53:21,954] jepsen worker 52 - jepsen.util 52 :ok :write [17 48] INFO [2022-01-10 12:53:21,955] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,955] jepsen worker 2 - jepsen.util 92 :ok :write [0 15] INFO [2022-01-10 12:53:21,956] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:21,956] jepsen worker 66 - jepsen.util 156 :ok :write [22 13] INFO [2022-01-10 12:53:21,956] jepsen worker 52 - jepsen.util 52 :ok :read [17 48] INFO [2022-01-10 12:53:21,956] jepsen worker 2 - jepsen.util 92 :ok :read [0 15] INFO [2022-01-10 12:53:21,957] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,957] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,958] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:21,958] jepsen worker 66 - jepsen.util 156 :ok :read [22 13] INFO [2022-01-10 12:53:21,958] jepsen worker 44 - jepsen.util 44 :ok :write [14 40] INFO [2022-01-10 12:53:21,959] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,959] jepsen worker 2 - jepsen.util 92 :ok :read [0 15] INFO [2022-01-10 12:53:21,959] jepsen worker 52 - jepsen.util 52 :ok :read [17 48] INFO [2022-01-10 12:53:21,959] jepsen worker 44 - jepsen.util 44 :invoke :write [14 24] INFO [2022-01-10 12:53:21,960] jepsen worker 66 - jepsen.util 156 :ok :read [22 13] INFO [2022-01-10 12:53:21,960] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:21,961] jepsen worker 2 - jepsen.util 92 :invoke :write [0 25] INFO [2022-01-10 12:53:21,961] jepsen worker 52 - jepsen.util 52 :ok :read [17 48] INFO [2022-01-10 12:53:21,961] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,962] jepsen worker 52 - jepsen.util 52 :invoke :write [17 28] INFO [2022-01-10 12:53:21,962] jepsen worker 66 - jepsen.util 156 :ok :read [22 13] INFO [2022-01-10 12:53:21,962] jepsen worker 13 - jepsen.util 13 :ok :write [4 25] INFO [2022-01-10 12:53:21,963] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,963] jepsen worker 13 - jepsen.util 13 :invoke :write [4 34] INFO [2022-01-10 12:53:21,964] jepsen worker 66 - jepsen.util 156 :ok :read [22 13] INFO [2022-01-10 12:53:21,964] jepsen worker 50 - jepsen.util 50 :ok :write [16 49] INFO [2022-01-10 12:53:21,964] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:21,965] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:21,965] jepsen worker 66 - jepsen.util 156 :ok :read [22 13] INFO [2022-01-10 12:53:21,966] jepsen worker 50 - jepsen.util 50 :ok :read [16 49] INFO [2022-01-10 12:53:21,966] jepsen worker 66 - jepsen.util 156 :invoke :write [22 14] INFO [2022-01-10 12:53:21,967] jepsen worker 50 - jepsen.util 50 :invoke :write [16 45] INFO [2022-01-10 12:53:21,968] jepsen worker 61 - jepsen.util 61 :ok :write [20 8] INFO [2022-01-10 12:53:21,969] jepsen worker 61 - jepsen.util 61 :invoke :write [20 26] INFO [2022-01-10 12:53:21,969] jepsen worker 40 - jepsen.util 130 :ok :write [13 35] INFO [2022-01-10 12:53:21,970] jepsen worker 40 - jepsen.util 130 :invoke :write [13 42] INFO [2022-01-10 12:53:21,972] jepsen worker 5 - jepsen.util 185 :ok :write [1 30] INFO [2022-01-10 12:53:21,972] jepsen worker 87 - jepsen.util 87 :ok :write [29 32] INFO [2022-01-10 12:53:21,973] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:21,974] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:21,974] jepsen worker 22 - jepsen.util 112 :ok :write [7 2] INFO [2022-01-10 12:53:21,975] jepsen worker 5 - jepsen.util 185 :ok :read [1 30] INFO [2022-01-10 12:53:21,975] jepsen worker 22 - jepsen.util 112 :invoke :write [7 22] INFO [2022-01-10 12:53:21,975] jepsen worker 87 - jepsen.util 87 :ok :read [29 32] INFO [2022-01-10 12:53:21,976] jepsen worker 5 - jepsen.util 185 :invoke :write [1 22] INFO [2022-01-10 12:53:21,976] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:21,977] jepsen worker 87 - jepsen.util 87 :ok :read [29 32] INFO [2022-01-10 12:53:21,978] jepsen worker 87 - jepsen.util 87 :invoke :write [29 6] INFO [2022-01-10 12:53:21,979] jepsen worker 2 - jepsen.util 92 :ok :write [0 25] INFO [2022-01-10 12:53:21,980] jepsen worker 2 - jepsen.util 92 :invoke :write [0 36] INFO [2022-01-10 12:53:21,981] jepsen worker 78 - jepsen.util 78 :ok :write [26 0] INFO [2022-01-10 12:53:21,982] jepsen worker 78 - jepsen.util 78 :invoke :write [26 16] INFO [2022-01-10 12:53:21,983] jepsen worker 81 - jepsen.util 171 :ok :write [27 37] INFO [2022-01-10 12:53:21,984] jepsen worker 81 - jepsen.util 171 :invoke :write [27 24] INFO [2022-01-10 12:53:21,986] jepsen worker 84 - jepsen.util 84 :ok :write [28 12] INFO [2022-01-10 12:53:21,986] jepsen worker 84 - jepsen.util 84 :invoke :write [28 37] INFO [2022-01-10 12:53:21,991] jepsen worker 8 - jepsen.util 8 :ok :write [2 49] INFO [2022-01-10 12:53:21,991] jepsen worker 37 - jepsen.util 127 :ok :write [12 32] INFO [2022-01-10 12:53:21,992] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:21,992] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,992] jepsen worker 8 - jepsen.util 8 :ok :read [2 49] INFO [2022-01-10 12:53:21,993] jepsen worker 37 - jepsen.util 127 :ok :read [12 32] INFO [2022-01-10 12:53:21,993] jepsen worker 8 - jepsen.util 8 :invoke :write [2 45] INFO [2022-01-10 12:53:21,994] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,994] jepsen worker 33 - jepsen.util 123 :ok :write [11 14] INFO [2022-01-10 12:53:21,995] jepsen worker 37 - jepsen.util 127 :ok :read [12 32] INFO [2022-01-10 12:53:21,995] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,995] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,996] jepsen worker 33 - jepsen.util 123 :ok :read [11 14] INFO [2022-01-10 12:53:21,996] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,996] jepsen worker 37 - jepsen.util 127 :ok :read [12 32] INFO [2022-01-10 12:53:21,997] jepsen worker 33 - jepsen.util 123 :ok :read [11 14] INFO [2022-01-10 12:53:21,997] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:21,998] jepsen worker 70 - jepsen.util 160 :ok :write [23 39] INFO [2022-01-10 12:53:21,998] jepsen worker 40 - jepsen.util 130 :ok :write [13 42] INFO [2022-01-10 12:53:21,998] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:21,999] jepsen worker 37 - jepsen.util 127 :ok :read [12 32] INFO [2022-01-10 12:53:21,999] jepsen worker 70 - jepsen.util 160 :invoke :write [23 30] INFO [2022-01-10 12:53:21,999] jepsen worker 33 - jepsen.util 123 :ok :read [11 14] INFO [2022-01-10 12:53:21,999] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:22,000] jepsen worker 37 - jepsen.util 127 :invoke :write [12 22] INFO [2022-01-10 12:53:22,000] jepsen worker 15 - jepsen.util 15 :ok :write [5 32] INFO [2022-01-10 12:53:22,000] jepsen worker 40 - jepsen.util 130 :ok :read [13 42] INFO [2022-01-10 12:53:22,000] jepsen worker 33 - jepsen.util 123 :invoke :write [11 4] INFO [2022-01-10 12:53:22,001] jepsen worker 40 - jepsen.util 130 :invoke :write [13 27] INFO [2022-01-10 12:53:22,001] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:22,002] jepsen worker 15 - jepsen.util 15 :ok :read [5 32] INFO [2022-01-10 12:53:22,002] jepsen worker 66 - jepsen.util 156 :ok :write [22 14] INFO [2022-01-10 12:53:22,002] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:22,003] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:22,003] jepsen worker 87 - jepsen.util 87 :ok :write [29 6] INFO [2022-01-10 12:53:22,004] jepsen worker 66 - jepsen.util 156 :ok :read [22 14] INFO [2022-01-10 12:53:22,004] jepsen worker 15 - jepsen.util 15 :ok :read [5 32] INFO [2022-01-10 12:53:22,004] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:22,005] jepsen worker 66 - jepsen.util 156 :invoke :write [22 10] INFO [2022-01-10 12:53:22,005] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:22,005] jepsen worker 15 - jepsen.util 15 :invoke :write [5 48] INFO [2022-01-10 12:53:22,006] jepsen worker 87 - jepsen.util 87 :invoke :write [29 29] INFO [2022-01-10 12:53:22,007] jepsen worker 52 - jepsen.util 52 :ok :write [17 28] INFO [2022-01-10 12:53:22,008] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:22,009] jepsen worker 44 - jepsen.util 44 :ok :write [14 24] INFO [2022-01-10 12:53:22,010] jepsen worker 44 - jepsen.util 44 :invoke :write [14 35] INFO [2022-01-10 12:53:22,011] jepsen worker 52 - jepsen.util 52 :ok :read [17 28] INFO [2022-01-10 12:53:22,012] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:22,013] jepsen worker 52 - jepsen.util 52 :ok :read [17 28] INFO [2022-01-10 12:53:22,013] jepsen worker 52 - jepsen.util 52 :invoke :write [17 32] INFO [2022-01-10 12:53:22,016] jepsen worker 50 - jepsen.util 50 :ok :write [16 45] INFO [2022-01-10 12:53:22,016] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:22,016] jepsen worker 13 - jepsen.util 13 :ok :write [4 34] INFO [2022-01-10 12:53:22,017] jepsen worker 50 - jepsen.util 50 :ok :read [16 45] INFO [2022-01-10 12:53:22,017] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:22,018] jepsen worker 50 - jepsen.util 50 :invoke :write [16 39] INFO [2022-01-10 12:53:22,019] jepsen worker 61 - jepsen.util 61 :ok :write [20 26] INFO [2022-01-10 12:53:22,019] jepsen worker 13 - jepsen.util 13 :ok :read [4 34] INFO [2022-01-10 12:53:22,019] jepsen worker 2 - jepsen.util 92 :ok :write [0 36] INFO [2022-01-10 12:53:22,019] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:22,020] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:22,020] jepsen worker 61 - jepsen.util 61 :ok :read [20 26] INFO [2022-01-10 12:53:22,020] jepsen worker 13 - jepsen.util 13 :invoke :write [4 13] INFO [2022-01-10 12:53:22,021] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:22,021] jepsen worker 2 - jepsen.util 92 :ok :read [0 36] INFO [2022-01-10 12:53:22,021] jepsen worker 2 - jepsen.util 92 :invoke :write [0 15] INFO [2022-01-10 12:53:22,022] jepsen worker 61 - jepsen.util 61 :ok :read [20 26] INFO [2022-01-10 12:53:22,022] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:22,023] jepsen worker 61 - jepsen.util 61 :ok :read [20 26] INFO [2022-01-10 12:53:22,023] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:22,024] jepsen worker 22 - jepsen.util 112 :ok :write [7 22] INFO [2022-01-10 12:53:22,024] jepsen worker 61 - jepsen.util 61 :ok :read [20 26] INFO [2022-01-10 12:53:22,024] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:22,024] jepsen worker 5 - jepsen.util 185 :ok :write [1 22] INFO [2022-01-10 12:53:22,024] jepsen worker 40 - jepsen.util 130 :ok :write [13 27] INFO [2022-01-10 12:53:22,025] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:22,025] jepsen worker 40 - jepsen.util 130 :invoke :write [13 46] INFO [2022-01-10 12:53:22,026] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:22,026] jepsen worker 22 - jepsen.util 112 :ok :read [7 22] INFO [2022-01-10 12:53:22,026] jepsen worker 61 - jepsen.util 61 :ok :read [20 26] INFO [2022-01-10 12:53:22,026] jepsen worker 61 - jepsen.util 61 :invoke :write [20 48] INFO [2022-01-10 12:53:22,027] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:22,027] jepsen worker 5 - jepsen.util 185 :ok :read [1 22] INFO [2022-01-10 12:53:22,027] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:22,028] jepsen worker 22 - jepsen.util 112 :ok :read [7 22] INFO [2022-01-10 12:53:22,028] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:22,028] jepsen worker 5 - jepsen.util 185 :ok :read [1 22] INFO [2022-01-10 12:53:22,029] jepsen worker 22 - jepsen.util 112 :ok :read [7 22] INFO [2022-01-10 12:53:22,029] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:22,029] jepsen worker 22 - jepsen.util 112 :invoke :write [7 40] INFO [2022-01-10 12:53:22,030] jepsen worker 5 - jepsen.util 185 :ok :read [1 22] INFO [2022-01-10 12:53:22,030] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:22,031] jepsen worker 5 - jepsen.util 185 :ok :read [1 22] INFO [2022-01-10 12:53:22,031] jepsen worker 81 - jepsen.util 171 :ok :write [27 24] INFO [2022-01-10 12:53:22,032] jepsen worker 5 - jepsen.util 185 :invoke :write [1 30] INFO [2022-01-10 12:53:22,032] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:22,033] jepsen worker 81 - jepsen.util 171 :ok :read [27 24] INFO [2022-01-10 12:53:22,033] jepsen worker 8 - jepsen.util 8 :ok :write [2 45] INFO [2022-01-10 12:53:22,033] jepsen worker 81 - jepsen.util 171 :invoke :write [27 14] INFO [2022-01-10 12:53:22,034] jepsen worker 78 - jepsen.util 78 :ok :write [26 16] INFO [2022-01-10 12:53:22,034] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:22,035] jepsen worker 78 - jepsen.util 78 :invoke :write [26 42] INFO [2022-01-10 12:53:22,035] jepsen worker 8 - jepsen.util 8 :ok :read [2 45] INFO [2022-01-10 12:53:22,036] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:22,036] jepsen worker 8 - jepsen.util 8 :ok :read [2 45] INFO [2022-01-10 12:53:22,037] jepsen worker 8 - jepsen.util 8 :invoke :write [2 16] INFO [2022-01-10 12:53:22,037] jepsen worker 87 - jepsen.util 87 :ok :write [29 29] INFO [2022-01-10 12:53:22,038] jepsen worker 87 - jepsen.util 87 :invoke :write [29 45] INFO [2022-01-10 12:53:22,039] jepsen worker 33 - jepsen.util 123 :ok :write [11 4] INFO [2022-01-10 12:53:22,039] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,040] jepsen worker 33 - jepsen.util 123 :ok :read [11 4] INFO [2022-01-10 12:53:22,040] jepsen worker 84 - jepsen.util 84 :ok :write [28 37] INFO [2022-01-10 12:53:22,041] jepsen worker 33 - jepsen.util 123 :invoke :write [11 22] INFO [2022-01-10 12:53:22,041] jepsen worker 84 - jepsen.util 84 :invoke :write [28 11] INFO [2022-01-10 12:53:22,045] jepsen worker 37 - jepsen.util 127 :ok :write [12 22] INFO [2022-01-10 12:53:22,046] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:22,046] jepsen worker 15 - jepsen.util 15 :ok :write [5 48] INFO [2022-01-10 12:53:22,046] jepsen worker 15 - jepsen.util 15 :invoke :write [5 47] INFO [2022-01-10 12:53:22,046] jepsen worker 37 - jepsen.util 127 :ok :read [12 22] INFO [2022-01-10 12:53:22,047] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:22,048] jepsen worker 37 - jepsen.util 127 :ok :read [12 22] INFO [2022-01-10 12:53:22,048] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:22,049] jepsen worker 37 - jepsen.util 127 :ok :read [12 22] INFO [2022-01-10 12:53:22,050] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:22,050] jepsen worker 37 - jepsen.util 127 :ok :read [12 22] INFO [2022-01-10 12:53:22,051] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:22,051] jepsen worker 70 - jepsen.util 160 :ok :write [23 30] INFO [2022-01-10 12:53:22,051] jepsen worker 37 - jepsen.util 127 :ok :read [12 22] INFO [2022-01-10 12:53:22,052] jepsen worker 70 - jepsen.util 160 :invoke :write [23 14] INFO [2022-01-10 12:53:22,052] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:22,053] jepsen worker 37 - jepsen.util 127 :ok :read [12 22] INFO [2022-01-10 12:53:22,053] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:22,054] jepsen worker 37 - jepsen.util 127 :ok :read [12 22] INFO [2022-01-10 12:53:22,055] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:22,056] jepsen worker 37 - jepsen.util 127 :ok :read [12 22] INFO [2022-01-10 12:53:22,056] jepsen worker 51 - jepsen.util 141 :ok :write [17 36] INFO [2022-01-10 12:53:22,056] jepsen worker 37 - jepsen.util 127 :invoke :write [12 36] INFO [2022-01-10 12:53:22,057] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:22,058] jepsen worker 51 - jepsen.util 141 :ok :read [17 36] INFO [2022-01-10 12:53:22,058] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:22,059] jepsen worker 51 - jepsen.util 141 :ok :read [17 36] INFO [2022-01-10 12:53:22,060] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:22,060] jepsen worker 13 - jepsen.util 13 :ok :write [4 13] INFO [2022-01-10 12:53:22,060] jepsen worker 51 - jepsen.util 141 :ok :read [17 36] INFO [2022-01-10 12:53:22,061] jepsen worker 51 - jepsen.util 141 :invoke :write [17 14] INFO [2022-01-10 12:53:22,062] jepsen worker 13 - jepsen.util 13 :invoke :write [4 12] INFO [2022-01-10 12:53:22,130] jepsen worker 5 - jepsen.util 185 :ok :write [1 30] INFO [2022-01-10 12:53:22,131] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:22,132] jepsen worker 5 - jepsen.util 185 :ok :read [1 30] INFO [2022-01-10 12:53:22,132] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:22,133] jepsen worker 5 - jepsen.util 185 :ok :read [1 30] INFO [2022-01-10 12:53:22,133] jepsen worker 5 - jepsen.util 185 :invoke :write [1 1] INFO [2022-01-10 12:53:22,136] jepsen worker 84 - jepsen.util 84 :ok :write [28 11] INFO [2022-01-10 12:53:22,136] jepsen worker 2 - jepsen.util 92 :ok :write [0 15] INFO [2022-01-10 12:53:22,137] jepsen worker 2 - jepsen.util 92 :invoke :write [0 45] INFO [2022-01-10 12:53:22,137] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:22,138] jepsen worker 84 - jepsen.util 84 :ok :read [28 11] INFO [2022-01-10 12:53:22,139] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:22,139] jepsen worker 78 - jepsen.util 78 :ok :write [26 42] INFO [2022-01-10 12:53:22,139] jepsen worker 66 - jepsen.util 156 :ok :write [22 10] INFO [2022-01-10 12:53:22,140] jepsen worker 66 - jepsen.util 156 :invoke :read [22 nil] INFO [2022-01-10 12:53:22,140] jepsen worker 78 - jepsen.util 78 :invoke :write [26 19] INFO [2022-01-10 12:53:22,141] jepsen worker 84 - jepsen.util 84 :ok :read [28 11] INFO [2022-01-10 12:53:22,141] jepsen worker 66 - jepsen.util 156 :ok :read [22 10] INFO [2022-01-10 12:53:22,142] jepsen worker 84 - jepsen.util 84 :invoke :write [28 5] INFO [2022-01-10 12:53:22,143] jepsen worker 66 - jepsen.util 156 :invoke :write [22 1] INFO [2022-01-10 12:53:22,143] jepsen worker 70 - jepsen.util 160 :ok :write [23 14] INFO [2022-01-10 12:53:22,143] jepsen worker 44 - jepsen.util 44 :ok :write [14 35] INFO [2022-01-10 12:53:22,144] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:22,145] jepsen worker 44 - jepsen.util 44 :invoke :write [14 7] INFO [2022-01-10 12:53:22,145] jepsen worker 70 - jepsen.util 160 :ok :read [23 14] INFO [2022-01-10 12:53:22,145] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:22,145] jepsen worker 40 - jepsen.util 130 :ok :write [13 46] INFO [2022-01-10 12:53:22,146] jepsen worker 70 - jepsen.util 160 :ok :read [23 14] INFO [2022-01-10 12:53:22,146] jepsen worker 37 - jepsen.util 127 :ok :write [12 36] INFO [2022-01-10 12:53:22,146] jepsen worker 40 - jepsen.util 130 :invoke :write [13 21] INFO [2022-01-10 12:53:22,147] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:22,147] jepsen worker 37 - jepsen.util 127 :invoke :write [12 25] INFO [2022-01-10 12:53:22,148] jepsen worker 70 - jepsen.util 160 :ok :read [23 14] INFO [2022-01-10 12:53:22,149] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:22,149] jepsen worker 52 - jepsen.util 52 :ok :write [17 32] INFO [2022-01-10 12:53:22,149] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:22,149] jepsen worker 70 - jepsen.util 160 :ok :read [23 14] INFO [2022-01-10 12:53:22,150] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:22,150] jepsen worker 52 - jepsen.util 52 :ok :read [17 32] INFO [2022-01-10 12:53:22,151] jepsen worker 52 - jepsen.util 52 :invoke :write [17 43] INFO [2022-01-10 12:53:22,151] jepsen worker 70 - jepsen.util 160 :ok :read [23 14] INFO [2022-01-10 12:53:22,151] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:22,152] jepsen worker 50 - jepsen.util 50 :ok :write [16 39] INFO [2022-01-10 12:53:22,152] jepsen worker 70 - jepsen.util 160 :ok :read [23 14] INFO [2022-01-10 12:53:22,153] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:22,153] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:22,154] jepsen worker 70 - jepsen.util 160 :ok :read [23 14] INFO [2022-01-10 12:53:22,154] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:22,154] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:22,155] jepsen worker 13 - jepsen.util 13 :ok :write [4 12] INFO [2022-01-10 12:53:22,155] jepsen worker 50 - jepsen.util 50 :invoke :write [16 36] INFO [2022-01-10 12:53:22,155] jepsen worker 70 - jepsen.util 160 :ok :read [23 14] INFO [2022-01-10 12:53:22,155] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:22,156] jepsen worker 5 - jepsen.util 185 :ok :write [1 1] INFO [2022-01-10 12:53:22,156] jepsen worker 70 - jepsen.util 160 :invoke :write [23 47] INFO [2022-01-10 12:53:22,156] jepsen worker 13 - jepsen.util 13 :ok :read [4 12] INFO [2022-01-10 12:53:22,156] jepsen worker 5 - jepsen.util 185 :invoke :write [1 23] INFO [2022-01-10 12:53:22,157] jepsen worker 13 - jepsen.util 13 :invoke :write [4 49] INFO [2022-01-10 12:53:22,157] jepsen worker 61 - jepsen.util 61 :ok :write [20 48] INFO [2022-01-10 12:53:22,158] jepsen worker 61 - jepsen.util 61 :invoke :write [20 38] INFO [2022-01-10 12:53:22,167] jepsen worker 37 - jepsen.util 127 :ok :write [12 25] INFO [2022-01-10 12:53:22,167] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:22,168] jepsen worker 87 - jepsen.util 87 :ok :write [29 45] INFO [2022-01-10 12:53:22,168] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:22,168] jepsen worker 37 - jepsen.util 127 :ok :read [12 25] INFO [2022-01-10 12:53:22,169] jepsen worker 87 - jepsen.util 87 :ok :read [29 45] INFO [2022-01-10 12:53:22,169] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:22,169] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:22,169] jepsen worker 37 - jepsen.util 127 :ok :read [12 25] INFO [2022-01-10 12:53:22,170] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:22,170] jepsen worker 87 - jepsen.util 87 :ok :read [29 45] INFO [2022-01-10 12:53:22,171] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:22,171] jepsen worker 37 - jepsen.util 127 :ok :read [12 25] INFO [2022-01-10 12:53:22,171] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:22,171] jepsen worker 87 - jepsen.util 87 :ok :read [29 45] INFO [2022-01-10 12:53:22,172] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:22,172] jepsen worker 37 - jepsen.util 127 :ok :read [12 25] INFO [2022-01-10 12:53:22,173] jepsen worker 87 - jepsen.util 87 :ok :read [29 45] INFO [2022-01-10 12:53:22,173] jepsen worker 37 - jepsen.util 127 :invoke :write [12 47] INFO [2022-01-10 12:53:22,173] jepsen worker 87 - jepsen.util 87 :invoke :write [29 35] INFO [2022-01-10 12:53:22,180] jepsen worker 2 - jepsen.util 92 :ok :write [0 45] INFO [2022-01-10 12:53:22,180] jepsen worker 2 - jepsen.util 92 :invoke :write [0 47] INFO [2022-01-10 12:53:22,188] jepsen worker 40 - jepsen.util 130 :ok :write [13 21] INFO [2022-01-10 12:53:22,188] jepsen worker 40 - jepsen.util 130 :invoke :write [13 40] INFO [2022-01-10 12:53:22,252] jepsen worker 57 - jepsen.util 147 :info :write [19 19] :RequestTimeoutException INFO [2022-01-10 12:53:22,253] jepsen worker 57 - jepsen.util 237 :invoke :write [19 26] INFO [2022-01-10 12:53:22,307] jepsen worker 78 - jepsen.util 78 :ok :write [26 19] INFO [2022-01-10 12:53:22,308] jepsen worker 78 - jepsen.util 78 :invoke :write [26 5] INFO [2022-01-10 12:53:22,308] jepsen worker 23 - jepsen.util 23 :ok :write [7 24] INFO [2022-01-10 12:53:22,308] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:22,309] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:22,310] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:22,310] jepsen worker 84 - jepsen.util 84 :ok :write [28 5] INFO [2022-01-10 12:53:22,310] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:22,311] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:22,311] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:22,311] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:22,312] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:22,312] jepsen worker 23 - jepsen.util 23 :ok :read [7 24] INFO [2022-01-10 12:53:22,312] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:22,312] jepsen worker 23 - jepsen.util 23 :invoke :write [7 31] INFO [2022-01-10 12:53:22,313] jepsen worker 84 - jepsen.util 84 :invoke :write [28 3] INFO [2022-01-10 12:53:22,316] jepsen worker 87 - jepsen.util 87 :ok :write [29 35] INFO [2022-01-10 12:53:22,316] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:22,317] jepsen worker 87 - jepsen.util 87 :ok :read [29 35] INFO [2022-01-10 12:53:22,318] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:22,318] jepsen worker 87 - jepsen.util 87 :ok :read [29 35] INFO [2022-01-10 12:53:22,319] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:22,320] jepsen worker 87 - jepsen.util 87 :ok :read [29 35] INFO [2022-01-10 12:53:22,320] jepsen worker 87 - jepsen.util 87 :invoke :write [29 46] INFO [2022-01-10 12:53:22,324] jepsen worker 2 - jepsen.util 92 :ok :write [0 47] INFO [2022-01-10 12:53:22,325] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:22,326] jepsen worker 2 - jepsen.util 92 :ok :read [0 47] INFO [2022-01-10 12:53:22,326] jepsen worker 2 - jepsen.util 92 :invoke :write [0 47] INFO [2022-01-10 12:53:22,329] jepsen worker 5 - jepsen.util 185 :ok :write [1 23] INFO [2022-01-10 12:53:22,329] jepsen worker 40 - jepsen.util 130 :ok :write [13 40] INFO [2022-01-10 12:53:22,329] jepsen worker 40 - jepsen.util 130 :invoke :write [13 39] INFO [2022-01-10 12:53:22,330] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:22,331] jepsen worker 5 - jepsen.util 185 :ok :read [1 23] INFO [2022-01-10 12:53:22,331] jepsen worker 5 - jepsen.util 185 :invoke :write [1 42] INFO [2022-01-10 12:53:22,337] jepsen worker 10 - jepsen.util 100 :info :write [3 35] :RequestTimeoutException INFO [2022-01-10 12:53:22,337] jepsen worker 10 - jepsen.util 190 :invoke :write [3 18] INFO [2022-01-10 12:53:22,430] jepsen worker 81 - jepsen.util 171 :ok :write [27 14] INFO [2022-01-10 12:53:22,431] jepsen worker 81 - jepsen.util 171 :invoke :write [27 33] INFO [2022-01-10 12:53:22,432] jepsen worker 52 - jepsen.util 52 :ok :write [17 43] INFO [2022-01-10 12:53:22,432] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:22,433] jepsen worker 52 - jepsen.util 52 :ok :read [17 43] INFO [2022-01-10 12:53:22,434] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:22,435] jepsen worker 52 - jepsen.util 52 :ok :read [17 43] INFO [2022-01-10 12:53:22,435] jepsen worker 8 - jepsen.util 8 :ok :write [2 16] INFO [2022-01-10 12:53:22,435] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:22,436] jepsen worker 8 - jepsen.util 8 :invoke :write [2 47] INFO [2022-01-10 12:53:22,436] jepsen worker 52 - jepsen.util 52 :ok :read [17 43] INFO [2022-01-10 12:53:22,436] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:22,437] jepsen worker 52 - jepsen.util 52 :ok :read [17 43] INFO [2022-01-10 12:53:22,437] jepsen worker 13 - jepsen.util 13 :ok :write [4 49] INFO [2022-01-10 12:53:22,438] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:22,438] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:22,439] jepsen worker 52 - jepsen.util 52 :ok :read [17 43] INFO [2022-01-10 12:53:22,439] jepsen worker 13 - jepsen.util 13 :ok :read [4 49] INFO [2022-01-10 12:53:22,439] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:22,440] jepsen worker 13 - jepsen.util 13 :invoke :write [4 35] INFO [2022-01-10 12:53:22,440] jepsen worker 52 - jepsen.util 52 :ok :read [17 43] INFO [2022-01-10 12:53:22,441] jepsen worker 52 - jepsen.util 52 :invoke :write [17 21] INFO [2022-01-10 12:53:22,443] jepsen worker 33 - jepsen.util 123 :ok :write [11 22] INFO [2022-01-10 12:53:22,444] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,444] jepsen worker 33 - jepsen.util 123 :ok :read [11 22] INFO [2022-01-10 12:53:22,445] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,446] jepsen worker 33 - jepsen.util 123 :ok :read [11 22] INFO [2022-01-10 12:53:22,446] jepsen worker 33 - jepsen.util 123 :invoke :write [11 41] INFO [2022-01-10 12:53:22,448] jepsen worker 15 - jepsen.util 15 :ok :write [5 47] INFO [2022-01-10 12:53:22,448] jepsen worker 15 - jepsen.util 15 :invoke :write [5 17] INFO [2022-01-10 12:53:22,453] jepsen worker 75 - jepsen.util 165 :ok :write [25 26] INFO [2022-01-10 12:53:22,454] jepsen worker 75 - jepsen.util 165 :invoke :write [25 32] INFO [2022-01-10 12:53:22,459] jepsen worker 66 - jepsen.util 156 :ok :write [22 1] INFO [2022-01-10 12:53:22,460] jepsen worker 66 - jepsen.util 156 :invoke :write [22 42] INFO [2022-01-10 12:53:22,464] jepsen worker 44 - jepsen.util 44 :ok :write [14 7] INFO [2022-01-10 12:53:22,464] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:22,465] jepsen worker 44 - jepsen.util 44 :ok :read [14 7] INFO [2022-01-10 12:53:22,465] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:22,465] jepsen worker 70 - jepsen.util 160 :ok :write [23 47] INFO [2022-01-10 12:53:22,466] jepsen worker 70 - jepsen.util 160 :invoke :write [23 34] INFO [2022-01-10 12:53:22,466] jepsen worker 44 - jepsen.util 44 :ok :read [14 7] INFO [2022-01-10 12:53:22,466] jepsen worker 44 - jepsen.util 44 :invoke :write [14 28] INFO [2022-01-10 12:53:22,471] jepsen worker 50 - jepsen.util 50 :ok :write [16 36] INFO [2022-01-10 12:53:22,472] jepsen worker 50 - jepsen.util 50 :invoke :write [16 41] INFO [2022-01-10 12:53:22,476] jepsen worker 61 - jepsen.util 61 :ok :write [20 38] INFO [2022-01-10 12:53:22,477] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:22,477] jepsen worker 61 - jepsen.util 61 :ok :read [20 38] INFO [2022-01-10 12:53:22,478] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:22,479] jepsen worker 61 - jepsen.util 61 :ok :read [20 38] INFO [2022-01-10 12:53:22,479] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:22,480] jepsen worker 61 - jepsen.util 61 :ok :read [20 38] INFO [2022-01-10 12:53:22,480] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:22,481] jepsen worker 61 - jepsen.util 61 :ok :read [20 38] INFO [2022-01-10 12:53:22,482] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:22,482] jepsen worker 23 - jepsen.util 23 :ok :write [7 31] INFO [2022-01-10 12:53:22,482] jepsen worker 61 - jepsen.util 61 :ok :read [20 38] INFO [2022-01-10 12:53:22,483] jepsen worker 23 - jepsen.util 23 :invoke :write [7 38] INFO [2022-01-10 12:53:22,483] jepsen worker 61 - jepsen.util 61 :invoke :write [20 28] INFO [2022-01-10 12:53:22,485] jepsen worker 87 - jepsen.util 87 :ok :write [29 46] INFO [2022-01-10 12:53:22,485] jepsen worker 37 - jepsen.util 127 :ok :write [12 47] INFO [2022-01-10 12:53:22,486] jepsen worker 87 - jepsen.util 87 :invoke :write [29 49] INFO [2022-01-10 12:53:22,486] jepsen worker 37 - jepsen.util 127 :invoke :write [12 23] INFO [2022-01-10 12:53:22,489] jepsen worker 81 - jepsen.util 171 :ok :write [27 33] INFO [2022-01-10 12:53:22,490] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:22,491] jepsen worker 81 - jepsen.util 171 :ok :read [27 33] INFO [2022-01-10 12:53:22,491] jepsen worker 81 - jepsen.util 171 :invoke :write [27 41] INFO [2022-01-10 12:53:22,491] jepsen worker 8 - jepsen.util 8 :ok :write [2 47] INFO [2022-01-10 12:53:22,492] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:22,493] jepsen worker 8 - jepsen.util 8 :ok :read [2 47] INFO [2022-01-10 12:53:22,494] jepsen worker 8 - jepsen.util 8 :invoke :write [2 18] INFO [2022-01-10 12:53:22,497] jepsen worker 33 - jepsen.util 123 :ok :write [11 41] INFO [2022-01-10 12:53:22,497] jepsen worker 33 - jepsen.util 123 :invoke :write [11 18] INFO [2022-01-10 12:53:22,502] jepsen worker 15 - jepsen.util 15 :ok :write [5 17] INFO [2022-01-10 12:53:22,503] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:22,503] jepsen worker 15 - jepsen.util 15 :ok :read [5 17] INFO [2022-01-10 12:53:22,504] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:22,505] jepsen worker 15 - jepsen.util 15 :ok :read [5 17] INFO [2022-01-10 12:53:22,505] jepsen worker 15 - jepsen.util 15 :invoke :write [5 42] INFO [2022-01-10 12:53:22,518] jepsen worker 50 - jepsen.util 50 :ok :write [16 41] INFO [2022-01-10 12:53:22,519] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:22,519] jepsen worker 50 - jepsen.util 50 :ok :read [16 41] INFO [2022-01-10 12:53:22,520] jepsen worker 50 - jepsen.util 50 :invoke :write [16 28] INFO [2022-01-10 12:53:22,529] jepsen worker 81 - jepsen.util 171 :ok :write [27 41] INFO [2022-01-10 12:53:22,530] jepsen worker 81 - jepsen.util 171 :invoke :write [27 26] INFO [2022-01-10 12:53:22,535] jepsen worker 8 - jepsen.util 8 :ok :write [2 18] INFO [2022-01-10 12:53:22,535] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:22,536] jepsen worker 8 - jepsen.util 8 :ok :read [2 18] INFO [2022-01-10 12:53:22,536] jepsen worker 8 - jepsen.util 8 :invoke :write [2 12] INFO [2022-01-10 12:53:22,540] jepsen worker 33 - jepsen.util 123 :ok :write [11 18] INFO [2022-01-10 12:53:22,541] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,542] jepsen worker 33 - jepsen.util 123 :ok :read [11 18] INFO [2022-01-10 12:53:22,542] jepsen worker 33 - jepsen.util 123 :invoke :write [11 6] INFO [2022-01-10 12:53:22,545] jepsen worker 15 - jepsen.util 15 :ok :write [5 42] INFO [2022-01-10 12:53:22,545] jepsen worker 15 - jepsen.util 15 :invoke :write [5 14] INFO [2022-01-10 12:53:22,550] jepsen worker 50 - jepsen.util 50 :ok :write [16 28] INFO [2022-01-10 12:53:22,550] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:22,551] jepsen worker 50 - jepsen.util 50 :ok :read [16 28] INFO [2022-01-10 12:53:22,552] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:22,552] jepsen worker 50 - jepsen.util 50 :ok :read [16 28] INFO [2022-01-10 12:53:22,553] jepsen worker 50 - jepsen.util 50 :invoke :write [16 45] INFO [2022-01-10 12:53:22,554] jepsen worker 81 - jepsen.util 171 :ok :write [27 26] INFO [2022-01-10 12:53:22,555] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:22,555] jepsen worker 81 - jepsen.util 171 :ok :read [27 26] INFO [2022-01-10 12:53:22,556] jepsen worker 81 - jepsen.util 171 :invoke :write [27 6] INFO [2022-01-10 12:53:22,558] jepsen worker 8 - jepsen.util 8 :ok :write [2 12] INFO [2022-01-10 12:53:22,558] jepsen worker 8 - jepsen.util 8 :invoke :write [2 19] INFO [2022-01-10 12:53:22,563] jepsen worker 33 - jepsen.util 123 :ok :write [11 6] INFO [2022-01-10 12:53:22,564] jepsen worker 33 - jepsen.util 123 :invoke :write [11 9] INFO [2022-01-10 12:53:22,568] jepsen worker 15 - jepsen.util 15 :ok :write [5 14] INFO [2022-01-10 12:53:22,568] jepsen worker 15 - jepsen.util 15 :invoke :write [5 24] INFO [2022-01-10 12:53:22,643] jepsen worker 45 - jepsen.util 225 :fail :read [15 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:22,644] jepsen worker 45 - jepsen.util 225 :invoke :read [15 nil] INFO [2022-01-10 12:53:22,648] jepsen worker 55 - jepsen.util 235 :fail :read [18 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:22,649] jepsen worker 55 - jepsen.util 235 :invoke :write [18 18] INFO [2022-01-10 12:53:22,715] jepsen worker 50 - jepsen.util 50 :ok :write [16 45] INFO [2022-01-10 12:53:22,715] jepsen worker 50 - jepsen.util 50 :invoke :write [16 37] INFO [2022-01-10 12:53:22,719] jepsen worker 81 - jepsen.util 171 :ok :write [27 6] INFO [2022-01-10 12:53:22,719] jepsen worker 81 - jepsen.util 171 :invoke :write [27 40] INFO [2022-01-10 12:53:22,723] jepsen worker 8 - jepsen.util 8 :ok :write [2 19] INFO [2022-01-10 12:53:22,723] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:22,724] jepsen worker 8 - jepsen.util 8 :ok :read [2 19] INFO [2022-01-10 12:53:22,724] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:22,725] jepsen worker 8 - jepsen.util 8 :ok :read [2 19] INFO [2022-01-10 12:53:22,726] jepsen worker 8 - jepsen.util 8 :invoke :write [2 26] INFO [2022-01-10 12:53:22,727] jepsen worker 33 - jepsen.util 123 :ok :write [11 9] INFO [2022-01-10 12:53:22,728] jepsen worker 33 - jepsen.util 123 :invoke :write [11 1] INFO [2022-01-10 12:53:22,732] jepsen worker 15 - jepsen.util 15 :ok :write [5 24] INFO [2022-01-10 12:53:22,733] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:22,733] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 12:53:22,734] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:22,735] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 12:53:22,735] jepsen worker 15 - jepsen.util 15 :invoke :write [5 22] INFO [2022-01-10 12:53:22,736] jepsen worker 68 - jepsen.util 68 :ok :write [22 2] INFO [2022-01-10 12:53:22,737] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:22,738] jepsen worker 68 - jepsen.util 68 :ok :read [22 2] INFO [2022-01-10 12:53:22,738] jepsen worker 68 - jepsen.util 68 :invoke :write [22 2] INFO [2022-01-10 12:53:22,739] jepsen worker 50 - jepsen.util 50 :ok :write [16 37] INFO [2022-01-10 12:53:22,740] jepsen worker 50 - jepsen.util 50 :invoke :write [16 9] INFO [2022-01-10 12:53:22,744] jepsen worker 81 - jepsen.util 171 :ok :write [27 40] INFO [2022-01-10 12:53:22,745] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:22,746] jepsen worker 81 - jepsen.util 171 :ok :read [27 40] INFO [2022-01-10 12:53:22,746] jepsen worker 81 - jepsen.util 171 :invoke :write [27 33] INFO [2022-01-10 12:53:22,749] jepsen worker 8 - jepsen.util 8 :ok :write [2 26] INFO [2022-01-10 12:53:22,750] jepsen worker 8 - jepsen.util 8 :invoke :write [2 0] INFO [2022-01-10 12:53:22,753] jepsen worker 2 - jepsen.util 92 :ok :write [0 47] INFO [2022-01-10 12:53:22,754] jepsen worker 2 - jepsen.util 92 :invoke :write [0 27] INFO [2022-01-10 12:53:22,760] jepsen worker 33 - jepsen.util 123 :ok :write [11 1] INFO [2022-01-10 12:53:22,761] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,761] jepsen worker 33 - jepsen.util 123 :ok :read [11 1] INFO [2022-01-10 12:53:22,762] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,763] jepsen worker 33 - jepsen.util 123 :ok :read [11 1] INFO [2022-01-10 12:53:22,763] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,764] jepsen worker 33 - jepsen.util 123 :ok :read [11 1] INFO [2022-01-10 12:53:22,764] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,765] jepsen worker 33 - jepsen.util 123 :ok :read [11 1] INFO [2022-01-10 12:53:22,765] jepsen worker 33 - jepsen.util 123 :invoke :write [11 2] INFO [2022-01-10 12:53:22,767] jepsen worker 15 - jepsen.util 15 :ok :write [5 22] INFO [2022-01-10 12:53:22,768] jepsen worker 15 - jepsen.util 15 :invoke :write [5 12] INFO [2022-01-10 12:53:22,857] jepsen worker 32 - jepsen.util 212 :fail :read [10 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:22,858] jepsen worker 32 - jepsen.util 212 :invoke :read [10 nil] INFO [2022-01-10 12:53:22,878] jepsen worker 78 - jepsen.util 78 :ok :write [26 5] INFO [2022-01-10 12:53:22,878] jepsen worker 78 - jepsen.util 78 :invoke :write [26 10] INFO [2022-01-10 12:53:22,881] jepsen worker 68 - jepsen.util 68 :ok :write [22 2] INFO [2022-01-10 12:53:22,882] jepsen worker 68 - jepsen.util 68 :invoke :write [22 9] INFO [2022-01-10 12:53:22,882] jepsen worker 40 - jepsen.util 130 :ok :write [13 39] INFO [2022-01-10 12:53:22,883] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:22,883] jepsen worker 40 - jepsen.util 130 :ok :read [13 39] INFO [2022-01-10 12:53:22,884] jepsen worker 40 - jepsen.util 130 :invoke :write [13 34] INFO [2022-01-10 12:53:22,885] jepsen worker 50 - jepsen.util 50 :ok :write [16 9] INFO [2022-01-10 12:53:22,885] jepsen worker 50 - jepsen.util 50 :invoke :write [16 6] INFO [2022-01-10 12:53:22,886] jepsen worker 84 - jepsen.util 84 :ok :write [28 3] INFO [2022-01-10 12:53:22,886] jepsen worker 75 - jepsen.util 165 :ok :write [25 32] INFO [2022-01-10 12:53:22,887] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:22,887] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:22,888] jepsen worker 84 - jepsen.util 84 :ok :read [28 3] INFO [2022-01-10 12:53:22,888] jepsen worker 75 - jepsen.util 165 :ok :read [25 32] INFO [2022-01-10 12:53:22,888] jepsen worker 81 - jepsen.util 171 :ok :write [27 33] INFO [2022-01-10 12:53:22,888] jepsen worker 84 - jepsen.util 84 :invoke :write [28 26] INFO [2022-01-10 12:53:22,889] jepsen worker 75 - jepsen.util 165 :invoke :write [25 37] INFO [2022-01-10 12:53:22,889] jepsen worker 81 - jepsen.util 171 :invoke :write [27 22] INFO [2022-01-10 12:53:22,893] jepsen worker 31 - jepsen.util 211 :fail :read [10 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:22,893] jepsen worker 5 - jepsen.util 185 :ok :write [1 42] INFO [2022-01-10 12:53:22,896] jepsen worker 44 - jepsen.util 44 :ok :write [14 28] INFO [2022-01-10 12:53:22,896] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:22,897] jepsen worker 31 - jepsen.util 211 :invoke :write [10 2] INFO [2022-01-10 12:53:22,897] jepsen worker 8 - jepsen.util 8 :ok :write [2 0] INFO [2022-01-10 12:53:22,897] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:22,898] jepsen worker 5 - jepsen.util 185 :ok :read [1 42] INFO [2022-01-10 12:53:22,898] jepsen worker 52 - jepsen.util 52 :ok :write [17 21] INFO [2022-01-10 12:53:22,898] jepsen worker 8 - jepsen.util 8 :invoke :write [2 45] INFO [2022-01-10 12:53:22,898] jepsen worker 44 - jepsen.util 44 :ok :read [14 28] INFO [2022-01-10 12:53:22,899] jepsen worker 5 - jepsen.util 185 :invoke :write [1 28] INFO [2022-01-10 12:53:22,900] jepsen worker 52 - jepsen.util 52 :invoke :write [17 19] INFO [2022-01-10 12:53:22,900] jepsen worker 44 - jepsen.util 44 :invoke :write [14 17] INFO [2022-01-10 12:53:22,907] jepsen worker 33 - jepsen.util 123 :ok :write [11 2] INFO [2022-01-10 12:53:22,907] jepsen worker 33 - jepsen.util 123 :invoke :write [11 5] INFO [2022-01-10 12:53:22,911] jepsen worker 15 - jepsen.util 15 :ok :write [5 12] INFO [2022-01-10 12:53:22,912] jepsen worker 15 - jepsen.util 15 :invoke :write [5 25] INFO [2022-01-10 12:53:22,919] jepsen worker 68 - jepsen.util 68 :ok :write [22 9] INFO [2022-01-10 12:53:22,920] jepsen worker 68 - jepsen.util 68 :invoke :write [22 5] INFO [2022-01-10 12:53:22,928] jepsen worker 8 - jepsen.util 8 :ok :write [2 45] INFO [2022-01-10 12:53:22,929] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:22,930] jepsen worker 8 - jepsen.util 8 :ok :read [2 45] INFO [2022-01-10 12:53:22,930] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:22,931] jepsen worker 8 - jepsen.util 8 :ok :read [2 45] INFO [2022-01-10 12:53:22,932] jepsen worker 8 - jepsen.util 8 :invoke :write [2 20] INFO [2022-01-10 12:53:22,932] jepsen worker 33 - jepsen.util 123 :ok :write [11 5] INFO [2022-01-10 12:53:22,933] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,933] jepsen worker 33 - jepsen.util 123 :ok :read [11 5] INFO [2022-01-10 12:53:22,934] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,935] jepsen worker 15 - jepsen.util 15 :ok :write [5 25] INFO [2022-01-10 12:53:22,935] jepsen worker 33 - jepsen.util 123 :ok :read [11 5] INFO [2022-01-10 12:53:22,935] jepsen worker 15 - jepsen.util 15 :invoke :write [5 34] INFO [2022-01-10 12:53:22,935] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,936] jepsen worker 33 - jepsen.util 123 :ok :read [11 5] INFO [2022-01-10 12:53:22,937] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,938] jepsen worker 33 - jepsen.util 123 :ok :read [11 5] INFO [2022-01-10 12:53:22,938] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,939] jepsen worker 33 - jepsen.util 123 :ok :read [11 5] INFO [2022-01-10 12:53:22,939] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:22,940] jepsen worker 33 - jepsen.util 123 :ok :read [11 5] INFO [2022-01-10 12:53:22,940] jepsen worker 33 - jepsen.util 123 :invoke :write [11 28] INFO [2022-01-10 12:53:23,067] jepsen worker 61 - jepsen.util 61 :ok :write [20 28] INFO [2022-01-10 12:53:23,067] jepsen worker 50 - jepsen.util 50 :ok :write [16 6] INFO [2022-01-10 12:53:23,068] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:23,068] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,069] jepsen worker 61 - jepsen.util 61 :ok :read [20 28] INFO [2022-01-10 12:53:23,069] jepsen worker 50 - jepsen.util 50 :ok :read [16 6] INFO [2022-01-10 12:53:23,070] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,070] jepsen worker 13 - jepsen.util 13 :ok :write [4 35] INFO [2022-01-10 12:53:23,070] jepsen worker 61 - jepsen.util 61 :ok :read [20 28] INFO [2022-01-10 12:53:23,071] jepsen worker 50 - jepsen.util 50 :invoke :write [16 2] INFO [2022-01-10 12:53:23,071] jepsen worker 13 - jepsen.util 13 :invoke :write [4 2] INFO [2022-01-10 12:53:23,072] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,073] jepsen worker 61 - jepsen.util 61 :ok :read [20 28] INFO [2022-01-10 12:53:23,073] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,073] jepsen worker 81 - jepsen.util 171 :ok :write [27 22] INFO [2022-01-10 12:53:23,074] jepsen worker 61 - jepsen.util 61 :ok :read [20 28] INFO [2022-01-10 12:53:23,074] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:23,075] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,075] jepsen worker 81 - jepsen.util 171 :ok :read [27 22] INFO [2022-01-10 12:53:23,075] jepsen worker 23 - jepsen.util 23 :ok :write [7 38] INFO [2022-01-10 12:53:23,076] jepsen worker 61 - jepsen.util 61 :ok :read [20 28] INFO [2022-01-10 12:53:23,076] jepsen worker 70 - jepsen.util 160 :ok :write [23 34] INFO [2022-01-10 12:53:23,076] jepsen worker 23 - jepsen.util 23 :invoke :write [7 36] INFO [2022-01-10 12:53:23,077] jepsen worker 81 - jepsen.util 171 :invoke :write [27 35] INFO [2022-01-10 12:53:23,078] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:23,078] jepsen worker 68 - jepsen.util 68 :ok :write [22 5] INFO [2022-01-10 12:53:23,078] jepsen worker 61 - jepsen.util 61 :invoke :write [20 40] INFO [2022-01-10 12:53:23,079] jepsen worker 70 - jepsen.util 160 :ok :read [23 34] INFO [2022-01-10 12:53:23,079] jepsen worker 68 - jepsen.util 68 :invoke :write [22 32] INFO [2022-01-10 12:53:23,079] jepsen worker 70 - jepsen.util 160 :invoke :write [23 7] INFO [2022-01-10 12:53:23,080] jepsen worker 87 - jepsen.util 87 :ok :write [29 49] INFO [2022-01-10 12:53:23,080] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,081] jepsen worker 87 - jepsen.util 87 :ok :read [29 49] INFO [2022-01-10 12:53:23,082] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,083] jepsen worker 8 - jepsen.util 8 :ok :write [2 20] INFO [2022-01-10 12:53:23,083] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,084] jepsen worker 87 - jepsen.util 87 :ok :read [29 49] INFO [2022-01-10 12:53:23,084] jepsen worker 36 - jepsen.util 36 :ok :write [12 2] INFO [2022-01-10 12:53:23,084] jepsen worker 8 - jepsen.util 8 :ok :read [2 20] INFO [2022-01-10 12:53:23,085] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,085] jepsen worker 2 - jepsen.util 92 :ok :write [0 27] INFO [2022-01-10 12:53:23,085] jepsen worker 87 - jepsen.util 87 :invoke :write [29 48] INFO [2022-01-10 12:53:23,086] jepsen worker 8 - jepsen.util 8 :ok :read [2 20] INFO [2022-01-10 12:53:23,086] jepsen worker 36 - jepsen.util 36 :invoke :write [12 30] INFO [2022-01-10 12:53:23,086] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,087] jepsen worker 2 - jepsen.util 92 :invoke :write [0 38] INFO [2022-01-10 12:53:23,087] jepsen worker 8 - jepsen.util 8 :ok :read [2 20] INFO [2022-01-10 12:53:23,088] jepsen worker 8 - jepsen.util 8 :invoke :write [2 44] INFO [2022-01-10 12:53:23,088] jepsen worker 78 - jepsen.util 78 :ok :write [26 10] INFO [2022-01-10 12:53:23,089] jepsen worker 78 - jepsen.util 78 :invoke :write [26 10] INFO [2022-01-10 12:53:23,089] jepsen worker 33 - jepsen.util 123 :ok :write [11 28] INFO [2022-01-10 12:53:23,090] jepsen worker 40 - jepsen.util 130 :ok :write [13 34] INFO [2022-01-10 12:53:23,090] jepsen worker 33 - jepsen.util 123 :invoke :write [11 39] INFO [2022-01-10 12:53:23,091] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,092] jepsen worker 40 - jepsen.util 130 :ok :read [13 34] INFO [2022-01-10 12:53:23,093] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,094] jepsen worker 40 - jepsen.util 130 :ok :read [13 34] INFO [2022-01-10 12:53:23,094] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,094] jepsen worker 15 - jepsen.util 15 :ok :write [5 34] INFO [2022-01-10 12:53:23,095] jepsen worker 84 - jepsen.util 84 :ok :write [28 26] INFO [2022-01-10 12:53:23,095] jepsen worker 75 - jepsen.util 165 :ok :write [25 37] INFO [2022-01-10 12:53:23,095] jepsen worker 40 - jepsen.util 130 :ok :read [13 34] INFO [2022-01-10 12:53:23,095] jepsen worker 84 - jepsen.util 84 :invoke :write [28 33] INFO [2022-01-10 12:53:23,096] jepsen worker 75 - jepsen.util 165 :invoke :write [25 13] INFO [2022-01-10 12:53:23,096] jepsen worker 40 - jepsen.util 130 :invoke :write [13 33] INFO [2022-01-10 12:53:23,097] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:23,098] jepsen worker 15 - jepsen.util 15 :ok :read [5 34] INFO [2022-01-10 12:53:23,098] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:23,099] jepsen worker 15 - jepsen.util 15 :ok :read [5 34] INFO [2022-01-10 12:53:23,100] jepsen worker 15 - jepsen.util 15 :invoke :write [5 27] INFO [2022-01-10 12:53:23,100] jepsen worker 44 - jepsen.util 44 :ok :write [14 17] INFO [2022-01-10 12:53:23,101] jepsen worker 44 - jepsen.util 44 :invoke :write [14 14] INFO [2022-01-10 12:53:23,101] jepsen worker 5 - jepsen.util 185 :ok :write [1 28] INFO [2022-01-10 12:53:23,102] jepsen worker 5 - jepsen.util 185 :invoke :write [1 19] INFO [2022-01-10 12:53:23,106] jepsen worker 61 - jepsen.util 61 :ok :write [20 40] INFO [2022-01-10 12:53:23,106] jepsen worker 52 - jepsen.util 52 :ok :write [17 19] INFO [2022-01-10 12:53:23,106] jepsen worker 61 - jepsen.util 61 :invoke :write [20 42] INFO [2022-01-10 12:53:23,107] jepsen worker 52 - jepsen.util 52 :invoke :write [17 29] INFO [2022-01-10 12:53:23,107] jepsen worker 50 - jepsen.util 50 :ok :write [16 2] INFO [2022-01-10 12:53:23,108] jepsen worker 50 - jepsen.util 50 :invoke :write [16 25] INFO [2022-01-10 12:53:23,111] jepsen worker 13 - jepsen.util 13 :ok :write [4 2] INFO [2022-01-10 12:53:23,111] jepsen worker 87 - jepsen.util 87 :ok :write [29 48] INFO [2022-01-10 12:53:23,112] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,113] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,113] jepsen worker 87 - jepsen.util 87 :ok :read [29 48] INFO [2022-01-10 12:53:23,114] jepsen worker 13 - jepsen.util 13 :ok :read [4 2] INFO [2022-01-10 12:53:23,114] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,114] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,115] jepsen worker 87 - jepsen.util 87 :ok :read [29 48] INFO [2022-01-10 12:53:23,115] jepsen worker 13 - jepsen.util 13 :ok :read [4 2] INFO [2022-01-10 12:53:23,115] jepsen worker 87 - jepsen.util 87 :invoke :write [29 22] INFO [2022-01-10 12:53:23,116] jepsen worker 13 - jepsen.util 13 :invoke :write [4 23] INFO [2022-01-10 12:53:23,116] jepsen worker 70 - jepsen.util 160 :ok :write [23 7] INFO [2022-01-10 12:53:23,117] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:23,117] jepsen worker 23 - jepsen.util 23 :ok :write [7 36] INFO [2022-01-10 12:53:23,117] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:23,118] jepsen worker 81 - jepsen.util 171 :ok :write [27 35] INFO [2022-01-10 12:53:23,118] jepsen worker 70 - jepsen.util 160 :ok :read [23 7] INFO [2022-01-10 12:53:23,118] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:23,119] jepsen worker 23 - jepsen.util 23 :ok :read [7 36] INFO [2022-01-10 12:53:23,119] jepsen worker 70 - jepsen.util 160 :invoke :write [23 28] INFO [2022-01-10 12:53:23,119] jepsen worker 81 - jepsen.util 171 :ok :read [27 35] INFO [2022-01-10 12:53:23,119] jepsen worker 23 - jepsen.util 23 :invoke :write [7 45] INFO [2022-01-10 12:53:23,120] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:23,121] jepsen worker 81 - jepsen.util 171 :ok :read [27 35] INFO [2022-01-10 12:53:23,122] jepsen worker 2 - jepsen.util 92 :ok :write [0 38] INFO [2022-01-10 12:53:23,122] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:23,122] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,122] jepsen worker 36 - jepsen.util 36 :ok :write [12 30] INFO [2022-01-10 12:53:23,123] jepsen worker 81 - jepsen.util 171 :ok :read [27 35] INFO [2022-01-10 12:53:23,123] jepsen worker 68 - jepsen.util 68 :ok :write [22 32] INFO [2022-01-10 12:53:23,123] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,123] jepsen worker 2 - jepsen.util 92 :ok :read [0 38] INFO [2022-01-10 12:53:23,124] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:23,124] jepsen worker 2 - jepsen.util 92 :invoke :write [0 32] INFO [2022-01-10 12:53:23,124] jepsen worker 36 - jepsen.util 36 :ok :read [12 30] INFO [2022-01-10 12:53:23,124] jepsen worker 81 - jepsen.util 171 :invoke :write [27 4] INFO [2022-01-10 12:53:23,124] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:53:23,125] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,125] jepsen worker 68 - jepsen.util 68 :invoke :write [22 19] INFO [2022-01-10 12:53:23,125] jepsen worker 36 - jepsen.util 36 :ok :read [12 30] INFO [2022-01-10 12:53:23,126] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,127] jepsen worker 8 - jepsen.util 8 :ok :write [2 44] INFO [2022-01-10 12:53:23,127] jepsen worker 36 - jepsen.util 36 :ok :read [12 30] INFO [2022-01-10 12:53:23,127] jepsen worker 8 - jepsen.util 8 :invoke :write [2 6] INFO [2022-01-10 12:53:23,128] jepsen worker 36 - jepsen.util 36 :invoke :write [12 17] INFO [2022-01-10 12:53:23,128] jepsen worker 40 - jepsen.util 130 :ok :write [13 33] INFO [2022-01-10 12:53:23,129] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,129] jepsen worker 78 - jepsen.util 78 :ok :write [26 10] INFO [2022-01-10 12:53:23,129] jepsen worker 40 - jepsen.util 130 :ok :read [13 33] INFO [2022-01-10 12:53:23,130] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:23,131] jepsen worker 40 - jepsen.util 130 :invoke :write [13 28] INFO [2022-01-10 12:53:23,131] jepsen worker 78 - jepsen.util 78 :ok :read [26 10] INFO [2022-01-10 12:53:23,132] jepsen worker 78 - jepsen.util 78 :invoke :write [26 3] INFO [2022-01-10 12:53:23,136] jepsen worker 33 - jepsen.util 123 :ok :write [11 39] INFO [2022-01-10 12:53:23,137] jepsen worker 33 - jepsen.util 123 :invoke :write [11 46] INFO [2022-01-10 12:53:23,137] jepsen worker 84 - jepsen.util 84 :ok :write [28 33] INFO [2022-01-10 12:53:23,138] jepsen worker 84 - jepsen.util 84 :invoke :write [28 21] INFO [2022-01-10 12:53:23,142] jepsen worker 5 - jepsen.util 185 :ok :write [1 19] INFO [2022-01-10 12:53:23,142] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,143] jepsen worker 5 - jepsen.util 185 :ok :read [1 19] INFO [2022-01-10 12:53:23,143] jepsen worker 75 - jepsen.util 165 :ok :write [25 13] INFO [2022-01-10 12:53:23,144] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,144] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:23,145] jepsen worker 75 - jepsen.util 165 :ok :read [25 13] INFO [2022-01-10 12:53:23,146] jepsen worker 75 - jepsen.util 165 :invoke :write [25 18] INFO [2022-01-10 12:53:23,146] jepsen worker 5 - jepsen.util 185 :ok :read [1 19] INFO [2022-01-10 12:53:23,146] jepsen worker 52 - jepsen.util 52 :ok :write [17 29] INFO [2022-01-10 12:53:23,147] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,147] jepsen worker 5 - jepsen.util 185 :invoke :write [1 7] INFO [2022-01-10 12:53:23,148] jepsen worker 52 - jepsen.util 52 :ok :read [17 29] INFO [2022-01-10 12:53:23,149] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,149] jepsen worker 52 - jepsen.util 52 :ok :read [17 29] INFO [2022-01-10 12:53:23,150] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,150] jepsen worker 15 - jepsen.util 15 :ok :write [5 27] INFO [2022-01-10 12:53:23,151] jepsen worker 15 - jepsen.util 15 :invoke :write [5 42] INFO [2022-01-10 12:53:23,151] jepsen worker 87 - jepsen.util 87 :ok :write [29 22] INFO [2022-01-10 12:53:23,152] jepsen worker 52 - jepsen.util 52 :ok :read [17 29] INFO [2022-01-10 12:53:23,152] jepsen worker 13 - jepsen.util 13 :ok :write [4 23] INFO [2022-01-10 12:53:23,152] jepsen worker 52 - jepsen.util 52 :invoke :write [17 45] INFO [2022-01-10 12:53:23,153] jepsen worker 13 - jepsen.util 13 :invoke :write [4 13] INFO [2022-01-10 12:53:23,153] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,155] jepsen worker 44 - jepsen.util 44 :ok :write [14 14] INFO [2022-01-10 12:53:23,156] jepsen worker 87 - jepsen.util 87 :ok :read [29 22] INFO [2022-01-10 12:53:23,156] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,156] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,157] jepsen worker 44 - jepsen.util 44 :ok :read [14 14] INFO [2022-01-10 12:53:23,157] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,158] jepsen worker 44 - jepsen.util 44 :ok :read [14 14] INFO [2022-01-10 12:53:23,158] jepsen worker 44 - jepsen.util 44 :invoke :write [14 49] INFO [2022-01-10 12:53:23,159] jepsen worker 70 - jepsen.util 160 :ok :write [23 28] INFO [2022-01-10 12:53:23,160] jepsen worker 87 - jepsen.util 87 :ok :read [29 22] INFO [2022-01-10 12:53:23,160] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:23,160] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,161] jepsen worker 70 - jepsen.util 160 :ok :read [23 28] INFO [2022-01-10 12:53:23,161] jepsen worker 87 - jepsen.util 87 :ok :read [29 22] INFO [2022-01-10 12:53:23,161] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:23,162] jepsen worker 61 - jepsen.util 61 :ok :write [20 42] INFO [2022-01-10 12:53:23,162] jepsen worker 87 - jepsen.util 87 :invoke :write [29 22] INFO [2022-01-10 12:53:23,162] jepsen worker 61 - jepsen.util 61 :invoke :write [20 4] INFO [2022-01-10 12:53:23,162] jepsen worker 70 - jepsen.util 160 :ok :read [23 28] INFO [2022-01-10 12:53:23,163] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:23,164] jepsen worker 70 - jepsen.util 160 :ok :read [23 28] INFO [2022-01-10 12:53:23,164] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:23,165] jepsen worker 70 - jepsen.util 160 :ok :read [23 28] INFO [2022-01-10 12:53:23,166] jepsen worker 70 - jepsen.util 160 :invoke :write [23 22] INFO [2022-01-10 12:53:23,167] jepsen worker 2 - jepsen.util 92 :ok :write [0 32] INFO [2022-01-10 12:53:23,167] jepsen worker 2 - jepsen.util 92 :invoke :write [0 7] INFO [2022-01-10 12:53:23,168] jepsen worker 50 - jepsen.util 50 :ok :write [16 25] INFO [2022-01-10 12:53:23,168] jepsen worker 36 - jepsen.util 36 :ok :write [12 17] INFO [2022-01-10 12:53:23,168] jepsen worker 50 - jepsen.util 50 :invoke :write [16 16] INFO [2022-01-10 12:53:23,169] jepsen worker 36 - jepsen.util 36 :invoke :write [12 24] INFO [2022-01-10 12:53:23,173] jepsen worker 23 - jepsen.util 23 :ok :write [7 45] INFO [2022-01-10 12:53:23,174] jepsen worker 78 - jepsen.util 78 :ok :write [26 3] INFO [2022-01-10 12:53:23,174] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:23,174] jepsen worker 40 - jepsen.util 130 :ok :write [13 28] INFO [2022-01-10 12:53:23,174] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:23,174] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,175] jepsen worker 23 - jepsen.util 23 :ok :read [7 45] INFO [2022-01-10 12:53:23,175] jepsen worker 78 - jepsen.util 78 :ok :read [26 3] INFO [2022-01-10 12:53:23,175] jepsen worker 40 - jepsen.util 130 :ok :read [13 28] INFO [2022-01-10 12:53:23,175] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:23,176] jepsen worker 78 - jepsen.util 78 :invoke :write [26 47] INFO [2022-01-10 12:53:23,176] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,177] jepsen worker 40 - jepsen.util 130 :ok :read [13 28] INFO [2022-01-10 12:53:23,177] jepsen worker 23 - jepsen.util 23 :ok :read [7 45] INFO [2022-01-10 12:53:23,177] jepsen worker 23 - jepsen.util 23 :invoke :write [7 47] INFO [2022-01-10 12:53:23,178] jepsen worker 40 - jepsen.util 130 :invoke :write [13 10] INFO [2022-01-10 12:53:23,183] jepsen worker 81 - jepsen.util 171 :ok :write [27 4] INFO [2022-01-10 12:53:23,183] jepsen worker 84 - jepsen.util 84 :ok :write [28 21] INFO [2022-01-10 12:53:23,184] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:23,184] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,184] jepsen worker 81 - jepsen.util 171 :ok :read [27 4] INFO [2022-01-10 12:53:23,185] jepsen worker 84 - jepsen.util 84 :ok :read [28 21] INFO [2022-01-10 12:53:23,185] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:23,186] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,186] jepsen worker 81 - jepsen.util 171 :ok :read [27 4] INFO [2022-01-10 12:53:23,187] jepsen worker 84 - jepsen.util 84 :ok :read [28 21] INFO [2022-01-10 12:53:23,187] jepsen worker 81 - jepsen.util 171 :invoke :write [27 21] INFO [2022-01-10 12:53:23,188] jepsen worker 84 - jepsen.util 84 :invoke :write [28 19] INFO [2022-01-10 12:53:23,188] jepsen worker 68 - jepsen.util 68 :ok :write [22 19] INFO [2022-01-10 12:53:23,189] jepsen worker 5 - jepsen.util 185 :ok :write [1 7] INFO [2022-01-10 12:53:23,189] jepsen worker 87 - jepsen.util 87 :ok :write [29 22] INFO [2022-01-10 12:53:23,189] jepsen worker 68 - jepsen.util 68 :invoke :write [22 22] INFO [2022-01-10 12:53:23,190] jepsen worker 87 - jepsen.util 87 :invoke :write [29 9] INFO [2022-01-10 12:53:23,190] jepsen worker 5 - jepsen.util 185 :invoke :write [1 43] INFO [2022-01-10 12:53:23,191] jepsen worker 8 - jepsen.util 8 :ok :write [2 6] INFO [2022-01-10 12:53:23,191] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,192] jepsen worker 8 - jepsen.util 8 :ok :read [2 6] INFO [2022-01-10 12:53:23,193] jepsen worker 51 - jepsen.util 141 :ok :write [17 14] INFO [2022-01-10 12:53:23,193] jepsen worker 8 - jepsen.util 8 :invoke :write [2 3] INFO [2022-01-10 12:53:23,193] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,194] jepsen worker 51 - jepsen.util 141 :ok :read [17 14] INFO [2022-01-10 12:53:23,195] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,195] jepsen worker 51 - jepsen.util 141 :ok :read [17 14] INFO [2022-01-10 12:53:23,196] jepsen worker 33 - jepsen.util 123 :ok :write [11 46] INFO [2022-01-10 12:53:23,196] jepsen worker 51 - jepsen.util 141 :invoke :write [17 9] INFO [2022-01-10 12:53:23,196] jepsen worker 33 - jepsen.util 123 :invoke :write [11 1] INFO [2022-01-10 12:53:23,200] jepsen worker 13 - jepsen.util 13 :ok :write [4 13] INFO [2022-01-10 12:53:23,200] jepsen worker 2 - jepsen.util 92 :ok :write [0 7] INFO [2022-01-10 12:53:23,200] jepsen worker 75 - jepsen.util 165 :ok :write [25 18] INFO [2022-01-10 12:53:23,201] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,201] jepsen worker 75 - jepsen.util 165 :invoke :write [25 3] INFO [2022-01-10 12:53:23,202] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,202] jepsen worker 13 - jepsen.util 13 :ok :read [4 13] INFO [2022-01-10 12:53:23,203] jepsen worker 13 - jepsen.util 13 :invoke :write [4 4] INFO [2022-01-10 12:53:23,203] jepsen worker 2 - jepsen.util 92 :ok :read [0 7] INFO [2022-01-10 12:53:23,204] jepsen worker 2 - jepsen.util 92 :invoke :write [0 18] INFO [2022-01-10 12:53:23,207] jepsen worker 15 - jepsen.util 15 :ok :write [5 42] INFO [2022-01-10 12:53:23,207] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:23,208] jepsen worker 15 - jepsen.util 15 :ok :read [5 42] INFO [2022-01-10 12:53:23,209] jepsen worker 15 - jepsen.util 15 :invoke :write [5 23] INFO [2022-01-10 12:53:23,213] jepsen worker 44 - jepsen.util 44 :ok :write [14 49] INFO [2022-01-10 12:53:23,213] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,214] jepsen worker 44 - jepsen.util 44 :ok :read [14 49] INFO [2022-01-10 12:53:23,215] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,216] jepsen worker 44 - jepsen.util 44 :ok :read [14 49] INFO [2022-01-10 12:53:23,216] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,217] jepsen worker 44 - jepsen.util 44 :ok :read [14 49] INFO [2022-01-10 12:53:23,217] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,218] jepsen worker 44 - jepsen.util 44 :ok :read [14 49] INFO [2022-01-10 12:53:23,219] jepsen worker 61 - jepsen.util 61 :ok :write [20 4] INFO [2022-01-10 12:53:23,219] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,220] jepsen worker 61 - jepsen.util 61 :invoke :write [20 33] INFO [2022-01-10 12:53:23,220] jepsen worker 44 - jepsen.util 44 :ok :read [14 49] INFO [2022-01-10 12:53:23,220] jepsen worker 44 - jepsen.util 44 :invoke :write [14 34] INFO [2022-01-10 12:53:23,223] jepsen worker 50 - jepsen.util 50 :ok :write [16 16] INFO [2022-01-10 12:53:23,224] jepsen worker 50 - jepsen.util 50 :invoke :write [16 34] INFO [2022-01-10 12:53:23,230] jepsen worker 81 - jepsen.util 171 :ok :write [27 21] INFO [2022-01-10 12:53:23,231] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:23,232] jepsen worker 81 - jepsen.util 171 :ok :read [27 21] INFO [2022-01-10 12:53:23,232] jepsen worker 81 - jepsen.util 171 :invoke :write [27 48] INFO [2022-01-10 12:53:23,234] jepsen worker 68 - jepsen.util 68 :ok :write [22 22] INFO [2022-01-10 12:53:23,235] jepsen worker 68 - jepsen.util 68 :invoke :write [22 4] INFO [2022-01-10 12:53:23,238] jepsen worker 8 - jepsen.util 8 :ok :write [2 3] INFO [2022-01-10 12:53:23,238] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,239] jepsen worker 8 - jepsen.util 8 :ok :read [2 3] INFO [2022-01-10 12:53:23,240] jepsen worker 8 - jepsen.util 8 :invoke :write [2 34] INFO [2022-01-10 12:53:23,242] jepsen worker 33 - jepsen.util 123 :ok :write [11 1] INFO [2022-01-10 12:53:23,242] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,243] jepsen worker 33 - jepsen.util 123 :ok :read [11 1] INFO [2022-01-10 12:53:23,244] jepsen worker 33 - jepsen.util 123 :invoke :write [11 12] INFO [2022-01-10 12:53:23,295] jepsen worker 70 - jepsen.util 160 :ok :write [23 22] INFO [2022-01-10 12:53:23,295] jepsen worker 23 - jepsen.util 23 :ok :write [7 47] INFO [2022-01-10 12:53:23,296] jepsen worker 70 - jepsen.util 160 :invoke :write [23 42] INFO [2022-01-10 12:53:23,296] jepsen worker 23 - jepsen.util 23 :invoke :write [7 17] INFO [2022-01-10 12:53:23,301] jepsen worker 40 - jepsen.util 130 :ok :write [13 10] INFO [2022-01-10 12:53:23,301] jepsen worker 36 - jepsen.util 36 :ok :write [12 24] INFO [2022-01-10 12:53:23,301] jepsen worker 40 - jepsen.util 130 :invoke :write [13 47] INFO [2022-01-10 12:53:23,302] jepsen worker 36 - jepsen.util 36 :invoke :write [12 2] INFO [2022-01-10 12:53:23,305] jepsen worker 87 - jepsen.util 87 :ok :write [29 9] INFO [2022-01-10 12:53:23,306] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,307] jepsen worker 87 - jepsen.util 87 :ok :read [29 9] INFO [2022-01-10 12:53:23,307] jepsen worker 78 - jepsen.util 78 :ok :write [26 47] INFO [2022-01-10 12:53:23,307] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:23,308] jepsen worker 87 - jepsen.util 87 :invoke :write [29 4] INFO [2022-01-10 12:53:23,308] jepsen worker 78 - jepsen.util 78 :ok :read [26 47] INFO [2022-01-10 12:53:23,309] jepsen worker 78 - jepsen.util 78 :invoke :write [26 45] INFO [2022-01-10 12:53:23,313] jepsen worker 84 - jepsen.util 84 :ok :write [28 19] INFO [2022-01-10 12:53:23,314] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,315] jepsen worker 84 - jepsen.util 84 :ok :read [28 19] INFO [2022-01-10 12:53:23,315] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,316] jepsen worker 84 - jepsen.util 84 :ok :read [28 19] INFO [2022-01-10 12:53:23,316] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,317] jepsen worker 84 - jepsen.util 84 :ok :read [28 19] INFO [2022-01-10 12:53:23,318] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,318] jepsen worker 84 - jepsen.util 84 :ok :read [28 19] INFO [2022-01-10 12:53:23,319] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,320] jepsen worker 84 - jepsen.util 84 :ok :read [28 19] INFO [2022-01-10 12:53:23,320] jepsen worker 84 - jepsen.util 84 :invoke :write [28 37] INFO [2022-01-10 12:53:23,332] jepsen worker 2 - jepsen.util 92 :ok :write [0 18] INFO [2022-01-10 12:53:23,332] jepsen worker 2 - jepsen.util 92 :invoke :write [0 21] INFO [2022-01-10 12:53:23,337] jepsen worker 15 - jepsen.util 15 :ok :write [5 23] INFO [2022-01-10 12:53:23,338] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:23,338] jepsen worker 15 - jepsen.util 15 :ok :read [5 23] INFO [2022-01-10 12:53:23,339] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:23,340] jepsen worker 15 - jepsen.util 15 :ok :read [5 23] INFO [2022-01-10 12:53:23,340] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:23,341] jepsen worker 15 - jepsen.util 15 :ok :read [5 23] INFO [2022-01-10 12:53:23,341] jepsen worker 15 - jepsen.util 15 :invoke :write [5 27] INFO [2022-01-10 12:53:23,343] jepsen worker 50 - jepsen.util 50 :ok :write [16 34] INFO [2022-01-10 12:53:23,344] jepsen worker 50 - jepsen.util 50 :invoke :write [16 14] INFO [2022-01-10 12:53:23,347] jepsen worker 81 - jepsen.util 171 :ok :write [27 48] INFO [2022-01-10 12:53:23,348] jepsen worker 81 - jepsen.util 171 :invoke :write [27 19] INFO [2022-01-10 12:53:23,352] jepsen worker 68 - jepsen.util 68 :ok :write [22 4] INFO [2022-01-10 12:53:23,353] jepsen worker 68 - jepsen.util 68 :invoke :write [22 16] INFO [2022-01-10 12:53:23,356] jepsen worker 8 - jepsen.util 8 :ok :write [2 34] INFO [2022-01-10 12:53:23,357] jepsen worker 8 - jepsen.util 8 :invoke :write [2 46] INFO [2022-01-10 12:53:23,360] jepsen worker 33 - jepsen.util 123 :ok :write [11 12] INFO [2022-01-10 12:53:23,361] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,362] jepsen worker 33 - jepsen.util 123 :ok :read [11 12] INFO [2022-01-10 12:53:23,362] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,363] jepsen worker 33 - jepsen.util 123 :ok :read [11 12] INFO [2022-01-10 12:53:23,363] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,364] jepsen worker 33 - jepsen.util 123 :ok :read [11 12] INFO [2022-01-10 12:53:23,364] jepsen worker 33 - jepsen.util 123 :invoke :write [11 3] INFO [2022-01-10 12:53:23,371] jepsen worker 15 - jepsen.util 15 :ok :write [5 27] INFO [2022-01-10 12:53:23,372] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:23,373] jepsen worker 15 - jepsen.util 15 :ok :read [5 27] INFO [2022-01-10 12:53:23,373] jepsen worker 15 - jepsen.util 15 :invoke :write [5 29] INFO [2022-01-10 12:53:23,376] jepsen worker 50 - jepsen.util 50 :ok :write [16 14] INFO [2022-01-10 12:53:23,376] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:23,377] jepsen worker 50 - jepsen.util 50 :ok :read [16 14] INFO [2022-01-10 12:53:23,377] jepsen worker 50 - jepsen.util 50 :invoke :write [16 22] INFO [2022-01-10 12:53:23,380] jepsen worker 81 - jepsen.util 171 :ok :write [27 19] INFO [2022-01-10 12:53:23,380] jepsen worker 81 - jepsen.util 171 :invoke :write [27 45] INFO [2022-01-10 12:53:23,432] jepsen worker 75 - jepsen.util 165 :ok :write [25 3] INFO [2022-01-10 12:53:23,433] jepsen worker 75 - jepsen.util 165 :invoke :write [25 28] INFO [2022-01-10 12:53:23,434] jepsen worker 5 - jepsen.util 185 :ok :write [1 43] INFO [2022-01-10 12:53:23,434] jepsen worker 5 - jepsen.util 185 :invoke :write [1 3] INFO [2022-01-10 12:53:23,438] jepsen worker 44 - jepsen.util 44 :ok :write [14 34] INFO [2022-01-10 12:53:23,438] jepsen worker 44 - jepsen.util 44 :invoke :write [14 16] INFO [2022-01-10 12:53:23,441] jepsen worker 61 - jepsen.util 61 :ok :write [20 33] INFO [2022-01-10 12:53:23,442] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,442] jepsen worker 51 - jepsen.util 141 :ok :write [17 9] INFO [2022-01-10 12:53:23,443] jepsen worker 61 - jepsen.util 61 :ok :read [20 33] INFO [2022-01-10 12:53:23,443] jepsen worker 51 - jepsen.util 141 :invoke :write [17 8] INFO [2022-01-10 12:53:23,443] jepsen worker 61 - jepsen.util 61 :invoke :write [20 6] INFO [2022-01-10 12:53:23,445] jepsen worker 23 - jepsen.util 23 :ok :write [7 17] INFO [2022-01-10 12:53:23,446] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:23,447] jepsen worker 23 - jepsen.util 23 :ok :read [7 17] INFO [2022-01-10 12:53:23,447] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:23,448] jepsen worker 23 - jepsen.util 23 :ok :read [7 17] INFO [2022-01-10 12:53:23,449] jepsen worker 23 - jepsen.util 23 :invoke :write [7 48] INFO [2022-01-10 12:53:23,450] jepsen worker 13 - jepsen.util 13 :ok :write [4 4] INFO [2022-01-10 12:53:23,450] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,451] jepsen worker 13 - jepsen.util 13 :ok :read [4 4] INFO [2022-01-10 12:53:23,451] jepsen worker 40 - jepsen.util 130 :ok :write [13 47] INFO [2022-01-10 12:53:23,452] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,452] jepsen worker 40 - jepsen.util 130 :invoke :write [13 1] INFO [2022-01-10 12:53:23,453] jepsen worker 13 - jepsen.util 13 :ok :read [4 4] INFO [2022-01-10 12:53:23,453] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,454] jepsen worker 13 - jepsen.util 13 :ok :read [4 4] INFO [2022-01-10 12:53:23,455] jepsen worker 13 - jepsen.util 13 :invoke :write [4 28] INFO [2022-01-10 12:53:23,455] jepsen worker 70 - jepsen.util 160 :ok :write [23 42] INFO [2022-01-10 12:53:23,455] jepsen worker 87 - jepsen.util 87 :ok :write [29 4] INFO [2022-01-10 12:53:23,456] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,457] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:23,457] jepsen worker 87 - jepsen.util 87 :ok :read [29 4] INFO [2022-01-10 12:53:23,458] jepsen worker 70 - jepsen.util 160 :ok :read [23 42] INFO [2022-01-10 12:53:23,458] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,458] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:23,458] jepsen worker 87 - jepsen.util 87 :ok :read [29 4] INFO [2022-01-10 12:53:23,459] jepsen worker 70 - jepsen.util 160 :ok :read [23 42] INFO [2022-01-10 12:53:23,459] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,460] jepsen worker 70 - jepsen.util 160 :invoke :write [23 44] INFO [2022-01-10 12:53:23,460] jepsen worker 87 - jepsen.util 87 :ok :read [29 4] INFO [2022-01-10 12:53:23,461] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,462] jepsen worker 87 - jepsen.util 87 :ok :read [29 4] INFO [2022-01-10 12:53:23,462] jepsen worker 87 - jepsen.util 87 :invoke :write [29 38] INFO [2022-01-10 12:53:23,463] jepsen worker 2 - jepsen.util 92 :ok :write [0 21] INFO [2022-01-10 12:53:23,464] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,464] jepsen worker 36 - jepsen.util 36 :ok :write [12 2] INFO [2022-01-10 12:53:23,465] jepsen worker 2 - jepsen.util 92 :ok :read [0 21] INFO [2022-01-10 12:53:23,465] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,466] jepsen worker 36 - jepsen.util 36 :ok :read [12 2] INFO [2022-01-10 12:53:23,466] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,467] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,467] jepsen worker 2 - jepsen.util 92 :ok :read [0 21] INFO [2022-01-10 12:53:23,468] jepsen worker 36 - jepsen.util 36 :ok :read [12 2] INFO [2022-01-10 12:53:23,468] jepsen worker 2 - jepsen.util 92 :invoke :write [0 45] INFO [2022-01-10 12:53:23,468] jepsen worker 36 - jepsen.util 36 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,469] jepsen worker 36 - jepsen.util 36 :ok :read [12 2] INFO [2022-01-10 12:53:23,470] jepsen worker 36 - jepsen.util 36 :invoke :write [12 20] INFO [2022-01-10 12:53:23,472] jepsen worker 78 - jepsen.util 78 :ok :write [26 45] INFO [2022-01-10 12:53:23,473] jepsen worker 78 - jepsen.util 78 :invoke :write [26 28] INFO [2022-01-10 12:53:23,482] jepsen worker 84 - jepsen.util 84 :ok :write [28 37] INFO [2022-01-10 12:53:23,483] jepsen worker 68 - jepsen.util 68 :ok :write [22 16] INFO [2022-01-10 12:53:23,483] jepsen worker 84 - jepsen.util 84 :invoke :write [28 5] INFO [2022-01-10 12:53:23,484] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:23,485] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:23,486] jepsen worker 5 - jepsen.util 185 :ok :write [1 3] INFO [2022-01-10 12:53:23,486] jepsen worker 68 - jepsen.util 68 :invoke :write [22 21] INFO [2022-01-10 12:53:23,487] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,487] jepsen worker 8 - jepsen.util 8 :ok :write [2 46] INFO [2022-01-10 12:53:23,488] jepsen worker 5 - jepsen.util 185 :ok :read [1 3] INFO [2022-01-10 12:53:23,489] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,489] jepsen worker 5 - jepsen.util 185 :invoke :write [1 31] INFO [2022-01-10 12:53:23,490] jepsen worker 8 - jepsen.util 8 :ok :read [2 46] INFO [2022-01-10 12:53:23,490] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,491] jepsen worker 8 - jepsen.util 8 :ok :read [2 46] INFO [2022-01-10 12:53:23,492] jepsen worker 8 - jepsen.util 8 :invoke :write [2 21] INFO [2022-01-10 12:53:23,493] jepsen worker 51 - jepsen.util 141 :ok :write [17 8] INFO [2022-01-10 12:53:23,493] jepsen worker 33 - jepsen.util 123 :ok :write [11 3] INFO [2022-01-10 12:53:23,494] jepsen worker 51 - jepsen.util 141 :invoke :write [17 26] INFO [2022-01-10 12:53:23,494] jepsen worker 40 - jepsen.util 130 :ok :write [13 1] INFO [2022-01-10 12:53:23,495] jepsen worker 33 - jepsen.util 123 :invoke :write [11 12] INFO [2022-01-10 12:53:23,495] jepsen worker 40 - jepsen.util 130 :invoke :write [13 22] INFO [2022-01-10 12:53:23,498] jepsen worker 87 - jepsen.util 87 :ok :write [29 38] INFO [2022-01-10 12:53:23,498] jepsen worker 13 - jepsen.util 13 :ok :write [4 28] INFO [2022-01-10 12:53:23,499] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,499] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,500] jepsen worker 13 - jepsen.util 13 :ok :read [4 28] INFO [2022-01-10 12:53:23,500] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,500] jepsen worker 87 - jepsen.util 87 :ok :read [29 38] INFO [2022-01-10 12:53:23,501] jepsen worker 13 - jepsen.util 13 :ok :read [4 28] INFO [2022-01-10 12:53:23,501] jepsen worker 87 - jepsen.util 87 :invoke :write [29 48] INFO [2022-01-10 12:53:23,501] jepsen worker 13 - jepsen.util 13 :invoke :write [4 10] INFO [2022-01-10 12:53:23,502] jepsen worker 15 - jepsen.util 15 :ok :write [5 29] INFO [2022-01-10 12:53:23,503] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:23,504] jepsen worker 15 - jepsen.util 15 :ok :read [5 29] INFO [2022-01-10 12:53:23,504] jepsen worker 70 - jepsen.util 160 :ok :write [23 44] INFO [2022-01-10 12:53:23,504] jepsen worker 2 - jepsen.util 92 :ok :write [0 45] INFO [2022-01-10 12:53:23,505] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:23,505] jepsen worker 2 - jepsen.util 92 :invoke :write [0 29] INFO [2022-01-10 12:53:23,505] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:23,505] jepsen worker 15 - jepsen.util 15 :ok :read [5 29] INFO [2022-01-10 12:53:23,506] jepsen worker 15 - jepsen.util 15 :invoke :write [5 46] INFO [2022-01-10 12:53:23,506] jepsen worker 70 - jepsen.util 160 :ok :read [23 44] INFO [2022-01-10 12:53:23,507] jepsen worker 50 - jepsen.util 50 :ok :write [16 22] INFO [2022-01-10 12:53:23,507] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:23,508] jepsen worker 50 - jepsen.util 50 :invoke :write [16 1] INFO [2022-01-10 12:53:23,508] jepsen worker 70 - jepsen.util 160 :ok :read [23 44] INFO [2022-01-10 12:53:23,509] jepsen worker 70 - jepsen.util 160 :invoke :write [23 18] INFO [2022-01-10 12:53:23,511] jepsen worker 37 - jepsen.util 127 :ok :write [12 23] INFO [2022-01-10 12:53:23,511] jepsen worker 81 - jepsen.util 171 :ok :write [27 45] INFO [2022-01-10 12:53:23,512] jepsen worker 40 - jepsen.util 130 :ok :write [13 22] INFO [2022-01-10 12:53:23,512] jepsen worker 81 - jepsen.util 171 :invoke :write [27 28] INFO [2022-01-10 12:53:23,513] jepsen worker 40 - jepsen.util 130 :invoke :write [13 14] INFO [2022-01-10 12:53:23,514] jepsen worker 37 - jepsen.util 127 :invoke :write [12 13] INFO [2022-01-10 12:53:23,516] jepsen worker 75 - jepsen.util 165 :ok :write [25 28] INFO [2022-01-10 12:53:23,516] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:23,516] jepsen worker 78 - jepsen.util 78 :ok :write [26 28] INFO [2022-01-10 12:53:23,517] jepsen worker 78 - jepsen.util 78 :invoke :write [26 43] INFO [2022-01-10 12:53:23,517] jepsen worker 75 - jepsen.util 165 :ok :read [25 28] INFO [2022-01-10 12:53:23,518] jepsen worker 75 - jepsen.util 165 :invoke :write [25 46] INFO [2022-01-10 12:53:23,522] jepsen worker 44 - jepsen.util 44 :ok :write [14 16] INFO [2022-01-10 12:53:23,523] jepsen worker 44 - jepsen.util 44 :invoke :write [14 4] INFO [2022-01-10 12:53:23,525] jepsen worker 84 - jepsen.util 84 :ok :write [28 5] INFO [2022-01-10 12:53:23,525] jepsen worker 87 - jepsen.util 87 :ok :write [29 48] INFO [2022-01-10 12:53:23,526] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,526] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,527] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:23,527] jepsen worker 87 - jepsen.util 87 :ok :read [29 48] INFO [2022-01-10 12:53:23,528] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,528] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:23,528] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,529] jepsen worker 61 - jepsen.util 61 :ok :write [20 6] INFO [2022-01-10 12:53:23,529] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,529] jepsen worker 87 - jepsen.util 87 :ok :read [29 48] INFO [2022-01-10 12:53:23,530] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,530] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:23,530] jepsen worker 5 - jepsen.util 185 :ok :write [1 31] INFO [2022-01-10 12:53:23,530] jepsen worker 87 - jepsen.util 87 :invoke :write [29 39] INFO [2022-01-10 12:53:23,531] jepsen worker 61 - jepsen.util 61 :ok :read [20 6] INFO [2022-01-10 12:53:23,531] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,532] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:23,532] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,533] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,533] jepsen worker 5 - jepsen.util 185 :ok :read [1 31] INFO [2022-01-10 12:53:23,533] jepsen worker 23 - jepsen.util 23 :ok :write [7 48] INFO [2022-01-10 12:53:23,534] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:23,534] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,535] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,535] jepsen worker 61 - jepsen.util 61 :ok :read [20 6] INFO [2022-01-10 12:53:23,535] jepsen worker 23 - jepsen.util 23 :invoke :write [7 40] INFO [2022-01-10 12:53:23,535] jepsen worker 5 - jepsen.util 185 :ok :read [1 31] INFO [2022-01-10 12:53:23,536] jepsen worker 2 - jepsen.util 92 :ok :write [0 29] INFO [2022-01-10 12:53:23,536] jepsen worker 51 - jepsen.util 141 :ok :write [17 26] INFO [2022-01-10 12:53:23,537] jepsen worker 5 - jepsen.util 185 :invoke :write [1 33] INFO [2022-01-10 12:53:23,537] jepsen worker 51 - jepsen.util 141 :invoke :write [17 0] INFO [2022-01-10 12:53:23,539] jepsen worker 61 - jepsen.util 61 :invoke :write [20 10] INFO [2022-01-10 12:53:23,539] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,540] jepsen worker 2 - jepsen.util 92 :invoke :write [0 39] INFO [2022-01-10 12:53:23,540] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:23,540] jepsen worker 68 - jepsen.util 68 :ok :write [22 21] INFO [2022-01-10 12:53:23,541] jepsen worker 84 - jepsen.util 84 :invoke :write [28 19] INFO [2022-01-10 12:53:23,542] jepsen worker 13 - jepsen.util 13 :ok :write [4 10] INFO [2022-01-10 12:53:23,542] jepsen worker 68 - jepsen.util 68 :invoke :write [22 36] INFO [2022-01-10 12:53:23,543] jepsen worker 13 - jepsen.util 13 :invoke :write [4 48] INFO [2022-01-10 12:53:23,546] jepsen worker 8 - jepsen.util 8 :ok :write [2 21] INFO [2022-01-10 12:53:23,547] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,548] jepsen worker 8 - jepsen.util 8 :ok :read [2 21] INFO [2022-01-10 12:53:23,549] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,549] jepsen worker 8 - jepsen.util 8 :ok :read [2 21] INFO [2022-01-10 12:53:23,550] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,550] jepsen worker 70 - jepsen.util 160 :ok :write [23 18] INFO [2022-01-10 12:53:23,551] jepsen worker 8 - jepsen.util 8 :ok :read [2 21] INFO [2022-01-10 12:53:23,551] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:23,551] jepsen worker 40 - jepsen.util 130 :ok :write [13 14] INFO [2022-01-10 12:53:23,552] jepsen worker 70 - jepsen.util 160 :ok :read [23 18] INFO [2022-01-10 12:53:23,552] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,553] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:23,553] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:23,553] jepsen worker 33 - jepsen.util 123 :ok :write [11 12] INFO [2022-01-10 12:53:23,554] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,554] jepsen worker 70 - jepsen.util 160 :ok :read [23 18] INFO [2022-01-10 12:53:23,554] jepsen worker 8 - jepsen.util 8 :ok :read [2 21] INFO [2022-01-10 12:53:23,555] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,555] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,556] jepsen worker 33 - jepsen.util 123 :ok :read [11 12] INFO [2022-01-10 12:53:23,556] jepsen worker 70 - jepsen.util 160 :invoke :write [23 10] INFO [2022-01-10 12:53:23,556] jepsen worker 37 - jepsen.util 127 :ok :write [12 13] INFO [2022-01-10 12:53:23,556] jepsen worker 8 - jepsen.util 8 :ok :read [2 21] INFO [2022-01-10 12:53:23,557] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,558] jepsen worker 87 - jepsen.util 87 :ok :write [29 39] INFO [2022-01-10 12:53:23,558] jepsen worker 37 - jepsen.util 127 :invoke :write [12 33] INFO [2022-01-10 12:53:23,558] jepsen worker 33 - jepsen.util 123 :ok :read [11 12] INFO [2022-01-10 12:53:23,559] jepsen worker 15 - jepsen.util 15 :ok :write [5 46] INFO [2022-01-10 12:53:23,559] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,560] jepsen worker 87 - jepsen.util 87 :ok :read [29 39] INFO [2022-01-10 12:53:23,560] jepsen worker 40 - jepsen.util 130 :invoke :write [13 29] INFO [2022-01-10 12:53:23,561] jepsen worker 8 - jepsen.util 8 :invoke :write [2 30] INFO [2022-01-10 12:53:23,562] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,563] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:23,563] jepsen worker 33 - jepsen.util 123 :ok :read [11 12] INFO [2022-01-10 12:53:23,563] jepsen worker 87 - jepsen.util 87 :invoke :write [29 16] INFO [2022-01-10 12:53:23,563] jepsen worker 15 - jepsen.util 15 :ok :read [5 46] INFO [2022-01-10 12:53:23,564] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,564] jepsen worker 50 - jepsen.util 50 :ok :write [16 1] INFO [2022-01-10 12:53:23,565] jepsen worker 33 - jepsen.util 123 :ok :read [11 12] INFO [2022-01-10 12:53:23,565] jepsen worker 15 - jepsen.util 15 :invoke :write [5 15] INFO [2022-01-10 12:53:23,565] jepsen worker 33 - jepsen.util 123 :invoke :write [11 21] INFO [2022-01-10 12:53:23,566] jepsen worker 78 - jepsen.util 78 :ok :write [26 43] INFO [2022-01-10 12:53:23,566] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:23,568] jepsen worker 62 - jepsen.util 152 :fail :read [20 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:23,568] jepsen worker 50 - jepsen.util 50 :ok :read [16 1] INFO [2022-01-10 12:53:23,568] jepsen worker 78 - jepsen.util 78 :invoke :write [26 42] INFO [2022-01-10 12:53:23,569] jepsen worker 62 - jepsen.util 152 :invoke :write [20 0] INFO [2022-01-10 12:53:23,570] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:23,570] jepsen worker 81 - jepsen.util 171 :ok :write [27 28] INFO [2022-01-10 12:53:23,571] jepsen worker 50 - jepsen.util 50 :ok :read [16 1] INFO [2022-01-10 12:53:23,571] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:23,572] jepsen worker 81 - jepsen.util 171 :ok :read [27 28] INFO [2022-01-10 12:53:23,572] jepsen worker 50 - jepsen.util 50 :invoke :write [16 49] INFO [2022-01-10 12:53:23,573] jepsen worker 81 - jepsen.util 171 :invoke :write [27 1] INFO [2022-01-10 12:53:23,575] jepsen worker 84 - jepsen.util 84 :ok :write [28 19] INFO [2022-01-10 12:53:23,575] jepsen worker 2 - jepsen.util 92 :ok :write [0 39] INFO [2022-01-10 12:53:23,576] jepsen worker 84 - jepsen.util 84 :invoke :write [28 42] INFO [2022-01-10 12:53:23,576] jepsen worker 75 - jepsen.util 165 :ok :write [25 46] INFO [2022-01-10 12:53:23,576] jepsen worker 2 - jepsen.util 92 :invoke :write [0 20] INFO [2022-01-10 12:53:23,577] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:23,578] jepsen worker 75 - jepsen.util 165 :ok :read [25 46] INFO [2022-01-10 12:53:23,578] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:23,579] jepsen worker 75 - jepsen.util 165 :ok :read [25 46] INFO [2022-01-10 12:53:23,579] jepsen worker 5 - jepsen.util 185 :ok :write [1 33] INFO [2022-01-10 12:53:23,579] jepsen worker 75 - jepsen.util 165 :invoke :write [25 30] INFO [2022-01-10 12:53:23,580] jepsen worker 5 - jepsen.util 185 :invoke :write [1 3] INFO [2022-01-10 12:53:23,583] jepsen worker 44 - jepsen.util 44 :ok :write [14 4] INFO [2022-01-10 12:53:23,583] jepsen worker 40 - jepsen.util 130 :ok :write [13 29] INFO [2022-01-10 12:53:23,583] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,584] jepsen worker 40 - jepsen.util 130 :invoke :write [13 17] INFO [2022-01-10 12:53:23,584] jepsen worker 44 - jepsen.util 44 :ok :read [14 4] INFO [2022-01-10 12:53:23,585] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,586] jepsen worker 44 - jepsen.util 44 :ok :read [14 4] INFO [2022-01-10 12:53:23,587] jepsen worker 60 - jepsen.util 150 :ok :write [20 11] INFO [2022-01-10 12:53:23,587] jepsen worker 51 - jepsen.util 141 :ok :write [17 0] INFO [2022-01-10 12:53:23,587] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,588] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,588] jepsen worker 60 - jepsen.util 150 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,588] jepsen worker 44 - jepsen.util 44 :ok :read [14 4] INFO [2022-01-10 12:53:23,589] jepsen worker 51 - jepsen.util 141 :ok :read [17 0] INFO [2022-01-10 12:53:23,589] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,589] jepsen worker 60 - jepsen.util 150 :ok :read [20 11] INFO [2022-01-10 12:53:23,590] jepsen worker 51 - jepsen.util 141 :invoke :write [17 2] INFO [2022-01-10 12:53:23,590] jepsen worker 44 - jepsen.util 44 :ok :read [14 4] INFO [2022-01-10 12:53:23,590] jepsen worker 60 - jepsen.util 150 :invoke :write [20 29] INFO [2022-01-10 12:53:23,591] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,593] jepsen worker 23 - jepsen.util 23 :ok :write [7 40] INFO [2022-01-10 12:53:23,593] jepsen worker 44 - jepsen.util 44 :ok :read [14 4] INFO [2022-01-10 12:53:23,593] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:23,593] jepsen worker 13 - jepsen.util 13 :ok :write [4 48] INFO [2022-01-10 12:53:23,593] jepsen worker 44 - jepsen.util 44 :invoke :write [14 41] INFO [2022-01-10 12:53:23,594] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,594] jepsen worker 23 - jepsen.util 23 :ok :read [7 40] INFO [2022-01-10 12:53:23,595] jepsen worker 13 - jepsen.util 13 :ok :read [4 48] INFO [2022-01-10 12:53:23,595] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:23,595] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,596] jepsen worker 23 - jepsen.util 23 :ok :read [7 40] INFO [2022-01-10 12:53:23,596] jepsen worker 13 - jepsen.util 13 :ok :read [4 48] INFO [2022-01-10 12:53:23,597] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:23,597] jepsen worker 13 - jepsen.util 13 :invoke :write [4 44] INFO [2022-01-10 12:53:23,597] jepsen worker 23 - jepsen.util 23 :ok :read [7 40] INFO [2022-01-10 12:53:23,598] jepsen worker 68 - jepsen.util 68 :ok :write [22 36] INFO [2022-01-10 12:53:23,598] jepsen worker 87 - jepsen.util 87 :ok :write [29 16] INFO [2022-01-10 12:53:23,598] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:23,599] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:23,599] jepsen worker 23 - jepsen.util 23 :ok :read [7 40] INFO [2022-01-10 12:53:23,599] jepsen worker 87 - jepsen.util 87 :invoke :write [29 17] INFO [2022-01-10 12:53:23,600] jepsen worker 68 - jepsen.util 68 :ok :read [22 36] INFO [2022-01-10 12:53:23,600] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:23,600] jepsen worker 68 - jepsen.util 68 :invoke :write [22 14] INFO [2022-01-10 12:53:23,601] jepsen worker 23 - jepsen.util 23 :ok :read [7 40] INFO [2022-01-10 12:53:23,602] jepsen worker 23 - jepsen.util 23 :invoke :write [7 37] INFO [2022-01-10 12:53:23,603] jepsen worker 8 - jepsen.util 8 :ok :write [2 30] INFO [2022-01-10 12:53:23,603] jepsen worker 70 - jepsen.util 160 :ok :write [23 10] INFO [2022-01-10 12:53:23,603] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,604] jepsen worker 70 - jepsen.util 160 :invoke :write [23 21] INFO [2022-01-10 12:53:23,605] jepsen worker 8 - jepsen.util 8 :ok :read [2 30] INFO [2022-01-10 12:53:23,605] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:23,606] jepsen worker 8 - jepsen.util 8 :ok :read [2 30] INFO [2022-01-10 12:53:23,607] jepsen worker 8 - jepsen.util 8 :invoke :write [2 25] INFO [2022-01-10 12:53:23,608] jepsen worker 37 - jepsen.util 127 :ok :write [12 33] INFO [2022-01-10 12:53:23,608] jepsen worker 33 - jepsen.util 123 :ok :write [11 21] INFO [2022-01-10 12:53:23,609] jepsen worker 37 - jepsen.util 127 :invoke :write [12 0] INFO [2022-01-10 12:53:23,609] jepsen worker 33 - jepsen.util 123 :invoke :write [11 16] INFO [2022-01-10 12:53:23,614] jepsen worker 78 - jepsen.util 78 :ok :write [26 42] INFO [2022-01-10 12:53:23,614] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:23,616] jepsen worker 78 - jepsen.util 78 :ok :read [26 42] INFO [2022-01-10 12:53:23,616] jepsen worker 15 - jepsen.util 15 :ok :write [5 15] INFO [2022-01-10 12:53:23,616] jepsen worker 2 - jepsen.util 92 :ok :write [0 20] INFO [2022-01-10 12:53:23,618] jepsen worker 9 - jepsen.util 189 :fail :read [3 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:23,619] jepsen worker 15 - jepsen.util 15 :invoke :write [5 1] INFO [2022-01-10 12:53:23,619] jepsen worker 9 - jepsen.util 189 :invoke :write [3 39] INFO [2022-01-10 12:53:23,620] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:23,621] jepsen worker 78 - jepsen.util 78 :ok :read [26 42] INFO [2022-01-10 12:53:23,622] jepsen worker 2 - jepsen.util 92 :invoke :write [0 39] INFO [2022-01-10 12:53:23,622] jepsen worker 84 - jepsen.util 84 :ok :write [28 42] INFO [2022-01-10 12:53:23,622] jepsen worker 50 - jepsen.util 50 :ok :write [16 49] INFO [2022-01-10 12:53:23,622] jepsen worker 78 - jepsen.util 78 :invoke :write [26 21] INFO [2022-01-10 12:53:23,623] jepsen worker 84 - jepsen.util 84 :invoke :write [28 45] INFO [2022-01-10 12:53:23,624] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:23,625] jepsen worker 50 - jepsen.util 50 :ok :read [16 49] INFO [2022-01-10 12:53:23,625] jepsen worker 5 - jepsen.util 185 :ok :write [1 3] INFO [2022-01-10 12:53:23,625] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:23,626] jepsen worker 81 - jepsen.util 171 :ok :write [27 1] INFO [2022-01-10 12:53:23,626] jepsen worker 5 - jepsen.util 185 :invoke :write [1 31] INFO [2022-01-10 12:53:23,626] jepsen worker 50 - jepsen.util 50 :ok :read [16 49] INFO [2022-01-10 12:53:23,627] jepsen worker 81 - jepsen.util 171 :invoke :write [27 39] INFO [2022-01-10 12:53:23,627] jepsen worker 50 - jepsen.util 50 :invoke :write [16 23] INFO [2022-01-10 12:53:23,630] jepsen worker 75 - jepsen.util 165 :ok :write [25 30] INFO [2022-01-10 12:53:23,631] jepsen worker 75 - jepsen.util 165 :invoke :write [25 2] INFO [2022-01-10 12:53:23,633] jepsen worker 51 - jepsen.util 141 :ok :write [17 2] INFO [2022-01-10 12:53:23,634] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,634] jepsen worker 44 - jepsen.util 44 :ok :write [14 41] INFO [2022-01-10 12:53:23,635] jepsen worker 51 - jepsen.util 141 :ok :read [17 2] INFO [2022-01-10 12:53:23,635] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,635] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,636] jepsen worker 44 - jepsen.util 44 :ok :read [14 41] INFO [2022-01-10 12:53:23,636] jepsen worker 51 - jepsen.util 141 :ok :read [17 2] INFO [2022-01-10 12:53:23,636] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,637] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,637] jepsen worker 44 - jepsen.util 44 :ok :read [14 41] INFO [2022-01-10 12:53:23,638] jepsen worker 51 - jepsen.util 141 :ok :read [17 2] INFO [2022-01-10 12:53:23,638] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,638] jepsen worker 51 - jepsen.util 141 :invoke :write [17 1] INFO [2022-01-10 12:53:23,639] jepsen worker 44 - jepsen.util 44 :ok :read [14 41] INFO [2022-01-10 12:53:23,640] jepsen worker 60 - jepsen.util 150 :ok :write [20 29] INFO [2022-01-10 12:53:23,640] jepsen worker 44 - jepsen.util 44 :invoke :write [14 37] INFO [2022-01-10 12:53:23,640] jepsen worker 60 - jepsen.util 150 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,640] jepsen worker 13 - jepsen.util 13 :ok :write [4 44] INFO [2022-01-10 12:53:23,641] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,642] jepsen worker 13 - jepsen.util 13 :ok :read [4 44] INFO [2022-01-10 12:53:23,642] jepsen worker 13 - jepsen.util 13 :invoke :write [4 39] INFO [2022-01-10 12:53:23,642] jepsen worker 60 - jepsen.util 150 :ok :read [20 29] INFO [2022-01-10 12:53:23,643] jepsen worker 60 - jepsen.util 150 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,645] jepsen worker 70 - jepsen.util 160 :ok :write [23 21] INFO [2022-01-10 12:53:23,645] jepsen worker 70 - jepsen.util 160 :invoke :write [23 41] INFO [2022-01-10 12:53:23,647] jepsen worker 23 - jepsen.util 23 :ok :write [7 37] INFO [2022-01-10 12:53:23,647] jepsen worker 60 - jepsen.util 150 :ok :read [20 29] INFO [2022-01-10 12:53:23,647] jepsen worker 60 - jepsen.util 150 :invoke :write [20 0] INFO [2022-01-10 12:53:23,648] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:23,650] jepsen worker 23 - jepsen.util 23 :ok :read [7 37] INFO [2022-01-10 12:53:23,650] jepsen worker 23 - jepsen.util 23 :invoke :write [7 17] INFO [2022-01-10 12:53:23,673] jepsen worker 74 - jepsen.util 254 :fail :read [24 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:23,673] jepsen worker 75 - jepsen.util 165 :ok :write [25 2] INFO [2022-01-10 12:53:23,674] jepsen worker 74 - jepsen.util 254 :invoke :read [24 nil] INFO [2022-01-10 12:53:23,674] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:23,675] jepsen worker 75 - jepsen.util 165 :ok :read [25 2] INFO [2022-01-10 12:53:23,675] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:23,676] jepsen worker 75 - jepsen.util 165 :ok :read [25 2] INFO [2022-01-10 12:53:23,676] jepsen worker 75 - jepsen.util 165 :invoke :write [25 34] INFO [2022-01-10 12:53:23,718] jepsen worker 40 - jepsen.util 130 :ok :write [13 17] INFO [2022-01-10 12:53:23,719] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,719] jepsen worker 40 - jepsen.util 130 :ok :read [13 17] INFO [2022-01-10 12:53:23,720] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,721] jepsen worker 40 - jepsen.util 130 :ok :read [13 17] INFO [2022-01-10 12:53:23,721] jepsen worker 40 - jepsen.util 130 :invoke :write [13 6] INFO [2022-01-10 12:53:23,725] jepsen worker 68 - jepsen.util 68 :ok :write [22 14] INFO [2022-01-10 12:53:23,725] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:23,726] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:23,727] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:23,728] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:23,728] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:23,729] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:23,730] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:23,730] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:23,731] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:23,731] jepsen worker 87 - jepsen.util 87 :ok :write [29 17] INFO [2022-01-10 12:53:23,732] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:23,732] jepsen worker 87 - jepsen.util 87 :invoke :write [29 41] INFO [2022-01-10 12:53:23,732] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:23,733] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:23,734] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:23,734] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:23,735] jepsen worker 68 - jepsen.util 68 :invoke :write [22 40] INFO [2022-01-10 12:53:23,735] jepsen worker 8 - jepsen.util 8 :ok :write [2 25] INFO [2022-01-10 12:53:23,736] jepsen worker 8 - jepsen.util 8 :invoke :write [2 20] INFO [2022-01-10 12:53:23,740] jepsen worker 33 - jepsen.util 123 :ok :write [11 16] INFO [2022-01-10 12:53:23,741] jepsen worker 33 - jepsen.util 123 :invoke :write [11 37] INFO [2022-01-10 12:53:23,744] jepsen worker 2 - jepsen.util 92 :ok :write [0 39] INFO [2022-01-10 12:53:23,745] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,745] jepsen worker 2 - jepsen.util 92 :ok :read [0 39] INFO [2022-01-10 12:53:23,746] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,747] jepsen worker 2 - jepsen.util 92 :ok :read [0 39] INFO [2022-01-10 12:53:23,747] jepsen worker 2 - jepsen.util 92 :invoke :write [0 38] INFO [2022-01-10 12:53:23,751] jepsen worker 15 - jepsen.util 15 :ok :write [5 1] INFO [2022-01-10 12:53:23,751] jepsen worker 15 - jepsen.util 15 :invoke :write [5 31] INFO [2022-01-10 12:53:23,755] jepsen worker 50 - jepsen.util 50 :ok :write [16 23] INFO [2022-01-10 12:53:23,756] jepsen worker 50 - jepsen.util 50 :invoke :write [16 5] INFO [2022-01-10 12:53:23,759] jepsen worker 81 - jepsen.util 171 :ok :write [27 39] INFO [2022-01-10 12:53:23,760] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:23,761] jepsen worker 81 - jepsen.util 171 :ok :read [27 39] INFO [2022-01-10 12:53:23,761] jepsen worker 81 - jepsen.util 171 :invoke :write [27 22] INFO [2022-01-10 12:53:23,765] jepsen worker 29 - jepsen.util 209 :fail :read [9 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:23,766] jepsen worker 29 - jepsen.util 209 :invoke :write [9 21] INFO [2022-01-10 12:53:23,767] jepsen worker 68 - jepsen.util 68 :ok :write [22 40] INFO [2022-01-10 12:53:23,768] jepsen worker 37 - jepsen.util 127 :ok :write [12 0] INFO [2022-01-10 12:53:23,768] jepsen worker 68 - jepsen.util 68 :invoke :write [22 17] INFO [2022-01-10 12:53:23,768] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,769] jepsen worker 37 - jepsen.util 127 :ok :read [12 0] INFO [2022-01-10 12:53:23,770] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,771] jepsen worker 37 - jepsen.util 127 :ok :read [12 0] INFO [2022-01-10 12:53:23,771] jepsen worker 61 - jepsen.util 61 :ok :write [20 10] INFO [2022-01-10 12:53:23,771] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,772] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,773] jepsen worker 61 - jepsen.util 61 :ok :read [20 10] INFO [2022-01-10 12:53:23,773] jepsen worker 78 - jepsen.util 78 :ok :write [26 21] INFO [2022-01-10 12:53:23,773] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,773] jepsen worker 37 - jepsen.util 127 :ok :read [12 0] INFO [2022-01-10 12:53:23,774] jepsen worker 78 - jepsen.util 78 :invoke :write [26 17] INFO [2022-01-10 12:53:23,774] jepsen worker 61 - jepsen.util 61 :ok :read [20 10] INFO [2022-01-10 12:53:23,775] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,775] jepsen worker 61 - jepsen.util 61 :invoke :write [20 28] INFO [2022-01-10 12:53:23,776] jepsen worker 37 - jepsen.util 127 :ok :read [12 0] INFO [2022-01-10 12:53:23,777] jepsen worker 37 - jepsen.util 127 :invoke :write [12 35] INFO [2022-01-10 12:53:23,779] jepsen worker 8 - jepsen.util 8 :ok :write [2 20] INFO [2022-01-10 12:53:23,779] jepsen worker 44 - jepsen.util 44 :ok :write [14 37] INFO [2022-01-10 12:53:23,780] jepsen worker 8 - jepsen.util 8 :invoke :write [2 25] INFO [2022-01-10 12:53:23,780] jepsen worker 84 - jepsen.util 84 :ok :write [28 45] INFO [2022-01-10 12:53:23,780] jepsen worker 44 - jepsen.util 44 :invoke :write [14 33] INFO [2022-01-10 12:53:23,781] jepsen worker 84 - jepsen.util 84 :invoke :write [28 20] INFO [2022-01-10 12:53:23,783] jepsen worker 23 - jepsen.util 23 :ok :write [7 17] INFO [2022-01-10 12:53:23,784] jepsen worker 23 - jepsen.util 23 :invoke :write [7 44] INFO [2022-01-10 12:53:23,785] jepsen worker 33 - jepsen.util 123 :ok :write [11 37] INFO [2022-01-10 12:53:23,786] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,786] jepsen worker 5 - jepsen.util 185 :ok :write [1 31] INFO [2022-01-10 12:53:23,786] jepsen worker 33 - jepsen.util 123 :ok :read [11 37] INFO [2022-01-10 12:53:23,787] jepsen worker 5 - jepsen.util 185 :invoke :write [1 9] INFO [2022-01-10 12:53:23,787] jepsen worker 75 - jepsen.util 165 :ok :write [25 34] INFO [2022-01-10 12:53:23,787] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,788] jepsen worker 75 - jepsen.util 165 :invoke :write [25 38] INFO [2022-01-10 12:53:23,788] jepsen worker 33 - jepsen.util 123 :ok :read [11 37] INFO [2022-01-10 12:53:23,789] jepsen worker 33 - jepsen.util 123 :invoke :write [11 2] INFO [2022-01-10 12:53:23,793] jepsen worker 51 - jepsen.util 141 :ok :write [17 1] INFO [2022-01-10 12:53:23,793] jepsen worker 40 - jepsen.util 130 :ok :write [13 6] INFO [2022-01-10 12:53:23,793] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,794] jepsen worker 40 - jepsen.util 130 :invoke :write [13 42] INFO [2022-01-10 12:53:23,794] jepsen worker 51 - jepsen.util 141 :ok :read [17 1] INFO [2022-01-10 12:53:23,794] jepsen worker 15 - jepsen.util 15 :ok :write [5 31] INFO [2022-01-10 12:53:23,795] jepsen worker 51 - jepsen.util 141 :invoke :write [17 39] INFO [2022-01-10 12:53:23,795] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:23,796] jepsen worker 15 - jepsen.util 15 :ok :read [5 31] INFO [2022-01-10 12:53:23,797] jepsen worker 15 - jepsen.util 15 :invoke :write [5 35] INFO [2022-01-10 12:53:23,798] jepsen worker 87 - jepsen.util 87 :ok :write [29 41] INFO [2022-01-10 12:53:23,798] jepsen worker 13 - jepsen.util 13 :ok :write [4 39] INFO [2022-01-10 12:53:23,799] jepsen worker 50 - jepsen.util 50 :ok :write [16 5] INFO [2022-01-10 12:53:23,799] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,800] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:23,800] jepsen worker 13 - jepsen.util 13 :ok :read [4 39] INFO [2022-01-10 12:53:23,800] jepsen worker 87 - jepsen.util 87 :invoke :write [29 41] INFO [2022-01-10 12:53:23,801] jepsen worker 50 - jepsen.util 50 :ok :read [16 5] INFO [2022-01-10 12:53:23,801] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,801] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:23,802] jepsen worker 13 - jepsen.util 13 :ok :read [4 39] INFO [2022-01-10 12:53:23,802] jepsen worker 13 - jepsen.util 13 :invoke :write [4 37] INFO [2022-01-10 12:53:23,802] jepsen worker 50 - jepsen.util 50 :ok :read [16 5] INFO [2022-01-10 12:53:23,803] jepsen worker 50 - jepsen.util 50 :invoke :write [16 47] INFO [2022-01-10 12:53:23,805] jepsen worker 81 - jepsen.util 171 :ok :write [27 22] INFO [2022-01-10 12:53:23,806] jepsen worker 70 - jepsen.util 160 :ok :write [23 41] INFO [2022-01-10 12:53:23,806] jepsen worker 2 - jepsen.util 92 :ok :write [0 38] INFO [2022-01-10 12:53:23,806] jepsen worker 81 - jepsen.util 171 :invoke :write [27 4] INFO [2022-01-10 12:53:23,807] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,807] jepsen worker 70 - jepsen.util 160 :invoke :write [23 46] INFO [2022-01-10 12:53:23,808] jepsen worker 2 - jepsen.util 92 :ok :read [0 38] INFO [2022-01-10 12:53:23,808] jepsen worker 2 - jepsen.util 92 :invoke :write [0 28] INFO [2022-01-10 12:53:23,812] jepsen worker 37 - jepsen.util 127 :ok :write [12 35] INFO [2022-01-10 12:53:23,812] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,813] jepsen worker 37 - jepsen.util 127 :ok :read [12 35] INFO [2022-01-10 12:53:23,814] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,814] jepsen worker 37 - jepsen.util 127 :ok :read [12 35] INFO [2022-01-10 12:53:23,815] jepsen worker 37 - jepsen.util 127 :invoke :write [12 39] INFO [2022-01-10 12:53:23,825] jepsen worker 84 - jepsen.util 84 :ok :write [28 20] INFO [2022-01-10 12:53:23,826] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,827] jepsen worker 84 - jepsen.util 84 :ok :read [28 20] INFO [2022-01-10 12:53:23,828] jepsen worker 84 - jepsen.util 84 :invoke :write [28 45] INFO [2022-01-10 12:53:23,832] jepsen worker 5 - jepsen.util 185 :ok :write [1 9] INFO [2022-01-10 12:53:23,832] jepsen worker 40 - jepsen.util 130 :ok :write [13 42] INFO [2022-01-10 12:53:23,832] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,832] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,833] jepsen worker 5 - jepsen.util 185 :ok :read [1 9] INFO [2022-01-10 12:53:23,833] jepsen worker 40 - jepsen.util 130 :ok :read [13 42] INFO [2022-01-10 12:53:23,834] jepsen worker 5 - jepsen.util 185 :invoke :write [1 41] INFO [2022-01-10 12:53:23,834] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,835] jepsen worker 40 - jepsen.util 130 :ok :read [13 42] INFO [2022-01-10 12:53:23,835] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,836] jepsen worker 40 - jepsen.util 130 :ok :read [13 42] INFO [2022-01-10 12:53:23,837] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,838] jepsen worker 40 - jepsen.util 130 :ok :read [13 42] INFO [2022-01-10 12:53:23,838] jepsen worker 40 - jepsen.util 130 :invoke :write [13 22] INFO [2022-01-10 12:53:23,839] jepsen worker 87 - jepsen.util 87 :ok :write [29 41] INFO [2022-01-10 12:53:23,839] jepsen worker 87 - jepsen.util 87 :invoke :write [29 32] INFO [2022-01-10 12:53:23,849] jepsen worker 37 - jepsen.util 127 :ok :write [12 39] INFO [2022-01-10 12:53:23,849] jepsen worker 37 - jepsen.util 127 :invoke :write [12 2] INFO [2022-01-10 12:53:23,850] jepsen worker 2 - jepsen.util 92 :ok :write [0 28] INFO [2022-01-10 12:53:23,850] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,851] jepsen worker 2 - jepsen.util 92 :ok :read [0 28] INFO [2022-01-10 12:53:23,852] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,853] jepsen worker 2 - jepsen.util 92 :ok :read [0 28] INFO [2022-01-10 12:53:23,853] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,854] jepsen worker 2 - jepsen.util 92 :ok :read [0 28] INFO [2022-01-10 12:53:23,855] jepsen worker 2 - jepsen.util 92 :invoke :write [0 31] INFO [2022-01-10 12:53:23,855] jepsen worker 84 - jepsen.util 84 :ok :write [28 45] INFO [2022-01-10 12:53:23,855] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,856] jepsen worker 40 - jepsen.util 130 :ok :write [13 22] INFO [2022-01-10 12:53:23,856] jepsen worker 84 - jepsen.util 84 :ok :read [28 45] INFO [2022-01-10 12:53:23,856] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,857] jepsen worker 84 - jepsen.util 84 :invoke :write [28 42] INFO [2022-01-10 12:53:23,857] jepsen worker 40 - jepsen.util 130 :ok :read [13 22] INFO [2022-01-10 12:53:23,858] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,858] jepsen worker 5 - jepsen.util 185 :ok :write [1 41] INFO [2022-01-10 12:53:23,859] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,859] jepsen worker 40 - jepsen.util 130 :ok :read [13 22] INFO [2022-01-10 12:53:23,859] jepsen worker 40 - jepsen.util 130 :invoke :write [13 7] INFO [2022-01-10 12:53:23,859] jepsen worker 5 - jepsen.util 185 :ok :read [1 41] INFO [2022-01-10 12:53:23,860] jepsen worker 5 - jepsen.util 185 :invoke :write [1 13] INFO [2022-01-10 12:53:23,860] jepsen worker 87 - jepsen.util 87 :ok :write [29 32] INFO [2022-01-10 12:53:23,861] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,862] jepsen worker 87 - jepsen.util 87 :ok :read [29 32] INFO [2022-01-10 12:53:23,862] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,863] jepsen worker 87 - jepsen.util 87 :ok :read [29 32] INFO [2022-01-10 12:53:23,864] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,865] jepsen worker 87 - jepsen.util 87 :ok :read [29 32] INFO [2022-01-10 12:53:23,865] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,866] jepsen worker 87 - jepsen.util 87 :ok :read [29 32] INFO [2022-01-10 12:53:23,867] jepsen worker 87 - jepsen.util 87 :invoke :write [29 48] INFO [2022-01-10 12:53:23,872] jepsen worker 37 - jepsen.util 127 :ok :write [12 2] INFO [2022-01-10 12:53:23,872] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,873] jepsen worker 2 - jepsen.util 92 :ok :write [0 31] INFO [2022-01-10 12:53:23,873] jepsen worker 37 - jepsen.util 127 :ok :read [12 2] INFO [2022-01-10 12:53:23,873] jepsen worker 2 - jepsen.util 92 :invoke :write [0 39] INFO [2022-01-10 12:53:23,874] jepsen worker 37 - jepsen.util 127 :invoke :write [12 42] INFO [2022-01-10 12:53:23,876] jepsen worker 84 - jepsen.util 84 :ok :write [28 42] INFO [2022-01-10 12:53:23,877] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,878] jepsen worker 84 - jepsen.util 84 :ok :read [28 42] INFO [2022-01-10 12:53:23,878] jepsen worker 84 - jepsen.util 84 :invoke :write [28 0] INFO [2022-01-10 12:53:23,879] jepsen worker 40 - jepsen.util 130 :ok :write [13 7] INFO [2022-01-10 12:53:23,879] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,880] jepsen worker 40 - jepsen.util 130 :ok :read [13 7] INFO [2022-01-10 12:53:23,880] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,881] jepsen worker 40 - jepsen.util 130 :ok :read [13 7] INFO [2022-01-10 12:53:23,881] jepsen worker 40 - jepsen.util 130 :invoke :write [13 24] INFO [2022-01-10 12:53:23,883] jepsen worker 5 - jepsen.util 185 :ok :write [1 13] INFO [2022-01-10 12:53:23,884] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,884] jepsen worker 5 - jepsen.util 185 :ok :read [1 13] INFO [2022-01-10 12:53:23,885] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,885] jepsen worker 87 - jepsen.util 87 :ok :write [29 48] INFO [2022-01-10 12:53:23,886] jepsen worker 87 - jepsen.util 87 :invoke :write [29 49] INFO [2022-01-10 12:53:23,886] jepsen worker 5 - jepsen.util 185 :ok :read [1 13] INFO [2022-01-10 12:53:23,886] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,887] jepsen worker 5 - jepsen.util 185 :ok :read [1 13] INFO [2022-01-10 12:53:23,888] jepsen worker 5 - jepsen.util 185 :invoke :write [1 43] INFO [2022-01-10 12:53:23,889] jepsen worker 37 - jepsen.util 127 :ok :write [12 42] INFO [2022-01-10 12:53:23,890] jepsen worker 37 - jepsen.util 127 :invoke :write [12 25] INFO [2022-01-10 12:53:23,894] jepsen worker 2 - jepsen.util 92 :ok :write [0 39] INFO [2022-01-10 12:53:23,894] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,895] jepsen worker 2 - jepsen.util 92 :ok :read [0 39] INFO [2022-01-10 12:53:23,895] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,896] jepsen worker 84 - jepsen.util 84 :ok :write [28 0] INFO [2022-01-10 12:53:23,896] jepsen worker 2 - jepsen.util 92 :ok :read [0 39] INFO [2022-01-10 12:53:23,897] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,897] jepsen worker 84 - jepsen.util 84 :invoke :write [28 12] INFO [2022-01-10 12:53:23,897] jepsen worker 2 - jepsen.util 92 :ok :read [0 39] INFO [2022-01-10 12:53:23,898] jepsen worker 40 - jepsen.util 130 :ok :write [13 24] INFO [2022-01-10 12:53:23,898] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,899] jepsen worker 2 - jepsen.util 92 :ok :read [0 39] INFO [2022-01-10 12:53:23,899] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:23,900] jepsen worker 40 - jepsen.util 130 :ok :read [13 24] INFO [2022-01-10 12:53:23,900] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,901] jepsen worker 40 - jepsen.util 130 :invoke :write [13 36] INFO [2022-01-10 12:53:23,901] jepsen worker 2 - jepsen.util 92 :ok :read [0 39] INFO [2022-01-10 12:53:23,901] jepsen worker 5 - jepsen.util 185 :ok :write [1 43] INFO [2022-01-10 12:53:23,901] jepsen worker 2 - jepsen.util 92 :invoke :write [0 32] INFO [2022-01-10 12:53:23,902] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,903] jepsen worker 5 - jepsen.util 185 :ok :read [1 43] INFO [2022-01-10 12:53:23,903] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,904] jepsen worker 87 - jepsen.util 87 :ok :write [29 49] INFO [2022-01-10 12:53:23,904] jepsen worker 87 - jepsen.util 87 :invoke :write [29 41] INFO [2022-01-10 12:53:23,904] jepsen worker 5 - jepsen.util 185 :ok :read [1 43] INFO [2022-01-10 12:53:23,905] jepsen worker 5 - jepsen.util 185 :invoke :write [1 43] INFO [2022-01-10 12:53:23,908] jepsen worker 37 - jepsen.util 127 :ok :write [12 25] INFO [2022-01-10 12:53:23,909] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,910] jepsen worker 37 - jepsen.util 127 :ok :read [12 25] INFO [2022-01-10 12:53:23,910] jepsen worker 37 - jepsen.util 127 :invoke :write [12 33] INFO [2022-01-10 12:53:23,912] jepsen worker 84 - jepsen.util 84 :ok :write [28 12] INFO [2022-01-10 12:53:23,913] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,913] jepsen worker 40 - jepsen.util 130 :ok :write [13 36] INFO [2022-01-10 12:53:23,914] jepsen worker 40 - jepsen.util 130 :invoke :write [13 9] INFO [2022-01-10 12:53:23,914] jepsen worker 84 - jepsen.util 84 :ok :read [28 12] INFO [2022-01-10 12:53:23,915] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:23,916] jepsen worker 84 - jepsen.util 84 :ok :read [28 12] INFO [2022-01-10 12:53:23,916] jepsen worker 84 - jepsen.util 84 :invoke :write [28 1] INFO [2022-01-10 12:53:23,918] jepsen worker 5 - jepsen.util 185 :ok :write [1 43] INFO [2022-01-10 12:53:23,919] jepsen worker 87 - jepsen.util 87 :ok :write [29 41] INFO [2022-01-10 12:53:23,919] jepsen worker 5 - jepsen.util 185 :invoke :write [1 19] INFO [2022-01-10 12:53:23,919] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,920] jepsen worker 87 - jepsen.util 87 :ok :read [29 41] INFO [2022-01-10 12:53:23,921] jepsen worker 2 - jepsen.util 92 :ok :write [0 32] INFO [2022-01-10 12:53:23,921] jepsen worker 87 - jepsen.util 87 :invoke :write [29 41] INFO [2022-01-10 12:53:23,922] jepsen worker 2 - jepsen.util 92 :invoke :write [0 6] INFO [2022-01-10 12:53:23,923] jepsen worker 37 - jepsen.util 127 :ok :write [12 33] INFO [2022-01-10 12:53:23,924] jepsen worker 37 - jepsen.util 127 :invoke :write [12 1] INFO [2022-01-10 12:53:23,930] jepsen worker 84 - jepsen.util 84 :ok :write [28 1] INFO [2022-01-10 12:53:23,930] jepsen worker 84 - jepsen.util 84 :invoke :write [28 14] INFO [2022-01-10 12:53:23,931] jepsen worker 40 - jepsen.util 130 :ok :write [13 9] INFO [2022-01-10 12:53:23,932] jepsen worker 40 - jepsen.util 130 :invoke :write [13 0] INFO [2022-01-10 12:53:23,932] jepsen worker 68 - jepsen.util 68 :ok :write [22 17] INFO [2022-01-10 12:53:23,933] jepsen worker 68 - jepsen.util 68 :invoke :write [22 27] INFO [2022-01-10 12:53:23,934] jepsen worker 87 - jepsen.util 87 :ok :write [29 41] INFO [2022-01-10 12:53:23,934] jepsen worker 78 - jepsen.util 78 :ok :write [26 17] INFO [2022-01-10 12:53:23,934] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,935] jepsen worker 61 - jepsen.util 61 :ok :write [20 28] INFO [2022-01-10 12:53:23,935] jepsen worker 78 - jepsen.util 78 :invoke :write [26 10] INFO [2022-01-10 12:53:23,935] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,935] jepsen worker 87 - jepsen.util 87 :ok :read [29 41] INFO [2022-01-10 12:53:23,936] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,936] jepsen worker 61 - jepsen.util 61 :ok :read [20 28] INFO [2022-01-10 12:53:23,936] jepsen worker 5 - jepsen.util 185 :ok :write [1 19] INFO [2022-01-10 12:53:23,936] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:23,937] jepsen worker 87 - jepsen.util 87 :ok :read [29 41] INFO [2022-01-10 12:53:23,937] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,937] jepsen worker 61 - jepsen.util 61 :ok :read [20 28] INFO [2022-01-10 12:53:23,938] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,938] jepsen worker 5 - jepsen.util 185 :ok :read [1 19] INFO [2022-01-10 12:53:23,938] jepsen worker 61 - jepsen.util 61 :invoke :write [20 45] INFO [2022-01-10 12:53:23,939] jepsen worker 87 - jepsen.util 87 :ok :read [29 41] INFO [2022-01-10 12:53:23,939] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,939] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:23,940] jepsen worker 5 - jepsen.util 185 :ok :read [1 19] INFO [2022-01-10 12:53:23,940] jepsen worker 51 - jepsen.util 141 :ok :write [17 39] INFO [2022-01-10 12:53:23,940] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:23,941] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,941] jepsen worker 87 - jepsen.util 87 :ok :read [29 41] INFO [2022-01-10 12:53:23,941] jepsen worker 2 - jepsen.util 92 :ok :write [0 6] INFO [2022-01-10 12:53:23,941] jepsen worker 51 - jepsen.util 141 :ok :read [17 39] INFO [2022-01-10 12:53:23,942] jepsen worker 5 - jepsen.util 185 :ok :read [1 19] INFO [2022-01-10 12:53:23,942] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:23,942] jepsen worker 87 - jepsen.util 87 :invoke :write [29 0] INFO [2022-01-10 12:53:23,942] jepsen worker 2 - jepsen.util 92 :ok :read [0 6] INFO [2022-01-10 12:53:23,943] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,943] jepsen worker 51 - jepsen.util 141 :ok :read [17 39] INFO [2022-01-10 12:53:23,944] jepsen worker 5 - jepsen.util 185 :invoke :write [1 33] INFO [2022-01-10 12:53:23,944] jepsen worker 8 - jepsen.util 8 :ok :write [2 25] INFO [2022-01-10 12:53:23,944] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,945] jepsen worker 8 - jepsen.util 8 :invoke :write [2 15] INFO [2022-01-10 12:53:23,945] jepsen worker 51 - jepsen.util 141 :ok :read [17 39] INFO [2022-01-10 12:53:23,945] jepsen worker 37 - jepsen.util 127 :ok :write [12 1] INFO [2022-01-10 12:53:23,945] jepsen worker 2 - jepsen.util 92 :invoke :write [0 1] INFO [2022-01-10 12:53:23,946] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:23,947] jepsen worker 51 - jepsen.util 141 :invoke :write [17 19] INFO [2022-01-10 12:53:23,948] jepsen worker 37 - jepsen.util 127 :ok :read [12 1] INFO [2022-01-10 12:53:23,948] jepsen worker 13 - jepsen.util 13 :ok :write [4 37] INFO [2022-01-10 12:53:23,949] jepsen worker 37 - jepsen.util 127 :invoke :write [12 0] INFO [2022-01-10 12:53:23,949] jepsen worker 13 - jepsen.util 13 :invoke :write [4 31] INFO [2022-01-10 12:53:23,950] jepsen worker 40 - jepsen.util 130 :ok :write [13 0] INFO [2022-01-10 12:53:23,950] jepsen worker 40 - jepsen.util 130 :invoke :write [13 1] INFO [2022-01-10 12:53:23,951] jepsen worker 44 - jepsen.util 44 :ok :write [14 33] INFO [2022-01-10 12:53:23,951] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:23,952] jepsen worker 44 - jepsen.util 44 :ok :read [14 33] INFO [2022-01-10 12:53:23,952] jepsen worker 44 - jepsen.util 44 :invoke :write [14 23] INFO [2022-01-10 12:53:23,955] jepsen worker 70 - jepsen.util 160 :ok :write [23 46] INFO [2022-01-10 12:53:23,956] jepsen worker 70 - jepsen.util 160 :invoke :write [23 48] INFO [2022-01-10 12:53:23,959] jepsen worker 23 - jepsen.util 23 :ok :write [7 44] INFO [2022-01-10 12:53:23,960] jepsen worker 23 - jepsen.util 23 :invoke :write [7 11] INFO [2022-01-10 12:53:23,961] jepsen worker 78 - jepsen.util 78 :ok :write [26 10] INFO [2022-01-10 12:53:23,961] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:23,962] jepsen worker 78 - jepsen.util 78 :ok :read [26 10] INFO [2022-01-10 12:53:23,963] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:23,964] jepsen worker 78 - jepsen.util 78 :ok :read [26 10] INFO [2022-01-10 12:53:23,964] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:23,965] jepsen worker 78 - jepsen.util 78 :ok :read [26 10] INFO [2022-01-10 12:53:23,966] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:23,967] jepsen worker 78 - jepsen.util 78 :ok :read [26 10] INFO [2022-01-10 12:53:23,967] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:23,967] jepsen worker 33 - jepsen.util 123 :ok :write [11 2] INFO [2022-01-10 12:53:23,968] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,968] jepsen worker 78 - jepsen.util 78 :ok :read [26 10] INFO [2022-01-10 12:53:23,968] jepsen worker 51 - jepsen.util 141 :ok :write [17 19] INFO [2022-01-10 12:53:23,969] jepsen worker 33 - jepsen.util 123 :ok :read [11 2] INFO [2022-01-10 12:53:23,969] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:23,970] jepsen worker 78 - jepsen.util 78 :ok :read [26 10] INFO [2022-01-10 12:53:23,970] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,970] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:23,971] jepsen worker 33 - jepsen.util 123 :ok :read [11 2] INFO [2022-01-10 12:53:23,971] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:23,971] jepsen worker 78 - jepsen.util 78 :ok :read [26 10] INFO [2022-01-10 12:53:23,971] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:23,972] jepsen worker 51 - jepsen.util 141 :ok :read [17 19] INFO [2022-01-10 12:53:23,972] jepsen worker 33 - jepsen.util 123 :ok :read [11 2] INFO [2022-01-10 12:53:23,972] jepsen worker 78 - jepsen.util 78 :invoke :write [26 48] INFO [2022-01-10 12:53:23,973] jepsen worker 51 - jepsen.util 141 :invoke :write [17 9] INFO [2022-01-10 12:53:23,973] jepsen worker 33 - jepsen.util 123 :invoke :write [11 28] INFO [2022-01-10 12:53:23,974] jepsen worker 13 - jepsen.util 13 :ok :write [4 31] INFO [2022-01-10 12:53:23,974] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:23,975] jepsen worker 13 - jepsen.util 13 :ok :read [4 31] INFO [2022-01-10 12:53:23,975] jepsen worker 75 - jepsen.util 165 :ok :write [25 38] INFO [2022-01-10 12:53:23,976] jepsen worker 13 - jepsen.util 13 :invoke :write [4 45] INFO [2022-01-10 12:53:23,976] jepsen worker 75 - jepsen.util 165 :invoke :write [25 5] INFO [2022-01-10 12:53:23,980] jepsen worker 69 - jepsen.util 159 :ok :write [23 28] INFO [2022-01-10 12:53:23,980] jepsen worker 69 - jepsen.util 159 :invoke :write [23 42] INFO [2022-01-10 12:53:23,981] jepsen worker 15 - jepsen.util 15 :ok :write [5 35] INFO [2022-01-10 12:53:23,982] jepsen worker 15 - jepsen.util 15 :invoke :write [5 3] INFO [2022-01-10 12:53:23,987] jepsen worker 50 - jepsen.util 50 :ok :write [16 47] INFO [2022-01-10 12:53:23,988] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:23,988] jepsen worker 50 - jepsen.util 50 :ok :read [16 47] INFO [2022-01-10 12:53:23,989] jepsen worker 50 - jepsen.util 50 :invoke :write [16 3] INFO [2022-01-10 12:53:23,991] jepsen worker 51 - jepsen.util 141 :ok :write [17 9] INFO [2022-01-10 12:53:23,992] jepsen worker 81 - jepsen.util 171 :ok :write [27 4] INFO [2022-01-10 12:53:23,992] jepsen worker 51 - jepsen.util 141 :invoke :write [17 10] INFO [2022-01-10 12:53:23,993] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:23,993] jepsen worker 81 - jepsen.util 171 :ok :read [27 4] INFO [2022-01-10 12:53:23,994] jepsen worker 81 - jepsen.util 171 :invoke :write [27 0] INFO [2022-01-10 12:53:23,996] jepsen worker 68 - jepsen.util 68 :ok :write [22 27] INFO [2022-01-10 12:53:23,997] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:23,997] jepsen worker 68 - jepsen.util 68 :ok :read [22 27] INFO [2022-01-10 12:53:23,998] jepsen worker 68 - jepsen.util 68 :invoke :write [22 27] INFO [2022-01-10 12:53:24,001] jepsen worker 61 - jepsen.util 61 :ok :write [20 45] INFO [2022-01-10 12:53:24,002] jepsen worker 61 - jepsen.util 61 :invoke :write [20 34] INFO [2022-01-10 12:53:24,011] jepsen worker 44 - jepsen.util 44 :ok :write [14 23] INFO [2022-01-10 12:53:24,012] jepsen worker 44 - jepsen.util 44 :invoke :write [14 27] INFO [2022-01-10 12:53:24,015] jepsen worker 23 - jepsen.util 23 :ok :write [7 11] INFO [2022-01-10 12:53:24,015] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:24,016] jepsen worker 23 - jepsen.util 23 :ok :read [7 11] INFO [2022-01-10 12:53:24,016] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:24,017] jepsen worker 23 - jepsen.util 23 :ok :read [7 11] INFO [2022-01-10 12:53:24,017] jepsen worker 23 - jepsen.util 23 :invoke :write [7 20] INFO [2022-01-10 12:53:24,061] jepsen worker 84 - jepsen.util 84 :ok :write [28 14] INFO [2022-01-10 12:53:24,061] jepsen worker 84 - jepsen.util 84 :invoke :write [28 30] INFO [2022-01-10 12:53:24,065] jepsen worker 2 - jepsen.util 92 :ok :write [0 1] INFO [2022-01-10 12:53:24,065] jepsen worker 2 - jepsen.util 92 :invoke :write [0 13] INFO [2022-01-10 12:53:24,066] jepsen worker 5 - jepsen.util 185 :ok :write [1 33] INFO [2022-01-10 12:53:24,066] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,067] jepsen worker 5 - jepsen.util 185 :ok :read [1 33] INFO [2022-01-10 12:53:24,068] jepsen worker 5 - jepsen.util 185 :invoke :write [1 30] INFO [2022-01-10 12:53:24,069] jepsen worker 37 - jepsen.util 127 :ok :write [12 0] INFO [2022-01-10 12:53:24,070] jepsen worker 37 - jepsen.util 127 :invoke :write [12 32] INFO [2022-01-10 12:53:24,071] jepsen worker 8 - jepsen.util 8 :ok :write [2 15] INFO [2022-01-10 12:53:24,071] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,072] jepsen worker 8 - jepsen.util 8 :ok :read [2 15] INFO [2022-01-10 12:53:24,072] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,073] jepsen worker 8 - jepsen.util 8 :ok :read [2 15] INFO [2022-01-10 12:53:24,074] jepsen worker 8 - jepsen.util 8 :invoke :write [2 15] INFO [2022-01-10 12:53:24,075] jepsen worker 87 - jepsen.util 87 :ok :write [29 0] INFO [2022-01-10 12:53:24,076] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,076] jepsen worker 87 - jepsen.util 87 :ok :read [29 0] INFO [2022-01-10 12:53:24,077] jepsen worker 87 - jepsen.util 87 :invoke :write [29 37] INFO [2022-01-10 12:53:24,080] jepsen worker 40 - jepsen.util 130 :ok :write [13 1] INFO [2022-01-10 12:53:24,080] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,081] jepsen worker 40 - jepsen.util 130 :ok :read [13 1] INFO [2022-01-10 12:53:24,082] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,083] jepsen worker 40 - jepsen.util 130 :ok :read [13 1] INFO [2022-01-10 12:53:24,083] jepsen worker 40 - jepsen.util 130 :invoke :write [13 40] INFO [2022-01-10 12:53:24,083] jepsen worker 33 - jepsen.util 123 :ok :write [11 28] INFO [2022-01-10 12:53:24,084] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:24,084] jepsen worker 33 - jepsen.util 123 :ok :read [11 28] INFO [2022-01-10 12:53:24,085] jepsen worker 33 - jepsen.util 123 :invoke :write [11 10] INFO [2022-01-10 12:53:24,088] jepsen worker 15 - jepsen.util 15 :ok :write [5 3] INFO [2022-01-10 12:53:24,089] jepsen worker 15 - jepsen.util 15 :invoke :write [5 25] INFO [2022-01-10 12:53:24,092] jepsen worker 50 - jepsen.util 50 :ok :write [16 3] INFO [2022-01-10 12:53:24,092] jepsen worker 50 - jepsen.util 50 :invoke :write [16 36] INFO [2022-01-10 12:53:24,094] jepsen worker 81 - jepsen.util 171 :ok :write [27 0] INFO [2022-01-10 12:53:24,094] jepsen worker 81 - jepsen.util 171 :invoke :write [27 2] INFO [2022-01-10 12:53:24,098] jepsen worker 68 - jepsen.util 68 :ok :write [22 27] INFO [2022-01-10 12:53:24,098] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:24,099] jepsen worker 68 - jepsen.util 68 :ok :read [22 27] INFO [2022-01-10 12:53:24,099] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:24,100] jepsen worker 68 - jepsen.util 68 :ok :read [22 27] INFO [2022-01-10 12:53:24,101] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:24,102] jepsen worker 68 - jepsen.util 68 :ok :read [22 27] INFO [2022-01-10 12:53:24,102] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:24,103] jepsen worker 68 - jepsen.util 68 :ok :read [22 27] INFO [2022-01-10 12:53:24,104] jepsen worker 68 - jepsen.util 68 :invoke :write [22 41] INFO [2022-01-10 12:53:24,104] jepsen worker 78 - jepsen.util 78 :ok :write [26 48] INFO [2022-01-10 12:53:24,104] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:24,105] jepsen worker 78 - jepsen.util 78 :ok :read [26 48] INFO [2022-01-10 12:53:24,106] jepsen worker 78 - jepsen.util 78 :invoke :write [26 5] INFO [2022-01-10 12:53:24,108] jepsen worker 75 - jepsen.util 165 :ok :write [25 5] INFO [2022-01-10 12:53:24,109] jepsen worker 8 - jepsen.util 8 :ok :write [2 15] INFO [2022-01-10 12:53:24,109] jepsen worker 75 - jepsen.util 165 :invoke :write [25 34] INFO [2022-01-10 12:53:24,109] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,110] jepsen worker 8 - jepsen.util 8 :ok :read [2 15] INFO [2022-01-10 12:53:24,110] jepsen worker 61 - jepsen.util 61 :ok :write [20 34] INFO [2022-01-10 12:53:24,111] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:24,111] jepsen worker 8 - jepsen.util 8 :invoke :write [2 29] INFO [2022-01-10 12:53:24,111] jepsen worker 61 - jepsen.util 61 :ok :read [20 34] INFO [2022-01-10 12:53:24,112] jepsen worker 13 - jepsen.util 13 :ok :write [4 45] INFO [2022-01-10 12:53:24,112] jepsen worker 61 - jepsen.util 61 :invoke :write [20 8] INFO [2022-01-10 12:53:24,113] jepsen worker 13 - jepsen.util 13 :invoke :write [4 14] INFO [2022-01-10 12:53:24,116] jepsen worker 33 - jepsen.util 123 :ok :write [11 10] INFO [2022-01-10 12:53:24,116] jepsen worker 44 - jepsen.util 44 :ok :write [14 27] INFO [2022-01-10 12:53:24,117] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:24,117] jepsen worker 44 - jepsen.util 44 :invoke :write [14 19] INFO [2022-01-10 12:53:24,118] jepsen worker 69 - jepsen.util 159 :ok :write [23 42] INFO [2022-01-10 12:53:24,118] jepsen worker 33 - jepsen.util 123 :ok :read [11 10] INFO [2022-01-10 12:53:24,119] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:24,119] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:24,120] jepsen worker 33 - jepsen.util 123 :ok :read [11 10] INFO [2022-01-10 12:53:24,120] jepsen worker 69 - jepsen.util 159 :ok :read [23 42] INFO [2022-01-10 12:53:24,120] jepsen worker 15 - jepsen.util 15 :ok :write [5 25] INFO [2022-01-10 12:53:24,120] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:24,121] jepsen worker 15 - jepsen.util 15 :invoke :write [5 33] INFO [2022-01-10 12:53:24,121] jepsen worker 33 - jepsen.util 123 :ok :read [11 10] INFO [2022-01-10 12:53:24,121] jepsen worker 69 - jepsen.util 159 :invoke :write [23 48] INFO [2022-01-10 12:53:24,122] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:24,122] jepsen worker 33 - jepsen.util 123 :ok :read [11 10] INFO [2022-01-10 12:53:24,123] jepsen worker 50 - jepsen.util 50 :ok :write [16 36] INFO [2022-01-10 12:53:24,123] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:24,124] jepsen worker 50 - jepsen.util 50 :invoke :write [16 17] INFO [2022-01-10 12:53:24,124] jepsen worker 33 - jepsen.util 123 :ok :read [11 10] INFO [2022-01-10 12:53:24,125] jepsen worker 23 - jepsen.util 23 :ok :write [7 20] INFO [2022-01-10 12:53:24,125] jepsen worker 51 - jepsen.util 141 :ok :write [17 10] INFO [2022-01-10 12:53:24,126] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,127] jepsen worker 51 - jepsen.util 141 :ok :read [17 10] INFO [2022-01-10 12:53:24,127] jepsen worker 33 - jepsen.util 123 :invoke :write [11 11] INFO [2022-01-10 12:53:24,127] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:24,128] jepsen worker 23 - jepsen.util 23 :ok :read [7 20] INFO [2022-01-10 12:53:24,128] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,129] jepsen worker 51 - jepsen.util 141 :ok :read [17 10] INFO [2022-01-10 12:53:24,129] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:24,130] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,130] jepsen worker 23 - jepsen.util 23 :ok :read [7 20] INFO [2022-01-10 12:53:24,131] jepsen worker 51 - jepsen.util 141 :ok :read [17 10] INFO [2022-01-10 12:53:24,131] jepsen worker 2 - jepsen.util 92 :ok :write [0 13] INFO [2022-01-10 12:53:24,131] jepsen worker 23 - jepsen.util 23 :invoke :write [7 10] INFO [2022-01-10 12:53:24,132] jepsen worker 84 - jepsen.util 84 :ok :write [28 30] INFO [2022-01-10 12:53:24,132] jepsen worker 51 - jepsen.util 141 :invoke :write [17 10] INFO [2022-01-10 12:53:24,133] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,134] jepsen worker 87 - jepsen.util 87 :ok :write [29 37] INFO [2022-01-10 12:53:24,134] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,134] jepsen worker 2 - jepsen.util 92 :ok :read [0 13] INFO [2022-01-10 12:53:24,135] jepsen worker 84 - jepsen.util 84 :ok :read [28 30] INFO [2022-01-10 12:53:24,135] jepsen worker 87 - jepsen.util 87 :invoke :write [29 44] INFO [2022-01-10 12:53:24,135] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,136] jepsen worker 84 - jepsen.util 84 :invoke :write [28 12] INFO [2022-01-10 12:53:24,136] jepsen worker 2 - jepsen.util 92 :ok :read [0 13] INFO [2022-01-10 12:53:24,137] jepsen worker 2 - jepsen.util 92 :invoke :write [0 14] INFO [2022-01-10 12:53:24,139] jepsen worker 5 - jepsen.util 185 :ok :write [1 30] INFO [2022-01-10 12:53:24,139] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,140] jepsen worker 5 - jepsen.util 185 :ok :read [1 30] INFO [2022-01-10 12:53:24,140] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,141] jepsen worker 40 - jepsen.util 130 :ok :write [13 40] INFO [2022-01-10 12:53:24,141] jepsen worker 40 - jepsen.util 130 :invoke :write [13 25] INFO [2022-01-10 12:53:24,141] jepsen worker 5 - jepsen.util 185 :ok :read [1 30] INFO [2022-01-10 12:53:24,142] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,142] jepsen worker 5 - jepsen.util 185 :ok :read [1 30] INFO [2022-01-10 12:53:24,143] jepsen worker 5 - jepsen.util 185 :invoke :write [1 39] INFO [2022-01-10 12:53:24,145] jepsen worker 37 - jepsen.util 127 :ok :write [12 32] INFO [2022-01-10 12:53:24,146] jepsen worker 37 - jepsen.util 127 :invoke :write [12 10] WARN [2022-01-10 12:53:24,150] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1033,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":22438,"total_server_duration_us":2786,"encode_duration_us":51,"operation_id":"0x13f3","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10009744},{"operation_name":"upsert","last_dispatch_duration_us":546,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":71843,"total_server_duration_us":2604,"encode_duration_us":31,"operation_id":"0x2214","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10008036},{"operation_name":"upsert","last_dispatch_duration_us":669,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":11529,"total_server_duration_us":1605,"encode_duration_us":20,"operation_id":"0x20f3","timeout_ms":10000,"last_server_duration_us":126,"total_duration_us":10007685},{"operation_name":"upsert","last_dispatch_duration_us":565,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000001/00000000100998FD","last_local_socket":"172.23.104.79:58632","total_dispatch_duration_us":19793,"total_server_duration_us":3194,"encode_duration_us":54,"operation_id":"0x17bd","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10006365},{"operation_name":"upsert","last_dispatch_duration_us":495,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":18728,"total_server_duration_us":2790,"encode_duration_us":41,"operation_id":"0x2007","timeout_ms":10000,"last_server_duration_us":56,"total_duration_us":10002225},{"operation_name":"upsert","last_dispatch_duration_us":817,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":20573,"total_server_duration_us":3149,"encode_duration_us":21,"operation_id":"0x1f56","timeout_ms":10000,"last_server_duration_us":293,"total_duration_us":10002105},{"operation_name":"upsert","last_dispatch_duration_us":580,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000001/000000005D716E8E","last_local_socket":"172.23.104.79:34680","total_dispatch_duration_us":18382,"total_server_duration_us":2818,"encode_duration_us":40,"operation_id":"0x1fbd","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10001291},{"operation_name":"upsert","last_dispatch_duration_us":9014510,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":9616827,"total_server_duration_us":9014337,"encode_duration_us":44,"operation_id":"0x1f78","timeout_ms":10000,"last_server_duration_us":9014029,"total_duration_us":9675533},{"operation_name":"upsert","last_dispatch_duration_us":9015196,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":9018548,"total_server_duration_us":9014170,"encode_duration_us":20,"operation_id":"0x20cc","timeout_ms":10000,"last_server_duration_us":9014029,"total_duration_us":9058377},{"operation_name":"upsert","last_dispatch_duration_us":9014452,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":9014452,"total_server_duration_us":9014029,"encode_duration_us":18,"operation_id":"0x20d0","timeout_ms":10000,"last_server_duration_us":9014029,"total_duration_us":9015076}],"total_count":23}} INFO [2022-01-10 12:53:24,166] jepsen worker 2 - jepsen.util 92 :ok :write [0 14] INFO [2022-01-10 12:53:24,167] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,168] jepsen worker 2 - jepsen.util 92 :ok :read [0 14] INFO [2022-01-10 12:53:24,169] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,169] jepsen worker 87 - jepsen.util 87 :ok :write [29 44] INFO [2022-01-10 12:53:24,169] jepsen worker 2 - jepsen.util 92 :ok :read [0 14] INFO [2022-01-10 12:53:24,170] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,170] jepsen worker 2 - jepsen.util 92 :invoke :write [0 29] INFO [2022-01-10 12:53:24,171] jepsen worker 87 - jepsen.util 87 :ok :read [29 44] INFO [2022-01-10 12:53:24,171] jepsen worker 84 - jepsen.util 84 :ok :write [28 12] INFO [2022-01-10 12:53:24,171] jepsen worker 87 - jepsen.util 87 :invoke :write [29 17] INFO [2022-01-10 12:53:24,172] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,172] jepsen worker 84 - jepsen.util 84 :ok :read [28 12] INFO [2022-01-10 12:53:24,173] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,174] jepsen worker 84 - jepsen.util 84 :ok :read [28 12] INFO [2022-01-10 12:53:24,175] jepsen worker 84 - jepsen.util 84 :invoke :write [28 33] INFO [2022-01-10 12:53:24,176] jepsen worker 5 - jepsen.util 185 :ok :write [1 39] INFO [2022-01-10 12:53:24,177] jepsen worker 40 - jepsen.util 130 :ok :write [13 25] INFO [2022-01-10 12:53:24,177] jepsen worker 5 - jepsen.util 185 :invoke :write [1 33] INFO [2022-01-10 12:53:24,177] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,178] jepsen worker 40 - jepsen.util 130 :ok :read [13 25] INFO [2022-01-10 12:53:24,179] jepsen worker 40 - jepsen.util 130 :invoke :write [13 37] INFO [2022-01-10 12:53:24,181] jepsen worker 2 - jepsen.util 92 :ok :write [0 29] INFO [2022-01-10 12:53:24,182] jepsen worker 37 - jepsen.util 127 :ok :write [12 10] INFO [2022-01-10 12:53:24,182] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,183] jepsen worker 37 - jepsen.util 127 :invoke :write [12 14] INFO [2022-01-10 12:53:24,183] jepsen worker 2 - jepsen.util 92 :ok :read [0 29] INFO [2022-01-10 12:53:24,184] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,185] jepsen worker 2 - jepsen.util 92 :ok :read [0 29] INFO [2022-01-10 12:53:24,186] jepsen worker 2 - jepsen.util 92 :invoke :write [0 27] INFO [2022-01-10 12:53:24,186] jepsen worker 87 - jepsen.util 87 :ok :write [29 17] INFO [2022-01-10 12:53:24,187] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,188] jepsen worker 87 - jepsen.util 87 :ok :read [29 17] INFO [2022-01-10 12:53:24,189] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,190] jepsen worker 84 - jepsen.util 84 :ok :write [28 33] INFO [2022-01-10 12:53:24,191] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,191] jepsen worker 87 - jepsen.util 87 :ok :read [29 17] INFO [2022-01-10 12:53:24,192] jepsen worker 84 - jepsen.util 84 :ok :read [28 33] INFO [2022-01-10 12:53:24,192] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,193] jepsen worker 40 - jepsen.util 130 :ok :write [13 37] INFO [2022-01-10 12:53:24,193] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,193] jepsen worker 87 - jepsen.util 87 :ok :read [29 17] INFO [2022-01-10 12:53:24,193] jepsen worker 5 - jepsen.util 185 :ok :write [1 33] INFO [2022-01-10 12:53:24,193] jepsen worker 84 - jepsen.util 84 :ok :read [28 33] INFO [2022-01-10 12:53:24,194] jepsen worker 40 - jepsen.util 130 :invoke :write [13 7] INFO [2022-01-10 12:53:24,194] jepsen worker 87 - jepsen.util 87 :invoke :write [29 2] INFO [2022-01-10 12:53:24,195] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,195] jepsen worker 5 - jepsen.util 185 :invoke :write [1 34] INFO [2022-01-10 12:53:24,196] jepsen worker 84 - jepsen.util 84 :ok :read [28 33] INFO [2022-01-10 12:53:24,196] jepsen worker 84 - jepsen.util 84 :invoke :write [28 20] INFO [2022-01-10 12:53:24,202] jepsen worker 37 - jepsen.util 127 :ok :write [12 14] INFO [2022-01-10 12:53:24,203] jepsen worker 81 - jepsen.util 171 :ok :write [27 2] INFO [2022-01-10 12:53:24,203] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,203] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:24,203] jepsen worker 37 - jepsen.util 127 :ok :read [12 14] INFO [2022-01-10 12:53:24,204] jepsen worker 81 - jepsen.util 171 :ok :read [27 2] INFO [2022-01-10 12:53:24,204] jepsen worker 37 - jepsen.util 127 :invoke :write [12 8] INFO [2022-01-10 12:53:24,205] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:24,205] jepsen worker 81 - jepsen.util 171 :ok :read [27 2] INFO [2022-01-10 12:53:24,206] jepsen worker 81 - jepsen.util 171 :invoke :write [27 0] INFO [2022-01-10 12:53:24,210] jepsen worker 87 - jepsen.util 87 :ok :write [29 2] INFO [2022-01-10 12:53:24,210] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,211] jepsen worker 87 - jepsen.util 87 :ok :read [29 2] INFO [2022-01-10 12:53:24,212] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,212] jepsen worker 2 - jepsen.util 92 :ok :write [0 27] INFO [2022-01-10 12:53:24,213] jepsen worker 87 - jepsen.util 87 :ok :read [29 2] INFO [2022-01-10 12:53:24,213] jepsen worker 84 - jepsen.util 84 :ok :write [28 20] INFO [2022-01-10 12:53:24,213] jepsen worker 2 - jepsen.util 92 :invoke :write [0 18] INFO [2022-01-10 12:53:24,213] jepsen worker 5 - jepsen.util 185 :ok :write [1 34] INFO [2022-01-10 12:53:24,214] jepsen worker 87 - jepsen.util 87 :invoke :write [29 14] INFO [2022-01-10 12:53:24,214] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,215] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,215] jepsen worker 84 - jepsen.util 84 :ok :read [28 20] INFO [2022-01-10 12:53:24,215] jepsen worker 5 - jepsen.util 185 :ok :read [1 34] INFO [2022-01-10 12:53:24,216] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,216] jepsen worker 5 - jepsen.util 185 :invoke :write [1 43] INFO [2022-01-10 12:53:24,216] jepsen worker 84 - jepsen.util 84 :ok :read [28 20] INFO [2022-01-10 12:53:24,217] jepsen worker 84 - jepsen.util 84 :invoke :write [28 18] INFO [2022-01-10 12:53:24,218] jepsen worker 40 - jepsen.util 130 :ok :write [13 7] INFO [2022-01-10 12:53:24,218] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,219] jepsen worker 40 - jepsen.util 130 :ok :read [13 7] INFO [2022-01-10 12:53:24,220] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,220] jepsen worker 40 - jepsen.util 130 :ok :read [13 7] INFO [2022-01-10 12:53:24,221] jepsen worker 40 - jepsen.util 130 :invoke :write [13 11] INFO [2022-01-10 12:53:24,356] jepsen worker 68 - jepsen.util 68 :ok :write [22 41] INFO [2022-01-10 12:53:24,356] jepsen worker 68 - jepsen.util 68 :invoke :write [22 44] INFO [2022-01-10 12:53:24,360] jepsen worker 75 - jepsen.util 165 :ok :write [25 34] INFO [2022-01-10 12:53:24,361] jepsen worker 78 - jepsen.util 78 :ok :write [26 5] INFO [2022-01-10 12:53:24,361] jepsen worker 75 - jepsen.util 165 :invoke :write [25 7] INFO [2022-01-10 12:53:24,362] jepsen worker 78 - jepsen.util 78 :invoke :write [26 8] INFO [2022-01-10 12:53:24,365] jepsen worker 13 - jepsen.util 13 :ok :write [4 14] INFO [2022-01-10 12:53:24,365] jepsen worker 8 - jepsen.util 8 :ok :write [2 29] INFO [2022-01-10 12:53:24,366] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,367] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:24,367] jepsen worker 8 - jepsen.util 8 :ok :read [2 29] INFO [2022-01-10 12:53:24,368] jepsen worker 13 - jepsen.util 13 :ok :read [4 14] INFO [2022-01-10 12:53:24,368] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,369] jepsen worker 8 - jepsen.util 8 :ok :read [2 29] INFO [2022-01-10 12:53:24,369] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:24,369] jepsen worker 8 - jepsen.util 8 :invoke :write [2 25] INFO [2022-01-10 12:53:24,369] jepsen worker 13 - jepsen.util 13 :ok :read [4 14] INFO [2022-01-10 12:53:24,371] jepsen worker 13 - jepsen.util 13 :invoke :write [4 31] INFO [2022-01-10 12:53:24,372] jepsen worker 69 - jepsen.util 159 :ok :write [23 48] INFO [2022-01-10 12:53:24,372] jepsen worker 69 - jepsen.util 159 :invoke :write [23 22] INFO [2022-01-10 12:53:24,374] jepsen worker 61 - jepsen.util 61 :ok :write [20 8] INFO [2022-01-10 12:53:24,375] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:24,376] jepsen worker 61 - jepsen.util 61 :ok :read [20 8] INFO [2022-01-10 12:53:24,376] jepsen worker 61 - jepsen.util 61 :invoke :write [20 46] INFO [2022-01-10 12:53:24,379] jepsen worker 51 - jepsen.util 141 :ok :write [17 10] INFO [2022-01-10 12:53:24,380] jepsen worker 51 - jepsen.util 141 :invoke :write [17 17] INFO [2022-01-10 12:53:24,381] jepsen worker 33 - jepsen.util 123 :ok :write [11 11] INFO [2022-01-10 12:53:24,381] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:24,382] jepsen worker 78 - jepsen.util 78 :ok :write [26 8] INFO [2022-01-10 12:53:24,382] jepsen worker 33 - jepsen.util 123 :ok :read [11 11] INFO [2022-01-10 12:53:24,383] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:24,384] jepsen worker 33 - jepsen.util 123 :invoke :write [11 36] INFO [2022-01-10 12:53:24,384] jepsen worker 78 - jepsen.util 78 :ok :read [26 8] INFO [2022-01-10 12:53:24,385] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:24,386] jepsen worker 78 - jepsen.util 78 :ok :read [26 8] INFO [2022-01-10 12:53:24,387] jepsen worker 78 - jepsen.util 78 :invoke :write [26 3] INFO [2022-01-10 12:53:24,388] jepsen worker 44 - jepsen.util 44 :ok :write [14 19] INFO [2022-01-10 12:53:24,389] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:24,389] jepsen worker 44 - jepsen.util 44 :ok :read [14 19] INFO [2022-01-10 12:53:24,390] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:24,391] jepsen worker 44 - jepsen.util 44 :ok :read [14 19] INFO [2022-01-10 12:53:24,391] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:24,391] jepsen worker 13 - jepsen.util 13 :ok :write [4 31] INFO [2022-01-10 12:53:24,392] jepsen worker 44 - jepsen.util 44 :ok :read [14 19] INFO [2022-01-10 12:53:24,392] jepsen worker 13 - jepsen.util 13 :invoke :write [4 22] INFO [2022-01-10 12:53:24,393] jepsen worker 44 - jepsen.util 44 :invoke :write [14 23] INFO [2022-01-10 12:53:24,394] jepsen worker 15 - jepsen.util 15 :ok :write [5 33] INFO [2022-01-10 12:53:24,395] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:24,396] jepsen worker 15 - jepsen.util 15 :ok :read [5 33] INFO [2022-01-10 12:53:24,396] jepsen worker 15 - jepsen.util 15 :invoke :write [5 40] INFO [2022-01-10 12:53:24,398] jepsen worker 69 - jepsen.util 159 :ok :write [23 22] INFO [2022-01-10 12:53:24,399] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:24,400] jepsen worker 69 - jepsen.util 159 :ok :read [23 22] INFO [2022-01-10 12:53:24,400] jepsen worker 23 - jepsen.util 23 :ok :write [7 10] INFO [2022-01-10 12:53:24,401] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:24,401] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:24,401] jepsen worker 69 - jepsen.util 159 :ok :read [23 22] INFO [2022-01-10 12:53:24,402] jepsen worker 23 - jepsen.util 23 :ok :read [7 10] INFO [2022-01-10 12:53:24,402] jepsen worker 23 - jepsen.util 23 :invoke :write [7 31] INFO [2022-01-10 12:53:24,403] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:24,404] jepsen worker 51 - jepsen.util 141 :ok :write [17 17] INFO [2022-01-10 12:53:24,404] jepsen worker 69 - jepsen.util 159 :ok :read [23 22] INFO [2022-01-10 12:53:24,404] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,405] jepsen worker 69 - jepsen.util 159 :invoke :write [23 33] INFO [2022-01-10 12:53:24,405] jepsen worker 50 - jepsen.util 50 :ok :write [16 17] INFO [2022-01-10 12:53:24,405] jepsen worker 51 - jepsen.util 141 :ok :read [17 17] INFO [2022-01-10 12:53:24,406] jepsen worker 50 - jepsen.util 50 :invoke :write [16 15] INFO [2022-01-10 12:53:24,407] jepsen worker 51 - jepsen.util 141 :invoke :write [17 27] INFO [2022-01-10 12:53:24,409] jepsen worker 78 - jepsen.util 78 :ok :write [26 3] INFO [2022-01-10 12:53:24,409] jepsen worker 78 - jepsen.util 78 :invoke :read [26 nil] INFO [2022-01-10 12:53:24,410] jepsen worker 81 - jepsen.util 171 :ok :write [27 0] INFO [2022-01-10 12:53:24,410] jepsen worker 78 - jepsen.util 78 :ok :read [26 3] INFO [2022-01-10 12:53:24,411] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:24,411] jepsen worker 78 - jepsen.util 78 :invoke :write [26 28] INFO [2022-01-10 12:53:24,411] jepsen worker 81 - jepsen.util 171 :ok :read [27 0] INFO [2022-01-10 12:53:24,412] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:24,412] jepsen worker 81 - jepsen.util 171 :ok :read [27 0] INFO [2022-01-10 12:53:24,413] jepsen worker 81 - jepsen.util 171 :invoke :write [27 42] INFO [2022-01-10 12:53:24,416] jepsen worker 13 - jepsen.util 13 :ok :write [4 22] INFO [2022-01-10 12:53:24,416] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:24,417] jepsen worker 13 - jepsen.util 13 :ok :read [4 22] INFO [2022-01-10 12:53:24,417] jepsen worker 13 - jepsen.util 13 :invoke :write [4 12] INFO [2022-01-10 12:53:24,418] jepsen worker 75 - jepsen.util 165 :ok :write [25 7] INFO [2022-01-10 12:53:24,419] jepsen worker 75 - jepsen.util 165 :invoke :write [25 32] INFO [2022-01-10 12:53:24,420] jepsen worker 69 - jepsen.util 159 :ok :write [23 33] INFO [2022-01-10 12:53:24,420] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:24,421] jepsen worker 69 - jepsen.util 159 :ok :read [23 33] INFO [2022-01-10 12:53:24,422] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:24,423] jepsen worker 69 - jepsen.util 159 :ok :read [23 33] INFO [2022-01-10 12:53:24,423] jepsen worker 69 - jepsen.util 159 :invoke :write [23 34] INFO [2022-01-10 12:53:24,424] jepsen worker 51 - jepsen.util 141 :ok :write [17 27] INFO [2022-01-10 12:53:24,426] jepsen worker 51 - jepsen.util 141 :invoke :write [17 28] INFO [2022-01-10 12:53:24,524] jepsen worker 37 - jepsen.util 127 :ok :write [12 8] INFO [2022-01-10 12:53:24,525] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,526] jepsen worker 37 - jepsen.util 127 :ok :read [12 8] INFO [2022-01-10 12:53:24,526] jepsen worker 37 - jepsen.util 127 :invoke :write [12 42] INFO [2022-01-10 12:53:24,541] jepsen worker 61 - jepsen.util 61 :ok :write [20 46] INFO [2022-01-10 12:53:24,542] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:24,543] jepsen worker 61 - jepsen.util 61 :ok :read [20 46] INFO [2022-01-10 12:53:24,544] jepsen worker 61 - jepsen.util 61 :invoke :write [20 22] INFO [2022-01-10 12:53:24,544] jepsen worker 78 - jepsen.util 78 :ok :write [26 28] INFO [2022-01-10 12:53:24,545] jepsen worker 78 - jepsen.util 78 :invoke :write [26 33] INFO [2022-01-10 12:53:24,546] jepsen worker 84 - jepsen.util 84 :ok :write [28 18] INFO [2022-01-10 12:53:24,546] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,547] jepsen worker 13 - jepsen.util 13 :ok :write [4 12] INFO [2022-01-10 12:53:24,547] jepsen worker 84 - jepsen.util 84 :ok :read [28 18] INFO [2022-01-10 12:53:24,548] jepsen worker 13 - jepsen.util 13 :invoke :write [4 12] INFO [2022-01-10 12:53:24,548] jepsen worker 84 - jepsen.util 84 :invoke :write [28 6] INFO [2022-01-10 12:53:24,549] jepsen worker 87 - jepsen.util 87 :ok :write [29 14] INFO [2022-01-10 12:53:24,550] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,551] jepsen worker 44 - jepsen.util 44 :ok :write [14 23] INFO [2022-01-10 12:53:24,551] jepsen worker 44 - jepsen.util 44 :invoke :write [14 0] INFO [2022-01-10 12:53:24,553] jepsen worker 87 - jepsen.util 87 :ok :read [29 14] INFO [2022-01-10 12:53:24,554] jepsen worker 87 - jepsen.util 87 :invoke :write [29 45] INFO [2022-01-10 12:53:24,554] jepsen worker 69 - jepsen.util 159 :ok :write [23 34] INFO [2022-01-10 12:53:24,555] jepsen worker 2 - jepsen.util 92 :ok :write [0 18] INFO [2022-01-10 12:53:24,555] jepsen worker 69 - jepsen.util 159 :invoke :write [23 3] INFO [2022-01-10 12:53:24,556] jepsen worker 2 - jepsen.util 92 :invoke :write [0 15] INFO [2022-01-10 12:53:24,557] jepsen worker 5 - jepsen.util 185 :ok :write [1 43] INFO [2022-01-10 12:53:24,557] jepsen worker 5 - jepsen.util 185 :invoke :write [1 22] INFO [2022-01-10 12:53:24,559] jepsen worker 53 - jepsen.util 143 :ok :write [17 36] INFO [2022-01-10 12:53:24,560] jepsen worker 37 - jepsen.util 127 :ok :write [12 42] INFO [2022-01-10 12:53:24,560] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,561] jepsen worker 37 - jepsen.util 127 :invoke :write [12 24] INFO [2022-01-10 12:53:24,561] jepsen worker 53 - jepsen.util 143 :ok :read [17 36] INFO [2022-01-10 12:53:24,562] jepsen worker 23 - jepsen.util 23 :ok :write [7 31] INFO [2022-01-10 12:53:24,562] jepsen worker 53 - jepsen.util 143 :invoke :write [17 38] INFO [2022-01-10 12:53:24,562] jepsen worker 23 - jepsen.util 23 :invoke :write [7 37] INFO [2022-01-10 12:53:24,562] jepsen worker 40 - jepsen.util 130 :ok :write [13 11] INFO [2022-01-10 12:53:24,563] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,564] jepsen worker 40 - jepsen.util 130 :ok :read [13 11] INFO [2022-01-10 12:53:24,564] jepsen worker 40 - jepsen.util 130 :invoke :write [13 2] INFO [2022-01-10 12:53:24,565] jepsen worker 84 - jepsen.util 84 :ok :write [28 6] INFO [2022-01-10 12:53:24,565] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,566] jepsen worker 84 - jepsen.util 84 :ok :read [28 6] INFO [2022-01-10 12:53:24,566] jepsen worker 80 - jepsen.util 260 :ok :write [26 7] INFO [2022-01-10 12:53:24,567] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,567] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:24,568] jepsen worker 84 - jepsen.util 84 :ok :read [28 6] INFO [2022-01-10 12:53:24,568] jepsen worker 80 - jepsen.util 260 :ok :read [26 7] INFO [2022-01-10 12:53:24,568] jepsen worker 84 - jepsen.util 84 :invoke :write [28 42] INFO [2022-01-10 12:53:24,569] jepsen worker 80 - jepsen.util 260 :invoke :write [26 15] INFO [2022-01-10 12:53:24,569] jepsen worker 13 - jepsen.util 13 :ok :write [4 12] INFO [2022-01-10 12:53:24,570] jepsen worker 13 - jepsen.util 13 :invoke :write [4 21] INFO [2022-01-10 12:53:24,570] jepsen worker 68 - jepsen.util 68 :ok :write [22 44] INFO [2022-01-10 12:53:24,571] jepsen worker 68 - jepsen.util 68 :invoke :write [22 19] INFO [2022-01-10 12:53:24,574] jepsen worker 70 - jepsen.util 160 :ok :write [23 48] INFO [2022-01-10 12:53:24,574] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:24,575] jepsen worker 75 - jepsen.util 165 :ok :write [25 32] INFO [2022-01-10 12:53:24,575] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:24,575] jepsen worker 70 - jepsen.util 160 :ok :read [23 48] INFO [2022-01-10 12:53:24,576] jepsen worker 70 - jepsen.util 160 :invoke :write [23 34] INFO [2022-01-10 12:53:24,576] jepsen worker 75 - jepsen.util 165 :ok :read [25 32] INFO [2022-01-10 12:53:24,577] jepsen worker 75 - jepsen.util 165 :invoke :write [25 44] INFO [2022-01-10 12:53:24,578] jepsen worker 8 - jepsen.util 8 :ok :write [2 25] INFO [2022-01-10 12:53:24,579] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,580] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:24,580] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,581] jepsen worker 5 - jepsen.util 185 :ok :write [1 22] INFO [2022-01-10 12:53:24,582] jepsen worker 33 - jepsen.util 123 :ok :write [11 36] INFO [2022-01-10 12:53:24,582] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,583] jepsen worker 33 - jepsen.util 123 :invoke :write [11 22] INFO [2022-01-10 12:53:24,583] jepsen worker 5 - jepsen.util 185 :ok :read [1 22] INFO [2022-01-10 12:53:24,583] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,584] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:24,584] jepsen worker 61 - jepsen.util 61 :ok :write [20 22] INFO [2022-01-10 12:53:24,584] jepsen worker 5 - jepsen.util 185 :ok :read [1 22] INFO [2022-01-10 12:53:24,584] jepsen worker 61 - jepsen.util 61 :invoke :write [20 47] INFO [2022-01-10 12:53:24,585] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,585] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,586] jepsen worker 5 - jepsen.util 185 :ok :read [1 22] INFO [2022-01-10 12:53:24,586] jepsen worker 15 - jepsen.util 15 :ok :write [5 40] INFO [2022-01-10 12:53:24,587] jepsen worker 5 - jepsen.util 185 :invoke :write [1 31] INFO [2022-01-10 12:53:24,587] jepsen worker 15 - jepsen.util 15 :invoke :write [5 14] INFO [2022-01-10 12:53:24,587] jepsen worker 44 - jepsen.util 44 :ok :write [14 0] INFO [2022-01-10 12:53:24,588] jepsen worker 44 - jepsen.util 44 :invoke :write [14 2] INFO [2022-01-10 12:53:24,589] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:24,589] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,590] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:24,591] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,592] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:24,592] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,593] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:24,594] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,595] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:24,595] jepsen worker 8 - jepsen.util 8 :invoke :write [2 39] INFO [2022-01-10 12:53:24,653] jepsen worker 45 - jepsen.util 225 :fail :read [15 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:24,654] jepsen worker 45 - jepsen.util 225 :invoke :read [15 nil] INFO [2022-01-10 12:53:24,698] jepsen worker 89 - jepsen.util 179 :info :write [29 46] :RequestTimeoutException INFO [2022-01-10 12:53:24,698] jepsen worker 89 - jepsen.util 269 :invoke :write [29 28] INFO [2022-01-10 12:53:24,728] jepsen worker 71 - jepsen.util 71 :info :write [23 28] :RequestTimeoutException INFO [2022-01-10 12:53:24,728] jepsen worker 71 - jepsen.util 161 :invoke :write [23 35] INFO [2022-01-10 12:53:24,734] jepsen worker 53 - jepsen.util 143 :ok :write [17 38] INFO [2022-01-10 12:53:24,735] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,735] jepsen worker 50 - jepsen.util 50 :ok :write [16 15] INFO [2022-01-10 12:53:24,735] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:24,736] jepsen worker 53 - jepsen.util 143 :ok :read [17 38] INFO [2022-01-10 12:53:24,736] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,736] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:24,737] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:24,737] jepsen worker 53 - jepsen.util 143 :ok :read [17 38] INFO [2022-01-10 12:53:24,738] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:24,738] jepsen worker 53 - jepsen.util 143 :invoke :write [17 24] INFO [2022-01-10 12:53:24,738] jepsen worker 50 - jepsen.util 50 :invoke :write [16 23] INFO [2022-01-10 12:53:24,741] jepsen worker 81 - jepsen.util 171 :ok :write [27 42] INFO [2022-01-10 12:53:24,741] jepsen worker 37 - jepsen.util 127 :ok :write [12 24] INFO [2022-01-10 12:53:24,742] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:24,742] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,743] jepsen worker 81 - jepsen.util 171 :ok :read [27 42] INFO [2022-01-10 12:53:24,743] jepsen worker 37 - jepsen.util 127 :ok :read [12 24] INFO [2022-01-10 12:53:24,743] jepsen worker 81 - jepsen.util 171 :invoke :write [27 43] INFO [2022-01-10 12:53:24,744] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,745] jepsen worker 37 - jepsen.util 127 :ok :read [12 24] INFO [2022-01-10 12:53:24,745] jepsen worker 23 - jepsen.util 23 :ok :write [7 37] INFO [2022-01-10 12:53:24,745] jepsen worker 23 - jepsen.util 23 :invoke :write [7 28] INFO [2022-01-10 12:53:24,746] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,747] jepsen worker 37 - jepsen.util 127 :ok :read [12 24] INFO [2022-01-10 12:53:24,748] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,749] jepsen worker 87 - jepsen.util 87 :ok :write [29 45] INFO [2022-01-10 12:53:24,749] jepsen worker 87 - jepsen.util 87 :invoke :write [29 18] INFO [2022-01-10 12:53:24,749] jepsen worker 37 - jepsen.util 127 :ok :read [12 24] INFO [2022-01-10 12:53:24,750] jepsen worker 80 - jepsen.util 260 :ok :write [26 15] INFO [2022-01-10 12:53:24,751] jepsen worker 37 - jepsen.util 127 :invoke :write [12 9] INFO [2022-01-10 12:53:24,751] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:24,753] jepsen worker 80 - jepsen.util 260 :ok :read [26 15] INFO [2022-01-10 12:53:24,753] jepsen worker 80 - jepsen.util 260 :invoke :write [26 27] INFO [2022-01-10 12:53:24,755] jepsen worker 84 - jepsen.util 84 :ok :write [28 42] INFO [2022-01-10 12:53:24,756] jepsen worker 84 - jepsen.util 84 :invoke :write [28 17] INFO [2022-01-10 12:53:24,757] jepsen worker 2 - jepsen.util 92 :ok :write [0 15] INFO [2022-01-10 12:53:24,758] jepsen worker 2 - jepsen.util 92 :invoke :write [0 36] INFO [2022-01-10 12:53:24,762] jepsen worker 13 - jepsen.util 13 :ok :write [4 21] INFO [2022-01-10 12:53:24,763] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:24,763] jepsen worker 75 - jepsen.util 165 :ok :write [25 44] INFO [2022-01-10 12:53:24,764] jepsen worker 75 - jepsen.util 165 :invoke :write [25 13] INFO [2022-01-10 12:53:24,764] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:24,764] jepsen worker 40 - jepsen.util 130 :ok :write [13 2] INFO [2022-01-10 12:53:24,765] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,765] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:24,766] jepsen worker 40 - jepsen.util 130 :ok :read [13 2] INFO [2022-01-10 12:53:24,767] jepsen worker 40 - jepsen.util 130 :invoke :write [13 40] INFO [2022-01-10 12:53:24,768] jepsen worker 69 - jepsen.util 159 :ok :write [23 3] INFO [2022-01-10 12:53:24,769] jepsen worker 69 - jepsen.util 159 :invoke :write [23 2] INFO [2022-01-10 12:53:24,769] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:24,770] jepsen worker 68 - jepsen.util 68 :ok :write [22 19] INFO [2022-01-10 12:53:24,770] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:24,771] jepsen worker 13 - jepsen.util 13 :ok :read [4 21] INFO [2022-01-10 12:53:24,771] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:24,772] jepsen worker 13 - jepsen.util 13 :invoke :write [4 24] INFO [2022-01-10 12:53:24,772] jepsen worker 68 - jepsen.util 68 :ok :read [22 19] INFO [2022-01-10 12:53:24,773] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:24,773] jepsen worker 5 - jepsen.util 185 :ok :write [1 31] INFO [2022-01-10 12:53:24,774] jepsen worker 68 - jepsen.util 68 :ok :read [22 19] INFO [2022-01-10 12:53:24,774] jepsen worker 5 - jepsen.util 185 :invoke :write [1 5] INFO [2022-01-10 12:53:24,774] jepsen worker 68 - jepsen.util 68 :invoke :write [22 25] INFO [2022-01-10 12:53:24,776] jepsen worker 61 - jepsen.util 61 :ok :write [20 47] INFO [2022-01-10 12:53:24,777] jepsen worker 61 - jepsen.util 61 :invoke :write [20 35] INFO [2022-01-10 12:53:24,778] jepsen worker 8 - jepsen.util 8 :ok :write [2 39] INFO [2022-01-10 12:53:24,778] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,779] jepsen worker 8 - jepsen.util 8 :ok :read [2 39] INFO [2022-01-10 12:53:24,780] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,780] jepsen worker 53 - jepsen.util 143 :ok :write [17 24] INFO [2022-01-10 12:53:24,781] jepsen worker 8 - jepsen.util 8 :ok :read [2 39] INFO [2022-01-10 12:53:24,781] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,781] jepsen worker 8 - jepsen.util 8 :invoke :write [2 25] INFO [2022-01-10 12:53:24,782] jepsen worker 53 - jepsen.util 143 :ok :read [17 24] INFO [2022-01-10 12:53:24,782] jepsen worker 44 - jepsen.util 44 :ok :write [14 2] INFO [2022-01-10 12:53:24,783] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,783] jepsen worker 44 - jepsen.util 44 :invoke :write [14 13] INFO [2022-01-10 12:53:24,783] jepsen worker 53 - jepsen.util 143 :ok :read [17 24] INFO [2022-01-10 12:53:24,784] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,785] jepsen worker 53 - jepsen.util 143 :ok :read [17 24] INFO [2022-01-10 12:53:24,785] jepsen worker 33 - jepsen.util 123 :ok :write [11 22] INFO [2022-01-10 12:53:24,786] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:24,786] jepsen worker 53 - jepsen.util 143 :invoke :write [17 23] INFO [2022-01-10 12:53:24,787] jepsen worker 33 - jepsen.util 123 :ok :read [11 22] INFO [2022-01-10 12:53:24,787] jepsen worker 33 - jepsen.util 123 :invoke :write [11 4] INFO [2022-01-10 12:53:24,787] jepsen worker 23 - jepsen.util 23 :ok :write [7 28] INFO [2022-01-10 12:53:24,788] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:24,788] jepsen worker 15 - jepsen.util 15 :ok :write [5 14] INFO [2022-01-10 12:53:24,789] jepsen worker 37 - jepsen.util 127 :ok :write [12 9] INFO [2022-01-10 12:53:24,789] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:24,789] jepsen worker 23 - jepsen.util 23 :ok :read [7 28] INFO [2022-01-10 12:53:24,790] jepsen worker 15 - jepsen.util 15 :ok :read [5 14] INFO [2022-01-10 12:53:24,791] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,791] jepsen worker 23 - jepsen.util 23 :invoke :write [7 22] INFO [2022-01-10 12:53:24,791] jepsen worker 37 - jepsen.util 127 :ok :read [12 9] INFO [2022-01-10 12:53:24,792] jepsen worker 15 - jepsen.util 15 :invoke :write [5 8] INFO [2022-01-10 12:53:24,792] jepsen worker 37 - jepsen.util 127 :invoke :write [12 30] INFO [2022-01-10 12:53:24,794] jepsen worker 80 - jepsen.util 260 :ok :write [26 27] INFO [2022-01-10 12:53:24,794] jepsen worker 80 - jepsen.util 260 :invoke :write [26 30] INFO [2022-01-10 12:53:24,796] jepsen worker 50 - jepsen.util 50 :ok :write [16 23] INFO [2022-01-10 12:53:24,797] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:24,797] jepsen worker 50 - jepsen.util 50 :ok :read [16 23] INFO [2022-01-10 12:53:24,798] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:24,799] jepsen worker 50 - jepsen.util 50 :ok :read [16 23] INFO [2022-01-10 12:53:24,799] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:24,800] jepsen worker 50 - jepsen.util 50 :ok :read [16 23] INFO [2022-01-10 12:53:24,801] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:24,801] jepsen worker 84 - jepsen.util 84 :ok :write [28 17] INFO [2022-01-10 12:53:24,801] jepsen worker 50 - jepsen.util 50 :ok :read [16 23] INFO [2022-01-10 12:53:24,802] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:53:24,803] jepsen worker 84 - jepsen.util 84 :invoke :write [28 22] INFO [2022-01-10 12:53:24,806] jepsen worker 81 - jepsen.util 171 :ok :write [27 43] INFO [2022-01-10 12:53:24,806] jepsen worker 75 - jepsen.util 165 :ok :write [25 13] INFO [2022-01-10 12:53:24,806] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:24,807] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:24,807] jepsen worker 81 - jepsen.util 171 :ok :read [27 43] INFO [2022-01-10 12:53:24,807] jepsen worker 75 - jepsen.util 165 :ok :read [25 13] INFO [2022-01-10 12:53:24,808] jepsen worker 81 - jepsen.util 171 :invoke :write [27 0] INFO [2022-01-10 12:53:24,808] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:24,809] jepsen worker 75 - jepsen.util 165 :ok :read [25 13] INFO [2022-01-10 12:53:24,809] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:24,810] jepsen worker 75 - jepsen.util 165 :ok :read [25 13] INFO [2022-01-10 12:53:24,811] jepsen worker 75 - jepsen.util 165 :invoke :write [25 6] INFO [2022-01-10 12:53:24,813] jepsen worker 87 - jepsen.util 87 :ok :write [29 18] INFO [2022-01-10 12:53:24,814] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,815] jepsen worker 87 - jepsen.util 87 :ok :read [29 18] INFO [2022-01-10 12:53:24,815] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,816] jepsen worker 87 - jepsen.util 87 :ok :read [29 18] INFO [2022-01-10 12:53:24,816] jepsen worker 2 - jepsen.util 92 :ok :write [0 36] INFO [2022-01-10 12:53:24,816] jepsen worker 5 - jepsen.util 185 :ok :write [1 5] INFO [2022-01-10 12:53:24,817] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,817] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,818] jepsen worker 5 - jepsen.util 185 :ok :read [1 5] INFO [2022-01-10 12:53:24,818] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,818] jepsen worker 87 - jepsen.util 87 :ok :read [29 18] INFO [2022-01-10 12:53:24,818] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,819] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,819] jepsen worker 2 - jepsen.util 92 :ok :read [0 36] INFO [2022-01-10 12:53:24,819] jepsen worker 5 - jepsen.util 185 :ok :read [1 5] INFO [2022-01-10 12:53:24,819] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,819] jepsen worker 87 - jepsen.util 87 :ok :read [29 18] INFO [2022-01-10 12:53:24,819] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,820] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,820] jepsen worker 2 - jepsen.util 92 :ok :read [0 36] INFO [2022-01-10 12:53:24,820] jepsen worker 5 - jepsen.util 185 :ok :read [1 5] INFO [2022-01-10 12:53:24,821] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,821] jepsen worker 87 - jepsen.util 87 :ok :read [29 18] INFO [2022-01-10 12:53:24,821] jepsen worker 5 - jepsen.util 185 :invoke :write [1 47] INFO [2022-01-10 12:53:24,822] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,822] jepsen worker 2 - jepsen.util 92 :ok :read [0 36] INFO [2022-01-10 12:53:24,822] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,822] jepsen worker 40 - jepsen.util 130 :ok :write [13 40] INFO [2022-01-10 12:53:24,823] jepsen worker 87 - jepsen.util 87 :ok :read [29 18] INFO [2022-01-10 12:53:24,823] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,823] jepsen worker 87 - jepsen.util 87 :invoke :write [29 21] INFO [2022-01-10 12:53:24,823] jepsen worker 2 - jepsen.util 92 :ok :read [0 36] INFO [2022-01-10 12:53:24,824] jepsen worker 40 - jepsen.util 130 :ok :read [13 40] INFO [2022-01-10 12:53:24,824] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,824] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,825] jepsen worker 2 - jepsen.util 92 :ok :read [0 36] INFO [2022-01-10 12:53:24,825] jepsen worker 40 - jepsen.util 130 :ok :read [13 40] INFO [2022-01-10 12:53:24,826] jepsen worker 2 - jepsen.util 92 :invoke :write [0 26] INFO [2022-01-10 12:53:24,826] jepsen worker 40 - jepsen.util 130 :invoke :write [13 37] INFO [2022-01-10 12:53:24,829] jepsen worker 37 - jepsen.util 127 :ok :write [12 30] INFO [2022-01-10 12:53:24,830] jepsen worker 37 - jepsen.util 127 :invoke :write [12 19] INFO [2022-01-10 12:53:24,842] jepsen worker 84 - jepsen.util 84 :ok :write [28 22] INFO [2022-01-10 12:53:24,843] jepsen worker 84 - jepsen.util 84 :invoke :write [28 48] INFO [2022-01-10 12:53:24,848] jepsen worker 5 - jepsen.util 185 :ok :write [1 47] INFO [2022-01-10 12:53:24,848] jepsen worker 5 - jepsen.util 185 :invoke :write [1 23] INFO [2022-01-10 12:53:24,854] jepsen worker 37 - jepsen.util 127 :ok :write [12 19] INFO [2022-01-10 12:53:24,855] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,856] jepsen worker 37 - jepsen.util 127 :ok :read [12 19] INFO [2022-01-10 12:53:24,856] jepsen worker 37 - jepsen.util 127 :invoke :write [12 19] INFO [2022-01-10 12:53:24,860] jepsen worker 84 - jepsen.util 84 :ok :write [28 48] INFO [2022-01-10 12:53:24,861] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,862] jepsen worker 84 - jepsen.util 84 :ok :read [28 48] INFO [2022-01-10 12:53:24,862] jepsen worker 87 - jepsen.util 87 :ok :write [29 21] INFO [2022-01-10 12:53:24,862] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,863] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,863] jepsen worker 84 - jepsen.util 84 :ok :read [28 48] INFO [2022-01-10 12:53:24,864] jepsen worker 87 - jepsen.util 87 :ok :read [29 21] INFO [2022-01-10 12:53:24,864] jepsen worker 84 - jepsen.util 84 :invoke :write [28 48] INFO [2022-01-10 12:53:24,864] jepsen worker 5 - jepsen.util 185 :ok :write [1 23] INFO [2022-01-10 12:53:24,864] jepsen worker 87 - jepsen.util 87 :invoke :write [29 22] INFO [2022-01-10 12:53:24,865] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,866] jepsen worker 5 - jepsen.util 185 :ok :read [1 23] INFO [2022-01-10 12:53:24,868] jepsen worker 2 - jepsen.util 92 :ok :write [0 26] INFO [2022-01-10 12:53:24,868] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,868] jepsen worker 32 - jepsen.util 212 :fail :read [10 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:24,869] jepsen worker 32 - jepsen.util 212 :invoke :write [10 11] INFO [2022-01-10 12:53:24,870] jepsen worker 5 - jepsen.util 185 :ok :read [1 23] INFO [2022-01-10 12:53:24,870] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,870] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:24,871] jepsen worker 2 - jepsen.util 92 :ok :read [0 26] INFO [2022-01-10 12:53:24,871] jepsen worker 5 - jepsen.util 185 :ok :read [1 23] INFO [2022-01-10 12:53:24,871] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,872] jepsen worker 5 - jepsen.util 185 :invoke :write [1 33] INFO [2022-01-10 12:53:24,872] jepsen worker 37 - jepsen.util 127 :ok :write [12 19] INFO [2022-01-10 12:53:24,872] jepsen worker 2 - jepsen.util 92 :ok :read [0 26] INFO [2022-01-10 12:53:24,873] jepsen worker 2 - jepsen.util 92 :invoke :write [0 45] INFO [2022-01-10 12:53:24,873] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,873] jepsen worker 40 - jepsen.util 130 :ok :write [13 37] INFO [2022-01-10 12:53:24,874] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,874] jepsen worker 37 - jepsen.util 127 :ok :read [12 19] INFO [2022-01-10 12:53:24,875] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,875] jepsen worker 40 - jepsen.util 130 :ok :read [13 37] INFO [2022-01-10 12:53:24,875] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,876] jepsen worker 37 - jepsen.util 127 :ok :read [12 19] INFO [2022-01-10 12:53:24,876] jepsen worker 40 - jepsen.util 130 :ok :read [13 37] INFO [2022-01-10 12:53:24,876] jepsen worker 37 - jepsen.util 127 :invoke :write [12 21] INFO [2022-01-10 12:53:24,877] jepsen worker 40 - jepsen.util 130 :invoke :write [13 27] INFO [2022-01-10 12:53:24,877] jepsen worker 84 - jepsen.util 84 :ok :write [28 48] INFO [2022-01-10 12:53:24,878] jepsen worker 87 - jepsen.util 87 :ok :write [29 22] INFO [2022-01-10 12:53:24,878] jepsen worker 84 - jepsen.util 84 :invoke :write [28 40] INFO [2022-01-10 12:53:24,879] jepsen worker 87 - jepsen.util 87 :invoke :write [29 0] INFO [2022-01-10 12:53:24,883] jepsen worker 5 - jepsen.util 185 :ok :write [1 33] INFO [2022-01-10 12:53:24,884] jepsen worker 5 - jepsen.util 185 :invoke :write [1 34] INFO [2022-01-10 12:53:24,889] jepsen worker 37 - jepsen.util 127 :ok :write [12 21] INFO [2022-01-10 12:53:24,890] jepsen worker 37 - jepsen.util 127 :invoke :write [12 46] INFO [2022-01-10 12:53:24,890] jepsen worker 40 - jepsen.util 130 :ok :write [13 27] INFO [2022-01-10 12:53:24,891] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,891] jepsen worker 2 - jepsen.util 92 :ok :write [0 45] INFO [2022-01-10 12:53:24,891] jepsen worker 40 - jepsen.util 130 :ok :read [13 27] INFO [2022-01-10 12:53:24,892] jepsen worker 2 - jepsen.util 92 :invoke :write [0 40] INFO [2022-01-10 12:53:24,892] jepsen worker 40 - jepsen.util 130 :invoke :write [13 21] INFO [2022-01-10 12:53:24,896] jepsen worker 84 - jepsen.util 84 :ok :write [28 40] INFO [2022-01-10 12:53:24,896] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,899] jepsen worker 84 - jepsen.util 84 :ok :read [28 40] INFO [2022-01-10 12:53:24,899] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,900] jepsen worker 84 - jepsen.util 84 :ok :read [28 40] INFO [2022-01-10 12:53:24,900] jepsen worker 5 - jepsen.util 185 :ok :write [1 34] INFO [2022-01-10 12:53:24,901] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,901] jepsen worker 5 - jepsen.util 185 :invoke :write [1 7] INFO [2022-01-10 12:53:24,901] jepsen worker 84 - jepsen.util 84 :ok :read [28 40] INFO [2022-01-10 12:53:24,902] jepsen worker 84 - jepsen.util 84 :invoke :write [28 49] INFO [2022-01-10 12:53:24,906] jepsen worker 37 - jepsen.util 127 :ok :write [12 46] INFO [2022-01-10 12:53:24,907] jepsen worker 87 - jepsen.util 87 :ok :write [29 0] INFO [2022-01-10 12:53:24,907] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,908] jepsen worker 87 - jepsen.util 87 :invoke :write [29 32] INFO [2022-01-10 12:53:24,908] jepsen worker 37 - jepsen.util 127 :ok :read [12 46] INFO [2022-01-10 12:53:24,909] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,910] jepsen worker 37 - jepsen.util 127 :ok :read [12 46] INFO [2022-01-10 12:53:24,910] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,911] jepsen worker 37 - jepsen.util 127 :ok :read [12 46] INFO [2022-01-10 12:53:24,912] jepsen worker 40 - jepsen.util 130 :ok :write [13 21] INFO [2022-01-10 12:53:24,912] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,913] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,913] jepsen worker 37 - jepsen.util 127 :ok :read [12 46] INFO [2022-01-10 12:53:24,913] jepsen worker 40 - jepsen.util 130 :ok :read [13 21] INFO [2022-01-10 12:53:24,914] jepsen worker 40 - jepsen.util 130 :invoke :write [13 30] INFO [2022-01-10 12:53:24,915] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,917] jepsen worker 37 - jepsen.util 127 :ok :read [12 46] INFO [2022-01-10 12:53:24,917] jepsen worker 2 - jepsen.util 92 :ok :write [0 40] INFO [2022-01-10 12:53:24,917] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,918] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,919] jepsen worker 2 - jepsen.util 92 :ok :read [0 40] INFO [2022-01-10 12:53:24,919] jepsen worker 37 - jepsen.util 127 :ok :read [12 46] INFO [2022-01-10 12:53:24,920] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,920] jepsen worker 2 - jepsen.util 92 :ok :read [0 40] INFO [2022-01-10 12:53:24,920] jepsen worker 37 - jepsen.util 127 :invoke :write [12 14] INFO [2022-01-10 12:53:24,921] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,922] jepsen worker 2 - jepsen.util 92 :ok :read [0 40] INFO [2022-01-10 12:53:24,922] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,923] jepsen worker 2 - jepsen.util 92 :ok :read [0 40] INFO [2022-01-10 12:53:24,924] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:24,925] jepsen worker 2 - jepsen.util 92 :ok :read [0 40] INFO [2022-01-10 12:53:24,925] jepsen worker 87 - jepsen.util 87 :ok :write [29 32] INFO [2022-01-10 12:53:24,925] jepsen worker 2 - jepsen.util 92 :invoke :write [0 32] INFO [2022-01-10 12:53:24,925] jepsen worker 61 - jepsen.util 61 :ok :write [20 35] INFO [2022-01-10 12:53:24,926] jepsen worker 69 - jepsen.util 159 :ok :write [23 2] INFO [2022-01-10 12:53:24,927] jepsen worker 61 - jepsen.util 61 :invoke :write [20 21] INFO [2022-01-10 12:53:24,927] jepsen worker 69 - jepsen.util 159 :invoke :write [23 41] INFO [2022-01-10 12:53:24,928] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:24,929] jepsen worker 87 - jepsen.util 87 :ok :read [29 32] INFO [2022-01-10 12:53:24,929] jepsen worker 87 - jepsen.util 87 :invoke :write [29 9] INFO [2022-01-10 12:53:24,930] jepsen worker 13 - jepsen.util 13 :ok :write [4 24] INFO [2022-01-10 12:53:24,930] jepsen worker 13 - jepsen.util 13 :invoke :write [4 40] INFO [2022-01-10 12:53:24,934] jepsen worker 40 - jepsen.util 130 :ok :write [13 30] INFO [2022-01-10 12:53:24,935] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,935] jepsen worker 8 - jepsen.util 8 :ok :write [2 25] INFO [2022-01-10 12:53:24,935] jepsen worker 40 - jepsen.util 130 :ok :read [13 30] INFO [2022-01-10 12:53:24,936] jepsen worker 53 - jepsen.util 143 :ok :write [17 23] INFO [2022-01-10 12:53:24,936] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,937] jepsen worker 40 - jepsen.util 130 :ok :read [13 30] INFO [2022-01-10 12:53:24,937] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:24,937] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,938] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:24,938] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,938] jepsen worker 53 - jepsen.util 143 :ok :read [17 23] INFO [2022-01-10 12:53:24,938] jepsen worker 8 - jepsen.util 8 :invoke :write [2 29] INFO [2022-01-10 12:53:24,939] jepsen worker 40 - jepsen.util 130 :ok :read [13 30] INFO [2022-01-10 12:53:24,939] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,939] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,940] jepsen worker 53 - jepsen.util 143 :ok :read [17 23] INFO [2022-01-10 12:53:24,940] jepsen worker 40 - jepsen.util 130 :ok :read [13 30] INFO [2022-01-10 12:53:24,940] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,941] jepsen worker 40 - jepsen.util 130 :invoke :write [13 40] INFO [2022-01-10 12:53:24,941] jepsen worker 53 - jepsen.util 143 :ok :read [17 23] INFO [2022-01-10 12:53:24,942] jepsen worker 2 - jepsen.util 92 :ok :write [0 32] INFO [2022-01-10 12:53:24,942] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,942] jepsen worker 2 - jepsen.util 92 :invoke :write [0 49] INFO [2022-01-10 12:53:24,943] jepsen worker 68 - jepsen.util 68 :ok :write [22 25] INFO [2022-01-10 12:53:24,944] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:24,945] jepsen worker 68 - jepsen.util 68 :ok :read [22 25] INFO [2022-01-10 12:53:24,946] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:24,946] jepsen worker 68 - jepsen.util 68 :ok :read [22 25] INFO [2022-01-10 12:53:24,946] jepsen worker 80 - jepsen.util 260 :ok :write [26 30] INFO [2022-01-10 12:53:24,947] jepsen worker 53 - jepsen.util 143 :ok :read [17 23] INFO [2022-01-10 12:53:24,947] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:24,948] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:24,948] jepsen worker 68 - jepsen.util 68 :ok :read [22 25] INFO [2022-01-10 12:53:24,948] jepsen worker 44 - jepsen.util 44 :ok :write [14 13] INFO [2022-01-10 12:53:24,948] jepsen worker 80 - jepsen.util 260 :invoke :write [26 35] INFO [2022-01-10 12:53:24,949] jepsen worker 53 - jepsen.util 143 :ok :read [17 23] INFO [2022-01-10 12:53:24,949] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:24,949] jepsen worker 53 - jepsen.util 143 :invoke :write [17 43] INFO [2022-01-10 12:53:24,949] jepsen worker 44 - jepsen.util 44 :ok :read [14 13] INFO [2022-01-10 12:53:24,950] jepsen worker 68 - jepsen.util 68 :invoke :write [22 7] INFO [2022-01-10 12:53:24,950] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:24,951] jepsen worker 44 - jepsen.util 44 :ok :read [14 13] INFO [2022-01-10 12:53:24,951] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:24,952] jepsen worker 44 - jepsen.util 44 :ok :read [14 13] INFO [2022-01-10 12:53:24,952] jepsen worker 69 - jepsen.util 159 :ok :write [23 41] INFO [2022-01-10 12:53:24,953] jepsen worker 44 - jepsen.util 44 :invoke :write [14 14] INFO [2022-01-10 12:53:24,954] jepsen worker 69 - jepsen.util 159 :invoke :write [23 26] INFO [2022-01-10 12:53:24,957] jepsen worker 13 - jepsen.util 13 :ok :write [4 40] INFO [2022-01-10 12:53:24,957] jepsen worker 33 - jepsen.util 123 :ok :write [11 4] INFO [2022-01-10 12:53:24,958] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:24,959] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:24,959] jepsen worker 13 - jepsen.util 13 :ok :read [4 40] INFO [2022-01-10 12:53:24,959] jepsen worker 33 - jepsen.util 123 :ok :read [11 4] INFO [2022-01-10 12:53:24,960] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:24,960] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:24,961] jepsen worker 87 - jepsen.util 87 :ok :write [29 9] INFO [2022-01-10 12:53:24,961] jepsen worker 13 - jepsen.util 13 :ok :read [4 40] INFO [2022-01-10 12:53:24,961] jepsen worker 33 - jepsen.util 123 :ok :read [11 4] INFO [2022-01-10 12:53:24,962] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:24,963] jepsen worker 13 - jepsen.util 13 :ok :read [4 40] INFO [2022-01-10 12:53:24,963] jepsen worker 87 - jepsen.util 87 :invoke :write [29 41] INFO [2022-01-10 12:53:24,963] jepsen worker 5 - jepsen.util 185 :ok :write [1 7] INFO [2022-01-10 12:53:24,963] jepsen worker 23 - jepsen.util 23 :ok :write [7 22] INFO [2022-01-10 12:53:24,963] jepsen worker 13 - jepsen.util 13 :invoke :write [4 5] INFO [2022-01-10 12:53:24,964] jepsen worker 5 - jepsen.util 185 :invoke :write [1 1] INFO [2022-01-10 12:53:24,965] jepsen worker 23 - jepsen.util 23 :invoke :write [7 6] INFO [2022-01-10 12:53:24,966] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:24,967] jepsen worker 33 - jepsen.util 123 :ok :read [11 4] INFO [2022-01-10 12:53:24,967] jepsen worker 33 - jepsen.util 123 :invoke :write [11 19] INFO [2022-01-10 12:53:24,970] jepsen worker 15 - jepsen.util 15 :ok :write [5 8] INFO [2022-01-10 12:53:24,971] jepsen worker 15 - jepsen.util 15 :invoke :write [5 2] INFO [2022-01-10 12:53:24,972] jepsen worker 40 - jepsen.util 130 :ok :write [13 40] INFO [2022-01-10 12:53:24,973] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,973] jepsen worker 40 - jepsen.util 130 :ok :read [13 40] INFO [2022-01-10 12:53:24,974] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,974] jepsen worker 84 - jepsen.util 84 :ok :write [28 49] INFO [2022-01-10 12:53:24,975] jepsen worker 40 - jepsen.util 130 :ok :read [13 40] INFO [2022-01-10 12:53:24,975] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:24,976] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:24,976] jepsen worker 84 - jepsen.util 84 :ok :read [28 49] INFO [2022-01-10 12:53:24,977] jepsen worker 40 - jepsen.util 130 :ok :read [13 40] INFO [2022-01-10 12:53:24,977] jepsen worker 84 - jepsen.util 84 :invoke :write [28 20] INFO [2022-01-10 12:53:24,978] jepsen worker 40 - jepsen.util 130 :invoke :write [13 27] INFO [2022-01-10 12:53:24,982] jepsen worker 37 - jepsen.util 127 :ok :write [12 14] INFO [2022-01-10 12:53:24,983] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:24,984] jepsen worker 37 - jepsen.util 127 :ok :read [12 14] INFO [2022-01-10 12:53:24,984] jepsen worker 37 - jepsen.util 127 :invoke :write [12 22] INFO [2022-01-10 12:53:25,074] jepsen worker 2 - jepsen.util 92 :ok :write [0 49] INFO [2022-01-10 12:53:25,075] jepsen worker 2 - jepsen.util 92 :invoke :write [0 8] INFO [2022-01-10 12:53:25,077] jepsen worker 53 - jepsen.util 143 :ok :write [17 43] INFO [2022-01-10 12:53:25,077] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,079] jepsen worker 53 - jepsen.util 143 :ok :read [17 43] INFO [2022-01-10 12:53:25,079] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,080] jepsen worker 53 - jepsen.util 143 :ok :read [17 43] INFO [2022-01-10 12:53:25,080] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,081] jepsen worker 53 - jepsen.util 143 :ok :read [17 43] INFO [2022-01-10 12:53:25,081] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,082] jepsen worker 80 - jepsen.util 260 :ok :write [26 35] INFO [2022-01-10 12:53:25,082] jepsen worker 80 - jepsen.util 260 :invoke :write [26 6] INFO [2022-01-10 12:53:25,082] jepsen worker 53 - jepsen.util 143 :ok :read [17 43] INFO [2022-01-10 12:53:25,083] jepsen worker 53 - jepsen.util 143 :invoke :write [17 30] INFO [2022-01-10 12:53:25,099] jepsen worker 5 - jepsen.util 185 :ok :write [1 1] INFO [2022-01-10 12:53:25,099] jepsen worker 5 - jepsen.util 185 :invoke :write [1 39] INFO [2022-01-10 12:53:25,106] jepsen worker 84 - jepsen.util 84 :ok :write [28 20] INFO [2022-01-10 12:53:25,106] jepsen worker 84 - jepsen.util 84 :invoke :write [28 12] INFO [2022-01-10 12:53:25,109] jepsen worker 37 - jepsen.util 127 :ok :write [12 22] INFO [2022-01-10 12:53:25,109] jepsen worker 37 - jepsen.util 127 :invoke :write [12 12] INFO [2022-01-10 12:53:25,134] jepsen worker 69 - jepsen.util 159 :ok :write [23 26] INFO [2022-01-10 12:53:25,134] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:53:25,135] jepsen worker 69 - jepsen.util 159 :invoke :write [23 49] INFO [2022-01-10 12:53:25,135] jepsen worker 50 - jepsen.util 50 :invoke :write [16 37] INFO [2022-01-10 12:53:25,139] jepsen worker 75 - jepsen.util 165 :ok :write [25 6] INFO [2022-01-10 12:53:25,139] jepsen worker 75 - jepsen.util 165 :invoke :write [25 1] INFO [2022-01-10 12:53:25,142] jepsen worker 87 - jepsen.util 87 :ok :write [29 41] INFO [2022-01-10 12:53:25,142] jepsen worker 87 - jepsen.util 87 :invoke :write [29 17] INFO [2022-01-10 12:53:25,144] jepsen worker 13 - jepsen.util 13 :ok :write [4 5] INFO [2022-01-10 12:53:25,144] jepsen worker 13 - jepsen.util 13 :invoke :write [4 15] INFO [2022-01-10 12:53:25,144] jepsen worker 81 - jepsen.util 171 :ok :write [27 0] INFO [2022-01-10 12:53:25,145] jepsen worker 81 - jepsen.util 171 :invoke :write [27 15] INFO [2022-01-10 12:53:25,147] jepsen worker 61 - jepsen.util 61 :ok :write [20 21] INFO [2022-01-10 12:53:25,148] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:25,149] jepsen worker 61 - jepsen.util 61 :ok :read [20 21] INFO [2022-01-10 12:53:25,149] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:25,150] jepsen worker 8 - jepsen.util 8 :ok :write [2 29] INFO [2022-01-10 12:53:25,151] jepsen worker 61 - jepsen.util 61 :ok :read [20 21] INFO [2022-01-10 12:53:25,151] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:25,152] jepsen worker 8 - jepsen.util 8 :ok :read [2 29] INFO [2022-01-10 12:53:25,152] jepsen worker 61 - jepsen.util 61 :invoke :write [20 37] INFO [2022-01-10 12:53:25,152] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:25,153] jepsen worker 53 - jepsen.util 143 :ok :write [17 30] INFO [2022-01-10 12:53:25,153] jepsen worker 8 - jepsen.util 8 :ok :read [2 29] INFO [2022-01-10 12:53:25,154] jepsen worker 53 - jepsen.util 143 :invoke :write [17 24] INFO [2022-01-10 12:53:25,154] jepsen worker 8 - jepsen.util 8 :invoke :write [2 28] INFO [2022-01-10 12:53:25,155] jepsen worker 5 - jepsen.util 185 :ok :write [1 39] INFO [2022-01-10 12:53:25,155] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:25,155] jepsen worker 68 - jepsen.util 68 :ok :write [22 7] INFO [2022-01-10 12:53:25,156] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:25,156] jepsen worker 5 - jepsen.util 185 :ok :read [1 39] INFO [2022-01-10 12:53:25,157] jepsen worker 68 - jepsen.util 68 :ok :read [22 7] INFO [2022-01-10 12:53:25,157] jepsen worker 5 - jepsen.util 185 :invoke :write [1 35] INFO [2022-01-10 12:53:25,158] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:25,158] jepsen worker 68 - jepsen.util 68 :ok :read [22 7] INFO [2022-01-10 12:53:25,159] jepsen worker 44 - jepsen.util 44 :ok :write [14 14] INFO [2022-01-10 12:53:25,159] jepsen worker 68 - jepsen.util 68 :invoke :write [22 16] INFO [2022-01-10 12:53:25,160] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:25,161] jepsen worker 44 - jepsen.util 44 :ok :read [14 14] INFO [2022-01-10 12:53:25,161] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:25,162] jepsen worker 80 - jepsen.util 260 :ok :write [26 6] INFO [2022-01-10 12:53:25,162] jepsen worker 44 - jepsen.util 44 :ok :read [14 14] INFO [2022-01-10 12:53:25,162] jepsen worker 84 - jepsen.util 84 :ok :write [28 12] INFO [2022-01-10 12:53:25,162] jepsen worker 80 - jepsen.util 260 :invoke :write [26 7] INFO [2022-01-10 12:53:25,163] jepsen worker 84 - jepsen.util 84 :invoke :write [28 46] INFO [2022-01-10 12:53:25,163] jepsen worker 40 - jepsen.util 130 :ok :write [13 27] INFO [2022-01-10 12:53:25,164] jepsen worker 33 - jepsen.util 123 :ok :write [11 19] INFO [2022-01-10 12:53:25,164] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:25,164] jepsen worker 44 - jepsen.util 44 :ok :read [14 14] INFO [2022-01-10 12:53:25,164] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,165] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:25,165] jepsen worker 40 - jepsen.util 130 :ok :read [13 27] INFO [2022-01-10 12:53:25,166] jepsen worker 33 - jepsen.util 123 :ok :read [11 19] INFO [2022-01-10 12:53:25,166] jepsen worker 44 - jepsen.util 44 :invoke :write [14 2] INFO [2022-01-10 12:53:25,167] jepsen worker 33 - jepsen.util 123 :invoke :write [11 45] INFO [2022-01-10 12:53:25,167] jepsen worker 40 - jepsen.util 130 :invoke :write [13 28] INFO [2022-01-10 12:53:25,170] jepsen worker 23 - jepsen.util 23 :ok :write [7 6] INFO [2022-01-10 12:53:25,170] jepsen worker 23 - jepsen.util 23 :invoke :write [7 40] INFO [2022-01-10 12:53:25,170] jepsen worker 2 - jepsen.util 92 :ok :write [0 8] INFO [2022-01-10 12:53:25,171] jepsen worker 2 - jepsen.util 92 :invoke :write [0 30] INFO [2022-01-10 12:53:25,171] jepsen worker 69 - jepsen.util 159 :ok :write [23 49] INFO [2022-01-10 12:53:25,171] jepsen worker 37 - jepsen.util 127 :ok :write [12 12] INFO [2022-01-10 12:53:25,171] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:25,172] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,172] jepsen worker 69 - jepsen.util 159 :ok :read [23 49] INFO [2022-01-10 12:53:25,173] jepsen worker 69 - jepsen.util 159 :invoke :write [23 32] INFO [2022-01-10 12:53:25,175] jepsen worker 37 - jepsen.util 127 :ok :read [12 12] INFO [2022-01-10 12:53:25,176] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,176] jepsen worker 37 - jepsen.util 127 :ok :read [12 12] INFO [2022-01-10 12:53:25,177] jepsen worker 5 - jepsen.util 185 :ok :write [1 35] INFO [2022-01-10 12:53:25,177] jepsen worker 13 - jepsen.util 13 :ok :write [4 15] INFO [2022-01-10 12:53:25,177] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,178] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:25,178] jepsen worker 37 - jepsen.util 127 :ok :read [12 12] INFO [2022-01-10 12:53:25,178] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:25,179] jepsen worker 13 - jepsen.util 13 :ok :read [4 15] INFO [2022-01-10 12:53:25,179] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,179] jepsen worker 5 - jepsen.util 185 :ok :read [1 35] INFO [2022-01-10 12:53:25,179] jepsen worker 13 - jepsen.util 13 :invoke :write [4 33] INFO [2022-01-10 12:53:25,180] jepsen worker 37 - jepsen.util 127 :ok :read [12 12] INFO [2022-01-10 12:53:25,180] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:25,181] jepsen worker 37 - jepsen.util 127 :invoke :write [12 8] INFO [2022-01-10 12:53:25,181] jepsen worker 15 - jepsen.util 15 :ok :write [5 2] INFO [2022-01-10 12:53:25,181] jepsen worker 5 - jepsen.util 185 :ok :read [1 35] INFO [2022-01-10 12:53:25,182] jepsen worker 53 - jepsen.util 143 :ok :write [17 24] INFO [2022-01-10 12:53:25,182] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:25,182] jepsen worker 5 - jepsen.util 185 :invoke :write [1 44] INFO [2022-01-10 12:53:25,183] jepsen worker 15 - jepsen.util 15 :ok :read [5 2] INFO [2022-01-10 12:53:25,183] jepsen worker 53 - jepsen.util 143 :invoke :write [17 29] INFO [2022-01-10 12:53:25,183] jepsen worker 15 - jepsen.util 15 :invoke :write [5 21] INFO [2022-01-10 12:53:25,186] jepsen worker 87 - jepsen.util 87 :ok :write [29 17] INFO [2022-01-10 12:53:25,187] jepsen worker 87 - jepsen.util 87 :invoke :write [29 36] INFO [2022-01-10 12:53:25,188] jepsen worker 50 - jepsen.util 50 :ok :write [16 37] INFO [2022-01-10 12:53:25,189] jepsen worker 80 - jepsen.util 260 :ok :write [26 7] INFO [2022-01-10 12:53:25,189] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:25,189] jepsen worker 80 - jepsen.util 260 :invoke :write [26 45] INFO [2022-01-10 12:53:25,190] jepsen worker 50 - jepsen.util 50 :ok :read [16 37] INFO [2022-01-10 12:53:25,191] jepsen worker 50 - jepsen.util 50 :invoke :write [16 34] INFO [2022-01-10 12:53:25,193] jepsen worker 75 - jepsen.util 165 :ok :write [25 1] INFO [2022-01-10 12:53:25,193] jepsen worker 75 - jepsen.util 165 :invoke :write [25 2] INFO [2022-01-10 12:53:25,195] jepsen worker 84 - jepsen.util 84 :ok :write [28 46] INFO [2022-01-10 12:53:25,195] jepsen worker 84 - jepsen.util 84 :invoke :write [28 26] INFO [2022-01-10 12:53:25,204] jepsen worker 37 - jepsen.util 127 :ok :write [12 8] INFO [2022-01-10 12:53:25,204] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,205] jepsen worker 37 - jepsen.util 127 :ok :read [12 8] INFO [2022-01-10 12:53:25,205] jepsen worker 37 - jepsen.util 127 :invoke :write [12 30] INFO [2022-01-10 12:53:25,208] jepsen worker 40 - jepsen.util 130 :ok :write [13 28] INFO [2022-01-10 12:53:25,209] jepsen worker 40 - jepsen.util 130 :invoke :write [13 30] INFO [2022-01-10 12:53:25,212] jepsen worker 5 - jepsen.util 185 :ok :write [1 44] INFO [2022-01-10 12:53:25,213] jepsen worker 5 - jepsen.util 185 :invoke :write [1 42] INFO [2022-01-10 12:53:25,221] jepsen worker 2 - jepsen.util 92 :ok :write [0 30] INFO [2022-01-10 12:53:25,221] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:25,222] jepsen worker 2 - jepsen.util 92 :ok :read [0 30] INFO [2022-01-10 12:53:25,223] jepsen worker 2 - jepsen.util 92 :invoke :write [0 26] INFO [2022-01-10 12:53:25,226] jepsen worker 84 - jepsen.util 84 :ok :write [28 26] INFO [2022-01-10 12:53:25,226] jepsen worker 84 - jepsen.util 84 :invoke :write [28 15] INFO [2022-01-10 12:53:25,232] jepsen worker 87 - jepsen.util 87 :ok :write [29 36] INFO [2022-01-10 12:53:25,232] jepsen worker 37 - jepsen.util 127 :ok :write [12 30] INFO [2022-01-10 12:53:25,233] jepsen worker 37 - jepsen.util 127 :invoke :write [12 28] INFO [2022-01-10 12:53:25,233] jepsen worker 87 - jepsen.util 87 :invoke :write [29 28] INFO [2022-01-10 12:53:25,238] jepsen worker 5 - jepsen.util 185 :ok :write [1 42] INFO [2022-01-10 12:53:25,239] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:25,240] jepsen worker 5 - jepsen.util 185 :ok :read [1 42] INFO [2022-01-10 12:53:25,240] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:25,241] jepsen worker 5 - jepsen.util 185 :ok :read [1 42] INFO [2022-01-10 12:53:25,241] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:25,242] jepsen worker 5 - jepsen.util 185 :ok :read [1 42] INFO [2022-01-10 12:53:25,242] jepsen worker 40 - jepsen.util 130 :ok :write [13 30] INFO [2022-01-10 12:53:25,243] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,243] jepsen worker 5 - jepsen.util 185 :invoke :write [1 32] INFO [2022-01-10 12:53:25,244] jepsen worker 40 - jepsen.util 130 :ok :read [13 30] INFO [2022-01-10 12:53:25,244] jepsen worker 40 - jepsen.util 130 :invoke :write [13 11] INFO [2022-01-10 12:53:25,245] jepsen worker 84 - jepsen.util 84 :ok :write [28 15] INFO [2022-01-10 12:53:25,246] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,248] jepsen worker 84 - jepsen.util 84 :ok :read [28 15] INFO [2022-01-10 12:53:25,248] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,249] jepsen worker 84 - jepsen.util 84 :ok :read [28 15] INFO [2022-01-10 12:53:25,249] jepsen worker 37 - jepsen.util 127 :ok :write [12 28] INFO [2022-01-10 12:53:25,249] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,250] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,250] jepsen worker 2 - jepsen.util 92 :ok :write [0 26] INFO [2022-01-10 12:53:25,251] jepsen worker 37 - jepsen.util 127 :ok :read [12 28] INFO [2022-01-10 12:53:25,251] jepsen worker 2 - jepsen.util 92 :invoke :write [0 0] INFO [2022-01-10 12:53:25,251] jepsen worker 84 - jepsen.util 84 :ok :read [28 15] INFO [2022-01-10 12:53:25,251] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,252] jepsen worker 84 - jepsen.util 84 :invoke :write [28 37] INFO [2022-01-10 12:53:25,252] jepsen worker 37 - jepsen.util 127 :ok :read [12 28] INFO [2022-01-10 12:53:25,253] jepsen worker 37 - jepsen.util 127 :invoke :write [12 8] INFO [2022-01-10 12:53:25,258] jepsen worker 87 - jepsen.util 87 :ok :write [29 28] INFO [2022-01-10 12:53:25,259] jepsen worker 87 - jepsen.util 87 :invoke :write [29 32] INFO [2022-01-10 12:53:25,262] jepsen worker 5 - jepsen.util 185 :ok :write [1 32] INFO [2022-01-10 12:53:25,263] jepsen worker 5 - jepsen.util 185 :invoke :write [1 27] INFO [2022-01-10 12:53:25,263] jepsen worker 40 - jepsen.util 130 :ok :write [13 11] INFO [2022-01-10 12:53:25,264] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,265] jepsen worker 40 - jepsen.util 130 :ok :read [13 11] INFO [2022-01-10 12:53:25,265] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,266] jepsen worker 40 - jepsen.util 130 :ok :read [13 11] INFO [2022-01-10 12:53:25,267] jepsen worker 40 - jepsen.util 130 :invoke :write [13 32] INFO [2022-01-10 12:53:25,270] jepsen worker 2 - jepsen.util 92 :ok :write [0 0] INFO [2022-01-10 12:53:25,271] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:25,272] jepsen worker 37 - jepsen.util 127 :ok :write [12 8] INFO [2022-01-10 12:53:25,272] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:25,272] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,273] jepsen worker 2 - jepsen.util 92 :invoke :write [0 40] INFO [2022-01-10 12:53:25,273] jepsen worker 37 - jepsen.util 127 :ok :read [12 8] INFO [2022-01-10 12:53:25,273] jepsen worker 37 - jepsen.util 127 :invoke :write [12 49] INFO [2022-01-10 12:53:25,277] jepsen worker 84 - jepsen.util 84 :ok :write [28 37] INFO [2022-01-10 12:53:25,278] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,278] jepsen worker 87 - jepsen.util 87 :ok :write [29 32] INFO [2022-01-10 12:53:25,279] jepsen worker 84 - jepsen.util 84 :ok :read [28 37] INFO [2022-01-10 12:53:25,279] jepsen worker 87 - jepsen.util 87 :invoke :write [29 24] INFO [2022-01-10 12:53:25,280] jepsen worker 84 - jepsen.util 84 :invoke :write [28 8] INFO [2022-01-10 12:53:25,284] jepsen worker 5 - jepsen.util 185 :ok :write [1 27] INFO [2022-01-10 12:53:25,284] jepsen worker 40 - jepsen.util 130 :ok :write [13 32] INFO [2022-01-10 12:53:25,285] jepsen worker 5 - jepsen.util 185 :invoke :write [1 48] INFO [2022-01-10 12:53:25,285] jepsen worker 40 - jepsen.util 130 :invoke :write [13 37] INFO [2022-01-10 12:53:25,291] jepsen worker 2 - jepsen.util 92 :ok :write [0 40] INFO [2022-01-10 12:53:25,291] jepsen worker 37 - jepsen.util 127 :ok :write [12 49] INFO [2022-01-10 12:53:25,291] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:25,292] jepsen worker 2 - jepsen.util 92 :ok :read [0 40] INFO [2022-01-10 12:53:25,292] jepsen worker 37 - jepsen.util 127 :invoke :write [12 7] INFO [2022-01-10 12:53:25,293] jepsen worker 2 - jepsen.util 92 :invoke :write [0 19] INFO [2022-01-10 12:53:25,297] jepsen worker 84 - jepsen.util 84 :ok :write [28 8] INFO [2022-01-10 12:53:25,297] jepsen worker 87 - jepsen.util 87 :ok :write [29 24] INFO [2022-01-10 12:53:25,297] jepsen worker 84 - jepsen.util 84 :invoke :write [28 24] INFO [2022-01-10 12:53:25,297] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:25,298] jepsen worker 87 - jepsen.util 87 :ok :read [29 24] INFO [2022-01-10 12:53:25,299] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:25,299] jepsen worker 87 - jepsen.util 87 :ok :read [29 24] INFO [2022-01-10 12:53:25,300] jepsen worker 87 - jepsen.util 87 :invoke :write [29 10] INFO [2022-01-10 12:53:25,305] jepsen worker 40 - jepsen.util 130 :ok :write [13 37] INFO [2022-01-10 12:53:25,305] jepsen worker 5 - jepsen.util 185 :ok :write [1 48] INFO [2022-01-10 12:53:25,306] jepsen worker 5 - jepsen.util 185 :invoke :write [1 42] INFO [2022-01-10 12:53:25,306] jepsen worker 40 - jepsen.util 130 :invoke :write [13 48] INFO [2022-01-10 12:53:25,345] jepsen worker 14 - jepsen.util 104 :info :write [4 19] :RequestTimeoutException INFO [2022-01-10 12:53:25,345] jepsen worker 14 - jepsen.util 194 :invoke :write [4 23] INFO [2022-01-10 12:53:25,369] jepsen worker 69 - jepsen.util 159 :ok :write [23 32] INFO [2022-01-10 12:53:25,369] jepsen worker 81 - jepsen.util 171 :ok :write [27 15] INFO [2022-01-10 12:53:25,370] jepsen worker 69 - jepsen.util 159 :invoke :write [23 26] INFO [2022-01-10 12:53:25,370] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:25,371] jepsen worker 81 - jepsen.util 171 :ok :read [27 15] INFO [2022-01-10 12:53:25,371] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:25,372] jepsen worker 81 - jepsen.util 171 :ok :read [27 15] INFO [2022-01-10 12:53:25,372] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:25,373] jepsen worker 81 - jepsen.util 171 :ok :read [27 15] INFO [2022-01-10 12:53:25,374] jepsen worker 81 - jepsen.util 171 :invoke :write [27 43] INFO [2022-01-10 12:53:25,374] jepsen worker 13 - jepsen.util 13 :ok :write [4 33] INFO [2022-01-10 12:53:25,375] jepsen worker 13 - jepsen.util 13 :invoke :write [4 43] INFO [2022-01-10 12:53:25,377] jepsen worker 53 - jepsen.util 143 :ok :write [17 29] INFO [2022-01-10 12:53:25,378] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,379] jepsen worker 53 - jepsen.util 143 :ok :read [17 29] INFO [2022-01-10 12:53:25,379] jepsen worker 61 - jepsen.util 61 :ok :write [20 37] INFO [2022-01-10 12:53:25,379] jepsen worker 53 - jepsen.util 143 :invoke :write [17 38] INFO [2022-01-10 12:53:25,380] jepsen worker 61 - jepsen.util 61 :invoke :write [20 11] INFO [2022-01-10 12:53:25,382] jepsen worker 80 - jepsen.util 260 :ok :write [26 45] INFO [2022-01-10 12:53:25,382] jepsen worker 80 - jepsen.util 260 :invoke :write [26 42] INFO [2022-01-10 12:53:25,384] jepsen worker 8 - jepsen.util 8 :ok :write [2 28] INFO [2022-01-10 12:53:25,385] jepsen worker 8 - jepsen.util 8 :invoke :write [2 11] INFO [2022-01-10 12:53:25,391] jepsen worker 68 - jepsen.util 68 :ok :write [22 16] INFO [2022-01-10 12:53:25,391] jepsen worker 68 - jepsen.util 68 :invoke :write [22 42] INFO [2022-01-10 12:53:25,397] jepsen worker 44 - jepsen.util 44 :ok :write [14 2] INFO [2022-01-10 12:53:25,398] jepsen worker 44 - jepsen.util 44 :invoke :write [14 41] INFO [2022-01-10 12:53:25,405] jepsen worker 33 - jepsen.util 123 :ok :write [11 45] INFO [2022-01-10 12:53:25,405] jepsen worker 33 - jepsen.util 123 :invoke :write [11 44] INFO [2022-01-10 12:53:25,410] jepsen worker 23 - jepsen.util 23 :ok :write [7 40] INFO [2022-01-10 12:53:25,411] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:25,412] jepsen worker 23 - jepsen.util 23 :ok :read [7 40] INFO [2022-01-10 12:53:25,412] jepsen worker 23 - jepsen.util 23 :invoke :write [7 47] INFO [2022-01-10 12:53:25,417] jepsen worker 15 - jepsen.util 15 :ok :write [5 21] INFO [2022-01-10 12:53:25,417] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:25,418] jepsen worker 15 - jepsen.util 15 :ok :read [5 21] INFO [2022-01-10 12:53:25,419] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:25,419] jepsen worker 15 - jepsen.util 15 :ok :read [5 21] INFO [2022-01-10 12:53:25,420] jepsen worker 15 - jepsen.util 15 :invoke :write [5 38] INFO [2022-01-10 12:53:25,423] jepsen worker 50 - jepsen.util 50 :ok :write [16 34] INFO [2022-01-10 12:53:25,424] jepsen worker 50 - jepsen.util 50 :invoke :write [16 20] INFO [2022-01-10 12:53:25,427] jepsen worker 75 - jepsen.util 165 :ok :write [25 2] INFO [2022-01-10 12:53:25,427] jepsen worker 75 - jepsen.util 165 :invoke :write [25 5] INFO [2022-01-10 12:53:25,432] jepsen worker 67 - jepsen.util 67 :info :write [22 48] :RequestTimeoutException INFO [2022-01-10 12:53:25,432] jepsen worker 81 - jepsen.util 171 :ok :write [27 43] INFO [2022-01-10 12:53:25,433] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:25,434] jepsen worker 67 - jepsen.util 157 :invoke :write [22 17] INFO [2022-01-10 12:53:25,434] jepsen worker 81 - jepsen.util 171 :ok :read [27 43] INFO [2022-01-10 12:53:25,435] jepsen worker 81 - jepsen.util 171 :invoke :write [27 22] INFO [2022-01-10 12:53:25,440] jepsen worker 8 - jepsen.util 8 :ok :write [2 11] INFO [2022-01-10 12:53:25,440] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:25,441] jepsen worker 8 - jepsen.util 8 :ok :read [2 11] INFO [2022-01-10 12:53:25,442] jepsen worker 8 - jepsen.util 8 :invoke :write [2 38] INFO [2022-01-10 12:53:25,447] jepsen worker 68 - jepsen.util 68 :ok :write [22 42] INFO [2022-01-10 12:53:25,448] jepsen worker 68 - jepsen.util 68 :invoke :write [22 9] INFO [2022-01-10 12:53:25,455] jepsen worker 33 - jepsen.util 123 :ok :write [11 44] INFO [2022-01-10 12:53:25,456] jepsen worker 33 - jepsen.util 123 :invoke :write [11 20] INFO [2022-01-10 12:53:25,461] jepsen worker 37 - jepsen.util 127 :ok :write [12 7] INFO [2022-01-10 12:53:25,461] jepsen worker 37 - jepsen.util 127 :invoke :write [12 41] INFO [2022-01-10 12:53:25,466] jepsen worker 15 - jepsen.util 15 :ok :write [5 38] INFO [2022-01-10 12:53:25,467] jepsen worker 15 - jepsen.util 15 :invoke :write [5 39] INFO [2022-01-10 12:53:25,467] jepsen worker 2 - jepsen.util 92 :ok :write [0 19] INFO [2022-01-10 12:53:25,468] jepsen worker 2 - jepsen.util 92 :invoke :write [0 12] INFO [2022-01-10 12:53:25,469] jepsen worker 84 - jepsen.util 84 :ok :write [28 24] INFO [2022-01-10 12:53:25,470] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,470] jepsen worker 84 - jepsen.util 84 :ok :read [28 24] INFO [2022-01-10 12:53:25,471] jepsen worker 50 - jepsen.util 50 :ok :write [16 20] INFO [2022-01-10 12:53:25,471] jepsen worker 84 - jepsen.util 84 :invoke :write [28 47] INFO [2022-01-10 12:53:25,472] jepsen worker 87 - jepsen.util 87 :ok :write [29 10] INFO [2022-01-10 12:53:25,472] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:25,472] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:25,473] jepsen worker 50 - jepsen.util 50 :ok :read [16 20] INFO [2022-01-10 12:53:25,473] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:25,473] jepsen worker 87 - jepsen.util 87 :ok :read [29 10] INFO [2022-01-10 12:53:25,474] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:25,474] jepsen worker 50 - jepsen.util 50 :ok :read [16 20] INFO [2022-01-10 12:53:25,475] jepsen worker 87 - jepsen.util 87 :ok :read [29 10] INFO [2022-01-10 12:53:25,475] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:25,475] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:25,476] jepsen worker 50 - jepsen.util 50 :ok :read [16 20] INFO [2022-01-10 12:53:25,476] jepsen worker 5 - jepsen.util 185 :ok :write [1 42] INFO [2022-01-10 12:53:25,477] jepsen worker 50 - jepsen.util 50 :invoke :write [16 27] INFO [2022-01-10 12:53:25,477] jepsen worker 5 - jepsen.util 185 :invoke :write [1 20] INFO [2022-01-10 12:53:25,477] jepsen worker 87 - jepsen.util 87 :ok :read [29 10] INFO [2022-01-10 12:53:25,478] jepsen worker 87 - jepsen.util 87 :invoke :write [29 36] INFO [2022-01-10 12:53:25,480] jepsen worker 81 - jepsen.util 171 :ok :write [27 22] INFO [2022-01-10 12:53:25,480] jepsen worker 40 - jepsen.util 130 :ok :write [13 48] INFO [2022-01-10 12:53:25,481] jepsen worker 81 - jepsen.util 171 :invoke :write [27 42] INFO [2022-01-10 12:53:25,481] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,482] jepsen worker 69 - jepsen.util 159 :ok :write [23 26] INFO [2022-01-10 12:53:25,482] jepsen worker 40 - jepsen.util 130 :ok :read [13 48] INFO [2022-01-10 12:53:25,483] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:25,483] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,483] jepsen worker 69 - jepsen.util 159 :ok :read [23 26] INFO [2022-01-10 12:53:25,483] jepsen worker 8 - jepsen.util 8 :ok :write [2 38] INFO [2022-01-10 12:53:25,484] jepsen worker 40 - jepsen.util 130 :ok :read [13 48] INFO [2022-01-10 12:53:25,485] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:25,485] jepsen worker 61 - jepsen.util 61 :ok :write [20 11] INFO [2022-01-10 12:53:25,485] jepsen worker 8 - jepsen.util 8 :ok :read [2 38] INFO [2022-01-10 12:53:25,485] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,486] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:25,486] jepsen worker 13 - jepsen.util 13 :ok :write [4 43] INFO [2022-01-10 12:53:25,487] jepsen worker 40 - jepsen.util 130 :ok :read [13 48] INFO [2022-01-10 12:53:25,487] jepsen worker 69 - jepsen.util 159 :invoke :write [23 9] INFO [2022-01-10 12:53:25,487] jepsen worker 8 - jepsen.util 8 :ok :read [2 38] INFO [2022-01-10 12:53:25,488] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,489] jepsen worker 61 - jepsen.util 61 :invoke :write [20 13] INFO [2022-01-10 12:53:25,489] jepsen worker 40 - jepsen.util 130 :ok :read [13 48] INFO [2022-01-10 12:53:25,490] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:25,490] jepsen worker 13 - jepsen.util 13 :invoke :write [4 12] INFO [2022-01-10 12:53:25,491] jepsen worker 8 - jepsen.util 8 :ok :read [2 38] INFO [2022-01-10 12:53:25,491] jepsen worker 40 - jepsen.util 130 :invoke :write [13 22] INFO [2022-01-10 12:53:25,491] jepsen worker 53 - jepsen.util 143 :ok :write [17 38] INFO [2022-01-10 12:53:25,491] jepsen worker 8 - jepsen.util 8 :invoke :write [2 42] INFO [2022-01-10 12:53:25,492] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,493] jepsen worker 53 - jepsen.util 143 :ok :read [17 38] INFO [2022-01-10 12:53:25,493] jepsen worker 53 - jepsen.util 143 :invoke :write [17 5] INFO [2022-01-10 12:53:25,495] jepsen worker 44 - jepsen.util 44 :ok :write [14 41] INFO [2022-01-10 12:53:25,495] jepsen worker 44 - jepsen.util 44 :invoke :write [14 29] INFO [2022-01-10 12:53:25,498] jepsen worker 80 - jepsen.util 260 :ok :write [26 42] INFO [2022-01-10 12:53:25,498] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:25,499] jepsen worker 80 - jepsen.util 260 :ok :read [26 42] INFO [2022-01-10 12:53:25,499] jepsen worker 68 - jepsen.util 68 :ok :write [22 9] INFO [2022-01-10 12:53:25,500] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:25,500] jepsen worker 80 - jepsen.util 260 :ok :read [26 42] INFO [2022-01-10 12:53:25,501] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:25,501] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:25,501] jepsen worker 68 - jepsen.util 68 :ok :read [22 9] INFO [2022-01-10 12:53:25,502] jepsen worker 23 - jepsen.util 23 :ok :write [7 47] INFO [2022-01-10 12:53:25,502] jepsen worker 80 - jepsen.util 260 :ok :read [26 42] INFO [2022-01-10 12:53:25,502] jepsen worker 68 - jepsen.util 68 :invoke :write [22 3] INFO [2022-01-10 12:53:25,503] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:25,503] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:25,503] jepsen worker 37 - jepsen.util 127 :ok :write [12 41] INFO [2022-01-10 12:53:25,504] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,504] jepsen worker 80 - jepsen.util 260 :ok :read [26 42] INFO [2022-01-10 12:53:25,504] jepsen worker 23 - jepsen.util 23 :ok :read [7 47] INFO [2022-01-10 12:53:25,505] jepsen worker 37 - jepsen.util 127 :ok :read [12 41] INFO [2022-01-10 12:53:25,505] jepsen worker 23 - jepsen.util 23 :invoke :write [7 8] INFO [2022-01-10 12:53:25,505] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,506] jepsen worker 80 - jepsen.util 260 :invoke :write [26 13] INFO [2022-01-10 12:53:25,506] jepsen worker 37 - jepsen.util 127 :ok :read [12 41] INFO [2022-01-10 12:53:25,507] jepsen worker 37 - jepsen.util 127 :invoke :write [12 28] INFO [2022-01-10 12:53:25,507] jepsen worker 75 - jepsen.util 165 :ok :write [25 5] INFO [2022-01-10 12:53:25,508] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:25,509] jepsen worker 75 - jepsen.util 165 :ok :read [25 5] INFO [2022-01-10 12:53:25,509] jepsen worker 84 - jepsen.util 84 :ok :write [28 47] INFO [2022-01-10 12:53:25,509] jepsen worker 75 - jepsen.util 165 :invoke :write [25 12] INFO [2022-01-10 12:53:25,510] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,511] jepsen worker 33 - jepsen.util 123 :ok :write [11 20] INFO [2022-01-10 12:53:25,511] jepsen worker 84 - jepsen.util 84 :ok :read [28 47] INFO [2022-01-10 12:53:25,511] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,512] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:25,512] jepsen worker 84 - jepsen.util 84 :ok :read [28 47] INFO [2022-01-10 12:53:25,513] jepsen worker 33 - jepsen.util 123 :ok :read [11 20] INFO [2022-01-10 12:53:25,513] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,513] jepsen worker 33 - jepsen.util 123 :invoke :write [11 42] INFO [2022-01-10 12:53:25,514] jepsen worker 84 - jepsen.util 84 :ok :read [28 47] INFO [2022-01-10 12:53:25,514] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,514] jepsen worker 2 - jepsen.util 92 :ok :write [0 12] INFO [2022-01-10 12:53:25,515] jepsen worker 84 - jepsen.util 84 :ok :read [28 47] INFO [2022-01-10 12:53:25,515] jepsen worker 4 - jepsen.util 4 :ok :write [1 44] INFO [2022-01-10 12:53:25,515] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:25,516] jepsen worker 4 - jepsen.util 4 :invoke :write [1 32] INFO [2022-01-10 12:53:25,516] jepsen worker 2 - jepsen.util 92 :ok :read [0 12] INFO [2022-01-10 12:53:25,516] jepsen worker 84 - jepsen.util 84 :invoke :write [28 42] INFO [2022-01-10 12:53:25,517] jepsen worker 2 - jepsen.util 92 :invoke :write [0 9] INFO [2022-01-10 12:53:25,518] jepsen worker 15 - jepsen.util 15 :ok :write [5 39] INFO [2022-01-10 12:53:25,519] jepsen worker 15 - jepsen.util 15 :invoke :write [5 16] INFO [2022-01-10 12:53:25,520] jepsen worker 69 - jepsen.util 159 :ok :write [23 9] INFO [2022-01-10 12:53:25,521] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:25,522] jepsen worker 69 - jepsen.util 159 :ok :read [23 9] INFO [2022-01-10 12:53:25,522] jepsen worker 50 - jepsen.util 50 :ok :write [16 27] INFO [2022-01-10 12:53:25,523] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:25,523] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:25,524] jepsen worker 50 - jepsen.util 50 :ok :read [16 27] INFO [2022-01-10 12:53:25,524] jepsen worker 50 - jepsen.util 50 :invoke :write [16 48] INFO [2022-01-10 12:53:25,524] jepsen worker 69 - jepsen.util 159 :ok :read [23 9] INFO [2022-01-10 12:53:25,525] jepsen worker 69 - jepsen.util 159 :invoke :write [23 36] INFO [2022-01-10 12:53:25,527] jepsen worker 13 - jepsen.util 13 :ok :write [4 12] INFO [2022-01-10 12:53:25,528] jepsen worker 13 - jepsen.util 13 :invoke :write [4 44] INFO [2022-01-10 12:53:25,532] jepsen worker 53 - jepsen.util 143 :ok :write [17 5] INFO [2022-01-10 12:53:25,533] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,534] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:25,534] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,535] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:25,535] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,536] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:25,537] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,537] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:25,538] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,539] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:25,539] jepsen worker 80 - jepsen.util 260 :ok :write [26 13] INFO [2022-01-10 12:53:25,539] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,539] jepsen worker 80 - jepsen.util 260 :invoke :write [26 9] INFO [2022-01-10 12:53:25,540] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:25,540] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,541] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:25,542] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,542] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:25,543] jepsen worker 53 - jepsen.util 143 :invoke :write [17 21] WARN [2022-01-10 12:53:25,552] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":2308304,"last_local_id":"2B94F89700000004/0000000077268043","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":2308304,"last_local_socket":"172.23.104.79:34634","total_server_duration_us":56,"operation_id":"0x2c9c","timeout_ms":2000,"last_server_duration_us":56,"total_duration_us":2005148}],"total_count":1}} WARN [2022-01-10 12:53:25,552] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7043225,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":7053667,"total_server_duration_us":1669,"encode_duration_us":18,"operation_id":"0x1f7c","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10010564},{"operation_name":"upsert","last_dispatch_duration_us":526,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000004/00000000C4B4FBBF","last_local_socket":"172.23.104.79:58638","total_dispatch_duration_us":17865,"total_server_duration_us":2903,"encode_duration_us":49,"operation_id":"0x18a7","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10007867},{"operation_name":"upsert","last_dispatch_duration_us":642,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":18258,"total_server_duration_us":2740,"encode_duration_us":31,"operation_id":"0x1f91","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10007620},{"operation_name":"upsert","last_dispatch_duration_us":7192473,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":7209093,"total_server_duration_us":2214,"encode_duration_us":28,"operation_id":"0x1a0d","timeout_ms":10000,"last_server_duration_us":84,"total_duration_us":10005174},{"operation_name":"upsert","last_dispatch_duration_us":608,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":20782,"total_server_duration_us":2772,"encode_duration_us":32,"operation_id":"0x1f40","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10004084},{"operation_name":"upsert","last_dispatch_duration_us":543,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":18496,"total_server_duration_us":2655,"encode_duration_us":20,"operation_id":"0x20fe","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10002595},{"operation_name":"upsert","last_dispatch_duration_us":544,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000004/0000000048226CCA","last_local_socket":"172.23.104.79:48384","total_dispatch_duration_us":58004,"total_server_duration_us":2491,"encode_duration_us":27,"operation_id":"0x1f59","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10002439},{"operation_name":"upsert","last_dispatch_duration_us":508,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000004/0000000048226CCA","last_local_socket":"172.23.104.79:48384","total_dispatch_duration_us":17505,"total_server_duration_us":2722,"encode_duration_us":51,"operation_id":"0x161f","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10001908},{"operation_name":"upsert","last_dispatch_duration_us":9016519,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":9016519,"total_server_duration_us":9015090,"encode_duration_us":20,"operation_id":"0x20e7","timeout_ms":10000,"last_server_duration_us":9015090,"total_duration_us":9017186},{"operation_name":"upsert","last_dispatch_duration_us":152589,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":7269470,"total_server_duration_us":152322,"encode_duration_us":36,"operation_id":"0x24b1","timeout_ms":10000,"last_server_duration_us":151432,"total_duration_us":8348571}],"total_count":16}} INFO [2022-01-10 12:53:25,559] jepsen worker 69 - jepsen.util 159 :ok :write [23 36] INFO [2022-01-10 12:53:25,560] jepsen worker 69 - jepsen.util 159 :invoke :write [23 13] INFO [2022-01-10 12:53:25,563] jepsen worker 13 - jepsen.util 13 :ok :write [4 44] INFO [2022-01-10 12:53:25,564] jepsen worker 13 - jepsen.util 13 :invoke :write [4 28] INFO [2022-01-10 12:53:25,644] jepsen worker 87 - jepsen.util 87 :ok :write [29 36] INFO [2022-01-10 12:53:25,645] jepsen worker 87 - jepsen.util 87 :invoke :write [29 28] INFO [2022-01-10 12:53:25,647] jepsen worker 37 - jepsen.util 127 :ok :write [12 28] INFO [2022-01-10 12:53:25,648] jepsen worker 37 - jepsen.util 127 :invoke :write [12 40] INFO [2022-01-10 12:53:25,649] jepsen worker 40 - jepsen.util 130 :ok :write [13 22] INFO [2022-01-10 12:53:25,649] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,650] jepsen worker 40 - jepsen.util 130 :ok :read [13 22] WARN [2022-01-10 12:53:25,651] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":546,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000002/00000000F0645BC3","last_local_socket":"172.23.104.79:34668","total_dispatch_duration_us":1498218,"total_server_duration_us":2670,"encode_duration_us":39,"operation_id":"0x1fc1","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10010920},{"operation_name":"upsert","last_dispatch_duration_us":1027443,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000002/00000000F0645BC3","last_local_socket":"172.23.104.79:34668","total_dispatch_duration_us":1077705,"total_server_duration_us":3529,"encode_duration_us":20,"operation_id":"0x1a5b","timeout_ms":10000,"last_server_duration_us":38,"total_duration_us":10006753},{"operation_name":"upsert","last_dispatch_duration_us":5922242,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","total_dispatch_duration_us":5922242,"total_server_duration_us":27,"encode_duration_us":24,"operation_id":"0x26a7","timeout_ms":10000,"last_server_duration_us":27,"total_duration_us":10005429},{"operation_name":"upsert","last_dispatch_duration_us":6106587,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","total_dispatch_duration_us":6117202,"total_server_duration_us":1414,"encode_duration_us":34,"operation_id":"0x1f5b","timeout_ms":10000,"last_server_duration_us":84,"total_duration_us":10004295},{"operation_name":"upsert","last_dispatch_duration_us":504,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","total_dispatch_duration_us":15533,"total_server_duration_us":1316,"encode_duration_us":25,"operation_id":"0x1f7d","timeout_ms":10000,"last_server_duration_us":56,"total_duration_us":10004000},{"operation_name":"upsert","last_dispatch_duration_us":5949444,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","total_dispatch_duration_us":5949444,"total_server_duration_us":49,"encode_duration_us":27,"operation_id":"0x269b","timeout_ms":10000,"last_server_duration_us":49,"total_duration_us":10002933},{"operation_name":"upsert","last_dispatch_duration_us":663,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":6150228,"total_server_duration_us":1348,"encode_duration_us":48,"operation_id":"0x150d","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10001956},{"operation_name":"upsert","last_dispatch_duration_us":9009577,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":9009577,"total_server_duration_us":9008729,"encode_duration_us":20,"operation_id":"0x2128","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":9010076},{"operation_name":"upsert","last_dispatch_duration_us":9006849,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","total_dispatch_duration_us":9006849,"total_server_duration_us":9006609,"encode_duration_us":23,"operation_id":"0x20f7","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":9007208},{"operation_name":"upsert","last_dispatch_duration_us":27156,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":996675,"total_server_duration_us":28296,"encode_duration_us":20,"operation_id":"0x20ce","timeout_ms":10000,"last_server_duration_us":26596,"total_duration_us":8194995}],"total_count":23}} WARN [2022-01-10 12:53:25,651] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":637,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":11627,"total_server_duration_us":1461,"encode_duration_us":25,"operation_id":"0x20e2","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10010524},{"operation_name":"upsert","last_dispatch_duration_us":576,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":21862,"total_server_duration_us":3615,"encode_duration_us":26,"operation_id":"0x2011","timeout_ms":10000,"last_server_duration_us":43,"total_duration_us":10007910},{"operation_name":"upsert","last_dispatch_duration_us":874,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":372801,"total_server_duration_us":3704,"encode_duration_us":21,"operation_id":"0x2780","timeout_ms":10000,"last_server_duration_us":38,"total_duration_us":10006764},{"operation_name":"upsert","last_dispatch_duration_us":726,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":11821,"total_server_duration_us":1424,"encode_duration_us":29,"operation_id":"0x20e6","timeout_ms":10000,"last_server_duration_us":117,"total_duration_us":10005755},{"operation_name":"upsert","last_dispatch_duration_us":3794967,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":3807356,"total_server_duration_us":1884,"encode_duration_us":34,"operation_id":"0x20fb","timeout_ms":10000,"last_server_duration_us":421,"total_duration_us":10005605},{"operation_name":"upsert","last_dispatch_duration_us":669,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":19240,"total_server_duration_us":3080,"encode_duration_us":33,"operation_id":"0x20e4","timeout_ms":10000,"last_server_duration_us":100,"total_duration_us":10005187},{"operation_name":"upsert","last_dispatch_duration_us":3865254,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":4665755,"total_server_duration_us":1522,"encode_duration_us":23,"operation_id":"0x1f1a","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10003986},{"operation_name":"upsert","last_dispatch_duration_us":9011824,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":9699614,"total_server_duration_us":9010895,"encode_duration_us":49,"operation_id":"0x1f58","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":9715092},{"operation_name":"upsert","last_dispatch_duration_us":102668,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":118393,"total_server_duration_us":104142,"encode_duration_us":31,"operation_id":"0x2034","timeout_ms":10000,"last_server_duration_us":102011,"total_duration_us":8326992},{"operation_name":"upsert","last_dispatch_duration_us":81396,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000003/000000000DBBABF1","last_local_socket":"172.23.104.79:34702","total_dispatch_duration_us":99469,"total_server_duration_us":46194,"encode_duration_us":27,"operation_id":"0x2137","timeout_ms":10000,"last_server_duration_us":39740,"total_duration_us":4740732}],"total_count":24}} INFO [2022-01-10 12:53:25,651] jepsen worker 40 - jepsen.util 130 :invoke :write [13 37] WARN [2022-01-10 12:53:25,651] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":10349837,"last_local_id":"2B94F89700000005/000000004F9479E2","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":10349837,"last_local_socket":"172.23.104.79:40240","total_server_duration_us":32,"operation_id":"0x1df4","timeout_ms":2000,"last_server_duration_us":32,"total_duration_us":2008909},{"operation_name":"upsert","last_dispatch_duration_us":15731178,"last_local_id":"2B94F89700000005/000000004F9479E2","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":15745643,"last_local_socket":"172.23.104.79:40240","total_server_duration_us":9017845,"encode_duration_us":80,"operation_id":"0xd60","timeout_ms":10000,"last_server_duration_us":9016150,"total_duration_us":10008291}],"total_count":2}} WARN [2022-01-10 12:53:25,652] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":872,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000005/0000000090CEECC6","last_local_socket":"172.23.104.79:48368","total_dispatch_duration_us":21862,"total_server_duration_us":2717,"encode_duration_us":25,"operation_id":"0x2119","timeout_ms":10000,"last_server_duration_us":186,"total_duration_us":10007412},{"operation_name":"upsert","last_dispatch_duration_us":787,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":8347971,"total_server_duration_us":281,"encode_duration_us":27,"operation_id":"0x2247","timeout_ms":10000,"last_server_duration_us":23,"total_duration_us":10007100},{"operation_name":"upsert","last_dispatch_duration_us":779,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":8927813,"total_server_duration_us":264,"encode_duration_us":43,"operation_id":"0x2126","timeout_ms":10000,"last_server_duration_us":27,"total_duration_us":10006773},{"operation_name":"upsert","last_dispatch_duration_us":806,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":9792316,"total_server_duration_us":455,"encode_duration_us":30,"operation_id":"0x1f50","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10006300},{"operation_name":"upsert","last_dispatch_duration_us":745,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":9066954,"total_server_duration_us":377,"encode_duration_us":20,"operation_id":"0x20e9","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10005942},{"operation_name":"upsert","last_dispatch_duration_us":985,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":1127475,"total_server_duration_us":4969,"encode_duration_us":28,"operation_id":"0x1f5a","timeout_ms":10000,"last_server_duration_us":145,"total_duration_us":10003730},{"operation_name":"upsert","last_dispatch_duration_us":855,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":9693071,"total_server_duration_us":390,"encode_duration_us":58,"operation_id":"0x1f8d","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10003222},{"operation_name":"upsert","last_dispatch_duration_us":1009,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":856312,"total_server_duration_us":2368,"encode_duration_us":87,"operation_id":"0x201d","timeout_ms":10000,"last_server_duration_us":108,"total_duration_us":10001801},{"operation_name":"upsert","last_dispatch_duration_us":38833,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":58516,"total_server_duration_us":40807,"encode_duration_us":88,"operation_id":"0x2d71","timeout_ms":10000,"last_server_duration_us":38170,"total_duration_us":6744328},{"operation_name":"upsert","last_dispatch_duration_us":396836,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000005/000000006FA84E55","last_local_socket":"172.23.104.79:34638","total_dispatch_duration_us":406724,"total_server_duration_us":397832,"encode_duration_us":33,"operation_id":"0x2cdb","timeout_ms":10000,"last_server_duration_us":396248,"total_duration_us":4031595}],"total_count":24}} INFO [2022-01-10 12:53:25,655] jepsen worker 81 - jepsen.util 171 :ok :write [27 42] INFO [2022-01-10 12:53:25,655] jepsen worker 84 - jepsen.util 84 :ok :write [28 42] INFO [2022-01-10 12:53:25,655] jepsen worker 61 - jepsen.util 61 :ok :write [20 13] INFO [2022-01-10 12:53:25,655] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:25,656] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,656] jepsen worker 81 - jepsen.util 171 :ok :read [27 42] INFO [2022-01-10 12:53:25,656] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:25,657] jepsen worker 81 - jepsen.util 171 :invoke :write [27 28] INFO [2022-01-10 12:53:25,657] jepsen worker 84 - jepsen.util 84 :ok :read [28 42] INFO [2022-01-10 12:53:25,657] jepsen worker 61 - jepsen.util 61 :ok :read [20 13] INFO [2022-01-10 12:53:25,658] jepsen worker 84 - jepsen.util 84 :invoke :write [28 32] INFO [2022-01-10 12:53:25,658] jepsen worker 61 - jepsen.util 61 :invoke :write [20 13] INFO [2022-01-10 12:53:25,661] jepsen worker 4 - jepsen.util 4 :ok :write [1 32] INFO [2022-01-10 12:53:25,662] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:25,662] jepsen worker 8 - jepsen.util 8 :ok :write [2 42] INFO [2022-01-10 12:53:25,663] jepsen worker 4 - jepsen.util 4 :ok :read [1 32] INFO [2022-01-10 12:53:25,663] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:25,663] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:25,664] jepsen worker 8 - jepsen.util 8 :ok :read [2 42] INFO [2022-01-10 12:53:25,664] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:25,665] jepsen worker 4 - jepsen.util 4 :ok :read [1 32] INFO [2022-01-10 12:53:25,665] jepsen worker 8 - jepsen.util 8 :ok :read [2 42] INFO [2022-01-10 12:53:25,665] jepsen worker 4 - jepsen.util 4 :invoke :write [1 30] INFO [2022-01-10 12:53:25,666] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:25,666] jepsen worker 37 - jepsen.util 127 :ok :write [12 40] INFO [2022-01-10 12:53:25,666] jepsen worker 8 - jepsen.util 8 :ok :read [2 42] INFO [2022-01-10 12:53:25,666] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,667] jepsen worker 8 - jepsen.util 8 :invoke :write [2 13] INFO [2022-01-10 12:53:25,668] jepsen worker 37 - jepsen.util 127 :ok :read [12 40] INFO [2022-01-10 12:53:25,668] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,669] jepsen worker 37 - jepsen.util 127 :ok :read [12 40] INFO [2022-01-10 12:53:25,670] jepsen worker 37 - jepsen.util 127 :invoke :write [12 41] INFO [2022-01-10 12:53:25,670] jepsen worker 84 - jepsen.util 84 :ok :write [28 32] INFO [2022-01-10 12:53:25,671] jepsen worker 84 - jepsen.util 84 :invoke :write [28 48] INFO [2022-01-10 12:53:25,675] jepsen worker 4 - jepsen.util 4 :ok :write [1 30] INFO [2022-01-10 12:53:25,676] jepsen worker 4 - jepsen.util 4 :invoke :write [1 11] INFO [2022-01-10 12:53:25,676] jepsen worker 80 - jepsen.util 260 :ok :write [26 9] INFO [2022-01-10 12:53:25,677] jepsen worker 80 - jepsen.util 260 :invoke :write [26 29] INFO [2022-01-10 12:53:25,678] jepsen worker 44 - jepsen.util 44 :ok :write [14 29] INFO [2022-01-10 12:53:25,679] jepsen worker 2 - jepsen.util 92 :ok :write [0 9] INFO [2022-01-10 12:53:25,679] jepsen worker 37 - jepsen.util 127 :ok :write [12 41] INFO [2022-01-10 12:53:25,679] jepsen worker 44 - jepsen.util 44 :invoke :write [14 47] INFO [2022-01-10 12:53:25,680] jepsen worker 2 - jepsen.util 92 :invoke :write [0 27] INFO [2022-01-10 12:53:25,680] jepsen worker 53 - jepsen.util 143 :ok :write [17 21] INFO [2022-01-10 12:53:25,681] jepsen worker 37 - jepsen.util 127 :invoke :write [12 43] INFO [2022-01-10 12:53:25,683] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,683] jepsen worker 74 - jepsen.util 254 :fail :read [24 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:25,683] jepsen worker 84 - jepsen.util 84 :ok :write [28 48] INFO [2022-01-10 12:53:25,683] jepsen worker 74 - jepsen.util 254 :invoke :write [24 20] INFO [2022-01-10 12:53:25,683] jepsen worker 53 - jepsen.util 143 :ok :read [17 21] INFO [2022-01-10 12:53:25,684] jepsen worker 84 - jepsen.util 84 :invoke :write [28 44] INFO [2022-01-10 12:53:25,684] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,685] jepsen worker 53 - jepsen.util 143 :ok :read [17 21] INFO [2022-01-10 12:53:25,685] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,686] jepsen worker 53 - jepsen.util 143 :ok :read [17 21] INFO [2022-01-10 12:53:25,686] jepsen worker 53 - jepsen.util 143 :invoke :write [17 45] INFO [2022-01-10 12:53:25,695] jepsen worker 37 - jepsen.util 127 :ok :write [12 43] INFO [2022-01-10 12:53:25,695] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,696] jepsen worker 37 - jepsen.util 127 :ok :read [12 43] INFO [2022-01-10 12:53:25,697] jepsen worker 37 - jepsen.util 127 :invoke :write [12 36] INFO [2022-01-10 12:53:25,698] jepsen worker 87 - jepsen.util 87 :ok :write [29 28] INFO [2022-01-10 12:53:25,698] jepsen worker 87 - jepsen.util 87 :invoke :write [29 2] INFO [2022-01-10 12:53:25,702] jepsen worker 40 - jepsen.util 130 :ok :write [13 37] INFO [2022-01-10 12:53:25,702] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,703] jepsen worker 4 - jepsen.util 4 :ok :write [1 11] INFO [2022-01-10 12:53:25,703] jepsen worker 40 - jepsen.util 130 :ok :read [13 37] INFO [2022-01-10 12:53:25,703] jepsen worker 4 - jepsen.util 4 :invoke :write [1 47] INFO [2022-01-10 12:53:25,703] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,704] jepsen worker 40 - jepsen.util 130 :ok :read [13 37] INFO [2022-01-10 12:53:25,705] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,706] jepsen worker 40 - jepsen.util 130 :ok :read [13 37] INFO [2022-01-10 12:53:25,707] jepsen worker 40 - jepsen.util 130 :invoke :write [13 36] INFO [2022-01-10 12:53:25,708] jepsen worker 84 - jepsen.util 84 :ok :write [28 44] INFO [2022-01-10 12:53:25,708] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,709] jepsen worker 84 - jepsen.util 84 :ok :read [28 44] INFO [2022-01-10 12:53:25,710] jepsen worker 84 - jepsen.util 84 :invoke :write [28 16] INFO [2022-01-10 12:53:25,712] jepsen worker 37 - jepsen.util 127 :ok :write [12 36] INFO [2022-01-10 12:53:25,712] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,713] jepsen worker 37 - jepsen.util 127 :ok :read [12 36] INFO [2022-01-10 12:53:25,714] jepsen worker 37 - jepsen.util 127 :invoke :write [12 16] INFO [2022-01-10 12:53:25,716] jepsen worker 4 - jepsen.util 4 :ok :write [1 47] INFO [2022-01-10 12:53:25,717] jepsen worker 4 - jepsen.util 4 :invoke :write [1 24] INFO [2022-01-10 12:53:25,718] jepsen worker 2 - jepsen.util 92 :ok :write [0 27] INFO [2022-01-10 12:53:25,719] jepsen worker 2 - jepsen.util 92 :invoke :write [0 13] INFO [2022-01-10 12:53:25,723] jepsen worker 84 - jepsen.util 84 :ok :write [28 16] INFO [2022-01-10 12:53:25,723] jepsen worker 84 - jepsen.util 84 :invoke :write [28 1] INFO [2022-01-10 12:53:25,726] jepsen worker 87 - jepsen.util 87 :ok :write [29 2] INFO [2022-01-10 12:53:25,727] jepsen worker 87 - jepsen.util 87 :invoke :write [29 3] INFO [2022-01-10 12:53:25,728] jepsen worker 69 - jepsen.util 159 :ok :write [23 13] INFO [2022-01-10 12:53:25,728] jepsen worker 40 - jepsen.util 130 :ok :write [13 36] INFO [2022-01-10 12:53:25,728] jepsen worker 69 - jepsen.util 159 :invoke :write [23 22] INFO [2022-01-10 12:53:25,728] jepsen worker 23 - jepsen.util 23 :ok :write [7 8] INFO [2022-01-10 12:53:25,729] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,729] jepsen worker 37 - jepsen.util 127 :ok :write [12 16] INFO [2022-01-10 12:53:25,729] jepsen worker 23 - jepsen.util 23 :invoke :write [7 16] INFO [2022-01-10 12:53:25,730] jepsen worker 40 - jepsen.util 130 :ok :read [13 36] INFO [2022-01-10 12:53:25,730] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:25,731] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,731] jepsen worker 37 - jepsen.util 127 :ok :read [12 16] INFO [2022-01-10 12:53:25,731] jepsen worker 40 - jepsen.util 130 :ok :read [13 36] INFO [2022-01-10 12:53:25,732] jepsen worker 37 - jepsen.util 127 :invoke :write [12 5] INFO [2022-01-10 12:53:25,732] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,733] jepsen worker 40 - jepsen.util 130 :ok :read [13 36] INFO [2022-01-10 12:53:25,733] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,735] jepsen worker 2 - jepsen.util 92 :ok :write [0 13] INFO [2022-01-10 12:53:25,735] jepsen worker 4 - jepsen.util 4 :ok :write [1 24] INFO [2022-01-10 12:53:25,735] jepsen worker 40 - jepsen.util 130 :ok :read [13 36] INFO [2022-01-10 12:53:25,735] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:25,736] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:25,736] jepsen worker 4 - jepsen.util 4 :ok :read [1 24] INFO [2022-01-10 12:53:25,736] jepsen worker 2 - jepsen.util 92 :invoke :write [0 45] INFO [2022-01-10 12:53:25,737] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:25,737] jepsen worker 40 - jepsen.util 130 :ok :read [13 36] INFO [2022-01-10 12:53:25,738] jepsen worker 4 - jepsen.util 4 :ok :read [1 24] INFO [2022-01-10 12:53:25,738] jepsen worker 40 - jepsen.util 130 :invoke :write [13 49] INFO [2022-01-10 12:53:25,738] jepsen worker 4 - jepsen.util 4 :invoke :write [1 2] INFO [2022-01-10 12:53:25,741] jepsen worker 84 - jepsen.util 84 :ok :write [28 1] INFO [2022-01-10 12:53:25,741] jepsen worker 87 - jepsen.util 87 :ok :write [29 3] INFO [2022-01-10 12:53:25,741] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,742] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:25,742] jepsen worker 84 - jepsen.util 84 :ok :read [28 1] INFO [2022-01-10 12:53:25,742] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:25,743] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:25,743] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,744] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:25,744] jepsen worker 84 - jepsen.util 84 :ok :read [28 1] INFO [2022-01-10 12:53:25,744] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:25,745] jepsen worker 87 - jepsen.util 87 :ok :read [29 3] INFO [2022-01-10 12:53:25,745] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,746] jepsen worker 84 - jepsen.util 84 :ok :read [28 1] INFO [2022-01-10 12:53:25,746] jepsen worker 87 - jepsen.util 87 :invoke :write [29 29] INFO [2022-01-10 12:53:25,747] jepsen worker 84 - jepsen.util 84 :invoke :write [28 30] WARN [2022-01-10 12:53:25,750] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1246444,"last_local_id":"2B94F89700000006/00000000A79E4C0A","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":1268335,"last_local_socket":"172.23.104.79:34662","total_server_duration_us":69714,"encode_duration_us":22,"operation_id":"0x1a2f","timeout_ms":10000,"last_server_duration_us":65253,"total_duration_us":10001985}],"total_count":1}} WARN [2022-01-10 12:53:25,750] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":513,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":996049,"total_server_duration_us":3236,"encode_duration_us":41,"operation_id":"0x20d9","timeout_ms":10000,"last_server_duration_us":49,"total_duration_us":10009814},{"operation_name":"upsert","last_dispatch_duration_us":623,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":3066744,"total_server_duration_us":1967,"encode_duration_us":28,"operation_id":"0x1a16","timeout_ms":10000,"last_server_duration_us":100,"total_duration_us":10008829},{"operation_name":"upsert","last_dispatch_duration_us":615,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":19967,"total_server_duration_us":3052,"encode_duration_us":48,"operation_id":"0x1782","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10006635},{"operation_name":"upsert","last_dispatch_duration_us":595,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":22587,"total_server_duration_us":4718,"encode_duration_us":26,"operation_id":"0x20f4","timeout_ms":10000,"last_server_duration_us":126,"total_duration_us":10005073},{"operation_name":"upsert","last_dispatch_duration_us":6018420,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000006/000000001E9CFABF","last_local_socket":"172.23.104.79:40226","total_dispatch_duration_us":6028018,"total_server_duration_us":1149,"encode_duration_us":28,"operation_id":"0x2009","timeout_ms":10000,"last_server_duration_us":108,"total_duration_us":10004236},{"operation_name":"upsert","last_dispatch_duration_us":744,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":1205085,"total_server_duration_us":2319,"encode_duration_us":22,"operation_id":"0x1e24","timeout_ms":10000,"last_server_duration_us":56,"total_duration_us":10003543},{"operation_name":"upsert","last_dispatch_duration_us":3017460,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000006/0000000044D7A75E","last_local_socket":"172.23.104.79:48380","total_dispatch_duration_us":3031959,"total_server_duration_us":2270,"encode_duration_us":26,"operation_id":"0x1e23","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10003442},{"operation_name":"upsert","last_dispatch_duration_us":520,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000006/000000001E9CFABF","last_local_socket":"172.23.104.79:40226","total_dispatch_duration_us":9833,"total_server_duration_us":1006,"encode_duration_us":51,"operation_id":"0x1fa2","timeout_ms":10000,"last_server_duration_us":84,"total_duration_us":10002346},{"operation_name":"upsert","last_dispatch_duration_us":1246444,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":1268335,"total_server_duration_us":69714,"encode_duration_us":22,"operation_id":"0x1a2f","timeout_ms":10000,"last_server_duration_us":65253,"total_duration_us":10001985},{"operation_name":"upsert","last_dispatch_duration_us":9008131,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000006/000000001E9CFABF","last_local_socket":"172.23.104.79:40226","total_dispatch_duration_us":9012424,"total_server_duration_us":9008292,"encode_duration_us":31,"operation_id":"0x20f1","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":9182172}],"total_count":24}} INFO [2022-01-10 12:53:25,764] jepsen worker 13 - jepsen.util 13 :ok :write [4 28] INFO [2022-01-10 12:53:25,765] jepsen worker 68 - jepsen.util 68 :ok :write [22 3] INFO [2022-01-10 12:53:25,765] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:25,766] jepsen worker 13 - jepsen.util 13 :ok :read [4 28] INFO [2022-01-10 12:53:25,766] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:25,766] jepsen worker 13 - jepsen.util 13 :invoke :write [4 23] INFO [2022-01-10 12:53:25,767] jepsen worker 68 - jepsen.util 68 :ok :read [22 3] INFO [2022-01-10 12:53:25,767] jepsen worker 68 - jepsen.util 68 :invoke :write [22 27] INFO [2022-01-10 12:53:25,769] jepsen worker 80 - jepsen.util 260 :ok :write [26 29] INFO [2022-01-10 12:53:25,769] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:25,770] jepsen worker 80 - jepsen.util 260 :ok :read [26 29] INFO [2022-01-10 12:53:25,771] jepsen worker 75 - jepsen.util 165 :ok :write [25 12] INFO [2022-01-10 12:53:25,771] jepsen worker 80 - jepsen.util 260 :invoke :write [26 18] INFO [2022-01-10 12:53:25,772] jepsen worker 75 - jepsen.util 165 :invoke :write [25 0] INFO [2022-01-10 12:53:25,820] jepsen worker 53 - jepsen.util 143 :ok :write [17 45] INFO [2022-01-10 12:53:25,820] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:25,821] jepsen worker 53 - jepsen.util 143 :ok :read [17 45] INFO [2022-01-10 12:53:25,822] jepsen worker 53 - jepsen.util 143 :invoke :write [17 46] INFO [2022-01-10 12:53:25,823] jepsen worker 33 - jepsen.util 123 :ok :write [11 42] INFO [2022-01-10 12:53:25,824] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:25,825] jepsen worker 33 - jepsen.util 123 :ok :read [11 42] INFO [2022-01-10 12:53:25,825] jepsen worker 69 - jepsen.util 159 :ok :write [23 22] INFO [2022-01-10 12:53:25,826] jepsen worker 33 - jepsen.util 123 :invoke :write [11 28] INFO [2022-01-10 12:53:25,826] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:25,827] jepsen worker 69 - jepsen.util 159 :ok :read [23 22] INFO [2022-01-10 12:53:25,827] jepsen worker 69 - jepsen.util 159 :invoke :write [23 19] INFO [2022-01-10 12:53:25,830] jepsen worker 15 - jepsen.util 15 :ok :write [5 16] INFO [2022-01-10 12:53:25,831] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:25,834] jepsen worker 50 - jepsen.util 50 :ok :write [16 48] INFO [2022-01-10 12:53:25,835] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:25,840] jepsen worker 61 - jepsen.util 61 :ok :write [20 13] INFO [2022-01-10 12:53:25,841] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:25,842] jepsen worker 61 - jepsen.util 61 :ok :read [20 13] INFO [2022-01-10 12:53:25,842] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:25,843] jepsen worker 61 - jepsen.util 61 :ok :read [20 13] INFO [2022-01-10 12:53:25,843] jepsen worker 61 - jepsen.util 61 :invoke :write [20 28] INFO [2022-01-10 12:53:25,845] jepsen worker 81 - jepsen.util 171 :ok :write [27 28] INFO [2022-01-10 12:53:25,846] jepsen worker 15 - jepsen.util 15 :ok :read [5 16] INFO [2022-01-10 12:53:25,846] jepsen worker 81 - jepsen.util 171 :invoke :write [27 47] INFO [2022-01-10 12:53:25,846] jepsen worker 50 - jepsen.util 50 :ok :read [16 48] INFO [2022-01-10 12:53:25,846] jepsen worker 50 - jepsen.util 50 :invoke :write [16 40] INFO [2022-01-10 12:53:25,847] jepsen worker 15 - jepsen.util 15 :invoke :write [5 21] INFO [2022-01-10 12:53:25,982] jepsen worker 37 - jepsen.util 127 :ok :write [12 5] INFO [2022-01-10 12:53:25,982] jepsen worker 37 - jepsen.util 127 :invoke :write [12 27] INFO [2022-01-10 12:53:25,989] jepsen worker 4 - jepsen.util 4 :ok :write [1 2] INFO [2022-01-10 12:53:25,989] jepsen worker 40 - jepsen.util 130 :ok :write [13 49] INFO [2022-01-10 12:53:25,989] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:25,990] jepsen worker 40 - jepsen.util 130 :invoke :write [13 38] INFO [2022-01-10 12:53:25,990] jepsen worker 4 - jepsen.util 4 :ok :read [1 2] INFO [2022-01-10 12:53:25,991] jepsen worker 4 - jepsen.util 4 :invoke :write [1 37] INFO [2022-01-10 12:53:25,993] jepsen worker 2 - jepsen.util 92 :ok :write [0 45] INFO [2022-01-10 12:53:25,993] jepsen worker 2 - jepsen.util 92 :invoke :write [0 5] INFO [2022-01-10 12:53:25,998] jepsen worker 84 - jepsen.util 84 :ok :write [28 30] INFO [2022-01-10 12:53:25,998] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:25,999] jepsen worker 84 - jepsen.util 84 :ok :read [28 30] INFO [2022-01-10 12:53:25,999] jepsen worker 8 - jepsen.util 8 :ok :write [2 13] INFO [2022-01-10 12:53:26,000] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,000] jepsen worker 8 - jepsen.util 8 :invoke :write [2 1] INFO [2022-01-10 12:53:26,000] jepsen worker 87 - jepsen.util 87 :ok :write [29 29] INFO [2022-01-10 12:53:26,001] jepsen worker 84 - jepsen.util 84 :ok :read [28 30] INFO [2022-01-10 12:53:26,001] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,001] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,002] jepsen worker 87 - jepsen.util 87 :ok :read [29 29] INFO [2022-01-10 12:53:26,002] jepsen worker 84 - jepsen.util 84 :ok :read [28 30] INFO [2022-01-10 12:53:26,002] jepsen worker 87 - jepsen.util 87 :invoke :write [29 6] INFO [2022-01-10 12:53:26,003] jepsen worker 13 - jepsen.util 13 :ok :write [4 23] INFO [2022-01-10 12:53:26,003] jepsen worker 84 - jepsen.util 84 :invoke :write [28 32] INFO [2022-01-10 12:53:26,003] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,004] jepsen worker 13 - jepsen.util 13 :ok :read [4 23] INFO [2022-01-10 12:53:26,005] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,006] jepsen worker 13 - jepsen.util 13 :ok :read [4 23] INFO [2022-01-10 12:53:26,006] jepsen worker 13 - jepsen.util 13 :invoke :write [4 17] INFO [2022-01-10 12:53:26,013] jepsen worker 80 - jepsen.util 260 :ok :write [26 18] INFO [2022-01-10 12:53:26,013] jepsen worker 80 - jepsen.util 260 :invoke :write [26 21] INFO [2022-01-10 12:53:26,017] jepsen worker 53 - jepsen.util 143 :ok :write [17 46] INFO [2022-01-10 12:53:26,017] jepsen worker 53 - jepsen.util 143 :invoke :write [17 27] INFO [2022-01-10 12:53:26,021] jepsen worker 69 - jepsen.util 159 :ok :write [23 19] INFO [2022-01-10 12:53:26,021] jepsen worker 69 - jepsen.util 159 :invoke :write [23 27] INFO [2022-01-10 12:53:26,025] jepsen worker 37 - jepsen.util 127 :ok :write [12 27] INFO [2022-01-10 12:53:26,025] jepsen worker 37 - jepsen.util 127 :invoke :write [12 15] INFO [2022-01-10 12:53:26,028] jepsen worker 4 - jepsen.util 4 :ok :write [1 37] INFO [2022-01-10 12:53:26,029] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,030] jepsen worker 4 - jepsen.util 4 :ok :read [1 37] INFO [2022-01-10 12:53:26,030] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,031] jepsen worker 4 - jepsen.util 4 :ok :read [1 37] INFO [2022-01-10 12:53:26,031] jepsen worker 4 - jepsen.util 4 :invoke :write [1 25] INFO [2022-01-10 12:53:26,039] jepsen worker 84 - jepsen.util 84 :ok :write [28 32] INFO [2022-01-10 12:53:26,039] jepsen worker 84 - jepsen.util 84 :invoke :write [28 27] INFO [2022-01-10 12:53:26,045] jepsen worker 40 - jepsen.util 130 :ok :write [13 38] INFO [2022-01-10 12:53:26,046] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:26,047] jepsen worker 40 - jepsen.util 130 :ok :read [13 38] INFO [2022-01-10 12:53:26,047] jepsen worker 40 - jepsen.util 130 :invoke :write [13 40] INFO [2022-01-10 12:53:26,050] jepsen worker 2 - jepsen.util 92 :ok :write [0 5] INFO [2022-01-10 12:53:26,051] jepsen worker 2 - jepsen.util 92 :invoke :write [0 14] INFO [2022-01-10 12:53:26,056] jepsen worker 87 - jepsen.util 87 :ok :write [29 6] INFO [2022-01-10 12:53:26,056] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,057] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:26,057] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,058] jepsen worker 87 - jepsen.util 87 :ok :read [29 6] INFO [2022-01-10 12:53:26,059] jepsen worker 87 - jepsen.util 87 :invoke :write [29 28] INFO [2022-01-10 12:53:26,061] jepsen worker 40 - jepsen.util 130 :ok :write [13 40] INFO [2022-01-10 12:53:26,061] jepsen worker 40 - jepsen.util 130 :invoke :write [13 19] INFO [2022-01-10 12:53:26,064] jepsen worker 37 - jepsen.util 127 :ok :write [12 15] INFO [2022-01-10 12:53:26,064] jepsen worker 37 - jepsen.util 127 :invoke :write [12 0] INFO [2022-01-10 12:53:26,065] jepsen worker 2 - jepsen.util 92 :ok :write [0 14] INFO [2022-01-10 12:53:26,066] jepsen worker 2 - jepsen.util 92 :invoke :write [0 13] INFO [2022-01-10 12:53:26,071] jepsen worker 4 - jepsen.util 4 :ok :write [1 25] INFO [2022-01-10 12:53:26,072] jepsen worker 4 - jepsen.util 4 :invoke :write [1 33] INFO [2022-01-10 12:53:26,075] jepsen worker 40 - jepsen.util 130 :ok :write [13 19] INFO [2022-01-10 12:53:26,076] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:26,077] jepsen worker 40 - jepsen.util 130 :ok :read [13 19] INFO [2022-01-10 12:53:26,077] jepsen worker 84 - jepsen.util 84 :ok :write [28 27] INFO [2022-01-10 12:53:26,078] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:26,078] jepsen worker 84 - jepsen.util 84 :invoke :write [28 10] INFO [2022-01-10 12:53:26,079] jepsen worker 40 - jepsen.util 130 :ok :read [13 19] INFO [2022-01-10 12:53:26,080] jepsen worker 40 - jepsen.util 130 :invoke :write [13 4] INFO [2022-01-10 12:53:26,103] jepsen worker 44 - jepsen.util 44 :ok :write [14 47] INFO [2022-01-10 12:53:26,104] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:26,105] jepsen worker 44 - jepsen.util 44 :ok :read [14 47] INFO [2022-01-10 12:53:26,105] jepsen worker 44 - jepsen.util 44 :invoke :write [14 11] INFO [2022-01-10 12:53:26,162] jepsen worker 87 - jepsen.util 87 :ok :write [29 28] INFO [2022-01-10 12:53:26,162] jepsen worker 87 - jepsen.util 87 :invoke :write [29 33] INFO [2022-01-10 12:53:26,164] jepsen worker 37 - jepsen.util 127 :ok :write [12 0] INFO [2022-01-10 12:53:26,165] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,166] jepsen worker 40 - jepsen.util 130 :ok :write [13 4] INFO [2022-01-10 12:53:26,166] jepsen worker 37 - jepsen.util 127 :ok :read [12 0] INFO [2022-01-10 12:53:26,166] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:26,166] jepsen worker 37 - jepsen.util 127 :invoke :write [12 49] INFO [2022-01-10 12:53:26,167] jepsen worker 40 - jepsen.util 130 :ok :read [13 4] INFO [2022-01-10 12:53:26,167] jepsen worker 40 - jepsen.util 130 :invoke :write [13 18] INFO [2022-01-10 12:53:26,170] jepsen worker 4 - jepsen.util 4 :ok :write [1 33] INFO [2022-01-10 12:53:26,170] jepsen worker 2 - jepsen.util 92 :ok :write [0 13] INFO [2022-01-10 12:53:26,170] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,171] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,171] jepsen worker 4 - jepsen.util 4 :ok :read [1 33] INFO [2022-01-10 12:53:26,172] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,172] jepsen worker 2 - jepsen.util 92 :ok :read [0 13] INFO [2022-01-10 12:53:26,172] jepsen worker 2 - jepsen.util 92 :invoke :write [0 39] INFO [2022-01-10 12:53:26,173] jepsen worker 84 - jepsen.util 84 :ok :write [28 10] INFO [2022-01-10 12:53:26,173] jepsen worker 4 - jepsen.util 4 :ok :read [1 33] INFO [2022-01-10 12:53:26,174] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,174] jepsen worker 4 - jepsen.util 4 :invoke :write [1 29] INFO [2022-01-10 12:53:26,175] jepsen worker 84 - jepsen.util 84 :ok :read [28 10] INFO [2022-01-10 12:53:26,175] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,176] jepsen worker 84 - jepsen.util 84 :ok :read [28 10] INFO [2022-01-10 12:53:26,176] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,177] jepsen worker 84 - jepsen.util 84 :ok :read [28 10] INFO [2022-01-10 12:53:26,178] jepsen worker 23 - jepsen.util 23 :ok :write [7 16] INFO [2022-01-10 12:53:26,178] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,179] jepsen worker 84 - jepsen.util 84 :ok :read [28 10] INFO [2022-01-10 12:53:26,179] jepsen worker 23 - jepsen.util 23 :invoke :write [7 5] INFO [2022-01-10 12:53:26,179] jepsen worker 87 - jepsen.util 87 :ok :write [29 33] INFO [2022-01-10 12:53:26,179] jepsen worker 13 - jepsen.util 13 :ok :write [4 17] INFO [2022-01-10 12:53:26,180] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,180] jepsen worker 13 - jepsen.util 13 :invoke :write [4 11] INFO [2022-01-10 12:53:26,181] jepsen worker 37 - jepsen.util 127 :ok :write [12 49] INFO [2022-01-10 12:53:26,181] jepsen worker 84 - jepsen.util 84 :ok :read [28 10] INFO [2022-01-10 12:53:26,181] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,182] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,182] jepsen worker 87 - jepsen.util 87 :ok :read [29 33] INFO [2022-01-10 12:53:26,182] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,183] jepsen worker 68 - jepsen.util 68 :ok :write [22 27] INFO [2022-01-10 12:53:26,183] jepsen worker 84 - jepsen.util 84 :ok :read [28 10] INFO [2022-01-10 12:53:26,183] jepsen worker 87 - jepsen.util 87 :invoke :write [29 7] INFO [2022-01-10 12:53:26,184] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,184] jepsen worker 80 - jepsen.util 260 :ok :write [26 21] INFO [2022-01-10 12:53:26,184] jepsen worker 68 - jepsen.util 68 :invoke :write [22 46] INFO [2022-01-10 12:53:26,185] jepsen worker 37 - jepsen.util 127 :ok :read [12 49] INFO [2022-01-10 12:53:26,185] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,185] jepsen worker 37 - jepsen.util 127 :invoke :write [12 7] INFO [2022-01-10 12:53:26,185] jepsen worker 84 - jepsen.util 84 :ok :read [28 10] INFO [2022-01-10 12:53:26,186] jepsen worker 80 - jepsen.util 260 :ok :read [26 21] INFO [2022-01-10 12:53:26,186] jepsen worker 84 - jepsen.util 84 :invoke :write [28 41] INFO [2022-01-10 12:53:26,186] jepsen worker 80 - jepsen.util 260 :invoke :write [26 6] INFO [2022-01-10 12:53:26,188] jepsen worker 53 - jepsen.util 143 :ok :write [17 27] INFO [2022-01-10 12:53:26,188] jepsen worker 75 - jepsen.util 165 :ok :write [25 0] INFO [2022-01-10 12:53:26,189] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:26,189] jepsen worker 75 - jepsen.util 165 :invoke :write [25 49] INFO [2022-01-10 12:53:26,191] jepsen worker 53 - jepsen.util 143 :ok :read [17 27] INFO [2022-01-10 12:53:26,191] jepsen worker 2 - jepsen.util 92 :ok :write [0 39] INFO [2022-01-10 12:53:26,191] jepsen worker 40 - jepsen.util 130 :ok :write [13 18] INFO [2022-01-10 12:53:26,191] jepsen worker 53 - jepsen.util 143 :invoke :write [17 38] INFO [2022-01-10 12:53:26,192] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,192] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:26,192] jepsen worker 2 - jepsen.util 92 :ok :read [0 39] INFO [2022-01-10 12:53:26,193] jepsen worker 40 - jepsen.util 130 :ok :read [13 18] INFO [2022-01-10 12:53:26,193] jepsen worker 2 - jepsen.util 92 :invoke :write [0 49] INFO [2022-01-10 12:53:26,193] jepsen worker 40 - jepsen.util 130 :invoke :write [13 16] INFO [2022-01-10 12:53:26,193] jepsen worker 69 - jepsen.util 159 :ok :write [23 27] INFO [2022-01-10 12:53:26,194] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,195] jepsen worker 69 - jepsen.util 159 :ok :read [23 27] INFO [2022-01-10 12:53:26,195] jepsen worker 69 - jepsen.util 159 :invoke :write [23 41] INFO [2022-01-10 12:53:26,196] jepsen worker 33 - jepsen.util 123 :ok :write [11 28] INFO [2022-01-10 12:53:26,196] jepsen worker 33 - jepsen.util 123 :invoke :write [11 48] INFO [2022-01-10 12:53:26,197] jepsen worker 4 - jepsen.util 4 :ok :write [1 29] INFO [2022-01-10 12:53:26,197] jepsen worker 4 - jepsen.util 4 :invoke :write [1 6] INFO [2022-01-10 12:53:26,199] jepsen worker 87 - jepsen.util 87 :ok :write [29 7] INFO [2022-01-10 12:53:26,200] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,201] jepsen worker 87 - jepsen.util 87 :ok :read [29 7] INFO [2022-01-10 12:53:26,201] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,202] jepsen worker 87 - jepsen.util 87 :ok :read [29 7] INFO [2022-01-10 12:53:26,202] jepsen worker 15 - jepsen.util 15 :ok :write [5 21] INFO [2022-01-10 12:53:26,203] jepsen worker 15 - jepsen.util 15 :invoke :write [5 24] INFO [2022-01-10 12:53:26,203] jepsen worker 13 - jepsen.util 13 :ok :write [4 11] INFO [2022-01-10 12:53:26,203] jepsen worker 87 - jepsen.util 87 :invoke :write [29 43] INFO [2022-01-10 12:53:26,204] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,205] jepsen worker 13 - jepsen.util 13 :ok :read [4 11] INFO [2022-01-10 12:53:26,206] jepsen worker 13 - jepsen.util 13 :invoke :write [4 30] INFO [2022-01-10 12:53:26,207] jepsen worker 48 - jepsen.util 228 :ok :write [16 33] INFO [2022-01-10 12:53:26,207] jepsen worker 37 - jepsen.util 127 :ok :write [12 7] INFO [2022-01-10 12:53:26,208] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:26,209] jepsen worker 37 - jepsen.util 127 :invoke :write [12 39] INFO [2022-01-10 12:53:26,209] jepsen worker 48 - jepsen.util 228 :ok :read [16 33] INFO [2022-01-10 12:53:26,210] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:26,211] jepsen worker 48 - jepsen.util 228 :ok :read [16 33] INFO [2022-01-10 12:53:26,212] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:26,212] jepsen worker 80 - jepsen.util 260 :ok :write [26 6] INFO [2022-01-10 12:53:26,212] jepsen worker 61 - jepsen.util 61 :ok :write [20 28] INFO [2022-01-10 12:53:26,213] jepsen worker 48 - jepsen.util 228 :ok :read [16 33] INFO [2022-01-10 12:53:26,213] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,214] jepsen worker 80 - jepsen.util 260 :ok :read [26 6] INFO [2022-01-10 12:53:26,214] jepsen worker 61 - jepsen.util 61 :invoke :write [20 47] INFO [2022-01-10 12:53:26,214] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:26,215] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,215] jepsen worker 48 - jepsen.util 228 :ok :read [16 33] INFO [2022-01-10 12:53:26,216] jepsen worker 80 - jepsen.util 260 :ok :read [26 6] INFO [2022-01-10 12:53:26,216] jepsen worker 2 - jepsen.util 92 :ok :write [0 49] INFO [2022-01-10 12:53:26,216] jepsen worker 48 - jepsen.util 228 :invoke :write [16 40] INFO [2022-01-10 12:53:26,216] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,217] jepsen worker 2 - jepsen.util 92 :invoke :write [0 9] INFO [2022-01-10 12:53:26,218] jepsen worker 80 - jepsen.util 260 :ok :read [26 6] INFO [2022-01-10 12:53:26,219] jepsen worker 80 - jepsen.util 260 :invoke :write [26 26] INFO [2022-01-10 12:53:26,220] jepsen worker 81 - jepsen.util 171 :ok :write [27 47] INFO [2022-01-10 12:53:26,220] jepsen worker 40 - jepsen.util 130 :ok :write [13 16] INFO [2022-01-10 12:53:26,221] jepsen worker 81 - jepsen.util 171 :invoke :write [27 45] INFO [2022-01-10 12:53:26,221] jepsen worker 40 - jepsen.util 130 :invoke :write [13 32] INFO [2022-01-10 12:53:26,222] jepsen worker 53 - jepsen.util 143 :ok :write [17 38] INFO [2022-01-10 12:53:26,223] jepsen worker 8 - jepsen.util 8 :ok :write [2 1] INFO [2022-01-10 12:53:26,223] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:26,224] jepsen worker 8 - jepsen.util 8 :invoke :write [2 27] INFO [2022-01-10 12:53:26,224] jepsen worker 53 - jepsen.util 143 :ok :read [17 38] INFO [2022-01-10 12:53:26,225] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:26,226] jepsen worker 53 - jepsen.util 143 :ok :read [17 38] INFO [2022-01-10 12:53:26,227] jepsen worker 53 - jepsen.util 143 :invoke :write [17 35] INFO [2022-01-10 12:53:26,227] jepsen worker 44 - jepsen.util 44 :ok :write [14 11] INFO [2022-01-10 12:53:26,228] jepsen worker 84 - jepsen.util 84 :ok :write [28 41] INFO [2022-01-10 12:53:26,228] jepsen worker 44 - jepsen.util 44 :invoke :write [14 24] INFO [2022-01-10 12:53:26,229] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,229] jepsen worker 87 - jepsen.util 87 :ok :write [29 43] INFO [2022-01-10 12:53:26,230] jepsen worker 84 - jepsen.util 84 :ok :read [28 41] INFO [2022-01-10 12:53:26,230] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,231] jepsen worker 84 - jepsen.util 84 :invoke :write [28 13] INFO [2022-01-10 12:53:26,231] jepsen worker 23 - jepsen.util 23 :ok :write [7 5] INFO [2022-01-10 12:53:26,232] jepsen worker 87 - jepsen.util 87 :ok :read [29 43] INFO [2022-01-10 12:53:26,232] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:26,233] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,233] jepsen worker 23 - jepsen.util 23 :ok :read [7 5] INFO [2022-01-10 12:53:26,234] jepsen worker 87 - jepsen.util 87 :ok :read [29 43] INFO [2022-01-10 12:53:26,234] jepsen worker 23 - jepsen.util 23 :invoke :write [7 35] INFO [2022-01-10 12:53:26,235] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,235] jepsen worker 87 - jepsen.util 87 :ok :read [29 43] INFO [2022-01-10 12:53:26,236] jepsen worker 69 - jepsen.util 159 :ok :write [23 41] INFO [2022-01-10 12:53:26,236] jepsen worker 68 - jepsen.util 68 :ok :write [22 46] INFO [2022-01-10 12:53:26,237] jepsen worker 87 - jepsen.util 87 :invoke :write [29 30] INFO [2022-01-10 12:53:26,237] jepsen worker 69 - jepsen.util 159 :invoke :write [23 36] INFO [2022-01-10 12:53:26,238] jepsen worker 68 - jepsen.util 68 :invoke :write [22 44] INFO [2022-01-10 12:53:26,241] jepsen worker 4 - jepsen.util 4 :ok :write [1 6] INFO [2022-01-10 12:53:26,242] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,243] jepsen worker 4 - jepsen.util 4 :ok :read [1 6] INFO [2022-01-10 12:53:26,243] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,244] jepsen worker 75 - jepsen.util 165 :ok :write [25 49] INFO [2022-01-10 12:53:26,244] jepsen worker 4 - jepsen.util 4 :ok :read [1 6] INFO [2022-01-10 12:53:26,244] jepsen worker 75 - jepsen.util 165 :invoke :write [25 14] INFO [2022-01-10 12:53:26,244] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,245] jepsen worker 4 - jepsen.util 4 :ok :read [1 6] INFO [2022-01-10 12:53:26,246] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,247] jepsen worker 4 - jepsen.util 4 :ok :read [1 6] INFO [2022-01-10 12:53:26,247] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,248] jepsen worker 4 - jepsen.util 4 :ok :read [1 6] INFO [2022-01-10 12:53:26,248] jepsen worker 13 - jepsen.util 13 :ok :write [4 30] INFO [2022-01-10 12:53:26,249] jepsen worker 4 - jepsen.util 4 :invoke :write [1 31] INFO [2022-01-10 12:53:26,249] jepsen worker 2 - jepsen.util 92 :ok :write [0 9] INFO [2022-01-10 12:53:26,249] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,250] jepsen worker 2 - jepsen.util 92 :invoke :write [0 42] INFO [2022-01-10 12:53:26,251] jepsen worker 13 - jepsen.util 13 :ok :read [4 30] INFO [2022-01-10 12:53:26,252] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,252] jepsen worker 13 - jepsen.util 13 :ok :read [4 30] INFO [2022-01-10 12:53:26,253] jepsen worker 13 - jepsen.util 13 :invoke :write [4 13] INFO [2022-01-10 12:53:26,255] jepsen worker 33 - jepsen.util 123 :ok :write [11 48] INFO [2022-01-10 12:53:26,255] jepsen worker 33 - jepsen.util 123 :invoke :write [11 5] INFO [2022-01-10 12:53:26,256] jepsen worker 37 - jepsen.util 127 :ok :write [12 39] INFO [2022-01-10 12:53:26,257] jepsen worker 37 - jepsen.util 127 :invoke :write [12 6] INFO [2022-01-10 12:53:26,260] jepsen worker 15 - jepsen.util 15 :ok :write [5 24] INFO [2022-01-10 12:53:26,260] jepsen worker 15 - jepsen.util 15 :invoke :write [5 25] INFO [2022-01-10 12:53:26,260] jepsen worker 80 - jepsen.util 260 :ok :write [26 26] INFO [2022-01-10 12:53:26,261] jepsen worker 40 - jepsen.util 130 :ok :write [13 32] INFO [2022-01-10 12:53:26,261] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,261] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:26,262] jepsen worker 80 - jepsen.util 260 :ok :read [26 26] INFO [2022-01-10 12:53:26,262] jepsen worker 40 - jepsen.util 130 :ok :read [13 32] INFO [2022-01-10 12:53:26,263] jepsen worker 80 - jepsen.util 260 :invoke :write [26 41] INFO [2022-01-10 12:53:26,263] jepsen worker 40 - jepsen.util 130 :invoke :write [13 27] INFO [2022-01-10 12:53:26,265] jepsen worker 48 - jepsen.util 228 :ok :write [16 40] INFO [2022-01-10 12:53:26,266] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:26,267] jepsen worker 48 - jepsen.util 228 :ok :read [16 40] INFO [2022-01-10 12:53:26,267] jepsen worker 48 - jepsen.util 228 :invoke :write [16 34] INFO [2022-01-10 12:53:26,268] jepsen worker 53 - jepsen.util 143 :ok :write [17 35] INFO [2022-01-10 12:53:26,268] jepsen worker 53 - jepsen.util 143 :invoke :write [17 46] INFO [2022-01-10 12:53:26,271] jepsen worker 61 - jepsen.util 61 :ok :write [20 47] INFO [2022-01-10 12:53:26,271] jepsen worker 61 - jepsen.util 61 :invoke :write [20 19] INFO [2022-01-10 12:53:26,274] jepsen worker 84 - jepsen.util 84 :ok :write [28 13] INFO [2022-01-10 12:53:26,274] jepsen worker 87 - jepsen.util 87 :ok :write [29 30] INFO [2022-01-10 12:53:26,274] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,275] jepsen worker 87 - jepsen.util 87 :invoke :write [29 39] INFO [2022-01-10 12:53:26,275] jepsen worker 84 - jepsen.util 84 :ok :read [28 13] INFO [2022-01-10 12:53:26,276] jepsen worker 84 - jepsen.util 84 :invoke :write [28 5] INFO [2022-01-10 12:53:26,279] jepsen worker 81 - jepsen.util 171 :ok :write [27 45] INFO [2022-01-10 12:53:26,280] jepsen worker 81 - jepsen.util 171 :invoke :write [27 40] INFO [2022-01-10 12:53:26,281] jepsen worker 8 - jepsen.util 8 :ok :write [2 27] INFO [2022-01-10 12:53:26,281] jepsen worker 8 - jepsen.util 8 :invoke :write [2 13] INFO [2022-01-10 12:53:26,285] jepsen worker 2 - jepsen.util 92 :ok :write [0 42] INFO [2022-01-10 12:53:26,285] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,285] jepsen worker 44 - jepsen.util 44 :ok :write [14 24] INFO [2022-01-10 12:53:26,286] jepsen worker 44 - jepsen.util 44 :invoke :write [14 37] INFO [2022-01-10 12:53:26,286] jepsen worker 2 - jepsen.util 92 :ok :read [0 42] INFO [2022-01-10 12:53:26,286] jepsen worker 2 - jepsen.util 92 :invoke :write [0 19] INFO [2022-01-10 12:53:26,291] jepsen worker 23 - jepsen.util 23 :ok :write [7 35] INFO [2022-01-10 12:53:26,291] jepsen worker 23 - jepsen.util 23 :invoke :write [7 41] INFO [2022-01-10 12:53:26,294] jepsen worker 68 - jepsen.util 68 :ok :write [22 44] INFO [2022-01-10 12:53:26,295] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,296] jepsen worker 68 - jepsen.util 68 :ok :read [22 44] INFO [2022-01-10 12:53:26,296] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,297] jepsen worker 68 - jepsen.util 68 :ok :read [22 44] INFO [2022-01-10 12:53:26,297] jepsen worker 75 - jepsen.util 165 :ok :write [25 14] INFO [2022-01-10 12:53:26,298] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,298] jepsen worker 75 - jepsen.util 165 :invoke :write [25 24] INFO [2022-01-10 12:53:26,299] jepsen worker 68 - jepsen.util 68 :ok :read [22 44] INFO [2022-01-10 12:53:26,299] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,300] jepsen worker 68 - jepsen.util 68 :ok :read [22 44] INFO [2022-01-10 12:53:26,301] jepsen worker 68 - jepsen.util 68 :invoke :write [22 47] INFO [2022-01-10 12:53:26,302] jepsen worker 33 - jepsen.util 123 :ok :write [11 5] INFO [2022-01-10 12:53:26,302] jepsen worker 33 - jepsen.util 123 :invoke :write [11 1] INFO [2022-01-10 12:53:26,306] jepsen worker 15 - jepsen.util 15 :ok :write [5 25] INFO [2022-01-10 12:53:26,306] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:26,307] jepsen worker 15 - jepsen.util 15 :ok :read [5 25] INFO [2022-01-10 12:53:26,308] jepsen worker 15 - jepsen.util 15 :invoke :write [5 32] INFO [2022-01-10 12:53:26,310] jepsen worker 48 - jepsen.util 228 :ok :write [16 34] INFO [2022-01-10 12:53:26,311] jepsen worker 48 - jepsen.util 228 :invoke :write [16 19] INFO [2022-01-10 12:53:26,317] jepsen worker 81 - jepsen.util 171 :ok :write [27 40] INFO [2022-01-10 12:53:26,317] jepsen worker 81 - jepsen.util 171 :invoke :write [27 41] INFO [2022-01-10 12:53:26,319] jepsen worker 8 - jepsen.util 8 :ok :write [2 13] INFO [2022-01-10 12:53:26,320] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:26,320] jepsen worker 8 - jepsen.util 8 :ok :read [2 13] INFO [2022-01-10 12:53:26,321] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:26,322] jepsen worker 8 - jepsen.util 8 :ok :read [2 13] INFO [2022-01-10 12:53:26,322] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:26,323] jepsen worker 8 - jepsen.util 8 :ok :read [2 13] INFO [2022-01-10 12:53:26,323] jepsen worker 8 - jepsen.util 8 :invoke :write [2 25] INFO [2022-01-10 12:53:26,328] jepsen worker 68 - jepsen.util 68 :ok :write [22 47] INFO [2022-01-10 12:53:26,329] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,330] jepsen worker 68 - jepsen.util 68 :ok :read [22 47] INFO [2022-01-10 12:53:26,330] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,331] jepsen worker 68 - jepsen.util 68 :ok :read [22 47] INFO [2022-01-10 12:53:26,332] jepsen worker 68 - jepsen.util 68 :invoke :write [22 16] INFO [2022-01-10 12:53:26,334] jepsen worker 33 - jepsen.util 123 :ok :write [11 1] INFO [2022-01-10 12:53:26,334] jepsen worker 33 - jepsen.util 123 :invoke :write [11 46] INFO [2022-01-10 12:53:26,338] jepsen worker 15 - jepsen.util 15 :ok :write [5 32] INFO [2022-01-10 12:53:26,338] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:26,339] jepsen worker 15 - jepsen.util 15 :ok :read [5 32] INFO [2022-01-10 12:53:26,340] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:26,340] jepsen worker 15 - jepsen.util 15 :ok :read [5 32] INFO [2022-01-10 12:53:26,341] jepsen worker 15 - jepsen.util 15 :invoke :write [5 28] INFO [2022-01-10 12:53:26,342] jepsen worker 48 - jepsen.util 228 :ok :write [16 19] INFO [2022-01-10 12:53:26,343] jepsen worker 48 - jepsen.util 228 :invoke :write [16 27] INFO [2022-01-10 12:53:26,346] jepsen worker 81 - jepsen.util 171 :ok :write [27 41] INFO [2022-01-10 12:53:26,346] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:26,347] jepsen worker 81 - jepsen.util 171 :ok :read [27 41] INFO [2022-01-10 12:53:26,348] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:26,349] jepsen worker 81 - jepsen.util 171 :ok :read [27 41] INFO [2022-01-10 12:53:26,349] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:26,350] jepsen worker 8 - jepsen.util 8 :ok :write [2 25] INFO [2022-01-10 12:53:26,350] jepsen worker 81 - jepsen.util 171 :ok :read [27 41] INFO [2022-01-10 12:53:26,350] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:26,351] jepsen worker 81 - jepsen.util 171 :invoke :write [27 1] INFO [2022-01-10 12:53:26,351] jepsen worker 8 - jepsen.util 8 :ok :read [2 25] INFO [2022-01-10 12:53:26,352] jepsen worker 8 - jepsen.util 8 :invoke :write [2 38] INFO [2022-01-10 12:53:26,354] jepsen worker 68 - jepsen.util 68 :ok :write [22 16] INFO [2022-01-10 12:53:26,355] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,355] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:26,356] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,357] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:26,357] jepsen worker 68 - jepsen.util 68 :invoke :write [22 15] INFO [2022-01-10 12:53:26,359] jepsen worker 33 - jepsen.util 123 :ok :write [11 46] INFO [2022-01-10 12:53:26,359] jepsen worker 33 - jepsen.util 123 :invoke :write [11 11] INFO [2022-01-10 12:53:26,363] jepsen worker 15 - jepsen.util 15 :ok :write [5 28] INFO [2022-01-10 12:53:26,363] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:26,364] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:26,364] jepsen worker 15 - jepsen.util 15 :invoke :write [5 3] INFO [2022-01-10 12:53:26,366] jepsen worker 48 - jepsen.util 228 :ok :write [16 27] INFO [2022-01-10 12:53:26,367] jepsen worker 48 - jepsen.util 228 :invoke :write [16 22] INFO [2022-01-10 12:53:26,371] jepsen worker 81 - jepsen.util 171 :ok :write [27 1] INFO [2022-01-10 12:53:26,371] jepsen worker 81 - jepsen.util 171 :invoke :write [27 46] INFO [2022-01-10 12:53:26,374] jepsen worker 8 - jepsen.util 8 :ok :write [2 38] INFO [2022-01-10 12:53:26,375] jepsen worker 8 - jepsen.util 8 :invoke :write [2 38] INFO [2022-01-10 12:53:26,378] jepsen worker 68 - jepsen.util 68 :ok :write [22 15] INFO [2022-01-10 12:53:26,379] jepsen worker 68 - jepsen.util 68 :invoke :write [22 29] INFO [2022-01-10 12:53:26,382] jepsen worker 33 - jepsen.util 123 :ok :write [11 11] INFO [2022-01-10 12:53:26,382] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:26,383] jepsen worker 33 - jepsen.util 123 :ok :read [11 11] INFO [2022-01-10 12:53:26,384] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:26,384] jepsen worker 33 - jepsen.util 123 :ok :read [11 11] INFO [2022-01-10 12:53:26,385] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:26,386] jepsen worker 15 - jepsen.util 15 :ok :write [5 3] INFO [2022-01-10 12:53:26,386] jepsen worker 33 - jepsen.util 123 :ok :read [11 11] INFO [2022-01-10 12:53:26,386] jepsen worker 33 - jepsen.util 123 :invoke :write [11 42] INFO [2022-01-10 12:53:26,387] jepsen worker 15 - jepsen.util 15 :invoke :write [5 8] INFO [2022-01-10 12:53:26,401] jepsen worker 69 - jepsen.util 159 :ok :write [23 36] INFO [2022-01-10 12:53:26,401] jepsen worker 40 - jepsen.util 130 :ok :write [13 27] INFO [2022-01-10 12:53:26,401] jepsen worker 40 - jepsen.util 130 :invoke :write [13 49] INFO [2022-01-10 12:53:26,402] jepsen worker 69 - jepsen.util 159 :invoke :write [23 26] INFO [2022-01-10 12:53:26,405] jepsen worker 4 - jepsen.util 4 :ok :write [1 31] INFO [2022-01-10 12:53:26,406] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,407] jepsen worker 4 - jepsen.util 4 :ok :read [1 31] INFO [2022-01-10 12:53:26,407] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,408] jepsen worker 4 - jepsen.util 4 :ok :read [1 31] INFO [2022-01-10 12:53:26,408] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,409] jepsen worker 4 - jepsen.util 4 :ok :read [1 31] INFO [2022-01-10 12:53:26,409] jepsen worker 61 - jepsen.util 61 :ok :write [20 19] INFO [2022-01-10 12:53:26,410] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:26,410] jepsen worker 4 - jepsen.util 4 :invoke :write [1 22] INFO [2022-01-10 12:53:26,411] jepsen worker 13 - jepsen.util 13 :ok :write [4 13] INFO [2022-01-10 12:53:26,411] jepsen worker 61 - jepsen.util 61 :ok :read [20 19] INFO [2022-01-10 12:53:26,411] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,411] jepsen worker 61 - jepsen.util 61 :invoke :write [20 26] INFO [2022-01-10 12:53:26,412] jepsen worker 13 - jepsen.util 13 :ok :read [4 13] INFO [2022-01-10 12:53:26,413] jepsen worker 13 - jepsen.util 13 :invoke :write [4 5] INFO [2022-01-10 12:53:26,417] jepsen worker 87 - jepsen.util 87 :ok :write [29 39] INFO [2022-01-10 12:53:26,417] jepsen worker 37 - jepsen.util 127 :ok :write [12 6] INFO [2022-01-10 12:53:26,418] jepsen worker 87 - jepsen.util 87 :invoke :write [29 42] INFO [2022-01-10 12:53:26,418] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,422] jepsen worker 80 - jepsen.util 260 :ok :write [26 41] INFO [2022-01-10 12:53:26,423] jepsen worker 80 - jepsen.util 260 :invoke :write [26 11] INFO [2022-01-10 12:53:26,426] jepsen worker 2 - jepsen.util 92 :ok :write [0 19] INFO [2022-01-10 12:53:26,427] jepsen worker 53 - jepsen.util 143 :ok :write [17 46] INFO [2022-01-10 12:53:26,427] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,427] jepsen worker 2 - jepsen.util 92 :ok :read [0 19] INFO [2022-01-10 12:53:26,428] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:26,428] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,429] jepsen worker 2 - jepsen.util 92 :ok :read [0 19] INFO [2022-01-10 12:53:26,429] jepsen worker 37 - jepsen.util 127 :ok :read [12 6] INFO [2022-01-10 12:53:26,430] jepsen worker 53 - jepsen.util 143 :ok :read [17 46] INFO [2022-01-10 12:53:26,430] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,430] jepsen worker 37 - jepsen.util 127 :invoke :write [12 0] INFO [2022-01-10 12:53:26,430] jepsen worker 2 - jepsen.util 92 :ok :read [0 19] INFO [2022-01-10 12:53:26,431] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:26,431] jepsen worker 2 - jepsen.util 92 :invoke :write [0 19] INFO [2022-01-10 12:53:26,432] jepsen worker 53 - jepsen.util 143 :ok :read [17 46] INFO [2022-01-10 12:53:26,433] jepsen worker 53 - jepsen.util 143 :invoke :write [17 7] INFO [2022-01-10 12:53:26,433] jepsen worker 44 - jepsen.util 44 :ok :write [14 37] INFO [2022-01-10 12:53:26,434] jepsen worker 84 - jepsen.util 84 :ok :write [28 5] INFO [2022-01-10 12:53:26,434] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,434] jepsen worker 44 - jepsen.util 44 :invoke :write [14 38] INFO [2022-01-10 12:53:26,435] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:26,435] jepsen worker 84 - jepsen.util 84 :invoke :write [28 5] INFO [2022-01-10 12:53:26,438] jepsen worker 23 - jepsen.util 23 :ok :write [7 41] INFO [2022-01-10 12:53:26,439] jepsen worker 23 - jepsen.util 23 :invoke :write [7 14] INFO [2022-01-10 12:53:26,441] jepsen worker 75 - jepsen.util 165 :ok :write [25 24] INFO [2022-01-10 12:53:26,441] jepsen worker 75 - jepsen.util 165 :invoke :write [25 4] INFO [2022-01-10 12:53:26,479] jepsen worker 48 - jepsen.util 228 :ok :write [16 22] INFO [2022-01-10 12:53:26,480] jepsen worker 48 - jepsen.util 228 :invoke :write [16 7] INFO [2022-01-10 12:53:26,483] jepsen worker 69 - jepsen.util 159 :ok :write [23 26] INFO [2022-01-10 12:53:26,483] jepsen worker 81 - jepsen.util 171 :ok :write [27 46] INFO [2022-01-10 12:53:26,484] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,484] jepsen worker 81 - jepsen.util 171 :invoke :write [27 49] INFO [2022-01-10 12:53:26,484] jepsen worker 69 - jepsen.util 159 :ok :read [23 26] INFO [2022-01-10 12:53:26,485] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,485] jepsen worker 4 - jepsen.util 4 :ok :write [1 22] INFO [2022-01-10 12:53:26,486] jepsen worker 69 - jepsen.util 159 :ok :read [23 26] INFO [2022-01-10 12:53:26,486] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,486] jepsen worker 69 - jepsen.util 159 :invoke :write [23 18] INFO [2022-01-10 12:53:26,487] jepsen worker 4 - jepsen.util 4 :ok :read [1 22] INFO [2022-01-10 12:53:26,487] jepsen worker 4 - jepsen.util 4 :invoke :write [1 20] INFO [2022-01-10 12:53:26,488] jepsen worker 13 - jepsen.util 13 :ok :write [4 5] INFO [2022-01-10 12:53:26,488] jepsen worker 8 - jepsen.util 8 :ok :write [2 38] INFO [2022-01-10 12:53:26,488] jepsen worker 13 - jepsen.util 13 :invoke :write [4 32] INFO [2022-01-10 12:53:26,489] jepsen worker 8 - jepsen.util 8 :invoke :write [2 29] INFO [2022-01-10 12:53:26,491] jepsen worker 37 - jepsen.util 127 :ok :write [12 0] INFO [2022-01-10 12:53:26,491] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,492] jepsen worker 37 - jepsen.util 127 :ok :read [12 0] INFO [2022-01-10 12:53:26,493] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,493] jepsen worker 37 - jepsen.util 127 :ok :read [12 0] INFO [2022-01-10 12:53:26,494] jepsen worker 80 - jepsen.util 260 :ok :write [26 11] INFO [2022-01-10 12:53:26,494] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,495] jepsen worker 37 - jepsen.util 127 :ok :read [12 0] INFO [2022-01-10 12:53:26,495] jepsen worker 80 - jepsen.util 260 :invoke :write [26 6] INFO [2022-01-10 12:53:26,495] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,496] jepsen worker 37 - jepsen.util 127 :ok :read [12 0] INFO [2022-01-10 12:53:26,496] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,497] jepsen worker 68 - jepsen.util 68 :ok :write [22 29] INFO [2022-01-10 12:53:26,497] jepsen worker 37 - jepsen.util 127 :ok :read [12 0] INFO [2022-01-10 12:53:26,497] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,498] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,499] jepsen worker 68 - jepsen.util 68 :ok :read [22 29] INFO [2022-01-10 12:53:26,499] jepsen worker 84 - jepsen.util 84 :ok :write [28 5] INFO [2022-01-10 12:53:26,499] jepsen worker 37 - jepsen.util 127 :ok :read [12 0] INFO [2022-01-10 12:53:26,500] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:26,500] jepsen worker 53 - jepsen.util 143 :ok :write [17 7] INFO [2022-01-10 12:53:26,500] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,501] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:26,501] jepsen worker 37 - jepsen.util 127 :invoke :write [12 45] INFO [2022-01-10 12:53:26,501] jepsen worker 68 - jepsen.util 68 :ok :read [22 29] INFO [2022-01-10 12:53:26,501] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:26,502] jepsen worker 84 - jepsen.util 84 :invoke :write [28 38] INFO [2022-01-10 12:53:26,503] jepsen worker 68 - jepsen.util 68 :invoke :write [22 29] INFO [2022-01-10 12:53:26,503] jepsen worker 53 - jepsen.util 143 :ok :read [17 7] INFO [2022-01-10 12:53:26,503] jepsen worker 40 - jepsen.util 130 :ok :write [13 49] INFO [2022-01-10 12:53:26,503] jepsen worker 33 - jepsen.util 123 :ok :write [11 42] INFO [2022-01-10 12:53:26,504] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:26,504] jepsen worker 33 - jepsen.util 123 :invoke :write [11 43] INFO [2022-01-10 12:53:26,504] jepsen worker 40 - jepsen.util 130 :invoke :write [13 2] INFO [2022-01-10 12:53:26,505] jepsen worker 53 - jepsen.util 143 :ok :read [17 7] INFO [2022-01-10 12:53:26,505] jepsen worker 69 - jepsen.util 159 :ok :write [23 18] INFO [2022-01-10 12:53:26,505] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:26,506] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,506] jepsen worker 53 - jepsen.util 143 :ok :read [17 7] INFO [2022-01-10 12:53:26,506] jepsen worker 4 - jepsen.util 4 :ok :write [1 20] INFO [2022-01-10 12:53:26,506] jepsen worker 69 - jepsen.util 159 :ok :read [23 18] INFO [2022-01-10 12:53:26,507] jepsen worker 53 - jepsen.util 143 :invoke :write [17 40] INFO [2022-01-10 12:53:26,507] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,508] jepsen worker 15 - jepsen.util 15 :ok :write [5 8] INFO [2022-01-10 12:53:26,508] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,508] jepsen worker 69 - jepsen.util 159 :ok :read [23 18] INFO [2022-01-10 12:53:26,508] jepsen worker 15 - jepsen.util 15 :invoke :write [5 37] INFO [2022-01-10 12:53:26,509] jepsen worker 4 - jepsen.util 4 :ok :read [1 20] INFO [2022-01-10 12:53:26,509] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,509] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,510] jepsen worker 69 - jepsen.util 159 :ok :read [23 18] INFO [2022-01-10 12:53:26,510] jepsen worker 4 - jepsen.util 4 :ok :read [1 20] INFO [2022-01-10 12:53:26,511] jepsen worker 69 - jepsen.util 159 :invoke :write [23 38] INFO [2022-01-10 12:53:26,511] jepsen worker 4 - jepsen.util 4 :invoke :write [1 41] INFO [2022-01-10 12:53:26,512] jepsen worker 13 - jepsen.util 13 :ok :write [4 32] INFO [2022-01-10 12:53:26,513] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,513] jepsen worker 87 - jepsen.util 87 :ok :write [29 42] INFO [2022-01-10 12:53:26,514] jepsen worker 61 - jepsen.util 61 :ok :write [20 26] INFO [2022-01-10 12:53:26,514] jepsen worker 13 - jepsen.util 13 :ok :read [4 32] INFO [2022-01-10 12:53:26,514] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,514] jepsen worker 37 - jepsen.util 127 :ok :write [12 45] INFO [2022-01-10 12:53:26,515] jepsen worker 87 - jepsen.util 87 :ok :read [29 42] INFO [2022-01-10 12:53:26,515] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,516] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,516] jepsen worker 13 - jepsen.util 13 :ok :read [4 32] INFO [2022-01-10 12:53:26,516] jepsen worker 37 - jepsen.util 127 :invoke :write [12 37] INFO [2022-01-10 12:53:26,516] jepsen worker 87 - jepsen.util 87 :ok :read [29 42] INFO [2022-01-10 12:53:26,517] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:26,517] jepsen worker 2 - jepsen.util 92 :ok :write [0 19] INFO [2022-01-10 12:53:26,518] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,518] jepsen worker 61 - jepsen.util 61 :ok :read [20 26] INFO [2022-01-10 12:53:26,518] jepsen worker 13 - jepsen.util 13 :invoke :write [4 34] INFO [2022-01-10 12:53:26,518] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,519] jepsen worker 87 - jepsen.util 87 :ok :read [29 42] INFO [2022-01-10 12:53:26,519] jepsen worker 61 - jepsen.util 61 :invoke :write [20 4] INFO [2022-01-10 12:53:26,519] jepsen worker 80 - jepsen.util 260 :ok :write [26 6] INFO [2022-01-10 12:53:26,519] jepsen worker 2 - jepsen.util 92 :ok :read [0 19] INFO [2022-01-10 12:53:26,520] jepsen worker 87 - jepsen.util 87 :invoke :write [29 17] INFO [2022-01-10 12:53:26,521] jepsen worker 2 - jepsen.util 92 :invoke :write [0 3] INFO [2022-01-10 12:53:26,521] jepsen worker 80 - jepsen.util 260 :invoke :write [26 12] INFO [2022-01-10 12:53:26,521] jepsen worker 43 - jepsen.util 43 :ok :write [14 1] INFO [2022-01-10 12:53:26,522] jepsen worker 43 - jepsen.util 43 :invoke :write [14 3] INFO [2022-01-10 12:53:26,525] jepsen worker 53 - jepsen.util 143 :ok :write [17 40] INFO [2022-01-10 12:53:26,525] jepsen worker 53 - jepsen.util 143 :invoke :write [17 36] INFO [2022-01-10 12:53:26,527] jepsen worker 84 - jepsen.util 84 :ok :write [28 38] INFO [2022-01-10 12:53:26,527] jepsen worker 23 - jepsen.util 23 :ok :write [7 14] INFO [2022-01-10 12:53:26,527] jepsen worker 84 - jepsen.util 84 :invoke :write [28 18] INFO [2022-01-10 12:53:26,528] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:26,529] jepsen worker 23 - jepsen.util 23 :ok :read [7 14] INFO [2022-01-10 12:53:26,529] jepsen worker 23 - jepsen.util 23 :invoke :write [7 41] INFO [2022-01-10 12:53:26,533] jepsen worker 69 - jepsen.util 159 :ok :write [23 38] INFO [2022-01-10 12:53:26,533] jepsen worker 75 - jepsen.util 165 :ok :write [25 4] INFO [2022-01-10 12:53:26,533] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:26,533] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,534] jepsen worker 75 - jepsen.util 165 :ok :read [25 4] INFO [2022-01-10 12:53:26,534] jepsen worker 69 - jepsen.util 159 :ok :read [23 38] INFO [2022-01-10 12:53:26,535] jepsen worker 75 - jepsen.util 165 :invoke :write [25 16] INFO [2022-01-10 12:53:26,535] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,536] jepsen worker 69 - jepsen.util 159 :ok :read [23 38] INFO [2022-01-10 12:53:26,536] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,537] jepsen worker 69 - jepsen.util 159 :ok :read [23 38] INFO [2022-01-10 12:53:26,538] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,539] jepsen worker 4 - jepsen.util 4 :ok :write [1 41] INFO [2022-01-10 12:53:26,539] jepsen worker 69 - jepsen.util 159 :ok :read [23 38] INFO [2022-01-10 12:53:26,540] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,540] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,541] jepsen worker 69 - jepsen.util 159 :ok :read [23 38] INFO [2022-01-10 12:53:26,541] jepsen worker 48 - jepsen.util 228 :ok :write [16 7] INFO [2022-01-10 12:53:26,541] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,541] jepsen worker 4 - jepsen.util 4 :ok :read [1 41] INFO [2022-01-10 12:53:26,542] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:26,542] jepsen worker 69 - jepsen.util 159 :ok :read [23 38] INFO [2022-01-10 12:53:26,542] jepsen worker 4 - jepsen.util 4 :invoke :write [1 23] INFO [2022-01-10 12:53:26,543] jepsen worker 48 - jepsen.util 228 :ok :read [16 7] INFO [2022-01-10 12:53:26,543] jepsen worker 69 - jepsen.util 159 :invoke :write [23 29] INFO [2022-01-10 12:53:26,543] jepsen worker 48 - jepsen.util 228 :invoke :write [16 32] INFO [2022-01-10 12:53:26,543] jepsen worker 81 - jepsen.util 171 :ok :write [27 49] INFO [2022-01-10 12:53:26,544] jepsen worker 13 - jepsen.util 13 :ok :write [4 34] INFO [2022-01-10 12:53:26,544] jepsen worker 81 - jepsen.util 171 :invoke :write [27 29] INFO [2022-01-10 12:53:26,544] jepsen worker 13 - jepsen.util 13 :invoke :write [4 42] INFO [2022-01-10 12:53:26,548] jepsen worker 40 - jepsen.util 130 :ok :write [13 2] INFO [2022-01-10 12:53:26,548] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:26,549] jepsen worker 40 - jepsen.util 130 :ok :read [13 2] INFO [2022-01-10 12:53:26,549] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:26,550] jepsen worker 8 - jepsen.util 8 :ok :write [2 29] INFO [2022-01-10 12:53:26,550] jepsen worker 40 - jepsen.util 130 :ok :read [13 2] INFO [2022-01-10 12:53:26,551] jepsen worker 8 - jepsen.util 8 :invoke :write [2 42] INFO [2022-01-10 12:53:26,551] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:26,552] jepsen worker 40 - jepsen.util 130 :ok :read [13 2] INFO [2022-01-10 12:53:26,553] jepsen worker 40 - jepsen.util 130 :invoke :write [13 39] INFO [2022-01-10 12:53:26,553] jepsen worker 37 - jepsen.util 127 :ok :write [12 37] INFO [2022-01-10 12:53:26,554] jepsen worker 37 - jepsen.util 127 :invoke :write [12 49] INFO [2022-01-10 12:53:26,558] jepsen worker 68 - jepsen.util 68 :ok :write [22 29] INFO [2022-01-10 12:53:26,559] jepsen worker 68 - jepsen.util 68 :invoke :write [22 34] INFO [2022-01-10 12:53:26,560] jepsen worker 80 - jepsen.util 260 :ok :write [26 12] INFO [2022-01-10 12:53:26,561] jepsen worker 80 - jepsen.util 260 :invoke :write [26 28] INFO [2022-01-10 12:53:26,562] jepsen worker 33 - jepsen.util 123 :ok :write [11 43] INFO [2022-01-10 12:53:26,563] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:26,564] jepsen worker 33 - jepsen.util 123 :ok :read [11 43] INFO [2022-01-10 12:53:26,565] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:26,565] jepsen worker 33 - jepsen.util 123 :ok :read [11 43] INFO [2022-01-10 12:53:26,566] jepsen worker 33 - jepsen.util 123 :invoke :write [11 4] INFO [2022-01-10 12:53:26,566] jepsen worker 53 - jepsen.util 143 :ok :write [17 36] INFO [2022-01-10 12:53:26,567] jepsen worker 53 - jepsen.util 143 :invoke :write [17 15] INFO [2022-01-10 12:53:26,569] jepsen worker 15 - jepsen.util 15 :ok :write [5 37] INFO [2022-01-10 12:53:26,570] jepsen worker 15 - jepsen.util 15 :invoke :write [5 4] INFO [2022-01-10 12:53:26,573] jepsen worker 84 - jepsen.util 84 :ok :write [28 18] INFO [2022-01-10 12:53:26,574] jepsen worker 88 - jepsen.util 178 :ok :write [29 34] INFO [2022-01-10 12:53:26,574] jepsen worker 84 - jepsen.util 84 :invoke :write [28 37] INFO [2022-01-10 12:53:26,574] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,575] jepsen worker 61 - jepsen.util 61 :ok :write [20 4] INFO [2022-01-10 12:53:26,575] jepsen worker 88 - jepsen.util 178 :ok :read [29 34] INFO [2022-01-10 12:53:26,576] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:26,576] jepsen worker 88 - jepsen.util 178 :invoke :write [29 19] INFO [2022-01-10 12:53:26,576] jepsen worker 61 - jepsen.util 61 :ok :read [20 4] INFO [2022-01-10 12:53:26,577] jepsen worker 61 - jepsen.util 61 :invoke :write [20 40] INFO [2022-01-10 12:53:26,577] jepsen worker 2 - jepsen.util 92 :ok :write [0 3] INFO [2022-01-10 12:53:26,578] jepsen worker 69 - jepsen.util 159 :ok :write [23 29] INFO [2022-01-10 12:53:26,578] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,578] jepsen worker 69 - jepsen.util 159 :invoke :write [23 7] INFO [2022-01-10 12:53:26,579] jepsen worker 2 - jepsen.util 92 :ok :read [0 3] INFO [2022-01-10 12:53:26,580] jepsen worker 2 - jepsen.util 92 :invoke :write [0 44] INFO [2022-01-10 12:53:26,583] jepsen worker 43 - jepsen.util 43 :ok :write [14 3] INFO [2022-01-10 12:53:26,583] jepsen worker 43 - jepsen.util 43 :invoke :write [14 22] INFO [2022-01-10 12:53:26,585] jepsen worker 4 - jepsen.util 4 :ok :write [1 23] INFO [2022-01-10 12:53:26,586] jepsen worker 4 - jepsen.util 4 :invoke :write [1 40] INFO [2022-01-10 12:53:26,589] jepsen worker 23 - jepsen.util 23 :ok :write [7 41] INFO [2022-01-10 12:53:26,589] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:26,590] jepsen worker 23 - jepsen.util 23 :ok :read [7 41] INFO [2022-01-10 12:53:26,591] jepsen worker 13 - jepsen.util 13 :ok :write [4 42] INFO [2022-01-10 12:53:26,591] jepsen worker 23 - jepsen.util 23 :invoke :write [7 23] INFO [2022-01-10 12:53:26,591] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,592] jepsen worker 13 - jepsen.util 13 :ok :read [4 42] INFO [2022-01-10 12:53:26,593] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,594] jepsen worker 13 - jepsen.util 13 :ok :read [4 42] INFO [2022-01-10 12:53:26,594] jepsen worker 13 - jepsen.util 13 :invoke :write [4 3] INFO [2022-01-10 12:53:26,595] jepsen worker 75 - jepsen.util 165 :ok :write [25 16] INFO [2022-01-10 12:53:26,596] jepsen worker 37 - jepsen.util 127 :ok :write [12 49] INFO [2022-01-10 12:53:26,596] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:26,596] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,597] jepsen worker 40 - jepsen.util 130 :ok :write [13 39] INFO [2022-01-10 12:53:26,597] jepsen worker 75 - jepsen.util 165 :ok :read [25 16] INFO [2022-01-10 12:53:26,597] jepsen worker 37 - jepsen.util 127 :ok :read [12 49] INFO [2022-01-10 12:53:26,597] jepsen worker 40 - jepsen.util 130 :invoke :write [13 1] INFO [2022-01-10 12:53:26,598] jepsen worker 75 - jepsen.util 165 :invoke :write [25 21] INFO [2022-01-10 12:53:26,598] jepsen worker 37 - jepsen.util 127 :invoke :write [12 21] INFO [2022-01-10 12:53:26,603] jepsen worker 48 - jepsen.util 228 :ok :write [16 32] INFO [2022-01-10 12:53:26,603] jepsen worker 48 - jepsen.util 228 :invoke :write [16 27] INFO [2022-01-10 12:53:26,604] jepsen worker 80 - jepsen.util 260 :ok :write [26 28] INFO [2022-01-10 12:53:26,604] jepsen worker 80 - jepsen.util 260 :invoke :write [26 35] INFO [2022-01-10 12:53:26,606] jepsen worker 81 - jepsen.util 171 :ok :write [27 29] INFO [2022-01-10 12:53:26,606] jepsen worker 81 - jepsen.util 171 :invoke :write [27 8] INFO [2022-01-10 12:53:26,610] jepsen worker 8 - jepsen.util 8 :ok :write [2 42] INFO [2022-01-10 12:53:26,610] jepsen worker 53 - jepsen.util 143 :ok :write [17 15] INFO [2022-01-10 12:53:26,611] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:26,611] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:26,612] jepsen worker 8 - jepsen.util 8 :ok :read [2 42] INFO [2022-01-10 12:53:26,612] jepsen worker 53 - jepsen.util 143 :ok :read [17 15] INFO [2022-01-10 12:53:26,612] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:26,613] jepsen worker 53 - jepsen.util 143 :invoke :write [17 16] INFO [2022-01-10 12:53:26,613] jepsen worker 8 - jepsen.util 8 :ok :read [2 42] INFO [2022-01-10 12:53:26,613] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:26,614] jepsen worker 8 - jepsen.util 8 :ok :read [2 42] INFO [2022-01-10 12:53:26,615] jepsen worker 8 - jepsen.util 8 :invoke :write [2 44] INFO [2022-01-10 12:53:26,617] jepsen worker 84 - jepsen.util 84 :ok :write [28 37] INFO [2022-01-10 12:53:26,617] jepsen worker 84 - jepsen.util 84 :invoke :write [28 40] INFO [2022-01-10 12:53:26,620] jepsen worker 68 - jepsen.util 68 :ok :write [22 34] INFO [2022-01-10 12:53:26,620] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,621] jepsen worker 68 - jepsen.util 68 :ok :read [22 34] INFO [2022-01-10 12:53:26,621] jepsen worker 69 - jepsen.util 159 :ok :write [23 7] INFO [2022-01-10 12:53:26,622] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,622] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,622] jepsen worker 68 - jepsen.util 68 :ok :read [22 34] INFO [2022-01-10 12:53:26,623] jepsen worker 69 - jepsen.util 159 :ok :read [23 7] INFO [2022-01-10 12:53:26,623] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,623] jepsen worker 33 - jepsen.util 123 :ok :write [11 4] INFO [2022-01-10 12:53:26,624] jepsen worker 68 - jepsen.util 68 :ok :read [22 34] INFO [2022-01-10 12:53:26,624] jepsen worker 33 - jepsen.util 123 :invoke :write [11 3] INFO [2022-01-10 12:53:26,625] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,625] jepsen worker 4 - jepsen.util 4 :ok :write [1 40] INFO [2022-01-10 12:53:26,625] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,625] jepsen worker 68 - jepsen.util 68 :ok :read [22 34] INFO [2022-01-10 12:53:26,626] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,626] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,626] jepsen worker 69 - jepsen.util 159 :ok :read [23 7] INFO [2022-01-10 12:53:26,627] jepsen worker 4 - jepsen.util 4 :ok :read [1 40] INFO [2022-01-10 12:53:26,627] jepsen worker 68 - jepsen.util 68 :ok :read [22 34] INFO [2022-01-10 12:53:26,627] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,627] jepsen worker 68 - jepsen.util 68 :invoke :write [22 27] INFO [2022-01-10 12:53:26,628] jepsen worker 69 - jepsen.util 159 :ok :read [23 7] INFO [2022-01-10 12:53:26,628] jepsen worker 4 - jepsen.util 4 :invoke :write [1 36] INFO [2022-01-10 12:53:26,628] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,628] jepsen worker 15 - jepsen.util 15 :ok :write [5 4] INFO [2022-01-10 12:53:26,629] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:26,629] jepsen worker 13 - jepsen.util 13 :ok :write [4 3] INFO [2022-01-10 12:53:26,629] jepsen worker 69 - jepsen.util 159 :ok :read [23 7] INFO [2022-01-10 12:53:26,630] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,630] jepsen worker 15 - jepsen.util 15 :ok :read [5 4] INFO [2022-01-10 12:53:26,630] jepsen worker 88 - jepsen.util 178 :ok :write [29 19] INFO [2022-01-10 12:53:26,630] jepsen worker 13 - jepsen.util 13 :ok :read [4 3] INFO [2022-01-10 12:53:26,631] jepsen worker 69 - jepsen.util 159 :invoke :write [23 38] INFO [2022-01-10 12:53:26,631] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,632] jepsen worker 88 - jepsen.util 178 :ok :read [29 19] INFO [2022-01-10 12:53:26,632] jepsen worker 13 - jepsen.util 13 :invoke :write [4 31] INFO [2022-01-10 12:53:26,632] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,633] jepsen worker 15 - jepsen.util 15 :invoke :write [5 14] INFO [2022-01-10 12:53:26,633] jepsen worker 88 - jepsen.util 178 :ok :read [29 19] INFO [2022-01-10 12:53:26,633] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,634] jepsen worker 61 - jepsen.util 61 :ok :write [20 40] INFO [2022-01-10 12:53:26,634] jepsen worker 88 - jepsen.util 178 :ok :read [29 19] INFO [2022-01-10 12:53:26,635] jepsen worker 61 - jepsen.util 61 :invoke :write [20 24] INFO [2022-01-10 12:53:26,635] jepsen worker 88 - jepsen.util 178 :invoke :write [29 24] INFO [2022-01-10 12:53:26,636] jepsen worker 2 - jepsen.util 92 :ok :write [0 44] INFO [2022-01-10 12:53:26,636] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,637] jepsen worker 2 - jepsen.util 92 :ok :read [0 44] INFO [2022-01-10 12:53:26,637] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,638] jepsen worker 2 - jepsen.util 92 :ok :read [0 44] INFO [2022-01-10 12:53:26,639] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,639] jepsen worker 43 - jepsen.util 43 :ok :write [14 22] INFO [2022-01-10 12:53:26,640] jepsen worker 2 - jepsen.util 92 :ok :read [0 44] INFO [2022-01-10 12:53:26,640] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:26,641] jepsen worker 2 - jepsen.util 92 :invoke :write [0 44] INFO [2022-01-10 12:53:26,641] jepsen worker 43 - jepsen.util 43 :ok :read [14 22] INFO [2022-01-10 12:53:26,642] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:26,642] jepsen worker 40 - jepsen.util 130 :ok :write [13 1] INFO [2022-01-10 12:53:26,642] jepsen worker 43 - jepsen.util 43 :ok :read [14 22] INFO [2022-01-10 12:53:26,643] jepsen worker 43 - jepsen.util 43 :invoke :write [14 25] INFO [2022-01-10 12:53:26,643] jepsen worker 40 - jepsen.util 130 :invoke :write [13 1] INFO [2022-01-10 12:53:26,644] jepsen worker 23 - jepsen.util 23 :ok :write [7 23] INFO [2022-01-10 12:53:26,644] jepsen worker 23 - jepsen.util 23 :invoke :write [7 27] INFO [2022-01-10 12:53:26,649] jepsen worker 75 - jepsen.util 165 :ok :write [25 21] INFO [2022-01-10 12:53:26,649] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:26,650] jepsen worker 75 - jepsen.util 165 :ok :read [25 21] INFO [2022-01-10 12:53:26,651] jepsen worker 75 - jepsen.util 165 :invoke :write [25 1] INFO [2022-01-10 12:53:26,654] jepsen worker 48 - jepsen.util 228 :ok :write [16 27] INFO [2022-01-10 12:53:26,655] jepsen worker 48 - jepsen.util 228 :invoke :write [16 35] INFO [2022-01-10 12:53:26,659] jepsen worker 81 - jepsen.util 171 :ok :write [27 8] INFO [2022-01-10 12:53:26,659] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:26,660] jepsen worker 81 - jepsen.util 171 :ok :read [27 8] INFO [2022-01-10 12:53:26,661] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:26,661] jepsen worker 8 - jepsen.util 8 :ok :write [2 44] INFO [2022-01-10 12:53:26,661] jepsen worker 81 - jepsen.util 171 :ok :read [27 8] INFO [2022-01-10 12:53:26,662] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:26,663] jepsen worker 45 - jepsen.util 225 :fail :read [15 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:26,663] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:26,664] jepsen worker 45 - jepsen.util 225 :invoke :read [15 nil] INFO [2022-01-10 12:53:26,664] jepsen worker 81 - jepsen.util 171 :ok :read [27 8] INFO [2022-01-10 12:53:26,664] jepsen worker 8 - jepsen.util 8 :ok :read [2 44] INFO [2022-01-10 12:53:26,665] jepsen worker 8 - jepsen.util 8 :invoke :write [2 31] INFO [2022-01-10 12:53:26,665] jepsen worker 81 - jepsen.util 171 :invoke :write [27 34] INFO [2022-01-10 12:53:26,666] jepsen worker 68 - jepsen.util 68 :ok :write [22 27] INFO [2022-01-10 12:53:26,667] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,668] jepsen worker 68 - jepsen.util 68 :ok :read [22 27] INFO [2022-01-10 12:53:26,668] jepsen worker 68 - jepsen.util 68 :invoke :write [22 6] INFO [2022-01-10 12:53:26,669] jepsen worker 33 - jepsen.util 123 :ok :write [11 3] INFO [2022-01-10 12:53:26,670] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:26,671] jepsen worker 33 - jepsen.util 123 :ok :read [11 3] INFO [2022-01-10 12:53:26,671] jepsen worker 33 - jepsen.util 123 :invoke :write [11 38] INFO [2022-01-10 12:53:26,674] jepsen worker 15 - jepsen.util 15 :ok :write [5 14] INFO [2022-01-10 12:53:26,674] jepsen worker 15 - jepsen.util 15 :invoke :write [5 8] INFO [2022-01-10 12:53:26,689] jepsen worker 48 - jepsen.util 228 :ok :write [16 35] INFO [2022-01-10 12:53:26,690] jepsen worker 48 - jepsen.util 228 :invoke :write [16 16] INFO [2022-01-10 12:53:26,693] jepsen worker 81 - jepsen.util 171 :ok :write [27 34] INFO [2022-01-10 12:53:26,694] jepsen worker 81 - jepsen.util 171 :invoke :write [27 29] INFO [2022-01-10 12:53:26,697] jepsen worker 8 - jepsen.util 8 :ok :write [2 31] INFO [2022-01-10 12:53:26,697] jepsen worker 6 - jepsen.util 96 :info :write [2 49] :RequestTimeoutException INFO [2022-01-10 12:53:26,697] jepsen worker 8 - jepsen.util 8 :invoke :write [2 36] INFO [2022-01-10 12:53:26,698] jepsen worker 6 - jepsen.util 186 :invoke :read [2 nil] INFO [2022-01-10 12:53:26,699] jepsen worker 6 - jepsen.util 186 :ok :read [2 31] INFO [2022-01-10 12:53:26,700] jepsen worker 6 - jepsen.util 186 :invoke :write [2 28] INFO [2022-01-10 12:53:26,703] jepsen worker 68 - jepsen.util 68 :ok :write [22 6] INFO [2022-01-10 12:53:26,703] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,704] jepsen worker 68 - jepsen.util 68 :ok :read [22 6] INFO [2022-01-10 12:53:26,704] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,705] jepsen worker 33 - jepsen.util 123 :ok :write [11 38] INFO [2022-01-10 12:53:26,705] jepsen worker 68 - jepsen.util 68 :ok :read [22 6] INFO [2022-01-10 12:53:26,705] jepsen worker 33 - jepsen.util 123 :invoke :write [11 19] INFO [2022-01-10 12:53:26,706] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:26,707] jepsen worker 68 - jepsen.util 68 :ok :read [22 6] INFO [2022-01-10 12:53:26,708] jepsen worker 68 - jepsen.util 68 :invoke :write [22 18] INFO [2022-01-10 12:53:26,766] jepsen worker 37 - jepsen.util 127 :ok :write [12 21] INFO [2022-01-10 12:53:26,767] jepsen worker 37 - jepsen.util 127 :invoke :write [12 15] INFO [2022-01-10 12:53:26,771] jepsen worker 80 - jepsen.util 260 :ok :write [26 35] INFO [2022-01-10 12:53:26,772] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,773] jepsen worker 61 - jepsen.util 61 :ok :write [20 24] INFO [2022-01-10 12:53:26,773] jepsen worker 80 - jepsen.util 260 :ok :read [26 35] INFO [2022-01-10 12:53:26,773] jepsen worker 61 - jepsen.util 61 :invoke :write [20 20] INFO [2022-01-10 12:53:26,774] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,775] jepsen worker 80 - jepsen.util 260 :ok :read [26 35] INFO [2022-01-10 12:53:26,775] jepsen worker 80 - jepsen.util 260 :invoke :write [26 38] INFO [2022-01-10 12:53:26,777] jepsen worker 88 - jepsen.util 178 :ok :write [29 24] INFO [2022-01-10 12:53:26,777] jepsen worker 53 - jepsen.util 143 :ok :write [17 16] INFO [2022-01-10 12:53:26,778] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:26,778] jepsen worker 88 - jepsen.util 178 :invoke :write [29 7] INFO [2022-01-10 12:53:26,779] jepsen worker 53 - jepsen.util 143 :ok :read [17 16] INFO [2022-01-10 12:53:26,779] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:26,780] jepsen worker 53 - jepsen.util 143 :ok :read [17 16] INFO [2022-01-10 12:53:26,781] jepsen worker 53 - jepsen.util 143 :invoke :write [17 46] INFO [2022-01-10 12:53:26,784] jepsen worker 2 - jepsen.util 92 :ok :write [0 44] INFO [2022-01-10 12:53:26,784] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,784] jepsen worker 84 - jepsen.util 84 :ok :write [28 40] INFO [2022-01-10 12:53:26,785] jepsen worker 84 - jepsen.util 84 :invoke :write [28 33] INFO [2022-01-10 12:53:26,785] jepsen worker 2 - jepsen.util 92 :ok :read [0 44] INFO [2022-01-10 12:53:26,786] jepsen worker 2 - jepsen.util 92 :invoke :write [0 17] INFO [2022-01-10 12:53:26,787] jepsen worker 43 - jepsen.util 43 :ok :write [14 25] INFO [2022-01-10 12:53:26,789] jepsen worker 43 - jepsen.util 43 :invoke :write [14 3] INFO [2022-01-10 12:53:26,790] jepsen worker 69 - jepsen.util 159 :ok :write [23 38] INFO [2022-01-10 12:53:26,790] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,791] jepsen worker 69 - jepsen.util 159 :ok :read [23 38] INFO [2022-01-10 12:53:26,791] jepsen worker 69 - jepsen.util 159 :invoke :write [23 21] INFO [2022-01-10 12:53:26,794] jepsen worker 40 - jepsen.util 130 :ok :write [13 1] INFO [2022-01-10 12:53:26,794] jepsen worker 40 - jepsen.util 130 :invoke :write [13 32] INFO [2022-01-10 12:53:26,795] jepsen worker 4 - jepsen.util 4 :ok :write [1 36] INFO [2022-01-10 12:53:26,796] jepsen worker 4 - jepsen.util 4 :invoke :write [1 5] INFO [2022-01-10 12:53:26,798] jepsen worker 23 - jepsen.util 23 :ok :write [7 27] INFO [2022-01-10 12:53:26,799] jepsen worker 23 - jepsen.util 23 :invoke :write [7 33] INFO [2022-01-10 12:53:26,801] jepsen worker 13 - jepsen.util 13 :ok :write [4 31] INFO [2022-01-10 12:53:26,802] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,802] jepsen worker 13 - jepsen.util 13 :ok :read [4 31] INFO [2022-01-10 12:53:26,803] jepsen worker 75 - jepsen.util 165 :ok :write [25 1] INFO [2022-01-10 12:53:26,803] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,803] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:26,804] jepsen worker 13 - jepsen.util 13 :ok :read [4 31] INFO [2022-01-10 12:53:26,804] jepsen worker 75 - jepsen.util 165 :ok :read [25 1] INFO [2022-01-10 12:53:26,804] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,805] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:26,805] jepsen worker 13 - jepsen.util 13 :ok :read [4 31] INFO [2022-01-10 12:53:26,806] jepsen worker 75 - jepsen.util 165 :ok :read [25 1] INFO [2022-01-10 12:53:26,806] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,806] jepsen worker 75 - jepsen.util 165 :invoke :write [25 31] INFO [2022-01-10 12:53:26,807] jepsen worker 13 - jepsen.util 13 :ok :read [4 31] INFO [2022-01-10 12:53:26,807] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,807] jepsen worker 37 - jepsen.util 127 :ok :write [12 15] INFO [2022-01-10 12:53:26,808] jepsen worker 13 - jepsen.util 13 :ok :read [4 31] INFO [2022-01-10 12:53:26,808] jepsen worker 37 - jepsen.util 127 :invoke :write [12 49] INFO [2022-01-10 12:53:26,808] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,809] jepsen worker 13 - jepsen.util 13 :ok :read [4 31] INFO [2022-01-10 12:53:26,809] jepsen worker 13 - jepsen.util 13 :invoke :write [4 45] INFO [2022-01-10 12:53:26,839] jepsen worker 15 - jepsen.util 15 :ok :write [5 8] INFO [2022-01-10 12:53:26,839] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:26,840] jepsen worker 15 - jepsen.util 15 :ok :read [5 8] INFO [2022-01-10 12:53:26,840] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:26,841] jepsen worker 15 - jepsen.util 15 :ok :read [5 8] INFO [2022-01-10 12:53:26,841] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:26,842] jepsen worker 15 - jepsen.util 15 :ok :read [5 8] INFO [2022-01-10 12:53:26,843] jepsen worker 48 - jepsen.util 228 :ok :write [16 16] INFO [2022-01-10 12:53:26,843] jepsen worker 15 - jepsen.util 15 :invoke :write [5 46] INFO [2022-01-10 12:53:26,843] jepsen worker 48 - jepsen.util 228 :invoke :write [16 48] INFO [2022-01-10 12:53:26,844] jepsen worker 80 - jepsen.util 260 :ok :write [26 38] INFO [2022-01-10 12:53:26,845] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,846] jepsen worker 80 - jepsen.util 260 :ok :read [26 38] INFO [2022-01-10 12:53:26,846] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,847] jepsen worker 81 - jepsen.util 171 :ok :write [27 29] INFO [2022-01-10 12:53:26,847] jepsen worker 80 - jepsen.util 260 :ok :read [26 38] INFO [2022-01-10 12:53:26,848] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:26,848] jepsen worker 53 - jepsen.util 143 :ok :write [17 46] INFO [2022-01-10 12:53:26,848] jepsen worker 81 - jepsen.util 171 :ok :read [27 29] INFO [2022-01-10 12:53:26,848] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,849] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:26,849] jepsen worker 80 - jepsen.util 260 :ok :read [26 38] INFO [2022-01-10 12:53:26,850] jepsen worker 53 - jepsen.util 143 :invoke :write [17 10] INFO [2022-01-10 12:53:26,850] jepsen worker 81 - jepsen.util 171 :ok :read [27 29] INFO [2022-01-10 12:53:26,850] jepsen worker 80 - jepsen.util 260 :invoke :write [26 21] INFO [2022-01-10 12:53:26,851] jepsen worker 81 - jepsen.util 171 :invoke :write [27 40] INFO [2022-01-10 12:53:26,854] jepsen worker 8 - jepsen.util 8 :ok :write [2 36] INFO [2022-01-10 12:53:26,855] jepsen worker 69 - jepsen.util 159 :ok :write [23 21] INFO [2022-01-10 12:53:26,855] jepsen worker 8 - jepsen.util 8 :invoke :write [2 40] INFO [2022-01-10 12:53:26,856] jepsen worker 69 - jepsen.util 159 :invoke :write [23 37] INFO [2022-01-10 12:53:26,861] jepsen worker 13 - jepsen.util 13 :ok :write [4 45] INFO [2022-01-10 12:53:26,861] jepsen worker 68 - jepsen.util 68 :ok :write [22 18] INFO [2022-01-10 12:53:26,862] jepsen worker 68 - jepsen.util 68 :invoke :write [22 2] INFO [2022-01-10 12:53:26,863] jepsen worker 13 - jepsen.util 13 :invoke :write [4 39] INFO [2022-01-10 12:53:26,868] jepsen worker 80 - jepsen.util 260 :ok :write [26 21] INFO [2022-01-10 12:53:26,868] jepsen worker 33 - jepsen.util 123 :ok :write [11 19] INFO [2022-01-10 12:53:26,869] jepsen worker 80 - jepsen.util 260 :invoke :write [26 3] INFO [2022-01-10 12:53:26,869] jepsen worker 33 - jepsen.util 123 :invoke :write [11 21] INFO [2022-01-10 12:53:26,873] jepsen worker 53 - jepsen.util 143 :ok :write [17 10] INFO [2022-01-10 12:53:26,874] jepsen worker 61 - jepsen.util 61 :ok :write [20 20] INFO [2022-01-10 12:53:26,874] jepsen worker 53 - jepsen.util 143 :invoke :write [17 31] INFO [2022-01-10 12:53:26,875] jepsen worker 61 - jepsen.util 61 :invoke :write [20 47] INFO [2022-01-10 12:53:26,879] jepsen worker 69 - jepsen.util 159 :ok :write [23 37] INFO [2022-01-10 12:53:26,879] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:26,880] jepsen worker 69 - jepsen.util 159 :ok :read [23 37] INFO [2022-01-10 12:53:26,880] jepsen worker 43 - jepsen.util 43 :ok :write [14 3] INFO [2022-01-10 12:53:26,881] jepsen worker 69 - jepsen.util 159 :invoke :write [23 13] INFO [2022-01-10 12:53:26,881] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:26,882] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:26,882] jepsen worker 13 - jepsen.util 13 :ok :write [4 39] INFO [2022-01-10 12:53:26,882] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:26,883] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:26,883] jepsen worker 13 - jepsen.util 13 :invoke :write [4 2] INFO [2022-01-10 12:53:26,884] jepsen worker 43 - jepsen.util 43 :invoke :write [14 44] INFO [2022-01-10 12:53:26,885] jepsen worker 23 - jepsen.util 23 :ok :write [7 33] INFO [2022-01-10 12:53:26,886] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:26,887] jepsen worker 23 - jepsen.util 23 :ok :read [7 33] INFO [2022-01-10 12:53:26,887] jepsen worker 23 - jepsen.util 23 :invoke :write [7 14] INFO [2022-01-10 12:53:26,887] jepsen worker 80 - jepsen.util 260 :ok :write [26 3] INFO [2022-01-10 12:53:26,888] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,889] jepsen worker 80 - jepsen.util 260 :ok :read [26 3] INFO [2022-01-10 12:53:26,889] jepsen worker 80 - jepsen.util 260 :invoke :write [26 15] INFO [2022-01-10 12:53:26,890] jepsen worker 75 - jepsen.util 165 :ok :write [25 31] INFO [2022-01-10 12:53:26,891] jepsen worker 75 - jepsen.util 165 :invoke :write [25 23] INFO [2022-01-10 12:53:26,892] jepsen worker 53 - jepsen.util 143 :ok :write [17 31] INFO [2022-01-10 12:53:26,892] jepsen worker 53 - jepsen.util 143 :invoke :write [17 44] INFO [2022-01-10 12:53:26,895] jepsen worker 69 - jepsen.util 159 :ok :write [23 13] INFO [2022-01-10 12:53:26,896] jepsen worker 69 - jepsen.util 159 :invoke :write [23 26] INFO [2022-01-10 12:53:26,902] jepsen worker 13 - jepsen.util 13 :ok :write [4 2] INFO [2022-01-10 12:53:26,903] jepsen worker 13 - jepsen.util 13 :invoke :write [4 32] INFO [2022-01-10 12:53:26,907] jepsen worker 88 - jepsen.util 178 :ok :write [29 7] INFO [2022-01-10 12:53:26,907] jepsen worker 80 - jepsen.util 260 :ok :write [26 15] INFO [2022-01-10 12:53:26,907] jepsen worker 84 - jepsen.util 84 :ok :write [28 33] INFO [2022-01-10 12:53:26,907] jepsen worker 80 - jepsen.util 260 :invoke :write [26 40] INFO [2022-01-10 12:53:26,908] jepsen worker 84 - jepsen.util 84 :invoke :write [28 21] INFO [2022-01-10 12:53:26,908] jepsen worker 88 - jepsen.util 178 :invoke :write [29 47] INFO [2022-01-10 12:53:26,912] jepsen worker 53 - jepsen.util 143 :ok :write [17 44] INFO [2022-01-10 12:53:26,912] jepsen worker 53 - jepsen.util 143 :invoke :write [17 4] INFO [2022-01-10 12:53:26,912] jepsen worker 2 - jepsen.util 92 :ok :write [0 17] INFO [2022-01-10 12:53:26,913] jepsen worker 4 - jepsen.util 4 :ok :write [1 5] INFO [2022-01-10 12:53:26,913] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,914] jepsen worker 4 - jepsen.util 4 :invoke :write [1 20] INFO [2022-01-10 12:53:26,914] jepsen worker 2 - jepsen.util 92 :ok :read [0 17] INFO [2022-01-10 12:53:26,915] jepsen worker 2 - jepsen.util 92 :invoke :write [0 8] INFO [2022-01-10 12:53:26,918] jepsen worker 69 - jepsen.util 159 :ok :write [23 26] INFO [2022-01-10 12:53:26,919] jepsen worker 37 - jepsen.util 127 :ok :write [12 49] INFO [2022-01-10 12:53:26,919] jepsen worker 69 - jepsen.util 159 :invoke :write [23 20] INFO [2022-01-10 12:53:26,919] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,920] jepsen worker 37 - jepsen.util 127 :ok :read [12 49] INFO [2022-01-10 12:53:26,921] jepsen worker 40 - jepsen.util 130 :ok :write [13 32] INFO [2022-01-10 12:53:26,921] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,921] jepsen worker 40 - jepsen.util 130 :invoke :write [13 38] INFO [2022-01-10 12:53:26,922] jepsen worker 37 - jepsen.util 127 :ok :read [12 49] INFO [2022-01-10 12:53:26,922] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,922] jepsen worker 13 - jepsen.util 13 :ok :write [4 32] INFO [2022-01-10 12:53:26,923] jepsen worker 37 - jepsen.util 127 :ok :read [12 49] INFO [2022-01-10 12:53:26,923] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,923] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,924] jepsen worker 13 - jepsen.util 13 :ok :read [4 32] INFO [2022-01-10 12:53:26,924] jepsen worker 37 - jepsen.util 127 :ok :read [12 49] INFO [2022-01-10 12:53:26,924] jepsen worker 13 - jepsen.util 13 :invoke :write [4 36] INFO [2022-01-10 12:53:26,925] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:26,926] jepsen worker 37 - jepsen.util 127 :ok :read [12 49] INFO [2022-01-10 12:53:26,927] jepsen worker 61 - jepsen.util 61 :ok :write [20 47] INFO [2022-01-10 12:53:26,927] jepsen worker 37 - jepsen.util 127 :invoke :write [12 3] INFO [2022-01-10 12:53:26,927] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:26,928] jepsen worker 61 - jepsen.util 61 :ok :read [20 47] INFO [2022-01-10 12:53:26,929] jepsen worker 61 - jepsen.util 61 :invoke :write [20 32] INFO [2022-01-10 12:53:26,929] jepsen worker 15 - jepsen.util 15 :ok :write [5 46] INFO [2022-01-10 12:53:26,929] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:26,930] jepsen worker 15 - jepsen.util 15 :ok :read [5 46] INFO [2022-01-10 12:53:26,931] jepsen worker 80 - jepsen.util 260 :ok :write [26 40] INFO [2022-01-10 12:53:26,931] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:26,931] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,932] jepsen worker 15 - jepsen.util 15 :ok :read [5 46] INFO [2022-01-10 12:53:26,932] jepsen worker 80 - jepsen.util 260 :ok :read [26 40] INFO [2022-01-10 12:53:26,932] jepsen worker 15 - jepsen.util 15 :invoke :write [5 35] INFO [2022-01-10 12:53:26,933] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,934] jepsen worker 43 - jepsen.util 43 :ok :write [14 44] INFO [2022-01-10 12:53:26,934] jepsen worker 80 - jepsen.util 260 :ok :read [26 40] INFO [2022-01-10 12:53:26,934] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:26,935] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,935] jepsen worker 43 - jepsen.util 43 :ok :read [14 44] INFO [2022-01-10 12:53:26,936] jepsen worker 80 - jepsen.util 260 :ok :read [26 40] INFO [2022-01-10 12:53:26,936] jepsen worker 43 - jepsen.util 43 :invoke :write [14 28] INFO [2022-01-10 12:53:26,936] jepsen worker 48 - jepsen.util 228 :ok :write [16 48] INFO [2022-01-10 12:53:26,937] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:26,937] jepsen worker 80 - jepsen.util 260 :invoke :write [26 44] INFO [2022-01-10 12:53:26,938] jepsen worker 84 - jepsen.util 84 :ok :write [28 21] INFO [2022-01-10 12:53:26,939] jepsen worker 48 - jepsen.util 228 :ok :read [16 48] INFO [2022-01-10 12:53:26,939] jepsen worker 84 - jepsen.util 84 :invoke :write [28 17] INFO [2022-01-10 12:53:26,939] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:26,940] jepsen worker 48 - jepsen.util 228 :ok :read [16 48] INFO [2022-01-10 12:53:26,941] jepsen worker 48 - jepsen.util 228 :invoke :write [16 47] INFO [2022-01-10 12:53:26,942] jepsen worker 23 - jepsen.util 23 :ok :write [7 14] INFO [2022-01-10 12:53:26,943] jepsen worker 53 - jepsen.util 143 :ok :write [17 4] INFO [2022-01-10 12:53:26,944] jepsen worker 81 - jepsen.util 171 :ok :write [27 40] INFO [2022-01-10 12:53:26,944] jepsen worker 53 - jepsen.util 143 :invoke :write [17 11] INFO [2022-01-10 12:53:26,945] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:26,945] jepsen worker 23 - jepsen.util 23 :invoke :write [7 2] INFO [2022-01-10 12:53:26,945] jepsen worker 81 - jepsen.util 171 :ok :read [27 40] INFO [2022-01-10 12:53:26,946] jepsen worker 81 - jepsen.util 171 :invoke :write [27 8] INFO [2022-01-10 12:53:26,949] jepsen worker 4 - jepsen.util 4 :ok :write [1 20] INFO [2022-01-10 12:53:26,950] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,950] jepsen worker 8 - jepsen.util 8 :ok :write [2 40] INFO [2022-01-10 12:53:26,950] jepsen worker 4 - jepsen.util 4 :ok :read [1 20] INFO [2022-01-10 12:53:26,951] jepsen worker 8 - jepsen.util 8 :invoke :write [2 27] INFO [2022-01-10 12:53:26,951] jepsen worker 75 - jepsen.util 165 :ok :write [25 23] INFO [2022-01-10 12:53:26,951] jepsen worker 4 - jepsen.util 4 :invoke :read [1 nil] INFO [2022-01-10 12:53:26,952] jepsen worker 75 - jepsen.util 165 :invoke :write [25 10] INFO [2022-01-10 12:53:26,952] jepsen worker 4 - jepsen.util 4 :ok :read [1 20] INFO [2022-01-10 12:53:26,953] jepsen worker 4 - jepsen.util 4 :invoke :write [1 46] INFO [2022-01-10 12:53:26,955] jepsen worker 69 - jepsen.util 159 :ok :write [23 20] INFO [2022-01-10 12:53:26,955] jepsen worker 68 - jepsen.util 68 :ok :write [22 2] INFO [2022-01-10 12:53:26,956] jepsen worker 69 - jepsen.util 159 :invoke :write [23 14] INFO [2022-01-10 12:53:26,956] jepsen worker 68 - jepsen.util 68 :invoke :write [22 37] INFO [2022-01-10 12:53:26,960] jepsen worker 61 - jepsen.util 61 :ok :write [20 32] INFO [2022-01-10 12:53:26,960] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:26,961] jepsen worker 61 - jepsen.util 61 :ok :read [20 32] INFO [2022-01-10 12:53:26,962] jepsen worker 61 - jepsen.util 61 :invoke :write [20 48] INFO [2022-01-10 12:53:26,962] jepsen worker 37 - jepsen.util 127 :ok :write [12 3] INFO [2022-01-10 12:53:26,963] jepsen worker 37 - jepsen.util 127 :invoke :write [12 28] INFO [2022-01-10 12:53:26,964] jepsen worker 33 - jepsen.util 123 :ok :write [11 21] INFO [2022-01-10 12:53:26,964] jepsen worker 33 - jepsen.util 123 :invoke :write [11 25] INFO [2022-01-10 12:53:26,968] jepsen worker 13 - jepsen.util 13 :ok :write [4 36] INFO [2022-01-10 12:53:26,968] jepsen worker 88 - jepsen.util 178 :ok :write [29 47] INFO [2022-01-10 12:53:26,969] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:26,970] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:26,970] jepsen worker 88 - jepsen.util 178 :ok :read [29 47] INFO [2022-01-10 12:53:26,970] jepsen worker 13 - jepsen.util 13 :ok :read [4 36] INFO [2022-01-10 12:53:26,971] jepsen worker 88 - jepsen.util 178 :invoke :write [29 8] INFO [2022-01-10 12:53:26,971] jepsen worker 13 - jepsen.util 13 :invoke :write [4 36] INFO [2022-01-10 12:53:26,972] jepsen worker 43 - jepsen.util 43 :ok :write [14 28] INFO [2022-01-10 12:53:26,972] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:26,973] jepsen worker 43 - jepsen.util 43 :ok :read [14 28] INFO [2022-01-10 12:53:26,974] jepsen worker 2 - jepsen.util 92 :ok :write [0 8] INFO [2022-01-10 12:53:26,974] jepsen worker 43 - jepsen.util 43 :invoke :write [14 42] INFO [2022-01-10 12:53:26,975] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:26,976] jepsen worker 2 - jepsen.util 92 :ok :read [0 8] INFO [2022-01-10 12:53:26,977] jepsen worker 80 - jepsen.util 260 :ok :write [26 44] INFO [2022-01-10 12:53:26,977] jepsen worker 2 - jepsen.util 92 :invoke :write [0 36] INFO [2022-01-10 12:53:26,977] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:26,978] jepsen worker 80 - jepsen.util 260 :ok :read [26 44] INFO [2022-01-10 12:53:26,979] jepsen worker 80 - jepsen.util 260 :invoke :write [26 3] INFO [2022-01-10 12:53:26,981] jepsen worker 40 - jepsen.util 130 :ok :write [13 38] INFO [2022-01-10 12:53:26,982] jepsen worker 40 - jepsen.util 130 :invoke :write [13 14] INFO [2022-01-10 12:53:26,985] jepsen worker 23 - jepsen.util 23 :ok :write [7 2] INFO [2022-01-10 12:53:26,985] jepsen worker 84 - jepsen.util 84 :ok :write [28 17] INFO [2022-01-10 12:53:26,986] jepsen worker 84 - jepsen.util 84 :invoke :write [28 17] INFO [2022-01-10 12:53:26,986] jepsen worker 23 - jepsen.util 23 :invoke :write [7 4] INFO [2022-01-10 12:53:26,988] jepsen worker 15 - jepsen.util 15 :ok :write [5 35] INFO [2022-01-10 12:53:26,989] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:26,990] jepsen worker 53 - jepsen.util 143 :ok :write [17 11] INFO [2022-01-10 12:53:26,990] jepsen worker 15 - jepsen.util 15 :ok :read [5 35] INFO [2022-01-10 12:53:26,990] jepsen worker 53 - jepsen.util 143 :invoke :write [17 41] INFO [2022-01-10 12:53:26,991] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:26,992] jepsen worker 15 - jepsen.util 15 :ok :read [5 35] INFO [2022-01-10 12:53:26,992] jepsen worker 15 - jepsen.util 15 :invoke :write [5 18] INFO [2022-01-10 12:53:26,995] jepsen worker 48 - jepsen.util 228 :ok :write [16 47] INFO [2022-01-10 12:53:26,995] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:26,996] jepsen worker 48 - jepsen.util 228 :ok :read [16 47] INFO [2022-01-10 12:53:26,997] jepsen worker 48 - jepsen.util 228 :invoke :write [16 25] INFO [2022-01-10 12:53:27,000] jepsen worker 75 - jepsen.util 165 :ok :write [25 10] INFO [2022-01-10 12:53:27,001] jepsen worker 81 - jepsen.util 171 :ok :write [27 8] INFO [2022-01-10 12:53:27,001] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,002] jepsen worker 75 - jepsen.util 165 :ok :read [25 10] INFO [2022-01-10 12:53:27,002] jepsen worker 81 - jepsen.util 171 :invoke :write [27 34] INFO [2022-01-10 12:53:27,002] jepsen worker 75 - jepsen.util 165 :invoke :write [25 13] INFO [2022-01-10 12:53:27,005] jepsen worker 8 - jepsen.util 8 :ok :write [2 27] INFO [2022-01-10 12:53:27,006] jepsen worker 8 - jepsen.util 8 :invoke :write [2 49] INFO [2022-01-10 12:53:27,009] jepsen worker 68 - jepsen.util 68 :ok :write [22 37] INFO [2022-01-10 12:53:27,010] jepsen worker 61 - jepsen.util 61 :ok :write [20 48] INFO [2022-01-10 12:53:27,010] jepsen worker 68 - jepsen.util 68 :invoke :write [22 15] INFO [2022-01-10 12:53:27,011] jepsen worker 61 - jepsen.util 61 :invoke :write [20 14] INFO [2022-01-10 12:53:27,015] jepsen worker 33 - jepsen.util 123 :ok :write [11 25] INFO [2022-01-10 12:53:27,015] jepsen worker 33 - jepsen.util 123 :invoke :write [11 48] INFO [2022-01-10 12:53:27,017] jepsen worker 43 - jepsen.util 43 :ok :write [14 42] INFO [2022-01-10 12:53:27,018] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,019] jepsen worker 43 - jepsen.util 43 :ok :read [14 42] INFO [2022-01-10 12:53:27,019] jepsen worker 88 - jepsen.util 178 :ok :write [29 8] INFO [2022-01-10 12:53:27,019] jepsen worker 43 - jepsen.util 43 :invoke :write [14 3] INFO [2022-01-10 12:53:27,020] jepsen worker 88 - jepsen.util 178 :invoke :write [29 23] INFO [2022-01-10 12:53:27,029] jepsen worker 15 - jepsen.util 15 :ok :write [5 18] INFO [2022-01-10 12:53:27,030] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:27,030] jepsen worker 15 - jepsen.util 15 :ok :read [5 18] INFO [2022-01-10 12:53:27,031] jepsen worker 15 - jepsen.util 15 :invoke :write [5 24] INFO [2022-01-10 12:53:27,034] jepsen worker 48 - jepsen.util 228 :ok :write [16 25] INFO [2022-01-10 12:53:27,034] jepsen worker 48 - jepsen.util 228 :invoke :write [16 6] INFO [2022-01-10 12:53:27,039] jepsen worker 81 - jepsen.util 171 :ok :write [27 34] INFO [2022-01-10 12:53:27,040] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:27,041] jepsen worker 81 - jepsen.util 171 :ok :read [27 34] INFO [2022-01-10 12:53:27,041] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:27,042] jepsen worker 81 - jepsen.util 171 :ok :read [27 34] INFO [2022-01-10 12:53:27,043] jepsen worker 81 - jepsen.util 171 :invoke :write [27 25] INFO [2022-01-10 12:53:27,044] jepsen worker 8 - jepsen.util 8 :ok :write [2 49] INFO [2022-01-10 12:53:27,045] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:27,046] jepsen worker 8 - jepsen.util 8 :ok :read [2 49] INFO [2022-01-10 12:53:27,046] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:27,047] jepsen worker 8 - jepsen.util 8 :ok :read [2 49] INFO [2022-01-10 12:53:27,047] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:27,048] jepsen worker 8 - jepsen.util 8 :ok :read [2 49] INFO [2022-01-10 12:53:27,048] jepsen worker 68 - jepsen.util 68 :ok :write [22 15] INFO [2022-01-10 12:53:27,049] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:27,049] jepsen worker 8 - jepsen.util 8 :invoke :write [2 35] INFO [2022-01-10 12:53:27,049] jepsen worker 68 - jepsen.util 68 :ok :read [22 15] INFO [2022-01-10 12:53:27,050] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:27,051] jepsen worker 68 - jepsen.util 68 :ok :read [22 15] INFO [2022-01-10 12:53:27,051] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:27,052] jepsen worker 68 - jepsen.util 68 :ok :read [22 15] INFO [2022-01-10 12:53:27,052] jepsen worker 68 - jepsen.util 68 :invoke :write [22 49] INFO [2022-01-10 12:53:27,054] jepsen worker 33 - jepsen.util 123 :ok :write [11 48] INFO [2022-01-10 12:53:27,055] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:27,055] jepsen worker 33 - jepsen.util 123 :ok :read [11 48] INFO [2022-01-10 12:53:27,056] jepsen worker 33 - jepsen.util 123 :invoke :write [11 49] INFO [2022-01-10 12:53:27,062] jepsen worker 15 - jepsen.util 15 :ok :write [5 24] INFO [2022-01-10 12:53:27,063] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:27,064] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 12:53:27,064] jepsen worker 15 - jepsen.util 15 :invoke :write [5 8] INFO [2022-01-10 12:53:27,066] jepsen worker 48 - jepsen.util 228 :ok :write [16 6] INFO [2022-01-10 12:53:27,066] jepsen worker 48 - jepsen.util 228 :invoke :write [16 44] INFO [2022-01-10 12:53:27,070] jepsen worker 81 - jepsen.util 171 :ok :write [27 25] INFO [2022-01-10 12:53:27,072] jepsen worker 81 - jepsen.util 171 :invoke :write [27 47] INFO [2022-01-10 12:53:27,076] jepsen worker 8 - jepsen.util 8 :ok :write [2 35] INFO [2022-01-10 12:53:27,076] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:27,077] jepsen worker 8 - jepsen.util 8 :ok :read [2 35] INFO [2022-01-10 12:53:27,078] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:27,079] jepsen worker 8 - jepsen.util 8 :ok :read [2 35] INFO [2022-01-10 12:53:27,079] jepsen worker 8 - jepsen.util 8 :invoke :write [2 3] INFO [2022-01-10 12:53:27,080] jepsen worker 68 - jepsen.util 68 :ok :write [22 49] INFO [2022-01-10 12:53:27,081] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:27,081] jepsen worker 68 - jepsen.util 68 :ok :read [22 49] INFO [2022-01-10 12:53:27,082] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:27,083] jepsen worker 68 - jepsen.util 68 :ok :read [22 49] INFO [2022-01-10 12:53:27,083] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:27,084] jepsen worker 68 - jepsen.util 68 :ok :read [22 49] INFO [2022-01-10 12:53:27,084] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:27,085] jepsen worker 68 - jepsen.util 68 :ok :read [22 49] INFO [2022-01-10 12:53:27,086] jepsen worker 68 - jepsen.util 68 :invoke :write [22 14] INFO [2022-01-10 12:53:27,086] jepsen worker 33 - jepsen.util 123 :ok :write [11 49] INFO [2022-01-10 12:53:27,086] jepsen worker 33 - jepsen.util 123 :invoke :write [11 46] INFO [2022-01-10 12:53:27,090] jepsen worker 15 - jepsen.util 15 :ok :write [5 8] INFO [2022-01-10 12:53:27,090] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:27,091] jepsen worker 15 - jepsen.util 15 :ok :read [5 8] INFO [2022-01-10 12:53:27,091] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:27,092] jepsen worker 15 - jepsen.util 15 :ok :read [5 8] INFO [2022-01-10 12:53:27,093] jepsen worker 15 - jepsen.util 15 :invoke :write [5 31] INFO [2022-01-10 12:53:27,094] jepsen worker 48 - jepsen.util 228 :ok :write [16 44] INFO [2022-01-10 12:53:27,095] jepsen worker 48 - jepsen.util 228 :invoke :write [16 48] INFO [2022-01-10 12:53:27,099] jepsen worker 81 - jepsen.util 171 :ok :write [27 47] INFO [2022-01-10 12:53:27,099] jepsen worker 81 - jepsen.util 171 :invoke :write [27 36] INFO [2022-01-10 12:53:27,104] jepsen worker 8 - jepsen.util 8 :ok :write [2 3] INFO [2022-01-10 12:53:27,104] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:27,105] jepsen worker 8 - jepsen.util 8 :ok :read [2 3] INFO [2022-01-10 12:53:27,106] jepsen worker 8 - jepsen.util 8 :invoke :write [2 48] INFO [2022-01-10 12:53:27,109] jepsen worker 68 - jepsen.util 68 :ok :write [22 14] INFO [2022-01-10 12:53:27,110] jepsen worker 68 - jepsen.util 68 :invoke :write [22 36] INFO [2022-01-10 12:53:27,111] jepsen worker 3 - jepsen.util 3 :ok :write [1 17] INFO [2022-01-10 12:53:27,111] jepsen worker 2 - jepsen.util 92 :ok :write [0 36] INFO [2022-01-10 12:53:27,112] jepsen worker 2 - jepsen.util 92 :invoke :write [0 26] INFO [2022-01-10 12:53:27,112] jepsen worker 3 - jepsen.util 3 :invoke :write [1 33] INFO [2022-01-10 12:53:27,116] jepsen worker 40 - jepsen.util 130 :ok :write [13 14] INFO [2022-01-10 12:53:27,117] jepsen worker 69 - jepsen.util 159 :ok :write [23 14] INFO [2022-01-10 12:53:27,117] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:27,117] jepsen worker 33 - jepsen.util 123 :ok :write [11 46] INFO [2022-01-10 12:53:27,118] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,118] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:27,118] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:27,119] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:27,119] jepsen worker 69 - jepsen.util 159 :ok :read [23 14] INFO [2022-01-10 12:53:27,119] jepsen worker 33 - jepsen.util 123 :ok :read [11 46] INFO [2022-01-10 12:53:27,120] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:27,120] jepsen worker 33 - jepsen.util 123 :invoke :write [11 37] INFO [2022-01-10 12:53:27,120] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,120] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:27,121] jepsen worker 69 - jepsen.util 159 :ok :read [23 14] INFO [2022-01-10 12:53:27,121] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:27,121] jepsen worker 23 - jepsen.util 23 :ok :write [7 4] INFO [2022-01-10 12:53:27,121] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,122] jepsen worker 37 - jepsen.util 127 :ok :write [12 28] INFO [2022-01-10 12:53:27,122] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:27,122] jepsen worker 69 - jepsen.util 159 :ok :read [23 14] INFO [2022-01-10 12:53:27,123] jepsen worker 37 - jepsen.util 127 :invoke :write [12 41] INFO [2022-01-10 12:53:27,123] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:27,124] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,124] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:27,124] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,125] jepsen worker 69 - jepsen.util 159 :ok :read [23 14] INFO [2022-01-10 12:53:27,125] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:27,125] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:27,125] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:27,126] jepsen worker 69 - jepsen.util 159 :invoke :write [23 44] INFO [2022-01-10 12:53:27,126] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,127] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:27,127] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:27,127] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:27,128] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,128] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:27,129] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:27,129] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:27,129] jepsen worker 13 - jepsen.util 13 :ok :write [4 36] INFO [2022-01-10 12:53:27,129] jepsen worker 75 - jepsen.util 165 :ok :write [25 13] INFO [2022-01-10 12:53:27,129] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:27,129] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,130] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:27,130] jepsen worker 13 - jepsen.util 13 :ok :read [4 36] INFO [2022-01-10 12:53:27,130] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,131] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:27,131] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,131] jepsen worker 75 - jepsen.util 165 :ok :read [25 13] INFO [2022-01-10 12:53:27,131] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:27,132] jepsen worker 23 - jepsen.util 23 :ok :read [7 4] INFO [2022-01-10 12:53:27,132] jepsen worker 75 - jepsen.util 165 :invoke :write [25 16] INFO [2022-01-10 12:53:27,132] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:27,132] jepsen worker 23 - jepsen.util 23 :invoke :write [7 6] INFO [2022-01-10 12:53:27,133] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,133] jepsen worker 40 - jepsen.util 130 :invoke :write [13 15] INFO [2022-01-10 12:53:27,134] jepsen worker 13 - jepsen.util 13 :ok :read [4 36] INFO [2022-01-10 12:53:27,134] jepsen worker 61 - jepsen.util 61 :ok :write [20 14] INFO [2022-01-10 12:53:27,135] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,135] jepsen worker 80 - jepsen.util 260 :ok :write [26 3] INFO [2022-01-10 12:53:27,135] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,135] jepsen worker 13 - jepsen.util 13 :ok :read [4 36] INFO [2022-01-10 12:53:27,136] jepsen worker 61 - jepsen.util 61 :ok :read [20 14] INFO [2022-01-10 12:53:27,136] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,137] jepsen worker 61 - jepsen.util 61 :invoke :write [20 31] INFO [2022-01-10 12:53:27,137] jepsen worker 80 - jepsen.util 260 :ok :read [26 3] INFO [2022-01-10 12:53:27,138] jepsen worker 13 - jepsen.util 13 :invoke :write [4 40] INFO [2022-01-10 12:53:27,138] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,139] jepsen worker 88 - jepsen.util 178 :ok :write [29 23] INFO [2022-01-10 12:53:27,139] jepsen worker 80 - jepsen.util 260 :ok :read [26 3] INFO [2022-01-10 12:53:27,139] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:27,140] jepsen worker 88 - jepsen.util 178 :ok :read [29 23] INFO [2022-01-10 12:53:27,140] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,141] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:27,141] jepsen worker 80 - jepsen.util 260 :ok :read [26 3] INFO [2022-01-10 12:53:27,142] jepsen worker 88 - jepsen.util 178 :ok :read [29 23] INFO [2022-01-10 12:53:27,142] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,142] jepsen worker 88 - jepsen.util 178 :invoke :write [29 39] INFO [2022-01-10 12:53:27,143] jepsen worker 84 - jepsen.util 84 :ok :write [28 17] INFO [2022-01-10 12:53:27,143] jepsen worker 80 - jepsen.util 260 :ok :read [26 3] INFO [2022-01-10 12:53:27,143] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:27,144] jepsen worker 80 - jepsen.util 260 :invoke :write [26 41] INFO [2022-01-10 12:53:27,144] jepsen worker 84 - jepsen.util 84 :ok :read [28 17] INFO [2022-01-10 12:53:27,145] jepsen worker 84 - jepsen.util 84 :invoke :write [28 42] INFO [2022-01-10 12:53:27,147] jepsen worker 43 - jepsen.util 43 :ok :write [14 3] INFO [2022-01-10 12:53:27,147] jepsen worker 53 - jepsen.util 143 :ok :write [17 41] INFO [2022-01-10 12:53:27,147] jepsen worker 43 - jepsen.util 43 :invoke :write [14 28] INFO [2022-01-10 12:53:27,148] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,149] jepsen worker 53 - jepsen.util 143 :ok :read [17 41] INFO [2022-01-10 12:53:27,149] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,150] jepsen worker 53 - jepsen.util 143 :ok :read [17 41] INFO [2022-01-10 12:53:27,150] jepsen worker 53 - jepsen.util 143 :invoke :write [17 42] INFO [2022-01-10 12:53:27,152] jepsen worker 2 - jepsen.util 92 :ok :write [0 26] INFO [2022-01-10 12:53:27,153] jepsen worker 2 - jepsen.util 92 :invoke :write [0 34] INFO [2022-01-10 12:53:27,153] jepsen worker 3 - jepsen.util 3 :ok :write [1 33] INFO [2022-01-10 12:53:27,154] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:27,154] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:27,155] jepsen worker 3 - jepsen.util 3 :invoke :write [1 21] INFO [2022-01-10 12:53:27,158] jepsen worker 40 - jepsen.util 130 :ok :write [13 15] INFO [2022-01-10 12:53:27,159] jepsen worker 40 - jepsen.util 130 :invoke :write [13 0] INFO [2022-01-10 12:53:27,165] jepsen worker 37 - jepsen.util 127 :ok :write [12 41] INFO [2022-01-10 12:53:27,166] jepsen worker 37 - jepsen.util 127 :invoke :write [12 34] INFO [2022-01-10 12:53:27,178] jepsen worker 88 - jepsen.util 178 :ok :write [29 39] INFO [2022-01-10 12:53:27,178] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:27,179] jepsen worker 88 - jepsen.util 178 :ok :read [29 39] INFO [2022-01-10 12:53:27,179] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:27,180] jepsen worker 88 - jepsen.util 178 :ok :read [29 39] INFO [2022-01-10 12:53:27,180] jepsen worker 88 - jepsen.util 178 :invoke :write [29 24] INFO [2022-01-10 12:53:27,182] jepsen worker 84 - jepsen.util 84 :ok :write [28 42] INFO [2022-01-10 12:53:27,183] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:27,184] jepsen worker 84 - jepsen.util 84 :ok :read [28 42] INFO [2022-01-10 12:53:27,184] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:27,185] jepsen worker 84 - jepsen.util 84 :ok :read [28 42] INFO [2022-01-10 12:53:27,186] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:27,187] jepsen worker 84 - jepsen.util 84 :ok :read [28 42] INFO [2022-01-10 12:53:27,187] jepsen worker 84 - jepsen.util 84 :invoke :write [28 8] INFO [2022-01-10 12:53:27,188] jepsen worker 2 - jepsen.util 92 :ok :write [0 34] INFO [2022-01-10 12:53:27,188] jepsen worker 2 - jepsen.util 92 :invoke :write [0 0] INFO [2022-01-10 12:53:27,193] jepsen worker 15 - jepsen.util 15 :ok :write [5 31] INFO [2022-01-10 12:53:27,194] jepsen worker 15 - jepsen.util 15 :invoke :write [5 4] INFO [2022-01-10 12:53:27,194] jepsen worker 3 - jepsen.util 3 :ok :write [1 21] INFO [2022-01-10 12:53:27,194] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:27,195] jepsen worker 3 - jepsen.util 3 :ok :read [1 21] INFO [2022-01-10 12:53:27,196] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:27,197] jepsen worker 3 - jepsen.util 3 :ok :read [1 21] INFO [2022-01-10 12:53:27,197] jepsen worker 3 - jepsen.util 3 :invoke :write [1 12] INFO [2022-01-10 12:53:27,199] jepsen worker 69 - jepsen.util 159 :ok :write [23 44] INFO [2022-01-10 12:53:27,199] jepsen worker 48 - jepsen.util 228 :ok :write [16 48] INFO [2022-01-10 12:53:27,199] jepsen worker 69 - jepsen.util 159 :invoke :write [23 14] INFO [2022-01-10 12:53:27,200] jepsen worker 48 - jepsen.util 228 :invoke :write [16 13] INFO [2022-01-10 12:53:27,201] jepsen worker 37 - jepsen.util 127 :ok :write [12 34] INFO [2022-01-10 12:53:27,201] jepsen worker 37 - jepsen.util 127 :invoke :write [12 0] INFO [2022-01-10 12:53:27,204] jepsen worker 13 - jepsen.util 13 :ok :write [4 40] INFO [2022-01-10 12:53:27,204] jepsen worker 13 - jepsen.util 13 :invoke :write [4 40] INFO [2022-01-10 12:53:27,204] jepsen worker 81 - jepsen.util 171 :ok :write [27 36] INFO [2022-01-10 12:53:27,205] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:27,206] jepsen worker 81 - jepsen.util 171 :ok :read [27 36] INFO [2022-01-10 12:53:27,206] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:27,207] jepsen worker 80 - jepsen.util 260 :ok :write [26 41] INFO [2022-01-10 12:53:27,207] jepsen worker 81 - jepsen.util 171 :ok :read [27 36] INFO [2022-01-10 12:53:27,207] jepsen worker 80 - jepsen.util 260 :invoke :write [26 29] INFO [2022-01-10 12:53:27,208] jepsen worker 81 - jepsen.util 171 :invoke :write [27 3] INFO [2022-01-10 12:53:27,210] jepsen worker 8 - jepsen.util 8 :ok :write [2 48] INFO [2022-01-10 12:53:27,210] jepsen worker 8 - jepsen.util 8 :invoke :write [2 22] INFO [2022-01-10 12:53:27,211] jepsen worker 53 - jepsen.util 143 :ok :write [17 42] INFO [2022-01-10 12:53:27,211] jepsen worker 53 - jepsen.util 143 :invoke :write [17 46] INFO [2022-01-10 12:53:27,215] jepsen worker 69 - jepsen.util 159 :ok :write [23 14] INFO [2022-01-10 12:53:27,216] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,216] jepsen worker 69 - jepsen.util 159 :ok :read [23 14] INFO [2022-01-10 12:53:27,217] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,218] jepsen worker 68 - jepsen.util 68 :ok :write [22 36] INFO [2022-01-10 12:53:27,218] jepsen worker 69 - jepsen.util 159 :ok :read [23 14] INFO [2022-01-10 12:53:27,218] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:27,218] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,219] jepsen worker 68 - jepsen.util 68 :ok :read [22 36] INFO [2022-01-10 12:53:27,219] jepsen worker 69 - jepsen.util 159 :ok :read [23 14] INFO [2022-01-10 12:53:27,219] jepsen worker 68 - jepsen.util 68 :invoke :write [22 12] INFO [2022-01-10 12:53:27,220] jepsen worker 69 - jepsen.util 159 :invoke :write [23 44] INFO [2022-01-10 12:53:27,222] jepsen worker 33 - jepsen.util 123 :ok :write [11 37] INFO [2022-01-10 12:53:27,223] jepsen worker 13 - jepsen.util 13 :ok :write [4 40] INFO [2022-01-10 12:53:27,223] jepsen worker 33 - jepsen.util 123 :invoke :write [11 48] INFO [2022-01-10 12:53:27,223] jepsen worker 13 - jepsen.util 13 :invoke :write [4 27] INFO [2022-01-10 12:53:27,227] jepsen worker 23 - jepsen.util 23 :ok :write [7 6] INFO [2022-01-10 12:53:27,227] jepsen worker 80 - jepsen.util 260 :ok :write [26 29] INFO [2022-01-10 12:53:27,228] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,228] jepsen worker 23 - jepsen.util 23 :invoke :write [7 28] INFO [2022-01-10 12:53:27,229] jepsen worker 80 - jepsen.util 260 :ok :read [26 29] INFO [2022-01-10 12:53:27,229] jepsen worker 80 - jepsen.util 260 :invoke :write [26 15] INFO [2022-01-10 12:53:27,233] jepsen worker 53 - jepsen.util 143 :ok :write [17 46] INFO [2022-01-10 12:53:27,234] jepsen worker 53 - jepsen.util 143 :invoke :write [17 19] INFO [2022-01-10 12:53:27,235] jepsen worker 75 - jepsen.util 165 :ok :write [25 16] INFO [2022-01-10 12:53:27,236] jepsen worker 75 - jepsen.util 165 :invoke :write [25 5] INFO [2022-01-10 12:53:27,237] jepsen worker 69 - jepsen.util 159 :ok :write [23 44] INFO [2022-01-10 12:53:27,237] jepsen worker 69 - jepsen.util 159 :invoke :write [23 28] INFO [2022-01-10 12:53:27,241] jepsen worker 13 - jepsen.util 13 :ok :write [4 27] INFO [2022-01-10 12:53:27,242] jepsen worker 13 - jepsen.util 13 :invoke :write [4 30] INFO [2022-01-10 12:53:27,244] jepsen worker 61 - jepsen.util 61 :ok :write [20 31] INFO [2022-01-10 12:53:27,244] jepsen worker 61 - jepsen.util 61 :invoke :write [20 31] INFO [2022-01-10 12:53:27,247] jepsen worker 80 - jepsen.util 260 :ok :write [26 15] INFO [2022-01-10 12:53:27,248] jepsen worker 80 - jepsen.util 260 :invoke :write [26 49] INFO [2022-01-10 12:53:27,249] jepsen worker 43 - jepsen.util 43 :ok :write [14 28] INFO [2022-01-10 12:53:27,250] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,251] jepsen worker 43 - jepsen.util 43 :ok :read [14 28] INFO [2022-01-10 12:53:27,251] jepsen worker 43 - jepsen.util 43 :invoke :write [14 26] INFO [2022-01-10 12:53:27,253] jepsen worker 53 - jepsen.util 143 :ok :write [17 19] INFO [2022-01-10 12:53:27,254] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,255] jepsen worker 53 - jepsen.util 143 :ok :read [17 19] INFO [2022-01-10 12:53:27,255] jepsen worker 53 - jepsen.util 143 :invoke :write [17 42] INFO [2022-01-10 12:53:27,260] jepsen worker 69 - jepsen.util 159 :ok :write [23 28] INFO [2022-01-10 12:53:27,261] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,262] jepsen worker 69 - jepsen.util 159 :ok :read [23 28] INFO [2022-01-10 12:53:27,263] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,263] jepsen worker 13 - jepsen.util 13 :ok :write [4 30] INFO [2022-01-10 12:53:27,263] jepsen worker 69 - jepsen.util 159 :ok :read [23 28] INFO [2022-01-10 12:53:27,264] jepsen worker 13 - jepsen.util 13 :invoke :write [4 26] INFO [2022-01-10 12:53:27,264] jepsen worker 69 - jepsen.util 159 :invoke :write [23 19] INFO [2022-01-10 12:53:27,267] jepsen worker 80 - jepsen.util 260 :ok :write [26 49] INFO [2022-01-10 12:53:27,268] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,269] jepsen worker 80 - jepsen.util 260 :ok :read [26 49] INFO [2022-01-10 12:53:27,269] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,270] jepsen worker 80 - jepsen.util 260 :ok :read [26 49] INFO [2022-01-10 12:53:27,271] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,272] jepsen worker 80 - jepsen.util 260 :ok :read [26 49] INFO [2022-01-10 12:53:27,272] jepsen worker 53 - jepsen.util 143 :ok :write [17 42] INFO [2022-01-10 12:53:27,272] jepsen worker 80 - jepsen.util 260 :invoke :write [26 7] INFO [2022-01-10 12:53:27,273] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,274] jepsen worker 53 - jepsen.util 143 :ok :read [17 42] INFO [2022-01-10 12:53:27,275] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,275] jepsen worker 69 - jepsen.util 159 :ok :write [23 19] INFO [2022-01-10 12:53:27,276] jepsen worker 53 - jepsen.util 143 :ok :read [17 42] INFO [2022-01-10 12:53:27,276] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,277] jepsen worker 69 - jepsen.util 159 :ok :read [23 19] INFO [2022-01-10 12:53:27,277] jepsen worker 53 - jepsen.util 143 :invoke :write [17 36] INFO [2022-01-10 12:53:27,277] jepsen worker 69 - jepsen.util 159 :invoke :write [23 9] INFO [2022-01-10 12:53:27,282] jepsen worker 13 - jepsen.util 13 :ok :write [4 26] INFO [2022-01-10 12:53:27,282] jepsen worker 13 - jepsen.util 13 :invoke :write [4 35] INFO [2022-01-10 12:53:27,286] jepsen worker 80 - jepsen.util 260 :ok :write [26 7] INFO [2022-01-10 12:53:27,286] jepsen worker 80 - jepsen.util 260 :invoke :write [26 45] INFO [2022-01-10 12:53:27,290] jepsen worker 23 - jepsen.util 23 :ok :write [7 28] INFO [2022-01-10 12:53:27,290] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,291] jepsen worker 53 - jepsen.util 143 :ok :write [17 36] INFO [2022-01-10 12:53:27,291] jepsen worker 23 - jepsen.util 23 :ok :read [7 28] INFO [2022-01-10 12:53:27,292] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,292] jepsen worker 23 - jepsen.util 23 :invoke :write [7 42] INFO [2022-01-10 12:53:27,293] jepsen worker 53 - jepsen.util 143 :ok :read [17 36] INFO [2022-01-10 12:53:27,293] jepsen worker 53 - jepsen.util 143 :invoke :write [17 2] INFO [2022-01-10 12:53:27,295] jepsen worker 75 - jepsen.util 165 :ok :write [25 5] INFO [2022-01-10 12:53:27,295] jepsen worker 69 - jepsen.util 159 :ok :write [23 9] INFO [2022-01-10 12:53:27,296] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,297] jepsen worker 69 - jepsen.util 159 :invoke :write [23 29] INFO [2022-01-10 12:53:27,297] jepsen worker 75 - jepsen.util 165 :ok :read [25 5] INFO [2022-01-10 12:53:27,297] jepsen worker 75 - jepsen.util 165 :invoke :write [25 5] INFO [2022-01-10 12:53:27,300] jepsen worker 13 - jepsen.util 13 :ok :write [4 35] INFO [2022-01-10 12:53:27,301] jepsen worker 61 - jepsen.util 61 :ok :write [20 31] INFO [2022-01-10 12:53:27,301] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,301] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,302] jepsen worker 13 - jepsen.util 13 :ok :read [4 35] INFO [2022-01-10 12:53:27,302] jepsen worker 61 - jepsen.util 61 :ok :read [20 31] INFO [2022-01-10 12:53:27,302] jepsen worker 13 - jepsen.util 13 :invoke :write [4 4] INFO [2022-01-10 12:53:27,303] jepsen worker 61 - jepsen.util 61 :invoke :write [20 38] INFO [2022-01-10 12:53:27,304] jepsen worker 80 - jepsen.util 260 :ok :write [26 45] INFO [2022-01-10 12:53:27,304] jepsen worker 80 - jepsen.util 260 :invoke :write [26 35] INFO [2022-01-10 12:53:27,308] jepsen worker 43 - jepsen.util 43 :ok :write [14 26] INFO [2022-01-10 12:53:27,309] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,310] jepsen worker 43 - jepsen.util 43 :ok :read [14 26] INFO [2022-01-10 12:53:27,310] jepsen worker 53 - jepsen.util 143 :ok :write [17 2] INFO [2022-01-10 12:53:27,310] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,310] jepsen worker 53 - jepsen.util 143 :invoke :write [17 29] INFO [2022-01-10 12:53:27,311] jepsen worker 43 - jepsen.util 43 :ok :read [14 26] INFO [2022-01-10 12:53:27,313] jepsen worker 43 - jepsen.util 43 :invoke :write [14 21] INFO [2022-01-10 12:53:27,315] jepsen worker 69 - jepsen.util 159 :ok :write [23 29] INFO [2022-01-10 12:53:27,316] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,316] jepsen worker 23 - jepsen.util 23 :ok :write [7 42] INFO [2022-01-10 12:53:27,317] jepsen worker 69 - jepsen.util 159 :ok :read [23 29] INFO [2022-01-10 12:53:27,317] jepsen worker 23 - jepsen.util 23 :invoke :write [7 12] INFO [2022-01-10 12:53:27,318] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,319] jepsen worker 69 - jepsen.util 159 :ok :read [23 29] INFO [2022-01-10 12:53:27,319] jepsen worker 69 - jepsen.util 159 :invoke :write [23 28] INFO [2022-01-10 12:53:27,321] jepsen worker 13 - jepsen.util 13 :ok :write [4 4] INFO [2022-01-10 12:53:27,322] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,322] jepsen worker 75 - jepsen.util 165 :ok :write [25 5] INFO [2022-01-10 12:53:27,322] jepsen worker 13 - jepsen.util 13 :ok :read [4 4] INFO [2022-01-10 12:53:27,323] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,324] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,324] jepsen worker 13 - jepsen.util 13 :ok :read [4 4] INFO [2022-01-10 12:53:27,324] jepsen worker 75 - jepsen.util 165 :ok :read [25 5] INFO [2022-01-10 12:53:27,325] jepsen worker 13 - jepsen.util 13 :invoke :write [4 6] INFO [2022-01-10 12:53:27,325] jepsen worker 75 - jepsen.util 165 :invoke :write [25 12] INFO [2022-01-10 12:53:27,328] jepsen worker 61 - jepsen.util 61 :ok :write [20 38] INFO [2022-01-10 12:53:27,329] jepsen worker 61 - jepsen.util 61 :invoke :write [20 22] INFO [2022-01-10 12:53:27,392] jepsen worker 80 - jepsen.util 260 :ok :write [26 35] INFO [2022-01-10 12:53:27,393] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,394] jepsen worker 80 - jepsen.util 260 :ok :read [26 35] INFO [2022-01-10 12:53:27,394] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,395] jepsen worker 80 - jepsen.util 260 :ok :read [26 35] INFO [2022-01-10 12:53:27,396] jepsen worker 80 - jepsen.util 260 :invoke :write [26 1] INFO [2022-01-10 12:53:27,398] jepsen worker 53 - jepsen.util 143 :ok :write [17 29] INFO [2022-01-10 12:53:27,399] jepsen worker 53 - jepsen.util 143 :invoke :write [17 1] INFO [2022-01-10 12:53:27,400] jepsen worker 43 - jepsen.util 43 :ok :write [14 21] INFO [2022-01-10 12:53:27,400] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,401] jepsen worker 43 - jepsen.util 43 :ok :read [14 21] INFO [2022-01-10 12:53:27,402] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,403] jepsen worker 43 - jepsen.util 43 :ok :read [14 21] INFO [2022-01-10 12:53:27,403] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,404] jepsen worker 43 - jepsen.util 43 :ok :read [14 21] INFO [2022-01-10 12:53:27,404] jepsen worker 69 - jepsen.util 159 :ok :write [23 28] INFO [2022-01-10 12:53:27,405] jepsen worker 23 - jepsen.util 23 :ok :write [7 12] INFO [2022-01-10 12:53:27,405] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,406] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,406] jepsen worker 43 - jepsen.util 43 :ok :read [14 21] INFO [2022-01-10 12:53:27,407] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,407] jepsen worker 23 - jepsen.util 23 :ok :read [7 12] INFO [2022-01-10 12:53:27,407] jepsen worker 69 - jepsen.util 159 :invoke :write [23 4] INFO [2022-01-10 12:53:27,408] jepsen worker 43 - jepsen.util 43 :ok :read [14 21] INFO [2022-01-10 12:53:27,408] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,409] jepsen worker 23 - jepsen.util 23 :ok :read [7 12] INFO [2022-01-10 12:53:27,409] jepsen worker 43 - jepsen.util 43 :invoke :write [14 25] INFO [2022-01-10 12:53:27,410] jepsen worker 23 - jepsen.util 23 :invoke :write [7 47] INFO [2022-01-10 12:53:27,410] jepsen worker 13 - jepsen.util 13 :ok :write [4 6] INFO [2022-01-10 12:53:27,411] jepsen worker 13 - jepsen.util 13 :invoke :write [4 43] INFO [2022-01-10 12:53:27,412] jepsen worker 75 - jepsen.util 165 :ok :write [25 12] INFO [2022-01-10 12:53:27,413] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,413] jepsen worker 75 - jepsen.util 165 :ok :read [25 12] INFO [2022-01-10 12:53:27,414] jepsen worker 75 - jepsen.util 165 :invoke :write [25 23] INFO [2022-01-10 12:53:27,416] jepsen worker 80 - jepsen.util 260 :ok :write [26 1] INFO [2022-01-10 12:53:27,417] jepsen worker 80 - jepsen.util 260 :invoke :write [26 34] INFO [2022-01-10 12:53:27,418] jepsen worker 61 - jepsen.util 61 :ok :write [20 22] INFO [2022-01-10 12:53:27,418] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,419] jepsen worker 61 - jepsen.util 61 :ok :read [20 22] INFO [2022-01-10 12:53:27,420] jepsen worker 61 - jepsen.util 61 :invoke :write [20 18] INFO [2022-01-10 12:53:27,422] jepsen worker 53 - jepsen.util 143 :ok :write [17 1] INFO [2022-01-10 12:53:27,422] jepsen worker 43 - jepsen.util 43 :ok :write [14 25] INFO [2022-01-10 12:53:27,423] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,423] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,423] jepsen worker 53 - jepsen.util 143 :ok :read [17 1] INFO [2022-01-10 12:53:27,424] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,424] jepsen worker 43 - jepsen.util 43 :ok :read [14 25] INFO [2022-01-10 12:53:27,425] jepsen worker 43 - jepsen.util 43 :invoke :write [14 40] INFO [2022-01-10 12:53:27,425] jepsen worker 53 - jepsen.util 143 :ok :read [17 1] INFO [2022-01-10 12:53:27,426] jepsen worker 53 - jepsen.util 143 :invoke :write [17 28] INFO [2022-01-10 12:53:27,427] jepsen worker 23 - jepsen.util 23 :ok :write [7 47] INFO [2022-01-10 12:53:27,427] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,428] jepsen worker 69 - jepsen.util 159 :ok :write [23 4] INFO [2022-01-10 12:53:27,428] jepsen worker 23 - jepsen.util 23 :ok :read [7 47] INFO [2022-01-10 12:53:27,429] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,429] jepsen worker 23 - jepsen.util 23 :invoke :write [7 14] INFO [2022-01-10 12:53:27,430] jepsen worker 69 - jepsen.util 159 :ok :read [23 4] INFO [2022-01-10 12:53:27,430] jepsen worker 69 - jepsen.util 159 :invoke :write [23 24] INFO [2022-01-10 12:53:27,433] jepsen worker 13 - jepsen.util 13 :ok :write [4 43] INFO [2022-01-10 12:53:27,433] jepsen worker 75 - jepsen.util 165 :ok :write [25 23] INFO [2022-01-10 12:53:27,434] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,434] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,434] jepsen worker 75 - jepsen.util 165 :ok :read [25 23] INFO [2022-01-10 12:53:27,435] jepsen worker 13 - jepsen.util 13 :ok :read [4 43] INFO [2022-01-10 12:53:27,435] jepsen worker 75 - jepsen.util 165 :invoke :write [25 17] INFO [2022-01-10 12:53:27,436] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,437] jepsen worker 13 - jepsen.util 13 :ok :read [4 43] INFO [2022-01-10 12:53:27,437] jepsen worker 13 - jepsen.util 13 :invoke :write [4 14] INFO [2022-01-10 12:53:27,438] jepsen worker 61 - jepsen.util 61 :ok :write [20 18] INFO [2022-01-10 12:53:27,438] jepsen worker 80 - jepsen.util 260 :ok :write [26 34] INFO [2022-01-10 12:53:27,439] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,439] jepsen worker 80 - jepsen.util 260 :invoke :write [26 25] INFO [2022-01-10 12:53:27,439] jepsen worker 61 - jepsen.util 61 :ok :read [20 18] INFO [2022-01-10 12:53:27,440] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,441] jepsen worker 61 - jepsen.util 61 :ok :read [20 18] INFO [2022-01-10 12:53:27,442] jepsen worker 61 - jepsen.util 61 :invoke :write [20 30] INFO [2022-01-10 12:53:27,443] jepsen worker 43 - jepsen.util 43 :ok :write [14 40] INFO [2022-01-10 12:53:27,445] jepsen worker 43 - jepsen.util 43 :invoke :write [14 47] INFO [2022-01-10 12:53:27,446] jepsen worker 53 - jepsen.util 143 :ok :write [17 28] INFO [2022-01-10 12:53:27,447] jepsen worker 53 - jepsen.util 143 :invoke :write [17 13] INFO [2022-01-10 12:53:27,448] jepsen worker 23 - jepsen.util 23 :ok :write [7 14] INFO [2022-01-10 12:53:27,448] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,449] jepsen worker 23 - jepsen.util 23 :ok :read [7 14] INFO [2022-01-10 12:53:27,450] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,451] jepsen worker 23 - jepsen.util 23 :ok :read [7 14] INFO [2022-01-10 12:53:27,451] jepsen worker 40 - jepsen.util 130 :ok :write [13 0] INFO [2022-01-10 12:53:27,451] jepsen worker 23 - jepsen.util 23 :invoke :write [7 17] INFO [2022-01-10 12:53:27,452] jepsen worker 40 - jepsen.util 130 :invoke :write [13 39] INFO [2022-01-10 12:53:27,552] jepsen worker 1 - jepsen.util 1 :info :write [0 40] :RequestTimeoutException INFO [2022-01-10 12:53:27,553] jepsen worker 1 - jepsen.util 91 :invoke :write [0 0] INFO [2022-01-10 12:53:27,565] jepsen worker 69 - jepsen.util 159 :ok :write [23 24] INFO [2022-01-10 12:53:27,566] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,567] jepsen worker 75 - jepsen.util 165 :ok :write [25 17] INFO [2022-01-10 12:53:27,567] jepsen worker 69 - jepsen.util 159 :ok :read [23 24] INFO [2022-01-10 12:53:27,567] jepsen worker 75 - jepsen.util 165 :invoke :write [25 12] INFO [2022-01-10 12:53:27,567] jepsen worker 69 - jepsen.util 159 :invoke :write [23 33] INFO [2022-01-10 12:53:27,571] jepsen worker 13 - jepsen.util 13 :ok :write [4 14] INFO [2022-01-10 12:53:27,572] jepsen worker 13 - jepsen.util 13 :invoke :write [4 2] INFO [2022-01-10 12:53:27,574] jepsen worker 61 - jepsen.util 61 :ok :write [20 30] INFO [2022-01-10 12:53:27,575] jepsen worker 61 - jepsen.util 61 :invoke :write [20 32] INFO [2022-01-10 12:53:27,576] jepsen worker 80 - jepsen.util 260 :ok :write [26 25] INFO [2022-01-10 12:53:27,577] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,578] jepsen worker 80 - jepsen.util 260 :ok :read [26 25] INFO [2022-01-10 12:53:27,579] jepsen worker 80 - jepsen.util 260 :invoke :write [26 2] INFO [2022-01-10 12:53:27,580] jepsen worker 43 - jepsen.util 43 :ok :write [14 47] INFO [2022-01-10 12:53:27,581] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,582] jepsen worker 53 - jepsen.util 143 :ok :write [17 13] INFO [2022-01-10 12:53:27,582] jepsen worker 43 - jepsen.util 43 :ok :read [14 47] INFO [2022-01-10 12:53:27,583] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,583] jepsen worker 43 - jepsen.util 43 :invoke :write [14 33] INFO [2022-01-10 12:53:27,583] jepsen worker 53 - jepsen.util 143 :ok :read [17 13] INFO [2022-01-10 12:53:27,584] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,585] jepsen worker 53 - jepsen.util 143 :ok :read [17 13] INFO [2022-01-10 12:53:27,586] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,586] jepsen worker 53 - jepsen.util 143 :ok :read [17 13] INFO [2022-01-10 12:53:27,587] jepsen worker 23 - jepsen.util 23 :ok :write [7 17] INFO [2022-01-10 12:53:27,587] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,588] jepsen worker 23 - jepsen.util 23 :invoke :write [7 37] INFO [2022-01-10 12:53:27,588] jepsen worker 53 - jepsen.util 143 :ok :read [17 13] INFO [2022-01-10 12:53:27,589] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,590] jepsen worker 69 - jepsen.util 159 :ok :write [23 33] INFO [2022-01-10 12:53:27,590] jepsen worker 53 - jepsen.util 143 :ok :read [17 13] INFO [2022-01-10 12:53:27,590] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,591] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,592] jepsen worker 53 - jepsen.util 143 :ok :read [17 13] INFO [2022-01-10 12:53:27,592] jepsen worker 69 - jepsen.util 159 :ok :read [23 33] INFO [2022-01-10 12:53:27,592] jepsen worker 53 - jepsen.util 143 :invoke :write [17 15] INFO [2022-01-10 12:53:27,593] jepsen worker 75 - jepsen.util 165 :ok :write [25 12] INFO [2022-01-10 12:53:27,593] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,594] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,595] jepsen worker 75 - jepsen.util 165 :ok :read [25 12] INFO [2022-01-10 12:53:27,595] jepsen worker 75 - jepsen.util 165 :invoke :write [25 6] INFO [2022-01-10 12:53:27,596] jepsen worker 69 - jepsen.util 159 :ok :read [23 33] INFO [2022-01-10 12:53:27,596] jepsen worker 69 - jepsen.util 159 :invoke :write [23 14] INFO [2022-01-10 12:53:27,596] jepsen worker 13 - jepsen.util 13 :ok :write [4 2] INFO [2022-01-10 12:53:27,597] jepsen worker 13 - jepsen.util 13 :invoke :write [4 25] INFO [2022-01-10 12:53:27,600] jepsen worker 61 - jepsen.util 61 :ok :write [20 32] INFO [2022-01-10 12:53:27,600] jepsen worker 80 - jepsen.util 260 :ok :write [26 2] INFO [2022-01-10 12:53:27,601] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,602] jepsen worker 80 - jepsen.util 260 :invoke :write [26 48] INFO [2022-01-10 12:53:27,602] jepsen worker 61 - jepsen.util 61 :ok :read [20 32] INFO [2022-01-10 12:53:27,603] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,603] jepsen worker 61 - jepsen.util 61 :ok :read [20 32] INFO [2022-01-10 12:53:27,604] jepsen worker 61 - jepsen.util 61 :invoke :write [20 48] INFO [2022-01-10 12:53:27,606] jepsen worker 43 - jepsen.util 43 :ok :write [14 33] INFO [2022-01-10 12:53:27,607] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,608] jepsen worker 43 - jepsen.util 43 :ok :read [14 33] INFO [2022-01-10 12:53:27,608] jepsen worker 43 - jepsen.util 43 :invoke :write [14 30] INFO [2022-01-10 12:53:27,610] jepsen worker 53 - jepsen.util 143 :ok :write [17 15] INFO [2022-01-10 12:53:27,610] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,611] jepsen worker 53 - jepsen.util 143 :ok :read [17 15] INFO [2022-01-10 12:53:27,612] jepsen worker 53 - jepsen.util 143 :invoke :write [17 13] INFO [2022-01-10 12:53:27,612] jepsen worker 23 - jepsen.util 23 :ok :write [7 37] INFO [2022-01-10 12:53:27,613] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,614] jepsen worker 23 - jepsen.util 23 :ok :read [7 37] INFO [2022-01-10 12:53:27,614] jepsen worker 69 - jepsen.util 159 :ok :write [23 14] INFO [2022-01-10 12:53:27,614] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,614] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,615] jepsen worker 23 - jepsen.util 23 :ok :read [7 37] INFO [2022-01-10 12:53:27,615] jepsen worker 69 - jepsen.util 159 :ok :read [23 14] INFO [2022-01-10 12:53:27,615] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,616] jepsen worker 69 - jepsen.util 159 :invoke :write [23 7] INFO [2022-01-10 12:53:27,616] jepsen worker 23 - jepsen.util 23 :ok :read [7 37] INFO [2022-01-10 12:53:27,617] jepsen worker 23 - jepsen.util 23 :invoke :write [7 36] INFO [2022-01-10 12:53:27,618] jepsen worker 13 - jepsen.util 13 :ok :write [4 25] INFO [2022-01-10 12:53:27,619] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,619] jepsen worker 13 - jepsen.util 13 :ok :read [4 25] INFO [2022-01-10 12:53:27,620] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,620] jepsen worker 75 - jepsen.util 165 :ok :write [25 6] INFO [2022-01-10 12:53:27,620] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,620] jepsen worker 13 - jepsen.util 13 :ok :read [4 25] INFO [2022-01-10 12:53:27,621] jepsen worker 13 - jepsen.util 13 :invoke :write [4 5] INFO [2022-01-10 12:53:27,621] jepsen worker 75 - jepsen.util 165 :ok :read [25 6] INFO [2022-01-10 12:53:27,622] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,622] jepsen worker 75 - jepsen.util 165 :ok :read [25 6] INFO [2022-01-10 12:53:27,623] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,624] jepsen worker 75 - jepsen.util 165 :ok :read [25 6] INFO [2022-01-10 12:53:27,624] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,625] jepsen worker 75 - jepsen.util 165 :ok :read [25 6] INFO [2022-01-10 12:53:27,625] jepsen worker 75 - jepsen.util 165 :invoke :write [25 13] INFO [2022-01-10 12:53:27,625] jepsen worker 80 - jepsen.util 260 :ok :write [26 48] INFO [2022-01-10 12:53:27,626] jepsen worker 80 - jepsen.util 260 :invoke :write [26 6] INFO [2022-01-10 12:53:27,626] jepsen worker 61 - jepsen.util 61 :ok :write [20 48] INFO [2022-01-10 12:53:27,627] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,628] jepsen worker 61 - jepsen.util 61 :ok :read [20 48] INFO [2022-01-10 12:53:27,628] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,629] jepsen worker 61 - jepsen.util 61 :ok :read [20 48] INFO [2022-01-10 12:53:27,630] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,631] jepsen worker 61 - jepsen.util 61 :ok :read [20 48] INFO [2022-01-10 12:53:27,631] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,632] jepsen worker 53 - jepsen.util 143 :ok :write [17 13] INFO [2022-01-10 12:53:27,632] jepsen worker 61 - jepsen.util 61 :ok :read [20 48] INFO [2022-01-10 12:53:27,632] jepsen worker 43 - jepsen.util 43 :ok :write [14 30] INFO [2022-01-10 12:53:27,632] jepsen worker 53 - jepsen.util 143 :invoke :write [17 18] INFO [2022-01-10 12:53:27,633] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,633] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,633] jepsen worker 61 - jepsen.util 61 :ok :read [20 48] INFO [2022-01-10 12:53:27,634] jepsen worker 61 - jepsen.util 61 :invoke :write [20 26] INFO [2022-01-10 12:53:27,634] jepsen worker 43 - jepsen.util 43 :ok :read [14 30] INFO [2022-01-10 12:53:27,635] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,636] jepsen worker 43 - jepsen.util 43 :ok :read [14 30] INFO [2022-01-10 12:53:27,636] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,637] jepsen worker 43 - jepsen.util 43 :ok :read [14 30] INFO [2022-01-10 12:53:27,637] jepsen worker 43 - jepsen.util 43 :invoke :write [14 4] INFO [2022-01-10 12:53:27,639] jepsen worker 23 - jepsen.util 23 :ok :write [7 36] INFO [2022-01-10 12:53:27,639] jepsen worker 69 - jepsen.util 159 :ok :write [23 7] INFO [2022-01-10 12:53:27,639] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,640] jepsen worker 69 - jepsen.util 159 :invoke :write [23 39] INFO [2022-01-10 12:53:27,640] jepsen worker 23 - jepsen.util 23 :ok :read [7 36] INFO [2022-01-10 12:53:27,641] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,642] jepsen worker 23 - jepsen.util 23 :ok :read [7 36] INFO [2022-01-10 12:53:27,642] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,643] jepsen worker 23 - jepsen.util 23 :ok :read [7 36] INFO [2022-01-10 12:53:27,644] jepsen worker 23 - jepsen.util 23 :invoke :write [7 3] INFO [2022-01-10 12:53:27,645] jepsen worker 13 - jepsen.util 13 :ok :write [4 5] INFO [2022-01-10 12:53:27,645] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,646] jepsen worker 13 - jepsen.util 13 :ok :read [4 5] INFO [2022-01-10 12:53:27,646] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,647] jepsen worker 75 - jepsen.util 165 :ok :write [25 13] INFO [2022-01-10 12:53:27,647] jepsen worker 13 - jepsen.util 13 :ok :read [4 5] INFO [2022-01-10 12:53:27,647] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,648] jepsen worker 13 - jepsen.util 13 :invoke :write [4 41] INFO [2022-01-10 12:53:27,648] jepsen worker 75 - jepsen.util 165 :ok :read [25 13] INFO [2022-01-10 12:53:27,648] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,649] jepsen worker 75 - jepsen.util 165 :ok :read [25 13] INFO [2022-01-10 12:53:27,649] jepsen worker 80 - jepsen.util 260 :ok :write [26 6] INFO [2022-01-10 12:53:27,650] jepsen worker 75 - jepsen.util 165 :invoke :write [25 42] INFO [2022-01-10 12:53:27,650] jepsen worker 80 - jepsen.util 260 :invoke :write [26 37] INFO [2022-01-10 12:53:27,656] jepsen worker 53 - jepsen.util 143 :ok :write [17 18] INFO [2022-01-10 12:53:27,656] jepsen worker 61 - jepsen.util 61 :ok :write [20 26] INFO [2022-01-10 12:53:27,657] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,657] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,657] jepsen worker 43 - jepsen.util 43 :ok :write [14 4] INFO [2022-01-10 12:53:27,658] jepsen worker 61 - jepsen.util 61 :ok :read [20 26] INFO [2022-01-10 12:53:27,658] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,658] jepsen worker 53 - jepsen.util 143 :ok :read [17 18] INFO [2022-01-10 12:53:27,659] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,659] jepsen worker 43 - jepsen.util 43 :ok :read [14 4] INFO [2022-01-10 12:53:27,659] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,659] jepsen worker 61 - jepsen.util 61 :ok :read [20 26] INFO [2022-01-10 12:53:27,660] jepsen worker 43 - jepsen.util 43 :invoke :write [14 28] INFO [2022-01-10 12:53:27,660] jepsen worker 53 - jepsen.util 143 :ok :read [17 18] INFO [2022-01-10 12:53:27,660] jepsen worker 61 - jepsen.util 61 :invoke :write [20 5] INFO [2022-01-10 12:53:27,661] jepsen worker 53 - jepsen.util 143 :invoke :write [17 5] INFO [2022-01-10 12:53:27,663] jepsen worker 69 - jepsen.util 159 :ok :write [23 39] INFO [2022-01-10 12:53:27,664] jepsen worker 23 - jepsen.util 23 :ok :write [7 3] INFO [2022-01-10 12:53:27,665] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,665] jepsen worker 69 - jepsen.util 159 :invoke :write [23 47] INFO [2022-01-10 12:53:27,666] jepsen worker 23 - jepsen.util 23 :ok :read [7 3] INFO [2022-01-10 12:53:27,666] jepsen worker 23 - jepsen.util 23 :invoke :write [7 19] INFO [2022-01-10 12:53:27,668] jepsen worker 13 - jepsen.util 13 :ok :write [4 41] INFO [2022-01-10 12:53:27,669] jepsen worker 13 - jepsen.util 13 :invoke :write [4 16] INFO [2022-01-10 12:53:27,671] jepsen worker 75 - jepsen.util 165 :ok :write [25 42] INFO [2022-01-10 12:53:27,672] jepsen worker 75 - jepsen.util 165 :invoke :write [25 41] INFO [2022-01-10 12:53:27,676] jepsen worker 80 - jepsen.util 260 :ok :write [26 37] INFO [2022-01-10 12:53:27,677] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,677] jepsen worker 80 - jepsen.util 260 :ok :read [26 37] INFO [2022-01-10 12:53:27,678] jepsen worker 80 - jepsen.util 260 :invoke :write [26 26] INFO [2022-01-10 12:53:27,679] jepsen worker 61 - jepsen.util 61 :ok :write [20 5] INFO [2022-01-10 12:53:27,679] jepsen worker 61 - jepsen.util 61 :invoke :write [20 8] INFO [2022-01-10 12:53:27,683] jepsen worker 53 - jepsen.util 143 :ok :write [17 5] INFO [2022-01-10 12:53:27,683] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,684] jepsen worker 43 - jepsen.util 43 :ok :write [14 28] INFO [2022-01-10 12:53:27,684] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,684] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:27,685] jepsen worker 53 - jepsen.util 143 :invoke :write [17 45] INFO [2022-01-10 12:53:27,685] jepsen worker 43 - jepsen.util 43 :ok :read [14 28] INFO [2022-01-10 12:53:27,686] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,687] jepsen worker 43 - jepsen.util 43 :ok :read [14 28] INFO [2022-01-10 12:53:27,688] jepsen worker 43 - jepsen.util 43 :invoke :write [14 44] INFO [2022-01-10 12:53:27,690] jepsen worker 69 - jepsen.util 159 :ok :write [23 47] INFO [2022-01-10 12:53:27,690] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,691] jepsen worker 69 - jepsen.util 159 :ok :read [23 47] INFO [2022-01-10 12:53:27,691] jepsen worker 23 - jepsen.util 23 :ok :write [7 19] INFO [2022-01-10 12:53:27,691] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,692] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,692] jepsen worker 23 - jepsen.util 23 :ok :read [7 19] INFO [2022-01-10 12:53:27,693] jepsen worker 69 - jepsen.util 159 :ok :read [23 47] INFO [2022-01-10 12:53:27,693] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,693] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,693] jepsen worker 23 - jepsen.util 23 :ok :read [7 19] INFO [2022-01-10 12:53:27,694] jepsen worker 69 - jepsen.util 159 :ok :read [23 47] INFO [2022-01-10 12:53:27,694] jepsen worker 23 - jepsen.util 23 :invoke :write [7 31] INFO [2022-01-10 12:53:27,694] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,695] jepsen worker 13 - jepsen.util 13 :ok :write [4 16] INFO [2022-01-10 12:53:27,695] jepsen worker 76 - jepsen.util 166 :ok :write [25 49] INFO [2022-01-10 12:53:27,695] jepsen worker 69 - jepsen.util 159 :ok :read [23 47] INFO [2022-01-10 12:53:27,696] jepsen worker 13 - jepsen.util 13 :invoke :write [4 13] INFO [2022-01-10 12:53:27,696] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,696] jepsen worker 76 - jepsen.util 166 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,697] jepsen worker 76 - jepsen.util 166 :ok :read [25 49] INFO [2022-01-10 12:53:27,697] jepsen worker 69 - jepsen.util 159 :ok :read [23 47] INFO [2022-01-10 12:53:27,698] jepsen worker 76 - jepsen.util 166 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,698] jepsen worker 69 - jepsen.util 159 :invoke :write [23 15] INFO [2022-01-10 12:53:27,699] jepsen worker 76 - jepsen.util 166 :ok :read [25 49] INFO [2022-01-10 12:53:27,699] jepsen worker 76 - jepsen.util 166 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,700] jepsen worker 61 - jepsen.util 61 :ok :write [20 8] INFO [2022-01-10 12:53:27,700] jepsen worker 76 - jepsen.util 166 :ok :read [25 49] INFO [2022-01-10 12:53:27,700] jepsen worker 61 - jepsen.util 61 :invoke :write [20 42] INFO [2022-01-10 12:53:27,701] jepsen worker 76 - jepsen.util 166 :invoke :write [25 40] INFO [2022-01-10 12:53:27,703] jepsen worker 80 - jepsen.util 260 :ok :write [26 26] INFO [2022-01-10 12:53:27,703] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,704] jepsen worker 80 - jepsen.util 260 :ok :read [26 26] INFO [2022-01-10 12:53:27,704] jepsen worker 80 - jepsen.util 260 :invoke :write [26 22] INFO [2022-01-10 12:53:27,705] jepsen worker 43 - jepsen.util 43 :ok :write [14 44] INFO [2022-01-10 12:53:27,705] jepsen worker 43 - jepsen.util 43 :invoke :write [14 11] INFO [2022-01-10 12:53:27,711] jepsen worker 53 - jepsen.util 143 :ok :write [17 45] INFO [2022-01-10 12:53:27,712] jepsen worker 53 - jepsen.util 143 :invoke :write [17 34] INFO [2022-01-10 12:53:27,714] jepsen worker 23 - jepsen.util 23 :ok :write [7 31] INFO [2022-01-10 12:53:27,714] jepsen worker 23 - jepsen.util 23 :invoke :write [7 40] INFO [2022-01-10 12:53:27,718] jepsen worker 75 - jepsen.util 165 :ok :write [25 41] INFO [2022-01-10 12:53:27,719] jepsen worker 75 - jepsen.util 165 :invoke :write [25 48] INFO [2022-01-10 12:53:27,720] jepsen worker 69 - jepsen.util 159 :ok :write [23 15] INFO [2022-01-10 12:53:27,720] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,721] jepsen worker 69 - jepsen.util 159 :ok :read [23 15] INFO [2022-01-10 12:53:27,722] jepsen worker 69 - jepsen.util 159 :invoke :write [23 22] INFO [2022-01-10 12:53:27,851] jepsen worker 61 - jepsen.util 61 :ok :write [20 42] INFO [2022-01-10 12:53:27,851] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,852] jepsen worker 61 - jepsen.util 61 :ok :read [20 42] INFO [2022-01-10 12:53:27,853] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,853] jepsen worker 61 - jepsen.util 61 :ok :read [20 42] INFO [2022-01-10 12:53:27,854] jepsen worker 13 - jepsen.util 13 :ok :write [4 13] INFO [2022-01-10 12:53:27,854] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,855] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,855] jepsen worker 61 - jepsen.util 61 :ok :read [20 42] INFO [2022-01-10 12:53:27,855] jepsen worker 43 - jepsen.util 43 :ok :write [14 11] INFO [2022-01-10 12:53:27,856] jepsen worker 13 - jepsen.util 13 :ok :read [4 13] INFO [2022-01-10 12:53:27,856] jepsen worker 43 - jepsen.util 43 :invoke :write [14 35] INFO [2022-01-10 12:53:27,856] jepsen worker 61 - jepsen.util 61 :invoke :write [20 29] INFO [2022-01-10 12:53:27,857] jepsen worker 13 - jepsen.util 13 :invoke :write [4 1] INFO [2022-01-10 12:53:27,860] jepsen worker 80 - jepsen.util 260 :ok :write [26 22] INFO [2022-01-10 12:53:27,861] jepsen worker 80 - jepsen.util 260 :invoke :write [26 32] INFO [2022-01-10 12:53:27,863] jepsen worker 23 - jepsen.util 23 :ok :write [7 40] INFO [2022-01-10 12:53:27,863] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,864] jepsen worker 23 - jepsen.util 23 :ok :read [7 40] INFO [2022-01-10 12:53:27,865] jepsen worker 23 - jepsen.util 23 :invoke :write [7 39] INFO [2022-01-10 12:53:27,866] jepsen worker 53 - jepsen.util 143 :ok :write [17 34] INFO [2022-01-10 12:53:27,866] jepsen worker 53 - jepsen.util 143 :invoke :write [17 5] INFO [2022-01-10 12:53:27,868] jepsen worker 75 - jepsen.util 165 :ok :write [25 48] INFO [2022-01-10 12:53:27,868] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,869] jepsen worker 75 - jepsen.util 165 :ok :read [25 48] INFO [2022-01-10 12:53:27,869] jepsen worker 75 - jepsen.util 165 :invoke :write [25 29] INFO [2022-01-10 12:53:27,871] jepsen worker 69 - jepsen.util 159 :ok :write [23 22] INFO [2022-01-10 12:53:27,872] jepsen worker 69 - jepsen.util 159 :invoke :write [23 32] INFO [2022-01-10 12:53:27,872] jepsen worker 61 - jepsen.util 61 :ok :write [20 29] INFO [2022-01-10 12:53:27,873] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:27,873] jepsen worker 61 - jepsen.util 61 :ok :read [20 29] INFO [2022-01-10 12:53:27,874] jepsen worker 61 - jepsen.util 61 :invoke :write [20 10] INFO [2022-01-10 12:53:27,876] jepsen worker 43 - jepsen.util 43 :ok :write [14 35] INFO [2022-01-10 12:53:27,877] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,877] jepsen worker 13 - jepsen.util 13 :ok :write [4 1] INFO [2022-01-10 12:53:27,878] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,878] jepsen worker 43 - jepsen.util 43 :ok :read [14 35] INFO [2022-01-10 12:53:27,878] jepsen worker 43 - jepsen.util 43 :invoke :write [14 14] INFO [2022-01-10 12:53:27,879] jepsen worker 13 - jepsen.util 13 :ok :read [4 1] INFO [2022-01-10 12:53:27,879] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:27,880] jepsen worker 13 - jepsen.util 13 :ok :read [4 1] INFO [2022-01-10 12:53:27,880] jepsen worker 13 - jepsen.util 13 :invoke :write [4 37] INFO [2022-01-10 12:53:27,882] jepsen worker 80 - jepsen.util 260 :ok :write [26 32] INFO [2022-01-10 12:53:27,883] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,883] jepsen worker 23 - jepsen.util 23 :ok :write [7 39] INFO [2022-01-10 12:53:27,883] jepsen worker 23 - jepsen.util 23 :invoke :write [7 43] INFO [2022-01-10 12:53:27,883] jepsen worker 80 - jepsen.util 260 :ok :read [26 32] INFO [2022-01-10 12:53:27,884] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,885] jepsen worker 80 - jepsen.util 260 :ok :read [26 32] INFO [2022-01-10 12:53:27,885] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,886] jepsen worker 80 - jepsen.util 260 :ok :read [26 32] INFO [2022-01-10 12:53:27,887] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:27,888] jepsen worker 75 - jepsen.util 165 :ok :write [25 29] INFO [2022-01-10 12:53:27,888] jepsen worker 80 - jepsen.util 260 :ok :read [26 32] INFO [2022-01-10 12:53:27,888] jepsen worker 53 - jepsen.util 143 :ok :write [17 5] INFO [2022-01-10 12:53:27,888] jepsen worker 80 - jepsen.util 260 :invoke :write [26 49] INFO [2022-01-10 12:53:27,889] jepsen worker 53 - jepsen.util 143 :invoke :write [17 16] INFO [2022-01-10 12:53:27,889] jepsen worker 75 - jepsen.util 165 :invoke :write [25 5] INFO [2022-01-10 12:53:27,892] jepsen worker 61 - jepsen.util 61 :ok :write [20 10] INFO [2022-01-10 12:53:27,892] jepsen worker 61 - jepsen.util 61 :invoke :write [20 45] INFO [2022-01-10 12:53:27,896] jepsen worker 69 - jepsen.util 159 :ok :write [23 32] INFO [2022-01-10 12:53:27,896] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,897] jepsen worker 43 - jepsen.util 43 :ok :write [14 14] INFO [2022-01-10 12:53:27,897] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,897] jepsen worker 69 - jepsen.util 159 :ok :read [23 32] INFO [2022-01-10 12:53:27,898] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,898] jepsen worker 43 - jepsen.util 43 :ok :read [14 14] INFO [2022-01-10 12:53:27,899] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,899] jepsen worker 69 - jepsen.util 159 :ok :read [23 32] INFO [2022-01-10 12:53:27,899] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:27,899] jepsen worker 43 - jepsen.util 43 :ok :read [14 14] INFO [2022-01-10 12:53:27,900] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,900] jepsen worker 69 - jepsen.util 159 :ok :read [23 32] INFO [2022-01-10 12:53:27,901] jepsen worker 69 - jepsen.util 159 :invoke :write [23 12] INFO [2022-01-10 12:53:27,901] jepsen worker 43 - jepsen.util 43 :ok :read [14 14] INFO [2022-01-10 12:53:27,901] jepsen worker 23 - jepsen.util 23 :ok :write [7 43] INFO [2022-01-10 12:53:27,901] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:27,902] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:27,902] jepsen worker 43 - jepsen.util 43 :ok :read [14 14] INFO [2022-01-10 12:53:27,903] jepsen worker 43 - jepsen.util 43 :invoke :write [14 49] INFO [2022-01-10 12:53:27,903] jepsen worker 23 - jepsen.util 23 :ok :read [7 43] INFO [2022-01-10 12:53:27,904] jepsen worker 23 - jepsen.util 23 :invoke :write [7 18] INFO [2022-01-10 12:53:27,954] jepsen worker 86 - jepsen.util 86 :info :write [28 15] :RequestTimeoutException INFO [2022-01-10 12:53:27,955] jepsen worker 86 - jepsen.util 176 :invoke :write [28 45] INFO [2022-01-10 12:53:27,985] jepsen worker 13 - jepsen.util 13 :ok :write [4 37] INFO [2022-01-10 12:53:27,986] jepsen worker 13 - jepsen.util 13 :invoke :write [4 39] INFO [2022-01-10 12:53:27,986] jepsen worker 75 - jepsen.util 165 :ok :write [25 5] INFO [2022-01-10 12:53:27,987] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,988] jepsen worker 75 - jepsen.util 165 :ok :read [25 5] INFO [2022-01-10 12:53:27,988] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,989] jepsen worker 75 - jepsen.util 165 :ok :read [25 5] INFO [2022-01-10 12:53:27,989] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:27,990] jepsen worker 75 - jepsen.util 165 :ok :read [25 5] INFO [2022-01-10 12:53:27,990] jepsen worker 79 - jepsen.util 259 :ok :write [26 34] INFO [2022-01-10 12:53:27,991] jepsen worker 75 - jepsen.util 165 :invoke :write [25 28] INFO [2022-01-10 12:53:27,991] jepsen worker 79 - jepsen.util 259 :invoke :write [26 40] INFO [2022-01-10 12:53:27,991] jepsen worker 61 - jepsen.util 61 :ok :write [20 45] INFO [2022-01-10 12:53:27,992] jepsen worker 61 - jepsen.util 61 :invoke :write [20 19] INFO [2022-01-10 12:53:27,996] jepsen worker 53 - jepsen.util 143 :ok :write [17 16] INFO [2022-01-10 12:53:27,997] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:27,998] jepsen worker 53 - jepsen.util 143 :ok :read [17 16] INFO [2022-01-10 12:53:27,998] jepsen worker 53 - jepsen.util 143 :invoke :write [17 40] INFO [2022-01-10 12:53:27,999] jepsen worker 43 - jepsen.util 43 :ok :write [14 49] INFO [2022-01-10 12:53:27,999] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:28,000] jepsen worker 43 - jepsen.util 43 :ok :read [14 49] INFO [2022-01-10 12:53:28,001] jepsen worker 43 - jepsen.util 43 :invoke :write [14 12] INFO [2022-01-10 12:53:28,002] jepsen worker 69 - jepsen.util 159 :ok :write [23 12] INFO [2022-01-10 12:53:28,002] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:28,003] jepsen worker 69 - jepsen.util 159 :ok :read [23 12] INFO [2022-01-10 12:53:28,004] jepsen worker 69 - jepsen.util 159 :invoke :write [23 14] INFO [2022-01-10 12:53:28,006] jepsen worker 23 - jepsen.util 23 :ok :write [7 18] INFO [2022-01-10 12:53:28,006] jepsen worker 23 - jepsen.util 23 :invoke :write [7 26] INFO [2022-01-10 12:53:28,008] jepsen worker 13 - jepsen.util 13 :ok :write [4 39] INFO [2022-01-10 12:53:28,008] jepsen worker 13 - jepsen.util 13 :invoke :write [4 39] INFO [2022-01-10 12:53:28,019] jepsen worker 79 - jepsen.util 259 :ok :write [26 40] INFO [2022-01-10 12:53:28,020] jepsen worker 79 - jepsen.util 259 :invoke :write [26 0] INFO [2022-01-10 12:53:28,020] jepsen worker 75 - jepsen.util 165 :ok :write [25 28] INFO [2022-01-10 12:53:28,021] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,022] jepsen worker 75 - jepsen.util 165 :ok :read [25 28] INFO [2022-01-10 12:53:28,023] jepsen worker 75 - jepsen.util 165 :invoke :write [25 49] INFO [2022-01-10 12:53:28,024] jepsen worker 53 - jepsen.util 143 :ok :write [17 40] INFO [2022-01-10 12:53:28,024] jepsen worker 53 - jepsen.util 143 :invoke :write [17 6] INFO [2022-01-10 12:53:28,027] jepsen worker 69 - jepsen.util 159 :ok :write [23 14] INFO [2022-01-10 12:53:28,027] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:28,028] jepsen worker 61 - jepsen.util 61 :ok :write [20 19] INFO [2022-01-10 12:53:28,028] jepsen worker 69 - jepsen.util 159 :ok :read [23 14] INFO [2022-01-10 12:53:28,029] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:28,030] jepsen worker 61 - jepsen.util 61 :ok :read [20 19] INFO [2022-01-10 12:53:28,030] jepsen worker 69 - jepsen.util 159 :invoke :write [23 35] INFO [2022-01-10 12:53:28,030] jepsen worker 13 - jepsen.util 13 :ok :write [4 39] INFO [2022-01-10 12:53:28,030] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:28,031] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,031] jepsen worker 61 - jepsen.util 61 :ok :read [20 19] INFO [2022-01-10 12:53:28,032] jepsen worker 13 - jepsen.util 13 :ok :read [4 39] INFO [2022-01-10 12:53:28,032] jepsen worker 61 - jepsen.util 61 :invoke :write [20 36] INFO [2022-01-10 12:53:28,033] jepsen worker 13 - jepsen.util 13 :invoke :write [4 1] INFO [2022-01-10 12:53:28,034] jepsen worker 79 - jepsen.util 259 :ok :write [26 0] INFO [2022-01-10 12:53:28,035] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,044] jepsen worker 43 - jepsen.util 43 :ok :write [14 12] INFO [2022-01-10 12:53:28,045] jepsen worker 53 - jepsen.util 143 :ok :write [17 6] INFO [2022-01-10 12:53:28,045] jepsen worker 69 - jepsen.util 159 :ok :write [23 35] INFO [2022-01-10 12:53:28,045] jepsen worker 79 - jepsen.util 259 :ok :read [26 0] INFO [2022-01-10 12:53:28,045] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:28,046] jepsen worker 43 - jepsen.util 43 :ok :read [14 12] INFO [2022-01-10 12:53:28,047] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,047] jepsen worker 69 - jepsen.util 159 :invoke :write [23 17] INFO [2022-01-10 12:53:28,048] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:28,048] jepsen worker 79 - jepsen.util 259 :ok :read [26 0] INFO [2022-01-10 12:53:28,048] jepsen worker 43 - jepsen.util 43 :invoke :write [14 32] INFO [2022-01-10 12:53:28,049] jepsen worker 53 - jepsen.util 143 :ok :read [17 6] INFO [2022-01-10 12:53:28,049] jepsen worker 23 - jepsen.util 23 :ok :write [7 26] INFO [2022-01-10 12:53:28,049] jepsen worker 53 - jepsen.util 143 :invoke :write [17 35] INFO [2022-01-10 12:53:28,050] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,051] jepsen worker 23 - jepsen.util 23 :invoke :write [7 21] INFO [2022-01-10 12:53:28,051] jepsen worker 79 - jepsen.util 259 :ok :read [26 0] INFO [2022-01-10 12:53:28,052] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,053] jepsen worker 79 - jepsen.util 259 :ok :read [26 0] INFO [2022-01-10 12:53:28,054] jepsen worker 79 - jepsen.util 259 :invoke :write [26 21] INFO [2022-01-10 12:53:28,063] jepsen worker 75 - jepsen.util 165 :ok :write [25 49] INFO [2022-01-10 12:53:28,064] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,064] jepsen worker 75 - jepsen.util 165 :ok :read [25 49] INFO [2022-01-10 12:53:28,065] jepsen worker 75 - jepsen.util 165 :invoke :write [25 35] INFO [2022-01-10 12:53:28,066] jepsen worker 13 - jepsen.util 13 :ok :write [4 1] INFO [2022-01-10 12:53:28,067] jepsen worker 13 - jepsen.util 13 :invoke :write [4 32] INFO [2022-01-10 12:53:28,071] jepsen worker 79 - jepsen.util 259 :ok :write [26 21] INFO [2022-01-10 12:53:28,071] jepsen worker 79 - jepsen.util 259 :invoke :write [26 28] INFO [2022-01-10 12:53:28,075] jepsen worker 53 - jepsen.util 143 :ok :write [17 35] INFO [2022-01-10 12:53:28,076] jepsen worker 53 - jepsen.util 143 :invoke :write [17 2] INFO [2022-01-10 12:53:28,076] jepsen worker 61 - jepsen.util 61 :ok :write [20 36] INFO [2022-01-10 12:53:28,077] jepsen worker 61 - jepsen.util 61 :invoke :write [20 9] INFO [2022-01-10 12:53:28,080] jepsen worker 69 - jepsen.util 159 :ok :write [23 17] INFO [2022-01-10 12:53:28,081] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:28,081] jepsen worker 69 - jepsen.util 159 :ok :read [23 17] INFO [2022-01-10 12:53:28,082] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:28,082] jepsen worker 69 - jepsen.util 159 :ok :read [23 17] INFO [2022-01-10 12:53:28,083] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:28,084] jepsen worker 43 - jepsen.util 43 :ok :write [14 32] INFO [2022-01-10 12:53:28,084] jepsen worker 69 - jepsen.util 159 :ok :read [23 17] INFO [2022-01-10 12:53:28,084] jepsen worker 43 - jepsen.util 43 :invoke :write [14 48] INFO [2022-01-10 12:53:28,085] jepsen worker 13 - jepsen.util 13 :ok :write [4 32] INFO [2022-01-10 12:53:28,085] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:28,085] jepsen worker 13 - jepsen.util 13 :invoke :write [4 22] INFO [2022-01-10 12:53:28,086] jepsen worker 69 - jepsen.util 159 :ok :read [23 17] INFO [2022-01-10 12:53:28,086] jepsen worker 69 - jepsen.util 159 :invoke :write [23 35] INFO [2022-01-10 12:53:28,088] jepsen worker 79 - jepsen.util 259 :ok :write [26 28] INFO [2022-01-10 12:53:28,089] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,090] jepsen worker 79 - jepsen.util 259 :ok :read [26 28] INFO [2022-01-10 12:53:28,090] jepsen worker 79 - jepsen.util 259 :invoke :write [26 26] INFO [2022-01-10 12:53:28,093] jepsen worker 53 - jepsen.util 143 :ok :write [17 2] INFO [2022-01-10 12:53:28,094] jepsen worker 23 - jepsen.util 23 :ok :write [7 21] INFO [2022-01-10 12:53:28,094] jepsen worker 53 - jepsen.util 143 :invoke :write [17 6] INFO [2022-01-10 12:53:28,094] jepsen worker 23 - jepsen.util 23 :invoke :write [7 40] INFO [2022-01-10 12:53:28,097] jepsen worker 69 - jepsen.util 159 :ok :write [23 35] INFO [2022-01-10 12:53:28,098] jepsen worker 69 - jepsen.util 159 :invoke :write [23 7] INFO [2022-01-10 12:53:28,100] jepsen worker 75 - jepsen.util 165 :ok :write [25 35] INFO [2022-01-10 12:53:28,101] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,102] jepsen worker 75 - jepsen.util 165 :ok :read [25 35] INFO [2022-01-10 12:53:28,102] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,103] jepsen worker 13 - jepsen.util 13 :ok :write [4 22] INFO [2022-01-10 12:53:28,103] jepsen worker 75 - jepsen.util 165 :ok :read [25 35] INFO [2022-01-10 12:53:28,104] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,104] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,105] jepsen worker 13 - jepsen.util 13 :ok :read [4 22] INFO [2022-01-10 12:53:28,105] jepsen worker 75 - jepsen.util 165 :ok :read [25 35] INFO [2022-01-10 12:53:28,105] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,106] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,106] jepsen worker 13 - jepsen.util 13 :ok :read [4 22] INFO [2022-01-10 12:53:28,107] jepsen worker 75 - jepsen.util 165 :ok :read [25 35] INFO [2022-01-10 12:53:28,107] jepsen worker 13 - jepsen.util 13 :invoke :write [4 44] INFO [2022-01-10 12:53:28,107] jepsen worker 75 - jepsen.util 165 :invoke :write [25 38] INFO [2022-01-10 12:53:28,108] jepsen worker 61 - jepsen.util 61 :ok :write [20 9] INFO [2022-01-10 12:53:28,108] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:28,109] jepsen worker 79 - jepsen.util 259 :ok :write [26 26] INFO [2022-01-10 12:53:28,109] jepsen worker 61 - jepsen.util 61 :ok :read [20 9] INFO [2022-01-10 12:53:28,109] jepsen worker 79 - jepsen.util 259 :invoke :write [26 13] INFO [2022-01-10 12:53:28,110] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:28,110] jepsen worker 61 - jepsen.util 61 :ok :read [20 9] INFO [2022-01-10 12:53:28,111] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:28,112] jepsen worker 61 - jepsen.util 61 :ok :read [20 9] INFO [2022-01-10 12:53:28,113] jepsen worker 61 - jepsen.util 61 :invoke :write [20 2] INFO [2022-01-10 12:53:28,113] jepsen worker 53 - jepsen.util 143 :ok :write [17 6] INFO [2022-01-10 12:53:28,113] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:28,114] jepsen worker 53 - jepsen.util 143 :ok :read [17 6] INFO [2022-01-10 12:53:28,115] jepsen worker 53 - jepsen.util 143 :invoke :write [17 29] INFO [2022-01-10 12:53:28,116] jepsen worker 43 - jepsen.util 43 :ok :write [14 48] INFO [2022-01-10 12:53:28,117] jepsen worker 43 - jepsen.util 43 :invoke :write [14 29] INFO [2022-01-10 12:53:28,118] jepsen worker 69 - jepsen.util 159 :ok :write [23 7] INFO [2022-01-10 12:53:28,119] jepsen worker 69 - jepsen.util 159 :invoke :write [23 49] INFO [2022-01-10 12:53:28,121] jepsen worker 13 - jepsen.util 13 :ok :write [4 44] INFO [2022-01-10 12:53:28,122] jepsen worker 13 - jepsen.util 13 :invoke :write [4 32] INFO [2022-01-10 12:53:28,124] jepsen worker 23 - jepsen.util 23 :ok :write [7 40] INFO [2022-01-10 12:53:28,125] jepsen worker 23 - jepsen.util 23 :invoke :write [7 11] INFO [2022-01-10 12:53:28,125] jepsen worker 79 - jepsen.util 259 :ok :write [26 13] INFO [2022-01-10 12:53:28,126] jepsen worker 79 - jepsen.util 259 :invoke :write [26 7] INFO [2022-01-10 12:53:28,132] jepsen worker 53 - jepsen.util 143 :ok :write [17 29] INFO [2022-01-10 12:53:28,132] jepsen worker 75 - jepsen.util 165 :ok :write [25 38] INFO [2022-01-10 12:53:28,132] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:28,133] jepsen worker 75 - jepsen.util 165 :invoke :write [25 43] INFO [2022-01-10 12:53:28,133] jepsen worker 53 - jepsen.util 143 :ok :read [17 29] INFO [2022-01-10 12:53:28,134] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:28,135] jepsen worker 53 - jepsen.util 143 :ok :read [17 29] INFO [2022-01-10 12:53:28,135] jepsen worker 53 - jepsen.util 143 :invoke :write [17 48] INFO [2022-01-10 12:53:28,135] jepsen worker 69 - jepsen.util 159 :ok :write [23 49] INFO [2022-01-10 12:53:28,136] jepsen worker 69 - jepsen.util 159 :invoke :write [23 1] INFO [2022-01-10 12:53:28,139] jepsen worker 13 - jepsen.util 13 :ok :write [4 32] INFO [2022-01-10 12:53:28,139] jepsen worker 61 - jepsen.util 61 :ok :write [20 2] INFO [2022-01-10 12:53:28,139] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,140] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:28,141] jepsen worker 13 - jepsen.util 13 :ok :read [4 32] INFO [2022-01-10 12:53:28,141] jepsen worker 61 - jepsen.util 61 :ok :read [20 2] INFO [2022-01-10 12:53:28,141] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,141] jepsen worker 61 - jepsen.util 61 :invoke :write [20 22] INFO [2022-01-10 12:53:28,142] jepsen worker 13 - jepsen.util 13 :ok :read [4 32] INFO [2022-01-10 12:53:28,142] jepsen worker 13 - jepsen.util 13 :invoke :write [4 15] INFO [2022-01-10 12:53:28,143] jepsen worker 79 - jepsen.util 259 :ok :write [26 7] INFO [2022-01-10 12:53:28,143] jepsen worker 79 - jepsen.util 259 :invoke :write [26 22] INFO [2022-01-10 12:53:28,145] jepsen worker 43 - jepsen.util 43 :ok :write [14 29] INFO [2022-01-10 12:53:28,145] jepsen worker 43 - jepsen.util 43 :invoke :write [14 13] INFO [2022-01-10 12:53:28,151] jepsen worker 53 - jepsen.util 143 :ok :write [17 48] INFO [2022-01-10 12:53:28,152] jepsen worker 23 - jepsen.util 23 :ok :write [7 11] INFO [2022-01-10 12:53:28,237] jepsen worker 69 - jepsen.util 159 :ok :write [23 1] INFO [2022-01-10 12:53:28,237] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:28,238] jepsen worker 75 - jepsen.util 165 :ok :write [25 43] INFO [2022-01-10 12:53:28,238] jepsen worker 23 - jepsen.util 23 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,238] jepsen worker 53 - jepsen.util 143 :ok :read [17 48] INFO [2022-01-10 12:53:28,238] jepsen worker 75 - jepsen.util 165 :invoke :write [25 21] INFO [2022-01-10 12:53:28,239] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:28,239] jepsen worker 23 - jepsen.util 23 :ok :read [7 11] INFO [2022-01-10 12:53:28,239] jepsen worker 53 - jepsen.util 143 :invoke :write [17 39] INFO [2022-01-10 12:53:28,240] jepsen worker 69 - jepsen.util 159 :ok :read [23 1] INFO [2022-01-10 12:53:28,240] jepsen worker 23 - jepsen.util 23 :invoke :write [7 22] INFO [2022-01-10 12:53:28,241] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:28,241] jepsen worker 69 - jepsen.util 159 :ok :read [23 1] INFO [2022-01-10 12:53:28,242] jepsen worker 69 - jepsen.util 159 :invoke :write [23 6] INFO [2022-01-10 12:53:28,274] jepsen worker 61 - jepsen.util 61 :ok :write [20 22] INFO [2022-01-10 12:53:28,275] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:28,275] jepsen worker 13 - jepsen.util 13 :ok :write [4 15] INFO [2022-01-10 12:53:28,276] jepsen worker 61 - jepsen.util 61 :ok :read [20 22] INFO [2022-01-10 12:53:28,277] jepsen worker 13 - jepsen.util 13 :invoke :write [4 28] INFO [2022-01-10 12:53:28,278] jepsen worker 61 - jepsen.util 61 :invoke :write [20 7] INFO [2022-01-10 12:53:28,282] jepsen worker 43 - jepsen.util 43 :ok :write [14 13] INFO [2022-01-10 12:53:28,282] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:28,282] jepsen worker 79 - jepsen.util 259 :ok :write [26 22] INFO [2022-01-10 12:53:28,283] jepsen worker 43 - jepsen.util 43 :ok :read [14 13] INFO [2022-01-10 12:53:28,284] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,284] jepsen worker 79 - jepsen.util 259 :ok :read [26 22] INFO [2022-01-10 12:53:28,285] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:28,285] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,285] jepsen worker 43 - jepsen.util 43 :ok :read [14 13] INFO [2022-01-10 12:53:28,286] jepsen worker 79 - jepsen.util 259 :ok :read [26 22] INFO [2022-01-10 12:53:28,286] jepsen worker 22 - jepsen.util 112 :ok :write [7 40] INFO [2022-01-10 12:53:28,287] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:28,287] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,288] jepsen worker 43 - jepsen.util 43 :ok :read [14 13] INFO [2022-01-10 12:53:28,288] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,288] jepsen worker 22 - jepsen.util 112 :ok :read [7 40] INFO [2022-01-10 12:53:28,289] jepsen worker 79 - jepsen.util 259 :ok :read [26 22] INFO [2022-01-10 12:53:28,289] jepsen worker 43 - jepsen.util 43 :invoke :write [14 5] INFO [2022-01-10 12:53:28,289] jepsen worker 79 - jepsen.util 259 :invoke :write [26 15] INFO [2022-01-10 12:53:28,290] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,290] jepsen worker 53 - jepsen.util 143 :ok :write [17 39] INFO [2022-01-10 12:53:28,291] jepsen worker 53 - jepsen.util 143 :invoke :write [17 40] INFO [2022-01-10 12:53:28,291] jepsen worker 22 - jepsen.util 112 :ok :read [7 40] INFO [2022-01-10 12:53:28,292] jepsen worker 22 - jepsen.util 112 :invoke :write [7 18] INFO [2022-01-10 12:53:28,293] jepsen worker 75 - jepsen.util 165 :ok :write [25 21] INFO [2022-01-10 12:53:28,294] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,295] jepsen worker 75 - jepsen.util 165 :ok :read [25 21] INFO [2022-01-10 12:53:28,295] jepsen worker 75 - jepsen.util 165 :invoke :write [25 17] INFO [2022-01-10 12:53:28,299] jepsen worker 70 - jepsen.util 160 :ok :write [23 34] INFO [2022-01-10 12:53:28,299] jepsen worker 70 - jepsen.util 160 :invoke :write [23 44] INFO [2022-01-10 12:53:28,302] jepsen worker 88 - jepsen.util 178 :ok :write [29 24] INFO [2022-01-10 12:53:28,302] jepsen worker 88 - jepsen.util 178 :invoke :write [29 30] INFO [2022-01-10 12:53:28,306] jepsen worker 84 - jepsen.util 84 :ok :write [28 8] INFO [2022-01-10 12:53:28,307] jepsen worker 2 - jepsen.util 92 :ok :write [0 0] INFO [2022-01-10 12:53:28,307] jepsen worker 84 - jepsen.util 84 :invoke :write [28 3] INFO [2022-01-10 12:53:28,307] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:28,308] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:28,309] jepsen worker 2 - jepsen.util 92 :invoke :write [0 38] INFO [2022-01-10 12:53:28,313] jepsen worker 3 - jepsen.util 3 :ok :write [1 12] INFO [2022-01-10 12:53:28,313] jepsen worker 3 - jepsen.util 3 :invoke :write [1 10] INFO [2022-01-10 12:53:28,317] jepsen worker 15 - jepsen.util 15 :ok :write [5 4] INFO [2022-01-10 12:53:28,317] jepsen worker 37 - jepsen.util 127 :ok :write [12 0] INFO [2022-01-10 12:53:28,317] jepsen worker 15 - jepsen.util 15 :invoke :write [5 12] INFO [2022-01-10 12:53:28,318] jepsen worker 37 - jepsen.util 127 :invoke :write [12 17] INFO [2022-01-10 12:53:28,323] jepsen worker 48 - jepsen.util 228 :ok :write [16 13] INFO [2022-01-10 12:53:28,324] jepsen worker 48 - jepsen.util 228 :invoke :write [16 32] INFO [2022-01-10 12:53:28,329] jepsen worker 81 - jepsen.util 171 :ok :write [27 3] INFO [2022-01-10 12:53:28,329] jepsen worker 81 - jepsen.util 171 :invoke :write [27 28] INFO [2022-01-10 12:53:28,335] jepsen worker 3 - jepsen.util 3 :ok :write [1 10] INFO [2022-01-10 12:53:28,336] jepsen worker 8 - jepsen.util 8 :ok :write [2 22] INFO [2022-01-10 12:53:28,337] jepsen worker 3 - jepsen.util 3 :invoke :write [1 26] INFO [2022-01-10 12:53:28,338] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:28,339] jepsen worker 8 - jepsen.util 8 :ok :read [2 22] INFO [2022-01-10 12:53:28,339] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:28,340] jepsen worker 8 - jepsen.util 8 :ok :read [2 22] INFO [2022-01-10 12:53:28,341] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:28,341] jepsen worker 37 - jepsen.util 127 :ok :write [12 17] INFO [2022-01-10 12:53:28,341] jepsen worker 8 - jepsen.util 8 :ok :read [2 22] INFO [2022-01-10 12:53:28,342] jepsen worker 37 - jepsen.util 127 :invoke :write [12 24] INFO [2022-01-10 12:53:28,342] jepsen worker 8 - jepsen.util 8 :invoke :read [2 nil] INFO [2022-01-10 12:53:28,342] jepsen worker 68 - jepsen.util 68 :ok :write [22 12] INFO [2022-01-10 12:53:28,343] jepsen worker 8 - jepsen.util 8 :ok :read [2 22] INFO [2022-01-10 12:53:28,343] jepsen worker 68 - jepsen.util 68 :invoke :write [22 18] INFO [2022-01-10 12:53:28,344] jepsen worker 8 - jepsen.util 8 :invoke :write [2 4] INFO [2022-01-10 12:53:28,350] jepsen worker 33 - jepsen.util 123 :ok :write [11 48] INFO [2022-01-10 12:53:28,351] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:28,351] jepsen worker 3 - jepsen.util 3 :ok :write [1 26] INFO [2022-01-10 12:53:28,351] jepsen worker 33 - jepsen.util 123 :ok :read [11 48] INFO [2022-01-10 12:53:28,352] jepsen worker 3 - jepsen.util 3 :invoke :write [1 33] INFO [2022-01-10 12:53:28,352] jepsen worker 33 - jepsen.util 123 :invoke :write [11 18] INFO [2022-01-10 12:53:28,358] jepsen worker 37 - jepsen.util 127 :ok :write [12 24] INFO [2022-01-10 12:53:28,359] jepsen worker 41 - jepsen.util 41 :ok :write [13 30] INFO [2022-01-10 12:53:28,359] jepsen worker 37 - jepsen.util 127 :invoke :write [12 9] INFO [2022-01-10 12:53:28,360] jepsen worker 41 - jepsen.util 41 :invoke :write [13 6] INFO [2022-01-10 12:53:28,367] jepsen worker 86 - jepsen.util 176 :ok :write [28 45] INFO [2022-01-10 12:53:28,367] jepsen worker 88 - jepsen.util 178 :ok :write [29 30] INFO [2022-01-10 12:53:28,368] jepsen worker 86 - jepsen.util 176 :invoke :write [28 41] INFO [2022-01-10 12:53:28,368] jepsen worker 88 - jepsen.util 178 :invoke :write [29 1] INFO [2022-01-10 12:53:28,371] jepsen worker 2 - jepsen.util 92 :ok :write [0 38] INFO [2022-01-10 12:53:28,372] jepsen worker 2 - jepsen.util 92 :invoke :write [0 20] INFO [2022-01-10 12:53:28,375] jepsen worker 3 - jepsen.util 3 :ok :write [1 33] INFO [2022-01-10 12:53:28,376] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:28,377] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:28,377] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:28,378] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:28,378] jepsen worker 3 - jepsen.util 3 :invoke :write [1 25] INFO [2022-01-10 12:53:28,382] jepsen worker 37 - jepsen.util 127 :ok :write [12 9] INFO [2022-01-10 12:53:28,383] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:28,384] jepsen worker 37 - jepsen.util 127 :ok :read [12 9] INFO [2022-01-10 12:53:28,385] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:28,385] jepsen worker 37 - jepsen.util 127 :ok :read [12 9] INFO [2022-01-10 12:53:28,387] jepsen worker 37 - jepsen.util 127 :invoke :write [12 23] INFO [2022-01-10 12:53:28,390] jepsen worker 86 - jepsen.util 176 :ok :write [28 41] INFO [2022-01-10 12:53:28,390] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:28,391] jepsen worker 86 - jepsen.util 176 :ok :read [28 41] INFO [2022-01-10 12:53:28,392] jepsen worker 86 - jepsen.util 176 :invoke :write [28 23] INFO [2022-01-10 12:53:28,398] jepsen worker 3 - jepsen.util 3 :ok :write [1 25] INFO [2022-01-10 12:53:28,399] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:28,400] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:28,401] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:28,402] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:28,402] jepsen worker 3 - jepsen.util 3 :invoke :write [1 32] INFO [2022-01-10 12:53:28,404] jepsen worker 37 - jepsen.util 127 :ok :write [12 23] INFO [2022-01-10 12:53:28,405] jepsen worker 37 - jepsen.util 127 :invoke :write [12 28] INFO [2022-01-10 12:53:28,410] jepsen worker 86 - jepsen.util 176 :ok :write [28 23] INFO [2022-01-10 12:53:28,410] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:28,411] jepsen worker 86 - jepsen.util 176 :ok :read [28 23] INFO [2022-01-10 12:53:28,412] jepsen worker 86 - jepsen.util 176 :invoke :write [28 5] INFO [2022-01-10 12:53:28,412] jepsen worker 41 - jepsen.util 41 :ok :write [13 6] INFO [2022-01-10 12:53:28,413] jepsen worker 41 - jepsen.util 41 :invoke :write [13 42] INFO [2022-01-10 12:53:28,415] jepsen worker 3 - jepsen.util 3 :ok :write [1 32] INFO [2022-01-10 12:53:28,416] jepsen worker 3 - jepsen.util 3 :invoke :write [1 45] INFO [2022-01-10 12:53:28,420] jepsen worker 88 - jepsen.util 178 :ok :write [29 1] INFO [2022-01-10 12:53:28,420] jepsen worker 88 - jepsen.util 178 :invoke :write [29 32] INFO [2022-01-10 12:53:28,422] jepsen worker 37 - jepsen.util 127 :ok :write [12 28] INFO [2022-01-10 12:53:28,423] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:28,424] jepsen worker 37 - jepsen.util 127 :ok :read [12 28] INFO [2022-01-10 12:53:28,425] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:28,426] jepsen worker 2 - jepsen.util 92 :ok :write [0 20] INFO [2022-01-10 12:53:28,426] jepsen worker 37 - jepsen.util 127 :ok :read [12 28] INFO [2022-01-10 12:53:28,426] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:28,427] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:28,427] jepsen worker 2 - jepsen.util 92 :ok :read [0 20] INFO [2022-01-10 12:53:28,428] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:28,428] jepsen worker 37 - jepsen.util 127 :ok :read [12 28] INFO [2022-01-10 12:53:28,428] jepsen worker 61 - jepsen.util 61 :ok :write [20 7] INFO [2022-01-10 12:53:28,429] jepsen worker 37 - jepsen.util 127 :invoke :write [12 34] INFO [2022-01-10 12:53:28,429] jepsen worker 2 - jepsen.util 92 :ok :read [0 20] INFO [2022-01-10 12:53:28,429] jepsen worker 2 - jepsen.util 92 :invoke :write [0 2] INFO [2022-01-10 12:53:28,430] jepsen worker 61 - jepsen.util 61 :invoke :write [20 25] INFO [2022-01-10 12:53:28,430] jepsen worker 86 - jepsen.util 176 :ok :write [28 5] INFO [2022-01-10 12:53:28,431] jepsen worker 86 - jepsen.util 176 :invoke :write [28 8] INFO [2022-01-10 12:53:28,432] jepsen worker 13 - jepsen.util 13 :ok :write [4 28] INFO [2022-01-10 12:53:28,432] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,433] jepsen worker 13 - jepsen.util 13 :ok :read [4 28] INFO [2022-01-10 12:53:28,434] jepsen worker 13 - jepsen.util 13 :invoke :write [4 38] INFO [2022-01-10 12:53:28,435] jepsen worker 3 - jepsen.util 3 :ok :write [1 45] INFO [2022-01-10 12:53:28,435] jepsen worker 41 - jepsen.util 41 :ok :write [13 42] INFO [2022-01-10 12:53:28,435] jepsen worker 43 - jepsen.util 43 :ok :write [14 5] INFO [2022-01-10 12:53:28,436] jepsen worker 3 - jepsen.util 3 :invoke :write [1 42] INFO [2022-01-10 12:53:28,436] jepsen worker 79 - jepsen.util 259 :ok :write [26 15] INFO [2022-01-10 12:53:28,436] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:28,437] jepsen worker 43 - jepsen.util 43 :invoke :write [14 36] INFO [2022-01-10 12:53:28,437] jepsen worker 41 - jepsen.util 41 :ok :read [13 42] INFO [2022-01-10 12:53:28,437] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,438] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:28,439] jepsen worker 41 - jepsen.util 41 :ok :read [13 42] INFO [2022-01-10 12:53:28,439] jepsen worker 41 - jepsen.util 41 :invoke :write [13 24] INFO [2022-01-10 12:53:28,441] jepsen worker 37 - jepsen.util 127 :ok :write [12 34] INFO [2022-01-10 12:53:28,441] jepsen worker 37 - jepsen.util 127 :invoke :write [12 10] INFO [2022-01-10 12:53:28,442] jepsen worker 22 - jepsen.util 112 :ok :write [7 18] INFO [2022-01-10 12:53:28,443] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,443] jepsen worker 53 - jepsen.util 143 :ok :write [17 40] INFO [2022-01-10 12:53:28,443] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:28,444] jepsen worker 88 - jepsen.util 178 :ok :write [29 32] INFO [2022-01-10 12:53:28,444] jepsen worker 53 - jepsen.util 143 :ok :read [17 40] INFO [2022-01-10 12:53:28,444] jepsen worker 79 - jepsen.util 259 :ok :read [26 15] INFO [2022-01-10 12:53:28,445] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:28,445] jepsen worker 22 - jepsen.util 112 :ok :read [7 18] INFO [2022-01-10 12:53:28,445] jepsen worker 53 - jepsen.util 143 :invoke :write [17 39] INFO [2022-01-10 12:53:28,446] jepsen worker 88 - jepsen.util 178 :ok :read [29 32] INFO [2022-01-10 12:53:28,447] jepsen worker 22 - jepsen.util 112 :invoke :write [7 37] INFO [2022-01-10 12:53:28,447] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:28,448] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,448] jepsen worker 88 - jepsen.util 178 :ok :read [29 32] INFO [2022-01-10 12:53:28,448] jepsen worker 70 - jepsen.util 160 :ok :write [23 44] INFO [2022-01-10 12:53:28,449] jepsen worker 79 - jepsen.util 259 :ok :read [26 15] INFO [2022-01-10 12:53:28,449] jepsen worker 75 - jepsen.util 165 :ok :write [25 17] INFO [2022-01-10 12:53:28,449] jepsen worker 88 - jepsen.util 178 :invoke :write [29 2] INFO [2022-01-10 12:53:28,450] jepsen worker 70 - jepsen.util 160 :invoke :write [23 24] INFO [2022-01-10 12:53:28,450] jepsen worker 75 - jepsen.util 165 :invoke :write [25 38] INFO [2022-01-10 12:53:28,451] jepsen worker 79 - jepsen.util 259 :invoke :write [26 32] INFO [2022-01-10 12:53:28,451] jepsen worker 2 - jepsen.util 92 :ok :write [0 2] INFO [2022-01-10 12:53:28,451] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:28,452] jepsen worker 2 - jepsen.util 92 :ok :read [0 2] INFO [2022-01-10 12:53:28,453] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:28,454] jepsen worker 2 - jepsen.util 92 :ok :read [0 2] INFO [2022-01-10 12:53:28,454] jepsen worker 2 - jepsen.util 92 :invoke :write [0 28] INFO [2022-01-10 12:53:28,457] jepsen worker 15 - jepsen.util 15 :ok :write [5 12] INFO [2022-01-10 12:53:28,457] jepsen worker 86 - jepsen.util 176 :ok :write [28 8] INFO [2022-01-10 12:53:28,458] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:28,459] jepsen worker 86 - jepsen.util 176 :invoke :write [28 2] INFO [2022-01-10 12:53:28,459] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:28,459] jepsen worker 15 - jepsen.util 15 :invoke :write [5 23] INFO [2022-01-10 12:53:28,462] jepsen worker 48 - jepsen.util 228 :ok :write [16 32] INFO [2022-01-10 12:53:28,462] jepsen worker 13 - jepsen.util 13 :ok :write [4 38] INFO [2022-01-10 12:53:28,463] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,463] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:28,464] jepsen worker 13 - jepsen.util 13 :ok :read [4 38] INFO [2022-01-10 12:53:28,464] jepsen worker 41 - jepsen.util 41 :ok :write [13 24] INFO [2022-01-10 12:53:28,464] jepsen worker 48 - jepsen.util 228 :ok :read [16 32] INFO [2022-01-10 12:53:28,464] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,465] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:28,465] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:28,466] jepsen worker 41 - jepsen.util 41 :ok :read [13 24] INFO [2022-01-10 12:53:28,466] jepsen worker 13 - jepsen.util 13 :ok :read [4 38] INFO [2022-01-10 12:53:28,466] jepsen worker 48 - jepsen.util 228 :ok :read [16 32] INFO [2022-01-10 12:53:28,467] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:28,467] jepsen worker 13 - jepsen.util 13 :invoke :write [4 32] INFO [2022-01-10 12:53:28,468] jepsen worker 41 - jepsen.util 41 :ok :read [13 24] INFO [2022-01-10 12:53:28,468] jepsen worker 48 - jepsen.util 228 :invoke :write [16 15] INFO [2022-01-10 12:53:28,468] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:28,469] jepsen worker 41 - jepsen.util 41 :ok :read [13 24] INFO [2022-01-10 12:53:28,470] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:28,470] jepsen worker 3 - jepsen.util 3 :ok :write [1 42] INFO [2022-01-10 12:53:28,471] jepsen worker 3 - jepsen.util 3 :invoke :write [1 20] INFO [2022-01-10 12:53:28,471] jepsen worker 41 - jepsen.util 41 :ok :read [13 24] INFO [2022-01-10 12:53:28,472] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:28,475] jepsen worker 7 - jepsen.util 7 :ok :write [2 47] INFO [2022-01-10 12:53:28,476] jepsen worker 7 - jepsen.util 7 :invoke :write [2 7] INFO [2022-01-10 12:53:28,476] jepsen worker 88 - jepsen.util 178 :ok :write [29 2] INFO [2022-01-10 12:53:28,476] jepsen worker 79 - jepsen.util 259 :ok :write [26 32] INFO [2022-01-10 12:53:28,477] jepsen worker 88 - jepsen.util 178 :invoke :write [29 35] INFO [2022-01-10 12:53:28,478] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,479] jepsen worker 79 - jepsen.util 259 :ok :read [26 32] INFO [2022-01-10 12:53:28,480] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,481] jepsen worker 79 - jepsen.util 259 :ok :read [26 32] INFO [2022-01-10 12:53:28,481] jepsen worker 79 - jepsen.util 259 :invoke :write [26 6] INFO [2022-01-10 12:53:28,484] jepsen worker 41 - jepsen.util 41 :ok :read [13 24] INFO [2022-01-10 12:53:28,484] jepsen worker 81 - jepsen.util 171 :ok :write [27 28] INFO [2022-01-10 12:53:28,485] jepsen worker 41 - jepsen.util 41 :invoke :write [13 8] INFO [2022-01-10 12:53:28,485] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:28,485] jepsen worker 68 - jepsen.util 68 :ok :write [22 18] INFO [2022-01-10 12:53:28,486] jepsen worker 81 - jepsen.util 171 :ok :read [27 28] INFO [2022-01-10 12:53:28,486] jepsen worker 68 - jepsen.util 68 :invoke :write [22 32] INFO [2022-01-10 12:53:28,486] jepsen worker 37 - jepsen.util 127 :ok :write [12 10] INFO [2022-01-10 12:53:28,487] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:28,487] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:28,488] jepsen worker 81 - jepsen.util 171 :ok :read [27 28] INFO [2022-01-10 12:53:28,489] jepsen worker 81 - jepsen.util 171 :invoke :write [27 12] INFO [2022-01-10 12:53:28,489] jepsen worker 37 - jepsen.util 127 :ok :read [12 10] INFO [2022-01-10 12:53:28,489] jepsen worker 33 - jepsen.util 123 :ok :write [11 18] INFO [2022-01-10 12:53:28,490] jepsen worker 37 - jepsen.util 127 :invoke :write [12 8] INFO [2022-01-10 12:53:28,490] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:28,491] jepsen worker 33 - jepsen.util 123 :ok :read [11 18] INFO [2022-01-10 12:53:28,492] jepsen worker 33 - jepsen.util 123 :invoke :write [11 28] INFO [2022-01-10 12:53:28,494] jepsen worker 53 - jepsen.util 143 :ok :write [17 39] INFO [2022-01-10 12:53:28,495] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:28,496] jepsen worker 53 - jepsen.util 143 :ok :read [17 39] INFO [2022-01-10 12:53:28,497] jepsen worker 53 - jepsen.util 143 :invoke :write [17 31] INFO [2022-01-10 12:53:28,500] jepsen worker 70 - jepsen.util 160 :ok :write [23 24] INFO [2022-01-10 12:53:28,501] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:28,501] jepsen worker 70 - jepsen.util 160 :ok :read [23 24] INFO [2022-01-10 12:53:28,502] jepsen worker 70 - jepsen.util 160 :invoke :write [23 40] INFO [2022-01-10 12:53:28,508] jepsen worker 84 - jepsen.util 84 :ok :write [28 3] INFO [2022-01-10 12:53:28,508] jepsen worker 84 - jepsen.util 84 :invoke :write [28 19] INFO [2022-01-10 12:53:28,512] jepsen worker 13 - jepsen.util 13 :ok :write [4 32] INFO [2022-01-10 12:53:28,513] jepsen worker 13 - jepsen.util 13 :invoke :write [4 39] INFO [2022-01-10 12:53:28,522] jepsen worker 79 - jepsen.util 259 :ok :write [26 6] INFO [2022-01-10 12:53:28,522] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,523] jepsen worker 79 - jepsen.util 259 :ok :read [26 6] INFO [2022-01-10 12:53:28,524] jepsen worker 79 - jepsen.util 259 :invoke :write [26 19] INFO [2022-01-10 12:53:28,568] jepsen worker 16 - jepsen.util 106 :info :write [5 27] :RequestTimeoutException INFO [2022-01-10 12:53:28,569] jepsen worker 16 - jepsen.util 196 :invoke :read [5 nil] INFO [2022-01-10 12:53:28,570] jepsen worker 16 - jepsen.util 196 :ok :read [5 12] INFO [2022-01-10 12:53:28,570] jepsen worker 16 - jepsen.util 196 :invoke :write [5 40] INFO [2022-01-10 12:53:28,586] jepsen worker 49 - jepsen.util 139 :info :write [16 42] :RequestTimeoutException INFO [2022-01-10 12:53:28,587] jepsen worker 49 - jepsen.util 229 :invoke :write [16 33] INFO [2022-01-10 12:53:28,596] jepsen worker 2 - jepsen.util 92 :ok :write [0 28] INFO [2022-01-10 12:53:28,597] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:28,598] jepsen worker 2 - jepsen.util 92 :ok :read [0 28] INFO [2022-01-10 12:53:28,599] jepsen worker 2 - jepsen.util 92 :invoke :write [0 48] INFO [2022-01-10 12:53:28,602] jepsen worker 3 - jepsen.util 3 :ok :write [1 20] INFO [2022-01-10 12:53:28,603] jepsen worker 3 - jepsen.util 3 :invoke :write [1 30] INFO [2022-01-10 12:53:28,605] jepsen worker 61 - jepsen.util 61 :ok :write [20 25] INFO [2022-01-10 12:53:28,606] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:28,607] jepsen worker 61 - jepsen.util 61 :ok :read [20 25] INFO [2022-01-10 12:53:28,607] jepsen worker 61 - jepsen.util 61 :invoke :write [20 19] INFO [2022-01-10 12:53:28,608] jepsen worker 37 - jepsen.util 127 :ok :write [12 8] INFO [2022-01-10 12:53:28,609] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:28,610] jepsen worker 37 - jepsen.util 127 :ok :read [12 8] INFO [2022-01-10 12:53:28,610] jepsen worker 37 - jepsen.util 127 :invoke :write [12 21] INFO [2022-01-10 12:53:28,612] jepsen worker 43 - jepsen.util 43 :ok :write [14 36] INFO [2022-01-10 12:53:28,613] jepsen worker 84 - jepsen.util 84 :ok :write [28 19] INFO [2022-01-10 12:53:28,613] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:28,613] jepsen worker 84 - jepsen.util 84 :invoke :write [28 29] INFO [2022-01-10 12:53:28,614] jepsen worker 43 - jepsen.util 43 :ok :read [14 36] INFO [2022-01-10 12:53:28,614] jepsen worker 53 - jepsen.util 143 :ok :write [17 31] INFO [2022-01-10 12:53:28,614] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:28,615] jepsen worker 53 - jepsen.util 143 :invoke :write [17 31] INFO [2022-01-10 12:53:28,615] jepsen worker 43 - jepsen.util 43 :ok :read [14 36] INFO [2022-01-10 12:53:28,615] jepsen worker 41 - jepsen.util 41 :ok :write [13 8] INFO [2022-01-10 12:53:28,616] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:28,616] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:28,616] jepsen worker 41 - jepsen.util 41 :ok :read [13 8] INFO [2022-01-10 12:53:28,617] jepsen worker 41 - jepsen.util 41 :invoke :write [13 18] INFO [2022-01-10 12:53:28,617] jepsen worker 43 - jepsen.util 43 :ok :read [14 36] INFO [2022-01-10 12:53:28,617] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:28,618] jepsen worker 43 - jepsen.util 43 :ok :read [14 36] INFO [2022-01-10 12:53:28,619] jepsen worker 43 - jepsen.util 43 :invoke :write [14 0] INFO [2022-01-10 12:53:28,619] jepsen worker 70 - jepsen.util 160 :ok :write [23 40] INFO [2022-01-10 12:53:28,619] jepsen worker 22 - jepsen.util 112 :ok :write [7 37] INFO [2022-01-10 12:53:28,620] jepsen worker 70 - jepsen.util 160 :invoke :write [23 28] INFO [2022-01-10 12:53:28,620] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,621] jepsen worker 22 - jepsen.util 112 :ok :read [7 37] INFO [2022-01-10 12:53:28,621] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,622] jepsen worker 22 - jepsen.util 112 :ok :read [7 37] INFO [2022-01-10 12:53:28,622] jepsen worker 22 - jepsen.util 112 :invoke :write [7 8] INFO [2022-01-10 12:53:28,627] jepsen worker 75 - jepsen.util 165 :ok :write [25 38] INFO [2022-01-10 12:53:28,627] jepsen worker 75 - jepsen.util 165 :invoke :write [25 40] INFO [2022-01-10 12:53:28,634] jepsen worker 15 - jepsen.util 15 :ok :write [5 23] INFO [2022-01-10 12:53:28,634] jepsen worker 15 - jepsen.util 15 :invoke :write [5 31] INFO [2022-01-10 12:53:28,638] jepsen worker 50 - jepsen.util 50 :ok :write [16 40] INFO [2022-01-10 12:53:28,639] jepsen worker 50 - jepsen.util 50 :invoke :write [16 32] INFO [2022-01-10 12:53:28,649] jepsen worker 81 - jepsen.util 171 :ok :write [27 12] INFO [2022-01-10 12:53:28,650] jepsen worker 13 - jepsen.util 13 :ok :write [4 39] INFO [2022-01-10 12:53:28,650] jepsen worker 81 - jepsen.util 171 :invoke :write [27 9] INFO [2022-01-10 12:53:28,651] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,652] jepsen worker 13 - jepsen.util 13 :ok :read [4 39] INFO [2022-01-10 12:53:28,652] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,654] jepsen worker 13 - jepsen.util 13 :ok :read [4 39] INFO [2022-01-10 12:53:28,654] jepsen worker 13 - jepsen.util 13 :invoke :write [4 5] INFO [2022-01-10 12:53:28,656] jepsen worker 79 - jepsen.util 259 :ok :write [26 19] INFO [2022-01-10 12:53:28,657] jepsen worker 79 - jepsen.util 259 :invoke :write [26 26] INFO [2022-01-10 12:53:28,664] jepsen worker 61 - jepsen.util 61 :ok :write [20 19] INFO [2022-01-10 12:53:28,665] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:28,666] jepsen worker 3 - jepsen.util 3 :ok :write [1 30] INFO [2022-01-10 12:53:28,666] jepsen worker 61 - jepsen.util 61 :ok :read [20 19] INFO [2022-01-10 12:53:28,666] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:28,667] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:28,667] jepsen worker 3 - jepsen.util 3 :ok :read [1 30] INFO [2022-01-10 12:53:28,668] jepsen worker 61 - jepsen.util 61 :ok :read [20 19] INFO [2022-01-10 12:53:28,668] jepsen worker 3 - jepsen.util 3 :invoke :write [1 19] INFO [2022-01-10 12:53:28,668] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:28,669] jepsen worker 61 - jepsen.util 61 :ok :read [20 19] INFO [2022-01-10 12:53:28,670] jepsen worker 43 - jepsen.util 43 :ok :write [14 0] INFO [2022-01-10 12:53:28,670] jepsen worker 61 - jepsen.util 61 :invoke :write [20 22] INFO [2022-01-10 12:53:28,671] jepsen worker 43 - jepsen.util 43 :invoke :write [14 6] INFO [2022-01-10 12:53:28,672] jepsen worker 37 - jepsen.util 127 :ok :write [12 21] INFO [2022-01-10 12:53:28,672] jepsen worker 45 - jepsen.util 225 :fail :read [15 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:28,673] jepsen worker 37 - jepsen.util 127 :invoke :write [12 32] INFO [2022-01-10 12:53:28,674] jepsen worker 45 - jepsen.util 225 :invoke :read [15 nil] INFO [2022-01-10 12:53:28,679] jepsen worker 53 - jepsen.util 143 :ok :write [17 31] INFO [2022-01-10 12:53:28,679] jepsen worker 22 - jepsen.util 112 :ok :write [7 8] INFO [2022-01-10 12:53:28,679] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:28,680] jepsen worker 22 - jepsen.util 112 :invoke :write [7 28] INFO [2022-01-10 12:53:28,681] jepsen worker 53 - jepsen.util 143 :ok :read [17 31] INFO [2022-01-10 12:53:28,681] jepsen worker 53 - jepsen.util 143 :invoke :write [17 22] INFO [2022-01-10 12:53:28,685] jepsen worker 75 - jepsen.util 165 :ok :write [25 40] INFO [2022-01-10 12:53:28,686] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,687] jepsen worker 75 - jepsen.util 165 :ok :read [25 40] INFO [2022-01-10 12:53:28,688] jepsen worker 75 - jepsen.util 165 :invoke :write [25 23] INFO [2022-01-10 12:53:28,691] jepsen worker 70 - jepsen.util 160 :ok :write [23 28] INFO [2022-01-10 12:53:28,691] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:28,692] jepsen worker 70 - jepsen.util 160 :ok :read [23 28] INFO [2022-01-10 12:53:28,693] jepsen worker 70 - jepsen.util 160 :invoke :write [23 17] INFO [2022-01-10 12:53:28,698] jepsen worker 13 - jepsen.util 13 :ok :write [4 5] INFO [2022-01-10 12:53:28,699] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,699] jepsen worker 13 - jepsen.util 13 :ok :read [4 5] INFO [2022-01-10 12:53:28,700] jepsen worker 13 - jepsen.util 13 :invoke :write [4 22] INFO [2022-01-10 12:53:28,703] jepsen worker 88 - jepsen.util 178 :ok :write [29 35] INFO [2022-01-10 12:53:28,703] jepsen worker 61 - jepsen.util 61 :ok :write [20 22] INFO [2022-01-10 12:53:28,703] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:28,704] jepsen worker 61 - jepsen.util 61 :invoke :write [20 48] INFO [2022-01-10 12:53:28,704] jepsen worker 84 - jepsen.util 84 :ok :write [28 29] INFO [2022-01-10 12:53:28,704] jepsen worker 88 - jepsen.util 178 :ok :read [29 35] INFO [2022-01-10 12:53:28,704] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:28,705] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:28,705] jepsen worker 84 - jepsen.util 84 :ok :read [28 29] INFO [2022-01-10 12:53:28,706] jepsen worker 88 - jepsen.util 178 :ok :read [29 35] INFO [2022-01-10 12:53:28,706] jepsen worker 84 - jepsen.util 84 :invoke :write [28 36] INFO [2022-01-10 12:53:28,706] jepsen worker 79 - jepsen.util 259 :ok :write [26 26] INFO [2022-01-10 12:53:28,706] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:28,707] jepsen worker 79 - jepsen.util 259 :invoke :write [26 2] INFO [2022-01-10 12:53:28,707] jepsen worker 88 - jepsen.util 178 :ok :read [29 35] INFO [2022-01-10 12:53:28,707] jepsen worker 88 - jepsen.util 178 :invoke :write [29 45] INFO [2022-01-10 12:53:28,711] jepsen worker 43 - jepsen.util 43 :ok :write [14 6] INFO [2022-01-10 12:53:28,711] jepsen worker 43 - jepsen.util 43 :invoke :write [14 9] INFO [2022-01-10 12:53:28,720] jepsen worker 22 - jepsen.util 112 :ok :write [7 28] INFO [2022-01-10 12:53:28,721] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,721] jepsen worker 22 - jepsen.util 112 :ok :read [7 28] INFO [2022-01-10 12:53:28,722] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,723] jepsen worker 22 - jepsen.util 112 :ok :read [7 28] INFO [2022-01-10 12:53:28,723] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,724] jepsen worker 22 - jepsen.util 112 :ok :read [7 28] INFO [2022-01-10 12:53:28,724] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,725] jepsen worker 22 - jepsen.util 112 :ok :read [7 28] INFO [2022-01-10 12:53:28,725] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,726] jepsen worker 22 - jepsen.util 112 :ok :read [7 28] INFO [2022-01-10 12:53:28,726] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,727] jepsen worker 75 - jepsen.util 165 :ok :write [25 23] INFO [2022-01-10 12:53:28,727] jepsen worker 22 - jepsen.util 112 :ok :read [7 28] INFO [2022-01-10 12:53:28,727] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,728] jepsen worker 53 - jepsen.util 143 :ok :write [17 22] INFO [2022-01-10 12:53:28,728] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,728] jepsen worker 75 - jepsen.util 165 :ok :read [25 23] INFO [2022-01-10 12:53:28,729] jepsen worker 53 - jepsen.util 143 :invoke :write [17 23] INFO [2022-01-10 12:53:28,729] jepsen worker 22 - jepsen.util 112 :ok :read [7 28] INFO [2022-01-10 12:53:28,729] jepsen worker 75 - jepsen.util 165 :invoke :write [25 18] INFO [2022-01-10 12:53:28,730] jepsen worker 22 - jepsen.util 112 :invoke :write [7 10] INFO [2022-01-10 12:53:28,734] jepsen worker 61 - jepsen.util 61 :ok :write [20 48] INFO [2022-01-10 12:53:28,735] jepsen worker 61 - jepsen.util 61 :invoke :write [20 32] INFO [2022-01-10 12:53:28,735] jepsen worker 70 - jepsen.util 160 :ok :write [23 17] INFO [2022-01-10 12:53:28,736] jepsen worker 70 - jepsen.util 160 :invoke :write [23 26] INFO [2022-01-10 12:53:28,742] jepsen worker 43 - jepsen.util 43 :ok :write [14 9] INFO [2022-01-10 12:53:28,742] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:28,743] jepsen worker 43 - jepsen.util 43 :ok :read [14 9] INFO [2022-01-10 12:53:28,744] jepsen worker 43 - jepsen.util 43 :invoke :write [14 46] INFO [2022-01-10 12:53:28,744] jepsen worker 13 - jepsen.util 13 :ok :write [4 22] INFO [2022-01-10 12:53:28,745] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,746] jepsen worker 13 - jepsen.util 13 :ok :read [4 22] INFO [2022-01-10 12:53:28,747] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,748] jepsen worker 13 - jepsen.util 13 :ok :read [4 22] INFO [2022-01-10 12:53:28,748] jepsen worker 13 - jepsen.util 13 :invoke :write [4 49] INFO [2022-01-10 12:53:28,750] jepsen worker 22 - jepsen.util 112 :ok :write [7 10] INFO [2022-01-10 12:53:28,750] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,751] jepsen worker 22 - jepsen.util 112 :ok :read [7 10] INFO [2022-01-10 12:53:28,752] jepsen worker 22 - jepsen.util 112 :invoke :write [7 35] INFO [2022-01-10 12:53:28,752] jepsen worker 75 - jepsen.util 165 :ok :write [25 18] INFO [2022-01-10 12:53:28,753] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,753] jepsen worker 75 - jepsen.util 165 :ok :read [25 18] INFO [2022-01-10 12:53:28,754] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,755] jepsen worker 75 - jepsen.util 165 :ok :read [25 18] INFO [2022-01-10 12:53:28,755] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,756] jepsen worker 75 - jepsen.util 165 :ok :read [25 18] INFO [2022-01-10 12:53:28,756] jepsen worker 75 - jepsen.util 165 :invoke :write [25 6] INFO [2022-01-10 12:53:28,757] jepsen worker 79 - jepsen.util 259 :ok :write [26 2] INFO [2022-01-10 12:53:28,758] jepsen worker 79 - jepsen.util 259 :invoke :write [26 21] INFO [2022-01-10 12:53:28,761] jepsen worker 61 - jepsen.util 61 :ok :write [20 32] INFO [2022-01-10 12:53:28,761] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:28,762] jepsen worker 61 - jepsen.util 61 :ok :read [20 32] INFO [2022-01-10 12:53:28,763] jepsen worker 53 - jepsen.util 143 :ok :write [17 23] INFO [2022-01-10 12:53:28,763] jepsen worker 61 - jepsen.util 61 :invoke :write [20 2] INFO [2022-01-10 12:53:28,763] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:28,764] jepsen worker 53 - jepsen.util 143 :ok :read [17 23] INFO [2022-01-10 12:53:28,765] jepsen worker 53 - jepsen.util 143 :invoke :write [17 38] INFO [2022-01-10 12:53:28,768] jepsen worker 42 - jepsen.util 132 :ok :write [14 4] INFO [2022-01-10 12:53:28,768] jepsen worker 42 - jepsen.util 132 :invoke :write [14 43] INFO [2022-01-10 12:53:28,771] jepsen worker 70 - jepsen.util 160 :ok :write [23 26] INFO [2022-01-10 12:53:28,772] jepsen worker 70 - jepsen.util 160 :invoke :write [23 49] INFO [2022-01-10 12:53:28,775] jepsen worker 22 - jepsen.util 112 :ok :write [7 35] INFO [2022-01-10 12:53:28,776] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,777] jepsen worker 22 - jepsen.util 112 :ok :read [7 35] INFO [2022-01-10 12:53:28,777] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,778] jepsen worker 22 - jepsen.util 112 :ok :read [7 35] INFO [2022-01-10 12:53:28,778] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,779] jepsen worker 22 - jepsen.util 112 :ok :read [7 35] INFO [2022-01-10 12:53:28,780] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,780] jepsen worker 13 - jepsen.util 13 :ok :write [4 49] INFO [2022-01-10 12:53:28,780] jepsen worker 22 - jepsen.util 112 :ok :read [7 35] INFO [2022-01-10 12:53:28,781] jepsen worker 13 - jepsen.util 13 :invoke :write [4 38] INFO [2022-01-10 12:53:28,781] jepsen worker 22 - jepsen.util 112 :invoke :write [7 31] INFO [2022-01-10 12:53:28,784] jepsen worker 75 - jepsen.util 165 :ok :write [25 6] INFO [2022-01-10 12:53:28,785] jepsen worker 75 - jepsen.util 165 :invoke :write [25 46] INFO [2022-01-10 12:53:28,789] jepsen worker 79 - jepsen.util 259 :ok :write [26 21] INFO [2022-01-10 12:53:28,790] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,791] jepsen worker 79 - jepsen.util 259 :ok :read [26 21] INFO [2022-01-10 12:53:28,791] jepsen worker 79 - jepsen.util 259 :invoke :write [26 14] INFO [2022-01-10 12:53:28,793] jepsen worker 61 - jepsen.util 61 :ok :write [20 2] INFO [2022-01-10 12:53:28,793] jepsen worker 61 - jepsen.util 61 :invoke :write [20 3] INFO [2022-01-10 12:53:28,797] jepsen worker 53 - jepsen.util 143 :ok :write [17 38] INFO [2022-01-10 12:53:28,797] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:28,798] jepsen worker 53 - jepsen.util 143 :ok :read [17 38] INFO [2022-01-10 12:53:28,799] jepsen worker 53 - jepsen.util 143 :invoke :write [17 21] INFO [2022-01-10 12:53:28,802] jepsen worker 42 - jepsen.util 132 :ok :write [14 43] INFO [2022-01-10 12:53:28,803] jepsen worker 42 - jepsen.util 132 :invoke :write [14 6] INFO [2022-01-10 12:53:28,805] jepsen worker 70 - jepsen.util 160 :ok :write [23 49] INFO [2022-01-10 12:53:28,806] jepsen worker 70 - jepsen.util 160 :invoke :write [23 28] INFO [2022-01-10 12:53:28,808] jepsen worker 22 - jepsen.util 112 :ok :write [7 31] INFO [2022-01-10 12:53:28,809] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,810] jepsen worker 22 - jepsen.util 112 :ok :read [7 31] INFO [2022-01-10 12:53:28,810] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,811] jepsen worker 22 - jepsen.util 112 :ok :read [7 31] INFO [2022-01-10 12:53:28,811] jepsen worker 13 - jepsen.util 13 :ok :write [4 38] INFO [2022-01-10 12:53:28,811] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,812] jepsen worker 13 - jepsen.util 13 :invoke :write [4 33] INFO [2022-01-10 12:53:28,812] jepsen worker 22 - jepsen.util 112 :ok :read [7 31] INFO [2022-01-10 12:53:28,813] jepsen worker 22 - jepsen.util 112 :invoke :write [7 18] INFO [2022-01-10 12:53:28,817] jepsen worker 79 - jepsen.util 259 :ok :write [26 14] INFO [2022-01-10 12:53:28,818] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,819] jepsen worker 79 - jepsen.util 259 :ok :read [26 14] INFO [2022-01-10 12:53:28,819] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,820] jepsen worker 79 - jepsen.util 259 :ok :read [26 14] INFO [2022-01-10 12:53:28,821] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,821] jepsen worker 79 - jepsen.util 259 :ok :read [26 14] INFO [2022-01-10 12:53:28,822] jepsen worker 79 - jepsen.util 259 :invoke :write [26 45] INFO [2022-01-10 12:53:28,824] jepsen worker 53 - jepsen.util 143 :ok :write [17 21] INFO [2022-01-10 12:53:28,825] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:28,826] jepsen worker 53 - jepsen.util 143 :ok :read [17 21] INFO [2022-01-10 12:53:28,827] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:28,828] jepsen worker 53 - jepsen.util 143 :ok :read [17 21] INFO [2022-01-10 12:53:28,828] jepsen worker 53 - jepsen.util 143 :invoke :write [17 23] INFO [2022-01-10 12:53:28,854] jepsen worker 7 - jepsen.util 7 :ok :write [2 7] INFO [2022-01-10 12:53:28,855] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:28,856] jepsen worker 7 - jepsen.util 7 :ok :read [2 7] INFO [2022-01-10 12:53:28,856] jepsen worker 3 - jepsen.util 3 :ok :write [1 19] INFO [2022-01-10 12:53:28,857] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:28,857] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:28,858] jepsen worker 7 - jepsen.util 7 :ok :read [2 7] INFO [2022-01-10 12:53:28,859] jepsen worker 3 - jepsen.util 3 :ok :read [1 19] INFO [2022-01-10 12:53:28,859] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:28,860] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:28,860] jepsen worker 68 - jepsen.util 68 :ok :write [22 32] INFO [2022-01-10 12:53:28,860] jepsen worker 7 - jepsen.util 7 :ok :read [2 7] INFO [2022-01-10 12:53:28,860] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:28,860] jepsen worker 3 - jepsen.util 3 :ok :read [1 19] INFO [2022-01-10 12:53:28,861] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:28,861] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:53:28,862] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:28,862] jepsen worker 7 - jepsen.util 7 :ok :read [2 7] INFO [2022-01-10 12:53:28,862] jepsen worker 3 - jepsen.util 3 :ok :read [1 19] INFO [2022-01-10 12:53:28,863] jepsen worker 37 - jepsen.util 127 :ok :write [12 32] INFO [2022-01-10 12:53:28,863] jepsen worker 68 - jepsen.util 68 :invoke :write [22 10] INFO [2022-01-10 12:53:28,863] jepsen worker 3 - jepsen.util 3 :invoke :write [1 38] INFO [2022-01-10 12:53:28,864] jepsen worker 7 - jepsen.util 7 :invoke :write [2 8] INFO [2022-01-10 12:53:28,865] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:28,865] jepsen worker 37 - jepsen.util 127 :ok :read [12 32] INFO [2022-01-10 12:53:28,866] jepsen worker 37 - jepsen.util 127 :invoke :write [12 1] INFO [2022-01-10 12:53:28,914] jepsen worker 75 - jepsen.util 165 :ok :write [25 46] INFO [2022-01-10 12:53:28,914] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,915] jepsen worker 75 - jepsen.util 165 :ok :read [25 46] INFO [2022-01-10 12:53:28,916] jepsen worker 75 - jepsen.util 165 :invoke :write [25 27] INFO [2022-01-10 12:53:28,917] jepsen worker 69 - jepsen.util 159 :ok :write [23 6] INFO [2022-01-10 12:53:28,918] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:28,918] jepsen worker 69 - jepsen.util 159 :ok :read [23 6] INFO [2022-01-10 12:53:28,919] jepsen worker 69 - jepsen.util 159 :invoke :write [23 39] INFO [2022-01-10 12:53:28,921] jepsen worker 61 - jepsen.util 61 :ok :write [20 3] INFO [2022-01-10 12:53:28,922] jepsen worker 61 - jepsen.util 61 :invoke :write [20 22] INFO [2022-01-10 12:53:28,924] jepsen worker 13 - jepsen.util 13 :ok :write [4 33] INFO [2022-01-10 12:53:28,925] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,926] jepsen worker 13 - jepsen.util 13 :ok :read [4 33] INFO [2022-01-10 12:53:28,927] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,927] jepsen worker 13 - jepsen.util 13 :ok :read [4 33] INFO [2022-01-10 12:53:28,927] jepsen worker 43 - jepsen.util 43 :ok :write [14 46] INFO [2022-01-10 12:53:28,928] jepsen worker 13 - jepsen.util 13 :invoke :write [4 14] INFO [2022-01-10 12:53:28,928] jepsen worker 43 - jepsen.util 43 :invoke :write [14 27] INFO [2022-01-10 12:53:28,930] jepsen worker 79 - jepsen.util 259 :ok :write [26 45] INFO [2022-01-10 12:53:28,931] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,932] jepsen worker 79 - jepsen.util 259 :ok :read [26 45] INFO [2022-01-10 12:53:28,932] jepsen worker 79 - jepsen.util 259 :invoke :write [26 22] INFO [2022-01-10 12:53:28,934] jepsen worker 22 - jepsen.util 112 :ok :write [7 18] INFO [2022-01-10 12:53:28,935] jepsen worker 53 - jepsen.util 143 :ok :write [17 23] INFO [2022-01-10 12:53:28,935] jepsen worker 22 - jepsen.util 112 :invoke :write [7 25] INFO [2022-01-10 12:53:28,936] jepsen worker 53 - jepsen.util 143 :invoke :write [17 41] INFO [2022-01-10 12:53:28,939] jepsen worker 70 - jepsen.util 160 :ok :write [23 28] INFO [2022-01-10 12:53:28,940] jepsen worker 70 - jepsen.util 160 :invoke :write [23 44] INFO [2022-01-10 12:53:28,943] jepsen worker 13 - jepsen.util 13 :ok :write [4 14] INFO [2022-01-10 12:53:28,944] jepsen worker 13 - jepsen.util 13 :invoke :write [4 48] INFO [2022-01-10 12:53:28,950] jepsen worker 75 - jepsen.util 165 :ok :write [25 27] INFO [2022-01-10 12:53:28,950] jepsen worker 79 - jepsen.util 259 :ok :write [26 22] INFO [2022-01-10 12:53:28,951] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,951] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,952] jepsen worker 79 - jepsen.util 259 :ok :read [26 22] INFO [2022-01-10 12:53:28,952] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,953] jepsen worker 75 - jepsen.util 165 :ok :read [25 27] INFO [2022-01-10 12:53:28,953] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,953] jepsen worker 79 - jepsen.util 259 :ok :read [26 22] INFO [2022-01-10 12:53:28,954] jepsen worker 75 - jepsen.util 165 :ok :read [25 27] INFO [2022-01-10 12:53:28,954] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,955] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:28,955] jepsen worker 79 - jepsen.util 259 :ok :read [26 22] INFO [2022-01-10 12:53:28,955] jepsen worker 53 - jepsen.util 143 :ok :write [17 41] INFO [2022-01-10 12:53:28,956] jepsen worker 75 - jepsen.util 165 :ok :read [25 27] INFO [2022-01-10 12:53:28,956] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:28,957] jepsen worker 79 - jepsen.util 259 :ok :read [26 22] INFO [2022-01-10 12:53:28,957] jepsen worker 53 - jepsen.util 143 :invoke :write [17 26] INFO [2022-01-10 12:53:28,957] jepsen worker 79 - jepsen.util 259 :invoke :write [26 28] INFO [2022-01-10 12:53:28,958] jepsen worker 61 - jepsen.util 61 :ok :write [20 22] INFO [2022-01-10 12:53:28,958] jepsen worker 75 - jepsen.util 165 :invoke :write [25 8] INFO [2022-01-10 12:53:28,959] jepsen worker 61 - jepsen.util 61 :invoke :write [20 2] INFO [2022-01-10 12:53:28,961] jepsen worker 70 - jepsen.util 160 :ok :write [23 44] INFO [2022-01-10 12:53:28,961] jepsen worker 70 - jepsen.util 160 :invoke :write [23 38] INFO [2022-01-10 12:53:28,964] jepsen worker 43 - jepsen.util 43 :ok :write [14 27] INFO [2022-01-10 12:53:28,964] jepsen worker 43 - jepsen.util 43 :invoke :write [14 3] INFO [2022-01-10 12:53:28,965] jepsen worker 13 - jepsen.util 13 :ok :write [4 48] INFO [2022-01-10 12:53:28,966] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,967] jepsen worker 13 - jepsen.util 13 :ok :read [4 48] INFO [2022-01-10 12:53:28,967] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:28,969] jepsen worker 13 - jepsen.util 13 :ok :read [4 48] INFO [2022-01-10 12:53:28,970] jepsen worker 13 - jepsen.util 13 :invoke :write [4 5] INFO [2022-01-10 12:53:28,971] jepsen worker 22 - jepsen.util 112 :ok :write [7 25] INFO [2022-01-10 12:53:28,971] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:28,972] jepsen worker 22 - jepsen.util 112 :ok :read [7 25] INFO [2022-01-10 12:53:28,973] jepsen worker 22 - jepsen.util 112 :invoke :write [7 1] INFO [2022-01-10 12:53:28,975] jepsen worker 79 - jepsen.util 259 :ok :write [26 28] INFO [2022-01-10 12:53:28,975] jepsen worker 79 - jepsen.util 259 :invoke :write [26 0] INFO [2022-01-10 12:53:28,978] jepsen worker 53 - jepsen.util 143 :ok :write [17 26] INFO [2022-01-10 12:53:28,979] jepsen worker 53 - jepsen.util 143 :invoke :write [17 14] INFO [2022-01-10 12:53:28,980] jepsen worker 75 - jepsen.util 165 :ok :write [25 8] INFO [2022-01-10 12:53:28,980] jepsen worker 75 - jepsen.util 165 :invoke :write [25 23] INFO [2022-01-10 12:53:29,010] jepsen worker 33 - jepsen.util 123 :ok :write [11 28] INFO [2022-01-10 12:53:29,010] jepsen worker 84 - jepsen.util 84 :ok :write [28 36] INFO [2022-01-10 12:53:29,011] jepsen worker 33 - jepsen.util 123 :invoke :write [11 44] INFO [2022-01-10 12:53:29,011] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,012] jepsen worker 84 - jepsen.util 84 :ok :read [28 36] INFO [2022-01-10 12:53:29,013] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,014] jepsen worker 84 - jepsen.util 84 :ok :read [28 36] INFO [2022-01-10 12:53:29,014] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,015] jepsen worker 2 - jepsen.util 92 :ok :write [0 48] INFO [2022-01-10 12:53:29,015] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,015] jepsen worker 84 - jepsen.util 84 :ok :read [28 36] INFO [2022-01-10 12:53:29,016] jepsen worker 3 - jepsen.util 3 :ok :write [1 38] INFO [2022-01-10 12:53:29,016] jepsen worker 2 - jepsen.util 92 :ok :read [0 48] INFO [2022-01-10 12:53:29,016] jepsen worker 84 - jepsen.util 84 :invoke :write [28 46] INFO [2022-01-10 12:53:29,017] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:29,017] jepsen worker 2 - jepsen.util 92 :invoke :write [0 8] INFO [2022-01-10 12:53:29,018] jepsen worker 3 - jepsen.util 3 :ok :read [1 38] INFO [2022-01-10 12:53:29,018] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:29,019] jepsen worker 3 - jepsen.util 3 :ok :read [1 38] INFO [2022-01-10 12:53:29,019] jepsen worker 3 - jepsen.util 3 :invoke :write [1 30] INFO [2022-01-10 12:53:29,169] jepsen worker 41 - jepsen.util 41 :ok :write [13 18] INFO [2022-01-10 12:53:29,170] jepsen worker 41 - jepsen.util 41 :invoke :write [13 10] INFO [2022-01-10 12:53:29,170] jepsen worker 37 - jepsen.util 127 :ok :write [12 1] INFO [2022-01-10 12:53:29,171] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,172] jepsen worker 37 - jepsen.util 127 :ok :read [12 1] INFO [2022-01-10 12:53:29,172] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,173] jepsen worker 37 - jepsen.util 127 :ok :read [12 1] INFO [2022-01-10 12:53:29,174] jepsen worker 37 - jepsen.util 127 :invoke :write [12 49] INFO [2022-01-10 12:53:29,196] jepsen worker 70 - jepsen.util 160 :ok :write [23 38] INFO [2022-01-10 12:53:29,197] jepsen worker 61 - jepsen.util 61 :ok :write [20 2] INFO [2022-01-10 12:53:29,197] jepsen worker 70 - jepsen.util 160 :invoke :write [23 20] INFO [2022-01-10 12:53:29,197] jepsen worker 61 - jepsen.util 61 :invoke :write [20 38] INFO [2022-01-10 12:53:29,201] jepsen worker 13 - jepsen.util 13 :ok :write [4 5] INFO [2022-01-10 12:53:29,202] jepsen worker 13 - jepsen.util 13 :invoke :write [4 48] INFO [2022-01-10 12:53:29,209] jepsen worker 79 - jepsen.util 259 :ok :write [26 0] INFO [2022-01-10 12:53:29,209] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:29,210] jepsen worker 43 - jepsen.util 43 :ok :write [14 3] INFO [2022-01-10 12:53:29,210] jepsen worker 79 - jepsen.util 259 :ok :read [26 0] INFO [2022-01-10 12:53:29,210] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,211] jepsen worker 79 - jepsen.util 259 :invoke :write [26 39] INFO [2022-01-10 12:53:29,211] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:29,212] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,213] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:29,214] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,214] jepsen worker 22 - jepsen.util 112 :ok :write [7 1] INFO [2022-01-10 12:53:29,215] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:29,215] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:29,215] jepsen worker 22 - jepsen.util 112 :ok :read [7 1] INFO [2022-01-10 12:53:29,215] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,216] jepsen worker 53 - jepsen.util 143 :ok :write [17 14] INFO [2022-01-10 12:53:29,216] jepsen worker 22 - jepsen.util 112 :invoke :write [7 46] INFO [2022-01-10 12:53:29,216] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:29,216] jepsen worker 53 - jepsen.util 143 :invoke :write [17 5] INFO [2022-01-10 12:53:29,217] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,218] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:29,218] jepsen worker 43 - jepsen.util 43 :invoke :write [14 3] INFO [2022-01-10 12:53:29,219] jepsen worker 75 - jepsen.util 165 :ok :write [25 23] INFO [2022-01-10 12:53:29,219] jepsen worker 75 - jepsen.util 165 :invoke :write [25 40] INFO [2022-01-10 12:53:29,233] jepsen worker 70 - jepsen.util 160 :ok :write [23 20] INFO [2022-01-10 12:53:29,233] jepsen worker 61 - jepsen.util 61 :ok :write [20 38] INFO [2022-01-10 12:53:29,233] jepsen worker 61 - jepsen.util 61 :invoke :write [20 1] INFO [2022-01-10 12:53:29,234] jepsen worker 70 - jepsen.util 160 :invoke :write [23 35] INFO [2022-01-10 12:53:29,238] jepsen worker 43 - jepsen.util 43 :ok :write [14 3] INFO [2022-01-10 12:53:29,238] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,239] jepsen worker 13 - jepsen.util 13 :ok :write [4 48] INFO [2022-01-10 12:53:29,239] jepsen worker 43 - jepsen.util 43 :ok :read [14 3] INFO [2022-01-10 12:53:29,240] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:29,240] jepsen worker 43 - jepsen.util 43 :invoke :write [14 11] INFO [2022-01-10 12:53:29,241] jepsen worker 13 - jepsen.util 13 :ok :read [4 48] INFO [2022-01-10 12:53:29,241] jepsen worker 13 - jepsen.util 13 :invoke :write [4 19] INFO [2022-01-10 12:53:29,244] jepsen worker 22 - jepsen.util 112 :ok :write [7 46] INFO [2022-01-10 12:53:29,245] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:29,245] jepsen worker 22 - jepsen.util 112 :ok :read [7 46] INFO [2022-01-10 12:53:29,246] jepsen worker 22 - jepsen.util 112 :invoke :write [7 23] INFO [2022-01-10 12:53:29,247] jepsen worker 79 - jepsen.util 259 :ok :write [26 39] INFO [2022-01-10 12:53:29,247] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:29,248] jepsen worker 79 - jepsen.util 259 :ok :read [26 39] INFO [2022-01-10 12:53:29,249] jepsen worker 79 - jepsen.util 259 :invoke :write [26 17] INFO [2022-01-10 12:53:29,249] jepsen worker 75 - jepsen.util 165 :ok :write [25 40] INFO [2022-01-10 12:53:29,250] jepsen worker 75 - jepsen.util 165 :invoke :write [25 30] INFO [2022-01-10 12:53:29,253] jepsen worker 84 - jepsen.util 84 :ok :write [28 46] INFO [2022-01-10 12:53:29,253] jepsen worker 15 - jepsen.util 15 :ok :write [5 31] INFO [2022-01-10 12:53:29,254] jepsen worker 84 - jepsen.util 84 :invoke :write [28 19] INFO [2022-01-10 12:53:29,254] jepsen worker 53 - jepsen.util 143 :ok :write [17 5] INFO [2022-01-10 12:53:29,254] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:29,255] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:29,255] jepsen worker 15 - jepsen.util 15 :ok :read [5 31] INFO [2022-01-10 12:53:29,255] jepsen worker 61 - jepsen.util 61 :ok :write [20 1] INFO [2022-01-10 12:53:29,256] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:29,256] jepsen worker 61 - jepsen.util 61 :invoke :write [20 7] INFO [2022-01-10 12:53:29,257] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:29,257] jepsen worker 53 - jepsen.util 143 :invoke :write [17 4] INFO [2022-01-10 12:53:29,257] jepsen worker 15 - jepsen.util 15 :ok :read [5 31] INFO [2022-01-10 12:53:29,258] jepsen worker 15 - jepsen.util 15 :invoke :write [5 5] INFO [2022-01-10 12:53:29,261] jepsen worker 43 - jepsen.util 43 :ok :write [14 11] INFO [2022-01-10 12:53:29,261] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,262] jepsen worker 43 - jepsen.util 43 :ok :read [14 11] INFO [2022-01-10 12:53:29,262] jepsen worker 70 - jepsen.util 160 :ok :write [23 35] INFO [2022-01-10 12:53:29,262] jepsen worker 43 - jepsen.util 43 :invoke :write [14 21] INFO [2022-01-10 12:53:29,263] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:29,264] jepsen worker 70 - jepsen.util 160 :ok :read [23 35] INFO [2022-01-10 12:53:29,264] jepsen worker 70 - jepsen.util 160 :invoke :write [23 41] INFO [2022-01-10 12:53:29,266] jepsen worker 22 - jepsen.util 112 :ok :write [7 23] INFO [2022-01-10 12:53:29,266] jepsen worker 22 - jepsen.util 112 :invoke :write [7 14] INFO [2022-01-10 12:53:29,269] jepsen worker 13 - jepsen.util 13 :ok :write [4 19] INFO [2022-01-10 12:53:29,270] jepsen worker 75 - jepsen.util 165 :ok :write [25 30] INFO [2022-01-10 12:53:29,270] jepsen worker 13 - jepsen.util 13 :invoke :write [4 45] INFO [2022-01-10 12:53:29,270] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:29,271] jepsen worker 75 - jepsen.util 165 :ok :read [25 30] INFO [2022-01-10 12:53:29,272] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:29,272] jepsen worker 75 - jepsen.util 165 :ok :read [25 30] INFO [2022-01-10 12:53:29,273] jepsen worker 75 - jepsen.util 165 :invoke :write [25 44] INFO [2022-01-10 12:53:29,275] jepsen worker 61 - jepsen.util 61 :ok :write [20 7] INFO [2022-01-10 12:53:29,275] jepsen worker 79 - jepsen.util 259 :ok :write [26 17] INFO [2022-01-10 12:53:29,276] jepsen worker 61 - jepsen.util 61 :invoke :write [20 24] INFO [2022-01-10 12:53:29,276] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:29,277] jepsen worker 79 - jepsen.util 259 :ok :read [26 17] INFO [2022-01-10 12:53:29,278] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:29,279] jepsen worker 79 - jepsen.util 259 :ok :read [26 17] INFO [2022-01-10 12:53:29,279] jepsen worker 79 - jepsen.util 259 :invoke :write [26 48] INFO [2022-01-10 12:53:29,279] jepsen worker 43 - jepsen.util 43 :ok :write [14 21] INFO [2022-01-10 12:53:29,280] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,281] jepsen worker 43 - jepsen.util 43 :ok :read [14 21] INFO [2022-01-10 12:53:29,281] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,282] jepsen worker 53 - jepsen.util 143 :ok :write [17 4] INFO [2022-01-10 12:53:29,282] jepsen worker 43 - jepsen.util 43 :ok :read [14 21] INFO [2022-01-10 12:53:29,283] jepsen worker 43 - jepsen.util 43 :invoke :write [14 43] INFO [2022-01-10 12:53:29,283] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:29,284] jepsen worker 53 - jepsen.util 143 :ok :read [17 4] INFO [2022-01-10 12:53:29,284] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:29,285] jepsen worker 53 - jepsen.util 143 :ok :read [17 4] INFO [2022-01-10 12:53:29,285] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:29,286] jepsen worker 53 - jepsen.util 143 :ok :read [17 4] INFO [2022-01-10 12:53:29,287] jepsen worker 53 - jepsen.util 143 :invoke :write [17 5] INFO [2022-01-10 12:53:29,287] jepsen worker 22 - jepsen.util 112 :ok :write [7 14] INFO [2022-01-10 12:53:29,288] jepsen worker 22 - jepsen.util 112 :invoke :write [7 8] INFO [2022-01-10 12:53:29,293] jepsen worker 75 - jepsen.util 165 :ok :write [25 44] INFO [2022-01-10 12:53:29,293] jepsen worker 75 - jepsen.util 165 :invoke :write [25 3] INFO [2022-01-10 12:53:29,294] jepsen worker 70 - jepsen.util 160 :ok :write [23 41] INFO [2022-01-10 12:53:29,295] jepsen worker 70 - jepsen.util 160 :invoke :write [23 3] INFO [2022-01-10 12:53:29,300] jepsen worker 61 - jepsen.util 61 :ok :write [20 24] INFO [2022-01-10 12:53:29,300] jepsen worker 13 - jepsen.util 13 :ok :write [4 45] INFO [2022-01-10 12:53:29,300] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:29,301] jepsen worker 13 - jepsen.util 13 :invoke :write [4 5] INFO [2022-01-10 12:53:29,301] jepsen worker 61 - jepsen.util 61 :ok :read [20 24] INFO [2022-01-10 12:53:29,302] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:29,303] jepsen worker 61 - jepsen.util 61 :ok :read [20 24] INFO [2022-01-10 12:53:29,303] jepsen worker 61 - jepsen.util 61 :invoke :write [20 37] INFO [2022-01-10 12:53:29,303] jepsen worker 43 - jepsen.util 43 :ok :write [14 43] INFO [2022-01-10 12:53:29,304] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,305] jepsen worker 43 - jepsen.util 43 :ok :read [14 43] INFO [2022-01-10 12:53:29,305] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,306] jepsen worker 79 - jepsen.util 259 :ok :write [26 48] INFO [2022-01-10 12:53:29,306] jepsen worker 43 - jepsen.util 43 :ok :read [14 43] INFO [2022-01-10 12:53:29,306] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:29,307] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,307] jepsen worker 79 - jepsen.util 259 :ok :read [26 48] INFO [2022-01-10 12:53:29,308] jepsen worker 43 - jepsen.util 43 :ok :read [14 43] INFO [2022-01-10 12:53:29,308] jepsen worker 79 - jepsen.util 259 :invoke :write [26 31] INFO [2022-01-10 12:53:29,308] jepsen worker 22 - jepsen.util 112 :ok :write [7 8] INFO [2022-01-10 12:53:29,308] jepsen worker 43 - jepsen.util 43 :invoke :write [14 28] INFO [2022-01-10 12:53:29,309] jepsen worker 22 - jepsen.util 112 :invoke :write [7 13] INFO [2022-01-10 12:53:29,312] jepsen worker 53 - jepsen.util 143 :ok :write [17 5] INFO [2022-01-10 12:53:29,312] jepsen worker 75 - jepsen.util 165 :ok :write [25 3] INFO [2022-01-10 12:53:29,313] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:29,313] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:29,314] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:29,314] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:29,314] jepsen worker 75 - jepsen.util 165 :ok :read [25 3] INFO [2022-01-10 12:53:29,315] jepsen worker 75 - jepsen.util 165 :invoke :write [25 1] INFO [2022-01-10 12:53:29,315] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:29,316] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:29,317] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:29,317] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:29,318] jepsen worker 53 - jepsen.util 143 :ok :read [17 5] INFO [2022-01-10 12:53:29,319] jepsen worker 53 - jepsen.util 143 :invoke :write [17 40] INFO [2022-01-10 12:53:29,319] jepsen worker 61 - jepsen.util 61 :ok :write [20 37] INFO [2022-01-10 12:53:29,319] jepsen worker 70 - jepsen.util 160 :ok :write [23 3] INFO [2022-01-10 12:53:29,319] jepsen worker 70 - jepsen.util 160 :invoke :write [23 32] INFO [2022-01-10 12:53:29,320] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:29,320] jepsen worker 61 - jepsen.util 61 :ok :read [20 37] INFO [2022-01-10 12:53:29,321] jepsen worker 61 - jepsen.util 61 :invoke :write [20 1] INFO [2022-01-10 12:53:29,373] jepsen worker 50 - jepsen.util 50 :ok :write [16 32] INFO [2022-01-10 12:53:29,374] jepsen worker 3 - jepsen.util 3 :ok :write [1 30] INFO [2022-01-10 12:53:29,374] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:29,374] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:29,375] jepsen worker 50 - jepsen.util 50 :ok :read [16 32] INFO [2022-01-10 12:53:29,375] jepsen worker 3 - jepsen.util 3 :ok :read [1 30] INFO [2022-01-10 12:53:29,376] jepsen worker 50 - jepsen.util 50 :invoke :write [16 23] INFO [2022-01-10 12:53:29,376] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 12:53:29,378] jepsen worker 81 - jepsen.util 171 :ok :write [27 9] INFO [2022-01-10 12:53:29,379] jepsen worker 37 - jepsen.util 127 :ok :write [12 49] INFO [2022-01-10 12:53:29,379] jepsen worker 37 - jepsen.util 127 :invoke :write [12 41] INFO [2022-01-10 12:53:29,379] jepsen worker 81 - jepsen.util 171 :invoke :write [27 18] INFO [2022-01-10 12:53:29,385] jepsen worker 88 - jepsen.util 178 :ok :write [29 45] INFO [2022-01-10 12:53:29,385] jepsen worker 84 - jepsen.util 84 :ok :write [28 19] INFO [2022-01-10 12:53:29,386] jepsen worker 88 - jepsen.util 178 :invoke :write [29 30] INFO [2022-01-10 12:53:29,386] jepsen worker 84 - jepsen.util 84 :invoke :write [28 9] INFO [2022-01-10 12:53:29,391] jepsen worker 7 - jepsen.util 7 :ok :write [2 8] INFO [2022-01-10 12:53:29,391] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:29,392] jepsen worker 3 - jepsen.util 3 :ok :write [1 40] INFO [2022-01-10 12:53:29,392] jepsen worker 7 - jepsen.util 7 :ok :read [2 8] INFO [2022-01-10 12:53:29,392] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:29,393] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:29,393] jepsen worker 3 - jepsen.util 3 :ok :read [1 40] INFO [2022-01-10 12:53:29,394] jepsen worker 7 - jepsen.util 7 :ok :read [2 8] INFO [2022-01-10 12:53:29,394] jepsen worker 68 - jepsen.util 68 :ok :write [22 10] INFO [2022-01-10 12:53:29,394] jepsen worker 3 - jepsen.util 3 :invoke :write [1 1] INFO [2022-01-10 12:53:29,395] jepsen worker 68 - jepsen.util 68 :invoke :write [22 36] INFO [2022-01-10 12:53:29,395] jepsen worker 7 - jepsen.util 7 :invoke :write [2 5] INFO [2022-01-10 12:53:29,397] jepsen worker 37 - jepsen.util 127 :ok :write [12 41] INFO [2022-01-10 12:53:29,398] jepsen worker 37 - jepsen.util 127 :invoke :write [12 16] INFO [2022-01-10 12:53:29,400] jepsen worker 33 - jepsen.util 123 :ok :write [11 44] INFO [2022-01-10 12:53:29,401] jepsen worker 33 - jepsen.util 123 :invoke :write [11 35] INFO [2022-01-10 12:53:29,401] jepsen worker 84 - jepsen.util 84 :ok :write [28 9] INFO [2022-01-10 12:53:29,402] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,403] jepsen worker 84 - jepsen.util 84 :ok :read [28 9] INFO [2022-01-10 12:53:29,403] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,404] jepsen worker 84 - jepsen.util 84 :ok :read [28 9] INFO [2022-01-10 12:53:29,405] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,405] jepsen worker 2 - jepsen.util 92 :ok :write [0 8] INFO [2022-01-10 12:53:29,405] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,406] jepsen worker 3 - jepsen.util 3 :ok :write [1 1] INFO [2022-01-10 12:53:29,406] jepsen worker 2 - jepsen.util 92 :ok :read [0 8] INFO [2022-01-10 12:53:29,406] jepsen worker 84 - jepsen.util 84 :ok :read [28 9] INFO [2022-01-10 12:53:29,407] jepsen worker 84 - jepsen.util 84 :invoke :write [28 27] INFO [2022-01-10 12:53:29,407] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 12:53:29,408] jepsen worker 2 - jepsen.util 92 :invoke :write [0 49] INFO [2022-01-10 12:53:29,410] jepsen worker 41 - jepsen.util 41 :ok :write [13 10] INFO [2022-01-10 12:53:29,410] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:29,411] jepsen worker 41 - jepsen.util 41 :ok :read [13 10] INFO [2022-01-10 12:53:29,411] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:29,412] jepsen worker 41 - jepsen.util 41 :ok :read [13 10] INFO [2022-01-10 12:53:29,413] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:29,414] jepsen worker 41 - jepsen.util 41 :ok :read [13 10] INFO [2022-01-10 12:53:29,414] jepsen worker 15 - jepsen.util 15 :ok :write [5 5] INFO [2022-01-10 12:53:29,414] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:29,414] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:29,415] jepsen worker 41 - jepsen.util 41 :ok :read [13 10] INFO [2022-01-10 12:53:29,415] jepsen worker 15 - jepsen.util 15 :ok :read [5 5] INFO [2022-01-10 12:53:29,415] jepsen worker 41 - jepsen.util 41 :invoke :write [13 36] INFO [2022-01-10 12:53:29,416] jepsen worker 15 - jepsen.util 15 :invoke :write [5 25] INFO [2022-01-10 12:53:29,420] jepsen worker 3 - jepsen.util 3 :ok :write [1 40] INFO [2022-01-10 12:53:29,421] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:29,422] jepsen worker 3 - jepsen.util 3 :ok :read [1 40] INFO [2022-01-10 12:53:29,422] jepsen worker 3 - jepsen.util 3 :invoke :write [1 34] INFO [2022-01-10 12:53:29,424] jepsen worker 84 - jepsen.util 84 :ok :write [28 27] INFO [2022-01-10 12:53:29,425] jepsen worker 84 - jepsen.util 84 :invoke :write [28 23] INFO [2022-01-10 12:53:29,429] jepsen worker 88 - jepsen.util 178 :ok :write [29 30] INFO [2022-01-10 12:53:29,430] jepsen worker 88 - jepsen.util 178 :invoke :write [29 22] INFO [2022-01-10 12:53:29,437] jepsen worker 84 - jepsen.util 84 :ok :write [28 23] INFO [2022-01-10 12:53:29,437] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,438] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:29,439] jepsen worker 3 - jepsen.util 3 :ok :write [1 34] INFO [2022-01-10 12:53:29,439] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,439] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:29,440] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:29,440] jepsen worker 3 - jepsen.util 3 :ok :read [1 34] INFO [2022-01-10 12:53:29,441] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,441] jepsen worker 3 - jepsen.util 3 :invoke :write [1 10] INFO [2022-01-10 12:53:29,442] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:29,442] jepsen worker 84 - jepsen.util 84 :invoke :write [28 17] INFO [2022-01-10 12:53:29,446] jepsen worker 43 - jepsen.util 43 :ok :write [14 28] INFO [2022-01-10 12:53:29,446] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,449] jepsen worker 13 - jepsen.util 13 :ok :write [4 5] INFO [2022-01-10 12:53:29,449] jepsen worker 13 - jepsen.util 13 :invoke :write [4 13] INFO [2022-01-10 12:53:29,450] jepsen worker 2 - jepsen.util 92 :ok :write [0 49] INFO [2022-01-10 12:53:29,450] jepsen worker 43 - jepsen.util 43 :ok :read [14 28] INFO [2022-01-10 12:53:29,451] jepsen worker 2 - jepsen.util 92 :invoke :write [0 21] INFO [2022-01-10 12:53:29,451] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,452] jepsen worker 43 - jepsen.util 43 :ok :read [14 28] INFO [2022-01-10 12:53:29,452] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,452] jepsen worker 22 - jepsen.util 112 :ok :write [7 13] INFO [2022-01-10 12:53:29,453] jepsen worker 43 - jepsen.util 43 :ok :read [14 28] INFO [2022-01-10 12:53:29,453] jepsen worker 22 - jepsen.util 112 :invoke :write [7 33] INFO [2022-01-10 12:53:29,454] jepsen worker 43 - jepsen.util 43 :invoke :write [14 22] INFO [2022-01-10 12:53:29,455] jepsen worker 79 - jepsen.util 259 :ok :write [26 31] INFO [2022-01-10 12:53:29,456] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:29,456] jepsen worker 41 - jepsen.util 41 :ok :write [13 36] INFO [2022-01-10 12:53:29,457] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:29,457] jepsen worker 79 - jepsen.util 259 :ok :read [26 31] INFO [2022-01-10 12:53:29,457] jepsen worker 41 - jepsen.util 41 :ok :read [13 36] INFO [2022-01-10 12:53:29,458] jepsen worker 79 - jepsen.util 259 :invoke :write [26 14] INFO [2022-01-10 12:53:29,458] jepsen worker 75 - jepsen.util 165 :ok :write [25 1] INFO [2022-01-10 12:53:29,458] jepsen worker 41 - jepsen.util 41 :invoke :write [13 41] INFO [2022-01-10 12:53:29,459] jepsen worker 75 - jepsen.util 165 :invoke :write [25 45] INFO [2022-01-10 12:53:29,463] jepsen worker 53 - jepsen.util 143 :ok :write [17 40] INFO [2022-01-10 12:53:29,463] jepsen worker 53 - jepsen.util 143 :invoke :write [17 22] INFO [2022-01-10 12:53:29,466] jepsen worker 88 - jepsen.util 178 :ok :write [29 22] INFO [2022-01-10 12:53:29,466] jepsen worker 61 - jepsen.util 61 :ok :write [20 1] INFO [2022-01-10 12:53:29,466] jepsen worker 88 - jepsen.util 178 :invoke :write [29 8] INFO [2022-01-10 12:53:29,467] jepsen worker 70 - jepsen.util 160 :ok :write [23 32] INFO [2022-01-10 12:53:29,467] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:29,467] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:29,468] jepsen worker 61 - jepsen.util 61 :ok :read [20 1] INFO [2022-01-10 12:53:29,468] jepsen worker 70 - jepsen.util 160 :ok :read [23 32] INFO [2022-01-10 12:53:29,468] jepsen worker 61 - jepsen.util 61 :invoke :write [20 42] INFO [2022-01-10 12:53:29,468] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:29,469] jepsen worker 70 - jepsen.util 160 :ok :read [23 32] INFO [2022-01-10 12:53:29,470] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:29,470] jepsen worker 70 - jepsen.util 160 :ok :read [23 32] INFO [2022-01-10 12:53:29,471] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:29,471] jepsen worker 70 - jepsen.util 160 :ok :read [23 32] INFO [2022-01-10 12:53:29,472] jepsen worker 70 - jepsen.util 160 :invoke :write [23 39] INFO [2022-01-10 12:53:29,473] jepsen worker 50 - jepsen.util 50 :ok :write [16 23] INFO [2022-01-10 12:53:29,473] jepsen worker 50 - jepsen.util 50 :invoke :write [16 16] INFO [2022-01-10 12:53:29,480] jepsen worker 81 - jepsen.util 171 :ok :write [27 18] INFO [2022-01-10 12:53:29,480] jepsen worker 81 - jepsen.util 171 :invoke :write [27 34] INFO [2022-01-10 12:53:29,481] jepsen worker 7 - jepsen.util 7 :ok :write [2 5] INFO [2022-01-10 12:53:29,481] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:29,482] jepsen worker 7 - jepsen.util 7 :ok :read [2 5] INFO [2022-01-10 12:53:29,483] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:29,483] jepsen worker 7 - jepsen.util 7 :ok :read [2 5] INFO [2022-01-10 12:53:29,484] jepsen worker 7 - jepsen.util 7 :invoke :write [2 44] INFO [2022-01-10 12:53:29,489] jepsen worker 68 - jepsen.util 68 :ok :write [22 36] INFO [2022-01-10 12:53:29,489] jepsen worker 68 - jepsen.util 68 :invoke :write [22 32] INFO [2022-01-10 12:53:29,493] jepsen worker 33 - jepsen.util 123 :ok :write [11 35] INFO [2022-01-10 12:53:29,493] jepsen worker 33 - jepsen.util 123 :invoke :write [11 22] INFO [2022-01-10 12:53:29,498] jepsen worker 15 - jepsen.util 15 :ok :write [5 25] INFO [2022-01-10 12:53:29,499] jepsen worker 15 - jepsen.util 15 :invoke :write [5 5] INFO [2022-01-10 12:53:29,514] jepsen worker 50 - jepsen.util 50 :ok :write [16 16] INFO [2022-01-10 12:53:29,515] jepsen worker 50 - jepsen.util 50 :invoke :write [16 24] INFO [2022-01-10 12:53:29,517] jepsen worker 7 - jepsen.util 7 :ok :write [2 44] INFO [2022-01-10 12:53:29,517] jepsen worker 7 - jepsen.util 7 :invoke :write [2 22] INFO [2022-01-10 12:53:29,522] jepsen worker 81 - jepsen.util 171 :ok :write [27 34] INFO [2022-01-10 12:53:29,523] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:29,524] jepsen worker 81 - jepsen.util 171 :ok :read [27 34] INFO [2022-01-10 12:53:29,525] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:29,525] jepsen worker 81 - jepsen.util 171 :ok :read [27 34] INFO [2022-01-10 12:53:29,526] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:29,527] jepsen worker 81 - jepsen.util 171 :ok :read [27 34] INFO [2022-01-10 12:53:29,527] jepsen worker 68 - jepsen.util 68 :ok :write [22 32] INFO [2022-01-10 12:53:29,527] jepsen worker 81 - jepsen.util 171 :invoke :write [27 20] INFO [2022-01-10 12:53:29,528] jepsen worker 68 - jepsen.util 68 :invoke :write [22 17] INFO [2022-01-10 12:53:29,530] jepsen worker 33 - jepsen.util 123 :ok :write [11 22] INFO [2022-01-10 12:53:29,530] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:29,531] jepsen worker 33 - jepsen.util 123 :ok :read [11 22] INFO [2022-01-10 12:53:29,532] jepsen worker 33 - jepsen.util 123 :invoke :write [11 45] INFO [2022-01-10 12:53:29,534] jepsen worker 15 - jepsen.util 15 :ok :write [5 5] INFO [2022-01-10 12:53:29,535] jepsen worker 15 - jepsen.util 15 :invoke :write [5 37] INFO [2022-01-10 12:53:29,538] jepsen worker 50 - jepsen.util 50 :ok :write [16 24] INFO [2022-01-10 12:53:29,539] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:29,539] jepsen worker 50 - jepsen.util 50 :ok :read [16 24] INFO [2022-01-10 12:53:29,540] jepsen worker 50 - jepsen.util 50 :invoke :write [16 40] INFO [2022-01-10 12:53:29,542] jepsen worker 7 - jepsen.util 7 :ok :write [2 22] INFO [2022-01-10 12:53:29,542] jepsen worker 7 - jepsen.util 7 :invoke :write [2 38] INFO [2022-01-10 12:53:29,547] jepsen worker 81 - jepsen.util 171 :ok :write [27 20] INFO [2022-01-10 12:53:29,547] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:29,548] jepsen worker 81 - jepsen.util 171 :ok :read [27 20] INFO [2022-01-10 12:53:29,548] jepsen worker 81 - jepsen.util 171 :invoke :write [27 26] INFO [2022-01-10 12:53:29,550] jepsen worker 68 - jepsen.util 68 :ok :write [22 17] INFO [2022-01-10 12:53:29,551] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:29,552] jepsen worker 68 - jepsen.util 68 :ok :read [22 17] INFO [2022-01-10 12:53:29,552] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:29,553] jepsen worker 68 - jepsen.util 68 :ok :read [22 17] INFO [2022-01-10 12:53:29,553] jepsen worker 68 - jepsen.util 68 :invoke :write [22 49] INFO [2022-01-10 12:53:29,554] jepsen worker 33 - jepsen.util 123 :ok :write [11 45] INFO [2022-01-10 12:53:29,554] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:29,555] jepsen worker 33 - jepsen.util 123 :ok :read [11 45] INFO [2022-01-10 12:53:29,555] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:29,556] jepsen worker 33 - jepsen.util 123 :ok :read [11 45] INFO [2022-01-10 12:53:29,557] jepsen worker 33 - jepsen.util 123 :invoke :write [11 5] INFO [2022-01-10 12:53:29,559] jepsen worker 15 - jepsen.util 15 :ok :write [5 37] INFO [2022-01-10 12:53:29,559] jepsen worker 15 - jepsen.util 15 :invoke :write [5 32] INFO [2022-01-10 12:53:29,561] jepsen worker 3 - jepsen.util 3 :ok :write [1 10] INFO [2022-01-10 12:53:29,562] jepsen worker 3 - jepsen.util 3 :invoke :write [1 8] INFO [2022-01-10 12:53:29,565] jepsen worker 50 - jepsen.util 50 :ok :write [16 40] INFO [2022-01-10 12:53:29,566] jepsen worker 50 - jepsen.util 50 :invoke :write [16 1] INFO [2022-01-10 12:53:29,568] jepsen worker 7 - jepsen.util 7 :ok :write [2 38] INFO [2022-01-10 12:53:29,568] jepsen worker 7 - jepsen.util 7 :invoke :write [2 12] INFO [2022-01-10 12:53:29,570] jepsen worker 84 - jepsen.util 84 :ok :write [28 17] INFO [2022-01-10 12:53:29,571] jepsen worker 2 - jepsen.util 92 :ok :write [0 21] INFO [2022-01-10 12:53:29,571] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,572] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,572] jepsen worker 84 - jepsen.util 84 :ok :read [28 17] INFO [2022-01-10 12:53:29,573] jepsen worker 2 - jepsen.util 92 :ok :read [0 21] INFO [2022-01-10 12:53:29,573] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,574] jepsen worker 2 - jepsen.util 92 :invoke :write [0 39] INFO [2022-01-10 12:53:29,574] jepsen worker 84 - jepsen.util 84 :ok :read [28 17] INFO [2022-01-10 12:53:29,575] jepsen worker 81 - jepsen.util 171 :ok :write [27 26] INFO [2022-01-10 12:53:29,575] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,575] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:29,576] jepsen worker 84 - jepsen.util 84 :ok :read [28 17] INFO [2022-01-10 12:53:29,576] jepsen worker 81 - jepsen.util 171 :ok :read [27 26] INFO [2022-01-10 12:53:29,577] jepsen worker 84 - jepsen.util 84 :invoke :write [28 30] INFO [2022-01-10 12:53:29,577] jepsen worker 13 - jepsen.util 13 :ok :write [4 13] INFO [2022-01-10 12:53:29,577] jepsen worker 81 - jepsen.util 171 :invoke :write [27 25] INFO [2022-01-10 12:53:29,578] jepsen worker 13 - jepsen.util 13 :invoke :write [4 6] INFO [2022-01-10 12:53:29,579] jepsen worker 22 - jepsen.util 112 :ok :write [7 33] INFO [2022-01-10 12:53:29,579] jepsen worker 68 - jepsen.util 68 :ok :write [22 49] INFO [2022-01-10 12:53:29,579] jepsen worker 22 - jepsen.util 112 :invoke :write [7 0] INFO [2022-01-10 12:53:29,580] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:29,581] jepsen worker 68 - jepsen.util 68 :ok :read [22 49] INFO [2022-01-10 12:53:29,581] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:29,582] jepsen worker 68 - jepsen.util 68 :ok :read [22 49] INFO [2022-01-10 12:53:29,582] jepsen worker 68 - jepsen.util 68 :invoke :write [22 35] INFO [2022-01-10 12:53:29,586] jepsen worker 33 - jepsen.util 123 :ok :write [11 5] INFO [2022-01-10 12:53:29,586] jepsen worker 43 - jepsen.util 43 :ok :write [14 22] INFO [2022-01-10 12:53:29,587] jepsen worker 33 - jepsen.util 123 :invoke :write [11 40] INFO [2022-01-10 12:53:29,587] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,588] jepsen worker 79 - jepsen.util 259 :ok :write [26 14] INFO [2022-01-10 12:53:29,588] jepsen worker 43 - jepsen.util 43 :ok :read [14 22] INFO [2022-01-10 12:53:29,589] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:29,589] jepsen worker 43 - jepsen.util 43 :invoke :write [14 13] INFO [2022-01-10 12:53:29,589] jepsen worker 79 - jepsen.util 259 :ok :read [26 14] INFO [2022-01-10 12:53:29,590] jepsen worker 79 - jepsen.util 259 :invoke :write [26 22] INFO [2022-01-10 12:53:29,592] jepsen worker 41 - jepsen.util 41 :ok :write [13 41] INFO [2022-01-10 12:53:29,593] jepsen worker 15 - jepsen.util 15 :ok :write [5 32] INFO [2022-01-10 12:53:29,593] jepsen worker 41 - jepsen.util 41 :invoke :write [13 7] INFO [2022-01-10 12:53:29,594] jepsen worker 15 - jepsen.util 15 :invoke :write [5 33] INFO [2022-01-10 12:53:29,597] jepsen worker 53 - jepsen.util 143 :ok :write [17 22] INFO [2022-01-10 12:53:29,598] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:29,599] jepsen worker 53 - jepsen.util 143 :ok :read [17 22] INFO [2022-01-10 12:53:29,599] jepsen worker 75 - jepsen.util 165 :ok :write [25 45] INFO [2022-01-10 12:53:29,599] jepsen worker 53 - jepsen.util 143 :invoke :write [17 33] INFO [2022-01-10 12:53:29,600] jepsen worker 75 - jepsen.util 165 :invoke :write [25 28] INFO [2022-01-10 12:53:29,603] jepsen worker 37 - jepsen.util 127 :ok :write [12 16] INFO [2022-01-10 12:53:29,604] jepsen worker 37 - jepsen.util 127 :invoke :write [12 26] INFO [2022-01-10 12:53:29,607] jepsen worker 58 - jepsen.util 238 :info :write [19 17] :SyncWriteAmbiguous INFO [2022-01-10 12:53:29,608] jepsen worker 88 - jepsen.util 178 :ok :write [29 8] INFO [2022-01-10 12:53:29,608] jepsen worker 58 - jepsen.util 328 :invoke :write [19 45] INFO [2022-01-10 12:53:29,609] jepsen worker 88 - jepsen.util 178 :invoke :write [29 25] INFO [2022-01-10 12:53:29,610] jepsen worker 70 - jepsen.util 160 :ok :write [23 39] INFO [2022-01-10 12:53:29,611] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:29,612] jepsen worker 70 - jepsen.util 160 :ok :read [23 39] INFO [2022-01-10 12:53:29,612] jepsen worker 70 - jepsen.util 160 :invoke :write [23 37] INFO [2022-01-10 12:53:29,614] jepsen worker 61 - jepsen.util 61 :ok :write [20 42] INFO [2022-01-10 12:53:29,614] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:29,615] jepsen worker 61 - jepsen.util 61 :ok :read [20 42] INFO [2022-01-10 12:53:29,615] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:29,616] jepsen worker 61 - jepsen.util 61 :ok :read [20 42] INFO [2022-01-10 12:53:29,617] jepsen worker 61 - jepsen.util 61 :invoke :write [20 49] INFO [2022-01-10 12:53:29,618] jepsen worker 3 - jepsen.util 3 :ok :write [1 8] INFO [2022-01-10 12:53:29,618] jepsen worker 3 - jepsen.util 3 :invoke :write [1 30] INFO [2022-01-10 12:53:29,625] jepsen worker 22 - jepsen.util 112 :ok :write [7 0] INFO [2022-01-10 12:53:29,625] jepsen worker 22 - jepsen.util 112 :invoke :write [7 29] INFO [2022-01-10 12:53:29,630] jepsen worker 43 - jepsen.util 43 :ok :write [14 13] INFO [2022-01-10 12:53:29,630] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,631] jepsen worker 43 - jepsen.util 43 :ok :read [14 13] INFO [2022-01-10 12:53:29,631] jepsen worker 43 - jepsen.util 43 :invoke :write [14 21] INFO [2022-01-10 12:53:29,635] jepsen worker 13 - jepsen.util 13 :ok :write [4 6] INFO [2022-01-10 12:53:29,635] jepsen worker 26 - jepsen.util 206 :info :write [8 32] :SyncWriteAmbiguous INFO [2022-01-10 12:53:29,636] jepsen worker 13 - jepsen.util 13 :invoke :write [4 49] INFO [2022-01-10 12:53:29,637] jepsen worker 63 - jepsen.util 243 :info :write [21 23] :SyncWriteAmbiguous INFO [2022-01-10 12:53:29,637] jepsen worker 26 - jepsen.util 296 :invoke :read [8 nil] INFO [2022-01-10 12:53:29,638] jepsen worker 63 - jepsen.util 333 :invoke :read [21 nil] INFO [2022-01-10 12:53:29,639] jepsen worker 26 - jepsen.util 296 :ok :read [8 25] INFO [2022-01-10 12:53:29,639] jepsen worker 26 - jepsen.util 296 :invoke :read [8 nil] INFO [2022-01-10 12:53:29,640] jepsen worker 26 - jepsen.util 296 :ok :read [8 25] INFO [2022-01-10 12:53:29,640] jepsen worker 26 - jepsen.util 296 :invoke :write [8 32] INFO [2022-01-10 12:53:29,640] jepsen worker 79 - jepsen.util 259 :ok :write [26 22] INFO [2022-01-10 12:53:29,641] jepsen worker 79 - jepsen.util 259 :invoke :write [26 8] INFO [2022-01-10 12:53:29,648] jepsen worker 53 - jepsen.util 143 :ok :write [17 33] INFO [2022-01-10 12:53:29,648] jepsen worker 45 - jepsen.util 225 :ok :read [15 10] INFO [2022-01-10 12:53:29,648] jepsen worker 53 - jepsen.util 143 :invoke :write [17 44] INFO [2022-01-10 12:53:29,649] jepsen worker 45 - jepsen.util 225 :invoke :write [15 7] INFO [2022-01-10 12:53:29,663] jepsen worker 70 - jepsen.util 160 :ok :write [23 37] INFO [2022-01-10 12:53:29,663] jepsen worker 70 - jepsen.util 160 :invoke :write [23 4] INFO [2022-01-10 12:53:29,755] jepsen worker 50 - jepsen.util 50 :ok :write [16 1] INFO [2022-01-10 12:53:29,755] jepsen worker 50 - jepsen.util 50 :invoke :write [16 34] INFO [2022-01-10 12:53:29,757] jepsen worker 84 - jepsen.util 84 :ok :write [28 30] INFO [2022-01-10 12:53:29,758] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,759] jepsen worker 84 - jepsen.util 84 :ok :read [28 30] INFO [2022-01-10 12:53:29,760] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,761] jepsen worker 84 - jepsen.util 84 :ok :read [28 30] INFO [2022-01-10 12:53:29,761] jepsen worker 7 - jepsen.util 7 :ok :write [2 12] INFO [2022-01-10 12:53:29,762] jepsen worker 7 - jepsen.util 7 :invoke :write [2 36] INFO [2022-01-10 12:53:29,763] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,764] jepsen worker 84 - jepsen.util 84 :ok :read [28 30] INFO [2022-01-10 12:53:29,765] jepsen worker 84 - jepsen.util 84 :invoke :write [28 33] INFO [2022-01-10 12:53:29,766] jepsen worker 2 - jepsen.util 92 :ok :write [0 39] INFO [2022-01-10 12:53:29,767] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,768] jepsen worker 2 - jepsen.util 92 :ok :read [0 39] INFO [2022-01-10 12:53:29,769] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,769] jepsen worker 2 - jepsen.util 92 :ok :read [0 39] INFO [2022-01-10 12:53:29,770] jepsen worker 37 - jepsen.util 127 :ok :write [12 26] INFO [2022-01-10 12:53:29,770] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,770] jepsen worker 37 - jepsen.util 127 :invoke :write [12 15] INFO [2022-01-10 12:53:29,771] jepsen worker 2 - jepsen.util 92 :ok :read [0 39] INFO [2022-01-10 12:53:29,771] jepsen worker 2 - jepsen.util 92 :invoke :write [0 38] INFO [2022-01-10 12:53:29,772] jepsen worker 81 - jepsen.util 171 :ok :write [27 25] INFO [2022-01-10 12:53:29,772] jepsen worker 81 - jepsen.util 171 :invoke :write [27 25] INFO [2022-01-10 12:53:29,776] jepsen worker 3 - jepsen.util 3 :ok :write [1 30] INFO [2022-01-10 12:53:29,777] jepsen worker 3 - jepsen.util 3 :invoke :write [1 13] INFO [2022-01-10 12:53:29,780] jepsen worker 68 - jepsen.util 68 :ok :write [22 35] INFO [2022-01-10 12:53:29,781] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:29,781] jepsen worker 84 - jepsen.util 84 :ok :write [28 33] INFO [2022-01-10 12:53:29,781] jepsen worker 68 - jepsen.util 68 :ok :read [22 35] INFO [2022-01-10 12:53:29,782] jepsen worker 84 - jepsen.util 84 :invoke :write [28 48] INFO [2022-01-10 12:53:29,782] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:29,784] jepsen worker 68 - jepsen.util 68 :ok :read [22 35] INFO [2022-01-10 12:53:29,784] jepsen worker 33 - jepsen.util 123 :ok :write [11 40] INFO [2022-01-10 12:53:29,784] jepsen worker 0 - jepsen.util 90 :info :write [0 3] :RequestTimeoutException INFO [2022-01-10 12:53:29,785] jepsen worker 68 - jepsen.util 68 :invoke :write [22 42] INFO [2022-01-10 12:53:29,785] jepsen worker 37 - jepsen.util 127 :ok :write [12 15] INFO [2022-01-10 12:53:29,785] jepsen worker 33 - jepsen.util 123 :invoke :write [11 41] INFO [2022-01-10 12:53:29,786] jepsen worker 37 - jepsen.util 127 :invoke :write [12 29] INFO [2022-01-10 12:53:29,786] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,788] jepsen worker 0 - jepsen.util 180 :ok :read [0 39] INFO [2022-01-10 12:53:29,789] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,790] jepsen worker 0 - jepsen.util 180 :ok :read [0 39] INFO [2022-01-10 12:53:29,790] jepsen worker 3 - jepsen.util 3 :ok :write [1 13] INFO [2022-01-10 12:53:29,791] jepsen worker 41 - jepsen.util 41 :ok :write [13 7] INFO [2022-01-10 12:53:29,791] jepsen worker 3 - jepsen.util 3 :invoke :write [1 35] INFO [2022-01-10 12:53:29,792] jepsen worker 41 - jepsen.util 41 :invoke :write [13 39] INFO [2022-01-10 12:53:29,793] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,794] jepsen worker 0 - jepsen.util 180 :ok :read [0 39] INFO [2022-01-10 12:53:29,795] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,795] jepsen worker 0 - jepsen.util 180 :ok :read [0 39] INFO [2022-01-10 12:53:29,796] jepsen worker 0 - jepsen.util 180 :invoke :write [0 48] INFO [2022-01-10 12:53:29,797] jepsen worker 15 - jepsen.util 15 :ok :write [5 33] INFO [2022-01-10 12:53:29,797] jepsen worker 84 - jepsen.util 84 :ok :write [28 48] INFO [2022-01-10 12:53:29,797] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:29,798] jepsen worker 15 - jepsen.util 15 :ok :read [5 33] INFO [2022-01-10 12:53:29,798] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,799] jepsen worker 15 - jepsen.util 15 :invoke :write [5 0] INFO [2022-01-10 12:53:29,799] jepsen worker 84 - jepsen.util 84 :ok :read [28 48] INFO [2022-01-10 12:53:29,800] jepsen worker 84 - jepsen.util 84 :invoke :write [28 46] INFO [2022-01-10 12:53:29,803] jepsen worker 88 - jepsen.util 178 :ok :write [29 25] INFO [2022-01-10 12:53:29,803] jepsen worker 37 - jepsen.util 127 :ok :write [12 29] INFO [2022-01-10 12:53:29,804] jepsen worker 88 - jepsen.util 178 :invoke :write [29 3] INFO [2022-01-10 12:53:29,804] jepsen worker 37 - jepsen.util 127 :invoke :write [12 7] INFO [2022-01-10 12:53:29,809] jepsen worker 3 - jepsen.util 3 :ok :write [1 35] INFO [2022-01-10 12:53:29,810] jepsen worker 3 - jepsen.util 3 :invoke :write [1 7] INFO [2022-01-10 12:53:29,815] jepsen worker 84 - jepsen.util 84 :ok :write [28 46] INFO [2022-01-10 12:53:29,815] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,816] jepsen worker 84 - jepsen.util 84 :ok :read [28 46] INFO [2022-01-10 12:53:29,817] jepsen worker 84 - jepsen.util 84 :invoke :write [28 30] INFO [2022-01-10 12:53:29,820] jepsen worker 37 - jepsen.util 127 :ok :write [12 7] INFO [2022-01-10 12:53:29,820] jepsen worker 37 - jepsen.util 127 :invoke :write [12 7] INFO [2022-01-10 12:53:29,821] jepsen worker 2 - jepsen.util 92 :ok :write [0 38] INFO [2022-01-10 12:53:29,823] jepsen worker 27 - jepsen.util 207 :info :write [9 7] :SyncWriteAmbiguous INFO [2022-01-10 12:53:29,823] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,823] jepsen worker 27 - jepsen.util 297 :invoke :read [9 nil] INFO [2022-01-10 12:53:29,824] jepsen worker 2 - jepsen.util 92 :ok :read [0 38] INFO [2022-01-10 12:53:29,824] jepsen worker 27 - jepsen.util 297 :ok :read [9 1] INFO [2022-01-10 12:53:29,824] jepsen worker 2 - jepsen.util 92 :invoke :write [0 13] INFO [2022-01-10 12:53:29,825] jepsen worker 27 - jepsen.util 297 :invoke :read [9 nil] INFO [2022-01-10 12:53:29,825] jepsen worker 27 - jepsen.util 297 :ok :read [9 1] INFO [2022-01-10 12:53:29,826] jepsen worker 27 - jepsen.util 297 :invoke :write [9 5] INFO [2022-01-10 12:53:29,827] jepsen worker 3 - jepsen.util 3 :ok :write [1 7] INFO [2022-01-10 12:53:29,827] jepsen worker 3 - jepsen.util 3 :invoke :write [1 44] INFO [2022-01-10 12:53:29,830] jepsen worker 84 - jepsen.util 84 :ok :write [28 30] INFO [2022-01-10 12:53:29,831] jepsen worker 84 - jepsen.util 84 :invoke :write [28 18] INFO [2022-01-10 12:53:29,838] jepsen worker 37 - jepsen.util 127 :ok :write [12 7] INFO [2022-01-10 12:53:29,839] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,840] jepsen worker 37 - jepsen.util 127 :ok :read [12 7] INFO [2022-01-10 12:53:29,841] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,842] jepsen worker 41 - jepsen.util 41 :ok :write [13 39] INFO [2022-01-10 12:53:29,842] jepsen worker 37 - jepsen.util 127 :ok :read [12 7] INFO [2022-01-10 12:53:29,843] jepsen worker 41 - jepsen.util 41 :invoke :write [13 10] INFO [2022-01-10 12:53:29,843] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,844] jepsen worker 37 - jepsen.util 127 :ok :read [12 7] INFO [2022-01-10 12:53:29,844] jepsen worker 37 - jepsen.util 127 :invoke :write [12 47] INFO [2022-01-10 12:53:29,850] jepsen worker 3 - jepsen.util 3 :ok :write [1 44] INFO [2022-01-10 12:53:29,850] jepsen worker 3 - jepsen.util 3 :invoke :write [1 33] INFO [2022-01-10 12:53:29,856] jepsen worker 88 - jepsen.util 178 :ok :write [29 3] INFO [2022-01-10 12:53:29,856] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:29,857] jepsen worker 84 - jepsen.util 84 :ok :write [28 18] INFO [2022-01-10 12:53:29,857] jepsen worker 88 - jepsen.util 178 :ok :read [29 3] INFO [2022-01-10 12:53:29,857] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,858] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:29,859] jepsen worker 2 - jepsen.util 92 :ok :write [0 13] INFO [2022-01-10 12:53:29,859] jepsen worker 88 - jepsen.util 178 :ok :read [29 3] INFO [2022-01-10 12:53:29,859] jepsen worker 84 - jepsen.util 84 :ok :read [28 18] INFO [2022-01-10 12:53:29,859] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,860] jepsen worker 84 - jepsen.util 84 :invoke :write [28 3] INFO [2022-01-10 12:53:29,860] jepsen worker 2 - jepsen.util 92 :ok :read [0 13] INFO [2022-01-10 12:53:29,861] jepsen worker 88 - jepsen.util 178 :invoke :write [29 46] INFO [2022-01-10 12:53:29,861] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,862] jepsen worker 2 - jepsen.util 92 :ok :read [0 13] INFO [2022-01-10 12:53:29,863] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,864] jepsen worker 37 - jepsen.util 127 :ok :write [12 47] INFO [2022-01-10 12:53:29,864] jepsen worker 2 - jepsen.util 92 :ok :read [0 13] INFO [2022-01-10 12:53:29,865] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,865] jepsen worker 2 - jepsen.util 92 :invoke :write [0 35] INFO [2022-01-10 12:53:29,866] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:29,867] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,868] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:29,869] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,870] jepsen worker 41 - jepsen.util 41 :ok :write [13 10] INFO [2022-01-10 12:53:29,870] jepsen worker 3 - jepsen.util 3 :ok :write [1 33] INFO [2022-01-10 12:53:29,871] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:29,871] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:29,872] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:29,872] jepsen worker 41 - jepsen.util 41 :ok :read [13 10] INFO [2022-01-10 12:53:29,872] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,872] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:29,873] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:29,873] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:29,874] jepsen worker 41 - jepsen.util 41 :ok :read [13 10] INFO [2022-01-10 12:53:29,874] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:29,874] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,875] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:29,875] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:29,875] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:29,876] jepsen worker 41 - jepsen.util 41 :ok :read [13 10] INFO [2022-01-10 12:53:29,876] jepsen worker 3 - jepsen.util 3 :invoke :write [1 29] INFO [2022-01-10 12:53:29,877] jepsen worker 41 - jepsen.util 41 :invoke :write [13 7] INFO [2022-01-10 12:53:29,877] jepsen worker 88 - jepsen.util 178 :ok :write [29 46] INFO [2022-01-10 12:53:29,877] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,878] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:29,878] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:29,878] jepsen worker 88 - jepsen.util 178 :ok :read [29 46] INFO [2022-01-10 12:53:29,879] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,879] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:29,879] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:29,880] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,880] jepsen worker 88 - jepsen.util 178 :ok :read [29 46] INFO [2022-01-10 12:53:29,880] jepsen worker 84 - jepsen.util 84 :ok :write [28 3] INFO [2022-01-10 12:53:29,881] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:29,881] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,882] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:29,882] jepsen worker 88 - jepsen.util 178 :ok :read [29 46] INFO [2022-01-10 12:53:29,882] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,882] jepsen worker 2 - jepsen.util 92 :ok :write [0 35] INFO [2022-01-10 12:53:29,883] jepsen worker 88 - jepsen.util 178 :invoke :write [29 13] INFO [2022-01-10 12:53:29,883] jepsen worker 84 - jepsen.util 84 :ok :read [28 3] INFO [2022-01-10 12:53:29,883] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:29,883] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:29,884] jepsen worker 84 - jepsen.util 84 :invoke :write [28 41] INFO [2022-01-10 12:53:29,884] jepsen worker 2 - jepsen.util 92 :ok :read [0 35] INFO [2022-01-10 12:53:29,884] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,884] jepsen worker 2 - jepsen.util 92 :invoke :write [0 16] INFO [2022-01-10 12:53:29,885] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:29,886] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:29,887] jepsen worker 37 - jepsen.util 127 :ok :read [12 47] INFO [2022-01-10 12:53:29,887] jepsen worker 37 - jepsen.util 127 :invoke :write [12 16] INFO [2022-01-10 12:53:29,889] jepsen worker 75 - jepsen.util 165 :ok :write [25 28] INFO [2022-01-10 12:53:29,890] jepsen worker 75 - jepsen.util 165 :invoke :write [25 19] INFO [2022-01-10 12:53:29,894] jepsen worker 61 - jepsen.util 61 :ok :write [20 49] INFO [2022-01-10 12:53:29,895] jepsen worker 61 - jepsen.util 61 :invoke :write [20 19] INFO [2022-01-10 12:53:29,895] jepsen worker 3 - jepsen.util 3 :ok :write [1 29] INFO [2022-01-10 12:53:29,895] jepsen worker 3 - jepsen.util 3 :invoke :write [1 21] INFO [2022-01-10 12:53:29,896] jepsen worker 13 - jepsen.util 13 :ok :write [4 49] INFO [2022-01-10 12:53:29,897] jepsen worker 13 - jepsen.util 13 :invoke :write [4 30] INFO [2022-01-10 12:53:29,900] jepsen worker 22 - jepsen.util 112 :ok :write [7 29] INFO [2022-01-10 12:53:29,901] jepsen worker 41 - jepsen.util 41 :ok :write [13 7] INFO [2022-01-10 12:53:29,901] jepsen worker 22 - jepsen.util 112 :invoke :write [7 9] INFO [2022-01-10 12:53:29,902] jepsen worker 41 - jepsen.util 41 :invoke :write [13 0] INFO [2022-01-10 12:53:29,902] jepsen worker 88 - jepsen.util 178 :ok :write [29 13] INFO [2022-01-10 12:53:29,903] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:29,904] jepsen worker 88 - jepsen.util 178 :ok :read [29 13] INFO [2022-01-10 12:53:29,905] jepsen worker 84 - jepsen.util 84 :ok :write [28 41] INFO [2022-01-10 12:53:29,905] jepsen worker 88 - jepsen.util 178 :invoke :write [29 33] INFO [2022-01-10 12:53:29,905] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:29,906] jepsen worker 84 - jepsen.util 84 :ok :read [28 41] INFO [2022-01-10 12:53:29,906] jepsen worker 79 - jepsen.util 259 :ok :write [26 8] INFO [2022-01-10 12:53:29,907] jepsen worker 43 - jepsen.util 43 :ok :write [14 21] INFO [2022-01-10 12:53:29,907] jepsen worker 84 - jepsen.util 84 :invoke :write [28 38] INFO [2022-01-10 12:53:29,907] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:29,908] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:29,908] jepsen worker 79 - jepsen.util 259 :ok :read [26 8] INFO [2022-01-10 12:53:29,909] jepsen worker 43 - jepsen.util 43 :ok :read [14 21] INFO [2022-01-10 12:53:29,909] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:29,909] jepsen worker 43 - jepsen.util 43 :invoke :write [14 16] INFO [2022-01-10 12:53:29,910] jepsen worker 79 - jepsen.util 259 :ok :read [26 8] INFO [2022-01-10 12:53:29,910] jepsen worker 79 - jepsen.util 259 :invoke :write [26 19] INFO [2022-01-10 12:53:29,913] jepsen worker 50 - jepsen.util 50 :ok :write [16 34] INFO [2022-01-10 12:53:29,913] jepsen worker 50 - jepsen.util 50 :invoke :write [16 39] INFO [2022-01-10 12:53:29,916] jepsen worker 53 - jepsen.util 143 :ok :write [17 44] INFO [2022-01-10 12:53:29,917] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:29,918] jepsen worker 53 - jepsen.util 143 :ok :read [17 44] INFO [2022-01-10 12:53:29,918] jepsen worker 53 - jepsen.util 143 :invoke :write [17 22] INFO [2022-01-10 12:53:29,974] jepsen worker 7 - jepsen.util 7 :ok :write [2 36] INFO [2022-01-10 12:53:29,974] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:29,975] jepsen worker 7 - jepsen.util 7 :ok :read [2 36] INFO [2022-01-10 12:53:29,975] jepsen worker 7 - jepsen.util 7 :invoke :write [2 42] INFO [2022-01-10 12:53:29,977] jepsen worker 2 - jepsen.util 92 :ok :write [0 16] INFO [2022-01-10 12:53:29,978] jepsen worker 37 - jepsen.util 127 :ok :write [12 16] INFO [2022-01-10 12:53:29,978] jepsen worker 2 - jepsen.util 92 :invoke :write [0 27] INFO [2022-01-10 12:53:29,978] jepsen worker 37 - jepsen.util 127 :invoke :write [12 5] INFO [2022-01-10 12:53:29,979] jepsen worker 70 - jepsen.util 160 :ok :write [23 4] INFO [2022-01-10 12:53:29,980] jepsen worker 70 - jepsen.util 160 :invoke :write [23 29] INFO [2022-01-10 12:53:29,980] jepsen worker 81 - jepsen.util 171 :ok :write [27 25] INFO [2022-01-10 12:53:29,981] jepsen worker 81 - jepsen.util 171 :invoke :write [27 5] INFO [2022-01-10 12:53:29,986] jepsen worker 68 - jepsen.util 68 :ok :write [22 42] INFO [2022-01-10 12:53:29,987] jepsen worker 68 - jepsen.util 68 :invoke :write [22 12] INFO [2022-01-10 12:53:29,988] jepsen worker 3 - jepsen.util 3 :ok :write [1 21] INFO [2022-01-10 12:53:29,989] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:29,989] jepsen worker 3 - jepsen.util 3 :ok :read [1 21] INFO [2022-01-10 12:53:29,990] jepsen worker 3 - jepsen.util 3 :invoke :write [1 13] INFO [2022-01-10 12:53:29,992] jepsen worker 33 - jepsen.util 123 :ok :write [11 41] INFO [2022-01-10 12:53:29,992] jepsen worker 33 - jepsen.util 123 :invoke :write [11 26] INFO [2022-01-10 12:53:29,998] jepsen worker 13 - jepsen.util 13 :ok :write [4 30] INFO [2022-01-10 12:53:29,998] jepsen worker 13 - jepsen.util 13 :invoke :write [4 9] INFO [2022-01-10 12:53:30,000] jepsen worker 15 - jepsen.util 15 :ok :write [5 0] INFO [2022-01-10 12:53:30,000] jepsen worker 15 - jepsen.util 15 :invoke :write [5 24] INFO [2022-01-10 12:53:30,004] jepsen worker 88 - jepsen.util 178 :ok :write [29 33] INFO [2022-01-10 12:53:30,005] jepsen worker 88 - jepsen.util 178 :invoke :write [29 28] INFO [2022-01-10 12:53:30,005] jepsen worker 75 - jepsen.util 165 :ok :write [25 19] INFO [2022-01-10 12:53:30,005] jepsen worker 84 - jepsen.util 84 :ok :write [28 38] INFO [2022-01-10 12:53:30,006] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:30,006] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,007] jepsen worker 84 - jepsen.util 84 :ok :read [28 38] INFO [2022-01-10 12:53:30,007] jepsen worker 75 - jepsen.util 165 :ok :read [25 19] INFO [2022-01-10 12:53:30,008] jepsen worker 84 - jepsen.util 84 :invoke :write [28 22] INFO [2022-01-10 12:53:30,008] jepsen worker 75 - jepsen.util 165 :invoke :write [25 26] INFO [2022-01-10 12:53:30,011] jepsen worker 61 - jepsen.util 61 :ok :write [20 19] INFO [2022-01-10 12:53:30,011] jepsen worker 61 - jepsen.util 61 :invoke :write [20 36] INFO [2022-01-10 12:53:30,014] jepsen worker 79 - jepsen.util 259 :ok :write [26 19] INFO [2022-01-10 12:53:30,015] jepsen worker 22 - jepsen.util 112 :ok :write [7 9] INFO [2022-01-10 12:53:30,015] jepsen worker 79 - jepsen.util 259 :invoke :write [26 22] INFO [2022-01-10 12:53:30,015] jepsen worker 41 - jepsen.util 41 :ok :write [13 0] INFO [2022-01-10 12:53:30,016] jepsen worker 22 - jepsen.util 112 :invoke :write [7 1] INFO [2022-01-10 12:53:30,016] jepsen worker 41 - jepsen.util 41 :invoke :write [13 28] INFO [2022-01-10 12:53:30,023] jepsen worker 50 - jepsen.util 50 :ok :write [16 39] INFO [2022-01-10 12:53:30,024] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,025] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:30,025] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,026] jepsen worker 53 - jepsen.util 143 :ok :write [17 22] INFO [2022-01-10 12:53:30,026] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:30,027] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,028] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,028] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:30,029] jepsen worker 53 - jepsen.util 143 :ok :read [17 22] INFO [2022-01-10 12:53:30,029] jepsen worker 7 - jepsen.util 7 :ok :write [2 42] INFO [2022-01-10 12:53:30,029] jepsen worker 43 - jepsen.util 43 :ok :write [14 16] INFO [2022-01-10 12:53:30,029] jepsen worker 50 - jepsen.util 50 :invoke :write [16 39] INFO [2022-01-10 12:53:30,030] jepsen worker 43 - jepsen.util 43 :invoke :write [14 48] INFO [2022-01-10 12:53:30,030] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,030] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,031] jepsen worker 53 - jepsen.util 143 :ok :read [17 22] INFO [2022-01-10 12:53:30,031] jepsen worker 0 - jepsen.util 180 :ok :write [0 48] INFO [2022-01-10 12:53:30,031] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,031] jepsen worker 37 - jepsen.util 127 :ok :write [12 5] INFO [2022-01-10 12:53:30,032] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,032] jepsen worker 7 - jepsen.util 7 :ok :read [2 42] INFO [2022-01-10 12:53:30,032] jepsen worker 0 - jepsen.util 180 :ok :read [0 48] INFO [2022-01-10 12:53:30,032] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,033] jepsen worker 53 - jepsen.util 143 :ok :read [17 22] INFO [2022-01-10 12:53:30,033] jepsen worker 7 - jepsen.util 7 :invoke :write [2 14] INFO [2022-01-10 12:53:30,033] jepsen worker 37 - jepsen.util 127 :ok :read [12 5] INFO [2022-01-10 12:53:30,034] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,034] jepsen worker 37 - jepsen.util 127 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,034] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,035] jepsen worker 53 - jepsen.util 143 :ok :read [17 22] INFO [2022-01-10 12:53:30,035] jepsen worker 37 - jepsen.util 127 :ok :read [12 5] INFO [2022-01-10 12:53:30,035] jepsen worker 0 - jepsen.util 180 :ok :read [0 48] INFO [2022-01-10 12:53:30,035] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,036] jepsen worker 37 - jepsen.util 127 :invoke :write [12 31] INFO [2022-01-10 12:53:30,036] jepsen worker 53 - jepsen.util 143 :ok :read [17 22] INFO [2022-01-10 12:53:30,036] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,037] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,037] jepsen worker 0 - jepsen.util 180 :ok :read [0 48] INFO [2022-01-10 12:53:30,037] jepsen worker 0 - jepsen.util 180 :invoke :write [0 47] INFO [2022-01-10 12:53:30,038] jepsen worker 53 - jepsen.util 143 :ok :read [17 22] INFO [2022-01-10 12:53:30,038] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,039] jepsen worker 53 - jepsen.util 143 :ok :read [17 22] INFO [2022-01-10 12:53:30,039] jepsen worker 53 - jepsen.util 143 :invoke :write [17 22] INFO [2022-01-10 12:53:30,042] jepsen worker 70 - jepsen.util 160 :ok :write [23 29] INFO [2022-01-10 12:53:30,042] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:30,045] jepsen worker 70 - jepsen.util 160 :ok :read [23 29] INFO [2022-01-10 12:53:30,045] jepsen worker 70 - jepsen.util 160 :invoke :write [23 25] INFO [2022-01-10 12:53:30,047] jepsen worker 81 - jepsen.util 171 :ok :write [27 5] INFO [2022-01-10 12:53:30,048] jepsen worker 81 - jepsen.util 171 :invoke :write [27 42] INFO [2022-01-10 12:53:30,052] jepsen worker 68 - jepsen.util 68 :ok :write [22 12] INFO [2022-01-10 12:53:30,053] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:30,053] jepsen worker 3 - jepsen.util 3 :ok :write [1 13] INFO [2022-01-10 12:53:30,054] jepsen worker 3 - jepsen.util 3 :invoke :write [1 41] INFO [2022-01-10 12:53:30,054] jepsen worker 68 - jepsen.util 68 :ok :read [22 12] INFO [2022-01-10 12:53:30,054] jepsen worker 68 - jepsen.util 68 :invoke :write [22 16] INFO [2022-01-10 12:53:30,060] jepsen worker 13 - jepsen.util 13 :ok :write [4 9] INFO [2022-01-10 12:53:30,061] jepsen worker 13 - jepsen.util 13 :invoke :write [4 25] INFO [2022-01-10 12:53:30,063] jepsen worker 33 - jepsen.util 123 :ok :write [11 26] INFO [2022-01-10 12:53:30,064] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,065] jepsen worker 33 - jepsen.util 123 :ok :read [11 26] INFO [2022-01-10 12:53:30,066] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,067] jepsen worker 33 - jepsen.util 123 :ok :read [11 26] INFO [2022-01-10 12:53:30,067] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,068] jepsen worker 33 - jepsen.util 123 :ok :read [11 26] INFO [2022-01-10 12:53:30,069] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,070] jepsen worker 33 - jepsen.util 123 :ok :read [11 26] INFO [2022-01-10 12:53:30,070] jepsen worker 33 - jepsen.util 123 :invoke :write [11 42] INFO [2022-01-10 12:53:30,075] jepsen worker 79 - jepsen.util 259 :ok :write [26 22] INFO [2022-01-10 12:53:30,075] jepsen worker 75 - jepsen.util 165 :ok :write [25 26] INFO [2022-01-10 12:53:30,076] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,077] jepsen worker 75 - jepsen.util 165 :invoke :write [25 31] INFO [2022-01-10 12:53:30,077] jepsen worker 79 - jepsen.util 259 :ok :read [26 22] INFO [2022-01-10 12:53:30,078] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,079] jepsen worker 79 - jepsen.util 259 :ok :read [26 22] INFO [2022-01-10 12:53:30,079] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,080] jepsen worker 79 - jepsen.util 259 :ok :read [26 22] INFO [2022-01-10 12:53:30,081] jepsen worker 79 - jepsen.util 259 :invoke :write [26 35] INFO [2022-01-10 12:53:30,083] jepsen worker 15 - jepsen.util 15 :ok :write [5 24] INFO [2022-01-10 12:53:30,083] jepsen worker 53 - jepsen.util 143 :ok :write [17 22] INFO [2022-01-10 12:53:30,084] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,085] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 12:53:30,085] jepsen worker 53 - jepsen.util 143 :invoke :write [17 5] INFO [2022-01-10 12:53:30,085] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,086] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 12:53:30,087] jepsen worker 84 - jepsen.util 84 :ok :write [28 22] INFO [2022-01-10 12:53:30,087] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,088] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 12:53:30,088] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:30,088] jepsen worker 15 - jepsen.util 15 :invoke :write [5 29] INFO [2022-01-10 12:53:30,089] jepsen worker 61 - jepsen.util 61 :ok :write [20 36] INFO [2022-01-10 12:53:30,089] jepsen worker 84 - jepsen.util 84 :ok :read [28 22] INFO [2022-01-10 12:53:30,089] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:30,090] jepsen worker 84 - jepsen.util 84 :invoke :write [28 10] INFO [2022-01-10 12:53:30,090] jepsen worker 61 - jepsen.util 61 :ok :read [20 36] INFO [2022-01-10 12:53:30,091] jepsen worker 61 - jepsen.util 61 :invoke :write [20 49] INFO [2022-01-10 12:53:30,091] jepsen worker 88 - jepsen.util 178 :ok :write [29 28] INFO [2022-01-10 12:53:30,092] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,093] jepsen worker 88 - jepsen.util 178 :ok :read [29 28] INFO [2022-01-10 12:53:30,093] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,094] jepsen worker 88 - jepsen.util 178 :ok :read [29 28] INFO [2022-01-10 12:53:30,095] jepsen worker 70 - jepsen.util 160 :ok :write [23 25] INFO [2022-01-10 12:53:30,095] jepsen worker 88 - jepsen.util 178 :invoke :write [29 37] INFO [2022-01-10 12:53:30,096] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:30,096] jepsen worker 38 - jepsen.util 38 :ok :write [12 21] INFO [2022-01-10 12:53:30,096] jepsen worker 22 - jepsen.util 112 :ok :write [7 1] INFO [2022-01-10 12:53:30,096] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,097] jepsen worker 70 - jepsen.util 160 :ok :read [23 25] INFO [2022-01-10 12:53:30,097] jepsen worker 38 - jepsen.util 38 :ok :read [12 21] INFO [2022-01-10 12:53:30,097] jepsen worker 22 - jepsen.util 112 :invoke :write [7 33] INFO [2022-01-10 12:53:30,098] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,098] jepsen worker 70 - jepsen.util 160 :invoke :write [23 25] INFO [2022-01-10 12:53:30,099] jepsen worker 38 - jepsen.util 38 :ok :read [12 21] INFO [2022-01-10 12:53:30,099] jepsen worker 41 - jepsen.util 41 :ok :write [13 28] INFO [2022-01-10 12:53:30,100] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,100] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,101] jepsen worker 38 - jepsen.util 38 :ok :read [12 21] INFO [2022-01-10 12:53:30,101] jepsen worker 41 - jepsen.util 41 :ok :read [13 28] INFO [2022-01-10 12:53:30,102] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,102] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,102] jepsen worker 38 - jepsen.util 38 :ok :read [12 21] INFO [2022-01-10 12:53:30,103] jepsen worker 41 - jepsen.util 41 :ok :read [13 28] INFO [2022-01-10 12:53:30,103] jepsen worker 38 - jepsen.util 38 :invoke :write [12 22] INFO [2022-01-10 12:53:30,104] jepsen worker 41 - jepsen.util 41 :invoke :write [13 17] INFO [2022-01-10 12:53:30,106] jepsen worker 50 - jepsen.util 50 :ok :write [16 39] INFO [2022-01-10 12:53:30,107] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,108] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:30,108] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,109] jepsen worker 43 - jepsen.util 43 :ok :write [14 48] INFO [2022-01-10 12:53:30,109] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:30,110] jepsen worker 43 - jepsen.util 43 :invoke :write [14 0] INFO [2022-01-10 12:53:30,110] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,111] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:30,112] jepsen worker 50 - jepsen.util 50 :invoke :write [16 21] INFO [2022-01-10 12:53:30,113] jepsen worker 3 - jepsen.util 3 :ok :write [1 41] INFO [2022-01-10 12:53:30,113] jepsen worker 3 - jepsen.util 3 :invoke :write [1 35] INFO [2022-01-10 12:53:30,118] jepsen worker 7 - jepsen.util 7 :ok :write [2 14] INFO [2022-01-10 12:53:30,118] jepsen worker 7 - jepsen.util 7 :invoke :write [2 44] INFO [2022-01-10 12:53:30,120] jepsen worker 13 - jepsen.util 13 :ok :write [4 25] INFO [2022-01-10 12:53:30,121] jepsen worker 0 - jepsen.util 180 :ok :write [0 47] INFO [2022-01-10 12:53:30,121] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:30,121] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,122] jepsen worker 13 - jepsen.util 13 :ok :read [4 25] INFO [2022-01-10 12:53:30,122] jepsen worker 0 - jepsen.util 180 :ok :read [0 47] INFO [2022-01-10 12:53:30,122] jepsen worker 13 - jepsen.util 13 :invoke :write [4 19] INFO [2022-01-10 12:53:30,123] jepsen worker 0 - jepsen.util 180 :invoke :write [0 3] INFO [2022-01-10 12:53:30,128] jepsen worker 81 - jepsen.util 171 :ok :write [27 42] INFO [2022-01-10 12:53:30,128] jepsen worker 81 - jepsen.util 171 :invoke :write [27 47] INFO [2022-01-10 12:53:30,134] jepsen worker 79 - jepsen.util 259 :ok :write [26 35] INFO [2022-01-10 12:53:30,134] jepsen worker 68 - jepsen.util 68 :ok :write [22 16] INFO [2022-01-10 12:53:30,134] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,134] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:30,135] jepsen worker 79 - jepsen.util 259 :ok :read [26 35] INFO [2022-01-10 12:53:30,135] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:30,136] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,136] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:30,137] jepsen worker 79 - jepsen.util 259 :ok :read [26 35] INFO [2022-01-10 12:53:30,137] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,137] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:30,138] jepsen worker 79 - jepsen.util 259 :ok :read [26 35] INFO [2022-01-10 12:53:30,138] jepsen worker 68 - jepsen.util 68 :invoke :write [22 46] INFO [2022-01-10 12:53:30,139] jepsen worker 79 - jepsen.util 259 :invoke :write [26 41] INFO [2022-01-10 12:53:30,139] jepsen worker 33 - jepsen.util 123 :ok :write [11 42] INFO [2022-01-10 12:53:30,140] jepsen worker 53 - jepsen.util 143 :ok :write [17 5] INFO [2022-01-10 12:53:30,140] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,140] jepsen worker 53 - jepsen.util 143 :invoke :write [17 39] INFO [2022-01-10 12:53:30,141] jepsen worker 33 - jepsen.util 123 :ok :read [11 42] INFO [2022-01-10 12:53:30,142] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,142] jepsen worker 33 - jepsen.util 123 :ok :read [11 42] INFO [2022-01-10 12:53:30,143] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,144] jepsen worker 33 - jepsen.util 123 :ok :read [11 42] INFO [2022-01-10 12:53:30,145] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,146] jepsen worker 33 - jepsen.util 123 :ok :read [11 42] INFO [2022-01-10 12:53:30,146] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,148] jepsen worker 33 - jepsen.util 123 :ok :read [11 42] INFO [2022-01-10 12:53:30,148] jepsen worker 33 - jepsen.util 123 :invoke :write [11 26] INFO [2022-01-10 12:53:30,149] jepsen worker 70 - jepsen.util 160 :ok :write [23 25] INFO [2022-01-10 12:53:30,150] jepsen worker 84 - jepsen.util 84 :ok :write [28 10] INFO [2022-01-10 12:53:30,150] jepsen worker 70 - jepsen.util 160 :invoke :write [23 31] INFO [2022-01-10 12:53:30,151] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:30,151] jepsen worker 84 - jepsen.util 84 :ok :read [28 10] INFO [2022-01-10 12:53:30,152] jepsen worker 84 - jepsen.util 84 :invoke :write [28 46] INFO [2022-01-10 12:53:30,153] jepsen worker 75 - jepsen.util 165 :ok :write [25 31] INFO [2022-01-10 12:53:30,154] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,154] jepsen worker 75 - jepsen.util 165 :ok :read [25 31] INFO [2022-01-10 12:53:30,155] jepsen worker 38 - jepsen.util 38 :ok :write [12 22] INFO [2022-01-10 12:53:30,155] jepsen worker 75 - jepsen.util 165 :invoke :write [25 5] INFO [2022-01-10 12:53:30,156] jepsen worker 38 - jepsen.util 38 :invoke :write [12 24] INFO [2022-01-10 12:53:30,159] jepsen worker 15 - jepsen.util 15 :ok :write [5 29] INFO [2022-01-10 12:53:30,160] jepsen worker 88 - jepsen.util 178 :ok :write [29 37] INFO [2022-01-10 12:53:30,160] jepsen worker 15 - jepsen.util 15 :invoke :write [5 3] INFO [2022-01-10 12:53:30,160] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,161] jepsen worker 88 - jepsen.util 178 :ok :read [29 37] INFO [2022-01-10 12:53:30,162] jepsen worker 88 - jepsen.util 178 :invoke :write [29 22] INFO [2022-01-10 12:53:30,162] jepsen worker 3 - jepsen.util 3 :ok :write [1 35] INFO [2022-01-10 12:53:30,162] jepsen worker 3 - jepsen.util 3 :invoke :write [1 33] INFO [2022-01-10 12:53:30,163] jepsen worker 61 - jepsen.util 61 :ok :write [20 49] INFO [2022-01-10 12:53:30,164] jepsen worker 61 - jepsen.util 61 :invoke :write [20 41] INFO [2022-01-10 12:53:30,166] jepsen worker 13 - jepsen.util 13 :ok :write [4 19] INFO [2022-01-10 12:53:30,167] jepsen worker 22 - jepsen.util 112 :ok :write [7 33] INFO [2022-01-10 12:53:30,167] jepsen worker 13 - jepsen.util 13 :invoke :write [4 17] INFO [2022-01-10 12:53:30,168] jepsen worker 22 - jepsen.util 112 :invoke :write [7 21] INFO [2022-01-10 12:53:30,169] jepsen worker 41 - jepsen.util 41 :ok :write [13 17] INFO [2022-01-10 12:53:30,169] jepsen worker 41 - jepsen.util 41 :invoke :write [13 33] INFO [2022-01-10 12:53:30,175] jepsen worker 43 - jepsen.util 43 :ok :write [14 0] INFO [2022-01-10 12:53:30,175] jepsen worker 43 - jepsen.util 43 :invoke :write [14 46] INFO [2022-01-10 12:53:30,176] jepsen worker 79 - jepsen.util 259 :ok :write [26 41] INFO [2022-01-10 12:53:30,176] jepsen worker 79 - jepsen.util 259 :invoke :write [26 12] INFO [2022-01-10 12:53:30,177] jepsen worker 50 - jepsen.util 50 :ok :write [16 21] INFO [2022-01-10 12:53:30,178] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,179] jepsen worker 50 - jepsen.util 50 :ok :read [16 21] INFO [2022-01-10 12:53:30,180] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,180] jepsen worker 50 - jepsen.util 50 :ok :read [16 21] INFO [2022-01-10 12:53:30,181] jepsen worker 50 - jepsen.util 50 :invoke :write [16 23] INFO [2022-01-10 12:53:30,184] jepsen worker 53 - jepsen.util 143 :ok :write [17 39] INFO [2022-01-10 12:53:30,185] jepsen worker 53 - jepsen.util 143 :invoke :write [17 15] INFO [2022-01-10 12:53:30,191] jepsen worker 70 - jepsen.util 160 :ok :write [23 31] INFO [2022-01-10 12:53:30,192] jepsen worker 70 - jepsen.util 160 :invoke :write [23 25] INFO [2022-01-10 12:53:30,198] jepsen worker 38 - jepsen.util 38 :ok :write [12 24] INFO [2022-01-10 12:53:30,198] jepsen worker 7 - jepsen.util 7 :ok :write [2 44] INFO [2022-01-10 12:53:30,198] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,199] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,199] jepsen worker 38 - jepsen.util 38 :ok :read [12 24] INFO [2022-01-10 12:53:30,200] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,200] jepsen worker 7 - jepsen.util 7 :ok :read [2 44] INFO [2022-01-10 12:53:30,201] jepsen worker 7 - jepsen.util 7 :invoke :write [2 15] INFO [2022-01-10 12:53:30,201] jepsen worker 38 - jepsen.util 38 :ok :read [12 24] INFO [2022-01-10 12:53:30,201] jepsen worker 0 - jepsen.util 180 :ok :write [0 3] INFO [2022-01-10 12:53:30,201] jepsen worker 38 - jepsen.util 38 :invoke :write [12 35] INFO [2022-01-10 12:53:30,202] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,202] jepsen worker 84 - jepsen.util 84 :ok :write [28 46] INFO [2022-01-10 12:53:30,203] jepsen worker 0 - jepsen.util 180 :ok :read [0 3] INFO [2022-01-10 12:53:30,203] jepsen worker 84 - jepsen.util 84 :invoke :write [28 13] INFO [2022-01-10 12:53:30,203] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,204] jepsen worker 0 - jepsen.util 180 :ok :read [0 3] INFO [2022-01-10 12:53:30,204] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,205] jepsen worker 81 - jepsen.util 171 :ok :write [27 47] INFO [2022-01-10 12:53:30,205] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,206] jepsen worker 0 - jepsen.util 180 :ok :read [0 3] INFO [2022-01-10 12:53:30,206] jepsen worker 3 - jepsen.util 3 :ok :write [1 33] INFO [2022-01-10 12:53:30,206] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,206] jepsen worker 81 - jepsen.util 171 :ok :read [27 47] INFO [2022-01-10 12:53:30,207] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,207] jepsen worker 0 - jepsen.util 180 :ok :read [0 3] INFO [2022-01-10 12:53:30,207] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,208] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:30,208] jepsen worker 0 - jepsen.util 180 :invoke :write [0 17] INFO [2022-01-10 12:53:30,208] jepsen worker 81 - jepsen.util 171 :ok :read [27 47] INFO [2022-01-10 12:53:30,208] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,209] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:30,209] jepsen worker 81 - jepsen.util 171 :invoke :write [27 36] INFO [2022-01-10 12:53:30,209] jepsen worker 3 - jepsen.util 3 :invoke :write [1 37] INFO [2022-01-10 12:53:30,210] jepsen worker 68 - jepsen.util 68 :ok :write [22 46] INFO [2022-01-10 12:53:30,210] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:30,211] jepsen worker 75 - jepsen.util 165 :ok :write [25 5] INFO [2022-01-10 12:53:30,212] jepsen worker 75 - jepsen.util 165 :invoke :write [25 46] INFO [2022-01-10 12:53:30,212] jepsen worker 68 - jepsen.util 68 :ok :read [22 46] INFO [2022-01-10 12:53:30,212] jepsen worker 12 - jepsen.util 192 :ok :write [4 28] INFO [2022-01-10 12:53:30,213] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:30,213] jepsen worker 68 - jepsen.util 68 :invoke :write [22 37] INFO [2022-01-10 12:53:30,213] jepsen worker 33 - jepsen.util 123 :ok :write [11 26] INFO [2022-01-10 12:53:30,213] jepsen worker 12 - jepsen.util 192 :ok :read [4 28] INFO [2022-01-10 12:53:30,214] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,214] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:30,215] jepsen worker 33 - jepsen.util 123 :ok :read [11 26] INFO [2022-01-10 12:53:30,215] jepsen worker 12 - jepsen.util 192 :ok :read [4 28] INFO [2022-01-10 12:53:30,215] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,216] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:30,216] jepsen worker 79 - jepsen.util 259 :ok :write [26 12] INFO [2022-01-10 12:53:30,216] jepsen worker 33 - jepsen.util 123 :ok :read [11 26] INFO [2022-01-10 12:53:30,217] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,217] jepsen worker 33 - jepsen.util 123 :invoke :write [11 46] INFO [2022-01-10 12:53:30,218] jepsen worker 79 - jepsen.util 259 :ok :read [26 12] INFO [2022-01-10 12:53:30,218] jepsen worker 12 - jepsen.util 192 :ok :read [4 28] INFO [2022-01-10 12:53:30,218] jepsen worker 79 - jepsen.util 259 :invoke :write [26 3] INFO [2022-01-10 12:53:30,219] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:30,220] jepsen worker 15 - jepsen.util 15 :ok :write [5 3] INFO [2022-01-10 12:53:30,220] jepsen worker 12 - jepsen.util 192 :ok :read [4 28] INFO [2022-01-10 12:53:30,220] jepsen worker 61 - jepsen.util 61 :ok :write [20 41] INFO [2022-01-10 12:53:30,221] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:30,222] jepsen worker 12 - jepsen.util 192 :invoke :write [4 28] INFO [2022-01-10 12:53:30,222] jepsen worker 61 - jepsen.util 61 :ok :read [20 41] INFO [2022-01-10 12:53:30,222] jepsen worker 15 - jepsen.util 15 :invoke :write [5 27] INFO [2022-01-10 12:53:30,223] jepsen worker 61 - jepsen.util 61 :invoke :write [20 46] INFO [2022-01-10 12:53:30,223] jepsen worker 53 - jepsen.util 143 :ok :write [17 15] INFO [2022-01-10 12:53:30,224] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,225] jepsen worker 53 - jepsen.util 143 :ok :read [17 15] INFO [2022-01-10 12:53:30,225] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,226] jepsen worker 53 - jepsen.util 143 :ok :read [17 15] INFO [2022-01-10 12:53:30,227] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,227] jepsen worker 22 - jepsen.util 112 :ok :write [7 21] INFO [2022-01-10 12:53:30,228] jepsen worker 70 - jepsen.util 160 :ok :write [23 25] INFO [2022-01-10 12:53:30,228] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,228] jepsen worker 53 - jepsen.util 143 :ok :read [17 15] INFO [2022-01-10 12:53:30,228] jepsen worker 70 - jepsen.util 160 :invoke :write [23 23] INFO [2022-01-10 12:53:30,229] jepsen worker 22 - jepsen.util 112 :ok :read [7 21] INFO [2022-01-10 12:53:30,229] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,230] jepsen worker 88 - jepsen.util 178 :ok :write [29 22] INFO [2022-01-10 12:53:30,230] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,230] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,231] jepsen worker 22 - jepsen.util 112 :ok :read [7 21] INFO [2022-01-10 12:53:30,231] jepsen worker 53 - jepsen.util 143 :ok :read [17 15] INFO [2022-01-10 12:53:30,232] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,232] jepsen worker 88 - jepsen.util 178 :ok :read [29 22] INFO [2022-01-10 12:53:30,232] jepsen worker 53 - jepsen.util 143 :invoke :write [17 11] INFO [2022-01-10 12:53:30,233] jepsen worker 22 - jepsen.util 112 :ok :read [7 21] INFO [2022-01-10 12:53:30,233] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,233] jepsen worker 43 - jepsen.util 43 :ok :write [14 46] INFO [2022-01-10 12:53:30,234] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,234] jepsen worker 88 - jepsen.util 178 :ok :read [29 22] INFO [2022-01-10 12:53:30,234] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,235] jepsen worker 22 - jepsen.util 112 :ok :read [7 21] INFO [2022-01-10 12:53:30,235] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,235] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:30,236] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,236] jepsen worker 88 - jepsen.util 178 :ok :read [29 22] INFO [2022-01-10 12:53:30,236] jepsen worker 41 - jepsen.util 41 :ok :write [13 33] INFO [2022-01-10 12:53:30,236] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,237] jepsen worker 22 - jepsen.util 112 :ok :read [7 21] INFO [2022-01-10 12:53:30,237] jepsen worker 43 - jepsen.util 43 :ok :read [14 46] INFO [2022-01-10 12:53:30,237] jepsen worker 88 - jepsen.util 178 :invoke :write [29 37] INFO [2022-01-10 12:53:30,238] jepsen worker 22 - jepsen.util 112 :invoke :write [7 28] INFO [2022-01-10 12:53:30,239] jepsen worker 43 - jepsen.util 43 :invoke :write [14 10] INFO [2022-01-10 12:53:30,239] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,240] jepsen worker 41 - jepsen.util 41 :ok :read [13 33] INFO [2022-01-10 12:53:30,240] jepsen worker 38 - jepsen.util 38 :ok :write [12 35] INFO [2022-01-10 12:53:30,241] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,241] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,241] jepsen worker 84 - jepsen.util 84 :ok :write [28 13] INFO [2022-01-10 12:53:30,242] jepsen worker 41 - jepsen.util 41 :ok :read [13 33] INFO [2022-01-10 12:53:30,242] jepsen worker 84 - jepsen.util 84 :invoke :write [28 8] INFO [2022-01-10 12:53:30,242] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,242] jepsen worker 38 - jepsen.util 38 :ok :read [12 35] INFO [2022-01-10 12:53:30,243] jepsen worker 41 - jepsen.util 41 :ok :read [13 33] INFO [2022-01-10 12:53:30,243] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,243] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,243] jepsen worker 50 - jepsen.util 50 :ok :write [16 23] INFO [2022-01-10 12:53:30,244] jepsen worker 50 - jepsen.util 50 :invoke :write [16 47] INFO [2022-01-10 12:53:30,244] jepsen worker 41 - jepsen.util 41 :ok :read [13 33] INFO [2022-01-10 12:53:30,244] jepsen worker 3 - jepsen.util 3 :ok :write [1 37] INFO [2022-01-10 12:53:30,245] jepsen worker 38 - jepsen.util 38 :ok :read [12 35] INFO [2022-01-10 12:53:30,245] jepsen worker 3 - jepsen.util 3 :invoke :write [1 14] INFO [2022-01-10 12:53:30,245] jepsen worker 41 - jepsen.util 41 :invoke :write [13 14] INFO [2022-01-10 12:53:30,246] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,247] jepsen worker 38 - jepsen.util 38 :ok :read [12 35] INFO [2022-01-10 12:53:30,247] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,249] jepsen worker 38 - jepsen.util 38 :ok :read [12 35] INFO [2022-01-10 12:53:30,249] jepsen worker 38 - jepsen.util 38 :invoke :write [12 26] INFO [2022-01-10 12:53:30,250] jepsen worker 13 - jepsen.util 13 :ok :write [4 17] INFO [2022-01-10 12:53:30,251] jepsen worker 7 - jepsen.util 7 :ok :write [2 15] INFO [2022-01-10 12:53:30,251] jepsen worker 13 - jepsen.util 13 :invoke :write [4 20] INFO [2022-01-10 12:53:30,251] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,252] jepsen worker 7 - jepsen.util 7 :ok :read [2 15] INFO [2022-01-10 12:53:30,253] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,254] jepsen worker 7 - jepsen.util 7 :ok :read [2 15] INFO [2022-01-10 12:53:30,254] jepsen worker 7 - jepsen.util 7 :invoke :write [2 2] INFO [2022-01-10 12:53:30,257] jepsen worker 75 - jepsen.util 165 :ok :write [25 46] INFO [2022-01-10 12:53:30,257] jepsen worker 75 - jepsen.util 165 :invoke :write [25 32] INFO [2022-01-10 12:53:30,264] jepsen worker 0 - jepsen.util 180 :ok :write [0 17] INFO [2022-01-10 12:53:30,264] jepsen worker 0 - jepsen.util 180 :invoke :write [0 38] INFO [2022-01-10 12:53:30,266] jepsen worker 81 - jepsen.util 171 :ok :write [27 36] INFO [2022-01-10 12:53:30,267] jepsen worker 81 - jepsen.util 171 :invoke :write [27 21] INFO [2022-01-10 12:53:30,272] jepsen worker 68 - jepsen.util 68 :ok :write [22 37] INFO [2022-01-10 12:53:30,272] jepsen worker 68 - jepsen.util 68 :invoke :write [22 28] INFO [2022-01-10 12:53:30,289] jepsen worker 88 - jepsen.util 178 :ok :write [29 37] INFO [2022-01-10 12:53:30,289] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,290] jepsen worker 88 - jepsen.util 178 :ok :read [29 37] INFO [2022-01-10 12:53:30,290] jepsen worker 88 - jepsen.util 178 :invoke :write [29 11] INFO [2022-01-10 12:53:30,310] jepsen worker 79 - jepsen.util 259 :ok :write [26 3] INFO [2022-01-10 12:53:30,310] jepsen worker 79 - jepsen.util 259 :invoke :write [26 41] INFO [2022-01-10 12:53:30,311] jepsen worker 33 - jepsen.util 123 :ok :write [11 46] INFO [2022-01-10 12:53:30,311] jepsen worker 33 - jepsen.util 123 :invoke :write [11 33] INFO [2022-01-10 12:53:30,315] jepsen worker 53 - jepsen.util 143 :ok :write [17 11] INFO [2022-01-10 12:53:30,316] jepsen worker 53 - jepsen.util 143 :invoke :write [17 10] INFO [2022-01-10 12:53:30,316] jepsen worker 61 - jepsen.util 61 :ok :write [20 46] INFO [2022-01-10 12:53:30,316] jepsen worker 41 - jepsen.util 41 :ok :write [13 14] INFO [2022-01-10 12:53:30,317] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:30,317] jepsen worker 41 - jepsen.util 41 :invoke :write [13 28] INFO [2022-01-10 12:53:30,318] jepsen worker 61 - jepsen.util 61 :ok :read [20 46] INFO [2022-01-10 12:53:30,319] jepsen worker 61 - jepsen.util 61 :invoke :write [20 6] INFO [2022-01-10 12:53:30,322] jepsen worker 70 - jepsen.util 160 :ok :write [23 23] INFO [2022-01-10 12:53:30,322] jepsen worker 70 - jepsen.util 160 :invoke :write [23 17] INFO [2022-01-10 12:53:30,326] jepsen worker 15 - jepsen.util 15 :ok :write [5 27] INFO [2022-01-10 12:53:30,327] jepsen worker 15 - jepsen.util 15 :invoke :write [5 39] INFO [2022-01-10 12:53:30,329] jepsen worker 38 - jepsen.util 38 :ok :write [12 26] INFO [2022-01-10 12:53:30,329] jepsen worker 0 - jepsen.util 180 :ok :write [0 38] INFO [2022-01-10 12:53:30,329] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,330] jepsen worker 22 - jepsen.util 112 :ok :write [7 28] INFO [2022-01-10 12:53:30,330] jepsen worker 0 - jepsen.util 180 :invoke :write [0 49] INFO [2022-01-10 12:53:30,330] jepsen worker 38 - jepsen.util 38 :ok :read [12 26] INFO [2022-01-10 12:53:30,331] jepsen worker 22 - jepsen.util 112 :invoke :write [7 4] INFO [2022-01-10 12:53:30,331] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,332] jepsen worker 38 - jepsen.util 38 :ok :read [12 26] INFO [2022-01-10 12:53:30,332] jepsen worker 38 - jepsen.util 38 :invoke :write [12 30] INFO [2022-01-10 12:53:30,333] jepsen worker 43 - jepsen.util 43 :ok :write [14 10] INFO [2022-01-10 12:53:30,333] jepsen worker 88 - jepsen.util 178 :ok :write [29 11] INFO [2022-01-10 12:53:30,334] jepsen worker 43 - jepsen.util 43 :invoke :write [14 41] INFO [2022-01-10 12:53:30,334] jepsen worker 84 - jepsen.util 84 :ok :write [28 8] INFO [2022-01-10 12:53:30,335] jepsen worker 21 - jepsen.util 111 :info :write [7 44] :RequestTimeoutException INFO [2022-01-10 12:53:30,335] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,335] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:30,335] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,336] jepsen worker 88 - jepsen.util 178 :ok :read [29 11] INFO [2022-01-10 12:53:30,336] jepsen worker 84 - jepsen.util 84 :ok :read [28 8] INFO [2022-01-10 12:53:30,336] jepsen worker 84 - jepsen.util 84 :invoke :write [28 13] INFO [2022-01-10 12:53:30,336] jepsen worker 21 - jepsen.util 201 :ok :read [7 28] INFO [2022-01-10 12:53:30,337] jepsen worker 88 - jepsen.util 178 :invoke :write [29 45] INFO [2022-01-10 12:53:30,337] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,338] jepsen worker 21 - jepsen.util 201 :ok :read [7 28] INFO [2022-01-10 12:53:30,338] jepsen worker 50 - jepsen.util 50 :ok :write [16 47] INFO [2022-01-10 12:53:30,338] jepsen worker 21 - jepsen.util 201 :invoke :write [7 45] INFO [2022-01-10 12:53:30,339] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:53:30,341] jepsen worker 7 - jepsen.util 7 :ok :write [2 2] INFO [2022-01-10 12:53:30,342] jepsen worker 3 - jepsen.util 3 :ok :write [1 14] INFO [2022-01-10 12:53:30,342] jepsen worker 41 - jepsen.util 41 :ok :write [13 28] INFO [2022-01-10 12:53:30,342] jepsen worker 7 - jepsen.util 7 :invoke :write [2 35] INFO [2022-01-10 12:53:30,343] jepsen worker 3 - jepsen.util 3 :invoke :write [1 7] INFO [2022-01-10 12:53:30,343] jepsen worker 41 - jepsen.util 41 :invoke :write [13 26] INFO [2022-01-10 12:53:30,345] jepsen worker 75 - jepsen.util 165 :ok :write [25 32] INFO [2022-01-10 12:53:30,346] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,347] jepsen worker 75 - jepsen.util 165 :ok :read [25 32] INFO [2022-01-10 12:53:30,347] jepsen worker 75 - jepsen.util 165 :invoke :write [25 9] INFO [2022-01-10 12:53:30,349] jepsen worker 13 - jepsen.util 13 :ok :write [4 20] INFO [2022-01-10 12:53:30,350] jepsen worker 13 - jepsen.util 13 :invoke :write [4 30] INFO [2022-01-10 12:53:30,351] jepsen worker 81 - jepsen.util 171 :ok :write [27 21] INFO [2022-01-10 12:53:30,352] jepsen worker 81 - jepsen.util 171 :invoke :write [27 12] INFO [2022-01-10 12:53:30,354] jepsen worker 0 - jepsen.util 180 :ok :write [0 49] INFO [2022-01-10 12:53:30,355] jepsen worker 0 - jepsen.util 180 :invoke :write [0 30] INFO [2022-01-10 12:53:30,356] jepsen worker 79 - jepsen.util 259 :ok :write [26 41] INFO [2022-01-10 12:53:30,356] jepsen worker 68 - jepsen.util 68 :ok :write [22 28] INFO [2022-01-10 12:53:30,357] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,357] jepsen worker 79 - jepsen.util 259 :ok :read [26 41] INFO [2022-01-10 12:53:30,357] jepsen worker 68 - jepsen.util 68 :invoke :write [22 28] INFO [2022-01-10 12:53:30,358] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,359] jepsen worker 79 - jepsen.util 259 :ok :read [26 41] INFO [2022-01-10 12:53:30,360] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,361] jepsen worker 33 - jepsen.util 123 :ok :write [11 33] INFO [2022-01-10 12:53:30,361] jepsen worker 79 - jepsen.util 259 :ok :read [26 41] INFO [2022-01-10 12:53:30,361] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,362] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,362] jepsen worker 53 - jepsen.util 143 :ok :write [17 10] INFO [2022-01-10 12:53:30,363] jepsen worker 33 - jepsen.util 123 :ok :read [11 33] INFO [2022-01-10 12:53:30,363] jepsen worker 79 - jepsen.util 259 :ok :read [26 41] INFO [2022-01-10 12:53:30,363] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,364] jepsen worker 33 - jepsen.util 123 :invoke :write [11 24] INFO [2022-01-10 12:53:30,364] jepsen worker 53 - jepsen.util 143 :ok :read [17 10] INFO [2022-01-10 12:53:30,364] jepsen worker 79 - jepsen.util 259 :invoke :write [26 31] INFO [2022-01-10 12:53:30,364] jepsen worker 88 - jepsen.util 178 :ok :write [29 45] INFO [2022-01-10 12:53:30,364] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,365] jepsen worker 88 - jepsen.util 178 :invoke :write [29 17] INFO [2022-01-10 12:53:30,366] jepsen worker 53 - jepsen.util 143 :ok :read [17 10] INFO [2022-01-10 12:53:30,366] jepsen worker 61 - jepsen.util 61 :ok :write [20 6] INFO [2022-01-10 12:53:30,366] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,367] jepsen worker 61 - jepsen.util 61 :invoke :write [20 23] INFO [2022-01-10 12:53:30,369] jepsen worker 53 - jepsen.util 143 :ok :read [17 10] INFO [2022-01-10 12:53:30,369] jepsen worker 70 - jepsen.util 160 :ok :write [23 17] INFO [2022-01-10 12:53:30,370] jepsen worker 53 - jepsen.util 143 :invoke :write [17 3] INFO [2022-01-10 12:53:30,370] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:30,371] jepsen worker 70 - jepsen.util 160 :ok :read [23 17] INFO [2022-01-10 12:53:30,371] jepsen worker 15 - jepsen.util 15 :ok :write [5 39] INFO [2022-01-10 12:53:30,372] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:30,372] jepsen worker 15 - jepsen.util 15 :invoke :write [5 21] INFO [2022-01-10 12:53:30,373] jepsen worker 70 - jepsen.util 160 :ok :read [23 17] INFO [2022-01-10 12:53:30,373] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:30,374] jepsen worker 70 - jepsen.util 160 :ok :read [23 17] INFO [2022-01-10 12:53:30,375] jepsen worker 70 - jepsen.util 160 :invoke :write [23 37] INFO [2022-01-10 12:53:30,376] jepsen worker 38 - jepsen.util 38 :ok :write [12 30] INFO [2022-01-10 12:53:30,377] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:30,378] jepsen worker 22 - jepsen.util 112 :ok :write [7 4] INFO [2022-01-10 12:53:30,379] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,379] jepsen worker 41 - jepsen.util 41 :ok :write [13 26] INFO [2022-01-10 12:53:30,380] jepsen worker 22 - jepsen.util 112 :ok :read [7 4] INFO [2022-01-10 12:53:30,380] jepsen worker 41 - jepsen.util 41 :invoke :write [13 44] INFO [2022-01-10 12:53:30,381] jepsen worker 22 - jepsen.util 112 :invoke :write [7 28] INFO [2022-01-10 12:53:30,383] jepsen worker 84 - jepsen.util 84 :ok :write [28 13] INFO [2022-01-10 12:53:30,383] jepsen worker 84 - jepsen.util 84 :invoke :write [28 15] INFO [2022-01-10 12:53:30,385] jepsen worker 43 - jepsen.util 43 :ok :write [14 41] INFO [2022-01-10 12:53:30,386] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,387] jepsen worker 43 - jepsen.util 43 :ok :read [14 41] INFO [2022-01-10 12:53:30,388] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,389] jepsen worker 43 - jepsen.util 43 :ok :read [14 41] INFO [2022-01-10 12:53:30,389] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,390] jepsen worker 43 - jepsen.util 43 :ok :read [14 41] INFO [2022-01-10 12:53:30,391] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,392] jepsen worker 43 - jepsen.util 43 :ok :read [14 41] INFO [2022-01-10 12:53:30,392] jepsen worker 43 - jepsen.util 43 :invoke :write [14 18] INFO [2022-01-10 12:53:30,392] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:53:30,393] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,394] jepsen worker 50 - jepsen.util 50 :ok :read [16 30] INFO [2022-01-10 12:53:30,394] jepsen worker 13 - jepsen.util 13 :ok :write [4 30] INFO [2022-01-10 12:53:30,395] jepsen worker 13 - jepsen.util 13 :invoke :write [4 23] INFO [2022-01-10 12:53:30,395] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,396] jepsen worker 3 - jepsen.util 3 :ok :write [1 7] INFO [2022-01-10 12:53:30,396] jepsen worker 50 - jepsen.util 50 :ok :read [16 30] INFO [2022-01-10 12:53:30,396] jepsen worker 3 - jepsen.util 3 :invoke :write [1 10] INFO [2022-01-10 12:53:30,397] jepsen worker 7 - jepsen.util 7 :ok :write [2 35] INFO [2022-01-10 12:53:30,397] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,397] jepsen worker 7 - jepsen.util 7 :invoke :write [2 48] INFO [2022-01-10 12:53:30,398] jepsen worker 50 - jepsen.util 50 :ok :read [16 30] INFO [2022-01-10 12:53:30,399] jepsen worker 50 - jepsen.util 50 :invoke :write [16 27] INFO [2022-01-10 12:53:30,402] jepsen worker 79 - jepsen.util 259 :ok :write [26 31] INFO [2022-01-10 12:53:30,403] jepsen worker 75 - jepsen.util 165 :ok :write [25 9] INFO [2022-01-10 12:53:30,403] jepsen worker 79 - jepsen.util 259 :invoke :write [26 1] INFO [2022-01-10 12:53:30,404] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,405] jepsen worker 75 - jepsen.util 165 :ok :read [25 9] INFO [2022-01-10 12:53:30,406] jepsen worker 75 - jepsen.util 165 :invoke :write [25 16] INFO [2022-01-10 12:53:30,407] jepsen worker 0 - jepsen.util 180 :ok :write [0 30] INFO [2022-01-10 12:53:30,407] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,408] jepsen worker 0 - jepsen.util 180 :ok :read [0 30] INFO [2022-01-10 12:53:30,408] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,409] jepsen worker 0 - jepsen.util 180 :ok :read [0 30] INFO [2022-01-10 12:53:30,410] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,410] jepsen worker 52 - jepsen.util 52 :ok :write [17 45] INFO [2022-01-10 12:53:30,410] jepsen worker 81 - jepsen.util 171 :ok :write [27 12] INFO [2022-01-10 12:53:30,410] jepsen worker 0 - jepsen.util 180 :ok :read [0 30] INFO [2022-01-10 12:53:30,411] jepsen worker 52 - jepsen.util 52 :invoke :write [17 15] INFO [2022-01-10 12:53:30,411] jepsen worker 81 - jepsen.util 171 :invoke :write [27 19] INFO [2022-01-10 12:53:30,411] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,412] jepsen worker 0 - jepsen.util 180 :ok :read [0 30] INFO [2022-01-10 12:53:30,413] jepsen worker 0 - jepsen.util 180 :invoke :write [0 2] INFO [2022-01-10 12:53:30,416] jepsen worker 68 - jepsen.util 68 :ok :write [22 28] INFO [2022-01-10 12:53:30,417] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:30,418] jepsen worker 70 - jepsen.util 160 :ok :write [23 37] INFO [2022-01-10 12:53:30,418] jepsen worker 68 - jepsen.util 68 :ok :read [22 28] INFO [2022-01-10 12:53:30,418] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:30,419] jepsen worker 68 - jepsen.util 68 :invoke :write [22 44] INFO [2022-01-10 12:53:30,419] jepsen worker 70 - jepsen.util 160 :ok :read [23 37] INFO [2022-01-10 12:53:30,420] jepsen worker 70 - jepsen.util 160 :invoke :write [23 15] INFO [2022-01-10 12:53:30,421] jepsen worker 88 - jepsen.util 178 :ok :write [29 17] INFO [2022-01-10 12:53:30,422] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,423] jepsen worker 88 - jepsen.util 178 :ok :read [29 17] INFO [2022-01-10 12:53:30,423] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,424] jepsen worker 33 - jepsen.util 123 :ok :write [11 24] INFO [2022-01-10 12:53:30,424] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,424] jepsen worker 88 - jepsen.util 178 :ok :read [29 17] INFO [2022-01-10 12:53:30,425] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,425] jepsen worker 33 - jepsen.util 123 :ok :read [11 24] INFO [2022-01-10 12:53:30,426] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,427] jepsen worker 88 - jepsen.util 178 :ok :read [29 17] INFO [2022-01-10 12:53:30,427] jepsen worker 33 - jepsen.util 123 :ok :read [11 24] INFO [2022-01-10 12:53:30,427] jepsen worker 88 - jepsen.util 178 :invoke :write [29 43] INFO [2022-01-10 12:53:30,427] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:30,428] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,428] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,429] jepsen worker 33 - jepsen.util 123 :ok :read [11 24] INFO [2022-01-10 12:53:30,429] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:30,429] jepsen worker 38 - jepsen.util 38 :invoke :write [12 34] INFO [2022-01-10 12:53:30,430] jepsen worker 33 - jepsen.util 123 :invoke :write [11 24] INFO [2022-01-10 12:53:30,430] jepsen worker 41 - jepsen.util 41 :ok :write [13 44] INFO [2022-01-10 12:53:30,431] jepsen worker 41 - jepsen.util 41 :invoke :write [13 8] INFO [2022-01-10 12:53:30,432] jepsen worker 61 - jepsen.util 61 :ok :write [20 23] INFO [2022-01-10 12:53:30,432] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:30,433] jepsen worker 61 - jepsen.util 61 :ok :read [20 23] INFO [2022-01-10 12:53:30,434] jepsen worker 61 - jepsen.util 61 :invoke :write [20 39] INFO [2022-01-10 12:53:30,435] jepsen worker 84 - jepsen.util 84 :ok :write [28 15] INFO [2022-01-10 12:53:30,435] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:30,436] jepsen worker 84 - jepsen.util 84 :ok :read [28 15] INFO [2022-01-10 12:53:30,437] jepsen worker 84 - jepsen.util 84 :invoke :write [28 10] INFO [2022-01-10 12:53:30,437] jepsen worker 15 - jepsen.util 15 :ok :write [5 21] INFO [2022-01-10 12:53:30,438] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,439] jepsen worker 13 - jepsen.util 13 :ok :write [4 23] INFO [2022-01-10 12:53:30,439] jepsen worker 15 - jepsen.util 15 :ok :read [5 21] INFO [2022-01-10 12:53:30,440] jepsen worker 13 - jepsen.util 13 :invoke :write [4 41] INFO [2022-01-10 12:53:30,440] jepsen worker 15 - jepsen.util 15 :invoke :write [5 19] INFO [2022-01-10 12:53:30,442] jepsen worker 21 - jepsen.util 201 :ok :write [7 45] INFO [2022-01-10 12:53:30,442] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,444] jepsen worker 21 - jepsen.util 201 :ok :read [7 45] INFO [2022-01-10 12:53:30,444] jepsen worker 21 - jepsen.util 201 :invoke :write [7 2] INFO [2022-01-10 12:53:30,446] jepsen worker 0 - jepsen.util 180 :ok :write [0 2] INFO [2022-01-10 12:53:30,446] jepsen worker 3 - jepsen.util 3 :ok :write [1 10] INFO [2022-01-10 12:53:30,447] jepsen worker 0 - jepsen.util 180 :invoke :write [0 20] INFO [2022-01-10 12:53:30,447] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,448] jepsen worker 3 - jepsen.util 3 :ok :read [1 10] INFO [2022-01-10 12:53:30,448] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,448] jepsen worker 43 - jepsen.util 43 :ok :write [14 18] INFO [2022-01-10 12:53:30,449] jepsen worker 43 - jepsen.util 43 :invoke :write [14 8] INFO [2022-01-10 12:53:30,449] jepsen worker 3 - jepsen.util 3 :ok :read [1 10] INFO [2022-01-10 12:53:30,450] jepsen worker 3 - jepsen.util 3 :invoke :write [1 7] INFO [2022-01-10 12:53:30,454] jepsen worker 79 - jepsen.util 259 :ok :write [26 1] INFO [2022-01-10 12:53:30,455] jepsen worker 50 - jepsen.util 50 :ok :write [16 27] INFO [2022-01-10 12:53:30,455] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,456] jepsen worker 50 - jepsen.util 50 :invoke :write [16 20] INFO [2022-01-10 12:53:30,456] jepsen worker 79 - jepsen.util 259 :ok :read [26 1] INFO [2022-01-10 12:53:30,457] jepsen worker 79 - jepsen.util 259 :invoke :write [26 7] INFO [2022-01-10 12:53:30,460] jepsen worker 7 - jepsen.util 7 :ok :write [2 48] INFO [2022-01-10 12:53:30,460] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,461] jepsen worker 52 - jepsen.util 52 :ok :write [17 15] INFO [2022-01-10 12:53:30,461] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:30,461] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,462] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,463] jepsen worker 88 - jepsen.util 178 :ok :write [29 43] INFO [2022-01-10 12:53:30,463] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:30,463] jepsen worker 52 - jepsen.util 52 :ok :read [17 15] INFO [2022-01-10 12:53:30,464] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,464] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:30,465] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,465] jepsen worker 75 - jepsen.util 165 :ok :write [25 16] INFO [2022-01-10 12:53:30,465] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,466] jepsen worker 52 - jepsen.util 52 :ok :read [17 15] INFO [2022-01-10 12:53:30,466] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:30,467] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,468] jepsen worker 52 - jepsen.util 52 :ok :read [17 15] INFO [2022-01-10 12:53:30,468] jepsen worker 75 - jepsen.util 165 :invoke :write [25 5] INFO [2022-01-10 12:53:30,468] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,469] jepsen worker 70 - jepsen.util 160 :ok :write [23 15] INFO [2022-01-10 12:53:30,469] jepsen worker 52 - jepsen.util 52 :ok :read [17 15] INFO [2022-01-10 12:53:30,469] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,470] jepsen worker 41 - jepsen.util 41 :ok :write [13 8] INFO [2022-01-10 12:53:30,470] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:30,470] jepsen worker 70 - jepsen.util 160 :invoke :write [23 18] INFO [2022-01-10 12:53:30,470] jepsen worker 81 - jepsen.util 171 :ok :write [27 19] INFO [2022-01-10 12:53:30,471] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,472] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:30,472] jepsen worker 88 - jepsen.util 178 :invoke :write [29 8] INFO [2022-01-10 12:53:30,473] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,474] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:30,474] jepsen worker 41 - jepsen.util 41 :invoke :write [13 35] INFO [2022-01-10 12:53:30,475] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,476] jepsen worker 52 - jepsen.util 52 :invoke :write [17 17] INFO [2022-01-10 12:53:30,476] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:30,476] jepsen worker 68 - jepsen.util 68 :ok :write [22 44] INFO [2022-01-10 12:53:30,477] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,478] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:30,478] jepsen worker 68 - jepsen.util 68 :invoke :write [22 27] INFO [2022-01-10 12:53:30,478] jepsen worker 38 - jepsen.util 38 :ok :write [12 34] INFO [2022-01-10 12:53:30,478] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,479] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,479] jepsen worker 35 - jepsen.util 125 :ok :write [11 22] INFO [2022-01-10 12:53:30,479] jepsen worker 81 - jepsen.util 171 :ok :read [27 19] INFO [2022-01-10 12:53:30,480] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:30,480] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,481] jepsen worker 7 - jepsen.util 7 :invoke :write [2 42] INFO [2022-01-10 12:53:30,482] jepsen worker 38 - jepsen.util 38 :ok :read [12 34] INFO [2022-01-10 12:53:30,482] jepsen worker 81 - jepsen.util 171 :invoke :write [27 47] INFO [2022-01-10 12:53:30,483] jepsen worker 35 - jepsen.util 125 :invoke :write [11 10] INFO [2022-01-10 12:53:30,484] jepsen worker 38 - jepsen.util 38 :invoke :write [12 25] INFO [2022-01-10 12:53:30,484] jepsen worker 0 - jepsen.util 180 :ok :write [0 20] INFO [2022-01-10 12:53:30,484] jepsen worker 61 - jepsen.util 61 :ok :write [20 39] INFO [2022-01-10 12:53:30,485] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,486] jepsen worker 61 - jepsen.util 61 :invoke :write [20 7] INFO [2022-01-10 12:53:30,486] jepsen worker 0 - jepsen.util 180 :ok :read [0 20] INFO [2022-01-10 12:53:30,486] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,487] jepsen worker 0 - jepsen.util 180 :ok :read [0 20] INFO [2022-01-10 12:53:30,488] jepsen worker 84 - jepsen.util 84 :ok :write [28 10] INFO [2022-01-10 12:53:30,488] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,489] jepsen worker 84 - jepsen.util 84 :invoke :write [28 44] INFO [2022-01-10 12:53:30,489] jepsen worker 0 - jepsen.util 180 :ok :read [0 20] INFO [2022-01-10 12:53:30,490] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,491] jepsen worker 15 - jepsen.util 15 :ok :write [5 19] INFO [2022-01-10 12:53:30,491] jepsen worker 0 - jepsen.util 180 :ok :read [0 20] INFO [2022-01-10 12:53:30,492] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,492] jepsen worker 15 - jepsen.util 15 :ok :read [5 19] INFO [2022-01-10 12:53:30,492] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,493] jepsen worker 0 - jepsen.util 180 :ok :read [0 20] INFO [2022-01-10 12:53:30,493] jepsen worker 15 - jepsen.util 15 :invoke :write [5 17] INFO [2022-01-10 12:53:30,494] jepsen worker 0 - jepsen.util 180 :invoke :write [0 29] INFO [2022-01-10 12:53:30,495] jepsen worker 13 - jepsen.util 13 :ok :write [4 41] INFO [2022-01-10 12:53:30,495] jepsen worker 22 - jepsen.util 112 :ok :write [7 28] INFO [2022-01-10 12:53:30,496] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:30,496] jepsen worker 22 - jepsen.util 112 :invoke :write [7 27] INFO [2022-01-10 12:53:30,500] jepsen worker 3 - jepsen.util 3 :ok :write [1 7] INFO [2022-01-10 12:53:30,501] jepsen worker 3 - jepsen.util 3 :invoke :write [1 29] INFO [2022-01-10 12:53:30,502] jepsen worker 43 - jepsen.util 43 :ok :write [14 8] INFO [2022-01-10 12:53:30,502] jepsen worker 43 - jepsen.util 43 :invoke :write [14 38] INFO [2022-01-10 12:53:30,505] jepsen worker 88 - jepsen.util 178 :ok :write [29 8] INFO [2022-01-10 12:53:30,506] jepsen worker 88 - jepsen.util 178 :invoke :write [29 5] INFO [2022-01-10 12:53:30,508] jepsen worker 50 - jepsen.util 50 :ok :write [16 20] INFO [2022-01-10 12:53:30,508] jepsen worker 79 - jepsen.util 259 :ok :write [26 7] INFO [2022-01-10 12:53:30,508] jepsen worker 50 - jepsen.util 50 :invoke :write [16 16] INFO [2022-01-10 12:53:30,509] jepsen worker 79 - jepsen.util 259 :invoke :write [26 38] INFO [2022-01-10 12:53:30,511] jepsen worker 7 - jepsen.util 7 :ok :write [2 42] INFO [2022-01-10 12:53:30,511] jepsen worker 41 - jepsen.util 41 :ok :write [13 35] INFO [2022-01-10 12:53:30,511] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,512] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,512] jepsen worker 7 - jepsen.util 7 :ok :read [2 42] INFO [2022-01-10 12:53:30,512] jepsen worker 41 - jepsen.util 41 :ok :read [13 35] INFO [2022-01-10 12:53:30,512] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,513] jepsen worker 41 - jepsen.util 41 :invoke :write [13 24] INFO [2022-01-10 12:53:30,513] jepsen worker 7 - jepsen.util 7 :ok :read [2 42] INFO [2022-01-10 12:53:30,514] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,514] jepsen worker 52 - jepsen.util 52 :ok :write [17 17] INFO [2022-01-10 12:53:30,515] jepsen worker 7 - jepsen.util 7 :ok :read [2 42] INFO [2022-01-10 12:53:30,516] jepsen worker 52 - jepsen.util 52 :invoke :write [17 42] INFO [2022-01-10 12:53:30,516] jepsen worker 7 - jepsen.util 7 :invoke :write [2 14] INFO [2022-01-10 12:53:30,517] jepsen worker 13 - jepsen.util 13 :ok :read [4 41] INFO [2022-01-10 12:53:30,518] jepsen worker 13 - jepsen.util 13 :invoke :write [4 40] INFO [2022-01-10 12:53:30,519] jepsen worker 75 - jepsen.util 165 :ok :write [25 5] INFO [2022-01-10 12:53:30,520] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,521] jepsen worker 75 - jepsen.util 165 :ok :read [25 5] INFO [2022-01-10 12:53:30,522] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,522] jepsen worker 70 - jepsen.util 160 :ok :write [23 18] INFO [2022-01-10 12:53:30,522] jepsen worker 70 - jepsen.util 160 :invoke :write [23 15] INFO [2022-01-10 12:53:30,522] jepsen worker 75 - jepsen.util 165 :ok :read [25 5] INFO [2022-01-10 12:53:30,523] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,524] jepsen worker 75 - jepsen.util 165 :ok :read [25 5] INFO [2022-01-10 12:53:30,524] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,525] jepsen worker 75 - jepsen.util 165 :ok :read [25 5] INFO [2022-01-10 12:53:30,525] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,526] jepsen worker 81 - jepsen.util 171 :ok :write [27 47] INFO [2022-01-10 12:53:30,526] jepsen worker 81 - jepsen.util 171 :invoke :write [27 41] INFO [2022-01-10 12:53:30,527] jepsen worker 75 - jepsen.util 165 :ok :read [25 5] INFO [2022-01-10 12:53:30,528] jepsen worker 38 - jepsen.util 38 :ok :write [12 25] INFO [2022-01-10 12:53:30,528] jepsen worker 75 - jepsen.util 165 :invoke :write [25 41] INFO [2022-01-10 12:53:30,529] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,530] jepsen worker 38 - jepsen.util 38 :ok :read [12 25] INFO [2022-01-10 12:53:30,530] jepsen worker 38 - jepsen.util 38 :invoke :write [12 31] INFO [2022-01-10 12:53:30,533] jepsen worker 0 - jepsen.util 180 :ok :write [0 29] INFO [2022-01-10 12:53:30,533] jepsen worker 0 - jepsen.util 180 :invoke :write [0 5] INFO [2022-01-10 12:53:30,534] jepsen worker 84 - jepsen.util 84 :ok :write [28 44] INFO [2022-01-10 12:53:30,535] jepsen worker 84 - jepsen.util 84 :invoke :write [28 37] INFO [2022-01-10 12:53:30,545] jepsen worker 88 - jepsen.util 178 :ok :write [29 5] INFO [2022-01-10 12:53:30,545] jepsen worker 3 - jepsen.util 3 :ok :write [1 29] INFO [2022-01-10 12:53:30,545] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,546] jepsen worker 3 - jepsen.util 3 :invoke :write [1 30] INFO [2022-01-10 12:53:30,547] jepsen worker 88 - jepsen.util 178 :ok :read [29 5] INFO [2022-01-10 12:53:30,547] jepsen worker 88 - jepsen.util 178 :invoke :write [29 36] INFO [2022-01-10 12:53:30,552] jepsen worker 79 - jepsen.util 259 :ok :write [26 38] INFO [2022-01-10 12:53:30,553] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,553] jepsen worker 79 - jepsen.util 259 :ok :read [26 38] INFO [2022-01-10 12:53:30,554] jepsen worker 79 - jepsen.util 259 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,555] jepsen worker 41 - jepsen.util 41 :ok :write [13 24] INFO [2022-01-10 12:53:30,555] jepsen worker 79 - jepsen.util 259 :ok :read [26 38] INFO [2022-01-10 12:53:30,555] jepsen worker 41 - jepsen.util 41 :invoke :write [13 38] INFO [2022-01-10 12:53:30,556] jepsen worker 79 - jepsen.util 259 :invoke :write [26 29] INFO [2022-01-10 12:53:30,559] jepsen worker 52 - jepsen.util 52 :ok :write [17 42] INFO [2022-01-10 12:53:30,559] jepsen worker 52 - jepsen.util 52 :invoke :write [17 40] INFO [2022-01-10 12:53:30,569] jepsen worker 0 - jepsen.util 180 :ok :write [0 5] INFO [2022-01-10 12:53:30,569] jepsen worker 0 - jepsen.util 180 :invoke :write [0 41] INFO [2022-01-10 12:53:30,573] jepsen worker 88 - jepsen.util 178 :ok :write [29 36] INFO [2022-01-10 12:53:30,574] jepsen worker 88 - jepsen.util 178 :invoke :write [29 28] INFO [2022-01-10 12:53:30,586] jepsen worker 17 - jepsen.util 197 :info :write [5 46] :RequestTimeoutException INFO [2022-01-10 12:53:30,587] jepsen worker 17 - jepsen.util 287 :invoke :write [5 19] INFO [2022-01-10 12:53:30,627] jepsen worker 19 - jepsen.util 199 :info :write [6 11] :RequestTimeoutException INFO [2022-01-10 12:53:30,627] jepsen worker 19 - jepsen.util 289 :invoke :write [6 35] INFO [2022-01-10 12:53:30,630] jepsen worker 68 - jepsen.util 68 :ok :write [22 27] INFO [2022-01-10 12:53:30,630] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:30,631] jepsen worker 68 - jepsen.util 68 :ok :read [22 27] INFO [2022-01-10 12:53:30,632] jepsen worker 13 - jepsen.util 13 :ok :write [4 40] INFO [2022-01-10 12:53:30,632] jepsen worker 68 - jepsen.util 68 :invoke :write [22 14] INFO [2022-01-10 12:53:30,633] jepsen worker 13 - jepsen.util 13 :invoke :write [4 3] INFO [2022-01-10 12:53:30,635] jepsen worker 33 - jepsen.util 123 :ok :write [11 24] INFO [2022-01-10 12:53:30,636] jepsen worker 33 - jepsen.util 123 :invoke :write [11 26] INFO [2022-01-10 12:53:30,643] jepsen worker 61 - jepsen.util 61 :ok :write [20 7] INFO [2022-01-10 12:53:30,644] jepsen worker 61 - jepsen.util 61 :invoke :write [20 28] INFO [2022-01-10 12:53:30,648] jepsen worker 15 - jepsen.util 15 :ok :write [5 17] INFO [2022-01-10 12:53:30,649] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,650] jepsen worker 15 - jepsen.util 15 :ok :read [5 17] INFO [2022-01-10 12:53:30,650] jepsen worker 15 - jepsen.util 15 :invoke :write [5 33] INFO [2022-01-10 12:53:30,652] jepsen worker 22 - jepsen.util 112 :ok :write [7 27] INFO [2022-01-10 12:53:30,653] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,654] jepsen worker 22 - jepsen.util 112 :ok :read [7 27] INFO [2022-01-10 12:53:30,655] jepsen worker 22 - jepsen.util 112 :invoke :write [7 12] INFO [2022-01-10 12:53:30,658] jepsen worker 43 - jepsen.util 43 :ok :write [14 38] INFO [2022-01-10 12:53:30,658] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,659] jepsen worker 43 - jepsen.util 43 :ok :read [14 38] INFO [2022-01-10 12:53:30,660] jepsen worker 43 - jepsen.util 43 :invoke :write [14 33] INFO [2022-01-10 12:53:30,663] jepsen worker 50 - jepsen.util 50 :ok :write [16 16] INFO [2022-01-10 12:53:30,664] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,664] jepsen worker 50 - jepsen.util 50 :ok :read [16 16] INFO [2022-01-10 12:53:30,665] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,666] jepsen worker 50 - jepsen.util 50 :ok :read [16 16] INFO [2022-01-10 12:53:30,666] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,667] jepsen worker 7 - jepsen.util 7 :ok :write [2 14] INFO [2022-01-10 12:53:30,667] jepsen worker 50 - jepsen.util 50 :ok :read [16 16] INFO [2022-01-10 12:53:30,668] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,668] jepsen worker 50 - jepsen.util 50 :invoke :write [16 39] INFO [2022-01-10 12:53:30,669] jepsen worker 7 - jepsen.util 7 :ok :read [2 14] INFO [2022-01-10 12:53:30,669] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,670] jepsen worker 7 - jepsen.util 7 :ok :read [2 14] INFO [2022-01-10 12:53:30,670] jepsen worker 7 - jepsen.util 7 :invoke :write [2 37] INFO [2022-01-10 12:53:30,672] jepsen worker 75 - jepsen.util 165 :ok :write [25 41] INFO [2022-01-10 12:53:30,673] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,674] jepsen worker 75 - jepsen.util 165 :ok :read [25 41] INFO [2022-01-10 12:53:30,674] jepsen worker 75 - jepsen.util 165 :invoke :write [25 16] INFO [2022-01-10 12:53:30,679] jepsen worker 81 - jepsen.util 171 :ok :write [27 41] INFO [2022-01-10 12:53:30,680] jepsen worker 81 - jepsen.util 171 :invoke :write [27 14] INFO [2022-01-10 12:53:30,682] jepsen worker 68 - jepsen.util 68 :ok :write [22 14] INFO [2022-01-10 12:53:30,682] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:30,683] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:30,683] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:30,684] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:30,685] jepsen worker 68 - jepsen.util 68 :invoke :write [22 23] INFO [2022-01-10 12:53:30,686] jepsen worker 33 - jepsen.util 123 :ok :write [11 26] INFO [2022-01-10 12:53:30,686] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:30,687] jepsen worker 33 - jepsen.util 123 :ok :read [11 26] INFO [2022-01-10 12:53:30,688] jepsen worker 33 - jepsen.util 123 :invoke :write [11 14] INFO [2022-01-10 12:53:30,694] jepsen worker 15 - jepsen.util 15 :ok :write [5 33] INFO [2022-01-10 12:53:30,695] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,695] jepsen worker 15 - jepsen.util 15 :ok :read [5 33] INFO [2022-01-10 12:53:30,696] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,697] jepsen worker 15 - jepsen.util 15 :ok :read [5 33] INFO [2022-01-10 12:53:30,697] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,698] jepsen worker 15 - jepsen.util 15 :ok :read [5 33] INFO [2022-01-10 12:53:30,699] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,700] jepsen worker 15 - jepsen.util 15 :ok :read [5 33] INFO [2022-01-10 12:53:30,700] jepsen worker 15 - jepsen.util 15 :invoke :write [5 22] INFO [2022-01-10 12:53:30,707] jepsen worker 50 - jepsen.util 50 :ok :write [16 39] INFO [2022-01-10 12:53:30,707] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,708] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:30,709] jepsen worker 50 - jepsen.util 50 :invoke :write [16 24] INFO [2022-01-10 12:53:30,710] jepsen worker 7 - jepsen.util 7 :ok :write [2 37] INFO [2022-01-10 12:53:30,711] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,712] jepsen worker 7 - jepsen.util 7 :ok :read [2 37] INFO [2022-01-10 12:53:30,712] jepsen worker 7 - jepsen.util 7 :invoke :write [2 46] INFO [2022-01-10 12:53:30,713] jepsen worker 70 - jepsen.util 160 :ok :write [23 15] INFO [2022-01-10 12:53:30,714] jepsen worker 70 - jepsen.util 160 :invoke :write [23 5] INFO [2022-01-10 12:53:30,715] jepsen worker 41 - jepsen.util 41 :ok :write [13 38] INFO [2022-01-10 12:53:30,715] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,716] jepsen worker 41 - jepsen.util 41 :ok :read [13 38] INFO [2022-01-10 12:53:30,716] jepsen worker 41 - jepsen.util 41 :invoke :write [13 17] INFO [2022-01-10 12:53:30,720] jepsen worker 38 - jepsen.util 38 :ok :write [12 31] INFO [2022-01-10 12:53:30,720] jepsen worker 81 - jepsen.util 171 :ok :write [27 14] INFO [2022-01-10 12:53:30,721] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,721] jepsen worker 0 - jepsen.util 180 :ok :write [0 41] INFO [2022-01-10 12:53:30,721] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,721] jepsen worker 38 - jepsen.util 38 :ok :read [12 31] INFO [2022-01-10 12:53:30,722] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,722] jepsen worker 81 - jepsen.util 171 :ok :read [27 14] INFO [2022-01-10 12:53:30,723] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,723] jepsen worker 0 - jepsen.util 180 :ok :read [0 41] INFO [2022-01-10 12:53:30,723] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,723] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,724] jepsen worker 81 - jepsen.util 171 :ok :read [27 14] INFO [2022-01-10 12:53:30,724] jepsen worker 38 - jepsen.util 38 :ok :read [12 31] INFO [2022-01-10 12:53:30,724] jepsen worker 0 - jepsen.util 180 :ok :read [0 41] INFO [2022-01-10 12:53:30,724] jepsen worker 68 - jepsen.util 68 :ok :write [22 23] INFO [2022-01-10 12:53:30,725] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,725] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,726] jepsen worker 81 - jepsen.util 171 :ok :read [27 14] INFO [2022-01-10 12:53:30,726] jepsen worker 68 - jepsen.util 68 :invoke :write [22 37] INFO [2022-01-10 12:53:30,726] jepsen worker 38 - jepsen.util 38 :ok :read [12 31] INFO [2022-01-10 12:53:30,727] jepsen worker 84 - jepsen.util 84 :ok :write [28 37] INFO [2022-01-10 12:53:30,727] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,727] jepsen worker 88 - jepsen.util 178 :ok :write [29 28] INFO [2022-01-10 12:53:30,727] jepsen worker 38 - jepsen.util 38 :invoke :write [12 39] INFO [2022-01-10 12:53:30,727] jepsen worker 81 - jepsen.util 171 :ok :read [27 14] INFO [2022-01-10 12:53:30,728] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,729] jepsen worker 0 - jepsen.util 180 :ok :read [0 41] INFO [2022-01-10 12:53:30,729] jepsen worker 88 - jepsen.util 178 :invoke :write [29 1] INFO [2022-01-10 12:53:30,729] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,730] jepsen worker 33 - jepsen.util 123 :ok :write [11 14] INFO [2022-01-10 12:53:30,730] jepsen worker 81 - jepsen.util 171 :invoke :write [27 45] INFO [2022-01-10 12:53:30,730] jepsen worker 0 - jepsen.util 180 :ok :read [0 41] INFO [2022-01-10 12:53:30,730] jepsen worker 84 - jepsen.util 84 :invoke :write [28 11] INFO [2022-01-10 12:53:30,730] jepsen worker 3 - jepsen.util 3 :ok :write [1 30] INFO [2022-01-10 12:53:30,731] jepsen worker 33 - jepsen.util 123 :invoke :write [11 39] INFO [2022-01-10 12:53:30,732] jepsen worker 0 - jepsen.util 180 :invoke :write [0 35] INFO [2022-01-10 12:53:30,732] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,733] jepsen worker 3 - jepsen.util 3 :ok :read [1 30] INFO [2022-01-10 12:53:30,733] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,734] jepsen worker 3 - jepsen.util 3 :ok :read [1 30] INFO [2022-01-10 12:53:30,735] jepsen worker 15 - jepsen.util 15 :ok :write [5 22] INFO [2022-01-10 12:53:30,735] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,735] jepsen worker 61 - jepsen.util 61 :ok :write [20 28] INFO [2022-01-10 12:53:30,735] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,736] jepsen worker 3 - jepsen.util 3 :ok :read [1 30] INFO [2022-01-10 12:53:30,736] jepsen worker 61 - jepsen.util 61 :invoke :write [20 12] INFO [2022-01-10 12:53:30,736] jepsen worker 15 - jepsen.util 15 :ok :read [5 22] INFO [2022-01-10 12:53:30,736] jepsen worker 3 - jepsen.util 3 :invoke :write [1 19] INFO [2022-01-10 12:53:30,737] jepsen worker 80 - jepsen.util 260 :ok :write [26 49] INFO [2022-01-10 12:53:30,737] jepsen worker 15 - jepsen.util 15 :invoke :write [5 36] INFO [2022-01-10 12:53:30,737] jepsen worker 50 - jepsen.util 50 :ok :write [16 24] INFO [2022-01-10 12:53:30,738] jepsen worker 80 - jepsen.util 260 :invoke :write [26 19] INFO [2022-01-10 12:53:30,738] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,739] jepsen worker 50 - jepsen.util 50 :ok :read [16 24] INFO [2022-01-10 12:53:30,740] jepsen worker 50 - jepsen.util 50 :invoke :write [16 40] INFO [2022-01-10 12:53:30,741] jepsen worker 22 - jepsen.util 112 :ok :write [7 12] INFO [2022-01-10 12:53:30,741] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,742] jepsen worker 22 - jepsen.util 112 :ok :read [7 12] INFO [2022-01-10 12:53:30,743] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,744] jepsen worker 22 - jepsen.util 112 :ok :read [7 12] INFO [2022-01-10 12:53:30,744] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,745] jepsen worker 22 - jepsen.util 112 :ok :read [7 12] INFO [2022-01-10 12:53:30,745] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,746] jepsen worker 43 - jepsen.util 43 :ok :write [14 33] INFO [2022-01-10 12:53:30,746] jepsen worker 22 - jepsen.util 112 :ok :read [7 12] INFO [2022-01-10 12:53:30,746] jepsen worker 43 - jepsen.util 43 :invoke :write [14 22] INFO [2022-01-10 12:53:30,747] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,748] jepsen worker 22 - jepsen.util 112 :ok :read [7 12] INFO [2022-01-10 12:53:30,748] jepsen worker 52 - jepsen.util 52 :ok :write [17 40] INFO [2022-01-10 12:53:30,748] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,749] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,749] jepsen worker 22 - jepsen.util 112 :ok :read [7 12] INFO [2022-01-10 12:53:30,749] jepsen worker 52 - jepsen.util 52 :ok :read [17 40] INFO [2022-01-10 12:53:30,749] jepsen worker 22 - jepsen.util 112 :invoke :write [7 34] INFO [2022-01-10 12:53:30,750] jepsen worker 75 - jepsen.util 165 :ok :write [25 16] INFO [2022-01-10 12:53:30,750] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,750] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,751] jepsen worker 52 - jepsen.util 52 :ok :read [17 40] INFO [2022-01-10 12:53:30,751] jepsen worker 75 - jepsen.util 165 :ok :read [25 16] INFO [2022-01-10 12:53:30,751] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:30,752] jepsen worker 75 - jepsen.util 165 :invoke :write [25 48] INFO [2022-01-10 12:53:30,752] jepsen worker 52 - jepsen.util 52 :ok :read [17 40] INFO [2022-01-10 12:53:30,752] jepsen worker 52 - jepsen.util 52 :invoke :write [17 3] INFO [2022-01-10 12:53:30,756] jepsen worker 13 - jepsen.util 13 :ok :write [4 3] INFO [2022-01-10 12:53:30,756] jepsen worker 13 - jepsen.util 13 :invoke :write [4 27] INFO [2022-01-10 12:53:30,763] jepsen worker 7 - jepsen.util 7 :ok :write [2 46] INFO [2022-01-10 12:53:30,763] jepsen worker 7 - jepsen.util 7 :invoke :write [2 5] INFO [2022-01-10 12:53:30,766] jepsen worker 70 - jepsen.util 160 :ok :write [23 5] INFO [2022-01-10 12:53:30,766] jepsen worker 41 - jepsen.util 41 :ok :write [13 17] INFO [2022-01-10 12:53:30,766] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:30,767] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,767] jepsen worker 70 - jepsen.util 160 :ok :read [23 5] INFO [2022-01-10 12:53:30,767] jepsen worker 41 - jepsen.util 41 :ok :read [13 17] INFO [2022-01-10 12:53:30,768] jepsen worker 70 - jepsen.util 160 :invoke :write [23 43] INFO [2022-01-10 12:53:30,769] jepsen worker 11 - jepsen.util 191 :info :write [3 13] :RequestTimeoutException INFO [2022-01-10 12:53:30,769] jepsen worker 41 - jepsen.util 41 :invoke :write [13 6] INFO [2022-01-10 12:53:30,769] jepsen worker 11 - jepsen.util 281 :invoke :read [3 nil] INFO [2022-01-10 12:53:30,771] jepsen worker 61 - jepsen.util 61 :ok :write [20 12] INFO [2022-01-10 12:53:30,772] jepsen worker 81 - jepsen.util 171 :ok :write [27 45] INFO [2022-01-10 12:53:30,772] jepsen worker 61 - jepsen.util 61 :invoke :write [20 47] INFO [2022-01-10 12:53:30,772] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,774] jepsen worker 81 - jepsen.util 171 :ok :read [27 45] INFO [2022-01-10 12:53:30,774] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,774] jepsen worker 30 - jepsen.util 210 :info :write [10 35] :RequestTimeoutException INFO [2022-01-10 12:53:30,775] jepsen worker 81 - jepsen.util 171 :ok :read [27 45] INFO [2022-01-10 12:53:30,775] jepsen worker 30 - jepsen.util 300 :invoke :read [10 nil] INFO [2022-01-10 12:53:30,776] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,777] jepsen worker 81 - jepsen.util 171 :ok :read [27 45] INFO [2022-01-10 12:53:30,777] jepsen worker 81 - jepsen.util 171 :invoke :write [27 25] INFO [2022-01-10 12:53:30,780] jepsen worker 0 - jepsen.util 180 :ok :write [0 35] INFO [2022-01-10 12:53:30,780] jepsen worker 22 - jepsen.util 112 :ok :write [7 34] INFO [2022-01-10 12:53:30,781] jepsen worker 0 - jepsen.util 180 :invoke :write [0 26] INFO [2022-01-10 12:53:30,781] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,782] jepsen worker 22 - jepsen.util 112 :ok :read [7 34] INFO [2022-01-10 12:53:30,783] jepsen worker 84 - jepsen.util 84 :ok :write [28 11] INFO [2022-01-10 12:53:30,783] jepsen worker 38 - jepsen.util 38 :ok :write [12 39] INFO [2022-01-10 12:53:30,783] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,784] jepsen worker 38 - jepsen.util 38 :invoke :write [12 10] INFO [2022-01-10 12:53:30,784] jepsen worker 22 - jepsen.util 112 :ok :read [7 34] INFO [2022-01-10 12:53:30,785] jepsen worker 84 - jepsen.util 84 :invoke :write [28 39] INFO [2022-01-10 12:53:30,786] jepsen worker 22 - jepsen.util 112 :invoke :write [7 48] INFO [2022-01-10 12:53:30,787] jepsen worker 43 - jepsen.util 43 :ok :write [14 22] INFO [2022-01-10 12:53:30,787] jepsen worker 3 - jepsen.util 3 :ok :write [1 19] INFO [2022-01-10 12:53:30,788] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,788] jepsen worker 68 - jepsen.util 68 :ok :write [22 37] INFO [2022-01-10 12:53:30,788] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,789] jepsen worker 43 - jepsen.util 43 :ok :read [14 22] INFO [2022-01-10 12:53:30,789] jepsen worker 3 - jepsen.util 3 :ok :read [1 19] INFO [2022-01-10 12:53:30,790] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,790] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:30,791] jepsen worker 43 - jepsen.util 43 :invoke :write [14 11] INFO [2022-01-10 12:53:30,791] jepsen worker 3 - jepsen.util 3 :ok :read [1 19] INFO [2022-01-10 12:53:30,791] jepsen worker 68 - jepsen.util 68 :ok :read [22 37] INFO [2022-01-10 12:53:30,792] jepsen worker 3 - jepsen.util 3 :invoke :write [1 48] INFO [2022-01-10 12:53:30,792] jepsen worker 68 - jepsen.util 68 :invoke :write [22 30] INFO [2022-01-10 12:53:30,793] jepsen worker 75 - jepsen.util 165 :ok :write [25 48] INFO [2022-01-10 12:53:30,793] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,794] jepsen worker 75 - jepsen.util 165 :ok :read [25 48] INFO [2022-01-10 12:53:30,794] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,795] jepsen worker 75 - jepsen.util 165 :ok :read [25 48] INFO [2022-01-10 12:53:30,795] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:30,796] jepsen worker 75 - jepsen.util 165 :ok :read [25 48] INFO [2022-01-10 12:53:30,797] jepsen worker 75 - jepsen.util 165 :invoke :write [25 12] INFO [2022-01-10 12:53:30,798] jepsen worker 80 - jepsen.util 260 :ok :write [26 19] INFO [2022-01-10 12:53:30,798] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,799] jepsen worker 88 - jepsen.util 178 :ok :write [29 1] INFO [2022-01-10 12:53:30,799] jepsen worker 88 - jepsen.util 178 :invoke :write [29 25] INFO [2022-01-10 12:53:30,800] jepsen worker 80 - jepsen.util 260 :ok :read [26 19] INFO [2022-01-10 12:53:30,800] jepsen worker 80 - jepsen.util 260 :invoke :write [26 4] INFO [2022-01-10 12:53:30,803] jepsen worker 33 - jepsen.util 123 :ok :write [11 39] INFO [2022-01-10 12:53:30,803] jepsen worker 33 - jepsen.util 123 :invoke :write [11 21] INFO [2022-01-10 12:53:30,804] jepsen worker 52 - jepsen.util 52 :ok :write [17 3] INFO [2022-01-10 12:53:30,805] jepsen worker 15 - jepsen.util 15 :ok :write [5 36] INFO [2022-01-10 12:53:30,805] jepsen worker 52 - jepsen.util 52 :invoke :write [17 45] INFO [2022-01-10 12:53:30,805] jepsen worker 15 - jepsen.util 15 :invoke :write [5 29] INFO [2022-01-10 12:53:30,809] jepsen worker 50 - jepsen.util 50 :ok :write [16 40] INFO [2022-01-10 12:53:30,810] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:30,811] jepsen worker 50 - jepsen.util 50 :ok :read [16 40] INFO [2022-01-10 12:53:30,811] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:53:30,812] jepsen worker 61 - jepsen.util 61 :ok :write [20 47] INFO [2022-01-10 12:53:30,812] jepsen worker 13 - jepsen.util 13 :ok :write [4 27] INFO [2022-01-10 12:53:30,812] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:30,813] jepsen worker 13 - jepsen.util 13 :invoke :write [4 0] INFO [2022-01-10 12:53:30,813] jepsen worker 61 - jepsen.util 61 :ok :read [20 47] INFO [2022-01-10 12:53:30,814] jepsen worker 73 - jepsen.util 253 :info :write [24 23] :RequestTimeoutException INFO [2022-01-10 12:53:30,814] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:30,815] jepsen worker 73 - jepsen.util 343 :invoke :write [24 43] INFO [2022-01-10 12:53:30,816] jepsen worker 61 - jepsen.util 61 :ok :read [20 47] INFO [2022-01-10 12:53:30,816] jepsen worker 61 - jepsen.util 61 :invoke :write [20 5] INFO [2022-01-10 12:53:30,817] jepsen worker 7 - jepsen.util 7 :ok :write [2 5] INFO [2022-01-10 12:53:30,818] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,819] jepsen worker 22 - jepsen.util 112 :ok :write [7 48] INFO [2022-01-10 12:53:30,819] jepsen worker 7 - jepsen.util 7 :ok :read [2 5] INFO [2022-01-10 12:53:30,820] jepsen worker 22 - jepsen.util 112 :invoke :write [7 8] INFO [2022-01-10 12:53:30,820] jepsen worker 41 - jepsen.util 41 :ok :write [13 6] INFO [2022-01-10 12:53:30,820] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,821] jepsen worker 70 - jepsen.util 160 :ok :write [23 43] INFO [2022-01-10 12:53:30,821] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,821] jepsen worker 7 - jepsen.util 7 :ok :read [2 5] INFO [2022-01-10 12:53:30,822] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:30,822] jepsen worker 41 - jepsen.util 41 :ok :read [13 6] INFO [2022-01-10 12:53:30,822] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,824] jepsen worker 7 - jepsen.util 7 :ok :read [2 5] INFO [2022-01-10 12:53:30,825] jepsen worker 70 - jepsen.util 160 :ok :read [23 43] INFO [2022-01-10 12:53:30,825] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,825] jepsen worker 81 - jepsen.util 171 :ok :write [27 25] INFO [2022-01-10 12:53:30,826] jepsen worker 41 - jepsen.util 41 :ok :read [13 6] INFO [2022-01-10 12:53:30,826] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:30,827] jepsen worker 7 - jepsen.util 7 :ok :read [2 5] INFO [2022-01-10 12:53:30,827] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,828] jepsen worker 38 - jepsen.util 38 :ok :write [12 10] INFO [2022-01-10 12:53:30,828] jepsen worker 7 - jepsen.util 7 :invoke :write [2 34] INFO [2022-01-10 12:53:30,828] jepsen worker 81 - jepsen.util 171 :ok :read [27 25] INFO [2022-01-10 12:53:30,829] jepsen worker 81 - jepsen.util 171 :invoke :write [27 31] INFO [2022-01-10 12:53:30,830] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,830] jepsen worker 43 - jepsen.util 43 :ok :write [14 11] INFO [2022-01-10 12:53:30,830] jepsen worker 70 - jepsen.util 160 :invoke :write [23 1] INFO [2022-01-10 12:53:30,831] jepsen worker 38 - jepsen.util 38 :ok :read [12 10] INFO [2022-01-10 12:53:30,831] jepsen worker 0 - jepsen.util 180 :ok :write [0 26] INFO [2022-01-10 12:53:30,831] jepsen worker 41 - jepsen.util 41 :invoke :write [13 9] INFO [2022-01-10 12:53:30,832] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,833] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,833] jepsen worker 0 - jepsen.util 180 :ok :read [0 26] INFO [2022-01-10 12:53:30,833] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,834] jepsen worker 0 - jepsen.util 180 :invoke :write [0 32] INFO [2022-01-10 12:53:30,834] jepsen worker 43 - jepsen.util 43 :ok :read [14 11] INFO [2022-01-10 12:53:30,834] jepsen worker 38 - jepsen.util 38 :ok :read [12 10] INFO [2022-01-10 12:53:30,835] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,835] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,836] jepsen worker 38 - jepsen.util 38 :ok :read [12 10] INFO [2022-01-10 12:53:30,837] jepsen worker 84 - jepsen.util 84 :ok :write [28 39] INFO [2022-01-10 12:53:30,837] jepsen worker 43 - jepsen.util 43 :ok :read [14 11] INFO [2022-01-10 12:53:30,837] jepsen worker 38 - jepsen.util 38 :invoke :write [12 38] INFO [2022-01-10 12:53:30,837] jepsen worker 75 - jepsen.util 165 :ok :write [25 12] INFO [2022-01-10 12:53:30,838] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,838] jepsen worker 68 - jepsen.util 68 :ok :write [22 30] INFO [2022-01-10 12:53:30,838] jepsen worker 75 - jepsen.util 165 :invoke :write [25 23] INFO [2022-01-10 12:53:30,839] jepsen worker 43 - jepsen.util 43 :ok :read [14 11] INFO [2022-01-10 12:53:30,839] jepsen worker 84 - jepsen.util 84 :invoke :write [28 27] INFO [2022-01-10 12:53:30,840] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,840] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:30,841] jepsen worker 43 - jepsen.util 43 :ok :read [14 11] INFO [2022-01-10 12:53:30,841] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:30,842] jepsen worker 68 - jepsen.util 68 :ok :read [22 30] INFO [2022-01-10 12:53:30,842] jepsen worker 43 - jepsen.util 43 :ok :read [14 11] INFO [2022-01-10 12:53:30,842] jepsen worker 3 - jepsen.util 3 :ok :write [1 48] INFO [2022-01-10 12:53:30,842] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:30,843] jepsen worker 43 - jepsen.util 43 :invoke :write [14 9] INFO [2022-01-10 12:53:30,843] jepsen worker 68 - jepsen.util 68 :ok :read [22 30] INFO [2022-01-10 12:53:30,843] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,844] jepsen worker 68 - jepsen.util 68 :invoke :write [22 45] INFO [2022-01-10 12:53:30,844] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:30,845] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,846] jepsen worker 88 - jepsen.util 178 :ok :write [29 25] INFO [2022-01-10 12:53:30,846] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:30,846] jepsen worker 88 - jepsen.util 178 :invoke :write [29 1] INFO [2022-01-10 12:53:30,847] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,848] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:30,848] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,849] jepsen worker 33 - jepsen.util 123 :ok :write [11 21] INFO [2022-01-10 12:53:30,849] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:30,850] jepsen worker 33 - jepsen.util 123 :invoke :write [11 23] INFO [2022-01-10 12:53:30,850] jepsen worker 61 - jepsen.util 61 :ok :write [20 5] INFO [2022-01-10 12:53:30,851] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,851] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:30,852] jepsen worker 3 - jepsen.util 3 :ok :read [1 48] INFO [2022-01-10 12:53:30,852] jepsen worker 80 - jepsen.util 260 :ok :write [26 4] INFO [2022-01-10 12:53:30,852] jepsen worker 61 - jepsen.util 61 :ok :read [20 5] INFO [2022-01-10 12:53:30,852] jepsen worker 3 - jepsen.util 3 :invoke :write [1 5] INFO [2022-01-10 12:53:30,853] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,853] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:30,853] jepsen worker 15 - jepsen.util 15 :ok :write [5 29] INFO [2022-01-10 12:53:30,854] jepsen worker 80 - jepsen.util 260 :ok :read [26 4] INFO [2022-01-10 12:53:30,854] jepsen worker 61 - jepsen.util 61 :ok :read [20 5] INFO [2022-01-10 12:53:30,854] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,855] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,855] jepsen worker 15 - jepsen.util 15 :ok :read [5 29] INFO [2022-01-10 12:53:30,856] jepsen worker 61 - jepsen.util 61 :invoke :write [20 42] INFO [2022-01-10 12:53:30,856] jepsen worker 80 - jepsen.util 260 :ok :read [26 4] INFO [2022-01-10 12:53:30,856] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:30,857] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,857] jepsen worker 15 - jepsen.util 15 :ok :read [5 29] INFO [2022-01-10 12:53:30,858] jepsen worker 15 - jepsen.util 15 :invoke :write [5 44] INFO [2022-01-10 12:53:30,858] jepsen worker 80 - jepsen.util 260 :ok :read [26 4] INFO [2022-01-10 12:53:30,859] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,860] jepsen worker 80 - jepsen.util 260 :ok :read [26 4] INFO [2022-01-10 12:53:30,860] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:30,861] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:53:30,861] jepsen worker 22 - jepsen.util 112 :ok :write [7 8] INFO [2022-01-10 12:53:30,861] jepsen worker 52 - jepsen.util 52 :ok :write [17 45] INFO [2022-01-10 12:53:30,861] jepsen worker 80 - jepsen.util 260 :ok :read [26 4] INFO [2022-01-10 12:53:30,861] jepsen worker 50 - jepsen.util 50 :invoke :write [16 12] INFO [2022-01-10 12:53:30,862] jepsen worker 52 - jepsen.util 52 :invoke :write [17 6] INFO [2022-01-10 12:53:30,863] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:30,863] jepsen worker 80 - jepsen.util 260 :invoke :write [26 17] INFO [2022-01-10 12:53:30,864] jepsen worker 22 - jepsen.util 112 :ok :read [7 8] INFO [2022-01-10 12:53:30,865] jepsen worker 22 - jepsen.util 112 :invoke :write [7 36] INFO [2022-01-10 12:53:30,865] jepsen worker 7 - jepsen.util 7 :ok :write [2 34] INFO [2022-01-10 12:53:30,866] jepsen worker 7 - jepsen.util 7 :invoke :write [2 4] INFO [2022-01-10 12:53:30,868] jepsen worker 41 - jepsen.util 41 :ok :write [13 9] INFO [2022-01-10 12:53:30,869] jepsen worker 41 - jepsen.util 41 :invoke :write [13 21] INFO [2022-01-10 12:53:30,871] jepsen worker 13 - jepsen.util 13 :ok :write [4 0] INFO [2022-01-10 12:53:30,871] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:30,872] jepsen worker 13 - jepsen.util 13 :ok :read [4 0] INFO [2022-01-10 12:53:30,873] jepsen worker 13 - jepsen.util 13 :invoke :write [4 10] INFO [2022-01-10 12:53:30,873] jepsen worker 81 - jepsen.util 171 :ok :write [27 31] INFO [2022-01-10 12:53:30,873] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,874] jepsen worker 81 - jepsen.util 171 :ok :read [27 31] INFO [2022-01-10 12:53:30,875] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,876] jepsen worker 81 - jepsen.util 171 :ok :read [27 31] INFO [2022-01-10 12:53:30,876] jepsen worker 0 - jepsen.util 180 :ok :write [0 32] INFO [2022-01-10 12:53:30,876] jepsen worker 70 - jepsen.util 160 :ok :write [23 1] INFO [2022-01-10 12:53:30,877] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:30,877] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,877] jepsen worker 81 - jepsen.util 171 :ok :read [27 31] INFO [2022-01-10 12:53:30,878] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:30,878] jepsen worker 0 - jepsen.util 180 :ok :read [0 32] INFO [2022-01-10 12:53:30,878] jepsen worker 81 - jepsen.util 171 :invoke :write [27 28] INFO [2022-01-10 12:53:30,878] jepsen worker 70 - jepsen.util 160 :ok :read [23 1] INFO [2022-01-10 12:53:30,879] jepsen worker 0 - jepsen.util 180 :invoke :write [0 6] INFO [2022-01-10 12:53:30,880] jepsen worker 70 - jepsen.util 160 :invoke :write [23 12] INFO [2022-01-10 12:53:30,883] jepsen worker 38 - jepsen.util 38 :ok :write [12 38] INFO [2022-01-10 12:53:30,884] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,885] jepsen worker 38 - jepsen.util 38 :ok :read [12 38] INFO [2022-01-10 12:53:30,885] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,885] jepsen worker 88 - jepsen.util 178 :ok :write [29 1] INFO [2022-01-10 12:53:30,886] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,886] jepsen worker 38 - jepsen.util 38 :ok :read [12 38] INFO [2022-01-10 12:53:30,887] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:30,887] jepsen worker 88 - jepsen.util 178 :ok :read [29 1] INFO [2022-01-10 12:53:30,888] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,888] jepsen worker 88 - jepsen.util 178 :ok :read [29 1] INFO [2022-01-10 12:53:30,889] jepsen worker 88 - jepsen.util 178 :invoke :write [29 24] INFO [2022-01-10 12:53:30,891] jepsen worker 84 - jepsen.util 84 :ok :write [28 27] INFO [2022-01-10 12:53:30,892] jepsen worker 84 - jepsen.util 84 :invoke :write [28 49] INFO [2022-01-10 12:53:30,897] jepsen worker 3 - jepsen.util 3 :ok :write [1 5] INFO [2022-01-10 12:53:30,898] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,898] jepsen worker 3 - jepsen.util 3 :ok :read [1 5] INFO [2022-01-10 12:53:30,899] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,899] jepsen worker 3 - jepsen.util 3 :ok :read [1 5] INFO [2022-01-10 12:53:30,900] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,901] jepsen worker 3 - jepsen.util 3 :ok :read [1 5] INFO [2022-01-10 12:53:30,901] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,902] jepsen worker 3 - jepsen.util 3 :ok :read [1 5] INFO [2022-01-10 12:53:30,902] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,903] jepsen worker 3 - jepsen.util 3 :ok :read [1 5] INFO [2022-01-10 12:53:30,904] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,904] jepsen worker 39 - jepsen.util 39 :info :write [13 44] :RequestTimeoutException INFO [2022-01-10 12:53:30,905] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,905] jepsen worker 3 - jepsen.util 3 :ok :read [1 5] INFO [2022-01-10 12:53:30,905] jepsen worker 3 - jepsen.util 3 :invoke :write [1 19] INFO [2022-01-10 12:53:30,906] jepsen worker 39 - jepsen.util 129 :ok :read [13 9] INFO [2022-01-10 12:53:30,906] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,907] jepsen worker 39 - jepsen.util 129 :ok :read [13 9] INFO [2022-01-10 12:53:30,908] jepsen worker 82 - jepsen.util 262 :info :write [27 16] :RequestTimeoutException INFO [2022-01-10 12:53:30,909] jepsen worker 39 - jepsen.util 129 :invoke :write [13 41] INFO [2022-01-10 12:53:30,909] jepsen worker 82 - jepsen.util 352 :invoke :write [27 4] INFO [2022-01-10 12:53:30,913] jepsen worker 41 - jepsen.util 41 :ok :write [13 21] INFO [2022-01-10 12:53:30,914] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,914] jepsen worker 59 - jepsen.util 239 :info :write [19 9] :RequestTimeoutException INFO [2022-01-10 12:53:30,915] jepsen worker 41 - jepsen.util 41 :ok :read [13 21] INFO [2022-01-10 12:53:30,915] jepsen worker 59 - jepsen.util 329 :invoke :write [19 21] INFO [2022-01-10 12:53:30,915] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,916] jepsen worker 41 - jepsen.util 41 :ok :read [13 21] INFO [2022-01-10 12:53:30,917] jepsen worker 41 - jepsen.util 41 :invoke :read [13 nil] INFO [2022-01-10 12:53:30,917] jepsen worker 41 - jepsen.util 41 :ok :read [13 21] INFO [2022-01-10 12:53:30,918] jepsen worker 41 - jepsen.util 41 :invoke :write [13 0] INFO [2022-01-10 12:53:30,923] jepsen worker 0 - jepsen.util 180 :ok :write [0 6] INFO [2022-01-10 12:53:30,924] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,925] jepsen worker 0 - jepsen.util 180 :ok :read [0 6] INFO [2022-01-10 12:53:30,925] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:30,926] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:30,926] jepsen worker 0 - jepsen.util 180 :ok :read [0 6] INFO [2022-01-10 12:53:30,926] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:30,927] jepsen worker 0 - jepsen.util 180 :invoke :write [0 8] INFO [2022-01-10 12:53:30,927] jepsen worker 38 - jepsen.util 38 :ok :read [12 11] INFO [2022-01-10 12:53:30,928] jepsen worker 38 - jepsen.util 38 :invoke :write [12 49] INFO [2022-01-10 12:53:30,930] jepsen worker 89 - jepsen.util 269 :ok :write [29 28] INFO [2022-01-10 12:53:30,930] jepsen worker 89 - jepsen.util 269 :invoke :write [29 20] INFO [2022-01-10 12:53:30,932] jepsen worker 84 - jepsen.util 84 :ok :write [28 49] INFO [2022-01-10 12:53:30,932] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:30,933] jepsen worker 84 - jepsen.util 84 :ok :read [28 49] INFO [2022-01-10 12:53:30,934] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:30,934] jepsen worker 84 - jepsen.util 84 :ok :read [28 49] INFO [2022-01-10 12:53:30,935] jepsen worker 84 - jepsen.util 84 :invoke :write [28 11] INFO [2022-01-10 12:53:30,935] jepsen worker 39 - jepsen.util 129 :ok :write [13 41] INFO [2022-01-10 12:53:30,936] jepsen worker 39 - jepsen.util 129 :invoke :write [13 48] INFO [2022-01-10 12:53:30,938] jepsen worker 3 - jepsen.util 3 :ok :write [1 19] INFO [2022-01-10 12:53:30,938] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:30,939] jepsen worker 3 - jepsen.util 3 :ok :read [1 19] INFO [2022-01-10 12:53:30,940] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 12:53:30,944] jepsen worker 0 - jepsen.util 180 :ok :write [0 8] INFO [2022-01-10 12:53:30,944] jepsen worker 0 - jepsen.util 180 :invoke :write [0 22] INFO [2022-01-10 12:53:30,949] jepsen worker 38 - jepsen.util 38 :ok :write [12 49] INFO [2022-01-10 12:53:30,949] jepsen worker 38 - jepsen.util 38 :invoke :write [12 4] INFO [2022-01-10 12:53:30,952] jepsen worker 89 - jepsen.util 269 :ok :write [29 20] INFO [2022-01-10 12:53:30,953] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:30,953] jepsen worker 89 - jepsen.util 269 :ok :read [29 20] INFO [2022-01-10 12:53:30,954] jepsen worker 89 - jepsen.util 269 :invoke :write [29 0] INFO [2022-01-10 12:53:30,977] jepsen worker 54 - jepsen.util 234 :info :write [18 1] :RequestTimeoutException INFO [2022-01-10 12:53:30,977] jepsen worker 54 - jepsen.util 324 :invoke :read [18 nil] INFO [2022-01-10 12:53:30,978] jepsen worker 54 - jepsen.util 324 :ok :read [18 46] INFO [2022-01-10 12:53:30,979] jepsen worker 54 - jepsen.util 324 :invoke :write [18 45] INFO [2022-01-10 12:53:30,998] jepsen worker 28 - jepsen.util 208 :info :write [9 38] :RequestTimeoutException INFO [2022-01-10 12:53:30,998] jepsen worker 28 - jepsen.util 298 :invoke :read [9 nil] INFO [2022-01-10 12:53:31,004] jepsen worker 46 - jepsen.util 226 :info :write [15 2] :RequestTimeoutException INFO [2022-01-10 12:53:31,004] jepsen worker 46 - jepsen.util 316 :invoke :read [15 nil] INFO [2022-01-10 12:53:31,005] jepsen worker 46 - jepsen.util 316 :ok :read [15 10] INFO [2022-01-10 12:53:31,006] jepsen worker 46 - jepsen.util 316 :invoke :read [15 nil] INFO [2022-01-10 12:53:31,006] jepsen worker 46 - jepsen.util 316 :ok :read [15 10] INFO [2022-01-10 12:53:31,007] jepsen worker 46 - jepsen.util 316 :invoke :write [15 16] INFO [2022-01-10 12:53:31,014] jepsen worker 72 - jepsen.util 252 :info :write [24 6] :RequestTimeoutException INFO [2022-01-10 12:53:31,014] jepsen worker 72 - jepsen.util 342 :invoke :write [24 19] INFO [2022-01-10 12:53:31,017] jepsen worker 52 - jepsen.util 52 :ok :write [17 6] INFO [2022-01-10 12:53:31,017] jepsen worker 43 - jepsen.util 43 :ok :write [14 9] INFO [2022-01-10 12:53:31,018] jepsen worker 43 - jepsen.util 43 :invoke :write [14 35] INFO [2022-01-10 12:53:31,019] jepsen worker 52 - jepsen.util 52 :invoke :write [17 14] INFO [2022-01-10 12:53:31,023] jepsen worker 80 - jepsen.util 260 :ok :write [26 17] INFO [2022-01-10 12:53:31,024] jepsen worker 75 - jepsen.util 165 :ok :write [25 23] INFO [2022-01-10 12:53:31,024] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,024] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:31,025] jepsen worker 80 - jepsen.util 260 :ok :read [26 17] INFO [2022-01-10 12:53:31,025] jepsen worker 75 - jepsen.util 165 :ok :read [25 23] INFO [2022-01-10 12:53:31,025] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,026] jepsen worker 75 - jepsen.util 165 :invoke :write [25 9] INFO [2022-01-10 12:53:31,026] jepsen worker 80 - jepsen.util 260 :ok :read [26 17] INFO [2022-01-10 12:53:31,027] jepsen worker 80 - jepsen.util 260 :invoke :write [26 47] INFO [2022-01-10 12:53:31,028] jepsen worker 13 - jepsen.util 13 :ok :write [4 10] INFO [2022-01-10 12:53:31,029] jepsen worker 13 - jepsen.util 13 :invoke :write [4 25] INFO [2022-01-10 12:53:31,030] jepsen worker 68 - jepsen.util 68 :ok :write [22 45] INFO [2022-01-10 12:53:31,031] jepsen worker 68 - jepsen.util 68 :invoke :write [22 0] INFO [2022-01-10 12:53:31,034] jepsen worker 33 - jepsen.util 123 :ok :write [11 23] INFO [2022-01-10 12:53:31,034] jepsen worker 70 - jepsen.util 160 :ok :write [23 12] INFO [2022-01-10 12:53:31,035] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:31,035] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,036] jepsen worker 33 - jepsen.util 123 :ok :read [11 23] INFO [2022-01-10 12:53:31,036] jepsen worker 70 - jepsen.util 160 :ok :read [23 12] INFO [2022-01-10 12:53:31,036] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:31,037] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,037] jepsen worker 33 - jepsen.util 123 :ok :read [11 23] INFO [2022-01-10 12:53:31,037] jepsen worker 70 - jepsen.util 160 :ok :read [23 12] INFO [2022-01-10 12:53:31,038] jepsen worker 33 - jepsen.util 123 :invoke :write [11 15] INFO [2022-01-10 12:53:31,038] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,039] jepsen worker 70 - jepsen.util 160 :ok :read [23 12] INFO [2022-01-10 12:53:31,039] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,040] jepsen worker 70 - jepsen.util 160 :ok :read [23 12] INFO [2022-01-10 12:53:31,041] jepsen worker 70 - jepsen.util 160 :invoke :write [23 32] INFO [2022-01-10 12:53:31,041] jepsen worker 61 - jepsen.util 61 :ok :write [20 42] INFO [2022-01-10 12:53:31,042] jepsen worker 61 - jepsen.util 61 :invoke :write [20 24] INFO [2022-01-10 12:53:31,048] jepsen worker 15 - jepsen.util 15 :ok :write [5 44] INFO [2022-01-10 12:53:31,048] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:31,049] jepsen worker 15 - jepsen.util 15 :ok :read [5 44] INFO [2022-01-10 12:53:31,049] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:31,050] jepsen worker 15 - jepsen.util 15 :ok :read [5 44] INFO [2022-01-10 12:53:31,051] jepsen worker 15 - jepsen.util 15 :invoke :write [5 37] INFO [2022-01-10 12:53:31,052] jepsen worker 22 - jepsen.util 112 :ok :write [7 36] INFO [2022-01-10 12:53:31,052] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,053] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,053] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,054] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,054] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,055] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,056] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,057] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,057] jepsen worker 25 - jepsen.util 205 :info :write [8 42] :RequestTimeoutException INFO [2022-01-10 12:53:31,057] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,058] jepsen worker 25 - jepsen.util 295 :invoke :read [8 nil] INFO [2022-01-10 12:53:31,058] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,058] jepsen worker 50 - jepsen.util 50 :ok :write [16 12] INFO [2022-01-10 12:53:31,059] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,059] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:53:31,059] jepsen worker 25 - jepsen.util 295 :ok :read [8 25] INFO [2022-01-10 12:53:31,060] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,060] jepsen worker 25 - jepsen.util 295 :invoke :read [8 nil] INFO [2022-01-10 12:53:31,060] jepsen worker 84 - jepsen.util 84 :ok :write [28 11] INFO [2022-01-10 12:53:31,060] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,061] jepsen worker 25 - jepsen.util 295 :ok :read [8 25] INFO [2022-01-10 12:53:31,061] jepsen worker 84 - jepsen.util 84 :invoke :write [28 32] INFO [2022-01-10 12:53:31,061] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,062] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,062] jepsen worker 39 - jepsen.util 129 :ok :write [13 48] INFO [2022-01-10 12:53:31,062] jepsen worker 25 - jepsen.util 295 :invoke :write [8 10] INFO [2022-01-10 12:53:31,062] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,063] jepsen worker 39 - jepsen.util 129 :invoke :write [13 15] INFO [2022-01-10 12:53:31,063] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,064] jepsen worker 7 - jepsen.util 7 :ok :write [2 4] INFO [2022-01-10 12:53:31,064] jepsen worker 3 - jepsen.util 3 :ok :write [1 40] INFO [2022-01-10 12:53:31,064] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,064] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,065] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,065] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:31,066] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:31,066] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,066] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,066] jepsen worker 24 - jepsen.util 204 :info :write [8 26] :RequestTimeoutException INFO [2022-01-10 12:53:31,067] jepsen worker 3 - jepsen.util 3 :ok :read [1 40] INFO [2022-01-10 12:53:31,067] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,067] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:31,067] jepsen worker 0 - jepsen.util 180 :ok :write [0 22] INFO [2022-01-10 12:53:31,067] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,068] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:31,068] jepsen worker 0 - jepsen.util 180 :invoke :write [0 28] INFO [2022-01-10 12:53:31,068] jepsen worker 3 - jepsen.util 3 :ok :read [1 40] INFO [2022-01-10 12:53:31,069] jepsen worker 7 - jepsen.util 7 :invoke :write [2 36] INFO [2022-01-10 12:53:31,069] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:31,070] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,070] jepsen worker 3 - jepsen.util 3 :ok :read [1 40] INFO [2022-01-10 12:53:31,070] jepsen worker 38 - jepsen.util 38 :ok :write [12 4] INFO [2022-01-10 12:53:31,070] jepsen worker 89 - jepsen.util 269 :ok :write [29 0] INFO [2022-01-10 12:53:31,070] jepsen worker 81 - jepsen.util 171 :ok :write [27 28] INFO [2022-01-10 12:53:31,071] jepsen worker 24 - jepsen.util 294 :invoke :read [8 nil] INFO [2022-01-10 12:53:31,071] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,071] jepsen worker 38 - jepsen.util 38 :invoke :write [12 23] INFO [2022-01-10 12:53:31,072] jepsen worker 24 - jepsen.util 294 :ok :read [8 25] INFO [2022-01-10 12:53:31,072] jepsen worker 81 - jepsen.util 171 :invoke :write [27 3] INFO [2022-01-10 12:53:31,073] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,074] jepsen worker 3 - jepsen.util 3 :invoke :write [1 42] INFO [2022-01-10 12:53:31,074] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,075] jepsen worker 24 - jepsen.util 294 :invoke :read [8 nil] INFO [2022-01-10 12:53:31,075] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:31,075] jepsen worker 24 - jepsen.util 294 :ok :read [8 25] INFO [2022-01-10 12:53:31,076] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,076] jepsen worker 89 - jepsen.util 269 :ok :read [29 0] INFO [2022-01-10 12:53:31,076] jepsen worker 24 - jepsen.util 294 :invoke :write [8 14] INFO [2022-01-10 12:53:31,077] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,077] jepsen worker 89 - jepsen.util 269 :invoke :write [29 20] INFO [2022-01-10 12:53:31,077] jepsen worker 52 - jepsen.util 52 :ok :write [17 14] INFO [2022-01-10 12:53:31,078] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,078] jepsen worker 43 - jepsen.util 43 :ok :write [14 35] INFO [2022-01-10 12:53:31,078] jepsen worker 52 - jepsen.util 52 :invoke :write [17 4] INFO [2022-01-10 12:53:31,078] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,079] jepsen worker 43 - jepsen.util 43 :invoke :write [14 16] INFO [2022-01-10 12:53:31,080] jepsen worker 22 - jepsen.util 112 :invoke :write [7 2] INFO [2022-01-10 12:53:31,083] jepsen worker 80 - jepsen.util 260 :ok :write [26 47] INFO [2022-01-10 12:53:31,083] jepsen worker 80 - jepsen.util 260 :invoke :write [26 30] INFO [2022-01-10 12:53:31,084] jepsen worker 75 - jepsen.util 165 :ok :write [25 9] INFO [2022-01-10 12:53:31,084] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:31,085] jepsen worker 75 - jepsen.util 165 :ok :read [25 9] INFO [2022-01-10 12:53:31,086] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:31,086] jepsen worker 68 - jepsen.util 68 :ok :write [22 0] INFO [2022-01-10 12:53:31,086] jepsen worker 75 - jepsen.util 165 :ok :read [25 9] INFO [2022-01-10 12:53:31,087] jepsen worker 68 - jepsen.util 68 :invoke :write [22 21] INFO [2022-01-10 12:53:31,088] jepsen worker 75 - jepsen.util 165 :invoke :write [25 4] INFO [2022-01-10 12:53:31,088] jepsen worker 13 - jepsen.util 13 :ok :write [4 25] INFO [2022-01-10 12:53:31,089] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,089] jepsen worker 13 - jepsen.util 13 :ok :read [4 25] INFO [2022-01-10 12:53:31,090] jepsen worker 13 - jepsen.util 13 :invoke :write [4 24] INFO [2022-01-10 12:53:31,091] jepsen worker 33 - jepsen.util 123 :ok :write [11 15] INFO [2022-01-10 12:53:31,092] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:31,093] jepsen worker 33 - jepsen.util 123 :ok :read [11 15] INFO [2022-01-10 12:53:31,093] jepsen worker 33 - jepsen.util 123 :invoke :write [11 12] INFO [2022-01-10 12:53:31,095] jepsen worker 70 - jepsen.util 160 :ok :write [23 32] INFO [2022-01-10 12:53:31,096] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,096] jepsen worker 70 - jepsen.util 160 :ok :read [23 32] INFO [2022-01-10 12:53:31,097] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,098] jepsen worker 70 - jepsen.util 160 :ok :read [23 32] INFO [2022-01-10 12:53:31,098] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,099] jepsen worker 61 - jepsen.util 61 :ok :write [20 24] INFO [2022-01-10 12:53:31,099] jepsen worker 70 - jepsen.util 160 :ok :read [23 32] INFO [2022-01-10 12:53:31,099] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:31,100] jepsen worker 70 - jepsen.util 160 :invoke :write [23 4] INFO [2022-01-10 12:53:31,100] jepsen worker 61 - jepsen.util 61 :ok :read [20 24] INFO [2022-01-10 12:53:31,100] jepsen worker 61 - jepsen.util 61 :invoke :write [20 8] INFO [2022-01-10 12:53:31,102] jepsen worker 84 - jepsen.util 84 :ok :write [28 32] INFO [2022-01-10 12:53:31,103] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:31,103] jepsen worker 15 - jepsen.util 15 :ok :write [5 37] INFO [2022-01-10 12:53:31,104] jepsen worker 84 - jepsen.util 84 :ok :read [28 32] INFO [2022-01-10 12:53:31,104] jepsen worker 15 - jepsen.util 15 :invoke :write [5 39] INFO [2022-01-10 12:53:31,104] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:31,105] jepsen worker 84 - jepsen.util 84 :ok :read [28 32] INFO [2022-01-10 12:53:31,106] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:31,107] jepsen worker 84 - jepsen.util 84 :ok :read [28 32] INFO [2022-01-10 12:53:31,107] jepsen worker 84 - jepsen.util 84 :invoke :write [28 35] INFO [2022-01-10 12:53:31,110] jepsen worker 22 - jepsen.util 112 :ok :write [7 2] INFO [2022-01-10 12:53:31,110] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,111] jepsen worker 22 - jepsen.util 112 :ok :read [7 2] INFO [2022-01-10 12:53:31,112] jepsen worker 22 - jepsen.util 112 :invoke :write [7 21] INFO [2022-01-10 12:53:31,118] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:53:31,118] jepsen worker 52 - jepsen.util 52 :ok :write [17 4] INFO [2022-01-10 12:53:31,118] jepsen worker 50 - jepsen.util 50 :invoke :write [16 41] INFO [2022-01-10 12:53:31,119] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:31,120] jepsen worker 52 - jepsen.util 52 :ok :read [17 4] INFO [2022-01-10 12:53:31,120] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:31,121] jepsen worker 52 - jepsen.util 52 :ok :read [17 4] INFO [2022-01-10 12:53:31,122] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:31,123] jepsen worker 52 - jepsen.util 52 :ok :read [17 4] INFO [2022-01-10 12:53:31,123] jepsen worker 52 - jepsen.util 52 :invoke :write [17 16] INFO [2022-01-10 12:53:31,126] jepsen worker 80 - jepsen.util 260 :ok :write [26 30] INFO [2022-01-10 12:53:31,127] jepsen worker 80 - jepsen.util 260 :invoke :write [26 43] INFO [2022-01-10 12:53:31,131] jepsen worker 13 - jepsen.util 13 :ok :write [4 24] INFO [2022-01-10 12:53:31,132] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,133] jepsen worker 43 - jepsen.util 43 :ok :write [14 16] INFO [2022-01-10 12:53:31,133] jepsen worker 13 - jepsen.util 13 :ok :read [4 24] INFO [2022-01-10 12:53:31,134] jepsen worker 39 - jepsen.util 129 :ok :write [13 15] INFO [2022-01-10 12:53:31,134] jepsen worker 43 - jepsen.util 43 :invoke :write [14 49] INFO [2022-01-10 12:53:31,135] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,135] jepsen worker 13 - jepsen.util 13 :invoke :write [4 14] INFO [2022-01-10 12:53:31,136] jepsen worker 39 - jepsen.util 129 :ok :read [13 15] INFO [2022-01-10 12:53:31,137] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,138] jepsen worker 39 - jepsen.util 129 :ok :read [13 15] INFO [2022-01-10 12:53:31,138] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,139] jepsen worker 39 - jepsen.util 129 :ok :read [13 15] INFO [2022-01-10 12:53:31,139] jepsen worker 70 - jepsen.util 160 :ok :write [23 4] INFO [2022-01-10 12:53:31,140] jepsen worker 38 - jepsen.util 38 :ok :write [12 23] INFO [2022-01-10 12:53:31,140] jepsen worker 39 - jepsen.util 129 :invoke :write [13 14] INFO [2022-01-10 12:53:31,140] jepsen worker 70 - jepsen.util 160 :invoke :write [23 37] INFO [2022-01-10 12:53:31,141] jepsen worker 75 - jepsen.util 165 :ok :write [25 4] INFO [2022-01-10 12:53:31,141] jepsen worker 38 - jepsen.util 38 :invoke :write [12 33] INFO [2022-01-10 12:53:31,142] jepsen worker 75 - jepsen.util 165 :invoke :write [25 49] INFO [2022-01-10 12:53:31,144] jepsen worker 7 - jepsen.util 7 :ok :write [2 36] INFO [2022-01-10 12:53:31,144] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,145] jepsen worker 3 - jepsen.util 3 :ok :write [1 42] INFO [2022-01-10 12:53:31,145] jepsen worker 7 - jepsen.util 7 :ok :read [2 36] INFO [2022-01-10 12:53:31,145] jepsen worker 3 - jepsen.util 3 :invoke :write [1 38] INFO [2022-01-10 12:53:31,146] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,147] jepsen worker 7 - jepsen.util 7 :ok :read [2 36] INFO [2022-01-10 12:53:31,147] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,148] jepsen worker 7 - jepsen.util 7 :ok :read [2 36] INFO [2022-01-10 12:53:31,148] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,149] jepsen worker 2 - jepsen.util 92 :ok :write [0 27] INFO [2022-01-10 12:53:31,149] jepsen worker 7 - jepsen.util 7 :ok :read [2 36] INFO [2022-01-10 12:53:31,150] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,150] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,150] jepsen worker 84 - jepsen.util 84 :ok :write [28 35] INFO [2022-01-10 12:53:31,151] jepsen worker 2 - jepsen.util 92 :ok :read [0 27] INFO [2022-01-10 12:53:31,151] jepsen worker 7 - jepsen.util 7 :ok :read [2 36] INFO [2022-01-10 12:53:31,151] jepsen worker 84 - jepsen.util 84 :invoke :write [28 29] INFO [2022-01-10 12:53:31,152] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,152] jepsen worker 51 - jepsen.util 141 :ok :write [17 28] INFO [2022-01-10 12:53:31,152] jepsen worker 2 - jepsen.util 92 :invoke :write [0 28] INFO [2022-01-10 12:53:31,152] jepsen worker 51 - jepsen.util 141 :invoke :write [17 1] INFO [2022-01-10 12:53:31,152] jepsen worker 7 - jepsen.util 7 :ok :read [2 36] INFO [2022-01-10 12:53:31,153] jepsen worker 7 - jepsen.util 7 :invoke :write [2 1] INFO [2022-01-10 12:53:31,156] jepsen worker 81 - jepsen.util 171 :ok :write [27 3] INFO [2022-01-10 12:53:31,156] jepsen worker 81 - jepsen.util 171 :invoke :write [27 43] INFO [2022-01-10 12:53:31,159] jepsen worker 61 - jepsen.util 61 :ok :write [20 8] INFO [2022-01-10 12:53:31,160] jepsen worker 61 - jepsen.util 61 :invoke :write [20 47] INFO [2022-01-10 12:53:31,161] jepsen worker 80 - jepsen.util 260 :ok :write [26 43] INFO [2022-01-10 12:53:31,162] jepsen worker 80 - jepsen.util 260 :invoke :write [26 48] INFO [2022-01-10 12:53:31,164] jepsen worker 88 - jepsen.util 178 :ok :write [29 24] INFO [2022-01-10 12:53:31,165] jepsen worker 88 - jepsen.util 178 :invoke :write [29 43] INFO [2022-01-10 12:53:31,168] jepsen worker 64 - jepsen.util 244 :info :write [21 25] :RequestTimeoutException INFO [2022-01-10 12:53:31,168] jepsen worker 64 - jepsen.util 334 :invoke :read [21 nil] INFO [2022-01-10 12:53:31,170] jepsen worker 68 - jepsen.util 68 :ok :write [22 21] INFO [2022-01-10 12:53:31,171] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:31,172] jepsen worker 68 - jepsen.util 68 :ok :read [22 21] INFO [2022-01-10 12:53:31,172] jepsen worker 13 - jepsen.util 13 :ok :write [4 14] INFO [2022-01-10 12:53:31,173] jepsen worker 68 - jepsen.util 68 :invoke :write [22 27] INFO [2022-01-10 12:53:31,173] jepsen worker 22 - jepsen.util 112 :ok :write [7 21] INFO [2022-01-10 12:53:31,174] jepsen worker 13 - jepsen.util 13 :invoke :write [4 4] INFO [2022-01-10 12:53:31,175] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,176] jepsen worker 22 - jepsen.util 112 :ok :read [7 21] INFO [2022-01-10 12:53:31,176] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,177] jepsen worker 70 - jepsen.util 160 :ok :write [23 37] INFO [2022-01-10 12:53:31,177] jepsen worker 33 - jepsen.util 123 :ok :write [11 12] INFO [2022-01-10 12:53:31,178] jepsen worker 22 - jepsen.util 112 :ok :read [7 21] INFO [2022-01-10 12:53:31,178] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,178] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,179] jepsen worker 70 - jepsen.util 160 :ok :read [23 37] INFO [2022-01-10 12:53:31,179] jepsen worker 33 - jepsen.util 123 :invoke :write [11 47] INFO [2022-01-10 12:53:31,179] jepsen worker 22 - jepsen.util 112 :ok :read [7 21] INFO [2022-01-10 12:53:31,180] jepsen worker 22 - jepsen.util 112 :invoke :write [7 25] INFO [2022-01-10 12:53:31,180] jepsen worker 70 - jepsen.util 160 :invoke :write [23 48] INFO [2022-01-10 12:53:31,181] jepsen worker 15 - jepsen.util 15 :ok :write [5 39] INFO [2022-01-10 12:53:31,182] jepsen worker 15 - jepsen.util 15 :invoke :write [5 6] INFO [2022-01-10 12:53:31,183] jepsen worker 38 - jepsen.util 38 :ok :write [12 33] INFO [2022-01-10 12:53:31,184] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:31,184] jepsen worker 38 - jepsen.util 38 :ok :read [12 33] INFO [2022-01-10 12:53:31,185] jepsen worker 38 - jepsen.util 38 :invoke :write [12 0] INFO [2022-01-10 12:53:31,186] jepsen worker 50 - jepsen.util 50 :ok :write [16 41] INFO [2022-01-10 12:53:31,187] jepsen worker 50 - jepsen.util 50 :invoke :write [16 6] INFO [2022-01-10 12:53:31,188] jepsen worker 43 - jepsen.util 43 :ok :write [14 49] INFO [2022-01-10 12:53:31,189] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:31,189] jepsen worker 43 - jepsen.util 43 :ok :read [14 49] INFO [2022-01-10 12:53:31,190] jepsen worker 3 - jepsen.util 3 :ok :write [1 38] INFO [2022-01-10 12:53:31,190] jepsen worker 43 - jepsen.util 43 :invoke :write [14 33] INFO [2022-01-10 12:53:31,191] jepsen worker 3 - jepsen.util 3 :invoke :write [1 15] INFO [2022-01-10 12:53:31,194] jepsen worker 39 - jepsen.util 129 :ok :write [13 14] INFO [2022-01-10 12:53:31,195] jepsen worker 75 - jepsen.util 165 :ok :write [25 49] INFO [2022-01-10 12:53:31,195] jepsen worker 39 - jepsen.util 129 :invoke :write [13 41] INFO [2022-01-10 12:53:31,195] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:31,196] jepsen worker 75 - jepsen.util 165 :ok :read [25 49] INFO [2022-01-10 12:53:31,197] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:31,198] jepsen worker 75 - jepsen.util 165 :ok :read [25 49] INFO [2022-01-10 12:53:31,198] jepsen worker 51 - jepsen.util 141 :ok :write [17 1] INFO [2022-01-10 12:53:31,198] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:31,199] jepsen worker 75 - jepsen.util 165 :invoke :write [25 17] INFO [2022-01-10 12:53:31,200] jepsen worker 51 - jepsen.util 141 :ok :read [17 1] INFO [2022-01-10 12:53:31,200] jepsen worker 51 - jepsen.util 141 :invoke :write [17 16] INFO [2022-01-10 12:53:31,200] jepsen worker 7 - jepsen.util 7 :ok :write [2 1] INFO [2022-01-10 12:53:31,201] jepsen worker 7 - jepsen.util 7 :invoke :write [2 30] INFO [2022-01-10 12:53:31,204] jepsen worker 2 - jepsen.util 92 :ok :write [0 28] INFO [2022-01-10 12:53:31,205] jepsen worker 2 - jepsen.util 92 :invoke :write [0 41] INFO [2022-01-10 12:53:31,205] jepsen worker 84 - jepsen.util 84 :ok :write [28 29] INFO [2022-01-10 12:53:31,206] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:31,207] jepsen worker 84 - jepsen.util 84 :ok :read [28 29] INFO [2022-01-10 12:53:31,208] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:31,208] jepsen worker 84 - jepsen.util 84 :ok :read [28 29] INFO [2022-01-10 12:53:31,209] jepsen worker 80 - jepsen.util 260 :ok :write [26 48] INFO [2022-01-10 12:53:31,209] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:31,210] jepsen worker 81 - jepsen.util 171 :ok :write [27 43] INFO [2022-01-10 12:53:31,210] jepsen worker 80 - jepsen.util 260 :invoke :write [26 37] INFO [2022-01-10 12:53:31,210] jepsen worker 84 - jepsen.util 84 :ok :read [28 29] INFO [2022-01-10 12:53:31,211] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:31,212] jepsen worker 84 - jepsen.util 84 :invoke :write [28 3] INFO [2022-01-10 12:53:31,212] jepsen worker 81 - jepsen.util 171 :ok :read [27 43] INFO [2022-01-10 12:53:31,213] jepsen worker 13 - jepsen.util 13 :ok :write [4 4] INFO [2022-01-10 12:53:31,213] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:31,214] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,214] jepsen worker 61 - jepsen.util 61 :ok :write [20 47] INFO [2022-01-10 12:53:31,214] jepsen worker 81 - jepsen.util 171 :ok :read [27 43] INFO [2022-01-10 12:53:31,214] jepsen worker 13 - jepsen.util 13 :ok :read [4 4] INFO [2022-01-10 12:53:31,215] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:31,215] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,216] jepsen worker 81 - jepsen.util 171 :ok :read [27 43] INFO [2022-01-10 12:53:31,216] jepsen worker 81 - jepsen.util 171 :invoke :write [27 13] INFO [2022-01-10 12:53:31,217] jepsen worker 13 - jepsen.util 13 :ok :read [4 4] INFO [2022-01-10 12:53:31,217] jepsen worker 88 - jepsen.util 178 :ok :write [29 43] INFO [2022-01-10 12:53:31,217] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:31,218] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,218] jepsen worker 61 - jepsen.util 61 :ok :read [20 47] INFO [2022-01-10 12:53:31,218] jepsen worker 88 - jepsen.util 178 :invoke :write [29 1] INFO [2022-01-10 12:53:31,219] jepsen worker 13 - jepsen.util 13 :ok :read [4 4] INFO [2022-01-10 12:53:31,219] jepsen worker 70 - jepsen.util 160 :ok :write [23 48] INFO [2022-01-10 12:53:31,219] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:31,220] jepsen worker 61 - jepsen.util 61 :ok :read [20 47] INFO [2022-01-10 12:53:31,220] jepsen worker 13 - jepsen.util 13 :invoke :write [4 43] INFO [2022-01-10 12:53:31,220] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,221] jepsen worker 68 - jepsen.util 68 :ok :write [22 27] INFO [2022-01-10 12:53:31,222] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:31,222] jepsen worker 61 - jepsen.util 61 :invoke :write [20 12] INFO [2022-01-10 12:53:31,223] jepsen worker 68 - jepsen.util 68 :ok :read [22 27] INFO [2022-01-10 12:53:31,223] jepsen worker 70 - jepsen.util 160 :ok :read [23 48] INFO [2022-01-10 12:53:31,223] jepsen worker 70 - jepsen.util 160 :invoke :write [23 6] INFO [2022-01-10 12:53:31,223] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:31,224] jepsen worker 68 - jepsen.util 68 :ok :read [22 27] INFO [2022-01-10 12:53:31,225] jepsen worker 68 - jepsen.util 68 :invoke :write [22 27] INFO [2022-01-10 12:53:31,225] jepsen worker 22 - jepsen.util 112 :ok :write [7 25] INFO [2022-01-10 12:53:31,225] jepsen worker 22 - jepsen.util 112 :invoke :write [7 10] INFO [2022-01-10 12:53:31,228] jepsen worker 38 - jepsen.util 38 :ok :write [12 0] INFO [2022-01-10 12:53:31,229] jepsen worker 38 - jepsen.util 38 :invoke :write [12 16] INFO [2022-01-10 12:53:31,231] jepsen worker 33 - jepsen.util 123 :ok :write [11 47] INFO [2022-01-10 12:53:31,231] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:31,232] jepsen worker 33 - jepsen.util 123 :ok :read [11 47] INFO [2022-01-10 12:53:31,233] jepsen worker 33 - jepsen.util 123 :invoke :write [11 45] INFO [2022-01-10 12:53:31,234] jepsen worker 3 - jepsen.util 3 :ok :write [1 15] INFO [2022-01-10 12:53:31,235] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:31,236] jepsen worker 3 - jepsen.util 3 :ok :read [1 15] INFO [2022-01-10 12:53:31,236] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:31,237] jepsen worker 3 - jepsen.util 3 :ok :read [1 15] INFO [2022-01-10 12:53:31,237] jepsen worker 3 - jepsen.util 3 :invoke :write [1 17] INFO [2022-01-10 12:53:31,238] jepsen worker 39 - jepsen.util 129 :ok :write [13 41] INFO [2022-01-10 12:53:31,238] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,239] jepsen worker 15 - jepsen.util 15 :ok :write [5 6] INFO [2022-01-10 12:53:31,239] jepsen worker 39 - jepsen.util 129 :ok :read [13 41] INFO [2022-01-10 12:53:31,240] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,240] jepsen worker 51 - jepsen.util 141 :ok :write [17 16] INFO [2022-01-10 12:53:31,240] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:31,241] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:31,241] jepsen worker 39 - jepsen.util 129 :ok :read [13 41] INFO [2022-01-10 12:53:31,241] jepsen worker 15 - jepsen.util 15 :ok :read [5 6] INFO [2022-01-10 12:53:31,241] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,241] jepsen worker 51 - jepsen.util 141 :ok :read [17 16] INFO [2022-01-10 12:53:31,242] jepsen worker 15 - jepsen.util 15 :invoke :write [5 7] INFO [2022-01-10 12:53:31,242] jepsen worker 39 - jepsen.util 129 :ok :read [13 41] INFO [2022-01-10 12:53:31,243] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:31,243] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,243] jepsen worker 51 - jepsen.util 141 :ok :read [17 16] INFO [2022-01-10 12:53:31,244] jepsen worker 43 - jepsen.util 43 :ok :write [14 33] INFO [2022-01-10 12:53:31,244] jepsen worker 51 - jepsen.util 141 :invoke :write [17 37] INFO [2022-01-10 12:53:31,244] jepsen worker 39 - jepsen.util 129 :ok :read [13 41] INFO [2022-01-10 12:53:31,244] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:31,245] jepsen worker 39 - jepsen.util 129 :invoke :write [13 47] INFO [2022-01-10 12:53:31,245] jepsen worker 43 - jepsen.util 43 :ok :read [14 33] INFO [2022-01-10 12:53:31,246] jepsen worker 43 - jepsen.util 43 :invoke :write [14 34] INFO [2022-01-10 12:53:31,249] jepsen worker 84 - jepsen.util 84 :ok :write [28 3] INFO [2022-01-10 12:53:31,250] jepsen worker 84 - jepsen.util 84 :invoke :write [28 44] INFO [2022-01-10 12:53:31,251] jepsen worker 2 - jepsen.util 92 :ok :write [0 41] INFO [2022-01-10 12:53:31,251] jepsen worker 50 - jepsen.util 50 :ok :write [16 6] INFO [2022-01-10 12:53:31,252] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,252] jepsen worker 50 - jepsen.util 50 :invoke :write [16 33] INFO [2022-01-10 12:53:31,252] jepsen worker 2 - jepsen.util 92 :ok :read [0 41] INFO [2022-01-10 12:53:31,253] jepsen worker 80 - jepsen.util 260 :ok :write [26 37] INFO [2022-01-10 12:53:31,253] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,253] jepsen worker 80 - jepsen.util 260 :invoke :write [26 1] INFO [2022-01-10 12:53:31,254] jepsen worker 2 - jepsen.util 92 :ok :read [0 41] INFO [2022-01-10 12:53:31,254] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,255] jepsen worker 2 - jepsen.util 92 :ok :read [0 41] INFO [2022-01-10 12:53:31,256] jepsen worker 75 - jepsen.util 165 :ok :write [25 17] INFO [2022-01-10 12:53:31,256] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,257] jepsen worker 75 - jepsen.util 165 :invoke :write [25 14] INFO [2022-01-10 12:53:31,257] jepsen worker 2 - jepsen.util 92 :ok :read [0 41] INFO [2022-01-10 12:53:31,257] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,258] jepsen worker 2 - jepsen.util 92 :ok :read [0 41] INFO [2022-01-10 12:53:31,258] jepsen worker 13 - jepsen.util 13 :ok :write [4 43] INFO [2022-01-10 12:53:31,259] jepsen worker 2 - jepsen.util 92 :invoke :write [0 1] INFO [2022-01-10 12:53:31,260] jepsen worker 13 - jepsen.util 13 :invoke :write [4 18] INFO [2022-01-10 12:53:31,263] jepsen worker 88 - jepsen.util 178 :ok :write [29 1] INFO [2022-01-10 12:53:31,264] jepsen worker 7 - jepsen.util 7 :ok :write [2 30] INFO [2022-01-10 12:53:31,264] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:31,264] jepsen worker 7 - jepsen.util 7 :invoke :write [2 4] INFO [2022-01-10 12:53:31,265] jepsen worker 88 - jepsen.util 178 :ok :read [29 1] INFO [2022-01-10 12:53:31,265] jepsen worker 70 - jepsen.util 160 :ok :write [23 6] INFO [2022-01-10 12:53:31,265] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:31,265] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,266] jepsen worker 88 - jepsen.util 178 :ok :read [29 1] INFO [2022-01-10 12:53:31,267] jepsen worker 88 - jepsen.util 178 :invoke :write [29 35] INFO [2022-01-10 12:53:31,267] jepsen worker 70 - jepsen.util 160 :ok :read [23 6] INFO [2022-01-10 12:53:31,267] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,268] jepsen worker 70 - jepsen.util 160 :ok :read [23 6] INFO [2022-01-10 12:53:31,268] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,269] jepsen worker 70 - jepsen.util 160 :ok :read [23 6] INFO [2022-01-10 12:53:31,270] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,270] jepsen worker 38 - jepsen.util 38 :ok :write [12 16] INFO [2022-01-10 12:53:31,271] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:31,271] jepsen worker 70 - jepsen.util 160 :ok :read [23 6] INFO [2022-01-10 12:53:31,271] jepsen worker 81 - jepsen.util 171 :ok :write [27 13] INFO [2022-01-10 12:53:31,272] jepsen worker 38 - jepsen.util 38 :ok :read [12 16] INFO [2022-01-10 12:53:31,272] jepsen worker 81 - jepsen.util 171 :invoke :write [27 11] INFO [2022-01-10 12:53:31,273] jepsen worker 39 - jepsen.util 129 :ok :write [13 47] INFO [2022-01-10 12:53:31,273] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:31,274] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,274] jepsen worker 38 - jepsen.util 38 :ok :read [12 16] INFO [2022-01-10 12:53:31,274] jepsen worker 70 - jepsen.util 160 :invoke :write [23 20] INFO [2022-01-10 12:53:31,275] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:31,275] jepsen worker 39 - jepsen.util 129 :ok :read [13 47] INFO [2022-01-10 12:53:31,275] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,276] jepsen worker 38 - jepsen.util 38 :ok :read [12 16] INFO [2022-01-10 12:53:31,276] jepsen worker 39 - jepsen.util 129 :ok :read [13 47] INFO [2022-01-10 12:53:31,276] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:31,277] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,277] jepsen worker 38 - jepsen.util 38 :ok :read [12 16] INFO [2022-01-10 12:53:31,278] jepsen worker 38 - jepsen.util 38 :invoke :write [12 24] INFO [2022-01-10 12:53:31,278] jepsen worker 3 - jepsen.util 3 :ok :write [1 17] INFO [2022-01-10 12:53:31,278] jepsen worker 39 - jepsen.util 129 :ok :read [13 47] INFO [2022-01-10 12:53:31,279] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:31,279] jepsen worker 39 - jepsen.util 129 :invoke :write [13 37] INFO [2022-01-10 12:53:31,280] jepsen worker 3 - jepsen.util 3 :ok :read [1 17] INFO [2022-01-10 12:53:31,280] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:31,281] jepsen worker 3 - jepsen.util 3 :ok :read [1 17] INFO [2022-01-10 12:53:31,281] jepsen worker 3 - jepsen.util 3 :invoke :write [1 49] INFO [2022-01-10 12:53:31,284] jepsen worker 2 - jepsen.util 92 :ok :write [0 1] INFO [2022-01-10 12:53:31,285] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,286] jepsen worker 2 - jepsen.util 92 :ok :read [0 1] INFO [2022-01-10 12:53:31,286] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,287] jepsen worker 2 - jepsen.util 92 :ok :read [0 1] INFO [2022-01-10 12:53:31,287] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,288] jepsen worker 2 - jepsen.util 92 :ok :read [0 1] INFO [2022-01-10 12:53:31,289] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,289] jepsen worker 2 - jepsen.util 92 :ok :read [0 1] INFO [2022-01-10 12:53:31,290] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,291] jepsen worker 2 - jepsen.util 92 :ok :read [0 1] INFO [2022-01-10 12:53:31,291] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,292] jepsen worker 2 - jepsen.util 92 :ok :read [0 1] INFO [2022-01-10 12:53:31,292] jepsen worker 2 - jepsen.util 92 :invoke :write [0 45] INFO [2022-01-10 12:53:31,298] jepsen worker 88 - jepsen.util 178 :ok :write [29 35] INFO [2022-01-10 12:53:31,299] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:31,299] jepsen worker 88 - jepsen.util 178 :ok :read [29 35] INFO [2022-01-10 12:53:31,300] jepsen worker 88 - jepsen.util 178 :invoke :write [29 7] INFO [2022-01-10 12:53:31,366] jepsen worker 61 - jepsen.util 61 :ok :write [20 12] INFO [2022-01-10 12:53:31,367] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:31,368] jepsen worker 51 - jepsen.util 141 :ok :write [17 37] INFO [2022-01-10 12:53:31,368] jepsen worker 61 - jepsen.util 61 :ok :read [20 12] INFO [2022-01-10 12:53:31,368] jepsen worker 61 - jepsen.util 61 :invoke :write [20 23] INFO [2022-01-10 12:53:31,369] jepsen worker 51 - jepsen.util 141 :invoke :write [17 19] INFO [2022-01-10 12:53:31,373] jepsen worker 84 - jepsen.util 84 :ok :write [28 44] INFO [2022-01-10 12:53:31,374] jepsen worker 68 - jepsen.util 68 :ok :write [22 27] INFO [2022-01-10 12:53:31,374] jepsen worker 84 - jepsen.util 84 :invoke :write [28 31] INFO [2022-01-10 12:53:31,375] jepsen worker 39 - jepsen.util 129 :ok :write [13 37] INFO [2022-01-10 12:53:31,375] jepsen worker 68 - jepsen.util 68 :invoke :write [22 22] INFO [2022-01-10 12:53:31,376] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,377] jepsen worker 80 - jepsen.util 260 :ok :write [26 1] INFO [2022-01-10 12:53:31,377] jepsen worker 39 - jepsen.util 129 :ok :read [13 37] INFO [2022-01-10 12:53:31,377] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,378] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,378] jepsen worker 80 - jepsen.util 260 :ok :read [26 1] INFO [2022-01-10 12:53:31,379] jepsen worker 39 - jepsen.util 129 :ok :read [13 37] INFO [2022-01-10 12:53:31,379] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,379] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,379] jepsen worker 22 - jepsen.util 112 :ok :write [7 10] INFO [2022-01-10 12:53:31,380] jepsen worker 80 - jepsen.util 260 :ok :read [26 1] INFO [2022-01-10 12:53:31,380] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,380] jepsen worker 39 - jepsen.util 129 :ok :read [13 37] INFO [2022-01-10 12:53:31,380] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,381] jepsen worker 2 - jepsen.util 92 :ok :write [0 45] INFO [2022-01-10 12:53:31,381] jepsen worker 22 - jepsen.util 112 :ok :read [7 10] INFO [2022-01-10 12:53:31,381] jepsen worker 39 - jepsen.util 129 :invoke :write [13 22] INFO [2022-01-10 12:53:31,381] jepsen worker 80 - jepsen.util 260 :ok :read [26 1] INFO [2022-01-10 12:53:31,382] jepsen worker 22 - jepsen.util 112 :invoke :write [7 16] INFO [2022-01-10 12:53:31,383] jepsen worker 2 - jepsen.util 92 :invoke :write [0 15] INFO [2022-01-10 12:53:31,384] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,384] jepsen worker 13 - jepsen.util 13 :ok :write [4 18] INFO [2022-01-10 12:53:31,384] jepsen worker 80 - jepsen.util 260 :ok :read [26 1] INFO [2022-01-10 12:53:31,385] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,385] jepsen worker 80 - jepsen.util 260 :invoke :write [26 42] INFO [2022-01-10 12:53:31,385] jepsen worker 88 - jepsen.util 178 :ok :write [29 7] INFO [2022-01-10 12:53:31,386] jepsen worker 13 - jepsen.util 13 :ok :read [4 18] INFO [2022-01-10 12:53:31,386] jepsen worker 33 - jepsen.util 123 :ok :write [11 45] INFO [2022-01-10 12:53:31,386] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:31,387] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:31,387] jepsen worker 88 - jepsen.util 178 :ok :read [29 7] INFO [2022-01-10 12:53:31,387] jepsen worker 33 - jepsen.util 123 :ok :read [11 45] INFO [2022-01-10 12:53:31,387] jepsen worker 13 - jepsen.util 13 :invoke :write [4 29] INFO [2022-01-10 12:53:31,388] jepsen worker 88 - jepsen.util 178 :invoke :write [29 0] INFO [2022-01-10 12:53:31,388] jepsen worker 33 - jepsen.util 123 :invoke :write [11 44] INFO [2022-01-10 12:53:31,389] jepsen worker 70 - jepsen.util 160 :ok :write [23 20] INFO [2022-01-10 12:53:31,390] jepsen worker 70 - jepsen.util 160 :invoke :write [23 17] INFO [2022-01-10 12:53:31,392] jepsen worker 15 - jepsen.util 15 :ok :write [5 7] INFO [2022-01-10 12:53:31,393] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:31,394] jepsen worker 15 - jepsen.util 15 :ok :read [5 7] INFO [2022-01-10 12:53:31,395] jepsen worker 15 - jepsen.util 15 :invoke :write [5 49] INFO [2022-01-10 12:53:31,395] jepsen worker 38 - jepsen.util 38 :ok :write [12 24] INFO [2022-01-10 12:53:31,396] jepsen worker 38 - jepsen.util 38 :invoke :write [12 6] INFO [2022-01-10 12:53:31,398] jepsen worker 43 - jepsen.util 43 :ok :write [14 34] INFO [2022-01-10 12:53:31,399] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:31,399] jepsen worker 3 - jepsen.util 3 :ok :write [1 49] INFO [2022-01-10 12:53:31,400] jepsen worker 43 - jepsen.util 43 :ok :read [14 34] INFO [2022-01-10 12:53:31,400] jepsen worker 3 - jepsen.util 3 :invoke :write [1 37] INFO [2022-01-10 12:53:31,401] jepsen worker 43 - jepsen.util 43 :invoke :write [14 32] INFO [2022-01-10 12:53:31,402] jepsen worker 39 - jepsen.util 129 :ok :write [13 22] INFO [2022-01-10 12:53:31,403] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,404] jepsen worker 50 - jepsen.util 50 :ok :write [16 33] INFO [2022-01-10 12:53:31,404] jepsen worker 39 - jepsen.util 129 :ok :read [13 22] INFO [2022-01-10 12:53:31,405] jepsen worker 51 - jepsen.util 141 :ok :write [17 19] INFO [2022-01-10 12:53:31,405] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:31,406] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,406] jepsen worker 50 - jepsen.util 50 :ok :read [16 33] INFO [2022-01-10 12:53:31,407] jepsen worker 39 - jepsen.util 129 :ok :read [13 22] INFO [2022-01-10 12:53:31,407] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:31,407] jepsen worker 50 - jepsen.util 50 :invoke :write [16 46] INFO [2022-01-10 12:53:31,408] jepsen worker 51 - jepsen.util 141 :ok :read [17 19] INFO [2022-01-10 12:53:31,408] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,409] jepsen worker 51 - jepsen.util 141 :invoke :write [17 15] INFO [2022-01-10 12:53:31,410] jepsen worker 2 - jepsen.util 92 :ok :write [0 15] INFO [2022-01-10 12:53:31,410] jepsen worker 39 - jepsen.util 129 :ok :read [13 22] INFO [2022-01-10 12:53:31,410] jepsen worker 39 - jepsen.util 129 :invoke :write [13 17] INFO [2022-01-10 12:53:31,411] jepsen worker 75 - jepsen.util 165 :ok :write [25 14] INFO [2022-01-10 12:53:31,411] jepsen worker 2 - jepsen.util 92 :invoke :write [0 6] INFO [2022-01-10 12:53:31,412] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:31,412] jepsen worker 75 - jepsen.util 165 :ok :read [25 14] INFO [2022-01-10 12:53:31,413] jepsen worker 75 - jepsen.util 165 :invoke :write [25 31] INFO [2022-01-10 12:53:31,413] jepsen worker 84 - jepsen.util 84 :ok :write [28 31] INFO [2022-01-10 12:53:31,414] jepsen worker 89 - jepsen.util 269 :ok :write [29 20] INFO [2022-01-10 12:53:31,414] jepsen worker 84 - jepsen.util 84 :invoke :write [28 27] INFO [2022-01-10 12:53:31,415] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:31,416] jepsen worker 89 - jepsen.util 269 :ok :read [29 20] INFO [2022-01-10 12:53:31,416] jepsen worker 89 - jepsen.util 269 :invoke :write [29 35] INFO [2022-01-10 12:53:31,417] jepsen worker 80 - jepsen.util 260 :ok :write [26 42] INFO [2022-01-10 12:53:31,418] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,419] jepsen worker 80 - jepsen.util 260 :ok :read [26 42] INFO [2022-01-10 12:53:31,419] jepsen worker 7 - jepsen.util 7 :ok :write [2 4] INFO [2022-01-10 12:53:31,419] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,420] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,420] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:31,421] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,421] jepsen worker 80 - jepsen.util 260 :ok :read [26 42] INFO [2022-01-10 12:53:31,422] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:31,422] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,423] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,423] jepsen worker 80 - jepsen.util 260 :ok :read [26 42] INFO [2022-01-10 12:53:31,424] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,425] jepsen worker 13 - jepsen.util 13 :ok :write [4 29] INFO [2022-01-10 12:53:31,425] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:31,426] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,426] jepsen worker 80 - jepsen.util 260 :ok :read [26 42] INFO [2022-01-10 12:53:31,426] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,427] jepsen worker 13 - jepsen.util 13 :ok :read [4 29] INFO [2022-01-10 12:53:31,427] jepsen worker 80 - jepsen.util 260 :invoke :write [26 38] INFO [2022-01-10 12:53:31,428] jepsen worker 81 - jepsen.util 171 :ok :write [27 11] INFO [2022-01-10 12:53:31,428] jepsen worker 39 - jepsen.util 129 :ok :write [13 17] INFO [2022-01-10 12:53:31,428] jepsen worker 13 - jepsen.util 13 :invoke :write [4 19] INFO [2022-01-10 12:53:31,428] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:31,429] jepsen worker 81 - jepsen.util 171 :invoke :write [27 20] INFO [2022-01-10 12:53:31,430] jepsen worker 70 - jepsen.util 160 :ok :write [23 17] INFO [2022-01-10 12:53:31,430] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,431] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,431] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:31,432] jepsen worker 70 - jepsen.util 160 :invoke :write [23 7] INFO [2022-01-10 12:53:31,432] jepsen worker 39 - jepsen.util 129 :ok :read [13 17] INFO [2022-01-10 12:53:31,432] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,432] jepsen worker 61 - jepsen.util 61 :ok :write [20 23] INFO [2022-01-10 12:53:31,433] jepsen worker 7 - jepsen.util 7 :ok :read [2 4] INFO [2022-01-10 12:53:31,433] jepsen worker 61 - jepsen.util 61 :invoke :write [20 4] INFO [2022-01-10 12:53:31,434] jepsen worker 7 - jepsen.util 7 :invoke :write [2 21] INFO [2022-01-10 12:53:31,435] jepsen worker 39 - jepsen.util 129 :invoke :write [13 34] INFO [2022-01-10 12:53:31,437] jepsen worker 2 - jepsen.util 92 :ok :write [0 6] INFO [2022-01-10 12:53:31,437] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,438] jepsen worker 68 - jepsen.util 68 :ok :write [22 22] INFO [2022-01-10 12:53:31,439] jepsen worker 2 - jepsen.util 92 :ok :read [0 6] INFO [2022-01-10 12:53:31,439] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:31,439] jepsen worker 38 - jepsen.util 38 :ok :write [12 6] INFO [2022-01-10 12:53:31,440] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,440] jepsen worker 38 - jepsen.util 38 :invoke :write [12 15] INFO [2022-01-10 12:53:31,440] jepsen worker 68 - jepsen.util 68 :ok :read [22 22] INFO [2022-01-10 12:53:31,441] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:31,441] jepsen worker 2 - jepsen.util 92 :ok :read [0 6] INFO [2022-01-10 12:53:31,442] jepsen worker 68 - jepsen.util 68 :ok :read [22 22] INFO [2022-01-10 12:53:31,442] jepsen worker 2 - jepsen.util 92 :invoke :write [0 46] INFO [2022-01-10 12:53:31,443] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:31,443] jepsen worker 68 - jepsen.util 68 :ok :read [22 22] INFO [2022-01-10 12:53:31,444] jepsen worker 3 - jepsen.util 3 :ok :write [1 37] INFO [2022-01-10 12:53:31,444] jepsen worker 68 - jepsen.util 68 :invoke :write [22 13] INFO [2022-01-10 12:53:31,445] jepsen worker 22 - jepsen.util 112 :ok :write [7 16] INFO [2022-01-10 12:53:31,445] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:31,446] jepsen worker 22 - jepsen.util 112 :invoke :write [7 4] INFO [2022-01-10 12:53:31,446] jepsen worker 3 - jepsen.util 3 :ok :read [1 37] INFO [2022-01-10 12:53:31,447] jepsen worker 3 - jepsen.util 3 :invoke :write [1 44] INFO [2022-01-10 12:53:31,450] jepsen worker 51 - jepsen.util 141 :ok :write [17 15] INFO [2022-01-10 12:53:31,451] jepsen worker 51 - jepsen.util 141 :invoke :write [17 48] INFO [2022-01-10 12:53:31,452] jepsen worker 33 - jepsen.util 123 :ok :write [11 44] INFO [2022-01-10 12:53:31,452] jepsen worker 33 - jepsen.util 123 :invoke :write [11 34] INFO [2022-01-10 12:53:31,456] jepsen worker 84 - jepsen.util 84 :ok :write [28 27] INFO [2022-01-10 12:53:31,457] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:31,457] jepsen worker 84 - jepsen.util 84 :ok :read [28 27] INFO [2022-01-10 12:53:31,458] jepsen worker 84 - jepsen.util 84 :invoke :write [28 30] INFO [2022-01-10 12:53:31,460] jepsen worker 15 - jepsen.util 15 :ok :write [5 49] INFO [2022-01-10 12:53:31,460] jepsen worker 80 - jepsen.util 260 :ok :write [26 38] INFO [2022-01-10 12:53:31,460] jepsen worker 15 - jepsen.util 15 :invoke :write [5 15] INFO [2022-01-10 12:53:31,461] jepsen worker 80 - jepsen.util 260 :invoke :write [26 19] INFO [2022-01-10 12:53:31,465] jepsen worker 13 - jepsen.util 13 :ok :write [4 19] INFO [2022-01-10 12:53:31,465] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,466] jepsen worker 43 - jepsen.util 43 :ok :write [14 32] INFO [2022-01-10 12:53:31,466] jepsen worker 13 - jepsen.util 13 :ok :read [4 19] INFO [2022-01-10 12:53:31,466] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:31,467] jepsen worker 13 - jepsen.util 13 :invoke :write [4 22] INFO [2022-01-10 12:53:31,467] jepsen worker 43 - jepsen.util 43 :ok :read [14 32] INFO [2022-01-10 12:53:31,468] jepsen worker 43 - jepsen.util 43 :invoke :write [14 48] INFO [2022-01-10 12:53:31,469] jepsen worker 71 - jepsen.util 161 :ok :write [23 35] INFO [2022-01-10 12:53:31,469] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,470] jepsen worker 71 - jepsen.util 161 :ok :read [23 35] INFO [2022-01-10 12:53:31,471] jepsen worker 71 - jepsen.util 161 :invoke :write [23 46] INFO [2022-01-10 12:53:31,472] jepsen worker 50 - jepsen.util 50 :ok :write [16 46] INFO [2022-01-10 12:53:31,472] jepsen worker 50 - jepsen.util 50 :invoke :write [16 10] INFO [2022-01-10 12:53:31,477] jepsen worker 75 - jepsen.util 165 :ok :write [25 31] INFO [2022-01-10 12:53:31,478] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:31,478] jepsen worker 75 - jepsen.util 165 :ok :read [25 31] INFO [2022-01-10 12:53:31,479] jepsen worker 75 - jepsen.util 165 :invoke :write [25 36] INFO [2022-01-10 12:53:31,483] jepsen worker 51 - jepsen.util 141 :ok :write [17 48] INFO [2022-01-10 12:53:31,484] jepsen worker 51 - jepsen.util 141 :invoke :write [17 40] INFO [2022-01-10 12:53:31,491] jepsen worker 80 - jepsen.util 260 :ok :write [26 19] INFO [2022-01-10 12:53:31,491] jepsen worker 61 - jepsen.util 61 :ok :write [20 4] INFO [2022-01-10 12:53:31,491] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,492] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:31,492] jepsen worker 80 - jepsen.util 260 :ok :read [26 19] INFO [2022-01-10 12:53:31,493] jepsen worker 61 - jepsen.util 61 :ok :read [20 4] INFO [2022-01-10 12:53:31,493] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,494] jepsen worker 80 - jepsen.util 260 :ok :read [26 19] INFO [2022-01-10 12:53:31,494] jepsen worker 61 - jepsen.util 61 :invoke :write [20 18] INFO [2022-01-10 12:53:31,494] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,495] jepsen worker 80 - jepsen.util 260 :ok :read [26 19] INFO [2022-01-10 12:53:31,496] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,496] jepsen worker 80 - jepsen.util 260 :ok :read [26 19] INFO [2022-01-10 12:53:31,497] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,497] jepsen worker 13 - jepsen.util 13 :ok :write [4 22] INFO [2022-01-10 12:53:31,498] jepsen worker 80 - jepsen.util 260 :ok :read [26 19] INFO [2022-01-10 12:53:31,498] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,498] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:31,499] jepsen worker 13 - jepsen.util 13 :ok :read [4 22] INFO [2022-01-10 12:53:31,499] jepsen worker 80 - jepsen.util 260 :ok :read [26 19] INFO [2022-01-10 12:53:31,499] jepsen worker 13 - jepsen.util 13 :invoke :write [4 6] INFO [2022-01-10 12:53:31,500] jepsen worker 80 - jepsen.util 260 :invoke :write [26 38] INFO [2022-01-10 12:53:31,501] jepsen worker 22 - jepsen.util 112 :ok :write [7 4] INFO [2022-01-10 12:53:31,501] jepsen worker 71 - jepsen.util 161 :ok :write [23 46] INFO [2022-01-10 12:53:31,501] jepsen worker 22 - jepsen.util 112 :invoke :write [7 44] INFO [2022-01-10 12:53:31,502] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,502] jepsen worker 71 - jepsen.util 161 :ok :read [23 46] INFO [2022-01-10 12:53:31,503] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,504] jepsen worker 71 - jepsen.util 161 :ok :read [23 46] INFO [2022-01-10 12:53:31,505] jepsen worker 51 - jepsen.util 141 :ok :write [17 40] INFO [2022-01-10 12:53:31,505] jepsen worker 71 - jepsen.util 161 :invoke :write [23 5] INFO [2022-01-10 12:53:31,505] jepsen worker 51 - jepsen.util 141 :invoke :write [17 28] INFO [2022-01-10 12:53:31,512] jepsen worker 80 - jepsen.util 260 :ok :write [26 38] INFO [2022-01-10 12:53:31,513] jepsen worker 80 - jepsen.util 260 :invoke :write [26 45] INFO [2022-01-10 12:53:31,515] jepsen worker 43 - jepsen.util 43 :ok :write [14 48] INFO [2022-01-10 12:53:31,516] jepsen worker 43 - jepsen.util 43 :invoke :write [14 35] INFO [2022-01-10 12:53:31,517] jepsen worker 13 - jepsen.util 13 :ok :write [4 6] INFO [2022-01-10 12:53:31,518] jepsen worker 13 - jepsen.util 13 :invoke :write [4 39] INFO [2022-01-10 12:53:31,522] jepsen worker 71 - jepsen.util 161 :ok :write [23 5] INFO [2022-01-10 12:53:31,523] jepsen worker 71 - jepsen.util 161 :invoke :write [23 48] INFO [2022-01-10 12:53:31,526] jepsen worker 75 - jepsen.util 165 :ok :write [25 36] INFO [2022-01-10 12:53:31,526] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:31,527] jepsen worker 75 - jepsen.util 165 :ok :read [25 36] INFO [2022-01-10 12:53:31,528] jepsen worker 51 - jepsen.util 141 :ok :write [17 28] INFO [2022-01-10 12:53:31,528] jepsen worker 75 - jepsen.util 165 :invoke :write [25 26] INFO [2022-01-10 12:53:31,528] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:31,529] jepsen worker 51 - jepsen.util 141 :ok :read [17 28] INFO [2022-01-10 12:53:31,530] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:31,531] jepsen worker 61 - jepsen.util 61 :ok :write [20 18] INFO [2022-01-10 12:53:31,531] jepsen worker 51 - jepsen.util 141 :ok :read [17 28] INFO [2022-01-10 12:53:31,531] jepsen worker 61 - jepsen.util 61 :invoke :write [20 35] INFO [2022-01-10 12:53:31,532] jepsen worker 51 - jepsen.util 141 :invoke :write [17 16] INFO [2022-01-10 12:53:31,534] jepsen worker 80 - jepsen.util 260 :ok :write [26 45] INFO [2022-01-10 12:53:31,535] jepsen worker 80 - jepsen.util 260 :invoke :write [26 31] INFO [2022-01-10 12:53:31,535] jepsen worker 22 - jepsen.util 112 :ok :write [7 44] INFO [2022-01-10 12:53:31,536] jepsen worker 22 - jepsen.util 112 :invoke :write [7 36] INFO [2022-01-10 12:53:31,539] jepsen worker 13 - jepsen.util 13 :ok :write [4 39] INFO [2022-01-10 12:53:31,540] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,541] jepsen worker 13 - jepsen.util 13 :ok :read [4 39] INFO [2022-01-10 12:53:31,541] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,542] jepsen worker 13 - jepsen.util 13 :ok :read [4 39] INFO [2022-01-10 12:53:31,542] jepsen worker 43 - jepsen.util 43 :ok :write [14 35] INFO [2022-01-10 12:53:31,542] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,542] jepsen worker 43 - jepsen.util 43 :invoke :write [14 23] INFO [2022-01-10 12:53:31,543] jepsen worker 13 - jepsen.util 13 :ok :read [4 39] INFO [2022-01-10 12:53:31,543] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,545] jepsen worker 13 - jepsen.util 13 :ok :read [4 39] INFO [2022-01-10 12:53:31,545] jepsen worker 13 - jepsen.util 13 :invoke :write [4 10] INFO [2022-01-10 12:53:31,545] jepsen worker 71 - jepsen.util 161 :ok :write [23 48] INFO [2022-01-10 12:53:31,546] jepsen worker 71 - jepsen.util 161 :invoke :write [23 8] INFO [2022-01-10 12:53:31,566] jepsen worker 77 - jepsen.util 167 :info :write [25 17] :RequestTimeoutException INFO [2022-01-10 12:53:31,567] jepsen worker 77 - jepsen.util 257 :invoke :read [25 nil] INFO [2022-01-10 12:53:31,579] jepsen worker 88 - jepsen.util 178 :ok :write [29 0] INFO [2022-01-10 12:53:31,580] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:31,581] jepsen worker 88 - jepsen.util 178 :ok :read [29 0] INFO [2022-01-10 12:53:31,581] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:31,582] jepsen worker 88 - jepsen.util 178 :ok :read [29 0] INFO [2022-01-10 12:53:31,583] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:31,583] jepsen worker 88 - jepsen.util 178 :ok :read [29 0] INFO [2022-01-10 12:53:31,584] jepsen worker 7 - jepsen.util 7 :ok :write [2 21] INFO [2022-01-10 12:53:31,584] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:31,584] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,585] jepsen worker 38 - jepsen.util 38 :ok :write [12 15] INFO [2022-01-10 12:53:31,585] jepsen worker 88 - jepsen.util 178 :ok :read [29 0] INFO [2022-01-10 12:53:31,585] jepsen worker 7 - jepsen.util 7 :ok :read [2 21] INFO [2022-01-10 12:53:31,586] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:31,586] jepsen worker 7 - jepsen.util 7 :invoke :write [2 28] INFO [2022-01-10 12:53:31,586] jepsen worker 38 - jepsen.util 38 :ok :read [12 15] INFO [2022-01-10 12:53:31,587] jepsen worker 88 - jepsen.util 178 :invoke :write [29 42] INFO [2022-01-10 12:53:31,587] jepsen worker 38 - jepsen.util 38 :invoke :write [12 39] INFO [2022-01-10 12:53:31,588] jepsen worker 81 - jepsen.util 171 :ok :write [27 20] INFO [2022-01-10 12:53:31,589] jepsen worker 81 - jepsen.util 171 :invoke :write [27 38] INFO [2022-01-10 12:53:31,590] jepsen worker 3 - jepsen.util 3 :ok :write [1 44] INFO [2022-01-10 12:53:31,590] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:31,591] jepsen worker 3 - jepsen.util 3 :ok :read [1 44] INFO [2022-01-10 12:53:31,591] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:31,592] jepsen worker 3 - jepsen.util 3 :ok :read [1 44] INFO [2022-01-10 12:53:31,593] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:31,593] jepsen worker 3 - jepsen.util 3 :ok :read [1 44] INFO [2022-01-10 12:53:31,594] jepsen worker 3 - jepsen.util 3 :invoke :write [1 31] INFO [2022-01-10 12:53:31,596] jepsen worker 39 - jepsen.util 129 :ok :write [13 34] INFO [2022-01-10 12:53:31,596] jepsen worker 84 - jepsen.util 84 :ok :write [28 30] INFO [2022-01-10 12:53:31,597] jepsen worker 39 - jepsen.util 129 :invoke :write [13 7] INFO [2022-01-10 12:53:31,597] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:31,602] jepsen worker 2 - jepsen.util 92 :ok :write [0 46] INFO [2022-01-10 12:53:31,602] jepsen worker 2 - jepsen.util 92 :invoke :write [0 0] INFO [2022-01-10 12:53:31,604] jepsen worker 85 - jepsen.util 175 :info :write [28 23] :RequestTimeoutException INFO [2022-01-10 12:53:31,604] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:31,605] jepsen worker 85 - jepsen.util 265 :ok :read [28 30] INFO [2022-01-10 12:53:31,606] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:31,606] jepsen worker 34 - jepsen.util 124 :info :write [11 4] :RequestTimeoutException INFO [2022-01-10 12:53:31,607] jepsen worker 85 - jepsen.util 265 :ok :read [28 30] INFO [2022-01-10 12:53:31,607] jepsen worker 34 - jepsen.util 214 :invoke :write [11 22] INFO [2022-01-10 12:53:31,607] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:31,608] jepsen worker 85 - jepsen.util 265 :ok :read [28 30] INFO [2022-01-10 12:53:31,609] jepsen worker 85 - jepsen.util 265 :invoke :write [28 13] INFO [2022-01-10 12:53:31,609] jepsen worker 68 - jepsen.util 68 :ok :write [22 13] INFO [2022-01-10 12:53:31,609] jepsen worker 84 - jepsen.util 84 :ok :read [28 30] INFO [2022-01-10 12:53:31,610] jepsen worker 68 - jepsen.util 68 :invoke :write [22 17] INFO [2022-01-10 12:53:31,610] jepsen worker 84 - jepsen.util 84 :invoke :write [28 37] INFO [2022-01-10 12:53:31,617] jepsen worker 33 - jepsen.util 123 :ok :write [11 34] INFO [2022-01-10 12:53:31,618] jepsen worker 33 - jepsen.util 123 :invoke :write [11 3] INFO [2022-01-10 12:53:31,621] jepsen worker 15 - jepsen.util 15 :ok :write [5 15] INFO [2022-01-10 12:53:31,622] jepsen worker 15 - jepsen.util 15 :invoke :write [5 0] INFO [2022-01-10 12:53:31,624] jepsen worker 65 - jepsen.util 245 :info :write [21 23] :RequestTimeoutException INFO [2022-01-10 12:53:31,624] jepsen worker 65 - jepsen.util 335 :invoke :write [21 0] INFO [2022-01-10 12:53:31,626] jepsen worker 50 - jepsen.util 50 :ok :write [16 10] INFO [2022-01-10 12:53:31,627] jepsen worker 50 - jepsen.util 50 :invoke :write [16 47] INFO [2022-01-10 12:53:31,628] jepsen worker 18 - jepsen.util 198 :info :write [6 28] :RequestTimeoutException INFO [2022-01-10 12:53:31,629] jepsen worker 18 - jepsen.util 288 :invoke :read [6 nil] INFO [2022-01-10 12:53:31,630] jepsen worker 18 - jepsen.util 288 :ok :read [6 8] INFO [2022-01-10 12:53:31,631] jepsen worker 18 - jepsen.util 288 :invoke :read [6 nil] INFO [2022-01-10 12:53:31,631] jepsen worker 18 - jepsen.util 288 :ok :read [6 8] INFO [2022-01-10 12:53:31,632] jepsen worker 18 - jepsen.util 288 :invoke :write [6 33] INFO [2022-01-10 12:53:31,634] jepsen worker 7 - jepsen.util 7 :ok :write [2 28] INFO [2022-01-10 12:53:31,635] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,636] jepsen worker 7 - jepsen.util 7 :ok :read [2 28] INFO [2022-01-10 12:53:31,636] jepsen worker 7 - jepsen.util 7 :invoke :write [2 36] INFO [2022-01-10 12:53:31,638] jepsen worker 81 - jepsen.util 171 :ok :write [27 38] INFO [2022-01-10 12:53:31,638] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:31,639] jepsen worker 81 - jepsen.util 171 :ok :read [27 38] INFO [2022-01-10 12:53:31,639] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:31,640] jepsen worker 81 - jepsen.util 171 :ok :read [27 38] INFO [2022-01-10 12:53:31,641] jepsen worker 81 - jepsen.util 171 :invoke :write [27 41] INFO [2022-01-10 12:53:31,644] jepsen worker 47 - jepsen.util 227 :info :write [15 26] :RequestTimeoutException INFO [2022-01-10 12:53:31,645] jepsen worker 47 - jepsen.util 317 :invoke :write [15 38] INFO [2022-01-10 12:53:31,646] jepsen worker 56 - jepsen.util 236 :info :write [18 0] :RequestTimeoutException INFO [2022-01-10 12:53:31,647] jepsen worker 63 - jepsen.util 333 :fail :read [21 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:31,648] jepsen worker 56 - jepsen.util 326 :invoke :write [18 33] INFO [2022-01-10 12:53:31,648] jepsen worker 63 - jepsen.util 333 :invoke :read [21 nil] INFO [2022-01-10 12:53:31,651] jepsen worker 68 - jepsen.util 68 :ok :write [22 17] INFO [2022-01-10 12:53:31,651] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:31,652] jepsen worker 68 - jepsen.util 68 :ok :read [22 17] INFO [2022-01-10 12:53:31,652] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:31,653] jepsen worker 68 - jepsen.util 68 :ok :read [22 17] INFO [2022-01-10 12:53:31,654] jepsen worker 68 - jepsen.util 68 :invoke :write [22 28] INFO [2022-01-10 12:53:31,676] jepsen worker 20 - jepsen.util 200 :info :write [6 3] :RequestTimeoutException INFO [2022-01-10 12:53:31,677] jepsen worker 20 - jepsen.util 290 :invoke :write [6 20] INFO [2022-01-10 12:53:31,687] jepsen worker 75 - jepsen.util 165 :ok :write [25 26] INFO [2022-01-10 12:53:31,688] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:31,689] jepsen worker 75 - jepsen.util 165 :ok :read [25 26] INFO [2022-01-10 12:53:31,689] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:31,690] jepsen worker 75 - jepsen.util 165 :ok :read [25 26] INFO [2022-01-10 12:53:31,690] jepsen worker 75 - jepsen.util 165 :invoke :write [25 31] INFO [2022-01-10 12:53:31,691] jepsen worker 51 - jepsen.util 141 :ok :write [17 16] INFO [2022-01-10 12:53:31,692] jepsen worker 51 - jepsen.util 141 :invoke :write [17 11] INFO [2022-01-10 12:53:31,730] jepsen worker 61 - jepsen.util 61 :ok :write [20 35] INFO [2022-01-10 12:53:31,730] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:31,731] jepsen worker 61 - jepsen.util 61 :ok :read [20 35] INFO [2022-01-10 12:53:31,731] jepsen worker 80 - jepsen.util 260 :ok :write [26 31] INFO [2022-01-10 12:53:31,732] jepsen worker 61 - jepsen.util 61 :invoke :write [20 11] INFO [2022-01-10 12:53:31,732] jepsen worker 80 - jepsen.util 260 :invoke :write [26 8] INFO [2022-01-10 12:53:31,736] jepsen worker 13 - jepsen.util 13 :ok :write [4 10] INFO [2022-01-10 12:53:31,737] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:31,738] jepsen worker 13 - jepsen.util 13 :ok :read [4 10] INFO [2022-01-10 12:53:31,738] jepsen worker 22 - jepsen.util 112 :ok :write [7 36] INFO [2022-01-10 12:53:31,738] jepsen worker 13 - jepsen.util 13 :invoke :write [4 38] INFO [2022-01-10 12:53:31,739] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,739] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:31,740] jepsen worker 22 - jepsen.util 112 :invoke :write [7 43] INFO [2022-01-10 12:53:31,741] jepsen worker 43 - jepsen.util 43 :ok :write [14 23] INFO [2022-01-10 12:53:31,741] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:31,742] jepsen worker 43 - jepsen.util 43 :ok :read [14 23] INFO [2022-01-10 12:53:31,743] jepsen worker 43 - jepsen.util 43 :invoke :write [14 32] INFO [2022-01-10 12:53:31,744] jepsen worker 88 - jepsen.util 178 :ok :write [29 42] INFO [2022-01-10 12:53:31,745] jepsen worker 88 - jepsen.util 178 :invoke :write [29 19] INFO [2022-01-10 12:53:31,761] jepsen worker 34 - jepsen.util 214 :ok :write [11 22] INFO [2022-01-10 12:53:31,761] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:31,762] jepsen worker 34 - jepsen.util 214 :ok :read [11 22] INFO [2022-01-10 12:53:31,762] jepsen worker 34 - jepsen.util 214 :invoke :write [11 7] INFO [2022-01-10 12:53:31,768] jepsen worker 15 - jepsen.util 15 :ok :write [5 0] INFO [2022-01-10 12:53:31,768] jepsen worker 15 - jepsen.util 15 :invoke :write [5 3] INFO [2022-01-10 12:53:31,771] jepsen worker 50 - jepsen.util 50 :ok :write [16 47] INFO [2022-01-10 12:53:31,771] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:31,771] jepsen worker 7 - jepsen.util 7 :ok :write [2 36] INFO [2022-01-10 12:53:31,772] jepsen worker 50 - jepsen.util 50 :ok :read [16 47] INFO [2022-01-10 12:53:31,772] jepsen worker 7 - jepsen.util 7 :invoke :write [2 8] INFO [2022-01-10 12:53:31,772] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:31,773] jepsen worker 50 - jepsen.util 50 :ok :read [16 47] INFO [2022-01-10 12:53:31,773] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:31,774] jepsen worker 50 - jepsen.util 50 :ok :read [16 47] INFO [2022-01-10 12:53:31,775] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:31,775] jepsen worker 50 - jepsen.util 50 :ok :read [16 47] INFO [2022-01-10 12:53:31,776] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:31,776] jepsen worker 50 - jepsen.util 50 :ok :read [16 47] INFO [2022-01-10 12:53:31,777] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:31,777] jepsen worker 50 - jepsen.util 50 :ok :read [16 47] INFO [2022-01-10 12:53:31,778] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:31,779] jepsen worker 81 - jepsen.util 171 :ok :write [27 41] INFO [2022-01-10 12:53:31,779] jepsen worker 50 - jepsen.util 50 :ok :read [16 47] INFO [2022-01-10 12:53:31,780] jepsen worker 50 - jepsen.util 50 :invoke :write [16 35] INFO [2022-01-10 12:53:31,780] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:31,781] jepsen worker 81 - jepsen.util 171 :ok :read [27 41] INFO [2022-01-10 12:53:31,782] jepsen worker 81 - jepsen.util 171 :invoke :write [27 29] INFO [2022-01-10 12:53:31,784] jepsen worker 68 - jepsen.util 68 :ok :write [22 28] INFO [2022-01-10 12:53:31,784] jepsen worker 68 - jepsen.util 68 :invoke :write [22 45] INFO [2022-01-10 12:53:31,796] jepsen worker 71 - jepsen.util 161 :ok :write [23 8] INFO [2022-01-10 12:53:31,796] jepsen worker 39 - jepsen.util 129 :ok :write [13 7] INFO [2022-01-10 12:53:31,797] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:31,797] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:31,798] jepsen worker 71 - jepsen.util 161 :ok :read [23 8] INFO [2022-01-10 12:53:31,799] jepsen worker 39 - jepsen.util 129 :ok :read [13 7] INFO [2022-01-10 12:53:31,799] jepsen worker 71 - jepsen.util 161 :invoke :write [23 15] INFO [2022-01-10 12:53:31,799] jepsen worker 2 - jepsen.util 92 :ok :write [0 0] INFO [2022-01-10 12:53:31,799] jepsen worker 39 - jepsen.util 129 :invoke :write [13 26] INFO [2022-01-10 12:53:31,800] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:31,801] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:31,802] jepsen worker 2 - jepsen.util 92 :invoke :write [0 37] INFO [2022-01-10 12:53:31,804] jepsen worker 34 - jepsen.util 214 :ok :write [11 7] INFO [2022-01-10 12:53:31,804] jepsen worker 34 - jepsen.util 214 :invoke :write [11 37] INFO [2022-01-10 12:53:31,811] jepsen worker 15 - jepsen.util 15 :ok :write [5 3] INFO [2022-01-10 12:53:31,812] jepsen worker 15 - jepsen.util 15 :invoke :write [5 24] INFO [2022-01-10 12:53:31,815] jepsen worker 7 - jepsen.util 7 :ok :write [2 8] INFO [2022-01-10 12:53:31,815] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,816] jepsen worker 7 - jepsen.util 7 :ok :read [2 8] INFO [2022-01-10 12:53:31,817] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,818] jepsen worker 7 - jepsen.util 7 :ok :read [2 8] INFO [2022-01-10 12:53:31,818] jepsen worker 7 - jepsen.util 7 :invoke :write [2 39] INFO [2022-01-10 12:53:31,819] jepsen worker 75 - jepsen.util 165 :ok :write [25 31] INFO [2022-01-10 12:53:31,820] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:31,820] jepsen worker 38 - jepsen.util 38 :ok :write [12 39] INFO [2022-01-10 12:53:31,821] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:31,821] jepsen worker 75 - jepsen.util 165 :ok :read [25 31] INFO [2022-01-10 12:53:31,822] jepsen worker 38 - jepsen.util 38 :ok :read [12 39] INFO [2022-01-10 12:53:31,822] jepsen worker 75 - jepsen.util 165 :invoke :write [25 10] INFO [2022-01-10 12:53:31,823] jepsen worker 38 - jepsen.util 38 :invoke :write [12 40] INFO [2022-01-10 12:53:31,823] jepsen worker 50 - jepsen.util 50 :ok :write [16 35] INFO [2022-01-10 12:53:31,824] jepsen worker 50 - jepsen.util 50 :invoke :write [16 0] INFO [2022-01-10 12:53:31,828] jepsen worker 81 - jepsen.util 171 :ok :write [27 29] INFO [2022-01-10 12:53:31,829] jepsen worker 81 - jepsen.util 171 :invoke :write [27 3] INFO [2022-01-10 12:53:31,833] jepsen worker 68 - jepsen.util 68 :ok :write [22 45] INFO [2022-01-10 12:53:31,834] jepsen worker 68 - jepsen.util 68 :invoke :write [22 44] INFO [2022-01-10 12:53:31,835] jepsen worker 34 - jepsen.util 214 :ok :write [11 37] INFO [2022-01-10 12:53:31,836] jepsen worker 34 - jepsen.util 214 :invoke :write [11 1] INFO [2022-01-10 12:53:31,841] jepsen worker 15 - jepsen.util 15 :ok :write [5 24] INFO [2022-01-10 12:53:31,842] jepsen worker 15 - jepsen.util 15 :invoke :write [5 27] INFO [2022-01-10 12:53:31,843] jepsen worker 7 - jepsen.util 7 :ok :write [2 39] INFO [2022-01-10 12:53:31,844] jepsen worker 7 - jepsen.util 7 :invoke :write [2 24] INFO [2022-01-10 12:53:31,852] jepsen worker 50 - jepsen.util 50 :ok :write [16 0] INFO [2022-01-10 12:53:31,853] jepsen worker 50 - jepsen.util 50 :invoke :write [16 5] INFO [2022-01-10 12:53:31,856] jepsen worker 81 - jepsen.util 171 :ok :write [27 3] INFO [2022-01-10 12:53:31,857] jepsen worker 81 - jepsen.util 171 :invoke :write [27 6] INFO [2022-01-10 12:53:31,860] jepsen worker 68 - jepsen.util 68 :ok :write [22 44] INFO [2022-01-10 12:53:31,861] jepsen worker 68 - jepsen.util 68 :invoke :write [22 15] INFO [2022-01-10 12:53:31,863] jepsen worker 34 - jepsen.util 214 :ok :write [11 1] INFO [2022-01-10 12:53:31,864] jepsen worker 34 - jepsen.util 214 :invoke :write [11 7] INFO [2022-01-10 12:53:31,869] jepsen worker 7 - jepsen.util 7 :ok :write [2 24] INFO [2022-01-10 12:53:31,869] jepsen worker 7 - jepsen.util 7 :invoke :write [2 38] INFO [2022-01-10 12:53:31,874] jepsen worker 15 - jepsen.util 15 :ok :write [5 27] INFO [2022-01-10 12:53:31,874] jepsen worker 15 - jepsen.util 15 :invoke :write [5 33] INFO [2022-01-10 12:53:31,877] jepsen worker 50 - jepsen.util 50 :ok :write [16 5] INFO [2022-01-10 12:53:31,878] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:31,879] jepsen worker 50 - jepsen.util 50 :ok :read [16 5] INFO [2022-01-10 12:53:31,879] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:31,880] jepsen worker 50 - jepsen.util 50 :ok :read [16 5] INFO [2022-01-10 12:53:31,880] jepsen worker 50 - jepsen.util 50 :invoke :write [16 12] INFO [2022-01-10 12:53:31,883] jepsen worker 81 - jepsen.util 171 :ok :write [27 6] INFO [2022-01-10 12:53:31,883] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:31,884] jepsen worker 81 - jepsen.util 171 :ok :read [27 6] INFO [2022-01-10 12:53:31,885] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:31,885] jepsen worker 81 - jepsen.util 171 :ok :read [27 6] INFO [2022-01-10 12:53:31,886] jepsen worker 81 - jepsen.util 171 :invoke :write [27 47] INFO [2022-01-10 12:53:31,886] jepsen worker 68 - jepsen.util 68 :ok :write [22 15] INFO [2022-01-10 12:53:31,887] jepsen worker 68 - jepsen.util 68 :invoke :write [22 37] INFO [2022-01-10 12:53:31,891] jepsen worker 34 - jepsen.util 214 :ok :write [11 7] INFO [2022-01-10 12:53:31,891] jepsen worker 34 - jepsen.util 214 :invoke :write [11 31] INFO [2022-01-10 12:53:31,895] jepsen worker 7 - jepsen.util 7 :ok :write [2 38] INFO [2022-01-10 12:53:31,896] jepsen worker 7 - jepsen.util 7 :invoke :write [2 45] INFO [2022-01-10 12:53:31,900] jepsen worker 15 - jepsen.util 15 :ok :write [5 33] INFO [2022-01-10 12:53:31,901] jepsen worker 15 - jepsen.util 15 :invoke :write [5 11] INFO [2022-01-10 12:53:31,903] jepsen worker 50 - jepsen.util 50 :ok :write [16 12] INFO [2022-01-10 12:53:31,904] jepsen worker 50 - jepsen.util 50 :invoke :write [16 41] INFO [2022-01-10 12:53:31,906] jepsen worker 81 - jepsen.util 171 :ok :write [27 47] INFO [2022-01-10 12:53:31,907] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:31,908] jepsen worker 81 - jepsen.util 171 :ok :read [27 47] INFO [2022-01-10 12:53:31,908] jepsen worker 81 - jepsen.util 171 :invoke :write [27 6] INFO [2022-01-10 12:53:31,911] jepsen worker 68 - jepsen.util 68 :ok :write [22 37] INFO [2022-01-10 12:53:31,911] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:31,912] jepsen worker 68 - jepsen.util 68 :ok :read [22 37] INFO [2022-01-10 12:53:31,912] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:31,913] jepsen worker 68 - jepsen.util 68 :ok :read [22 37] INFO [2022-01-10 12:53:31,914] jepsen worker 68 - jepsen.util 68 :invoke :write [22 22] INFO [2022-01-10 12:53:31,938] jepsen worker 83 - jepsen.util 173 :info :write [27 9] :RequestTimeoutException INFO [2022-01-10 12:53:31,939] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:31,940] jepsen worker 83 - jepsen.util 263 :ok :read [27 47] INFO [2022-01-10 12:53:31,940] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:31,941] jepsen worker 83 - jepsen.util 263 :ok :read [27 47] INFO [2022-01-10 12:53:31,942] jepsen worker 83 - jepsen.util 263 :invoke :write [27 37] INFO [2022-01-10 12:53:31,976] jepsen worker 3 - jepsen.util 3 :ok :write [1 31] INFO [2022-01-10 12:53:31,976] jepsen worker 61 - jepsen.util 61 :ok :write [20 11] INFO [2022-01-10 12:53:31,976] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:31,977] jepsen worker 3 - jepsen.util 3 :invoke :write [1 34] INFO [2022-01-10 12:53:31,978] jepsen worker 61 - jepsen.util 61 :ok :read [20 11] INFO [2022-01-10 12:53:31,979] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:31,980] jepsen worker 61 - jepsen.util 61 :ok :read [20 11] INFO [2022-01-10 12:53:31,980] jepsen worker 22 - jepsen.util 112 :ok :write [7 43] INFO [2022-01-10 12:53:31,980] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:31,981] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,981] jepsen worker 34 - jepsen.util 214 :ok :write [11 31] INFO [2022-01-10 12:53:31,981] jepsen worker 61 - jepsen.util 61 :ok :read [20 11] INFO [2022-01-10 12:53:31,981] jepsen worker 86 - jepsen.util 176 :ok :write [28 2] INFO [2022-01-10 12:53:31,982] jepsen worker 22 - jepsen.util 112 :ok :read [7 43] INFO [2022-01-10 12:53:31,982] jepsen worker 34 - jepsen.util 214 :invoke :write [11 1] INFO [2022-01-10 12:53:31,983] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:31,983] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:31,983] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,984] jepsen worker 61 - jepsen.util 61 :ok :read [20 11] INFO [2022-01-10 12:53:31,984] jepsen worker 86 - jepsen.util 176 :ok :read [28 2] INFO [2022-01-10 12:53:31,984] jepsen worker 61 - jepsen.util 61 :invoke :write [20 16] INFO [2022-01-10 12:53:31,985] jepsen worker 86 - jepsen.util 176 :invoke :write [28 34] INFO [2022-01-10 12:53:31,985] jepsen worker 22 - jepsen.util 112 :ok :read [7 43] INFO [2022-01-10 12:53:31,985] jepsen worker 43 - jepsen.util 43 :ok :write [14 32] INFO [2022-01-10 12:53:31,985] jepsen worker 51 - jepsen.util 141 :ok :write [17 11] INFO [2022-01-10 12:53:31,986] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,986] jepsen worker 43 - jepsen.util 43 :invoke :write [14 35] INFO [2022-01-10 12:53:31,987] jepsen worker 51 - jepsen.util 141 :invoke :write [17 44] INFO [2022-01-10 12:53:31,989] jepsen worker 88 - jepsen.util 178 :ok :write [29 19] INFO [2022-01-10 12:53:31,989] jepsen worker 22 - jepsen.util 112 :ok :read [7 43] INFO [2022-01-10 12:53:31,990] jepsen worker 88 - jepsen.util 178 :invoke :write [29 22] INFO [2022-01-10 12:53:31,990] jepsen worker 7 - jepsen.util 7 :ok :write [2 45] INFO [2022-01-10 12:53:31,990] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:31,991] jepsen worker 80 - jepsen.util 260 :ok :write [26 8] INFO [2022-01-10 12:53:31,991] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,991] jepsen worker 22 - jepsen.util 112 :ok :read [7 43] INFO [2022-01-10 12:53:31,991] jepsen worker 80 - jepsen.util 260 :invoke :write [26 20] INFO [2022-01-10 12:53:31,992] jepsen worker 7 - jepsen.util 7 :ok :read [2 45] INFO [2022-01-10 12:53:31,992] jepsen worker 22 - jepsen.util 112 :invoke :write [7 0] INFO [2022-01-10 12:53:31,992] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,993] jepsen worker 7 - jepsen.util 7 :ok :read [2 45] INFO [2022-01-10 12:53:31,993] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,994] jepsen worker 7 - jepsen.util 7 :ok :read [2 45] INFO [2022-01-10 12:53:31,994] jepsen worker 15 - jepsen.util 15 :ok :write [5 11] INFO [2022-01-10 12:53:31,995] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:31,995] jepsen worker 15 - jepsen.util 15 :invoke :write [5 28] INFO [2022-01-10 12:53:31,996] jepsen worker 7 - jepsen.util 7 :ok :read [2 45] INFO [2022-01-10 12:53:31,996] jepsen worker 7 - jepsen.util 7 :invoke :write [2 4] INFO [2022-01-10 12:53:31,998] jepsen worker 39 - jepsen.util 129 :ok :write [13 26] INFO [2022-01-10 12:53:31,999] jepsen worker 39 - jepsen.util 129 :invoke :write [13 2] INFO [2022-01-10 12:53:31,999] jepsen worker 50 - jepsen.util 50 :ok :write [16 41] INFO [2022-01-10 12:53:31,999] jepsen worker 13 - jepsen.util 13 :ok :write [4 38] INFO [2022-01-10 12:53:32,000] jepsen worker 50 - jepsen.util 50 :invoke :write [16 20] INFO [2022-01-10 12:53:32,000] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:32,001] jepsen worker 13 - jepsen.util 13 :ok :read [4 38] INFO [2022-01-10 12:53:32,001] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:32,002] jepsen worker 13 - jepsen.util 13 :ok :read [4 38] INFO [2022-01-10 12:53:32,003] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:32,003] jepsen worker 13 - jepsen.util 13 :ok :read [4 38] INFO [2022-01-10 12:53:32,004] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:32,005] jepsen worker 13 - jepsen.util 13 :ok :read [4 38] INFO [2022-01-10 12:53:32,005] jepsen worker 13 - jepsen.util 13 :invoke :write [4 0] INFO [2022-01-10 12:53:32,007] jepsen worker 68 - jepsen.util 68 :ok :write [22 22] INFO [2022-01-10 12:53:32,008] jepsen worker 68 - jepsen.util 68 :invoke :write [22 15] INFO [2022-01-10 12:53:32,010] jepsen worker 2 - jepsen.util 92 :ok :write [0 37] INFO [2022-01-10 12:53:32,011] jepsen worker 71 - jepsen.util 161 :ok :write [23 15] INFO [2022-01-10 12:53:32,011] jepsen worker 2 - jepsen.util 92 :invoke :write [0 23] INFO [2022-01-10 12:53:32,011] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,012] jepsen worker 71 - jepsen.util 161 :ok :read [23 15] INFO [2022-01-10 12:53:32,012] jepsen worker 34 - jepsen.util 214 :ok :write [11 1] INFO [2022-01-10 12:53:32,013] jepsen worker 71 - jepsen.util 161 :invoke :write [23 31] INFO [2022-01-10 12:53:32,013] jepsen worker 34 - jepsen.util 214 :invoke :write [11 9] INFO [2022-01-10 12:53:32,016] jepsen worker 75 - jepsen.util 165 :ok :write [25 10] INFO [2022-01-10 12:53:32,016] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:32,017] jepsen worker 7 - jepsen.util 7 :ok :write [2 4] INFO [2022-01-10 12:53:32,017] jepsen worker 75 - jepsen.util 165 :ok :read [25 10] INFO [2022-01-10 12:53:32,017] jepsen worker 38 - jepsen.util 38 :ok :write [12 40] INFO [2022-01-10 12:53:32,018] jepsen worker 7 - jepsen.util 7 :invoke :write [2 8] INFO [2022-01-10 12:53:32,018] jepsen worker 38 - jepsen.util 38 :invoke :write [12 25] INFO [2022-01-10 12:53:32,018] jepsen worker 75 - jepsen.util 165 :invoke :write [25 19] INFO [2022-01-10 12:53:32,020] jepsen worker 61 - jepsen.util 61 :ok :write [20 16] INFO [2022-01-10 12:53:32,021] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:32,022] jepsen worker 61 - jepsen.util 61 :ok :read [20 16] INFO [2022-01-10 12:53:32,022] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:32,022] jepsen worker 3 - jepsen.util 3 :ok :write [1 34] INFO [2022-01-10 12:53:32,023] jepsen worker 3 - jepsen.util 3 :invoke :write [1 39] INFO [2022-01-10 12:53:32,023] jepsen worker 61 - jepsen.util 61 :ok :read [20 16] INFO [2022-01-10 12:53:32,024] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:32,024] jepsen worker 22 - jepsen.util 112 :ok :write [7 0] INFO [2022-01-10 12:53:32,024] jepsen worker 61 - jepsen.util 61 :ok :read [20 16] INFO [2022-01-10 12:53:32,025] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:32,025] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:32,025] jepsen worker 22 - jepsen.util 112 :ok :read [7 0] INFO [2022-01-10 12:53:32,026] jepsen worker 61 - jepsen.util 61 :ok :read [20 16] INFO [2022-01-10 12:53:32,026] jepsen worker 22 - jepsen.util 112 :invoke :write [7 29] INFO [2022-01-10 12:53:32,026] jepsen worker 61 - jepsen.util 61 :invoke :write [20 30] INFO [2022-01-10 12:53:32,027] jepsen worker 15 - jepsen.util 15 :ok :write [5 28] INFO [2022-01-10 12:53:32,027] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,028] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:32,029] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,030] jepsen worker 86 - jepsen.util 176 :ok :write [28 34] INFO [2022-01-10 12:53:32,030] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:32,030] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:32,031] jepsen worker 43 - jepsen.util 43 :ok :write [14 35] INFO [2022-01-10 12:53:32,031] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,032] jepsen worker 86 - jepsen.util 176 :ok :read [28 34] INFO [2022-01-10 12:53:32,032] jepsen worker 43 - jepsen.util 43 :invoke :write [14 10] INFO [2022-01-10 12:53:32,032] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:32,032] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:32,033] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,033] jepsen worker 86 - jepsen.util 176 :ok :read [28 34] INFO [2022-01-10 12:53:32,033] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:32,034] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:32,034] jepsen worker 51 - jepsen.util 141 :ok :write [17 44] INFO [2022-01-10 12:53:32,034] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,034] jepsen worker 86 - jepsen.util 176 :ok :read [28 34] INFO [2022-01-10 12:53:32,035] jepsen worker 51 - jepsen.util 141 :invoke :write [17 18] INFO [2022-01-10 12:53:32,035] jepsen worker 86 - jepsen.util 176 :invoke :write [28 45] INFO [2022-01-10 12:53:32,035] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:32,036] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,036] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:32,037] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,038] jepsen worker 88 - jepsen.util 178 :ok :write [29 22] INFO [2022-01-10 12:53:32,038] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:32,039] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,039] jepsen worker 88 - jepsen.util 178 :invoke :write [29 46] INFO [2022-01-10 12:53:32,039] jepsen worker 50 - jepsen.util 50 :ok :write [16 20] INFO [2022-01-10 12:53:32,040] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:32,040] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:32,040] jepsen worker 15 - jepsen.util 15 :invoke :write [5 16] INFO [2022-01-10 12:53:32,041] jepsen worker 80 - jepsen.util 260 :ok :write [26 20] INFO [2022-01-10 12:53:32,042] jepsen worker 50 - jepsen.util 50 :ok :read [16 20] INFO [2022-01-10 12:53:32,042] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:32,042] jepsen worker 50 - jepsen.util 50 :invoke :write [16 12] INFO [2022-01-10 12:53:32,043] jepsen worker 83 - jepsen.util 263 :ok :write [27 37] INFO [2022-01-10 12:53:32,043] jepsen worker 80 - jepsen.util 260 :ok :read [26 20] INFO [2022-01-10 12:53:32,044] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:32,044] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:32,045] jepsen worker 80 - jepsen.util 260 :ok :read [26 20] INFO [2022-01-10 12:53:32,045] jepsen worker 39 - jepsen.util 129 :ok :write [13 2] INFO [2022-01-10 12:53:32,045] jepsen worker 83 - jepsen.util 263 :ok :read [27 37] INFO [2022-01-10 12:53:32,045] jepsen worker 39 - jepsen.util 129 :invoke :write [13 26] INFO [2022-01-10 12:53:32,046] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:32,046] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:32,046] jepsen worker 83 - jepsen.util 263 :ok :read [27 37] INFO [2022-01-10 12:53:32,047] jepsen worker 80 - jepsen.util 260 :ok :read [26 20] INFO [2022-01-10 12:53:32,047] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:32,047] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:32,048] jepsen worker 13 - jepsen.util 13 :ok :write [4 0] INFO [2022-01-10 12:53:32,048] jepsen worker 68 - jepsen.util 68 :ok :write [22 15] INFO [2022-01-10 12:53:32,048] jepsen worker 83 - jepsen.util 263 :ok :read [27 37] INFO [2022-01-10 12:53:32,048] jepsen worker 80 - jepsen.util 260 :ok :read [26 20] INFO [2022-01-10 12:53:32,048] jepsen worker 13 - jepsen.util 13 :invoke :write [4 48] INFO [2022-01-10 12:53:32,049] jepsen worker 68 - jepsen.util 68 :invoke :write [22 5] INFO [2022-01-10 12:53:32,049] jepsen worker 80 - jepsen.util 260 :invoke :write [26 11] INFO [2022-01-10 12:53:32,050] jepsen worker 83 - jepsen.util 263 :invoke :write [27 38] INFO [2022-01-10 12:53:32,053] jepsen worker 2 - jepsen.util 92 :ok :write [0 23] INFO [2022-01-10 12:53:32,053] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,054] jepsen worker 71 - jepsen.util 161 :ok :write [23 31] INFO [2022-01-10 12:53:32,054] jepsen worker 2 - jepsen.util 92 :ok :read [0 23] INFO [2022-01-10 12:53:32,054] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,054] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,055] jepsen worker 71 - jepsen.util 161 :ok :read [23 31] INFO [2022-01-10 12:53:32,055] jepsen worker 2 - jepsen.util 92 :ok :read [0 23] INFO [2022-01-10 12:53:32,056] jepsen worker 71 - jepsen.util 161 :invoke :write [23 30] INFO [2022-01-10 12:53:32,056] jepsen worker 2 - jepsen.util 92 :invoke :write [0 11] INFO [2022-01-10 12:53:32,235] jepsen worker 38 - jepsen.util 38 :ok :write [12 25] INFO [2022-01-10 12:53:32,236] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:32,237] jepsen worker 38 - jepsen.util 38 :ok :read [12 25] INFO [2022-01-10 12:53:32,238] jepsen worker 38 - jepsen.util 38 :invoke :write [12 44] INFO [2022-01-10 12:53:32,247] jepsen worker 3 - jepsen.util 3 :ok :write [1 39] INFO [2022-01-10 12:53:32,248] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,249] jepsen worker 3 - jepsen.util 3 :ok :read [1 39] INFO [2022-01-10 12:53:32,249] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,250] jepsen worker 3 - jepsen.util 3 :ok :read [1 39] INFO [2022-01-10 12:53:32,250] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,251] jepsen worker 3 - jepsen.util 3 :ok :read [1 39] INFO [2022-01-10 12:53:32,252] jepsen worker 3 - jepsen.util 3 :invoke :write [1 14] INFO [2022-01-10 12:53:32,253] jepsen worker 86 - jepsen.util 176 :ok :write [28 45] INFO [2022-01-10 12:53:32,253] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:32,254] jepsen worker 86 - jepsen.util 176 :ok :read [28 45] INFO [2022-01-10 12:53:32,255] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:32,256] jepsen worker 86 - jepsen.util 176 :ok :read [28 45] INFO [2022-01-10 12:53:32,256] jepsen worker 88 - jepsen.util 178 :ok :write [29 46] INFO [2022-01-10 12:53:32,256] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:32,257] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:32,257] jepsen worker 86 - jepsen.util 176 :ok :read [28 45] INFO [2022-01-10 12:53:32,257] jepsen worker 88 - jepsen.util 178 :ok :read [29 46] INFO [2022-01-10 12:53:32,258] jepsen worker 86 - jepsen.util 176 :invoke :write [28 29] INFO [2022-01-10 12:53:32,258] jepsen worker 88 - jepsen.util 178 :invoke :write [29 24] INFO [2022-01-10 12:53:32,262] jepsen worker 39 - jepsen.util 129 :ok :write [13 26] INFO [2022-01-10 12:53:32,263] jepsen worker 39 - jepsen.util 129 :invoke :write [13 15] INFO [2022-01-10 12:53:32,264] jepsen worker 57 - jepsen.util 237 :info :write [19 26] :RequestTimeoutException INFO [2022-01-10 12:53:32,265] jepsen worker 57 - jepsen.util 327 :invoke :write [19 39] INFO [2022-01-10 12:53:32,267] jepsen worker 2 - jepsen.util 92 :ok :write [0 11] INFO [2022-01-10 12:53:32,268] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,269] jepsen worker 2 - jepsen.util 92 :ok :read [0 11] INFO [2022-01-10 12:53:32,269] jepsen worker 2 - jepsen.util 92 :invoke :write [0 20] INFO [2022-01-10 12:53:32,272] jepsen worker 88 - jepsen.util 178 :ok :write [29 24] INFO [2022-01-10 12:53:32,272] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:32,273] jepsen worker 88 - jepsen.util 178 :ok :read [29 24] INFO [2022-01-10 12:53:32,273] jepsen worker 88 - jepsen.util 178 :invoke :write [29 25] INFO [2022-01-10 12:53:32,275] jepsen worker 39 - jepsen.util 129 :ok :write [13 15] INFO [2022-01-10 12:53:32,276] jepsen worker 39 - jepsen.util 129 :invoke :write [13 25] INFO [2022-01-10 12:53:32,280] jepsen worker 2 - jepsen.util 92 :ok :write [0 20] INFO [2022-01-10 12:53:32,281] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,281] jepsen worker 34 - jepsen.util 214 :ok :write [11 9] INFO [2022-01-10 12:53:32,282] jepsen worker 2 - jepsen.util 92 :ok :read [0 20] INFO [2022-01-10 12:53:32,282] jepsen worker 34 - jepsen.util 214 :invoke :write [11 38] INFO [2022-01-10 12:53:32,283] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,283] jepsen worker 38 - jepsen.util 38 :ok :write [12 44] INFO [2022-01-10 12:53:32,283] jepsen worker 2 - jepsen.util 92 :ok :read [0 20] INFO [2022-01-10 12:53:32,284] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:32,284] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,285] jepsen worker 2 - jepsen.util 92 :ok :read [0 20] INFO [2022-01-10 12:53:32,286] jepsen worker 88 - jepsen.util 178 :ok :write [29 25] INFO [2022-01-10 12:53:32,286] jepsen worker 2 - jepsen.util 92 :invoke :write [0 35] INFO [2022-01-10 12:53:32,286] jepsen worker 75 - jepsen.util 165 :ok :write [25 19] INFO [2022-01-10 12:53:32,287] jepsen worker 88 - jepsen.util 178 :invoke :write [29 26] INFO [2022-01-10 12:53:32,287] jepsen worker 75 - jepsen.util 165 :invoke :write [25 43] INFO [2022-01-10 12:53:32,288] jepsen worker 3 - jepsen.util 3 :ok :write [1 14] INFO [2022-01-10 12:53:32,289] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,290] jepsen worker 3 - jepsen.util 3 :ok :read [1 14] INFO [2022-01-10 12:53:32,291] jepsen worker 7 - jepsen.util 7 :ok :write [2 8] INFO [2022-01-10 12:53:32,291] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,291] jepsen worker 3 - jepsen.util 3 :ok :read [1 14] INFO [2022-01-10 12:53:32,292] jepsen worker 7 - jepsen.util 7 :invoke :write [2 33] INFO [2022-01-10 12:53:32,292] jepsen worker 51 - jepsen.util 141 :ok :write [17 18] INFO [2022-01-10 12:53:32,292] jepsen worker 51 - jepsen.util 141 :invoke :write [17 3] INFO [2022-01-10 12:53:32,293] jepsen worker 3 - jepsen.util 3 :invoke :write [1 44] INFO [2022-01-10 12:53:32,294] jepsen worker 86 - jepsen.util 176 :ok :write [28 29] INFO [2022-01-10 12:53:32,295] jepsen worker 86 - jepsen.util 176 :invoke :write [28 20] INFO [2022-01-10 12:53:32,297] jepsen worker 80 - jepsen.util 260 :ok :write [26 11] INFO [2022-01-10 12:53:32,298] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:32,299] jepsen worker 80 - jepsen.util 260 :ok :read [26 11] INFO [2022-01-10 12:53:32,299] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:32,300] jepsen worker 80 - jepsen.util 260 :ok :read [26 11] INFO [2022-01-10 12:53:32,301] jepsen worker 80 - jepsen.util 260 :invoke :write [26 12] INFO [2022-01-10 12:53:32,302] jepsen worker 13 - jepsen.util 13 :ok :write [4 48] INFO [2022-01-10 12:53:32,302] jepsen worker 61 - jepsen.util 61 :ok :write [20 30] INFO [2022-01-10 12:53:32,303] jepsen worker 13 - jepsen.util 13 :invoke :write [4 49] INFO [2022-01-10 12:53:32,303] jepsen worker 61 - jepsen.util 61 :invoke :write [20 34] INFO [2022-01-10 12:53:32,307] jepsen worker 71 - jepsen.util 161 :ok :write [23 30] INFO [2022-01-10 12:53:32,307] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,308] jepsen worker 71 - jepsen.util 161 :ok :read [23 30] INFO [2022-01-10 12:53:32,309] jepsen worker 22 - jepsen.util 112 :ok :write [7 29] INFO [2022-01-10 12:53:32,309] jepsen worker 71 - jepsen.util 161 :invoke :write [23 16] INFO [2022-01-10 12:53:32,309] jepsen worker 22 - jepsen.util 112 :invoke :write [7 33] INFO [2022-01-10 12:53:32,315] jepsen worker 15 - jepsen.util 15 :ok :write [5 16] INFO [2022-01-10 12:53:32,315] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,316] jepsen worker 15 - jepsen.util 15 :ok :read [5 16] INFO [2022-01-10 12:53:32,316] jepsen worker 15 - jepsen.util 15 :invoke :write [5 46] INFO [2022-01-10 12:53:32,318] jepsen worker 43 - jepsen.util 43 :ok :write [14 10] INFO [2022-01-10 12:53:32,318] jepsen worker 43 - jepsen.util 43 :invoke :write [14 39] INFO [2022-01-10 12:53:32,323] jepsen worker 50 - jepsen.util 50 :ok :write [16 12] INFO [2022-01-10 12:53:32,324] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:32,325] jepsen worker 50 - jepsen.util 50 :ok :read [16 12] INFO [2022-01-10 12:53:32,325] jepsen worker 50 - jepsen.util 50 :invoke :write [16 43] INFO [2022-01-10 12:53:32,327] jepsen worker 83 - jepsen.util 263 :ok :write [27 38] INFO [2022-01-10 12:53:32,328] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:32,329] jepsen worker 83 - jepsen.util 263 :ok :read [27 38] INFO [2022-01-10 12:53:32,329] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:32,330] jepsen worker 83 - jepsen.util 263 :ok :read [27 38] INFO [2022-01-10 12:53:32,330] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:32,331] jepsen worker 68 - jepsen.util 68 :ok :write [22 5] INFO [2022-01-10 12:53:32,331] jepsen worker 83 - jepsen.util 263 :ok :read [27 38] INFO [2022-01-10 12:53:32,332] jepsen worker 83 - jepsen.util 263 :invoke :write [27 9] INFO [2022-01-10 12:53:32,333] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:32,334] jepsen worker 68 - jepsen.util 68 :ok :read [22 5] INFO [2022-01-10 12:53:32,334] jepsen worker 34 - jepsen.util 214 :ok :write [11 38] INFO [2022-01-10 12:53:32,335] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:32,335] jepsen worker 34 - jepsen.util 214 :invoke :write [11 25] INFO [2022-01-10 12:53:32,335] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:32,335] jepsen worker 68 - jepsen.util 68 :ok :read [22 5] INFO [2022-01-10 12:53:32,336] jepsen worker 68 - jepsen.util 68 :invoke :write [22 37] INFO [2022-01-10 12:53:32,337] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:32,338] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:32,338] jepsen worker 38 - jepsen.util 38 :invoke :write [12 24] INFO [2022-01-10 12:53:32,340] jepsen worker 39 - jepsen.util 129 :ok :write [13 25] INFO [2022-01-10 12:53:32,341] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:32,341] jepsen worker 7 - jepsen.util 7 :ok :write [2 33] INFO [2022-01-10 12:53:32,342] jepsen worker 39 - jepsen.util 129 :ok :read [13 25] INFO [2022-01-10 12:53:32,342] jepsen worker 3 - jepsen.util 3 :ok :write [1 44] INFO [2022-01-10 12:53:32,342] jepsen worker 7 - jepsen.util 7 :invoke :write [2 30] INFO [2022-01-10 12:53:32,342] jepsen worker 2 - jepsen.util 92 :ok :write [0 35] INFO [2022-01-10 12:53:32,343] jepsen worker 3 - jepsen.util 3 :invoke :write [1 45] INFO [2022-01-10 12:53:32,344] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,344] jepsen worker 39 - jepsen.util 129 :invoke :write [13 22] INFO [2022-01-10 12:53:32,345] jepsen worker 2 - jepsen.util 92 :ok :read [0 35] INFO [2022-01-10 12:53:32,345] jepsen worker 2 - jepsen.util 92 :invoke :write [0 41] INFO [2022-01-10 12:53:32,347] jepsen worker 10 - jepsen.util 190 :info :write [3 18] :RequestTimeoutException INFO [2022-01-10 12:53:32,348] jepsen worker 10 - jepsen.util 280 :invoke :write [3 5] INFO [2022-01-10 12:53:32,350] jepsen worker 51 - jepsen.util 141 :ok :write [17 3] INFO [2022-01-10 12:53:32,350] jepsen worker 88 - jepsen.util 178 :ok :write [29 26] INFO [2022-01-10 12:53:32,350] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:32,351] jepsen worker 88 - jepsen.util 178 :invoke :write [29 35] INFO [2022-01-10 12:53:32,351] jepsen worker 51 - jepsen.util 141 :ok :read [17 3] INFO [2022-01-10 12:53:32,352] jepsen worker 51 - jepsen.util 141 :invoke :write [17 20] INFO [2022-01-10 12:53:32,355] jepsen worker 15 - jepsen.util 15 :ok :write [5 46] INFO [2022-01-10 12:53:32,356] jepsen worker 15 - jepsen.util 15 :invoke :write [5 0] INFO [2022-01-10 12:53:32,356] jepsen worker 75 - jepsen.util 165 :ok :write [25 43] INFO [2022-01-10 12:53:32,357] jepsen worker 86 - jepsen.util 176 :ok :write [28 20] INFO [2022-01-10 12:53:32,357] jepsen worker 75 - jepsen.util 165 :invoke :write [25 37] INFO [2022-01-10 12:53:32,358] jepsen worker 86 - jepsen.util 176 :invoke :write [28 37] INFO [2022-01-10 12:53:32,360] jepsen worker 61 - jepsen.util 61 :ok :write [20 34] INFO [2022-01-10 12:53:32,361] jepsen worker 80 - jepsen.util 260 :ok :write [26 12] INFO [2022-01-10 12:53:32,361] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:32,362] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:32,362] jepsen worker 61 - jepsen.util 61 :ok :read [20 34] INFO [2022-01-10 12:53:32,363] jepsen worker 80 - jepsen.util 260 :ok :read [26 12] INFO [2022-01-10 12:53:32,363] jepsen worker 61 - jepsen.util 61 :invoke :write [20 0] INFO [2022-01-10 12:53:32,363] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:32,364] jepsen worker 50 - jepsen.util 50 :ok :write [16 43] INFO [2022-01-10 12:53:32,364] jepsen worker 80 - jepsen.util 260 :ok :read [26 12] INFO [2022-01-10 12:53:32,365] jepsen worker 50 - jepsen.util 50 :invoke :write [16 37] INFO [2022-01-10 12:53:32,365] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:32,366] jepsen worker 80 - jepsen.util 260 :ok :read [26 12] INFO [2022-01-10 12:53:32,367] jepsen worker 22 - jepsen.util 112 :ok :write [7 33] INFO [2022-01-10 12:53:32,367] jepsen worker 80 - jepsen.util 260 :invoke :write [26 20] INFO [2022-01-10 12:53:32,367] jepsen worker 22 - jepsen.util 112 :invoke :write [7 36] INFO [2022-01-10 12:53:32,369] jepsen worker 13 - jepsen.util 13 :ok :write [4 49] INFO [2022-01-10 12:53:32,370] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:32,370] jepsen worker 83 - jepsen.util 263 :ok :write [27 9] INFO [2022-01-10 12:53:32,370] jepsen worker 83 - jepsen.util 263 :invoke :write [27 36] INFO [2022-01-10 12:53:32,371] jepsen worker 13 - jepsen.util 13 :ok :read [4 49] INFO [2022-01-10 12:53:32,371] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:32,373] jepsen worker 13 - jepsen.util 13 :ok :read [4 49] INFO [2022-01-10 12:53:32,373] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:32,374] jepsen worker 43 - jepsen.util 43 :ok :write [14 39] INFO [2022-01-10 12:53:32,374] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:32,374] jepsen worker 68 - jepsen.util 68 :ok :write [22 37] INFO [2022-01-10 12:53:32,375] jepsen worker 13 - jepsen.util 13 :ok :read [4 49] INFO [2022-01-10 12:53:32,375] jepsen worker 68 - jepsen.util 68 :invoke :write [22 44] INFO [2022-01-10 12:53:32,375] jepsen worker 43 - jepsen.util 43 :ok :read [14 39] INFO [2022-01-10 12:53:32,375] jepsen worker 13 - jepsen.util 13 :invoke :write [4 42] INFO [2022-01-10 12:53:32,376] jepsen worker 43 - jepsen.util 43 :invoke :read [14 nil] INFO [2022-01-10 12:53:32,376] jepsen worker 43 - jepsen.util 43 :ok :read [14 39] INFO [2022-01-10 12:53:32,377] jepsen worker 43 - jepsen.util 43 :invoke :write [14 25] INFO [2022-01-10 12:53:32,379] jepsen worker 71 - jepsen.util 161 :ok :write [23 16] INFO [2022-01-10 12:53:32,379] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,380] jepsen worker 39 - jepsen.util 129 :ok :write [13 22] INFO [2022-01-10 12:53:32,380] jepsen worker 71 - jepsen.util 161 :ok :read [23 16] INFO [2022-01-10 12:53:32,380] jepsen worker 39 - jepsen.util 129 :invoke :write [13 2] INFO [2022-01-10 12:53:32,381] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,381] jepsen worker 71 - jepsen.util 161 :ok :read [23 16] INFO [2022-01-10 12:53:32,382] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,383] jepsen worker 2 - jepsen.util 92 :ok :write [0 41] INFO [2022-01-10 12:53:32,383] jepsen worker 71 - jepsen.util 161 :ok :read [23 16] INFO [2022-01-10 12:53:32,383] jepsen worker 2 - jepsen.util 92 :invoke :write [0 38] INFO [2022-01-10 12:53:32,384] jepsen worker 71 - jepsen.util 161 :invoke :write [23 4] INFO [2022-01-10 12:53:32,389] jepsen worker 88 - jepsen.util 178 :ok :write [29 35] INFO [2022-01-10 12:53:32,390] jepsen worker 38 - jepsen.util 38 :ok :write [12 24] INFO [2022-01-10 12:53:32,390] jepsen worker 88 - jepsen.util 178 :invoke :write [29 6] INFO [2022-01-10 12:53:32,390] jepsen worker 38 - jepsen.util 38 :invoke :write [12 17] INFO [2022-01-10 12:53:32,397] jepsen worker 3 - jepsen.util 3 :ok :write [1 45] INFO [2022-01-10 12:53:32,397] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,398] jepsen worker 3 - jepsen.util 3 :ok :read [1 45] INFO [2022-01-10 12:53:32,398] jepsen worker 3 - jepsen.util 3 :invoke :write [1 24] INFO [2022-01-10 12:53:32,402] jepsen worker 51 - jepsen.util 141 :ok :write [17 20] INFO [2022-01-10 12:53:32,403] jepsen worker 51 - jepsen.util 141 :invoke :write [17 5] INFO [2022-01-10 12:53:32,420] jepsen worker 34 - jepsen.util 214 :ok :write [11 25] INFO [2022-01-10 12:53:32,421] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:32,422] jepsen worker 34 - jepsen.util 214 :ok :read [11 25] INFO [2022-01-10 12:53:32,423] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:32,424] jepsen worker 34 - jepsen.util 214 :ok :read [11 25] INFO [2022-01-10 12:53:32,424] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:32,425] jepsen worker 34 - jepsen.util 214 :ok :read [11 25] INFO [2022-01-10 12:53:32,425] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:32,426] jepsen worker 34 - jepsen.util 214 :ok :read [11 25] INFO [2022-01-10 12:53:32,427] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:32,427] jepsen worker 34 - jepsen.util 214 :ok :read [11 25] INFO [2022-01-10 12:53:32,428] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:32,429] jepsen worker 34 - jepsen.util 214 :ok :read [11 25] INFO [2022-01-10 12:53:32,429] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:32,430] jepsen worker 34 - jepsen.util 214 :ok :read [11 25] INFO [2022-01-10 12:53:32,430] jepsen worker 34 - jepsen.util 214 :invoke :write [11 40] INFO [2022-01-10 12:53:32,464] jepsen worker 66 - jepsen.util 156 :info :write [22 42] :RequestTimeoutException INFO [2022-01-10 12:53:32,465] jepsen worker 66 - jepsen.util 246 :invoke :read [22 nil] INFO [2022-01-10 12:53:32,466] jepsen worker 66 - jepsen.util 246 :ok :read [22 37] INFO [2022-01-10 12:53:32,466] jepsen worker 66 - jepsen.util 246 :invoke :read [22 nil] INFO [2022-01-10 12:53:32,467] jepsen worker 66 - jepsen.util 246 :ok :read [22 37] INFO [2022-01-10 12:53:32,468] jepsen worker 66 - jepsen.util 246 :invoke :write [22 10] INFO [2022-01-10 12:53:32,598] jepsen worker 86 - jepsen.util 176 :ok :write [28 37] INFO [2022-01-10 12:53:32,598] jepsen worker 86 - jepsen.util 176 :invoke :write [28 14] INFO [2022-01-10 12:53:32,602] jepsen worker 7 - jepsen.util 7 :ok :write [2 30] INFO [2022-01-10 12:53:32,602] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:32,602] jepsen worker 80 - jepsen.util 260 :ok :write [26 20] INFO [2022-01-10 12:53:32,603] jepsen worker 80 - jepsen.util 260 :invoke :write [26 49] INFO [2022-01-10 12:53:32,603] jepsen worker 7 - jepsen.util 7 :ok :read [2 30] INFO [2022-01-10 12:53:32,603] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:32,604] jepsen worker 39 - jepsen.util 129 :ok :write [13 2] INFO [2022-01-10 12:53:32,604] jepsen worker 7 - jepsen.util 7 :ok :read [2 30] INFO [2022-01-10 12:53:32,604] jepsen worker 39 - jepsen.util 129 :invoke :write [13 8] INFO [2022-01-10 12:53:32,605] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:32,606] jepsen worker 7 - jepsen.util 7 :ok :read [2 30] INFO [2022-01-10 12:53:32,606] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:32,607] jepsen worker 7 - jepsen.util 7 :ok :read [2 30] INFO [2022-01-10 12:53:32,608] jepsen worker 2 - jepsen.util 92 :ok :write [0 38] INFO [2022-01-10 12:53:32,608] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:32,608] jepsen worker 75 - jepsen.util 165 :ok :write [25 37] INFO [2022-01-10 12:53:32,608] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,609] jepsen worker 13 - jepsen.util 13 :ok :write [4 42] INFO [2022-01-10 12:53:32,609] jepsen worker 75 - jepsen.util 165 :invoke :write [25 21] INFO [2022-01-10 12:53:32,609] jepsen worker 7 - jepsen.util 7 :ok :read [2 30] INFO [2022-01-10 12:53:32,609] jepsen worker 2 - jepsen.util 92 :ok :read [0 38] INFO [2022-01-10 12:53:32,609] jepsen worker 13 - jepsen.util 13 :invoke :write [4 20] INFO [2022-01-10 12:53:32,610] jepsen worker 7 - jepsen.util 7 :invoke :write [2 12] INFO [2022-01-10 12:53:32,610] jepsen worker 2 - jepsen.util 92 :invoke :write [0 43] INFO [2022-01-10 12:53:32,614] jepsen worker 15 - jepsen.util 15 :ok :write [5 0] INFO [2022-01-10 12:53:32,615] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,615] jepsen worker 88 - jepsen.util 178 :ok :write [29 6] INFO [2022-01-10 12:53:32,615] jepsen worker 71 - jepsen.util 161 :ok :write [23 4] INFO [2022-01-10 12:53:32,615] jepsen worker 88 - jepsen.util 178 :invoke :write [29 24] INFO [2022-01-10 12:53:32,615] jepsen worker 15 - jepsen.util 15 :ok :read [5 0] INFO [2022-01-10 12:53:32,616] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,616] jepsen worker 15 - jepsen.util 15 :invoke :write [5 42] INFO [2022-01-10 12:53:32,617] jepsen worker 71 - jepsen.util 161 :ok :read [23 4] INFO [2022-01-10 12:53:32,618] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,619] jepsen worker 71 - jepsen.util 161 :ok :read [23 4] INFO [2022-01-10 12:53:32,619] jepsen worker 61 - jepsen.util 61 :ok :write [20 0] INFO [2022-01-10 12:53:32,619] jepsen worker 71 - jepsen.util 161 :invoke :write [23 42] INFO [2022-01-10 12:53:32,620] jepsen worker 61 - jepsen.util 61 :invoke :write [20 7] INFO [2022-01-10 12:53:32,624] jepsen worker 38 - jepsen.util 38 :ok :write [12 17] INFO [2022-01-10 12:53:32,624] jepsen worker 39 - jepsen.util 129 :ok :write [13 8] INFO [2022-01-10 12:53:32,624] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:32,625] jepsen worker 39 - jepsen.util 129 :invoke :write [13 5] INFO [2022-01-10 12:53:32,625] jepsen worker 38 - jepsen.util 38 :ok :read [12 17] INFO [2022-01-10 12:53:32,626] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:32,626] jepsen worker 50 - jepsen.util 50 :ok :write [16 37] INFO [2022-01-10 12:53:32,627] jepsen worker 38 - jepsen.util 38 :ok :read [12 17] INFO [2022-01-10 12:53:32,627] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:32,628] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:32,628] jepsen worker 50 - jepsen.util 50 :ok :read [16 37] INFO [2022-01-10 12:53:32,629] jepsen worker 38 - jepsen.util 38 :ok :read [12 17] INFO [2022-01-10 12:53:32,629] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:32,629] jepsen worker 2 - jepsen.util 92 :ok :write [0 43] INFO [2022-01-10 12:53:32,629] jepsen worker 38 - jepsen.util 38 :invoke :write [12 38] INFO [2022-01-10 12:53:32,630] jepsen worker 3 - jepsen.util 3 :ok :write [1 24] INFO [2022-01-10 12:53:32,630] jepsen worker 50 - jepsen.util 50 :ok :read [16 37] INFO [2022-01-10 12:53:32,630] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,630] jepsen worker 22 - jepsen.util 112 :ok :write [7 36] INFO [2022-01-10 12:53:32,631] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,631] jepsen worker 2 - jepsen.util 92 :ok :read [0 43] INFO [2022-01-10 12:53:32,631] jepsen worker 50 - jepsen.util 50 :invoke :write [16 46] INFO [2022-01-10 12:53:32,632] jepsen worker 3 - jepsen.util 3 :ok :read [1 24] INFO [2022-01-10 12:53:32,632] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:32,633] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,633] jepsen worker 22 - jepsen.util 112 :ok :read [7 36] INFO [2022-01-10 12:53:32,633] jepsen worker 2 - jepsen.util 92 :invoke :write [0 24] INFO [2022-01-10 12:53:32,634] jepsen worker 3 - jepsen.util 3 :ok :read [1 24] INFO [2022-01-10 12:53:32,634] jepsen worker 22 - jepsen.util 112 :invoke :write [7 32] INFO [2022-01-10 12:53:32,634] jepsen worker 3 - jepsen.util 3 :invoke :write [1 43] INFO [2022-01-10 12:53:32,637] jepsen worker 51 - jepsen.util 141 :ok :write [17 5] INFO [2022-01-10 12:53:32,637] jepsen worker 83 - jepsen.util 263 :ok :write [27 36] INFO [2022-01-10 12:53:32,637] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:32,638] jepsen worker 83 - jepsen.util 263 :invoke :write [27 41] INFO [2022-01-10 12:53:32,638] jepsen worker 51 - jepsen.util 141 :ok :read [17 5] INFO [2022-01-10 12:53:32,639] jepsen worker 51 - jepsen.util 141 :invoke :write [17 18] INFO [2022-01-10 12:53:32,641] jepsen worker 43 - jepsen.util 43 :ok :write [14 25] INFO [2022-01-10 12:53:32,642] jepsen worker 88 - jepsen.util 178 :ok :write [29 24] INFO [2022-01-10 12:53:32,642] jepsen worker 43 - jepsen.util 43 :invoke :write [14 47] INFO [2022-01-10 12:53:32,643] jepsen worker 88 - jepsen.util 178 :invoke :write [29 40] INFO [2022-01-10 12:53:32,646] jepsen worker 86 - jepsen.util 176 :ok :write [28 14] INFO [2022-01-10 12:53:32,646] jepsen worker 86 - jepsen.util 176 :invoke :write [28 34] INFO [2022-01-10 12:53:32,652] jepsen worker 68 - jepsen.util 68 :ok :write [22 44] INFO [2022-01-10 12:53:32,653] jepsen worker 68 - jepsen.util 68 :invoke :write [22 8] INFO [2022-01-10 12:53:32,654] jepsen worker 34 - jepsen.util 214 :ok :write [11 40] INFO [2022-01-10 12:53:32,655] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:32,656] jepsen worker 34 - jepsen.util 214 :ok :read [11 40] INFO [2022-01-10 12:53:32,656] jepsen worker 34 - jepsen.util 214 :invoke :write [11 12] INFO [2022-01-10 12:53:32,658] jepsen worker 55 - jepsen.util 235 :info :write [18 18] :RequestTimeoutException INFO [2022-01-10 12:53:32,659] jepsen worker 55 - jepsen.util 325 :invoke :write [18 38] INFO [2022-01-10 12:53:32,660] jepsen worker 7 - jepsen.util 7 :ok :write [2 12] INFO [2022-01-10 12:53:32,661] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:32,662] jepsen worker 7 - jepsen.util 7 :ok :read [2 12] INFO [2022-01-10 12:53:32,662] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:32,663] jepsen worker 7 - jepsen.util 7 :ok :read [2 12] INFO [2022-01-10 12:53:32,663] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:32,664] jepsen worker 7 - jepsen.util 7 :ok :read [2 12] INFO [2022-01-10 12:53:32,664] jepsen worker 75 - jepsen.util 165 :ok :write [25 21] INFO [2022-01-10 12:53:32,665] jepsen worker 7 - jepsen.util 7 :invoke :write [2 28] INFO [2022-01-10 12:53:32,665] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:32,666] jepsen worker 75 - jepsen.util 165 :ok :read [25 21] INFO [2022-01-10 12:53:32,666] jepsen worker 75 - jepsen.util 165 :invoke :write [25 43] INFO [2022-01-10 12:53:32,670] jepsen worker 15 - jepsen.util 15 :ok :write [5 42] INFO [2022-01-10 12:53:32,670] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,671] jepsen worker 15 - jepsen.util 15 :ok :read [5 42] INFO [2022-01-10 12:53:32,672] jepsen worker 15 - jepsen.util 15 :invoke :write [5 2] INFO [2022-01-10 12:53:32,672] jepsen worker 61 - jepsen.util 61 :ok :write [20 7] INFO [2022-01-10 12:53:32,673] jepsen worker 61 - jepsen.util 61 :invoke :write [20 29] INFO [2022-01-10 12:53:32,674] jepsen worker 80 - jepsen.util 260 :ok :write [26 49] INFO [2022-01-10 12:53:32,674] jepsen worker 80 - jepsen.util 260 :invoke :write [26 49] INFO [2022-01-10 12:53:32,677] jepsen worker 50 - jepsen.util 50 :ok :write [16 46] INFO [2022-01-10 12:53:32,677] jepsen worker 50 - jepsen.util 50 :invoke :write [16 39] INFO [2022-01-10 12:53:32,679] jepsen worker 39 - jepsen.util 129 :ok :write [13 5] INFO [2022-01-10 12:53:32,679] jepsen worker 39 - jepsen.util 129 :invoke :write [13 39] INFO [2022-01-10 12:53:32,683] jepsen worker 22 - jepsen.util 112 :ok :write [7 32] INFO [2022-01-10 12:53:32,683] jepsen worker 13 - jepsen.util 13 :ok :write [4 20] INFO [2022-01-10 12:53:32,683] jepsen worker 22 - jepsen.util 112 :invoke :write [7 14] INFO [2022-01-10 12:53:32,684] jepsen worker 13 - jepsen.util 13 :invoke :write [4 41] INFO [2022-01-10 12:53:32,686] jepsen worker 71 - jepsen.util 161 :ok :write [23 42] INFO [2022-01-10 12:53:32,686] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,686] jepsen worker 83 - jepsen.util 263 :ok :write [27 41] INFO [2022-01-10 12:53:32,687] jepsen worker 71 - jepsen.util 161 :ok :read [23 42] INFO [2022-01-10 12:53:32,687] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:32,688] jepsen worker 71 - jepsen.util 161 :invoke :write [23 9] INFO [2022-01-10 12:53:32,688] jepsen worker 83 - jepsen.util 263 :ok :read [27 41] INFO [2022-01-10 12:53:32,689] jepsen worker 83 - jepsen.util 263 :invoke :write [27 48] INFO [2022-01-10 12:53:32,691] jepsen worker 2 - jepsen.util 92 :ok :write [0 24] INFO [2022-01-10 12:53:32,691] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,692] jepsen worker 2 - jepsen.util 92 :ok :read [0 24] INFO [2022-01-10 12:53:32,693] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,693] jepsen worker 38 - jepsen.util 38 :ok :write [12 38] INFO [2022-01-10 12:53:32,693] jepsen worker 2 - jepsen.util 92 :ok :read [0 24] INFO [2022-01-10 12:53:32,694] jepsen worker 88 - jepsen.util 178 :ok :write [29 40] INFO [2022-01-10 12:53:32,694] jepsen worker 38 - jepsen.util 38 :invoke :write [12 18] INFO [2022-01-10 12:53:32,695] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:32,695] jepsen worker 2 - jepsen.util 92 :invoke :write [0 19] INFO [2022-01-10 12:53:32,696] jepsen worker 88 - jepsen.util 178 :ok :read [29 40] INFO [2022-01-10 12:53:32,696] jepsen worker 3 - jepsen.util 3 :ok :write [1 43] INFO [2022-01-10 12:53:32,696] jepsen worker 88 - jepsen.util 178 :invoke :write [29 27] INFO [2022-01-10 12:53:32,697] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,698] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:32,699] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,699] jepsen worker 68 - jepsen.util 68 :ok :write [22 8] INFO [2022-01-10 12:53:32,700] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:32,700] jepsen worker 44 - jepsen.util 44 :ok :write [14 38] INFO [2022-01-10 12:53:32,700] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:32,700] jepsen worker 44 - jepsen.util 44 :invoke :write [14 42] INFO [2022-01-10 12:53:32,701] jepsen worker 3 - jepsen.util 3 :invoke :write [1 16] INFO [2022-01-10 12:53:32,701] jepsen worker 68 - jepsen.util 68 :ok :read [22 8] INFO [2022-01-10 12:53:32,702] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:32,703] jepsen worker 68 - jepsen.util 68 :ok :read [22 8] INFO [2022-01-10 12:53:32,703] jepsen worker 68 - jepsen.util 68 :invoke :write [22 26] INFO [2022-01-10 12:53:32,704] jepsen worker 51 - jepsen.util 141 :ok :write [17 18] INFO [2022-01-10 12:53:32,704] jepsen worker 51 - jepsen.util 141 :invoke :write [17 5] INFO [2022-01-10 12:53:32,707] jepsen worker 34 - jepsen.util 214 :ok :write [11 12] INFO [2022-01-10 12:53:32,708] jepsen worker 34 - jepsen.util 214 :invoke :write [11 3] INFO [2022-01-10 12:53:32,710] jepsen worker 86 - jepsen.util 176 :ok :write [28 34] INFO [2022-01-10 12:53:32,710] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:32,711] jepsen worker 86 - jepsen.util 176 :ok :read [28 34] INFO [2022-01-10 12:53:32,712] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:32,713] jepsen worker 80 - jepsen.util 260 :ok :write [26 49] INFO [2022-01-10 12:53:32,713] jepsen worker 39 - jepsen.util 129 :ok :write [13 39] INFO [2022-01-10 12:53:32,713] jepsen worker 86 - jepsen.util 176 :ok :read [28 34] INFO [2022-01-10 12:53:32,713] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:32,713] jepsen worker 7 - jepsen.util 7 :ok :write [2 28] INFO [2022-01-10 12:53:32,714] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:32,714] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:32,715] jepsen worker 80 - jepsen.util 260 :ok :read [26 49] INFO [2022-01-10 12:53:32,715] jepsen worker 39 - jepsen.util 129 :invoke :write [13 29] INFO [2022-01-10 12:53:32,715] jepsen worker 86 - jepsen.util 176 :ok :read [28 34] INFO [2022-01-10 12:53:32,715] jepsen worker 7 - jepsen.util 7 :ok :read [2 28] INFO [2022-01-10 12:53:32,715] jepsen worker 80 - jepsen.util 260 :invoke :write [26 16] INFO [2022-01-10 12:53:32,716] jepsen worker 86 - jepsen.util 176 :invoke :write [28 43] INFO [2022-01-10 12:53:32,716] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:32,717] jepsen worker 7 - jepsen.util 7 :ok :read [2 28] INFO [2022-01-10 12:53:32,717] jepsen worker 7 - jepsen.util 7 :invoke :write [2 29] INFO [2022-01-10 12:53:32,719] jepsen worker 75 - jepsen.util 165 :ok :write [25 43] INFO [2022-01-10 12:53:32,720] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:32,720] jepsen worker 75 - jepsen.util 165 :ok :read [25 43] INFO [2022-01-10 12:53:32,721] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:32,721] jepsen worker 13 - jepsen.util 13 :ok :write [4 41] INFO [2022-01-10 12:53:32,722] jepsen worker 75 - jepsen.util 165 :ok :read [25 43] INFO [2022-01-10 12:53:32,722] jepsen worker 13 - jepsen.util 13 :invoke :write [4 27] INFO [2022-01-10 12:53:32,722] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:32,723] jepsen worker 75 - jepsen.util 165 :ok :read [25 43] INFO [2022-01-10 12:53:32,724] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:32,725] jepsen worker 75 - jepsen.util 165 :ok :read [25 43] INFO [2022-01-10 12:53:32,725] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:32,726] jepsen worker 15 - jepsen.util 15 :ok :write [5 2] INFO [2022-01-10 12:53:32,726] jepsen worker 2 - jepsen.util 92 :ok :write [0 19] INFO [2022-01-10 12:53:32,726] jepsen worker 71 - jepsen.util 161 :ok :write [23 9] INFO [2022-01-10 12:53:32,726] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,726] jepsen worker 75 - jepsen.util 165 :ok :read [25 43] INFO [2022-01-10 12:53:32,727] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:32,727] jepsen worker 15 - jepsen.util 15 :ok :read [5 2] INFO [2022-01-10 12:53:32,727] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,728] jepsen worker 75 - jepsen.util 165 :ok :read [25 43] INFO [2022-01-10 12:53:32,728] jepsen worker 2 - jepsen.util 92 :invoke :write [0 1] INFO [2022-01-10 12:53:32,728] jepsen worker 71 - jepsen.util 161 :ok :read [23 9] INFO [2022-01-10 12:53:32,729] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,729] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,729] jepsen worker 88 - jepsen.util 178 :ok :write [29 27] INFO [2022-01-10 12:53:32,729] jepsen worker 15 - jepsen.util 15 :ok :read [5 2] INFO [2022-01-10 12:53:32,730] jepsen worker 75 - jepsen.util 165 :invoke :write [25 28] INFO [2022-01-10 12:53:32,730] jepsen worker 71 - jepsen.util 161 :ok :read [23 9] INFO [2022-01-10 12:53:32,730] jepsen worker 61 - jepsen.util 61 :ok :write [20 29] INFO [2022-01-10 12:53:32,730] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,731] jepsen worker 88 - jepsen.util 178 :invoke :write [29 9] INFO [2022-01-10 12:53:32,731] jepsen worker 15 - jepsen.util 15 :ok :read [5 2] INFO [2022-01-10 12:53:32,732] jepsen worker 61 - jepsen.util 61 :invoke :write [20 47] INFO [2022-01-10 12:53:32,732] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,733] jepsen worker 71 - jepsen.util 161 :invoke :write [23 23] INFO [2022-01-10 12:53:32,733] jepsen worker 15 - jepsen.util 15 :ok :read [5 2] INFO [2022-01-10 12:53:32,734] jepsen worker 15 - jepsen.util 15 :invoke :write [5 4] INFO [2022-01-10 12:53:32,734] jepsen worker 38 - jepsen.util 38 :ok :write [12 18] INFO [2022-01-10 12:53:32,735] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:32,739] jepsen worker 3 - jepsen.util 3 :ok :write [1 16] INFO [2022-01-10 12:53:32,739] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,740] jepsen worker 39 - jepsen.util 129 :ok :write [13 29] INFO [2022-01-10 12:53:32,740] jepsen worker 3 - jepsen.util 3 :ok :read [1 16] INFO [2022-01-10 12:53:32,740] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:32,741] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,742] jepsen worker 39 - jepsen.util 129 :ok :read [13 29] INFO [2022-01-10 12:53:32,742] jepsen worker 3 - jepsen.util 3 :ok :read [1 16] INFO [2022-01-10 12:53:32,742] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:32,743] jepsen worker 3 - jepsen.util 3 :invoke :write [1 43] INFO [2022-01-10 12:53:32,743] jepsen worker 39 - jepsen.util 129 :ok :read [13 29] INFO [2022-01-10 12:53:32,744] jepsen worker 39 - jepsen.util 129 :invoke :write [13 19] INFO [2022-01-10 12:53:32,745] jepsen worker 51 - jepsen.util 141 :ok :write [17 5] INFO [2022-01-10 12:53:32,746] jepsen worker 51 - jepsen.util 141 :invoke :write [17 22] INFO [2022-01-10 12:53:32,751] jepsen worker 86 - jepsen.util 176 :ok :write [28 43] INFO [2022-01-10 12:53:32,752] jepsen worker 86 - jepsen.util 176 :invoke :write [28 5] INFO [2022-01-10 12:53:32,754] jepsen worker 2 - jepsen.util 92 :ok :write [0 1] INFO [2022-01-10 12:53:32,755] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,755] jepsen worker 80 - jepsen.util 260 :ok :write [26 16] INFO [2022-01-10 12:53:32,756] jepsen worker 2 - jepsen.util 92 :ok :read [0 1] INFO [2022-01-10 12:53:32,756] jepsen worker 80 - jepsen.util 260 :invoke :write [26 35] INFO [2022-01-10 12:53:32,756] jepsen worker 2 - jepsen.util 92 :invoke :write [0 41] INFO [2022-01-10 12:53:32,761] jepsen worker 88 - jepsen.util 178 :ok :write [29 9] INFO [2022-01-10 12:53:32,761] jepsen worker 88 - jepsen.util 178 :invoke :write [29 4] INFO [2022-01-10 12:53:32,765] jepsen worker 39 - jepsen.util 129 :ok :write [13 19] INFO [2022-01-10 12:53:32,765] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:32,766] jepsen worker 39 - jepsen.util 129 :ok :read [13 19] INFO [2022-01-10 12:53:32,767] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:32,768] jepsen worker 39 - jepsen.util 129 :ok :read [13 19] INFO [2022-01-10 12:53:32,768] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:32,769] jepsen worker 39 - jepsen.util 129 :ok :read [13 19] INFO [2022-01-10 12:53:32,769] jepsen worker 2 - jepsen.util 92 :ok :write [0 41] INFO [2022-01-10 12:53:32,770] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:32,770] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,770] jepsen worker 39 - jepsen.util 129 :ok :read [13 19] INFO [2022-01-10 12:53:32,771] jepsen worker 2 - jepsen.util 92 :ok :read [0 41] INFO [2022-01-10 12:53:32,771] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:32,771] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,772] jepsen worker 39 - jepsen.util 129 :ok :read [13 19] INFO [2022-01-10 12:53:32,772] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:32,772] jepsen worker 2 - jepsen.util 92 :ok :read [0 41] INFO [2022-01-10 12:53:32,772] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:32,773] jepsen worker 39 - jepsen.util 129 :ok :read [13 19] INFO [2022-01-10 12:53:32,773] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,774] jepsen worker 38 - jepsen.util 38 :invoke :write [12 5] INFO [2022-01-10 12:53:32,774] jepsen worker 2 - jepsen.util 92 :ok :read [0 41] INFO [2022-01-10 12:53:32,774] jepsen worker 39 - jepsen.util 129 :invoke :write [13 26] INFO [2022-01-10 12:53:32,775] jepsen worker 2 - jepsen.util 92 :invoke :write [0 48] INFO [2022-01-10 12:53:32,777] jepsen worker 3 - jepsen.util 3 :ok :write [1 43] INFO [2022-01-10 12:53:32,777] jepsen worker 11 - jepsen.util 281 :fail :read [3 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:32,778] jepsen worker 88 - jepsen.util 178 :ok :write [29 4] INFO [2022-01-10 12:53:32,779] jepsen worker 11 - jepsen.util 281 :invoke :write [3 26] INFO [2022-01-10 12:53:32,779] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:32,779] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,780] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:32,780] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:32,780] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:32,781] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,781] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:32,781] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:32,782] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:32,782] jepsen worker 3 - jepsen.util 3 :invoke :write [1 24] INFO [2022-01-10 12:53:32,782] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:32,783] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:32,784] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:32,784] jepsen worker 50 - jepsen.util 50 :ok :write [16 39] INFO [2022-01-10 12:53:32,785] jepsen worker 50 - jepsen.util 50 :invoke :write [16 6] INFO [2022-01-10 12:53:32,785] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:32,785] jepsen worker 13 - jepsen.util 13 :ok :write [4 27] INFO [2022-01-10 12:53:32,786] jepsen worker 88 - jepsen.util 178 :ok :read [29 4] INFO [2022-01-10 12:53:32,786] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:32,787] jepsen worker 30 - jepsen.util 300 :fail :read [10 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:32,788] jepsen worker 88 - jepsen.util 178 :invoke :write [29 30] INFO [2022-01-10 12:53:32,788] jepsen worker 13 - jepsen.util 13 :ok :read [4 27] INFO [2022-01-10 12:53:32,788] jepsen worker 30 - jepsen.util 300 :invoke :read [10 nil] INFO [2022-01-10 12:53:32,789] jepsen worker 86 - jepsen.util 176 :ok :write [28 5] INFO [2022-01-10 12:53:32,790] jepsen worker 86 - jepsen.util 176 :invoke :write [28 35] INFO [2022-01-10 12:53:32,791] jepsen worker 22 - jepsen.util 112 :ok :write [7 14] INFO [2022-01-10 12:53:32,791] jepsen worker 13 - jepsen.util 13 :invoke :write [4 14] INFO [2022-01-10 12:53:32,792] jepsen worker 22 - jepsen.util 112 :invoke :write [7 24] INFO [2022-01-10 12:53:32,792] jepsen worker 71 - jepsen.util 161 :ok :write [23 23] INFO [2022-01-10 12:53:32,793] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,794] jepsen worker 71 - jepsen.util 161 :ok :read [23 23] INFO [2022-01-10 12:53:32,794] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:32,795] jepsen worker 39 - jepsen.util 129 :ok :write [13 26] INFO [2022-01-10 12:53:32,795] jepsen worker 71 - jepsen.util 161 :ok :read [23 23] INFO [2022-01-10 12:53:32,795] jepsen worker 39 - jepsen.util 129 :invoke :write [13 19] INFO [2022-01-10 12:53:32,796] jepsen worker 71 - jepsen.util 161 :invoke :write [23 14] INFO [2022-01-10 12:53:32,797] jepsen worker 83 - jepsen.util 263 :ok :write [27 48] INFO [2022-01-10 12:53:32,798] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:32,799] jepsen worker 83 - jepsen.util 263 :ok :read [27 48] INFO [2022-01-10 12:53:32,799] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:32,801] jepsen worker 83 - jepsen.util 263 :ok :read [27 48] INFO [2022-01-10 12:53:32,801] jepsen worker 88 - jepsen.util 178 :ok :write [29 30] INFO [2022-01-10 12:53:32,802] jepsen worker 83 - jepsen.util 263 :invoke :write [27 26] INFO [2022-01-10 12:53:32,802] jepsen worker 44 - jepsen.util 44 :ok :write [14 42] INFO [2022-01-10 12:53:32,802] jepsen worker 2 - jepsen.util 92 :ok :write [0 48] INFO [2022-01-10 12:53:32,802] jepsen worker 88 - jepsen.util 178 :invoke :write [29 8] INFO [2022-01-10 12:53:32,803] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:32,804] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:32,804] jepsen worker 44 - jepsen.util 44 :ok :read [14 42] INFO [2022-01-10 12:53:32,804] jepsen worker 2 - jepsen.util 92 :ok :read [0 48] INFO [2022-01-10 12:53:32,805] jepsen worker 44 - jepsen.util 44 :invoke :write [14 41] INFO [2022-01-10 12:53:32,805] jepsen worker 3 - jepsen.util 3 :ok :write [1 24] INFO [2022-01-10 12:53:32,805] jepsen worker 2 - jepsen.util 92 :invoke :write [0 12] INFO [2022-01-10 12:53:32,806] jepsen worker 3 - jepsen.util 3 :invoke :write [1 25] INFO [2022-01-10 12:53:32,807] jepsen worker 38 - jepsen.util 38 :ok :write [12 5] INFO [2022-01-10 12:53:32,808] jepsen worker 51 - jepsen.util 141 :ok :write [17 22] INFO [2022-01-10 12:53:32,809] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:32,809] jepsen worker 51 - jepsen.util 141 :invoke :write [17 11] INFO [2022-01-10 12:53:32,810] jepsen worker 38 - jepsen.util 38 :ok :read [12 5] INFO [2022-01-10 12:53:32,811] jepsen worker 80 - jepsen.util 260 :ok :write [26 35] INFO [2022-01-10 12:53:32,811] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:32,812] jepsen worker 80 - jepsen.util 260 :invoke :write [26 10] INFO [2022-01-10 12:53:32,813] jepsen worker 38 - jepsen.util 38 :ok :read [12 5] INFO [2022-01-10 12:53:32,814] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:32,815] jepsen worker 38 - jepsen.util 38 :ok :read [12 5] INFO [2022-01-10 12:53:32,815] jepsen worker 39 - jepsen.util 129 :ok :write [13 19] INFO [2022-01-10 12:53:32,815] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:32,816] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:32,817] jepsen worker 38 - jepsen.util 38 :ok :read [12 5] INFO [2022-01-10 12:53:32,818] jepsen worker 39 - jepsen.util 129 :ok :read [13 19] INFO [2022-01-10 12:53:32,818] jepsen worker 38 - jepsen.util 38 :invoke :write [12 19] INFO [2022-01-10 12:53:32,818] jepsen worker 39 - jepsen.util 129 :invoke :write [13 19] INFO [2022-01-10 12:53:32,828] jepsen worker 86 - jepsen.util 176 :ok :write [28 35] INFO [2022-01-10 12:53:32,829] jepsen worker 86 - jepsen.util 176 :invoke :write [28 9] INFO [2022-01-10 12:53:32,832] jepsen worker 2 - jepsen.util 92 :ok :write [0 12] INFO [2022-01-10 12:53:32,832] jepsen worker 2 - jepsen.util 92 :invoke :write [0 33] INFO [2022-01-10 12:53:32,836] jepsen worker 3 - jepsen.util 3 :ok :write [1 25] INFO [2022-01-10 12:53:32,836] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,837] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:32,837] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:32,838] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:32,838] jepsen worker 3 - jepsen.util 3 :invoke :write [1 37] INFO [2022-01-10 12:53:32,902] jepsen worker 31 - jepsen.util 211 :info :write [10 2] :RequestTimeoutException INFO [2022-01-10 12:53:32,903] jepsen worker 31 - jepsen.util 301 :invoke :write [10 43] INFO [2022-01-10 12:53:32,917] jepsen worker 88 - jepsen.util 178 :ok :write [29 8] INFO [2022-01-10 12:53:32,917] jepsen worker 68 - jepsen.util 68 :ok :write [22 26] INFO [2022-01-10 12:53:32,918] jepsen worker 88 - jepsen.util 178 :invoke :write [29 48] INFO [2022-01-10 12:53:32,918] jepsen worker 68 - jepsen.util 68 :invoke :write [22 26] INFO [2022-01-10 12:53:32,920] jepsen worker 34 - jepsen.util 214 :ok :write [11 3] INFO [2022-01-10 12:53:32,921] jepsen worker 34 - jepsen.util 214 :invoke :write [11 49] INFO [2022-01-10 12:53:32,921] jepsen worker 39 - jepsen.util 129 :ok :write [13 19] INFO [2022-01-10 12:53:32,922] jepsen worker 38 - jepsen.util 38 :ok :write [12 19] INFO [2022-01-10 12:53:32,922] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:32,922] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:32,923] jepsen worker 39 - jepsen.util 129 :ok :read [13 19] INFO [2022-01-10 12:53:32,923] jepsen worker 38 - jepsen.util 38 :ok :read [12 19] INFO [2022-01-10 12:53:32,924] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:32,924] jepsen worker 13 - jepsen.util 13 :ok :write [4 14] INFO [2022-01-10 12:53:32,924] jepsen worker 39 - jepsen.util 129 :ok :read [13 19] INFO [2022-01-10 12:53:32,925] jepsen worker 38 - jepsen.util 38 :invoke :write [12 10] INFO [2022-01-10 12:53:32,925] jepsen worker 13 - jepsen.util 13 :invoke :write [4 15] INFO [2022-01-10 12:53:32,926] jepsen worker 7 - jepsen.util 7 :ok :write [2 29] INFO [2022-01-10 12:53:32,926] jepsen worker 39 - jepsen.util 129 :invoke :write [13 9] INFO [2022-01-10 12:53:32,927] jepsen worker 7 - jepsen.util 7 :invoke :write [2 43] INFO [2022-01-10 12:53:32,929] jepsen worker 71 - jepsen.util 161 :ok :write [23 14] INFO [2022-01-10 12:53:32,929] jepsen worker 71 - jepsen.util 161 :invoke :write [23 1] INFO [2022-01-10 12:53:32,929] jepsen worker 75 - jepsen.util 165 :ok :write [25 28] INFO [2022-01-10 12:53:32,930] jepsen worker 75 - jepsen.util 165 :invoke :write [25 33] INFO [2022-01-10 12:53:32,930] jepsen worker 2 - jepsen.util 92 :ok :write [0 33] INFO [2022-01-10 12:53:32,931] jepsen worker 2 - jepsen.util 92 :invoke :write [0 35] INFO [2022-01-10 12:53:32,934] jepsen worker 15 - jepsen.util 15 :ok :write [5 4] INFO [2022-01-10 12:53:32,935] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,936] jepsen worker 15 - jepsen.util 15 :ok :read [5 4] INFO [2022-01-10 12:53:32,936] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:32,937] jepsen worker 15 - jepsen.util 15 :ok :read [5 4] INFO [2022-01-10 12:53:32,937] jepsen worker 15 - jepsen.util 15 :invoke :write [5 45] INFO [2022-01-10 12:53:33,009] jepsen worker 28 - jepsen.util 298 :fail :read [9 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:33,010] jepsen worker 28 - jepsen.util 298 :invoke :write [9 37] INFO [2022-01-10 12:53:33,012] jepsen worker 51 - jepsen.util 141 :ok :write [17 11] INFO [2022-01-10 12:53:33,014] jepsen worker 51 - jepsen.util 141 :invoke :write [17 22] INFO [2022-01-10 12:53:33,017] jepsen worker 61 - jepsen.util 61 :ok :write [20 47] INFO [2022-01-10 12:53:33,018] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:33,019] jepsen worker 61 - jepsen.util 61 :ok :read [20 47] INFO [2022-01-10 12:53:33,019] jepsen worker 61 - jepsen.util 61 :invoke :write [20 37] INFO [2022-01-10 12:53:33,024] jepsen worker 80 - jepsen.util 260 :ok :write [26 10] INFO [2022-01-10 12:53:33,024] jepsen worker 88 - jepsen.util 178 :ok :write [29 48] INFO [2022-01-10 12:53:33,024] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,025] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:33,026] jepsen worker 88 - jepsen.util 178 :ok :read [29 48] INFO [2022-01-10 12:53:33,026] jepsen worker 80 - jepsen.util 260 :ok :read [26 10] INFO [2022-01-10 12:53:33,026] jepsen worker 88 - jepsen.util 178 :invoke :write [29 5] INFO [2022-01-10 12:53:33,027] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:33,028] jepsen worker 80 - jepsen.util 260 :ok :read [26 10] INFO [2022-01-10 12:53:33,028] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:33,029] jepsen worker 80 - jepsen.util 260 :ok :read [26 10] INFO [2022-01-10 12:53:33,030] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:33,030] jepsen worker 86 - jepsen.util 176 :ok :write [28 9] INFO [2022-01-10 12:53:33,031] jepsen worker 80 - jepsen.util 260 :ok :read [26 10] INFO [2022-01-10 12:53:33,031] jepsen worker 39 - jepsen.util 129 :ok :write [13 9] INFO [2022-01-10 12:53:33,031] jepsen worker 86 - jepsen.util 176 :invoke :write [28 17] INFO [2022-01-10 12:53:33,032] jepsen worker 50 - jepsen.util 50 :ok :write [16 6] INFO [2022-01-10 12:53:33,032] jepsen worker 39 - jepsen.util 129 :invoke :write [13 49] INFO [2022-01-10 12:53:33,033] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:33,033] jepsen worker 80 - jepsen.util 260 :invoke :write [26 33] INFO [2022-01-10 12:53:33,034] jepsen worker 50 - jepsen.util 50 :ok :read [16 6] INFO [2022-01-10 12:53:33,035] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:33,036] jepsen worker 50 - jepsen.util 50 :ok :read [16 6] INFO [2022-01-10 12:53:33,036] jepsen worker 50 - jepsen.util 50 :invoke :write [16 12] INFO [2022-01-10 12:53:33,037] jepsen worker 22 - jepsen.util 112 :ok :write [7 24] INFO [2022-01-10 12:53:33,038] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:33,038] jepsen worker 3 - jepsen.util 3 :ok :write [1 37] INFO [2022-01-10 12:53:33,039] jepsen worker 22 - jepsen.util 112 :ok :read [7 24] INFO [2022-01-10 12:53:33,039] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,039] jepsen worker 22 - jepsen.util 112 :invoke :write [7 25] INFO [2022-01-10 12:53:33,040] jepsen worker 3 - jepsen.util 3 :ok :read [1 37] INFO [2022-01-10 12:53:33,040] jepsen worker 3 - jepsen.util 3 :invoke :write [1 26] INFO [2022-01-10 12:53:33,044] jepsen worker 83 - jepsen.util 263 :ok :write [27 26] INFO [2022-01-10 12:53:33,045] jepsen worker 83 - jepsen.util 263 :invoke :write [27 23] INFO [2022-01-10 12:53:33,045] jepsen worker 38 - jepsen.util 38 :ok :write [12 10] INFO [2022-01-10 12:53:33,045] jepsen worker 2 - jepsen.util 92 :ok :write [0 35] INFO [2022-01-10 12:53:33,046] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,046] jepsen worker 38 - jepsen.util 38 :invoke :write [12 48] INFO [2022-01-10 12:53:33,047] jepsen worker 2 - jepsen.util 92 :ok :read [0 35] INFO [2022-01-10 12:53:33,047] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,048] jepsen worker 2 - jepsen.util 92 :ok :read [0 35] INFO [2022-01-10 12:53:33,049] jepsen worker 2 - jepsen.util 92 :invoke :write [0 4] INFO [2022-01-10 12:53:33,049] jepsen worker 44 - jepsen.util 44 :ok :write [14 41] INFO [2022-01-10 12:53:33,050] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:33,051] jepsen worker 44 - jepsen.util 44 :ok :read [14 41] INFO [2022-01-10 12:53:33,051] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:33,052] jepsen worker 44 - jepsen.util 44 :ok :read [14 41] INFO [2022-01-10 12:53:33,052] jepsen worker 13 - jepsen.util 13 :ok :write [4 15] INFO [2022-01-10 12:53:33,053] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:33,053] jepsen worker 13 - jepsen.util 13 :invoke :write [4 34] INFO [2022-01-10 12:53:33,053] jepsen worker 44 - jepsen.util 44 :ok :read [14 41] INFO [2022-01-10 12:53:33,054] jepsen worker 44 - jepsen.util 44 :invoke :write [14 3] INFO [2022-01-10 12:53:33,057] jepsen worker 71 - jepsen.util 161 :ok :write [23 1] INFO [2022-01-10 12:53:33,057] jepsen worker 88 - jepsen.util 178 :ok :write [29 5] INFO [2022-01-10 12:53:33,058] jepsen worker 88 - jepsen.util 178 :invoke :write [29 40] INFO [2022-01-10 12:53:33,059] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,060] jepsen worker 71 - jepsen.util 161 :ok :read [23 1] INFO [2022-01-10 12:53:33,060] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,061] jepsen worker 71 - jepsen.util 161 :ok :read [23 1] INFO [2022-01-10 12:53:33,062] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,062] jepsen worker 71 - jepsen.util 161 :ok :read [23 1] INFO [2022-01-10 12:53:33,063] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,064] jepsen worker 71 - jepsen.util 161 :ok :read [23 1] INFO [2022-01-10 12:53:33,065] jepsen worker 39 - jepsen.util 129 :ok :write [13 49] INFO [2022-01-10 12:53:33,065] jepsen worker 71 - jepsen.util 161 :invoke :write [23 14] INFO [2022-01-10 12:53:33,065] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,067] jepsen worker 51 - jepsen.util 141 :ok :write [17 22] INFO [2022-01-10 12:53:33,067] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:33,067] jepsen worker 39 - jepsen.util 129 :ok :read [13 49] INFO [2022-01-10 12:53:33,068] jepsen worker 51 - jepsen.util 141 :ok :read [17 22] INFO [2022-01-10 12:53:33,068] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,069] jepsen worker 51 - jepsen.util 141 :invoke :write [17 16] INFO [2022-01-10 12:53:33,069] jepsen worker 39 - jepsen.util 129 :ok :read [13 49] INFO [2022-01-10 12:53:33,069] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,070] jepsen worker 39 - jepsen.util 129 :ok :read [13 49] INFO [2022-01-10 12:53:33,071] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,072] jepsen worker 39 - jepsen.util 129 :ok :read [13 49] INFO [2022-01-10 12:53:33,072] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,073] jepsen worker 39 - jepsen.util 129 :ok :read [13 49] INFO [2022-01-10 12:53:33,074] jepsen worker 39 - jepsen.util 129 :invoke :write [13 21] INFO [2022-01-10 12:53:33,085] jepsen worker 80 - jepsen.util 260 :ok :write [26 33] INFO [2022-01-10 12:53:33,085] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:33,086] jepsen worker 80 - jepsen.util 260 :ok :read [26 33] INFO [2022-01-10 12:53:33,087] jepsen worker 80 - jepsen.util 260 :invoke :write [26 45] INFO [2022-01-10 12:53:33,087] jepsen worker 2 - jepsen.util 92 :ok :write [0 4] INFO [2022-01-10 12:53:33,088] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,089] jepsen worker 2 - jepsen.util 92 :ok :read [0 4] INFO [2022-01-10 12:53:33,089] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,090] jepsen worker 2 - jepsen.util 92 :ok :read [0 4] INFO [2022-01-10 12:53:33,091] jepsen worker 2 - jepsen.util 92 :invoke :write [0 24] INFO [2022-01-10 12:53:33,093] jepsen worker 86 - jepsen.util 176 :ok :write [28 17] INFO [2022-01-10 12:53:33,094] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,095] jepsen worker 86 - jepsen.util 176 :ok :read [28 17] INFO [2022-01-10 12:53:33,095] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,096] jepsen worker 86 - jepsen.util 176 :ok :read [28 17] INFO [2022-01-10 12:53:33,096] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,097] jepsen worker 86 - jepsen.util 176 :ok :read [28 17] INFO [2022-01-10 12:53:33,098] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,099] jepsen worker 86 - jepsen.util 176 :ok :read [28 17] INFO [2022-01-10 12:53:33,099] jepsen worker 86 - jepsen.util 176 :invoke :write [28 39] INFO [2022-01-10 12:53:33,103] jepsen worker 3 - jepsen.util 3 :ok :write [1 26] INFO [2022-01-10 12:53:33,103] jepsen worker 88 - jepsen.util 178 :ok :write [29 40] INFO [2022-01-10 12:53:33,104] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,104] jepsen worker 3 - jepsen.util 3 :invoke :write [1 26] INFO [2022-01-10 12:53:33,105] jepsen worker 88 - jepsen.util 178 :ok :read [29 40] INFO [2022-01-10 12:53:33,105] jepsen worker 88 - jepsen.util 178 :invoke :write [29 48] INFO [2022-01-10 12:53:33,109] jepsen worker 38 - jepsen.util 38 :ok :write [12 48] INFO [2022-01-10 12:53:33,110] jepsen worker 38 - jepsen.util 38 :invoke :write [12 30] INFO [2022-01-10 12:53:33,112] jepsen worker 39 - jepsen.util 129 :ok :write [13 21] INFO [2022-01-10 12:53:33,113] jepsen worker 39 - jepsen.util 129 :invoke :write [13 30] INFO [2022-01-10 12:53:33,117] jepsen worker 2 - jepsen.util 92 :ok :write [0 24] INFO [2022-01-10 12:53:33,118] jepsen worker 2 - jepsen.util 92 :invoke :write [0 12] INFO [2022-01-10 12:53:33,123] jepsen worker 88 - jepsen.util 178 :ok :write [29 48] INFO [2022-01-10 12:53:33,123] jepsen worker 88 - jepsen.util 178 :invoke :write [29 48] INFO [2022-01-10 12:53:33,128] jepsen worker 39 - jepsen.util 129 :ok :write [13 30] INFO [2022-01-10 12:53:33,129] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,129] jepsen worker 39 - jepsen.util 129 :ok :read [13 30] INFO [2022-01-10 12:53:33,130] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,131] jepsen worker 39 - jepsen.util 129 :ok :read [13 30] INFO [2022-01-10 12:53:33,131] jepsen worker 39 - jepsen.util 129 :invoke :write [13 14] INFO [2022-01-10 12:53:33,134] jepsen worker 2 - jepsen.util 92 :ok :write [0 12] INFO [2022-01-10 12:53:33,134] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,135] jepsen worker 2 - jepsen.util 92 :ok :read [0 12] INFO [2022-01-10 12:53:33,136] jepsen worker 2 - jepsen.util 92 :invoke :write [0 10] INFO [2022-01-10 12:53:33,139] jepsen worker 86 - jepsen.util 176 :ok :write [28 39] INFO [2022-01-10 12:53:33,140] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,140] jepsen worker 88 - jepsen.util 178 :ok :write [29 48] INFO [2022-01-10 12:53:33,141] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,141] jepsen worker 86 - jepsen.util 176 :ok :read [28 39] INFO [2022-01-10 12:53:33,141] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,142] jepsen worker 88 - jepsen.util 178 :ok :read [29 48] INFO [2022-01-10 12:53:33,142] jepsen worker 86 - jepsen.util 176 :ok :read [28 39] INFO [2022-01-10 12:53:33,142] jepsen worker 88 - jepsen.util 178 :invoke :write [29 18] INFO [2022-01-10 12:53:33,143] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,144] jepsen worker 86 - jepsen.util 176 :ok :read [28 39] INFO [2022-01-10 12:53:33,144] jepsen worker 86 - jepsen.util 176 :invoke :write [28 25] INFO [2022-01-10 12:53:33,145] jepsen worker 3 - jepsen.util 3 :ok :write [1 26] INFO [2022-01-10 12:53:33,146] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,146] jepsen worker 68 - jepsen.util 68 :ok :write [22 26] INFO [2022-01-10 12:53:33,147] jepsen worker 68 - jepsen.util 68 :invoke :write [22 48] INFO [2022-01-10 12:53:33,147] jepsen worker 3 - jepsen.util 3 :ok :read [1 26] INFO [2022-01-10 12:53:33,147] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,148] jepsen worker 39 - jepsen.util 129 :ok :write [13 14] INFO [2022-01-10 12:53:33,148] jepsen worker 3 - jepsen.util 3 :ok :read [1 26] INFO [2022-01-10 12:53:33,148] jepsen worker 39 - jepsen.util 129 :invoke :write [13 9] INFO [2022-01-10 12:53:33,149] jepsen worker 3 - jepsen.util 3 :invoke :write [1 42] INFO [2022-01-10 12:53:33,151] jepsen worker 34 - jepsen.util 214 :ok :write [11 49] INFO [2022-01-10 12:53:33,152] jepsen worker 34 - jepsen.util 214 :invoke :write [11 9] INFO [2022-01-10 12:53:33,153] jepsen worker 38 - jepsen.util 38 :ok :write [12 30] INFO [2022-01-10 12:53:33,153] jepsen worker 2 - jepsen.util 92 :ok :write [0 10] INFO [2022-01-10 12:53:33,154] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:33,154] jepsen worker 2 - jepsen.util 92 :invoke :write [0 16] INFO [2022-01-10 12:53:33,155] jepsen worker 38 - jepsen.util 38 :ok :read [12 30] INFO [2022-01-10 12:53:33,156] jepsen worker 13 - jepsen.util 13 :ok :write [4 34] INFO [2022-01-10 12:53:33,156] jepsen worker 38 - jepsen.util 38 :invoke :write [12 35] INFO [2022-01-10 12:53:33,157] jepsen worker 13 - jepsen.util 13 :invoke :write [4 30] INFO [2022-01-10 12:53:33,161] jepsen worker 7 - jepsen.util 7 :ok :write [2 43] INFO [2022-01-10 12:53:33,161] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:33,162] jepsen worker 71 - jepsen.util 161 :ok :write [23 14] INFO [2022-01-10 12:53:33,162] jepsen worker 7 - jepsen.util 7 :ok :read [2 43] INFO [2022-01-10 12:53:33,162] jepsen worker 86 - jepsen.util 176 :ok :write [28 25] INFO [2022-01-10 12:53:33,162] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,163] jepsen worker 88 - jepsen.util 178 :ok :write [29 18] INFO [2022-01-10 12:53:33,163] jepsen worker 71 - jepsen.util 161 :ok :read [23 14] INFO [2022-01-10 12:53:33,164] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,164] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:33,165] jepsen worker 86 - jepsen.util 176 :ok :read [28 25] INFO [2022-01-10 12:53:33,165] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,165] jepsen worker 7 - jepsen.util 7 :ok :read [2 43] INFO [2022-01-10 12:53:33,165] jepsen worker 71 - jepsen.util 161 :invoke :write [23 33] INFO [2022-01-10 12:53:33,166] jepsen worker 88 - jepsen.util 178 :ok :read [29 18] INFO [2022-01-10 12:53:33,167] jepsen worker 86 - jepsen.util 176 :invoke :write [28 10] INFO [2022-01-10 12:53:33,167] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:33,168] jepsen worker 75 - jepsen.util 165 :ok :write [25 33] INFO [2022-01-10 12:53:33,168] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,168] jepsen worker 7 - jepsen.util 7 :ok :read [2 43] INFO [2022-01-10 12:53:33,168] jepsen worker 3 - jepsen.util 3 :ok :write [1 42] INFO [2022-01-10 12:53:33,169] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:33,169] jepsen worker 88 - jepsen.util 178 :ok :read [29 18] INFO [2022-01-10 12:53:33,169] jepsen worker 3 - jepsen.util 3 :invoke :write [1 38] INFO [2022-01-10 12:53:33,170] jepsen worker 7 - jepsen.util 7 :ok :read [2 43] INFO [2022-01-10 12:53:33,170] jepsen worker 88 - jepsen.util 178 :invoke :write [29 33] INFO [2022-01-10 12:53:33,171] jepsen worker 75 - jepsen.util 165 :invoke :write [25 15] INFO [2022-01-10 12:53:33,171] jepsen worker 7 - jepsen.util 7 :invoke :write [2 9] INFO [2022-01-10 12:53:33,172] jepsen worker 51 - jepsen.util 141 :ok :write [17 16] INFO [2022-01-10 12:53:33,173] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:33,174] jepsen worker 51 - jepsen.util 141 :ok :read [17 16] INFO [2022-01-10 12:53:33,174] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:33,175] jepsen worker 51 - jepsen.util 141 :ok :read [17 16] INFO [2022-01-10 12:53:33,176] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:33,176] jepsen worker 38 - jepsen.util 38 :ok :write [12 35] INFO [2022-01-10 12:53:33,177] jepsen worker 51 - jepsen.util 141 :ok :read [17 16] INFO [2022-01-10 12:53:33,178] jepsen worker 64 - jepsen.util 334 :fail :read [21 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:33,178] jepsen worker 15 - jepsen.util 15 :ok :write [5 45] INFO [2022-01-10 12:53:33,178] jepsen worker 39 - jepsen.util 129 :ok :write [13 9] INFO [2022-01-10 12:53:33,179] jepsen worker 38 - jepsen.util 38 :invoke :write [12 30] INFO [2022-01-10 12:53:33,180] jepsen worker 80 - jepsen.util 260 :ok :write [26 45] INFO [2022-01-10 12:53:33,180] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:33,181] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:33,181] jepsen worker 15 - jepsen.util 15 :ok :read [5 45] INFO [2022-01-10 12:53:33,182] jepsen worker 80 - jepsen.util 260 :invoke :write [26 39] INFO [2022-01-10 12:53:33,182] jepsen worker 51 - jepsen.util 141 :ok :read [17 16] INFO [2022-01-10 12:53:33,182] jepsen worker 15 - jepsen.util 15 :invoke :write [5 22] INFO [2022-01-10 12:53:33,183] jepsen worker 64 - jepsen.util 334 :invoke :read [21 nil] INFO [2022-01-10 12:53:33,184] jepsen worker 51 - jepsen.util 141 :invoke :write [17 29] INFO [2022-01-10 12:53:33,184] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,185] jepsen worker 39 - jepsen.util 129 :ok :read [13 9] INFO [2022-01-10 12:53:33,185] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,186] jepsen worker 2 - jepsen.util 92 :ok :write [0 16] INFO [2022-01-10 12:53:33,186] jepsen worker 39 - jepsen.util 129 :ok :read [13 9] INFO [2022-01-10 12:53:33,187] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,188] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,188] jepsen worker 39 - jepsen.util 129 :ok :read [13 9] INFO [2022-01-10 12:53:33,189] jepsen worker 61 - jepsen.util 61 :ok :write [20 37] INFO [2022-01-10 12:53:33,189] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,189] jepsen worker 2 - jepsen.util 92 :ok :read [0 16] INFO [2022-01-10 12:53:33,190] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:33,190] jepsen worker 39 - jepsen.util 129 :ok :read [13 9] INFO [2022-01-10 12:53:33,191] jepsen worker 61 - jepsen.util 61 :ok :read [20 37] INFO [2022-01-10 12:53:33,191] jepsen worker 39 - jepsen.util 129 :invoke :write [13 41] INFO [2022-01-10 12:53:33,191] jepsen worker 13 - jepsen.util 13 :ok :write [4 30] INFO [2022-01-10 12:53:33,192] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:33,192] jepsen worker 13 - jepsen.util 13 :invoke :write [4 3] INFO [2022-01-10 12:53:33,193] jepsen worker 61 - jepsen.util 61 :ok :read [20 37] INFO [2022-01-10 12:53:33,193] jepsen worker 2 - jepsen.util 92 :invoke :write [0 2] INFO [2022-01-10 12:53:33,193] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:33,194] jepsen worker 61 - jepsen.util 61 :ok :read [20 37] INFO [2022-01-10 12:53:33,195] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:33,196] jepsen worker 50 - jepsen.util 50 :ok :write [16 12] INFO [2022-01-10 12:53:33,196] jepsen worker 61 - jepsen.util 61 :ok :read [20 37] INFO [2022-01-10 12:53:33,196] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:33,196] jepsen worker 71 - jepsen.util 161 :ok :write [23 33] INFO [2022-01-10 12:53:33,197] jepsen worker 50 - jepsen.util 50 :ok :read [16 12] INFO [2022-01-10 12:53:33,197] jepsen worker 86 - jepsen.util 176 :ok :write [28 10] INFO [2022-01-10 12:53:33,197] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:33,198] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,198] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:33,198] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,198] jepsen worker 61 - jepsen.util 61 :ok :read [20 37] INFO [2022-01-10 12:53:33,199] jepsen worker 61 - jepsen.util 61 :invoke :write [20 35] INFO [2022-01-10 12:53:33,199] jepsen worker 50 - jepsen.util 50 :ok :read [16 12] INFO [2022-01-10 12:53:33,199] jepsen worker 71 - jepsen.util 161 :ok :read [23 33] INFO [2022-01-10 12:53:33,199] jepsen worker 86 - jepsen.util 176 :ok :read [28 10] INFO [2022-01-10 12:53:33,200] jepsen worker 50 - jepsen.util 50 :invoke :write [16 21] INFO [2022-01-10 12:53:33,200] jepsen worker 86 - jepsen.util 176 :invoke :write [28 29] INFO [2022-01-10 12:53:33,201] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,201] jepsen worker 22 - jepsen.util 112 :ok :write [7 25] INFO [2022-01-10 12:53:33,201] jepsen worker 71 - jepsen.util 161 :ok :read [23 33] INFO [2022-01-10 12:53:33,202] jepsen worker 22 - jepsen.util 112 :invoke :write [7 15] INFO [2022-01-10 12:53:33,202] jepsen worker 3 - jepsen.util 3 :ok :write [1 38] INFO [2022-01-10 12:53:33,202] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,203] jepsen worker 3 - jepsen.util 3 :invoke :write [1 46] INFO [2022-01-10 12:53:33,203] jepsen worker 71 - jepsen.util 161 :ok :read [23 33] INFO [2022-01-10 12:53:33,204] jepsen worker 71 - jepsen.util 161 :invoke :write [23 26] INFO [2022-01-10 12:53:33,208] jepsen worker 51 - jepsen.util 141 :ok :write [17 29] INFO [2022-01-10 12:53:33,209] jepsen worker 51 - jepsen.util 141 :invoke :write [17 36] INFO [2022-01-10 12:53:33,211] jepsen worker 83 - jepsen.util 263 :ok :write [27 23] INFO [2022-01-10 12:53:33,211] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:33,212] jepsen worker 88 - jepsen.util 178 :ok :write [29 33] INFO [2022-01-10 12:53:33,212] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,212] jepsen worker 83 - jepsen.util 263 :ok :read [27 23] INFO [2022-01-10 12:53:33,213] jepsen worker 83 - jepsen.util 263 :invoke :write [27 49] INFO [2022-01-10 12:53:33,213] jepsen worker 88 - jepsen.util 178 :ok :read [29 33] INFO [2022-01-10 12:53:33,214] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,214] jepsen worker 39 - jepsen.util 129 :ok :write [13 41] INFO [2022-01-10 12:53:33,214] jepsen worker 88 - jepsen.util 178 :ok :read [29 33] INFO [2022-01-10 12:53:33,215] jepsen worker 88 - jepsen.util 178 :invoke :write [29 28] INFO [2022-01-10 12:53:33,215] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,216] jepsen worker 38 - jepsen.util 38 :ok :write [12 30] INFO [2022-01-10 12:53:33,216] jepsen worker 39 - jepsen.util 129 :ok :read [13 41] INFO [2022-01-10 12:53:33,217] jepsen worker 44 - jepsen.util 44 :ok :write [14 3] INFO [2022-01-10 12:53:33,217] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,217] jepsen worker 38 - jepsen.util 38 :invoke :write [12 43] INFO [2022-01-10 12:53:33,218] jepsen worker 39 - jepsen.util 129 :ok :read [13 41] INFO [2022-01-10 12:53:33,218] jepsen worker 44 - jepsen.util 44 :invoke :write [14 33] INFO [2022-01-10 12:53:33,218] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,219] jepsen worker 39 - jepsen.util 129 :ok :read [13 41] INFO [2022-01-10 12:53:33,220] jepsen worker 39 - jepsen.util 129 :invoke :write [13 42] INFO [2022-01-10 12:53:33,221] jepsen worker 80 - jepsen.util 260 :ok :write [26 39] INFO [2022-01-10 12:53:33,222] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:33,223] jepsen worker 80 - jepsen.util 260 :ok :read [26 39] INFO [2022-01-10 12:53:33,223] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:33,225] jepsen worker 68 - jepsen.util 68 :ok :write [22 48] INFO [2022-01-10 12:53:33,225] jepsen worker 80 - jepsen.util 260 :ok :read [26 39] INFO [2022-01-10 12:53:33,225] jepsen worker 2 - jepsen.util 92 :ok :write [0 2] INFO [2022-01-10 12:53:33,225] jepsen worker 68 - jepsen.util 68 :invoke :write [22 39] INFO [2022-01-10 12:53:33,226] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:33,226] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,227] jepsen worker 80 - jepsen.util 260 :ok :read [26 39] INFO [2022-01-10 12:53:33,227] jepsen worker 80 - jepsen.util 260 :invoke :write [26 34] INFO [2022-01-10 12:53:33,227] jepsen worker 2 - jepsen.util 92 :ok :read [0 2] INFO [2022-01-10 12:53:33,228] jepsen worker 2 - jepsen.util 92 :invoke :write [0 35] INFO [2022-01-10 12:53:33,229] jepsen worker 13 - jepsen.util 13 :ok :write [4 3] INFO [2022-01-10 12:53:33,229] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:33,230] jepsen worker 13 - jepsen.util 13 :ok :read [4 3] INFO [2022-01-10 12:53:33,231] jepsen worker 13 - jepsen.util 13 :invoke :write [4 42] INFO [2022-01-10 12:53:33,232] jepsen worker 34 - jepsen.util 214 :ok :write [11 9] INFO [2022-01-10 12:53:33,233] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:33,234] jepsen worker 34 - jepsen.util 214 :ok :read [11 9] INFO [2022-01-10 12:53:33,234] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:33,235] jepsen worker 34 - jepsen.util 214 :ok :read [11 9] INFO [2022-01-10 12:53:33,236] jepsen worker 71 - jepsen.util 161 :ok :write [23 26] INFO [2022-01-10 12:53:33,236] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:33,237] jepsen worker 71 - jepsen.util 161 :invoke :write [23 40] INFO [2022-01-10 12:53:33,237] jepsen worker 34 - jepsen.util 214 :ok :read [11 9] INFO [2022-01-10 12:53:33,238] jepsen worker 34 - jepsen.util 214 :invoke :write [11 27] INFO [2022-01-10 12:53:33,240] jepsen worker 7 - jepsen.util 7 :ok :write [2 9] INFO [2022-01-10 12:53:33,240] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:33,241] jepsen worker 7 - jepsen.util 7 :ok :read [2 9] INFO [2022-01-10 12:53:33,241] jepsen worker 7 - jepsen.util 7 :invoke :write [2 7] INFO [2022-01-10 12:53:33,242] jepsen worker 88 - jepsen.util 178 :ok :write [29 28] INFO [2022-01-10 12:53:33,243] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,244] jepsen worker 88 - jepsen.util 178 :ok :read [29 28] INFO [2022-01-10 12:53:33,244] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,245] jepsen worker 88 - jepsen.util 178 :ok :read [29 28] INFO [2022-01-10 12:53:33,246] jepsen worker 88 - jepsen.util 178 :invoke :write [29 20] INFO [2022-01-10 12:53:33,246] jepsen worker 39 - jepsen.util 129 :ok :write [13 42] INFO [2022-01-10 12:53:33,246] jepsen worker 86 - jepsen.util 176 :ok :write [28 29] INFO [2022-01-10 12:53:33,247] jepsen worker 39 - jepsen.util 129 :invoke :write [13 38] INFO [2022-01-10 12:53:33,247] jepsen worker 75 - jepsen.util 165 :ok :write [25 15] INFO [2022-01-10 12:53:33,248] jepsen worker 75 - jepsen.util 165 :invoke :write [25 36] INFO [2022-01-10 12:53:33,249] jepsen worker 86 - jepsen.util 176 :invoke :write [28 15] INFO [2022-01-10 12:53:33,252] jepsen worker 3 - jepsen.util 3 :ok :write [1 46] INFO [2022-01-10 12:53:33,252] jepsen worker 3 - jepsen.util 3 :invoke :write [1 11] INFO [2022-01-10 12:53:33,255] jepsen worker 15 - jepsen.util 15 :ok :write [5 22] INFO [2022-01-10 12:53:33,256] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:33,257] jepsen worker 15 - jepsen.util 15 :ok :read [5 22] INFO [2022-01-10 12:53:33,258] jepsen worker 51 - jepsen.util 141 :ok :write [17 36] INFO [2022-01-10 12:53:33,258] jepsen worker 15 - jepsen.util 15 :invoke :write [5 7] INFO [2022-01-10 12:53:33,258] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:33,259] jepsen worker 51 - jepsen.util 141 :ok :read [17 36] INFO [2022-01-10 12:53:33,260] jepsen worker 61 - jepsen.util 61 :ok :write [20 35] INFO [2022-01-10 12:53:33,260] jepsen worker 51 - jepsen.util 141 :invoke :write [17 43] INFO [2022-01-10 12:53:33,261] jepsen worker 61 - jepsen.util 61 :invoke :write [20 20] INFO [2022-01-10 12:53:33,263] jepsen worker 1 - jepsen.util 91 :ok :write [0 0] INFO [2022-01-10 12:53:33,264] jepsen worker 1 - jepsen.util 91 :invoke :write [0 15] INFO [2022-01-10 12:53:33,265] jepsen worker 38 - jepsen.util 38 :ok :write [12 43] INFO [2022-01-10 12:53:33,266] jepsen worker 50 - jepsen.util 50 :ok :write [16 21] INFO [2022-01-10 12:53:33,267] jepsen worker 38 - jepsen.util 38 :invoke :write [12 29] INFO [2022-01-10 12:53:33,267] jepsen worker 50 - jepsen.util 50 :invoke :write [16 2] INFO [2022-01-10 12:53:33,271] jepsen worker 22 - jepsen.util 112 :ok :write [7 15] INFO [2022-01-10 12:53:33,272] jepsen worker 22 - jepsen.util 112 :invoke :write [7 40] INFO [2022-01-10 12:53:33,274] jepsen worker 80 - jepsen.util 260 :ok :write [26 34] INFO [2022-01-10 12:53:33,275] jepsen worker 80 - jepsen.util 260 :invoke :write [26 32] INFO [2022-01-10 12:53:33,278] jepsen worker 83 - jepsen.util 263 :ok :write [27 49] INFO [2022-01-10 12:53:33,279] jepsen worker 83 - jepsen.util 263 :invoke :write [27 13] INFO [2022-01-10 12:53:33,279] jepsen worker 13 - jepsen.util 13 :ok :write [4 42] INFO [2022-01-10 12:53:33,280] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:33,281] jepsen worker 13 - jepsen.util 13 :ok :read [4 42] INFO [2022-01-10 12:53:33,281] jepsen worker 13 - jepsen.util 13 :invoke :write [4 1] INFO [2022-01-10 12:53:33,284] jepsen worker 44 - jepsen.util 44 :ok :write [14 33] INFO [2022-01-10 12:53:33,284] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:33,285] jepsen worker 44 - jepsen.util 44 :ok :read [14 33] INFO [2022-01-10 12:53:33,286] jepsen worker 44 - jepsen.util 44 :invoke :write [14 16] INFO [2022-01-10 12:53:33,290] jepsen worker 68 - jepsen.util 68 :ok :write [22 39] INFO [2022-01-10 12:53:33,291] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:33,292] jepsen worker 68 - jepsen.util 68 :ok :read [22 39] INFO [2022-01-10 12:53:33,292] jepsen worker 68 - jepsen.util 68 :invoke :write [22 32] INFO [2022-01-10 12:53:33,403] jepsen worker 71 - jepsen.util 161 :ok :write [23 40] INFO [2022-01-10 12:53:33,404] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,405] jepsen worker 34 - jepsen.util 214 :ok :write [11 27] INFO [2022-01-10 12:53:33,405] jepsen worker 71 - jepsen.util 161 :ok :read [23 40] INFO [2022-01-10 12:53:33,405] jepsen worker 34 - jepsen.util 214 :invoke :write [11 10] INFO [2022-01-10 12:53:33,406] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,407] jepsen worker 71 - jepsen.util 161 :ok :read [23 40] INFO [2022-01-10 12:53:33,407] jepsen worker 71 - jepsen.util 161 :invoke :write [23 1] INFO [2022-01-10 12:53:33,410] jepsen worker 7 - jepsen.util 7 :ok :write [2 7] INFO [2022-01-10 12:53:33,411] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:33,411] jepsen worker 86 - jepsen.util 176 :ok :write [28 15] INFO [2022-01-10 12:53:33,412] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,413] jepsen worker 86 - jepsen.util 176 :ok :read [28 15] INFO [2022-01-10 12:53:33,414] jepsen worker 7 - jepsen.util 7 :ok :read [2 7] INFO [2022-01-10 12:53:33,414] jepsen worker 86 - jepsen.util 176 :invoke :write [28 32] INFO [2022-01-10 12:53:33,415] jepsen worker 88 - jepsen.util 178 :ok :write [29 20] INFO [2022-01-10 12:53:33,415] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:33,416] jepsen worker 7 - jepsen.util 7 :ok :read [2 7] INFO [2022-01-10 12:53:33,416] jepsen worker 88 - jepsen.util 178 :invoke :write [29 5] INFO [2022-01-10 12:53:33,417] jepsen worker 3 - jepsen.util 3 :ok :write [1 11] INFO [2022-01-10 12:53:33,417] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:33,418] jepsen worker 7 - jepsen.util 7 :ok :read [2 7] INFO [2022-01-10 12:53:33,418] jepsen worker 3 - jepsen.util 3 :invoke :write [1 13] INFO [2022-01-10 12:53:33,419] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:33,419] jepsen worker 39 - jepsen.util 129 :ok :write [13 38] INFO [2022-01-10 12:53:33,420] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,420] jepsen worker 7 - jepsen.util 7 :ok :read [2 7] INFO [2022-01-10 12:53:33,420] jepsen worker 7 - jepsen.util 7 :invoke :write [2 36] INFO [2022-01-10 12:53:33,424] jepsen worker 51 - jepsen.util 141 :ok :write [17 43] INFO [2022-01-10 12:53:33,425] jepsen worker 51 - jepsen.util 141 :invoke :write [17 25] INFO [2022-01-10 12:53:33,429] jepsen worker 1 - jepsen.util 91 :ok :write [0 15] INFO [2022-01-10 12:53:33,429] jepsen worker 38 - jepsen.util 38 :ok :write [12 29] INFO [2022-01-10 12:53:33,430] jepsen worker 38 - jepsen.util 38 :invoke :write [12 38] INFO [2022-01-10 12:53:33,430] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,431] jepsen worker 1 - jepsen.util 91 :ok :read [0 15] INFO [2022-01-10 12:53:33,431] jepsen worker 1 - jepsen.util 91 :invoke :write [0 4] INFO [2022-01-10 12:53:33,437] jepsen worker 80 - jepsen.util 260 :ok :write [26 32] INFO [2022-01-10 12:53:33,438] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:33,439] jepsen worker 80 - jepsen.util 260 :ok :read [26 32] INFO [2022-01-10 12:53:33,439] jepsen worker 80 - jepsen.util 260 :invoke :write [26 12] INFO [2022-01-10 12:53:33,443] jepsen worker 13 - jepsen.util 13 :ok :write [4 1] INFO [2022-01-10 12:53:33,443] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:33,444] jepsen worker 13 - jepsen.util 13 :ok :read [4 1] INFO [2022-01-10 12:53:33,445] jepsen worker 13 - jepsen.util 13 :invoke :write [4 6] INFO [2022-01-10 12:53:33,453] jepsen worker 86 - jepsen.util 176 :ok :write [28 32] INFO [2022-01-10 12:53:33,454] jepsen worker 86 - jepsen.util 176 :invoke :write [28 4] INFO [2022-01-10 12:53:33,456] jepsen worker 3 - jepsen.util 3 :ok :write [1 13] INFO [2022-01-10 12:53:33,457] jepsen worker 88 - jepsen.util 178 :ok :write [29 5] INFO [2022-01-10 12:53:33,457] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,458] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,458] jepsen worker 3 - jepsen.util 3 :ok :read [1 13] INFO [2022-01-10 12:53:33,458] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,458] jepsen worker 88 - jepsen.util 178 :ok :read [29 5] INFO [2022-01-10 12:53:33,459] jepsen worker 3 - jepsen.util 3 :ok :read [1 13] INFO [2022-01-10 12:53:33,459] jepsen worker 88 - jepsen.util 178 :invoke :write [29 37] INFO [2022-01-10 12:53:33,460] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,460] jepsen worker 3 - jepsen.util 3 :ok :read [1 13] INFO [2022-01-10 12:53:33,461] jepsen worker 3 - jepsen.util 3 :invoke :write [1 8] INFO [2022-01-10 12:53:33,462] jepsen worker 75 - jepsen.util 165 :ok :write [25 36] INFO [2022-01-10 12:53:33,463] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:33,464] jepsen worker 75 - jepsen.util 165 :ok :read [25 36] INFO [2022-01-10 12:53:33,464] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:33,465] jepsen worker 75 - jepsen.util 165 :ok :read [25 36] INFO [2022-01-10 12:53:33,465] jepsen worker 75 - jepsen.util 165 :invoke :write [25 20] INFO [2022-01-10 12:53:33,466] jepsen worker 71 - jepsen.util 161 :ok :write [23 1] INFO [2022-01-10 12:53:33,467] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,467] jepsen worker 1 - jepsen.util 91 :ok :write [0 4] INFO [2022-01-10 12:53:33,468] jepsen worker 1 - jepsen.util 91 :invoke :write [0 5] INFO [2022-01-10 12:53:33,468] jepsen worker 71 - jepsen.util 161 :ok :read [23 1] INFO [2022-01-10 12:53:33,468] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,469] jepsen worker 71 - jepsen.util 161 :ok :read [23 1] INFO [2022-01-10 12:53:33,469] jepsen worker 71 - jepsen.util 161 :invoke :write [23 46] INFO [2022-01-10 12:53:33,470] jepsen worker 38 - jepsen.util 38 :ok :write [12 38] INFO [2022-01-10 12:53:33,470] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:33,471] jepsen worker 38 - jepsen.util 38 :ok :read [12 38] INFO [2022-01-10 12:53:33,472] jepsen worker 38 - jepsen.util 38 :invoke :write [12 40] INFO [2022-01-10 12:53:33,472] jepsen worker 88 - jepsen.util 178 :ok :write [29 37] INFO [2022-01-10 12:53:33,473] jepsen worker 88 - jepsen.util 178 :invoke :write [29 19] INFO [2022-01-10 12:53:33,477] jepsen worker 36 - jepsen.util 36 :info :write [12 20] :RequestTimeoutException INFO [2022-01-10 12:53:33,478] jepsen worker 36 - jepsen.util 126 :invoke :write [12 27] INFO [2022-01-10 12:53:33,484] jepsen worker 1 - jepsen.util 91 :ok :write [0 5] INFO [2022-01-10 12:53:33,485] jepsen worker 86 - jepsen.util 176 :ok :write [28 4] INFO [2022-01-10 12:53:33,485] jepsen worker 1 - jepsen.util 91 :invoke :write [0 13] INFO [2022-01-10 12:53:33,485] jepsen worker 86 - jepsen.util 176 :invoke :write [28 1] INFO [2022-01-10 12:53:33,488] jepsen worker 88 - jepsen.util 178 :ok :write [29 19] INFO [2022-01-10 12:53:33,489] jepsen worker 3 - jepsen.util 3 :ok :write [1 8] INFO [2022-01-10 12:53:33,489] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,490] jepsen worker 15 - jepsen.util 15 :ok :write [5 7] INFO [2022-01-10 12:53:33,490] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,490] jepsen worker 88 - jepsen.util 178 :ok :read [29 19] INFO [2022-01-10 12:53:33,491] jepsen worker 3 - jepsen.util 3 :ok :read [1 8] INFO [2022-01-10 12:53:33,491] jepsen worker 51 - jepsen.util 141 :ok :write [17 25] INFO [2022-01-10 12:53:33,491] jepsen worker 15 - jepsen.util 15 :invoke :write [5 36] INFO [2022-01-10 12:53:33,492] jepsen worker 3 - jepsen.util 3 :invoke :write [1 11] INFO [2022-01-10 12:53:33,493] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,493] jepsen worker 88 - jepsen.util 178 :ok :read [29 19] INFO [2022-01-10 12:53:33,493] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:33,494] jepsen worker 88 - jepsen.util 178 :invoke :write [29 43] INFO [2022-01-10 12:53:33,494] jepsen worker 51 - jepsen.util 141 :ok :read [17 25] INFO [2022-01-10 12:53:33,495] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:33,496] jepsen worker 51 - jepsen.util 141 :ok :read [17 25] INFO [2022-01-10 12:53:33,496] jepsen worker 1 - jepsen.util 91 :ok :write [0 13] INFO [2022-01-10 12:53:33,497] jepsen worker 1 - jepsen.util 91 :invoke :write [0 5] INFO [2022-01-10 12:53:33,497] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:33,499] jepsen worker 38 - jepsen.util 38 :ok :write [12 40] INFO [2022-01-10 12:53:33,500] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:33,501] jepsen worker 38 - jepsen.util 38 :ok :read [12 40] INFO [2022-01-10 12:53:33,501] jepsen worker 51 - jepsen.util 141 :ok :read [17 25] INFO [2022-01-10 12:53:33,501] jepsen worker 38 - jepsen.util 38 :invoke :write [12 26] INFO [2022-01-10 12:53:33,502] jepsen worker 51 - jepsen.util 141 :invoke :write [17 22] INFO [2022-01-10 12:53:33,507] jepsen worker 86 - jepsen.util 176 :ok :write [28 1] INFO [2022-01-10 12:53:33,508] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,508] jepsen worker 86 - jepsen.util 176 :ok :read [28 1] INFO [2022-01-10 12:53:33,509] jepsen worker 3 - jepsen.util 3 :ok :write [1 11] INFO [2022-01-10 12:53:33,509] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,510] jepsen worker 3 - jepsen.util 3 :invoke :write [1 23] INFO [2022-01-10 12:53:33,510] jepsen worker 86 - jepsen.util 176 :ok :read [28 1] INFO [2022-01-10 12:53:33,511] jepsen worker 86 - jepsen.util 176 :invoke :write [28 28] INFO [2022-01-10 12:53:33,512] jepsen worker 1 - jepsen.util 91 :ok :write [0 5] INFO [2022-01-10 12:53:33,513] jepsen worker 1 - jepsen.util 91 :invoke :write [0 5] INFO [2022-01-10 12:53:33,516] jepsen worker 38 - jepsen.util 38 :ok :write [12 26] INFO [2022-01-10 12:53:33,517] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:33,517] jepsen worker 38 - jepsen.util 38 :ok :read [12 26] INFO [2022-01-10 12:53:33,518] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:33,519] jepsen worker 38 - jepsen.util 38 :ok :read [12 26] INFO [2022-01-10 12:53:33,519] jepsen worker 88 - jepsen.util 178 :ok :write [29 43] INFO [2022-01-10 12:53:33,520] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:33,520] jepsen worker 88 - jepsen.util 178 :invoke :write [29 29] INFO [2022-01-10 12:53:33,521] jepsen worker 38 - jepsen.util 38 :ok :read [12 26] INFO [2022-01-10 12:53:33,521] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:33,522] jepsen worker 86 - jepsen.util 176 :ok :write [28 28] INFO [2022-01-10 12:53:33,522] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,522] jepsen worker 38 - jepsen.util 38 :ok :read [12 26] INFO [2022-01-10 12:53:33,523] jepsen worker 86 - jepsen.util 176 :ok :read [28 28] INFO [2022-01-10 12:53:33,523] jepsen worker 38 - jepsen.util 38 :invoke :write [12 34] INFO [2022-01-10 12:53:33,524] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,524] jepsen worker 86 - jepsen.util 176 :ok :read [28 28] INFO [2022-01-10 12:53:33,525] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,526] jepsen worker 86 - jepsen.util 176 :ok :read [28 28] INFO [2022-01-10 12:53:33,526] jepsen worker 86 - jepsen.util 176 :invoke :write [28 36] INFO [2022-01-10 12:53:33,526] jepsen worker 1 - jepsen.util 91 :ok :write [0 5] INFO [2022-01-10 12:53:33,526] jepsen worker 3 - jepsen.util 3 :ok :write [1 23] INFO [2022-01-10 12:53:33,527] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,527] jepsen worker 3 - jepsen.util 3 :invoke :write [1 10] INFO [2022-01-10 12:53:33,528] jepsen worker 1 - jepsen.util 91 :ok :read [0 5] INFO [2022-01-10 12:53:33,528] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,529] jepsen worker 1 - jepsen.util 91 :ok :read [0 5] INFO [2022-01-10 12:53:33,530] jepsen worker 1 - jepsen.util 91 :invoke :write [0 37] INFO [2022-01-10 12:53:33,536] jepsen worker 88 - jepsen.util 178 :ok :write [29 29] INFO [2022-01-10 12:53:33,537] jepsen worker 88 - jepsen.util 178 :invoke :write [29 23] INFO [2022-01-10 12:53:33,538] jepsen worker 86 - jepsen.util 176 :ok :write [28 36] INFO [2022-01-10 12:53:33,538] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,539] jepsen worker 86 - jepsen.util 176 :ok :read [28 36] INFO [2022-01-10 12:53:33,540] jepsen worker 86 - jepsen.util 176 :invoke :write [28 20] INFO [2022-01-10 12:53:33,542] jepsen worker 1 - jepsen.util 91 :ok :write [0 37] INFO [2022-01-10 12:53:33,543] jepsen worker 1 - jepsen.util 91 :invoke :write [0 49] INFO [2022-01-10 12:53:33,545] jepsen worker 38 - jepsen.util 38 :ok :write [12 34] INFO [2022-01-10 12:53:33,546] jepsen worker 38 - jepsen.util 38 :invoke :write [12 25] INFO [2022-01-10 12:53:33,549] jepsen worker 88 - jepsen.util 178 :ok :write [29 23] INFO [2022-01-10 12:53:33,550] jepsen worker 88 - jepsen.util 178 :invoke :write [29 16] INFO [2022-01-10 12:53:33,550] jepsen worker 3 - jepsen.util 3 :ok :write [1 10] INFO [2022-01-10 12:53:33,551] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,551] jepsen worker 3 - jepsen.util 3 :ok :read [1 10] INFO [2022-01-10 12:53:33,552] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 12:53:33,553] jepsen worker 1 - jepsen.util 91 :ok :write [0 49] INFO [2022-01-10 12:53:33,554] jepsen worker 1 - jepsen.util 91 :invoke :write [0 21] INFO [2022-01-10 12:53:33,557] jepsen worker 86 - jepsen.util 176 :ok :write [28 20] INFO [2022-01-10 12:53:33,558] jepsen worker 86 - jepsen.util 176 :invoke :write [28 14] INFO [2022-01-10 12:53:33,561] jepsen worker 88 - jepsen.util 178 :ok :write [29 16] INFO [2022-01-10 12:53:33,561] jepsen worker 88 - jepsen.util 178 :invoke :write [29 41] INFO [2022-01-10 12:53:33,566] jepsen worker 38 - jepsen.util 38 :ok :write [12 25] INFO [2022-01-10 12:53:33,566] jepsen worker 38 - jepsen.util 38 :invoke :write [12 29] INFO [2022-01-10 12:53:33,567] jepsen worker 1 - jepsen.util 91 :ok :write [0 21] INFO [2022-01-10 12:53:33,568] jepsen worker 1 - jepsen.util 91 :invoke :write [0 0] INFO [2022-01-10 12:53:33,571] jepsen worker 3 - jepsen.util 3 :ok :write [1 40] INFO [2022-01-10 12:53:33,572] jepsen worker 3 - jepsen.util 3 :invoke :write [1 3] INFO [2022-01-10 12:53:33,575] jepsen worker 88 - jepsen.util 178 :ok :write [29 41] INFO [2022-01-10 12:53:33,575] jepsen worker 88 - jepsen.util 178 :invoke :write [29 39] INFO [2022-01-10 12:53:33,575] jepsen worker 86 - jepsen.util 176 :ok :write [28 14] INFO [2022-01-10 12:53:33,576] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,577] jepsen worker 62 - jepsen.util 152 :info :write [20 0] :RequestTimeoutException INFO [2022-01-10 12:53:33,578] jepsen worker 77 - jepsen.util 257 :fail :read [25 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:33,578] jepsen worker 86 - jepsen.util 176 :ok :read [28 14] INFO [2022-01-10 12:53:33,579] jepsen worker 62 - jepsen.util 242 :invoke :write [20 33] INFO [2022-01-10 12:53:33,579] jepsen worker 77 - jepsen.util 257 :invoke :write [25 5] INFO [2022-01-10 12:53:33,580] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,581] jepsen worker 86 - jepsen.util 176 :ok :read [28 14] INFO [2022-01-10 12:53:33,581] jepsen worker 1 - jepsen.util 91 :ok :write [0 0] INFO [2022-01-10 12:53:33,581] jepsen worker 1 - jepsen.util 91 :invoke :write [0 15] INFO [2022-01-10 12:53:33,582] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,583] jepsen worker 86 - jepsen.util 176 :ok :read [28 14] INFO [2022-01-10 12:53:33,583] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,583] jepsen worker 38 - jepsen.util 38 :ok :write [12 29] INFO [2022-01-10 12:53:33,584] jepsen worker 86 - jepsen.util 176 :ok :read [28 14] INFO [2022-01-10 12:53:33,584] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:33,584] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,585] jepsen worker 38 - jepsen.util 38 :ok :read [12 29] INFO [2022-01-10 12:53:33,585] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:33,585] jepsen worker 86 - jepsen.util 176 :ok :read [28 14] INFO [2022-01-10 12:53:33,586] jepsen worker 38 - jepsen.util 38 :ok :read [12 29] INFO [2022-01-10 12:53:33,586] jepsen worker 86 - jepsen.util 176 :invoke :write [28 38] INFO [2022-01-10 12:53:33,586] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:33,587] jepsen worker 38 - jepsen.util 38 :ok :read [12 29] INFO [2022-01-10 12:53:33,588] jepsen worker 38 - jepsen.util 38 :invoke :write [12 33] INFO [2022-01-10 12:53:33,589] jepsen worker 3 - jepsen.util 3 :ok :write [1 3] INFO [2022-01-10 12:53:33,589] jepsen worker 88 - jepsen.util 178 :ok :write [29 39] INFO [2022-01-10 12:53:33,589] jepsen worker 3 - jepsen.util 3 :invoke :write [1 17] INFO [2022-01-10 12:53:33,590] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,590] jepsen worker 88 - jepsen.util 178 :ok :read [29 39] INFO [2022-01-10 12:53:33,591] jepsen worker 88 - jepsen.util 178 :invoke :write [29 9] INFO [2022-01-10 12:53:33,598] jepsen worker 1 - jepsen.util 91 :ok :write [0 15] INFO [2022-01-10 12:53:33,599] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,600] jepsen worker 1 - jepsen.util 91 :ok :read [0 15] INFO [2022-01-10 12:53:33,600] jepsen worker 1 - jepsen.util 91 :invoke :write [0 32] INFO [2022-01-10 12:53:33,604] jepsen worker 88 - jepsen.util 178 :ok :write [29 9] INFO [2022-01-10 12:53:33,604] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,605] jepsen worker 88 - jepsen.util 178 :ok :read [29 9] INFO [2022-01-10 12:53:33,606] jepsen worker 88 - jepsen.util 178 :invoke :write [29 16] INFO [2022-01-10 12:53:33,606] jepsen worker 86 - jepsen.util 176 :ok :write [28 38] INFO [2022-01-10 12:53:33,606] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,607] jepsen worker 86 - jepsen.util 176 :ok :read [28 38] INFO [2022-01-10 12:53:33,607] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,608] jepsen worker 38 - jepsen.util 38 :ok :write [12 33] INFO [2022-01-10 12:53:33,608] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:33,608] jepsen worker 86 - jepsen.util 176 :ok :read [28 38] INFO [2022-01-10 12:53:33,609] jepsen worker 1 - jepsen.util 91 :ok :write [0 32] INFO [2022-01-10 12:53:33,609] jepsen worker 38 - jepsen.util 38 :ok :read [12 33] INFO [2022-01-10 12:53:33,609] jepsen worker 86 - jepsen.util 176 :invoke :write [28 29] INFO [2022-01-10 12:53:33,609] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,610] jepsen worker 38 - jepsen.util 38 :invoke :write [12 28] INFO [2022-01-10 12:53:33,610] jepsen worker 1 - jepsen.util 91 :ok :read [0 32] INFO [2022-01-10 12:53:33,611] jepsen worker 1 - jepsen.util 91 :invoke :write [0 25] INFO [2022-01-10 12:53:33,613] jepsen worker 3 - jepsen.util 3 :ok :write [1 17] INFO [2022-01-10 12:53:33,613] jepsen worker 3 - jepsen.util 3 :invoke :write [1 30] INFO [2022-01-10 12:53:33,626] jepsen worker 9 - jepsen.util 189 :info :write [3 39] :RequestTimeoutException INFO [2022-01-10 12:53:33,627] jepsen worker 9 - jepsen.util 279 :invoke :write [3 9] INFO [2022-01-10 12:53:33,658] jepsen worker 63 - jepsen.util 333 :fail :read [21 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:33,658] jepsen worker 63 - jepsen.util 333 :invoke :write [21 16] INFO [2022-01-10 12:53:33,659] jepsen worker 60 - jepsen.util 150 :info :write [20 0] :RequestTimeoutException INFO [2022-01-10 12:53:33,659] jepsen worker 60 - jepsen.util 240 :invoke :write [20 21] INFO [2022-01-10 12:53:33,697] jepsen worker 61 - jepsen.util 61 :ok :write [20 20] INFO [2022-01-10 12:53:33,698] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:33,699] jepsen worker 61 - jepsen.util 61 :ok :read [20 20] INFO [2022-01-10 12:53:33,699] jepsen worker 80 - jepsen.util 260 :ok :write [26 12] INFO [2022-01-10 12:53:33,699] jepsen worker 61 - jepsen.util 61 :invoke :write [20 31] INFO [2022-01-10 12:53:33,699] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:33,700] jepsen worker 80 - jepsen.util 260 :ok :read [26 12] INFO [2022-01-10 12:53:33,701] jepsen worker 80 - jepsen.util 260 :invoke :write [26 32] INFO [2022-01-10 12:53:33,702] jepsen worker 86 - jepsen.util 176 :ok :write [28 29] INFO [2022-01-10 12:53:33,703] jepsen worker 86 - jepsen.util 176 :invoke :write [28 25] INFO [2022-01-10 12:53:33,706] jepsen worker 13 - jepsen.util 13 :ok :write [4 6] INFO [2022-01-10 12:53:33,706] jepsen worker 50 - jepsen.util 50 :ok :write [16 2] INFO [2022-01-10 12:53:33,706] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:33,707] jepsen worker 50 - jepsen.util 50 :invoke :write [16 6] INFO [2022-01-10 12:53:33,707] jepsen worker 13 - jepsen.util 13 :ok :read [4 6] INFO [2022-01-10 12:53:33,708] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:33,709] jepsen worker 1 - jepsen.util 91 :ok :write [0 25] INFO [2022-01-10 12:53:33,709] jepsen worker 13 - jepsen.util 13 :ok :read [4 6] INFO [2022-01-10 12:53:33,709] jepsen worker 38 - jepsen.util 38 :ok :write [12 28] INFO [2022-01-10 12:53:33,709] jepsen worker 13 - jepsen.util 13 :invoke :write [4 11] INFO [2022-01-10 12:53:33,710] jepsen worker 38 - jepsen.util 38 :invoke :write [12 17] INFO [2022-01-10 12:53:33,710] jepsen worker 71 - jepsen.util 161 :ok :write [23 46] INFO [2022-01-10 12:53:33,711] jepsen worker 22 - jepsen.util 112 :ok :write [7 40] INFO [2022-01-10 12:53:33,711] jepsen worker 1 - jepsen.util 91 :invoke :write [0 24] INFO [2022-01-10 12:53:33,711] jepsen worker 22 - jepsen.util 112 :invoke :write [7 39] INFO [2022-01-10 12:53:33,712] jepsen worker 88 - jepsen.util 178 :ok :write [29 16] INFO [2022-01-10 12:53:33,712] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,712] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,713] jepsen worker 71 - jepsen.util 161 :ok :read [23 46] INFO [2022-01-10 12:53:33,713] jepsen worker 88 - jepsen.util 178 :ok :read [29 16] INFO [2022-01-10 12:53:33,714] jepsen worker 71 - jepsen.util 161 :invoke :write [23 30] INFO [2022-01-10 12:53:33,714] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,715] jepsen worker 88 - jepsen.util 178 :ok :read [29 16] INFO [2022-01-10 12:53:33,715] jepsen worker 88 - jepsen.util 178 :invoke :write [29 3] INFO [2022-01-10 12:53:33,716] jepsen worker 51 - jepsen.util 141 :ok :write [17 22] INFO [2022-01-10 12:53:33,716] jepsen worker 51 - jepsen.util 141 :invoke :write [17 10] INFO [2022-01-10 12:53:33,719] jepsen worker 83 - jepsen.util 263 :ok :write [27 13] INFO [2022-01-10 12:53:33,719] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:33,720] jepsen worker 3 - jepsen.util 3 :ok :write [1 30] INFO [2022-01-10 12:53:33,720] jepsen worker 39 - jepsen.util 129 :ok :read [13 38] INFO [2022-01-10 12:53:33,720] jepsen worker 83 - jepsen.util 263 :ok :read [27 13] INFO [2022-01-10 12:53:33,720] jepsen worker 39 - jepsen.util 129 :invoke :write [13 15] INFO [2022-01-10 12:53:33,721] jepsen worker 83 - jepsen.util 263 :invoke :write [27 9] INFO [2022-01-10 12:53:33,722] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,723] jepsen worker 3 - jepsen.util 3 :ok :read [1 30] INFO [2022-01-10 12:53:33,724] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,724] jepsen worker 44 - jepsen.util 44 :ok :write [14 16] INFO [2022-01-10 12:53:33,724] jepsen worker 3 - jepsen.util 3 :ok :read [1 30] INFO [2022-01-10 12:53:33,725] jepsen worker 44 - jepsen.util 44 :invoke :write [14 37] INFO [2022-01-10 12:53:33,726] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,726] jepsen worker 3 - jepsen.util 3 :ok :read [1 30] INFO [2022-01-10 12:53:33,727] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,728] jepsen worker 1 - jepsen.util 91 :ok :write [0 24] INFO [2022-01-10 12:53:33,728] jepsen worker 3 - jepsen.util 3 :ok :read [1 30] INFO [2022-01-10 12:53:33,728] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,729] jepsen worker 80 - jepsen.util 260 :ok :write [26 32] INFO [2022-01-10 12:53:33,729] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,729] jepsen worker 1 - jepsen.util 91 :ok :read [0 24] INFO [2022-01-10 12:53:33,729] jepsen worker 80 - jepsen.util 260 :invoke :write [26 1] INFO [2022-01-10 12:53:33,730] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,730] jepsen worker 3 - jepsen.util 3 :ok :read [1 30] INFO [2022-01-10 12:53:33,730] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,731] jepsen worker 1 - jepsen.util 91 :ok :read [0 24] INFO [2022-01-10 12:53:33,731] jepsen worker 1 - jepsen.util 91 :invoke :write [0 42] INFO [2022-01-10 12:53:33,731] jepsen worker 3 - jepsen.util 3 :ok :read [1 30] INFO [2022-01-10 12:53:33,732] jepsen worker 68 - jepsen.util 68 :ok :write [22 32] INFO [2022-01-10 12:53:33,732] jepsen worker 3 - jepsen.util 3 :invoke :write [1 22] INFO [2022-01-10 12:53:33,732] jepsen worker 88 - jepsen.util 178 :ok :write [29 3] INFO [2022-01-10 12:53:33,732] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:33,733] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,734] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:53:33,734] jepsen worker 88 - jepsen.util 178 :ok :read [29 3] INFO [2022-01-10 12:53:33,734] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:33,735] jepsen worker 88 - jepsen.util 178 :invoke :write [29 30] INFO [2022-01-10 12:53:33,735] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:53:33,736] jepsen worker 68 - jepsen.util 68 :invoke :write [22 32] INFO [2022-01-10 12:53:33,736] jepsen worker 86 - jepsen.util 176 :ok :write [28 25] INFO [2022-01-10 12:53:33,737] jepsen worker 86 - jepsen.util 176 :invoke :write [28 43] INFO [2022-01-10 12:53:33,737] jepsen worker 34 - jepsen.util 214 :ok :write [11 10] INFO [2022-01-10 12:53:33,738] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:33,739] jepsen worker 34 - jepsen.util 214 :ok :read [11 10] INFO [2022-01-10 12:53:33,739] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:33,740] jepsen worker 34 - jepsen.util 214 :ok :read [11 10] INFO [2022-01-10 12:53:33,740] jepsen worker 34 - jepsen.util 214 :invoke :write [11 45] INFO [2022-01-10 12:53:33,743] jepsen worker 7 - jepsen.util 7 :ok :write [2 36] INFO [2022-01-10 12:53:33,743] jepsen worker 7 - jepsen.util 7 :invoke :write [2 33] INFO [2022-01-10 12:53:33,748] jepsen worker 75 - jepsen.util 165 :ok :write [25 20] INFO [2022-01-10 12:53:33,749] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:33,750] jepsen worker 75 - jepsen.util 165 :ok :read [25 20] INFO [2022-01-10 12:53:33,750] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:33,751] jepsen worker 75 - jepsen.util 165 :ok :read [25 20] INFO [2022-01-10 12:53:33,751] jepsen worker 75 - jepsen.util 165 :invoke :write [25 0] INFO [2022-01-10 12:53:33,755] jepsen worker 15 - jepsen.util 15 :ok :write [5 36] INFO [2022-01-10 12:53:33,756] jepsen worker 15 - jepsen.util 15 :invoke :write [5 25] INFO [2022-01-10 12:53:33,758] jepsen worker 61 - jepsen.util 61 :ok :write [20 31] INFO [2022-01-10 12:53:33,759] jepsen worker 61 - jepsen.util 61 :invoke :write [20 36] INFO [2022-01-10 12:53:33,764] jepsen worker 50 - jepsen.util 50 :ok :write [16 6] INFO [2022-01-10 12:53:33,765] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:33,766] jepsen worker 50 - jepsen.util 50 :ok :read [16 6] INFO [2022-01-10 12:53:33,767] jepsen worker 50 - jepsen.util 50 :invoke :write [16 20] INFO [2022-01-10 12:53:33,773] jepsen worker 83 - jepsen.util 263 :ok :write [27 9] INFO [2022-01-10 12:53:33,773] jepsen worker 83 - jepsen.util 263 :invoke :write [27 36] INFO [2022-01-10 12:53:33,774] jepsen worker 29 - jepsen.util 209 :info :write [9 21] :RequestTimeoutException INFO [2022-01-10 12:53:33,775] jepsen worker 29 - jepsen.util 299 :invoke :read [9 nil] INFO [2022-01-10 12:53:33,780] jepsen worker 68 - jepsen.util 68 :ok :write [22 32] INFO [2022-01-10 12:53:33,780] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:33,782] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:53:33,782] jepsen worker 68 - jepsen.util 68 :invoke :write [22 45] INFO [2022-01-10 12:53:33,783] jepsen worker 34 - jepsen.util 214 :ok :write [11 45] INFO [2022-01-10 12:53:33,783] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:33,784] jepsen worker 34 - jepsen.util 214 :ok :read [11 45] INFO [2022-01-10 12:53:33,784] jepsen worker 34 - jepsen.util 214 :invoke :write [11 29] INFO [2022-01-10 12:53:33,787] jepsen worker 7 - jepsen.util 7 :ok :write [2 33] INFO [2022-01-10 12:53:33,788] jepsen worker 7 - jepsen.util 7 :invoke :write [2 25] INFO [2022-01-10 12:53:33,796] jepsen worker 15 - jepsen.util 15 :ok :write [5 25] INFO [2022-01-10 12:53:33,797] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:33,798] jepsen worker 15 - jepsen.util 15 :ok :read [5 25] INFO [2022-01-10 12:53:33,799] jepsen worker 15 - jepsen.util 15 :invoke :write [5 31] INFO [2022-01-10 12:53:33,802] jepsen worker 22 - jepsen.util 112 :ok :write [7 39] INFO [2022-01-10 12:53:33,803] jepsen worker 13 - jepsen.util 13 :ok :write [4 11] INFO [2022-01-10 12:53:33,803] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:33,804] jepsen worker 13 - jepsen.util 13 :invoke :write [4 13] INFO [2022-01-10 12:53:33,804] jepsen worker 50 - jepsen.util 50 :ok :write [16 20] INFO [2022-01-10 12:53:33,804] jepsen worker 22 - jepsen.util 112 :ok :read [7 39] INFO [2022-01-10 12:53:33,805] jepsen worker 50 - jepsen.util 50 :invoke :write [16 17] INFO [2022-01-10 12:53:33,806] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:33,806] jepsen worker 40 - jepsen.util 130 :ok :write [13 39] INFO [2022-01-10 12:53:33,807] jepsen worker 22 - jepsen.util 112 :ok :read [7 39] INFO [2022-01-10 12:53:33,807] jepsen worker 40 - jepsen.util 130 :invoke :write [13 14] INFO [2022-01-10 12:53:33,807] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:33,808] jepsen worker 38 - jepsen.util 38 :ok :write [12 17] INFO [2022-01-10 12:53:33,809] jepsen worker 22 - jepsen.util 112 :ok :read [7 39] INFO [2022-01-10 12:53:33,809] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:33,809] jepsen worker 44 - jepsen.util 44 :ok :write [14 37] INFO [2022-01-10 12:53:33,810] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:33,811] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:33,811] jepsen worker 22 - jepsen.util 112 :ok :read [7 39] INFO [2022-01-10 12:53:33,811] jepsen worker 22 - jepsen.util 112 :invoke :write [7 3] INFO [2022-01-10 12:53:33,811] jepsen worker 44 - jepsen.util 44 :ok :read [14 37] INFO [2022-01-10 12:53:33,812] jepsen worker 83 - jepsen.util 263 :ok :write [27 36] INFO [2022-01-10 12:53:33,813] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:33,813] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:33,814] jepsen worker 44 - jepsen.util 44 :ok :read [14 37] INFO [2022-01-10 12:53:33,814] jepsen worker 1 - jepsen.util 91 :ok :write [0 42] INFO [2022-01-10 12:53:33,814] jepsen worker 83 - jepsen.util 263 :ok :read [27 36] INFO [2022-01-10 12:53:33,814] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,815] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:33,815] jepsen worker 1 - jepsen.util 91 :ok :read [0 42] INFO [2022-01-10 12:53:33,815] jepsen worker 83 - jepsen.util 263 :invoke :write [27 24] INFO [2022-01-10 12:53:33,815] jepsen worker 44 - jepsen.util 44 :ok :read [14 37] INFO [2022-01-10 12:53:33,816] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,816] jepsen worker 71 - jepsen.util 161 :ok :write [23 30] INFO [2022-01-10 12:53:33,816] jepsen worker 44 - jepsen.util 44 :invoke :write [14 33] INFO [2022-01-10 12:53:33,816] jepsen worker 1 - jepsen.util 91 :ok :read [0 42] INFO [2022-01-10 12:53:33,817] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,817] jepsen worker 1 - jepsen.util 91 :invoke :write [0 37] INFO [2022-01-10 12:53:33,818] jepsen worker 88 - jepsen.util 178 :ok :write [29 30] INFO [2022-01-10 12:53:33,818] jepsen worker 71 - jepsen.util 161 :ok :read [23 30] INFO [2022-01-10 12:53:33,818] jepsen worker 88 - jepsen.util 178 :invoke :write [29 27] INFO [2022-01-10 12:53:33,819] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,820] jepsen worker 71 - jepsen.util 161 :ok :read [23 30] INFO [2022-01-10 12:53:33,821] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,821] jepsen worker 51 - jepsen.util 141 :ok :write [17 10] INFO [2022-01-10 12:53:33,821] jepsen worker 68 - jepsen.util 68 :ok :write [22 45] INFO [2022-01-10 12:53:33,821] jepsen worker 71 - jepsen.util 161 :ok :read [23 30] INFO [2022-01-10 12:53:33,822] jepsen worker 51 - jepsen.util 141 :invoke :write [17 45] INFO [2022-01-10 12:53:33,823] jepsen worker 68 - jepsen.util 68 :invoke :write [22 45] INFO [2022-01-10 12:53:33,823] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,824] jepsen worker 71 - jepsen.util 161 :ok :read [23 30] INFO [2022-01-10 12:53:33,825] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,826] jepsen worker 71 - jepsen.util 161 :ok :read [23 30] INFO [2022-01-10 12:53:33,826] jepsen worker 71 - jepsen.util 161 :invoke :write [23 40] INFO [2022-01-10 12:53:33,826] jepsen worker 75 - jepsen.util 165 :ok :write [25 0] INFO [2022-01-10 12:53:33,827] jepsen worker 3 - jepsen.util 3 :ok :write [1 22] INFO [2022-01-10 12:53:33,827] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:33,827] jepsen worker 34 - jepsen.util 214 :ok :write [11 29] INFO [2022-01-10 12:53:33,827] jepsen worker 3 - jepsen.util 3 :invoke :write [1 4] INFO [2022-01-10 12:53:33,828] jepsen worker 75 - jepsen.util 165 :ok :read [25 0] INFO [2022-01-10 12:53:33,828] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:33,828] jepsen worker 75 - jepsen.util 165 :invoke :write [25 16] INFO [2022-01-10 12:53:33,829] jepsen worker 34 - jepsen.util 214 :ok :read [11 29] INFO [2022-01-10 12:53:33,829] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:33,830] jepsen worker 34 - jepsen.util 214 :ok :read [11 29] INFO [2022-01-10 12:53:33,830] jepsen worker 61 - jepsen.util 61 :ok :write [20 36] INFO [2022-01-10 12:53:33,831] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:33,832] jepsen worker 34 - jepsen.util 214 :invoke :write [11 2] INFO [2022-01-10 12:53:33,832] jepsen worker 7 - jepsen.util 7 :ok :write [2 25] INFO [2022-01-10 12:53:33,832] jepsen worker 61 - jepsen.util 61 :ok :read [20 36] INFO [2022-01-10 12:53:33,833] jepsen worker 7 - jepsen.util 7 :invoke :write [2 12] INFO [2022-01-10 12:53:33,833] jepsen worker 61 - jepsen.util 61 :invoke :write [20 36] INFO [2022-01-10 12:53:33,836] jepsen worker 80 - jepsen.util 260 :ok :write [26 1] INFO [2022-01-10 12:53:33,837] jepsen worker 80 - jepsen.util 260 :invoke :write [26 21] INFO [2022-01-10 12:53:33,839] jepsen worker 15 - jepsen.util 15 :ok :write [5 31] INFO [2022-01-10 12:53:33,839] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:33,841] jepsen worker 15 - jepsen.util 15 :ok :read [5 31] INFO [2022-01-10 12:53:33,841] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:33,842] jepsen worker 40 - jepsen.util 130 :ok :write [13 14] INFO [2022-01-10 12:53:33,842] jepsen worker 86 - jepsen.util 176 :ok :write [28 43] INFO [2022-01-10 12:53:33,842] jepsen worker 22 - jepsen.util 112 :ok :write [7 3] INFO [2022-01-10 12:53:33,842] jepsen worker 15 - jepsen.util 15 :ok :read [5 31] INFO [2022-01-10 12:53:33,843] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,843] jepsen worker 22 - jepsen.util 112 :invoke :write [7 46] INFO [2022-01-10 12:53:33,844] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:33,844] jepsen worker 15 - jepsen.util 15 :invoke :write [5 5] INFO [2022-01-10 12:53:33,844] jepsen worker 40 - jepsen.util 130 :invoke :write [13 16] INFO [2022-01-10 12:53:33,844] jepsen worker 86 - jepsen.util 176 :invoke :write [28 38] INFO [2022-01-10 12:53:33,846] jepsen worker 13 - jepsen.util 13 :ok :write [4 13] INFO [2022-01-10 12:53:33,847] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:33,848] jepsen worker 13 - jepsen.util 13 :ok :read [4 13] INFO [2022-01-10 12:53:33,848] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:33,849] jepsen worker 13 - jepsen.util 13 :ok :read [4 13] INFO [2022-01-10 12:53:33,850] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:33,851] jepsen worker 13 - jepsen.util 13 :ok :read [4 13] INFO [2022-01-10 12:53:33,851] jepsen worker 50 - jepsen.util 50 :ok :write [16 17] INFO [2022-01-10 12:53:33,851] jepsen worker 13 - jepsen.util 13 :invoke :write [4 15] INFO [2022-01-10 12:53:33,852] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:33,852] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:33,852] jepsen worker 1 - jepsen.util 91 :ok :write [0 37] INFO [2022-01-10 12:53:33,852] jepsen worker 50 - jepsen.util 50 :ok :read [16 17] INFO [2022-01-10 12:53:33,853] jepsen worker 38 - jepsen.util 38 :invoke :write [12 31] INFO [2022-01-10 12:53:33,853] jepsen worker 50 - jepsen.util 50 :invoke :write [16 34] INFO [2022-01-10 12:53:33,854] jepsen worker 1 - jepsen.util 91 :invoke :write [0 41] INFO [2022-01-10 12:53:33,854] jepsen worker 44 - jepsen.util 44 :ok :write [14 33] INFO [2022-01-10 12:53:33,855] jepsen worker 44 - jepsen.util 44 :invoke :write [14 12] INFO [2022-01-10 12:53:33,857] jepsen worker 88 - jepsen.util 178 :ok :write [29 27] INFO [2022-01-10 12:53:33,858] jepsen worker 71 - jepsen.util 161 :ok :write [23 40] INFO [2022-01-10 12:53:33,858] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,859] jepsen worker 71 - jepsen.util 161 :invoke :write [23 17] INFO [2022-01-10 12:53:33,859] jepsen worker 88 - jepsen.util 178 :ok :read [29 27] INFO [2022-01-10 12:53:33,860] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,861] jepsen worker 88 - jepsen.util 178 :ok :read [29 27] INFO [2022-01-10 12:53:33,861] jepsen worker 88 - jepsen.util 178 :invoke :write [29 42] INFO [2022-01-10 12:53:33,862] jepsen worker 83 - jepsen.util 263 :ok :write [27 24] INFO [2022-01-10 12:53:33,863] jepsen worker 83 - jepsen.util 263 :invoke :write [27 6] INFO [2022-01-10 12:53:33,866] jepsen worker 51 - jepsen.util 141 :ok :write [17 45] INFO [2022-01-10 12:53:33,866] jepsen worker 68 - jepsen.util 68 :ok :write [22 45] INFO [2022-01-10 12:53:33,867] jepsen worker 51 - jepsen.util 141 :invoke :write [17 20] INFO [2022-01-10 12:53:33,868] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:33,868] jepsen worker 68 - jepsen.util 68 :ok :read [22 45] INFO [2022-01-10 12:53:33,869] jepsen worker 68 - jepsen.util 68 :invoke :write [22 31] INFO [2022-01-10 12:53:33,870] jepsen worker 3 - jepsen.util 3 :ok :write [1 4] INFO [2022-01-10 12:53:33,870] jepsen worker 3 - jepsen.util 3 :invoke :write [1 30] INFO [2022-01-10 12:53:33,872] jepsen worker 75 - jepsen.util 165 :ok :write [25 16] INFO [2022-01-10 12:53:33,873] jepsen worker 75 - jepsen.util 165 :invoke :write [25 19] INFO [2022-01-10 12:53:33,878] jepsen worker 34 - jepsen.util 214 :ok :write [11 2] INFO [2022-01-10 12:53:33,878] jepsen worker 80 - jepsen.util 260 :ok :write [26 21] INFO [2022-01-10 12:53:33,878] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:33,879] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:33,879] jepsen worker 80 - jepsen.util 260 :ok :read [26 21] INFO [2022-01-10 12:53:33,880] jepsen worker 34 - jepsen.util 214 :ok :read [11 2] INFO [2022-01-10 12:53:33,880] jepsen worker 80 - jepsen.util 260 :invoke :write [26 13] INFO [2022-01-10 12:53:33,880] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:33,881] jepsen worker 40 - jepsen.util 130 :ok :write [13 16] INFO [2022-01-10 12:53:33,881] jepsen worker 34 - jepsen.util 214 :ok :read [11 2] INFO [2022-01-10 12:53:33,882] jepsen worker 40 - jepsen.util 130 :invoke :write [13 15] INFO [2022-01-10 12:53:33,882] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:33,883] jepsen worker 34 - jepsen.util 214 :ok :read [11 2] INFO [2022-01-10 12:53:33,884] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:33,884] jepsen worker 61 - jepsen.util 61 :ok :write [20 36] INFO [2022-01-10 12:53:33,884] jepsen worker 61 - jepsen.util 61 :invoke :read [20 nil] INFO [2022-01-10 12:53:33,885] jepsen worker 34 - jepsen.util 214 :ok :read [11 2] INFO [2022-01-10 12:53:33,885] jepsen worker 34 - jepsen.util 214 :invoke :write [11 49] INFO [2022-01-10 12:53:33,885] jepsen worker 61 - jepsen.util 61 :ok :read [20 36] INFO [2022-01-10 12:53:33,886] jepsen worker 61 - jepsen.util 61 :invoke :write [20 33] INFO [2022-01-10 12:53:33,886] jepsen worker 86 - jepsen.util 176 :ok :write [28 38] INFO [2022-01-10 12:53:33,887] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,887] jepsen worker 86 - jepsen.util 176 :ok :read [28 38] INFO [2022-01-10 12:53:33,888] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,888] jepsen worker 13 - jepsen.util 13 :ok :write [4 15] INFO [2022-01-10 12:53:33,889] jepsen worker 13 - jepsen.util 13 :invoke :write [4 31] INFO [2022-01-10 12:53:33,889] jepsen worker 86 - jepsen.util 176 :ok :read [28 38] INFO [2022-01-10 12:53:33,889] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,890] jepsen worker 7 - jepsen.util 7 :ok :write [2 12] INFO [2022-01-10 12:53:33,890] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:33,890] jepsen worker 86 - jepsen.util 176 :ok :read [28 38] INFO [2022-01-10 12:53:33,891] jepsen worker 7 - jepsen.util 7 :ok :read [2 12] INFO [2022-01-10 12:53:33,891] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,891] jepsen worker 7 - jepsen.util 7 :invoke :write [2 0] INFO [2022-01-10 12:53:33,891] jepsen worker 86 - jepsen.util 176 :ok :read [28 38] INFO [2022-01-10 12:53:33,892] jepsen worker 86 - jepsen.util 176 :invoke :write [28 10] INFO [2022-01-10 12:53:33,895] jepsen worker 1 - jepsen.util 91 :ok :write [0 41] INFO [2022-01-10 12:53:33,895] jepsen worker 1 - jepsen.util 91 :invoke :write [0 1] INFO [2022-01-10 12:53:33,895] jepsen worker 38 - jepsen.util 38 :ok :write [12 31] INFO [2022-01-10 12:53:33,896] jepsen worker 38 - jepsen.util 38 :invoke :write [12 1] INFO [2022-01-10 12:53:33,897] jepsen worker 15 - jepsen.util 15 :ok :write [5 5] INFO [2022-01-10 12:53:33,897] jepsen worker 15 - jepsen.util 15 :invoke :write [5 6] INFO [2022-01-10 12:53:33,899] jepsen worker 88 - jepsen.util 178 :ok :write [29 42] INFO [2022-01-10 12:53:33,899] jepsen worker 22 - jepsen.util 112 :ok :write [7 46] INFO [2022-01-10 12:53:33,900] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,900] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:33,901] jepsen worker 88 - jepsen.util 178 :ok :read [29 42] INFO [2022-01-10 12:53:33,901] jepsen worker 71 - jepsen.util 161 :ok :write [23 17] INFO [2022-01-10 12:53:33,901] jepsen worker 22 - jepsen.util 112 :ok :read [7 46] INFO [2022-01-10 12:53:33,902] jepsen worker 71 - jepsen.util 161 :invoke :write [23 47] INFO [2022-01-10 12:53:33,902] jepsen worker 22 - jepsen.util 112 :invoke :write [7 31] INFO [2022-01-10 12:53:33,903] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,903] jepsen worker 88 - jepsen.util 178 :ok :read [29 42] INFO [2022-01-10 12:53:33,904] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,905] jepsen worker 88 - jepsen.util 178 :ok :read [29 42] INFO [2022-01-10 12:53:33,905] jepsen worker 50 - jepsen.util 50 :ok :write [16 34] INFO [2022-01-10 12:53:33,905] jepsen worker 88 - jepsen.util 178 :invoke :write [29 30] INFO [2022-01-10 12:53:33,906] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:33,906] jepsen worker 50 - jepsen.util 50 :ok :read [16 34] INFO [2022-01-10 12:53:33,907] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:33,908] jepsen worker 51 - jepsen.util 141 :ok :write [17 20] INFO [2022-01-10 12:53:33,908] jepsen worker 50 - jepsen.util 50 :ok :read [16 34] INFO [2022-01-10 12:53:33,908] jepsen worker 51 - jepsen.util 141 :invoke :write [17 11] INFO [2022-01-10 12:53:33,909] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:33,909] jepsen worker 44 - jepsen.util 44 :ok :write [14 12] INFO [2022-01-10 12:53:33,910] jepsen worker 50 - jepsen.util 50 :ok :read [16 34] INFO [2022-01-10 12:53:33,910] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:33,911] jepsen worker 50 - jepsen.util 50 :invoke :write [16 40] INFO [2022-01-10 12:53:33,911] jepsen worker 44 - jepsen.util 44 :ok :read [14 12] INFO [2022-01-10 12:53:33,911] jepsen worker 44 - jepsen.util 44 :invoke :write [14 4] INFO [2022-01-10 12:53:33,913] jepsen worker 3 - jepsen.util 3 :ok :write [1 30] INFO [2022-01-10 12:53:33,913] jepsen worker 3 - jepsen.util 3 :invoke :write [1 42] INFO [2022-01-10 12:53:33,915] jepsen worker 83 - jepsen.util 263 :ok :write [27 6] INFO [2022-01-10 12:53:33,916] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:33,917] jepsen worker 40 - jepsen.util 130 :ok :write [13 15] INFO [2022-01-10 12:53:33,917] jepsen worker 83 - jepsen.util 263 :ok :read [27 6] INFO [2022-01-10 12:53:33,918] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,918] jepsen worker 83 - jepsen.util 263 :invoke :write [27 2] INFO [2022-01-10 12:53:33,919] jepsen worker 40 - jepsen.util 130 :ok :read [13 15] INFO [2022-01-10 12:53:33,919] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,920] jepsen worker 40 - jepsen.util 130 :ok :read [13 15] INFO [2022-01-10 12:53:33,921] jepsen worker 80 - jepsen.util 260 :ok :write [26 13] INFO [2022-01-10 12:53:33,921] jepsen worker 40 - jepsen.util 130 :invoke :write [13 23] INFO [2022-01-10 12:53:33,921] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:33,922] jepsen worker 80 - jepsen.util 260 :ok :read [26 13] INFO [2022-01-10 12:53:33,923] jepsen worker 80 - jepsen.util 260 :invoke :write [26 31] INFO [2022-01-10 12:53:33,927] jepsen worker 86 - jepsen.util 176 :ok :write [28 10] INFO [2022-01-10 12:53:33,928] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,928] jepsen worker 86 - jepsen.util 176 :ok :read [28 10] INFO [2022-01-10 12:53:33,929] jepsen worker 86 - jepsen.util 176 :invoke :write [28 21] INFO [2022-01-10 12:53:33,931] jepsen worker 13 - jepsen.util 13 :ok :write [4 31] INFO [2022-01-10 12:53:33,932] jepsen worker 13 - jepsen.util 13 :invoke :write [4 22] INFO [2022-01-10 12:53:33,935] jepsen worker 1 - jepsen.util 91 :ok :write [0 1] INFO [2022-01-10 12:53:33,935] jepsen worker 1 - jepsen.util 91 :invoke :write [0 17] INFO [2022-01-10 12:53:33,937] jepsen worker 38 - jepsen.util 38 :ok :write [12 1] INFO [2022-01-10 12:53:33,938] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:33,939] jepsen worker 38 - jepsen.util 38 :ok :read [12 1] INFO [2022-01-10 12:53:33,939] jepsen worker 38 - jepsen.util 38 :invoke :write [12 29] INFO [2022-01-10 12:53:33,943] jepsen worker 71 - jepsen.util 161 :ok :write [23 47] INFO [2022-01-10 12:53:33,943] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:33,944] jepsen worker 71 - jepsen.util 161 :ok :read [23 47] INFO [2022-01-10 12:53:33,945] jepsen worker 71 - jepsen.util 161 :invoke :write [23 12] INFO [2022-01-10 12:53:33,946] jepsen worker 88 - jepsen.util 178 :ok :write [29 30] INFO [2022-01-10 12:53:33,946] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,947] jepsen worker 88 - jepsen.util 178 :ok :read [29 30] INFO [2022-01-10 12:53:33,948] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,948] jepsen worker 88 - jepsen.util 178 :ok :read [29 30] INFO [2022-01-10 12:53:33,949] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,950] jepsen worker 88 - jepsen.util 178 :ok :read [29 30] INFO [2022-01-10 12:53:33,950] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:33,951] jepsen worker 88 - jepsen.util 178 :ok :read [29 30] INFO [2022-01-10 12:53:33,951] jepsen worker 88 - jepsen.util 178 :invoke :write [29 0] INFO [2022-01-10 12:53:33,954] jepsen worker 3 - jepsen.util 3 :ok :write [1 42] INFO [2022-01-10 12:53:33,954] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,955] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:33,956] jepsen worker 3 - jepsen.util 3 :invoke :write [1 4] INFO [2022-01-10 12:53:33,964] jepsen worker 86 - jepsen.util 176 :ok :write [28 21] INFO [2022-01-10 12:53:33,964] jepsen worker 40 - jepsen.util 130 :ok :write [13 23] INFO [2022-01-10 12:53:33,964] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,965] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,965] jepsen worker 86 - jepsen.util 176 :ok :read [28 21] INFO [2022-01-10 12:53:33,966] jepsen worker 40 - jepsen.util 130 :ok :read [13 23] INFO [2022-01-10 12:53:33,966] jepsen worker 86 - jepsen.util 176 :invoke :write [28 13] INFO [2022-01-10 12:53:33,966] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,967] jepsen worker 40 - jepsen.util 130 :ok :read [13 23] INFO [2022-01-10 12:53:33,968] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,969] jepsen worker 40 - jepsen.util 130 :ok :read [13 23] INFO [2022-01-10 12:53:33,969] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,970] jepsen worker 40 - jepsen.util 130 :ok :read [13 23] INFO [2022-01-10 12:53:33,971] jepsen worker 38 - jepsen.util 38 :ok :write [12 29] INFO [2022-01-10 12:53:33,971] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:33,971] jepsen worker 38 - jepsen.util 38 :invoke :write [12 46] INFO [2022-01-10 12:53:33,972] jepsen worker 40 - jepsen.util 130 :ok :read [13 23] INFO [2022-01-10 12:53:33,972] jepsen worker 40 - jepsen.util 130 :invoke :write [13 48] INFO [2022-01-10 12:53:33,973] jepsen worker 1 - jepsen.util 91 :ok :write [0 17] INFO [2022-01-10 12:53:33,974] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,975] jepsen worker 1 - jepsen.util 91 :ok :read [0 17] INFO [2022-01-10 12:53:33,976] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,977] jepsen worker 1 - jepsen.util 91 :ok :read [0 17] INFO [2022-01-10 12:53:33,977] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:33,979] jepsen worker 1 - jepsen.util 91 :ok :read [0 17] INFO [2022-01-10 12:53:33,979] jepsen worker 1 - jepsen.util 91 :invoke :write [0 48] INFO [2022-01-10 12:53:33,980] jepsen worker 88 - jepsen.util 178 :ok :write [29 0] INFO [2022-01-10 12:53:33,980] jepsen worker 88 - jepsen.util 178 :invoke :write [29 1] INFO [2022-01-10 12:53:33,980] jepsen worker 3 - jepsen.util 3 :ok :write [1 4] INFO [2022-01-10 12:53:33,981] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,982] jepsen worker 3 - jepsen.util 3 :ok :read [1 4] INFO [2022-01-10 12:53:33,982] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,983] jepsen worker 3 - jepsen.util 3 :ok :read [1 4] INFO [2022-01-10 12:53:33,983] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,984] jepsen worker 3 - jepsen.util 3 :ok :read [1 4] INFO [2022-01-10 12:53:33,985] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,986] jepsen worker 3 - jepsen.util 3 :ok :read [1 4] INFO [2022-01-10 12:53:33,986] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:33,987] jepsen worker 3 - jepsen.util 3 :ok :read [1 4] INFO [2022-01-10 12:53:33,987] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 12:53:33,989] jepsen worker 86 - jepsen.util 176 :ok :write [28 13] INFO [2022-01-10 12:53:33,989] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:33,990] jepsen worker 86 - jepsen.util 176 :ok :read [28 13] INFO [2022-01-10 12:53:33,990] jepsen worker 86 - jepsen.util 176 :invoke :write [28 27] INFO [2022-01-10 12:53:34,018] jepsen worker 68 - jepsen.util 68 :ok :write [22 31] INFO [2022-01-10 12:53:34,019] jepsen worker 68 - jepsen.util 68 :invoke :write [22 47] INFO [2022-01-10 12:53:34,020] jepsen worker 51 - jepsen.util 141 :ok :write [17 11] INFO [2022-01-10 12:53:34,020] jepsen worker 51 - jepsen.util 141 :invoke :write [17 22] INFO [2022-01-10 12:53:34,023] jepsen worker 75 - jepsen.util 165 :ok :write [25 19] INFO [2022-01-10 12:53:34,024] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:34,025] jepsen worker 75 - jepsen.util 165 :ok :read [25 19] INFO [2022-01-10 12:53:34,025] jepsen worker 80 - jepsen.util 260 :ok :write [26 31] INFO [2022-01-10 12:53:34,026] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:34,027] jepsen worker 80 - jepsen.util 260 :invoke :write [26 36] INFO [2022-01-10 12:53:34,027] jepsen worker 75 - jepsen.util 165 :ok :read [25 19] INFO [2022-01-10 12:53:34,028] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:34,029] jepsen worker 75 - jepsen.util 165 :ok :read [25 19] INFO [2022-01-10 12:53:34,029] jepsen worker 34 - jepsen.util 214 :ok :write [11 49] INFO [2022-01-10 12:53:34,029] jepsen worker 75 - jepsen.util 165 :invoke :write [25 5] INFO [2022-01-10 12:53:34,030] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:34,031] jepsen worker 34 - jepsen.util 214 :ok :read [11 49] INFO [2022-01-10 12:53:34,031] jepsen worker 13 - jepsen.util 13 :ok :write [4 22] INFO [2022-01-10 12:53:34,032] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:34,032] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:34,032] jepsen worker 34 - jepsen.util 214 :ok :read [11 49] INFO [2022-01-10 12:53:34,033] jepsen worker 13 - jepsen.util 13 :ok :read [4 22] INFO [2022-01-10 12:53:34,034] jepsen worker 34 - jepsen.util 214 :invoke :write [11 32] INFO [2022-01-10 12:53:34,034] jepsen worker 62 - jepsen.util 242 :ok :write [20 33] INFO [2022-01-10 12:53:34,035] jepsen worker 13 - jepsen.util 13 :invoke :write [4 2] INFO [2022-01-10 12:53:34,035] jepsen worker 62 - jepsen.util 242 :invoke :write [20 43] INFO [2022-01-10 12:53:34,039] jepsen worker 71 - jepsen.util 161 :ok :write [23 12] INFO [2022-01-10 12:53:34,040] jepsen worker 71 - jepsen.util 161 :invoke :write [23 41] INFO [2022-01-10 12:53:34,041] jepsen worker 7 - jepsen.util 7 :ok :write [2 0] INFO [2022-01-10 12:53:34,041] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:34,041] jepsen worker 51 - jepsen.util 141 :ok :write [17 22] INFO [2022-01-10 12:53:34,042] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:34,042] jepsen worker 7 - jepsen.util 7 :ok :read [2 0] INFO [2022-01-10 12:53:34,043] jepsen worker 7 - jepsen.util 7 :invoke :write [2 28] INFO [2022-01-10 12:53:34,044] jepsen worker 51 - jepsen.util 141 :ok :read [17 22] INFO [2022-01-10 12:53:34,044] jepsen worker 51 - jepsen.util 141 :invoke :write [17 20] INFO [2022-01-10 12:53:34,046] jepsen worker 22 - jepsen.util 112 :ok :write [7 31] INFO [2022-01-10 12:53:34,047] jepsen worker 22 - jepsen.util 112 :invoke :write [7 49] INFO [2022-01-10 12:53:34,053] jepsen worker 15 - jepsen.util 15 :ok :write [5 6] INFO [2022-01-10 12:53:34,053] jepsen worker 15 - jepsen.util 15 :invoke :write [5 30] INFO [2022-01-10 12:53:34,056] jepsen worker 50 - jepsen.util 50 :ok :write [16 40] INFO [2022-01-10 12:53:34,057] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:34,058] jepsen worker 50 - jepsen.util 50 :ok :read [16 40] INFO [2022-01-10 12:53:34,058] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:34,059] jepsen worker 44 - jepsen.util 44 :ok :write [14 4] INFO [2022-01-10 12:53:34,059] jepsen worker 50 - jepsen.util 50 :ok :read [16 40] INFO [2022-01-10 12:53:34,059] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:34,060] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:34,060] jepsen worker 44 - jepsen.util 44 :ok :read [14 4] INFO [2022-01-10 12:53:34,060] jepsen worker 50 - jepsen.util 50 :ok :read [16 40] INFO [2022-01-10 12:53:34,061] jepsen worker 44 - jepsen.util 44 :invoke :write [14 12] INFO [2022-01-10 12:53:34,061] jepsen worker 50 - jepsen.util 50 :invoke :write [16 5] INFO [2022-01-10 12:53:34,063] jepsen worker 83 - jepsen.util 263 :ok :write [27 2] INFO [2022-01-10 12:53:34,063] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:34,064] jepsen worker 83 - jepsen.util 263 :ok :read [27 2] INFO [2022-01-10 12:53:34,065] jepsen worker 83 - jepsen.util 263 :invoke :write [27 15] INFO [2022-01-10 12:53:34,065] jepsen worker 80 - jepsen.util 260 :ok :write [26 36] INFO [2022-01-10 12:53:34,066] jepsen worker 80 - jepsen.util 260 :invoke :write [26 40] INFO [2022-01-10 12:53:34,071] jepsen worker 75 - jepsen.util 165 :ok :write [25 5] INFO [2022-01-10 12:53:34,071] jepsen worker 75 - jepsen.util 165 :invoke :write [25 29] INFO [2022-01-10 12:53:34,074] jepsen worker 13 - jepsen.util 13 :ok :write [4 2] INFO [2022-01-10 12:53:34,075] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:34,076] jepsen worker 13 - jepsen.util 13 :ok :read [4 2] INFO [2022-01-10 12:53:34,076] jepsen worker 13 - jepsen.util 13 :invoke :write [4 41] INFO [2022-01-10 12:53:34,078] jepsen worker 62 - jepsen.util 242 :ok :write [20 43] INFO [2022-01-10 12:53:34,079] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:34,080] jepsen worker 62 - jepsen.util 242 :ok :read [20 43] INFO [2022-01-10 12:53:34,080] jepsen worker 62 - jepsen.util 242 :invoke :write [20 17] INFO [2022-01-10 12:53:34,096] jepsen worker 40 - jepsen.util 130 :ok :write [13 48] INFO [2022-01-10 12:53:34,097] jepsen worker 38 - jepsen.util 38 :ok :write [12 46] INFO [2022-01-10 12:53:34,097] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,098] jepsen worker 38 - jepsen.util 38 :invoke :write [12 21] INFO [2022-01-10 12:53:34,098] jepsen worker 40 - jepsen.util 130 :ok :read [13 48] INFO [2022-01-10 12:53:34,099] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,099] jepsen worker 40 - jepsen.util 130 :ok :read [13 48] INFO [2022-01-10 12:53:34,100] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,101] jepsen worker 40 - jepsen.util 130 :ok :read [13 48] INFO [2022-01-10 12:53:34,101] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,102] jepsen worker 1 - jepsen.util 91 :ok :write [0 48] INFO [2022-01-10 12:53:34,102] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,102] jepsen worker 40 - jepsen.util 130 :ok :read [13 48] INFO [2022-01-10 12:53:34,103] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,103] jepsen worker 1 - jepsen.util 91 :ok :read [0 48] INFO [2022-01-10 12:53:34,103] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,104] jepsen worker 40 - jepsen.util 130 :ok :read [13 48] INFO [2022-01-10 12:53:34,104] jepsen worker 40 - jepsen.util 130 :invoke :write [13 22] INFO [2022-01-10 12:53:34,104] jepsen worker 1 - jepsen.util 91 :ok :read [0 48] INFO [2022-01-10 12:53:34,105] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,105] jepsen worker 3 - jepsen.util 3 :ok :write [1 40] INFO [2022-01-10 12:53:34,105] jepsen worker 1 - jepsen.util 91 :ok :read [0 48] INFO [2022-01-10 12:53:34,106] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,106] jepsen worker 1 - jepsen.util 91 :invoke :write [0 17] INFO [2022-01-10 12:53:34,106] jepsen worker 3 - jepsen.util 3 :ok :read [1 40] INFO [2022-01-10 12:53:34,107] jepsen worker 3 - jepsen.util 3 :invoke :write [1 20] INFO [2022-01-10 12:53:34,109] jepsen worker 88 - jepsen.util 178 :ok :write [29 1] INFO [2022-01-10 12:53:34,109] jepsen worker 88 - jepsen.util 178 :invoke :write [29 24] INFO [2022-01-10 12:53:34,110] jepsen worker 86 - jepsen.util 176 :ok :write [28 27] INFO [2022-01-10 12:53:34,111] jepsen worker 86 - jepsen.util 176 :invoke :write [28 37] INFO [2022-01-10 12:53:34,114] jepsen worker 68 - jepsen.util 68 :ok :write [22 47] INFO [2022-01-10 12:53:34,114] jepsen worker 68 - jepsen.util 68 :invoke :write [22 34] INFO [2022-01-10 12:53:34,119] jepsen worker 34 - jepsen.util 214 :ok :write [11 32] INFO [2022-01-10 12:53:34,120] jepsen worker 34 - jepsen.util 214 :invoke :write [11 30] INFO [2022-01-10 12:53:34,125] jepsen worker 7 - jepsen.util 7 :ok :write [2 28] INFO [2022-01-10 12:53:34,125] jepsen worker 7 - jepsen.util 7 :invoke :write [2 35] INFO [2022-01-10 12:53:34,129] jepsen worker 15 - jepsen.util 15 :ok :write [5 30] INFO [2022-01-10 12:53:34,130] jepsen worker 15 - jepsen.util 15 :invoke :write [5 10] INFO [2022-01-10 12:53:34,133] jepsen worker 50 - jepsen.util 50 :ok :write [16 5] INFO [2022-01-10 12:53:34,133] jepsen worker 50 - jepsen.util 50 :invoke :write [16 22] INFO [2022-01-10 12:53:34,136] jepsen worker 83 - jepsen.util 263 :ok :write [27 15] INFO [2022-01-10 12:53:34,137] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:34,138] jepsen worker 83 - jepsen.util 263 :ok :read [27 15] INFO [2022-01-10 12:53:34,138] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:34,139] jepsen worker 83 - jepsen.util 263 :ok :read [27 15] INFO [2022-01-10 12:53:34,140] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:34,141] jepsen worker 83 - jepsen.util 263 :ok :read [27 15] INFO [2022-01-10 12:53:34,141] jepsen worker 83 - jepsen.util 263 :invoke :write [27 45] INFO [2022-01-10 12:53:34,152] jepsen worker 68 - jepsen.util 68 :ok :write [22 34] INFO [2022-01-10 12:53:34,152] jepsen worker 68 - jepsen.util 68 :invoke :write [22 0] INFO [2022-01-10 12:53:34,159] jepsen worker 34 - jepsen.util 214 :ok :write [11 30] INFO [2022-01-10 12:53:34,159] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:34,160] jepsen worker 34 - jepsen.util 214 :ok :read [11 30] INFO [2022-01-10 12:53:34,161] jepsen worker 7 - jepsen.util 7 :ok :write [2 35] INFO [2022-01-10 12:53:34,161] jepsen worker 34 - jepsen.util 214 :invoke :write [11 24] INFO [2022-01-10 12:53:34,162] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:34,162] jepsen worker 7 - jepsen.util 7 :ok :read [2 35] INFO [2022-01-10 12:53:34,163] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:34,164] jepsen worker 7 - jepsen.util 7 :ok :read [2 35] INFO [2022-01-10 12:53:34,165] jepsen worker 7 - jepsen.util 7 :invoke :write [2 30] INFO [2022-01-10 12:53:34,165] jepsen worker 15 - jepsen.util 15 :ok :write [5 10] INFO [2022-01-10 12:53:34,166] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:34,167] jepsen worker 15 - jepsen.util 15 :ok :read [5 10] INFO [2022-01-10 12:53:34,167] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:34,168] jepsen worker 15 - jepsen.util 15 :ok :read [5 10] INFO [2022-01-10 12:53:34,169] jepsen worker 15 - jepsen.util 15 :invoke :write [5 16] INFO [2022-01-10 12:53:34,170] jepsen worker 50 - jepsen.util 50 :ok :write [16 22] INFO [2022-01-10 12:53:34,171] jepsen worker 50 - jepsen.util 50 :invoke :write [16 4] INFO [2022-01-10 12:53:34,175] jepsen worker 83 - jepsen.util 263 :ok :write [27 45] INFO [2022-01-10 12:53:34,176] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:34,176] jepsen worker 83 - jepsen.util 263 :ok :read [27 45] INFO [2022-01-10 12:53:34,177] jepsen worker 83 - jepsen.util 263 :invoke :write [27 29] INFO [2022-01-10 12:53:34,181] jepsen worker 68 - jepsen.util 68 :ok :write [22 0] INFO [2022-01-10 12:53:34,182] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:34,182] jepsen worker 68 - jepsen.util 68 :ok :read [22 0] INFO [2022-01-10 12:53:34,183] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:34,184] jepsen worker 68 - jepsen.util 68 :ok :read [22 0] INFO [2022-01-10 12:53:34,184] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:34,185] jepsen worker 68 - jepsen.util 68 :ok :read [22 0] INFO [2022-01-10 12:53:34,186] jepsen worker 68 - jepsen.util 68 :invoke :write [22 46] INFO [2022-01-10 12:53:34,187] jepsen worker 34 - jepsen.util 214 :ok :write [11 24] INFO [2022-01-10 12:53:34,188] jepsen worker 34 - jepsen.util 214 :invoke :write [11 38] INFO [2022-01-10 12:53:34,194] jepsen worker 7 - jepsen.util 7 :ok :write [2 30] INFO [2022-01-10 12:53:34,194] jepsen worker 7 - jepsen.util 7 :invoke :write [2 15] INFO [2022-01-10 12:53:34,194] jepsen worker 15 - jepsen.util 15 :ok :write [5 16] INFO [2022-01-10 12:53:34,195] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:34,196] jepsen worker 15 - jepsen.util 15 :ok :read [5 16] INFO [2022-01-10 12:53:34,197] jepsen worker 15 - jepsen.util 15 :invoke :write [5 37] INFO [2022-01-10 12:53:34,201] jepsen worker 50 - jepsen.util 50 :ok :write [16 4] INFO [2022-01-10 12:53:34,202] jepsen worker 50 - jepsen.util 50 :invoke :write [16 11] INFO [2022-01-10 12:53:34,206] jepsen worker 83 - jepsen.util 263 :ok :write [27 29] INFO [2022-01-10 12:53:34,207] jepsen worker 83 - jepsen.util 263 :invoke :write [27 19] INFO [2022-01-10 12:53:34,212] jepsen worker 68 - jepsen.util 68 :ok :write [22 46] INFO [2022-01-10 12:53:34,213] jepsen worker 22 - jepsen.util 112 :ok :write [7 49] INFO [2022-01-10 12:53:34,213] jepsen worker 68 - jepsen.util 68 :invoke :write [22 22] INFO [2022-01-10 12:53:34,213] jepsen worker 71 - jepsen.util 161 :ok :write [23 41] INFO [2022-01-10 12:53:34,214] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,215] jepsen worker 22 - jepsen.util 112 :invoke :write [7 10] INFO [2022-01-10 12:53:34,215] jepsen worker 71 - jepsen.util 161 :ok :read [23 41] INFO [2022-01-10 12:53:34,216] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,216] jepsen worker 71 - jepsen.util 161 :ok :read [23 41] INFO [2022-01-10 12:53:34,217] jepsen worker 71 - jepsen.util 161 :invoke :write [23 47] INFO [2022-01-10 12:53:34,219] jepsen worker 51 - jepsen.util 141 :ok :write [17 20] INFO [2022-01-10 12:53:34,219] jepsen worker 44 - jepsen.util 44 :ok :write [14 12] INFO [2022-01-10 12:53:34,220] jepsen worker 34 - jepsen.util 214 :ok :write [11 38] INFO [2022-01-10 12:53:34,220] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:34,220] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:34,221] jepsen worker 51 - jepsen.util 141 :ok :read [17 20] INFO [2022-01-10 12:53:34,221] jepsen worker 44 - jepsen.util 44 :invoke :write [14 44] INFO [2022-01-10 12:53:34,221] jepsen worker 34 - jepsen.util 214 :ok :read [11 38] INFO [2022-01-10 12:53:34,222] jepsen worker 51 - jepsen.util 141 :invoke :write [17 41] INFO [2022-01-10 12:53:34,222] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:34,223] jepsen worker 34 - jepsen.util 214 :ok :read [11 38] INFO [2022-01-10 12:53:34,224] jepsen worker 7 - jepsen.util 7 :ok :write [2 15] INFO [2022-01-10 12:53:34,224] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:34,225] jepsen worker 7 - jepsen.util 7 :invoke :write [2 42] INFO [2022-01-10 12:53:34,225] jepsen worker 34 - jepsen.util 214 :ok :read [11 38] INFO [2022-01-10 12:53:34,226] jepsen worker 34 - jepsen.util 214 :invoke :write [11 4] INFO [2022-01-10 12:53:34,227] jepsen worker 15 - jepsen.util 15 :ok :write [5 37] INFO [2022-01-10 12:53:34,227] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:34,228] jepsen worker 80 - jepsen.util 260 :ok :write [26 40] INFO [2022-01-10 12:53:34,228] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:34,229] jepsen worker 15 - jepsen.util 15 :ok :read [5 37] INFO [2022-01-10 12:53:34,229] jepsen worker 80 - jepsen.util 260 :ok :read [26 40] INFO [2022-01-10 12:53:34,229] jepsen worker 15 - jepsen.util 15 :invoke :write [5 14] INFO [2022-01-10 12:53:34,230] jepsen worker 80 - jepsen.util 260 :invoke :write [26 25] INFO [2022-01-10 12:53:34,230] jepsen worker 75 - jepsen.util 165 :ok :write [25 29] INFO [2022-01-10 12:53:34,231] jepsen worker 75 - jepsen.util 165 :invoke :write [25 23] INFO [2022-01-10 12:53:34,231] jepsen worker 62 - jepsen.util 242 :ok :write [20 17] INFO [2022-01-10 12:53:34,232] jepsen worker 62 - jepsen.util 242 :invoke :write [20 8] INFO [2022-01-10 12:53:34,233] jepsen worker 50 - jepsen.util 50 :ok :write [16 11] INFO [2022-01-10 12:53:34,234] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:34,235] jepsen worker 50 - jepsen.util 50 :ok :read [16 11] INFO [2022-01-10 12:53:34,236] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:34,236] jepsen worker 13 - jepsen.util 13 :ok :write [4 41] INFO [2022-01-10 12:53:34,237] jepsen worker 50 - jepsen.util 50 :ok :read [16 11] INFO [2022-01-10 12:53:34,237] jepsen worker 13 - jepsen.util 13 :invoke :write [4 41] INFO [2022-01-10 12:53:34,237] jepsen worker 40 - jepsen.util 130 :ok :write [13 22] INFO [2022-01-10 12:53:34,237] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:34,238] jepsen worker 40 - jepsen.util 130 :invoke :write [13 47] INFO [2022-01-10 12:53:34,238] jepsen worker 50 - jepsen.util 50 :ok :read [16 11] INFO [2022-01-10 12:53:34,239] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:34,240] jepsen worker 83 - jepsen.util 263 :ok :write [27 19] INFO [2022-01-10 12:53:34,240] jepsen worker 50 - jepsen.util 50 :ok :read [16 11] INFO [2022-01-10 12:53:34,240] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:34,241] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:34,242] jepsen worker 50 - jepsen.util 50 :ok :read [16 11] INFO [2022-01-10 12:53:34,242] jepsen worker 83 - jepsen.util 263 :ok :read [27 19] INFO [2022-01-10 12:53:34,242] jepsen worker 38 - jepsen.util 38 :ok :write [12 21] INFO [2022-01-10 12:53:34,242] jepsen worker 1 - jepsen.util 91 :ok :write [0 17] INFO [2022-01-10 12:53:34,242] jepsen worker 50 - jepsen.util 50 :invoke :write [16 39] INFO [2022-01-10 12:53:34,243] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,244] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:34,244] jepsen worker 1 - jepsen.util 91 :ok :read [0 17] INFO [2022-01-10 12:53:34,244] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:34,245] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,245] jepsen worker 83 - jepsen.util 263 :ok :read [27 19] INFO [2022-01-10 12:53:34,246] jepsen worker 88 - jepsen.util 178 :ok :write [29 24] INFO [2022-01-10 12:53:34,246] jepsen worker 1 - jepsen.util 91 :ok :read [0 17] INFO [2022-01-10 12:53:34,246] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] WARN [2022-01-10 12:53:34,247] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":655,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":20568,"total_server_duration_us":4032,"encode_duration_us":19,"operation_id":"0x27c2","timeout_ms":10000,"last_server_duration_us":135,"total_duration_us":10010336},{"operation_name":"upsert","last_dispatch_duration_us":681,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":21566,"total_server_duration_us":6659,"encode_duration_us":29,"operation_id":"0x2bed","timeout_ms":10000,"last_server_duration_us":108,"total_duration_us":10007734},{"operation_name":"upsert","last_dispatch_duration_us":544,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":6759497,"total_server_duration_us":1342,"encode_duration_us":25,"operation_id":"0x340c","timeout_ms":10000,"last_server_duration_us":56,"total_duration_us":10005207},{"operation_name":"upsert","last_dispatch_duration_us":26888,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000001/000000005D716E8E","last_local_socket":"172.23.104.79:34680","total_dispatch_duration_us":43955,"total_server_duration_us":28796,"encode_duration_us":20,"operation_id":"0x2fa3","timeout_ms":10000,"last_server_duration_us":26241,"total_duration_us":8832102},{"operation_name":"upsert","last_dispatch_duration_us":37045,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000001/00000000100998FD","last_local_socket":"172.23.104.79:58632","total_dispatch_duration_us":52911,"total_server_duration_us":39133,"encode_duration_us":29,"operation_id":"0x372a","timeout_ms":10000,"last_server_duration_us":36322,"total_duration_us":6739876},{"operation_name":"upsert","last_dispatch_duration_us":30562,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000001/00000000100998FD","last_local_socket":"172.23.104.79:58632","total_dispatch_duration_us":43118,"total_server_duration_us":31625,"encode_duration_us":17,"operation_id":"0x36ec","timeout_ms":10000,"last_server_duration_us":29975,"total_duration_us":6725433},{"operation_name":"upsert","last_dispatch_duration_us":23654,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000001/000000005D716E8E","last_local_socket":"172.23.104.79:34680","total_dispatch_duration_us":42098,"total_server_duration_us":30047,"encode_duration_us":20,"operation_id":"0x30f5","timeout_ms":10000,"last_server_duration_us":23054,"total_duration_us":6718687},{"operation_name":"upsert","last_dispatch_duration_us":159501,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000001/00000000100998FD","last_local_socket":"172.23.104.79:58632","total_dispatch_duration_us":259155,"total_server_duration_us":160575,"encode_duration_us":28,"operation_id":"0x3179","timeout_ms":10000,"last_server_duration_us":158779,"total_duration_us":5914868},{"operation_name":"upsert","last_dispatch_duration_us":405925,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":416278,"total_server_duration_us":363276,"encode_duration_us":35,"operation_id":"0x325d","timeout_ms":10000,"last_server_duration_us":361821,"total_duration_us":4555220},{"operation_name":"upsert","last_dispatch_duration_us":177106,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":185322,"total_server_duration_us":178225,"encode_duration_us":22,"operation_id":"0x3927","timeout_ms":10000,"last_server_duration_us":176692,"total_duration_us":2791820}],"total_count":24}} WARN [2022-01-10 12:53:34,247] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":8758689,"last_local_id":"2B94F89700000001/0000000023B95902","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":8758689,"last_local_socket":"172.23.104.79:40178","total_server_duration_us":15,"operation_id":"0x30e6","timeout_ms":2000,"last_server_duration_us":15,"total_duration_us":2002139},{"operation_name":"get","last_dispatch_duration_us":7981332,"last_local_id":"2B94F89700000001/0000000023B95902","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":7981332,"last_local_socket":"172.23.104.79:40178","total_server_duration_us":11,"operation_id":"0x3263","timeout_ms":2000,"last_server_duration_us":11,"total_duration_us":2005166},{"operation_name":"get","last_dispatch_duration_us":9008972,"last_local_id":"2B94F89700000001/0000000023B95902","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":9008972,"last_local_socket":"172.23.104.79:40178","total_server_duration_us":38,"operation_id":"0x307e","timeout_ms":2000,"last_server_duration_us":38,"total_duration_us":2003183},{"operation_name":"get","last_dispatch_duration_us":2983917,"last_local_id":"2B94F89700000001/0000000023B95902","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":2983917,"last_local_socket":"172.23.104.79:40178","total_server_duration_us":23,"operation_id":"0x3aa7","timeout_ms":2000,"last_server_duration_us":23,"total_duration_us":2007246},{"operation_name":"get","last_dispatch_duration_us":7003705,"last_local_id":"2B94F89700000001/0000000023B95902","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":7003705,"last_local_socket":"172.23.104.79:40178","total_server_duration_us":11,"operation_id":"0x33e2","timeout_ms":2000,"last_server_duration_us":11,"total_duration_us":2007502},{"operation_name":"get","last_dispatch_duration_us":5973946,"last_local_id":"2B94F89700000001/0000000023B95902","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":5973946,"last_local_socket":"172.23.104.79:40178","total_server_duration_us":15,"operation_id":"0x358a","timeout_ms":2000,"last_server_duration_us":15,"total_duration_us":2007137},{"operation_name":"get","last_dispatch_duration_us":4994170,"last_local_id":"2B94F89700000001/0000000023B95902","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":4994170,"last_local_socket":"172.23.104.79:40178","total_server_duration_us":11,"operation_id":"0x3728","timeout_ms":2000,"last_server_duration_us":11,"total_duration_us":2008202},{"operation_name":"upsert","last_dispatch_duration_us":12855940,"last_local_id":"2B94F89700000001/0000000023B95902","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":12867470,"last_local_socket":"172.23.104.79:40178","total_server_duration_us":9012454,"encode_duration_us":20,"operation_id":"0x20f3","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10007685}],"total_count":8}} INFO [2022-01-10 12:53:34,247] jepsen worker 83 - jepsen.util 263 :ok :read [27 19] INFO [2022-01-10 12:53:34,247] jepsen worker 88 - jepsen.util 178 :invoke :write [29 23] INFO [2022-01-10 12:53:34,248] jepsen worker 1 - jepsen.util 91 :invoke :write [0 40] INFO [2022-01-10 12:53:34,248] jepsen worker 68 - jepsen.util 68 :ok :write [22 22] INFO [2022-01-10 12:53:34,248] jepsen worker 3 - jepsen.util 3 :ok :write [1 20] INFO [2022-01-10 12:53:34,249] jepsen worker 83 - jepsen.util 263 :invoke :write [27 20] INFO [2022-01-10 12:53:34,249] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,250] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:34,250] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:53:34,251] jepsen worker 68 - jepsen.util 68 :ok :read [22 22] INFO [2022-01-10 12:53:34,251] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,251] jepsen worker 68 - jepsen.util 68 :invoke :write [22 14] INFO [2022-01-10 12:53:34,252] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:53:34,252] jepsen worker 22 - jepsen.util 112 :ok :write [7 10] INFO [2022-01-10 12:53:34,252] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:34,253] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,253] jepsen worker 22 - jepsen.util 112 :ok :read [7 10] INFO [2022-01-10 12:53:34,254] jepsen worker 3 - jepsen.util 3 :ok :read [1 20] INFO [2022-01-10 12:53:34,254] jepsen worker 86 - jepsen.util 176 :ok :write [28 37] INFO [2022-01-10 12:53:34,254] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:34,255] jepsen worker 3 - jepsen.util 3 :invoke :write [1 46] INFO [2022-01-10 12:53:34,255] jepsen worker 22 - jepsen.util 112 :ok :read [7 10] INFO [2022-01-10 12:53:34,255] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:34,256] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:34,256] jepsen worker 86 - jepsen.util 176 :ok :read [28 37] INFO [2022-01-10 12:53:34,257] jepsen worker 22 - jepsen.util 112 :ok :read [7 10] INFO [2022-01-10 12:53:34,257] jepsen worker 71 - jepsen.util 161 :ok :write [23 47] INFO [2022-01-10 12:53:34,257] jepsen worker 86 - jepsen.util 176 :invoke :write [28 45] INFO [2022-01-10 12:53:34,258] jepsen worker 71 - jepsen.util 161 :invoke :write [23 42] INFO [2022-01-10 12:53:34,258] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:34,258] jepsen worker 44 - jepsen.util 44 :ok :write [14 44] INFO [2022-01-10 12:53:34,259] jepsen worker 44 - jepsen.util 44 :invoke :write [14 21] INFO [2022-01-10 12:53:34,259] jepsen worker 22 - jepsen.util 112 :ok :read [7 10] INFO [2022-01-10 12:53:34,260] jepsen worker 22 - jepsen.util 112 :invoke :write [7 6] INFO [2022-01-10 12:53:34,263] jepsen worker 51 - jepsen.util 141 :ok :write [17 41] INFO [2022-01-10 12:53:34,264] jepsen worker 51 - jepsen.util 141 :invoke :write [17 38] INFO [2022-01-10 12:53:34,276] jepsen worker 40 - jepsen.util 130 :ok :write [13 47] INFO [2022-01-10 12:53:34,276] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:34,276] jepsen worker 40 - jepsen.util 130 :invoke :write [13 47] INFO [2022-01-10 12:53:34,277] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,278] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:34,278] jepsen worker 38 - jepsen.util 38 :invoke :write [12 39] INFO [2022-01-10 12:53:34,281] jepsen worker 3 - jepsen.util 3 :ok :write [1 46] INFO [2022-01-10 12:53:34,282] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,283] jepsen worker 3 - jepsen.util 3 :ok :read [1 46] INFO [2022-01-10 12:53:34,283] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,284] jepsen worker 3 - jepsen.util 3 :ok :read [1 46] INFO [2022-01-10 12:53:34,284] jepsen worker 3 - jepsen.util 3 :invoke :write [1 8] INFO [2022-01-10 12:53:34,285] jepsen worker 1 - jepsen.util 91 :ok :write [0 40] INFO [2022-01-10 12:53:34,285] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,286] jepsen worker 1 - jepsen.util 91 :ok :read [0 40] INFO [2022-01-10 12:53:34,287] jepsen worker 1 - jepsen.util 91 :invoke :write [0 16] INFO [2022-01-10 12:53:34,288] jepsen worker 86 - jepsen.util 176 :ok :write [28 45] INFO [2022-01-10 12:53:34,289] jepsen worker 86 - jepsen.util 176 :invoke :write [28 23] INFO [2022-01-10 12:53:34,295] jepsen worker 87 - jepsen.util 87 :ok :write [29 17] INFO [2022-01-10 12:53:34,295] jepsen worker 87 - jepsen.util 87 :invoke :write [29 37] INFO [2022-01-10 12:53:34,303] jepsen worker 40 - jepsen.util 130 :ok :write [13 47] INFO [2022-01-10 12:53:34,304] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,304] jepsen worker 38 - jepsen.util 38 :ok :write [12 39] INFO [2022-01-10 12:53:34,304] jepsen worker 40 - jepsen.util 130 :ok :read [13 47] INFO [2022-01-10 12:53:34,304] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,305] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,305] jepsen worker 38 - jepsen.util 38 :ok :read [12 39] INFO [2022-01-10 12:53:34,305] jepsen worker 40 - jepsen.util 130 :ok :read [13 47] INFO [2022-01-10 12:53:34,306] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,306] jepsen worker 40 - jepsen.util 130 :invoke :write [13 6] INFO [2022-01-10 12:53:34,307] jepsen worker 38 - jepsen.util 38 :ok :read [12 39] INFO [2022-01-10 12:53:34,307] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,308] jepsen worker 38 - jepsen.util 38 :ok :read [12 39] INFO [2022-01-10 12:53:34,308] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,309] jepsen worker 3 - jepsen.util 3 :ok :write [1 8] INFO [2022-01-10 12:53:34,310] jepsen worker 38 - jepsen.util 38 :ok :read [12 39] INFO [2022-01-10 12:53:34,310] jepsen worker 3 - jepsen.util 3 :invoke :write [1 30] INFO [2022-01-10 12:53:34,310] jepsen worker 38 - jepsen.util 38 :invoke :write [12 21] INFO [2022-01-10 12:53:34,316] jepsen worker 1 - jepsen.util 91 :ok :write [0 16] INFO [2022-01-10 12:53:34,317] jepsen worker 86 - jepsen.util 176 :ok :write [28 23] INFO [2022-01-10 12:53:34,317] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,318] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:34,318] jepsen worker 1 - jepsen.util 91 :ok :read [0 16] INFO [2022-01-10 12:53:34,319] jepsen worker 86 - jepsen.util 176 :ok :read [28 23] INFO [2022-01-10 12:53:34,319] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,320] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:34,320] jepsen worker 1 - jepsen.util 91 :ok :read [0 16] INFO [2022-01-10 12:53:34,320] jepsen worker 86 - jepsen.util 176 :ok :read [28 23] INFO [2022-01-10 12:53:34,321] jepsen worker 1 - jepsen.util 91 :invoke :write [0 48] INFO [2022-01-10 12:53:34,321] jepsen worker 86 - jepsen.util 176 :invoke :write [28 13] INFO [2022-01-10 12:53:34,326] jepsen worker 88 - jepsen.util 178 :ok :write [29 23] INFO [2022-01-10 12:53:34,326] jepsen worker 3 - jepsen.util 3 :ok :write [1 30] INFO [2022-01-10 12:53:34,326] jepsen worker 88 - jepsen.util 178 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,327] jepsen worker 3 - jepsen.util 3 :invoke :write [1 36] INFO [2022-01-10 12:53:34,327] jepsen worker 34 - jepsen.util 214 :ok :write [11 4] INFO [2022-01-10 12:53:34,327] jepsen worker 88 - jepsen.util 178 :ok :read [29 23] INFO [2022-01-10 12:53:34,328] jepsen worker 88 - jepsen.util 178 :invoke :write [29 19] INFO [2022-01-10 12:53:34,328] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:34,329] jepsen worker 34 - jepsen.util 214 :ok :read [11 4] INFO [2022-01-10 12:53:34,330] jepsen worker 34 - jepsen.util 214 :invoke :write [11 28] INFO [2022-01-10 12:53:34,333] jepsen worker 40 - jepsen.util 130 :ok :write [13 6] INFO [2022-01-10 12:53:34,334] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,334] jepsen worker 80 - jepsen.util 260 :ok :write [26 25] INFO [2022-01-10 12:53:34,334] jepsen worker 7 - jepsen.util 7 :ok :write [2 42] INFO [2022-01-10 12:53:34,335] jepsen worker 40 - jepsen.util 130 :ok :read [13 6] INFO [2022-01-10 12:53:34,335] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:34,335] jepsen worker 38 - jepsen.util 38 :ok :write [12 21] INFO [2022-01-10 12:53:34,336] jepsen worker 7 - jepsen.util 7 :ok :read [2 42] INFO [2022-01-10 12:53:34,336] jepsen worker 40 - jepsen.util 130 :invoke :write [13 11] INFO [2022-01-10 12:53:34,337] jepsen worker 7 - jepsen.util 7 :invoke :write [2 48] INFO [2022-01-10 12:53:34,338] jepsen worker 80 - jepsen.util 260 :invoke :write [26 6] INFO [2022-01-10 12:53:34,338] jepsen worker 38 - jepsen.util 38 :invoke :write [12 47] INFO [2022-01-10 12:53:34,340] jepsen worker 75 - jepsen.util 165 :ok :write [25 23] INFO [2022-01-10 12:53:34,341] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:34,341] jepsen worker 13 - jepsen.util 13 :ok :write [4 41] INFO [2022-01-10 12:53:34,341] jepsen worker 86 - jepsen.util 176 :ok :write [28 13] INFO [2022-01-10 12:53:34,341] jepsen worker 1 - jepsen.util 91 :ok :write [0 48] INFO [2022-01-10 12:53:34,341] jepsen worker 75 - jepsen.util 165 :ok :read [25 23] INFO [2022-01-10 12:53:34,343] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:34,343] jepsen worker 13 - jepsen.util 13 :invoke :write [4 0] INFO [2022-01-10 12:53:34,344] jepsen worker 86 - jepsen.util 176 :ok :read [28 13] INFO [2022-01-10 12:53:34,344] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:34,345] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,345] jepsen worker 71 - jepsen.util 161 :ok :write [23 42] INFO [2022-01-10 12:53:34,345] jepsen worker 75 - jepsen.util 165 :ok :read [25 23] INFO [2022-01-10 12:53:34,346] jepsen worker 86 - jepsen.util 176 :invoke :write [28 25] INFO [2022-01-10 12:53:34,346] jepsen worker 1 - jepsen.util 91 :ok :read [0 48] INFO [2022-01-10 12:53:34,347] jepsen worker 71 - jepsen.util 161 :invoke :write [23 42] INFO [2022-01-10 12:53:34,347] jepsen worker 15 - jepsen.util 15 :ok :write [5 14] INFO [2022-01-10 12:53:34,347] jepsen worker 1 - jepsen.util 91 :invoke :write [0 29] INFO [2022-01-10 12:53:34,348] jepsen worker 75 - jepsen.util 165 :invoke :write [25 12] INFO [2022-01-10 12:53:34,348] jepsen worker 87 - jepsen.util 87 :ok :write [29 37] INFO [2022-01-10 12:53:34,349] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:34,349] jepsen worker 87 - jepsen.util 87 :invoke :write [29 9] INFO [2022-01-10 12:53:34,350] jepsen worker 15 - jepsen.util 15 :ok :read [5 14] INFO [2022-01-10 12:53:34,350] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:34,351] jepsen worker 51 - jepsen.util 141 :ok :write [17 38] INFO [2022-01-10 12:53:34,351] jepsen worker 15 - jepsen.util 15 :ok :read [5 14] INFO [2022-01-10 12:53:34,351] jepsen worker 51 - jepsen.util 141 :invoke :write [17 35] INFO [2022-01-10 12:53:34,352] jepsen worker 62 - jepsen.util 242 :ok :write [20 8] INFO [2022-01-10 12:53:34,352] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:34,353] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:34,353] jepsen worker 15 - jepsen.util 15 :ok :read [5 14] INFO [2022-01-10 12:53:34,353] jepsen worker 15 - jepsen.util 15 :invoke :write [5 2] INFO [2022-01-10 12:53:34,354] jepsen worker 62 - jepsen.util 242 :ok :read [20 8] INFO [2022-01-10 12:53:34,354] jepsen worker 3 - jepsen.util 3 :ok :write [1 36] INFO [2022-01-10 12:53:34,354] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:34,355] jepsen worker 3 - jepsen.util 3 :invoke :write [1 36] INFO [2022-01-10 12:53:34,355] jepsen worker 62 - jepsen.util 242 :ok :read [20 8] INFO [2022-01-10 12:53:34,356] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:34,356] jepsen worker 62 - jepsen.util 242 :ok :read [20 8] INFO [2022-01-10 12:53:34,357] jepsen worker 62 - jepsen.util 242 :invoke :write [20 11] INFO [2022-01-10 12:53:34,358] jepsen worker 80 - jepsen.util 260 :ok :write [26 6] INFO [2022-01-10 12:53:34,359] jepsen worker 80 - jepsen.util 260 :invoke :write [26 20] INFO [2022-01-10 12:53:34,360] jepsen worker 50 - jepsen.util 50 :ok :write [16 39] INFO [2022-01-10 12:53:34,360] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:34,361] jepsen worker 40 - jepsen.util 130 :ok :write [13 11] INFO [2022-01-10 12:53:34,361] jepsen worker 38 - jepsen.util 38 :ok :write [12 47] INFO [2022-01-10 12:53:34,361] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,362] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,362] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:34,362] jepsen worker 40 - jepsen.util 130 :ok :read [13 11] INFO [2022-01-10 12:53:34,362] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:34,363] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,363] jepsen worker 13 - jepsen.util 13 :ok :write [4 0] INFO [2022-01-10 12:53:34,363] jepsen worker 38 - jepsen.util 38 :ok :read [12 47] INFO [2022-01-10 12:53:34,363] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:34,364] jepsen worker 40 - jepsen.util 130 :ok :read [13 11] INFO [2022-01-10 12:53:34,364] jepsen worker 13 - jepsen.util 13 :invoke :write [4 22] INFO [2022-01-10 12:53:34,364] jepsen worker 86 - jepsen.util 176 :ok :write [28 25] INFO [2022-01-10 12:53:34,365] jepsen worker 40 - jepsen.util 130 :invoke :write [13 21] INFO [2022-01-10 12:53:34,365] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:34,366] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,366] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:34,367] jepsen worker 86 - jepsen.util 176 :invoke :write [28 36] INFO [2022-01-10 12:53:34,367] jepsen worker 83 - jepsen.util 263 :ok :write [27 20] INFO [2022-01-10 12:53:34,367] jepsen worker 50 - jepsen.util 50 :invoke :write [16 12] INFO [2022-01-10 12:53:34,368] jepsen worker 71 - jepsen.util 161 :ok :write [23 42] INFO [2022-01-10 12:53:34,368] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:34,369] jepsen worker 71 - jepsen.util 161 :invoke :write [23 9] INFO [2022-01-10 12:53:34,369] jepsen worker 83 - jepsen.util 263 :ok :read [27 20] INFO [2022-01-10 12:53:34,369] jepsen worker 38 - jepsen.util 38 :ok :read [12 47] INFO [2022-01-10 12:53:34,369] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:34,370] jepsen worker 38 - jepsen.util 38 :invoke :write [12 23] INFO [2022-01-10 12:53:34,370] jepsen worker 83 - jepsen.util 263 :ok :read [27 20] INFO [2022-01-10 12:53:34,371] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:34,372] jepsen worker 83 - jepsen.util 263 :ok :read [27 20] INFO [2022-01-10 12:53:34,372] jepsen worker 68 - jepsen.util 68 :ok :write [22 14] INFO [2022-01-10 12:53:34,373] jepsen worker 83 - jepsen.util 263 :invoke :write [27 22] INFO [2022-01-10 12:53:34,373] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:34,374] jepsen worker 51 - jepsen.util 141 :ok :write [17 35] INFO [2022-01-10 12:53:34,374] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:34,374] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:34,375] jepsen worker 68 - jepsen.util 68 :invoke :write [22 14] INFO [2022-01-10 12:53:34,375] jepsen worker 51 - jepsen.util 141 :ok :read [17 35] INFO [2022-01-10 12:53:34,376] jepsen worker 51 - jepsen.util 141 :invoke :write [17 11] INFO [2022-01-10 12:53:34,377] jepsen worker 22 - jepsen.util 112 :ok :write [7 6] INFO [2022-01-10 12:53:34,378] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:34,378] jepsen worker 22 - jepsen.util 112 :ok :read [7 6] INFO [2022-01-10 12:53:34,379] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:34,380] jepsen worker 22 - jepsen.util 112 :ok :read [7 6] INFO [2022-01-10 12:53:34,380] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:34,381] jepsen worker 22 - jepsen.util 112 :ok :read [7 6] INFO [2022-01-10 12:53:34,381] jepsen worker 80 - jepsen.util 260 :ok :write [26 20] INFO [2022-01-10 12:53:34,381] jepsen worker 22 - jepsen.util 112 :invoke :write [7 11] INFO [2022-01-10 12:53:34,382] jepsen worker 80 - jepsen.util 260 :invoke :write [26 35] INFO [2022-01-10 12:53:34,389] jepsen worker 44 - jepsen.util 44 :ok :write [14 21] INFO [2022-01-10 12:53:34,389] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:34,390] jepsen worker 44 - jepsen.util 44 :ok :read [14 21] INFO [2022-01-10 12:53:34,390] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:34,391] jepsen worker 13 - jepsen.util 13 :ok :write [4 22] INFO [2022-01-10 12:53:34,391] jepsen worker 44 - jepsen.util 44 :ok :read [14 21] INFO [2022-01-10 12:53:34,391] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:34,392] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:34,392] jepsen worker 13 - jepsen.util 13 :ok :read [4 22] INFO [2022-01-10 12:53:34,393] jepsen worker 44 - jepsen.util 44 :ok :read [14 21] INFO [2022-01-10 12:53:34,393] jepsen worker 13 - jepsen.util 13 :invoke :write [4 1] INFO [2022-01-10 12:53:34,393] jepsen worker 44 - jepsen.util 44 :invoke :write [14 46] INFO [2022-01-10 12:53:34,396] jepsen worker 34 - jepsen.util 214 :ok :write [11 28] INFO [2022-01-10 12:53:34,396] jepsen worker 34 - jepsen.util 214 :invoke :write [11 24] INFO [2022-01-10 12:53:34,402] jepsen worker 71 - jepsen.util 161 :ok :write [23 9] INFO [2022-01-10 12:53:34,402] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,403] jepsen worker 71 - jepsen.util 161 :ok :read [23 9] INFO [2022-01-10 12:53:34,403] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,404] jepsen worker 71 - jepsen.util 161 :ok :read [23 9] INFO [2022-01-10 12:53:34,405] jepsen worker 71 - jepsen.util 161 :invoke :write [23 46] INFO [2022-01-10 12:53:34,407] jepsen worker 51 - jepsen.util 141 :ok :write [17 11] INFO [2022-01-10 12:53:34,407] jepsen worker 51 - jepsen.util 141 :invoke :write [17 34] INFO [2022-01-10 12:53:34,412] jepsen worker 75 - jepsen.util 165 :ok :write [25 12] INFO [2022-01-10 12:53:34,412] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:34,414] jepsen worker 75 - jepsen.util 165 :ok :read [25 12] INFO [2022-01-10 12:53:34,414] jepsen worker 80 - jepsen.util 260 :ok :write [26 35] INFO [2022-01-10 12:53:34,414] jepsen worker 80 - jepsen.util 260 :invoke :write [26 47] INFO [2022-01-10 12:53:34,415] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:34,415] jepsen worker 75 - jepsen.util 165 :ok :read [25 12] INFO [2022-01-10 12:53:34,416] jepsen worker 75 - jepsen.util 165 :invoke :write [25 49] INFO [2022-01-10 12:53:34,420] jepsen worker 13 - jepsen.util 13 :ok :write [4 1] INFO [2022-01-10 12:53:34,421] jepsen worker 13 - jepsen.util 13 :invoke :write [4 46] INFO [2022-01-10 12:53:34,427] jepsen worker 62 - jepsen.util 242 :ok :write [20 11] INFO [2022-01-10 12:53:34,428] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:34,428] jepsen worker 62 - jepsen.util 242 :ok :read [20 11] INFO [2022-01-10 12:53:34,429] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:34,430] jepsen worker 71 - jepsen.util 161 :ok :write [23 46] INFO [2022-01-10 12:53:34,430] jepsen worker 62 - jepsen.util 242 :ok :read [20 11] INFO [2022-01-10 12:53:34,430] jepsen worker 71 - jepsen.util 161 :invoke :write [23 32] INFO [2022-01-10 12:53:34,431] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:34,431] jepsen worker 62 - jepsen.util 242 :ok :read [20 11] INFO [2022-01-10 12:53:34,432] jepsen worker 62 - jepsen.util 242 :invoke :write [20 18] INFO [2022-01-10 12:53:34,436] jepsen worker 51 - jepsen.util 141 :ok :write [17 34] INFO [2022-01-10 12:53:34,436] jepsen worker 51 - jepsen.util 141 :invoke :write [17 23] INFO [2022-01-10 12:53:34,442] jepsen worker 80 - jepsen.util 260 :ok :write [26 47] INFO [2022-01-10 12:53:34,442] jepsen worker 80 - jepsen.util 260 :invoke :write [26 11] INFO [2022-01-10 12:53:34,447] jepsen worker 13 - jepsen.util 13 :ok :write [4 46] INFO [2022-01-10 12:53:34,448] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:34,449] jepsen worker 13 - jepsen.util 13 :ok :read [4 46] INFO [2022-01-10 12:53:34,449] jepsen worker 13 - jepsen.util 13 :invoke :write [4 29] INFO [2022-01-10 12:53:34,451] jepsen worker 22 - jepsen.util 112 :ok :write [7 11] INFO [2022-01-10 12:53:34,452] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:34,452] jepsen worker 71 - jepsen.util 161 :ok :write [23 32] INFO [2022-01-10 12:53:34,453] jepsen worker 22 - jepsen.util 112 :ok :read [7 11] INFO [2022-01-10 12:53:34,453] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,453] jepsen worker 22 - jepsen.util 112 :invoke :write [7 23] INFO [2022-01-10 12:53:34,454] jepsen worker 71 - jepsen.util 161 :ok :read [23 32] INFO [2022-01-10 12:53:34,454] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,455] jepsen worker 71 - jepsen.util 161 :ok :read [23 32] INFO [2022-01-10 12:53:34,456] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,456] jepsen worker 71 - jepsen.util 161 :ok :read [23 32] INFO [2022-01-10 12:53:34,457] jepsen worker 51 - jepsen.util 141 :ok :write [17 23] INFO [2022-01-10 12:53:34,457] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,457] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:34,458] jepsen worker 71 - jepsen.util 161 :ok :read [23 32] INFO [2022-01-10 12:53:34,458] jepsen worker 44 - jepsen.util 44 :ok :write [14 46] INFO [2022-01-10 12:53:34,458] jepsen worker 51 - jepsen.util 141 :ok :read [17 23] INFO [2022-01-10 12:53:34,459] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:34,459] jepsen worker 44 - jepsen.util 44 :ok :read [14 46] INFO [2022-01-10 12:53:34,459] jepsen worker 71 - jepsen.util 161 :invoke :write [23 6] INFO [2022-01-10 12:53:34,460] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:34,460] jepsen worker 51 - jepsen.util 141 :ok :read [17 23] INFO [2022-01-10 12:53:34,460] jepsen worker 44 - jepsen.util 44 :invoke :write [14 1] INFO [2022-01-10 12:53:34,461] jepsen worker 80 - jepsen.util 260 :ok :write [26 11] INFO [2022-01-10 12:53:34,461] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:34,462] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:34,462] jepsen worker 51 - jepsen.util 141 :ok :read [17 23] INFO [2022-01-10 12:53:34,463] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:34,463] jepsen worker 80 - jepsen.util 260 :ok :read [26 11] INFO [2022-01-10 12:53:34,463] jepsen worker 80 - jepsen.util 260 :invoke :write [26 39] INFO [2022-01-10 12:53:34,463] jepsen worker 51 - jepsen.util 141 :ok :read [17 23] INFO [2022-01-10 12:53:34,464] jepsen worker 51 - jepsen.util 141 :invoke :write [17 31] INFO [2022-01-10 12:53:34,469] jepsen worker 75 - jepsen.util 165 :ok :write [25 49] INFO [2022-01-10 12:53:34,470] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:34,470] jepsen worker 75 - jepsen.util 165 :ok :read [25 49] INFO [2022-01-10 12:53:34,471] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:34,471] jepsen worker 75 - jepsen.util 165 :ok :read [25 49] INFO [2022-01-10 12:53:34,472] jepsen worker 75 - jepsen.util 165 :invoke :write [25 12] INFO [2022-01-10 12:53:34,476] jepsen worker 62 - jepsen.util 242 :ok :write [20 18] INFO [2022-01-10 12:53:34,477] jepsen worker 62 - jepsen.util 242 :invoke :write [20 25] INFO [2022-01-10 12:53:34,512] jepsen worker 7 - jepsen.util 7 :ok :write [2 48] INFO [2022-01-10 12:53:34,513] jepsen worker 7 - jepsen.util 7 :invoke :write [2 16] INFO [2022-01-10 12:53:34,513] jepsen worker 3 - jepsen.util 3 :ok :write [1 36] INFO [2022-01-10 12:53:34,514] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,514] jepsen worker 3 - jepsen.util 3 :ok :read [1 36] INFO [2022-01-10 12:53:34,515] jepsen worker 3 - jepsen.util 3 :invoke :write [1 6] INFO [2022-01-10 12:53:34,518] jepsen worker 1 - jepsen.util 91 :ok :write [0 29] INFO [2022-01-10 12:53:34,518] jepsen worker 38 - jepsen.util 38 :ok :write [12 23] INFO [2022-01-10 12:53:34,518] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,519] jepsen worker 1 - jepsen.util 91 :invoke :write [0 5] INFO [2022-01-10 12:53:34,520] jepsen worker 38 - jepsen.util 38 :ok :read [12 23] INFO [2022-01-10 12:53:34,520] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:34,525] jepsen worker 15 - jepsen.util 15 :ok :write [5 2] INFO [2022-01-10 12:53:34,525] jepsen worker 86 - jepsen.util 176 :ok :write [28 36] INFO [2022-01-10 12:53:34,526] jepsen worker 15 - jepsen.util 15 :invoke :write [5 23] INFO [2022-01-10 12:53:34,527] jepsen worker 86 - jepsen.util 176 :invoke :write [28 9] INFO [2022-01-10 12:53:34,531] jepsen worker 87 - jepsen.util 87 :ok :write [29 9] INFO [2022-01-10 12:53:34,531] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,533] jepsen worker 87 - jepsen.util 87 :ok :read [29 9] INFO [2022-01-10 12:53:34,533] jepsen worker 87 - jepsen.util 87 :invoke :write [29 25] INFO [2022-01-10 12:53:34,536] jepsen worker 40 - jepsen.util 130 :ok :write [13 21] INFO [2022-01-10 12:53:34,537] jepsen worker 40 - jepsen.util 130 :invoke :write [13 40] INFO [2022-01-10 12:53:34,542] jepsen worker 50 - jepsen.util 50 :ok :write [16 12] INFO [2022-01-10 12:53:34,542] jepsen worker 50 - jepsen.util 50 :invoke :write [16 48] INFO [2022-01-10 12:53:34,546] jepsen worker 83 - jepsen.util 263 :ok :write [27 22] INFO [2022-01-10 12:53:34,546] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:34,547] jepsen worker 83 - jepsen.util 263 :ok :read [27 22] INFO [2022-01-10 12:53:34,548] jepsen worker 83 - jepsen.util 263 :invoke :write [27 35] INFO [2022-01-10 12:53:34,549] jepsen worker 68 - jepsen.util 68 :ok :write [22 14] INFO [2022-01-10 12:53:34,550] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:34,551] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:34,551] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:34,552] jepsen worker 78 - jepsen.util 78 :info :write [26 33] :RequestTimeoutException INFO [2022-01-10 12:53:34,552] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:34,552] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:34,553] jepsen worker 68 - jepsen.util 68 :invoke :write [22 20] INFO [2022-01-10 12:53:34,553] jepsen worker 78 - jepsen.util 168 :ok :read [26 11] INFO [2022-01-10 12:53:34,554] jepsen worker 78 - jepsen.util 168 :invoke :write [26 12] INFO [2022-01-10 12:53:34,557] jepsen worker 34 - jepsen.util 214 :ok :write [11 24] INFO [2022-01-10 12:53:34,558] jepsen worker 34 - jepsen.util 214 :invoke :write [11 28] INFO [2022-01-10 12:53:34,561] jepsen worker 7 - jepsen.util 7 :ok :write [2 16] INFO [2022-01-10 12:53:34,561] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:34,562] jepsen worker 7 - jepsen.util 7 :ok :read [2 16] INFO [2022-01-10 12:53:34,562] jepsen worker 7 - jepsen.util 7 :invoke :write [2 31] INFO [2022-01-10 12:53:34,568] jepsen worker 15 - jepsen.util 15 :ok :write [5 23] INFO [2022-01-10 12:53:34,569] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:34,570] jepsen worker 15 - jepsen.util 15 :ok :read [5 23] INFO [2022-01-10 12:53:34,570] jepsen worker 15 - jepsen.util 15 :invoke :write [5 40] INFO [2022-01-10 12:53:34,572] jepsen worker 13 - jepsen.util 13 :ok :write [4 29] INFO [2022-01-10 12:53:34,572] jepsen worker 13 - jepsen.util 13 :invoke :write [4 30] INFO [2022-01-10 12:53:34,573] jepsen worker 22 - jepsen.util 112 :ok :write [7 23] INFO [2022-01-10 12:53:34,573] jepsen worker 22 - jepsen.util 112 :invoke :write [7 44] INFO [2022-01-10 12:53:34,578] jepsen worker 71 - jepsen.util 161 :ok :write [23 6] INFO [2022-01-10 12:53:34,578] jepsen worker 44 - jepsen.util 44 :ok :write [14 1] INFO [2022-01-10 12:53:34,578] jepsen worker 71 - jepsen.util 161 :invoke :write [23 10] INFO [2022-01-10 12:53:34,579] jepsen worker 44 - jepsen.util 44 :invoke :write [14 16] INFO [2022-01-10 12:53:34,580] jepsen worker 50 - jepsen.util 50 :ok :write [16 48] INFO [2022-01-10 12:53:34,581] jepsen worker 50 - jepsen.util 50 :invoke :write [16 9] INFO [2022-01-10 12:53:34,583] jepsen worker 75 - jepsen.util 165 :ok :write [25 12] INFO [2022-01-10 12:53:34,584] jepsen worker 75 - jepsen.util 165 :invoke :write [25 30] INFO [2022-01-10 12:53:34,584] jepsen worker 51 - jepsen.util 141 :ok :write [17 31] INFO [2022-01-10 12:53:34,585] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:34,585] jepsen worker 83 - jepsen.util 263 :ok :write [27 35] INFO [2022-01-10 12:53:34,586] jepsen worker 51 - jepsen.util 141 :ok :read [17 31] INFO [2022-01-10 12:53:34,586] jepsen worker 83 - jepsen.util 263 :invoke :write [27 3] INFO [2022-01-10 12:53:34,587] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:34,587] jepsen worker 51 - jepsen.util 141 :ok :read [17 31] INFO [2022-01-10 12:53:34,588] jepsen worker 51 - jepsen.util 141 :invoke :write [17 9] INFO [2022-01-10 12:53:34,589] jepsen worker 62 - jepsen.util 242 :ok :write [20 25] INFO [2022-01-10 12:53:34,589] jepsen worker 68 - jepsen.util 68 :ok :write [22 20] INFO [2022-01-10 12:53:34,590] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:34,590] jepsen worker 62 - jepsen.util 242 :invoke :write [20 18] INFO [2022-01-10 12:53:34,591] jepsen worker 68 - jepsen.util 68 :ok :read [22 20] INFO [2022-01-10 12:53:34,591] jepsen worker 68 - jepsen.util 68 :invoke :write [22 25] INFO [2022-01-10 12:53:34,592] jepsen worker 80 - jepsen.util 260 :ok :write [26 39] INFO [2022-01-10 12:53:34,592] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:34,593] jepsen worker 80 - jepsen.util 260 :ok :read [26 39] INFO [2022-01-10 12:53:34,594] jepsen worker 34 - jepsen.util 214 :ok :write [11 28] INFO [2022-01-10 12:53:34,594] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:34,594] jepsen worker 34 - jepsen.util 214 :invoke :write [11 16] INFO [2022-01-10 12:53:34,595] jepsen worker 80 - jepsen.util 260 :ok :read [26 39] INFO [2022-01-10 12:53:34,596] jepsen worker 80 - jepsen.util 260 :invoke :write [26 11] INFO [2022-01-10 12:53:34,596] jepsen worker 1 - jepsen.util 91 :ok :write [0 5] INFO [2022-01-10 12:53:34,596] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,597] jepsen worker 1 - jepsen.util 91 :ok :read [0 5] INFO [2022-01-10 12:53:34,598] jepsen worker 1 - jepsen.util 91 :invoke :write [0 18] INFO [2022-01-10 12:53:34,598] jepsen worker 7 - jepsen.util 7 :ok :write [2 31] INFO [2022-01-10 12:53:34,598] jepsen worker 7 - jepsen.util 7 :invoke :write [2 32] INFO [2022-01-10 12:53:34,600] jepsen worker 3 - jepsen.util 3 :ok :write [1 6] INFO [2022-01-10 12:53:34,600] jepsen worker 87 - jepsen.util 87 :ok :write [29 25] INFO [2022-01-10 12:53:34,601] jepsen worker 3 - jepsen.util 3 :invoke :write [1 21] INFO [2022-01-10 12:53:34,601] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,602] jepsen worker 15 - jepsen.util 15 :ok :write [5 40] INFO [2022-01-10 12:53:34,602] jepsen worker 87 - jepsen.util 87 :ok :read [29 25] INFO [2022-01-10 12:53:34,602] jepsen worker 15 - jepsen.util 15 :invoke :write [5 5] INFO [2022-01-10 12:53:34,603] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,603] jepsen worker 87 - jepsen.util 87 :ok :read [29 25] INFO [2022-01-10 12:53:34,604] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,605] jepsen worker 87 - jepsen.util 87 :ok :read [29 25] INFO [2022-01-10 12:53:34,605] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,606] jepsen worker 40 - jepsen.util 130 :ok :write [13 40] INFO [2022-01-10 12:53:34,606] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,607] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:34,607] jepsen worker 40 - jepsen.util 130 :ok :read [13 40] INFO [2022-01-10 12:53:34,607] jepsen worker 87 - jepsen.util 87 :ok :read [29 25] INFO [2022-01-10 12:53:34,608] jepsen worker 50 - jepsen.util 50 :ok :write [16 9] INFO [2022-01-10 12:53:34,608] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,608] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,609] jepsen worker 87 - jepsen.util 87 :ok :read [29 25] INFO [2022-01-10 12:53:34,609] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,609] jepsen worker 40 - jepsen.util 130 :ok :read [13 40] INFO [2022-01-10 12:53:34,610] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,610] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:34,610] jepsen worker 50 - jepsen.util 50 :invoke :write [16 10] INFO [2022-01-10 12:53:34,611] jepsen worker 87 - jepsen.util 87 :ok :read [29 25] INFO [2022-01-10 12:53:34,611] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,612] jepsen worker 22 - jepsen.util 112 :ok :write [7 44] INFO [2022-01-10 12:53:34,612] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:34,612] jepsen worker 87 - jepsen.util 87 :invoke :write [29 46] INFO [2022-01-10 12:53:34,613] jepsen worker 40 - jepsen.util 130 :invoke :write [13 43] INFO [2022-01-10 12:53:34,614] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:34,614] jepsen worker 22 - jepsen.util 112 :invoke :write [7 46] INFO [2022-01-10 12:53:34,615] jepsen worker 83 - jepsen.util 263 :ok :write [27 3] INFO [2022-01-10 12:53:34,616] jepsen worker 83 - jepsen.util 263 :invoke :write [27 36] INFO [2022-01-10 12:53:34,616] jepsen worker 86 - jepsen.util 176 :ok :write [28 9] INFO [2022-01-10 12:53:34,617] jepsen worker 86 - jepsen.util 176 :invoke :write [28 14] INFO [2022-01-10 12:53:34,617] jepsen worker 44 - jepsen.util 44 :ok :write [14 16] INFO [2022-01-10 12:53:34,618] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:34,619] jepsen worker 44 - jepsen.util 44 :ok :read [14 16] INFO [2022-01-10 12:53:34,620] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:34,620] jepsen worker 44 - jepsen.util 44 :ok :read [14 16] INFO [2022-01-10 12:53:34,621] jepsen worker 44 - jepsen.util 44 :invoke :write [14 42] INFO [2022-01-10 12:53:34,623] jepsen worker 13 - jepsen.util 13 :ok :write [4 30] INFO [2022-01-10 12:53:34,624] jepsen worker 13 - jepsen.util 13 :invoke :write [4 1] INFO [2022-01-10 12:53:34,625] jepsen worker 68 - jepsen.util 68 :ok :write [22 25] INFO [2022-01-10 12:53:34,625] jepsen worker 68 - jepsen.util 68 :invoke :write [22 30] INFO [2022-01-10 12:53:34,628] jepsen worker 75 - jepsen.util 165 :ok :write [25 30] INFO [2022-01-10 12:53:34,629] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:34,629] jepsen worker 75 - jepsen.util 165 :ok :read [25 30] INFO [2022-01-10 12:53:34,630] jepsen worker 75 - jepsen.util 165 :invoke :write [25 23] INFO [2022-01-10 12:53:34,631] jepsen worker 71 - jepsen.util 161 :ok :write [23 10] INFO [2022-01-10 12:53:34,631] jepsen worker 34 - jepsen.util 214 :ok :write [11 16] INFO [2022-01-10 12:53:34,632] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,632] jepsen worker 71 - jepsen.util 161 :ok :read [23 10] INFO [2022-01-10 12:53:34,632] jepsen worker 62 - jepsen.util 242 :ok :write [20 18] INFO [2022-01-10 12:53:34,633] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:34,633] jepsen worker 62 - jepsen.util 242 :invoke :write [20 46] INFO [2022-01-10 12:53:34,633] jepsen worker 34 - jepsen.util 214 :ok :read [11 16] INFO [2022-01-10 12:53:34,634] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,634] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:34,634] jepsen worker 71 - jepsen.util 161 :ok :read [23 10] INFO [2022-01-10 12:53:34,635] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,635] jepsen worker 34 - jepsen.util 214 :ok :read [11 16] INFO [2022-01-10 12:53:34,636] jepsen worker 34 - jepsen.util 214 :invoke :write [11 10] INFO [2022-01-10 12:53:34,636] jepsen worker 71 - jepsen.util 161 :ok :read [23 10] INFO [2022-01-10 12:53:34,637] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,637] jepsen worker 51 - jepsen.util 141 :ok :write [17 9] INFO [2022-01-10 12:53:34,638] jepsen worker 71 - jepsen.util 161 :ok :read [23 10] INFO [2022-01-10 12:53:34,638] jepsen worker 51 - jepsen.util 141 :invoke :write [17 22] INFO [2022-01-10 12:53:34,639] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,639] jepsen worker 1 - jepsen.util 91 :ok :write [0 18] INFO [2022-01-10 12:53:34,640] jepsen worker 71 - jepsen.util 161 :ok :read [23 10] INFO [2022-01-10 12:53:34,640] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,640] jepsen worker 71 - jepsen.util 161 :invoke :write [23 5] INFO [2022-01-10 12:53:34,641] jepsen worker 1 - jepsen.util 91 :ok :read [0 18] INFO [2022-01-10 12:53:34,641] jepsen worker 1 - jepsen.util 91 :invoke :write [0 10] INFO [2022-01-10 12:53:34,642] jepsen worker 7 - jepsen.util 7 :ok :write [2 32] INFO [2022-01-10 12:53:34,642] jepsen worker 78 - jepsen.util 168 :ok :write [26 12] INFO [2022-01-10 12:53:34,642] jepsen worker 78 - jepsen.util 168 :invoke :write [26 10] INFO [2022-01-10 12:53:34,643] jepsen worker 7 - jepsen.util 7 :invoke :write [2 32] INFO [2022-01-10 12:53:34,649] jepsen worker 3 - jepsen.util 3 :ok :write [1 21] INFO [2022-01-10 12:53:34,649] jepsen worker 87 - jepsen.util 87 :ok :write [29 46] INFO [2022-01-10 12:53:34,650] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,650] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,650] jepsen worker 3 - jepsen.util 3 :ok :read [1 21] INFO [2022-01-10 12:53:34,651] jepsen worker 3 - jepsen.util 3 :invoke :write [1 43] INFO [2022-01-10 12:53:34,651] jepsen worker 87 - jepsen.util 87 :ok :read [29 46] INFO [2022-01-10 12:53:34,652] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,653] jepsen worker 87 - jepsen.util 87 :ok :read [29 46] INFO [2022-01-10 12:53:34,653] jepsen worker 87 - jepsen.util 87 :invoke :write [29 5] INFO [2022-01-10 12:53:34,657] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:34,658] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,659] jepsen worker 38 - jepsen.util 38 :ok :read [12 11] INFO [2022-01-10 12:53:34,659] jepsen worker 40 - jepsen.util 130 :ok :write [13 43] INFO [2022-01-10 12:53:34,659] jepsen worker 38 - jepsen.util 38 :invoke :write [12 43] INFO [2022-01-10 12:53:34,660] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,661] jepsen worker 40 - jepsen.util 130 :ok :read [13 43] INFO [2022-01-10 12:53:34,661] jepsen worker 40 - jepsen.util 130 :invoke :write [13 41] INFO [2022-01-10 12:53:34,664] jepsen worker 86 - jepsen.util 176 :ok :write [28 14] INFO [2022-01-10 12:53:34,664] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:34,665] jepsen worker 86 - jepsen.util 176 :ok :read [28 14] INFO [2022-01-10 12:53:34,666] jepsen worker 86 - jepsen.util 176 :invoke :write [28 7] INFO [2022-01-10 12:53:34,669] jepsen worker 13 - jepsen.util 13 :ok :write [4 1] INFO [2022-01-10 12:53:34,670] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:34,671] jepsen worker 13 - jepsen.util 13 :ok :read [4 1] INFO [2022-01-10 12:53:34,671] jepsen worker 13 - jepsen.util 13 :invoke :write [4 11] INFO [2022-01-10 12:53:34,686] jepsen worker 1 - jepsen.util 91 :ok :write [0 10] INFO [2022-01-10 12:53:34,687] jepsen worker 1 - jepsen.util 91 :invoke :write [0 42] INFO [2022-01-10 12:53:34,691] jepsen worker 87 - jepsen.util 87 :ok :write [29 5] INFO [2022-01-10 12:53:34,692] jepsen worker 87 - jepsen.util 87 :invoke :write [29 12] INFO [2022-01-10 12:53:34,693] jepsen worker 3 - jepsen.util 3 :ok :write [1 43] INFO [2022-01-10 12:53:34,694] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,694] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:34,695] jepsen worker 3 - jepsen.util 3 :invoke :write [1 19] INFO [2022-01-10 12:53:34,698] jepsen worker 40 - jepsen.util 130 :ok :write [13 41] INFO [2022-01-10 12:53:34,698] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,699] jepsen worker 40 - jepsen.util 130 :ok :read [13 41] INFO [2022-01-10 12:53:34,700] jepsen worker 38 - jepsen.util 38 :ok :write [12 43] INFO [2022-01-10 12:53:34,700] jepsen worker 40 - jepsen.util 130 :invoke :write [13 31] INFO [2022-01-10 12:53:34,701] jepsen worker 38 - jepsen.util 38 :invoke :write [12 30] INFO [2022-01-10 12:53:34,706] jepsen worker 1 - jepsen.util 91 :ok :write [0 42] INFO [2022-01-10 12:53:34,706] jepsen worker 1 - jepsen.util 91 :invoke :write [0 31] INFO [2022-01-10 12:53:34,712] jepsen worker 86 - jepsen.util 176 :ok :write [28 7] INFO [2022-01-10 12:53:34,713] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:34,713] jepsen worker 86 - jepsen.util 176 :ok :read [28 7] INFO [2022-01-10 12:53:34,714] jepsen worker 87 - jepsen.util 87 :ok :write [29 12] INFO [2022-01-10 12:53:34,715] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:34,715] jepsen worker 86 - jepsen.util 176 :ok :read [28 7] INFO [2022-01-10 12:53:34,715] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,716] jepsen worker 86 - jepsen.util 176 :invoke :write [28 6] INFO [2022-01-10 12:53:34,716] jepsen worker 87 - jepsen.util 87 :ok :read [29 12] INFO [2022-01-10 12:53:34,717] jepsen worker 87 - jepsen.util 87 :invoke :write [29 32] INFO [2022-01-10 12:53:34,722] jepsen worker 3 - jepsen.util 3 :ok :write [1 19] INFO [2022-01-10 12:53:34,722] jepsen worker 40 - jepsen.util 130 :ok :write [13 31] INFO [2022-01-10 12:53:34,722] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,723] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,723] jepsen worker 40 - jepsen.util 130 :ok :read [13 31] INFO [2022-01-10 12:53:34,724] jepsen worker 3 - jepsen.util 3 :ok :read [1 19] INFO [2022-01-10 12:53:34,724] jepsen worker 40 - jepsen.util 130 :invoke :write [13 38] INFO [2022-01-10 12:53:34,725] jepsen worker 3 - jepsen.util 3 :invoke :write [1 35] INFO [2022-01-10 12:53:34,728] jepsen worker 0 - jepsen.util 180 :ok :write [0 28] INFO [2022-01-10 12:53:34,728] jepsen worker 0 - jepsen.util 180 :invoke :write [0 46] INFO [2022-01-10 12:53:34,728] jepsen worker 38 - jepsen.util 38 :ok :write [12 30] INFO [2022-01-10 12:53:34,729] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,730] jepsen worker 38 - jepsen.util 38 :ok :read [12 30] INFO [2022-01-10 12:53:34,730] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,731] jepsen worker 38 - jepsen.util 38 :ok :read [12 30] INFO [2022-01-10 12:53:34,732] jepsen worker 87 - jepsen.util 87 :ok :write [29 32] INFO [2022-01-10 12:53:34,732] jepsen worker 38 - jepsen.util 38 :invoke :write [12 0] INFO [2022-01-10 12:53:34,733] jepsen worker 87 - jepsen.util 87 :invoke :write [29 40] INFO [2022-01-10 12:53:34,735] jepsen worker 86 - jepsen.util 176 :ok :write [28 6] INFO [2022-01-10 12:53:34,736] jepsen worker 86 - jepsen.util 176 :invoke :write [28 3] INFO [2022-01-10 12:53:34,739] jepsen worker 40 - jepsen.util 130 :ok :write [13 38] INFO [2022-01-10 12:53:34,740] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:34,741] jepsen worker 40 - jepsen.util 130 :ok :read [13 38] INFO [2022-01-10 12:53:34,742] jepsen worker 40 - jepsen.util 130 :invoke :write [13 6] INFO [2022-01-10 12:53:34,742] jepsen worker 5 - jepsen.util 185 :ok :write [1 20] INFO [2022-01-10 12:53:34,743] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,744] jepsen worker 5 - jepsen.util 185 :ok :read [1 20] INFO [2022-01-10 12:53:34,744] jepsen worker 0 - jepsen.util 180 :ok :write [0 46] INFO [2022-01-10 12:53:34,744] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,745] jepsen worker 0 - jepsen.util 180 :invoke :write [0 21] INFO [2022-01-10 12:53:34,745] jepsen worker 5 - jepsen.util 185 :ok :read [1 20] INFO [2022-01-10 12:53:34,746] jepsen worker 5 - jepsen.util 185 :invoke :write [1 40] INFO [2022-01-10 12:53:34,748] jepsen worker 38 - jepsen.util 38 :ok :write [12 0] INFO [2022-01-10 12:53:34,748] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,749] jepsen worker 38 - jepsen.util 38 :ok :read [12 0] INFO [2022-01-10 12:53:34,750] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,751] jepsen worker 38 - jepsen.util 38 :ok :read [12 0] INFO [2022-01-10 12:53:34,751] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,752] jepsen worker 38 - jepsen.util 38 :ok :read [12 0] INFO [2022-01-10 12:53:34,752] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,753] jepsen worker 38 - jepsen.util 38 :ok :read [12 0] INFO [2022-01-10 12:53:34,754] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,755] jepsen worker 38 - jepsen.util 38 :ok :read [12 0] INFO [2022-01-10 12:53:34,755] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:34,756] jepsen worker 38 - jepsen.util 38 :ok :read [12 0] INFO [2022-01-10 12:53:34,756] jepsen worker 38 - jepsen.util 38 :invoke :write [12 39] INFO [2022-01-10 12:53:34,797] jepsen worker 30 - jepsen.util 300 :fail :read [10 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:34,798] jepsen worker 30 - jepsen.util 300 :invoke :write [10 17] INFO [2022-01-10 12:53:34,838] jepsen worker 15 - jepsen.util 15 :ok :write [5 5] INFO [2022-01-10 12:53:34,838] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:34,842] jepsen worker 51 - jepsen.util 141 :ok :write [17 22] INFO [2022-01-10 12:53:34,843] jepsen worker 50 - jepsen.util 50 :ok :write [16 10] INFO [2022-01-10 12:53:34,845] jepsen worker 15 - jepsen.util 15 :ok :read [5 5] INFO [2022-01-10 12:53:34,846] jepsen worker 51 - jepsen.util 141 :invoke :write [17 36] INFO [2022-01-10 12:53:34,846] jepsen worker 50 - jepsen.util 50 :invoke :write [16 11] INFO [2022-01-10 12:53:34,847] jepsen worker 15 - jepsen.util 15 :invoke :write [5 38] INFO [2022-01-10 12:53:34,850] jepsen worker 71 - jepsen.util 161 :ok :write [23 5] INFO [2022-01-10 12:53:34,850] jepsen worker 22 - jepsen.util 112 :ok :write [7 46] INFO [2022-01-10 12:53:34,851] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,851] jepsen worker 22 - jepsen.util 112 :invoke :write [7 18] INFO [2022-01-10 12:53:34,855] jepsen worker 78 - jepsen.util 168 :ok :write [26 10] INFO [2022-01-10 12:53:34,856] jepsen worker 83 - jepsen.util 263 :ok :write [27 36] INFO [2022-01-10 12:53:34,856] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:34,857] jepsen worker 83 - jepsen.util 263 :invoke :write [27 25] INFO [2022-01-10 12:53:34,861] jepsen worker 44 - jepsen.util 44 :ok :write [14 42] INFO [2022-01-10 12:53:34,861] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:34,862] jepsen worker 13 - jepsen.util 13 :ok :write [4 11] INFO [2022-01-10 12:53:34,862] jepsen worker 44 - jepsen.util 44 :ok :read [14 42] INFO [2022-01-10 12:53:34,863] jepsen worker 13 - jepsen.util 13 :invoke :write [4 36] INFO [2022-01-10 12:53:34,863] jepsen worker 44 - jepsen.util 44 :invoke :write [14 13] INFO [2022-01-10 12:53:34,868] jepsen worker 68 - jepsen.util 68 :ok :write [22 30] INFO [2022-01-10 12:53:34,869] jepsen worker 68 - jepsen.util 68 :invoke :write [22 19] INFO [2022-01-10 12:53:34,870] jepsen worker 87 - jepsen.util 87 :ok :write [29 40] INFO [2022-01-10 12:53:34,871] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,872] jepsen worker 87 - jepsen.util 87 :ok :read [29 40] INFO [2022-01-10 12:53:34,873] jepsen worker 87 - jepsen.util 87 :invoke :read [29 nil] INFO [2022-01-10 12:53:34,873] jepsen worker 84 - jepsen.util 84 :ok :write [28 37] INFO [2022-01-10 12:53:34,874] jepsen worker 87 - jepsen.util 87 :ok :read [29 40] INFO [2022-01-10 12:53:34,874] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:34,875] jepsen worker 40 - jepsen.util 130 :ok :write [13 6] INFO [2022-01-10 12:53:34,875] jepsen worker 87 - jepsen.util 87 :invoke :write [29 42] INFO [2022-01-10 12:53:34,875] jepsen worker 84 - jepsen.util 84 :ok :read [28 37] INFO [2022-01-10 12:53:34,876] jepsen worker 40 - jepsen.util 130 :invoke :write [13 28] INFO [2022-01-10 12:53:34,876] jepsen worker 32 - jepsen.util 212 :info :write [10 11] :RequestTimeoutException INFO [2022-01-10 12:53:34,876] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:34,877] jepsen worker 84 - jepsen.util 84 :ok :read [28 37] INFO [2022-01-10 12:53:34,877] jepsen worker 32 - jepsen.util 302 :invoke :read [10 nil] INFO [2022-01-10 12:53:34,878] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:34,878] jepsen worker 32 - jepsen.util 302 :ok :read [10 38] INFO [2022-01-10 12:53:34,878] jepsen worker 84 - jepsen.util 84 :ok :read [28 37] INFO [2022-01-10 12:53:34,879] jepsen worker 32 - jepsen.util 302 :invoke :read [10 nil] INFO [2022-01-10 12:53:34,879] jepsen worker 84 - jepsen.util 84 :invoke :write [28 48] INFO [2022-01-10 12:53:34,879] jepsen worker 32 - jepsen.util 302 :ok :read [10 38] INFO [2022-01-10 12:53:34,880] jepsen worker 32 - jepsen.util 302 :invoke :write [10 44] INFO [2022-01-10 12:53:34,882] jepsen worker 0 - jepsen.util 180 :ok :write [0 21] INFO [2022-01-10 12:53:34,882] jepsen worker 3 - jepsen.util 3 :ok :write [1 35] INFO [2022-01-10 12:53:34,882] jepsen worker 71 - jepsen.util 161 :ok :read [23 5] INFO [2022-01-10 12:53:34,882] jepsen worker 78 - jepsen.util 168 :ok :read [26 10] INFO [2022-01-10 12:53:34,882] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,883] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,883] jepsen worker 3 - jepsen.util 3 :ok :read [1 35] INFO [2022-01-10 12:53:34,883] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:34,884] jepsen worker 0 - jepsen.util 180 :ok :read [0 21] INFO [2022-01-10 12:53:34,884] jepsen worker 78 - jepsen.util 168 :ok :read [26 10] INFO [2022-01-10 12:53:34,884] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,885] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:34,885] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,885] jepsen worker 3 - jepsen.util 3 :ok :read [1 35] INFO [2022-01-10 12:53:34,885] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,885] jepsen worker 78 - jepsen.util 168 :ok :read [26 10] INFO [2022-01-10 12:53:34,886] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:34,886] jepsen worker 0 - jepsen.util 180 :ok :read [0 21] INFO [2022-01-10 12:53:34,886] jepsen worker 78 - jepsen.util 168 :invoke :write [26 34] INFO [2022-01-10 12:53:34,886] jepsen worker 71 - jepsen.util 161 :ok :read [23 5] INFO [2022-01-10 12:53:34,887] jepsen worker 3 - jepsen.util 3 :ok :read [1 35] INFO [2022-01-10 12:53:34,887] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:34,887] jepsen worker 3 - jepsen.util 3 :invoke :write [1 46] INFO [2022-01-10 12:53:34,887] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,888] jepsen worker 0 - jepsen.util 180 :ok :read [0 21] INFO [2022-01-10 12:53:34,888] jepsen worker 0 - jepsen.util 180 :invoke :write [0 26] INFO [2022-01-10 12:53:34,888] jepsen worker 71 - jepsen.util 161 :ok :read [23 5] INFO [2022-01-10 12:53:34,889] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,890] jepsen worker 71 - jepsen.util 161 :ok :read [23 5] INFO [2022-01-10 12:53:34,890] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,890] jepsen worker 38 - jepsen.util 38 :ok :write [12 39] INFO [2022-01-10 12:53:34,891] jepsen worker 71 - jepsen.util 161 :ok :read [23 5] INFO [2022-01-10 12:53:34,891] jepsen worker 38 - jepsen.util 38 :invoke :write [12 40] INFO [2022-01-10 12:53:34,892] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:34,892] jepsen worker 71 - jepsen.util 161 :ok :read [23 5] INFO [2022-01-10 12:53:34,893] jepsen worker 71 - jepsen.util 161 :invoke :write [23 7] INFO [2022-01-10 12:53:34,906] jepsen worker 87 - jepsen.util 87 :ok :write [29 42] INFO [2022-01-10 12:53:34,907] jepsen worker 87 - jepsen.util 87 :invoke :write [29 43] INFO [2022-01-10 12:53:34,909] jepsen worker 40 - jepsen.util 130 :ok :write [13 28] INFO [2022-01-10 12:53:34,910] jepsen worker 40 - jepsen.util 130 :invoke :write [13 14] INFO [2022-01-10 12:53:34,912] jepsen worker 1 - jepsen.util 91 :ok :write [0 31] INFO [2022-01-10 12:53:34,913] jepsen worker 1 - jepsen.util 91 :invoke :write [0 22] INFO [2022-01-10 12:53:35,050] jepsen worker 84 - jepsen.util 84 :ok :write [28 48] INFO [2022-01-10 12:53:35,051] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:35,052] jepsen worker 84 - jepsen.util 84 :ok :read [28 48] INFO [2022-01-10 12:53:35,052] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:35,053] jepsen worker 84 - jepsen.util 84 :ok :read [28 48] INFO [2022-01-10 12:53:35,053] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:35,054] jepsen worker 84 - jepsen.util 84 :ok :read [28 48] INFO [2022-01-10 12:53:35,055] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:35,056] jepsen worker 84 - jepsen.util 84 :ok :read [28 48] INFO [2022-01-10 12:53:35,056] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:35,057] jepsen worker 87 - jepsen.util 87 :ok :write [29 43] INFO [2022-01-10 12:53:35,057] jepsen worker 84 - jepsen.util 84 :ok :read [28 48] INFO [2022-01-10 12:53:35,057] jepsen worker 87 - jepsen.util 87 :invoke :write [29 22] INFO [2022-01-10 12:53:35,058] jepsen worker 84 - jepsen.util 84 :invoke :write [28 19] INFO [2022-01-10 12:53:35,060] jepsen worker 75 - jepsen.util 165 :ok :write [25 23] INFO [2022-01-10 12:53:35,060] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:35,061] jepsen worker 75 - jepsen.util 165 :ok :read [25 23] INFO [2022-01-10 12:53:35,061] jepsen worker 75 - jepsen.util 165 :invoke :write [25 17] INFO [2022-01-10 12:53:35,062] jepsen worker 78 - jepsen.util 168 :ok :write [26 34] INFO [2022-01-10 12:53:35,063] jepsen worker 78 - jepsen.util 168 :invoke :write [26 33] INFO [2022-01-10 12:53:35,064] jepsen worker 40 - jepsen.util 130 :ok :write [13 14] INFO [2022-01-10 12:53:35,065] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:35,066] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:35,066] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:35,066] jepsen worker 13 - jepsen.util 13 :ok :write [4 36] INFO [2022-01-10 12:53:35,066] jepsen worker 34 - jepsen.util 214 :ok :write [11 10] INFO [2022-01-10 12:53:35,067] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,067] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:35,067] jepsen worker 3 - jepsen.util 3 :ok :write [1 46] INFO [2022-01-10 12:53:35,068] jepsen worker 34 - jepsen.util 214 :invoke :write [11 24] INFO [2022-01-10 12:53:35,068] jepsen worker 13 - jepsen.util 13 :ok :read [4 36] INFO [2022-01-10 12:53:35,068] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:35,068] jepsen worker 1 - jepsen.util 91 :ok :write [0 22] INFO [2022-01-10 12:53:35,069] jepsen worker 40 - jepsen.util 130 :invoke :write [13 1] INFO [2022-01-10 12:53:35,069] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,070] jepsen worker 3 - jepsen.util 3 :ok :read [1 46] INFO [2022-01-10 12:53:35,070] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,070] jepsen worker 13 - jepsen.util 13 :ok :read [4 36] INFO [2022-01-10 12:53:35,071] jepsen worker 3 - jepsen.util 3 :invoke :write [1 18] INFO [2022-01-10 12:53:35,071] jepsen worker 51 - jepsen.util 141 :ok :write [17 36] INFO [2022-01-10 12:53:35,071] jepsen worker 1 - jepsen.util 91 :ok :read [0 22] INFO [2022-01-10 12:53:35,071] jepsen worker 13 - jepsen.util 13 :invoke :write [4 17] INFO [2022-01-10 12:53:35,072] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,072] jepsen worker 51 - jepsen.util 141 :invoke :write [17 23] INFO [2022-01-10 12:53:35,073] jepsen worker 1 - jepsen.util 91 :ok :read [0 22] INFO [2022-01-10 12:53:35,073] jepsen worker 89 - jepsen.util 269 :ok :write [29 35] INFO [2022-01-10 12:53:35,074] jepsen worker 89 - jepsen.util 269 :invoke :write [29 37] INFO [2022-01-10 12:53:35,074] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,074] jepsen worker 62 - jepsen.util 242 :ok :write [20 46] INFO [2022-01-10 12:53:35,075] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:35,075] jepsen worker 1 - jepsen.util 91 :ok :read [0 22] INFO [2022-01-10 12:53:35,075] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,076] jepsen worker 62 - jepsen.util 242 :ok :read [20 46] INFO [2022-01-10 12:53:35,076] jepsen worker 38 - jepsen.util 38 :ok :write [12 40] INFO [2022-01-10 12:53:35,076] jepsen worker 62 - jepsen.util 242 :invoke :write [20 42] INFO [2022-01-10 12:53:35,076] jepsen worker 1 - jepsen.util 91 :ok :read [0 22] INFO [2022-01-10 12:53:35,077] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:35,077] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,078] jepsen worker 1 - jepsen.util 91 :ok :read [0 22] INFO [2022-01-10 12:53:35,078] jepsen worker 1 - jepsen.util 91 :invoke :write [0 24] INFO [2022-01-10 12:53:35,081] jepsen worker 7 - jepsen.util 7 :ok :write [2 32] INFO [2022-01-10 12:53:35,081] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,082] jepsen worker 7 - jepsen.util 7 :ok :read [2 32] INFO [2022-01-10 12:53:35,082] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,083] jepsen worker 7 - jepsen.util 7 :ok :read [2 32] INFO [2022-01-10 12:53:35,084] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,085] jepsen worker 7 - jepsen.util 7 :ok :read [2 32] INFO [2022-01-10 12:53:35,085] jepsen worker 7 - jepsen.util 7 :invoke :write [2 31] INFO [2022-01-10 12:53:35,085] jepsen worker 71 - jepsen.util 161 :ok :write [23 7] INFO [2022-01-10 12:53:35,086] jepsen worker 71 - jepsen.util 161 :invoke :write [23 1] INFO [2022-01-10 12:53:35,088] jepsen worker 15 - jepsen.util 15 :ok :write [5 38] INFO [2022-01-10 12:53:35,089] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,090] jepsen worker 15 - jepsen.util 15 :ok :read [5 38] INFO [2022-01-10 12:53:35,090] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,091] jepsen worker 15 - jepsen.util 15 :ok :read [5 38] INFO [2022-01-10 12:53:35,091] jepsen worker 15 - jepsen.util 15 :invoke :write [5 41] INFO [2022-01-10 12:53:35,092] jepsen worker 84 - jepsen.util 84 :ok :write [28 19] INFO [2022-01-10 12:53:35,092] jepsen worker 40 - jepsen.util 130 :ok :write [13 1] INFO [2022-01-10 12:53:35,093] jepsen worker 84 - jepsen.util 84 :invoke :write [28 23] INFO [2022-01-10 12:53:35,093] jepsen worker 40 - jepsen.util 130 :invoke :write [13 16] INFO [2022-01-10 12:53:35,094] jepsen worker 50 - jepsen.util 50 :ok :write [16 11] INFO [2022-01-10 12:53:35,095] jepsen worker 50 - jepsen.util 50 :invoke :write [16 39] INFO [2022-01-10 12:53:35,099] jepsen worker 22 - jepsen.util 112 :ok :write [7 18] INFO [2022-01-10 12:53:35,100] jepsen worker 22 - jepsen.util 112 :invoke :write [7 49] INFO [2022-01-10 12:53:35,105] jepsen worker 83 - jepsen.util 263 :ok :write [27 25] INFO [2022-01-10 12:53:35,106] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,106] jepsen worker 83 - jepsen.util 263 :ok :read [27 25] INFO [2022-01-10 12:53:35,107] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,108] jepsen worker 83 - jepsen.util 263 :ok :read [27 25] INFO [2022-01-10 12:53:35,108] jepsen worker 89 - jepsen.util 269 :ok :write [29 37] INFO [2022-01-10 12:53:35,108] jepsen worker 44 - jepsen.util 44 :ok :write [14 13] INFO [2022-01-10 12:53:35,108] jepsen worker 83 - jepsen.util 263 :invoke :write [27 45] INFO [2022-01-10 12:53:35,109] jepsen worker 89 - jepsen.util 269 :invoke :write [29 22] INFO [2022-01-10 12:53:35,110] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:35,111] jepsen worker 44 - jepsen.util 44 :ok :read [14 13] INFO [2022-01-10 12:53:35,111] jepsen worker 44 - jepsen.util 44 :invoke :write [14 49] INFO [2022-01-10 12:53:35,113] jepsen worker 1 - jepsen.util 91 :ok :write [0 24] INFO [2022-01-10 12:53:35,114] jepsen worker 1 - jepsen.util 91 :invoke :write [0 24] INFO [2022-01-10 12:53:35,114] jepsen worker 68 - jepsen.util 68 :ok :write [22 19] INFO [2022-01-10 12:53:35,114] jepsen worker 68 - jepsen.util 68 :invoke :write [22 26] INFO [2022-01-10 12:53:35,118] jepsen worker 75 - jepsen.util 165 :ok :write [25 17] INFO [2022-01-10 12:53:35,119] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:35,120] jepsen worker 75 - jepsen.util 165 :ok :read [25 17] INFO [2022-01-10 12:53:35,120] jepsen worker 75 - jepsen.util 165 :invoke :write [25 42] INFO [2022-01-10 12:53:35,122] jepsen worker 34 - jepsen.util 214 :ok :write [11 24] INFO [2022-01-10 12:53:35,123] jepsen worker 34 - jepsen.util 214 :invoke :write [11 2] INFO [2022-01-10 12:53:35,129] jepsen worker 7 - jepsen.util 7 :ok :write [2 31] INFO [2022-01-10 12:53:35,130] jepsen worker 7 - jepsen.util 7 :invoke :write [2 37] INFO [2022-01-10 12:53:35,134] jepsen worker 15 - jepsen.util 15 :ok :write [5 41] INFO [2022-01-10 12:53:35,135] jepsen worker 15 - jepsen.util 15 :invoke :write [5 47] INFO [2022-01-10 12:53:35,138] jepsen worker 50 - jepsen.util 50 :ok :write [16 39] INFO [2022-01-10 12:53:35,139] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:35,140] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:35,141] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:35,141] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:35,142] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:35,143] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:35,144] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:35,144] jepsen worker 83 - jepsen.util 263 :ok :write [27 45] INFO [2022-01-10 12:53:35,144] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,144] jepsen worker 50 - jepsen.util 50 :ok :read [16 39] INFO [2022-01-10 12:53:35,145] jepsen worker 83 - jepsen.util 263 :ok :read [27 45] INFO [2022-01-10 12:53:35,145] jepsen worker 50 - jepsen.util 50 :invoke :write [16 33] INFO [2022-01-10 12:53:35,146] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,147] jepsen worker 83 - jepsen.util 263 :ok :read [27 45] INFO [2022-01-10 12:53:35,147] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,148] jepsen worker 83 - jepsen.util 263 :ok :read [27 45] INFO [2022-01-10 12:53:35,149] jepsen worker 83 - jepsen.util 263 :invoke :write [27 10] INFO [2022-01-10 12:53:35,154] jepsen worker 68 - jepsen.util 68 :ok :write [22 26] INFO [2022-01-10 12:53:35,155] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,156] jepsen worker 68 - jepsen.util 68 :ok :read [22 26] INFO [2022-01-10 12:53:35,157] jepsen worker 68 - jepsen.util 68 :invoke :write [22 38] INFO [2022-01-10 12:53:35,166] jepsen worker 34 - jepsen.util 214 :ok :write [11 2] INFO [2022-01-10 12:53:35,167] jepsen worker 34 - jepsen.util 214 :invoke :write [11 21] INFO [2022-01-10 12:53:35,171] jepsen worker 7 - jepsen.util 7 :ok :write [2 37] INFO [2022-01-10 12:53:35,171] jepsen worker 7 - jepsen.util 7 :invoke :write [2 42] INFO [2022-01-10 12:53:35,179] jepsen worker 15 - jepsen.util 15 :ok :write [5 47] INFO [2022-01-10 12:53:35,179] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,180] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:35,180] jepsen worker 15 - jepsen.util 15 :invoke :write [5 12] INFO [2022-01-10 12:53:35,186] jepsen worker 50 - jepsen.util 50 :ok :write [16 33] INFO [2022-01-10 12:53:35,188] jepsen worker 50 - jepsen.util 50 :invoke :write [16 28] INFO [2022-01-10 12:53:35,188] jepsen worker 64 - jepsen.util 334 :fail :read [21 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:35,188] jepsen worker 64 - jepsen.util 334 :invoke :read [21 nil] INFO [2022-01-10 12:53:35,191] jepsen worker 83 - jepsen.util 263 :ok :write [27 10] INFO [2022-01-10 12:53:35,192] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,193] jepsen worker 83 - jepsen.util 263 :ok :read [27 10] INFO [2022-01-10 12:53:35,193] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,194] jepsen worker 83 - jepsen.util 263 :ok :read [27 10] INFO [2022-01-10 12:53:35,194] jepsen worker 68 - jepsen.util 68 :ok :write [22 38] INFO [2022-01-10 12:53:35,195] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,195] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,196] jepsen worker 68 - jepsen.util 68 :ok :read [22 38] INFO [2022-01-10 12:53:35,196] jepsen worker 83 - jepsen.util 263 :ok :read [27 10] INFO [2022-01-10 12:53:35,196] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,197] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,197] jepsen worker 34 - jepsen.util 214 :ok :write [11 21] INFO [2022-01-10 12:53:35,197] jepsen worker 68 - jepsen.util 68 :ok :read [22 38] INFO [2022-01-10 12:53:35,198] jepsen worker 83 - jepsen.util 263 :ok :read [27 10] INFO [2022-01-10 12:53:35,198] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,198] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,199] jepsen worker 68 - jepsen.util 68 :ok :read [22 38] INFO [2022-01-10 12:53:35,199] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,199] jepsen worker 83 - jepsen.util 263 :ok :read [27 10] INFO [2022-01-10 12:53:35,199] jepsen worker 68 - jepsen.util 68 :invoke :write [22 7] INFO [2022-01-10 12:53:35,200] jepsen worker 34 - jepsen.util 214 :ok :read [11 21] INFO [2022-01-10 12:53:35,200] jepsen worker 83 - jepsen.util 263 :invoke :write [27 2] INFO [2022-01-10 12:53:35,200] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,201] jepsen worker 7 - jepsen.util 7 :ok :write [2 42] INFO [2022-01-10 12:53:35,201] jepsen worker 34 - jepsen.util 214 :ok :read [11 21] INFO [2022-01-10 12:53:35,202] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,203] jepsen worker 34 - jepsen.util 214 :invoke :write [11 22] INFO [2022-01-10 12:53:35,203] jepsen worker 7 - jepsen.util 7 :ok :read [2 42] INFO [2022-01-10 12:53:35,204] jepsen worker 7 - jepsen.util 7 :invoke :write [2 12] INFO [2022-01-10 12:53:35,210] jepsen worker 15 - jepsen.util 15 :ok :write [5 12] INFO [2022-01-10 12:53:35,211] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,212] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:35,212] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,213] jepsen worker 15 - jepsen.util 15 :ok :read [5 12] INFO [2022-01-10 12:53:35,214] jepsen worker 50 - jepsen.util 50 :ok :write [16 28] INFO [2022-01-10 12:53:35,214] jepsen worker 15 - jepsen.util 15 :invoke :write [5 48] INFO [2022-01-10 12:53:35,215] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:35,216] jepsen worker 50 - jepsen.util 50 :ok :read [16 28] INFO [2022-01-10 12:53:35,216] jepsen worker 50 - jepsen.util 50 :invoke :write [16 37] INFO [2022-01-10 12:53:35,219] jepsen worker 83 - jepsen.util 263 :ok :write [27 2] INFO [2022-01-10 12:53:35,220] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,220] jepsen worker 83 - jepsen.util 263 :ok :read [27 2] INFO [2022-01-10 12:53:35,221] jepsen worker 83 - jepsen.util 263 :invoke :write [27 4] INFO [2022-01-10 12:53:35,222] jepsen worker 68 - jepsen.util 68 :ok :write [22 7] INFO [2022-01-10 12:53:35,223] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,224] jepsen worker 68 - jepsen.util 68 :ok :read [22 7] INFO [2022-01-10 12:53:35,224] jepsen worker 68 - jepsen.util 68 :invoke :write [22 2] INFO [2022-01-10 12:53:35,225] jepsen worker 34 - jepsen.util 214 :ok :write [11 22] INFO [2022-01-10 12:53:35,226] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,227] jepsen worker 34 - jepsen.util 214 :ok :read [11 22] INFO [2022-01-10 12:53:35,228] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,228] jepsen worker 34 - jepsen.util 214 :ok :read [11 22] INFO [2022-01-10 12:53:35,229] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,229] jepsen worker 7 - jepsen.util 7 :ok :write [2 12] INFO [2022-01-10 12:53:35,230] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,230] jepsen worker 34 - jepsen.util 214 :ok :read [11 22] INFO [2022-01-10 12:53:35,230] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,230] jepsen worker 7 - jepsen.util 7 :ok :read [2 12] INFO [2022-01-10 12:53:35,231] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,231] jepsen worker 34 - jepsen.util 214 :ok :read [11 22] INFO [2022-01-10 12:53:35,232] jepsen worker 34 - jepsen.util 214 :invoke :write [11 31] INFO [2022-01-10 12:53:35,232] jepsen worker 7 - jepsen.util 7 :ok :read [2 12] INFO [2022-01-10 12:53:35,232] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,233] jepsen worker 7 - jepsen.util 7 :ok :read [2 12] INFO [2022-01-10 12:53:35,234] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,234] jepsen worker 15 - jepsen.util 15 :ok :write [5 48] INFO [2022-01-10 12:53:35,234] jepsen worker 7 - jepsen.util 7 :ok :read [2 12] INFO [2022-01-10 12:53:35,235] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,235] jepsen worker 7 - jepsen.util 7 :invoke :write [2 35] INFO [2022-01-10 12:53:35,236] jepsen worker 15 - jepsen.util 15 :ok :read [5 48] INFO [2022-01-10 12:53:35,236] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,237] jepsen worker 15 - jepsen.util 15 :ok :read [5 48] INFO [2022-01-10 12:53:35,238] jepsen worker 15 - jepsen.util 15 :invoke :write [5 3] INFO [2022-01-10 12:53:35,239] jepsen worker 50 - jepsen.util 50 :ok :write [16 37] INFO [2022-01-10 12:53:35,239] jepsen worker 50 - jepsen.util 50 :invoke :write [16 48] INFO [2022-01-10 12:53:35,245] jepsen worker 83 - jepsen.util 263 :ok :write [27 4] INFO [2022-01-10 12:53:35,245] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,246] jepsen worker 83 - jepsen.util 263 :ok :read [27 4] INFO [2022-01-10 12:53:35,247] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,247] jepsen worker 83 - jepsen.util 263 :ok :read [27 4] INFO [2022-01-10 12:53:35,248] jepsen worker 83 - jepsen.util 263 :invoke :write [27 14] INFO [2022-01-10 12:53:35,248] jepsen worker 68 - jepsen.util 68 :ok :write [22 2] INFO [2022-01-10 12:53:35,249] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,250] jepsen worker 68 - jepsen.util 68 :ok :read [22 2] INFO [2022-01-10 12:53:35,250] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,251] jepsen worker 68 - jepsen.util 68 :ok :read [22 2] INFO [2022-01-10 12:53:35,252] jepsen worker 68 - jepsen.util 68 :invoke :write [22 46] INFO [2022-01-10 12:53:35,261] jepsen worker 34 - jepsen.util 214 :ok :write [11 31] INFO [2022-01-10 12:53:35,261] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,262] jepsen worker 34 - jepsen.util 214 :ok :read [11 31] INFO [2022-01-10 12:53:35,263] jepsen worker 34 - jepsen.util 214 :invoke :write [11 16] INFO [2022-01-10 12:53:35,267] jepsen worker 7 - jepsen.util 7 :ok :write [2 35] INFO [2022-01-10 12:53:35,267] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,269] jepsen worker 7 - jepsen.util 7 :ok :read [2 35] INFO [2022-01-10 12:53:35,269] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,270] jepsen worker 7 - jepsen.util 7 :ok :read [2 35] INFO [2022-01-10 12:53:35,270] jepsen worker 7 - jepsen.util 7 :invoke :write [2 22] INFO [2022-01-10 12:53:35,280] jepsen worker 15 - jepsen.util 15 :ok :write [5 3] INFO [2022-01-10 12:53:35,280] jepsen worker 15 - jepsen.util 15 :invoke :write [5 24] INFO [2022-01-10 12:53:35,286] jepsen worker 50 - jepsen.util 50 :ok :write [16 48] INFO [2022-01-10 12:53:35,287] jepsen worker 50 - jepsen.util 50 :invoke :write [16 1] INFO [2022-01-10 12:53:35,294] jepsen worker 34 - jepsen.util 214 :ok :write [11 16] INFO [2022-01-10 12:53:35,295] jepsen worker 34 - jepsen.util 214 :invoke :write [11 16] INFO [2022-01-10 12:53:35,296] jepsen worker 68 - jepsen.util 68 :ok :write [22 46] INFO [2022-01-10 12:53:35,296] jepsen worker 83 - jepsen.util 263 :ok :write [27 14] INFO [2022-01-10 12:53:35,296] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,297] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,297] jepsen worker 68 - jepsen.util 68 :ok :read [22 46] INFO [2022-01-10 12:53:35,298] jepsen worker 83 - jepsen.util 263 :ok :read [27 14] INFO [2022-01-10 12:53:35,298] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,298] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,299] jepsen worker 68 - jepsen.util 68 :ok :read [22 46] INFO [2022-01-10 12:53:35,299] jepsen worker 83 - jepsen.util 263 :ok :read [27 14] INFO [2022-01-10 12:53:35,299] jepsen worker 68 - jepsen.util 68 :invoke :write [22 33] INFO [2022-01-10 12:53:35,300] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,301] jepsen worker 83 - jepsen.util 263 :ok :read [27 14] INFO [2022-01-10 12:53:35,301] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,302] jepsen worker 83 - jepsen.util 263 :ok :read [27 14] INFO [2022-01-10 12:53:35,302] jepsen worker 83 - jepsen.util 263 :invoke :write [27 46] INFO [2022-01-10 12:53:35,348] jepsen worker 14 - jepsen.util 194 :info :write [4 23] :RequestTimeoutException INFO [2022-01-10 12:53:35,348] jepsen worker 14 - jepsen.util 284 :invoke :write [4 5] INFO [2022-01-10 12:53:35,443] jepsen worker 7 - jepsen.util 7 :ok :write [2 22] INFO [2022-01-10 12:53:35,444] jepsen worker 7 - jepsen.util 7 :invoke :write [2 43] INFO [2022-01-10 12:53:35,445] jepsen worker 67 - jepsen.util 157 :info :write [22 17] :RequestTimeoutException INFO [2022-01-10 12:53:35,446] jepsen worker 67 - jepsen.util 247 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,447] jepsen worker 67 - jepsen.util 247 :ok :read [22 46] INFO [2022-01-10 12:53:35,448] jepsen worker 67 - jepsen.util 247 :invoke :write [22 32] INFO [2022-01-10 12:53:35,451] jepsen worker 15 - jepsen.util 15 :ok :write [5 24] INFO [2022-01-10 12:53:35,451] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,452] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 12:53:35,452] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,453] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 12:53:35,454] jepsen worker 15 - jepsen.util 15 :invoke :write [5 7] INFO [2022-01-10 12:53:35,458] jepsen worker 50 - jepsen.util 50 :ok :write [16 1] INFO [2022-01-10 12:53:35,458] jepsen worker 78 - jepsen.util 168 :ok :write [26 33] INFO [2022-01-10 12:53:35,458] jepsen worker 50 - jepsen.util 50 :invoke :write [16 6] INFO [2022-01-10 12:53:35,459] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:35,460] jepsen worker 78 - jepsen.util 168 :ok :read [26 33] INFO [2022-01-10 12:53:35,460] jepsen worker 62 - jepsen.util 242 :ok :write [20 42] INFO [2022-01-10 12:53:35,460] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:35,461] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:35,461] jepsen worker 62 - jepsen.util 242 :ok :read [20 42] INFO [2022-01-10 12:53:35,462] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:35,462] jepsen worker 78 - jepsen.util 168 :ok :read [26 33] INFO [2022-01-10 12:53:35,462] jepsen worker 62 - jepsen.util 242 :ok :read [20 42] INFO [2022-01-10 12:53:35,462] jepsen worker 78 - jepsen.util 168 :invoke :write [26 46] INFO [2022-01-10 12:53:35,463] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:35,463] jepsen worker 62 - jepsen.util 242 :ok :read [20 42] INFO [2022-01-10 12:53:35,464] jepsen worker 83 - jepsen.util 263 :ok :write [27 46] INFO [2022-01-10 12:53:35,464] jepsen worker 62 - jepsen.util 242 :invoke :write [20 7] INFO [2022-01-10 12:53:35,465] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,465] jepsen worker 13 - jepsen.util 13 :ok :write [4 17] INFO [2022-01-10 12:53:35,466] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,466] jepsen worker 83 - jepsen.util 263 :ok :read [27 46] INFO [2022-01-10 12:53:35,466] jepsen worker 13 - jepsen.util 13 :ok :read [4 17] INFO [2022-01-10 12:53:35,467] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,467] jepsen worker 40 - jepsen.util 130 :ok :write [13 16] INFO [2022-01-10 12:53:35,467] jepsen worker 13 - jepsen.util 13 :invoke :write [4 45] INFO [2022-01-10 12:53:35,467] jepsen worker 83 - jepsen.util 263 :ok :read [27 46] INFO [2022-01-10 12:53:35,468] jepsen worker 40 - jepsen.util 130 :invoke :write [13 4] INFO [2022-01-10 12:53:35,468] jepsen worker 83 - jepsen.util 263 :invoke :write [27 48] INFO [2022-01-10 12:53:35,471] jepsen worker 3 - jepsen.util 3 :ok :write [1 18] INFO [2022-01-10 12:53:35,472] jepsen worker 3 - jepsen.util 3 :invoke :write [1 43] INFO [2022-01-10 12:53:35,472] jepsen worker 34 - jepsen.util 214 :ok :write [11 16] INFO [2022-01-10 12:53:35,473] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,473] jepsen worker 34 - jepsen.util 214 :ok :read [11 16] INFO [2022-01-10 12:53:35,474] jepsen worker 22 - jepsen.util 112 :ok :write [7 49] INFO [2022-01-10 12:53:35,474] jepsen worker 34 - jepsen.util 214 :invoke :write [11 22] INFO [2022-01-10 12:53:35,475] jepsen worker 22 - jepsen.util 112 :invoke :write [7 45] INFO [2022-01-10 12:53:35,481] jepsen worker 89 - jepsen.util 269 :ok :write [29 22] INFO [2022-01-10 12:53:35,481] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:35,481] jepsen worker 51 - jepsen.util 141 :ok :write [17 23] INFO [2022-01-10 12:53:35,482] jepsen worker 51 - jepsen.util 141 :invoke :write [17 34] INFO [2022-01-10 12:53:35,482] jepsen worker 89 - jepsen.util 269 :ok :read [29 22] INFO [2022-01-10 12:53:35,483] jepsen worker 89 - jepsen.util 269 :invoke :write [29 44] INFO [2022-01-10 12:53:35,485] jepsen worker 68 - jepsen.util 68 :ok :write [22 33] INFO [2022-01-10 12:53:35,485] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,486] jepsen worker 68 - jepsen.util 68 :ok :read [22 33] INFO [2022-01-10 12:53:35,487] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,488] jepsen worker 68 - jepsen.util 68 :ok :read [22 33] INFO [2022-01-10 12:53:35,489] jepsen worker 68 - jepsen.util 68 :invoke :write [22 29] INFO [2022-01-10 12:53:35,490] jepsen worker 44 - jepsen.util 44 :ok :write [14 49] INFO [2022-01-10 12:53:35,490] jepsen worker 7 - jepsen.util 7 :ok :write [2 43] INFO [2022-01-10 12:53:35,491] jepsen worker 44 - jepsen.util 44 :invoke :write [14 40] INFO [2022-01-10 12:53:35,491] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:35,491] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,492] jepsen worker 38 - jepsen.util 38 :invoke :write [12 25] INFO [2022-01-10 12:53:35,492] jepsen worker 7 - jepsen.util 7 :ok :read [2 43] INFO [2022-01-10 12:53:35,492] jepsen worker 7 - jepsen.util 7 :invoke :write [2 19] INFO [2022-01-10 12:53:35,495] jepsen worker 15 - jepsen.util 15 :ok :write [5 7] INFO [2022-01-10 12:53:35,496] jepsen worker 15 - jepsen.util 15 :invoke :write [5 44] INFO [2022-01-10 12:53:35,497] jepsen worker 1 - jepsen.util 91 :ok :write [0 24] INFO [2022-01-10 12:53:35,497] jepsen worker 1 - jepsen.util 91 :invoke :write [0 21] INFO [2022-01-10 12:53:35,498] jepsen worker 71 - jepsen.util 161 :ok :write [23 1] INFO [2022-01-10 12:53:35,499] jepsen worker 71 - jepsen.util 161 :invoke :write [23 31] INFO [2022-01-10 12:53:35,501] jepsen worker 75 - jepsen.util 165 :ok :write [25 42] INFO [2022-01-10 12:53:35,502] jepsen worker 50 - jepsen.util 50 :ok :write [16 6] INFO [2022-01-10 12:53:35,502] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:35,502] jepsen worker 50 - jepsen.util 50 :invoke :write [16 6] INFO [2022-01-10 12:53:35,503] jepsen worker 75 - jepsen.util 165 :ok :read [25 42] INFO [2022-01-10 12:53:35,504] jepsen worker 75 - jepsen.util 165 :invoke :write [25 26] INFO [2022-01-10 12:53:35,506] jepsen worker 62 - jepsen.util 242 :ok :write [20 7] INFO [2022-01-10 12:53:35,506] jepsen worker 84 - jepsen.util 84 :ok :write [28 23] INFO [2022-01-10 12:53:35,507] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:35,508] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:35,508] jepsen worker 83 - jepsen.util 263 :ok :write [27 48] INFO [2022-01-10 12:53:35,508] jepsen worker 84 - jepsen.util 84 :ok :read [28 23] INFO [2022-01-10 12:53:35,509] jepsen worker 83 - jepsen.util 263 :invoke :write [27 22] INFO [2022-01-10 12:53:35,509] jepsen worker 62 - jepsen.util 242 :ok :read [20 7] INFO [2022-01-10 12:53:35,509] jepsen worker 84 - jepsen.util 84 :invoke :write [28 37] INFO [2022-01-10 12:53:35,510] jepsen worker 78 - jepsen.util 168 :ok :write [26 46] INFO [2022-01-10 12:53:35,510] jepsen worker 62 - jepsen.util 242 :invoke :write [20 38] INFO [2022-01-10 12:53:35,510] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:35,511] jepsen worker 78 - jepsen.util 168 :ok :read [26 46] INFO [2022-01-10 12:53:35,512] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:35,513] jepsen worker 78 - jepsen.util 168 :ok :read [26 46] INFO [2022-01-10 12:53:35,513] jepsen worker 78 - jepsen.util 168 :invoke :write [26 28] INFO [2022-01-10 12:53:35,514] jepsen worker 40 - jepsen.util 130 :ok :write [13 4] INFO [2022-01-10 12:53:35,515] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:35,516] jepsen worker 40 - jepsen.util 130 :ok :read [13 4] INFO [2022-01-10 12:53:35,516] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:35,516] jepsen worker 13 - jepsen.util 13 :ok :write [4 45] INFO [2022-01-10 12:53:35,517] jepsen worker 40 - jepsen.util 130 :ok :read [13 4] INFO [2022-01-10 12:53:35,517] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,518] jepsen worker 40 - jepsen.util 130 :invoke :write [13 22] INFO [2022-01-10 12:53:35,518] jepsen worker 13 - jepsen.util 13 :ok :read [4 45] INFO [2022-01-10 12:53:35,519] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,519] jepsen worker 22 - jepsen.util 112 :ok :write [7 45] INFO [2022-01-10 12:53:35,520] jepsen worker 13 - jepsen.util 13 :ok :read [4 45] INFO [2022-01-10 12:53:35,520] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:35,522] jepsen worker 34 - jepsen.util 214 :ok :write [11 22] INFO [2022-01-10 12:53:35,522] jepsen worker 22 - jepsen.util 112 :ok :read [7 45] INFO [2022-01-10 12:53:35,523] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,523] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:35,524] jepsen worker 3 - jepsen.util 3 :ok :write [1 43] INFO [2022-01-10 12:53:35,524] jepsen worker 13 - jepsen.util 13 :invoke :write [4 2] INFO [2022-01-10 12:53:35,524] jepsen worker 22 - jepsen.util 112 :ok :read [7 45] INFO [2022-01-10 12:53:35,524] jepsen worker 34 - jepsen.util 214 :ok :read [11 22] INFO [2022-01-10 12:53:35,524] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:35,525] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:35,526] jepsen worker 89 - jepsen.util 269 :ok :write [29 44] INFO [2022-01-10 12:53:35,526] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,526] jepsen worker 22 - jepsen.util 112 :invoke :write [7 38] INFO [2022-01-10 12:53:35,527] jepsen worker 34 - jepsen.util 214 :ok :read [11 22] INFO [2022-01-10 12:53:35,527] jepsen worker 89 - jepsen.util 269 :invoke :write [29 5] INFO [2022-01-10 12:53:35,528] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:35,529] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,530] jepsen worker 3 - jepsen.util 3 :ok :read [1 43] INFO [2022-01-10 12:53:35,530] jepsen worker 34 - jepsen.util 214 :ok :read [11 22] INFO [2022-01-10 12:53:35,531] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,531] jepsen worker 3 - jepsen.util 3 :invoke :write [1 31] INFO [2022-01-10 12:53:35,531] jepsen worker 68 - jepsen.util 68 :ok :write [22 29] INFO [2022-01-10 12:53:35,531] jepsen worker 44 - jepsen.util 44 :ok :write [14 40] INFO [2022-01-10 12:53:35,531] jepsen worker 51 - jepsen.util 141 :ok :write [17 34] INFO [2022-01-10 12:53:35,532] jepsen worker 68 - jepsen.util 68 :invoke :write [22 10] INFO [2022-01-10 12:53:35,532] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:35,533] jepsen worker 44 - jepsen.util 44 :invoke :write [14 20] INFO [2022-01-10 12:53:35,533] jepsen worker 34 - jepsen.util 214 :ok :read [11 22] INFO [2022-01-10 12:53:35,533] jepsen worker 51 - jepsen.util 141 :ok :read [17 34] INFO [2022-01-10 12:53:35,533] jepsen worker 34 - jepsen.util 214 :invoke :write [11 23] INFO [2022-01-10 12:53:35,534] jepsen worker 51 - jepsen.util 141 :invoke :write [17 42] INFO [2022-01-10 12:53:35,538] jepsen worker 7 - jepsen.util 7 :ok :write [2 19] INFO [2022-01-10 12:53:35,539] jepsen worker 7 - jepsen.util 7 :invoke :write [2 13] INFO [2022-01-10 12:53:35,539] jepsen worker 38 - jepsen.util 38 :ok :write [12 25] INFO [2022-01-10 12:53:35,540] jepsen worker 38 - jepsen.util 38 :invoke :write [12 37] INFO [2022-01-10 12:53:35,542] jepsen worker 15 - jepsen.util 15 :ok :write [5 44] INFO [2022-01-10 12:53:35,543] jepsen worker 15 - jepsen.util 15 :invoke :write [5 28] INFO [2022-01-10 12:53:35,543] jepsen worker 75 - jepsen.util 165 :ok :write [25 26] INFO [2022-01-10 12:53:35,544] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:35,545] jepsen worker 75 - jepsen.util 165 :ok :read [25 26] INFO [2022-01-10 12:53:35,546] jepsen worker 71 - jepsen.util 161 :ok :write [23 31] INFO [2022-01-10 12:53:35,546] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:35,547] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:35,547] jepsen worker 75 - jepsen.util 165 :ok :read [25 26] INFO [2022-01-10 12:53:35,547] jepsen worker 75 - jepsen.util 165 :invoke :write [25 3] INFO [2022-01-10 12:53:35,548] jepsen worker 71 - jepsen.util 161 :ok :read [23 31] INFO [2022-01-10 12:53:35,548] jepsen worker 62 - jepsen.util 242 :ok :write [20 38] INFO [2022-01-10 12:53:35,548] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:35,549] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:35,549] jepsen worker 71 - jepsen.util 161 :ok :read [23 31] INFO [2022-01-10 12:53:35,550] jepsen worker 71 - jepsen.util 161 :invoke :write [23 38] INFO [2022-01-10 12:53:35,550] jepsen worker 62 - jepsen.util 242 :ok :read [20 38] INFO [2022-01-10 12:53:35,551] jepsen worker 1 - jepsen.util 91 :ok :write [0 21] INFO [2022-01-10 12:53:35,551] jepsen worker 62 - jepsen.util 242 :invoke :write [20 14] INFO [2022-01-10 12:53:35,551] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,552] jepsen worker 1 - jepsen.util 91 :ok :read [0 21] INFO [2022-01-10 12:53:35,553] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,554] jepsen worker 1 - jepsen.util 91 :ok :read [0 21] INFO [2022-01-10 12:53:35,554] jepsen worker 1 - jepsen.util 91 :invoke :write [0 49] INFO [2022-01-10 12:53:35,556] jepsen worker 84 - jepsen.util 84 :ok :write [28 37] INFO [2022-01-10 12:53:35,556] jepsen worker 84 - jepsen.util 84 :invoke :write [28 25] INFO [2022-01-10 12:53:35,559] jepsen worker 50 - jepsen.util 50 :ok :write [16 6] INFO [2022-01-10 12:53:35,559] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:35,560] jepsen worker 50 - jepsen.util 50 :ok :read [16 6] INFO [2022-01-10 12:53:35,560] jepsen worker 78 - jepsen.util 168 :ok :write [26 28] INFO [2022-01-10 12:53:35,560] jepsen worker 22 - jepsen.util 112 :ok :write [7 38] INFO [2022-01-10 12:53:35,561] jepsen worker 50 - jepsen.util 50 :invoke :write [16 38] INFO [2022-01-10 12:53:35,561] jepsen worker 78 - jepsen.util 168 :invoke :write [26 40] INFO [2022-01-10 12:53:35,562] jepsen worker 22 - jepsen.util 112 :invoke :write [7 31] INFO [2022-01-10 12:53:35,565] jepsen worker 83 - jepsen.util 263 :ok :write [27 22] INFO [2022-01-10 12:53:35,566] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,567] jepsen worker 83 - jepsen.util 263 :ok :read [27 22] INFO [2022-01-10 12:53:35,567] jepsen worker 83 - jepsen.util 263 :invoke :write [27 3] INFO [2022-01-10 12:53:35,568] jepsen worker 13 - jepsen.util 13 :ok :write [4 2] INFO [2022-01-10 12:53:35,568] jepsen worker 13 - jepsen.util 13 :invoke :write [4 32] INFO [2022-01-10 12:53:35,571] jepsen worker 40 - jepsen.util 130 :ok :write [13 22] INFO [2022-01-10 12:53:35,571] jepsen worker 40 - jepsen.util 130 :invoke :write [13 17] INFO [2022-01-10 12:53:35,572] jepsen worker 44 - jepsen.util 44 :ok :write [14 20] INFO [2022-01-10 12:53:35,572] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:35,573] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:35,574] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:35,574] jepsen worker 3 - jepsen.util 3 :ok :write [1 31] INFO [2022-01-10 12:53:35,574] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:35,575] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:35,575] jepsen worker 3 - jepsen.util 3 :ok :read [1 31] INFO [2022-01-10 12:53:35,576] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:35,576] jepsen worker 3 - jepsen.util 3 :invoke :write [1 1] INFO [2022-01-10 12:53:35,577] jepsen worker 34 - jepsen.util 214 :ok :write [11 23] INFO [2022-01-10 12:53:35,577] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:35,577] jepsen worker 44 - jepsen.util 44 :invoke :write [14 3] INFO [2022-01-10 12:53:35,578] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,579] jepsen worker 34 - jepsen.util 214 :ok :read [11 23] INFO [2022-01-10 12:53:35,579] jepsen worker 75 - jepsen.util 165 :ok :write [25 3] INFO [2022-01-10 12:53:35,579] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,580] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:35,580] jepsen worker 34 - jepsen.util 214 :ok :read [11 23] INFO [2022-01-10 12:53:35,580] jepsen worker 75 - jepsen.util 165 :ok :read [25 3] INFO [2022-01-10 12:53:35,581] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,581] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:35,582] jepsen worker 34 - jepsen.util 214 :ok :read [11 23] INFO [2022-01-10 12:53:35,582] jepsen worker 51 - jepsen.util 141 :ok :write [17 42] INFO [2022-01-10 12:53:35,582] jepsen worker 75 - jepsen.util 165 :ok :read [25 3] INFO [2022-01-10 12:53:35,582] jepsen worker 34 - jepsen.util 214 :invoke :write [11 30] INFO [2022-01-10 12:53:35,583] jepsen worker 89 - jepsen.util 269 :ok :write [29 5] INFO [2022-01-10 12:53:35,583] jepsen worker 51 - jepsen.util 141 :invoke :write [17 27] INFO [2022-01-10 12:53:35,584] jepsen worker 75 - jepsen.util 165 :invoke :write [25 35] INFO [2022-01-10 12:53:35,584] jepsen worker 89 - jepsen.util 269 :invoke :write [29 24] INFO [2022-01-10 12:53:35,586] jepsen worker 62 - jepsen.util 242 :ok :write [20 14] INFO [2022-01-10 12:53:35,587] jepsen worker 62 - jepsen.util 242 :invoke :write [20 16] INFO [2022-01-10 12:53:35,588] jepsen worker 68 - jepsen.util 68 :ok :write [22 10] INFO [2022-01-10 12:53:35,589] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,590] jepsen worker 68 - jepsen.util 68 :ok :read [22 10] INFO [2022-01-10 12:53:35,590] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,591] jepsen worker 68 - jepsen.util 68 :ok :read [22 10] INFO [2022-01-10 12:53:35,592] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,592] jepsen worker 38 - jepsen.util 38 :ok :write [12 37] INFO [2022-01-10 12:53:35,592] jepsen worker 38 - jepsen.util 38 :invoke :write [12 30] INFO [2022-01-10 12:53:35,592] jepsen worker 68 - jepsen.util 68 :ok :read [22 10] INFO [2022-01-10 12:53:35,593] jepsen worker 68 - jepsen.util 68 :invoke :write [22 36] INFO [2022-01-10 12:53:35,594] jepsen worker 7 - jepsen.util 7 :ok :write [2 13] INFO [2022-01-10 12:53:35,595] jepsen worker 7 - jepsen.util 7 :invoke :write [2 13] INFO [2022-01-10 12:53:35,598] jepsen worker 22 - jepsen.util 112 :ok :write [7 31] INFO [2022-01-10 12:53:35,599] jepsen worker 71 - jepsen.util 161 :ok :write [23 38] INFO [2022-01-10 12:53:35,599] jepsen worker 15 - jepsen.util 15 :ok :write [5 28] INFO [2022-01-10 12:53:35,599] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:35,600] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,600] jepsen worker 22 - jepsen.util 112 :ok :read [7 31] INFO [2022-01-10 12:53:35,601] jepsen worker 71 - jepsen.util 161 :invoke :write [23 11] INFO [2022-01-10 12:53:35,602] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:35,602] jepsen worker 22 - jepsen.util 112 :invoke :write [7 6] INFO [2022-01-10 12:53:35,603] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,603] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:35,604] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,605] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:35,605] jepsen worker 84 - jepsen.util 84 :ok :write [28 25] INFO [2022-01-10 12:53:35,606] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,606] jepsen worker 1 - jepsen.util 91 :ok :write [0 49] INFO [2022-01-10 12:53:35,606] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:35,606] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:35,607] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,607] jepsen worker 84 - jepsen.util 84 :ok :read [28 25] INFO [2022-01-10 12:53:35,607] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,608] jepsen worker 1 - jepsen.util 91 :ok :read [0 49] INFO [2022-01-10 12:53:35,608] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:35,608] jepsen worker 15 - jepsen.util 15 :ok :read [5 28] INFO [2022-01-10 12:53:35,609] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,609] jepsen worker 84 - jepsen.util 84 :ok :read [28 25] INFO [2022-01-10 12:53:35,609] jepsen worker 15 - jepsen.util 15 :invoke :write [5 19] INFO [2022-01-10 12:53:35,610] jepsen worker 1 - jepsen.util 91 :ok :read [0 49] INFO [2022-01-10 12:53:35,610] jepsen worker 84 - jepsen.util 84 :invoke :write [28 13] INFO [2022-01-10 12:53:35,610] jepsen worker 1 - jepsen.util 91 :invoke :write [0 46] INFO [2022-01-10 12:53:35,612] jepsen worker 44 - jepsen.util 44 :ok :write [14 3] INFO [2022-01-10 12:53:35,612] jepsen worker 78 - jepsen.util 168 :ok :write [26 40] INFO [2022-01-10 12:53:35,613] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:35,613] jepsen worker 50 - jepsen.util 50 :ok :write [16 38] INFO [2022-01-10 12:53:35,613] jepsen worker 50 - jepsen.util 50 :invoke :write [16 20] INFO [2022-01-10 12:53:35,613] jepsen worker 44 - jepsen.util 44 :ok :read [14 3] INFO [2022-01-10 12:53:35,614] jepsen worker 78 - jepsen.util 168 :invoke :write [26 48] INFO [2022-01-10 12:53:35,615] jepsen worker 44 - jepsen.util 44 :invoke :write [14 9] INFO [2022-01-10 12:53:35,620] jepsen worker 75 - jepsen.util 165 :ok :write [25 35] INFO [2022-01-10 12:53:35,620] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:35,620] jepsen worker 83 - jepsen.util 263 :ok :write [27 3] INFO [2022-01-10 12:53:35,620] jepsen worker 13 - jepsen.util 13 :ok :write [4 32] INFO [2022-01-10 12:53:35,621] jepsen worker 75 - jepsen.util 165 :ok :read [25 35] INFO [2022-01-10 12:53:35,622] jepsen worker 83 - jepsen.util 263 :invoke :write [27 26] INFO [2022-01-10 12:53:35,622] jepsen worker 75 - jepsen.util 165 :invoke :write [25 45] INFO [2022-01-10 12:53:35,623] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,624] jepsen worker 13 - jepsen.util 13 :ok :read [4 32] INFO [2022-01-10 12:53:35,624] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,625] jepsen worker 40 - jepsen.util 130 :ok :write [13 17] INFO [2022-01-10 12:53:35,625] jepsen worker 13 - jepsen.util 13 :ok :read [4 32] INFO [2022-01-10 12:53:35,625] jepsen worker 40 - jepsen.util 130 :invoke :write [13 15] INFO [2022-01-10 12:53:35,626] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,627] jepsen worker 13 - jepsen.util 13 :ok :read [4 32] INFO [2022-01-10 12:53:35,627] jepsen worker 13 - jepsen.util 13 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,628] jepsen worker 13 - jepsen.util 13 :ok :read [4 32] INFO [2022-01-10 12:53:35,628] jepsen worker 13 - jepsen.util 13 :invoke :write [4 17] INFO [2022-01-10 12:53:35,629] jepsen worker 3 - jepsen.util 3 :ok :write [1 1] INFO [2022-01-10 12:53:35,629] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:35,630] jepsen worker 62 - jepsen.util 242 :ok :write [20 16] INFO [2022-01-10 12:53:35,630] jepsen worker 34 - jepsen.util 214 :ok :write [11 30] INFO [2022-01-10 12:53:35,630] jepsen worker 3 - jepsen.util 3 :ok :read [1 1] INFO [2022-01-10 12:53:35,630] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,631] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:35,631] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:35,631] jepsen worker 34 - jepsen.util 214 :ok :read [11 30] INFO [2022-01-10 12:53:35,632] jepsen worker 62 - jepsen.util 242 :ok :read [20 16] INFO [2022-01-10 12:53:35,632] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,632] jepsen worker 3 - jepsen.util 3 :ok :read [1 1] INFO [2022-01-10 12:53:35,632] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:35,633] jepsen worker 34 - jepsen.util 214 :ok :read [11 30] INFO [2022-01-10 12:53:35,633] jepsen worker 89 - jepsen.util 269 :ok :write [29 24] INFO [2022-01-10 12:53:35,633] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:35,633] jepsen worker 62 - jepsen.util 242 :ok :read [20 16] INFO [2022-01-10 12:53:35,634] jepsen worker 89 - jepsen.util 269 :invoke :write [29 28] INFO [2022-01-10 12:53:35,634] jepsen worker 3 - jepsen.util 3 :ok :read [1 1] INFO [2022-01-10 12:53:35,634] jepsen worker 34 - jepsen.util 214 :invoke :write [11 11] INFO [2022-01-10 12:53:35,634] jepsen worker 62 - jepsen.util 242 :invoke :write [20 1] INFO [2022-01-10 12:53:35,635] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:35,636] jepsen worker 3 - jepsen.util 3 :ok :read [1 1] INFO [2022-01-10 12:53:35,636] jepsen worker 3 - jepsen.util 3 :invoke :write [1 15] INFO [2022-01-10 12:53:35,637] jepsen worker 51 - jepsen.util 141 :ok :write [17 27] INFO [2022-01-10 12:53:35,638] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:35,638] jepsen worker 68 - jepsen.util 68 :ok :write [22 36] INFO [2022-01-10 12:53:35,638] jepsen worker 51 - jepsen.util 141 :ok :read [17 27] INFO [2022-01-10 12:53:35,639] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,639] jepsen worker 51 - jepsen.util 141 :invoke :write [17 17] INFO [2022-01-10 12:53:35,640] jepsen worker 68 - jepsen.util 68 :ok :read [22 36] INFO [2022-01-10 12:53:35,640] jepsen worker 68 - jepsen.util 68 :invoke :write [22 14] INFO [2022-01-10 12:53:35,642] jepsen worker 22 - jepsen.util 112 :ok :write [7 6] INFO [2022-01-10 12:53:35,643] jepsen worker 22 - jepsen.util 112 :invoke :write [7 40] INFO [2022-01-10 12:53:35,646] jepsen worker 38 - jepsen.util 38 :ok :write [12 30] INFO [2022-01-10 12:53:35,647] jepsen worker 7 - jepsen.util 7 :ok :write [2 13] INFO [2022-01-10 12:53:35,647] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:35,647] jepsen worker 7 - jepsen.util 7 :invoke :write [2 18] INFO [2022-01-10 12:53:35,648] jepsen worker 38 - jepsen.util 38 :ok :read [12 30] WARN [2022-01-10 12:53:35,649] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9005825,"last_local_id":"2B94F89700000002/00000000E26E7095","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":9005825,"last_local_socket":"172.23.104.79:58606","total_server_duration_us":62,"operation_id":"0x307f","timeout_ms":2000,"last_server_duration_us":62,"total_duration_us":2002415},{"operation_name":"upsert","last_dispatch_duration_us":15042813,"last_local_id":"2B94F89700000002/00000000E26E7095","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":15058346,"last_local_socket":"172.23.104.79:58606","total_server_duration_us":9008985,"encode_duration_us":25,"operation_id":"0x1f7d","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10004000},{"operation_name":"upsert","last_dispatch_duration_us":96671,"last_local_id":"2B94F89700000002/000000004AD6A961","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":844113,"last_local_socket":"172.23.104.79:48334","total_server_duration_us":99538,"encode_duration_us":39,"operation_id":"0x32df","timeout_ms":10000,"last_server_duration_us":96096,"total_duration_us":10005798}],"total_count":3}} INFO [2022-01-10 12:53:35,649] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] WARN [2022-01-10 12:53:35,649] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":7865816,"last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":7865816,"last_local_socket":"172.23.104.79:40254","total_server_duration_us":19,"operation_id":"0x3295","timeout_ms":2000,"last_server_duration_us":19,"total_duration_us":2005621},{"operation_name":"upsert","last_dispatch_duration_us":12860229,"last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":12871856,"last_local_socket":"172.23.104.79:40254","total_server_duration_us":9009130,"encode_duration_us":25,"operation_id":"0x20e2","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10010524},{"operation_name":"upsert","last_dispatch_duration_us":12850121,"last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":12861942,"last_local_socket":"172.23.104.79:40254","total_server_duration_us":9008033,"encode_duration_us":29,"operation_id":"0x20e6","timeout_ms":10000,"last_server_duration_us":9006609,"total_duration_us":10005755}],"total_count":3}} WARN [2022-01-10 12:53:35,649] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":528,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000002/00000000F0645BC3","last_local_socket":"172.23.104.79:34668","total_dispatch_duration_us":560168,"total_server_duration_us":4239,"encode_duration_us":63,"operation_id":"0x3587","timeout_ms":10000,"last_server_duration_us":84,"total_duration_us":10010565},{"operation_name":"upsert","last_dispatch_duration_us":502,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":17258,"total_server_duration_us":2503,"encode_duration_us":37,"operation_id":"0x3237","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10010423},{"operation_name":"upsert","last_dispatch_duration_us":821,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":18475,"total_server_duration_us":2778,"encode_duration_us":30,"operation_id":"0x3158","timeout_ms":10000,"last_server_duration_us":117,"total_duration_us":10009815},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","encode_duration_us":27,"operation_id":"0x33e3","timeout_ms":10000,"total_duration_us":10008803},{"operation_name":"upsert","last_dispatch_duration_us":628,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":17488,"total_server_duration_us":2528,"encode_duration_us":23,"operation_id":"0x30fb","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10007514},{"operation_name":"upsert","last_dispatch_duration_us":860555,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":1029896,"total_server_duration_us":3262,"encode_duration_us":38,"operation_id":"0x2d15","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10007476},{"operation_name":"upsert","last_dispatch_duration_us":850,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":757531,"total_server_duration_us":3468,"encode_duration_us":19,"operation_id":"0x30ed","timeout_ms":10000,"last_server_duration_us":243,"total_duration_us":10006709},{"operation_name":"upsert","last_dispatch_duration_us":96671,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":844113,"total_server_duration_us":99538,"encode_duration_us":39,"operation_id":"0x32df","timeout_ms":10000,"last_server_duration_us":96096,"total_duration_us":10005798},{"operation_name":"upsert","last_dispatch_duration_us":665,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":19070,"total_server_duration_us":2741,"encode_duration_us":27,"operation_id":"0x30b8","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10004340},{"operation_name":"upsert","last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","encode_duration_us":21,"operation_id":"0x384f","timeout_ms":10000,"total_duration_us":10001398}],"total_count":18}} WARN [2022-01-10 12:53:35,649] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":7005644,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":7017403,"total_server_duration_us":1927,"encode_duration_us":85,"operation_id":"0x2c65","timeout_ms":10000,"last_server_duration_us":56,"total_duration_us":10010179},{"operation_name":"upsert","last_dispatch_duration_us":4174975,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":4174975,"total_server_duration_us":38,"encode_duration_us":20,"operation_id":"0x3872","timeout_ms":10000,"last_server_duration_us":38,"total_duration_us":10010049},{"operation_name":"upsert","last_dispatch_duration_us":806,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":8866664,"total_server_duration_us":567,"encode_duration_us":23,"operation_id":"0x30b9","timeout_ms":10000,"last_server_duration_us":100,"total_duration_us":10009010},{"operation_name":"upsert","last_dispatch_duration_us":676,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":7450,"total_server_duration_us":1011,"encode_duration_us":20,"operation_id":"0x30ee","timeout_ms":10000,"last_server_duration_us":186,"total_duration_us":10008226},{"operation_name":"upsert","last_dispatch_duration_us":743,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":8640020,"total_server_duration_us":910,"encode_duration_us":22,"operation_id":"0x3100","timeout_ms":10000,"last_server_duration_us":100,"total_duration_us":10008181},{"operation_name":"upsert","last_dispatch_duration_us":930,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":5871224,"total_server_duration_us":788,"encode_duration_us":20,"operation_id":"0x35a3","timeout_ms":10000,"last_server_duration_us":100,"total_duration_us":10007299},{"operation_name":"upsert","last_dispatch_duration_us":7116024,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":7122834,"total_server_duration_us":906,"encode_duration_us":19,"operation_id":"0x30ec","timeout_ms":10000,"last_server_duration_us":56,"total_duration_us":10005331},{"operation_name":"upsert","last_dispatch_duration_us":949,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":8005726,"total_server_duration_us":46,"encode_duration_us":28,"operation_id":"0x323a","timeout_ms":10000,"last_server_duration_us":23,"total_duration_us":10004090},{"operation_name":"upsert","last_dispatch_duration_us":7149097,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":7149097,"total_server_duration_us":334,"encode_duration_us":22,"operation_id":"0x33b5","timeout_ms":10000,"last_server_duration_us":334,"total_duration_us":10003622},{"operation_name":"upsert","last_dispatch_duration_us":18117,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":38796,"total_server_duration_us":20063,"encode_duration_us":27,"operation_id":"0x3884","timeout_ms":10000,"last_server_duration_us":17588,"total_duration_us":9264654}],"total_count":15}} INFO [2022-01-10 12:53:35,650] jepsen worker 38 - jepsen.util 38 :ok :read [12 30] INFO [2022-01-10 12:53:35,650] jepsen worker 38 - jepsen.util 38 :invoke :write [12 14] INFO [2022-01-10 12:53:35,652] jepsen worker 15 - jepsen.util 15 :ok :write [5 19] INFO [2022-01-10 12:53:35,652] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,653] jepsen worker 15 - jepsen.util 15 :ok :read [5 19] INFO [2022-01-10 12:53:35,653] jepsen worker 71 - jepsen.util 161 :ok :write [23 11] INFO [2022-01-10 12:53:35,654] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:35,655] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,655] jepsen worker 71 - jepsen.util 161 :ok :read [23 11] INFO [2022-01-10 12:53:35,655] jepsen worker 44 - jepsen.util 44 :ok :write [14 9] INFO [2022-01-10 12:53:35,656] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:35,656] jepsen worker 15 - jepsen.util 15 :ok :read [5 19] INFO [2022-01-10 12:53:35,656] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:35,657] jepsen worker 71 - jepsen.util 161 :ok :read [23 11] INFO [2022-01-10 12:53:35,657] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,657] jepsen worker 44 - jepsen.util 44 :ok :read [14 9] INFO [2022-01-10 12:53:35,657] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:35,658] jepsen worker 15 - jepsen.util 15 :ok :read [5 19] INFO [2022-01-10 12:53:35,658] jepsen worker 44 - jepsen.util 44 :invoke :write [14 0] INFO [2022-01-10 12:53:35,658] jepsen worker 71 - jepsen.util 161 :ok :read [23 11] INFO [2022-01-10 12:53:35,658] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,659] jepsen worker 71 - jepsen.util 161 :invoke :write [23 14] INFO [2022-01-10 12:53:35,659] jepsen worker 1 - jepsen.util 91 :ok :write [0 46] INFO [2022-01-10 12:53:35,660] jepsen worker 15 - jepsen.util 15 :ok :read [5 19] INFO [2022-01-10 12:53:35,660] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,661] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,661] jepsen worker 15 - jepsen.util 15 :ok :read [5 19] INFO [2022-01-10 12:53:35,662] jepsen worker 1 - jepsen.util 91 :ok :read [0 46] INFO [2022-01-10 12:53:35,662] jepsen worker 15 - jepsen.util 15 :invoke :write [5 10] INFO [2022-01-10 12:53:35,662] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,663] jepsen worker 1 - jepsen.util 91 :ok :read [0 46] INFO [2022-01-10 12:53:35,664] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,665] jepsen worker 1 - jepsen.util 91 :ok :read [0 46] INFO [2022-01-10 12:53:35,665] jepsen worker 1 - jepsen.util 91 :invoke :write [0 39] INFO [2022-01-10 12:53:35,692] jepsen worker 74 - jepsen.util 254 :info :write [24 20] :RequestTimeoutException INFO [2022-01-10 12:53:35,692] jepsen worker 74 - jepsen.util 344 :invoke :read [24 nil] INFO [2022-01-10 12:53:35,742] jepsen worker 50 - jepsen.util 50 :ok :write [16 20] INFO [2022-01-10 12:53:35,742] jepsen worker 84 - jepsen.util 84 :ok :write [28 13] INFO [2022-01-10 12:53:35,743] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:35,743] jepsen worker 84 - jepsen.util 84 :invoke :write [28 36] INFO [2022-01-10 12:53:35,744] jepsen worker 50 - jepsen.util 50 :ok :read [16 20] INFO [2022-01-10 12:53:35,744] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:35,745] jepsen worker 50 - jepsen.util 50 :ok :read [16 20] INFO [2022-01-10 12:53:35,746] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:35,747] jepsen worker 75 - jepsen.util 165 :ok :write [25 45] INFO [2022-01-10 12:53:35,747] jepsen worker 50 - jepsen.util 50 :ok :read [16 20] INFO [2022-01-10 12:53:35,747] jepsen worker 78 - jepsen.util 168 :ok :write [26 48] INFO [2022-01-10 12:53:35,748] jepsen worker 50 - jepsen.util 50 :invoke :write [16 22] WARN [2022-01-10 12:53:35,748] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":497,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000004/0000000048226CCA","last_local_socket":"172.23.104.79:48384","total_dispatch_duration_us":18519,"total_server_duration_us":2353,"encode_duration_us":61,"operation_id":"0x3383","timeout_ms":10000,"last_server_duration_us":49,"total_duration_us":10009937},{"operation_name":"upsert","last_dispatch_duration_us":743,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":322603,"total_server_duration_us":2015,"encode_duration_us":24,"operation_id":"0x300d","timeout_ms":10000,"last_server_duration_us":100,"total_duration_us":10008599},{"operation_name":"upsert","last_dispatch_duration_us":612,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":6922421,"total_server_duration_us":1387,"encode_duration_us":31,"operation_id":"0x3247","timeout_ms":10000,"last_server_duration_us":100,"total_duration_us":10007171},{"operation_name":"upsert","last_dispatch_duration_us":619,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000004/0000000048226CCA","last_local_socket":"172.23.104.79:48384","total_dispatch_duration_us":19024,"total_server_duration_us":3118,"encode_duration_us":26,"operation_id":"0x2f77","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10005933},{"operation_name":"upsert","last_dispatch_duration_us":896,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":7042766,"total_server_duration_us":1327,"encode_duration_us":15,"operation_id":"0x310f","timeout_ms":10000,"last_server_duration_us":108,"total_duration_us":10003959},{"operation_name":"upsert","last_dispatch_duration_us":910,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":6733668,"total_server_duration_us":1278,"encode_duration_us":60,"operation_id":"0x30c0","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10003444},{"operation_name":"upsert","last_dispatch_duration_us":535,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000004/0000000048226CCA","last_local_socket":"172.23.104.79:48384","total_dispatch_duration_us":19312,"total_server_duration_us":2779,"encode_duration_us":29,"operation_id":"0x322a","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10000699},{"operation_name":"upsert","last_dispatch_duration_us":9011906,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":9011906,"total_server_duration_us":9010849,"encode_duration_us":27,"operation_id":"0x3076","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":9012246},{"operation_name":"upsert","last_dispatch_duration_us":60417,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000004/00000000C4B4FBBF","last_local_socket":"172.23.104.79:58638","total_dispatch_duration_us":129749,"total_server_duration_us":62020,"encode_duration_us":42,"operation_id":"0x2d56","timeout_ms":10000,"last_server_duration_us":59625,"total_duration_us":7827793},{"operation_name":"upsert","last_dispatch_duration_us":106441,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000004/00000000C4B4FBBF","last_local_socket":"172.23.104.79:58638","total_dispatch_duration_us":121940,"total_server_duration_us":108475,"encode_duration_us":24,"operation_id":"0x3162","timeout_ms":10000,"last_server_duration_us":105824,"total_duration_us":6818185}],"total_count":14}} INFO [2022-01-10 12:53:35,748] jepsen worker 75 - jepsen.util 165 :invoke :write [25 26] INFO [2022-01-10 12:53:35,748] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:35,749] jepsen worker 40 - jepsen.util 130 :ok :write [13 15] INFO [2022-01-10 12:53:35,750] jepsen worker 78 - jepsen.util 168 :ok :read [26 48] INFO [2022-01-10 12:53:35,750] jepsen worker 40 - jepsen.util 130 :invoke :write [13 46] INFO [2022-01-10 12:53:35,750] jepsen worker 78 - jepsen.util 168 :invoke :write [26 1] INFO [2022-01-10 12:53:35,752] jepsen worker 83 - jepsen.util 263 :ok :write [27 26] INFO [2022-01-10 12:53:35,753] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:35,754] jepsen worker 83 - jepsen.util 263 :ok :read [27 26] INFO [2022-01-10 12:53:35,754] jepsen worker 14 - jepsen.util 284 :ok :write [4 5] INFO [2022-01-10 12:53:35,754] jepsen worker 83 - jepsen.util 263 :invoke :write [27 30] INFO [2022-01-10 12:53:35,755] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,756] jepsen worker 14 - jepsen.util 284 :ok :read [4 5] INFO [2022-01-10 12:53:35,756] jepsen worker 14 - jepsen.util 284 :invoke :write [4 44] INFO [2022-01-10 12:53:35,760] jepsen worker 34 - jepsen.util 214 :ok :write [11 11] INFO [2022-01-10 12:53:35,760] jepsen worker 34 - jepsen.util 214 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,761] jepsen worker 89 - jepsen.util 269 :ok :write [29 28] INFO [2022-01-10 12:53:35,761] jepsen worker 34 - jepsen.util 214 :ok :read [11 11] INFO [2022-01-10 12:53:35,761] jepsen worker 3 - jepsen.util 3 :ok :write [1 15] INFO [2022-01-10 12:53:35,761] jepsen worker 89 - jepsen.util 269 :invoke :write [29 12] INFO [2022-01-10 12:53:35,762] jepsen worker 34 - jepsen.util 214 :invoke :write [11 1] INFO [2022-01-10 12:53:35,762] jepsen worker 3 - jepsen.util 3 :invoke :write [1 24] INFO [2022-01-10 12:53:35,767] jepsen worker 51 - jepsen.util 141 :ok :write [17 17] INFO [2022-01-10 12:53:35,768] jepsen worker 51 - jepsen.util 141 :invoke :write [17 22] INFO [2022-01-10 12:53:35,769] jepsen worker 62 - jepsen.util 242 :ok :write [20 1] INFO [2022-01-10 12:53:35,770] jepsen worker 62 - jepsen.util 242 :invoke :write [20 17] INFO [2022-01-10 12:53:35,773] jepsen worker 38 - jepsen.util 38 :ok :write [12 14] INFO [2022-01-10 12:53:35,773] jepsen worker 38 - jepsen.util 38 :invoke :write [12 6] INFO [2022-01-10 12:53:35,777] jepsen worker 68 - jepsen.util 68 :ok :write [22 14] INFO [2022-01-10 12:53:35,777] jepsen worker 68 - jepsen.util 68 :invoke :write [22 6] INFO [2022-01-10 12:53:35,778] jepsen worker 71 - jepsen.util 161 :ok :write [23 14] INFO [2022-01-10 12:53:35,778] jepsen worker 1 - jepsen.util 91 :ok :write [0 39] INFO [2022-01-10 12:53:35,779] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,779] jepsen worker 22 - jepsen.util 112 :ok :write [7 40] INFO [2022-01-10 12:53:35,779] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:35,780] jepsen worker 1 - jepsen.util 91 :ok :read [0 39] INFO [2022-01-10 12:53:35,780] jepsen worker 22 - jepsen.util 112 :invoke :write [7 8] INFO [2022-01-10 12:53:35,781] jepsen worker 71 - jepsen.util 161 :ok :read [23 14] INFO [2022-01-10 12:53:35,781] jepsen worker 1 - jepsen.util 91 :invoke :write [0 27] INFO [2022-01-10 12:53:35,781] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:35,782] jepsen worker 71 - jepsen.util 161 :ok :read [23 14] INFO [2022-01-10 12:53:35,783] jepsen worker 71 - jepsen.util 161 :invoke :write [23 16] INFO [2022-01-10 12:53:35,786] jepsen worker 7 - jepsen.util 7 :ok :write [2 18] INFO [2022-01-10 12:53:35,788] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,788] jepsen worker 29 - jepsen.util 299 :fail :read [9 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:35,789] jepsen worker 7 - jepsen.util 7 :ok :read [2 18] INFO [2022-01-10 12:53:35,789] jepsen worker 29 - jepsen.util 299 :invoke :read [9 nil] INFO [2022-01-10 12:53:35,789] jepsen worker 7 - jepsen.util 7 :invoke :write [2 17] INFO [2022-01-10 12:53:35,791] jepsen worker 78 - jepsen.util 168 :ok :write [26 1] INFO [2022-01-10 12:53:35,792] jepsen worker 15 - jepsen.util 15 :ok :write [5 10] INFO [2022-01-10 12:53:35,792] jepsen worker 78 - jepsen.util 168 :invoke :write [26 23] INFO [2022-01-10 12:53:35,793] jepsen worker 15 - jepsen.util 15 :invoke :write [5 6] INFO [2022-01-10 12:53:35,797] jepsen worker 14 - jepsen.util 284 :ok :write [4 44] INFO [2022-01-10 12:53:35,797] jepsen worker 44 - jepsen.util 44 :ok :write [14 0] INFO [2022-01-10 12:53:35,798] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,799] jepsen worker 44 - jepsen.util 44 :invoke :write [14 12] INFO [2022-01-10 12:53:35,799] jepsen worker 14 - jepsen.util 284 :ok :read [4 44] INFO [2022-01-10 12:53:35,800] jepsen worker 14 - jepsen.util 284 :invoke :write [4 13] INFO [2022-01-10 12:53:35,808] jepsen worker 75 - jepsen.util 165 :ok :write [25 26] INFO [2022-01-10 12:53:35,808] jepsen worker 51 - jepsen.util 141 :ok :write [17 22] INFO [2022-01-10 12:53:35,809] jepsen worker 51 - jepsen.util 141 :invoke :write [17 36] INFO [2022-01-10 12:53:35,809] jepsen worker 75 - jepsen.util 165 :invoke :write [25 39] INFO [2022-01-10 12:53:35,819] jepsen worker 71 - jepsen.util 161 :ok :write [23 16] INFO [2022-01-10 12:53:35,820] jepsen worker 71 - jepsen.util 161 :invoke :write [23 14] INFO [2022-01-10 12:53:35,823] jepsen worker 62 - jepsen.util 242 :ok :write [20 17] INFO [2022-01-10 12:53:35,823] jepsen worker 78 - jepsen.util 168 :ok :write [26 23] INFO [2022-01-10 12:53:35,823] jepsen worker 62 - jepsen.util 242 :invoke :write [20 14] INFO [2022-01-10 12:53:35,824] jepsen worker 78 - jepsen.util 168 :invoke :write [26 17] INFO [2022-01-10 12:53:35,828] jepsen worker 14 - jepsen.util 284 :ok :write [4 13] INFO [2022-01-10 12:53:35,829] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,830] jepsen worker 14 - jepsen.util 284 :ok :read [4 13] INFO [2022-01-10 12:53:35,830] jepsen worker 22 - jepsen.util 112 :ok :write [7 8] INFO [2022-01-10 12:53:35,830] jepsen worker 14 - jepsen.util 284 :invoke :write [4 25] INFO [2022-01-10 12:53:35,831] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:35,831] jepsen worker 22 - jepsen.util 112 :ok :read [7 8] INFO [2022-01-10 12:53:35,832] jepsen worker 22 - jepsen.util 112 :invoke :write [7 11] INFO [2022-01-10 12:53:35,835] jepsen worker 51 - jepsen.util 141 :ok :write [17 36] INFO [2022-01-10 12:53:35,836] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:35,837] jepsen worker 51 - jepsen.util 141 :ok :read [17 36] INFO [2022-01-10 12:53:35,837] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:35,838] jepsen worker 51 - jepsen.util 141 :ok :read [17 36] INFO [2022-01-10 12:53:35,839] jepsen worker 51 - jepsen.util 141 :invoke :write [17 12] INFO [2022-01-10 12:53:35,841] jepsen worker 71 - jepsen.util 161 :ok :write [23 14] INFO [2022-01-10 12:53:35,841] jepsen worker 71 - jepsen.util 161 :invoke :write [23 42] INFO [2022-01-10 12:53:35,845] jepsen worker 78 - jepsen.util 168 :ok :write [26 17] INFO [2022-01-10 12:53:35,845] jepsen worker 78 - jepsen.util 168 :invoke :write [26 43] WARN [2022-01-10 12:53:35,848] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":8071197,"last_local_id":"2B94F89700000006/00000000A79E4C0A","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":8071197,"last_local_socket":"172.23.104.79:34662","total_server_duration_us":19,"operation_id":"0x3209","timeout_ms":2000,"last_server_duration_us":19,"total_duration_us":2003759},{"operation_name":"get","last_dispatch_duration_us":6964626,"last_local_id":"2B94F89700000006/000000001E9CFABF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":6964626,"last_local_socket":"172.23.104.79:40226","total_server_duration_us":32,"operation_id":"0x3400","timeout_ms":2000,"last_server_duration_us":32,"total_duration_us":2008227},{"operation_name":"get","last_dispatch_duration_us":8975441,"last_local_id":"2B94F89700000006/000000001E9CFABF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":8975441,"last_local_socket":"172.23.104.79:40226","total_server_duration_us":32,"operation_id":"0x30cf","timeout_ms":2000,"last_server_duration_us":32,"total_duration_us":2008844},{"operation_name":"upsert","last_dispatch_duration_us":15174710,"last_local_id":"2B94F89700000006/000000001E9CFABF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":15184543,"last_local_socket":"172.23.104.79:40226","total_server_duration_us":9017156,"encode_duration_us":51,"operation_id":"0x1fa2","timeout_ms":10000,"last_server_duration_us":9016150,"total_duration_us":10002346},{"operation_name":"get","last_dispatch_duration_us":8215928,"last_local_id":"2B94F89700000006/000000001E9CFABF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":8215928,"last_local_socket":"172.23.104.79:40226","total_server_duration_us":15,"operation_id":"0x322c","timeout_ms":2000,"last_server_duration_us":15,"total_duration_us":2010024}],"total_count":5}} WARN [2022-01-10 12:53:35,848] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":8078266,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":8078266,"total_server_duration_us":62,"encode_duration_us":19,"operation_id":"0x3206","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10010543},{"operation_name":"upsert","last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000006/000000001E9CFABF","last_local_socket":"172.23.104.79:40226","encode_duration_us":56,"operation_id":"0x3262","timeout_ms":10000,"total_duration_us":10010053},{"operation_name":"upsert","last_dispatch_duration_us":6064184,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":6064184,"total_server_duration_us":38,"encode_duration_us":33,"operation_id":"0x3545","timeout_ms":10000,"last_server_duration_us":38,"total_duration_us":10006775},{"operation_name":"upsert","last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000006/000000001E9CFABF","last_local_socket":"172.23.104.79:40226","encode_duration_us":24,"operation_id":"0x378b","timeout_ms":10000,"total_duration_us":10005981},{"operation_name":"upsert","last_dispatch_duration_us":6202712,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000006/000000001E9CFABF","last_local_socket":"172.23.104.79:40226","total_dispatch_duration_us":6202712,"total_server_duration_us":255,"encode_duration_us":20,"operation_id":"0x356f","timeout_ms":10000,"last_server_duration_us":255,"total_duration_us":10005959},{"operation_name":"upsert","last_dispatch_duration_us":689,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":32613,"total_server_duration_us":2862,"encode_duration_us":26,"operation_id":"0x30f3","timeout_ms":10000,"last_server_duration_us":117,"total_duration_us":10005395},{"operation_name":"upsert","last_dispatch_duration_us":692,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":7350,"total_server_duration_us":1158,"encode_duration_us":13,"operation_id":"0x312c","timeout_ms":10000,"last_server_duration_us":84,"total_duration_us":10002767},{"operation_name":"upsert","last_dispatch_duration_us":997,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000006/0000000044D7A75E","last_local_socket":"172.23.104.79:48380","total_dispatch_duration_us":568642,"total_server_duration_us":2257,"encode_duration_us":23,"operation_id":"0x305a","timeout_ms":10000,"last_server_duration_us":117,"total_duration_us":10000790},{"operation_name":"upsert","last_dispatch_duration_us":9003253,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":9003253,"total_server_duration_us":9002371,"encode_duration_us":15,"operation_id":"0x3078","timeout_ms":10000,"last_server_duration_us":9002371,"total_duration_us":9003792},{"operation_name":"upsert","last_dispatch_duration_us":43749,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":59393,"total_server_duration_us":44521,"encode_duration_us":31,"operation_id":"0x3729","timeout_ms":10000,"last_server_duration_us":42961,"total_duration_us":6230921}],"total_count":23}} WARN [2022-01-10 12:53:35,848] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":553,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000005/000000006FA84E55","last_local_socket":"172.23.104.79:34638","total_dispatch_duration_us":483893,"total_server_duration_us":2422,"encode_duration_us":22,"operation_id":"0x2a88","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10009026},{"operation_name":"upsert","last_dispatch_duration_us":7483285,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":7483285,"total_server_duration_us":27,"encode_duration_us":19,"operation_id":"0x3395","timeout_ms":10000,"last_server_duration_us":27,"total_duration_us":10008633},{"operation_name":"upsert","last_dispatch_duration_us":775,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000005/000000006FA84E55","last_local_socket":"172.23.104.79:34638","total_dispatch_duration_us":18059,"total_server_duration_us":2765,"encode_duration_us":20,"operation_id":"0x311e","timeout_ms":10000,"last_server_duration_us":117,"total_duration_us":10007605},{"operation_name":"upsert","last_dispatch_duration_us":653,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":20285,"total_server_duration_us":3011,"encode_duration_us":24,"operation_id":"0x324b","timeout_ms":10000,"last_server_duration_us":100,"total_duration_us":10007461},{"operation_name":"upsert","last_dispatch_duration_us":573,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":22267,"total_server_duration_us":2947,"encode_duration_us":20,"operation_id":"0x34f2","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10006973},{"operation_name":"upsert","last_dispatch_duration_us":515,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000005/000000006FA84E55","last_local_socket":"172.23.104.79:34638","total_dispatch_duration_us":19367,"total_server_duration_us":3403,"encode_duration_us":45,"operation_id":"0x3229","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10003830},{"operation_name":"upsert","last_dispatch_duration_us":836,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000005/0000000090CEECC6","last_local_socket":"172.23.104.79:48368","total_dispatch_duration_us":16987,"total_server_duration_us":2112,"encode_duration_us":21,"operation_id":"0x2d23","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10003390},{"operation_name":"upsert","last_dispatch_duration_us":8064890,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","total_dispatch_duration_us":8118401,"total_server_duration_us":371,"encode_duration_us":14,"operation_id":"0x3077","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10002315},{"operation_name":"upsert","last_dispatch_duration_us":52835,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000005/000000006FA84E55","last_local_socket":"172.23.104.79:34638","total_dispatch_duration_us":69587,"total_server_duration_us":54604,"encode_duration_us":14,"operation_id":"0x3095","timeout_ms":10000,"last_server_duration_us":52301,"total_duration_us":9804860},{"operation_name":"upsert","last_dispatch_duration_us":46037,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":62630,"total_server_duration_us":47410,"encode_duration_us":17,"operation_id":"0x30f1","timeout_ms":10000,"last_server_duration_us":45167,"total_duration_us":9295937}],"total_count":19}} WARN [2022-01-10 12:53:35,848] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9011529,"last_local_id":"2B94F89700000005/000000004F9479E2","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":18803846,"last_local_socket":"172.23.104.79:40240","total_server_duration_us":9011304,"encode_duration_us":30,"operation_id":"0x1f50","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10006300}],"total_count":1}} INFO [2022-01-10 12:53:35,849] jepsen worker 14 - jepsen.util 284 :ok :write [4 25] INFO [2022-01-10 12:53:35,849] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,850] jepsen worker 14 - jepsen.util 284 :ok :read [4 25] INFO [2022-01-10 12:53:35,850] jepsen worker 44 - jepsen.util 44 :ok :write [14 12] INFO [2022-01-10 12:53:35,850] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,851] jepsen worker 44 - jepsen.util 44 :invoke :write [14 35] INFO [2022-01-10 12:53:35,851] jepsen worker 14 - jepsen.util 284 :ok :read [4 25] INFO [2022-01-10 12:53:35,852] jepsen worker 14 - jepsen.util 284 :invoke :write [4 16] INFO [2022-01-10 12:53:35,854] jepsen worker 51 - jepsen.util 141 :ok :write [17 12] INFO [2022-01-10 12:53:35,854] jepsen worker 51 - jepsen.util 141 :invoke :write [17 3] INFO [2022-01-10 12:53:35,858] jepsen worker 75 - jepsen.util 165 :ok :write [25 39] INFO [2022-01-10 12:53:35,858] jepsen worker 75 - jepsen.util 165 :invoke :write [25 7] INFO [2022-01-10 12:53:35,858] jepsen worker 71 - jepsen.util 161 :ok :write [23 42] INFO [2022-01-10 12:53:35,859] jepsen worker 71 - jepsen.util 161 :invoke :write [23 31] INFO [2022-01-10 12:53:35,863] jepsen worker 62 - jepsen.util 242 :ok :write [20 14] INFO [2022-01-10 12:53:35,863] jepsen worker 62 - jepsen.util 242 :invoke :write [20 19] INFO [2022-01-10 12:53:35,864] jepsen worker 78 - jepsen.util 168 :ok :write [26 43] INFO [2022-01-10 12:53:35,864] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:35,865] jepsen worker 78 - jepsen.util 168 :ok :read [26 43] INFO [2022-01-10 12:53:35,865] jepsen worker 78 - jepsen.util 168 :invoke :write [26 44] INFO [2022-01-10 12:53:35,868] jepsen worker 22 - jepsen.util 112 :ok :write [7 11] INFO [2022-01-10 12:53:35,868] jepsen worker 22 - jepsen.util 112 :invoke :write [7 15] INFO [2022-01-10 12:53:35,868] jepsen worker 14 - jepsen.util 284 :ok :write [4 16] INFO [2022-01-10 12:53:35,869] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,870] jepsen worker 14 - jepsen.util 284 :ok :read [4 16] INFO [2022-01-10 12:53:35,871] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,872] jepsen worker 14 - jepsen.util 284 :ok :read [4 16] INFO [2022-01-10 12:53:35,872] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,873] jepsen worker 14 - jepsen.util 284 :ok :read [4 16] INFO [2022-01-10 12:53:35,874] jepsen worker 51 - jepsen.util 141 :ok :write [17 3] INFO [2022-01-10 12:53:35,874] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,874] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:35,875] jepsen worker 14 - jepsen.util 284 :ok :read [4 16] INFO [2022-01-10 12:53:35,875] jepsen worker 51 - jepsen.util 141 :ok :read [17 3] INFO [2022-01-10 12:53:35,875] jepsen worker 14 - jepsen.util 284 :invoke :write [4 34] INFO [2022-01-10 12:53:35,876] jepsen worker 44 - jepsen.util 44 :ok :write [14 35] INFO [2022-01-10 12:53:35,876] jepsen worker 51 - jepsen.util 141 :invoke :write [17 47] INFO [2022-01-10 12:53:35,876] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:35,877] jepsen worker 44 - jepsen.util 44 :ok :read [14 35] INFO [2022-01-10 12:53:35,878] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:35,878] jepsen worker 44 - jepsen.util 44 :ok :read [14 35] INFO [2022-01-10 12:53:35,879] jepsen worker 71 - jepsen.util 161 :ok :write [23 31] INFO [2022-01-10 12:53:35,879] jepsen worker 44 - jepsen.util 44 :invoke :write [14 17] INFO [2022-01-10 12:53:35,879] jepsen worker 71 - jepsen.util 161 :invoke :write [23 4] INFO [2022-01-10 12:53:35,883] jepsen worker 75 - jepsen.util 165 :ok :write [25 7] INFO [2022-01-10 12:53:35,884] jepsen worker 75 - jepsen.util 165 :invoke :write [25 32] INFO [2022-01-10 12:53:35,885] jepsen worker 78 - jepsen.util 168 :ok :write [26 44] INFO [2022-01-10 12:53:35,886] jepsen worker 78 - jepsen.util 168 :invoke :write [26 38] INFO [2022-01-10 12:53:35,886] jepsen worker 50 - jepsen.util 50 :ok :write [16 22] INFO [2022-01-10 12:53:35,886] jepsen worker 84 - jepsen.util 84 :ok :write [28 36] INFO [2022-01-10 12:53:35,887] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:53:35,888] jepsen worker 84 - jepsen.util 84 :invoke :write [28 13] INFO [2022-01-10 12:53:35,888] jepsen worker 62 - jepsen.util 242 :ok :write [20 19] INFO [2022-01-10 12:53:35,889] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:35,890] jepsen worker 62 - jepsen.util 242 :ok :read [20 19] INFO [2022-01-10 12:53:35,891] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:35,892] jepsen worker 62 - jepsen.util 242 :ok :read [20 19] INFO [2022-01-10 12:53:35,892] jepsen worker 14 - jepsen.util 284 :ok :write [4 34] INFO [2022-01-10 12:53:35,892] jepsen worker 3 - jepsen.util 3 :ok :write [1 24] INFO [2022-01-10 12:53:35,892] jepsen worker 40 - jepsen.util 130 :ok :write [13 46] INFO [2022-01-10 12:53:35,893] jepsen worker 62 - jepsen.util 242 :invoke :write [20 18] INFO [2022-01-10 12:53:35,894] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:35,895] jepsen worker 40 - jepsen.util 130 :ok :read [13 46] INFO [2022-01-10 12:53:35,895] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,895] jepsen worker 14 - jepsen.util 284 :ok :read [4 34] INFO [2022-01-10 12:53:35,895] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:35,896] jepsen worker 22 - jepsen.util 112 :ok :write [7 15] INFO [2022-01-10 12:53:35,896] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:35,896] jepsen worker 3 - jepsen.util 3 :ok :read [1 24] INFO [2022-01-10 12:53:35,897] jepsen worker 40 - jepsen.util 130 :ok :read [13 46] INFO [2022-01-10 12:53:35,897] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:35,898] jepsen worker 83 - jepsen.util 263 :ok :write [27 30] INFO [2022-01-10 12:53:35,899] jepsen worker 14 - jepsen.util 284 :ok :read [4 34] INFO [2022-01-10 12:53:35,899] jepsen worker 38 - jepsen.util 38 :ok :write [12 6] INFO [2022-01-10 12:53:35,900] jepsen worker 83 - jepsen.util 263 :invoke :write [27 9] INFO [2022-01-10 12:53:35,900] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:35,900] jepsen worker 51 - jepsen.util 141 :ok :write [17 47] INFO [2022-01-10 12:53:35,901] jepsen worker 3 - jepsen.util 3 :ok :read [1 24] INFO [2022-01-10 12:53:35,901] jepsen worker 14 - jepsen.util 284 :invoke :write [4 30] INFO [2022-01-10 12:53:35,902] jepsen worker 22 - jepsen.util 112 :invoke :write [7 15] INFO [2022-01-10 12:53:35,903] jepsen worker 44 - jepsen.util 44 :ok :write [14 17] INFO [2022-01-10 12:53:35,903] jepsen worker 51 - jepsen.util 141 :invoke :write [17 48] INFO [2022-01-10 12:53:35,904] jepsen worker 40 - jepsen.util 130 :invoke :write [13 44] INFO [2022-01-10 12:53:35,905] jepsen worker 33 - jepsen.util 123 :ok :write [11 3] INFO [2022-01-10 12:53:35,905] jepsen worker 44 - jepsen.util 44 :invoke :write [14 37] INFO [2022-01-10 12:53:35,906] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,906] jepsen worker 3 - jepsen.util 3 :invoke :write [1 37] INFO [2022-01-10 12:53:35,907] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:35,907] jepsen worker 33 - jepsen.util 123 :ok :read [11 3] INFO [2022-01-10 12:53:35,908] jepsen worker 38 - jepsen.util 38 :ok :read [12 6] INFO [2022-01-10 12:53:35,908] jepsen worker 33 - jepsen.util 123 :invoke :write [11 1] INFO [2022-01-10 12:53:35,909] jepsen worker 38 - jepsen.util 38 :invoke :write [12 9] INFO [2022-01-10 12:53:35,909] jepsen worker 71 - jepsen.util 161 :ok :write [23 4] INFO [2022-01-10 12:53:35,910] jepsen worker 71 - jepsen.util 161 :invoke :write [23 45] INFO [2022-01-10 12:53:35,910] jepsen worker 89 - jepsen.util 269 :ok :write [29 12] INFO [2022-01-10 12:53:35,911] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:35,912] jepsen worker 89 - jepsen.util 269 :ok :read [29 12] INFO [2022-01-10 12:53:35,913] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:35,913] jepsen worker 89 - jepsen.util 269 :ok :read [29 12] INFO [2022-01-10 12:53:35,914] jepsen worker 75 - jepsen.util 165 :ok :write [25 32] INFO [2022-01-10 12:53:35,914] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:35,915] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:35,915] jepsen worker 89 - jepsen.util 269 :ok :read [29 12] INFO [2022-01-10 12:53:35,915] jepsen worker 75 - jepsen.util 165 :ok :read [25 32] INFO [2022-01-10 12:53:35,916] jepsen worker 89 - jepsen.util 269 :invoke :write [29 3] INFO [2022-01-10 12:53:35,916] jepsen worker 75 - jepsen.util 165 :invoke :write [25 15] INFO [2022-01-10 12:53:35,917] jepsen worker 68 - jepsen.util 68 :ok :write [22 6] INFO [2022-01-10 12:53:35,917] jepsen worker 78 - jepsen.util 168 :ok :write [26 38] INFO [2022-01-10 12:53:35,918] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,918] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:35,919] jepsen worker 68 - jepsen.util 68 :ok :read [22 6] INFO [2022-01-10 12:53:35,919] jepsen worker 78 - jepsen.util 168 :ok :read [26 38] INFO [2022-01-10 12:53:35,920] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,920] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:35,920] jepsen worker 68 - jepsen.util 68 :ok :read [22 6] INFO [2022-01-10 12:53:35,921] jepsen worker 78 - jepsen.util 168 :ok :read [26 38] INFO [2022-01-10 12:53:35,922] jepsen worker 68 - jepsen.util 68 :invoke :write [22 24] INFO [2022-01-10 12:53:35,922] jepsen worker 78 - jepsen.util 168 :invoke :write [26 23] INFO [2022-01-10 12:53:35,923] jepsen worker 1 - jepsen.util 91 :ok :write [0 27] INFO [2022-01-10 12:53:35,924] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,925] jepsen worker 1 - jepsen.util 91 :ok :read [0 27] INFO [2022-01-10 12:53:35,926] jepsen worker 1 - jepsen.util 91 :invoke :write [0 38] INFO [2022-01-10 12:53:35,926] jepsen worker 84 - jepsen.util 84 :ok :write [28 13] INFO [2022-01-10 12:53:35,927] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:35,927] jepsen worker 62 - jepsen.util 242 :ok :write [20 18] INFO [2022-01-10 12:53:35,928] jepsen worker 84 - jepsen.util 84 :ok :read [28 13] INFO [2022-01-10 12:53:35,928] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:35,928] jepsen worker 7 - jepsen.util 7 :ok :write [2 17] INFO [2022-01-10 12:53:35,929] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:35,929] jepsen worker 62 - jepsen.util 242 :ok :read [20 18] INFO [2022-01-10 12:53:35,929] jepsen worker 7 - jepsen.util 7 :invoke :write [2 48] INFO [2022-01-10 12:53:35,930] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:35,930] jepsen worker 84 - jepsen.util 84 :ok :read [28 13] INFO [2022-01-10 12:53:35,931] jepsen worker 62 - jepsen.util 242 :ok :read [20 18] INFO [2022-01-10 12:53:35,931] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:35,932] jepsen worker 62 - jepsen.util 242 :invoke :write [20 37] INFO [2022-01-10 12:53:35,932] jepsen worker 84 - jepsen.util 84 :ok :read [28 13] INFO [2022-01-10 12:53:35,932] jepsen worker 12 - jepsen.util 192 :ok :write [4 28] INFO [2022-01-10 12:53:35,933] jepsen worker 12 - jepsen.util 192 :invoke :write [4 23] INFO [2022-01-10 12:53:35,934] jepsen worker 84 - jepsen.util 84 :invoke :write [28 5] INFO [2022-01-10 12:53:35,935] jepsen worker 15 - jepsen.util 15 :ok :write [5 6] INFO [2022-01-10 12:53:35,935] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,936] jepsen worker 22 - jepsen.util 112 :ok :write [7 15] INFO [2022-01-10 12:53:35,936] jepsen worker 15 - jepsen.util 15 :ok :read [5 6] INFO [2022-01-10 12:53:35,937] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,937] jepsen worker 22 - jepsen.util 112 :invoke :write [7 34] INFO [2022-01-10 12:53:35,938] jepsen worker 15 - jepsen.util 15 :ok :read [5 6] INFO [2022-01-10 12:53:35,938] jepsen worker 3 - jepsen.util 3 :ok :write [1 37] INFO [2022-01-10 12:53:35,938] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,939] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:35,939] jepsen worker 15 - jepsen.util 15 :ok :read [5 6] INFO [2022-01-10 12:53:35,940] jepsen worker 3 - jepsen.util 3 :ok :read [1 37] INFO [2022-01-10 12:53:35,940] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,940] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:35,941] jepsen worker 15 - jepsen.util 15 :ok :read [5 6] INFO [2022-01-10 12:53:35,941] jepsen worker 3 - jepsen.util 3 :ok :read [1 37] INFO [2022-01-10 12:53:35,941] jepsen worker 15 - jepsen.util 15 :invoke :write [5 44] INFO [2022-01-10 12:53:35,941] jepsen worker 3 - jepsen.util 3 :invoke :write [1 33] INFO [2022-01-10 12:53:35,942] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:53:35,943] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:35,944] jepsen worker 50 - jepsen.util 50 :ok :read [16 30] INFO [2022-01-10 12:53:35,945] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:35,946] jepsen worker 50 - jepsen.util 50 :ok :read [16 30] INFO [2022-01-10 12:53:35,946] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:53:35,946] jepsen worker 38 - jepsen.util 38 :ok :write [12 9] INFO [2022-01-10 12:53:35,947] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:35,947] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:35,948] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:35,948] jepsen worker 40 - jepsen.util 130 :ok :write [13 44] INFO [2022-01-10 12:53:35,949] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:35,949] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:35,949] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:35,949] jepsen worker 40 - jepsen.util 130 :ok :read [13 44] INFO [2022-01-10 12:53:35,950] jepsen worker 40 - jepsen.util 130 :invoke :write [13 11] INFO [2022-01-10 12:53:35,950] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:35,951] jepsen worker 44 - jepsen.util 44 :ok :write [14 37] INFO [2022-01-10 12:53:35,951] jepsen worker 38 - jepsen.util 38 :invoke :write [12 46] INFO [2022-01-10 12:53:35,951] jepsen worker 44 - jepsen.util 44 :invoke :write [14 24] INFO [2022-01-10 12:53:35,955] jepsen worker 51 - jepsen.util 141 :ok :write [17 48] INFO [2022-01-10 12:53:35,955] jepsen worker 51 - jepsen.util 141 :invoke :write [17 38] INFO [2022-01-10 12:53:35,956] jepsen worker 83 - jepsen.util 263 :ok :write [27 9] INFO [2022-01-10 12:53:35,956] jepsen worker 83 - jepsen.util 263 :invoke :write [27 44] INFO [2022-01-10 12:53:35,961] jepsen worker 71 - jepsen.util 161 :ok :write [23 45] INFO [2022-01-10 12:53:35,961] jepsen worker 71 - jepsen.util 161 :invoke :write [23 27] INFO [2022-01-10 12:53:35,964] jepsen worker 75 - jepsen.util 165 :ok :write [25 15] INFO [2022-01-10 12:53:35,965] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:35,965] jepsen worker 75 - jepsen.util 165 :ok :read [25 15] INFO [2022-01-10 12:53:35,966] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:35,967] jepsen worker 75 - jepsen.util 165 :ok :read [25 15] INFO [2022-01-10 12:53:35,967] jepsen worker 75 - jepsen.util 165 :invoke :write [25 25] INFO [2022-01-10 12:53:35,967] jepsen worker 33 - jepsen.util 123 :ok :write [11 1] INFO [2022-01-10 12:53:35,968] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:35,969] jepsen worker 33 - jepsen.util 123 :ok :read [11 1] INFO [2022-01-10 12:53:35,970] jepsen worker 33 - jepsen.util 123 :invoke :write [11 49] INFO [2022-01-10 12:53:35,972] jepsen worker 89 - jepsen.util 269 :ok :write [29 3] INFO [2022-01-10 12:53:35,972] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:35,973] jepsen worker 89 - jepsen.util 269 :ok :read [29 3] INFO [2022-01-10 12:53:35,973] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:35,974] jepsen worker 62 - jepsen.util 242 :ok :write [20 37] INFO [2022-01-10 12:53:35,974] jepsen worker 89 - jepsen.util 269 :ok :read [29 3] INFO [2022-01-10 12:53:35,974] jepsen worker 62 - jepsen.util 242 :invoke :write [20 13] INFO [2022-01-10 12:53:35,975] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:35,975] jepsen worker 89 - jepsen.util 269 :ok :read [29 3] INFO [2022-01-10 12:53:35,976] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:35,977] jepsen worker 68 - jepsen.util 68 :ok :write [22 24] INFO [2022-01-10 12:53:35,977] jepsen worker 89 - jepsen.util 269 :ok :read [29 3] INFO [2022-01-10 12:53:35,977] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:35,978] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:35,978] jepsen worker 68 - jepsen.util 68 :ok :read [22 24] INFO [2022-01-10 12:53:35,978] jepsen worker 68 - jepsen.util 68 :invoke :write [22 49] INFO [2022-01-10 12:53:35,978] jepsen worker 89 - jepsen.util 269 :ok :read [29 3] INFO [2022-01-10 12:53:35,979] jepsen worker 89 - jepsen.util 269 :invoke :write [29 44] INFO [2022-01-10 12:53:35,983] jepsen worker 1 - jepsen.util 91 :ok :write [0 38] INFO [2022-01-10 12:53:35,983] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,984] jepsen worker 1 - jepsen.util 91 :ok :read [0 38] INFO [2022-01-10 12:53:35,985] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,985] jepsen worker 1 - jepsen.util 91 :ok :read [0 38] INFO [2022-01-10 12:53:35,986] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,987] jepsen worker 1 - jepsen.util 91 :ok :read [0 38] INFO [2022-01-10 12:53:35,988] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,988] jepsen worker 7 - jepsen.util 7 :ok :write [2 48] INFO [2022-01-10 12:53:35,988] jepsen worker 1 - jepsen.util 91 :ok :read [0 38] INFO [2022-01-10 12:53:35,989] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:35,989] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:35,990] jepsen worker 7 - jepsen.util 7 :ok :read [2 48] INFO [2022-01-10 12:53:35,990] jepsen worker 1 - jepsen.util 91 :ok :read [0 38] INFO [2022-01-10 12:53:35,990] jepsen worker 7 - jepsen.util 7 :invoke :write [2 11] INFO [2022-01-10 12:53:35,990] jepsen worker 1 - jepsen.util 91 :invoke :write [0 15] INFO [2022-01-10 12:53:35,993] jepsen worker 15 - jepsen.util 15 :ok :write [5 44] INFO [2022-01-10 12:53:35,994] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,995] jepsen worker 15 - jepsen.util 15 :ok :read [5 44] INFO [2022-01-10 12:53:35,995] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:35,996] jepsen worker 15 - jepsen.util 15 :ok :read [5 44] INFO [2022-01-10 12:53:35,996] jepsen worker 15 - jepsen.util 15 :invoke :write [5 38] INFO [2022-01-10 12:53:35,998] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:53:35,999] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:36,000] jepsen worker 50 - jepsen.util 50 :ok :read [16 30] INFO [2022-01-10 12:53:36,000] jepsen worker 50 - jepsen.util 50 :invoke :write [16 9] INFO [2022-01-10 12:53:36,008] jepsen worker 83 - jepsen.util 263 :ok :write [27 44] INFO [2022-01-10 12:53:36,008] jepsen worker 83 - jepsen.util 263 :invoke :write [27 49] INFO [2022-01-10 12:53:36,076] jepsen worker 22 - jepsen.util 112 :ok :write [7 34] INFO [2022-01-10 12:53:36,077] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,078] jepsen worker 22 - jepsen.util 112 :ok :read [7 34] INFO [2022-01-10 12:53:36,078] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,079] jepsen worker 78 - jepsen.util 168 :ok :write [26 23] INFO [2022-01-10 12:53:36,079] jepsen worker 22 - jepsen.util 112 :ok :read [7 34] INFO [2022-01-10 12:53:36,080] jepsen worker 78 - jepsen.util 168 :invoke :write [26 39] INFO [2022-01-10 12:53:36,080] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,081] jepsen worker 22 - jepsen.util 112 :ok :read [7 34] INFO [2022-01-10 12:53:36,081] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,082] jepsen worker 40 - jepsen.util 130 :ok :write [13 11] INFO [2022-01-10 12:53:36,082] jepsen worker 22 - jepsen.util 112 :ok :read [7 34] INFO [2022-01-10 12:53:36,083] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:36,084] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,084] jepsen worker 40 - jepsen.util 130 :ok :read [13 11] INFO [2022-01-10 12:53:36,084] jepsen worker 22 - jepsen.util 112 :ok :read [7 34] INFO [2022-01-10 12:53:36,085] jepsen worker 40 - jepsen.util 130 :invoke :write [13 13] INFO [2022-01-10 12:53:36,086] jepsen worker 22 - jepsen.util 112 :invoke :write [7 18] INFO [2022-01-10 12:53:36,086] jepsen worker 44 - jepsen.util 44 :ok :write [14 24] INFO [2022-01-10 12:53:36,086] jepsen worker 84 - jepsen.util 84 :ok :write [28 5] INFO [2022-01-10 12:53:36,087] jepsen worker 44 - jepsen.util 44 :invoke :write [14 38] INFO [2022-01-10 12:53:36,087] jepsen worker 84 - jepsen.util 84 :invoke :write [28 11] INFO [2022-01-10 12:53:36,092] jepsen worker 12 - jepsen.util 192 :ok :write [4 23] INFO [2022-01-10 12:53:36,092] jepsen worker 12 - jepsen.util 192 :invoke :write [4 32] INFO [2022-01-10 12:53:36,096] jepsen worker 89 - jepsen.util 269 :ok :write [29 44] INFO [2022-01-10 12:53:36,097] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:36,098] jepsen worker 89 - jepsen.util 269 :ok :read [29 44] INFO [2022-01-10 12:53:36,098] jepsen worker 3 - jepsen.util 3 :ok :write [1 33] INFO [2022-01-10 12:53:36,098] jepsen worker 89 - jepsen.util 269 :invoke :write [29 20] INFO [2022-01-10 12:53:36,099] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:36,100] jepsen worker 3 - jepsen.util 3 :ok :read [1 33] INFO [2022-01-10 12:53:36,100] jepsen worker 3 - jepsen.util 3 :invoke :write [1 40] INFO [2022-01-10 12:53:36,102] jepsen worker 75 - jepsen.util 165 :ok :write [25 25] INFO [2022-01-10 12:53:36,103] jepsen worker 75 - jepsen.util 165 :invoke :write [25 30] INFO [2022-01-10 12:53:36,106] jepsen worker 38 - jepsen.util 38 :ok :write [12 46] INFO [2022-01-10 12:53:36,106] jepsen worker 38 - jepsen.util 38 :invoke :write [12 47] INFO [2022-01-10 12:53:36,110] jepsen worker 0 - jepsen.util 180 :ok :write [0 26] INFO [2022-01-10 12:53:36,110] jepsen worker 0 - jepsen.util 180 :invoke :write [0 25] INFO [2022-01-10 12:53:36,111] jepsen worker 33 - jepsen.util 123 :ok :write [11 49] INFO [2022-01-10 12:53:36,111] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,112] jepsen worker 33 - jepsen.util 123 :ok :read [11 49] INFO [2022-01-10 12:53:36,112] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,113] jepsen worker 33 - jepsen.util 123 :ok :read [11 49] INFO [2022-01-10 12:53:36,114] jepsen worker 33 - jepsen.util 123 :invoke :write [11 12] INFO [2022-01-10 12:53:36,114] jepsen worker 51 - jepsen.util 141 :ok :write [17 38] INFO [2022-01-10 12:53:36,115] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,115] jepsen worker 62 - jepsen.util 242 :ok :write [20 13] INFO [2022-01-10 12:53:36,116] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,116] jepsen worker 51 - jepsen.util 141 :ok :read [17 38] INFO [2022-01-10 12:53:36,116] jepsen worker 51 - jepsen.util 141 :invoke :write [17 32] INFO [2022-01-10 12:53:36,117] jepsen worker 62 - jepsen.util 242 :ok :read [20 13] INFO [2022-01-10 12:53:36,117] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,118] jepsen worker 62 - jepsen.util 242 :ok :read [20 13] INFO [2022-01-10 12:53:36,119] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,121] jepsen worker 40 - jepsen.util 130 :ok :write [13 13] INFO [2022-01-10 12:53:36,121] jepsen worker 71 - jepsen.util 161 :ok :write [23 27] INFO [2022-01-10 12:53:36,121] jepsen worker 62 - jepsen.util 242 :ok :read [20 13] INFO [2022-01-10 12:53:36,122] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,123] jepsen worker 71 - jepsen.util 161 :ok :read [23 27] INFO [2022-01-10 12:53:36,123] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,124] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,124] jepsen worker 68 - jepsen.util 68 :ok :write [22 49] INFO [2022-01-10 12:53:36,124] jepsen worker 62 - jepsen.util 242 :ok :read [20 13] INFO [2022-01-10 12:53:36,125] jepsen worker 71 - jepsen.util 161 :ok :read [23 27] INFO [2022-01-10 12:53:36,125] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,126] jepsen worker 62 - jepsen.util 242 :ok :read [20 13] INFO [2022-01-10 12:53:36,126] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,126] jepsen worker 40 - jepsen.util 130 :invoke :write [13 13] INFO [2022-01-10 12:53:36,127] jepsen worker 62 - jepsen.util 242 :invoke :write [20 1] INFO [2022-01-10 12:53:36,127] jepsen worker 71 - jepsen.util 161 :ok :read [23 27] INFO [2022-01-10 12:53:36,127] jepsen worker 68 - jepsen.util 68 :invoke :write [22 24] INFO [2022-01-10 12:53:36,128] jepsen worker 71 - jepsen.util 161 :invoke :write [23 24] INFO [2022-01-10 12:53:36,129] jepsen worker 7 - jepsen.util 7 :ok :write [2 11] INFO [2022-01-10 12:53:36,129] jepsen worker 7 - jepsen.util 7 :invoke :write [2 21] INFO [2022-01-10 12:53:36,130] jepsen worker 78 - jepsen.util 168 :ok :write [26 39] INFO [2022-01-10 12:53:36,131] jepsen worker 78 - jepsen.util 168 :invoke :write [26 44] INFO [2022-01-10 12:53:36,131] jepsen worker 89 - jepsen.util 269 :ok :write [29 20] INFO [2022-01-10 12:53:36,132] jepsen worker 89 - jepsen.util 269 :invoke :write [29 40] INFO [2022-01-10 12:53:36,137] jepsen worker 15 - jepsen.util 15 :ok :write [5 38] INFO [2022-01-10 12:53:36,137] jepsen worker 84 - jepsen.util 84 :ok :write [28 11] INFO [2022-01-10 12:53:36,138] jepsen worker 84 - jepsen.util 84 :invoke :write [28 14] INFO [2022-01-10 12:53:36,139] jepsen worker 15 - jepsen.util 15 :invoke :write [5 45] INFO [2022-01-10 12:53:36,142] jepsen worker 0 - jepsen.util 180 :ok :write [0 25] INFO [2022-01-10 12:53:36,142] jepsen worker 0 - jepsen.util 180 :invoke :write [0 1] INFO [2022-01-10 12:53:36,142] jepsen worker 12 - jepsen.util 192 :ok :write [4 32] INFO [2022-01-10 12:53:36,143] jepsen worker 50 - jepsen.util 50 :ok :write [16 9] INFO [2022-01-10 12:53:36,143] jepsen worker 12 - jepsen.util 192 :invoke :write [4 23] INFO [2022-01-10 12:53:36,144] jepsen worker 50 - jepsen.util 50 :invoke :write [16 29] INFO [2022-01-10 12:53:36,150] jepsen worker 83 - jepsen.util 263 :ok :write [27 49] INFO [2022-01-10 12:53:36,151] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:36,152] jepsen worker 3 - jepsen.util 3 :ok :write [1 40] INFO [2022-01-10 12:53:36,152] jepsen worker 83 - jepsen.util 263 :ok :read [27 49] INFO [2022-01-10 12:53:36,153] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:36,153] jepsen worker 22 - jepsen.util 112 :ok :write [7 18] INFO [2022-01-10 12:53:36,153] jepsen worker 40 - jepsen.util 130 :ok :write [13 13] INFO [2022-01-10 12:53:36,154] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:36,154] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,155] jepsen worker 83 - jepsen.util 263 :ok :read [27 49] INFO [2022-01-10 12:53:36,155] jepsen worker 3 - jepsen.util 3 :ok :read [1 40] INFO [2022-01-10 12:53:36,155] jepsen worker 40 - jepsen.util 130 :invoke :write [13 40] INFO [2022-01-10 12:53:36,155] jepsen worker 22 - jepsen.util 112 :ok :read [7 18] INFO [2022-01-10 12:53:36,156] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:36,156] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:36,157] jepsen worker 44 - jepsen.util 44 :ok :write [14 38] INFO [2022-01-10 12:53:36,157] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,157] jepsen worker 83 - jepsen.util 263 :ok :read [27 49] INFO [2022-01-10 12:53:36,157] jepsen worker 38 - jepsen.util 38 :ok :write [12 47] INFO [2022-01-10 12:53:36,157] jepsen worker 3 - jepsen.util 3 :ok :read [1 40] INFO [2022-01-10 12:53:36,158] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,158] jepsen worker 89 - jepsen.util 269 :ok :write [29 40] INFO [2022-01-10 12:53:36,158] jepsen worker 22 - jepsen.util 112 :ok :read [7 18] INFO [2022-01-10 12:53:36,159] jepsen worker 44 - jepsen.util 44 :ok :read [14 38] INFO [2022-01-10 12:53:36,159] jepsen worker 83 - jepsen.util 263 :invoke :write [27 31] INFO [2022-01-10 12:53:36,160] jepsen worker 89 - jepsen.util 269 :invoke :write [29 21] INFO [2022-01-10 12:53:36,160] jepsen worker 75 - jepsen.util 165 :ok :write [25 30] INFO [2022-01-10 12:53:36,161] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,161] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:36,162] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,162] jepsen worker 22 - jepsen.util 112 :ok :read [7 18] INFO [2022-01-10 12:53:36,162] jepsen worker 3 - jepsen.util 3 :ok :read [1 40] INFO [2022-01-10 12:53:36,163] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,163] jepsen worker 75 - jepsen.util 165 :ok :read [25 30] INFO [2022-01-10 12:53:36,163] jepsen worker 22 - jepsen.util 112 :ok :read [7 18] INFO [2022-01-10 12:53:36,163] jepsen worker 44 - jepsen.util 44 :invoke :write [14 30] INFO [2022-01-10 12:53:36,164] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,165] jepsen worker 75 - jepsen.util 165 :ok :read [25 30] INFO [2022-01-10 12:53:36,165] jepsen worker 22 - jepsen.util 112 :invoke :write [7 42] INFO [2022-01-10 12:53:36,165] jepsen worker 51 - jepsen.util 141 :ok :write [17 32] INFO [2022-01-10 12:53:36,166] jepsen worker 75 - jepsen.util 165 :invoke :write [25 8] INFO [2022-01-10 12:53:36,166] jepsen worker 33 - jepsen.util 123 :ok :write [11 12] INFO [2022-01-10 12:53:36,167] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:36,167] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,168] jepsen worker 38 - jepsen.util 38 :ok :read [12 47] INFO [2022-01-10 12:53:36,168] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,168] jepsen worker 33 - jepsen.util 123 :ok :read [11 12] INFO [2022-01-10 12:53:36,168] jepsen worker 0 - jepsen.util 180 :ok :write [0 1] INFO [2022-01-10 12:53:36,168] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:36,169] jepsen worker 33 - jepsen.util 123 :invoke :write [11 42] INFO [2022-01-10 12:53:36,169] jepsen worker 51 - jepsen.util 141 :ok :read [17 32] INFO [2022-01-10 12:53:36,169] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:36,170] jepsen worker 38 - jepsen.util 38 :ok :read [12 47] INFO [2022-01-10 12:53:36,170] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,170] jepsen worker 0 - jepsen.util 180 :ok :read [0 1] INFO [2022-01-10 12:53:36,170] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:36,171] jepsen worker 3 - jepsen.util 3 :invoke :write [1 47] INFO [2022-01-10 12:53:36,171] jepsen worker 51 - jepsen.util 141 :ok :read [17 32] INFO [2022-01-10 12:53:36,171] jepsen worker 0 - jepsen.util 180 :invoke :read [0 nil] INFO [2022-01-10 12:53:36,172] jepsen worker 38 - jepsen.util 38 :ok :read [12 47] INFO [2022-01-10 12:53:36,172] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,172] jepsen worker 0 - jepsen.util 180 :ok :read [0 1] INFO [2022-01-10 12:53:36,172] jepsen worker 38 - jepsen.util 38 :invoke :write [12 45] INFO [2022-01-10 12:53:36,172] jepsen worker 69 - jepsen.util 159 :ok :write [23 39] INFO [2022-01-10 12:53:36,173] jepsen worker 0 - jepsen.util 180 :invoke :write [0 0] INFO [2022-01-10 12:53:36,173] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,174] jepsen worker 62 - jepsen.util 242 :ok :write [20 1] INFO [2022-01-10 12:53:36,174] jepsen worker 51 - jepsen.util 141 :ok :read [17 32] INFO [2022-01-10 12:53:36,174] jepsen worker 69 - jepsen.util 159 :ok :read [23 39] INFO [2022-01-10 12:53:36,174] jepsen worker 62 - jepsen.util 242 :invoke :write [20 29] INFO [2022-01-10 12:53:36,175] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,175] jepsen worker 69 - jepsen.util 159 :invoke :write [23 12] INFO [2022-01-10 12:53:36,176] jepsen worker 51 - jepsen.util 141 :ok :read [17 32] INFO [2022-01-10 12:53:36,176] jepsen worker 51 - jepsen.util 141 :invoke :write [17 19] INFO [2022-01-10 12:53:36,182] jepsen worker 68 - jepsen.util 68 :ok :write [22 24] INFO [2022-01-10 12:53:36,182] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:36,183] jepsen worker 78 - jepsen.util 168 :ok :write [26 44] INFO [2022-01-10 12:53:36,183] jepsen worker 68 - jepsen.util 68 :ok :read [22 24] INFO [2022-01-10 12:53:36,184] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,184] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:36,185] jepsen worker 78 - jepsen.util 168 :ok :read [26 44] INFO [2022-01-10 12:53:36,185] jepsen worker 68 - jepsen.util 68 :ok :read [22 24] INFO [2022-01-10 12:53:36,185] jepsen worker 78 - jepsen.util 168 :invoke :write [26 48] INFO [2022-01-10 12:53:36,186] jepsen worker 40 - jepsen.util 130 :ok :write [13 40] INFO [2022-01-10 12:53:36,186] jepsen worker 68 - jepsen.util 68 :invoke :write [22 22] INFO [2022-01-10 12:53:36,186] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:36,187] jepsen worker 7 - jepsen.util 7 :ok :write [2 21] INFO [2022-01-10 12:53:36,187] jepsen worker 7 - jepsen.util 7 :invoke :write [2 16] INFO [2022-01-10 12:53:36,188] jepsen worker 40 - jepsen.util 130 :ok :read [13 40] INFO [2022-01-10 12:53:36,188] jepsen worker 40 - jepsen.util 130 :invoke :write [13 26] INFO [2022-01-10 12:53:36,191] jepsen worker 84 - jepsen.util 84 :ok :write [28 14] INFO [2022-01-10 12:53:36,192] jepsen worker 84 - jepsen.util 84 :invoke :write [28 32] INFO [2022-01-10 12:53:36,196] jepsen worker 15 - jepsen.util 15 :ok :write [5 45] INFO [2022-01-10 12:53:36,197] jepsen worker 15 - jepsen.util 15 :invoke :write [5 38] INFO [2022-01-10 12:53:36,197] jepsen worker 12 - jepsen.util 192 :ok :write [4 23] INFO [2022-01-10 12:53:36,198] jepsen worker 12 - jepsen.util 192 :invoke :write [4 27] INFO [2022-01-10 12:53:36,201] jepsen worker 50 - jepsen.util 50 :ok :write [16 29] INFO [2022-01-10 12:53:36,201] jepsen worker 50 - jepsen.util 50 :invoke :write [16 42] INFO [2022-01-10 12:53:36,203] jepsen worker 3 - jepsen.util 3 :ok :write [1 47] INFO [2022-01-10 12:53:36,204] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:36,204] jepsen worker 3 - jepsen.util 3 :ok :read [1 47] INFO [2022-01-10 12:53:36,205] jepsen worker 3 - jepsen.util 3 :invoke :write [1 42] INFO [2022-01-10 12:53:36,207] jepsen worker 83 - jepsen.util 263 :ok :write [27 31] INFO [2022-01-10 12:53:36,207] jepsen worker 83 - jepsen.util 263 :invoke :write [27 34] INFO [2022-01-10 12:53:36,210] jepsen worker 38 - jepsen.util 38 :ok :write [12 45] INFO [2022-01-10 12:53:36,211] jepsen worker 22 - jepsen.util 112 :ok :write [7 42] INFO [2022-01-10 12:53:36,211] jepsen worker 38 - jepsen.util 38 :invoke :write [12 45] INFO [2022-01-10 12:53:36,211] jepsen worker 22 - jepsen.util 112 :invoke :write [7 44] INFO [2022-01-10 12:53:36,214] jepsen worker 44 - jepsen.util 44 :ok :write [14 30] INFO [2022-01-10 12:53:36,215] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,216] jepsen worker 44 - jepsen.util 44 :ok :read [14 30] INFO [2022-01-10 12:53:36,216] jepsen worker 44 - jepsen.util 44 :invoke :write [14 35] INFO [2022-01-10 12:53:36,218] jepsen worker 51 - jepsen.util 141 :ok :write [17 19] INFO [2022-01-10 12:53:36,218] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,219] jepsen worker 75 - jepsen.util 165 :ok :write [25 8] INFO [2022-01-10 12:53:36,219] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,219] jepsen worker 51 - jepsen.util 141 :ok :read [17 19] INFO [2022-01-10 12:53:36,220] jepsen worker 75 - jepsen.util 165 :ok :read [25 8] INFO [2022-01-10 12:53:36,220] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,220] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,221] jepsen worker 75 - jepsen.util 165 :ok :read [25 8] INFO [2022-01-10 12:53:36,221] jepsen worker 51 - jepsen.util 141 :ok :read [17 19] INFO [2022-01-10 12:53:36,222] jepsen worker 75 - jepsen.util 165 :invoke :write [25 23] INFO [2022-01-10 12:53:36,222] jepsen worker 51 - jepsen.util 141 :invoke :write [17 25] INFO [2022-01-10 12:53:36,226] jepsen worker 71 - jepsen.util 161 :ok :write [23 24] INFO [2022-01-10 12:53:36,226] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,228] jepsen worker 71 - jepsen.util 161 :ok :read [23 24] INFO [2022-01-10 12:53:36,228] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,228] jepsen worker 62 - jepsen.util 242 :ok :write [20 29] INFO [2022-01-10 12:53:36,229] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,229] jepsen worker 71 - jepsen.util 161 :ok :read [23 24] INFO [2022-01-10 12:53:36,230] jepsen worker 62 - jepsen.util 242 :ok :read [20 29] INFO [2022-01-10 12:53:36,230] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,230] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,231] jepsen worker 71 - jepsen.util 161 :ok :read [23 24] INFO [2022-01-10 12:53:36,231] jepsen worker 78 - jepsen.util 168 :ok :write [26 48] INFO [2022-01-10 12:53:36,231] jepsen worker 62 - jepsen.util 242 :ok :read [20 29] INFO [2022-01-10 12:53:36,232] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,232] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,233] jepsen worker 71 - jepsen.util 161 :ok :read [23 24] INFO [2022-01-10 12:53:36,233] jepsen worker 78 - jepsen.util 168 :invoke :write [26 10] INFO [2022-01-10 12:53:36,233] jepsen worker 62 - jepsen.util 242 :ok :read [20 29] INFO [2022-01-10 12:53:36,233] jepsen worker 71 - jepsen.util 161 :invoke :write [23 47] INFO [2022-01-10 12:53:36,234] jepsen worker 62 - jepsen.util 242 :invoke :write [20 32] INFO [2022-01-10 12:53:36,241] jepsen worker 12 - jepsen.util 192 :ok :write [4 27] INFO [2022-01-10 12:53:36,242] jepsen worker 12 - jepsen.util 192 :invoke :write [4 32] INFO [2022-01-10 12:53:36,256] jepsen worker 51 - jepsen.util 141 :ok :write [17 25] INFO [2022-01-10 12:53:36,257] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,258] jepsen worker 51 - jepsen.util 141 :ok :read [17 25] INFO [2022-01-10 12:53:36,258] jepsen worker 22 - jepsen.util 112 :ok :write [7 44] INFO [2022-01-10 12:53:36,258] jepsen worker 51 - jepsen.util 141 :invoke :write [17 22] INFO [2022-01-10 12:53:36,259] jepsen worker 22 - jepsen.util 112 :invoke :write [7 30] INFO [2022-01-10 12:53:36,264] jepsen worker 69 - jepsen.util 159 :ok :write [23 12] INFO [2022-01-10 12:53:36,264] jepsen worker 44 - jepsen.util 44 :ok :write [14 35] INFO [2022-01-10 12:53:36,265] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,265] jepsen worker 69 - jepsen.util 159 :invoke :write [23 5] INFO [2022-01-10 12:53:36,266] jepsen worker 44 - jepsen.util 44 :ok :read [14 35] INFO [2022-01-10 12:53:36,266] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,267] jepsen worker 44 - jepsen.util 44 :ok :read [14 35] INFO [2022-01-10 12:53:36,268] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,268] jepsen worker 75 - jepsen.util 165 :ok :write [25 23] INFO [2022-01-10 12:53:36,269] jepsen worker 75 - jepsen.util 165 :invoke :write [25 40] INFO [2022-01-10 12:53:36,269] jepsen worker 44 - jepsen.util 44 :ok :read [14 35] INFO [2022-01-10 12:53:36,270] jepsen worker 44 - jepsen.util 44 :invoke :write [14 42] INFO [2022-01-10 12:53:36,270] jepsen worker 78 - jepsen.util 168 :ok :write [26 10] INFO [2022-01-10 12:53:36,271] jepsen worker 78 - jepsen.util 168 :invoke :write [26 29] INFO [2022-01-10 12:53:36,274] jepsen worker 62 - jepsen.util 242 :ok :write [20 32] INFO [2022-01-10 12:53:36,275] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,276] jepsen worker 62 - jepsen.util 242 :ok :read [20 32] INFO [2022-01-10 12:53:36,276] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,277] jepsen worker 62 - jepsen.util 242 :ok :read [20 32] INFO [2022-01-10 12:53:36,278] jepsen worker 12 - jepsen.util 192 :ok :write [4 32] INFO [2022-01-10 12:53:36,278] jepsen worker 62 - jepsen.util 242 :invoke :write [20 24] INFO [2022-01-10 12:53:36,278] jepsen worker 12 - jepsen.util 192 :invoke :write [4 45] INFO [2022-01-10 12:53:36,282] jepsen worker 22 - jepsen.util 112 :ok :write [7 30] INFO [2022-01-10 12:53:36,282] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,283] jepsen worker 51 - jepsen.util 141 :ok :write [17 22] INFO [2022-01-10 12:53:36,283] jepsen worker 22 - jepsen.util 112 :ok :read [7 30] INFO [2022-01-10 12:53:36,283] jepsen worker 51 - jepsen.util 141 :invoke :write [17 15] INFO [2022-01-10 12:53:36,284] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,285] jepsen worker 22 - jepsen.util 112 :ok :read [7 30] INFO [2022-01-10 12:53:36,285] jepsen worker 44 - jepsen.util 44 :ok :write [14 42] INFO [2022-01-10 12:53:36,285] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,286] jepsen worker 22 - jepsen.util 112 :ok :read [7 30] INFO [2022-01-10 12:53:36,286] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,287] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,287] jepsen worker 44 - jepsen.util 44 :ok :read [14 42] INFO [2022-01-10 12:53:36,288] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,288] jepsen worker 22 - jepsen.util 112 :ok :read [7 30] INFO [2022-01-10 12:53:36,288] jepsen worker 22 - jepsen.util 112 :invoke :write [7 21] INFO [2022-01-10 12:53:36,288] jepsen worker 44 - jepsen.util 44 :ok :read [14 42] INFO [2022-01-10 12:53:36,289] jepsen worker 69 - jepsen.util 159 :ok :write [23 5] INFO [2022-01-10 12:53:36,289] jepsen worker 44 - jepsen.util 44 :invoke :write [14 14] INFO [2022-01-10 12:53:36,290] jepsen worker 69 - jepsen.util 159 :invoke :write [23 24] INFO [2022-01-10 12:53:36,294] jepsen worker 75 - jepsen.util 165 :ok :write [25 40] INFO [2022-01-10 12:53:36,294] jepsen worker 75 - jepsen.util 165 :invoke :write [25 9] INFO [2022-01-10 12:53:36,296] jepsen worker 78 - jepsen.util 168 :ok :write [26 29] INFO [2022-01-10 12:53:36,297] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,297] jepsen worker 78 - jepsen.util 168 :ok :read [26 29] INFO [2022-01-10 12:53:36,298] jepsen worker 78 - jepsen.util 168 :invoke :write [26 24] INFO [2022-01-10 12:53:36,299] jepsen worker 62 - jepsen.util 242 :ok :write [20 24] INFO [2022-01-10 12:53:36,300] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,301] jepsen worker 62 - jepsen.util 242 :ok :read [20 24] INFO [2022-01-10 12:53:36,301] jepsen worker 12 - jepsen.util 192 :ok :write [4 45] INFO [2022-01-10 12:53:36,302] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,302] jepsen worker 62 - jepsen.util 242 :invoke :write [20 9] INFO [2022-01-10 12:53:36,302] jepsen worker 12 - jepsen.util 192 :ok :read [4 45] INFO [2022-01-10 12:53:36,303] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,304] jepsen worker 12 - jepsen.util 192 :ok :read [4 45] INFO [2022-01-10 12:53:36,305] jepsen worker 12 - jepsen.util 192 :invoke :write [4 13] INFO [2022-01-10 12:53:36,305] jepsen worker 22 - jepsen.util 112 :ok :write [7 21] INFO [2022-01-10 12:53:36,306] jepsen worker 22 - jepsen.util 112 :invoke :write [7 24] INFO [2022-01-10 12:53:36,307] jepsen worker 51 - jepsen.util 141 :ok :write [17 15] INFO [2022-01-10 12:53:36,308] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,309] jepsen worker 51 - jepsen.util 141 :ok :read [17 15] INFO [2022-01-10 12:53:36,309] jepsen worker 51 - jepsen.util 141 :invoke :write [17 32] INFO [2022-01-10 12:53:36,310] jepsen worker 44 - jepsen.util 44 :ok :write [14 14] INFO [2022-01-10 12:53:36,311] jepsen worker 44 - jepsen.util 44 :invoke :write [14 42] INFO [2022-01-10 12:53:36,312] jepsen worker 69 - jepsen.util 159 :ok :write [23 24] INFO [2022-01-10 12:53:36,313] jepsen worker 69 - jepsen.util 159 :invoke :write [23 31] INFO [2022-01-10 12:53:36,316] jepsen worker 75 - jepsen.util 165 :ok :write [25 9] INFO [2022-01-10 12:53:36,316] jepsen worker 75 - jepsen.util 165 :invoke :write [25 5] INFO [2022-01-10 12:53:36,318] jepsen worker 78 - jepsen.util 168 :ok :write [26 24] INFO [2022-01-10 12:53:36,318] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,319] jepsen worker 78 - jepsen.util 168 :ok :read [26 24] INFO [2022-01-10 12:53:36,320] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,321] jepsen worker 78 - jepsen.util 168 :ok :read [26 24] INFO [2022-01-10 12:53:36,321] jepsen worker 62 - jepsen.util 242 :ok :write [20 9] INFO [2022-01-10 12:53:36,321] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,322] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,322] jepsen worker 78 - jepsen.util 168 :ok :read [26 24] INFO [2022-01-10 12:53:36,323] jepsen worker 62 - jepsen.util 242 :ok :read [20 9] INFO [2022-01-10 12:53:36,323] jepsen worker 78 - jepsen.util 168 :invoke :write [26 7] INFO [2022-01-10 12:53:36,324] jepsen worker 62 - jepsen.util 242 :invoke :write [20 45] INFO [2022-01-10 12:53:36,336] jepsen worker 89 - jepsen.util 269 :ok :write [29 21] INFO [2022-01-10 12:53:36,337] jepsen worker 89 - jepsen.util 269 :invoke :write [29 9] INFO [2022-01-10 12:53:36,341] jepsen worker 1 - jepsen.util 91 :ok :write [0 15] INFO [2022-01-10 12:53:36,342] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:36,343] jepsen worker 1 - jepsen.util 91 :ok :read [0 15] INFO [2022-01-10 12:53:36,343] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:36,344] jepsen worker 1 - jepsen.util 91 :ok :read [0 15] INFO [2022-01-10 12:53:36,344] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:36,345] jepsen worker 1 - jepsen.util 91 :ok :read [0 15] INFO [2022-01-10 12:53:36,345] jepsen worker 1 - jepsen.util 91 :invoke :write [0 34] INFO [2022-01-10 12:53:36,346] jepsen worker 33 - jepsen.util 123 :ok :write [11 42] INFO [2022-01-10 12:53:36,346] jepsen worker 84 - jepsen.util 84 :ok :write [28 32] INFO [2022-01-10 12:53:36,347] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:36,348] jepsen worker 33 - jepsen.util 123 :invoke :write [11 32] INFO [2022-01-10 12:53:36,348] jepsen worker 84 - jepsen.util 84 :ok :read [28 32] INFO [2022-01-10 12:53:36,349] jepsen worker 84 - jepsen.util 84 :invoke :write [28 21] INFO [2022-01-10 12:53:36,353] jepsen worker 3 - jepsen.util 3 :ok :write [1 42] INFO [2022-01-10 12:53:36,353] jepsen worker 68 - jepsen.util 68 :ok :write [22 22] INFO [2022-01-10 12:53:36,353] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:36,353] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:36,354] jepsen worker 3 - jepsen.util 3 :ok :read [1 42] INFO [2022-01-10 12:53:36,354] jepsen worker 68 - jepsen.util 68 :ok :read [22 22] INFO [2022-01-10 12:53:36,354] jepsen worker 3 - jepsen.util 3 :invoke :write [1 25] INFO [2022-01-10 12:53:36,355] jepsen worker 68 - jepsen.util 68 :invoke :write [22 24] INFO [2022-01-10 12:53:36,359] jepsen worker 38 - jepsen.util 38 :ok :write [12 45] INFO [2022-01-10 12:53:36,360] jepsen worker 38 - jepsen.util 38 :invoke :write [12 42] INFO [2022-01-10 12:53:36,363] jepsen worker 40 - jepsen.util 130 :ok :write [13 26] INFO [2022-01-10 12:53:36,363] jepsen worker 40 - jepsen.util 130 :invoke :write [13 19] INFO [2022-01-10 12:53:36,368] jepsen worker 7 - jepsen.util 7 :ok :write [2 16] INFO [2022-01-10 12:53:36,369] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:36,369] jepsen worker 7 - jepsen.util 7 :ok :read [2 16] INFO [2022-01-10 12:53:36,370] jepsen worker 7 - jepsen.util 7 :invoke :write [2 21] INFO [2022-01-10 12:53:36,374] jepsen worker 15 - jepsen.util 15 :ok :write [5 38] INFO [2022-01-10 12:53:36,375] jepsen worker 15 - jepsen.util 15 :invoke :write [5 47] INFO [2022-01-10 12:53:36,378] jepsen worker 50 - jepsen.util 50 :ok :write [16 42] INFO [2022-01-10 12:53:36,378] jepsen worker 50 - jepsen.util 50 :invoke :write [16 34] INFO [2022-01-10 12:53:36,383] jepsen worker 83 - jepsen.util 263 :ok :write [27 34] INFO [2022-01-10 12:53:36,383] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:36,384] jepsen worker 83 - jepsen.util 263 :ok :read [27 34] INFO [2022-01-10 12:53:36,385] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:36,385] jepsen worker 83 - jepsen.util 263 :ok :read [27 34] INFO [2022-01-10 12:53:36,386] jepsen worker 83 - jepsen.util 263 :invoke :write [27 6] INFO [2022-01-10 12:53:36,432] jepsen worker 22 - jepsen.util 112 :ok :write [7 24] INFO [2022-01-10 12:53:36,432] jepsen worker 12 - jepsen.util 192 :ok :write [4 13] INFO [2022-01-10 12:53:36,433] jepsen worker 22 - jepsen.util 112 :invoke :write [7 18] INFO [2022-01-10 12:53:36,433] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,435] jepsen worker 12 - jepsen.util 192 :ok :read [4 13] INFO [2022-01-10 12:53:36,435] jepsen worker 12 - jepsen.util 192 :invoke :write [4 35] INFO [2022-01-10 12:53:36,438] jepsen worker 44 - jepsen.util 44 :ok :write [14 42] INFO [2022-01-10 12:53:36,439] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,440] jepsen worker 44 - jepsen.util 44 :ok :read [14 42] INFO [2022-01-10 12:53:36,440] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,441] jepsen worker 44 - jepsen.util 44 :ok :read [14 42] INFO [2022-01-10 12:53:36,441] jepsen worker 44 - jepsen.util 44 :invoke :write [14 8] INFO [2022-01-10 12:53:36,443] jepsen worker 51 - jepsen.util 141 :ok :write [17 32] INFO [2022-01-10 12:53:36,444] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,445] jepsen worker 51 - jepsen.util 141 :ok :read [17 32] INFO [2022-01-10 12:53:36,445] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,446] jepsen worker 51 - jepsen.util 141 :ok :read [17 32] INFO [2022-01-10 12:53:36,446] jepsen worker 51 - jepsen.util 141 :invoke :write [17 47] INFO [2022-01-10 12:53:36,446] jepsen worker 75 - jepsen.util 165 :ok :write [25 5] INFO [2022-01-10 12:53:36,447] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,448] jepsen worker 75 - jepsen.util 165 :ok :read [25 5] INFO [2022-01-10 12:53:36,448] jepsen worker 75 - jepsen.util 165 :invoke :write [25 9] INFO [2022-01-10 12:53:36,451] jepsen worker 69 - jepsen.util 159 :ok :write [23 31] INFO [2022-01-10 12:53:36,452] jepsen worker 62 - jepsen.util 242 :ok :write [20 45] INFO [2022-01-10 12:53:36,452] jepsen worker 69 - jepsen.util 159 :invoke :write [23 2] INFO [2022-01-10 12:53:36,452] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,453] jepsen worker 62 - jepsen.util 242 :ok :read [20 45] INFO [2022-01-10 12:53:36,454] jepsen worker 62 - jepsen.util 242 :invoke :write [20 4] INFO [2022-01-10 12:53:36,459] jepsen worker 78 - jepsen.util 168 :ok :write [26 7] INFO [2022-01-10 12:53:36,459] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,460] jepsen worker 78 - jepsen.util 168 :ok :read [26 7] INFO [2022-01-10 12:53:36,461] jepsen worker 78 - jepsen.util 168 :invoke :write [26 27] INFO [2022-01-10 12:53:36,466] jepsen worker 84 - jepsen.util 84 :ok :write [28 21] INFO [2022-01-10 12:53:36,467] jepsen worker 84 - jepsen.util 84 :invoke :write [28 6] INFO [2022-01-10 12:53:36,471] jepsen worker 3 - jepsen.util 3 :ok :write [1 25] INFO [2022-01-10 12:53:36,472] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:36,473] jepsen worker 3 - jepsen.util 3 :ok :read [1 25] INFO [2022-01-10 12:53:36,473] jepsen worker 3 - jepsen.util 3 :invoke :write [1 27] INFO [2022-01-10 12:53:36,476] jepsen worker 38 - jepsen.util 38 :ok :write [12 42] INFO [2022-01-10 12:53:36,477] jepsen worker 38 - jepsen.util 38 :invoke :write [12 36] INFO [2022-01-10 12:53:36,484] jepsen worker 12 - jepsen.util 192 :ok :write [4 35] INFO [2022-01-10 12:53:36,484] jepsen worker 12 - jepsen.util 192 :invoke :write [4 43] INFO [2022-01-10 12:53:36,487] jepsen worker 22 - jepsen.util 112 :ok :write [7 18] INFO [2022-01-10 12:53:36,487] jepsen worker 22 - jepsen.util 112 :invoke :write [7 8] INFO [2022-01-10 12:53:36,493] jepsen worker 51 - jepsen.util 141 :ok :write [17 47] INFO [2022-01-10 12:53:36,493] jepsen worker 51 - jepsen.util 141 :invoke :write [17 0] INFO [2022-01-10 12:53:36,493] jepsen worker 44 - jepsen.util 44 :ok :write [14 8] INFO [2022-01-10 12:53:36,494] jepsen worker 44 - jepsen.util 44 :invoke :write [14 24] INFO [2022-01-10 12:53:36,499] jepsen worker 75 - jepsen.util 165 :ok :write [25 9] INFO [2022-01-10 12:53:36,499] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,500] jepsen worker 75 - jepsen.util 165 :ok :read [25 9] INFO [2022-01-10 12:53:36,500] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,501] jepsen worker 75 - jepsen.util 165 :ok :read [25 9] INFO [2022-01-10 12:53:36,502] jepsen worker 69 - jepsen.util 159 :ok :write [23 2] INFO [2022-01-10 12:53:36,502] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,503] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,503] jepsen worker 75 - jepsen.util 165 :ok :read [25 9] INFO [2022-01-10 12:53:36,503] jepsen worker 62 - jepsen.util 242 :ok :write [20 4] INFO [2022-01-10 12:53:36,504] jepsen worker 69 - jepsen.util 159 :ok :read [23 2] INFO [2022-01-10 12:53:36,504] jepsen worker 75 - jepsen.util 165 :invoke :write [25 19] INFO [2022-01-10 12:53:36,505] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,505] jepsen worker 62 - jepsen.util 242 :invoke :write [20 12] INFO [2022-01-10 12:53:36,506] jepsen worker 69 - jepsen.util 159 :ok :read [23 2] INFO [2022-01-10 12:53:36,507] jepsen worker 69 - jepsen.util 159 :invoke :write [23 18] INFO [2022-01-10 12:53:36,510] jepsen worker 78 - jepsen.util 168 :ok :write [26 27] INFO [2022-01-10 12:53:36,510] jepsen worker 22 - jepsen.util 112 :ok :write [7 8] INFO [2022-01-10 12:53:36,511] jepsen worker 78 - jepsen.util 168 :invoke :write [26 41] INFO [2022-01-10 12:53:36,511] jepsen worker 22 - jepsen.util 112 :invoke :write [7 31] INFO [2022-01-10 12:53:36,514] jepsen worker 44 - jepsen.util 44 :ok :write [14 24] INFO [2022-01-10 12:53:36,514] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,515] jepsen worker 44 - jepsen.util 44 :ok :read [14 24] INFO [2022-01-10 12:53:36,515] jepsen worker 44 - jepsen.util 44 :invoke :write [14 5] INFO [2022-01-10 12:53:36,519] jepsen worker 75 - jepsen.util 165 :ok :write [25 19] INFO [2022-01-10 12:53:36,520] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,521] jepsen worker 75 - jepsen.util 165 :ok :read [25 19] INFO [2022-01-10 12:53:36,522] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,522] jepsen worker 75 - jepsen.util 165 :ok :read [25 19] INFO [2022-01-10 12:53:36,523] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,524] jepsen worker 75 - jepsen.util 165 :ok :read [25 19] INFO [2022-01-10 12:53:36,524] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,524] jepsen worker 61 - jepsen.util 61 :ok :write [20 33] INFO [2022-01-10 12:53:36,525] jepsen worker 75 - jepsen.util 165 :ok :read [25 19] INFO [2022-01-10 12:53:36,525] jepsen worker 61 - jepsen.util 61 :invoke :write [20 22] INFO [2022-01-10 12:53:36,525] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,526] jepsen worker 12 - jepsen.util 192 :ok :write [4 43] INFO [2022-01-10 12:53:36,526] jepsen worker 75 - jepsen.util 165 :ok :read [25 19] INFO [2022-01-10 12:53:36,526] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,527] jepsen worker 75 - jepsen.util 165 :invoke :write [25 12] INFO [2022-01-10 12:53:36,529] jepsen worker 12 - jepsen.util 192 :ok :read [4 43] INFO [2022-01-10 12:53:36,529] jepsen worker 22 - jepsen.util 112 :ok :write [7 31] INFO [2022-01-10 12:53:36,529] jepsen worker 12 - jepsen.util 192 :invoke :write [4 18] INFO [2022-01-10 12:53:36,530] jepsen worker 22 - jepsen.util 112 :invoke :write [7 10] INFO [2022-01-10 12:53:36,532] jepsen worker 51 - jepsen.util 141 :ok :write [17 0] INFO [2022-01-10 12:53:36,532] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,533] jepsen worker 51 - jepsen.util 141 :ok :read [17 0] INFO [2022-01-10 12:53:36,533] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,535] jepsen worker 51 - jepsen.util 141 :ok :read [17 0] INFO [2022-01-10 12:53:36,535] jepsen worker 44 - jepsen.util 44 :ok :write [14 5] INFO [2022-01-10 12:53:36,535] jepsen worker 44 - jepsen.util 44 :invoke :write [14 28] INFO [2022-01-10 12:53:36,536] jepsen worker 51 - jepsen.util 141 :invoke :write [17 19] INFO [2022-01-10 12:53:36,537] jepsen worker 69 - jepsen.util 159 :ok :write [23 18] INFO [2022-01-10 12:53:36,537] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,538] jepsen worker 69 - jepsen.util 159 :ok :read [23 18] INFO [2022-01-10 12:53:36,539] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,540] jepsen worker 69 - jepsen.util 159 :ok :read [23 18] INFO [2022-01-10 12:53:36,541] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,543] jepsen worker 69 - jepsen.util 159 :ok :read [23 18] INFO [2022-01-10 12:53:36,543] jepsen worker 78 - jepsen.util 168 :ok :write [26 41] INFO [2022-01-10 12:53:36,544] jepsen worker 78 - jepsen.util 168 :invoke :write [26 13] INFO [2022-01-10 12:53:36,544] jepsen worker 69 - jepsen.util 159 :invoke :write [23 29] INFO [2022-01-10 12:53:36,546] jepsen worker 12 - jepsen.util 192 :ok :write [4 18] INFO [2022-01-10 12:53:36,547] jepsen worker 12 - jepsen.util 192 :invoke :write [4 42] INFO [2022-01-10 12:53:36,547] jepsen worker 62 - jepsen.util 242 :ok :write [20 12] INFO [2022-01-10 12:53:36,548] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,549] jepsen worker 62 - jepsen.util 242 :ok :read [20 12] INFO [2022-01-10 12:53:36,549] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,550] jepsen worker 62 - jepsen.util 242 :ok :read [20 12] INFO [2022-01-10 12:53:36,551] jepsen worker 62 - jepsen.util 242 :invoke :write [20 16] INFO [2022-01-10 12:53:36,551] jepsen worker 1 - jepsen.util 91 :ok :write [0 34] INFO [2022-01-10 12:53:36,551] jepsen worker 89 - jepsen.util 269 :ok :write [29 9] INFO [2022-01-10 12:53:36,552] jepsen worker 1 - jepsen.util 91 :invoke :write [0 44] INFO [2022-01-10 12:53:36,552] jepsen worker 89 - jepsen.util 269 :invoke :write [29 19] INFO [2022-01-10 12:53:36,553] jepsen worker 75 - jepsen.util 165 :ok :write [25 12] INFO [2022-01-10 12:53:36,553] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,554] jepsen worker 75 - jepsen.util 165 :ok :read [25 12] INFO [2022-01-10 12:53:36,555] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,555] jepsen worker 75 - jepsen.util 165 :ok :read [25 12] INFO [2022-01-10 12:53:36,556] jepsen worker 22 - jepsen.util 112 :ok :write [7 10] INFO [2022-01-10 12:53:36,556] jepsen worker 75 - jepsen.util 165 :invoke :write [25 13] INFO [2022-01-10 12:53:36,556] jepsen worker 51 - jepsen.util 141 :ok :write [17 19] INFO [2022-01-10 12:53:36,557] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,557] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,558] jepsen worker 22 - jepsen.util 112 :ok :read [7 10] INFO [2022-01-10 12:53:36,558] jepsen worker 84 - jepsen.util 84 :ok :write [28 6] INFO [2022-01-10 12:53:36,558] jepsen worker 84 - jepsen.util 84 :invoke :write [28 22] INFO [2022-01-10 12:53:36,559] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,559] jepsen worker 22 - jepsen.util 112 :ok :read [7 10] INFO [2022-01-10 12:53:36,560] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,560] jepsen worker 51 - jepsen.util 141 :ok :read [17 19] INFO [2022-01-10 12:53:36,560] jepsen worker 44 - jepsen.util 44 :ok :write [14 28] INFO [2022-01-10 12:53:36,561] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,561] jepsen worker 22 - jepsen.util 112 :ok :read [7 10] INFO [2022-01-10 12:53:36,561] jepsen worker 44 - jepsen.util 44 :invoke :write [14 49] INFO [2022-01-10 12:53:36,561] jepsen worker 51 - jepsen.util 141 :ok :read [17 19] INFO [2022-01-10 12:53:36,561] jepsen worker 22 - jepsen.util 112 :invoke :write [7 46] INFO [2022-01-10 12:53:36,562] jepsen worker 51 - jepsen.util 141 :invoke :write [17 2] INFO [2022-01-10 12:53:36,566] jepsen worker 78 - jepsen.util 168 :ok :write [26 13] INFO [2022-01-10 12:53:36,567] jepsen worker 62 - jepsen.util 242 :ok :write [20 16] INFO [2022-01-10 12:53:36,567] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,568] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,568] jepsen worker 78 - jepsen.util 168 :ok :read [26 13] INFO [2022-01-10 12:53:36,569] jepsen worker 62 - jepsen.util 242 :ok :read [20 16] INFO [2022-01-10 12:53:36,569] jepsen worker 78 - jepsen.util 168 :invoke :write [26 31] INFO [2022-01-10 12:53:36,569] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,570] jepsen worker 62 - jepsen.util 242 :ok :read [20 16] INFO [2022-01-10 12:53:36,571] jepsen worker 62 - jepsen.util 242 :invoke :write [20 13] INFO [2022-01-10 12:53:36,571] jepsen worker 12 - jepsen.util 192 :ok :write [4 42] INFO [2022-01-10 12:53:36,572] jepsen worker 12 - jepsen.util 192 :invoke :write [4 30] INFO [2022-01-10 12:53:36,573] jepsen worker 69 - jepsen.util 159 :ok :write [23 29] INFO [2022-01-10 12:53:36,574] jepsen worker 69 - jepsen.util 159 :invoke :write [23 7] INFO [2022-01-10 12:53:36,575] jepsen worker 75 - jepsen.util 165 :ok :write [25 13] INFO [2022-01-10 12:53:36,576] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,576] jepsen worker 75 - jepsen.util 165 :ok :read [25 13] INFO [2022-01-10 12:53:36,577] jepsen worker 75 - jepsen.util 165 :invoke :write [25 17] INFO [2022-01-10 12:53:36,579] jepsen worker 51 - jepsen.util 141 :ok :write [17 2] INFO [2022-01-10 12:53:36,579] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,580] jepsen worker 51 - jepsen.util 141 :ok :read [17 2] INFO [2022-01-10 12:53:36,581] jepsen worker 51 - jepsen.util 141 :invoke :write [17 44] INFO [2022-01-10 12:53:36,583] jepsen worker 22 - jepsen.util 112 :ok :write [7 46] INFO [2022-01-10 12:53:36,584] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,585] jepsen worker 22 - jepsen.util 112 :ok :read [7 46] INFO [2022-01-10 12:53:36,586] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,586] jepsen worker 33 - jepsen.util 123 :ok :write [11 32] INFO [2022-01-10 12:53:36,586] jepsen worker 22 - jepsen.util 112 :ok :read [7 46] INFO [2022-01-10 12:53:36,587] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,587] jepsen worker 3 - jepsen.util 3 :ok :write [1 27] INFO [2022-01-10 12:53:36,587] jepsen worker 22 - jepsen.util 112 :invoke :write [7 10] INFO [2022-01-10 12:53:36,588] jepsen worker 33 - jepsen.util 123 :ok :read [11 32] INFO [2022-01-10 12:53:36,588] jepsen worker 3 - jepsen.util 3 :invoke :read [1 nil] INFO [2022-01-10 12:53:36,589] jepsen worker 33 - jepsen.util 123 :invoke :write [11 13] INFO [2022-01-10 12:53:36,589] jepsen worker 3 - jepsen.util 3 :ok :read [1 27] INFO [2022-01-10 12:53:36,590] jepsen worker 3 - jepsen.util 3 :invoke :write [1 28] INFO [2022-01-10 12:53:36,590] jepsen worker 68 - jepsen.util 68 :ok :write [22 24] INFO [2022-01-10 12:53:36,591] jepsen worker 68 - jepsen.util 68 :invoke :write [22 0] INFO [2022-01-10 12:53:36,591] jepsen worker 38 - jepsen.util 38 :ok :write [12 36] INFO [2022-01-10 12:53:36,591] jepsen worker 44 - jepsen.util 44 :ok :write [14 49] INFO [2022-01-10 12:53:36,591] jepsen worker 38 - jepsen.util 38 :invoke :write [12 27] INFO [2022-01-10 12:53:36,592] jepsen worker 44 - jepsen.util 44 :invoke :write [14 10] INFO [2022-01-10 12:53:36,595] jepsen worker 78 - jepsen.util 168 :ok :write [26 31] INFO [2022-01-10 12:53:36,596] jepsen worker 84 - jepsen.util 84 :ok :write [28 22] INFO [2022-01-10 12:53:36,596] jepsen worker 78 - jepsen.util 168 :invoke :write [26 31] INFO [2022-01-10 12:53:36,596] jepsen worker 84 - jepsen.util 84 :invoke :write [28 26] INFO [2022-01-10 12:53:36,596] jepsen worker 40 - jepsen.util 130 :ok :write [13 19] INFO [2022-01-10 12:53:36,597] jepsen worker 40 - jepsen.util 130 :invoke :write [13 25] INFO [2022-01-10 12:53:36,598] jepsen worker 62 - jepsen.util 242 :ok :write [20 13] INFO [2022-01-10 12:53:36,598] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,599] jepsen worker 62 - jepsen.util 242 :ok :read [20 13] INFO [2022-01-10 12:53:36,600] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,601] jepsen worker 62 - jepsen.util 242 :ok :read [20 13] INFO [2022-01-10 12:53:36,601] jepsen worker 62 - jepsen.util 242 :invoke :write [20 44] INFO [2022-01-10 12:53:36,604] jepsen worker 12 - jepsen.util 192 :ok :write [4 30] INFO [2022-01-10 12:53:36,605] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,606] jepsen worker 12 - jepsen.util 192 :ok :read [4 30] INFO [2022-01-10 12:53:36,606] jepsen worker 69 - jepsen.util 159 :ok :write [23 7] INFO [2022-01-10 12:53:36,606] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,607] jepsen worker 69 - jepsen.util 159 :invoke :write [23 32] INFO [2022-01-10 12:53:36,607] jepsen worker 12 - jepsen.util 192 :ok :read [4 30] INFO [2022-01-10 12:53:36,607] jepsen worker 75 - jepsen.util 165 :ok :write [25 17] INFO [2022-01-10 12:53:36,608] jepsen worker 75 - jepsen.util 165 :invoke :write [25 12] INFO [2022-01-10 12:53:36,608] jepsen worker 22 - jepsen.util 112 :ok :write [7 10] INFO [2022-01-10 12:53:36,608] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,608] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,609] jepsen worker 12 - jepsen.util 192 :ok :read [4 30] INFO [2022-01-10 12:53:36,609] jepsen worker 22 - jepsen.util 112 :ok :read [7 10] INFO [2022-01-10 12:53:36,609] jepsen worker 12 - jepsen.util 192 :invoke :write [4 6] INFO [2022-01-10 12:53:36,610] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,610] jepsen worker 22 - jepsen.util 112 :ok :read [7 10] INFO [2022-01-10 12:53:36,611] jepsen worker 22 - jepsen.util 112 :invoke :write [7 21] INFO [2022-01-10 12:53:36,612] jepsen worker 51 - jepsen.util 141 :ok :write [17 44] INFO [2022-01-10 12:53:36,612] jepsen worker 44 - jepsen.util 44 :ok :write [14 10] INFO [2022-01-10 12:53:36,612] jepsen worker 51 - jepsen.util 141 :invoke :write [17 4] INFO [2022-01-10 12:53:36,612] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,613] jepsen worker 44 - jepsen.util 44 :ok :read [14 10] INFO [2022-01-10 12:53:36,614] jepsen worker 44 - jepsen.util 44 :invoke :write [14 24] INFO [2022-01-10 12:53:36,623] jepsen worker 62 - jepsen.util 242 :ok :write [20 44] INFO [2022-01-10 12:53:36,624] jepsen worker 78 - jepsen.util 168 :ok :write [26 31] INFO [2022-01-10 12:53:36,624] jepsen worker 62 - jepsen.util 242 :invoke :write [20 23] INFO [2022-01-10 12:53:36,624] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,625] jepsen worker 78 - jepsen.util 168 :ok :read [26 31] INFO [2022-01-10 12:53:36,626] jepsen worker 78 - jepsen.util 168 :invoke :write [26 22] INFO [2022-01-10 12:53:36,628] jepsen worker 75 - jepsen.util 165 :ok :write [25 12] INFO [2022-01-10 12:53:36,629] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,630] jepsen worker 75 - jepsen.util 165 :ok :read [25 12] INFO [2022-01-10 12:53:36,630] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,631] jepsen worker 75 - jepsen.util 165 :ok :read [25 12] INFO [2022-01-10 12:53:36,631] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,632] jepsen worker 75 - jepsen.util 165 :ok :read [25 12] INFO [2022-01-10 12:53:36,633] jepsen worker 70 - jepsen.util 160 :ok :write [23 7] INFO [2022-01-10 12:53:36,633] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,633] jepsen worker 70 - jepsen.util 160 :invoke :write [23 10] INFO [2022-01-10 12:53:36,634] jepsen worker 75 - jepsen.util 165 :ok :read [25 12] INFO [2022-01-10 12:53:36,635] jepsen worker 75 - jepsen.util 165 :invoke :write [25 8] INFO [2022-01-10 12:53:36,635] jepsen worker 22 - jepsen.util 112 :ok :write [7 21] INFO [2022-01-10 12:53:36,636] jepsen worker 22 - jepsen.util 112 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,637] jepsen worker 22 - jepsen.util 112 :ok :read [7 21] INFO [2022-01-10 12:53:36,637] jepsen worker 12 - jepsen.util 192 :ok :write [4 6] INFO [2022-01-10 12:53:36,637] jepsen worker 12 - jepsen.util 192 :invoke :write [4 41] INFO [2022-01-10 12:53:36,638] jepsen worker 22 - jepsen.util 112 :invoke :write [7 2] INFO [2022-01-10 12:53:36,641] jepsen worker 44 - jepsen.util 44 :ok :write [14 24] INFO [2022-01-10 12:53:36,642] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,643] jepsen worker 44 - jepsen.util 44 :ok :read [14 24] INFO [2022-01-10 12:53:36,643] jepsen worker 44 - jepsen.util 44 :invoke :write [14 14] INFO [2022-01-10 12:53:36,644] jepsen worker 51 - jepsen.util 141 :ok :write [17 4] INFO [2022-01-10 12:53:36,645] jepsen worker 51 - jepsen.util 141 :invoke :write [17 16] INFO [2022-01-10 12:53:36,646] jepsen worker 62 - jepsen.util 242 :ok :write [20 23] INFO [2022-01-10 12:53:36,647] jepsen worker 62 - jepsen.util 242 :invoke :write [20 34] INFO [2022-01-10 12:53:36,652] jepsen worker 78 - jepsen.util 168 :ok :write [26 22] INFO [2022-01-10 12:53:36,652] jepsen worker 78 - jepsen.util 168 :invoke :write [26 31] INFO [2022-01-10 12:53:36,655] jepsen worker 75 - jepsen.util 165 :ok :write [25 8] INFO [2022-01-10 12:53:36,656] jepsen worker 75 - jepsen.util 165 :invoke :write [25 21] INFO [2022-01-10 12:53:36,685] jepsen worker 7 - jepsen.util 7 :ok :write [2 21] INFO [2022-01-10 12:53:36,686] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:36,687] jepsen worker 7 - jepsen.util 7 :ok :read [2 21] INFO [2022-01-10 12:53:36,687] jepsen worker 7 - jepsen.util 7 :invoke :write [2 35] INFO [2022-01-10 12:53:36,690] jepsen worker 15 - jepsen.util 15 :ok :write [5 47] INFO [2022-01-10 12:53:36,690] jepsen worker 38 - jepsen.util 38 :ok :write [12 27] INFO [2022-01-10 12:53:36,690] jepsen worker 15 - jepsen.util 15 :invoke :write [5 25] INFO [2022-01-10 12:53:36,691] jepsen worker 38 - jepsen.util 38 :invoke :write [12 43] INFO [2022-01-10 12:53:36,700] jepsen worker 84 - jepsen.util 84 :ok :write [28 26] INFO [2022-01-10 12:53:36,701] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:36,701] jepsen worker 50 - jepsen.util 50 :ok :write [16 34] INFO [2022-01-10 12:53:36,701] jepsen worker 84 - jepsen.util 84 :ok :read [28 26] INFO [2022-01-10 12:53:36,701] jepsen worker 50 - jepsen.util 50 :invoke :write [16 9] INFO [2022-01-10 12:53:36,702] jepsen worker 84 - jepsen.util 84 :invoke :write [28 0] INFO [2022-01-10 12:53:36,704] jepsen worker 6 - jepsen.util 186 :info :write [2 28] :RequestTimeoutException INFO [2022-01-10 12:53:36,705] jepsen worker 6 - jepsen.util 276 :invoke :write [2 42] INFO [2022-01-10 12:53:36,708] jepsen worker 3 - jepsen.util 3 :ok :write [1 28] INFO [2022-01-10 12:53:36,708] jepsen worker 83 - jepsen.util 263 :ok :write [27 6] INFO [2022-01-10 12:53:36,708] jepsen worker 3 - jepsen.util 3 :invoke :write [1 38] INFO [2022-01-10 12:53:36,709] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:36,710] jepsen worker 83 - jepsen.util 263 :ok :read [27 6] INFO [2022-01-10 12:53:36,710] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:36,711] jepsen worker 83 - jepsen.util 263 :ok :read [27 6] INFO [2022-01-10 12:53:36,711] jepsen worker 83 - jepsen.util 263 :invoke :write [27 22] INFO [2022-01-10 12:53:36,714] jepsen worker 89 - jepsen.util 269 :ok :write [29 19] INFO [2022-01-10 12:53:36,715] jepsen worker 89 - jepsen.util 269 :invoke :write [29 12] INFO [2022-01-10 12:53:36,718] jepsen worker 1 - jepsen.util 91 :ok :write [0 44] INFO [2022-01-10 12:53:36,718] jepsen worker 1 - jepsen.util 91 :invoke :write [0 26] INFO [2022-01-10 12:53:36,723] jepsen worker 33 - jepsen.util 123 :ok :write [11 13] INFO [2022-01-10 12:53:36,724] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,724] jepsen worker 33 - jepsen.util 123 :ok :read [11 13] INFO [2022-01-10 12:53:36,725] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,726] jepsen worker 33 - jepsen.util 123 :ok :read [11 13] INFO [2022-01-10 12:53:36,726] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,726] jepsen worker 68 - jepsen.util 68 :ok :write [22 0] INFO [2022-01-10 12:53:36,727] jepsen worker 33 - jepsen.util 123 :ok :read [11 13] INFO [2022-01-10 12:53:36,727] jepsen worker 68 - jepsen.util 68 :invoke :write [22 6] INFO [2022-01-10 12:53:36,727] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,728] jepsen worker 33 - jepsen.util 123 :ok :read [11 13] INFO [2022-01-10 12:53:36,728] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,729] jepsen worker 33 - jepsen.util 123 :ok :read [11 13] INFO [2022-01-10 12:53:36,729] jepsen worker 33 - jepsen.util 123 :invoke :write [11 19] INFO [2022-01-10 12:53:36,730] jepsen worker 40 - jepsen.util 130 :ok :write [13 25] INFO [2022-01-10 12:53:36,731] jepsen worker 40 - jepsen.util 130 :invoke :write [13 21] INFO [2022-01-10 12:53:36,795] jepsen worker 70 - jepsen.util 160 :ok :write [23 10] INFO [2022-01-10 12:53:36,796] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,796] jepsen worker 21 - jepsen.util 201 :ok :write [7 2] INFO [2022-01-10 12:53:36,796] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,797] jepsen worker 70 - jepsen.util 160 :ok :read [23 10] INFO [2022-01-10 12:53:36,797] jepsen worker 21 - jepsen.util 201 :ok :read [7 2] INFO [2022-01-10 12:53:36,797] jepsen worker 70 - jepsen.util 160 :invoke :write [23 35] INFO [2022-01-10 12:53:36,798] jepsen worker 21 - jepsen.util 201 :invoke :write [7 20] INFO [2022-01-10 12:53:36,799] jepsen worker 12 - jepsen.util 192 :ok :write [4 41] INFO [2022-01-10 12:53:36,800] jepsen worker 12 - jepsen.util 192 :invoke :write [4 13] INFO [2022-01-10 12:53:36,803] jepsen worker 44 - jepsen.util 44 :ok :write [14 14] INFO [2022-01-10 12:53:36,804] jepsen worker 44 - jepsen.util 44 :invoke :write [14 27] INFO [2022-01-10 12:53:36,807] jepsen worker 51 - jepsen.util 141 :ok :write [17 16] INFO [2022-01-10 12:53:36,808] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,809] jepsen worker 62 - jepsen.util 242 :ok :write [20 34] INFO [2022-01-10 12:53:36,809] jepsen worker 51 - jepsen.util 141 :ok :read [17 16] INFO [2022-01-10 12:53:36,809] jepsen worker 62 - jepsen.util 242 :invoke :write [20 29] INFO [2022-01-10 12:53:36,810] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,810] jepsen worker 51 - jepsen.util 141 :ok :read [17 16] INFO [2022-01-10 12:53:36,811] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,812] jepsen worker 51 - jepsen.util 141 :ok :read [17 16] INFO [2022-01-10 12:53:36,812] jepsen worker 89 - jepsen.util 269 :ok :write [29 12] INFO [2022-01-10 12:53:36,812] jepsen worker 51 - jepsen.util 141 :invoke :write [17 4] INFO [2022-01-10 12:53:36,813] jepsen worker 89 - jepsen.util 269 :invoke :write [29 26] INFO [2022-01-10 12:53:36,815] jepsen worker 75 - jepsen.util 165 :ok :write [25 21] INFO [2022-01-10 12:53:36,815] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,816] jepsen worker 75 - jepsen.util 165 :ok :read [25 21] INFO [2022-01-10 12:53:36,817] jepsen worker 75 - jepsen.util 165 :invoke :write [25 30] INFO [2022-01-10 12:53:36,817] jepsen worker 78 - jepsen.util 168 :ok :write [26 31] INFO [2022-01-10 12:53:36,817] jepsen worker 1 - jepsen.util 91 :ok :write [0 26] INFO [2022-01-10 12:53:36,818] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,819] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:36,820] jepsen worker 78 - jepsen.util 168 :ok :read [26 31] INFO [2022-01-10 12:53:36,820] jepsen worker 1 - jepsen.util 91 :ok :read [0 26] INFO [2022-01-10 12:53:36,820] jepsen worker 7 - jepsen.util 7 :ok :write [2 35] INFO [2022-01-10 12:53:36,821] jepsen worker 7 - jepsen.util 7 :invoke :write [2 34] INFO [2022-01-10 12:53:36,821] jepsen worker 78 - jepsen.util 168 :invoke :write [26 42] INFO [2022-01-10 12:53:36,821] jepsen worker 40 - jepsen.util 130 :ok :write [13 21] INFO [2022-01-10 12:53:36,822] jepsen worker 1 - jepsen.util 91 :invoke :write [0 45] INFO [2022-01-10 12:53:36,822] jepsen worker 40 - jepsen.util 130 :invoke :write [13 3] INFO [2022-01-10 12:53:36,826] jepsen worker 15 - jepsen.util 15 :ok :write [5 25] INFO [2022-01-10 12:53:36,827] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:36,828] jepsen worker 15 - jepsen.util 15 :ok :read [5 25] INFO [2022-01-10 12:53:36,828] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:36,829] jepsen worker 15 - jepsen.util 15 :ok :read [5 25] INFO [2022-01-10 12:53:36,829] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:36,830] jepsen worker 48 - jepsen.util 228 :ok :write [16 15] INFO [2022-01-10 12:53:36,830] jepsen worker 38 - jepsen.util 38 :ok :write [12 43] INFO [2022-01-10 12:53:36,830] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:36,831] jepsen worker 15 - jepsen.util 15 :ok :read [5 25] INFO [2022-01-10 12:53:36,831] jepsen worker 48 - jepsen.util 228 :ok :read [16 15] INFO [2022-01-10 12:53:36,832] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:36,832] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:36,833] jepsen worker 48 - jepsen.util 228 :invoke :write [16 7] INFO [2022-01-10 12:53:36,833] jepsen worker 15 - jepsen.util 15 :ok :read [5 25] INFO [2022-01-10 12:53:36,833] jepsen worker 38 - jepsen.util 38 :ok :read [12 43] INFO [2022-01-10 12:53:36,833] jepsen worker 15 - jepsen.util 15 :invoke :write [5 19] INFO [2022-01-10 12:53:36,834] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:36,834] jepsen worker 83 - jepsen.util 263 :ok :write [27 22] INFO [2022-01-10 12:53:36,835] jepsen worker 38 - jepsen.util 38 :ok :read [12 43] INFO [2022-01-10 12:53:36,835] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:36,835] jepsen worker 38 - jepsen.util 38 :invoke :write [12 9] INFO [2022-01-10 12:53:36,836] jepsen worker 83 - jepsen.util 263 :ok :read [27 22] INFO [2022-01-10 12:53:36,836] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:36,837] jepsen worker 83 - jepsen.util 263 :ok :read [27 22] INFO [2022-01-10 12:53:36,837] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:36,838] jepsen worker 83 - jepsen.util 263 :ok :read [27 22] INFO [2022-01-10 12:53:36,838] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:36,839] jepsen worker 33 - jepsen.util 123 :ok :write [11 19] INFO [2022-01-10 12:53:36,839] jepsen worker 84 - jepsen.util 84 :ok :write [28 0] INFO [2022-01-10 12:53:36,839] jepsen worker 83 - jepsen.util 263 :ok :read [27 22] INFO [2022-01-10 12:53:36,839] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,840] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:36,840] jepsen worker 33 - jepsen.util 123 :ok :read [11 19] INFO [2022-01-10 12:53:36,840] jepsen worker 83 - jepsen.util 263 :ok :read [27 22] INFO [2022-01-10 12:53:36,841] jepsen worker 84 - jepsen.util 84 :invoke :write [28 22] INFO [2022-01-10 12:53:36,841] jepsen worker 83 - jepsen.util 263 :invoke :write [27 33] INFO [2022-01-10 12:53:36,841] jepsen worker 89 - jepsen.util 269 :ok :write [29 26] INFO [2022-01-10 12:53:36,842] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,842] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:36,842] jepsen worker 33 - jepsen.util 123 :ok :read [11 19] INFO [2022-01-10 12:53:36,843] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,843] jepsen worker 89 - jepsen.util 269 :ok :read [29 26] INFO [2022-01-10 12:53:36,843] jepsen worker 3 - jepsen.util 3 :ok :write [1 38] INFO [2022-01-10 12:53:36,844] jepsen worker 68 - jepsen.util 68 :ok :write [22 6] INFO [2022-01-10 12:53:36,844] jepsen worker 33 - jepsen.util 123 :ok :read [11 19] INFO [2022-01-10 12:53:36,844] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:36,845] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:36,845] jepsen worker 89 - jepsen.util 269 :ok :read [29 26] INFO [2022-01-10 12:53:36,845] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:36,845] jepsen worker 33 - jepsen.util 123 :ok :read [11 19] INFO [2022-01-10 12:53:36,846] jepsen worker 33 - jepsen.util 123 :invoke :write [11 1] INFO [2022-01-10 12:53:36,846] jepsen worker 68 - jepsen.util 68 :ok :read [22 6] INFO [2022-01-10 12:53:36,847] jepsen worker 3 - jepsen.util 3 :invoke :write [1 44] INFO [2022-01-10 12:53:36,847] jepsen worker 21 - jepsen.util 201 :ok :write [7 20] INFO [2022-01-10 12:53:36,847] jepsen worker 68 - jepsen.util 68 :invoke :write [22 6] INFO [2022-01-10 12:53:36,848] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,848] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:36,849] jepsen worker 21 - jepsen.util 201 :ok :read [7 20] INFO [2022-01-10 12:53:36,849] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,850] jepsen worker 89 - jepsen.util 269 :ok :read [29 26] INFO [2022-01-10 12:53:36,850] jepsen worker 89 - jepsen.util 269 :invoke :write [29 41] INFO [2022-01-10 12:53:36,851] jepsen worker 21 - jepsen.util 201 :ok :read [7 20] INFO [2022-01-10 12:53:36,851] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,852] jepsen worker 21 - jepsen.util 201 :ok :read [7 20] INFO [2022-01-10 12:53:36,852] jepsen worker 21 - jepsen.util 201 :invoke :write [7 46] INFO [2022-01-10 12:53:36,853] jepsen worker 1 - jepsen.util 91 :ok :write [0 45] INFO [2022-01-10 12:53:36,853] jepsen worker 44 - jepsen.util 44 :ok :write [14 27] INFO [2022-01-10 12:53:36,854] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:36,854] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,855] jepsen worker 1 - jepsen.util 91 :ok :read [0 45] INFO [2022-01-10 12:53:36,855] jepsen worker 69 - jepsen.util 159 :ok :write [23 32] INFO [2022-01-10 12:53:36,855] jepsen worker 44 - jepsen.util 44 :ok :read [14 27] INFO [2022-01-10 12:53:36,855] jepsen worker 69 - jepsen.util 159 :invoke :write [23 27] INFO [2022-01-10 12:53:36,856] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,856] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:36,857] jepsen worker 1 - jepsen.util 91 :ok :read [0 45] INFO [2022-01-10 12:53:36,857] jepsen worker 44 - jepsen.util 44 :ok :read [14 27] INFO [2022-01-10 12:53:36,857] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:36,858] jepsen worker 1 - jepsen.util 91 :invoke :write [0 49] INFO [2022-01-10 12:53:36,859] jepsen worker 44 - jepsen.util 44 :ok :read [14 27] INFO [2022-01-10 12:53:36,859] jepsen worker 62 - jepsen.util 242 :ok :write [20 29] INFO [2022-01-10 12:53:36,859] jepsen worker 62 - jepsen.util 242 :invoke :write [20 21] INFO [2022-01-10 12:53:36,859] jepsen worker 44 - jepsen.util 44 :invoke :write [14 43] INFO [2022-01-10 12:53:36,863] jepsen worker 12 - jepsen.util 192 :ok :write [4 13] INFO [2022-01-10 12:53:36,863] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,864] jepsen worker 12 - jepsen.util 192 :ok :read [4 13] INFO [2022-01-10 12:53:36,865] jepsen worker 75 - jepsen.util 165 :ok :write [25 30] INFO [2022-01-10 12:53:36,865] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,865] jepsen worker 75 - jepsen.util 165 :invoke :write [25 39] INFO [2022-01-10 12:53:36,865] jepsen worker 12 - jepsen.util 192 :ok :read [4 13] INFO [2022-01-10 12:53:36,866] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,867] jepsen worker 12 - jepsen.util 192 :ok :read [4 13] INFO [2022-01-10 12:53:36,867] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,868] jepsen worker 51 - jepsen.util 141 :ok :write [17 4] INFO [2022-01-10 12:53:36,868] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,869] jepsen worker 12 - jepsen.util 192 :ok :read [4 13] INFO [2022-01-10 12:53:36,869] jepsen worker 12 - jepsen.util 192 :invoke :write [4 39] INFO [2022-01-10 12:53:36,869] jepsen worker 51 - jepsen.util 141 :ok :read [17 4] INFO [2022-01-10 12:53:36,870] jepsen worker 51 - jepsen.util 141 :invoke :write [17 29] INFO [2022-01-10 12:53:36,873] jepsen worker 78 - jepsen.util 168 :ok :write [26 42] INFO [2022-01-10 12:53:36,873] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,874] jepsen worker 78 - jepsen.util 168 :ok :read [26 42] INFO [2022-01-10 12:53:36,875] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,876] jepsen worker 78 - jepsen.util 168 :ok :read [26 42] INFO [2022-01-10 12:53:36,876] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,877] jepsen worker 78 - jepsen.util 168 :ok :read [26 42] INFO [2022-01-10 12:53:36,878] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,879] jepsen worker 78 - jepsen.util 168 :ok :read [26 42] INFO [2022-01-10 12:53:36,879] jepsen worker 78 - jepsen.util 168 :invoke :write [26 22] INFO [2022-01-10 12:53:36,888] jepsen worker 70 - jepsen.util 160 :ok :write [23 35] INFO [2022-01-10 12:53:36,888] jepsen worker 70 - jepsen.util 160 :invoke :write [23 22] INFO [2022-01-10 12:53:36,891] jepsen worker 12 - jepsen.util 192 :ok :write [4 39] INFO [2022-01-10 12:53:36,892] jepsen worker 12 - jepsen.util 192 :invoke :write [4 41] INFO [2022-01-10 12:53:36,894] jepsen worker 51 - jepsen.util 141 :ok :write [17 29] INFO [2022-01-10 12:53:36,895] jepsen worker 51 - jepsen.util 141 :invoke :write [17 19] INFO [2022-01-10 12:53:36,900] jepsen worker 78 - jepsen.util 168 :ok :write [26 22] INFO [2022-01-10 12:53:36,900] jepsen worker 78 - jepsen.util 168 :invoke :write [26 0] INFO [2022-01-10 12:53:36,902] jepsen worker 70 - jepsen.util 160 :ok :write [23 22] INFO [2022-01-10 12:53:36,903] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,904] jepsen worker 70 - jepsen.util 160 :ok :read [23 22] INFO [2022-01-10 12:53:36,904] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,904] jepsen worker 21 - jepsen.util 201 :ok :write [7 46] INFO [2022-01-10 12:53:36,905] jepsen worker 70 - jepsen.util 160 :ok :read [23 22] INFO [2022-01-10 12:53:36,905] jepsen worker 21 - jepsen.util 201 :invoke :write [7 22] INFO [2022-01-10 12:53:36,906] jepsen worker 70 - jepsen.util 160 :invoke :write [23 27] INFO [2022-01-10 12:53:36,907] jepsen worker 12 - jepsen.util 192 :ok :write [4 41] INFO [2022-01-10 12:53:36,907] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,908] jepsen worker 12 - jepsen.util 192 :ok :read [4 41] INFO [2022-01-10 12:53:36,909] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,909] jepsen worker 12 - jepsen.util 192 :ok :read [4 41] INFO [2022-01-10 12:53:36,909] jepsen worker 44 - jepsen.util 44 :ok :write [14 43] INFO [2022-01-10 12:53:36,910] jepsen worker 44 - jepsen.util 44 :invoke :write [14 40] INFO [2022-01-10 12:53:36,911] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,912] jepsen worker 12 - jepsen.util 192 :ok :read [4 41] INFO [2022-01-10 12:53:36,912] jepsen worker 12 - jepsen.util 192 :invoke :write [4 35] INFO [2022-01-10 12:53:36,913] jepsen worker 51 - jepsen.util 141 :ok :write [17 19] INFO [2022-01-10 12:53:36,913] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:36,914] jepsen worker 51 - jepsen.util 141 :ok :read [17 19] INFO [2022-01-10 12:53:36,915] jepsen worker 51 - jepsen.util 141 :invoke :write [17 27] INFO [2022-01-10 12:53:36,916] jepsen worker 62 - jepsen.util 242 :ok :write [20 21] INFO [2022-01-10 12:53:36,916] jepsen worker 62 - jepsen.util 242 :invoke :write [20 34] INFO [2022-01-10 12:53:36,919] jepsen worker 78 - jepsen.util 168 :ok :write [26 0] INFO [2022-01-10 12:53:36,920] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,921] jepsen worker 78 - jepsen.util 168 :ok :read [26 0] INFO [2022-01-10 12:53:36,921] jepsen worker 78 - jepsen.util 168 :invoke :write [26 2] INFO [2022-01-10 12:53:36,922] jepsen worker 75 - jepsen.util 165 :ok :write [25 39] INFO [2022-01-10 12:53:36,923] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:36,923] jepsen worker 75 - jepsen.util 165 :ok :read [25 39] INFO [2022-01-10 12:53:36,924] jepsen worker 75 - jepsen.util 165 :invoke :write [25 16] INFO [2022-01-10 12:53:36,924] jepsen worker 69 - jepsen.util 159 :ok :write [23 27] INFO [2022-01-10 12:53:36,925] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,925] jepsen worker 69 - jepsen.util 159 :ok :read [23 27] INFO [2022-01-10 12:53:36,926] jepsen worker 69 - jepsen.util 159 :invoke :write [23 28] INFO [2022-01-10 12:53:36,927] jepsen worker 21 - jepsen.util 201 :ok :write [7 22] INFO [2022-01-10 12:53:36,927] jepsen worker 12 - jepsen.util 192 :ok :write [4 35] INFO [2022-01-10 12:53:36,928] jepsen worker 21 - jepsen.util 201 :invoke :write [7 46] INFO [2022-01-10 12:53:36,928] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,929] jepsen worker 12 - jepsen.util 192 :ok :read [4 35] INFO [2022-01-10 12:53:36,929] jepsen worker 12 - jepsen.util 192 :invoke :write [4 24] INFO [2022-01-10 12:53:36,931] jepsen worker 51 - jepsen.util 141 :ok :write [17 27] INFO [2022-01-10 12:53:36,932] jepsen worker 51 - jepsen.util 141 :invoke :write [17 20] INFO [2022-01-10 12:53:36,933] jepsen worker 44 - jepsen.util 44 :ok :write [14 40] INFO [2022-01-10 12:53:36,933] jepsen worker 44 - jepsen.util 44 :invoke :write [14 24] INFO [2022-01-10 12:53:36,937] jepsen worker 40 - jepsen.util 130 :ok :write [13 3] INFO [2022-01-10 12:53:36,937] jepsen worker 40 - jepsen.util 130 :invoke :write [13 4] INFO [2022-01-10 12:53:36,939] jepsen worker 62 - jepsen.util 242 :ok :write [20 34] INFO [2022-01-10 12:53:36,939] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,940] jepsen worker 62 - jepsen.util 242 :ok :read [20 34] INFO [2022-01-10 12:53:36,941] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:36,942] jepsen worker 62 - jepsen.util 242 :ok :read [20 34] INFO [2022-01-10 12:53:36,942] jepsen worker 62 - jepsen.util 242 :invoke :write [20 20] INFO [2022-01-10 12:53:36,943] jepsen worker 78 - jepsen.util 168 :ok :write [26 2] INFO [2022-01-10 12:53:36,944] jepsen worker 7 - jepsen.util 7 :ok :write [2 34] INFO [2022-01-10 12:53:36,944] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:36,944] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:36,945] jepsen worker 78 - jepsen.util 168 :ok :read [26 2] INFO [2022-01-10 12:53:36,945] jepsen worker 7 - jepsen.util 7 :ok :read [2 34] INFO [2022-01-10 12:53:36,945] jepsen worker 78 - jepsen.util 168 :invoke :write [26 26] INFO [2022-01-10 12:53:36,946] jepsen worker 7 - jepsen.util 7 :invoke :write [2 48] INFO [2022-01-10 12:53:36,946] jepsen worker 75 - jepsen.util 165 :ok :write [25 16] INFO [2022-01-10 12:53:36,946] jepsen worker 84 - jepsen.util 84 :ok :write [28 22] INFO [2022-01-10 12:53:36,947] jepsen worker 75 - jepsen.util 165 :invoke :write [25 3] INFO [2022-01-10 12:53:36,947] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:36,948] jepsen worker 84 - jepsen.util 84 :ok :read [28 22] INFO [2022-01-10 12:53:36,949] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:36,949] jepsen worker 15 - jepsen.util 15 :ok :write [5 19] INFO [2022-01-10 12:53:36,950] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:36,950] jepsen worker 84 - jepsen.util 84 :ok :read [28 22] INFO [2022-01-10 12:53:36,951] jepsen worker 84 - jepsen.util 84 :invoke :write [28 5] INFO [2022-01-10 12:53:36,951] jepsen worker 70 - jepsen.util 160 :ok :write [23 27] INFO [2022-01-10 12:53:36,951] jepsen worker 15 - jepsen.util 15 :ok :read [5 19] INFO [2022-01-10 12:53:36,951] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,951] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:36,952] jepsen worker 21 - jepsen.util 201 :ok :write [7 46] INFO [2022-01-10 12:53:36,952] jepsen worker 70 - jepsen.util 160 :ok :read [23 27] INFO [2022-01-10 12:53:36,952] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,952] jepsen worker 15 - jepsen.util 15 :ok :read [5 19] INFO [2022-01-10 12:53:36,953] jepsen worker 70 - jepsen.util 160 :invoke :write [23 9] INFO [2022-01-10 12:53:36,953] jepsen worker 21 - jepsen.util 201 :ok :read [7 46] INFO [2022-01-10 12:53:36,953] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:36,954] jepsen worker 21 - jepsen.util 201 :invoke :write [7 47] INFO [2022-01-10 12:53:36,954] jepsen worker 15 - jepsen.util 15 :ok :read [5 19] INFO [2022-01-10 12:53:36,955] jepsen worker 15 - jepsen.util 15 :invoke :write [5 1] INFO [2022-01-10 12:53:36,957] jepsen worker 4 - jepsen.util 4 :info :write [1 46] :RequestTimeoutException INFO [2022-01-10 12:53:36,957] jepsen worker 12 - jepsen.util 192 :ok :write [4 24] INFO [2022-01-10 12:53:36,957] jepsen worker 38 - jepsen.util 38 :ok :write [12 9] INFO [2022-01-10 12:53:36,958] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:36,958] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:36,959] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:36,959] jepsen worker 51 - jepsen.util 141 :ok :write [17 20] INFO [2022-01-10 12:53:36,960] jepsen worker 51 - jepsen.util 141 :invoke :write [17 23] INFO [2022-01-10 12:53:36,960] jepsen worker 38 - jepsen.util 38 :ok :read [12 9] INFO [2022-01-10 12:53:36,960] jepsen worker 12 - jepsen.util 192 :ok :read [4 24] INFO [2022-01-10 12:53:36,961] jepsen worker 38 - jepsen.util 38 :invoke :write [12 40] INFO [2022-01-10 12:53:36,962] jepsen worker 12 - jepsen.util 192 :invoke :write [4 31] INFO [2022-01-10 12:53:36,962] jepsen worker 4 - jepsen.util 94 :ok :read [1 38] INFO [2022-01-10 12:53:36,963] jepsen worker 4 - jepsen.util 94 :invoke :write [1 42] INFO [2022-01-10 12:53:36,964] jepsen worker 44 - jepsen.util 44 :ok :write [14 24] INFO [2022-01-10 12:53:36,964] jepsen worker 78 - jepsen.util 168 :ok :write [26 26] INFO [2022-01-10 12:53:36,964] jepsen worker 44 - jepsen.util 44 :invoke :write [14 43] INFO [2022-01-10 12:53:36,965] jepsen worker 78 - jepsen.util 168 :invoke :write [26 42] INFO [2022-01-10 12:53:36,973] jepsen worker 62 - jepsen.util 242 :ok :write [20 20] INFO [2022-01-10 12:53:36,973] jepsen worker 62 - jepsen.util 242 :invoke :write [20 49] INFO [2022-01-10 12:53:36,977] jepsen worker 75 - jepsen.util 165 :ok :write [25 3] INFO [2022-01-10 12:53:36,977] jepsen worker 75 - jepsen.util 165 :invoke :write [25 13] INFO [2022-01-10 12:53:36,981] jepsen worker 70 - jepsen.util 160 :ok :write [23 9] INFO [2022-01-10 12:53:36,982] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,983] jepsen worker 70 - jepsen.util 160 :ok :read [23 9] INFO [2022-01-10 12:53:36,984] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:36,985] jepsen worker 70 - jepsen.util 160 :ok :read [23 9] INFO [2022-01-10 12:53:36,985] jepsen worker 70 - jepsen.util 160 :invoke :write [23 12] INFO [2022-01-10 12:53:36,989] jepsen worker 51 - jepsen.util 141 :ok :write [17 23] INFO [2022-01-10 12:53:36,990] jepsen worker 51 - jepsen.util 141 :invoke :write [17 19] INFO [2022-01-10 12:53:36,991] jepsen worker 21 - jepsen.util 201 :ok :write [7 47] INFO [2022-01-10 12:53:36,992] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:36,992] jepsen worker 21 - jepsen.util 201 :ok :read [7 47] INFO [2022-01-10 12:53:36,993] jepsen worker 21 - jepsen.util 201 :invoke :write [7 7] INFO [2022-01-10 12:53:36,994] jepsen worker 12 - jepsen.util 192 :ok :write [4 31] INFO [2022-01-10 12:53:36,995] jepsen worker 12 - jepsen.util 192 :invoke :write [4 8] INFO [2022-01-10 12:53:36,998] jepsen worker 44 - jepsen.util 44 :ok :write [14 43] INFO [2022-01-10 12:53:36,999] jepsen worker 44 - jepsen.util 44 :invoke :write [14 41] INFO [2022-01-10 12:53:37,001] jepsen worker 78 - jepsen.util 168 :ok :write [26 42] INFO [2022-01-10 12:53:37,002] jepsen worker 78 - jepsen.util 168 :invoke :write [26 16] INFO [2022-01-10 12:53:37,007] jepsen worker 62 - jepsen.util 242 :ok :write [20 49] INFO [2022-01-10 12:53:37,007] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,008] jepsen worker 62 - jepsen.util 242 :ok :read [20 49] INFO [2022-01-10 12:53:37,008] jepsen worker 62 - jepsen.util 242 :invoke :write [20 49] INFO [2022-01-10 12:53:37,009] jepsen worker 70 - jepsen.util 160 :ok :write [23 12] INFO [2022-01-10 12:53:37,009] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:37,010] jepsen worker 70 - jepsen.util 160 :ok :read [23 12] INFO [2022-01-10 12:53:37,011] jepsen worker 70 - jepsen.util 160 :invoke :write [23 34] INFO [2022-01-10 12:53:37,197] jepsen worker 64 - jepsen.util 334 :fail :read [21 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:37,198] jepsen worker 64 - jepsen.util 334 :invoke :read [21 nil] INFO [2022-01-10 12:53:37,200] jepsen worker 48 - jepsen.util 228 :ok :write [16 7] INFO [2022-01-10 12:53:37,201] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,201] jepsen worker 48 - jepsen.util 228 :ok :read [16 7] INFO [2022-01-10 12:53:37,202] jepsen worker 48 - jepsen.util 228 :invoke :write [16 11] INFO [2022-01-10 12:53:37,205] jepsen worker 3 - jepsen.util 3 :ok :write [1 44] INFO [2022-01-10 12:53:37,205] jepsen worker 3 - jepsen.util 3 :invoke :write [1 27] INFO [2022-01-10 12:53:37,206] jepsen worker 83 - jepsen.util 263 :ok :write [27 33] INFO [2022-01-10 12:53:37,206] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:37,207] jepsen worker 83 - jepsen.util 263 :ok :read [27 33] INFO [2022-01-10 12:53:37,208] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:37,209] jepsen worker 83 - jepsen.util 263 :ok :read [27 33] INFO [2022-01-10 12:53:37,210] jepsen worker 83 - jepsen.util 263 :invoke :write [27 0] INFO [2022-01-10 12:53:37,210] jepsen worker 84 - jepsen.util 84 :ok :write [28 5] INFO [2022-01-10 12:53:37,211] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,212] jepsen worker 84 - jepsen.util 84 :ok :read [28 5] INFO [2022-01-10 12:53:37,212] jepsen worker 33 - jepsen.util 123 :ok :write [11 1] INFO [2022-01-10 12:53:37,212] jepsen worker 84 - jepsen.util 84 :invoke :write [28 18] INFO [2022-01-10 12:53:37,213] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:37,213] jepsen worker 33 - jepsen.util 123 :ok :read [11 1] INFO [2022-01-10 12:53:37,214] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:37,215] jepsen worker 33 - jepsen.util 123 :ok :read [11 1] INFO [2022-01-10 12:53:37,215] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:37,216] jepsen worker 33 - jepsen.util 123 :ok :read [11 1] INFO [2022-01-10 12:53:37,216] jepsen worker 33 - jepsen.util 123 :invoke :write [11 23] INFO [2022-01-10 12:53:37,218] jepsen worker 68 - jepsen.util 68 :ok :write [22 6] INFO [2022-01-10 12:53:37,218] jepsen worker 68 - jepsen.util 68 :invoke :write [22 40] INFO [2022-01-10 12:53:37,221] jepsen worker 38 - jepsen.util 38 :ok :write [12 40] INFO [2022-01-10 12:53:37,221] jepsen worker 38 - jepsen.util 38 :invoke :write [12 43] INFO [2022-01-10 12:53:37,225] jepsen worker 89 - jepsen.util 269 :ok :write [29 41] INFO [2022-01-10 12:53:37,226] jepsen worker 89 - jepsen.util 269 :invoke :write [29 0] INFO [2022-01-10 12:53:37,230] jepsen worker 1 - jepsen.util 91 :ok :write [0 49] INFO [2022-01-10 12:53:37,231] jepsen worker 1 - jepsen.util 91 :invoke :write [0 25] INFO [2022-01-10 12:53:37,263] jepsen worker 75 - jepsen.util 165 :ok :write [25 13] INFO [2022-01-10 12:53:37,263] jepsen worker 51 - jepsen.util 141 :ok :write [17 19] INFO [2022-01-10 12:53:37,264] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,265] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,265] jepsen worker 75 - jepsen.util 165 :ok :read [25 13] INFO [2022-01-10 12:53:37,266] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,267] jepsen worker 51 - jepsen.util 141 :ok :read [17 19] INFO [2022-01-10 12:53:37,267] jepsen worker 75 - jepsen.util 165 :ok :read [25 13] INFO [2022-01-10 12:53:37,267] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,268] jepsen worker 75 - jepsen.util 165 :invoke :write [25 23] INFO [2022-01-10 12:53:37,270] jepsen worker 12 - jepsen.util 192 :ok :write [4 8] INFO [2022-01-10 12:53:37,271] jepsen worker 21 - jepsen.util 201 :ok :write [7 7] INFO [2022-01-10 12:53:37,271] jepsen worker 12 - jepsen.util 192 :invoke :write [4 30] INFO [2022-01-10 12:53:37,271] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,272] jepsen worker 21 - jepsen.util 201 :ok :read [7 7] INFO [2022-01-10 12:53:37,273] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,274] jepsen worker 21 - jepsen.util 201 :ok :read [7 7] INFO [2022-01-10 12:53:37,274] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,276] jepsen worker 21 - jepsen.util 201 :ok :read [7 7] INFO [2022-01-10 12:53:37,276] jepsen worker 3 - jepsen.util 3 :ok :write [1 27] INFO [2022-01-10 12:53:37,276] jepsen worker 51 - jepsen.util 141 :ok :read [17 19] INFO [2022-01-10 12:53:37,276] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,277] jepsen worker 3 - jepsen.util 3 :invoke :write [1 10] INFO [2022-01-10 12:53:37,277] jepsen worker 21 - jepsen.util 201 :ok :read [7 7] INFO [2022-01-10 12:53:37,277] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,278] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,278] jepsen worker 78 - jepsen.util 168 :ok :write [26 16] INFO [2022-01-10 12:53:37,279] jepsen worker 21 - jepsen.util 201 :ok :read [7 7] INFO [2022-01-10 12:53:37,279] jepsen worker 78 - jepsen.util 168 :invoke :write [26 32] INFO [2022-01-10 12:53:37,279] jepsen worker 44 - jepsen.util 44 :ok :write [14 41] INFO [2022-01-10 12:53:37,279] jepsen worker 51 - jepsen.util 141 :ok :read [17 19] INFO [2022-01-10 12:53:37,280] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,280] jepsen worker 84 - jepsen.util 84 :ok :write [28 18] INFO [2022-01-10 12:53:37,280] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,281] jepsen worker 21 - jepsen.util 201 :ok :read [7 7] INFO [2022-01-10 12:53:37,281] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,282] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:37,282] jepsen worker 84 - jepsen.util 84 :ok :read [28 18] INFO [2022-01-10 12:53:37,282] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,283] jepsen worker 44 - jepsen.util 44 :ok :read [14 41] INFO [2022-01-10 12:53:37,283] jepsen worker 84 - jepsen.util 84 :invoke :write [28 35] INFO [2022-01-10 12:53:37,283] jepsen worker 21 - jepsen.util 201 :ok :read [7 7] INFO [2022-01-10 12:53:37,283] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:37,284] jepsen worker 44 - jepsen.util 44 :ok :read [14 41] INFO [2022-01-10 12:53:37,284] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,285] jepsen worker 44 - jepsen.util 44 :invoke :write [14 41] INFO [2022-01-10 12:53:37,285] jepsen worker 21 - jepsen.util 201 :ok :read [7 7] INFO [2022-01-10 12:53:37,286] jepsen worker 51 - jepsen.util 141 :ok :read [17 19] INFO [2022-01-10 12:53:37,286] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,286] jepsen worker 69 - jepsen.util 159 :ok :write [23 28] INFO [2022-01-10 12:53:37,287] jepsen worker 51 - jepsen.util 141 :invoke :write [17 34] INFO [2022-01-10 12:53:37,287] jepsen worker 21 - jepsen.util 201 :ok :read [7 7] INFO [2022-01-10 12:53:37,287] jepsen worker 62 - jepsen.util 242 :ok :write [20 49] INFO [2022-01-10 12:53:37,288] jepsen worker 69 - jepsen.util 159 :invoke :write [23 10] INFO [2022-01-10 12:53:37,288] jepsen worker 21 - jepsen.util 201 :invoke :write [7 32] INFO [2022-01-10 12:53:37,289] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,290] jepsen worker 62 - jepsen.util 242 :ok :read [20 49] INFO [2022-01-10 12:53:37,291] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,292] jepsen worker 62 - jepsen.util 242 :ok :read [20 49] INFO [2022-01-10 12:53:37,292] jepsen worker 62 - jepsen.util 242 :invoke :write [20 24] INFO [2022-01-10 12:53:37,301] jepsen worker 12 - jepsen.util 192 :ok :write [4 30] INFO [2022-01-10 12:53:37,301] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,302] jepsen worker 12 - jepsen.util 192 :ok :read [4 30] INFO [2022-01-10 12:53:37,302] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,303] jepsen worker 12 - jepsen.util 192 :ok :read [4 30] INFO [2022-01-10 12:53:37,304] jepsen worker 12 - jepsen.util 192 :invoke :write [4 9] INFO [2022-01-10 12:53:37,306] jepsen worker 78 - jepsen.util 168 :ok :write [26 32] INFO [2022-01-10 12:53:37,307] jepsen worker 78 - jepsen.util 168 :invoke :write [26 40] INFO [2022-01-10 12:53:37,313] jepsen worker 75 - jepsen.util 165 :ok :write [25 23] INFO [2022-01-10 12:53:37,314] jepsen worker 75 - jepsen.util 165 :invoke :write [25 30] INFO [2022-01-10 12:53:37,319] jepsen worker 69 - jepsen.util 159 :ok :write [23 10] INFO [2022-01-10 12:53:37,319] jepsen worker 69 - jepsen.util 159 :invoke :write [23 7] INFO [2022-01-10 12:53:37,321] jepsen worker 21 - jepsen.util 201 :ok :write [7 32] INFO [2022-01-10 12:53:37,322] jepsen worker 21 - jepsen.util 201 :invoke :write [7 6] INFO [2022-01-10 12:53:37,325] jepsen worker 51 - jepsen.util 141 :ok :write [17 34] INFO [2022-01-10 12:53:37,326] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,327] jepsen worker 40 - jepsen.util 130 :ok :write [13 4] INFO [2022-01-10 12:53:37,328] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:37,328] jepsen worker 40 - jepsen.util 130 :ok :read [13 4] INFO [2022-01-10 12:53:37,329] jepsen worker 40 - jepsen.util 130 :invoke :write [13 36] INFO [2022-01-10 12:53:37,330] jepsen worker 44 - jepsen.util 44 :ok :write [14 41] INFO [2022-01-10 12:53:37,330] jepsen worker 38 - jepsen.util 38 :ok :write [12 43] INFO [2022-01-10 12:53:37,330] jepsen worker 12 - jepsen.util 192 :ok :write [4 9] INFO [2022-01-10 12:53:37,330] jepsen worker 38 - jepsen.util 38 :invoke :write [12 33] INFO [2022-01-10 12:53:37,331] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,331] jepsen worker 44 - jepsen.util 44 :invoke :write [14 38] INFO [2022-01-10 12:53:37,333] jepsen worker 12 - jepsen.util 192 :ok :read [4 9] INFO [2022-01-10 12:53:37,333] jepsen worker 12 - jepsen.util 192 :invoke :write [4 35] INFO [2022-01-10 12:53:37,335] jepsen worker 7 - jepsen.util 7 :ok :write [2 48] INFO [2022-01-10 12:53:37,335] jepsen worker 7 - jepsen.util 7 :invoke :write [2 0] INFO [2022-01-10 12:53:37,336] jepsen worker 51 - jepsen.util 141 :ok :read [17 34] INFO [2022-01-10 12:53:37,336] jepsen worker 4 - jepsen.util 94 :ok :write [1 42] INFO [2022-01-10 12:53:37,336] jepsen worker 78 - jepsen.util 168 :ok :write [26 40] INFO [2022-01-10 12:53:37,337] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,337] jepsen worker 62 - jepsen.util 242 :ok :write [20 24] INFO [2022-01-10 12:53:37,338] jepsen worker 51 - jepsen.util 141 :ok :read [17 34] INFO [2022-01-10 12:53:37,338] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,339] jepsen worker 62 - jepsen.util 242 :invoke :write [20 18] INFO [2022-01-10 12:53:37,339] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,340] jepsen worker 4 - jepsen.util 94 :invoke :write [1 29] INFO [2022-01-10 12:53:37,340] jepsen worker 51 - jepsen.util 141 :ok :read [17 34] INFO [2022-01-10 12:53:37,340] jepsen worker 78 - jepsen.util 168 :ok :read [26 40] INFO [2022-01-10 12:53:37,341] jepsen worker 70 - jepsen.util 160 :ok :write [23 34] INFO [2022-01-10 12:53:37,341] jepsen worker 51 - jepsen.util 141 :invoke :write [17 24] INFO [2022-01-10 12:53:37,342] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,342] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:37,343] jepsen worker 75 - jepsen.util 165 :ok :write [25 30] INFO [2022-01-10 12:53:37,344] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,345] jepsen worker 75 - jepsen.util 165 :ok :read [25 30] INFO [2022-01-10 12:53:37,345] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,346] jepsen worker 78 - jepsen.util 168 :ok :read [26 40] INFO [2022-01-10 12:53:37,346] jepsen worker 70 - jepsen.util 160 :ok :read [23 34] INFO [2022-01-10 12:53:37,346] jepsen worker 75 - jepsen.util 165 :ok :read [25 30] INFO [2022-01-10 12:53:37,346] jepsen worker 70 - jepsen.util 160 :invoke :write [23 24] INFO [2022-01-10 12:53:37,347] jepsen worker 75 - jepsen.util 165 :invoke :write [25 9] INFO [2022-01-10 12:53:37,348] jepsen worker 78 - jepsen.util 168 :invoke :write [26 36] INFO [2022-01-10 12:53:37,350] jepsen worker 21 - jepsen.util 201 :ok :write [7 6] INFO [2022-01-10 12:53:37,351] jepsen worker 21 - jepsen.util 201 :invoke :write [7 36] INFO [2022-01-10 12:53:37,353] jepsen worker 12 - jepsen.util 192 :ok :write [4 35] INFO [2022-01-10 12:53:37,354] jepsen worker 12 - jepsen.util 192 :invoke :write [4 24] INFO [2022-01-10 12:53:37,358] jepsen worker 44 - jepsen.util 44 :ok :write [14 38] INFO [2022-01-10 12:53:37,358] jepsen worker 44 - jepsen.util 44 :invoke :write [14 20] INFO [2022-01-10 12:53:37,366] jepsen worker 78 - jepsen.util 168 :ok :write [26 36] INFO [2022-01-10 12:53:37,366] jepsen worker 78 - jepsen.util 168 :invoke :write [26 7] INFO [2022-01-10 12:53:37,367] jepsen worker 62 - jepsen.util 242 :ok :write [20 18] INFO [2022-01-10 12:53:37,367] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,368] jepsen worker 62 - jepsen.util 242 :ok :read [20 18] INFO [2022-01-10 12:53:37,368] jepsen worker 62 - jepsen.util 242 :invoke :write [20 6] INFO [2022-01-10 12:53:37,370] jepsen worker 71 - jepsen.util 161 :ok :write [23 47] INFO [2022-01-10 12:53:37,371] jepsen worker 71 - jepsen.util 161 :invoke :write [23 45] INFO [2022-01-10 12:53:37,373] jepsen worker 75 - jepsen.util 165 :ok :write [25 9] INFO [2022-01-10 12:53:37,373] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,374] jepsen worker 75 - jepsen.util 165 :ok :read [25 9] INFO [2022-01-10 12:53:37,375] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,376] jepsen worker 75 - jepsen.util 165 :ok :read [25 9] INFO [2022-01-10 12:53:37,376] jepsen worker 75 - jepsen.util 165 :invoke :write [25 4] INFO [2022-01-10 12:53:37,377] jepsen worker 51 - jepsen.util 141 :ok :write [17 24] INFO [2022-01-10 12:53:37,378] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,378] jepsen worker 21 - jepsen.util 201 :ok :write [7 36] INFO [2022-01-10 12:53:37,378] jepsen worker 51 - jepsen.util 141 :ok :read [17 24] INFO [2022-01-10 12:53:37,379] jepsen worker 21 - jepsen.util 201 :invoke :write [7 16] INFO [2022-01-10 12:53:37,379] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,383] jepsen worker 12 - jepsen.util 192 :ok :write [4 24] INFO [2022-01-10 12:53:37,383] jepsen worker 44 - jepsen.util 44 :ok :write [14 20] INFO [2022-01-10 12:53:37,383] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,383] jepsen worker 51 - jepsen.util 141 :ok :read [17 24] INFO [2022-01-10 12:53:37,384] jepsen worker 44 - jepsen.util 44 :invoke :write [14 23] INFO [2022-01-10 12:53:37,384] jepsen worker 12 - jepsen.util 192 :ok :read [4 24] INFO [2022-01-10 12:53:37,384] jepsen worker 51 - jepsen.util 141 :invoke :write [17 38] INFO [2022-01-10 12:53:37,384] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,386] jepsen worker 12 - jepsen.util 192 :ok :read [4 24] INFO [2022-01-10 12:53:37,386] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,388] jepsen worker 12 - jepsen.util 192 :ok :read [4 24] INFO [2022-01-10 12:53:37,388] jepsen worker 12 - jepsen.util 192 :invoke :write [4 40] INFO [2022-01-10 12:53:37,389] jepsen worker 62 - jepsen.util 242 :ok :write [20 6] INFO [2022-01-10 12:53:37,389] jepsen worker 62 - jepsen.util 242 :invoke :write [20 32] INFO [2022-01-10 12:53:37,391] jepsen worker 78 - jepsen.util 168 :ok :write [26 7] INFO [2022-01-10 12:53:37,391] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,394] jepsen worker 75 - jepsen.util 165 :ok :write [25 4] INFO [2022-01-10 12:53:37,394] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,395] jepsen worker 75 - jepsen.util 165 :ok :read [25 4] INFO [2022-01-10 12:53:37,396] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,396] jepsen worker 71 - jepsen.util 161 :ok :write [23 45] INFO [2022-01-10 12:53:37,396] jepsen worker 78 - jepsen.util 168 :ok :read [26 7] INFO [2022-01-10 12:53:37,396] jepsen worker 75 - jepsen.util 165 :ok :read [25 4] INFO [2022-01-10 12:53:37,397] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,397] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,398] jepsen worker 75 - jepsen.util 165 :ok :read [25 4] INFO [2022-01-10 12:53:37,398] jepsen worker 71 - jepsen.util 161 :invoke :write [23 40] INFO [2022-01-10 12:53:37,398] jepsen worker 78 - jepsen.util 168 :ok :read [26 7] INFO [2022-01-10 12:53:37,398] jepsen worker 75 - jepsen.util 165 :invoke :write [25 37] INFO [2022-01-10 12:53:37,398] jepsen worker 21 - jepsen.util 201 :ok :write [7 16] INFO [2022-01-10 12:53:37,399] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,399] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,400] jepsen worker 78 - jepsen.util 168 :ok :read [26 7] INFO [2022-01-10 12:53:37,400] jepsen worker 21 - jepsen.util 201 :ok :read [7 16] INFO [2022-01-10 12:53:37,400] jepsen worker 78 - jepsen.util 168 :invoke :write [26 35] INFO [2022-01-10 12:53:37,401] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,401] jepsen worker 21 - jepsen.util 201 :ok :read [7 16] INFO [2022-01-10 12:53:37,402] jepsen worker 21 - jepsen.util 201 :invoke :write [7 0] INFO [2022-01-10 12:53:37,404] jepsen worker 12 - jepsen.util 192 :ok :write [4 40] INFO [2022-01-10 12:53:37,404] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,405] jepsen worker 44 - jepsen.util 44 :ok :write [14 23] INFO [2022-01-10 12:53:37,406] jepsen worker 12 - jepsen.util 192 :ok :read [4 40] INFO [2022-01-10 12:53:37,406] jepsen worker 44 - jepsen.util 44 :invoke :write [14 21] INFO [2022-01-10 12:53:37,406] jepsen worker 12 - jepsen.util 192 :invoke :write [4 47] INFO [2022-01-10 12:53:37,407] jepsen worker 15 - jepsen.util 15 :ok :write [5 1] INFO [2022-01-10 12:53:37,407] jepsen worker 85 - jepsen.util 265 :ok :write [28 13] INFO [2022-01-10 12:53:37,408] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,408] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:37,409] jepsen worker 85 - jepsen.util 265 :ok :read [28 13] INFO [2022-01-10 12:53:37,409] jepsen worker 15 - jepsen.util 15 :ok :read [5 1] INFO [2022-01-10 12:53:37,409] jepsen worker 85 - jepsen.util 265 :invoke :write [28 33] INFO [2022-01-10 12:53:37,409] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:37,411] jepsen worker 15 - jepsen.util 15 :ok :read [5 1] INFO [2022-01-10 12:53:37,411] jepsen worker 15 - jepsen.util 15 :invoke :write [5 13] INFO [2022-01-10 12:53:37,412] jepsen worker 48 - jepsen.util 228 :ok :write [16 11] INFO [2022-01-10 12:53:37,412] jepsen worker 51 - jepsen.util 141 :ok :write [17 38] INFO [2022-01-10 12:53:37,412] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,413] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,413] jepsen worker 48 - jepsen.util 228 :ok :read [16 11] INFO [2022-01-10 12:53:37,414] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,414] jepsen worker 62 - jepsen.util 242 :ok :write [20 32] INFO [2022-01-10 12:53:37,414] jepsen worker 62 - jepsen.util 242 :invoke :write [20 46] INFO [2022-01-10 12:53:37,415] jepsen worker 48 - jepsen.util 228 :ok :read [16 11] INFO [2022-01-10 12:53:37,415] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,416] jepsen worker 51 - jepsen.util 141 :ok :read [17 38] INFO [2022-01-10 12:53:37,416] jepsen worker 70 - jepsen.util 160 :ok :write [23 24] INFO [2022-01-10 12:53:37,416] jepsen worker 48 - jepsen.util 228 :ok :read [16 11] INFO [2022-01-10 12:53:37,416] jepsen worker 51 - jepsen.util 141 :invoke :write [17 24] INFO [2022-01-10 12:53:37,416] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:37,417] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,417] jepsen worker 83 - jepsen.util 263 :ok :write [27 0] INFO [2022-01-10 12:53:37,418] jepsen worker 48 - jepsen.util 228 :ok :read [16 11] INFO [2022-01-10 12:53:37,418] jepsen worker 38 - jepsen.util 38 :ok :write [12 33] INFO [2022-01-10 12:53:37,418] jepsen worker 83 - jepsen.util 263 :invoke :write [27 11] INFO [2022-01-10 12:53:37,419] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,419] jepsen worker 70 - jepsen.util 160 :ok :read [23 24] INFO [2022-01-10 12:53:37,420] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,420] jepsen worker 38 - jepsen.util 38 :ok :read [12 33] INFO [2022-01-10 12:53:37,420] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:37,421] jepsen worker 48 - jepsen.util 228 :ok :read [16 11] INFO [2022-01-10 12:53:37,421] jepsen worker 38 - jepsen.util 38 :invoke :write [12 49] INFO [2022-01-10 12:53:37,421] jepsen worker 70 - jepsen.util 160 :ok :read [23 24] INFO [2022-01-10 12:53:37,422] jepsen worker 48 - jepsen.util 228 :invoke :write [16 49] INFO [2022-01-10 12:53:37,422] jepsen worker 70 - jepsen.util 160 :invoke :write [23 16] INFO [2022-01-10 12:53:37,425] jepsen worker 75 - jepsen.util 165 :ok :write [25 37] INFO [2022-01-10 12:53:37,426] jepsen worker 75 - jepsen.util 165 :invoke :write [25 46] INFO [2022-01-10 12:53:37,427] jepsen worker 33 - jepsen.util 123 :ok :write [11 23] INFO [2022-01-10 12:53:37,428] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:37,429] jepsen worker 33 - jepsen.util 123 :ok :read [11 23] INFO [2022-01-10 12:53:37,429] jepsen worker 33 - jepsen.util 123 :invoke :write [11 30] INFO [2022-01-10 12:53:37,431] jepsen worker 12 - jepsen.util 192 :ok :write [4 47] INFO [2022-01-10 12:53:37,431] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,432] jepsen worker 12 - jepsen.util 192 :ok :read [4 47] INFO [2022-01-10 12:53:37,432] jepsen worker 85 - jepsen.util 265 :ok :write [28 33] INFO [2022-01-10 12:53:37,433] jepsen worker 4 - jepsen.util 94 :ok :write [1 29] INFO [2022-01-10 12:53:37,433] jepsen worker 78 - jepsen.util 168 :ok :write [26 35] INFO [2022-01-10 12:53:37,433] jepsen worker 12 - jepsen.util 192 :invoke :write [4 14] INFO [2022-01-10 12:53:37,434] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,434] jepsen worker 78 - jepsen.util 168 :invoke :write [26 32] INFO [2022-01-10 12:53:37,434] jepsen worker 85 - jepsen.util 265 :ok :read [28 33] INFO [2022-01-10 12:53:37,435] jepsen worker 21 - jepsen.util 201 :ok :write [7 0] INFO [2022-01-10 12:53:37,435] jepsen worker 4 - jepsen.util 94 :invoke :write [1 37] INFO [2022-01-10 12:53:37,435] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,436] jepsen worker 68 - jepsen.util 68 :ok :write [22 40] INFO [2022-01-10 12:53:37,436] jepsen worker 21 - jepsen.util 201 :invoke :write [7 46] INFO [2022-01-10 12:53:37,436] jepsen worker 85 - jepsen.util 265 :ok :read [28 33] INFO [2022-01-10 12:53:37,436] jepsen worker 68 - jepsen.util 68 :invoke :write [22 40] INFO [2022-01-10 12:53:37,437] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,438] jepsen worker 51 - jepsen.util 141 :ok :write [17 24] INFO [2022-01-10 12:53:37,438] jepsen worker 85 - jepsen.util 265 :ok :read [28 33] INFO [2022-01-10 12:53:37,439] jepsen worker 51 - jepsen.util 141 :invoke :write [17 37] INFO [2022-01-10 12:53:37,439] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,440] jepsen worker 85 - jepsen.util 265 :ok :read [28 33] INFO [2022-01-10 12:53:37,441] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,442] jepsen worker 85 - jepsen.util 265 :ok :read [28 33] INFO [2022-01-10 12:53:37,442] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,443] jepsen worker 44 - jepsen.util 44 :ok :write [14 21] INFO [2022-01-10 12:53:37,443] jepsen worker 85 - jepsen.util 265 :ok :read [28 33] INFO [2022-01-10 12:53:37,443] jepsen worker 38 - jepsen.util 38 :ok :write [12 49] INFO [2022-01-10 12:53:37,443] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:37,444] jepsen worker 85 - jepsen.util 265 :invoke :write [28 34] INFO [2022-01-10 12:53:37,444] jepsen worker 44 - jepsen.util 44 :ok :read [14 21] INFO [2022-01-10 12:53:37,444] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,445] jepsen worker 71 - jepsen.util 161 :ok :write [23 40] INFO [2022-01-10 12:53:37,445] jepsen worker 44 - jepsen.util 44 :invoke :write [14 6] INFO [2022-01-10 12:53:37,445] jepsen worker 38 - jepsen.util 38 :ok :read [12 49] INFO [2022-01-10 12:53:37,446] jepsen worker 71 - jepsen.util 161 :invoke :write [23 17] INFO [2022-01-10 12:53:37,446] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,447] jepsen worker 38 - jepsen.util 38 :ok :read [12 49] INFO [2022-01-10 12:53:37,448] jepsen worker 38 - jepsen.util 38 :invoke :write [12 43] INFO [2022-01-10 12:53:37,457] jepsen worker 78 - jepsen.util 168 :ok :write [26 32] INFO [2022-01-10 12:53:37,457] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,458] jepsen worker 78 - jepsen.util 168 :ok :read [26 32] INFO [2022-01-10 12:53:37,459] jepsen worker 78 - jepsen.util 168 :invoke :write [26 35] INFO [2022-01-10 12:53:37,459] jepsen worker 51 - jepsen.util 141 :ok :write [17 37] INFO [2022-01-10 12:53:37,460] jepsen worker 62 - jepsen.util 242 :ok :write [20 46] INFO [2022-01-10 12:53:37,460] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,461] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,461] jepsen worker 51 - jepsen.util 141 :ok :read [17 37] INFO [2022-01-10 12:53:37,462] jepsen worker 62 - jepsen.util 242 :ok :read [20 46] INFO [2022-01-10 12:53:37,462] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,463] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,463] jepsen worker 51 - jepsen.util 141 :ok :read [17 37] INFO [2022-01-10 12:53:37,464] jepsen worker 62 - jepsen.util 242 :ok :read [20 46] INFO [2022-01-10 12:53:37,464] jepsen worker 51 - jepsen.util 141 :invoke :write [17 2] INFO [2022-01-10 12:53:37,465] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,466] jepsen worker 62 - jepsen.util 242 :ok :read [20 46] INFO [2022-01-10 12:53:37,467] jepsen worker 62 - jepsen.util 242 :invoke :write [20 2] INFO [2022-01-10 12:53:37,468] jepsen worker 89 - jepsen.util 269 :ok :write [29 0] INFO [2022-01-10 12:53:37,469] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:37,470] jepsen worker 89 - jepsen.util 269 :ok :read [29 0] INFO [2022-01-10 12:53:37,470] jepsen worker 89 - jepsen.util 269 :invoke :write [29 22] INFO [2022-01-10 12:53:37,471] jepsen worker 4 - jepsen.util 94 :ok :write [1 37] INFO [2022-01-10 12:53:37,471] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:37,472] jepsen worker 4 - jepsen.util 94 :ok :read [1 37] INFO [2022-01-10 12:53:37,473] jepsen worker 4 - jepsen.util 94 :invoke :write [1 37] INFO [2022-01-10 12:53:37,474] jepsen worker 75 - jepsen.util 165 :ok :write [25 46] INFO [2022-01-10 12:53:37,474] jepsen worker 75 - jepsen.util 165 :invoke :write [25 15] INFO [2022-01-10 12:53:37,479] jepsen worker 1 - jepsen.util 91 :ok :write [0 25] INFO [2022-01-10 12:53:37,479] jepsen worker 1 - jepsen.util 91 :invoke :write [0 36] INFO [2022-01-10 12:53:37,481] jepsen worker 21 - jepsen.util 201 :ok :write [7 46] INFO [2022-01-10 12:53:37,482] jepsen worker 21 - jepsen.util 201 :invoke :write [7 40] INFO [2022-01-10 12:53:37,506] jepsen worker 40 - jepsen.util 130 :ok :write [13 36] INFO [2022-01-10 12:53:37,507] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:37,508] jepsen worker 40 - jepsen.util 130 :ok :read [13 36] INFO [2022-01-10 12:53:37,509] jepsen worker 40 - jepsen.util 130 :invoke :write [13 33] INFO [2022-01-10 12:53:37,513] jepsen worker 7 - jepsen.util 7 :ok :write [2 0] INFO [2022-01-10 12:53:37,514] jepsen worker 7 - jepsen.util 7 :invoke :write [2 39] INFO [2022-01-10 12:53:37,518] jepsen worker 15 - jepsen.util 15 :ok :write [5 13] INFO [2022-01-10 12:53:37,519] jepsen worker 15 - jepsen.util 15 :invoke :write [5 16] INFO [2022-01-10 12:53:37,523] jepsen worker 48 - jepsen.util 228 :ok :write [16 49] INFO [2022-01-10 12:53:37,523] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,524] jepsen worker 48 - jepsen.util 228 :ok :read [16 49] INFO [2022-01-10 12:53:37,525] jepsen worker 48 - jepsen.util 228 :invoke :write [16 23] INFO [2022-01-10 12:53:37,528] jepsen worker 83 - jepsen.util 263 :ok :write [27 11] INFO [2022-01-10 12:53:37,528] jepsen worker 83 - jepsen.util 263 :invoke :write [27 16] INFO [2022-01-10 12:53:37,533] jepsen worker 33 - jepsen.util 123 :ok :write [11 30] INFO [2022-01-10 12:53:37,533] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:37,534] jepsen worker 33 - jepsen.util 123 :ok :read [11 30] INFO [2022-01-10 12:53:37,534] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:37,535] jepsen worker 33 - jepsen.util 123 :ok :read [11 30] INFO [2022-01-10 12:53:37,536] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:37,536] jepsen worker 33 - jepsen.util 123 :ok :read [11 30] INFO [2022-01-10 12:53:37,537] jepsen worker 33 - jepsen.util 123 :invoke :write [11 7] INFO [2022-01-10 12:53:37,579] jepsen worker 12 - jepsen.util 192 :ok :write [4 14] INFO [2022-01-10 12:53:37,579] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,580] jepsen worker 12 - jepsen.util 192 :ok :read [4 14] INFO [2022-01-10 12:53:37,581] jepsen worker 12 - jepsen.util 192 :invoke :write [4 26] INFO [2022-01-10 12:53:37,584] jepsen worker 71 - jepsen.util 161 :ok :write [23 17] INFO [2022-01-10 12:53:37,584] jepsen worker 89 - jepsen.util 269 :ok :write [29 22] INFO [2022-01-10 12:53:37,585] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:37,586] jepsen worker 89 - jepsen.util 269 :invoke :write [29 35] INFO [2022-01-10 12:53:37,586] jepsen worker 71 - jepsen.util 161 :ok :read [23 17] INFO [2022-01-10 12:53:37,587] jepsen worker 68 - jepsen.util 68 :ok :write [22 40] INFO [2022-01-10 12:53:37,587] jepsen worker 71 - jepsen.util 161 :invoke :write [23 20] INFO [2022-01-10 12:53:37,588] jepsen worker 68 - jepsen.util 68 :invoke :write [22 10] INFO [2022-01-10 12:53:37,592] jepsen worker 44 - jepsen.util 44 :ok :write [14 6] INFO [2022-01-10 12:53:37,593] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:37,594] jepsen worker 44 - jepsen.util 44 :ok :read [14 6] INFO [2022-01-10 12:53:37,594] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:37,594] jepsen worker 85 - jepsen.util 265 :ok :write [28 34] INFO [2022-01-10 12:53:37,595] jepsen worker 85 - jepsen.util 265 :invoke :write [28 30] INFO [2022-01-10 12:53:37,595] jepsen worker 44 - jepsen.util 44 :ok :read [14 6] INFO [2022-01-10 12:53:37,596] jepsen worker 44 - jepsen.util 44 :invoke :write [14 32] INFO [2022-01-10 12:53:37,599] jepsen worker 62 - jepsen.util 242 :ok :write [20 2] INFO [2022-01-10 12:53:37,600] jepsen worker 62 - jepsen.util 242 :invoke :write [20 40] INFO [2022-01-10 12:53:37,602] jepsen worker 38 - jepsen.util 38 :ok :write [12 43] INFO [2022-01-10 12:53:37,602] jepsen worker 38 - jepsen.util 38 :invoke :write [12 30] INFO [2022-01-10 12:53:37,604] jepsen worker 1 - jepsen.util 91 :ok :write [0 36] INFO [2022-01-10 12:53:37,605] jepsen worker 1 - jepsen.util 91 :invoke :write [0 48] INFO [2022-01-10 12:53:37,606] jepsen worker 75 - jepsen.util 165 :ok :write [25 15] INFO [2022-01-10 12:53:37,606] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,607] jepsen worker 75 - jepsen.util 165 :ok :read [25 15] INFO [2022-01-10 12:53:37,608] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,609] jepsen worker 75 - jepsen.util 165 :ok :read [25 15] INFO [2022-01-10 12:53:37,609] jepsen worker 75 - jepsen.util 165 :invoke :write [25 16] INFO [2022-01-10 12:53:37,610] jepsen worker 78 - jepsen.util 168 :ok :write [26 35] INFO [2022-01-10 12:53:37,611] jepsen worker 21 - jepsen.util 201 :ok :write [7 40] INFO [2022-01-10 12:53:37,611] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,612] jepsen worker 21 - jepsen.util 201 :invoke :write [7 47] INFO [2022-01-10 12:53:37,613] jepsen worker 78 - jepsen.util 168 :ok :read [26 35] INFO [2022-01-10 12:53:37,613] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,614] jepsen worker 78 - jepsen.util 168 :ok :read [26 35] INFO [2022-01-10 12:53:37,615] jepsen worker 78 - jepsen.util 168 :invoke :write [26 46] INFO [2022-01-10 12:53:37,616] jepsen worker 7 - jepsen.util 7 :ok :write [2 39] INFO [2022-01-10 12:53:37,616] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:37,616] jepsen worker 51 - jepsen.util 141 :ok :write [17 2] INFO [2022-01-10 12:53:37,617] jepsen worker 7 - jepsen.util 7 :ok :read [2 39] INFO [2022-01-10 12:53:37,617] jepsen worker 51 - jepsen.util 141 :invoke :write [17 40] INFO [2022-01-10 12:53:37,618] jepsen worker 7 - jepsen.util 7 :invoke :write [2 13] INFO [2022-01-10 12:53:37,621] jepsen worker 4 - jepsen.util 94 :ok :write [1 37] INFO [2022-01-10 12:53:37,622] jepsen worker 40 - jepsen.util 130 :ok :write [13 33] INFO [2022-01-10 12:53:37,622] jepsen worker 4 - jepsen.util 94 :invoke :write [1 9] INFO [2022-01-10 12:53:37,622] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:37,623] jepsen worker 40 - jepsen.util 130 :ok :read [13 33] INFO [2022-01-10 12:53:37,623] jepsen worker 15 - jepsen.util 15 :ok :write [5 16] INFO [2022-01-10 12:53:37,624] jepsen worker 40 - jepsen.util 130 :invoke :write [13 43] INFO [2022-01-10 12:53:37,624] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:37,625] jepsen worker 15 - jepsen.util 15 :ok :read [5 16] INFO [2022-01-10 12:53:37,626] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:37,627] jepsen worker 15 - jepsen.util 15 :ok :read [5 16] INFO [2022-01-10 12:53:37,627] jepsen worker 89 - jepsen.util 269 :ok :write [29 35] INFO [2022-01-10 12:53:37,627] jepsen worker 12 - jepsen.util 192 :ok :write [4 26] INFO [2022-01-10 12:53:37,627] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:37,628] jepsen worker 48 - jepsen.util 228 :ok :write [16 23] INFO [2022-01-10 12:53:37,628] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,628] jepsen worker 15 - jepsen.util 15 :ok :read [5 16] INFO [2022-01-10 12:53:37,628] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,629] jepsen worker 12 - jepsen.util 192 :ok :read [4 26] INFO [2022-01-10 12:53:37,629] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:37,629] jepsen worker 48 - jepsen.util 228 :ok :read [16 23] INFO [2022-01-10 12:53:37,629] jepsen worker 89 - jepsen.util 269 :invoke :write [29 42] INFO [2022-01-10 12:53:37,630] jepsen worker 15 - jepsen.util 15 :ok :read [5 16] INFO [2022-01-10 12:53:37,630] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,630] jepsen worker 15 - jepsen.util 15 :invoke :write [5 23] INFO [2022-01-10 12:53:37,631] jepsen worker 12 - jepsen.util 192 :ok :read [4 26] INFO [2022-01-10 12:53:37,631] jepsen worker 48 - jepsen.util 228 :invoke :write [16 36] INFO [2022-01-10 12:53:37,631] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,632] jepsen worker 12 - jepsen.util 192 :ok :read [4 26] INFO [2022-01-10 12:53:37,633] jepsen worker 12 - jepsen.util 192 :invoke :write [4 32] INFO [2022-01-10 12:53:37,633] jepsen worker 83 - jepsen.util 263 :ok :write [27 16] INFO [2022-01-10 12:53:37,633] jepsen worker 70 - jepsen.util 160 :ok :write [23 16] INFO [2022-01-10 12:53:37,633] jepsen worker 70 - jepsen.util 160 :invoke :write [23 10] INFO [2022-01-10 12:53:37,634] jepsen worker 83 - jepsen.util 263 :invoke :write [27 37] INFO [2022-01-10 12:53:37,638] jepsen worker 33 - jepsen.util 123 :ok :write [11 7] INFO [2022-01-10 12:53:37,639] jepsen worker 33 - jepsen.util 123 :invoke :write [11 49] INFO [2022-01-10 12:53:37,641] jepsen worker 85 - jepsen.util 265 :ok :write [28 30] INFO [2022-01-10 12:53:37,641] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,642] jepsen worker 85 - jepsen.util 265 :ok :read [28 30] INFO [2022-01-10 12:53:37,643] jepsen worker 85 - jepsen.util 265 :invoke :write [28 10] INFO [2022-01-10 12:53:37,644] jepsen worker 1 - jepsen.util 91 :ok :write [0 48] INFO [2022-01-10 12:53:37,645] jepsen worker 38 - jepsen.util 38 :ok :write [12 30] INFO [2022-01-10 12:53:37,645] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,645] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,646] jepsen worker 1 - jepsen.util 91 :ok :read [0 48] INFO [2022-01-10 12:53:37,646] jepsen worker 68 - jepsen.util 68 :ok :write [22 10] INFO [2022-01-10 12:53:37,646] jepsen worker 38 - jepsen.util 38 :ok :read [12 30] INFO [2022-01-10 12:53:37,646] jepsen worker 1 - jepsen.util 91 :invoke :write [0 14] INFO [2022-01-10 12:53:37,647] jepsen worker 38 - jepsen.util 38 :invoke :write [12 19] INFO [2022-01-10 12:53:37,647] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:37,648] jepsen worker 68 - jepsen.util 68 :ok :read [22 10] INFO [2022-01-10 12:53:37,649] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:37,650] jepsen worker 68 - jepsen.util 68 :ok :read [22 10] INFO [2022-01-10 12:53:37,650] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:37,651] jepsen worker 44 - jepsen.util 44 :ok :write [14 32] INFO [2022-01-10 12:53:37,651] jepsen worker 68 - jepsen.util 68 :ok :read [22 10] INFO [2022-01-10 12:53:37,651] jepsen worker 44 - jepsen.util 44 :invoke :write [14 13] INFO [2022-01-10 12:53:37,652] jepsen worker 68 - jepsen.util 68 :invoke :write [22 40] INFO [2022-01-10 12:53:37,652] jepsen worker 78 - jepsen.util 168 :ok :write [26 46] INFO [2022-01-10 12:53:37,652] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,653] jepsen worker 78 - jepsen.util 168 :ok :read [26 46] INFO [2022-01-10 12:53:37,653] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,654] jepsen worker 78 - jepsen.util 168 :ok :read [26 46] INFO [2022-01-10 12:53:37,655] jepsen worker 78 - jepsen.util 168 :invoke :write [26 19] INFO [2022-01-10 12:53:37,657] jepsen worker 62 - jepsen.util 242 :ok :write [20 40] INFO [2022-01-10 12:53:37,657] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,658] jepsen worker 62 - jepsen.util 242 :ok :read [20 40] INFO [2022-01-10 12:53:37,658] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,659] jepsen worker 51 - jepsen.util 141 :ok :write [17 40] INFO [2022-01-10 12:53:37,659] jepsen worker 62 - jepsen.util 242 :ok :read [20 40] INFO [2022-01-10 12:53:37,660] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,660] jepsen worker 51 - jepsen.util 141 :invoke :write [17 12] INFO [2022-01-10 12:53:37,661] jepsen worker 62 - jepsen.util 242 :ok :read [20 40] INFO [2022-01-10 12:53:37,661] jepsen worker 62 - jepsen.util 242 :invoke :write [20 37] INFO [2022-01-10 12:53:37,663] jepsen worker 40 - jepsen.util 130 :ok :write [13 43] INFO [2022-01-10 12:53:37,663] jepsen worker 75 - jepsen.util 165 :ok :write [25 16] INFO [2022-01-10 12:53:37,663] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:37,664] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,664] jepsen worker 40 - jepsen.util 130 :ok :read [13 43] INFO [2022-01-10 12:53:37,665] jepsen worker 75 - jepsen.util 165 :ok :read [25 16] INFO [2022-01-10 12:53:37,665] jepsen worker 40 - jepsen.util 130 :invoke :write [13 13] INFO [2022-01-10 12:53:37,665] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,666] jepsen worker 75 - jepsen.util 165 :ok :read [25 16] INFO [2022-01-10 12:53:37,666] jepsen worker 4 - jepsen.util 94 :ok :write [1 9] INFO [2022-01-10 12:53:37,666] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,667] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:37,667] jepsen worker 75 - jepsen.util 165 :ok :read [25 16] INFO [2022-01-10 12:53:37,668] jepsen worker 4 - jepsen.util 94 :ok :read [1 9] INFO [2022-01-10 12:53:37,668] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,668] jepsen worker 89 - jepsen.util 269 :ok :write [29 42] INFO [2022-01-10 12:53:37,668] jepsen worker 21 - jepsen.util 201 :ok :write [7 47] INFO [2022-01-10 12:53:37,669] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:37,669] jepsen worker 75 - jepsen.util 165 :ok :read [25 16] INFO [2022-01-10 12:53:37,669] jepsen worker 89 - jepsen.util 269 :invoke :write [29 42] INFO [2022-01-10 12:53:37,669] jepsen worker 4 - jepsen.util 94 :ok :read [1 9] INFO [2022-01-10 12:53:37,669] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,670] jepsen worker 21 - jepsen.util 201 :invoke :write [7 13] INFO [2022-01-10 12:53:37,670] jepsen worker 4 - jepsen.util 94 :invoke :write [1 2] INFO [2022-01-10 12:53:37,672] jepsen worker 75 - jepsen.util 165 :ok :read [25 16] INFO [2022-01-10 12:53:37,672] jepsen worker 75 - jepsen.util 165 :invoke :write [25 26] INFO [2022-01-10 12:53:37,673] jepsen worker 12 - jepsen.util 192 :ok :write [4 32] INFO [2022-01-10 12:53:37,673] jepsen worker 1 - jepsen.util 91 :ok :write [0 14] INFO [2022-01-10 12:53:37,674] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,674] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,675] jepsen worker 12 - jepsen.util 192 :ok :read [4 32] INFO [2022-01-10 12:53:37,675] jepsen worker 1 - jepsen.util 91 :ok :read [0 14] INFO [2022-01-10 12:53:37,675] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,676] jepsen worker 7 - jepsen.util 7 :ok :write [2 13] INFO [2022-01-10 12:53:37,676] jepsen worker 1 - jepsen.util 91 :ok :read [0 14] INFO [2022-01-10 12:53:37,676] jepsen worker 12 - jepsen.util 192 :invoke :write [4 21] INFO [2022-01-10 12:53:37,677] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,677] jepsen worker 7 - jepsen.util 7 :invoke :write [2 4] INFO [2022-01-10 12:53:37,678] jepsen worker 1 - jepsen.util 91 :ok :read [0 14] INFO [2022-01-10 12:53:37,678] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,679] jepsen worker 1 - jepsen.util 91 :ok :read [0 14] INFO [2022-01-10 12:53:37,680] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,680] jepsen worker 1 - jepsen.util 91 :ok :read [0 14] INFO [2022-01-10 12:53:37,681] jepsen worker 1 - jepsen.util 91 :invoke :write [0 5] INFO [2022-01-10 12:53:37,681] jepsen worker 70 - jepsen.util 160 :ok :write [23 10] INFO [2022-01-10 12:53:37,682] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:37,682] jepsen worker 15 - jepsen.util 15 :ok :write [5 23] INFO [2022-01-10 12:53:37,683] jepsen worker 70 - jepsen.util 160 :ok :read [23 10] INFO [2022-01-10 12:53:37,683] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:37,683] jepsen worker 70 - jepsen.util 160 :invoke :write [23 21] INFO [2022-01-10 12:53:37,684] jepsen worker 15 - jepsen.util 15 :ok :read [5 23] INFO [2022-01-10 12:53:37,684] jepsen worker 15 - jepsen.util 15 :invoke :write [5 47] INFO [2022-01-10 12:53:37,687] jepsen worker 48 - jepsen.util 228 :ok :write [16 36] INFO [2022-01-10 12:53:37,688] jepsen worker 48 - jepsen.util 228 :invoke :write [16 24] INFO [2022-01-10 12:53:37,690] jepsen worker 85 - jepsen.util 265 :ok :write [28 10] INFO [2022-01-10 12:53:37,690] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,691] jepsen worker 85 - jepsen.util 265 :ok :read [28 10] INFO [2022-01-10 12:53:37,692] jepsen worker 85 - jepsen.util 265 :invoke :write [28 46] INFO [2022-01-10 12:53:37,693] jepsen worker 83 - jepsen.util 263 :ok :write [27 37] INFO [2022-01-10 12:53:37,694] jepsen worker 83 - jepsen.util 263 :invoke :write [27 36] INFO [2022-01-10 12:53:37,696] jepsen worker 40 - jepsen.util 130 :ok :write [13 13] INFO [2022-01-10 12:53:37,696] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:37,698] jepsen worker 74 - jepsen.util 344 :fail :read [24 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:37,698] jepsen worker 40 - jepsen.util 130 :ok :read [13 13] INFO [2022-01-10 12:53:37,698] jepsen worker 74 - jepsen.util 344 :invoke :write [24 46] INFO [2022-01-10 12:53:37,699] jepsen worker 33 - jepsen.util 123 :ok :write [11 49] INFO [2022-01-10 12:53:37,699] jepsen worker 38 - jepsen.util 38 :ok :write [12 19] INFO [2022-01-10 12:53:37,699] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:37,700] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,700] jepsen worker 40 - jepsen.util 130 :ok :read [13 13] INFO [2022-01-10 12:53:37,700] jepsen worker 33 - jepsen.util 123 :invoke :write [11 13] INFO [2022-01-10 12:53:37,701] jepsen worker 40 - jepsen.util 130 :invoke :write [13 4] INFO [2022-01-10 12:53:37,701] jepsen worker 38 - jepsen.util 38 :ok :read [12 19] INFO [2022-01-10 12:53:37,702] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,702] jepsen worker 38 - jepsen.util 38 :ok :read [12 19] INFO [2022-01-10 12:53:37,703] jepsen worker 68 - jepsen.util 68 :ok :write [22 40] INFO [2022-01-10 12:53:37,703] jepsen worker 78 - jepsen.util 168 :ok :write [26 19] INFO [2022-01-10 12:53:37,703] jepsen worker 38 - jepsen.util 38 :invoke :write [12 7] INFO [2022-01-10 12:53:37,704] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,704] jepsen worker 68 - jepsen.util 68 :invoke :write [22 49] INFO [2022-01-10 12:53:37,705] jepsen worker 78 - jepsen.util 168 :ok :read [26 19] INFO [2022-01-10 12:53:37,705] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,705] jepsen worker 44 - jepsen.util 44 :ok :write [14 13] INFO [2022-01-10 12:53:37,706] jepsen worker 44 - jepsen.util 44 :invoke :write [14 14] INFO [2022-01-10 12:53:37,706] jepsen worker 78 - jepsen.util 168 :ok :read [26 19] INFO [2022-01-10 12:53:37,707] jepsen worker 89 - jepsen.util 269 :ok :write [29 42] INFO [2022-01-10 12:53:37,707] jepsen worker 78 - jepsen.util 168 :invoke :write [26 40] INFO [2022-01-10 12:53:37,708] jepsen worker 89 - jepsen.util 269 :invoke :write [29 26] INFO [2022-01-10 12:53:37,710] jepsen worker 51 - jepsen.util 141 :ok :write [17 12] INFO [2022-01-10 12:53:37,710] jepsen worker 51 - jepsen.util 141 :invoke :write [17 44] INFO [2022-01-10 12:53:37,711] jepsen worker 62 - jepsen.util 242 :ok :write [20 37] INFO [2022-01-10 12:53:37,711] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,712] jepsen worker 62 - jepsen.util 242 :ok :read [20 37] INFO [2022-01-10 12:53:37,712] jepsen worker 76 - jepsen.util 166 :info :write [25 40] :RequestTimeoutException INFO [2022-01-10 12:53:37,712] jepsen worker 62 - jepsen.util 242 :invoke :write [20 40] INFO [2022-01-10 12:53:37,713] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,714] jepsen worker 76 - jepsen.util 256 :ok :read [25 16] INFO [2022-01-10 12:53:37,715] jepsen worker 76 - jepsen.util 256 :invoke :write [25 46] INFO [2022-01-10 12:53:37,717] jepsen worker 4 - jepsen.util 94 :ok :write [1 2] INFO [2022-01-10 12:53:37,717] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:37,719] jepsen worker 4 - jepsen.util 94 :ok :read [1 2] INFO [2022-01-10 12:53:37,719] jepsen worker 4 - jepsen.util 94 :invoke :write [1 38] INFO [2022-01-10 12:53:37,720] jepsen worker 1 - jepsen.util 91 :ok :write [0 5] INFO [2022-01-10 12:53:37,720] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,721] jepsen worker 1 - jepsen.util 91 :ok :read [0 5] INFO [2022-01-10 12:53:37,722] jepsen worker 1 - jepsen.util 91 :invoke :write [0 41] INFO [2022-01-10 12:53:37,724] jepsen worker 12 - jepsen.util 192 :ok :write [4 21] INFO [2022-01-10 12:53:37,724] jepsen worker 40 - jepsen.util 130 :ok :write [13 4] INFO [2022-01-10 12:53:37,725] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:37,725] jepsen worker 12 - jepsen.util 192 :invoke :write [4 32] INFO [2022-01-10 12:53:37,726] jepsen worker 40 - jepsen.util 130 :ok :read [13 4] INFO [2022-01-10 12:53:37,726] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:37,727] jepsen worker 40 - jepsen.util 130 :ok :read [13 4] INFO [2022-01-10 12:53:37,728] jepsen worker 40 - jepsen.util 130 :invoke :write [13 17] INFO [2022-01-10 12:53:37,729] jepsen worker 70 - jepsen.util 160 :ok :write [23 21] INFO [2022-01-10 12:53:37,730] jepsen worker 70 - jepsen.util 160 :invoke :write [23 2] INFO [2022-01-10 12:53:37,730] jepsen worker 21 - jepsen.util 201 :ok :write [7 13] INFO [2022-01-10 12:53:37,731] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,732] jepsen worker 21 - jepsen.util 201 :ok :read [7 13] INFO [2022-01-10 12:53:37,732] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,733] jepsen worker 21 - jepsen.util 201 :ok :read [7 13] INFO [2022-01-10 12:53:37,734] jepsen worker 21 - jepsen.util 201 :invoke :write [7 48] INFO [2022-01-10 12:53:37,736] jepsen worker 85 - jepsen.util 265 :ok :write [28 46] INFO [2022-01-10 12:53:37,737] jepsen worker 85 - jepsen.util 265 :invoke :write [28 45] INFO [2022-01-10 12:53:37,737] jepsen worker 7 - jepsen.util 7 :ok :write [2 4] INFO [2022-01-10 12:53:37,737] jepsen worker 7 - jepsen.util 7 :invoke :write [2 12] INFO [2022-01-10 12:53:37,739] jepsen worker 89 - jepsen.util 269 :ok :write [29 26] INFO [2022-01-10 12:53:37,739] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:37,740] jepsen worker 89 - jepsen.util 269 :ok :read [29 26] INFO [2022-01-10 12:53:37,741] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:37,742] jepsen worker 89 - jepsen.util 269 :ok :read [29 26] INFO [2022-01-10 12:53:37,742] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:37,743] jepsen worker 38 - jepsen.util 38 :ok :write [12 7] INFO [2022-01-10 12:53:37,743] jepsen worker 89 - jepsen.util 269 :ok :read [29 26] INFO [2022-01-10 12:53:37,743] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,744] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:37,744] jepsen worker 38 - jepsen.util 38 :ok :read [12 7] INFO [2022-01-10 12:53:37,745] jepsen worker 89 - jepsen.util 269 :ok :read [29 26] INFO [2022-01-10 12:53:37,745] jepsen worker 15 - jepsen.util 15 :ok :write [5 47] INFO [2022-01-10 12:53:37,745] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,746] jepsen worker 89 - jepsen.util 269 :invoke :write [29 33] INFO [2022-01-10 12:53:37,746] jepsen worker 38 - jepsen.util 38 :ok :read [12 7] INFO [2022-01-10 12:53:37,746] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:37,747] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,747] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:37,747] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:37,748] jepsen worker 38 - jepsen.util 38 :ok :read [12 7] INFO [2022-01-10 12:53:37,748] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:37,748] jepsen worker 38 - jepsen.util 38 :invoke :write [12 8] INFO [2022-01-10 12:53:37,749] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:37,750] jepsen worker 48 - jepsen.util 228 :ok :write [16 24] INFO [2022-01-10 12:53:37,750] jepsen worker 15 - jepsen.util 15 :ok :read [5 47] INFO [2022-01-10 12:53:37,750] jepsen worker 78 - jepsen.util 168 :ok :write [26 40] INFO [2022-01-10 12:53:37,750] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,751] jepsen worker 15 - jepsen.util 15 :invoke :write [5 13] INFO [2022-01-10 12:53:37,751] jepsen worker 48 - jepsen.util 228 :ok :read [16 24] INFO [2022-01-10 12:53:37,752] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,752] jepsen worker 48 - jepsen.util 228 :invoke :write [16 8] INFO [2022-01-10 12:53:37,752] jepsen worker 78 - jepsen.util 168 :ok :read [26 40] INFO [2022-01-10 12:53:37,753] jepsen worker 78 - jepsen.util 168 :invoke :write [26 27] INFO [2022-01-10 12:53:37,755] jepsen worker 83 - jepsen.util 263 :ok :write [27 36] INFO [2022-01-10 12:53:37,756] jepsen worker 83 - jepsen.util 263 :invoke :write [27 13] INFO [2022-01-10 12:53:37,758] jepsen worker 51 - jepsen.util 141 :ok :write [17 44] INFO [2022-01-10 12:53:37,758] jepsen worker 1 - jepsen.util 91 :ok :write [0 41] INFO [2022-01-10 12:53:37,758] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,759] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,759] jepsen worker 51 - jepsen.util 141 :ok :read [17 44] INFO [2022-01-10 12:53:37,760] jepsen worker 1 - jepsen.util 91 :ok :read [0 41] INFO [2022-01-10 12:53:37,760] jepsen worker 33 - jepsen.util 123 :ok :write [11 13] INFO [2022-01-10 12:53:37,760] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,761] jepsen worker 33 - jepsen.util 123 :invoke :write [11 3] INFO [2022-01-10 12:53:37,761] jepsen worker 51 - jepsen.util 141 :ok :read [17 44] INFO [2022-01-10 12:53:37,761] jepsen worker 1 - jepsen.util 91 :invoke :write [0 9] INFO [2022-01-10 12:53:37,762] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,763] jepsen worker 51 - jepsen.util 141 :ok :read [17 44] INFO [2022-01-10 12:53:37,763] jepsen worker 51 - jepsen.util 141 :invoke :write [17 49] INFO [2022-01-10 12:53:37,764] jepsen worker 40 - jepsen.util 130 :ok :write [13 17] INFO [2022-01-10 12:53:37,765] jepsen worker 40 - jepsen.util 130 :invoke :write [13 31] INFO [2022-01-10 12:53:37,765] jepsen worker 4 - jepsen.util 94 :ok :write [1 38] INFO [2022-01-10 12:53:37,766] jepsen worker 4 - jepsen.util 94 :invoke :write [1 37] INFO [2022-01-10 12:53:37,767] jepsen worker 68 - jepsen.util 68 :ok :write [22 49] INFO [2022-01-10 12:53:37,768] jepsen worker 68 - jepsen.util 68 :invoke :write [22 31] INFO [2022-01-10 12:53:37,772] jepsen worker 44 - jepsen.util 44 :ok :write [14 14] INFO [2022-01-10 12:53:37,772] jepsen worker 44 - jepsen.util 44 :invoke :write [14 17] INFO [2022-01-10 12:53:37,773] jepsen worker 89 - jepsen.util 269 :ok :write [29 33] INFO [2022-01-10 12:53:37,773] jepsen worker 12 - jepsen.util 192 :ok :write [4 32] INFO [2022-01-10 12:53:37,774] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,775] jepsen worker 89 - jepsen.util 269 :invoke :write [29 36] INFO [2022-01-10 12:53:37,775] jepsen worker 12 - jepsen.util 192 :ok :read [4 32] INFO [2022-01-10 12:53:37,776] jepsen worker 62 - jepsen.util 242 :ok :write [20 40] INFO [2022-01-10 12:53:37,776] jepsen worker 12 - jepsen.util 192 :invoke :write [4 35] INFO [2022-01-10 12:53:37,777] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,778] jepsen worker 62 - jepsen.util 242 :ok :read [20 40] INFO [2022-01-10 12:53:37,778] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,779] jepsen worker 62 - jepsen.util 242 :ok :read [20 40] INFO [2022-01-10 12:53:37,779] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,780] jepsen worker 70 - jepsen.util 160 :ok :write [23 2] INFO [2022-01-10 12:53:37,780] jepsen worker 62 - jepsen.util 242 :ok :read [20 40] INFO [2022-01-10 12:53:37,780] jepsen worker 70 - jepsen.util 160 :invoke :write [23 48] INFO [2022-01-10 12:53:37,781] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,782] jepsen worker 62 - jepsen.util 242 :ok :read [20 40] INFO [2022-01-10 12:53:37,782] jepsen worker 62 - jepsen.util 242 :invoke :write [20 48] INFO [2022-01-10 12:53:37,787] jepsen worker 85 - jepsen.util 265 :ok :write [28 45] INFO [2022-01-10 12:53:37,787] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,793] jepsen worker 38 - jepsen.util 38 :ok :write [12 8] INFO [2022-01-10 12:53:37,793] jepsen worker 1 - jepsen.util 91 :ok :write [0 9] INFO [2022-01-10 12:53:37,794] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:37,794] jepsen worker 1 - jepsen.util 91 :invoke :write [0 47] INFO [2022-01-10 12:53:37,797] jepsen worker 29 - jepsen.util 299 :fail :read [9 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:37,798] jepsen worker 29 - jepsen.util 299 :invoke :write [9 21] INFO [2022-01-10 12:53:37,799] jepsen worker 78 - jepsen.util 168 :ok :write [26 27] INFO [2022-01-10 12:53:37,799] jepsen worker 78 - jepsen.util 168 :invoke :write [26 1] INFO [2022-01-10 12:53:37,799] jepsen worker 75 - jepsen.util 165 :ok :write [25 26] INFO [2022-01-10 12:53:37,800] jepsen worker 75 - jepsen.util 165 :invoke :write [25 26] INFO [2022-01-10 12:53:37,804] jepsen worker 21 - jepsen.util 201 :ok :write [7 48] INFO [2022-01-10 12:53:37,805] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:37,805] jepsen worker 40 - jepsen.util 130 :ok :write [13 31] INFO [2022-01-10 12:53:37,806] jepsen worker 21 - jepsen.util 201 :ok :read [7 48] INFO [2022-01-10 12:53:37,806] jepsen worker 40 - jepsen.util 130 :invoke :write [13 38] INFO [2022-01-10 12:53:37,807] jepsen worker 51 - jepsen.util 141 :ok :write [17 49] INFO [2022-01-10 12:53:37,807] jepsen worker 21 - jepsen.util 201 :invoke :write [7 4] INFO [2022-01-10 12:53:37,808] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,809] jepsen worker 7 - jepsen.util 7 :ok :write [2 12] INFO [2022-01-10 12:53:37,810] jepsen worker 7 - jepsen.util 7 :invoke :write [2 49] INFO [2022-01-10 12:53:37,814] jepsen worker 4 - jepsen.util 94 :ok :write [1 37] INFO [2022-01-10 12:53:37,815] jepsen worker 89 - jepsen.util 269 :ok :write [29 36] INFO [2022-01-10 12:53:37,815] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:37,816] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:37,816] jepsen worker 51 - jepsen.util 141 :ok :read [17 49] INFO [2022-01-10 12:53:37,817] jepsen worker 89 - jepsen.util 269 :ok :read [29 36] INFO [2022-01-10 12:53:37,817] jepsen worker 4 - jepsen.util 94 :ok :read [1 37] INFO [2022-01-10 12:53:37,817] jepsen worker 15 - jepsen.util 15 :ok :write [5 13] INFO [2022-01-10 12:53:37,817] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,818] jepsen worker 51 - jepsen.util 141 :ok :read [17 49] INFO [2022-01-10 12:53:37,818] jepsen worker 1 - jepsen.util 91 :ok :write [0 47] INFO [2022-01-10 12:53:37,818] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:37,819] jepsen worker 15 - jepsen.util 15 :invoke :write [5 37] INFO [2022-01-10 12:53:37,820] jepsen worker 51 - jepsen.util 141 :invoke :write [17 3] INFO [2022-01-10 12:53:37,820] jepsen worker 89 - jepsen.util 269 :ok :read [29 36] INFO [2022-01-10 12:53:37,820] jepsen worker 1 - jepsen.util 91 :invoke :write [0 43] INFO [2022-01-10 12:53:37,821] jepsen worker 89 - jepsen.util 269 :invoke :write [29 9] INFO [2022-01-10 12:53:37,821] jepsen worker 48 - jepsen.util 228 :ok :write [16 8] INFO [2022-01-10 12:53:37,821] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:37,822] jepsen worker 85 - jepsen.util 265 :ok :read [28 45] INFO [2022-01-10 12:53:37,822] jepsen worker 48 - jepsen.util 228 :invoke :write [16 23] INFO [2022-01-10 12:53:37,822] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,823] jepsen worker 4 - jepsen.util 94 :ok :read [1 37] INFO [2022-01-10 12:53:37,823] jepsen worker 85 - jepsen.util 265 :ok :read [28 45] INFO [2022-01-10 12:53:37,823] jepsen worker 4 - jepsen.util 94 :invoke :write [1 30] INFO [2022-01-10 12:53:37,824] jepsen worker 85 - jepsen.util 265 :invoke :write [28 10] INFO [2022-01-10 12:53:37,825] jepsen worker 12 - jepsen.util 192 :ok :write [4 35] INFO [2022-01-10 12:53:37,825] jepsen worker 12 - jepsen.util 192 :invoke :write [4 3] INFO [2022-01-10 12:53:37,873] jepsen worker 70 - jepsen.util 160 :ok :write [23 48] INFO [2022-01-10 12:53:37,874] jepsen worker 70 - jepsen.util 160 :invoke :write [23 34] INFO [2022-01-10 12:53:37,875] jepsen worker 83 - jepsen.util 263 :ok :write [27 13] INFO [2022-01-10 12:53:37,875] jepsen worker 83 - jepsen.util 263 :invoke :write [27 45] INFO [2022-01-10 12:53:37,879] jepsen worker 33 - jepsen.util 123 :ok :write [11 3] INFO [2022-01-10 12:53:37,879] jepsen worker 85 - jepsen.util 265 :ok :write [28 10] INFO [2022-01-10 12:53:37,880] jepsen worker 40 - jepsen.util 130 :ok :write [13 38] INFO [2022-01-10 12:53:37,880] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:37,881] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,881] jepsen worker 33 - jepsen.util 123 :ok :read [11 3] INFO [2022-01-10 12:53:37,882] jepsen worker 85 - jepsen.util 265 :ok :read [28 10] INFO [2022-01-10 12:53:37,882] jepsen worker 40 - jepsen.util 130 :invoke :write [13 46] INFO [2022-01-10 12:53:37,883] jepsen worker 33 - jepsen.util 123 :invoke :write [11 49] INFO [2022-01-10 12:53:37,883] jepsen worker 85 - jepsen.util 265 :invoke :write [28 0] INFO [2022-01-10 12:53:37,884] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:37,885] jepsen worker 38 - jepsen.util 38 :invoke :write [12 40] INFO [2022-01-10 12:53:37,885] jepsen worker 89 - jepsen.util 269 :ok :write [29 9] INFO [2022-01-10 12:53:37,886] jepsen worker 89 - jepsen.util 269 :invoke :write [29 28] INFO [2022-01-10 12:53:37,887] jepsen worker 44 - jepsen.util 44 :ok :write [14 17] INFO [2022-01-10 12:53:37,887] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:37,888] jepsen worker 44 - jepsen.util 44 :ok :read [14 17] INFO [2022-01-10 12:53:37,889] jepsen worker 44 - jepsen.util 44 :invoke :write [14 47] INFO [2022-01-10 12:53:37,893] jepsen worker 78 - jepsen.util 168 :ok :write [26 1] INFO [2022-01-10 12:53:37,894] jepsen worker 1 - jepsen.util 91 :ok :write [0 43] INFO [2022-01-10 12:53:37,894] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,894] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,895] jepsen worker 78 - jepsen.util 168 :ok :read [26 1] INFO [2022-01-10 12:53:37,895] jepsen worker 68 - jepsen.util 68 :ok :write [22 31] INFO [2022-01-10 12:53:37,895] jepsen worker 1 - jepsen.util 91 :ok :read [0 43] INFO [2022-01-10 12:53:37,896] jepsen worker 78 - jepsen.util 168 :invoke :write [26 37] INFO [2022-01-10 12:53:37,897] jepsen worker 1 - jepsen.util 91 :invoke :write [0 49] INFO [2022-01-10 12:53:37,897] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:37,898] jepsen worker 68 - jepsen.util 68 :ok :read [22 31] INFO [2022-01-10 12:53:37,898] jepsen worker 68 - jepsen.util 68 :invoke :write [22 31] INFO [2022-01-10 12:53:37,898] jepsen worker 62 - jepsen.util 242 :ok :write [20 48] INFO [2022-01-10 12:53:37,899] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,900] jepsen worker 62 - jepsen.util 242 :ok :read [20 48] INFO [2022-01-10 12:53:37,900] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,901] jepsen worker 40 - jepsen.util 130 :ok :write [13 46] INFO [2022-01-10 12:53:37,901] jepsen worker 62 - jepsen.util 242 :ok :read [20 48] INFO [2022-01-10 12:53:37,901] jepsen worker 40 - jepsen.util 130 :invoke :write [13 18] INFO [2022-01-10 12:53:37,901] jepsen worker 52 - jepsen.util 52 :ok :write [17 16] INFO [2022-01-10 12:53:37,902] jepsen worker 62 - jepsen.util 242 :invoke :write [20 10] INFO [2022-01-10 12:53:37,902] jepsen worker 52 - jepsen.util 52 :invoke :write [17 41] INFO [2022-01-10 12:53:37,904] jepsen worker 75 - jepsen.util 165 :ok :write [25 26] INFO [2022-01-10 12:53:37,904] jepsen worker 75 - jepsen.util 165 :invoke :write [25 46] INFO [2022-01-10 12:53:37,908] jepsen worker 89 - jepsen.util 269 :ok :write [29 28] INFO [2022-01-10 12:53:37,909] jepsen worker 4 - jepsen.util 94 :ok :write [1 30] INFO [2022-01-10 12:53:37,909] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:37,909] jepsen worker 4 - jepsen.util 94 :invoke :write [1 38] INFO [2022-01-10 12:53:37,910] jepsen worker 89 - jepsen.util 269 :ok :read [29 28] INFO [2022-01-10 12:53:37,910] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:37,910] jepsen worker 21 - jepsen.util 201 :ok :write [7 4] INFO [2022-01-10 12:53:37,911] jepsen worker 89 - jepsen.util 269 :ok :read [29 28] INFO [2022-01-10 12:53:37,911] jepsen worker 21 - jepsen.util 201 :invoke :write [7 38] INFO [2022-01-10 12:53:37,912] jepsen worker 89 - jepsen.util 269 :invoke :write [29 13] INFO [2022-01-10 12:53:37,915] jepsen worker 7 - jepsen.util 7 :ok :write [2 49] INFO [2022-01-10 12:53:37,915] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:37,916] jepsen worker 7 - jepsen.util 7 :ok :read [2 49] INFO [2022-01-10 12:53:37,917] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:37,917] jepsen worker 12 - jepsen.util 192 :ok :write [4 3] INFO [2022-01-10 12:53:37,918] jepsen worker 7 - jepsen.util 7 :ok :read [2 49] INFO [2022-01-10 12:53:37,918] jepsen worker 1 - jepsen.util 91 :ok :write [0 49] INFO [2022-01-10 12:53:37,918] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:37,919] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:37,919] jepsen worker 12 - jepsen.util 192 :ok :read [4 3] INFO [2022-01-10 12:53:37,920] jepsen worker 1 - jepsen.util 91 :invoke :write [0 20] INFO [2022-01-10 12:53:37,920] jepsen worker 7 - jepsen.util 7 :ok :read [2 49] INFO [2022-01-10 12:53:37,920] jepsen worker 15 - jepsen.util 15 :ok :write [5 37] INFO [2022-01-10 12:53:37,920] jepsen worker 12 - jepsen.util 192 :invoke :write [4 9] INFO [2022-01-10 12:53:37,921] jepsen worker 7 - jepsen.util 7 :invoke :write [2 36] INFO [2022-01-10 12:53:37,922] jepsen worker 15 - jepsen.util 15 :invoke :write [5 36] INFO [2022-01-10 12:53:37,923] jepsen worker 70 - jepsen.util 160 :ok :write [23 34] INFO [2022-01-10 12:53:37,924] jepsen worker 70 - jepsen.util 160 :invoke :write [23 47] INFO [2022-01-10 12:53:37,926] jepsen worker 50 - jepsen.util 50 :ok :write [16 9] INFO [2022-01-10 12:53:37,926] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,927] jepsen worker 50 - jepsen.util 50 :ok :read [16 9] INFO [2022-01-10 12:53:37,928] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,929] jepsen worker 50 - jepsen.util 50 :ok :read [16 9] INFO [2022-01-10 12:53:37,929] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,930] jepsen worker 40 - jepsen.util 130 :ok :write [13 18] INFO [2022-01-10 12:53:37,930] jepsen worker 85 - jepsen.util 265 :ok :write [28 0] INFO [2022-01-10 12:53:37,930] jepsen worker 50 - jepsen.util 50 :ok :read [16 9] INFO [2022-01-10 12:53:37,931] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:37,932] jepsen worker 83 - jepsen.util 263 :ok :write [27 45] INFO [2022-01-10 12:53:37,932] jepsen worker 85 - jepsen.util 265 :invoke :write [28 3] INFO [2022-01-10 12:53:37,932] jepsen worker 50 - jepsen.util 50 :invoke :write [16 19] INFO [2022-01-10 12:53:37,933] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:37,933] jepsen worker 40 - jepsen.util 130 :ok :read [13 18] INFO [2022-01-10 12:53:37,933] jepsen worker 40 - jepsen.util 130 :invoke :write [13 12] INFO [2022-01-10 12:53:37,934] jepsen worker 38 - jepsen.util 38 :ok :write [12 40] INFO [2022-01-10 12:53:37,934] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,934] jepsen worker 83 - jepsen.util 263 :ok :read [27 45] INFO [2022-01-10 12:53:37,935] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:37,935] jepsen worker 38 - jepsen.util 38 :ok :read [12 40] INFO [2022-01-10 12:53:37,936] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,936] jepsen worker 33 - jepsen.util 123 :ok :write [11 49] INFO [2022-01-10 12:53:37,936] jepsen worker 83 - jepsen.util 263 :ok :read [27 45] INFO [2022-01-10 12:53:37,936] jepsen worker 33 - jepsen.util 123 :invoke :write [11 34] INFO [2022-01-10 12:53:37,937] jepsen worker 38 - jepsen.util 38 :ok :read [12 40] INFO [2022-01-10 12:53:37,937] jepsen worker 83 - jepsen.util 263 :invoke :write [27 1] INFO [2022-01-10 12:53:37,937] jepsen worker 38 - jepsen.util 38 :invoke :write [12 47] INFO [2022-01-10 12:53:37,938] jepsen worker 89 - jepsen.util 269 :ok :write [29 13] INFO [2022-01-10 12:53:37,939] jepsen worker 89 - jepsen.util 269 :invoke :write [29 27] INFO [2022-01-10 12:53:37,941] jepsen worker 78 - jepsen.util 168 :ok :write [26 37] INFO [2022-01-10 12:53:37,941] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,942] jepsen worker 78 - jepsen.util 168 :ok :read [26 37] INFO [2022-01-10 12:53:37,943] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,943] jepsen worker 44 - jepsen.util 44 :ok :write [14 47] INFO [2022-01-10 12:53:37,944] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:37,944] jepsen worker 78 - jepsen.util 168 :ok :read [26 37] INFO [2022-01-10 12:53:37,944] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,945] jepsen worker 44 - jepsen.util 44 :ok :read [14 47] INFO [2022-01-10 12:53:37,945] jepsen worker 78 - jepsen.util 168 :ok :read [26 37] INFO [2022-01-10 12:53:37,945] jepsen worker 44 - jepsen.util 44 :invoke :write [14 8] INFO [2022-01-10 12:53:37,946] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,947] jepsen worker 78 - jepsen.util 168 :ok :read [26 37] INFO [2022-01-10 12:53:37,947] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,948] jepsen worker 52 - jepsen.util 52 :ok :write [17 41] INFO [2022-01-10 12:53:37,949] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,949] jepsen worker 78 - jepsen.util 168 :ok :read [26 37] INFO [2022-01-10 12:53:37,949] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:37,949] jepsen worker 68 - jepsen.util 68 :ok :write [22 31] INFO [2022-01-10 12:53:37,950] jepsen worker 52 - jepsen.util 52 :ok :read [17 41] INFO [2022-01-10 12:53:37,950] jepsen worker 78 - jepsen.util 168 :ok :read [26 37] INFO [2022-01-10 12:53:37,950] jepsen worker 68 - jepsen.util 68 :invoke :write [22 7] INFO [2022-01-10 12:53:37,951] jepsen worker 1 - jepsen.util 91 :ok :write [0 20] INFO [2022-01-10 12:53:37,951] jepsen worker 78 - jepsen.util 168 :invoke :write [26 26] INFO [2022-01-10 12:53:37,952] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,952] jepsen worker 62 - jepsen.util 242 :ok :write [20 10] INFO [2022-01-10 12:53:37,953] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,953] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,953] jepsen worker 1 - jepsen.util 91 :ok :read [0 20] INFO [2022-01-10 12:53:37,954] jepsen worker 62 - jepsen.util 242 :ok :read [20 10] INFO [2022-01-10 12:53:37,954] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,955] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,955] jepsen worker 1 - jepsen.util 91 :ok :read [0 20] INFO [2022-01-10 12:53:37,955] jepsen worker 4 - jepsen.util 94 :ok :write [1 38] INFO [2022-01-10 12:53:37,955] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,956] jepsen worker 62 - jepsen.util 242 :ok :read [20 10] INFO [2022-01-10 12:53:37,956] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:37,956] jepsen worker 1 - jepsen.util 91 :ok :read [0 20] INFO [2022-01-10 12:53:37,956] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,957] jepsen worker 4 - jepsen.util 94 :ok :read [1 38] INFO [2022-01-10 12:53:37,957] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:37,957] jepsen worker 62 - jepsen.util 242 :ok :read [20 10] INFO [2022-01-10 12:53:37,958] jepsen worker 4 - jepsen.util 94 :invoke :write [1 29] INFO [2022-01-10 12:53:37,958] jepsen worker 75 - jepsen.util 165 :ok :write [25 46] INFO [2022-01-10 12:53:37,958] jepsen worker 1 - jepsen.util 91 :ok :read [0 20] INFO [2022-01-10 12:53:37,958] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,959] jepsen worker 62 - jepsen.util 242 :ok :read [20 10] INFO [2022-01-10 12:53:37,959] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:37,959] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:37,960] jepsen worker 75 - jepsen.util 165 :ok :read [25 46] INFO [2022-01-10 12:53:37,960] jepsen worker 1 - jepsen.util 91 :invoke :write [0 18] INFO [2022-01-10 12:53:37,960] jepsen worker 62 - jepsen.util 242 :ok :read [20 10] INFO [2022-01-10 12:53:37,960] jepsen worker 75 - jepsen.util 165 :invoke :write [25 1] INFO [2022-01-10 12:53:37,961] jepsen worker 62 - jepsen.util 242 :invoke :write [20 40] INFO [2022-01-10 12:53:37,961] jepsen worker 52 - jepsen.util 52 :ok :read [17 41] INFO [2022-01-10 12:53:37,961] jepsen worker 12 - jepsen.util 192 :ok :write [4 9] INFO [2022-01-10 12:53:37,962] jepsen worker 12 - jepsen.util 192 :invoke :write [4 15] INFO [2022-01-10 12:53:37,963] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,963] jepsen worker 21 - jepsen.util 201 :ok :write [7 38] INFO [2022-01-10 12:53:37,964] jepsen worker 21 - jepsen.util 201 :invoke :write [7 13] INFO [2022-01-10 12:53:37,966] jepsen worker 52 - jepsen.util 52 :ok :read [17 41] INFO [2022-01-10 12:53:37,966] jepsen worker 52 - jepsen.util 52 :invoke :write [17 39] INFO [2022-01-10 12:53:37,967] jepsen worker 40 - jepsen.util 130 :ok :write [13 12] INFO [2022-01-10 12:53:37,968] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:37,969] jepsen worker 70 - jepsen.util 160 :ok :write [23 47] INFO [2022-01-10 12:53:37,969] jepsen worker 40 - jepsen.util 130 :ok :read [13 12] INFO [2022-01-10 12:53:37,969] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:37,970] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:37,970] jepsen worker 7 - jepsen.util 7 :ok :write [2 36] INFO [2022-01-10 12:53:37,970] jepsen worker 7 - jepsen.util 7 :invoke :write [2 1] INFO [2022-01-10 12:53:37,970] jepsen worker 70 - jepsen.util 160 :ok :read [23 47] INFO [2022-01-10 12:53:37,971] jepsen worker 40 - jepsen.util 130 :ok :read [13 12] INFO [2022-01-10 12:53:37,971] jepsen worker 70 - jepsen.util 160 :invoke :write [23 12] INFO [2022-01-10 12:53:37,972] jepsen worker 40 - jepsen.util 130 :invoke :write [13 6] INFO [2022-01-10 12:53:37,978] jepsen worker 85 - jepsen.util 265 :ok :write [28 3] INFO [2022-01-10 12:53:37,978] jepsen worker 15 - jepsen.util 15 :ok :write [5 36] INFO [2022-01-10 12:53:37,978] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,979] jepsen worker 15 - jepsen.util 15 :invoke :write [5 17] INFO [2022-01-10 12:53:37,979] jepsen worker 85 - jepsen.util 265 :ok :read [28 3] INFO [2022-01-10 12:53:37,980] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,981] jepsen worker 85 - jepsen.util 265 :ok :read [28 3] INFO [2022-01-10 12:53:37,982] jepsen worker 48 - jepsen.util 228 :ok :write [16 23] INFO [2022-01-10 12:53:37,982] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,982] jepsen worker 38 - jepsen.util 38 :ok :write [12 47] INFO [2022-01-10 12:53:37,982] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:37,983] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,983] jepsen worker 85 - jepsen.util 265 :ok :read [28 3] INFO [2022-01-10 12:53:37,983] jepsen worker 48 - jepsen.util 228 :ok :read [16 23] INFO [2022-01-10 12:53:37,984] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,984] jepsen worker 38 - jepsen.util 38 :ok :read [12 47] INFO [2022-01-10 12:53:37,984] jepsen worker 48 - jepsen.util 228 :invoke :write [16 7] INFO [2022-01-10 12:53:37,984] jepsen worker 85 - jepsen.util 265 :ok :read [28 3] INFO [2022-01-10 12:53:37,984] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:37,985] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,985] jepsen worker 38 - jepsen.util 38 :ok :read [12 47] INFO [2022-01-10 12:53:37,986] jepsen worker 85 - jepsen.util 265 :ok :read [28 3] INFO [2022-01-10 12:53:37,986] jepsen worker 38 - jepsen.util 38 :invoke :write [12 0] INFO [2022-01-10 12:53:37,986] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,987] jepsen worker 85 - jepsen.util 265 :ok :read [28 3] INFO [2022-01-10 12:53:37,987] jepsen worker 83 - jepsen.util 263 :ok :write [27 1] INFO [2022-01-10 12:53:37,988] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,988] jepsen worker 83 - jepsen.util 263 :invoke :write [27 0] INFO [2022-01-10 12:53:37,989] jepsen worker 85 - jepsen.util 265 :ok :read [28 3] INFO [2022-01-10 12:53:37,989] jepsen worker 78 - jepsen.util 168 :ok :write [26 26] INFO [2022-01-10 12:53:37,989] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,989] jepsen worker 78 - jepsen.util 168 :invoke :write [26 27] INFO [2022-01-10 12:53:37,991] jepsen worker 85 - jepsen.util 265 :ok :read [28 3] INFO [2022-01-10 12:53:37,991] jepsen worker 33 - jepsen.util 123 :ok :write [11 34] INFO [2022-01-10 12:53:37,991] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:37,991] jepsen worker 89 - jepsen.util 269 :ok :write [29 27] INFO [2022-01-10 12:53:37,991] jepsen worker 33 - jepsen.util 123 :invoke :write [11 6] INFO [2022-01-10 12:53:37,992] jepsen worker 89 - jepsen.util 269 :invoke :write [29 15] INFO [2022-01-10 12:53:37,992] jepsen worker 85 - jepsen.util 265 :ok :read [28 3] INFO [2022-01-10 12:53:37,993] jepsen worker 85 - jepsen.util 265 :invoke :write [28 18] INFO [2022-01-10 12:53:37,995] jepsen worker 52 - jepsen.util 52 :ok :write [17 39] INFO [2022-01-10 12:53:37,996] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:37,996] jepsen worker 44 - jepsen.util 44 :ok :write [14 8] INFO [2022-01-10 12:53:37,997] jepsen worker 52 - jepsen.util 52 :ok :read [17 39] INFO [2022-01-10 12:53:37,997] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:37,997] jepsen worker 52 - jepsen.util 52 :invoke :write [17 1] INFO [2022-01-10 12:53:37,998] jepsen worker 44 - jepsen.util 44 :ok :read [14 8] INFO [2022-01-10 12:53:37,999] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:38,000] jepsen worker 44 - jepsen.util 44 :ok :read [14 8] INFO [2022-01-10 12:53:38,000] jepsen worker 68 - jepsen.util 68 :ok :write [22 7] INFO [2022-01-10 12:53:38,000] jepsen worker 44 - jepsen.util 44 :invoke :write [14 47] INFO [2022-01-10 12:53:38,001] jepsen worker 4 - jepsen.util 94 :ok :write [1 29] INFO [2022-01-10 12:53:38,001] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:38,002] jepsen worker 4 - jepsen.util 94 :invoke :write [1 43] INFO [2022-01-10 12:53:38,002] jepsen worker 68 - jepsen.util 68 :ok :read [22 7] INFO [2022-01-10 12:53:38,002] jepsen worker 68 - jepsen.util 68 :invoke :write [22 47] INFO [2022-01-10 12:53:38,003] jepsen worker 1 - jepsen.util 91 :ok :write [0 18] INFO [2022-01-10 12:53:38,003] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,004] jepsen worker 62 - jepsen.util 242 :ok :write [20 40] INFO [2022-01-10 12:53:38,005] jepsen worker 1 - jepsen.util 91 :ok :read [0 18] INFO [2022-01-10 12:53:38,005] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:38,005] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,006] jepsen worker 1 - jepsen.util 91 :ok :read [0 18] INFO [2022-01-10 12:53:38,006] jepsen worker 62 - jepsen.util 242 :ok :read [20 40] INFO [2022-01-10 12:53:38,007] jepsen worker 12 - jepsen.util 192 :ok :write [4 15] INFO [2022-01-10 12:53:38,007] jepsen worker 62 - jepsen.util 242 :invoke :write [20 34] INFO [2022-01-10 12:53:38,007] jepsen worker 12 - jepsen.util 192 :invoke :write [4 24] INFO [2022-01-10 12:53:38,008] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,009] jepsen worker 1 - jepsen.util 91 :ok :read [0 18] INFO [2022-01-10 12:53:38,010] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,010] jepsen worker 1 - jepsen.util 91 :ok :read [0 18] INFO [2022-01-10 12:53:38,011] jepsen worker 75 - jepsen.util 165 :ok :write [25 1] INFO [2022-01-10 12:53:38,011] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:38,012] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,012] jepsen worker 75 - jepsen.util 165 :ok :read [25 1] INFO [2022-01-10 12:53:38,012] jepsen worker 1 - jepsen.util 91 :ok :read [0 18] INFO [2022-01-10 12:53:38,013] jepsen worker 70 - jepsen.util 160 :ok :write [23 12] INFO [2022-01-10 12:53:38,013] jepsen worker 75 - jepsen.util 165 :invoke :write [25 0] INFO [2022-01-10 12:53:38,014] jepsen worker 1 - jepsen.util 91 :invoke :write [0 17] INFO [2022-01-10 12:53:38,014] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:38,015] jepsen worker 70 - jepsen.util 160 :ok :read [23 12] INFO [2022-01-10 12:53:38,016] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:38,017] jepsen worker 70 - jepsen.util 160 :ok :read [23 12] INFO [2022-01-10 12:53:38,018] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:38,019] jepsen worker 70 - jepsen.util 160 :ok :read [23 12] INFO [2022-01-10 12:53:38,019] jepsen worker 70 - jepsen.util 160 :invoke :write [23 41] INFO [2022-01-10 12:53:38,027] jepsen worker 38 - jepsen.util 38 :ok :write [12 0] INFO [2022-01-10 12:53:38,028] jepsen worker 38 - jepsen.util 38 :invoke :write [12 38] INFO [2022-01-10 12:53:38,030] jepsen worker 21 - jepsen.util 201 :ok :write [7 13] INFO [2022-01-10 12:53:38,031] jepsen worker 21 - jepsen.util 201 :invoke :write [7 46] INFO [2022-01-10 12:53:38,036] jepsen worker 78 - jepsen.util 168 :ok :write [26 27] INFO [2022-01-10 12:53:38,037] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,037] jepsen worker 78 - jepsen.util 168 :ok :read [26 27] INFO [2022-01-10 12:53:38,038] jepsen worker 78 - jepsen.util 168 :invoke :write [26 22] INFO [2022-01-10 12:53:38,043] jepsen worker 52 - jepsen.util 52 :ok :write [17 1] INFO [2022-01-10 12:53:38,043] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,044] jepsen worker 52 - jepsen.util 52 :ok :read [17 1] INFO [2022-01-10 12:53:38,045] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,046] jepsen worker 52 - jepsen.util 52 :ok :read [17 1] INFO [2022-01-10 12:53:38,046] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,047] jepsen worker 52 - jepsen.util 52 :ok :read [17 1] INFO [2022-01-10 12:53:38,048] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,049] jepsen worker 52 - jepsen.util 52 :ok :read [17 1] INFO [2022-01-10 12:53:38,049] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,050] jepsen worker 52 - jepsen.util 52 :ok :read [17 1] INFO [2022-01-10 12:53:38,050] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,051] jepsen worker 52 - jepsen.util 52 :ok :read [17 1] INFO [2022-01-10 12:53:38,052] jepsen worker 52 - jepsen.util 52 :invoke :write [17 8] INFO [2022-01-10 12:53:38,056] jepsen worker 12 - jepsen.util 192 :ok :write [4 24] INFO [2022-01-10 12:53:38,056] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:38,057] jepsen worker 12 - jepsen.util 192 :ok :read [4 24] INFO [2022-01-10 12:53:38,058] jepsen worker 12 - jepsen.util 192 :invoke :write [4 23] INFO [2022-01-10 12:53:38,060] jepsen worker 70 - jepsen.util 160 :ok :write [23 41] INFO [2022-01-10 12:53:38,060] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:38,061] jepsen worker 70 - jepsen.util 160 :ok :read [23 41] INFO [2022-01-10 12:53:38,062] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:38,062] jepsen worker 70 - jepsen.util 160 :ok :read [23 41] INFO [2022-01-10 12:53:38,063] jepsen worker 70 - jepsen.util 160 :invoke :write [23 49] INFO [2022-01-10 12:53:38,068] jepsen worker 44 - jepsen.util 44 :ok :write [14 47] INFO [2022-01-10 12:53:38,068] jepsen worker 78 - jepsen.util 168 :ok :write [26 22] INFO [2022-01-10 12:53:38,069] jepsen worker 44 - jepsen.util 44 :invoke :write [14 24] INFO [2022-01-10 12:53:38,069] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,070] jepsen worker 78 - jepsen.util 168 :ok :read [26 22] INFO [2022-01-10 12:53:38,071] jepsen worker 78 - jepsen.util 168 :invoke :write [26 10] INFO [2022-01-10 12:53:38,081] jepsen worker 52 - jepsen.util 52 :ok :write [17 8] INFO [2022-01-10 12:53:38,081] jepsen worker 62 - jepsen.util 242 :ok :write [20 34] INFO [2022-01-10 12:53:38,082] jepsen worker 52 - jepsen.util 52 :invoke :write [17 32] INFO [2022-01-10 12:53:38,083] jepsen worker 62 - jepsen.util 242 :invoke :write [20 15] INFO [2022-01-10 12:53:38,085] jepsen worker 85 - jepsen.util 265 :ok :write [28 18] INFO [2022-01-10 12:53:38,085] jepsen worker 40 - jepsen.util 130 :ok :write [13 6] INFO [2022-01-10 12:53:38,086] jepsen worker 85 - jepsen.util 265 :invoke :write [28 28] INFO [2022-01-10 12:53:38,086] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,087] jepsen worker 12 - jepsen.util 192 :ok :write [4 23] INFO [2022-01-10 12:53:38,087] jepsen worker 12 - jepsen.util 192 :invoke :write [4 2] INFO [2022-01-10 12:53:38,087] jepsen worker 40 - jepsen.util 130 :ok :read [13 6] INFO [2022-01-10 12:53:38,088] jepsen worker 75 - jepsen.util 165 :ok :write [25 0] INFO [2022-01-10 12:53:38,088] jepsen worker 40 - jepsen.util 130 :invoke :write [13 39] INFO [2022-01-10 12:53:38,088] jepsen worker 75 - jepsen.util 165 :invoke :write [25 6] INFO [2022-01-10 12:53:38,091] jepsen worker 4 - jepsen.util 94 :ok :write [1 43] INFO [2022-01-10 12:53:38,091] jepsen worker 7 - jepsen.util 7 :ok :write [2 1] INFO [2022-01-10 12:53:38,092] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,092] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,093] jepsen worker 4 - jepsen.util 94 :ok :read [1 43] INFO [2022-01-10 12:53:38,093] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:53:38,094] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,094] jepsen worker 7 - jepsen.util 7 :invoke :write [2 21] INFO [2022-01-10 12:53:38,094] jepsen worker 21 - jepsen.util 201 :ok :write [7 46] INFO [2022-01-10 12:53:38,095] jepsen worker 4 - jepsen.util 94 :ok :read [1 43] INFO [2022-01-10 12:53:38,095] jepsen worker 70 - jepsen.util 160 :ok :write [23 49] INFO [2022-01-10 12:53:38,095] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:38,096] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,096] jepsen worker 21 - jepsen.util 201 :ok :read [7 46] INFO [2022-01-10 12:53:38,096] jepsen worker 70 - jepsen.util 160 :invoke :write [23 7] INFO [2022-01-10 12:53:38,096] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:38,097] jepsen worker 4 - jepsen.util 94 :ok :read [1 43] INFO [2022-01-10 12:53:38,097] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,097] jepsen worker 21 - jepsen.util 201 :ok :read [7 46] INFO [2022-01-10 12:53:38,098] jepsen worker 38 - jepsen.util 38 :ok :write [12 38] INFO [2022-01-10 12:53:38,098] jepsen worker 21 - jepsen.util 201 :invoke :write [7 36] INFO [2022-01-10 12:53:38,098] jepsen worker 15 - jepsen.util 15 :ok :write [5 17] INFO [2022-01-10 12:53:38,098] jepsen worker 4 - jepsen.util 94 :ok :read [1 43] INFO [2022-01-10 12:53:38,098] jepsen worker 38 - jepsen.util 38 :invoke :write [12 17] INFO [2022-01-10 12:53:38,099] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:38,099] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,100] jepsen worker 15 - jepsen.util 15 :ok :read [5 17] INFO [2022-01-10 12:53:38,100] jepsen worker 4 - jepsen.util 94 :ok :read [1 43] INFO [2022-01-10 12:53:38,100] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:38,100] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,101] jepsen worker 15 - jepsen.util 15 :ok :read [5 17] INFO [2022-01-10 12:53:38,101] jepsen worker 4 - jepsen.util 94 :ok :read [1 43] INFO [2022-01-10 12:53:38,101] jepsen worker 15 - jepsen.util 15 :invoke :write [5 2] INFO [2022-01-10 12:53:38,102] jepsen worker 4 - jepsen.util 94 :invoke :write [1 27] INFO [2022-01-10 12:53:38,103] jepsen worker 44 - jepsen.util 44 :ok :write [14 24] INFO [2022-01-10 12:53:38,103] jepsen worker 78 - jepsen.util 168 :ok :write [26 10] INFO [2022-01-10 12:53:38,103] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:38,103] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,104] jepsen worker 44 - jepsen.util 44 :ok :read [14 24] INFO [2022-01-10 12:53:38,104] jepsen worker 78 - jepsen.util 168 :ok :read [26 10] INFO [2022-01-10 12:53:38,104] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:38,105] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,105] jepsen worker 50 - jepsen.util 50 :ok :write [16 19] INFO [2022-01-10 12:53:38,105] jepsen worker 44 - jepsen.util 44 :ok :read [14 24] INFO [2022-01-10 12:53:38,105] jepsen worker 50 - jepsen.util 50 :invoke :write [16 22] INFO [2022-01-10 12:53:38,106] jepsen worker 78 - jepsen.util 168 :ok :read [26 10] INFO [2022-01-10 12:53:38,106] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:38,106] jepsen worker 78 - jepsen.util 168 :invoke :write [26 5] INFO [2022-01-10 12:53:38,107] jepsen worker 44 - jepsen.util 44 :ok :read [14 24] INFO [2022-01-10 12:53:38,107] jepsen worker 44 - jepsen.util 44 :invoke :write [14 42] INFO [2022-01-10 12:53:38,107] jepsen worker 62 - jepsen.util 242 :ok :write [20 15] INFO [2022-01-10 12:53:38,108] jepsen worker 62 - jepsen.util 242 :invoke :write [20 9] INFO [2022-01-10 12:53:38,109] jepsen worker 52 - jepsen.util 52 :ok :write [17 32] INFO [2022-01-10 12:53:38,109] jepsen worker 83 - jepsen.util 263 :ok :write [27 0] INFO [2022-01-10 12:53:38,109] jepsen worker 52 - jepsen.util 52 :invoke :write [17 23] INFO [2022-01-10 12:53:38,110] jepsen worker 83 - jepsen.util 263 :invoke :write [27 47] INFO [2022-01-10 12:53:38,113] jepsen worker 75 - jepsen.util 165 :ok :write [25 6] INFO [2022-01-10 12:53:38,113] jepsen worker 12 - jepsen.util 192 :ok :write [4 2] INFO [2022-01-10 12:53:38,114] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:38,114] jepsen worker 12 - jepsen.util 192 :invoke :write [4 43] INFO [2022-01-10 12:53:38,115] jepsen worker 75 - jepsen.util 165 :ok :read [25 6] INFO [2022-01-10 12:53:38,115] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:38,115] jepsen worker 89 - jepsen.util 269 :ok :write [29 15] INFO [2022-01-10 12:53:38,116] jepsen worker 89 - jepsen.util 269 :invoke :write [29 18] INFO [2022-01-10 12:53:38,116] jepsen worker 75 - jepsen.util 165 :ok :read [25 6] INFO [2022-01-10 12:53:38,117] jepsen worker 75 - jepsen.util 165 :invoke :read [25 nil] INFO [2022-01-10 12:53:38,118] jepsen worker 75 - jepsen.util 165 :ok :read [25 6] INFO [2022-01-10 12:53:38,118] jepsen worker 75 - jepsen.util 165 :invoke :write [25 44] INFO [2022-01-10 12:53:38,120] jepsen worker 33 - jepsen.util 123 :ok :write [11 6] INFO [2022-01-10 12:53:38,121] jepsen worker 33 - jepsen.util 123 :invoke :write [11 7] INFO [2022-01-10 12:53:38,123] jepsen worker 68 - jepsen.util 68 :ok :write [22 47] INFO [2022-01-10 12:53:38,124] jepsen worker 68 - jepsen.util 68 :invoke :write [22 39] INFO [2022-01-10 12:53:38,127] jepsen worker 1 - jepsen.util 91 :ok :write [0 17] INFO [2022-01-10 12:53:38,127] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,128] jepsen worker 1 - jepsen.util 91 :ok :read [0 17] INFO [2022-01-10 12:53:38,128] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,129] jepsen worker 1 - jepsen.util 91 :ok :read [0 17] INFO [2022-01-10 12:53:38,129] jepsen worker 1 - jepsen.util 91 :invoke :write [0 9] INFO [2022-01-10 12:53:38,132] jepsen worker 7 - jepsen.util 7 :ok :write [2 21] INFO [2022-01-10 12:53:38,133] jepsen worker 7 - jepsen.util 7 :invoke :write [2 33] INFO [2022-01-10 12:53:38,244] jepsen worker 23 - jepsen.util 23 :info :write [7 22] :RequestTimeoutException INFO [2022-01-10 12:53:38,244] jepsen worker 23 - jepsen.util 113 :invoke :read [7 nil] INFO [2022-01-10 12:53:38,246] jepsen worker 23 - jepsen.util 113 :ok :read [7 46] INFO [2022-01-10 12:53:38,246] jepsen worker 23 - jepsen.util 113 :invoke :write [7 29] INFO [2022-01-10 12:53:38,259] jepsen worker 40 - jepsen.util 130 :ok :write [13 39] INFO [2022-01-10 12:53:38,260] jepsen worker 40 - jepsen.util 130 :invoke :write [13 25] INFO [2022-01-10 12:53:38,260] jepsen worker 15 - jepsen.util 15 :ok :write [5 2] INFO [2022-01-10 12:53:38,261] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:38,261] jepsen worker 85 - jepsen.util 265 :ok :write [28 28] INFO [2022-01-10 12:53:38,262] jepsen worker 15 - jepsen.util 15 :ok :read [5 2] INFO [2022-01-10 12:53:38,262] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:38,262] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:38,263] jepsen worker 85 - jepsen.util 265 :ok :read [28 28] INFO [2022-01-10 12:53:38,263] jepsen worker 15 - jepsen.util 15 :ok :read [5 2] INFO [2022-01-10 12:53:38,263] jepsen worker 85 - jepsen.util 265 :invoke :write [28 8] INFO [2022-01-10 12:53:38,263] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:38,264] jepsen worker 15 - jepsen.util 15 :ok :read [5 2] INFO [2022-01-10 12:53:38,265] jepsen worker 15 - jepsen.util 15 :invoke :write [5 28] INFO [2022-01-10 12:53:38,267] jepsen worker 21 - jepsen.util 201 :ok :write [7 36] INFO [2022-01-10 12:53:38,268] jepsen worker 4 - jepsen.util 94 :ok :write [1 27] INFO [2022-01-10 12:53:38,268] jepsen worker 21 - jepsen.util 201 :invoke :write [7 20] INFO [2022-01-10 12:53:38,268] jepsen worker 4 - jepsen.util 94 :invoke :write [1 14] INFO [2022-01-10 12:53:38,270] jepsen worker 50 - jepsen.util 50 :ok :write [16 22] INFO [2022-01-10 12:53:38,271] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,272] jepsen worker 50 - jepsen.util 50 :ok :read [16 22] INFO [2022-01-10 12:53:38,272] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,273] jepsen worker 44 - jepsen.util 44 :ok :write [14 42] INFO [2022-01-10 12:53:38,273] jepsen worker 50 - jepsen.util 50 :ok :read [16 22] INFO [2022-01-10 12:53:38,273] jepsen worker 44 - jepsen.util 44 :invoke :write [14 46] INFO [2022-01-10 12:53:38,274] jepsen worker 50 - jepsen.util 50 :invoke :write [16 34] INFO [2022-01-10 12:53:38,277] jepsen worker 83 - jepsen.util 263 :ok :write [27 47] INFO [2022-01-10 12:53:38,278] jepsen worker 70 - jepsen.util 160 :ok :write [23 7] INFO [2022-01-10 12:53:38,278] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:38,278] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:38,279] jepsen worker 83 - jepsen.util 263 :ok :read [27 47] INFO [2022-01-10 12:53:38,279] jepsen worker 70 - jepsen.util 160 :ok :read [23 7] INFO [2022-01-10 12:53:38,279] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:38,279] jepsen worker 62 - jepsen.util 242 :ok :write [20 9] INFO [2022-01-10 12:53:38,280] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:38,280] jepsen worker 83 - jepsen.util 263 :ok :read [27 47] INFO [2022-01-10 12:53:38,280] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:38,281] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:38,281] jepsen worker 70 - jepsen.util 160 :ok :read [23 7] INFO [2022-01-10 12:53:38,281] jepsen worker 70 - jepsen.util 160 :invoke :write [23 29] INFO [2022-01-10 12:53:38,281] jepsen worker 83 - jepsen.util 263 :ok :read [27 47] INFO [2022-01-10 12:53:38,281] jepsen worker 62 - jepsen.util 242 :ok :read [20 9] INFO [2022-01-10 12:53:38,282] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:38,282] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:38,283] jepsen worker 62 - jepsen.util 242 :ok :read [20 9] INFO [2022-01-10 12:53:38,283] jepsen worker 83 - jepsen.util 263 :ok :read [27 47] INFO [2022-01-10 12:53:38,283] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:38,284] jepsen worker 83 - jepsen.util 263 :invoke :write [27 30] INFO [2022-01-10 12:53:38,285] jepsen worker 38 - jepsen.util 38 :ok :write [12 17] INFO [2022-01-10 12:53:38,285] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,286] jepsen worker 38 - jepsen.util 38 :ok :read [12 17] INFO [2022-01-10 12:53:38,287] jepsen worker 33 - jepsen.util 123 :ok :write [11 7] INFO [2022-01-10 12:53:38,287] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,287] jepsen worker 62 - jepsen.util 242 :ok :read [20 9] INFO [2022-01-10 12:53:38,287] jepsen worker 75 - jepsen.util 165 :ok :write [25 44] INFO [2022-01-10 12:53:38,287] jepsen worker 33 - jepsen.util 123 :invoke :write [11 11] INFO [2022-01-10 12:53:38,288] jepsen worker 62 - jepsen.util 242 :invoke :write [20 6] INFO [2022-01-10 12:53:38,288] jepsen worker 38 - jepsen.util 38 :ok :read [12 17] INFO [2022-01-10 12:53:38,288] jepsen worker 75 - jepsen.util 165 :invoke :write [25 11] INFO [2022-01-10 12:53:38,289] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,290] jepsen worker 38 - jepsen.util 38 :ok :read [12 17] INFO [2022-01-10 12:53:38,290] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,291] jepsen worker 78 - jepsen.util 168 :ok :write [26 5] INFO [2022-01-10 12:53:38,291] jepsen worker 89 - jepsen.util 269 :ok :write [29 18] INFO [2022-01-10 12:53:38,291] jepsen worker 38 - jepsen.util 38 :ok :read [12 17] INFO [2022-01-10 12:53:38,291] jepsen worker 78 - jepsen.util 168 :invoke :write [26 4] INFO [2022-01-10 12:53:38,292] jepsen worker 89 - jepsen.util 269 :invoke :write [29 24] INFO [2022-01-10 12:53:38,292] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,293] jepsen worker 38 - jepsen.util 38 :ok :read [12 17] INFO [2022-01-10 12:53:38,294] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,295] jepsen worker 68 - jepsen.util 68 :ok :write [22 39] INFO [2022-01-10 12:53:38,295] jepsen worker 38 - jepsen.util 38 :ok :read [12 17] INFO [2022-01-10 12:53:38,296] jepsen worker 38 - jepsen.util 38 :invoke :write [12 35] INFO [2022-01-10 12:53:38,296] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:38,297] jepsen worker 68 - jepsen.util 68 :ok :read [22 39] INFO [2022-01-10 12:53:38,297] jepsen worker 68 - jepsen.util 68 :invoke :write [22 23] INFO [2022-01-10 12:53:38,298] jepsen worker 1 - jepsen.util 91 :ok :write [0 9] INFO [2022-01-10 12:53:38,298] jepsen worker 52 - jepsen.util 52 :ok :write [17 23] INFO [2022-01-10 12:53:38,299] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,300] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,300] jepsen worker 52 - jepsen.util 52 :ok :read [17 23] INFO [2022-01-10 12:53:38,301] jepsen worker 52 - jepsen.util 52 :invoke :write [17 33] INFO [2022-01-10 12:53:38,301] jepsen worker 1 - jepsen.util 91 :ok :read [0 9] INFO [2022-01-10 12:53:38,301] jepsen worker 7 - jepsen.util 7 :ok :write [2 33] INFO [2022-01-10 12:53:38,302] jepsen worker 1 - jepsen.util 91 :invoke :write [0 39] INFO [2022-01-10 12:53:38,303] jepsen worker 7 - jepsen.util 7 :invoke :write [2 19] INFO [2022-01-10 12:53:38,305] jepsen worker 12 - jepsen.util 192 :ok :write [4 43] INFO [2022-01-10 12:53:38,305] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:38,306] jepsen worker 40 - jepsen.util 130 :ok :write [13 25] INFO [2022-01-10 12:53:38,306] jepsen worker 12 - jepsen.util 192 :ok :read [4 43] INFO [2022-01-10 12:53:38,307] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:38,307] jepsen worker 15 - jepsen.util 15 :ok :write [5 28] INFO [2022-01-10 12:53:38,307] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,308] jepsen worker 12 - jepsen.util 192 :ok :read [4 43] INFO [2022-01-10 12:53:38,308] jepsen worker 15 - jepsen.util 15 :invoke :write [5 44] INFO [2022-01-10 12:53:38,308] jepsen worker 40 - jepsen.util 130 :ok :read [13 25] INFO [2022-01-10 12:53:38,309] jepsen worker 12 - jepsen.util 192 :invoke :write [4 49] INFO [2022-01-10 12:53:38,309] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,310] jepsen worker 40 - jepsen.util 130 :ok :read [13 25] INFO [2022-01-10 12:53:38,311] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,312] jepsen worker 40 - jepsen.util 130 :ok :read [13 25] INFO [2022-01-10 12:53:38,312] jepsen worker 85 - jepsen.util 265 :ok :write [28 8] INFO [2022-01-10 12:53:38,312] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,313] jepsen worker 21 - jepsen.util 201 :ok :write [7 20] INFO [2022-01-10 12:53:38,313] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:38,313] jepsen worker 21 - jepsen.util 201 :invoke :write [7 12] INFO [2022-01-10 12:53:38,314] jepsen worker 85 - jepsen.util 265 :ok :read [28 8] INFO [2022-01-10 12:53:38,314] jepsen worker 40 - jepsen.util 130 :ok :read [13 25] INFO [2022-01-10 12:53:38,315] jepsen worker 85 - jepsen.util 265 :invoke :write [28 16] INFO [2022-01-10 12:53:38,315] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,316] jepsen worker 40 - jepsen.util 130 :ok :read [13 25] INFO [2022-01-10 12:53:38,316] jepsen worker 4 - jepsen.util 94 :ok :write [1 14] INFO [2022-01-10 12:53:38,317] jepsen worker 40 - jepsen.util 130 :invoke :write [13 8] INFO [2022-01-10 12:53:38,317] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,318] jepsen worker 4 - jepsen.util 94 :ok :read [1 14] INFO [2022-01-10 12:53:38,319] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,319] jepsen worker 44 - jepsen.util 44 :ok :write [14 46] INFO [2022-01-10 12:53:38,320] jepsen worker 4 - jepsen.util 94 :ok :read [1 14] INFO [2022-01-10 12:53:38,320] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:38,320] jepsen worker 4 - jepsen.util 94 :invoke :write [1 41] INFO [2022-01-10 12:53:38,321] jepsen worker 44 - jepsen.util 44 :ok :read [14 46] INFO [2022-01-10 12:53:38,321] jepsen worker 44 - jepsen.util 44 :invoke :write [14 32] INFO [2022-01-10 12:53:38,325] jepsen worker 89 - jepsen.util 269 :ok :write [29 24] INFO [2022-01-10 12:53:38,325] jepsen worker 70 - jepsen.util 160 :ok :write [23 29] INFO [2022-01-10 12:53:38,325] jepsen worker 70 - jepsen.util 160 :invoke :write [23 44] INFO [2022-01-10 12:53:38,326] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,326] jepsen worker 50 - jepsen.util 50 :ok :write [16 34] INFO [2022-01-10 12:53:38,327] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,327] jepsen worker 89 - jepsen.util 269 :ok :read [29 24] INFO [2022-01-10 12:53:38,328] jepsen worker 89 - jepsen.util 269 :invoke :write [29 24] INFO [2022-01-10 12:53:38,328] jepsen worker 50 - jepsen.util 50 :ok :read [16 34] INFO [2022-01-10 12:53:38,329] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,330] jepsen worker 50 - jepsen.util 50 :ok :read [16 34] INFO [2022-01-10 12:53:38,330] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,331] jepsen worker 38 - jepsen.util 38 :ok :write [12 35] INFO [2022-01-10 12:53:38,331] jepsen worker 62 - jepsen.util 242 :ok :write [20 6] INFO [2022-01-10 12:53:38,331] jepsen worker 50 - jepsen.util 50 :ok :read [16 34] INFO [2022-01-10 12:53:38,331] jepsen worker 38 - jepsen.util 38 :invoke :write [12 19] INFO [2022-01-10 12:53:38,332] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,332] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:38,333] jepsen worker 50 - jepsen.util 50 :ok :read [16 34] INFO [2022-01-10 12:53:38,333] jepsen worker 62 - jepsen.util 242 :ok :read [20 6] INFO [2022-01-10 12:53:38,334] jepsen worker 50 - jepsen.util 50 :invoke :write [16 41] INFO [2022-01-10 12:53:38,334] jepsen worker 62 - jepsen.util 242 :invoke :write [20 29] INFO [2022-01-10 12:53:38,338] jepsen worker 83 - jepsen.util 263 :ok :write [27 30] INFO [2022-01-10 12:53:38,338] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:38,339] jepsen worker 83 - jepsen.util 263 :ok :read [27 30] INFO [2022-01-10 12:53:38,340] jepsen worker 83 - jepsen.util 263 :invoke :write [27 3] INFO [2022-01-10 12:53:38,341] jepsen worker 1 - jepsen.util 91 :ok :write [0 39] INFO [2022-01-10 12:53:38,341] jepsen worker 78 - jepsen.util 168 :ok :write [26 4] INFO [2022-01-10 12:53:38,341] jepsen worker 76 - jepsen.util 256 :ok :write [25 46] INFO [2022-01-10 12:53:38,342] jepsen worker 76 - jepsen.util 256 :invoke :write [25 7] INFO [2022-01-10 12:53:38,342] jepsen worker 1 - jepsen.util 91 :invoke :write [0 25] INFO [2022-01-10 12:53:38,343] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,343] jepsen worker 78 - jepsen.util 168 :ok :read [26 4] INFO [2022-01-10 12:53:38,344] jepsen worker 41 - jepsen.util 41 :ok :write [13 0] INFO [2022-01-10 12:53:38,344] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,344] jepsen worker 41 - jepsen.util 41 :invoke :write [13 34] INFO [2022-01-10 12:53:38,345] jepsen worker 78 - jepsen.util 168 :ok :read [26 4] INFO [2022-01-10 12:53:38,346] jepsen worker 78 - jepsen.util 168 :invoke :write [26 29] INFO [2022-01-10 12:53:38,346] jepsen worker 33 - jepsen.util 123 :ok :write [11 11] INFO [2022-01-10 12:53:38,347] jepsen worker 33 - jepsen.util 123 :invoke :write [11 17] INFO [2022-01-10 12:53:38,348] jepsen worker 8 - jepsen.util 8 :info :write [2 4] :RequestTimeoutException INFO [2022-01-10 12:53:38,349] jepsen worker 8 - jepsen.util 98 :invoke :write [2 3] INFO [2022-01-10 12:53:38,351] jepsen worker 52 - jepsen.util 52 :ok :write [17 33] INFO [2022-01-10 12:53:38,351] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,352] jepsen worker 52 - jepsen.util 52 :ok :read [17 33] INFO [2022-01-10 12:53:38,353] jepsen worker 68 - jepsen.util 68 :ok :write [22 23] INFO [2022-01-10 12:53:38,353] jepsen worker 52 - jepsen.util 52 :invoke :write [17 34] INFO [2022-01-10 12:53:38,354] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:38,355] jepsen worker 68 - jepsen.util 68 :ok :read [22 23] INFO [2022-01-10 12:53:38,356] jepsen worker 68 - jepsen.util 68 :invoke :write [22 16] INFO [2022-01-10 12:53:38,357] jepsen worker 7 - jepsen.util 7 :ok :write [2 19] INFO [2022-01-10 12:53:38,358] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,358] jepsen worker 12 - jepsen.util 192 :ok :write [4 49] INFO [2022-01-10 12:53:38,358] jepsen worker 89 - jepsen.util 269 :ok :write [29 24] INFO [2022-01-10 12:53:38,359] jepsen worker 7 - jepsen.util 7 :ok :read [2 19] INFO [2022-01-10 12:53:38,359] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:38,359] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,360] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,360] jepsen worker 12 - jepsen.util 192 :ok :read [4 49] INFO [2022-01-10 12:53:38,360] jepsen worker 89 - jepsen.util 269 :ok :read [29 24] INFO [2022-01-10 12:53:38,361] jepsen worker 7 - jepsen.util 7 :ok :read [2 19] INFO [2022-01-10 12:53:38,361] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,361] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:38,362] jepsen worker 89 - jepsen.util 269 :ok :read [29 24] INFO [2022-01-10 12:53:38,362] jepsen worker 7 - jepsen.util 7 :invoke :write [2 0] INFO [2022-01-10 12:53:38,362] jepsen worker 12 - jepsen.util 192 :ok :read [4 49] INFO [2022-01-10 12:53:38,363] jepsen worker 89 - jepsen.util 269 :invoke :write [29 49] INFO [2022-01-10 12:53:38,363] jepsen worker 12 - jepsen.util 192 :invoke :write [4 38] INFO [2022-01-10 12:53:38,366] jepsen worker 85 - jepsen.util 265 :ok :write [28 16] INFO [2022-01-10 12:53:38,366] jepsen worker 15 - jepsen.util 15 :ok :write [5 44] INFO [2022-01-10 12:53:38,367] jepsen worker 15 - jepsen.util 15 :invoke :write [5 38] INFO [2022-01-10 12:53:38,367] jepsen worker 85 - jepsen.util 265 :invoke :write [28 4] INFO [2022-01-10 12:53:38,371] jepsen worker 21 - jepsen.util 201 :ok :write [7 12] INFO [2022-01-10 12:53:38,371] jepsen worker 1 - jepsen.util 91 :ok :write [0 25] INFO [2022-01-10 12:53:38,372] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:38,372] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,372] jepsen worker 21 - jepsen.util 201 :ok :read [7 12] INFO [2022-01-10 12:53:38,373] jepsen worker 1 - jepsen.util 91 :ok :read [0 25] INFO [2022-01-10 12:53:38,373] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:38,374] jepsen worker 1 - jepsen.util 91 :invoke :write [0 21] INFO [2022-01-10 12:53:38,374] jepsen worker 21 - jepsen.util 201 :ok :read [7 12] INFO [2022-01-10 12:53:38,375] jepsen worker 21 - jepsen.util 201 :invoke :write [7 43] INFO [2022-01-10 12:53:38,375] jepsen worker 44 - jepsen.util 44 :ok :write [14 32] INFO [2022-01-10 12:53:38,376] jepsen worker 44 - jepsen.util 44 :invoke :write [14 24] INFO [2022-01-10 12:53:38,377] jepsen worker 41 - jepsen.util 41 :ok :write [13 34] INFO [2022-01-10 12:53:38,377] jepsen worker 41 - jepsen.util 41 :invoke :write [13 21] INFO [2022-01-10 12:53:38,378] jepsen worker 4 - jepsen.util 94 :ok :write [1 41] INFO [2022-01-10 12:53:38,379] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,380] jepsen worker 4 - jepsen.util 94 :ok :read [1 41] INFO [2022-01-10 12:53:38,380] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,381] jepsen worker 4 - jepsen.util 94 :ok :read [1 41] INFO [2022-01-10 12:53:38,381] jepsen worker 4 - jepsen.util 94 :invoke :write [1 21] INFO [2022-01-10 12:53:38,383] jepsen worker 50 - jepsen.util 50 :ok :write [16 41] INFO [2022-01-10 12:53:38,383] jepsen worker 62 - jepsen.util 242 :ok :write [20 29] INFO [2022-01-10 12:53:38,384] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,384] jepsen worker 62 - jepsen.util 242 :invoke :write [20 33] INFO [2022-01-10 12:53:38,385] jepsen worker 50 - jepsen.util 50 :ok :read [16 41] INFO [2022-01-10 12:53:38,385] jepsen worker 70 - jepsen.util 160 :ok :write [23 44] INFO [2022-01-10 12:53:38,386] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,387] jepsen worker 50 - jepsen.util 50 :ok :read [16 41] INFO [2022-01-10 12:53:38,387] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:38,388] jepsen worker 50 - jepsen.util 50 :invoke :write [16 13] INFO [2022-01-10 12:53:38,388] jepsen worker 83 - jepsen.util 263 :ok :write [27 3] INFO [2022-01-10 12:53:38,388] jepsen worker 70 - jepsen.util 160 :ok :read [23 44] INFO [2022-01-10 12:53:38,389] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:38,389] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:38,390] jepsen worker 83 - jepsen.util 263 :ok :read [27 3] INFO [2022-01-10 12:53:38,390] jepsen worker 70 - jepsen.util 160 :ok :read [23 44] INFO [2022-01-10 12:53:38,390] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:38,391] jepsen worker 76 - jepsen.util 256 :ok :write [25 7] INFO [2022-01-10 12:53:38,391] jepsen worker 70 - jepsen.util 160 :invoke :write [23 35] INFO [2022-01-10 12:53:38,391] jepsen worker 83 - jepsen.util 263 :ok :read [27 3] INFO [2022-01-10 12:53:38,392] jepsen worker 76 - jepsen.util 256 :invoke :write [25 8] INFO [2022-01-10 12:53:38,392] jepsen worker 83 - jepsen.util 263 :invoke :write [27 24] INFO [2022-01-10 12:53:38,394] jepsen worker 38 - jepsen.util 38 :ok :write [12 19] INFO [2022-01-10 12:53:38,394] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,395] jepsen worker 89 - jepsen.util 269 :ok :write [29 49] INFO [2022-01-10 12:53:38,396] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,397] jepsen worker 38 - jepsen.util 38 :ok :read [12 19] INFO [2022-01-10 12:53:38,397] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,398] jepsen worker 33 - jepsen.util 123 :ok :write [11 17] INFO [2022-01-10 12:53:38,398] jepsen worker 89 - jepsen.util 269 :ok :read [29 49] INFO [2022-01-10 12:53:38,398] jepsen worker 38 - jepsen.util 38 :ok :read [12 19] INFO [2022-01-10 12:53:38,399] jepsen worker 33 - jepsen.util 123 :invoke :write [11 31] INFO [2022-01-10 12:53:38,399] jepsen worker 78 - jepsen.util 168 :ok :write [26 29] INFO [2022-01-10 12:53:38,400] jepsen worker 89 - jepsen.util 269 :invoke :write [29 16] INFO [2022-01-10 12:53:38,401] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,401] jepsen worker 38 - jepsen.util 38 :invoke :write [12 40] INFO [2022-01-10 12:53:38,401] jepsen worker 78 - jepsen.util 168 :ok :read [26 29] INFO [2022-01-10 12:53:38,402] jepsen worker 78 - jepsen.util 168 :invoke :write [26 32] INFO [2022-01-10 12:53:38,404] jepsen worker 1 - jepsen.util 91 :ok :write [0 21] INFO [2022-01-10 12:53:38,404] jepsen worker 1 - jepsen.util 91 :invoke :write [0 29] INFO [2022-01-10 12:53:38,405] jepsen worker 68 - jepsen.util 68 :ok :write [22 16] INFO [2022-01-10 12:53:38,406] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:38,406] jepsen worker 52 - jepsen.util 52 :ok :write [17 34] INFO [2022-01-10 12:53:38,407] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:38,407] jepsen worker 7 - jepsen.util 7 :ok :write [2 0] INFO [2022-01-10 12:53:38,408] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,408] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,409] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:38,409] jepsen worker 52 - jepsen.util 52 :ok :read [17 34] INFO [2022-01-10 12:53:38,409] jepsen worker 7 - jepsen.util 7 :ok :read [2 0] INFO [2022-01-10 12:53:38,409] jepsen worker 52 - jepsen.util 52 :invoke :write [17 42] INFO [2022-01-10 12:53:38,409] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:38,410] jepsen worker 39 - jepsen.util 129 :ok :write [13 15] INFO [2022-01-10 12:53:38,410] jepsen worker 7 - jepsen.util 7 :invoke :write [2 25] INFO [2022-01-10 12:53:38,411] jepsen worker 39 - jepsen.util 129 :invoke :write [13 37] INFO [2022-01-10 12:53:38,412] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:38,412] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:38,413] jepsen worker 68 - jepsen.util 68 :invoke :write [22 23] INFO [2022-01-10 12:53:38,413] jepsen worker 15 - jepsen.util 15 :ok :write [5 38] INFO [2022-01-10 12:53:38,414] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:38,414] jepsen worker 12 - jepsen.util 192 :ok :write [4 38] INFO [2022-01-10 12:53:38,415] jepsen worker 15 - jepsen.util 15 :ok :read [5 38] INFO [2022-01-10 12:53:38,415] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:38,415] jepsen worker 15 - jepsen.util 15 :invoke :write [5 39] INFO [2022-01-10 12:53:38,416] jepsen worker 12 - jepsen.util 192 :ok :read [4 38] INFO [2022-01-10 12:53:38,416] jepsen worker 12 - jepsen.util 192 :invoke :write [4 11] INFO [2022-01-10 12:53:38,417] jepsen worker 21 - jepsen.util 201 :ok :write [7 43] INFO [2022-01-10 12:53:38,418] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:38,419] jepsen worker 21 - jepsen.util 201 :ok :read [7 43] INFO [2022-01-10 12:53:38,419] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:38,420] jepsen worker 21 - jepsen.util 201 :ok :read [7 43] INFO [2022-01-10 12:53:38,421] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:38,422] jepsen worker 21 - jepsen.util 201 :ok :read [7 43] INFO [2022-01-10 12:53:38,422] jepsen worker 84 - jepsen.util 84 :ok :write [28 35] INFO [2022-01-10 12:53:38,422] jepsen worker 44 - jepsen.util 44 :ok :write [14 24] INFO [2022-01-10 12:53:38,422] jepsen worker 21 - jepsen.util 201 :invoke :write [7 25] INFO [2022-01-10 12:53:38,423] jepsen worker 84 - jepsen.util 84 :invoke :write [28 27] INFO [2022-01-10 12:53:38,423] jepsen worker 44 - jepsen.util 44 :invoke :write [14 20] INFO [2022-01-10 12:53:38,427] jepsen worker 4 - jepsen.util 94 :ok :write [1 21] INFO [2022-01-10 12:53:38,427] jepsen worker 4 - jepsen.util 94 :invoke :write [1 19] INFO [2022-01-10 12:53:38,428] jepsen worker 62 - jepsen.util 242 :ok :write [20 33] INFO [2022-01-10 12:53:38,428] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:38,429] jepsen worker 62 - jepsen.util 242 :ok :read [20 33] INFO [2022-01-10 12:53:38,430] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:38,431] jepsen worker 62 - jepsen.util 242 :ok :read [20 33] INFO [2022-01-10 12:53:38,431] jepsen worker 62 - jepsen.util 242 :invoke :write [20 31] INFO [2022-01-10 12:53:38,432] jepsen worker 50 - jepsen.util 50 :ok :write [16 13] INFO [2022-01-10 12:53:38,433] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,434] jepsen worker 50 - jepsen.util 50 :ok :read [16 13] INFO [2022-01-10 12:53:38,434] jepsen worker 70 - jepsen.util 160 :ok :write [23 35] INFO [2022-01-10 12:53:38,434] jepsen worker 50 - jepsen.util 50 :invoke :write [16 13] INFO [2022-01-10 12:53:38,435] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:38,436] jepsen worker 83 - jepsen.util 263 :ok :write [27 24] INFO [2022-01-10 12:53:38,436] jepsen worker 70 - jepsen.util 160 :ok :read [23 35] INFO [2022-01-10 12:53:38,436] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:38,437] jepsen worker 70 - jepsen.util 160 :invoke :write [23 11] INFO [2022-01-10 12:53:38,437] jepsen worker 83 - jepsen.util 263 :ok :read [27 24] INFO [2022-01-10 12:53:38,438] jepsen worker 83 - jepsen.util 263 :invoke :write [27 26] INFO [2022-01-10 12:53:38,439] jepsen worker 76 - jepsen.util 256 :ok :write [25 8] INFO [2022-01-10 12:53:38,439] jepsen worker 76 - jepsen.util 256 :invoke :write [25 26] INFO [2022-01-10 12:53:38,440] jepsen worker 38 - jepsen.util 38 :ok :write [12 40] INFO [2022-01-10 12:53:38,441] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,442] jepsen worker 38 - jepsen.util 38 :ok :read [12 40] INFO [2022-01-10 12:53:38,442] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,443] jepsen worker 38 - jepsen.util 38 :ok :read [12 40] INFO [2022-01-10 12:53:38,443] jepsen worker 38 - jepsen.util 38 :invoke :write [12 21] INFO [2022-01-10 12:53:38,445] jepsen worker 78 - jepsen.util 168 :ok :write [26 32] INFO [2022-01-10 12:53:38,446] jepsen worker 78 - jepsen.util 168 :invoke :write [26 8] INFO [2022-01-10 12:53:38,449] jepsen worker 52 - jepsen.util 52 :ok :write [17 42] INFO [2022-01-10 12:53:38,450] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,450] jepsen worker 52 - jepsen.util 52 :ok :read [17 42] INFO [2022-01-10 12:53:38,451] jepsen worker 52 - jepsen.util 52 :invoke :write [17 33] INFO [2022-01-10 12:53:38,453] jepsen worker 12 - jepsen.util 192 :ok :write [4 11] INFO [2022-01-10 12:53:38,453] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:38,454] jepsen worker 12 - jepsen.util 192 :ok :read [4 11] INFO [2022-01-10 12:53:38,455] jepsen worker 12 - jepsen.util 192 :invoke :write [4 24] INFO [2022-01-10 12:53:38,462] jepsen worker 21 - jepsen.util 201 :ok :write [7 25] INFO [2022-01-10 12:53:38,462] jepsen worker 21 - jepsen.util 201 :invoke :write [7 13] INFO [2022-01-10 12:53:38,466] jepsen worker 70 - jepsen.util 160 :ok :write [23 11] INFO [2022-01-10 12:53:38,466] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:38,467] jepsen worker 44 - jepsen.util 44 :ok :write [14 20] INFO [2022-01-10 12:53:38,467] jepsen worker 70 - jepsen.util 160 :ok :read [23 11] INFO [2022-01-10 12:53:38,467] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:38,467] jepsen worker 70 - jepsen.util 160 :invoke :write [23 42] INFO [2022-01-10 12:53:38,468] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:38,469] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:38,469] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:38,470] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:38,471] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:38,471] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:38,472] jepsen worker 62 - jepsen.util 242 :ok :write [20 31] INFO [2022-01-10 12:53:38,472] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:38,472] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:38,472] jepsen worker 44 - jepsen.util 44 :invoke :write [14 23] INFO [2022-01-10 12:53:38,473] jepsen worker 62 - jepsen.util 242 :ok :read [20 31] INFO [2022-01-10 12:53:38,473] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:38,474] jepsen worker 62 - jepsen.util 242 :ok :read [20 31] INFO [2022-01-10 12:53:38,475] jepsen worker 62 - jepsen.util 242 :invoke :write [20 8] INFO [2022-01-10 12:53:38,476] jepsen worker 78 - jepsen.util 168 :ok :write [26 8] INFO [2022-01-10 12:53:38,476] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,477] jepsen worker 78 - jepsen.util 168 :ok :read [26 8] INFO [2022-01-10 12:53:38,477] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,478] jepsen worker 78 - jepsen.util 168 :ok :read [26 8] INFO [2022-01-10 12:53:38,479] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,479] jepsen worker 78 - jepsen.util 168 :ok :read [26 8] INFO [2022-01-10 12:53:38,480] jepsen worker 52 - jepsen.util 52 :ok :write [17 33] INFO [2022-01-10 12:53:38,480] jepsen worker 78 - jepsen.util 168 :invoke :write [26 14] INFO [2022-01-10 12:53:38,480] jepsen worker 52 - jepsen.util 52 :invoke :write [17 44] INFO [2022-01-10 12:53:38,510] jepsen worker 89 - jepsen.util 269 :ok :write [29 16] INFO [2022-01-10 12:53:38,511] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,512] jepsen worker 89 - jepsen.util 269 :ok :read [29 16] INFO [2022-01-10 12:53:38,512] jepsen worker 89 - jepsen.util 269 :invoke :write [29 24] INFO [2022-01-10 12:53:38,514] jepsen worker 33 - jepsen.util 123 :ok :write [11 31] INFO [2022-01-10 12:53:38,515] jepsen worker 33 - jepsen.util 123 :invoke :write [11 0] INFO [2022-01-10 12:53:38,517] jepsen worker 84 - jepsen.util 84 :ok :write [28 27] INFO [2022-01-10 12:53:38,517] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:38,518] jepsen worker 1 - jepsen.util 91 :ok :write [0 29] INFO [2022-01-10 12:53:38,518] jepsen worker 84 - jepsen.util 84 :ok :read [28 27] INFO [2022-01-10 12:53:38,519] jepsen worker 84 - jepsen.util 84 :invoke :write [28 43] INFO [2022-01-10 12:53:38,519] jepsen worker 1 - jepsen.util 91 :invoke :write [0 41] INFO [2022-01-10 12:53:38,524] jepsen worker 68 - jepsen.util 68 :ok :write [22 23] INFO [2022-01-10 12:53:38,524] jepsen worker 4 - jepsen.util 94 :ok :write [1 19] INFO [2022-01-10 12:53:38,524] jepsen worker 68 - jepsen.util 68 :invoke :write [22 35] INFO [2022-01-10 12:53:38,525] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,526] jepsen worker 4 - jepsen.util 94 :ok :read [1 19] INFO [2022-01-10 12:53:38,526] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,527] jepsen worker 4 - jepsen.util 94 :ok :read [1 19] INFO [2022-01-10 12:53:38,527] jepsen worker 4 - jepsen.util 94 :invoke :write [1 38] INFO [2022-01-10 12:53:38,529] jepsen worker 38 - jepsen.util 38 :ok :write [12 21] INFO [2022-01-10 12:53:38,529] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,530] jepsen worker 38 - jepsen.util 38 :ok :read [12 21] INFO [2022-01-10 12:53:38,531] jepsen worker 7 - jepsen.util 7 :ok :write [2 25] INFO [2022-01-10 12:53:38,531] jepsen worker 38 - jepsen.util 38 :invoke :write [12 28] INFO [2022-01-10 12:53:38,531] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,532] jepsen worker 7 - jepsen.util 7 :ok :read [2 25] INFO [2022-01-10 12:53:38,532] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,533] jepsen worker 7 - jepsen.util 7 :ok :read [2 25] INFO [2022-01-10 12:53:38,533] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,535] jepsen worker 7 - jepsen.util 7 :ok :read [2 25] INFO [2022-01-10 12:53:38,535] jepsen worker 39 - jepsen.util 129 :ok :write [13 37] INFO [2022-01-10 12:53:38,535] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,535] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,536] jepsen worker 7 - jepsen.util 7 :ok :read [2 25] INFO [2022-01-10 12:53:38,536] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,536] jepsen worker 39 - jepsen.util 129 :ok :read [13 37] INFO [2022-01-10 12:53:38,537] jepsen worker 39 - jepsen.util 129 :invoke :write [13 29] INFO [2022-01-10 12:53:38,537] jepsen worker 7 - jepsen.util 7 :ok :read [2 25] INFO [2022-01-10 12:53:38,537] jepsen worker 7 - jepsen.util 7 :invoke :write [2 46] INFO [2022-01-10 12:53:38,541] jepsen worker 15 - jepsen.util 15 :ok :write [5 39] INFO [2022-01-10 12:53:38,541] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:38,542] jepsen worker 15 - jepsen.util 15 :ok :read [5 39] INFO [2022-01-10 12:53:38,543] jepsen worker 15 - jepsen.util 15 :invoke :write [5 9] INFO [2022-01-10 12:53:38,572] jepsen worker 16 - jepsen.util 196 :info :write [5 40] :RequestTimeoutException INFO [2022-01-10 12:53:38,573] jepsen worker 16 - jepsen.util 286 :invoke :write [5 21] INFO [2022-01-10 12:53:38,592] jepsen worker 50 - jepsen.util 50 :ok :write [16 13] INFO [2022-01-10 12:53:38,593] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,594] jepsen worker 50 - jepsen.util 50 :ok :read [16 13] INFO [2022-01-10 12:53:38,594] jepsen worker 50 - jepsen.util 50 :invoke :write [16 19] INFO [2022-01-10 12:53:38,598] jepsen worker 49 - jepsen.util 229 :info :write [16 33] :RequestTimeoutException INFO [2022-01-10 12:53:38,599] jepsen worker 83 - jepsen.util 263 :ok :write [27 26] INFO [2022-01-10 12:53:38,599] jepsen worker 49 - jepsen.util 319 :invoke :write [16 11] INFO [2022-01-10 12:53:38,599] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:38,600] jepsen worker 83 - jepsen.util 263 :ok :read [27 26] INFO [2022-01-10 12:53:38,601] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:38,601] jepsen worker 83 - jepsen.util 263 :ok :read [27 26] INFO [2022-01-10 12:53:38,602] jepsen worker 83 - jepsen.util 263 :invoke :write [27 16] INFO [2022-01-10 12:53:38,605] jepsen worker 76 - jepsen.util 256 :ok :write [25 26] INFO [2022-01-10 12:53:38,605] jepsen worker 76 - jepsen.util 256 :invoke :write [25 30] INFO [2022-01-10 12:53:38,611] jepsen worker 21 - jepsen.util 201 :ok :write [7 13] INFO [2022-01-10 12:53:38,611] jepsen worker 21 - jepsen.util 201 :invoke :write [7 21] INFO [2022-01-10 12:53:38,613] jepsen worker 12 - jepsen.util 192 :ok :write [4 24] INFO [2022-01-10 12:53:38,613] jepsen worker 12 - jepsen.util 192 :invoke :write [4 28] INFO [2022-01-10 12:53:38,618] jepsen worker 44 - jepsen.util 44 :ok :write [14 23] INFO [2022-01-10 12:53:38,618] jepsen worker 44 - jepsen.util 44 :invoke :write [14 8] INFO [2022-01-10 12:53:38,618] jepsen worker 70 - jepsen.util 160 :ok :write [23 42] INFO [2022-01-10 12:53:38,619] jepsen worker 70 - jepsen.util 160 :invoke :write [23 30] INFO [2022-01-10 12:53:38,624] jepsen worker 78 - jepsen.util 168 :ok :write [26 14] INFO [2022-01-10 12:53:38,624] jepsen worker 89 - jepsen.util 269 :ok :write [29 24] INFO [2022-01-10 12:53:38,625] jepsen worker 62 - jepsen.util 242 :ok :write [20 8] INFO [2022-01-10 12:53:38,625] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,626] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,626] jepsen worker 62 - jepsen.util 242 :invoke :write [20 49] INFO [2022-01-10 12:53:38,627] jepsen worker 89 - jepsen.util 269 :ok :read [29 24] INFO [2022-01-10 12:53:38,627] jepsen worker 78 - jepsen.util 168 :ok :read [26 14] INFO [2022-01-10 12:53:38,628] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,628] jepsen worker 89 - jepsen.util 269 :invoke :write [29 40] INFO [2022-01-10 12:53:38,629] jepsen worker 78 - jepsen.util 168 :ok :read [26 14] INFO [2022-01-10 12:53:38,629] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,630] jepsen worker 78 - jepsen.util 168 :ok :read [26 14] INFO [2022-01-10 12:53:38,631] jepsen worker 52 - jepsen.util 52 :ok :write [17 44] INFO [2022-01-10 12:53:38,631] jepsen worker 1 - jepsen.util 91 :ok :write [0 41] INFO [2022-01-10 12:53:38,631] jepsen worker 78 - jepsen.util 168 :invoke :write [26 22] INFO [2022-01-10 12:53:38,632] jepsen worker 33 - jepsen.util 123 :ok :write [11 0] INFO [2022-01-10 12:53:38,632] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,633] jepsen worker 52 - jepsen.util 52 :invoke :write [17 42] INFO [2022-01-10 12:53:38,633] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:38,633] jepsen worker 1 - jepsen.util 91 :ok :read [0 41] INFO [2022-01-10 12:53:38,634] jepsen worker 33 - jepsen.util 123 :ok :read [11 0] INFO [2022-01-10 12:53:38,635] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:38,635] jepsen worker 33 - jepsen.util 123 :ok :read [11 0] INFO [2022-01-10 12:53:38,636] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,636] jepsen worker 33 - jepsen.util 123 :invoke :write [11 46] INFO [2022-01-10 12:53:38,637] jepsen worker 39 - jepsen.util 129 :ok :write [13 29] INFO [2022-01-10 12:53:38,637] jepsen worker 39 - jepsen.util 129 :invoke :write [13 32] INFO [2022-01-10 12:53:38,639] jepsen worker 84 - jepsen.util 84 :ok :write [28 43] INFO [2022-01-10 12:53:38,639] jepsen worker 68 - jepsen.util 68 :ok :write [22 35] INFO [2022-01-10 12:53:38,639] jepsen worker 1 - jepsen.util 91 :ok :read [0 41] INFO [2022-01-10 12:53:38,640] jepsen worker 84 - jepsen.util 84 :invoke :write [28 23] INFO [2022-01-10 12:53:38,640] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,641] jepsen worker 68 - jepsen.util 68 :invoke :write [22 39] INFO [2022-01-10 12:53:38,641] jepsen worker 4 - jepsen.util 94 :ok :write [1 38] INFO [2022-01-10 12:53:38,642] jepsen worker 1 - jepsen.util 91 :ok :read [0 41] INFO [2022-01-10 12:53:38,642] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,643] jepsen worker 4 - jepsen.util 94 :ok :read [1 38] INFO [2022-01-10 12:53:38,643] jepsen worker 1 - jepsen.util 91 :invoke :write [0 31] INFO [2022-01-10 12:53:38,643] jepsen worker 4 - jepsen.util 94 :invoke :write [1 33] INFO [2022-01-10 12:53:38,645] jepsen worker 8 - jepsen.util 98 :ok :write [2 3] INFO [2022-01-10 12:53:38,646] jepsen worker 8 - jepsen.util 98 :invoke :write [2 4] INFO [2022-01-10 12:53:38,648] jepsen worker 38 - jepsen.util 38 :ok :write [12 28] INFO [2022-01-10 12:53:38,648] jepsen worker 38 - jepsen.util 38 :invoke :write [12 3] INFO [2022-01-10 12:53:38,654] jepsen worker 12 - jepsen.util 192 :ok :write [4 28] INFO [2022-01-10 12:53:38,655] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:38,655] jepsen worker 15 - jepsen.util 15 :ok :write [5 9] INFO [2022-01-10 12:53:38,656] jepsen worker 12 - jepsen.util 192 :ok :read [4 28] INFO [2022-01-10 12:53:38,656] jepsen worker 15 - jepsen.util 15 :invoke :write [5 15] INFO [2022-01-10 12:53:38,656] jepsen worker 12 - jepsen.util 192 :invoke :write [4 47] INFO [2022-01-10 12:53:38,659] jepsen worker 89 - jepsen.util 269 :ok :write [29 40] INFO [2022-01-10 12:53:38,660] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,661] jepsen worker 89 - jepsen.util 269 :ok :read [29 40] INFO [2022-01-10 12:53:38,661] jepsen worker 70 - jepsen.util 160 :ok :write [23 30] INFO [2022-01-10 12:53:38,664] jepsen worker 45 - jepsen.util 225 :info :write [15 7] :SyncWriteAmbiguous INFO [2022-01-10 12:53:38,664] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,664] jepsen worker 70 - jepsen.util 160 :invoke :write [23 8] INFO [2022-01-10 12:53:38,665] jepsen worker 89 - jepsen.util 269 :ok :read [29 40] INFO [2022-01-10 12:53:38,665] jepsen worker 45 - jepsen.util 315 :invoke :write [15 44] INFO [2022-01-10 12:53:38,665] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,666] jepsen worker 89 - jepsen.util 269 :ok :read [29 40] INFO [2022-01-10 12:53:38,667] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,668] jepsen worker 89 - jepsen.util 269 :ok :read [29 40] INFO [2022-01-10 12:53:38,668] jepsen worker 89 - jepsen.util 269 :invoke :write [29 47] INFO [2022-01-10 12:53:38,670] jepsen worker 1 - jepsen.util 91 :ok :write [0 31] INFO [2022-01-10 12:53:38,671] jepsen worker 78 - jepsen.util 168 :ok :write [26 22] INFO [2022-01-10 12:53:38,671] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,672] jepsen worker 78 - jepsen.util 168 :invoke :write [26 20] INFO [2022-01-10 12:53:38,672] jepsen worker 1 - jepsen.util 91 :ok :read [0 31] INFO [2022-01-10 12:53:38,673] jepsen worker 1 - jepsen.util 91 :invoke :write [0 40] INFO [2022-01-10 12:53:38,675] jepsen worker 39 - jepsen.util 129 :ok :write [13 32] INFO [2022-01-10 12:53:38,675] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,676] jepsen worker 39 - jepsen.util 129 :ok :read [13 32] INFO [2022-01-10 12:53:38,676] jepsen worker 52 - jepsen.util 52 :ok :write [17 42] INFO [2022-01-10 12:53:38,677] jepsen worker 39 - jepsen.util 129 :invoke :write [13 42] INFO [2022-01-10 12:53:38,677] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,678] jepsen worker 52 - jepsen.util 52 :ok :read [17 42] INFO [2022-01-10 12:53:38,679] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,680] jepsen worker 52 - jepsen.util 52 :ok :read [17 42] INFO [2022-01-10 12:53:38,680] jepsen worker 52 - jepsen.util 52 :invoke :write [17 23] INFO [2022-01-10 12:53:38,682] jepsen worker 89 - jepsen.util 269 :ok :write [29 47] INFO [2022-01-10 12:53:38,682] jepsen worker 89 - jepsen.util 269 :invoke :write [29 30] INFO [2022-01-10 12:53:38,684] jepsen worker 84 - jepsen.util 84 :ok :write [28 23] INFO [2022-01-10 12:53:38,685] jepsen worker 84 - jepsen.util 84 :invoke :write [28 11] INFO [2022-01-10 12:53:38,689] jepsen worker 4 - jepsen.util 94 :ok :write [1 33] INFO [2022-01-10 12:53:38,690] jepsen worker 4 - jepsen.util 94 :invoke :write [1 44] INFO [2022-01-10 12:53:38,691] jepsen worker 1 - jepsen.util 91 :ok :write [0 40] INFO [2022-01-10 12:53:38,691] jepsen worker 1 - jepsen.util 91 :invoke :write [0 22] INFO [2022-01-10 12:53:38,696] jepsen worker 38 - jepsen.util 38 :ok :write [12 3] INFO [2022-01-10 12:53:38,696] jepsen worker 39 - jepsen.util 129 :ok :write [13 42] INFO [2022-01-10 12:53:38,697] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,697] jepsen worker 38 - jepsen.util 38 :invoke :write [12 18] INFO [2022-01-10 12:53:38,698] jepsen worker 39 - jepsen.util 129 :ok :read [13 42] INFO [2022-01-10 12:53:38,699] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,699] jepsen worker 39 - jepsen.util 129 :ok :read [13 42] INFO [2022-01-10 12:53:38,700] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,701] jepsen worker 39 - jepsen.util 129 :ok :read [13 42] INFO [2022-01-10 12:53:38,702] jepsen worker 39 - jepsen.util 129 :invoke :write [13 8] INFO [2022-01-10 12:53:38,703] jepsen worker 89 - jepsen.util 269 :ok :write [29 30] INFO [2022-01-10 12:53:38,704] jepsen worker 89 - jepsen.util 269 :invoke :write [29 14] INFO [2022-01-10 12:53:38,711] jepsen worker 1 - jepsen.util 91 :ok :write [0 22] INFO [2022-01-10 12:53:38,711] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,712] jepsen worker 1 - jepsen.util 91 :ok :read [0 22] INFO [2022-01-10 12:53:38,712] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,713] jepsen worker 1 - jepsen.util 91 :ok :read [0 22] INFO [2022-01-10 12:53:38,714] jepsen worker 1 - jepsen.util 91 :invoke :write [0 47] INFO [2022-01-10 12:53:38,720] jepsen worker 39 - jepsen.util 129 :ok :write [13 8] INFO [2022-01-10 12:53:38,720] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,721] jepsen worker 39 - jepsen.util 129 :ok :read [13 8] INFO [2022-01-10 12:53:38,721] jepsen worker 39 - jepsen.util 129 :invoke :write [13 42] INFO [2022-01-10 12:53:38,722] jepsen worker 89 - jepsen.util 269 :ok :write [29 14] INFO [2022-01-10 12:53:38,723] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,724] jepsen worker 89 - jepsen.util 269 :ok :read [29 14] INFO [2022-01-10 12:53:38,725] jepsen worker 89 - jepsen.util 269 :invoke :write [29 11] INFO [2022-01-10 12:53:38,726] jepsen worker 84 - jepsen.util 84 :ok :write [28 11] INFO [2022-01-10 12:53:38,726] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:38,727] jepsen worker 84 - jepsen.util 84 :ok :read [28 11] INFO [2022-01-10 12:53:38,728] jepsen worker 84 - jepsen.util 84 :invoke :write [28 19] INFO [2022-01-10 12:53:38,731] jepsen worker 1 - jepsen.util 91 :ok :write [0 47] INFO [2022-01-10 12:53:38,731] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,732] jepsen worker 1 - jepsen.util 91 :ok :read [0 47] INFO [2022-01-10 12:53:38,732] jepsen worker 1 - jepsen.util 91 :invoke :write [0 0] INFO [2022-01-10 12:53:38,733] jepsen worker 4 - jepsen.util 94 :ok :write [1 44] INFO [2022-01-10 12:53:38,734] jepsen worker 4 - jepsen.util 94 :invoke :write [1 21] INFO [2022-01-10 12:53:38,735] jepsen worker 39 - jepsen.util 129 :ok :write [13 42] INFO [2022-01-10 12:53:38,736] jepsen worker 39 - jepsen.util 129 :invoke :write [13 33] INFO [2022-01-10 12:53:38,741] jepsen worker 89 - jepsen.util 269 :ok :write [29 11] INFO [2022-01-10 12:53:38,742] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,743] jepsen worker 89 - jepsen.util 269 :ok :read [29 11] INFO [2022-01-10 12:53:38,743] jepsen worker 38 - jepsen.util 38 :ok :write [12 18] INFO [2022-01-10 12:53:38,743] jepsen worker 89 - jepsen.util 269 :invoke :write [29 39] INFO [2022-01-10 12:53:38,744] jepsen worker 38 - jepsen.util 38 :invoke :write [12 49] INFO [2022-01-10 12:53:38,749] jepsen worker 1 - jepsen.util 91 :ok :write [0 0] INFO [2022-01-10 12:53:38,749] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,751] jepsen worker 1 - jepsen.util 91 :ok :read [0 0] INFO [2022-01-10 12:53:38,751] jepsen worker 1 - jepsen.util 91 :invoke :write [0 1] INFO [2022-01-10 12:53:38,751] jepsen worker 84 - jepsen.util 84 :ok :write [28 19] INFO [2022-01-10 12:53:38,752] jepsen worker 84 - jepsen.util 84 :invoke :write [28 7] INFO [2022-01-10 12:53:38,754] jepsen worker 4 - jepsen.util 94 :ok :write [1 21] INFO [2022-01-10 12:53:38,755] jepsen worker 39 - jepsen.util 129 :ok :write [13 33] INFO [2022-01-10 12:53:38,755] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,755] jepsen worker 39 - jepsen.util 129 :invoke :write [13 48] INFO [2022-01-10 12:53:38,756] jepsen worker 4 - jepsen.util 94 :ok :read [1 21] INFO [2022-01-10 12:53:38,756] jepsen worker 4 - jepsen.util 94 :invoke :write [1 23] INFO [2022-01-10 12:53:38,759] jepsen worker 89 - jepsen.util 269 :ok :write [29 39] INFO [2022-01-10 12:53:38,760] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,761] jepsen worker 89 - jepsen.util 269 :ok :read [29 39] INFO [2022-01-10 12:53:38,761] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,762] jepsen worker 89 - jepsen.util 269 :ok :read [29 39] INFO [2022-01-10 12:53:38,762] jepsen worker 38 - jepsen.util 38 :ok :write [12 49] INFO [2022-01-10 12:53:38,763] jepsen worker 89 - jepsen.util 269 :invoke :write [29 20] INFO [2022-01-10 12:53:38,763] jepsen worker 38 - jepsen.util 38 :invoke :write [12 16] INFO [2022-01-10 12:53:38,768] jepsen worker 4 - jepsen.util 94 :ok :write [1 23] INFO [2022-01-10 12:53:38,768] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,769] jepsen worker 4 - jepsen.util 94 :ok :read [1 23] INFO [2022-01-10 12:53:38,769] jepsen worker 4 - jepsen.util 94 :invoke :write [1 7] INFO [2022-01-10 12:53:38,773] jepsen worker 84 - jepsen.util 84 :ok :write [28 7] INFO [2022-01-10 12:53:38,773] jepsen worker 1 - jepsen.util 91 :ok :write [0 1] INFO [2022-01-10 12:53:38,773] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:38,774] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,774] jepsen worker 84 - jepsen.util 84 :ok :read [28 7] INFO [2022-01-10 12:53:38,775] jepsen worker 1 - jepsen.util 91 :ok :read [0 1] INFO [2022-01-10 12:53:38,775] jepsen worker 84 - jepsen.util 84 :invoke :write [28 26] INFO [2022-01-10 12:53:38,775] jepsen worker 1 - jepsen.util 91 :invoke :write [0 32] INFO [2022-01-10 12:53:38,792] jepsen worker 50 - jepsen.util 50 :ok :write [16 19] INFO [2022-01-10 12:53:38,792] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,793] jepsen worker 50 - jepsen.util 50 :ok :read [16 19] INFO [2022-01-10 12:53:38,793] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,794] jepsen worker 50 - jepsen.util 50 :ok :read [16 19] INFO [2022-01-10 12:53:38,795] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:38,796] jepsen worker 50 - jepsen.util 50 :ok :read [16 19] INFO [2022-01-10 12:53:38,796] jepsen worker 50 - jepsen.util 50 :invoke :write [16 20] INFO [2022-01-10 12:53:38,797] jepsen worker 12 - jepsen.util 192 :ok :write [4 47] INFO [2022-01-10 12:53:38,798] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:38,798] jepsen worker 12 - jepsen.util 192 :ok :read [4 47] INFO [2022-01-10 12:53:38,799] jepsen worker 12 - jepsen.util 192 :invoke :write [4 12] INFO [2022-01-10 12:53:38,799] jepsen worker 81 - jepsen.util 171 :ok :write [27 6] INFO [2022-01-10 12:53:38,800] jepsen worker 81 - jepsen.util 171 :invoke :write [27 3] INFO [2022-01-10 12:53:38,812] jepsen worker 42 - jepsen.util 132 :info :write [14 6] :RequestTimeoutException INFO [2022-01-10 12:53:38,813] jepsen worker 42 - jepsen.util 222 :invoke :write [14 20] INFO [2022-01-10 12:53:38,838] jepsen worker 27 - jepsen.util 297 :info :write [9 5] :SyncWriteAmbiguous INFO [2022-01-10 12:53:38,839] jepsen worker 27 - jepsen.util 387 :invoke :write [9 24] INFO [2022-01-10 12:53:38,907] jepsen worker 39 - jepsen.util 129 :ok :write [13 48] INFO [2022-01-10 12:53:38,908] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:38,909] jepsen worker 39 - jepsen.util 129 :ok :read [13 48] INFO [2022-01-10 12:53:38,910] jepsen worker 39 - jepsen.util 129 :invoke :write [13 19] INFO [2022-01-10 12:53:38,914] jepsen worker 89 - jepsen.util 269 :ok :write [29 20] INFO [2022-01-10 12:53:38,915] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,917] jepsen worker 89 - jepsen.util 269 :ok :read [29 20] INFO [2022-01-10 12:53:38,917] jepsen worker 76 - jepsen.util 256 :ok :write [25 30] INFO [2022-01-10 12:53:38,918] jepsen worker 1 - jepsen.util 91 :ok :write [0 32] INFO [2022-01-10 12:53:38,918] jepsen worker 76 - jepsen.util 256 :invoke :write [25 17] INFO [2022-01-10 12:53:38,919] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:38,920] jepsen worker 89 - jepsen.util 269 :ok :read [29 20] INFO [2022-01-10 12:53:38,920] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:38,921] jepsen worker 1 - jepsen.util 91 :ok :read [0 32] INFO [2022-01-10 12:53:38,921] jepsen worker 89 - jepsen.util 269 :invoke :write [29 48] INFO [2022-01-10 12:53:38,921] jepsen worker 38 - jepsen.util 38 :ok :write [12 16] INFO [2022-01-10 12:53:38,922] jepsen worker 1 - jepsen.util 91 :invoke :write [0 6] INFO [2022-01-10 12:53:38,923] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,924] jepsen worker 4 - jepsen.util 94 :ok :write [1 7] INFO [2022-01-10 12:53:38,924] jepsen worker 21 - jepsen.util 201 :ok :write [7 21] INFO [2022-01-10 12:53:38,925] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:38,926] jepsen worker 4 - jepsen.util 94 :invoke :write [1 33] INFO [2022-01-10 12:53:38,926] jepsen worker 21 - jepsen.util 201 :ok :read [7 21] INFO [2022-01-10 12:53:38,927] jepsen worker 21 - jepsen.util 201 :invoke :write [7 1] INFO [2022-01-10 12:53:38,930] jepsen worker 38 - jepsen.util 38 :ok :read [12 16] INFO [2022-01-10 12:53:38,930] jepsen worker 38 - jepsen.util 38 :invoke :write [12 38] INFO [2022-01-10 12:53:38,932] jepsen worker 44 - jepsen.util 44 :ok :write [14 8] INFO [2022-01-10 12:53:38,932] jepsen worker 70 - jepsen.util 160 :ok :write [23 8] INFO [2022-01-10 12:53:38,933] jepsen worker 44 - jepsen.util 44 :invoke :write [14 33] INFO [2022-01-10 12:53:38,933] jepsen worker 84 - jepsen.util 84 :ok :write [28 26] INFO [2022-01-10 12:53:38,934] jepsen worker 70 - jepsen.util 160 :invoke :write [23 30] INFO [2022-01-10 12:53:38,935] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:38,936] jepsen worker 84 - jepsen.util 84 :ok :read [28 26] INFO [2022-01-10 12:53:38,937] jepsen worker 84 - jepsen.util 84 :invoke :write [28 2] INFO [2022-01-10 12:53:38,940] jepsen worker 62 - jepsen.util 242 :ok :write [20 49] INFO [2022-01-10 12:53:38,940] jepsen worker 78 - jepsen.util 168 :ok :write [26 20] INFO [2022-01-10 12:53:38,941] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:38,941] jepsen worker 62 - jepsen.util 242 :ok :read [20 49] INFO [2022-01-10 12:53:38,942] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:38,942] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:38,943] jepsen worker 78 - jepsen.util 168 :ok :read [26 20] INFO [2022-01-10 12:53:38,943] jepsen worker 62 - jepsen.util 242 :ok :read [20 49] INFO [2022-01-10 12:53:38,944] jepsen worker 78 - jepsen.util 168 :invoke :write [26 34] INFO [2022-01-10 12:53:38,944] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:38,945] jepsen worker 62 - jepsen.util 242 :ok :read [20 49] INFO [2022-01-10 12:53:38,945] jepsen worker 39 - jepsen.util 129 :ok :write [13 19] INFO [2022-01-10 12:53:38,946] jepsen worker 4 - jepsen.util 94 :ok :write [1 33] INFO [2022-01-10 12:53:38,946] jepsen worker 39 - jepsen.util 129 :invoke :write [13 41] INFO [2022-01-10 12:53:38,946] jepsen worker 52 - jepsen.util 52 :ok :write [17 23] INFO [2022-01-10 12:53:38,947] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,947] jepsen worker 33 - jepsen.util 123 :ok :write [11 46] INFO [2022-01-10 12:53:38,947] jepsen worker 4 - jepsen.util 94 :ok :read [1 33] INFO [2022-01-10 12:53:38,948] jepsen worker 62 - jepsen.util 242 :invoke :write [20 35] INFO [2022-01-10 12:53:38,949] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,949] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:38,950] jepsen worker 52 - jepsen.util 52 :ok :read [17 23] INFO [2022-01-10 12:53:38,950] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:38,950] jepsen worker 33 - jepsen.util 123 :ok :read [11 46] INFO [2022-01-10 12:53:38,951] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,951] jepsen worker 4 - jepsen.util 94 :ok :read [1 33] INFO [2022-01-10 12:53:38,952] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:38,952] jepsen worker 52 - jepsen.util 52 :ok :read [17 23] INFO [2022-01-10 12:53:38,952] jepsen worker 33 - jepsen.util 123 :ok :read [11 46] INFO [2022-01-10 12:53:38,953] jepsen worker 4 - jepsen.util 94 :invoke :write [1 25] INFO [2022-01-10 12:53:38,953] jepsen worker 89 - jepsen.util 269 :ok :write [29 48] INFO [2022-01-10 12:53:38,953] jepsen worker 33 - jepsen.util 123 :invoke :write [11 48] INFO [2022-01-10 12:53:38,954] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:38,955] jepsen worker 68 - jepsen.util 68 :ok :write [22 39] INFO [2022-01-10 12:53:38,955] jepsen worker 52 - jepsen.util 52 :ok :read [17 23] INFO [2022-01-10 12:53:38,955] jepsen worker 89 - jepsen.util 269 :invoke :write [29 41] INFO [2022-01-10 12:53:38,956] jepsen worker 12 - jepsen.util 192 :ok :write [4 12] INFO [2022-01-10 12:53:38,956] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:38,956] jepsen worker 1 - jepsen.util 91 :ok :write [0 6] INFO [2022-01-10 12:53:38,957] jepsen worker 68 - jepsen.util 68 :ok :read [22 39] INFO [2022-01-10 12:53:38,957] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:38,958] jepsen worker 1 - jepsen.util 91 :invoke :write [0 20] INFO [2022-01-10 12:53:38,958] jepsen worker 12 - jepsen.util 192 :ok :read [4 12] INFO [2022-01-10 12:53:38,959] jepsen worker 52 - jepsen.util 52 :invoke :write [17 4] INFO [2022-01-10 12:53:38,960] jepsen worker 12 - jepsen.util 192 :invoke :write [4 3] INFO [2022-01-10 12:53:38,960] jepsen worker 6 - jepsen.util 276 :ok :write [2 42] INFO [2022-01-10 12:53:38,960] jepsen worker 68 - jepsen.util 68 :invoke :write [22 5] INFO [2022-01-10 12:53:38,961] jepsen worker 6 - jepsen.util 276 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,962] jepsen worker 6 - jepsen.util 276 :ok :read [2 42] INFO [2022-01-10 12:53:38,962] jepsen worker 38 - jepsen.util 38 :ok :write [12 38] INFO [2022-01-10 12:53:38,963] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,964] jepsen worker 6 - jepsen.util 276 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,964] jepsen worker 38 - jepsen.util 38 :ok :read [12 38] INFO [2022-01-10 12:53:38,964] jepsen worker 6 - jepsen.util 276 :ok :read [2 42] INFO [2022-01-10 12:53:38,965] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:38,965] jepsen worker 6 - jepsen.util 276 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,966] jepsen worker 38 - jepsen.util 38 :ok :read [12 38] INFO [2022-01-10 12:53:38,966] jepsen worker 6 - jepsen.util 276 :ok :read [2 42] INFO [2022-01-10 12:53:38,967] jepsen worker 6 - jepsen.util 276 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,968] jepsen worker 15 - jepsen.util 15 :ok :write [5 15] INFO [2022-01-10 12:53:38,968] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:38,968] jepsen worker 6 - jepsen.util 276 :ok :read [2 42] INFO [2022-01-10 12:53:38,968] jepsen worker 15 - jepsen.util 15 :invoke :write [5 24] INFO [2022-01-10 12:53:38,969] jepsen worker 6 - jepsen.util 276 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,970] jepsen worker 6 - jepsen.util 276 :ok :read [2 42] INFO [2022-01-10 12:53:38,971] jepsen worker 6 - jepsen.util 276 :invoke :read [2 nil] INFO [2022-01-10 12:53:38,971] jepsen worker 6 - jepsen.util 276 :ok :read [2 42] INFO [2022-01-10 12:53:38,972] jepsen worker 6 - jepsen.util 276 :invoke :write [2 26] INFO [2022-01-10 12:53:38,972] jepsen worker 50 - jepsen.util 50 :ok :write [16 20] INFO [2022-01-10 12:53:38,973] jepsen worker 50 - jepsen.util 50 :invoke :write [16 11] INFO [2022-01-10 12:53:38,979] jepsen worker 81 - jepsen.util 171 :ok :write [27 3] INFO [2022-01-10 12:53:38,980] jepsen worker 81 - jepsen.util 171 :invoke :write [27 4] INFO [2022-01-10 12:53:38,984] jepsen worker 76 - jepsen.util 256 :ok :write [25 17] INFO [2022-01-10 12:53:38,984] jepsen worker 76 - jepsen.util 256 :invoke :write [25 44] INFO [2022-01-10 12:53:39,002] jepsen worker 33 - jepsen.util 123 :ok :write [11 48] INFO [2022-01-10 12:53:39,003] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,003] jepsen worker 33 - jepsen.util 123 :ok :read [11 48] INFO [2022-01-10 12:53:39,004] jepsen worker 33 - jepsen.util 123 :invoke :write [11 24] INFO [2022-01-10 12:53:39,007] jepsen worker 68 - jepsen.util 68 :ok :write [22 5] INFO [2022-01-10 12:53:39,007] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:39,008] jepsen worker 68 - jepsen.util 68 :ok :read [22 5] INFO [2022-01-10 12:53:39,009] jepsen worker 68 - jepsen.util 68 :invoke :write [22 0] INFO [2022-01-10 12:53:39,010] jepsen worker 6 - jepsen.util 276 :ok :write [2 26] INFO [2022-01-10 12:53:39,011] jepsen worker 6 - jepsen.util 276 :invoke :write [2 12] INFO [2022-01-10 12:53:39,015] jepsen worker 15 - jepsen.util 15 :ok :write [5 24] INFO [2022-01-10 12:53:39,016] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,017] jepsen worker 15 - jepsen.util 15 :ok :read [5 24] INFO [2022-01-10 12:53:39,017] jepsen worker 15 - jepsen.util 15 :invoke :write [5 38] INFO [2022-01-10 12:53:39,019] jepsen worker 50 - jepsen.util 50 :ok :write [16 11] INFO [2022-01-10 12:53:39,020] jepsen worker 50 - jepsen.util 50 :invoke :write [16 15] INFO [2022-01-10 12:53:39,022] jepsen worker 81 - jepsen.util 171 :ok :write [27 4] INFO [2022-01-10 12:53:39,023] jepsen worker 81 - jepsen.util 171 :invoke :write [27 33] INFO [2022-01-10 12:53:39,030] jepsen worker 33 - jepsen.util 123 :ok :write [11 24] INFO [2022-01-10 12:53:39,031] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,032] jepsen worker 33 - jepsen.util 123 :ok :read [11 24] INFO [2022-01-10 12:53:39,032] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,033] jepsen worker 33 - jepsen.util 123 :ok :read [11 24] INFO [2022-01-10 12:53:39,034] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,035] jepsen worker 33 - jepsen.util 123 :ok :read [11 24] INFO [2022-01-10 12:53:39,035] jepsen worker 68 - jepsen.util 68 :ok :write [22 0] INFO [2022-01-10 12:53:39,036] jepsen worker 33 - jepsen.util 123 :invoke :write [11 7] INFO [2022-01-10 12:53:39,036] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:39,037] jepsen worker 68 - jepsen.util 68 :ok :read [22 0] INFO [2022-01-10 12:53:39,038] jepsen worker 68 - jepsen.util 68 :invoke :write [22 24] INFO [2022-01-10 12:53:39,039] jepsen worker 6 - jepsen.util 276 :ok :write [2 12] INFO [2022-01-10 12:53:39,039] jepsen worker 6 - jepsen.util 276 :invoke :read [2 nil] INFO [2022-01-10 12:53:39,040] jepsen worker 6 - jepsen.util 276 :ok :read [2 12] INFO [2022-01-10 12:53:39,040] jepsen worker 6 - jepsen.util 276 :invoke :read [2 nil] INFO [2022-01-10 12:53:39,041] jepsen worker 6 - jepsen.util 276 :ok :read [2 12] INFO [2022-01-10 12:53:39,042] jepsen worker 6 - jepsen.util 276 :invoke :write [2 18] INFO [2022-01-10 12:53:39,044] jepsen worker 15 - jepsen.util 15 :ok :write [5 38] INFO [2022-01-10 12:53:39,044] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,045] jepsen worker 15 - jepsen.util 15 :ok :read [5 38] INFO [2022-01-10 12:53:39,046] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,047] jepsen worker 15 - jepsen.util 15 :ok :read [5 38] INFO [2022-01-10 12:53:39,047] jepsen worker 15 - jepsen.util 15 :invoke :write [5 30] INFO [2022-01-10 12:53:39,048] jepsen worker 50 - jepsen.util 50 :ok :write [16 15] INFO [2022-01-10 12:53:39,049] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,050] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:39,050] jepsen worker 21 - jepsen.util 201 :ok :write [7 1] INFO [2022-01-10 12:53:39,050] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,051] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:39,051] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,052] jepsen worker 21 - jepsen.util 201 :ok :read [7 1] INFO [2022-01-10 12:53:39,052] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,053] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,053] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:39,053] jepsen worker 21 - jepsen.util 201 :ok :read [7 1] INFO [2022-01-10 12:53:39,054] jepsen worker 70 - jepsen.util 160 :ok :write [23 30] INFO [2022-01-10 12:53:39,054] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,054] jepsen worker 81 - jepsen.util 171 :ok :write [27 33] INFO [2022-01-10 12:53:39,055] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,055] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:39,056] jepsen worker 21 - jepsen.util 201 :ok :read [7 1] INFO [2022-01-10 12:53:39,056] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,057] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,057] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:39,057] jepsen worker 70 - jepsen.util 160 :invoke :write [23 4] INFO [2022-01-10 12:53:39,058] jepsen worker 21 - jepsen.util 201 :ok :read [7 1] INFO [2022-01-10 12:53:39,058] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:39,059] jepsen worker 81 - jepsen.util 171 :ok :read [27 33] INFO [2022-01-10 12:53:39,059] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,060] jepsen worker 50 - jepsen.util 50 :invoke :write [16 10] INFO [2022-01-10 12:53:39,060] jepsen worker 21 - jepsen.util 201 :ok :read [7 1] INFO [2022-01-10 12:53:39,061] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:39,061] jepsen worker 81 - jepsen.util 171 :ok :read [27 33] INFO [2022-01-10 12:53:39,061] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,062] jepsen worker 33 - jepsen.util 123 :ok :write [11 7] INFO [2022-01-10 12:53:39,062] jepsen worker 44 - jepsen.util 44 :ok :write [14 33] INFO [2022-01-10 12:53:39,062] jepsen worker 62 - jepsen.util 242 :ok :write [20 35] INFO [2022-01-10 12:53:39,062] jepsen worker 85 - jepsen.util 265 :ok :write [28 4] INFO [2022-01-10 12:53:39,062] jepsen worker 21 - jepsen.util 201 :ok :read [7 1] INFO [2022-01-10 12:53:39,063] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:39,064] jepsen worker 81 - jepsen.util 171 :ok :read [27 33] INFO [2022-01-10 12:53:39,065] jepsen worker 44 - jepsen.util 44 :invoke :write [14 47] INFO [2022-01-10 12:53:39,065] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:39,066] jepsen worker 78 - jepsen.util 168 :ok :write [26 34] INFO [2022-01-10 12:53:39,066] jepsen worker 85 - jepsen.util 265 :ok :read [28 4] INFO [2022-01-10 12:53:39,067] jepsen worker 33 - jepsen.util 123 :invoke :write [11 3] INFO [2022-01-10 12:53:39,068] jepsen worker 39 - jepsen.util 129 :ok :write [13 41] INFO [2022-01-10 12:53:39,068] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:39,069] jepsen worker 81 - jepsen.util 171 :ok :read [27 33] INFO [2022-01-10 12:53:39,069] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,070] jepsen worker 21 - jepsen.util 201 :ok :read [7 1] INFO [2022-01-10 12:53:39,070] jepsen worker 68 - jepsen.util 68 :ok :write [22 24] INFO [2022-01-10 12:53:39,071] jepsen worker 78 - jepsen.util 168 :invoke :write [26 11] INFO [2022-01-10 12:53:39,072] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,073] jepsen worker 21 - jepsen.util 201 :ok :read [7 1] INFO [2022-01-10 12:53:39,073] jepsen worker 89 - jepsen.util 269 :ok :write [29 41] INFO [2022-01-10 12:53:39,073] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,074] jepsen worker 62 - jepsen.util 242 :ok :read [20 35] INFO [2022-01-10 12:53:39,075] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:39,075] jepsen worker 52 - jepsen.util 52 :ok :write [17 4] INFO [2022-01-10 12:53:39,076] jepsen worker 89 - jepsen.util 269 :ok :read [29 41] INFO [2022-01-10 12:53:39,076] jepsen worker 21 - jepsen.util 201 :invoke :write [7 31] INFO [2022-01-10 12:53:39,077] jepsen worker 6 - jepsen.util 276 :ok :write [2 18] INFO [2022-01-10 12:53:39,077] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,078] jepsen worker 62 - jepsen.util 242 :ok :read [20 35] INFO [2022-01-10 12:53:39,079] jepsen worker 6 - jepsen.util 276 :invoke :read [2 nil] INFO [2022-01-10 12:53:39,080] jepsen worker 6 - jepsen.util 276 :ok :read [2 18] INFO [2022-01-10 12:53:39,080] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:39,080] jepsen worker 4 - jepsen.util 94 :ok :write [1 25] INFO [2022-01-10 12:53:39,081] jepsen worker 1 - jepsen.util 91 :ok :write [0 20] INFO [2022-01-10 12:53:39,081] jepsen worker 81 - jepsen.util 171 :ok :read [27 33] INFO [2022-01-10 12:53:39,082] jepsen worker 85 - jepsen.util 265 :invoke :write [28 6] INFO [2022-01-10 12:53:39,084] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:39,085] jepsen worker 15 - jepsen.util 15 :ok :write [5 30] INFO [2022-01-10 12:53:39,085] jepsen worker 52 - jepsen.util 52 :ok :read [17 4] INFO [2022-01-10 12:53:39,086] jepsen worker 12 - jepsen.util 192 :ok :write [4 3] INFO [2022-01-10 12:53:39,087] jepsen worker 4 - jepsen.util 94 :invoke :write [1 26] INFO [2022-01-10 12:53:39,087] jepsen worker 76 - jepsen.util 256 :ok :write [25 44] INFO [2022-01-10 12:53:39,088] jepsen worker 39 - jepsen.util 129 :invoke :write [13 39] INFO [2022-01-10 12:53:39,089] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,091] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:39,091] jepsen worker 62 - jepsen.util 242 :ok :read [20 35] INFO [2022-01-10 12:53:39,092] jepsen worker 52 - jepsen.util 52 :ok :read [17 4] INFO [2022-01-10 12:53:39,092] jepsen worker 50 - jepsen.util 50 :ok :write [16 10] INFO [2022-01-10 12:53:39,092] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,093] jepsen worker 62 - jepsen.util 242 :ok :read [20 35] INFO [2022-01-10 12:53:39,093] jepsen worker 12 - jepsen.util 192 :invoke :write [4 14] INFO [2022-01-10 12:53:39,094] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:39,095] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,096] jepsen worker 15 - jepsen.util 15 :ok :read [5 30] INFO [2022-01-10 12:53:39,096] jepsen worker 76 - jepsen.util 256 :invoke :write [25 41] INFO [2022-01-10 12:53:39,097] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:39,098] jepsen worker 68 - jepsen.util 68 :ok :read [22 24] INFO [2022-01-10 12:53:39,099] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,099] jepsen worker 50 - jepsen.util 50 :ok :read [16 10] INFO [2022-01-10 12:53:39,100] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,101] jepsen worker 15 - jepsen.util 15 :ok :read [5 30] INFO [2022-01-10 12:53:39,101] jepsen worker 62 - jepsen.util 242 :invoke :write [20 6] INFO [2022-01-10 12:53:39,102] jepsen worker 70 - jepsen.util 160 :ok :write [23 4] INFO [2022-01-10 12:53:39,196] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:39,196] jepsen worker 12 - jepsen.util 192 :ok :write [4 14] INFO [2022-01-10 12:53:39,196] jepsen worker 44 - jepsen.util 44 :ok :write [14 47] INFO [2022-01-10 12:53:39,196] jepsen worker 21 - jepsen.util 201 :ok :write [7 31] INFO [2022-01-10 12:53:39,197] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:39,197] jepsen worker 39 - jepsen.util 129 :ok :write [13 39] INFO [2022-01-10 12:53:39,197] jepsen worker 4 - jepsen.util 94 :ok :write [1 26] INFO [2022-01-10 12:53:39,197] jepsen worker 78 - jepsen.util 168 :ok :write [26 11] INFO [2022-01-10 12:53:39,198] jepsen worker 85 - jepsen.util 265 :ok :write [28 6] INFO [2022-01-10 12:53:39,199] jepsen worker 70 - jepsen.util 160 :invoke :write [23 11] INFO [2022-01-10 12:53:39,199] jepsen worker 33 - jepsen.util 123 :ok :write [11 3] INFO [2022-01-10 12:53:39,201] jepsen worker 78 - jepsen.util 168 :invoke :write [26 29] INFO [2022-01-10 12:53:39,202] jepsen worker 89 - jepsen.util 269 :invoke :write [29 37] INFO [2022-01-10 12:53:39,203] jepsen worker 62 - jepsen.util 242 :ok :write [20 6] INFO [2022-01-10 12:53:39,203] jepsen worker 12 - jepsen.util 192 :invoke :write [4 0] INFO [2022-01-10 12:53:39,204] jepsen worker 4 - jepsen.util 94 :invoke :write [1 30] INFO [2022-01-10 12:53:39,206] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,208] jepsen worker 64 - jepsen.util 334 :fail :read [21 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:39,208] jepsen worker 15 - jepsen.util 15 :ok :read [5 30] INFO [2022-01-10 12:53:39,208] jepsen worker 68 - jepsen.util 68 :invoke :write [22 42] INFO [2022-01-10 12:53:39,209] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,210] jepsen worker 50 - jepsen.util 50 :ok :read [16 10] INFO [2022-01-10 12:53:39,210] jepsen worker 64 - jepsen.util 334 :invoke :write [21 38] INFO [2022-01-10 12:53:39,211] jepsen worker 71 - jepsen.util 161 :ok :write [23 20] INFO [2022-01-10 12:53:39,211] jepsen worker 6 - jepsen.util 276 :invoke :write [2 28] INFO [2022-01-10 12:53:39,212] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:39,213] jepsen worker 85 - jepsen.util 265 :ok :read [28 6] INFO [2022-01-10 12:53:39,213] jepsen worker 39 - jepsen.util 129 :invoke :write [13 14] INFO [2022-01-10 12:53:39,215] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,216] jepsen worker 62 - jepsen.util 242 :ok :read [20 6] INFO [2022-01-10 12:53:39,216] jepsen worker 81 - jepsen.util 171 :invoke :read [27 nil] INFO [2022-01-10 12:53:39,216] jepsen worker 53 - jepsen.util 143 :ok :write [17 3] INFO [2022-01-10 12:53:39,216] jepsen worker 52 - jepsen.util 52 :invoke :read [17 nil] INFO [2022-01-10 12:53:39,217] jepsen worker 81 - jepsen.util 171 :ok :read [27 33] INFO [2022-01-10 12:53:39,217] jepsen worker 52 - jepsen.util 52 :ok :read [17 3] INFO [2022-01-10 12:53:39,218] jepsen worker 66 - jepsen.util 246 :ok :write [22 10] INFO [2022-01-10 12:53:39,218] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,219] jepsen worker 33 - jepsen.util 123 :ok :read [11 3] INFO [2022-01-10 12:53:39,219] jepsen worker 50 - jepsen.util 50 :invoke :write [16 23] INFO [2022-01-10 12:53:39,220] jepsen worker 76 - jepsen.util 256 :ok :write [25 41] INFO [2022-01-10 12:53:39,220] jepsen worker 33 - jepsen.util 123 :invoke :write [11 10] INFO [2022-01-10 12:53:39,222] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:39,223] jepsen worker 76 - jepsen.util 256 :ok :read [25 41] INFO [2022-01-10 12:53:39,223] jepsen worker 16 - jepsen.util 286 :ok :write [5 21] INFO [2022-01-10 12:53:39,223] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,224] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:39,225] jepsen worker 21 - jepsen.util 201 :ok :read [7 31] INFO [2022-01-10 12:53:39,225] jepsen worker 84 - jepsen.util 84 :ok :write [28 2] INFO [2022-01-10 12:53:39,225] jepsen worker 71 - jepsen.util 161 :ok :read [23 20] INFO [2022-01-10 12:53:39,225] jepsen worker 76 - jepsen.util 256 :invoke :write [25 43] INFO [2022-01-10 12:53:39,227] jepsen worker 38 - jepsen.util 38 :invoke :write [12 4] INFO [2022-01-10 12:53:39,227] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:39,228] jepsen worker 81 - jepsen.util 171 :invoke :write [27 43] INFO [2022-01-10 12:53:39,229] jepsen worker 85 - jepsen.util 265 :ok :read [28 2] INFO [2022-01-10 12:53:39,230] jepsen worker 66 - jepsen.util 246 :invoke :write [22 44] INFO [2022-01-10 12:53:39,230] jepsen worker 49 - jepsen.util 319 :ok :write [16 11] INFO [2022-01-10 12:53:39,231] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,231] jepsen worker 78 - jepsen.util 168 :ok :write [26 29] INFO [2022-01-10 12:53:39,232] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:39,232] jepsen worker 15 - jepsen.util 15 :ok :read [5 21] INFO [2022-01-10 12:53:39,232] jepsen worker 84 - jepsen.util 84 :ok :read [28 2] INFO [2022-01-10 12:53:39,233] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:39,233] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:39,233] jepsen worker 71 - jepsen.util 161 :ok :read [23 20] INFO [2022-01-10 12:53:39,234] jepsen worker 1 - jepsen.util 91 :invoke :read [0 nil] INFO [2022-01-10 12:53:39,234] jepsen worker 53 - jepsen.util 143 :ok :read [17 3] INFO [2022-01-10 12:53:39,235] jepsen worker 1 - jepsen.util 91 :ok :read [0 20] INFO [2022-01-10 12:53:39,236] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:39,236] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,236] jepsen worker 78 - jepsen.util 168 :ok :read [26 29] INFO [2022-01-10 12:53:39,237] jepsen worker 12 - jepsen.util 192 :ok :write [4 0] INFO [2022-01-10 12:53:39,237] jepsen worker 42 - jepsen.util 222 :ok :write [14 20] INFO [2022-01-10 12:53:39,237] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,238] jepsen worker 62 - jepsen.util 242 :ok :read [20 6] INFO [2022-01-10 12:53:39,238] jepsen worker 85 - jepsen.util 265 :invoke :write [28 41] INFO [2022-01-10 12:53:39,238] jepsen worker 21 - jepsen.util 201 :ok :read [7 31] INFO [2022-01-10 12:53:39,239] jepsen worker 49 - jepsen.util 319 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,240] jepsen worker 49 - jepsen.util 319 :ok :read [16 11] INFO [2022-01-10 12:53:39,240] jepsen worker 12 - jepsen.util 192 :invoke :write [4 1] INFO [2022-01-10 12:53:39,240] jepsen worker 4 - jepsen.util 94 :ok :write [1 30] INFO [2022-01-10 12:53:39,241] jepsen worker 62 - jepsen.util 242 :invoke :write [20 28] INFO [2022-01-10 12:53:39,241] jepsen worker 1 - jepsen.util 91 :invoke :write [0 3] INFO [2022-01-10 12:53:39,242] jepsen worker 53 - jepsen.util 143 :invoke :write [17 6] INFO [2022-01-10 12:53:39,243] jepsen worker 52 - jepsen.util 52 :invoke :write [17 24] INFO [2022-01-10 12:53:39,244] jepsen worker 49 - jepsen.util 319 :invoke :write [16 31] INFO [2022-01-10 12:53:39,245] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,246] jepsen worker 84 - jepsen.util 84 :invoke :read [28 nil] INFO [2022-01-10 12:53:39,246] jepsen worker 21 - jepsen.util 201 :ok :read [7 31] INFO [2022-01-10 12:53:39,246] jepsen worker 83 - jepsen.util 263 :ok :write [27 16] INFO [2022-01-10 12:53:39,246] jepsen worker 84 - jepsen.util 84 :ok :read [28 2] INFO [2022-01-10 12:53:39,246] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:39,247] jepsen worker 4 - jepsen.util 94 :ok :read [1 30] INFO [2022-01-10 12:53:39,248] jepsen worker 84 - jepsen.util 84 :invoke :write [28 25] INFO [2022-01-10 12:53:39,249] jepsen worker 83 - jepsen.util 263 :invoke :write [27 45] INFO [2022-01-10 12:53:39,249] jepsen worker 70 - jepsen.util 160 :ok :write [23 11] INFO [2022-01-10 12:53:39,250] jepsen worker 21 - jepsen.util 201 :invoke :write [7 15] INFO [2022-01-10 12:53:39,251] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,252] jepsen worker 15 - jepsen.util 15 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,252] jepsen worker 16 - jepsen.util 286 :ok :read [5 21] INFO [2022-01-10 12:53:39,252] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:39,252] jepsen worker 15 - jepsen.util 15 :ok :read [5 21] INFO [2022-01-10 12:53:39,253] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:39,253] jepsen worker 78 - jepsen.util 168 :ok :read [26 29] INFO [2022-01-10 12:53:39,253] jepsen worker 44 - jepsen.util 44 :invoke :write [14 7] INFO [2022-01-10 12:53:39,254] jepsen worker 4 - jepsen.util 94 :invoke :write [1 12] INFO [2022-01-10 12:53:39,254] jepsen worker 42 - jepsen.util 222 :ok :read [14 20] INFO [2022-01-10 12:53:39,255] jepsen worker 16 - jepsen.util 286 :invoke :write [5 30] INFO [2022-01-10 12:53:39,255] jepsen worker 62 - jepsen.util 242 :ok :write [20 28] INFO [2022-01-10 12:53:39,255] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:39,256] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:39,256] jepsen worker 42 - jepsen.util 222 :invoke :write [14 15] INFO [2022-01-10 12:53:39,257] jepsen worker 15 - jepsen.util 15 :invoke :write [5 18] INFO [2022-01-10 12:53:39,257] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,257] jepsen worker 78 - jepsen.util 168 :ok :read [26 29] INFO [2022-01-10 12:53:39,258] jepsen worker 71 - jepsen.util 161 :ok :read [23 11] INFO [2022-01-10 12:53:39,258] jepsen worker 70 - jepsen.util 160 :invoke :write [23 12] INFO [2022-01-10 12:53:39,258] jepsen worker 71 - jepsen.util 161 :invoke :write [23 32] INFO [2022-01-10 12:53:39,258] jepsen worker 62 - jepsen.util 242 :ok :read [20 28] INFO [2022-01-10 12:53:39,258] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:39,259] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,259] jepsen worker 76 - jepsen.util 256 :ok :write [25 43] INFO [2022-01-10 12:53:39,259] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:39,260] jepsen worker 62 - jepsen.util 242 :ok :read [20 28] INFO [2022-01-10 12:53:39,260] jepsen worker 62 - jepsen.util 242 :invoke :write [20 24] INFO [2022-01-10 12:53:39,260] jepsen worker 76 - jepsen.util 256 :ok :read [25 43] INFO [2022-01-10 12:53:39,260] jepsen worker 78 - jepsen.util 168 :ok :read [26 29] INFO [2022-01-10 12:53:39,261] jepsen worker 76 - jepsen.util 256 :invoke :write [25 35] INFO [2022-01-10 12:53:39,261] jepsen worker 78 - jepsen.util 168 :invoke :write [26 7] INFO [2022-01-10 12:53:39,271] jepsen worker 80 - jepsen.util 260 :ok :write [26 11] INFO [2022-01-10 12:53:39,271] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:39,272] jepsen worker 80 - jepsen.util 260 :ok :read [26 11] INFO [2022-01-10 12:53:39,273] jepsen worker 80 - jepsen.util 260 :invoke :write [26 18] INFO [2022-01-10 12:53:39,274] jepsen worker 12 - jepsen.util 192 :ok :write [4 1] INFO [2022-01-10 12:53:39,275] jepsen worker 44 - jepsen.util 44 :ok :write [14 7] INFO [2022-01-10 12:53:39,275] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:39,276] jepsen worker 44 - jepsen.util 44 :invoke :write [14 33] INFO [2022-01-10 12:53:39,276] jepsen worker 12 - jepsen.util 192 :ok :read [4 1] INFO [2022-01-10 12:53:39,277] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:39,277] jepsen worker 12 - jepsen.util 192 :ok :read [4 1] INFO [2022-01-10 12:53:39,278] jepsen worker 12 - jepsen.util 192 :invoke :read [4 nil] INFO [2022-01-10 12:53:39,279] jepsen worker 12 - jepsen.util 192 :ok :read [4 1] INFO [2022-01-10 12:53:39,279] jepsen worker 12 - jepsen.util 192 :invoke :write [4 33] INFO [2022-01-10 12:53:39,282] jepsen worker 21 - jepsen.util 201 :ok :write [7 15] INFO [2022-01-10 12:53:39,283] jepsen worker 21 - jepsen.util 201 :invoke :write [7 5] INFO [2022-01-10 12:53:39,285] jepsen worker 62 - jepsen.util 242 :ok :write [20 24] INFO [2022-01-10 12:53:39,286] jepsen worker 62 - jepsen.util 242 :invoke :write [20 29] INFO [2022-01-10 12:53:39,290] jepsen worker 53 - jepsen.util 143 :ok :write [17 6] INFO [2022-01-10 12:53:39,290] jepsen worker 76 - jepsen.util 256 :ok :write [25 35] INFO [2022-01-10 12:53:39,291] jepsen worker 53 - jepsen.util 143 :invoke :write [17 18] INFO [2022-01-10 12:53:39,291] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:39,292] jepsen worker 76 - jepsen.util 256 :ok :read [25 35] INFO [2022-01-10 12:53:39,293] jepsen worker 76 - jepsen.util 256 :invoke :write [25 25] INFO [2022-01-10 12:53:39,299] jepsen worker 44 - jepsen.util 44 :ok :write [14 33] INFO [2022-01-10 12:53:39,300] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:39,301] jepsen worker 44 - jepsen.util 44 :ok :read [14 33] INFO [2022-01-10 12:53:39,301] jepsen worker 44 - jepsen.util 44 :invoke :write [14 33] INFO [2022-01-10 12:53:39,301] jepsen worker 70 - jepsen.util 160 :ok :write [23 12] INFO [2022-01-10 12:53:39,302] jepsen worker 70 - jepsen.util 160 :invoke :read [23 nil] INFO [2022-01-10 12:53:39,303] jepsen worker 70 - jepsen.util 160 :ok :read [23 12] INFO [2022-01-10 12:53:39,303] jepsen worker 70 - jepsen.util 160 :invoke :write [23 45] INFO [2022-01-10 12:53:39,311] jepsen worker 21 - jepsen.util 201 :ok :write [7 5] INFO [2022-01-10 12:53:39,311] jepsen worker 21 - jepsen.util 201 :invoke :write [7 47] INFO [2022-01-10 12:53:39,314] jepsen worker 62 - jepsen.util 242 :ok :write [20 29] INFO [2022-01-10 12:53:39,314] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,315] jepsen worker 62 - jepsen.util 242 :ok :read [20 29] INFO [2022-01-10 12:53:39,315] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,316] jepsen worker 62 - jepsen.util 242 :ok :read [20 29] INFO [2022-01-10 12:53:39,316] jepsen worker 62 - jepsen.util 242 :invoke :write [20 19] INFO [2022-01-10 12:53:39,318] jepsen worker 78 - jepsen.util 168 :ok :write [26 7] INFO [2022-01-10 12:53:39,318] jepsen worker 78 - jepsen.util 168 :invoke :write [26 17] INFO [2022-01-10 12:53:39,319] jepsen worker 76 - jepsen.util 256 :ok :write [25 25] INFO [2022-01-10 12:53:39,320] jepsen worker 76 - jepsen.util 256 :invoke :write [25 49] INFO [2022-01-10 12:53:39,324] jepsen worker 42 - jepsen.util 222 :ok :write [14 15] INFO [2022-01-10 12:53:39,325] jepsen worker 12 - jepsen.util 192 :ok :write [4 33] INFO [2022-01-10 12:53:39,325] jepsen worker 42 - jepsen.util 222 :invoke :write [14 1] INFO [2022-01-10 12:53:39,325] jepsen worker 12 - jepsen.util 192 :invoke :write [4 9] INFO [2022-01-10 12:53:39,329] jepsen worker 53 - jepsen.util 143 :ok :write [17 18] INFO [2022-01-10 12:53:39,329] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:39,330] jepsen worker 21 - jepsen.util 201 :ok :write [7 47] INFO [2022-01-10 12:53:39,330] jepsen worker 53 - jepsen.util 143 :ok :read [17 18] INFO [2022-01-10 12:53:39,331] jepsen worker 21 - jepsen.util 201 :invoke :write [7 12] INFO [2022-01-10 12:53:39,331] jepsen worker 53 - jepsen.util 143 :invoke :write [17 48] INFO [2022-01-10 12:53:39,433] jepsen worker 7 - jepsen.util 7 :ok :write [2 46] INFO [2022-01-10 12:53:39,434] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:39,434] jepsen worker 38 - jepsen.util 38 :ok :write [12 4] INFO [2022-01-10 12:53:39,435] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:39,436] jepsen worker 38 - jepsen.util 38 :ok :read [12 4] INFO [2022-01-10 12:53:39,437] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:39,438] jepsen worker 38 - jepsen.util 38 :ok :read [12 4] INFO [2022-01-10 12:53:39,438] jepsen worker 89 - jepsen.util 269 :ok :write [29 37] INFO [2022-01-10 12:53:39,438] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:39,439] jepsen worker 85 - jepsen.util 265 :ok :write [28 41] INFO [2022-01-10 12:53:39,439] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:39,439] jepsen worker 38 - jepsen.util 38 :ok :read [12 4] INFO [2022-01-10 12:53:39,439] jepsen worker 85 - jepsen.util 265 :invoke :write [28 14] INFO [2022-01-10 12:53:39,440] jepsen worker 89 - jepsen.util 269 :ok :read [29 37] INFO [2022-01-10 12:53:39,440] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:39,440] jepsen worker 89 - jepsen.util 269 :invoke :write [29 13] INFO [2022-01-10 12:53:39,441] jepsen worker 38 - jepsen.util 38 :ok :read [12 4] INFO [2022-01-10 12:53:39,441] jepsen worker 38 - jepsen.util 38 :invoke :write [12 37] INFO [2022-01-10 12:53:39,443] jepsen worker 39 - jepsen.util 129 :ok :write [13 14] INFO [2022-01-10 12:53:39,444] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:39,445] jepsen worker 4 - jepsen.util 94 :ok :write [1 12] INFO [2022-01-10 12:53:39,446] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:39,446] jepsen worker 4 - jepsen.util 94 :ok :read [1 12] INFO [2022-01-10 12:53:39,447] jepsen worker 4 - jepsen.util 94 :invoke :write [1 36] INFO [2022-01-10 12:53:39,448] jepsen worker 68 - jepsen.util 68 :ok :write [22 42] INFO [2022-01-10 12:53:39,449] jepsen worker 68 - jepsen.util 68 :invoke :write [22 0] INFO [2022-01-10 12:53:39,453] jepsen worker 33 - jepsen.util 123 :ok :write [11 10] INFO [2022-01-10 12:53:39,454] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,455] jepsen worker 33 - jepsen.util 123 :ok :read [11 10] INFO [2022-01-10 12:53:39,456] jepsen worker 33 - jepsen.util 123 :invoke :write [11 34] INFO [2022-01-10 12:53:39,457] jepsen worker 16 - jepsen.util 286 :ok :write [5 30] INFO [2022-01-10 12:53:39,458] jepsen worker 16 - jepsen.util 286 :invoke :write [5 29] INFO [2022-01-10 12:53:39,461] jepsen worker 2 - jepsen.util 92 :ok :write [0 35] INFO [2022-01-10 12:53:39,462] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:39,467] jepsen worker 50 - jepsen.util 50 :ok :write [16 23] INFO [2022-01-10 12:53:39,467] jepsen worker 50 - jepsen.util 50 :invoke :write [16 36] INFO [2022-01-10 12:53:39,471] jepsen worker 83 - jepsen.util 263 :ok :write [27 45] INFO [2022-01-10 12:53:39,472] jepsen worker 39 - jepsen.util 129 :ok :read [13 14] INFO [2022-01-10 12:53:39,472] jepsen worker 2 - jepsen.util 92 :ok :read [0 35] INFO [2022-01-10 12:53:39,472] jepsen worker 83 - jepsen.util 263 :invoke :write [27 2] INFO [2022-01-10 12:53:39,473] jepsen worker 39 - jepsen.util 129 :invoke :write [13 8] INFO [2022-01-10 12:53:39,473] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:39,474] jepsen worker 2 - jepsen.util 92 :ok :read [0 35] INFO [2022-01-10 12:53:39,475] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:39,475] jepsen worker 2 - jepsen.util 92 :ok :read [0 35] INFO [2022-01-10 12:53:39,476] jepsen worker 2 - jepsen.util 92 :invoke :write [0 23] INFO [2022-01-10 12:53:39,487] jepsen worker 68 - jepsen.util 68 :ok :write [22 0] INFO [2022-01-10 12:53:39,487] jepsen worker 68 - jepsen.util 68 :invoke :write [22 38] INFO [2022-01-10 12:53:39,492] jepsen worker 33 - jepsen.util 123 :ok :write [11 34] INFO [2022-01-10 12:53:39,492] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,493] jepsen worker 33 - jepsen.util 123 :ok :read [11 34] INFO [2022-01-10 12:53:39,494] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,494] jepsen worker 16 - jepsen.util 286 :ok :write [5 29] INFO [2022-01-10 12:53:39,494] jepsen worker 33 - jepsen.util 123 :ok :read [11 34] INFO [2022-01-10 12:53:39,495] jepsen worker 16 - jepsen.util 286 :invoke :write [5 1] INFO [2022-01-10 12:53:39,496] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,496] jepsen worker 33 - jepsen.util 123 :ok :read [11 34] INFO [2022-01-10 12:53:39,497] jepsen worker 33 - jepsen.util 123 :invoke :write [11 23] INFO [2022-01-10 12:53:39,501] jepsen worker 50 - jepsen.util 50 :ok :write [16 36] INFO [2022-01-10 12:53:39,501] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,502] jepsen worker 50 - jepsen.util 50 :ok :read [16 36] INFO [2022-01-10 12:53:39,503] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,504] jepsen worker 50 - jepsen.util 50 :ok :read [16 36] INFO [2022-01-10 12:53:39,504] jepsen worker 50 - jepsen.util 50 :invoke :write [16 15] INFO [2022-01-10 12:53:39,506] jepsen worker 83 - jepsen.util 263 :ok :write [27 2] INFO [2022-01-10 12:53:39,506] jepsen worker 83 - jepsen.util 263 :invoke :write [27 39] INFO [2022-01-10 12:53:39,507] jepsen worker 62 - jepsen.util 242 :ok :write [20 19] INFO [2022-01-10 12:53:39,507] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,508] jepsen worker 62 - jepsen.util 242 :ok :read [20 19] INFO [2022-01-10 12:53:39,509] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,509] jepsen worker 62 - jepsen.util 242 :ok :read [20 19] INFO [2022-01-10 12:53:39,510] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,510] jepsen worker 68 - jepsen.util 68 :ok :write [22 38] INFO [2022-01-10 12:53:39,511] jepsen worker 62 - jepsen.util 242 :ok :read [20 19] INFO [2022-01-10 12:53:39,511] jepsen worker 68 - jepsen.util 68 :invoke :write [22 30] INFO [2022-01-10 12:53:39,511] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,512] jepsen worker 62 - jepsen.util 242 :ok :read [20 19] INFO [2022-01-10 12:53:39,512] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,513] jepsen worker 62 - jepsen.util 242 :ok :read [20 19] INFO [2022-01-10 12:53:39,514] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,514] jepsen worker 62 - jepsen.util 242 :ok :read [20 19] INFO [2022-01-10 12:53:39,515] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,515] jepsen worker 33 - jepsen.util 123 :ok :write [11 23] INFO [2022-01-10 12:53:39,516] jepsen worker 33 - jepsen.util 123 :invoke :write [11 26] INFO [2022-01-10 12:53:39,516] jepsen worker 62 - jepsen.util 242 :ok :read [20 19] INFO [2022-01-10 12:53:39,516] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,517] jepsen worker 62 - jepsen.util 242 :ok :read [20 19] INFO [2022-01-10 12:53:39,518] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,518] jepsen worker 62 - jepsen.util 242 :ok :read [20 19] INFO [2022-01-10 12:53:39,519] jepsen worker 62 - jepsen.util 242 :invoke :write [20 3] INFO [2022-01-10 12:53:39,519] jepsen worker 16 - jepsen.util 286 :ok :write [5 1] INFO [2022-01-10 12:53:39,520] jepsen worker 16 - jepsen.util 286 :invoke :write [5 40] INFO [2022-01-10 12:53:39,538] jepsen worker 71 - jepsen.util 161 :ok :write [23 32] INFO [2022-01-10 12:53:39,539] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:39,540] jepsen worker 71 - jepsen.util 161 :ok :read [23 32] INFO [2022-01-10 12:53:39,540] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:39,541] jepsen worker 71 - jepsen.util 161 :ok :read [23 32] INFO [2022-01-10 12:53:39,541] jepsen worker 76 - jepsen.util 256 :ok :write [25 49] INFO [2022-01-10 12:53:39,542] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:39,543] jepsen worker 76 - jepsen.util 256 :invoke :write [25 27] INFO [2022-01-10 12:53:39,543] jepsen worker 71 - jepsen.util 161 :ok :read [23 32] INFO [2022-01-10 12:53:39,544] jepsen worker 71 - jepsen.util 161 :invoke :write [23 14] INFO [2022-01-10 12:53:39,544] jepsen worker 78 - jepsen.util 168 :ok :write [26 17] INFO [2022-01-10 12:53:39,544] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:39,545] jepsen worker 78 - jepsen.util 168 :ok :read [26 17] INFO [2022-01-10 12:53:39,546] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:39,546] jepsen worker 78 - jepsen.util 168 :ok :read [26 17] INFO [2022-01-10 12:53:39,547] jepsen worker 78 - jepsen.util 168 :invoke :write [26 14] INFO [2022-01-10 12:53:39,548] jepsen worker 44 - jepsen.util 44 :ok :write [14 33] INFO [2022-01-10 12:53:39,548] jepsen worker 44 - jepsen.util 44 :invoke :write [14 29] INFO [2022-01-10 12:53:39,550] jepsen worker 12 - jepsen.util 192 :ok :write [4 9] INFO [2022-01-10 12:53:39,550] jepsen worker 12 - jepsen.util 192 :invoke :write [4 14] INFO [2022-01-10 12:53:39,554] jepsen worker 21 - jepsen.util 201 :ok :write [7 12] INFO [2022-01-10 12:53:39,554] jepsen worker 21 - jepsen.util 201 :invoke :write [7 18] INFO [2022-01-10 12:53:39,554] jepsen worker 7 - jepsen.util 7 :ok :read [2 46] INFO [2022-01-10 12:53:39,554] jepsen worker 53 - jepsen.util 143 :ok :write [17 48] INFO [2022-01-10 12:53:39,555] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:39,555] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:39,556] jepsen worker 53 - jepsen.util 143 :ok :read [17 48] INFO [2022-01-10 12:53:39,557] jepsen worker 7 - jepsen.util 7 :ok :read [2 46] INFO [2022-01-10 12:53:39,557] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:39,557] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:39,558] jepsen worker 53 - jepsen.util 143 :ok :read [17 48] INFO [2022-01-10 12:53:39,558] jepsen worker 89 - jepsen.util 269 :ok :write [29 13] INFO [2022-01-10 12:53:39,558] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:39,558] jepsen worker 7 - jepsen.util 7 :ok :read [2 46] INFO [2022-01-10 12:53:39,559] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:39,559] jepsen worker 53 - jepsen.util 143 :ok :read [17 48] INFO [2022-01-10 12:53:39,559] jepsen worker 89 - jepsen.util 269 :invoke :write [29 46] INFO [2022-01-10 12:53:39,560] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:39,560] jepsen worker 7 - jepsen.util 7 :ok :read [2 46] INFO [2022-01-10 12:53:39,560] jepsen worker 38 - jepsen.util 38 :ok :write [12 37] INFO [2022-01-10 12:53:39,560] jepsen worker 7 - jepsen.util 7 :invoke :write [2 14] INFO [2022-01-10 12:53:39,561] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:39,561] jepsen worker 53 - jepsen.util 143 :ok :read [17 48] INFO [2022-01-10 12:53:39,561] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:39,562] jepsen worker 38 - jepsen.util 38 :ok :read [12 37] INFO [2022-01-10 12:53:39,562] jepsen worker 38 - jepsen.util 38 :invoke :write [12 18] INFO [2022-01-10 12:53:39,563] jepsen worker 53 - jepsen.util 143 :ok :read [17 48] INFO [2022-01-10 12:53:39,563] jepsen worker 53 - jepsen.util 143 :invoke :write [17 32] INFO [2022-01-10 12:53:39,566] jepsen worker 85 - jepsen.util 265 :ok :write [28 14] INFO [2022-01-10 12:53:39,566] jepsen worker 40 - jepsen.util 130 :ok :write [13 8] INFO [2022-01-10 12:53:39,567] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:39,567] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:39,567] jepsen worker 85 - jepsen.util 265 :ok :read [28 14] INFO [2022-01-10 12:53:39,568] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:39,568] jepsen worker 40 - jepsen.util 130 :ok :read [13 8] INFO [2022-01-10 12:53:39,568] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:39,569] jepsen worker 85 - jepsen.util 265 :ok :read [28 14] INFO [2022-01-10 12:53:39,569] jepsen worker 85 - jepsen.util 265 :invoke :write [28 24] INFO [2022-01-10 12:53:39,569] jepsen worker 40 - jepsen.util 130 :ok :read [13 8] INFO [2022-01-10 12:53:39,570] jepsen worker 40 - jepsen.util 130 :invoke :write [13 32] INFO [2022-01-10 12:53:39,570] jepsen worker 4 - jepsen.util 94 :ok :write [1 36] INFO [2022-01-10 12:53:39,571] jepsen worker 4 - jepsen.util 94 :invoke :write [1 27] INFO [2022-01-10 12:53:39,574] jepsen worker 2 - jepsen.util 92 :ok :write [0 23] INFO [2022-01-10 12:53:39,574] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:39,575] jepsen worker 2 - jepsen.util 92 :ok :read [0 23] INFO [2022-01-10 12:53:39,575] jepsen worker 2 - jepsen.util 92 :invoke :write [0 33] INFO [2022-01-10 12:53:39,597] jepsen worker 38 - jepsen.util 38 :ok :write [12 18] INFO [2022-01-10 12:53:39,598] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:39,598] jepsen worker 89 - jepsen.util 269 :ok :write [29 46] INFO [2022-01-10 12:53:39,598] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:39,599] jepsen worker 38 - jepsen.util 38 :ok :read [12 18] INFO [2022-01-10 12:53:39,599] jepsen worker 38 - jepsen.util 38 :invoke :write [12 9] INFO [2022-01-10 12:53:39,599] jepsen worker 89 - jepsen.util 269 :ok :read [29 46] INFO [2022-01-10 12:53:39,600] jepsen worker 89 - jepsen.util 269 :invoke :write [29 19] INFO [2022-01-10 12:53:39,603] jepsen worker 85 - jepsen.util 265 :ok :write [28 24] INFO [2022-01-10 12:53:39,604] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:39,605] jepsen worker 85 - jepsen.util 265 :ok :read [28 24] INFO [2022-01-10 12:53:39,605] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:39,606] jepsen worker 85 - jepsen.util 265 :ok :read [28 24] INFO [2022-01-10 12:53:39,606] jepsen worker 40 - jepsen.util 130 :ok :write [13 32] INFO [2022-01-10 12:53:39,606] jepsen worker 85 - jepsen.util 265 :invoke :write [28 48] INFO [2022-01-10 12:53:39,607] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:39,607] jepsen worker 4 - jepsen.util 94 :ok :write [1 27] INFO [2022-01-10 12:53:39,607] jepsen worker 4 - jepsen.util 94 :invoke :write [1 33] INFO [2022-01-10 12:53:39,608] jepsen worker 40 - jepsen.util 130 :ok :read [13 32] INFO [2022-01-10 12:53:39,608] jepsen worker 40 - jepsen.util 130 :invoke :write [13 33] INFO [2022-01-10 12:53:39,610] jepsen worker 2 - jepsen.util 92 :ok :write [0 33] INFO [2022-01-10 12:53:39,610] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:39,611] jepsen worker 2 - jepsen.util 92 :ok :read [0 33] INFO [2022-01-10 12:53:39,612] jepsen worker 2 - jepsen.util 92 :invoke :write [0 22] INFO [2022-01-10 12:53:39,615] jepsen worker 58 - jepsen.util 328 :info :write [19 45] :RequestTimeoutException INFO [2022-01-10 12:53:39,615] jepsen worker 58 - jepsen.util 418 :invoke :read [19 nil] INFO [2022-01-10 12:53:39,615] jepsen worker 89 - jepsen.util 269 :ok :write [29 19] INFO [2022-01-10 12:53:39,616] jepsen worker 89 - jepsen.util 269 :invoke :write [29 10] INFO [2022-01-10 12:53:39,617] jepsen worker 58 - jepsen.util 418 :ok :read [19 48] INFO [2022-01-10 12:53:39,617] jepsen worker 58 - jepsen.util 418 :invoke :read [19 nil] INFO [2022-01-10 12:53:39,618] jepsen worker 58 - jepsen.util 418 :ok :read [19 48] INFO [2022-01-10 12:53:39,618] jepsen worker 58 - jepsen.util 418 :invoke :read [19 nil] INFO [2022-01-10 12:53:39,619] jepsen worker 58 - jepsen.util 418 :ok :read [19 48] INFO [2022-01-10 12:53:39,620] jepsen worker 58 - jepsen.util 418 :invoke :read [19 nil] INFO [2022-01-10 12:53:39,621] jepsen worker 58 - jepsen.util 418 :ok :read [19 48] INFO [2022-01-10 12:53:39,621] jepsen worker 58 - jepsen.util 418 :invoke :write [19 32] INFO [2022-01-10 12:53:39,621] jepsen worker 85 - jepsen.util 265 :ok :write [28 48] INFO [2022-01-10 12:53:39,622] jepsen worker 40 - jepsen.util 130 :ok :write [13 33] INFO [2022-01-10 12:53:39,622] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:39,622] jepsen worker 40 - jepsen.util 130 :invoke :write [13 10] INFO [2022-01-10 12:53:39,623] jepsen worker 85 - jepsen.util 265 :ok :read [28 48] INFO [2022-01-10 12:53:39,623] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:39,624] jepsen worker 85 - jepsen.util 265 :ok :read [28 48] INFO [2022-01-10 12:53:39,625] jepsen worker 85 - jepsen.util 265 :invoke :write [28 33] INFO [2022-01-10 12:53:39,627] jepsen worker 2 - jepsen.util 92 :ok :write [0 22] INFO [2022-01-10 12:53:39,628] jepsen worker 38 - jepsen.util 38 :ok :write [12 9] INFO [2022-01-10 12:53:39,628] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:39,628] jepsen worker 38 - jepsen.util 38 :invoke :write [12 21] INFO [2022-01-10 12:53:39,629] jepsen worker 2 - jepsen.util 92 :ok :read [0 22] INFO [2022-01-10 12:53:39,629] jepsen worker 2 - jepsen.util 92 :invoke :write [0 36] INFO [2022-01-10 12:53:39,633] jepsen worker 89 - jepsen.util 269 :ok :write [29 10] INFO [2022-01-10 12:53:39,634] jepsen worker 89 - jepsen.util 269 :invoke :write [29 39] INFO [2022-01-10 12:53:39,638] jepsen worker 40 - jepsen.util 130 :ok :write [13 10] INFO [2022-01-10 12:53:39,638] jepsen worker 40 - jepsen.util 130 :invoke :write [13 9] INFO [2022-01-10 12:53:39,639] jepsen worker 50 - jepsen.util 50 :ok :write [16 15] INFO [2022-01-10 12:53:39,640] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,641] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:39,641] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,642] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:39,643] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,643] jepsen worker 2 - jepsen.util 92 :ok :write [0 36] INFO [2022-01-10 12:53:39,643] jepsen worker 62 - jepsen.util 242 :ok :write [20 3] INFO [2022-01-10 12:53:39,644] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:39,644] jepsen worker 2 - jepsen.util 92 :invoke :write [0 2] INFO [2022-01-10 12:53:39,644] jepsen worker 78 - jepsen.util 168 :ok :write [26 14] INFO [2022-01-10 12:53:39,645] jepsen worker 26 - jepsen.util 296 :info :write [8 32] :RequestTimeoutException INFO [2022-01-10 12:53:39,645] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,645] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:39,646] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:39,646] jepsen worker 26 - jepsen.util 386 :invoke :read [8 nil] INFO [2022-01-10 12:53:39,646] jepsen worker 78 - jepsen.util 168 :ok :read [26 14] INFO [2022-01-10 12:53:39,646] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,647] jepsen worker 4 - jepsen.util 94 :ok :write [1 33] INFO [2022-01-10 12:53:39,647] jepsen worker 78 - jepsen.util 168 :invoke :write [26 18] INFO [2022-01-10 12:53:39,647] jepsen worker 26 - jepsen.util 386 :ok :read [8 25] INFO [2022-01-10 12:53:39,647] jepsen worker 50 - jepsen.util 50 :ok :read [16 15] INFO [2022-01-10 12:53:39,648] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,649] jepsen worker 26 - jepsen.util 386 :invoke :write [8 36] INFO [2022-01-10 12:53:39,649] jepsen worker 62 - jepsen.util 242 :ok :read [20 3] INFO [2022-01-10 12:53:39,649] jepsen worker 50 - jepsen.util 50 :invoke :write [16 4] INFO [2022-01-10 12:53:39,650] jepsen worker 89 - jepsen.util 269 :ok :write [29 39] INFO [2022-01-10 12:53:39,650] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:39,651] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:39,651] jepsen worker 4 - jepsen.util 94 :ok :read [1 33] INFO [2022-01-10 12:53:39,651] jepsen worker 62 - jepsen.util 242 :invoke :write [20 10] INFO [2022-01-10 12:53:39,651] jepsen worker 89 - jepsen.util 269 :ok :read [29 39] INFO [2022-01-10 12:53:39,652] jepsen worker 83 - jepsen.util 263 :ok :write [27 39] INFO [2022-01-10 12:53:39,652] jepsen worker 71 - jepsen.util 161 :ok :write [23 14] INFO [2022-01-10 12:53:39,652] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:39,653] jepsen worker 4 - jepsen.util 94 :ok :read [1 33] INFO [2022-01-10 12:53:39,653] jepsen worker 83 - jepsen.util 263 :invoke :write [27 40] INFO [2022-01-10 12:53:39,654] jepsen worker 71 - jepsen.util 161 :invoke :write [23 25] INFO [2022-01-10 12:53:39,654] jepsen worker 85 - jepsen.util 265 :ok :write [28 33] INFO [2022-01-10 12:53:39,655] jepsen worker 89 - jepsen.util 269 :invoke :write [29 22] INFO [2022-01-10 12:53:39,655] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:39,656] jepsen worker 85 - jepsen.util 265 :invoke :write [28 25] INFO [2022-01-10 12:53:39,656] jepsen worker 68 - jepsen.util 68 :ok :write [22 30] INFO [2022-01-10 12:53:39,657] jepsen worker 4 - jepsen.util 94 :ok :read [1 33] INFO [2022-01-10 12:53:39,657] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:39,657] jepsen worker 4 - jepsen.util 94 :invoke :write [1 16] INFO [2022-01-10 12:53:39,658] jepsen worker 68 - jepsen.util 68 :ok :read [22 30] INFO [2022-01-10 12:53:39,658] jepsen worker 40 - jepsen.util 130 :ok :write [13 9] INFO [2022-01-10 12:53:39,658] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:39,659] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:39,659] jepsen worker 68 - jepsen.util 68 :ok :read [22 30] INFO [2022-01-10 12:53:39,660] jepsen worker 40 - jepsen.util 130 :ok :read [13 9] INFO [2022-01-10 12:53:39,660] jepsen worker 68 - jepsen.util 68 :invoke :write [22 46] INFO [2022-01-10 12:53:39,660] jepsen worker 40 - jepsen.util 130 :invoke :write [13 9] INFO [2022-01-10 12:53:39,662] jepsen worker 33 - jepsen.util 123 :ok :write [11 26] INFO [2022-01-10 12:53:39,662] jepsen worker 33 - jepsen.util 123 :invoke :write [11 14] INFO [2022-01-10 12:53:39,663] jepsen worker 38 - jepsen.util 38 :ok :write [12 21] INFO [2022-01-10 12:53:39,663] jepsen worker 53 - jepsen.util 143 :ok :write [17 32] INFO [2022-01-10 12:53:39,663] jepsen worker 38 - jepsen.util 38 :invoke :write [12 40] INFO [2022-01-10 12:53:39,664] jepsen worker 53 - jepsen.util 143 :invoke :write [17 39] INFO [2022-01-10 12:53:39,667] jepsen worker 6 - jepsen.util 276 :ok :write [2 28] INFO [2022-01-10 12:53:39,668] jepsen worker 6 - jepsen.util 276 :invoke :read [2 nil] INFO [2022-01-10 12:53:39,668] jepsen worker 6 - jepsen.util 276 :ok :read [2 28] INFO [2022-01-10 12:53:39,669] jepsen worker 6 - jepsen.util 276 :invoke :read [2 nil] INFO [2022-01-10 12:53:39,669] jepsen worker 6 - jepsen.util 276 :ok :read [2 28] INFO [2022-01-10 12:53:39,670] jepsen worker 6 - jepsen.util 276 :invoke :read [2 nil] INFO [2022-01-10 12:53:39,671] jepsen worker 6 - jepsen.util 276 :ok :read [2 28] INFO [2022-01-10 12:53:39,671] jepsen worker 6 - jepsen.util 276 :invoke :write [2 7] INFO [2022-01-10 12:53:39,674] jepsen worker 16 - jepsen.util 286 :ok :write [5 40] INFO [2022-01-10 12:53:39,675] jepsen worker 16 - jepsen.util 286 :invoke :write [5 24] INFO [2022-01-10 12:53:39,679] jepsen worker 76 - jepsen.util 256 :ok :write [25 27] INFO [2022-01-10 12:53:39,680] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:39,680] jepsen worker 76 - jepsen.util 256 :ok :read [25 27] INFO [2022-01-10 12:53:39,681] jepsen worker 76 - jepsen.util 256 :invoke :write [25 16] INFO [2022-01-10 12:53:39,685] jepsen worker 44 - jepsen.util 44 :ok :write [14 29] INFO [2022-01-10 12:53:39,685] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:39,686] jepsen worker 44 - jepsen.util 44 :ok :read [14 29] INFO [2022-01-10 12:53:39,687] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:39,688] jepsen worker 44 - jepsen.util 44 :ok :read [14 29] INFO [2022-01-10 12:53:39,688] jepsen worker 44 - jepsen.util 44 :invoke :write [14 13] INFO [2022-01-10 12:53:39,689] jepsen worker 21 - jepsen.util 201 :ok :write [7 18] INFO [2022-01-10 12:53:39,690] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,691] jepsen worker 21 - jepsen.util 201 :ok :read [7 18] INFO [2022-01-10 12:53:39,691] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,692] jepsen worker 21 - jepsen.util 201 :ok :read [7 18] INFO [2022-01-10 12:53:39,692] jepsen worker 21 - jepsen.util 201 :invoke :write [7 2] INFO [2022-01-10 12:53:39,694] jepsen worker 50 - jepsen.util 50 :ok :write [16 4] INFO [2022-01-10 12:53:39,695] jepsen worker 50 - jepsen.util 50 :invoke :write [16 11] INFO [2022-01-10 12:53:39,701] jepsen worker 83 - jepsen.util 263 :ok :write [27 40] INFO [2022-01-10 12:53:39,702] jepsen worker 83 - jepsen.util 263 :invoke :write [27 13] INFO [2022-01-10 12:53:39,728] jepsen worker 62 - jepsen.util 242 :ok :write [20 10] INFO [2022-01-10 12:53:39,728] jepsen worker 78 - jepsen.util 168 :ok :write [26 18] INFO [2022-01-10 12:53:39,729] jepsen worker 62 - jepsen.util 242 :invoke :write [20 24] INFO [2022-01-10 12:53:39,730] jepsen worker 78 - jepsen.util 168 :invoke :write [26 33] INFO [2022-01-10 12:53:39,731] jepsen worker 2 - jepsen.util 92 :ok :write [0 2] INFO [2022-01-10 12:53:39,732] jepsen worker 2 - jepsen.util 92 :invoke :write [0 37] INFO [2022-01-10 12:53:39,735] jepsen worker 71 - jepsen.util 161 :ok :write [23 25] INFO [2022-01-10 12:53:39,735] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:39,736] jepsen worker 71 - jepsen.util 161 :ok :read [23 25] INFO [2022-01-10 12:53:39,736] jepsen worker 71 - jepsen.util 161 :invoke :write [23 25] INFO [2022-01-10 12:53:39,741] jepsen worker 85 - jepsen.util 265 :ok :write [28 25] INFO [2022-01-10 12:53:39,741] jepsen worker 85 - jepsen.util 265 :invoke :read [28 nil] INFO [2022-01-10 12:53:39,742] jepsen worker 85 - jepsen.util 265 :ok :read [28 25] INFO [2022-01-10 12:53:39,742] jepsen worker 85 - jepsen.util 265 :invoke :write [28 14] INFO [2022-01-10 12:53:39,745] jepsen worker 76 - jepsen.util 256 :ok :write [25 16] INFO [2022-01-10 12:53:39,745] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:39,746] jepsen worker 76 - jepsen.util 256 :ok :read [25 16] INFO [2022-01-10 12:53:39,747] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:39,747] jepsen worker 4 - jepsen.util 94 :ok :write [1 16] INFO [2022-01-10 12:53:39,748] jepsen worker 76 - jepsen.util 256 :ok :read [25 16] INFO [2022-01-10 12:53:39,748] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:39,748] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:39,749] jepsen worker 4 - jepsen.util 94 :ok :read [1 16] INFO [2022-01-10 12:53:39,749] jepsen worker 44 - jepsen.util 44 :ok :write [14 13] INFO [2022-01-10 12:53:39,749] jepsen worker 4 - jepsen.util 94 :invoke :read [1 nil] INFO [2022-01-10 12:53:39,749] jepsen worker 76 - jepsen.util 256 :ok :read [25 16] INFO [2022-01-10 12:53:39,750] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:39,750] jepsen worker 76 - jepsen.util 256 :invoke :write [25 31] INFO [2022-01-10 12:53:39,750] jepsen worker 4 - jepsen.util 94 :ok :read [1 16] INFO [2022-01-10 12:53:39,750] jepsen worker 44 - jepsen.util 44 :ok :read [14 13] INFO [2022-01-10 12:53:39,751] jepsen worker 4 - jepsen.util 94 :invoke :write [1 45] INFO [2022-01-10 12:53:39,751] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:39,752] jepsen worker 44 - jepsen.util 44 :ok :read [14 13] INFO [2022-01-10 12:53:39,752] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:39,753] jepsen worker 44 - jepsen.util 44 :ok :read [14 13] INFO [2022-01-10 12:53:39,754] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:39,754] jepsen worker 44 - jepsen.util 44 :ok :read [14 13] INFO [2022-01-10 12:53:39,755] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:39,756] jepsen worker 44 - jepsen.util 44 :ok :read [14 13] INFO [2022-01-10 12:53:39,756] jepsen worker 44 - jepsen.util 44 :invoke :write [14 23] INFO [2022-01-10 12:53:39,814] jepsen worker 89 - jepsen.util 269 :ok :write [29 22] INFO [2022-01-10 12:53:39,814] jepsen worker 89 - jepsen.util 269 :invoke :write [29 10] INFO [2022-01-10 12:53:39,817] jepsen worker 68 - jepsen.util 68 :ok :write [22 46] INFO [2022-01-10 12:53:39,818] jepsen worker 68 - jepsen.util 68 :invoke :write [22 27] INFO [2022-01-10 12:53:39,822] jepsen worker 40 - jepsen.util 130 :ok :write [13 9] INFO [2022-01-10 12:53:39,822] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:39,823] jepsen worker 40 - jepsen.util 130 :ok :read [13 9] INFO [2022-01-10 12:53:39,823] jepsen worker 40 - jepsen.util 130 :invoke :write [13 44] INFO [2022-01-10 12:53:39,827] jepsen worker 33 - jepsen.util 123 :ok :write [11 14] INFO [2022-01-10 12:53:39,827] jepsen worker 33 - jepsen.util 123 :invoke :write [11 27] INFO [2022-01-10 12:53:39,829] jepsen worker 7 - jepsen.util 7 :ok :write [2 14] INFO [2022-01-10 12:53:39,830] jepsen worker 7 - jepsen.util 7 :invoke :write [2 33] INFO [2022-01-10 12:53:39,832] jepsen worker 16 - jepsen.util 286 :ok :write [5 24] INFO [2022-01-10 12:53:39,833] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,834] jepsen worker 16 - jepsen.util 286 :ok :read [5 24] INFO [2022-01-10 12:53:39,834] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,835] jepsen worker 16 - jepsen.util 286 :ok :read [5 24] INFO [2022-01-10 12:53:39,836] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,836] jepsen worker 50 - jepsen.util 50 :ok :write [16 11] INFO [2022-01-10 12:53:39,837] jepsen worker 50 - jepsen.util 50 :invoke :write [16 31] INFO [2022-01-10 12:53:39,837] jepsen worker 16 - jepsen.util 286 :ok :read [5 24] INFO [2022-01-10 12:53:39,837] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,838] jepsen worker 16 - jepsen.util 286 :ok :read [5 24] INFO [2022-01-10 12:53:39,839] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,839] jepsen worker 16 - jepsen.util 286 :ok :read [5 24] INFO [2022-01-10 12:53:39,840] jepsen worker 16 - jepsen.util 286 :invoke :write [5 6] INFO [2022-01-10 12:53:39,840] jepsen worker 83 - jepsen.util 263 :ok :write [27 13] INFO [2022-01-10 12:53:39,841] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:39,842] jepsen worker 83 - jepsen.util 263 :ok :read [27 13] INFO [2022-01-10 12:53:39,842] jepsen worker 83 - jepsen.util 263 :invoke :write [27 19] INFO [2022-01-10 12:53:39,851] jepsen worker 68 - jepsen.util 68 :ok :write [22 27] INFO [2022-01-10 12:53:39,852] jepsen worker 68 - jepsen.util 68 :invoke :write [22 33] INFO [2022-01-10 12:53:39,860] jepsen worker 33 - jepsen.util 123 :ok :write [11 27] INFO [2022-01-10 12:53:39,861] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,861] jepsen worker 33 - jepsen.util 123 :ok :read [11 27] INFO [2022-01-10 12:53:39,862] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,863] jepsen worker 33 - jepsen.util 123 :ok :read [11 27] INFO [2022-01-10 12:53:39,863] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,864] jepsen worker 7 - jepsen.util 7 :ok :write [2 33] INFO [2022-01-10 12:53:39,864] jepsen worker 33 - jepsen.util 123 :ok :read [11 27] INFO [2022-01-10 12:53:39,864] jepsen worker 7 - jepsen.util 7 :invoke :write [2 42] INFO [2022-01-10 12:53:39,865] jepsen worker 33 - jepsen.util 123 :invoke :write [11 49] INFO [2022-01-10 12:53:39,868] jepsen worker 16 - jepsen.util 286 :ok :write [5 6] INFO [2022-01-10 12:53:39,869] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,870] jepsen worker 16 - jepsen.util 286 :ok :read [5 6] INFO [2022-01-10 12:53:39,871] jepsen worker 16 - jepsen.util 286 :invoke :write [5 17] INFO [2022-01-10 12:53:39,872] jepsen worker 50 - jepsen.util 50 :ok :write [16 31] INFO [2022-01-10 12:53:39,873] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,874] jepsen worker 50 - jepsen.util 50 :ok :read [16 31] INFO [2022-01-10 12:53:39,874] jepsen worker 50 - jepsen.util 50 :invoke :write [16 33] INFO [2022-01-10 12:53:39,877] jepsen worker 83 - jepsen.util 263 :ok :write [27 19] INFO [2022-01-10 12:53:39,877] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:39,878] jepsen worker 83 - jepsen.util 263 :ok :read [27 19] INFO [2022-01-10 12:53:39,879] jepsen worker 83 - jepsen.util 263 :invoke :write [27 18] INFO [2022-01-10 12:53:39,880] jepsen worker 38 - jepsen.util 38 :ok :write [12 40] INFO [2022-01-10 12:53:39,881] jepsen worker 68 - jepsen.util 68 :ok :write [22 33] INFO [2022-01-10 12:53:39,881] jepsen worker 38 - jepsen.util 38 :invoke :write [12 41] INFO [2022-01-10 12:53:39,883] jepsen worker 68 - jepsen.util 68 :invoke :write [22 31] INFO [2022-01-10 12:53:39,883] jepsen worker 21 - jepsen.util 201 :ok :write [7 2] INFO [2022-01-10 12:53:39,884] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,884] jepsen worker 21 - jepsen.util 201 :ok :read [7 2] INFO [2022-01-10 12:53:39,885] jepsen worker 62 - jepsen.util 242 :ok :write [20 24] INFO [2022-01-10 12:53:39,885] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:39,886] jepsen worker 53 - jepsen.util 143 :ok :write [17 39] INFO [2022-01-10 12:53:39,886] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,886] jepsen worker 21 - jepsen.util 201 :ok :read [7 2] INFO [2022-01-10 12:53:39,887] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:39,887] jepsen worker 21 - jepsen.util 201 :invoke :write [7 38] INFO [2022-01-10 12:53:39,888] jepsen worker 62 - jepsen.util 242 :ok :read [20 24] INFO [2022-01-10 12:53:39,888] jepsen worker 53 - jepsen.util 143 :ok :read [17 39] INFO [2022-01-10 12:53:39,888] jepsen worker 62 - jepsen.util 242 :invoke :write [20 35] INFO [2022-01-10 12:53:39,889] jepsen worker 33 - jepsen.util 123 :ok :write [11 49] INFO [2022-01-10 12:53:39,889] jepsen worker 53 - jepsen.util 143 :invoke :write [17 15] INFO [2022-01-10 12:53:39,890] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:39,891] jepsen worker 33 - jepsen.util 123 :ok :read [11 49] INFO [2022-01-10 12:53:39,891] jepsen worker 2 - jepsen.util 92 :ok :write [0 37] INFO [2022-01-10 12:53:39,891] jepsen worker 33 - jepsen.util 123 :invoke :write [11 1] INFO [2022-01-10 12:53:39,892] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:39,893] jepsen worker 2 - jepsen.util 92 :ok :read [0 37] INFO [2022-01-10 12:53:39,893] jepsen worker 78 - jepsen.util 168 :ok :write [26 33] INFO [2022-01-10 12:53:39,893] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:39,894] jepsen worker 78 - jepsen.util 168 :invoke :write [26 7] INFO [2022-01-10 12:53:39,894] jepsen worker 2 - jepsen.util 92 :ok :read [0 37] INFO [2022-01-10 12:53:39,895] jepsen worker 2 - jepsen.util 92 :invoke :write [0 6] INFO [2022-01-10 12:53:39,895] jepsen worker 7 - jepsen.util 7 :ok :write [2 42] INFO [2022-01-10 12:53:39,895] jepsen worker 76 - jepsen.util 256 :ok :write [25 31] INFO [2022-01-10 12:53:39,896] jepsen worker 7 - jepsen.util 7 :invoke :write [2 1] INFO [2022-01-10 12:53:39,896] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:39,897] jepsen worker 76 - jepsen.util 256 :ok :read [25 31] INFO [2022-01-10 12:53:39,897] jepsen worker 76 - jepsen.util 256 :invoke :write [25 8] INFO [2022-01-10 12:53:39,899] jepsen worker 16 - jepsen.util 286 :ok :write [5 17] INFO [2022-01-10 12:53:39,899] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:39,900] jepsen worker 71 - jepsen.util 161 :ok :write [23 25] INFO [2022-01-10 12:53:39,900] jepsen worker 71 - jepsen.util 161 :invoke :write [23 30] INFO [2022-01-10 12:53:39,900] jepsen worker 16 - jepsen.util 286 :ok :read [5 17] INFO [2022-01-10 12:53:39,901] jepsen worker 44 - jepsen.util 44 :ok :write [14 23] INFO [2022-01-10 12:53:39,901] jepsen worker 16 - jepsen.util 286 :invoke :write [5 11] INFO [2022-01-10 12:53:39,901] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:39,903] jepsen worker 44 - jepsen.util 44 :ok :read [14 23] INFO [2022-01-10 12:53:39,904] jepsen worker 85 - jepsen.util 265 :ok :write [28 14] INFO [2022-01-10 12:53:39,904] jepsen worker 50 - jepsen.util 50 :ok :write [16 33] INFO [2022-01-10 12:53:39,905] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:39,905] jepsen worker 44 - jepsen.util 44 :ok :read [14 23] INFO [2022-01-10 12:53:39,905] jepsen worker 85 - jepsen.util 265 :invoke :write [28 16] INFO [2022-01-10 12:53:39,906] jepsen worker 50 - jepsen.util 50 :invoke :write [16 18] INFO [2022-01-10 12:53:39,907] jepsen worker 89 - jepsen.util 269 :ok :write [29 10] INFO [2022-01-10 12:53:39,907] jepsen worker 44 - jepsen.util 44 :invoke :write [14 37] INFO [2022-01-10 12:53:39,908] jepsen worker 89 - jepsen.util 269 :invoke :write [29 36] INFO [2022-01-10 12:53:39,910] jepsen worker 83 - jepsen.util 263 :ok :write [27 18] INFO [2022-01-10 12:53:39,911] jepsen worker 83 - jepsen.util 263 :invoke :write [27 20] INFO [2022-01-10 12:53:39,912] jepsen worker 40 - jepsen.util 130 :ok :write [13 44] INFO [2022-01-10 12:53:39,912] jepsen worker 4 - jepsen.util 94 :ok :write [1 45] INFO [2022-01-10 12:53:39,913] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:39,913] jepsen worker 40 - jepsen.util 130 :ok :read [13 44] INFO [2022-01-10 12:53:39,913] jepsen worker 4 - jepsen.util 94 :invoke :write [1 22] INFO [2022-01-10 12:53:39,914] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:39,915] jepsen worker 40 - jepsen.util 130 :ok :read [13 44] INFO [2022-01-10 12:53:39,915] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:39,916] jepsen worker 38 - jepsen.util 38 :ok :write [12 41] INFO [2022-01-10 12:53:39,916] jepsen worker 40 - jepsen.util 130 :ok :read [13 44] INFO [2022-01-10 12:53:39,916] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:39,916] jepsen worker 21 - jepsen.util 201 :ok :write [7 38] INFO [2022-01-10 12:53:39,917] jepsen worker 21 - jepsen.util 201 :invoke :write [7 39] INFO [2022-01-10 12:53:39,918] jepsen worker 38 - jepsen.util 38 :ok :read [12 41] INFO [2022-01-10 12:53:39,918] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:39,919] jepsen worker 38 - jepsen.util 38 :invoke :write [12 13] INFO [2022-01-10 12:53:39,919] jepsen worker 68 - jepsen.util 68 :ok :write [22 31] INFO [2022-01-10 12:53:39,919] jepsen worker 40 - jepsen.util 130 :ok :read [13 44] INFO [2022-01-10 12:53:39,920] jepsen worker 68 - jepsen.util 68 :invoke :write [22 19] INFO [2022-01-10 12:53:39,920] jepsen worker 40 - jepsen.util 130 :invoke :write [13 23] INFO [2022-01-10 12:53:39,922] jepsen worker 53 - jepsen.util 143 :ok :write [17 15] INFO [2022-01-10 12:53:39,922] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:39,923] jepsen worker 62 - jepsen.util 242 :ok :write [20 35] INFO [2022-01-10 12:53:39,923] jepsen worker 53 - jepsen.util 143 :ok :read [17 15] INFO [2022-01-10 12:53:39,923] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:39,924] jepsen worker 53 - jepsen.util 143 :invoke :write [17 17] INFO [2022-01-10 12:53:39,924] jepsen worker 62 - jepsen.util 242 :ok :read [20 35] INFO [2022-01-10 12:53:39,925] jepsen worker 62 - jepsen.util 242 :invoke :write [20 43] INFO [2022-01-10 12:53:39,929] jepsen worker 78 - jepsen.util 168 :ok :write [26 7] INFO [2022-01-10 12:53:39,930] jepsen worker 2 - jepsen.util 92 :ok :write [0 6] INFO [2022-01-10 12:53:39,930] jepsen worker 78 - jepsen.util 168 :invoke :write [26 3] INFO [2022-01-10 12:53:39,931] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:39,932] jepsen worker 33 - jepsen.util 123 :ok :write [11 1] INFO [2022-01-10 12:53:39,932] jepsen worker 33 - jepsen.util 123 :invoke :write [11 45] INFO [2022-01-10 12:53:39,932] jepsen worker 2 - jepsen.util 92 :ok :read [0 6] INFO [2022-01-10 12:53:39,933] jepsen worker 2 - jepsen.util 92 :invoke :write [0 21] INFO [2022-01-10 12:53:39,937] jepsen worker 7 - jepsen.util 7 :ok :write [2 1] INFO [2022-01-10 12:53:39,937] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:39,938] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:53:39,938] jepsen worker 7 - jepsen.util 7 :invoke :write [2 20] INFO [2022-01-10 12:53:39,940] jepsen worker 16 - jepsen.util 286 :ok :write [5 11] INFO [2022-01-10 12:53:39,941] jepsen worker 16 - jepsen.util 286 :invoke :write [5 42] INFO [2022-01-10 12:53:39,943] jepsen worker 50 - jepsen.util 50 :ok :write [16 18] INFO [2022-01-10 12:53:39,944] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,945] jepsen worker 50 - jepsen.util 50 :ok :read [16 18] INFO [2022-01-10 12:53:39,946] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:39,946] jepsen worker 50 - jepsen.util 50 :ok :read [16 18] INFO [2022-01-10 12:53:39,947] jepsen worker 50 - jepsen.util 50 :invoke :write [16 48] INFO [2022-01-10 12:53:39,951] jepsen worker 83 - jepsen.util 263 :ok :write [27 20] INFO [2022-01-10 12:53:39,951] jepsen worker 83 - jepsen.util 263 :invoke :write [27 8] INFO [2022-01-10 12:53:40,044] jepsen worker 37 - jepsen.util 127 :info :write [12 31] :RequestTimeoutException INFO [2022-01-10 12:53:40,045] jepsen worker 37 - jepsen.util 217 :invoke :read [12 nil] INFO [2022-01-10 12:53:40,046] jepsen worker 37 - jepsen.util 217 :ok :read [12 41] INFO [2022-01-10 12:53:40,047] jepsen worker 37 - jepsen.util 217 :invoke :write [12 43] INFO [2022-01-10 12:53:40,094] jepsen worker 68 - jepsen.util 68 :ok :write [22 19] INFO [2022-01-10 12:53:40,094] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,095] jepsen worker 68 - jepsen.util 68 :ok :read [22 19] INFO [2022-01-10 12:53:40,096] jepsen worker 68 - jepsen.util 68 :invoke :write [22 21] INFO [2022-01-10 12:53:40,102] jepsen worker 33 - jepsen.util 123 :ok :write [11 45] INFO [2022-01-10 12:53:40,102] jepsen worker 33 - jepsen.util 123 :invoke :write [11 43] INFO [2022-01-10 12:53:40,105] jepsen worker 7 - jepsen.util 7 :ok :write [2 20] INFO [2022-01-10 12:53:40,105] jepsen worker 7 - jepsen.util 7 :invoke :write [2 47] INFO [2022-01-10 12:53:40,109] jepsen worker 16 - jepsen.util 286 :ok :write [5 42] INFO [2022-01-10 12:53:40,110] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:40,111] jepsen worker 16 - jepsen.util 286 :ok :read [5 42] INFO [2022-01-10 12:53:40,111] jepsen worker 16 - jepsen.util 286 :invoke :write [5 43] INFO [2022-01-10 12:53:40,113] jepsen worker 50 - jepsen.util 50 :ok :write [16 48] INFO [2022-01-10 12:53:40,114] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:40,115] jepsen worker 50 - jepsen.util 50 :ok :read [16 48] INFO [2022-01-10 12:53:40,115] jepsen worker 50 - jepsen.util 50 :invoke :write [16 37] INFO [2022-01-10 12:53:40,118] jepsen worker 83 - jepsen.util 263 :ok :write [27 8] INFO [2022-01-10 12:53:40,118] jepsen worker 83 - jepsen.util 263 :invoke :write [27 29] INFO [2022-01-10 12:53:40,122] jepsen worker 68 - jepsen.util 68 :ok :write [22 21] INFO [2022-01-10 12:53:40,123] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,124] jepsen worker 68 - jepsen.util 68 :ok :read [22 21] INFO [2022-01-10 12:53:40,124] jepsen worker 33 - jepsen.util 123 :ok :write [11 43] INFO [2022-01-10 12:53:40,125] jepsen worker 68 - jepsen.util 68 :invoke :write [22 3] INFO [2022-01-10 12:53:40,125] jepsen worker 33 - jepsen.util 123 :invoke :write [11 7] INFO [2022-01-10 12:53:40,128] jepsen worker 7 - jepsen.util 7 :ok :write [2 47] INFO [2022-01-10 12:53:40,128] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,129] jepsen worker 7 - jepsen.util 7 :ok :read [2 47] INFO [2022-01-10 12:53:40,130] jepsen worker 7 - jepsen.util 7 :invoke :write [2 16] INFO [2022-01-10 12:53:40,132] jepsen worker 16 - jepsen.util 286 :ok :write [5 43] INFO [2022-01-10 12:53:40,133] jepsen worker 16 - jepsen.util 286 :invoke :write [5 20] INFO [2022-01-10 12:53:40,135] jepsen worker 50 - jepsen.util 50 :ok :write [16 37] INFO [2022-01-10 12:53:40,135] jepsen worker 50 - jepsen.util 50 :invoke :write [16 30] INFO [2022-01-10 12:53:40,138] jepsen worker 83 - jepsen.util 263 :ok :write [27 29] INFO [2022-01-10 12:53:40,139] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,140] jepsen worker 83 - jepsen.util 263 :ok :read [27 29] INFO [2022-01-10 12:53:40,140] jepsen worker 83 - jepsen.util 263 :invoke :write [27 28] INFO [2022-01-10 12:53:40,143] jepsen worker 68 - jepsen.util 68 :ok :write [22 3] INFO [2022-01-10 12:53:40,143] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,144] jepsen worker 68 - jepsen.util 68 :ok :read [22 3] INFO [2022-01-10 12:53:40,145] jepsen worker 68 - jepsen.util 68 :invoke :write [22 23] INFO [2022-01-10 12:53:40,146] jepsen worker 33 - jepsen.util 123 :ok :write [11 7] INFO [2022-01-10 12:53:40,146] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:40,147] jepsen worker 33 - jepsen.util 123 :ok :read [11 7] INFO [2022-01-10 12:53:40,147] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:40,148] jepsen worker 33 - jepsen.util 123 :ok :read [11 7] INFO [2022-01-10 12:53:40,149] jepsen worker 33 - jepsen.util 123 :invoke :write [11 47] INFO [2022-01-10 12:53:40,149] jepsen worker 7 - jepsen.util 7 :ok :write [2 16] INFO [2022-01-10 12:53:40,149] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,151] jepsen worker 7 - jepsen.util 7 :ok :read [2 16] INFO [2022-01-10 12:53:40,151] jepsen worker 7 - jepsen.util 7 :invoke :write [2 11] INFO [2022-01-10 12:53:40,155] jepsen worker 16 - jepsen.util 286 :ok :write [5 20] INFO [2022-01-10 12:53:40,159] jepsen worker 50 - jepsen.util 50 :ok :write [16 30] INFO [2022-01-10 12:53:40,160] jepsen worker 16 - jepsen.util 286 :invoke :write [5 38] INFO [2022-01-10 12:53:40,161] jepsen worker 50 - jepsen.util 50 :invoke :write [16 45] INFO [2022-01-10 12:53:40,164] jepsen worker 83 - jepsen.util 263 :ok :write [27 28] INFO [2022-01-10 12:53:40,165] jepsen worker 83 - jepsen.util 263 :invoke :write [27 22] INFO [2022-01-10 12:53:40,167] jepsen worker 68 - jepsen.util 68 :ok :write [22 23] INFO [2022-01-10 12:53:40,168] jepsen worker 68 - jepsen.util 68 :invoke :write [22 14] INFO [2022-01-10 12:53:40,169] jepsen worker 76 - jepsen.util 256 :ok :write [25 8] INFO [2022-01-10 12:53:40,169] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,171] jepsen worker 76 - jepsen.util 256 :ok :read [25 8] INFO [2022-01-10 12:53:40,171] jepsen worker 71 - jepsen.util 161 :ok :write [23 30] INFO [2022-01-10 12:53:40,171] jepsen worker 33 - jepsen.util 123 :ok :write [11 47] INFO [2022-01-10 12:53:40,172] jepsen worker 76 - jepsen.util 256 :invoke :write [25 6] INFO [2022-01-10 12:53:40,172] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:40,172] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,173] jepsen worker 33 - jepsen.util 123 :ok :read [11 47] INFO [2022-01-10 12:53:40,173] jepsen worker 71 - jepsen.util 161 :ok :read [23 30] INFO [2022-01-10 12:53:40,174] jepsen worker 44 - jepsen.util 44 :ok :write [14 37] INFO [2022-01-10 12:53:40,178] jepsen worker 33 - jepsen.util 123 :invoke :write [11 8] INFO [2022-01-10 12:53:40,179] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,179] jepsen worker 44 - jepsen.util 44 :invoke :write [14 20] INFO [2022-01-10 12:53:40,180] jepsen worker 71 - jepsen.util 161 :ok :read [23 30] INFO [2022-01-10 12:53:40,180] jepsen worker 7 - jepsen.util 7 :ok :write [2 11] INFO [2022-01-10 12:53:40,180] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,181] jepsen worker 71 - jepsen.util 161 :ok :read [23 30] INFO [2022-01-10 12:53:40,181] jepsen worker 7 - jepsen.util 7 :invoke :write [2 1] INFO [2022-01-10 12:53:40,182] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,183] jepsen worker 71 - jepsen.util 161 :ok :read [23 30] INFO [2022-01-10 12:53:40,184] jepsen worker 71 - jepsen.util 161 :invoke :write [23 15] INFO [2022-01-10 12:53:40,205] jepsen worker 16 - jepsen.util 286 :ok :write [5 38] INFO [2022-01-10 12:53:40,206] jepsen worker 16 - jepsen.util 286 :invoke :write [5 49] INFO [2022-01-10 12:53:40,210] jepsen worker 50 - jepsen.util 50 :ok :write [16 45] INFO [2022-01-10 12:53:40,211] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:40,212] jepsen worker 50 - jepsen.util 50 :ok :read [16 45] INFO [2022-01-10 12:53:40,212] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:40,213] jepsen worker 50 - jepsen.util 50 :ok :read [16 45] INFO [2022-01-10 12:53:40,213] jepsen worker 50 - jepsen.util 50 :invoke :write [16 23] INFO [2022-01-10 12:53:40,214] jepsen worker 83 - jepsen.util 263 :ok :write [27 22] INFO [2022-01-10 12:53:40,214] jepsen worker 83 - jepsen.util 263 :invoke :write [27 45] INFO [2022-01-10 12:53:40,222] jepsen worker 68 - jepsen.util 68 :ok :write [22 14] INFO [2022-01-10 12:53:40,222] jepsen worker 68 - jepsen.util 68 :invoke :write [22 46] INFO [2022-01-10 12:53:40,226] jepsen worker 33 - jepsen.util 123 :ok :write [11 8] INFO [2022-01-10 12:53:40,226] jepsen worker 33 - jepsen.util 123 :invoke :write [11 12] INFO [2022-01-10 12:53:40,230] jepsen worker 86 - jepsen.util 176 :ok :write [28 3] INFO [2022-01-10 12:53:40,231] jepsen worker 86 - jepsen.util 176 :invoke :write [28 41] INFO [2022-01-10 12:53:40,233] jepsen worker 89 - jepsen.util 269 :ok :write [29 36] INFO [2022-01-10 12:53:40,234] jepsen worker 89 - jepsen.util 269 :invoke :write [29 26] INFO [2022-01-10 12:53:40,311] jepsen worker 7 - jepsen.util 7 :ok :write [2 1] INFO [2022-01-10 12:53:40,311] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,312] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:53:40,312] jepsen worker 7 - jepsen.util 7 :invoke :write [2 12] INFO [2022-01-10 12:53:40,317] jepsen worker 16 - jepsen.util 286 :ok :write [5 49] INFO [2022-01-10 12:53:40,317] jepsen worker 16 - jepsen.util 286 :invoke :write [5 19] INFO [2022-01-10 12:53:40,321] jepsen worker 50 - jepsen.util 50 :ok :write [16 23] INFO [2022-01-10 12:53:40,321] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:40,322] jepsen worker 50 - jepsen.util 50 :ok :read [16 23] INFO [2022-01-10 12:53:40,322] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:40,323] jepsen worker 50 - jepsen.util 50 :ok :read [16 23] INFO [2022-01-10 12:53:40,324] jepsen worker 50 - jepsen.util 50 :invoke :write [16 49] INFO [2022-01-10 12:53:40,325] jepsen worker 83 - jepsen.util 263 :ok :write [27 45] INFO [2022-01-10 12:53:40,325] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,326] jepsen worker 83 - jepsen.util 263 :ok :read [27 45] INFO [2022-01-10 12:53:40,327] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,328] jepsen worker 83 - jepsen.util 263 :ok :read [27 45] INFO [2022-01-10 12:53:40,328] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,329] jepsen worker 83 - jepsen.util 263 :ok :read [27 45] INFO [2022-01-10 12:53:40,329] jepsen worker 83 - jepsen.util 263 :invoke :write [27 22] INFO [2022-01-10 12:53:40,330] jepsen worker 68 - jepsen.util 68 :ok :write [22 46] INFO [2022-01-10 12:53:40,330] jepsen worker 68 - jepsen.util 68 :invoke :write [22 10] INFO [2022-01-10 12:53:40,335] jepsen worker 33 - jepsen.util 123 :ok :write [11 12] INFO [2022-01-10 12:53:40,336] jepsen worker 33 - jepsen.util 123 :invoke :write [11 33] INFO [2022-01-10 12:53:40,341] jepsen worker 7 - jepsen.util 7 :ok :write [2 12] INFO [2022-01-10 12:53:40,341] jepsen worker 7 - jepsen.util 7 :invoke :write [2 44] INFO [2022-01-10 12:53:40,344] jepsen worker 16 - jepsen.util 286 :ok :write [5 19] INFO [2022-01-10 12:53:40,344] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:40,345] jepsen worker 16 - jepsen.util 286 :ok :read [5 19] INFO [2022-01-10 12:53:40,345] jepsen worker 16 - jepsen.util 286 :invoke :write [5 10] INFO [2022-01-10 12:53:40,349] jepsen worker 50 - jepsen.util 50 :ok :write [16 49] INFO [2022-01-10 12:53:40,350] jepsen worker 50 - jepsen.util 50 :invoke :write [16 28] INFO [2022-01-10 12:53:40,350] jepsen worker 5 - jepsen.util 185 :ok :write [1 40] INFO [2022-01-10 12:53:40,351] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:40,352] jepsen worker 5 - jepsen.util 185 :ok :read [1 40] INFO [2022-01-10 12:53:40,352] jepsen worker 5 - jepsen.util 185 :invoke :write [1 27] INFO [2022-01-10 12:53:40,353] jepsen worker 40 - jepsen.util 130 :ok :write [13 23] INFO [2022-01-10 12:53:40,353] jepsen worker 40 - jepsen.util 130 :invoke :write [13 5] INFO [2022-01-10 12:53:40,354] jepsen worker 83 - jepsen.util 263 :ok :write [27 22] INFO [2022-01-10 12:53:40,354] jepsen worker 83 - jepsen.util 263 :invoke :write [27 12] INFO [2022-01-10 12:53:40,357] jepsen worker 38 - jepsen.util 38 :ok :write [12 13] INFO [2022-01-10 12:53:40,357] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:40,358] jepsen worker 68 - jepsen.util 68 :ok :write [22 10] INFO [2022-01-10 12:53:40,358] jepsen worker 21 - jepsen.util 201 :ok :write [7 39] INFO [2022-01-10 12:53:40,358] jepsen worker 38 - jepsen.util 38 :ok :read [12 13] INFO [2022-01-10 12:53:40,358] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,359] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:40,359] jepsen worker 38 - jepsen.util 38 :invoke :write [12 16] INFO [2022-01-10 12:53:40,359] jepsen worker 68 - jepsen.util 68 :ok :read [22 10] INFO [2022-01-10 12:53:40,359] jepsen worker 21 - jepsen.util 201 :ok :read [7 39] INFO [2022-01-10 12:53:40,360] jepsen worker 68 - jepsen.util 68 :invoke :write [22 6] INFO [2022-01-10 12:53:40,360] jepsen worker 21 - jepsen.util 201 :invoke :write [7 7] INFO [2022-01-10 12:53:40,363] jepsen worker 33 - jepsen.util 123 :ok :write [11 33] INFO [2022-01-10 12:53:40,364] jepsen worker 33 - jepsen.util 123 :invoke :write [11 27] INFO [2022-01-10 12:53:40,370] jepsen worker 7 - jepsen.util 7 :ok :write [2 44] INFO [2022-01-10 12:53:40,370] jepsen worker 7 - jepsen.util 7 :invoke :write [2 24] INFO [2022-01-10 12:53:40,375] jepsen worker 16 - jepsen.util 286 :ok :write [5 10] INFO [2022-01-10 12:53:40,376] jepsen worker 16 - jepsen.util 286 :invoke :write [5 39] INFO [2022-01-10 12:53:40,380] jepsen worker 50 - jepsen.util 50 :ok :write [16 28] INFO [2022-01-10 12:53:40,380] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:40,381] jepsen worker 50 - jepsen.util 50 :ok :read [16 28] INFO [2022-01-10 12:53:40,381] jepsen worker 50 - jepsen.util 50 :invoke :write [16 36] INFO [2022-01-10 12:53:40,392] jepsen worker 83 - jepsen.util 263 :ok :write [27 12] INFO [2022-01-10 12:53:40,393] jepsen worker 83 - jepsen.util 263 :invoke :write [27 29] INFO [2022-01-10 12:53:40,395] jepsen worker 68 - jepsen.util 68 :ok :write [22 6] INFO [2022-01-10 12:53:40,396] jepsen worker 68 - jepsen.util 68 :invoke :write [22 41] INFO [2022-01-10 12:53:40,416] jepsen worker 33 - jepsen.util 123 :ok :write [11 27] INFO [2022-01-10 12:53:40,417] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:40,418] jepsen worker 33 - jepsen.util 123 :ok :read [11 27] INFO [2022-01-10 12:53:40,418] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:40,419] jepsen worker 33 - jepsen.util 123 :ok :read [11 27] INFO [2022-01-10 12:53:40,420] jepsen worker 33 - jepsen.util 123 :invoke :write [11 16] INFO [2022-01-10 12:53:40,420] jepsen worker 7 - jepsen.util 7 :ok :write [2 24] INFO [2022-01-10 12:53:40,420] jepsen worker 7 - jepsen.util 7 :invoke :write [2 38] INFO [2022-01-10 12:53:40,423] jepsen worker 16 - jepsen.util 286 :ok :write [5 39] INFO [2022-01-10 12:53:40,423] jepsen worker 16 - jepsen.util 286 :invoke :write [5 33] INFO [2022-01-10 12:53:40,427] jepsen worker 50 - jepsen.util 50 :ok :write [16 36] INFO [2022-01-10 12:53:40,428] jepsen worker 50 - jepsen.util 50 :invoke :write [16 3] INFO [2022-01-10 12:53:40,432] jepsen worker 83 - jepsen.util 263 :ok :write [27 29] INFO [2022-01-10 12:53:40,432] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,433] jepsen worker 83 - jepsen.util 263 :ok :read [27 29] INFO [2022-01-10 12:53:40,433] jepsen worker 53 - jepsen.util 143 :ok :write [17 17] INFO [2022-01-10 12:53:40,434] jepsen worker 83 - jepsen.util 263 :invoke :write [27 27] INFO [2022-01-10 12:53:40,435] jepsen worker 62 - jepsen.util 242 :ok :write [20 43] INFO [2022-01-10 12:53:40,435] jepsen worker 68 - jepsen.util 68 :ok :write [22 41] INFO [2022-01-10 12:53:40,435] jepsen worker 53 - jepsen.util 143 :invoke :write [17 33] INFO [2022-01-10 12:53:40,436] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,437] jepsen worker 68 - jepsen.util 68 :ok :read [22 41] INFO [2022-01-10 12:53:40,437] jepsen worker 62 - jepsen.util 242 :invoke :write [20 0] INFO [2022-01-10 12:53:40,438] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,439] jepsen worker 68 - jepsen.util 68 :ok :read [22 41] INFO [2022-01-10 12:53:40,439] jepsen worker 2 - jepsen.util 92 :ok :write [0 21] INFO [2022-01-10 12:53:40,440] jepsen worker 68 - jepsen.util 68 :invoke :write [22 38] INFO [2022-01-10 12:53:40,440] jepsen worker 78 - jepsen.util 168 :ok :write [26 3] INFO [2022-01-10 12:53:40,441] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:40,441] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:40,442] jepsen worker 2 - jepsen.util 92 :ok :read [0 21] INFO [2022-01-10 12:53:40,442] jepsen worker 33 - jepsen.util 123 :ok :write [11 16] INFO [2022-01-10 12:53:40,442] jepsen worker 2 - jepsen.util 92 :invoke :write [0 3] INFO [2022-01-10 12:53:40,442] jepsen worker 78 - jepsen.util 168 :ok :read [26 3] INFO [2022-01-10 12:53:40,443] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:40,443] jepsen worker 78 - jepsen.util 168 :invoke :write [26 46] INFO [2022-01-10 12:53:40,443] jepsen worker 33 - jepsen.util 123 :ok :read [11 16] INFO [2022-01-10 12:53:40,444] jepsen worker 33 - jepsen.util 123 :invoke :write [11 38] INFO [2022-01-10 12:53:40,446] jepsen worker 76 - jepsen.util 256 :ok :write [25 6] INFO [2022-01-10 12:53:40,447] jepsen worker 7 - jepsen.util 7 :ok :write [2 38] INFO [2022-01-10 12:53:40,447] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,448] jepsen worker 7 - jepsen.util 7 :invoke :write [2 49] INFO [2022-01-10 12:53:40,448] jepsen worker 76 - jepsen.util 256 :ok :read [25 6] INFO [2022-01-10 12:53:40,448] jepsen worker 76 - jepsen.util 256 :invoke :write [25 1] INFO [2022-01-10 12:53:40,449] jepsen worker 14 - jepsen.util 284 :ok :write [4 30] INFO [2022-01-10 12:53:40,450] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:40,450] jepsen worker 14 - jepsen.util 284 :ok :read [4 30] INFO [2022-01-10 12:53:40,451] jepsen worker 44 - jepsen.util 44 :ok :write [14 20] INFO [2022-01-10 12:53:40,451] jepsen worker 14 - jepsen.util 284 :invoke :write [4 16] INFO [2022-01-10 12:53:40,452] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:40,452] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:40,453] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:40,454] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:40,454] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:40,455] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:40,455] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:40,456] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:40,456] jepsen worker 71 - jepsen.util 161 :ok :write [23 15] INFO [2022-01-10 12:53:40,456] jepsen worker 89 - jepsen.util 269 :ok :write [29 26] INFO [2022-01-10 12:53:40,457] jepsen worker 44 - jepsen.util 44 :invoke :write [14 36] INFO [2022-01-10 12:53:40,457] jepsen worker 71 - jepsen.util 161 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,458] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,458] jepsen worker 71 - jepsen.util 161 :ok :read [23 15] INFO [2022-01-10 12:53:40,459] jepsen worker 89 - jepsen.util 269 :ok :read [29 26] INFO [2022-01-10 12:53:40,459] jepsen worker 71 - jepsen.util 161 :invoke :write [23 6] INFO [2022-01-10 12:53:40,459] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,460] jepsen worker 89 - jepsen.util 269 :ok :read [29 26] INFO [2022-01-10 12:53:40,460] jepsen worker 89 - jepsen.util 269 :invoke :write [29 34] INFO [2022-01-10 12:53:40,487] jepsen worker 35 - jepsen.util 125 :info :write [11 10] :RequestTimeoutException INFO [2022-01-10 12:53:40,487] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:40,488] jepsen worker 35 - jepsen.util 215 :ok :read [11 16] INFO [2022-01-10 12:53:40,489] jepsen worker 35 - jepsen.util 215 :invoke :write [11 39] INFO [2022-01-10 12:53:40,502] jepsen worker 86 - jepsen.util 176 :ok :write [28 41] INFO [2022-01-10 12:53:40,503] jepsen worker 40 - jepsen.util 130 :ok :write [13 5] INFO [2022-01-10 12:53:40,503] jepsen worker 86 - jepsen.util 176 :invoke :write [28 23] INFO [2022-01-10 12:53:40,503] jepsen worker 40 - jepsen.util 130 :invoke :write [13 43] INFO [2022-01-10 12:53:40,534] jepsen worker 21 - jepsen.util 201 :ok :write [7 7] INFO [2022-01-10 12:53:40,534] jepsen worker 5 - jepsen.util 185 :ok :write [1 27] INFO [2022-01-10 12:53:40,534] jepsen worker 16 - jepsen.util 286 :ok :write [5 33] INFO [2022-01-10 12:53:40,535] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:40,535] jepsen worker 16 - jepsen.util 286 :invoke :write [5 48] INFO [2022-01-10 12:53:40,535] jepsen worker 21 - jepsen.util 201 :ok :read [7 7] INFO [2022-01-10 12:53:40,536] jepsen worker 5 - jepsen.util 185 :invoke :write [1 15] INFO [2022-01-10 12:53:40,536] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:40,537] jepsen worker 21 - jepsen.util 201 :ok :read [7 7] INFO [2022-01-10 12:53:40,538] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:40,539] jepsen worker 21 - jepsen.util 201 :ok :read [7 7] INFO [2022-01-10 12:53:40,539] jepsen worker 21 - jepsen.util 201 :invoke :write [7 18] INFO [2022-01-10 12:53:40,540] jepsen worker 50 - jepsen.util 50 :ok :write [16 3] INFO [2022-01-10 12:53:40,540] jepsen worker 38 - jepsen.util 38 :ok :write [12 16] INFO [2022-01-10 12:53:40,541] jepsen worker 50 - jepsen.util 50 :invoke :write [16 37] INFO [2022-01-10 12:53:40,541] jepsen worker 38 - jepsen.util 38 :invoke :write [12 5] INFO [2022-01-10 12:53:40,545] jepsen worker 2 - jepsen.util 92 :ok :write [0 3] INFO [2022-01-10 12:53:40,545] jepsen worker 2 - jepsen.util 92 :invoke :write [0 14] INFO [2022-01-10 12:53:40,546] jepsen worker 83 - jepsen.util 263 :ok :write [27 27] INFO [2022-01-10 12:53:40,546] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,547] jepsen worker 83 - jepsen.util 263 :ok :read [27 27] INFO [2022-01-10 12:53:40,548] jepsen worker 83 - jepsen.util 263 :invoke :write [27 34] INFO [2022-01-10 12:53:40,549] jepsen worker 53 - jepsen.util 143 :ok :write [17 33] INFO [2022-01-10 12:53:40,549] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:40,550] jepsen worker 53 - jepsen.util 143 :ok :read [17 33] INFO [2022-01-10 12:53:40,551] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:40,552] jepsen worker 53 - jepsen.util 143 :ok :read [17 33] INFO [2022-01-10 12:53:40,552] jepsen worker 62 - jepsen.util 242 :ok :write [20 0] INFO [2022-01-10 12:53:40,552] jepsen worker 53 - jepsen.util 143 :invoke :write [17 15] INFO [2022-01-10 12:53:40,552] jepsen worker 62 - jepsen.util 242 :invoke :write [20 14] INFO [2022-01-10 12:53:40,556] jepsen worker 78 - jepsen.util 168 :ok :write [26 46] INFO [2022-01-10 12:53:40,557] jepsen worker 78 - jepsen.util 168 :invoke :write [26 29] INFO [2022-01-10 12:53:40,559] jepsen worker 68 - jepsen.util 68 :ok :write [22 38] INFO [2022-01-10 12:53:40,559] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,562] jepsen worker 89 - jepsen.util 269 :ok :write [29 34] INFO [2022-01-10 12:53:40,562] jepsen worker 14 - jepsen.util 284 :ok :write [4 16] INFO [2022-01-10 12:53:40,563] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,563] jepsen worker 33 - jepsen.util 123 :ok :write [11 38] INFO [2022-01-10 12:53:40,563] jepsen worker 14 - jepsen.util 284 :invoke :write [4 39] INFO [2022-01-10 12:53:40,564] jepsen worker 89 - jepsen.util 269 :ok :read [29 34] INFO [2022-01-10 12:53:40,564] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:40,564] jepsen worker 89 - jepsen.util 269 :invoke :write [29 32] INFO [2022-01-10 12:53:40,565] jepsen worker 79 - jepsen.util 259 :info :write [26 29] :RequestTimeoutException INFO [2022-01-10 12:53:40,565] jepsen worker 33 - jepsen.util 123 :ok :read [11 38] INFO [2022-01-10 12:53:40,565] jepsen worker 79 - jepsen.util 349 :invoke :write [26 43] INFO [2022-01-10 12:53:40,566] jepsen worker 33 - jepsen.util 123 :invoke :write [11 3] INFO [2022-01-10 12:53:40,568] jepsen worker 40 - jepsen.util 130 :ok :write [13 43] INFO [2022-01-10 12:53:40,569] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:40,570] jepsen worker 40 - jepsen.util 130 :ok :read [13 43] INFO [2022-01-10 12:53:40,570] jepsen worker 40 - jepsen.util 130 :invoke :write [13 45] INFO [2022-01-10 12:53:40,571] jepsen worker 76 - jepsen.util 256 :ok :write [25 1] INFO [2022-01-10 12:53:40,571] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,572] jepsen worker 76 - jepsen.util 256 :ok :read [25 1] INFO [2022-01-10 12:53:40,573] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,573] jepsen worker 69 - jepsen.util 159 :ok :write [23 7] INFO [2022-01-10 12:53:40,573] jepsen worker 69 - jepsen.util 159 :invoke :write [23 29] INFO [2022-01-10 12:53:40,574] jepsen worker 76 - jepsen.util 256 :ok :read [25 1] INFO [2022-01-10 12:53:40,574] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,575] jepsen worker 76 - jepsen.util 256 :ok :read [25 1] INFO [2022-01-10 12:53:40,576] jepsen worker 76 - jepsen.util 256 :invoke :write [25 42] INFO [2022-01-10 12:53:40,576] jepsen worker 7 - jepsen.util 7 :ok :write [2 49] INFO [2022-01-10 12:53:40,577] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,578] jepsen worker 7 - jepsen.util 7 :ok :read [2 49] INFO [2022-01-10 12:53:40,578] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,579] jepsen worker 7 - jepsen.util 7 :ok :read [2 49] INFO [2022-01-10 12:53:40,579] jepsen worker 2 - jepsen.util 92 :ok :write [0 14] INFO [2022-01-10 12:53:40,579] jepsen worker 86 - jepsen.util 176 :ok :write [28 23] INFO [2022-01-10 12:53:40,580] jepsen worker 68 - jepsen.util 68 :ok :read [22 38] INFO [2022-01-10 12:53:40,580] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,580] jepsen worker 86 - jepsen.util 176 :invoke :write [28 28] INFO [2022-01-10 12:53:40,581] jepsen worker 7 - jepsen.util 7 :ok :read [2 49] INFO [2022-01-10 12:53:40,581] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,581] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,582] jepsen worker 2 - jepsen.util 92 :invoke :write [0 10] INFO [2022-01-10 12:53:40,582] jepsen worker 68 - jepsen.util 68 :ok :read [22 38] INFO [2022-01-10 12:53:40,582] jepsen worker 7 - jepsen.util 7 :ok :read [2 49] INFO [2022-01-10 12:53:40,582] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,583] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,583] jepsen worker 44 - jepsen.util 44 :ok :write [14 36] INFO [2022-01-10 12:53:40,583] jepsen worker 68 - jepsen.util 68 :ok :read [22 38] INFO [2022-01-10 12:53:40,584] jepsen worker 68 - jepsen.util 68 :invoke :write [22 22] INFO [2022-01-10 12:53:40,584] jepsen worker 7 - jepsen.util 7 :ok :read [2 49] INFO [2022-01-10 12:53:40,584] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:40,585] jepsen worker 5 - jepsen.util 185 :ok :write [1 15] INFO [2022-01-10 12:53:40,585] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,585] jepsen worker 44 - jepsen.util 44 :ok :read [14 36] INFO [2022-01-10 12:53:40,585] jepsen worker 5 - jepsen.util 185 :invoke :write [1 6] INFO [2022-01-10 12:53:40,586] jepsen worker 7 - jepsen.util 7 :ok :read [2 49] INFO [2022-01-10 12:53:40,586] jepsen worker 44 - jepsen.util 44 :invoke :write [14 1] INFO [2022-01-10 12:53:40,586] jepsen worker 7 - jepsen.util 7 :invoke :write [2 4] INFO [2022-01-10 12:53:40,589] jepsen worker 21 - jepsen.util 201 :ok :write [7 18] INFO [2022-01-10 12:53:40,589] jepsen worker 21 - jepsen.util 201 :invoke :write [7 2] INFO [2022-01-10 12:53:40,592] jepsen worker 38 - jepsen.util 38 :ok :write [12 5] INFO [2022-01-10 12:53:40,592] jepsen worker 89 - jepsen.util 269 :ok :write [29 32] INFO [2022-01-10 12:53:40,593] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:40,593] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,594] jepsen worker 89 - jepsen.util 269 :ok :read [29 32] INFO [2022-01-10 12:53:40,594] jepsen worker 17 - jepsen.util 287 :info :write [5 19] :RequestTimeoutException INFO [2022-01-10 12:53:40,595] jepsen worker 89 - jepsen.util 269 :invoke :write [29 12] INFO [2022-01-10 12:53:40,595] jepsen worker 17 - jepsen.util 377 :invoke :write [5 36] INFO [2022-01-10 12:53:40,596] jepsen worker 16 - jepsen.util 286 :ok :write [5 48] INFO [2022-01-10 12:53:40,596] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:40,597] jepsen worker 16 - jepsen.util 286 :ok :read [5 48] INFO [2022-01-10 12:53:40,598] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:40,599] jepsen worker 16 - jepsen.util 286 :ok :read [5 48] INFO [2022-01-10 12:53:40,599] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:40,600] jepsen worker 40 - jepsen.util 130 :ok :write [13 45] INFO [2022-01-10 12:53:40,600] jepsen worker 16 - jepsen.util 286 :ok :read [5 48] INFO [2022-01-10 12:53:40,601] jepsen worker 16 - jepsen.util 286 :invoke :write [5 45] INFO [2022-01-10 12:53:40,601] jepsen worker 40 - jepsen.util 130 :invoke :write [13 1] INFO [2022-01-10 12:53:40,601] jepsen worker 53 - jepsen.util 143 :ok :write [17 15] INFO [2022-01-10 12:53:40,601] jepsen worker 50 - jepsen.util 50 :ok :write [16 37] INFO [2022-01-10 12:53:40,602] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:40,602] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:40,603] jepsen worker 50 - jepsen.util 50 :ok :read [16 37] INFO [2022-01-10 12:53:40,603] jepsen worker 53 - jepsen.util 143 :ok :read [17 15] INFO [2022-01-10 12:53:40,603] jepsen worker 50 - jepsen.util 50 :invoke :read [16 nil] INFO [2022-01-10 12:53:40,604] jepsen worker 53 - jepsen.util 143 :invoke :write [17 2] INFO [2022-01-10 12:53:40,604] jepsen worker 50 - jepsen.util 50 :ok :read [16 37] INFO [2022-01-10 12:53:40,605] jepsen worker 50 - jepsen.util 50 :invoke :write [16 2] INFO [2022-01-10 12:53:40,605] jepsen worker 83 - jepsen.util 263 :ok :write [27 34] INFO [2022-01-10 12:53:40,606] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,607] jepsen worker 83 - jepsen.util 263 :ok :read [27 34] INFO [2022-01-10 12:53:40,608] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,608] jepsen worker 83 - jepsen.util 263 :ok :read [27 34] INFO [2022-01-10 12:53:40,609] jepsen worker 83 - jepsen.util 263 :invoke :write [27 5] INFO [2022-01-10 12:53:40,611] jepsen worker 62 - jepsen.util 242 :ok :write [20 14] INFO [2022-01-10 12:53:40,612] jepsen worker 62 - jepsen.util 242 :invoke :write [20 35] INFO [2022-01-10 12:53:40,613] jepsen worker 78 - jepsen.util 168 :ok :write [26 29] INFO [2022-01-10 12:53:40,614] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:40,614] jepsen worker 2 - jepsen.util 92 :ok :write [0 10] INFO [2022-01-10 12:53:40,614] jepsen worker 2 - jepsen.util 92 :invoke :write [0 8] INFO [2022-01-10 12:53:40,614] jepsen worker 78 - jepsen.util 168 :ok :read [26 29] INFO [2022-01-10 12:53:40,615] jepsen worker 78 - jepsen.util 168 :invoke :write [26 21] INFO [2022-01-10 12:53:40,618] jepsen worker 14 - jepsen.util 284 :ok :write [4 39] INFO [2022-01-10 12:53:40,619] jepsen worker 68 - jepsen.util 68 :ok :write [22 22] INFO [2022-01-10 12:53:40,619] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:40,620] jepsen worker 68 - jepsen.util 68 :invoke :write [22 10] INFO [2022-01-10 12:53:40,620] jepsen worker 14 - jepsen.util 284 :ok :read [4 39] INFO [2022-01-10 12:53:40,620] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:40,621] jepsen worker 14 - jepsen.util 284 :ok :read [4 39] INFO [2022-01-10 12:53:40,622] jepsen worker 14 - jepsen.util 284 :invoke :write [4 7] INFO [2022-01-10 12:53:40,625] jepsen worker 33 - jepsen.util 123 :ok :write [11 3] INFO [2022-01-10 12:53:40,626] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:40,626] jepsen worker 69 - jepsen.util 159 :ok :write [23 29] INFO [2022-01-10 12:53:40,627] jepsen worker 33 - jepsen.util 123 :ok :read [11 3] INFO [2022-01-10 12:53:40,627] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,627] jepsen worker 33 - jepsen.util 123 :invoke :write [11 32] INFO [2022-01-10 12:53:40,628] jepsen worker 69 - jepsen.util 159 :ok :read [23 29] INFO [2022-01-10 12:53:40,628] jepsen worker 69 - jepsen.util 159 :invoke :write [23 24] INFO [2022-01-10 12:53:40,632] jepsen worker 76 - jepsen.util 256 :ok :write [25 42] INFO [2022-01-10 12:53:40,632] jepsen worker 76 - jepsen.util 256 :invoke :write [25 35] INFO [2022-01-10 12:53:40,634] jepsen worker 86 - jepsen.util 176 :ok :write [28 28] INFO [2022-01-10 12:53:40,634] jepsen worker 89 - jepsen.util 269 :ok :write [29 12] INFO [2022-01-10 12:53:40,634] jepsen worker 19 - jepsen.util 289 :info :write [6 35] :RequestTimeoutException INFO [2022-01-10 12:53:40,635] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:40,635] jepsen worker 19 - jepsen.util 379 :invoke :read [6 nil] INFO [2022-01-10 12:53:40,636] jepsen worker 89 - jepsen.util 269 :invoke :write [29 42] INFO [2022-01-10 12:53:40,636] jepsen worker 86 - jepsen.util 176 :ok :read [28 28] INFO [2022-01-10 12:53:40,636] jepsen worker 86 - jepsen.util 176 :invoke :write [28 49] INFO [2022-01-10 12:53:40,637] jepsen worker 19 - jepsen.util 379 :ok :read [6 8] INFO [2022-01-10 12:53:40,637] jepsen worker 19 - jepsen.util 379 :invoke :read [6 nil] INFO [2022-01-10 12:53:40,638] jepsen worker 19 - jepsen.util 379 :ok :read [6 8] INFO [2022-01-10 12:53:40,638] jepsen worker 7 - jepsen.util 7 :ok :write [2 4] INFO [2022-01-10 12:53:40,639] jepsen worker 19 - jepsen.util 379 :invoke :write [6 32] INFO [2022-01-10 12:53:40,639] jepsen worker 7 - jepsen.util 7 :invoke :write [2 44] INFO [2022-01-10 12:53:40,642] jepsen worker 5 - jepsen.util 185 :ok :write [1 6] INFO [2022-01-10 12:53:40,642] jepsen worker 40 - jepsen.util 130 :ok :write [13 1] INFO [2022-01-10 12:53:40,642] jepsen worker 5 - jepsen.util 185 :invoke :write [1 37] INFO [2022-01-10 12:53:40,643] jepsen worker 40 - jepsen.util 130 :invoke :write [13 36] INFO [2022-01-10 12:53:40,644] jepsen worker 44 - jepsen.util 44 :ok :write [14 1] INFO [2022-01-10 12:53:40,644] jepsen worker 44 - jepsen.util 44 :invoke :write [14 34] INFO [2022-01-10 12:53:40,648] jepsen worker 21 - jepsen.util 201 :ok :write [7 2] INFO [2022-01-10 12:53:40,649] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:40,649] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:40,649] jepsen worker 21 - jepsen.util 201 :ok :read [7 2] INFO [2022-01-10 12:53:40,650] jepsen worker 38 - jepsen.util 38 :invoke :write [12 29] INFO [2022-01-10 12:53:40,650] jepsen worker 21 - jepsen.util 201 :invoke :write [7 27] INFO [2022-01-10 12:53:40,655] jepsen worker 16 - jepsen.util 286 :ok :write [5 45] INFO [2022-01-10 12:53:40,656] jepsen worker 16 - jepsen.util 286 :invoke :write [5 33] INFO [2022-01-10 12:53:40,657] jepsen worker 53 - jepsen.util 143 :ok :write [17 2] INFO [2022-01-10 12:53:40,658] jepsen worker 53 - jepsen.util 143 :invoke :write [17 4] INFO [2022-01-10 12:53:40,664] jepsen worker 78 - jepsen.util 168 :ok :write [26 21] INFO [2022-01-10 12:53:40,665] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:40,666] jepsen worker 78 - jepsen.util 168 :ok :read [26 21] INFO [2022-01-10 12:53:40,666] jepsen worker 78 - jepsen.util 168 :invoke :write [26 40] INFO [2022-01-10 12:53:40,668] jepsen worker 62 - jepsen.util 242 :ok :write [20 35] INFO [2022-01-10 12:53:40,669] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,669] jepsen worker 62 - jepsen.util 242 :ok :read [20 35] INFO [2022-01-10 12:53:40,670] jepsen worker 62 - jepsen.util 242 :invoke :write [20 27] INFO [2022-01-10 12:53:40,671] jepsen worker 14 - jepsen.util 284 :ok :write [4 7] INFO [2022-01-10 12:53:40,672] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:40,673] jepsen worker 14 - jepsen.util 284 :ok :read [4 7] INFO [2022-01-10 12:53:40,673] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:40,674] jepsen worker 14 - jepsen.util 284 :ok :read [4 7] INFO [2022-01-10 12:53:40,675] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:40,675] jepsen worker 14 - jepsen.util 284 :ok :read [4 7] INFO [2022-01-10 12:53:40,676] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:40,677] jepsen worker 14 - jepsen.util 284 :ok :read [4 7] INFO [2022-01-10 12:53:40,677] jepsen worker 14 - jepsen.util 284 :invoke :write [4 36] INFO [2022-01-10 12:53:40,678] jepsen worker 69 - jepsen.util 159 :ok :write [23 24] INFO [2022-01-10 12:53:40,678] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,679] jepsen worker 69 - jepsen.util 159 :ok :read [23 24] INFO [2022-01-10 12:53:40,679] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,680] jepsen worker 69 - jepsen.util 159 :ok :read [23 24] INFO [2022-01-10 12:53:40,680] jepsen worker 69 - jepsen.util 159 :invoke :write [23 6] INFO [2022-01-10 12:53:40,684] jepsen worker 76 - jepsen.util 256 :ok :write [25 35] INFO [2022-01-10 12:53:40,685] jepsen worker 76 - jepsen.util 256 :invoke :write [25 11] INFO [2022-01-10 12:53:40,696] jepsen worker 44 - jepsen.util 44 :ok :write [14 34] INFO [2022-01-10 12:53:40,697] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:40,698] jepsen worker 44 - jepsen.util 44 :ok :read [14 34] INFO [2022-01-10 12:53:40,698] jepsen worker 44 - jepsen.util 44 :invoke :write [14 27] INFO [2022-01-10 12:53:40,699] jepsen worker 53 - jepsen.util 143 :ok :write [17 4] INFO [2022-01-10 12:53:40,700] jepsen worker 53 - jepsen.util 143 :invoke :write [17 32] INFO [2022-01-10 12:53:40,703] jepsen worker 21 - jepsen.util 201 :ok :write [7 27] INFO [2022-01-10 12:53:40,703] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:40,704] jepsen worker 21 - jepsen.util 201 :ok :read [7 27] INFO [2022-01-10 12:53:40,705] jepsen worker 78 - jepsen.util 168 :ok :write [26 40] INFO [2022-01-10 12:53:40,705] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:40,706] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:40,707] jepsen worker 21 - jepsen.util 201 :ok :read [7 27] INFO [2022-01-10 12:53:40,707] jepsen worker 78 - jepsen.util 168 :ok :read [26 40] INFO [2022-01-10 12:53:40,707] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:40,708] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:40,708] jepsen worker 21 - jepsen.util 201 :ok :read [7 27] INFO [2022-01-10 12:53:40,708] jepsen worker 78 - jepsen.util 168 :ok :read [26 40] INFO [2022-01-10 12:53:40,709] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:40,710] jepsen worker 78 - jepsen.util 168 :ok :read [26 40] INFO [2022-01-10 12:53:40,710] jepsen worker 21 - jepsen.util 201 :invoke :write [7 7] INFO [2022-01-10 12:53:40,710] jepsen worker 78 - jepsen.util 168 :invoke :write [26 12] INFO [2022-01-10 12:53:40,716] jepsen worker 62 - jepsen.util 242 :ok :write [20 27] INFO [2022-01-10 12:53:40,717] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,718] jepsen worker 62 - jepsen.util 242 :ok :read [20 27] INFO [2022-01-10 12:53:40,718] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,719] jepsen worker 62 - jepsen.util 242 :ok :read [20 27] INFO [2022-01-10 12:53:40,719] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,720] jepsen worker 62 - jepsen.util 242 :ok :read [20 27] INFO [2022-01-10 12:53:40,721] jepsen worker 48 - jepsen.util 228 :ok :write [16 7] INFO [2022-01-10 12:53:40,721] jepsen worker 62 - jepsen.util 242 :invoke :write [20 12] INFO [2022-01-10 12:53:40,721] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:40,722] jepsen worker 48 - jepsen.util 228 :ok :read [16 7] INFO [2022-01-10 12:53:40,723] jepsen worker 48 - jepsen.util 228 :invoke :write [16 4] INFO [2022-01-10 12:53:40,723] jepsen worker 76 - jepsen.util 256 :ok :write [25 11] INFO [2022-01-10 12:53:40,724] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,725] jepsen worker 76 - jepsen.util 256 :ok :read [25 11] INFO [2022-01-10 12:53:40,725] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,726] jepsen worker 76 - jepsen.util 256 :ok :read [25 11] INFO [2022-01-10 12:53:40,726] jepsen worker 83 - jepsen.util 263 :ok :write [27 5] INFO [2022-01-10 12:53:40,727] jepsen worker 76 - jepsen.util 256 :invoke :write [25 16] INFO [2022-01-10 12:53:40,727] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,728] jepsen worker 83 - jepsen.util 263 :ok :read [27 5] INFO [2022-01-10 12:53:40,729] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,729] jepsen worker 86 - jepsen.util 176 :ok :write [28 49] INFO [2022-01-10 12:53:40,729] jepsen worker 86 - jepsen.util 176 :invoke :write [28 42] INFO [2022-01-10 12:53:40,729] jepsen worker 83 - jepsen.util 263 :ok :read [27 5] INFO [2022-01-10 12:53:40,730] jepsen worker 83 - jepsen.util 263 :invoke :write [27 35] INFO [2022-01-10 12:53:40,733] jepsen worker 2 - jepsen.util 92 :ok :write [0 8] INFO [2022-01-10 12:53:40,734] jepsen worker 44 - jepsen.util 44 :ok :write [14 27] INFO [2022-01-10 12:53:40,734] jepsen worker 14 - jepsen.util 284 :ok :write [4 36] INFO [2022-01-10 12:53:40,734] jepsen worker 2 - jepsen.util 92 :invoke :write [0 13] INFO [2022-01-10 12:53:40,735] jepsen worker 44 - jepsen.util 44 :invoke :write [14 12] INFO [2022-01-10 12:53:40,736] jepsen worker 14 - jepsen.util 284 :invoke :write [4 41] INFO [2022-01-10 12:53:40,736] jepsen worker 5 - jepsen.util 185 :ok :write [1 37] INFO [2022-01-10 12:53:40,737] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:40,738] jepsen worker 5 - jepsen.util 185 :ok :read [1 37] INFO [2022-01-10 12:53:40,739] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:40,740] jepsen worker 5 - jepsen.util 185 :ok :read [1 37] INFO [2022-01-10 12:53:40,741] jepsen worker 5 - jepsen.util 185 :invoke :write [1 11] INFO [2022-01-10 12:53:40,741] jepsen worker 68 - jepsen.util 68 :ok :write [22 10] INFO [2022-01-10 12:53:40,742] jepsen worker 68 - jepsen.util 68 :invoke :write [22 32] INFO [2022-01-10 12:53:40,742] jepsen worker 38 - jepsen.util 38 :ok :write [12 29] INFO [2022-01-10 12:53:40,743] jepsen worker 21 - jepsen.util 201 :ok :write [7 7] INFO [2022-01-10 12:53:40,743] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:40,744] jepsen worker 21 - jepsen.util 201 :invoke :write [7 16] INFO [2022-01-10 12:53:40,744] jepsen worker 38 - jepsen.util 38 :ok :read [12 29] INFO [2022-01-10 12:53:40,745] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:40,746] jepsen worker 38 - jepsen.util 38 :ok :read [12 29] INFO [2022-01-10 12:53:40,746] jepsen worker 69 - jepsen.util 159 :ok :write [23 6] INFO [2022-01-10 12:53:40,747] jepsen worker 38 - jepsen.util 38 :invoke :write [12 10] INFO [2022-01-10 12:53:40,747] jepsen worker 33 - jepsen.util 123 :ok :write [11 32] INFO [2022-01-10 12:53:40,747] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,748] jepsen worker 33 - jepsen.util 123 :invoke :read [11 nil] INFO [2022-01-10 12:53:40,748] jepsen worker 69 - jepsen.util 159 :ok :read [23 6] INFO [2022-01-10 12:53:40,748] jepsen worker 33 - jepsen.util 123 :ok :read [11 32] INFO [2022-01-10 12:53:40,749] jepsen worker 69 - jepsen.util 159 :invoke :write [23 38] INFO [2022-01-10 12:53:40,749] jepsen worker 33 - jepsen.util 123 :invoke :write [11 28] INFO [2022-01-10 12:53:40,752] jepsen worker 89 - jepsen.util 269 :ok :write [29 42] INFO [2022-01-10 12:53:40,752] jepsen worker 62 - jepsen.util 242 :ok :write [20 12] INFO [2022-01-10 12:53:40,752] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,753] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,753] jepsen worker 89 - jepsen.util 269 :ok :read [29 42] INFO [2022-01-10 12:53:40,754] jepsen worker 62 - jepsen.util 242 :ok :read [20 12] INFO [2022-01-10 12:53:40,754] jepsen worker 89 - jepsen.util 269 :invoke :write [29 31] INFO [2022-01-10 12:53:40,754] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,755] jepsen worker 53 - jepsen.util 143 :ok :write [17 32] INFO [2022-01-10 12:53:40,755] jepsen worker 62 - jepsen.util 242 :ok :read [20 12] INFO [2022-01-10 12:53:40,755] jepsen worker 53 - jepsen.util 143 :invoke :write [17 31] INFO [2022-01-10 12:53:40,756] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,757] jepsen worker 62 - jepsen.util 242 :ok :read [20 12] INFO [2022-01-10 12:53:40,757] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,758] jepsen worker 76 - jepsen.util 256 :ok :write [25 16] INFO [2022-01-10 12:53:40,758] jepsen worker 7 - jepsen.util 7 :ok :write [2 44] INFO [2022-01-10 12:53:40,758] jepsen worker 76 - jepsen.util 256 :invoke :write [25 34] INFO [2022-01-10 12:53:40,759] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,759] jepsen worker 62 - jepsen.util 242 :ok :read [20 12] INFO [2022-01-10 12:53:40,759] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,761] jepsen worker 62 - jepsen.util 242 :ok :read [20 12] INFO [2022-01-10 12:53:40,761] jepsen worker 7 - jepsen.util 7 :ok :read [2 44] INFO [2022-01-10 12:53:40,761] jepsen worker 78 - jepsen.util 168 :ok :write [26 12] INFO [2022-01-10 12:53:40,762] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,762] jepsen worker 40 - jepsen.util 130 :ok :write [13 36] INFO [2022-01-10 12:53:40,763] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:40,763] jepsen worker 7 - jepsen.util 7 :ok :read [2 44] INFO [2022-01-10 12:53:40,763] jepsen worker 78 - jepsen.util 168 :ok :read [26 12] INFO [2022-01-10 12:53:40,763] jepsen worker 40 - jepsen.util 130 :invoke :write [13 14] INFO [2022-01-10 12:53:40,764] jepsen worker 78 - jepsen.util 168 :invoke :write [26 47] INFO [2022-01-10 12:53:40,765] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,765] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,766] jepsen worker 44 - jepsen.util 44 :ok :write [14 12] INFO [2022-01-10 12:53:40,766] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:40,766] jepsen worker 7 - jepsen.util 7 :ok :read [2 44] INFO [2022-01-10 12:53:40,767] jepsen worker 62 - jepsen.util 242 :ok :read [20 12] INFO [2022-01-10 12:53:40,767] jepsen worker 7 - jepsen.util 7 :invoke :write [2 8] INFO [2022-01-10 12:53:40,767] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,767] jepsen worker 44 - jepsen.util 44 :ok :read [14 12] INFO [2022-01-10 12:53:40,768] jepsen worker 44 - jepsen.util 44 :invoke :write [14 6] INFO [2022-01-10 12:53:40,769] jepsen worker 16 - jepsen.util 286 :ok :write [5 33] INFO [2022-01-10 12:53:40,769] jepsen worker 86 - jepsen.util 176 :ok :write [28 42] INFO [2022-01-10 12:53:40,769] jepsen worker 16 - jepsen.util 286 :invoke :write [5 48] INFO [2022-01-10 12:53:40,770] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:40,770] jepsen worker 62 - jepsen.util 242 :ok :read [20 12] INFO [2022-01-10 12:53:40,770] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,770] jepsen worker 86 - jepsen.util 176 :ok :read [28 42] INFO [2022-01-10 12:53:40,771] jepsen worker 86 - jepsen.util 176 :invoke :write [28 15] INFO [2022-01-10 12:53:40,773] jepsen worker 62 - jepsen.util 242 :ok :read [20 12] INFO [2022-01-10 12:53:40,773] jepsen worker 62 - jepsen.util 242 :invoke :write [20 46] INFO [2022-01-10 12:53:40,775] jepsen worker 14 - jepsen.util 284 :ok :write [4 41] INFO [2022-01-10 12:53:40,776] jepsen worker 14 - jepsen.util 284 :invoke :write [4 4] INFO [2022-01-10 12:53:40,777] jepsen worker 48 - jepsen.util 228 :ok :write [16 4] INFO [2022-01-10 12:53:40,777] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:40,778] jepsen worker 48 - jepsen.util 228 :ok :read [16 4] INFO [2022-01-10 12:53:40,779] jepsen worker 48 - jepsen.util 228 :invoke :write [16 45] INFO [2022-01-10 12:53:40,779] jepsen worker 21 - jepsen.util 201 :ok :write [7 16] INFO [2022-01-10 12:53:40,779] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:40,780] jepsen worker 21 - jepsen.util 201 :ok :read [7 16] INFO [2022-01-10 12:53:40,781] jepsen worker 21 - jepsen.util 201 :invoke :write [7 4] INFO [2022-01-10 12:53:40,784] jepsen worker 83 - jepsen.util 263 :ok :write [27 35] INFO [2022-01-10 12:53:40,784] jepsen worker 5 - jepsen.util 185 :ok :write [1 11] INFO [2022-01-10 12:53:40,784] jepsen worker 5 - jepsen.util 185 :invoke :write [1 36] INFO [2022-01-10 12:53:40,785] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,786] jepsen worker 83 - jepsen.util 263 :ok :read [27 35] INFO [2022-01-10 12:53:40,786] jepsen worker 83 - jepsen.util 263 :invoke :write [27 34] INFO [2022-01-10 12:53:40,792] jepsen worker 38 - jepsen.util 38 :ok :write [12 10] INFO [2022-01-10 12:53:40,793] jepsen worker 38 - jepsen.util 38 :invoke :write [12 16] INFO [2022-01-10 12:53:40,793] jepsen worker 62 - jepsen.util 242 :ok :write [20 46] INFO [2022-01-10 12:53:40,794] jepsen worker 2 - jepsen.util 92 :ok :write [0 13] INFO [2022-01-10 12:53:40,794] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,795] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:40,796] jepsen worker 62 - jepsen.util 242 :ok :read [20 46] INFO [2022-01-10 12:53:40,796] jepsen worker 2 - jepsen.util 92 :ok :read [0 13] INFO [2022-01-10 12:53:40,797] jepsen worker 2 - jepsen.util 92 :invoke :write [0 25] INFO [2022-01-10 12:53:40,797] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,799] jepsen worker 62 - jepsen.util 242 :ok :read [20 46] INFO [2022-01-10 12:53:40,799] jepsen worker 68 - jepsen.util 68 :ok :write [22 32] INFO [2022-01-10 12:53:40,800] jepsen worker 69 - jepsen.util 159 :ok :write [23 38] INFO [2022-01-10 12:53:40,800] jepsen worker 76 - jepsen.util 256 :ok :write [25 34] INFO [2022-01-10 12:53:40,800] jepsen worker 62 - jepsen.util 242 :invoke :write [20 39] INFO [2022-01-10 12:53:40,801] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,802] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,802] jepsen worker 69 - jepsen.util 159 :ok :read [23 38] INFO [2022-01-10 12:53:40,802] jepsen worker 76 - jepsen.util 256 :invoke :write [25 5] INFO [2022-01-10 12:53:40,803] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:53:40,803] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,803] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,804] jepsen worker 69 - jepsen.util 159 :ok :read [23 38] INFO [2022-01-10 12:53:40,804] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:53:40,804] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,805] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,805] jepsen worker 69 - jepsen.util 159 :ok :read [23 38] INFO [2022-01-10 12:53:40,806] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,806] jepsen worker 33 - jepsen.util 123 :ok :write [11 28] INFO [2022-01-10 12:53:40,806] jepsen worker 68 - jepsen.util 68 :ok :read [22 32] INFO [2022-01-10 12:53:40,807] jepsen worker 69 - jepsen.util 159 :ok :read [23 38] INFO [2022-01-10 12:53:40,807] jepsen worker 68 - jepsen.util 68 :invoke :write [22 14] INFO [2022-01-10 12:53:40,808] jepsen worker 69 - jepsen.util 159 :invoke :write [23 45] INFO [2022-01-10 12:53:40,808] jepsen worker 33 - jepsen.util 123 :invoke :write [11 27] INFO [2022-01-10 12:53:40,812] jepsen worker 44 - jepsen.util 44 :ok :write [14 6] INFO [2022-01-10 12:53:40,812] jepsen worker 44 - jepsen.util 44 :invoke :write [14 8] INFO [2022-01-10 12:53:40,813] jepsen worker 89 - jepsen.util 269 :ok :write [29 31] INFO [2022-01-10 12:53:40,813] jepsen worker 89 - jepsen.util 269 :invoke :write [29 45] INFO [2022-01-10 12:53:40,815] jepsen worker 7 - jepsen.util 7 :ok :write [2 8] INFO [2022-01-10 12:53:40,816] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,817] jepsen worker 78 - jepsen.util 168 :ok :write [26 47] INFO [2022-01-10 12:53:40,817] jepsen worker 53 - jepsen.util 143 :ok :write [17 31] INFO [2022-01-10 12:53:40,817] jepsen worker 7 - jepsen.util 7 :ok :read [2 8] INFO [2022-01-10 12:53:40,817] jepsen worker 78 - jepsen.util 168 :invoke :write [26 38] INFO [2022-01-10 12:53:40,818] jepsen worker 21 - jepsen.util 201 :ok :write [7 4] INFO [2022-01-10 12:53:40,818] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:40,818] jepsen worker 7 - jepsen.util 7 :invoke :write [2 26] INFO [2022-01-10 12:53:40,819] jepsen worker 53 - jepsen.util 143 :ok :read [17 31] INFO [2022-01-10 12:53:40,819] jepsen worker 21 - jepsen.util 201 :invoke :write [7 42] INFO [2022-01-10 12:53:40,819] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:40,820] jepsen worker 53 - jepsen.util 143 :ok :read [17 31] INFO [2022-01-10 12:53:40,820] jepsen worker 40 - jepsen.util 130 :ok :write [13 14] INFO [2022-01-10 12:53:40,821] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:40,821] jepsen worker 40 - jepsen.util 130 :invoke :write [13 49] INFO [2022-01-10 12:53:40,822] jepsen worker 53 - jepsen.util 143 :ok :read [17 31] INFO [2022-01-10 12:53:40,822] jepsen worker 73 - jepsen.util 343 :info :write [24 43] :RequestTimeoutException INFO [2022-01-10 12:53:40,822] jepsen worker 53 - jepsen.util 143 :invoke :write [17 12] INFO [2022-01-10 12:53:40,823] jepsen worker 86 - jepsen.util 176 :ok :write [28 15] INFO [2022-01-10 12:53:40,824] jepsen worker 73 - jepsen.util 433 :invoke :read [24 nil] INFO [2022-01-10 12:53:40,824] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:40,825] jepsen worker 86 - jepsen.util 176 :ok :read [28 15] INFO [2022-01-10 12:53:40,825] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:40,826] jepsen worker 86 - jepsen.util 176 :ok :read [28 15] INFO [2022-01-10 12:53:40,827] jepsen worker 86 - jepsen.util 176 :invoke :write [28 42] INFO [2022-01-10 12:53:40,829] jepsen worker 14 - jepsen.util 284 :ok :write [4 4] INFO [2022-01-10 12:53:40,829] jepsen worker 14 - jepsen.util 284 :invoke :write [4 8] INFO [2022-01-10 12:53:40,830] jepsen worker 16 - jepsen.util 286 :ok :write [5 48] INFO [2022-01-10 12:53:40,830] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:40,831] jepsen worker 16 - jepsen.util 286 :ok :read [5 48] INFO [2022-01-10 12:53:40,832] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:40,833] jepsen worker 16 - jepsen.util 286 :ok :read [5 48] INFO [2022-01-10 12:53:40,833] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:40,834] jepsen worker 16 - jepsen.util 286 :ok :read [5 48] INFO [2022-01-10 12:53:40,835] jepsen worker 16 - jepsen.util 286 :invoke :write [5 26] INFO [2022-01-10 12:53:40,836] jepsen worker 5 - jepsen.util 185 :ok :write [1 36] INFO [2022-01-10 12:53:40,836] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:40,837] jepsen worker 5 - jepsen.util 185 :ok :read [1 36] INFO [2022-01-10 12:53:40,838] jepsen worker 48 - jepsen.util 228 :ok :write [16 45] INFO [2022-01-10 12:53:40,838] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:40,838] jepsen worker 48 - jepsen.util 228 :invoke :write [16 9] INFO [2022-01-10 12:53:40,839] jepsen worker 5 - jepsen.util 185 :ok :read [1 36] INFO [2022-01-10 12:53:40,839] jepsen worker 5 - jepsen.util 185 :invoke :write [1 29] INFO [2022-01-10 12:53:40,841] jepsen worker 62 - jepsen.util 242 :ok :write [20 39] INFO [2022-01-10 12:53:40,842] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,843] jepsen worker 62 - jepsen.util 242 :ok :read [20 39] INFO [2022-01-10 12:53:40,843] jepsen worker 38 - jepsen.util 38 :ok :write [12 16] INFO [2022-01-10 12:53:40,844] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,844] jepsen worker 38 - jepsen.util 38 :invoke :write [12 45] INFO [2022-01-10 12:53:40,845] jepsen worker 62 - jepsen.util 242 :ok :read [20 39] INFO [2022-01-10 12:53:40,845] jepsen worker 83 - jepsen.util 263 :ok :write [27 34] INFO [2022-01-10 12:53:40,846] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,847] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,847] jepsen worker 62 - jepsen.util 242 :ok :read [20 39] INFO [2022-01-10 12:53:40,847] jepsen worker 62 - jepsen.util 242 :invoke :write [20 48] INFO [2022-01-10 12:53:40,848] jepsen worker 83 - jepsen.util 263 :ok :read [27 34] INFO [2022-01-10 12:53:40,848] jepsen worker 83 - jepsen.util 263 :invoke :write [27 27] INFO [2022-01-10 12:53:40,849] jepsen worker 76 - jepsen.util 256 :ok :write [25 5] INFO [2022-01-10 12:53:40,850] jepsen worker 76 - jepsen.util 256 :invoke :write [25 7] INFO [2022-01-10 12:53:40,852] jepsen worker 2 - jepsen.util 92 :ok :write [0 25] INFO [2022-01-10 12:53:40,852] jepsen worker 69 - jepsen.util 159 :ok :write [23 45] INFO [2022-01-10 12:53:40,853] jepsen worker 2 - jepsen.util 92 :invoke :write [0 30] INFO [2022-01-10 12:53:40,853] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,855] jepsen worker 69 - jepsen.util 159 :ok :read [23 45] INFO [2022-01-10 12:53:40,855] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,856] jepsen worker 69 - jepsen.util 159 :ok :read [23 45] INFO [2022-01-10 12:53:40,857] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:40,857] jepsen worker 69 - jepsen.util 159 :ok :read [23 45] INFO [2022-01-10 12:53:40,858] jepsen worker 69 - jepsen.util 159 :invoke :write [23 23] INFO [2022-01-10 12:53:40,859] jepsen worker 68 - jepsen.util 68 :ok :write [22 14] INFO [2022-01-10 12:53:40,860] jepsen worker 68 - jepsen.util 68 :invoke :write [22 21] INFO [2022-01-10 12:53:40,861] jepsen worker 53 - jepsen.util 143 :ok :write [17 12] INFO [2022-01-10 12:53:40,861] jepsen worker 53 - jepsen.util 143 :invoke :write [17 14] INFO [2022-01-10 12:53:40,866] jepsen worker 35 - jepsen.util 215 :ok :write [11 39] INFO [2022-01-10 12:53:40,867] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:40,868] jepsen worker 35 - jepsen.util 215 :ok :read [11 39] INFO [2022-01-10 12:53:40,868] jepsen worker 78 - jepsen.util 168 :ok :write [26 38] INFO [2022-01-10 12:53:40,868] jepsen worker 44 - jepsen.util 44 :ok :write [14 8] INFO [2022-01-10 12:53:40,869] jepsen worker 35 - jepsen.util 215 :invoke :write [11 31] INFO [2022-01-10 12:53:40,869] jepsen worker 44 - jepsen.util 44 :invoke :write [14 10] INFO [2022-01-10 12:53:40,870] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:40,871] jepsen worker 78 - jepsen.util 168 :ok :read [26 38] INFO [2022-01-10 12:53:40,871] jepsen worker 78 - jepsen.util 168 :invoke :read [26 nil] INFO [2022-01-10 12:53:40,873] jepsen worker 78 - jepsen.util 168 :ok :read [26 38] INFO [2022-01-10 12:53:40,874] jepsen worker 89 - jepsen.util 269 :ok :write [29 45] INFO [2022-01-10 12:53:40,874] jepsen worker 78 - jepsen.util 168 :invoke :write [26 9] INFO [2022-01-10 12:53:40,875] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,877] jepsen worker 89 - jepsen.util 269 :ok :read [29 45] INFO [2022-01-10 12:53:40,877] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,878] jepsen worker 7 - jepsen.util 7 :ok :write [2 26] INFO [2022-01-10 12:53:40,878] jepsen worker 89 - jepsen.util 269 :ok :read [29 45] INFO [2022-01-10 12:53:40,879] jepsen worker 86 - jepsen.util 176 :ok :write [28 42] INFO [2022-01-10 12:53:40,879] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,880] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,880] jepsen worker 21 - jepsen.util 201 :ok :write [7 42] INFO [2022-01-10 12:53:40,880] jepsen worker 7 - jepsen.util 7 :ok :read [2 26] INFO [2022-01-10 12:53:40,880] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:40,881] jepsen worker 89 - jepsen.util 269 :ok :read [29 45] INFO [2022-01-10 12:53:40,881] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:40,882] jepsen worker 86 - jepsen.util 176 :ok :read [28 42] INFO [2022-01-10 12:53:40,882] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:40,882] jepsen worker 21 - jepsen.util 201 :ok :read [7 42] INFO [2022-01-10 12:53:40,883] jepsen worker 7 - jepsen.util 7 :ok :read [2 26] INFO [2022-01-10 12:53:40,883] jepsen worker 86 - jepsen.util 176 :invoke :write [28 36] INFO [2022-01-10 12:53:40,883] jepsen worker 40 - jepsen.util 130 :ok :write [13 49] INFO [2022-01-10 12:53:40,884] jepsen worker 21 - jepsen.util 201 :invoke :write [7 10] INFO [2022-01-10 12:53:40,885] jepsen worker 7 - jepsen.util 7 :invoke :write [2 26] INFO [2022-01-10 12:53:40,886] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:40,886] jepsen worker 14 - jepsen.util 284 :ok :write [4 8] INFO [2022-01-10 12:53:40,887] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,887] jepsen worker 40 - jepsen.util 130 :ok :read [13 49] INFO [2022-01-10 12:53:40,887] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:40,888] jepsen worker 89 - jepsen.util 269 :ok :read [29 45] INFO [2022-01-10 12:53:40,888] jepsen worker 40 - jepsen.util 130 :invoke :write [13 25] INFO [2022-01-10 12:53:40,888] jepsen worker 14 - jepsen.util 284 :ok :read [4 8] INFO [2022-01-10 12:53:40,888] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,889] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:40,890] jepsen worker 14 - jepsen.util 284 :ok :read [4 8] INFO [2022-01-10 12:53:40,890] jepsen worker 89 - jepsen.util 269 :ok :read [29 45] INFO [2022-01-10 12:53:40,891] jepsen worker 89 - jepsen.util 269 :invoke :write [29 10] INFO [2022-01-10 12:53:40,891] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:40,892] jepsen worker 16 - jepsen.util 286 :ok :write [5 26] INFO [2022-01-10 12:53:40,892] jepsen worker 14 - jepsen.util 284 :ok :read [4 8] INFO [2022-01-10 12:53:40,893] jepsen worker 16 - jepsen.util 286 :invoke :write [5 2] INFO [2022-01-10 12:53:40,893] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:40,894] jepsen worker 14 - jepsen.util 284 :ok :read [4 8] INFO [2022-01-10 12:53:40,894] jepsen worker 5 - jepsen.util 185 :ok :write [1 29] INFO [2022-01-10 12:53:40,895] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:40,896] jepsen worker 14 - jepsen.util 284 :ok :read [4 8] INFO [2022-01-10 12:53:40,896] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:40,896] jepsen worker 14 - jepsen.util 284 :invoke :write [4 31] INFO [2022-01-10 12:53:40,897] jepsen worker 5 - jepsen.util 185 :ok :read [1 29] INFO [2022-01-10 12:53:40,897] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:40,898] jepsen worker 5 - jepsen.util 185 :ok :read [1 29] INFO [2022-01-10 12:53:40,899] jepsen worker 62 - jepsen.util 242 :ok :write [20 48] INFO [2022-01-10 12:53:40,899] jepsen worker 5 - jepsen.util 185 :invoke :write [1 33] INFO [2022-01-10 12:53:40,900] jepsen worker 48 - jepsen.util 228 :ok :write [16 9] INFO [2022-01-10 12:53:40,900] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,900] jepsen worker 48 - jepsen.util 228 :invoke :write [16 0] INFO [2022-01-10 12:53:40,901] jepsen worker 62 - jepsen.util 242 :ok :read [20 48] INFO [2022-01-10 12:53:40,901] jepsen worker 38 - jepsen.util 38 :ok :write [12 45] INFO [2022-01-10 12:53:40,902] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,902] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:40,903] jepsen worker 62 - jepsen.util 242 :ok :read [20 48] INFO [2022-01-10 12:53:40,903] jepsen worker 38 - jepsen.util 38 :ok :read [12 45] INFO [2022-01-10 12:53:40,904] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:40,905] jepsen worker 62 - jepsen.util 242 :invoke :read [20 nil] INFO [2022-01-10 12:53:40,905] jepsen worker 38 - jepsen.util 38 :ok :read [12 45] INFO [2022-01-10 12:53:40,906] jepsen worker 62 - jepsen.util 242 :ok :read [20 48] INFO [2022-01-10 12:53:40,906] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:40,907] jepsen worker 62 - jepsen.util 242 :invoke :write [20 21] INFO [2022-01-10 12:53:40,907] jepsen worker 38 - jepsen.util 38 :ok :read [12 45] INFO [2022-01-10 12:53:40,908] jepsen worker 83 - jepsen.util 263 :ok :write [27 27] INFO [2022-01-10 12:53:40,908] jepsen worker 38 - jepsen.util 38 :invoke :write [12 24] INFO [2022-01-10 12:53:40,908] jepsen worker 83 - jepsen.util 263 :invoke :write [27 33] INFO [2022-01-10 12:53:40,910] jepsen worker 76 - jepsen.util 256 :ok :write [25 7] INFO [2022-01-10 12:53:40,910] jepsen worker 69 - jepsen.util 159 :ok :write [23 23] INFO [2022-01-10 12:53:40,911] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,911] jepsen worker 69 - jepsen.util 159 :invoke :write [23 4] INFO [2022-01-10 12:53:40,912] jepsen worker 76 - jepsen.util 256 :ok :read [25 7] INFO [2022-01-10 12:53:40,912] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,913] jepsen worker 76 - jepsen.util 256 :ok :read [25 7] INFO [2022-01-10 12:53:40,914] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,914] jepsen worker 82 - jepsen.util 352 :info :write [27 4] :RequestTimeoutException INFO [2022-01-10 12:53:40,915] jepsen worker 76 - jepsen.util 256 :ok :read [25 7] INFO [2022-01-10 12:53:40,915] jepsen worker 2 - jepsen.util 92 :ok :write [0 30] INFO [2022-01-10 12:53:40,915] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,916] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:40,916] jepsen worker 76 - jepsen.util 256 :ok :read [25 7] INFO [2022-01-10 12:53:40,916] jepsen worker 82 - jepsen.util 442 :invoke :write [27 10] INFO [2022-01-10 12:53:40,917] jepsen worker 2 - jepsen.util 92 :ok :read [0 30] INFO [2022-01-10 12:53:40,917] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,918] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:40,918] jepsen worker 76 - jepsen.util 256 :ok :read [25 7] INFO [2022-01-10 12:53:40,919] jepsen worker 53 - jepsen.util 143 :ok :write [17 14] INFO [2022-01-10 12:53:40,919] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,919] jepsen worker 2 - jepsen.util 92 :ok :read [0 30] INFO [2022-01-10 12:53:40,919] jepsen worker 53 - jepsen.util 143 :invoke :write [17 7] INFO [2022-01-10 12:53:40,919] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:40,920] jepsen worker 76 - jepsen.util 256 :ok :read [25 7] INFO [2022-01-10 12:53:40,920] jepsen worker 76 - jepsen.util 256 :invoke :write [25 45] INFO [2022-01-10 12:53:40,920] jepsen worker 2 - jepsen.util 92 :ok :read [0 30] INFO [2022-01-10 12:53:40,921] jepsen worker 2 - jepsen.util 92 :invoke :write [0 35] INFO [2022-01-10 12:53:40,923] jepsen worker 68 - jepsen.util 68 :ok :write [22 21] INFO [2022-01-10 12:53:40,924] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,924] jepsen worker 59 - jepsen.util 329 :info :write [19 21] :RequestTimeoutException INFO [2022-01-10 12:53:40,925] jepsen worker 59 - jepsen.util 419 :invoke :read [19 nil] INFO [2022-01-10 12:53:40,925] jepsen worker 68 - jepsen.util 68 :ok :read [22 21] INFO [2022-01-10 12:53:40,925] jepsen worker 80 - jepsen.util 260 :ok :write [26 18] INFO [2022-01-10 12:53:40,926] jepsen worker 68 - jepsen.util 68 :invoke :write [22 30] INFO [2022-01-10 12:53:40,926] jepsen worker 59 - jepsen.util 419 :ok :read [19 48] INFO [2022-01-10 12:53:40,927] jepsen worker 80 - jepsen.util 260 :invoke :write [26 39] INFO [2022-01-10 12:53:40,927] jepsen worker 44 - jepsen.util 44 :ok :write [14 10] INFO [2022-01-10 12:53:40,927] jepsen worker 59 - jepsen.util 419 :invoke :read [19 nil] INFO [2022-01-10 12:53:40,928] jepsen worker 44 - jepsen.util 44 :invoke :write [14 23] INFO [2022-01-10 12:53:40,928] jepsen worker 59 - jepsen.util 419 :ok :read [19 48] INFO [2022-01-10 12:53:40,929] jepsen worker 59 - jepsen.util 419 :invoke :write [19 47] INFO [2022-01-10 12:53:40,932] jepsen worker 35 - jepsen.util 215 :ok :write [11 31] INFO [2022-01-10 12:53:40,933] jepsen worker 35 - jepsen.util 215 :invoke :write [11 10] INFO [2022-01-10 12:53:40,933] jepsen worker 86 - jepsen.util 176 :ok :write [28 36] INFO [2022-01-10 12:53:40,934] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:40,935] jepsen worker 86 - jepsen.util 176 :ok :read [28 36] INFO [2022-01-10 12:53:40,936] jepsen worker 86 - jepsen.util 176 :invoke :write [28 2] INFO [2022-01-10 12:53:40,937] jepsen worker 89 - jepsen.util 269 :ok :write [29 10] INFO [2022-01-10 12:53:40,938] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,938] jepsen worker 89 - jepsen.util 269 :ok :read [29 10] INFO [2022-01-10 12:53:40,939] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,941] jepsen worker 89 - jepsen.util 269 :ok :read [29 10] INFO [2022-01-10 12:53:40,941] jepsen worker 14 - jepsen.util 284 :ok :write [4 31] INFO [2022-01-10 12:53:40,941] jepsen worker 21 - jepsen.util 201 :ok :write [7 10] INFO [2022-01-10 12:53:40,942] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:40,942] jepsen worker 14 - jepsen.util 284 :invoke :write [4 25] INFO [2022-01-10 12:53:40,942] jepsen worker 21 - jepsen.util 201 :ok :read [7 10] INFO [2022-01-10 12:53:40,943] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,943] jepsen worker 7 - jepsen.util 7 :ok :write [2 26] INFO [2022-01-10 12:53:40,944] jepsen worker 21 - jepsen.util 201 :invoke :write [7 3] INFO [2022-01-10 12:53:40,944] jepsen worker 89 - jepsen.util 269 :ok :read [29 10] INFO [2022-01-10 12:53:40,944] jepsen worker 7 - jepsen.util 7 :invoke :write [2 41] INFO [2022-01-10 12:53:40,945] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,946] jepsen worker 89 - jepsen.util 269 :ok :read [29 10] INFO [2022-01-10 12:53:40,946] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:40,947] jepsen worker 89 - jepsen.util 269 :ok :read [29 10] INFO [2022-01-10 12:53:40,947] jepsen worker 40 - jepsen.util 130 :ok :write [13 25] INFO [2022-01-10 12:53:40,948] jepsen worker 5 - jepsen.util 185 :ok :write [1 33] INFO [2022-01-10 12:53:40,948] jepsen worker 40 - jepsen.util 130 :invoke :write [13 6] INFO [2022-01-10 12:53:40,949] jepsen worker 89 - jepsen.util 269 :invoke :write [29 6] INFO [2022-01-10 12:53:40,949] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:40,950] jepsen worker 5 - jepsen.util 185 :ok :read [1 33] INFO [2022-01-10 12:53:40,951] jepsen worker 5 - jepsen.util 185 :invoke :write [1 27] INFO [2022-01-10 12:53:40,956] jepsen worker 16 - jepsen.util 286 :ok :write [5 2] INFO [2022-01-10 12:53:40,956] jepsen worker 62 - jepsen.util 242 :ok :write [20 21] INFO [2022-01-10 12:53:40,957] jepsen worker 16 - jepsen.util 286 :invoke :write [5 37] INFO [2022-01-10 12:53:40,957] jepsen worker 62 - jepsen.util 242 :invoke :write [20 5] INFO [2022-01-10 12:53:40,958] jepsen worker 38 - jepsen.util 38 :ok :write [12 24] INFO [2022-01-10 12:53:40,958] jepsen worker 38 - jepsen.util 38 :invoke :write [12 1] INFO [2022-01-10 12:53:40,961] jepsen worker 48 - jepsen.util 228 :ok :write [16 0] INFO [2022-01-10 12:53:40,962] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:40,963] jepsen worker 48 - jepsen.util 228 :ok :read [16 0] INFO [2022-01-10 12:53:40,963] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:40,964] jepsen worker 48 - jepsen.util 228 :ok :read [16 0] INFO [2022-01-10 12:53:40,964] jepsen worker 48 - jepsen.util 228 :invoke :write [16 23] INFO [2022-01-10 12:53:40,967] jepsen worker 69 - jepsen.util 159 :ok :write [23 4] INFO [2022-01-10 12:53:40,967] jepsen worker 69 - jepsen.util 159 :invoke :write [23 42] INFO [2022-01-10 12:53:40,970] jepsen worker 83 - jepsen.util 263 :ok :write [27 33] INFO [2022-01-10 12:53:40,970] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,971] jepsen worker 83 - jepsen.util 263 :ok :read [27 33] INFO [2022-01-10 12:53:40,971] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,972] jepsen worker 76 - jepsen.util 256 :ok :write [25 45] INFO [2022-01-10 12:53:40,973] jepsen worker 83 - jepsen.util 263 :ok :read [27 33] INFO [2022-01-10 12:53:40,973] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:40,973] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:40,974] jepsen worker 53 - jepsen.util 143 :ok :write [17 7] INFO [2022-01-10 12:53:40,974] jepsen worker 76 - jepsen.util 256 :ok :read [25 45] INFO [2022-01-10 12:53:40,974] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:40,974] jepsen worker 83 - jepsen.util 263 :ok :read [27 33] INFO [2022-01-10 12:53:40,975] jepsen worker 76 - jepsen.util 256 :invoke :write [25 36] INFO [2022-01-10 12:53:40,975] jepsen worker 53 - jepsen.util 143 :ok :read [17 7] INFO [2022-01-10 12:53:40,975] jepsen worker 83 - jepsen.util 263 :invoke :write [27 26] INFO [2022-01-10 12:53:40,975] jepsen worker 53 - jepsen.util 143 :invoke :write [17 24] INFO [2022-01-10 12:53:40,976] jepsen worker 2 - jepsen.util 92 :ok :write [0 35] INFO [2022-01-10 12:53:40,977] jepsen worker 2 - jepsen.util 92 :invoke :write [0 0] INFO [2022-01-10 12:53:40,981] jepsen worker 80 - jepsen.util 260 :ok :write [26 39] INFO [2022-01-10 12:53:40,982] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:40,983] jepsen worker 80 - jepsen.util 260 :ok :read [26 39] INFO [2022-01-10 12:53:40,983] jepsen worker 80 - jepsen.util 260 :invoke :write [26 4] INFO [2022-01-10 12:53:40,986] jepsen worker 44 - jepsen.util 44 :ok :write [14 23] INFO [2022-01-10 12:53:40,987] jepsen worker 54 - jepsen.util 324 :info :write [18 45] :RequestTimeoutException INFO [2022-01-10 12:53:40,987] jepsen worker 68 - jepsen.util 68 :ok :write [22 30] INFO [2022-01-10 12:53:40,987] jepsen worker 44 - jepsen.util 44 :invoke :write [14 20] INFO [2022-01-10 12:53:40,987] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,988] jepsen worker 54 - jepsen.util 414 :invoke :read [18 nil] INFO [2022-01-10 12:53:40,988] jepsen worker 68 - jepsen.util 68 :ok :read [22 30] INFO [2022-01-10 12:53:40,989] jepsen worker 54 - jepsen.util 414 :ok :read [18 46] INFO [2022-01-10 12:53:40,989] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,989] jepsen worker 54 - jepsen.util 414 :invoke :write [18 8] INFO [2022-01-10 12:53:40,990] jepsen worker 68 - jepsen.util 68 :ok :read [22 30] INFO [2022-01-10 12:53:40,990] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,991] jepsen worker 68 - jepsen.util 68 :ok :read [22 30] INFO [2022-01-10 12:53:40,992] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:40,993] jepsen worker 68 - jepsen.util 68 :ok :read [22 30] INFO [2022-01-10 12:53:40,993] jepsen worker 68 - jepsen.util 68 :invoke :write [22 6] INFO [2022-01-10 12:53:40,993] jepsen worker 35 - jepsen.util 215 :ok :write [11 10] INFO [2022-01-10 12:53:40,994] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:40,995] jepsen worker 35 - jepsen.util 215 :ok :read [11 10] INFO [2022-01-10 12:53:40,995] jepsen worker 35 - jepsen.util 215 :invoke :write [11 44] INFO [2022-01-10 12:53:40,998] jepsen worker 89 - jepsen.util 269 :ok :write [29 6] INFO [2022-01-10 12:53:40,999] jepsen worker 21 - jepsen.util 201 :ok :write [7 3] INFO [2022-01-10 12:53:40,999] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,000] jepsen worker 21 - jepsen.util 201 :invoke :write [7 4] INFO [2022-01-10 12:53:41,000] jepsen worker 89 - jepsen.util 269 :ok :read [29 6] INFO [2022-01-10 12:53:41,001] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,001] jepsen worker 89 - jepsen.util 269 :ok :read [29 6] INFO [2022-01-10 12:53:41,002] jepsen worker 89 - jepsen.util 269 :invoke :write [29 31] INFO [2022-01-10 12:53:41,005] jepsen worker 7 - jepsen.util 7 :ok :write [2 41] INFO [2022-01-10 12:53:41,006] jepsen worker 7 - jepsen.util 7 :invoke :write [2 42] INFO [2022-01-10 12:53:41,008] jepsen worker 40 - jepsen.util 130 :ok :write [13 6] INFO [2022-01-10 12:53:41,009] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:41,009] jepsen worker 40 - jepsen.util 130 :ok :read [13 6] INFO [2022-01-10 12:53:41,010] jepsen worker 60 - jepsen.util 240 :ok :write [20 21] INFO [2022-01-10 12:53:41,010] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:41,011] jepsen worker 60 - jepsen.util 240 :invoke :write [20 22] INFO [2022-01-10 12:53:41,011] jepsen worker 40 - jepsen.util 130 :ok :read [13 6] INFO [2022-01-10 12:53:41,012] jepsen worker 40 - jepsen.util 130 :invoke :write [13 9] INFO [2022-01-10 12:53:41,012] jepsen worker 46 - jepsen.util 316 :info :write [15 16] :RequestTimeoutException INFO [2022-01-10 12:53:41,013] jepsen worker 46 - jepsen.util 406 :invoke :read [15 nil] INFO [2022-01-10 12:53:41,017] jepsen worker 16 - jepsen.util 286 :ok :write [5 37] INFO [2022-01-10 12:53:41,017] jepsen worker 16 - jepsen.util 286 :invoke :write [5 10] INFO [2022-01-10 12:53:41,018] jepsen worker 72 - jepsen.util 342 :info :write [24 19] :RequestTimeoutException INFO [2022-01-10 12:53:41,019] jepsen worker 72 - jepsen.util 432 :invoke :read [24 nil] INFO [2022-01-10 12:53:41,020] jepsen worker 72 - jepsen.util 432 :ok :read [24 37] INFO [2022-01-10 12:53:41,020] jepsen worker 72 - jepsen.util 432 :invoke :write [24 44] INFO [2022-01-10 12:53:41,021] jepsen worker 48 - jepsen.util 228 :ok :write [16 23] INFO [2022-01-10 12:53:41,021] jepsen worker 48 - jepsen.util 228 :invoke :write [16 20] INFO [2022-01-10 12:53:41,025] jepsen worker 83 - jepsen.util 263 :ok :write [27 26] INFO [2022-01-10 12:53:41,025] jepsen worker 83 - jepsen.util 263 :invoke :write [27 48] INFO [2022-01-10 12:53:41,030] jepsen worker 76 - jepsen.util 256 :ok :write [25 36] INFO [2022-01-10 12:53:41,030] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:41,031] jepsen worker 76 - jepsen.util 256 :ok :read [25 36] INFO [2022-01-10 12:53:41,032] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:41,033] jepsen worker 76 - jepsen.util 256 :ok :read [25 36] INFO [2022-01-10 12:53:41,033] jepsen worker 76 - jepsen.util 256 :invoke :write [25 45] INFO [2022-01-10 12:53:41,064] jepsen worker 25 - jepsen.util 295 :info :write [8 10] :RequestTimeoutException INFO [2022-01-10 12:53:41,064] jepsen worker 25 - jepsen.util 385 :invoke :read [8 nil] INFO [2022-01-10 12:53:41,065] jepsen worker 25 - jepsen.util 385 :ok :read [8 25] INFO [2022-01-10 12:53:41,066] jepsen worker 25 - jepsen.util 385 :invoke :read [8 nil] INFO [2022-01-10 12:53:41,067] jepsen worker 25 - jepsen.util 385 :ok :read [8 25] INFO [2022-01-10 12:53:41,067] jepsen worker 25 - jepsen.util 385 :invoke :read [8 nil] INFO [2022-01-10 12:53:41,068] jepsen worker 25 - jepsen.util 385 :ok :read [8 25] INFO [2022-01-10 12:53:41,068] jepsen worker 25 - jepsen.util 385 :invoke :read [8 nil] INFO [2022-01-10 12:53:41,069] jepsen worker 25 - jepsen.util 385 :ok :read [8 25] INFO [2022-01-10 12:53:41,069] jepsen worker 25 - jepsen.util 385 :invoke :read [8 nil] INFO [2022-01-10 12:53:41,070] jepsen worker 25 - jepsen.util 385 :ok :read [8 25] INFO [2022-01-10 12:53:41,071] jepsen worker 25 - jepsen.util 385 :invoke :write [8 24] INFO [2022-01-10 12:53:41,084] jepsen worker 24 - jepsen.util 294 :info :write [8 14] :RequestTimeoutException INFO [2022-01-10 12:53:41,084] jepsen worker 2 - jepsen.util 92 :ok :write [0 0] INFO [2022-01-10 12:53:41,084] jepsen worker 24 - jepsen.util 384 :invoke :read [8 nil] INFO [2022-01-10 12:53:41,084] jepsen worker 86 - jepsen.util 176 :ok :write [28 2] INFO [2022-01-10 12:53:41,085] jepsen worker 2 - jepsen.util 92 :invoke :write [0 4] INFO [2022-01-10 12:53:41,085] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,085] jepsen worker 24 - jepsen.util 384 :ok :read [8 25] INFO [2022-01-10 12:53:41,086] jepsen worker 86 - jepsen.util 176 :ok :read [28 2] INFO [2022-01-10 12:53:41,086] jepsen worker 24 - jepsen.util 384 :invoke :read [8 nil] INFO [2022-01-10 12:53:41,086] jepsen worker 86 - jepsen.util 176 :invoke :write [28 12] INFO [2022-01-10 12:53:41,087] jepsen worker 24 - jepsen.util 384 :ok :read [8 25] INFO [2022-01-10 12:53:41,087] jepsen worker 24 - jepsen.util 384 :invoke :read [8 nil] INFO [2022-01-10 12:53:41,088] jepsen worker 24 - jepsen.util 384 :ok :read [8 25] INFO [2022-01-10 12:53:41,088] jepsen worker 24 - jepsen.util 384 :invoke :write [8 13] INFO [2022-01-10 12:53:41,089] jepsen worker 14 - jepsen.util 284 :ok :write [4 25] INFO [2022-01-10 12:53:41,090] jepsen worker 14 - jepsen.util 284 :invoke :write [4 38] INFO [2022-01-10 12:53:41,091] jepsen worker 44 - jepsen.util 44 :ok :write [14 20] INFO [2022-01-10 12:53:41,091] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:41,092] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:41,093] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:41,094] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:41,094] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:41,095] jepsen worker 44 - jepsen.util 44 :ok :read [14 20] INFO [2022-01-10 12:53:41,096] jepsen worker 5 - jepsen.util 185 :ok :write [1 27] INFO [2022-01-10 12:53:41,096] jepsen worker 44 - jepsen.util 44 :invoke :write [14 30] INFO [2022-01-10 12:53:41,097] jepsen worker 5 - jepsen.util 185 :invoke :write [1 43] INFO [2022-01-10 12:53:41,100] jepsen worker 21 - jepsen.util 201 :ok :write [7 4] INFO [2022-01-10 12:53:41,101] jepsen worker 21 - jepsen.util 201 :invoke :write [7 27] INFO [2022-01-10 12:53:41,105] jepsen worker 38 - jepsen.util 38 :ok :write [12 1] INFO [2022-01-10 12:53:41,105] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,106] jepsen worker 38 - jepsen.util 38 :ok :read [12 1] INFO [2022-01-10 12:53:41,107] jepsen worker 38 - jepsen.util 38 :invoke :write [12 44] INFO [2022-01-10 12:53:41,109] jepsen worker 68 - jepsen.util 68 :ok :write [22 6] INFO [2022-01-10 12:53:41,110] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:41,110] jepsen worker 68 - jepsen.util 68 :ok :read [22 6] INFO [2022-01-10 12:53:41,111] jepsen worker 68 - jepsen.util 68 :invoke :write [22 32] INFO [2022-01-10 12:53:41,112] jepsen worker 60 - jepsen.util 240 :ok :write [20 22] INFO [2022-01-10 12:53:41,113] jepsen worker 60 - jepsen.util 240 :invoke :write [20 10] INFO [2022-01-10 12:53:41,113] jepsen worker 69 - jepsen.util 159 :ok :write [23 42] INFO [2022-01-10 12:53:41,114] jepsen worker 69 - jepsen.util 159 :invoke :write [23 35] INFO [2022-01-10 12:53:41,118] jepsen worker 76 - jepsen.util 256 :ok :write [25 45] INFO [2022-01-10 12:53:41,118] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:41,119] jepsen worker 53 - jepsen.util 143 :ok :write [17 24] INFO [2022-01-10 12:53:41,119] jepsen worker 35 - jepsen.util 215 :ok :write [11 44] INFO [2022-01-10 12:53:41,119] jepsen worker 53 - jepsen.util 143 :invoke :write [17 33] INFO [2022-01-10 12:53:41,120] jepsen worker 35 - jepsen.util 215 :invoke :write [11 7] INFO [2022-01-10 12:53:41,123] jepsen worker 76 - jepsen.util 256 :ok :read [25 45] INFO [2022-01-10 12:53:41,124] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:41,125] jepsen worker 76 - jepsen.util 256 :ok :read [25 45] INFO [2022-01-10 12:53:41,126] jepsen worker 76 - jepsen.util 256 :invoke :write [25 37] INFO [2022-01-10 12:53:41,127] jepsen worker 89 - jepsen.util 269 :ok :write [29 31] INFO [2022-01-10 12:53:41,127] jepsen worker 89 - jepsen.util 269 :invoke :write [29 27] INFO [2022-01-10 12:53:41,129] jepsen worker 80 - jepsen.util 260 :ok :write [26 4] INFO [2022-01-10 12:53:41,130] jepsen worker 80 - jepsen.util 260 :invoke :write [26 7] INFO [2022-01-10 12:53:41,132] jepsen worker 44 - jepsen.util 44 :ok :write [14 30] INFO [2022-01-10 12:53:41,132] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:41,135] jepsen worker 44 - jepsen.util 44 :ok :read [14 30] INFO [2022-01-10 12:53:41,135] jepsen worker 44 - jepsen.util 44 :invoke :write [14 32] INFO [2022-01-10 12:53:41,136] jepsen worker 86 - jepsen.util 176 :ok :write [28 12] INFO [2022-01-10 12:53:41,136] jepsen worker 7 - jepsen.util 7 :ok :write [2 42] INFO [2022-01-10 12:53:41,137] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:41,137] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,138] jepsen worker 86 - jepsen.util 176 :ok :read [28 12] INFO [2022-01-10 12:53:41,138] jepsen worker 21 - jepsen.util 201 :ok :write [7 27] INFO [2022-01-10 12:53:41,138] jepsen worker 7 - jepsen.util 7 :ok :read [2 42] INFO [2022-01-10 12:53:41,139] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,140] jepsen worker 40 - jepsen.util 130 :ok :write [13 9] INFO [2022-01-10 12:53:41,140] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:41,140] jepsen worker 86 - jepsen.util 176 :ok :read [28 12] INFO [2022-01-10 12:53:41,140] jepsen worker 14 - jepsen.util 284 :ok :write [4 38] INFO [2022-01-10 12:53:41,141] jepsen worker 7 - jepsen.util 7 :invoke :write [2 42] INFO [2022-01-10 12:53:41,141] jepsen worker 21 - jepsen.util 201 :ok :read [7 27] INFO [2022-01-10 12:53:41,141] jepsen worker 14 - jepsen.util 284 :invoke :write [4 17] INFO [2022-01-10 12:53:41,142] jepsen worker 40 - jepsen.util 130 :invoke :write [13 13] INFO [2022-01-10 12:53:41,142] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:41,142] jepsen worker 60 - jepsen.util 240 :ok :write [20 10] INFO [2022-01-10 12:53:41,143] jepsen worker 86 - jepsen.util 176 :invoke :write [28 34] INFO [2022-01-10 12:53:41,143] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:41,144] jepsen worker 60 - jepsen.util 240 :ok :read [20 10] INFO [2022-01-10 12:53:41,145] jepsen worker 60 - jepsen.util 240 :invoke :write [20 29] INFO [2022-01-10 12:53:41,145] jepsen worker 21 - jepsen.util 201 :ok :read [7 27] INFO [2022-01-10 12:53:41,146] jepsen worker 5 - jepsen.util 185 :ok :write [1 43] INFO [2022-01-10 12:53:41,146] jepsen worker 21 - jepsen.util 201 :invoke :write [7 2] INFO [2022-01-10 12:53:41,146] jepsen worker 5 - jepsen.util 185 :invoke :write [1 12] INFO [2022-01-10 12:53:41,148] jepsen worker 16 - jepsen.util 286 :ok :write [5 10] INFO [2022-01-10 12:53:41,149] jepsen worker 16 - jepsen.util 286 :invoke :write [5 14] INFO [2022-01-10 12:53:41,150] jepsen worker 76 - jepsen.util 256 :ok :write [25 37] INFO [2022-01-10 12:53:41,150] jepsen worker 76 - jepsen.util 256 :invoke :write [25 0] INFO [2022-01-10 12:53:41,153] jepsen worker 38 - jepsen.util 38 :ok :write [12 44] INFO [2022-01-10 12:53:41,154] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,155] jepsen worker 38 - jepsen.util 38 :ok :read [12 44] INFO [2022-01-10 12:53:41,155] jepsen worker 38 - jepsen.util 38 :invoke :write [12 35] INFO [2022-01-10 12:53:41,157] jepsen worker 48 - jepsen.util 228 :ok :write [16 20] INFO [2022-01-10 12:53:41,158] jepsen worker 42 - jepsen.util 222 :ok :write [14 1] INFO [2022-01-10 12:53:41,158] jepsen worker 48 - jepsen.util 228 :invoke :write [16 39] INFO [2022-01-10 12:53:41,158] jepsen worker 69 - jepsen.util 159 :ok :write [23 35] INFO [2022-01-10 12:53:41,158] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:41,159] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,159] jepsen worker 42 - jepsen.util 222 :ok :read [14 1] INFO [2022-01-10 12:53:41,160] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:41,161] jepsen worker 69 - jepsen.util 159 :ok :read [23 35] INFO [2022-01-10 12:53:41,161] jepsen worker 42 - jepsen.util 222 :ok :read [14 1] INFO [2022-01-10 12:53:41,161] jepsen worker 69 - jepsen.util 159 :invoke :write [23 32] INFO [2022-01-10 12:53:41,162] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:41,163] jepsen worker 42 - jepsen.util 222 :ok :read [14 1] INFO [2022-01-10 12:53:41,163] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:41,165] jepsen worker 83 - jepsen.util 263 :ok :write [27 48] INFO [2022-01-10 12:53:41,166] jepsen worker 53 - jepsen.util 143 :ok :write [17 33] INFO [2022-01-10 12:53:41,166] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:41,166] jepsen worker 21 - jepsen.util 201 :ok :write [7 2] INFO [2022-01-10 12:53:41,167] jepsen worker 42 - jepsen.util 222 :ok :read [14 1] INFO [2022-01-10 12:53:41,167] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:41,167] jepsen worker 83 - jepsen.util 263 :ok :read [27 48] INFO [2022-01-10 12:53:41,168] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:41,168] jepsen worker 53 - jepsen.util 143 :ok :read [17 33] INFO [2022-01-10 12:53:41,168] jepsen worker 83 - jepsen.util 263 :invoke :write [27 4] INFO [2022-01-10 12:53:41,169] jepsen worker 42 - jepsen.util 222 :ok :read [14 1] INFO [2022-01-10 12:53:41,169] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:41,170] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:41,170] jepsen worker 21 - jepsen.util 201 :ok :read [7 2] INFO [2022-01-10 12:53:41,170] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:41,171] jepsen worker 53 - jepsen.util 143 :ok :read [17 33] INFO [2022-01-10 12:53:41,171] jepsen worker 21 - jepsen.util 201 :invoke :write [7 9] INFO [2022-01-10 12:53:41,171] jepsen worker 42 - jepsen.util 222 :ok :read [14 1] INFO [2022-01-10 12:53:41,172] jepsen worker 53 - jepsen.util 143 :invoke :write [17 24] INFO [2022-01-10 12:53:41,172] jepsen worker 42 - jepsen.util 222 :invoke :write [14 8] INFO [2022-01-10 12:53:41,173] jepsen worker 60 - jepsen.util 240 :ok :write [20 29] INFO [2022-01-10 12:53:41,173] jepsen worker 80 - jepsen.util 260 :ok :write [26 7] INFO [2022-01-10 12:53:41,174] jepsen worker 2 - jepsen.util 92 :ok :write [0 4] INFO [2022-01-10 12:53:41,174] jepsen worker 80 - jepsen.util 260 :invoke :write [26 31] INFO [2022-01-10 12:53:41,175] jepsen worker 60 - jepsen.util 240 :invoke :write [20 42] INFO [2022-01-10 12:53:41,176] jepsen worker 2 - jepsen.util 92 :invoke :write [0 37] INFO [2022-01-10 12:53:41,180] jepsen worker 76 - jepsen.util 256 :ok :write [25 0] INFO [2022-01-10 12:53:41,181] jepsen worker 86 - jepsen.util 176 :ok :write [28 34] INFO [2022-01-10 12:53:41,181] jepsen worker 76 - jepsen.util 256 :invoke :write [25 23] INFO [2022-01-10 12:53:41,182] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,183] jepsen worker 68 - jepsen.util 68 :ok :write [22 32] INFO [2022-01-10 12:53:41,183] jepsen worker 86 - jepsen.util 176 :ok :read [28 34] INFO [2022-01-10 12:53:41,183] jepsen worker 68 - jepsen.util 68 :invoke :write [22 41] INFO [2022-01-10 12:53:41,184] jepsen worker 86 - jepsen.util 176 :invoke :write [28 11] INFO [2022-01-10 12:53:41,185] jepsen worker 14 - jepsen.util 284 :ok :write [4 17] INFO [2022-01-10 12:53:41,186] jepsen worker 14 - jepsen.util 284 :invoke :write [4 5] INFO [2022-01-10 12:53:41,191] jepsen worker 5 - jepsen.util 185 :ok :write [1 12] INFO [2022-01-10 12:53:41,191] jepsen worker 5 - jepsen.util 185 :invoke :write [1 2] INFO [2022-01-10 12:53:41,194] jepsen worker 35 - jepsen.util 215 :ok :write [11 7] INFO [2022-01-10 12:53:41,195] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:41,196] jepsen worker 35 - jepsen.util 215 :ok :read [11 7] INFO [2022-01-10 12:53:41,196] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:41,197] jepsen worker 35 - jepsen.util 215 :ok :read [11 7] INFO [2022-01-10 12:53:41,197] jepsen worker 35 - jepsen.util 215 :invoke :write [11 45] INFO [2022-01-10 12:53:41,198] jepsen worker 38 - jepsen.util 38 :ok :write [12 35] INFO [2022-01-10 12:53:41,199] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,199] jepsen worker 38 - jepsen.util 38 :ok :read [12 35] INFO [2022-01-10 12:53:41,200] jepsen worker 89 - jepsen.util 269 :ok :write [29 27] INFO [2022-01-10 12:53:41,200] jepsen worker 38 - jepsen.util 38 :invoke :write [12 46] INFO [2022-01-10 12:53:41,200] jepsen worker 89 - jepsen.util 269 :invoke :write [29 17] INFO [2022-01-10 12:53:41,204] jepsen worker 7 - jepsen.util 7 :ok :write [2 42] INFO [2022-01-10 12:53:41,205] jepsen worker 69 - jepsen.util 159 :ok :write [23 32] INFO [2022-01-10 12:53:41,205] jepsen worker 7 - jepsen.util 7 :invoke :write [2 10] INFO [2022-01-10 12:53:41,205] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,206] jepsen worker 69 - jepsen.util 159 :ok :read [23 32] INFO [2022-01-10 12:53:41,207] jepsen worker 69 - jepsen.util 159 :invoke :write [23 34] INFO [2022-01-10 12:53:41,211] jepsen worker 40 - jepsen.util 130 :ok :write [13 13] INFO [2022-01-10 12:53:41,211] jepsen worker 40 - jepsen.util 130 :invoke :write [13 30] INFO [2022-01-10 12:53:41,221] jepsen worker 86 - jepsen.util 176 :ok :write [28 11] INFO [2022-01-10 12:53:41,222] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,222] jepsen worker 86 - jepsen.util 176 :ok :read [28 11] INFO [2022-01-10 12:53:41,223] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,224] jepsen worker 86 - jepsen.util 176 :ok :read [28 11] INFO [2022-01-10 12:53:41,224] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,225] jepsen worker 86 - jepsen.util 176 :ok :read [28 11] INFO [2022-01-10 12:53:41,226] jepsen worker 86 - jepsen.util 176 :invoke :write [28 23] INFO [2022-01-10 12:53:41,229] jepsen worker 5 - jepsen.util 185 :ok :write [1 2] INFO [2022-01-10 12:53:41,229] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,230] jepsen worker 5 - jepsen.util 185 :ok :read [1 2] INFO [2022-01-10 12:53:41,231] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,231] jepsen worker 2 - jepsen.util 92 :ok :write [0 37] INFO [2022-01-10 12:53:41,232] jepsen worker 5 - jepsen.util 185 :ok :read [1 2] INFO [2022-01-10 12:53:41,232] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,233] jepsen worker 5 - jepsen.util 185 :invoke :write [1 17] INFO [2022-01-10 12:53:41,233] jepsen worker 2 - jepsen.util 92 :ok :read [0 37] INFO [2022-01-10 12:53:41,233] jepsen worker 38 - jepsen.util 38 :ok :write [12 46] INFO [2022-01-10 12:53:41,234] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,235] jepsen worker 38 - jepsen.util 38 :ok :read [12 46] INFO [2022-01-10 12:53:41,235] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,236] jepsen worker 2 - jepsen.util 92 :ok :read [0 37] INFO [2022-01-10 12:53:41,236] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,237] jepsen worker 38 - jepsen.util 38 :ok :read [12 46] INFO [2022-01-10 12:53:41,237] jepsen worker 2 - jepsen.util 92 :invoke :write [0 21] INFO [2022-01-10 12:53:41,237] jepsen worker 38 - jepsen.util 38 :invoke :write [12 45] INFO [2022-01-10 12:53:41,246] jepsen worker 86 - jepsen.util 176 :ok :write [28 23] INFO [2022-01-10 12:53:41,247] jepsen worker 86 - jepsen.util 176 :invoke :write [28 9] INFO [2022-01-10 12:53:41,252] jepsen worker 5 - jepsen.util 185 :ok :write [1 17] INFO [2022-01-10 12:53:41,252] jepsen worker 89 - jepsen.util 269 :ok :write [29 17] INFO [2022-01-10 12:53:41,252] jepsen worker 5 - jepsen.util 185 :invoke :write [1 17] INFO [2022-01-10 12:53:41,253] jepsen worker 89 - jepsen.util 269 :invoke :write [29 10] INFO [2022-01-10 12:53:41,257] jepsen worker 38 - jepsen.util 38 :ok :write [12 45] INFO [2022-01-10 12:53:41,258] jepsen worker 38 - jepsen.util 38 :invoke :write [12 27] INFO [2022-01-10 12:53:41,262] jepsen worker 40 - jepsen.util 130 :ok :write [13 30] INFO [2022-01-10 12:53:41,263] jepsen worker 40 - jepsen.util 130 :invoke :write [13 22] INFO [2022-01-10 12:53:41,264] jepsen worker 86 - jepsen.util 176 :ok :write [28 9] INFO [2022-01-10 12:53:41,264] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,265] jepsen worker 86 - jepsen.util 176 :ok :read [28 9] INFO [2022-01-10 12:53:41,266] jepsen worker 86 - jepsen.util 176 :invoke :write [28 0] INFO [2022-01-10 12:53:41,271] jepsen worker 2 - jepsen.util 92 :ok :write [0 21] INFO [2022-01-10 12:53:41,272] jepsen worker 5 - jepsen.util 185 :ok :write [1 17] INFO [2022-01-10 12:53:41,272] jepsen worker 2 - jepsen.util 92 :invoke :write [0 0] INFO [2022-01-10 12:53:41,273] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,274] jepsen worker 5 - jepsen.util 185 :ok :read [1 17] INFO [2022-01-10 12:53:41,274] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,275] jepsen worker 5 - jepsen.util 185 :ok :read [1 17] INFO [2022-01-10 12:53:41,275] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,276] jepsen worker 5 - jepsen.util 185 :ok :read [1 17] INFO [2022-01-10 12:53:41,277] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,277] jepsen worker 38 - jepsen.util 38 :ok :write [12 27] INFO [2022-01-10 12:53:41,278] jepsen worker 5 - jepsen.util 185 :ok :read [1 17] INFO [2022-01-10 12:53:41,278] jepsen worker 38 - jepsen.util 38 :invoke :write [12 37] INFO [2022-01-10 12:53:41,279] jepsen worker 89 - jepsen.util 269 :ok :write [29 10] INFO [2022-01-10 12:53:41,279] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,280] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,280] jepsen worker 5 - jepsen.util 185 :ok :read [1 17] INFO [2022-01-10 12:53:41,281] jepsen worker 89 - jepsen.util 269 :ok :read [29 10] INFO [2022-01-10 12:53:41,281] jepsen worker 89 - jepsen.util 269 :invoke :write [29 35] INFO [2022-01-10 12:53:41,282] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,282] jepsen worker 86 - jepsen.util 176 :ok :write [28 0] INFO [2022-01-10 12:53:41,283] jepsen worker 5 - jepsen.util 185 :ok :read [1 17] INFO [2022-01-10 12:53:41,283] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,284] jepsen worker 5 - jepsen.util 185 :invoke :write [1 9] INFO [2022-01-10 12:53:41,284] jepsen worker 86 - jepsen.util 176 :ok :read [28 0] INFO [2022-01-10 12:53:41,284] jepsen worker 40 - jepsen.util 130 :ok :write [13 22] INFO [2022-01-10 12:53:41,284] jepsen worker 86 - jepsen.util 176 :invoke :write [28 10] INFO [2022-01-10 12:53:41,285] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:41,286] jepsen worker 40 - jepsen.util 130 :ok :read [13 22] INFO [2022-01-10 12:53:41,286] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:41,287] jepsen worker 40 - jepsen.util 130 :ok :read [13 22] INFO [2022-01-10 12:53:41,288] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:41,288] jepsen worker 40 - jepsen.util 130 :ok :read [13 22] INFO [2022-01-10 12:53:41,289] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:41,290] jepsen worker 40 - jepsen.util 130 :ok :read [13 22] INFO [2022-01-10 12:53:41,290] jepsen worker 40 - jepsen.util 130 :invoke :write [13 49] INFO [2022-01-10 12:53:41,293] jepsen worker 2 - jepsen.util 92 :ok :write [0 0] INFO [2022-01-10 12:53:41,294] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,295] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:41,295] jepsen worker 2 - jepsen.util 92 :invoke :write [0 14] INFO [2022-01-10 12:53:41,296] jepsen worker 38 - jepsen.util 38 :ok :write [12 37] INFO [2022-01-10 12:53:41,296] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,297] jepsen worker 38 - jepsen.util 38 :ok :read [12 37] INFO [2022-01-10 12:53:41,297] jepsen worker 38 - jepsen.util 38 :invoke :write [12 18] INFO [2022-01-10 12:53:41,304] jepsen worker 89 - jepsen.util 269 :ok :write [29 35] INFO [2022-01-10 12:53:41,304] jepsen worker 86 - jepsen.util 176 :ok :write [28 10] INFO [2022-01-10 12:53:41,305] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,306] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,306] jepsen worker 89 - jepsen.util 269 :ok :read [29 35] INFO [2022-01-10 12:53:41,307] jepsen worker 86 - jepsen.util 176 :ok :read [28 10] INFO [2022-01-10 12:53:41,307] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,308] jepsen worker 86 - jepsen.util 176 :invoke :write [28 23] INFO [2022-01-10 12:53:41,308] jepsen worker 89 - jepsen.util 269 :ok :read [29 35] INFO [2022-01-10 12:53:41,309] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,309] jepsen worker 89 - jepsen.util 269 :ok :read [29 35] INFO [2022-01-10 12:53:41,310] jepsen worker 89 - jepsen.util 269 :invoke :write [29 18] INFO [2022-01-10 12:53:41,311] jepsen worker 40 - jepsen.util 130 :ok :write [13 49] INFO [2022-01-10 12:53:41,311] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:41,312] jepsen worker 40 - jepsen.util 130 :ok :read [13 49] INFO [2022-01-10 12:53:41,313] jepsen worker 5 - jepsen.util 185 :ok :write [1 9] INFO [2022-01-10 12:53:41,313] jepsen worker 40 - jepsen.util 130 :invoke :write [13 37] INFO [2022-01-10 12:53:41,313] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,314] jepsen worker 5 - jepsen.util 185 :ok :read [1 9] INFO [2022-01-10 12:53:41,314] jepsen worker 5 - jepsen.util 185 :invoke :write [1 8] INFO [2022-01-10 12:53:41,440] jepsen worker 16 - jepsen.util 286 :ok :write [5 14] INFO [2022-01-10 12:53:41,440] jepsen worker 80 - jepsen.util 260 :ok :write [26 31] INFO [2022-01-10 12:53:41,441] jepsen worker 16 - jepsen.util 286 :invoke :write [5 38] INFO [2022-01-10 12:53:41,441] jepsen worker 80 - jepsen.util 260 :invoke :write [26 36] INFO [2022-01-10 12:53:41,444] jepsen worker 44 - jepsen.util 44 :ok :write [14 32] INFO [2022-01-10 12:53:41,445] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:41,449] jepsen worker 48 - jepsen.util 228 :ok :write [16 39] INFO [2022-01-10 12:53:41,449] jepsen worker 53 - jepsen.util 143 :ok :write [17 24] INFO [2022-01-10 12:53:41,449] jepsen worker 2 - jepsen.util 92 :ok :write [0 14] INFO [2022-01-10 12:53:41,450] jepsen worker 48 - jepsen.util 228 :invoke :write [16 43] INFO [2022-01-10 12:53:41,450] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,451] jepsen worker 53 - jepsen.util 143 :invoke :write [17 21] INFO [2022-01-10 12:53:41,451] jepsen worker 38 - jepsen.util 38 :ok :write [12 18] INFO [2022-01-10 12:53:41,452] jepsen worker 2 - jepsen.util 92 :ok :read [0 14] INFO [2022-01-10 12:53:41,452] jepsen worker 38 - jepsen.util 38 :invoke :write [12 37] INFO [2022-01-10 12:53:41,452] jepsen worker 89 - jepsen.util 269 :ok :write [29 18] INFO [2022-01-10 12:53:41,453] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,453] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,453] jepsen worker 14 - jepsen.util 284 :ok :write [4 5] INFO [2022-01-10 12:53:41,453] jepsen worker 2 - jepsen.util 92 :ok :read [0 14] INFO [2022-01-10 12:53:41,454] jepsen worker 89 - jepsen.util 269 :ok :read [29 18] INFO [2022-01-10 12:53:41,454] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:41,454] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,455] jepsen worker 21 - jepsen.util 201 :ok :write [7 9] INFO [2022-01-10 12:53:41,455] jepsen worker 89 - jepsen.util 269 :invoke :write [29 8] INFO [2022-01-10 12:53:41,455] jepsen worker 2 - jepsen.util 92 :ok :read [0 14] INFO [2022-01-10 12:53:41,455] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:41,456] jepsen worker 2 - jepsen.util 92 :invoke :write [0 28] INFO [2022-01-10 12:53:41,456] jepsen worker 21 - jepsen.util 201 :ok :read [7 9] INFO [2022-01-10 12:53:41,457] jepsen worker 21 - jepsen.util 201 :invoke :write [7 18] INFO [2022-01-10 12:53:41,457] jepsen worker 69 - jepsen.util 159 :ok :write [23 34] INFO [2022-01-10 12:53:41,458] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,459] jepsen worker 69 - jepsen.util 159 :ok :read [23 34] INFO [2022-01-10 12:53:41,459] jepsen worker 69 - jepsen.util 159 :invoke :write [23 40] INFO [2022-01-10 12:53:41,461] jepsen worker 86 - jepsen.util 176 :ok :write [28 23] INFO [2022-01-10 12:53:41,461] jepsen worker 40 - jepsen.util 130 :ok :write [13 37] INFO [2022-01-10 12:53:41,462] jepsen worker 86 - jepsen.util 176 :invoke :write [28 18] INFO [2022-01-10 12:53:41,462] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:41,463] jepsen worker 40 - jepsen.util 130 :ok :read [13 37] INFO [2022-01-10 12:53:41,463] jepsen worker 83 - jepsen.util 263 :ok :write [27 4] INFO [2022-01-10 12:53:41,464] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:41,464] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:41,465] jepsen worker 40 - jepsen.util 130 :ok :read [13 37] INFO [2022-01-10 12:53:41,465] jepsen worker 83 - jepsen.util 263 :ok :read [27 4] INFO [2022-01-10 12:53:41,466] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:41,466] jepsen worker 40 - jepsen.util 130 :ok :read [13 37] INFO [2022-01-10 12:53:41,467] jepsen worker 83 - jepsen.util 263 :invoke :write [27 45] INFO [2022-01-10 12:53:41,467] jepsen worker 5 - jepsen.util 185 :ok :write [1 8] INFO [2022-01-10 12:53:41,467] jepsen worker 60 - jepsen.util 240 :ok :write [20 42] INFO [2022-01-10 12:53:41,468] jepsen worker 40 - jepsen.util 130 :invoke :write [13 40] INFO [2022-01-10 12:53:41,469] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,469] jepsen worker 14 - jepsen.util 284 :ok :read [4 5] INFO [2022-01-10 12:53:41,469] jepsen worker 60 - jepsen.util 240 :invoke :write [20 20] INFO [2022-01-10 12:53:41,469] jepsen worker 5 - jepsen.util 185 :ok :read [1 8] INFO [2022-01-10 12:53:41,470] jepsen worker 14 - jepsen.util 284 :invoke :write [4 19] INFO [2022-01-10 12:53:41,470] jepsen worker 5 - jepsen.util 185 :invoke :write [1 17] INFO [2022-01-10 12:53:41,472] jepsen worker 76 - jepsen.util 256 :ok :write [25 23] INFO [2022-01-10 12:53:41,473] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:41,474] jepsen worker 76 - jepsen.util 256 :ok :read [25 23] INFO [2022-01-10 12:53:41,474] jepsen worker 76 - jepsen.util 256 :invoke :write [25 47] INFO [2022-01-10 12:53:41,476] jepsen worker 80 - jepsen.util 260 :ok :write [26 36] INFO [2022-01-10 12:53:41,477] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:41,477] jepsen worker 80 - jepsen.util 260 :ok :read [26 36] INFO [2022-01-10 12:53:41,478] jepsen worker 68 - jepsen.util 68 :ok :write [22 41] INFO [2022-01-10 12:53:41,478] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:41,479] jepsen worker 68 - jepsen.util 68 :invoke :write [22 1] INFO [2022-01-10 12:53:41,479] jepsen worker 80 - jepsen.util 260 :ok :read [26 36] INFO [2022-01-10 12:53:41,480] jepsen worker 80 - jepsen.util 260 :invoke :write [26 15] INFO [2022-01-10 12:53:41,482] jepsen worker 35 - jepsen.util 215 :ok :write [11 45] INFO [2022-01-10 12:53:41,482] jepsen worker 35 - jepsen.util 215 :invoke :write [11 20] INFO [2022-01-10 12:53:41,483] jepsen worker 89 - jepsen.util 269 :ok :write [29 8] INFO [2022-01-10 12:53:41,483] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,484] jepsen worker 89 - jepsen.util 269 :ok :read [29 8] INFO [2022-01-10 12:53:41,485] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,485] jepsen worker 2 - jepsen.util 92 :ok :write [0 28] INFO [2022-01-10 12:53:41,486] jepsen worker 2 - jepsen.util 92 :invoke :write [0 15] INFO [2022-01-10 12:53:41,486] jepsen worker 89 - jepsen.util 269 :ok :read [29 8] INFO [2022-01-10 12:53:41,487] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,488] jepsen worker 89 - jepsen.util 269 :ok :read [29 8] INFO [2022-01-10 12:53:41,488] jepsen worker 89 - jepsen.util 269 :invoke :write [29 34] INFO [2022-01-10 12:53:41,492] jepsen worker 38 - jepsen.util 38 :ok :write [12 37] INFO [2022-01-10 12:53:41,492] jepsen worker 38 - jepsen.util 38 :invoke :write [12 10] INFO [2022-01-10 12:53:41,509] jepsen worker 86 - jepsen.util 176 :ok :write [28 18] INFO [2022-01-10 12:53:41,509] jepsen worker 86 - jepsen.util 176 :invoke :write [28 48] INFO [2022-01-10 12:53:41,518] jepsen worker 5 - jepsen.util 185 :ok :write [1 17] INFO [2022-01-10 12:53:41,518] jepsen worker 5 - jepsen.util 185 :invoke :write [1 9] INFO [2022-01-10 12:53:41,594] jepsen worker 69 - jepsen.util 159 :ok :write [23 40] INFO [2022-01-10 12:53:41,594] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,595] jepsen worker 7 - jepsen.util 7 :ok :write [2 10] INFO [2022-01-10 12:53:41,595] jepsen worker 69 - jepsen.util 159 :ok :read [23 40] INFO [2022-01-10 12:53:41,595] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:41,596] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,596] jepsen worker 7 - jepsen.util 7 :ok :read [2 10] INFO [2022-01-10 12:53:41,597] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:41,597] jepsen worker 69 - jepsen.util 159 :ok :read [23 40] INFO [2022-01-10 12:53:41,597] jepsen worker 69 - jepsen.util 159 :invoke :write [23 17] INFO [2022-01-10 12:53:41,597] jepsen worker 7 - jepsen.util 7 :ok :read [2 10] INFO [2022-01-10 12:53:41,598] jepsen worker 7 - jepsen.util 7 :invoke :write [2 18] INFO [2022-01-10 12:53:41,635] jepsen worker 65 - jepsen.util 335 :info :write [21 0] :RequestTimeoutException INFO [2022-01-10 12:53:41,635] jepsen worker 18 - jepsen.util 288 :info :write [6 33] :RequestTimeoutException INFO [2022-01-10 12:53:41,635] jepsen worker 65 - jepsen.util 425 :invoke :write [21 14] INFO [2022-01-10 12:53:41,636] jepsen worker 18 - jepsen.util 378 :invoke :write [6 10] INFO [2022-01-10 12:53:41,640] jepsen worker 40 - jepsen.util 130 :ok :write [13 40] INFO [2022-01-10 12:53:41,641] jepsen worker 40 - jepsen.util 130 :invoke :write [13 17] INFO [2022-01-10 12:53:41,645] jepsen worker 16 - jepsen.util 286 :ok :write [5 38] INFO [2022-01-10 12:53:41,646] jepsen worker 53 - jepsen.util 143 :ok :write [17 21] INFO [2022-01-10 12:53:41,646] jepsen worker 16 - jepsen.util 286 :invoke :write [5 5] INFO [2022-01-10 12:53:41,647] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:41,647] jepsen worker 47 - jepsen.util 317 :info :write [15 38] :RequestTimeoutException INFO [2022-01-10 12:53:41,648] jepsen worker 47 - jepsen.util 407 :invoke :write [15 23] INFO [2022-01-10 12:53:41,648] jepsen worker 53 - jepsen.util 143 :ok :read [17 21] INFO [2022-01-10 12:53:41,649] jepsen worker 38 - jepsen.util 38 :ok :write [12 10] INFO [2022-01-10 12:53:41,649] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:41,649] jepsen worker 38 - jepsen.util 38 :invoke :write [12 21] INFO [2022-01-10 12:53:41,650] jepsen worker 53 - jepsen.util 143 :ok :read [17 21] INFO [2022-01-10 12:53:41,650] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:41,651] jepsen worker 14 - jepsen.util 284 :ok :write [4 19] INFO [2022-01-10 12:53:41,651] jepsen worker 53 - jepsen.util 143 :ok :read [17 21] INFO [2022-01-10 12:53:41,651] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:41,652] jepsen worker 42 - jepsen.util 222 :ok :write [14 8] INFO [2022-01-10 12:53:41,652] jepsen worker 14 - jepsen.util 284 :ok :read [4 19] INFO [2022-01-10 12:53:41,652] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:41,653] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:41,653] jepsen worker 53 - jepsen.util 143 :ok :read [17 21] INFO [2022-01-10 12:53:41,653] jepsen worker 42 - jepsen.util 222 :invoke :write [14 0] INFO [2022-01-10 12:53:41,654] jepsen worker 14 - jepsen.util 284 :ok :read [4 19] INFO [2022-01-10 12:53:41,654] jepsen worker 53 - jepsen.util 143 :invoke :write [17 38] INFO [2022-01-10 12:53:41,654] jepsen worker 80 - jepsen.util 260 :ok :write [26 15] INFO [2022-01-10 12:53:41,655] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:41,655] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:41,655] jepsen worker 14 - jepsen.util 284 :ok :read [4 19] INFO [2022-01-10 12:53:41,655] jepsen worker 86 - jepsen.util 176 :ok :write [28 48] INFO [2022-01-10 12:53:41,656] jepsen worker 80 - jepsen.util 260 :ok :read [26 15] INFO [2022-01-10 12:53:41,656] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,657] jepsen worker 56 - jepsen.util 326 :info :write [18 33] :RequestTimeoutException INFO [2022-01-10 12:53:41,657] jepsen worker 14 - jepsen.util 284 :invoke :write [4 2] INFO [2022-01-10 12:53:41,658] jepsen worker 86 - jepsen.util 176 :ok :read [28 48] INFO [2022-01-10 12:53:41,658] jepsen worker 56 - jepsen.util 416 :invoke :read [18 nil] INFO [2022-01-10 12:53:41,659] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:41,659] jepsen worker 56 - jepsen.util 416 :ok :read [18 46] INFO [2022-01-10 12:53:41,659] jepsen worker 86 - jepsen.util 176 :invoke :write [28 1] INFO [2022-01-10 12:53:41,660] jepsen worker 80 - jepsen.util 260 :ok :read [26 15] INFO [2022-01-10 12:53:41,660] jepsen worker 69 - jepsen.util 159 :ok :write [23 17] INFO [2022-01-10 12:53:41,660] jepsen worker 56 - jepsen.util 416 :invoke :read [18 nil] INFO [2022-01-10 12:53:41,661] jepsen worker 48 - jepsen.util 228 :ok :write [16 43] INFO [2022-01-10 12:53:41,661] jepsen worker 56 - jepsen.util 416 :ok :read [18 46] INFO [2022-01-10 12:53:41,661] jepsen worker 69 - jepsen.util 159 :invoke :write [23 34] INFO [2022-01-10 12:53:41,662] jepsen worker 80 - jepsen.util 260 :invoke :write [26 35] INFO [2022-01-10 12:53:41,662] jepsen worker 48 - jepsen.util 228 :invoke :write [16 2] INFO [2022-01-10 12:53:41,663] jepsen worker 56 - jepsen.util 416 :invoke :read [18 nil] INFO [2022-01-10 12:53:41,664] jepsen worker 56 - jepsen.util 416 :ok :read [18 46] INFO [2022-01-10 12:53:41,664] jepsen worker 5 - jepsen.util 185 :ok :write [1 9] INFO [2022-01-10 12:53:41,664] jepsen worker 56 - jepsen.util 416 :invoke :read [18 nil] INFO [2022-01-10 12:53:41,665] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,665] jepsen worker 56 - jepsen.util 416 :ok :read [18 46] INFO [2022-01-10 12:53:41,666] jepsen worker 56 - jepsen.util 416 :invoke :write [18 25] INFO [2022-01-10 12:53:41,666] jepsen worker 5 - jepsen.util 185 :ok :read [1 9] INFO [2022-01-10 12:53:41,667] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,668] jepsen worker 5 - jepsen.util 185 :ok :read [1 9] INFO [2022-01-10 12:53:41,668] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,669] jepsen worker 21 - jepsen.util 201 :ok :write [7 18] INFO [2022-01-10 12:53:41,669] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:41,669] jepsen worker 5 - jepsen.util 185 :ok :read [1 9] INFO [2022-01-10 12:53:41,670] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,670] jepsen worker 21 - jepsen.util 201 :ok :read [7 18] INFO [2022-01-10 12:53:41,670] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:41,670] jepsen worker 5 - jepsen.util 185 :ok :read [1 9] INFO [2022-01-10 12:53:41,671] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,671] jepsen worker 89 - jepsen.util 269 :ok :write [29 34] INFO [2022-01-10 12:53:41,671] jepsen worker 21 - jepsen.util 201 :ok :read [7 18] INFO [2022-01-10 12:53:41,672] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:41,672] jepsen worker 5 - jepsen.util 185 :ok :read [1 9] INFO [2022-01-10 12:53:41,672] jepsen worker 38 - jepsen.util 38 :ok :write [12 21] INFO [2022-01-10 12:53:41,672] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,673] jepsen worker 21 - jepsen.util 201 :ok :read [7 18] INFO [2022-01-10 12:53:41,673] jepsen worker 89 - jepsen.util 269 :ok :read [29 34] INFO [2022-01-10 12:53:41,673] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,674] jepsen worker 38 - jepsen.util 38 :invoke :write [12 15] INFO [2022-01-10 12:53:41,674] jepsen worker 89 - jepsen.util 269 :invoke :write [29 1] INFO [2022-01-10 12:53:41,674] jepsen worker 5 - jepsen.util 185 :ok :read [1 9] INFO [2022-01-10 12:53:41,675] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:41,675] jepsen worker 5 - jepsen.util 185 :invoke :write [1 28] INFO [2022-01-10 12:53:41,676] jepsen worker 21 - jepsen.util 201 :ok :read [7 18] INFO [2022-01-10 12:53:41,676] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:41,677] jepsen worker 2 - jepsen.util 92 :ok :write [0 15] INFO [2022-01-10 12:53:41,678] jepsen worker 21 - jepsen.util 201 :ok :read [7 18] INFO [2022-01-10 12:53:41,678] jepsen worker 2 - jepsen.util 92 :invoke :write [0 6] INFO [2022-01-10 12:53:41,678] jepsen worker 21 - jepsen.util 201 :invoke :write [7 26] INFO [2022-01-10 12:53:41,678] jepsen worker 53 - jepsen.util 143 :ok :write [17 38] INFO [2022-01-10 12:53:41,679] jepsen worker 80 - jepsen.util 260 :ok :write [26 35] INFO [2022-01-10 12:53:41,679] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:41,679] jepsen worker 80 - jepsen.util 260 :invoke :write [26 36] INFO [2022-01-10 12:53:41,680] jepsen worker 53 - jepsen.util 143 :ok :read [17 38] INFO [2022-01-10 12:53:41,680] jepsen worker 83 - jepsen.util 263 :ok :write [27 45] INFO [2022-01-10 12:53:41,680] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:41,681] jepsen worker 83 - jepsen.util 263 :invoke :write [27 33] INFO [2022-01-10 12:53:41,681] jepsen worker 53 - jepsen.util 143 :ok :read [17 38] INFO [2022-01-10 12:53:41,682] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:41,682] jepsen worker 20 - jepsen.util 290 :info :write [6 20] :RequestTimeoutException INFO [2022-01-10 12:53:41,683] jepsen worker 20 - jepsen.util 380 :invoke :read [6 nil] INFO [2022-01-10 12:53:41,684] jepsen worker 53 - jepsen.util 143 :ok :read [17 38] INFO [2022-01-10 12:53:41,684] jepsen worker 53 - jepsen.util 143 :invoke :read [17 nil] INFO [2022-01-10 12:53:41,685] jepsen worker 53 - jepsen.util 143 :ok :read [17 38] INFO [2022-01-10 12:53:41,685] jepsen worker 53 - jepsen.util 143 :invoke :write [17 17] INFO [2022-01-10 12:53:41,685] jepsen worker 86 - jepsen.util 176 :ok :write [28 1] INFO [2022-01-10 12:53:41,686] jepsen worker 86 - jepsen.util 176 :invoke :write [28 21] INFO [2022-01-10 12:53:41,690] jepsen worker 14 - jepsen.util 284 :ok :write [4 2] INFO [2022-01-10 12:53:41,690] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:41,691] jepsen worker 14 - jepsen.util 284 :ok :read [4 2] INFO [2022-01-10 12:53:41,692] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:41,692] jepsen worker 14 - jepsen.util 284 :ok :read [4 2] INFO [2022-01-10 12:53:41,693] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:41,693] jepsen worker 40 - jepsen.util 130 :ok :write [13 17] INFO [2022-01-10 12:53:41,694] jepsen worker 14 - jepsen.util 284 :ok :read [4 2] INFO [2022-01-10 12:53:41,694] jepsen worker 40 - jepsen.util 130 :invoke :write [13 36] INFO [2022-01-10 12:53:41,695] jepsen worker 14 - jepsen.util 284 :invoke :write [4 27] INFO [2022-01-10 12:53:41,699] jepsen worker 69 - jepsen.util 159 :ok :write [23 34] INFO [2022-01-10 12:53:41,699] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,700] jepsen worker 69 - jepsen.util 159 :ok :read [23 34] INFO [2022-01-10 12:53:41,700] jepsen worker 60 - jepsen.util 240 :ok :write [20 20] INFO [2022-01-10 12:53:41,701] jepsen worker 69 - jepsen.util 159 :invoke :write [23 34] INFO [2022-01-10 12:53:41,701] jepsen worker 60 - jepsen.util 240 :invoke :write [20 25] INFO [2022-01-10 12:53:41,707] jepsen worker 76 - jepsen.util 256 :ok :write [25 47] INFO [2022-01-10 12:53:41,708] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:41,708] jepsen worker 5 - jepsen.util 185 :ok :write [1 28] INFO [2022-01-10 12:53:41,709] jepsen worker 80 - jepsen.util 260 :ok :write [26 36] INFO [2022-01-10 12:53:41,709] jepsen worker 76 - jepsen.util 256 :ok :read [25 47] INFO [2022-01-10 12:53:41,709] jepsen worker 5 - jepsen.util 185 :invoke :write [1 2] INFO [2022-01-10 12:53:41,710] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:41,710] jepsen worker 76 - jepsen.util 256 :invoke :write [25 3] INFO [2022-01-10 12:53:41,710] jepsen worker 89 - jepsen.util 269 :ok :write [29 1] INFO [2022-01-10 12:53:41,711] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,711] jepsen worker 80 - jepsen.util 260 :ok :read [26 36] INFO [2022-01-10 12:53:41,712] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:41,712] jepsen worker 89 - jepsen.util 269 :ok :read [29 1] INFO [2022-01-10 12:53:41,712] jepsen worker 80 - jepsen.util 260 :ok :read [26 36] INFO [2022-01-10 12:53:41,712] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,713] jepsen worker 80 - jepsen.util 260 :invoke :write [26 3] INFO [2022-01-10 12:53:41,713] jepsen worker 89 - jepsen.util 269 :ok :read [29 1] INFO [2022-01-10 12:53:41,714] jepsen worker 89 - jepsen.util 269 :invoke :write [29 12] INFO [2022-01-10 12:53:41,715] jepsen worker 68 - jepsen.util 68 :ok :write [22 1] INFO [2022-01-10 12:53:41,716] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:41,717] jepsen worker 68 - jepsen.util 68 :ok :read [22 1] INFO [2022-01-10 12:53:41,717] jepsen worker 38 - jepsen.util 38 :ok :write [12 15] INFO [2022-01-10 12:53:41,717] jepsen worker 38 - jepsen.util 38 :invoke :write [12 49] INFO [2022-01-10 12:53:41,718] jepsen worker 68 - jepsen.util 68 :invoke :write [22 6] INFO [2022-01-10 12:53:41,718] jepsen worker 2 - jepsen.util 92 :ok :write [0 6] INFO [2022-01-10 12:53:41,719] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,719] jepsen worker 2 - jepsen.util 92 :ok :read [0 6] INFO [2022-01-10 12:53:41,720] jepsen worker 35 - jepsen.util 215 :ok :write [11 20] INFO [2022-01-10 12:53:41,720] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,721] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:41,721] jepsen worker 2 - jepsen.util 92 :ok :read [0 6] INFO [2022-01-10 12:53:41,721] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,721] jepsen worker 35 - jepsen.util 215 :ok :read [11 20] INFO [2022-01-10 12:53:41,722] jepsen worker 35 - jepsen.util 215 :invoke :write [11 27] INFO [2022-01-10 12:53:41,722] jepsen worker 2 - jepsen.util 92 :ok :read [0 6] INFO [2022-01-10 12:53:41,723] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,724] jepsen worker 2 - jepsen.util 92 :ok :read [0 6] INFO [2022-01-10 12:53:41,724] jepsen worker 2 - jepsen.util 92 :invoke :write [0 29] INFO [2022-01-10 12:53:41,725] jepsen worker 53 - jepsen.util 143 :ok :write [17 17] INFO [2022-01-10 12:53:41,726] jepsen worker 53 - jepsen.util 143 :invoke :write [17 23] INFO [2022-01-10 12:53:41,726] jepsen worker 7 - jepsen.util 7 :ok :write [2 18] INFO [2022-01-10 12:53:41,727] jepsen worker 7 - jepsen.util 7 :invoke :write [2 30] INFO [2022-01-10 12:53:41,732] jepsen worker 16 - jepsen.util 286 :ok :write [5 5] INFO [2022-01-10 12:53:41,732] jepsen worker 86 - jepsen.util 176 :ok :write [28 21] INFO [2022-01-10 12:53:41,733] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:41,733] jepsen worker 86 - jepsen.util 176 :invoke :write [28 20] INFO [2022-01-10 12:53:41,733] jepsen worker 16 - jepsen.util 286 :ok :read [5 5] INFO [2022-01-10 12:53:41,734] jepsen worker 16 - jepsen.util 286 :invoke :write [5 44] INFO [2022-01-10 12:53:41,737] jepsen worker 14 - jepsen.util 284 :ok :write [4 27] INFO [2022-01-10 12:53:41,737] jepsen worker 40 - jepsen.util 130 :ok :write [13 36] INFO [2022-01-10 12:53:41,738] jepsen worker 40 - jepsen.util 130 :invoke :write [13 4] INFO [2022-01-10 12:53:41,738] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:41,739] jepsen worker 14 - jepsen.util 284 :ok :read [4 27] INFO [2022-01-10 12:53:41,740] jepsen worker 14 - jepsen.util 284 :invoke :write [4 27] INFO [2022-01-10 12:53:41,744] jepsen worker 69 - jepsen.util 159 :ok :write [23 34] INFO [2022-01-10 12:53:41,744] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,745] jepsen worker 69 - jepsen.util 159 :ok :read [23 34] INFO [2022-01-10 12:53:41,745] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,746] jepsen worker 89 - jepsen.util 269 :ok :write [29 12] INFO [2022-01-10 12:53:41,746] jepsen worker 69 - jepsen.util 159 :ok :read [23 34] INFO [2022-01-10 12:53:41,746] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,747] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,747] jepsen worker 89 - jepsen.util 269 :ok :read [29 12] INFO [2022-01-10 12:53:41,747] jepsen worker 69 - jepsen.util 159 :ok :read [23 34] INFO [2022-01-10 12:53:41,748] jepsen worker 89 - jepsen.util 269 :invoke :write [29 39] INFO [2022-01-10 12:53:41,748] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,749] jepsen worker 5 - jepsen.util 185 :ok :write [1 2] INFO [2022-01-10 12:53:41,749] jepsen worker 69 - jepsen.util 159 :ok :read [23 34] INFO [2022-01-10 12:53:41,749] jepsen worker 5 - jepsen.util 185 :invoke :write [1 42] INFO [2022-01-10 12:53:41,750] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,751] jepsen worker 69 - jepsen.util 159 :ok :read [23 34] INFO [2022-01-10 12:53:41,751] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,752] jepsen worker 69 - jepsen.util 159 :ok :read [23 34] INFO [2022-01-10 12:53:41,753] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,753] jepsen worker 69 - jepsen.util 159 :ok :read [23 34] INFO [2022-01-10 12:53:41,754] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,755] jepsen worker 69 - jepsen.util 159 :ok :read [23 34] INFO [2022-01-10 12:53:41,755] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:41,756] jepsen worker 69 - jepsen.util 159 :ok :read [23 34] INFO [2022-01-10 12:53:41,756] jepsen worker 80 - jepsen.util 260 :ok :write [26 3] INFO [2022-01-10 12:53:41,756] jepsen worker 69 - jepsen.util 159 :invoke :write [23 28] INFO [2022-01-10 12:53:41,757] jepsen worker 80 - jepsen.util 260 :invoke :write [26 13] INFO [2022-01-10 12:53:41,757] jepsen worker 2 - jepsen.util 92 :ok :write [0 29] INFO [2022-01-10 12:53:41,757] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,759] jepsen worker 2 - jepsen.util 92 :ok :read [0 29] INFO [2022-01-10 12:53:41,759] jepsen worker 2 - jepsen.util 92 :invoke :write [0 36] INFO [2022-01-10 12:53:41,761] jepsen worker 38 - jepsen.util 38 :ok :write [12 49] INFO [2022-01-10 12:53:41,762] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,763] jepsen worker 38 - jepsen.util 38 :ok :read [12 49] INFO [2022-01-10 12:53:41,763] jepsen worker 38 - jepsen.util 38 :invoke :write [12 31] INFO [2022-01-10 12:53:41,772] jepsen worker 86 - jepsen.util 176 :ok :write [28 20] INFO [2022-01-10 12:53:41,773] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,774] jepsen worker 86 - jepsen.util 176 :ok :read [28 20] INFO [2022-01-10 12:53:41,775] jepsen worker 86 - jepsen.util 176 :invoke :write [28 47] INFO [2022-01-10 12:53:41,779] jepsen worker 40 - jepsen.util 130 :ok :write [13 4] INFO [2022-01-10 12:53:41,780] jepsen worker 40 - jepsen.util 130 :invoke :write [13 41] INFO [2022-01-10 12:53:41,786] jepsen worker 89 - jepsen.util 269 :ok :write [29 39] INFO [2022-01-10 12:53:41,786] jepsen worker 5 - jepsen.util 185 :ok :write [1 42] INFO [2022-01-10 12:53:41,786] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,787] jepsen worker 5 - jepsen.util 185 :invoke :write [1 43] INFO [2022-01-10 12:53:41,787] jepsen worker 89 - jepsen.util 269 :ok :read [29 39] INFO [2022-01-10 12:53:41,788] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,789] jepsen worker 89 - jepsen.util 269 :ok :read [29 39] INFO [2022-01-10 12:53:41,790] jepsen worker 89 - jepsen.util 269 :invoke :write [29 3] INFO [2022-01-10 12:53:41,792] jepsen worker 2 - jepsen.util 92 :ok :write [0 36] INFO [2022-01-10 12:53:41,792] jepsen worker 2 - jepsen.util 92 :invoke :write [0 37] INFO [2022-01-10 12:53:41,800] jepsen worker 40 - jepsen.util 130 :ok :write [13 41] INFO [2022-01-10 12:53:41,801] jepsen worker 40 - jepsen.util 130 :invoke :write [13 7] INFO [2022-01-10 12:53:41,894] jepsen worker 42 - jepsen.util 222 :ok :write [14 0] INFO [2022-01-10 12:53:41,894] jepsen worker 42 - jepsen.util 222 :invoke :write [14 33] INFO [2022-01-10 12:53:41,903] jepsen worker 38 - jepsen.util 38 :ok :write [12 31] INFO [2022-01-10 12:53:41,904] jepsen worker 38 - jepsen.util 38 :invoke :write [12 4] INFO [2022-01-10 12:53:41,908] jepsen worker 86 - jepsen.util 176 :ok :write [28 47] INFO [2022-01-10 12:53:41,909] jepsen worker 86 - jepsen.util 176 :invoke :write [28 31] INFO [2022-01-10 12:53:41,910] jepsen worker 89 - jepsen.util 269 :ok :write [29 3] INFO [2022-01-10 12:53:41,911] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:41,911] jepsen worker 5 - jepsen.util 185 :ok :write [1 43] INFO [2022-01-10 12:53:41,911] jepsen worker 89 - jepsen.util 269 :ok :read [29 3] INFO [2022-01-10 12:53:41,912] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,912] jepsen worker 89 - jepsen.util 269 :invoke :write [29 2] INFO [2022-01-10 12:53:41,913] jepsen worker 5 - jepsen.util 185 :ok :read [1 43] INFO [2022-01-10 12:53:41,913] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,914] jepsen worker 5 - jepsen.util 185 :ok :read [1 43] INFO [2022-01-10 12:53:41,914] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:41,915] jepsen worker 5 - jepsen.util 185 :ok :read [1 43] INFO [2022-01-10 12:53:41,915] jepsen worker 5 - jepsen.util 185 :invoke :write [1 25] INFO [2022-01-10 12:53:41,918] jepsen worker 2 - jepsen.util 92 :ok :write [0 37] INFO [2022-01-10 12:53:41,918] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,919] jepsen worker 2 - jepsen.util 92 :ok :read [0 37] INFO [2022-01-10 12:53:41,919] jepsen worker 2 - jepsen.util 92 :invoke :write [0 5] INFO [2022-01-10 12:53:41,923] jepsen worker 40 - jepsen.util 130 :ok :write [13 7] INFO [2022-01-10 12:53:41,923] jepsen worker 40 - jepsen.util 130 :invoke :write [13 14] INFO [2022-01-10 12:53:41,963] jepsen worker 53 - jepsen.util 143 :ok :write [17 23] INFO [2022-01-10 12:53:41,963] jepsen worker 48 - jepsen.util 228 :ok :write [16 2] INFO [2022-01-10 12:53:41,963] jepsen worker 53 - jepsen.util 143 :invoke :write [17 46] INFO [2022-01-10 12:53:41,964] jepsen worker 48 - jepsen.util 228 :invoke :write [16 34] INFO [2022-01-10 12:53:41,967] jepsen worker 14 - jepsen.util 284 :ok :write [4 27] INFO [2022-01-10 12:53:41,967] jepsen worker 14 - jepsen.util 284 :invoke :write [4 31] INFO [2022-01-10 12:53:41,969] jepsen worker 21 - jepsen.util 201 :ok :write [7 26] INFO [2022-01-10 12:53:41,969] jepsen worker 21 - jepsen.util 201 :invoke :write [7 22] INFO [2022-01-10 12:53:41,971] jepsen worker 69 - jepsen.util 159 :ok :write [23 28] INFO [2022-01-10 12:53:41,971] jepsen worker 69 - jepsen.util 159 :invoke :write [23 10] INFO [2022-01-10 12:53:41,971] jepsen worker 89 - jepsen.util 269 :ok :write [29 2] INFO [2022-01-10 12:53:41,972] jepsen worker 89 - jepsen.util 269 :invoke :write [29 15] INFO [2022-01-10 12:53:41,974] jepsen worker 38 - jepsen.util 38 :ok :write [12 4] INFO [2022-01-10 12:53:41,974] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,975] jepsen worker 80 - jepsen.util 260 :ok :write [26 13] INFO [2022-01-10 12:53:41,975] jepsen worker 38 - jepsen.util 38 :ok :read [12 4] INFO [2022-01-10 12:53:41,976] jepsen worker 2 - jepsen.util 92 :ok :write [0 5] INFO [2022-01-10 12:53:41,976] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:41,976] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,977] jepsen worker 80 - jepsen.util 260 :ok :read [26 13] INFO [2022-01-10 12:53:41,977] jepsen worker 83 - jepsen.util 263 :ok :write [27 33] INFO [2022-01-10 12:53:41,977] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:41,977] jepsen worker 38 - jepsen.util 38 :ok :read [12 4] INFO [2022-01-10 12:53:41,978] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:41,978] jepsen worker 2 - jepsen.util 92 :ok :read [0 5] INFO [2022-01-10 12:53:41,979] jepsen worker 80 - jepsen.util 260 :ok :read [26 13] INFO [2022-01-10 12:53:41,979] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,980] jepsen worker 83 - jepsen.util 263 :invoke :write [27 26] INFO [2022-01-10 12:53:41,980] jepsen worker 38 - jepsen.util 38 :ok :read [12 4] INFO [2022-01-10 12:53:41,980] jepsen worker 53 - jepsen.util 143 :ok :write [17 46] INFO [2022-01-10 12:53:41,980] jepsen worker 80 - jepsen.util 260 :invoke :write [26 8] INFO [2022-01-10 12:53:41,981] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,981] jepsen worker 40 - jepsen.util 130 :ok :write [13 14] INFO [2022-01-10 12:53:41,981] jepsen worker 2 - jepsen.util 92 :invoke :write [0 8] INFO [2022-01-10 12:53:41,982] jepsen worker 38 - jepsen.util 38 :ok :read [12 4] INFO [2022-01-10 12:53:41,982] jepsen worker 53 - jepsen.util 143 :invoke :write [17 30] INFO [2022-01-10 12:53:41,982] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:41,983] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,983] jepsen worker 60 - jepsen.util 240 :ok :write [20 25] INFO [2022-01-10 12:53:41,983] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:41,983] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:41,984] jepsen worker 40 - jepsen.util 130 :invoke :write [13 25] INFO [2022-01-10 12:53:41,984] jepsen worker 38 - jepsen.util 38 :ok :read [12 4] INFO [2022-01-10 12:53:41,984] jepsen worker 60 - jepsen.util 240 :ok :read [20 25] INFO [2022-01-10 12:53:41,984] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:41,985] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:41,985] jepsen worker 38 - jepsen.util 38 :ok :read [12 4] INFO [2022-01-10 12:53:41,986] jepsen worker 60 - jepsen.util 240 :ok :read [20 25] INFO [2022-01-10 12:53:41,986] jepsen worker 38 - jepsen.util 38 :invoke :write [12 0] INFO [2022-01-10 12:53:41,986] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:41,987] jepsen worker 86 - jepsen.util 176 :ok :write [28 31] INFO [2022-01-10 12:53:41,987] jepsen worker 60 - jepsen.util 240 :ok :read [20 25] INFO [2022-01-10 12:53:41,987] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,988] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:41,988] jepsen worker 86 - jepsen.util 176 :ok :read [28 31] INFO [2022-01-10 12:53:41,989] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,989] jepsen worker 60 - jepsen.util 240 :ok :read [20 25] INFO [2022-01-10 12:53:41,989] jepsen worker 86 - jepsen.util 176 :ok :read [28 31] INFO [2022-01-10 12:53:41,990] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:41,990] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,990] jepsen worker 60 - jepsen.util 240 :ok :read [20 25] INFO [2022-01-10 12:53:41,991] jepsen worker 86 - jepsen.util 176 :ok :read [28 31] INFO [2022-01-10 12:53:41,991] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:41,991] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:41,992] jepsen worker 60 - jepsen.util 240 :ok :read [20 25] INFO [2022-01-10 12:53:41,992] jepsen worker 86 - jepsen.util 176 :ok :read [28 31] INFO [2022-01-10 12:53:41,992] jepsen worker 60 - jepsen.util 240 :invoke :write [20 19] INFO [2022-01-10 12:53:41,992] jepsen worker 86 - jepsen.util 176 :invoke :write [28 47] INFO [2022-01-10 12:53:42,059] jepsen worker 14 - jepsen.util 284 :ok :write [4 31] INFO [2022-01-10 12:53:42,059] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,060] jepsen worker 14 - jepsen.util 284 :ok :read [4 31] INFO [2022-01-10 12:53:42,061] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,062] jepsen worker 14 - jepsen.util 284 :ok :read [4 31] INFO [2022-01-10 12:53:42,062] jepsen worker 14 - jepsen.util 284 :invoke :write [4 29] INFO [2022-01-10 12:53:42,064] jepsen worker 76 - jepsen.util 256 :ok :write [25 3] INFO [2022-01-10 12:53:42,064] jepsen worker 5 - jepsen.util 185 :ok :write [1 25] INFO [2022-01-10 12:53:42,064] jepsen worker 89 - jepsen.util 269 :ok :write [29 15] INFO [2022-01-10 12:53:42,065] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,066] jepsen worker 76 - jepsen.util 256 :invoke :write [25 38] INFO [2022-01-10 12:53:42,066] jepsen worker 5 - jepsen.util 185 :ok :read [1 25] INFO [2022-01-10 12:53:42,066] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,067] jepsen worker 5 - jepsen.util 185 :invoke :write [1 42] INFO [2022-01-10 12:53:42,067] jepsen worker 69 - jepsen.util 159 :ok :write [23 10] INFO [2022-01-10 12:53:42,067] jepsen worker 89 - jepsen.util 269 :ok :read [29 15] INFO [2022-01-10 12:53:42,068] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,069] jepsen worker 69 - jepsen.util 159 :invoke :write [23 39] INFO [2022-01-10 12:53:42,069] jepsen worker 89 - jepsen.util 269 :ok :read [29 15] INFO [2022-01-10 12:53:42,069] jepsen worker 2 - jepsen.util 92 :ok :write [0 8] INFO [2022-01-10 12:53:42,070] jepsen worker 2 - jepsen.util 92 :invoke :write [0 19] INFO [2022-01-10 12:53:42,070] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,071] jepsen worker 38 - jepsen.util 38 :ok :write [12 0] INFO [2022-01-10 12:53:42,071] jepsen worker 89 - jepsen.util 269 :ok :read [29 15] INFO [2022-01-10 12:53:42,072] jepsen worker 38 - jepsen.util 38 :invoke :write [12 22] INFO [2022-01-10 12:53:42,072] jepsen worker 89 - jepsen.util 269 :invoke :write [29 23] INFO [2022-01-10 12:53:42,073] jepsen worker 68 - jepsen.util 68 :ok :write [22 6] INFO [2022-01-10 12:53:42,073] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,075] jepsen worker 68 - jepsen.util 68 :ok :read [22 6] INFO [2022-01-10 12:53:42,075] jepsen worker 68 - jepsen.util 68 :invoke :write [22 16] INFO [2022-01-10 12:53:42,079] jepsen worker 80 - jepsen.util 260 :ok :write [26 8] INFO [2022-01-10 12:53:42,080] jepsen worker 80 - jepsen.util 260 :invoke :write [26 38] INFO [2022-01-10 12:53:42,081] jepsen worker 35 - jepsen.util 215 :ok :write [11 27] INFO [2022-01-10 12:53:42,081] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,082] jepsen worker 35 - jepsen.util 215 :ok :read [11 27] INFO [2022-01-10 12:53:42,082] jepsen worker 86 - jepsen.util 176 :ok :write [28 47] INFO [2022-01-10 12:53:42,083] jepsen worker 51 - jepsen.util 141 :ok :write [17 3] INFO [2022-01-10 12:53:42,083] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,083] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,084] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:42,084] jepsen worker 35 - jepsen.util 215 :ok :read [11 27] INFO [2022-01-10 12:53:42,084] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,085] jepsen worker 51 - jepsen.util 141 :ok :read [17 3] INFO [2022-01-10 12:53:42,085] jepsen worker 86 - jepsen.util 176 :ok :read [28 47] INFO [2022-01-10 12:53:42,085] jepsen worker 51 - jepsen.util 141 :invoke :write [17 35] INFO [2022-01-10 12:53:42,086] jepsen worker 35 - jepsen.util 215 :ok :read [11 27] INFO [2022-01-10 12:53:42,086] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,086] jepsen worker 35 - jepsen.util 215 :invoke :write [11 30] INFO [2022-01-10 12:53:42,087] jepsen worker 86 - jepsen.util 176 :ok :read [28 47] INFO [2022-01-10 12:53:42,088] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,088] jepsen worker 7 - jepsen.util 7 :ok :write [2 30] INFO [2022-01-10 12:53:42,089] jepsen worker 7 - jepsen.util 7 :invoke :write [2 45] INFO [2022-01-10 12:53:42,090] jepsen worker 86 - jepsen.util 176 :ok :read [28 47] INFO [2022-01-10 12:53:42,090] jepsen worker 14 - jepsen.util 284 :ok :write [4 29] INFO [2022-01-10 12:53:42,091] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,091] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,092] jepsen worker 14 - jepsen.util 284 :ok :read [4 29] INFO [2022-01-10 12:53:42,092] jepsen worker 14 - jepsen.util 284 :invoke :write [4 0] INFO [2022-01-10 12:53:42,092] jepsen worker 86 - jepsen.util 176 :ok :read [28 47] INFO [2022-01-10 12:53:42,093] jepsen worker 86 - jepsen.util 176 :invoke :write [28 38] INFO [2022-01-10 12:53:42,095] jepsen worker 5 - jepsen.util 185 :ok :write [1 42] INFO [2022-01-10 12:53:42,095] jepsen worker 5 - jepsen.util 185 :invoke :write [1 30] INFO [2022-01-10 12:53:42,096] jepsen worker 40 - jepsen.util 130 :ok :write [13 25] INFO [2022-01-10 12:53:42,097] jepsen worker 16 - jepsen.util 286 :ok :write [5 44] INFO [2022-01-10 12:53:42,097] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:42,097] jepsen worker 69 - jepsen.util 159 :ok :write [23 39] INFO [2022-01-10 12:53:42,097] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,098] jepsen worker 69 - jepsen.util 159 :invoke :write [23 1] INFO [2022-01-10 12:53:42,098] jepsen worker 40 - jepsen.util 130 :ok :read [13 25] INFO [2022-01-10 12:53:42,098] jepsen worker 16 - jepsen.util 286 :ok :read [5 44] INFO [2022-01-10 12:53:42,099] jepsen worker 40 - jepsen.util 130 :invoke :write [13 48] INFO [2022-01-10 12:53:42,099] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,101] jepsen worker 16 - jepsen.util 286 :ok :read [5 44] INFO [2022-01-10 12:53:42,101] jepsen worker 16 - jepsen.util 286 :invoke :write [5 17] INFO [2022-01-10 12:53:42,101] jepsen worker 42 - jepsen.util 222 :ok :write [14 33] INFO [2022-01-10 12:53:42,102] jepsen worker 42 - jepsen.util 222 :invoke :write [14 19] INFO [2022-01-10 12:53:42,103] jepsen worker 38 - jepsen.util 38 :ok :write [12 22] INFO [2022-01-10 12:53:42,103] jepsen worker 38 - jepsen.util 38 :invoke :write [12 0] INFO [2022-01-10 12:53:42,106] jepsen worker 80 - jepsen.util 260 :ok :write [26 38] INFO [2022-01-10 12:53:42,107] jepsen worker 80 - jepsen.util 260 :invoke :write [26 0] INFO [2022-01-10 12:53:42,107] jepsen worker 89 - jepsen.util 269 :ok :write [29 23] INFO [2022-01-10 12:53:42,108] jepsen worker 48 - jepsen.util 228 :ok :write [16 34] INFO [2022-01-10 12:53:42,108] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,108] jepsen worker 48 - jepsen.util 228 :invoke :write [16 36] INFO [2022-01-10 12:53:42,109] jepsen worker 89 - jepsen.util 269 :ok :read [29 23] INFO [2022-01-10 12:53:42,109] jepsen worker 89 - jepsen.util 269 :invoke :write [29 43] INFO [2022-01-10 12:53:42,111] jepsen worker 86 - jepsen.util 176 :ok :write [28 38] INFO [2022-01-10 12:53:42,112] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,112] jepsen worker 21 - jepsen.util 201 :ok :write [7 22] INFO [2022-01-10 12:53:42,113] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,113] jepsen worker 86 - jepsen.util 176 :ok :read [28 38] INFO [2022-01-10 12:53:42,114] jepsen worker 51 - jepsen.util 141 :ok :write [17 35] INFO [2022-01-10 12:53:42,114] jepsen worker 21 - jepsen.util 201 :ok :read [7 22] INFO [2022-01-10 12:53:42,114] jepsen worker 86 - jepsen.util 176 :invoke :write [28 4] INFO [2022-01-10 12:53:42,115] jepsen worker 21 - jepsen.util 201 :invoke :write [7 6] INFO [2022-01-10 12:53:42,115] jepsen worker 2 - jepsen.util 92 :ok :write [0 19] INFO [2022-01-10 12:53:42,115] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,116] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:42,116] jepsen worker 2 - jepsen.util 92 :ok :read [0 19] INFO [2022-01-10 12:53:42,117] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,117] jepsen worker 51 - jepsen.util 141 :ok :read [17 35] INFO [2022-01-10 12:53:42,117] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:42,118] jepsen worker 2 - jepsen.util 92 :ok :read [0 19] INFO [2022-01-10 12:53:42,118] jepsen worker 51 - jepsen.util 141 :ok :read [17 35] INFO [2022-01-10 12:53:42,118] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,119] jepsen worker 14 - jepsen.util 284 :ok :write [4 0] INFO [2022-01-10 12:53:42,119] jepsen worker 51 - jepsen.util 141 :invoke :write [17 20] INFO [2022-01-10 12:53:42,120] jepsen worker 2 - jepsen.util 92 :ok :read [0 19] INFO [2022-01-10 12:53:42,120] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,120] jepsen worker 2 - jepsen.util 92 :invoke :write [0 32] INFO [2022-01-10 12:53:42,120] jepsen worker 14 - jepsen.util 284 :ok :read [4 0] INFO [2022-01-10 12:53:42,121] jepsen worker 83 - jepsen.util 263 :ok :write [27 26] INFO [2022-01-10 12:53:42,121] jepsen worker 14 - jepsen.util 284 :invoke :write [4 15] INFO [2022-01-10 12:53:42,121] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,123] jepsen worker 83 - jepsen.util 263 :ok :read [27 26] INFO [2022-01-10 12:53:42,123] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,124] jepsen worker 83 - jepsen.util 263 :ok :read [27 26] INFO [2022-01-10 12:53:42,125] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,125] jepsen worker 5 - jepsen.util 185 :ok :write [1 30] INFO [2022-01-10 12:53:42,126] jepsen worker 5 - jepsen.util 185 :invoke :write [1 7] INFO [2022-01-10 12:53:42,126] jepsen worker 83 - jepsen.util 263 :ok :read [27 26] INFO [2022-01-10 12:53:42,127] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,128] jepsen worker 83 - jepsen.util 263 :ok :read [27 26] INFO [2022-01-10 12:53:42,128] jepsen worker 60 - jepsen.util 240 :ok :write [20 19] INFO [2022-01-10 12:53:42,128] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,129] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,129] jepsen worker 83 - jepsen.util 263 :ok :read [27 26] INFO [2022-01-10 12:53:42,130] jepsen worker 60 - jepsen.util 240 :ok :read [20 19] INFO [2022-01-10 12:53:42,130] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,131] jepsen worker 60 - jepsen.util 240 :invoke :write [20 36] INFO [2022-01-10 12:53:42,131] jepsen worker 83 - jepsen.util 263 :ok :read [27 26] INFO [2022-01-10 12:53:42,132] jepsen worker 83 - jepsen.util 263 :invoke :write [27 49] INFO [2022-01-10 12:53:42,132] jepsen worker 69 - jepsen.util 159 :ok :write [23 1] INFO [2022-01-10 12:53:42,132] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,134] jepsen worker 69 - jepsen.util 159 :ok :read [23 1] INFO [2022-01-10 12:53:42,134] jepsen worker 69 - jepsen.util 159 :invoke :write [23 47] INFO [2022-01-10 12:53:42,138] jepsen worker 38 - jepsen.util 38 :ok :write [12 0] INFO [2022-01-10 12:53:42,139] jepsen worker 38 - jepsen.util 38 :invoke :write [12 38] INFO [2022-01-10 12:53:42,139] jepsen worker 76 - jepsen.util 256 :ok :write [25 38] INFO [2022-01-10 12:53:42,139] jepsen worker 76 - jepsen.util 256 :invoke :write [25 8] INFO [2022-01-10 12:53:42,141] jepsen worker 40 - jepsen.util 130 :ok :write [13 48] INFO [2022-01-10 12:53:42,142] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:42,143] jepsen worker 40 - jepsen.util 130 :ok :read [13 48] INFO [2022-01-10 12:53:42,143] jepsen worker 40 - jepsen.util 130 :invoke :write [13 20] INFO [2022-01-10 12:53:42,148] jepsen worker 80 - jepsen.util 260 :ok :write [26 0] INFO [2022-01-10 12:53:42,148] jepsen worker 68 - jepsen.util 68 :ok :write [22 16] INFO [2022-01-10 12:53:42,149] jepsen worker 80 - jepsen.util 260 :invoke :write [26 17] INFO [2022-01-10 12:53:42,149] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,150] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:42,151] jepsen worker 35 - jepsen.util 215 :ok :write [11 30] INFO [2022-01-10 12:53:42,151] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,152] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,152] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:42,152] jepsen worker 35 - jepsen.util 215 :ok :read [11 30] INFO [2022-01-10 12:53:42,153] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,153] jepsen worker 35 - jepsen.util 215 :invoke :write [11 24] INFO [2022-01-10 12:53:42,154] jepsen worker 68 - jepsen.util 68 :ok :read [22 16] INFO [2022-01-10 12:53:42,154] jepsen worker 68 - jepsen.util 68 :invoke :write [22 48] INFO [2022-01-10 12:53:42,157] jepsen worker 51 - jepsen.util 141 :ok :write [17 20] INFO [2022-01-10 12:53:42,158] jepsen worker 51 - jepsen.util 141 :invoke :write [17 7] INFO [2022-01-10 12:53:42,161] jepsen worker 7 - jepsen.util 7 :ok :write [2 45] INFO [2022-01-10 12:53:42,161] jepsen worker 89 - jepsen.util 269 :ok :write [29 43] INFO [2022-01-10 12:53:42,162] jepsen worker 7 - jepsen.util 7 :invoke :write [2 6] INFO [2022-01-10 12:53:42,162] jepsen worker 89 - jepsen.util 269 :invoke :write [29 27] INFO [2022-01-10 12:53:42,164] jepsen worker 86 - jepsen.util 176 :ok :write [28 4] INFO [2022-01-10 12:53:42,165] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,166] jepsen worker 86 - jepsen.util 176 :ok :read [28 4] INFO [2022-01-10 12:53:42,167] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,168] jepsen worker 86 - jepsen.util 176 :ok :read [28 4] INFO [2022-01-10 12:53:42,168] jepsen worker 86 - jepsen.util 176 :invoke :write [28 3] INFO [2022-01-10 12:53:42,169] jepsen worker 16 - jepsen.util 286 :ok :write [5 17] INFO [2022-01-10 12:53:42,170] jepsen worker 2 - jepsen.util 92 :ok :write [0 32] INFO [2022-01-10 12:53:42,170] jepsen worker 16 - jepsen.util 286 :invoke :write [5 19] INFO [2022-01-10 12:53:42,171] jepsen worker 14 - jepsen.util 284 :ok :write [4 15] INFO [2022-01-10 12:53:42,171] jepsen worker 2 - jepsen.util 92 :invoke :write [0 49] INFO [2022-01-10 12:53:42,171] jepsen worker 14 - jepsen.util 284 :invoke :write [4 31] INFO [2022-01-10 12:53:42,175] jepsen worker 42 - jepsen.util 222 :ok :write [14 19] INFO [2022-01-10 12:53:42,176] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:42,177] jepsen worker 42 - jepsen.util 222 :ok :read [14 19] INFO [2022-01-10 12:53:42,177] jepsen worker 42 - jepsen.util 222 :invoke :write [14 22] INFO [2022-01-10 12:53:42,179] jepsen worker 5 - jepsen.util 185 :ok :write [1 7] INFO [2022-01-10 12:53:42,180] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,180] jepsen worker 48 - jepsen.util 228 :ok :write [16 36] INFO [2022-01-10 12:53:42,181] jepsen worker 5 - jepsen.util 185 :ok :read [1 7] INFO [2022-01-10 12:53:42,181] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:42,182] jepsen worker 5 - jepsen.util 185 :invoke :write [1 21] INFO [2022-01-10 12:53:42,182] jepsen worker 48 - jepsen.util 228 :ok :read [16 36] INFO [2022-01-10 12:53:42,183] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:42,183] jepsen worker 21 - jepsen.util 201 :ok :write [7 6] INFO [2022-01-10 12:53:42,183] jepsen worker 40 - jepsen.util 130 :ok :write [13 20] INFO [2022-01-10 12:53:42,183] jepsen worker 48 - jepsen.util 228 :ok :read [16 36] INFO [2022-01-10 12:53:42,184] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,185] jepsen worker 48 - jepsen.util 228 :invoke :write [16 3] INFO [2022-01-10 12:53:42,185] jepsen worker 21 - jepsen.util 201 :ok :read [7 6] INFO [2022-01-10 12:53:42,185] jepsen worker 40 - jepsen.util 130 :invoke :write [13 49] INFO [2022-01-10 12:53:42,186] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,187] jepsen worker 21 - jepsen.util 201 :ok :read [7 6] INFO [2022-01-10 12:53:42,188] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,188] jepsen worker 69 - jepsen.util 159 :ok :write [23 47] INFO [2022-01-10 12:53:42,188] jepsen worker 21 - jepsen.util 201 :ok :read [7 6] INFO [2022-01-10 12:53:42,189] jepsen worker 69 - jepsen.util 159 :invoke :write [23 5] INFO [2022-01-10 12:53:42,189] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,190] jepsen worker 21 - jepsen.util 201 :ok :read [7 6] INFO [2022-01-10 12:53:42,191] jepsen worker 83 - jepsen.util 263 :ok :write [27 49] INFO [2022-01-10 12:53:42,191] jepsen worker 21 - jepsen.util 201 :invoke :write [7 10] INFO [2022-01-10 12:53:42,192] jepsen worker 83 - jepsen.util 263 :invoke :write [27 34] INFO [2022-01-10 12:53:42,195] jepsen worker 60 - jepsen.util 240 :ok :write [20 36] INFO [2022-01-10 12:53:42,195] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,196] jepsen worker 60 - jepsen.util 240 :ok :read [20 36] INFO [2022-01-10 12:53:42,196] jepsen worker 38 - jepsen.util 38 :ok :write [12 38] INFO [2022-01-10 12:53:42,197] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,197] jepsen worker 38 - jepsen.util 38 :invoke :write [12 44] INFO [2022-01-10 12:53:42,197] jepsen worker 60 - jepsen.util 240 :ok :read [20 36] INFO [2022-01-10 12:53:42,198] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,200] jepsen worker 60 - jepsen.util 240 :ok :read [20 36] INFO [2022-01-10 12:53:42,200] jepsen worker 76 - jepsen.util 256 :ok :write [25 8] INFO [2022-01-10 12:53:42,200] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,201] jepsen worker 60 - jepsen.util 240 :ok :read [20 36] INFO [2022-01-10 12:53:42,201] jepsen worker 76 - jepsen.util 256 :invoke :write [25 17] INFO [2022-01-10 12:53:42,202] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,203] jepsen worker 60 - jepsen.util 240 :ok :read [20 36] INFO [2022-01-10 12:53:42,204] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,204] jepsen worker 80 - jepsen.util 260 :ok :write [26 17] INFO [2022-01-10 12:53:42,204] jepsen worker 60 - jepsen.util 240 :ok :read [20 36] INFO [2022-01-10 12:53:42,205] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,205] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,206] jepsen worker 80 - jepsen.util 260 :ok :read [26 17] INFO [2022-01-10 12:53:42,206] jepsen worker 60 - jepsen.util 240 :ok :read [20 36] INFO [2022-01-10 12:53:42,206] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,207] jepsen worker 68 - jepsen.util 68 :ok :write [22 48] INFO [2022-01-10 12:53:42,207] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,207] jepsen worker 80 - jepsen.util 260 :ok :read [26 17] INFO [2022-01-10 12:53:42,208] jepsen worker 60 - jepsen.util 240 :ok :read [20 36] INFO [2022-01-10 12:53:42,208] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,208] jepsen worker 60 - jepsen.util 240 :invoke :write [20 12] INFO [2022-01-10 12:53:42,209] jepsen worker 80 - jepsen.util 260 :ok :read [26 17] INFO [2022-01-10 12:53:42,209] jepsen worker 68 - jepsen.util 68 :invoke :write [22 14] INFO [2022-01-10 12:53:42,210] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,211] jepsen worker 80 - jepsen.util 260 :ok :read [26 17] INFO [2022-01-10 12:53:42,211] jepsen worker 89 - jepsen.util 269 :ok :write [29 27] INFO [2022-01-10 12:53:42,212] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,212] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,212] jepsen worker 35 - jepsen.util 215 :ok :write [11 24] INFO [2022-01-10 12:53:42,213] jepsen worker 89 - jepsen.util 269 :ok :read [29 27] INFO [2022-01-10 12:53:42,213] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,213] jepsen worker 80 - jepsen.util 260 :ok :read [26 17] INFO [2022-01-10 12:53:42,213] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,214] jepsen worker 35 - jepsen.util 215 :ok :read [11 24] INFO [2022-01-10 12:53:42,214] jepsen worker 80 - jepsen.util 260 :invoke :write [26 19] INFO [2022-01-10 12:53:42,214] jepsen worker 89 - jepsen.util 269 :ok :read [29 27] INFO [2022-01-10 12:53:42,214] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,215] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,215] jepsen worker 35 - jepsen.util 215 :ok :read [11 24] INFO [2022-01-10 12:53:42,215] jepsen worker 89 - jepsen.util 269 :ok :read [29 27] INFO [2022-01-10 12:53:42,215] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,216] jepsen worker 2 - jepsen.util 92 :ok :write [0 49] INFO [2022-01-10 12:53:42,216] jepsen worker 89 - jepsen.util 269 :invoke :write [29 15] INFO [2022-01-10 12:53:42,216] jepsen worker 35 - jepsen.util 215 :ok :read [11 24] INFO [2022-01-10 12:53:42,216] jepsen worker 2 - jepsen.util 92 :invoke :write [0 45] INFO [2022-01-10 12:53:42,217] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,218] jepsen worker 35 - jepsen.util 215 :ok :read [11 24] INFO [2022-01-10 12:53:42,218] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,218] jepsen worker 7 - jepsen.util 7 :ok :write [2 6] INFO [2022-01-10 12:53:42,219] jepsen worker 35 - jepsen.util 215 :ok :read [11 24] INFO [2022-01-10 12:53:42,219] jepsen worker 7 - jepsen.util 7 :invoke :write [2 37] INFO [2022-01-10 12:53:42,219] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,220] jepsen worker 35 - jepsen.util 215 :ok :read [11 24] INFO [2022-01-10 12:53:42,221] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,221] jepsen worker 35 - jepsen.util 215 :ok :read [11 24] INFO [2022-01-10 12:53:42,222] jepsen worker 35 - jepsen.util 215 :invoke :write [11 42] INFO [2022-01-10 12:53:42,225] jepsen worker 16 - jepsen.util 286 :ok :write [5 19] INFO [2022-01-10 12:53:42,226] jepsen worker 16 - jepsen.util 286 :invoke :write [5 49] INFO [2022-01-10 12:53:42,228] jepsen worker 42 - jepsen.util 222 :ok :write [14 22] INFO [2022-01-10 12:53:42,229] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:42,230] jepsen worker 42 - jepsen.util 222 :ok :read [14 22] INFO [2022-01-10 12:53:42,230] jepsen worker 42 - jepsen.util 222 :invoke :write [14 23] INFO [2022-01-10 12:53:42,234] jepsen worker 21 - jepsen.util 201 :ok :write [7 10] INFO [2022-01-10 12:53:42,234] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,235] jepsen worker 21 - jepsen.util 201 :ok :read [7 10] INFO [2022-01-10 12:53:42,236] jepsen worker 21 - jepsen.util 201 :invoke :write [7 1] INFO [2022-01-10 12:53:42,240] jepsen worker 48 - jepsen.util 228 :ok :write [16 3] INFO [2022-01-10 12:53:42,241] jepsen worker 48 - jepsen.util 228 :invoke :write [16 1] INFO [2022-01-10 12:53:42,244] jepsen worker 83 - jepsen.util 263 :ok :write [27 34] INFO [2022-01-10 12:53:42,244] jepsen worker 83 - jepsen.util 263 :invoke :write [27 28] INFO [2022-01-10 12:53:42,251] jepsen worker 40 - jepsen.util 130 :ok :write [13 49] INFO [2022-01-10 12:53:42,251] jepsen worker 40 - jepsen.util 130 :invoke :write [13 7] INFO [2022-01-10 12:53:42,252] jepsen worker 51 - jepsen.util 141 :ok :write [17 7] INFO [2022-01-10 12:53:42,253] jepsen worker 51 - jepsen.util 141 :invoke :write [17 9] INFO [2022-01-10 12:53:42,255] jepsen worker 60 - jepsen.util 240 :ok :write [20 12] INFO [2022-01-10 12:53:42,256] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,257] jepsen worker 60 - jepsen.util 240 :ok :read [20 12] INFO [2022-01-10 12:53:42,257] jepsen worker 60 - jepsen.util 240 :invoke :write [20 34] INFO [2022-01-10 12:53:42,259] jepsen worker 68 - jepsen.util 68 :ok :write [22 14] INFO [2022-01-10 12:53:42,260] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,261] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:42,261] jepsen worker 76 - jepsen.util 256 :ok :write [25 17] INFO [2022-01-10 12:53:42,261] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,262] jepsen worker 86 - jepsen.util 176 :ok :write [28 3] INFO [2022-01-10 12:53:42,262] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,262] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,262] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:42,263] jepsen worker 76 - jepsen.util 256 :ok :read [25 17] INFO [2022-01-10 12:53:42,263] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,264] jepsen worker 35 - jepsen.util 215 :ok :write [11 42] INFO [2022-01-10 12:53:42,264] jepsen worker 86 - jepsen.util 176 :ok :read [28 3] INFO [2022-01-10 12:53:42,264] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,264] jepsen worker 68 - jepsen.util 68 :ok :read [22 14] INFO [2022-01-10 12:53:42,265] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,265] jepsen worker 76 - jepsen.util 256 :ok :read [25 17] INFO [2022-01-10 12:53:42,265] jepsen worker 68 - jepsen.util 68 :invoke :write [22 0] INFO [2022-01-10 12:53:42,266] jepsen worker 35 - jepsen.util 215 :ok :read [11 42] INFO [2022-01-10 12:53:42,266] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,267] jepsen worker 14 - jepsen.util 284 :ok :write [4 31] INFO [2022-01-10 12:53:42,267] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,267] jepsen worker 86 - jepsen.util 176 :ok :read [28 3] INFO [2022-01-10 12:53:42,268] jepsen worker 14 - jepsen.util 284 :invoke :write [4 11] INFO [2022-01-10 12:53:42,268] jepsen worker 89 - jepsen.util 269 :ok :write [29 15] INFO [2022-01-10 12:53:42,268] jepsen worker 35 - jepsen.util 215 :ok :read [11 42] INFO [2022-01-10 12:53:42,268] jepsen worker 76 - jepsen.util 256 :invoke :write [25 13] INFO [2022-01-10 12:53:42,269] jepsen worker 35 - jepsen.util 215 :invoke :write [11 44] INFO [2022-01-10 12:53:42,270] jepsen worker 89 - jepsen.util 269 :invoke :write [29 27] INFO [2022-01-10 12:53:42,270] jepsen worker 86 - jepsen.util 176 :invoke :write [28 2] INFO [2022-01-10 12:53:42,271] jepsen worker 7 - jepsen.util 7 :ok :write [2 37] INFO [2022-01-10 12:53:42,272] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:42,272] jepsen worker 7 - jepsen.util 7 :ok :read [2 37] INFO [2022-01-10 12:53:42,273] jepsen worker 7 - jepsen.util 7 :invoke :write [2 24] INFO [2022-01-10 12:53:42,274] jepsen worker 57 - jepsen.util 327 :info :write [19 39] :RequestTimeoutException INFO [2022-01-10 12:53:42,275] jepsen worker 5 - jepsen.util 185 :ok :write [1 21] INFO [2022-01-10 12:53:42,275] jepsen worker 57 - jepsen.util 417 :invoke :read [19 nil] INFO [2022-01-10 12:53:42,276] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,276] jepsen worker 57 - jepsen.util 417 :ok :read [19 48] INFO [2022-01-10 12:53:42,277] jepsen worker 5 - jepsen.util 185 :ok :read [1 21] INFO [2022-01-10 12:53:42,277] jepsen worker 57 - jepsen.util 417 :invoke :read [19 nil] INFO [2022-01-10 12:53:42,277] jepsen worker 2 - jepsen.util 92 :ok :write [0 45] INFO [2022-01-10 12:53:42,278] jepsen worker 16 - jepsen.util 286 :ok :write [5 49] INFO [2022-01-10 12:53:42,278] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,278] jepsen worker 57 - jepsen.util 417 :ok :read [19 48] INFO [2022-01-10 12:53:42,278] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,279] jepsen worker 5 - jepsen.util 185 :ok :read [1 21] INFO [2022-01-10 12:53:42,279] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,279] jepsen worker 16 - jepsen.util 286 :ok :read [5 49] INFO [2022-01-10 12:53:42,280] jepsen worker 5 - jepsen.util 185 :invoke :write [1 26] INFO [2022-01-10 12:53:42,280] jepsen worker 2 - jepsen.util 92 :ok :read [0 45] INFO [2022-01-10 12:53:42,280] jepsen worker 69 - jepsen.util 159 :ok :write [23 5] INFO [2022-01-10 12:53:42,281] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,281] jepsen worker 2 - jepsen.util 92 :invoke :write [0 24] INFO [2022-01-10 12:53:42,282] jepsen worker 16 - jepsen.util 286 :ok :read [5 49] INFO [2022-01-10 12:53:42,282] jepsen worker 69 - jepsen.util 159 :invoke :write [23 32] INFO [2022-01-10 12:53:42,282] jepsen worker 57 - jepsen.util 417 :invoke :write [19 39] INFO [2022-01-10 12:53:42,283] jepsen worker 42 - jepsen.util 222 :ok :write [14 23] INFO [2022-01-10 12:53:42,283] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,283] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:42,284] jepsen worker 16 - jepsen.util 286 :ok :read [5 49] INFO [2022-01-10 12:53:42,284] jepsen worker 42 - jepsen.util 222 :ok :read [14 23] INFO [2022-01-10 12:53:42,284] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,285] jepsen worker 42 - jepsen.util 222 :invoke :write [14 12] INFO [2022-01-10 12:53:42,285] jepsen worker 16 - jepsen.util 286 :ok :read [5 49] INFO [2022-01-10 12:53:42,286] jepsen worker 16 - jepsen.util 286 :invoke :write [5 49] INFO [2022-01-10 12:53:42,287] jepsen worker 38 - jepsen.util 38 :ok :write [12 44] INFO [2022-01-10 12:53:42,288] jepsen worker 38 - jepsen.util 38 :invoke :write [12 40] INFO [2022-01-10 12:53:42,292] jepsen worker 80 - jepsen.util 260 :ok :write [26 19] INFO [2022-01-10 12:53:42,292] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,293] jepsen worker 80 - jepsen.util 260 :ok :read [26 19] INFO [2022-01-10 12:53:42,293] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,294] jepsen worker 80 - jepsen.util 260 :ok :read [26 19] INFO [2022-01-10 12:53:42,295] jepsen worker 80 - jepsen.util 260 :invoke :write [26 32] INFO [2022-01-10 12:53:42,297] jepsen worker 51 - jepsen.util 141 :ok :write [17 9] INFO [2022-01-10 12:53:42,297] jepsen worker 40 - jepsen.util 130 :ok :write [13 7] INFO [2022-01-10 12:53:42,298] jepsen worker 51 - jepsen.util 141 :invoke :write [17 2] INFO [2022-01-10 12:53:42,298] jepsen worker 40 - jepsen.util 130 :invoke :write [13 46] INFO [2022-01-10 12:53:42,304] jepsen worker 86 - jepsen.util 176 :ok :write [28 2] INFO [2022-01-10 12:53:42,304] jepsen worker 86 - jepsen.util 176 :invoke :write [28 23] INFO [2022-01-10 12:53:42,309] jepsen worker 14 - jepsen.util 284 :ok :write [4 11] INFO [2022-01-10 12:53:42,309] jepsen worker 14 - jepsen.util 284 :invoke :write [4 43] INFO [2022-01-10 12:53:42,314] jepsen worker 5 - jepsen.util 185 :ok :write [1 26] INFO [2022-01-10 12:53:42,315] jepsen worker 89 - jepsen.util 269 :ok :write [29 27] INFO [2022-01-10 12:53:42,315] jepsen worker 5 - jepsen.util 185 :invoke :write [1 15] INFO [2022-01-10 12:53:42,316] jepsen worker 89 - jepsen.util 269 :invoke :write [29 7] INFO [2022-01-10 12:53:42,321] jepsen worker 2 - jepsen.util 92 :ok :write [0 24] INFO [2022-01-10 12:53:42,321] jepsen worker 2 - jepsen.util 92 :invoke :write [0 44] INFO [2022-01-10 12:53:42,325] jepsen worker 38 - jepsen.util 38 :ok :write [12 40] INFO [2022-01-10 12:53:42,326] jepsen worker 38 - jepsen.util 38 :invoke :write [12 38] INFO [2022-01-10 12:53:42,332] jepsen worker 40 - jepsen.util 130 :ok :write [13 46] INFO [2022-01-10 12:53:42,333] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:42,334] jepsen worker 40 - jepsen.util 130 :ok :read [13 46] INFO [2022-01-10 12:53:42,334] jepsen worker 40 - jepsen.util 130 :invoke :write [13 41] INFO [2022-01-10 12:53:42,338] jepsen worker 89 - jepsen.util 269 :ok :write [29 7] INFO [2022-01-10 12:53:42,338] jepsen worker 89 - jepsen.util 269 :invoke :write [29 12] INFO [2022-01-10 12:53:42,343] jepsen worker 2 - jepsen.util 92 :ok :write [0 44] INFO [2022-01-10 12:53:42,343] jepsen worker 86 - jepsen.util 176 :ok :write [28 23] INFO [2022-01-10 12:53:42,343] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,344] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,345] jepsen worker 86 - jepsen.util 176 :ok :read [28 23] INFO [2022-01-10 12:53:42,345] jepsen worker 2 - jepsen.util 92 :ok :read [0 44] INFO [2022-01-10 12:53:42,345] jepsen worker 86 - jepsen.util 176 :invoke :write [28 48] INFO [2022-01-10 12:53:42,346] jepsen worker 2 - jepsen.util 92 :invoke :write [0 48] INFO [2022-01-10 12:53:42,351] jepsen worker 40 - jepsen.util 130 :ok :write [13 41] INFO [2022-01-10 12:53:42,352] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:42,354] jepsen worker 21 - jepsen.util 201 :ok :write [7 1] INFO [2022-01-10 12:53:42,354] jepsen worker 5 - jepsen.util 185 :ok :write [1 15] INFO [2022-01-10 12:53:42,355] jepsen worker 10 - jepsen.util 280 :info :write [3 5] :RequestTimeoutException INFO [2022-01-10 12:53:42,355] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,355] jepsen worker 21 - jepsen.util 201 :invoke :write [7 11] INFO [2022-01-10 12:53:42,356] jepsen worker 5 - jepsen.util 185 :ok :read [1 15] INFO [2022-01-10 12:53:42,356] jepsen worker 10 - jepsen.util 370 :invoke :read [3 nil] INFO [2022-01-10 12:53:42,357] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,358] jepsen worker 5 - jepsen.util 185 :ok :read [1 15] INFO [2022-01-10 12:53:42,358] jepsen worker 5 - jepsen.util 185 :invoke :write [1 47] INFO [2022-01-10 12:53:42,359] jepsen worker 89 - jepsen.util 269 :ok :write [29 12] INFO [2022-01-10 12:53:42,359] jepsen worker 89 - jepsen.util 269 :invoke :write [29 4] INFO [2022-01-10 12:53:42,361] jepsen worker 48 - jepsen.util 228 :ok :write [16 1] INFO [2022-01-10 12:53:42,362] jepsen worker 48 - jepsen.util 228 :invoke :write [16 15] INFO [2022-01-10 12:53:42,364] jepsen worker 38 - jepsen.util 38 :ok :write [12 38] INFO [2022-01-10 12:53:42,364] jepsen worker 83 - jepsen.util 263 :ok :write [27 28] INFO [2022-01-10 12:53:42,365] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:42,365] jepsen worker 83 - jepsen.util 263 :invoke :write [27 48] INFO [2022-01-10 12:53:42,366] jepsen worker 38 - jepsen.util 38 :ok :read [12 38] INFO [2022-01-10 12:53:42,367] jepsen worker 69 - jepsen.util 159 :ok :write [23 32] INFO [2022-01-10 12:53:42,367] jepsen worker 38 - jepsen.util 38 :invoke :write [12 34] INFO [2022-01-10 12:53:42,367] jepsen worker 60 - jepsen.util 240 :ok :write [20 34] INFO [2022-01-10 12:53:42,367] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,368] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,368] jepsen worker 69 - jepsen.util 159 :ok :read [23 32] INFO [2022-01-10 12:53:42,369] jepsen worker 69 - jepsen.util 159 :invoke :write [23 3] INFO [2022-01-10 12:53:42,369] jepsen worker 60 - jepsen.util 240 :ok :read [20 34] INFO [2022-01-10 12:53:42,370] jepsen worker 60 - jepsen.util 240 :invoke :write [20 46] INFO [2022-01-10 12:53:42,373] jepsen worker 86 - jepsen.util 176 :ok :write [28 48] INFO [2022-01-10 12:53:42,373] jepsen worker 2 - jepsen.util 92 :ok :write [0 48] INFO [2022-01-10 12:53:42,373] jepsen worker 86 - jepsen.util 176 :invoke :write [28 44] INFO [2022-01-10 12:53:42,374] jepsen worker 2 - jepsen.util 92 :invoke :write [0 38] INFO [2022-01-10 12:53:42,375] jepsen worker 40 - jepsen.util 130 :ok :read [13 41] INFO [2022-01-10 12:53:42,375] jepsen worker 68 - jepsen.util 68 :ok :write [22 0] INFO [2022-01-10 12:53:42,375] jepsen worker 40 - jepsen.util 130 :invoke :write [13 46] INFO [2022-01-10 12:53:42,376] jepsen worker 80 - jepsen.util 260 :ok :write [26 32] INFO [2022-01-10 12:53:42,376] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,377] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,377] jepsen worker 68 - jepsen.util 68 :ok :read [22 0] INFO [2022-01-10 12:53:42,378] jepsen worker 80 - jepsen.util 260 :ok :read [26 32] INFO [2022-01-10 12:53:42,378] jepsen worker 76 - jepsen.util 256 :ok :write [25 13] INFO [2022-01-10 12:53:42,378] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,379] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,379] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,380] jepsen worker 68 - jepsen.util 68 :ok :read [22 0] INFO [2022-01-10 12:53:42,380] jepsen worker 5 - jepsen.util 185 :ok :write [1 47] INFO [2022-01-10 12:53:42,380] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,381] jepsen worker 76 - jepsen.util 256 :ok :read [25 13] INFO [2022-01-10 12:53:42,381] jepsen worker 5 - jepsen.util 185 :invoke :write [1 23] INFO [2022-01-10 12:53:42,381] jepsen worker 80 - jepsen.util 260 :ok :read [26 32] INFO [2022-01-10 12:53:42,381] jepsen worker 68 - jepsen.util 68 :ok :read [22 0] INFO [2022-01-10 12:53:42,381] jepsen worker 76 - jepsen.util 256 :invoke :write [25 16] INFO [2022-01-10 12:53:42,382] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,382] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,383] jepsen worker 51 - jepsen.util 141 :ok :write [17 2] INFO [2022-01-10 12:53:42,383] jepsen worker 80 - jepsen.util 260 :ok :read [26 32] INFO [2022-01-10 12:53:42,383] jepsen worker 35 - jepsen.util 215 :ok :write [11 44] INFO [2022-01-10 12:53:42,384] jepsen worker 68 - jepsen.util 68 :ok :read [22 0] INFO [2022-01-10 12:53:42,384] jepsen worker 51 - jepsen.util 141 :invoke :write [17 42] INFO [2022-01-10 12:53:42,385] jepsen worker 35 - jepsen.util 215 :invoke :write [11 44] INFO [2022-01-10 12:53:42,385] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,386] jepsen worker 80 - jepsen.util 260 :invoke :write [26 34] INFO [2022-01-10 12:53:42,386] jepsen worker 68 - jepsen.util 68 :ok :read [22 0] INFO [2022-01-10 12:53:42,388] jepsen worker 68 - jepsen.util 68 :invoke :write [22 18] INFO [2022-01-10 12:53:42,389] jepsen worker 14 - jepsen.util 284 :ok :write [4 43] INFO [2022-01-10 12:53:42,390] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,391] jepsen worker 14 - jepsen.util 284 :ok :read [4 43] INFO [2022-01-10 12:53:42,391] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,391] jepsen worker 89 - jepsen.util 269 :ok :write [29 4] INFO [2022-01-10 12:53:42,391] jepsen worker 38 - jepsen.util 38 :ok :write [12 34] INFO [2022-01-10 12:53:42,392] jepsen worker 14 - jepsen.util 284 :ok :read [4 43] INFO [2022-01-10 12:53:42,392] jepsen worker 89 - jepsen.util 269 :invoke :write [29 4] INFO [2022-01-10 12:53:42,392] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:42,393] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,393] jepsen worker 38 - jepsen.util 38 :ok :read [12 34] INFO [2022-01-10 12:53:42,393] jepsen worker 14 - jepsen.util 284 :ok :read [4 43] INFO [2022-01-10 12:53:42,394] jepsen worker 38 - jepsen.util 38 :invoke :write [12 12] INFO [2022-01-10 12:53:42,394] jepsen worker 14 - jepsen.util 284 :invoke :write [4 29] INFO [2022-01-10 12:53:42,396] jepsen worker 7 - jepsen.util 7 :ok :write [2 24] INFO [2022-01-10 12:53:42,396] jepsen worker 7 - jepsen.util 7 :invoke :write [2 49] INFO [2022-01-10 12:53:42,401] jepsen worker 69 - jepsen.util 159 :ok :write [23 3] INFO [2022-01-10 12:53:42,401] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,402] jepsen worker 69 - jepsen.util 159 :ok :read [23 3] INFO [2022-01-10 12:53:42,403] jepsen worker 69 - jepsen.util 159 :invoke :write [23 19] INFO [2022-01-10 12:53:42,404] jepsen worker 16 - jepsen.util 286 :ok :write [5 49] INFO [2022-01-10 12:53:42,404] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,405] jepsen worker 16 - jepsen.util 286 :ok :read [5 49] INFO [2022-01-10 12:53:42,406] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,407] jepsen worker 16 - jepsen.util 286 :ok :read [5 49] INFO [2022-01-10 12:53:42,407] jepsen worker 86 - jepsen.util 176 :ok :write [28 44] INFO [2022-01-10 12:53:42,407] jepsen worker 0 - jepsen.util 180 :ok :write [0 0] INFO [2022-01-10 12:53:42,408] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,408] jepsen worker 0 - jepsen.util 180 :invoke :write [0 48] INFO [2022-01-10 12:53:42,409] jepsen worker 86 - jepsen.util 176 :ok :read [28 44] INFO [2022-01-10 12:53:42,409] jepsen worker 16 - jepsen.util 286 :invoke :write [5 18] INFO [2022-01-10 12:53:42,409] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,410] jepsen worker 42 - jepsen.util 222 :ok :write [14 12] INFO [2022-01-10 12:53:42,411] jepsen worker 86 - jepsen.util 176 :ok :read [28 44] INFO [2022-01-10 12:53:42,411] jepsen worker 80 - jepsen.util 260 :ok :write [26 34] INFO [2022-01-10 12:53:42,411] jepsen worker 86 - jepsen.util 176 :invoke :write [28 30] INFO [2022-01-10 12:53:42,412] jepsen worker 42 - jepsen.util 222 :invoke :write [14 27] INFO [2022-01-10 12:53:42,412] jepsen worker 80 - jepsen.util 260 :invoke :write [26 30] INFO [2022-01-10 12:53:42,417] jepsen worker 21 - jepsen.util 201 :ok :write [7 11] INFO [2022-01-10 12:53:42,418] jepsen worker 21 - jepsen.util 201 :invoke :write [7 22] INFO [2022-01-10 12:53:42,420] jepsen worker 5 - jepsen.util 185 :ok :write [1 23] INFO [2022-01-10 12:53:42,420] jepsen worker 5 - jepsen.util 185 :invoke :write [1 9] INFO [2022-01-10 12:53:42,421] jepsen worker 40 - jepsen.util 130 :ok :write [13 46] INFO [2022-01-10 12:53:42,422] jepsen worker 40 - jepsen.util 130 :invoke :write [13 37] INFO [2022-01-10 12:53:42,422] jepsen worker 48 - jepsen.util 228 :ok :write [16 15] INFO [2022-01-10 12:53:42,423] jepsen worker 48 - jepsen.util 228 :invoke :write [16 15] INFO [2022-01-10 12:53:42,428] jepsen worker 51 - jepsen.util 141 :ok :write [17 42] INFO [2022-01-10 12:53:42,429] jepsen worker 51 - jepsen.util 141 :invoke :write [17 2] INFO [2022-01-10 12:53:42,431] jepsen worker 83 - jepsen.util 263 :ok :write [27 48] INFO [2022-01-10 12:53:42,432] jepsen worker 83 - jepsen.util 263 :invoke :write [27 13] INFO [2022-01-10 12:53:42,435] jepsen worker 14 - jepsen.util 284 :ok :write [4 29] INFO [2022-01-10 12:53:42,435] jepsen worker 89 - jepsen.util 269 :ok :write [29 4] INFO [2022-01-10 12:53:42,435] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,436] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,436] jepsen worker 14 - jepsen.util 284 :ok :read [4 29] INFO [2022-01-10 12:53:42,437] jepsen worker 89 - jepsen.util 269 :ok :read [29 4] INFO [2022-01-10 12:53:42,437] jepsen worker 14 - jepsen.util 284 :invoke :write [4 29] INFO [2022-01-10 12:53:42,437] jepsen worker 89 - jepsen.util 269 :invoke :write [29 10] INFO [2022-01-10 12:53:42,438] jepsen worker 60 - jepsen.util 240 :ok :write [20 46] INFO [2022-01-10 12:53:42,438] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,439] jepsen worker 60 - jepsen.util 240 :ok :read [20 46] INFO [2022-01-10 12:53:42,440] jepsen worker 60 - jepsen.util 240 :invoke :write [20 23] INFO [2022-01-10 12:53:42,443] jepsen worker 76 - jepsen.util 256 :ok :write [25 16] INFO [2022-01-10 12:53:42,444] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,444] jepsen worker 38 - jepsen.util 38 :ok :write [12 12] INFO [2022-01-10 12:53:42,444] jepsen worker 76 - jepsen.util 256 :ok :read [25 16] INFO [2022-01-10 12:53:42,445] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:42,446] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,446] jepsen worker 38 - jepsen.util 38 :ok :read [12 12] INFO [2022-01-10 12:53:42,446] jepsen worker 76 - jepsen.util 256 :ok :read [25 16] INFO [2022-01-10 12:53:42,447] jepsen worker 38 - jepsen.util 38 :invoke :write [12 38] INFO [2022-01-10 12:53:42,447] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,448] jepsen worker 76 - jepsen.util 256 :ok :read [25 16] INFO [2022-01-10 12:53:42,449] jepsen worker 76 - jepsen.util 256 :invoke :write [25 2] INFO [2022-01-10 12:53:42,451] jepsen worker 68 - jepsen.util 68 :ok :write [22 18] INFO [2022-01-10 12:53:42,451] jepsen worker 68 - jepsen.util 68 :invoke :write [22 43] INFO [2022-01-10 12:53:42,453] jepsen worker 69 - jepsen.util 159 :ok :write [23 19] INFO [2022-01-10 12:53:42,454] jepsen worker 69 - jepsen.util 159 :invoke :write [23 27] INFO [2022-01-10 12:53:42,454] jepsen worker 35 - jepsen.util 215 :ok :write [11 44] INFO [2022-01-10 12:53:42,455] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,455] jepsen worker 35 - jepsen.util 215 :ok :read [11 44] INFO [2022-01-10 12:53:42,456] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,457] jepsen worker 35 - jepsen.util 215 :ok :read [11 44] INFO [2022-01-10 12:53:42,457] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,458] jepsen worker 35 - jepsen.util 215 :ok :read [11 44] INFO [2022-01-10 12:53:42,459] jepsen worker 35 - jepsen.util 215 :invoke :write [11 44] INFO [2022-01-10 12:53:42,460] jepsen worker 7 - jepsen.util 7 :ok :write [2 49] INFO [2022-01-10 12:53:42,461] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:42,461] jepsen worker 86 - jepsen.util 176 :ok :write [28 30] INFO [2022-01-10 12:53:42,462] jepsen worker 7 - jepsen.util 7 :ok :read [2 49] INFO [2022-01-10 12:53:42,462] jepsen worker 86 - jepsen.util 176 :invoke :write [28 11] INFO [2022-01-10 12:53:42,463] jepsen worker 7 - jepsen.util 7 :invoke :write [2 26] INFO [2022-01-10 12:53:42,465] jepsen worker 80 - jepsen.util 260 :ok :write [26 30] INFO [2022-01-10 12:53:42,466] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,466] jepsen worker 80 - jepsen.util 260 :ok :read [26 30] INFO [2022-01-10 12:53:42,467] jepsen worker 16 - jepsen.util 286 :ok :write [5 18] INFO [2022-01-10 12:53:42,467] jepsen worker 80 - jepsen.util 260 :invoke :write [26 9] INFO [2022-01-10 12:53:42,467] jepsen worker 16 - jepsen.util 286 :invoke :write [5 19] INFO [2022-01-10 12:53:42,471] jepsen worker 42 - jepsen.util 222 :ok :write [14 27] INFO [2022-01-10 12:53:42,471] jepsen worker 5 - jepsen.util 185 :ok :write [1 9] INFO [2022-01-10 12:53:42,472] jepsen worker 42 - jepsen.util 222 :invoke :write [14 36] INFO [2022-01-10 12:53:42,472] jepsen worker 5 - jepsen.util 185 :invoke :write [1 2] INFO [2022-01-10 12:53:42,474] jepsen worker 21 - jepsen.util 201 :ok :write [7 22] INFO [2022-01-10 12:53:42,475] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,476] jepsen worker 21 - jepsen.util 201 :ok :read [7 22] INFO [2022-01-10 12:53:42,476] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,477] jepsen worker 21 - jepsen.util 201 :ok :read [7 22] INFO [2022-01-10 12:53:42,478] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,478] jepsen worker 51 - jepsen.util 141 :ok :write [17 2] INFO [2022-01-10 12:53:42,479] jepsen worker 21 - jepsen.util 201 :ok :read [7 22] INFO [2022-01-10 12:53:42,479] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:42,479] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,480] jepsen worker 51 - jepsen.util 141 :ok :read [17 2] INFO [2022-01-10 12:53:42,480] jepsen worker 21 - jepsen.util 201 :ok :read [7 22] INFO [2022-01-10 12:53:42,480] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:42,481] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,481] jepsen worker 51 - jepsen.util 141 :ok :read [17 2] INFO [2022-01-10 12:53:42,482] jepsen worker 21 - jepsen.util 201 :ok :read [7 22] INFO [2022-01-10 12:53:42,482] jepsen worker 51 - jepsen.util 141 :invoke :write [17 17] INFO [2022-01-10 12:53:42,483] jepsen worker 21 - jepsen.util 201 :invoke :write [7 21] INFO [2022-01-10 12:53:42,488] jepsen worker 14 - jepsen.util 284 :ok :write [4 29] INFO [2022-01-10 12:53:42,489] jepsen worker 14 - jepsen.util 284 :invoke :write [4 34] INFO [2022-01-10 12:53:42,491] jepsen worker 60 - jepsen.util 240 :ok :write [20 23] INFO [2022-01-10 12:53:42,496] jepsen worker 60 - jepsen.util 240 :invoke :write [20 4] INFO [2022-01-10 12:53:42,496] jepsen worker 76 - jepsen.util 256 :ok :write [25 2] INFO [2022-01-10 12:53:42,497] jepsen worker 76 - jepsen.util 256 :invoke :write [25 19] INFO [2022-01-10 12:53:42,499] jepsen worker 69 - jepsen.util 159 :ok :write [23 27] INFO [2022-01-10 12:53:42,499] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,500] jepsen worker 69 - jepsen.util 159 :ok :read [23 27] INFO [2022-01-10 12:53:42,501] jepsen worker 69 - jepsen.util 159 :invoke :write [23 6] INFO [2022-01-10 12:53:42,521] jepsen worker 42 - jepsen.util 222 :ok :write [14 36] INFO [2022-01-10 12:53:42,522] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:42,523] jepsen worker 42 - jepsen.util 222 :ok :read [14 36] INFO [2022-01-10 12:53:42,523] jepsen worker 42 - jepsen.util 222 :invoke :write [14 32] INFO [2022-01-10 12:53:42,527] jepsen worker 21 - jepsen.util 201 :ok :write [7 21] INFO [2022-01-10 12:53:42,527] jepsen worker 21 - jepsen.util 201 :invoke :write [7 34] INFO [2022-01-10 12:53:42,574] jepsen worker 2 - jepsen.util 92 :ok :write [0 38] INFO [2022-01-10 12:53:42,575] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,576] jepsen worker 2 - jepsen.util 92 :ok :read [0 38] INFO [2022-01-10 12:53:42,576] jepsen worker 2 - jepsen.util 92 :invoke :write [0 0] INFO [2022-01-10 12:53:42,579] jepsen worker 40 - jepsen.util 130 :ok :write [13 37] INFO [2022-01-10 12:53:42,579] jepsen worker 38 - jepsen.util 38 :ok :write [12 38] INFO [2022-01-10 12:53:42,580] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:42,580] jepsen worker 38 - jepsen.util 38 :invoke :write [12 27] INFO [2022-01-10 12:53:42,581] jepsen worker 40 - jepsen.util 130 :ok :read [13 37] INFO [2022-01-10 12:53:42,581] jepsen worker 40 - jepsen.util 130 :invoke :write [13 41] INFO [2022-01-10 12:53:42,585] jepsen worker 76 - jepsen.util 256 :ok :write [25 19] INFO [2022-01-10 12:53:42,585] jepsen worker 76 - jepsen.util 256 :invoke :write [25 27] INFO [2022-01-10 12:53:42,586] jepsen worker 86 - jepsen.util 176 :ok :write [28 11] INFO [2022-01-10 12:53:42,586] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,587] jepsen worker 86 - jepsen.util 176 :ok :read [28 11] INFO [2022-01-10 12:53:42,587] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,588] jepsen worker 86 - jepsen.util 176 :ok :read [28 11] INFO [2022-01-10 12:53:42,589] jepsen worker 48 - jepsen.util 228 :ok :write [16 15] INFO [2022-01-10 12:53:42,589] jepsen worker 60 - jepsen.util 240 :ok :write [20 4] INFO [2022-01-10 12:53:42,589] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:42,590] jepsen worker 48 - jepsen.util 228 :ok :read [16 15] INFO [2022-01-10 12:53:42,591] jepsen worker 80 - jepsen.util 260 :ok :write [26 9] INFO [2022-01-10 12:53:42,591] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,591] jepsen worker 80 - jepsen.util 260 :invoke :write [26 24] INFO [2022-01-10 12:53:42,592] jepsen worker 60 - jepsen.util 240 :ok :read [20 4] INFO [2022-01-10 12:53:42,592] jepsen worker 86 - jepsen.util 176 :invoke :write [28 10] INFO [2022-01-10 12:53:42,593] jepsen worker 60 - jepsen.util 240 :invoke :write [20 16] INFO [2022-01-10 12:53:42,593] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:42,594] jepsen worker 48 - jepsen.util 228 :ok :read [16 15] INFO [2022-01-10 12:53:42,594] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:42,595] jepsen worker 83 - jepsen.util 263 :ok :write [27 13] INFO [2022-01-10 12:53:42,595] jepsen worker 42 - jepsen.util 222 :ok :write [14 32] INFO [2022-01-10 12:53:42,595] jepsen worker 48 - jepsen.util 228 :ok :read [16 15] INFO [2022-01-10 12:53:42,596] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,596] jepsen worker 5 - jepsen.util 185 :ok :write [1 2] INFO [2022-01-10 12:53:42,596] jepsen worker 83 - jepsen.util 263 :ok :read [27 13] INFO [2022-01-10 12:53:42,597] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:42,597] jepsen worker 42 - jepsen.util 222 :ok :read [14 32] INFO [2022-01-10 12:53:42,598] jepsen worker 5 - jepsen.util 185 :invoke :write [1 30] INFO [2022-01-10 12:53:42,598] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,599] jepsen worker 83 - jepsen.util 263 :ok :read [27 13] INFO [2022-01-10 12:53:42,599] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:42,599] jepsen worker 48 - jepsen.util 228 :invoke :write [16 19] INFO [2022-01-10 12:53:42,600] jepsen worker 42 - jepsen.util 222 :ok :read [14 32] INFO [2022-01-10 12:53:42,600] jepsen worker 83 - jepsen.util 263 :invoke :write [27 42] INFO [2022-01-10 12:53:42,601] jepsen worker 21 - jepsen.util 201 :ok :write [7 34] INFO [2022-01-10 12:53:42,601] jepsen worker 42 - jepsen.util 222 :invoke :write [14 0] INFO [2022-01-10 12:53:42,602] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,603] jepsen worker 89 - jepsen.util 269 :ok :write [29 10] INFO [2022-01-10 12:53:42,603] jepsen worker 21 - jepsen.util 201 :ok :read [7 34] INFO [2022-01-10 12:53:42,603] jepsen worker 51 - jepsen.util 141 :ok :write [17 17] INFO [2022-01-10 12:53:42,603] jepsen worker 89 - jepsen.util 269 :invoke :write [29 18] INFO [2022-01-10 12:53:42,604] jepsen worker 21 - jepsen.util 201 :invoke :write [7 27] INFO [2022-01-10 12:53:42,604] jepsen worker 51 - jepsen.util 141 :invoke :write [17 29] INFO [2022-01-10 12:53:42,606] jepsen worker 68 - jepsen.util 68 :ok :write [22 43] INFO [2022-01-10 12:53:42,606] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,607] jepsen worker 68 - jepsen.util 68 :ok :read [22 43] INFO [2022-01-10 12:53:42,608] jepsen worker 68 - jepsen.util 68 :invoke :write [22 46] INFO [2022-01-10 12:53:42,611] jepsen worker 14 - jepsen.util 284 :ok :write [4 34] INFO [2022-01-10 12:53:42,611] jepsen worker 35 - jepsen.util 215 :ok :write [11 44] INFO [2022-01-10 12:53:42,612] jepsen worker 35 - jepsen.util 215 :invoke :write [11 13] INFO [2022-01-10 12:53:42,612] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,613] jepsen worker 14 - jepsen.util 284 :ok :read [4 34] INFO [2022-01-10 12:53:42,614] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,615] jepsen worker 14 - jepsen.util 284 :ok :read [4 34] INFO [2022-01-10 12:53:42,615] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,616] jepsen worker 7 - jepsen.util 7 :ok :write [2 26] INFO [2022-01-10 12:53:42,616] jepsen worker 14 - jepsen.util 284 :ok :read [4 34] INFO [2022-01-10 12:53:42,616] jepsen worker 7 - jepsen.util 7 :invoke :write [2 1] INFO [2022-01-10 12:53:42,617] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,619] jepsen worker 69 - jepsen.util 159 :ok :write [23 6] INFO [2022-01-10 12:53:42,619] jepsen worker 14 - jepsen.util 284 :ok :read [4 34] INFO [2022-01-10 12:53:42,620] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,620] jepsen worker 16 - jepsen.util 286 :ok :write [5 19] INFO [2022-01-10 12:53:42,620] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,620] jepsen worker 76 - jepsen.util 256 :ok :write [25 27] INFO [2022-01-10 12:53:42,620] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,621] jepsen worker 69 - jepsen.util 159 :ok :read [23 6] INFO [2022-01-10 12:53:42,621] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,621] jepsen worker 16 - jepsen.util 286 :ok :read [5 19] INFO [2022-01-10 12:53:42,621] jepsen worker 14 - jepsen.util 284 :ok :read [4 34] INFO [2022-01-10 12:53:42,622] jepsen worker 69 - jepsen.util 159 :invoke :write [23 46] INFO [2022-01-10 12:53:42,622] jepsen worker 76 - jepsen.util 256 :ok :read [25 27] INFO [2022-01-10 12:53:42,622] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,623] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,623] jepsen worker 16 - jepsen.util 286 :ok :read [5 19] INFO [2022-01-10 12:53:42,624] jepsen worker 2 - jepsen.util 92 :ok :write [0 0] INFO [2022-01-10 12:53:42,624] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,624] jepsen worker 76 - jepsen.util 256 :ok :read [25 27] INFO [2022-01-10 12:53:42,624] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,625] jepsen worker 16 - jepsen.util 286 :ok :read [5 19] INFO [2022-01-10 12:53:42,625] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,625] jepsen worker 14 - jepsen.util 284 :ok :read [4 34] INFO [2022-01-10 12:53:42,626] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,626] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:42,626] jepsen worker 38 - jepsen.util 38 :ok :write [12 27] INFO [2022-01-10 12:53:42,627] jepsen worker 76 - jepsen.util 256 :ok :read [25 27] INFO [2022-01-10 12:53:42,627] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,627] jepsen worker 60 - jepsen.util 240 :ok :write [20 16] INFO [2022-01-10 12:53:42,628] jepsen worker 14 - jepsen.util 284 :invoke :write [4 16] INFO [2022-01-10 12:53:42,628] jepsen worker 16 - jepsen.util 286 :ok :read [5 19] INFO [2022-01-10 12:53:42,628] jepsen worker 60 - jepsen.util 240 :invoke :write [20 5] INFO [2022-01-10 12:53:42,629] jepsen worker 16 - jepsen.util 286 :invoke :write [5 26] INFO [2022-01-10 12:53:42,629] jepsen worker 40 - jepsen.util 130 :ok :write [13 41] INFO [2022-01-10 12:53:42,629] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:42,630] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,630] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:42,631] jepsen worker 38 - jepsen.util 38 :ok :read [12 27] INFO [2022-01-10 12:53:42,631] jepsen worker 40 - jepsen.util 130 :ok :read [13 41] INFO [2022-01-10 12:53:42,631] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:42,632] jepsen worker 76 - jepsen.util 256 :ok :read [25 27] INFO [2022-01-10 12:53:42,632] jepsen worker 40 - jepsen.util 130 :invoke :write [13 49] INFO [2022-01-10 12:53:42,632] jepsen worker 38 - jepsen.util 38 :ok :read [12 27] INFO [2022-01-10 12:53:42,633] jepsen worker 76 - jepsen.util 256 :invoke :write [25 0] INFO [2022-01-10 12:53:42,633] jepsen worker 38 - jepsen.util 38 :invoke :write [12 6] INFO [2022-01-10 12:53:42,633] jepsen worker 42 - jepsen.util 222 :ok :write [14 0] INFO [2022-01-10 12:53:42,634] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,634] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:42,634] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:42,635] jepsen worker 86 - jepsen.util 176 :ok :write [28 10] INFO [2022-01-10 12:53:42,635] jepsen worker 42 - jepsen.util 222 :ok :read [14 0] INFO [2022-01-10 12:53:42,635] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,636] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:42,636] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:42,636] jepsen worker 86 - jepsen.util 176 :invoke :write [28 38] INFO [2022-01-10 12:53:42,637] jepsen worker 42 - jepsen.util 222 :ok :read [14 0] INFO [2022-01-10 12:53:42,637] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,638] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:42,638] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:42,638] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,638] jepsen worker 42 - jepsen.util 222 :ok :read [14 0] INFO [2022-01-10 12:53:42,639] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:42,639] jepsen worker 42 - jepsen.util 222 :invoke :write [14 30] INFO [2022-01-10 12:53:42,640] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,640] jepsen worker 80 - jepsen.util 260 :ok :write [26 24] INFO [2022-01-10 12:53:42,640] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:42,641] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,641] jepsen worker 80 - jepsen.util 260 :invoke :write [26 48] INFO [2022-01-10 12:53:42,642] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:42,642] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,643] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:42,643] jepsen worker 83 - jepsen.util 263 :ok :write [27 42] INFO [2022-01-10 12:53:42,644] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,644] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,644] jepsen worker 43 - jepsen.util 43 :info :write [14 47] :RequestTimeoutException INFO [2022-01-10 12:53:42,644] jepsen worker 83 - jepsen.util 263 :ok :read [27 42] INFO [2022-01-10 12:53:42,645] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:42,645] jepsen worker 43 - jepsen.util 133 :invoke :write [14 8] INFO [2022-01-10 12:53:42,645] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,646] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,646] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:42,646] jepsen worker 5 - jepsen.util 185 :ok :write [1 30] INFO [2022-01-10 12:53:42,647] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,647] jepsen worker 83 - jepsen.util 263 :ok :read [27 42] INFO [2022-01-10 12:53:42,647] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,647] jepsen worker 2 - jepsen.util 92 :ok :read [0 0] INFO [2022-01-10 12:53:42,647] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,648] jepsen worker 2 - jepsen.util 92 :invoke :write [0 35] INFO [2022-01-10 12:53:42,648] jepsen worker 5 - jepsen.util 185 :ok :read [1 30] INFO [2022-01-10 12:53:42,648] jepsen worker 83 - jepsen.util 263 :ok :read [27 42] INFO [2022-01-10 12:53:42,649] jepsen worker 5 - jepsen.util 185 :invoke :write [1 3] INFO [2022-01-10 12:53:42,649] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,650] jepsen worker 83 - jepsen.util 263 :ok :read [27 42] INFO [2022-01-10 12:53:42,650] jepsen worker 83 - jepsen.util 263 :invoke :read [27 nil] INFO [2022-01-10 12:53:42,651] jepsen worker 89 - jepsen.util 269 :ok :write [29 18] INFO [2022-01-10 12:53:42,651] jepsen worker 83 - jepsen.util 263 :ok :read [27 42] INFO [2022-01-10 12:53:42,652] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,652] jepsen worker 83 - jepsen.util 263 :invoke :write [27 34] INFO [2022-01-10 12:53:42,652] jepsen worker 89 - jepsen.util 269 :ok :read [29 18] INFO [2022-01-10 12:53:42,653] jepsen worker 89 - jepsen.util 269 :invoke :write [29 36] INFO [2022-01-10 12:53:42,653] jepsen worker 51 - jepsen.util 141 :ok :write [17 29] INFO [2022-01-10 12:53:42,654] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:42,655] jepsen worker 51 - jepsen.util 141 :ok :read [17 29] INFO [2022-01-10 12:53:42,656] jepsen worker 51 - jepsen.util 141 :invoke :read [17 nil] INFO [2022-01-10 12:53:42,657] jepsen worker 51 - jepsen.util 141 :ok :read [17 29] INFO [2022-01-10 12:53:42,657] jepsen worker 51 - jepsen.util 141 :invoke :write [17 20] INFO [2022-01-10 12:53:42,666] jepsen worker 55 - jepsen.util 325 :info :write [18 38] :RequestTimeoutException INFO [2022-01-10 12:53:42,667] jepsen worker 55 - jepsen.util 415 :invoke :write [18 29] INFO [2022-01-10 12:53:42,670] jepsen worker 2 - jepsen.util 92 :ok :write [0 35] INFO [2022-01-10 12:53:42,670] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,671] jepsen worker 38 - jepsen.util 38 :ok :write [12 6] INFO [2022-01-10 12:53:42,671] jepsen worker 2 - jepsen.util 92 :ok :read [0 35] INFO [2022-01-10 12:53:42,672] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:42,672] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,672] jepsen worker 38 - jepsen.util 38 :ok :read [12 6] INFO [2022-01-10 12:53:42,673] jepsen worker 2 - jepsen.util 92 :ok :read [0 35] INFO [2022-01-10 12:53:42,673] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:42,673] jepsen worker 2 - jepsen.util 92 :invoke :write [0 25] INFO [2022-01-10 12:53:42,674] jepsen worker 38 - jepsen.util 38 :ok :read [12 6] INFO [2022-01-10 12:53:42,674] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:42,675] jepsen worker 40 - jepsen.util 130 :ok :write [13 49] INFO [2022-01-10 12:53:42,675] jepsen worker 38 - jepsen.util 38 :ok :read [12 6] INFO [2022-01-10 12:53:42,675] jepsen worker 40 - jepsen.util 130 :invoke :write [13 20] INFO [2022-01-10 12:53:42,676] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:42,677] jepsen worker 38 - jepsen.util 38 :ok :read [12 6] INFO [2022-01-10 12:53:42,677] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:42,678] jepsen worker 86 - jepsen.util 176 :ok :write [28 38] INFO [2022-01-10 12:53:42,678] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,678] jepsen worker 38 - jepsen.util 38 :ok :read [12 6] INFO [2022-01-10 12:53:42,679] jepsen worker 86 - jepsen.util 176 :ok :read [28 38] INFO [2022-01-10 12:53:42,679] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:42,679] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,680] jepsen worker 38 - jepsen.util 38 :ok :read [12 6] INFO [2022-01-10 12:53:42,680] jepsen worker 86 - jepsen.util 176 :ok :read [28 38] INFO [2022-01-10 12:53:42,680] jepsen worker 38 - jepsen.util 38 :invoke :write [12 41] INFO [2022-01-10 12:53:42,681] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,682] jepsen worker 86 - jepsen.util 176 :ok :read [28 38] INFO [2022-01-10 12:53:42,682] jepsen worker 86 - jepsen.util 176 :invoke :write [28 20] INFO [2022-01-10 12:53:42,689] jepsen worker 89 - jepsen.util 269 :ok :write [29 36] INFO [2022-01-10 12:53:42,690] jepsen worker 5 - jepsen.util 185 :ok :write [1 3] INFO [2022-01-10 12:53:42,690] jepsen worker 89 - jepsen.util 269 :invoke :write [29 49] INFO [2022-01-10 12:53:42,690] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,692] jepsen worker 5 - jepsen.util 185 :ok :read [1 3] INFO [2022-01-10 12:53:42,692] jepsen worker 5 - jepsen.util 185 :invoke :write [1 7] INFO [2022-01-10 12:53:42,692] jepsen worker 2 - jepsen.util 92 :ok :write [0 25] INFO [2022-01-10 12:53:42,693] jepsen worker 2 - jepsen.util 92 :invoke :write [0 14] INFO [2022-01-10 12:53:42,696] jepsen worker 40 - jepsen.util 130 :ok :write [13 20] INFO [2022-01-10 12:53:42,696] jepsen worker 40 - jepsen.util 130 :invoke :write [13 10] INFO [2022-01-10 12:53:42,702] jepsen worker 89 - jepsen.util 269 :ok :write [29 49] INFO [2022-01-10 12:53:42,702] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,703] jepsen worker 38 - jepsen.util 38 :ok :write [12 41] INFO [2022-01-10 12:53:42,703] jepsen worker 89 - jepsen.util 269 :ok :read [29 49] INFO [2022-01-10 12:53:42,703] jepsen worker 38 - jepsen.util 38 :invoke :write [12 0] INFO [2022-01-10 12:53:42,704] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,704] jepsen worker 89 - jepsen.util 269 :ok :read [29 49] INFO [2022-01-10 12:53:42,705] jepsen worker 89 - jepsen.util 269 :invoke :write [29 34] INFO [2022-01-10 12:53:42,708] jepsen worker 2 - jepsen.util 92 :ok :write [0 14] INFO [2022-01-10 12:53:42,708] jepsen worker 86 - jepsen.util 176 :ok :write [28 20] INFO [2022-01-10 12:53:42,709] jepsen worker 2 - jepsen.util 92 :invoke :write [0 18] INFO [2022-01-10 12:53:42,709] jepsen worker 86 - jepsen.util 176 :invoke :write [28 4] INFO [2022-01-10 12:53:42,713] jepsen worker 40 - jepsen.util 130 :ok :write [13 10] INFO [2022-01-10 12:53:42,713] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:42,714] jepsen worker 40 - jepsen.util 130 :ok :read [13 10] INFO [2022-01-10 12:53:42,714] jepsen worker 5 - jepsen.util 185 :ok :write [1 7] INFO [2022-01-10 12:53:42,714] jepsen worker 40 - jepsen.util 130 :invoke :write [13 31] INFO [2022-01-10 12:53:42,715] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,717] jepsen worker 89 - jepsen.util 269 :ok :write [29 34] INFO [2022-01-10 12:53:42,718] jepsen worker 89 - jepsen.util 269 :invoke :write [29 13] INFO [2022-01-10 12:53:42,718] jepsen worker 5 - jepsen.util 185 :ok :read [1 7] INFO [2022-01-10 12:53:42,719] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,720] jepsen worker 5 - jepsen.util 185 :ok :read [1 7] INFO [2022-01-10 12:53:42,720] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,721] jepsen worker 5 - jepsen.util 185 :ok :read [1 7] INFO [2022-01-10 12:53:42,722] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,722] jepsen worker 38 - jepsen.util 38 :ok :write [12 0] INFO [2022-01-10 12:53:42,722] jepsen worker 5 - jepsen.util 185 :ok :read [1 7] INFO [2022-01-10 12:53:42,723] jepsen worker 38 - jepsen.util 38 :invoke :write [12 0] INFO [2022-01-10 12:53:42,723] jepsen worker 5 - jepsen.util 185 :invoke :write [1 46] INFO [2022-01-10 12:53:42,724] jepsen worker 2 - jepsen.util 92 :ok :write [0 18] INFO [2022-01-10 12:53:42,724] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,725] jepsen worker 2 - jepsen.util 92 :ok :read [0 18] INFO [2022-01-10 12:53:42,726] jepsen worker 2 - jepsen.util 92 :invoke :write [0 33] INFO [2022-01-10 12:53:42,731] jepsen worker 86 - jepsen.util 176 :ok :write [28 4] INFO [2022-01-10 12:53:42,732] jepsen worker 86 - jepsen.util 176 :invoke :write [28 49] INFO [2022-01-10 12:53:42,732] jepsen worker 40 - jepsen.util 130 :ok :write [13 31] INFO [2022-01-10 12:53:42,732] jepsen worker 21 - jepsen.util 201 :ok :write [7 27] INFO [2022-01-10 12:53:42,733] jepsen worker 40 - jepsen.util 130 :invoke :write [13 37] INFO [2022-01-10 12:53:42,734] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,735] jepsen worker 21 - jepsen.util 201 :ok :read [7 27] INFO [2022-01-10 12:53:42,735] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,736] jepsen worker 89 - jepsen.util 269 :ok :write [29 13] INFO [2022-01-10 12:53:42,736] jepsen worker 21 - jepsen.util 201 :ok :read [7 27] INFO [2022-01-10 12:53:42,737] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,737] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,737] jepsen worker 89 - jepsen.util 269 :ok :read [29 13] INFO [2022-01-10 12:53:42,738] jepsen worker 21 - jepsen.util 201 :ok :read [7 27] INFO [2022-01-10 12:53:42,738] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,738] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,739] jepsen worker 89 - jepsen.util 269 :ok :read [29 13] INFO [2022-01-10 12:53:42,739] jepsen worker 48 - jepsen.util 228 :ok :write [16 19] INFO [2022-01-10 12:53:42,740] jepsen worker 21 - jepsen.util 201 :ok :read [7 27] INFO [2022-01-10 12:53:42,740] jepsen worker 69 - jepsen.util 159 :ok :write [23 46] INFO [2022-01-10 12:53:42,740] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,741] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,741] jepsen worker 89 - jepsen.util 269 :ok :read [29 13] INFO [2022-01-10 12:53:42,741] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,742] jepsen worker 69 - jepsen.util 159 :ok :read [23 46] INFO [2022-01-10 12:53:42,742] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:42,742] jepsen worker 5 - jepsen.util 185 :ok :write [1 46] INFO [2022-01-10 12:53:42,742] jepsen worker 21 - jepsen.util 201 :ok :read [7 27] INFO [2022-01-10 12:53:42,742] jepsen worker 38 - jepsen.util 38 :ok :write [12 0] INFO [2022-01-10 12:53:42,743] jepsen worker 48 - jepsen.util 228 :ok :read [16 19] INFO [2022-01-10 12:53:42,743] jepsen worker 5 - jepsen.util 185 :invoke :write [1 5] INFO [2022-01-10 12:53:42,744] jepsen worker 21 - jepsen.util 201 :invoke :write [7 35] INFO [2022-01-10 12:53:42,745] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:42,746] jepsen worker 48 - jepsen.util 228 :ok :read [16 19] INFO [2022-01-10 12:53:42,746] jepsen worker 2 - jepsen.util 92 :ok :write [0 33] INFO [2022-01-10 12:53:42,747] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,747] jepsen worker 68 - jepsen.util 68 :ok :write [22 46] INFO [2022-01-10 12:53:42,747] jepsen worker 89 - jepsen.util 269 :ok :read [29 13] INFO [2022-01-10 12:53:42,747] jepsen worker 48 - jepsen.util 228 :invoke :write [16 19] INFO [2022-01-10 12:53:42,748] jepsen worker 14 - jepsen.util 284 :ok :write [4 16] INFO [2022-01-10 12:53:42,749] jepsen worker 14 - jepsen.util 284 :invoke :write [4 12] INFO [2022-01-10 12:53:42,749] jepsen worker 40 - jepsen.util 130 :ok :write [13 37] INFO [2022-01-10 12:53:42,750] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,751] jepsen worker 35 - jepsen.util 215 :ok :write [11 13] INFO [2022-01-10 12:53:42,751] jepsen worker 89 - jepsen.util 269 :invoke :write [29 23] INFO [2022-01-10 12:53:42,752] jepsen worker 2 - jepsen.util 92 :ok :read [0 33] INFO [2022-01-10 12:53:42,752] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,753] jepsen worker 80 - jepsen.util 260 :ok :write [26 48] INFO [2022-01-10 12:53:42,753] jepsen worker 68 - jepsen.util 68 :ok :read [22 46] INFO [2022-01-10 12:53:42,753] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,754] jepsen worker 80 - jepsen.util 260 :invoke :write [26 38] INFO [2022-01-10 12:53:42,754] jepsen worker 86 - jepsen.util 176 :ok :write [28 49] INFO [2022-01-10 12:53:42,755] jepsen worker 69 - jepsen.util 159 :ok :read [23 46] INFO [2022-01-10 12:53:42,755] jepsen worker 38 - jepsen.util 38 :invoke :write [12 24] INFO [2022-01-10 12:53:42,756] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,757] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,757] jepsen worker 68 - jepsen.util 68 :ok :read [22 46] INFO [2022-01-10 12:53:42,757] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,758] jepsen worker 69 - jepsen.util 159 :ok :read [23 46] INFO [2022-01-10 12:53:42,758] jepsen worker 40 - jepsen.util 130 :invoke :write [13 14] INFO [2022-01-10 12:53:42,758] jepsen worker 7 - jepsen.util 7 :ok :write [2 1] INFO [2022-01-10 12:53:42,759] jepsen worker 35 - jepsen.util 215 :ok :read [11 13] INFO [2022-01-10 12:53:42,759] jepsen worker 86 - jepsen.util 176 :invoke :write [28 37] INFO [2022-01-10 12:53:42,760] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,761] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,761] jepsen worker 2 - jepsen.util 92 :invoke :write [0 10] INFO [2022-01-10 12:53:42,761] jepsen worker 35 - jepsen.util 215 :ok :read [11 13] INFO [2022-01-10 12:53:42,762] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:42,762] jepsen worker 51 - jepsen.util 141 :ok :write [17 20] INFO [2022-01-10 12:53:42,762] jepsen worker 69 - jepsen.util 159 :ok :read [23 46] INFO [2022-01-10 12:53:42,762] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,763] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:53:42,763] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,764] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:42,764] jepsen worker 68 - jepsen.util 68 :ok :read [22 46] INFO [2022-01-10 12:53:42,764] jepsen worker 35 - jepsen.util 215 :invoke :write [11 10] INFO [2022-01-10 12:53:42,764] jepsen worker 69 - jepsen.util 159 :ok :read [23 46] INFO [2022-01-10 12:53:42,765] jepsen worker 5 - jepsen.util 185 :ok :write [1 5] INFO [2022-01-10 12:53:42,765] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:53:42,765] jepsen worker 51 - jepsen.util 141 :invoke :write [17 25] INFO [2022-01-10 12:53:42,765] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:42,766] jepsen worker 5 - jepsen.util 185 :invoke :write [1 34] INFO [2022-01-10 12:53:42,766] jepsen worker 7 - jepsen.util 7 :ok :read [2 1] INFO [2022-01-10 12:53:42,767] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,767] jepsen worker 69 - jepsen.util 159 :invoke :write [23 36] INFO [2022-01-10 12:53:42,767] jepsen worker 76 - jepsen.util 256 :ok :write [25 0] INFO [2022-01-10 12:53:42,768] jepsen worker 7 - jepsen.util 7 :invoke :write [2 20] INFO [2022-01-10 12:53:42,768] jepsen worker 68 - jepsen.util 68 :ok :read [22 46] INFO [2022-01-10 12:53:42,768] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,769] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,770] jepsen worker 68 - jepsen.util 68 :ok :read [22 46] INFO [2022-01-10 12:53:42,771] jepsen worker 38 - jepsen.util 38 :ok :write [12 24] INFO [2022-01-10 12:53:42,771] jepsen worker 76 - jepsen.util 256 :ok :read [25 0] INFO [2022-01-10 12:53:42,771] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,771] jepsen worker 38 - jepsen.util 38 :invoke :write [12 24] INFO [2022-01-10 12:53:42,772] jepsen worker 68 - jepsen.util 68 :ok :read [22 46] INFO [2022-01-10 12:53:42,772] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,773] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,773] jepsen worker 76 - jepsen.util 256 :ok :read [25 0] INFO [2022-01-10 12:53:42,774] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,774] jepsen worker 68 - jepsen.util 68 :ok :read [22 46] INFO [2022-01-10 12:53:42,774] jepsen worker 76 - jepsen.util 256 :ok :read [25 0] INFO [2022-01-10 12:53:42,775] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,775] jepsen worker 76 - jepsen.util 256 :invoke :write [25 8] INFO [2022-01-10 12:53:42,776] jepsen worker 68 - jepsen.util 68 :ok :read [22 46] INFO [2022-01-10 12:53:42,776] jepsen worker 68 - jepsen.util 68 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,777] jepsen worker 14 - jepsen.util 284 :ok :write [4 12] INFO [2022-01-10 12:53:42,777] jepsen worker 68 - jepsen.util 68 :ok :read [22 46] INFO [2022-01-10 12:53:42,778] jepsen worker 14 - jepsen.util 284 :invoke :write [4 0] INFO [2022-01-10 12:53:42,778] jepsen worker 40 - jepsen.util 130 :ok :write [13 14] INFO [2022-01-10 12:53:42,778] jepsen worker 16 - jepsen.util 286 :ok :write [5 26] INFO [2022-01-10 12:53:42,779] jepsen worker 68 - jepsen.util 68 :invoke :write [22 21] INFO [2022-01-10 12:53:42,779] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,780] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:42,780] jepsen worker 16 - jepsen.util 286 :ok :read [5 26] INFO [2022-01-10 12:53:42,781] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:42,781] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,781] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:42,782] jepsen worker 16 - jepsen.util 286 :ok :read [5 26] INFO [2022-01-10 12:53:42,782] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:42,783] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,783] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:42,783] jepsen worker 16 - jepsen.util 286 :ok :read [5 26] INFO [2022-01-10 12:53:42,784] jepsen worker 40 - jepsen.util 130 :ok :read [13 14] INFO [2022-01-10 12:53:42,784] jepsen worker 16 - jepsen.util 286 :invoke :write [5 31] INFO [2022-01-10 12:53:42,785] jepsen worker 40 - jepsen.util 130 :invoke :write [13 33] INFO [2022-01-10 12:53:42,785] jepsen worker 80 - jepsen.util 260 :ok :write [26 38] INFO [2022-01-10 12:53:42,786] jepsen worker 89 - jepsen.util 269 :ok :write [29 23] INFO [2022-01-10 12:53:42,786] jepsen worker 60 - jepsen.util 240 :ok :write [20 5] INFO [2022-01-10 12:53:42,786] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,787] jepsen worker 11 - jepsen.util 281 :info :write [3 26] :RequestTimeoutException INFO [2022-01-10 12:53:42,787] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,787] jepsen worker 80 - jepsen.util 260 :ok :read [26 38] INFO [2022-01-10 12:53:42,788] jepsen worker 60 - jepsen.util 240 :ok :read [20 5] INFO [2022-01-10 12:53:42,788] jepsen worker 11 - jepsen.util 371 :invoke :write [3 19] INFO [2022-01-10 12:53:42,789] jepsen worker 60 - jepsen.util 240 :invoke :write [20 23] INFO [2022-01-10 12:53:42,790] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:42,791] jepsen worker 86 - jepsen.util 176 :ok :write [28 37] INFO [2022-01-10 12:53:42,791] jepsen worker 89 - jepsen.util 269 :ok :read [29 23] INFO [2022-01-10 12:53:42,791] jepsen worker 80 - jepsen.util 260 :invoke :write [26 49] INFO [2022-01-10 12:53:42,792] jepsen worker 89 - jepsen.util 269 :invoke :write [29 2] INFO [2022-01-10 12:53:42,792] jepsen worker 86 - jepsen.util 176 :invoke :write [28 36] INFO [2022-01-10 12:53:42,795] jepsen worker 42 - jepsen.util 222 :ok :write [14 30] INFO [2022-01-10 12:53:42,796] jepsen worker 42 - jepsen.util 222 :invoke :write [14 33] INFO [2022-01-10 12:53:42,796] jepsen worker 51 - jepsen.util 141 :ok :write [17 25] INFO [2022-01-10 12:53:42,797] jepsen worker 51 - jepsen.util 141 :invoke :write [17 34] INFO [2022-01-10 12:53:42,802] jepsen worker 83 - jepsen.util 263 :ok :write [27 34] INFO [2022-01-10 12:53:42,802] jepsen worker 5 - jepsen.util 185 :ok :write [1 34] INFO [2022-01-10 12:53:42,803] jepsen worker 83 - jepsen.util 263 :invoke :write [27 22] INFO [2022-01-10 12:53:42,803] jepsen worker 5 - jepsen.util 185 :invoke :write [1 15] INFO [2022-01-10 12:53:42,806] jepsen worker 21 - jepsen.util 201 :ok :write [7 35] INFO [2022-01-10 12:53:42,807] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,808] jepsen worker 21 - jepsen.util 201 :ok :read [7 35] INFO [2022-01-10 12:53:42,808] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,809] jepsen worker 69 - jepsen.util 159 :ok :write [23 36] INFO [2022-01-10 12:53:42,809] jepsen worker 21 - jepsen.util 201 :ok :read [7 35] INFO [2022-01-10 12:53:42,809] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,810] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,810] jepsen worker 69 - jepsen.util 159 :ok :read [23 36] INFO [2022-01-10 12:53:42,811] jepsen worker 69 - jepsen.util 159 :invoke :write [23 3] INFO [2022-01-10 12:53:42,811] jepsen worker 21 - jepsen.util 201 :ok :read [7 35] INFO [2022-01-10 12:53:42,812] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,812] jepsen worker 48 - jepsen.util 228 :ok :write [16 19] INFO [2022-01-10 12:53:42,812] jepsen worker 21 - jepsen.util 201 :ok :read [7 35] INFO [2022-01-10 12:53:42,813] jepsen worker 48 - jepsen.util 228 :invoke :read [16 nil] INFO [2022-01-10 12:53:42,813] jepsen worker 21 - jepsen.util 201 :invoke :write [7 34] INFO [2022-01-10 12:53:42,813] jepsen worker 48 - jepsen.util 228 :ok :read [16 19] INFO [2022-01-10 12:53:42,814] jepsen worker 48 - jepsen.util 228 :invoke :write [16 35] INFO [2022-01-10 12:53:42,816] jepsen worker 38 - jepsen.util 38 :ok :write [12 24] INFO [2022-01-10 12:53:42,817] jepsen worker 38 - jepsen.util 38 :invoke :write [12 0] INFO [2022-01-10 12:53:42,823] jepsen worker 35 - jepsen.util 215 :ok :write [11 10] INFO [2022-01-10 12:53:42,823] jepsen worker 35 - jepsen.util 215 :invoke :read [11 nil] INFO [2022-01-10 12:53:42,824] jepsen worker 35 - jepsen.util 215 :ok :read [11 10] INFO [2022-01-10 12:53:42,825] jepsen worker 35 - jepsen.util 215 :invoke :write [11 26] INFO [2022-01-10 12:53:42,826] jepsen worker 14 - jepsen.util 284 :ok :write [4 0] INFO [2022-01-10 12:53:42,827] jepsen worker 73 - jepsen.util 433 :fail :read [24 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:42,826] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:42,828] jepsen worker 73 - jepsen.util 433 :invoke :write [24 9] INFO [2022-01-10 12:53:42,829] jepsen worker 14 - jepsen.util 284 :ok :read [4 0] INFO [2022-01-10 12:53:42,829] jepsen worker 14 - jepsen.util 284 :invoke :write [4 34] INFO [2022-01-10 12:53:42,834] jepsen worker 80 - jepsen.util 260 :ok :write [26 49] INFO [2022-01-10 12:53:42,835] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,835] jepsen worker 76 - jepsen.util 256 :ok :write [25 8] INFO [2022-01-10 12:53:42,835] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,836] jepsen worker 80 - jepsen.util 260 :ok :read [26 49] INFO [2022-01-10 12:53:42,836] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,836] jepsen worker 76 - jepsen.util 256 :ok :read [25 8] INFO [2022-01-10 12:53:42,837] jepsen worker 76 - jepsen.util 256 :invoke :write [25 43] INFO [2022-01-10 12:53:42,837] jepsen worker 80 - jepsen.util 260 :ok :read [26 49] INFO [2022-01-10 12:53:42,837] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:42,838] jepsen worker 80 - jepsen.util 260 :ok :read [26 49] INFO [2022-01-10 12:53:42,839] jepsen worker 80 - jepsen.util 260 :invoke :write [26 2] INFO [2022-01-10 12:53:42,845] jepsen worker 60 - jepsen.util 240 :ok :write [20 23] INFO [2022-01-10 12:53:42,846] jepsen worker 60 - jepsen.util 240 :invoke :write [20 31] INFO [2022-01-10 12:53:42,851] jepsen worker 42 - jepsen.util 222 :ok :write [14 33] INFO [2022-01-10 12:53:42,852] jepsen worker 42 - jepsen.util 222 :invoke :write [14 38] INFO [2022-01-10 12:53:42,912] jepsen worker 31 - jepsen.util 301 :info :write [10 43] :RequestTimeoutException INFO [2022-01-10 12:53:42,913] jepsen worker 31 - jepsen.util 391 :invoke :write [10 24] INFO [2022-01-10 12:53:42,930] jepsen worker 21 - jepsen.util 201 :ok :write [7 34] INFO [2022-01-10 12:53:42,930] jepsen worker 7 - jepsen.util 7 :ok :write [2 20] INFO [2022-01-10 12:53:42,931] jepsen worker 7 - jepsen.util 7 :invoke :write [2 3] INFO [2022-01-10 12:53:42,931] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,932] jepsen worker 21 - jepsen.util 201 :ok :read [7 34] INFO [2022-01-10 12:53:42,933] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,934] jepsen worker 21 - jepsen.util 201 :ok :read [7 34] INFO [2022-01-10 12:53:42,934] jepsen worker 2 - jepsen.util 92 :ok :write [0 10] INFO [2022-01-10 12:53:42,935] jepsen worker 21 - jepsen.util 201 :invoke :write [7 33] INFO [2022-01-10 12:53:42,935] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,936] jepsen worker 76 - jepsen.util 256 :ok :write [25 43] INFO [2022-01-10 12:53:42,936] jepsen worker 2 - jepsen.util 92 :ok :read [0 10] INFO [2022-01-10 12:53:42,937] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:42,938] jepsen worker 86 - jepsen.util 176 :ok :write [28 36] INFO [2022-01-10 12:53:42,938] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,938] jepsen worker 2 - jepsen.util 92 :ok :read [0 10] INFO [2022-01-10 12:53:42,938] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,939] jepsen worker 76 - jepsen.util 256 :ok :read [25 43] INFO [2022-01-10 12:53:42,939] jepsen worker 2 - jepsen.util 92 :invoke :write [0 25] INFO [2022-01-10 12:53:42,939] jepsen worker 86 - jepsen.util 176 :ok :read [28 36] INFO [2022-01-10 12:53:42,939] jepsen worker 51 - jepsen.util 141 :ok :write [17 34] INFO [2022-01-10 12:53:42,940] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,940] jepsen worker 86 - jepsen.util 176 :invoke :write [28 20] INFO [2022-01-10 12:53:42,941] jepsen worker 76 - jepsen.util 256 :ok :read [25 43] INFO [2022-01-10 12:53:42,941] jepsen worker 67 - jepsen.util 247 :ok :write [22 32] INFO [2022-01-10 12:53:42,941] jepsen worker 51 - jepsen.util 141 :invoke :write [17 47] INFO [2022-01-10 12:53:42,942] jepsen worker 60 - jepsen.util 240 :ok :write [20 31] INFO [2022-01-10 12:53:42,942] jepsen worker 67 - jepsen.util 247 :invoke :write [22 25] INFO [2022-01-10 12:53:42,943] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,943] jepsen worker 76 - jepsen.util 256 :invoke :write [25 19] INFO [2022-01-10 12:53:42,944] jepsen worker 60 - jepsen.util 240 :ok :read [20 31] INFO [2022-01-10 12:53:42,945] jepsen worker 60 - jepsen.util 240 :invoke :write [20 14] INFO [2022-01-10 12:53:42,945] jepsen worker 40 - jepsen.util 130 :ok :write [13 33] INFO [2022-01-10 12:53:42,945] jepsen worker 5 - jepsen.util 185 :ok :write [1 15] INFO [2022-01-10 12:53:42,946] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,946] jepsen worker 40 - jepsen.util 130 :invoke :write [13 17] INFO [2022-01-10 12:53:42,947] jepsen worker 5 - jepsen.util 185 :ok :read [1 15] INFO [2022-01-10 12:53:42,948] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,948] jepsen worker 42 - jepsen.util 222 :ok :write [14 38] INFO [2022-01-10 12:53:42,949] jepsen worker 5 - jepsen.util 185 :ok :read [1 15] INFO [2022-01-10 12:53:42,949] jepsen worker 42 - jepsen.util 222 :invoke :read [14 nil] INFO [2022-01-10 12:53:42,950] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,950] jepsen worker 42 - jepsen.util 222 :ok :read [14 38] INFO [2022-01-10 12:53:42,951] jepsen worker 5 - jepsen.util 185 :ok :read [1 15] INFO [2022-01-10 12:53:42,951] jepsen worker 69 - jepsen.util 159 :ok :write [23 3] INFO [2022-01-10 12:53:42,951] jepsen worker 16 - jepsen.util 286 :ok :write [5 31] INFO [2022-01-10 12:53:42,951] jepsen worker 42 - jepsen.util 222 :invoke :write [14 14] INFO [2022-01-10 12:53:42,952] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,953] jepsen worker 16 - jepsen.util 286 :invoke :read [5 nil] INFO [2022-01-10 12:53:42,953] jepsen worker 69 - jepsen.util 159 :ok :read [23 3] INFO [2022-01-10 12:53:42,953] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:42,954] jepsen worker 16 - jepsen.util 286 :ok :read [5 31] INFO [2022-01-10 12:53:42,954] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,955] jepsen worker 5 - jepsen.util 185 :ok :read [1 15] INFO [2022-01-10 12:53:42,955] jepsen worker 69 - jepsen.util 159 :ok :read [23 3] INFO [2022-01-10 12:53:42,955] jepsen worker 16 - jepsen.util 286 :invoke :write [5 49] INFO [2022-01-10 12:53:42,956] jepsen worker 5 - jepsen.util 185 :invoke :write [1 14] INFO [2022-01-10 12:53:42,956] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,956] jepsen worker 38 - jepsen.util 38 :ok :write [12 0] INFO [2022-01-10 12:53:42,957] jepsen worker 69 - jepsen.util 159 :ok :read [23 3] INFO [2022-01-10 12:53:42,957] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:42,957] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,958] jepsen worker 38 - jepsen.util 38 :ok :read [12 0] INFO [2022-01-10 12:53:42,959] jepsen worker 38 - jepsen.util 38 :invoke :write [12 32] INFO [2022-01-10 12:53:42,959] jepsen worker 89 - jepsen.util 269 :ok :write [29 2] INFO [2022-01-10 12:53:42,959] jepsen worker 69 - jepsen.util 159 :ok :read [23 3] INFO [2022-01-10 12:53:42,960] jepsen worker 89 - jepsen.util 269 :invoke :write [29 44] INFO [2022-01-10 12:53:42,961] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,961] jepsen worker 83 - jepsen.util 263 :ok :write [27 22] INFO [2022-01-10 12:53:42,962] jepsen worker 69 - jepsen.util 159 :ok :read [23 3] INFO [2022-01-10 12:53:42,962] jepsen worker 83 - jepsen.util 263 :invoke :write [27 46] INFO [2022-01-10 12:53:42,962] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:42,963] jepsen worker 14 - jepsen.util 284 :ok :write [4 34] INFO [2022-01-10 12:53:42,963] jepsen worker 69 - jepsen.util 159 :ok :read [23 3] INFO [2022-01-10 12:53:42,964] jepsen worker 14 - jepsen.util 284 :invoke :write [4 33] INFO [2022-01-10 12:53:42,964] jepsen worker 69 - jepsen.util 159 :invoke :write [23 30] INFO [2022-01-10 12:53:42,967] jepsen worker 21 - jepsen.util 201 :ok :write [7 33] INFO [2022-01-10 12:53:42,968] jepsen worker 21 - jepsen.util 201 :invoke :read [7 nil] INFO [2022-01-10 12:53:42,968] jepsen worker 21 - jepsen.util 201 :ok :read [7 33] INFO [2022-01-10 12:53:42,969] jepsen worker 80 - jepsen.util 260 :ok :write [26 2] INFO [2022-01-10 12:53:42,969] jepsen worker 21 - jepsen.util 201 :invoke :write [7 10] INFO [2022-01-10 12:53:42,970] jepsen worker 48 - jepsen.util 228 :ok :write [16 35] INFO [2022-01-10 12:53:42,970] jepsen worker 80 - jepsen.util 260 :invoke :write [26 48] INFO [2022-01-10 12:53:42,971] jepsen worker 48 - jepsen.util 228 :invoke :write [16 13] INFO [2022-01-10 12:53:42,975] jepsen worker 35 - jepsen.util 215 :ok :write [11 26] INFO [2022-01-10 12:53:42,976] jepsen worker 35 - jepsen.util 215 :invoke :write [11 1] INFO [2022-01-10 12:53:42,978] jepsen worker 51 - jepsen.util 141 :ok :write [17 47] INFO [2022-01-10 12:53:42,978] jepsen worker 7 - jepsen.util 7 :ok :write [2 3] INFO [2022-01-10 12:53:42,979] jepsen worker 51 - jepsen.util 141 :invoke :write [17 21] INFO [2022-01-10 12:53:42,980] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:42,981] jepsen worker 7 - jepsen.util 7 :ok :read [2 3] INFO [2022-01-10 12:53:42,981] jepsen worker 60 - jepsen.util 240 :ok :write [20 14] INFO [2022-01-10 12:53:42,981] jepsen worker 2 - jepsen.util 92 :ok :write [0 25] INFO [2022-01-10 12:53:42,981] jepsen worker 7 - jepsen.util 7 :invoke :read [2 nil] INFO [2022-01-10 12:53:42,982] jepsen worker 60 - jepsen.util 240 :invoke :read [20 nil] INFO [2022-01-10 12:53:42,982] jepsen worker 7 - jepsen.util 7 :ok :read [2 3] INFO [2022-01-10 12:53:42,982] jepsen worker 2 - jepsen.util 92 :invoke :write [0 20] INFO [2022-01-10 12:53:42,983] jepsen worker 7 - jepsen.util 7 :invoke :write [2 40] INFO [2022-01-10 12:53:42,983] jepsen worker 60 - jepsen.util 240 :ok :read [20 14] INFO [2022-01-10 12:53:42,984] jepsen worker 60 - jepsen.util 240 :invoke :write [20 37] INFO [2022-01-10 12:53:42,986] jepsen worker 67 - jepsen.util 247 :ok :write [22 25] INFO [2022-01-10 12:53:42,986] jepsen worker 67 - jepsen.util 247 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,987] jepsen worker 67 - jepsen.util 247 :ok :read [22 25] INFO [2022-01-10 12:53:42,988] jepsen worker 67 - jepsen.util 247 :invoke :read [22 nil] INFO [2022-01-10 12:53:42,989] jepsen worker 67 - jepsen.util 247 :ok :read [22 25] INFO [2022-01-10 12:53:42,989] jepsen worker 67 - jepsen.util 247 :invoke :write [22 48] INFO [2022-01-10 12:53:42,990] jepsen worker 86 - jepsen.util 176 :ok :write [28 20] INFO [2022-01-10 12:53:42,991] jepsen worker 76 - jepsen.util 256 :ok :write [25 19] INFO [2022-01-10 12:53:42,991] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:42,991] jepsen worker 76 - jepsen.util 256 :invoke :read [25 nil] INFO [2022-01-10 12:53:42,992] jepsen worker 86 - jepsen.util 176 :ok :read [28 20] INFO [2022-01-10 12:53:42,992] jepsen worker 86 - jepsen.util 176 :invoke :write [28 21] INFO [2022-01-10 12:53:42,992] jepsen worker 76 - jepsen.util 256 :ok :read [25 19] INFO [2022-01-10 12:53:42,993] jepsen worker 76 - jepsen.util 256 :invoke :write [25 25] INFO [2022-01-10 12:53:42,994] jepsen worker 5 - jepsen.util 185 :ok :write [1 14] INFO [2022-01-10 12:53:42,994] jepsen worker 5 - jepsen.util 185 :invoke :write [1 46] INFO [2022-01-10 12:53:42,994] jepsen worker 40 - jepsen.util 130 :ok :write [13 17] INFO [2022-01-10 12:53:42,995] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:42,996] jepsen worker 40 - jepsen.util 130 :ok :read [13 17] INFO [2022-01-10 12:53:42,997] jepsen worker 40 - jepsen.util 130 :invoke :write [13 13] INFO [2022-01-10 12:53:42,999] jepsen worker 43 - jepsen.util 133 :ok :write [14 8] INFO [2022-01-10 12:53:43,000] jepsen worker 43 - jepsen.util 133 :invoke :read [14 nil] INFO [2022-01-10 12:53:43,001] jepsen worker 43 - jepsen.util 133 :ok :read [14 8] INFO [2022-01-10 12:53:43,001] jepsen worker 43 - jepsen.util 133 :invoke :read [14 nil] INFO [2022-01-10 12:53:43,002] jepsen worker 43 - jepsen.util 133 :ok :read [14 8] INFO [2022-01-10 12:53:43,003] jepsen worker 43 - jepsen.util 133 :invoke :read [14 nil] INFO [2022-01-10 12:53:43,004] jepsen worker 43 - jepsen.util 133 :ok :read [14 8] INFO [2022-01-10 12:53:43,004] jepsen worker 69 - jepsen.util 159 :ok :write [23 30] INFO [2022-01-10 12:53:43,005] jepsen worker 43 - jepsen.util 133 :invoke :read [14 nil] INFO [2022-01-10 12:53:43,005] jepsen worker 89 - jepsen.util 269 :ok :write [29 44] INFO [2022-01-10 12:53:43,006] jepsen worker 43 - jepsen.util 133 :ok :read [14 8] INFO [2022-01-10 12:53:43,006] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:43,007] jepsen worker 69 - jepsen.util 159 :ok :read [23 30] INFO [2022-01-10 12:53:43,007] jepsen worker 43 - jepsen.util 133 :invoke :write [14 7] INFO [2022-01-10 12:53:43,008] jepsen worker 69 - jepsen.util 159 :invoke :read [23 nil] INFO [2022-01-10 12:53:43,009] jepsen worker 38 - jepsen.util 38 :ok :write [12 32] INFO [2022-01-10 12:53:43,009] jepsen worker 69 - jepsen.util 159 :ok :read [23 30] INFO [2022-01-10 12:53:43,009] jepsen worker 89 - jepsen.util 269 :invoke :write [29 28] INFO [2022-01-10 12:53:43,010] jepsen worker 69 - jepsen.util 159 :invoke :write [23 28] INFO [2022-01-10 12:53:43,011] jepsen worker 38 - jepsen.util 38 :invoke :write [12 8] INFO [2022-01-10 12:53:43,016] jepsen worker 28 - jepsen.util 298 :info :write [9 37] :RequestTimeoutException INFO [2022-01-10 12:53:43,017] jepsen worker 28 - jepsen.util 388 :invoke :read [9 nil] INFO [2022-01-10 12:53:43,025] jepsen worker 46 - jepsen.util 406 :fail :read [15 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:43,025] jepsen worker 46 - jepsen.util 406 :invoke :write [15 13] INFO [2022-01-10 12:53:43,028] jepsen worker 14 - jepsen.util 284 :ok :write [4 33] INFO [2022-01-10 12:53:43,028] jepsen worker 14 - jepsen.util 284 :invoke :read [4 nil] INFO [2022-01-10 12:53:43,029] jepsen worker 14 - jepsen.util 284 :ok :read [4 33] INFO [2022-01-10 12:53:43,029] jepsen worker 14 - jepsen.util 284 :invoke :write [4 46] INFO [2022-01-10 12:53:43,031] jepsen worker 2 - jepsen.util 92 :ok :write [0 20] INFO [2022-01-10 12:53:43,032] jepsen worker 2 - jepsen.util 92 :invoke :write [0 38] INFO [2022-01-10 12:53:43,033] jepsen worker 80 - jepsen.util 260 :ok :write [26 48] INFO [2022-01-10 12:53:43,033] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:43,034] jepsen worker 80 - jepsen.util 260 :ok :read [26 48] INFO [2022-01-10 12:53:43,035] jepsen worker 80 - jepsen.util 260 :invoke :read [26 nil] INFO [2022-01-10 12:53:43,036] jepsen worker 80 - jepsen.util 260 :ok :read [26 48] INFO [2022-01-10 12:53:43,037] jepsen worker 80 - jepsen.util 260 :invoke :write [26 42] INFO [2022-01-10 12:53:43,044] jepsen worker 40 - jepsen.util 130 :ok :write [13 13] INFO [2022-01-10 12:53:43,045] jepsen worker 40 - jepsen.util 130 :invoke :write [13 6] INFO [2022-01-10 12:53:43,048] jepsen worker 5 - jepsen.util 185 :ok :write [1 46] INFO [2022-01-10 12:53:43,048] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:43,049] jepsen worker 5 - jepsen.util 185 :ok :read [1 46] INFO [2022-01-10 12:53:43,050] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:43,050] jepsen worker 5 - jepsen.util 185 :ok :read [1 46] INFO [2022-01-10 12:53:43,051] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:43,052] jepsen worker 5 - jepsen.util 185 :ok :read [1 46] INFO [2022-01-10 12:53:43,052] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:43,053] jepsen worker 5 - jepsen.util 185 :ok :read [1 46] INFO [2022-01-10 12:53:43,053] jepsen worker 86 - jepsen.util 176 :ok :write [28 21] INFO [2022-01-10 12:53:43,053] jepsen worker 5 - jepsen.util 185 :invoke :write [1 28] INFO [2022-01-10 12:53:43,054] jepsen worker 86 - jepsen.util 176 :invoke :write [28 43] INFO [2022-01-10 12:53:43,061] jepsen worker 89 - jepsen.util 269 :ok :write [29 28] INFO [2022-01-10 12:53:43,062] jepsen worker 89 - jepsen.util 269 :invoke :write [29 4] INFO [2022-01-10 12:53:43,069] jepsen worker 2 - jepsen.util 92 :ok :write [0 38] INFO [2022-01-10 12:53:43,070] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:43,071] jepsen worker 2 - jepsen.util 92 :ok :read [0 38] INFO [2022-01-10 12:53:43,071] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:43,073] jepsen worker 2 - jepsen.util 92 :ok :read [0 38] INFO [2022-01-10 12:53:43,073] jepsen worker 2 - jepsen.util 92 :invoke :write [0 16] INFO [2022-01-10 12:53:43,076] jepsen worker 40 - jepsen.util 130 :ok :write [13 6] INFO [2022-01-10 12:53:43,077] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:43,078] jepsen worker 40 - jepsen.util 130 :ok :read [13 6] INFO [2022-01-10 12:53:43,079] jepsen worker 40 - jepsen.util 130 :invoke :write [13 15] INFO [2022-01-10 12:53:43,081] jepsen worker 38 - jepsen.util 38 :ok :write [12 8] INFO [2022-01-10 12:53:43,082] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:43,082] jepsen worker 89 - jepsen.util 269 :ok :write [29 4] INFO [2022-01-10 12:53:43,083] jepsen worker 38 - jepsen.util 38 :ok :read [12 8] INFO [2022-01-10 12:53:43,083] jepsen worker 89 - jepsen.util 269 :invoke :write [29 4] INFO [2022-01-10 12:53:43,084] jepsen worker 38 - jepsen.util 38 :invoke :write [12 21] INFO [2022-01-10 12:53:43,085] jepsen worker 2 - jepsen.util 92 :ok :write [0 16] INFO [2022-01-10 12:53:43,086] jepsen worker 2 - jepsen.util 92 :invoke :write [0 12] INFO [2022-01-10 12:53:43,092] jepsen worker 40 - jepsen.util 130 :ok :write [13 15] INFO [2022-01-10 12:53:43,093] jepsen worker 40 - jepsen.util 130 :invoke :write [13 5] INFO [2022-01-10 12:53:43,099] jepsen worker 89 - jepsen.util 269 :ok :write [29 4] INFO [2022-01-10 12:53:43,100] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:43,100] jepsen worker 89 - jepsen.util 269 :ok :read [29 4] INFO [2022-01-10 12:53:43,101] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:43,101] jepsen worker 89 - jepsen.util 269 :ok :read [29 4] INFO [2022-01-10 12:53:43,102] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:43,102] jepsen worker 2 - jepsen.util 92 :ok :write [0 12] INFO [2022-01-10 12:53:43,103] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:43,103] jepsen worker 89 - jepsen.util 269 :ok :read [29 4] INFO [2022-01-10 12:53:43,104] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:43,104] jepsen worker 2 - jepsen.util 92 :ok :read [0 12] INFO [2022-01-10 12:53:43,104] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:43,104] jepsen worker 89 - jepsen.util 269 :ok :read [29 4] INFO [2022-01-10 12:53:43,105] jepsen worker 2 - jepsen.util 92 :ok :read [0 12] INFO [2022-01-10 12:53:43,105] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:43,106] jepsen worker 89 - jepsen.util 269 :ok :read [29 4] INFO [2022-01-10 12:53:43,106] jepsen worker 2 - jepsen.util 92 :invoke :write [0 7] INFO [2022-01-10 12:53:43,106] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:43,107] jepsen worker 40 - jepsen.util 130 :ok :write [13 5] INFO [2022-01-10 12:53:43,107] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:43,107] jepsen worker 89 - jepsen.util 269 :ok :read [29 4] INFO [2022-01-10 12:53:43,108] jepsen worker 89 - jepsen.util 269 :invoke :write [29 41] INFO [2022-01-10 12:53:43,108] jepsen worker 40 - jepsen.util 130 :ok :read [13 5] INFO [2022-01-10 12:53:43,109] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:43,110] jepsen worker 40 - jepsen.util 130 :ok :read [13 5] INFO [2022-01-10 12:53:43,110] jepsen worker 40 - jepsen.util 130 :invoke :write [13 36] INFO [2022-01-10 12:53:43,110] jepsen worker 5 - jepsen.util 185 :ok :write [1 28] INFO [2022-01-10 12:53:43,111] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:43,112] jepsen worker 5 - jepsen.util 185 :ok :read [1 28] INFO [2022-01-10 12:53:43,112] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:43,113] jepsen worker 5 - jepsen.util 185 :ok :read [1 28] INFO [2022-01-10 12:53:43,113] jepsen worker 5 - jepsen.util 185 :invoke :write [1 44] INFO [2022-01-10 12:53:43,117] jepsen worker 2 - jepsen.util 92 :ok :write [0 7] INFO [2022-01-10 12:53:43,118] jepsen worker 2 - jepsen.util 92 :invoke :write [0 34] INFO [2022-01-10 12:53:43,119] jepsen worker 86 - jepsen.util 176 :ok :write [28 43] INFO [2022-01-10 12:53:43,119] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,120] jepsen worker 86 - jepsen.util 176 :ok :read [28 43] INFO [2022-01-10 12:53:43,120] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,121] jepsen worker 86 - jepsen.util 176 :ok :read [28 43] INFO [2022-01-10 12:53:43,121] jepsen worker 86 - jepsen.util 176 :invoke :write [28 43] INFO [2022-01-10 12:53:43,125] jepsen worker 39 - jepsen.util 129 :ok :write [13 8] INFO [2022-01-10 12:53:43,125] jepsen worker 39 - jepsen.util 129 :invoke :read [13 nil] INFO [2022-01-10 12:53:43,126] jepsen worker 39 - jepsen.util 129 :ok :read [13 8] INFO [2022-01-10 12:53:43,126] jepsen worker 39 - jepsen.util 129 :invoke :write [13 0] INFO [2022-01-10 12:53:43,126] jepsen worker 38 - jepsen.util 38 :ok :write [12 21] INFO [2022-01-10 12:53:43,127] jepsen worker 38 - jepsen.util 38 :invoke :write [12 4] INFO [2022-01-10 12:53:43,131] jepsen worker 89 - jepsen.util 269 :ok :write [29 41] INFO [2022-01-10 12:53:43,132] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:43,133] jepsen worker 89 - jepsen.util 269 :ok :read [29 41] INFO [2022-01-10 12:53:43,133] jepsen worker 89 - jepsen.util 269 :invoke :write [29 14] INFO [2022-01-10 12:53:43,134] jepsen worker 5 - jepsen.util 185 :ok :write [1 44] INFO [2022-01-10 12:53:43,135] jepsen worker 5 - jepsen.util 185 :invoke :write [1 41] INFO [2022-01-10 12:53:43,139] jepsen worker 2 - jepsen.util 92 :ok :write [0 34] INFO [2022-01-10 12:53:43,139] jepsen worker 2 - jepsen.util 92 :invoke :write [0 40] INFO [2022-01-10 12:53:43,144] jepsen worker 86 - jepsen.util 176 :ok :write [28 43] INFO [2022-01-10 12:53:43,144] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,145] jepsen worker 86 - jepsen.util 176 :ok :read [28 43] INFO [2022-01-10 12:53:43,145] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,147] jepsen worker 86 - jepsen.util 176 :ok :read [28 43] INFO [2022-01-10 12:53:43,147] jepsen worker 39 - jepsen.util 129 :ok :write [13 0] INFO [2022-01-10 12:53:43,147] jepsen worker 86 - jepsen.util 176 :invoke :write [28 39] INFO [2022-01-10 12:53:43,148] jepsen worker 39 - jepsen.util 129 :invoke :write [13 39] INFO [2022-01-10 12:53:43,152] jepsen worker 38 - jepsen.util 38 :ok :write [12 4] INFO [2022-01-10 12:53:43,152] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:43,153] jepsen worker 38 - jepsen.util 38 :ok :read [12 4] INFO [2022-01-10 12:53:43,154] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:43,156] jepsen worker 89 - jepsen.util 269 :ok :write [29 14] INFO [2022-01-10 12:53:43,156] jepsen worker 5 - jepsen.util 185 :ok :write [1 41] INFO [2022-01-10 12:53:43,157] jepsen worker 5 - jepsen.util 185 :invoke :write [1 4] INFO [2022-01-10 12:53:43,158] jepsen worker 89 - jepsen.util 269 :invoke :write [29 27] INFO [2022-01-10 12:53:43,161] jepsen worker 86 - jepsen.util 176 :ok :write [28 39] INFO [2022-01-10 12:53:43,162] jepsen worker 86 - jepsen.util 176 :invoke :write [28 9] INFO [2022-01-10 12:53:43,162] jepsen worker 2 - jepsen.util 92 :ok :write [0 40] INFO [2022-01-10 12:53:43,163] jepsen worker 2 - jepsen.util 92 :invoke :write [0 18] INFO [2022-01-10 12:53:43,169] jepsen worker 40 - jepsen.util 130 :ok :write [13 36] INFO [2022-01-10 12:53:43,169] jepsen worker 40 - jepsen.util 130 :invoke :write [13 46] INFO [2022-01-10 12:53:43,170] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:43,170] jepsen worker 38 - jepsen.util 38 :invoke :write [12 36] INFO [2022-01-10 12:53:43,174] jepsen worker 89 - jepsen.util 269 :ok :write [29 27] INFO [2022-01-10 12:53:43,174] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:43,175] jepsen worker 89 - jepsen.util 269 :ok :read [29 27] INFO [2022-01-10 12:53:43,176] jepsen worker 89 - jepsen.util 269 :invoke :write [29 2] WARN [2022-01-10 12:53:43,254] jepsen worker nemesis - jepsen.control.retry com.jcraft.jsch.JSchException with conn [:control "172.23.105.187"]; reopening. INFO [2022-01-10 12:53:43,336] jepsen worker 5 - jepsen.util 185 :ok :write [1 4] INFO [2022-01-10 12:53:43,337] jepsen worker 5 - jepsen.util 185 :invoke :write [1 7] INFO [2022-01-10 12:53:43,340] jepsen worker 2 - jepsen.util 92 :ok :write [0 18] INFO [2022-01-10 12:53:43,340] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:43,341] jepsen worker 2 - jepsen.util 92 :ok :read [0 18] INFO [2022-01-10 12:53:43,342] jepsen worker 2 - jepsen.util 92 :invoke :write [0 25] INFO [2022-01-10 12:53:43,342] jepsen worker 86 - jepsen.util 176 :ok :write [28 9] INFO [2022-01-10 12:53:43,343] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,344] jepsen worker 86 - jepsen.util 176 :ok :read [28 9] INFO [2022-01-10 12:53:43,345] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,346] jepsen worker 38 - jepsen.util 38 :ok :write [12 36] INFO [2022-01-10 12:53:43,346] jepsen worker 86 - jepsen.util 176 :ok :read [28 9] INFO [2022-01-10 12:53:43,347] jepsen worker 38 - jepsen.util 38 :invoke :write [12 11] INFO [2022-01-10 12:53:43,347] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,349] jepsen worker 86 - jepsen.util 176 :ok :read [28 9] INFO [2022-01-10 12:53:43,350] jepsen worker 86 - jepsen.util 176 :invoke :write [28 16] INFO [2022-01-10 12:53:43,351] jepsen worker 40 - jepsen.util 130 :ok :write [13 46] INFO [2022-01-10 12:53:43,352] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:43,353] jepsen worker 40 - jepsen.util 130 :ok :read [13 46] INFO [2022-01-10 12:53:43,353] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:43,354] jepsen worker 40 - jepsen.util 130 :ok :read [13 46] INFO [2022-01-10 12:53:43,355] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:43,355] jepsen worker 40 - jepsen.util 130 :ok :read [13 46] INFO [2022-01-10 12:53:43,356] jepsen worker 89 - jepsen.util 269 :ok :write [29 2] INFO [2022-01-10 12:53:43,356] jepsen worker 40 - jepsen.util 130 :invoke :write [13 18] INFO [2022-01-10 12:53:43,357] jepsen worker 89 - jepsen.util 269 :invoke :write [29 5] INFO [2022-01-10 12:53:43,358] jepsen worker 5 - jepsen.util 185 :ok :write [1 7] INFO [2022-01-10 12:53:43,359] jepsen worker 5 - jepsen.util 185 :invoke :write [1 45] INFO [2022-01-10 12:53:43,363] jepsen worker 2 - jepsen.util 92 :ok :write [0 25] INFO [2022-01-10 12:53:43,364] jepsen worker 38 - jepsen.util 38 :ok :write [12 11] INFO [2022-01-10 12:53:43,364] jepsen worker 2 - jepsen.util 92 :invoke :write [0 47] INFO [2022-01-10 12:53:43,365] jepsen worker 38 - jepsen.util 38 :invoke :write [12 42] INFO [2022-01-10 12:53:43,367] jepsen worker 40 - jepsen.util 130 :ok :write [13 18] INFO [2022-01-10 12:53:43,368] jepsen worker 40 - jepsen.util 130 :invoke :write [13 39] INFO [2022-01-10 12:53:43,371] jepsen worker 86 - jepsen.util 176 :ok :write [28 16] INFO [2022-01-10 12:53:43,372] jepsen worker 89 - jepsen.util 269 :ok :write [29 5] INFO [2022-01-10 12:53:43,372] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,373] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:43,373] jepsen worker 86 - jepsen.util 176 :ok :read [28 16] INFO [2022-01-10 12:53:43,373] jepsen worker 89 - jepsen.util 269 :ok :read [29 5] INFO [2022-01-10 12:53:43,374] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,374] jepsen worker 89 - jepsen.util 269 :invoke :write [29 41] INFO [2022-01-10 12:53:43,374] jepsen worker 86 - jepsen.util 176 :ok :read [28 16] INFO [2022-01-10 12:53:43,375] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,376] jepsen worker 86 - jepsen.util 176 :ok :read [28 16] INFO [2022-01-10 12:53:43,377] jepsen worker 86 - jepsen.util 176 :invoke :write [28 35] INFO [2022-01-10 12:53:43,378] jepsen worker 5 - jepsen.util 185 :ok :write [1 45] INFO [2022-01-10 12:53:43,378] jepsen worker 2 - jepsen.util 92 :ok :write [0 47] INFO [2022-01-10 12:53:43,379] jepsen worker 5 - jepsen.util 185 :invoke :write [1 9] INFO [2022-01-10 12:53:43,379] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:43,380] jepsen worker 2 - jepsen.util 92 :ok :read [0 47] INFO [2022-01-10 12:53:43,381] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:43,382] jepsen worker 2 - jepsen.util 92 :ok :read [0 47] INFO [2022-01-10 12:53:43,383] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:43,384] jepsen worker 2 - jepsen.util 92 :ok :read [0 47] INFO [2022-01-10 12:53:43,384] jepsen worker 2 - jepsen.util 92 :invoke :write [0 28] INFO [2022-01-10 12:53:43,385] jepsen worker 40 - jepsen.util 130 :ok :write [13 39] INFO [2022-01-10 12:53:43,386] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:43,386] jepsen worker 38 - jepsen.util 38 :ok :write [12 42] INFO [2022-01-10 12:53:43,387] jepsen worker 40 - jepsen.util 130 :ok :read [13 39] INFO [2022-01-10 12:53:43,387] jepsen worker 38 - jepsen.util 38 :invoke :write [12 27] INFO [2022-01-10 12:53:43,388] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:43,389] jepsen worker 40 - jepsen.util 130 :ok :read [13 39] INFO [2022-01-10 12:53:43,390] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:43,392] jepsen worker 40 - jepsen.util 130 :ok :read [13 39] INFO [2022-01-10 12:53:43,392] jepsen worker 86 - jepsen.util 176 :ok :write [28 35] INFO [2022-01-10 12:53:43,392] jepsen worker 89 - jepsen.util 269 :ok :write [29 41] INFO [2022-01-10 12:53:43,392] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:43,393] jepsen worker 40 - jepsen.util 130 :ok :read [13 39] INFO [2022-01-10 12:53:43,393] jepsen worker 89 - jepsen.util 269 :invoke :write [29 48] INFO [2022-01-10 12:53:43,394] jepsen worker 86 - jepsen.util 176 :invoke :write [28 7] INFO [2022-01-10 12:53:43,394] jepsen worker 40 - jepsen.util 130 :invoke :write [13 22] INFO [2022-01-10 12:53:43,398] jepsen worker 2 - jepsen.util 92 :ok :write [0 28] INFO [2022-01-10 12:53:43,399] jepsen worker 2 - jepsen.util 92 :invoke :write [0 28] INFO [2022-01-10 12:53:43,403] jepsen worker 5 - jepsen.util 185 :ok :write [1 9] INFO [2022-01-10 12:53:43,403] jepsen worker 5 - jepsen.util 185 :invoke :read [1 nil] INFO [2022-01-10 12:53:43,405] jepsen worker 5 - jepsen.util 185 :ok :read [1 9] INFO [2022-01-10 12:53:43,405] jepsen worker 5 - jepsen.util 185 :invoke :write [1 8] INFO [2022-01-10 12:53:43,409] jepsen worker 38 - jepsen.util 38 :ok :write [12 27] INFO [2022-01-10 12:53:43,409] jepsen worker 38 - jepsen.util 38 :invoke :write [12 0] INFO [2022-01-10 12:53:43,409] jepsen worker 89 - jepsen.util 269 :ok :write [29 48] INFO [2022-01-10 12:53:43,410] jepsen worker 89 - jepsen.util 269 :invoke :write [29 24] INFO [2022-01-10 12:53:43,414] jepsen worker 40 - jepsen.util 130 :ok :write [13 22] INFO [2022-01-10 12:53:43,414] jepsen worker 40 - jepsen.util 130 :invoke :write [13 15] INFO [2022-01-10 12:53:43,417] jepsen worker 2 - jepsen.util 92 :ok :write [0 28] INFO [2022-01-10 12:53:43,417] jepsen worker 2 - jepsen.util 92 :invoke :write [0 14] INFO [2022-01-10 12:53:43,419] jepsen worker 86 - jepsen.util 176 :ok :write [28 7] INFO [2022-01-10 12:53:43,419] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,420] jepsen worker 86 - jepsen.util 176 :ok :read [28 7] INFO [2022-01-10 12:53:43,421] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,422] jepsen worker 86 - jepsen.util 176 :ok :read [28 7] INFO [2022-01-10 12:53:43,423] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,426] jepsen worker 5 - jepsen.util 185 :ok :write [1 8] INFO [2022-01-10 12:53:43,427] jepsen worker 5 - jepsen.util 185 :invoke :write [1 15] INFO [2022-01-10 12:53:43,427] jepsen worker 86 - jepsen.util 176 :ok :read [28 7] INFO [2022-01-10 12:53:43,428] jepsen worker 89 - jepsen.util 269 :ok :write [29 24] INFO [2022-01-10 12:53:43,428] jepsen worker 86 - jepsen.util 176 :invoke :write [28 3] INFO [2022-01-10 12:53:43,429] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:43,429] jepsen worker 89 - jepsen.util 269 :ok :read [29 24] INFO [2022-01-10 12:53:43,430] jepsen worker 89 - jepsen.util 269 :invoke :write [29 34] INFO [2022-01-10 12:53:43,431] jepsen worker 40 - jepsen.util 130 :ok :write [13 15] INFO [2022-01-10 12:53:43,432] jepsen worker 40 - jepsen.util 130 :invoke :read [13 nil] INFO [2022-01-10 12:53:43,432] jepsen worker 40 - jepsen.util 130 :ok :read [13 15] INFO [2022-01-10 12:53:43,433] jepsen worker 40 - jepsen.util 130 :invoke :write [13 15] INFO [2022-01-10 12:53:43,436] jepsen worker 38 - jepsen.util 38 :ok :write [12 0] INFO [2022-01-10 12:53:43,437] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:43,437] jepsen worker 2 - jepsen.util 92 :ok :write [0 14] INFO [2022-01-10 12:53:43,437] jepsen worker 38 - jepsen.util 38 :ok :read [12 0] INFO [2022-01-10 12:53:43,437] jepsen worker 2 - jepsen.util 92 :invoke :write [0 20] INFO [2022-01-10 12:53:43,438] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:43,439] jepsen worker 38 - jepsen.util 38 :ok :read [12 0] INFO [2022-01-10 12:53:43,439] jepsen worker 38 - jepsen.util 38 :invoke :write [12 48] INFO [2022-01-10 12:53:43,443] jepsen worker 89 - jepsen.util 269 :ok :write [29 34] INFO [2022-01-10 12:53:43,443] jepsen worker 86 - jepsen.util 176 :ok :write [28 3] INFO [2022-01-10 12:53:43,443] jepsen worker 89 - jepsen.util 269 :invoke :write [29 5] INFO [2022-01-10 12:53:43,444] jepsen worker 86 - jepsen.util 176 :invoke :write [28 24] INFO [2022-01-10 12:53:43,446] jepsen worker 40 - jepsen.util 130 :ok :write [13 15] INFO [2022-01-10 12:53:43,447] jepsen worker 40 - jepsen.util 130 :invoke :write [13 8] INFO [2022-01-10 12:53:43,448] jepsen worker 5 - jepsen.util 185 :ok :write [1 15] INFO [2022-01-10 12:53:43,449] jepsen worker 5 - jepsen.util 185 :invoke :write [1 14] INFO [2022-01-10 12:53:43,453] jepsen worker 16 - jepsen.util 286 :ok :write [5 49] INFO [2022-01-10 12:53:43,454] jepsen worker 16 - jepsen.util 286 :invoke :write [5 23] INFO [2022-01-10 12:53:43,455] jepsen worker 44 - jepsen.util 44 :fail :read [14 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:43,456] jepsen worker 2 - jepsen.util 92 :ok :write [0 20] INFO [2022-01-10 12:53:43,456] jepsen worker 38 - jepsen.util 38 :ok :write [12 48] INFO [2022-01-10 12:53:43,456] jepsen worker 44 - jepsen.util 44 :invoke :read [14 nil] INFO [2022-01-10 12:53:43,457] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:43,457] jepsen worker 2 - jepsen.util 92 :invoke :write [0 45] INFO [2022-01-10 12:53:43,457] jepsen worker 38 - jepsen.util 38 :ok :read [12 48] INFO [2022-01-10 12:53:43,458] jepsen worker 38 - jepsen.util 38 :invoke :write [12 22] INFO [2022-01-10 12:53:43,466] jepsen worker 86 - jepsen.util 176 :ok :write [28 24] INFO [2022-01-10 12:53:43,467] jepsen worker 89 - jepsen.util 269 :ok :write [29 5] INFO [2022-01-10 12:53:43,467] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,467] jepsen worker 89 - jepsen.util 269 :invoke :write [29 28] INFO [2022-01-10 12:53:43,467] jepsen worker 86 - jepsen.util 176 :ok :read [28 24] INFO [2022-01-10 12:53:43,468] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,469] jepsen worker 86 - jepsen.util 176 :ok :read [28 24] INFO [2022-01-10 12:53:43,469] jepsen worker 86 - jepsen.util 176 :invoke :write [28 23] INFO [2022-01-10 12:53:43,472] jepsen worker 5 - jepsen.util 185 :ok :write [1 14] INFO [2022-01-10 12:53:43,473] jepsen worker 5 - jepsen.util 185 :invoke :write [1 42] INFO [2022-01-10 12:53:43,473] jepsen worker 40 - jepsen.util 130 :ok :write [13 8] INFO [2022-01-10 12:53:43,474] jepsen worker 40 - jepsen.util 130 :invoke :write [13 38] INFO [2022-01-10 12:53:43,477] jepsen worker 38 - jepsen.util 38 :ok :write [12 22] INFO [2022-01-10 12:53:43,478] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:43,479] jepsen worker 38 - jepsen.util 38 :ok :read [12 22] INFO [2022-01-10 12:53:43,480] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:43,481] jepsen worker 38 - jepsen.util 38 :ok :read [12 22] INFO [2022-01-10 12:53:43,481] jepsen worker 38 - jepsen.util 38 :invoke :write [12 36] INFO [2022-01-10 12:53:43,483] jepsen worker 86 - jepsen.util 176 :ok :write [28 23] INFO [2022-01-10 12:53:43,483] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,484] jepsen worker 86 - jepsen.util 176 :ok :read [28 23] INFO [2022-01-10 12:53:43,485] jepsen worker 2 - jepsen.util 92 :ok :write [0 45] INFO [2022-01-10 12:53:43,485] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,486] jepsen worker 86 - jepsen.util 176 :ok :read [28 23] INFO [2022-01-10 12:53:43,486] jepsen worker 2 - jepsen.util 92 :invoke :write [0 12] INFO [2022-01-10 12:53:43,486] jepsen worker 86 - jepsen.util 176 :invoke :read [28 nil] INFO [2022-01-10 12:53:43,488] jepsen worker 86 - jepsen.util 176 :ok :read [28 23] INFO [2022-01-10 12:53:43,488] jepsen worker 36 - jepsen.util 126 :info :write [12 27] :RequestTimeoutException INFO [2022-01-10 12:53:43,489] jepsen worker 36 - jepsen.util 216 :invoke :write [12 47] INFO [2022-01-10 12:53:43,490] jepsen worker 3 - jepsen.util 3 :ok :write [1 10] INFO [2022-01-10 12:53:43,490] jepsen worker 89 - jepsen.util 269 :ok :write [29 28] INFO [2022-01-10 12:53:43,491] jepsen worker 86 - jepsen.util 176 :invoke :write [28 46] INFO [2022-01-10 12:53:43,491] jepsen worker 3 - jepsen.util 3 :invoke :write [1 11] INFO [2022-01-10 12:53:43,492] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:43,493] jepsen worker 89 - jepsen.util 269 :ok :read [29 28] INFO [2022-01-10 12:53:43,493] jepsen worker 89 - jepsen.util 269 :invoke :read [29 nil] INFO [2022-01-10 12:53:43,494] jepsen worker 89 - jepsen.util 269 :ok :read [29 28] INFO [2022-01-10 12:53:43,495] jepsen worker 38 - jepsen.util 38 :ok :write [12 36] INFO [2022-01-10 12:53:43,495] jepsen worker 89 - jepsen.util 269 :invoke :write [29 19] INFO [2022-01-10 12:53:43,496] jepsen worker 40 - jepsen.util 130 :ok :write [13 38] INFO [2022-01-10 12:53:43,496] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:43,497] jepsen worker 40 - jepsen.util 130 :invoke :write [13 30] INFO [2022-01-10 12:53:43,497] jepsen worker 38 - jepsen.util 38 :ok :read [12 36] INFO [2022-01-10 12:53:43,498] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:43,498] jepsen worker 38 - jepsen.util 38 :ok :read [12 36] INFO [2022-01-10 12:53:43,499] jepsen worker 38 - jepsen.util 38 :invoke :read [12 nil] INFO [2022-01-10 12:53:43,500] jepsen worker 38 - jepsen.util 38 :ok :read [12 36] WARN [2022-01-10 12:53:43,499] jepsen worker nemesis - jepsen.generator.interpreter Process :nemesis crashed com.jcraft.jsch.JSchException: channel is not opened. at com.jcraft.jsch.Channel.sendChannelOpen(Channel.java:765) at com.jcraft.jsch.Channel.connect(Channel.java:151) at com.jcraft.jsch.Channel.connect(Channel.java:145) at clj_ssh.ssh$connect_channel.invokeStatic(ssh.clj:512) at clj_ssh.ssh$connect_channel.invoke(ssh.clj:509) at clj_ssh.ssh$ssh_exec_proc.invokeStatic(ssh.clj:667) at clj_ssh.ssh$ssh_exec_proc.invoke(ssh.clj:646) at clj_ssh.ssh$ssh_exec.invokeStatic(ssh.clj:677) at clj_ssh.ssh$ssh_exec.invoke(ssh.clj:670) at clj_ssh.ssh$ssh.invokeStatic(ssh.clj:723) at clj_ssh.ssh$ssh.invoke(ssh.clj:699) at jepsen.control.clj_ssh.Remote.execute_BANG_(clj_ssh.clj:73) at jepsen.control.scp.Remote.execute_BANG_(scp.clj:91) at jepsen.control.retry.Remote$fn__3118.invoke(retry.clj:53) at jepsen.control.retry.Remote.execute_BANG_(retry.clj:53) at jepsen.control$ssh_STAR_.invokeStatic(control.clj:134) at jepsen.control$ssh_STAR_.invoke(control.clj:126) at jepsen.control$exec_STAR_.invokeStatic(control.clj:141) at jepsen.control$exec_STAR_.doInvoke(control.clj:138) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$apply.invoke(core.clj:662) at jepsen.control$exec.invokeStatic(control.clj:157) at jepsen.control$exec.doInvoke(control.clj:151) at clojure.lang.RestFn.invoke(RestFn.java:436) at couchbase.util$perform_kill_command.invokeStatic(util.clj:75) at couchbase.util$perform_kill_command.invoke(util.clj:53) at couchbase.util$kill_process.invokeStatic(util.clj:80) at couchbase.util$kill_process.invoke(util.clj:77) at couchbase.nemesis$kill_process.invokeStatic(nemesis.clj:267) at couchbase.nemesis$kill_process.invoke(nemesis.clj:261) at couchbase.nemesis$couchbase$reify__3984.invoke_BANG_(nemesis.clj:413) at jepsen.generator.interpreter.NemesisWorker.invoke_BANG_(interpreter.clj:74) at jepsen.generator.interpreter$spawn_worker$fn__9837$fn__9838.invoke(interpreter.clj:137) at jepsen.generator.interpreter$spawn_worker$fn__9837.invoke(interpreter.clj:120) at clojure.core$binding_conveyor_fn$fn__5772.invoke(core.clj:2034) at clojure.lang.AFn.call(AFn.java:18) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) INFO [2022-01-10 12:53:43,500] jepsen worker 38 - jepsen.util 38 :invoke :write [12 43] INFO [2022-01-10 12:53:43,504] jepsen worker 2 - jepsen.util 92 :ok :write [0 12] INFO [2022-01-10 12:53:43,504] jepsen worker 2 - jepsen.util 92 :invoke :read [0 nil] INFO [2022-01-10 12:53:43,505] jepsen worker nemesis - jepsen.generator.interpreter Aborting test due to nemesis failure. INFO [2022-01-10 12:53:43,505] jepsen worker 2 - jepsen.util 92 :ok :read [0 12] INFO [2022-01-10 12:53:43,508] jepsen worker 3 - jepsen.util 3 :ok :write [1 11] INFO [2022-01-10 12:53:43,510] jepsen worker 89 - jepsen.util 269 :ok :write [29 19] INFO [2022-01-10 12:53:43,514] jepsen worker 40 - jepsen.util 130 :ok :write [13 30] INFO [2022-01-10 12:53:43,515] jepsen worker 86 - jepsen.util 176 :ok :write [28 46] INFO [2022-01-10 12:53:43,519] jepsen worker 38 - jepsen.util 38 :ok :write [12 43] INFO [2022-01-10 12:53:43,526] jepsen worker 5 - jepsen.util 185 :ok :write [1 42] INFO [2022-01-10 12:53:43,533] jepsen worker 36 - jepsen.util 216 :ok :write [12 47] INFO [2022-01-10 12:53:43,548] jepsen worker 85 - jepsen.util 265 :ok :write [28 16] INFO [2022-01-10 12:53:43,557] jepsen worker 41 - jepsen.util 41 :ok :write [13 21] INFO [2022-01-10 12:53:43,560] jepsen worker 4 - jepsen.util 94 :ok :write [1 22] INFO [2022-01-10 12:53:43,586] jepsen worker 77 - jepsen.util 257 :info :write [25 5] :RequestTimeoutException INFO [2022-01-10 12:53:43,635] jepsen worker 9 - jepsen.util 279 :info :write [3 9] :RequestTimeoutException INFO [2022-01-10 12:53:43,643] jepsen worker 51 - jepsen.util 141 :ok :write [17 21] INFO [2022-01-10 12:53:43,667] jepsen worker 63 - jepsen.util 333 :info :write [21 16] :RequestTimeoutException INFO [2022-01-10 12:53:43,688] jepsen worker 20 - jepsen.util 380 :fail :read [6 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:43,737] jepsen worker 88 - jepsen.util 178 :ok :write [29 19] INFO [2022-01-10 12:53:43,742] jepsen worker 39 - jepsen.util 129 :ok :write [13 39] INFO [2022-01-10 12:53:43,812] jepsen worker 21 - jepsen.util 201 :ok :write [7 10] INFO [2022-01-10 12:53:43,814] jepsen worker 69 - jepsen.util 159 :ok :write [23 28] INFO [2022-01-10 12:53:43,823] jepsen worker 83 - jepsen.util 263 :ok :write [27 46] INFO [2022-01-10 12:53:43,825] jepsen worker 14 - jepsen.util 284 :ok :write [4 46] INFO [2022-01-10 12:53:43,825] jepsen worker 37 - jepsen.util 217 :ok :write [12 43] INFO [2022-01-10 12:53:43,829] jepsen worker 48 - jepsen.util 228 :ok :write [16 13] INFO [2022-01-10 12:53:43,830] jepsen worker 80 - jepsen.util 260 :ok :write [26 42] INFO [2022-01-10 12:53:43,836] jepsen worker 35 - jepsen.util 215 :ok :write [11 1] INFO [2022-01-10 12:53:43,840] jepsen worker 7 - jepsen.util 7 :ok :write [2 40] INFO [2022-01-10 12:53:43,847] jepsen worker 60 - jepsen.util 240 :ok :write [20 37] INFO [2022-01-10 12:53:43,854] jepsen worker 67 - jepsen.util 247 :ok :write [22 48] INFO [2022-01-10 12:53:43,858] jepsen worker 76 - jepsen.util 256 :ok :write [25 25] INFO [2022-01-10 12:53:43,864] jepsen worker 43 - jepsen.util 133 :ok :write [14 7] INFO [2022-01-10 12:53:43,869] jepsen worker 16 - jepsen.util 286 :ok :write [5 23] INFO [2022-01-10 12:53:43,874] jepsen worker 22 - jepsen.util 112 :ok :write [7 2] INFO [2022-01-10 12:53:43,899] jepsen worker 68 - jepsen.util 68 :ok :write [22 21] INFO [2022-01-10 12:53:43,904] jepsen worker 1 - jepsen.util 91 :ok :write [0 3] INFO [2022-01-10 12:53:43,913] jepsen worker 84 - jepsen.util 84 :ok :write [28 25] INFO [2022-01-10 12:53:43,919] jepsen worker 81 - jepsen.util 171 :ok :write [27 43] INFO [2022-01-10 12:53:43,926] jepsen worker 49 - jepsen.util 319 :ok :write [16 31] INFO [2022-01-10 12:53:43,931] jepsen worker 15 - jepsen.util 15 :ok :write [5 18] INFO [2022-01-10 12:53:43,947] jepsen worker 33 - jepsen.util 123 :ok :write [11 27] INFO [2022-01-10 12:53:43,963] jepsen worker 70 - jepsen.util 160 :ok :write [23 45] INFO [2022-01-10 12:53:43,974] jepsen worker 75 - jepsen.util 165 :ok :write [25 11] INFO [2022-01-10 12:53:44,013] jepsen worker 0 - jepsen.util 180 :ok :write [0 48] INFO [2022-01-10 12:53:44,018] jepsen worker 78 - jepsen.util 168 :ok :write [26 9] INFO [2022-01-10 12:53:44,045] jepsen worker 42 - jepsen.util 222 :ok :write [14 14] INFO [2022-01-10 12:53:44,062] jepsen worker 82 - jepsen.util 442 :ok :write [27 10] INFO [2022-01-10 12:53:44,098] jepsen worker 71 - jepsen.util 161 :ok :write [23 6] INFO [2022-01-10 12:53:44,107] jepsen worker 53 - jepsen.util 143 :ok :write [17 30] INFO [2022-01-10 12:53:44,146] jepsen worker 34 - jepsen.util 214 :ok :write [11 1] INFO [2022-01-10 12:53:44,225] jepsen worker 79 - jepsen.util 349 :ok :write [26 43] INFO [2022-01-10 12:53:44,247] jepsen worker 50 - jepsen.util 50 :ok :write [16 2] INFO [2022-01-10 12:53:44,301] jepsen worker 61 - jepsen.util 61 :ok :write [20 22] WARN [2022-01-10 12:53:44,304] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":45724,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000001/000000005D716E8E","last_local_socket":"172.23.104.79:34680","total_dispatch_duration_us":281068,"total_server_duration_us":47560,"encode_duration_us":14,"operation_id":"0x3ca3","timeout_ms":10000,"last_server_duration_us":45167,"total_duration_us":10010732},{"operation_name":"upsert","last_dispatch_duration_us":775,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000001/000000005D716E8E","last_local_socket":"172.23.104.79:34680","total_dispatch_duration_us":223537,"total_server_duration_us":2570,"encode_duration_us":19,"operation_id":"0x3e36","timeout_ms":10000,"last_server_duration_us":155,"total_duration_us":10008661},{"operation_name":"upsert","last_dispatch_duration_us":5933998,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":5933998,"total_server_duration_us":38,"encode_duration_us":16,"operation_id":"0x45ce","timeout_ms":10000,"last_server_duration_us":38,"total_duration_us":10007930},{"operation_name":"upsert","last_dispatch_duration_us":9002558,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":12971158,"total_server_duration_us":9001665,"encode_duration_us":22,"operation_id":"0x38e3","timeout_ms":10000,"last_server_duration_us":9001311,"total_duration_us":10007609},{"operation_name":"upsert","last_dispatch_duration_us":607,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000001/00000000100998FD","last_local_socket":"172.23.104.79:58632","total_dispatch_duration_us":17766,"total_server_duration_us":2322,"encode_duration_us":30,"operation_id":"0x36fd","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10006454},{"operation_name":"upsert","last_dispatch_duration_us":676,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":6337,"total_server_duration_us":901,"encode_duration_us":12,"operation_id":"0x41eb","timeout_ms":10000,"last_server_duration_us":76,"total_duration_us":10006245},{"operation_name":"upsert","last_dispatch_duration_us":649,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":6130,"total_server_duration_us":767,"encode_duration_us":10,"operation_id":"0x425b","timeout_ms":10000,"last_server_duration_us":108,"total_duration_us":10004892},{"operation_name":"upsert","last_dispatch_duration_us":6986362,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":6990729,"total_server_duration_us":550,"encode_duration_us":17,"operation_id":"0x43f9","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10004366},{"operation_name":"upsert","last_dispatch_duration_us":644,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000001/000000005CDEF525","last_local_socket":"172.23.104.79:48370","total_dispatch_duration_us":16577,"total_server_duration_us":2359,"encode_duration_us":18,"operation_id":"0x3dcc","timeout_ms":10000,"last_server_duration_us":208,"total_duration_us":10001192},{"operation_name":"upsert","last_dispatch_duration_us":9011628,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000001/0000000023B95902","last_local_socket":"172.23.104.79:40178","total_dispatch_duration_us":9011628,"total_server_duration_us":9010849,"encode_duration_us":14,"operation_id":"0x3f6a","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":9011995}],"total_count":33}} WARN [2022-01-10 12:53:44,304] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9002558,"last_local_id":"2B94F89700000001/0000000023B95902","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":12971158,"last_local_socket":"172.23.104.79:40178","total_server_duration_us":9001665,"encode_duration_us":22,"operation_id":"0x38e3","timeout_ms":10000,"last_server_duration_us":9001311,"total_duration_us":10007609},{"operation_name":"upsert","last_dispatch_duration_us":45724,"last_local_id":"2B94F89700000001/000000005D716E8E","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":281068,"last_local_socket":"172.23.104.79:34680","total_server_duration_us":47560,"encode_duration_us":14,"operation_id":"0x3ca3","timeout_ms":10000,"last_server_duration_us":45167,"total_duration_us":10010732}],"total_count":2}} INFO [2022-01-10 12:53:44,329] jepsen worker 8 - jepsen.util 98 :ok :write [2 4] INFO [2022-01-10 12:53:44,363] jepsen worker 10 - jepsen.util 370 :fail :read [3 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:44,402] jepsen worker 66 - jepsen.util 246 :ok :write [22 44] INFO [2022-01-10 12:53:44,806] jepsen worker 30 - jepsen.util 300 :info :write [10 17] :RequestTimeoutException INFO [2022-01-10 12:53:44,848] jepsen worker 6 - jepsen.util 276 :ok :write [2 7] INFO [2022-01-10 12:53:44,888] jepsen worker 32 - jepsen.util 302 :info :write [10 44] :RequestTimeoutException INFO [2022-01-10 12:53:45,027] jepsen worker 28 - jepsen.util 388 :fail :read [9 nil] :UnambiguousTimeoutException INFO [2022-01-10 12:53:45,067] jepsen worker 87 - jepsen.util 87 :info :write [29 22] :RequestTimeoutException INFO [2022-01-10 12:53:45,465] jepsen worker 44 - jepsen.util 44 :fail :read [14 nil] :UnambiguousTimeoutException WARN [2022-01-10 12:53:45,605] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":16014084,"last_local_id":"2B94F89700000002/00000000E26E7095","last_remote_socket":"172.23.107.227:11210","total_dispatch_duration_us":16014084,"last_local_socket":"172.23.104.79:58606","total_server_duration_us":9010849,"encode_duration_us":27,"operation_id":"0x33e3","timeout_ms":10000,"last_server_duration_us":9010849,"total_duration_us":10008803}],"total_count":1}} INFO [2022-01-10 12:53:45,637] jepsen worker 13 - jepsen.util 13 :info :write [4 17] :RequestTimeoutException WARN [2022-01-10 12:53:45,705] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":920,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000004/0000000048226CCA","last_local_socket":"172.23.104.79:48384","total_dispatch_duration_us":18335,"total_server_duration_us":2355,"encode_duration_us":13,"operation_id":"0x4242","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10008578},{"operation_name":"upsert","last_dispatch_duration_us":601,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000004/00000000C4B4FBBF","last_local_socket":"172.23.104.79:58638","total_dispatch_duration_us":22272,"total_server_duration_us":3030,"encode_duration_us":16,"operation_id":"0x4169","timeout_ms":10000,"last_server_duration_us":117,"total_duration_us":10008171},{"operation_name":"upsert","last_dispatch_duration_us":700,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":26983,"total_server_duration_us":3574,"encode_duration_us":12,"operation_id":"0x4293","timeout_ms":10000,"last_server_duration_us":117,"total_duration_us":10006758},{"operation_name":"upsert","last_dispatch_duration_us":647,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":17279,"total_server_duration_us":2340,"encode_duration_us":13,"operation_id":"0x4720","timeout_ms":10000,"last_server_duration_us":126,"total_duration_us":10006538},{"operation_name":"upsert","last_dispatch_duration_us":572,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":17478,"total_server_duration_us":2306,"encode_duration_us":23,"operation_id":"0x416e","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10005984},{"operation_name":"upsert","last_dispatch_duration_us":723,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":16549,"total_server_duration_us":1991,"encode_duration_us":20,"operation_id":"0x44df","timeout_ms":10000,"last_server_duration_us":49,"total_duration_us":10005169},{"operation_name":"upsert","last_dispatch_duration_us":838,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":19655,"total_server_duration_us":2829,"encode_duration_us":14,"operation_id":"0x3ab3","timeout_ms":10000,"last_server_duration_us":268,"total_duration_us":10003516},{"operation_name":"upsert","last_dispatch_duration_us":459,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000004/0000000077268043","last_local_socket":"172.23.104.79:34634","total_dispatch_duration_us":18314,"total_server_duration_us":2823,"encode_duration_us":23,"operation_id":"0x3f67","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10003459},{"operation_name":"upsert","last_dispatch_duration_us":459,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000004/000000002DD27EA7","last_local_socket":"172.23.104.79:40242","total_dispatch_duration_us":18316,"total_server_duration_us":2209,"encode_duration_us":11,"operation_id":"0x43ed","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10001809},{"operation_name":"upsert","last_dispatch_duration_us":153309,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000004/00000000C4B4FBBF","last_local_socket":"172.23.104.79:58638","total_dispatch_duration_us":179755,"total_server_duration_us":156247,"encode_duration_us":10,"operation_id":"0x488f","timeout_ms":10000,"last_server_duration_us":152740,"total_duration_us":9408383}],"total_count":19}} WARN [2022-01-10 12:53:45,706] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1236,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":31540,"total_server_duration_us":8989,"encode_duration_us":19,"operation_id":"0x3dcd","timeout_ms":10000,"last_server_duration_us":320,"total_duration_us":10010491},{"operation_name":"upsert","last_dispatch_duration_us":721,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000002/00000000E26E7095","last_local_socket":"172.23.104.79:58606","total_dispatch_duration_us":5201335,"total_server_duration_us":6715,"encode_duration_us":32,"operation_id":"0x46c9","timeout_ms":10000,"last_server_duration_us":92,"total_duration_us":10008889},{"operation_name":"upsert","last_dispatch_duration_us":539,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":17877,"total_server_duration_us":3016,"encode_duration_us":11,"operation_id":"0x49ac","timeout_ms":10000,"last_server_duration_us":100,"total_duration_us":10007603},{"operation_name":"upsert","last_dispatch_duration_us":1298,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000002/00000000F0645BC3","last_local_socket":"172.23.104.79:34668","total_dispatch_duration_us":19727,"total_server_duration_us":3153,"encode_duration_us":14,"operation_id":"0x3d5a","timeout_ms":10000,"last_server_duration_us":49,"total_duration_us":10003477},{"operation_name":"upsert","last_dispatch_duration_us":632,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000002/00000000F44673B9","last_local_socket":"172.23.104.79:40206","total_dispatch_duration_us":17505,"total_server_duration_us":2887,"encode_duration_us":13,"operation_id":"0x425c","timeout_ms":10000,"last_server_duration_us":100,"total_duration_us":10002599},{"operation_name":"upsert","last_dispatch_duration_us":7794,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000002/00000000F0645BC3","last_local_socket":"172.23.104.79:34668","total_dispatch_duration_us":180748,"total_server_duration_us":9838,"encode_duration_us":40,"operation_id":"0x4b1d","timeout_ms":10000,"last_server_duration_us":7231,"total_duration_us":8383365},{"operation_name":"upsert","last_dispatch_duration_us":36056,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":231641,"total_server_duration_us":37232,"encode_duration_us":17,"operation_id":"0x4245","timeout_ms":10000,"last_server_duration_us":35513,"total_duration_us":7425672},{"operation_name":"upsert","last_dispatch_duration_us":158466,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000002/00000000F0645BC3","last_local_socket":"172.23.104.79:34668","total_dispatch_duration_us":172832,"total_server_duration_us":159759,"encode_duration_us":16,"operation_id":"0x4112","timeout_ms":10000,"last_server_duration_us":157828,"total_duration_us":6351399},{"operation_name":"upsert","last_dispatch_duration_us":233847,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":559464,"total_server_duration_us":235198,"encode_duration_us":21,"operation_id":"0x4674","timeout_ms":10000,"last_server_duration_us":233435,"total_duration_us":6233306},{"operation_name":"upsert","last_dispatch_duration_us":8906,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000002/000000004AD6A961","last_local_socket":"172.23.104.79:48334","total_dispatch_duration_us":28937,"total_server_duration_us":10070,"encode_duration_us":10,"operation_id":"0x506d","timeout_ms":10000,"last_server_duration_us":8284,"total_duration_us":5178755}],"total_count":26}} WARN [2022-01-10 12:53:45,805] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":9016337,"last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":17022064,"last_local_socket":"172.23.104.79:40254","total_server_duration_us":9015136,"encode_duration_us":28,"operation_id":"0x323a","timeout_ms":10000,"last_server_duration_us":9015090,"total_duration_us":10004090},{"operation_name":"upsert","last_dispatch_duration_us":16108444,"last_local_id":"2B94F89700000003/0000000020258EC7","last_remote_socket":"172.23.106.61:11210","total_dispatch_duration_us":16115894,"last_local_socket":"172.23.104.79:48408","total_server_duration_us":9008680,"encode_duration_us":20,"operation_id":"0x30ee","timeout_ms":10000,"last_server_duration_us":9007669,"total_duration_us":10008226},{"operation_name":"upsert","last_dispatch_duration_us":9013215,"last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":17879879,"last_local_socket":"172.23.104.79:40254","total_server_duration_us":9013536,"encode_duration_us":23,"operation_id":"0x30b9","timeout_ms":10000,"last_server_duration_us":9012969,"total_duration_us":10009010}],"total_count":3}} WARN [2022-01-10 12:53:45,805] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":9185988,"last_local_id":"2B94F89700000005/000000004F9479E2","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":9185988,"last_local_socket":"172.23.104.79:40240","total_server_duration_us":38,"operation_id":"0x3f65","timeout_ms":2000,"last_server_duration_us":38,"total_duration_us":2007812},{"operation_name":"upsert","last_dispatch_duration_us":17067794,"last_local_id":"2B94F89700000005/000000004F9479E2","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":25415766,"last_local_socket":"172.23.104.79:40240","total_server_duration_us":9003711,"encode_duration_us":27,"operation_id":"0x2247","timeout_ms":10000,"last_server_duration_us":9003430,"total_duration_us":10007100}],"total_count":2}} WARN [2022-01-10 12:53:45,806] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":574,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000003/00000000B6BEC0CE","last_local_socket":"172.23.104.79:40254","total_dispatch_duration_us":6182,"total_server_duration_us":808,"encode_duration_us":10,"operation_id":"0x43e9","timeout_ms":10000,"last_server_duration_us":108,"total_duration_us":10009317},{"operation_name":"upsert","last_dispatch_duration_us":6347344,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":6347344,"total_server_duration_us":23,"encode_duration_us":20,"operation_id":"0x44ab","timeout_ms":10000,"last_server_duration_us":23,"total_duration_us":10008254},{"operation_name":"upsert","last_dispatch_duration_us":597,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":19321,"total_server_duration_us":2887,"encode_duration_us":12,"operation_id":"0x3ffb","timeout_ms":10000,"last_server_duration_us":117,"total_duration_us":10007962},{"operation_name":"upsert","last_dispatch_duration_us":8025456,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":8025456,"total_server_duration_us":38,"encode_duration_us":15,"operation_id":"0x416d","timeout_ms":10000,"last_server_duration_us":38,"total_duration_us":10006725},{"operation_name":"upsert","last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","encode_duration_us":17,"operation_id":"0x3f58","timeout_ms":10000,"total_duration_us":10005279},{"operation_name":"upsert","last_dispatch_duration_us":7702710,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000003/0000000020258EC7","last_local_socket":"172.23.104.79:48408","total_dispatch_duration_us":7702710,"total_server_duration_us":23,"encode_duration_us":11,"operation_id":"0x4240","timeout_ms":10000,"last_server_duration_us":23,"total_duration_us":10004193},{"operation_name":"upsert","last_dispatch_duration_us":550,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000003/000000000DBBABF1","last_local_socket":"172.23.104.79:34702","total_dispatch_duration_us":20538,"total_server_duration_us":2359,"encode_duration_us":30,"operation_id":"0x3d35","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10003134},{"operation_name":"upsert","last_dispatch_duration_us":736,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000003/000000000DBBABF1","last_local_socket":"172.23.104.79:34702","total_dispatch_duration_us":19172,"total_server_duration_us":3184,"encode_duration_us":15,"operation_id":"0x4542","timeout_ms":10000,"last_server_duration_us":56,"total_duration_us":10001004},{"operation_name":"upsert","last_dispatch_duration_us":2865,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000003/000000000DBBABF1","last_local_socket":"172.23.104.79:34702","total_dispatch_duration_us":22081,"total_server_duration_us":3814,"encode_duration_us":19,"operation_id":"0x427e","timeout_ms":10000,"last_server_duration_us":197,"total_duration_us":10000838},{"operation_name":"upsert","last_dispatch_duration_us":17958,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000003/00000000B68ED78B","last_local_socket":"172.23.104.79:58580","total_dispatch_duration_us":39101,"total_server_duration_us":21197,"encode_duration_us":19,"operation_id":"0x40d4","timeout_ms":10000,"last_server_duration_us":17439,"total_duration_us":8846313}],"total_count":19}} WARN [2022-01-10 12:53:45,806] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":1870,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":13475,"total_server_duration_us":1345,"encode_duration_us":19,"operation_id":"0x49c7","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10009009},{"operation_name":"upsert","last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","encode_duration_us":12,"operation_id":"0x4721","timeout_ms":10000,"total_duration_us":10008138},{"operation_name":"upsert","last_dispatch_duration_us":597,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":11441,"total_server_duration_us":1476,"encode_duration_us":11,"operation_id":"0x4ae7","timeout_ms":10000,"last_server_duration_us":62,"total_duration_us":10007950},{"operation_name":"upsert","last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","encode_duration_us":12,"operation_id":"0x45a0","timeout_ms":10000,"total_duration_us":10007271},{"operation_name":"upsert","last_dispatch_duration_us":601,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":14420,"total_server_duration_us":2101,"encode_duration_us":13,"operation_id":"0x4257","timeout_ms":10000,"last_server_duration_us":27,"total_duration_us":10006984},{"operation_name":"upsert","last_dispatch_duration_us":485,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":72839,"total_server_duration_us":2417,"encode_duration_us":16,"operation_id":"0x3b24","timeout_ms":10000,"last_server_duration_us":56,"total_duration_us":10003277},{"operation_name":"upsert","last_dispatch_duration_us":580,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000005/000000006FA84E55","last_local_socket":"172.23.104.79:34638","total_dispatch_duration_us":164014,"total_server_duration_us":3184,"encode_duration_us":18,"operation_id":"0x4133","timeout_ms":10000,"last_server_duration_us":49,"total_duration_us":10003111},{"operation_name":"upsert","last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000005/000000004F9479E2","last_local_socket":"172.23.104.79:40240","encode_duration_us":17,"operation_id":"0x43f3","timeout_ms":10000,"total_duration_us":10001454},{"operation_name":"upsert","last_dispatch_duration_us":53508,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000005/000000006FA84E55","last_local_socket":"172.23.104.79:34638","total_dispatch_duration_us":156511,"total_server_duration_us":54953,"encode_duration_us":12,"operation_id":"0x4722","timeout_ms":10000,"last_server_duration_us":52896,"total_duration_us":7350107},{"operation_name":"upsert","last_dispatch_duration_us":85311,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000005/00000000E496BE9B","last_local_socket":"172.23.104.79:58646","total_dispatch_duration_us":112905,"total_server_duration_us":92784,"encode_duration_us":12,"operation_id":"0x42b3","timeout_ms":10000,"last_server_duration_us":84429,"total_duration_us":6777966}],"total_count":22}} WARN [2022-01-10 12:53:45,905] cb-events - com.couchbase.tracing [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"upsert","last_dispatch_duration_us":680,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":23193,"total_server_duration_us":2298,"encode_duration_us":13,"operation_id":"0x43f4","timeout_ms":10000,"last_server_duration_us":69,"total_duration_us":10007864},{"operation_name":"upsert","last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000006/000000001E9CFABF","last_local_socket":"172.23.104.79:40226","encode_duration_us":19,"operation_id":"0x4995","timeout_ms":10000,"total_duration_us":10007418},{"operation_name":"upsert","last_dispatch_duration_us":452,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":5077340,"total_server_duration_us":1940,"encode_duration_us":10,"operation_id":"0x4706","timeout_ms":10000,"last_server_duration_us":56,"total_duration_us":10006645},{"operation_name":"upsert","last_dispatch_duration_us":707,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":21035,"total_server_duration_us":2807,"encode_duration_us":10,"operation_id":"0x4548","timeout_ms":10000,"last_server_duration_us":231,"total_duration_us":10006638},{"operation_name":"upsert","last_dispatch_duration_us":5816378,"last_remote_socket":"172.23.105.187:11210","last_local_id":"2B94F89700000006/000000001E9CFABF","last_local_socket":"172.23.104.79:40226","total_dispatch_duration_us":5816378,"total_server_duration_us":27,"encode_duration_us":19,"operation_id":"0x45df","timeout_ms":10000,"last_server_duration_us":27,"total_duration_us":10005990},{"operation_name":"upsert","last_dispatch_duration_us":46736,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":61522,"total_server_duration_us":48239,"encode_duration_us":12,"operation_id":"0x3e54","timeout_ms":10000,"last_server_duration_us":46062,"total_duration_us":7253230},{"operation_name":"upsert","last_dispatch_duration_us":40701,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000006/0000000044D7A75E","last_local_socket":"172.23.104.79:48380","total_dispatch_duration_us":56287,"total_server_duration_us":42742,"encode_duration_us":12,"operation_id":"0x4c05","timeout_ms":10000,"last_server_duration_us":40163,"total_duration_us":6233958},{"operation_name":"upsert","last_dispatch_duration_us":324498,"last_remote_socket":"172.23.106.61:11210","last_local_id":"2B94F89700000006/0000000044D7A75E","last_local_socket":"172.23.104.79:48380","total_dispatch_duration_us":335410,"total_server_duration_us":325422,"encode_duration_us":12,"operation_id":"0x4981","timeout_ms":10000,"last_server_duration_us":323822,"total_duration_us":5493930},{"operation_name":"upsert","last_dispatch_duration_us":37316,"last_remote_socket":"172.23.107.227:11210","last_local_id":"2B94F89700000006/0000000042177E4B","last_local_socket":"172.23.104.79:58592","total_dispatch_duration_us":51801,"total_server_duration_us":36686,"encode_duration_us":15,"operation_id":"0x4382","timeout_ms":10000,"last_server_duration_us":35011,"total_duration_us":5200655},{"operation_name":"upsert","last_dispatch_duration_us":11819,"last_remote_socket":"172.23.107.228:11210","last_local_id":"2B94F89700000006/00000000A79E4C0A","last_local_socket":"172.23.104.79:34662","total_dispatch_duration_us":23382,"total_server_duration_us":13581,"encode_duration_us":14,"operation_id":"0x52c0","timeout_ms":10000,"last_server_duration_us":11241,"total_duration_us":5176394}],"total_count":26}} WARN [2022-01-10 12:53:45,905] cb-events - com.couchbase.tracing [com.couchbase.tracing][OrphansRecordedEvent][10s] Orphaned requests found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":3037331,"last_local_id":"2B94F89700000006/000000001E9CFABF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":3037331,"last_local_socket":"172.23.104.79:40226","total_server_duration_us":38,"operation_id":"0x4b2b","timeout_ms":2000,"last_server_duration_us":38,"total_duration_us":2006602},{"operation_name":"get","last_dispatch_duration_us":6037496,"last_local_id":"2B94F89700000006/000000001E9CFABF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":6037496,"last_local_socket":"172.23.104.79:40226","total_server_duration_us":15,"operation_id":"0x45ac","timeout_ms":2000,"last_server_duration_us":15,"total_duration_us":2007149},{"operation_name":"get","last_dispatch_duration_us":7827584,"last_local_id":"2B94F89700000006/000000001E9CFABF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":7827584,"last_local_socket":"172.23.104.79:40226","total_server_duration_us":15,"operation_id":"0x4258","timeout_ms":2000,"last_server_duration_us":15,"total_duration_us":2007454},{"operation_name":"get","last_dispatch_duration_us":7075351,"last_local_id":"2B94F89700000006/00000000A79E4C0A","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":7075351,"last_local_socket":"172.23.104.79:34662","total_server_duration_us":15,"operation_id":"0x43ce","timeout_ms":2000,"last_server_duration_us":15,"total_duration_us":2009162},{"operation_name":"get","last_dispatch_duration_us":8050464,"last_local_id":"2B94F89700000006/000000001E9CFABF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":8050464,"last_local_socket":"172.23.104.79:40226","total_server_duration_us":38,"operation_id":"0x41c9","timeout_ms":2000,"last_server_duration_us":38,"total_duration_us":2010180},{"operation_name":"upsert","last_dispatch_duration_us":17158888,"last_local_id":"2B94F89700000006/000000001E9CFABF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":17158888,"last_local_socket":"172.23.104.79:40226","total_server_duration_us":9003430,"encode_duration_us":56,"operation_id":"0x3262","timeout_ms":10000,"last_server_duration_us":9003430,"total_duration_us":10010053},{"operation_name":"get","last_dispatch_duration_us":5050990,"last_local_id":"2B94F89700000006/000000001E9CFABF","last_remote_socket":"172.23.105.187:11210","total_dispatch_duration_us":5050990,"last_local_socket":"172.23.104.79:40226","total_server_duration_us":11,"operation_id":"0x4756","timeout_ms":2000,"last_server_duration_us":11,"total_duration_us":2010842},{"operation_name":"upsert","last_dispatch_duration_us":17016095,"last_local_id":"2B94F89700000006/00000000A79E4C0A","last_remote_socket":"172.23.107.228:11210","total_dispatch_duration_us":17023446,"last_local_socket":"172.23.104.79:34662","total_server_duration_us":9009887,"encode_duration_us":13,"operation_id":"0x312c","timeout_ms":10000,"last_server_duration_us":9008729,"total_duration_us":10002767}],"total_count":8}} INFO [2022-01-10 12:53:47,707] jepsen worker 74 - jepsen.util 344 :info :write [24 46] :RequestTimeoutException INFO [2022-01-10 12:53:47,721] jepsen worker 12 - jepsen.util 192 :ok :write [4 14] INFO [2022-01-10 12:53:47,721] jepsen worker 52 - jepsen.util 52 :ok :write [17 24] INFO [2022-01-10 12:53:47,806] jepsen worker 29 - jepsen.util 299 :info :write [9 21] :RequestTimeoutException INFO [2022-01-10 12:53:47,940] jepsen worker 62 - jepsen.util 242 :ok :write [20 5] INFO [2022-01-10 12:53:48,130] jepsen worker 23 - jepsen.util 113 :ok :write [7 29] INFO [2022-01-10 12:53:48,672] jepsen worker 45 - jepsen.util 315 :info :write [15 44] :RequestTimeoutException INFO [2022-01-10 12:53:48,845] jepsen worker 27 - jepsen.util 387 :info :write [9 24] :RequestTimeoutException INFO [2022-01-10 12:53:49,217] jepsen worker 64 - jepsen.util 334 :info :write [21 38] :RequestTimeoutException INFO [2022-01-10 12:53:49,624] jepsen worker 58 - jepsen.util 418 :info :write [19 32] :RequestTimeoutException INFO [2022-01-10 12:53:49,657] jepsen worker 26 - jepsen.util 386 :info :write [8 36] :RequestTimeoutException INFO [2022-01-10 12:53:50,605] jepsen worker 17 - jepsen.util 377 :info :write [5 36] :RequestTimeoutException INFO [2022-01-10 12:53:50,644] jepsen worker 19 - jepsen.util 379 :info :write [6 32] :RequestTimeoutException INFO [2022-01-10 12:53:50,932] jepsen worker 59 - jepsen.util 419 :info :write [19 47] :RequestTimeoutException INFO [2022-01-10 12:53:50,998] jepsen worker 54 - jepsen.util 414 :info :write [18 8] :RequestTimeoutException INFO [2022-01-10 12:53:51,024] jepsen worker 72 - jepsen.util 432 :info :write [24 44] :RequestTimeoutException INFO [2022-01-10 12:53:51,076] jepsen worker 25 - jepsen.util 385 :info :write [8 24] :RequestTimeoutException INFO [2022-01-10 12:53:51,096] jepsen worker 24 - jepsen.util 384 :info :write [8 13] :RequestTimeoutException INFO [2022-01-10 12:53:51,638] jepsen worker 18 - jepsen.util 378 :info :write [6 10] :RequestTimeoutException INFO [2022-01-10 12:53:51,642] jepsen worker 65 - jepsen.util 425 :info :write [21 14] :RequestTimeoutException INFO [2022-01-10 12:53:51,654] jepsen worker 47 - jepsen.util 407 :info :write [15 23] :RequestTimeoutException INFO [2022-01-10 12:53:51,674] jepsen worker 56 - jepsen.util 416 :info :write [18 25] :RequestTimeoutException INFO [2022-01-10 12:53:52,287] jepsen worker 57 - jepsen.util 417 :info :write [19 39] :RequestTimeoutException INFO [2022-01-10 12:53:52,674] jepsen worker 55 - jepsen.util 415 :info :write [18 29] :RequestTimeoutException INFO [2022-01-10 12:53:52,794] jepsen worker 11 - jepsen.util 371 :info :write [3 19] :RequestTimeoutException INFO [2022-01-10 12:53:52,837] jepsen worker 73 - jepsen.util 433 :info :write [24 9] :RequestTimeoutException INFO [2022-01-10 12:53:52,916] jepsen worker 31 - jepsen.util 391 :info :write [10 24] :RequestTimeoutException INFO [2022-01-10 12:53:53,035] jepsen worker 46 - jepsen.util 406 :info :write [15 13] :RequestTimeoutException INFO [2022-01-10 12:53:53,115] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][7799us] Closed bucket "default" {"coreId":"0x2b94f89700000002"} INFO [2022-01-10 12:53:53,115] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][888us] Node disconnected {"coreId":"0x2b94f89700000002","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 12:53:53,115] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][299us] Node disconnected {"coreId":"0x2b94f89700000002","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 12:53:53,115] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][166us] Node disconnected {"coreId":"0x2b94f89700000002","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 12:53:53,116] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][183us] Node disconnected {"coreId":"0x2b94f89700000002","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 12:53:53,116] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][1258us] Closed bucket "default" {"coreId":"0x2b94f89700000003"} INFO [2022-01-10 12:53:53,116] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][55ms] Completed shutdown and closed all open buckets {"coreId":"0x2b94f89700000002"} INFO [2022-01-10 12:53:53,116] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][242us] Node disconnected {"coreId":"0x2b94f89700000003","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 12:53:53,116] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][157us] Node disconnected {"coreId":"0x2b94f89700000003","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 12:53:53,116] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][156us] Node disconnected {"coreId":"0x2b94f89700000003","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 12:53:53,116] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][338us] Node disconnected {"coreId":"0x2b94f89700000003","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 12:53:53,215] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][1226us] Closed bucket "default" {"coreId":"0x2b94f89700000001"} INFO [2022-01-10 12:53:53,215] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][1019us] Closed bucket "default" {"coreId":"0x2b94f89700000005"} INFO [2022-01-10 12:53:53,215] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][196us] Node disconnected {"coreId":"0x2b94f89700000005","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 12:53:53,215] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][1092us] Closed bucket "default" {"coreId":"0x2b94f89700000004"} INFO [2022-01-10 12:53:53,216] cb-events - com.couchbase.core [com.couchbase.core][BucketClosedEvent][1182us] Closed bucket "default" {"coreId":"0x2b94f89700000006"} INFO [2022-01-10 12:53:53,216] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][143us] Node disconnected {"coreId":"0x2b94f89700000001","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 12:53:53,216] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][16ms] Completed shutdown and closed all open buckets {"coreId":"0x2b94f89700000003"} INFO [2022-01-10 12:53:53,216] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][149us] Node disconnected {"coreId":"0x2b94f89700000001","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 12:53:53,216] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][149us] Node disconnected {"coreId":"0x2b94f89700000006","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 12:53:53,216] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][306us] Node disconnected {"coreId":"0x2b94f89700000004","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][146us] Node disconnected {"coreId":"0x2b94f89700000001","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][229us] Node disconnected {"coreId":"0x2b94f89700000006","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][175us] Node disconnected {"coreId":"0x2b94f89700000004","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][361us] Node disconnected {"coreId":"0x2b94f89700000001","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][159us] Node disconnected {"coreId":"0x2b94f89700000006","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][17ms] Completed shutdown and closed all open buckets {"coreId":"0x2b94f89700000001"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][239us] Node disconnected {"coreId":"0x2b94f89700000004","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][394us] Node disconnected {"coreId":"0x2b94f89700000006","managerPort":"8091","remote":"172.23.107.228"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][280us] Node disconnected {"coreId":"0x2b94f89700000004","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][197us] Node disconnected {"coreId":"0x2b94f89700000005","managerPort":"8091","remote":"172.23.105.187"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][19ms] Completed shutdown and closed all open buckets {"coreId":"0x2b94f89700000006"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][16ms] Completed shutdown and closed all open buckets {"coreId":"0x2b94f89700000004"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][321us] Node disconnected {"coreId":"0x2b94f89700000005","managerPort":"8091","remote":"172.23.106.61"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.node [com.couchbase.node][NodeDisconnectedEvent][152us] Node disconnected {"coreId":"0x2b94f89700000005","managerPort":"8091","remote":"172.23.107.227"} INFO [2022-01-10 12:53:53,217] cb-events - com.couchbase.core [com.couchbase.core][ShutdownCompletedEvent][18ms] Completed shutdown and closed all open buckets {"coreId":"0x2b94f89700000005"} INFO [2022-01-10 12:53:53,382] jepsen test runner - jepsen.core Run complete, writing INFO [2022-01-10 12:53:53,384] jepsen fressian - couchbase.core Not writing fressian INFO [2022-01-10 12:53:54,247] jepsen test runner - jepsen.core Snarfing log files INFO [2022-01-10 12:53:54,555] jepsen node 172.23.107.227 - couchbase.util Running cbcollect_info on 172.23.107.227 INFO [2022-01-10 12:53:54,555] jepsen node 172.23.106.61 - couchbase.util Running cbcollect_info on 172.23.106.61 INFO [2022-01-10 12:53:54,654] jepsen node 172.23.107.228 - couchbase.util Running cbcollect_info on 172.23.107.228 INFO [2022-01-10 12:53:54,654] jepsen node 172.23.105.187 - couchbase.util Running cbcollect_info on 172.23.105.187 INFO [2022-01-10 12:59:38,877] jepsen node 172.23.106.61 - couchbase.util Creating hashtable dump on 172.23.106.61 INFO [2022-01-10 12:59:39,683] jepsen node 172.23.107.227 - couchbase.util Creating hashtable dump on 172.23.107.227 INFO [2022-01-10 12:59:40,079] jepsen node 172.23.107.228 - couchbase.util Creating hashtable dump on 172.23.107.228 INFO [2022-01-10 13:00:47,755] jepsen node 172.23.105.187 - couchbase.util Creating hashtable dump on 172.23.105.187 INFO [2022-01-10 13:02:57,627] jepsen node 172.23.107.228 - couchbase.util Collecting tcp packet capture INFO [2022-01-10 13:02:58,123] jepsen node 172.23.106.61 - couchbase.util Collecting tcp packet capture INFO [2022-01-10 13:02:59,427] jepsen node 172.23.107.227 - couchbase.util Collecting tcp packet capture INFO [2022-01-10 13:02:59,742] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/goxdcr_cprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/goxdcr_cprof.log INFO [2022-01-10 13:03:00,339] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/cbcollect_info.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/cbcollect_info.log INFO [2022-01-10 13:03:01,739] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/stats_snapshot/01FS2WM1NXT476108KW8SYZDCJ/chunks/000001 to cbcollect_info_ns_1@172.23.107.227_20220110-205355/stats_snapshot/01FS2WM1NXT476108KW8SYZDCJ/chunks/000001 INFO [2022-01-10 13:03:02,815] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/kv_trace.json to cbcollect_info_ns_1@172.23.107.228_20220110-205354/kv_trace.json INFO [2022-01-10 13:03:03,493] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.fts.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.fts.log INFO [2022-01-10 13:03:04,299] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/couchstore_local.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/couchstore_local.log INFO [2022-01-10 13:03:04,583] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_warn.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_warn.log INFO [2022-01-10 13:03:04,654] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/master_events.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/master_events.log INFO [2022-01-10 13:03:05,227] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/stats_snapshot/01FS2WM1VZZX2X3FKP044FNM84/meta.json to cbcollect_info_ns_1@172.23.107.228_20220110-205354/stats_snapshot/01FS2WM1VZZX2X3FKP044FNM84/meta.json INFO [2022-01-10 13:03:05,836] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.xdcr_target.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.xdcr_target.log INFO [2022-01-10 13:03:05,860] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/stats_snapshot/01FS2WM0SAPWGKBG1TM10H928G/index to cbcollect_info_ns_1@172.23.106.61_20220110-205354/stats_snapshot/01FS2WM0SAPWGKBG1TM10H928G/index INFO [2022-01-10 13:03:06,633] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/goxdcr_remote_clusters.json to cbcollect_info_ns_1@172.23.107.228_20220110-205354/goxdcr_remote_clusters.json INFO [2022-01-10 13:03:07,158] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/chronicle_logs.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/chronicle_logs.log INFO [2022-01-10 13:03:07,200] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.backup_service.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.backup_service.log INFO [2022-01-10 13:03:07,929] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/event_log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/event_log INFO [2022-01-10 13:03:08,508] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_error.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_error.log INFO [2022-01-10 13:03:08,544] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/systemd_journal.gz to cbcollect_info_ns_1@172.23.106.61_20220110-205354/systemd_journal.gz INFO [2022-01-10 13:03:09,258] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_dcpdebug.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_dcpdebug.log INFO [2022-01-10 13:03:09,829] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.metakv.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.metakv.log INFO [2022-01-10 13:03:10,604] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/Couchbase-172.23.107.228.pcap.1641847924.gz to Couchbase-172.23.107.228.pcap.1641847924.gz INFO [2022-01-10 13:03:11,166] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/kv_trace.json to cbcollect_info_ns_1@172.23.107.227_20220110-205355/kv_trace.json INFO [2022-01-10 13:03:12,233] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/projector_cprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/projector_cprof.log INFO [2022-01-10 13:03:12,530] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.projector.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.projector.log INFO [2022-01-10 13:03:12,694] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ddocs.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ddocs.log INFO [2022-01-10 13:03:13,613] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.xdcr_target.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.xdcr_target.log INFO [2022-01-10 13:03:13,867] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_shutdown.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_shutdown.log INFO [2022-01-10 13:03:14,076] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.json_rpc.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.json_rpc.log INFO [2022-01-10 13:03:14,934] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.babysitter.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.babysitter.log INFO [2022-01-10 13:03:15,193] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/goxdcr_replications.json to cbcollect_info_ns_1@172.23.107.228_20220110-205354/goxdcr_replications.json INFO [2022-01-10 13:03:15,361] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.http_access_internal.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.http_access_internal.log INFO [2022-01-10 13:03:16,281] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.reports.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.reports.log INFO [2022-01-10 13:03:16,537] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_access.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_access.log INFO [2022-01-10 13:03:16,692] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/goxdcr_remote_clusters.json to cbcollect_info_ns_1@172.23.107.227_20220110-205355/goxdcr_remote_clusters.json INFO [2022-01-10 13:03:17,618] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/Couchbase-172.23.106.61.pcap.1641847925.gz to Couchbase-172.23.106.61.pcap.1641847925.gz INFO [2022-01-10 13:03:17,848] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_debug.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_debug.log INFO [2022-01-10 13:03:18,010] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.debug.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.debug.log INFO [2022-01-10 13:03:19,156] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.error.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.error.log INFO [2022-01-10 13:03:19,377] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.babysitter.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.babysitter.log INFO [2022-01-10 13:03:19,558] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/event_log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/event_log INFO [2022-01-10 13:03:20,524] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.xdcr_target.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.xdcr_target.log INFO [2022-01-10 13:03:20,709] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ini.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ini.log INFO [2022-01-10 13:03:20,900] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.query.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.query.log INFO [2022-01-10 13:03:21,820] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.babysitter.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.babysitter.log INFO [2022-01-10 13:03:22,029] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.fts.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.fts.log INFO [2022-01-10 13:03:22,190] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.goxdcr.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.goxdcr.log INFO [2022-01-10 13:03:23,149] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.http_access.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.http_access.log INFO [2022-01-10 13:03:23,402] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/systemd_journal.gz to cbcollect_info_ns_1@172.23.107.227_20220110-205355/systemd_journal.gz INFO [2022-01-10 13:03:23,470] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/goxdcr_pprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/goxdcr_pprof.log INFO [2022-01-10 13:03:24,354] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.indexer.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.indexer.log INFO [2022-01-10 13:03:24,393] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/stats.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/stats.log INFO [2022-01-10 13:03:25,659] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/projector_cprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/projector_cprof.log INFO [2022-01-10 13:03:25,838] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/stats_snapshot/01FS2WM0SAPWGKBG1TM10H928G/meta.json to cbcollect_info_ns_1@172.23.106.61_20220110-205354/stats_snapshot/01FS2WM0SAPWGKBG1TM10H928G/meta.json INFO [2022-01-10 13:03:26,297] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_info.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_info.log INFO [2022-01-10 13:03:27,127] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.error.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.error.log INFO [2022-01-10 13:03:27,145] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/cbcollect_info.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/cbcollect_info.log INFO [2022-01-10 13:03:27,460] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.error.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.error.log INFO [2022-01-10 13:03:28,379] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.indexer_stats.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.indexer_stats.log INFO [2022-01-10 13:03:28,455] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.info.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.info.log INFO [2022-01-10 13:03:28,700] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/goxdcr_replications.json to cbcollect_info_ns_1@172.23.107.227_20220110-205355/goxdcr_replications.json INFO [2022-01-10 13:03:29,740] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.http_access_internal.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.http_access_internal.log INFO [2022-01-10 13:03:29,803] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_warn.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_warn.log INFO [2022-01-10 13:03:30,002] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_access.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_access.log INFO [2022-01-10 13:03:31,084] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/projector_mprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/projector_mprof.log INFO [2022-01-10 13:03:31,147] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ddocs.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ddocs.log INFO [2022-01-10 13:03:31,344] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/users.dets to cbcollect_info_ns_1@172.23.107.227_20220110-205355/users.dets INFO [2022-01-10 13:03:32,472] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_info.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_info.log INFO [2022-01-10 13:03:32,492] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.debug.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.debug.log INFO [2022-01-10 13:03:32,670] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/event_log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/event_log INFO [2022-01-10 13:03:33,767] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/mdocs.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/mdocs.log INFO [2022-01-10 13:03:33,941] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.eventing.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.eventing.log INFO [2022-01-10 13:03:34,003] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.views.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.views.log INFO [2022-01-10 13:03:35,103] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_error.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_error.log INFO [2022-01-10 13:03:35,279] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.ns_couchdb.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.ns_couchdb.log INFO [2022-01-10 13:03:35,378] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/cbcollect_info.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/cbcollect_info.log INFO [2022-01-10 13:03:36,432] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_log INFO [2022-01-10 13:03:36,626] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/goxdcr_mprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/goxdcr_mprof.log INFO [2022-01-10 13:03:36,699] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.prometheus.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.prometheus.log INFO [2022-01-10 13:03:37,743] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/chronicle_logs.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/chronicle_logs.log INFO [2022-01-10 13:03:37,927] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.views.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.views.log INFO [2022-01-10 13:03:37,990] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/couchbase.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/couchbase.log INFO [2022-01-10 13:03:39,085] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.ns_couchdb.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.ns_couchdb.log INFO [2022-01-10 13:03:39,295] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_error.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_error.log INFO [2022-01-10 13:03:39,535] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/stats_snapshot/01FS2WM1NXT476108KW8SYZDCJ/tombstones to cbcollect_info_ns_1@172.23.107.227_20220110-205355/stats_snapshot/01FS2WM1NXT476108KW8SYZDCJ/tombstones INFO [2022-01-10 13:03:40,457] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/stats_snapshot/01FS2WM0SAPWGKBG1TM10H928G/tombstones to cbcollect_info_ns_1@172.23.106.61_20220110-205354/stats_snapshot/01FS2WM0SAPWGKBG1TM10H928G/tombstones INFO [2022-01-10 13:03:40,636] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/stats_snapshot/01FS2WM1VZZX2X3FKP044FNM84/tombstones to cbcollect_info_ns_1@172.23.107.228_20220110-205354/stats_snapshot/01FS2WM1VZZX2X3FKP044FNM84/tombstones INFO [2022-01-10 13:03:40,867] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/goxdcr_cprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/goxdcr_cprof.log INFO [2022-01-10 13:03:41,799] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_access.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_access.log INFO [2022-01-10 13:03:41,918] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/couchbase.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/couchbase.log INFO [2022-01-10 13:03:42,176] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.eventing.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.eventing.log INFO [2022-01-10 13:03:43,123] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_cbas_debug.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_cbas_debug.log INFO [2022-01-10 13:03:43,483] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/rebalance_report_20220110T205241.json to cbcollect_info_ns_1@172.23.107.228_20220110-205354/rebalance_report_20220110T205241.json INFO [2022-01-10 13:03:43,508] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/goxdcr_mprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/goxdcr_mprof.log INFO [2022-01-10 13:03:44,447] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.views.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.views.log INFO [2022-01-10 13:03:44,811] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/stats_snapshot/01FS2WM1VZZX2X3FKP044FNM84/chunks/000001 to cbcollect_info_ns_1@172.23.107.228_20220110-205354/stats_snapshot/01FS2WM1VZZX2X3FKP044FNM84/chunks/000001 INFO [2022-01-10 13:03:44,847] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_dcpdebug.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_dcpdebug.log INFO [2022-01-10 13:03:45,827] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_trace.json to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_trace.json INFO [2022-01-10 13:03:46,185] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/stats_snapshot/01FS2WM1VZZX2X3FKP044FNM84/index to cbcollect_info_ns_1@172.23.107.228_20220110-205354/stats_snapshot/01FS2WM1VZZX2X3FKP044FNM84/index INFO [2022-01-10 13:03:46,199] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/projector_cprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/projector_cprof.log INFO [2022-01-10 13:03:47,175] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/couchstore_local.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/couchstore_local.log INFO [2022-01-10 13:03:47,519] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.info.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.info.log INFO [2022-01-10 13:03:47,522] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.goxdcr.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.goxdcr.log INFO [2022-01-10 13:03:48,535] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.metakv.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.metakv.log INFO [2022-01-10 13:03:48,876] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.indexer.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.indexer.log INFO [2022-01-10 13:03:48,888] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.stats.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.stats.log INFO [2022-01-10 13:03:49,872] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_shutdown.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_shutdown.log INFO [2022-01-10 13:03:50,191] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.reports.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.reports.log INFO [2022-01-10 13:03:50,225] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/chronicle_logs.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/chronicle_logs.log INFO [2022-01-10 13:03:51,235] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/kv_trace.json to cbcollect_info_ns_1@172.23.106.61_20220110-205354/kv_trace.json INFO [2022-01-10 13:03:51,552] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/Couchbase-172.23.107.227.pcap.1641847925.gz to Couchbase-172.23.107.227.pcap.1641847925.gz INFO [2022-01-10 13:03:51,595] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/systemd_journal.gz to cbcollect_info_ns_1@172.23.107.228_20220110-205354/systemd_journal.gz INFO [2022-01-10 13:03:52,623] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.projector.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.projector.log INFO [2022-01-10 13:03:53,298] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.mapreduce_errors.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.mapreduce_errors.log INFO [2022-01-10 13:03:53,980] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/goxdcr_cprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/goxdcr_cprof.log INFO [2022-01-10 13:03:54,649] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.couchdb.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.couchdb.log INFO [2022-01-10 13:03:55,192] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/syslog.tar.gz to cbcollect_info_ns_1@172.23.107.228_20220110-205354/syslog.tar.gz INFO [2022-01-10 13:03:55,323] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.debug.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.debug.log INFO [2022-01-10 13:03:56,014] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/syslog.tar.gz to cbcollect_info_ns_1@172.23.107.227_20220110-205355/syslog.tar.gz INFO [2022-01-10 13:03:56,665] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.couchdb.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.couchdb.log INFO [2022-01-10 13:03:56,758] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/couchbase.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/couchbase.log INFO [2022-01-10 13:03:57,486] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_log INFO [2022-01-10 13:03:57,982] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.fts.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.fts.log INFO [2022-01-10 13:03:58,192] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_warn.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_warn.log INFO [2022-01-10 13:03:58,476] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.ns_couchdb.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.ns_couchdb.log INFO [2022-01-10 13:03:59,271] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_log INFO [2022-01-10 13:03:59,398] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/stats_snapshot/01FS2WM0SAPWGKBG1TM10H928G/chunks/000001 to cbcollect_info_ns_1@172.23.106.61_20220110-205354/stats_snapshot/01FS2WM0SAPWGKBG1TM10H928G/chunks/000001 INFO [2022-01-10 13:03:59,758] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/mdocs.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/mdocs.log INFO [2022-01-10 13:04:00,570] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.indexer_stats.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.indexer_stats.log INFO [2022-01-10 13:04:00,707] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/users.dets to cbcollect_info_ns_1@172.23.106.61_20220110-205354/users.dets INFO [2022-01-10 13:04:01,101] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/hashtable_dump.txt to hashtable_dump.txt INFO [2022-01-10 13:04:01,923] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/hashtable_dump.txt to hashtable_dump.txt INFO [2022-01-10 13:04:02,059] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/syslog.tar.gz to cbcollect_info_ns_1@172.23.106.61_20220110-205354/syslog.tar.gz INFO [2022-01-10 13:04:02,483] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/projector_pprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/projector_pprof.log INFO [2022-01-10 13:04:03,369] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.reports.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.reports.log INFO [2022-01-10 13:04:03,596] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/diag.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/diag.log INFO [2022-01-10 13:04:03,841] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/stats.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/stats.log INFO [2022-01-10 13:04:04,701] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/master_events.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/master_events.log INFO [2022-01-10 13:04:05,081] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/projector_pprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/projector_pprof.log INFO [2022-01-10 13:04:05,332] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/couchstore_local.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/couchstore_local.log INFO [2022-01-10 13:04:06,030] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/users.dets to cbcollect_info_ns_1@172.23.107.228_20220110-205354/users.dets INFO [2022-01-10 13:04:06,436] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/hashtable_dump.txt to hashtable_dump.txt INFO [2022-01-10 13:04:06,649] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/stats_snapshot/01FS2WM1NXT476108KW8SYZDCJ/meta.json to cbcollect_info_ns_1@172.23.107.227_20220110-205355/stats_snapshot/01FS2WM1NXT476108KW8SYZDCJ/meta.json INFO [2022-01-10 13:04:07,358] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/goxdcr_pprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/goxdcr_pprof.log INFO [2022-01-10 13:04:07,850] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.json_rpc.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.json_rpc.log INFO [2022-01-10 13:04:07,922] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_cbas_debug.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_cbas_debug.log INFO [2022-01-10 13:04:08,708] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/diag.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/diag.log INFO [2022-01-10 13:04:09,172] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.eventing.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.eventing.log INFO [2022-01-10 13:04:09,386] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/goxdcr_pprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/goxdcr_pprof.log INFO [2022-01-10 13:04:10,163] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/projector_mprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/projector_mprof.log INFO [2022-01-10 13:04:10,507] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.mapreduce_errors.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.mapreduce_errors.log INFO [2022-01-10 13:04:10,765] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/diag.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/diag.log INFO [2022-01-10 13:04:11,563] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.backup_service.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.backup_service.log INFO [2022-01-10 13:04:11,906] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.indexer.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.indexer.log INFO [2022-01-10 13:04:12,287] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/.ignore to .ignore INFO [2022-01-10 13:04:12,861] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.http_access_internal.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.http_access_internal.log INFO [2022-01-10 13:04:13,083] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.stats.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.stats.log INFO [2022-01-10 13:04:13,395] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/rebalance_report_20220110T205241.json to cbcollect_info_ns_1@172.23.107.227_20220110-205355/rebalance_report_20220110T205241.json INFO [2022-01-10 13:04:14,131] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/.ignore to .ignore INFO [2022-01-10 13:04:14,436] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/.ignore to .ignore INFO [2022-01-10 13:04:14,720] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_trace.json to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_trace.json INFO [2022-01-10 13:04:15,448] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/mdocs.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/mdocs.log INFO [2022-01-10 13:04:15,750] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/goxdcr_mprof.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/goxdcr_mprof.log INFO [2022-01-10 13:04:16,051] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/stats_snapshot/01FS2WM1NXT476108KW8SYZDCJ/index to cbcollect_info_ns_1@172.23.107.227_20220110-205355/stats_snapshot/01FS2WM1NXT476108KW8SYZDCJ/index INFO [2022-01-10 13:04:16,504] jepsen node 172.23.105.187 - couchbase.util Collecting tcp packet capture INFO [2022-01-10 13:04:16,851] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.json_rpc.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.json_rpc.log INFO [2022-01-10 13:04:17,096] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/goxdcr_replications.json to cbcollect_info_ns_1@172.23.106.61_20220110-205354/goxdcr_replications.json INFO [2022-01-10 13:04:17,473] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.query.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.query.log INFO [2022-01-10 13:04:18,193] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/memcached.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/memcached.log INFO [2022-01-10 13:04:18,441] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.couchdb.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.couchdb.log INFO [2022-01-10 13:04:18,693] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/projector_mprof.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/projector_mprof.log INFO [2022-01-10 13:04:18,815] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/kv_trace.json to cbcollect_info_ns_1@172.23.105.187_20220110-205355/kv_trace.json INFO [2022-01-10 13:04:19,700] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_trace.json to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_trace.json INFO [2022-01-10 13:04:19,827] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ini.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ini.log INFO [2022-01-10 13:04:20,080] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.indexer_stats.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.indexer_stats.log INFO [2022-01-10 13:04:21,059] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.mapreduce_errors.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.mapreduce_errors.log INFO [2022-01-10 13:04:21,192] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/goxdcr_remote_clusters.json to cbcollect_info_ns_1@172.23.106.61_20220110-205354/goxdcr_remote_clusters.json INFO [2022-01-10 13:04:21,459] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.projector.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.projector.log INFO [2022-01-10 13:04:22,370] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.metakv.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.metakv.log INFO [2022-01-10 13:04:22,545] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.prometheus.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.prometheus.log INFO [2022-01-10 13:04:22,744] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.http_access.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.http_access.log INFO [2022-01-10 13:04:23,062] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_log INFO [2022-01-10 13:04:23,591] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_info.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_info.log INFO [2022-01-10 13:04:23,819] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/goxdcr_remote_clusters.json to cbcollect_info_ns_1@172.23.105.187_20220110-205355/goxdcr_remote_clusters.json INFO [2022-01-10 13:04:23,891] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.info.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.info.log INFO [2022-01-10 13:04:24,078] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.stats.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.stats.log INFO [2022-01-10 13:04:24,516] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.error.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.error.log INFO [2022-01-10 13:04:24,923] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.prometheus.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.prometheus.log INFO [2022-01-10 13:04:25,230] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_debug.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_debug.log INFO [2022-01-10 13:04:25,242] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/couchbase.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/couchbase.log INFO [2022-01-10 13:04:25,363] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_shutdown.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_shutdown.log INFO [2022-01-10 13:04:26,245] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/stats_snapshot/01FS2WMK7ZRGHQSHSC2JD6MRMD/tombstones to cbcollect_info_ns_1@172.23.105.187_20220110-205355/stats_snapshot/01FS2WMK7ZRGHQSHSC2JD6MRMD/tombstones INFO [2022-01-10 13:04:26,410] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ini.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ini.log INFO [2022-01-10 13:04:26,734] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/memcached.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/memcached.log INFO [2022-01-10 13:04:26,747] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ddocs.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ddocs.log INFO [2022-01-10 13:04:27,071] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/chronicle_logs.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/chronicle_logs.log INFO [2022-01-10 13:04:27,801] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/stats.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/stats.log INFO [2022-01-10 13:04:27,819] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/projector_mprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/projector_mprof.log INFO [2022-01-10 13:04:28,055] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/memcached.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/memcached.log INFO [2022-01-10 13:04:28,576] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/stats_snapshot/01FS2WMK7ZRGHQSHSC2JD6MRMD/index to cbcollect_info_ns_1@172.23.105.187_20220110-205355/stats_snapshot/01FS2WMK7ZRGHQSHSC2JD6MRMD/index INFO [2022-01-10 13:04:28,663] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.goxdcr.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.goxdcr.log INFO [2022-01-10 13:04:29,275] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_info.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_info.log INFO [2022-01-10 13:04:29,319] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/projector_pprof.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/projector_pprof.log INFO [2022-01-10 13:04:29,932] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/rebalance_report_20220110T205241.json to cbcollect_info_ns_1@172.23.106.61_20220110-205354/rebalance_report_20220110T205241.json INFO [2022-01-10 13:04:29,994] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/stats_snapshot/01FS2WMK7ZRGHQSHSC2JD6MRMD/meta.json to cbcollect_info_ns_1@172.23.105.187_20220110-205355/stats_snapshot/01FS2WMK7ZRGHQSHSC2JD6MRMD/meta.json INFO [2022-01-10 13:04:30,032] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_debug.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.analytics_debug.log INFO [2022-01-10 13:04:30,672] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.query.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.query.log INFO [2022-01-10 13:04:30,795] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.couchdb.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.couchdb.log INFO [2022-01-10 13:04:31,238] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_dcpdebug.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.analytics_dcpdebug.log INFO [2022-01-10 13:04:31,368] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/master_events.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/master_events.log INFO [2022-01-10 13:04:31,481] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/rebalance_report_20220110T205241.json to cbcollect_info_ns_1@172.23.105.187_20220110-205355/rebalance_report_20220110T205241.json INFO [2022-01-10 13:04:32,004] jepsen node 172.23.107.228 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_cbas_debug.log to cbcollect_info_ns_1@172.23.107.228_20220110-205354/ns_server.analytics_cbas_debug.log INFO [2022-01-10 13:04:32,171] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.prometheus.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.prometheus.log INFO [2022-01-10 13:04:32,558] jepsen node 172.23.106.61 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.http_access.log to cbcollect_info_ns_1@172.23.106.61_20220110-205354/ns_server.http_access.log INFO [2022-01-10 13:04:32,690] jepsen node 172.23.107.227 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.backup_service.log to cbcollect_info_ns_1@172.23.107.227_20220110-205355/ns_server.backup_service.log INFO [2022-01-10 13:04:32,895] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.goxdcr.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.goxdcr.log INFO [2022-01-10 13:04:33,624] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/goxdcr_cprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/goxdcr_cprof.log INFO [2022-01-10 13:04:34,319] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.projector.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.projector.log INFO [2022-01-10 13:04:35,011] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/cbcollect_info.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/cbcollect_info.log INFO [2022-01-10 13:04:35,695] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_cbas_debug.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_cbas_debug.log INFO [2022-01-10 13:04:36,362] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.metakv.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.metakv.log INFO [2022-01-10 13:04:37,078] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.http_access_internal.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.http_access_internal.log INFO [2022-01-10 13:04:37,786] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/systemd_journal.gz to cbcollect_info_ns_1@172.23.105.187_20220110-205355/systemd_journal.gz INFO [2022-01-10 13:04:40,636] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/syslog.tar.gz to cbcollect_info_ns_1@172.23.105.187_20220110-205355/syslog.tar.gz INFO [2022-01-10 13:04:41,545] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_access.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_access.log INFO [2022-01-10 13:04:42,312] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.babysitter.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.babysitter.log INFO [2022-01-10 13:04:43,098] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/users.dets to cbcollect_info_ns_1@172.23.105.187_20220110-205355/users.dets INFO [2022-01-10 13:04:43,790] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.indexer.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.indexer.log INFO [2022-01-10 13:04:44,503] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.http_access.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.http_access.log INFO [2022-01-10 13:04:45,275] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.fts.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.fts.log INFO [2022-01-10 13:04:45,957] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ini.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ini.log INFO [2022-01-10 13:04:46,644] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.eventing.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.eventing.log INFO [2022-01-10 13:04:47,317] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_shutdown.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_shutdown.log INFO [2022-01-10 13:04:48,010] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/event_log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/event_log INFO [2022-01-10 13:04:48,703] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/stats_snapshot/01FS2WMK7ZRGHQSHSC2JD6MRMD/chunks/000001 to cbcollect_info_ns_1@172.23.105.187_20220110-205355/stats_snapshot/01FS2WMK7ZRGHQSHSC2JD6MRMD/chunks/000001 INFO [2022-01-10 13:04:49,376] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/master_events.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/master_events.log INFO [2022-01-10 13:04:50,078] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/goxdcr_mprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/goxdcr_mprof.log INFO [2022-01-10 13:04:50,756] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.mapreduce_errors.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.mapreduce_errors.log INFO [2022-01-10 13:04:51,443] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.views.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.views.log INFO [2022-01-10 13:04:52,129] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_warn.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_warn.log INFO [2022-01-10 13:04:52,810] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/projector_cprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/projector_cprof.log INFO [2022-01-10 13:04:53,482] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/hashtable_dump.txt to hashtable_dump.txt INFO [2022-01-10 13:04:54,314] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.indexer_stats.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.indexer_stats.log INFO [2022-01-10 13:04:55,030] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.reports.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.reports.log INFO [2022-01-10 13:04:55,693] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_error.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_error.log INFO [2022-01-10 13:04:56,349] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/Couchbase-172.23.105.187.pcap.1641847927.gz to Couchbase-172.23.105.187.pcap.1641847927.gz INFO [2022-01-10 13:04:57,869] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.info.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.info.log INFO [2022-01-10 13:04:58,533] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/stats.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/stats.log INFO [2022-01-10 13:04:59,472] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.xdcr_target.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.xdcr_target.log INFO [2022-01-10 13:05:00,246] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/.ignore to .ignore INFO [2022-01-10 13:05:00,910] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.ns_couchdb.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.ns_couchdb.log INFO [2022-01-10 13:05:01,629] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_debug.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_debug.log INFO [2022-01-10 13:05:02,378] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_dcpdebug.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_dcpdebug.log INFO [2022-01-10 13:05:03,066] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_trace.json to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.analytics_trace.json INFO [2022-01-10 13:05:03,801] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.stats.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.stats.log INFO [2022-01-10 13:05:04,472] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.query.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.query.log INFO [2022-01-10 13:05:05,153] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.backup_service.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.backup_service.log INFO [2022-01-10 13:05:05,834] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/goxdcr_replications.json to cbcollect_info_ns_1@172.23.105.187_20220110-205355/goxdcr_replications.json INFO [2022-01-10 13:05:06,579] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/projector_pprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/projector_pprof.log INFO [2022-01-10 13:05:07,280] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/memcached.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/memcached.log INFO [2022-01-10 13:05:44,460] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/diag.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/diag.log INFO [2022-01-10 13:05:45,399] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/goxdcr_pprof.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/goxdcr_pprof.log INFO [2022-01-10 13:05:46,122] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.debug.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.debug.log INFO [2022-01-10 13:05:46,872] jepsen node 172.23.105.187 - jepsen.core downloading /tmp/jepsen-logs/cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.json_rpc.log to cbcollect_info_ns_1@172.23.105.187_20220110-205355/ns_server.json_rpc.log INFO [2022-01-10 13:05:47,438] jepsen test runner - jepsen.core Snarfing log files WARN [2022-01-10 13:05:47,440] jepsen node 172.23.105.187 - couchbase.core Ignoring duplicate log collection request WARN [2022-01-10 13:05:47,440] jepsen node 172.23.107.227 - couchbase.core Ignoring duplicate log collection request WARN [2022-01-10 13:05:47,440] jepsen node 172.23.106.61 - couchbase.core Ignoring duplicate log collection request WARN [2022-01-10 13:05:47,440] jepsen node 172.23.107.228 - couchbase.core Ignoring duplicate log collection request INFO [2022-01-10 13:05:47,443] jepsen node 172.23.105.187 - couchbase.util Tearing down couchbase node INFO [2022-01-10 13:05:47,443] jepsen node 172.23.107.228 - couchbase.util Tearing down couchbase node INFO [2022-01-10 13:05:47,443] jepsen node 172.23.106.61 - couchbase.util Tearing down couchbase node INFO [2022-01-10 13:05:47,443] jepsen node 172.23.107.227 - couchbase.util Tearing down couchbase node INFO [2022-01-10 13:05:50,963] jepsen node 172.23.107.227 - couchbase.util Teardown Complete INFO [2022-01-10 13:05:51,168] jepsen node 172.23.106.61 - couchbase.util Teardown Complete INFO [2022-01-10 13:05:51,373] jepsen node 172.23.107.228 - couchbase.util Teardown Complete INFO [2022-01-10 13:07:21,251] jepsen node 172.23.105.187 - couchbase.util Teardown Complete INFO [2022-01-10 13:07:21,254] jepsen test runner - jepsen.core Analyzing... INFO [2022-01-10 13:07:21,678] clojure-agent-send-off-pool-108 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:21,748] clojure-agent-send-off-pool-118 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:21,749] clojure-agent-send-off-pool-112 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:21,759] clojure-agent-send-off-pool-110 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:21,852] clojure-agent-send-off-pool-122 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:21,919] clojure-agent-send-off-pool-121 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:21,965] clojure-agent-send-off-pool-120 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:22,237] Thread-176 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:22,289] Thread-179 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:22,322] Thread-186 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:22,323] Thread-173 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:22,330] Thread-188 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:22,344] Thread-177 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:22,394] Thread-175 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:22,644] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/27/results.edn INFO [2022-01-10 13:07:22,674] clojure-agent-send-pool-8 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/7/results.edn INFO [2022-01-10 13:07:22,748] clojure-agent-send-pool-0 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/20/results.edn INFO [2022-01-10 13:07:22,809] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/24/results.edn INFO [2022-01-10 13:07:22,813] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/27/history.edn INFO [2022-01-10 13:07:22,817] clojure-agent-send-pool-1 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/4/results.edn INFO [2022-01-10 13:07:22,830] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/24/history.edn INFO [2022-01-10 13:07:22,834] clojure-agent-send-pool-8 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/7/history.edn INFO [2022-01-10 13:07:22,838] clojure-agent-send-pool-6 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/13/results.edn INFO [2022-01-10 13:07:22,889] clojure-agent-send-pool-0 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/20/history.edn INFO [2022-01-10 13:07:22,897] clojure-agent-send-pool-2 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/0/results.edn INFO [2022-01-10 13:07:22,915] clojure-agent-send-pool-9 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/1/results.edn INFO [2022-01-10 13:07:22,921] clojure-agent-send-off-pool-116 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:22,932] clojure-agent-send-off-pool-105 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:22,992] clojure-agent-send-pool-1 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/4/history.edn INFO [2022-01-10 13:07:23,041] clojure-agent-send-pool-2 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/0/history.edn INFO [2022-01-10 13:07:23,042] clojure-agent-send-off-pool-110 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,048] clojure-agent-send-pool-6 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/13/history.edn INFO [2022-01-10 13:07:23,084] clojure-agent-send-pool-9 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/1/history.edn INFO [2022-01-10 13:07:23,108] Thread-197 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,136] clojure-agent-send-off-pool-120 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,212] clojure-agent-send-off-pool-118 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,239] Thread-199 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,305] Thread-201 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,308] clojure-agent-send-off-pool-130 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,308] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/22/results.edn INFO [2022-01-10 13:07:23,364] Thread-205 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,413] Thread-212 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,417] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/22/history.edn INFO [2022-01-10 13:07:23,446] Thread-210 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,495] clojure-agent-send-off-pool-110 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,560] clojure-agent-send-pool-5 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/15/results.edn INFO [2022-01-10 13:07:23,568] clojure-agent-send-pool-5 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/15/history.edn INFO [2022-01-10 13:07:23,601] clojure-agent-send-pool-1 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/25/results.edn INFO [2022-01-10 13:07:23,610] clojure-agent-send-pool-4 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/21/results.edn INFO [2022-01-10 13:07:23,619] clojure-agent-send-pool-4 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/21/history.edn INFO [2022-01-10 13:07:23,632] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/29/results.edn INFO [2022-01-10 13:07:23,655] Thread-216 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,668] clojure-agent-send-pool-0 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/28/results.edn INFO [2022-01-10 13:07:23,685] clojure-agent-send-off-pool-119 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,752] clojure-agent-send-off-pool-120 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,794] Thread-218 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,808] clojure-agent-send-pool-1 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/25/history.edn INFO [2022-01-10 13:07:23,840] clojure-agent-send-pool-2 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/17/results.edn INFO [2022-01-10 13:07:23,868] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/2/results.edn INFO [2022-01-10 13:07:23,876] clojure-agent-send-pool-0 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/28/history.edn INFO [2022-01-10 13:07:23,886] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/29/history.edn INFO [2022-01-10 13:07:23,886] Thread-220 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,902] clojure-agent-send-pool-9 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/12/results.edn INFO [2022-01-10 13:07:23,908] clojure-agent-send-off-pool-113 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,930] clojure-agent-send-pool-2 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/17/history.edn INFO [2022-01-10 13:07:23,951] clojure-agent-send-off-pool-133 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:23,954] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/2/history.edn INFO [2022-01-10 13:07:23,966] clojure-agent-send-pool-9 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/12/history.edn INFO [2022-01-10 13:07:23,985] clojure-agent-send-pool-8 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/6/results.edn INFO [2022-01-10 13:07:23,991] clojure-agent-send-pool-8 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/6/history.edn INFO [2022-01-10 13:07:23,997] clojure-agent-send-off-pool-129 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:24,023] Thread-224 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:24,033] clojure-agent-send-off-pool-135 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:24,057] clojure-agent-send-pool-5 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/23/results.edn INFO [2022-01-10 13:07:24,078] Thread-232 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:24,079] Thread-226 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:24,108] clojure-agent-send-pool-5 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/23/history.edn INFO [2022-01-10 13:07:24,153] Thread-236 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:24,203] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/5/results.edn INFO [2022-01-10 13:07:24,233] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/5/history.edn INFO [2022-01-10 13:07:24,241] clojure-agent-send-pool-1 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/11/results.edn INFO [2022-01-10 13:07:24,248] clojure-agent-send-off-pool-142 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:24,284] clojure-agent-send-pool-6 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/3/results.edn INFO [2022-01-10 13:07:24,290] clojure-agent-send-pool-6 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/3/history.edn INFO [2022-01-10 13:07:24,301] Thread-243 - knossos.model.memo More than 1024 reachable models; not memoizing models for this search INFO [2022-01-10 13:07:24,306] clojure-agent-send-pool-1 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/11/history.edn INFO [2022-01-10 13:07:24,380] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/26/results.edn INFO [2022-01-10 13:07:24,414] clojure-agent-send-pool-2 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/14/results.edn INFO [2022-01-10 13:07:24,450] clojure-agent-send-pool-9 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/16/results.edn INFO [2022-01-10 13:07:24,486] clojure-agent-send-pool-2 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/14/history.edn INFO [2022-01-10 13:07:24,494] clojure-agent-send-pool-9 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/16/history.edn INFO [2022-01-10 13:07:24,498] clojure-agent-send-pool-3 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/26/history.edn INFO [2022-01-10 13:07:24,632] clojure-agent-send-pool-8 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/10/results.edn INFO [2022-01-10 13:07:24,638] clojure-agent-send-pool-8 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/10/history.edn INFO [2022-01-10 13:07:24,763] clojure-agent-send-pool-5 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/18/results.edn INFO [2022-01-10 13:07:24,770] clojure-agent-send-pool-5 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/18/history.edn INFO [2022-01-10 13:07:24,783] clojure-agent-send-pool-0 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/9/results.edn INFO [2022-01-10 13:07:24,790] clojure-agent-send-pool-0 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/9/history.edn INFO [2022-01-10 13:07:25,270] clojure-agent-send-pool-4 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/19/results.edn INFO [2022-01-10 13:07:25,277] clojure-agent-send-pool-4 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/19/history.edn INFO [2022-01-10 13:07:25,436] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/8/results.edn INFO [2022-01-10 13:07:25,442] clojure-agent-send-pool-7 - jepsen.independent Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/independent/8/history.edn INFO [2022-01-10 13:07:25,443] jepsen test runner - jepsen.core Analysis complete INFO [2022-01-10 13:07:25,445] jepsen fressian - couchbase.core Not writing fressian INFO [2022-01-10 13:07:26,164] jepsen results - jepsen.store Wrote /mnt/data-store/jenkins/workspace/kv-engine-neo-jepsen-nightly/store/Couchbase/20220110T125017.000-0800/results.edn INFO [2022-01-10 13:07:26,692] jepsen test runner - jepsen.core {:indep {:valid? true, :results {0 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 12}, :final-paths (), :configs ()}, :valid? true}, 7 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 10}, :final-paths (), :configs ()}, :valid? true}, 20 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 37}, :final-paths (), :configs ()}, :valid? true}, 27 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 46}, :final-paths (), :configs ()}, :valid? true}, 1 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 11}, :final-paths (), :configs ()}, :valid? true}, 24 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 9}, :final-paths (), :configs ()}, :valid? true}, 4 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 46}, :final-paths (), :configs ()}, :valid? true}, 15 {:timeline {:valid? true}, :linear {:valid? true, :configs ({:model #knossos.model.CASRegister{:value 10}, :last-op {:process 316, :type :ok, :f :read, :value 10, :index 31167, :time 47962222805, :cas 1641847982539210752}, :pending [{:persist-to 0, :index 5699, :value 35, :time 17903486891, :process 45, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 5722, :value 26, :time 17921939573, :process 46, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 7575, :value 45, :time 19573262548, :process 47, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14269, :value 3, :time 27915492608, :process 135, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14316, :value 44, :time 27941093576, :process 136, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14976, :value 22, :time 28579609778, :process 137, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 29716, :value 7, :time 46604300596, :process 225, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22762, :value 2, :time 37950164003, :process 226, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 23369, :value 26, :time 38590988428, :process 227, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 38304, :value 44, :time 55620148003, :process 315, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 31168, :value 16, :time 47962238000, :process 316, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 31942, :value 38, :time 48600289031, :process 317, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 42208, :value 13, :time 59980861435, :process 406, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 40704, :value 23, :time 58603487293, :process 407, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 10}, :last-op {:process 316, :type :ok, :f :read, :value 10, :index 31167, :time 47962222805, :cas 1641847982539210752}, :pending [{:persist-to 0, :index 5722, :value 26, :time 17921939573, :process 46, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 7575, :value 45, :time 19573262548, :process 47, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14269, :value 3, :time 27915492608, :process 135, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14316, :value 44, :time 27941093576, :process 136, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14976, :value 22, :time 28579609778, :process 137, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 29716, :value 7, :time 46604300596, :process 225, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22762, :value 2, :time 37950164003, :process 226, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 23369, :value 26, :time 38590988428, :process 227, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 38304, :value 44, :time 55620148003, :process 315, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 31168, :value 16, :time 47962238000, :process 316, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 31942, :value 38, :time 48600289031, :process 317, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 42208, :value 13, :time 59980861435, :process 406, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 40704, :value 23, :time 58603487293, :process 407, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 10}, :last-op {:process 316, :type :ok, :f :read, :value 10, :index 31167, :time 47962222805, :cas 1641847982539210752}, :pending [{:persist-to 0, :index 5699, :value 35, :time 17903486891, :process 45, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 7575, :value 45, :time 19573262548, :process 47, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14269, :value 3, :time 27915492608, :process 135, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14316, :value 44, :time 27941093576, :process 136, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14976, :value 22, :time 28579609778, :process 137, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 29716, :value 7, :time 46604300596, :process 225, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22762, :value 2, :time 37950164003, :process 226, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 23369, :value 26, :time 38590988428, :process 227, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 38304, :value 44, :time 55620148003, :process 315, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 31168, :value 16, :time 47962238000, :process 316, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 31942, :value 38, :time 48600289031, :process 317, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 42208, :value 13, :time 59980861435, :process 406, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 40704, :value 23, :time 58603487293, :process 407, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 10}, :last-op {:process 316, :type :ok, :f :read, :value 10, :index 31167, :time 47962222805, :cas 1641847982539210752}, :pending [{:persist-to 0, :index 7575, :value 45, :time 19573262548, :process 47, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14269, :value 3, :time 27915492608, :process 135, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14316, :value 44, :time 27941093576, :process 136, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14976, :value 22, :time 28579609778, :process 137, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 29716, :value 7, :time 46604300596, :process 225, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22762, :value 2, :time 37950164003, :process 226, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 23369, :value 26, :time 38590988428, :process 227, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 38304, :value 44, :time 55620148003, :process 315, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 31168, :value 16, :time 47962238000, :process 316, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 31942, :value 38, :time 48600289031, :process 317, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 42208, :value 13, :time 59980861435, :process 406, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 40704, :value 23, :time 58603487293, :process 407, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]}), :final-paths ()}, :valid? true}, 21 {:timeline {:valid? true}, :linear {:valid? true, :configs ({:model #knossos.model.CASRegister{:value 19}, :last-op {:process 153, :type :ok, :f :read, :value 19, :index 14897, :time 28513413540, :cas 1641847982510637056}, :pending [{:persist-to 0, :index 5925, :value 34, :time 18074339895, :process 63, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 7535, :value 4, :time 19543018765, :process 65, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14898, :value 4, :time 28513433375, :process 153, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14561, :value 41, :time 28106108232, :process 154, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14957, :value 26, :time 28565423722, :process 155, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22493, :value 23, :time 37578183625, :process 243, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22938, :value 25, :time 38112452897, :process 244, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 23343, :value 23, :time 38574319136, :process 245, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 33474, :value 16, :time 50613984349, :process 333, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 38696, :value 38, :time 56165157857, :process 334, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 31922, :value 0, :time 48580031006, :process 335, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 40695, :value 14, :time 58590917735, :process 425, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 19}, :last-op {:process 154, :type :ok, :f :read, :value 19, :index 14556, :time 28104796005, :cas 1641847982510637056}, :pending [{:persist-to 0, :index 7535, :value 4, :time 19543018765, :process 65, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14898, :value 4, :time 28513433375, :process 153, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14561, :value 41, :time 28106108232, :process 154, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14957, :value 26, :time 28565423722, :process 155, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22493, :value 23, :time 37578183625, :process 243, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22938, :value 25, :time 38112452897, :process 244, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 23343, :value 23, :time 38574319136, :process 245, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 33474, :value 16, :time 50613984349, :process 333, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 38696, :value 38, :time 56165157857, :process 334, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 31922, :value 0, :time 48580031006, :process 335, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 40695, :value 14, :time 58590917735, :process 425, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 19}, :last-op {:process 153, :type :ok, :f :read, :value 19, :index 14897, :time 28513413540, :cas 1641847982510637056}, :pending [{:persist-to 0, :index 5962, :value 47, :time 18092301120, :process 64, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 7535, :value 4, :time 19543018765, :process 65, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14898, :value 4, :time 28513433375, :process 153, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14561, :value 41, :time 28106108232, :process 154, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14957, :value 26, :time 28565423722, :process 155, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22493, :value 23, :time 37578183625, :process 243, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22938, :value 25, :time 38112452897, :process 244, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 23343, :value 23, :time 38574319136, :process 245, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 33474, :value 16, :time 50613984349, :process 333, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 38696, :value 38, :time 56165157857, :process 334, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 31922, :value 0, :time 48580031006, :process 335, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 40695, :value 14, :time 58590917735, :process 425, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 19}, :last-op {:process 154, :type :ok, :f :read, :value 19, :index 14556, :time 28104796005, :cas 1641847982510637056}, :pending [{:persist-to 0, :index 5925, :value 34, :time 18074339895, :process 63, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 5962, :value 47, :time 18092301120, :process 64, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 7535, :value 4, :time 19543018765, :process 65, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14898, :value 4, :time 28513433375, :process 153, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14561, :value 41, :time 28106108232, :process 154, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14957, :value 26, :time 28565423722, :process 155, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22493, :value 23, :time 37578183625, :process 243, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22938, :value 25, :time 38112452897, :process 244, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 23343, :value 23, :time 38574319136, :process 245, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 33474, :value 16, :time 50613984349, :process 333, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 38696, :value 38, :time 56165157857, :process 334, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 31922, :value 0, :time 48580031006, :process 335, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 40695, :value 14, :time 58590917735, :process 425, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]}), :final-paths ()}, :valid? true}, 13 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 30}, :final-paths (), :configs ()}, :valid? true}, 22 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 48}, :final-paths (), :configs ()}, :valid? true}, 29 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 19}, :final-paths (), :configs ()}, :valid? true}, 6 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 10}, :final-paths (), :configs ()}, :valid? true}, 28 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 46}, :final-paths (), :configs ()}, :valid? true}, 25 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 25}, :final-paths (), :configs ()}, :valid? true}, 17 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 21}, :final-paths (), :configs ()}, :valid? true}, 3 {:timeline {:valid? true}, :linear {:valid? true, :configs ({:model #knossos.model.CASRegister{:value 30}, :last-op {:process 191, :type :ok, :f :read, :value 30, :index 22623, :time 37717014798, :cas 1641847982375370752}, :pending [{:persist-to 0, :index 5709, :value 35, :time 17912016030, :process 9, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 5119, :value 25, :time 17247773239, :process 10, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 7502, :value 41, :time 19506347269, :process 11, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14942, :value 29, :time 28548730684, :process 99, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 15650, :value 35, :time 29284640660, :process 100, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14906, :value 33, :time 28516728076, :process 101, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 24896, :value 39, :time 40574586823, :process 189, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 24030, :value 18, :time 39293123263, :process 190, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22624, :value 13, :time 37717034834, :process 191, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 33472, :value 9, :time 50582388021, :process 279, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 32396, :value 5, :time 49303344827, :process 280, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 32754, :value 26, :time 49733805187, :process 281, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 42000, :value 19, :time 59743128344, :process 371, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 30}, :last-op {:process 191, :type :ok, :f :read, :value 30, :index 22623, :time 37717014798, :cas 1641847982375370752}, :pending [{:persist-to 0, :index 7502, :value 41, :time 19506347269, :process 11, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14942, :value 29, :time 28548730684, :process 99, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 15650, :value 35, :time 29284640660, :process 100, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14906, :value 33, :time 28516728076, :process 101, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 24896, :value 39, :time 40574586823, :process 189, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 24030, :value 18, :time 39293123263, :process 190, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22624, :value 13, :time 37717034834, :process 191, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 33472, :value 9, :time 50582388021, :process 279, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 32396, :value 5, :time 49303344827, :process 280, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 32754, :value 26, :time 49733805187, :process 281, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 42000, :value 19, :time 59743128344, :process 371, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 30}, :last-op {:process 191, :type :ok, :f :read, :value 30, :index 22623, :time 37717014798, :cas 1641847982375370752}, :pending [{:persist-to 0, :index 5119, :value 25, :time 17247773239, :process 10, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 7502, :value 41, :time 19506347269, :process 11, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14942, :value 29, :time 28548730684, :process 99, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 15650, :value 35, :time 29284640660, :process 100, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14906, :value 33, :time 28516728076, :process 101, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 24896, :value 39, :time 40574586823, :process 189, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 24030, :value 18, :time 39293123263, :process 190, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22624, :value 13, :time 37717034834, :process 191, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 33472, :value 9, :time 50582388021, :process 279, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 32396, :value 5, :time 49303344827, :process 280, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 32754, :value 26, :time 49733805187, :process 281, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 42000, :value 19, :time 59743128344, :process 371, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 30}, :last-op {:process 191, :type :ok, :f :read, :value 30, :index 22623, :time 37717014798, :cas 1641847982375370752}, :pending [{:persist-to 0, :index 5709, :value 35, :time 17912016030, :process 9, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 7502, :value 41, :time 19506347269, :process 11, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14942, :value 29, :time 28548730684, :process 99, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 15650, :value 35, :time 29284640660, :process 100, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14906, :value 33, :time 28516728076, :process 101, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 24896, :value 39, :time 40574586823, :process 189, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 24030, :value 18, :time 39293123263, :process 190, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22624, :value 13, :time 37717034834, :process 191, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 33472, :value 9, :time 50582388021, :process 279, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 32396, :value 5, :time 49303344827, :process 280, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 32754, :value 26, :time 49733805187, :process 281, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 42000, :value 19, :time 59743128344, :process 371, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]}), :final-paths ()}, :valid? true}, 12 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 43}, :final-paths (), :configs ()}, :valid? true}, 2 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 40}, :final-paths (), :configs ()}, :valid? true}, 23 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 28}, :final-paths (), :configs ()}, :valid? true}, 19 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 39}, :final-paths (), :configs ()}, :valid? true}, 11 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 1}, :final-paths (), :configs ()}, :valid? true}, 9 {:timeline {:valid? true}, :linear {:valid? true, :configs ({:model #knossos.model.CASRegister{:value 1}, :last-op {:process 297, :type :ok, :f :read, :value 1, :index 29799, :time 46781465234, :cas 1641847982545174528}, :pending [{:persist-to 0, :index 7652, :value 47, :time 19690420279, :process 29, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14972, :value 22, :time 28576369737, :process 117, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14304, :value 42, :time 27934167264, :process 118, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 15080, :value 23, :time 28704131089, :process 119, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22630, :value 7, :time 37762633849, :process 207, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22752, :value 38, :time 37945025183, :process 208, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 24998, :value 21, :time 40721557003, :process 209, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 29800, :value 5, :time 46781484938, :process 297, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 32872, :value 37, :time 49965082631, :process 298, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 37374, :value 21, :time 54753072085, :process 299, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 38428, :value 24, :time 55793969446, :process 387, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 1}, :last-op {:process 297, :type :ok, :f :read, :value 1, :index 29799, :time 46781465234, :cas 1641847982545174528}, :pending [{:persist-to 0, :index 6449, :value 26, :time 18564613986, :process 27, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 7652, :value 47, :time 19690420279, :process 29, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14972, :value 22, :time 28576369737, :process 117, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14304, :value 42, :time 27934167264, :process 118, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 15080, :value 23, :time 28704131089, :process 119, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22630, :value 7, :time 37762633849, :process 207, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22752, :value 38, :time 37945025183, :process 208, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 24998, :value 21, :time 40721557003, :process 209, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 29800, :value 5, :time 46781484938, :process 297, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 32872, :value 37, :time 49965082631, :process 298, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 37374, :value 21, :time 54753072085, :process 299, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 38428, :value 24, :time 55793969446, :process 387, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 1}, :last-op {:process 297, :type :ok, :f :read, :value 1, :index 29799, :time 46781465234, :cas 1641847982545174528}, :pending [{:persist-to 0, :index 5721, :value 24, :time 17920789042, :process 28, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 7652, :value 47, :time 19690420279, :process 29, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14972, :value 22, :time 28576369737, :process 117, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14304, :value 42, :time 27934167264, :process 118, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 15080, :value 23, :time 28704131089, :process 119, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22630, :value 7, :time 37762633849, :process 207, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22752, :value 38, :time 37945025183, :process 208, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 24998, :value 21, :time 40721557003, :process 209, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 29800, :value 5, :time 46781484938, :process 297, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 32872, :value 37, :time 49965082631, :process 298, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 37374, :value 21, :time 54753072085, :process 299, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 38428, :value 24, :time 55793969446, :process 387, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]} {:model #knossos.model.CASRegister{:value 1}, :last-op {:process 297, :type :ok, :f :read, :value 1, :index 29799, :time 46781465234, :cas 1641847982545174528}, :pending [{:persist-to 0, :index 6449, :value 26, :time 18564613986, :process 27, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 5721, :value 24, :time 17920789042, :process 28, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 7652, :value 47, :time 19690420279, :process 29, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14972, :value 22, :time 28576369737, :process 117, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 14304, :value 42, :time 27934167264, :process 118, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 15080, :value 23, :time 28704131089, :process 119, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22630, :value 7, :time 37762633849, :process 207, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 22752, :value 38, :time 37945025183, :process 208, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 24998, :value 21, :time 40721557003, :process 209, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 29800, :value 5, :time 46781484938, :process 297, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 32872, :value 37, :time 49965082631, :process 298, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 37374, :value 21, :time 54753072085, :process 299, :replicate-to 0, :type :invoke, :f :write, :durability-level 3} {:persist-to 0, :index 38428, :value 24, :time 55793969446, :process 387, :replicate-to 0, :type :invoke, :f :write, :durability-level 3}]}), :final-paths ()}, :valid? true}, 5 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 23}, :final-paths (), :configs ()}, :valid? true}, 14 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 7}, :final-paths (), :configs ()}, :valid? true}, 26 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 42}, :final-paths (), :configs ()}, :valid? true}, 16 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 13}, :final-paths (), :configs ()}, :valid? true}, 10 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 24}, :final-paths (), :configs ()}, :valid? true}, 18 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 29}, :final-paths (), :configs ()}, :valid? true}, 8 {:timeline {:valid? true}, :linear {:valid? true, :model #knossos.model.CASRegister{:value 13}, :final-paths (), :configs ()}, :valid? true}}, :failures []}, :sanity {:valid? :unknown, :error "Test aborted due to nemesis exception"}, :valid? :unknown} Errors occurred during analysis, but no anomalies found. ಠ~ಠ Jepsen failed with unknown failure Test crashed! ###### Current Test Report ######### pass: 31 unknown: 0 fail: 0 crash: 3 memcached errors (not reported as total): 0 31/34 = 91.17% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #34 :: --workload=kill-n-disk-failure --manipulate-disks --replicas=1 --durability=0:0:0:100 --recovery-type=delta --eviction-policy=full --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ ###### Finished Running Suite ######### Jepsen tests complete! ###### Final Test Report ######### pass: 31 unknown: 0 fail: 0 crash: 3 memcached errors (not reported as total): 0 31/34 = 91.17% ###### Crashed Tests ######## #8 :: --workload=rebalance --scenario=swap-rebalance --no-autofailover --replicas=1 --disrupt-count=1 --kv-timeout=30 --durability=0:0:0:100 --use-json-docs --doc-padding-size=4096 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #15 :: --workload=set-kill --scenario=kill-memcached --replicas=1 --no-autofailover --durability=0:0:100:0 --eviction-policy=full --use-json-docs --doc-padding-size=64 --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 #34 :: --workload=kill-n-disk-failure --manipulate-disks --replicas=1 --durability=0:0:0:100 --recovery-type=delta --eviction-policy=full --enable-tcp-capture --hashdump --enable-memcached-debug-log-level --net-interface=eth0 ################################################################ Error log messages where found in memached.log's ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:52:39.604218-08:00 CRITICAL *** Fatal error encountered during exception handling *** ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:52:39.970915-08:00 CRITICAL Caught unhandled std::exception-derived exception. what(): GSL: Precondition failure: 'prepareSeqno >= checkpointManager->getOpenSnapshotStartSeqno()' at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/vbucket.cc:1007 ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.253841-08:00 CRITICAL Detected previous crash ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254040-08:00 CRITICAL Breakpad caught a crash (Couchbase version 7.1.0-2024). Writing crash dump to /opt/couchbase/var/lib/couchbase/crash/c54761bb-f626-41ca-27ebe899-d5b9fcb5.dmp before terminating. ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254064-08:00 CRITICAL Stack backtrace of crashed thread: ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254066-08:00 CRITICAL #0 /opt/couchbase/bin/memcached() [0x400000+0x726438] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254068-08:00 CRITICAL #1 /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler12GenerateDumpEPNS0_12CrashContextE+0x3ea) [0x400000+0x777b1a] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254071-08:00 CRITICAL #2 /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler13SignalHandlerEiP9siginfo_tPv+0xb8) [0x400000+0x777e58] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254073-08:00 CRITICAL #3 /lib/x86_64-linux-gnu/libpthread.so.0() [0x7f2c5c227000+0x128a0] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254075-08:00 CRITICAL #4 /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7) [0x7f2c599b2000+0x3ef47] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254076-08:00 CRITICAL #5 /lib/x86_64-linux-gnu/libc.so.6(abort+0x141) [0x7f2c599b2000+0x408b1] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254077-08:00 CRITICAL #6 /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f2c5a359000+0x9963c] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254096-08:00 CRITICAL #7 /opt/couchbase/bin/memcached() [0x400000+0x734a9b] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254098-08:00 CRITICAL #8 /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f2c5a359000+0xa48f6] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254099-08:00 CRITICAL #9 /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f2c5a359000+0xa4961] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254101-08:00 CRITICAL #10 /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f2c5a359000+0xa4bf4] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254102-08:00 CRITICAL #11 /opt/couchbase/bin/memcached() [0x400000+0x50a7b] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254104-08:00 CRITICAL #12 /opt/couchbase/bin/memcached() [0x400000+0xa74d6] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254105-08:00 CRITICAL #13 /opt/couchbase/bin/memcached() [0x400000+0x4e1c38] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254107-08:00 CRITICAL #14 /opt/couchbase/bin/memcached() [0x400000+0x4e3bac] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254108-08:00 CRITICAL #15 /opt/couchbase/bin/memcached() [0x400000+0x4d6672] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254109-08:00 CRITICAL #16 /opt/couchbase/bin/memcached() [0x400000+0x4db7a7] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254110-08:00 CRITICAL #17 /opt/couchbase/bin/memcached() [0x400000+0x4def85] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254112-08:00 CRITICAL #18 /opt/couchbase/bin/memcached() [0x400000+0x6a3bb2] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254113-08:00 CRITICAL #19 /opt/couchbase/bin/memcached() [0x400000+0x6a0dc5] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254114-08:00 CRITICAL #20 /opt/couchbase/bin/memcached() [0x400000+0x7f7440] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254115-08:00 CRITICAL #21 /opt/couchbase/bin/memcached() [0x400000+0x7df1fa] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254117-08:00 CRITICAL #22 /opt/couchbase/bin/memcached() [0x400000+0x7fa3f9] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254118-08:00 CRITICAL #23 /opt/couchbase/bin/memcached() [0x400000+0x6a0a54] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254119-08:00 CRITICAL #24 /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f2c5a359000+0xcdd40] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254121-08:00 CRITICAL #25 /lib/x86_64-linux-gnu/libpthread.so.0() [0x7f2c5c227000+0x76db] ./20220109T053708.000-0800/172.23.107.227/cbcollect_info_ns_1@172.23.107.227_20220109-135231/memcached.log:2022-01-09T05:53:23.254122-08:00 CRITICAL #26 /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f2c599b2000+0x121a3f] EXIT CODE IS 6 JEPSEN CRASHED AND ERRORS FOUND IN MEMCACHED LOGS Build step 'Execute shell' marked build as failure Archiving artifacts Sending e-mails to: kv_engine_dev@couchbase.com asad.zaidi@couchbase.com Finished: FAILURE